2024-09-21T03:23:36.521 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-21T03:23:36.542 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-21T03:23:36.618 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913473 branch: wip-pdonnell-testing-20240920.212106-debug description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/damage} email: pdonnell@redhat.com first_in_suite: false job_id: '7913473' kernel: client: sha1: distro kdb: 1 sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-pdonnell-testing-20240920.212106-debug ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd cluster-conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 2 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: client mount timeout: 30 debug client: 20 debug mgr: 20 debug ms: 1 mon warn on pool no app: false mon: mon op complaint time: 120 osd: osd op complaint time: 180 conf: client: pid file: '' global: ms die on skipped message: false mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - fs.*is degraded - filesystem is degraded - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - filesystem is offline - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - insufficient standby MDS daemons available - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - bad backtrace - object missing on disk - error reading table object - error reading sessionmap - Error loading MDS rank - missing journal object - Error recovering journal - error decoding table object - failed to read JournalPointer - Corrupt directory entry - Corrupt fnode header - corrupt sessionmap header - Corrupt dentry - Scrub error on inode - Metadata damage detected - MDS_READ_ONLY - force file system read-only - with standby daemon mds - MDS abort because newly corrupt dentry sha1: 1843221895adccfb3b692060251dd6a0183bc410 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 1843221895adccfb3b692060251dd6a0183bc410 kclient: syntax: v1 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-pdonnell-testing-20240920.212106-debug sha1: 1843221895adccfb3b692060251dd6a0183bc410 owner: scheduled_pdonnell@teuthology priority: 90 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 7864 sha1: 1843221895adccfb3b692060251dd6a0183bc410 sleep_before_teardown: 0 subset: 67/512 suite: fs suite_branch: wip-pdonnell-testing-20240920.212106-debug suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_1843221895adccfb3b692060251dd6a0183bc410/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 1843221895adccfb3b692060251dd6a0183bc410 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi038.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEMKrMeI/grdgdieR3gKFoSocAczE1VXEQqTtfimHxvz9zBS1jbD9PnWVgWpnmedtrXbAZCSFsEfPyOQlaFR4VM= smithi195.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNPRj7Xa1rHyZCZzDGJgB4qh8J16e3iQN7219mvcU4gAAUE3aZuCQnL0NG9riL3CqFqAwuUw07eV+Y3YgPOYtnE= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - kclient: null - cephfs_test_runner: modules: - tasks.cephfs.test_damage teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-20_23:48:22 tube: smithi user: pdonnell verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2902 2024-09-21T03:23:36.618 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_1843221895adccfb3b692060251dd6a0183bc410/qa; will attempt to use it 2024-09-21T03:23:36.619 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_1843221895adccfb3b692060251dd6a0183bc410/qa/tasks 2024-09-21T03:23:36.619 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-21T03:23:36.620 INFO:teuthology.task.internal:Checking packages... 2024-09-21T03:23:36.640 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '1843221895adccfb3b692060251dd6a0183bc410' 2024-09-21T03:23:36.640 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-21T03:23:36.640 INFO:teuthology.packaging:ref: None 2024-09-21T03:23:36.641 INFO:teuthology.packaging:tag: None 2024-09-21T03:23:36.641 INFO:teuthology.packaging:branch: wip-pdonnell-testing-20240920.212106-debug 2024-09-21T03:23:36.641 INFO:teuthology.packaging:sha1: 1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T03:23:36.641 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-pdonnell-testing-20240920.212106-debug 2024-09-21T03:23:36.785 INFO:teuthology.task.internal:Found packages for ceph version 19.3.0-5120.g18432218 2024-09-21T03:23:36.786 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-21T03:23:36.793 INFO:teuthology.task.internal:no buildpackages task found 2024-09-21T03:23:36.793 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-21T03:23:36.878 INFO:teuthology.task.internal:Saving configuration 2024-09-21T03:23:36.894 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-21T03:23:36.901 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-21T03:23:36.926 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi038.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913473', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-21 03:18:52.326050', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEMKrMeI/grdgdieR3gKFoSocAczE1VXEQqTtfimHxvz9zBS1jbD9PnWVgWpnmedtrXbAZCSFsEfPyOQlaFR4VM='} 2024-09-21T03:23:36.946 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi195.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913473', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-21 03:18:52.327965', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNPRj7Xa1rHyZCZzDGJgB4qh8J16e3iQN7219mvcU4gAAUE3aZuCQnL0NG9riL3CqFqAwuUw07eV+Y3YgPOYtnE='} 2024-09-21T03:23:36.946 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-21T03:23:36.953 INFO:teuthology.task.internal:roles: ubuntu@smithi038.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-09-21T03:23:36.953 INFO:teuthology.task.internal:roles: ubuntu@smithi195.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-09-21T03:23:36.953 INFO:teuthology.run_tasks:Running task console_log... 2024-09-21T03:23:37.006 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f5a148c0160>, signals=[15]) 2024-09-21T03:23:37.006 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-21T03:23:37.012 INFO:teuthology.task.internal:Opening connections... 2024-09-21T03:23:37.012 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi038.front.sepia.ceph.com 2024-09-21T03:23:37.013 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi038.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T03:23:37.088 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi195.front.sepia.ceph.com 2024-09-21T03:23:37.089 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi195.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T03:23:37.161 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-21T03:23:37.167 DEBUG:teuthology.orchestra.run.smithi038:> uname -m 2024-09-21T03:23:37.185 INFO:teuthology.orchestra.run.smithi038.stdout:x86_64 2024-09-21T03:23:37.185 DEBUG:teuthology.orchestra.run.smithi038:> cat /etc/os-release 2024-09-21T03:23:37.243 INFO:teuthology.orchestra.run.smithi038.stdout:NAME="CentOS Stream" 2024-09-21T03:23:37.243 INFO:teuthology.orchestra.run.smithi038.stdout:VERSION="9" 2024-09-21T03:23:37.243 INFO:teuthology.orchestra.run.smithi038.stdout:ID="centos" 2024-09-21T03:23:37.243 INFO:teuthology.orchestra.run.smithi038.stdout:ID_LIKE="rhel fedora" 2024-09-21T03:23:37.243 INFO:teuthology.orchestra.run.smithi038.stdout:VERSION_ID="9" 2024-09-21T03:23:37.243 INFO:teuthology.orchestra.run.smithi038.stdout:PLATFORM_ID="platform:el9" 2024-09-21T03:23:37.243 INFO:teuthology.orchestra.run.smithi038.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-21T03:23:37.243 INFO:teuthology.orchestra.run.smithi038.stdout:ANSI_COLOR="0;31" 2024-09-21T03:23:37.243 INFO:teuthology.orchestra.run.smithi038.stdout:LOGO="fedora-logo-icon" 2024-09-21T03:23:37.243 INFO:teuthology.orchestra.run.smithi038.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-21T03:23:37.244 INFO:teuthology.orchestra.run.smithi038.stdout:HOME_URL="https://centos.org/" 2024-09-21T03:23:37.244 INFO:teuthology.orchestra.run.smithi038.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-21T03:23:37.244 INFO:teuthology.orchestra.run.smithi038.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-21T03:23:37.244 INFO:teuthology.orchestra.run.smithi038.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-21T03:23:37.244 INFO:teuthology.lock.ops:Updating smithi038.front.sepia.ceph.com on lock server 2024-09-21T03:23:37.267 DEBUG:teuthology.orchestra.run.smithi195:> uname -m 2024-09-21T03:23:37.285 INFO:teuthology.orchestra.run.smithi195.stdout:x86_64 2024-09-21T03:23:37.285 DEBUG:teuthology.orchestra.run.smithi195:> cat /etc/os-release 2024-09-21T03:23:37.342 INFO:teuthology.orchestra.run.smithi195.stdout:NAME="CentOS Stream" 2024-09-21T03:23:37.342 INFO:teuthology.orchestra.run.smithi195.stdout:VERSION="9" 2024-09-21T03:23:37.342 INFO:teuthology.orchestra.run.smithi195.stdout:ID="centos" 2024-09-21T03:23:37.342 INFO:teuthology.orchestra.run.smithi195.stdout:ID_LIKE="rhel fedora" 2024-09-21T03:23:37.342 INFO:teuthology.orchestra.run.smithi195.stdout:VERSION_ID="9" 2024-09-21T03:23:37.342 INFO:teuthology.orchestra.run.smithi195.stdout:PLATFORM_ID="platform:el9" 2024-09-21T03:23:37.342 INFO:teuthology.orchestra.run.smithi195.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-21T03:23:37.342 INFO:teuthology.orchestra.run.smithi195.stdout:ANSI_COLOR="0;31" 2024-09-21T03:23:37.342 INFO:teuthology.orchestra.run.smithi195.stdout:LOGO="fedora-logo-icon" 2024-09-21T03:23:37.343 INFO:teuthology.orchestra.run.smithi195.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-21T03:23:37.343 INFO:teuthology.orchestra.run.smithi195.stdout:HOME_URL="https://centos.org/" 2024-09-21T03:23:37.343 INFO:teuthology.orchestra.run.smithi195.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-21T03:23:37.343 INFO:teuthology.orchestra.run.smithi195.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-21T03:23:37.343 INFO:teuthology.orchestra.run.smithi195.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-21T03:23:37.343 INFO:teuthology.lock.ops:Updating smithi195.front.sepia.ceph.com on lock server 2024-09-21T03:23:37.367 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-21T03:23:37.376 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-21T03:23:37.382 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-21T03:23:37.382 DEBUG:teuthology.orchestra.run.smithi038:> test '!' -e /home/ubuntu/cephtest 2024-09-21T03:23:37.384 DEBUG:teuthology.orchestra.run.smithi195:> test '!' -e /home/ubuntu/cephtest 2024-09-21T03:23:37.400 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-21T03:23:37.405 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-21T03:23:37.405 DEBUG:teuthology.orchestra.run.smithi038:> test -z $(ls -A /var/lib/ceph) 2024-09-21T03:23:37.442 DEBUG:teuthology.orchestra.run.smithi195:> test -z $(ls -A /var/lib/ceph) 2024-09-21T03:23:37.474 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-21T03:23:37.525 INFO:teuthology.run_tasks:Running task kernel... 2024-09-21T03:23:37.538 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': 1, 'sha1': 'distro'} 2024-09-21T03:23:37.538 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}, 'client.2': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}, 'client.3': {'sha1': 'distro'}} 2024-09-21T03:23:37.538 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}}, timeout 300 2024-09-21T03:23:37.538 DEBUG:teuthology.orchestra.run.smithi038:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-21T03:23:37.539 DEBUG:teuthology.orchestra.run.smithi195:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-21T03:23:37.554 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T03:23:37.554 DEBUG:teuthology.orchestra.run.smithi195:> uname -r 2024-09-21T03:23:37.555 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T03:23:37.555 DEBUG:teuthology.orchestra.run.smithi038:> uname -r 2024-09-21T03:23:37.610 INFO:teuthology.orchestra.run.smithi195.stdout:5.14.0-503.el9.x86_64 2024-09-21T03:23:37.611 INFO:teuthology.task.kernel:Running kernel on smithi195: 5.14.0-503.el9.x86_64 2024-09-21T03:23:37.611 DEBUG:teuthology.orchestra.run.smithi195:> sudo yum install -y kernel 2024-09-21T03:23:37.613 INFO:teuthology.orchestra.run.smithi038.stdout:5.14.0-503.el9.x86_64 2024-09-21T03:23:37.613 INFO:teuthology.task.kernel:Running kernel on smithi038: 5.14.0-503.el9.x86_64 2024-09-21T03:23:37.613 DEBUG:teuthology.orchestra.run.smithi038:> sudo yum install -y kernel 2024-09-21T03:23:39.977 INFO:teuthology.orchestra.run.smithi038.stdout:CentOS Stream 9 - BaseOS 14 MB/s | 8.3 MB 00:00 2024-09-21T03:23:41.291 INFO:teuthology.orchestra.run.smithi195.stdout:CentOS Stream 9 - BaseOS 4.3 MB/s | 8.3 MB 00:01 2024-09-21T03:23:43.092 INFO:teuthology.orchestra.run.smithi195.stdout:CentOS Stream 9 - AppStream 28 MB/s | 20 MB 00:00 2024-09-21T03:23:48.467 INFO:teuthology.orchestra.run.smithi195.stdout:CentOS Stream 9 - CRB 4.1 MB/s | 6.5 MB 00:01 2024-09-21T03:23:50.001 INFO:teuthology.orchestra.run.smithi195.stdout:CentOS Stream 9 - Extras packages 57 kB/s | 19 kB 00:00 2024-09-21T03:23:50.789 INFO:teuthology.orchestra.run.smithi195.stdout:Extra Packages for Enterprise Linux 34 MB/s | 23 MB 00:00 2024-09-21T03:23:51.219 INFO:teuthology.orchestra.run.smithi038.stdout:CentOS Stream 9 - AppStream 2.0 MB/s | 20 MB 00:10 2024-09-21T03:23:55.662 INFO:teuthology.orchestra.run.smithi038.stdout:CentOS Stream 9 - CRB 13 MB/s | 6.5 MB 00:00 2024-09-21T03:23:56.186 INFO:teuthology.orchestra.run.smithi195.stdout:lab-extras 44 kB/s | 1.7 kB 00:00 2024-09-21T03:23:57.137 INFO:teuthology.orchestra.run.smithi038.stdout:CentOS Stream 9 - Extras packages 85 kB/s | 19 kB 00:00 2024-09-21T03:23:58.041 INFO:teuthology.orchestra.run.smithi038.stdout:Extra Packages for Enterprise Linux 29 MB/s | 23 MB 00:00 2024-09-21T03:23:58.109 INFO:teuthology.orchestra.run.smithi195.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-21T03:23:58.109 INFO:teuthology.orchestra.run.smithi195.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-21T03:23:58.189 INFO:teuthology.orchestra.run.smithi195.stdout:Dependencies resolved. 2024-09-21T03:23:58.199 INFO:teuthology.orchestra.run.smithi195.stdout:================================================================================ 2024-09-21T03:23:58.199 INFO:teuthology.orchestra.run.smithi195.stdout: Package Architecture Version Repository Size 2024-09-21T03:23:58.199 INFO:teuthology.orchestra.run.smithi195.stdout:================================================================================ 2024-09-21T03:23:58.199 INFO:teuthology.orchestra.run.smithi195.stdout:Installing: 2024-09-21T03:23:58.199 INFO:teuthology.orchestra.run.smithi195.stdout: kernel x86_64 5.14.0-508.el9 baseos 36 k 2024-09-21T03:23:58.199 INFO:teuthology.orchestra.run.smithi195.stdout:Installing dependencies: 2024-09-21T03:23:58.199 INFO:teuthology.orchestra.run.smithi195.stdout: kernel-core x86_64 5.14.0-508.el9 baseos 16 M 2024-09-21T03:23:58.199 INFO:teuthology.orchestra.run.smithi195.stdout: kernel-modules x86_64 5.14.0-508.el9 baseos 35 M 2024-09-21T03:23:58.199 INFO:teuthology.orchestra.run.smithi195.stdout: kernel-modules-core x86_64 5.14.0-508.el9 baseos 29 M 2024-09-21T03:23:58.199 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-21T03:23:58.199 INFO:teuthology.orchestra.run.smithi195.stdout:Transaction Summary 2024-09-21T03:23:58.200 INFO:teuthology.orchestra.run.smithi195.stdout:================================================================================ 2024-09-21T03:23:58.200 INFO:teuthology.orchestra.run.smithi195.stdout:Install 4 Packages 2024-09-21T03:23:58.200 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-21T03:23:58.200 INFO:teuthology.orchestra.run.smithi195.stdout:Total download size: 79 M 2024-09-21T03:23:58.200 INFO:teuthology.orchestra.run.smithi195.stdout:Installed size: 126 M 2024-09-21T03:23:58.200 INFO:teuthology.orchestra.run.smithi195.stdout:Downloading Packages: 2024-09-21T03:23:58.563 INFO:teuthology.orchestra.run.smithi195.stdout:(1/4): kernel-5.14.0-508.el9.x86_64.rpm 338 kB/s | 36 kB 00:00 2024-09-21T03:23:59.030 INFO:teuthology.orchestra.run.smithi195.stdout:(2/4): kernel-core-5.14.0-508.el9.x86_64.rpm 27 MB/s | 16 MB 00:00 2024-09-21T03:23:59.363 INFO:teuthology.orchestra.run.smithi195.stdout:(3/4): kernel-modules-5.14.0-508.el9.x86_64.rpm 38 MB/s | 35 MB 00:00 2024-09-21T03:23:59.722 INFO:teuthology.orchestra.run.smithi195.stdout:(4/4): kernel-modules-core-5.14.0-508.el9.x86_6 25 MB/s | 29 MB 00:01 2024-09-21T03:23:59.722 INFO:teuthology.orchestra.run.smithi195.stdout:-------------------------------------------------------------------------------- 2024-09-21T03:23:59.722 INFO:teuthology.orchestra.run.smithi195.stdout:Total 52 MB/s | 79 MB 00:01 2024-09-21T03:24:00.408 INFO:teuthology.orchestra.run.smithi195.stdout:Running transaction check 2024-09-21T03:24:00.737 INFO:teuthology.orchestra.run.smithi195.stdout:Transaction check succeeded. 2024-09-21T03:24:00.737 INFO:teuthology.orchestra.run.smithi195.stdout:Running transaction test 2024-09-21T03:24:01.693 INFO:teuthology.orchestra.run.smithi195.stdout:Transaction test succeeded. 2024-09-21T03:24:01.693 INFO:teuthology.orchestra.run.smithi195.stdout:Running transaction 2024-09-21T03:24:03.622 INFO:teuthology.orchestra.run.smithi038.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-09-21T03:24:04.101 INFO:teuthology.orchestra.run.smithi195.stdout: Preparing : 1/1 2024-09-21T03:24:04.788 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : kernel-modules-core-5.14.0-508.el9.x86_64 1/4 2024-09-21T03:24:04.864 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-21T03:24:05.604 INFO:teuthology.orchestra.run.smithi038.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-21T03:24:05.604 INFO:teuthology.orchestra.run.smithi038.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-21T03:24:05.706 INFO:teuthology.orchestra.run.smithi038.stdout:Dependencies resolved. 2024-09-21T03:24:05.713 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2024-09-21T03:24:05.713 INFO:teuthology.orchestra.run.smithi038.stdout: Package Architecture Version Repository Size 2024-09-21T03:24:05.713 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2024-09-21T03:24:05.714 INFO:teuthology.orchestra.run.smithi038.stdout:Installing: 2024-09-21T03:24:05.714 INFO:teuthology.orchestra.run.smithi038.stdout: kernel x86_64 5.14.0-508.el9 baseos 36 k 2024-09-21T03:24:05.714 INFO:teuthology.orchestra.run.smithi038.stdout:Installing dependencies: 2024-09-21T03:24:05.714 INFO:teuthology.orchestra.run.smithi038.stdout: kernel-core x86_64 5.14.0-508.el9 baseos 16 M 2024-09-21T03:24:05.714 INFO:teuthology.orchestra.run.smithi038.stdout: kernel-modules x86_64 5.14.0-508.el9 baseos 35 M 2024-09-21T03:24:05.714 INFO:teuthology.orchestra.run.smithi038.stdout: kernel-modules-core x86_64 5.14.0-508.el9 baseos 29 M 2024-09-21T03:24:05.714 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:24:05.714 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction Summary 2024-09-21T03:24:05.714 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2024-09-21T03:24:05.714 INFO:teuthology.orchestra.run.smithi038.stdout:Install 4 Packages 2024-09-21T03:24:05.714 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:24:05.715 INFO:teuthology.orchestra.run.smithi038.stdout:Total download size: 79 M 2024-09-21T03:24:05.715 INFO:teuthology.orchestra.run.smithi038.stdout:Installed size: 126 M 2024-09-21T03:24:05.715 INFO:teuthology.orchestra.run.smithi038.stdout:Downloading Packages: 2024-09-21T03:24:05.832 INFO:teuthology.orchestra.run.smithi038.stdout:(1/4): kernel-5.14.0-508.el9.x86_64.rpm 375 kB/s | 36 kB 00:00 2024-09-21T03:24:06.233 INFO:teuthology.orchestra.run.smithi038.stdout:(2/4): kernel-core-5.14.0-508.el9.x86_64.rpm 32 MB/s | 16 MB 00:00 2024-09-21T03:24:06.617 INFO:teuthology.orchestra.run.smithi038.stdout:(3/4): kernel-modules-core-5.14.0-508.el9.x86_6 36 MB/s | 29 MB 00:00 2024-09-21T03:24:06.933 INFO:teuthology.orchestra.run.smithi038.stdout:(4/4): kernel-modules-5.14.0-508.el9.x86_64.rpm 29 MB/s | 35 MB 00:01 2024-09-21T03:24:06.934 INFO:teuthology.orchestra.run.smithi038.stdout:-------------------------------------------------------------------------------- 2024-09-21T03:24:06.934 INFO:teuthology.orchestra.run.smithi038.stdout:Total 65 MB/s | 79 MB 00:01 2024-09-21T03:24:07.620 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-21T03:24:07.643 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction check 2024-09-21T03:24:07.751 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-21T03:24:08.040 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction check succeeded. 2024-09-21T03:24:08.040 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction test 2024-09-21T03:24:09.008 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction test succeeded. 2024-09-21T03:24:09.008 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction 2024-09-21T03:24:11.568 INFO:teuthology.orchestra.run.smithi038.stdout: Preparing : 1/1 2024-09-21T03:24:12.231 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : kernel-modules-core-5.14.0-508.el9.x86_64 1/4 2024-09-21T03:24:12.335 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-21T03:24:12.577 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-21T03:24:12.660 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : kernel-5.14.0-508.el9.x86_64 4/4 2024-09-21T03:24:15.117 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-21T03:24:15.248 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-21T03:24:17.506 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-21T03:24:20.377 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-21T03:24:20.407 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : kernel-5.14.0-508.el9.x86_64 4/4 2024-09-21T03:24:25.708 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-21T03:24:55.339 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 4/4 2024-09-21T03:24:55.379 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 4/4 2024-09-21T03:24:55.803 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: kernel-5.14.0-508.el9.x86_64 4/4 2024-09-21T03:24:55.803 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : kernel-5.14.0-508.el9.x86_64 1/4 2024-09-21T03:24:55.803 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-21T03:24:55.803 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-21T03:24:56.655 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-21T03:24:56.655 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-21T03:24:56.655 INFO:teuthology.orchestra.run.smithi195.stdout:Installed: 2024-09-21T03:24:56.655 INFO:teuthology.orchestra.run.smithi195.stdout: kernel-5.14.0-508.el9.x86_64 2024-09-21T03:24:56.655 INFO:teuthology.orchestra.run.smithi195.stdout: kernel-core-5.14.0-508.el9.x86_64 2024-09-21T03:24:56.655 INFO:teuthology.orchestra.run.smithi195.stdout: kernel-modules-5.14.0-508.el9.x86_64 2024-09-21T03:24:56.655 INFO:teuthology.orchestra.run.smithi195.stdout: kernel-modules-core-5.14.0-508.el9.x86_64 2024-09-21T03:24:56.655 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-21T03:24:56.656 INFO:teuthology.orchestra.run.smithi195.stdout:Complete! 2024-09-21T03:24:56.991 DEBUG:teuthology.orchestra.run.smithi195:> rpm -q kernel | sort -rV | head -n 1 2024-09-21T03:24:57.074 INFO:teuthology.orchestra.run.smithi195.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-21T03:24:57.075 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-21T03:24:57.075 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-508.el9.x86_64 2024-09-21T03:24:57.075 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-21T03:24:57.076 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-21T03:24:57.076 INFO:teuthology.task.kernel:Installing distro kernel on client.1... 2024-09-21T03:24:57.076 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi195.front.sepia.ceph.com, path=None, version=distro) 2024-09-21T03:24:57.076 DEBUG:teuthology.orchestra.run.smithi195:> rpm -q kernel | sort -rV | head -n 1 2024-09-21T03:24:57.134 INFO:teuthology.orchestra.run.smithi195.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-21T03:24:57.135 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-21T03:24:57.135 DEBUG:teuthology.orchestra.run.smithi195:> sudo rpm -qi grub2-tools 2024-09-21T03:24:57.172 INFO:teuthology.orchestra.run.smithi195.stdout:Name : grub2-tools 2024-09-21T03:24:57.172 INFO:teuthology.orchestra.run.smithi195.stdout:Epoch : 1 2024-09-21T03:24:57.172 INFO:teuthology.orchestra.run.smithi195.stdout:Version : 2.06 2024-09-21T03:24:57.172 INFO:teuthology.orchestra.run.smithi195.stdout:Release : 82.el9 2024-09-21T03:24:57.172 INFO:teuthology.orchestra.run.smithi195.stdout:Architecture: x86_64 2024-09-21T03:24:57.172 INFO:teuthology.orchestra.run.smithi195.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-21T03:24:57.172 INFO:teuthology.orchestra.run.smithi195.stdout:Group : Unspecified 2024-09-21T03:24:57.172 INFO:teuthology.orchestra.run.smithi195.stdout:Size : 8274670 2024-09-21T03:24:57.172 INFO:teuthology.orchestra.run.smithi195.stdout:License : GPLv3+ 2024-09-21T03:24:57.172 INFO:teuthology.orchestra.run.smithi195.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-21T03:24:57.172 INFO:teuthology.orchestra.run.smithi195.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-21T03:24:57.173 INFO:teuthology.orchestra.run.smithi195.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-21T03:24:57.173 INFO:teuthology.orchestra.run.smithi195.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-21T03:24:57.173 INFO:teuthology.orchestra.run.smithi195.stdout:Packager : builder@centos.org 2024-09-21T03:24:57.173 INFO:teuthology.orchestra.run.smithi195.stdout:Vendor : CentOS 2024-09-21T03:24:57.173 INFO:teuthology.orchestra.run.smithi195.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-21T03:24:57.173 INFO:teuthology.orchestra.run.smithi195.stdout:Summary : Support tools for GRUB. 2024-09-21T03:24:57.173 INFO:teuthology.orchestra.run.smithi195.stdout:Description : 2024-09-21T03:24:57.173 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-21T03:24:57.173 INFO:teuthology.orchestra.run.smithi195.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-21T03:24:57.173 INFO:teuthology.orchestra.run.smithi195.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-21T03:24:57.173 INFO:teuthology.orchestra.run.smithi195.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-21T03:24:57.173 INFO:teuthology.orchestra.run.smithi195.stdout:hardware devices. 2024-09-21T03:24:57.173 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-21T03:24:57.174 INFO:teuthology.orchestra.run.smithi195.stdout:This subpackage provides tools for support of all platforms. 2024-09-21T03:24:57.174 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-21T03:24:57.174 INFO:teuthology.task.kernel:Updating grub on smithi195 to boot 5.14.0-508.el9.x86_64 2024-09-21T03:24:57.175 DEBUG:teuthology.orchestra.run.smithi195:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-21T03:24:57.843 INFO:teuthology.orchestra.run.smithi195.stderr:Generating grub configuration file ... 2024-09-21T03:24:59.159 INFO:teuthology.orchestra.run.smithi195.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-21T03:24:59.187 INFO:teuthology.orchestra.run.smithi195.stderr:done 2024-09-21T03:24:59.189 DEBUG:teuthology.orchestra.run.smithi195:> mktemp 2024-09-21T03:24:59.204 INFO:teuthology.orchestra.run.smithi195.stdout:/tmp/tmp.fIcOwvKCoP 2024-09-21T03:24:59.204 DEBUG:teuthology.orchestra.run.smithi195:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.fIcOwvKCoP 2024-09-21T03:24:59.270 DEBUG:teuthology.orchestra.run.smithi195:> sudo chmod 0666 /tmp/tmp.fIcOwvKCoP 2024-09-21T03:24:59.404 DEBUG:teuthology.orchestra.remote:smithi195:/tmp/tmp.fIcOwvKCoP is 6KB 2024-09-21T03:24:59.451 DEBUG:teuthology.orchestra.run.smithi195:> rm -fr /tmp/tmp.fIcOwvKCoP 2024-09-21T03:24:59.466 DEBUG:teuthology.orchestra.run.smithi195:> sudo /bin/ls /boot/loader/entries || true 2024-09-21T03:24:59.528 INFO:teuthology.orchestra.run.smithi195.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-21T03:24:59.528 INFO:teuthology.orchestra.run.smithi195.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-21T03:24:59.529 INFO:teuthology.orchestra.run.smithi195.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-21T03:24:59.529 INFO:teuthology.orchestra.run.smithi195.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64.conf 2024-09-21T03:24:59.530 DEBUG:teuthology.orchestra.run.smithi195:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64 2024-09-21T03:24:59.694 DEBUG:teuthology.orchestra.run.smithi195:> sudo shutdown -r now 2024-09-21T03:25:05.056 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 4/4 2024-09-21T03:25:05.097 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 4/4 2024-09-21T03:25:05.478 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: kernel-5.14.0-508.el9.x86_64 4/4 2024-09-21T03:25:05.478 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : kernel-5.14.0-508.el9.x86_64 1/4 2024-09-21T03:25:05.478 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-21T03:25:05.478 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-21T03:25:06.350 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-21T03:25:06.350 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:25:06.350 INFO:teuthology.orchestra.run.smithi038.stdout:Installed: 2024-09-21T03:25:06.350 INFO:teuthology.orchestra.run.smithi038.stdout: kernel-5.14.0-508.el9.x86_64 2024-09-21T03:25:06.350 INFO:teuthology.orchestra.run.smithi038.stdout: kernel-core-5.14.0-508.el9.x86_64 2024-09-21T03:25:06.350 INFO:teuthology.orchestra.run.smithi038.stdout: kernel-modules-5.14.0-508.el9.x86_64 2024-09-21T03:25:06.350 INFO:teuthology.orchestra.run.smithi038.stdout: kernel-modules-core-5.14.0-508.el9.x86_64 2024-09-21T03:25:06.350 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:25:06.350 INFO:teuthology.orchestra.run.smithi038.stdout:Complete! 2024-09-21T03:25:06.698 DEBUG:teuthology.orchestra.run.smithi038:> rpm -q kernel | sort -rV | head -n 1 2024-09-21T03:25:06.773 INFO:teuthology.orchestra.run.smithi038.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-21T03:25:06.773 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-21T03:25:06.773 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-508.el9.x86_64 2024-09-21T03:25:06.773 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-21T03:25:06.773 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-21T03:25:06.773 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-09-21T03:25:06.773 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi038.front.sepia.ceph.com, path=None, version=distro) 2024-09-21T03:25:06.774 DEBUG:teuthology.orchestra.run.smithi038:> rpm -q kernel | sort -rV | head -n 1 2024-09-21T03:25:06.831 INFO:teuthology.orchestra.run.smithi038.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-21T03:25:06.831 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-21T03:25:06.831 DEBUG:teuthology.orchestra.run.smithi038:> sudo rpm -qi grub2-tools 2024-09-21T03:25:06.862 INFO:teuthology.orchestra.run.smithi038.stdout:Name : grub2-tools 2024-09-21T03:25:06.862 INFO:teuthology.orchestra.run.smithi038.stdout:Epoch : 1 2024-09-21T03:25:06.863 INFO:teuthology.orchestra.run.smithi038.stdout:Version : 2.06 2024-09-21T03:25:06.863 INFO:teuthology.orchestra.run.smithi038.stdout:Release : 82.el9 2024-09-21T03:25:06.863 INFO:teuthology.orchestra.run.smithi038.stdout:Architecture: x86_64 2024-09-21T03:25:06.863 INFO:teuthology.orchestra.run.smithi038.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-21T03:25:06.863 INFO:teuthology.orchestra.run.smithi038.stdout:Group : Unspecified 2024-09-21T03:25:06.863 INFO:teuthology.orchestra.run.smithi038.stdout:Size : 8274670 2024-09-21T03:25:06.863 INFO:teuthology.orchestra.run.smithi038.stdout:License : GPLv3+ 2024-09-21T03:25:06.863 INFO:teuthology.orchestra.run.smithi038.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-21T03:25:06.863 INFO:teuthology.orchestra.run.smithi038.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-21T03:25:06.863 INFO:teuthology.orchestra.run.smithi038.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-21T03:25:06.863 INFO:teuthology.orchestra.run.smithi038.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-21T03:25:06.863 INFO:teuthology.orchestra.run.smithi038.stdout:Packager : builder@centos.org 2024-09-21T03:25:06.863 INFO:teuthology.orchestra.run.smithi038.stdout:Vendor : CentOS 2024-09-21T03:25:06.863 INFO:teuthology.orchestra.run.smithi038.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-21T03:25:06.864 INFO:teuthology.orchestra.run.smithi038.stdout:Summary : Support tools for GRUB. 2024-09-21T03:25:06.864 INFO:teuthology.orchestra.run.smithi038.stdout:Description : 2024-09-21T03:25:06.864 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:25:06.864 INFO:teuthology.orchestra.run.smithi038.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-21T03:25:06.864 INFO:teuthology.orchestra.run.smithi038.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-21T03:25:06.864 INFO:teuthology.orchestra.run.smithi038.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-21T03:25:06.864 INFO:teuthology.orchestra.run.smithi038.stdout:hardware devices. 2024-09-21T03:25:06.864 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:25:06.864 INFO:teuthology.orchestra.run.smithi038.stdout:This subpackage provides tools for support of all platforms. 2024-09-21T03:25:06.865 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-21T03:25:06.865 INFO:teuthology.task.kernel:Updating grub on smithi038 to boot 5.14.0-508.el9.x86_64 2024-09-21T03:25:06.865 DEBUG:teuthology.orchestra.run.smithi038:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-21T03:25:07.599 INFO:teuthology.orchestra.run.smithi038.stderr:Generating grub configuration file ... 2024-09-21T03:25:08.902 INFO:teuthology.orchestra.run.smithi038.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-21T03:25:08.920 INFO:teuthology.orchestra.run.smithi038.stderr:done 2024-09-21T03:25:08.923 DEBUG:teuthology.orchestra.run.smithi038:> mktemp 2024-09-21T03:25:08.938 INFO:teuthology.orchestra.run.smithi038.stdout:/tmp/tmp.C97I64FEej 2024-09-21T03:25:08.938 DEBUG:teuthology.orchestra.run.smithi038:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.C97I64FEej 2024-09-21T03:25:09.005 DEBUG:teuthology.orchestra.run.smithi038:> sudo chmod 0666 /tmp/tmp.C97I64FEej 2024-09-21T03:25:09.150 DEBUG:teuthology.orchestra.remote:smithi038:/tmp/tmp.C97I64FEej is 6KB 2024-09-21T03:25:09.170 DEBUG:teuthology.orchestra.run.smithi038:> rm -fr /tmp/tmp.C97I64FEej 2024-09-21T03:25:09.184 DEBUG:teuthology.orchestra.run.smithi038:> sudo /bin/ls /boot/loader/entries || true 2024-09-21T03:25:09.255 INFO:teuthology.orchestra.run.smithi038.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-21T03:25:09.255 INFO:teuthology.orchestra.run.smithi038.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-21T03:25:09.255 INFO:teuthology.orchestra.run.smithi038.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-21T03:25:09.255 INFO:teuthology.orchestra.run.smithi038.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64.conf 2024-09-21T03:25:09.257 DEBUG:teuthology.orchestra.run.smithi038:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64 2024-09-21T03:25:09.410 DEBUG:teuthology.orchestra.run.smithi038:> sudo shutdown -r now 2024-09-21T03:25:29.715 INFO:teuthology.task.kernel:Checking client client.1 for new kernel version... 2024-09-21T03:25:29.821 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi195.front.sepia.ceph.com' 2024-09-21T03:25:29.822 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi195.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T03:25:39.418 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-09-21T03:25:39.419 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi038.front.sepia.ceph.com' 2024-09-21T03:25:39.419 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi038.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T03:26:29.867 DEBUG:teuthology.orchestra.remote:timed out 2024-09-21T03:26:38.874 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi195.front.sepia.ceph.com' 2024-09-21T03:26:38.875 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi195.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T03:26:39.197 DEBUG:teuthology.orchestra.run.smithi195:> true 2024-09-21T03:26:39.421 DEBUG:teuthology.orchestra.remote:timed out 2024-09-21T03:26:39.585 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi195.front.sepia.ceph.com' 2024-09-21T03:26:39.586 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "5.14.0-508.el9.x86_64"... 2024-09-21T03:26:39.586 DEBUG:teuthology.orchestra.run.smithi195:> uname -r 2024-09-21T03:26:39.600 INFO:teuthology.orchestra.run.smithi195.stdout:5.14.0-508.el9.x86_64 2024-09-21T03:26:39.600 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-508.el9.x86_64 vs 5.14.0-508.el9.x86_64 2024-09-21T03:26:39.600 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-21T03:26:39.601 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-21T03:26:40.602 DEBUG:teuthology.parallel:result is None 2024-09-21T03:26:48.422 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi038.front.sepia.ceph.com' 2024-09-21T03:26:48.422 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi038.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T03:26:48.797 DEBUG:teuthology.orchestra.run.smithi038:> true 2024-09-21T03:26:49.381 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi038.front.sepia.ceph.com' 2024-09-21T03:26:49.382 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-508.el9.x86_64"... 2024-09-21T03:26:49.382 DEBUG:teuthology.orchestra.run.smithi038:> uname -r 2024-09-21T03:26:49.397 INFO:teuthology.orchestra.run.smithi038.stdout:5.14.0-508.el9.x86_64 2024-09-21T03:26:49.397 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-508.el9.x86_64 vs 5.14.0-508.el9.x86_64 2024-09-21T03:26:49.397 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-21T03:26:49.397 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-21T03:26:50.398 DEBUG:teuthology.parallel:result is None 2024-09-21T03:26:50.399 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-21T03:26:50.406 INFO:teuthology.task.internal:Creating test directory... 2024-09-21T03:26:50.406 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-21T03:26:50.409 DEBUG:teuthology.orchestra.run.smithi195:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-21T03:26:50.429 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-21T03:26:50.435 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-21T03:26:50.488 INFO:teuthology.task.internal:Creating archive directory... 2024-09-21T03:26:50.488 DEBUG:teuthology.orchestra.run.smithi038:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-21T03:26:50.491 DEBUG:teuthology.orchestra.run.smithi195:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-21T03:26:50.610 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-21T03:26:50.617 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-21T03:26:50.617 DEBUG:teuthology.orchestra.run.smithi038:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-21T03:26:50.619 DEBUG:teuthology.orchestra.run.smithi195:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-21T03:26:50.714 INFO:teuthology.orchestra.run.smithi195.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T03:26:50.719 INFO:teuthology.orchestra.run.smithi038.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T03:26:50.746 INFO:teuthology.orchestra.run.smithi195.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T03:26:50.769 INFO:teuthology.orchestra.run.smithi038.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T03:26:50.771 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-21T03:26:50.778 INFO:teuthology.task.internal:Configuring sudo... 2024-09-21T03:26:50.778 DEBUG:teuthology.orchestra.run.smithi038:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-21T03:26:50.780 DEBUG:teuthology.orchestra.run.smithi195:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-21T03:26:50.810 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-21T03:26:50.827 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-21T03:26:50.827 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-21T03:26:50.849 DEBUG:teuthology.orchestra.run.smithi195:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-21T03:26:50.868 DEBUG:teuthology.orchestra.run.smithi038:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T03:26:50.946 DEBUG:teuthology.orchestra.run.smithi038:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T03:26:50.992 DEBUG:teuthology.orchestra.run.smithi038:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T03:26:51.074 DEBUG:teuthology.orchestra.run.smithi038:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T03:26:51.117 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-21T03:26:51.117 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-21T03:26:51.189 DEBUG:teuthology.orchestra.run.smithi195:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T03:26:51.232 DEBUG:teuthology.orchestra.run.smithi195:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T03:26:51.287 DEBUG:teuthology.orchestra.run.smithi195:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T03:26:51.373 DEBUG:teuthology.orchestra.run.smithi195:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T03:26:51.413 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-09-21T03:26:51.413 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-21T03:26:51.480 DEBUG:teuthology.orchestra.run.smithi038:> sudo service rsyslog restart 2024-09-21T03:26:51.482 DEBUG:teuthology.orchestra.run.smithi195:> sudo service rsyslog restart 2024-09-21T03:26:51.528 INFO:teuthology.orchestra.run.smithi038.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T03:26:51.566 INFO:teuthology.orchestra.run.smithi195.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T03:26:51.857 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-21T03:26:51.864 INFO:teuthology.task.internal:Starting timer... 2024-09-21T03:26:51.865 INFO:teuthology.run_tasks:Running task pcp... 2024-09-21T03:26:51.887 INFO:teuthology.run_tasks:Running task selinux... 2024-09-21T03:26:51.896 DEBUG:teuthology.orchestra.run.smithi038:> sudo service auditd rotate 2024-09-21T03:26:51.941 INFO:teuthology.orchestra.run.smithi038.stdout:Rotating logs: 2024-09-21T03:26:51.943 DEBUG:teuthology.orchestra.run.smithi195:> sudo service auditd rotate 2024-09-21T03:26:51.996 INFO:teuthology.orchestra.run.smithi195.stdout:Rotating logs: 2024-09-21T03:26:51.998 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-21T03:26:51.998 DEBUG:teuthology.orchestra.run.smithi038:> /usr/sbin/getenforce 2024-09-21T03:26:52.017 INFO:teuthology.orchestra.run.smithi038.stdout:Permissive 2024-09-21T03:26:52.018 DEBUG:teuthology.orchestra.run.smithi195:> /usr/sbin/getenforce 2024-09-21T03:26:52.040 INFO:teuthology.orchestra.run.smithi195.stdout:Permissive 2024-09-21T03:26:52.040 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi038.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi195.front.sepia.ceph.com': 'permissive'} 2024-09-21T03:26:52.040 DEBUG:teuthology.orchestra.run.smithi038:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-21T03:26:52.085 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T03:26:52.086 DEBUG:teuthology.orchestra.run.smithi195:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-21T03:26:52.111 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T03:26:52.112 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-21T03:26:52.112 DEBUG:teuthology.orchestra.run.smithi038:> sudo /usr/sbin/setenforce permissive 2024-09-21T03:26:52.206 DEBUG:teuthology.orchestra.run.smithi195:> sudo /usr/sbin/setenforce permissive 2024-09-21T03:26:52.247 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-21T03:26:52.256 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-21T03:26:52.268 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-21T03:26:52.269 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi038.front.sepia.ceph.com,smithi195.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-21T03:30:41.277 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi038.front.sepia.ceph.com'), Remote(name='ubuntu@smithi195.front.sepia.ceph.com')] 2024-09-21T03:30:41.278 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi038.front.sepia.ceph.com' 2024-09-21T03:30:41.279 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi038.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T03:30:41.357 DEBUG:teuthology.orchestra.run.smithi038:> true 2024-09-21T03:30:41.440 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi038.front.sepia.ceph.com' 2024-09-21T03:30:41.441 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi195.front.sepia.ceph.com' 2024-09-21T03:30:41.441 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi195.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T03:30:41.515 DEBUG:teuthology.orchestra.run.smithi195:> true 2024-09-21T03:30:41.604 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi195.front.sepia.ceph.com' 2024-09-21T03:30:41.604 INFO:teuthology.run_tasks:Running task clock... 2024-09-21T03:30:41.613 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-21T03:30:41.613 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-21T03:30:41.614 DEBUG:teuthology.orchestra.run.smithi038:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T03:30:41.616 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-21T03:30:41.616 DEBUG:teuthology.orchestra.run.smithi195:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T03:30:41.649 INFO:teuthology.orchestra.run.smithi038.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-21T03:30:41.666 INFO:teuthology.orchestra.run.smithi038.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-21T03:30:41.677 INFO:teuthology.orchestra.run.smithi195.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-21T03:30:41.693 INFO:teuthology.orchestra.run.smithi195.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-21T03:30:41.694 INFO:teuthology.orchestra.run.smithi038.stderr:sudo: ntpd: command not found 2024-09-21T03:30:41.706 INFO:teuthology.orchestra.run.smithi038.stdout:506 Cannot talk to daemon 2024-09-21T03:30:41.719 INFO:teuthology.orchestra.run.smithi195.stderr:sudo: ntpd: command not found 2024-09-21T03:30:41.722 INFO:teuthology.orchestra.run.smithi038.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-21T03:30:41.731 INFO:teuthology.orchestra.run.smithi195.stdout:506 Cannot talk to daemon 2024-09-21T03:30:41.738 INFO:teuthology.orchestra.run.smithi038.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-21T03:30:41.745 INFO:teuthology.orchestra.run.smithi195.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-21T03:30:41.760 INFO:teuthology.orchestra.run.smithi195.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-21T03:30:41.794 INFO:teuthology.orchestra.run.smithi038.stderr:bash: line 1: ntpq: command not found 2024-09-21T03:30:41.797 INFO:teuthology.orchestra.run.smithi038.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T03:30:41.797 INFO:teuthology.orchestra.run.smithi038.stdout:=============================================================================== 2024-09-21T03:30:41.797 INFO:teuthology.orchestra.run.smithi038.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T03:30:41.797 INFO:teuthology.orchestra.run.smithi038.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T03:30:41.797 INFO:teuthology.orchestra.run.smithi038.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T03:30:41.797 INFO:teuthology.orchestra.run.smithi038.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T03:30:41.825 INFO:teuthology.orchestra.run.smithi195.stderr:bash: line 1: ntpq: command not found 2024-09-21T03:30:41.830 INFO:teuthology.orchestra.run.smithi195.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T03:30:41.830 INFO:teuthology.orchestra.run.smithi195.stdout:=============================================================================== 2024-09-21T03:30:41.830 INFO:teuthology.orchestra.run.smithi195.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T03:30:41.830 INFO:teuthology.orchestra.run.smithi195.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T03:30:41.830 INFO:teuthology.orchestra.run.smithi195.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T03:30:41.830 INFO:teuthology.orchestra.run.smithi195.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T03:30:41.831 INFO:teuthology.run_tasks:Running task install... 2024-09-21T03:30:41.845 DEBUG:teuthology.task.install:project ceph 2024-09-21T03:30:41.845 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '1843221895adccfb3b692060251dd6a0183bc410'}} 2024-09-21T03:30:41.845 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '1843221895adccfb3b692060251dd6a0183bc410'} 2024-09-21T03:30:41.845 INFO:teuthology.task.install:Using flavor: default 2024-09-21T03:30:41.850 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-21T03:30:41.850 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-09-21T03:30:41.850 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'tag': None, 'wait_for_package': False} 2024-09-21T03:30:41.851 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T03:30:41.852 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'tag': None, 'wait_for_package': False} 2024-09-21T03:30:41.852 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T03:30:41.990 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/ 2024-09-21T03:30:41.990 INFO:teuthology.task.install.rpm:Package version is 19.3.0-5120.g18432218 2024-09-21T03:30:42.003 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/ 2024-09-21T03:30:42.003 INFO:teuthology.task.install.rpm:Package version is 19.3.0-5120.g18432218 2024-09-21T03:30:42.100 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-21T03:30:42.100 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-21T03:30:42.100 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-21T03:30:42.109 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-21T03:30:42.110 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-09-21T03:30:42.110 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-21T03:30:42.128 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-21T03:30:42.129 DEBUG:teuthology.orchestra.run.smithi038:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1843221895adccfb3b692060251dd6a0183bc410/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-21T03:30:42.136 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-21T03:30:42.136 DEBUG:teuthology.orchestra.run.smithi195:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1843221895adccfb3b692060251dd6a0183bc410/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-21T03:30:42.196 DEBUG:teuthology.orchestra.run.smithi038:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-21T03:30:42.201 DEBUG:teuthology.orchestra.run.smithi195:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-21T03:30:42.274 DEBUG:teuthology.orchestra.run.smithi038:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-21T03:30:42.275 DEBUG:teuthology.orchestra.run.smithi195:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-21T03:30:42.339 INFO:teuthology.orchestra.run.smithi038.stdout:check_obsoletes = 1 2024-09-21T03:30:42.339 INFO:teuthology.orchestra.run.smithi195.stdout:check_obsoletes = 1 2024-09-21T03:30:42.340 DEBUG:teuthology.orchestra.run.smithi038:> sudo yum clean all 2024-09-21T03:30:42.341 DEBUG:teuthology.orchestra.run.smithi195:> sudo yum clean all 2024-09-21T03:30:42.577 INFO:teuthology.orchestra.run.smithi195.stdout:45 files removed 2024-09-21T03:30:42.578 INFO:teuthology.orchestra.run.smithi038.stdout:45 files removed 2024-09-21T03:30:42.605 DEBUG:teuthology.orchestra.run.smithi195:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-21T03:30:42.606 DEBUG:teuthology.orchestra.run.smithi038:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-21T03:30:43.168 INFO:teuthology.orchestra.run.smithi195.stdout:ceph packages for x86_64 305 kB/s | 84 kB 00:00 2024-09-21T03:30:43.178 INFO:teuthology.orchestra.run.smithi038.stdout:ceph packages for x86_64 309 kB/s | 84 kB 00:00 2024-09-21T03:30:43.468 INFO:teuthology.orchestra.run.smithi195.stdout:ceph noarch packages 94 kB/s | 24 kB 00:00 2024-09-21T03:30:43.471 INFO:teuthology.orchestra.run.smithi038.stdout:ceph noarch packages 98 kB/s | 24 kB 00:00 2024-09-21T03:30:43.729 INFO:teuthology.orchestra.run.smithi038.stdout:ceph source packages 8.7 kB/s | 1.9 kB 00:00 2024-09-21T03:30:43.752 INFO:teuthology.orchestra.run.smithi195.stdout:ceph source packages 7.7 kB/s | 1.9 kB 00:00 2024-09-21T03:30:44.202 INFO:teuthology.orchestra.run.smithi195.stdout:CentOS Stream 9 - BaseOS 20 MB/s | 8.3 MB 00:00 2024-09-21T03:30:44.355 INFO:teuthology.orchestra.run.smithi038.stdout:CentOS Stream 9 - BaseOS 14 MB/s | 8.3 MB 00:00 2024-09-21T03:30:45.904 INFO:teuthology.orchestra.run.smithi195.stdout:CentOS Stream 9 - AppStream 32 MB/s | 20 MB 00:00 2024-09-21T03:30:51.396 INFO:teuthology.orchestra.run.smithi195.stdout:CentOS Stream 9 - CRB 3.9 MB/s | 6.5 MB 00:01 2024-09-21T03:30:52.913 INFO:teuthology.orchestra.run.smithi195.stdout:CentOS Stream 9 - Extras packages 64 kB/s | 19 kB 00:00 2024-09-21T03:30:53.497 INFO:teuthology.orchestra.run.smithi038.stdout:CentOS Stream 9 - AppStream 2.5 MB/s | 20 MB 00:08 2024-09-21T03:30:54.076 INFO:teuthology.orchestra.run.smithi195.stdout:Extra Packages for Enterprise Linux 22 MB/s | 23 MB 00:01 2024-09-21T03:30:59.098 INFO:teuthology.orchestra.run.smithi038.stdout:CentOS Stream 9 - CRB 4.1 MB/s | 6.5 MB 00:01 2024-09-21T03:30:59.557 INFO:teuthology.orchestra.run.smithi195.stdout:lab-extras 33 kB/s | 1.7 kB 00:00 2024-09-21T03:31:00.857 INFO:teuthology.orchestra.run.smithi038.stdout:CentOS Stream 9 - Extras packages 41 kB/s | 19 kB 00:00 2024-09-21T03:31:01.753 INFO:teuthology.orchestra.run.smithi195.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-21T03:31:01.753 INFO:teuthology.orchestra.run.smithi195.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-21T03:31:01.765 INFO:teuthology.orchestra.run.smithi195.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-21T03:31:01.766 INFO:teuthology.orchestra.run.smithi195.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-21T03:31:01.767 INFO:teuthology.orchestra.run.smithi195.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-21T03:31:01.768 INFO:teuthology.orchestra.run.smithi195.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-21T03:31:01.770 INFO:teuthology.orchestra.run.smithi195.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-21T03:31:01.772 INFO:teuthology.orchestra.run.smithi195.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-21T03:31:01.773 INFO:teuthology.orchestra.run.smithi195.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-21T03:31:01.836 INFO:teuthology.orchestra.run.smithi195.stdout:Dependencies resolved. 2024-09-21T03:31:01.853 INFO:teuthology.orchestra.run.smithi195.stdout:======================================================================================= 2024-09-21T03:31:01.853 INFO:teuthology.orchestra.run.smithi195.stdout: Package Arch Version Repository Size 2024-09-21T03:31:01.854 INFO:teuthology.orchestra.run.smithi195.stdout:======================================================================================= 2024-09-21T03:31:01.854 INFO:teuthology.orchestra.run.smithi195.stdout:Installing: 2024-09-21T03:31:01.854 INFO:teuthology.orchestra.run.smithi195.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-21T03:31:01.854 INFO:teuthology.orchestra.run.smithi195.stdout: ceph x86_64 2:19.3.0-5120.g18432218.el9 ceph 6.6 k 2024-09-21T03:31:01.854 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-base x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.5 M 2024-09-21T03:31:01.854 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-fuse x86_64 2:19.3.0-5120.g18432218.el9 ceph 873 k 2024-09-21T03:31:01.854 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-5120.g18432218.el9 ceph 148 k 2024-09-21T03:31:01.854 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mgr x86_64 2:19.3.0-5120.g18432218.el9 ceph 1.5 M 2024-09-21T03:31:01.854 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mgr-cephadm noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 154 k 2024-09-21T03:31:01.854 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mgr-dashboard noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 14 M 2024-09-21T03:31:01.854 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 7.4 M 2024-09-21T03:31:01.854 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mgr-rook noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 49 k 2024-09-21T03:31:01.854 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-radosgw x86_64 2:19.3.0-5120.g18432218.el9 ceph 10 M 2024-09-21T03:31:01.855 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-test x86_64 2:19.3.0-5120.g18432218.el9 ceph 40 M 2024-09-21T03:31:01.855 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-volume noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 285 k 2024-09-21T03:31:01.855 INFO:teuthology.orchestra.run.smithi195.stdout: cephadm noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 784 k 2024-09-21T03:31:01.855 INFO:teuthology.orchestra.run.smithi195.stdout: cephfs-mirror x86_64 2:19.3.0-5120.g18432218.el9 ceph 226 k 2024-09-21T03:31:01.855 INFO:teuthology.orchestra.run.smithi195.stdout: cephfs-shell x86_64 2:19.3.0-5120.g18432218.el9 ceph 27 k 2024-09-21T03:31:01.855 INFO:teuthology.orchestra.run.smithi195.stdout: cephfs-top noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 18 k 2024-09-21T03:31:01.855 INFO:teuthology.orchestra.run.smithi195.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-21T03:31:01.855 INFO:teuthology.orchestra.run.smithi195.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-21T03:31:01.855 INFO:teuthology.orchestra.run.smithi195.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-21T03:31:01.855 INFO:teuthology.orchestra.run.smithi195.stdout: libcephfs-devel x86_64 2:19.3.0-5120.g18432218.el9 ceph 32 k 2024-09-21T03:31:01.855 INFO:teuthology.orchestra.run.smithi195.stdout: libcephfs2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 766 k 2024-09-21T03:31:01.855 INFO:teuthology.orchestra.run.smithi195.stdout: librados-devel x86_64 2:19.3.0-5120.g18432218.el9 ceph 127 k 2024-09-21T03:31:01.855 INFO:teuthology.orchestra.run.smithi195.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-21T03:31:01.856 INFO:teuthology.orchestra.run.smithi195.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-21T03:31:01.856 INFO:teuthology.orchestra.run.smithi195.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-21T03:31:01.856 INFO:teuthology.orchestra.run.smithi195.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-21T03:31:01.856 INFO:teuthology.orchestra.run.smithi195.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-21T03:31:01.856 INFO:teuthology.orchestra.run.smithi195.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-21T03:31:01.856 INFO:teuthology.orchestra.run.smithi195.stdout: python3-cephfs x86_64 2:19.3.0-5120.g18432218.el9 ceph 161 k 2024-09-21T03:31:01.856 INFO:teuthology.orchestra.run.smithi195.stdout: python3-rados x86_64 2:19.3.0-5120.g18432218.el9 ceph 321 k 2024-09-21T03:31:01.856 INFO:teuthology.orchestra.run.smithi195.stdout: python3-rbd x86_64 2:19.3.0-5120.g18432218.el9 ceph 307 k 2024-09-21T03:31:01.856 INFO:teuthology.orchestra.run.smithi195.stdout: python3-rgw x86_64 2:19.3.0-5120.g18432218.el9 ceph 99 k 2024-09-21T03:31:01.856 INFO:teuthology.orchestra.run.smithi195.stdout: rbd-fuse x86_64 2:19.3.0-5120.g18432218.el9 ceph 86 k 2024-09-21T03:31:01.856 INFO:teuthology.orchestra.run.smithi195.stdout: rbd-mirror x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.1 M 2024-09-21T03:31:01.856 INFO:teuthology.orchestra.run.smithi195.stdout: rbd-nbd x86_64 2:19.3.0-5120.g18432218.el9 ceph 170 k 2024-09-21T03:31:01.857 INFO:teuthology.orchestra.run.smithi195.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-21T03:31:01.857 INFO:teuthology.orchestra.run.smithi195.stdout:Upgrading: 2024-09-21T03:31:01.857 INFO:teuthology.orchestra.run.smithi195.stdout: NetworkManager x86_64 1:1.51.0-1.el9 baseos 2.3 M 2024-09-21T03:31:01.857 INFO:teuthology.orchestra.run.smithi195.stdout: NetworkManager-libnm x86_64 1:1.51.0-1.el9 baseos 1.8 M 2024-09-21T03:31:01.857 INFO:teuthology.orchestra.run.smithi195.stdout: NetworkManager-team x86_64 1:1.51.0-1.el9 baseos 41 k 2024-09-21T03:31:01.857 INFO:teuthology.orchestra.run.smithi195.stdout: NetworkManager-tui x86_64 1:1.51.0-1.el9 baseos 249 k 2024-09-21T03:31:01.857 INFO:teuthology.orchestra.run.smithi195.stdout: librados2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.4 M 2024-09-21T03:31:01.857 INFO:teuthology.orchestra.run.smithi195.stdout: librbd1 x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.2 M 2024-09-21T03:31:01.857 INFO:teuthology.orchestra.run.smithi195.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-21T03:31:01.857 INFO:teuthology.orchestra.run.smithi195.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-21T03:31:01.857 INFO:teuthology.orchestra.run.smithi195.stdout:Installing dependencies: 2024-09-21T03:31:01.857 INFO:teuthology.orchestra.run.smithi195.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-21T03:31:01.857 INFO:teuthology.orchestra.run.smithi195.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-21T03:31:01.858 INFO:teuthology.orchestra.run.smithi195.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-21T03:31:01.858 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-common x86_64 2:19.3.0-5120.g18432218.el9 ceph 20 M 2024-09-21T03:31:01.858 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-grafana-dashboards noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 34 k 2024-09-21T03:31:01.858 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mds x86_64 2:19.3.0-5120.g18432218.el9 ceph 2.3 M 2024-09-21T03:31:01.858 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mgr-modules-core noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 292 k 2024-09-21T03:31:01.858 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mon x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.0 M 2024-09-21T03:31:01.858 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-osd x86_64 2:19.3.0-5120.g18432218.el9 ceph 18 M 2024-09-21T03:31:01.858 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-prometheus-alerts noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 16 k 2024-09-21T03:31:01.858 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-selinux x86_64 2:19.3.0-5120.g18432218.el9 ceph 25 k 2024-09-21T03:31:01.858 INFO:teuthology.orchestra.run.smithi195.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-21T03:31:01.858 INFO:teuthology.orchestra.run.smithi195.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-21T03:31:01.858 INFO:teuthology.orchestra.run.smithi195.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-21T03:31:01.858 INFO:teuthology.orchestra.run.smithi195.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-21T03:31:01.859 INFO:teuthology.orchestra.run.smithi195.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-21T03:31:01.859 INFO:teuthology.orchestra.run.smithi195.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-21T03:31:01.859 INFO:teuthology.orchestra.run.smithi195.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-21T03:31:01.859 INFO:teuthology.orchestra.run.smithi195.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-21T03:31:01.859 INFO:teuthology.orchestra.run.smithi195.stdout: libcephsqlite x86_64 2:19.3.0-5120.g18432218.el9 ceph 158 k 2024-09-21T03:31:01.859 INFO:teuthology.orchestra.run.smithi195.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-21T03:31:01.859 INFO:teuthology.orchestra.run.smithi195.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-21T03:31:01.859 INFO:teuthology.orchestra.run.smithi195.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-21T03:31:01.859 INFO:teuthology.orchestra.run.smithi195.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-21T03:31:01.859 INFO:teuthology.orchestra.run.smithi195.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-21T03:31:01.859 INFO:teuthology.orchestra.run.smithi195.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-21T03:31:01.859 INFO:teuthology.orchestra.run.smithi195.stdout: libradosstriper1 x86_64 2:19.3.0-5120.g18432218.el9 ceph 507 k 2024-09-21T03:31:01.859 INFO:teuthology.orchestra.run.smithi195.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-21T03:31:01.860 INFO:teuthology.orchestra.run.smithi195.stdout: librgw2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.3 M 2024-09-21T03:31:01.860 INFO:teuthology.orchestra.run.smithi195.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-21T03:31:01.860 INFO:teuthology.orchestra.run.smithi195.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-21T03:31:01.860 INFO:teuthology.orchestra.run.smithi195.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-21T03:31:01.860 INFO:teuthology.orchestra.run.smithi195.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-21T03:31:01.860 INFO:teuthology.orchestra.run.smithi195.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-21T03:31:01.860 INFO:teuthology.orchestra.run.smithi195.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-21T03:31:01.860 INFO:teuthology.orchestra.run.smithi195.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-21T03:31:01.860 INFO:teuthology.orchestra.run.smithi195.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-21T03:31:01.860 INFO:teuthology.orchestra.run.smithi195.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-21T03:31:01.860 INFO:teuthology.orchestra.run.smithi195.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-21T03:31:01.860 INFO:teuthology.orchestra.run.smithi195.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-21T03:31:01.860 INFO:teuthology.orchestra.run.smithi195.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-21T03:31:01.860 INFO:teuthology.orchestra.run.smithi195.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-21T03:31:01.861 INFO:teuthology.orchestra.run.smithi195.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-21T03:31:01.861 INFO:teuthology.orchestra.run.smithi195.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-21T03:31:01.861 INFO:teuthology.orchestra.run.smithi195.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-21T03:31:01.861 INFO:teuthology.orchestra.run.smithi195.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-21T03:31:01.861 INFO:teuthology.orchestra.run.smithi195.stdout: python3-ceph-argparse x86_64 2:19.3.0-5120.g18432218.el9 ceph 45 k 2024-09-21T03:31:01.861 INFO:teuthology.orchestra.run.smithi195.stdout: python3-ceph-common x86_64 2:19.3.0-5120.g18432218.el9 ceph 148 k 2024-09-21T03:31:01.861 INFO:teuthology.orchestra.run.smithi195.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-21T03:31:01.861 INFO:teuthology.orchestra.run.smithi195.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-21T03:31:01.861 INFO:teuthology.orchestra.run.smithi195.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-21T03:31:01.861 INFO:teuthology.orchestra.run.smithi195.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-21T03:31:01.861 INFO:teuthology.orchestra.run.smithi195.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-21T03:31:01.861 INFO:teuthology.orchestra.run.smithi195.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-21T03:31:01.861 INFO:teuthology.orchestra.run.smithi195.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-21T03:31:01.862 INFO:teuthology.orchestra.run.smithi195.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-21T03:31:01.862 INFO:teuthology.orchestra.run.smithi195.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-21T03:31:01.862 INFO:teuthology.orchestra.run.smithi195.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-21T03:31:01.862 INFO:teuthology.orchestra.run.smithi195.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-21T03:31:01.862 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-21T03:31:01.862 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-21T03:31:01.862 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-21T03:31:01.862 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-21T03:31:01.862 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-21T03:31:01.862 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-21T03:31:01.862 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-21T03:31:01.862 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-21T03:31:01.863 INFO:teuthology.orchestra.run.smithi195.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-21T03:31:01.863 INFO:teuthology.orchestra.run.smithi195.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-21T03:31:01.863 INFO:teuthology.orchestra.run.smithi195.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-21T03:31:01.863 INFO:teuthology.orchestra.run.smithi195.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-21T03:31:01.863 INFO:teuthology.orchestra.run.smithi195.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-21T03:31:01.863 INFO:teuthology.orchestra.run.smithi195.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-21T03:31:01.863 INFO:teuthology.orchestra.run.smithi195.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-21T03:31:01.863 INFO:teuthology.orchestra.run.smithi195.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-21T03:31:01.863 INFO:teuthology.orchestra.run.smithi195.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-21T03:31:01.863 INFO:teuthology.orchestra.run.smithi195.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-21T03:31:01.863 INFO:teuthology.orchestra.run.smithi195.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-21T03:31:01.863 INFO:teuthology.orchestra.run.smithi195.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-21T03:31:01.863 INFO:teuthology.orchestra.run.smithi195.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-21T03:31:01.864 INFO:teuthology.orchestra.run.smithi195.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-21T03:31:01.864 INFO:teuthology.orchestra.run.smithi195.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-21T03:31:01.864 INFO:teuthology.orchestra.run.smithi195.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-21T03:31:01.864 INFO:teuthology.orchestra.run.smithi195.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-21T03:31:01.864 INFO:teuthology.orchestra.run.smithi195.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-21T03:31:01.864 INFO:teuthology.orchestra.run.smithi195.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-21T03:31:01.864 INFO:teuthology.orchestra.run.smithi195.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-21T03:31:01.864 INFO:teuthology.orchestra.run.smithi195.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-21T03:31:01.864 INFO:teuthology.orchestra.run.smithi195.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-21T03:31:01.864 INFO:teuthology.orchestra.run.smithi195.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-21T03:31:01.864 INFO:teuthology.orchestra.run.smithi195.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-21T03:31:01.864 INFO:teuthology.orchestra.run.smithi195.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-21T03:31:01.864 INFO:teuthology.orchestra.run.smithi195.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-21T03:31:01.864 INFO:teuthology.orchestra.run.smithi195.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-21T03:31:01.865 INFO:teuthology.orchestra.run.smithi195.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-21T03:31:01.865 INFO:teuthology.orchestra.run.smithi195.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-21T03:31:01.865 INFO:teuthology.orchestra.run.smithi195.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-21T03:31:01.865 INFO:teuthology.orchestra.run.smithi195.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-21T03:31:01.865 INFO:teuthology.orchestra.run.smithi195.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-21T03:31:01.865 INFO:teuthology.orchestra.run.smithi195.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-21T03:31:01.865 INFO:teuthology.orchestra.run.smithi195.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-21T03:31:01.865 INFO:teuthology.orchestra.run.smithi195.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-21T03:31:01.865 INFO:teuthology.orchestra.run.smithi195.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-21T03:31:01.865 INFO:teuthology.orchestra.run.smithi195.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-21T03:31:01.865 INFO:teuthology.orchestra.run.smithi195.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-21T03:31:01.865 INFO:teuthology.orchestra.run.smithi195.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-21T03:31:01.865 INFO:teuthology.orchestra.run.smithi195.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-21T03:31:01.866 INFO:teuthology.orchestra.run.smithi195.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-21T03:31:01.866 INFO:teuthology.orchestra.run.smithi195.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-21T03:31:01.866 INFO:teuthology.orchestra.run.smithi195.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-21T03:31:01.866 INFO:teuthology.orchestra.run.smithi195.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-21T03:31:01.866 INFO:teuthology.orchestra.run.smithi195.stdout:Installing weak dependencies: 2024-09-21T03:31:01.866 INFO:teuthology.orchestra.run.smithi195.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-21T03:31:01.866 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-21T03:31:01.866 INFO:teuthology.orchestra.run.smithi195.stdout:Transaction Summary 2024-09-21T03:31:01.866 INFO:teuthology.orchestra.run.smithi195.stdout:======================================================================================= 2024-09-21T03:31:01.866 INFO:teuthology.orchestra.run.smithi195.stdout:Install 149 Packages 2024-09-21T03:31:01.866 INFO:teuthology.orchestra.run.smithi195.stdout:Upgrade 8 Packages 2024-09-21T03:31:01.866 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-21T03:31:01.867 INFO:teuthology.orchestra.run.smithi195.stdout:Total download size: 228 M 2024-09-21T03:31:01.867 INFO:teuthology.orchestra.run.smithi195.stdout:Downloading Packages: 2024-09-21T03:31:02.002 INFO:teuthology.orchestra.run.smithi038.stdout:Extra Packages for Enterprise Linux 22 MB/s | 23 MB 00:01 2024-09-21T03:31:02.702 INFO:teuthology.orchestra.run.smithi195.stdout:(1/157): ceph-19.3.0-5120.g18432218.el9.x86_64. 45 kB/s | 6.6 kB 00:00 2024-09-21T03:31:03.135 INFO:teuthology.orchestra.run.smithi195.stdout:(2/157): ceph-fuse-19.3.0-5120.g18432218.el9.x8 2.0 MB/s | 873 kB 00:00 2024-09-21T03:31:03.210 INFO:teuthology.orchestra.run.smithi195.stdout:(3/157): ceph-immutable-object-cache-19.3.0-512 1.9 MB/s | 148 kB 00:00 2024-09-21T03:31:04.302 INFO:teuthology.orchestra.run.smithi195.stdout:(4/157): ceph-common-19.3.0-5120.g18432218.el9. 11 MB/s | 20 MB 00:01 2024-09-21T03:31:04.386 INFO:teuthology.orchestra.run.smithi195.stdout:(5/157): ceph-mds-19.3.0-5120.g18432218.el9.x86 1.9 MB/s | 2.3 MB 00:01 2024-09-21T03:31:04.428 INFO:teuthology.orchestra.run.smithi195.stdout:(6/157): ceph-mgr-19.3.0-5120.g18432218.el9.x86 12 MB/s | 1.5 MB 00:00 2024-09-21T03:31:04.511 INFO:teuthology.orchestra.run.smithi195.stdout:(7/157): ceph-base-19.3.0-5120.g18432218.el9.x8 2.8 MB/s | 5.5 MB 00:01 2024-09-21T03:31:05.512 INFO:teuthology.orchestra.run.smithi195.stdout:(8/157): ceph-osd-19.3.0-5120.g18432218.el9.x86 16 MB/s | 18 MB 00:01 2024-09-21T03:31:05.553 INFO:teuthology.orchestra.run.smithi195.stdout:(9/157): ceph-selinux-19.3.0-5120.g18432218.el9 610 kB/s | 25 kB 00:00 2024-09-21T03:31:05.862 INFO:teuthology.orchestra.run.smithi195.stdout:(10/157): ceph-mon-19.3.0-5120.g18432218.el9.x8 3.4 MB/s | 5.0 MB 00:01 2024-09-21T03:31:05.963 INFO:teuthology.orchestra.run.smithi195.stdout:(11/157): cephfs-mirror-19.3.0-5120.g18432218.e 2.2 MB/s | 226 kB 00:00 2024-09-21T03:31:06.013 INFO:teuthology.orchestra.run.smithi195.stdout:(12/157): cephfs-shell-19.3.0-5120.g18432218.el 542 kB/s | 27 kB 00:00 2024-09-21T03:31:06.063 INFO:teuthology.orchestra.run.smithi195.stdout:(13/157): libcephfs-devel-19.3.0-5120.g18432218 636 kB/s | 32 kB 00:00 2024-09-21T03:31:06.230 INFO:teuthology.orchestra.run.smithi195.stdout:(14/157): libcephfs2-19.3.0-5120.g18432218.el9. 4.5 MB/s | 766 kB 00:00 2024-09-21T03:31:06.305 INFO:teuthology.orchestra.run.smithi195.stdout:(15/157): libcephsqlite-19.3.0-5120.g18432218.e 2.1 MB/s | 158 kB 00:00 2024-09-21T03:31:06.447 INFO:teuthology.orchestra.run.smithi195.stdout:(16/157): ceph-radosgw-19.3.0-5120.g18432218.el 5.4 MB/s | 10 MB 00:01 2024-09-21T03:31:06.464 INFO:teuthology.orchestra.run.smithi195.stdout:(17/157): librados-devel-19.3.0-5120.g18432218. 799 kB/s | 127 kB 00:00 2024-09-21T03:31:06.558 INFO:teuthology.orchestra.run.smithi195.stdout:(18/157): libradosstriper1-19.3.0-5120.g1843221 4.5 MB/s | 507 kB 00:00 2024-09-21T03:31:06.606 INFO:teuthology.orchestra.run.smithi195.stdout:(19/157): python3-ceph-argparse-19.3.0-5120.g18 946 kB/s | 45 kB 00:00 2024-09-21T03:31:06.665 INFO:teuthology.orchestra.run.smithi195.stdout:(20/157): python3-ceph-common-19.3.0-5120.g1843 2.5 MB/s | 148 kB 00:00 2024-09-21T03:31:06.723 INFO:teuthology.orchestra.run.smithi195.stdout:(21/157): python3-cephfs-19.3.0-5120.g18432218. 2.7 MB/s | 161 kB 00:00 2024-09-21T03:31:06.798 INFO:teuthology.orchestra.run.smithi195.stdout:(22/157): python3-rados-19.3.0-5120.g18432218.e 4.2 MB/s | 321 kB 00:00 2024-09-21T03:31:07.199 INFO:teuthology.orchestra.run.smithi195.stdout:(23/157): ceph-test-19.3.0-5120.g18432218.el9.x 24 MB/s | 40 MB 00:01 2024-09-21T03:31:07.224 INFO:teuthology.orchestra.run.smithi195.stdout:(24/157): python3-rbd-19.3.0-5120.g18432218.el9 721 kB/s | 307 kB 00:00 2024-09-21T03:31:07.249 INFO:teuthology.orchestra.run.smithi195.stdout:(25/157): python3-rgw-19.3.0-5120.g18432218.el9 1.9 MB/s | 99 kB 00:00 2024-09-21T03:31:07.275 INFO:teuthology.orchestra.run.smithi195.stdout:(26/157): rbd-fuse-19.3.0-5120.g18432218.el9.x8 1.7 MB/s | 86 kB 00:00 2024-09-21T03:31:07.350 INFO:teuthology.orchestra.run.smithi195.stdout:(27/157): rbd-nbd-19.3.0-5120.g18432218.el9.x86 2.2 MB/s | 170 kB 00:00 2024-09-21T03:31:07.417 INFO:teuthology.orchestra.run.smithi195.stdout:(28/157): rbd-mirror-19.3.0-5120.g18432218.el9. 19 MB/s | 3.1 MB 00:00 2024-09-21T03:31:07.434 INFO:teuthology.orchestra.run.smithi195.stdout:(29/157): ceph-grafana-dashboards-19.3.0-5120.g 411 kB/s | 34 kB 00:00 2024-09-21T03:31:07.468 INFO:teuthology.orchestra.run.smithi195.stdout:(30/157): ceph-mgr-cephadm-19.3.0-5120.g1843221 3.0 MB/s | 154 kB 00:00 2024-09-21T03:31:07.625 INFO:teuthology.orchestra.run.smithi038.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-09-21T03:31:07.643 INFO:teuthology.orchestra.run.smithi195.stdout:(31/157): librgw2-19.3.0-5120.g18432218.el9.x86 4.5 MB/s | 5.3 MB 00:01 2024-09-21T03:31:07.793 INFO:teuthology.orchestra.run.smithi195.stdout:(32/157): ceph-mgr-diskprediction-local-19.3.0- 23 MB/s | 7.4 MB 00:00 2024-09-21T03:31:07.819 INFO:teuthology.orchestra.run.smithi195.stdout:(33/157): ceph-mgr-modules-core-19.3.0-5120.g18 1.6 MB/s | 292 kB 00:00 2024-09-21T03:31:07.844 INFO:teuthology.orchestra.run.smithi195.stdout:(34/157): ceph-mgr-rook-19.3.0-5120.g18432218.e 990 kB/s | 49 kB 00:00 2024-09-21T03:31:07.869 INFO:teuthology.orchestra.run.smithi195.stdout:(35/157): ceph-prometheus-alerts-19.3.0-5120.g1 328 kB/s | 16 kB 00:00 2024-09-21T03:31:07.894 INFO:teuthology.orchestra.run.smithi195.stdout:(36/157): ceph-volume-19.3.0-5120.g18432218.el9 5.6 MB/s | 285 kB 00:00 2024-09-21T03:31:07.936 INFO:teuthology.orchestra.run.smithi195.stdout:(37/157): cephfs-top-19.3.0-5120.g18432218.el9. 443 kB/s | 18 kB 00:00 2024-09-21T03:31:08.011 INFO:teuthology.orchestra.run.smithi195.stdout:(38/157): cephadm-19.3.0-5120.g18432218.el9.noa 5.4 MB/s | 784 kB 00:00 2024-09-21T03:31:08.570 INFO:teuthology.orchestra.run.smithi195.stdout:(39/157): fuse-2.9.9-16.el9.x86_64.rpm 126 kB/s | 80 kB 00:00 2024-09-21T03:31:08.887 INFO:teuthology.orchestra.run.smithi195.stdout:(40/157): libgfortran-11.5.0-2.el9.x86_64.rpm 916 kB/s | 801 kB 00:00 2024-09-21T03:31:09.321 INFO:teuthology.orchestra.run.smithi195.stdout:(41/157): libquadmath-11.5.0-2.el9.x86_64.rpm 255 kB/s | 191 kB 00:00 2024-09-21T03:31:09.654 INFO:teuthology.orchestra.run.smithi195.stdout:(42/157): ceph-mgr-dashboard-19.3.0-5120.g18432 6.4 MB/s | 14 MB 00:02 2024-09-21T03:31:09.687 INFO:teuthology.orchestra.run.smithi195.stdout:(43/157): python3-cffi-1.14.5-5.el9.x86_64.rpm 317 kB/s | 253 kB 00:00 2024-09-21T03:31:09.903 INFO:teuthology.orchestra.run.smithi038.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-21T03:31:09.904 INFO:teuthology.orchestra.run.smithi038.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-21T03:31:09.916 INFO:teuthology.orchestra.run.smithi038.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-21T03:31:09.917 INFO:teuthology.orchestra.run.smithi038.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-21T03:31:09.918 INFO:teuthology.orchestra.run.smithi038.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-21T03:31:09.919 INFO:teuthology.orchestra.run.smithi038.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-21T03:31:09.921 INFO:teuthology.orchestra.run.smithi038.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-21T03:31:09.923 INFO:teuthology.orchestra.run.smithi038.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-21T03:31:09.924 INFO:teuthology.orchestra.run.smithi038.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-21T03:31:09.991 INFO:teuthology.orchestra.run.smithi038.stdout:Dependencies resolved. 2024-09-21T03:31:10.010 INFO:teuthology.orchestra.run.smithi038.stdout:======================================================================================= 2024-09-21T03:31:10.010 INFO:teuthology.orchestra.run.smithi038.stdout: Package Arch Version Repository Size 2024-09-21T03:31:10.010 INFO:teuthology.orchestra.run.smithi038.stdout:======================================================================================= 2024-09-21T03:31:10.010 INFO:teuthology.orchestra.run.smithi038.stdout:Installing: 2024-09-21T03:31:10.011 INFO:teuthology.orchestra.run.smithi038.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-21T03:31:10.011 INFO:teuthology.orchestra.run.smithi038.stdout: ceph x86_64 2:19.3.0-5120.g18432218.el9 ceph 6.6 k 2024-09-21T03:31:10.011 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-base x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.5 M 2024-09-21T03:31:10.011 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-fuse x86_64 2:19.3.0-5120.g18432218.el9 ceph 873 k 2024-09-21T03:31:10.011 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-5120.g18432218.el9 ceph 148 k 2024-09-21T03:31:10.011 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mgr x86_64 2:19.3.0-5120.g18432218.el9 ceph 1.5 M 2024-09-21T03:31:10.011 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mgr-cephadm noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 154 k 2024-09-21T03:31:10.011 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mgr-dashboard noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 14 M 2024-09-21T03:31:10.011 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 7.4 M 2024-09-21T03:31:10.011 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mgr-rook noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 49 k 2024-09-21T03:31:10.011 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-radosgw x86_64 2:19.3.0-5120.g18432218.el9 ceph 10 M 2024-09-21T03:31:10.011 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-test x86_64 2:19.3.0-5120.g18432218.el9 ceph 40 M 2024-09-21T03:31:10.012 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-volume noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 285 k 2024-09-21T03:31:10.012 INFO:teuthology.orchestra.run.smithi038.stdout: cephadm noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 784 k 2024-09-21T03:31:10.012 INFO:teuthology.orchestra.run.smithi038.stdout: cephfs-mirror x86_64 2:19.3.0-5120.g18432218.el9 ceph 226 k 2024-09-21T03:31:10.012 INFO:teuthology.orchestra.run.smithi038.stdout: cephfs-shell x86_64 2:19.3.0-5120.g18432218.el9 ceph 27 k 2024-09-21T03:31:10.012 INFO:teuthology.orchestra.run.smithi038.stdout: cephfs-top noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 18 k 2024-09-21T03:31:10.012 INFO:teuthology.orchestra.run.smithi038.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-21T03:31:10.012 INFO:teuthology.orchestra.run.smithi038.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-21T03:31:10.012 INFO:teuthology.orchestra.run.smithi038.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-21T03:31:10.012 INFO:teuthology.orchestra.run.smithi038.stdout: libcephfs-devel x86_64 2:19.3.0-5120.g18432218.el9 ceph 32 k 2024-09-21T03:31:10.012 INFO:teuthology.orchestra.run.smithi038.stdout: libcephfs2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 766 k 2024-09-21T03:31:10.012 INFO:teuthology.orchestra.run.smithi038.stdout: librados-devel x86_64 2:19.3.0-5120.g18432218.el9 ceph 127 k 2024-09-21T03:31:10.012 INFO:teuthology.orchestra.run.smithi038.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-21T03:31:10.013 INFO:teuthology.orchestra.run.smithi038.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-21T03:31:10.013 INFO:teuthology.orchestra.run.smithi038.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-21T03:31:10.013 INFO:teuthology.orchestra.run.smithi038.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-21T03:31:10.013 INFO:teuthology.orchestra.run.smithi038.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-21T03:31:10.013 INFO:teuthology.orchestra.run.smithi038.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-21T03:31:10.013 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cephfs x86_64 2:19.3.0-5120.g18432218.el9 ceph 161 k 2024-09-21T03:31:10.013 INFO:teuthology.orchestra.run.smithi038.stdout: python3-rados x86_64 2:19.3.0-5120.g18432218.el9 ceph 321 k 2024-09-21T03:31:10.013 INFO:teuthology.orchestra.run.smithi038.stdout: python3-rbd x86_64 2:19.3.0-5120.g18432218.el9 ceph 307 k 2024-09-21T03:31:10.013 INFO:teuthology.orchestra.run.smithi038.stdout: python3-rgw x86_64 2:19.3.0-5120.g18432218.el9 ceph 99 k 2024-09-21T03:31:10.013 INFO:teuthology.orchestra.run.smithi038.stdout: rbd-fuse x86_64 2:19.3.0-5120.g18432218.el9 ceph 86 k 2024-09-21T03:31:10.013 INFO:teuthology.orchestra.run.smithi038.stdout: rbd-mirror x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.1 M 2024-09-21T03:31:10.013 INFO:teuthology.orchestra.run.smithi038.stdout: rbd-nbd x86_64 2:19.3.0-5120.g18432218.el9 ceph 170 k 2024-09-21T03:31:10.013 INFO:teuthology.orchestra.run.smithi038.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-21T03:31:10.013 INFO:teuthology.orchestra.run.smithi038.stdout:Upgrading: 2024-09-21T03:31:10.014 INFO:teuthology.orchestra.run.smithi038.stdout: NetworkManager x86_64 1:1.51.0-1.el9 baseos 2.3 M 2024-09-21T03:31:10.014 INFO:teuthology.orchestra.run.smithi038.stdout: NetworkManager-libnm x86_64 1:1.51.0-1.el9 baseos 1.8 M 2024-09-21T03:31:10.014 INFO:teuthology.orchestra.run.smithi038.stdout: NetworkManager-team x86_64 1:1.51.0-1.el9 baseos 41 k 2024-09-21T03:31:10.014 INFO:teuthology.orchestra.run.smithi038.stdout: NetworkManager-tui x86_64 1:1.51.0-1.el9 baseos 249 k 2024-09-21T03:31:10.014 INFO:teuthology.orchestra.run.smithi038.stdout: librados2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.4 M 2024-09-21T03:31:10.014 INFO:teuthology.orchestra.run.smithi038.stdout: librbd1 x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.2 M 2024-09-21T03:31:10.014 INFO:teuthology.orchestra.run.smithi038.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-21T03:31:10.014 INFO:teuthology.orchestra.run.smithi038.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-21T03:31:10.014 INFO:teuthology.orchestra.run.smithi038.stdout:Installing dependencies: 2024-09-21T03:31:10.014 INFO:teuthology.orchestra.run.smithi038.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-21T03:31:10.014 INFO:teuthology.orchestra.run.smithi038.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-21T03:31:10.015 INFO:teuthology.orchestra.run.smithi038.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-21T03:31:10.015 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-common x86_64 2:19.3.0-5120.g18432218.el9 ceph 20 M 2024-09-21T03:31:10.015 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-grafana-dashboards noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 34 k 2024-09-21T03:31:10.015 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mds x86_64 2:19.3.0-5120.g18432218.el9 ceph 2.3 M 2024-09-21T03:31:10.015 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mgr-modules-core noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 292 k 2024-09-21T03:31:10.015 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mon x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.0 M 2024-09-21T03:31:10.015 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-osd x86_64 2:19.3.0-5120.g18432218.el9 ceph 18 M 2024-09-21T03:31:10.015 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-prometheus-alerts noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 16 k 2024-09-21T03:31:10.015 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-selinux x86_64 2:19.3.0-5120.g18432218.el9 ceph 25 k 2024-09-21T03:31:10.015 INFO:teuthology.orchestra.run.smithi038.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-21T03:31:10.015 INFO:teuthology.orchestra.run.smithi038.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-21T03:31:10.015 INFO:teuthology.orchestra.run.smithi038.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-21T03:31:10.015 INFO:teuthology.orchestra.run.smithi038.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-21T03:31:10.016 INFO:teuthology.orchestra.run.smithi038.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-21T03:31:10.016 INFO:teuthology.orchestra.run.smithi038.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-21T03:31:10.016 INFO:teuthology.orchestra.run.smithi038.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-21T03:31:10.016 INFO:teuthology.orchestra.run.smithi038.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-21T03:31:10.016 INFO:teuthology.orchestra.run.smithi038.stdout: libcephsqlite x86_64 2:19.3.0-5120.g18432218.el9 ceph 158 k 2024-09-21T03:31:10.016 INFO:teuthology.orchestra.run.smithi038.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-21T03:31:10.016 INFO:teuthology.orchestra.run.smithi038.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-21T03:31:10.016 INFO:teuthology.orchestra.run.smithi038.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-21T03:31:10.016 INFO:teuthology.orchestra.run.smithi038.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-21T03:31:10.016 INFO:teuthology.orchestra.run.smithi038.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-21T03:31:10.016 INFO:teuthology.orchestra.run.smithi038.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-21T03:31:10.016 INFO:teuthology.orchestra.run.smithi038.stdout: libradosstriper1 x86_64 2:19.3.0-5120.g18432218.el9 ceph 507 k 2024-09-21T03:31:10.016 INFO:teuthology.orchestra.run.smithi038.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-21T03:31:10.017 INFO:teuthology.orchestra.run.smithi038.stdout: librgw2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.3 M 2024-09-21T03:31:10.017 INFO:teuthology.orchestra.run.smithi038.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-21T03:31:10.017 INFO:teuthology.orchestra.run.smithi038.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-21T03:31:10.017 INFO:teuthology.orchestra.run.smithi038.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-21T03:31:10.017 INFO:teuthology.orchestra.run.smithi038.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-21T03:31:10.017 INFO:teuthology.orchestra.run.smithi038.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-21T03:31:10.017 INFO:teuthology.orchestra.run.smithi038.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-21T03:31:10.017 INFO:teuthology.orchestra.run.smithi038.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-21T03:31:10.017 INFO:teuthology.orchestra.run.smithi038.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-21T03:31:10.017 INFO:teuthology.orchestra.run.smithi038.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-21T03:31:10.017 INFO:teuthology.orchestra.run.smithi038.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-21T03:31:10.018 INFO:teuthology.orchestra.run.smithi038.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-21T03:31:10.018 INFO:teuthology.orchestra.run.smithi038.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-21T03:31:10.018 INFO:teuthology.orchestra.run.smithi038.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-21T03:31:10.018 INFO:teuthology.orchestra.run.smithi038.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-21T03:31:10.018 INFO:teuthology.orchestra.run.smithi038.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-21T03:31:10.018 INFO:teuthology.orchestra.run.smithi038.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-21T03:31:10.018 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-21T03:31:10.018 INFO:teuthology.orchestra.run.smithi038.stdout: python3-ceph-argparse x86_64 2:19.3.0-5120.g18432218.el9 ceph 45 k 2024-09-21T03:31:10.018 INFO:teuthology.orchestra.run.smithi038.stdout: python3-ceph-common x86_64 2:19.3.0-5120.g18432218.el9 ceph 148 k 2024-09-21T03:31:10.018 INFO:teuthology.orchestra.run.smithi038.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-21T03:31:10.018 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-21T03:31:10.018 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-21T03:31:10.019 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-21T03:31:10.019 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-21T03:31:10.019 INFO:teuthology.orchestra.run.smithi038.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-21T03:31:10.019 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-21T03:31:10.019 INFO:teuthology.orchestra.run.smithi038.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-21T03:31:10.019 INFO:teuthology.orchestra.run.smithi038.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-21T03:31:10.019 INFO:teuthology.orchestra.run.smithi038.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-21T03:31:10.019 INFO:teuthology.orchestra.run.smithi038.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-21T03:31:10.019 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-21T03:31:10.019 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-21T03:31:10.019 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-21T03:31:10.020 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-21T03:31:10.020 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-21T03:31:10.020 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-21T03:31:10.020 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-21T03:31:10.020 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-21T03:31:10.020 INFO:teuthology.orchestra.run.smithi038.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-21T03:31:10.020 INFO:teuthology.orchestra.run.smithi038.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-21T03:31:10.020 INFO:teuthology.orchestra.run.smithi038.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-21T03:31:10.020 INFO:teuthology.orchestra.run.smithi038.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-21T03:31:10.020 INFO:teuthology.orchestra.run.smithi038.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-21T03:31:10.020 INFO:teuthology.orchestra.run.smithi038.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-21T03:31:10.020 INFO:teuthology.orchestra.run.smithi038.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-21T03:31:10.020 INFO:teuthology.orchestra.run.smithi038.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-21T03:31:10.020 INFO:teuthology.orchestra.run.smithi038.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-21T03:31:10.021 INFO:teuthology.orchestra.run.smithi038.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-21T03:31:10.021 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-21T03:31:10.021 INFO:teuthology.orchestra.run.smithi038.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-21T03:31:10.021 INFO:teuthology.orchestra.run.smithi038.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-21T03:31:10.021 INFO:teuthology.orchestra.run.smithi038.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-21T03:31:10.021 INFO:teuthology.orchestra.run.smithi038.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-21T03:31:10.021 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-21T03:31:10.021 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-21T03:31:10.021 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-21T03:31:10.021 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-21T03:31:10.021 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-21T03:31:10.021 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-21T03:31:10.021 INFO:teuthology.orchestra.run.smithi038.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-21T03:31:10.021 INFO:teuthology.orchestra.run.smithi038.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-21T03:31:10.021 INFO:teuthology.orchestra.run.smithi038.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-21T03:31:10.022 INFO:teuthology.orchestra.run.smithi038.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-21T03:31:10.022 INFO:teuthology.orchestra.run.smithi038.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-21T03:31:10.022 INFO:teuthology.orchestra.run.smithi038.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-21T03:31:10.022 INFO:teuthology.orchestra.run.smithi038.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-21T03:31:10.022 INFO:teuthology.orchestra.run.smithi038.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-21T03:31:10.022 INFO:teuthology.orchestra.run.smithi038.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-21T03:31:10.022 INFO:teuthology.orchestra.run.smithi038.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-21T03:31:10.022 INFO:teuthology.orchestra.run.smithi038.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-21T03:31:10.022 INFO:teuthology.orchestra.run.smithi038.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-21T03:31:10.022 INFO:teuthology.orchestra.run.smithi038.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-21T03:31:10.022 INFO:teuthology.orchestra.run.smithi038.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-21T03:31:10.022 INFO:teuthology.orchestra.run.smithi038.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-21T03:31:10.022 INFO:teuthology.orchestra.run.smithi038.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-21T03:31:10.023 INFO:teuthology.orchestra.run.smithi038.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-21T03:31:10.023 INFO:teuthology.orchestra.run.smithi038.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-21T03:31:10.023 INFO:teuthology.orchestra.run.smithi038.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-21T03:31:10.023 INFO:teuthology.orchestra.run.smithi038.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-21T03:31:10.023 INFO:teuthology.orchestra.run.smithi038.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-21T03:31:10.023 INFO:teuthology.orchestra.run.smithi038.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-21T03:31:10.023 INFO:teuthology.orchestra.run.smithi038.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-21T03:31:10.023 INFO:teuthology.orchestra.run.smithi038.stdout:Installing weak dependencies: 2024-09-21T03:31:10.023 INFO:teuthology.orchestra.run.smithi038.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-21T03:31:10.023 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:31:10.023 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction Summary 2024-09-21T03:31:10.023 INFO:teuthology.orchestra.run.smithi038.stdout:======================================================================================= 2024-09-21T03:31:10.024 INFO:teuthology.orchestra.run.smithi038.stdout:Install 149 Packages 2024-09-21T03:31:10.024 INFO:teuthology.orchestra.run.smithi038.stdout:Upgrade 8 Packages 2024-09-21T03:31:10.024 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:31:10.024 INFO:teuthology.orchestra.run.smithi038.stdout:Total download size: 228 M 2024-09-21T03:31:10.024 INFO:teuthology.orchestra.run.smithi038.stdout:Downloading Packages: 2024-09-21T03:31:10.329 INFO:teuthology.orchestra.run.smithi195.stdout:(44/157): python3-ply-3.11-14.el9.noarch.rpm 160 kB/s | 106 kB 00:00 2024-09-21T03:31:10.380 INFO:teuthology.orchestra.run.smithi195.stdout:(45/157): python3-pycparser-2.20-6.el9.noarch.r 195 kB/s | 135 kB 00:00 2024-09-21T03:31:10.413 INFO:teuthology.orchestra.run.smithi195.stdout:(46/157): python3-cryptography-36.0.1-4.el9.x86 1.1 MB/s | 1.2 MB 00:01 2024-09-21T03:31:10.630 INFO:teuthology.orchestra.run.smithi195.stdout:(47/157): automake-1.16.2-8.el9.noarch.rpm 3.1 MB/s | 689 kB 00:00 2024-09-21T03:31:10.714 INFO:teuthology.orchestra.run.smithi195.stdout:(48/157): bison-3.7.4-5.el9.x86_64.rpm 12 MB/s | 943 kB 00:00 2024-09-21T03:31:10.764 INFO:teuthology.orchestra.run.smithi195.stdout:(49/157): boost-program-options-1.75.0-8.el9.x8 2.1 MB/s | 107 kB 00:00 2024-09-21T03:31:10.806 INFO:teuthology.orchestra.run.smithi195.stdout:(50/157): elfutils-libelf-devel-0.191-4.el9.x86 520 kB/s | 22 kB 00:00 2024-09-21T03:31:10.856 INFO:teuthology.orchestra.run.smithi195.stdout:(51/157): flex-2.6.4-9.el9.x86_64.rpm 6.1 MB/s | 313 kB 00:00 2024-09-21T03:31:10.898 INFO:teuthology.orchestra.run.smithi195.stdout:(52/157): flexiblas-3.0.4-8.el9.x86_64.rpm 789 kB/s | 33 kB 00:00 2024-09-21T03:31:10.998 INFO:teuthology.orchestra.run.smithi195.stdout:(53/157): flexiblas-netlib-3.0.4-8.el9.x86_64.r 30 MB/s | 3.0 MB 00:00 2024-09-21T03:31:11.010 INFO:teuthology.orchestra.run.smithi038.stdout:(1/157): ceph-19.3.0-5120.g18432218.el9.x86_64. 34 kB/s | 6.6 kB 00:00 2024-09-21T03:31:11.032 INFO:teuthology.orchestra.run.smithi195.stdout:(54/157): python3-requests-2.25.1-8.el9.noarch. 178 kB/s | 125 kB 00:00 2024-09-21T03:31:11.057 INFO:teuthology.orchestra.run.smithi195.stdout:(55/157): flexiblas-openblas-openmp-3.0.4-8.el9 310 kB/s | 18 kB 00:00 2024-09-21T03:31:11.099 INFO:teuthology.orchestra.run.smithi195.stdout:(56/157): libaio-devel-0.3.111-13.el9.x86_64.rp 167 kB/s | 11 kB 00:00 2024-09-21T03:31:11.124 INFO:teuthology.orchestra.run.smithi195.stdout:(57/157): libnbd-1.20.2-2.el9.x86_64.rpm 2.5 MB/s | 171 kB 00:00 2024-09-21T03:31:11.175 INFO:teuthology.orchestra.run.smithi195.stdout:(58/157): python3-urllib3-1.26.5-6.el9.noarch.r 270 kB/s | 215 kB 00:00 2024-09-21T03:31:11.200 INFO:teuthology.orchestra.run.smithi195.stdout:(59/157): librabbitmq-0.11.0-7.el9.x86_64.rpm 600 kB/s | 45 kB 00:00 2024-09-21T03:31:11.225 INFO:teuthology.orchestra.run.smithi195.stdout:(60/157): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.2 MB/s | 160 kB 00:00 2024-09-21T03:31:11.251 INFO:teuthology.orchestra.run.smithi195.stdout:(61/157): libtool-2.4.6-46.el9.x86_64.rpm 11 MB/s | 581 kB 00:00 2024-09-21T03:31:11.276 INFO:teuthology.orchestra.run.smithi195.stdout:(62/157): libuuid-devel-2.37.4-20.el9.x86_64.rp 488 kB/s | 24 kB 00:00 2024-09-21T03:31:11.285 INFO:teuthology.orchestra.run.smithi038.stdout:(2/157): ceph-fuse-19.3.0-5120.g18432218.el9.x8 3.1 MB/s | 873 kB 00:00 2024-09-21T03:31:11.301 INFO:teuthology.orchestra.run.smithi195.stdout:(63/157): libzstd-devel-1.5.1-2.el9.x86_64.rpm 946 kB/s | 47 kB 00:00 2024-09-21T03:31:11.351 INFO:teuthology.orchestra.run.smithi195.stdout:(64/157): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 188 kB 00:00 2024-09-21T03:31:11.352 INFO:teuthology.orchestra.run.smithi038.stdout:(3/157): ceph-immutable-object-cache-19.3.0-512 2.2 MB/s | 148 kB 00:00 2024-09-21T03:31:11.377 INFO:teuthology.orchestra.run.smithi195.stdout:(65/157): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.8 MB/s | 292 kB 00:00 2024-09-21T03:31:11.402 INFO:teuthology.orchestra.run.smithi195.stdout:(66/157): openblas-0.3.26-2.el9.x86_64.rpm 766 kB/s | 38 kB 00:00 2024-09-21T03:31:11.427 INFO:teuthology.orchestra.run.smithi195.stdout:(67/157): librdkafka-1.6.1-102.el9.x86_64.rpm 2.6 MB/s | 662 kB 00:00 2024-09-21T03:31:11.511 INFO:teuthology.orchestra.run.smithi195.stdout:(68/157): openssl-devel-3.2.2-6.el9.x86_64.rpm 41 MB/s | 4.4 MB 00:00 2024-09-21T03:31:11.553 INFO:teuthology.orchestra.run.smithi195.stdout:(69/157): postgresql-13.16-1.el9.x86_64.rpm 13 MB/s | 1.6 MB 00:00 2024-09-21T03:31:11.586 INFO:teuthology.orchestra.run.smithi195.stdout:(70/157): postgresql-contrib-13.16-1.el9.x86_64 12 MB/s | 910 kB 00:00 2024-09-21T03:31:11.611 INFO:teuthology.orchestra.run.smithi195.stdout:(71/157): postgresql-private-libs-13.16-1.el9.x 2.3 MB/s | 137 kB 00:00 2024-09-21T03:31:11.686 INFO:teuthology.orchestra.run.smithi038.stdout:(4/157): ceph-mds-19.3.0-5120.g18432218.el9.x86 6.9 MB/s | 2.3 MB 00:00 2024-09-21T03:31:11.687 INFO:teuthology.orchestra.run.smithi195.stdout:(72/157): openblas-openmp-0.3.26-2.el9.x86_64.r 16 MB/s | 4.9 MB 00:00 2024-09-21T03:31:11.721 INFO:teuthology.orchestra.run.smithi195.stdout:(73/157): protobuf-3.14.0-13.el9.x86_64.rpm 9.3 MB/s | 1.0 MB 00:00 2024-09-21T03:31:11.746 INFO:teuthology.orchestra.run.smithi195.stdout:(74/157): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-09-21T03:31:11.761 INFO:teuthology.orchestra.run.smithi038.stdout:(5/157): ceph-base-19.3.0-5120.g18432218.el9.x8 5.9 MB/s | 5.5 MB 00:00 2024-09-21T03:31:11.838 INFO:teuthology.orchestra.run.smithi195.stdout:(75/157): postgresql-server-13.16-1.el9.x86_64. 23 MB/s | 5.8 MB 00:00 2024-09-21T03:31:11.861 INFO:teuthology.orchestra.run.smithi038.stdout:(6/157): ceph-mgr-19.3.0-5120.g18432218.el9.x86 8.7 MB/s | 1.5 MB 00:00 2024-09-21T03:31:11.863 INFO:teuthology.orchestra.run.smithi195.stdout:(76/157): python3-colorama-0.4.6-3.el9.noarch.r 523 kB/s | 61 kB 00:00 2024-09-21T03:31:11.888 INFO:teuthology.orchestra.run.smithi195.stdout:(77/157): python3-devel-3.9.19-8.el9.x86_64.rpm 4.8 MB/s | 245 kB 00:00 2024-09-21T03:31:11.914 INFO:teuthology.orchestra.run.smithi195.stdout:(78/157): python3-jinja2-2.11.3-6.el9.noarch.rp 4.8 MB/s | 248 kB 00:00 2024-09-21T03:31:11.930 INFO:teuthology.orchestra.run.smithi195.stdout:(79/157): python3-jsonpatch-1.21-16.el9.noarch. 613 kB/s | 26 kB 00:00 2024-09-21T03:31:11.947 INFO:teuthology.orchestra.run.smithi195.stdout:(80/157): python3-jsonpointer-2.0-4.el9.noarch. 574 kB/s | 19 kB 00:00 2024-09-21T03:31:11.981 INFO:teuthology.orchestra.run.smithi195.stdout:(81/157): python3-mako-1.1.4-6.el9.noarch.rpm 3.3 MB/s | 172 kB 00:00 2024-09-21T03:31:11.998 INFO:teuthology.orchestra.run.smithi195.stdout:(82/157): python3-markupsafe-1.1.1-12.el9.x86_6 689 kB/s | 35 kB 00:00 2024-09-21T03:31:12.073 INFO:teuthology.orchestra.run.smithi195.stdout:(83/157): python3-babel-2.9.1-2.el9.noarch.rpm 17 MB/s | 6.0 MB 00:00 2024-09-21T03:31:12.099 INFO:teuthology.orchestra.run.smithi195.stdout:(84/157): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.4 MB/s | 250 kB 00:00 2024-09-21T03:31:12.111 INFO:teuthology.orchestra.run.smithi038.stdout:(7/157): ceph-mon-19.3.0-5120.g18432218.el9.x86 14 MB/s | 5.0 MB 00:00 2024-09-21T03:31:12.124 INFO:teuthology.orchestra.run.smithi195.stdout:(85/157): python3-oauthlib-3.1.1-5.el9.noarch.r 4.3 MB/s | 222 kB 00:00 2024-09-21T03:31:12.149 INFO:teuthology.orchestra.run.smithi195.stdout:(86/157): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-09-21T03:31:12.174 INFO:teuthology.orchestra.run.smithi195.stdout:(87/157): python3-prettytable-0.7.2-27.el9.noar 839 kB/s | 42 kB 00:00 2024-09-21T03:31:12.200 INFO:teuthology.orchestra.run.smithi195.stdout:(88/157): python3-protobuf-3.14.0-13.el9.noarch 5.2 MB/s | 269 kB 00:00 2024-09-21T03:31:12.225 INFO:teuthology.orchestra.run.smithi195.stdout:(89/157): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-09-21T03:31:12.251 INFO:teuthology.orchestra.run.smithi195.stdout:(90/157): python3-pyasn1-modules-0.4.8-6.el9.no 5.4 MB/s | 279 kB 00:00 2024-09-21T03:31:12.276 INFO:teuthology.orchestra.run.smithi195.stdout:(91/157): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-09-21T03:31:12.301 INFO:teuthology.orchestra.run.smithi195.stdout:(92/157): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-09-21T03:31:12.385 INFO:teuthology.orchestra.run.smithi195.stdout:(93/157): python3-numpy-1.20.1-5.el9.x86_64.rpm 13 MB/s | 5.1 MB 00:00 2024-09-21T03:31:12.410 INFO:teuthology.orchestra.run.smithi195.stdout:(94/157): python3-toml-0.10.2-6.el9.noarch.rpm 384 kB/s | 42 kB 00:00 2024-09-21T03:31:12.435 INFO:teuthology.orchestra.run.smithi195.stdout:(95/157): python3-wcwidth-0.2.5-8.el9.noarch.rp 881 kB/s | 44 kB 00:00 2024-09-21T03:31:12.460 INFO:teuthology.orchestra.run.smithi195.stdout:(96/157): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-09-21T03:31:12.486 INFO:teuthology.orchestra.run.smithi195.stdout:(97/157): qatlib-service-24.02.0-1.el9.x86_64.r 719 kB/s | 36 kB 00:00 2024-09-21T03:31:12.511 INFO:teuthology.orchestra.run.smithi195.stdout:(98/157): qatzip-libs-1.2.0-1.el9.x86_64.rpm 942 kB/s | 47 kB 00:00 2024-09-21T03:31:12.536 INFO:teuthology.orchestra.run.smithi195.stdout:(99/157): socat-1.7.4.1-6.el9.x86_64.rpm 5.9 MB/s | 304 kB 00:00 2024-09-21T03:31:12.562 INFO:teuthology.orchestra.run.smithi195.stdout:(100/157): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-09-21T03:31:12.587 INFO:teuthology.orchestra.run.smithi195.stdout:(101/157): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.9 MB/s | 96 kB 00:00 2024-09-21T03:31:12.603 INFO:teuthology.orchestra.run.smithi038.stdout:(8/157): ceph-common-19.3.0-5120.g18432218.el9. 11 MB/s | 20 MB 00:01 2024-09-21T03:31:12.612 INFO:teuthology.orchestra.run.smithi195.stdout:(102/157): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-09-21T03:31:12.637 INFO:teuthology.orchestra.run.smithi195.stdout:(103/157): zlib-devel-1.2.11-41.el9.x86_64.rpm 893 kB/s | 45 kB 00:00 2024-09-21T03:31:12.653 INFO:teuthology.orchestra.run.smithi038.stdout:(9/157): ceph-selinux-19.3.0-5120.g18432218.el9 505 kB/s | 25 kB 00:00 2024-09-21T03:31:12.795 INFO:teuthology.orchestra.run.smithi038.stdout:(10/157): ceph-radosgw-19.3.0-5120.g18432218.el 15 MB/s | 10 MB 00:00 2024-09-21T03:31:12.854 INFO:teuthology.orchestra.run.smithi038.stdout:(11/157): cephfs-mirror-19.3.0-5120.g18432218.e 3.8 MB/s | 226 kB 00:00 2024-09-21T03:31:12.913 INFO:teuthology.orchestra.run.smithi195.stdout:(104/157): python3-scipy-1.6.2-8.el9.x86_64.rpm 26 MB/s | 16 MB 00:00 2024-09-21T03:31:12.979 INFO:teuthology.orchestra.run.smithi195.stdout:(105/157): abseil-cpp-20211102.0-3.el9.x86_64.r 9.2 MB/s | 548 kB 00:00 2024-09-21T03:31:13.005 INFO:teuthology.orchestra.run.smithi195.stdout:(106/157): gperftools-libs-2.9.1-3.el9.x86_64.r 12 MB/s | 308 kB 00:00 2024-09-21T03:31:13.038 INFO:teuthology.orchestra.run.smithi195.stdout:(107/157): grpc-data-1.46.7-10.el9.noarch.rpm 766 kB/s | 19 kB 00:00 2024-09-21T03:31:13.046 INFO:teuthology.orchestra.run.smithi038.stdout:(12/157): ceph-osd-19.3.0-5120.g18432218.el9.x8 15 MB/s | 18 MB 00:01 2024-09-21T03:31:13.071 INFO:teuthology.orchestra.run.smithi038.stdout:(13/157): cephfs-shell-19.3.0-5120.g18432218.el 124 kB/s | 27 kB 00:00 2024-09-21T03:31:13.097 INFO:teuthology.orchestra.run.smithi038.stdout:(14/157): libcephfs-devel-19.3.0-5120.g18432218 634 kB/s | 32 kB 00:00 2024-09-21T03:31:13.139 INFO:teuthology.orchestra.run.smithi038.stdout:(15/157): libcephfs2-19.3.0-5120.g18432218.el9. 11 MB/s | 766 kB 00:00 2024-09-21T03:31:13.164 INFO:teuthology.orchestra.run.smithi038.stdout:(16/157): libcephsqlite-19.3.0-5120.g18432218.e 2.3 MB/s | 158 kB 00:00 2024-09-21T03:31:13.172 INFO:teuthology.orchestra.run.smithi195.stdout:(108/157): libarrow-9.0.0-11.el9.x86_64.rpm 33 MB/s | 4.4 MB 00:00 2024-09-21T03:31:13.189 INFO:teuthology.orchestra.run.smithi038.stdout:(17/157): librados-devel-19.3.0-5120.g18432218. 2.5 MB/s | 127 kB 00:00 2024-09-21T03:31:13.197 INFO:teuthology.orchestra.run.smithi195.stdout:(109/157): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-09-21T03:31:13.239 INFO:teuthology.orchestra.run.smithi195.stdout:(110/157): liboath-2.6.7-2.el9.x86_64.rpm 1.1 MB/s | 49 kB 00:00 2024-09-21T03:31:13.264 INFO:teuthology.orchestra.run.smithi195.stdout:(111/157): lua-devel-5.4.4-4.el9.x86_64.rpm 34 kB/s | 22 kB 00:00 2024-09-21T03:31:13.281 INFO:teuthology.orchestra.run.smithi195.stdout:(112/157): libunwind-1.6.2-1.el9.x86_64.rpm 1.6 MB/s | 67 kB 00:00 2024-09-21T03:31:13.306 INFO:teuthology.orchestra.run.smithi038.stdout:(18/157): libradosstriper1-19.3.0-5120.g1843221 3.5 MB/s | 507 kB 00:00 2024-09-21T03:31:13.332 INFO:teuthology.orchestra.run.smithi195.stdout:(113/157): parquet-libs-9.0.0-11.el9.x86_64.rpm 19 MB/s | 839 kB 00:00 2024-09-21T03:31:13.357 INFO:teuthology.orchestra.run.smithi195.stdout:(114/157): luarocks-3.9.2-5.el9.noarch.rpm 1.6 MB/s | 151 kB 00:00 2024-09-21T03:31:13.366 INFO:teuthology.orchestra.run.smithi038.stdout:(19/157): python3-ceph-argparse-19.3.0-5120.g18 774 kB/s | 45 kB 00:00 2024-09-21T03:31:13.382 INFO:teuthology.orchestra.run.smithi195.stdout:(115/157): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-21T03:31:13.407 INFO:teuthology.orchestra.run.smithi195.stdout:(116/157): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-09-21T03:31:13.432 INFO:teuthology.orchestra.run.smithi195.stdout:(117/157): python3-certifi-2023.05.07-4.el9.noa 563 kB/s | 14 kB 00:00 2024-09-21T03:31:13.458 INFO:teuthology.orchestra.run.smithi195.stdout:(118/157): python3-asyncssh-2.13.2-5.el9.noarch 4.2 MB/s | 548 kB 00:00 2024-09-21T03:31:13.482 INFO:teuthology.orchestra.run.smithi038.stdout:(20/157): librgw2-19.3.0-5120.g18432218.el9.x86 18 MB/s | 5.3 MB 00:00 2024-09-21T03:31:13.492 INFO:teuthology.orchestra.run.smithi195.stdout:(119/157): python3-cheroot-8.6.0-4.el9.noarch.r 2.9 MB/s | 172 kB 00:00 2024-09-21T03:31:13.507 INFO:teuthology.orchestra.run.smithi038.stdout:(21/157): python3-ceph-common-19.3.0-5120.g1843 1.0 MB/s | 148 kB 00:00 2024-09-21T03:31:13.517 INFO:teuthology.orchestra.run.smithi195.stdout:(120/157): python3-cherrypy-18.6.1-2.el9.noarch 6.0 MB/s | 358 kB 00:00 2024-09-21T03:31:13.532 INFO:teuthology.orchestra.run.smithi038.stdout:(22/157): python3-cephfs-19.3.0-5120.g18432218. 3.2 MB/s | 161 kB 00:00 2024-09-21T03:31:13.542 INFO:teuthology.orchestra.run.smithi195.stdout:(121/157): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.9 MB/s | 303 kB 00:00 2024-09-21T03:31:13.558 INFO:teuthology.orchestra.run.smithi038.stdout:(23/157): python3-rados-19.3.0-5120.g18432218.e 6.2 MB/s | 321 kB 00:00 2024-09-21T03:31:13.568 INFO:teuthology.orchestra.run.smithi195.stdout:(122/157): python3-google-auth-2.34.0-1.el9.noa 4.3 MB/s | 220 kB 00:00 2024-09-21T03:31:13.583 INFO:teuthology.orchestra.run.smithi038.stdout:(24/157): python3-rbd-19.3.0-5120.g18432218.el9 6.0 MB/s | 307 kB 00:00 2024-09-21T03:31:13.593 INFO:teuthology.orchestra.run.smithi195.stdout:(123/157): python3-grpcio-tools-1.46.7-10.el9.x 5.6 MB/s | 144 kB 00:00 2024-09-21T03:31:13.608 INFO:teuthology.orchestra.run.smithi038.stdout:(25/157): python3-rgw-19.3.0-5120.g18432218.el9 1.9 MB/s | 99 kB 00:00 2024-09-21T03:31:13.610 INFO:teuthology.orchestra.run.smithi195.stdout:(124/157): python3-jaraco-8.2.1-3.el9.noarch.rp 648 kB/s | 11 kB 00:00 2024-09-21T03:31:13.627 INFO:teuthology.orchestra.run.smithi195.stdout:(125/157): python3-jaraco-classes-3.2.1-5.el9.n 1.0 MB/s | 18 kB 00:00 2024-09-21T03:31:13.633 INFO:teuthology.orchestra.run.smithi038.stdout:(26/157): rbd-fuse-19.3.0-5120.g18432218.el9.x8 1.7 MB/s | 86 kB 00:00 2024-09-21T03:31:13.644 INFO:teuthology.orchestra.run.smithi195.stdout:(126/157): python3-jaraco-collections-3.0.0-8.e 1.4 MB/s | 23 kB 00:00 2024-09-21T03:31:13.661 INFO:teuthology.orchestra.run.smithi195.stdout:(127/157): python3-jaraco-functools-3.5.0-2.el9 1.1 MB/s | 19 kB 00:00 2024-09-21T03:31:13.678 INFO:teuthology.orchestra.run.smithi195.stdout:(128/157): python3-jaraco-text-3.2.0-6.el9.noar 1.2 MB/s | 20 kB 00:00 2024-09-21T03:31:13.692 INFO:teuthology.orchestra.run.smithi038.stdout:(27/157): rbd-nbd-19.3.0-5120.g18432218.el9.x86 2.9 MB/s | 170 kB 00:00 2024-09-21T03:31:13.736 INFO:teuthology.orchestra.run.smithi195.stdout:(129/157): python3-grpcio-1.46.7-10.el9.x86_64. 11 MB/s | 2.0 MB 00:00 2024-09-21T03:31:13.759 INFO:teuthology.orchestra.run.smithi038.stdout:(28/157): rbd-mirror-19.3.0-5120.g18432218.el9. 21 MB/s | 3.1 MB 00:00 2024-09-21T03:31:13.770 INFO:teuthology.orchestra.run.smithi195.stdout:(130/157): protobuf-compiler-3.14.0-13.el9.x86_ 762 kB/s | 863 kB 00:01 2024-09-21T03:31:13.776 INFO:teuthology.orchestra.run.smithi038.stdout:(29/157): ceph-grafana-dashboards-19.3.0-5120.g 411 kB/s | 34 kB 00:00 2024-09-21T03:31:13.840 INFO:teuthology.orchestra.run.smithi038.stdout:(30/157): ceph-mgr-cephadm-19.3.0-5120.g1843221 3.0 MB/s | 154 kB 00:00 2024-09-21T03:31:13.841 INFO:teuthology.orchestra.run.smithi195.stdout:(131/157): python3-logutils-0.3.5-21.el9.noarch 806 kB/s | 46 kB 00:00 2024-09-21T03:31:13.841 INFO:teuthology.orchestra.run.smithi195.stdout:(132/157): python3-natsort-7.1.1-5.el9.noarch.r 2.2 MB/s | 58 kB 00:00 2024-09-21T03:31:13.863 INFO:teuthology.orchestra.run.smithi195.stdout:(133/157): python3-pecan-1.4.2-3.el9.noarch.rpm 8.5 MB/s | 272 kB 00:00 2024-09-21T03:31:13.888 INFO:teuthology.orchestra.run.smithi195.stdout:(134/157): python3-more-itertools-8.12.0-2.el9. 672 kB/s | 79 kB 00:00 2024-09-21T03:31:13.913 INFO:teuthology.orchestra.run.smithi195.stdout:(135/157): python3-portend-3.1.0-2.el9.noarch.r 327 kB/s | 16 kB 00:00 2024-09-21T03:31:13.938 INFO:teuthology.orchestra.run.smithi195.stdout:(136/157): python3-pyperclip-1.8.2-2.el9.noarch 1.1 MB/s | 28 kB 00:00 2024-09-21T03:31:13.972 INFO:teuthology.orchestra.run.smithi195.stdout:(137/157): python3-kubernetes-26.1.0-2.el9.noar 3.5 MB/s | 1.0 MB 00:00 2024-09-21T03:31:13.997 INFO:teuthology.orchestra.run.smithi195.stdout:(138/157): python3-pyOpenSSL-21.0.0-1.el9.noarc 826 kB/s | 90 kB 00:00 2024-09-21T03:31:14.022 INFO:teuthology.orchestra.run.smithi195.stdout:(139/157): python3-repoze-lru-0.7-16.el9.noarch 367 kB/s | 31 kB 00:00 2024-09-21T03:31:14.048 INFO:teuthology.orchestra.run.smithi195.stdout:(140/157): python3-rsa-4.9-2.el9.noarch.rpm 1.2 MB/s | 59 kB 00:00 2024-09-21T03:31:14.073 INFO:teuthology.orchestra.run.smithi195.stdout:(141/157): python3-tempora-5.0.0-2.el9.noarch.r 715 kB/s | 36 kB 00:00 2024-09-21T03:31:14.098 INFO:teuthology.orchestra.run.smithi195.stdout:(142/157): python3-typing-extensions-4.12.2-2.e 1.4 MB/s | 74 kB 00:00 2024-09-21T03:31:14.123 INFO:teuthology.orchestra.run.smithi195.stdout:(143/157): python3-routes-2.5.1-5.el9.noarch.rp 1.5 MB/s | 188 kB 00:00 2024-09-21T03:31:14.157 INFO:teuthology.orchestra.run.smithi195.stdout:(144/157): python3-websocket-client-1.2.3-2.el9 1.5 MB/s | 90 kB 00:00 2024-09-21T03:31:14.182 INFO:teuthology.orchestra.run.smithi195.stdout:(145/157): python3-zc-lockfile-2.0-10.el9.noarc 806 kB/s | 20 kB 00:00 2024-09-21T03:31:14.207 INFO:teuthology.orchestra.run.smithi195.stdout:(146/157): python3-webob-1.8.8-2.el9.noarch.rpm 1.7 MB/s | 230 kB 00:00 2024-09-21T03:31:14.241 INFO:teuthology.orchestra.run.smithi195.stdout:(147/157): python3-werkzeug-2.0.3-3.el9.1.noarc 3.6 MB/s | 427 kB 00:00 2024-09-21T03:31:14.266 INFO:teuthology.orchestra.run.smithi195.stdout:(148/157): re2-20211101-20.el9.x86_64.rpm 2.2 MB/s | 191 kB 00:00 2024-09-21T03:31:14.317 INFO:teuthology.orchestra.run.smithi195.stdout:(149/157): thrift-0.15.0-2.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-09-21T03:31:14.343 INFO:teuthology.orchestra.run.smithi038.stdout:(31/157): ceph-mgr-diskprediction-local-19.3.0- 14 MB/s | 7.4 MB 00:00 2024-09-21T03:31:14.410 INFO:teuthology.orchestra.run.smithi038.stdout:(32/157): ceph-mgr-modules-core-19.3.0-5120.g18 4.3 MB/s | 292 kB 00:00 2024-09-21T03:31:14.602 INFO:teuthology.orchestra.run.smithi038.stdout:(33/157): ceph-mgr-dashboard-19.3.0-5120.g18432 17 MB/s | 14 MB 00:00 2024-09-21T03:31:14.627 INFO:teuthology.orchestra.run.smithi038.stdout:(34/157): ceph-mgr-rook-19.3.0-5120.g18432218.e 227 kB/s | 49 kB 00:00 2024-09-21T03:31:14.652 INFO:teuthology.orchestra.run.smithi038.stdout:(35/157): ceph-prometheus-alerts-19.3.0-5120.g1 329 kB/s | 16 kB 00:00 2024-09-21T03:31:14.678 INFO:teuthology.orchestra.run.smithi038.stdout:(36/157): ceph-volume-19.3.0-5120.g18432218.el9 5.5 MB/s | 285 kB 00:00 2024-09-21T03:31:14.720 INFO:teuthology.orchestra.run.smithi038.stdout:(37/157): cephadm-19.3.0-5120.g18432218.el9.noa 11 MB/s | 784 kB 00:00 2024-09-21T03:31:14.745 INFO:teuthology.orchestra.run.smithi038.stdout:(38/157): cephfs-top-19.3.0-5120.g18432218.el9. 275 kB/s | 18 kB 00:00 2024-09-21T03:31:14.850 INFO:teuthology.orchestra.run.smithi195.stdout:(150/157): librbd1-19.3.0-5120.g18432218.el9.x8 5.4 MB/s | 3.2 MB 00:00 2024-09-21T03:31:14.926 INFO:teuthology.orchestra.run.smithi195.stdout:(151/157): librados2-19.3.0-5120.g18432218.el9. 5.0 MB/s | 3.4 MB 00:00 2024-09-21T03:31:15.112 INFO:teuthology.orchestra.run.smithi038.stdout:(39/157): ceph-test-19.3.0-5120.g18432218.el9.x 16 MB/s | 40 MB 00:02 2024-09-21T03:31:15.237 INFO:teuthology.orchestra.run.smithi038.stdout:(40/157): fuse-2.9.9-16.el9.x86_64.rpm 154 kB/s | 80 kB 00:00 2024-09-21T03:31:15.346 INFO:teuthology.orchestra.run.smithi038.stdout:(41/157): libgfortran-11.5.0-2.el9.x86_64.rpm 1.3 MB/s | 801 kB 00:00 2024-09-21T03:31:15.371 INFO:teuthology.orchestra.run.smithi038.stdout:(42/157): libquadmath-11.5.0-2.el9.x86_64.rpm 738 kB/s | 191 kB 00:00 2024-09-21T03:31:15.396 INFO:teuthology.orchestra.run.smithi038.stdout:(43/157): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.6 MB/s | 253 kB 00:00 2024-09-21T03:31:15.422 INFO:teuthology.orchestra.run.smithi038.stdout:(44/157): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-09-21T03:31:15.447 INFO:teuthology.orchestra.run.smithi038.stdout:(45/157): python3-pycparser-2.20-6.el9.noarch.r 2.6 MB/s | 135 kB 00:00 2024-09-21T03:31:15.459 INFO:teuthology.orchestra.run.smithi195.stdout:(152/157): NetworkManager-1.51.0-1.el9.x86_64.r 2.0 MB/s | 2.3 MB 00:01 2024-09-21T03:31:15.472 INFO:teuthology.orchestra.run.smithi038.stdout:(46/157): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-09-21T03:31:15.484 INFO:teuthology.orchestra.run.smithi195.stdout:(153/157): NetworkManager-team-1.51.0-1.el9.x86 73 kB/s | 41 kB 00:00 2024-09-21T03:31:15.506 INFO:teuthology.orchestra.run.smithi038.stdout:(47/157): python3-urllib3-1.26.5-6.el9.noarch.r 3.6 MB/s | 215 kB 00:00 2024-09-21T03:31:15.706 INFO:teuthology.orchestra.run.smithi038.stdout:(48/157): automake-1.16.2-8.el9.noarch.rpm 2.9 MB/s | 689 kB 00:00 2024-09-21T03:31:15.748 INFO:teuthology.orchestra.run.smithi038.stdout:(49/157): python3-cryptography-36.0.1-4.el9.x86 3.1 MB/s | 1.2 MB 00:00 2024-09-21T03:31:15.773 INFO:teuthology.orchestra.run.smithi038.stdout:(50/157): boost-program-options-1.75.0-8.el9.x8 1.6 MB/s | 107 kB 00:00 2024-09-21T03:31:15.832 INFO:teuthology.orchestra.run.smithi038.stdout:(51/157): flex-2.6.4-9.el9.x86_64.rpm 5.3 MB/s | 313 kB 00:00 2024-09-21T03:31:15.843 INFO:teuthology.orchestra.run.smithi195.stdout:(154/157): NetworkManager-libnm-1.51.0-1.el9.x8 1.8 MB/s | 1.8 MB 00:00 2024-09-21T03:31:15.865 INFO:teuthology.orchestra.run.smithi038.stdout:(52/157): bison-3.7.4-5.el9.x86_64.rpm 2.6 MB/s | 943 kB 00:00 2024-09-21T03:31:15.891 INFO:teuthology.orchestra.run.smithi038.stdout:(53/157): elfutils-libelf-devel-0.191-4.el9.x86 152 kB/s | 22 kB 00:00 2024-09-21T03:31:15.916 INFO:teuthology.orchestra.run.smithi038.stdout:(54/157): flexiblas-3.0.4-8.el9.x86_64.rpm 390 kB/s | 33 kB 00:00 2024-09-21T03:31:15.941 INFO:teuthology.orchestra.run.smithi038.stdout:(55/157): flexiblas-openblas-openmp-3.0.4-8.el9 363 kB/s | 18 kB 00:00 2024-09-21T03:31:15.967 INFO:teuthology.orchestra.run.smithi038.stdout:(56/157): libaio-devel-0.3.111-13.el9.x86_64.rp 223 kB/s | 11 kB 00:00 2024-09-21T03:31:16.042 INFO:teuthology.orchestra.run.smithi038.stdout:(57/157): flexiblas-netlib-3.0.4-8.el9.x86_64.r 17 MB/s | 3.0 MB 00:00 2024-09-21T03:31:16.067 INFO:teuthology.orchestra.run.smithi038.stdout:(58/157): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.6 MB/s | 160 kB 00:00 2024-09-21T03:31:16.093 INFO:teuthology.orchestra.run.smithi038.stdout:(59/157): libnbd-1.20.2-2.el9.x86_64.rpm 1.1 MB/s | 171 kB 00:00 2024-09-21T03:31:16.118 INFO:teuthology.orchestra.run.smithi038.stdout:(60/157): librabbitmq-0.11.0-7.el9.x86_64.rpm 601 kB/s | 45 kB 00:00 2024-09-21T03:31:16.151 INFO:teuthology.orchestra.run.smithi038.stdout:(61/157): librdkafka-1.6.1-102.el9.x86_64.rpm 7.7 MB/s | 662 kB 00:00 2024-09-21T03:31:16.168 INFO:teuthology.orchestra.run.smithi038.stdout:(62/157): libuuid-devel-2.37.4-20.el9.x86_64.rp 487 kB/s | 24 kB 00:00 2024-09-21T03:31:16.193 INFO:teuthology.orchestra.run.smithi038.stdout:(63/157): libzstd-devel-1.5.1-2.el9.x86_64.rpm 1.1 MB/s | 47 kB 00:00 2024-09-21T03:31:16.227 INFO:teuthology.orchestra.run.smithi038.stdout:(64/157): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.9 MB/s | 292 kB 00:00 2024-09-21T03:31:16.235 INFO:teuthology.orchestra.run.smithi195.stdout:(155/157): NetworkManager-tui-1.51.0-1.el9.x86_ 321 kB/s | 249 kB 00:00 2024-09-21T03:31:16.252 INFO:teuthology.orchestra.run.smithi038.stdout:(65/157): lua-5.4.4-4.el9.x86_64.rpm 3.1 MB/s | 188 kB 00:00 2024-09-21T03:31:16.277 INFO:teuthology.orchestra.run.smithi038.stdout:(66/157): libtool-2.4.6-46.el9.x86_64.rpm 3.1 MB/s | 581 kB 00:00 2024-09-21T03:31:16.303 INFO:teuthology.orchestra.run.smithi038.stdout:(67/157): openblas-0.3.26-2.el9.x86_64.rpm 509 kB/s | 38 kB 00:00 2024-09-21T03:31:16.420 INFO:teuthology.orchestra.run.smithi038.stdout:(68/157): postgresql-13.16-1.el9.x86_64.rpm 14 MB/s | 1.6 MB 00:00 2024-09-21T03:31:16.427 INFO:teuthology.orchestra.run.smithi195.stdout:(156/157): openssl-3.2.2-6.el9.x86_64.rpm 1.4 MB/s | 1.4 MB 00:00 2024-09-21T03:31:16.503 INFO:teuthology.orchestra.run.smithi038.stdout:(69/157): openblas-openmp-0.3.26-2.el9.x86_64.r 20 MB/s | 4.9 MB 00:00 2024-09-21T03:31:16.537 INFO:teuthology.orchestra.run.smithi038.stdout:(70/157): postgresql-contrib-13.16-1.el9.x86_64 7.6 MB/s | 910 kB 00:00 2024-09-21T03:31:16.604 INFO:teuthology.orchestra.run.smithi038.stdout:(71/157): openssl-devel-3.2.2-6.el9.x86_64.rpm 14 MB/s | 4.4 MB 00:00 2024-09-21T03:31:16.638 INFO:teuthology.orchestra.run.smithi038.stdout:(72/157): postgresql-private-libs-13.16-1.el9.x 1.0 MB/s | 137 kB 00:00 2024-09-21T03:31:16.696 INFO:teuthology.orchestra.run.smithi038.stdout:(73/157): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-09-21T03:31:16.738 INFO:teuthology.orchestra.run.smithi038.stdout:(74/157): protobuf-3.14.0-13.el9.x86_64.rpm 10 MB/s | 1.0 MB 00:00 2024-09-21T03:31:16.827 INFO:teuthology.orchestra.run.smithi038.stdout:(75/157): python3-colorama-0.4.6-3.el9.noarch.r 692 kB/s | 61 kB 00:00 2024-09-21T03:31:16.906 INFO:teuthology.orchestra.run.smithi038.stdout:(76/157): postgresql-server-13.16-1.el9.x86_64. 16 MB/s | 5.8 MB 00:00 2024-09-21T03:31:16.931 INFO:teuthology.orchestra.run.smithi038.stdout:(77/157): python3-devel-3.9.19-8.el9.x86_64.rpm 2.3 MB/s | 245 kB 00:00 2024-09-21T03:31:16.964 INFO:teuthology.orchestra.run.smithi038.stdout:(78/157): python3-jinja2-2.11.3-6.el9.noarch.rp 4.2 MB/s | 248 kB 00:00 2024-09-21T03:31:16.969 INFO:teuthology.orchestra.run.smithi195.stdout:(157/157): openssl-libs-3.2.2-6.el9.x86_64.rpm 2.1 MB/s | 2.4 MB 00:01 2024-09-21T03:31:16.976 INFO:teuthology.orchestra.run.smithi195.stdout:-------------------------------------------------------------------------------- 2024-09-21T03:31:16.976 INFO:teuthology.orchestra.run.smithi195.stdout:Total 15 MB/s | 228 MB 00:15 2024-09-21T03:31:16.989 INFO:teuthology.orchestra.run.smithi038.stdout:(79/157): python3-jsonpatch-1.21-16.el9.noarch. 437 kB/s | 26 kB 00:00 2024-09-21T03:31:17.015 INFO:teuthology.orchestra.run.smithi038.stdout:(80/157): python3-jsonpointer-2.0-4.el9.noarch. 380 kB/s | 19 kB 00:00 2024-09-21T03:31:17.057 INFO:teuthology.orchestra.run.smithi038.stdout:(81/157): python3-mako-1.1.4-6.el9.noarch.rpm 2.5 MB/s | 172 kB 00:00 2024-09-21T03:31:17.082 INFO:teuthology.orchestra.run.smithi038.stdout:(82/157): python3-markupsafe-1.1.1-12.el9.x86_6 519 kB/s | 35 kB 00:00 2024-09-21T03:31:17.157 INFO:teuthology.orchestra.run.smithi038.stdout:(83/157): python3-babel-2.9.1-2.el9.noarch.rpm 13 MB/s | 6.0 MB 00:00 2024-09-21T03:31:17.183 INFO:teuthology.orchestra.run.smithi038.stdout:(84/157): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.4 MB/s | 250 kB 00:00 2024-09-21T03:31:17.216 INFO:teuthology.orchestra.run.smithi038.stdout:(85/157): python3-oauthlib-3.1.1-5.el9.noarch.r 3.7 MB/s | 222 kB 00:00 2024-09-21T03:31:17.233 INFO:teuthology.orchestra.run.smithi038.stdout:(86/157): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-09-21T03:31:17.275 INFO:teuthology.orchestra.run.smithi038.stdout:(87/157): python3-prettytable-0.7.2-27.el9.noar 721 kB/s | 42 kB 00:00 2024-09-21T03:31:17.300 INFO:teuthology.orchestra.run.smithi038.stdout:(88/157): python3-protobuf-3.14.0-13.el9.noarch 3.9 MB/s | 269 kB 00:00 2024-09-21T03:31:17.367 INFO:teuthology.orchestra.run.smithi038.stdout:(89/157): python3-numpy-1.20.1-5.el9.x86_64.rpm 16 MB/s | 5.1 MB 00:00 2024-09-21T03:31:17.392 INFO:teuthology.orchestra.run.smithi038.stdout:(90/157): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.3 MB/s | 159 kB 00:00 2024-09-21T03:31:17.418 INFO:teuthology.orchestra.run.smithi038.stdout:(91/157): python3-pyasn1-modules-0.4.8-6.el9.no 2.3 MB/s | 279 kB 00:00 2024-09-21T03:31:17.443 INFO:teuthology.orchestra.run.smithi038.stdout:(92/157): python3-pytz-2021.1-5.el9.noarch.rpm 676 kB/s | 51 kB 00:00 2024-09-21T03:31:17.468 INFO:teuthology.orchestra.run.smithi038.stdout:(93/157): python3-requests-oauthlib-1.3.0-12.el 711 kB/s | 54 kB 00:00 2024-09-21T03:31:17.527 INFO:teuthology.orchestra.run.smithi038.stdout:(94/157): python3-toml-0.10.2-6.el9.noarch.rpm 499 kB/s | 42 kB 00:00 2024-09-21T03:31:17.552 INFO:teuthology.orchestra.run.smithi038.stdout:(95/157): python3-wcwidth-0.2.5-8.el9.noarch.rp 527 kB/s | 44 kB 00:00 2024-09-21T03:31:17.619 INFO:teuthology.orchestra.run.smithi038.stdout:(96/157): qatlib-24.02.0-1.el9.x86_64.rpm 2.3 MB/s | 221 kB 00:00 2024-09-21T03:31:17.636 INFO:teuthology.orchestra.run.smithi038.stdout:(97/157): qatlib-service-24.02.0-1.el9.x86_64.r 430 kB/s | 36 kB 00:00 2024-09-21T03:31:17.686 INFO:teuthology.orchestra.run.smithi038.stdout:(98/157): qatzip-libs-1.2.0-1.el9.x86_64.rpm 708 kB/s | 47 kB 00:00 2024-09-21T03:31:17.762 INFO:teuthology.orchestra.run.smithi038.stdout:(99/157): uuid-1.6.2-55.el9.x86_64.rpm 771 kB/s | 58 kB 00:00 2024-09-21T03:31:17.820 INFO:teuthology.orchestra.run.smithi038.stdout:(100/157): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.6 MB/s | 96 kB 00:00 2024-09-21T03:31:17.870 INFO:teuthology.orchestra.run.smithi038.stdout:(101/157): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-09-21T03:31:17.937 INFO:teuthology.orchestra.run.smithi038.stdout:(102/157): zlib-devel-1.2.11-41.el9.x86_64.rpm 669 kB/s | 45 kB 00:00 2024-09-21T03:31:18.004 INFO:teuthology.orchestra.run.smithi038.stdout:(103/157): lua-devel-5.4.4-4.el9.x86_64.rpm 335 kB/s | 22 kB 00:00 2024-09-21T03:31:18.172 INFO:teuthology.orchestra.run.smithi195.stdout:Running transaction check 2024-09-21T03:31:18.238 INFO:teuthology.orchestra.run.smithi038.stdout:(104/157): python3-scipy-1.6.2-8.el9.x86_64.rpm 20 MB/s | 16 MB 00:00 2024-09-21T03:31:18.263 INFO:teuthology.orchestra.run.smithi038.stdout:(105/157): socat-1.7.4.1-6.el9.x86_64.rpm 485 kB/s | 304 kB 00:00 2024-09-21T03:31:18.322 INFO:teuthology.orchestra.run.smithi038.stdout:(106/157): protobuf-compiler-3.14.0-13.el9.x86_ 2.7 MB/s | 863 kB 00:00 2024-09-21T03:31:18.324 INFO:teuthology.orchestra.run.smithi195.stdout:Transaction check succeeded. 2024-09-21T03:31:18.324 INFO:teuthology.orchestra.run.smithi195.stdout:Running transaction test 2024-09-21T03:31:18.406 INFO:teuthology.orchestra.run.smithi038.stdout:(107/157): abseil-cpp-20211102.0-3.el9.x86_64.r 3.2 MB/s | 548 kB 00:00 2024-09-21T03:31:18.431 INFO:teuthology.orchestra.run.smithi038.stdout:(108/157): grpc-data-1.46.7-10.el9.noarch.rpm 183 kB/s | 19 kB 00:00 2024-09-21T03:31:18.456 INFO:teuthology.orchestra.run.smithi038.stdout:(109/157): gperftools-libs-2.9.1-3.el9.x86_64.r 1.6 MB/s | 308 kB 00:00 2024-09-21T03:31:18.482 INFO:teuthology.orchestra.run.smithi038.stdout:(110/157): libarrow-doc-9.0.0-11.el9.noarch.rpm 517 kB/s | 26 kB 00:00 2024-09-21T03:31:18.507 INFO:teuthology.orchestra.run.smithi038.stdout:(111/157): liboath-2.6.7-2.el9.x86_64.rpm 968 kB/s | 49 kB 00:00 2024-09-21T03:31:18.532 INFO:teuthology.orchestra.run.smithi038.stdout:(112/157): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-09-21T03:31:18.566 INFO:teuthology.orchestra.run.smithi038.stdout:(113/157): luarocks-3.9.2-5.el9.noarch.rpm 2.5 MB/s | 151 kB 00:00 2024-09-21T03:31:18.624 INFO:teuthology.orchestra.run.smithi038.stdout:(114/157): python3-asyncssh-2.13.2-5.el9.noarch 9.2 MB/s | 548 kB 00:00 2024-09-21T03:31:18.649 INFO:teuthology.orchestra.run.smithi038.stdout:(115/157): parquet-libs-9.0.0-11.el9.x86_64.rpm 7.0 MB/s | 839 kB 00:00 2024-09-21T03:31:18.675 INFO:teuthology.orchestra.run.smithi038.stdout:(116/157): python3-bcrypt-3.2.2-1.el9.x86_64.rp 867 kB/s | 43 kB 00:00 2024-09-21T03:31:18.700 INFO:teuthology.orchestra.run.smithi038.stdout:(117/157): python3-cachetools-4.2.4-1.el9.noarc 640 kB/s | 32 kB 00:00 2024-09-21T03:31:18.726 INFO:teuthology.orchestra.run.smithi038.stdout:(118/157): python3-certifi-2023.05.07-4.el9.noa 280 kB/s | 14 kB 00:00 2024-09-21T03:31:18.751 INFO:teuthology.orchestra.run.smithi038.stdout:(119/157): python3-cheroot-8.6.0-4.el9.noarch.r 3.3 MB/s | 172 kB 00:00 2024-09-21T03:31:18.784 INFO:teuthology.orchestra.run.smithi038.stdout:(120/157): python3-cherrypy-18.6.1-2.el9.noarch 6.0 MB/s | 358 kB 00:00 2024-09-21T03:31:18.810 INFO:teuthology.orchestra.run.smithi038.stdout:(121/157): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.1 MB/s | 303 kB 00:00 2024-09-21T03:31:18.885 INFO:teuthology.orchestra.run.smithi038.stdout:(122/157): libarrow-9.0.0-11.el9.x86_64.rpm 9.2 MB/s | 4.4 MB 00:00 2024-09-21T03:31:18.910 INFO:teuthology.orchestra.run.smithi038.stdout:(123/157): python3-google-auth-2.34.0-1.el9.noa 1.7 MB/s | 220 kB 00:00 2024-09-21T03:31:18.935 INFO:teuthology.orchestra.run.smithi038.stdout:(124/157): python3-grpcio-tools-1.46.7-10.el9.x 2.8 MB/s | 144 kB 00:00 2024-09-21T03:31:18.961 INFO:teuthology.orchestra.run.smithi038.stdout:(125/157): python3-jaraco-8.2.1-3.el9.noarch.rp 214 kB/s | 11 kB 00:00 2024-09-21T03:31:18.986 INFO:teuthology.orchestra.run.smithi038.stdout:(126/157): python3-jaraco-classes-3.2.1-5.el9.n 352 kB/s | 18 kB 00:00 2024-09-21T03:31:19.011 INFO:teuthology.orchestra.run.smithi038.stdout:(127/157): python3-jaraco-collections-3.0.0-8.e 461 kB/s | 23 kB 00:00 2024-09-21T03:31:19.036 INFO:teuthology.orchestra.run.smithi038.stdout:(128/157): python3-jaraco-functools-3.5.0-2.el9 387 kB/s | 19 kB 00:00 2024-09-21T03:31:19.062 INFO:teuthology.orchestra.run.smithi038.stdout:(129/157): python3-jaraco-text-3.2.0-6.el9.noar 393 kB/s | 20 kB 00:00 2024-09-21T03:31:19.087 INFO:teuthology.orchestra.run.smithi038.stdout:(130/157): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-21T03:31:19.112 INFO:teuthology.orchestra.run.smithi038.stdout:(131/157): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-21T03:31:19.138 INFO:teuthology.orchestra.run.smithi038.stdout:(132/157): python3-natsort-7.1.1-5.el9.noarch.r 2.2 MB/s | 58 kB 00:00 2024-09-21T03:31:19.180 INFO:teuthology.orchestra.run.smithi038.stdout:(133/157): python3-kubernetes-26.1.0-2.el9.noar 7.1 MB/s | 1.0 MB 00:00 2024-09-21T03:31:19.205 INFO:teuthology.orchestra.run.smithi038.stdout:(134/157): python3-portend-3.1.0-2.el9.noarch.r 665 kB/s | 16 kB 00:00 2024-09-21T03:31:19.230 INFO:teuthology.orchestra.run.smithi038.stdout:(135/157): python3-pyOpenSSL-21.0.0-1.el9.noarc 3.5 MB/s | 90 kB 00:00 2024-09-21T03:31:19.256 INFO:teuthology.orchestra.run.smithi038.stdout:(136/157): python3-pecan-1.4.2-3.el9.noarch.rpm 2.3 MB/s | 272 kB 00:00 2024-09-21T03:31:19.281 INFO:teuthology.orchestra.run.smithi038.stdout:(137/157): python3-pyperclip-1.8.2-2.el9.noarch 562 kB/s | 28 kB 00:00 2024-09-21T03:31:19.306 INFO:teuthology.orchestra.run.smithi038.stdout:(138/157): python3-repoze-lru-0.7-16.el9.noarch 611 kB/s | 31 kB 00:00 2024-09-21T03:31:19.331 INFO:teuthology.orchestra.run.smithi038.stdout:(139/157): python3-routes-2.5.1-5.el9.noarch.rp 3.7 MB/s | 188 kB 00:00 2024-09-21T03:31:19.357 INFO:teuthology.orchestra.run.smithi038.stdout:(140/157): python3-rsa-4.9-2.el9.noarch.rpm 1.2 MB/s | 59 kB 00:00 2024-09-21T03:31:19.382 INFO:teuthology.orchestra.run.smithi038.stdout:(141/157): python3-tempora-5.0.0-2.el9.noarch.r 711 kB/s | 36 kB 00:00 2024-09-21T03:31:19.432 INFO:teuthology.orchestra.run.smithi038.stdout:(142/157): python3-grpcio-1.46.7-10.el9.x86_64. 3.3 MB/s | 2.0 MB 00:00 2024-09-21T03:31:19.458 INFO:teuthology.orchestra.run.smithi038.stdout:(143/157): python3-typing-extensions-4.12.2-2.e 740 kB/s | 74 kB 00:00 2024-09-21T03:31:19.483 INFO:teuthology.orchestra.run.smithi038.stdout:(144/157): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-09-21T03:31:19.508 INFO:teuthology.orchestra.run.smithi038.stdout:(145/157): python3-webob-1.8.8-2.el9.noarch.rpm 1.8 MB/s | 230 kB 00:00 2024-09-21T03:31:19.525 INFO:teuthology.orchestra.run.smithi038.stdout:(146/157): python3-zc-lockfile-2.0-10.el9.noarc 478 kB/s | 20 kB 00:00 2024-09-21T03:31:19.559 INFO:teuthology.orchestra.run.smithi038.stdout:(147/157): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-21T03:31:19.584 INFO:teuthology.orchestra.run.smithi038.stdout:(148/157): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2024-09-21T03:31:19.634 INFO:teuthology.orchestra.run.smithi038.stdout:(149/157): thrift-0.15.0-2.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-09-21T03:31:19.851 INFO:teuthology.orchestra.run.smithi038.stdout:(150/157): librbd1-19.3.0-5120.g18432218.el9.x8 12 MB/s | 3.2 MB 00:00 2024-09-21T03:31:19.918 INFO:teuthology.orchestra.run.smithi038.stdout:(151/157): librados2-19.3.0-5120.g18432218.el9. 9.4 MB/s | 3.4 MB 00:00 2024-09-21T03:31:20.035 INFO:teuthology.orchestra.run.smithi038.stdout:(152/157): NetworkManager-team-1.51.0-1.el9.x86 351 kB/s | 41 kB 00:00 2024-09-21T03:31:20.193 INFO:teuthology.orchestra.run.smithi038.stdout:(153/157): NetworkManager-1.51.0-1.el9.x86_64.r 4.2 MB/s | 2.3 MB 00:00 2024-09-21T03:31:20.219 INFO:teuthology.orchestra.run.smithi038.stdout:(154/157): NetworkManager-tui-1.51.0-1.el9.x86_ 1.3 MB/s | 249 kB 00:00 2024-09-21T03:31:20.377 INFO:teuthology.orchestra.run.smithi038.stdout:(155/157): NetworkManager-libnm-1.51.0-1.el9.x8 3.4 MB/s | 1.8 MB 00:00 2024-09-21T03:31:20.494 INFO:teuthology.orchestra.run.smithi038.stdout:(156/157): openssl-3.2.2-6.el9.x86_64.rpm 4.5 MB/s | 1.4 MB 00:00 2024-09-21T03:31:20.543 INFO:teuthology.orchestra.run.smithi195.stdout:Transaction test succeeded. 2024-09-21T03:31:20.543 INFO:teuthology.orchestra.run.smithi195.stdout:Running transaction 2024-09-21T03:31:20.653 INFO:teuthology.orchestra.run.smithi038.stdout:(157/157): openssl-libs-3.2.2-6.el9.x86_64.rpm 5.5 MB/s | 2.4 MB 00:00 2024-09-21T03:31:20.660 INFO:teuthology.orchestra.run.smithi038.stdout:-------------------------------------------------------------------------------- 2024-09-21T03:31:20.660 INFO:teuthology.orchestra.run.smithi038.stdout:Total 21 MB/s | 228 MB 00:10 2024-09-21T03:31:21.942 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction check 2024-09-21T03:31:22.114 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction check succeeded. 2024-09-21T03:31:22.115 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction test 2024-09-21T03:31:24.493 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction test succeeded. 2024-09-21T03:31:24.493 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction 2024-09-21T03:31:27.035 INFO:teuthology.orchestra.run.smithi195.stdout: Preparing : 1/1 2024-09-21T03:31:27.173 INFO:teuthology.orchestra.run.smithi195.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/165 2024-09-21T03:31:27.225 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/165 2024-09-21T03:31:27.281 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/165 2024-09-21T03:31:27.314 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/165 2024-09-21T03:31:27.340 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/165 2024-09-21T03:31:27.393 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/165 2024-09-21T03:31:27.741 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/165 2024-09-21T03:31:27.860 INFO:teuthology.orchestra.run.smithi195.stdout: Upgrading : librados2-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T03:31:27.986 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: librados2-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T03:31:28.010 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T03:31:28.102 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T03:31:28.140 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 10/165 2024-09-21T03:31:28.170 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/165 2024-09-21T03:31:28.349 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 12/165 2024-09-21T03:31:28.404 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : postgresql-13.16-1.el9.x86_64 13/165 2024-09-21T03:31:28.432 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T03:31:28.538 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T03:31:28.566 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 15/165 2024-09-21T03:31:28.629 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 15/165 2024-09-21T03:31:28.651 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-21T03:31:28.680 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-21T03:31:28.831 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-21T03:31:29.023 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-21T03:31:29.153 INFO:teuthology.orchestra.run.smithi195.stdout: Upgrading : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 18/165 2024-09-21T03:31:29.263 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T03:31:29.326 INFO:teuthology.orchestra.run.smithi195.stdout: Upgrading : NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T03:31:29.563 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T03:31:29.677 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : re2-1:20211101-20.el9.x86_64 20/165 2024-09-21T03:31:29.708 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/165 2024-09-21T03:31:29.748 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/165 2024-09-21T03:31:29.830 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/165 2024-09-21T03:31:29.874 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/165 2024-09-21T03:31:29.906 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/165 2024-09-21T03:31:29.975 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/165 2024-09-21T03:31:30.007 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/165 2024-09-21T03:31:30.035 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/165 2024-09-21T03:31:30.104 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/165 2024-09-21T03:31:30.157 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/165 2024-09-21T03:31:30.205 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/165 2024-09-21T03:31:30.253 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/165 2024-09-21T03:31:30.330 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/165 2024-09-21T03:31:30.359 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-ceph-common-2:19.3.0-5120.g18432218.el9. 34/165 2024-09-21T03:31:30.403 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-ceph-argparse-2:19.3.0-5120.g18432218.el 35/165 2024-09-21T03:31:30.431 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_6 36/165 2024-09-21T03:31:30.502 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/165 2024-09-21T03:31:30.546 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/165 2024-09-21T03:31:30.620 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/165 2024-09-21T03:31:30.761 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/165 2024-09-21T03:31:30.815 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/165 2024-09-21T03:31:31.069 INFO:teuthology.orchestra.run.smithi038.stdout: Preparing : 1/1 2024-09-21T03:31:31.157 INFO:teuthology.orchestra.run.smithi038.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/165 2024-09-21T03:31:31.303 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/165 2024-09-21T03:31:31.337 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/165 2024-09-21T03:31:31.419 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/165 2024-09-21T03:31:31.457 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/165 2024-09-21T03:31:31.496 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/165 2024-09-21T03:31:31.506 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/165 2024-09-21T03:31:31.543 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/165 2024-09-21T03:31:31.546 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : librados-devel-2:19.3.0-5120.g18432218.el9.x86_6 46/165 2024-09-21T03:31:31.575 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/165 2024-09-21T03:31:31.588 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/165 2024-09-21T03:31:31.641 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/165 2024-09-21T03:31:31.750 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/165 2024-09-21T03:31:31.807 INFO:teuthology.orchestra.run.smithi195.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/165 2024-09-21T03:31:31.899 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/165 2024-09-21T03:31:31.963 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 51/165 2024-09-21T03:31:32.044 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/165 2024-09-21T03:31:32.047 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/165 2024-09-21T03:31:32.076 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 53/165 2024-09-21T03:31:32.112 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 54/165 2024-09-21T03:31:32.125 INFO:teuthology.orchestra.run.smithi038.stdout: Upgrading : librados2-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T03:31:32.159 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/165 2024-09-21T03:31:32.194 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/165 2024-09-21T03:31:32.246 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/165 2024-09-21T03:31:32.255 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: librados2-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T03:31:32.285 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T03:31:32.288 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/165 2024-09-21T03:31:32.378 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T03:31:32.407 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 10/165 2024-09-21T03:31:32.425 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/165 2024-09-21T03:31:32.436 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/165 2024-09-21T03:31:32.465 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/165 2024-09-21T03:31:32.506 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 61/165 2024-09-21T03:31:32.610 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/165 2024-09-21T03:31:32.618 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 12/165 2024-09-21T03:31:32.649 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 63/165 2024-09-21T03:31:32.671 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : postgresql-13.16-1.el9.x86_64 13/165 2024-09-21T03:31:32.699 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T03:31:32.793 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/165 2024-09-21T03:31:32.825 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T03:31:32.858 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 15/165 2024-09-21T03:31:32.912 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 15/165 2024-09-21T03:31:32.942 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-21T03:31:32.964 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-21T03:31:32.998 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 65/165 2024-09-21T03:31:33.039 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/165 2024-09-21T03:31:33.154 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-21T03:31:33.156 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/165 2024-09-21T03:31:33.285 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/165 2024-09-21T03:31:33.307 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-21T03:31:33.321 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/165 2024-09-21T03:31:33.349 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/165 2024-09-21T03:31:33.383 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/165 2024-09-21T03:31:33.415 INFO:teuthology.orchestra.run.smithi038.stdout: Upgrading : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 18/165 2024-09-21T03:31:33.422 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/165 2024-09-21T03:31:33.454 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/165 2024-09-21T03:31:33.484 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/165 2024-09-21T03:31:33.522 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/165 2024-09-21T03:31:33.535 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T03:31:33.555 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/165 2024-09-21T03:31:33.597 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/165 2024-09-21T03:31:33.617 INFO:teuthology.orchestra.run.smithi038.stdout: Upgrading : NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T03:31:33.631 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/165 2024-09-21T03:31:33.718 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/165 2024-09-21T03:31:33.762 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/165 2024-09-21T03:31:33.801 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/165 2024-09-21T03:31:33.851 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T03:31:33.904 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/165 2024-09-21T03:31:33.953 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/165 2024-09-21T03:31:33.986 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : re2-1:20211101-20.el9.x86_64 20/165 2024-09-21T03:31:34.021 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/165 2024-09-21T03:31:34.057 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/165 2024-09-21T03:31:34.134 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/165 2024-09-21T03:31:34.184 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/165 2024-09-21T03:31:34.223 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/165 2024-09-21T03:31:34.266 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/165 2024-09-21T03:31:34.299 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/165 2024-09-21T03:31:34.335 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/165 2024-09-21T03:31:34.407 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/165 2024-09-21T03:31:34.459 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/165 2024-09-21T03:31:34.488 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/165 2024-09-21T03:31:34.532 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/165 2024-09-21T03:31:34.615 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/165 2024-09-21T03:31:34.650 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-ceph-common-2:19.3.0-5120.g18432218.el9. 34/165 2024-09-21T03:31:34.676 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/165 2024-09-21T03:31:34.694 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-ceph-argparse-2:19.3.0-5120.g18432218.el 35/165 2024-09-21T03:31:34.731 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_6 36/165 2024-09-21T03:31:34.733 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/165 2024-09-21T03:31:34.767 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/165 2024-09-21T03:31:34.814 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/165 2024-09-21T03:31:34.863 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/165 2024-09-21T03:31:34.875 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/165 2024-09-21T03:31:34.930 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 88/165 2024-09-21T03:31:34.941 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/165 2024-09-21T03:31:34.976 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 89/165 2024-09-21T03:31:35.062 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 90/165 2024-09-21T03:31:35.088 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/165 2024-09-21T03:31:35.095 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 91/165 2024-09-21T03:31:35.150 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/165 2024-09-21T03:31:35.279 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : openblas-0.3.26-2.el9.x86_64 92/165 2024-09-21T03:31:35.300 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 93/165 2024-09-21T03:31:35.386 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 94/165 2024-09-21T03:31:35.719 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/165 2024-09-21T03:31:35.806 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/165 2024-09-21T03:31:35.839 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/165 2024-09-21T03:31:35.882 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/165 2024-09-21T03:31:35.920 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : librados-devel-2:19.3.0-5120.g18432218.el9.x86_6 46/165 2024-09-21T03:31:35.972 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/165 2024-09-21T03:31:36.045 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 95/165 2024-09-21T03:31:36.138 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/165 2024-09-21T03:31:36.192 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 96/165 2024-09-21T03:31:36.199 INFO:teuthology.orchestra.run.smithi038.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/165 2024-09-21T03:31:36.295 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/165 2024-09-21T03:31:36.356 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 51/165 2024-09-21T03:31:36.430 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/165 2024-09-21T03:31:36.475 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 53/165 2024-09-21T03:31:36.511 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 54/165 2024-09-21T03:31:36.560 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/165 2024-09-21T03:31:36.594 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/165 2024-09-21T03:31:36.638 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/165 2024-09-21T03:31:36.680 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/165 2024-09-21T03:31:36.825 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/165 2024-09-21T03:31:36.956 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/165 2024-09-21T03:31:36.998 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 61/165 2024-09-21T03:31:37.115 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/165 2024-09-21T03:31:37.156 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 63/165 2024-09-21T03:31:37.220 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/165 2024-09-21T03:31:37.254 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 65/165 2024-09-21T03:31:37.279 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/165 2024-09-21T03:31:37.378 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/165 2024-09-21T03:31:37.449 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 97/165 2024-09-21T03:31:37.512 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/165 2024-09-21T03:31:37.539 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 98/165 2024-09-21T03:31:37.553 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/165 2024-09-21T03:31:37.567 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 99/165 2024-09-21T03:31:37.589 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/165 2024-09-21T03:31:37.600 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 100/165 2024-09-21T03:31:37.624 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/165 2024-09-21T03:31:37.663 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/165 2024-09-21T03:31:37.703 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/165 2024-09-21T03:31:37.732 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/165 2024-09-21T03:31:37.767 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/165 2024-09-21T03:31:37.800 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/165 2024-09-21T03:31:37.842 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/165 2024-09-21T03:31:37.880 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/165 2024-09-21T03:31:37.928 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 101/165 2024-09-21T03:31:37.971 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/165 2024-09-21T03:31:37.972 INFO:teuthology.orchestra.run.smithi195.stdout: Upgrading : librbd1-2:19.3.0-5120.g18432218.el9.x86_64 102/165 2024-09-21T03:31:38.036 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/165 2024-09-21T03:31:38.064 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: librbd1-2:19.3.0-5120.g18432218.el9.x86_64 102/165 2024-09-21T03:31:38.075 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/165 2024-09-21T03:31:38.093 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 103/165 2024-09-21T03:31:38.129 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 104/165 2024-09-21T03:31:38.183 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/165 2024-09-21T03:31:38.244 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/165 2024-09-21T03:31:38.627 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 105/165 2024-09-21T03:31:38.658 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : librgw2-2:19.3.0-5120.g18432218.el9.x86_64 106/165 2024-09-21T03:31:38.728 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: librgw2-2:19.3.0-5120.g18432218.el9.x86_64 106/165 2024-09-21T03:31:38.751 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 107/165 2024-09-21T03:31:39.023 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/165 2024-09-21T03:31:39.073 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/165 2024-09-21T03:31:39.115 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/165 2024-09-21T03:31:39.222 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/165 2024-09-21T03:31:39.271 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 88/165 2024-09-21T03:31:39.318 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 89/165 2024-09-21T03:31:39.408 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 90/165 2024-09-21T03:31:39.459 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 91/165 2024-09-21T03:31:39.650 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : openblas-0.3.26-2.el9.x86_64 92/165 2024-09-21T03:31:39.682 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 93/165 2024-09-21T03:31:39.761 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 94/165 2024-09-21T03:31:40.458 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 95/165 2024-09-21T03:31:40.586 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 96/165 2024-09-21T03:31:41.401 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T03:31:41.447 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T03:31:41.940 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 97/165 2024-09-21T03:31:42.027 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T03:31:42.039 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 98/165 2024-09-21T03:31:42.067 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 99/165 2024-09-21T03:31:42.067 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 109/165 2024-09-21T03:31:42.100 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 100/165 2024-09-21T03:31:42.158 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 109/165 2024-09-21T03:31:42.158 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-21T03:31:42.158 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-21T03:31:42.158 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-21T03:31:42.211 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 110/165 2024-09-21T03:31:42.445 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 101/165 2024-09-21T03:31:42.504 INFO:teuthology.orchestra.run.smithi038.stdout: Upgrading : librbd1-2:19.3.0-5120.g18432218.el9.x86_64 102/165 2024-09-21T03:31:42.588 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: librbd1-2:19.3.0-5120.g18432218.el9.x86_64 102/165 2024-09-21T03:31:42.625 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 103/165 2024-09-21T03:31:42.662 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 104/165 2024-09-21T03:31:43.196 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 105/165 2024-09-21T03:31:43.223 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : librgw2-2:19.3.0-5120.g18432218.el9.x86_64 106/165 2024-09-21T03:31:43.294 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: librgw2-2:19.3.0-5120.g18432218.el9.x86_64 106/165 2024-09-21T03:31:43.317 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 107/165 2024-09-21T03:31:46.196 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T03:31:46.230 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T03:31:46.841 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T03:31:46.883 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 109/165 2024-09-21T03:31:46.984 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 109/165 2024-09-21T03:31:46.984 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-21T03:31:46.984 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-21T03:31:46.984 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:31:47.035 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 110/165 2024-09-21T03:31:54.189 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 110/165 2024-09-21T03:31:54.189 INFO:teuthology.orchestra.run.smithi195.stdout:skipping the directory /sys 2024-09-21T03:31:54.189 INFO:teuthology.orchestra.run.smithi195.stdout:skipping the directory /proc 2024-09-21T03:31:54.189 INFO:teuthology.orchestra.run.smithi195.stdout:skipping the directory /mnt 2024-09-21T03:31:54.189 INFO:teuthology.orchestra.run.smithi195.stdout:skipping the directory /var/tmp 2024-09-21T03:31:54.189 INFO:teuthology.orchestra.run.smithi195.stdout:skipping the directory /home 2024-09-21T03:31:54.189 INFO:teuthology.orchestra.run.smithi195.stdout:skipping the directory /root 2024-09-21T03:31:54.189 INFO:teuthology.orchestra.run.smithi195.stdout:skipping the directory /tmp 2024-09-21T03:31:54.189 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-21T03:31:54.366 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 111/165 2024-09-21T03:31:54.382 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 111/165 2024-09-21T03:31:54.382 INFO:teuthology.orchestra.run.smithi195.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:31:54.382 INFO:teuthology.orchestra.run.smithi195.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-21T03:31:54.382 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-21T03:31:55.986 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 112/165 2024-09-21T03:31:56.012 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 112/165 2024-09-21T03:31:56.012 INFO:teuthology.orchestra.run.smithi195.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:31:56.012 INFO:teuthology.orchestra.run.smithi195.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-21T03:31:56.012 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T03:31:56.012 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T03:31:56.012 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-21T03:31:56.245 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 113/165 2024-09-21T03:31:56.268 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 113/165 2024-09-21T03:31:56.268 INFO:teuthology.orchestra.run.smithi195.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:31:56.268 INFO:teuthology.orchestra.run.smithi195.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-21T03:31:56.268 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T03:31:56.268 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T03:31:56.268 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-21T03:31:56.746 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 114/165 2024-09-21T03:31:56.768 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 114/165 2024-09-21T03:31:56.769 INFO:teuthology.orchestra.run.smithi195.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:31:56.769 INFO:teuthology.orchestra.run.smithi195.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-21T03:31:56.769 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T03:31:56.769 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T03:31:56.769 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-21T03:31:56.875 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : automake-1.16.2-8.el9.noarch 115/165 2024-09-21T03:31:56.924 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-ply-3.11-14.el9.noarch 116/165 2024-09-21T03:31:56.979 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 117/165 2024-09-21T03:31:57.174 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 118/165 2024-09-21T03:31:57.221 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 119/165 2024-09-21T03:31:57.299 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 120/165 2024-09-21T03:31:57.416 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 121/165 2024-09-21T03:31:57.568 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 122/165 2024-09-21T03:31:57.613 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 123/165 2024-09-21T03:31:57.645 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 124/165 2024-09-21T03:31:57.673 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : fuse-2.9.9-16.el9.x86_64 125/165 2024-09-21T03:31:58.133 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: cephadm-2:19.3.0-5120.g18432218.el9.noarch 126/165 2024-09-21T03:31:58.163 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : cephadm-2:19.3.0-5120.g18432218.el9.noarch 126/165 2024-09-21T03:31:58.195 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-5120.g18432218.e 127/165 2024-09-21T03:31:58.261 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-5120.g18432218. 128/165 2024-09-21T03:31:58.286 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 129/165 2024-09-21T03:31:59.538 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 110/165 2024-09-21T03:31:59.538 INFO:teuthology.orchestra.run.smithi038.stdout:skipping the directory /sys 2024-09-21T03:31:59.538 INFO:teuthology.orchestra.run.smithi038.stdout:skipping the directory /proc 2024-09-21T03:31:59.538 INFO:teuthology.orchestra.run.smithi038.stdout:skipping the directory /mnt 2024-09-21T03:31:59.538 INFO:teuthology.orchestra.run.smithi038.stdout:skipping the directory /var/tmp 2024-09-21T03:31:59.539 INFO:teuthology.orchestra.run.smithi038.stdout:skipping the directory /home 2024-09-21T03:31:59.539 INFO:teuthology.orchestra.run.smithi038.stdout:skipping the directory /root 2024-09-21T03:31:59.539 INFO:teuthology.orchestra.run.smithi038.stdout:skipping the directory /tmp 2024-09-21T03:31:59.539 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:31:59.729 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 111/165 2024-09-21T03:31:59.746 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 111/165 2024-09-21T03:31:59.746 INFO:teuthology.orchestra.run.smithi038.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:31:59.746 INFO:teuthology.orchestra.run.smithi038.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-21T03:31:59.746 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:32:01.391 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 112/165 2024-09-21T03:32:01.418 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 112/165 2024-09-21T03:32:01.419 INFO:teuthology.orchestra.run.smithi038.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:32:01.419 INFO:teuthology.orchestra.run.smithi038.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-21T03:32:01.419 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T03:32:01.419 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T03:32:01.419 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:32:01.484 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 129/165 2024-09-21T03:32:01.543 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 130/165 2024-09-21T03:32:01.649 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 113/165 2024-09-21T03:32:01.674 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 113/165 2024-09-21T03:32:01.674 INFO:teuthology.orchestra.run.smithi038.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:32:01.675 INFO:teuthology.orchestra.run.smithi038.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-21T03:32:01.675 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T03:32:01.675 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T03:32:01.675 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:32:02.175 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 114/165 2024-09-21T03:32:02.200 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 114/165 2024-09-21T03:32:02.200 INFO:teuthology.orchestra.run.smithi038.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:32:02.200 INFO:teuthology.orchestra.run.smithi038.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-21T03:32:02.201 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T03:32:02.201 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T03:32:02.201 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:32:02.315 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : automake-1.16.2-8.el9.noarch 115/165 2024-09-21T03:32:02.373 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-ply-3.11-14.el9.noarch 116/165 2024-09-21T03:32:02.445 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 117/165 2024-09-21T03:32:02.504 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 130/165 2024-09-21T03:32:02.534 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 131/165 2024-09-21T03:32:02.649 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 131/165 2024-09-21T03:32:02.650 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 118/165 2024-09-21T03:32:02.702 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 119/165 2024-09-21T03:32:02.783 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 120/165 2024-09-21T03:32:02.819 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el 132/165 2024-09-21T03:32:02.862 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 133/165 2024-09-21T03:32:02.881 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 121/165 2024-09-21T03:32:02.885 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 133/165 2024-09-21T03:32:02.885 INFO:teuthology.orchestra.run.smithi195.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:32:02.885 INFO:teuthology.orchestra.run.smithi195.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-21T03:32:02.885 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T03:32:02.885 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T03:32:02.885 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-21T03:32:02.937 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 134/165 2024-09-21T03:32:02.949 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 134/165 2024-09-21T03:32:03.041 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 122/165 2024-09-21T03:32:03.048 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-2:19.3.0-5120.g18432218.el9.x86_64 135/165 2024-09-21T03:32:03.085 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 123/165 2024-09-21T03:32:03.103 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 136/165 2024-09-21T03:32:03.125 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 124/165 2024-09-21T03:32:03.153 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : fuse-2.9.9-16.el9.x86_64 125/165 2024-09-21T03:32:03.605 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: cephadm-2:19.3.0-5120.g18432218.el9.noarch 126/165 2024-09-21T03:32:03.635 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : cephadm-2:19.3.0-5120.g18432218.el9.noarch 126/165 2024-09-21T03:32:03.667 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-5120.g18432218.e 127/165 2024-09-21T03:32:03.737 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-5120.g18432218. 128/165 2024-09-21T03:32:03.774 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 129/165 2024-09-21T03:32:04.069 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : libtool-2.4.6-46.el9.x86_64 137/165 2024-09-21T03:32:04.098 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 138/165 2024-09-21T03:32:04.121 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 138/165 2024-09-21T03:32:04.122 INFO:teuthology.orchestra.run.smithi195.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:32:04.122 INFO:teuthology.orchestra.run.smithi195.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-21T03:32:04.122 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T03:32:04.122 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T03:32:04.122 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-21T03:32:04.166 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-5120.g18432 139/165 2024-09-21T03:32:04.188 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-5120.g18432 139/165 2024-09-21T03:32:04.188 INFO:teuthology.orchestra.run.smithi195.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:32:04.189 INFO:teuthology.orchestra.run.smithi195.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-21T03:32:04.189 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-21T03:32:04.241 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 140/165 2024-09-21T03:32:04.264 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 140/165 2024-09-21T03:32:04.264 INFO:teuthology.orchestra.run.smithi195.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:32:04.264 INFO:teuthology.orchestra.run.smithi195.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-21T03:32:04.264 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-21T03:32:04.264 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-21T03:32:04.264 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-21T03:32:04.592 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 141/165 2024-09-21T03:32:04.615 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 141/165 2024-09-21T03:32:04.615 INFO:teuthology.orchestra.run.smithi195.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:32:04.615 INFO:teuthology.orchestra.run.smithi195.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-21T03:32:04.615 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T03:32:04.615 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T03:32:04.615 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-21T03:32:07.191 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 129/165 2024-09-21T03:32:07.256 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 130/165 2024-09-21T03:32:08.269 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 130/165 2024-09-21T03:32:08.297 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 131/165 2024-09-21T03:32:08.421 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 131/165 2024-09-21T03:32:08.589 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el 132/165 2024-09-21T03:32:08.616 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 133/165 2024-09-21T03:32:08.639 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 142/165 2024-09-21T03:32:08.642 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 133/165 2024-09-21T03:32:08.642 INFO:teuthology.orchestra.run.smithi038.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:32:08.642 INFO:teuthology.orchestra.run.smithi038.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-21T03:32:08.642 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T03:32:08.642 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T03:32:08.642 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:32:08.687 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 143/165 2024-09-21T03:32:08.692 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 134/165 2024-09-21T03:32:08.704 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 134/165 2024-09-21T03:32:08.729 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 144/165 2024-09-21T03:32:08.760 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 145/165 2024-09-21T03:32:08.788 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 146/165 2024-09-21T03:32:08.806 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-2:19.3.0-5120.g18432218.el9.x86_64 135/165 2024-09-21T03:32:08.877 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 136/165 2024-09-21T03:32:09.007 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 147/165 2024-09-21T03:32:09.060 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 148/165 2024-09-21T03:32:09.090 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_ 149/165 2024-09-21T03:32:09.119 INFO:teuthology.orchestra.run.smithi195.stdout: Upgrading : NetworkManager-team-1:1.51.0-1.el9.x86_64 150/165 2024-09-21T03:32:09.166 INFO:teuthology.orchestra.run.smithi195.stdout: Upgrading : NetworkManager-tui-1:1.51.0-1.el9.x86_64 151/165 2024-09-21T03:32:09.808 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T03:32:09.855 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T03:32:09.874 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T03:32:09.893 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : libtool-2.4.6-46.el9.x86_64 137/165 2024-09-21T03:32:09.928 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 138/165 2024-09-21T03:32:09.952 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 138/165 2024-09-21T03:32:09.953 INFO:teuthology.orchestra.run.smithi038.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:32:09.953 INFO:teuthology.orchestra.run.smithi038.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-21T03:32:09.953 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T03:32:09.953 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T03:32:09.953 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:32:09.995 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-5120.g18432 139/165 2024-09-21T03:32:10.019 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-5120.g18432 139/165 2024-09-21T03:32:10.019 INFO:teuthology.orchestra.run.smithi038.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:32:10.020 INFO:teuthology.orchestra.run.smithi038.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-21T03:32:10.020 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:32:10.070 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 140/165 2024-09-21T03:32:10.095 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 140/165 2024-09-21T03:32:10.095 INFO:teuthology.orchestra.run.smithi038.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:32:10.095 INFO:teuthology.orchestra.run.smithi038.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-21T03:32:10.095 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-21T03:32:10.096 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-21T03:32:10.096 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:32:10.430 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 141/165 2024-09-21T03:32:10.454 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 141/165 2024-09-21T03:32:10.454 INFO:teuthology.orchestra.run.smithi038.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:32:10.455 INFO:teuthology.orchestra.run.smithi038.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-21T03:32:10.455 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T03:32:10.455 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T03:32:10.455 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:32:10.715 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 153/165 2024-09-21T03:32:10.801 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 154/165 2024-09-21T03:32:10.848 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 155/165 2024-09-21T03:32:10.947 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : flex-2.6.4-9.el9.x86_64 156/165 2024-09-21T03:32:10.980 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : bison-3.7.4-5.el9.x86_64 157/165 2024-09-21T03:32:11.005 INFO:teuthology.orchestra.run.smithi195.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 158/165 2024-09-21T03:32:11.031 INFO:teuthology.orchestra.run.smithi195.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 159/165 2024-09-21T03:32:11.031 INFO:teuthology.orchestra.run.smithi195.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-21T03:32:11.107 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-21T03:32:11.107 INFO:teuthology.orchestra.run.smithi195.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-21T03:32:11.174 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-21T03:32:11.199 INFO:teuthology.orchestra.run.smithi195.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 162/165 2024-09-21T03:32:11.204 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T03:32:11.204 INFO:teuthology.orchestra.run.smithi195.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T03:32:11.384 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T03:32:11.384 INFO:teuthology.orchestra.run.smithi195.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-21T03:32:11.477 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-21T03:32:11.542 INFO:teuthology.orchestra.run.smithi195.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T03:32:14.710 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 142/165 2024-09-21T03:32:14.751 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 143/165 2024-09-21T03:32:14.791 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 144/165 2024-09-21T03:32:14.823 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 145/165 2024-09-21T03:32:14.850 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 146/165 2024-09-21T03:32:15.085 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 147/165 2024-09-21T03:32:15.130 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 148/165 2024-09-21T03:32:15.160 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_ 149/165 2024-09-21T03:32:15.199 INFO:teuthology.orchestra.run.smithi038.stdout: Upgrading : NetworkManager-team-1:1.51.0-1.el9.x86_64 150/165 2024-09-21T03:32:15.245 INFO:teuthology.orchestra.run.smithi038.stdout: Upgrading : NetworkManager-tui-1:1.51.0-1.el9.x86_64 151/165 2024-09-21T03:32:15.895 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T03:32:15.934 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T03:32:15.955 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T03:32:16.861 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 153/165 2024-09-21T03:32:16.947 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 154/165 2024-09-21T03:32:17.008 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 155/165 2024-09-21T03:32:17.112 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : flex-2.6.4-9.el9.x86_64 156/165 2024-09-21T03:32:17.150 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : bison-3.7.4-5.el9.x86_64 157/165 2024-09-21T03:32:17.175 INFO:teuthology.orchestra.run.smithi038.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 158/165 2024-09-21T03:32:17.201 INFO:teuthology.orchestra.run.smithi038.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 159/165 2024-09-21T03:32:17.201 INFO:teuthology.orchestra.run.smithi038.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-21T03:32:17.294 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-21T03:32:17.294 INFO:teuthology.orchestra.run.smithi038.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-21T03:32:17.378 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-21T03:32:17.402 INFO:teuthology.orchestra.run.smithi038.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 162/165 2024-09-21T03:32:17.408 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T03:32:17.408 INFO:teuthology.orchestra.run.smithi038.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T03:32:17.596 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T03:32:17.596 INFO:teuthology.orchestra.run.smithi038.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-21T03:32:17.681 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-21T03:32:17.748 INFO:teuthology.orchestra.run.smithi038.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T03:32:18.058 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T03:32:18.058 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-2:19.3.0-5120.g18432218.el9.x86_64 1/165 2024-09-21T03:32:18.058 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 2/165 2024-09-21T03:32:18.058 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 3/165 2024-09-21T03:32:18.058 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 4/165 2024-09-21T03:32:18.058 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-5120.g18432 5/165 2024-09-21T03:32:18.058 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 6/165 2024-09-21T03:32:18.058 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 7/165 2024-09-21T03:32:18.058 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T03:32:18.058 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T03:32:18.058 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 10/165 2024-09-21T03:32:18.058 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 11/165 2024-09-21T03:32:18.058 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 12/165 2024-09-21T03:32:18.059 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 13/165 2024-09-21T03:32:18.059 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T03:32:18.059 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_ 15/165 2024-09-21T03:32:18.059 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 16/165 2024-09-21T03:32:18.059 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 17/165 2024-09-21T03:32:18.059 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : librados-devel-2:19.3.0-5120.g18432218.el9.x86_6 18/165 2024-09-21T03:32:18.059 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 19/165 2024-09-21T03:32:18.059 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : librgw2-2:19.3.0-5120.g18432218.el9.x86_64 20/165 2024-09-21T03:32:18.059 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-ceph-argparse-2:19.3.0-5120.g18432218.el 21/165 2024-09-21T03:32:18.059 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-ceph-common-2:19.3.0-5120.g18432218.el9. 22/165 2024-09-21T03:32:18.059 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_6 23/165 2024-09-21T03:32:18.059 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 24/165 2024-09-21T03:32:18.059 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 25/165 2024-09-21T03:32:18.059 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 26/165 2024-09-21T03:32:18.060 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 27/165 2024-09-21T03:32:18.060 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 28/165 2024-09-21T03:32:18.060 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 29/165 2024-09-21T03:32:18.060 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-5120.g18432218. 30/165 2024-09-21T03:32:18.060 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 31/165 2024-09-21T03:32:18.060 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 32/165 2024-09-21T03:32:18.060 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 33/165 2024-09-21T03:32:18.060 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el 34/165 2024-09-21T03:32:18.060 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 35/165 2024-09-21T03:32:18.060 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-5120.g18432218.e 36/165 2024-09-21T03:32:18.060 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 37/165 2024-09-21T03:32:18.060 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : cephadm-2:19.3.0-5120.g18432218.el9.noarch 38/165 2024-09-21T03:32:18.060 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 39/165 2024-09-21T03:32:18.060 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/165 2024-09-21T03:32:18.060 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/165 2024-09-21T03:32:18.061 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/165 2024-09-21T03:32:18.061 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/165 2024-09-21T03:32:18.061 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/165 2024-09-21T03:32:18.061 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/165 2024-09-21T03:32:18.061 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/165 2024-09-21T03:32:18.061 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/165 2024-09-21T03:32:18.061 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/165 2024-09-21T03:32:18.061 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/165 2024-09-21T03:32:18.061 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/165 2024-09-21T03:32:18.061 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/165 2024-09-21T03:32:18.061 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/165 2024-09-21T03:32:18.061 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/165 2024-09-21T03:32:18.061 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/165 2024-09-21T03:32:18.061 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/165 2024-09-21T03:32:18.061 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/165 2024-09-21T03:32:18.062 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/165 2024-09-21T03:32:18.062 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 58/165 2024-09-21T03:32:18.062 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 59/165 2024-09-21T03:32:18.062 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 60/165 2024-09-21T03:32:18.062 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 61/165 2024-09-21T03:32:18.062 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 62/165 2024-09-21T03:32:18.062 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 63/165 2024-09-21T03:32:18.062 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 64/165 2024-09-21T03:32:18.062 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 65/165 2024-09-21T03:32:18.062 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : lua-5.4.4-4.el9.x86_64 66/165 2024-09-21T03:32:18.062 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 67/165 2024-09-21T03:32:18.062 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 68/165 2024-09-21T03:32:18.062 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 69/165 2024-09-21T03:32:18.062 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : postgresql-13.16-1.el9.x86_64 70/165 2024-09-21T03:32:18.062 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 71/165 2024-09-21T03:32:18.063 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 72/165 2024-09-21T03:32:18.063 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 73/165 2024-09-21T03:32:18.063 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 74/165 2024-09-21T03:32:18.063 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 75/165 2024-09-21T03:32:18.063 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 76/165 2024-09-21T03:32:18.063 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 77/165 2024-09-21T03:32:18.063 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 78/165 2024-09-21T03:32:18.063 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 79/165 2024-09-21T03:32:18.063 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 80/165 2024-09-21T03:32:18.063 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 81/165 2024-09-21T03:32:18.063 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 82/165 2024-09-21T03:32:18.063 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 83/165 2024-09-21T03:32:18.063 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 84/165 2024-09-21T03:32:18.063 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 85/165 2024-09-21T03:32:18.064 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 86/165 2024-09-21T03:32:18.064 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 87/165 2024-09-21T03:32:18.064 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 88/165 2024-09-21T03:32:18.064 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 89/165 2024-09-21T03:32:18.064 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 90/165 2024-09-21T03:32:18.064 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 91/165 2024-09-21T03:32:18.064 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/165 2024-09-21T03:32:18.064 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 93/165 2024-09-21T03:32:18.064 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 94/165 2024-09-21T03:32:18.064 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 95/165 2024-09-21T03:32:18.064 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 96/165 2024-09-21T03:32:18.064 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 97/165 2024-09-21T03:32:18.064 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 98/165 2024-09-21T03:32:18.064 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 99/165 2024-09-21T03:32:18.064 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 100/165 2024-09-21T03:32:18.064 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 101/165 2024-09-21T03:32:18.065 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 102/165 2024-09-21T03:32:18.065 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 103/165 2024-09-21T03:32:18.065 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 104/165 2024-09-21T03:32:18.065 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/165 2024-09-21T03:32:18.065 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/165 2024-09-21T03:32:18.065 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/165 2024-09-21T03:32:18.065 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 108/165 2024-09-21T03:32:18.065 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/165 2024-09-21T03:32:18.065 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 110/165 2024-09-21T03:32:18.065 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 111/165 2024-09-21T03:32:18.065 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/165 2024-09-21T03:32:18.065 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/165 2024-09-21T03:32:18.065 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 114/165 2024-09-21T03:32:18.065 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 115/165 2024-09-21T03:32:18.065 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/165 2024-09-21T03:32:18.066 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/165 2024-09-21T03:32:18.066 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/165 2024-09-21T03:32:18.066 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 119/165 2024-09-21T03:32:18.066 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/165 2024-09-21T03:32:18.066 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/165 2024-09-21T03:32:18.066 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/165 2024-09-21T03:32:18.066 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 123/165 2024-09-21T03:32:18.066 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 124/165 2024-09-21T03:32:18.066 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 125/165 2024-09-21T03:32:18.066 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 126/165 2024-09-21T03:32:18.066 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 127/165 2024-09-21T03:32:18.066 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 128/165 2024-09-21T03:32:18.066 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 129/165 2024-09-21T03:32:18.066 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 130/165 2024-09-21T03:32:18.067 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 131/165 2024-09-21T03:32:18.067 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 132/165 2024-09-21T03:32:18.067 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 133/165 2024-09-21T03:32:18.067 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 134/165 2024-09-21T03:32:18.067 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 135/165 2024-09-21T03:32:18.067 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 136/165 2024-09-21T03:32:18.067 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 137/165 2024-09-21T03:32:18.067 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 138/165 2024-09-21T03:32:18.067 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 139/165 2024-09-21T03:32:18.067 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 140/165 2024-09-21T03:32:18.067 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 141/165 2024-09-21T03:32:18.067 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 142/165 2024-09-21T03:32:18.067 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 143/165 2024-09-21T03:32:18.067 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 144/165 2024-09-21T03:32:18.067 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 145/165 2024-09-21T03:32:18.068 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 146/165 2024-09-21T03:32:18.068 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 147/165 2024-09-21T03:32:18.068 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : re2-1:20211101-20.el9.x86_64 148/165 2024-09-21T03:32:18.068 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 149/165 2024-09-21T03:32:18.068 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : librados2-2:19.3.0-5120.g18432218.el9.x86_64 150/165 2024-09-21T03:32:18.068 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 151/165 2024-09-21T03:32:18.068 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : librbd1-2:19.3.0-5120.g18432218.el9.x86_64 152/165 2024-09-21T03:32:18.068 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 153/165 2024-09-21T03:32:18.068 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : NetworkManager-1:1.51.0-1.el9.x86_64 154/165 2024-09-21T03:32:18.068 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 155/165 2024-09-21T03:32:18.068 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 156/165 2024-09-21T03:32:18.068 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 157/165 2024-09-21T03:32:18.068 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : NetworkManager-team-1:1.51.0-1.el9.x86_64 158/165 2024-09-21T03:32:18.068 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 159/165 2024-09-21T03:32:18.068 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : NetworkManager-tui-1:1.51.0-1.el9.x86_64 160/165 2024-09-21T03:32:18.069 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 161/165 2024-09-21T03:32:18.069 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 162/165 2024-09-21T03:32:18.069 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 163/165 2024-09-21T03:32:18.069 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 164/165 2024-09-21T03:32:20.795 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T03:32:20.795 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-21T03:32:20.795 INFO:teuthology.orchestra.run.smithi195.stdout:Upgraded: 2024-09-21T03:32:20.795 INFO:teuthology.orchestra.run.smithi195.stdout: NetworkManager-1:1.51.0-1.el9.x86_64 2024-09-21T03:32:20.795 INFO:teuthology.orchestra.run.smithi195.stdout: NetworkManager-libnm-1:1.51.0-1.el9.x86_64 2024-09-21T03:32:20.795 INFO:teuthology.orchestra.run.smithi195.stdout: NetworkManager-team-1:1.51.0-1.el9.x86_64 2024-09-21T03:32:20.795 INFO:teuthology.orchestra.run.smithi195.stdout: NetworkManager-tui-1:1.51.0-1.el9.x86_64 2024-09-21T03:32:20.795 INFO:teuthology.orchestra.run.smithi195.stdout: librados2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:32:20.795 INFO:teuthology.orchestra.run.smithi195.stdout: librbd1-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:32:20.795 INFO:teuthology.orchestra.run.smithi195.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-21T03:32:20.796 INFO:teuthology.orchestra.run.smithi195.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-21T03:32:20.796 INFO:teuthology.orchestra.run.smithi195.stdout:Installed: 2024-09-21T03:32:20.796 INFO:teuthology.orchestra.run.smithi195.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-21T03:32:20.796 INFO:teuthology.orchestra.run.smithi195.stdout: automake-1.16.2-8.el9.noarch 2024-09-21T03:32:20.796 INFO:teuthology.orchestra.run.smithi195.stdout: bison-3.7.4-5.el9.x86_64 2024-09-21T03:32:20.796 INFO:teuthology.orchestra.run.smithi195.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-21T03:32:20.796 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:32:20.796 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:32:20.796 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:32:20.796 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:32:20.796 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-grafana-dashboards-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:32:20.796 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-immutable-object-cache-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:32:20.796 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:32:20.796 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:32:20.796 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:32:20.797 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:32:20.797 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mgr-diskprediction-local-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:32:20.797 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:32:20.797 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:32:20.797 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:32:20.797 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:32:20.797 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-prometheus-alerts-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:32:20.797 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:32:20.797 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:32:20.797 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:32:20.797 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:32:20.797 INFO:teuthology.orchestra.run.smithi195.stdout: cephadm-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:32:20.797 INFO:teuthology.orchestra.run.smithi195.stdout: cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:32:20.797 INFO:teuthology.orchestra.run.smithi195.stdout: cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:32:20.798 INFO:teuthology.orchestra.run.smithi195.stdout: cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:32:20.798 INFO:teuthology.orchestra.run.smithi195.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-21T03:32:20.798 INFO:teuthology.orchestra.run.smithi195.stdout: flex-2.6.4-9.el9.x86_64 2024-09-21T03:32:20.798 INFO:teuthology.orchestra.run.smithi195.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-21T03:32:20.798 INFO:teuthology.orchestra.run.smithi195.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-21T03:32:20.798 INFO:teuthology.orchestra.run.smithi195.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-21T03:32:20.798 INFO:teuthology.orchestra.run.smithi195.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-21T03:32:20.798 INFO:teuthology.orchestra.run.smithi195.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-21T03:32:20.798 INFO:teuthology.orchestra.run.smithi195.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-21T03:32:20.798 INFO:teuthology.orchestra.run.smithi195.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-21T03:32:20.798 INFO:teuthology.orchestra.run.smithi195.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-21T03:32:20.798 INFO:teuthology.orchestra.run.smithi195.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-21T03:32:20.798 INFO:teuthology.orchestra.run.smithi195.stdout: libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:32:20.799 INFO:teuthology.orchestra.run.smithi195.stdout: libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:32:20.799 INFO:teuthology.orchestra.run.smithi195.stdout: libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:32:20.799 INFO:teuthology.orchestra.run.smithi195.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-21T03:32:20.799 INFO:teuthology.orchestra.run.smithi195.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-21T03:32:20.799 INFO:teuthology.orchestra.run.smithi195.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-21T03:32:20.799 INFO:teuthology.orchestra.run.smithi195.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-21T03:32:20.799 INFO:teuthology.orchestra.run.smithi195.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-21T03:32:20.799 INFO:teuthology.orchestra.run.smithi195.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-21T03:32:20.799 INFO:teuthology.orchestra.run.smithi195.stdout: librados-devel-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:32:20.799 INFO:teuthology.orchestra.run.smithi195.stdout: libradosstriper1-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:32:20.799 INFO:teuthology.orchestra.run.smithi195.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-21T03:32:20.799 INFO:teuthology.orchestra.run.smithi195.stdout: librgw2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:32:20.799 INFO:teuthology.orchestra.run.smithi195.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-21T03:32:20.799 INFO:teuthology.orchestra.run.smithi195.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-21T03:32:20.800 INFO:teuthology.orchestra.run.smithi195.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-21T03:32:20.800 INFO:teuthology.orchestra.run.smithi195.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-21T03:32:20.800 INFO:teuthology.orchestra.run.smithi195.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-21T03:32:20.800 INFO:teuthology.orchestra.run.smithi195.stdout: lua-5.4.4-4.el9.x86_64 2024-09-21T03:32:20.800 INFO:teuthology.orchestra.run.smithi195.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-21T03:32:20.800 INFO:teuthology.orchestra.run.smithi195.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-21T03:32:20.800 INFO:teuthology.orchestra.run.smithi195.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-21T03:32:20.800 INFO:teuthology.orchestra.run.smithi195.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-21T03:32:20.800 INFO:teuthology.orchestra.run.smithi195.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-21T03:32:20.800 INFO:teuthology.orchestra.run.smithi195.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-21T03:32:20.800 INFO:teuthology.orchestra.run.smithi195.stdout: postgresql-13.16-1.el9.x86_64 2024-09-21T03:32:20.800 INFO:teuthology.orchestra.run.smithi195.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-21T03:32:20.800 INFO:teuthology.orchestra.run.smithi195.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-21T03:32:20.801 INFO:teuthology.orchestra.run.smithi195.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-21T03:32:20.801 INFO:teuthology.orchestra.run.smithi195.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-21T03:32:20.801 INFO:teuthology.orchestra.run.smithi195.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-21T03:32:20.801 INFO:teuthology.orchestra.run.smithi195.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-21T03:32:20.801 INFO:teuthology.orchestra.run.smithi195.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-21T03:32:20.801 INFO:teuthology.orchestra.run.smithi195.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-21T03:32:20.801 INFO:teuthology.orchestra.run.smithi195.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-21T03:32:20.801 INFO:teuthology.orchestra.run.smithi195.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-21T03:32:20.801 INFO:teuthology.orchestra.run.smithi195.stdout: python3-ceph-argparse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:32:20.801 INFO:teuthology.orchestra.run.smithi195.stdout: python3-ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:32:20.801 INFO:teuthology.orchestra.run.smithi195.stdout: python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:32:20.801 INFO:teuthology.orchestra.run.smithi195.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-21T03:32:20.801 INFO:teuthology.orchestra.run.smithi195.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-21T03:32:20.802 INFO:teuthology.orchestra.run.smithi195.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-21T03:32:20.802 INFO:teuthology.orchestra.run.smithi195.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-21T03:32:20.802 INFO:teuthology.orchestra.run.smithi195.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-21T03:32:20.802 INFO:teuthology.orchestra.run.smithi195.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-21T03:32:20.802 INFO:teuthology.orchestra.run.smithi195.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-21T03:32:20.802 INFO:teuthology.orchestra.run.smithi195.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-21T03:32:20.802 INFO:teuthology.orchestra.run.smithi195.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-21T03:32:20.802 INFO:teuthology.orchestra.run.smithi195.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-21T03:32:20.802 INFO:teuthology.orchestra.run.smithi195.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-21T03:32:20.802 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-21T03:32:20.802 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-21T03:32:20.802 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-21T03:32:20.803 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-21T03:32:20.803 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-21T03:32:20.803 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-21T03:32:20.803 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-21T03:32:20.803 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-21T03:32:20.803 INFO:teuthology.orchestra.run.smithi195.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-21T03:32:20.803 INFO:teuthology.orchestra.run.smithi195.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-21T03:32:20.803 INFO:teuthology.orchestra.run.smithi195.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-21T03:32:20.803 INFO:teuthology.orchestra.run.smithi195.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-21T03:32:20.803 INFO:teuthology.orchestra.run.smithi195.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-21T03:32:20.803 INFO:teuthology.orchestra.run.smithi195.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-21T03:32:20.803 INFO:teuthology.orchestra.run.smithi195.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-21T03:32:20.803 INFO:teuthology.orchestra.run.smithi195.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-21T03:32:20.803 INFO:teuthology.orchestra.run.smithi195.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-21T03:32:20.804 INFO:teuthology.orchestra.run.smithi195.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-21T03:32:20.804 INFO:teuthology.orchestra.run.smithi195.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-21T03:32:20.804 INFO:teuthology.orchestra.run.smithi195.stdout: python3-ply-3.11-14.el9.noarch 2024-09-21T03:32:20.804 INFO:teuthology.orchestra.run.smithi195.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-21T03:32:20.804 INFO:teuthology.orchestra.run.smithi195.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-21T03:32:20.804 INFO:teuthology.orchestra.run.smithi195.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-21T03:32:20.804 INFO:teuthology.orchestra.run.smithi195.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-21T03:32:20.804 INFO:teuthology.orchestra.run.smithi195.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-21T03:32:20.804 INFO:teuthology.orchestra.run.smithi195.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-21T03:32:20.804 INFO:teuthology.orchestra.run.smithi195.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-21T03:32:20.804 INFO:teuthology.orchestra.run.smithi195.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-21T03:32:20.804 INFO:teuthology.orchestra.run.smithi195.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-21T03:32:20.804 INFO:teuthology.orchestra.run.smithi195.stdout: python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:32:20.804 INFO:teuthology.orchestra.run.smithi195.stdout: python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:32:20.805 INFO:teuthology.orchestra.run.smithi195.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-21T03:32:20.805 INFO:teuthology.orchestra.run.smithi195.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-21T03:32:20.805 INFO:teuthology.orchestra.run.smithi195.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-21T03:32:20.805 INFO:teuthology.orchestra.run.smithi195.stdout: python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:32:20.805 INFO:teuthology.orchestra.run.smithi195.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-21T03:32:20.805 INFO:teuthology.orchestra.run.smithi195.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-21T03:32:20.805 INFO:teuthology.orchestra.run.smithi195.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-21T03:32:20.805 INFO:teuthology.orchestra.run.smithi195.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-21T03:32:20.805 INFO:teuthology.orchestra.run.smithi195.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-21T03:32:20.805 INFO:teuthology.orchestra.run.smithi195.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-21T03:32:20.805 INFO:teuthology.orchestra.run.smithi195.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-21T03:32:20.805 INFO:teuthology.orchestra.run.smithi195.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-21T03:32:20.805 INFO:teuthology.orchestra.run.smithi195.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-21T03:32:20.806 INFO:teuthology.orchestra.run.smithi195.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-21T03:32:20.806 INFO:teuthology.orchestra.run.smithi195.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-21T03:32:20.806 INFO:teuthology.orchestra.run.smithi195.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-21T03:32:20.806 INFO:teuthology.orchestra.run.smithi195.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-21T03:32:20.806 INFO:teuthology.orchestra.run.smithi195.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-21T03:32:20.806 INFO:teuthology.orchestra.run.smithi195.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-21T03:32:20.806 INFO:teuthology.orchestra.run.smithi195.stdout: rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:32:20.806 INFO:teuthology.orchestra.run.smithi195.stdout: rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:32:20.806 INFO:teuthology.orchestra.run.smithi195.stdout: rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:32:20.806 INFO:teuthology.orchestra.run.smithi195.stdout: re2-1:20211101-20.el9.x86_64 2024-09-21T03:32:20.806 INFO:teuthology.orchestra.run.smithi195.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-21T03:32:20.806 INFO:teuthology.orchestra.run.smithi195.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-21T03:32:20.806 INFO:teuthology.orchestra.run.smithi195.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-21T03:32:20.807 INFO:teuthology.orchestra.run.smithi195.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-21T03:32:20.807 INFO:teuthology.orchestra.run.smithi195.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-21T03:32:20.807 INFO:teuthology.orchestra.run.smithi195.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-21T03:32:20.807 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-21T03:32:20.807 INFO:teuthology.orchestra.run.smithi195.stdout:Complete! 2024-09-21T03:32:21.229 DEBUG:teuthology.parallel:result is None 2024-09-21T03:32:24.281 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T03:32:24.281 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-2:19.3.0-5120.g18432218.el9.x86_64 1/165 2024-09-21T03:32:24.281 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 2/165 2024-09-21T03:32:24.282 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 3/165 2024-09-21T03:32:24.282 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 4/165 2024-09-21T03:32:24.282 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-5120.g18432 5/165 2024-09-21T03:32:24.282 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 6/165 2024-09-21T03:32:24.282 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 7/165 2024-09-21T03:32:24.282 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T03:32:24.282 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T03:32:24.282 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 10/165 2024-09-21T03:32:24.282 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 11/165 2024-09-21T03:32:24.282 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 12/165 2024-09-21T03:32:24.282 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 13/165 2024-09-21T03:32:24.282 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T03:32:24.283 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_ 15/165 2024-09-21T03:32:24.283 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 16/165 2024-09-21T03:32:24.283 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 17/165 2024-09-21T03:32:24.283 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : librados-devel-2:19.3.0-5120.g18432218.el9.x86_6 18/165 2024-09-21T03:32:24.283 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 19/165 2024-09-21T03:32:24.283 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : librgw2-2:19.3.0-5120.g18432218.el9.x86_64 20/165 2024-09-21T03:32:24.283 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-ceph-argparse-2:19.3.0-5120.g18432218.el 21/165 2024-09-21T03:32:24.283 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-ceph-common-2:19.3.0-5120.g18432218.el9. 22/165 2024-09-21T03:32:24.283 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_6 23/165 2024-09-21T03:32:24.283 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 24/165 2024-09-21T03:32:24.283 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 25/165 2024-09-21T03:32:24.283 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 26/165 2024-09-21T03:32:24.283 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 27/165 2024-09-21T03:32:24.284 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 28/165 2024-09-21T03:32:24.284 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 29/165 2024-09-21T03:32:24.294 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-5120.g18432218. 30/165 2024-09-21T03:32:24.294 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 31/165 2024-09-21T03:32:24.294 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 32/165 2024-09-21T03:32:24.294 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 33/165 2024-09-21T03:32:24.295 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el 34/165 2024-09-21T03:32:24.295 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 35/165 2024-09-21T03:32:24.295 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-5120.g18432218.e 36/165 2024-09-21T03:32:24.295 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 37/165 2024-09-21T03:32:24.295 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : cephadm-2:19.3.0-5120.g18432218.el9.noarch 38/165 2024-09-21T03:32:24.295 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 39/165 2024-09-21T03:32:24.295 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/165 2024-09-21T03:32:24.295 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/165 2024-09-21T03:32:24.295 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/165 2024-09-21T03:32:24.295 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/165 2024-09-21T03:32:24.295 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/165 2024-09-21T03:32:24.295 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/165 2024-09-21T03:32:24.295 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/165 2024-09-21T03:32:24.296 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/165 2024-09-21T03:32:24.296 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/165 2024-09-21T03:32:24.296 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/165 2024-09-21T03:32:24.296 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/165 2024-09-21T03:32:24.296 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/165 2024-09-21T03:32:24.296 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/165 2024-09-21T03:32:24.296 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/165 2024-09-21T03:32:24.296 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/165 2024-09-21T03:32:24.296 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/165 2024-09-21T03:32:24.296 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/165 2024-09-21T03:32:24.296 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/165 2024-09-21T03:32:24.296 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 58/165 2024-09-21T03:32:24.297 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 59/165 2024-09-21T03:32:24.297 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 60/165 2024-09-21T03:32:24.297 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 61/165 2024-09-21T03:32:24.297 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 62/165 2024-09-21T03:32:24.297 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 63/165 2024-09-21T03:32:24.297 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 64/165 2024-09-21T03:32:24.297 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 65/165 2024-09-21T03:32:24.297 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : lua-5.4.4-4.el9.x86_64 66/165 2024-09-21T03:32:24.297 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 67/165 2024-09-21T03:32:24.297 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 68/165 2024-09-21T03:32:24.297 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 69/165 2024-09-21T03:32:24.297 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : postgresql-13.16-1.el9.x86_64 70/165 2024-09-21T03:32:24.297 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 71/165 2024-09-21T03:32:24.297 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 72/165 2024-09-21T03:32:24.298 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 73/165 2024-09-21T03:32:24.298 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 74/165 2024-09-21T03:32:24.298 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 75/165 2024-09-21T03:32:24.298 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 76/165 2024-09-21T03:32:24.298 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 77/165 2024-09-21T03:32:24.298 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 78/165 2024-09-21T03:32:24.298 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 79/165 2024-09-21T03:32:24.298 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 80/165 2024-09-21T03:32:24.298 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 81/165 2024-09-21T03:32:24.298 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 82/165 2024-09-21T03:32:24.298 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 83/165 2024-09-21T03:32:24.298 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 84/165 2024-09-21T03:32:24.298 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 85/165 2024-09-21T03:32:24.299 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 86/165 2024-09-21T03:32:24.299 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 87/165 2024-09-21T03:32:24.299 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 88/165 2024-09-21T03:32:24.299 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 89/165 2024-09-21T03:32:24.299 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 90/165 2024-09-21T03:32:24.299 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 91/165 2024-09-21T03:32:24.299 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/165 2024-09-21T03:32:24.299 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 93/165 2024-09-21T03:32:24.299 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 94/165 2024-09-21T03:32:24.299 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 95/165 2024-09-21T03:32:24.299 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 96/165 2024-09-21T03:32:24.299 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 97/165 2024-09-21T03:32:24.299 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 98/165 2024-09-21T03:32:24.299 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 99/165 2024-09-21T03:32:24.300 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 100/165 2024-09-21T03:32:24.300 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 101/165 2024-09-21T03:32:24.300 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 102/165 2024-09-21T03:32:24.300 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 103/165 2024-09-21T03:32:24.300 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 104/165 2024-09-21T03:32:24.300 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/165 2024-09-21T03:32:24.300 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/165 2024-09-21T03:32:24.300 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/165 2024-09-21T03:32:24.300 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 108/165 2024-09-21T03:32:24.300 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/165 2024-09-21T03:32:24.300 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 110/165 2024-09-21T03:32:24.300 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 111/165 2024-09-21T03:32:24.300 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/165 2024-09-21T03:32:24.300 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/165 2024-09-21T03:32:24.301 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 114/165 2024-09-21T03:32:24.301 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 115/165 2024-09-21T03:32:24.301 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/165 2024-09-21T03:32:24.301 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/165 2024-09-21T03:32:24.301 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/165 2024-09-21T03:32:24.301 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 119/165 2024-09-21T03:32:24.301 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/165 2024-09-21T03:32:24.301 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/165 2024-09-21T03:32:24.301 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/165 2024-09-21T03:32:24.301 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 123/165 2024-09-21T03:32:24.301 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 124/165 2024-09-21T03:32:24.301 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 125/165 2024-09-21T03:32:24.301 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 126/165 2024-09-21T03:32:24.301 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 127/165 2024-09-21T03:32:24.302 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 128/165 2024-09-21T03:32:24.302 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 129/165 2024-09-21T03:32:24.302 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 130/165 2024-09-21T03:32:24.302 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 131/165 2024-09-21T03:32:24.302 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 132/165 2024-09-21T03:32:24.302 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 133/165 2024-09-21T03:32:24.302 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 134/165 2024-09-21T03:32:24.302 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 135/165 2024-09-21T03:32:24.302 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 136/165 2024-09-21T03:32:24.302 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 137/165 2024-09-21T03:32:24.302 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 138/165 2024-09-21T03:32:24.302 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 139/165 2024-09-21T03:32:24.302 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 140/165 2024-09-21T03:32:24.303 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 141/165 2024-09-21T03:32:24.303 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 142/165 2024-09-21T03:32:24.303 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 143/165 2024-09-21T03:32:24.303 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 144/165 2024-09-21T03:32:24.303 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 145/165 2024-09-21T03:32:24.303 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 146/165 2024-09-21T03:32:24.303 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 147/165 2024-09-21T03:32:24.303 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : re2-1:20211101-20.el9.x86_64 148/165 2024-09-21T03:32:24.303 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 149/165 2024-09-21T03:32:24.303 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : librados2-2:19.3.0-5120.g18432218.el9.x86_64 150/165 2024-09-21T03:32:24.303 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 151/165 2024-09-21T03:32:24.303 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : librbd1-2:19.3.0-5120.g18432218.el9.x86_64 152/165 2024-09-21T03:32:24.303 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 153/165 2024-09-21T03:32:24.304 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : NetworkManager-1:1.51.0-1.el9.x86_64 154/165 2024-09-21T03:32:24.304 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 155/165 2024-09-21T03:32:24.304 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 156/165 2024-09-21T03:32:24.304 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 157/165 2024-09-21T03:32:24.304 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : NetworkManager-team-1:1.51.0-1.el9.x86_64 158/165 2024-09-21T03:32:24.304 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 159/165 2024-09-21T03:32:24.304 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : NetworkManager-tui-1:1.51.0-1.el9.x86_64 160/165 2024-09-21T03:32:24.304 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 161/165 2024-09-21T03:32:24.304 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 162/165 2024-09-21T03:32:24.304 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 163/165 2024-09-21T03:32:24.304 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 164/165 2024-09-21T03:32:27.067 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T03:32:27.067 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:32:27.067 INFO:teuthology.orchestra.run.smithi038.stdout:Upgraded: 2024-09-21T03:32:27.067 INFO:teuthology.orchestra.run.smithi038.stdout: NetworkManager-1:1.51.0-1.el9.x86_64 2024-09-21T03:32:27.068 INFO:teuthology.orchestra.run.smithi038.stdout: NetworkManager-libnm-1:1.51.0-1.el9.x86_64 2024-09-21T03:32:27.068 INFO:teuthology.orchestra.run.smithi038.stdout: NetworkManager-team-1:1.51.0-1.el9.x86_64 2024-09-21T03:32:27.068 INFO:teuthology.orchestra.run.smithi038.stdout: NetworkManager-tui-1:1.51.0-1.el9.x86_64 2024-09-21T03:32:27.068 INFO:teuthology.orchestra.run.smithi038.stdout: librados2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:32:27.068 INFO:teuthology.orchestra.run.smithi038.stdout: librbd1-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:32:27.068 INFO:teuthology.orchestra.run.smithi038.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-21T03:32:27.068 INFO:teuthology.orchestra.run.smithi038.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-21T03:32:27.068 INFO:teuthology.orchestra.run.smithi038.stdout:Installed: 2024-09-21T03:32:27.068 INFO:teuthology.orchestra.run.smithi038.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-21T03:32:27.068 INFO:teuthology.orchestra.run.smithi038.stdout: automake-1.16.2-8.el9.noarch 2024-09-21T03:32:27.068 INFO:teuthology.orchestra.run.smithi038.stdout: bison-3.7.4-5.el9.x86_64 2024-09-21T03:32:27.068 INFO:teuthology.orchestra.run.smithi038.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-21T03:32:27.068 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:32:27.068 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:32:27.069 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:32:27.069 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:32:27.069 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-grafana-dashboards-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:32:27.069 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-immutable-object-cache-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:32:27.069 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:32:27.069 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:32:27.069 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:32:27.069 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:32:27.069 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mgr-diskprediction-local-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:32:27.069 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:32:27.069 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:32:27.069 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:32:27.069 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:32:27.069 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-prometheus-alerts-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:32:27.069 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:32:27.070 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:32:27.070 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:32:27.070 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:32:27.070 INFO:teuthology.orchestra.run.smithi038.stdout: cephadm-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:32:27.070 INFO:teuthology.orchestra.run.smithi038.stdout: cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:32:27.070 INFO:teuthology.orchestra.run.smithi038.stdout: cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:32:27.070 INFO:teuthology.orchestra.run.smithi038.stdout: cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:32:27.070 INFO:teuthology.orchestra.run.smithi038.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-21T03:32:27.070 INFO:teuthology.orchestra.run.smithi038.stdout: flex-2.6.4-9.el9.x86_64 2024-09-21T03:32:27.070 INFO:teuthology.orchestra.run.smithi038.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-21T03:32:27.070 INFO:teuthology.orchestra.run.smithi038.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-21T03:32:27.070 INFO:teuthology.orchestra.run.smithi038.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-21T03:32:27.070 INFO:teuthology.orchestra.run.smithi038.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-21T03:32:27.070 INFO:teuthology.orchestra.run.smithi038.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-21T03:32:27.071 INFO:teuthology.orchestra.run.smithi038.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-21T03:32:27.071 INFO:teuthology.orchestra.run.smithi038.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-21T03:32:27.071 INFO:teuthology.orchestra.run.smithi038.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-21T03:32:27.071 INFO:teuthology.orchestra.run.smithi038.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-21T03:32:27.071 INFO:teuthology.orchestra.run.smithi038.stdout: libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:32:27.071 INFO:teuthology.orchestra.run.smithi038.stdout: libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:32:27.071 INFO:teuthology.orchestra.run.smithi038.stdout: libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:32:27.071 INFO:teuthology.orchestra.run.smithi038.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-21T03:32:27.071 INFO:teuthology.orchestra.run.smithi038.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-21T03:32:27.071 INFO:teuthology.orchestra.run.smithi038.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-21T03:32:27.071 INFO:teuthology.orchestra.run.smithi038.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-21T03:32:27.071 INFO:teuthology.orchestra.run.smithi038.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-21T03:32:27.071 INFO:teuthology.orchestra.run.smithi038.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-21T03:32:27.071 INFO:teuthology.orchestra.run.smithi038.stdout: librados-devel-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:32:27.072 INFO:teuthology.orchestra.run.smithi038.stdout: libradosstriper1-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:32:27.072 INFO:teuthology.orchestra.run.smithi038.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-21T03:32:27.072 INFO:teuthology.orchestra.run.smithi038.stdout: librgw2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:32:27.072 INFO:teuthology.orchestra.run.smithi038.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-21T03:32:27.072 INFO:teuthology.orchestra.run.smithi038.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-21T03:32:27.072 INFO:teuthology.orchestra.run.smithi038.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-21T03:32:27.072 INFO:teuthology.orchestra.run.smithi038.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-21T03:32:27.072 INFO:teuthology.orchestra.run.smithi038.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-21T03:32:27.072 INFO:teuthology.orchestra.run.smithi038.stdout: lua-5.4.4-4.el9.x86_64 2024-09-21T03:32:27.072 INFO:teuthology.orchestra.run.smithi038.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-21T03:32:27.072 INFO:teuthology.orchestra.run.smithi038.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-21T03:32:27.072 INFO:teuthology.orchestra.run.smithi038.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-21T03:32:27.072 INFO:teuthology.orchestra.run.smithi038.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-21T03:32:27.072 INFO:teuthology.orchestra.run.smithi038.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-21T03:32:27.072 INFO:teuthology.orchestra.run.smithi038.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-21T03:32:27.073 INFO:teuthology.orchestra.run.smithi038.stdout: postgresql-13.16-1.el9.x86_64 2024-09-21T03:32:27.073 INFO:teuthology.orchestra.run.smithi038.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-21T03:32:27.073 INFO:teuthology.orchestra.run.smithi038.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-21T03:32:27.073 INFO:teuthology.orchestra.run.smithi038.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-21T03:32:27.073 INFO:teuthology.orchestra.run.smithi038.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-21T03:32:27.073 INFO:teuthology.orchestra.run.smithi038.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-21T03:32:27.073 INFO:teuthology.orchestra.run.smithi038.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-21T03:32:27.073 INFO:teuthology.orchestra.run.smithi038.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-21T03:32:27.073 INFO:teuthology.orchestra.run.smithi038.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-21T03:32:27.073 INFO:teuthology.orchestra.run.smithi038.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-21T03:32:27.073 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-21T03:32:27.073 INFO:teuthology.orchestra.run.smithi038.stdout: python3-ceph-argparse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:32:27.073 INFO:teuthology.orchestra.run.smithi038.stdout: python3-ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:32:27.073 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:32:27.073 INFO:teuthology.orchestra.run.smithi038.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-21T03:32:27.074 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-21T03:32:27.074 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-21T03:32:27.074 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-21T03:32:27.074 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-21T03:32:27.074 INFO:teuthology.orchestra.run.smithi038.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-21T03:32:27.074 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-21T03:32:27.074 INFO:teuthology.orchestra.run.smithi038.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-21T03:32:27.074 INFO:teuthology.orchestra.run.smithi038.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-21T03:32:27.074 INFO:teuthology.orchestra.run.smithi038.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-21T03:32:27.074 INFO:teuthology.orchestra.run.smithi038.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-21T03:32:27.074 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-21T03:32:27.074 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-21T03:32:27.074 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-21T03:32:27.074 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-21T03:32:27.074 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-21T03:32:27.075 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-21T03:32:27.075 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-21T03:32:27.075 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-21T03:32:27.075 INFO:teuthology.orchestra.run.smithi038.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-21T03:32:27.075 INFO:teuthology.orchestra.run.smithi038.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-21T03:32:27.075 INFO:teuthology.orchestra.run.smithi038.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-21T03:32:27.075 INFO:teuthology.orchestra.run.smithi038.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-21T03:32:27.075 INFO:teuthology.orchestra.run.smithi038.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-21T03:32:27.075 INFO:teuthology.orchestra.run.smithi038.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-21T03:32:27.075 INFO:teuthology.orchestra.run.smithi038.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-21T03:32:27.075 INFO:teuthology.orchestra.run.smithi038.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-21T03:32:27.075 INFO:teuthology.orchestra.run.smithi038.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-21T03:32:27.075 INFO:teuthology.orchestra.run.smithi038.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-21T03:32:27.075 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-21T03:32:27.075 INFO:teuthology.orchestra.run.smithi038.stdout: python3-ply-3.11-14.el9.noarch 2024-09-21T03:32:27.076 INFO:teuthology.orchestra.run.smithi038.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-21T03:32:27.076 INFO:teuthology.orchestra.run.smithi038.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-21T03:32:27.076 INFO:teuthology.orchestra.run.smithi038.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-21T03:32:27.076 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-21T03:32:27.076 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-21T03:32:27.076 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-21T03:32:27.076 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-21T03:32:27.076 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-21T03:32:27.076 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-21T03:32:27.076 INFO:teuthology.orchestra.run.smithi038.stdout: python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:32:27.076 INFO:teuthology.orchestra.run.smithi038.stdout: python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:32:27.076 INFO:teuthology.orchestra.run.smithi038.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-21T03:32:27.076 INFO:teuthology.orchestra.run.smithi038.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-21T03:32:27.076 INFO:teuthology.orchestra.run.smithi038.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-21T03:32:27.076 INFO:teuthology.orchestra.run.smithi038.stdout: python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:32:27.077 INFO:teuthology.orchestra.run.smithi038.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-21T03:32:27.077 INFO:teuthology.orchestra.run.smithi038.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-21T03:32:27.077 INFO:teuthology.orchestra.run.smithi038.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-21T03:32:27.077 INFO:teuthology.orchestra.run.smithi038.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-21T03:32:27.077 INFO:teuthology.orchestra.run.smithi038.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-21T03:32:27.077 INFO:teuthology.orchestra.run.smithi038.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-21T03:32:27.077 INFO:teuthology.orchestra.run.smithi038.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-21T03:32:27.077 INFO:teuthology.orchestra.run.smithi038.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-21T03:32:27.077 INFO:teuthology.orchestra.run.smithi038.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-21T03:32:27.077 INFO:teuthology.orchestra.run.smithi038.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-21T03:32:27.077 INFO:teuthology.orchestra.run.smithi038.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-21T03:32:27.077 INFO:teuthology.orchestra.run.smithi038.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-21T03:32:27.077 INFO:teuthology.orchestra.run.smithi038.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-21T03:32:27.077 INFO:teuthology.orchestra.run.smithi038.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-21T03:32:27.077 INFO:teuthology.orchestra.run.smithi038.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-21T03:32:27.078 INFO:teuthology.orchestra.run.smithi038.stdout: rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:32:27.078 INFO:teuthology.orchestra.run.smithi038.stdout: rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:32:27.078 INFO:teuthology.orchestra.run.smithi038.stdout: rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:32:27.078 INFO:teuthology.orchestra.run.smithi038.stdout: re2-1:20211101-20.el9.x86_64 2024-09-21T03:32:27.078 INFO:teuthology.orchestra.run.smithi038.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-21T03:32:27.078 INFO:teuthology.orchestra.run.smithi038.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-21T03:32:27.078 INFO:teuthology.orchestra.run.smithi038.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-21T03:32:27.078 INFO:teuthology.orchestra.run.smithi038.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-21T03:32:27.078 INFO:teuthology.orchestra.run.smithi038.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-21T03:32:27.078 INFO:teuthology.orchestra.run.smithi038.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-21T03:32:27.078 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:32:27.079 INFO:teuthology.orchestra.run.smithi038.stdout:Complete! 2024-09-21T03:32:27.660 DEBUG:teuthology.parallel:result is None 2024-09-21T03:32:27.661 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T03:32:27.810 DEBUG:teuthology.orchestra.run.smithi038:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-21T03:32:27.834 INFO:teuthology.orchestra.run.smithi038.stdout:19.3.0-5120.g18432218.el9 2024-09-21T03:32:27.834 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-5120.g18432218.el9 2024-09-21T03:32:27.834 INFO:teuthology.task.install:The correct ceph version 19.3.0-5120.g18432218 is installed. 2024-09-21T03:32:27.836 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T03:32:27.975 DEBUG:teuthology.orchestra.run.smithi195:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-21T03:32:28.001 INFO:teuthology.orchestra.run.smithi195.stdout:19.3.0-5120.g18432218.el9 2024-09-21T03:32:28.001 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-5120.g18432218.el9 2024-09-21T03:32:28.001 INFO:teuthology.task.install:The correct ceph version 19.3.0-5120.g18432218 is installed. 2024-09-21T03:32:28.002 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-21T03:32:28.003 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-21T03:32:28.003 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-21T03:32:28.031 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-09-21T03:32:28.031 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-21T03:32:28.072 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-21T03:32:28.072 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-21T03:32:28.072 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd of=/usr/bin/daemon-helper 2024-09-21T03:32:28.099 DEBUG:teuthology.orchestra.run.smithi038:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-21T03:32:28.165 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-09-21T03:32:28.165 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd of=/usr/bin/daemon-helper 2024-09-21T03:32:28.193 DEBUG:teuthology.orchestra.run.smithi195:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-21T03:32:28.261 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-21T03:32:28.261 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-21T03:32:28.261 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-21T03:32:28.287 DEBUG:teuthology.orchestra.run.smithi038:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-21T03:32:28.353 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-09-21T03:32:28.353 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-21T03:32:28.380 DEBUG:teuthology.orchestra.run.smithi195:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-21T03:32:28.447 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-21T03:32:28.447 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-21T03:32:28.447 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd of=/usr/bin/stdin-killer 2024-09-21T03:32:28.473 DEBUG:teuthology.orchestra.run.smithi038:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-21T03:32:28.539 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-09-21T03:32:28.539 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd of=/usr/bin/stdin-killer 2024-09-21T03:32:28.565 DEBUG:teuthology.orchestra.run.smithi195:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-21T03:32:28.631 INFO:teuthology.run_tasks:Running task ceph... 2024-09-21T03:32:28.709 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-09-21T03:32:28.709 DEBUG:teuthology.orchestra.run.smithi038:> sudo chmod 777 /var/log/ceph 2024-09-21T03:32:28.711 DEBUG:teuthology.orchestra.run.smithi195:> sudo chmod 777 /var/log/ceph 2024-09-21T03:32:28.736 INFO:tasks.ceph:Disabling ceph logrotate... 2024-09-21T03:32:28.737 DEBUG:teuthology.orchestra.run.smithi038:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-21T03:32:28.777 DEBUG:teuthology.orchestra.run.smithi195:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-21T03:32:28.800 INFO:tasks.ceph:Creating extra log directories... 2024-09-21T03:32:28.801 DEBUG:teuthology.orchestra.run.smithi038:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-21T03:32:28.842 DEBUG:teuthology.orchestra.run.smithi195:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-21T03:32:28.871 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-09-21T03:32:28.872 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-09-21T03:32:28.872 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-09-21T03:32:28.872 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-21T03:32:28.872 DEBUG:teuthology.orchestra.run.smithi038:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-09-21T03:32:28.921 DEBUG:teuthology.orchestra.run.smithi038:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-09-21T03:32:29.031 DEBUG:teuthology.orchestra.run.smithi038:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-09-21T03:32:29.057 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-09-21T03:32:29.057 DEBUG:teuthology.orchestra.run.smithi195:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-09-21T03:32:29.072 DEBUG:teuthology.orchestra.run.smithi195:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-09-21T03:32:29.183 DEBUG:teuthology.orchestra.run.smithi195:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-09-21T03:32:29.211 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-09-21T03:32:29.211 INFO:tasks.ceph:config {'conf': {'client': {'pid file': ''}, 'global': {'ms die on skipped message': False}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'fs.*is degraded', 'filesystem is degraded', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'filesystem is offline', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'insufficient standby MDS daemons available', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'bad backtrace', 'object missing on disk', 'error reading table object', 'error reading sessionmap', 'Error loading MDS rank', 'missing journal object', 'Error recovering journal', 'error decoding table object', 'failed to read JournalPointer', 'Corrupt directory entry', 'Corrupt fnode header', 'corrupt sessionmap header', 'Corrupt dentry', 'Scrub error on inode', 'Metadata damage detected', 'MDS_READ_ONLY', 'force file system read-only', 'with standby daemon mds', 'MDS abort because newly corrupt dentry'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-09-21T03:32:29.211 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913473', 'branch': 'wip-pdonnell-testing-20240920.212106-debug', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/damage}', 'email': 'pdonnell@redhat.com', 'first_in_suite': False, 'job_id': '7913473', 'kernel': {'client': {'sha1': 'distro'}, 'kdb': 1, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi', 'no_nested_subset': False, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-pdonnell-testing-20240920.212106-debug'}, 'ceph': {'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'cluster-conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 2, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'client mount timeout': 30, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1, 'mon warn on pool no app': False}, 'mon': {'mon op complaint time': 120}, 'osd': {'osd op complaint time': 180}}, 'conf': {'client': {'pid file': ''}, 'global': {'ms die on skipped message': False}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'fs.*is degraded', 'filesystem is degraded', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'filesystem is offline', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'insufficient standby MDS daemons available', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'bad backtrace', 'object missing on disk', 'error reading table object', 'error reading sessionmap', 'Error loading MDS rank', 'missing journal object', 'Error recovering journal', 'error decoding table object', 'failed to read JournalPointer', 'Corrupt directory entry', 'Corrupt fnode header', 'corrupt sessionmap header', 'Corrupt dentry', 'Scrub error on inode', 'Metadata damage detected', 'MDS_READ_ONLY', 'force file system read-only', 'with standby daemon mds', 'MDS abort because newly corrupt dentry'], 'sha1': '1843221895adccfb3b692060251dd6a0183bc410'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '1843221895adccfb3b692060251dd6a0183bc410'}}, 'kclient': {'syntax': 'v1'}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-pdonnell-testing-20240920.212106-debug', 'sha1': '1843221895adccfb3b692060251dd6a0183bc410'}}, 'owner': 'scheduled_pdonnell@teuthology', 'priority': 90, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 7864, 'sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'sleep_before_teardown': 0, 'subset': '67/512', 'suite': 'fs', 'suite_branch': 'wip-pdonnell-testing-20240920.212106-debug', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_1843221895adccfb3b692060251dd6a0183bc410/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi038.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEMKrMeI/grdgdieR3gKFoSocAczE1VXEQqTtfimHxvz9zBS1jbD9PnWVgWpnmedtrXbAZCSFsEfPyOQlaFR4VM=', 'smithi195.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNPRj7Xa1rHyZCZzDGJgB4qh8J16e3iQN7219mvcU4gAAUE3aZuCQnL0NG9riL3CqFqAwuUw07eV+Y3YgPOYtnE='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'sha1': 'distro'}, 'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '1843221895adccfb3b692060251dd6a0183bc410'}}, {'ceph': {'create_rbd_pool': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'cluster-conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 2, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'client mount timeout': 30, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1, 'mon warn on pool no app': False}, 'mon': {'mon op complaint time': 120}, 'osd': {'osd op complaint time': 180}}, 'conf': {'client': {'pid file': ''}, 'global': {'ms die on skipped message': False}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'fs.*is degraded', 'filesystem is degraded', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'filesystem is offline', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'insufficient standby MDS daemons available', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'bad backtrace', 'object missing on disk', 'error reading table object', 'error reading sessionmap', 'Error loading MDS rank', 'missing journal object', 'Error recovering journal', 'error decoding table object', 'failed to read JournalPointer', 'Corrupt directory entry', 'Corrupt fnode header', 'corrupt sessionmap header', 'Corrupt dentry', 'Scrub error on inode', 'Metadata damage detected', 'MDS_READ_ONLY', 'force file system read-only', 'with standby daemon mds', 'MDS abort because newly corrupt dentry'], 'sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'cluster': 'ceph'}}, {'kclient': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_damage']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': '3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a', 'timestamp': '2024-09-20_23:48:22', 'tube': 'smithi', 'user': 'pdonnell', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2902'} 2024-09-21T03:32:29.212 DEBUG:teuthology.orchestra.run.smithi038:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-21T03:32:29.215 DEBUG:teuthology.orchestra.run.smithi195:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-21T03:32:29.263 DEBUG:teuthology.orchestra.run.smithi038:> sudo install -d -m0777 -- /var/run/ceph 2024-09-21T03:32:29.271 DEBUG:teuthology.orchestra.run.smithi195:> sudo install -d -m0777 -- /var/run/ceph 2024-09-21T03:32:29.328 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-21T03:32:29.328 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/scratch_devs of=/dev/stdout 2024-09-21T03:32:29.349 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-21T03:32:29.349 DEBUG:teuthology.orchestra.run.smithi038:> stat /dev/vg_nvme/lv_1 2024-09-21T03:32:29.405 INFO:teuthology.orchestra.run.smithi038.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-21T03:32:29.405 INFO:teuthology.orchestra.run.smithi038.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T03:32:29.405 INFO:teuthology.orchestra.run.smithi038.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-09-21T03:32:29.405 INFO:teuthology.orchestra.run.smithi038.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T03:32:29.405 INFO:teuthology.orchestra.run.smithi038.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T03:32:29.405 INFO:teuthology.orchestra.run.smithi038.stdout:Access: 2024-09-21 03:32:18.104769653 +0000 2024-09-21T03:32:29.406 INFO:teuthology.orchestra.run.smithi038.stdout:Modify: 2024-09-21 03:30:21.652561609 +0000 2024-09-21T03:32:29.406 INFO:teuthology.orchestra.run.smithi038.stdout:Change: 2024-09-21 03:30:21.652561609 +0000 2024-09-21T03:32:29.406 INFO:teuthology.orchestra.run.smithi038.stdout: Birth: 2024-09-21 03:30:21.652561609 +0000 2024-09-21T03:32:29.406 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-21T03:32:29.469 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-09-21T03:32:29.469 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-09-21T03:32:29.469 INFO:teuthology.orchestra.run.smithi038.stderr:512 bytes copied, 0.000287299 s, 1.8 MB/s 2024-09-21T03:32:29.470 DEBUG:teuthology.orchestra.run.smithi038:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-21T03:32:29.527 DEBUG:teuthology.orchestra.run.smithi038:> stat /dev/vg_nvme/lv_2 2024-09-21T03:32:29.583 INFO:teuthology.orchestra.run.smithi038.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-21T03:32:29.583 INFO:teuthology.orchestra.run.smithi038.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T03:32:29.583 INFO:teuthology.orchestra.run.smithi038.stdout:Device: 5h/5d Inode: 687 Links: 1 2024-09-21T03:32:29.583 INFO:teuthology.orchestra.run.smithi038.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T03:32:29.583 INFO:teuthology.orchestra.run.smithi038.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T03:32:29.583 INFO:teuthology.orchestra.run.smithi038.stdout:Access: 2024-09-21 03:32:18.104769653 +0000 2024-09-21T03:32:29.583 INFO:teuthology.orchestra.run.smithi038.stdout:Modify: 2024-09-21 03:30:21.986555411 +0000 2024-09-21T03:32:29.583 INFO:teuthology.orchestra.run.smithi038.stdout:Change: 2024-09-21 03:30:21.986555411 +0000 2024-09-21T03:32:29.583 INFO:teuthology.orchestra.run.smithi038.stdout: Birth: 2024-09-21 03:30:21.986555411 +0000 2024-09-21T03:32:29.584 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-21T03:32:29.647 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-09-21T03:32:29.647 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-09-21T03:32:29.647 INFO:teuthology.orchestra.run.smithi038.stderr:512 bytes copied, 0.000261807 s, 2.0 MB/s 2024-09-21T03:32:29.649 DEBUG:teuthology.orchestra.run.smithi038:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-21T03:32:29.705 DEBUG:teuthology.orchestra.run.smithi038:> stat /dev/vg_nvme/lv_3 2024-09-21T03:32:29.761 INFO:teuthology.orchestra.run.smithi038.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-21T03:32:29.761 INFO:teuthology.orchestra.run.smithi038.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T03:32:29.761 INFO:teuthology.orchestra.run.smithi038.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-09-21T03:32:29.761 INFO:teuthology.orchestra.run.smithi038.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T03:32:29.761 INFO:teuthology.orchestra.run.smithi038.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T03:32:29.761 INFO:teuthology.orchestra.run.smithi038.stdout:Access: 2024-09-21 03:32:18.105769635 +0000 2024-09-21T03:32:29.761 INFO:teuthology.orchestra.run.smithi038.stdout:Modify: 2024-09-21 03:30:22.318549251 +0000 2024-09-21T03:32:29.761 INFO:teuthology.orchestra.run.smithi038.stdout:Change: 2024-09-21 03:30:22.318549251 +0000 2024-09-21T03:32:29.762 INFO:teuthology.orchestra.run.smithi038.stdout: Birth: 2024-09-21 03:30:22.318549251 +0000 2024-09-21T03:32:29.762 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-21T03:32:29.825 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-09-21T03:32:29.825 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-09-21T03:32:29.825 INFO:teuthology.orchestra.run.smithi038.stderr:512 bytes copied, 0.000259606 s, 2.0 MB/s 2024-09-21T03:32:29.827 DEBUG:teuthology.orchestra.run.smithi038:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-21T03:32:29.883 DEBUG:teuthology.orchestra.run.smithi038:> stat /dev/vg_nvme/lv_4 2024-09-21T03:32:29.938 INFO:teuthology.orchestra.run.smithi038.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-21T03:32:29.938 INFO:teuthology.orchestra.run.smithi038.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T03:32:29.938 INFO:teuthology.orchestra.run.smithi038.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-09-21T03:32:29.939 INFO:teuthology.orchestra.run.smithi038.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T03:32:29.939 INFO:teuthology.orchestra.run.smithi038.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T03:32:29.939 INFO:teuthology.orchestra.run.smithi038.stdout:Access: 2024-09-21 03:32:18.105769635 +0000 2024-09-21T03:32:29.939 INFO:teuthology.orchestra.run.smithi038.stdout:Modify: 2024-09-21 03:30:22.640543276 +0000 2024-09-21T03:32:29.939 INFO:teuthology.orchestra.run.smithi038.stdout:Change: 2024-09-21 03:30:22.640543276 +0000 2024-09-21T03:32:29.939 INFO:teuthology.orchestra.run.smithi038.stdout: Birth: 2024-09-21 03:30:22.640543276 +0000 2024-09-21T03:32:29.939 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-21T03:32:30.002 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-09-21T03:32:30.002 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-09-21T03:32:30.002 INFO:teuthology.orchestra.run.smithi038.stderr:512 bytes copied, 0.000278399 s, 1.8 MB/s 2024-09-21T03:32:30.004 DEBUG:teuthology.orchestra.run.smithi038:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-21T03:32:30.060 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-21T03:32:30.060 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-09-21T03:32:30.060 DEBUG:teuthology.orchestra.run.smithi195:> dd if=/scratch_devs of=/dev/stdout 2024-09-21T03:32:30.077 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-21T03:32:30.077 DEBUG:teuthology.orchestra.run.smithi195:> stat /dev/vg_nvme/lv_1 2024-09-21T03:32:30.133 INFO:teuthology.orchestra.run.smithi195.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-21T03:32:30.133 INFO:teuthology.orchestra.run.smithi195.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T03:32:30.134 INFO:teuthology.orchestra.run.smithi195.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-09-21T03:32:30.134 INFO:teuthology.orchestra.run.smithi195.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T03:32:30.134 INFO:teuthology.orchestra.run.smithi195.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T03:32:30.134 INFO:teuthology.orchestra.run.smithi195.stdout:Access: 2024-09-21 03:32:11.936141758 +0000 2024-09-21T03:32:30.134 INFO:teuthology.orchestra.run.smithi195.stdout:Modify: 2024-09-21 03:30:16.874351075 +0000 2024-09-21T03:32:30.134 INFO:teuthology.orchestra.run.smithi195.stdout:Change: 2024-09-21 03:30:16.874351075 +0000 2024-09-21T03:32:30.134 INFO:teuthology.orchestra.run.smithi195.stdout: Birth: 2024-09-21 03:30:16.874351075 +0000 2024-09-21T03:32:30.134 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-21T03:32:30.198 INFO:teuthology.orchestra.run.smithi195.stderr:1+0 records in 2024-09-21T03:32:30.198 INFO:teuthology.orchestra.run.smithi195.stderr:1+0 records out 2024-09-21T03:32:30.198 INFO:teuthology.orchestra.run.smithi195.stderr:512 bytes copied, 0.000113007 s, 4.5 MB/s 2024-09-21T03:32:30.199 DEBUG:teuthology.orchestra.run.smithi195:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-21T03:32:30.256 DEBUG:teuthology.orchestra.run.smithi195:> stat /dev/vg_nvme/lv_2 2024-09-21T03:32:30.312 INFO:teuthology.orchestra.run.smithi195.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-21T03:32:30.312 INFO:teuthology.orchestra.run.smithi195.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T03:32:30.312 INFO:teuthology.orchestra.run.smithi195.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-09-21T03:32:30.312 INFO:teuthology.orchestra.run.smithi195.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T03:32:30.313 INFO:teuthology.orchestra.run.smithi195.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T03:32:30.313 INFO:teuthology.orchestra.run.smithi195.stdout:Access: 2024-09-21 03:32:11.937141739 +0000 2024-09-21T03:32:30.313 INFO:teuthology.orchestra.run.smithi195.stdout:Modify: 2024-09-21 03:30:17.284343103 +0000 2024-09-21T03:32:30.313 INFO:teuthology.orchestra.run.smithi195.stdout:Change: 2024-09-21 03:30:17.284343103 +0000 2024-09-21T03:32:30.313 INFO:teuthology.orchestra.run.smithi195.stdout: Birth: 2024-09-21 03:30:17.284343103 +0000 2024-09-21T03:32:30.313 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-21T03:32:30.376 INFO:teuthology.orchestra.run.smithi195.stderr:1+0 records in 2024-09-21T03:32:30.376 INFO:teuthology.orchestra.run.smithi195.stderr:1+0 records out 2024-09-21T03:32:30.376 INFO:teuthology.orchestra.run.smithi195.stderr:512 bytes copied, 0.000103484 s, 4.9 MB/s 2024-09-21T03:32:30.377 DEBUG:teuthology.orchestra.run.smithi195:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-21T03:32:30.434 DEBUG:teuthology.orchestra.run.smithi195:> stat /dev/vg_nvme/lv_3 2024-09-21T03:32:30.489 INFO:teuthology.orchestra.run.smithi195.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-21T03:32:30.489 INFO:teuthology.orchestra.run.smithi195.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T03:32:30.489 INFO:teuthology.orchestra.run.smithi195.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-09-21T03:32:30.489 INFO:teuthology.orchestra.run.smithi195.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T03:32:30.489 INFO:teuthology.orchestra.run.smithi195.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T03:32:30.489 INFO:teuthology.orchestra.run.smithi195.stdout:Access: 2024-09-21 03:32:11.937141739 +0000 2024-09-21T03:32:30.489 INFO:teuthology.orchestra.run.smithi195.stdout:Modify: 2024-09-21 03:30:17.592337115 +0000 2024-09-21T03:32:30.489 INFO:teuthology.orchestra.run.smithi195.stdout:Change: 2024-09-21 03:30:17.592337115 +0000 2024-09-21T03:32:30.489 INFO:teuthology.orchestra.run.smithi195.stdout: Birth: 2024-09-21 03:30:17.592337115 +0000 2024-09-21T03:32:30.489 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-21T03:32:30.552 INFO:teuthology.orchestra.run.smithi195.stderr:1+0 records in 2024-09-21T03:32:30.552 INFO:teuthology.orchestra.run.smithi195.stderr:1+0 records out 2024-09-21T03:32:30.552 INFO:teuthology.orchestra.run.smithi195.stderr:512 bytes copied, 0.000102088 s, 5.0 MB/s 2024-09-21T03:32:30.553 DEBUG:teuthology.orchestra.run.smithi195:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-21T03:32:30.610 DEBUG:teuthology.orchestra.run.smithi195:> stat /dev/vg_nvme/lv_4 2024-09-21T03:32:30.665 INFO:teuthology.orchestra.run.smithi195.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-21T03:32:30.666 INFO:teuthology.orchestra.run.smithi195.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T03:32:30.666 INFO:teuthology.orchestra.run.smithi195.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-09-21T03:32:30.666 INFO:teuthology.orchestra.run.smithi195.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T03:32:30.666 INFO:teuthology.orchestra.run.smithi195.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T03:32:30.666 INFO:teuthology.orchestra.run.smithi195.stdout:Access: 2024-09-21 03:32:11.937141739 +0000 2024-09-21T03:32:30.666 INFO:teuthology.orchestra.run.smithi195.stdout:Modify: 2024-09-21 03:30:17.916330816 +0000 2024-09-21T03:32:30.666 INFO:teuthology.orchestra.run.smithi195.stdout:Change: 2024-09-21 03:30:17.916330816 +0000 2024-09-21T03:32:30.666 INFO:teuthology.orchestra.run.smithi195.stdout: Birth: 2024-09-21 03:30:17.916330816 +0000 2024-09-21T03:32:30.666 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-21T03:32:30.728 INFO:teuthology.orchestra.run.smithi195.stderr:1+0 records in 2024-09-21T03:32:30.728 INFO:teuthology.orchestra.run.smithi195.stderr:1+0 records out 2024-09-21T03:32:30.729 INFO:teuthology.orchestra.run.smithi195.stderr:512 bytes copied, 0.000103719 s, 4.9 MB/s 2024-09-21T03:32:30.730 DEBUG:teuthology.orchestra.run.smithi195:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-21T03:32:30.786 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-21T03:32:30.786 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi038.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi195.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-21T03:32:30.786 INFO:tasks.ceph:Generating config... 2024-09-21T03:32:30.788 INFO:tasks.ceph:[client] pid file = 2024-09-21T03:32:30.788 INFO:tasks.ceph:[global] ms die on skipped message = False 2024-09-21T03:32:30.788 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-09-21T03:32:30.788 INFO:tasks.ceph:[mgr] debug ms = 1 2024-09-21T03:32:30.788 INFO:tasks.ceph:[mon] debug mon = 20 2024-09-21T03:32:30.788 INFO:tasks.ceph:[mon] debug ms = 1 2024-09-21T03:32:30.788 INFO:tasks.ceph:[mon] debug paxos = 20 2024-09-21T03:32:30.788 INFO:tasks.ceph:[mon] mon down mkfs grace = 300 2024-09-21T03:32:30.788 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-09-21T03:32:30.788 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-09-21T03:32:30.788 INFO:tasks.ceph:[osd] debug bluefs = 20 2024-09-21T03:32:30.788 INFO:tasks.ceph:[osd] debug bluestore = 20 2024-09-21T03:32:30.788 INFO:tasks.ceph:[osd] debug ms = 1 2024-09-21T03:32:30.788 INFO:tasks.ceph:[osd] debug osd = 20 2024-09-21T03:32:30.789 INFO:tasks.ceph:[osd] debug rocksdb = 10 2024-09-21T03:32:30.789 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-09-21T03:32:30.789 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-09-21T03:32:30.789 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-09-21T03:32:30.789 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-09-21T03:32:30.789 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-09-21T03:32:30.789 INFO:tasks.ceph:Setting up mon.a... 2024-09-21T03:32:30.789 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-09-21T03:32:30.858 INFO:teuthology.orchestra.run.smithi038.stdout:creating /etc/ceph/ceph.keyring 2024-09-21T03:32:30.861 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-09-21T03:32:30.918 DEBUG:teuthology.orchestra.run.smithi038:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-21T03:32:30.944 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.38'), ('mon.b', '172.21.15.195'), ('mon.c', '[v2:172.21.15.195:3301,v1:172.21.15.195:6790]')] 2024-09-21T03:32:30.944 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.38,172.21.15.195,[v2:172.21.15.195:3301,v1:172.21.15.195:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon down mkfs grace': 300}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'pid file': ''}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}} 2024-09-21T03:32:30.945 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-21T03:32:30.945 DEBUG:teuthology.orchestra.run.smithi038:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-09-21T03:32:31.001 DEBUG:teuthology.orchestra.run.smithi038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.38 --add b 172.21.15.195 --addv c '[v2:172.21.15.195:3301,v1:172.21.15.195:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-09-21T03:32:31.100 INFO:teuthology.orchestra.run.smithi038.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-09-21T03:32:31.100 INFO:teuthology.orchestra.run.smithi038.stdout:monmaptool: generated fsid 9d50d66f-656b-4fe2-8e3b-fe53b79c4d43 2024-09-21T03:32:31.100 INFO:teuthology.orchestra.run.smithi038.stdout:setting min_mon_release = quincy 2024-09-21T03:32:31.100 INFO:teuthology.orchestra.run.smithi038.stdout:epoch 0 2024-09-21T03:32:31.101 INFO:teuthology.orchestra.run.smithi038.stdout:fsid 9d50d66f-656b-4fe2-8e3b-fe53b79c4d43 2024-09-21T03:32:31.101 INFO:teuthology.orchestra.run.smithi038.stdout:last_changed 2024-09-21T03:32:31.099811+0000 2024-09-21T03:32:31.101 INFO:teuthology.orchestra.run.smithi038.stdout:created 2024-09-21T03:32:31.099811+0000 2024-09-21T03:32:31.101 INFO:teuthology.orchestra.run.smithi038.stdout:min_mon_release 17 (quincy) 2024-09-21T03:32:31.101 INFO:teuthology.orchestra.run.smithi038.stdout:election_strategy: 1 2024-09-21T03:32:31.101 INFO:teuthology.orchestra.run.smithi038.stdout:0: [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] mon.a 2024-09-21T03:32:31.101 INFO:teuthology.orchestra.run.smithi038.stdout:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-09-21T03:32:31.101 INFO:teuthology.orchestra.run.smithi038.stdout:2: [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] mon.c 2024-09-21T03:32:31.101 INFO:teuthology.orchestra.run.smithi038.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-09-21T03:32:31.103 DEBUG:teuthology.orchestra.run.smithi038:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-09-21T03:32:31.118 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 9d50d66f-656b-4fe2-8e3b-fe53b79c4d43... 2024-09-21T03:32:31.119 DEBUG:teuthology.orchestra.run.smithi038:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-21T03:32:31.161 DEBUG:teuthology.orchestra.run.smithi195:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-21T03:32:31.203 INFO:teuthology.orchestra.run.smithi038.stdout:[global] 2024-09-21T03:32:31.203 INFO:teuthology.orchestra.run.smithi038.stdout: chdir = "" 2024-09-21T03:32:31.203 INFO:teuthology.orchestra.run.smithi038.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-21T03:32:31.203 INFO:teuthology.orchestra.run.smithi038.stdout: auth supported = cephx 2024-09-21T03:32:31.203 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:32:31.203 INFO:teuthology.orchestra.run.smithi038.stdout: filestore xattr use omap = true 2024-09-21T03:32:31.203 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:32:31.203 INFO:teuthology.orchestra.run.smithi038.stdout: mon clock drift allowed = 1.000 2024-09-21T03:32:31.203 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:32:31.203 INFO:teuthology.orchestra.run.smithi038.stdout: osd crush chooseleaf type = 0 2024-09-21T03:32:31.203 INFO:teuthology.orchestra.run.smithi038.stdout: auth debug = true 2024-09-21T03:32:31.203 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:32:31.204 INFO:teuthology.orchestra.run.smithi038.stdout: ms die on old message = true 2024-09-21T03:32:31.204 INFO:teuthology.orchestra.run.smithi038.stdout: ms die on bug = true 2024-09-21T03:32:31.204 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:32:31.204 INFO:teuthology.orchestra.run.smithi038.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-21T03:32:31.204 INFO:teuthology.orchestra.run.smithi038.stdout: mon pg warn max object skew = 0 2024-09-21T03:32:31.204 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:32:31.204 INFO:teuthology.orchestra.run.smithi038.stdout: # disable pg_autoscaler by default for new pools 2024-09-21T03:32:31.204 INFO:teuthology.orchestra.run.smithi038.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-21T03:32:31.204 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:32:31.204 INFO:teuthology.orchestra.run.smithi038.stdout: osd pool default size = 2 2024-09-21T03:32:31.204 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:32:31.204 INFO:teuthology.orchestra.run.smithi038.stdout: mon osd allow primary affinity = true 2024-09-21T03:32:31.204 INFO:teuthology.orchestra.run.smithi038.stdout: mon osd allow pg remap = true 2024-09-21T03:32:31.204 INFO:teuthology.orchestra.run.smithi038.stdout: mon warn on legacy crush tunables = false 2024-09-21T03:32:31.204 INFO:teuthology.orchestra.run.smithi038.stdout: mon warn on crush straw calc version zero = false 2024-09-21T03:32:31.205 INFO:teuthology.orchestra.run.smithi038.stdout: mon warn on no sortbitwise = false 2024-09-21T03:32:31.205 INFO:teuthology.orchestra.run.smithi038.stdout: mon warn on osd down out interval zero = false 2024-09-21T03:32:31.205 INFO:teuthology.orchestra.run.smithi038.stdout: mon warn on too few osds = false 2024-09-21T03:32:31.205 INFO:teuthology.orchestra.run.smithi038.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-21T03:32:31.205 INFO:teuthology.orchestra.run.smithi038.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-21T03:32:31.205 INFO:teuthology.orchestra.run.smithi038.stdout: mon_allow_pool_size_one = true 2024-09-21T03:32:31.205 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:32:31.205 INFO:teuthology.orchestra.run.smithi038.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-21T03:32:31.205 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:32:31.205 INFO:teuthology.orchestra.run.smithi038.stdout: osd default data pool replay window = 5 2024-09-21T03:32:31.205 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:32:31.205 INFO:teuthology.orchestra.run.smithi038.stdout: mon allow pool delete = true 2024-09-21T03:32:31.205 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:32:31.205 INFO:teuthology.orchestra.run.smithi038.stdout: mon cluster log file level = debug 2024-09-21T03:32:31.205 INFO:teuthology.orchestra.run.smithi038.stdout: debug asserts on shutdown = true 2024-09-21T03:32:31.205 INFO:teuthology.orchestra.run.smithi038.stdout: mon health detail to clog = false 2024-09-21T03:32:31.206 INFO:teuthology.orchestra.run.smithi038.stdout: mon host = "172.21.15.38,172.21.15.195,[v2:172.21.15.195:3301,v1:172.21.15.195:6790]" 2024-09-21T03:32:31.206 INFO:teuthology.orchestra.run.smithi038.stdout: ms die on skipped message = False 2024-09-21T03:32:31.206 INFO:teuthology.orchestra.run.smithi038.stdout: fsid = 9d50d66f-656b-4fe2-8e3b-fe53b79c4d43 2024-09-21T03:32:31.206 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:32:31.206 INFO:teuthology.orchestra.run.smithi038.stdout:[osd] 2024-09-21T03:32:31.206 INFO:teuthology.orchestra.run.smithi038.stdout: osd journal size = 100 2024-09-21T03:32:31.206 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:32:31.206 INFO:teuthology.orchestra.run.smithi038.stdout: osd scrub load threshold = 5.0 2024-09-21T03:32:31.206 INFO:teuthology.orchestra.run.smithi038.stdout: osd scrub max interval = 600 2024-09-21T03:32:31.206 INFO:teuthology.orchestra.run.smithi038.stdout: osd mclock profile = high_recovery_ops 2024-09-21T03:32:31.206 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:32:31.206 INFO:teuthology.orchestra.run.smithi038.stdout: osd recover clone overlap = true 2024-09-21T03:32:31.206 INFO:teuthology.orchestra.run.smithi038.stdout: osd recovery max chunk = 1048576 2024-09-21T03:32:31.206 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:32:31.207 INFO:teuthology.orchestra.run.smithi038.stdout: osd debug shutdown = true 2024-09-21T03:32:31.207 INFO:teuthology.orchestra.run.smithi038.stdout: osd debug op order = true 2024-09-21T03:32:31.207 INFO:teuthology.orchestra.run.smithi038.stdout: osd debug verify stray on activate = true 2024-09-21T03:32:31.207 INFO:teuthology.orchestra.run.smithi038.stdout: osd debug trim objects = true 2024-09-21T03:32:31.207 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:32:31.207 INFO:teuthology.orchestra.run.smithi038.stdout: osd open classes on start = true 2024-09-21T03:32:31.207 INFO:teuthology.orchestra.run.smithi038.stdout: osd debug pg log writeout = true 2024-09-21T03:32:31.207 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:32:31.207 INFO:teuthology.orchestra.run.smithi038.stdout: osd deep scrub update digest min age = 30 2024-09-21T03:32:31.207 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:32:31.207 INFO:teuthology.orchestra.run.smithi038.stdout: osd map max advance = 10 2024-09-21T03:32:31.207 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:32:31.207 INFO:teuthology.orchestra.run.smithi038.stdout: journal zero on create = true 2024-09-21T03:32:31.207 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:32:31.207 INFO:teuthology.orchestra.run.smithi038.stdout: filestore ondisk finisher threads = 3 2024-09-21T03:32:31.207 INFO:teuthology.orchestra.run.smithi038.stdout: filestore apply finisher threads = 3 2024-09-21T03:32:31.208 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:32:31.208 INFO:teuthology.orchestra.run.smithi038.stdout: bdev debug aio = true 2024-09-21T03:32:31.208 INFO:teuthology.orchestra.run.smithi038.stdout: osd debug misdirected ops = true 2024-09-21T03:32:31.208 INFO:teuthology.orchestra.run.smithi038.stdout: bluestore block size = 96636764160 2024-09-21T03:32:31.208 INFO:teuthology.orchestra.run.smithi038.stdout: bluestore fsck on mount = True 2024-09-21T03:32:31.208 INFO:teuthology.orchestra.run.smithi038.stdout: debug bluefs = 20 2024-09-21T03:32:31.208 INFO:teuthology.orchestra.run.smithi038.stdout: debug bluestore = 20 2024-09-21T03:32:31.208 INFO:teuthology.orchestra.run.smithi038.stdout: debug ms = 1 2024-09-21T03:32:31.208 INFO:teuthology.orchestra.run.smithi038.stdout: debug osd = 20 2024-09-21T03:32:31.208 INFO:teuthology.orchestra.run.smithi038.stdout: debug rocksdb = 10 2024-09-21T03:32:31.208 INFO:teuthology.orchestra.run.smithi038.stdout: mon osd backfillfull_ratio = 0.85 2024-09-21T03:32:31.208 INFO:teuthology.orchestra.run.smithi038.stdout: mon osd full ratio = 0.9 2024-09-21T03:32:31.208 INFO:teuthology.orchestra.run.smithi038.stdout: mon osd nearfull ratio = 0.8 2024-09-21T03:32:31.208 INFO:teuthology.orchestra.run.smithi038.stdout: osd failsafe full ratio = 0.95 2024-09-21T03:32:31.208 INFO:teuthology.orchestra.run.smithi038.stdout: osd objectstore = bluestore 2024-09-21T03:32:31.208 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:32:31.209 INFO:teuthology.orchestra.run.smithi038.stdout:[mgr] 2024-09-21T03:32:31.209 INFO:teuthology.orchestra.run.smithi038.stdout: debug ms = 1 2024-09-21T03:32:31.209 INFO:teuthology.orchestra.run.smithi038.stdout: debug mgr = 20 2024-09-21T03:32:31.209 INFO:teuthology.orchestra.run.smithi038.stdout: debug mon = 20 2024-09-21T03:32:31.209 INFO:teuthology.orchestra.run.smithi038.stdout: debug auth = 20 2024-09-21T03:32:31.209 INFO:teuthology.orchestra.run.smithi038.stdout: mon reweight min pgs per osd = 4 2024-09-21T03:32:31.209 INFO:teuthology.orchestra.run.smithi038.stdout: mon reweight min bytes per osd = 10 2024-09-21T03:32:31.209 INFO:teuthology.orchestra.run.smithi038.stdout: mgr/telemetry/nag = false 2024-09-21T03:32:31.209 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:32:31.209 INFO:teuthology.orchestra.run.smithi038.stdout:[mon] 2024-09-21T03:32:31.209 INFO:teuthology.orchestra.run.smithi038.stdout: debug ms = 1 2024-09-21T03:32:31.209 INFO:teuthology.orchestra.run.smithi038.stdout: debug mon = 20 2024-09-21T03:32:31.209 INFO:teuthology.orchestra.run.smithi038.stdout: debug paxos = 20 2024-09-21T03:32:31.209 INFO:teuthology.orchestra.run.smithi038.stdout: debug auth = 20 2024-09-21T03:32:31.209 INFO:teuthology.orchestra.run.smithi038.stdout: mon data avail warn = 5 2024-09-21T03:32:31.209 INFO:teuthology.orchestra.run.smithi038.stdout: mon mgr mkfs grace = 240 2024-09-21T03:32:31.210 INFO:teuthology.orchestra.run.smithi038.stdout: mon reweight min pgs per osd = 4 2024-09-21T03:32:31.210 INFO:teuthology.orchestra.run.smithi038.stdout: mon osd reporter subtree level = osd 2024-09-21T03:32:31.210 INFO:teuthology.orchestra.run.smithi038.stdout: mon osd prime pg temp = true 2024-09-21T03:32:31.210 INFO:teuthology.orchestra.run.smithi038.stdout: mon reweight min bytes per osd = 10 2024-09-21T03:32:31.210 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:32:31.210 INFO:teuthology.orchestra.run.smithi038.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-21T03:32:31.210 INFO:teuthology.orchestra.run.smithi038.stdout: auth mon ticket ttl = 660 # 11m 2024-09-21T03:32:31.210 INFO:teuthology.orchestra.run.smithi038.stdout: auth service ticket ttl = 240 # 4m 2024-09-21T03:32:31.210 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:32:31.210 INFO:teuthology.orchestra.run.smithi038.stdout: # don't complain about insecure global_id in the test suite 2024-09-21T03:32:31.210 INFO:teuthology.orchestra.run.smithi038.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-21T03:32:31.210 INFO:teuthology.orchestra.run.smithi038.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-21T03:32:31.210 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:32:31.210 INFO:teuthology.orchestra.run.smithi038.stdout: # 1m isn't quite enough 2024-09-21T03:32:31.210 INFO:teuthology.orchestra.run.smithi038.stdout: mon_down_mkfs_grace = 2m 2024-09-21T03:32:31.211 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:32:31.211 INFO:teuthology.orchestra.run.smithi038.stdout: mon_warn_on_filestore_osds = false 2024-09-21T03:32:31.211 INFO:teuthology.orchestra.run.smithi038.stdout: mon down mkfs grace = 300 2024-09-21T03:32:31.211 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:32:31.211 INFO:teuthology.orchestra.run.smithi038.stdout:[client] 2024-09-21T03:32:31.211 INFO:teuthology.orchestra.run.smithi038.stdout: rgw cache enabled = true 2024-09-21T03:32:31.211 INFO:teuthology.orchestra.run.smithi038.stdout: rgw enable ops log = true 2024-09-21T03:32:31.211 INFO:teuthology.orchestra.run.smithi038.stdout: rgw enable usage log = true 2024-09-21T03:32:31.211 INFO:teuthology.orchestra.run.smithi038.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-21T03:32:31.211 INFO:teuthology.orchestra.run.smithi038.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-21T03:32:31.211 INFO:teuthology.orchestra.run.smithi038.stdout: pid file = "" 2024-09-21T03:32:31.211 INFO:teuthology.orchestra.run.smithi038.stdout:[mon.a] 2024-09-21T03:32:31.211 INFO:teuthology.orchestra.run.smithi038.stdout:[mon.b] 2024-09-21T03:32:31.211 INFO:teuthology.orchestra.run.smithi038.stdout:[mon.c] 2024-09-21T03:32:31.211 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.a] 2024-09-21T03:32:31.211 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.c] 2024-09-21T03:32:31.212 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.b] 2024-09-21T03:32:31.212 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.d] 2024-09-21T03:32:31.212 INFO:teuthology.orchestra.run.smithi195.stdout:[global] 2024-09-21T03:32:31.212 INFO:teuthology.orchestra.run.smithi195.stdout: chdir = "" 2024-09-21T03:32:31.212 INFO:teuthology.orchestra.run.smithi195.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-21T03:32:31.212 INFO:teuthology.orchestra.run.smithi195.stdout: auth supported = cephx 2024-09-21T03:32:31.212 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-21T03:32:31.212 INFO:teuthology.orchestra.run.smithi195.stdout: filestore xattr use omap = true 2024-09-21T03:32:31.212 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-21T03:32:31.212 INFO:teuthology.orchestra.run.smithi195.stdout: mon clock drift allowed = 1.000 2024-09-21T03:32:31.212 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-21T03:32:31.212 INFO:teuthology.orchestra.run.smithi195.stdout: osd crush chooseleaf type = 0 2024-09-21T03:32:31.213 INFO:teuthology.orchestra.run.smithi195.stdout: auth debug = true 2024-09-21T03:32:31.213 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-21T03:32:31.213 INFO:teuthology.orchestra.run.smithi195.stdout: ms die on old message = true 2024-09-21T03:32:31.213 INFO:teuthology.orchestra.run.smithi195.stdout: ms die on bug = true 2024-09-21T03:32:31.213 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-21T03:32:31.213 INFO:teuthology.orchestra.run.smithi195.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-21T03:32:31.213 INFO:teuthology.orchestra.run.smithi195.stdout: mon pg warn max object skew = 0 2024-09-21T03:32:31.213 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-21T03:32:31.213 INFO:teuthology.orchestra.run.smithi195.stdout: # disable pg_autoscaler by default for new pools 2024-09-21T03:32:31.213 INFO:teuthology.orchestra.run.smithi195.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-21T03:32:31.213 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-21T03:32:31.213 INFO:teuthology.orchestra.run.smithi195.stdout: osd pool default size = 2 2024-09-21T03:32:31.213 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-21T03:32:31.214 INFO:teuthology.orchestra.run.smithi195.stdout: mon osd allow primary affinity = true 2024-09-21T03:32:31.214 INFO:teuthology.orchestra.run.smithi195.stdout: mon osd allow pg remap = true 2024-09-21T03:32:31.214 INFO:teuthology.orchestra.run.smithi195.stdout: mon warn on legacy crush tunables = false 2024-09-21T03:32:31.214 INFO:teuthology.orchestra.run.smithi195.stdout: mon warn on crush straw calc version zero = false 2024-09-21T03:32:31.214 INFO:teuthology.orchestra.run.smithi195.stdout: mon warn on no sortbitwise = false 2024-09-21T03:32:31.214 INFO:teuthology.orchestra.run.smithi195.stdout: mon warn on osd down out interval zero = false 2024-09-21T03:32:31.214 INFO:teuthology.orchestra.run.smithi195.stdout: mon warn on too few osds = false 2024-09-21T03:32:31.214 INFO:teuthology.orchestra.run.smithi195.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-21T03:32:31.214 INFO:teuthology.orchestra.run.smithi195.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-21T03:32:31.214 INFO:teuthology.orchestra.run.smithi195.stdout: mon_allow_pool_size_one = true 2024-09-21T03:32:31.214 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-21T03:32:31.214 INFO:teuthology.orchestra.run.smithi195.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-21T03:32:31.214 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-21T03:32:31.214 INFO:teuthology.orchestra.run.smithi195.stdout: osd default data pool replay window = 5 2024-09-21T03:32:31.215 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-21T03:32:31.215 INFO:teuthology.orchestra.run.smithi195.stdout: mon allow pool delete = true 2024-09-21T03:32:31.215 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-21T03:32:31.215 INFO:teuthology.orchestra.run.smithi195.stdout: mon cluster log file level = debug 2024-09-21T03:32:31.215 INFO:teuthology.orchestra.run.smithi195.stdout: debug asserts on shutdown = true 2024-09-21T03:32:31.215 INFO:teuthology.orchestra.run.smithi195.stdout: mon health detail to clog = false 2024-09-21T03:32:31.215 INFO:teuthology.orchestra.run.smithi195.stdout: mon host = "172.21.15.38,172.21.15.195,[v2:172.21.15.195:3301,v1:172.21.15.195:6790]" 2024-09-21T03:32:31.215 INFO:teuthology.orchestra.run.smithi195.stdout: ms die on skipped message = False 2024-09-21T03:32:31.215 INFO:teuthology.orchestra.run.smithi195.stdout: fsid = 9d50d66f-656b-4fe2-8e3b-fe53b79c4d43 2024-09-21T03:32:31.215 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-21T03:32:31.215 INFO:teuthology.orchestra.run.smithi195.stdout:[osd] 2024-09-21T03:32:31.215 INFO:teuthology.orchestra.run.smithi195.stdout: osd journal size = 100 2024-09-21T03:32:31.215 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-21T03:32:31.215 INFO:teuthology.orchestra.run.smithi195.stdout: osd scrub load threshold = 5.0 2024-09-21T03:32:31.216 INFO:teuthology.orchestra.run.smithi195.stdout: osd scrub max interval = 600 2024-09-21T03:32:31.216 INFO:teuthology.orchestra.run.smithi195.stdout: osd mclock profile = high_recovery_ops 2024-09-21T03:32:31.216 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-21T03:32:31.216 INFO:teuthology.orchestra.run.smithi195.stdout: osd recover clone overlap = true 2024-09-21T03:32:31.216 INFO:teuthology.orchestra.run.smithi195.stdout: osd recovery max chunk = 1048576 2024-09-21T03:32:31.216 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-21T03:32:31.216 INFO:teuthology.orchestra.run.smithi195.stdout: osd debug shutdown = true 2024-09-21T03:32:31.216 INFO:teuthology.orchestra.run.smithi195.stdout: osd debug op order = true 2024-09-21T03:32:31.216 INFO:teuthology.orchestra.run.smithi195.stdout: osd debug verify stray on activate = true 2024-09-21T03:32:31.216 INFO:teuthology.orchestra.run.smithi195.stdout: osd debug trim objects = true 2024-09-21T03:32:31.216 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-21T03:32:31.216 INFO:teuthology.orchestra.run.smithi195.stdout: osd open classes on start = true 2024-09-21T03:32:31.216 INFO:teuthology.orchestra.run.smithi195.stdout: osd debug pg log writeout = true 2024-09-21T03:32:31.216 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-21T03:32:31.216 INFO:teuthology.orchestra.run.smithi195.stdout: osd deep scrub update digest min age = 30 2024-09-21T03:32:31.217 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-21T03:32:31.217 INFO:teuthology.orchestra.run.smithi195.stdout: osd map max advance = 10 2024-09-21T03:32:31.217 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-21T03:32:31.217 INFO:teuthology.orchestra.run.smithi195.stdout: journal zero on create = true 2024-09-21T03:32:31.217 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-21T03:32:31.217 INFO:teuthology.orchestra.run.smithi195.stdout: filestore ondisk finisher threads = 3 2024-09-21T03:32:31.217 INFO:teuthology.orchestra.run.smithi195.stdout: filestore apply finisher threads = 3 2024-09-21T03:32:31.217 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-21T03:32:31.217 INFO:teuthology.orchestra.run.smithi195.stdout: bdev debug aio = true 2024-09-21T03:32:31.217 INFO:teuthology.orchestra.run.smithi195.stdout: osd debug misdirected ops = true 2024-09-21T03:32:31.217 INFO:teuthology.orchestra.run.smithi195.stdout: bluestore block size = 96636764160 2024-09-21T03:32:31.217 INFO:teuthology.orchestra.run.smithi195.stdout: bluestore fsck on mount = True 2024-09-21T03:32:31.217 INFO:teuthology.orchestra.run.smithi195.stdout: debug bluefs = 20 2024-09-21T03:32:31.217 INFO:teuthology.orchestra.run.smithi195.stdout: debug bluestore = 20 2024-09-21T03:32:31.217 INFO:teuthology.orchestra.run.smithi195.stdout: debug ms = 1 2024-09-21T03:32:31.218 INFO:teuthology.orchestra.run.smithi195.stdout: debug osd = 20 2024-09-21T03:32:31.218 INFO:teuthology.orchestra.run.smithi195.stdout: debug rocksdb = 10 2024-09-21T03:32:31.218 INFO:teuthology.orchestra.run.smithi195.stdout: mon osd backfillfull_ratio = 0.85 2024-09-21T03:32:31.218 INFO:teuthology.orchestra.run.smithi195.stdout: mon osd full ratio = 0.9 2024-09-21T03:32:31.218 INFO:teuthology.orchestra.run.smithi195.stdout: mon osd nearfull ratio = 0.8 2024-09-21T03:32:31.218 INFO:teuthology.orchestra.run.smithi195.stdout: osd failsafe full ratio = 0.95 2024-09-21T03:32:31.218 INFO:teuthology.orchestra.run.smithi195.stdout: osd objectstore = bluestore 2024-09-21T03:32:31.218 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-21T03:32:31.218 INFO:teuthology.orchestra.run.smithi195.stdout:[mgr] 2024-09-21T03:32:31.218 INFO:teuthology.orchestra.run.smithi195.stdout: debug ms = 1 2024-09-21T03:32:31.218 INFO:teuthology.orchestra.run.smithi195.stdout: debug mgr = 20 2024-09-21T03:32:31.218 INFO:teuthology.orchestra.run.smithi195.stdout: debug mon = 20 2024-09-21T03:32:31.218 INFO:teuthology.orchestra.run.smithi195.stdout: debug auth = 20 2024-09-21T03:32:31.218 INFO:teuthology.orchestra.run.smithi195.stdout: mon reweight min pgs per osd = 4 2024-09-21T03:32:31.219 INFO:teuthology.orchestra.run.smithi195.stdout: mon reweight min bytes per osd = 10 2024-09-21T03:32:31.219 INFO:teuthology.orchestra.run.smithi195.stdout: mgr/telemetry/nag = false 2024-09-21T03:32:31.219 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-21T03:32:31.219 INFO:teuthology.orchestra.run.smithi195.stdout:[mon] 2024-09-21T03:32:31.219 INFO:teuthology.orchestra.run.smithi195.stdout: debug ms = 1 2024-09-21T03:32:31.219 INFO:teuthology.orchestra.run.smithi195.stdout: debug mon = 20 2024-09-21T03:32:31.219 INFO:teuthology.orchestra.run.smithi195.stdout: debug paxos = 20 2024-09-21T03:32:31.219 INFO:teuthology.orchestra.run.smithi195.stdout: debug auth = 20 2024-09-21T03:32:31.219 INFO:teuthology.orchestra.run.smithi195.stdout: mon data avail warn = 5 2024-09-21T03:32:31.219 INFO:teuthology.orchestra.run.smithi195.stdout: mon mgr mkfs grace = 240 2024-09-21T03:32:31.219 INFO:teuthology.orchestra.run.smithi195.stdout: mon reweight min pgs per osd = 4 2024-09-21T03:32:31.219 INFO:teuthology.orchestra.run.smithi195.stdout: mon osd reporter subtree level = osd 2024-09-21T03:32:31.219 INFO:teuthology.orchestra.run.smithi195.stdout: mon osd prime pg temp = true 2024-09-21T03:32:31.219 INFO:teuthology.orchestra.run.smithi195.stdout: mon reweight min bytes per osd = 10 2024-09-21T03:32:31.220 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-21T03:32:31.220 INFO:teuthology.orchestra.run.smithi195.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-21T03:32:31.220 INFO:teuthology.orchestra.run.smithi195.stdout: auth mon ticket ttl = 660 # 11m 2024-09-21T03:32:31.220 INFO:teuthology.orchestra.run.smithi195.stdout: auth service ticket ttl = 240 # 4m 2024-09-21T03:32:31.220 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-21T03:32:31.220 INFO:teuthology.orchestra.run.smithi195.stdout: # don't complain about insecure global_id in the test suite 2024-09-21T03:32:31.220 INFO:teuthology.orchestra.run.smithi195.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-21T03:32:31.220 INFO:teuthology.orchestra.run.smithi195.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-21T03:32:31.220 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-21T03:32:31.220 INFO:teuthology.orchestra.run.smithi195.stdout: # 1m isn't quite enough 2024-09-21T03:32:31.220 INFO:teuthology.orchestra.run.smithi195.stdout: mon_down_mkfs_grace = 2m 2024-09-21T03:32:31.220 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-21T03:32:31.220 INFO:teuthology.orchestra.run.smithi195.stdout: mon_warn_on_filestore_osds = false 2024-09-21T03:32:31.220 INFO:teuthology.orchestra.run.smithi195.stdout: mon down mkfs grace = 300 2024-09-21T03:32:31.221 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-21T03:32:31.221 INFO:teuthology.orchestra.run.smithi195.stdout:[client] 2024-09-21T03:32:31.221 INFO:teuthology.orchestra.run.smithi195.stdout: rgw cache enabled = true 2024-09-21T03:32:31.221 INFO:teuthology.orchestra.run.smithi195.stdout: rgw enable ops log = true 2024-09-21T03:32:31.221 INFO:teuthology.orchestra.run.smithi195.stdout: rgw enable usage log = true 2024-09-21T03:32:31.221 INFO:teuthology.orchestra.run.smithi195.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-21T03:32:31.221 INFO:teuthology.orchestra.run.smithi195.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-21T03:32:31.221 INFO:teuthology.orchestra.run.smithi195.stdout: pid file = "" 2024-09-21T03:32:31.221 INFO:teuthology.orchestra.run.smithi195.stdout:[mon.a] 2024-09-21T03:32:31.221 INFO:teuthology.orchestra.run.smithi195.stdout:[mon.b] 2024-09-21T03:32:31.221 INFO:teuthology.orchestra.run.smithi195.stdout:[mon.c] 2024-09-21T03:32:31.221 INFO:teuthology.orchestra.run.smithi195.stdout:[mds.a] 2024-09-21T03:32:31.221 INFO:teuthology.orchestra.run.smithi195.stdout:[mds.c] 2024-09-21T03:32:31.221 INFO:teuthology.orchestra.run.smithi195.stdout:[mds.b] 2024-09-21T03:32:31.221 INFO:teuthology.orchestra.run.smithi195.stdout:[mds.d] 2024-09-21T03:32:31.223 INFO:tasks.ceph:Creating admin key on mon.a... 2024-09-21T03:32:31.223 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-09-21T03:32:31.329 INFO:tasks.ceph:Copying monmap to all nodes... 2024-09-21T03:32:31.329 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-21T03:32:31.329 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-09-21T03:32:31.344 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-21T03:32:31.344 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-09-21T03:32:31.400 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi038.front.sepia.ceph.com 2024-09-21T03:32:31.400 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-21T03:32:31.400 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-21T03:32:31.400 DEBUG:teuthology.orchestra.run.smithi038:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-21T03:32:31.476 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-21T03:32:31.477 DEBUG:teuthology.orchestra.run.smithi038:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-21T03:32:31.532 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi195.front.sepia.ceph.com 2024-09-21T03:32:31.532 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-09-21T03:32:31.532 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-21T03:32:31.532 DEBUG:teuthology.orchestra.run.smithi195:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-21T03:32:31.568 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-09-21T03:32:31.568 DEBUG:teuthology.orchestra.run.smithi195:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-21T03:32:31.625 INFO:tasks.ceph:Setting up mon nodes... 2024-09-21T03:32:31.625 INFO:tasks.ceph:Setting up mgr nodes... 2024-09-21T03:32:31.625 DEBUG:teuthology.orchestra.run.smithi038:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-09-21T03:32:31.688 INFO:teuthology.orchestra.run.smithi038.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-09-21T03:32:31.691 DEBUG:teuthology.orchestra.run.smithi195:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-09-21T03:32:31.766 INFO:teuthology.orchestra.run.smithi195.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-09-21T03:32:31.769 INFO:tasks.ceph:Setting up mds nodes... 2024-09-21T03:32:31.769 DEBUG:teuthology.orchestra.run.smithi038:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-09-21T03:32:31.831 INFO:teuthology.orchestra.run.smithi038.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-09-21T03:32:31.834 DEBUG:teuthology.orchestra.run.smithi038:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-09-21T03:32:31.859 DEBUG:teuthology.orchestra.run.smithi038:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-09-21T03:32:31.961 INFO:teuthology.orchestra.run.smithi038.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-09-21T03:32:31.964 DEBUG:teuthology.orchestra.run.smithi038:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-09-21T03:32:31.989 DEBUG:teuthology.orchestra.run.smithi195:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-09-21T03:32:32.049 INFO:teuthology.orchestra.run.smithi195.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-09-21T03:32:32.052 DEBUG:teuthology.orchestra.run.smithi195:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-09-21T03:32:32.077 DEBUG:teuthology.orchestra.run.smithi195:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-09-21T03:32:32.177 INFO:teuthology.orchestra.run.smithi195.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-09-21T03:32:32.180 DEBUG:teuthology.orchestra.run.smithi195:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-09-21T03:32:32.204 INFO:tasks.ceph_client:Setting up client nodes... 2024-09-21T03:32:32.204 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-21T03:32:32.256 INFO:teuthology.orchestra.run.smithi038.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-09-21T03:32:32.270 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-09-21T03:32:32.321 INFO:teuthology.orchestra.run.smithi038.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-09-21T03:32:32.335 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-09-21T03:32:32.385 INFO:teuthology.orchestra.run.smithi195.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-09-21T03:32:32.399 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-09-21T03:32:32.448 INFO:teuthology.orchestra.run.smithi195.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-09-21T03:32:32.461 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-09-21T03:32:32.462 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi038.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi195.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-21T03:32:32.462 DEBUG:teuthology.orchestra.run.smithi038:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-09-21T03:32:32.487 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-21T03:32:32.487 INFO:tasks.ceph:role: osd.0 2024-09-21T03:32:32.487 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi038.front.sepia.ceph.com 2024-09-21T03:32:32.487 DEBUG:teuthology.orchestra.run.smithi038:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-21T03:32:32.581 INFO:teuthology.orchestra.run.smithi038.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-21T03:32:32.581 INFO:teuthology.orchestra.run.smithi038.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-21T03:32:32.581 INFO:teuthology.orchestra.run.smithi038.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-21T03:32:32.582 INFO:teuthology.orchestra.run.smithi038.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-21T03:32:32.582 INFO:teuthology.orchestra.run.smithi038.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-21T03:32:32.582 INFO:teuthology.orchestra.run.smithi038.stdout: = sunit=0 swidth=0 blks 2024-09-21T03:32:32.582 INFO:teuthology.orchestra.run.smithi038.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-21T03:32:32.582 INFO:teuthology.orchestra.run.smithi038.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-21T03:32:32.582 INFO:teuthology.orchestra.run.smithi038.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-21T03:32:32.582 INFO:teuthology.orchestra.run.smithi038.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-21T03:32:33.928 INFO:teuthology.orchestra.run.smithi038.stdout:Discarding blocks...Done. 2024-09-21T03:32:33.930 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi038.front.sepia.ceph.com -o noatime 2024-09-21T03:32:33.931 DEBUG:teuthology.orchestra.run.smithi038:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-09-21T03:32:33.967 DEBUG:teuthology.orchestra.run.smithi038:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-09-21T03:32:34.039 DEBUG:teuthology.orchestra.run.smithi038:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-09-21T03:32:34.107 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-21T03:32:34.107 INFO:tasks.ceph:role: osd.1 2024-09-21T03:32:34.107 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi038.front.sepia.ceph.com 2024-09-21T03:32:34.107 DEBUG:teuthology.orchestra.run.smithi038:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-21T03:32:34.202 INFO:teuthology.orchestra.run.smithi038.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-21T03:32:34.202 INFO:teuthology.orchestra.run.smithi038.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-21T03:32:34.202 INFO:teuthology.orchestra.run.smithi038.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-21T03:32:34.202 INFO:teuthology.orchestra.run.smithi038.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-21T03:32:34.203 INFO:teuthology.orchestra.run.smithi038.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-21T03:32:34.203 INFO:teuthology.orchestra.run.smithi038.stdout: = sunit=0 swidth=0 blks 2024-09-21T03:32:34.203 INFO:teuthology.orchestra.run.smithi038.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-21T03:32:34.203 INFO:teuthology.orchestra.run.smithi038.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-21T03:32:34.203 INFO:teuthology.orchestra.run.smithi038.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-21T03:32:34.203 INFO:teuthology.orchestra.run.smithi038.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-21T03:32:35.552 INFO:teuthology.orchestra.run.smithi038.stdout:Discarding blocks...Done. 2024-09-21T03:32:35.554 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi038.front.sepia.ceph.com -o noatime 2024-09-21T03:32:35.554 DEBUG:teuthology.orchestra.run.smithi038:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-09-21T03:32:35.586 DEBUG:teuthology.orchestra.run.smithi038:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-09-21T03:32:35.657 DEBUG:teuthology.orchestra.run.smithi038:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-09-21T03:32:35.723 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-21T03:32:35.846 INFO:tasks.ceph:role: osd.2 2024-09-21T03:32:35.846 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi038.front.sepia.ceph.com 2024-09-21T03:32:35.846 DEBUG:teuthology.orchestra.run.smithi038:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-21T03:32:35.902 INFO:teuthology.orchestra.run.smithi038.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-21T03:32:35.902 INFO:teuthology.orchestra.run.smithi038.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-21T03:32:35.902 INFO:teuthology.orchestra.run.smithi038.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-21T03:32:35.902 INFO:teuthology.orchestra.run.smithi038.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-21T03:32:35.902 INFO:teuthology.orchestra.run.smithi038.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-21T03:32:35.902 INFO:teuthology.orchestra.run.smithi038.stdout: = sunit=0 swidth=0 blks 2024-09-21T03:32:35.902 INFO:teuthology.orchestra.run.smithi038.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-21T03:32:35.902 INFO:teuthology.orchestra.run.smithi038.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-21T03:32:35.903 INFO:teuthology.orchestra.run.smithi038.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-21T03:32:35.903 INFO:teuthology.orchestra.run.smithi038.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-21T03:32:37.218 INFO:teuthology.orchestra.run.smithi038.stdout:Discarding blocks...Done. 2024-09-21T03:32:37.221 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi038.front.sepia.ceph.com -o noatime 2024-09-21T03:32:37.221 DEBUG:teuthology.orchestra.run.smithi038:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-09-21T03:32:37.257 DEBUG:teuthology.orchestra.run.smithi038:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-09-21T03:32:37.331 DEBUG:teuthology.orchestra.run.smithi038:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-09-21T03:32:37.400 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-21T03:32:37.400 INFO:tasks.ceph:role: osd.3 2024-09-21T03:32:37.400 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi038.front.sepia.ceph.com 2024-09-21T03:32:37.400 DEBUG:teuthology.orchestra.run.smithi038:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-21T03:32:37.496 INFO:teuthology.orchestra.run.smithi038.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-21T03:32:37.496 INFO:teuthology.orchestra.run.smithi038.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-21T03:32:37.496 INFO:teuthology.orchestra.run.smithi038.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-21T03:32:37.496 INFO:teuthology.orchestra.run.smithi038.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-21T03:32:37.496 INFO:teuthology.orchestra.run.smithi038.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-21T03:32:37.496 INFO:teuthology.orchestra.run.smithi038.stdout: = sunit=0 swidth=0 blks 2024-09-21T03:32:37.497 INFO:teuthology.orchestra.run.smithi038.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-21T03:32:37.497 INFO:teuthology.orchestra.run.smithi038.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-21T03:32:37.497 INFO:teuthology.orchestra.run.smithi038.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-21T03:32:37.497 INFO:teuthology.orchestra.run.smithi038.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-21T03:32:38.812 INFO:teuthology.orchestra.run.smithi038.stdout:Discarding blocks...Done. 2024-09-21T03:32:38.814 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi038.front.sepia.ceph.com -o noatime 2024-09-21T03:32:38.815 DEBUG:teuthology.orchestra.run.smithi038:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-09-21T03:32:38.851 DEBUG:teuthology.orchestra.run.smithi038:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-09-21T03:32:38.925 DEBUG:teuthology.orchestra.run.smithi038:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-21T03:32:39.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:32:39.034+0000 7fc01e80e640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-09-21T03:32:39.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:32:39.035+0000 7fc01e80e640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-09-21T03:32:39.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:32:39.035+0000 7fc01e80e640 -1 bdev(0x5587c7159900 /var/lib/ceph/osd/ceph-0/block) open stat got: (1) Operation not permitted 2024-09-21T03:32:39.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:32:39.035+0000 7fc01e80e640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-09-21T03:32:42.448 DEBUG:teuthology.orchestra.run.smithi038:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-21T03:32:42.479 DEBUG:teuthology.orchestra.run.smithi038:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-21T03:32:42.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:32:42.582+0000 7fb8a9a1f640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-09-21T03:32:42.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:32:42.582+0000 7fb8a9a1f640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-09-21T03:32:42.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:32:42.582+0000 7fb8a9a1f640 -1 bdev(0x564cf305d900 /var/lib/ceph/osd/ceph-1/block) open stat got: (1) Operation not permitted 2024-09-21T03:32:42.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:32:42.583+0000 7fb8a9a1f640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-09-21T03:32:46.003 DEBUG:teuthology.orchestra.run.smithi038:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-21T03:32:46.034 DEBUG:teuthology.orchestra.run.smithi038:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-21T03:32:46.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:32:46.139+0000 7ff86f4d6640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-09-21T03:32:46.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:32:46.140+0000 7ff86f4d6640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-09-21T03:32:46.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:32:46.140+0000 7ff86f4d6640 -1 bdev(0x55ff5d6e9900 /var/lib/ceph/osd/ceph-2/block) open stat got: (1) Operation not permitted 2024-09-21T03:32:46.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:32:46.140+0000 7ff86f4d6640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-09-21T03:32:49.550 DEBUG:teuthology.orchestra.run.smithi038:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-21T03:32:49.581 DEBUG:teuthology.orchestra.run.smithi038:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-21T03:32:49.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:32:49.684+0000 7fc514c14640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-09-21T03:32:49.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:32:49.685+0000 7fc514c14640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-09-21T03:32:49.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:32:49.685+0000 7fc514c14640 -1 bdev(0x5596b5361900 /var/lib/ceph/osd/ceph-3/block) open stat got: (1) Operation not permitted 2024-09-21T03:32:49.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:32:49.685+0000 7fc514c14640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-09-21T03:32:52.616 DEBUG:teuthology.orchestra.run.smithi038:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-21T03:32:52.646 DEBUG:teuthology.orchestra.run.smithi195:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-09-21T03:32:52.675 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-21T03:32:52.676 INFO:tasks.ceph:role: osd.4 2024-09-21T03:32:52.676 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi195.front.sepia.ceph.com 2024-09-21T03:32:52.676 DEBUG:teuthology.orchestra.run.smithi195:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-21T03:32:52.743 INFO:teuthology.orchestra.run.smithi195.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-21T03:32:52.744 INFO:teuthology.orchestra.run.smithi195.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-21T03:32:52.744 INFO:teuthology.orchestra.run.smithi195.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-21T03:32:52.744 INFO:teuthology.orchestra.run.smithi195.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-21T03:32:52.744 INFO:teuthology.orchestra.run.smithi195.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-21T03:32:52.744 INFO:teuthology.orchestra.run.smithi195.stdout: = sunit=0 swidth=0 blks 2024-09-21T03:32:52.744 INFO:teuthology.orchestra.run.smithi195.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-21T03:32:52.744 INFO:teuthology.orchestra.run.smithi195.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-21T03:32:52.744 INFO:teuthology.orchestra.run.smithi195.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-21T03:32:52.744 INFO:teuthology.orchestra.run.smithi195.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-21T03:32:52.776 INFO:teuthology.orchestra.run.smithi195.stdout:Discarding blocks...Done. 2024-09-21T03:32:52.779 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi195.front.sepia.ceph.com -o noatime 2024-09-21T03:32:52.779 DEBUG:teuthology.orchestra.run.smithi195:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-09-21T03:32:52.848 DEBUG:teuthology.orchestra.run.smithi195:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-09-21T03:32:52.921 DEBUG:teuthology.orchestra.run.smithi195:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-09-21T03:32:52.987 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-21T03:32:52.987 INFO:tasks.ceph:role: osd.5 2024-09-21T03:32:52.987 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi195.front.sepia.ceph.com 2024-09-21T03:32:52.988 DEBUG:teuthology.orchestra.run.smithi195:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-21T03:32:53.053 INFO:teuthology.orchestra.run.smithi195.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-21T03:32:53.053 INFO:teuthology.orchestra.run.smithi195.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-21T03:32:53.053 INFO:teuthology.orchestra.run.smithi195.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-21T03:32:53.053 INFO:teuthology.orchestra.run.smithi195.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-21T03:32:53.053 INFO:teuthology.orchestra.run.smithi195.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-21T03:32:53.053 INFO:teuthology.orchestra.run.smithi195.stdout: = sunit=0 swidth=0 blks 2024-09-21T03:32:53.053 INFO:teuthology.orchestra.run.smithi195.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-21T03:32:53.053 INFO:teuthology.orchestra.run.smithi195.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-21T03:32:53.053 INFO:teuthology.orchestra.run.smithi195.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-21T03:32:53.053 INFO:teuthology.orchestra.run.smithi195.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-21T03:32:53.085 INFO:teuthology.orchestra.run.smithi195.stdout:Discarding blocks...Done. 2024-09-21T03:32:53.087 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi195.front.sepia.ceph.com -o noatime 2024-09-21T03:32:53.087 DEBUG:teuthology.orchestra.run.smithi195:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-09-21T03:32:53.156 DEBUG:teuthology.orchestra.run.smithi195:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-09-21T03:32:53.224 DEBUG:teuthology.orchestra.run.smithi195:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-09-21T03:32:53.290 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-21T03:32:53.291 INFO:tasks.ceph:role: osd.6 2024-09-21T03:32:53.291 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi195.front.sepia.ceph.com 2024-09-21T03:32:53.291 DEBUG:teuthology.orchestra.run.smithi195:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-21T03:32:53.355 INFO:teuthology.orchestra.run.smithi195.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-21T03:32:53.355 INFO:teuthology.orchestra.run.smithi195.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-21T03:32:53.355 INFO:teuthology.orchestra.run.smithi195.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-21T03:32:53.355 INFO:teuthology.orchestra.run.smithi195.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-21T03:32:53.355 INFO:teuthology.orchestra.run.smithi195.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-21T03:32:53.355 INFO:teuthology.orchestra.run.smithi195.stdout: = sunit=0 swidth=0 blks 2024-09-21T03:32:53.355 INFO:teuthology.orchestra.run.smithi195.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-21T03:32:53.355 INFO:teuthology.orchestra.run.smithi195.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-21T03:32:53.356 INFO:teuthology.orchestra.run.smithi195.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-21T03:32:53.356 INFO:teuthology.orchestra.run.smithi195.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-21T03:32:53.387 INFO:teuthology.orchestra.run.smithi195.stdout:Discarding blocks...Done. 2024-09-21T03:32:53.389 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi195.front.sepia.ceph.com -o noatime 2024-09-21T03:32:53.389 DEBUG:teuthology.orchestra.run.smithi195:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-09-21T03:32:53.457 DEBUG:teuthology.orchestra.run.smithi195:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-09-21T03:32:53.527 DEBUG:teuthology.orchestra.run.smithi195:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-09-21T03:32:53.591 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-21T03:32:53.592 INFO:tasks.ceph:role: osd.7 2024-09-21T03:32:53.592 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi195.front.sepia.ceph.com 2024-09-21T03:32:53.592 DEBUG:teuthology.orchestra.run.smithi195:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-21T03:32:53.656 INFO:teuthology.orchestra.run.smithi195.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-21T03:32:53.656 INFO:teuthology.orchestra.run.smithi195.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-21T03:32:53.656 INFO:teuthology.orchestra.run.smithi195.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-21T03:32:53.656 INFO:teuthology.orchestra.run.smithi195.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-21T03:32:53.656 INFO:teuthology.orchestra.run.smithi195.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-21T03:32:53.656 INFO:teuthology.orchestra.run.smithi195.stdout: = sunit=0 swidth=0 blks 2024-09-21T03:32:53.656 INFO:teuthology.orchestra.run.smithi195.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-21T03:32:53.656 INFO:teuthology.orchestra.run.smithi195.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-21T03:32:53.657 INFO:teuthology.orchestra.run.smithi195.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-21T03:32:53.657 INFO:teuthology.orchestra.run.smithi195.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-21T03:32:53.688 INFO:teuthology.orchestra.run.smithi195.stdout:Discarding blocks...Done. 2024-09-21T03:32:53.690 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi195.front.sepia.ceph.com -o noatime 2024-09-21T03:32:53.690 DEBUG:teuthology.orchestra.run.smithi195:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-09-21T03:32:53.758 DEBUG:teuthology.orchestra.run.smithi195:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-09-21T03:32:53.827 DEBUG:teuthology.orchestra.run.smithi195:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-21T03:32:53.931 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:32:53.929+0000 7fe5b581c640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-09-21T03:32:53.931 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:32:53.929+0000 7fe5b581c640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-09-21T03:32:53.931 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:32:53.929+0000 7fe5b581c640 -1 bdev(0x564e80629900 /var/lib/ceph/osd/ceph-4/block) open stat got: (1) Operation not permitted 2024-09-21T03:32:53.931 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:32:53.929+0000 7fe5b581c640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-09-21T03:32:57.080 DEBUG:teuthology.orchestra.run.smithi195:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-21T03:32:57.110 DEBUG:teuthology.orchestra.run.smithi195:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-21T03:32:57.213 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:32:57.211+0000 7f5ae7018640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-09-21T03:32:57.213 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:32:57.211+0000 7f5ae7018640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-09-21T03:32:57.213 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:32:57.211+0000 7f5ae7018640 -1 bdev(0x56328dbf9900 /var/lib/ceph/osd/ceph-5/block) open stat got: (1) Operation not permitted 2024-09-21T03:32:57.213 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:32:57.211+0000 7f5ae7018640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-09-21T03:32:59.213 DEBUG:teuthology.orchestra.run.smithi038:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T03:32:59.216 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T03:33:00.627 DEBUG:teuthology.orchestra.run.smithi195:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-09-21T03:33:00.655 DEBUG:teuthology.orchestra.run.smithi195:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-21T03:33:00.757 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:00.755+0000 7f7d10616640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-09-21T03:33:00.758 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:00.755+0000 7f7d10616640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-09-21T03:33:00.758 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:00.756+0000 7f7d10616640 -1 bdev(0x55811e343900 /var/lib/ceph/osd/ceph-6/block) open stat got: (1) Operation not permitted 2024-09-21T03:33:00.758 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:00.756+0000 7f7d10616640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-09-21T03:33:04.162 DEBUG:teuthology.orchestra.run.smithi195:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-09-21T03:33:04.188 DEBUG:teuthology.orchestra.run.smithi195:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-21T03:33:04.287 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:04.285+0000 7fa94db89640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-09-21T03:33:04.287 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:04.286+0000 7fa94db89640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-09-21T03:33:04.288 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:04.286+0000 7fa94db89640 -1 bdev(0x55f73375f900 /var/lib/ceph/osd/ceph-7/block) open stat got: (1) Operation not permitted 2024-09-21T03:33:04.288 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:04.286+0000 7fa94db89640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-09-21T03:33:07.446 DEBUG:teuthology.orchestra.run.smithi195:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-09-21T03:33:07.474 INFO:tasks.ceph:Reading keys from all nodes... 2024-09-21T03:33:07.474 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-21T03:33:07.475 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-09-21T03:33:07.507 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-21T03:33:07.507 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-09-21T03:33:07.576 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-21T03:33:07.576 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-09-21T03:33:07.645 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-21T03:33:07.645 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-09-21T03:33:07.713 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-21T03:33:07.713 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-09-21T03:33:07.779 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-21T03:33:07.780 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-09-21T03:33:07.846 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-21T03:33:07.846 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-09-21T03:33:07.913 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-09-21T03:33:07.913 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-09-21T03:33:07.943 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-09-21T03:33:07.943 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-09-21T03:33:08.012 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-09-21T03:33:08.013 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-09-21T03:33:08.080 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-09-21T03:33:08.080 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-09-21T03:33:08.148 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-09-21T03:33:08.148 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-09-21T03:33:08.214 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-09-21T03:33:08.214 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-09-21T03:33:08.281 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-09-21T03:33:08.281 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-09-21T03:33:08.347 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-21T03:33:08.347 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-09-21T03:33:08.366 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-21T03:33:08.366 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-09-21T03:33:08.425 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-09-21T03:33:08.425 DEBUG:teuthology.orchestra.run.smithi195:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-09-21T03:33:08.441 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-09-21T03:33:08.442 DEBUG:teuthology.orchestra.run.smithi195:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-09-21T03:33:08.497 INFO:tasks.ceph:Adding keys to all mons... 2024-09-21T03:33:08.497 DEBUG:teuthology.orchestra.run.smithi038:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-21T03:33:08.500 DEBUG:teuthology.orchestra.run.smithi195:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-21T03:33:08.541 INFO:teuthology.orchestra.run.smithi038.stdout:[mgr.y] 2024-09-21T03:33:08.541 INFO:teuthology.orchestra.run.smithi038.stdout: key = AQBPPu5mWEz+KBAASx2KzDril3lD9cEawy56jA== 2024-09-21T03:33:08.541 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.a] 2024-09-21T03:33:08.541 INFO:teuthology.orchestra.run.smithi038.stdout: key = AQBPPu5mfzGIMRAAmq0TaC3J1VU3GxzhTgJ/Zw== 2024-09-21T03:33:08.541 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.c] 2024-09-21T03:33:08.541 INFO:teuthology.orchestra.run.smithi038.stdout: key = AQBPPu5mQipIORAArRn8k8SvvpG8x03F9I2/gQ== 2024-09-21T03:33:08.541 INFO:teuthology.orchestra.run.smithi038.stdout:[osd.0] 2024-09-21T03:33:08.542 INFO:teuthology.orchestra.run.smithi038.stdout: key = AQBXPu5mMZggAhAAtMPdJipheizZhpby9APj4A== 2024-09-21T03:33:08.542 INFO:teuthology.orchestra.run.smithi038.stdout:[osd.1] 2024-09-21T03:33:08.542 INFO:teuthology.orchestra.run.smithi038.stdout: key = AQBaPu5mm9TFIhAAWPJCvBYMgMbKr7Z7Diq58A== 2024-09-21T03:33:08.542 INFO:teuthology.orchestra.run.smithi038.stdout:[osd.2] 2024-09-21T03:33:08.542 INFO:teuthology.orchestra.run.smithi038.stdout: key = AQBePu5mqjljCBAAcdEpzJq6D8e1L+q1s0GjWQ== 2024-09-21T03:33:08.542 INFO:teuthology.orchestra.run.smithi038.stdout:[osd.3] 2024-09-21T03:33:08.542 INFO:teuthology.orchestra.run.smithi038.stdout: key = AQBhPu5mRjfdKBAAJ6+PlEiRp/xvxny6SjHV8Q== 2024-09-21T03:33:08.542 INFO:teuthology.orchestra.run.smithi038.stdout:[mgr.x] 2024-09-21T03:33:08.542 INFO:teuthology.orchestra.run.smithi038.stdout: key = AQBPPu5m6K2oLRAAzldoBXAMSnWhxVFwPZkHOw== 2024-09-21T03:33:08.542 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.b] 2024-09-21T03:33:08.542 INFO:teuthology.orchestra.run.smithi038.stdout: key = AQBQPu5m/fzuAhAApyigRlQ8OrV/sf0ap6Fn6A== 2024-09-21T03:33:08.542 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.d] 2024-09-21T03:33:08.542 INFO:teuthology.orchestra.run.smithi038.stdout: key = AQBQPu5mzlyDChAAqbPKXS3EtnOqqeHhA1ehIw== 2024-09-21T03:33:08.543 INFO:teuthology.orchestra.run.smithi038.stdout:[osd.4] 2024-09-21T03:33:08.543 INFO:teuthology.orchestra.run.smithi038.stdout: key = AQBlPu5m8bNyNxAA2rERruGojEDKmcXN2K6uog== 2024-09-21T03:33:08.543 INFO:teuthology.orchestra.run.smithi038.stdout:[osd.5] 2024-09-21T03:33:08.543 INFO:teuthology.orchestra.run.smithi038.stdout: key = AQBpPu5mBB2iDBAAvsvhAPMvNWII2Zr9Je45YQ== 2024-09-21T03:33:08.543 INFO:teuthology.orchestra.run.smithi038.stdout:[osd.6] 2024-09-21T03:33:08.543 INFO:teuthology.orchestra.run.smithi038.stdout: key = AQBsPu5mMQMaLRAAznzYdoNGHWOI2p2mNN/rQw== 2024-09-21T03:33:08.543 INFO:teuthology.orchestra.run.smithi038.stdout:[osd.7] 2024-09-21T03:33:08.543 INFO:teuthology.orchestra.run.smithi038.stdout: key = AQBwPu5mHu0WERAAir8bmtuNOOj/2D4hLaoYOA== 2024-09-21T03:33:08.543 INFO:teuthology.orchestra.run.smithi038.stdout:[client.0] 2024-09-21T03:33:08.543 INFO:teuthology.orchestra.run.smithi038.stdout: key = AQBQPu5m6ydBDxAAlWZ2XzQTPBcYPriPGRJVeA== 2024-09-21T03:33:08.543 INFO:teuthology.orchestra.run.smithi038.stdout:[client.2] 2024-09-21T03:33:08.543 INFO:teuthology.orchestra.run.smithi038.stdout: key = AQBQPu5m48oaExAAOzYrR/SsykpzH2TkWHs39Q== 2024-09-21T03:33:08.543 INFO:teuthology.orchestra.run.smithi038.stdout:[client.1] 2024-09-21T03:33:08.543 INFO:teuthology.orchestra.run.smithi038.stdout: key = AQBQPu5mlWP0FhAAkNwfaJ6rPr3r4ZRVbPxX8g== 2024-09-21T03:33:08.544 INFO:teuthology.orchestra.run.smithi038.stdout:[client.3] 2024-09-21T03:33:08.544 INFO:teuthology.orchestra.run.smithi038.stdout: key = AQBQPu5mVQSzGhAAAo6vizLc7GOnenQRsgtn1A== 2024-09-21T03:33:08.561 INFO:teuthology.orchestra.run.smithi195.stdout:[mgr.y] 2024-09-21T03:33:08.562 INFO:teuthology.orchestra.run.smithi195.stdout: key = AQBPPu5mWEz+KBAASx2KzDril3lD9cEawy56jA== 2024-09-21T03:33:08.562 INFO:teuthology.orchestra.run.smithi195.stdout:[mds.a] 2024-09-21T03:33:08.562 INFO:teuthology.orchestra.run.smithi195.stdout: key = AQBPPu5mfzGIMRAAmq0TaC3J1VU3GxzhTgJ/Zw== 2024-09-21T03:33:08.562 INFO:teuthology.orchestra.run.smithi195.stdout:[mds.c] 2024-09-21T03:33:08.562 INFO:teuthology.orchestra.run.smithi195.stdout: key = AQBPPu5mQipIORAArRn8k8SvvpG8x03F9I2/gQ== 2024-09-21T03:33:08.562 INFO:teuthology.orchestra.run.smithi195.stdout:[osd.0] 2024-09-21T03:33:08.562 INFO:teuthology.orchestra.run.smithi195.stdout: key = AQBXPu5mMZggAhAAtMPdJipheizZhpby9APj4A== 2024-09-21T03:33:08.562 INFO:teuthology.orchestra.run.smithi195.stdout:[osd.1] 2024-09-21T03:33:08.562 INFO:teuthology.orchestra.run.smithi195.stdout: key = AQBaPu5mm9TFIhAAWPJCvBYMgMbKr7Z7Diq58A== 2024-09-21T03:33:08.562 INFO:teuthology.orchestra.run.smithi195.stdout:[osd.2] 2024-09-21T03:33:08.562 INFO:teuthology.orchestra.run.smithi195.stdout: key = AQBePu5mqjljCBAAcdEpzJq6D8e1L+q1s0GjWQ== 2024-09-21T03:33:08.562 INFO:teuthology.orchestra.run.smithi195.stdout:[osd.3] 2024-09-21T03:33:08.562 INFO:teuthology.orchestra.run.smithi195.stdout: key = AQBhPu5mRjfdKBAAJ6+PlEiRp/xvxny6SjHV8Q== 2024-09-21T03:33:08.563 INFO:teuthology.orchestra.run.smithi195.stdout:[mgr.x] 2024-09-21T03:33:08.563 INFO:teuthology.orchestra.run.smithi195.stdout: key = AQBPPu5m6K2oLRAAzldoBXAMSnWhxVFwPZkHOw== 2024-09-21T03:33:08.563 INFO:teuthology.orchestra.run.smithi195.stdout:[mds.b] 2024-09-21T03:33:08.563 INFO:teuthology.orchestra.run.smithi195.stdout: key = AQBQPu5m/fzuAhAApyigRlQ8OrV/sf0ap6Fn6A== 2024-09-21T03:33:08.563 INFO:teuthology.orchestra.run.smithi195.stdout:[mds.d] 2024-09-21T03:33:08.563 INFO:teuthology.orchestra.run.smithi195.stdout: key = AQBQPu5mzlyDChAAqbPKXS3EtnOqqeHhA1ehIw== 2024-09-21T03:33:08.563 INFO:teuthology.orchestra.run.smithi195.stdout:[osd.4] 2024-09-21T03:33:08.563 INFO:teuthology.orchestra.run.smithi195.stdout: key = AQBlPu5m8bNyNxAA2rERruGojEDKmcXN2K6uog== 2024-09-21T03:33:08.563 INFO:teuthology.orchestra.run.smithi195.stdout:[osd.5] 2024-09-21T03:33:08.563 INFO:teuthology.orchestra.run.smithi195.stdout: key = AQBpPu5mBB2iDBAAvsvhAPMvNWII2Zr9Je45YQ== 2024-09-21T03:33:08.563 INFO:teuthology.orchestra.run.smithi195.stdout:[osd.6] 2024-09-21T03:33:08.563 INFO:teuthology.orchestra.run.smithi195.stdout: key = AQBsPu5mMQMaLRAAznzYdoNGHWOI2p2mNN/rQw== 2024-09-21T03:33:08.563 INFO:teuthology.orchestra.run.smithi195.stdout:[osd.7] 2024-09-21T03:33:08.563 INFO:teuthology.orchestra.run.smithi195.stdout: key = AQBwPu5mHu0WERAAir8bmtuNOOj/2D4hLaoYOA== 2024-09-21T03:33:08.564 INFO:teuthology.orchestra.run.smithi195.stdout:[client.0] 2024-09-21T03:33:08.564 INFO:teuthology.orchestra.run.smithi195.stdout: key = AQBQPu5m6ydBDxAAlWZ2XzQTPBcYPriPGRJVeA== 2024-09-21T03:33:08.564 INFO:teuthology.orchestra.run.smithi195.stdout:[client.2] 2024-09-21T03:33:08.564 INFO:teuthology.orchestra.run.smithi195.stdout: key = AQBQPu5m48oaExAAOzYrR/SsykpzH2TkWHs39Q== 2024-09-21T03:33:08.564 INFO:teuthology.orchestra.run.smithi195.stdout:[client.1] 2024-09-21T03:33:08.564 INFO:teuthology.orchestra.run.smithi195.stdout: key = AQBQPu5mlWP0FhAAkNwfaJ6rPr3r4ZRVbPxX8g== 2024-09-21T03:33:08.564 INFO:teuthology.orchestra.run.smithi195.stdout:[client.3] 2024-09-21T03:33:08.564 INFO:teuthology.orchestra.run.smithi195.stdout: key = AQBQPu5mVQSzGhAAAo6vizLc7GOnenQRsgtn1A== 2024-09-21T03:33:08.565 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-21T03:33:08.587 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-21T03:33:08.667 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-21T03:33:08.669 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-21T03:33:08.736 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-21T03:33:08.738 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-21T03:33:08.803 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T03:33:08.805 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T03:33:08.865 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T03:33:08.867 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T03:33:08.929 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T03:33:08.931 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T03:33:08.992 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T03:33:08.995 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T03:33:09.056 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-21T03:33:09.058 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-21T03:33:09.119 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-21T03:33:09.121 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-21T03:33:09.185 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-21T03:33:09.187 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-21T03:33:09.251 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T03:33:09.253 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T03:33:09.314 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T03:33:09.317 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T03:33:09.377 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T03:33:09.380 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T03:33:09.441 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T03:33:09.444 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T03:33:09.505 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-21T03:33:09.507 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-21T03:33:09.572 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-21T03:33:09.574 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-21T03:33:09.638 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-21T03:33:09.641 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-21T03:33:09.705 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-21T03:33:09.708 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-21T03:33:09.772 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-09-21T03:33:09.772 DEBUG:teuthology.orchestra.run.smithi038:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-09-21T03:33:09.798 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-21T03:33:09.916 DEBUG:teuthology.orchestra.run.smithi038:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-09-21T03:33:09.941 DEBUG:teuthology.orchestra.run.smithi195:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-09-21T03:33:09.967 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-21T03:33:10.074 DEBUG:teuthology.orchestra.run.smithi195:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-09-21T03:33:10.100 DEBUG:teuthology.orchestra.run.smithi195:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-09-21T03:33:10.164 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-21T03:33:10.271 DEBUG:teuthology.orchestra.run.smithi195:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-09-21T03:33:10.295 DEBUG:teuthology.orchestra.run.smithi038:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-21T03:33:10.298 DEBUG:teuthology.orchestra.run.smithi195:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-21T03:33:10.350 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-09-21T03:33:10.351 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-21T03:33:10.351 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-21T03:33:10.354 INFO:tasks.ceph.mon.a:Started 2024-09-21T03:33:10.354 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-21T03:33:10.354 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-21T03:33:10.393 INFO:tasks.ceph.mon.b:Started 2024-09-21T03:33:10.393 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-21T03:33:10.393 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-21T03:33:10.395 INFO:tasks.ceph.mon.c:Started 2024-09-21T03:33:10.395 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-09-21T03:33:10.396 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-09-21T03:33:10.396 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-09-21T03:33:10.398 INFO:tasks.ceph.mgr.y:Started 2024-09-21T03:33:10.398 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-09-21T03:33:10.398 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-09-21T03:33:10.400 INFO:tasks.ceph.mgr.x:Started 2024-09-21T03:33:10.400 INFO:tasks.ceph:setting config [client] client_mount_timeout = 600 2024-09-21T03:33:10.400 DEBUG:teuthology.orchestra.run.smithi038:> sudo ceph --cluster ceph config set client client_mount_timeout 600 2024-09-21T03:33:10.402 INFO:tasks.ceph:setting config [client] debug_client = 20 2024-09-21T03:33:10.402 DEBUG:teuthology.orchestra.run.smithi038:> sudo ceph --cluster ceph config set client debug_client 20 2024-09-21T03:33:10.445 INFO:tasks.ceph:setting config [client] debug_ms = 1 2024-09-21T03:33:10.446 DEBUG:teuthology.orchestra.run.smithi038:> sudo ceph --cluster ceph config set client debug_ms 1 2024-09-21T03:33:10.488 INFO:tasks.ceph:setting config [client] rados_mon_op_timeout = 900 2024-09-21T03:33:10.488 DEBUG:teuthology.orchestra.run.smithi038:> sudo ceph --cluster ceph config set client rados_mon_op_timeout 900 2024-09-21T03:33:10.531 INFO:tasks.ceph:setting config [client] rados_osd_op_timeout = 900 2024-09-21T03:33:10.531 DEBUG:teuthology.orchestra.run.smithi038:> sudo ceph --cluster ceph config set client rados_osd_op_timeout 900 2024-09-21T03:33:10.575 INFO:tasks.ceph:setting config [mds] debug_mds = 20 2024-09-21T03:33:10.575 DEBUG:teuthology.orchestra.run.smithi038:> sudo ceph --cluster ceph config set mds debug_mds 20 2024-09-21T03:33:10.618 INFO:tasks.ceph:setting config [mds] debug_mds_balancer = 20 2024-09-21T03:33:10.618 DEBUG:teuthology.orchestra.run.smithi038:> sudo ceph --cluster ceph config set mds debug_mds_balancer 20 2024-09-21T03:33:10.662 INFO:tasks.ceph:setting config [mds] debug_ms = 2 2024-09-21T03:33:10.663 DEBUG:teuthology.orchestra.run.smithi038:> sudo ceph --cluster ceph config set mds debug_ms 2 2024-09-21T03:33:10.705 INFO:tasks.ceph:setting config [mds] mds_debug_frag = True 2024-09-21T03:33:10.706 DEBUG:teuthology.orchestra.run.smithi038:> sudo ceph --cluster ceph config set mds mds_debug_frag True 2024-09-21T03:33:10.749 INFO:tasks.ceph:setting config [mds] mds_debug_scatterstat = True 2024-09-21T03:33:10.749 DEBUG:teuthology.orchestra.run.smithi038:> sudo ceph --cluster ceph config set mds mds_debug_scatterstat True 2024-09-21T03:33:10.795 INFO:tasks.ceph:setting config [mds] mds_op_complaint_time = 180 2024-09-21T03:33:10.795 DEBUG:teuthology.orchestra.run.smithi038:> sudo ceph --cluster ceph config set mds mds_op_complaint_time 180 2024-09-21T03:33:10.839 INFO:tasks.ceph:setting config [mds] mds_verify_scatter = True 2024-09-21T03:33:10.839 DEBUG:teuthology.orchestra.run.smithi038:> sudo ceph --cluster ceph config set mds mds_verify_scatter True 2024-09-21T03:33:10.883 INFO:tasks.ceph:setting config [mds] osd_op_complaint_time = 180 2024-09-21T03:33:10.883 DEBUG:teuthology.orchestra.run.smithi038:> sudo ceph --cluster ceph config set mds osd_op_complaint_time 180 2024-09-21T03:33:10.897 INFO:tasks.ceph:setting config [mds] rados_mon_op_timeout = 900 2024-09-21T03:33:10.897 DEBUG:teuthology.orchestra.run.smithi038:> sudo ceph --cluster ceph config set mds rados_mon_op_timeout 900 2024-09-21T03:33:10.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:10.929+0000 7fa5ecff9640 1 -- 172.21.15.38:0/4013463774 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa5f80b3eb0 msgr2=0x7fa5f8148c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:10.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:10.929+0000 7fa5ecff9640 1 --2- 172.21.15.38:0/4013463774 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa5f80b3eb0 0x7fa5f8148c80 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fa5e8006f60 tx=0x7fa5f815fe10 comp rx=0 tx=0).stop 2024-09-21T03:33:10.932 INFO:tasks.ceph:setting config [mds] rados_osd_op_timeout = 900 2024-09-21T03:33:10.932 DEBUG:teuthology.orchestra.run.smithi038:> sudo ceph --cluster ceph config set mds rados_osd_op_timeout 900 2024-09-21T03:33:10.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:10.947+0000 7fa5ecff9640 1 -- 172.21.15.38:0/4013463774 shutdown_connections 2024-09-21T03:33:10.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:10.947+0000 7fa5ecff9640 1 --2- 172.21.15.38:0/4013463774 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa5f81f0b10 0x7fa5f81f0f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:10.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:10.947+0000 7fa5ecff9640 1 --2- 172.21.15.38:0/4013463774 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa5f80b3eb0 0x7fa5f8148c80 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:10.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:10.947+0000 7fa5ecff9640 1 --2- 172.21.15.38:0/4013463774 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa5e40037a0 0x7fa5f815e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:10.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:10.947+0000 7fa5ecff9640 1 -- 172.21.15.38:0/4013463774 >> 172.21.15.38:0/4013463774 conn(0x7fa5f80735d0 msgr2=0x7fa5f80cb6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:10.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:10.947+0000 7fa5ecff9640 1 -- 172.21.15.38:0/4013463774 shutdown_connections 2024-09-21T03:33:10.951 INFO:tasks.ceph:setting config [mgr] client_mount_timeout = 30 2024-09-21T03:33:10.951 DEBUG:teuthology.orchestra.run.smithi038:> sudo ceph --cluster ceph config set mgr client_mount_timeout 30 2024-09-21T03:33:10.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:10.948+0000 7fa5ecff9640 1 -- 172.21.15.38:0/4013463774 wait complete. 2024-09-21T03:33:10.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:10.952+0000 7f7aff0f9640 1 -- 172.21.15.38:0/1124902099 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7af806edf0 msgr2=0x7f7af80b3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:10.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:10.952+0000 7f7aff0f9640 1 --2- 172.21.15.38:0/1124902099 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7af806edf0 0x7f7af80b3a50 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f7ae8009860 tx=0x7f7ae802a380 comp rx=0 tx=0).stop 2024-09-21T03:33:10.955 INFO:tasks.ceph:setting config [mgr] debug_client = 20 2024-09-21T03:33:10.956 DEBUG:teuthology.orchestra.run.smithi038:> sudo ceph --cluster ceph config set mgr debug_client 20 2024-09-21T03:33:10.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:10.955+0000 7ff3f77fe640 1 -- 172.21.15.38:0/1445732049 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command([{prefix=config set, name=debug_client}] v 0) -- 0x7ff3e40052f0 con 0x7ff40c1f1350 2024-09-21T03:33:10.959 INFO:tasks.ceph:setting config [mgr] debug_mgr = 20 2024-09-21T03:33:10.959 DEBUG:teuthology.orchestra.run.smithi038:> sudo ceph --cluster ceph config set mgr debug_mgr 20 2024-09-21T03:33:10.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:10.958+0000 7f7aff0f9640 1 -- 172.21.15.38:0/1124902099 shutdown_connections 2024-09-21T03:33:10.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:10.958+0000 7f7aff0f9640 1 --2- 172.21.15.38:0/1124902099 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7af806edf0 0x7f7af80b3a50 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:10.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:10.958+0000 7f7aff0f9640 1 -- 172.21.15.38:0/1124902099 >> 172.21.15.38:0/1124902099 conn(0x7f7af806cde0 msgr2=0x7f7af806d1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:10.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:10.963+0000 7ff0527fc640 1 -- 172.21.15.38:0/2900396950 <== mon.0 v2:172.21.15.38:3300/0 10 ==== config(4 keys) ==== 110+0+0 (secure 0 0 0) 0x7ff054042420 con 0x7ff05c0d02b0 2024-09-21T03:33:10.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:10.963+0000 7f8ff2ffd640 1 -- 172.21.15.38:0/3263739705 <== mon.2 v2:172.21.15.195:3301/0 10 ==== config(4 keys) ==== 110+0+0 (secure 0 0 0) 0x7f8ffc01b710 con 0x7f9004157050 2024-09-21T03:33:10.971 INFO:tasks.ceph:setting config [mgr] debug_ms = 1 2024-09-21T03:33:10.971 DEBUG:teuthology.orchestra.run.smithi038:> sudo ceph --cluster ceph config set mgr debug_ms 1 2024-09-21T03:33:10.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:10.964+0000 7ff401ffb640 1 -- 172.21.15.38:0/1445732049 <== mon.0 v2:172.21.15.38:3300/0 10 ==== mon_command_ack([{prefix=config set, name=debug_client}]=0 v5)=0 v5) ==== 114+0+0 (secure 0 0 0) 0x7ff404027c20 con 0x7ff40c1f1350 2024-09-21T03:33:10.972 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:10.964+0000 7ff401ffb640 1 -- 172.21.15.38:0/1445732049 <== mon.0 v2:172.21.15.38:3300/0 11 ==== config(4 keys) ==== 110+0+0 (secure 0 0 0) 0x7ff404027e00 con 0x7ff40c1f1350 2024-09-21T03:33:10.972 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:10.964+0000 7ff284ff9640 1 -- 172.21.15.38:0/830217705 <== mon.0 v2:172.21.15.38:3300/0 9 ==== config(4 keys) ==== 110+0+0 (secure 0 0 0) 0x7ff278010ad0 con 0x7ff2880cfac0 2024-09-21T03:33:10.972 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:10.967+0000 7ff3f77fe640 1 -- 172.21.15.38:0/1445732049 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff40c1f1350 msgr2=0x7ff40c150940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:10.972 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:10.967+0000 7ff3f77fe640 1 --2- 172.21.15.38:0/1445732049 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff40c1f1350 0x7ff40c150940 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ff40c073ca0 tx=0x7ff404006720 comp rx=0 tx=0).stop 2024-09-21T03:33:10.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:10.968+0000 7f7aff0f9640 1 -- 172.21.15.38:0/1124902099 shutdown_connections 2024-09-21T03:33:10.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:10.969+0000 7f7aff0f9640 1 -- 172.21.15.38:0/1124902099 wait complete. 2024-09-21T03:33:10.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:10.970+0000 7ff3f77fe640 1 -- 172.21.15.38:0/1445732049 shutdown_connections 2024-09-21T03:33:10.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:10.970+0000 7ff3f77fe640 1 --2- 172.21.15.38:0/1445732049 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff40c1f1350 0x7ff40c150940 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:10.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:10.970+0000 7ff3f77fe640 1 --2- 172.21.15.38:0/1445732049 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff40c151af0 0x7ff40c0ccb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:10.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:10.970+0000 7ff3f77fe640 1 --2- 172.21.15.38:0/1445732049 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff3f8003e20 0x7ff40c14e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:10.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:10.970+0000 7ff3f77fe640 1 -- 172.21.15.38:0/1445732049 >> 172.21.15.38:0/1445732049 conn(0x7ff40c0cdca0 msgr2=0x7ff40c155840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:10.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:10.970+0000 7ff03ffff640 1 -- 172.21.15.38:0/2900396950 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command([{prefix=config set, name=rados_osd_op_timeout}] v 0) -- 0x7ff038005310 con 0x7ff05c0d02b0 2024-09-21T03:33:10.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:10.972+0000 7f7aff0f9640 1 Processor -- start 2024-09-21T03:33:10.979 INFO:tasks.ceph:setting config [mgr] mon_warn_on_pool_no_app = False 2024-09-21T03:33:10.980 DEBUG:teuthology.orchestra.run.smithi038:> sudo ceph --cluster ceph config set mgr mon_warn_on_pool_no_app False 2024-09-21T03:33:10.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:10.974+0000 7ff3f77fe640 1 -- 172.21.15.38:0/1445732049 shutdown_connections 2024-09-21T03:33:10.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:10.974+0000 7ff3f77fe640 1 -- 172.21.15.38:0/1445732049 wait complete. 2024-09-21T03:33:10.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:10.972+0000 7f7aff0f9640 1 -- start start 2024-09-21T03:33:10.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:10.975+0000 7f7aff0f9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7af806edf0 0x7f7af813d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:10.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:10.975+0000 7f7aff0f9640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7af80aec20 0x7f7af815af10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:10.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:10.975+0000 7f7aff0f9640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7af80b3f90 0x7f7af815bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:10.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:10.975+0000 7f7afce95640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7af806edf0 0x7f7af813d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:10.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:10.975+0000 7f7aff0f9640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f7af80af8c0 con 0x7f7af806edf0 2024-09-21T03:33:10.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:10.975+0000 7f7aff0f9640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f7af80af740 con 0x7f7af80aec20 2024-09-21T03:33:10.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:10.975+0000 7f7aff0f9640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f7af815d0c0 con 0x7f7af80b3f90 2024-09-21T03:33:10.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:10.975+0000 7f7afce95640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7af806edf0 0x7f7af813d7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41340/0 (socket says 172.21.15.38:41340) 2024-09-21T03:33:10.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:10.975+0000 7f7afce95640 1 -- 172.21.15.38:0/423395403 learned_addr learned my addr 172.21.15.38:0/423395403 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:33:10.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:10.976+0000 7f7afd696640 1 --2- 172.21.15.38:0/423395403 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7af80b3f90 0x7f7af815bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:10.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:10.978+0000 7f7af7fff640 1 --2- 172.21.15.38:0/423395403 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7af80aec20 0x7f7af815af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:10.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:10.979+0000 7ff284ff9640 1 -- 172.21.15.38:0/830217705 <== mon.0 v2:172.21.15.38:3300/0 10 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff27800e4f0 con 0x7ff2880cfac0 2024-09-21T03:33:10.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:10.980+0000 7f8ff2ffd640 1 -- 172.21.15.38:0/3263739705 <== mon.2 v2:172.21.15.195:3301/0 11 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8ffc01ba90 con 0x7f9004157050 2024-09-21T03:33:10.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:10.981+0000 7ff0527fc640 1 -- 172.21.15.38:0/2900396950 <== mon.0 v2:172.21.15.38:3300/0 11 ==== mon_command_ack([{prefix=config set, name=rados_osd_op_timeout}]=0 v6)=0 v6) ==== 123+0+0 (secure 0 0 0) 0x7ff054032030 con 0x7ff05c0d02b0 2024-09-21T03:33:10.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:10.981+0000 7ff0527fc640 1 -- 172.21.15.38:0/2900396950 <== mon.0 v2:172.21.15.38:3300/0 12 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff054042720 con 0x7ff05c0d02b0 2024-09-21T03:33:10.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:10.979+0000 7f7afd696640 1 -- 172.21.15.38:0/423395403 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7af80aec20 msgr2=0x7f7af815af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:10.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:10.979+0000 7f7afd696640 1 --2- 172.21.15.38:0/423395403 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7af80aec20 0x7f7af815af10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:10.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:10.979+0000 7f7afd696640 1 -- 172.21.15.38:0/423395403 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7af806edf0 msgr2=0x7f7af813d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:10.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:10.979+0000 7f7afd696640 1 --2- 172.21.15.38:0/423395403 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7af806edf0 0x7f7af813d7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:10.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:10.979+0000 7f7afd696640 1 -- 172.21.15.38:0/423395403 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7af813dd10 con 0x7f7af80b3f90 2024-09-21T03:33:10.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:10.981+0000 7f7afd696640 1 --2- 172.21.15.38:0/423395403 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7af80b3f90 0x7f7af815bfa0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f7af001ce00 tx=0x7f7af001cef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:10.984 INFO:tasks.ceph:setting config [mon] mon_op_complaint_time = 120 2024-09-21T03:33:10.984 DEBUG:teuthology.orchestra.run.smithi038:> sudo ceph --cluster ceph config set mon mon_op_complaint_time 120 2024-09-21T03:33:10.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:10.988+0000 7ff0641df640 1 -- 172.21.15.38:0/2900396950 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff05c0d02b0 msgr2=0x7ff05c068d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:10.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:10.988+0000 7ff0641df640 1 --2- 172.21.15.38:0/2900396950 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff05c0d02b0 0x7ff05c068d00 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7ff05c0cd2d0 tx=0x7ff054004a40 comp rx=0 tx=0).stop 2024-09-21T03:33:10.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:10.982+0000 7f7af57fa640 1 -- 172.21.15.38:0/423395403 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7af002a070 con 0x7f7af80b3f90 2024-09-21T03:33:10.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:10.983+0000 7f7aff0f9640 1 -- 172.21.15.38:0/423395403 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7af815b7e0 con 0x7f7af80b3f90 2024-09-21T03:33:10.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:10.983+0000 7f7aff0f9640 1 -- 172.21.15.38:0/423395403 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f7af815ba80 con 0x7f7af80b3f90 2024-09-21T03:33:10.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:10.984+0000 7f7af57fa640 1 -- 172.21.15.38:0/423395403 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7af001dbc0 con 0x7f7af80b3f90 2024-09-21T03:33:10.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:10.984+0000 7f7af57fa640 1 -- 172.21.15.38:0/423395403 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7af0025440 con 0x7f7af80b3f90 2024-09-21T03:33:10.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:10.984+0000 7f7af57fa640 1 -- 172.21.15.38:0/423395403 <== mon.2 v2:172.21.15.195:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f7af002d070 con 0x7f7af80b3f90 2024-09-21T03:33:10.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:10.984+0000 7f7af57fa640 1 -- 172.21.15.38:0/423395403 <== mon.2 v2:172.21.15.195:3301/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f7af002fcb0 con 0x7f7af80b3f90 2024-09-21T03:33:10.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:10.987+0000 7f7adeffd640 1 -- 172.21.15.38:0/423395403 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ad8005290 con 0x7f7af80b3f90 2024-09-21T03:33:10.995 INFO:tasks.ceph:setting config [osd] osd_op_complaint_time = 180 2024-09-21T03:33:10.995 DEBUG:teuthology.orchestra.run.smithi038:> sudo ceph --cluster ceph config set osd osd_op_complaint_time 180 2024-09-21T03:33:10.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:10.990+0000 7f7af57fa640 1 -- 172.21.15.38:0/423395403 <== mon.2 v2:172.21.15.195:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f7af002f440 con 0x7f7af80b3f90 2024-09-21T03:33:10.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:10.994+0000 7ff0641df640 1 -- 172.21.15.38:0/2900396950 shutdown_connections 2024-09-21T03:33:10.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:10.994+0000 7ff0641df640 1 --2- 172.21.15.38:0/2900396950 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff05c0d02b0 0x7ff05c068d00 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:10.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:10.994+0000 7ff0641df640 1 --2- 172.21.15.38:0/2900396950 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff05c0ceca0 0x7ff05c14bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:10.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:10.994+0000 7ff0641df640 1 --2- 172.21.15.38:0/2900396950 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff05c06ddf0 0x7ff05c148e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:10.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:10.994+0000 7ff0641df640 1 -- 172.21.15.38:0/2900396950 >> 172.21.15.38:0/2900396950 conn(0x7ff05c0735d0 msgr2=0x7ff05c15c290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:10.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:10.996+0000 7ff0641df640 1 -- 172.21.15.38:0/2900396950 shutdown_connections 2024-09-21T03:33:10.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:10.996+0000 7ff0641df640 1 -- 172.21.15.38:0/2900396950 wait complete. 2024-09-21T03:33:11.001 DEBUG:tasks.ceph:set 22 configs 2024-09-21T03:33:11.001 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'client_mount_timeout', '600'], check_status=True, hostname='smithi038') 2024-09-21T03:33:11.001 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'debug_client', '20'], check_status=True, hostname='smithi038') 2024-09-21T03:33:11.012 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'debug_ms', '1'], check_status=True, hostname='smithi038') 2024-09-21T03:33:11.017 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi038') 2024-09-21T03:33:11.017 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi038') 2024-09-21T03:33:11.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.029+0000 7fe740e22640 1 -- 172.21.15.38:0/1717276092 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe73c08bb20 msgr2=0x7fe73c16b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:11.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.029+0000 7fe740e22640 1 --2- 172.21.15.38:0/1717276092 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe73c08bb20 0x7fe73c16b340 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fe72c005fd0 tx=0x7fe72c02b640 comp rx=0 tx=0).stop 2024-09-21T03:33:11.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.030+0000 7fe740e22640 1 -- 172.21.15.38:0/1717276092 shutdown_connections 2024-09-21T03:33:11.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.030+0000 7fe740e22640 1 --2- 172.21.15.38:0/1717276092 >> v2:172.21.15.195:3300/0 conn(0x7fe73c0b5190 0x7fe73c06cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:11.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.030+0000 7fe740e22640 1 --2- 172.21.15.38:0/1717276092 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe73c08bb20 0x7fe73c16b340 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:11.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.030+0000 7fe740e22640 1 -- 172.21.15.38:0/1717276092 >> 172.21.15.38:0/1717276092 conn(0x7fe73c0cfec0 msgr2=0x7fe73c0d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:11.041 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'debug_mds', '20'], check_status=True, hostname='smithi038') 2024-09-21T03:33:11.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.030+0000 7fe740e22640 1 -- 172.21.15.38:0/1717276092 shutdown_connections 2024-09-21T03:33:11.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.030+0000 7fe740e22640 1 -- 172.21.15.38:0/1717276092 wait complete. 2024-09-21T03:33:11.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.034+0000 7fe740e22640 1 Processor -- start 2024-09-21T03:33:11.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.035+0000 7fe740e22640 1 -- start start 2024-09-21T03:33:11.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.040+0000 7fe740e22640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe73c06e530 0x7fe73c0aed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:11.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.040+0000 7fe740e22640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe73c0af260 0x7fe73c2094a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:11.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.041+0000 7fe73affd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe73c0af260 0x7fe73c2094a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:11.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.041+0000 7fe73affd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe73c0af260 0x7fe73c2094a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41346/0 (socket says 172.21.15.38:41346) 2024-09-21T03:33:11.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.041+0000 7fe73affd640 1 -- 172.21.15.38:0/1364528189 learned_addr learned my addr 172.21.15.38:0/1364528189 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:33:11.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.041+0000 7fe740e22640 1 --2- 172.21.15.38:0/1364528189 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe73c0af670 0x7fe73c2099e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:11.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.041+0000 7fe740e22640 1 -- 172.21.15.38:0/1364528189 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fe73c06d940 con 0x7fe73c0af260 2024-09-21T03:33:11.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.041+0000 7fe740e22640 1 -- 172.21.15.38:0/1364528189 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fe73c06d7c0 con 0x7fe73c06e530 2024-09-21T03:33:11.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.041+0000 7fe740e22640 1 -- 172.21.15.38:0/1364528189 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fe73c06dac0 con 0x7fe73c0af670 2024-09-21T03:33:11.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.041+0000 7fe73affd640 1 -- 172.21.15.38:0/1364528189 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe73c0af670 msgr2=0x7fe73c2099e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T03:33:11.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.041+0000 7fe73affd640 1 --2- 172.21.15.38:0/1364528189 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe73c0af670 0x7fe73c2099e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:11.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.041+0000 7fe73affd640 1 -- 172.21.15.38:0/1364528189 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe73c06e530 msgr2=0x7fe73c0aed20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T03:33:11.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.041+0000 7fe73affd640 1 --2- 172.21.15.38:0/1364528189 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe73c06e530 0x7fe73c0aed20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:11.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.041+0000 7fe73affd640 1 -- 172.21.15.38:0/1364528189 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe73c169c40 con 0x7fe73c0af260 2024-09-21T03:33:11.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.043+0000 7fe73affd640 1 --2- 172.21.15.38:0/1364528189 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe73c0af260 0x7fe73c2094a0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fe73400caf0 tx=0x7fe73400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:11.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.044+0000 7fe723fff640 1 -- 172.21.15.38:0/1364528189 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe734017070 con 0x7fe73c0af260 2024-09-21T03:33:11.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.044+0000 7fe740e22640 1 -- 172.21.15.38:0/1364528189 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe73c169f20 con 0x7fe73c0af260 2024-09-21T03:33:11.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.045+0000 7fe740e22640 1 -- 172.21.15.38:0/1364528189 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe73c0ab9d0 con 0x7fe73c0af260 2024-09-21T03:33:11.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.046+0000 7fe723fff640 1 -- 172.21.15.38:0/1364528189 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe734003c60 con 0x7fe73c0af260 2024-09-21T03:33:11.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.046+0000 7fe723fff640 1 -- 172.21.15.38:0/1364528189 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe73400aaf0 con 0x7fe73c0af260 2024-09-21T03:33:11.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.046+0000 7fe723fff640 1 -- 172.21.15.38:0/1364528189 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fe73401a070 con 0x7fe73c0af260 2024-09-21T03:33:11.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.046+0000 7fe723fff640 1 -- 172.21.15.38:0/1364528189 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fe7340128d0 con 0x7fe73c0af260 2024-09-21T03:33:11.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.053+0000 7fe721ffb640 1 -- 172.21.15.38:0/1364528189 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe71c005290 con 0x7fe73c0af260 2024-09-21T03:33:11.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.057+0000 7fe723fff640 1 -- 172.21.15.38:0/1364528189 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fe73400ac90 con 0x7fe73c0af260 2024-09-21T03:33:11.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.123+0000 7fd13a7da640 1 -- 172.21.15.38:0/1438027780 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd12c0e7200 msgr2=0x7fd12c005e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:11.126 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.123+0000 7fd13a7da640 1 --2- 172.21.15.38:0/1438027780 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd12c0e7200 0x7fd12c005e10 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fd12c00eef0 tx=0x7fd12802a510 comp rx=0 tx=0).stop 2024-09-21T03:33:11.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.125+0000 7fd13a7da640 1 -- 172.21.15.38:0/1438027780 shutdown_connections 2024-09-21T03:33:11.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.125+0000 7fd13a7da640 1 --2- 172.21.15.38:0/1438027780 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd12c0e7200 0x7fd12c005e10 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:11.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.125+0000 7fd13a7da640 1 -- 172.21.15.38:0/1438027780 >> 172.21.15.38:0/1438027780 conn(0x7fd12c01d9d0 msgr2=0x7fd12c039690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:11.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.125+0000 7fd13a7da640 1 -- 172.21.15.38:0/1438027780 shutdown_connections 2024-09-21T03:33:11.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.125+0000 7fd13a7da640 1 -- 172.21.15.38:0/1438027780 wait complete. 2024-09-21T03:33:11.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.128+0000 7fd13a7da640 1 Processor -- start 2024-09-21T03:33:11.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.129+0000 7fd13a7da640 1 -- start start 2024-09-21T03:33:11.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.131+0000 7fd13a7da640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd12c006350 0x7fd12c0d7580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:11.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.131+0000 7fd13a7da640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd12c052390 0x7fd12c0df0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:11.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.131+0000 7fd13a7da640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd12c0e7200 0x7fd12c031d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:11.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.131+0000 7fd13a7da640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fd12c006760 con 0x7fd12c006350 2024-09-21T03:33:11.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.131+0000 7fd13a7da640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd12c0e3650 con 0x7fd12c0e7200 2024-09-21T03:33:11.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.131+0000 7fd13a7da640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fd12c0068e0 con 0x7fd12c052390 2024-09-21T03:33:11.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.143+0000 7fd133fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd12c006350 0x7fd12c0d7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:11.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.143+0000 7fd133fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd12c006350 0x7fd12c0d7580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41352/0 (socket says 172.21.15.38:41352) 2024-09-21T03:33:11.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.143+0000 7fd133fff640 1 -- 172.21.15.38:0/1547583706 learned_addr learned my addr 172.21.15.38:0/1547583706 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:33:11.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.143+0000 7fd138d77640 1 --2- 172.21.15.38:0/1547583706 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd12c0e7200 0x7fd12c031d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:11.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.144+0000 7fd1337fe640 1 --2- 172.21.15.38:0/1547583706 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd12c052390 0x7fd12c0df0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:11.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.144+0000 7fd133fff640 1 -- 172.21.15.38:0/1547583706 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd12c052390 msgr2=0x7fd12c0df0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:11.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.144+0000 7fd133fff640 1 --2- 172.21.15.38:0/1547583706 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd12c052390 0x7fd12c0df0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:11.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.144+0000 7fd133fff640 1 -- 172.21.15.38:0/1547583706 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd12c0e7200 msgr2=0x7fd12c031d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:11.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.144+0000 7fd133fff640 1 --2- 172.21.15.38:0/1547583706 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd12c0e7200 0x7fd12c031d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:11.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.144+0000 7fd133fff640 1 -- 172.21.15.38:0/1547583706 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd12c0dc9a0 con 0x7fd12c006350 2024-09-21T03:33:11.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.145+0000 7fd133fff640 1 --2- 172.21.15.38:0/1547583706 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd12c006350 0x7fd12c0d7580 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fd120001eb0 tx=0x7fd12000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:11.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.145+0000 7fd1337fe640 1 --2- 172.21.15.38:0/1547583706 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd12c052390 0x7fd12c0df0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:33:11.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.146+0000 7fd138d77640 1 --2- 172.21.15.38:0/1547583706 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd12c0e7200 0x7fd12c031d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:33:11.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.146+0000 7fd130ff9640 1 -- 172.21.15.38:0/1547583706 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd120011790 con 0x7fd12c006350 2024-09-21T03:33:11.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.146+0000 7fd13a7da640 1 -- 172.21.15.38:0/1547583706 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd12c0dce10 con 0x7fd12c006350 2024-09-21T03:33:11.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.146+0000 7fd13a7da640 1 -- 172.21.15.38:0/1547583706 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd12c031510 con 0x7fd12c006350 2024-09-21T03:33:11.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.148+0000 7fd130ff9640 1 -- 172.21.15.38:0/1547583706 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd120017070 con 0x7fd12c006350 2024-09-21T03:33:11.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.148+0000 7fd130ff9640 1 -- 172.21.15.38:0/1547583706 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd120011cc0 con 0x7fd12c006350 2024-09-21T03:33:11.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.149+0000 7fd130ff9640 1 -- 172.21.15.38:0/1547583706 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fd12001c460 con 0x7fd12c006350 2024-09-21T03:33:11.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.150+0000 7fd130ff9640 1 -- 172.21.15.38:0/1547583706 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fd120012870 con 0x7fd12c006350 2024-09-21T03:33:11.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.150+0000 7fd11e7fc640 1 -- 172.21.15.38:0/1547583706 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd114005290 con 0x7fd12c006350 2024-09-21T03:33:11.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.151+0000 7f9008f62640 1 -- 172.21.15.38:0/3263739705 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_command([{prefix=config set, name=debug_mds}] v 0) -- 0x7f8fdc0052f0 con 0x7f9004157050 2024-09-21T03:33:11.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.154+0000 7fd130ff9640 1 -- 172.21.15.38:0/1547583706 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fd120012420 con 0x7fd12c006350 2024-09-21T03:33:11.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.160+0000 7f8ff2ffd640 1 -- 172.21.15.38:0/3263739705 <== mon.2 v2:172.21.15.195:3301/0 12 ==== mon_command_ack([{prefix=config set, name=debug_mds}]=0 v7)=0 v7) ==== 108+0+0 (secure 0 0 0) 0x7f8ffc01bc40 con 0x7f9004157050 2024-09-21T03:33:11.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.162+0000 7f8ff0ff9640 1 -- 172.21.15.38:0/3263739705 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9004157050 msgr2=0x7f90040b4050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:11.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.162+0000 7f8ff0ff9640 1 --2- 172.21.15.38:0/3263739705 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9004157050 0x7f90040b4050 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f8ffc00c950 tx=0x7f8ffc00ce10 comp rx=0 tx=0).stop 2024-09-21T03:33:11.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.162+0000 7f8ff0ff9640 1 -- 172.21.15.38:0/3263739705 shutdown_connections 2024-09-21T03:33:11.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.162+0000 7f8ff0ff9640 1 --2- 172.21.15.38:0/3263739705 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9004157050 0x7f90040b4050 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:11.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.163+0000 7f8ff0ff9640 1 --2- 172.21.15.38:0/3263739705 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9004160300 0x7f90040b3b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:11.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.163+0000 7f8ff0ff9640 1 --2- 172.21.15.38:0/3263739705 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f900406fed0 0x7f9004156350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:11.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.163+0000 7f8ff0ff9640 1 -- 172.21.15.38:0/3263739705 >> 172.21.15.38:0/3263739705 conn(0x7f90040b1b90 msgr2=0x7f9004073df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:11.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.163+0000 7f8ff0ff9640 1 -- 172.21.15.38:0/3263739705 shutdown_connections 2024-09-21T03:33:11.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.163+0000 7f8ff0ff9640 1 -- 172.21.15.38:0/3263739705 wait complete. 2024-09-21T03:33:11.211 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'debug_mds_balancer', '20'], check_status=True, hostname='smithi038') 2024-09-21T03:33:11.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.219+0000 7ff28e67e640 1 -- 172.21.15.38:0/830217705 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command([{prefix=config set, name=debug_mds_balancer}] v 0) -- 0x7ff2680016f0 con 0x7ff2880cfac0 2024-09-21T03:33:11.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.226+0000 7ff284ff9640 1 -- 172.21.15.38:0/830217705 <== mon.0 v2:172.21.15.38:3300/0 11 ==== mon_command_ack([{prefix=config set, name=debug_mds_balancer}]=0 v8)=0 v8) ==== 117+0+0 (secure 0 0 0) 0x7ff27800e800 con 0x7ff2880cfac0 2024-09-21T03:33:11.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.229+0000 7ff26e7fc640 1 -- 172.21.15.38:0/830217705 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff2880cfac0 msgr2=0x7ff28809e380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:11.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.229+0000 7ff26e7fc640 1 --2- 172.21.15.38:0/830217705 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff2880cfac0 0x7ff28809e380 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7ff27800c570 tx=0x7ff27800ca30 comp rx=0 tx=0).stop 2024-09-21T03:33:11.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.230+0000 7ff26e7fc640 1 -- 172.21.15.38:0/830217705 shutdown_connections 2024-09-21T03:33:11.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.230+0000 7ff26e7fc640 1 --2- 172.21.15.38:0/830217705 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff2880cfac0 0x7ff28809e380 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:11.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.230+0000 7ff26e7fc640 1 --2- 172.21.15.38:0/830217705 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff2880cf6b0 0x7ff2881f8590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:11.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.230+0000 7ff26e7fc640 1 --2- 172.21.15.38:0/830217705 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff288073000 0x7ff28809c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:11.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.230+0000 7ff26e7fc640 1 -- 172.21.15.38:0/830217705 >> 172.21.15.38:0/830217705 conn(0x7ff2880b1b90 msgr2=0x7ff2880b1460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:11.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.230+0000 7ff26e7fc640 1 -- 172.21.15.38:0/830217705 shutdown_connections 2024-09-21T03:33:11.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.230+0000 7ff26e7fc640 1 -- 172.21.15.38:0/830217705 wait complete. 2024-09-21T03:33:11.272 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'debug_ms', '2'], check_status=True, hostname='smithi038') 2024-09-21T03:33:11.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.286+0000 7f030759e640 1 -- 172.21.15.38:0/2816435765 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f030806e530 msgr2=0x7f03080b4a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:11.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.286+0000 7f030759e640 1 --2- 172.21.15.38:0/2816435765 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f030806e530 0x7f03080b4a80 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f0308071b30 tx=0x7f030002a850 comp rx=0 tx=0).stop 2024-09-21T03:33:11.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.289+0000 7f4ad10d5640 1 -- 172.21.15.38:0/4077462586 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4acc08b9e0 msgr2=0x7f4acc16ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:11.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.289+0000 7f4ad10d5640 1 --2- 172.21.15.38:0/4077462586 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4acc08b9e0 0x7f4acc16ec10 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f4abc0098b0 tx=0x7f4abc02a4b0 comp rx=0 tx=0).stop 2024-09-21T03:33:11.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.289+0000 7f0306d9d640 1 --2- 172.21.15.38:0/2816435765 >> v2:172.21.15.38:3300/0 conn(0x7f03080b4fc0 0x7f03080cddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:33:11.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.289+0000 7f030759e640 1 -- 172.21.15.38:0/2816435765 shutdown_connections 2024-09-21T03:33:11.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.289+0000 7f030759e640 1 --2- 172.21.15.38:0/2816435765 >> v2:172.21.15.38:3300/0 conn(0x7f03080b4fc0 0x7f03080cddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:11.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.289+0000 7f030759e640 1 --2- 172.21.15.38:0/2816435765 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f030806e530 0x7f03080b4a80 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:11.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.289+0000 7f030759e640 1 --2- 172.21.15.38:0/2816435765 >> v2:172.21.15.195:3300/0 conn(0x7f030808bb20 0x7f030816b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:11.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.289+0000 7f030759e640 1 -- 172.21.15.38:0/2816435765 >> 172.21.15.38:0/2816435765 conn(0x7f03080cfec0 msgr2=0x7f03080d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:11.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.290+0000 7f030759e640 1 -- 172.21.15.38:0/2816435765 shutdown_connections 2024-09-21T03:33:11.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.290+0000 7f4ad10d5640 1 -- 172.21.15.38:0/4077462586 shutdown_connections 2024-09-21T03:33:11.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.290+0000 7f4ad10d5640 1 --2- 172.21.15.38:0/4077462586 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4acc16f150 0x7f4acc0b3080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:11.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.290+0000 7f4ad10d5640 1 --2- 172.21.15.38:0/4077462586 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4acc08b9e0 0x7f4acc16ec10 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:11.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.290+0000 7f4ad10d5640 1 -- 172.21.15.38:0/4077462586 >> 172.21.15.38:0/4077462586 conn(0x7f4acc06cde0 msgr2=0x7f4acc06d1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:11.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.291+0000 7f4ad10d5640 1 -- 172.21.15.38:0/4077462586 shutdown_connections 2024-09-21T03:33:11.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.298+0000 7f030759e640 1 -- 172.21.15.38:0/2816435765 wait complete. 2024-09-21T03:33:11.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.301+0000 7f030759e640 1 Processor -- start 2024-09-21T03:33:11.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.299+0000 7f4ad10d5640 1 -- 172.21.15.38:0/4077462586 wait complete. 2024-09-21T03:33:11.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.302+0000 7f4ad10d5640 1 Processor -- start 2024-09-21T03:33:11.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.302+0000 7f4ad10d5640 1 -- start start 2024-09-21T03:33:11.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.304+0000 7f4ad10d5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4acc073070 0x7f4acc167000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:11.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.304+0000 7f4ad10d5640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4acc08b9e0 0x7f4acc212a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:11.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.305+0000 7f4ad10d5640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4acc16f150 0x7f4acc212fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:11.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.303+0000 7f030759e640 1 -- start start 2024-09-21T03:33:11.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.305+0000 7f4ad10d5640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f4acc0b3950 con 0x7f4acc073070 2024-09-21T03:33:11.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.305+0000 7f4ad10d5640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f4acc0b37d0 con 0x7f4acc16f150 2024-09-21T03:33:11.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.305+0000 7f4ad10d5640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f4acc0b3ad0 con 0x7f4acc08b9e0 2024-09-21T03:33:11.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.305+0000 7f4acad9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4acc073070 0x7f4acc167000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:11.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.305+0000 7f4acad9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4acc073070 0x7f4acc167000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41384/0 (socket says 172.21.15.38:41384) 2024-09-21T03:33:11.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.306+0000 7f4acad9d640 1 -- 172.21.15.38:0/3724294675 learned_addr learned my addr 172.21.15.38:0/3724294675 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:33:11.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.306+0000 7f4acb59e640 1 --2- 172.21.15.38:0/3724294675 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4acc16f150 0x7f4acc212fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:11.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.306+0000 7f4aca59c640 1 --2- 172.21.15.38:0/3724294675 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4acc08b9e0 0x7f4acc212a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:11.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.306+0000 7f4acad9d640 1 -- 172.21.15.38:0/3724294675 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4acc08b9e0 msgr2=0x7f4acc212a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:11.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.306+0000 7f4acad9d640 1 --2- 172.21.15.38:0/3724294675 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4acc08b9e0 0x7f4acc212a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:11.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.306+0000 7f4acad9d640 1 -- 172.21.15.38:0/3724294675 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4acc16f150 msgr2=0x7f4acc212fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:11.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.306+0000 7f4acad9d640 1 --2- 172.21.15.38:0/3724294675 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4acc16f150 0x7f4acc212fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:11.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.306+0000 7f4acad9d640 1 -- 172.21.15.38:0/3724294675 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4acc169740 con 0x7f4acc073070 2024-09-21T03:33:11.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.305+0000 7f030759e640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f030806e530 0x7f03082133a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:11.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.305+0000 7f030759e640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f030808bb20 0x7f03080cc760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:11.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.305+0000 7f030759e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f03080b4fc0 0x7f030820cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:11.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.305+0000 7f030759e640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f03080ce4c0 con 0x7f03080b4fc0 2024-09-21T03:33:11.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.307+0000 7f4acad9d640 1 --2- 172.21.15.38:0/3724294675 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4acc073070 0x7f4acc167000 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f4ac400cb40 tx=0x7f4ac400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:11.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.305+0000 7f030759e640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f03080ce340 con 0x7f030806e530 2024-09-21T03:33:11.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.305+0000 7f030759e640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f03080ce640 con 0x7f030808bb20 2024-09-21T03:33:11.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.308+0000 7f4ab37fe640 1 -- 172.21.15.38:0/3724294675 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ac4017070 con 0x7f4acc073070 2024-09-21T03:33:11.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.308+0000 7f030659c640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f030806e530 0x7f03082133a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:11.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.308+0000 7f4ab37fe640 1 -- 172.21.15.38:0/3724294675 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4ac400a890 con 0x7f4acc073070 2024-09-21T03:33:11.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.308+0000 7f4ad10d5640 1 -- 172.21.15.38:0/3724294675 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4acc169a20 con 0x7f4acc073070 2024-09-21T03:33:11.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.308+0000 7f030659c640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f030806e530 0x7f03082133a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.38:60282/0 (socket says 172.21.15.38:60282) 2024-09-21T03:33:11.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.308+0000 7f030659c640 1 -- 172.21.15.38:0/2998531551 learned_addr learned my addr 172.21.15.38:0/2998531551 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:33:11.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.309+0000 7f4ad10d5640 1 -- 172.21.15.38:0/3724294675 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4acc167540 con 0x7f4acc073070 2024-09-21T03:33:11.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.309+0000 7f0305d9b640 1 --2- 172.21.15.38:0/2998531551 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f030808bb20 0x7f03080cc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:11.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.309+0000 7f4ab37fe640 1 -- 172.21.15.38:0/3724294675 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ac400acc0 con 0x7f4acc073070 2024-09-21T03:33:11.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.311+0000 7f0306d9d640 1 --2- 172.21.15.38:0/2998531551 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f03080b4fc0 0x7f030820cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:11.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.311+0000 7f030659c640 1 -- 172.21.15.38:0/2998531551 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f030808bb20 msgr2=0x7f03080cc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:11.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.311+0000 7f030659c640 1 --2- 172.21.15.38:0/2998531551 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f030808bb20 0x7f03080cc760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:11.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.311+0000 7f030659c640 1 -- 172.21.15.38:0/2998531551 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f03080b4fc0 msgr2=0x7f030820cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:11.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.311+0000 7f030659c640 1 --2- 172.21.15.38:0/2998531551 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f03080b4fc0 0x7f030820cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:11.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.311+0000 7f030659c640 1 -- 172.21.15.38:0/2998531551 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0308213d80 con 0x7f030806e530 2024-09-21T03:33:11.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.311+0000 7f0306d9d640 1 --2- 172.21.15.38:0/2998531551 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f03080b4fc0 0x7f030820cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:33:11.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.312+0000 7f030659c640 1 --2- 172.21.15.38:0/2998531551 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f030806e530 0x7f03082133a0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f02f800e840 tx=0x7f02f800ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:11.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.312+0000 7f02f6ffd640 1 -- 172.21.15.38:0/2998531551 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02f800cc60 con 0x7f030806e530 2024-09-21T03:33:11.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.312+0000 7f02f6ffd640 1 -- 172.21.15.38:0/2998531551 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f02f8005020 con 0x7f030806e530 2024-09-21T03:33:11.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.312+0000 7f030759e640 1 -- 172.21.15.38:0/2998531551 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0308214060 con 0x7f030806e530 2024-09-21T03:33:11.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.313+0000 7f030759e640 1 -- 172.21.15.38:0/2998531551 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03080c52c0 con 0x7f030806e530 2024-09-21T03:33:11.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.313+0000 7f02f6ffd640 1 -- 172.21.15.38:0/2998531551 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02f8005470 con 0x7f030806e530 2024-09-21T03:33:11.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.314+0000 7f02f6ffd640 1 -- 172.21.15.38:0/2998531551 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f02f80047a0 con 0x7f030806e530 2024-09-21T03:33:11.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.314+0000 7f02f6ffd640 1 -- 172.21.15.38:0/2998531551 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f02f8016c90 con 0x7f030806e530 2024-09-21T03:33:11.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.314+0000 7f4ab37fe640 1 -- 172.21.15.38:0/3724294675 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f4ac401a070 con 0x7f4acc073070 2024-09-21T03:33:11.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.316+0000 7f030759e640 1 -- 172.21.15.38:0/2998531551 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02ec005290 con 0x7f030806e530 2024-09-21T03:33:11.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.316+0000 7f782e377640 1 -- 172.21.15.38:0/2720599130 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7828073070 msgr2=0x7f782808b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:11.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.316+0000 7f782e377640 1 --2- 172.21.15.38:0/2720599130 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7828073070 0x7f782808b9e0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f7818009830 tx=0x7f781802a530 comp rx=0 tx=0).stop 2024-09-21T03:33:11.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.316+0000 7f782e377640 1 -- 172.21.15.38:0/2720599130 shutdown_connections 2024-09-21T03:33:11.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.316+0000 7f782e377640 1 --2- 172.21.15.38:0/2720599130 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7828073070 0x7f782808b9e0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:11.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.316+0000 7f782e377640 1 -- 172.21.15.38:0/2720599130 >> 172.21.15.38:0/2720599130 conn(0x7f782806cde0 msgr2=0x7f782806d1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:11.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.321+0000 7f782e377640 1 -- 172.21.15.38:0/2720599130 shutdown_connections 2024-09-21T03:33:11.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.325+0000 7f4ab37fe640 1 -- 172.21.15.38:0/3724294675 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f4ac401ca70 con 0x7f4acc073070 2024-09-21T03:33:11.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.325+0000 7f4ad10d5640 1 -- 172.21.15.38:0/3724294675 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4aac005290 con 0x7f4acc073070 2024-09-21T03:33:11.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.327+0000 7f02f6ffd640 1 -- 172.21.15.38:0/2998531551 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f02f8013070 con 0x7f030806e530 2024-09-21T03:33:11.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.321+0000 7f782e377640 1 -- 172.21.15.38:0/2720599130 wait complete. 2024-09-21T03:33:11.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.324+0000 7f782e377640 1 Processor -- start 2024-09-21T03:33:11.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.324+0000 7f782e377640 1 -- start start 2024-09-21T03:33:11.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.327+0000 7f782e377640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f782806edf0 0x7f78281fa6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:11.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.327+0000 7f782e377640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f78281644b0 0x7f78280cf250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:11.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.327+0000 7f782e377640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f78281648c0 0x7f78280cf790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:11.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.327+0000 7f782e377640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f78280b3b20 con 0x7f782806edf0 2024-09-21T03:33:11.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.327+0000 7f782e377640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f78280b39a0 con 0x7f78281648c0 2024-09-21T03:33:11.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.327+0000 7f782e377640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f78280b3ca0 con 0x7f78281644b0 2024-09-21T03:33:11.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.332+0000 7f7827fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f782806edf0 0x7f78281fa6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:11.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.332+0000 7f7827fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f782806edf0 0x7f78281fa6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41408/0 (socket says 172.21.15.38:41408) 2024-09-21T03:33:11.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.332+0000 7f7827fff640 1 -- 172.21.15.38:0/1825845608 learned_addr learned my addr 172.21.15.38:0/1825845608 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:33:11.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.332+0000 7f7827fff640 1 -- 172.21.15.38:0/1825845608 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f78281644b0 msgr2=0x7f78280cf250 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T03:33:11.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.332+0000 7f7827fff640 1 --2- 172.21.15.38:0/1825845608 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f78281644b0 0x7f78280cf250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:11.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.332+0000 7f7827fff640 1 -- 172.21.15.38:0/1825845608 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f78281648c0 msgr2=0x7f78280cf790 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T03:33:11.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.332+0000 7f7827fff640 1 --2- 172.21.15.38:0/1825845608 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f78281648c0 0x7f78280cf790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:11.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.332+0000 7f7827fff640 1 -- 172.21.15.38:0/1825845608 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7828204fd0 con 0x7f782806edf0 2024-09-21T03:33:11.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.333+0000 7f7827fff640 1 --2- 172.21.15.38:0/1825845608 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f782806edf0 0x7f78281fa6f0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f7818009950 tx=0x7f78180038b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:11.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.334+0000 7f7824ff9640 1 -- 172.21.15.38:0/1825845608 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7818041070 con 0x7f782806edf0 2024-09-21T03:33:11.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.331+0000 7f4ab37fe640 1 -- 172.21.15.38:0/3724294675 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f4ac4021070 con 0x7f4acc073070 2024-09-21T03:33:11.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.334+0000 7f7824ff9640 1 -- 172.21.15.38:0/1825845608 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7818003bd0 con 0x7f782806edf0 2024-09-21T03:33:11.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.334+0000 7f7824ff9640 1 -- 172.21.15.38:0/1825845608 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f781803b440 con 0x7f782806edf0 2024-09-21T03:33:11.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.334+0000 7f782e377640 1 -- 172.21.15.38:0/1825845608 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78282052b0 con 0x7f782806edf0 2024-09-21T03:33:11.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.334+0000 7f782e377640 1 -- 172.21.15.38:0/1825845608 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7828205870 con 0x7f782806edf0 2024-09-21T03:33:11.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.337+0000 7f7c399cc640 1 -- 172.21.15.38:0/852164721 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7c340b3f90 msgr2=0x7f7c34073bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:11.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.337+0000 7f780e7fc640 1 -- 172.21.15.38:0/1825845608 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7808005290 con 0x7f782806edf0 2024-09-21T03:33:11.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.341+0000 7f7824ff9640 1 -- 172.21.15.38:0/1825845608 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f7818044070 con 0x7f782806edf0 2024-09-21T03:33:11.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.341+0000 7f7824ff9640 1 -- 172.21.15.38:0/1825845608 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f7818046790 con 0x7f782806edf0 2024-09-21T03:33:11.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.341+0000 7f7824ff9640 1 -- 172.21.15.38:0/1825845608 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f781803bdf0 con 0x7f782806edf0 2024-09-21T03:33:11.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.337+0000 7f7c399cc640 1 --2- 172.21.15.38:0/852164721 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7c340b3f90 0x7f7c34073bd0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f7c24005fd0 tx=0x7f7c2402a410 comp rx=0 tx=0).stop 2024-09-21T03:33:11.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.343+0000 7f7c399cc640 1 -- 172.21.15.38:0/852164721 shutdown_connections 2024-09-21T03:33:11.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.343+0000 7f7c399cc640 1 --2- 172.21.15.38:0/852164721 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7c340b3f90 0x7f7c34073bd0 secure :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f7c24005fd0 tx=0x7f7c2402a410 comp rx=0 tx=0).stop 2024-09-21T03:33:11.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.343+0000 7f7c399cc640 1 --2- 172.21.15.38:0/852164721 >> v2:172.21.15.195:3300/0 conn(0x7f7c3406edf0 0x7f7c340b3a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:11.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.343+0000 7f7c399cc640 1 -- 172.21.15.38:0/852164721 >> 172.21.15.38:0/852164721 conn(0x7f7c3406cde0 msgr2=0x7f7c3406d1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:11.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.347+0000 7fe721ffb640 1 -- 172.21.15.38:0/1364528189 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command([{prefix=config set, name=mds_debug_frag}] v 0) -- 0x7fe71c005850 con 0x7fe73c0af260 2024-09-21T03:33:11.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.355+0000 7fe723fff640 1 -- 172.21.15.38:0/1364528189 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_debug_frag}]=0 v9)=0 v9) ==== 115+0+0 (secure 0 0 0) 0x7fe734012ad0 con 0x7fe73c0af260 2024-09-21T03:33:11.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.357+0000 7f7c399cc640 1 -- 172.21.15.38:0/852164721 shutdown_connections 2024-09-21T03:33:11.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.357+0000 7f7c399cc640 1 -- 172.21.15.38:0/852164721 wait complete. 2024-09-21T03:33:11.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.360+0000 7f7c399cc640 1 Processor -- start 2024-09-21T03:33:11.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.360+0000 7f7c399cc640 1 -- start start 2024-09-21T03:33:11.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.362+0000 7f7c399cc640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7c3406edf0 0x7f7c34159520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:11.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.362+0000 7f7c399cc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7c34074110 0x7f7c34160710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:11.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.362+0000 7f7c399cc640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7c34206540 0x7f7c34205930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:11.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.362+0000 7f7c399cc640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f7c340af780 con 0x7f7c34074110 2024-09-21T03:33:11.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.362+0000 7f7c399cc640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f7c340af600 con 0x7f7c3406edf0 2024-09-21T03:33:11.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.362+0000 7f7c399cc640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f7c34159de0 con 0x7f7c34206540 2024-09-21T03:33:11.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.363+0000 7f7c32ffd640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7c3406edf0 0x7f7c34159520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:11.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.363+0000 7f7c32ffd640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7c3406edf0 0x7f7c34159520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.38:60286/0 (socket says 172.21.15.38:60286) 2024-09-21T03:33:11.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.363+0000 7f7c32ffd640 1 -- 172.21.15.38:0/660629611 learned_addr learned my addr 172.21.15.38:0/660629611 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:33:11.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.363+0000 7f7c337fe640 1 --2- 172.21.15.38:0/660629611 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7c34206540 0x7f7c34205930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:11.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.364+0000 7fe721ffb640 1 -- 172.21.15.38:0/1364528189 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe73c0af260 msgr2=0x7fe73c2094a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:11.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.364+0000 7fe721ffb640 1 --2- 172.21.15.38:0/1364528189 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe73c0af260 0x7fe73c2094a0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fe73400caf0 tx=0x7fe73400a060 comp rx=0 tx=0).stop 2024-09-21T03:33:11.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.364+0000 7fe721ffb640 1 -- 172.21.15.38:0/1364528189 shutdown_connections 2024-09-21T03:33:11.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.364+0000 7fe721ffb640 1 --2- 172.21.15.38:0/1364528189 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe73c0af670 0x7fe73c2099e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:11.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.364+0000 7fe721ffb640 1 --2- 172.21.15.38:0/1364528189 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe73c0af260 0x7fe73c2094a0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:11.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.364+0000 7fe721ffb640 1 --2- 172.21.15.38:0/1364528189 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe73c06e530 0x7fe73c0aed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:11.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.364+0000 7fe721ffb640 1 -- 172.21.15.38:0/1364528189 >> 172.21.15.38:0/1364528189 conn(0x7fe73c0cfec0 msgr2=0x7fe73c16a890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:11.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.363+0000 7f7c32ffd640 1 -- 172.21.15.38:0/660629611 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7c34206540 msgr2=0x7f7c34205930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:11.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.363+0000 7f7c32ffd640 1 --2- 172.21.15.38:0/660629611 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7c34206540 0x7f7c34205930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:11.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.363+0000 7f7c32ffd640 1 -- 172.21.15.38:0/660629611 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7c34074110 msgr2=0x7f7c34160710 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T03:33:11.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.363+0000 7f7c32ffd640 1 --2- 172.21.15.38:0/660629611 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7c34074110 0x7f7c34160710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:11.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.363+0000 7f7c32ffd640 1 -- 172.21.15.38:0/660629611 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c34160c50 con 0x7f7c3406edf0 2024-09-21T03:33:11.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.364+0000 7fe721ffb640 1 -- 172.21.15.38:0/1364528189 shutdown_connections 2024-09-21T03:33:11.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.364+0000 7f7c32ffd640 1 --2- 172.21.15.38:0/660629611 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7c3406edf0 0x7f7c34159520 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f7c2800d720 tx=0x7f7c2800dbe0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:11.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.364+0000 7fe721ffb640 1 -- 172.21.15.38:0/1364528189 wait complete. 2024-09-21T03:33:11.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.365+0000 7f7c1bfff640 1 -- 172.21.15.38:0/660629611 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c28017070 con 0x7f7c3406edf0 2024-09-21T03:33:11.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.365+0000 7f7c399cc640 1 -- 172.21.15.38:0/660629611 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c340ab7d0 con 0x7f7c3406edf0 2024-09-21T03:33:11.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.365+0000 7f7c399cc640 1 -- 172.21.15.38:0/660629611 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c340abd90 con 0x7f7c3406edf0 2024-09-21T03:33:11.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.366+0000 7f7c1bfff640 1 -- 172.21.15.38:0/660629611 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7c28009ab0 con 0x7f7c3406edf0 2024-09-21T03:33:11.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.366+0000 7f7c1bfff640 1 -- 172.21.15.38:0/660629611 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c28010460 con 0x7f7c3406edf0 2024-09-21T03:33:11.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.368+0000 7f7c1bfff640 1 -- 172.21.15.38:0/660629611 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f7c2801a070 con 0x7f7c3406edf0 2024-09-21T03:33:11.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.368+0000 7f7c1bfff640 1 -- 172.21.15.38:0/660629611 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f7c28003930 con 0x7f7c3406edf0 2024-09-21T03:33:11.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.370+0000 7f7c399cc640 1 -- 172.21.15.38:0/660629611 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c14005290 con 0x7f7c3406edf0 2024-09-21T03:33:11.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.377+0000 7fda1abda640 1 -- 172.21.15.38:0/810151798 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fda140ca6f0 msgr2=0x7fda140cf6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:11.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.377+0000 7fda1abda640 1 --2- 172.21.15.38:0/810151798 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fda140ca6f0 0x7fda140cf6e0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fda0c005fd0 tx=0x7fda0c02cb20 comp rx=0 tx=0).stop 2024-09-21T03:33:11.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.380+0000 7f7c1bfff640 1 -- 172.21.15.38:0/660629611 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f7c28010d90 con 0x7f7c3406edf0 2024-09-21T03:33:11.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.380+0000 7fda1abda640 1 -- 172.21.15.38:0/810151798 shutdown_connections 2024-09-21T03:33:11.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.380+0000 7fda1abda640 1 --2- 172.21.15.38:0/810151798 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fda140cc0b0 0x7fda140cc4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:11.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.380+0000 7fda1abda640 1 --2- 172.21.15.38:0/810151798 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fda140ca6f0 0x7fda140cf6e0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:11.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.380+0000 7fda1abda640 1 -- 172.21.15.38:0/810151798 >> 172.21.15.38:0/810151798 conn(0x7fda140b1b90 msgr2=0x7fda140b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:11.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.384+0000 7fa7da00a640 1 -- 172.21.15.38:0/2161108938 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa7d40cf0b0 msgr2=0x7fa7d40cf4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:11.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.384+0000 7fa7da00a640 1 --2- 172.21.15.38:0/2161108938 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa7d40cf0b0 0x7fa7d40cf4c0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fa7c8009830 tx=0x7fa7c802a440 comp rx=0 tx=0).stop 2024-09-21T03:33:11.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.384+0000 7fa7da00a640 1 -- 172.21.15.38:0/2161108938 shutdown_connections 2024-09-21T03:33:11.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.384+0000 7fa7da00a640 1 --2- 172.21.15.38:0/2161108938 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa7d40cf0b0 0x7fa7d40cf4c0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:11.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.384+0000 7fa7da00a640 1 --2- 172.21.15.38:0/2161108938 >> v2:172.21.15.195:3300/0 conn(0x7fa7d40ad850 0x7fa7d4155560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:11.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.384+0000 7fa7da00a640 1 -- 172.21.15.38:0/2161108938 >> 172.21.15.38:0/2161108938 conn(0x7fa7d40ccb40 msgr2=0x7fa7d40ccf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:11.391 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.389+0000 7fd11e7fc640 1 -- 172.21.15.38:0/1547583706 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command([{prefix=config set, name=mds_debug_scatterstat}] v 0) -- 0x7fd114005580 con 0x7fd12c006350 2024-09-21T03:33:11.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.390+0000 7f7adeffd640 1 -- 172.21.15.38:0/423395403 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_command([{prefix=config set, name=debug_ms}] v 0) -- 0x7f7ad8005580 con 0x7f7af80b3f90 2024-09-21T03:33:11.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.386+0000 7fa7da00a640 1 -- 172.21.15.38:0/2161108938 shutdown_connections 2024-09-21T03:33:11.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.380+0000 7fda1abda640 1 -- 172.21.15.38:0/810151798 shutdown_connections 2024-09-21T03:33:11.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.380+0000 7fda1abda640 1 -- 172.21.15.38:0/810151798 wait complete. 2024-09-21T03:33:11.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.383+0000 7fda1abda640 1 Processor -- start 2024-09-21T03:33:11.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.383+0000 7fda1abda640 1 -- start start 2024-09-21T03:33:11.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.386+0000 7fda1abda640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fda1406ddf0 0x7fda140c2b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:11.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.386+0000 7fda1abda640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fda140ca6f0 0x7fda140b4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:11.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.391+0000 7fda13fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fda140ca6f0 0x7fda140b4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:11.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.392+0000 7fda13fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fda140ca6f0 0x7fda140b4790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41420/0 (socket says 172.21.15.38:41420) 2024-09-21T03:33:11.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.392+0000 7fda18976640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fda1406ddf0 0x7fda140c2b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:11.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.392+0000 7fda18976640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fda1406ddf0 0x7fda140c2b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.38:60308/0 (socket says 172.21.15.38:60308) 2024-09-21T03:33:11.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.392+0000 7fda13fff640 1 -- 172.21.15.38:0/3337678306 learned_addr learned my addr 172.21.15.38:0/3337678306 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:33:11.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.392+0000 7fda1abda640 1 --2- 172.21.15.38:0/3337678306 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fda140cc0b0 0x7fda140b4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:11.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.392+0000 7fda1abda640 1 -- 172.21.15.38:0/3337678306 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fda14071000 con 0x7fda140ca6f0 2024-09-21T03:33:11.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.392+0000 7fda1abda640 1 -- 172.21.15.38:0/3337678306 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fda14070e80 con 0x7fda1406ddf0 2024-09-21T03:33:11.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.392+0000 7fda1abda640 1 -- 172.21.15.38:0/3337678306 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fda14071180 con 0x7fda140cc0b0 2024-09-21T03:33:11.398 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.396+0000 7fd130ff9640 1 -- 172.21.15.38:0/1547583706 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_debug_scatterstat}]=0 v10)=0 v10) ==== 122+0+0 (secure 0 0 0) 0x7fd120012ac0 con 0x7fd12c006350 2024-09-21T03:33:11.398 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.395+0000 7fa7da00a640 1 -- 172.21.15.38:0/2161108938 wait complete. 2024-09-21T03:33:11.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.398+0000 7fd13a7da640 1 -- 172.21.15.38:0/1547583706 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd12c006350 msgr2=0x7fd12c0d7580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:11.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.398+0000 7fd13a7da640 1 --2- 172.21.15.38:0/1547583706 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd12c006350 0x7fd12c0d7580 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fd120001eb0 tx=0x7fd12000c890 comp rx=0 tx=0).stop 2024-09-21T03:33:11.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.398+0000 7fd13a7da640 1 -- 172.21.15.38:0/1547583706 shutdown_connections 2024-09-21T03:33:11.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.398+0000 7fd13a7da640 1 --2- 172.21.15.38:0/1547583706 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd12c0e7200 0x7fd12c031d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:11.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.398+0000 7fd13a7da640 1 --2- 172.21.15.38:0/1547583706 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd12c052390 0x7fd12c0df0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:11.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.398+0000 7fd13a7da640 1 --2- 172.21.15.38:0/1547583706 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd12c006350 0x7fd12c0d7580 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:11.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.398+0000 7fa7da00a640 1 Processor -- start 2024-09-21T03:33:11.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.399+0000 7fd13a7da640 1 -- 172.21.15.38:0/1547583706 >> 172.21.15.38:0/1547583706 conn(0x7fd12c01d9d0 msgr2=0x7fd12c0e1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:11.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.399+0000 7fd13a7da640 1 -- 172.21.15.38:0/1547583706 shutdown_connections 2024-09-21T03:33:11.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.399+0000 7fd13a7da640 1 -- 172.21.15.38:0/1547583706 wait complete. 2024-09-21T03:33:11.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.398+0000 7fda19177640 1 --2- 172.21.15.38:0/3337678306 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fda140cc0b0 0x7fda140b4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:11.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.399+0000 7fa7da00a640 1 -- start start 2024-09-21T03:33:11.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.401+0000 7fa7da00a640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa7d40ad850 0x7fa7d4152fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:11.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.401+0000 7fa7da00a640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa7d40cf0b0 0x7fa7d4162090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:11.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.401+0000 7fa7da00a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa7d415ed20 0x7fa7d41625d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:11.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.401+0000 7fa7da00a640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa7d40703e0 con 0x7fa7d415ed20 2024-09-21T03:33:11.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.401+0000 7fa7da00a640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fa7d4070260 con 0x7fa7d40ad850 2024-09-21T03:33:11.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.401+0000 7fa7da00a640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fa7d4070560 con 0x7fa7d40cf0b0 2024-09-21T03:33:11.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.402+0000 7fa7d37fe640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa7d40ad850 0x7fa7d4152fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:11.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.402+0000 7fa7d37fe640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa7d40ad850 0x7fa7d4152fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.38:60332/0 (socket says 172.21.15.38:60332) 2024-09-21T03:33:11.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.402+0000 7fa7d37fe640 1 -- 172.21.15.38:0/1803892417 learned_addr learned my addr 172.21.15.38:0/1803892417 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:33:11.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.402+0000 7fda18976640 1 -- 172.21.15.38:0/3337678306 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fda140cc0b0 msgr2=0x7fda140b4cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:11.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.402+0000 7fda18976640 1 --2- 172.21.15.38:0/3337678306 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fda140cc0b0 0x7fda140b4cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:11.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.402+0000 7fda18976640 1 -- 172.21.15.38:0/3337678306 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fda140ca6f0 msgr2=0x7fda140b4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:11.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.403+0000 7fa7d37fe640 1 -- 172.21.15.38:0/1803892417 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa7d40cf0b0 msgr2=0x7fa7d4162090 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T03:33:11.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.403+0000 7fa7d37fe640 1 --2- 172.21.15.38:0/1803892417 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa7d40cf0b0 0x7fa7d4162090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:11.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.403+0000 7fa7d37fe640 1 -- 172.21.15.38:0/1803892417 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa7d415ed20 msgr2=0x7fa7d41625d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:11.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.403+0000 7fa7d37fe640 1 --2- 172.21.15.38:0/1803892417 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa7d415ed20 0x7fa7d41625d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:11.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.403+0000 7fa7d37fe640 1 -- 172.21.15.38:0/1803892417 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7d41589d0 con 0x7fa7d40ad850 2024-09-21T03:33:11.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.402+0000 7fda18976640 1 --2- 172.21.15.38:0/3337678306 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fda140ca6f0 0x7fda140b4790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:11.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.402+0000 7fda18976640 1 -- 172.21.15.38:0/3337678306 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda1409f3d0 con 0x7fda1406ddf0 2024-09-21T03:33:11.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.402+0000 7fda13fff640 1 --2- 172.21.15.38:0/3337678306 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fda140ca6f0 0x7fda140b4790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:33:11.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.403+0000 7fda18976640 1 --2- 172.21.15.38:0/3337678306 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fda1406ddf0 0x7fda140c2b40 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fda140b2840 tx=0x7fda0c02ef60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:11.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.405+0000 7f2b302b6640 1 -- 172.21.15.38:0/511402847 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2b2806ddf0 msgr2=0x7f2b2806e200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:11.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.405+0000 7f2b302b6640 1 --2- 172.21.15.38:0/511402847 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2b2806ddf0 0x7f2b2806e200 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f2b20005fd0 tx=0x7f2b2002a8e0 comp rx=0 tx=0).stop 2024-09-21T03:33:11.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.405+0000 7fa7d37fe640 1 --2- 172.21.15.38:0/1803892417 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa7d40ad850 0x7fa7d4152fd0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fa7c000c7d0 tx=0x7fa7c000cc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:11.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.405+0000 7fda117fa640 1 -- 172.21.15.38:0/3337678306 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda0c044030 con 0x7fda1406ddf0 2024-09-21T03:33:11.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.407+0000 7fa7bffff640 1 -- 172.21.15.38:0/1803892417 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7c0017070 con 0x7fa7d40ad850 2024-09-21T03:33:11.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.407+0000 7fa7bffff640 1 -- 172.21.15.38:0/1803892417 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa7c00035f0 con 0x7fa7d40ad850 2024-09-21T03:33:11.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.405+0000 7fda1abda640 1 -- 172.21.15.38:0/3337678306 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda14203550 con 0x7fda1406ddf0 2024-09-21T03:33:11.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.406+0000 7fda1abda640 1 -- 172.21.15.38:0/3337678306 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda1406feb0 con 0x7fda1406ddf0 2024-09-21T03:33:11.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.407+0000 7fa7da00a640 1 -- 172.21.15.38:0/1803892417 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7d4158cb0 con 0x7fa7d40ad850 2024-09-21T03:33:11.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.407+0000 7fa7da00a640 1 -- 172.21.15.38:0/1803892417 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7d415bd90 con 0x7fa7d40ad850 2024-09-21T03:33:11.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.407+0000 7fa7bffff640 1 -- 172.21.15.38:0/1803892417 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7c00039e0 con 0x7fa7d40ad850 2024-09-21T03:33:11.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.407+0000 7f5819feb640 1 -- 172.21.15.38:0/1300228153 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f581406d2b0 msgr2=0x7f58141621d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:11.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.407+0000 7f5819feb640 1 --2- 172.21.15.38:0/1300228153 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f581406d2b0 0x7f58141621d0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f5804009830 tx=0x7f580402a490 comp rx=0 tx=0).stop 2024-09-21T03:33:11.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.407+0000 7fda117fa640 1 -- 172.21.15.38:0/3337678306 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fda0c004b40 con 0x7fda1406ddf0 2024-09-21T03:33:11.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.407+0000 7fda117fa640 1 -- 172.21.15.38:0/3337678306 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda0c02f880 con 0x7fda1406ddf0 2024-09-21T03:33:11.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.407+0000 7fda117fa640 1 -- 172.21.15.38:0/3337678306 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fda0c047070 con 0x7fda1406ddf0 2024-09-21T03:33:11.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.407+0000 7fda117fa640 1 -- 172.21.15.38:0/3337678306 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fda0c040550 con 0x7fda1406ddf0 2024-09-21T03:33:11.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.407+0000 7fa7bffff640 1 -- 172.21.15.38:0/1803892417 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fa7c001a070 con 0x7fa7d40ad850 2024-09-21T03:33:11.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.407+0000 7fa7bffff640 1 -- 172.21.15.38:0/1803892417 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fa7c001c780 con 0x7fa7d40ad850 2024-09-21T03:33:11.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.408+0000 7f7af57fa640 1 -- 172.21.15.38:0/423395403 <== mon.2 v2:172.21.15.195:3301/0 7 ==== mon_command_ack([{prefix=config set, name=debug_ms}]=0 v11)=0 v11) ==== 106+0+0 (secure 0 0 0) 0x7f7af0003a40 con 0x7f7af80b3f90 2024-09-21T03:33:11.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.409+0000 7fd9faffd640 1 -- 172.21.15.38:0/3337678306 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9f4005290 con 0x7fda1406ddf0 2024-09-21T03:33:11.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.409+0000 7f5819feb640 1 -- 172.21.15.38:0/1300228153 shutdown_connections 2024-09-21T03:33:11.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.409+0000 7f5819feb640 1 --2- 172.21.15.38:0/1300228153 >> v2:172.21.15.38:3300/0 conn(0x7f5814162710 0x7f58140cf300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:11.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.409+0000 7f5819feb640 1 --2- 172.21.15.38:0/1300228153 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f581406d2b0 0x7f58141621d0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:11.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.409+0000 7f5819feb640 1 -- 172.21.15.38:0/1300228153 >> 172.21.15.38:0/1300228153 conn(0x7f58140d1960 msgr2=0x7f58140d42e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:11.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.411+0000 7f5819feb640 1 -- 172.21.15.38:0/1300228153 shutdown_connections 2024-09-21T03:33:11.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.411+0000 7f5819feb640 1 -- 172.21.15.38:0/1300228153 wait complete. 2024-09-21T03:33:11.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.414+0000 7f5819feb640 1 Processor -- start 2024-09-21T03:33:11.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.411+0000 7f7adeffd640 1 -- 172.21.15.38:0/423395403 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7af80b3f90 msgr2=0x7f7af815bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:11.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.411+0000 7f7adeffd640 1 --2- 172.21.15.38:0/423395403 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7af80b3f90 0x7f7af815bfa0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f7af001ce00 tx=0x7f7af001cef0 comp rx=0 tx=0).stop 2024-09-21T03:33:11.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.412+0000 7f2b302b6640 1 -- 172.21.15.38:0/511402847 shutdown_connections 2024-09-21T03:33:11.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.412+0000 7f2b302b6640 1 --2- 172.21.15.38:0/511402847 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2b2806ddf0 0x7f2b2806e200 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:11.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.412+0000 7f2b302b6640 1 --2- 172.21.15.38:0/511402847 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2b28074590 0x7f2b2815e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:11.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.412+0000 7f2b302b6640 1 -- 172.21.15.38:0/511402847 >> 172.21.15.38:0/511402847 conn(0x7f2b280735d0 msgr2=0x7f2b280739e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:11.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.412+0000 7f2b302b6640 1 -- 172.21.15.38:0/511402847 shutdown_connections 2024-09-21T03:33:11.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.412+0000 7fa7da00a640 1 -- 172.21.15.38:0/1803892417 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7b4005290 con 0x7fa7d40ad850 2024-09-21T03:33:11.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.414+0000 7f7adeffd640 1 -- 172.21.15.38:0/423395403 shutdown_connections 2024-09-21T03:33:11.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.414+0000 7f7adeffd640 1 --2- 172.21.15.38:0/423395403 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7af80b3f90 0x7f7af815bfa0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:11.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.414+0000 7f7adeffd640 1 --2- 172.21.15.38:0/423395403 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7af80aec20 0x7f7af815af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:11.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.414+0000 7f7adeffd640 1 --2- 172.21.15.38:0/423395403 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7af806edf0 0x7f7af813d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:11.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.414+0000 7f7adeffd640 1 -- 172.21.15.38:0/423395403 >> 172.21.15.38:0/423395403 conn(0x7f7af806cde0 msgr2=0x7f7af806d6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:11.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.414+0000 7f7adeffd640 1 -- 172.21.15.38:0/423395403 shutdown_connections 2024-09-21T03:33:11.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.414+0000 7f7adeffd640 1 -- 172.21.15.38:0/423395403 wait complete. 2024-09-21T03:33:11.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.415+0000 7f5819feb640 1 -- start start 2024-09-21T03:33:11.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.412+0000 7f2b302b6640 1 -- 172.21.15.38:0/511402847 wait complete. 2024-09-21T03:33:11.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.415+0000 7f2b302b6640 1 Processor -- start 2024-09-21T03:33:11.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.415+0000 7f2b302b6640 1 -- start start 2024-09-21T03:33:11.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.417+0000 7f5819feb640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f581406d2b0 0x7f58141613c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:11.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.417+0000 7fa7bffff640 1 -- 172.21.15.38:0/1803892417 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fa7c0021020 con 0x7fa7d40ad850 2024-09-21T03:33:11.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.418+0000 7fda117fa640 1 -- 172.21.15.38:0/3337678306 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fda0c034030 con 0x7fda1406ddf0 2024-09-21T03:33:11.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.418+0000 7f5819feb640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f58140cf840 0x7f5814161900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:11.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.419+0000 7f5818fe9640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f581406d2b0 0x7f58141613c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:11.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.419+0000 7f5818fe9640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f581406d2b0 0x7f58141613c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.38:60338/0 (socket says 172.21.15.38:60338) 2024-09-21T03:33:11.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.419+0000 7f5818fe9640 1 -- 172.21.15.38:0/2617562833 learned_addr learned my addr 172.21.15.38:0/2617562833 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:33:11.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.419+0000 7f5813fff640 1 --2- 172.21.15.38:0/2617562833 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f58140cf840 0x7f5814161900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:11.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.419+0000 7f5819feb640 1 --2- 172.21.15.38:0/2617562833 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5814162710 0x7f58141f79c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:11.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.419+0000 7f5819feb640 1 -- 172.21.15.38:0/2617562833 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f5814155980 con 0x7f5814162710 2024-09-21T03:33:11.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.419+0000 7f5819feb640 1 -- 172.21.15.38:0/2617562833 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f5814155800 con 0x7f581406d2b0 2024-09-21T03:33:11.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.419+0000 7f5819feb640 1 -- 172.21.15.38:0/2617562833 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f5814155b00 con 0x7f58140cf840 2024-09-21T03:33:11.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.422+0000 7f5813fff640 1 -- 172.21.15.38:0/2617562833 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f581406d2b0 msgr2=0x7f58141613c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:11.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.422+0000 7f5813fff640 1 --2- 172.21.15.38:0/2617562833 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f581406d2b0 0x7f58141613c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:11.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.422+0000 7f5813fff640 1 -- 172.21.15.38:0/2617562833 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5814162710 msgr2=0x7f58141f79c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T03:33:11.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.422+0000 7f5813fff640 1 --2- 172.21.15.38:0/2617562833 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5814162710 0x7f58141f79c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:11.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.422+0000 7f5813fff640 1 -- 172.21.15.38:0/2617562833 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f581415c330 con 0x7f58140cf840 2024-09-21T03:33:11.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.424+0000 7fee22d69640 1 -- 172.21.15.38:0/1665913663 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fee04003a50 msgr2=0x7fee04003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:11.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.424+0000 7fee22d69640 1 --2- 172.21.15.38:0/1665913663 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fee04003a50 0x7fee04003ec0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fee0c0016b0 tx=0x7fee04005830 comp rx=0 tx=0).stop 2024-09-21T03:33:11.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.428+0000 7fee22d69640 1 -- 172.21.15.38:0/1665913663 shutdown_connections 2024-09-21T03:33:11.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.428+0000 7fee22d69640 1 --2- 172.21.15.38:0/1665913663 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fee04003a50 0x7fee04003ec0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:11.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.428+0000 7fee22d69640 1 -- 172.21.15.38:0/1665913663 >> 172.21.15.38:0/1665913663 conn(0x7fee1c0b1b90 msgr2=0x7fee1c0b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:11.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.431+0000 7f5813fff640 1 --2- 172.21.15.38:0/2617562833 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f58140cf840 0x7f5814161900 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f580800e700 tx=0x7f580800ebc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:11.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.431+0000 7fee22d69640 1 -- 172.21.15.38:0/1665913663 shutdown_connections 2024-09-21T03:33:11.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.431+0000 7f58117fa640 1 -- 172.21.15.38:0/2617562833 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f580800cab0 con 0x7f58140cf840 2024-09-21T03:33:11.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.431+0000 7fee22d69640 1 -- 172.21.15.38:0/1665913663 wait complete. 2024-09-21T03:33:11.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.431+0000 7f5819feb640 1 -- 172.21.15.38:0/2617562833 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f581415c610 con 0x7f58140cf840 2024-09-21T03:33:11.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.431+0000 7f5819feb640 1 -- 172.21.15.38:0/2617562833 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f581415cbd0 con 0x7f58140cf840 2024-09-21T03:33:11.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.431+0000 7f58117fa640 1 -- 172.21.15.38:0/2617562833 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f580800cd40 con 0x7f58140cf840 2024-09-21T03:33:11.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.432+0000 7f58117fa640 1 -- 172.21.15.38:0/2617562833 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5808010690 con 0x7f58140cf840 2024-09-21T03:33:11.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.432+0000 7f58117fa640 1 -- 172.21.15.38:0/2617562833 <== mon.2 v2:172.21.15.195:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f5808010870 con 0x7f58140cf840 2024-09-21T03:33:11.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.432+0000 7f58117fa640 1 -- 172.21.15.38:0/2617562833 <== mon.2 v2:172.21.15.195:3301/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f5808018ce0 con 0x7f58140cf840 2024-09-21T03:33:11.437 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_debug_frag', 'True'], check_status=True, hostname='smithi038') 2024-09-21T03:33:11.438 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_debug_scatterstat', 'True'], check_status=True, hostname='smithi038') 2024-09-21T03:33:11.438 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_op_complaint_time', '180'], check_status=True, hostname='smithi038') 2024-09-21T03:33:11.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.435+0000 7f2b302b6640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2b2806ddf0 0x7f2b281588a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:11.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.435+0000 7f2b302b6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2b280711b0 0x7f2b281f8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:11.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.435+0000 7f2b302b6640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2b28074590 0x7f2b28205b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:11.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.435+0000 7f2b302b6640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f2b280cbf50 con 0x7f2b280711b0 2024-09-21T03:33:11.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.435+0000 7f2b302b6640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f2b280cbdd0 con 0x7f2b2806ddf0 2024-09-21T03:33:11.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.435+0000 7f2b302b6640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f2b280cc0d0 con 0x7f2b28074590 2024-09-21T03:33:11.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.437+0000 7f89eca9b640 1 -- 172.21.15.38:0/2501065028 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f89e80ca6f0 msgr2=0x7f89e80cf6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:11.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.437+0000 7f89eca9b640 1 --2- 172.21.15.38:0/2501065028 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f89e80ca6f0 0x7f89e80cf6e0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f89d8009880 tx=0x7f89d802c650 comp rx=0 tx=0).stop 2024-09-21T03:33:11.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.437+0000 7f89eca9b640 1 -- 172.21.15.38:0/2501065028 shutdown_connections 2024-09-21T03:33:11.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.437+0000 7f89eca9b640 1 --2- 172.21.15.38:0/2501065028 >> v2:172.21.15.195:3300/0 conn(0x7f89e806e500 0x7f89e80cc4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:11.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.437+0000 7f89eca9b640 1 --2- 172.21.15.38:0/2501065028 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f89e80ca6f0 0x7f89e80cf6e0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:11.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.437+0000 7f89eca9b640 1 -- 172.21.15.38:0/2501065028 >> 172.21.15.38:0/2501065028 conn(0x7f89e80b1b90 msgr2=0x7f89e80b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:11.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.438+0000 7f5819feb640 1 -- 172.21.15.38:0/2617562833 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58141621d0 con 0x7f58140cf840 2024-09-21T03:33:11.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.438+0000 7fee22d69640 1 Processor -- start 2024-09-21T03:33:11.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.438+0000 7fee22d69640 1 -- start start 2024-09-21T03:33:11.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.442+0000 7fee22d69640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fee04003a50 0x7fee1c1f9180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:11.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.442+0000 7fee22d69640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fee1c0b0760 0x7fee1c0c5da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:11.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.443+0000 7f58117fa640 1 -- 172.21.15.38:0/2617562833 <== mon.2 v2:172.21.15.195:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f5808010cf0 con 0x7f58140cf840 2024-09-21T03:33:11.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.444+0000 7fee22d69640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fee1c0c62e0 0x7fee1c14c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:11.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.444+0000 7fee22d69640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fee1c0cc2e0 con 0x7fee1c0b0760 2024-09-21T03:33:11.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.444+0000 7fee22d69640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fee1c0cc160 con 0x7fee1c0c62e0 2024-09-21T03:33:11.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.444+0000 7fee22d69640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fee1c0cc460 con 0x7fee04003a50 2024-09-21T03:33:11.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.440+0000 7f89eca9b640 1 -- 172.21.15.38:0/2501065028 shutdown_connections 2024-09-21T03:33:11.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.440+0000 7f89eca9b640 1 -- 172.21.15.38:0/2501065028 wait complete. 2024-09-21T03:33:11.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.443+0000 7f89eca9b640 1 Processor -- start 2024-09-21T03:33:11.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.443+0000 7f89eca9b640 1 -- start start 2024-09-21T03:33:11.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.445+0000 7f89eca9b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f89e80ca6f0 0x7f89e80a0550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:11.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.445+0000 7f89eca9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f89e80cc0b0 0x7f89e80a0a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:11.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.445+0000 7f89eca9b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f89e809b3f0 0x7f89e809b800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:11.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.445+0000 7f89eca9b640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f89e8071000 con 0x7f89e80cc0b0 2024-09-21T03:33:11.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.445+0000 7f89e659c640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f89e80ca6f0 0x7f89e80a0550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:11.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.445+0000 7f89e659c640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f89e80ca6f0 0x7f89e80a0550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.38:39478/0 (socket says 172.21.15.38:39478) 2024-09-21T03:33:11.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.445+0000 7f89e659c640 1 -- 172.21.15.38:0/1672971093 learned_addr learned my addr 172.21.15.38:0/1672971093 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:33:11.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.445+0000 7f89eca9b640 1 -- 172.21.15.38:0/1672971093 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f89e8070e80 con 0x7f89e809b3f0 2024-09-21T03:33:11.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.445+0000 7f89eca9b640 1 -- 172.21.15.38:0/1672971093 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f89e8071180 con 0x7f89e80ca6f0 2024-09-21T03:33:11.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.446+0000 7fee21306640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fee1c0c62e0 0x7fee1c14c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:11.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.446+0000 7fee21306640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fee1c0c62e0 0x7fee1c14c5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.38:60350/0 (socket says 172.21.15.38:60350) 2024-09-21T03:33:11.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.446+0000 7fee21306640 1 -- 172.21.15.38:0/3749813913 learned_addr learned my addr 172.21.15.38:0/3749813913 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:33:11.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.443+0000 7fee20b05640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fee04003a50 0x7fee1c1f9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:11.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.446+0000 7f89e659c640 1 -- 172.21.15.38:0/1672971093 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f89e809b3f0 msgr2=0x7f89e809b800 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T03:33:11.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.446+0000 7f89e659c640 1 --2- 172.21.15.38:0/1672971093 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f89e809b3f0 0x7f89e809b800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:11.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.446+0000 7f89e659c640 1 -- 172.21.15.38:0/1672971093 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f89e80cc0b0 msgr2=0x7f89e80a0a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:11.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.446+0000 7f89e659c640 1 --2- 172.21.15.38:0/1672971093 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f89e80cc0b0 0x7f89e80a0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:11.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.446+0000 7f89e659c640 1 -- 172.21.15.38:0/1672971093 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89e80855f0 con 0x7f89e80ca6f0 2024-09-21T03:33:11.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.446+0000 7f89e659c640 1 --2- 172.21.15.38:0/1672971093 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f89e80ca6f0 0x7f89e80a0550 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f89d8009970 tx=0x7f89d8033b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:11.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.447+0000 7fee1bfff640 1 --2- 172.21.15.38:0/3749813913 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fee1c0b0760 0x7fee1c0c5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:11.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.447+0000 7f89d6ffd640 1 -- 172.21.15.38:0/1672971093 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89d8040020 con 0x7f89e80ca6f0 2024-09-21T03:33:11.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.447+0000 7fee21306640 1 -- 172.21.15.38:0/3749813913 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fee04003a50 msgr2=0x7fee1c1f9180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:11.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.447+0000 7fee21306640 1 --2- 172.21.15.38:0/3749813913 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fee04003a50 0x7fee1c1f9180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:11.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.447+0000 7fee21306640 1 -- 172.21.15.38:0/3749813913 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fee1c0b0760 msgr2=0x7fee1c0c5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:11.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.447+0000 7fee21306640 1 --2- 172.21.15.38:0/3749813913 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fee1c0b0760 0x7fee1c0c5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:11.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.447+0000 7fee21306640 1 -- 172.21.15.38:0/3749813913 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee1c0c66f0 con 0x7fee1c0c62e0 2024-09-21T03:33:11.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.448+0000 7fee21306640 1 --2- 172.21.15.38:0/3749813913 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fee1c0c62e0 0x7fee1c14c5a0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fee1c072d20 tx=0x7fee1400ed90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:11.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.448+0000 7fee197fa640 1 -- 172.21.15.38:0/3749813913 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee1400cc20 con 0x7fee1c0c62e0 2024-09-21T03:33:11.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.448+0000 7fee197fa640 1 -- 172.21.15.38:0/3749813913 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fee14018070 con 0x7fee1c0c62e0 2024-09-21T03:33:11.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.448+0000 7fee197fa640 1 -- 172.21.15.38:0/3749813913 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee140116c0 con 0x7fee1c0c62e0 2024-09-21T03:33:11.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.447+0000 7f89eca9b640 1 -- 172.21.15.38:0/1672971093 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89e81686f0 con 0x7f89e80ca6f0 2024-09-21T03:33:11.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.449+0000 7fee22d69640 1 -- 172.21.15.38:0/3749813913 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee1c1f96c0 con 0x7fee1c0c62e0 2024-09-21T03:33:11.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.449+0000 7fee22d69640 1 -- 172.21.15.38:0/3749813913 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee1c09ed90 con 0x7fee1c0c62e0 2024-09-21T03:33:11.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.449+0000 7fee1bfff640 1 --2- 172.21.15.38:0/3749813913 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fee1c0b0760 0x7fee1c0c5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:33:11.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.447+0000 7f89eca9b640 1 -- 172.21.15.38:0/1672971093 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f89e8211220 con 0x7f89e80ca6f0 2024-09-21T03:33:11.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.449+0000 7f89d6ffd640 1 -- 172.21.15.38:0/1672971093 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f89d8003af0 con 0x7f89e80ca6f0 2024-09-21T03:33:11.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.450+0000 7f89d6ffd640 1 -- 172.21.15.38:0/1672971093 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89d803c430 con 0x7f89e80ca6f0 2024-09-21T03:33:11.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.450+0000 7f89d6ffd640 1 -- 172.21.15.38:0/1672971093 <== mon.2 v2:172.21.15.195:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f89d8043020 con 0x7f89e80ca6f0 2024-09-21T03:33:11.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.450+0000 7f89d6ffd640 1 -- 172.21.15.38:0/1672971093 <== mon.2 v2:172.21.15.195:3301/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f89d8045660 con 0x7f89e80ca6f0 2024-09-21T03:33:11.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.450+0000 7fd7d6ef5640 1 -- 172.21.15.38:0/3359392858 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd7c8009ba0 msgr2=0x7fd7c80df180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:11.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.450+0000 7fd7d6ef5640 1 --2- 172.21.15.38:0/3359392858 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd7c8009ba0 0x7fd7c80df180 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc009fd0 tx=0x7fd7cc02b830 comp rx=0 tx=0).stop 2024-09-21T03:33:11.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.450+0000 7fd7d6ef5640 1 -- 172.21.15.38:0/3359392858 shutdown_connections 2024-09-21T03:33:11.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.450+0000 7fd7d6ef5640 1 --2- 172.21.15.38:0/3359392858 >> v2:172.21.15.38:3300/0 conn(0x7fd7c8005c60 0x7fd7c8036430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:11.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.450+0000 7fd7d6ef5640 1 --2- 172.21.15.38:0/3359392858 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd7c8009ba0 0x7fd7c80df180 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:11.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.450+0000 7fd7d6ef5640 1 -- 172.21.15.38:0/3359392858 >> 172.21.15.38:0/3359392858 conn(0x7fd7c8051420 msgr2=0x7fd7c8051860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:11.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.450+0000 7fd7d6ef5640 1 -- 172.21.15.38:0/3359392858 shutdown_connections 2024-09-21T03:33:11.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.450+0000 7fee02ffd640 1 -- 172.21.15.38:0/3749813913 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fedf4005290 con 0x7fee1c0c62e0 2024-09-21T03:33:11.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.451+0000 7f89eca9b640 1 -- 172.21.15.38:0/1672971093 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89cc005290 con 0x7f89e80ca6f0 2024-09-21T03:33:11.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.452+0000 7fee197fa640 1 -- 172.21.15.38:0/3749813913 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fee14011bf0 con 0x7fee1c0c62e0 2024-09-21T03:33:11.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.452+0000 7fee197fa640 1 -- 172.21.15.38:0/3749813913 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fee1401ce00 con 0x7fee1c0c62e0 2024-09-21T03:33:11.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.446+0000 7f2b2e853640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2b28074590 0x7f2b28205b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:11.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.446+0000 7f2b2e052640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2b2806ddf0 0x7f2b281588a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:11.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.446+0000 7f2b2d851640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2b280711b0 0x7f2b281f8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:11.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.446+0000 7f2b2d851640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2b280711b0 0x7f2b281f8ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41478/0 (socket says 172.21.15.38:41478) 2024-09-21T03:33:11.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.446+0000 7f2b2d851640 1 -- 172.21.15.38:0/908039980 learned_addr learned my addr 172.21.15.38:0/908039980 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:33:11.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.447+0000 7f2b2d851640 1 -- 172.21.15.38:0/908039980 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2b28074590 msgr2=0x7f2b28205b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:11.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.447+0000 7f2b2d851640 1 --2- 172.21.15.38:0/908039980 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2b28074590 0x7f2b28205b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:11.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.447+0000 7f2b2d851640 1 -- 172.21.15.38:0/908039980 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2b2806ddf0 msgr2=0x7f2b281588a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:11.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.447+0000 7f2b2d851640 1 --2- 172.21.15.38:0/908039980 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2b2806ddf0 0x7f2b281588a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:11.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.447+0000 7f2b2d851640 1 -- 172.21.15.38:0/908039980 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b280aeb80 con 0x7f2b280711b0 2024-09-21T03:33:11.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.453+0000 7f2b2d851640 1 --2- 172.21.15.38:0/908039980 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2b280711b0 0x7f2b281f8ed0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f2b1c00ea20 tx=0x7f2b1c00eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:11.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.451+0000 7fd7d6ef5640 1 -- 172.21.15.38:0/3359392858 wait complete. 2024-09-21T03:33:11.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.456+0000 7fee197fa640 1 -- 172.21.15.38:0/3749813913 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fee14021020 con 0x7fee1c0c62e0 2024-09-21T03:33:11.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.454+0000 7f2b26ffd640 1 -- 172.21.15.38:0/908039980 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b1c00cc30 con 0x7f2b280711b0 2024-09-21T03:33:11.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.454+0000 7f2b26ffd640 1 -- 172.21.15.38:0/908039980 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2b1c010430 con 0x7f2b280711b0 2024-09-21T03:33:11.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.454+0000 7f2b26ffd640 1 -- 172.21.15.38:0/908039980 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b1c010860 con 0x7f2b280711b0 2024-09-21T03:33:11.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.455+0000 7f2b302b6640 1 -- 172.21.15.38:0/908039980 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b28157ee0 con 0x7f2b280711b0 2024-09-21T03:33:11.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.455+0000 7f2b302b6640 1 -- 172.21.15.38:0/908039980 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b28157970 con 0x7f2b280711b0 2024-09-21T03:33:11.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.457+0000 7fd7d6ef5640 1 Processor -- start 2024-09-21T03:33:11.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.459+0000 7f7282863640 1 -- 172.21.15.38:0/1843641197 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f727c15ac20 msgr2=0x7f727c070070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:11.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.459+0000 7f2b26ffd640 1 -- 172.21.15.38:0/908039980 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f2b1c01f020 con 0x7f2b280711b0 2024-09-21T03:33:11.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.459+0000 7f2b26ffd640 1 -- 172.21.15.38:0/908039980 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f2b1c003960 con 0x7f2b280711b0 2024-09-21T03:33:11.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.460+0000 7f2b24ff9640 1 -- 172.21.15.38:0/908039980 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b280ae990 con 0x7f2b280711b0 2024-09-21T03:33:11.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.460+0000 7f89d6ffd640 1 -- 172.21.15.38:0/1672971093 <== mon.2 v2:172.21.15.195:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f89d804a030 con 0x7f89e80ca6f0 2024-09-21T03:33:11.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.460+0000 7fd7d6ef5640 1 -- start start 2024-09-21T03:33:11.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.462+0000 7fd7d6ef5640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd7c8005c60 0x7fd7c80d4df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:11.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.462+0000 7fd7d6ef5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7c8036970 0x7fd7c8021890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:11.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.462+0000 7fd7d6ef5640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd7c8047da0 0x7fd7c801f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:11.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.462+0000 7fd7d6ef5640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fd7c800bdb0 con 0x7fd7c8036970 2024-09-21T03:33:11.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.462+0000 7fd7d6ef5640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd7c800bc30 con 0x7fd7c8047da0 2024-09-21T03:33:11.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.462+0000 7fd7d6ef5640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fd7c801ccc0 con 0x7fd7c8005c60 2024-09-21T03:33:11.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.463+0000 7fd7d5ef3640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd7c8005c60 0x7fd7c80d4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:11.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.463+0000 7fd7d66f4640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd7c8047da0 0x7fd7c801f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:11.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.463+0000 7fd7d66f4640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd7c8047da0 0x7fd7c801f420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.38:60384/0 (socket says 172.21.15.38:60384) 2024-09-21T03:33:11.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.463+0000 7fd7d66f4640 1 -- 172.21.15.38:0/724932207 learned_addr learned my addr 172.21.15.38:0/724932207 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:33:11.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.464+0000 7fd7d5ef3640 1 -- 172.21.15.38:0/724932207 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd7c8047da0 msgr2=0x7fd7c801f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:11.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.464+0000 7fd7d5ef3640 1 --2- 172.21.15.38:0/724932207 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd7c8047da0 0x7fd7c801f420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:11.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.464+0000 7fd7d5ef3640 1 -- 172.21.15.38:0/724932207 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7c8036970 msgr2=0x7fd7c8021890 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T03:33:11.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.464+0000 7fd7d5ef3640 1 --2- 172.21.15.38:0/724932207 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7c8036970 0x7fd7c8021890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:11.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.464+0000 7fd7d5ef3640 1 -- 172.21.15.38:0/724932207 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7c8021dd0 con 0x7fd7c8005c60 2024-09-21T03:33:11.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.464+0000 7f2b26ffd640 1 -- 172.21.15.38:0/908039980 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f2b1c026070 con 0x7f2b280711b0 2024-09-21T03:33:11.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.464+0000 7fd7d5ef3640 1 --2- 172.21.15.38:0/724932207 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd7c8005c60 0x7fd7c80d4df0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc033000 tx=0x7fd7cc004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:11.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.464+0000 7fd7c67fc640 1 -- 172.21.15.38:0/724932207 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7cc005220 con 0x7fd7c8005c60 2024-09-21T03:33:11.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.464+0000 7fd7d6ef5640 1 -- 172.21.15.38:0/724932207 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7c80d4030 con 0x7fd7c8005c60 2024-09-21T03:33:11.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.464+0000 7fd7d6ef5640 1 -- 172.21.15.38:0/724932207 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7c80d45f0 con 0x7fd7c8005c60 2024-09-21T03:33:11.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.466+0000 7fd7c67fc640 1 -- 172.21.15.38:0/724932207 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd7cc0053c0 con 0x7fd7c8005c60 2024-09-21T03:33:11.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.466+0000 7fd7c67fc640 1 -- 172.21.15.38:0/724932207 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7cc003800 con 0x7fd7c8005c60 2024-09-21T03:33:11.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.459+0000 7f7282863640 1 --2- 172.21.15.38:0/1843641197 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f727c15ac20 0x7f727c070070 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f726c009830 tx=0x7f726c02a510 comp rx=0 tx=0).stop 2024-09-21T03:33:11.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.468+0000 7fd7c67fc640 1 -- 172.21.15.38:0/724932207 <== mon.2 v2:172.21.15.195:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fd7cc0047a0 con 0x7fd7c8005c60 2024-09-21T03:33:11.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.468+0000 7fd7c67fc640 1 -- 172.21.15.38:0/724932207 <== mon.2 v2:172.21.15.195:3301/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fd7cc044780 con 0x7fd7c8005c60 2024-09-21T03:33:11.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.468+0000 7fd7d6ef5640 1 -- 172.21.15.38:0/724932207 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7b8005290 con 0x7fd7c8005c60 2024-09-21T03:33:11.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.474+0000 7f7282863640 1 -- 172.21.15.38:0/1843641197 shutdown_connections 2024-09-21T03:33:11.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.474+0000 7f7282863640 1 --2- 172.21.15.38:0/1843641197 >> v2:172.21.15.195:3300/0 conn(0x7f727c072d90 0x7f727c0731a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:11.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.474+0000 7f7282863640 1 --2- 172.21.15.38:0/1843641197 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f727c15ac20 0x7f727c070070 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:11.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.474+0000 7f7282863640 1 -- 172.21.15.38:0/1843641197 >> 172.21.15.38:0/1843641197 conn(0x7f727c0b1b90 msgr2=0x7f727c0b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:11.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.478+0000 7fd7c67fc640 1 -- 172.21.15.38:0/724932207 <== mon.2 v2:172.21.15.195:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fd7cc040070 con 0x7fd7c8005c60 2024-09-21T03:33:11.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.483+0000 7f7282863640 1 -- 172.21.15.38:0/1843641197 shutdown_connections 2024-09-21T03:33:11.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.486+0000 7f4ad10d5640 1 -- 172.21.15.38:0/3724294675 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command([{prefix=config set, name=mds_verify_scatter}] v 0) -- 0x7f4aac005850 con 0x7f4acc073070 2024-09-21T03:33:11.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.487+0000 7f7282863640 1 -- 172.21.15.38:0/1843641197 wait complete. 2024-09-21T03:33:11.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.490+0000 7f7282863640 1 Processor -- start 2024-09-21T03:33:11.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.493+0000 7f4ab37fe640 1 -- 172.21.15.38:0/3724294675 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_verify_scatter}]=0 v12)=0 v12) ==== 119+0+0 (secure 0 0 0) 0x7f4acc16f560 con 0x7f4acc073070 2024-09-21T03:33:11.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.494+0000 7f7282863640 1 -- start start 2024-09-21T03:33:11.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.495+0000 7f4ad10d5640 1 -- 172.21.15.38:0/3724294675 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4acc073070 msgr2=0x7f4acc167000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:11.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.495+0000 7f4ad10d5640 1 --2- 172.21.15.38:0/3724294675 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4acc073070 0x7f4acc167000 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f4ac400cb40 tx=0x7f4ac400a060 comp rx=0 tx=0).stop 2024-09-21T03:33:11.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.495+0000 7f4ad10d5640 1 -- 172.21.15.38:0/3724294675 shutdown_connections 2024-09-21T03:33:11.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.495+0000 7f4ad10d5640 1 --2- 172.21.15.38:0/3724294675 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4acc16f150 0x7f4acc212fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:11.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.495+0000 7f4ad10d5640 1 --2- 172.21.15.38:0/3724294675 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4acc08b9e0 0x7f4acc212a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:11.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.495+0000 7f4ad10d5640 1 --2- 172.21.15.38:0/3724294675 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4acc073070 0x7f4acc167000 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:11.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.495+0000 7f4ad10d5640 1 -- 172.21.15.38:0/3724294675 >> 172.21.15.38:0/3724294675 conn(0x7f4acc06cde0 msgr2=0x7f4acc16dc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:11.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.495+0000 7f4ad10d5640 1 -- 172.21.15.38:0/3724294675 shutdown_connections 2024-09-21T03:33:11.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.496+0000 7f4ad10d5640 1 -- 172.21.15.38:0/3724294675 wait complete. 2024-09-21T03:33:11.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.497+0000 7f7282863640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f727c06ddf0 0x7f727c0af030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:11.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.497+0000 7f7282863640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f727c072d90 0x7f727c1529c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:11.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.497+0000 7f7282863640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f727c15ac20 0x7f727c0c4440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:11.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.497+0000 7f727b7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f727c072d90 0x7f727c1529c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:11.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.497+0000 7f727b7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f727c072d90 0x7f727c1529c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41492/0 (socket says 172.21.15.38:41492) 2024-09-21T03:33:11.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.506+0000 7f727bfff640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f727c06ddf0 0x7f727c0af030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:11.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.507+0000 7f7282863640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f727c071090 con 0x7f727c072d90 2024-09-21T03:33:11.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.507+0000 7f7282863640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f727c0707d0 con 0x7f727c06ddf0 2024-09-21T03:33:11.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.507+0000 7f7282863640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f727c0736e0 con 0x7f727c15ac20 2024-09-21T03:33:11.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.508+0000 7f727b7fe640 1 -- 172.21.15.38:0/199173135 learned_addr learned my addr 172.21.15.38:0/199173135 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:33:11.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.508+0000 7f727b7fe640 1 -- 172.21.15.38:0/199173135 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f727c15ac20 msgr2=0x7f727c0c4440 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T03:33:11.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.508+0000 7f727b7fe640 1 --2- 172.21.15.38:0/199173135 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f727c15ac20 0x7f727c0c4440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:11.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.508+0000 7f727b7fe640 1 -- 172.21.15.38:0/199173135 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f727c06ddf0 msgr2=0x7f727c0af030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:11.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.508+0000 7f727b7fe640 1 --2- 172.21.15.38:0/199173135 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f727c06ddf0 0x7f727c0af030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:11.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.508+0000 7f727b7fe640 1 -- 172.21.15.38:0/199173135 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f727c152f00 con 0x7f727c072d90 2024-09-21T03:33:11.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.509+0000 7f727b7fe640 1 --2- 172.21.15.38:0/199173135 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f727c072d90 0x7f727c1529c0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f7270001eb0 tx=0x7f727000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:11.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.509+0000 7f7278ff9640 1 -- 172.21.15.38:0/199173135 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7270011790 con 0x7f727c072d90 2024-09-21T03:33:11.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.509+0000 7f7278ff9640 1 -- 172.21.15.38:0/199173135 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7270017070 con 0x7f727c072d90 2024-09-21T03:33:11.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.511+0000 7f7278ff9640 1 -- 172.21.15.38:0/199173135 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7270011ca0 con 0x7f727c072d90 2024-09-21T03:33:11.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.516+0000 7f7282863640 1 -- 172.21.15.38:0/199173135 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f727c1f6220 con 0x7f727c072d90 2024-09-21T03:33:11.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.516+0000 7f7282863640 1 -- 172.21.15.38:0/199173135 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f727c1f67e0 con 0x7f727c072d90 2024-09-21T03:33:11.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.519+0000 7f030759e640 1 -- 172.21.15.38:0/2998531551 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command([{prefix=config set, name=mds_op_complaint_time}] v 0) -- 0x7f02ec005850 con 0x7f030806e530 2024-09-21T03:33:11.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.518+0000 7f7278ff9640 1 -- 172.21.15.38:0/199173135 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f727001c460 con 0x7f727c072d90 2024-09-21T03:33:11.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.518+0000 7f7278ff9640 1 -- 172.21.15.38:0/199173135 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f72700127d0 con 0x7f727c072d90 2024-09-21T03:33:11.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.525+0000 7f7282863640 1 -- 172.21.15.38:0/199173135 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f727c074450 con 0x7f727c072d90 2024-09-21T03:33:11.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.531+0000 7f02f6ffd640 1 -- 172.21.15.38:0/2998531551 <== mon.1 v2:172.21.15.195:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_op_complaint_time}]=0 v13)=0 v13) ==== 121+0+0 (secure 0 0 0) 0x7f02f80163f0 con 0x7f030806e530 2024-09-21T03:33:11.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.531+0000 7f7278ff9640 1 -- 172.21.15.38:0/199173135 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f7270012a20 con 0x7f727c072d90 2024-09-21T03:33:11.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.536+0000 7f030759e640 1 -- 172.21.15.38:0/2998531551 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f030806e530 msgr2=0x7f03082133a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:11.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.536+0000 7f030759e640 1 --2- 172.21.15.38:0/2998531551 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f030806e530 0x7f03082133a0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f02f800e840 tx=0x7f02f800ed00 comp rx=0 tx=0).stop 2024-09-21T03:33:11.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.536+0000 7f030759e640 1 -- 172.21.15.38:0/2998531551 shutdown_connections 2024-09-21T03:33:11.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.536+0000 7f030759e640 1 --2- 172.21.15.38:0/2998531551 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f03080b4fc0 0x7f030820cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:11.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.536+0000 7f030759e640 1 --2- 172.21.15.38:0/2998531551 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f030808bb20 0x7f03080cc760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:11.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.536+0000 7f030759e640 1 --2- 172.21.15.38:0/2998531551 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f030806e530 0x7f03082133a0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:11.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.536+0000 7f030759e640 1 -- 172.21.15.38:0/2998531551 >> 172.21.15.38:0/2998531551 conn(0x7f03080cfec0 msgr2=0x7f03080b3a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:11.541 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.540+0000 7f030759e640 1 -- 172.21.15.38:0/2998531551 shutdown_connections 2024-09-21T03:33:11.541 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.540+0000 7f030759e640 1 -- 172.21.15.38:0/2998531551 wait complete. 2024-09-21T03:33:11.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.588+0000 7f780e7fc640 1 -- 172.21.15.38:0/1825845608 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command([{prefix=config set, name=rados_mon_op_timeout}] v 0) -- 0x7f7808005850 con 0x7f782806edf0 2024-09-21T03:33:11.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.592+0000 7f7824ff9640 1 -- 172.21.15.38:0/1825845608 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_mon_op_timeout}]=0 v14)=0 v14) ==== 120+0+0 (secure 0 0 0) 0x7f7818031970 con 0x7f782806edf0 2024-09-21T03:33:11.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.604+0000 7fd9faffd640 1 -- 172.21.15.38:0/3337678306 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_pool_no_app}] v 0) -- 0x7fd9f4005580 con 0x7fda1406ddf0 2024-09-21T03:33:11.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.605+0000 7f7c399cc640 1 -- 172.21.15.38:0/660629611 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) -- 0x7f7c14005580 con 0x7f7c3406edf0 2024-09-21T03:33:11.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.605+0000 7f780e7fc640 1 -- 172.21.15.38:0/1825845608 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f782806edf0 msgr2=0x7f78281fa6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:11.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.605+0000 7f780e7fc640 1 --2- 172.21.15.38:0/1825845608 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f782806edf0 0x7f78281fa6f0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f7818009950 tx=0x7f78180038b0 comp rx=0 tx=0).stop 2024-09-21T03:33:11.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.608+0000 7f780e7fc640 1 -- 172.21.15.38:0/1825845608 shutdown_connections 2024-09-21T03:33:11.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.608+0000 7f780e7fc640 1 --2- 172.21.15.38:0/1825845608 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f78281648c0 0x7f78280cf790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:11.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.608+0000 7f780e7fc640 1 --2- 172.21.15.38:0/1825845608 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f78281644b0 0x7f78280cf250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:11.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.608+0000 7f780e7fc640 1 --2- 172.21.15.38:0/1825845608 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f782806edf0 0x7f78281fa6f0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:11.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.608+0000 7f780e7fc640 1 -- 172.21.15.38:0/1825845608 >> 172.21.15.38:0/1825845608 conn(0x7f782806cde0 msgr2=0x7f78280aff90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:11.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.608+0000 7f780e7fc640 1 -- 172.21.15.38:0/1825845608 shutdown_connections 2024-09-21T03:33:11.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.608+0000 7f780e7fc640 1 -- 172.21.15.38:0/1825845608 wait complete. 2024-09-21T03:33:11.612 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_verify_scatter', 'True'], check_status=True, hostname='smithi038') 2024-09-21T03:33:11.612 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi038') 2024-09-21T03:33:11.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.611+0000 7fda117fa640 1 -- 172.21.15.38:0/3337678306 <== mon.1 v2:172.21.15.195:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_pool_no_app}]=0 v15)=0 v15) ==== 125+0+0 (secure 0 0 0) 0x7fda0c004040 con 0x7fda1406ddf0 2024-09-21T03:33:11.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.614+0000 7fda1abda640 1 -- 172.21.15.38:0/3337678306 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fda1406ddf0 msgr2=0x7fda140c2b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:11.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.614+0000 7fda1abda640 1 --2- 172.21.15.38:0/3337678306 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fda1406ddf0 0x7fda140c2b40 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fda140b2840 tx=0x7fda0c02ef60 comp rx=0 tx=0).stop 2024-09-21T03:33:11.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.616+0000 7fda1abda640 1 -- 172.21.15.38:0/3337678306 shutdown_connections 2024-09-21T03:33:11.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.616+0000 7fda1abda640 1 --2- 172.21.15.38:0/3337678306 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fda140cc0b0 0x7fda140b4cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:11.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.616+0000 7fda1abda640 1 --2- 172.21.15.38:0/3337678306 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fda140ca6f0 0x7fda140b4790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:11.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.616+0000 7fda1abda640 1 --2- 172.21.15.38:0/3337678306 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fda1406ddf0 0x7fda140c2b40 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:11.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.616+0000 7fda1abda640 1 -- 172.21.15.38:0/3337678306 >> 172.21.15.38:0/3337678306 conn(0x7fda140b1b90 msgr2=0x7fda140cdc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:11.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.617+0000 7fda1abda640 1 -- 172.21.15.38:0/3337678306 shutdown_connections 2024-09-21T03:33:11.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.617+0000 7fda1abda640 1 -- 172.21.15.38:0/3337678306 wait complete. 2024-09-21T03:33:11.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.618+0000 7f7c1bfff640 1 -- 172.21.15.38:0/660629611 <== mon.1 v2:172.21.15.195:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v16)=0 v16) ==== 121+0+0 (secure 0 0 0) 0x7f7c280106e0 con 0x7f7c3406edf0 2024-09-21T03:33:11.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.619+0000 7fa7da00a640 1 -- 172.21.15.38:0/1803892417 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command([{prefix=config set, name=mon_op_complaint_time}] v 0) -- 0x7fa7b4005580 con 0x7fa7d40ad850 2024-09-21T03:33:11.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.628+0000 7fee02ffd640 1 -- 172.21.15.38:0/3749813913 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command([{prefix=config set, name=client_mount_timeout}] v 0) -- 0x7fedf4005850 con 0x7fee1c0c62e0 2024-09-21T03:33:11.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.629+0000 7f7c19ffb640 1 -- 172.21.15.38:0/660629611 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7c3406edf0 msgr2=0x7f7c34159520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:11.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.629+0000 7f7c19ffb640 1 --2- 172.21.15.38:0/660629611 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7c3406edf0 0x7f7c34159520 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f7c2800d720 tx=0x7f7c2800dbe0 comp rx=0 tx=0).stop 2024-09-21T03:33:11.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.629+0000 7f7c19ffb640 1 -- 172.21.15.38:0/660629611 shutdown_connections 2024-09-21T03:33:11.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.629+0000 7f7c19ffb640 1 --2- 172.21.15.38:0/660629611 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7c34206540 0x7f7c34205930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:11.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.629+0000 7f7c19ffb640 1 --2- 172.21.15.38:0/660629611 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7c34074110 0x7f7c34160710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:11.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.629+0000 7f7c19ffb640 1 --2- 172.21.15.38:0/660629611 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7c3406edf0 0x7f7c34159520 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:11.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.629+0000 7f7c19ffb640 1 -- 172.21.15.38:0/660629611 >> 172.21.15.38:0/660629611 conn(0x7f7c3406cde0 msgr2=0x7f7c340aeb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:11.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.630+0000 7f2b24ff9640 1 -- 172.21.15.38:0/908039980 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command([{prefix=config set, name=debug_ms}] v 0) -- 0x7f2b2806e200 con 0x7f2b280711b0 2024-09-21T03:33:11.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.630+0000 7f7c19ffb640 1 -- 172.21.15.38:0/660629611 shutdown_connections 2024-09-21T03:33:11.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.630+0000 7f7c19ffb640 1 -- 172.21.15.38:0/660629611 wait complete. 2024-09-21T03:33:11.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.633+0000 7fa7bffff640 1 -- 172.21.15.38:0/1803892417 <== mon.1 v2:172.21.15.195:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_op_complaint_time}]=0 v17)=0 v17) ==== 121+0+0 (secure 0 0 0) 0x7fa7d40cfa00 con 0x7fa7d40ad850 2024-09-21T03:33:11.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.640+0000 7fee197fa640 1 -- 172.21.15.38:0/3749813913 <== mon.1 v2:172.21.15.195:3300/0 7 ==== mon_command_ack([{prefix=config set, name=client_mount_timeout}]=0 v18)=0 v18) ==== 119+0+0 (secure 0 0 0) 0x7fee1c0c66f0 con 0x7fee1c0c62e0 2024-09-21T03:33:11.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.642+0000 7fee22d69640 1 -- 172.21.15.38:0/3749813913 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fee1c0c62e0 msgr2=0x7fee1c14c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:11.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.642+0000 7fee22d69640 1 --2- 172.21.15.38:0/3749813913 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fee1c0c62e0 0x7fee1c14c5a0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fee1c072d20 tx=0x7fee1400ed90 comp rx=0 tx=0).stop 2024-09-21T03:33:11.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.642+0000 7fa7da00a640 1 -- 172.21.15.38:0/1803892417 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa7d40ad850 msgr2=0x7fa7d4152fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:11.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.642+0000 7fa7da00a640 1 --2- 172.21.15.38:0/1803892417 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa7d40ad850 0x7fa7d4152fd0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fa7c000c7d0 tx=0x7fa7c000cc90 comp rx=0 tx=0).stop 2024-09-21T03:33:11.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.642+0000 7fa7da00a640 1 -- 172.21.15.38:0/1803892417 shutdown_connections 2024-09-21T03:33:11.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.642+0000 7fa7da00a640 1 --2- 172.21.15.38:0/1803892417 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa7d415ed20 0x7fa7d41625d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:11.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.642+0000 7fa7da00a640 1 --2- 172.21.15.38:0/1803892417 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa7d40cf0b0 0x7fa7d4162090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:11.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.642+0000 7fa7da00a640 1 --2- 172.21.15.38:0/1803892417 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa7d40ad850 0x7fa7d4152fd0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:11.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.642+0000 7fee22d69640 1 -- 172.21.15.38:0/3749813913 shutdown_connections 2024-09-21T03:33:11.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.642+0000 7fee22d69640 1 --2- 172.21.15.38:0/3749813913 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fee1c0c62e0 0x7fee1c14c5a0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:11.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.642+0000 7fee22d69640 1 --2- 172.21.15.38:0/3749813913 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fee1c0b0760 0x7fee1c0c5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:11.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.642+0000 7fee22d69640 1 --2- 172.21.15.38:0/3749813913 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fee04003a50 0x7fee1c1f9180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:11.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.642+0000 7fee22d69640 1 -- 172.21.15.38:0/3749813913 >> 172.21.15.38:0/3749813913 conn(0x7fee1c0b1b90 msgr2=0x7fee1c070fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:11.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.642+0000 7fa7da00a640 1 -- 172.21.15.38:0/1803892417 >> 172.21.15.38:0/1803892417 conn(0x7fa7d40ccb40 msgr2=0x7fa7d40ccf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:11.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.642+0000 7fee22d69640 1 -- 172.21.15.38:0/3749813913 shutdown_connections 2024-09-21T03:33:11.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.643+0000 7fee22d69640 1 -- 172.21.15.38:0/3749813913 wait complete. 2024-09-21T03:33:11.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.643+0000 7fa7da00a640 1 -- 172.21.15.38:0/1803892417 shutdown_connections 2024-09-21T03:33:11.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.643+0000 7fa7da00a640 1 -- 172.21.15.38:0/1803892417 wait complete. 2024-09-21T03:33:11.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.646+0000 7f2b26ffd640 1 -- 172.21.15.38:0/908039980 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_ms}]=0 v19)=0 v19) ==== 106+0+0 (secure 0 0 0) 0x7f2b1c003bb0 con 0x7f2b280711b0 2024-09-21T03:33:11.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.648+0000 7f2b24ff9640 1 -- 172.21.15.38:0/908039980 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2b280711b0 msgr2=0x7f2b281f8ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:11.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.648+0000 7f2b24ff9640 1 --2- 172.21.15.38:0/908039980 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2b280711b0 0x7f2b281f8ed0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f2b1c00ea20 tx=0x7f2b1c00eee0 comp rx=0 tx=0).stop 2024-09-21T03:33:11.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.651+0000 7f2b24ff9640 1 -- 172.21.15.38:0/908039980 shutdown_connections 2024-09-21T03:33:11.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.651+0000 7f2b24ff9640 1 --2- 172.21.15.38:0/908039980 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2b28074590 0x7f2b28205b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:11.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.651+0000 7f2b24ff9640 1 --2- 172.21.15.38:0/908039980 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2b280711b0 0x7f2b281f8ed0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:11.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.651+0000 7f2b24ff9640 1 --2- 172.21.15.38:0/908039980 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2b2806ddf0 0x7f2b281588a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:11.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.651+0000 7f2b24ff9640 1 -- 172.21.15.38:0/908039980 >> 172.21.15.38:0/908039980 conn(0x7f2b280735d0 msgr2=0x7f2b280b1bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:11.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.651+0000 7f2b24ff9640 1 -- 172.21.15.38:0/908039980 shutdown_connections 2024-09-21T03:33:11.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.651+0000 7f2b24ff9640 1 -- 172.21.15.38:0/908039980 wait complete. 2024-09-21T03:33:11.666 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi038') 2024-09-21T03:33:11.666 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi038') 2024-09-21T03:33:11.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.677+0000 7fd7d6ef5640 1 -- 172.21.15.38:0/724932207 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) -- 0x7fd7b8005580 con 0x7fd7c8005c60 2024-09-21T03:33:11.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.678+0000 7f5819feb640 1 -- 172.21.15.38:0/2617562833 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_command([{prefix=config set, name=rados_osd_op_timeout}] v 0) -- 0x7f58140cf430 con 0x7f58140cf840 2024-09-21T03:33:11.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.680+0000 7fd7c67fc640 1 -- 172.21.15.38:0/724932207 <== mon.2 v2:172.21.15.195:3301/0 7 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v20)=0 v20) ==== 121+0+0 (secure 0 0 0) 0x7fd7cc009ce0 con 0x7fd7c8005c60 2024-09-21T03:33:11.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.681+0000 7fd7d6ef5640 1 -- 172.21.15.38:0/724932207 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd7c8005c60 msgr2=0x7fd7c80d4df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:11.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.681+0000 7fd7d6ef5640 1 --2- 172.21.15.38:0/724932207 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd7c8005c60 0x7fd7c80d4df0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc033000 tx=0x7fd7cc004270 comp rx=0 tx=0).stop 2024-09-21T03:33:11.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.681+0000 7fd7d6ef5640 1 -- 172.21.15.38:0/724932207 shutdown_connections 2024-09-21T03:33:11.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.682+0000 7fd7d6ef5640 1 --2- 172.21.15.38:0/724932207 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd7c8047da0 0x7fd7c801f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:11.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.682+0000 7fd7d6ef5640 1 --2- 172.21.15.38:0/724932207 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7c8036970 0x7fd7c8021890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:11.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.682+0000 7fd7d6ef5640 1 --2- 172.21.15.38:0/724932207 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd7c8005c60 0x7fd7c80d4df0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:11.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.682+0000 7fd7d6ef5640 1 -- 172.21.15.38:0/724932207 >> 172.21.15.38:0/724932207 conn(0x7fd7c8051420 msgr2=0x7fd7c80d9990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:11.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.682+0000 7fd7d6ef5640 1 -- 172.21.15.38:0/724932207 shutdown_connections 2024-09-21T03:33:11.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.682+0000 7fd7d6ef5640 1 -- 172.21.15.38:0/724932207 wait complete. 2024-09-21T03:33:11.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.684+0000 7f58117fa640 1 -- 172.21.15.38:0/2617562833 <== mon.2 v2:172.21.15.195:3301/0 7 ==== mon_command_ack([{prefix=config set, name=rados_osd_op_timeout}]=0 v21)=0 v21) ==== 120+0+0 (secure 0 0 0) 0x7f5808018420 con 0x7f58140cf840 2024-09-21T03:33:11.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.686+0000 7f5819feb640 1 -- 172.21.15.38:0/2617562833 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f58140cf840 msgr2=0x7f5814161900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:11.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.686+0000 7f5819feb640 1 --2- 172.21.15.38:0/2617562833 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f58140cf840 0x7f5814161900 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f580800e700 tx=0x7f580800ebc0 comp rx=0 tx=0).stop 2024-09-21T03:33:11.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.686+0000 7f5819feb640 1 -- 172.21.15.38:0/2617562833 shutdown_connections 2024-09-21T03:33:11.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.686+0000 7f5819feb640 1 --2- 172.21.15.38:0/2617562833 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5814162710 0x7f58141f79c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:11.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.686+0000 7f5819feb640 1 --2- 172.21.15.38:0/2617562833 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f58140cf840 0x7f5814161900 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:11.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.686+0000 7f5819feb640 1 --2- 172.21.15.38:0/2617562833 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f581406d2b0 0x7f58141613c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:11.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.686+0000 7f5819feb640 1 -- 172.21.15.38:0/2617562833 >> 172.21.15.38:0/2617562833 conn(0x7f58140d1960 msgr2=0x7f58140b9710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:11.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.686+0000 7f5819feb640 1 -- 172.21.15.38:0/2617562833 shutdown_connections 2024-09-21T03:33:11.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.686+0000 7f5819feb640 1 -- 172.21.15.38:0/2617562833 wait complete. 2024-09-21T03:33:11.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.688+0000 7f89eca9b640 1 -- 172.21.15.38:0/1672971093 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_command([{prefix=config set, name=debug_client}] v 0) -- 0x7f89cc005580 con 0x7f89e80ca6f0 2024-09-21T03:33:11.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.694+0000 7f89d6ffd640 1 -- 172.21.15.38:0/1672971093 <== mon.2 v2:172.21.15.195:3301/0 7 ==== mon_command_ack([{prefix=config set, name=debug_client}]=0 v22)=0 v22) ==== 111+0+0 (secure 0 0 0) 0x7f89d803caf0 con 0x7f89e80ca6f0 2024-09-21T03:33:11.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.696+0000 7f89eca9b640 1 -- 172.21.15.38:0/1672971093 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f89e80ca6f0 msgr2=0x7f89e80a0550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:11.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.696+0000 7f89eca9b640 1 --2- 172.21.15.38:0/1672971093 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f89e80ca6f0 0x7f89e80a0550 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f89d8009970 tx=0x7f89d8033b80 comp rx=0 tx=0).stop 2024-09-21T03:33:11.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.696+0000 7f89eca9b640 1 -- 172.21.15.38:0/1672971093 shutdown_connections 2024-09-21T03:33:11.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.696+0000 7f89eca9b640 1 --2- 172.21.15.38:0/1672971093 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f89e809b3f0 0x7f89e809b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:11.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.696+0000 7f89eca9b640 1 --2- 172.21.15.38:0/1672971093 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f89e80cc0b0 0x7f89e80a0a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:11.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.696+0000 7f89eca9b640 1 --2- 172.21.15.38:0/1672971093 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f89e80ca6f0 0x7f89e80a0550 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:11.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.696+0000 7f89eca9b640 1 -- 172.21.15.38:0/1672971093 >> 172.21.15.38:0/1672971093 conn(0x7f89e80b1b90 msgr2=0x7f89e80703b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:11.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.696+0000 7f89eca9b640 1 -- 172.21.15.38:0/1672971093 shutdown_connections 2024-09-21T03:33:11.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.696+0000 7f89eca9b640 1 -- 172.21.15.38:0/1672971093 wait complete. 2024-09-21T03:33:11.701 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mgr', 'client_mount_timeout', '30'], check_status=True, hostname='smithi038') 2024-09-21T03:33:11.701 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mgr', 'debug_client', '20'], check_status=True, hostname='smithi038') 2024-09-21T03:33:11.709 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mgr', 'debug_mgr', '20'], check_status=True, hostname='smithi038') 2024-09-21T03:33:11.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.708+0000 7f7282863640 1 -- 172.21.15.38:0/199173135 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command([{prefix=config set, name=debug_mgr}] v 0) -- 0x7f727c0b5f90 con 0x7f727c072d90 2024-09-21T03:33:11.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.710+0000 7f7278ff9640 1 -- 172.21.15.38:0/199173135 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_mgr}]=0 v23)=0 v23) ==== 108+0+0 (secure 0 0 0) 0x7f7270012c00 con 0x7f727c072d90 2024-09-21T03:33:11.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.711+0000 7f7282863640 1 -- 172.21.15.38:0/199173135 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f727c072d90 msgr2=0x7f727c1529c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:11.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.711+0000 7f7282863640 1 --2- 172.21.15.38:0/199173135 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f727c072d90 0x7f727c1529c0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f7270001eb0 tx=0x7f727000c890 comp rx=0 tx=0).stop 2024-09-21T03:33:11.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.711+0000 7f7282863640 1 -- 172.21.15.38:0/199173135 shutdown_connections 2024-09-21T03:33:11.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.711+0000 7f7282863640 1 --2- 172.21.15.38:0/199173135 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f727c15ac20 0x7f727c0c4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:11.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.711+0000 7f7282863640 1 --2- 172.21.15.38:0/199173135 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f727c072d90 0x7f727c1529c0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:11.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.711+0000 7f7282863640 1 --2- 172.21.15.38:0/199173135 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f727c06ddf0 0x7f727c0af030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:11.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.711+0000 7f7282863640 1 -- 172.21.15.38:0/199173135 >> 172.21.15.38:0/199173135 conn(0x7f727c0b1b90 msgr2=0x7f727c0a0af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:11.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.711+0000 7f7282863640 1 -- 172.21.15.38:0/199173135 shutdown_connections 2024-09-21T03:33:11.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.711+0000 7f7282863640 1 -- 172.21.15.38:0/199173135 wait complete. 2024-09-21T03:33:11.724 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mgr', 'debug_ms', '1'], check_status=True, hostname='smithi038') 2024-09-21T03:33:11.724 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mgr', 'mon_warn_on_pool_no_app', 'False'], check_status=True, hostname='smithi038') 2024-09-21T03:33:11.724 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mon', 'mon_op_complaint_time', '120'], check_status=True, hostname='smithi038') 2024-09-21T03:33:11.724 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'osd', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi038') 2024-09-21T03:33:11.724 DEBUG:teuthology.orchestra.run.smithi038:> sudo ceph --cluster ceph config dump 2024-09-21T03:33:11.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.877+0000 7f951680e640 1 -- 172.21.15.38:0/197601789 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9510162440 msgr2=0x7f9510162850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:11.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.877+0000 7f951680e640 1 --2- 172.21.15.38:0/197601789 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9510162440 0x7f9510162850 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f9504005fd0 tx=0x7f950402a670 comp rx=0 tx=0).stop 2024-09-21T03:33:11.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.877+0000 7f951680e640 1 -- 172.21.15.38:0/197601789 shutdown_connections 2024-09-21T03:33:11.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.877+0000 7f951680e640 1 --2- 172.21.15.38:0/197601789 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9510162440 0x7f9510162850 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:11.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.877+0000 7f951680e640 1 -- 172.21.15.38:0/197601789 >> 172.21.15.38:0/197601789 conn(0x7f9510073e30 msgr2=0x7f9510074240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:11.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.877+0000 7f951680e640 1 -- 172.21.15.38:0/197601789 shutdown_connections 2024-09-21T03:33:11.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.877+0000 7f951680e640 1 -- 172.21.15.38:0/197601789 wait complete. 2024-09-21T03:33:11.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.879+0000 7f951680e640 1 Processor -- start 2024-09-21T03:33:11.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.879+0000 7f951680e640 1 -- start start 2024-09-21T03:33:11.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.880+0000 7f951680e640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f95100d0300 0x7f9510143e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:11.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.880+0000 7f951680e640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9510155910 0x7f9510156340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:11.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.880+0000 7f951680e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9510162440 0x7f95100b1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:11.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.880+0000 7f951680e640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f951006d4f0 con 0x7f9510162440 2024-09-21T03:33:11.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.880+0000 7f951680e640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f95100b50d0 con 0x7f95100d0300 2024-09-21T03:33:11.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.880+0000 7f951680e640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f9510158450 con 0x7f9510155910 2024-09-21T03:33:11.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.880+0000 7f9514dab640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9510162440 0x7f95100b1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:11.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.880+0000 7f9514dab640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9510162440 0x7f95100b1eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41516/0 (socket says 172.21.15.38:41516) 2024-09-21T03:33:11.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.880+0000 7f9514dab640 1 -- 172.21.15.38:0/4149903334 learned_addr learned my addr 172.21.15.38:0/4149903334 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:33:11.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.880+0000 7f950f7fe640 1 --2- 172.21.15.38:0/4149903334 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9510155910 0x7f9510156340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:11.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.880+0000 7f950ffff640 1 --2- 172.21.15.38:0/4149903334 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f95100d0300 0x7f9510143e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:11.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.881+0000 7f9514dab640 1 -- 172.21.15.38:0/4149903334 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9510155910 msgr2=0x7f9510156340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:11.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.881+0000 7f9514dab640 1 --2- 172.21.15.38:0/4149903334 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9510155910 0x7f9510156340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:11.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.881+0000 7f9514dab640 1 -- 172.21.15.38:0/4149903334 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f95100d0300 msgr2=0x7f9510143e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:11.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.881+0000 7f9514dab640 1 --2- 172.21.15.38:0/4149903334 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f95100d0300 0x7f9510143e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:11.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.881+0000 7f9514dab640 1 -- 172.21.15.38:0/4149903334 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9510146a20 con 0x7f9510162440 2024-09-21T03:33:11.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.881+0000 7f9514dab640 1 --2- 172.21.15.38:0/4149903334 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9510162440 0x7f95100b1eb0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f950800c730 tx=0x7f950800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:11.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.881+0000 7f950cff9640 1 -- 172.21.15.38:0/4149903334 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9508003b70 con 0x7f9510162440 2024-09-21T03:33:11.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.881+0000 7f950cff9640 1 -- 172.21.15.38:0/4149903334 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9508003d10 con 0x7f9510162440 2024-09-21T03:33:11.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.881+0000 7f951680e640 1 -- 172.21.15.38:0/4149903334 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9510146d00 con 0x7f9510162440 2024-09-21T03:33:11.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.881+0000 7f950cff9640 1 -- 172.21.15.38:0/4149903334 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f950800aab0 con 0x7f9510162440 2024-09-21T03:33:11.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.881+0000 7f951680e640 1 -- 172.21.15.38:0/4149903334 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95101472c0 con 0x7f9510162440 2024-09-21T03:33:11.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.881+0000 7f950cff9640 1 -- 172.21.15.38:0/4149903334 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f950800ac50 con 0x7f9510162440 2024-09-21T03:33:11.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.881+0000 7f950cff9640 1 -- 172.21.15.38:0/4149903334 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f9508012930 con 0x7f9510162440 2024-09-21T03:33:11.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.882+0000 7f951680e640 1 -- 172.21.15.38:0/4149903334 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9510162850 con 0x7f9510162440 2024-09-21T03:33:11.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.884+0000 7f950cff9640 1 -- 172.21.15.38:0/4149903334 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f95080050d0 con 0x7f9510162440 2024-09-21T03:33:11.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.946+0000 7f951680e640 1 -- 172.21.15.38:0/4149903334 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f9510074d60 con 0x7f9510162440 2024-09-21T03:33:11.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.946+0000 7f950cff9640 1 -- 172.21.15.38:0/4149903334 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v23) ==== 59+0+1472 (secure 0 0 0) 0x7f95080052b0 con 0x7f9510162440 2024-09-21T03:33:11.948 INFO:teuthology.orchestra.run.smithi038.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-21T03:33:11.948 INFO:teuthology.orchestra.run.smithi038.stdout:mon advanced mon_op_complaint_time 120 2024-09-21T03:33:11.948 INFO:teuthology.orchestra.run.smithi038.stdout:mgr advanced client_mount_timeout 30 2024-09-21T03:33:11.948 INFO:teuthology.orchestra.run.smithi038.stdout:mgr advanced debug_client 20/20 2024-09-21T03:33:11.948 INFO:teuthology.orchestra.run.smithi038.stdout:mgr advanced debug_mgr 20/20 2024-09-21T03:33:11.948 INFO:teuthology.orchestra.run.smithi038.stdout:mgr advanced debug_ms 1/1 2024-09-21T03:33:11.949 INFO:teuthology.orchestra.run.smithi038.stdout:mgr dev mon_warn_on_pool_no_app false 2024-09-21T03:33:11.949 INFO:teuthology.orchestra.run.smithi038.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-21T03:33:11.949 INFO:teuthology.orchestra.run.smithi038.stdout:mds advanced debug_mds 20/20 2024-09-21T03:33:11.949 INFO:teuthology.orchestra.run.smithi038.stdout:mds advanced debug_mds_balancer 20/20 2024-09-21T03:33:11.949 INFO:teuthology.orchestra.run.smithi038.stdout:mds advanced debug_ms 2/2 2024-09-21T03:33:11.949 INFO:teuthology.orchestra.run.smithi038.stdout:mds dev mds_debug_frag true 2024-09-21T03:33:11.949 INFO:teuthology.orchestra.run.smithi038.stdout:mds dev mds_debug_scatterstat true 2024-09-21T03:33:11.949 INFO:teuthology.orchestra.run.smithi038.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-21T03:33:11.949 INFO:teuthology.orchestra.run.smithi038.stdout:mds dev mds_verify_scatter true 2024-09-21T03:33:11.949 INFO:teuthology.orchestra.run.smithi038.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-21T03:33:11.949 INFO:teuthology.orchestra.run.smithi038.stdout:mds advanced rados_mon_op_timeout 900 2024-09-21T03:33:11.949 INFO:teuthology.orchestra.run.smithi038.stdout:mds advanced rados_osd_op_timeout 900 2024-09-21T03:33:11.949 INFO:teuthology.orchestra.run.smithi038.stdout:client advanced client_mount_timeout 600 2024-09-21T03:33:11.949 INFO:teuthology.orchestra.run.smithi038.stdout:client advanced debug_client 20/20 2024-09-21T03:33:11.950 INFO:teuthology.orchestra.run.smithi038.stdout:client advanced debug_ms 1/1 2024-09-21T03:33:11.950 INFO:teuthology.orchestra.run.smithi038.stdout:client advanced rados_mon_op_timeout 900 2024-09-21T03:33:11.950 INFO:teuthology.orchestra.run.smithi038.stdout:client advanced rados_osd_op_timeout 900 2024-09-21T03:33:11.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.947+0000 7f951680e640 1 -- 172.21.15.38:0/4149903334 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9510162440 msgr2=0x7f95100b1eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:11.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.947+0000 7f951680e640 1 --2- 172.21.15.38:0/4149903334 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9510162440 0x7f95100b1eb0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f950800c730 tx=0x7f950800cbf0 comp rx=0 tx=0).stop 2024-09-21T03:33:11.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.947+0000 7f951680e640 1 -- 172.21.15.38:0/4149903334 shutdown_connections 2024-09-21T03:33:11.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.947+0000 7f951680e640 1 --2- 172.21.15.38:0/4149903334 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9510162440 0x7f95100b1eb0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:11.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.947+0000 7f951680e640 1 --2- 172.21.15.38:0/4149903334 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9510155910 0x7f9510156340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:11.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.947+0000 7f951680e640 1 --2- 172.21.15.38:0/4149903334 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f95100d0300 0x7f9510143e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:11.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.947+0000 7f951680e640 1 -- 172.21.15.38:0/4149903334 >> 172.21.15.38:0/4149903334 conn(0x7f9510073e30 msgr2=0x7f951014d550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:11.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.947+0000 7f951680e640 1 -- 172.21.15.38:0/4149903334 shutdown_connections 2024-09-21T03:33:11.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:11.947+0000 7f951680e640 1 -- 172.21.15.38:0/4149903334 wait complete. 2024-09-21T03:33:11.959 INFO:tasks.ceph:Setting crush tunables to default 2024-09-21T03:33:11.960 DEBUG:teuthology.orchestra.run.smithi038:> sudo ceph --cluster ceph osd crush tunables default 2024-09-21T03:33:12.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:12.074+0000 7f4d43b12640 1 -- 172.21.15.38:0/2271803321 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4d3c15a300 msgr2=0x7f4d3c06f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:12.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:12.074+0000 7f4d43b12640 1 --2- 172.21.15.38:0/2271803321 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4d3c15a300 0x7f4d3c06f4c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f4d3c075160 tx=0x7f4d3002a530 comp rx=0 tx=0).stop 2024-09-21T03:33:12.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:12.074+0000 7f4d43b12640 1 -- 172.21.15.38:0/2271803321 shutdown_connections 2024-09-21T03:33:12.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:12.074+0000 7f4d43b12640 1 --2- 172.21.15.38:0/2271803321 >> v2:172.21.15.195:3300/0 conn(0x7f4d3c0b2ca0 0x7f4d3c0b30b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:12.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:12.074+0000 7f4d43b12640 1 --2- 172.21.15.38:0/2271803321 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4d3c15a300 0x7f4d3c06f4c0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:12.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:12.074+0000 7f4d43b12640 1 -- 172.21.15.38:0/2271803321 >> 172.21.15.38:0/2271803321 conn(0x7f4d3c0cdca0 msgr2=0x7f4d3c0ce0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:12.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:12.074+0000 7f4d43b12640 1 -- 172.21.15.38:0/2271803321 shutdown_connections 2024-09-21T03:33:12.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:12.074+0000 7f4d43b12640 1 -- 172.21.15.38:0/2271803321 wait complete. 2024-09-21T03:33:12.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:12.076+0000 7f4d43b12640 1 Processor -- start 2024-09-21T03:33:12.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:12.076+0000 7f4d43b12640 1 -- start start 2024-09-21T03:33:12.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:12.077+0000 7f4d43b12640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4d3c0ae970 0x7f4d3c1f76e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:12.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:12.077+0000 7f4d43b12640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4d3c0b2ca0 0x7f4d3c1595b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:12.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:12.077+0000 7f4d43b12640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4d3c15a300 0x7f4d3c0b1d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:12.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:12.077+0000 7f4d43b12640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f4d3c160530 con 0x7f4d3c0b2ca0 2024-09-21T03:33:12.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:12.077+0000 7f4d43b12640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f4d3c1603b0 con 0x7f4d3c0ae970 2024-09-21T03:33:12.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:12.077+0000 7f4d43b12640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f4d3c1606b0 con 0x7f4d3c15a300 2024-09-21T03:33:12.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:12.077+0000 7f4d410ad640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4d3c0b2ca0 0x7f4d3c1595b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:12.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:12.077+0000 7f4d410ad640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4d3c0b2ca0 0x7f4d3c1595b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41534/0 (socket says 172.21.15.38:41534) 2024-09-21T03:33:12.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:12.077+0000 7f4d410ad640 1 -- 172.21.15.38:0/2278771198 learned_addr learned my addr 172.21.15.38:0/2278771198 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:33:12.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:12.077+0000 7f4d418ae640 1 --2- 172.21.15.38:0/2278771198 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4d3c0ae970 0x7f4d3c1f76e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:12.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:12.077+0000 7f4d420af640 1 --2- 172.21.15.38:0/2278771198 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4d3c15a300 0x7f4d3c0b1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:12.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:12.078+0000 7f4d410ad640 1 -- 172.21.15.38:0/2278771198 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4d3c15a300 msgr2=0x7f4d3c0b1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:12.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:12.078+0000 7f4d410ad640 1 --2- 172.21.15.38:0/2278771198 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4d3c15a300 0x7f4d3c0b1d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:12.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:12.078+0000 7f4d410ad640 1 -- 172.21.15.38:0/2278771198 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4d3c0ae970 msgr2=0x7f4d3c1f76e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:12.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:12.078+0000 7f4d410ad640 1 --2- 172.21.15.38:0/2278771198 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4d3c0ae970 0x7f4d3c1f76e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:12.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:12.078+0000 7f4d410ad640 1 -- 172.21.15.38:0/2278771198 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d3c1f7c20 con 0x7f4d3c0b2ca0 2024-09-21T03:33:12.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:12.078+0000 7f4d410ad640 1 --2- 172.21.15.38:0/2278771198 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4d3c0b2ca0 0x7f4d3c1595b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f4d3800c3f0 tx=0x7f4d3800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:12.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:12.078+0000 7f4d2e7fc640 1 -- 172.21.15.38:0/2278771198 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d38006890 con 0x7f4d3c0b2ca0 2024-09-21T03:33:12.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:12.078+0000 7f4d2e7fc640 1 -- 172.21.15.38:0/2278771198 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4d38006b20 con 0x7f4d3c0b2ca0 2024-09-21T03:33:12.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:12.078+0000 7f4d43b12640 1 -- 172.21.15.38:0/2278771198 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d3c09de70 con 0x7f4d3c0b2ca0 2024-09-21T03:33:12.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:12.078+0000 7f4d2e7fc640 1 -- 172.21.15.38:0/2278771198 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d38007020 con 0x7f4d3c0b2ca0 2024-09-21T03:33:12.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:12.078+0000 7f4d43b12640 1 -- 172.21.15.38:0/2278771198 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d3c09e430 con 0x7f4d3c0b2ca0 2024-09-21T03:33:12.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:12.078+0000 7f4d2e7fc640 1 -- 172.21.15.38:0/2278771198 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f4d38019020 con 0x7f4d3c0b2ca0 2024-09-21T03:33:12.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:12.078+0000 7f4d2e7fc640 1 -- 172.21.15.38:0/2278771198 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f4d38004940 con 0x7f4d3c0b2ca0 2024-09-21T03:33:12.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:12.079+0000 7f4d43b12640 1 -- 172.21.15.38:0/2278771198 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d3c160b60 con 0x7f4d3c0b2ca0 2024-09-21T03:33:12.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:12.081+0000 7f4d2e7fc640 1 -- 172.21.15.38:0/2278771198 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f4d38020020 con 0x7f4d3c0b2ca0 2024-09-21T03:33:12.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:12.147+0000 7f4d43b12640 1 -- 172.21.15.38:0/2278771198 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f4d3c09db90 con 0x7f4d3c0b2ca0 2024-09-21T03:33:12.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:12.149+0000 7f4d2e7fc640 1 -- 172.21.15.38:0/2278771198 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7f4d3c1f7c20 con 0x7f4d3c0b2ca0 2024-09-21T03:33:12.151 INFO:teuthology.orchestra.run.smithi038.stderr:adjusted tunables profile to default 2024-09-21T03:33:12.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:12.150+0000 7f4d43b12640 1 -- 172.21.15.38:0/2278771198 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4d3c0b2ca0 msgr2=0x7f4d3c1595b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:12.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:12.150+0000 7f4d43b12640 1 --2- 172.21.15.38:0/2278771198 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4d3c0b2ca0 0x7f4d3c1595b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f4d3800c3f0 tx=0x7f4d3800c8b0 comp rx=0 tx=0).stop 2024-09-21T03:33:12.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:12.150+0000 7f4d43b12640 1 -- 172.21.15.38:0/2278771198 shutdown_connections 2024-09-21T03:33:12.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:12.150+0000 7f4d43b12640 1 --2- 172.21.15.38:0/2278771198 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4d3c15a300 0x7f4d3c0b1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:12.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:12.150+0000 7f4d43b12640 1 --2- 172.21.15.38:0/2278771198 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4d3c0b2ca0 0x7f4d3c1595b0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:12.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:12.150+0000 7f4d43b12640 1 --2- 172.21.15.38:0/2278771198 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4d3c0ae970 0x7f4d3c1f76e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:12.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:12.150+0000 7f4d43b12640 1 -- 172.21.15.38:0/2278771198 >> 172.21.15.38:0/2278771198 conn(0x7f4d3c0cdca0 msgr2=0x7f4d3c06d5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:12.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:12.150+0000 7f4d43b12640 1 -- 172.21.15.38:0/2278771198 shutdown_connections 2024-09-21T03:33:12.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:12.150+0000 7f4d43b12640 1 -- 172.21.15.38:0/2278771198 wait complete. 2024-09-21T03:33:12.163 INFO:tasks.ceph:check_enable_crimson: False 2024-09-21T03:33:12.163 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-09-21T03:33:12.163 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-21T03:33:12.163 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-09-21T03:33:12.188 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-21T03:33:12.188 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-09-21T03:33:12.254 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-21T03:33:12.254 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-09-21T03:33:12.319 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-21T03:33:12.319 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-09-21T03:33:12.385 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-09-21T03:33:12.385 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-09-21T03:33:12.410 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-09-21T03:33:12.410 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-09-21T03:33:12.475 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-09-21T03:33:12.475 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-09-21T03:33:12.540 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-09-21T03:33:12.540 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-09-21T03:33:12.605 DEBUG:teuthology.orchestra.run.smithi195:> sudo ceph --cluster ceph osd new dbab183f-7e63-48b8-960a-ae55fd5bf565 0 2024-09-21T03:33:12.760 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:12.757+0000 7fa066f2b640 1 -- 172.21.15.195:0/2410545650 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa0600d0bb0 msgr2=0x7fa0600b5be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:12.760 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:12.757+0000 7fa066f2b640 1 --2- 172.21.15.195:0/2410545650 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa0600d0bb0 0x7fa0600b5be0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fa058005fd0 tx=0x7fa05802a820 comp rx=0 tx=0).stop 2024-09-21T03:33:12.760 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:12.757+0000 7fa066f2b640 1 -- 172.21.15.195:0/2410545650 shutdown_connections 2024-09-21T03:33:12.760 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:12.757+0000 7fa066f2b640 1 --2- 172.21.15.195:0/2410545650 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa0600d0bb0 0x7fa0600b5be0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:12.761 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:12.757+0000 7fa066f2b640 1 --2- 172.21.15.195:0/2410545650 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa06015ca90 0x7fa0600d0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:12.761 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:12.757+0000 7fa066f2b640 1 -- 172.21.15.195:0/2410545650 >> 172.21.15.195:0/2410545650 conn(0x7fa0600b46e0 msgr2=0x7fa0600d23e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:12.761 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:12.757+0000 7fa066f2b640 1 -- 172.21.15.195:0/2410545650 shutdown_connections 2024-09-21T03:33:12.761 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:12.757+0000 7fa066f2b640 1 -- 172.21.15.195:0/2410545650 wait complete. 2024-09-21T03:33:12.761 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:12.758+0000 7fa066f2b640 1 Processor -- start 2024-09-21T03:33:12.761 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:12.758+0000 7fa066f2b640 1 -- start start 2024-09-21T03:33:12.765 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:12.760+0000 7fa066f2b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa0600ccc40 0x7fa0601e1d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:12.765 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:12.760+0000 7fa066f2b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa0600d0bb0 0x7fa0601f24c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:12.765 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:12.760+0000 7fa066f2b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa06015ca90 0x7fa0601f2a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:12.765 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:12.760+0000 7fa066f2b640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa0600b6670 con 0x7fa06015ca90 2024-09-21T03:33:12.765 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:12.760+0000 7fa066f2b640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fa0600b64f0 con 0x7fa0600ccc40 2024-09-21T03:33:12.765 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:12.760+0000 7fa066f2b640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fa0600b67f0 con 0x7fa0600d0bb0 2024-09-21T03:33:12.765 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:12.760+0000 7fa05ffff640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa0600d0bb0 0x7fa0601f24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:12.765 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:12.760+0000 7fa064cc7640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa0600ccc40 0x7fa0601e1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:12.765 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:12.760+0000 7fa05ffff640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa0600d0bb0 0x7fa0601f24c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.195:56702/0 (socket says 172.21.15.195:56702) 2024-09-21T03:33:12.766 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:12.760+0000 7fa05ffff640 1 -- 172.21.15.195:0/49124018 learned_addr learned my addr 172.21.15.195:0/49124018 (peer_addr_for_me v2:172.21.15.195:0/0) 2024-09-21T03:33:12.766 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:12.760+0000 7fa0654c8640 1 --2- 172.21.15.195:0/49124018 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa06015ca90 0x7fa0601f2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:12.766 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:12.760+0000 7fa064cc7640 1 -- 172.21.15.195:0/49124018 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa0600d0bb0 msgr2=0x7fa0601f24c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:12.766 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:12.760+0000 7fa064cc7640 1 --2- 172.21.15.195:0/49124018 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa0600d0bb0 0x7fa0601f24c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:12.766 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:12.760+0000 7fa064cc7640 1 -- 172.21.15.195:0/49124018 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa06015ca90 msgr2=0x7fa0601f2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:12.766 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:12.760+0000 7fa064cc7640 1 --2- 172.21.15.195:0/49124018 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa06015ca90 0x7fa0601f2a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:12.766 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:12.760+0000 7fa064cc7640 1 -- 172.21.15.195:0/49124018 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0601f2f40 con 0x7fa0600ccc40 2024-09-21T03:33:12.766 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:12.760+0000 7fa05ffff640 1 --2- 172.21.15.195:0/49124018 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa0600d0bb0 0x7fa0601f24c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:33:12.766 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:12.761+0000 7fa064cc7640 1 --2- 172.21.15.195:0/49124018 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa0600ccc40 0x7fa0601e1d20 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fa05400c3f0 tx=0x7fa05400c8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:12.766 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:12.761+0000 7fa05d7fa640 1 -- 172.21.15.195:0/49124018 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa054011790 con 0x7fa0600ccc40 2024-09-21T03:33:12.767 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:12.761+0000 7fa05d7fa640 1 -- 172.21.15.195:0/49124018 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa054017070 con 0x7fa0600ccc40 2024-09-21T03:33:12.767 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:12.761+0000 7fa05d7fa640 1 -- 172.21.15.195:0/49124018 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa054011cc0 con 0x7fa0600ccc40 2024-09-21T03:33:12.767 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:12.761+0000 7fa066f2b640 1 -- 172.21.15.195:0/49124018 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0601e2970 con 0x7fa0600ccc40 2024-09-21T03:33:12.767 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:12.761+0000 7fa066f2b640 1 -- 172.21.15.195:0/49124018 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0601e2f30 con 0x7fa0600ccc40 2024-09-21T03:33:12.767 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:12.761+0000 7fa05d7fa640 1 -- 172.21.15.195:0/49124018 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fa05401c460 con 0x7fa0600ccc40 2024-09-21T03:33:12.767 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:12.761+0000 7fa05d7fa640 1 -- 172.21.15.195:0/49124018 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7fa054012810 con 0x7fa0600ccc40 2024-09-21T03:33:12.767 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:12.762+0000 7fa066f2b640 1 -- 172.21.15.195:0/49124018 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0600b6d80 con 0x7fa0600ccc40 2024-09-21T03:33:12.767 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:12.764+0000 7fa05d7fa640 1 -- 172.21.15.195:0/49124018 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fa0600d0fc0 con 0x7fa0600ccc40 2024-09-21T03:33:12.837 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:12.834+0000 7fa066f2b640 1 -- 172.21.15.195:0/49124018 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "osd new", "uuid": "dbab183f-7e63-48b8-960a-ae55fd5bf565", "id": 0} v 0) -- 0x7fa0600762a0 con 0x7fa0600ccc40 2024-09-21T03:33:12.845 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:12.842+0000 7fa05d7fa640 1 -- 172.21.15.195:0/49124018 <== mon.1 v2:172.21.15.195:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "dbab183f-7e63-48b8-960a-ae55fd5bf565", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7fa054012a60 con 0x7fa0600ccc40 2024-09-21T03:33:12.845 INFO:teuthology.orchestra.run.smithi195.stdout:0 2024-09-21T03:33:12.846 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:12.843+0000 7fa066f2b640 1 -- 172.21.15.195:0/49124018 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa0600ccc40 msgr2=0x7fa0601e1d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:12.846 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:12.843+0000 7fa066f2b640 1 --2- 172.21.15.195:0/49124018 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa0600ccc40 0x7fa0601e1d20 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fa05400c3f0 tx=0x7fa05400c8b0 comp rx=0 tx=0).stop 2024-09-21T03:33:12.846 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:12.843+0000 7fa066f2b640 1 -- 172.21.15.195:0/49124018 shutdown_connections 2024-09-21T03:33:12.846 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:12.843+0000 7fa066f2b640 1 --2- 172.21.15.195:0/49124018 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa06015ca90 0x7fa0601f2a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:12.846 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:12.843+0000 7fa066f2b640 1 --2- 172.21.15.195:0/49124018 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa0600d0bb0 0x7fa0601f24c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:12.846 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:12.843+0000 7fa066f2b640 1 --2- 172.21.15.195:0/49124018 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa0600ccc40 0x7fa0601e1d20 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:12.847 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:12.843+0000 7fa066f2b640 1 -- 172.21.15.195:0/49124018 >> 172.21.15.195:0/49124018 conn(0x7fa0600b46e0 msgr2=0x7fa0600cd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:12.847 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:12.843+0000 7fa066f2b640 1 -- 172.21.15.195:0/49124018 shutdown_connections 2024-09-21T03:33:12.847 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:12.843+0000 7fa066f2b640 1 -- 172.21.15.195:0/49124018 wait complete. 2024-09-21T03:33:12.857 DEBUG:teuthology.orchestra.run.smithi195:> sudo ceph --cluster ceph osd new fa1c0e8a-ebc1-4359-8146-bd5342abbc16 1 2024-09-21T03:33:12.970 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:12.966+0000 7f7bd13c6640 1 -- 172.21.15.195:0/3873863892 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7bcc0b5260 msgr2=0x7f7bcc15ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:12.970 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:12.966+0000 7f7bd13c6640 1 --2- 172.21.15.195:0/3873863892 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7bcc0b5260 0x7f7bcc15ba80 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f7bc0005fd0 tx=0x7f7bc002a670 comp rx=0 tx=0).stop 2024-09-21T03:33:12.971 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:12.966+0000 7f7bd13c6640 1 -- 172.21.15.195:0/3873863892 shutdown_connections 2024-09-21T03:33:12.971 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:12.966+0000 7f7bd13c6640 1 --2- 172.21.15.195:0/3873863892 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7bcc0b5260 0x7f7bcc15ba80 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:12.971 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:12.966+0000 7f7bd13c6640 1 -- 172.21.15.195:0/3873863892 >> 172.21.15.195:0/3873863892 conn(0x7f7bcc0cdca0 msgr2=0x7f7bcc0ce0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:12.971 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:12.966+0000 7f7bd13c6640 1 -- 172.21.15.195:0/3873863892 shutdown_connections 2024-09-21T03:33:12.971 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:12.966+0000 7f7bd13c6640 1 -- 172.21.15.195:0/3873863892 wait complete. 2024-09-21T03:33:12.975 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:12.968+0000 7f7bd13c6640 1 Processor -- start 2024-09-21T03:33:12.976 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:12.968+0000 7f7bd13c6640 1 -- start start 2024-09-21T03:33:12.976 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:12.970+0000 7f7bd13c6640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7bcc06def0 0x7f7bcc14d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:12.976 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:12.970+0000 7f7bd13c6640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7bcc0b5260 0x7f7bcc157e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:12.976 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:12.970+0000 7f7bd13c6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7bcc15bfc0 0x7f7bcc06cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:12.976 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:12.970+0000 7f7bd13c6640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f7bcc071300 con 0x7f7bcc15bfc0 2024-09-21T03:33:12.977 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:12.970+0000 7f7bd13c6640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f7bcc071480 con 0x7f7bcc06def0 2024-09-21T03:33:12.977 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:12.970+0000 7f7bd13c6640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f7bcc071600 con 0x7f7bcc0b5260 2024-09-21T03:33:12.977 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:12.970+0000 7f7bca7fc640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7bcc0b5260 0x7f7bcc157e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:12.977 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:12.970+0000 7f7bcaffd640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7bcc06def0 0x7f7bcc14d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:12.977 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:12.970+0000 7f7bca7fc640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7bcc0b5260 0x7f7bcc157e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.195:56712/0 (socket says 172.21.15.195:56712) 2024-09-21T03:33:12.977 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:12.970+0000 7f7bca7fc640 1 -- 172.21.15.195:0/2850393845 learned_addr learned my addr 172.21.15.195:0/2850393845 (peer_addr_for_me v2:172.21.15.195:0/0) 2024-09-21T03:33:12.977 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:12.970+0000 7f7bcb7fe640 1 --2- 172.21.15.195:0/2850393845 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7bcc15bfc0 0x7f7bcc06cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:12.977 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:12.970+0000 7f7bca7fc640 1 -- 172.21.15.195:0/2850393845 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7bcc06def0 msgr2=0x7f7bcc14d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:12.978 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:12.970+0000 7f7bca7fc640 1 --2- 172.21.15.195:0/2850393845 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7bcc06def0 0x7f7bcc14d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:12.978 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:12.970+0000 7f7bca7fc640 1 -- 172.21.15.195:0/2850393845 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7bcc15bfc0 msgr2=0x7f7bcc06cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:12.978 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:12.970+0000 7f7bca7fc640 1 --2- 172.21.15.195:0/2850393845 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7bcc15bfc0 0x7f7bcc06cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:12.978 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:12.970+0000 7f7bca7fc640 1 -- 172.21.15.195:0/2850393845 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bcc14dbc0 con 0x7f7bcc0b5260 2024-09-21T03:33:12.978 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:12.970+0000 7f7bcaffd640 1 --2- 172.21.15.195:0/2850393845 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7bcc06def0 0x7f7bcc14d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:33:12.978 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:12.970+0000 7f7bca7fc640 1 --2- 172.21.15.195:0/2850393845 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7bcc0b5260 0x7f7bcc157e40 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f7bc0034040 tx=0x7f7bc00039e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:12.978 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:12.970+0000 7f7bb37fe640 1 -- 172.21.15.195:0/2850393845 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bc003e030 con 0x7f7bcc0b5260 2024-09-21T03:33:12.978 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:12.970+0000 7f7bb37fe640 1 -- 172.21.15.195:0/2850393845 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7bc00047a0 con 0x7f7bcc0b5260 2024-09-21T03:33:12.978 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:12.970+0000 7f7bd13c6640 1 -- 172.21.15.195:0/2850393845 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bcc1f6ee0 con 0x7f7bcc0b5260 2024-09-21T03:33:12.978 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:12.970+0000 7f7bb37fe640 1 -- 172.21.15.195:0/2850393845 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bc0035040 con 0x7f7bcc0b5260 2024-09-21T03:33:12.978 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:12.971+0000 7f7bd13c6640 1 -- 172.21.15.195:0/2850393845 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bcc1f74a0 con 0x7f7bcc0b5260 2024-09-21T03:33:12.979 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:12.971+0000 7f7bb37fe640 1 -- 172.21.15.195:0/2850393845 <== mon.2 v2:172.21.15.195:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f7bc0041030 con 0x7f7bcc0b5260 2024-09-21T03:33:12.979 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:12.971+0000 7f7bb37fe640 1 -- 172.21.15.195:0/2850393845 <== mon.2 v2:172.21.15.195:3301/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7f7bc0043da0 con 0x7f7bcc0b5260 2024-09-21T03:33:12.979 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:12.971+0000 7f7bd13c6640 1 -- 172.21.15.195:0/2850393845 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7bcc1587e0 con 0x7f7bcc0b5260 2024-09-21T03:33:12.979 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:12.975+0000 7f7bb37fe640 1 -- 172.21.15.195:0/2850393845 <== mon.2 v2:172.21.15.195:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f7bc0043410 con 0x7f7bcc0b5260 2024-09-21T03:33:13.049 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.046+0000 7f7bd13c6640 1 -- 172.21.15.195:0/2850393845 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_command({"prefix": "osd new", "uuid": "fa1c0e8a-ebc1-4359-8146-bd5342abbc16", "id": 1} v 0) -- 0x7f7bcc071b40 con 0x7f7bcc0b5260 2024-09-21T03:33:13.057 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.054+0000 7f7bb37fe640 1 -- 172.21.15.195:0/2850393845 <== mon.2 v2:172.21.15.195:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "fa1c0e8a-ebc1-4359-8146-bd5342abbc16", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7f7bc004d440 con 0x7f7bcc0b5260 2024-09-21T03:33:13.057 INFO:teuthology.orchestra.run.smithi195.stdout:1 2024-09-21T03:33:13.058 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.055+0000 7f7bd13c6640 1 -- 172.21.15.195:0/2850393845 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7bcc0b5260 msgr2=0x7f7bcc157e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:13.058 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.055+0000 7f7bd13c6640 1 --2- 172.21.15.195:0/2850393845 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7bcc0b5260 0x7f7bcc157e40 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f7bc0034040 tx=0x7f7bc00039e0 comp rx=0 tx=0).stop 2024-09-21T03:33:13.058 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.055+0000 7f7bd13c6640 1 -- 172.21.15.195:0/2850393845 shutdown_connections 2024-09-21T03:33:13.059 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.055+0000 7f7bd13c6640 1 --2- 172.21.15.195:0/2850393845 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7bcc15bfc0 0x7f7bcc06cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:13.059 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.055+0000 7f7bd13c6640 1 --2- 172.21.15.195:0/2850393845 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7bcc0b5260 0x7f7bcc157e40 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:13.059 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.055+0000 7f7bd13c6640 1 --2- 172.21.15.195:0/2850393845 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7bcc06def0 0x7f7bcc14d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:13.059 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.055+0000 7f7bd13c6640 1 -- 172.21.15.195:0/2850393845 >> 172.21.15.195:0/2850393845 conn(0x7f7bcc0cdca0 msgr2=0x7f7bcc15d220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:13.059 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.055+0000 7f7bd13c6640 1 -- 172.21.15.195:0/2850393845 shutdown_connections 2024-09-21T03:33:13.059 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.055+0000 7f7bd13c6640 1 -- 172.21.15.195:0/2850393845 wait complete. 2024-09-21T03:33:13.069 DEBUG:teuthology.orchestra.run.smithi195:> sudo ceph --cluster ceph osd new 2ed3cd2d-385d-4fc6-9869-214552a84803 2 2024-09-21T03:33:13.182 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.178+0000 7fc215900640 1 -- 172.21.15.195:0/4161358676 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc2100cd440 msgr2=0x7fc2100d0670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:13.182 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.178+0000 7fc215900640 1 --2- 172.21.15.195:0/4161358676 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc2100cd440 0x7fc2100d0670 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fc200005fd0 tx=0x7fc20002a540 comp rx=0 tx=0).stop 2024-09-21T03:33:13.182 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.178+0000 7fc215900640 1 -- 172.21.15.195:0/4161358676 shutdown_connections 2024-09-21T03:33:13.182 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.178+0000 7fc215900640 1 --2- 172.21.15.195:0/4161358676 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc2100cd440 0x7fc2100d0670 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:13.182 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.178+0000 7fc215900640 1 --2- 172.21.15.195:0/4161358676 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc2100ccc40 0x7fc21015ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:13.182 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.178+0000 7fc215900640 1 -- 172.21.15.195:0/4161358676 >> 172.21.15.195:0/4161358676 conn(0x7fc2100b46e0 msgr2=0x7fc2100d23e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:13.182 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.178+0000 7fc215900640 1 -- 172.21.15.195:0/4161358676 shutdown_connections 2024-09-21T03:33:13.182 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.178+0000 7fc215900640 1 -- 172.21.15.195:0/4161358676 wait complete. 2024-09-21T03:33:13.183 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.180+0000 7fc215900640 1 Processor -- start 2024-09-21T03:33:13.183 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.180+0000 7fc215900640 1 -- start start 2024-09-21T03:33:13.186 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.181+0000 7fc215900640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc2100ccc40 0x7fc2101f5770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:13.186 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.181+0000 7fc215900640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc2100cd440 0x7fc2101518e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:13.186 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.181+0000 7fc215900640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc2100d0bb0 0x7fc21006def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:13.186 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.181+0000 7fc215900640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fc210070810 con 0x7fc2100cd440 2024-09-21T03:33:13.186 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.181+0000 7fc215900640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fc2100b6e00 con 0x7fc2100ccc40 2024-09-21T03:33:13.186 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.181+0000 7fc215900640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fc210070990 con 0x7fc2100d0bb0 2024-09-21T03:33:13.186 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.181+0000 7fc20effd640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc2100ccc40 0x7fc2101f5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:13.186 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.181+0000 7fc20effd640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc2100ccc40 0x7fc2101f5770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.195:35240/0 (socket says 172.21.15.195:35240) 2024-09-21T03:33:13.186 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.181+0000 7fc20effd640 1 -- 172.21.15.195:0/3943678357 learned_addr learned my addr 172.21.15.195:0/3943678357 (peer_addr_for_me v2:172.21.15.195:0/0) 2024-09-21T03:33:13.187 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.182+0000 7fc20f7fe640 1 --2- 172.21.15.195:0/3943678357 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc2100d0bb0 0x7fc21006def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:13.187 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.182+0000 7fc20e7fc640 1 --2- 172.21.15.195:0/3943678357 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc2100cd440 0x7fc2101518e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:13.187 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.182+0000 7fc20effd640 1 -- 172.21.15.195:0/3943678357 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc2100d0bb0 msgr2=0x7fc21006def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:13.187 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.182+0000 7fc20effd640 1 --2- 172.21.15.195:0/3943678357 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc2100d0bb0 0x7fc21006def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:13.187 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.182+0000 7fc20effd640 1 -- 172.21.15.195:0/3943678357 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc2100cd440 msgr2=0x7fc2101518e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:13.187 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.182+0000 7fc20effd640 1 --2- 172.21.15.195:0/3943678357 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc2100cd440 0x7fc2101518e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:13.187 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.182+0000 7fc20effd640 1 -- 172.21.15.195:0/3943678357 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2100caa80 con 0x7fc2100ccc40 2024-09-21T03:33:13.187 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.182+0000 7fc20effd640 1 --2- 172.21.15.195:0/3943678357 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc2100ccc40 0x7fc2101f5770 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fc20400ea50 tx=0x7fc20400ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:13.187 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.182+0000 7fc1f7fff640 1 -- 172.21.15.195:0/3943678357 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc20400cdc0 con 0x7fc2100ccc40 2024-09-21T03:33:13.187 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.182+0000 7fc1f7fff640 1 -- 172.21.15.195:0/3943678357 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc204004020 con 0x7fc2100ccc40 2024-09-21T03:33:13.187 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.182+0000 7fc1f7fff640 1 -- 172.21.15.195:0/3943678357 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc204004470 con 0x7fc2100ccc40 2024-09-21T03:33:13.187 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.182+0000 7fc215900640 1 -- 172.21.15.195:0/3943678357 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2100cad60 con 0x7fc2100ccc40 2024-09-21T03:33:13.187 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.182+0000 7fc215900640 1 -- 172.21.15.195:0/3943678357 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2100cb320 con 0x7fc2100ccc40 2024-09-21T03:33:13.188 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.182+0000 7fc1f7fff640 1 -- 172.21.15.195:0/3943678357 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fc204003980 con 0x7fc2100ccc40 2024-09-21T03:33:13.188 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.182+0000 7fc1f7fff640 1 -- 172.21.15.195:0/3943678357 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7fc204018ca0 con 0x7fc2100ccc40 2024-09-21T03:33:13.188 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.183+0000 7fc215900640 1 -- 172.21.15.195:0/3943678357 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc210070ed0 con 0x7fc2100ccc40 2024-09-21T03:33:13.188 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.185+0000 7fc1f7fff640 1 -- 172.21.15.195:0/3943678357 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fc204026070 con 0x7fc2100ccc40 2024-09-21T03:33:13.257 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.254+0000 7fc215900640 1 -- 172.21.15.195:0/3943678357 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "osd new", "uuid": "2ed3cd2d-385d-4fc6-9869-214552a84803", "id": 2} v 0) -- 0x7fc21015bea0 con 0x7fc2100ccc40 2024-09-21T03:33:13.265 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.262+0000 7fc1f7fff640 1 -- 172.21.15.195:0/3943678357 <== mon.1 v2:172.21.15.195:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "2ed3cd2d-385d-4fc6-9869-214552a84803", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7fc204009490 con 0x7fc2100ccc40 2024-09-21T03:33:13.265 INFO:teuthology.orchestra.run.smithi195.stdout:2 2024-09-21T03:33:13.266 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.263+0000 7fc215900640 1 -- 172.21.15.195:0/3943678357 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc2100ccc40 msgr2=0x7fc2101f5770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:13.266 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.263+0000 7fc215900640 1 --2- 172.21.15.195:0/3943678357 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc2100ccc40 0x7fc2101f5770 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fc20400ea50 tx=0x7fc20400ef10 comp rx=0 tx=0).stop 2024-09-21T03:33:13.267 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.263+0000 7fc215900640 1 -- 172.21.15.195:0/3943678357 shutdown_connections 2024-09-21T03:33:13.267 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.263+0000 7fc215900640 1 --2- 172.21.15.195:0/3943678357 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc2100d0bb0 0x7fc21006def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:13.267 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.263+0000 7fc215900640 1 --2- 172.21.15.195:0/3943678357 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc2100cd440 0x7fc2101518e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:13.267 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.263+0000 7fc215900640 1 --2- 172.21.15.195:0/3943678357 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc2100ccc40 0x7fc2101f5770 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:13.267 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.263+0000 7fc215900640 1 -- 172.21.15.195:0/3943678357 >> 172.21.15.195:0/3943678357 conn(0x7fc2100b46e0 msgr2=0x7fc210070100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:13.267 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.263+0000 7fc215900640 1 -- 172.21.15.195:0/3943678357 shutdown_connections 2024-09-21T03:33:13.267 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.264+0000 7fc215900640 1 -- 172.21.15.195:0/3943678357 wait complete. 2024-09-21T03:33:13.279 DEBUG:teuthology.orchestra.run.smithi195:> sudo ceph --cluster ceph osd new 62d1d95e-919d-4250-be71-1d34cfa70dbb 3 2024-09-21T03:33:13.391 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.388+0000 7f89d8551640 1 -- 172.21.15.195:0/3475410403 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f89d00d0d80 msgr2=0x7f89d00b5be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:13.391 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.388+0000 7f89d8551640 1 --2- 172.21.15.195:0/3475410403 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f89d00d0d80 0x7f89d00b5be0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f89c8005fd0 tx=0x7f89c802a820 comp rx=0 tx=0).stop 2024-09-21T03:33:13.392 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.388+0000 7f89d8551640 1 -- 172.21.15.195:0/3475410403 shutdown_connections 2024-09-21T03:33:13.392 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.388+0000 7f89d8551640 1 --2- 172.21.15.195:0/3475410403 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f89d00d0d80 0x7f89d00b5be0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:13.392 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.388+0000 7f89d8551640 1 --2- 172.21.15.195:0/3475410403 >> v2:172.21.15.195:3300/0 conn(0x7f89d00ccc40 0x7f89d015ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:13.392 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.388+0000 7f89d8551640 1 -- 172.21.15.195:0/3475410403 >> 172.21.15.195:0/3475410403 conn(0x7f89d00b46e0 msgr2=0x7f89d00d23e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:13.392 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.388+0000 7f89d8551640 1 -- 172.21.15.195:0/3475410403 shutdown_connections 2024-09-21T03:33:13.392 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.388+0000 7f89d8551640 1 -- 172.21.15.195:0/3475410403 wait complete. 2024-09-21T03:33:13.393 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.390+0000 7f89d8551640 1 Processor -- start 2024-09-21T03:33:13.393 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.390+0000 7f89d8551640 1 -- start start 2024-09-21T03:33:13.396 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.391+0000 7f89d8551640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f89d00ccc40 0x7f89d015e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:13.396 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.391+0000 7f89d8551640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f89d00cd440 0x7f89d00c7a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:13.396 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.391+0000 7f89d8551640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f89d015ca90 0x7f89d00c7fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:13.396 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.391+0000 7f89d8551640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f89d00b66a0 con 0x7f89d00cd440 2024-09-21T03:33:13.396 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.391+0000 7f89d8551640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f89d00b6520 con 0x7f89d00ccc40 2024-09-21T03:33:13.396 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.391+0000 7f89d8551640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f89d00b6820 con 0x7f89d015ca90 2024-09-21T03:33:13.397 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.391+0000 7f89d62ed640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f89d00ccc40 0x7f89d015e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:13.397 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.391+0000 7f89d6aee640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f89d015ca90 0x7f89d00c7fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:13.397 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.391+0000 7f89d62ed640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f89d00ccc40 0x7f89d015e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.195:35264/0 (socket says 172.21.15.195:35264) 2024-09-21T03:33:13.397 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.391+0000 7f89d62ed640 1 -- 172.21.15.195:0/2259086886 learned_addr learned my addr 172.21.15.195:0/2259086886 (peer_addr_for_me v2:172.21.15.195:0/0) 2024-09-21T03:33:13.397 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.392+0000 7f89d5aec640 1 --2- 172.21.15.195:0/2259086886 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f89d00cd440 0x7f89d00c7a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:13.397 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.392+0000 7f89d6aee640 1 -- 172.21.15.195:0/2259086886 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f89d00ccc40 msgr2=0x7f89d015e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:13.397 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.392+0000 7f89d6aee640 1 --2- 172.21.15.195:0/2259086886 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f89d00ccc40 0x7f89d015e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:13.397 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.392+0000 7f89d6aee640 1 -- 172.21.15.195:0/2259086886 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f89d00cd440 msgr2=0x7f89d00c7a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:13.397 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.392+0000 7f89d6aee640 1 --2- 172.21.15.195:0/2259086886 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f89d00cd440 0x7f89d00c7a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:13.397 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.392+0000 7f89d6aee640 1 -- 172.21.15.195:0/2259086886 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89d01f59f0 con 0x7f89d015ca90 2024-09-21T03:33:13.397 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.392+0000 7f89d62ed640 1 --2- 172.21.15.195:0/2259086886 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f89d00ccc40 0x7f89d015e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:33:13.397 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.392+0000 7f89d5aec640 1 --2- 172.21.15.195:0/2259086886 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f89d00cd440 0x7f89d00c7a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:33:13.397 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.392+0000 7f89d6aee640 1 --2- 172.21.15.195:0/2259086886 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f89d015ca90 0x7f89d00c7fd0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f89c8009000 tx=0x7f89c8032c90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:13.398 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.392+0000 7f89c6ffd640 1 -- 172.21.15.195:0/2259086886 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89c8037040 con 0x7f89d015ca90 2024-09-21T03:33:13.398 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.392+0000 7f89c6ffd640 1 -- 172.21.15.195:0/2259086886 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f89c8042070 con 0x7f89d015ca90 2024-09-21T03:33:13.398 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.392+0000 7f89d8551640 1 -- 172.21.15.195:0/2259086886 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89d01f5cd0 con 0x7f89d015ca90 2024-09-21T03:33:13.398 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.392+0000 7f89c6ffd640 1 -- 172.21.15.195:0/2259086886 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89c80042a0 con 0x7f89d015ca90 2024-09-21T03:33:13.398 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.392+0000 7f89d8551640 1 -- 172.21.15.195:0/2259086886 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f89d015cf00 con 0x7f89d015ca90 2024-09-21T03:33:13.398 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.392+0000 7f89c6ffd640 1 -- 172.21.15.195:0/2259086886 <== mon.2 v2:172.21.15.195:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f89c8004440 con 0x7f89d015ca90 2024-09-21T03:33:13.398 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.392+0000 7f89c6ffd640 1 -- 172.21.15.195:0/2259086886 <== mon.2 v2:172.21.15.195:3301/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7f89c8047d50 con 0x7f89d015ca90 2024-09-21T03:33:13.398 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.393+0000 7f89d8551640 1 -- 172.21.15.195:0/2259086886 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89d00b6d40 con 0x7f89d015ca90 2024-09-21T03:33:13.398 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.395+0000 7f89c6ffd640 1 -- 172.21.15.195:0/2259086886 <== mon.2 v2:172.21.15.195:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f89c8047460 con 0x7f89d015ca90 2024-09-21T03:33:13.467 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.464+0000 7f89d8551640 1 -- 172.21.15.195:0/2259086886 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_command({"prefix": "osd new", "uuid": "62d1d95e-919d-4250-be71-1d34cfa70dbb", "id": 3} v 0) -- 0x7f89d00d28d0 con 0x7f89d015ca90 2024-09-21T03:33:13.470 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.467+0000 7f89c6ffd640 1 -- 172.21.15.195:0/2259086886 <== mon.2 v2:172.21.15.195:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "62d1d95e-919d-4250-be71-1d34cfa70dbb", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7f89c80478c0 con 0x7f89d015ca90 2024-09-21T03:33:13.470 INFO:teuthology.orchestra.run.smithi195.stdout:3 2024-09-21T03:33:13.471 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.468+0000 7f89d8551640 1 -- 172.21.15.195:0/2259086886 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f89d015ca90 msgr2=0x7f89d00c7fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:13.471 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.468+0000 7f89d8551640 1 --2- 172.21.15.195:0/2259086886 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f89d015ca90 0x7f89d00c7fd0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f89c8009000 tx=0x7f89c8032c90 comp rx=0 tx=0).stop 2024-09-21T03:33:13.472 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.468+0000 7f89d8551640 1 -- 172.21.15.195:0/2259086886 shutdown_connections 2024-09-21T03:33:13.472 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.468+0000 7f89d8551640 1 --2- 172.21.15.195:0/2259086886 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f89d015ca90 0x7f89d00c7fd0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:13.472 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.468+0000 7f89d8551640 1 --2- 172.21.15.195:0/2259086886 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f89d00cd440 0x7f89d00c7a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:13.472 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.468+0000 7f89d8551640 1 --2- 172.21.15.195:0/2259086886 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f89d00ccc40 0x7f89d015e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:13.472 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.468+0000 7f89d8551640 1 -- 172.21.15.195:0/2259086886 >> 172.21.15.195:0/2259086886 conn(0x7f89d00b46e0 msgr2=0x7f89d00b6f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:13.472 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.468+0000 7f89d8551640 1 -- 172.21.15.195:0/2259086886 shutdown_connections 2024-09-21T03:33:13.472 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.468+0000 7f89d8551640 1 -- 172.21.15.195:0/2259086886 wait complete. 2024-09-21T03:33:13.482 DEBUG:teuthology.orchestra.run.smithi195:> sudo ceph --cluster ceph osd new 7c9f9314-99d7-4d73-8392-37809a9f54b5 4 2024-09-21T03:33:13.595 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.592+0000 7f9017a39640 1 -- 172.21.15.195:0/769635358 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f90100ccc40 msgr2=0x7f901015ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:13.595 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.592+0000 7f9017a39640 1 --2- 172.21.15.195:0/769635358 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f90100ccc40 0x7f901015ca90 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f9000005fd0 tx=0x7f900002a540 comp rx=0 tx=0).stop 2024-09-21T03:33:13.595 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.592+0000 7f9017a39640 1 -- 172.21.15.195:0/769635358 shutdown_connections 2024-09-21T03:33:13.596 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.592+0000 7f9017a39640 1 --2- 172.21.15.195:0/769635358 >> v2:172.21.15.38:3300/0 conn(0x7f90100cd440 0x7f90100d0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:13.596 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.592+0000 7f9017a39640 1 --2- 172.21.15.195:0/769635358 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f90100ccc40 0x7f901015ca90 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:13.596 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.592+0000 7f9017a39640 1 -- 172.21.15.195:0/769635358 >> 172.21.15.195:0/769635358 conn(0x7f90100b46e0 msgr2=0x7f90100d23e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:13.596 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.592+0000 7f9017a39640 1 -- 172.21.15.195:0/769635358 shutdown_connections 2024-09-21T03:33:13.596 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.592+0000 7f9017a39640 1 -- 172.21.15.195:0/769635358 wait complete. 2024-09-21T03:33:13.596 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.594+0000 7f9017a39640 1 Processor -- start 2024-09-21T03:33:13.597 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.594+0000 7f9017a39640 1 -- start start 2024-09-21T03:33:13.600 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.595+0000 7f9017a39640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f90100ccc40 0x7f9010157940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:13.600 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.595+0000 7f9017a39640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f90100cd440 0x7f90101581a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:13.600 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.595+0000 7f9017a39640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f90100d0bb0 0x7f90101586e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:13.600 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.595+0000 7f9017a39640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f9010070810 con 0x7f90100cd440 2024-09-21T03:33:13.600 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.595+0000 7f9017a39640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f90100b6e00 con 0x7f90100d0bb0 2024-09-21T03:33:13.600 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.595+0000 7f9017a39640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f9010070990 con 0x7f90100ccc40 2024-09-21T03:33:13.600 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.595+0000 7f90157d5640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f90100ccc40 0x7f9010157940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:13.600 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.595+0000 7f90157d5640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f90100ccc40 0x7f9010157940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.195:56766/0 (socket says 172.21.15.195:56766) 2024-09-21T03:33:13.600 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.595+0000 7f90157d5640 1 -- 172.21.15.195:0/4029222923 learned_addr learned my addr 172.21.15.195:0/4029222923 (peer_addr_for_me v2:172.21.15.195:0/0) 2024-09-21T03:33:13.600 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.595+0000 7f9015fd6640 1 --2- 172.21.15.195:0/4029222923 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f90100d0bb0 0x7f90101586e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:13.600 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.595+0000 7f9014fd4640 1 --2- 172.21.15.195:0/4029222923 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f90100cd440 0x7f90101581a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:13.601 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.596+0000 7f90157d5640 1 -- 172.21.15.195:0/4029222923 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f90100d0bb0 msgr2=0x7f90101586e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:13.601 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.596+0000 7f90157d5640 1 --2- 172.21.15.195:0/4029222923 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f90100d0bb0 0x7f90101586e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:13.601 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.596+0000 7f90157d5640 1 -- 172.21.15.195:0/4029222923 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f90100cd440 msgr2=0x7f90101581a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:13.601 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.596+0000 7f90157d5640 1 --2- 172.21.15.195:0/4029222923 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f90100cd440 0x7f90101581a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:13.601 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.596+0000 7f90157d5640 1 -- 172.21.15.195:0/4029222923 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90100cbbe0 con 0x7f90100ccc40 2024-09-21T03:33:13.601 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.596+0000 7f90157d5640 1 --2- 172.21.15.195:0/4029222923 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f90100ccc40 0x7f9010157940 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f9000034040 tx=0x7f90000038a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:13.601 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.596+0000 7f9005ffb640 1 -- 172.21.15.195:0/4029222923 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f900003e030 con 0x7f90100ccc40 2024-09-21T03:33:13.602 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.596+0000 7f9005ffb640 1 -- 172.21.15.195:0/4029222923 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9000003db0 con 0x7f90100ccc40 2024-09-21T03:33:13.602 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.596+0000 7f9005ffb640 1 -- 172.21.15.195:0/4029222923 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9000035040 con 0x7f90100ccc40 2024-09-21T03:33:13.602 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.596+0000 7f9017a39640 1 -- 172.21.15.195:0/4029222923 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90100c9ea0 con 0x7f90100ccc40 2024-09-21T03:33:13.602 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.596+0000 7f9017a39640 1 -- 172.21.15.195:0/4029222923 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f90100ca460 con 0x7f90100ccc40 2024-09-21T03:33:13.602 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.596+0000 7f9005ffb640 1 -- 172.21.15.195:0/4029222923 <== mon.2 v2:172.21.15.195:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f9000041030 con 0x7f90100ccc40 2024-09-21T03:33:13.602 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.596+0000 7f9005ffb640 1 -- 172.21.15.195:0/4029222923 <== mon.2 v2:172.21.15.195:3301/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7f9000043d30 con 0x7f90100ccc40 2024-09-21T03:33:13.602 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.597+0000 7f9017a39640 1 -- 172.21.15.195:0/4029222923 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9010070ed0 con 0x7f90100ccc40 2024-09-21T03:33:13.602 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.599+0000 7f9005ffb640 1 -- 172.21.15.195:0/4029222923 <== mon.2 v2:172.21.15.195:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f9000043410 con 0x7f90100ccc40 2024-09-21T03:33:13.670 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.667+0000 7f9017a39640 1 -- 172.21.15.195:0/4029222923 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_command({"prefix": "osd new", "uuid": "7c9f9314-99d7-4d73-8392-37809a9f54b5", "id": 4} v 0) -- 0x7f901015c2f0 con 0x7f90100ccc40 2024-09-21T03:33:13.678 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.675+0000 7f9005ffb640 1 -- 172.21.15.195:0/4029222923 <== mon.2 v2:172.21.15.195:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "7c9f9314-99d7-4d73-8392-37809a9f54b5", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7f9000004f20 con 0x7f90100ccc40 2024-09-21T03:33:13.678 INFO:teuthology.orchestra.run.smithi195.stdout:4 2024-09-21T03:33:13.679 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.676+0000 7f9017a39640 1 -- 172.21.15.195:0/4029222923 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f90100ccc40 msgr2=0x7f9010157940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:13.680 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.676+0000 7f9017a39640 1 --2- 172.21.15.195:0/4029222923 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f90100ccc40 0x7f9010157940 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f9000034040 tx=0x7f90000038a0 comp rx=0 tx=0).stop 2024-09-21T03:33:13.680 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.676+0000 7f9017a39640 1 -- 172.21.15.195:0/4029222923 shutdown_connections 2024-09-21T03:33:13.680 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.676+0000 7f9017a39640 1 --2- 172.21.15.195:0/4029222923 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f90100d0bb0 0x7f90101586e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:13.680 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.676+0000 7f9017a39640 1 --2- 172.21.15.195:0/4029222923 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f90100cd440 0x7f90101581a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:13.680 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.676+0000 7f9017a39640 1 --2- 172.21.15.195:0/4029222923 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f90100ccc40 0x7f9010157940 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:13.680 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.676+0000 7f9017a39640 1 -- 172.21.15.195:0/4029222923 >> 172.21.15.195:0/4029222923 conn(0x7f90100b46e0 msgr2=0x7f90100b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:13.680 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.676+0000 7f9017a39640 1 -- 172.21.15.195:0/4029222923 shutdown_connections 2024-09-21T03:33:13.680 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.677+0000 7f9017a39640 1 -- 172.21.15.195:0/4029222923 wait complete. 2024-09-21T03:33:13.690 DEBUG:teuthology.orchestra.run.smithi195:> sudo ceph --cluster ceph osd new bde4fe9f-8414-4bba-bf4a-43973d1976bf 5 2024-09-21T03:33:13.801 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.798+0000 7f6241e64640 1 -- 172.21.15.195:0/2632333014 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f623c0cd440 msgr2=0x7f623c0d0670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:13.801 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.798+0000 7f6241e64640 1 --2- 172.21.15.195:0/2632333014 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f623c0cd440 0x7f623c0d0670 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f622c005fd0 tx=0x7f622c02a540 comp rx=0 tx=0).stop 2024-09-21T03:33:13.802 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.798+0000 7f6241e64640 1 -- 172.21.15.195:0/2632333014 shutdown_connections 2024-09-21T03:33:13.802 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.798+0000 7f6241e64640 1 --2- 172.21.15.195:0/2632333014 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f623c0cd440 0x7f623c0d0670 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:13.802 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.798+0000 7f6241e64640 1 --2- 172.21.15.195:0/2632333014 >> v2:172.21.15.38:3300/0 conn(0x7f623c0ccc40 0x7f623c15ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:13.802 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.798+0000 7f6241e64640 1 -- 172.21.15.195:0/2632333014 >> 172.21.15.195:0/2632333014 conn(0x7f623c0b46e0 msgr2=0x7f623c0d23e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:13.802 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.798+0000 7f6241e64640 1 -- 172.21.15.195:0/2632333014 shutdown_connections 2024-09-21T03:33:13.803 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.798+0000 7f6241e64640 1 -- 172.21.15.195:0/2632333014 wait complete. 2024-09-21T03:33:13.803 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.800+0000 7f6241e64640 1 Processor -- start 2024-09-21T03:33:13.803 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.800+0000 7f6241e64640 1 -- start start 2024-09-21T03:33:13.807 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.801+0000 7f6241e64640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f623c0ccc40 0x7f623c157820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:13.808 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.801+0000 7f6241e64640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f623c0cd440 0x7f623c15ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:13.808 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.801+0000 7f6241e64640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f623c0d0bb0 0x7f623c0b6f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:13.808 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.801+0000 7f6241e64640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f623c070810 con 0x7f623c0cd440 2024-09-21T03:33:13.808 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.801+0000 7f6241e64640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f623c0b6e00 con 0x7f623c0ccc40 2024-09-21T03:33:13.809 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.801+0000 7f6241e64640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f623c070990 con 0x7f623c0d0bb0 2024-09-21T03:33:13.809 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.801+0000 7f623b7fe640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f623c0ccc40 0x7f623c157820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:13.809 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.801+0000 7f623bfff640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f623c0d0bb0 0x7f623c0b6f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:13.809 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.801+0000 7f623b7fe640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f623c0ccc40 0x7f623c157820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.195:35268/0 (socket says 172.21.15.195:35268) 2024-09-21T03:33:13.809 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.801+0000 7f623b7fe640 1 -- 172.21.15.195:0/1839178803 learned_addr learned my addr 172.21.15.195:0/1839178803 (peer_addr_for_me v2:172.21.15.195:0/0) 2024-09-21T03:33:13.809 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.801+0000 7f623affd640 1 --2- 172.21.15.195:0/1839178803 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f623c0cd440 0x7f623c15ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:13.810 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.801+0000 7f623b7fe640 1 -- 172.21.15.195:0/1839178803 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f623c0d0bb0 msgr2=0x7f623c0b6f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:13.810 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.801+0000 7f623b7fe640 1 --2- 172.21.15.195:0/1839178803 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f623c0d0bb0 0x7f623c0b6f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:13.810 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.801+0000 7f623b7fe640 1 -- 172.21.15.195:0/1839178803 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f623c0cd440 msgr2=0x7f623c15ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:13.810 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.801+0000 7f623b7fe640 1 --2- 172.21.15.195:0/1839178803 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f623c0cd440 0x7f623c15ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:13.810 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.801+0000 7f623b7fe640 1 -- 172.21.15.195:0/1839178803 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f623c1f3780 con 0x7f623c0ccc40 2024-09-21T03:33:13.811 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.802+0000 7f623b7fe640 1 --2- 172.21.15.195:0/1839178803 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f623c0ccc40 0x7f623c157820 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f623000e790 tx=0x7f623000ec50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:13.811 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.802+0000 7f6223fff640 1 -- 172.21.15.195:0/1839178803 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f623000eea0 con 0x7f623c0ccc40 2024-09-21T03:33:13.811 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.802+0000 7f6223fff640 1 -- 172.21.15.195:0/1839178803 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f623000cdb0 con 0x7f623c0ccc40 2024-09-21T03:33:13.811 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.802+0000 7f6223fff640 1 -- 172.21.15.195:0/1839178803 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6230010690 con 0x7f623c0ccc40 2024-09-21T03:33:13.811 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.802+0000 7f6241e64640 1 -- 172.21.15.195:0/1839178803 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f623c1f3a60 con 0x7f623c0ccc40 2024-09-21T03:33:13.812 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.802+0000 7f6241e64640 1 -- 172.21.15.195:0/1839178803 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f623c1f4020 con 0x7f623c0ccc40 2024-09-21T03:33:13.812 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.802+0000 7f6223fff640 1 -- 172.21.15.195:0/1839178803 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f6230010830 con 0x7f623c0ccc40 2024-09-21T03:33:13.812 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.802+0000 7f6223fff640 1 -- 172.21.15.195:0/1839178803 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7f6230018c40 con 0x7f623c0ccc40 2024-09-21T03:33:13.812 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.803+0000 7f6241e64640 1 -- 172.21.15.195:0/1839178803 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f623c070ed0 con 0x7f623c0ccc40 2024-09-21T03:33:13.813 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.805+0000 7f6223fff640 1 -- 172.21.15.195:0/1839178803 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f6230010d00 con 0x7f623c0ccc40 2024-09-21T03:33:13.877 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.874+0000 7f6241e64640 1 -- 172.21.15.195:0/1839178803 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "osd new", "uuid": "bde4fe9f-8414-4bba-bf4a-43973d1976bf", "id": 5} v 0) -- 0x7f623c0c8be0 con 0x7f623c0ccc40 2024-09-21T03:33:13.885 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.882+0000 7f6223fff640 1 -- 172.21.15.195:0/1839178803 <== mon.1 v2:172.21.15.195:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "bde4fe9f-8414-4bba-bf4a-43973d1976bf", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7f6230018420 con 0x7f623c0ccc40 2024-09-21T03:33:13.886 INFO:teuthology.orchestra.run.smithi195.stdout:5 2024-09-21T03:33:13.887 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.883+0000 7f6241e64640 1 -- 172.21.15.195:0/1839178803 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f623c0ccc40 msgr2=0x7f623c157820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:13.887 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.883+0000 7f6241e64640 1 --2- 172.21.15.195:0/1839178803 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f623c0ccc40 0x7f623c157820 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f623000e790 tx=0x7f623000ec50 comp rx=0 tx=0).stop 2024-09-21T03:33:13.888 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.883+0000 7f6241e64640 1 -- 172.21.15.195:0/1839178803 shutdown_connections 2024-09-21T03:33:13.888 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.883+0000 7f6241e64640 1 --2- 172.21.15.195:0/1839178803 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f623c0d0bb0 0x7f623c0b6f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:13.888 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.883+0000 7f6241e64640 1 --2- 172.21.15.195:0/1839178803 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f623c0cd440 0x7f623c15ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:13.888 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.883+0000 7f6241e64640 1 --2- 172.21.15.195:0/1839178803 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f623c0ccc40 0x7f623c157820 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:13.888 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.883+0000 7f6241e64640 1 -- 172.21.15.195:0/1839178803 >> 172.21.15.195:0/1839178803 conn(0x7f623c0b46e0 msgr2=0x7f623c070100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:13.888 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.883+0000 7f6241e64640 1 -- 172.21.15.195:0/1839178803 shutdown_connections 2024-09-21T03:33:13.888 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:13.883+0000 7f6241e64640 1 -- 172.21.15.195:0/1839178803 wait complete. 2024-09-21T03:33:13.897 DEBUG:teuthology.orchestra.run.smithi195:> sudo ceph --cluster ceph osd new 6ee6d7f4-1a5f-4a4c-ac16-48fe05f6a002 6 2024-09-21T03:33:14.011 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:14.008+0000 7fe45f27e640 1 -- 172.21.15.195:0/1883485478 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe4580b5260 msgr2=0x7fe45815ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:14.011 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:14.008+0000 7fe45f27e640 1 --2- 172.21.15.195:0/1883485478 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe4580b5260 0x7fe45815ba80 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fe458075150 tx=0x7fe45402a5f0 comp rx=0 tx=0).stop 2024-09-21T03:33:14.011 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:14.008+0000 7fe45f27e640 1 -- 172.21.15.195:0/1883485478 shutdown_connections 2024-09-21T03:33:14.012 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:14.008+0000 7fe45f27e640 1 --2- 172.21.15.195:0/1883485478 >> v2:172.21.15.38:3300/0 conn(0x7fe45815bfc0 0x7fe45815f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:14.012 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:14.008+0000 7fe45f27e640 1 --2- 172.21.15.195:0/1883485478 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe4580b5260 0x7fe45815ba80 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:14.012 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:14.008+0000 7fe45f27e640 1 -- 172.21.15.195:0/1883485478 >> 172.21.15.195:0/1883485478 conn(0x7fe4580cdca0 msgr2=0x7fe4580ce0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:14.012 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:14.008+0000 7fe45f27e640 1 -- 172.21.15.195:0/1883485478 shutdown_connections 2024-09-21T03:33:14.012 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:14.008+0000 7fe45f27e640 1 -- 172.21.15.195:0/1883485478 wait complete. 2024-09-21T03:33:14.012 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:14.010+0000 7fe45f27e640 1 Processor -- start 2024-09-21T03:33:14.013 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:14.010+0000 7fe45f27e640 1 -- start start 2024-09-21T03:33:14.016 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:14.011+0000 7fe45f27e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe45806def0 0x7fe458157840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:14.016 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:14.011+0000 7fe45f27e640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe4580b5260 0x7fe4580b29f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:14.016 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:14.011+0000 7fe45f27e640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe45815bfc0 0x7fe45814f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:14.016 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:14.011+0000 7fe45f27e640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fe45815ff40 con 0x7fe45806def0 2024-09-21T03:33:14.016 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:14.011+0000 7fe45f27e640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fe45815fdc0 con 0x7fe4580b5260 2024-09-21T03:33:14.016 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:14.011+0000 7fe45f27e640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fe458071300 con 0x7fe45815bfc0 2024-09-21T03:33:14.016 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:14.011+0000 7fe45c819640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe4580b5260 0x7fe4580b29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:14.017 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:14.011+0000 7fe45d81b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe45815bfc0 0x7fe45814f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:14.017 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:14.011+0000 7fe45c819640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe4580b5260 0x7fe4580b29f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.195:35300/0 (socket says 172.21.15.195:35300) 2024-09-21T03:33:14.017 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:14.011+0000 7fe45c819640 1 -- 172.21.15.195:0/2549149434 learned_addr learned my addr 172.21.15.195:0/2549149434 (peer_addr_for_me v2:172.21.15.195:0/0) 2024-09-21T03:33:14.017 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:14.011+0000 7fe45d01a640 1 --2- 172.21.15.195:0/2549149434 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe45806def0 0x7fe458157840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:14.017 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:14.011+0000 7fe45c819640 1 -- 172.21.15.195:0/2549149434 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe45815bfc0 msgr2=0x7fe45814f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:14.017 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:14.011+0000 7fe45c819640 1 --2- 172.21.15.195:0/2549149434 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe45815bfc0 0x7fe45814f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:14.017 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:14.011+0000 7fe45c819640 1 -- 172.21.15.195:0/2549149434 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe45806def0 msgr2=0x7fe458157840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:14.017 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:14.011+0000 7fe45c819640 1 --2- 172.21.15.195:0/2549149434 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe45806def0 0x7fe458157840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:14.017 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:14.011+0000 7fe45c819640 1 -- 172.21.15.195:0/2549149434 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe45814fc50 con 0x7fe4580b5260 2024-09-21T03:33:14.017 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:14.011+0000 7fe45d81b640 1 --2- 172.21.15.195:0/2549149434 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe45815bfc0 0x7fe45814f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:33:14.018 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:14.012+0000 7fe45c819640 1 --2- 172.21.15.195:0/2549149434 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe4580b5260 0x7fe4580b29f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe454034040 tx=0x7fe4540039e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:14.018 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:14.012+0000 7fe44dffb640 1 -- 172.21.15.195:0/2549149434 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe45403e030 con 0x7fe4580b5260 2024-09-21T03:33:14.018 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:14.012+0000 7fe44dffb640 1 -- 172.21.15.195:0/2549149434 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe4540047a0 con 0x7fe4580b5260 2024-09-21T03:33:14.018 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:14.012+0000 7fe45f27e640 1 -- 172.21.15.195:0/2549149434 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4581f8e50 con 0x7fe4580b5260 2024-09-21T03:33:14.018 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:14.012+0000 7fe44dffb640 1 -- 172.21.15.195:0/2549149434 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe454035040 con 0x7fe4580b5260 2024-09-21T03:33:14.018 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:14.012+0000 7fe45f27e640 1 -- 172.21.15.195:0/2549149434 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4581f9410 con 0x7fe4580b5260 2024-09-21T03:33:14.018 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:14.012+0000 7fe44dffb640 1 -- 172.21.15.195:0/2549149434 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fe454041030 con 0x7fe4580b5260 2024-09-21T03:33:14.018 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:14.012+0000 7fe44dffb640 1 -- 172.21.15.195:0/2549149434 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7fe454031400 con 0x7fe4580b5260 2024-09-21T03:33:14.018 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:14.013+0000 7fe45f27e640 1 -- 172.21.15.195:0/2549149434 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe45815ba80 con 0x7fe4580b5260 2024-09-21T03:33:14.019 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:14.015+0000 7fe44dffb640 1 -- 172.21.15.195:0/2549149434 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fe45814fc50 con 0x7fe4580b5260 2024-09-21T03:33:14.087 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:14.084+0000 7fe45f27e640 1 -- 172.21.15.195:0/2549149434 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "osd new", "uuid": "6ee6d7f4-1a5f-4a4c-ac16-48fe05f6a002", "id": 6} v 0) -- 0x7fe45815f720 con 0x7fe4580b5260 2024-09-21T03:33:14.095 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:14.092+0000 7fe44dffb640 1 -- 172.21.15.195:0/2549149434 <== mon.1 v2:172.21.15.195:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "6ee6d7f4-1a5f-4a4c-ac16-48fe05f6a002", "id": 6}]=0 v9) ==== 112+0+2 (secure 0 0 0) 0x7fe454043410 con 0x7fe4580b5260 2024-09-21T03:33:14.095 INFO:teuthology.orchestra.run.smithi195.stdout:6 2024-09-21T03:33:14.096 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:14.093+0000 7fe45f27e640 1 -- 172.21.15.195:0/2549149434 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe4580b5260 msgr2=0x7fe4580b29f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:14.097 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:14.093+0000 7fe45f27e640 1 --2- 172.21.15.195:0/2549149434 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe4580b5260 0x7fe4580b29f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe454034040 tx=0x7fe4540039e0 comp rx=0 tx=0).stop 2024-09-21T03:33:14.097 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:14.093+0000 7fe45f27e640 1 -- 172.21.15.195:0/2549149434 shutdown_connections 2024-09-21T03:33:14.097 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:14.093+0000 7fe45f27e640 1 --2- 172.21.15.195:0/2549149434 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe45815bfc0 0x7fe45814f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:14.097 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:14.093+0000 7fe45f27e640 1 --2- 172.21.15.195:0/2549149434 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe4580b5260 0x7fe4580b29f0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:14.097 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:14.093+0000 7fe45f27e640 1 --2- 172.21.15.195:0/2549149434 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe45806def0 0x7fe458157840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:14.097 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:14.093+0000 7fe45f27e640 1 -- 172.21.15.195:0/2549149434 >> 172.21.15.195:0/2549149434 conn(0x7fe4580cdca0 msgr2=0x7fe458068d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:14.097 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:14.094+0000 7fe45f27e640 1 -- 172.21.15.195:0/2549149434 shutdown_connections 2024-09-21T03:33:14.097 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:14.094+0000 7fe45f27e640 1 -- 172.21.15.195:0/2549149434 wait complete. 2024-09-21T03:33:14.107 DEBUG:teuthology.orchestra.run.smithi195:> sudo ceph --cluster ceph osd new efeb5f51-329a-4484-8290-92e7d3503d05 7 2024-09-21T03:33:14.220 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:14.217+0000 7f6ab9d90640 1 -- 172.21.15.195:0/3712668502 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6ab406def0 msgr2=0x7f6ab406f410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:14.220 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:14.217+0000 7f6ab9d90640 1 --2- 172.21.15.195:0/3712668502 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6ab406def0 0x7f6ab406f410 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f6aa4005fd0 tx=0x7f6aa402a5d0 comp rx=0 tx=0).stop 2024-09-21T03:33:14.221 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:14.217+0000 7f6ab9d90640 1 -- 172.21.15.195:0/3712668502 shutdown_connections 2024-09-21T03:33:14.221 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:14.217+0000 7f6ab9d90640 1 --2- 172.21.15.195:0/3712668502 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6ab40b5260 0x7f6ab415ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:14.221 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:14.217+0000 7f6ab9d90640 1 --2- 172.21.15.195:0/3712668502 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6ab406def0 0x7f6ab406f410 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:14.221 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:14.217+0000 7f6ab9d90640 1 -- 172.21.15.195:0/3712668502 >> 172.21.15.195:0/3712668502 conn(0x7f6ab40cdca0 msgr2=0x7f6ab40ce0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:14.221 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:14.217+0000 7f6ab9d90640 1 -- 172.21.15.195:0/3712668502 shutdown_connections 2024-09-21T03:33:14.221 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:14.217+0000 7f6ab9d90640 1 -- 172.21.15.195:0/3712668502 wait complete. 2024-09-21T03:33:14.226 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:14.219+0000 7f6ab9d90640 1 Processor -- start 2024-09-21T03:33:14.226 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:14.219+0000 7f6ab9d90640 1 -- start start 2024-09-21T03:33:14.226 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:14.220+0000 7f6ab9d90640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6ab406def0 0x7f6ab406cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:14.226 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:14.220+0000 7f6ab9d90640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6ab40b5260 0x7f6ab406d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:14.226 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:14.220+0000 7f6ab9d90640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6ab415bfc0 0x7f6ab4149c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:14.227 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:14.220+0000 7f6ab9d90640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f6ab40b5a00 con 0x7f6ab415bfc0 2024-09-21T03:33:14.227 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:14.220+0000 7f6ab9d90640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f6ab40b5880 con 0x7f6ab40b5260 2024-09-21T03:33:14.227 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:14.220+0000 7f6ab9d90640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f6ab4071300 con 0x7f6ab406def0 2024-09-21T03:33:14.227 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:14.220+0000 7f6aabfff640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6ab40b5260 0x7f6ab406d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:14.227 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:14.220+0000 7f6ab37fe640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6ab406def0 0x7f6ab406cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:14.227 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:14.220+0000 7f6ab37fe640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6ab406def0 0x7f6ab406cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.195:56816/0 (socket says 172.21.15.195:56816) 2024-09-21T03:33:14.227 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:14.220+0000 7f6aabfff640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6ab40b5260 0x7f6ab406d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.195:35322/0 (socket says 172.21.15.195:35322) 2024-09-21T03:33:14.227 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:14.220+0000 7f6ab37fe640 1 -- 172.21.15.195:0/1362879197 learned_addr learned my addr 172.21.15.195:0/1362879197 (peer_addr_for_me v2:172.21.15.195:0/0) 2024-09-21T03:33:14.227 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:14.220+0000 7f6ab3fff640 1 --2- 172.21.15.195:0/1362879197 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6ab415bfc0 0x7f6ab4149c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:14.227 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:14.220+0000 7f6aabfff640 1 -- 172.21.15.195:0/1362879197 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6ab406def0 msgr2=0x7f6ab406cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:14.228 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:14.220+0000 7f6aabfff640 1 --2- 172.21.15.195:0/1362879197 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6ab406def0 0x7f6ab406cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:14.228 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:14.220+0000 7f6aabfff640 1 -- 172.21.15.195:0/1362879197 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6ab415bfc0 msgr2=0x7f6ab4149c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:14.228 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:14.220+0000 7f6aabfff640 1 --2- 172.21.15.195:0/1362879197 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6ab415bfc0 0x7f6ab4149c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:14.228 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:14.220+0000 7f6aabfff640 1 -- 172.21.15.195:0/1362879197 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ab415d340 con 0x7f6ab40b5260 2024-09-21T03:33:14.228 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:14.220+0000 7f6ab37fe640 1 --2- 172.21.15.195:0/1362879197 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6ab406def0 0x7f6ab406cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:33:14.228 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:14.220+0000 7f6aabfff640 1 --2- 172.21.15.195:0/1362879197 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6ab40b5260 0x7f6ab406d320 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f6a9c00ea50 tx=0x7f6a9c00ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:14.228 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:14.221+0000 7f6ab0ff9640 1 -- 172.21.15.195:0/1362879197 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a9c00cd00 con 0x7f6ab40b5260 2024-09-21T03:33:14.228 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:14.221+0000 7f6ab0ff9640 1 -- 172.21.15.195:0/1362879197 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6a9c004020 con 0x7f6ab40b5260 2024-09-21T03:33:14.228 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:14.221+0000 7f6ab9d90640 1 -- 172.21.15.195:0/1362879197 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ab415d620 con 0x7f6ab40b5260 2024-09-21T03:33:14.228 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:14.221+0000 7f6ab0ff9640 1 -- 172.21.15.195:0/1362879197 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a9c00cea0 con 0x7f6ab40b5260 2024-09-21T03:33:14.228 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:14.221+0000 7f6ab9d90640 1 -- 172.21.15.195:0/1362879197 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ab4149330 con 0x7f6ab40b5260 2024-09-21T03:33:14.228 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:14.221+0000 7f6ab0ff9640 1 -- 172.21.15.195:0/1362879197 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f6a9c003980 con 0x7f6ab40b5260 2024-09-21T03:33:14.228 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:14.221+0000 7f6ab0ff9640 1 -- 172.21.15.195:0/1362879197 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1530+0+0 (secure 0 0 0) 0x7f6a9c018a90 con 0x7f6ab40b5260 2024-09-21T03:33:14.229 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:14.222+0000 7f6ab9d90640 1 -- 172.21.15.195:0/1362879197 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ab406f410 con 0x7f6ab40b5260 2024-09-21T03:33:14.229 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:14.224+0000 7f6ab0ff9640 1 -- 172.21.15.195:0/1362879197 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f6a9c038860 con 0x7f6ab40b5260 2024-09-21T03:33:14.295 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:14.293+0000 7f6ab9d90640 1 -- 172.21.15.195:0/1362879197 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "osd new", "uuid": "efeb5f51-329a-4484-8290-92e7d3503d05", "id": 7} v 0) -- 0x7f6ab415eb40 con 0x7f6ab40b5260 2024-09-21T03:33:14.299 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:14.297+0000 7f6ab0ff9640 1 -- 172.21.15.195:0/1362879197 <== mon.1 v2:172.21.15.195:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "efeb5f51-329a-4484-8290-92e7d3503d05", "id": 7}]=0 v10) ==== 112+0+2 (secure 0 0 0) 0x7f6a9c026020 con 0x7f6ab40b5260 2024-09-21T03:33:14.300 INFO:teuthology.orchestra.run.smithi195.stdout:7 2024-09-21T03:33:14.301 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:14.298+0000 7f6ab9d90640 1 -- 172.21.15.195:0/1362879197 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6ab40b5260 msgr2=0x7f6ab406d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:14.301 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:14.298+0000 7f6ab9d90640 1 --2- 172.21.15.195:0/1362879197 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6ab40b5260 0x7f6ab406d320 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f6a9c00ea50 tx=0x7f6a9c00ef10 comp rx=0 tx=0).stop 2024-09-21T03:33:14.301 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:14.298+0000 7f6ab9d90640 1 -- 172.21.15.195:0/1362879197 shutdown_connections 2024-09-21T03:33:14.301 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:14.298+0000 7f6ab9d90640 1 --2- 172.21.15.195:0/1362879197 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6ab415bfc0 0x7f6ab4149c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:14.301 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:14.298+0000 7f6ab9d90640 1 --2- 172.21.15.195:0/1362879197 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6ab40b5260 0x7f6ab406d320 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:14.301 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:14.298+0000 7f6ab9d90640 1 --2- 172.21.15.195:0/1362879197 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6ab406def0 0x7f6ab406cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:14.301 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:14.298+0000 7f6ab9d90640 1 -- 172.21.15.195:0/1362879197 >> 172.21.15.195:0/1362879197 conn(0x7f6ab40cdca0 msgr2=0x7f6ab415df80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:14.301 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:14.298+0000 7f6ab9d90640 1 -- 172.21.15.195:0/1362879197 shutdown_connections 2024-09-21T03:33:14.301 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:33:14.298+0000 7f6ab9d90640 1 -- 172.21.15.195:0/1362879197 wait complete. 2024-09-21T03:33:14.311 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-21T03:33:14.311 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-09-21T03:33:14.314 INFO:tasks.ceph.osd.0:Started 2024-09-21T03:33:14.314 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-21T03:33:14.314 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-21T03:33:14.316 INFO:tasks.ceph.osd.1:Started 2024-09-21T03:33:14.316 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-21T03:33:14.316 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-21T03:33:14.318 INFO:tasks.ceph.osd.2:Started 2024-09-21T03:33:14.319 INFO:tasks.ceph.osd.3:Restarting daemon 2024-09-21T03:33:14.319 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-09-21T03:33:14.321 INFO:tasks.ceph.osd.3:Started 2024-09-21T03:33:14.321 INFO:tasks.ceph.osd.4:Restarting daemon 2024-09-21T03:33:14.321 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-09-21T03:33:14.323 INFO:tasks.ceph.osd.4:Started 2024-09-21T03:33:14.324 INFO:tasks.ceph.osd.5:Restarting daemon 2024-09-21T03:33:14.324 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-09-21T03:33:14.326 INFO:tasks.ceph.osd.5:Started 2024-09-21T03:33:14.326 INFO:tasks.ceph.osd.6:Restarting daemon 2024-09-21T03:33:14.326 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-09-21T03:33:14.328 INFO:tasks.ceph.osd.6:Started 2024-09-21T03:33:14.328 INFO:tasks.ceph.osd.7:Restarting daemon 2024-09-21T03:33:14.328 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-09-21T03:33:14.330 INFO:tasks.ceph.osd.7:Started 2024-09-21T03:33:14.330 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T03:33:14.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:14.486+0000 7f51aa172640 1 -- 172.21.15.38:0/2150241859 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f51a40cfe00 msgr2=0x7f51a40d2a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:14.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:14.486+0000 7f51aa172640 1 --2- 172.21.15.38:0/2150241859 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f51a40cfe00 0x7f51a40d2a20 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f5194009830 tx=0x7f519402a510 comp rx=0 tx=0).stop 2024-09-21T03:33:14.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:14.487+0000 7f51aa172640 1 -- 172.21.15.38:0/2150241859 shutdown_connections 2024-09-21T03:33:14.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:14.487+0000 7f51aa172640 1 --2- 172.21.15.38:0/2150241859 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f51a4162590 0x7f51a41629a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:14.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:14.487+0000 7f51aa172640 1 --2- 172.21.15.38:0/2150241859 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f51a40cfe00 0x7f51a40d2a20 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:14.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:14.487+0000 7f51aa172640 1 -- 172.21.15.38:0/2150241859 >> 172.21.15.38:0/2150241859 conn(0x7f51a40d4440 msgr2=0x7f51a40d4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:14.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:14.487+0000 7f51aa172640 1 -- 172.21.15.38:0/2150241859 shutdown_connections 2024-09-21T03:33:14.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:14.487+0000 7f51aa172640 1 -- 172.21.15.38:0/2150241859 wait complete. 2024-09-21T03:33:14.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:14.488+0000 7f51aa172640 1 Processor -- start 2024-09-21T03:33:14.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:14.488+0000 7f51aa172640 1 -- start start 2024-09-21T03:33:14.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:14.490+0000 7f51aa172640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f51a40b2e60 0x7f51a414e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:14.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:14.490+0000 7f51aa172640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f51a40cfe00 0x7f51a40c8930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:14.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:14.490+0000 7f51aa172640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f51a4162590 0x7f51a40c8e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:14.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:14.490+0000 7f51aa172640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f51a40b3980 con 0x7f51a4162590 2024-09-21T03:33:14.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:14.490+0000 7f51aa172640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f51a40cf5a0 con 0x7f51a40cfe00 2024-09-21T03:33:14.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:14.490+0000 7f51aa172640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f51a40b3b00 con 0x7f51a40b2e60 2024-09-21T03:33:14.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:14.490+0000 7f51a3fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f51a4162590 0x7f51a40c8e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:14.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:14.490+0000 7f51a3fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f51a4162590 0x7f51a40c8e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:56108/0 (socket says 172.21.15.38:56108) 2024-09-21T03:33:14.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:14.490+0000 7f51a3fff640 1 -- 172.21.15.38:0/859264564 learned_addr learned my addr 172.21.15.38:0/859264564 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:33:14.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:14.490+0000 7f51a37fe640 1 --2- 172.21.15.38:0/859264564 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f51a40b2e60 0x7f51a414e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:14.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:14.490+0000 7f519bfff640 1 --2- 172.21.15.38:0/859264564 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f51a40cfe00 0x7f51a40c8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:14.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:14.490+0000 7f51a3fff640 1 -- 172.21.15.38:0/859264564 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f51a40b2e60 msgr2=0x7f51a414e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:14.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:14.490+0000 7f51a3fff640 1 --2- 172.21.15.38:0/859264564 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f51a40b2e60 0x7f51a414e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:14.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:14.490+0000 7f51a3fff640 1 -- 172.21.15.38:0/859264564 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f51a40cfe00 msgr2=0x7f51a40c8930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:14.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:14.490+0000 7f51a3fff640 1 --2- 172.21.15.38:0/859264564 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f51a40cfe00 0x7f51a40c8930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:14.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:14.490+0000 7f51a3fff640 1 -- 172.21.15.38:0/859264564 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51a414ecc0 con 0x7f51a4162590 2024-09-21T03:33:14.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:14.490+0000 7f51a37fe640 1 --2- 172.21.15.38:0/859264564 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f51a40b2e60 0x7f51a414e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:33:14.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:14.491+0000 7f51a3fff640 1 --2- 172.21.15.38:0/859264564 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f51a4162590 0x7f51a40c8e70 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f519c00caa0 tx=0x7f519c00cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:14.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:14.491+0000 7f51a0ff9640 1 -- 172.21.15.38:0/859264564 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f519c016020 con 0x7f51a4162590 2024-09-21T03:33:14.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:14.491+0000 7f51a0ff9640 1 -- 172.21.15.38:0/859264564 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f519c00acf0 con 0x7f51a4162590 2024-09-21T03:33:14.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:14.491+0000 7f51aa172640 1 -- 172.21.15.38:0/859264564 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51a415dbf0 con 0x7f51a4162590 2024-09-21T03:33:14.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:14.491+0000 7f51a0ff9640 1 -- 172.21.15.38:0/859264564 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f519c012680 con 0x7f51a4162590 2024-09-21T03:33:14.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:14.491+0000 7f51aa172640 1 -- 172.21.15.38:0/859264564 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51a415e1b0 con 0x7f51a4162590 2024-09-21T03:33:14.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:14.491+0000 7f51a0ff9640 1 -- 172.21.15.38:0/859264564 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f519c019020 con 0x7f51a4162590 2024-09-21T03:33:14.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:14.491+0000 7f51a0ff9640 1 -- 172.21.15.38:0/859264564 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f519c01bcc0 con 0x7f51a4162590 2024-09-21T03:33:14.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:14.492+0000 7f51aa172640 1 -- 172.21.15.38:0/859264564 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51a415d960 con 0x7f51a4162590 2024-09-21T03:33:14.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:14.494+0000 7f51a0ff9640 1 -- 172.21.15.38:0/859264564 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f519c020030 con 0x7f51a4162590 2024-09-21T03:33:14.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:14.556+0000 7f51aa172640 1 -- 172.21.15.38:0/859264564 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f51a4158b60 con 0x7f51a4162590 2024-09-21T03:33:14.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:14.556+0000 7f51a0ff9640 1 -- 172.21.15.38:0/859264564 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f51a4162ee0 con 0x7f51a4162590 2024-09-21T03:33:14.559 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:33:14.559 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":10,"fsid":"9d50d66f-656b-4fe2-8e3b-fe53b79c4d43","created":"2024-09-21T03:33:10.677865+0000","modified":"2024-09-21T03:33:14.296193+0000","last_up_change":"0.000000","last_in_change":"2024-09-21T03:33:14.296193+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"dbab183f-7e63-48b8-960a-ae55fd5bf565","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"fa1c0e8a-ebc1-4359-8146-bd5342abbc16","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"2ed3cd2d-385d-4fc6-9869-214552a84803","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"62d1d95e-919d-4250-be71-1d34cfa70dbb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"7c9f9314-99d7-4d73-8392-37809a9f54b5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"bde4fe9f-8414-4bba-bf4a-43973d1976bf","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"6ee6d7f4-1a5f-4a4c-ac16-48fe05f6a002","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"efeb5f51-329a-4484-8290-92e7d3503d05","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T03:33:14.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:14.557+0000 7f51aa172640 1 -- 172.21.15.38:0/859264564 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f51a4162590 msgr2=0x7f51a40c8e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:14.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:14.557+0000 7f51aa172640 1 --2- 172.21.15.38:0/859264564 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f51a4162590 0x7f51a40c8e70 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f519c00caa0 tx=0x7f519c00cf60 comp rx=0 tx=0).stop 2024-09-21T03:33:14.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:14.557+0000 7f51aa172640 1 -- 172.21.15.38:0/859264564 shutdown_connections 2024-09-21T03:33:14.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:14.557+0000 7f51aa172640 1 --2- 172.21.15.38:0/859264564 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f51a4162590 0x7f51a40c8e70 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:14.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:14.557+0000 7f51aa172640 1 --2- 172.21.15.38:0/859264564 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f51a40cfe00 0x7f51a40c8930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:14.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:14.557+0000 7f51aa172640 1 --2- 172.21.15.38:0/859264564 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f51a40b2e60 0x7f51a414e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:14.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:14.557+0000 7f51aa172640 1 -- 172.21.15.38:0/859264564 >> 172.21.15.38:0/859264564 conn(0x7f51a40d4440 msgr2=0x7f51a40b3270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:14.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:14.557+0000 7f51aa172640 1 -- 172.21.15.38:0/859264564 shutdown_connections 2024-09-21T03:33:14.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:14.557+0000 7f51aa172640 1 -- 172.21.15.38:0/859264564 wait complete. 2024-09-21T03:33:14.570 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-09-21T03:33:14.571 INFO:tasks.ceph:Waiting for OSDs to come up 2024-09-21T03:33:15.312 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-21T03:33:15.310+0000 7f8ba4011640 -1 Falling back to public interface 2024-09-21T03:33:15.317 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-21T03:33:15.314+0000 7fe0db13e640 -1 Falling back to public interface 2024-09-21T03:33:15.377 DEBUG:teuthology.orchestra.run.smithi038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-21T03:33:15.485 INFO:teuthology.misc.health.smithi038.stderr:2024-09-21T03:33:15.482+0000 7f5988f57640 1 -- 172.21.15.38:0/1152440319 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f59840afbd0 msgr2=0x7f59840cf320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:15.485 INFO:teuthology.misc.health.smithi038.stderr:2024-09-21T03:33:15.482+0000 7f5988f57640 1 --2- 172.21.15.38:0/1152440319 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f59840afbd0 0x7f59840cf320 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f5974005fd0 tx=0x7f597402a4b0 comp rx=0 tx=0).stop 2024-09-21T03:33:15.485 INFO:teuthology.misc.health.smithi038.stderr:2024-09-21T03:33:15.483+0000 7f5988f57640 1 -- 172.21.15.38:0/1152440319 shutdown_connections 2024-09-21T03:33:15.485 INFO:teuthology.misc.health.smithi038.stderr:2024-09-21T03:33:15.483+0000 7f5988f57640 1 --2- 172.21.15.38:0/1152440319 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f598415cae0 0x7f598406ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:15.485 INFO:teuthology.misc.health.smithi038.stderr:2024-09-21T03:33:15.483+0000 7f5988f57640 1 --2- 172.21.15.38:0/1152440319 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f59840afbd0 0x7f59840cf320 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:15.486 INFO:teuthology.misc.health.smithi038.stderr:2024-09-21T03:33:15.483+0000 7f5988f57640 1 -- 172.21.15.38:0/1152440319 >> 172.21.15.38:0/1152440319 conn(0x7f5984073560 msgr2=0x7f5984073970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:15.486 INFO:teuthology.misc.health.smithi038.stderr:2024-09-21T03:33:15.483+0000 7f5988f57640 1 -- 172.21.15.38:0/1152440319 shutdown_connections 2024-09-21T03:33:15.486 INFO:teuthology.misc.health.smithi038.stderr:2024-09-21T03:33:15.483+0000 7f5988f57640 1 -- 172.21.15.38:0/1152440319 wait complete. 2024-09-21T03:33:15.487 INFO:teuthology.misc.health.smithi038.stderr:2024-09-21T03:33:15.485+0000 7f5988f57640 1 Processor -- start 2024-09-21T03:33:15.487 INFO:teuthology.misc.health.smithi038.stderr:2024-09-21T03:33:15.485+0000 7f5988f57640 1 -- start start 2024-09-21T03:33:15.489 INFO:teuthology.misc.health.smithi038.stderr:2024-09-21T03:33:15.486+0000 7f5988f57640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f598406e330 0x7f59840ca5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:15.489 INFO:teuthology.misc.health.smithi038.stderr:2024-09-21T03:33:15.486+0000 7f5988f57640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f59840afbd0 0x7f59840b5b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:15.489 INFO:teuthology.misc.health.smithi038.stderr:2024-09-21T03:33:15.486+0000 7f5988f57640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f598415cae0 0x7f5984154370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:15.489 INFO:teuthology.misc.health.smithi038.stderr:2024-09-21T03:33:15.486+0000 7f5988f57640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f598415d070 con 0x7f59840afbd0 2024-09-21T03:33:15.489 INFO:teuthology.misc.health.smithi038.stderr:2024-09-21T03:33:15.486+0000 7f5988f57640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f598415cef0 con 0x7f598415cae0 2024-09-21T03:33:15.489 INFO:teuthology.misc.health.smithi038.stderr:2024-09-21T03:33:15.486+0000 7f5988f57640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f598415c480 con 0x7f598406e330 2024-09-21T03:33:15.489 INFO:teuthology.misc.health.smithi038.stderr:2024-09-21T03:33:15.486+0000 7f5979d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f59840afbd0 0x7f59840b5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:15.490 INFO:teuthology.misc.health.smithi038.stderr:2024-09-21T03:33:15.486+0000 7f5979d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f59840afbd0 0x7f59840b5b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:56120/0 (socket says 172.21.15.38:56120) 2024-09-21T03:33:15.490 INFO:teuthology.misc.health.smithi038.stderr:2024-09-21T03:33:15.486+0000 7f598259c640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f598406e330 0x7f59840ca5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:15.490 INFO:teuthology.misc.health.smithi038.stderr:2024-09-21T03:33:15.486+0000 7f5979d9b640 1 -- 172.21.15.38:0/2084121195 learned_addr learned my addr 172.21.15.38:0/2084121195 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:33:15.490 INFO:teuthology.misc.health.smithi038.stderr:2024-09-21T03:33:15.486+0000 7f5982d9d640 1 --2- 172.21.15.38:0/2084121195 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f598415cae0 0x7f5984154370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:15.490 INFO:teuthology.misc.health.smithi038.stderr:2024-09-21T03:33:15.487+0000 7f598259c640 1 -- 172.21.15.38:0/2084121195 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f598415cae0 msgr2=0x7f5984154370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:15.490 INFO:teuthology.misc.health.smithi038.stderr:2024-09-21T03:33:15.487+0000 7f598259c640 1 --2- 172.21.15.38:0/2084121195 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f598415cae0 0x7f5984154370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:15.490 INFO:teuthology.misc.health.smithi038.stderr:2024-09-21T03:33:15.487+0000 7f598259c640 1 -- 172.21.15.38:0/2084121195 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f59840afbd0 msgr2=0x7f59840b5b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:15.490 INFO:teuthology.misc.health.smithi038.stderr:2024-09-21T03:33:15.487+0000 7f598259c640 1 --2- 172.21.15.38:0/2084121195 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f59840afbd0 0x7f59840b5b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:15.490 INFO:teuthology.misc.health.smithi038.stderr:2024-09-21T03:33:15.487+0000 7f598259c640 1 -- 172.21.15.38:0/2084121195 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59841f37c0 con 0x7f598406e330 2024-09-21T03:33:15.490 INFO:teuthology.misc.health.smithi038.stderr:2024-09-21T03:33:15.487+0000 7f598259c640 1 --2- 172.21.15.38:0/2084121195 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f598406e330 0x7f59840ca5c0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f5974031000 tx=0x7f59740038a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:15.490 INFO:teuthology.misc.health.smithi038.stderr:2024-09-21T03:33:15.487+0000 7f597b7fe640 1 -- 172.21.15.38:0/2084121195 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f597403e030 con 0x7f598406e330 2024-09-21T03:33:15.490 INFO:teuthology.misc.health.smithi038.stderr:2024-09-21T03:33:15.487+0000 7f597b7fe640 1 -- 172.21.15.38:0/2084121195 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5974003ca0 con 0x7f598406e330 2024-09-21T03:33:15.491 INFO:teuthology.misc.health.smithi038.stderr:2024-09-21T03:33:15.487+0000 7f5988f57640 1 -- 172.21.15.38:0/2084121195 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59841f3aa0 con 0x7f598406e330 2024-09-21T03:33:15.491 INFO:teuthology.misc.health.smithi038.stderr:2024-09-21T03:33:15.487+0000 7f597b7fe640 1 -- 172.21.15.38:0/2084121195 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5974035040 con 0x7f598406e330 2024-09-21T03:33:15.491 INFO:teuthology.misc.health.smithi038.stderr:2024-09-21T03:33:15.487+0000 7f5988f57640 1 -- 172.21.15.38:0/2084121195 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f59841f4060 con 0x7f598406e330 2024-09-21T03:33:15.491 INFO:teuthology.misc.health.smithi038.stderr:2024-09-21T03:33:15.488+0000 7f597b7fe640 1 -- 172.21.15.38:0/2084121195 <== mon.2 v2:172.21.15.195:3301/0 4 ==== mgrmap(e 2) ==== 50742+0+0 (secure 0 0 0) 0x7f5974041030 con 0x7f598406e330 2024-09-21T03:33:15.491 INFO:teuthology.misc.health.smithi038.stderr:2024-09-21T03:33:15.488+0000 7f597b7fe640 1 -- 172.21.15.38:0/2084121195 <== mon.2 v2:172.21.15.195:3301/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f5974072b20 con 0x7f598406e330 2024-09-21T03:33:15.491 INFO:teuthology.misc.health.smithi038.stderr:2024-09-21T03:33:15.488+0000 7f5988f57640 1 -- 172.21.15.38:0/2084121195 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59841f6950 con 0x7f598406e330 2024-09-21T03:33:15.492 INFO:teuthology.misc.health.smithi038.stderr:2024-09-21T03:33:15.491+0000 7f597b7fe640 1 -- 172.21.15.38:0/2084121195 <== mon.2 v2:172.21.15.195:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f5974004d80 con 0x7f598406e330 2024-09-21T03:33:15.554 INFO:teuthology.misc.health.smithi038.stderr:2024-09-21T03:33:15.553+0000 7f5988f57640 1 -- 172.21.15.38:0/2084121195 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f59840cf320 con 0x7f598406e330 2024-09-21T03:33:15.555 INFO:teuthology.misc.health.smithi038.stderr:2024-09-21T03:33:15.553+0000 7f597b7fe640 1 -- 172.21.15.38:0/2084121195 <== mon.2 v2:172.21.15.195:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f5974062130 con 0x7f598406e330 2024-09-21T03:33:15.555 INFO:teuthology.misc.health.smithi038.stdout: 2024-09-21T03:33:15.556 INFO:teuthology.misc.health.smithi038.stdout:{"epoch":10,"fsid":"9d50d66f-656b-4fe2-8e3b-fe53b79c4d43","created":"2024-09-21T03:33:10.677865+0000","modified":"2024-09-21T03:33:14.296193+0000","last_up_change":"0.000000","last_in_change":"2024-09-21T03:33:14.296193+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"dbab183f-7e63-48b8-960a-ae55fd5bf565","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"fa1c0e8a-ebc1-4359-8146-bd5342abbc16","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"2ed3cd2d-385d-4fc6-9869-214552a84803","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"62d1d95e-919d-4250-be71-1d34cfa70dbb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"7c9f9314-99d7-4d73-8392-37809a9f54b5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"bde4fe9f-8414-4bba-bf4a-43973d1976bf","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"6ee6d7f4-1a5f-4a4c-ac16-48fe05f6a002","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"efeb5f51-329a-4484-8290-92e7d3503d05","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T03:33:15.557 INFO:teuthology.misc.health.smithi038.stderr:2024-09-21T03:33:15.554+0000 7f5988f57640 1 -- 172.21.15.38:0/2084121195 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f598406e330 msgr2=0x7f59840ca5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:15.557 INFO:teuthology.misc.health.smithi038.stderr:2024-09-21T03:33:15.554+0000 7f5988f57640 1 --2- 172.21.15.38:0/2084121195 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f598406e330 0x7f59840ca5c0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f5974031000 tx=0x7f59740038a0 comp rx=0 tx=0).stop 2024-09-21T03:33:15.557 INFO:teuthology.misc.health.smithi038.stderr:2024-09-21T03:33:15.554+0000 7f5988f57640 1 -- 172.21.15.38:0/2084121195 shutdown_connections 2024-09-21T03:33:15.557 INFO:teuthology.misc.health.smithi038.stderr:2024-09-21T03:33:15.554+0000 7f5988f57640 1 --2- 172.21.15.38:0/2084121195 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f598415cae0 0x7f5984154370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:15.557 INFO:teuthology.misc.health.smithi038.stderr:2024-09-21T03:33:15.554+0000 7f5988f57640 1 --2- 172.21.15.38:0/2084121195 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f59840afbd0 0x7f59840b5b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:15.557 INFO:teuthology.misc.health.smithi038.stderr:2024-09-21T03:33:15.554+0000 7f5988f57640 1 --2- 172.21.15.38:0/2084121195 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f598406e330 0x7f59840ca5c0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:15.557 INFO:teuthology.misc.health.smithi038.stderr:2024-09-21T03:33:15.555+0000 7f5988f57640 1 -- 172.21.15.38:0/2084121195 >> 172.21.15.38:0/2084121195 conn(0x7f5984073560 msgr2=0x7f598415bd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:15.557 INFO:teuthology.misc.health.smithi038.stderr:2024-09-21T03:33:15.555+0000 7f5988f57640 1 -- 172.21.15.38:0/2084121195 shutdown_connections 2024-09-21T03:33:15.557 INFO:teuthology.misc.health.smithi038.stderr:2024-09-21T03:33:15.555+0000 7f5988f57640 1 -- 172.21.15.38:0/2084121195 wait complete. 2024-09-21T03:33:15.566 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-09-21T03:33:15.808 INFO:tasks.ceph.osd.0.smithi038.stderr:2024-09-21T03:33:15.807+0000 7fa4e7e30640 -1 Falling back to public interface 2024-09-21T03:33:15.985 INFO:tasks.ceph.mgr.x.smithi195.stderr:2024-09-21T03:33:15.982+0000 7fe673483640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-21T03:33:15.985 INFO:tasks.ceph.mgr.x.smithi195.stderr:2024-09-21T03:33:15.982+0000 7fe673483640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-21T03:33:16.043 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-21T03:33:16.041+0000 7f04ec3ae640 -1 Falling back to public interface 2024-09-21T03:33:16.051 INFO:tasks.ceph.osd.3.smithi038.stderr:2024-09-21T03:33:16.049+0000 7f87c0efe640 -1 Falling back to public interface 2024-09-21T03:33:16.068 INFO:tasks.ceph.osd.1.smithi038.stderr:2024-09-21T03:33:16.067+0000 7fad54214640 -1 Falling back to public interface 2024-09-21T03:33:16.094 INFO:tasks.ceph.osd.2.smithi038.stderr:2024-09-21T03:33:16.093+0000 7f266c1dd640 -1 Falling back to public interface 2024-09-21T03:33:16.313 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-21T03:33:16.311+0000 7fb7107d4640 -1 Falling back to public interface 2024-09-21T03:33:18.857 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-21T03:33:18.855+0000 7fe0db13e640 -1 osd.5 0 log_to_monitors true 2024-09-21T03:33:19.113 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-21T03:33:19.111+0000 7f8ba4011640 -1 osd.4 0 log_to_monitors true 2024-09-21T03:33:19.832 INFO:tasks.ceph.osd.3.smithi038.stderr:2024-09-21T03:33:19.830+0000 7f87c0efe640 -1 osd.3 0 log_to_monitors true 2024-09-21T03:33:19.832 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-21T03:33:19.830+0000 7fb7107d4640 -1 osd.6 0 log_to_monitors true 2024-09-21T03:33:19.891 INFO:tasks.ceph.osd.0.smithi038.stderr:2024-09-21T03:33:19.890+0000 7fa4e7e30640 -1 osd.0 0 log_to_monitors true 2024-09-21T03:33:19.944 INFO:tasks.ceph.osd.2.smithi038.stderr:2024-09-21T03:33:19.942+0000 7f266c1dd640 -1 osd.2 0 log_to_monitors true 2024-09-21T03:33:20.050 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-21T03:33:20.048+0000 7f04ec3ae640 -1 osd.7 0 log_to_monitors true 2024-09-21T03:33:20.114 INFO:tasks.ceph.osd.1.smithi038.stderr:2024-09-21T03:33:20.112+0000 7fad54214640 -1 osd.1 0 log_to_monitors true 2024-09-21T03:33:20.630 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-21T03:33:20.628+0000 7fe0d78e1640 -1 osd.5 0 waiting for initial osdmap 2024-09-21T03:33:20.649 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-21T03:33:20.647+0000 7fe0ceec2640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T03:33:21.946 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-21T03:33:21.944+0000 7fb70cf77640 -1 osd.6 0 waiting for initial osdmap 2024-09-21T03:33:21.955 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-21T03:33:21.954+0000 7fb704558640 -1 osd.6 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T03:33:21.967 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-21T03:33:21.966+0000 7f8b9f8ec640 -1 osd.4 0 waiting for initial osdmap 2024-09-21T03:33:21.977 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-21T03:33:21.976+0000 7f8b976e2640 -1 osd.4 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T03:33:22.372 DEBUG:teuthology.orchestra.run.smithi038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-21T03:33:22.544 INFO:teuthology.misc.health.smithi038.stderr:2024-09-21T03:33:22.542+0000 7f1030add640 1 -- 172.21.15.38:0/1000630955 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f102c0ce860 msgr2=0x7f102c15f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:22.545 INFO:teuthology.misc.health.smithi038.stderr:2024-09-21T03:33:22.542+0000 7f1030add640 1 --2- 172.21.15.38:0/1000630955 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f102c0ce860 0x7f102c15f5a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f1024009830 tx=0x7f102402a410 comp rx=0 tx=0).stop 2024-09-21T03:33:22.545 INFO:teuthology.misc.health.smithi038.stderr:2024-09-21T03:33:22.542+0000 7f1030add640 1 -- 172.21.15.38:0/1000630955 shutdown_connections 2024-09-21T03:33:22.545 INFO:teuthology.misc.health.smithi038.stderr:2024-09-21T03:33:22.542+0000 7f1030add640 1 --2- 172.21.15.38:0/1000630955 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f102c0ce860 0x7f102c15f5a0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:22.545 INFO:teuthology.misc.health.smithi038.stderr:2024-09-21T03:33:22.542+0000 7f1030add640 1 --2- 172.21.15.38:0/1000630955 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f102c0cf930 0x7f102c0ce320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:22.545 INFO:teuthology.misc.health.smithi038.stderr:2024-09-21T03:33:22.542+0000 7f1030add640 1 -- 172.21.15.38:0/1000630955 >> 172.21.15.38:0/1000630955 conn(0x7f102c0cca40 msgr2=0x7f102c0cce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:22.545 INFO:teuthology.misc.health.smithi038.stderr:2024-09-21T03:33:22.543+0000 7f1030add640 1 -- 172.21.15.38:0/1000630955 shutdown_connections 2024-09-21T03:33:22.545 INFO:teuthology.misc.health.smithi038.stderr:2024-09-21T03:33:22.543+0000 7f1030add640 1 -- 172.21.15.38:0/1000630955 wait complete. 2024-09-21T03:33:22.547 INFO:teuthology.misc.health.smithi038.stderr:2024-09-21T03:33:22.546+0000 7f1030add640 1 Processor -- start 2024-09-21T03:33:22.550 INFO:teuthology.misc.health.smithi038.stderr:2024-09-21T03:33:22.546+0000 7f1030add640 1 -- start start 2024-09-21T03:33:22.550 INFO:teuthology.misc.health.smithi038.stderr:2024-09-21T03:33:22.548+0000 7f1030add640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f102c0ce860 0x7f102c157dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:22.552 INFO:teuthology.misc.health.smithi038.stderr:2024-09-21T03:33:22.548+0000 7f1030add640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f102c0cf930 0x7f102c1f5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:22.552 INFO:teuthology.misc.health.smithi038.stderr:2024-09-21T03:33:22.549+0000 7f1030add640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f102c15b9f0 0x7f102c0c4190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:22.552 INFO:teuthology.misc.health.smithi038.stderr:2024-09-21T03:33:22.549+0000 7f1030add640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f102c06df70 con 0x7f102c0ce860 2024-09-21T03:33:22.552 INFO:teuthology.misc.health.smithi038.stderr:2024-09-21T03:33:22.549+0000 7f1030add640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f102c06ddf0 con 0x7f102c0cf930 2024-09-21T03:33:22.552 INFO:teuthology.misc.health.smithi038.stderr:2024-09-21T03:33:22.549+0000 7f1030add640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f102c06e0f0 con 0x7f102c15b9f0 2024-09-21T03:33:22.552 INFO:teuthology.misc.health.smithi038.stderr:2024-09-21T03:33:22.549+0000 7f102a59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f102c0ce860 0x7f102c157dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:22.552 INFO:teuthology.misc.health.smithi038.stderr:2024-09-21T03:33:22.549+0000 7f102a59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f102c0ce860 0x7f102c157dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:58296/0 (socket says 172.21.15.38:58296) 2024-09-21T03:33:22.552 INFO:teuthology.misc.health.smithi038.stderr:2024-09-21T03:33:22.549+0000 7f102a59c640 1 -- 172.21.15.38:0/3521632988 learned_addr learned my addr 172.21.15.38:0/3521632988 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:33:22.553 INFO:teuthology.misc.health.smithi038.stderr:2024-09-21T03:33:22.549+0000 7f1029d9b640 1 --2- 172.21.15.38:0/3521632988 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f102c0cf930 0x7f102c1f5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:22.553 INFO:teuthology.misc.health.smithi038.stderr:2024-09-21T03:33:22.549+0000 7f102ad9d640 1 --2- 172.21.15.38:0/3521632988 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f102c15b9f0 0x7f102c0c4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:22.553 INFO:teuthology.misc.health.smithi038.stderr:2024-09-21T03:33:22.549+0000 7f102a59c640 1 -- 172.21.15.38:0/3521632988 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f102c15b9f0 msgr2=0x7f102c0c4190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:22.553 INFO:teuthology.misc.health.smithi038.stderr:2024-09-21T03:33:22.549+0000 7f102a59c640 1 --2- 172.21.15.38:0/3521632988 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f102c15b9f0 0x7f102c0c4190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:22.553 INFO:teuthology.misc.health.smithi038.stderr:2024-09-21T03:33:22.549+0000 7f102a59c640 1 -- 172.21.15.38:0/3521632988 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f102c0cf930 msgr2=0x7f102c1f5130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:22.553 INFO:teuthology.misc.health.smithi038.stderr:2024-09-21T03:33:22.549+0000 7f102a59c640 1 --2- 172.21.15.38:0/3521632988 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f102c0cf930 0x7f102c1f5130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:22.553 INFO:teuthology.misc.health.smithi038.stderr:2024-09-21T03:33:22.549+0000 7f102a59c640 1 -- 172.21.15.38:0/3521632988 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f102c1562d0 con 0x7f102c0ce860 2024-09-21T03:33:22.553 INFO:teuthology.misc.health.smithi038.stderr:2024-09-21T03:33:22.550+0000 7f102a59c640 1 --2- 172.21.15.38:0/3521632988 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f102c0ce860 0x7f102c157dd0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f101c0142b0 tx=0x7f101c014770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:22.553 INFO:teuthology.misc.health.smithi038.stderr:2024-09-21T03:33:22.550+0000 7f1022ffd640 1 -- 172.21.15.38:0/3521632988 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f101c014e20 con 0x7f102c0ce860 2024-09-21T03:33:22.553 INFO:teuthology.misc.health.smithi038.stderr:2024-09-21T03:33:22.550+0000 7f1030add640 1 -- 172.21.15.38:0/3521632988 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f102c1572f0 con 0x7f102c0ce860 2024-09-21T03:33:22.553 INFO:teuthology.misc.health.smithi038.stderr:2024-09-21T03:33:22.550+0000 7f1030add640 1 -- 172.21.15.38:0/3521632988 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f102c0a1190 con 0x7f102c0ce860 2024-09-21T03:33:22.553 INFO:teuthology.misc.health.smithi038.stderr:2024-09-21T03:33:22.550+0000 7f1022ffd640 1 -- 172.21.15.38:0/3521632988 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f101c003ba0 con 0x7f102c0ce860 2024-09-21T03:33:22.553 INFO:teuthology.misc.health.smithi038.stderr:2024-09-21T03:33:22.550+0000 7f1022ffd640 1 -- 172.21.15.38:0/3521632988 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f101c0120d0 con 0x7f102c0ce860 2024-09-21T03:33:22.554 INFO:teuthology.misc.health.smithi038.stderr:2024-09-21T03:33:22.552+0000 7f1030add640 1 -- 172.21.15.38:0/3521632988 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1010005290 con 0x7f102c0ce860 2024-09-21T03:33:22.555 INFO:teuthology.misc.health.smithi038.stderr:2024-09-21T03:33:22.554+0000 7f1022ffd640 1 -- 172.21.15.38:0/3521632988 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 4) ==== 101385+0+0 (secure 0 0 0) 0x7f101c012270 con 0x7f102c0ce860 2024-09-21T03:33:22.556 INFO:teuthology.misc.health.smithi038.stderr:2024-09-21T03:33:22.555+0000 7f1022ffd640 1 --2- 172.21.15.38:0/3521632988 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f100c078b30 0x7f100c07b040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:22.558 INFO:teuthology.misc.health.smithi038.stderr:2024-09-21T03:33:22.556+0000 7f1029d9b640 1 --2- 172.21.15.38:0/3521632988 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f100c078b30 0x7f100c07b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:22.559 INFO:teuthology.misc.health.smithi038.stderr:2024-09-21T03:33:22.556+0000 7f1022ffd640 1 -- 172.21.15.38:0/3521632988 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 3071+0+0 (secure 0 0 0) 0x7f101c0a22a0 con 0x7f102c0ce860 2024-09-21T03:33:22.559 INFO:teuthology.misc.health.smithi038.stderr:2024-09-21T03:33:22.557+0000 7f1029d9b640 1 --2- 172.21.15.38:0/3521632988 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f100c078b30 0x7f100c07b040 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f102c0ae7f0 tx=0x7f101400b2e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:22.568 INFO:teuthology.misc.health.smithi038.stderr:2024-09-21T03:33:22.565+0000 7f1022ffd640 1 -- 172.21.15.38:0/3521632988 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f101c06b200 con 0x7f102c0ce860 2024-09-21T03:33:22.630 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-21T03:33:22.628+0000 7f04e833c640 -1 osd.7 0 waiting for initial osdmap 2024-09-21T03:33:22.643 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-21T03:33:22.642+0000 7f04e0132640 -1 osd.7 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T03:33:22.796 INFO:tasks.ceph.osd.2.smithi038.stderr:2024-09-21T03:33:22.793+0000 7f266816b640 -1 osd.2 0 waiting for initial osdmap 2024-09-21T03:33:22.813 INFO:tasks.ceph.osd.2.smithi038.stderr:2024-09-21T03:33:22.812+0000 7f265ff61640 -1 osd.2 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T03:33:22.866 INFO:tasks.ceph.osd.3.smithi038.stderr:2024-09-21T03:33:22.864+0000 7f87bd6a1640 -1 osd.3 0 waiting for initial osdmap 2024-09-21T03:33:22.868 INFO:tasks.ceph.osd.0.smithi038.stderr:2024-09-21T03:33:22.866+0000 7fa4e3371640 -1 osd.0 0 waiting for initial osdmap 2024-09-21T03:33:22.882 INFO:tasks.ceph.osd.0.smithi038.stderr:2024-09-21T03:33:22.881+0000 7fa4db167640 -1 osd.0 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T03:33:22.883 INFO:tasks.ceph.osd.3.smithi038.stderr:2024-09-21T03:33:22.882+0000 7f87b4c82640 -1 osd.3 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T03:33:22.949 INFO:teuthology.misc.health.smithi038.stderr:2024-09-21T03:33:22.948+0000 7f1030add640 1 -- 172.21.15.38:0/3521632988 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1010005580 con 0x7f102c0ce860 2024-09-21T03:33:22.950 INFO:teuthology.misc.health.smithi038.stderr:2024-09-21T03:33:22.949+0000 7f1022ffd640 1 -- 172.21.15.38:0/3521632988 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v14) ==== 74+0+8562 (secure 0 0 0) 0x7f101c06ab00 con 0x7f102c0ce860 2024-09-21T03:33:22.950 INFO:teuthology.misc.health.smithi038.stdout: 2024-09-21T03:33:22.951 INFO:teuthology.misc.health.smithi038.stdout:{"epoch":14,"fsid":"9d50d66f-656b-4fe2-8e3b-fe53b79c4d43","created":"2024-09-21T03:33:10.677865+0000","modified":"2024-09-21T03:33:22.002522+0000","last_up_change":"2024-09-21T03:33:22.002522+0000","last_in_change":"2024-09-21T03:33:14.296193+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"dbab183f-7e63-48b8-960a-ae55fd5bf565","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"fa1c0e8a-ebc1-4359-8146-bd5342abbc16","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"2ed3cd2d-385d-4fc6-9869-214552a84803","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"62d1d95e-919d-4250-be71-1d34cfa70dbb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"7c9f9314-99d7-4d73-8392-37809a9f54b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6803","nonce":3401837571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6805","nonce":3401837571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6809","nonce":3401837571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6807","nonce":3401837571}]},"public_addr":"172.21.15.195:6803/3401837571","cluster_addr":"172.21.15.195:6805/3401837571","heartbeat_back_addr":"172.21.15.195:6809/3401837571","heartbeat_front_addr":"172.21.15.195:6807/3401837571","state":["exists","up"]},{"osd":5,"uuid":"bde4fe9f-8414-4bba-bf4a-43973d1976bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6811","nonce":2080960180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6813","nonce":2080960180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6817","nonce":2080960180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6815","nonce":2080960180}]},"public_addr":"172.21.15.195:6811/2080960180","cluster_addr":"172.21.15.195:6813/2080960180","heartbeat_back_addr":"172.21.15.195:6817/2080960180","heartbeat_front_addr":"172.21.15.195:6815/2080960180","state":["exists","up"]},{"osd":6,"uuid":"6ee6d7f4-1a5f-4a4c-ac16-48fe05f6a002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6827","nonce":4055825675}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6829","nonce":4055825675}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6833","nonce":4055825675}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6831","nonce":4055825675}]},"public_addr":"172.21.15.195:6827/4055825675","cluster_addr":"172.21.15.195:6829/4055825675","heartbeat_back_addr":"172.21.15.195:6833/4055825675","heartbeat_front_addr":"172.21.15.195:6831/4055825675","state":["exists","up"]},{"osd":7,"uuid":"efeb5f51-329a-4484-8290-92e7d3503d05","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:19.817560+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T03:33:22.954 INFO:teuthology.misc.health.smithi038.stderr:2024-09-21T03:33:22.952+0000 7f1030add640 1 -- 172.21.15.38:0/3521632988 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f100c078b30 msgr2=0x7f100c07b040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:22.954 INFO:teuthology.misc.health.smithi038.stderr:2024-09-21T03:33:22.952+0000 7f1030add640 1 --2- 172.21.15.38:0/3521632988 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f100c078b30 0x7f100c07b040 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f102c0ae7f0 tx=0x7f101400b2e0 comp rx=0 tx=0).stop 2024-09-21T03:33:22.954 INFO:teuthology.misc.health.smithi038.stderr:2024-09-21T03:33:22.952+0000 7f1030add640 1 -- 172.21.15.38:0/3521632988 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f102c0ce860 msgr2=0x7f102c157dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:22.954 INFO:teuthology.misc.health.smithi038.stderr:2024-09-21T03:33:22.952+0000 7f1030add640 1 --2- 172.21.15.38:0/3521632988 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f102c0ce860 0x7f102c157dd0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f101c0142b0 tx=0x7f101c014770 comp rx=0 tx=0).stop 2024-09-21T03:33:22.954 INFO:teuthology.misc.health.smithi038.stderr:2024-09-21T03:33:22.952+0000 7f1030add640 1 -- 172.21.15.38:0/3521632988 shutdown_connections 2024-09-21T03:33:22.954 INFO:teuthology.misc.health.smithi038.stderr:2024-09-21T03:33:22.952+0000 7f1030add640 1 --2- 172.21.15.38:0/3521632988 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f100c078b30 0x7f100c07b040 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:22.954 INFO:teuthology.misc.health.smithi038.stderr:2024-09-21T03:33:22.952+0000 7f1030add640 1 --2- 172.21.15.38:0/3521632988 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f102c15b9f0 0x7f102c0c4190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:22.954 INFO:teuthology.misc.health.smithi038.stderr:2024-09-21T03:33:22.952+0000 7f1030add640 1 --2- 172.21.15.38:0/3521632988 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f102c0cf930 0x7f102c1f5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:22.955 INFO:teuthology.misc.health.smithi038.stderr:2024-09-21T03:33:22.952+0000 7f1030add640 1 --2- 172.21.15.38:0/3521632988 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f102c0ce860 0x7f102c157dd0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:22.955 INFO:teuthology.misc.health.smithi038.stderr:2024-09-21T03:33:22.952+0000 7f1030add640 1 -- 172.21.15.38:0/3521632988 >> 172.21.15.38:0/3521632988 conn(0x7f102c0cca40 msgr2=0x7f102c15df80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:22.955 INFO:teuthology.misc.health.smithi038.stderr:2024-09-21T03:33:22.952+0000 7f1030add640 1 -- 172.21.15.38:0/3521632988 shutdown_connections 2024-09-21T03:33:22.955 INFO:teuthology.misc.health.smithi038.stderr:2024-09-21T03:33:22.952+0000 7f1030add640 1 -- 172.21.15.38:0/3521632988 wait complete. 2024-09-21T03:33:22.964 DEBUG:teuthology.misc:3 of 8 OSDs are up 2024-09-21T03:33:23.081 INFO:tasks.ceph.osd.1.smithi038.stderr:2024-09-21T03:33:23.079+0000 7fad502b3640 -1 osd.1 0 waiting for initial osdmap 2024-09-21T03:33:23.095 INFO:tasks.ceph.osd.1.smithi038.stderr:2024-09-21T03:33:23.094+0000 7fad47894640 -1 osd.1 15 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T03:33:29.300 DEBUG:teuthology.orchestra.run.smithi038:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T03:33:29.303 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T03:33:29.769 DEBUG:teuthology.orchestra.run.smithi038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-21T03:33:29.892 INFO:teuthology.misc.health.smithi038.stderr:2024-09-21T03:33:29.890+0000 7f263a69b640 1 -- 172.21.15.38:0/4157798872 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f261c005280 msgr2=0x7f261c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:29.892 INFO:teuthology.misc.health.smithi038.stderr:2024-09-21T03:33:29.890+0000 7f263a69b640 1 --2- 172.21.15.38:0/4157798872 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f261c005280 0x7f261c00b9c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f262c009fd0 tx=0x7f261c005830 comp rx=0 tx=0).stop 2024-09-21T03:33:29.892 INFO:teuthology.misc.health.smithi038.stderr:2024-09-21T03:33:29.890+0000 7f263a69b640 1 -- 172.21.15.38:0/4157798872 shutdown_connections 2024-09-21T03:33:29.892 INFO:teuthology.misc.health.smithi038.stderr:2024-09-21T03:33:29.890+0000 7f263a69b640 1 --2- 172.21.15.38:0/4157798872 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f261c005280 0x7f261c00b9c0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:29.893 INFO:teuthology.misc.health.smithi038.stderr:2024-09-21T03:33:29.890+0000 7f263a69b640 1 -- 172.21.15.38:0/4157798872 >> 172.21.15.38:0/4157798872 conn(0x7f26340d4440 msgr2=0x7f26340d4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:29.893 INFO:teuthology.misc.health.smithi038.stderr:2024-09-21T03:33:29.890+0000 7f263a69b640 1 -- 172.21.15.38:0/4157798872 shutdown_connections 2024-09-21T03:33:29.893 INFO:teuthology.misc.health.smithi038.stderr:2024-09-21T03:33:29.890+0000 7f263a69b640 1 -- 172.21.15.38:0/4157798872 wait complete. 2024-09-21T03:33:29.893 INFO:teuthology.misc.health.smithi038.stderr:2024-09-21T03:33:29.892+0000 7f263a69b640 1 Processor -- start 2024-09-21T03:33:29.894 INFO:teuthology.misc.health.smithi038.stderr:2024-09-21T03:33:29.892+0000 7f263a69b640 1 -- start start 2024-09-21T03:33:29.896 INFO:teuthology.misc.health.smithi038.stderr:2024-09-21T03:33:29.893+0000 7f263a69b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f261c005280 0x7f2634157030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:29.896 INFO:teuthology.misc.health.smithi038.stderr:2024-09-21T03:33:29.893+0000 7f263a69b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2634155530 0x7f263415d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:29.897 INFO:teuthology.misc.health.smithi038.stderr:2024-09-21T03:33:29.893+0000 7f263a69b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2634155940 0x7f26341595d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:29.897 INFO:teuthology.misc.health.smithi038.stderr:2024-09-21T03:33:29.893+0000 7f263a69b640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f2634162de0 con 0x7f2634155940 2024-09-21T03:33:29.897 INFO:teuthology.misc.health.smithi038.stderr:2024-09-21T03:33:29.893+0000 7f263a69b640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f2634162c60 con 0x7f261c005280 2024-09-21T03:33:29.897 INFO:teuthology.misc.health.smithi038.stderr:2024-09-21T03:33:29.893+0000 7f263a69b640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f26340d2a20 con 0x7f2634155530 2024-09-21T03:33:29.897 INFO:teuthology.misc.health.smithi038.stderr:2024-09-21T03:33:29.894+0000 7f2638c38640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2634155940 0x7f26341595d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:29.897 INFO:teuthology.misc.health.smithi038.stderr:2024-09-21T03:33:29.894+0000 7f2638c38640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2634155940 0x7f26341595d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:58314/0 (socket says 172.21.15.38:58314) 2024-09-21T03:33:29.897 INFO:teuthology.misc.health.smithi038.stderr:2024-09-21T03:33:29.894+0000 7f2638c38640 1 -- 172.21.15.38:0/2646648724 learned_addr learned my addr 172.21.15.38:0/2646648724 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:33:29.897 INFO:teuthology.misc.health.smithi038.stderr:2024-09-21T03:33:29.894+0000 7f2633fff640 1 --2- 172.21.15.38:0/2646648724 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f261c005280 0x7f2634157030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:29.897 INFO:teuthology.misc.health.smithi038.stderr:2024-09-21T03:33:29.894+0000 7f26337fe640 1 --2- 172.21.15.38:0/2646648724 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2634155530 0x7f263415d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:29.897 INFO:teuthology.misc.health.smithi038.stderr:2024-09-21T03:33:29.894+0000 7f2638c38640 1 -- 172.21.15.38:0/2646648724 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2634155530 msgr2=0x7f263415d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:29.897 INFO:teuthology.misc.health.smithi038.stderr:2024-09-21T03:33:29.894+0000 7f2638c38640 1 --2- 172.21.15.38:0/2646648724 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2634155530 0x7f263415d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:29.897 INFO:teuthology.misc.health.smithi038.stderr:2024-09-21T03:33:29.894+0000 7f2638c38640 1 -- 172.21.15.38:0/2646648724 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f261c005280 msgr2=0x7f2634157030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:29.898 INFO:teuthology.misc.health.smithi038.stderr:2024-09-21T03:33:29.894+0000 7f2638c38640 1 --2- 172.21.15.38:0/2646648724 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f261c005280 0x7f2634157030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:29.898 INFO:teuthology.misc.health.smithi038.stderr:2024-09-21T03:33:29.894+0000 7f2638c38640 1 -- 172.21.15.38:0/2646648724 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2634156a50 con 0x7f2634155940 2024-09-21T03:33:29.898 INFO:teuthology.misc.health.smithi038.stderr:2024-09-21T03:33:29.894+0000 7f2638c38640 1 --2- 172.21.15.38:0/2646648724 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2634155940 0x7f26341595d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f262c009fd0 tx=0x7f262c004870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:29.898 INFO:teuthology.misc.health.smithi038.stderr:2024-09-21T03:33:29.894+0000 7f2630ff9640 1 -- 172.21.15.38:0/2646648724 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f262c009ca0 con 0x7f2634155940 2024-09-21T03:33:29.898 INFO:teuthology.misc.health.smithi038.stderr:2024-09-21T03:33:29.894+0000 7f2630ff9640 1 -- 172.21.15.38:0/2646648724 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f262c03b450 con 0x7f2634155940 2024-09-21T03:33:29.898 INFO:teuthology.misc.health.smithi038.stderr:2024-09-21T03:33:29.894+0000 7f263a69b640 1 -- 172.21.15.38:0/2646648724 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f263415ce70 con 0x7f2634155940 2024-09-21T03:33:29.898 INFO:teuthology.misc.health.smithi038.stderr:2024-09-21T03:33:29.894+0000 7f2630ff9640 1 -- 172.21.15.38:0/2646648724 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f262c03b880 con 0x7f2634155940 2024-09-21T03:33:29.898 INFO:teuthology.misc.health.smithi038.stderr:2024-09-21T03:33:29.894+0000 7f263a69b640 1 -- 172.21.15.38:0/2646648724 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f263415cb40 con 0x7f2634155940 2024-09-21T03:33:29.899 INFO:teuthology.misc.health.smithi038.stderr:2024-09-21T03:33:29.895+0000 7f263a69b640 1 -- 172.21.15.38:0/2646648724 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26340d2f80 con 0x7f2634155940 2024-09-21T03:33:29.899 INFO:teuthology.misc.health.smithi038.stderr:2024-09-21T03:33:29.896+0000 7f2630ff9640 1 -- 172.21.15.38:0/2646648724 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f262c044020 con 0x7f2634155940 2024-09-21T03:33:29.899 INFO:teuthology.misc.health.smithi038.stderr:2024-09-21T03:33:29.897+0000 7f2630ff9640 1 --2- 172.21.15.38:0/2646648724 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2620078b90 0x7f262007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:29.899 INFO:teuthology.misc.health.smithi038.stderr:2024-09-21T03:33:29.897+0000 7f2630ff9640 1 -- 172.21.15.38:0/2646648724 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4857+0+0 (secure 0 0 0) 0x7f262c0bb650 con 0x7f2634155940 2024-09-21T03:33:29.899 INFO:teuthology.misc.health.smithi038.stderr:2024-09-21T03:33:29.897+0000 7f2633fff640 1 --2- 172.21.15.38:0/2646648724 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2620078b90 0x7f262007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:29.899 INFO:teuthology.misc.health.smithi038.stderr:2024-09-21T03:33:29.897+0000 7f2633fff640 1 --2- 172.21.15.38:0/2646648724 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2620078b90 0x7f262007b0a0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f2624001050 tx=0x7f26240011c0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:29.904 INFO:teuthology.misc.health.smithi038.stderr:2024-09-21T03:33:29.902+0000 7f2630ff9640 1 -- 172.21.15.38:0/2646648724 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f262c04b020 con 0x7f2634155940 2024-09-21T03:33:30.094 INFO:teuthology.misc.health.smithi038.stderr:2024-09-21T03:33:30.092+0000 7f263a69b640 1 -- 172.21.15.38:0/2646648724 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2634076180 con 0x7f2634155940 2024-09-21T03:33:30.094 INFO:teuthology.misc.health.smithi038.stderr:2024-09-21T03:33:30.093+0000 7f2630ff9640 1 -- 172.21.15.38:0/2646648724 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) ==== 74+0+12921 (secure 0 0 0) 0x7f262c0836f0 con 0x7f2634155940 2024-09-21T03:33:30.095 INFO:teuthology.misc.health.smithi038.stdout: 2024-09-21T03:33:30.095 INFO:teuthology.misc.health.smithi038.stdout:{"epoch":18,"fsid":"9d50d66f-656b-4fe2-8e3b-fe53b79c4d43","created":"2024-09-21T03:33:10.677865+0000","modified":"2024-09-21T03:33:26.038933+0000","last_up_change":"2024-09-21T03:33:24.023421+0000","last_in_change":"2024-09-21T03:33:14.296193+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T03:33:22.992573+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dbab183f-7e63-48b8-960a-ae55fd5bf565","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6801","nonce":2381639433}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6803","nonce":2381639433}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6807","nonce":2381639433}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6805","nonce":2381639433}]},"public_addr":"172.21.15.38:6801/2381639433","cluster_addr":"172.21.15.38:6803/2381639433","heartbeat_back_addr":"172.21.15.38:6807/2381639433","heartbeat_front_addr":"172.21.15.38:6805/2381639433","state":["exists","up"]},{"osd":1,"uuid":"fa1c0e8a-ebc1-4359-8146-bd5342abbc16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6817","nonce":1491818732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6819","nonce":1491818732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6823","nonce":1491818732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6821","nonce":1491818732}]},"public_addr":"172.21.15.38:6817/1491818732","cluster_addr":"172.21.15.38:6819/1491818732","heartbeat_back_addr":"172.21.15.38:6823/1491818732","heartbeat_front_addr":"172.21.15.38:6821/1491818732","state":["exists","up"]},{"osd":2,"uuid":"2ed3cd2d-385d-4fc6-9869-214552a84803","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6825","nonce":2577858902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6827","nonce":2577858902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6831","nonce":2577858902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6829","nonce":2577858902}]},"public_addr":"172.21.15.38:6825/2577858902","cluster_addr":"172.21.15.38:6827/2577858902","heartbeat_back_addr":"172.21.15.38:6831/2577858902","heartbeat_front_addr":"172.21.15.38:6829/2577858902","state":["exists","up"]},{"osd":3,"uuid":"62d1d95e-919d-4250-be71-1d34cfa70dbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6809","nonce":1286810294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6811","nonce":1286810294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6815","nonce":1286810294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6813","nonce":1286810294}]},"public_addr":"172.21.15.38:6809/1286810294","cluster_addr":"172.21.15.38:6811/1286810294","heartbeat_back_addr":"172.21.15.38:6815/1286810294","heartbeat_front_addr":"172.21.15.38:6813/1286810294","state":["exists","up"]},{"osd":4,"uuid":"7c9f9314-99d7-4d73-8392-37809a9f54b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6803","nonce":3401837571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6805","nonce":3401837571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6809","nonce":3401837571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6807","nonce":3401837571}]},"public_addr":"172.21.15.195:6803/3401837571","cluster_addr":"172.21.15.195:6805/3401837571","heartbeat_back_addr":"172.21.15.195:6809/3401837571","heartbeat_front_addr":"172.21.15.195:6807/3401837571","state":["exists","up"]},{"osd":5,"uuid":"bde4fe9f-8414-4bba-bf4a-43973d1976bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6811","nonce":2080960180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6813","nonce":2080960180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6817","nonce":2080960180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6815","nonce":2080960180}]},"public_addr":"172.21.15.195:6811/2080960180","cluster_addr":"172.21.15.195:6813/2080960180","heartbeat_back_addr":"172.21.15.195:6817/2080960180","heartbeat_front_addr":"172.21.15.195:6815/2080960180","state":["exists","up"]},{"osd":6,"uuid":"6ee6d7f4-1a5f-4a4c-ac16-48fe05f6a002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6827","nonce":4055825675}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6829","nonce":4055825675}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6833","nonce":4055825675}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6831","nonce":4055825675}]},"public_addr":"172.21.15.195:6827/4055825675","cluster_addr":"172.21.15.195:6829/4055825675","heartbeat_back_addr":"172.21.15.195:6833/4055825675","heartbeat_front_addr":"172.21.15.195:6831/4055825675","state":["exists","up"]},{"osd":7,"uuid":"efeb5f51-329a-4484-8290-92e7d3503d05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6819","nonce":1606106342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6821","nonce":1606106342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6825","nonce":1606106342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6823","nonce":1606106342}]},"public_addr":"172.21.15.195:6819/1606106342","cluster_addr":"172.21.15.195:6821/1606106342","heartbeat_back_addr":"172.21.15.195:6825/1606106342","heartbeat_front_addr":"172.21.15.195:6823/1606106342","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.901343+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:21.115003+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.901623+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.835238+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.132199+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:19.817560+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.829690+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:21.087300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T03:33:30.097 INFO:teuthology.misc.health.smithi038.stderr:2024-09-21T03:33:30.095+0000 7f263a69b640 1 -- 172.21.15.38:0/2646648724 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2620078b90 msgr2=0x7f262007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:30.097 INFO:teuthology.misc.health.smithi038.stderr:2024-09-21T03:33:30.095+0000 7f263a69b640 1 --2- 172.21.15.38:0/2646648724 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2620078b90 0x7f262007b0a0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f2624001050 tx=0x7f26240011c0 comp rx=0 tx=0).stop 2024-09-21T03:33:30.098 INFO:teuthology.misc.health.smithi038.stderr:2024-09-21T03:33:30.095+0000 7f263a69b640 1 -- 172.21.15.38:0/2646648724 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2634155940 msgr2=0x7f26341595d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:30.098 INFO:teuthology.misc.health.smithi038.stderr:2024-09-21T03:33:30.095+0000 7f263a69b640 1 --2- 172.21.15.38:0/2646648724 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2634155940 0x7f26341595d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f262c009fd0 tx=0x7f262c004870 comp rx=0 tx=0).stop 2024-09-21T03:33:30.098 INFO:teuthology.misc.health.smithi038.stderr:2024-09-21T03:33:30.095+0000 7f263a69b640 1 -- 172.21.15.38:0/2646648724 shutdown_connections 2024-09-21T03:33:30.098 INFO:teuthology.misc.health.smithi038.stderr:2024-09-21T03:33:30.095+0000 7f263a69b640 1 --2- 172.21.15.38:0/2646648724 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2634155940 0x7f26341595d0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:30.098 INFO:teuthology.misc.health.smithi038.stderr:2024-09-21T03:33:30.095+0000 7f263a69b640 1 --2- 172.21.15.38:0/2646648724 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2620078b90 0x7f262007b0a0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:30.098 INFO:teuthology.misc.health.smithi038.stderr:2024-09-21T03:33:30.095+0000 7f263a69b640 1 --2- 172.21.15.38:0/2646648724 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2634155530 0x7f263415d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:30.098 INFO:teuthology.misc.health.smithi038.stderr:2024-09-21T03:33:30.095+0000 7f263a69b640 1 --2- 172.21.15.38:0/2646648724 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f261c005280 0x7f2634157030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:30.098 INFO:teuthology.misc.health.smithi038.stderr:2024-09-21T03:33:30.095+0000 7f263a69b640 1 -- 172.21.15.38:0/2646648724 >> 172.21.15.38:0/2646648724 conn(0x7f26340d4440 msgr2=0x7f263406d120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:30.098 INFO:teuthology.misc.health.smithi038.stderr:2024-09-21T03:33:30.095+0000 7f263a69b640 1 -- 172.21.15.38:0/2646648724 shutdown_connections 2024-09-21T03:33:30.098 INFO:teuthology.misc.health.smithi038.stderr:2024-09-21T03:33:30.095+0000 7f263a69b640 1 -- 172.21.15.38:0/2646648724 wait complete. 2024-09-21T03:33:30.107 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-09-21T03:33:30.107 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-09-21T03:33:30.108 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-21T03:33:30.108 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-21T03:33:30.110 INFO:tasks.ceph.mds.a:Started 2024-09-21T03:33:30.110 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-21T03:33:30.110 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-21T03:33:30.112 INFO:tasks.ceph.mds.c:Started 2024-09-21T03:33:30.112 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-21T03:33:30.112 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-21T03:33:30.115 INFO:tasks.ceph.mds.b:Started 2024-09-21T03:33:30.115 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-21T03:33:30.115 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-21T03:33:30.118 INFO:tasks.ceph.mds.d:Started 2024-09-21T03:33:30.118 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-09-21T03:33:30.118 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:33:30.200 INFO:tasks.ceph.mds.c.smithi038.stdout:starting mds.c at 2024-09-21T03:33:30.201 INFO:tasks.ceph.mds.a.smithi038.stdout:starting mds.a at 2024-09-21T03:33:30.213 INFO:tasks.ceph.mds.b.smithi195.stdout:starting mds.b at 2024-09-21T03:33:30.213 INFO:tasks.ceph.mds.d.smithi195.stdout:starting mds.d at 2024-09-21T03:33:30.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:30.240+0000 7fcffa4e6640 1 -- 172.21.15.38:0/62859955 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcff41536e0 msgr2=0x7fcff40d17c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:30.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:30.240+0000 7fcffa4e6640 1 --2- 172.21.15.38:0/62859955 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcff41536e0 0x7fcff40d17c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fcff40b2db0 tx=0x7fcfe402a5f0 comp rx=0 tx=0).stop 2024-09-21T03:33:30.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:30.240+0000 7fcffa4e6640 1 -- 172.21.15.38:0/62859955 shutdown_connections 2024-09-21T03:33:30.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:30.240+0000 7fcffa4e6640 1 --2- 172.21.15.38:0/62859955 >> v2:172.21.15.195:3300/0 conn(0x7fcff40cd040 0x7fcff40d04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:30.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:30.240+0000 7fcffa4e6640 1 --2- 172.21.15.38:0/62859955 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcff41536e0 0x7fcff40d17c0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:30.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:30.240+0000 7fcffa4e6640 1 -- 172.21.15.38:0/62859955 >> 172.21.15.38:0/62859955 conn(0x7fcff40cef40 msgr2=0x7fcff40cf350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:30.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:30.240+0000 7fcffa4e6640 1 -- 172.21.15.38:0/62859955 shutdown_connections 2024-09-21T03:33:30.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:30.240+0000 7fcffa4e6640 1 -- 172.21.15.38:0/62859955 wait complete. 2024-09-21T03:33:30.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:30.242+0000 7fcffa4e6640 1 Processor -- start 2024-09-21T03:33:30.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:30.242+0000 7fcffa4e6640 1 -- start start 2024-09-21T03:33:30.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:30.243+0000 7fcffa4e6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcff40d1d00 0x7fcff4150d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:30.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:30.243+0000 7fcffa4e6640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcff41536e0 0x7fcff41e6220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:30.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:30.243+0000 7fcffa4e6640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcff41e3c50 0x7fcff41e6760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:30.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:30.243+0000 7fcffa4e6640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fcff40d0d90 con 0x7fcff40d1d00 2024-09-21T03:33:30.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:30.243+0000 7fcffa4e6640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fcff40d0c10 con 0x7fcff41e3c50 2024-09-21T03:33:30.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:30.243+0000 7fcffa4e6640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fcff40d0f10 con 0x7fcff41536e0 2024-09-21T03:33:30.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:30.243+0000 7fcff3fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcff40d1d00 0x7fcff4150d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:30.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:30.243+0000 7fcff3fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcff40d1d00 0x7fcff4150d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:58354/0 (socket says 172.21.15.38:58354) 2024-09-21T03:33:30.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:30.243+0000 7fcff3fff640 1 -- 172.21.15.38:0/85318773 learned_addr learned my addr 172.21.15.38:0/85318773 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:33:30.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:30.243+0000 7fcfebfff640 1 --2- 172.21.15.38:0/85318773 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcff41536e0 0x7fcff41e6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:30.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:30.243+0000 7fcff8a83640 1 --2- 172.21.15.38:0/85318773 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcff41e3c50 0x7fcff41e6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:30.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:30.243+0000 7fcff3fff640 1 -- 172.21.15.38:0/85318773 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcff41536e0 msgr2=0x7fcff41e6220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:30.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:30.243+0000 7fcff3fff640 1 --2- 172.21.15.38:0/85318773 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcff41536e0 0x7fcff41e6220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:30.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:30.243+0000 7fcff3fff640 1 -- 172.21.15.38:0/85318773 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcff41e3c50 msgr2=0x7fcff41e6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:30.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:30.243+0000 7fcff3fff640 1 --2- 172.21.15.38:0/85318773 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcff41e3c50 0x7fcff41e6760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:30.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:30.243+0000 7fcff3fff640 1 -- 172.21.15.38:0/85318773 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcff4151ff0 con 0x7fcff40d1d00 2024-09-21T03:33:30.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:30.243+0000 7fcff8a83640 1 --2- 172.21.15.38:0/85318773 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcff41e3c50 0x7fcff41e6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:33:30.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:30.244+0000 7fcff3fff640 1 --2- 172.21.15.38:0/85318773 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcff40d1d00 0x7fcff4150d80 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fcfe4034040 tx=0x7fcfe4003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:30.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:30.244+0000 7fcff17fa640 1 -- 172.21.15.38:0/85318773 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfe403e030 con 0x7fcff40d1d00 2024-09-21T03:33:30.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:30.244+0000 7fcff17fa640 1 -- 172.21.15.38:0/85318773 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcfe4003d10 con 0x7fcff40d1d00 2024-09-21T03:33:30.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:30.244+0000 7fcffa4e6640 1 -- 172.21.15.38:0/85318773 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcff41e2ed0 con 0x7fcff40d1d00 2024-09-21T03:33:30.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:30.244+0000 7fcff17fa640 1 -- 172.21.15.38:0/85318773 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfe4035040 con 0x7fcff40d1d00 2024-09-21T03:33:30.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:30.244+0000 7fcffa4e6640 1 -- 172.21.15.38:0/85318773 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcff41e3170 con 0x7fcff40d1d00 2024-09-21T03:33:30.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:30.245+0000 7fcffa4e6640 1 -- 172.21.15.38:0/85318773 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcff40d04c0 con 0x7fcff40d1d00 2024-09-21T03:33:30.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:30.245+0000 7fcff17fa640 1 -- 172.21.15.38:0/85318773 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fcfe4041030 con 0x7fcff40d1d00 2024-09-21T03:33:30.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:30.246+0000 7fcff17fa640 1 --2- 172.21.15.38:0/85318773 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fcfd4078b90 0x7fcfd407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:30.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:30.246+0000 7fcff17fa640 1 -- 172.21.15.38:0/85318773 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4857+0+0 (secure 0 0 0) 0x7fcfe40bb080 con 0x7fcff40d1d00 2024-09-21T03:33:30.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:30.246+0000 7fcfebfff640 1 --2- 172.21.15.38:0/85318773 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fcfd4078b90 0x7fcfd407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:30.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:30.247+0000 7fcfebfff640 1 --2- 172.21.15.38:0/85318773 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fcfd4078b90 0x7fcfd407b0a0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fcff41517a0 tx=0x7fcfdc009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:30.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:30.251+0000 7fcff17fa640 1 -- 172.21.15.38:0/85318773 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fcfe4083250 con 0x7fcff40d1d00 2024-09-21T03:33:30.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:30.478+0000 7fcffa4e6640 1 -- 172.21.15.38:0/85318773 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcff40b7cb0 con 0x7fcff40d1d00 2024-09-21T03:33:30.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:30.479+0000 7fcff17fa640 1 -- 172.21.15.38:0/85318773 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1431 (secure 0 0 0) 0x7fcfe4087c80 con 0x7fcff40d1d00 2024-09-21T03:33:30.482 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:33:30.482 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":2,"btime":"2024-09-21T03:33:30:205509+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4374,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/663402481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":663402481},{"type":"v1","addr":"172.21.15.38:6833","nonce":663402481}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-21T03:33:30.483 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 2 2024-09-21T03:33:30.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:30.481+0000 7fcffa4e6640 1 -- 172.21.15.38:0/85318773 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fcfd4078b90 msgr2=0x7fcfd407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:30.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:30.481+0000 7fcffa4e6640 1 --2- 172.21.15.38:0/85318773 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fcfd4078b90 0x7fcfd407b0a0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fcff41517a0 tx=0x7fcfdc009040 comp rx=0 tx=0).stop 2024-09-21T03:33:30.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:30.481+0000 7fcffa4e6640 1 -- 172.21.15.38:0/85318773 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcff40d1d00 msgr2=0x7fcff4150d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:30.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:30.481+0000 7fcffa4e6640 1 --2- 172.21.15.38:0/85318773 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcff40d1d00 0x7fcff4150d80 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fcfe4034040 tx=0x7fcfe4003820 comp rx=0 tx=0).stop 2024-09-21T03:33:30.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:30.481+0000 7fcffa4e6640 1 -- 172.21.15.38:0/85318773 shutdown_connections 2024-09-21T03:33:30.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:30.481+0000 7fcffa4e6640 1 --2- 172.21.15.38:0/85318773 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcff41e3c50 0x7fcff41e6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:30.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:30.481+0000 7fcffa4e6640 1 --2- 172.21.15.38:0/85318773 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fcfd4078b90 0x7fcfd407b0a0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:30.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:30.481+0000 7fcffa4e6640 1 --2- 172.21.15.38:0/85318773 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcff41536e0 0x7fcff41e6220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:30.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:30.481+0000 7fcffa4e6640 1 --2- 172.21.15.38:0/85318773 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcff40d1d00 0x7fcff4150d80 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:30.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:30.481+0000 7fcffa4e6640 1 -- 172.21.15.38:0/85318773 >> 172.21.15.38:0/85318773 conn(0x7fcff40cef40 msgr2=0x7fcff40b5e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:30.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:30.481+0000 7fcffa4e6640 1 -- 172.21.15.38:0/85318773 shutdown_connections 2024-09-21T03:33:30.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:30.481+0000 7fcffa4e6640 1 -- 172.21.15.38:0/85318773 wait complete. 2024-09-21T03:33:32.495 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:33:32.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:32.624+0000 7f2e45913640 1 -- 172.21.15.38:0/3118181020 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2e4006e500 msgr2=0x7f2e4015bd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:32.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:32.624+0000 7f2e45913640 1 --2- 172.21.15.38:0/3118181020 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2e4006e500 0x7f2e4015bd70 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2e38009830 tx=0x7f2e3802a4a0 comp rx=0 tx=0).stop 2024-09-21T03:33:32.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:32.624+0000 7f2e45913640 1 -- 172.21.15.38:0/3118181020 shutdown_connections 2024-09-21T03:33:32.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:32.624+0000 7f2e45913640 1 --2- 172.21.15.38:0/3118181020 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2e4006e500 0x7f2e4015bd70 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:32.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:32.624+0000 7f2e45913640 1 -- 172.21.15.38:0/3118181020 >> 172.21.15.38:0/3118181020 conn(0x7f2e40073560 msgr2=0x7f2e40073970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:32.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:32.624+0000 7f2e45913640 1 -- 172.21.15.38:0/3118181020 shutdown_connections 2024-09-21T03:33:32.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:32.624+0000 7f2e45913640 1 -- 172.21.15.38:0/3118181020 wait complete. 2024-09-21T03:33:32.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:32.626+0000 7f2e45913640 1 Processor -- start 2024-09-21T03:33:32.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:32.626+0000 7f2e45913640 1 -- start start 2024-09-21T03:33:32.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:32.628+0000 7f2e45913640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2e400afbd0 0x7f2e400ae980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:32.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:32.628+0000 7f2e45913640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2e4015cae0 0x7f2e401f6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:32.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:32.628+0000 7f2e45913640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2e400b4f80 0x7f2e400b5390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:32.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:32.628+0000 7f2e45913640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f2e4015c430 con 0x7f2e400b4f80 2024-09-21T03:33:32.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:32.628+0000 7f2e45913640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f2e4015c2b0 con 0x7f2e4015cae0 2024-09-21T03:33:32.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:32.628+0000 7f2e45913640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f2e4015c5b0 con 0x7f2e400afbd0 2024-09-21T03:33:32.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:32.628+0000 7f2e3f7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2e400b4f80 0x7f2e400b5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:32.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:32.628+0000 7f2e3f7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2e400b4f80 0x7f2e400b5390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:39362/0 (socket says 172.21.15.38:39362) 2024-09-21T03:33:32.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:32.628+0000 7f2e3f7fe640 1 -- 172.21.15.38:0/2030354558 learned_addr learned my addr 172.21.15.38:0/2030354558 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:33:32.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:32.628+0000 7f2e3effd640 1 --2- 172.21.15.38:0/2030354558 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2e400afbd0 0x7f2e400ae980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:32.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:32.628+0000 7f2e3e7fc640 1 --2- 172.21.15.38:0/2030354558 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2e4015cae0 0x7f2e401f6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:32.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:32.628+0000 7f2e3f7fe640 1 -- 172.21.15.38:0/2030354558 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2e400afbd0 msgr2=0x7f2e400ae980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:32.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:32.628+0000 7f2e3f7fe640 1 --2- 172.21.15.38:0/2030354558 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2e400afbd0 0x7f2e400ae980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:32.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:32.628+0000 7f2e3f7fe640 1 -- 172.21.15.38:0/2030354558 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2e4015cae0 msgr2=0x7f2e401f6240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:32.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:32.628+0000 7f2e3f7fe640 1 --2- 172.21.15.38:0/2030354558 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2e4015cae0 0x7f2e401f6240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:32.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:32.628+0000 7f2e3f7fe640 1 -- 172.21.15.38:0/2030354558 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e400b58d0 con 0x7f2e400b4f80 2024-09-21T03:33:32.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:32.628+0000 7f2e3e7fc640 1 --2- 172.21.15.38:0/2030354558 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2e4015cae0 0x7f2e401f6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:33:32.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:32.628+0000 7f2e3f7fe640 1 --2- 172.21.15.38:0/2030354558 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2e400b4f80 0x7f2e400b5390 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2e38009950 tx=0x7f2e38033ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:32.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:32.628+0000 7f2e37fff640 1 -- 172.21.15.38:0/2030354558 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e38031d60 con 0x7f2e400b4f80 2024-09-21T03:33:32.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:32.629+0000 7f2e37fff640 1 -- 172.21.15.38:0/2030354558 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2e38005020 con 0x7f2e400b4f80 2024-09-21T03:33:32.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:32.629+0000 7f2e45913640 1 -- 172.21.15.38:0/2030354558 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e400ad980 con 0x7f2e400b4f80 2024-09-21T03:33:32.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:32.629+0000 7f2e37fff640 1 -- 172.21.15.38:0/2030354558 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e38005450 con 0x7f2e400b4f80 2024-09-21T03:33:32.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:32.629+0000 7f2e45913640 1 -- 172.21.15.38:0/2030354558 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e400adf00 con 0x7f2e400b4f80 2024-09-21T03:33:32.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:32.630+0000 7f2e45913640 1 -- 172.21.15.38:0/2030354558 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e4015bd70 con 0x7f2e400b4f80 2024-09-21T03:33:32.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:32.630+0000 7f2e37fff640 1 -- 172.21.15.38:0/2030354558 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f2e380047a0 con 0x7f2e400b4f80 2024-09-21T03:33:32.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:32.631+0000 7f2e37fff640 1 --2- 172.21.15.38:0/2030354558 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2e20078b90 0x7f2e2007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:32.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:32.631+0000 7f2e37fff640 1 -- 172.21.15.38:0/2030354558 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4857+0+0 (secure 0 0 0) 0x7f2e380bbd00 con 0x7f2e400b4f80 2024-09-21T03:33:32.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:32.631+0000 7f2e3effd640 1 --2- 172.21.15.38:0/2030354558 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2e20078b90 0x7f2e2007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:32.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:32.632+0000 7f2e3effd640 1 --2- 172.21.15.38:0/2030354558 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2e20078b90 0x7f2e2007b0a0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f2e30004a30 tx=0x7f2e30001a50 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:32.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:32.636+0000 7f2e37fff640 1 -- 172.21.15.38:0/2030354558 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f2e400b58d0 con 0x7f2e400b4f80 2024-09-21T03:33:32.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:32.861+0000 7f2e45913640 1 -- 172.21.15.38:0/2030354558 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2e40159360 con 0x7f2e400b4f80 2024-09-21T03:33:32.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:32.862+0000 7f2e37fff640 1 -- 172.21.15.38:0/2030354558 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+3906 (secure 0 0 0) 0x7f2e40159360 con 0x7f2e400b4f80 2024-09-21T03:33:32.864 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:33:32.864 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":3,"btime":"2024-09-21T03:33:31:206628+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4322,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/593838590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":593838590},{"type":"v1","addr":"172.21.15.195:6836","nonce":593838590}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4374,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/663402481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":663402481},{"type":"v1","addr":"172.21.15.38:6833","nonce":663402481}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4377,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/2614829035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":2614829035},{"type":"v1","addr":"172.21.15.38:6835","nonce":2614829035}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4402,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/1420805977","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":1420805977},{"type":"v1","addr":"172.21.15.195:6837","nonce":1420805977}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-09-21T03:33:32.864 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 3 2024-09-21T03:33:32.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:32.864+0000 7f2e45913640 1 -- 172.21.15.38:0/2030354558 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2e20078b90 msgr2=0x7f2e2007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:32.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:32.864+0000 7f2e45913640 1 --2- 172.21.15.38:0/2030354558 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2e20078b90 0x7f2e2007b0a0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f2e30004a30 tx=0x7f2e30001a50 comp rx=0 tx=0).stop 2024-09-21T03:33:32.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:32.864+0000 7f2e45913640 1 -- 172.21.15.38:0/2030354558 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2e400b4f80 msgr2=0x7f2e400b5390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:32.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:32.864+0000 7f2e45913640 1 --2- 172.21.15.38:0/2030354558 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2e400b4f80 0x7f2e400b5390 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2e38009950 tx=0x7f2e38033ce0 comp rx=0 tx=0).stop 2024-09-21T03:33:32.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:32.864+0000 7f2e45913640 1 -- 172.21.15.38:0/2030354558 shutdown_connections 2024-09-21T03:33:32.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:32.864+0000 7f2e45913640 1 --2- 172.21.15.38:0/2030354558 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2e400b4f80 0x7f2e400b5390 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:32.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:32.864+0000 7f2e45913640 1 --2- 172.21.15.38:0/2030354558 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2e4015cae0 0x7f2e401f6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:32.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:32.864+0000 7f2e45913640 1 --2- 172.21.15.38:0/2030354558 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2e20078b90 0x7f2e2007b0a0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:32.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:32.864+0000 7f2e45913640 1 --2- 172.21.15.38:0/2030354558 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2e400afbd0 0x7f2e400ae980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:32.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:32.864+0000 7f2e45913640 1 -- 172.21.15.38:0/2030354558 >> 172.21.15.38:0/2030354558 conn(0x7f2e40073560 msgr2=0x7f2e400712f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:32.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:32.865+0000 7f2e45913640 1 -- 172.21.15.38:0/2030354558 shutdown_connections 2024-09-21T03:33:32.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:32.865+0000 7f2e45913640 1 -- 172.21.15.38:0/2030354558 wait complete. 2024-09-21T03:33:32.877 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-21T03:33:32.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:32.993+0000 7f0d3934a640 1 -- 172.21.15.38:0/723743187 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0d340d0bd0 msgr2=0x7f0d340ccab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:32.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:32.993+0000 7f0d3934a640 1 --2- 172.21.15.38:0/723743187 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0d340d0bd0 0x7f0d340ccab0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f0d1c009830 tx=0x7f0d1c02a510 comp rx=0 tx=0).stop 2024-09-21T03:33:32.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:32.993+0000 7f0d3934a640 1 -- 172.21.15.38:0/723743187 shutdown_connections 2024-09-21T03:33:32.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:32.993+0000 7f0d3934a640 1 --2- 172.21.15.38:0/723743187 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0d340d0bd0 0x7f0d340ccab0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:32.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:32.993+0000 7f0d3934a640 1 --2- 172.21.15.38:0/723743187 >> v2:172.21.15.195:3300/0 conn(0x7f0d3415a350 0x7f0d340d0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:32.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:32.993+0000 7f0d3934a640 1 -- 172.21.15.38:0/723743187 >> 172.21.15.38:0/723743187 conn(0x7f0d340d2250 msgr2=0x7f0d340d2660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:32.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:32.993+0000 7f0d3934a640 1 -- 172.21.15.38:0/723743187 shutdown_connections 2024-09-21T03:33:32.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:32.993+0000 7f0d3934a640 1 -- 172.21.15.38:0/723743187 wait complete. 2024-09-21T03:33:32.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:32.995+0000 7f0d3934a640 1 Processor -- start 2024-09-21T03:33:32.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:32.995+0000 7f0d3934a640 1 -- start start 2024-09-21T03:33:33.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:32.997+0000 7f0d3934a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0d340ccff0 0x7f0d341f3030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:33.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:32.997+0000 7f0d3934a640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0d340d0bd0 0x7f0d34156480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:33.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:32.997+0000 7f0d3934a640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0d3415a350 0x7f0d3415bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:33.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:32.997+0000 7f0d3934a640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f0d34160a70 con 0x7f0d340ccff0 2024-09-21T03:33:33.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:32.997+0000 7f0d3934a640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f0d340b2c60 con 0x7f0d3415a350 2024-09-21T03:33:33.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:32.997+0000 7f0d3934a640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f0d34160bf0 con 0x7f0d340d0bd0 2024-09-21T03:33:33.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:32.997+0000 7f0d32ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0d340ccff0 0x7f0d341f3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:33.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:32.997+0000 7f0d32ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0d340ccff0 0x7f0d341f3030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:39372/0 (socket says 172.21.15.38:39372) 2024-09-21T03:33:33.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:32.997+0000 7f0d32ffd640 1 -- 172.21.15.38:0/952499122 learned_addr learned my addr 172.21.15.38:0/952499122 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:33:33.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:32.997+0000 7f0d327fc640 1 --2- 172.21.15.38:0/952499122 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0d340d0bd0 0x7f0d34156480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:33.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:32.997+0000 7f0d337fe640 1 --2- 172.21.15.38:0/952499122 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0d3415a350 0x7f0d3415bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:33.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:32.997+0000 7f0d32ffd640 1 -- 172.21.15.38:0/952499122 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0d340d0bd0 msgr2=0x7f0d34156480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:33.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:32.997+0000 7f0d32ffd640 1 --2- 172.21.15.38:0/952499122 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0d340d0bd0 0x7f0d34156480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:33.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:32.997+0000 7f0d32ffd640 1 -- 172.21.15.38:0/952499122 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0d3415a350 msgr2=0x7f0d3415bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:33.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:32.997+0000 7f0d32ffd640 1 --2- 172.21.15.38:0/952499122 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0d3415a350 0x7f0d3415bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:33.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:32.997+0000 7f0d32ffd640 1 -- 172.21.15.38:0/952499122 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d341f3a00 con 0x7f0d340ccff0 2024-09-21T03:33:33.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:32.997+0000 7f0d337fe640 1 --2- 172.21.15.38:0/952499122 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0d3415a350 0x7f0d3415bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:33:33.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:32.997+0000 7f0d327fc640 1 --2- 172.21.15.38:0/952499122 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0d340d0bd0 0x7f0d34156480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:33:33.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:32.997+0000 7f0d32ffd640 1 --2- 172.21.15.38:0/952499122 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0d340ccff0 0x7f0d341f3030 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f0d2400e790 tx=0x7f0d2400ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:33.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:32.997+0000 7f0d2b7fe640 1 -- 172.21.15.38:0/952499122 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d2400eea0 con 0x7f0d340ccff0 2024-09-21T03:33:33.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:32.998+0000 7f0d2b7fe640 1 -- 172.21.15.38:0/952499122 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0d2400cdb0 con 0x7f0d340ccff0 2024-09-21T03:33:33.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:32.998+0000 7f0d3934a640 1 -- 172.21.15.38:0/952499122 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d341f3ce0 con 0x7f0d340ccff0 2024-09-21T03:33:33.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:32.998+0000 7f0d2b7fe640 1 -- 172.21.15.38:0/952499122 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d24010690 con 0x7f0d340ccff0 2024-09-21T03:33:33.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:32.998+0000 7f0d3934a640 1 -- 172.21.15.38:0/952499122 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d341f42a0 con 0x7f0d340ccff0 2024-09-21T03:33:33.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:32.999+0000 7f0d3934a640 1 -- 172.21.15.38:0/952499122 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d34161110 con 0x7f0d340ccff0 2024-09-21T03:33:33.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:32.999+0000 7f0d2b7fe640 1 -- 172.21.15.38:0/952499122 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f0d24010830 con 0x7f0d340ccff0 2024-09-21T03:33:33.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:33.000+0000 7f0d2b7fe640 1 --2- 172.21.15.38:0/952499122 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0d14078bc0 0x7f0d1407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:33.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:33.000+0000 7f0d2b7fe640 1 -- 172.21.15.38:0/952499122 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4857+0+0 (secure 0 0 0) 0x7f0d2409b3a0 con 0x7f0d340ccff0 2024-09-21T03:33:33.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:33.000+0000 7f0d327fc640 1 --2- 172.21.15.38:0/952499122 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0d14078bc0 0x7f0d1407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:33.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:33.001+0000 7f0d327fc640 1 --2- 172.21.15.38:0/952499122 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0d14078bc0 0x7f0d1407b0d0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f0d1c034000 tx=0x7f0d1c004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:33.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:33.005+0000 7f0d2b7fe640 1 -- 172.21.15.38:0/952499122 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f0d24063b40 con 0x7f0d340ccff0 2024-09-21T03:33:33.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:33.193+0000 7f0d3934a640 1 -- 172.21.15.38:0/952499122 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f0d3415ce50 con 0x7f0d340ccff0 2024-09-21T03:33:33.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:33.193+0000 7f0d2b7fe640 1 -- 172.21.15.38:0/952499122 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v18) ==== 84+0+65 (secure 0 0 0) 0x7f0d24063440 con 0x7f0d340ccff0 2024-09-21T03:33:33.195 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:33:33.195 INFO:teuthology.orchestra.run.smithi038.stdout:[ 2024-09-21T03:33:33.195 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-21T03:33:33.195 INFO:teuthology.orchestra.run.smithi038.stdout: "poolnum": 1, 2024-09-21T03:33:33.195 INFO:teuthology.orchestra.run.smithi038.stdout: "poolname": ".mgr" 2024-09-21T03:33:33.196 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-21T03:33:33.196 INFO:teuthology.orchestra.run.smithi038.stdout:] 2024-09-21T03:33:33.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:33.195+0000 7f0d3934a640 1 -- 172.21.15.38:0/952499122 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0d14078bc0 msgr2=0x7f0d1407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:33.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:33.195+0000 7f0d3934a640 1 --2- 172.21.15.38:0/952499122 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0d14078bc0 0x7f0d1407b0d0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f0d1c034000 tx=0x7f0d1c004730 comp rx=0 tx=0).stop 2024-09-21T03:33:33.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:33.195+0000 7f0d3934a640 1 -- 172.21.15.38:0/952499122 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0d340ccff0 msgr2=0x7f0d341f3030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:33.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:33.195+0000 7f0d3934a640 1 --2- 172.21.15.38:0/952499122 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0d340ccff0 0x7f0d341f3030 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f0d2400e790 tx=0x7f0d2400ec50 comp rx=0 tx=0).stop 2024-09-21T03:33:33.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:33.195+0000 7f0d3934a640 1 -- 172.21.15.38:0/952499122 shutdown_connections 2024-09-21T03:33:33.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:33.195+0000 7f0d3934a640 1 --2- 172.21.15.38:0/952499122 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0d3415a350 0x7f0d3415bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:33.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:33.195+0000 7f0d3934a640 1 --2- 172.21.15.38:0/952499122 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0d14078bc0 0x7f0d1407b0d0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:33.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:33.195+0000 7f0d3934a640 1 --2- 172.21.15.38:0/952499122 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0d340d0bd0 0x7f0d34156480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:33.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:33.195+0000 7f0d3934a640 1 --2- 172.21.15.38:0/952499122 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0d340ccff0 0x7f0d341f3030 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:33.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:33.195+0000 7f0d3934a640 1 -- 172.21.15.38:0/952499122 >> 172.21.15.38:0/952499122 conn(0x7f0d340d2250 msgr2=0x7f0d3415f980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:33.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:33.195+0000 7f0d3934a640 1 -- 172.21.15.38:0/952499122 shutdown_connections 2024-09-21T03:33:33.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:33.195+0000 7f0d3934a640 1 -- 172.21.15.38:0/952499122 wait complete. 2024-09-21T03:33:33.208 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-21T03:33:33.208 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-21T03:33:33.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:33.327+0000 7f6d7b269640 1 -- 172.21.15.38:0/518455896 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6d7415c650 msgr2=0x7f6d741627b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:33.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:33.327+0000 7f6d7b269640 1 --2- 172.21.15.38:0/518455896 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6d7415c650 0x7f6d741627b0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f6d68009830 tx=0x7f6d6802a3f0 comp rx=0 tx=0).stop 2024-09-21T03:33:33.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:33.327+0000 7f6d7b269640 1 -- 172.21.15.38:0/518455896 shutdown_connections 2024-09-21T03:33:33.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:33.327+0000 7f6d7b269640 1 --2- 172.21.15.38:0/518455896 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6d74162cf0 0x7f6d740b3000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:33.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:33.327+0000 7f6d7b269640 1 --2- 172.21.15.38:0/518455896 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6d7415c650 0x7f6d741627b0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:33.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:33.327+0000 7f6d7b269640 1 -- 172.21.15.38:0/518455896 >> 172.21.15.38:0/518455896 conn(0x7f6d7406cde0 msgr2=0x7f6d7406d1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:33.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:33.327+0000 7f6d7b269640 1 -- 172.21.15.38:0/518455896 shutdown_connections 2024-09-21T03:33:33.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:33.327+0000 7f6d7b269640 1 -- 172.21.15.38:0/518455896 wait complete. 2024-09-21T03:33:33.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:33.329+0000 7f6d7b269640 1 Processor -- start 2024-09-21T03:33:33.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:33.329+0000 7f6d7b269640 1 -- start start 2024-09-21T03:33:33.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:33.330+0000 7f6d7b269640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6d7415c650 0x7f6d741e8560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:33.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:33.330+0000 7f6d7b269640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6d74162cf0 0x7f6d74154f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:33.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:33.330+0000 7f6d7b269640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6d74163350 0x7f6d741f60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:33.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:33.330+0000 7f6d7b269640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f6d740b3a10 con 0x7f6d74163350 2024-09-21T03:33:33.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:33.330+0000 7f6d7b269640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f6d740b3890 con 0x7f6d7415c650 2024-09-21T03:33:33.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:33.330+0000 7f6d7b269640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f6d740b3b90 con 0x7f6d74162cf0 2024-09-21T03:33:33.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:33.330+0000 7f6d79806640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6d74163350 0x7f6d741f60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:33.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:33.330+0000 7f6d79806640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6d74163350 0x7f6d741f60e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:39388/0 (socket says 172.21.15.38:39388) 2024-09-21T03:33:33.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:33.330+0000 7f6d79806640 1 -- 172.21.15.38:0/1637499511 learned_addr learned my addr 172.21.15.38:0/1637499511 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:33:33.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:33.330+0000 7f6d78804640 1 --2- 172.21.15.38:0/1637499511 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6d74162cf0 0x7f6d74154f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:33.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:33.330+0000 7f6d79005640 1 --2- 172.21.15.38:0/1637499511 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6d7415c650 0x7f6d741e8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:33.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:33.331+0000 7f6d79806640 1 -- 172.21.15.38:0/1637499511 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6d74162cf0 msgr2=0x7f6d74154f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:33.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:33.331+0000 7f6d79806640 1 --2- 172.21.15.38:0/1637499511 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6d74162cf0 0x7f6d74154f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:33.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:33.331+0000 7f6d79806640 1 -- 172.21.15.38:0/1637499511 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6d7415c650 msgr2=0x7f6d741e8560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:33.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:33.331+0000 7f6d79806640 1 --2- 172.21.15.38:0/1637499511 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6d7415c650 0x7f6d741e8560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:33.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:33.331+0000 7f6d79806640 1 -- 172.21.15.38:0/1637499511 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d740b88c0 con 0x7f6d74163350 2024-09-21T03:33:33.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:33.331+0000 7f6d79005640 1 --2- 172.21.15.38:0/1637499511 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6d7415c650 0x7f6d741e8560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:33:33.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:33.331+0000 7f6d79806640 1 --2- 172.21.15.38:0/1637499511 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6d74163350 0x7f6d741f60e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f6d6400e9c0 tx=0x7f6d6400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:33.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:33.331+0000 7f6d71ffb640 1 -- 172.21.15.38:0/1637499511 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d64004d20 con 0x7f6d74163350 2024-09-21T03:33:33.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:33.331+0000 7f6d71ffb640 1 -- 172.21.15.38:0/1637499511 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6d6400cb20 con 0x7f6d74163350 2024-09-21T03:33:33.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:33.331+0000 7f6d7b269640 1 -- 172.21.15.38:0/1637499511 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d741f6620 con 0x7f6d74163350 2024-09-21T03:33:33.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:33.331+0000 7f6d71ffb640 1 -- 172.21.15.38:0/1637499511 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d64005020 con 0x7f6d74163350 2024-09-21T03:33:33.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:33.331+0000 7f6d7b269640 1 -- 172.21.15.38:0/1637499511 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d740b8e40 con 0x7f6d74163350 2024-09-21T03:33:33.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:33.332+0000 7f6d7b269640 1 -- 172.21.15.38:0/1637499511 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d741627b0 con 0x7f6d74163350 2024-09-21T03:33:33.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:33.333+0000 7f6d71ffb640 1 -- 172.21.15.38:0/1637499511 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f6d640051c0 con 0x7f6d74163350 2024-09-21T03:33:33.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:33.334+0000 7f6d71ffb640 1 --2- 172.21.15.38:0/1637499511 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6d60078bc0 0x7f6d6007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:33.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:33.334+0000 7f6d71ffb640 1 -- 172.21.15.38:0/1637499511 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4857+0+0 (secure 0 0 0) 0x7f6d6409b280 con 0x7f6d74163350 2024-09-21T03:33:33.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:33.334+0000 7f6d79005640 1 --2- 172.21.15.38:0/1637499511 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6d60078bc0 0x7f6d6007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:33.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:33.335+0000 7f6d79005640 1 --2- 172.21.15.38:0/1637499511 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6d60078bc0 0x7f6d6007b0d0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c007a50 tx=0x7f6d6c009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:33.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:33.338+0000 7f6d71ffb640 1 -- 172.21.15.38:0/1637499511 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f6d741f6620 con 0x7f6d74163350 2024-09-21T03:33:33.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:33.526+0000 7f6d7b269640 1 -- 172.21.15.38:0/1637499511 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f6d7415d470 con 0x7f6d74163350 2024-09-21T03:33:34.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:34.225+0000 7f6d71ffb640 1 -- 172.21.15.38:0/1637499511 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v19) ==== 138+0+0 (secure 0 0 0) 0x7f6d740b88c0 con 0x7f6d74163350 2024-09-21T03:33:34.227 INFO:teuthology.orchestra.run.smithi038.stderr:pool 'cephfs_metadata' created 2024-09-21T03:33:34.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:34.231+0000 7f6d7b269640 1 -- 172.21.15.38:0/1637499511 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6d60078bc0 msgr2=0x7f6d6007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:34.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:34.231+0000 7f6d7b269640 1 --2- 172.21.15.38:0/1637499511 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6d60078bc0 0x7f6d6007b0d0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c007a50 tx=0x7f6d6c009040 comp rx=0 tx=0).stop 2024-09-21T03:33:34.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:34.231+0000 7f6d7b269640 1 -- 172.21.15.38:0/1637499511 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6d74163350 msgr2=0x7f6d741f60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:34.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:34.231+0000 7f6d7b269640 1 --2- 172.21.15.38:0/1637499511 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6d74163350 0x7f6d741f60e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f6d6400e9c0 tx=0x7f6d6400ee80 comp rx=0 tx=0).stop 2024-09-21T03:33:34.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:34.231+0000 7f6d7b269640 1 -- 172.21.15.38:0/1637499511 shutdown_connections 2024-09-21T03:33:34.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:34.231+0000 7f6d7b269640 1 --2- 172.21.15.38:0/1637499511 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6d74163350 0x7f6d741f60e0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:34.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:34.231+0000 7f6d7b269640 1 --2- 172.21.15.38:0/1637499511 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6d60078bc0 0x7f6d6007b0d0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:34.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:34.231+0000 7f6d7b269640 1 --2- 172.21.15.38:0/1637499511 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6d74162cf0 0x7f6d74154f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:34.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:34.231+0000 7f6d7b269640 1 --2- 172.21.15.38:0/1637499511 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6d7415c650 0x7f6d741e8560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:34.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:34.231+0000 7f6d7b269640 1 -- 172.21.15.38:0/1637499511 >> 172.21.15.38:0/1637499511 conn(0x7f6d7406cde0 msgr2=0x7f6d740cfdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:34.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:34.232+0000 7f6d7b269640 1 -- 172.21.15.38:0/1637499511 shutdown_connections 2024-09-21T03:33:34.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:34.233+0000 7f6d7b269640 1 -- 172.21.15.38:0/1637499511 wait complete. 2024-09-21T03:33:34.250 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-09-21T03:33:34.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:34.379+0000 7f6fbd2f2640 1 -- 172.21.15.38:0/3758820509 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6fb806f290 msgr2=0x7f6fb80b40c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:34.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:34.379+0000 7f6fbd2f2640 1 --2- 172.21.15.38:0/3758820509 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6fb806f290 0x7f6fb80b40c0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6fb0005fd0 tx=0x7f6fb002a970 comp rx=0 tx=0).stop 2024-09-21T03:33:34.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:34.379+0000 7f6fbd2f2640 1 -- 172.21.15.38:0/3758820509 shutdown_connections 2024-09-21T03:33:34.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:34.379+0000 7f6fbd2f2640 1 --2- 172.21.15.38:0/3758820509 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6fb806f290 0x7f6fb80b40c0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:34.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:34.379+0000 7f6fbd2f2640 1 --2- 172.21.15.38:0/3758820509 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6fb806e940 0x7f6fb806ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:34.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:34.379+0000 7f6fbd2f2640 1 -- 172.21.15.38:0/3758820509 >> 172.21.15.38:0/3758820509 conn(0x7f6fb8073e30 msgr2=0x7f6fb8074240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:34.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:34.379+0000 7f6fbd2f2640 1 -- 172.21.15.38:0/3758820509 shutdown_connections 2024-09-21T03:33:34.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:34.379+0000 7f6fbd2f2640 1 -- 172.21.15.38:0/3758820509 wait complete. 2024-09-21T03:33:34.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:34.381+0000 7f6fbd2f2640 1 Processor -- start 2024-09-21T03:33:34.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:34.381+0000 7f6fbd2f2640 1 -- start start 2024-09-21T03:33:34.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:34.383+0000 7f6fbd2f2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6fb806e940 0x7f6fb81e58a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:34.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:34.383+0000 7f6fbd2f2640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6fb806f290 0x7f6fb815a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:34.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:34.383+0000 7f6fbd2f2640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6fb80cee70 0x7f6fb81f6860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:34.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:34.383+0000 7f6fbd2f2640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f6fb80b4a10 con 0x7f6fb806e940 2024-09-21T03:33:34.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:34.383+0000 7f6fbd2f2640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f6fb80b4890 con 0x7f6fb806f290 2024-09-21T03:33:34.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:34.383+0000 7f6fbd2f2640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f6fb80b4b90 con 0x7f6fb80cee70 2024-09-21T03:33:34.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:34.383+0000 7f6fb6ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6fb806e940 0x7f6fb81e58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:34.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:34.383+0000 7f6fb6ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6fb806e940 0x7f6fb81e58a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:39400/0 (socket says 172.21.15.38:39400) 2024-09-21T03:33:34.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:34.383+0000 7f6fb6ffd640 1 -- 172.21.15.38:0/1522522811 learned_addr learned my addr 172.21.15.38:0/1522522811 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:33:34.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:34.383+0000 7f6fb77fe640 1 --2- 172.21.15.38:0/1522522811 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6fb80cee70 0x7f6fb81f6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:34.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:34.383+0000 7f6fb67fc640 1 --2- 172.21.15.38:0/1522522811 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6fb806f290 0x7f6fb815a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:34.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:34.383+0000 7f6fb6ffd640 1 -- 172.21.15.38:0/1522522811 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6fb80cee70 msgr2=0x7f6fb81f6860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:34.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:34.383+0000 7f6fb6ffd640 1 --2- 172.21.15.38:0/1522522811 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6fb80cee70 0x7f6fb81f6860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:34.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:34.383+0000 7f6fb6ffd640 1 -- 172.21.15.38:0/1522522811 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6fb806f290 msgr2=0x7f6fb815a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:34.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:34.383+0000 7f6fb6ffd640 1 --2- 172.21.15.38:0/1522522811 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6fb806f290 0x7f6fb815a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:34.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:34.383+0000 7f6fb6ffd640 1 -- 172.21.15.38:0/1522522811 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fb8154230 con 0x7f6fb806e940 2024-09-21T03:33:34.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:34.383+0000 7f6fb6ffd640 1 --2- 172.21.15.38:0/1522522811 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6fb806e940 0x7f6fb81e58a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6fa800b3f0 tx=0x7f6fa800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:34.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:34.383+0000 7f6f9f7fe640 1 -- 172.21.15.38:0/1522522811 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fa8015020 con 0x7f6fb806e940 2024-09-21T03:33:34.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:34.383+0000 7f6f9f7fe640 1 -- 172.21.15.38:0/1522522811 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6fa8011920 con 0x7f6fb806e940 2024-09-21T03:33:34.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:34.383+0000 7f6fbd2f2640 1 -- 172.21.15.38:0/1522522811 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fb8154510 con 0x7f6fb806e940 2024-09-21T03:33:34.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:34.383+0000 7f6f9f7fe640 1 -- 172.21.15.38:0/1522522811 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fa8011d70 con 0x7f6fb806e940 2024-09-21T03:33:34.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:34.383+0000 7f6fbd2f2640 1 -- 172.21.15.38:0/1522522811 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fb8154ad0 con 0x7f6fb806e940 2024-09-21T03:33:34.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:34.385+0000 7f6fbd2f2640 1 -- 172.21.15.38:0/1522522811 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6fb80b5040 con 0x7f6fb806e940 2024-09-21T03:33:34.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:34.385+0000 7f6f9f7fe640 1 -- 172.21.15.38:0/1522522811 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f6fa8018020 con 0x7f6fb806e940 2024-09-21T03:33:34.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:34.386+0000 7f6f9f7fe640 1 --2- 172.21.15.38:0/1522522811 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6fa4078b90 0x7f6fa407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:34.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:34.386+0000 7f6f9f7fe640 1 -- 172.21.15.38:0/1522522811 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5227+0+0 (secure 0 0 0) 0x7f6fa809a700 con 0x7f6fb806e940 2024-09-21T03:33:34.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:34.386+0000 7f6fb67fc640 1 --2- 172.21.15.38:0/1522522811 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6fa4078b90 0x7f6fa407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:34.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:34.387+0000 7f6fb67fc640 1 --2- 172.21.15.38:0/1522522811 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6fa4078b90 0x7f6fa407b0a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f6fb8158040 tx=0x7f6fac005d80 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:34.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:34.391+0000 7f6f9f7fe640 1 -- 172.21.15.38:0/1522522811 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f6fa801f030 con 0x7f6fb806e940 2024-09-21T03:33:34.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:34.581+0000 7f6fbd2f2640 1 -- 172.21.15.38:0/1522522811 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7f6fb80b5850 con 0x7f6fb806e940 2024-09-21T03:33:35.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:35.231+0000 7f6f9f7fe640 1 -- 172.21.15.38:0/1522522811 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v20) ==== 171+0+0 (secure 0 0 0) 0x7f6fa8062e40 con 0x7f6fb806e940 2024-09-21T03:33:35.233 INFO:teuthology.orchestra.run.smithi038.stderr:pool 'cephfs_data' created 2024-09-21T03:33:35.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:35.234+0000 7f6fbd2f2640 1 -- 172.21.15.38:0/1522522811 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6fa4078b90 msgr2=0x7f6fa407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:35.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:35.234+0000 7f6fbd2f2640 1 --2- 172.21.15.38:0/1522522811 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6fa4078b90 0x7f6fa407b0a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f6fb8158040 tx=0x7f6fac005d80 comp rx=0 tx=0).stop 2024-09-21T03:33:35.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:35.234+0000 7f6fbd2f2640 1 -- 172.21.15.38:0/1522522811 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6fb806e940 msgr2=0x7f6fb81e58a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:35.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:35.234+0000 7f6fbd2f2640 1 --2- 172.21.15.38:0/1522522811 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6fb806e940 0x7f6fb81e58a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6fa800b3f0 tx=0x7f6fa800b8b0 comp rx=0 tx=0).stop 2024-09-21T03:33:35.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:35.234+0000 7f6fbd2f2640 1 -- 172.21.15.38:0/1522522811 shutdown_connections 2024-09-21T03:33:35.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:35.234+0000 7f6fbd2f2640 1 --2- 172.21.15.38:0/1522522811 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6fa4078b90 0x7f6fa407b0a0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:35.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:35.234+0000 7f6fbd2f2640 1 --2- 172.21.15.38:0/1522522811 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6fb80cee70 0x7f6fb81f6860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:35.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:35.234+0000 7f6fbd2f2640 1 --2- 172.21.15.38:0/1522522811 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6fb806f290 0x7f6fb815a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:35.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:35.234+0000 7f6fbd2f2640 1 --2- 172.21.15.38:0/1522522811 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6fb806e940 0x7f6fb81e58a0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:35.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:35.234+0000 7f6fbd2f2640 1 -- 172.21.15.38:0/1522522811 >> 172.21.15.38:0/1522522811 conn(0x7f6fb8073e30 msgr2=0x7f6fb8074240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:35.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:35.235+0000 7f6fbd2f2640 1 -- 172.21.15.38:0/1522522811 shutdown_connections 2024-09-21T03:33:35.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:35.235+0000 7f6fbd2f2640 1 -- 172.21.15.38:0/1522522811 wait complete. 2024-09-21T03:33:35.260 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-21T03:33:35.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:35.385+0000 7f1d64c3f640 1 -- 172.21.15.38:0/2938887025 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1d600d0170 msgr2=0x7f1d600d0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:35.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:35.385+0000 7f1d64c3f640 1 --2- 172.21.15.38:0/2938887025 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1d600d0170 0x7f1d600d0be0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f1d48009820 tx=0x7f1d4802a410 comp rx=0 tx=0).stop 2024-09-21T03:33:35.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:35.385+0000 7f1d64c3f640 1 -- 172.21.15.38:0/2938887025 shutdown_connections 2024-09-21T03:33:35.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:35.385+0000 7f1d64c3f640 1 --2- 172.21.15.38:0/2938887025 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1d600d0170 0x7f1d600d0be0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:35.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:35.385+0000 7f1d64c3f640 1 -- 172.21.15.38:0/2938887025 >> 172.21.15.38:0/2938887025 conn(0x7f1d6009d830 msgr2=0x7f1d600b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:35.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:35.385+0000 7f1d64c3f640 1 -- 172.21.15.38:0/2938887025 shutdown_connections 2024-09-21T03:33:35.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:35.385+0000 7f1d64c3f640 1 -- 172.21.15.38:0/2938887025 wait complete. 2024-09-21T03:33:35.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:35.387+0000 7f1d64c3f640 1 Processor -- start 2024-09-21T03:33:35.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:35.387+0000 7f1d64c3f640 1 -- start start 2024-09-21T03:33:35.391 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:35.389+0000 7f1d64c3f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1d600d0170 0x7f1d600b2780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:35.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:35.389+0000 7f1d64c3f640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1d600d1120 0x7f1d600b2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:35.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:35.389+0000 7f1d64c3f640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1d6015ee80 0x7f1d6006ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:35.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:35.389+0000 7f1d64c3f640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f1d600b6910 con 0x7f1d600d0170 2024-09-21T03:33:35.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:35.389+0000 7f1d64c3f640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f1d600b6790 con 0x7f1d6015ee80 2024-09-21T03:33:35.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:35.389+0000 7f1d64c3f640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f1d600b6a90 con 0x7f1d600d1120 2024-09-21T03:33:35.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:35.389+0000 7f1d5e59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1d600d0170 0x7f1d600b2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:35.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:35.389+0000 7f1d5e59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1d600d0170 0x7f1d600b2780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:39406/0 (socket says 172.21.15.38:39406) 2024-09-21T03:33:35.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:35.389+0000 7f1d5e59c640 1 -- 172.21.15.38:0/2428139183 learned_addr learned my addr 172.21.15.38:0/2428139183 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:33:35.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:35.389+0000 7f1d5dd9b640 1 --2- 172.21.15.38:0/2428139183 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1d600d1120 0x7f1d600b2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:35.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:35.389+0000 7f1d5ed9d640 1 --2- 172.21.15.38:0/2428139183 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1d6015ee80 0x7f1d6006ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:35.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:35.389+0000 7f1d5e59c640 1 -- 172.21.15.38:0/2428139183 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1d600d1120 msgr2=0x7f1d600b2cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:35.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:35.389+0000 7f1d5e59c640 1 --2- 172.21.15.38:0/2428139183 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1d600d1120 0x7f1d600b2cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:35.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:35.389+0000 7f1d5e59c640 1 -- 172.21.15.38:0/2428139183 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1d6015ee80 msgr2=0x7f1d6006ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:35.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:35.389+0000 7f1d5e59c640 1 --2- 172.21.15.38:0/2428139183 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1d6015ee80 0x7f1d6006ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:35.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:35.389+0000 7f1d5e59c640 1 -- 172.21.15.38:0/2428139183 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d6006e360 con 0x7f1d600d0170 2024-09-21T03:33:35.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:35.389+0000 7f1d5e59c640 1 --2- 172.21.15.38:0/2428139183 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1d600d0170 0x7f1d600b2780 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1d5800cc70 tx=0x7f1d5800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:35.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:35.390+0000 7f1d56ffd640 1 -- 172.21.15.38:0/2428139183 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d58016020 con 0x7f1d600d0170 2024-09-21T03:33:35.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:35.390+0000 7f1d56ffd640 1 -- 172.21.15.38:0/2428139183 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1d58012420 con 0x7f1d600d0170 2024-09-21T03:33:35.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:35.390+0000 7f1d64c3f640 1 -- 172.21.15.38:0/2428139183 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d6015c230 con 0x7f1d600d0170 2024-09-21T03:33:35.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:35.390+0000 7f1d56ffd640 1 -- 172.21.15.38:0/2428139183 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d58012810 con 0x7f1d600d0170 2024-09-21T03:33:35.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:35.390+0000 7f1d64c3f640 1 -- 172.21.15.38:0/2428139183 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d6006e620 con 0x7f1d600d0170 2024-09-21T03:33:35.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:35.391+0000 7f1d64c3f640 1 -- 172.21.15.38:0/2428139183 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d6015bd50 con 0x7f1d600d0170 2024-09-21T03:33:35.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:35.392+0000 7f1d56ffd640 1 -- 172.21.15.38:0/2428139183 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f1d58019020 con 0x7f1d600d0170 2024-09-21T03:33:35.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:35.392+0000 7f1d56ffd640 1 --2- 172.21.15.38:0/2428139183 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1d40078b90 0x7f1d4007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:35.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:35.392+0000 7f1d56ffd640 1 -- 172.21.15.38:0/2428139183 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5609+0+0 (secure 0 0 0) 0x7f1d5809afb0 con 0x7f1d600d0170 2024-09-21T03:33:35.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:35.393+0000 7f1d5dd9b640 1 --2- 172.21.15.38:0/2428139183 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1d40078b90 0x7f1d4007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:35.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:35.393+0000 7f1d5dd9b640 1 --2- 172.21.15.38:0/2428139183 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1d40078b90 0x7f1d4007b0a0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f1d48034000 tx=0x7f1d48004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:35.399 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:35.397+0000 7f1d56ffd640 1 -- 172.21.15.38:0/2428139183 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f1d6015bd50 con 0x7f1d600d0170 2024-09-21T03:33:35.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:35.621+0000 7f1d64c3f640 1 -- 172.21.15.38:0/2428139183 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f1d600d0be0 con 0x7f1d600d0170 2024-09-21T03:33:36.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:36.233+0000 7f1d56ffd640 1 -- 172.21.15.38:0/2428139183 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) ==== 172+0+0 (secure 0 0 0) 0x7f1d58020030 con 0x7f1d600d0170 2024-09-21T03:33:36.235 INFO:teuthology.orchestra.run.smithi038.stderr:new fs with metadata pool 2 and data pool 3 2024-09-21T03:33:36.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:36.237+0000 7f1d64c3f640 1 -- 172.21.15.38:0/2428139183 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1d40078b90 msgr2=0x7f1d4007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:36.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:36.237+0000 7f1d64c3f640 1 --2- 172.21.15.38:0/2428139183 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1d40078b90 0x7f1d4007b0a0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f1d48034000 tx=0x7f1d48004730 comp rx=0 tx=0).stop 2024-09-21T03:33:36.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:36.237+0000 7f1d64c3f640 1 -- 172.21.15.38:0/2428139183 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1d600d0170 msgr2=0x7f1d600b2780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:36.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:36.237+0000 7f1d64c3f640 1 --2- 172.21.15.38:0/2428139183 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1d600d0170 0x7f1d600b2780 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1d5800cc70 tx=0x7f1d5800a430 comp rx=0 tx=0).stop 2024-09-21T03:33:36.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:36.237+0000 7f1d64c3f640 1 -- 172.21.15.38:0/2428139183 shutdown_connections 2024-09-21T03:33:36.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:36.237+0000 7f1d64c3f640 1 --2- 172.21.15.38:0/2428139183 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1d6015ee80 0x7f1d6006ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:36.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:36.237+0000 7f1d64c3f640 1 --2- 172.21.15.38:0/2428139183 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1d40078b90 0x7f1d4007b0a0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:36.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:36.237+0000 7f1d64c3f640 1 --2- 172.21.15.38:0/2428139183 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1d600d1120 0x7f1d600b2cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:36.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:36.237+0000 7f1d64c3f640 1 --2- 172.21.15.38:0/2428139183 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1d600d0170 0x7f1d600b2780 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:36.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:36.238+0000 7f1d64c3f640 1 -- 172.21.15.38:0/2428139183 >> 172.21.15.38:0/2428139183 conn(0x7f1d6009d830 msgr2=0x7f1d6015d320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:36.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:36.238+0000 7f1d64c3f640 1 -- 172.21.15.38:0/2428139183 shutdown_connections 2024-09-21T03:33:36.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:36.238+0000 7f1d64c3f640 1 -- 172.21.15.38:0/2428139183 wait complete. 2024-09-21T03:33:36.273 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-09-21T03:33:36.273 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-09-21T03:33:36.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:36.400+0000 7fb8e74c5640 1 -- 172.21.15.38:0/3136267795 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb8e0162ec0 msgr2=0x7fb8e00b3000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:36.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:36.400+0000 7fb8e74c5640 1 --2- 172.21.15.38:0/3136267795 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb8e0162ec0 0x7fb8e00b3000 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb8e006dc80 tx=0x7fb8d802a440 comp rx=0 tx=0).stop 2024-09-21T03:33:36.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:36.400+0000 7fb8e74c5640 1 -- 172.21.15.38:0/3136267795 shutdown_connections 2024-09-21T03:33:36.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:36.400+0000 7fb8e74c5640 1 --2- 172.21.15.38:0/3136267795 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb8e0162ec0 0x7fb8e00b3000 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:36.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:36.400+0000 7fb8e74c5640 1 --2- 172.21.15.38:0/3136267795 >> v2:172.21.15.195:3300/0 conn(0x7fb8e0163350 0x7fb8e015c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:36.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:36.400+0000 7fb8e74c5640 1 -- 172.21.15.38:0/3136267795 >> 172.21.15.38:0/3136267795 conn(0x7fb8e006cde0 msgr2=0x7fb8e006d1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:36.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:36.400+0000 7fb8e74c5640 1 -- 172.21.15.38:0/3136267795 shutdown_connections 2024-09-21T03:33:36.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:36.400+0000 7fb8e74c5640 1 -- 172.21.15.38:0/3136267795 wait complete. 2024-09-21T03:33:36.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:36.402+0000 7fb8e74c5640 1 Processor -- start 2024-09-21T03:33:36.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:36.402+0000 7fb8e74c5640 1 -- start start 2024-09-21T03:33:36.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:36.403+0000 7fb8e74c5640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb8e00d0e40 0x7fb8e00c72b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:36.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:36.403+0000 7fb8e74c5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb8e0163350 0x7fb8e015de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:36.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:36.403+0000 7fb8e74c5640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb8e01f9110 0x7fb8e015e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:36.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:36.403+0000 7fb8e74c5640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fb8e00b39b0 con 0x7fb8e0163350 2024-09-21T03:33:36.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:36.403+0000 7fb8e74c5640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fb8e00b3830 con 0x7fb8e00d0e40 2024-09-21T03:33:36.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:36.403+0000 7fb8e74c5640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fb8e00b3b30 con 0x7fb8e01f9110 2024-09-21T03:33:36.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:36.403+0000 7fb8e4a60640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb8e0163350 0x7fb8e015de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:36.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:36.403+0000 7fb8e4a60640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb8e0163350 0x7fb8e015de90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:39428/0 (socket says 172.21.15.38:39428) 2024-09-21T03:33:36.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:36.403+0000 7fb8e4a60640 1 -- 172.21.15.38:0/900274276 learned_addr learned my addr 172.21.15.38:0/900274276 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:33:36.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:36.404+0000 7fb8e5261640 1 --2- 172.21.15.38:0/900274276 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb8e00d0e40 0x7fb8e00c72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:36.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:36.404+0000 7fb8e5a62640 1 --2- 172.21.15.38:0/900274276 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb8e01f9110 0x7fb8e015e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:36.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:36.404+0000 7fb8e4a60640 1 -- 172.21.15.38:0/900274276 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb8e01f9110 msgr2=0x7fb8e015e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:36.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:36.404+0000 7fb8e4a60640 1 --2- 172.21.15.38:0/900274276 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb8e01f9110 0x7fb8e015e3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:36.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:36.404+0000 7fb8e4a60640 1 -- 172.21.15.38:0/900274276 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb8e00d0e40 msgr2=0x7fb8e00c72b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:36.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:36.404+0000 7fb8e4a60640 1 --2- 172.21.15.38:0/900274276 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb8e00d0e40 0x7fb8e00c72b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:36.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:36.404+0000 7fb8e4a60640 1 -- 172.21.15.38:0/900274276 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8e00c82b0 con 0x7fb8e0163350 2024-09-21T03:33:36.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:36.404+0000 7fb8e5261640 1 --2- 172.21.15.38:0/900274276 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb8e00d0e40 0x7fb8e00c72b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:33:36.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:36.404+0000 7fb8e5a62640 1 --2- 172.21.15.38:0/900274276 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb8e01f9110 0x7fb8e015e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:33:36.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:36.404+0000 7fb8e4a60640 1 --2- 172.21.15.38:0/900274276 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb8e0163350 0x7fb8e015de90 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb8d400b3c0 tx=0x7fb8d400b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:36.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:36.404+0000 7fb8ddffb640 1 -- 172.21.15.38:0/900274276 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8d4011830 con 0x7fb8e0163350 2024-09-21T03:33:36.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:36.404+0000 7fb8ddffb640 1 -- 172.21.15.38:0/900274276 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb8d4016070 con 0x7fb8e0163350 2024-09-21T03:33:36.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:36.404+0000 7fb8e74c5640 1 -- 172.21.15.38:0/900274276 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8e01594a0 con 0x7fb8e0163350 2024-09-21T03:33:36.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:36.404+0000 7fb8ddffb640 1 -- 172.21.15.38:0/900274276 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8d4011da0 con 0x7fb8e0163350 2024-09-21T03:33:36.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:36.404+0000 7fb8e74c5640 1 -- 172.21.15.38:0/900274276 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8e0159a60 con 0x7fb8e0163350 2024-09-21T03:33:36.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:36.405+0000 7fb8e74c5640 1 -- 172.21.15.38:0/900274276 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8e00c8020 con 0x7fb8e0163350 2024-09-21T03:33:36.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:36.406+0000 7fb8ddffb640 1 -- 172.21.15.38:0/900274276 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb8d401b460 con 0x7fb8e0163350 2024-09-21T03:33:36.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:36.407+0000 7fb8ddffb640 1 --2- 172.21.15.38:0/900274276 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb8cc078bc0 0x7fb8cc07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:36.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:36.407+0000 7fb8ddffb640 1 -- 172.21.15.38:0/900274276 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fb8d409aa50 con 0x7fb8e0163350 2024-09-21T03:33:36.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:36.407+0000 7fb8e5261640 1 --2- 172.21.15.38:0/900274276 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb8cc078bc0 0x7fb8cc07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:36.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:36.407+0000 7fb8e5261640 1 --2- 172.21.15.38:0/900274276 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb8cc078bc0 0x7fb8cc07b0d0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fb8d0005fd0 tx=0x7fb8d0009340 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:36.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:36.411+0000 7fb8ddffb640 1 -- 172.21.15.38:0/900274276 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fb8d4062df0 con 0x7fb8e0163350 2024-09-21T03:33:36.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:36.606+0000 7fb8e74c5640 1 -- 172.21.15.38:0/900274276 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7fb8e00b3000 con 0x7fb8e0163350 2024-09-21T03:33:37.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:37.250+0000 7fb8ddffb640 1 -- 172.21.15.38:0/900274276 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v22) ==== 157+0+0 (secure 0 0 0) 0x7fb8d40626f0 con 0x7fb8e0163350 2024-09-21T03:33:37.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:37.253+0000 7fb8e74c5640 1 -- 172.21.15.38:0/900274276 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb8cc078bc0 msgr2=0x7fb8cc07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:37.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:37.253+0000 7fb8e74c5640 1 --2- 172.21.15.38:0/900274276 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb8cc078bc0 0x7fb8cc07b0d0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fb8d0005fd0 tx=0x7fb8d0009340 comp rx=0 tx=0).stop 2024-09-21T03:33:37.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:37.253+0000 7fb8e74c5640 1 -- 172.21.15.38:0/900274276 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb8e0163350 msgr2=0x7fb8e015de90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:37.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:37.253+0000 7fb8e74c5640 1 --2- 172.21.15.38:0/900274276 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb8e0163350 0x7fb8e015de90 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb8d400b3c0 tx=0x7fb8d400b880 comp rx=0 tx=0).stop 2024-09-21T03:33:37.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:37.253+0000 7fb8e74c5640 1 -- 172.21.15.38:0/900274276 shutdown_connections 2024-09-21T03:33:37.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:37.253+0000 7fb8e74c5640 1 --2- 172.21.15.38:0/900274276 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb8cc078bc0 0x7fb8cc07b0d0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:37.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:37.253+0000 7fb8e74c5640 1 --2- 172.21.15.38:0/900274276 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb8e01f9110 0x7fb8e015e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:37.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:37.253+0000 7fb8e74c5640 1 --2- 172.21.15.38:0/900274276 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb8e0163350 0x7fb8e015de90 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:37.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:37.253+0000 7fb8e74c5640 1 --2- 172.21.15.38:0/900274276 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb8e00d0e40 0x7fb8e00c72b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:37.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:37.253+0000 7fb8e74c5640 1 -- 172.21.15.38:0/900274276 >> 172.21.15.38:0/900274276 conn(0x7fb8e006cde0 msgr2=0x7fb8e01628c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:37.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:37.253+0000 7fb8e74c5640 1 -- 172.21.15.38:0/900274276 shutdown_connections 2024-09-21T03:33:37.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:37.253+0000 7fb8e74c5640 1 -- 172.21.15.38:0/900274276 wait complete. 2024-09-21T03:33:37.271 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-09-21T03:33:37.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:37.401+0000 7fcf8c9e6640 1 -- 172.21.15.38:0/318790507 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcf88162cf0 msgr2=0x7fcf880b3000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:37.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:37.401+0000 7fcf8c9e6640 1 --2- 172.21.15.38:0/318790507 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcf88162cf0 0x7fcf880b3000 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fcf880d2120 tx=0x7fcf8002a930 comp rx=0 tx=0).stop 2024-09-21T03:33:37.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:37.401+0000 7fcf8c9e6640 1 -- 172.21.15.38:0/318790507 shutdown_connections 2024-09-21T03:33:37.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:37.401+0000 7fcf8c9e6640 1 --2- 172.21.15.38:0/318790507 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcf88162cf0 0x7fcf880b3000 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:37.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:37.401+0000 7fcf8c9e6640 1 --2- 172.21.15.38:0/318790507 >> v2:172.21.15.195:3300/0 conn(0x7fcf880d0e40 0x7fcf881627b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:37.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:37.401+0000 7fcf8c9e6640 1 --2- 172.21.15.38:0/318790507 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcf88163350 0x7fcf8815c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:37.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:37.401+0000 7fcf8c9e6640 1 -- 172.21.15.38:0/318790507 >> 172.21.15.38:0/318790507 conn(0x7fcf8806cde0 msgr2=0x7fcf8806d1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:37.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:37.401+0000 7fcf8c9e6640 1 -- 172.21.15.38:0/318790507 shutdown_connections 2024-09-21T03:33:37.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:37.401+0000 7fcf8c9e6640 1 -- 172.21.15.38:0/318790507 wait complete. 2024-09-21T03:33:37.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:37.403+0000 7fcf8c9e6640 1 Processor -- start 2024-09-21T03:33:37.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:37.403+0000 7fcf8c9e6640 1 -- start start 2024-09-21T03:33:37.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:37.404+0000 7fcf8c9e6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcf880d0e40 0x7fcf880b5750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:37.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:37.404+0000 7fcf8c9e6640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcf88162cf0 0x7fcf881f9660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:37.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:37.404+0000 7fcf8c9e6640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcf88163350 0x7fcf88161fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:37.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:37.404+0000 7fcf8c9e6640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fcf880b38d0 con 0x7fcf880d0e40 2024-09-21T03:33:37.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:37.404+0000 7fcf8c9e6640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fcf880b3750 con 0x7fcf88162cf0 2024-09-21T03:33:37.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:37.404+0000 7fcf8c9e6640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fcf880b3a50 con 0x7fcf88163350 2024-09-21T03:33:37.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:37.404+0000 7fcf8659c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcf880d0e40 0x7fcf880b5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:37.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:37.404+0000 7fcf8659c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcf880d0e40 0x7fcf880b5750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:39442/0 (socket says 172.21.15.38:39442) 2024-09-21T03:33:37.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:37.404+0000 7fcf8659c640 1 -- 172.21.15.38:0/4292492699 learned_addr learned my addr 172.21.15.38:0/4292492699 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:33:37.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:37.405+0000 7fcf86d9d640 1 --2- 172.21.15.38:0/4292492699 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcf88163350 0x7fcf88161fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:37.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:37.405+0000 7fcf85d9b640 1 --2- 172.21.15.38:0/4292492699 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcf88162cf0 0x7fcf881f9660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:37.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:37.405+0000 7fcf8659c640 1 -- 172.21.15.38:0/4292492699 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcf88163350 msgr2=0x7fcf88161fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:37.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:37.405+0000 7fcf8659c640 1 --2- 172.21.15.38:0/4292492699 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcf88163350 0x7fcf88161fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:37.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:37.405+0000 7fcf8659c640 1 -- 172.21.15.38:0/4292492699 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcf88162cf0 msgr2=0x7fcf881f9660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:37.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:37.405+0000 7fcf8659c640 1 --2- 172.21.15.38:0/4292492699 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcf88162cf0 0x7fcf881f9660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:37.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:37.405+0000 7fcf8659c640 1 -- 172.21.15.38:0/4292492699 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf881f5db0 con 0x7fcf880d0e40 2024-09-21T03:33:37.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:37.405+0000 7fcf8659c640 1 --2- 172.21.15.38:0/4292492699 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcf880d0e40 0x7fcf880b5750 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fcf7800d750 tx=0x7fcf7800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:37.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:37.405+0000 7fcf76ffd640 1 -- 172.21.15.38:0/4292492699 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf78017070 con 0x7fcf880d0e40 2024-09-21T03:33:37.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:37.405+0000 7fcf76ffd640 1 -- 172.21.15.38:0/4292492699 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcf78004310 con 0x7fcf880d0e40 2024-09-21T03:33:37.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:37.405+0000 7fcf8c9e6640 1 -- 172.21.15.38:0/4292492699 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf881f6090 con 0x7fcf880d0e40 2024-09-21T03:33:37.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:37.405+0000 7fcf76ffd640 1 -- 172.21.15.38:0/4292492699 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf7800de60 con 0x7fcf880d0e40 2024-09-21T03:33:37.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:37.405+0000 7fcf8c9e6640 1 -- 172.21.15.38:0/4292492699 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf881f6650 con 0x7fcf880d0e40 2024-09-21T03:33:37.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:37.406+0000 7fcf8c9e6640 1 -- 172.21.15.38:0/4292492699 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf880b3000 con 0x7fcf880d0e40 2024-09-21T03:33:37.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:37.407+0000 7fcf76ffd640 1 -- 172.21.15.38:0/4292492699 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fcf78009c30 con 0x7fcf880d0e40 2024-09-21T03:33:37.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:37.408+0000 7fcf76ffd640 1 --2- 172.21.15.38:0/4292492699 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fcf68078b90 0x7fcf6807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:37.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:37.408+0000 7fcf76ffd640 1 -- 172.21.15.38:0/4292492699 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 6007+0+0 (secure 0 0 0) 0x7fcf7809b3b0 con 0x7fcf880d0e40 2024-09-21T03:33:37.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:37.408+0000 7fcf85d9b640 1 --2- 172.21.15.38:0/4292492699 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fcf68078b90 0x7fcf6807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:37.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:37.408+0000 7fcf85d9b640 1 --2- 172.21.15.38:0/4292492699 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fcf68078b90 0x7fcf6807b0a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fcf88155720 tx=0x7fcf7c005d70 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:37.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:37.412+0000 7fcf76ffd640 1 -- 172.21.15.38:0/4292492699 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fcf78020020 con 0x7fcf880d0e40 2024-09-21T03:33:37.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:37.600+0000 7fcf8c9e6640 1 -- 172.21.15.38:0/4292492699 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fcf8815e7b0 con 0x7fcf880d0e40 2024-09-21T03:33:39.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:39.267+0000 7fcf76ffd640 1 -- 172.21.15.38:0/4292492699 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v24) ==== 228+0+0 (secure 0 0 0) 0x7fcf780635a0 con 0x7fcf880d0e40 2024-09-21T03:33:39.269 INFO:teuthology.orchestra.run.smithi038.stderr:pool 'cephfs_data_ec' created 2024-09-21T03:33:39.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:39.270+0000 7fcf8c9e6640 1 -- 172.21.15.38:0/4292492699 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fcf68078b90 msgr2=0x7fcf6807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:39.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:39.270+0000 7fcf8c9e6640 1 --2- 172.21.15.38:0/4292492699 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fcf68078b90 0x7fcf6807b0a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fcf88155720 tx=0x7fcf7c005d70 comp rx=0 tx=0).stop 2024-09-21T03:33:39.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:39.270+0000 7fcf8c9e6640 1 -- 172.21.15.38:0/4292492699 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcf880d0e40 msgr2=0x7fcf880b5750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:39.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:39.270+0000 7fcf8c9e6640 1 --2- 172.21.15.38:0/4292492699 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcf880d0e40 0x7fcf880b5750 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fcf7800d750 tx=0x7fcf7800dc10 comp rx=0 tx=0).stop 2024-09-21T03:33:39.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:39.271+0000 7fcf8c9e6640 1 -- 172.21.15.38:0/4292492699 shutdown_connections 2024-09-21T03:33:39.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:39.271+0000 7fcf8c9e6640 1 --2- 172.21.15.38:0/4292492699 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fcf68078b90 0x7fcf6807b0a0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:39.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:39.271+0000 7fcf8c9e6640 1 --2- 172.21.15.38:0/4292492699 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcf88163350 0x7fcf88161fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:39.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:39.271+0000 7fcf8c9e6640 1 --2- 172.21.15.38:0/4292492699 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcf88162cf0 0x7fcf881f9660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:39.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:39.271+0000 7fcf8c9e6640 1 --2- 172.21.15.38:0/4292492699 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcf880d0e40 0x7fcf880b5750 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:39.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:39.271+0000 7fcf8c9e6640 1 -- 172.21.15.38:0/4292492699 >> 172.21.15.38:0/4292492699 conn(0x7fcf8806cde0 msgr2=0x7fcf880d2ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:39.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:39.271+0000 7fcf8c9e6640 1 -- 172.21.15.38:0/4292492699 shutdown_connections 2024-09-21T03:33:39.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:39.272+0000 7fcf8c9e6640 1 -- 172.21.15.38:0/4292492699 wait complete. 2024-09-21T03:33:39.293 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-09-21T03:33:39.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:39.419+0000 7fcbaa39d640 1 -- 172.21.15.38:0/1281772865 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcba40b5370 msgr2=0x7fcba41681c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:39.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:39.419+0000 7fcbaa39d640 1 --2- 172.21.15.38:0/1281772865 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcba40b5370 0x7fcba41681c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fcb9c009830 tx=0x7fcb9c02a510 comp rx=0 tx=0).stop 2024-09-21T03:33:39.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:39.419+0000 7fcbaa39d640 1 -- 172.21.15.38:0/1281772865 shutdown_connections 2024-09-21T03:33:39.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:39.419+0000 7fcbaa39d640 1 --2- 172.21.15.38:0/1281772865 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcba40b5370 0x7fcba41681c0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:39.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:39.419+0000 7fcbaa39d640 1 --2- 172.21.15.38:0/1281772865 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcba40b4a20 0x7fcba40b4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:39.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:39.419+0000 7fcbaa39d640 1 -- 172.21.15.38:0/1281772865 >> 172.21.15.38:0/1281772865 conn(0x7fcba40d18c0 msgr2=0x7fcba40d4180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:39.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:39.419+0000 7fcbaa39d640 1 -- 172.21.15.38:0/1281772865 shutdown_connections 2024-09-21T03:33:39.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:39.419+0000 7fcbaa39d640 1 -- 172.21.15.38:0/1281772865 wait complete. 2024-09-21T03:33:39.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:39.420+0000 7fcbaa39d640 1 Processor -- start 2024-09-21T03:33:39.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:39.421+0000 7fcbaa39d640 1 -- start start 2024-09-21T03:33:39.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:39.422+0000 7fcbaa39d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcba40b4a20 0x7fcba414cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:39.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:39.422+0000 7fcbaa39d640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcba40b5370 0x7fcba41f96a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:39.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:39.422+0000 7fcbaa39d640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcba40ced00 0x7fcba41520f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:39.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:39.422+0000 7fcbaa39d640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fcba4168880 con 0x7fcba40b4a20 2024-09-21T03:33:39.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:39.422+0000 7fcbaa39d640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fcba4168700 con 0x7fcba40ced00 2024-09-21T03:33:39.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:39.422+0000 7fcbaa39d640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fcba4168a00 con 0x7fcba40b5370 2024-09-21T03:33:39.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:39.422+0000 7fcba939b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcba40b4a20 0x7fcba414cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:39.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:39.422+0000 7fcba939b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcba40b4a20 0x7fcba414cd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:39444/0 (socket says 172.21.15.38:39444) 2024-09-21T03:33:39.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:39.422+0000 7fcba939b640 1 -- 172.21.15.38:0/2898971024 learned_addr learned my addr 172.21.15.38:0/2898971024 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:33:39.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:39.422+0000 7fcba9b9c640 1 --2- 172.21.15.38:0/2898971024 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcba40ced00 0x7fcba41520f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:39.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:39.422+0000 7fcba8b9a640 1 --2- 172.21.15.38:0/2898971024 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcba40b5370 0x7fcba41f96a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:39.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:39.422+0000 7fcba939b640 1 -- 172.21.15.38:0/2898971024 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcba40b5370 msgr2=0x7fcba41f96a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:39.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:39.422+0000 7fcba939b640 1 --2- 172.21.15.38:0/2898971024 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcba40b5370 0x7fcba41f96a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:39.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:39.422+0000 7fcba939b640 1 -- 172.21.15.38:0/2898971024 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcba40ced00 msgr2=0x7fcba41520f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:39.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:39.422+0000 7fcba939b640 1 --2- 172.21.15.38:0/2898971024 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcba40ced00 0x7fcba41520f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:39.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:39.422+0000 7fcba939b640 1 -- 172.21.15.38:0/2898971024 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcba40ba150 con 0x7fcba40b4a20 2024-09-21T03:33:39.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:39.422+0000 7fcba939b640 1 --2- 172.21.15.38:0/2898971024 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcba40b4a20 0x7fcba414cd70 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fcba000c7a0 tx=0x7fcba000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:39.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:39.423+0000 7fcb99ffb640 1 -- 172.21.15.38:0/2898971024 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcba0017070 con 0x7fcba40b4a20 2024-09-21T03:33:39.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:39.423+0000 7fcb99ffb640 1 -- 172.21.15.38:0/2898971024 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcba0005070 con 0x7fcba40b4a20 2024-09-21T03:33:39.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:39.423+0000 7fcbaa39d640 1 -- 172.21.15.38:0/2898971024 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcba41f9be0 con 0x7fcba40b4a20 2024-09-21T03:33:39.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:39.423+0000 7fcb99ffb640 1 -- 172.21.15.38:0/2898971024 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcba00054a0 con 0x7fcba40b4a20 2024-09-21T03:33:39.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:39.423+0000 7fcbaa39d640 1 -- 172.21.15.38:0/2898971024 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcba40b9b90 con 0x7fcba40b4a20 2024-09-21T03:33:39.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:39.424+0000 7fcbaa39d640 1 -- 172.21.15.38:0/2898971024 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcba40b4e30 con 0x7fcba40b4a20 2024-09-21T03:33:39.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:39.425+0000 7fcb99ffb640 1 -- 172.21.15.38:0/2898971024 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fcba001a070 con 0x7fcba40b4a20 2024-09-21T03:33:39.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:39.425+0000 7fcb99ffb640 1 --2- 172.21.15.38:0/2898971024 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fcb88078bc0 0x7fcb8807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:39.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:39.425+0000 7fcb99ffb640 1 -- 172.21.15.38:0/2898971024 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6500+0+0 (secure 0 0 0) 0x7fcba009a690 con 0x7fcba40b4a20 2024-09-21T03:33:39.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:39.426+0000 7fcba8b9a640 1 --2- 172.21.15.38:0/2898971024 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fcb88078bc0 0x7fcb8807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:39.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:39.426+0000 7fcba8b9a640 1 --2- 172.21.15.38:0/2898971024 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fcb88078bc0 0x7fcb8807b0d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fcb940096c0 tx=0x7fcb94009260 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:39.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:39.430+0000 7fcb99ffb640 1 -- 172.21.15.38:0/2898971024 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fcba0062160 con 0x7fcba40b4a20 2024-09-21T03:33:39.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:39.616+0000 7fcbaa39d640 1 -- 172.21.15.38:0/2898971024 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7fcba4151ed0 con 0x7fcba40b4a20 2024-09-21T03:33:40.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:40.268+0000 7fcb99ffb640 1 -- 172.21.15.38:0/2898971024 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v25) ==== 169+0+0 (secure 0 0 0) 0x7fcba0066b90 con 0x7fcba40b4a20 2024-09-21T03:33:40.271 INFO:teuthology.orchestra.run.smithi038.stderr:set pool 4 allow_ec_overwrites to true 2024-09-21T03:33:40.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:40.271+0000 7fcbaa39d640 1 -- 172.21.15.38:0/2898971024 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fcb88078bc0 msgr2=0x7fcb8807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:40.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:40.271+0000 7fcbaa39d640 1 --2- 172.21.15.38:0/2898971024 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fcb88078bc0 0x7fcb8807b0d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fcb940096c0 tx=0x7fcb94009260 comp rx=0 tx=0).stop 2024-09-21T03:33:40.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:40.271+0000 7fcbaa39d640 1 -- 172.21.15.38:0/2898971024 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcba40b4a20 msgr2=0x7fcba414cd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:40.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:40.271+0000 7fcbaa39d640 1 --2- 172.21.15.38:0/2898971024 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcba40b4a20 0x7fcba414cd70 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fcba000c7a0 tx=0x7fcba000cc60 comp rx=0 tx=0).stop 2024-09-21T03:33:40.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:40.271+0000 7fcbaa39d640 1 -- 172.21.15.38:0/2898971024 shutdown_connections 2024-09-21T03:33:40.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:40.271+0000 7fcbaa39d640 1 --2- 172.21.15.38:0/2898971024 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcba40ced00 0x7fcba41520f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:40.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:40.271+0000 7fcbaa39d640 1 --2- 172.21.15.38:0/2898971024 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fcb88078bc0 0x7fcb8807b0d0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:40.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:40.271+0000 7fcbaa39d640 1 --2- 172.21.15.38:0/2898971024 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcba40b5370 0x7fcba41f96a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:40.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:40.271+0000 7fcbaa39d640 1 --2- 172.21.15.38:0/2898971024 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcba40b4a20 0x7fcba414cd70 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:40.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:40.271+0000 7fcbaa39d640 1 -- 172.21.15.38:0/2898971024 >> 172.21.15.38:0/2898971024 conn(0x7fcba40d18c0 msgr2=0x7fcba4167e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:40.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:40.272+0000 7fcbaa39d640 1 -- 172.21.15.38:0/2898971024 shutdown_connections 2024-09-21T03:33:40.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:40.272+0000 7fcbaa39d640 1 -- 172.21.15.38:0/2898971024 wait complete. 2024-09-21T03:33:40.300 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2024-09-21T03:33:40.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:40.426+0000 7f47c7f7b640 1 -- 172.21.15.38:0/1859293619 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47ac0053a0 msgr2=0x7f47ac00ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:40.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:40.426+0000 7f47c7f7b640 1 --2- 172.21.15.38:0/1859293619 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47ac0053a0 0x7f47ac00ba50 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f47b8009ed0 tx=0x7f47ac005830 comp rx=0 tx=0).stop 2024-09-21T03:33:40.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:40.426+0000 7f47c7f7b640 1 -- 172.21.15.38:0/1859293619 shutdown_connections 2024-09-21T03:33:40.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:40.426+0000 7f47c7f7b640 1 --2- 172.21.15.38:0/1859293619 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47ac0053a0 0x7f47ac00ba50 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:40.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:40.426+0000 7f47c7f7b640 1 -- 172.21.15.38:0/1859293619 >> 172.21.15.38:0/1859293619 conn(0x7f47c009d830 msgr2=0x7f47c00b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:40.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:40.426+0000 7f47c7f7b640 1 -- 172.21.15.38:0/1859293619 shutdown_connections 2024-09-21T03:33:40.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:40.426+0000 7f47c7f7b640 1 -- 172.21.15.38:0/1859293619 wait complete. 2024-09-21T03:33:40.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:40.428+0000 7f47c7f7b640 1 Processor -- start 2024-09-21T03:33:40.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:40.428+0000 7f47c7f7b640 1 -- start start 2024-09-21T03:33:40.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:40.429+0000 7f47c7f7b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47ac0053a0 0x7f47c01f6480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:40.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:40.430+0000 7f47c7f7b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f47c01589c0 0x7f47c006eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:40.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:40.430+0000 7f47c7f7b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f47c006f530 0x7f47c006cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:40.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:40.430+0000 7f47c7f7b640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f47c00b9c50 con 0x7f47ac0053a0 2024-09-21T03:33:40.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:40.430+0000 7f47c7f7b640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f47c00b9ad0 con 0x7f47c01589c0 2024-09-21T03:33:40.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:40.430+0000 7f47c7f7b640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f47c00b9dd0 con 0x7f47c006f530 2024-09-21T03:33:40.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:40.430+0000 7f47c5d17640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47ac0053a0 0x7f47c01f6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:40.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:40.430+0000 7f47c5d17640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47ac0053a0 0x7f47c01f6480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:39474/0 (socket says 172.21.15.38:39474) 2024-09-21T03:33:40.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:40.430+0000 7f47c5d17640 1 -- 172.21.15.38:0/3234180008 learned_addr learned my addr 172.21.15.38:0/3234180008 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:33:40.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:40.430+0000 7f47c6518640 1 --2- 172.21.15.38:0/3234180008 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f47c006f530 0x7f47c006cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:40.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:40.430+0000 7f47c5516640 1 --2- 172.21.15.38:0/3234180008 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f47c01589c0 0x7f47c006eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:40.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:40.430+0000 7f47c5d17640 1 -- 172.21.15.38:0/3234180008 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f47c006f530 msgr2=0x7f47c006cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:40.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:40.430+0000 7f47c5d17640 1 --2- 172.21.15.38:0/3234180008 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f47c006f530 0x7f47c006cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:40.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:40.430+0000 7f47c5d17640 1 -- 172.21.15.38:0/3234180008 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f47c01589c0 msgr2=0x7f47c006eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:40.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:40.430+0000 7f47c5d17640 1 --2- 172.21.15.38:0/3234180008 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f47c01589c0 0x7f47c006eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:40.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:40.430+0000 7f47c5d17640 1 -- 172.21.15.38:0/3234180008 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47c01f69c0 con 0x7f47ac0053a0 2024-09-21T03:33:40.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:40.430+0000 7f47c5d17640 1 --2- 172.21.15.38:0/3234180008 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47ac0053a0 0x7f47c01f6480 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f47bc003f00 tx=0x7f47bc001630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:40.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:40.430+0000 7f47b67fc640 1 -- 172.21.15.38:0/3234180008 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47bc011640 con 0x7f47ac0053a0 2024-09-21T03:33:40.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:40.430+0000 7f47b67fc640 1 -- 172.21.15.38:0/3234180008 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f47bc016020 con 0x7f47ac0053a0 2024-09-21T03:33:40.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:40.430+0000 7f47b67fc640 1 -- 172.21.15.38:0/3234180008 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47bc011a80 con 0x7f47ac0053a0 2024-09-21T03:33:40.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:40.430+0000 7f47c7f7b640 1 -- 172.21.15.38:0/3234180008 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47c006f940 con 0x7f47ac0053a0 2024-09-21T03:33:40.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:40.430+0000 7f47c7f7b640 1 -- 172.21.15.38:0/3234180008 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47c00cbc60 con 0x7f47ac0053a0 2024-09-21T03:33:40.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:40.431+0000 7f47c7f7b640 1 -- 172.21.15.38:0/3234180008 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47c00d0be0 con 0x7f47ac0053a0 2024-09-21T03:33:40.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:40.432+0000 7f47b67fc640 1 -- 172.21.15.38:0/3234180008 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f47bc011c20 con 0x7f47ac0053a0 2024-09-21T03:33:40.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:40.433+0000 7f47b67fc640 1 --2- 172.21.15.38:0/3234180008 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f47a4078bc0 0x7f47a407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:40.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:40.433+0000 7f47b67fc640 1 -- 172.21.15.38:0/3234180008 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6500+0+0 (secure 0 0 0) 0x7f47bc09bbe0 con 0x7f47ac0053a0 2024-09-21T03:33:40.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:40.433+0000 7f47c5516640 1 --2- 172.21.15.38:0/3234180008 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f47a4078bc0 0x7f47a407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:40.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:40.433+0000 7f47c5516640 1 --2- 172.21.15.38:0/3234180008 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f47a4078bc0 0x7f47a407b0d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f47c006ebd0 tx=0x7f47b0006910 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:40.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:40.437+0000 7f47b67fc640 1 -- 172.21.15.38:0/3234180008 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f47bc020020 con 0x7f47ac0053a0 2024-09-21T03:33:40.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:40.664+0000 7f47c7f7b640 1 -- 172.21.15.38:0/3234180008 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7f47c00b53e0 con 0x7f47ac0053a0 2024-09-21T03:33:41.216 INFO:tasks.ceph.mds.d.smithi195.stderr:2024-09-21T03:33:41.214+0000 7fdfaef82640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T03:33:41.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:41.287+0000 7f47b67fc640 1 -- 172.21.15.38:0/3234180008 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v7) ==== 137+0+0 (secure 0 0 0) 0x7f47c006f940 con 0x7f47ac0053a0 2024-09-21T03:33:41.289 INFO:teuthology.orchestra.run.smithi038.stderr:added data pool 4 to fsmap 2024-09-21T03:33:41.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:41.290+0000 7f47c7f7b640 1 -- 172.21.15.38:0/3234180008 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f47a4078bc0 msgr2=0x7f47a407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:41.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:41.290+0000 7f47c7f7b640 1 --2- 172.21.15.38:0/3234180008 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f47a4078bc0 0x7f47a407b0d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f47c006ebd0 tx=0x7f47b0006910 comp rx=0 tx=0).stop 2024-09-21T03:33:41.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:41.290+0000 7f47c7f7b640 1 -- 172.21.15.38:0/3234180008 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47ac0053a0 msgr2=0x7f47c01f6480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:41.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:41.290+0000 7f47c7f7b640 1 --2- 172.21.15.38:0/3234180008 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47ac0053a0 0x7f47c01f6480 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f47bc003f00 tx=0x7f47bc001630 comp rx=0 tx=0).stop 2024-09-21T03:33:41.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:41.290+0000 7f47c7f7b640 1 -- 172.21.15.38:0/3234180008 shutdown_connections 2024-09-21T03:33:41.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:41.290+0000 7f47c7f7b640 1 --2- 172.21.15.38:0/3234180008 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f47a4078bc0 0x7f47a407b0d0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:41.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:41.290+0000 7f47c7f7b640 1 --2- 172.21.15.38:0/3234180008 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f47c006f530 0x7f47c006cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:41.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:41.290+0000 7f47c7f7b640 1 --2- 172.21.15.38:0/3234180008 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f47c01589c0 0x7f47c006eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:41.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:41.290+0000 7f47c7f7b640 1 --2- 172.21.15.38:0/3234180008 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47ac0053a0 0x7f47c01f6480 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:41.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:41.290+0000 7f47c7f7b640 1 -- 172.21.15.38:0/3234180008 >> 172.21.15.38:0/3234180008 conn(0x7f47c009d830 msgr2=0x7f47c00b31a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:41.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:41.290+0000 7f47c7f7b640 1 -- 172.21.15.38:0/3234180008 shutdown_connections 2024-09-21T03:33:41.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:41.290+0000 7f47c7f7b640 1 -- 172.21.15.38:0/3234180008 wait complete. 2024-09-21T03:33:41.306 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:33:41.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:41.428+0000 7fc5eeda8640 1 -- 172.21.15.38:0/2879413434 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5d4004800 msgr2=0x7fc5d4004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:41.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:41.428+0000 7fc5eeda8640 1 --2- 172.21.15.38:0/2879413434 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5d4004800 0x7fc5d4004c10 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc5d8009b50 tx=0x7fc5d4005830 comp rx=0 tx=0).stop 2024-09-21T03:33:41.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:41.428+0000 7fc5ed345640 1 -- 172.21.15.38:0/2879413434 reap_dead start 2024-09-21T03:33:41.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:41.428+0000 7fc5eeda8640 1 -- 172.21.15.38:0/2879413434 shutdown_connections 2024-09-21T03:33:41.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:41.428+0000 7fc5eeda8640 1 -- 172.21.15.38:0/2879413434 >> 172.21.15.38:0/2879413434 conn(0x7fc5e80d4440 msgr2=0x7fc5e80d4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:41.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:41.428+0000 7fc5eeda8640 1 -- 172.21.15.38:0/2879413434 shutdown_connections 2024-09-21T03:33:41.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:41.428+0000 7fc5eeda8640 1 -- 172.21.15.38:0/2879413434 wait complete. 2024-09-21T03:33:41.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:41.430+0000 7fc5eeda8640 1 Processor -- start 2024-09-21T03:33:41.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:41.430+0000 7fc5eeda8640 1 -- start start 2024-09-21T03:33:41.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:41.431+0000 7fc5eeda8640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc5d4004800 0x7fc5e80c7f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:41.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:41.431+0000 7fc5eeda8640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc5e814c450 0x7fc5e814c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:41.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:41.431+0000 7fc5eeda8640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5e814cda0 0x7fc5e81f89f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:41.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:41.431+0000 7fc5eeda8640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fc5e8162b20 con 0x7fc5e814cda0 2024-09-21T03:33:41.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:41.431+0000 7fc5eeda8640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fc5e81629a0 con 0x7fc5e814c450 2024-09-21T03:33:41.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:41.431+0000 7fc5eeda8640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fc5e8162ca0 con 0x7fc5d4004800 2024-09-21T03:33:41.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:41.431+0000 7fc5ed345640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5e814cda0 0x7fc5e81f89f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:41.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:41.431+0000 7fc5ed345640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5e814cda0 0x7fc5e81f89f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:39500/0 (socket says 172.21.15.38:39500) 2024-09-21T03:33:41.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:41.431+0000 7fc5ed345640 1 -- 172.21.15.38:0/421330300 learned_addr learned my addr 172.21.15.38:0/421330300 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:33:41.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:41.431+0000 7fc5ecb44640 1 --2- 172.21.15.38:0/421330300 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc5d4004800 0x7fc5e80c7f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:41.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:41.432+0000 7fc5e7fff640 1 --2- 172.21.15.38:0/421330300 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc5e814c450 0x7fc5e814c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:41.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:41.432+0000 7fc5ed345640 1 -- 172.21.15.38:0/421330300 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc5d4004800 msgr2=0x7fc5e80c7f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:41.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:41.432+0000 7fc5ed345640 1 --2- 172.21.15.38:0/421330300 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc5d4004800 0x7fc5e80c7f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:41.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:41.432+0000 7fc5ed345640 1 -- 172.21.15.38:0/421330300 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc5e814c450 msgr2=0x7fc5e814c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:41.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:41.432+0000 7fc5ed345640 1 --2- 172.21.15.38:0/421330300 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc5e814c450 0x7fc5e814c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:41.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:41.432+0000 7fc5ed345640 1 -- 172.21.15.38:0/421330300 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5e80c88c0 con 0x7fc5e814cda0 2024-09-21T03:33:41.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:41.432+0000 7fc5ed345640 1 --2- 172.21.15.38:0/421330300 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5e814cda0 0x7fc5e81f89f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc5e80cea40 tx=0x7fc5e000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:41.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:41.432+0000 7fc5e57fa640 1 -- 172.21.15.38:0/421330300 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5e0003d80 con 0x7fc5e814cda0 2024-09-21T03:33:41.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:41.432+0000 7fc5e57fa640 1 -- 172.21.15.38:0/421330300 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc5e0004b20 con 0x7fc5e814cda0 2024-09-21T03:33:41.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:41.432+0000 7fc5eeda8640 1 -- 172.21.15.38:0/421330300 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5e80c8ba0 con 0x7fc5e814cda0 2024-09-21T03:33:41.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:41.432+0000 7fc5e57fa640 1 -- 172.21.15.38:0/421330300 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5e0004030 con 0x7fc5e814cda0 2024-09-21T03:33:41.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:41.432+0000 7fc5eeda8640 1 -- 172.21.15.38:0/421330300 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5e80c9160 con 0x7fc5e814cda0 2024-09-21T03:33:41.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:41.433+0000 7fc5eeda8640 1 -- 172.21.15.38:0/421330300 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5e80b3a70 con 0x7fc5e814cda0 2024-09-21T03:33:41.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:41.434+0000 7fc5e57fa640 1 -- 172.21.15.38:0/421330300 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc5e00041d0 con 0x7fc5e814cda0 2024-09-21T03:33:41.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:41.434+0000 7fc5e57fa640 1 --2- 172.21.15.38:0/421330300 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc5c8078b90 0x7fc5c807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:41.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:41.435+0000 7fc5e57fa640 1 -- 172.21.15.38:0/421330300 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fc5e009bdf0 con 0x7fc5e814cda0 2024-09-21T03:33:41.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:41.435+0000 7fc5ecb44640 1 --2- 172.21.15.38:0/421330300 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc5c8078b90 0x7fc5c807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:41.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:41.435+0000 7fc5ecb44640 1 --2- 172.21.15.38:0/421330300 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc5c8078b90 0x7fc5c807b0a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc5d0000fd0 tx=0x7fc5d0006930 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:41.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:41.439+0000 7fc5e57fa640 1 -- 172.21.15.38:0/421330300 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fc5e00638a0 con 0x7fc5e814cda0 2024-09-21T03:33:41.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:41.663+0000 7fc5eeda8640 1 -- 172.21.15.38:0/421330300 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc5e80c9960 con 0x7fc5e814cda0 2024-09-21T03:33:41.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:41.663+0000 7fc5e57fa640 1 -- 172.21.15.38:0/421330300 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5219 (secure 0 0 0) 0x7fc5e00682d0 con 0x7fc5e814cda0 2024-09-21T03:33:41.665 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:33:41.665 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":7,"btime":"2024-09-21T03:33:41:279513+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4322,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/593838590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":593838590},{"type":"v1","addr":"172.21.15.195:6836","nonce":593838590}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4374,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/663402481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":663402481},{"type":"v1","addr":"172.21.15.38:6833","nonce":663402481}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4377,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/2614829035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":2614829035},{"type":"v1","addr":"172.21.15.38:6835","nonce":2614829035}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:33:35.622623+0000","modified":"2024-09-21T03:33:40.666159+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4402},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4402":{"gid":4402,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.195:6837/1420805977","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":1420805977},{"type":"v1","addr":"172.21.15.195:6837","nonce":1420805977}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":4402,"qdb_cluster":[4402]},"id":1}]} 2024-09-21T03:33:41.665 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 7 2024-09-21T03:33:41.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:41.665+0000 7fc5eeda8640 1 -- 172.21.15.38:0/421330300 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc5c8078b90 msgr2=0x7fc5c807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:41.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:41.665+0000 7fc5eeda8640 1 --2- 172.21.15.38:0/421330300 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc5c8078b90 0x7fc5c807b0a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc5d0000fd0 tx=0x7fc5d0006930 comp rx=0 tx=0).stop 2024-09-21T03:33:41.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:41.665+0000 7fc5eeda8640 1 -- 172.21.15.38:0/421330300 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5e814cda0 msgr2=0x7fc5e81f89f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:41.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:41.665+0000 7fc5eeda8640 1 --2- 172.21.15.38:0/421330300 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5e814cda0 0x7fc5e81f89f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc5e80cea40 tx=0x7fc5e000f790 comp rx=0 tx=0).stop 2024-09-21T03:33:41.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:41.665+0000 7fc5eeda8640 1 -- 172.21.15.38:0/421330300 shutdown_connections 2024-09-21T03:33:41.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:41.665+0000 7fc5eeda8640 1 --2- 172.21.15.38:0/421330300 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5e814cda0 0x7fc5e81f89f0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:41.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:41.665+0000 7fc5eeda8640 1 --2- 172.21.15.38:0/421330300 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc5e814c450 0x7fc5e814c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:41.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:41.665+0000 7fc5eeda8640 1 --2- 172.21.15.38:0/421330300 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc5c8078b90 0x7fc5c807b0a0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:41.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:41.665+0000 7fc5eeda8640 1 --2- 172.21.15.38:0/421330300 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc5d4004800 0x7fc5e80c7f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:41.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:41.665+0000 7fc5eeda8640 1 -- 172.21.15.38:0/421330300 >> 172.21.15.38:0/421330300 conn(0x7fc5e80d4440 msgr2=0x7fc5e80d4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:41.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:41.665+0000 7fc5eeda8640 1 -- 172.21.15.38:0/421330300 shutdown_connections 2024-09-21T03:33:41.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:41.665+0000 7fc5eeda8640 1 -- 172.21.15.38:0/421330300 wait complete. 2024-09-21T03:33:41.678 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T03:33:41.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:41.793+0000 7f215a1a1640 1 -- 172.21.15.38:0/3793056784 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f21540b2e60 msgr2=0x7f21540b3270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:41.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:41.793+0000 7f215a1a1640 1 --2- 172.21.15.38:0/3793056784 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f21540b2e60 0x7f21540b3270 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f215408d3b0 tx=0x7f214c02a8b0 comp rx=0 tx=0).stop 2024-09-21T03:33:41.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:41.793+0000 7f215a1a1640 1 -- 172.21.15.38:0/3793056784 shutdown_connections 2024-09-21T03:33:41.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:41.793+0000 7f215a1a1640 1 --2- 172.21.15.38:0/3793056784 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f21540b2e60 0x7f21540b3270 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:41.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:41.793+0000 7f215a1a1640 1 --2- 172.21.15.38:0/3793056784 >> v2:172.21.15.195:3300/0 conn(0x7f21540cfe00 0x7f21540d2a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:41.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:41.793+0000 7f215a1a1640 1 -- 172.21.15.38:0/3793056784 >> 172.21.15.38:0/3793056784 conn(0x7f21540d4440 msgr2=0x7f21540d4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:41.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:41.793+0000 7f215a1a1640 1 -- 172.21.15.38:0/3793056784 shutdown_connections 2024-09-21T03:33:41.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:41.793+0000 7f215a1a1640 1 -- 172.21.15.38:0/3793056784 wait complete. 2024-09-21T03:33:41.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:41.794+0000 7f215a1a1640 1 Processor -- start 2024-09-21T03:33:41.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:41.795+0000 7f215a1a1640 1 -- start start 2024-09-21T03:33:41.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:41.796+0000 7f215a1a1640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f21540b2e60 0x7f21541539e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:41.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:41.796+0000 7f215a1a1640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f21540cfe00 0x7f2154155730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:41.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:41.796+0000 7f215a1a1640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2154162590 0x7f2154160000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:41.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:41.796+0000 7f215a1a1640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f21540b3930 con 0x7f21540b2e60 2024-09-21T03:33:41.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:41.796+0000 7f215a1a1640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f21540b37b0 con 0x7f2154162590 2024-09-21T03:33:41.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:41.796+0000 7f215a1a1640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f21540b3ab0 con 0x7f21540cfe00 2024-09-21T03:33:41.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:41.796+0000 7f21537fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f21540b2e60 0x7f21541539e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:41.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:41.796+0000 7f21537fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f21540b2e60 0x7f21541539e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:39520/0 (socket says 172.21.15.38:39520) 2024-09-21T03:33:41.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:41.796+0000 7f21537fe640 1 -- 172.21.15.38:0/2863742417 learned_addr learned my addr 172.21.15.38:0/2863742417 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:33:41.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:41.796+0000 7f2152ffd640 1 --2- 172.21.15.38:0/2863742417 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f21540cfe00 0x7f2154155730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:41.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:41.796+0000 7f2153fff640 1 --2- 172.21.15.38:0/2863742417 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2154162590 0x7f2154160000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:41.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:41.796+0000 7f21537fe640 1 -- 172.21.15.38:0/2863742417 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f21540cfe00 msgr2=0x7f2154155730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:41.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:41.796+0000 7f21537fe640 1 --2- 172.21.15.38:0/2863742417 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f21540cfe00 0x7f2154155730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:41.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:41.796+0000 7f21537fe640 1 -- 172.21.15.38:0/2863742417 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2154162590 msgr2=0x7f2154160000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:41.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:41.796+0000 7f21537fe640 1 --2- 172.21.15.38:0/2863742417 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2154162590 0x7f2154160000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:41.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:41.796+0000 7f21537fe640 1 -- 172.21.15.38:0/2863742417 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f215415cf30 con 0x7f21540b2e60 2024-09-21T03:33:41.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:41.796+0000 7f2153fff640 1 --2- 172.21.15.38:0/2863742417 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2154162590 0x7f2154160000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:33:41.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:41.797+0000 7f21537fe640 1 --2- 172.21.15.38:0/2863742417 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f21540b2e60 0x7f21541539e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f214400d770 tx=0x7f214400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:41.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:41.797+0000 7f213bfff640 1 -- 172.21.15.38:0/2863742417 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2144017070 con 0x7f21540b2e60 2024-09-21T03:33:41.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:41.797+0000 7f213bfff640 1 -- 172.21.15.38:0/2863742417 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2144009d20 con 0x7f21540b2e60 2024-09-21T03:33:41.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:41.797+0000 7f215a1a1640 1 -- 172.21.15.38:0/2863742417 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21541f91e0 con 0x7f21540b2e60 2024-09-21T03:33:41.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:41.797+0000 7f213bfff640 1 -- 172.21.15.38:0/2863742417 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21440106c0 con 0x7f21540b2e60 2024-09-21T03:33:41.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:41.797+0000 7f215a1a1640 1 -- 172.21.15.38:0/2863742417 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21541f97a0 con 0x7f21540b2e60 2024-09-21T03:33:41.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:41.798+0000 7f215a1a1640 1 -- 172.21.15.38:0/2863742417 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21540b3270 con 0x7f21540b2e60 2024-09-21T03:33:41.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:41.798+0000 7f213bfff640 1 -- 172.21.15.38:0/2863742417 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f214401a070 con 0x7f21540b2e60 2024-09-21T03:33:41.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:41.799+0000 7f213bfff640 1 --2- 172.21.15.38:0/2863742417 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2140078b90 0x7f214007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:41.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:41.799+0000 7f213bfff640 1 -- 172.21.15.38:0/2863742417 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f214409b190 con 0x7f21540b2e60 2024-09-21T03:33:41.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:41.799+0000 7f2152ffd640 1 --2- 172.21.15.38:0/2863742417 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2140078b90 0x7f214007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:41.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:41.800+0000 7f2152ffd640 1 --2- 172.21.15.38:0/2863742417 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2140078b90 0x7f214007b0a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f215415a9c0 tx=0x7f2148009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:41.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:41.804+0000 7f213bfff640 1 -- 172.21.15.38:0/2863742417 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f21440632a0 con 0x7f21540b2e60 2024-09-21T03:33:41.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:41.991+0000 7f215a1a1640 1 -- 172.21.15.38:0/2863742417 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f215415ea50 con 0x7f21540b2e60 2024-09-21T03:33:41.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:41.992+0000 7f213bfff640 1 -- 172.21.15.38:0/2863742417 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18626 (secure 0 0 0) 0x7f2144062ba0 con 0x7f21540b2e60 2024-09-21T03:33:41.994 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:33:41.995 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":26,"fsid":"9d50d66f-656b-4fe2-8e3b-fe53b79c4d43","created":"2024-09-21T03:33:10.677865+0000","modified":"2024-09-21T03:33:40.665991+0000","last_up_change":"2024-09-21T03:33:24.023421+0000","last_in_change":"2024-09-21T03:33:14.296193+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T03:33:22.992573+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T03:33:33.530201+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T03:33:34.585875+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T03:33:38.268346+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"dbab183f-7e63-48b8-960a-ae55fd5bf565","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6801","nonce":2381639433}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6803","nonce":2381639433}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6807","nonce":2381639433}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6805","nonce":2381639433}]},"public_addr":"172.21.15.38:6801/2381639433","cluster_addr":"172.21.15.38:6803/2381639433","heartbeat_back_addr":"172.21.15.38:6807/2381639433","heartbeat_front_addr":"172.21.15.38:6805/2381639433","state":["exists","up"]},{"osd":1,"uuid":"fa1c0e8a-ebc1-4359-8146-bd5342abbc16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6817","nonce":1491818732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6819","nonce":1491818732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6823","nonce":1491818732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6821","nonce":1491818732}]},"public_addr":"172.21.15.38:6817/1491818732","cluster_addr":"172.21.15.38:6819/1491818732","heartbeat_back_addr":"172.21.15.38:6823/1491818732","heartbeat_front_addr":"172.21.15.38:6821/1491818732","state":["exists","up"]},{"osd":2,"uuid":"2ed3cd2d-385d-4fc6-9869-214552a84803","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6825","nonce":2577858902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6827","nonce":2577858902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6831","nonce":2577858902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6829","nonce":2577858902}]},"public_addr":"172.21.15.38:6825/2577858902","cluster_addr":"172.21.15.38:6827/2577858902","heartbeat_back_addr":"172.21.15.38:6831/2577858902","heartbeat_front_addr":"172.21.15.38:6829/2577858902","state":["exists","up"]},{"osd":3,"uuid":"62d1d95e-919d-4250-be71-1d34cfa70dbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6809","nonce":1286810294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6811","nonce":1286810294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6815","nonce":1286810294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6813","nonce":1286810294}]},"public_addr":"172.21.15.38:6809/1286810294","cluster_addr":"172.21.15.38:6811/1286810294","heartbeat_back_addr":"172.21.15.38:6815/1286810294","heartbeat_front_addr":"172.21.15.38:6813/1286810294","state":["exists","up"]},{"osd":4,"uuid":"7c9f9314-99d7-4d73-8392-37809a9f54b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6803","nonce":3401837571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6805","nonce":3401837571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6809","nonce":3401837571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6807","nonce":3401837571}]},"public_addr":"172.21.15.195:6803/3401837571","cluster_addr":"172.21.15.195:6805/3401837571","heartbeat_back_addr":"172.21.15.195:6809/3401837571","heartbeat_front_addr":"172.21.15.195:6807/3401837571","state":["exists","up"]},{"osd":5,"uuid":"bde4fe9f-8414-4bba-bf4a-43973d1976bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6811","nonce":2080960180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6813","nonce":2080960180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6817","nonce":2080960180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6815","nonce":2080960180}]},"public_addr":"172.21.15.195:6811/2080960180","cluster_addr":"172.21.15.195:6813/2080960180","heartbeat_back_addr":"172.21.15.195:6817/2080960180","heartbeat_front_addr":"172.21.15.195:6815/2080960180","state":["exists","up"]},{"osd":6,"uuid":"6ee6d7f4-1a5f-4a4c-ac16-48fe05f6a002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6827","nonce":4055825675}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6829","nonce":4055825675}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6833","nonce":4055825675}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6831","nonce":4055825675}]},"public_addr":"172.21.15.195:6827/4055825675","cluster_addr":"172.21.15.195:6829/4055825675","heartbeat_back_addr":"172.21.15.195:6833/4055825675","heartbeat_front_addr":"172.21.15.195:6831/4055825675","state":["exists","up"]},{"osd":7,"uuid":"efeb5f51-329a-4484-8290-92e7d3503d05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6819","nonce":1606106342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6821","nonce":1606106342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6825","nonce":1606106342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6823","nonce":1606106342}]},"public_addr":"172.21.15.195:6819/1606106342","cluster_addr":"172.21.15.195:6821/1606106342","heartbeat_back_addr":"172.21.15.195:6825/1606106342","heartbeat_front_addr":"172.21.15.195:6823/1606106342","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.901343+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:21.115003+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.901623+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.835238+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.132199+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:19.817560+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.829690+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:21.087300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T03:33:41.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:41.995+0000 7f215a1a1640 1 -- 172.21.15.38:0/2863742417 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2140078b90 msgr2=0x7f214007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:41.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:41.995+0000 7f215a1a1640 1 --2- 172.21.15.38:0/2863742417 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2140078b90 0x7f214007b0a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f215415a9c0 tx=0x7f2148009040 comp rx=0 tx=0).stop 2024-09-21T03:33:41.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:41.995+0000 7f215a1a1640 1 -- 172.21.15.38:0/2863742417 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f21540b2e60 msgr2=0x7f21541539e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:41.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:41.995+0000 7f215a1a1640 1 --2- 172.21.15.38:0/2863742417 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f21540b2e60 0x7f21541539e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f214400d770 tx=0x7f214400dc30 comp rx=0 tx=0).stop 2024-09-21T03:33:41.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:41.995+0000 7f215a1a1640 1 -- 172.21.15.38:0/2863742417 shutdown_connections 2024-09-21T03:33:41.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:41.995+0000 7f215a1a1640 1 --2- 172.21.15.38:0/2863742417 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2154162590 0x7f2154160000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:41.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:41.995+0000 7f215a1a1640 1 --2- 172.21.15.38:0/2863742417 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2140078b90 0x7f214007b0a0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:41.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:41.995+0000 7f215a1a1640 1 --2- 172.21.15.38:0/2863742417 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f21540cfe00 0x7f2154155730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:41.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:41.995+0000 7f215a1a1640 1 --2- 172.21.15.38:0/2863742417 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f21540b2e60 0x7f21541539e0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:41.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:41.995+0000 7f215a1a1640 1 -- 172.21.15.38:0/2863742417 >> 172.21.15.38:0/2863742417 conn(0x7f21540d4440 msgr2=0x7f2154157ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:41.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:41.995+0000 7f215a1a1640 1 -- 172.21.15.38:0/2863742417 shutdown_connections 2024-09-21T03:33:41.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:41.995+0000 7f215a1a1640 1 -- 172.21.15.38:0/2863742417 wait complete. 2024-09-21T03:33:42.008 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T03:33:42.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:42.122+0000 7f29a2225640 1 -- 172.21.15.38:0/3615013458 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f299c0cfe00 msgr2=0x7f299c0d2a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:42.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:42.122+0000 7f29a2225640 1 --2- 172.21.15.38:0/3615013458 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f299c0cfe00 0x7f299c0d2a20 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f298c005fd0 tx=0x7f298c02a5d0 comp rx=0 tx=0).stop 2024-09-21T03:33:42.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:42.122+0000 7f29a2225640 1 -- 172.21.15.38:0/3615013458 shutdown_connections 2024-09-21T03:33:42.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:42.122+0000 7f29a2225640 1 --2- 172.21.15.38:0/3615013458 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f299c0b2e60 0x7f299c0b3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:42.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:42.122+0000 7f29a2225640 1 --2- 172.21.15.38:0/3615013458 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f299c0cfe00 0x7f299c0d2a20 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:42.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:42.122+0000 7f29a2225640 1 -- 172.21.15.38:0/3615013458 >> 172.21.15.38:0/3615013458 conn(0x7f299c0d4440 msgr2=0x7f299c0d4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:42.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:42.122+0000 7f29a2225640 1 -- 172.21.15.38:0/3615013458 shutdown_connections 2024-09-21T03:33:42.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:42.122+0000 7f29a2225640 1 -- 172.21.15.38:0/3615013458 wait complete. 2024-09-21T03:33:42.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:42.124+0000 7f29a2225640 1 Processor -- start 2024-09-21T03:33:42.126 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:42.124+0000 7f29a2225640 1 -- start start 2024-09-21T03:33:42.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:42.125+0000 7f29a2225640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f299c0b2e60 0x7f299c154080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:42.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:42.125+0000 7f29a2225640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f299c0cfe00 0x7f299c14a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:42.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:42.125+0000 7f29a2225640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f299c162590 0x7f299c15a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:42.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:42.125+0000 7f29a2225640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f299c0b3930 con 0x7f299c0cfe00 2024-09-21T03:33:42.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:42.125+0000 7f29a2225640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f299c0b37b0 con 0x7f299c162590 2024-09-21T03:33:42.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:42.125+0000 7f29a2225640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f299c0b3ab0 con 0x7f299c0b2e60 2024-09-21T03:33:42.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:42.125+0000 7f299affd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f299c0cfe00 0x7f299c14a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:42.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:42.125+0000 7f299affd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f299c0cfe00 0x7f299c14a7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:39540/0 (socket says 172.21.15.38:39540) 2024-09-21T03:33:42.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:42.125+0000 7f299affd640 1 -- 172.21.15.38:0/1345674766 learned_addr learned my addr 172.21.15.38:0/1345674766 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:33:42.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:42.125+0000 7f299b7fe640 1 --2- 172.21.15.38:0/1345674766 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f299c0b2e60 0x7f299c154080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:42.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:42.125+0000 7f299bfff640 1 --2- 172.21.15.38:0/1345674766 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f299c162590 0x7f299c15a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:42.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:42.126+0000 7f299affd640 1 -- 172.21.15.38:0/1345674766 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f299c0b2e60 msgr2=0x7f299c154080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:42.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:42.126+0000 7f299affd640 1 --2- 172.21.15.38:0/1345674766 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f299c0b2e60 0x7f299c154080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:42.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:42.126+0000 7f299affd640 1 -- 172.21.15.38:0/1345674766 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f299c162590 msgr2=0x7f299c15a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:42.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:42.126+0000 7f299affd640 1 --2- 172.21.15.38:0/1345674766 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f299c162590 0x7f299c15a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:42.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:42.126+0000 7f299affd640 1 -- 172.21.15.38:0/1345674766 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f299c15b8a0 con 0x7f299c0cfe00 2024-09-21T03:33:42.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:42.126+0000 7f299affd640 1 --2- 172.21.15.38:0/1345674766 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f299c0cfe00 0x7f299c14a7b0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f299000c400 tx=0x7f299000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:42.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:42.126+0000 7f2983fff640 1 -- 172.21.15.38:0/1345674766 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2990017070 con 0x7f299c0cfe00 2024-09-21T03:33:42.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:42.126+0000 7f2983fff640 1 -- 172.21.15.38:0/1345674766 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f29900053c0 con 0x7f299c0cfe00 2024-09-21T03:33:42.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:42.126+0000 7f29a2225640 1 -- 172.21.15.38:0/1345674766 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f299c15d2d0 con 0x7f299c0cfe00 2024-09-21T03:33:42.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:42.126+0000 7f2983fff640 1 -- 172.21.15.38:0/1345674766 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2990003850 con 0x7f299c0cfe00 2024-09-21T03:33:42.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:42.126+0000 7f29a2225640 1 -- 172.21.15.38:0/1345674766 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f299c15d890 con 0x7f299c0cfe00 2024-09-21T03:33:42.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:42.127+0000 7f29a2225640 1 -- 172.21.15.38:0/1345674766 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f299c15b610 con 0x7f299c0cfe00 2024-09-21T03:33:42.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:42.128+0000 7f2983fff640 1 -- 172.21.15.38:0/1345674766 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f299001a070 con 0x7f299c0cfe00 2024-09-21T03:33:42.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:42.129+0000 7f2983fff640 1 --2- 172.21.15.38:0/1345674766 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2988078bc0 0x7f298807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:42.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:42.129+0000 7f2983fff640 1 -- 172.21.15.38:0/1345674766 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f299009a3d0 con 0x7f299c0cfe00 2024-09-21T03:33:42.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:42.129+0000 7f299b7fe640 1 --2- 172.21.15.38:0/1345674766 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2988078bc0 0x7f298807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:42.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:42.129+0000 7f299b7fe640 1 --2- 172.21.15.38:0/1345674766 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2988078bc0 0x7f298807b0d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f298c033000 tx=0x7f298c004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:42.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:42.133+0000 7f2983fff640 1 -- 172.21.15.38:0/1345674766 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f2990061e80 con 0x7f299c0cfe00 2024-09-21T03:33:42.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:42.323+0000 7f29a2225640 1 -- 172.21.15.38:0/1345674766 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f299c0b3270 con 0x7f299c0cfe00 2024-09-21T03:33:42.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:42.325+0000 7f2983fff640 1 -- 172.21.15.38:0/1345674766 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18626 (secure 0 0 0) 0x7f29900668b0 con 0x7f299c0cfe00 2024-09-21T03:33:42.327 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:33:42.327 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":26,"fsid":"9d50d66f-656b-4fe2-8e3b-fe53b79c4d43","created":"2024-09-21T03:33:10.677865+0000","modified":"2024-09-21T03:33:40.665991+0000","last_up_change":"2024-09-21T03:33:24.023421+0000","last_in_change":"2024-09-21T03:33:14.296193+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T03:33:22.992573+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T03:33:33.530201+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T03:33:34.585875+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T03:33:38.268346+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"dbab183f-7e63-48b8-960a-ae55fd5bf565","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6801","nonce":2381639433}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6803","nonce":2381639433}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6807","nonce":2381639433}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6805","nonce":2381639433}]},"public_addr":"172.21.15.38:6801/2381639433","cluster_addr":"172.21.15.38:6803/2381639433","heartbeat_back_addr":"172.21.15.38:6807/2381639433","heartbeat_front_addr":"172.21.15.38:6805/2381639433","state":["exists","up"]},{"osd":1,"uuid":"fa1c0e8a-ebc1-4359-8146-bd5342abbc16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6817","nonce":1491818732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6819","nonce":1491818732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6823","nonce":1491818732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6821","nonce":1491818732}]},"public_addr":"172.21.15.38:6817/1491818732","cluster_addr":"172.21.15.38:6819/1491818732","heartbeat_back_addr":"172.21.15.38:6823/1491818732","heartbeat_front_addr":"172.21.15.38:6821/1491818732","state":["exists","up"]},{"osd":2,"uuid":"2ed3cd2d-385d-4fc6-9869-214552a84803","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6825","nonce":2577858902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6827","nonce":2577858902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6831","nonce":2577858902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6829","nonce":2577858902}]},"public_addr":"172.21.15.38:6825/2577858902","cluster_addr":"172.21.15.38:6827/2577858902","heartbeat_back_addr":"172.21.15.38:6831/2577858902","heartbeat_front_addr":"172.21.15.38:6829/2577858902","state":["exists","up"]},{"osd":3,"uuid":"62d1d95e-919d-4250-be71-1d34cfa70dbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6809","nonce":1286810294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6811","nonce":1286810294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6815","nonce":1286810294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6813","nonce":1286810294}]},"public_addr":"172.21.15.38:6809/1286810294","cluster_addr":"172.21.15.38:6811/1286810294","heartbeat_back_addr":"172.21.15.38:6815/1286810294","heartbeat_front_addr":"172.21.15.38:6813/1286810294","state":["exists","up"]},{"osd":4,"uuid":"7c9f9314-99d7-4d73-8392-37809a9f54b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6803","nonce":3401837571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6805","nonce":3401837571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6809","nonce":3401837571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6807","nonce":3401837571}]},"public_addr":"172.21.15.195:6803/3401837571","cluster_addr":"172.21.15.195:6805/3401837571","heartbeat_back_addr":"172.21.15.195:6809/3401837571","heartbeat_front_addr":"172.21.15.195:6807/3401837571","state":["exists","up"]},{"osd":5,"uuid":"bde4fe9f-8414-4bba-bf4a-43973d1976bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6811","nonce":2080960180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6813","nonce":2080960180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6817","nonce":2080960180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6815","nonce":2080960180}]},"public_addr":"172.21.15.195:6811/2080960180","cluster_addr":"172.21.15.195:6813/2080960180","heartbeat_back_addr":"172.21.15.195:6817/2080960180","heartbeat_front_addr":"172.21.15.195:6815/2080960180","state":["exists","up"]},{"osd":6,"uuid":"6ee6d7f4-1a5f-4a4c-ac16-48fe05f6a002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6827","nonce":4055825675}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6829","nonce":4055825675}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6833","nonce":4055825675}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6831","nonce":4055825675}]},"public_addr":"172.21.15.195:6827/4055825675","cluster_addr":"172.21.15.195:6829/4055825675","heartbeat_back_addr":"172.21.15.195:6833/4055825675","heartbeat_front_addr":"172.21.15.195:6831/4055825675","state":["exists","up"]},{"osd":7,"uuid":"efeb5f51-329a-4484-8290-92e7d3503d05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6819","nonce":1606106342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6821","nonce":1606106342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6825","nonce":1606106342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6823","nonce":1606106342}]},"public_addr":"172.21.15.195:6819/1606106342","cluster_addr":"172.21.15.195:6821/1606106342","heartbeat_back_addr":"172.21.15.195:6825/1606106342","heartbeat_front_addr":"172.21.15.195:6823/1606106342","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.901343+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:21.115003+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.901623+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.835238+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.132199+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:19.817560+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.829690+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:21.087300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T03:33:42.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:42.327+0000 7f29a2225640 1 -- 172.21.15.38:0/1345674766 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2988078bc0 msgr2=0x7f298807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:42.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:42.327+0000 7f29a2225640 1 --2- 172.21.15.38:0/1345674766 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2988078bc0 0x7f298807b0d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f298c033000 tx=0x7f298c004730 comp rx=0 tx=0).stop 2024-09-21T03:33:42.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:42.327+0000 7f29a2225640 1 -- 172.21.15.38:0/1345674766 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f299c0cfe00 msgr2=0x7f299c14a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:42.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:42.327+0000 7f29a2225640 1 --2- 172.21.15.38:0/1345674766 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f299c0cfe00 0x7f299c14a7b0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f299000c400 tx=0x7f299000c8c0 comp rx=0 tx=0).stop 2024-09-21T03:33:42.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:42.327+0000 7f29a2225640 1 -- 172.21.15.38:0/1345674766 shutdown_connections 2024-09-21T03:33:42.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:42.327+0000 7f29a2225640 1 --2- 172.21.15.38:0/1345674766 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f299c162590 0x7f299c15a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:42.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:42.327+0000 7f29a2225640 1 --2- 172.21.15.38:0/1345674766 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f299c0cfe00 0x7f299c14a7b0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:42.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:42.327+0000 7f29a2225640 1 --2- 172.21.15.38:0/1345674766 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2988078bc0 0x7f298807b0d0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:42.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:42.327+0000 7f29a2225640 1 --2- 172.21.15.38:0/1345674766 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f299c0b2e60 0x7f299c154080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:42.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:42.327+0000 7f29a2225640 1 -- 172.21.15.38:0/1345674766 >> 172.21.15.38:0/1345674766 conn(0x7f299c0d4440 msgr2=0x7f299c0b47f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:42.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:42.327+0000 7f29a2225640 1 -- 172.21.15.38:0/1345674766 shutdown_connections 2024-09-21T03:33:42.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:42.327+0000 7f29a2225640 1 -- 172.21.15.38:0/1345674766 wait complete. 2024-09-21T03:33:42.356 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:33:42.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:42.469+0000 7f6ab3e56640 1 -- 172.21.15.38:0/4044616461 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6aac0b2e60 msgr2=0x7f6aac0b3270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:42.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:42.469+0000 7f6ab3e56640 1 --2- 172.21.15.38:0/4044616461 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6aac0b2e60 0x7f6aac0b3270 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f6aac08d3b0 tx=0x7f6aa402aa30 comp rx=0 tx=0).stop 2024-09-21T03:33:42.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:42.469+0000 7f6ab3e56640 1 -- 172.21.15.38:0/4044616461 shutdown_connections 2024-09-21T03:33:42.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:42.469+0000 7f6ab3e56640 1 --2- 172.21.15.38:0/4044616461 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6aac0b2e60 0x7f6aac0b3270 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:42.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:42.469+0000 7f6ab3e56640 1 --2- 172.21.15.38:0/4044616461 >> v2:172.21.15.195:3300/0 conn(0x7f6aac0cfe00 0x7f6aac0d2a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:42.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:42.469+0000 7f6ab3e56640 1 -- 172.21.15.38:0/4044616461 >> 172.21.15.38:0/4044616461 conn(0x7f6aac0d4440 msgr2=0x7f6aac0d4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:42.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:42.470+0000 7f6ab3e56640 1 -- 172.21.15.38:0/4044616461 shutdown_connections 2024-09-21T03:33:42.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:42.470+0000 7f6ab3e56640 1 -- 172.21.15.38:0/4044616461 wait complete. 2024-09-21T03:33:42.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:42.471+0000 7f6ab3e56640 1 Processor -- start 2024-09-21T03:33:42.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:42.471+0000 7f6ab3e56640 1 -- start start 2024-09-21T03:33:42.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:42.473+0000 7f6ab3e56640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6aac0b2e60 0x7f6aac15b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:42.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:42.473+0000 7f6ab3e56640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6aac0cfe00 0x7f6aac1595d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:42.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:42.473+0000 7f6ab3e56640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6aac162590 0x7f6aac155710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:42.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:42.473+0000 7f6ab3e56640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f6aac0b3930 con 0x7f6aac0cfe00 2024-09-21T03:33:42.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:42.473+0000 7f6ab3e56640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f6aac0b37b0 con 0x7f6aac0b2e60 2024-09-21T03:33:42.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:42.473+0000 7f6ab3e56640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f6aac0b3ab0 con 0x7f6aac162590 2024-09-21T03:33:42.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:42.473+0000 7f6ab13f1640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6aac0cfe00 0x7f6aac1595d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:42.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:42.473+0000 7f6ab13f1640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6aac0cfe00 0x7f6aac1595d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35794/0 (socket says 172.21.15.38:35794) 2024-09-21T03:33:42.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:42.473+0000 7f6ab13f1640 1 -- 172.21.15.38:0/1987598815 learned_addr learned my addr 172.21.15.38:0/1987598815 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:33:42.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:42.473+0000 7f6ab1bf2640 1 --2- 172.21.15.38:0/1987598815 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6aac0b2e60 0x7f6aac15b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:42.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:42.473+0000 7f6ab23f3640 1 --2- 172.21.15.38:0/1987598815 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6aac162590 0x7f6aac155710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:42.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:42.473+0000 7f6ab13f1640 1 -- 172.21.15.38:0/1987598815 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6aac162590 msgr2=0x7f6aac155710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:42.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:42.473+0000 7f6ab13f1640 1 --2- 172.21.15.38:0/1987598815 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6aac162590 0x7f6aac155710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:42.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:42.473+0000 7f6ab13f1640 1 -- 172.21.15.38:0/1987598815 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6aac0b2e60 msgr2=0x7f6aac15b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:42.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:42.473+0000 7f6ab13f1640 1 --2- 172.21.15.38:0/1987598815 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6aac0b2e60 0x7f6aac15b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:42.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:42.473+0000 7f6ab13f1640 1 -- 172.21.15.38:0/1987598815 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6aac1eb320 con 0x7f6aac0cfe00 2024-09-21T03:33:42.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:42.473+0000 7f6ab13f1640 1 --2- 172.21.15.38:0/1987598815 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6aac0cfe00 0x7f6aac1595d0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f6a9c00b410 tx=0x7f6a9c00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:42.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:42.473+0000 7f6aa27fc640 1 -- 172.21.15.38:0/1987598815 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a9c015020 con 0x7f6aac0cfe00 2024-09-21T03:33:42.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:42.473+0000 7f6aa27fc640 1 -- 172.21.15.38:0/1987598815 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6a9c011920 con 0x7f6aac0cfe00 2024-09-21T03:33:42.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:42.473+0000 7f6ab3e56640 1 -- 172.21.15.38:0/1987598815 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6aac1eb600 con 0x7f6aac0cfe00 2024-09-21T03:33:42.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:42.473+0000 7f6aa27fc640 1 -- 172.21.15.38:0/1987598815 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a9c011d70 con 0x7f6aac0cfe00 2024-09-21T03:33:42.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:42.473+0000 7f6ab3e56640 1 -- 172.21.15.38:0/1987598815 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6aac1e8890 con 0x7f6aac0cfe00 2024-09-21T03:33:42.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:42.474+0000 7f6ab3e56640 1 -- 172.21.15.38:0/1987598815 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6aac0b3270 con 0x7f6aac0cfe00 2024-09-21T03:33:42.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:42.475+0000 7f6aa27fc640 1 -- 172.21.15.38:0/1987598815 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f6a9c018020 con 0x7f6aac0cfe00 2024-09-21T03:33:42.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:42.476+0000 7f6aa27fc640 1 --2- 172.21.15.38:0/1987598815 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6a90078b90 0x7f6a9007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:42.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:42.476+0000 7f6aa27fc640 1 -- 172.21.15.38:0/1987598815 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f6a9c09ae70 con 0x7f6aac0cfe00 2024-09-21T03:33:42.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:42.476+0000 7f6ab1bf2640 1 --2- 172.21.15.38:0/1987598815 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6a90078b90 0x7f6a9007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:42.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:42.477+0000 7f6ab1bf2640 1 --2- 172.21.15.38:0/1987598815 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6a90078b90 0x7f6a9007b0a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f6aa80098e0 tx=0x7f6aa8009060 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:42.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:42.480+0000 7f6aa27fc640 1 -- 172.21.15.38:0/1987598815 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f6a9c01f020 con 0x7f6aac0cfe00 2024-09-21T03:33:42.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:42.706+0000 7f6ab3e56640 1 -- 172.21.15.38:0/1987598815 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6aac156080 con 0x7f6aac0cfe00 2024-09-21T03:33:42.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:42.706+0000 7f6aa27fc640 1 -- 172.21.15.38:0/1987598815 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5219 (secure 0 0 0) 0x7f6a9c062a40 con 0x7f6aac0cfe00 2024-09-21T03:33:42.708 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:33:42.708 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":7,"btime":"2024-09-21T03:33:41:279513+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4322,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/593838590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":593838590},{"type":"v1","addr":"172.21.15.195:6836","nonce":593838590}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4374,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/663402481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":663402481},{"type":"v1","addr":"172.21.15.38:6833","nonce":663402481}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4377,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/2614829035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":2614829035},{"type":"v1","addr":"172.21.15.38:6835","nonce":2614829035}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:33:35.622623+0000","modified":"2024-09-21T03:33:40.666159+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4402},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4402":{"gid":4402,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.195:6837/1420805977","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":1420805977},{"type":"v1","addr":"172.21.15.195:6837","nonce":1420805977}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":4402,"qdb_cluster":[4402]},"id":1}]} 2024-09-21T03:33:42.709 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 7 2024-09-21T03:33:42.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:42.709+0000 7f6ab3e56640 1 -- 172.21.15.38:0/1987598815 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6a90078b90 msgr2=0x7f6a9007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:42.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:42.709+0000 7f6ab3e56640 1 --2- 172.21.15.38:0/1987598815 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6a90078b90 0x7f6a9007b0a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f6aa80098e0 tx=0x7f6aa8009060 comp rx=0 tx=0).stop 2024-09-21T03:33:42.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:42.709+0000 7f6ab3e56640 1 -- 172.21.15.38:0/1987598815 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6aac0cfe00 msgr2=0x7f6aac1595d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:42.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:42.709+0000 7f6ab3e56640 1 --2- 172.21.15.38:0/1987598815 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6aac0cfe00 0x7f6aac1595d0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f6a9c00b410 tx=0x7f6a9c00b8d0 comp rx=0 tx=0).stop 2024-09-21T03:33:42.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:42.709+0000 7f6ab3e56640 1 -- 172.21.15.38:0/1987598815 shutdown_connections 2024-09-21T03:33:42.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:42.709+0000 7f6ab3e56640 1 --2- 172.21.15.38:0/1987598815 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6a90078b90 0x7f6a9007b0a0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:42.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:42.709+0000 7f6ab3e56640 1 --2- 172.21.15.38:0/1987598815 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6aac162590 0x7f6aac155710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:42.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:42.709+0000 7f6ab3e56640 1 --2- 172.21.15.38:0/1987598815 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6aac0cfe00 0x7f6aac1595d0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:42.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:42.709+0000 7f6ab3e56640 1 --2- 172.21.15.38:0/1987598815 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6aac0b2e60 0x7f6aac15b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:42.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:42.709+0000 7f6ab3e56640 1 -- 172.21.15.38:0/1987598815 >> 172.21.15.38:0/1987598815 conn(0x7f6aac0d4440 msgr2=0x7f6aac06cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:42.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:42.709+0000 7f6ab3e56640 1 -- 172.21.15.38:0/1987598815 shutdown_connections 2024-09-21T03:33:42.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:42.709+0000 7f6ab3e56640 1 -- 172.21.15.38:0/1987598815 wait complete. 2024-09-21T03:33:42.722 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T03:33:42.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:42.836+0000 7f1d1c0d5640 1 -- 172.21.15.38:0/4024761882 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1d14073d50 msgr2=0x7f1d14074160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:42.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:42.836+0000 7f1d1c0d5640 1 --2- 172.21.15.38:0/4024761882 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1d14073d50 0x7f1d14074160 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f1d04005fd0 tx=0x7f1d0402a8a0 comp rx=0 tx=0).stop 2024-09-21T03:33:42.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:42.836+0000 7f1d1c0d5640 1 -- 172.21.15.38:0/4024761882 shutdown_connections 2024-09-21T03:33:42.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:42.836+0000 7f1d1c0d5640 1 --2- 172.21.15.38:0/4024761882 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1d14073d50 0x7f1d14074160 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:42.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:42.836+0000 7f1d1c0d5640 1 --2- 172.21.15.38:0/4024761882 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1d140d0b40 0x7f1d1406e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:42.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:42.836+0000 7f1d1c0d5640 1 -- 172.21.15.38:0/4024761882 >> 172.21.15.38:0/4024761882 conn(0x7f1d140b2cc0 msgr2=0x7f1d140b30d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:42.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:42.836+0000 7f1d1c0d5640 1 -- 172.21.15.38:0/4024761882 shutdown_connections 2024-09-21T03:33:42.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:42.836+0000 7f1d1c0d5640 1 -- 172.21.15.38:0/4024761882 wait complete. 2024-09-21T03:33:42.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:42.838+0000 7f1d1c0d5640 1 Processor -- start 2024-09-21T03:33:42.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:42.838+0000 7f1d1c0d5640 1 -- start start 2024-09-21T03:33:42.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:42.839+0000 7f1d1c0d5640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1d1406ec30 0x7f1d1414ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:42.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:42.839+0000 7f1d1c0d5640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1d14073d50 0x7f1d141f6b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:42.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:42.839+0000 7f1d1c0d5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1d140d0b40 0x7f1d141f70c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:42.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:42.839+0000 7f1d1c0d5640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f1d1415b750 con 0x7f1d140d0b40 2024-09-21T03:33:42.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:42.839+0000 7f1d1c0d5640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f1d1415b5d0 con 0x7f1d1406ec30 2024-09-21T03:33:42.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:42.839+0000 7f1d1c0d5640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f1d1415b8d0 con 0x7f1d14073d50 2024-09-21T03:33:42.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:42.839+0000 7f1d1a672640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1d140d0b40 0x7f1d141f70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:42.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:42.839+0000 7f1d1a672640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1d140d0b40 0x7f1d141f70c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35798/0 (socket says 172.21.15.38:35798) 2024-09-21T03:33:42.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:42.839+0000 7f1d1a672640 1 -- 172.21.15.38:0/1158856136 learned_addr learned my addr 172.21.15.38:0/1158856136 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:33:42.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:42.839+0000 7f1d19e71640 1 --2- 172.21.15.38:0/1158856136 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1d1406ec30 0x7f1d1414ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:42.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:42.839+0000 7f1d19670640 1 --2- 172.21.15.38:0/1158856136 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1d14073d50 0x7f1d141f6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:42.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:42.839+0000 7f1d1a672640 1 -- 172.21.15.38:0/1158856136 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1d14073d50 msgr2=0x7f1d141f6b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:42.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:42.839+0000 7f1d1a672640 1 --2- 172.21.15.38:0/1158856136 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1d14073d50 0x7f1d141f6b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:42.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:42.839+0000 7f1d1a672640 1 -- 172.21.15.38:0/1158856136 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1d1406ec30 msgr2=0x7f1d1414ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:42.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:42.839+0000 7f1d1a672640 1 --2- 172.21.15.38:0/1158856136 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1d1406ec30 0x7f1d1414ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:42.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:42.839+0000 7f1d1a672640 1 -- 172.21.15.38:0/1158856136 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d140cc410 con 0x7f1d140d0b40 2024-09-21T03:33:42.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:42.840+0000 7f1d1a672640 1 --2- 172.21.15.38:0/1158856136 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1d140d0b40 0x7f1d141f70c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f1d04009b30 tx=0x7f1d04032d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:42.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:42.840+0000 7f1d127fc640 1 -- 172.21.15.38:0/1158856136 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d04041070 con 0x7f1d140d0b40 2024-09-21T03:33:42.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:42.840+0000 7f1d127fc640 1 -- 172.21.15.38:0/1158856136 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1d04004450 con 0x7f1d140d0b40 2024-09-21T03:33:42.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:42.840+0000 7f1d1c0d5640 1 -- 172.21.15.38:0/1158856136 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d140cc130 con 0x7f1d140d0b40 2024-09-21T03:33:42.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:42.840+0000 7f1d127fc640 1 -- 172.21.15.38:0/1158856136 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d04031430 con 0x7f1d140d0b40 2024-09-21T03:33:42.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:42.840+0000 7f1d1c0d5640 1 -- 172.21.15.38:0/1158856136 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d141e7860 con 0x7f1d140d0b40 2024-09-21T03:33:42.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:42.841+0000 7f1d1c0d5640 1 -- 172.21.15.38:0/1158856136 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d140750c0 con 0x7f1d140d0b40 2024-09-21T03:33:42.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:42.842+0000 7f1d127fc640 1 -- 172.21.15.38:0/1158856136 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f1d04044070 con 0x7f1d140d0b40 2024-09-21T03:33:42.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:42.842+0000 7f1d127fc640 1 --2- 172.21.15.38:0/1158856136 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1d00078b90 0x7f1d0007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:42.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:42.842+0000 7f1d127fc640 1 -- 172.21.15.38:0/1158856136 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f1d0402e020 con 0x7f1d140d0b40 2024-09-21T03:33:42.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:42.843+0000 7f1d19e71640 1 --2- 172.21.15.38:0/1158856136 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1d00078b90 0x7f1d0007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:42.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:42.843+0000 7f1d19e71640 1 --2- 172.21.15.38:0/1158856136 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1d00078b90 0x7f1d0007b0a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f1d0c009530 tx=0x7f1d0c009260 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:42.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:42.847+0000 7f1d127fc640 1 -- 172.21.15.38:0/1158856136 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f1d140750c0 con 0x7f1d140d0b40 2024-09-21T03:33:43.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.037+0000 7f1d1c0d5640 1 -- 172.21.15.38:0/1158856136 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1d1415bd80 con 0x7f1d140d0b40 2024-09-21T03:33:43.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.038+0000 7f1d127fc640 1 -- 172.21.15.38:0/1158856136 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18626 (secure 0 0 0) 0x7f1d0404b030 con 0x7f1d140d0b40 2024-09-21T03:33:43.041 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:33:43.041 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":26,"fsid":"9d50d66f-656b-4fe2-8e3b-fe53b79c4d43","created":"2024-09-21T03:33:10.677865+0000","modified":"2024-09-21T03:33:40.665991+0000","last_up_change":"2024-09-21T03:33:24.023421+0000","last_in_change":"2024-09-21T03:33:14.296193+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T03:33:22.992573+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T03:33:33.530201+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T03:33:34.585875+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T03:33:38.268346+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"dbab183f-7e63-48b8-960a-ae55fd5bf565","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6801","nonce":2381639433}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6803","nonce":2381639433}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6807","nonce":2381639433}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6805","nonce":2381639433}]},"public_addr":"172.21.15.38:6801/2381639433","cluster_addr":"172.21.15.38:6803/2381639433","heartbeat_back_addr":"172.21.15.38:6807/2381639433","heartbeat_front_addr":"172.21.15.38:6805/2381639433","state":["exists","up"]},{"osd":1,"uuid":"fa1c0e8a-ebc1-4359-8146-bd5342abbc16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6817","nonce":1491818732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6819","nonce":1491818732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6823","nonce":1491818732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6821","nonce":1491818732}]},"public_addr":"172.21.15.38:6817/1491818732","cluster_addr":"172.21.15.38:6819/1491818732","heartbeat_back_addr":"172.21.15.38:6823/1491818732","heartbeat_front_addr":"172.21.15.38:6821/1491818732","state":["exists","up"]},{"osd":2,"uuid":"2ed3cd2d-385d-4fc6-9869-214552a84803","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6825","nonce":2577858902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6827","nonce":2577858902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6831","nonce":2577858902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6829","nonce":2577858902}]},"public_addr":"172.21.15.38:6825/2577858902","cluster_addr":"172.21.15.38:6827/2577858902","heartbeat_back_addr":"172.21.15.38:6831/2577858902","heartbeat_front_addr":"172.21.15.38:6829/2577858902","state":["exists","up"]},{"osd":3,"uuid":"62d1d95e-919d-4250-be71-1d34cfa70dbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6809","nonce":1286810294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6811","nonce":1286810294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6815","nonce":1286810294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6813","nonce":1286810294}]},"public_addr":"172.21.15.38:6809/1286810294","cluster_addr":"172.21.15.38:6811/1286810294","heartbeat_back_addr":"172.21.15.38:6815/1286810294","heartbeat_front_addr":"172.21.15.38:6813/1286810294","state":["exists","up"]},{"osd":4,"uuid":"7c9f9314-99d7-4d73-8392-37809a9f54b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6803","nonce":3401837571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6805","nonce":3401837571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6809","nonce":3401837571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6807","nonce":3401837571}]},"public_addr":"172.21.15.195:6803/3401837571","cluster_addr":"172.21.15.195:6805/3401837571","heartbeat_back_addr":"172.21.15.195:6809/3401837571","heartbeat_front_addr":"172.21.15.195:6807/3401837571","state":["exists","up"]},{"osd":5,"uuid":"bde4fe9f-8414-4bba-bf4a-43973d1976bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6811","nonce":2080960180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6813","nonce":2080960180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6817","nonce":2080960180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6815","nonce":2080960180}]},"public_addr":"172.21.15.195:6811/2080960180","cluster_addr":"172.21.15.195:6813/2080960180","heartbeat_back_addr":"172.21.15.195:6817/2080960180","heartbeat_front_addr":"172.21.15.195:6815/2080960180","state":["exists","up"]},{"osd":6,"uuid":"6ee6d7f4-1a5f-4a4c-ac16-48fe05f6a002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6827","nonce":4055825675}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6829","nonce":4055825675}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6833","nonce":4055825675}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6831","nonce":4055825675}]},"public_addr":"172.21.15.195:6827/4055825675","cluster_addr":"172.21.15.195:6829/4055825675","heartbeat_back_addr":"172.21.15.195:6833/4055825675","heartbeat_front_addr":"172.21.15.195:6831/4055825675","state":["exists","up"]},{"osd":7,"uuid":"efeb5f51-329a-4484-8290-92e7d3503d05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6819","nonce":1606106342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6821","nonce":1606106342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6825","nonce":1606106342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6823","nonce":1606106342}]},"public_addr":"172.21.15.195:6819/1606106342","cluster_addr":"172.21.15.195:6821/1606106342","heartbeat_back_addr":"172.21.15.195:6825/1606106342","heartbeat_front_addr":"172.21.15.195:6823/1606106342","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.901343+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:21.115003+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.901623+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.835238+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.132199+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:19.817560+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.829690+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:21.087300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T03:33:43.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.041+0000 7f1d1c0d5640 1 -- 172.21.15.38:0/1158856136 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1d00078b90 msgr2=0x7f1d0007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:43.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.041+0000 7f1d1c0d5640 1 --2- 172.21.15.38:0/1158856136 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1d00078b90 0x7f1d0007b0a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f1d0c009530 tx=0x7f1d0c009260 comp rx=0 tx=0).stop 2024-09-21T03:33:43.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.041+0000 7f1d1c0d5640 1 -- 172.21.15.38:0/1158856136 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1d140d0b40 msgr2=0x7f1d141f70c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:43.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.041+0000 7f1d1c0d5640 1 --2- 172.21.15.38:0/1158856136 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1d140d0b40 0x7f1d141f70c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f1d04009b30 tx=0x7f1d04032d30 comp rx=0 tx=0).stop 2024-09-21T03:33:43.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.041+0000 7f1d1c0d5640 1 -- 172.21.15.38:0/1158856136 shutdown_connections 2024-09-21T03:33:43.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.041+0000 7f1d1c0d5640 1 --2- 172.21.15.38:0/1158856136 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1d140d0b40 0x7f1d141f70c0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:43.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.041+0000 7f1d1c0d5640 1 --2- 172.21.15.38:0/1158856136 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1d00078b90 0x7f1d0007b0a0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:43.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.041+0000 7f1d1c0d5640 1 --2- 172.21.15.38:0/1158856136 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1d14073d50 0x7f1d141f6b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:43.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.041+0000 7f1d1c0d5640 1 --2- 172.21.15.38:0/1158856136 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1d1406ec30 0x7f1d1414ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:43.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.041+0000 7f1d1c0d5640 1 -- 172.21.15.38:0/1158856136 >> 172.21.15.38:0/1158856136 conn(0x7f1d140b2cc0 msgr2=0x7f1d140b30d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:43.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.041+0000 7f1d1c0d5640 1 -- 172.21.15.38:0/1158856136 shutdown_connections 2024-09-21T03:33:43.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.041+0000 7f1d1c0d5640 1 -- 172.21.15.38:0/1158856136 wait complete. 2024-09-21T03:33:43.456 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T03:33:35.622623+0000', 'modified': '2024-09-21T03:33:40.666159+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4402}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4402': {'gid': 4402, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.195:6837/1420805977', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6835', 'nonce': 1420805977}, {'type': 'v1', 'addr': '172.21.15.195:6837', 'nonce': 1420805977}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 4402, 'qdb_cluster': [4402]} 2024-09-21T03:33:43.456 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-21T03:33:43.456 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-09-21T03:33:43.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.574+0000 7f8fef582640 1 -- 172.21.15.38:0/2303305504 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8fd4004d40 msgr2=0x7f8fd4005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:43.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.574+0000 7f8fef582640 1 --2- 172.21.15.38:0/2303305504 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8fd4004d40 0x7f8fd4005210 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f8fe80cc9e0 tx=0x7f8fd4005830 comp rx=0 tx=0).stop 2024-09-21T03:33:43.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.574+0000 7f8fedb1f640 1 -- 172.21.15.38:0/2303305504 reap_dead start 2024-09-21T03:33:43.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.574+0000 7f8fef582640 1 -- 172.21.15.38:0/2303305504 shutdown_connections 2024-09-21T03:33:43.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.574+0000 7f8fef582640 1 -- 172.21.15.38:0/2303305504 >> 172.21.15.38:0/2303305504 conn(0x7f8fe80b46e0 msgr2=0x7f8fe80d23e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:43.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.574+0000 7f8fef582640 1 -- 172.21.15.38:0/2303305504 shutdown_connections 2024-09-21T03:33:43.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.574+0000 7f8fef582640 1 -- 172.21.15.38:0/2303305504 wait complete. 2024-09-21T03:33:43.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.576+0000 7f8fef582640 1 Processor -- start 2024-09-21T03:33:43.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.576+0000 7f8fef582640 1 -- start start 2024-09-21T03:33:43.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.577+0000 7f8fef582640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8fd4004d40 0x7f8fe81573f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:43.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.577+0000 7f8fef582640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8fe80b3220 0x7f8fe8070af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:43.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.577+0000 7f8fef582640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8fe8071030 0x7f8fe81e30e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:43.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.577+0000 7f8fef582640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f8fe80b62f0 con 0x7f8fe80b3220 2024-09-21T03:33:43.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.577+0000 7f8fef582640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f8fe815ce30 con 0x7f8fd4004d40 2024-09-21T03:33:43.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.577+0000 7f8fef582640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f8fe80b6470 con 0x7f8fe8071030 2024-09-21T03:33:43.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.577+0000 7f8fecb1d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8fe80b3220 0x7f8fe8070af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:43.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.577+0000 7f8fecb1d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8fe80b3220 0x7f8fe8070af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35820/0 (socket says 172.21.15.38:35820) 2024-09-21T03:33:43.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.577+0000 7f8fecb1d640 1 -- 172.21.15.38:0/4096239534 learned_addr learned my addr 172.21.15.38:0/4096239534 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:33:43.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.578+0000 7f8fedb1f640 1 --2- 172.21.15.38:0/4096239534 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8fe8071030 0x7f8fe81e30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:43.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.578+0000 7f8fecb1d640 1 -- 172.21.15.38:0/4096239534 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8fe8071030 msgr2=0x7f8fe81e30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:43.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.578+0000 7f8fecb1d640 1 --2- 172.21.15.38:0/4096239534 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8fe8071030 0x7f8fe81e30e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:43.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.578+0000 7f8fecb1d640 1 -- 172.21.15.38:0/4096239534 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8fd4004d40 msgr2=0x7f8fe81573f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:43.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.578+0000 7f8fed31e640 1 --2- 172.21.15.38:0/4096239534 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8fd4004d40 0x7f8fe81573f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:43.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.578+0000 7f8fecb1d640 1 --2- 172.21.15.38:0/4096239534 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8fd4004d40 0x7f8fe81573f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:43.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.578+0000 7f8fecb1d640 1 -- 172.21.15.38:0/4096239534 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fe81f5fe0 con 0x7f8fe80b3220 2024-09-21T03:33:43.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.578+0000 7f8fecb1d640 1 --2- 172.21.15.38:0/4096239534 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8fe80b3220 0x7f8fe8070af0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f8fd8003ec0 tx=0x7f8fd8001800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:43.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.578+0000 7f8fddffb640 1 -- 172.21.15.38:0/4096239534 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fd8015070 con 0x7f8fe80b3220 2024-09-21T03:33:43.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.578+0000 7f8fddffb640 1 -- 172.21.15.38:0/4096239534 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8fd8011900 con 0x7f8fe80b3220 2024-09-21T03:33:43.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.578+0000 7f8fef582640 1 -- 172.21.15.38:0/4096239534 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fe81e4b80 con 0x7f8fe80b3220 2024-09-21T03:33:43.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.578+0000 7f8fddffb640 1 -- 172.21.15.38:0/4096239534 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fd8011d50 con 0x7f8fe80b3220 2024-09-21T03:33:43.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.578+0000 7f8fef582640 1 -- 172.21.15.38:0/4096239534 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fe81e48a0 con 0x7f8fe80b3220 2024-09-21T03:33:43.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.580+0000 7f8fddffb640 1 -- 172.21.15.38:0/4096239534 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f8fd8018070 con 0x7f8fe80b3220 2024-09-21T03:33:43.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.580+0000 7f8fddffb640 1 --2- 172.21.15.38:0/4096239534 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8fcc078bc0 0x7f8fcc07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:43.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.581+0000 7f8fddffb640 1 -- 172.21.15.38:0/4096239534 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f8fd809bea0 con 0x7f8fe80b3220 2024-09-21T03:33:43.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.581+0000 7f8fed31e640 1 --2- 172.21.15.38:0/4096239534 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8fcc078bc0 0x7f8fcc07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:43.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.581+0000 7f8fed31e640 1 --2- 172.21.15.38:0/4096239534 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8fcc078bc0 0x7f8fcc07b0d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8fe4009a80 tx=0x7f8fe4005f90 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:43.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.581+0000 7f8ff65cd740 1 Processor -- start 2024-09-21T03:33:43.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.581+0000 7f8ff65cd740 1 -- start start 2024-09-21T03:33:43.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.583+0000 7f8ff65cd740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5611a9bc7270 0x5611a9f98fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:43.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.583+0000 7f8ff65cd740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5611a9b9e3b0 0x5611a9f994e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:43.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.583+0000 7f8ff65cd740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5611a9ba6590 0x5611a9fb63e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:43.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.583+0000 7f8ff65cd740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5611a9ee1820 con 0x5611a9b9e3b0 2024-09-21T03:33:43.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.583+0000 7f8ff65cd740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5611a9ed0c50 con 0x5611a9bc7270 2024-09-21T03:33:43.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.583+0000 7f8ff65cd740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x5611a9ed4a60 con 0x5611a9ba6590 2024-09-21T03:33:43.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.583+0000 7f8fecb1d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5611a9b9e3b0 0x5611a9f994e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:43.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.583+0000 7f8fecb1d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5611a9b9e3b0 0x5611a9f994e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35836/0 (socket says 172.21.15.38:35836) 2024-09-21T03:33:43.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.583+0000 7f8fecb1d640 1 -- 172.21.15.38:0/1991702683 learned_addr learned my addr 172.21.15.38:0/1991702683 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:33:43.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.583+0000 7f8fed31e640 1 --2- 172.21.15.38:0/1991702683 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5611a9bc7270 0x5611a9f98fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:43.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.583+0000 7f8fedb1f640 1 --2- 172.21.15.38:0/1991702683 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5611a9ba6590 0x5611a9fb63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:43.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.583+0000 7f8fecb1d640 1 -- 172.21.15.38:0/1991702683 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5611a9ba6590 msgr2=0x5611a9fb63e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:33:43.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.583+0000 7f8fecb1d640 1 --2- 172.21.15.38:0/1991702683 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5611a9ba6590 0x5611a9fb63e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:43.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.583+0000 7f8fecb1d640 1 -- 172.21.15.38:0/1991702683 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5611a9bc7270 msgr2=0x5611a9f98fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:33:43.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.583+0000 7f8fecb1d640 1 --2- 172.21.15.38:0/1991702683 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5611a9bc7270 0x5611a9f98fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:43.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.583+0000 7f8fecb1d640 1 -- 172.21.15.38:0/1991702683 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5611a9c32850 con 0x5611a9b9e3b0 2024-09-21T03:33:43.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.583+0000 7f8fedb1f640 1 --2- 172.21.15.38:0/1991702683 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5611a9ba6590 0x5611a9fb63e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:33:43.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.583+0000 7f8fecb1d640 1 --2- 172.21.15.38:0/1991702683 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5611a9b9e3b0 0x5611a9f994e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f8fd8003e90 tx=0x7f8fd80633a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9d5fd0438573c9d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:43.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.583+0000 7f8fd2ffd640 1 -- 172.21.15.38:0/1991702683 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fd801f040 con 0x5611a9b9e3b0 2024-09-21T03:33:43.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.584+0000 7f8fd2ffd640 1 -- 172.21.15.38:0/1991702683 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8fd8063890 con 0x5611a9b9e3b0 2024-09-21T03:33:43.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.584+0000 7f8fd2ffd640 1 -- 172.21.15.38:0/1991702683 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fd8068370 con 0x5611a9b9e3b0 2024-09-21T03:33:43.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.584+0000 7f8ff65cd740 1 -- 172.21.15.38:0/1991702683 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5611a9b9e3b0 msgr2=0x5611a9f994e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:43.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.584+0000 7f8ff65cd740 1 --2- 172.21.15.38:0/1991702683 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5611a9b9e3b0 0x5611a9f994e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f8fd8003e90 tx=0x7f8fd80633a0 comp rx=0 tx=0).stop 2024-09-21T03:33:43.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.584+0000 7f8ff65cd740 1 -- 172.21.15.38:0/1991702683 shutdown_connections 2024-09-21T03:33:43.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.584+0000 7f8ff65cd740 1 --2- 172.21.15.38:0/1991702683 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5611a9ba6590 0x5611a9fb63e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:43.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.584+0000 7f8ff65cd740 1 --2- 172.21.15.38:0/1991702683 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5611a9b9e3b0 0x5611a9f994e0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:43.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.584+0000 7f8ff65cd740 1 --2- 172.21.15.38:0/1991702683 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5611a9bc7270 0x5611a9f98fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:43.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.584+0000 7f8ff65cd740 1 -- 172.21.15.38:0/1991702683 >> 172.21.15.38:0/1991702683 conn(0x5611a9bbb3c0 msgr2=0x5611a9f6f260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:43.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.584+0000 7f8ff65cd740 1 -- 172.21.15.38:0/1991702683 shutdown_connections 2024-09-21T03:33:43.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.584+0000 7f8ff65cd740 1 -- 172.21.15.38:0/1991702683 wait complete. 2024-09-21T03:33:43.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.585+0000 7f8ff65cd740 1 -- start start 2024-09-21T03:33:43.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.585+0000 7f8ff65cd740 1 Processor -- start 2024-09-21T03:33:43.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.587+0000 7f8ff65cd740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5611a9b9e3b0 0x5611a9f86bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:43.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.587+0000 7f8ff65cd740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5611a9ba6590 0x5611a9fadb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:43.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.587+0000 7f8ff65cd740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5611a9bc7270 0x5611a9facd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:43.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.587+0000 7f8ff65cd740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5611a9c41620 con 0x5611a9ba6590 2024-09-21T03:33:43.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.587+0000 7f8ff65cd740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5611a9e765f0 con 0x5611a9bc7270 2024-09-21T03:33:43.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.587+0000 7f8ff65cd740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x5611a9f99a20 con 0x5611a9b9e3b0 2024-09-21T03:33:43.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.587+0000 7f8ff65cd740 10 client.0 _ll_register_callbacks cb 0x5611a9e7a0a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:33:43.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.587+0000 7f8fecb1d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5611a9ba6590 0x5611a9fadb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:43.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.587+0000 7f8fecb1d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5611a9ba6590 0x5611a9fadb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35838/0 (socket says 172.21.15.38:35838) 2024-09-21T03:33:43.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.587+0000 7f8fecb1d640 1 -- 172.21.15.38:0/3371578756 learned_addr learned my addr 172.21.15.38:0/3371578756 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:33:43.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.587+0000 7f8fed31e640 1 --2- 172.21.15.38:0/3371578756 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5611a9b9e3b0 0x5611a9f86bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:43.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.587+0000 7f8fedb1f640 1 --2- 172.21.15.38:0/3371578756 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5611a9bc7270 0x5611a9facd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:43.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.587+0000 7f8fecb1d640 1 -- 172.21.15.38:0/3371578756 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5611a9b9e3b0 msgr2=0x5611a9f86bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:33:43.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.587+0000 7f8fecb1d640 1 --2- 172.21.15.38:0/3371578756 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5611a9b9e3b0 0x5611a9f86bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:43.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.587+0000 7f8fecb1d640 1 -- 172.21.15.38:0/3371578756 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5611a9bc7270 msgr2=0x5611a9facd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:33:43.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.587+0000 7f8fecb1d640 1 --2- 172.21.15.38:0/3371578756 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5611a9bc7270 0x5611a9facd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:43.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.587+0000 7f8fecb1d640 1 -- 172.21.15.38:0/3371578756 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5611a9ca96b0 con 0x5611a9ba6590 2024-09-21T03:33:43.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.587+0000 7f8fed31e640 1 --2- 172.21.15.38:0/3371578756 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5611a9b9e3b0 0x5611a9f86bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:33:43.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.587+0000 7f8fedb1f640 1 --2- 172.21.15.38:0/3371578756 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5611a9bc7270 0x5611a9facd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:33:43.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.587+0000 7f8fecb1d640 1 --2- 172.21.15.38:0/3371578756 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5611a9ba6590 0x5611a9fadb10 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f8fd8063590 tx=0x7f8fd805f800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed8aaea1a176e4fa server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:43.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.587+0000 7f8fc7fff640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-21T03:33:43.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.588+0000 7f8fc7fff640 1 -- 172.21.15.38:0/3371578756 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fd801f020 con 0x5611a9ba6590 2024-09-21T03:33:43.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.588+0000 7f8fc7fff640 1 -- 172.21.15.38:0/3371578756 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f8fd8064dd0 con 0x5611a9ba6590 2024-09-21T03:33:43.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.588+0000 7f8ff65cd740 1 -- 172.21.15.38:0/3371578756 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8fd8063890 con 0x5611a9ba6590 2024-09-21T03:33:43.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.588+0000 7f8fc7fff640 1 -- 172.21.15.38:0/3371578756 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) ==== 24+0+0 (secure 0 0 0) 0x7f8fd8082050 con 0x5611a9ba6590 2024-09-21T03:33:43.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.588+0000 7f8ff65cd740 10 client.4491 fetch_fsmap learned FSMap version 7 2024-09-21T03:33:43.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.588+0000 7f8ff65cd740 1 -- 172.21.15.38:0/3371578756 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) -- 0x5611a9fabed0 con 0x5611a9ba6590 2024-09-21T03:33:43.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.588+0000 7f8fc7fff640 1 -- 172.21.15.38:0/3371578756 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8fd8063890 con 0x5611a9ba6590 2024-09-21T03:33:43.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.588+0000 7f8fc7fff640 1 -- 172.21.15.38:0/3371578756 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 7) ==== 3510+0+0 (secure 0 0 0) 0x7f8fd8085020 con 0x5611a9ba6590 2024-09-21T03:33:43.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.588+0000 7f8ff65cd740 10 client.4491 fetch_fsmap finished waiting for FSMap version 7 2024-09-21T03:33:43.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.588+0000 7f8ff65cd740 10 client.4491 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-21T03:33:43.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.588+0000 7f8ff65cd740 20 client.4491 populate_metadata read hostname 'smithi038' 2024-09-21T03:33:43.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.588+0000 7f8ff65cd740 1 --2- 172.21.15.38:0/3371578756 >> [v2:172.21.15.195:6835/1420805977,v1:172.21.15.195:6837/1420805977] conn(0x5611a9b85e40 0x5611a9fc6d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:43.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.588+0000 7f8ff65cd740 4 client.4491 mds_command: new command op to 4402 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:33:43.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.588+0000 7f8ff65cd740 1 -- 172.21.15.38:0/3371578756 --> [v2:172.21.15.195:6835/1420805977,v1:172.21.15.195:6837/1420805977] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5611a9ee3770 con 0x5611a9b85e40 2024-09-21T03:33:43.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.588+0000 7f8fed31e640 1 --2- 172.21.15.38:0/3371578756 >> [v2:172.21.15.195:6835/1420805977,v1:172.21.15.195:6837/1420805977] conn(0x5611a9b85e40 0x5611a9fc6d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:43.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.589+0000 7f8fc7fff640 10 client.4491 ms_handle_connect on v2:172.21.15.195:6835/1420805977 2024-09-21T03:33:43.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.589+0000 7f8fed31e640 1 --2- 172.21.15.38:0/3371578756 >> [v2:172.21.15.195:6835/1420805977,v1:172.21.15.195:6837/1420805977] conn(0x5611a9b85e40 0x5611a9fc6d40 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=199d56eaeaf80bc server_cookie=8ff2186d7e7d7759 in_seq=0 out_seq=0 2024-09-21T03:33:43.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.590+0000 7f8fc7fff640 1 -- 172.21.15.38:0/3371578756 <== mds.0 v2:172.21.15.195:6835/1420805977 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5611a9ee3770 con 0x5611a9b85e40 2024-09-21T03:33:43.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.590+0000 7f8fc7fff640 10 client.4491 handle_command_reply: tid=0 2024-09-21T03:33:43.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.590+0000 7f8ff65cd740 1 client.4491 shutdown 2024-09-21T03:33:43.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.590+0000 7f8ff65cd740 1 -- 172.21.15.38:0/3371578756 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5611a9ba6590 msgr2=0x5611a9fadb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:43.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.590+0000 7f8ff65cd740 1 --2- 172.21.15.38:0/3371578756 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5611a9ba6590 0x5611a9fadb10 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f8fd8063590 tx=0x7f8fd805f800 comp rx=0 tx=0).stop 2024-09-21T03:33:43.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.590+0000 7f8ff65cd740 1 -- 172.21.15.38:0/3371578756 shutdown_connections 2024-09-21T03:33:43.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.590+0000 7f8ff65cd740 1 --2- 172.21.15.38:0/3371578756 >> [v2:172.21.15.195:6835/1420805977,v1:172.21.15.195:6837/1420805977] conn(0x5611a9b85e40 0x5611a9fc6d40 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:43.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.590+0000 7f8ff65cd740 1 --2- 172.21.15.38:0/3371578756 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5611a9bc7270 0x5611a9facd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:43.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.590+0000 7f8fc7fff640 0 client.4491 ms_handle_reset on v2:172.21.15.195:6835/1420805977 2024-09-21T03:33:43.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.590+0000 7f8ff65cd740 1 --2- 172.21.15.38:0/3371578756 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5611a9ba6590 0x5611a9fadb10 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:43.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.590+0000 7f8ff65cd740 1 --2- 172.21.15.38:0/3371578756 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5611a9b9e3b0 0x5611a9f86bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:43.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.590+0000 7f8ff65cd740 1 -- 172.21.15.38:0/3371578756 >> 172.21.15.38:0/3371578756 conn(0x5611a9bbb3c0 msgr2=0x5611a9f88760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:43.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.590+0000 7f8ff65cd740 1 -- 172.21.15.38:0/3371578756 shutdown_connections 2024-09-21T03:33:43.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.590+0000 7f8ff65cd740 1 -- 172.21.15.38:0/3371578756 wait complete. 2024-09-21T03:33:43.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.590+0000 7f8ff65cd740 20 client.4491 trim_cache size 0 max 16384 2024-09-21T03:33:43.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.609+0000 7f8ff65cd740 1 Processor -- start 2024-09-21T03:33:43.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.609+0000 7f8ff65cd740 1 -- start start 2024-09-21T03:33:43.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.610+0000 7f8ff65cd740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5611a9b85e40 0x5611a9facce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:43.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.610+0000 7f8ff65cd740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5611a9b9e3b0 0x5611a9f86bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:43.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.610+0000 7f8ff65cd740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5611a9ba6590 0x5611a9f98ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:43.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.610+0000 7f8ff65cd740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f8fe8202590 con 0x5611a9b85e40 2024-09-21T03:33:43.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.610+0000 7f8ff65cd740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f8fe81f9380 con 0x5611a9ba6590 2024-09-21T03:33:43.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.610+0000 7f8ff65cd740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f8fe8201f70 con 0x5611a9b9e3b0 2024-09-21T03:33:43.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.610+0000 7f8fed31e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5611a9b85e40 0x5611a9facce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:43.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.610+0000 7f8fed31e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5611a9b85e40 0x5611a9facce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35844/0 (socket says 172.21.15.38:35844) 2024-09-21T03:33:43.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.610+0000 7f8fed31e640 1 -- 172.21.15.38:0/2027379330 learned_addr learned my addr 172.21.15.38:0/2027379330 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:33:43.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.610+0000 7f8fecb1d640 1 --2- 172.21.15.38:0/2027379330 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5611a9b9e3b0 0x5611a9f86bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:43.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.610+0000 7f8fedb1f640 1 --2- 172.21.15.38:0/2027379330 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5611a9ba6590 0x5611a9f98ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:43.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.610+0000 7f8fed31e640 1 -- 172.21.15.38:0/2027379330 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5611a9b9e3b0 msgr2=0x5611a9f86bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:33:43.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.610+0000 7f8fed31e640 1 --2- 172.21.15.38:0/2027379330 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5611a9b9e3b0 0x5611a9f86bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:43.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.610+0000 7f8fed31e640 1 -- 172.21.15.38:0/2027379330 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5611a9ba6590 msgr2=0x5611a9f98ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:33:43.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.610+0000 7f8fed31e640 1 --2- 172.21.15.38:0/2027379330 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5611a9ba6590 0x5611a9f98ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:43.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.610+0000 7f8fed31e640 1 -- 172.21.15.38:0/2027379330 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5611a9f6be30 con 0x5611a9b85e40 2024-09-21T03:33:43.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.610+0000 7f8fed31e640 1 --2- 172.21.15.38:0/2027379330 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5611a9b85e40 0x5611a9facce0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f8fe40115d0 tx=0x7f8fe4016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7835fff3e1979c15 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:43.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.610+0000 7f8fd27fc640 1 -- 172.21.15.38:0/2027379330 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fe402c070 con 0x5611a9b85e40 2024-09-21T03:33:43.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.611+0000 7f8fd27fc640 1 -- 172.21.15.38:0/2027379330 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8fe402a630 con 0x5611a9b85e40 2024-09-21T03:33:43.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.611+0000 7f8fd27fc640 1 -- 172.21.15.38:0/2027379330 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fe402aa60 con 0x5611a9b85e40 2024-09-21T03:33:43.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.611+0000 7f8ff65cd740 1 -- 172.21.15.38:0/2027379330 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5611a9b85e40 msgr2=0x5611a9facce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:43.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.611+0000 7f8ff65cd740 1 --2- 172.21.15.38:0/2027379330 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5611a9b85e40 0x5611a9facce0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f8fe40115d0 tx=0x7f8fe4016000 comp rx=0 tx=0).stop 2024-09-21T03:33:43.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.611+0000 7f8ff65cd740 1 -- 172.21.15.38:0/2027379330 shutdown_connections 2024-09-21T03:33:43.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.611+0000 7f8ff65cd740 1 --2- 172.21.15.38:0/2027379330 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5611a9ba6590 0x5611a9f98ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:43.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.611+0000 7f8ff65cd740 1 --2- 172.21.15.38:0/2027379330 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5611a9b9e3b0 0x5611a9f86bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:43.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.611+0000 7f8ff65cd740 1 --2- 172.21.15.38:0/2027379330 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5611a9b85e40 0x5611a9facce0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:43.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.611+0000 7f8ff65cd740 1 -- 172.21.15.38:0/2027379330 >> 172.21.15.38:0/2027379330 conn(0x5611a9bbb3c0 msgr2=0x5611a9f6a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:43.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.611+0000 7f8ff65cd740 1 -- 172.21.15.38:0/2027379330 shutdown_connections 2024-09-21T03:33:43.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.611+0000 7f8ff65cd740 1 -- 172.21.15.38:0/2027379330 wait complete. 2024-09-21T03:33:43.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.612+0000 7f8ff65cd740 1 -- start start 2024-09-21T03:33:43.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.612+0000 7f8ff65cd740 1 Processor -- start 2024-09-21T03:33:43.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.614+0000 7f8ff65cd740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5611a9b85e40 0x5611aa0325c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:43.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.614+0000 7f8ff65cd740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5611a9b9e3b0 0x5611a9fb1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:43.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.614+0000 7f8ff65cd740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5611a9ba6590 0x5611a9fd6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:43.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.614+0000 7f8ff65cd740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5611a9c41620 con 0x5611a9ba6590 2024-09-21T03:33:43.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.614+0000 7f8ff65cd740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5611a9f99a20 con 0x5611a9b85e40 2024-09-21T03:33:43.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.614+0000 7f8ff65cd740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x5611a9e765f0 con 0x5611a9b9e3b0 2024-09-21T03:33:43.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.614+0000 7f8ff65cd740 10 client.0 _ll_register_callbacks cb 0x7f8fd80631e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:33:43.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.614+0000 7f8fedb1f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5611a9ba6590 0x5611a9fd6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:43.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.614+0000 7f8fedb1f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5611a9ba6590 0x5611a9fd6c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35854/0 (socket says 172.21.15.38:35854) 2024-09-21T03:33:43.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.614+0000 7f8fedb1f640 1 -- 172.21.15.38:0/3268729454 learned_addr learned my addr 172.21.15.38:0/3268729454 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:33:43.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.614+0000 7f8fed31e640 1 --2- 172.21.15.38:0/3268729454 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5611a9b85e40 0x5611aa0325c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:43.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.614+0000 7f8fecb1d640 1 --2- 172.21.15.38:0/3268729454 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5611a9b9e3b0 0x5611a9fb1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:43.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.614+0000 7f8fedb1f640 1 -- 172.21.15.38:0/3268729454 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5611a9b9e3b0 msgr2=0x5611a9fb1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:33:43.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.614+0000 7f8fedb1f640 1 --2- 172.21.15.38:0/3268729454 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5611a9b9e3b0 0x5611a9fb1210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:43.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.614+0000 7f8fedb1f640 1 -- 172.21.15.38:0/3268729454 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5611a9b85e40 msgr2=0x5611aa0325c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:33:43.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.614+0000 7f8fedb1f640 1 --2- 172.21.15.38:0/3268729454 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5611a9b85e40 0x5611aa0325c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:43.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.614+0000 7f8fedb1f640 1 -- 172.21.15.38:0/3268729454 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5611a9f6f290 con 0x5611a9ba6590 2024-09-21T03:33:43.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.614+0000 7f8fed31e640 1 --2- 172.21.15.38:0/3268729454 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5611a9b85e40 0x5611aa0325c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:33:43.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.614+0000 7f8fedb1f640 1 --2- 172.21.15.38:0/3268729454 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5611a9ba6590 0x5611a9fd6c10 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f8fe000c650 tx=0x7f8fe0001560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf6bbac915bc3e5a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:43.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.614+0000 7f8fc7fff640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-21T03:33:43.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.614+0000 7f8fc7fff640 1 -- 172.21.15.38:0/3268729454 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fe0003b70 con 0x5611a9ba6590 2024-09-21T03:33:43.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.615+0000 7f8fc7fff640 1 -- 172.21.15.38:0/3268729454 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f8fe0003d10 con 0x5611a9ba6590 2024-09-21T03:33:43.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.615+0000 7f8ff65cd740 1 -- 172.21.15.38:0/3268729454 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8fe402a630 con 0x5611a9ba6590 2024-09-21T03:33:43.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.615+0000 7f8fc7fff640 1 -- 172.21.15.38:0/3268729454 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) ==== 24+0+0 (secure 0 0 0) 0x7f8fe004a050 con 0x5611a9ba6590 2024-09-21T03:33:43.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.615+0000 7f8ff65cd740 10 client.4497 fetch_fsmap learned FSMap version 7 2024-09-21T03:33:43.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.615+0000 7f8ff65cd740 1 -- 172.21.15.38:0/3268729454 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) -- 0x5611a9fb1b00 con 0x5611a9ba6590 2024-09-21T03:33:43.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.615+0000 7f8fc7fff640 1 -- 172.21.15.38:0/3268729454 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8fe402a630 con 0x5611a9ba6590 2024-09-21T03:33:43.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.615+0000 7f8fc7fff640 1 -- 172.21.15.38:0/3268729454 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 7) ==== 3510+0+0 (secure 0 0 0) 0x7f8fe004d020 con 0x5611a9ba6590 2024-09-21T03:33:43.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.615+0000 7f8ff65cd740 10 client.4497 fetch_fsmap finished waiting for FSMap version 7 2024-09-21T03:33:43.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.615+0000 7f8ff65cd740 10 client.4497 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-21T03:33:43.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.615+0000 7f8ff65cd740 20 client.4497 populate_metadata read hostname 'smithi038' 2024-09-21T03:33:43.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.615+0000 7f8ff65cd740 1 --2- 172.21.15.38:0/3268729454 >> [v2:172.21.15.195:6835/1420805977,v1:172.21.15.195:6837/1420805977] conn(0x5611a9b5b280 0x5611a9fae190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:43.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.615+0000 7f8ff65cd740 4 client.4497 mds_command: new command op to 4402 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T03:33:43.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.615+0000 7f8ff65cd740 1 -- 172.21.15.38:0/3268729454 --> [v2:172.21.15.195:6835/1420805977,v1:172.21.15.195:6837/1420805977] -- command(tid 0: {"prefix": "status"}) -- 0x5611a9edf220 con 0x5611a9b5b280 2024-09-21T03:33:43.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.615+0000 7f8fed31e640 1 --2- 172.21.15.38:0/3268729454 >> [v2:172.21.15.195:6835/1420805977,v1:172.21.15.195:6837/1420805977] conn(0x5611a9b5b280 0x5611a9fae190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:43.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.616+0000 7f8fc7fff640 10 client.4497 ms_handle_connect on v2:172.21.15.195:6835/1420805977 2024-09-21T03:33:43.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.616+0000 7f8fed31e640 1 --2- 172.21.15.38:0/3268729454 >> [v2:172.21.15.195:6835/1420805977,v1:172.21.15.195:6837/1420805977] conn(0x5611a9b5b280 0x5611a9fae190 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b641218c83016040 server_cookie=f22dc4976cd824d8 in_seq=0 out_seq=0 2024-09-21T03:33:43.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.616+0000 7f8fc7fff640 1 -- 172.21.15.38:0/3268729454 <== mds.0 v2:172.21.15.195:6835/1420805977 1 ==== command_reply(tid 0: 0 ) ==== 8+0+324 (crc 0 0 0) 0x5611a9edf220 con 0x5611a9b5b280 2024-09-21T03:33:43.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.616+0000 7f8fc7fff640 10 client.4497 handle_command_reply: tid=0 2024-09-21T03:33:43.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.616+0000 7f8ff65cd740 1 client.4497 shutdown 2024-09-21T03:33:43.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.616+0000 7f8ff65cd740 1 -- 172.21.15.38:0/3268729454 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5611a9ba6590 msgr2=0x5611a9fd6c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:43.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.616+0000 7f8ff65cd740 1 --2- 172.21.15.38:0/3268729454 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5611a9ba6590 0x5611a9fd6c10 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f8fe000c650 tx=0x7f8fe0001560 comp rx=0 tx=0).stop 2024-09-21T03:33:43.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.616+0000 7f8ff65cd740 1 -- 172.21.15.38:0/3268729454 shutdown_connections 2024-09-21T03:33:43.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.616+0000 7f8ff65cd740 1 --2- 172.21.15.38:0/3268729454 >> [v2:172.21.15.195:6835/1420805977,v1:172.21.15.195:6837/1420805977] conn(0x5611a9b5b280 0x5611a9fae190 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:43.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.616+0000 7f8ff65cd740 1 --2- 172.21.15.38:0/3268729454 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5611a9ba6590 0x5611a9fd6c10 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:43.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.616+0000 7f8ff65cd740 1 --2- 172.21.15.38:0/3268729454 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5611a9b9e3b0 0x5611a9fb1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:43.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.616+0000 7f8ff65cd740 1 --2- 172.21.15.38:0/3268729454 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5611a9b85e40 0x5611aa0325c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:43.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.616+0000 7f8ff65cd740 1 -- 172.21.15.38:0/3268729454 >> 172.21.15.38:0/3268729454 conn(0x5611a9bbb3c0 msgr2=0x5611a9fc5a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:43.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.616+0000 7f8fc7fff640 0 client.4497 ms_handle_reset on v2:172.21.15.195:6835/1420805977 2024-09-21T03:33:43.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.616+0000 7f8ff65cd740 1 -- 172.21.15.38:0/3268729454 shutdown_connections 2024-09-21T03:33:43.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.616+0000 7f8ff65cd740 1 -- 172.21.15.38:0/3268729454 wait complete. 2024-09-21T03:33:43.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.616+0000 7f8ff65cd740 20 client.4497 trim_cache size 0 max 16384 2024-09-21T03:33:43.623 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-09-21T03:33:43.624 INFO:teuthology.orchestra.run.smithi038.stdout: "cluster_fsid": "9d50d66f-656b-4fe2-8e3b-fe53b79c4d43", 2024-09-21T03:33:43.624 INFO:teuthology.orchestra.run.smithi038.stdout: "whoami": 0, 2024-09-21T03:33:43.624 INFO:teuthology.orchestra.run.smithi038.stdout: "id": 4402, 2024-09-21T03:33:43.624 INFO:teuthology.orchestra.run.smithi038.stdout: "want_state": "up:active", 2024-09-21T03:33:43.624 INFO:teuthology.orchestra.run.smithi038.stdout: "state": "up:active", 2024-09-21T03:33:43.624 INFO:teuthology.orchestra.run.smithi038.stdout: "fs_name": "cephfs", 2024-09-21T03:33:43.624 INFO:teuthology.orchestra.run.smithi038.stdout: "rank_uptime": 7.3821051100000004, 2024-09-21T03:33:43.624 INFO:teuthology.orchestra.run.smithi038.stdout: "mdsmap_epoch": 7, 2024-09-21T03:33:43.624 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch": 26, 2024-09-21T03:33:43.624 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch_barrier": 26, 2024-09-21T03:33:43.624 INFO:teuthology.orchestra.run.smithi038.stdout: "uptime": 13.403101882 2024-09-21T03:33:43.624 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-09-21T03:33:43.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.617+0000 7f8fef582640 1 -- 172.21.15.38:0/4096239534 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8fcc078bc0 msgr2=0x7f8fcc07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:43.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.617+0000 7f8fef582640 1 --2- 172.21.15.38:0/4096239534 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8fcc078bc0 0x7f8fcc07b0d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8fe4009a80 tx=0x7f8fe4005f90 comp rx=0 tx=0).stop 2024-09-21T03:33:43.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.617+0000 7f8fef582640 1 -- 172.21.15.38:0/4096239534 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8fe80b3220 msgr2=0x7f8fe8070af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:43.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.617+0000 7f8fef582640 1 --2- 172.21.15.38:0/4096239534 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8fe80b3220 0x7f8fe8070af0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f8fd8003ec0 tx=0x7f8fd8001800 comp rx=0 tx=0).stop 2024-09-21T03:33:43.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.617+0000 7f8fef582640 1 -- 172.21.15.38:0/4096239534 shutdown_connections 2024-09-21T03:33:43.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.617+0000 7f8fef582640 1 --2- 172.21.15.38:0/4096239534 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8fcc078bc0 0x7f8fcc07b0d0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:43.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.617+0000 7f8fef582640 1 --2- 172.21.15.38:0/4096239534 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8fe8071030 0x7f8fe81e30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:43.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.617+0000 7f8fef582640 1 --2- 172.21.15.38:0/4096239534 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8fe80b3220 0x7f8fe8070af0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:43.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.617+0000 7f8fef582640 1 --2- 172.21.15.38:0/4096239534 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8fd4004d40 0x7f8fe81573f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:43.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.617+0000 7f8fef582640 1 -- 172.21.15.38:0/4096239534 >> 172.21.15.38:0/4096239534 conn(0x7f8fe80b46e0 msgr2=0x7f8fe8070100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:43.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.617+0000 7f8fef582640 1 -- 172.21.15.38:0/4096239534 shutdown_connections 2024-09-21T03:33:43.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.617+0000 7f8fef582640 1 -- 172.21.15.38:0/4096239534 wait complete. 2024-09-21T03:33:43.630 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-21T03:33:43.630 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-09-21T03:33:43.630 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T03:33:43.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.745+0000 7f87b94f2640 1 -- 172.21.15.38:0/116235912 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f87b4162590 msgr2=0x7f87b41629a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:43.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.745+0000 7f87b94f2640 1 --2- 172.21.15.38:0/116235912 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f87b4162590 0x7f87b41629a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f879c009860 tx=0x7f879c02a540 comp rx=0 tx=0).stop 2024-09-21T03:33:43.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.745+0000 7f87b94f2640 1 -- 172.21.15.38:0/116235912 shutdown_connections 2024-09-21T03:33:43.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.745+0000 7f87b94f2640 1 --2- 172.21.15.38:0/116235912 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f87b40b2e60 0x7f87b40b3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:43.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.745+0000 7f87b94f2640 1 --2- 172.21.15.38:0/116235912 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f87b4162590 0x7f87b41629a0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:43.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.745+0000 7f87b94f2640 1 -- 172.21.15.38:0/116235912 >> 172.21.15.38:0/116235912 conn(0x7f87b40d4440 msgr2=0x7f87b40d4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:43.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.745+0000 7f87b94f2640 1 -- 172.21.15.38:0/116235912 shutdown_connections 2024-09-21T03:33:43.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.745+0000 7f87b94f2640 1 -- 172.21.15.38:0/116235912 wait complete. 2024-09-21T03:33:43.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.747+0000 7f87b94f2640 1 Processor -- start 2024-09-21T03:33:43.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.747+0000 7f87b94f2640 1 -- start start 2024-09-21T03:33:43.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.748+0000 7f87b94f2640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f87b40b2e60 0x7f87b414fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:43.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.748+0000 7f87b94f2640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f87b40cfe00 0x7f87b415afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:43.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.748+0000 7f87b94f2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f87b4162590 0x7f87b41f62b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:43.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.748+0000 7f87b94f2640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f87b40b3b40 con 0x7f87b4162590 2024-09-21T03:33:43.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.748+0000 7f87b94f2640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f87b40b39c0 con 0x7f87b40cfe00 2024-09-21T03:33:43.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.748+0000 7f87b94f2640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f87b40cf590 con 0x7f87b40b2e60 2024-09-21T03:33:43.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.749+0000 7f87b37fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f87b4162590 0x7f87b41f62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:43.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.749+0000 7f87b37fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f87b4162590 0x7f87b41f62b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35884/0 (socket says 172.21.15.38:35884) 2024-09-21T03:33:43.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.749+0000 7f87b37fe640 1 -- 172.21.15.38:0/2060224778 learned_addr learned my addr 172.21.15.38:0/2060224778 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:33:43.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.749+0000 7f87b2ffd640 1 --2- 172.21.15.38:0/2060224778 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f87b40b2e60 0x7f87b414fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:43.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.749+0000 7f87b27fc640 1 --2- 172.21.15.38:0/2060224778 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f87b40cfe00 0x7f87b415afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:43.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.749+0000 7f87b37fe640 1 -- 172.21.15.38:0/2060224778 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f87b40b2e60 msgr2=0x7f87b414fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:43.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.749+0000 7f87b37fe640 1 --2- 172.21.15.38:0/2060224778 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f87b40b2e60 0x7f87b414fa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:43.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.749+0000 7f87b37fe640 1 -- 172.21.15.38:0/2060224778 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f87b40cfe00 msgr2=0x7f87b415afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:43.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.749+0000 7f87b37fe640 1 --2- 172.21.15.38:0/2060224778 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f87b40cfe00 0x7f87b415afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:43.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.749+0000 7f87b37fe640 1 -- 172.21.15.38:0/2060224778 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87b41f9c20 con 0x7f87b4162590 2024-09-21T03:33:43.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.749+0000 7f87b2ffd640 1 --2- 172.21.15.38:0/2060224778 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f87b40b2e60 0x7f87b414fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:33:43.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.749+0000 7f87b37fe640 1 --2- 172.21.15.38:0/2060224778 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f87b4162590 0x7f87b41f62b0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f87a400eab0 tx=0x7f87a400ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:43.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.749+0000 7f87ab7fe640 1 -- 172.21.15.38:0/2060224778 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87a400ce40 con 0x7f87b4162590 2024-09-21T03:33:43.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.749+0000 7f87ab7fe640 1 -- 172.21.15.38:0/2060224778 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f87a4004b20 con 0x7f87b4162590 2024-09-21T03:33:43.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.749+0000 7f87b94f2640 1 -- 172.21.15.38:0/2060224778 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87b41f9ec0 con 0x7f87b4162590 2024-09-21T03:33:43.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.749+0000 7f87ab7fe640 1 -- 172.21.15.38:0/2060224778 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87a4005020 con 0x7f87b4162590 2024-09-21T03:33:43.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.749+0000 7f87b94f2640 1 -- 172.21.15.38:0/2060224778 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87b415ea10 con 0x7f87b4162590 2024-09-21T03:33:43.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.750+0000 7f87b94f2640 1 -- 172.21.15.38:0/2060224778 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87b415e780 con 0x7f87b4162590 2024-09-21T03:33:43.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.751+0000 7f87ab7fe640 1 -- 172.21.15.38:0/2060224778 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f87a40051c0 con 0x7f87b4162590 2024-09-21T03:33:43.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.751+0000 7f87ab7fe640 1 --2- 172.21.15.38:0/2060224778 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8794078b90 0x7f879407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:43.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.752+0000 7f87ab7fe640 1 -- 172.21.15.38:0/2060224778 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f87a409bc50 con 0x7f87b4162590 2024-09-21T03:33:43.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.752+0000 7f87b2ffd640 1 --2- 172.21.15.38:0/2060224778 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8794078b90 0x7f879407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:43.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.752+0000 7f87b2ffd640 1 --2- 172.21.15.38:0/2060224778 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8794078b90 0x7f879407b0a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f87ac000f60 tx=0x7f87ac009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:43.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.756+0000 7f87ab7fe640 1 -- 172.21.15.38:0/2060224778 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f87a40a1020 con 0x7f87b4162590 2024-09-21T03:33:43.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.983+0000 7f87b94f2640 1 -- 172.21.15.38:0/2060224778 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f87b41f5b80 con 0x7f87b4162590 2024-09-21T03:33:43.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.983+0000 7f87ab7fe640 1 -- 172.21.15.38:0/2060224778 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) ==== 53+0+88 (secure 0 0 0) 0x7f87a4063de0 con 0x7f87b4162590 2024-09-21T03:33:43.987 INFO:teuthology.orchestra.run.smithi038.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-21T03:33:43.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.986+0000 7f87b94f2640 1 -- 172.21.15.38:0/2060224778 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8794078b90 msgr2=0x7f879407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:43.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.986+0000 7f87b94f2640 1 --2- 172.21.15.38:0/2060224778 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8794078b90 0x7f879407b0a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f87ac000f60 tx=0x7f87ac009040 comp rx=0 tx=0).stop 2024-09-21T03:33:43.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.986+0000 7f87b94f2640 1 -- 172.21.15.38:0/2060224778 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f87b4162590 msgr2=0x7f87b41f62b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:43.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.986+0000 7f87b94f2640 1 --2- 172.21.15.38:0/2060224778 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f87b4162590 0x7f87b41f62b0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f87a400eab0 tx=0x7f87a400ef70 comp rx=0 tx=0).stop 2024-09-21T03:33:43.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.986+0000 7f87b94f2640 1 -- 172.21.15.38:0/2060224778 shutdown_connections 2024-09-21T03:33:43.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.986+0000 7f87b94f2640 1 --2- 172.21.15.38:0/2060224778 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f87b4162590 0x7f87b41f62b0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:43.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.986+0000 7f87b94f2640 1 --2- 172.21.15.38:0/2060224778 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f87b40cfe00 0x7f87b415afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:43.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.986+0000 7f87b94f2640 1 --2- 172.21.15.38:0/2060224778 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8794078b90 0x7f879407b0a0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:43.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.986+0000 7f87b94f2640 1 --2- 172.21.15.38:0/2060224778 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f87b40b2e60 0x7f87b414fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:43.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.986+0000 7f87b94f2640 1 -- 172.21.15.38:0/2060224778 >> 172.21.15.38:0/2060224778 conn(0x7f87b40d4440 msgr2=0x7f87b406cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:43.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.986+0000 7f87b94f2640 1 -- 172.21.15.38:0/2060224778 shutdown_connections 2024-09-21T03:33:43.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:43.986+0000 7f87b94f2640 1 -- 172.21.15.38:0/2060224778 wait complete. 2024-09-21T03:33:43.999 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-21T03:33:43.999 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T03:33:43.999 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi038.front.sepia.ceph.com 2024-09-21T03:33:43.999 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-09-21T03:33:43.999 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-09-21T03:33:43.999 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-21T03:33:43.999 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-21T03:33:43.999 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T03:33:43.999 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-09-21T03:33:43.999 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:33:43.999 DEBUG:teuthology.orchestra.run.smithi038:> ip addr 2024-09-21T03:33:44.015 INFO:teuthology.orchestra.run.smithi038.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T03:33:44.015 INFO:teuthology.orchestra.run.smithi038.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T03:33:44.016 INFO:teuthology.orchestra.run.smithi038.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T03:33:44.016 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:33:44.016 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 ::1/128 scope host 2024-09-21T03:33:44.016 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:33:44.016 INFO:teuthology.orchestra.run.smithi038.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T03:33:44.016 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:6b:fd:a2 brd ff:ff:ff:ff:ff:ff 2024-09-21T03:33:44.016 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp7s0f0 2024-09-21T03:33:44.016 INFO:teuthology.orchestra.run.smithi038.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T03:33:44.016 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:6b:fd:a3 brd ff:ff:ff:ff:ff:ff 2024-09-21T03:33:44.016 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp7s0f1 2024-09-21T03:33:44.016 INFO:teuthology.orchestra.run.smithi038.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T03:33:44.016 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:bd:15:fc brd ff:ff:ff:ff:ff:ff 2024-09-21T03:33:44.017 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp1s0f0 2024-09-21T03:33:44.017 INFO:teuthology.orchestra.run.smithi038.stdout: inet 172.21.15.38/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-21T03:33:44.017 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft 42785sec preferred_lft 42785sec 2024-09-21T03:33:44.017 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 fe80::ec4:7aff:febd:15fc/64 scope link 2024-09-21T03:33:44.017 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:33:44.017 INFO:teuthology.orchestra.run.smithi038.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T03:33:44.017 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:bd:15:fd brd ff:ff:ff:ff:ff:ff 2024-09-21T03:33:44.017 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp1s0f1 2024-09-21T03:33:44.017 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-21T03:33:44.018 DEBUG:teuthology.orchestra.run.smithi038:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T03:33:44.018 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-09-21T03:33:44.018 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip link add name ceph-brx type bridge 2024-09-21T03:33:44.018 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip addr flush dev ceph-brx 2024-09-21T03:33:44.018 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip link set ceph-brx up 2024-09-21T03:33:44.018 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-21T03:33:44.018 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-09-21T03:33:44.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:44 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T03:33:44.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:33:44.216 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:33:44.216 DEBUG:teuthology.orchestra.run.smithi038:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T03:33:44.242 INFO:teuthology.orchestra.run.smithi038.stdout:1 2024-09-21T03:33:44.243 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:33:44.243 DEBUG:teuthology.orchestra.run.smithi038:> route 2024-09-21T03:33:44.308 INFO:teuthology.orchestra.run.smithi038.stdout:Kernel IP routing table 2024-09-21T03:33:44.308 INFO:teuthology.orchestra.run.smithi038.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T03:33:44.308 INFO:teuthology.orchestra.run.smithi038.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-21T03:33:44.308 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-21T03:33:44.308 INFO:teuthology.orchestra.run.smithi038.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T03:33:44.308 DEBUG:teuthology.orchestra.run.smithi038:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T03:33:44.308 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-09-21T03:33:44.309 DEBUG:teuthology.orchestra.run.smithi038:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-21T03:33:44.309 DEBUG:teuthology.orchestra.run.smithi038:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-21T03:33:44.309 DEBUG:teuthology.orchestra.run.smithi038:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-21T03:33:44.309 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-09-21T03:33:44.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:44 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T03:33:44.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:33:44.609 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:33:44.610 DEBUG:teuthology.orchestra.run.smithi038:> ip netns list 2024-09-21T03:33:44.626 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:33:44.626 DEBUG:teuthology.orchestra.run.smithi038:> ip netns list-id 2024-09-21T03:33:44.683 DEBUG:teuthology.orchestra.run.smithi038:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T03:33:44.683 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-09-21T03:33:44.683 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-21T03:33:44.683 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-09-21T03:33:44.683 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-09-21T03:33:44.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:44 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T03:33:44.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:33:44.803 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-09-21T03:33:44.803 DEBUG:teuthology.orchestra.run.smithi038:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T03:33:44.803 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-09-21T03:33:44.804 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-09-21T03:33:44.804 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-21T03:33:44.804 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-09-21T03:33:44.804 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-09-21T03:33:44.804 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-09-21T03:33:44.804 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-09-21T03:33:44.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:44 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T03:33:44.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:33:44.988 DEBUG:teuthology.orchestra.run.smithi038:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T03:33:44.988 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-09-21T03:33:44.988 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip link set brx.0 up 2024-09-21T03:33:44.989 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip link set dev brx.0 master ceph-brx 2024-09-21T03:33:44.989 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-09-21T03:33:45.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:45 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T03:33:45.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:33:45.069 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-09-21T03:33:45.069 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T03:33:45.069 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-09-21T03:33:45.097 INFO:teuthology.orchestra.run.smithi038.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-09-21T03:33:45.097 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T03:33:45.097 DEBUG:teuthology.orchestra.run.smithi038:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-09-21T03:33:45.153 DEBUG:teuthology.orchestra.run.smithi038:> sudo modprobe fuse 2024-09-21T03:33:45.220 DEBUG:teuthology.orchestra.run.smithi038:> cat /proc/self/mounts | awk '{print $2}' 2024-09-21T03:33:45.276 INFO:teuthology.orchestra.run.smithi038.stdout:/proc 2024-09-21T03:33:45.277 INFO:teuthology.orchestra.run.smithi038.stdout:/sys 2024-09-21T03:33:45.277 INFO:teuthology.orchestra.run.smithi038.stdout:/dev 2024-09-21T03:33:45.277 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/kernel/security 2024-09-21T03:33:45.277 INFO:teuthology.orchestra.run.smithi038.stdout:/dev/shm 2024-09-21T03:33:45.277 INFO:teuthology.orchestra.run.smithi038.stdout:/dev/pts 2024-09-21T03:33:45.277 INFO:teuthology.orchestra.run.smithi038.stdout:/run 2024-09-21T03:33:45.277 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/cgroup 2024-09-21T03:33:45.277 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/pstore 2024-09-21T03:33:45.277 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/bpf 2024-09-21T03:33:45.277 INFO:teuthology.orchestra.run.smithi038.stdout:/ 2024-09-21T03:33:45.277 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/selinux 2024-09-21T03:33:45.277 INFO:teuthology.orchestra.run.smithi038.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T03:33:45.278 INFO:teuthology.orchestra.run.smithi038.stdout:/dev/mqueue 2024-09-21T03:33:45.278 INFO:teuthology.orchestra.run.smithi038.stdout:/dev/hugepages 2024-09-21T03:33:45.278 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/kernel/debug 2024-09-21T03:33:45.278 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/kernel/tracing 2024-09-21T03:33:45.278 INFO:teuthology.orchestra.run.smithi038.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-21T03:33:45.278 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/fuse/connections 2024-09-21T03:33:45.278 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/kernel/config 2024-09-21T03:33:45.278 INFO:teuthology.orchestra.run.smithi038.stdout:/run/credentials/systemd-sysctl.service 2024-09-21T03:33:45.278 INFO:teuthology.orchestra.run.smithi038.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-21T03:33:45.278 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/nfs/rpc_pipefs 2024-09-21T03:33:45.278 INFO:teuthology.orchestra.run.smithi038.stdout:/run/user/1000 2024-09-21T03:33:45.278 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/ceph 2024-09-21T03:33:45.278 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/ceph/osd/ceph-0 2024-09-21T03:33:45.279 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/ceph/osd/ceph-1 2024-09-21T03:33:45.279 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/ceph/osd/ceph-2 2024-09-21T03:33:45.279 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/ceph/osd/ceph-3 2024-09-21T03:33:45.279 INFO:teuthology.orchestra.run.smithi038.stdout:/run/netns 2024-09-21T03:33:45.279 INFO:teuthology.orchestra.run.smithi038.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-21T03:33:45.279 INFO:teuthology.orchestra.run.smithi038.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-21T03:33:45.279 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:33:45.279 DEBUG:teuthology.orchestra.run.smithi038:> ls /sys/fs/fuse/connections 2024-09-21T03:33:45.332 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-21T03:33:45.332 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-09-21T03:33:45.375 DEBUG:teuthology.orchestra.run.smithi038:> sudo modprobe fuse 2024-09-21T03:33:45.400 DEBUG:teuthology.orchestra.run.smithi038:> cat /proc/self/mounts | awk '{print $2}' 2024-09-21T03:33:45.458 INFO:teuthology.orchestra.run.smithi038.stdout:/proc 2024-09-21T03:33:45.458 INFO:teuthology.orchestra.run.smithi038.stdout:/sys 2024-09-21T03:33:45.459 INFO:teuthology.orchestra.run.smithi038.stdout:/dev 2024-09-21T03:33:45.459 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/kernel/security 2024-09-21T03:33:45.459 INFO:teuthology.orchestra.run.smithi038.stdout:/dev/shm 2024-09-21T03:33:45.459 INFO:teuthology.orchestra.run.smithi038.stdout:/dev/pts 2024-09-21T03:33:45.459 INFO:teuthology.orchestra.run.smithi038.stdout:/run 2024-09-21T03:33:45.459 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/cgroup 2024-09-21T03:33:45.459 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/pstore 2024-09-21T03:33:45.459 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/bpf 2024-09-21T03:33:45.459 INFO:teuthology.orchestra.run.smithi038.stdout:/ 2024-09-21T03:33:45.460 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/selinux 2024-09-21T03:33:45.460 INFO:teuthology.orchestra.run.smithi038.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T03:33:45.460 INFO:teuthology.orchestra.run.smithi038.stdout:/dev/mqueue 2024-09-21T03:33:45.460 INFO:teuthology.orchestra.run.smithi038.stdout:/dev/hugepages 2024-09-21T03:33:45.460 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/kernel/debug 2024-09-21T03:33:45.460 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/kernel/tracing 2024-09-21T03:33:45.460 INFO:teuthology.orchestra.run.smithi038.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-21T03:33:45.460 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/fuse/connections 2024-09-21T03:33:45.460 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/kernel/config 2024-09-21T03:33:45.460 INFO:teuthology.orchestra.run.smithi038.stdout:/run/credentials/systemd-sysctl.service 2024-09-21T03:33:45.461 INFO:teuthology.orchestra.run.smithi038.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-21T03:33:45.461 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/nfs/rpc_pipefs 2024-09-21T03:33:45.461 INFO:teuthology.orchestra.run.smithi038.stdout:/run/user/1000 2024-09-21T03:33:45.461 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/ceph 2024-09-21T03:33:45.461 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/ceph/osd/ceph-0 2024-09-21T03:33:45.461 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/ceph/osd/ceph-1 2024-09-21T03:33:45.461 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/ceph/osd/ceph-2 2024-09-21T03:33:45.461 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/ceph/osd/ceph-3 2024-09-21T03:33:45.461 INFO:teuthology.orchestra.run.smithi038.stdout:/run/netns 2024-09-21T03:33:45.461 INFO:teuthology.orchestra.run.smithi038.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-21T03:33:45.461 INFO:teuthology.orchestra.run.smithi038.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-21T03:33:45.461 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:33:45.462 DEBUG:teuthology.orchestra.run.smithi038:> ls /sys/fs/fuse/connections 2024-09-21T03:33:45.513 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi038.stderr:2024-09-21T03:33:45.511+0000 7f01bb506480 -1 init, newargv = 0x5563ef3554e0 newargc=15 2024-09-21T03:33:45.513 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi038.stderr:ceph-fuse[43963]: starting ceph client 2024-09-21T03:33:45.521 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi038.stderr:ceph-fuse[43963]: starting fuse 2024-09-21T03:33:46.616 DEBUG:teuthology.orchestra.run.smithi038:> sudo modprobe fuse 2024-09-21T03:33:46.650 DEBUG:teuthology.orchestra.run.smithi038:> cat /proc/self/mounts | awk '{print $2}' 2024-09-21T03:33:46.710 INFO:teuthology.orchestra.run.smithi038.stdout:/proc 2024-09-21T03:33:46.710 INFO:teuthology.orchestra.run.smithi038.stdout:/sys 2024-09-21T03:33:46.710 INFO:teuthology.orchestra.run.smithi038.stdout:/dev 2024-09-21T03:33:46.710 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/kernel/security 2024-09-21T03:33:46.711 INFO:teuthology.orchestra.run.smithi038.stdout:/dev/shm 2024-09-21T03:33:46.711 INFO:teuthology.orchestra.run.smithi038.stdout:/dev/pts 2024-09-21T03:33:46.711 INFO:teuthology.orchestra.run.smithi038.stdout:/run 2024-09-21T03:33:46.711 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/cgroup 2024-09-21T03:33:46.711 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/pstore 2024-09-21T03:33:46.711 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/bpf 2024-09-21T03:33:46.711 INFO:teuthology.orchestra.run.smithi038.stdout:/ 2024-09-21T03:33:46.711 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/selinux 2024-09-21T03:33:46.711 INFO:teuthology.orchestra.run.smithi038.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T03:33:46.711 INFO:teuthology.orchestra.run.smithi038.stdout:/dev/mqueue 2024-09-21T03:33:46.711 INFO:teuthology.orchestra.run.smithi038.stdout:/dev/hugepages 2024-09-21T03:33:46.711 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/kernel/debug 2024-09-21T03:33:46.712 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/kernel/tracing 2024-09-21T03:33:46.712 INFO:teuthology.orchestra.run.smithi038.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-21T03:33:46.712 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/fuse/connections 2024-09-21T03:33:46.712 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/kernel/config 2024-09-21T03:33:46.712 INFO:teuthology.orchestra.run.smithi038.stdout:/run/credentials/systemd-sysctl.service 2024-09-21T03:33:46.712 INFO:teuthology.orchestra.run.smithi038.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-21T03:33:46.712 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/nfs/rpc_pipefs 2024-09-21T03:33:46.712 INFO:teuthology.orchestra.run.smithi038.stdout:/run/user/1000 2024-09-21T03:33:46.712 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/ceph 2024-09-21T03:33:46.712 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/ceph/osd/ceph-0 2024-09-21T03:33:46.712 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/ceph/osd/ceph-1 2024-09-21T03:33:46.712 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/ceph/osd/ceph-2 2024-09-21T03:33:46.712 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/ceph/osd/ceph-3 2024-09-21T03:33:46.712 INFO:teuthology.orchestra.run.smithi038.stdout:/run/netns 2024-09-21T03:33:46.713 INFO:teuthology.orchestra.run.smithi038.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-21T03:33:46.713 INFO:teuthology.orchestra.run.smithi038.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-21T03:33:46.713 INFO:teuthology.orchestra.run.smithi038.stdout:/home/ubuntu/cephtest/mnt.admin 2024-09-21T03:33:46.713 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:33:46.713 DEBUG:teuthology.orchestra.run.smithi038:> ls /sys/fs/fuse/connections 2024-09-21T03:33:46.768 INFO:teuthology.orchestra.run.smithi038.stdout:41 2024-09-21T03:33:46.769 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2024-09-21T03:33:46.769 DEBUG:teuthology.orchestra.run.smithi038:> sudo stdin-killer -- python3 -c ' 2024-09-21T03:33:46.769 DEBUG:teuthology.orchestra.run.smithi038:> import glob 2024-09-21T03:33:46.769 DEBUG:teuthology.orchestra.run.smithi038:> import re 2024-09-21T03:33:46.769 DEBUG:teuthology.orchestra.run.smithi038:> import os 2024-09-21T03:33:46.769 DEBUG:teuthology.orchestra.run.smithi038:> import subprocess 2024-09-21T03:33:46.769 DEBUG:teuthology.orchestra.run.smithi038:> 2024-09-21T03:33:46.769 DEBUG:teuthology.orchestra.run.smithi038:> def _find_admin_socket(client_name): 2024-09-21T03:33:46.770 DEBUG:teuthology.orchestra.run.smithi038:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-09-21T03:33:46.770 DEBUG:teuthology.orchestra.run.smithi038:> files = glob.glob(asok_path) 2024-09-21T03:33:46.770 DEBUG:teuthology.orchestra.run.smithi038:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-09-21T03:33:46.770 DEBUG:teuthology.orchestra.run.smithi038:> 2024-09-21T03:33:46.770 DEBUG:teuthology.orchestra.run.smithi038:> # Given a non-glob path, it better be there 2024-09-21T03:33:46.770 DEBUG:teuthology.orchestra.run.smithi038:> if "*" not in asok_path: 2024-09-21T03:33:46.770 DEBUG:teuthology.orchestra.run.smithi038:> assert(len(files) == 1) 2024-09-21T03:33:46.770 DEBUG:teuthology.orchestra.run.smithi038:> return files[0] 2024-09-21T03:33:46.770 DEBUG:teuthology.orchestra.run.smithi038:> 2024-09-21T03:33:46.770 DEBUG:teuthology.orchestra.run.smithi038:> for f in files: 2024-09-21T03:33:46.770 DEBUG:teuthology.orchestra.run.smithi038:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-21T03:33:46.770 DEBUG:teuthology.orchestra.run.smithi038:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-21T03:33:46.770 DEBUG:teuthology.orchestra.run.smithi038:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-21T03:33:46.771 DEBUG:teuthology.orchestra.run.smithi038:> contents = proc_f.read() 2024-09-21T03:33:46.771 DEBUG:teuthology.orchestra.run.smithi038:> if mountpoint in contents: 2024-09-21T03:33:46.771 DEBUG:teuthology.orchestra.run.smithi038:> return f 2024-09-21T03:33:46.771 DEBUG:teuthology.orchestra.run.smithi038:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-21T03:33:46.771 DEBUG:teuthology.orchestra.run.smithi038:> 2024-09-21T03:33:46.771 DEBUG:teuthology.orchestra.run.smithi038:> print(_find_admin_socket("client.admin")) 2024-09-21T03:33:46.771 DEBUG:teuthology.orchestra.run.smithi038:> ' 2024-09-21T03:33:46.896 INFO:teuthology.orchestra.run.smithi038.stdout:/var/run/ceph/ceph-client.admin.43963.asok 2024-09-21T03:33:46.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:33:46.907 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.43963.asok 2024-09-21T03:33:46.907 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:33:46.908 DEBUG:teuthology.orchestra.run.smithi038:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.43963.asok status 2024-09-21T03:33:47.055 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-09-21T03:33:47.055 INFO:teuthology.orchestra.run.smithi038.stdout: "metadata": { 2024-09-21T03:33:47.056 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph_sha1": "1843221895adccfb3b692060251dd6a0183bc410", 2024-09-21T03:33:47.056 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph_version": "ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)", 2024-09-21T03:33:47.056 INFO:teuthology.orchestra.run.smithi038.stdout: "entity_id": "admin", 2024-09-21T03:33:47.056 INFO:teuthology.orchestra.run.smithi038.stdout: "hostname": "smithi038", 2024-09-21T03:33:47.056 INFO:teuthology.orchestra.run.smithi038.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-09-21T03:33:47.056 INFO:teuthology.orchestra.run.smithi038.stdout: "pid": "43963", 2024-09-21T03:33:47.056 INFO:teuthology.orchestra.run.smithi038.stdout: "root": "/" 2024-09-21T03:33:47.056 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-21T03:33:47.056 INFO:teuthology.orchestra.run.smithi038.stdout: "dentry_count": 0, 2024-09-21T03:33:47.056 INFO:teuthology.orchestra.run.smithi038.stdout: "dentry_pinned_count": 0, 2024-09-21T03:33:47.056 INFO:teuthology.orchestra.run.smithi038.stdout: "id": 4509, 2024-09-21T03:33:47.056 INFO:teuthology.orchestra.run.smithi038.stdout: "inst": { 2024-09-21T03:33:47.056 INFO:teuthology.orchestra.run.smithi038.stdout: "name": { 2024-09-21T03:33:47.057 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "client", 2024-09-21T03:33:47.057 INFO:teuthology.orchestra.run.smithi038.stdout: "num": 4509 2024-09-21T03:33:47.057 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-21T03:33:47.057 INFO:teuthology.orchestra.run.smithi038.stdout: "addr": { 2024-09-21T03:33:47.057 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "v1", 2024-09-21T03:33:47.057 INFO:teuthology.orchestra.run.smithi038.stdout: "addr": "192.168.0.1:0", 2024-09-21T03:33:47.057 INFO:teuthology.orchestra.run.smithi038.stdout: "nonce": 2973579098 2024-09-21T03:33:47.057 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-21T03:33:47.057 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-21T03:33:47.057 INFO:teuthology.orchestra.run.smithi038.stdout: "addr": { 2024-09-21T03:33:47.057 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "v1", 2024-09-21T03:33:47.057 INFO:teuthology.orchestra.run.smithi038.stdout: "addr": "192.168.0.1:0", 2024-09-21T03:33:47.057 INFO:teuthology.orchestra.run.smithi038.stdout: "nonce": 2973579098 2024-09-21T03:33:47.057 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-21T03:33:47.058 INFO:teuthology.orchestra.run.smithi038.stdout: "inst_str": "client.4509 192.168.0.1:0/2973579098", 2024-09-21T03:33:47.058 INFO:teuthology.orchestra.run.smithi038.stdout: "addr_str": "192.168.0.1:0/2973579098", 2024-09-21T03:33:47.058 INFO:teuthology.orchestra.run.smithi038.stdout: "inode_count": 1, 2024-09-21T03:33:47.058 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_epoch": 7, 2024-09-21T03:33:47.058 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_epoch": 26, 2024-09-21T03:33:47.058 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_epoch_barrier": 0, 2024-09-21T03:33:47.058 INFO:teuthology.orchestra.run.smithi038.stdout: "blocklisted": false, 2024-09-21T03:33:47.058 INFO:teuthology.orchestra.run.smithi038.stdout: "fs_name": "cephfs" 2024-09-21T03:33:47.058 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-09-21T03:33:47.064 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:33:47.064 DEBUG:teuthology.orchestra.run.smithi038:> stat --file-system '--printf=%T 2024-09-21T03:33:47.064 DEBUG:teuthology.orchestra.run.smithi038:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-09-21T03:33:47.080 INFO:teuthology.orchestra.run.smithi038.stdout:fuseblk 2024-09-21T03:33:47.080 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-09-21T03:33:47.080 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:33:47.080 DEBUG:teuthology.orchestra.run.smithi038:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-09-21T03:33:47.147 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-09-21T03:33:47.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:47 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T03:33:47.262 INFO:teuthology.orchestra.run.smithi038.stdout:# file: . 2024-09-21T03:33:47.263 INFO:teuthology.orchestra.run.smithi038.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-09-21T03:33:47.263 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:33:47.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:33:47.269 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-21T03:33:47.269 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:33:47.285 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T03:33:47.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:47.439+0000 7f1c970d5640 1 -- 172.21.15.38:0/2534106 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1c900cf7d0 msgr2=0x7f1c90072d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:47.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:47.439+0000 7f1c970d5640 1 --2- 172.21.15.38:0/2534106 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1c900cf7d0 0x7f1c90072d90 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f1c88009830 tx=0x7f1c8802a500 comp rx=0 tx=0).stop 2024-09-21T03:33:47.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:47.439+0000 7f1c970d5640 1 -- 172.21.15.38:0/2534106 shutdown_connections 2024-09-21T03:33:47.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:47.439+0000 7f1c970d5640 1 --2- 172.21.15.38:0/2534106 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1c900cf7d0 0x7f1c90072d90 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:47.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:47.439+0000 7f1c970d5640 1 --2- 172.21.15.38:0/2534106 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1c9006e500 0x7f1c900cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:47.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:47.439+0000 7f1c970d5640 1 -- 172.21.15.38:0/2534106 >> 172.21.15.38:0/2534106 conn(0x7f1c900b1b90 msgr2=0x7f1c900b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:47.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:47.439+0000 7f1c970d5640 1 -- 172.21.15.38:0/2534106 shutdown_connections 2024-09-21T03:33:47.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:47.439+0000 7f1c970d5640 1 -- 172.21.15.38:0/2534106 wait complete. 2024-09-21T03:33:47.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:47.441+0000 7f1c970d5640 1 Processor -- start 2024-09-21T03:33:47.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:47.441+0000 7f1c970d5640 1 -- start start 2024-09-21T03:33:47.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:47.443+0000 7f1c970d5640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1c900cf7d0 0x7f1c90158e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:47.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:47.443+0000 7f1c970d5640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1c9015fc50 0x7f1c901e7190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:47.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:47.443+0000 7f1c970d5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1c900a04a0 0x7f1c901e76d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:47.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:47.443+0000 7f1c970d5640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f1c90070d20 con 0x7f1c900a04a0 2024-09-21T03:33:47.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:47.443+0000 7f1c970d5640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f1c90070ba0 con 0x7f1c900cf7d0 2024-09-21T03:33:47.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:47.443+0000 7f1c970d5640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f1c900732d0 con 0x7f1c9015fc50 2024-09-21T03:33:47.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:47.443+0000 7f1c95672640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1c900a04a0 0x7f1c901e76d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:47.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:47.443+0000 7f1c95672640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1c900a04a0 0x7f1c901e76d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35894/0 (socket says 172.21.15.38:35894) 2024-09-21T03:33:47.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:47.443+0000 7f1c95672640 1 -- 172.21.15.38:0/2534371020 learned_addr learned my addr 172.21.15.38:0/2534371020 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:33:47.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:47.443+0000 7f1c87fff640 1 --2- 172.21.15.38:0/2534371020 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1c9015fc50 0x7f1c901e7190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:47.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:47.443+0000 7f1c94e71640 1 --2- 172.21.15.38:0/2534371020 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1c900cf7d0 0x7f1c90158e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:47.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:47.443+0000 7f1c95672640 1 -- 172.21.15.38:0/2534371020 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1c9015fc50 msgr2=0x7f1c901e7190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:47.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:47.443+0000 7f1c95672640 1 --2- 172.21.15.38:0/2534371020 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1c9015fc50 0x7f1c901e7190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:47.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:47.443+0000 7f1c95672640 1 -- 172.21.15.38:0/2534371020 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1c900cf7d0 msgr2=0x7f1c90158e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:47.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:47.443+0000 7f1c95672640 1 --2- 172.21.15.38:0/2534371020 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1c900cf7d0 0x7f1c90158e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:47.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:47.443+0000 7f1c95672640 1 -- 172.21.15.38:0/2534371020 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c900a0c80 con 0x7f1c900a04a0 2024-09-21T03:33:47.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:47.443+0000 7f1c95672640 1 --2- 172.21.15.38:0/2534371020 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1c900a04a0 0x7f1c901e76d0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f1c8802aaf0 tx=0x7f1c88033c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:47.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:47.444+0000 7f1c857fa640 1 -- 172.21.15.38:0/2534371020 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c88031b60 con 0x7f1c900a04a0 2024-09-21T03:33:47.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:47.444+0000 7f1c970d5640 1 -- 172.21.15.38:0/2534371020 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c900a0f60 con 0x7f1c900a04a0 2024-09-21T03:33:47.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:47.444+0000 7f1c857fa640 1 -- 172.21.15.38:0/2534371020 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1c88031df0 con 0x7f1c900a04a0 2024-09-21T03:33:47.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:47.444+0000 7f1c857fa640 1 -- 172.21.15.38:0/2534371020 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c880332d0 con 0x7f1c900a04a0 2024-09-21T03:33:47.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:47.444+0000 7f1c970d5640 1 -- 172.21.15.38:0/2534371020 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c900c8410 con 0x7f1c900a04a0 2024-09-21T03:33:47.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:47.445+0000 7f1c970d5640 1 -- 172.21.15.38:0/2534371020 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c90073760 con 0x7f1c900a04a0 2024-09-21T03:33:47.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:47.445+0000 7f1c857fa640 1 -- 172.21.15.38:0/2534371020 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f1c88033470 con 0x7f1c900a04a0 2024-09-21T03:33:47.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:47.446+0000 7f1c857fa640 1 --2- 172.21.15.38:0/2534371020 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1c7c078bc0 0x7f1c7c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:47.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:47.446+0000 7f1c857fa640 1 -- 172.21.15.38:0/2534371020 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f1c880bc310 con 0x7f1c900a04a0 2024-09-21T03:33:47.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:47.446+0000 7f1c94e71640 1 --2- 172.21.15.38:0/2534371020 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1c7c078bc0 0x7f1c7c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:47.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:47.447+0000 7f1c94e71640 1 --2- 172.21.15.38:0/2534371020 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1c7c078bc0 0x7f1c7c07b0d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1c8c002200 tx=0x7f1c8c0019f0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:47.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:47.450+0000 7f1c857fa640 1 -- 172.21.15.38:0/2534371020 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f1c8804a020 con 0x7f1c900a04a0 2024-09-21T03:33:47.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:47.652+0000 7f1c970d5640 1 -- 172.21.15.38:0/2534371020 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f1c900b1500 con 0x7f1c900a04a0 2024-09-21T03:33:47.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:47.652+0000 7f1c857fa640 1 -- 172.21.15.38:0/2534371020 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) ==== 80+0+0 (secure 0 0 0) 0x7f1c8803f030 con 0x7f1c900a04a0 2024-09-21T03:33:47.654 INFO:teuthology.orchestra.run.smithi038.stderr:listed 0 entries 2024-09-21T03:33:47.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:47.654+0000 7f1c970d5640 1 -- 172.21.15.38:0/2534371020 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1c7c078bc0 msgr2=0x7f1c7c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:47.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:47.654+0000 7f1c970d5640 1 --2- 172.21.15.38:0/2534371020 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1c7c078bc0 0x7f1c7c07b0d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1c8c002200 tx=0x7f1c8c0019f0 comp rx=0 tx=0).stop 2024-09-21T03:33:47.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:47.654+0000 7f1c970d5640 1 -- 172.21.15.38:0/2534371020 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1c900a04a0 msgr2=0x7f1c901e76d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:47.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:47.654+0000 7f1c970d5640 1 --2- 172.21.15.38:0/2534371020 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1c900a04a0 0x7f1c901e76d0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f1c8802aaf0 tx=0x7f1c88033c10 comp rx=0 tx=0).stop 2024-09-21T03:33:47.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:47.654+0000 7f1c970d5640 1 -- 172.21.15.38:0/2534371020 shutdown_connections 2024-09-21T03:33:47.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:47.654+0000 7f1c970d5640 1 --2- 172.21.15.38:0/2534371020 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1c900a04a0 0x7f1c901e76d0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:47.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:47.654+0000 7f1c970d5640 1 --2- 172.21.15.38:0/2534371020 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1c7c078bc0 0x7f1c7c07b0d0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:47.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:47.654+0000 7f1c970d5640 1 --2- 172.21.15.38:0/2534371020 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1c9015fc50 0x7f1c901e7190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:47.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:47.654+0000 7f1c970d5640 1 --2- 172.21.15.38:0/2534371020 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1c900cf7d0 0x7f1c90158e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:47.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:47.654+0000 7f1c970d5640 1 -- 172.21.15.38:0/2534371020 >> 172.21.15.38:0/2534371020 conn(0x7f1c900b1b90 msgr2=0x7f1c90159700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:47.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:47.655+0000 7f1c970d5640 1 -- 172.21.15.38:0/2534371020 shutdown_connections 2024-09-21T03:33:47.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:47.655+0000 7f1c970d5640 1 -- 172.21.15.38:0/2534371020 wait complete. 2024-09-21T03:33:47.668 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-21T03:33:47.668 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:33:47.685 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T03:33:47.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:47.840+0000 7f3b355ad640 1 -- 172.21.15.38:0/1070938201 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3b300b9ad0 msgr2=0x7f3b300d0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:47.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:47.840+0000 7f3b355ad640 1 --2- 172.21.15.38:0/1070938201 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3b300b9ad0 0x7f3b300d0be0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f3b300cfba0 tx=0x7f3b2402a4b0 comp rx=0 tx=0).stop 2024-09-21T03:33:47.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:47.840+0000 7f3b355ad640 1 -- 172.21.15.38:0/1070938201 shutdown_connections 2024-09-21T03:33:47.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:47.840+0000 7f3b355ad640 1 --2- 172.21.15.38:0/1070938201 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3b300d1120 0x7f3b300ceb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:47.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:47.840+0000 7f3b355ad640 1 --2- 172.21.15.38:0/1070938201 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3b300b9ad0 0x7f3b300d0be0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:47.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:47.840+0000 7f3b355ad640 1 --2- 172.21.15.38:0/1070938201 >> v2:172.21.15.195:3300/0 conn(0x7f3b3015ee80 0x7f3b300d0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:47.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:47.840+0000 7f3b355ad640 1 -- 172.21.15.38:0/1070938201 >> 172.21.15.38:0/1070938201 conn(0x7f3b3009d830 msgr2=0x7f3b300b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:47.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:47.840+0000 7f3b355ad640 1 -- 172.21.15.38:0/1070938201 shutdown_connections 2024-09-21T03:33:47.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:47.840+0000 7f3b355ad640 1 -- 172.21.15.38:0/1070938201 wait complete. 2024-09-21T03:33:47.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:47.842+0000 7f3b355ad640 1 Processor -- start 2024-09-21T03:33:47.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:47.842+0000 7f3b355ad640 1 -- start start 2024-09-21T03:33:47.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:47.843+0000 7f3b355ad640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3b300b9ad0 0x7f3b3014fa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:47.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:47.843+0000 7f3b355ad640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3b300d1120 0x7f3b300d3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:47.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:47.843+0000 7f3b355ad640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3b3015ee80 0x7f3b300d4130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:47.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:47.843+0000 7f3b355ad640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f3b300cf520 con 0x7f3b300d1120 2024-09-21T03:33:47.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:47.843+0000 7f3b355ad640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f3b300cf3a0 con 0x7f3b300b9ad0 2024-09-21T03:33:47.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:47.843+0000 7f3b355ad640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f3b300b6580 con 0x7f3b3015ee80 2024-09-21T03:33:47.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:47.843+0000 7f3b2e7fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3b300d1120 0x7f3b300d3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:47.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:47.843+0000 7f3b2e7fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3b300d1120 0x7f3b300d3bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35916/0 (socket says 172.21.15.38:35916) 2024-09-21T03:33:47.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:47.843+0000 7f3b2e7fc640 1 -- 172.21.15.38:0/1421567461 learned_addr learned my addr 172.21.15.38:0/1421567461 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:33:47.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:47.844+0000 7f3b2effd640 1 --2- 172.21.15.38:0/1421567461 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3b300b9ad0 0x7f3b3014fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:47.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:47.844+0000 7f3b2f7fe640 1 --2- 172.21.15.38:0/1421567461 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3b3015ee80 0x7f3b300d4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:47.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:47.844+0000 7f3b2e7fc640 1 -- 172.21.15.38:0/1421567461 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3b3015ee80 msgr2=0x7f3b300d4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:47.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:47.844+0000 7f3b2e7fc640 1 --2- 172.21.15.38:0/1421567461 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3b3015ee80 0x7f3b300d4130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:47.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:47.844+0000 7f3b2e7fc640 1 -- 172.21.15.38:0/1421567461 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3b300b9ad0 msgr2=0x7f3b3014fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:47.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:47.844+0000 7f3b2e7fc640 1 --2- 172.21.15.38:0/1421567461 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3b300b9ad0 0x7f3b3014fa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:47.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:47.844+0000 7f3b2e7fc640 1 -- 172.21.15.38:0/1421567461 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b3006e750 con 0x7f3b300d1120 2024-09-21T03:33:47.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:47.844+0000 7f3b2e7fc640 1 --2- 172.21.15.38:0/1421567461 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3b300d1120 0x7f3b300d3bf0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f3b2402aaa0 tx=0x7f3b240049d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:47.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:47.844+0000 7f3b177fe640 1 -- 172.21.15.38:0/1421567461 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b24031950 con 0x7f3b300d1120 2024-09-21T03:33:47.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:47.844+0000 7f3b177fe640 1 -- 172.21.15.38:0/1421567461 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3b24031be0 con 0x7f3b300d1120 2024-09-21T03:33:47.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:47.844+0000 7f3b355ad640 1 -- 172.21.15.38:0/1421567461 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b3006ea30 con 0x7f3b300d1120 2024-09-21T03:33:47.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:47.844+0000 7f3b177fe640 1 -- 172.21.15.38:0/1421567461 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b240330d0 con 0x7f3b300d1120 2024-09-21T03:33:47.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:47.844+0000 7f3b355ad640 1 -- 172.21.15.38:0/1421567461 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b3006eff0 con 0x7f3b300d1120 2024-09-21T03:33:47.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:47.845+0000 7f3b355ad640 1 -- 172.21.15.38:0/1421567461 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b3006e4c0 con 0x7f3b300d1120 2024-09-21T03:33:47.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:47.846+0000 7f3b177fe640 1 -- 172.21.15.38:0/1421567461 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f3b24033290 con 0x7f3b300d1120 2024-09-21T03:33:47.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:47.846+0000 7f3b177fe640 1 --2- 172.21.15.38:0/1421567461 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3b1c078bc0 0x7f3b1c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:47.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:47.847+0000 7f3b177fe640 1 -- 172.21.15.38:0/1421567461 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f3b240bc230 con 0x7f3b300d1120 2024-09-21T03:33:47.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:47.847+0000 7f3b2effd640 1 --2- 172.21.15.38:0/1421567461 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3b1c078bc0 0x7f3b1c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:47.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:47.847+0000 7f3b2effd640 1 --2- 172.21.15.38:0/1421567461 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3b1c078bc0 0x7f3b1c07b0d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f3b20005fd0 tx=0x7f3b20009660 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:47.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:47.851+0000 7f3b177fe640 1 -- 172.21.15.38:0/1421567461 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f3b2403f030 con 0x7f3b300d1120 2024-09-21T03:33:48.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:48.050+0000 7f3b355ad640 1 -- 172.21.15.38:0/1421567461 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f3b300b5bd0 con 0x7f3b300d1120 2024-09-21T03:33:48.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:48.051+0000 7f3b177fe640 1 -- 172.21.15.38:0/1421567461 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) ==== 80+0+0 (secure 0 0 0) 0x7f3b300b5bd0 con 0x7f3b300d1120 2024-09-21T03:33:48.053 INFO:teuthology.orchestra.run.smithi038.stderr:listed 0 entries 2024-09-21T03:33:48.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:48.053+0000 7f3b355ad640 1 -- 172.21.15.38:0/1421567461 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3b1c078bc0 msgr2=0x7f3b1c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:48.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:48.053+0000 7f3b355ad640 1 --2- 172.21.15.38:0/1421567461 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3b1c078bc0 0x7f3b1c07b0d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f3b20005fd0 tx=0x7f3b20009660 comp rx=0 tx=0).stop 2024-09-21T03:33:48.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:48.053+0000 7f3b355ad640 1 -- 172.21.15.38:0/1421567461 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3b300d1120 msgr2=0x7f3b300d3bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:48.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:48.053+0000 7f3b355ad640 1 --2- 172.21.15.38:0/1421567461 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3b300d1120 0x7f3b300d3bf0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f3b2402aaa0 tx=0x7f3b240049d0 comp rx=0 tx=0).stop 2024-09-21T03:33:48.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:48.053+0000 7f3b355ad640 1 -- 172.21.15.38:0/1421567461 shutdown_connections 2024-09-21T03:33:48.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:48.053+0000 7f3b355ad640 1 --2- 172.21.15.38:0/1421567461 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3b1c078bc0 0x7f3b1c07b0d0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:48.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:48.053+0000 7f3b355ad640 1 --2- 172.21.15.38:0/1421567461 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3b3015ee80 0x7f3b300d4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:48.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:48.053+0000 7f3b355ad640 1 --2- 172.21.15.38:0/1421567461 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3b300d1120 0x7f3b300d3bf0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:48.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:48.053+0000 7f3b355ad640 1 --2- 172.21.15.38:0/1421567461 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3b300b9ad0 0x7f3b3014fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:48.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:48.053+0000 7f3b355ad640 1 -- 172.21.15.38:0/1421567461 >> 172.21.15.38:0/1421567461 conn(0x7f3b3009d830 msgr2=0x7f3b30161b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:48.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:48.053+0000 7f3b355ad640 1 -- 172.21.15.38:0/1421567461 shutdown_connections 2024-09-21T03:33:48.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:48.053+0000 7f3b355ad640 1 -- 172.21.15.38:0/1421567461 wait complete. 2024-09-21T03:33:48.066 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi038.front.sepia.ceph.com... 2024-09-21T03:33:48.066 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:33:48.066 DEBUG:teuthology.orchestra.run.smithi038:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-09-21T03:33:48.090 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi038.stderr:ceph-fuse[43963]: fuse finished with error 0 and tester_r 0 2024-09-21T03:33:48.092 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T03:33:54.193 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi038.front.sepia.ceph.com 2024-09-21T03:33:54.193 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:33:54.194 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-09-21T03:33:54.222 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T03:33:54.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:54.392+0000 7f563ba5c640 1 -- 172.21.15.38:0/2243945819 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f563415b9f0 msgr2=0x7f56340cf930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:54.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:54.392+0000 7f563ba5c640 1 --2- 172.21.15.38:0/2243945819 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f563415b9f0 0x7f56340cf930 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f5630009830 tx=0x7f563002a510 comp rx=0 tx=0).stop 2024-09-21T03:33:54.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:54.392+0000 7f563ba5c640 1 -- 172.21.15.38:0/2243945819 shutdown_connections 2024-09-21T03:33:54.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:54.392+0000 7f563ba5c640 1 --2- 172.21.15.38:0/2243945819 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f56340cefa0 0x7f56340ce320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:54.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:54.392+0000 7f563ba5c640 1 --2- 172.21.15.38:0/2243945819 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f563415b9f0 0x7f56340cf930 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:54.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:54.392+0000 7f563ba5c640 1 -- 172.21.15.38:0/2243945819 >> 172.21.15.38:0/2243945819 conn(0x7f56340cca40 msgr2=0x7f56340cce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:54.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:54.392+0000 7f563ba5c640 1 -- 172.21.15.38:0/2243945819 shutdown_connections 2024-09-21T03:33:54.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:54.392+0000 7f563ba5c640 1 -- 172.21.15.38:0/2243945819 wait complete. 2024-09-21T03:33:54.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:54.394+0000 7f563ba5c640 1 Processor -- start 2024-09-21T03:33:54.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:54.394+0000 7f563ba5c640 1 -- start start 2024-09-21T03:33:54.399 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:54.395+0000 7f563ba5c640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f56340ce860 0x7f56341f6480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:54.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:54.396+0000 7f563ba5c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f56340cefa0 0x7f56341f2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:54.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:54.396+0000 7f563ba5c640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f563415b9f0 0x7f563409b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:54.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:54.396+0000 7f563ba5c640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f563406df70 con 0x7f56340cefa0 2024-09-21T03:33:54.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:54.396+0000 7f563ba5c640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f563406ddf0 con 0x7f563415b9f0 2024-09-21T03:33:54.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:54.396+0000 7f563ba5c640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f563406e0f0 con 0x7f56340ce860 2024-09-21T03:33:54.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:54.396+0000 7f5638ff7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f56340cefa0 0x7f56341f2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:54.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:54.396+0000 7f5638ff7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f56340cefa0 0x7f56341f2ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41842/0 (socket says 172.21.15.38:41842) 2024-09-21T03:33:54.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:54.396+0000 7f5638ff7640 1 -- 172.21.15.38:0/2667248074 learned_addr learned my addr 172.21.15.38:0/2667248074 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:33:54.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:54.396+0000 7f56397f8640 1 --2- 172.21.15.38:0/2667248074 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f56340ce860 0x7f56341f6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:54.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:54.396+0000 7f5639ff9640 1 --2- 172.21.15.38:0/2667248074 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f563415b9f0 0x7f563409b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:54.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:54.396+0000 7f5638ff7640 1 -- 172.21.15.38:0/2667248074 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f56340ce860 msgr2=0x7f56341f6480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:54.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:54.396+0000 7f5638ff7640 1 --2- 172.21.15.38:0/2667248074 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f56340ce860 0x7f56341f6480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:54.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:54.396+0000 7f5638ff7640 1 -- 172.21.15.38:0/2667248074 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f563415b9f0 msgr2=0x7f563409b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:54.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:54.396+0000 7f5638ff7640 1 --2- 172.21.15.38:0/2667248074 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f563415b9f0 0x7f563409b5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:54.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:54.396+0000 7f5638ff7640 1 -- 172.21.15.38:0/2667248074 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56341f69c0 con 0x7f56340cefa0 2024-09-21T03:33:54.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:54.396+0000 7f5639ff9640 1 --2- 172.21.15.38:0/2667248074 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f563415b9f0 0x7f563409b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:33:54.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:54.396+0000 7f5638ff7640 1 --2- 172.21.15.38:0/2667248074 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f56340cefa0 0x7f56341f2ed0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f562400d770 tx=0x7f562400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:54.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:54.396+0000 7f5629ffb640 1 -- 172.21.15.38:0/2667248074 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5624017070 con 0x7f56340cefa0 2024-09-21T03:33:54.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:54.396+0000 7f5629ffb640 1 -- 172.21.15.38:0/2667248074 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5624009d20 con 0x7f56340cefa0 2024-09-21T03:33:54.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:54.396+0000 7f563ba5c640 1 -- 172.21.15.38:0/2667248074 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f563409c460 con 0x7f56340cefa0 2024-09-21T03:33:54.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:54.396+0000 7f5629ffb640 1 -- 172.21.15.38:0/2667248074 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5624004280 con 0x7f56340cefa0 2024-09-21T03:33:54.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:54.397+0000 7f563ba5c640 1 -- 172.21.15.38:0/2667248074 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f563409adf0 con 0x7f56340cefa0 2024-09-21T03:33:54.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:54.398+0000 7f563ba5c640 1 -- 172.21.15.38:0/2667248074 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f563409c1d0 con 0x7f56340cefa0 2024-09-21T03:33:54.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:54.399+0000 7f5629ffb640 1 -- 172.21.15.38:0/2667248074 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f56240047a0 con 0x7f56340cefa0 2024-09-21T03:33:54.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:54.399+0000 7f5629ffb640 1 --2- 172.21.15.38:0/2667248074 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5618078bc0 0x7f561807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:54.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:54.399+0000 7f5629ffb640 1 -- 172.21.15.38:0/2667248074 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f562409b3b0 con 0x7f56340cefa0 2024-09-21T03:33:54.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:54.400+0000 7f56397f8640 1 --2- 172.21.15.38:0/2667248074 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5618078bc0 0x7f561807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:54.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:54.400+0000 7f56397f8640 1 --2- 172.21.15.38:0/2667248074 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5618078bc0 0x7f561807b0d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5630034000 tx=0x7f5630004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:54.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:54.404+0000 7f5629ffb640 1 -- 172.21.15.38:0/2667248074 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f5624020020 con 0x7f56340cefa0 2024-09-21T03:33:54.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:54.593+0000 7f563ba5c640 1 -- 172.21.15.38:0/2667248074 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f563406e5c0 con 0x7f56340cefa0 2024-09-21T03:33:54.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:54.594+0000 7f5629ffb640 1 -- 172.21.15.38:0/2667248074 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18626 (secure 0 0 0) 0x7f5624063540 con 0x7f56340cefa0 2024-09-21T03:33:54.597 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:33:54.597 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":26,"fsid":"9d50d66f-656b-4fe2-8e3b-fe53b79c4d43","created":"2024-09-21T03:33:10.677865+0000","modified":"2024-09-21T03:33:40.665991+0000","last_up_change":"2024-09-21T03:33:24.023421+0000","last_in_change":"2024-09-21T03:33:14.296193+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T03:33:22.992573+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T03:33:33.530201+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T03:33:34.585875+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T03:33:38.268346+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"dbab183f-7e63-48b8-960a-ae55fd5bf565","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6801","nonce":2381639433}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6803","nonce":2381639433}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6807","nonce":2381639433}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6805","nonce":2381639433}]},"public_addr":"172.21.15.38:6801/2381639433","cluster_addr":"172.21.15.38:6803/2381639433","heartbeat_back_addr":"172.21.15.38:6807/2381639433","heartbeat_front_addr":"172.21.15.38:6805/2381639433","state":["exists","up"]},{"osd":1,"uuid":"fa1c0e8a-ebc1-4359-8146-bd5342abbc16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6817","nonce":1491818732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6819","nonce":1491818732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6823","nonce":1491818732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6821","nonce":1491818732}]},"public_addr":"172.21.15.38:6817/1491818732","cluster_addr":"172.21.15.38:6819/1491818732","heartbeat_back_addr":"172.21.15.38:6823/1491818732","heartbeat_front_addr":"172.21.15.38:6821/1491818732","state":["exists","up"]},{"osd":2,"uuid":"2ed3cd2d-385d-4fc6-9869-214552a84803","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6825","nonce":2577858902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6827","nonce":2577858902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6831","nonce":2577858902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6829","nonce":2577858902}]},"public_addr":"172.21.15.38:6825/2577858902","cluster_addr":"172.21.15.38:6827/2577858902","heartbeat_back_addr":"172.21.15.38:6831/2577858902","heartbeat_front_addr":"172.21.15.38:6829/2577858902","state":["exists","up"]},{"osd":3,"uuid":"62d1d95e-919d-4250-be71-1d34cfa70dbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6809","nonce":1286810294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6811","nonce":1286810294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6815","nonce":1286810294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6813","nonce":1286810294}]},"public_addr":"172.21.15.38:6809/1286810294","cluster_addr":"172.21.15.38:6811/1286810294","heartbeat_back_addr":"172.21.15.38:6815/1286810294","heartbeat_front_addr":"172.21.15.38:6813/1286810294","state":["exists","up"]},{"osd":4,"uuid":"7c9f9314-99d7-4d73-8392-37809a9f54b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6803","nonce":3401837571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6805","nonce":3401837571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6809","nonce":3401837571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6807","nonce":3401837571}]},"public_addr":"172.21.15.195:6803/3401837571","cluster_addr":"172.21.15.195:6805/3401837571","heartbeat_back_addr":"172.21.15.195:6809/3401837571","heartbeat_front_addr":"172.21.15.195:6807/3401837571","state":["exists","up"]},{"osd":5,"uuid":"bde4fe9f-8414-4bba-bf4a-43973d1976bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6811","nonce":2080960180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6813","nonce":2080960180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6817","nonce":2080960180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6815","nonce":2080960180}]},"public_addr":"172.21.15.195:6811/2080960180","cluster_addr":"172.21.15.195:6813/2080960180","heartbeat_back_addr":"172.21.15.195:6817/2080960180","heartbeat_front_addr":"172.21.15.195:6815/2080960180","state":["exists","up"]},{"osd":6,"uuid":"6ee6d7f4-1a5f-4a4c-ac16-48fe05f6a002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6827","nonce":4055825675}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6829","nonce":4055825675}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6833","nonce":4055825675}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6831","nonce":4055825675}]},"public_addr":"172.21.15.195:6827/4055825675","cluster_addr":"172.21.15.195:6829/4055825675","heartbeat_back_addr":"172.21.15.195:6833/4055825675","heartbeat_front_addr":"172.21.15.195:6831/4055825675","state":["exists","up"]},{"osd":7,"uuid":"efeb5f51-329a-4484-8290-92e7d3503d05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6819","nonce":1606106342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6821","nonce":1606106342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6825","nonce":1606106342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6823","nonce":1606106342}]},"public_addr":"172.21.15.195:6819/1606106342","cluster_addr":"172.21.15.195:6821/1606106342","heartbeat_back_addr":"172.21.15.195:6825/1606106342","heartbeat_front_addr":"172.21.15.195:6823/1606106342","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.901343+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:21.115003+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.901623+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.835238+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.132199+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:19.817560+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.829690+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:21.087300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T03:33:54.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:54.596+0000 7f563ba5c640 1 -- 172.21.15.38:0/2667248074 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5618078bc0 msgr2=0x7f561807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:54.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:54.596+0000 7f563ba5c640 1 --2- 172.21.15.38:0/2667248074 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5618078bc0 0x7f561807b0d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5630034000 tx=0x7f5630004730 comp rx=0 tx=0).stop 2024-09-21T03:33:54.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:54.596+0000 7f563ba5c640 1 -- 172.21.15.38:0/2667248074 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f56340cefa0 msgr2=0x7f56341f2ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:54.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:54.596+0000 7f563ba5c640 1 --2- 172.21.15.38:0/2667248074 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f56340cefa0 0x7f56341f2ed0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f562400d770 tx=0x7f562400dc30 comp rx=0 tx=0).stop 2024-09-21T03:33:54.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:54.596+0000 7f563ba5c640 1 -- 172.21.15.38:0/2667248074 shutdown_connections 2024-09-21T03:33:54.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:54.596+0000 7f563ba5c640 1 --2- 172.21.15.38:0/2667248074 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f563415b9f0 0x7f563409b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:54.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:54.596+0000 7f563ba5c640 1 --2- 172.21.15.38:0/2667248074 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f56340cefa0 0x7f56341f2ed0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:54.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:54.596+0000 7f563ba5c640 1 --2- 172.21.15.38:0/2667248074 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5618078bc0 0x7f561807b0d0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:54.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:54.596+0000 7f563ba5c640 1 --2- 172.21.15.38:0/2667248074 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f56340ce860 0x7f56341f6480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:54.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:54.596+0000 7f563ba5c640 1 -- 172.21.15.38:0/2667248074 >> 172.21.15.38:0/2667248074 conn(0x7f56340cca40 msgr2=0x7f563415aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:54.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:54.596+0000 7f563ba5c640 1 -- 172.21.15.38:0/2667248074 shutdown_connections 2024-09-21T03:33:54.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:54.596+0000 7f563ba5c640 1 -- 172.21.15.38:0/2667248074 wait complete. 2024-09-21T03:33:54.610 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T03:33:54.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:54.724+0000 7fed381c6640 1 -- 172.21.15.38:0/3186685134 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed18003a00 msgr2=0x7fed18003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:54.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:54.725+0000 7fed381c6640 1 --2- 172.21.15.38:0/3186685134 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed18003a00 0x7fed18003e70 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fed20009ed0 tx=0x7fed18005830 comp rx=0 tx=0).stop 2024-09-21T03:33:54.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:54.725+0000 7fed36763640 1 -- 172.21.15.38:0/3186685134 reap_dead start 2024-09-21T03:33:54.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:54.725+0000 7fed381c6640 1 -- 172.21.15.38:0/3186685134 shutdown_connections 2024-09-21T03:33:54.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:54.725+0000 7fed381c6640 1 -- 172.21.15.38:0/3186685134 >> 172.21.15.38:0/3186685134 conn(0x7fed300d4440 msgr2=0x7fed300d4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:54.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:54.725+0000 7fed381c6640 1 -- 172.21.15.38:0/3186685134 shutdown_connections 2024-09-21T03:33:54.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:54.725+0000 7fed381c6640 1 -- 172.21.15.38:0/3186685134 wait complete. 2024-09-21T03:33:54.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:54.726+0000 7fed381c6640 1 Processor -- start 2024-09-21T03:33:54.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:54.727+0000 7fed381c6640 1 -- start start 2024-09-21T03:33:54.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:54.728+0000 7fed381c6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed18003a00 0x7fed3014c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:54.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:54.728+0000 7fed381c6640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fed3014d360 0x7fed3014d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:54.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:54.728+0000 7fed381c6640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fed3014dcb0 0x7fed3014e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:54.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:54.728+0000 7fed381c6640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fed300b3980 con 0x7fed18003a00 2024-09-21T03:33:54.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:54.728+0000 7fed381c6640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fed300cf5a0 con 0x7fed3014d360 2024-09-21T03:33:54.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:54.728+0000 7fed381c6640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fed300b3b00 con 0x7fed3014dcb0 2024-09-21T03:33:54.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:54.728+0000 7fed35f62640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed18003a00 0x7fed3014c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:54.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:54.728+0000 7fed35f62640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed18003a00 0x7fed3014c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41864/0 (socket says 172.21.15.38:41864) 2024-09-21T03:33:54.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:54.728+0000 7fed35f62640 1 -- 172.21.15.38:0/2310934649 learned_addr learned my addr 172.21.15.38:0/2310934649 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:33:54.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:54.728+0000 7fed35761640 1 --2- 172.21.15.38:0/2310934649 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fed3014d360 0x7fed3014d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:54.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:54.728+0000 7fed36763640 1 --2- 172.21.15.38:0/2310934649 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fed3014dcb0 0x7fed3014e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:54.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:54.728+0000 7fed35f62640 1 -- 172.21.15.38:0/2310934649 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fed3014dcb0 msgr2=0x7fed3014e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:54.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:54.728+0000 7fed35f62640 1 --2- 172.21.15.38:0/2310934649 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fed3014dcb0 0x7fed3014e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:54.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:54.728+0000 7fed35f62640 1 -- 172.21.15.38:0/2310934649 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fed3014d360 msgr2=0x7fed3014d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:54.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:54.728+0000 7fed35f62640 1 --2- 172.21.15.38:0/2310934649 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fed3014d360 0x7fed3014d770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:54.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:54.728+0000 7fed35f62640 1 -- 172.21.15.38:0/2310934649 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed300c9640 con 0x7fed18003a00 2024-09-21T03:33:54.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:54.728+0000 7fed35761640 1 --2- 172.21.15.38:0/2310934649 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fed3014d360 0x7fed3014d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:33:54.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:54.728+0000 7fed35f62640 1 --2- 172.21.15.38:0/2310934649 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed18003a00 0x7fed3014c150 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fed20036040 tx=0x7fed20033d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:54.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:54.729+0000 7fed2e7fc640 1 -- 172.21.15.38:0/2310934649 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed20038050 con 0x7fed18003a00 2024-09-21T03:33:54.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:54.729+0000 7fed2e7fc640 1 -- 172.21.15.38:0/2310934649 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fed20039040 con 0x7fed18003a00 2024-09-21T03:33:54.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:54.729+0000 7fed381c6640 1 -- 172.21.15.38:0/2310934649 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed300c9920 con 0x7fed18003a00 2024-09-21T03:33:54.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:54.729+0000 7fed2e7fc640 1 -- 172.21.15.38:0/2310934649 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed20009200 con 0x7fed18003a00 2024-09-21T03:33:54.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:54.729+0000 7fed381c6640 1 -- 172.21.15.38:0/2310934649 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed3014e3e0 con 0x7fed18003a00 2024-09-21T03:33:54.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:54.730+0000 7fed381c6640 1 -- 172.21.15.38:0/2310934649 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed30162bb0 con 0x7fed18003a00 2024-09-21T03:33:54.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:54.731+0000 7fed2e7fc640 1 -- 172.21.15.38:0/2310934649 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fed20004030 con 0x7fed18003a00 2024-09-21T03:33:54.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:54.731+0000 7fed2e7fc640 1 --2- 172.21.15.38:0/2310934649 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fed1c078b90 0x7fed1c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:54.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:54.732+0000 7fed2e7fc640 1 -- 172.21.15.38:0/2310934649 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fed200bc450 con 0x7fed18003a00 2024-09-21T03:33:54.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:54.732+0000 7fed35761640 1 --2- 172.21.15.38:0/2310934649 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fed1c078b90 0x7fed1c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:54.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:54.732+0000 7fed35761640 1 --2- 172.21.15.38:0/2310934649 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fed1c078b90 0x7fed1c07b0a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fed3015e500 tx=0x7fed24005f90 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:54.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:54.736+0000 7fed2e7fc640 1 -- 172.21.15.38:0/2310934649 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fed200845e0 con 0x7fed18003a00 2024-09-21T03:33:54.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:54.923+0000 7fed381c6640 1 -- 172.21.15.38:0/2310934649 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fed3006da60 con 0x7fed18003a00 2024-09-21T03:33:54.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:54.924+0000 7fed2e7fc640 1 -- 172.21.15.38:0/2310934649 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18626 (secure 0 0 0) 0x7fed20083ee0 con 0x7fed18003a00 2024-09-21T03:33:54.928 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:33:54.928 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":26,"fsid":"9d50d66f-656b-4fe2-8e3b-fe53b79c4d43","created":"2024-09-21T03:33:10.677865+0000","modified":"2024-09-21T03:33:40.665991+0000","last_up_change":"2024-09-21T03:33:24.023421+0000","last_in_change":"2024-09-21T03:33:14.296193+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T03:33:22.992573+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T03:33:33.530201+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T03:33:34.585875+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T03:33:38.268346+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"dbab183f-7e63-48b8-960a-ae55fd5bf565","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6801","nonce":2381639433}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6803","nonce":2381639433}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6807","nonce":2381639433}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6805","nonce":2381639433}]},"public_addr":"172.21.15.38:6801/2381639433","cluster_addr":"172.21.15.38:6803/2381639433","heartbeat_back_addr":"172.21.15.38:6807/2381639433","heartbeat_front_addr":"172.21.15.38:6805/2381639433","state":["exists","up"]},{"osd":1,"uuid":"fa1c0e8a-ebc1-4359-8146-bd5342abbc16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6817","nonce":1491818732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6819","nonce":1491818732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6823","nonce":1491818732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6821","nonce":1491818732}]},"public_addr":"172.21.15.38:6817/1491818732","cluster_addr":"172.21.15.38:6819/1491818732","heartbeat_back_addr":"172.21.15.38:6823/1491818732","heartbeat_front_addr":"172.21.15.38:6821/1491818732","state":["exists","up"]},{"osd":2,"uuid":"2ed3cd2d-385d-4fc6-9869-214552a84803","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6825","nonce":2577858902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6827","nonce":2577858902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6831","nonce":2577858902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6829","nonce":2577858902}]},"public_addr":"172.21.15.38:6825/2577858902","cluster_addr":"172.21.15.38:6827/2577858902","heartbeat_back_addr":"172.21.15.38:6831/2577858902","heartbeat_front_addr":"172.21.15.38:6829/2577858902","state":["exists","up"]},{"osd":3,"uuid":"62d1d95e-919d-4250-be71-1d34cfa70dbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6809","nonce":1286810294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6811","nonce":1286810294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6815","nonce":1286810294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6813","nonce":1286810294}]},"public_addr":"172.21.15.38:6809/1286810294","cluster_addr":"172.21.15.38:6811/1286810294","heartbeat_back_addr":"172.21.15.38:6815/1286810294","heartbeat_front_addr":"172.21.15.38:6813/1286810294","state":["exists","up"]},{"osd":4,"uuid":"7c9f9314-99d7-4d73-8392-37809a9f54b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6803","nonce":3401837571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6805","nonce":3401837571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6809","nonce":3401837571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6807","nonce":3401837571}]},"public_addr":"172.21.15.195:6803/3401837571","cluster_addr":"172.21.15.195:6805/3401837571","heartbeat_back_addr":"172.21.15.195:6809/3401837571","heartbeat_front_addr":"172.21.15.195:6807/3401837571","state":["exists","up"]},{"osd":5,"uuid":"bde4fe9f-8414-4bba-bf4a-43973d1976bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6811","nonce":2080960180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6813","nonce":2080960180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6817","nonce":2080960180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6815","nonce":2080960180}]},"public_addr":"172.21.15.195:6811/2080960180","cluster_addr":"172.21.15.195:6813/2080960180","heartbeat_back_addr":"172.21.15.195:6817/2080960180","heartbeat_front_addr":"172.21.15.195:6815/2080960180","state":["exists","up"]},{"osd":6,"uuid":"6ee6d7f4-1a5f-4a4c-ac16-48fe05f6a002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6827","nonce":4055825675}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6829","nonce":4055825675}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6833","nonce":4055825675}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6831","nonce":4055825675}]},"public_addr":"172.21.15.195:6827/4055825675","cluster_addr":"172.21.15.195:6829/4055825675","heartbeat_back_addr":"172.21.15.195:6833/4055825675","heartbeat_front_addr":"172.21.15.195:6831/4055825675","state":["exists","up"]},{"osd":7,"uuid":"efeb5f51-329a-4484-8290-92e7d3503d05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6819","nonce":1606106342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6821","nonce":1606106342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6825","nonce":1606106342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6823","nonce":1606106342}]},"public_addr":"172.21.15.195:6819/1606106342","cluster_addr":"172.21.15.195:6821/1606106342","heartbeat_back_addr":"172.21.15.195:6825/1606106342","heartbeat_front_addr":"172.21.15.195:6823/1606106342","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.901343+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:21.115003+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.901623+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.835238+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.132199+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:19.817560+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.829690+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:21.087300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T03:33:54.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:54.927+0000 7fed381c6640 1 -- 172.21.15.38:0/2310934649 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fed1c078b90 msgr2=0x7fed1c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:54.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:54.927+0000 7fed381c6640 1 --2- 172.21.15.38:0/2310934649 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fed1c078b90 0x7fed1c07b0a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fed3015e500 tx=0x7fed24005f90 comp rx=0 tx=0).stop 2024-09-21T03:33:54.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:54.927+0000 7fed381c6640 1 -- 172.21.15.38:0/2310934649 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed18003a00 msgr2=0x7fed3014c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:54.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:54.927+0000 7fed381c6640 1 --2- 172.21.15.38:0/2310934649 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed18003a00 0x7fed3014c150 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fed20036040 tx=0x7fed20033d60 comp rx=0 tx=0).stop 2024-09-21T03:33:54.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:54.927+0000 7fed381c6640 1 -- 172.21.15.38:0/2310934649 shutdown_connections 2024-09-21T03:33:54.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:54.927+0000 7fed381c6640 1 --2- 172.21.15.38:0/2310934649 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fed1c078b90 0x7fed1c07b0a0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:54.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:54.927+0000 7fed381c6640 1 --2- 172.21.15.38:0/2310934649 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fed3014dcb0 0x7fed3014e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:54.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:54.927+0000 7fed381c6640 1 --2- 172.21.15.38:0/2310934649 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fed3014d360 0x7fed3014d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:54.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:54.927+0000 7fed381c6640 1 --2- 172.21.15.38:0/2310934649 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed18003a00 0x7fed3014c150 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:54.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:54.927+0000 7fed381c6640 1 -- 172.21.15.38:0/2310934649 >> 172.21.15.38:0/2310934649 conn(0x7fed300d4440 msgr2=0x7fed30068d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:54.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:54.927+0000 7fed381c6640 1 -- 172.21.15.38:0/2310934649 shutdown_connections 2024-09-21T03:33:54.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:54.927+0000 7fed381c6640 1 -- 172.21.15.38:0/2310934649 wait complete. 2024-09-21T03:33:54.941 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-21T03:33:55.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:55.059+0000 7f41ba630640 1 -- 172.21.15.38:0/3065966028 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f41b415ee80 msgr2=0x7f41b40d0170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:55.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:55.059+0000 7f41ba630640 1 --2- 172.21.15.38:0/3065966028 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f41b415ee80 0x7f41b40d0170 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f41a4009830 tx=0x7f41a402a420 comp rx=0 tx=0).stop 2024-09-21T03:33:55.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:55.059+0000 7f41ba630640 1 -- 172.21.15.38:0/3065966028 shutdown_connections 2024-09-21T03:33:55.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:55.059+0000 7f41ba630640 1 --2- 172.21.15.38:0/3065966028 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f41b415ee80 0x7f41b40d0170 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:55.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:55.059+0000 7f41ba630640 1 -- 172.21.15.38:0/3065966028 >> 172.21.15.38:0/3065966028 conn(0x7f41b409d830 msgr2=0x7f41b40b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:55.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:55.059+0000 7f41ba630640 1 -- 172.21.15.38:0/3065966028 shutdown_connections 2024-09-21T03:33:55.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:55.059+0000 7f41ba630640 1 -- 172.21.15.38:0/3065966028 wait complete. 2024-09-21T03:33:55.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:55.060+0000 7f41ba630640 1 Processor -- start 2024-09-21T03:33:55.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:55.061+0000 7f41ba630640 1 -- start start 2024-09-21T03:33:55.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:55.062+0000 7f41ba630640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f41b40b9ad0 0x7f41b406d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:55.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:55.062+0000 7f41ba630640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f41b40ceb90 0x7f41b4157b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:55.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:55.062+0000 7f41ba630640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f41b415ee80 0x7f41b40b2950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:55.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:55.062+0000 7f41ba630640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f41b40b6910 con 0x7f41b415ee80 2024-09-21T03:33:55.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:55.062+0000 7f41ba630640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f41b40b6790 con 0x7f41b40ceb90 2024-09-21T03:33:55.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:55.062+0000 7f41ba630640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f41b40b6a90 con 0x7f41b40b9ad0 2024-09-21T03:33:55.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:55.062+0000 7f41b8bcd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f41b415ee80 0x7f41b40b2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:55.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:55.062+0000 7f41b8bcd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f41b415ee80 0x7f41b40b2950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41880/0 (socket says 172.21.15.38:41880) 2024-09-21T03:33:55.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:55.062+0000 7f41b8bcd640 1 -- 172.21.15.38:0/3589705735 learned_addr learned my addr 172.21.15.38:0/3589705735 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:33:55.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:55.062+0000 7f41b3fff640 1 --2- 172.21.15.38:0/3589705735 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f41b40b9ad0 0x7f41b406d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:55.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:55.062+0000 7f41b37fe640 1 --2- 172.21.15.38:0/3589705735 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f41b40ceb90 0x7f41b4157b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:55.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:55.062+0000 7f41b8bcd640 1 -- 172.21.15.38:0/3589705735 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f41b40b9ad0 msgr2=0x7f41b406d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:55.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:55.062+0000 7f41b8bcd640 1 --2- 172.21.15.38:0/3589705735 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f41b40b9ad0 0x7f41b406d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:55.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:55.063+0000 7f41b8bcd640 1 -- 172.21.15.38:0/3589705735 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f41b40ceb90 msgr2=0x7f41b4157b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:55.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:55.063+0000 7f41b8bcd640 1 --2- 172.21.15.38:0/3589705735 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f41b40ceb90 0x7f41b4157b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:55.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:55.063+0000 7f41b8bcd640 1 -- 172.21.15.38:0/3589705735 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41b406dd30 con 0x7f41b415ee80 2024-09-21T03:33:55.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:55.063+0000 7f41b8bcd640 1 --2- 172.21.15.38:0/3589705735 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f41b415ee80 0x7f41b40b2950 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f41ac00cbc0 tx=0x7f41ac00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:55.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:55.063+0000 7f41b0ff9640 1 -- 172.21.15.38:0/3589705735 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41ac016020 con 0x7f41b415ee80 2024-09-21T03:33:55.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:55.063+0000 7f41b0ff9640 1 -- 172.21.15.38:0/3589705735 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f41ac012420 con 0x7f41b415ee80 2024-09-21T03:33:55.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:55.063+0000 7f41ba630640 1 -- 172.21.15.38:0/3589705735 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41b406ebd0 con 0x7f41b415ee80 2024-09-21T03:33:55.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:55.063+0000 7f41b0ff9640 1 -- 172.21.15.38:0/3589705735 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41ac012870 con 0x7f41b415ee80 2024-09-21T03:33:55.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:55.063+0000 7f41ba630640 1 -- 172.21.15.38:0/3589705735 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41b4159700 con 0x7f41b415ee80 2024-09-21T03:33:55.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:55.064+0000 7f41ba630640 1 -- 172.21.15.38:0/3589705735 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41b40d0170 con 0x7f41b415ee80 2024-09-21T03:33:55.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:55.065+0000 7f41b0ff9640 1 -- 172.21.15.38:0/3589705735 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f41ac019020 con 0x7f41b415ee80 2024-09-21T03:33:55.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:55.065+0000 7f41b0ff9640 1 --2- 172.21.15.38:0/3589705735 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f41a0078b90 0x7f41a007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:55.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:55.065+0000 7f41b0ff9640 1 -- 172.21.15.38:0/3589705735 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f41ac09b300 con 0x7f41b415ee80 2024-09-21T03:33:55.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:55.066+0000 7f41b3fff640 1 --2- 172.21.15.38:0/3589705735 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f41a0078b90 0x7f41a007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:55.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:55.066+0000 7f41b3fff640 1 --2- 172.21.15.38:0/3589705735 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f41a0078b90 0x7f41a007b0a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f41a4034000 tx=0x7f41a4004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:55.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:55.070+0000 7f41b0ff9640 1 -- 172.21.15.38:0/3589705735 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f41ac020030 con 0x7f41b415ee80 2024-09-21T03:33:55.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:55.318+0000 7f41ba630640 1 -- 172.21.15.38:0/3589705735 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f41b406e930 con 0x7f41b415ee80 2024-09-21T03:33:55.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:55.327+0000 7f41b0ff9640 1 -- 172.21.15.38:0/3589705735 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v8) ==== 118+0+0 (secure 0 0 0) 0x7f41b41592d0 con 0x7f41b415ee80 2024-09-21T03:33:55.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:55.330+0000 7f41ba630640 1 -- 172.21.15.38:0/3589705735 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f41a0078b90 msgr2=0x7f41a007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:55.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:55.330+0000 7f41ba630640 1 --2- 172.21.15.38:0/3589705735 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f41a0078b90 0x7f41a007b0a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f41a4034000 tx=0x7f41a4004730 comp rx=0 tx=0).stop 2024-09-21T03:33:55.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:55.330+0000 7f41ba630640 1 -- 172.21.15.38:0/3589705735 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f41b415ee80 msgr2=0x7f41b40b2950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:55.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:55.330+0000 7f41ba630640 1 --2- 172.21.15.38:0/3589705735 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f41b415ee80 0x7f41b40b2950 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f41ac00cbc0 tx=0x7f41ac00a430 comp rx=0 tx=0).stop 2024-09-21T03:33:55.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:55.330+0000 7f41ba630640 1 -- 172.21.15.38:0/3589705735 shutdown_connections 2024-09-21T03:33:55.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:55.330+0000 7f41ba630640 1 --2- 172.21.15.38:0/3589705735 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f41b415ee80 0x7f41b40b2950 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:55.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:55.330+0000 7f41ba630640 1 --2- 172.21.15.38:0/3589705735 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f41b40ceb90 0x7f41b4157b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:55.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:55.330+0000 7f41ba630640 1 --2- 172.21.15.38:0/3589705735 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f41a0078b90 0x7f41a007b0a0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:55.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:55.330+0000 7f41ba630640 1 --2- 172.21.15.38:0/3589705735 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f41b40b9ad0 0x7f41b406d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:55.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:55.330+0000 7f41ba630640 1 -- 172.21.15.38:0/3589705735 >> 172.21.15.38:0/3589705735 conn(0x7f41b409d830 msgr2=0x7f41b40b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:55.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:55.330+0000 7f41ba630640 1 -- 172.21.15.38:0/3589705735 shutdown_connections 2024-09-21T03:33:55.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:55.330+0000 7f41ba630640 1 -- 172.21.15.38:0/3589705735 wait complete. 2024-09-21T03:33:55.344 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']} 2024-09-21T03:33:55.344 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-21T03:33:55.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:55.462+0000 7ff77d4f9640 1 -- 172.21.15.38:0/2607256274 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff760003a30 msgr2=0x7ff760003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:55.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:55.462+0000 7ff77d4f9640 1 --2- 172.21.15.38:0/2607256274 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff760003a30 0x7ff760003ea0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ff768009b50 tx=0x7ff760005830 comp rx=0 tx=0).stop 2024-09-21T03:33:55.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:55.462+0000 7ff7777fe640 1 -- 172.21.15.38:0/2607256274 reap_dead start 2024-09-21T03:33:55.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:55.462+0000 7ff77d4f9640 1 -- 172.21.15.38:0/2607256274 shutdown_connections 2024-09-21T03:33:55.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:55.462+0000 7ff77d4f9640 1 -- 172.21.15.38:0/2607256274 >> 172.21.15.38:0/2607256274 conn(0x7ff77809d830 msgr2=0x7ff7780b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:55.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:55.462+0000 7ff77d4f9640 1 -- 172.21.15.38:0/2607256274 shutdown_connections 2024-09-21T03:33:55.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:55.462+0000 7ff77d4f9640 1 -- 172.21.15.38:0/2607256274 wait complete. 2024-09-21T03:33:55.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:55.464+0000 7ff77d4f9640 1 Processor -- start 2024-09-21T03:33:55.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:55.464+0000 7ff77d4f9640 1 -- start start 2024-09-21T03:33:55.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:55.465+0000 7ff77d4f9640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff760003a30 0x7ff77814fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:55.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:55.465+0000 7ff77d4f9640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff778159ec0 0x7ff77815a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:55.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:55.465+0000 7ff77d4f9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff77815aed0 0x7ff7781f8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:55.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:55.465+0000 7ff77d4f9640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ff7780d0500 con 0x7ff77815aed0 2024-09-21T03:33:55.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:55.465+0000 7ff77d4f9640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ff7780d0380 con 0x7ff760003a30 2024-09-21T03:33:55.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:55.465+0000 7ff77d4f9640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7ff7780b6580 con 0x7ff778159ec0 2024-09-21T03:33:55.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:55.465+0000 7ff7777fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff77815aed0 0x7ff7781f8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:55.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:55.465+0000 7ff7777fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff77815aed0 0x7ff7781f8cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41892/0 (socket says 172.21.15.38:41892) 2024-09-21T03:33:55.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:55.465+0000 7ff7777fe640 1 -- 172.21.15.38:0/3637785483 learned_addr learned my addr 172.21.15.38:0/3637785483 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:33:55.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:55.466+0000 7ff7767fc640 1 --2- 172.21.15.38:0/3637785483 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff778159ec0 0x7ff77815a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:55.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:55.466+0000 7ff776ffd640 1 --2- 172.21.15.38:0/3637785483 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff760003a30 0x7ff77814fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:55.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:55.466+0000 7ff7777fe640 1 -- 172.21.15.38:0/3637785483 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff778159ec0 msgr2=0x7ff77815a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:55.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:55.466+0000 7ff7777fe640 1 --2- 172.21.15.38:0/3637785483 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff778159ec0 0x7ff77815a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:55.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:55.466+0000 7ff7777fe640 1 -- 172.21.15.38:0/3637785483 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff760003a30 msgr2=0x7ff77814fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:55.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:55.466+0000 7ff7777fe640 1 --2- 172.21.15.38:0/3637785483 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff760003a30 0x7ff77814fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:55.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:55.466+0000 7ff7777fe640 1 -- 172.21.15.38:0/3637785483 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff77815b2e0 con 0x7ff77815aed0 2024-09-21T03:33:55.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:55.466+0000 7ff7777fe640 1 --2- 172.21.15.38:0/3637785483 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff77815aed0 0x7ff7781f8cc0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ff7780d06d0 tx=0x7ff7700019b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:55.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:55.466+0000 7ff75f7fe640 1 -- 172.21.15.38:0/3637785483 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7700107d0 con 0x7ff77815aed0 2024-09-21T03:33:55.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:55.466+0000 7ff75f7fe640 1 -- 172.21.15.38:0/3637785483 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff770010970 con 0x7ff77815aed0 2024-09-21T03:33:55.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:55.466+0000 7ff77d4f9640 1 -- 172.21.15.38:0/3637785483 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff77806ce80 con 0x7ff77815aed0 2024-09-21T03:33:55.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:55.466+0000 7ff75f7fe640 1 -- 172.21.15.38:0/3637785483 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff770010da0 con 0x7ff77815aed0 2024-09-21T03:33:55.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:55.466+0000 7ff77d4f9640 1 -- 172.21.15.38:0/3637785483 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff77806d440 con 0x7ff77815aed0 2024-09-21T03:33:55.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:55.467+0000 7ff77d4f9640 1 -- 172.21.15.38:0/3637785483 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7780b6ae0 con 0x7ff77815aed0 2024-09-21T03:33:55.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:55.468+0000 7ff75f7fe640 1 -- 172.21.15.38:0/3637785483 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ff77001a400 con 0x7ff77815aed0 2024-09-21T03:33:55.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:55.468+0000 7ff75f7fe640 1 --2- 172.21.15.38:0/3637785483 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff764078bc0 0x7ff76407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:55.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:55.468+0000 7ff75f7fe640 1 -- 172.21.15.38:0/3637785483 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7ff77009b3f0 con 0x7ff77815aed0 2024-09-21T03:33:55.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:55.469+0000 7ff776ffd640 1 --2- 172.21.15.38:0/3637785483 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff764078bc0 0x7ff76407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:55.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:55.469+0000 7ff776ffd640 1 --2- 172.21.15.38:0/3637785483 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff764078bc0 0x7ff76407b0d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ff768035000 tx=0x7ff76802d040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:55.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:55.473+0000 7ff75f7fe640 1 -- 172.21.15.38:0/3637785483 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7ff77815b2e0 con 0x7ff77815aed0 2024-09-21T03:33:55.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:55.702+0000 7ff77d4f9640 1 -- 172.21.15.38:0/3637785483 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7ff7780b1510 con 0x7ff77815aed0 2024-09-21T03:33:56.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:56.333+0000 7ff75f7fe640 1 -- 172.21.15.38:0/3637785483 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v9) ==== 122+0+0 (secure 0 0 0) 0x7ff770062f20 con 0x7ff77815aed0 2024-09-21T03:33:56.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:56.336+0000 7ff77d4f9640 1 -- 172.21.15.38:0/3637785483 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff764078bc0 msgr2=0x7ff76407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:56.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:56.336+0000 7ff77d4f9640 1 --2- 172.21.15.38:0/3637785483 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff764078bc0 0x7ff76407b0d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ff768035000 tx=0x7ff76802d040 comp rx=0 tx=0).stop 2024-09-21T03:33:56.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:56.336+0000 7ff77d4f9640 1 -- 172.21.15.38:0/3637785483 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff77815aed0 msgr2=0x7ff7781f8cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:56.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:56.336+0000 7ff77d4f9640 1 --2- 172.21.15.38:0/3637785483 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff77815aed0 0x7ff7781f8cc0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ff7780d06d0 tx=0x7ff7700019b0 comp rx=0 tx=0).stop 2024-09-21T03:33:56.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:56.336+0000 7ff77d4f9640 1 -- 172.21.15.38:0/3637785483 shutdown_connections 2024-09-21T03:33:56.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:56.336+0000 7ff77d4f9640 1 --2- 172.21.15.38:0/3637785483 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff77815aed0 0x7ff7781f8cc0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:56.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:56.336+0000 7ff77d4f9640 1 --2- 172.21.15.38:0/3637785483 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff764078bc0 0x7ff76407b0d0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:56.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:56.336+0000 7ff77d4f9640 1 --2- 172.21.15.38:0/3637785483 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff778159ec0 0x7ff77815a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:56.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:56.336+0000 7ff77d4f9640 1 --2- 172.21.15.38:0/3637785483 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff760003a30 0x7ff77814fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:56.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:56.336+0000 7ff77d4f9640 1 -- 172.21.15.38:0/3637785483 >> 172.21.15.38:0/3637785483 conn(0x7ff77809d830 msgr2=0x7ff778160eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:56.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:56.336+0000 7ff77d4f9640 1 -- 172.21.15.38:0/3637785483 shutdown_connections 2024-09-21T03:33:56.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:56.336+0000 7ff77d4f9640 1 -- 172.21.15.38:0/3637785483 wait complete. 2024-09-21T03:33:56.352 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-21T03:33:56.352 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:33:56.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:56.469+0000 7f9bb1638640 1 -- 172.21.15.38:0/236415374 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9b94004d40 msgr2=0x7f9b94005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:56.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:56.469+0000 7f9bb1638640 1 --2- 172.21.15.38:0/236415374 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9b94004d40 0x7f9b94005150 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f9ba400c470 tx=0x7f9b94005830 comp rx=0 tx=0).stop 2024-09-21T03:33:56.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:56.469+0000 7f9bab7fe640 1 -- 172.21.15.38:0/236415374 reap_dead start 2024-09-21T03:33:56.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:56.470+0000 7f9bb1638640 1 -- 172.21.15.38:0/236415374 shutdown_connections 2024-09-21T03:33:56.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:56.470+0000 7f9bb1638640 1 -- 172.21.15.38:0/236415374 >> 172.21.15.38:0/236415374 conn(0x7f9bac0d4440 msgr2=0x7f9bac0d4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:56.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:56.470+0000 7f9bb1638640 1 -- 172.21.15.38:0/236415374 shutdown_connections 2024-09-21T03:33:56.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:56.470+0000 7f9bb1638640 1 -- 172.21.15.38:0/236415374 wait complete. 2024-09-21T03:33:56.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:56.471+0000 7f9bb1638640 1 Processor -- start 2024-09-21T03:33:56.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:56.472+0000 7f9bb1638640 1 -- start start 2024-09-21T03:33:56.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:56.473+0000 7f9bb1638640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9b94004d40 0x7f9bac14d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:56.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:56.473+0000 7f9bb1638640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9bac1f5e50 0x7f9bac0ca660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:56.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:56.473+0000 7f9bb1638640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9bac1f6260 0x7f9bac14cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:56.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:56.473+0000 7f9bb1638640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f9bac0d2a20 con 0x7f9bac1f5e50 2024-09-21T03:33:56.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:56.473+0000 7f9bb1638640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f9bac0b3b90 con 0x7f9b94004d40 2024-09-21T03:33:56.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:56.473+0000 7f9bb1638640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f9bac0d2ba0 con 0x7f9bac1f6260 2024-09-21T03:33:56.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:56.473+0000 7f9baa7fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9bac1f5e50 0x7f9bac0ca660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:56.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:56.473+0000 7f9baa7fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9bac1f5e50 0x7f9bac0ca660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41912/0 (socket says 172.21.15.38:41912) 2024-09-21T03:33:56.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:56.473+0000 7f9baa7fc640 1 -- 172.21.15.38:0/187956280 learned_addr learned my addr 172.21.15.38:0/187956280 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:33:56.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:56.473+0000 7f9baaffd640 1 --2- 172.21.15.38:0/187956280 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9b94004d40 0x7f9bac14d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:56.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:56.473+0000 7f9bab7fe640 1 --2- 172.21.15.38:0/187956280 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9bac1f6260 0x7f9bac14cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:56.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:56.473+0000 7f9baa7fc640 1 -- 172.21.15.38:0/187956280 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9bac1f6260 msgr2=0x7f9bac14cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:56.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:56.473+0000 7f9baa7fc640 1 --2- 172.21.15.38:0/187956280 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9bac1f6260 0x7f9bac14cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:56.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:56.473+0000 7f9baa7fc640 1 -- 172.21.15.38:0/187956280 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9b94004d40 msgr2=0x7f9bac14d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:56.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:56.473+0000 7f9baa7fc640 1 --2- 172.21.15.38:0/187956280 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9b94004d40 0x7f9bac14d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:56.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:56.473+0000 7f9baa7fc640 1 -- 172.21.15.38:0/187956280 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bac0c95c0 con 0x7f9bac1f5e50 2024-09-21T03:33:56.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:56.474+0000 7f9baa7fc640 1 --2- 172.21.15.38:0/187956280 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9bac1f5e50 0x7f9bac0ca660 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f9ba0010aa0 tx=0x7f9ba0010f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:56.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:56.474+0000 7f9b937fe640 1 -- 172.21.15.38:0/187956280 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ba000ec20 con 0x7f9bac1f5e50 2024-09-21T03:33:56.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:56.474+0000 7f9b937fe640 1 -- 172.21.15.38:0/187956280 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9ba000edc0 con 0x7f9bac1f5e50 2024-09-21T03:33:56.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:56.474+0000 7f9bb1638640 1 -- 172.21.15.38:0/187956280 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bac0c98a0 con 0x7f9bac1f5e50 2024-09-21T03:33:56.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:56.474+0000 7f9b937fe640 1 -- 172.21.15.38:0/187956280 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ba0005260 con 0x7f9bac1f5e50 2024-09-21T03:33:56.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:56.474+0000 7f9bb1638640 1 -- 172.21.15.38:0/187956280 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bac0c9e60 con 0x7f9bac1f5e50 2024-09-21T03:33:56.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:56.475+0000 7f9bb1638640 1 -- 172.21.15.38:0/187956280 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9bac1629a0 con 0x7f9bac1f5e50 2024-09-21T03:33:56.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:56.476+0000 7f9b937fe640 1 -- 172.21.15.38:0/187956280 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f9ba0005400 con 0x7f9bac1f5e50 2024-09-21T03:33:56.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:56.477+0000 7f9b937fe640 1 --2- 172.21.15.38:0/187956280 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9b98078b90 0x7f9b9807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:56.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:56.477+0000 7f9b937fe640 1 -- 172.21.15.38:0/187956280 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f9ba009c230 con 0x7f9bac1f5e50 2024-09-21T03:33:56.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:56.477+0000 7f9baaffd640 1 --2- 172.21.15.38:0/187956280 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9b98078b90 0x7f9b9807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:56.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:56.478+0000 7f9baaffd640 1 --2- 172.21.15.38:0/187956280 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9b98078b90 0x7f9b9807b0a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f9b9c005fd0 tx=0x7f9b9c009500 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:56.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:56.480+0000 7f9b937fe640 1 -- 172.21.15.38:0/187956280 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f9ba0029080 con 0x7f9bac1f5e50 2024-09-21T03:33:56.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:56.704+0000 7f9bb1638640 1 -- 172.21.15.38:0/187956280 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9bac0cecc0 con 0x7f9bac1f5e50 2024-09-21T03:33:56.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:56.704+0000 7f9b937fe640 1 -- 172.21.15.38:0/187956280 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5219 (secure 0 0 0) 0x7f9ba0063d70 con 0x7f9bac1f5e50 2024-09-21T03:33:56.707 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:33:56.707 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":9,"btime":"2024-09-21T03:33:56:329606+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4322,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/593838590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":593838590},{"type":"v1","addr":"172.21.15.195:6836","nonce":593838590}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4374,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/663402481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":663402481},{"type":"v1","addr":"172.21.15.38:6833","nonce":663402481}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4377,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/2614829035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":2614829035},{"type":"v1","addr":"172.21.15.38:6835","nonce":2614829035}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:33:35.622623+0000","modified":"2024-09-21T03:33:55.704494+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4402},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4402":{"gid":4402,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.195:6837/1420805977","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":1420805977},{"type":"v1","addr":"172.21.15.195:6837","nonce":1420805977}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4402,"qdb_cluster":[4402]},"id":1}]} 2024-09-21T03:33:56.708 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 9 2024-09-21T03:33:56.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:56.706+0000 7f9bb1638640 1 -- 172.21.15.38:0/187956280 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9b98078b90 msgr2=0x7f9b9807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:56.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:56.706+0000 7f9bb1638640 1 --2- 172.21.15.38:0/187956280 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9b98078b90 0x7f9b9807b0a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f9b9c005fd0 tx=0x7f9b9c009500 comp rx=0 tx=0).stop 2024-09-21T03:33:56.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:56.706+0000 7f9bb1638640 1 -- 172.21.15.38:0/187956280 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9bac1f5e50 msgr2=0x7f9bac0ca660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:56.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:56.706+0000 7f9bb1638640 1 --2- 172.21.15.38:0/187956280 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9bac1f5e50 0x7f9bac0ca660 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f9ba0010aa0 tx=0x7f9ba0010f60 comp rx=0 tx=0).stop 2024-09-21T03:33:56.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:56.707+0000 7f9bb1638640 1 -- 172.21.15.38:0/187956280 shutdown_connections 2024-09-21T03:33:56.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:56.707+0000 7f9bb1638640 1 --2- 172.21.15.38:0/187956280 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9b98078b90 0x7f9b9807b0a0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:56.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:56.707+0000 7f9bb1638640 1 --2- 172.21.15.38:0/187956280 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9bac1f6260 0x7f9bac14cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:56.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:56.707+0000 7f9bb1638640 1 --2- 172.21.15.38:0/187956280 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9bac1f5e50 0x7f9bac0ca660 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:56.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:56.707+0000 7f9bb1638640 1 --2- 172.21.15.38:0/187956280 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9b94004d40 0x7f9bac14d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:56.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:56.707+0000 7f9bb1638640 1 -- 172.21.15.38:0/187956280 >> 172.21.15.38:0/187956280 conn(0x7f9bac0d4440 msgr2=0x7f9bac0b3270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:56.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:56.707+0000 7f9bb1638640 1 -- 172.21.15.38:0/187956280 shutdown_connections 2024-09-21T03:33:56.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:56.707+0000 7f9bb1638640 1 -- 172.21.15.38:0/187956280 wait complete. 2024-09-21T03:33:56.721 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T03:33:56.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:56.834+0000 7fb2b40a9640 1 -- 172.21.15.38:0/3941346415 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb298004d40 msgr2=0x7fb298005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:56.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:56.834+0000 7fb2b40a9640 1 --2- 172.21.15.38:0/3941346415 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb298004d40 0x7fb298005150 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb2a4005fd0 tx=0x7fb298005830 comp rx=0 tx=0).stop 2024-09-21T03:33:56.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:56.834+0000 7fb2b2646640 1 -- 172.21.15.38:0/3941346415 reap_dead start 2024-09-21T03:33:56.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:56.834+0000 7fb2b40a9640 1 -- 172.21.15.38:0/3941346415 shutdown_connections 2024-09-21T03:33:56.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:56.834+0000 7fb2b40a9640 1 -- 172.21.15.38:0/3941346415 >> 172.21.15.38:0/3941346415 conn(0x7fb2ac0d4440 msgr2=0x7fb2ac0d4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:56.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:56.834+0000 7fb2b40a9640 1 -- 172.21.15.38:0/3941346415 shutdown_connections 2024-09-21T03:33:56.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:56.834+0000 7fb2b40a9640 1 -- 172.21.15.38:0/3941346415 wait complete. 2024-09-21T03:33:56.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:56.836+0000 7fb2b40a9640 1 Processor -- start 2024-09-21T03:33:56.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:56.836+0000 7fb2b40a9640 1 -- start start 2024-09-21T03:33:56.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:56.837+0000 7fb2b40a9640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb298004d40 0x7fb2ac0c86d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:56.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:56.838+0000 7fb2b40a9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb2ac15e460 0x7fb2ac15cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:56.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:56.838+0000 7fb2b40a9640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb2ac15d2c0 0x7fb2ac14df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:56.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:56.838+0000 7fb2b40a9640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fb2ac0b3b90 con 0x7fb2ac15e460 2024-09-21T03:33:56.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:56.838+0000 7fb2b40a9640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fb2ac0b3a10 con 0x7fb298004d40 2024-09-21T03:33:56.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:56.838+0000 7fb2b40a9640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fb2ac0d2a20 con 0x7fb2ac15d2c0 2024-09-21T03:33:56.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:56.838+0000 7fb2b1644640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb2ac15e460 0x7fb2ac15cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:56.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:56.838+0000 7fb2b1644640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb2ac15e460 0x7fb2ac15cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41930/0 (socket says 172.21.15.38:41930) 2024-09-21T03:33:56.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:56.838+0000 7fb2b1644640 1 -- 172.21.15.38:0/3695927336 learned_addr learned my addr 172.21.15.38:0/3695927336 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:33:56.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:56.838+0000 7fb2b2646640 1 --2- 172.21.15.38:0/3695927336 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb2ac15d2c0 0x7fb2ac14df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:56.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:56.838+0000 7fb2b1e45640 1 --2- 172.21.15.38:0/3695927336 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb298004d40 0x7fb2ac0c86d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:56.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:56.838+0000 7fb2b1644640 1 -- 172.21.15.38:0/3695927336 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb2ac15d2c0 msgr2=0x7fb2ac14df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:56.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:56.838+0000 7fb2b1644640 1 --2- 172.21.15.38:0/3695927336 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb2ac15d2c0 0x7fb2ac14df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:56.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:56.838+0000 7fb2b1644640 1 -- 172.21.15.38:0/3695927336 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb298004d40 msgr2=0x7fb2ac0c86d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:56.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:56.838+0000 7fb2b1644640 1 --2- 172.21.15.38:0/3695927336 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb298004d40 0x7fb2ac0c86d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:56.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:56.838+0000 7fb2b1644640 1 -- 172.21.15.38:0/3695927336 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2ac15b6e0 con 0x7fb2ac15e460 2024-09-21T03:33:56.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:56.838+0000 7fb2b1644640 1 --2- 172.21.15.38:0/3695927336 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb2ac15e460 0x7fb2ac15cd80 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fb29c0066b0 tx=0x7fb29c0039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:56.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:56.838+0000 7fb2a27fc640 1 -- 172.21.15.38:0/3695927336 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb29c015070 con 0x7fb2ac15e460 2024-09-21T03:33:56.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:56.838+0000 7fb2a27fc640 1 -- 172.21.15.38:0/3695927336 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb29c011920 con 0x7fb2ac15e460 2024-09-21T03:33:56.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:56.838+0000 7fb2b40a9640 1 -- 172.21.15.38:0/3695927336 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2ac0c8f00 con 0x7fb2ac15e460 2024-09-21T03:33:56.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:56.838+0000 7fb2a27fc640 1 -- 172.21.15.38:0/3695927336 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb29c011d70 con 0x7fb2ac15e460 2024-09-21T03:33:56.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:56.838+0000 7fb2b40a9640 1 -- 172.21.15.38:0/3695927336 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2ac15bc60 con 0x7fb2ac15e460 2024-09-21T03:33:56.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:56.839+0000 7fb2b40a9640 1 -- 172.21.15.38:0/3695927336 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2ac0d2fb0 con 0x7fb2ac15e460 2024-09-21T03:33:56.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:56.840+0000 7fb2a27fc640 1 -- 172.21.15.38:0/3695927336 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb29c010420 con 0x7fb2ac15e460 2024-09-21T03:33:56.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:56.841+0000 7fb2a27fc640 1 --2- 172.21.15.38:0/3695927336 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb290078b90 0x7fb29007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:56.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:56.841+0000 7fb2a27fc640 1 -- 172.21.15.38:0/3695927336 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fb29c09be90 con 0x7fb2ac15e460 2024-09-21T03:33:56.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:56.841+0000 7fb2b1e45640 1 --2- 172.21.15.38:0/3695927336 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb290078b90 0x7fb29007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:56.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:56.841+0000 7fb2b1e45640 1 --2- 172.21.15.38:0/3695927336 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb290078b90 0x7fb29007b0a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb2a8009b90 tx=0x7fb2a8005f90 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:56.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:56.845+0000 7fb2a27fc640 1 -- 172.21.15.38:0/3695927336 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fb29c063f00 con 0x7fb2ac15e460 2024-09-21T03:33:57.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:57.033+0000 7fb2b40a9640 1 -- 172.21.15.38:0/3695927336 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb2ac0c8c40 con 0x7fb2ac15e460 2024-09-21T03:33:57.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:57.034+0000 7fb2a27fc640 1 -- 172.21.15.38:0/3695927336 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18626 (secure 0 0 0) 0x7fb29c09d090 con 0x7fb2ac15e460 2024-09-21T03:33:57.037 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:33:57.038 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":26,"fsid":"9d50d66f-656b-4fe2-8e3b-fe53b79c4d43","created":"2024-09-21T03:33:10.677865+0000","modified":"2024-09-21T03:33:40.665991+0000","last_up_change":"2024-09-21T03:33:24.023421+0000","last_in_change":"2024-09-21T03:33:14.296193+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T03:33:22.992573+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T03:33:33.530201+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T03:33:34.585875+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T03:33:38.268346+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"dbab183f-7e63-48b8-960a-ae55fd5bf565","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6801","nonce":2381639433}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6803","nonce":2381639433}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6807","nonce":2381639433}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6805","nonce":2381639433}]},"public_addr":"172.21.15.38:6801/2381639433","cluster_addr":"172.21.15.38:6803/2381639433","heartbeat_back_addr":"172.21.15.38:6807/2381639433","heartbeat_front_addr":"172.21.15.38:6805/2381639433","state":["exists","up"]},{"osd":1,"uuid":"fa1c0e8a-ebc1-4359-8146-bd5342abbc16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6817","nonce":1491818732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6819","nonce":1491818732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6823","nonce":1491818732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6821","nonce":1491818732}]},"public_addr":"172.21.15.38:6817/1491818732","cluster_addr":"172.21.15.38:6819/1491818732","heartbeat_back_addr":"172.21.15.38:6823/1491818732","heartbeat_front_addr":"172.21.15.38:6821/1491818732","state":["exists","up"]},{"osd":2,"uuid":"2ed3cd2d-385d-4fc6-9869-214552a84803","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6825","nonce":2577858902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6827","nonce":2577858902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6831","nonce":2577858902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6829","nonce":2577858902}]},"public_addr":"172.21.15.38:6825/2577858902","cluster_addr":"172.21.15.38:6827/2577858902","heartbeat_back_addr":"172.21.15.38:6831/2577858902","heartbeat_front_addr":"172.21.15.38:6829/2577858902","state":["exists","up"]},{"osd":3,"uuid":"62d1d95e-919d-4250-be71-1d34cfa70dbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6809","nonce":1286810294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6811","nonce":1286810294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6815","nonce":1286810294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6813","nonce":1286810294}]},"public_addr":"172.21.15.38:6809/1286810294","cluster_addr":"172.21.15.38:6811/1286810294","heartbeat_back_addr":"172.21.15.38:6815/1286810294","heartbeat_front_addr":"172.21.15.38:6813/1286810294","state":["exists","up"]},{"osd":4,"uuid":"7c9f9314-99d7-4d73-8392-37809a9f54b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6803","nonce":3401837571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6805","nonce":3401837571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6809","nonce":3401837571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6807","nonce":3401837571}]},"public_addr":"172.21.15.195:6803/3401837571","cluster_addr":"172.21.15.195:6805/3401837571","heartbeat_back_addr":"172.21.15.195:6809/3401837571","heartbeat_front_addr":"172.21.15.195:6807/3401837571","state":["exists","up"]},{"osd":5,"uuid":"bde4fe9f-8414-4bba-bf4a-43973d1976bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6811","nonce":2080960180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6813","nonce":2080960180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6817","nonce":2080960180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6815","nonce":2080960180}]},"public_addr":"172.21.15.195:6811/2080960180","cluster_addr":"172.21.15.195:6813/2080960180","heartbeat_back_addr":"172.21.15.195:6817/2080960180","heartbeat_front_addr":"172.21.15.195:6815/2080960180","state":["exists","up"]},{"osd":6,"uuid":"6ee6d7f4-1a5f-4a4c-ac16-48fe05f6a002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6827","nonce":4055825675}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6829","nonce":4055825675}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6833","nonce":4055825675}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6831","nonce":4055825675}]},"public_addr":"172.21.15.195:6827/4055825675","cluster_addr":"172.21.15.195:6829/4055825675","heartbeat_back_addr":"172.21.15.195:6833/4055825675","heartbeat_front_addr":"172.21.15.195:6831/4055825675","state":["exists","up"]},{"osd":7,"uuid":"efeb5f51-329a-4484-8290-92e7d3503d05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6819","nonce":1606106342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6821","nonce":1606106342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6825","nonce":1606106342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6823","nonce":1606106342}]},"public_addr":"172.21.15.195:6819/1606106342","cluster_addr":"172.21.15.195:6821/1606106342","heartbeat_back_addr":"172.21.15.195:6825/1606106342","heartbeat_front_addr":"172.21.15.195:6823/1606106342","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.901343+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:21.115003+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.901623+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.835238+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.132199+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:19.817560+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.829690+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:21.087300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T03:33:57.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:57.037+0000 7fb2b40a9640 1 -- 172.21.15.38:0/3695927336 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb290078b90 msgr2=0x7fb29007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:57.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:57.037+0000 7fb2b40a9640 1 --2- 172.21.15.38:0/3695927336 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb290078b90 0x7fb29007b0a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb2a8009b90 tx=0x7fb2a8005f90 comp rx=0 tx=0).stop 2024-09-21T03:33:57.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:57.037+0000 7fb2b40a9640 1 -- 172.21.15.38:0/3695927336 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb2ac15e460 msgr2=0x7fb2ac15cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:57.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:57.037+0000 7fb2b40a9640 1 --2- 172.21.15.38:0/3695927336 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb2ac15e460 0x7fb2ac15cd80 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fb29c0066b0 tx=0x7fb29c0039c0 comp rx=0 tx=0).stop 2024-09-21T03:33:57.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:57.037+0000 7fb2b40a9640 1 -- 172.21.15.38:0/3695927336 shutdown_connections 2024-09-21T03:33:57.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:57.037+0000 7fb2b40a9640 1 --2- 172.21.15.38:0/3695927336 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb290078b90 0x7fb29007b0a0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:57.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:57.037+0000 7fb2b40a9640 1 --2- 172.21.15.38:0/3695927336 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb2ac15d2c0 0x7fb2ac14df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:57.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:57.037+0000 7fb2b40a9640 1 --2- 172.21.15.38:0/3695927336 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb2ac15e460 0x7fb2ac15cd80 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:57.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:57.037+0000 7fb2b40a9640 1 --2- 172.21.15.38:0/3695927336 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb298004d40 0x7fb2ac0c86d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:57.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:57.037+0000 7fb2b40a9640 1 -- 172.21.15.38:0/3695927336 >> 172.21.15.38:0/3695927336 conn(0x7fb2ac0d4440 msgr2=0x7fb2ac0d4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:57.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:57.037+0000 7fb2b40a9640 1 -- 172.21.15.38:0/3695927336 shutdown_connections 2024-09-21T03:33:57.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:57.037+0000 7fb2b40a9640 1 -- 172.21.15.38:0/3695927336 wait complete. 2024-09-21T03:33:57.051 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-21T03:33:57.051 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T03:33:57.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:57.164+0000 7f8634b43640 1 -- 172.21.15.38:0/167830827 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f863015cf20 msgr2=0x7f86300b6960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:57.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:57.164+0000 7f8634b43640 1 --2- 172.21.15.38:0/167830827 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f863015cf20 0x7f86300b6960 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f8628009be0 tx=0x7f862802a890 comp rx=0 tx=0).stop 2024-09-21T03:33:57.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:57.164+0000 7f8634b43640 1 -- 172.21.15.38:0/167830827 shutdown_connections 2024-09-21T03:33:57.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:57.164+0000 7f8634b43640 1 --2- 172.21.15.38:0/167830827 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f863015cf20 0x7f86300b6960 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:57.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:57.164+0000 7f8634b43640 1 -- 172.21.15.38:0/167830827 >> 172.21.15.38:0/167830827 conn(0x7f86300d2350 msgr2=0x7f86300d2760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:57.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:57.165+0000 7f8634b43640 1 -- 172.21.15.38:0/167830827 shutdown_connections 2024-09-21T03:33:57.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:57.165+0000 7f8634b43640 1 -- 172.21.15.38:0/167830827 wait complete. 2024-09-21T03:33:57.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:57.167+0000 7f8634b43640 1 Processor -- start 2024-09-21T03:33:57.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:57.167+0000 7f8634b43640 1 -- start start 2024-09-21T03:33:57.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:57.168+0000 7f8634b43640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f86300d19f0 0x7f8630154640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:57.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:57.168+0000 7f8634b43640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f863015cb10 0x7f863013f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:57.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:57.168+0000 7f8634b43640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f863015cf20 0x7f863013fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:57.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:57.168+0000 7f8634b43640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f86300b7020 con 0x7f863015cf20 2024-09-21T03:33:57.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:57.168+0000 7f8634b43640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f86300b6ea0 con 0x7f86300d19f0 2024-09-21T03:33:57.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:57.168+0000 7f8634b43640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f86300b71a0 con 0x7f863015cb10 2024-09-21T03:33:57.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:57.168+0000 7f862ed9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f863015cf20 0x7f863013fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:57.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:57.168+0000 7f862ed9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f863015cf20 0x7f863013fc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41938/0 (socket says 172.21.15.38:41938) 2024-09-21T03:33:57.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:57.168+0000 7f862ed9d640 1 -- 172.21.15.38:0/3432501875 learned_addr learned my addr 172.21.15.38:0/3432501875 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:33:57.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:57.168+0000 7f862e59c640 1 --2- 172.21.15.38:0/3432501875 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f86300d19f0 0x7f8630154640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:57.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:57.168+0000 7f8625d9b640 1 --2- 172.21.15.38:0/3432501875 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f863015cb10 0x7f863013f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:57.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:57.168+0000 7f862ed9d640 1 -- 172.21.15.38:0/3432501875 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f863015cb10 msgr2=0x7f863013f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:57.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:57.168+0000 7f862ed9d640 1 --2- 172.21.15.38:0/3432501875 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f863015cb10 0x7f863013f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:57.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:57.168+0000 7f862ed9d640 1 -- 172.21.15.38:0/3432501875 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f86300d19f0 msgr2=0x7f8630154640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:57.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:57.168+0000 7f862ed9d640 1 --2- 172.21.15.38:0/3432501875 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f86300d19f0 0x7f8630154640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:57.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:57.168+0000 7f862ed9d640 1 -- 172.21.15.38:0/3432501875 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8630154e70 con 0x7f863015cf20 2024-09-21T03:33:57.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:57.169+0000 7f862ed9d640 1 --2- 172.21.15.38:0/3432501875 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f863015cf20 0x7f863013fc40 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f8628035040 tx=0x7f8628004380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:57.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:57.169+0000 7f86277fe640 1 -- 172.21.15.38:0/3432501875 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8628041020 con 0x7f863015cf20 2024-09-21T03:33:57.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:57.169+0000 7f86277fe640 1 -- 172.21.15.38:0/3432501875 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8628004880 con 0x7f863015cf20 2024-09-21T03:33:57.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:57.169+0000 7f8634b43640 1 -- 172.21.15.38:0/3432501875 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8630155360 con 0x7f863015cf20 2024-09-21T03:33:57.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:57.169+0000 7f86277fe640 1 -- 172.21.15.38:0/3432501875 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8628037040 con 0x7f863015cf20 2024-09-21T03:33:57.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:57.169+0000 7f8634b43640 1 -- 172.21.15.38:0/3432501875 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8630155920 con 0x7f863015cf20 2024-09-21T03:33:57.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:57.170+0000 7f8634b43640 1 -- 172.21.15.38:0/3432501875 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86300d12e0 con 0x7f863015cf20 2024-09-21T03:33:57.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:57.171+0000 7f86277fe640 1 -- 172.21.15.38:0/3432501875 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f8628044020 con 0x7f863015cf20 2024-09-21T03:33:57.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:57.171+0000 7f86277fe640 1 --2- 172.21.15.38:0/3432501875 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8610078b90 0x7f861007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:57.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:57.171+0000 7f86277fe640 1 -- 172.21.15.38:0/3432501875 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f86280bb7e0 con 0x7f863015cf20 2024-09-21T03:33:57.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:57.171+0000 7f862e59c640 1 --2- 172.21.15.38:0/3432501875 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8610078b90 0x7f861007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:57.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:57.172+0000 7f862e59c640 1 --2- 172.21.15.38:0/3432501875 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8610078b90 0x7f861007b0a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f8620001860 tx=0x7f8620009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:57.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:57.176+0000 7f86277fe640 1 -- 172.21.15.38:0/3432501875 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f862804b070 con 0x7f863015cf20 2024-09-21T03:33:57.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:57.363+0000 7f8634b43640 1 -- 172.21.15.38:0/3432501875 --> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f863014bc20 con 0x7f8610078b90 2024-09-21T03:33:57.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:57.370+0000 7f86277fe640 1 -- 172.21.15.38:0/3432501875 <== mgr.4117 v2:172.21.15.195:6800/1631901344 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332519 (secure 0 0 0) 0x7f863014bc20 con 0x7f8610078b90 2024-09-21T03:33:57.376 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:33:57.377 INFO:teuthology.orchestra.run.smithi038.stderr:dumped all 2024-09-21T03:33:57.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:57.373+0000 7f8634b43640 1 -- 172.21.15.38:0/3432501875 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8610078b90 msgr2=0x7f861007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:57.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:57.373+0000 7f8634b43640 1 --2- 172.21.15.38:0/3432501875 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8610078b90 0x7f861007b0a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f8620001860 tx=0x7f8620009040 comp rx=0 tx=0).stop 2024-09-21T03:33:57.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:57.373+0000 7f8634b43640 1 -- 172.21.15.38:0/3432501875 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f863015cf20 msgr2=0x7f863013fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:57.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:57.373+0000 7f8634b43640 1 --2- 172.21.15.38:0/3432501875 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f863015cf20 0x7f863013fc40 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f8628035040 tx=0x7f8628004380 comp rx=0 tx=0).stop 2024-09-21T03:33:57.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:57.373+0000 7f8634b43640 1 -- 172.21.15.38:0/3432501875 shutdown_connections 2024-09-21T03:33:57.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:57.373+0000 7f8634b43640 1 --2- 172.21.15.38:0/3432501875 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f863015cf20 0x7f863013fc40 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:57.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:57.373+0000 7f8634b43640 1 --2- 172.21.15.38:0/3432501875 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8610078b90 0x7f861007b0a0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:57.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:57.373+0000 7f8634b43640 1 --2- 172.21.15.38:0/3432501875 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f863015cb10 0x7f863013f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:57.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:57.373+0000 7f8634b43640 1 --2- 172.21.15.38:0/3432501875 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f86300d19f0 0x7f8630154640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:57.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:57.373+0000 7f8634b43640 1 -- 172.21.15.38:0/3432501875 >> 172.21.15.38:0/3432501875 conn(0x7f86300d2350 msgr2=0x7f8630070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:57.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:57.373+0000 7f8634b43640 1 -- 172.21.15.38:0/3432501875 shutdown_connections 2024-09-21T03:33:57.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:57.373+0000 7f8634b43640 1 -- 172.21.15.38:0/3432501875 wait complete. 2024-09-21T03:33:57.387 INFO:teuthology.orchestra.run.smithi038.stdout:{"pg_ready":true,"pg_map":{"version":36,"stamp":"2024-09-21T03:33:56.982031+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":149,"num_write_kb":1646,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":114,"ondisk_log_size":114,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":252840,"kb_used_data":3336,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754721880,"statfs":{"total":773094113280,"available":772835205120,"internally_reserved":0,"allocated":3416064,"data_stored":2048944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12748,"internal_metadata":255315508},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2230,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003913"},"pg_stats":[{"pgid":"2.1f","version":"21'4","reported_seq":27,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.285664+0000","last_change":"2024-09-21T03:33:35.252626+0000","last_active":"2024-09-21T03:33:41.285664+0000","last_peered":"2024-09-21T03:33:41.285664+0000","last_clean":"2024-09-21T03:33:41.285664+0000","last_became_active":"2024-09-21T03:33:35.252204+0000","last_became_peered":"2024-09-21T03:33:35.252204+0000","last_unstale":"2024-09-21T03:33:41.285664+0000","last_undegraded":"2024-09-21T03:33:41.285664+0000","last_fullsized":"2024-09-21T03:33:41.285664+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_clean_scrub_stamp":"2024-09-21T03:33:34.220881+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:42:59.930617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.920863+0000","last_change":"2024-09-21T03:33:36.262395+0000","last_active":"2024-09-21T03:33:42.920863+0000","last_peered":"2024-09-21T03:33:42.920863+0000","last_clean":"2024-09-21T03:33:42.920863+0000","last_became_active":"2024-09-21T03:33:36.262085+0000","last_became_peered":"2024-09-21T03:33:36.262085+0000","last_unstale":"2024-09-21T03:33:42.920863+0000","last_undegraded":"2024-09-21T03:33:42.920863+0000","last_fullsized":"2024-09-21T03:33:42.920863+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:13:46.036072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.754945+0000","last_change":"2024-09-21T03:33:40.303707+0000","last_active":"2024-09-21T03:33:41.754945+0000","last_peered":"2024-09-21T03:33:41.754945+0000","last_clean":"2024-09-21T03:33:41.754945+0000","last_became_active":"2024-09-21T03:33:40.303328+0000","last_became_peered":"2024-09-21T03:33:40.303328+0000","last_unstale":"2024-09-21T03:33:41.754945+0000","last_undegraded":"2024-09-21T03:33:41.754945+0000","last_fullsized":"2024-09-21T03:33:41.754945+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_clean_scrub_stamp":"2024-09-21T03:33:39.262231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:58:01.084602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4,1],"acting":[7,6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.921084+0000","last_change":"2024-09-21T03:33:35.253147+0000","last_active":"2024-09-21T03:33:42.921084+0000","last_peered":"2024-09-21T03:33:42.921084+0000","last_clean":"2024-09-21T03:33:42.921084+0000","last_became_active":"2024-09-21T03:33:35.251243+0000","last_became_peered":"2024-09-21T03:33:35.251243+0000","last_unstale":"2024-09-21T03:33:42.921084+0000","last_undegraded":"2024-09-21T03:33:42.921084+0000","last_fullsized":"2024-09-21T03:33:42.921084+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_clean_scrub_stamp":"2024-09-21T03:33:34.220881+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:21:52.640936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.920870+0000","last_change":"2024-09-21T03:33:36.255323+0000","last_active":"2024-09-21T03:33:42.920870+0000","last_peered":"2024-09-21T03:33:42.920870+0000","last_clean":"2024-09-21T03:33:42.920870+0000","last_became_active":"2024-09-21T03:33:36.253192+0000","last_became_peered":"2024-09-21T03:33:36.253192+0000","last_unstale":"2024-09-21T03:33:42.920870+0000","last_undegraded":"2024-09-21T03:33:42.920870+0000","last_fullsized":"2024-09-21T03:33:42.920870+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:21:43.036932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.285988+0000","last_change":"2024-09-21T03:33:40.294983+0000","last_active":"2024-09-21T03:33:41.285988+0000","last_peered":"2024-09-21T03:33:41.285988+0000","last_clean":"2024-09-21T03:33:41.285988+0000","last_became_active":"2024-09-21T03:33:40.294528+0000","last_became_peered":"2024-09-21T03:33:40.294528+0000","last_unstale":"2024-09-21T03:33:41.285988+0000","last_undegraded":"2024-09-21T03:33:41.285988+0000","last_fullsized":"2024-09-21T03:33:41.285988+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_clean_scrub_stamp":"2024-09-21T03:33:39.262231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:39:42.878029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4,6],"acting":[2,7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.298804+0000","last_change":"2024-09-21T03:33:35.260455+0000","last_active":"2024-09-21T03:33:41.298804+0000","last_peered":"2024-09-21T03:33:41.298804+0000","last_clean":"2024-09-21T03:33:41.298804+0000","last_became_active":"2024-09-21T03:33:35.260074+0000","last_became_peered":"2024-09-21T03:33:35.260074+0000","last_unstale":"2024-09-21T03:33:41.298804+0000","last_undegraded":"2024-09-21T03:33:41.298804+0000","last_fullsized":"2024-09-21T03:33:41.298804+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_clean_scrub_stamp":"2024-09-21T03:33:34.220881+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:55:02.392036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.920774+0000","last_change":"2024-09-21T03:33:36.274462+0000","last_active":"2024-09-21T03:33:42.920774+0000","last_peered":"2024-09-21T03:33:42.920774+0000","last_clean":"2024-09-21T03:33:42.920774+0000","last_became_active":"2024-09-21T03:33:36.274080+0000","last_became_peered":"2024-09-21T03:33:36.274080+0000","last_unstale":"2024-09-21T03:33:42.920774+0000","last_undegraded":"2024-09-21T03:33:42.920774+0000","last_fullsized":"2024-09-21T03:33:42.920774+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:55:20.815479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.286750+0000","last_change":"2024-09-21T03:33:40.294613+0000","last_active":"2024-09-21T03:33:41.286750+0000","last_peered":"2024-09-21T03:33:41.286750+0000","last_clean":"2024-09-21T03:33:41.286750+0000","last_became_active":"2024-09-21T03:33:40.294138+0000","last_became_peered":"2024-09-21T03:33:40.294138+0000","last_unstale":"2024-09-21T03:33:41.286750+0000","last_undegraded":"2024-09-21T03:33:41.286750+0000","last_fullsized":"2024-09-21T03:33:41.286750+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_clean_scrub_stamp":"2024-09-21T03:33:39.262231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:10:29.758718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3,6],"acting":[2,5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.298963+0000","last_change":"2024-09-21T03:33:35.265118+0000","last_active":"2024-09-21T03:33:41.298963+0000","last_peered":"2024-09-21T03:33:41.298963+0000","last_clean":"2024-09-21T03:33:41.298963+0000","last_became_active":"2024-09-21T03:33:35.262875+0000","last_became_peered":"2024-09-21T03:33:35.262875+0000","last_unstale":"2024-09-21T03:33:41.298963+0000","last_undegraded":"2024-09-21T03:33:41.298963+0000","last_fullsized":"2024-09-21T03:33:41.298963+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_clean_scrub_stamp":"2024-09-21T03:33:34.220881+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:40:22.752363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.921523+0000","last_change":"2024-09-21T03:33:36.266776+0000","last_active":"2024-09-21T03:33:42.921523+0000","last_peered":"2024-09-21T03:33:42.921523+0000","last_clean":"2024-09-21T03:33:42.921523+0000","last_became_active":"2024-09-21T03:33:36.266449+0000","last_became_peered":"2024-09-21T03:33:36.266449+0000","last_unstale":"2024-09-21T03:33:42.921523+0000","last_undegraded":"2024-09-21T03:33:42.921523+0000","last_fullsized":"2024-09-21T03:33:42.921523+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:39:22.951079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.286642+0000","last_change":"2024-09-21T03:33:40.293654+0000","last_active":"2024-09-21T03:33:41.286642+0000","last_peered":"2024-09-21T03:33:41.286642+0000","last_clean":"2024-09-21T03:33:41.286642+0000","last_became_active":"2024-09-21T03:33:40.293236+0000","last_became_peered":"2024-09-21T03:33:40.293236+0000","last_unstale":"2024-09-21T03:33:41.286642+0000","last_undegraded":"2024-09-21T03:33:41.286642+0000","last_fullsized":"2024-09-21T03:33:41.286642+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_clean_scrub_stamp":"2024-09-21T03:33:39.262231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:20:09.214266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,5,7],"acting":[2,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.920921+0000","last_change":"2024-09-21T03:33:36.252854+0000","last_active":"2024-09-21T03:33:42.920921+0000","last_peered":"2024-09-21T03:33:42.920921+0000","last_clean":"2024-09-21T03:33:42.920921+0000","last_became_active":"2024-09-21T03:33:36.252384+0000","last_became_peered":"2024-09-21T03:33:36.252384+0000","last_unstale":"2024-09-21T03:33:42.920921+0000","last_undegraded":"2024-09-21T03:33:42.920921+0000","last_fullsized":"2024-09-21T03:33:42.920921+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:35:32.535895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.921139+0000","last_change":"2024-09-21T03:33:35.252643+0000","last_active":"2024-09-21T03:33:42.921139+0000","last_peered":"2024-09-21T03:33:42.921139+0000","last_clean":"2024-09-21T03:33:42.921139+0000","last_became_active":"2024-09-21T03:33:35.252335+0000","last_became_peered":"2024-09-21T03:33:35.252335+0000","last_unstale":"2024-09-21T03:33:42.921139+0000","last_undegraded":"2024-09-21T03:33:42.921139+0000","last_fullsized":"2024-09-21T03:33:42.921139+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_clean_scrub_stamp":"2024-09-21T03:33:34.220881+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:07:29.037511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.921479+0000","last_change":"2024-09-21T03:33:40.307347+0000","last_active":"2024-09-21T03:33:42.921479+0000","last_peered":"2024-09-21T03:33:42.921479+0000","last_clean":"2024-09-21T03:33:42.921479+0000","last_became_active":"2024-09-21T03:33:40.306899+0000","last_became_peered":"2024-09-21T03:33:40.306899+0000","last_unstale":"2024-09-21T03:33:42.921479+0000","last_undegraded":"2024-09-21T03:33:42.921479+0000","last_fullsized":"2024-09-21T03:33:42.921479+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_clean_scrub_stamp":"2024-09-21T03:33:39.262231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:45:45.403094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,5],"acting":[3,1,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.754733+0000","last_change":"2024-09-21T03:33:36.257562+0000","last_active":"2024-09-21T03:33:41.754733+0000","last_peered":"2024-09-21T03:33:41.754733+0000","last_clean":"2024-09-21T03:33:41.754733+0000","last_became_active":"2024-09-21T03:33:36.255065+0000","last_became_peered":"2024-09-21T03:33:36.255065+0000","last_unstale":"2024-09-21T03:33:41.754733+0000","last_undegraded":"2024-09-21T03:33:41.754733+0000","last_fullsized":"2024-09-21T03:33:41.754733+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:35:07.883056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.284335+0000","last_change":"2024-09-21T03:33:35.254523+0000","last_active":"2024-09-21T03:33:41.284335+0000","last_peered":"2024-09-21T03:33:41.284335+0000","last_clean":"2024-09-21T03:33:41.284335+0000","last_became_active":"2024-09-21T03:33:35.254185+0000","last_became_peered":"2024-09-21T03:33:35.254185+0000","last_unstale":"2024-09-21T03:33:41.284335+0000","last_undegraded":"2024-09-21T03:33:41.284335+0000","last_fullsized":"2024-09-21T03:33:41.284335+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_clean_scrub_stamp":"2024-09-21T03:33:34.220881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:06:37.946009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.298659+0000","last_change":"2024-09-21T03:33:40.304238+0000","last_active":"2024-09-21T03:33:41.298659+0000","last_peered":"2024-09-21T03:33:41.298659+0000","last_clean":"2024-09-21T03:33:41.298659+0000","last_became_active":"2024-09-21T03:33:40.303785+0000","last_became_peered":"2024-09-21T03:33:40.303785+0000","last_unstale":"2024-09-21T03:33:41.298659+0000","last_undegraded":"2024-09-21T03:33:41.298659+0000","last_fullsized":"2024-09-21T03:33:41.298659+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_clean_scrub_stamp":"2024-09-21T03:33:39.262231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:28:07.313548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6,1],"acting":[0,4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.283514+0000","last_change":"2024-09-21T03:33:36.255712+0000","last_active":"2024-09-21T03:33:41.283514+0000","last_peered":"2024-09-21T03:33:41.283514+0000","last_clean":"2024-09-21T03:33:41.283514+0000","last_became_active":"2024-09-21T03:33:36.255386+0000","last_became_peered":"2024-09-21T03:33:36.255386+0000","last_unstale":"2024-09-21T03:33:41.283514+0000","last_undegraded":"2024-09-21T03:33:41.283514+0000","last_fullsized":"2024-09-21T03:33:41.283514+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:49:00.717844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.284629+0000","last_change":"2024-09-21T03:33:35.266345+0000","last_active":"2024-09-21T03:33:41.284629+0000","last_peered":"2024-09-21T03:33:41.284629+0000","last_clean":"2024-09-21T03:33:41.284629+0000","last_became_active":"2024-09-21T03:33:35.266059+0000","last_became_peered":"2024-09-21T03:33:35.266059+0000","last_unstale":"2024-09-21T03:33:41.284629+0000","last_undegraded":"2024-09-21T03:33:41.284629+0000","last_fullsized":"2024-09-21T03:33:41.284629+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_clean_scrub_stamp":"2024-09-21T03:33:34.220881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:54:40.741741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.755284+0000","last_change":"2024-09-21T03:33:40.303927+0000","last_active":"2024-09-21T03:33:41.755284+0000","last_peered":"2024-09-21T03:33:41.755284+0000","last_clean":"2024-09-21T03:33:41.755284+0000","last_became_active":"2024-09-21T03:33:40.303511+0000","last_became_peered":"2024-09-21T03:33:40.303511+0000","last_unstale":"2024-09-21T03:33:41.755284+0000","last_undegraded":"2024-09-21T03:33:41.755284+0000","last_fullsized":"2024-09-21T03:33:41.755284+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_clean_scrub_stamp":"2024-09-21T03:33:39.262231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:27:26.725520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2,3],"acting":[7,4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.920506+0000","last_change":"2024-09-21T03:33:36.249470+0000","last_active":"2024-09-21T03:33:42.920506+0000","last_peered":"2024-09-21T03:33:42.920506+0000","last_clean":"2024-09-21T03:33:42.920506+0000","last_became_active":"2024-09-21T03:33:36.248726+0000","last_became_peered":"2024-09-21T03:33:36.248726+0000","last_unstale":"2024-09-21T03:33:42.920506+0000","last_undegraded":"2024-09-21T03:33:42.920506+0000","last_fullsized":"2024-09-21T03:33:42.920506+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:56:46.691011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.297935+0000","last_change":"2024-09-21T03:33:35.260543+0000","last_active":"2024-09-21T03:33:41.297935+0000","last_peered":"2024-09-21T03:33:41.297935+0000","last_clean":"2024-09-21T03:33:41.297935+0000","last_became_active":"2024-09-21T03:33:35.260041+0000","last_became_peered":"2024-09-21T03:33:35.260041+0000","last_unstale":"2024-09-21T03:33:41.297935+0000","last_undegraded":"2024-09-21T03:33:41.297935+0000","last_fullsized":"2024-09-21T03:33:41.297935+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_clean_scrub_stamp":"2024-09-21T03:33:34.220881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:59:06.265387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.286314+0000","last_change":"2024-09-21T03:33:40.309460+0000","last_active":"2024-09-21T03:33:41.286314+0000","last_peered":"2024-09-21T03:33:41.286314+0000","last_clean":"2024-09-21T03:33:41.286314+0000","last_became_active":"2024-09-21T03:33:40.306652+0000","last_became_peered":"2024-09-21T03:33:40.306652+0000","last_unstale":"2024-09-21T03:33:41.286314+0000","last_undegraded":"2024-09-21T03:33:41.286314+0000","last_fullsized":"2024-09-21T03:33:41.286314+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_clean_scrub_stamp":"2024-09-21T03:33:39.262231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:43:35.273085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,7],"acting":[2,1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.284755+0000","last_change":"2024-09-21T03:33:36.256025+0000","last_active":"2024-09-21T03:33:41.284755+0000","last_peered":"2024-09-21T03:33:41.284755+0000","last_clean":"2024-09-21T03:33:41.284755+0000","last_became_active":"2024-09-21T03:33:36.255648+0000","last_became_peered":"2024-09-21T03:33:36.255648+0000","last_unstale":"2024-09-21T03:33:41.284755+0000","last_undegraded":"2024-09-21T03:33:41.284755+0000","last_fullsized":"2024-09-21T03:33:41.284755+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:48:33.535308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.921585+0000","last_change":"2024-09-21T03:33:35.264889+0000","last_active":"2024-09-21T03:33:42.921585+0000","last_peered":"2024-09-21T03:33:42.921585+0000","last_clean":"2024-09-21T03:33:42.921585+0000","last_became_active":"2024-09-21T03:33:35.264160+0000","last_became_peered":"2024-09-21T03:33:35.264160+0000","last_unstale":"2024-09-21T03:33:42.921585+0000","last_undegraded":"2024-09-21T03:33:42.921585+0000","last_fullsized":"2024-09-21T03:33:42.921585+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_clean_scrub_stamp":"2024-09-21T03:33:34.220881+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:48:59.712004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.286424+0000","last_change":"2024-09-21T03:33:40.304067+0000","last_active":"2024-09-21T03:33:41.286424+0000","last_peered":"2024-09-21T03:33:41.286424+0000","last_clean":"2024-09-21T03:33:41.286424+0000","last_became_active":"2024-09-21T03:33:40.303613+0000","last_became_peered":"2024-09-21T03:33:40.303613+0000","last_unstale":"2024-09-21T03:33:41.286424+0000","last_undegraded":"2024-09-21T03:33:41.286424+0000","last_fullsized":"2024-09-21T03:33:41.286424+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_clean_scrub_stamp":"2024-09-21T03:33:39.262231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:51:21.940594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,3,1],"acting":[2,6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.285607+0000","last_change":"2024-09-21T03:33:36.258150+0000","last_active":"2024-09-21T03:33:41.285607+0000","last_peered":"2024-09-21T03:33:41.285607+0000","last_clean":"2024-09-21T03:33:41.285607+0000","last_became_active":"2024-09-21T03:33:36.257655+0000","last_became_peered":"2024-09-21T03:33:36.257655+0000","last_unstale":"2024-09-21T03:33:41.285607+0000","last_undegraded":"2024-09-21T03:33:41.285607+0000","last_fullsized":"2024-09-21T03:33:41.285607+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:10:32.046638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"22'1","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.285234+0000","last_change":"2024-09-21T03:33:35.264750+0000","last_active":"2024-09-21T03:33:41.285234+0000","last_peered":"2024-09-21T03:33:41.285234+0000","last_clean":"2024-09-21T03:33:41.285234+0000","last_became_active":"2024-09-21T03:33:35.264380+0000","last_became_peered":"2024-09-21T03:33:35.264380+0000","last_unstale":"2024-09-21T03:33:41.285234+0000","last_undegraded":"2024-09-21T03:33:41.285234+0000","last_fullsized":"2024-09-21T03:33:41.285234+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_clean_scrub_stamp":"2024-09-21T03:33:34.220881+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:12:06.105531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.284754+0000","last_change":"2024-09-21T03:33:40.303103+0000","last_active":"2024-09-21T03:33:41.284754+0000","last_peered":"2024-09-21T03:33:41.284754+0000","last_clean":"2024-09-21T03:33:41.284754+0000","last_became_active":"2024-09-21T03:33:40.301739+0000","last_became_peered":"2024-09-21T03:33:40.301739+0000","last_unstale":"2024-09-21T03:33:41.284754+0000","last_undegraded":"2024-09-21T03:33:41.284754+0000","last_fullsized":"2024-09-21T03:33:41.284754+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_clean_scrub_stamp":"2024-09-21T03:33:39.262231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:00:58.389938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6,3],"acting":[4,1,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.285678+0000","last_change":"2024-09-21T03:33:36.268661+0000","last_active":"2024-09-21T03:33:41.285678+0000","last_peered":"2024-09-21T03:33:41.285678+0000","last_clean":"2024-09-21T03:33:41.285678+0000","last_became_active":"2024-09-21T03:33:36.268317+0000","last_became_peered":"2024-09-21T03:33:36.268317+0000","last_unstale":"2024-09-21T03:33:41.285678+0000","last_undegraded":"2024-09-21T03:33:41.285678+0000","last_fullsized":"2024-09-21T03:33:41.285678+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:19:27.002524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.921773+0000","last_change":"2024-09-21T03:33:35.250205+0000","last_active":"2024-09-21T03:33:42.921773+0000","last_peered":"2024-09-21T03:33:42.921773+0000","last_clean":"2024-09-21T03:33:42.921773+0000","last_became_active":"2024-09-21T03:33:35.249883+0000","last_became_peered":"2024-09-21T03:33:35.249883+0000","last_unstale":"2024-09-21T03:33:42.921773+0000","last_undegraded":"2024-09-21T03:33:42.921773+0000","last_fullsized":"2024-09-21T03:33:42.921773+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_clean_scrub_stamp":"2024-09-21T03:33:34.220881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:48:54.078693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.283974+0000","last_change":"2024-09-21T03:33:40.298889+0000","last_active":"2024-09-21T03:33:41.283974+0000","last_peered":"2024-09-21T03:33:41.283974+0000","last_clean":"2024-09-21T03:33:41.283974+0000","last_became_active":"2024-09-21T03:33:40.298435+0000","last_became_peered":"2024-09-21T03:33:40.298435+0000","last_unstale":"2024-09-21T03:33:41.283974+0000","last_undegraded":"2024-09-21T03:33:41.283974+0000","last_fullsized":"2024-09-21T03:33:41.283974+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_clean_scrub_stamp":"2024-09-21T03:33:39.262231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:29:46.742857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,2,7],"acting":[6,0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.286003+0000","last_change":"2024-09-21T03:33:36.258037+0000","last_active":"2024-09-21T03:33:41.286003+0000","last_peered":"2024-09-21T03:33:41.286003+0000","last_clean":"2024-09-21T03:33:41.286003+0000","last_became_active":"2024-09-21T03:33:36.257687+0000","last_became_peered":"2024-09-21T03:33:36.257687+0000","last_unstale":"2024-09-21T03:33:41.286003+0000","last_undegraded":"2024-09-21T03:33:41.286003+0000","last_fullsized":"2024-09-21T03:33:41.286003+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:12:07.111371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.920392+0000","last_change":"2024-09-21T03:33:35.254341+0000","last_active":"2024-09-21T03:33:42.920392+0000","last_peered":"2024-09-21T03:33:42.920392+0000","last_clean":"2024-09-21T03:33:42.920392+0000","last_became_active":"2024-09-21T03:33:35.254031+0000","last_became_peered":"2024-09-21T03:33:35.254031+0000","last_unstale":"2024-09-21T03:33:42.920392+0000","last_undegraded":"2024-09-21T03:33:42.920392+0000","last_fullsized":"2024-09-21T03:33:42.920392+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_clean_scrub_stamp":"2024-09-21T03:33:34.220881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:39:48.189397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.284028+0000","last_change":"2024-09-21T03:33:40.295123+0000","last_active":"2024-09-21T03:33:41.284028+0000","last_peered":"2024-09-21T03:33:41.284028+0000","last_clean":"2024-09-21T03:33:41.284028+0000","last_became_active":"2024-09-21T03:33:40.294672+0000","last_became_peered":"2024-09-21T03:33:40.294672+0000","last_unstale":"2024-09-21T03:33:41.284028+0000","last_undegraded":"2024-09-21T03:33:41.284028+0000","last_fullsized":"2024-09-21T03:33:41.284028+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_clean_scrub_stamp":"2024-09-21T03:33:39.262231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:43:35.273085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5,7],"acting":[6,3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.298299+0000","last_change":"2024-09-21T03:33:36.277694+0000","last_active":"2024-09-21T03:33:41.298299+0000","last_peered":"2024-09-21T03:33:41.298299+0000","last_clean":"2024-09-21T03:33:41.298299+0000","last_became_active":"2024-09-21T03:33:36.277375+0000","last_became_peered":"2024-09-21T03:33:36.277375+0000","last_unstale":"2024-09-21T03:33:41.298299+0000","last_undegraded":"2024-09-21T03:33:41.298299+0000","last_fullsized":"2024-09-21T03:33:41.298299+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:39:38.842518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.921283+0000","last_change":"2024-09-21T03:33:35.250423+0000","last_active":"2024-09-21T03:33:42.921283+0000","last_peered":"2024-09-21T03:33:42.921283+0000","last_clean":"2024-09-21T03:33:42.921283+0000","last_became_active":"2024-09-21T03:33:35.248426+0000","last_became_peered":"2024-09-21T03:33:35.248426+0000","last_unstale":"2024-09-21T03:33:42.921283+0000","last_undegraded":"2024-09-21T03:33:42.921283+0000","last_fullsized":"2024-09-21T03:33:42.921283+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_clean_scrub_stamp":"2024-09-21T03:33:34.220881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:56:24.872394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.285593+0000","last_change":"2024-09-21T03:33:40.302877+0000","last_active":"2024-09-21T03:33:41.285593+0000","last_peered":"2024-09-21T03:33:41.285593+0000","last_clean":"2024-09-21T03:33:41.285593+0000","last_became_active":"2024-09-21T03:33:40.301015+0000","last_became_peered":"2024-09-21T03:33:40.301015+0000","last_unstale":"2024-09-21T03:33:41.285593+0000","last_undegraded":"2024-09-21T03:33:41.285593+0000","last_fullsized":"2024-09-21T03:33:41.285593+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_clean_scrub_stamp":"2024-09-21T03:33:39.262231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:20:09.214266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0,3],"acting":[6,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.919937+0000","last_change":"2024-09-21T03:33:36.255208+0000","last_active":"2024-09-21T03:33:42.919937+0000","last_peered":"2024-09-21T03:33:42.919937+0000","last_clean":"2024-09-21T03:33:42.919937+0000","last_became_active":"2024-09-21T03:33:36.254340+0000","last_became_peered":"2024-09-21T03:33:36.254340+0000","last_unstale":"2024-09-21T03:33:42.919937+0000","last_undegraded":"2024-09-21T03:33:42.919937+0000","last_fullsized":"2024-09-21T03:33:42.919937+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:59:34.251362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.754771+0000","last_change":"2024-09-21T03:33:35.255282+0000","last_active":"2024-09-21T03:33:41.754771+0000","last_peered":"2024-09-21T03:33:41.754771+0000","last_clean":"2024-09-21T03:33:41.754771+0000","last_became_active":"2024-09-21T03:33:35.254934+0000","last_became_peered":"2024-09-21T03:33:35.254934+0000","last_unstale":"2024-09-21T03:33:41.754771+0000","last_undegraded":"2024-09-21T03:33:41.754771+0000","last_fullsized":"2024-09-21T03:33:41.754771+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_clean_scrub_stamp":"2024-09-21T03:33:34.220881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:08:00.579605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.285752+0000","last_change":"2024-09-21T03:33:40.305801+0000","last_active":"2024-09-21T03:33:41.285752+0000","last_peered":"2024-09-21T03:33:41.285752+0000","last_clean":"2024-09-21T03:33:41.285752+0000","last_became_active":"2024-09-21T03:33:40.305316+0000","last_became_peered":"2024-09-21T03:33:40.305316+0000","last_unstale":"2024-09-21T03:33:41.285752+0000","last_undegraded":"2024-09-21T03:33:41.285752+0000","last_fullsized":"2024-09-21T03:33:41.285752+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_clean_scrub_stamp":"2024-09-21T03:33:39.262231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:10:29.758718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,7],"acting":[6,0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.284256+0000","last_change":"2024-09-21T03:33:36.264028+0000","last_active":"2024-09-21T03:33:41.284256+0000","last_peered":"2024-09-21T03:33:41.284256+0000","last_clean":"2024-09-21T03:33:41.284256+0000","last_became_active":"2024-09-21T03:33:36.263725+0000","last_became_peered":"2024-09-21T03:33:36.263725+0000","last_unstale":"2024-09-21T03:33:41.284256+0000","last_undegraded":"2024-09-21T03:33:41.284256+0000","last_fullsized":"2024-09-21T03:33:41.284256+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:48:55.084533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.920078+0000","last_change":"2024-09-21T03:33:35.260963+0000","last_active":"2024-09-21T03:33:42.920078+0000","last_peered":"2024-09-21T03:33:42.920078+0000","last_clean":"2024-09-21T03:33:42.920078+0000","last_became_active":"2024-09-21T03:33:35.257593+0000","last_became_peered":"2024-09-21T03:33:35.257593+0000","last_unstale":"2024-09-21T03:33:42.920078+0000","last_undegraded":"2024-09-21T03:33:42.920078+0000","last_fullsized":"2024-09-21T03:33:42.920078+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_clean_scrub_stamp":"2024-09-21T03:33:34.220881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:35:18.509895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.920658+0000","last_change":"2024-09-21T03:33:40.298927+0000","last_active":"2024-09-21T03:33:42.920658+0000","last_peered":"2024-09-21T03:33:42.920658+0000","last_clean":"2024-09-21T03:33:42.920658+0000","last_became_active":"2024-09-21T03:33:40.298272+0000","last_became_peered":"2024-09-21T03:33:40.298272+0000","last_unstale":"2024-09-21T03:33:42.920658+0000","last_undegraded":"2024-09-21T03:33:42.920658+0000","last_fullsized":"2024-09-21T03:33:42.920658+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_clean_scrub_stamp":"2024-09-21T03:33:39.262231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:44:52.684204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6,1],"acting":[5,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.284686+0000","last_change":"2024-09-21T03:33:36.263187+0000","last_active":"2024-09-21T03:33:41.284686+0000","last_peered":"2024-09-21T03:33:41.284686+0000","last_clean":"2024-09-21T03:33:41.284686+0000","last_became_active":"2024-09-21T03:33:36.262852+0000","last_became_peered":"2024-09-21T03:33:36.262852+0000","last_unstale":"2024-09-21T03:33:41.284686+0000","last_undegraded":"2024-09-21T03:33:41.284686+0000","last_fullsized":"2024-09-21T03:33:41.284686+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:28:30.977653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.921266+0000","last_change":"2024-09-21T03:33:35.249580+0000","last_active":"2024-09-21T03:33:42.921266+0000","last_peered":"2024-09-21T03:33:42.921266+0000","last_clean":"2024-09-21T03:33:42.921266+0000","last_became_active":"2024-09-21T03:33:35.248574+0000","last_became_peered":"2024-09-21T03:33:35.248574+0000","last_unstale":"2024-09-21T03:33:42.921266+0000","last_undegraded":"2024-09-21T03:33:42.921266+0000","last_fullsized":"2024-09-21T03:33:42.921266+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_clean_scrub_stamp":"2024-09-21T03:33:34.220881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:19:25.996685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":119,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.919871+0000","last_change":"2024-09-21T03:33:25.319776+0000","last_active":"2024-09-21T03:33:42.919871+0000","last_peered":"2024-09-21T03:33:42.919871+0000","last_clean":"2024-09-21T03:33:42.919871+0000","last_became_active":"2024-09-21T03:33:25.318399+0000","last_became_peered":"2024-09-21T03:33:25.318399+0000","last_unstale":"2024-09-21T03:33:42.919871+0000","last_undegraded":"2024-09-21T03:33:42.919871+0000","last_fullsized":"2024-09-21T03:33:42.919871+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:23.016672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:23.016672+0000","last_clean_scrub_stamp":"2024-09-21T03:33:23.016672+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:32:35.906114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.921733+0000","last_change":"2024-09-21T03:33:40.304303+0000","last_active":"2024-09-21T03:33:42.921733+0000","last_peered":"2024-09-21T03:33:42.921733+0000","last_clean":"2024-09-21T03:33:42.921733+0000","last_became_active":"2024-09-21T03:33:40.303880+0000","last_became_peered":"2024-09-21T03:33:40.303880+0000","last_unstale":"2024-09-21T03:33:42.921733+0000","last_undegraded":"2024-09-21T03:33:42.921733+0000","last_fullsized":"2024-09-21T03:33:42.921733+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_clean_scrub_stamp":"2024-09-21T03:33:39.262231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:22:59.076888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2,0],"acting":[3,1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.921586+0000","last_change":"2024-09-21T03:33:36.256735+0000","last_active":"2024-09-21T03:33:42.921586+0000","last_peered":"2024-09-21T03:33:42.921586+0000","last_clean":"2024-09-21T03:33:42.921586+0000","last_became_active":"2024-09-21T03:33:36.256296+0000","last_became_peered":"2024-09-21T03:33:36.256296+0000","last_unstale":"2024-09-21T03:33:42.921586+0000","last_undegraded":"2024-09-21T03:33:42.921586+0000","last_fullsized":"2024-09-21T03:33:42.921586+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:20:58.478604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.285704+0000","last_change":"2024-09-21T03:33:35.263469+0000","last_active":"2024-09-21T03:33:41.285704+0000","last_peered":"2024-09-21T03:33:41.285704+0000","last_clean":"2024-09-21T03:33:41.285704+0000","last_became_active":"2024-09-21T03:33:35.263119+0000","last_became_peered":"2024-09-21T03:33:35.263119+0000","last_unstale":"2024-09-21T03:33:41.285704+0000","last_undegraded":"2024-09-21T03:33:41.285704+0000","last_fullsized":"2024-09-21T03:33:41.285704+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_clean_scrub_stamp":"2024-09-21T03:33:34.220881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:54:40.741741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.286176+0000","last_change":"2024-09-21T03:33:40.304199+0000","last_active":"2024-09-21T03:33:41.286176+0000","last_peered":"2024-09-21T03:33:41.286176+0000","last_clean":"2024-09-21T03:33:41.286176+0000","last_became_active":"2024-09-21T03:33:40.303770+0000","last_became_peered":"2024-09-21T03:33:40.303770+0000","last_unstale":"2024-09-21T03:33:41.286176+0000","last_undegraded":"2024-09-21T03:33:41.286176+0000","last_fullsized":"2024-09-21T03:33:41.286176+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_clean_scrub_stamp":"2024-09-21T03:33:39.262231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:41:52.568236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7,4],"acting":[2,3,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.298815+0000","last_change":"2024-09-21T03:33:36.269892+0000","last_active":"2024-09-21T03:33:41.298815+0000","last_peered":"2024-09-21T03:33:41.298815+0000","last_clean":"2024-09-21T03:33:41.298815+0000","last_became_active":"2024-09-21T03:33:36.269521+0000","last_became_peered":"2024-09-21T03:33:36.269521+0000","last_unstale":"2024-09-21T03:33:41.298815+0000","last_undegraded":"2024-09-21T03:33:41.298815+0000","last_fullsized":"2024-09-21T03:33:41.298815+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:43:04.668863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.920621+0000","last_change":"2024-09-21T03:33:35.256681+0000","last_active":"2024-09-21T03:33:42.920621+0000","last_peered":"2024-09-21T03:33:42.920621+0000","last_clean":"2024-09-21T03:33:42.920621+0000","last_became_active":"2024-09-21T03:33:35.256325+0000","last_became_peered":"2024-09-21T03:33:35.256325+0000","last_unstale":"2024-09-21T03:33:42.920621+0000","last_undegraded":"2024-09-21T03:33:42.920621+0000","last_fullsized":"2024-09-21T03:33:42.920621+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_clean_scrub_stamp":"2024-09-21T03:33:34.220881+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:49:36.514638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.920396+0000","last_change":"2024-09-21T03:33:40.298797+0000","last_active":"2024-09-21T03:33:42.920396+0000","last_peered":"2024-09-21T03:33:42.920396+0000","last_clean":"2024-09-21T03:33:42.920396+0000","last_became_active":"2024-09-21T03:33:40.298254+0000","last_became_peered":"2024-09-21T03:33:40.298254+0000","last_unstale":"2024-09-21T03:33:42.920396+0000","last_undegraded":"2024-09-21T03:33:42.920396+0000","last_fullsized":"2024-09-21T03:33:42.920396+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_clean_scrub_stamp":"2024-09-21T03:33:39.262231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:33:49.804176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1,3],"acting":[5,4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.299252+0000","last_change":"2024-09-21T03:33:36.274252+0000","last_active":"2024-09-21T03:33:41.299252+0000","last_peered":"2024-09-21T03:33:41.299252+0000","last_clean":"2024-09-21T03:33:41.299252+0000","last_became_active":"2024-09-21T03:33:36.273890+0000","last_became_peered":"2024-09-21T03:33:36.273890+0000","last_unstale":"2024-09-21T03:33:41.299252+0000","last_undegraded":"2024-09-21T03:33:41.299252+0000","last_fullsized":"2024-09-21T03:33:41.299252+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:55:43.689119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.284896+0000","last_change":"2024-09-21T03:33:35.255320+0000","last_active":"2024-09-21T03:33:41.284896+0000","last_peered":"2024-09-21T03:33:41.284896+0000","last_clean":"2024-09-21T03:33:41.284896+0000","last_became_active":"2024-09-21T03:33:35.254912+0000","last_became_peered":"2024-09-21T03:33:35.254912+0000","last_unstale":"2024-09-21T03:33:41.284896+0000","last_undegraded":"2024-09-21T03:33:41.284896+0000","last_fullsized":"2024-09-21T03:33:41.284896+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_clean_scrub_stamp":"2024-09-21T03:33:34.220881+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:55:02.392036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.919867+0000","last_change":"2024-09-21T03:33:40.304053+0000","last_active":"2024-09-21T03:33:42.919867+0000","last_peered":"2024-09-21T03:33:42.919867+0000","last_clean":"2024-09-21T03:33:42.919867+0000","last_became_active":"2024-09-21T03:33:40.303606+0000","last_became_peered":"2024-09-21T03:33:40.303606+0000","last_unstale":"2024-09-21T03:33:42.919867+0000","last_undegraded":"2024-09-21T03:33:42.919867+0000","last_fullsized":"2024-09-21T03:33:42.919867+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_clean_scrub_stamp":"2024-09-21T03:33:39.262231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:43:29.231059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.286527+0000","last_change":"2024-09-21T03:33:36.270385+0000","last_active":"2024-09-21T03:33:41.286527+0000","last_peered":"2024-09-21T03:33:41.286527+0000","last_clean":"2024-09-21T03:33:41.286527+0000","last_became_active":"2024-09-21T03:33:36.267107+0000","last_became_peered":"2024-09-21T03:33:36.267107+0000","last_unstale":"2024-09-21T03:33:41.286527+0000","last_undegraded":"2024-09-21T03:33:41.286527+0000","last_fullsized":"2024-09-21T03:33:41.286527+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:48:55.084533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.299383+0000","last_change":"2024-09-21T03:33:35.263063+0000","last_active":"2024-09-21T03:33:41.299383+0000","last_peered":"2024-09-21T03:33:41.299383+0000","last_clean":"2024-09-21T03:33:41.299383+0000","last_became_active":"2024-09-21T03:33:35.262324+0000","last_became_peered":"2024-09-21T03:33:35.262324+0000","last_unstale":"2024-09-21T03:33:41.299383+0000","last_undegraded":"2024-09-21T03:33:41.299383+0000","last_fullsized":"2024-09-21T03:33:41.299383+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_clean_scrub_stamp":"2024-09-21T03:33:34.220881+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:35:18.509895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.920221+0000","last_change":"2024-09-21T03:33:40.298024+0000","last_active":"2024-09-21T03:33:42.920221+0000","last_peered":"2024-09-21T03:33:42.920221+0000","last_clean":"2024-09-21T03:33:42.920221+0000","last_became_active":"2024-09-21T03:33:40.297580+0000","last_became_peered":"2024-09-21T03:33:40.297580+0000","last_unstale":"2024-09-21T03:33:42.920221+0000","last_undegraded":"2024-09-21T03:33:42.920221+0000","last_fullsized":"2024-09-21T03:33:42.920221+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_clean_scrub_stamp":"2024-09-21T03:33:39.262231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:34:07.830134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,2],"acting":[1,5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.920389+0000","last_change":"2024-09-21T03:33:36.256567+0000","last_active":"2024-09-21T03:33:42.920389+0000","last_peered":"2024-09-21T03:33:42.920389+0000","last_clean":"2024-09-21T03:33:42.920389+0000","last_became_active":"2024-09-21T03:33:36.252598+0000","last_became_peered":"2024-09-21T03:33:36.252598+0000","last_unstale":"2024-09-21T03:33:42.920389+0000","last_undegraded":"2024-09-21T03:33:42.920389+0000","last_fullsized":"2024-09-21T03:33:42.920389+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:53:59.995385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":31,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.283961+0000","last_change":"2024-09-21T03:33:35.246577+0000","last_active":"2024-09-21T03:33:41.283961+0000","last_peered":"2024-09-21T03:33:41.283961+0000","last_clean":"2024-09-21T03:33:41.283961+0000","last_became_active":"2024-09-21T03:33:35.245954+0000","last_became_peered":"2024-09-21T03:33:35.245954+0000","last_unstale":"2024-09-21T03:33:41.283961+0000","last_undegraded":"2024-09-21T03:33:41.283961+0000","last_fullsized":"2024-09-21T03:33:41.283961+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_clean_scrub_stamp":"2024-09-21T03:33:34.220881+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:59:06.265387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.920916+0000","last_change":"2024-09-21T03:33:40.299966+0000","last_active":"2024-09-21T03:33:42.920916+0000","last_peered":"2024-09-21T03:33:42.920916+0000","last_clean":"2024-09-21T03:33:42.920916+0000","last_became_active":"2024-09-21T03:33:40.298455+0000","last_became_peered":"2024-09-21T03:33:40.298455+0000","last_unstale":"2024-09-21T03:33:42.920916+0000","last_undegraded":"2024-09-21T03:33:42.920916+0000","last_fullsized":"2024-09-21T03:33:42.920916+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_clean_scrub_stamp":"2024-09-21T03:33:39.262231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:59:07.141809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1,6],"acting":[5,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.919715+0000","last_change":"2024-09-21T03:33:36.261762+0000","last_active":"2024-09-21T03:33:42.919715+0000","last_peered":"2024-09-21T03:33:42.919715+0000","last_clean":"2024-09-21T03:33:42.919715+0000","last_became_active":"2024-09-21T03:33:36.261379+0000","last_became_peered":"2024-09-21T03:33:36.261379+0000","last_unstale":"2024-09-21T03:33:42.919715+0000","last_undegraded":"2024-09-21T03:33:42.919715+0000","last_fullsized":"2024-09-21T03:33:42.919715+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:35:07.883056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.298133+0000","last_change":"2024-09-21T03:33:35.264258+0000","last_active":"2024-09-21T03:33:41.298133+0000","last_peered":"2024-09-21T03:33:41.298133+0000","last_clean":"2024-09-21T03:33:41.298133+0000","last_became_active":"2024-09-21T03:33:35.262742+0000","last_became_peered":"2024-09-21T03:33:35.262742+0000","last_unstale":"2024-09-21T03:33:41.298133+0000","last_undegraded":"2024-09-21T03:33:41.298133+0000","last_fullsized":"2024-09-21T03:33:41.298133+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_clean_scrub_stamp":"2024-09-21T03:33:34.220881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:42:59.930617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.920862+0000","last_change":"2024-09-21T03:33:40.298402+0000","last_active":"2024-09-21T03:33:42.920862+0000","last_peered":"2024-09-21T03:33:42.920862+0000","last_clean":"2024-09-21T03:33:42.920862+0000","last_became_active":"2024-09-21T03:33:40.297795+0000","last_became_peered":"2024-09-21T03:33:40.297795+0000","last_unstale":"2024-09-21T03:33:42.920862+0000","last_undegraded":"2024-09-21T03:33:42.920862+0000","last_fullsized":"2024-09-21T03:33:42.920862+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_clean_scrub_stamp":"2024-09-21T03:33:39.262231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:18:24.844623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6,4],"acting":[5,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.920969+0000","last_change":"2024-09-21T03:33:36.269233+0000","last_active":"2024-09-21T03:33:42.920969+0000","last_peered":"2024-09-21T03:33:42.920969+0000","last_clean":"2024-09-21T03:33:42.920969+0000","last_became_active":"2024-09-21T03:33:36.268680+0000","last_became_peered":"2024-09-21T03:33:36.268680+0000","last_unstale":"2024-09-21T03:33:42.920969+0000","last_undegraded":"2024-09-21T03:33:42.920969+0000","last_fullsized":"2024-09-21T03:33:42.920969+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:55:43.689119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.921162+0000","last_change":"2024-09-21T03:33:35.251566+0000","last_active":"2024-09-21T03:33:42.921162+0000","last_peered":"2024-09-21T03:33:42.921162+0000","last_clean":"2024-09-21T03:33:42.921162+0000","last_became_active":"2024-09-21T03:33:35.251252+0000","last_became_peered":"2024-09-21T03:33:35.251252+0000","last_unstale":"2024-09-21T03:33:42.921162+0000","last_undegraded":"2024-09-21T03:33:42.921162+0000","last_fullsized":"2024-09-21T03:33:42.921162+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_clean_scrub_stamp":"2024-09-21T03:33:34.220881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:39:01.410384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.285366+0000","last_change":"2024-09-21T03:33:40.300353+0000","last_active":"2024-09-21T03:33:41.285366+0000","last_peered":"2024-09-21T03:33:41.285366+0000","last_clean":"2024-09-21T03:33:41.285366+0000","last_became_active":"2024-09-21T03:33:40.299481+0000","last_became_peered":"2024-09-21T03:33:40.299481+0000","last_unstale":"2024-09-21T03:33:41.285366+0000","last_undegraded":"2024-09-21T03:33:41.285366+0000","last_fullsized":"2024-09-21T03:33:41.285366+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_clean_scrub_stamp":"2024-09-21T03:33:39.262231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:51:21.940594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0,7],"acting":[6,4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.284147+0000","last_change":"2024-09-21T03:33:36.265957+0000","last_active":"2024-09-21T03:33:41.284147+0000","last_peered":"2024-09-21T03:33:41.284147+0000","last_clean":"2024-09-21T03:33:41.284147+0000","last_became_active":"2024-09-21T03:33:36.265578+0000","last_became_peered":"2024-09-21T03:33:36.265578+0000","last_unstale":"2024-09-21T03:33:41.284147+0000","last_undegraded":"2024-09-21T03:33:41.284147+0000","last_fullsized":"2024-09-21T03:33:41.284147+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:06:36.428086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.754862+0000","last_change":"2024-09-21T03:33:35.263298+0000","last_active":"2024-09-21T03:33:41.754862+0000","last_peered":"2024-09-21T03:33:41.754862+0000","last_clean":"2024-09-21T03:33:41.754862+0000","last_became_active":"2024-09-21T03:33:35.262833+0000","last_became_peered":"2024-09-21T03:33:35.262833+0000","last_unstale":"2024-09-21T03:33:41.754862+0000","last_undegraded":"2024-09-21T03:33:41.754862+0000","last_fullsized":"2024-09-21T03:33:41.754862+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_clean_scrub_stamp":"2024-09-21T03:33:34.220881+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:36:04.049350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.921613+0000","last_change":"2024-09-21T03:33:40.299399+0000","last_active":"2024-09-21T03:33:42.921613+0000","last_peered":"2024-09-21T03:33:42.921613+0000","last_clean":"2024-09-21T03:33:42.921613+0000","last_became_active":"2024-09-21T03:33:40.298855+0000","last_became_peered":"2024-09-21T03:33:40.298855+0000","last_unstale":"2024-09-21T03:33:42.921613+0000","last_undegraded":"2024-09-21T03:33:42.921613+0000","last_fullsized":"2024-09-21T03:33:42.921613+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_clean_scrub_stamp":"2024-09-21T03:33:39.262231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:38:17.611449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5,6],"acting":[3,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.298648+0000","last_change":"2024-09-21T03:33:36.268494+0000","last_active":"2024-09-21T03:33:41.298648+0000","last_peered":"2024-09-21T03:33:41.298648+0000","last_clean":"2024-09-21T03:33:41.298648+0000","last_became_active":"2024-09-21T03:33:36.268126+0000","last_became_peered":"2024-09-21T03:33:36.268126+0000","last_unstale":"2024-09-21T03:33:41.298648+0000","last_undegraded":"2024-09-21T03:33:41.298648+0000","last_fullsized":"2024-09-21T03:33:41.298648+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:00:54.354428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.14","version":"26'9","reported_seq":32,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:47.259231+0000","last_change":"2024-09-21T03:33:35.249689+0000","last_active":"2024-09-21T03:33:47.259231+0000","last_peered":"2024-09-21T03:33:47.259231+0000","last_clean":"2024-09-21T03:33:47.259231+0000","last_became_active":"2024-09-21T03:33:35.249355+0000","last_became_peered":"2024-09-21T03:33:35.249355+0000","last_unstale":"2024-09-21T03:33:47.259231+0000","last_undegraded":"2024-09-21T03:33:47.259231+0000","last_fullsized":"2024-09-21T03:33:47.259231+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_clean_scrub_stamp":"2024-09-21T03:33:34.220881+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:01:23.098006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.284651+0000","last_change":"2024-09-21T03:33:40.295609+0000","last_active":"2024-09-21T03:33:41.284651+0000","last_peered":"2024-09-21T03:33:41.284651+0000","last_clean":"2024-09-21T03:33:41.284651+0000","last_became_active":"2024-09-21T03:33:40.294963+0000","last_became_peered":"2024-09-21T03:33:40.294963+0000","last_unstale":"2024-09-21T03:33:41.284651+0000","last_undegraded":"2024-09-21T03:33:41.284651+0000","last_fullsized":"2024-09-21T03:33:41.284651+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_clean_scrub_stamp":"2024-09-21T03:33:39.262231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:55:47.724630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5,0],"acting":[4,3,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.921108+0000","last_change":"2024-09-21T03:33:36.260379+0000","last_active":"2024-09-21T03:33:42.921108+0000","last_peered":"2024-09-21T03:33:42.921108+0000","last_clean":"2024-09-21T03:33:42.921108+0000","last_became_active":"2024-09-21T03:33:36.260035+0000","last_became_peered":"2024-09-21T03:33:36.260035+0000","last_unstale":"2024-09-21T03:33:42.921108+0000","last_undegraded":"2024-09-21T03:33:42.921108+0000","last_fullsized":"2024-09-21T03:33:42.921108+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:28:03.278038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.755172+0000","last_change":"2024-09-21T03:33:35.249062+0000","last_active":"2024-09-21T03:33:41.755172+0000","last_peered":"2024-09-21T03:33:41.755172+0000","last_clean":"2024-09-21T03:33:41.755172+0000","last_became_active":"2024-09-21T03:33:35.248630+0000","last_became_peered":"2024-09-21T03:33:35.248630+0000","last_unstale":"2024-09-21T03:33:41.755172+0000","last_undegraded":"2024-09-21T03:33:41.755172+0000","last_fullsized":"2024-09-21T03:33:41.755172+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_clean_scrub_stamp":"2024-09-21T03:33:34.220881+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:49:09.367077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.286199+0000","last_change":"2024-09-21T03:33:40.305392+0000","last_active":"2024-09-21T03:33:41.286199+0000","last_peered":"2024-09-21T03:33:41.286199+0000","last_clean":"2024-09-21T03:33:41.286199+0000","last_became_active":"2024-09-21T03:33:40.304899+0000","last_became_peered":"2024-09-21T03:33:40.304899+0000","last_unstale":"2024-09-21T03:33:41.286199+0000","last_undegraded":"2024-09-21T03:33:41.286199+0000","last_fullsized":"2024-09-21T03:33:41.286199+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_clean_scrub_stamp":"2024-09-21T03:33:39.262231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:29:46.742857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1,7],"acting":[2,3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.920967+0000","last_change":"2024-09-21T03:33:36.256188+0000","last_active":"2024-09-21T03:33:42.920967+0000","last_peered":"2024-09-21T03:33:42.920967+0000","last_clean":"2024-09-21T03:33:42.920967+0000","last_became_active":"2024-09-21T03:33:36.254900+0000","last_became_peered":"2024-09-21T03:33:36.254900+0000","last_unstale":"2024-09-21T03:33:42.920967+0000","last_undegraded":"2024-09-21T03:33:42.920967+0000","last_fullsized":"2024-09-21T03:33:42.920967+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:20:54.467912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.299116+0000","last_change":"2024-09-21T03:33:35.259866+0000","last_active":"2024-09-21T03:33:41.299116+0000","last_peered":"2024-09-21T03:33:41.299116+0000","last_clean":"2024-09-21T03:33:41.299116+0000","last_became_active":"2024-09-21T03:33:35.259368+0000","last_became_peered":"2024-09-21T03:33:35.259368+0000","last_unstale":"2024-09-21T03:33:41.299116+0000","last_undegraded":"2024-09-21T03:33:41.299116+0000","last_fullsized":"2024-09-21T03:33:41.299116+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_clean_scrub_stamp":"2024-09-21T03:33:34.220881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:12:06.105531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.754662+0000","last_change":"2024-09-21T03:33:40.304856+0000","last_active":"2024-09-21T03:33:41.754662+0000","last_peered":"2024-09-21T03:33:41.754662+0000","last_clean":"2024-09-21T03:33:41.754662+0000","last_became_active":"2024-09-21T03:33:40.304421+0000","last_became_peered":"2024-09-21T03:33:40.304421+0000","last_unstale":"2024-09-21T03:33:41.754662+0000","last_undegraded":"2024-09-21T03:33:41.754662+0000","last_fullsized":"2024-09-21T03:33:41.754662+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_clean_scrub_stamp":"2024-09-21T03:33:39.262231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:10:13.916812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3,2],"acting":[7,1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.286504+0000","last_change":"2024-09-21T03:33:36.265331+0000","last_active":"2024-09-21T03:33:41.286504+0000","last_peered":"2024-09-21T03:33:41.286504+0000","last_clean":"2024-09-21T03:33:41.286504+0000","last_became_active":"2024-09-21T03:33:36.264928+0000","last_became_peered":"2024-09-21T03:33:36.264928+0000","last_unstale":"2024-09-21T03:33:41.286504+0000","last_undegraded":"2024-09-21T03:33:41.286504+0000","last_fullsized":"2024-09-21T03:33:41.286504+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:56:25.878234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.920745+0000","last_change":"2024-09-21T03:33:35.249377+0000","last_active":"2024-09-21T03:33:42.920745+0000","last_peered":"2024-09-21T03:33:42.920745+0000","last_clean":"2024-09-21T03:33:42.920745+0000","last_became_active":"2024-09-21T03:33:35.249027+0000","last_became_peered":"2024-09-21T03:33:35.249027+0000","last_unstale":"2024-09-21T03:33:42.920745+0000","last_undegraded":"2024-09-21T03:33:42.920745+0000","last_fullsized":"2024-09-21T03:33:42.920745+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_clean_scrub_stamp":"2024-09-21T03:33:34.220881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:48:36.003541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.284509+0000","last_change":"2024-09-21T03:33:40.294482+0000","last_active":"2024-09-21T03:33:41.284509+0000","last_peered":"2024-09-21T03:33:41.284509+0000","last_clean":"2024-09-21T03:33:41.284509+0000","last_became_active":"2024-09-21T03:33:40.293996+0000","last_became_peered":"2024-09-21T03:33:40.293996+0000","last_unstale":"2024-09-21T03:33:41.284509+0000","last_undegraded":"2024-09-21T03:33:41.284509+0000","last_fullsized":"2024-09-21T03:33:41.284509+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_clean_scrub_stamp":"2024-09-21T03:33:39.262231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:43:08.704373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,3,7],"acting":[4,0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.283820+0000","last_change":"2024-09-21T03:33:40.300111+0000","last_active":"2024-09-21T03:33:41.283820+0000","last_peered":"2024-09-21T03:33:41.283820+0000","last_clean":"2024-09-21T03:33:41.283820+0000","last_became_active":"2024-09-21T03:33:40.299645+0000","last_became_peered":"2024-09-21T03:33:40.299645+0000","last_unstale":"2024-09-21T03:33:41.283820+0000","last_undegraded":"2024-09-21T03:33:41.283820+0000","last_fullsized":"2024-09-21T03:33:41.283820+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_clean_scrub_stamp":"2024-09-21T03:33:39.262231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:10:29.758718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,5,3],"acting":[4,1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.284548+0000","last_change":"2024-09-21T03:33:36.248593+0000","last_active":"2024-09-21T03:33:41.284548+0000","last_peered":"2024-09-21T03:33:41.284548+0000","last_clean":"2024-09-21T03:33:41.284548+0000","last_became_active":"2024-09-21T03:33:36.248274+0000","last_became_peered":"2024-09-21T03:33:36.248274+0000","last_unstale":"2024-09-21T03:33:41.284548+0000","last_undegraded":"2024-09-21T03:33:41.284548+0000","last_fullsized":"2024-09-21T03:33:41.284548+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:36:57.787555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.920023+0000","last_change":"2024-09-21T03:33:35.261733+0000","last_active":"2024-09-21T03:33:42.920023+0000","last_peered":"2024-09-21T03:33:42.920023+0000","last_clean":"2024-09-21T03:33:42.920023+0000","last_became_active":"2024-09-21T03:33:35.261427+0000","last_became_peered":"2024-09-21T03:33:35.261427+0000","last_unstale":"2024-09-21T03:33:42.920023+0000","last_undegraded":"2024-09-21T03:33:42.920023+0000","last_fullsized":"2024-09-21T03:33:42.920023+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_clean_scrub_stamp":"2024-09-21T03:33:34.220881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:40:22.752363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.921462+0000","last_change":"2024-09-21T03:33:40.306313+0000","last_active":"2024-09-21T03:33:42.921462+0000","last_peered":"2024-09-21T03:33:42.921462+0000","last_clean":"2024-09-21T03:33:42.921462+0000","last_became_active":"2024-09-21T03:33:40.305650+0000","last_became_peered":"2024-09-21T03:33:40.305650+0000","last_unstale":"2024-09-21T03:33:42.921462+0000","last_undegraded":"2024-09-21T03:33:42.921462+0000","last_fullsized":"2024-09-21T03:33:42.921462+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_clean_scrub_stamp":"2024-09-21T03:33:39.262231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:45:58.941848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5,1],"acting":[3,2,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.920101+0000","last_change":"2024-09-21T03:33:36.255929+0000","last_active":"2024-09-21T03:33:42.920101+0000","last_peered":"2024-09-21T03:33:42.920101+0000","last_clean":"2024-09-21T03:33:42.920101+0000","last_became_active":"2024-09-21T03:33:36.255538+0000","last_became_peered":"2024-09-21T03:33:36.255538+0000","last_unstale":"2024-09-21T03:33:42.920101+0000","last_undegraded":"2024-09-21T03:33:42.920101+0000","last_fullsized":"2024-09-21T03:33:42.920101+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:33:10.543079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.284381+0000","last_change":"2024-09-21T03:33:35.264572+0000","last_active":"2024-09-21T03:33:41.284381+0000","last_peered":"2024-09-21T03:33:41.284381+0000","last_clean":"2024-09-21T03:33:41.284381+0000","last_became_active":"2024-09-21T03:33:35.264251+0000","last_became_peered":"2024-09-21T03:33:35.264251+0000","last_unstale":"2024-09-21T03:33:41.284381+0000","last_undegraded":"2024-09-21T03:33:41.284381+0000","last_fullsized":"2024-09-21T03:33:41.284381+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_clean_scrub_stamp":"2024-09-21T03:33:34.220881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:16:25.260763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.921171+0000","last_change":"2024-09-21T03:33:35.258225+0000","last_active":"2024-09-21T03:33:42.921171+0000","last_peered":"2024-09-21T03:33:42.921171+0000","last_clean":"2024-09-21T03:33:42.921171+0000","last_became_active":"2024-09-21T03:33:35.257903+0000","last_became_peered":"2024-09-21T03:33:35.257903+0000","last_unstale":"2024-09-21T03:33:42.921171+0000","last_undegraded":"2024-09-21T03:33:42.921171+0000","last_fullsized":"2024-09-21T03:33:42.921171+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_clean_scrub_stamp":"2024-09-21T03:33:34.220881+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:12:06.105531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.283808+0000","last_change":"2024-09-21T03:33:36.267650+0000","last_active":"2024-09-21T03:33:41.283808+0000","last_peered":"2024-09-21T03:33:41.283808+0000","last_clean":"2024-09-21T03:33:41.283808+0000","last_became_active":"2024-09-21T03:33:36.267315+0000","last_became_peered":"2024-09-21T03:33:36.267315+0000","last_unstale":"2024-09-21T03:33:41.283808+0000","last_undegraded":"2024-09-21T03:33:41.283808+0000","last_fullsized":"2024-09-21T03:33:41.283808+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:21:56.947372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.285062+0000","last_change":"2024-09-21T03:33:40.290894+0000","last_active":"2024-09-21T03:33:41.285062+0000","last_peered":"2024-09-21T03:33:41.285062+0000","last_clean":"2024-09-21T03:33:41.285062+0000","last_became_active":"2024-09-21T03:33:40.290424+0000","last_became_peered":"2024-09-21T03:33:40.290424+0000","last_unstale":"2024-09-21T03:33:41.285062+0000","last_undegraded":"2024-09-21T03:33:41.285062+0000","last_fullsized":"2024-09-21T03:33:41.285062+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_clean_scrub_stamp":"2024-09-21T03:33:39.262231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:56:50.726521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,7],"acting":[4,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.920464+0000","last_change":"2024-09-21T03:33:35.251276+0000","last_active":"2024-09-21T03:33:42.920464+0000","last_peered":"2024-09-21T03:33:42.920464+0000","last_clean":"2024-09-21T03:33:42.920464+0000","last_became_active":"2024-09-21T03:33:35.250758+0000","last_became_peered":"2024-09-21T03:33:35.250758+0000","last_unstale":"2024-09-21T03:33:42.920464+0000","last_undegraded":"2024-09-21T03:33:42.920464+0000","last_fullsized":"2024-09-21T03:33:42.920464+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_clean_scrub_stamp":"2024-09-21T03:33:34.220881+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:03:09.683999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.920393+0000","last_change":"2024-09-21T03:33:36.261907+0000","last_active":"2024-09-21T03:33:42.920393+0000","last_peered":"2024-09-21T03:33:42.920393+0000","last_clean":"2024-09-21T03:33:42.920393+0000","last_became_active":"2024-09-21T03:33:36.261483+0000","last_became_peered":"2024-09-21T03:33:36.261483+0000","last_unstale":"2024-09-21T03:33:42.920393+0000","last_undegraded":"2024-09-21T03:33:42.920393+0000","last_fullsized":"2024-09-21T03:33:42.920393+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:08:27.276982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.755140+0000","last_change":"2024-09-21T03:33:40.305034+0000","last_active":"2024-09-21T03:33:41.755140+0000","last_peered":"2024-09-21T03:33:41.755140+0000","last_clean":"2024-09-21T03:33:41.755140+0000","last_became_active":"2024-09-21T03:33:40.304555+0000","last_became_peered":"2024-09-21T03:33:40.304555+0000","last_unstale":"2024-09-21T03:33:41.755140+0000","last_undegraded":"2024-09-21T03:33:41.755140+0000","last_fullsized":"2024-09-21T03:33:41.755140+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_clean_scrub_stamp":"2024-09-21T03:33:39.262231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:09:41.615406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1,0],"acting":[7,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.285867+0000","last_change":"2024-09-21T03:33:36.258191+0000","last_active":"2024-09-21T03:33:41.285867+0000","last_peered":"2024-09-21T03:33:41.285867+0000","last_clean":"2024-09-21T03:33:41.285867+0000","last_became_active":"2024-09-21T03:33:36.257866+0000","last_became_peered":"2024-09-21T03:33:36.257866+0000","last_unstale":"2024-09-21T03:33:41.285867+0000","last_undegraded":"2024-09-21T03:33:41.285867+0000","last_fullsized":"2024-09-21T03:33:41.285867+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:07:30.043350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.754532+0000","last_change":"2024-09-21T03:33:36.248020+0000","last_active":"2024-09-21T03:33:41.754532+0000","last_peered":"2024-09-21T03:33:41.754532+0000","last_clean":"2024-09-21T03:33:41.754532+0000","last_became_active":"2024-09-21T03:33:36.247696+0000","last_became_peered":"2024-09-21T03:33:36.247696+0000","last_unstale":"2024-09-21T03:33:41.754532+0000","last_undegraded":"2024-09-21T03:33:41.754532+0000","last_fullsized":"2024-09-21T03:33:41.754532+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:11:14.232403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.920927+0000","last_change":"2024-09-21T03:33:36.264435+0000","last_active":"2024-09-21T03:33:42.920927+0000","last_peered":"2024-09-21T03:33:42.920927+0000","last_clean":"2024-09-21T03:33:42.920927+0000","last_became_active":"2024-09-21T03:33:36.263834+0000","last_became_peered":"2024-09-21T03:33:36.263834+0000","last_unstale":"2024-09-21T03:33:42.920927+0000","last_undegraded":"2024-09-21T03:33:42.920927+0000","last_fullsized":"2024-09-21T03:33:42.920927+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:00:54.354428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.920341+0000","last_change":"2024-09-21T03:33:36.256675+0000","last_active":"2024-09-21T03:33:42.920341+0000","last_peered":"2024-09-21T03:33:42.920341+0000","last_clean":"2024-09-21T03:33:42.920341+0000","last_became_active":"2024-09-21T03:33:36.256184+0000","last_became_peered":"2024-09-21T03:33:36.256184+0000","last_unstale":"2024-09-21T03:33:42.920341+0000","last_undegraded":"2024-09-21T03:33:42.920341+0000","last_fullsized":"2024-09-21T03:33:42.920341+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:10:44.754143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.285553+0000","last_change":"2024-09-21T03:33:36.271076+0000","last_active":"2024-09-21T03:33:41.285553+0000","last_peered":"2024-09-21T03:33:41.285553+0000","last_clean":"2024-09-21T03:33:41.285553+0000","last_became_active":"2024-09-21T03:33:36.270707+0000","last_became_peered":"2024-09-21T03:33:36.270707+0000","last_unstale":"2024-09-21T03:33:41.285553+0000","last_undegraded":"2024-09-21T03:33:41.285553+0000","last_fullsized":"2024-09-21T03:33:41.285553+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:07:30.043350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.285361+0000","last_change":"2024-09-21T03:33:36.261405+0000","last_active":"2024-09-21T03:33:41.285361+0000","last_peered":"2024-09-21T03:33:41.285361+0000","last_clean":"2024-09-21T03:33:41.285361+0000","last_became_active":"2024-09-21T03:33:36.261048+0000","last_became_peered":"2024-09-21T03:33:36.261048+0000","last_unstale":"2024-09-21T03:33:41.285361+0000","last_undegraded":"2024-09-21T03:33:41.285361+0000","last_fullsized":"2024-09-21T03:33:41.285361+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:56:25.878234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.286335+0000","last_change":"2024-09-21T03:33:36.272514+0000","last_active":"2024-09-21T03:33:41.286335+0000","last_peered":"2024-09-21T03:33:41.286335+0000","last_clean":"2024-09-21T03:33:41.286335+0000","last_became_active":"2024-09-21T03:33:36.272186+0000","last_became_peered":"2024-09-21T03:33:36.272186+0000","last_unstale":"2024-09-21T03:33:41.286335+0000","last_undegraded":"2024-09-21T03:33:41.286335+0000","last_fullsized":"2024-09-21T03:33:41.286335+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:39:02.416224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.920780+0000","last_change":"2024-09-21T03:33:36.255650+0000","last_active":"2024-09-21T03:33:42.920780+0000","last_peered":"2024-09-21T03:33:42.920780+0000","last_clean":"2024-09-21T03:33:42.920780+0000","last_became_active":"2024-09-21T03:33:36.253524+0000","last_became_peered":"2024-09-21T03:33:36.253524+0000","last_unstale":"2024-09-21T03:33:42.920780+0000","last_undegraded":"2024-09-21T03:33:42.920780+0000","last_fullsized":"2024-09-21T03:33:42.920780+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:51:37.424690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.919971+0000","last_change":"2024-09-21T03:33:36.262263+0000","last_active":"2024-09-21T03:33:42.919971+0000","last_peered":"2024-09-21T03:33:42.919971+0000","last_clean":"2024-09-21T03:33:42.919971+0000","last_became_active":"2024-09-21T03:33:36.261962+0000","last_became_peered":"2024-09-21T03:33:36.261962+0000","last_unstale":"2024-09-21T03:33:42.919971+0000","last_undegraded":"2024-09-21T03:33:42.919971+0000","last_fullsized":"2024-09-21T03:33:42.919971+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:10:25.723208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.920014+0000","last_change":"2024-09-21T03:33:36.261266+0000","last_active":"2024-09-21T03:33:42.920014+0000","last_peered":"2024-09-21T03:33:42.920014+0000","last_clean":"2024-09-21T03:33:42.920014+0000","last_became_active":"2024-09-21T03:33:36.260938+0000","last_became_peered":"2024-09-21T03:33:36.260938+0000","last_unstale":"2024-09-21T03:33:42.920014+0000","last_undegraded":"2024-09-21T03:33:42.920014+0000","last_fullsized":"2024-09-21T03:33:42.920014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:54:42.460406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.920279+0000","last_change":"2024-09-21T03:33:36.268419+0000","last_active":"2024-09-21T03:33:42.920279+0000","last_peered":"2024-09-21T03:33:42.920279+0000","last_clean":"2024-09-21T03:33:42.920279+0000","last_became_active":"2024-09-21T03:33:36.268118+0000","last_became_peered":"2024-09-21T03:33:36.268118+0000","last_unstale":"2024-09-21T03:33:42.920279+0000","last_undegraded":"2024-09-21T03:33:42.920279+0000","last_fullsized":"2024-09-21T03:33:42.920279+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:15:19.693603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.283681+0000","last_change":"2024-09-21T03:33:36.265448+0000","last_active":"2024-09-21T03:33:41.283681+0000","last_peered":"2024-09-21T03:33:41.283681+0000","last_clean":"2024-09-21T03:33:41.283681+0000","last_became_active":"2024-09-21T03:33:36.263265+0000","last_became_peered":"2024-09-21T03:33:36.263265+0000","last_unstale":"2024-09-21T03:33:41.283681+0000","last_undegraded":"2024-09-21T03:33:41.283681+0000","last_fullsized":"2024-09-21T03:33:41.283681+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:06:36.428086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.298767+0000","last_change":"2024-09-21T03:33:36.274784+0000","last_active":"2024-09-21T03:33:41.298767+0000","last_peered":"2024-09-21T03:33:41.298767+0000","last_clean":"2024-09-21T03:33:41.298767+0000","last_became_active":"2024-09-21T03:33:36.273608+0000","last_became_peered":"2024-09-21T03:33:36.273608+0000","last_unstale":"2024-09-21T03:33:41.298767+0000","last_undegraded":"2024-09-21T03:33:41.298767+0000","last_fullsized":"2024-09-21T03:33:41.298767+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:43:31.237575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.921685+0000","last_change":"2024-09-21T03:33:36.253771+0000","last_active":"2024-09-21T03:33:42.921685+0000","last_peered":"2024-09-21T03:33:42.921685+0000","last_clean":"2024-09-21T03:33:42.921685+0000","last_became_active":"2024-09-21T03:33:36.253068+0000","last_became_peered":"2024-09-21T03:33:36.253068+0000","last_unstale":"2024-09-21T03:33:42.921685+0000","last_undegraded":"2024-09-21T03:33:42.921685+0000","last_fullsized":"2024-09-21T03:33:42.921685+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:33:50.625863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.755254+0000","last_change":"2024-09-21T03:33:36.255996+0000","last_active":"2024-09-21T03:33:41.755254+0000","last_peered":"2024-09-21T03:33:41.755254+0000","last_clean":"2024-09-21T03:33:41.755254+0000","last_became_active":"2024-09-21T03:33:36.255696+0000","last_became_peered":"2024-09-21T03:33:36.255696+0000","last_unstale":"2024-09-21T03:33:41.755254+0000","last_undegraded":"2024-09-21T03:33:41.755254+0000","last_fullsized":"2024-09-21T03:33:41.755254+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:15:19.693603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.921552+0000","last_change":"2024-09-21T03:33:36.271593+0000","last_active":"2024-09-21T03:33:42.921552+0000","last_peered":"2024-09-21T03:33:42.921552+0000","last_clean":"2024-09-21T03:33:42.921552+0000","last_became_active":"2024-09-21T03:33:36.271244+0000","last_became_peered":"2024-09-21T03:33:36.271244+0000","last_unstale":"2024-09-21T03:33:42.921552+0000","last_undegraded":"2024-09-21T03:33:42.921552+0000","last_fullsized":"2024-09-21T03:33:42.921552+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:14:54.862806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.919854+0000","last_change":"2024-09-21T03:33:36.252117+0000","last_active":"2024-09-21T03:33:42.919854+0000","last_peered":"2024-09-21T03:33:42.919854+0000","last_clean":"2024-09-21T03:33:42.919854+0000","last_became_active":"2024-09-21T03:33:36.251757+0000","last_became_peered":"2024-09-21T03:33:36.251757+0000","last_unstale":"2024-09-21T03:33:42.919854+0000","last_undegraded":"2024-09-21T03:33:42.919854+0000","last_fullsized":"2024-09-21T03:33:42.919854+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:59:02.176819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.919914+0000","last_change":"2024-09-21T03:33:36.257727+0000","last_active":"2024-09-21T03:33:42.919914+0000","last_peered":"2024-09-21T03:33:42.919914+0000","last_clean":"2024-09-21T03:33:42.919914+0000","last_became_active":"2024-09-21T03:33:36.257423+0000","last_became_peered":"2024-09-21T03:33:36.257423+0000","last_unstale":"2024-09-21T03:33:42.919914+0000","last_undegraded":"2024-09-21T03:33:42.919914+0000","last_fullsized":"2024-09-21T03:33:42.919914+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:11:43.225156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.284433+0000","last_change":"2024-09-21T03:33:36.257080+0000","last_active":"2024-09-21T03:33:41.284433+0000","last_peered":"2024-09-21T03:33:41.284433+0000","last_clean":"2024-09-21T03:33:41.284433+0000","last_became_active":"2024-09-21T03:33:36.256763+0000","last_became_peered":"2024-09-21T03:33:36.256763+0000","last_unstale":"2024-09-21T03:33:41.284433+0000","last_undegraded":"2024-09-21T03:33:41.284433+0000","last_fullsized":"2024-09-21T03:33:41.284433+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:36:57.787555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.285409+0000","last_change":"2024-09-21T03:33:36.265166+0000","last_active":"2024-09-21T03:33:41.285409+0000","last_peered":"2024-09-21T03:33:41.285409+0000","last_clean":"2024-09-21T03:33:41.285409+0000","last_became_active":"2024-09-21T03:33:36.264841+0000","last_became_peered":"2024-09-21T03:33:36.264841+0000","last_unstale":"2024-09-21T03:33:41.285409+0000","last_undegraded":"2024-09-21T03:33:41.285409+0000","last_fullsized":"2024-09-21T03:33:41.285409+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:49:00.717844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.298659+0000","last_change":"2024-09-21T03:33:36.274919+0000","last_active":"2024-09-21T03:33:41.298659+0000","last_peered":"2024-09-21T03:33:41.298659+0000","last_clean":"2024-09-21T03:33:41.298659+0000","last_became_active":"2024-09-21T03:33:36.274111+0000","last_became_peered":"2024-09-21T03:33:36.274111+0000","last_unstale":"2024-09-21T03:33:41.298659+0000","last_undegraded":"2024-09-21T03:33:41.298659+0000","last_fullsized":"2024-09-21T03:33:41.298659+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:29:42.707347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.298611+0000","last_change":"2024-09-21T03:33:36.272954+0000","last_active":"2024-09-21T03:33:41.298611+0000","last_peered":"2024-09-21T03:33:41.298611+0000","last_clean":"2024-09-21T03:33:41.298611+0000","last_became_active":"2024-09-21T03:33:36.272551+0000","last_became_peered":"2024-09-21T03:33:36.272551+0000","last_unstale":"2024-09-21T03:33:41.298611+0000","last_undegraded":"2024-09-21T03:33:41.298611+0000","last_fullsized":"2024-09-21T03:33:41.298611+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:41:48.532726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.298922+0000","last_change":"2024-09-21T03:33:36.276841+0000","last_active":"2024-09-21T03:33:41.298922+0000","last_peered":"2024-09-21T03:33:41.298922+0000","last_clean":"2024-09-21T03:33:41.298922+0000","last_became_active":"2024-09-21T03:33:36.276522+0000","last_became_peered":"2024-09-21T03:33:36.276522+0000","last_unstale":"2024-09-21T03:33:41.298922+0000","last_undegraded":"2024-09-21T03:33:41.298922+0000","last_fullsized":"2024-09-21T03:33:41.298922+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:20:05.178755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.920682+0000","last_change":"2024-09-21T03:33:36.251561+0000","last_active":"2024-09-21T03:33:42.920682+0000","last_peered":"2024-09-21T03:33:42.920682+0000","last_clean":"2024-09-21T03:33:42.920682+0000","last_became_active":"2024-09-21T03:33:36.251161+0000","last_became_peered":"2024-09-21T03:33:36.251161+0000","last_unstale":"2024-09-21T03:33:42.920682+0000","last_undegraded":"2024-09-21T03:33:42.920682+0000","last_fullsized":"2024-09-21T03:33:42.920682+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:22:37.114294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.919931+0000","last_change":"2024-09-21T03:33:36.258249+0000","last_active":"2024-09-21T03:33:42.919931+0000","last_peered":"2024-09-21T03:33:42.919931+0000","last_clean":"2024-09-21T03:33:42.919931+0000","last_became_active":"2024-09-21T03:33:36.257558+0000","last_became_peered":"2024-09-21T03:33:36.257558+0000","last_unstale":"2024-09-21T03:33:42.919931+0000","last_undegraded":"2024-09-21T03:33:42.919931+0000","last_fullsized":"2024-09-21T03:33:42.919931+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:14:09.890548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.298137+0000","last_change":"2024-09-21T03:33:36.265238+0000","last_active":"2024-09-21T03:33:41.298137+0000","last_peered":"2024-09-21T03:33:41.298137+0000","last_clean":"2024-09-21T03:33:41.298137+0000","last_became_active":"2024-09-21T03:33:36.264904+0000","last_became_peered":"2024-09-21T03:33:36.264904+0000","last_unstale":"2024-09-21T03:33:41.298137+0000","last_undegraded":"2024-09-21T03:33:41.298137+0000","last_fullsized":"2024-09-21T03:33:41.298137+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:10:32.046638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.286379+0000","last_change":"2024-09-21T03:33:36.270243+0000","last_active":"2024-09-21T03:33:41.286379+0000","last_peered":"2024-09-21T03:33:41.286379+0000","last_clean":"2024-09-21T03:33:41.286379+0000","last_became_active":"2024-09-21T03:33:36.267458+0000","last_became_peered":"2024-09-21T03:33:36.267458+0000","last_unstale":"2024-09-21T03:33:41.286379+0000","last_undegraded":"2024-09-21T03:33:41.286379+0000","last_fullsized":"2024-09-21T03:33:41.286379+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:40:23.758202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.298523+0000","last_change":"2024-09-21T03:33:36.273294+0000","last_active":"2024-09-21T03:33:41.298523+0000","last_peered":"2024-09-21T03:33:41.298523+0000","last_clean":"2024-09-21T03:33:41.298523+0000","last_became_active":"2024-09-21T03:33:36.272841+0000","last_became_peered":"2024-09-21T03:33:36.272841+0000","last_unstale":"2024-09-21T03:33:41.298523+0000","last_undegraded":"2024-09-21T03:33:41.298523+0000","last_fullsized":"2024-09-21T03:33:41.298523+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:43:32.821740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.285970+0000","last_change":"2024-09-21T03:33:36.267665+0000","last_active":"2024-09-21T03:33:41.285970+0000","last_peered":"2024-09-21T03:33:41.285970+0000","last_clean":"2024-09-21T03:33:41.285970+0000","last_became_active":"2024-09-21T03:33:36.267339+0000","last_became_peered":"2024-09-21T03:33:36.267339+0000","last_unstale":"2024-09-21T03:33:41.285970+0000","last_undegraded":"2024-09-21T03:33:41.285970+0000","last_fullsized":"2024-09-21T03:33:41.285970+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:35:19.515735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.920418+0000","last_change":"2024-09-21T03:33:36.260290+0000","last_active":"2024-09-21T03:33:42.920418+0000","last_peered":"2024-09-21T03:33:42.920418+0000","last_clean":"2024-09-21T03:33:42.920418+0000","last_became_active":"2024-09-21T03:33:36.259984+0000","last_became_peered":"2024-09-21T03:33:36.259984+0000","last_unstale":"2024-09-21T03:33:42.920418+0000","last_undegraded":"2024-09-21T03:33:42.920418+0000","last_fullsized":"2024-09-21T03:33:42.920418+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:15:12.112267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.920700+0000","last_change":"2024-09-21T03:33:36.256894+0000","last_active":"2024-09-21T03:33:42.920700+0000","last_peered":"2024-09-21T03:33:42.920700+0000","last_clean":"2024-09-21T03:33:42.920700+0000","last_became_active":"2024-09-21T03:33:36.256598+0000","last_became_peered":"2024-09-21T03:33:36.256598+0000","last_unstale":"2024-09-21T03:33:42.920700+0000","last_undegraded":"2024-09-21T03:33:42.920700+0000","last_fullsized":"2024-09-21T03:33:42.920700+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:59:34.604906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.298773+0000","last_change":"2024-09-21T03:33:36.273537+0000","last_active":"2024-09-21T03:33:41.298773+0000","last_peered":"2024-09-21T03:33:41.298773+0000","last_clean":"2024-09-21T03:33:41.298773+0000","last_became_active":"2024-09-21T03:33:36.272386+0000","last_became_peered":"2024-09-21T03:33:36.272386+0000","last_unstale":"2024-09-21T03:33:41.298773+0000","last_undegraded":"2024-09-21T03:33:41.298773+0000","last_fullsized":"2024-09-21T03:33:41.298773+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:10:25.723208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":1,"up_from":16,"seq":68719476745,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32036,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339804,"statfs":{"total":96636764160,"available":96603959296,"internally_reserved":0,"allocated":868352,"data_stored":697804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509449,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31460,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340380,"statfs":{"total":96636764160,"available":96604549120,"internally_reserved":0,"allocated":278528,"data_stored":107490,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509448,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31456,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340384,"statfs":{"total":96636764160,"available":96604553216,"internally_reserved":0,"allocated":274432,"data_stored":107948,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509449,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":282624,"data_stored":111208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509448,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31464,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340376,"statfs":{"total":96636764160,"available":96604545024,"internally_reserved":0,"allocated":278528,"data_stored":107422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542152,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31456,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340384,"statfs":{"total":96636764160,"available":96604553216,"internally_reserved":0,"allocated":270336,"data_stored":107366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32040,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339800,"statfs":{"total":96636764160,"available":96603955200,"internally_reserved":0,"allocated":872448,"data_stored":697826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542153,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35628,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336212,"statfs":{"total":96636764160,"available":96600281088,"internally_reserved":0,"allocated":290816,"data_stored":111880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":4514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T03:33:57.393 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T03:33:57.544 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:57.541+0000 7f49b582e640 1 -- 172.21.15.38:0/1604089093 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f49b0162590 msgr2=0x7f49b01629a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:57.544 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:57.541+0000 7f49b582e640 1 --2- 172.21.15.38:0/1604089093 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f49b0162590 0x7f49b01629a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f49b00b5a40 tx=0x7f49a402a560 comp rx=0 tx=0).stop 2024-09-21T03:33:57.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:57.541+0000 7f49b582e640 1 -- 172.21.15.38:0/1604089093 shutdown_connections 2024-09-21T03:33:57.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:57.541+0000 7f49b582e640 1 --2- 172.21.15.38:0/1604089093 >> v2:172.21.15.195:3300/0 conn(0x7f49b00b2e60 0x7f49b00b3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:57.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:57.541+0000 7f49b582e640 1 --2- 172.21.15.38:0/1604089093 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f49b0162590 0x7f49b01629a0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:57.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:57.541+0000 7f49b582e640 1 -- 172.21.15.38:0/1604089093 >> 172.21.15.38:0/1604089093 conn(0x7f49b00d4440 msgr2=0x7f49b00d4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:57.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:57.541+0000 7f49b582e640 1 -- 172.21.15.38:0/1604089093 shutdown_connections 2024-09-21T03:33:57.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:57.541+0000 7f49b582e640 1 -- 172.21.15.38:0/1604089093 wait complete. 2024-09-21T03:33:57.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:57.543+0000 7f49b582e640 1 Processor -- start 2024-09-21T03:33:57.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:57.543+0000 7f49b582e640 1 -- start start 2024-09-21T03:33:57.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:57.545+0000 7f49b582e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f49b00b2e60 0x7f49b01e9e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:57.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:57.545+0000 7f49b582e640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f49b00cfe00 0x7f49b00c7180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:57.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:57.545+0000 7f49b582e640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f49b0162590 0x7f49b00b4140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:57.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:57.545+0000 7f49b582e640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f49b00b3930 con 0x7f49b00b2e60 2024-09-21T03:33:57.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:57.545+0000 7f49b582e640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f49b00b37b0 con 0x7f49b0162590 2024-09-21T03:33:57.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:57.545+0000 7f49b582e640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f49b00b3ab0 con 0x7f49b00cfe00 2024-09-21T03:33:57.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:57.545+0000 7f49aeffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f49b00b2e60 0x7f49b01e9e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:57.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:57.545+0000 7f49aeffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f49b00b2e60 0x7f49b01e9e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41960/0 (socket says 172.21.15.38:41960) 2024-09-21T03:33:57.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:57.545+0000 7f49aeffd640 1 -- 172.21.15.38:0/836982614 learned_addr learned my addr 172.21.15.38:0/836982614 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:33:57.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:57.545+0000 7f49ae7fc640 1 --2- 172.21.15.38:0/836982614 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f49b00cfe00 0x7f49b00c7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:57.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:57.545+0000 7f49af7fe640 1 --2- 172.21.15.38:0/836982614 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f49b0162590 0x7f49b00b4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:57.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:57.545+0000 7f49aeffd640 1 -- 172.21.15.38:0/836982614 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f49b00cfe00 msgr2=0x7f49b00c7180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:57.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:57.545+0000 7f49aeffd640 1 --2- 172.21.15.38:0/836982614 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f49b00cfe00 0x7f49b00c7180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:57.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:57.545+0000 7f49aeffd640 1 -- 172.21.15.38:0/836982614 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f49b0162590 msgr2=0x7f49b00b4140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:57.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:57.545+0000 7f49aeffd640 1 --2- 172.21.15.38:0/836982614 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f49b0162590 0x7f49b00b4140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:57.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:57.545+0000 7f49aeffd640 1 -- 172.21.15.38:0/836982614 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49b01f92a0 con 0x7f49b00b2e60 2024-09-21T03:33:57.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:57.545+0000 7f49aeffd640 1 --2- 172.21.15.38:0/836982614 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f49b00b2e60 0x7f49b01e9e70 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f49a000ba80 tx=0x7f49a000bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:57.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:57.545+0000 7f4997fff640 1 -- 172.21.15.38:0/836982614 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49a0019070 con 0x7f49b00b2e60 2024-09-21T03:33:57.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:57.545+0000 7f4997fff640 1 -- 172.21.15.38:0/836982614 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f49a000cd80 con 0x7f49b00b2e60 2024-09-21T03:33:57.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:57.545+0000 7f49b582e640 1 -- 172.21.15.38:0/836982614 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49b01f9580 con 0x7f49b00b2e60 2024-09-21T03:33:57.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:57.545+0000 7f4997fff640 1 -- 172.21.15.38:0/836982614 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49a00146a0 con 0x7f49b00b2e60 2024-09-21T03:33:57.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:57.546+0000 7f49b582e640 1 -- 172.21.15.38:0/836982614 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49b01ea3b0 con 0x7f49b00b2e60 2024-09-21T03:33:57.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:57.547+0000 7f49b582e640 1 -- 172.21.15.38:0/836982614 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49b00ce2e0 con 0x7f49b00b2e60 2024-09-21T03:33:57.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:57.547+0000 7f4997fff640 1 -- 172.21.15.38:0/836982614 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f49a001c070 con 0x7f49b00b2e60 2024-09-21T03:33:57.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:57.548+0000 7f4997fff640 1 --2- 172.21.15.38:0/836982614 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f499c078b90 0x7f499c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:57.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:57.548+0000 7f4997fff640 1 -- 172.21.15.38:0/836982614 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f49a009ae00 con 0x7f49b00b2e60 2024-09-21T03:33:57.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:57.548+0000 7f49ae7fc640 1 --2- 172.21.15.38:0/836982614 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f499c078b90 0x7f499c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:57.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:57.549+0000 7f49ae7fc640 1 --2- 172.21.15.38:0/836982614 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f499c078b90 0x7f499c07b0a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f49a4033000 tx=0x7f49a4004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:57.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:57.553+0000 7f4997fff640 1 -- 172.21.15.38:0/836982614 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f49a00628b0 con 0x7f49b00b2e60 2024-09-21T03:33:57.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:57.736+0000 7f49b582e640 1 -- 172.21.15.38:0/836982614 --> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f49b01ea8f0 con 0x7f499c078b90 2024-09-21T03:33:57.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:57.743+0000 7f4997fff640 1 -- 172.21.15.38:0/836982614 <== mgr.4117 v2:172.21.15.195:6800/1631901344 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332519 (secure 0 0 0) 0x7f49b01ea8f0 con 0x7f499c078b90 2024-09-21T03:33:57.749 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:33:57.750 INFO:teuthology.orchestra.run.smithi038.stderr:dumped all 2024-09-21T03:33:57.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:57.746+0000 7f49b582e640 1 -- 172.21.15.38:0/836982614 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f499c078b90 msgr2=0x7f499c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:57.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:57.746+0000 7f49b582e640 1 --2- 172.21.15.38:0/836982614 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f499c078b90 0x7f499c07b0a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f49a4033000 tx=0x7f49a4004730 comp rx=0 tx=0).stop 2024-09-21T03:33:57.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:57.746+0000 7f49b582e640 1 -- 172.21.15.38:0/836982614 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f49b00b2e60 msgr2=0x7f49b01e9e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:57.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:57.746+0000 7f49b582e640 1 --2- 172.21.15.38:0/836982614 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f49b00b2e60 0x7f49b01e9e70 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f49a000ba80 tx=0x7f49a000bf40 comp rx=0 tx=0).stop 2024-09-21T03:33:57.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:57.746+0000 7f49b582e640 1 -- 172.21.15.38:0/836982614 shutdown_connections 2024-09-21T03:33:57.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:57.746+0000 7f49b582e640 1 --2- 172.21.15.38:0/836982614 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f49b0162590 0x7f49b00b4140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:57.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:57.746+0000 7f49b582e640 1 --2- 172.21.15.38:0/836982614 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f499c078b90 0x7f499c07b0a0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:57.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:57.746+0000 7f49b582e640 1 --2- 172.21.15.38:0/836982614 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f49b00cfe00 0x7f49b00c7180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:57.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:57.746+0000 7f49b582e640 1 --2- 172.21.15.38:0/836982614 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f49b00b2e60 0x7f49b01e9e70 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:57.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:57.746+0000 7f49b582e640 1 -- 172.21.15.38:0/836982614 >> 172.21.15.38:0/836982614 conn(0x7f49b00d4440 msgr2=0x7f49b00d4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:57.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:57.746+0000 7f49b582e640 1 -- 172.21.15.38:0/836982614 shutdown_connections 2024-09-21T03:33:57.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:57.746+0000 7f49b582e640 1 -- 172.21.15.38:0/836982614 wait complete. 2024-09-21T03:33:57.760 INFO:teuthology.orchestra.run.smithi038.stdout:{"pg_ready":true,"pg_map":{"version":36,"stamp":"2024-09-21T03:33:56.982031+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":149,"num_write_kb":1646,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":114,"ondisk_log_size":114,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":252840,"kb_used_data":3336,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754721880,"statfs":{"total":773094113280,"available":772835205120,"internally_reserved":0,"allocated":3416064,"data_stored":2048944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12748,"internal_metadata":255315508},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2230,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003913"},"pg_stats":[{"pgid":"2.1f","version":"21'4","reported_seq":27,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.285664+0000","last_change":"2024-09-21T03:33:35.252626+0000","last_active":"2024-09-21T03:33:41.285664+0000","last_peered":"2024-09-21T03:33:41.285664+0000","last_clean":"2024-09-21T03:33:41.285664+0000","last_became_active":"2024-09-21T03:33:35.252204+0000","last_became_peered":"2024-09-21T03:33:35.252204+0000","last_unstale":"2024-09-21T03:33:41.285664+0000","last_undegraded":"2024-09-21T03:33:41.285664+0000","last_fullsized":"2024-09-21T03:33:41.285664+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_clean_scrub_stamp":"2024-09-21T03:33:34.220881+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:42:59.930617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.920863+0000","last_change":"2024-09-21T03:33:36.262395+0000","last_active":"2024-09-21T03:33:42.920863+0000","last_peered":"2024-09-21T03:33:42.920863+0000","last_clean":"2024-09-21T03:33:42.920863+0000","last_became_active":"2024-09-21T03:33:36.262085+0000","last_became_peered":"2024-09-21T03:33:36.262085+0000","last_unstale":"2024-09-21T03:33:42.920863+0000","last_undegraded":"2024-09-21T03:33:42.920863+0000","last_fullsized":"2024-09-21T03:33:42.920863+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:13:46.036072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.754945+0000","last_change":"2024-09-21T03:33:40.303707+0000","last_active":"2024-09-21T03:33:41.754945+0000","last_peered":"2024-09-21T03:33:41.754945+0000","last_clean":"2024-09-21T03:33:41.754945+0000","last_became_active":"2024-09-21T03:33:40.303328+0000","last_became_peered":"2024-09-21T03:33:40.303328+0000","last_unstale":"2024-09-21T03:33:41.754945+0000","last_undegraded":"2024-09-21T03:33:41.754945+0000","last_fullsized":"2024-09-21T03:33:41.754945+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_clean_scrub_stamp":"2024-09-21T03:33:39.262231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:58:01.084602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4,1],"acting":[7,6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.921084+0000","last_change":"2024-09-21T03:33:35.253147+0000","last_active":"2024-09-21T03:33:42.921084+0000","last_peered":"2024-09-21T03:33:42.921084+0000","last_clean":"2024-09-21T03:33:42.921084+0000","last_became_active":"2024-09-21T03:33:35.251243+0000","last_became_peered":"2024-09-21T03:33:35.251243+0000","last_unstale":"2024-09-21T03:33:42.921084+0000","last_undegraded":"2024-09-21T03:33:42.921084+0000","last_fullsized":"2024-09-21T03:33:42.921084+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_clean_scrub_stamp":"2024-09-21T03:33:34.220881+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:21:52.640936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.920870+0000","last_change":"2024-09-21T03:33:36.255323+0000","last_active":"2024-09-21T03:33:42.920870+0000","last_peered":"2024-09-21T03:33:42.920870+0000","last_clean":"2024-09-21T03:33:42.920870+0000","last_became_active":"2024-09-21T03:33:36.253192+0000","last_became_peered":"2024-09-21T03:33:36.253192+0000","last_unstale":"2024-09-21T03:33:42.920870+0000","last_undegraded":"2024-09-21T03:33:42.920870+0000","last_fullsized":"2024-09-21T03:33:42.920870+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:21:43.036932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.285988+0000","last_change":"2024-09-21T03:33:40.294983+0000","last_active":"2024-09-21T03:33:41.285988+0000","last_peered":"2024-09-21T03:33:41.285988+0000","last_clean":"2024-09-21T03:33:41.285988+0000","last_became_active":"2024-09-21T03:33:40.294528+0000","last_became_peered":"2024-09-21T03:33:40.294528+0000","last_unstale":"2024-09-21T03:33:41.285988+0000","last_undegraded":"2024-09-21T03:33:41.285988+0000","last_fullsized":"2024-09-21T03:33:41.285988+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_clean_scrub_stamp":"2024-09-21T03:33:39.262231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:39:42.878029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4,6],"acting":[2,7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.298804+0000","last_change":"2024-09-21T03:33:35.260455+0000","last_active":"2024-09-21T03:33:41.298804+0000","last_peered":"2024-09-21T03:33:41.298804+0000","last_clean":"2024-09-21T03:33:41.298804+0000","last_became_active":"2024-09-21T03:33:35.260074+0000","last_became_peered":"2024-09-21T03:33:35.260074+0000","last_unstale":"2024-09-21T03:33:41.298804+0000","last_undegraded":"2024-09-21T03:33:41.298804+0000","last_fullsized":"2024-09-21T03:33:41.298804+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_clean_scrub_stamp":"2024-09-21T03:33:34.220881+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:55:02.392036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.920774+0000","last_change":"2024-09-21T03:33:36.274462+0000","last_active":"2024-09-21T03:33:42.920774+0000","last_peered":"2024-09-21T03:33:42.920774+0000","last_clean":"2024-09-21T03:33:42.920774+0000","last_became_active":"2024-09-21T03:33:36.274080+0000","last_became_peered":"2024-09-21T03:33:36.274080+0000","last_unstale":"2024-09-21T03:33:42.920774+0000","last_undegraded":"2024-09-21T03:33:42.920774+0000","last_fullsized":"2024-09-21T03:33:42.920774+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:55:20.815479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.286750+0000","last_change":"2024-09-21T03:33:40.294613+0000","last_active":"2024-09-21T03:33:41.286750+0000","last_peered":"2024-09-21T03:33:41.286750+0000","last_clean":"2024-09-21T03:33:41.286750+0000","last_became_active":"2024-09-21T03:33:40.294138+0000","last_became_peered":"2024-09-21T03:33:40.294138+0000","last_unstale":"2024-09-21T03:33:41.286750+0000","last_undegraded":"2024-09-21T03:33:41.286750+0000","last_fullsized":"2024-09-21T03:33:41.286750+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_clean_scrub_stamp":"2024-09-21T03:33:39.262231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:10:29.758718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3,6],"acting":[2,5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.298963+0000","last_change":"2024-09-21T03:33:35.265118+0000","last_active":"2024-09-21T03:33:41.298963+0000","last_peered":"2024-09-21T03:33:41.298963+0000","last_clean":"2024-09-21T03:33:41.298963+0000","last_became_active":"2024-09-21T03:33:35.262875+0000","last_became_peered":"2024-09-21T03:33:35.262875+0000","last_unstale":"2024-09-21T03:33:41.298963+0000","last_undegraded":"2024-09-21T03:33:41.298963+0000","last_fullsized":"2024-09-21T03:33:41.298963+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_clean_scrub_stamp":"2024-09-21T03:33:34.220881+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:40:22.752363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.921523+0000","last_change":"2024-09-21T03:33:36.266776+0000","last_active":"2024-09-21T03:33:42.921523+0000","last_peered":"2024-09-21T03:33:42.921523+0000","last_clean":"2024-09-21T03:33:42.921523+0000","last_became_active":"2024-09-21T03:33:36.266449+0000","last_became_peered":"2024-09-21T03:33:36.266449+0000","last_unstale":"2024-09-21T03:33:42.921523+0000","last_undegraded":"2024-09-21T03:33:42.921523+0000","last_fullsized":"2024-09-21T03:33:42.921523+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:39:22.951079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.286642+0000","last_change":"2024-09-21T03:33:40.293654+0000","last_active":"2024-09-21T03:33:41.286642+0000","last_peered":"2024-09-21T03:33:41.286642+0000","last_clean":"2024-09-21T03:33:41.286642+0000","last_became_active":"2024-09-21T03:33:40.293236+0000","last_became_peered":"2024-09-21T03:33:40.293236+0000","last_unstale":"2024-09-21T03:33:41.286642+0000","last_undegraded":"2024-09-21T03:33:41.286642+0000","last_fullsized":"2024-09-21T03:33:41.286642+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_clean_scrub_stamp":"2024-09-21T03:33:39.262231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:20:09.214266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,5,7],"acting":[2,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.920921+0000","last_change":"2024-09-21T03:33:36.252854+0000","last_active":"2024-09-21T03:33:42.920921+0000","last_peered":"2024-09-21T03:33:42.920921+0000","last_clean":"2024-09-21T03:33:42.920921+0000","last_became_active":"2024-09-21T03:33:36.252384+0000","last_became_peered":"2024-09-21T03:33:36.252384+0000","last_unstale":"2024-09-21T03:33:42.920921+0000","last_undegraded":"2024-09-21T03:33:42.920921+0000","last_fullsized":"2024-09-21T03:33:42.920921+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:35:32.535895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.921139+0000","last_change":"2024-09-21T03:33:35.252643+0000","last_active":"2024-09-21T03:33:42.921139+0000","last_peered":"2024-09-21T03:33:42.921139+0000","last_clean":"2024-09-21T03:33:42.921139+0000","last_became_active":"2024-09-21T03:33:35.252335+0000","last_became_peered":"2024-09-21T03:33:35.252335+0000","last_unstale":"2024-09-21T03:33:42.921139+0000","last_undegraded":"2024-09-21T03:33:42.921139+0000","last_fullsized":"2024-09-21T03:33:42.921139+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_clean_scrub_stamp":"2024-09-21T03:33:34.220881+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:07:29.037511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.921479+0000","last_change":"2024-09-21T03:33:40.307347+0000","last_active":"2024-09-21T03:33:42.921479+0000","last_peered":"2024-09-21T03:33:42.921479+0000","last_clean":"2024-09-21T03:33:42.921479+0000","last_became_active":"2024-09-21T03:33:40.306899+0000","last_became_peered":"2024-09-21T03:33:40.306899+0000","last_unstale":"2024-09-21T03:33:42.921479+0000","last_undegraded":"2024-09-21T03:33:42.921479+0000","last_fullsized":"2024-09-21T03:33:42.921479+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_clean_scrub_stamp":"2024-09-21T03:33:39.262231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:45:45.403094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,5],"acting":[3,1,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.754733+0000","last_change":"2024-09-21T03:33:36.257562+0000","last_active":"2024-09-21T03:33:41.754733+0000","last_peered":"2024-09-21T03:33:41.754733+0000","last_clean":"2024-09-21T03:33:41.754733+0000","last_became_active":"2024-09-21T03:33:36.255065+0000","last_became_peered":"2024-09-21T03:33:36.255065+0000","last_unstale":"2024-09-21T03:33:41.754733+0000","last_undegraded":"2024-09-21T03:33:41.754733+0000","last_fullsized":"2024-09-21T03:33:41.754733+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:35:07.883056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.284335+0000","last_change":"2024-09-21T03:33:35.254523+0000","last_active":"2024-09-21T03:33:41.284335+0000","last_peered":"2024-09-21T03:33:41.284335+0000","last_clean":"2024-09-21T03:33:41.284335+0000","last_became_active":"2024-09-21T03:33:35.254185+0000","last_became_peered":"2024-09-21T03:33:35.254185+0000","last_unstale":"2024-09-21T03:33:41.284335+0000","last_undegraded":"2024-09-21T03:33:41.284335+0000","last_fullsized":"2024-09-21T03:33:41.284335+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_clean_scrub_stamp":"2024-09-21T03:33:34.220881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:06:37.946009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.298659+0000","last_change":"2024-09-21T03:33:40.304238+0000","last_active":"2024-09-21T03:33:41.298659+0000","last_peered":"2024-09-21T03:33:41.298659+0000","last_clean":"2024-09-21T03:33:41.298659+0000","last_became_active":"2024-09-21T03:33:40.303785+0000","last_became_peered":"2024-09-21T03:33:40.303785+0000","last_unstale":"2024-09-21T03:33:41.298659+0000","last_undegraded":"2024-09-21T03:33:41.298659+0000","last_fullsized":"2024-09-21T03:33:41.298659+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_clean_scrub_stamp":"2024-09-21T03:33:39.262231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:28:07.313548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6,1],"acting":[0,4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.283514+0000","last_change":"2024-09-21T03:33:36.255712+0000","last_active":"2024-09-21T03:33:41.283514+0000","last_peered":"2024-09-21T03:33:41.283514+0000","last_clean":"2024-09-21T03:33:41.283514+0000","last_became_active":"2024-09-21T03:33:36.255386+0000","last_became_peered":"2024-09-21T03:33:36.255386+0000","last_unstale":"2024-09-21T03:33:41.283514+0000","last_undegraded":"2024-09-21T03:33:41.283514+0000","last_fullsized":"2024-09-21T03:33:41.283514+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:49:00.717844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.284629+0000","last_change":"2024-09-21T03:33:35.266345+0000","last_active":"2024-09-21T03:33:41.284629+0000","last_peered":"2024-09-21T03:33:41.284629+0000","last_clean":"2024-09-21T03:33:41.284629+0000","last_became_active":"2024-09-21T03:33:35.266059+0000","last_became_peered":"2024-09-21T03:33:35.266059+0000","last_unstale":"2024-09-21T03:33:41.284629+0000","last_undegraded":"2024-09-21T03:33:41.284629+0000","last_fullsized":"2024-09-21T03:33:41.284629+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_clean_scrub_stamp":"2024-09-21T03:33:34.220881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:54:40.741741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.755284+0000","last_change":"2024-09-21T03:33:40.303927+0000","last_active":"2024-09-21T03:33:41.755284+0000","last_peered":"2024-09-21T03:33:41.755284+0000","last_clean":"2024-09-21T03:33:41.755284+0000","last_became_active":"2024-09-21T03:33:40.303511+0000","last_became_peered":"2024-09-21T03:33:40.303511+0000","last_unstale":"2024-09-21T03:33:41.755284+0000","last_undegraded":"2024-09-21T03:33:41.755284+0000","last_fullsized":"2024-09-21T03:33:41.755284+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_clean_scrub_stamp":"2024-09-21T03:33:39.262231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:27:26.725520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2,3],"acting":[7,4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.920506+0000","last_change":"2024-09-21T03:33:36.249470+0000","last_active":"2024-09-21T03:33:42.920506+0000","last_peered":"2024-09-21T03:33:42.920506+0000","last_clean":"2024-09-21T03:33:42.920506+0000","last_became_active":"2024-09-21T03:33:36.248726+0000","last_became_peered":"2024-09-21T03:33:36.248726+0000","last_unstale":"2024-09-21T03:33:42.920506+0000","last_undegraded":"2024-09-21T03:33:42.920506+0000","last_fullsized":"2024-09-21T03:33:42.920506+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:56:46.691011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.297935+0000","last_change":"2024-09-21T03:33:35.260543+0000","last_active":"2024-09-21T03:33:41.297935+0000","last_peered":"2024-09-21T03:33:41.297935+0000","last_clean":"2024-09-21T03:33:41.297935+0000","last_became_active":"2024-09-21T03:33:35.260041+0000","last_became_peered":"2024-09-21T03:33:35.260041+0000","last_unstale":"2024-09-21T03:33:41.297935+0000","last_undegraded":"2024-09-21T03:33:41.297935+0000","last_fullsized":"2024-09-21T03:33:41.297935+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_clean_scrub_stamp":"2024-09-21T03:33:34.220881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:59:06.265387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.286314+0000","last_change":"2024-09-21T03:33:40.309460+0000","last_active":"2024-09-21T03:33:41.286314+0000","last_peered":"2024-09-21T03:33:41.286314+0000","last_clean":"2024-09-21T03:33:41.286314+0000","last_became_active":"2024-09-21T03:33:40.306652+0000","last_became_peered":"2024-09-21T03:33:40.306652+0000","last_unstale":"2024-09-21T03:33:41.286314+0000","last_undegraded":"2024-09-21T03:33:41.286314+0000","last_fullsized":"2024-09-21T03:33:41.286314+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_clean_scrub_stamp":"2024-09-21T03:33:39.262231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:43:35.273085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,7],"acting":[2,1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.284755+0000","last_change":"2024-09-21T03:33:36.256025+0000","last_active":"2024-09-21T03:33:41.284755+0000","last_peered":"2024-09-21T03:33:41.284755+0000","last_clean":"2024-09-21T03:33:41.284755+0000","last_became_active":"2024-09-21T03:33:36.255648+0000","last_became_peered":"2024-09-21T03:33:36.255648+0000","last_unstale":"2024-09-21T03:33:41.284755+0000","last_undegraded":"2024-09-21T03:33:41.284755+0000","last_fullsized":"2024-09-21T03:33:41.284755+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:48:33.535308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.921585+0000","last_change":"2024-09-21T03:33:35.264889+0000","last_active":"2024-09-21T03:33:42.921585+0000","last_peered":"2024-09-21T03:33:42.921585+0000","last_clean":"2024-09-21T03:33:42.921585+0000","last_became_active":"2024-09-21T03:33:35.264160+0000","last_became_peered":"2024-09-21T03:33:35.264160+0000","last_unstale":"2024-09-21T03:33:42.921585+0000","last_undegraded":"2024-09-21T03:33:42.921585+0000","last_fullsized":"2024-09-21T03:33:42.921585+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_clean_scrub_stamp":"2024-09-21T03:33:34.220881+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:48:59.712004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.286424+0000","last_change":"2024-09-21T03:33:40.304067+0000","last_active":"2024-09-21T03:33:41.286424+0000","last_peered":"2024-09-21T03:33:41.286424+0000","last_clean":"2024-09-21T03:33:41.286424+0000","last_became_active":"2024-09-21T03:33:40.303613+0000","last_became_peered":"2024-09-21T03:33:40.303613+0000","last_unstale":"2024-09-21T03:33:41.286424+0000","last_undegraded":"2024-09-21T03:33:41.286424+0000","last_fullsized":"2024-09-21T03:33:41.286424+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_clean_scrub_stamp":"2024-09-21T03:33:39.262231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:51:21.940594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,3,1],"acting":[2,6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.285607+0000","last_change":"2024-09-21T03:33:36.258150+0000","last_active":"2024-09-21T03:33:41.285607+0000","last_peered":"2024-09-21T03:33:41.285607+0000","last_clean":"2024-09-21T03:33:41.285607+0000","last_became_active":"2024-09-21T03:33:36.257655+0000","last_became_peered":"2024-09-21T03:33:36.257655+0000","last_unstale":"2024-09-21T03:33:41.285607+0000","last_undegraded":"2024-09-21T03:33:41.285607+0000","last_fullsized":"2024-09-21T03:33:41.285607+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:10:32.046638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"22'1","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.285234+0000","last_change":"2024-09-21T03:33:35.264750+0000","last_active":"2024-09-21T03:33:41.285234+0000","last_peered":"2024-09-21T03:33:41.285234+0000","last_clean":"2024-09-21T03:33:41.285234+0000","last_became_active":"2024-09-21T03:33:35.264380+0000","last_became_peered":"2024-09-21T03:33:35.264380+0000","last_unstale":"2024-09-21T03:33:41.285234+0000","last_undegraded":"2024-09-21T03:33:41.285234+0000","last_fullsized":"2024-09-21T03:33:41.285234+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_clean_scrub_stamp":"2024-09-21T03:33:34.220881+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:12:06.105531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.284754+0000","last_change":"2024-09-21T03:33:40.303103+0000","last_active":"2024-09-21T03:33:41.284754+0000","last_peered":"2024-09-21T03:33:41.284754+0000","last_clean":"2024-09-21T03:33:41.284754+0000","last_became_active":"2024-09-21T03:33:40.301739+0000","last_became_peered":"2024-09-21T03:33:40.301739+0000","last_unstale":"2024-09-21T03:33:41.284754+0000","last_undegraded":"2024-09-21T03:33:41.284754+0000","last_fullsized":"2024-09-21T03:33:41.284754+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_clean_scrub_stamp":"2024-09-21T03:33:39.262231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:00:58.389938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6,3],"acting":[4,1,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.285678+0000","last_change":"2024-09-21T03:33:36.268661+0000","last_active":"2024-09-21T03:33:41.285678+0000","last_peered":"2024-09-21T03:33:41.285678+0000","last_clean":"2024-09-21T03:33:41.285678+0000","last_became_active":"2024-09-21T03:33:36.268317+0000","last_became_peered":"2024-09-21T03:33:36.268317+0000","last_unstale":"2024-09-21T03:33:41.285678+0000","last_undegraded":"2024-09-21T03:33:41.285678+0000","last_fullsized":"2024-09-21T03:33:41.285678+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:19:27.002524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.921773+0000","last_change":"2024-09-21T03:33:35.250205+0000","last_active":"2024-09-21T03:33:42.921773+0000","last_peered":"2024-09-21T03:33:42.921773+0000","last_clean":"2024-09-21T03:33:42.921773+0000","last_became_active":"2024-09-21T03:33:35.249883+0000","last_became_peered":"2024-09-21T03:33:35.249883+0000","last_unstale":"2024-09-21T03:33:42.921773+0000","last_undegraded":"2024-09-21T03:33:42.921773+0000","last_fullsized":"2024-09-21T03:33:42.921773+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_clean_scrub_stamp":"2024-09-21T03:33:34.220881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:48:54.078693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.283974+0000","last_change":"2024-09-21T03:33:40.298889+0000","last_active":"2024-09-21T03:33:41.283974+0000","last_peered":"2024-09-21T03:33:41.283974+0000","last_clean":"2024-09-21T03:33:41.283974+0000","last_became_active":"2024-09-21T03:33:40.298435+0000","last_became_peered":"2024-09-21T03:33:40.298435+0000","last_unstale":"2024-09-21T03:33:41.283974+0000","last_undegraded":"2024-09-21T03:33:41.283974+0000","last_fullsized":"2024-09-21T03:33:41.283974+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_clean_scrub_stamp":"2024-09-21T03:33:39.262231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:29:46.742857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,2,7],"acting":[6,0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.286003+0000","last_change":"2024-09-21T03:33:36.258037+0000","last_active":"2024-09-21T03:33:41.286003+0000","last_peered":"2024-09-21T03:33:41.286003+0000","last_clean":"2024-09-21T03:33:41.286003+0000","last_became_active":"2024-09-21T03:33:36.257687+0000","last_became_peered":"2024-09-21T03:33:36.257687+0000","last_unstale":"2024-09-21T03:33:41.286003+0000","last_undegraded":"2024-09-21T03:33:41.286003+0000","last_fullsized":"2024-09-21T03:33:41.286003+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:12:07.111371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.920392+0000","last_change":"2024-09-21T03:33:35.254341+0000","last_active":"2024-09-21T03:33:42.920392+0000","last_peered":"2024-09-21T03:33:42.920392+0000","last_clean":"2024-09-21T03:33:42.920392+0000","last_became_active":"2024-09-21T03:33:35.254031+0000","last_became_peered":"2024-09-21T03:33:35.254031+0000","last_unstale":"2024-09-21T03:33:42.920392+0000","last_undegraded":"2024-09-21T03:33:42.920392+0000","last_fullsized":"2024-09-21T03:33:42.920392+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_clean_scrub_stamp":"2024-09-21T03:33:34.220881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:39:48.189397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.284028+0000","last_change":"2024-09-21T03:33:40.295123+0000","last_active":"2024-09-21T03:33:41.284028+0000","last_peered":"2024-09-21T03:33:41.284028+0000","last_clean":"2024-09-21T03:33:41.284028+0000","last_became_active":"2024-09-21T03:33:40.294672+0000","last_became_peered":"2024-09-21T03:33:40.294672+0000","last_unstale":"2024-09-21T03:33:41.284028+0000","last_undegraded":"2024-09-21T03:33:41.284028+0000","last_fullsized":"2024-09-21T03:33:41.284028+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_clean_scrub_stamp":"2024-09-21T03:33:39.262231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:43:35.273085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5,7],"acting":[6,3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.298299+0000","last_change":"2024-09-21T03:33:36.277694+0000","last_active":"2024-09-21T03:33:41.298299+0000","last_peered":"2024-09-21T03:33:41.298299+0000","last_clean":"2024-09-21T03:33:41.298299+0000","last_became_active":"2024-09-21T03:33:36.277375+0000","last_became_peered":"2024-09-21T03:33:36.277375+0000","last_unstale":"2024-09-21T03:33:41.298299+0000","last_undegraded":"2024-09-21T03:33:41.298299+0000","last_fullsized":"2024-09-21T03:33:41.298299+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:39:38.842518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.921283+0000","last_change":"2024-09-21T03:33:35.250423+0000","last_active":"2024-09-21T03:33:42.921283+0000","last_peered":"2024-09-21T03:33:42.921283+0000","last_clean":"2024-09-21T03:33:42.921283+0000","last_became_active":"2024-09-21T03:33:35.248426+0000","last_became_peered":"2024-09-21T03:33:35.248426+0000","last_unstale":"2024-09-21T03:33:42.921283+0000","last_undegraded":"2024-09-21T03:33:42.921283+0000","last_fullsized":"2024-09-21T03:33:42.921283+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_clean_scrub_stamp":"2024-09-21T03:33:34.220881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:56:24.872394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.285593+0000","last_change":"2024-09-21T03:33:40.302877+0000","last_active":"2024-09-21T03:33:41.285593+0000","last_peered":"2024-09-21T03:33:41.285593+0000","last_clean":"2024-09-21T03:33:41.285593+0000","last_became_active":"2024-09-21T03:33:40.301015+0000","last_became_peered":"2024-09-21T03:33:40.301015+0000","last_unstale":"2024-09-21T03:33:41.285593+0000","last_undegraded":"2024-09-21T03:33:41.285593+0000","last_fullsized":"2024-09-21T03:33:41.285593+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_clean_scrub_stamp":"2024-09-21T03:33:39.262231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:20:09.214266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0,3],"acting":[6,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.919937+0000","last_change":"2024-09-21T03:33:36.255208+0000","last_active":"2024-09-21T03:33:42.919937+0000","last_peered":"2024-09-21T03:33:42.919937+0000","last_clean":"2024-09-21T03:33:42.919937+0000","last_became_active":"2024-09-21T03:33:36.254340+0000","last_became_peered":"2024-09-21T03:33:36.254340+0000","last_unstale":"2024-09-21T03:33:42.919937+0000","last_undegraded":"2024-09-21T03:33:42.919937+0000","last_fullsized":"2024-09-21T03:33:42.919937+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:59:34.251362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.754771+0000","last_change":"2024-09-21T03:33:35.255282+0000","last_active":"2024-09-21T03:33:41.754771+0000","last_peered":"2024-09-21T03:33:41.754771+0000","last_clean":"2024-09-21T03:33:41.754771+0000","last_became_active":"2024-09-21T03:33:35.254934+0000","last_became_peered":"2024-09-21T03:33:35.254934+0000","last_unstale":"2024-09-21T03:33:41.754771+0000","last_undegraded":"2024-09-21T03:33:41.754771+0000","last_fullsized":"2024-09-21T03:33:41.754771+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_clean_scrub_stamp":"2024-09-21T03:33:34.220881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:08:00.579605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.285752+0000","last_change":"2024-09-21T03:33:40.305801+0000","last_active":"2024-09-21T03:33:41.285752+0000","last_peered":"2024-09-21T03:33:41.285752+0000","last_clean":"2024-09-21T03:33:41.285752+0000","last_became_active":"2024-09-21T03:33:40.305316+0000","last_became_peered":"2024-09-21T03:33:40.305316+0000","last_unstale":"2024-09-21T03:33:41.285752+0000","last_undegraded":"2024-09-21T03:33:41.285752+0000","last_fullsized":"2024-09-21T03:33:41.285752+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_clean_scrub_stamp":"2024-09-21T03:33:39.262231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:10:29.758718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,7],"acting":[6,0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.284256+0000","last_change":"2024-09-21T03:33:36.264028+0000","last_active":"2024-09-21T03:33:41.284256+0000","last_peered":"2024-09-21T03:33:41.284256+0000","last_clean":"2024-09-21T03:33:41.284256+0000","last_became_active":"2024-09-21T03:33:36.263725+0000","last_became_peered":"2024-09-21T03:33:36.263725+0000","last_unstale":"2024-09-21T03:33:41.284256+0000","last_undegraded":"2024-09-21T03:33:41.284256+0000","last_fullsized":"2024-09-21T03:33:41.284256+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:48:55.084533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.920078+0000","last_change":"2024-09-21T03:33:35.260963+0000","last_active":"2024-09-21T03:33:42.920078+0000","last_peered":"2024-09-21T03:33:42.920078+0000","last_clean":"2024-09-21T03:33:42.920078+0000","last_became_active":"2024-09-21T03:33:35.257593+0000","last_became_peered":"2024-09-21T03:33:35.257593+0000","last_unstale":"2024-09-21T03:33:42.920078+0000","last_undegraded":"2024-09-21T03:33:42.920078+0000","last_fullsized":"2024-09-21T03:33:42.920078+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_clean_scrub_stamp":"2024-09-21T03:33:34.220881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:35:18.509895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.920658+0000","last_change":"2024-09-21T03:33:40.298927+0000","last_active":"2024-09-21T03:33:42.920658+0000","last_peered":"2024-09-21T03:33:42.920658+0000","last_clean":"2024-09-21T03:33:42.920658+0000","last_became_active":"2024-09-21T03:33:40.298272+0000","last_became_peered":"2024-09-21T03:33:40.298272+0000","last_unstale":"2024-09-21T03:33:42.920658+0000","last_undegraded":"2024-09-21T03:33:42.920658+0000","last_fullsized":"2024-09-21T03:33:42.920658+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_clean_scrub_stamp":"2024-09-21T03:33:39.262231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:44:52.684204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6,1],"acting":[5,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.284686+0000","last_change":"2024-09-21T03:33:36.263187+0000","last_active":"2024-09-21T03:33:41.284686+0000","last_peered":"2024-09-21T03:33:41.284686+0000","last_clean":"2024-09-21T03:33:41.284686+0000","last_became_active":"2024-09-21T03:33:36.262852+0000","last_became_peered":"2024-09-21T03:33:36.262852+0000","last_unstale":"2024-09-21T03:33:41.284686+0000","last_undegraded":"2024-09-21T03:33:41.284686+0000","last_fullsized":"2024-09-21T03:33:41.284686+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:28:30.977653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.921266+0000","last_change":"2024-09-21T03:33:35.249580+0000","last_active":"2024-09-21T03:33:42.921266+0000","last_peered":"2024-09-21T03:33:42.921266+0000","last_clean":"2024-09-21T03:33:42.921266+0000","last_became_active":"2024-09-21T03:33:35.248574+0000","last_became_peered":"2024-09-21T03:33:35.248574+0000","last_unstale":"2024-09-21T03:33:42.921266+0000","last_undegraded":"2024-09-21T03:33:42.921266+0000","last_fullsized":"2024-09-21T03:33:42.921266+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_clean_scrub_stamp":"2024-09-21T03:33:34.220881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:19:25.996685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":119,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.919871+0000","last_change":"2024-09-21T03:33:25.319776+0000","last_active":"2024-09-21T03:33:42.919871+0000","last_peered":"2024-09-21T03:33:42.919871+0000","last_clean":"2024-09-21T03:33:42.919871+0000","last_became_active":"2024-09-21T03:33:25.318399+0000","last_became_peered":"2024-09-21T03:33:25.318399+0000","last_unstale":"2024-09-21T03:33:42.919871+0000","last_undegraded":"2024-09-21T03:33:42.919871+0000","last_fullsized":"2024-09-21T03:33:42.919871+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:23.016672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:23.016672+0000","last_clean_scrub_stamp":"2024-09-21T03:33:23.016672+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:32:35.906114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.921733+0000","last_change":"2024-09-21T03:33:40.304303+0000","last_active":"2024-09-21T03:33:42.921733+0000","last_peered":"2024-09-21T03:33:42.921733+0000","last_clean":"2024-09-21T03:33:42.921733+0000","last_became_active":"2024-09-21T03:33:40.303880+0000","last_became_peered":"2024-09-21T03:33:40.303880+0000","last_unstale":"2024-09-21T03:33:42.921733+0000","last_undegraded":"2024-09-21T03:33:42.921733+0000","last_fullsized":"2024-09-21T03:33:42.921733+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_clean_scrub_stamp":"2024-09-21T03:33:39.262231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:22:59.076888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2,0],"acting":[3,1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.921586+0000","last_change":"2024-09-21T03:33:36.256735+0000","last_active":"2024-09-21T03:33:42.921586+0000","last_peered":"2024-09-21T03:33:42.921586+0000","last_clean":"2024-09-21T03:33:42.921586+0000","last_became_active":"2024-09-21T03:33:36.256296+0000","last_became_peered":"2024-09-21T03:33:36.256296+0000","last_unstale":"2024-09-21T03:33:42.921586+0000","last_undegraded":"2024-09-21T03:33:42.921586+0000","last_fullsized":"2024-09-21T03:33:42.921586+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:20:58.478604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.285704+0000","last_change":"2024-09-21T03:33:35.263469+0000","last_active":"2024-09-21T03:33:41.285704+0000","last_peered":"2024-09-21T03:33:41.285704+0000","last_clean":"2024-09-21T03:33:41.285704+0000","last_became_active":"2024-09-21T03:33:35.263119+0000","last_became_peered":"2024-09-21T03:33:35.263119+0000","last_unstale":"2024-09-21T03:33:41.285704+0000","last_undegraded":"2024-09-21T03:33:41.285704+0000","last_fullsized":"2024-09-21T03:33:41.285704+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_clean_scrub_stamp":"2024-09-21T03:33:34.220881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:54:40.741741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.286176+0000","last_change":"2024-09-21T03:33:40.304199+0000","last_active":"2024-09-21T03:33:41.286176+0000","last_peered":"2024-09-21T03:33:41.286176+0000","last_clean":"2024-09-21T03:33:41.286176+0000","last_became_active":"2024-09-21T03:33:40.303770+0000","last_became_peered":"2024-09-21T03:33:40.303770+0000","last_unstale":"2024-09-21T03:33:41.286176+0000","last_undegraded":"2024-09-21T03:33:41.286176+0000","last_fullsized":"2024-09-21T03:33:41.286176+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_clean_scrub_stamp":"2024-09-21T03:33:39.262231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:41:52.568236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7,4],"acting":[2,3,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.298815+0000","last_change":"2024-09-21T03:33:36.269892+0000","last_active":"2024-09-21T03:33:41.298815+0000","last_peered":"2024-09-21T03:33:41.298815+0000","last_clean":"2024-09-21T03:33:41.298815+0000","last_became_active":"2024-09-21T03:33:36.269521+0000","last_became_peered":"2024-09-21T03:33:36.269521+0000","last_unstale":"2024-09-21T03:33:41.298815+0000","last_undegraded":"2024-09-21T03:33:41.298815+0000","last_fullsized":"2024-09-21T03:33:41.298815+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:43:04.668863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.920621+0000","last_change":"2024-09-21T03:33:35.256681+0000","last_active":"2024-09-21T03:33:42.920621+0000","last_peered":"2024-09-21T03:33:42.920621+0000","last_clean":"2024-09-21T03:33:42.920621+0000","last_became_active":"2024-09-21T03:33:35.256325+0000","last_became_peered":"2024-09-21T03:33:35.256325+0000","last_unstale":"2024-09-21T03:33:42.920621+0000","last_undegraded":"2024-09-21T03:33:42.920621+0000","last_fullsized":"2024-09-21T03:33:42.920621+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_clean_scrub_stamp":"2024-09-21T03:33:34.220881+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:49:36.514638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.920396+0000","last_change":"2024-09-21T03:33:40.298797+0000","last_active":"2024-09-21T03:33:42.920396+0000","last_peered":"2024-09-21T03:33:42.920396+0000","last_clean":"2024-09-21T03:33:42.920396+0000","last_became_active":"2024-09-21T03:33:40.298254+0000","last_became_peered":"2024-09-21T03:33:40.298254+0000","last_unstale":"2024-09-21T03:33:42.920396+0000","last_undegraded":"2024-09-21T03:33:42.920396+0000","last_fullsized":"2024-09-21T03:33:42.920396+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_clean_scrub_stamp":"2024-09-21T03:33:39.262231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:33:49.804176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1,3],"acting":[5,4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.299252+0000","last_change":"2024-09-21T03:33:36.274252+0000","last_active":"2024-09-21T03:33:41.299252+0000","last_peered":"2024-09-21T03:33:41.299252+0000","last_clean":"2024-09-21T03:33:41.299252+0000","last_became_active":"2024-09-21T03:33:36.273890+0000","last_became_peered":"2024-09-21T03:33:36.273890+0000","last_unstale":"2024-09-21T03:33:41.299252+0000","last_undegraded":"2024-09-21T03:33:41.299252+0000","last_fullsized":"2024-09-21T03:33:41.299252+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:55:43.689119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.284896+0000","last_change":"2024-09-21T03:33:35.255320+0000","last_active":"2024-09-21T03:33:41.284896+0000","last_peered":"2024-09-21T03:33:41.284896+0000","last_clean":"2024-09-21T03:33:41.284896+0000","last_became_active":"2024-09-21T03:33:35.254912+0000","last_became_peered":"2024-09-21T03:33:35.254912+0000","last_unstale":"2024-09-21T03:33:41.284896+0000","last_undegraded":"2024-09-21T03:33:41.284896+0000","last_fullsized":"2024-09-21T03:33:41.284896+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_clean_scrub_stamp":"2024-09-21T03:33:34.220881+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:55:02.392036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.919867+0000","last_change":"2024-09-21T03:33:40.304053+0000","last_active":"2024-09-21T03:33:42.919867+0000","last_peered":"2024-09-21T03:33:42.919867+0000","last_clean":"2024-09-21T03:33:42.919867+0000","last_became_active":"2024-09-21T03:33:40.303606+0000","last_became_peered":"2024-09-21T03:33:40.303606+0000","last_unstale":"2024-09-21T03:33:42.919867+0000","last_undegraded":"2024-09-21T03:33:42.919867+0000","last_fullsized":"2024-09-21T03:33:42.919867+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_clean_scrub_stamp":"2024-09-21T03:33:39.262231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:43:29.231059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.286527+0000","last_change":"2024-09-21T03:33:36.270385+0000","last_active":"2024-09-21T03:33:41.286527+0000","last_peered":"2024-09-21T03:33:41.286527+0000","last_clean":"2024-09-21T03:33:41.286527+0000","last_became_active":"2024-09-21T03:33:36.267107+0000","last_became_peered":"2024-09-21T03:33:36.267107+0000","last_unstale":"2024-09-21T03:33:41.286527+0000","last_undegraded":"2024-09-21T03:33:41.286527+0000","last_fullsized":"2024-09-21T03:33:41.286527+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:48:55.084533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.299383+0000","last_change":"2024-09-21T03:33:35.263063+0000","last_active":"2024-09-21T03:33:41.299383+0000","last_peered":"2024-09-21T03:33:41.299383+0000","last_clean":"2024-09-21T03:33:41.299383+0000","last_became_active":"2024-09-21T03:33:35.262324+0000","last_became_peered":"2024-09-21T03:33:35.262324+0000","last_unstale":"2024-09-21T03:33:41.299383+0000","last_undegraded":"2024-09-21T03:33:41.299383+0000","last_fullsized":"2024-09-21T03:33:41.299383+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_clean_scrub_stamp":"2024-09-21T03:33:34.220881+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:35:18.509895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.920221+0000","last_change":"2024-09-21T03:33:40.298024+0000","last_active":"2024-09-21T03:33:42.920221+0000","last_peered":"2024-09-21T03:33:42.920221+0000","last_clean":"2024-09-21T03:33:42.920221+0000","last_became_active":"2024-09-21T03:33:40.297580+0000","last_became_peered":"2024-09-21T03:33:40.297580+0000","last_unstale":"2024-09-21T03:33:42.920221+0000","last_undegraded":"2024-09-21T03:33:42.920221+0000","last_fullsized":"2024-09-21T03:33:42.920221+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_clean_scrub_stamp":"2024-09-21T03:33:39.262231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:34:07.830134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,2],"acting":[1,5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.920389+0000","last_change":"2024-09-21T03:33:36.256567+0000","last_active":"2024-09-21T03:33:42.920389+0000","last_peered":"2024-09-21T03:33:42.920389+0000","last_clean":"2024-09-21T03:33:42.920389+0000","last_became_active":"2024-09-21T03:33:36.252598+0000","last_became_peered":"2024-09-21T03:33:36.252598+0000","last_unstale":"2024-09-21T03:33:42.920389+0000","last_undegraded":"2024-09-21T03:33:42.920389+0000","last_fullsized":"2024-09-21T03:33:42.920389+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:53:59.995385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":31,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.283961+0000","last_change":"2024-09-21T03:33:35.246577+0000","last_active":"2024-09-21T03:33:41.283961+0000","last_peered":"2024-09-21T03:33:41.283961+0000","last_clean":"2024-09-21T03:33:41.283961+0000","last_became_active":"2024-09-21T03:33:35.245954+0000","last_became_peered":"2024-09-21T03:33:35.245954+0000","last_unstale":"2024-09-21T03:33:41.283961+0000","last_undegraded":"2024-09-21T03:33:41.283961+0000","last_fullsized":"2024-09-21T03:33:41.283961+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_clean_scrub_stamp":"2024-09-21T03:33:34.220881+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:59:06.265387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.920916+0000","last_change":"2024-09-21T03:33:40.299966+0000","last_active":"2024-09-21T03:33:42.920916+0000","last_peered":"2024-09-21T03:33:42.920916+0000","last_clean":"2024-09-21T03:33:42.920916+0000","last_became_active":"2024-09-21T03:33:40.298455+0000","last_became_peered":"2024-09-21T03:33:40.298455+0000","last_unstale":"2024-09-21T03:33:42.920916+0000","last_undegraded":"2024-09-21T03:33:42.920916+0000","last_fullsized":"2024-09-21T03:33:42.920916+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_clean_scrub_stamp":"2024-09-21T03:33:39.262231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:59:07.141809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1,6],"acting":[5,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.919715+0000","last_change":"2024-09-21T03:33:36.261762+0000","last_active":"2024-09-21T03:33:42.919715+0000","last_peered":"2024-09-21T03:33:42.919715+0000","last_clean":"2024-09-21T03:33:42.919715+0000","last_became_active":"2024-09-21T03:33:36.261379+0000","last_became_peered":"2024-09-21T03:33:36.261379+0000","last_unstale":"2024-09-21T03:33:42.919715+0000","last_undegraded":"2024-09-21T03:33:42.919715+0000","last_fullsized":"2024-09-21T03:33:42.919715+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:35:07.883056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.298133+0000","last_change":"2024-09-21T03:33:35.264258+0000","last_active":"2024-09-21T03:33:41.298133+0000","last_peered":"2024-09-21T03:33:41.298133+0000","last_clean":"2024-09-21T03:33:41.298133+0000","last_became_active":"2024-09-21T03:33:35.262742+0000","last_became_peered":"2024-09-21T03:33:35.262742+0000","last_unstale":"2024-09-21T03:33:41.298133+0000","last_undegraded":"2024-09-21T03:33:41.298133+0000","last_fullsized":"2024-09-21T03:33:41.298133+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_clean_scrub_stamp":"2024-09-21T03:33:34.220881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:42:59.930617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.920862+0000","last_change":"2024-09-21T03:33:40.298402+0000","last_active":"2024-09-21T03:33:42.920862+0000","last_peered":"2024-09-21T03:33:42.920862+0000","last_clean":"2024-09-21T03:33:42.920862+0000","last_became_active":"2024-09-21T03:33:40.297795+0000","last_became_peered":"2024-09-21T03:33:40.297795+0000","last_unstale":"2024-09-21T03:33:42.920862+0000","last_undegraded":"2024-09-21T03:33:42.920862+0000","last_fullsized":"2024-09-21T03:33:42.920862+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_clean_scrub_stamp":"2024-09-21T03:33:39.262231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:18:24.844623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6,4],"acting":[5,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.920969+0000","last_change":"2024-09-21T03:33:36.269233+0000","last_active":"2024-09-21T03:33:42.920969+0000","last_peered":"2024-09-21T03:33:42.920969+0000","last_clean":"2024-09-21T03:33:42.920969+0000","last_became_active":"2024-09-21T03:33:36.268680+0000","last_became_peered":"2024-09-21T03:33:36.268680+0000","last_unstale":"2024-09-21T03:33:42.920969+0000","last_undegraded":"2024-09-21T03:33:42.920969+0000","last_fullsized":"2024-09-21T03:33:42.920969+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:55:43.689119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.921162+0000","last_change":"2024-09-21T03:33:35.251566+0000","last_active":"2024-09-21T03:33:42.921162+0000","last_peered":"2024-09-21T03:33:42.921162+0000","last_clean":"2024-09-21T03:33:42.921162+0000","last_became_active":"2024-09-21T03:33:35.251252+0000","last_became_peered":"2024-09-21T03:33:35.251252+0000","last_unstale":"2024-09-21T03:33:42.921162+0000","last_undegraded":"2024-09-21T03:33:42.921162+0000","last_fullsized":"2024-09-21T03:33:42.921162+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_clean_scrub_stamp":"2024-09-21T03:33:34.220881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:39:01.410384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.285366+0000","last_change":"2024-09-21T03:33:40.300353+0000","last_active":"2024-09-21T03:33:41.285366+0000","last_peered":"2024-09-21T03:33:41.285366+0000","last_clean":"2024-09-21T03:33:41.285366+0000","last_became_active":"2024-09-21T03:33:40.299481+0000","last_became_peered":"2024-09-21T03:33:40.299481+0000","last_unstale":"2024-09-21T03:33:41.285366+0000","last_undegraded":"2024-09-21T03:33:41.285366+0000","last_fullsized":"2024-09-21T03:33:41.285366+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_clean_scrub_stamp":"2024-09-21T03:33:39.262231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:51:21.940594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0,7],"acting":[6,4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.284147+0000","last_change":"2024-09-21T03:33:36.265957+0000","last_active":"2024-09-21T03:33:41.284147+0000","last_peered":"2024-09-21T03:33:41.284147+0000","last_clean":"2024-09-21T03:33:41.284147+0000","last_became_active":"2024-09-21T03:33:36.265578+0000","last_became_peered":"2024-09-21T03:33:36.265578+0000","last_unstale":"2024-09-21T03:33:41.284147+0000","last_undegraded":"2024-09-21T03:33:41.284147+0000","last_fullsized":"2024-09-21T03:33:41.284147+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:06:36.428086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.754862+0000","last_change":"2024-09-21T03:33:35.263298+0000","last_active":"2024-09-21T03:33:41.754862+0000","last_peered":"2024-09-21T03:33:41.754862+0000","last_clean":"2024-09-21T03:33:41.754862+0000","last_became_active":"2024-09-21T03:33:35.262833+0000","last_became_peered":"2024-09-21T03:33:35.262833+0000","last_unstale":"2024-09-21T03:33:41.754862+0000","last_undegraded":"2024-09-21T03:33:41.754862+0000","last_fullsized":"2024-09-21T03:33:41.754862+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_clean_scrub_stamp":"2024-09-21T03:33:34.220881+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:36:04.049350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.921613+0000","last_change":"2024-09-21T03:33:40.299399+0000","last_active":"2024-09-21T03:33:42.921613+0000","last_peered":"2024-09-21T03:33:42.921613+0000","last_clean":"2024-09-21T03:33:42.921613+0000","last_became_active":"2024-09-21T03:33:40.298855+0000","last_became_peered":"2024-09-21T03:33:40.298855+0000","last_unstale":"2024-09-21T03:33:42.921613+0000","last_undegraded":"2024-09-21T03:33:42.921613+0000","last_fullsized":"2024-09-21T03:33:42.921613+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_clean_scrub_stamp":"2024-09-21T03:33:39.262231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:38:17.611449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5,6],"acting":[3,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.298648+0000","last_change":"2024-09-21T03:33:36.268494+0000","last_active":"2024-09-21T03:33:41.298648+0000","last_peered":"2024-09-21T03:33:41.298648+0000","last_clean":"2024-09-21T03:33:41.298648+0000","last_became_active":"2024-09-21T03:33:36.268126+0000","last_became_peered":"2024-09-21T03:33:36.268126+0000","last_unstale":"2024-09-21T03:33:41.298648+0000","last_undegraded":"2024-09-21T03:33:41.298648+0000","last_fullsized":"2024-09-21T03:33:41.298648+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:00:54.354428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.14","version":"26'9","reported_seq":32,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:47.259231+0000","last_change":"2024-09-21T03:33:35.249689+0000","last_active":"2024-09-21T03:33:47.259231+0000","last_peered":"2024-09-21T03:33:47.259231+0000","last_clean":"2024-09-21T03:33:47.259231+0000","last_became_active":"2024-09-21T03:33:35.249355+0000","last_became_peered":"2024-09-21T03:33:35.249355+0000","last_unstale":"2024-09-21T03:33:47.259231+0000","last_undegraded":"2024-09-21T03:33:47.259231+0000","last_fullsized":"2024-09-21T03:33:47.259231+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_clean_scrub_stamp":"2024-09-21T03:33:34.220881+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:01:23.098006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.284651+0000","last_change":"2024-09-21T03:33:40.295609+0000","last_active":"2024-09-21T03:33:41.284651+0000","last_peered":"2024-09-21T03:33:41.284651+0000","last_clean":"2024-09-21T03:33:41.284651+0000","last_became_active":"2024-09-21T03:33:40.294963+0000","last_became_peered":"2024-09-21T03:33:40.294963+0000","last_unstale":"2024-09-21T03:33:41.284651+0000","last_undegraded":"2024-09-21T03:33:41.284651+0000","last_fullsized":"2024-09-21T03:33:41.284651+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_clean_scrub_stamp":"2024-09-21T03:33:39.262231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:55:47.724630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5,0],"acting":[4,3,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.921108+0000","last_change":"2024-09-21T03:33:36.260379+0000","last_active":"2024-09-21T03:33:42.921108+0000","last_peered":"2024-09-21T03:33:42.921108+0000","last_clean":"2024-09-21T03:33:42.921108+0000","last_became_active":"2024-09-21T03:33:36.260035+0000","last_became_peered":"2024-09-21T03:33:36.260035+0000","last_unstale":"2024-09-21T03:33:42.921108+0000","last_undegraded":"2024-09-21T03:33:42.921108+0000","last_fullsized":"2024-09-21T03:33:42.921108+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:28:03.278038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.755172+0000","last_change":"2024-09-21T03:33:35.249062+0000","last_active":"2024-09-21T03:33:41.755172+0000","last_peered":"2024-09-21T03:33:41.755172+0000","last_clean":"2024-09-21T03:33:41.755172+0000","last_became_active":"2024-09-21T03:33:35.248630+0000","last_became_peered":"2024-09-21T03:33:35.248630+0000","last_unstale":"2024-09-21T03:33:41.755172+0000","last_undegraded":"2024-09-21T03:33:41.755172+0000","last_fullsized":"2024-09-21T03:33:41.755172+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_clean_scrub_stamp":"2024-09-21T03:33:34.220881+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:49:09.367077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.286199+0000","last_change":"2024-09-21T03:33:40.305392+0000","last_active":"2024-09-21T03:33:41.286199+0000","last_peered":"2024-09-21T03:33:41.286199+0000","last_clean":"2024-09-21T03:33:41.286199+0000","last_became_active":"2024-09-21T03:33:40.304899+0000","last_became_peered":"2024-09-21T03:33:40.304899+0000","last_unstale":"2024-09-21T03:33:41.286199+0000","last_undegraded":"2024-09-21T03:33:41.286199+0000","last_fullsized":"2024-09-21T03:33:41.286199+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_clean_scrub_stamp":"2024-09-21T03:33:39.262231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:29:46.742857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1,7],"acting":[2,3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.920967+0000","last_change":"2024-09-21T03:33:36.256188+0000","last_active":"2024-09-21T03:33:42.920967+0000","last_peered":"2024-09-21T03:33:42.920967+0000","last_clean":"2024-09-21T03:33:42.920967+0000","last_became_active":"2024-09-21T03:33:36.254900+0000","last_became_peered":"2024-09-21T03:33:36.254900+0000","last_unstale":"2024-09-21T03:33:42.920967+0000","last_undegraded":"2024-09-21T03:33:42.920967+0000","last_fullsized":"2024-09-21T03:33:42.920967+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:20:54.467912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.299116+0000","last_change":"2024-09-21T03:33:35.259866+0000","last_active":"2024-09-21T03:33:41.299116+0000","last_peered":"2024-09-21T03:33:41.299116+0000","last_clean":"2024-09-21T03:33:41.299116+0000","last_became_active":"2024-09-21T03:33:35.259368+0000","last_became_peered":"2024-09-21T03:33:35.259368+0000","last_unstale":"2024-09-21T03:33:41.299116+0000","last_undegraded":"2024-09-21T03:33:41.299116+0000","last_fullsized":"2024-09-21T03:33:41.299116+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_clean_scrub_stamp":"2024-09-21T03:33:34.220881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:12:06.105531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.754662+0000","last_change":"2024-09-21T03:33:40.304856+0000","last_active":"2024-09-21T03:33:41.754662+0000","last_peered":"2024-09-21T03:33:41.754662+0000","last_clean":"2024-09-21T03:33:41.754662+0000","last_became_active":"2024-09-21T03:33:40.304421+0000","last_became_peered":"2024-09-21T03:33:40.304421+0000","last_unstale":"2024-09-21T03:33:41.754662+0000","last_undegraded":"2024-09-21T03:33:41.754662+0000","last_fullsized":"2024-09-21T03:33:41.754662+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_clean_scrub_stamp":"2024-09-21T03:33:39.262231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:10:13.916812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3,2],"acting":[7,1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.286504+0000","last_change":"2024-09-21T03:33:36.265331+0000","last_active":"2024-09-21T03:33:41.286504+0000","last_peered":"2024-09-21T03:33:41.286504+0000","last_clean":"2024-09-21T03:33:41.286504+0000","last_became_active":"2024-09-21T03:33:36.264928+0000","last_became_peered":"2024-09-21T03:33:36.264928+0000","last_unstale":"2024-09-21T03:33:41.286504+0000","last_undegraded":"2024-09-21T03:33:41.286504+0000","last_fullsized":"2024-09-21T03:33:41.286504+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:56:25.878234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.920745+0000","last_change":"2024-09-21T03:33:35.249377+0000","last_active":"2024-09-21T03:33:42.920745+0000","last_peered":"2024-09-21T03:33:42.920745+0000","last_clean":"2024-09-21T03:33:42.920745+0000","last_became_active":"2024-09-21T03:33:35.249027+0000","last_became_peered":"2024-09-21T03:33:35.249027+0000","last_unstale":"2024-09-21T03:33:42.920745+0000","last_undegraded":"2024-09-21T03:33:42.920745+0000","last_fullsized":"2024-09-21T03:33:42.920745+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_clean_scrub_stamp":"2024-09-21T03:33:34.220881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:48:36.003541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.284509+0000","last_change":"2024-09-21T03:33:40.294482+0000","last_active":"2024-09-21T03:33:41.284509+0000","last_peered":"2024-09-21T03:33:41.284509+0000","last_clean":"2024-09-21T03:33:41.284509+0000","last_became_active":"2024-09-21T03:33:40.293996+0000","last_became_peered":"2024-09-21T03:33:40.293996+0000","last_unstale":"2024-09-21T03:33:41.284509+0000","last_undegraded":"2024-09-21T03:33:41.284509+0000","last_fullsized":"2024-09-21T03:33:41.284509+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_clean_scrub_stamp":"2024-09-21T03:33:39.262231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:43:08.704373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,3,7],"acting":[4,0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.283820+0000","last_change":"2024-09-21T03:33:40.300111+0000","last_active":"2024-09-21T03:33:41.283820+0000","last_peered":"2024-09-21T03:33:41.283820+0000","last_clean":"2024-09-21T03:33:41.283820+0000","last_became_active":"2024-09-21T03:33:40.299645+0000","last_became_peered":"2024-09-21T03:33:40.299645+0000","last_unstale":"2024-09-21T03:33:41.283820+0000","last_undegraded":"2024-09-21T03:33:41.283820+0000","last_fullsized":"2024-09-21T03:33:41.283820+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_clean_scrub_stamp":"2024-09-21T03:33:39.262231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:10:29.758718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,5,3],"acting":[4,1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.284548+0000","last_change":"2024-09-21T03:33:36.248593+0000","last_active":"2024-09-21T03:33:41.284548+0000","last_peered":"2024-09-21T03:33:41.284548+0000","last_clean":"2024-09-21T03:33:41.284548+0000","last_became_active":"2024-09-21T03:33:36.248274+0000","last_became_peered":"2024-09-21T03:33:36.248274+0000","last_unstale":"2024-09-21T03:33:41.284548+0000","last_undegraded":"2024-09-21T03:33:41.284548+0000","last_fullsized":"2024-09-21T03:33:41.284548+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:36:57.787555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.920023+0000","last_change":"2024-09-21T03:33:35.261733+0000","last_active":"2024-09-21T03:33:42.920023+0000","last_peered":"2024-09-21T03:33:42.920023+0000","last_clean":"2024-09-21T03:33:42.920023+0000","last_became_active":"2024-09-21T03:33:35.261427+0000","last_became_peered":"2024-09-21T03:33:35.261427+0000","last_unstale":"2024-09-21T03:33:42.920023+0000","last_undegraded":"2024-09-21T03:33:42.920023+0000","last_fullsized":"2024-09-21T03:33:42.920023+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_clean_scrub_stamp":"2024-09-21T03:33:34.220881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:40:22.752363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.921462+0000","last_change":"2024-09-21T03:33:40.306313+0000","last_active":"2024-09-21T03:33:42.921462+0000","last_peered":"2024-09-21T03:33:42.921462+0000","last_clean":"2024-09-21T03:33:42.921462+0000","last_became_active":"2024-09-21T03:33:40.305650+0000","last_became_peered":"2024-09-21T03:33:40.305650+0000","last_unstale":"2024-09-21T03:33:42.921462+0000","last_undegraded":"2024-09-21T03:33:42.921462+0000","last_fullsized":"2024-09-21T03:33:42.921462+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_clean_scrub_stamp":"2024-09-21T03:33:39.262231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:45:58.941848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5,1],"acting":[3,2,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.920101+0000","last_change":"2024-09-21T03:33:36.255929+0000","last_active":"2024-09-21T03:33:42.920101+0000","last_peered":"2024-09-21T03:33:42.920101+0000","last_clean":"2024-09-21T03:33:42.920101+0000","last_became_active":"2024-09-21T03:33:36.255538+0000","last_became_peered":"2024-09-21T03:33:36.255538+0000","last_unstale":"2024-09-21T03:33:42.920101+0000","last_undegraded":"2024-09-21T03:33:42.920101+0000","last_fullsized":"2024-09-21T03:33:42.920101+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:33:10.543079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.284381+0000","last_change":"2024-09-21T03:33:35.264572+0000","last_active":"2024-09-21T03:33:41.284381+0000","last_peered":"2024-09-21T03:33:41.284381+0000","last_clean":"2024-09-21T03:33:41.284381+0000","last_became_active":"2024-09-21T03:33:35.264251+0000","last_became_peered":"2024-09-21T03:33:35.264251+0000","last_unstale":"2024-09-21T03:33:41.284381+0000","last_undegraded":"2024-09-21T03:33:41.284381+0000","last_fullsized":"2024-09-21T03:33:41.284381+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_clean_scrub_stamp":"2024-09-21T03:33:34.220881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:16:25.260763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.921171+0000","last_change":"2024-09-21T03:33:35.258225+0000","last_active":"2024-09-21T03:33:42.921171+0000","last_peered":"2024-09-21T03:33:42.921171+0000","last_clean":"2024-09-21T03:33:42.921171+0000","last_became_active":"2024-09-21T03:33:35.257903+0000","last_became_peered":"2024-09-21T03:33:35.257903+0000","last_unstale":"2024-09-21T03:33:42.921171+0000","last_undegraded":"2024-09-21T03:33:42.921171+0000","last_fullsized":"2024-09-21T03:33:42.921171+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_clean_scrub_stamp":"2024-09-21T03:33:34.220881+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:12:06.105531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.283808+0000","last_change":"2024-09-21T03:33:36.267650+0000","last_active":"2024-09-21T03:33:41.283808+0000","last_peered":"2024-09-21T03:33:41.283808+0000","last_clean":"2024-09-21T03:33:41.283808+0000","last_became_active":"2024-09-21T03:33:36.267315+0000","last_became_peered":"2024-09-21T03:33:36.267315+0000","last_unstale":"2024-09-21T03:33:41.283808+0000","last_undegraded":"2024-09-21T03:33:41.283808+0000","last_fullsized":"2024-09-21T03:33:41.283808+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:21:56.947372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.285062+0000","last_change":"2024-09-21T03:33:40.290894+0000","last_active":"2024-09-21T03:33:41.285062+0000","last_peered":"2024-09-21T03:33:41.285062+0000","last_clean":"2024-09-21T03:33:41.285062+0000","last_became_active":"2024-09-21T03:33:40.290424+0000","last_became_peered":"2024-09-21T03:33:40.290424+0000","last_unstale":"2024-09-21T03:33:41.285062+0000","last_undegraded":"2024-09-21T03:33:41.285062+0000","last_fullsized":"2024-09-21T03:33:41.285062+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_clean_scrub_stamp":"2024-09-21T03:33:39.262231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:56:50.726521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,7],"acting":[4,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.920464+0000","last_change":"2024-09-21T03:33:35.251276+0000","last_active":"2024-09-21T03:33:42.920464+0000","last_peered":"2024-09-21T03:33:42.920464+0000","last_clean":"2024-09-21T03:33:42.920464+0000","last_became_active":"2024-09-21T03:33:35.250758+0000","last_became_peered":"2024-09-21T03:33:35.250758+0000","last_unstale":"2024-09-21T03:33:42.920464+0000","last_undegraded":"2024-09-21T03:33:42.920464+0000","last_fullsized":"2024-09-21T03:33:42.920464+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_clean_scrub_stamp":"2024-09-21T03:33:34.220881+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:03:09.683999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.920393+0000","last_change":"2024-09-21T03:33:36.261907+0000","last_active":"2024-09-21T03:33:42.920393+0000","last_peered":"2024-09-21T03:33:42.920393+0000","last_clean":"2024-09-21T03:33:42.920393+0000","last_became_active":"2024-09-21T03:33:36.261483+0000","last_became_peered":"2024-09-21T03:33:36.261483+0000","last_unstale":"2024-09-21T03:33:42.920393+0000","last_undegraded":"2024-09-21T03:33:42.920393+0000","last_fullsized":"2024-09-21T03:33:42.920393+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:08:27.276982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.755140+0000","last_change":"2024-09-21T03:33:40.305034+0000","last_active":"2024-09-21T03:33:41.755140+0000","last_peered":"2024-09-21T03:33:41.755140+0000","last_clean":"2024-09-21T03:33:41.755140+0000","last_became_active":"2024-09-21T03:33:40.304555+0000","last_became_peered":"2024-09-21T03:33:40.304555+0000","last_unstale":"2024-09-21T03:33:41.755140+0000","last_undegraded":"2024-09-21T03:33:41.755140+0000","last_fullsized":"2024-09-21T03:33:41.755140+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_clean_scrub_stamp":"2024-09-21T03:33:39.262231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:09:41.615406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1,0],"acting":[7,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.285867+0000","last_change":"2024-09-21T03:33:36.258191+0000","last_active":"2024-09-21T03:33:41.285867+0000","last_peered":"2024-09-21T03:33:41.285867+0000","last_clean":"2024-09-21T03:33:41.285867+0000","last_became_active":"2024-09-21T03:33:36.257866+0000","last_became_peered":"2024-09-21T03:33:36.257866+0000","last_unstale":"2024-09-21T03:33:41.285867+0000","last_undegraded":"2024-09-21T03:33:41.285867+0000","last_fullsized":"2024-09-21T03:33:41.285867+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:07:30.043350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.754532+0000","last_change":"2024-09-21T03:33:36.248020+0000","last_active":"2024-09-21T03:33:41.754532+0000","last_peered":"2024-09-21T03:33:41.754532+0000","last_clean":"2024-09-21T03:33:41.754532+0000","last_became_active":"2024-09-21T03:33:36.247696+0000","last_became_peered":"2024-09-21T03:33:36.247696+0000","last_unstale":"2024-09-21T03:33:41.754532+0000","last_undegraded":"2024-09-21T03:33:41.754532+0000","last_fullsized":"2024-09-21T03:33:41.754532+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:11:14.232403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.920927+0000","last_change":"2024-09-21T03:33:36.264435+0000","last_active":"2024-09-21T03:33:42.920927+0000","last_peered":"2024-09-21T03:33:42.920927+0000","last_clean":"2024-09-21T03:33:42.920927+0000","last_became_active":"2024-09-21T03:33:36.263834+0000","last_became_peered":"2024-09-21T03:33:36.263834+0000","last_unstale":"2024-09-21T03:33:42.920927+0000","last_undegraded":"2024-09-21T03:33:42.920927+0000","last_fullsized":"2024-09-21T03:33:42.920927+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:00:54.354428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.920341+0000","last_change":"2024-09-21T03:33:36.256675+0000","last_active":"2024-09-21T03:33:42.920341+0000","last_peered":"2024-09-21T03:33:42.920341+0000","last_clean":"2024-09-21T03:33:42.920341+0000","last_became_active":"2024-09-21T03:33:36.256184+0000","last_became_peered":"2024-09-21T03:33:36.256184+0000","last_unstale":"2024-09-21T03:33:42.920341+0000","last_undegraded":"2024-09-21T03:33:42.920341+0000","last_fullsized":"2024-09-21T03:33:42.920341+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:10:44.754143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.285553+0000","last_change":"2024-09-21T03:33:36.271076+0000","last_active":"2024-09-21T03:33:41.285553+0000","last_peered":"2024-09-21T03:33:41.285553+0000","last_clean":"2024-09-21T03:33:41.285553+0000","last_became_active":"2024-09-21T03:33:36.270707+0000","last_became_peered":"2024-09-21T03:33:36.270707+0000","last_unstale":"2024-09-21T03:33:41.285553+0000","last_undegraded":"2024-09-21T03:33:41.285553+0000","last_fullsized":"2024-09-21T03:33:41.285553+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:07:30.043350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.285361+0000","last_change":"2024-09-21T03:33:36.261405+0000","last_active":"2024-09-21T03:33:41.285361+0000","last_peered":"2024-09-21T03:33:41.285361+0000","last_clean":"2024-09-21T03:33:41.285361+0000","last_became_active":"2024-09-21T03:33:36.261048+0000","last_became_peered":"2024-09-21T03:33:36.261048+0000","last_unstale":"2024-09-21T03:33:41.285361+0000","last_undegraded":"2024-09-21T03:33:41.285361+0000","last_fullsized":"2024-09-21T03:33:41.285361+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:56:25.878234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.286335+0000","last_change":"2024-09-21T03:33:36.272514+0000","last_active":"2024-09-21T03:33:41.286335+0000","last_peered":"2024-09-21T03:33:41.286335+0000","last_clean":"2024-09-21T03:33:41.286335+0000","last_became_active":"2024-09-21T03:33:36.272186+0000","last_became_peered":"2024-09-21T03:33:36.272186+0000","last_unstale":"2024-09-21T03:33:41.286335+0000","last_undegraded":"2024-09-21T03:33:41.286335+0000","last_fullsized":"2024-09-21T03:33:41.286335+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:39:02.416224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.920780+0000","last_change":"2024-09-21T03:33:36.255650+0000","last_active":"2024-09-21T03:33:42.920780+0000","last_peered":"2024-09-21T03:33:42.920780+0000","last_clean":"2024-09-21T03:33:42.920780+0000","last_became_active":"2024-09-21T03:33:36.253524+0000","last_became_peered":"2024-09-21T03:33:36.253524+0000","last_unstale":"2024-09-21T03:33:42.920780+0000","last_undegraded":"2024-09-21T03:33:42.920780+0000","last_fullsized":"2024-09-21T03:33:42.920780+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:51:37.424690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.919971+0000","last_change":"2024-09-21T03:33:36.262263+0000","last_active":"2024-09-21T03:33:42.919971+0000","last_peered":"2024-09-21T03:33:42.919971+0000","last_clean":"2024-09-21T03:33:42.919971+0000","last_became_active":"2024-09-21T03:33:36.261962+0000","last_became_peered":"2024-09-21T03:33:36.261962+0000","last_unstale":"2024-09-21T03:33:42.919971+0000","last_undegraded":"2024-09-21T03:33:42.919971+0000","last_fullsized":"2024-09-21T03:33:42.919971+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:10:25.723208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.920014+0000","last_change":"2024-09-21T03:33:36.261266+0000","last_active":"2024-09-21T03:33:42.920014+0000","last_peered":"2024-09-21T03:33:42.920014+0000","last_clean":"2024-09-21T03:33:42.920014+0000","last_became_active":"2024-09-21T03:33:36.260938+0000","last_became_peered":"2024-09-21T03:33:36.260938+0000","last_unstale":"2024-09-21T03:33:42.920014+0000","last_undegraded":"2024-09-21T03:33:42.920014+0000","last_fullsized":"2024-09-21T03:33:42.920014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:54:42.460406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.920279+0000","last_change":"2024-09-21T03:33:36.268419+0000","last_active":"2024-09-21T03:33:42.920279+0000","last_peered":"2024-09-21T03:33:42.920279+0000","last_clean":"2024-09-21T03:33:42.920279+0000","last_became_active":"2024-09-21T03:33:36.268118+0000","last_became_peered":"2024-09-21T03:33:36.268118+0000","last_unstale":"2024-09-21T03:33:42.920279+0000","last_undegraded":"2024-09-21T03:33:42.920279+0000","last_fullsized":"2024-09-21T03:33:42.920279+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:15:19.693603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.283681+0000","last_change":"2024-09-21T03:33:36.265448+0000","last_active":"2024-09-21T03:33:41.283681+0000","last_peered":"2024-09-21T03:33:41.283681+0000","last_clean":"2024-09-21T03:33:41.283681+0000","last_became_active":"2024-09-21T03:33:36.263265+0000","last_became_peered":"2024-09-21T03:33:36.263265+0000","last_unstale":"2024-09-21T03:33:41.283681+0000","last_undegraded":"2024-09-21T03:33:41.283681+0000","last_fullsized":"2024-09-21T03:33:41.283681+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:06:36.428086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.298767+0000","last_change":"2024-09-21T03:33:36.274784+0000","last_active":"2024-09-21T03:33:41.298767+0000","last_peered":"2024-09-21T03:33:41.298767+0000","last_clean":"2024-09-21T03:33:41.298767+0000","last_became_active":"2024-09-21T03:33:36.273608+0000","last_became_peered":"2024-09-21T03:33:36.273608+0000","last_unstale":"2024-09-21T03:33:41.298767+0000","last_undegraded":"2024-09-21T03:33:41.298767+0000","last_fullsized":"2024-09-21T03:33:41.298767+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:43:31.237575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.921685+0000","last_change":"2024-09-21T03:33:36.253771+0000","last_active":"2024-09-21T03:33:42.921685+0000","last_peered":"2024-09-21T03:33:42.921685+0000","last_clean":"2024-09-21T03:33:42.921685+0000","last_became_active":"2024-09-21T03:33:36.253068+0000","last_became_peered":"2024-09-21T03:33:36.253068+0000","last_unstale":"2024-09-21T03:33:42.921685+0000","last_undegraded":"2024-09-21T03:33:42.921685+0000","last_fullsized":"2024-09-21T03:33:42.921685+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:33:50.625863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.755254+0000","last_change":"2024-09-21T03:33:36.255996+0000","last_active":"2024-09-21T03:33:41.755254+0000","last_peered":"2024-09-21T03:33:41.755254+0000","last_clean":"2024-09-21T03:33:41.755254+0000","last_became_active":"2024-09-21T03:33:36.255696+0000","last_became_peered":"2024-09-21T03:33:36.255696+0000","last_unstale":"2024-09-21T03:33:41.755254+0000","last_undegraded":"2024-09-21T03:33:41.755254+0000","last_fullsized":"2024-09-21T03:33:41.755254+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:15:19.693603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.921552+0000","last_change":"2024-09-21T03:33:36.271593+0000","last_active":"2024-09-21T03:33:42.921552+0000","last_peered":"2024-09-21T03:33:42.921552+0000","last_clean":"2024-09-21T03:33:42.921552+0000","last_became_active":"2024-09-21T03:33:36.271244+0000","last_became_peered":"2024-09-21T03:33:36.271244+0000","last_unstale":"2024-09-21T03:33:42.921552+0000","last_undegraded":"2024-09-21T03:33:42.921552+0000","last_fullsized":"2024-09-21T03:33:42.921552+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:14:54.862806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.919854+0000","last_change":"2024-09-21T03:33:36.252117+0000","last_active":"2024-09-21T03:33:42.919854+0000","last_peered":"2024-09-21T03:33:42.919854+0000","last_clean":"2024-09-21T03:33:42.919854+0000","last_became_active":"2024-09-21T03:33:36.251757+0000","last_became_peered":"2024-09-21T03:33:36.251757+0000","last_unstale":"2024-09-21T03:33:42.919854+0000","last_undegraded":"2024-09-21T03:33:42.919854+0000","last_fullsized":"2024-09-21T03:33:42.919854+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:59:02.176819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.919914+0000","last_change":"2024-09-21T03:33:36.257727+0000","last_active":"2024-09-21T03:33:42.919914+0000","last_peered":"2024-09-21T03:33:42.919914+0000","last_clean":"2024-09-21T03:33:42.919914+0000","last_became_active":"2024-09-21T03:33:36.257423+0000","last_became_peered":"2024-09-21T03:33:36.257423+0000","last_unstale":"2024-09-21T03:33:42.919914+0000","last_undegraded":"2024-09-21T03:33:42.919914+0000","last_fullsized":"2024-09-21T03:33:42.919914+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:11:43.225156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.284433+0000","last_change":"2024-09-21T03:33:36.257080+0000","last_active":"2024-09-21T03:33:41.284433+0000","last_peered":"2024-09-21T03:33:41.284433+0000","last_clean":"2024-09-21T03:33:41.284433+0000","last_became_active":"2024-09-21T03:33:36.256763+0000","last_became_peered":"2024-09-21T03:33:36.256763+0000","last_unstale":"2024-09-21T03:33:41.284433+0000","last_undegraded":"2024-09-21T03:33:41.284433+0000","last_fullsized":"2024-09-21T03:33:41.284433+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:36:57.787555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.285409+0000","last_change":"2024-09-21T03:33:36.265166+0000","last_active":"2024-09-21T03:33:41.285409+0000","last_peered":"2024-09-21T03:33:41.285409+0000","last_clean":"2024-09-21T03:33:41.285409+0000","last_became_active":"2024-09-21T03:33:36.264841+0000","last_became_peered":"2024-09-21T03:33:36.264841+0000","last_unstale":"2024-09-21T03:33:41.285409+0000","last_undegraded":"2024-09-21T03:33:41.285409+0000","last_fullsized":"2024-09-21T03:33:41.285409+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:49:00.717844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.298659+0000","last_change":"2024-09-21T03:33:36.274919+0000","last_active":"2024-09-21T03:33:41.298659+0000","last_peered":"2024-09-21T03:33:41.298659+0000","last_clean":"2024-09-21T03:33:41.298659+0000","last_became_active":"2024-09-21T03:33:36.274111+0000","last_became_peered":"2024-09-21T03:33:36.274111+0000","last_unstale":"2024-09-21T03:33:41.298659+0000","last_undegraded":"2024-09-21T03:33:41.298659+0000","last_fullsized":"2024-09-21T03:33:41.298659+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:29:42.707347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.298611+0000","last_change":"2024-09-21T03:33:36.272954+0000","last_active":"2024-09-21T03:33:41.298611+0000","last_peered":"2024-09-21T03:33:41.298611+0000","last_clean":"2024-09-21T03:33:41.298611+0000","last_became_active":"2024-09-21T03:33:36.272551+0000","last_became_peered":"2024-09-21T03:33:36.272551+0000","last_unstale":"2024-09-21T03:33:41.298611+0000","last_undegraded":"2024-09-21T03:33:41.298611+0000","last_fullsized":"2024-09-21T03:33:41.298611+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:41:48.532726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.298922+0000","last_change":"2024-09-21T03:33:36.276841+0000","last_active":"2024-09-21T03:33:41.298922+0000","last_peered":"2024-09-21T03:33:41.298922+0000","last_clean":"2024-09-21T03:33:41.298922+0000","last_became_active":"2024-09-21T03:33:36.276522+0000","last_became_peered":"2024-09-21T03:33:36.276522+0000","last_unstale":"2024-09-21T03:33:41.298922+0000","last_undegraded":"2024-09-21T03:33:41.298922+0000","last_fullsized":"2024-09-21T03:33:41.298922+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:20:05.178755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.920682+0000","last_change":"2024-09-21T03:33:36.251561+0000","last_active":"2024-09-21T03:33:42.920682+0000","last_peered":"2024-09-21T03:33:42.920682+0000","last_clean":"2024-09-21T03:33:42.920682+0000","last_became_active":"2024-09-21T03:33:36.251161+0000","last_became_peered":"2024-09-21T03:33:36.251161+0000","last_unstale":"2024-09-21T03:33:42.920682+0000","last_undegraded":"2024-09-21T03:33:42.920682+0000","last_fullsized":"2024-09-21T03:33:42.920682+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:22:37.114294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.919931+0000","last_change":"2024-09-21T03:33:36.258249+0000","last_active":"2024-09-21T03:33:42.919931+0000","last_peered":"2024-09-21T03:33:42.919931+0000","last_clean":"2024-09-21T03:33:42.919931+0000","last_became_active":"2024-09-21T03:33:36.257558+0000","last_became_peered":"2024-09-21T03:33:36.257558+0000","last_unstale":"2024-09-21T03:33:42.919931+0000","last_undegraded":"2024-09-21T03:33:42.919931+0000","last_fullsized":"2024-09-21T03:33:42.919931+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:14:09.890548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.298137+0000","last_change":"2024-09-21T03:33:36.265238+0000","last_active":"2024-09-21T03:33:41.298137+0000","last_peered":"2024-09-21T03:33:41.298137+0000","last_clean":"2024-09-21T03:33:41.298137+0000","last_became_active":"2024-09-21T03:33:36.264904+0000","last_became_peered":"2024-09-21T03:33:36.264904+0000","last_unstale":"2024-09-21T03:33:41.298137+0000","last_undegraded":"2024-09-21T03:33:41.298137+0000","last_fullsized":"2024-09-21T03:33:41.298137+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:10:32.046638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.286379+0000","last_change":"2024-09-21T03:33:36.270243+0000","last_active":"2024-09-21T03:33:41.286379+0000","last_peered":"2024-09-21T03:33:41.286379+0000","last_clean":"2024-09-21T03:33:41.286379+0000","last_became_active":"2024-09-21T03:33:36.267458+0000","last_became_peered":"2024-09-21T03:33:36.267458+0000","last_unstale":"2024-09-21T03:33:41.286379+0000","last_undegraded":"2024-09-21T03:33:41.286379+0000","last_fullsized":"2024-09-21T03:33:41.286379+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:40:23.758202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.298523+0000","last_change":"2024-09-21T03:33:36.273294+0000","last_active":"2024-09-21T03:33:41.298523+0000","last_peered":"2024-09-21T03:33:41.298523+0000","last_clean":"2024-09-21T03:33:41.298523+0000","last_became_active":"2024-09-21T03:33:36.272841+0000","last_became_peered":"2024-09-21T03:33:36.272841+0000","last_unstale":"2024-09-21T03:33:41.298523+0000","last_undegraded":"2024-09-21T03:33:41.298523+0000","last_fullsized":"2024-09-21T03:33:41.298523+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:43:32.821740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.285970+0000","last_change":"2024-09-21T03:33:36.267665+0000","last_active":"2024-09-21T03:33:41.285970+0000","last_peered":"2024-09-21T03:33:41.285970+0000","last_clean":"2024-09-21T03:33:41.285970+0000","last_became_active":"2024-09-21T03:33:36.267339+0000","last_became_peered":"2024-09-21T03:33:36.267339+0000","last_unstale":"2024-09-21T03:33:41.285970+0000","last_undegraded":"2024-09-21T03:33:41.285970+0000","last_fullsized":"2024-09-21T03:33:41.285970+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:35:19.515735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.920418+0000","last_change":"2024-09-21T03:33:36.260290+0000","last_active":"2024-09-21T03:33:42.920418+0000","last_peered":"2024-09-21T03:33:42.920418+0000","last_clean":"2024-09-21T03:33:42.920418+0000","last_became_active":"2024-09-21T03:33:36.259984+0000","last_became_peered":"2024-09-21T03:33:36.259984+0000","last_unstale":"2024-09-21T03:33:42.920418+0000","last_undegraded":"2024-09-21T03:33:42.920418+0000","last_fullsized":"2024-09-21T03:33:42.920418+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:15:12.112267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.920700+0000","last_change":"2024-09-21T03:33:36.256894+0000","last_active":"2024-09-21T03:33:42.920700+0000","last_peered":"2024-09-21T03:33:42.920700+0000","last_clean":"2024-09-21T03:33:42.920700+0000","last_became_active":"2024-09-21T03:33:36.256598+0000","last_became_peered":"2024-09-21T03:33:36.256598+0000","last_unstale":"2024-09-21T03:33:42.920700+0000","last_undegraded":"2024-09-21T03:33:42.920700+0000","last_fullsized":"2024-09-21T03:33:42.920700+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:59:34.604906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.298773+0000","last_change":"2024-09-21T03:33:36.273537+0000","last_active":"2024-09-21T03:33:41.298773+0000","last_peered":"2024-09-21T03:33:41.298773+0000","last_clean":"2024-09-21T03:33:41.298773+0000","last_became_active":"2024-09-21T03:33:36.272386+0000","last_became_peered":"2024-09-21T03:33:36.272386+0000","last_unstale":"2024-09-21T03:33:41.298773+0000","last_undegraded":"2024-09-21T03:33:41.298773+0000","last_fullsized":"2024-09-21T03:33:41.298773+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:10:25.723208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":1,"up_from":16,"seq":68719476745,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32036,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339804,"statfs":{"total":96636764160,"available":96603959296,"internally_reserved":0,"allocated":868352,"data_stored":697804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509449,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31460,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340380,"statfs":{"total":96636764160,"available":96604549120,"internally_reserved":0,"allocated":278528,"data_stored":107490,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509448,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31456,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340384,"statfs":{"total":96636764160,"available":96604553216,"internally_reserved":0,"allocated":274432,"data_stored":107948,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509449,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":282624,"data_stored":111208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509448,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31464,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340376,"statfs":{"total":96636764160,"available":96604545024,"internally_reserved":0,"allocated":278528,"data_stored":107422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542152,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31456,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340384,"statfs":{"total":96636764160,"available":96604553216,"internally_reserved":0,"allocated":270336,"data_stored":107366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32040,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339800,"statfs":{"total":96636764160,"available":96603955200,"internally_reserved":0,"allocated":872448,"data_stored":697826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542153,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35628,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336212,"statfs":{"total":96636764160,"available":96600281088,"internally_reserved":0,"allocated":290816,"data_stored":111880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":4514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T03:33:57.765 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-21T03:33:57.765 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config log 1 --format=json 2024-09-21T03:33:57.766 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-09-21T03:33:57.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:57.915+0000 7fce4dade640 1 -- 172.21.15.38:0/506196973 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fce480d1120 msgr2=0x7fce480ceb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:57.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:57.915+0000 7fce4dade640 1 --2- 172.21.15.38:0/506196973 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fce480d1120 0x7fce480ceb90 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fce40005fd0 tx=0x7fce4002aa10 comp rx=0 tx=0).stop 2024-09-21T03:33:57.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:57.915+0000 7fce4dade640 1 -- 172.21.15.38:0/506196973 shutdown_connections 2024-09-21T03:33:57.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:57.915+0000 7fce4dade640 1 --2- 172.21.15.38:0/506196973 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fce480d1120 0x7fce480ceb90 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:57.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:57.915+0000 7fce4dade640 1 --2- 172.21.15.38:0/506196973 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fce480d0170 0x7fce480d0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:57.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:57.915+0000 7fce4dade640 1 -- 172.21.15.38:0/506196973 >> 172.21.15.38:0/506196973 conn(0x7fce4809d830 msgr2=0x7fce480b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:57.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:57.915+0000 7fce4dade640 1 -- 172.21.15.38:0/506196973 shutdown_connections 2024-09-21T03:33:57.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:57.915+0000 7fce4dade640 1 -- 172.21.15.38:0/506196973 wait complete. 2024-09-21T03:33:57.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:57.917+0000 7fce4dade640 1 Processor -- start 2024-09-21T03:33:57.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:57.917+0000 7fce4dade640 1 -- start start 2024-09-21T03:33:57.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:57.918+0000 7fce4dade640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fce480d0170 0x7fce480cb610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:57.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:57.918+0000 7fce4dade640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fce480d1120 0x7fce4806e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:57.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:57.918+0000 7fce4dade640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fce4815ee80 0x7fce4806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:57.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:57.918+0000 7fce4dade640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fce480b6700 con 0x7fce480d1120 2024-09-21T03:33:57.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:57.918+0000 7fce4dade640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fce480b6580 con 0x7fce4815ee80 2024-09-21T03:33:57.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:57.918+0000 7fce4dade640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fce480b6880 con 0x7fce480d0170 2024-09-21T03:33:57.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:57.918+0000 7fce46ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fce480d1120 0x7fce4806e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:57.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:57.919+0000 7fce46ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fce480d1120 0x7fce4806e460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41968/0 (socket says 172.21.15.38:41968) 2024-09-21T03:33:57.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:57.919+0000 7fce46ffd640 1 -- 172.21.15.38:0/214295093 learned_addr learned my addr 172.21.15.38:0/214295093 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:33:57.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:57.919+0000 7fce47fff640 1 --2- 172.21.15.38:0/214295093 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fce4815ee80 0x7fce4806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:57.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:57.919+0000 7fce477fe640 1 --2- 172.21.15.38:0/214295093 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fce480d0170 0x7fce480cb610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:57.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:57.919+0000 7fce46ffd640 1 -- 172.21.15.38:0/214295093 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fce480d0170 msgr2=0x7fce480cb610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:57.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:57.919+0000 7fce46ffd640 1 --2- 172.21.15.38:0/214295093 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fce480d0170 0x7fce480cb610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:57.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:57.919+0000 7fce46ffd640 1 -- 172.21.15.38:0/214295093 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fce4815ee80 msgr2=0x7fce4806e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:57.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:57.919+0000 7fce46ffd640 1 --2- 172.21.15.38:0/214295093 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fce4815ee80 0x7fce4806e9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:57.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:57.919+0000 7fce46ffd640 1 -- 172.21.15.38:0/214295093 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce481488d0 con 0x7fce480d1120 2024-09-21T03:33:57.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:57.919+0000 7fce46ffd640 1 --2- 172.21.15.38:0/214295093 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fce480d1120 0x7fce4806e460 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fce3c00e9f0 tx=0x7fce3c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:57.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:57.919+0000 7fce2ffff640 1 -- 172.21.15.38:0/214295093 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce3c00ccc0 con 0x7fce480d1120 2024-09-21T03:33:57.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:57.919+0000 7fce2ffff640 1 -- 172.21.15.38:0/214295093 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fce3c010430 con 0x7fce480d1120 2024-09-21T03:33:57.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:57.919+0000 7fce4dade640 1 -- 172.21.15.38:0/214295093 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce480cbd20 con 0x7fce480d1120 2024-09-21T03:33:57.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:57.919+0000 7fce2ffff640 1 -- 172.21.15.38:0/214295093 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce3c010840 con 0x7fce480d1120 2024-09-21T03:33:57.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:57.919+0000 7fce4dade640 1 -- 172.21.15.38:0/214295093 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce48148e70 con 0x7fce480d1120 2024-09-21T03:33:57.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:57.920+0000 7fce4dade640 1 -- 172.21.15.38:0/214295093 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce480b6da0 con 0x7fce480d1120 2024-09-21T03:33:57.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:57.921+0000 7fce2ffff640 1 -- 172.21.15.38:0/214295093 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fce3c0109e0 con 0x7fce480d1120 2024-09-21T03:33:57.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:57.921+0000 7fce2ffff640 1 --2- 172.21.15.38:0/214295093 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fce34078bc0 0x7fce3407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:57.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:57.922+0000 7fce2ffff640 1 -- 172.21.15.38:0/214295093 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fce3c09bad0 con 0x7fce480d1120 2024-09-21T03:33:57.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:57.922+0000 7fce477fe640 1 --2- 172.21.15.38:0/214295093 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fce34078bc0 0x7fce3407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:57.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:57.922+0000 7fce477fe640 1 --2- 172.21.15.38:0/214295093 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fce34078bc0 0x7fce3407b0d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fce38001720 tx=0x7fce38009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:57.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:57.926+0000 7fce2ffff640 1 -- 172.21.15.38:0/214295093 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fce3c063580 con 0x7fce480d1120 2024-09-21T03:33:58.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:58.114+0000 7fce4dade640 1 -- 172.21.15.38:0/214295093 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "config log", "num": 1, "format": "json"} v 0) -- 0x7fce480cb1b0 con 0x7fce480d1120 2024-09-21T03:33:58.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:58.115+0000 7fce2ffff640 1 -- 172.21.15.38:0/214295093 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "config log", "num": 1, "format": "json"}]=0 v31) ==== 86+0+160 (secure 0 0 0) 0x7fce3c067fb0 con 0x7fce480d1120 2024-09-21T03:33:58.118 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:33:58.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:58.117+0000 7fce4dade640 1 -- 172.21.15.38:0/214295093 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fce34078bc0 msgr2=0x7fce3407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:58.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:58.117+0000 7fce4dade640 1 --2- 172.21.15.38:0/214295093 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fce34078bc0 0x7fce3407b0d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fce38001720 tx=0x7fce38009040 comp rx=0 tx=0).stop 2024-09-21T03:33:58.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:58.117+0000 7fce4dade640 1 -- 172.21.15.38:0/214295093 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fce480d1120 msgr2=0x7fce4806e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:58.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:58.117+0000 7fce4dade640 1 --2- 172.21.15.38:0/214295093 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fce480d1120 0x7fce4806e460 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fce3c00e9f0 tx=0x7fce3c00eeb0 comp rx=0 tx=0).stop 2024-09-21T03:33:58.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:58.117+0000 7fce4dade640 1 -- 172.21.15.38:0/214295093 shutdown_connections 2024-09-21T03:33:58.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:58.117+0000 7fce4dade640 1 --2- 172.21.15.38:0/214295093 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fce4815ee80 0x7fce4806e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:58.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:58.117+0000 7fce4dade640 1 --2- 172.21.15.38:0/214295093 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fce480d1120 0x7fce4806e460 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:58.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:58.117+0000 7fce4dade640 1 --2- 172.21.15.38:0/214295093 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fce34078bc0 0x7fce3407b0d0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:58.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:58.117+0000 7fce4dade640 1 --2- 172.21.15.38:0/214295093 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fce480d0170 0x7fce480cb610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:58.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:58.117+0000 7fce4dade640 1 -- 172.21.15.38:0/214295093 >> 172.21.15.38:0/214295093 conn(0x7fce4809d830 msgr2=0x7fce480b5490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:58.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:58.117+0000 7fce4dade640 1 -- 172.21.15.38:0/214295093 shutdown_connections 2024-09-21T03:33:58.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:58.117+0000 7fce4dade640 1 -- 172.21.15.38:0/214295093 wait complete. 2024-09-21T03:33:58.131 INFO:teuthology.orchestra.run.smithi038.stdout:[{"version":31,"timestamp":"2024-09-21T03:33:23.080483+0000","name":"","changes":[{"name":"osd.1/osd_mclock_max_capacity_iops_ssd","new_value":"4916.015278"}]}] 2024-09-21T03:33:58.132 INFO:tasks.ceph_manager:config epoch is 31 2024-09-21T03:33:58.132 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-21T03:33:58.132 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-09-21T03:33:58.132 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-09-21T03:33:58.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:58.246+0000 7f5db0edc640 1 -- 172.21.15.38:0/774799896 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5d940041c0 msgr2=0x7f5d94004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:58.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:58.246+0000 7f5db0edc640 1 --2- 172.21.15.38:0/774799896 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5d940041c0 0x7f5d94004800 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f5da0006f90 tx=0x7f5d94005830 comp rx=0 tx=0).stop 2024-09-21T03:33:58.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:58.246+0000 7f5daad9d640 1 -- 172.21.15.38:0/774799896 reap_dead start 2024-09-21T03:33:58.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:58.246+0000 7f5db0edc640 1 -- 172.21.15.38:0/774799896 shutdown_connections 2024-09-21T03:33:58.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:58.247+0000 7f5db0edc640 1 -- 172.21.15.38:0/774799896 >> 172.21.15.38:0/774799896 conn(0x7f5dac0d4440 msgr2=0x7f5dac0d4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:58.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:58.247+0000 7f5db0edc640 1 -- 172.21.15.38:0/774799896 shutdown_connections 2024-09-21T03:33:58.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:58.247+0000 7f5db0edc640 1 -- 172.21.15.38:0/774799896 wait complete. 2024-09-21T03:33:58.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:58.248+0000 7f5db0edc640 1 Processor -- start 2024-09-21T03:33:58.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:58.249+0000 7f5db0edc640 1 -- start start 2024-09-21T03:33:58.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:58.250+0000 7f5db0edc640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5d940041c0 0x7f5dac1f8b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:58.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:58.250+0000 7f5db0edc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5dac1f90b0 0x7f5dac0c9030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:58.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:58.250+0000 7f5db0edc640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5dac0c9570 0x7f5dac1f7b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:58.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:58.250+0000 7f5db0edc640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f5dac0b31f0 con 0x7f5dac1f90b0 2024-09-21T03:33:58.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:58.250+0000 7f5db0edc640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f5dac0b3070 con 0x7f5dac0c9570 2024-09-21T03:33:58.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:58.250+0000 7f5db0edc640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f5dac0b3370 con 0x7f5d940041c0 2024-09-21T03:33:58.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:58.250+0000 7f5da9d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5dac1f90b0 0x7f5dac0c9030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:58.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:58.250+0000 7f5da9d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5dac1f90b0 0x7f5dac0c9030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41982/0 (socket says 172.21.15.38:41982) 2024-09-21T03:33:58.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:58.250+0000 7f5da9d9b640 1 -- 172.21.15.38:0/158101077 learned_addr learned my addr 172.21.15.38:0/158101077 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:33:58.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:58.250+0000 7f5daa59c640 1 --2- 172.21.15.38:0/158101077 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5d940041c0 0x7f5dac1f8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:58.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:58.250+0000 7f5daad9d640 1 --2- 172.21.15.38:0/158101077 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5dac0c9570 0x7f5dac1f7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:58.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:58.250+0000 7f5da9d9b640 1 -- 172.21.15.38:0/158101077 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5d940041c0 msgr2=0x7f5dac1f8b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:58.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:58.250+0000 7f5da9d9b640 1 --2- 172.21.15.38:0/158101077 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5d940041c0 0x7f5dac1f8b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:58.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:58.250+0000 7f5da9d9b640 1 -- 172.21.15.38:0/158101077 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5dac0c9570 msgr2=0x7f5dac1f7b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:58.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:58.250+0000 7f5da9d9b640 1 --2- 172.21.15.38:0/158101077 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5dac0c9570 0x7f5dac1f7b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:58.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:58.250+0000 7f5da9d9b640 1 -- 172.21.15.38:0/158101077 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dac0c9da0 con 0x7f5dac1f90b0 2024-09-21T03:33:58.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:58.251+0000 7f5da9d9b640 1 --2- 172.21.15.38:0/158101077 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5dac1f90b0 0x7f5dac0c9030 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f5da0031040 tx=0x7f5da0006ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:58.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:58.251+0000 7f5d9affd640 1 -- 172.21.15.38:0/158101077 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5da0032030 con 0x7f5dac1f90b0 2024-09-21T03:33:58.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:58.251+0000 7f5d9affd640 1 -- 172.21.15.38:0/158101077 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5da0003af0 con 0x7f5dac1f90b0 2024-09-21T03:33:58.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:58.251+0000 7f5d9affd640 1 -- 172.21.15.38:0/158101077 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5da002e460 con 0x7f5dac1f90b0 2024-09-21T03:33:58.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:58.251+0000 7f5db0edc640 1 -- 172.21.15.38:0/158101077 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5dac0ca080 con 0x7f5dac1f90b0 2024-09-21T03:33:58.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:58.251+0000 7f5db0edc640 1 -- 172.21.15.38:0/158101077 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5dac0ca640 con 0x7f5dac1f90b0 2024-09-21T03:33:58.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:58.252+0000 7f5db0edc640 1 -- 172.21.15.38:0/158101077 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5dac0c9b10 con 0x7f5dac1f90b0 2024-09-21T03:33:58.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:58.253+0000 7f5d9affd640 1 -- 172.21.15.38:0/158101077 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f5da0035020 con 0x7f5dac1f90b0 2024-09-21T03:33:58.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:58.253+0000 7f5d9affd640 1 --2- 172.21.15.38:0/158101077 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5d8c078bc0 0x7f5d8c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:58.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:58.253+0000 7f5d9affd640 1 -- 172.21.15.38:0/158101077 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f5da00bb3f0 con 0x7f5dac1f90b0 2024-09-21T03:33:58.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:58.253+0000 7f5daa59c640 1 --2- 172.21.15.38:0/158101077 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5d8c078bc0 0x7f5d8c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:58.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:58.254+0000 7f5daa59c640 1 --2- 172.21.15.38:0/158101077 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5d8c078bc0 0x7f5d8c07b0d0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c006560 tx=0x7f5d9c006b10 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:58.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:58.258+0000 7f5d9affd640 1 -- 172.21.15.38:0/158101077 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f5da0047030 con 0x7f5dac1f90b0 2024-09-21T03:33:58.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:58.495+0000 7f5db0edc640 1 -- 172.21.15.38:0/158101077 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f5dac0b38d0 con 0x7f5dac1f90b0 2024-09-21T03:33:58.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:58.499+0000 7f5d9affd640 1 -- 172.21.15.38:0/158101077 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+194721 (secure 0 0 0) 0x7f5da0083610 con 0x7f5dac1f90b0 2024-09-21T03:33:58.504 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:33:58.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:58.501+0000 7f5db0edc640 1 -- 172.21.15.38:0/158101077 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5d8c078bc0 msgr2=0x7f5d8c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:58.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:58.501+0000 7f5db0edc640 1 --2- 172.21.15.38:0/158101077 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5d8c078bc0 0x7f5d8c07b0d0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c006560 tx=0x7f5d9c006b10 comp rx=0 tx=0).stop 2024-09-21T03:33:58.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:58.501+0000 7f5db0edc640 1 -- 172.21.15.38:0/158101077 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5dac1f90b0 msgr2=0x7f5dac0c9030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:58.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:58.501+0000 7f5db0edc640 1 --2- 172.21.15.38:0/158101077 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5dac1f90b0 0x7f5dac0c9030 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f5da0031040 tx=0x7f5da0006ca0 comp rx=0 tx=0).stop 2024-09-21T03:33:58.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:58.502+0000 7f5db0edc640 1 -- 172.21.15.38:0/158101077 shutdown_connections 2024-09-21T03:33:58.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:58.502+0000 7f5db0edc640 1 --2- 172.21.15.38:0/158101077 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5dac0c9570 0x7f5dac1f7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:58.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:58.502+0000 7f5db0edc640 1 --2- 172.21.15.38:0/158101077 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5dac1f90b0 0x7f5dac0c9030 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:58.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:58.502+0000 7f5db0edc640 1 --2- 172.21.15.38:0/158101077 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5d8c078bc0 0x7f5d8c07b0d0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:58.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:58.502+0000 7f5db0edc640 1 --2- 172.21.15.38:0/158101077 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5d940041c0 0x7f5dac1f8b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:58.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:58.502+0000 7f5db0edc640 1 -- 172.21.15.38:0/158101077 >> 172.21.15.38:0/158101077 conn(0x7f5dac0d4440 msgr2=0x7f5dac06ddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:58.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:58.502+0000 7f5db0edc640 1 -- 172.21.15.38:0/158101077 shutdown_connections 2024-09-21T03:33:58.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:58.502+0000 7f5db0edc640 1 -- 172.21.15.38:0/158101077 wait complete. 2024-09-21T03:33:58.516 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":5,"flags":0,"active_gid":4117,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":1631901344},{"type":"v1","addr":"172.21.15.195:6801","nonce":1631901344}]},"active_addr":"172.21.15.195:6801/1631901344","active_change":"2024-09-21T03:33:14.967561+0000","active_mgr_features":4540719139924082687,"available":true,"standbys":[{"gid":4103,"name":"y","mgr_features":4540719139924082687,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.8","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sso_oauth2":{"name":"sso_oauth2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.8","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sso_oauth2":{"name":"sso_oauth2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.195:0","nonce":757005916}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.195:0","nonce":2518406595}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.195:0","nonce":2524721327}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.195:0","nonce":3841133909}]}]} 2024-09-21T03:33:58.521 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-09-21T03:33:58.521 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-21T03:33:58.521 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T03:33:58.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:58.635+0000 7f02cfe0e640 1 -- 172.21.15.38:0/1275676520 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f02b4004d90 msgr2=0x7f02b4005220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:58.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:58.635+0000 7f02cfe0e640 1 --2- 172.21.15.38:0/1275676520 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f02b4004d90 0x7f02b4005220 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f02c0006f80 tx=0x7f02b4005830 comp rx=0 tx=0).stop 2024-09-21T03:33:58.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:58.635+0000 7f02ce3ab640 1 -- 172.21.15.38:0/1275676520 reap_dead start 2024-09-21T03:33:58.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:58.635+0000 7f02cfe0e640 1 -- 172.21.15.38:0/1275676520 shutdown_connections 2024-09-21T03:33:58.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:58.635+0000 7f02cfe0e640 1 -- 172.21.15.38:0/1275676520 >> 172.21.15.38:0/1275676520 conn(0x7f02c80d4440 msgr2=0x7f02c80d4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:58.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:58.635+0000 7f02cfe0e640 1 -- 172.21.15.38:0/1275676520 shutdown_connections 2024-09-21T03:33:58.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:58.635+0000 7f02cfe0e640 1 -- 172.21.15.38:0/1275676520 wait complete. 2024-09-21T03:33:58.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:58.637+0000 7f02cfe0e640 1 Processor -- start 2024-09-21T03:33:58.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:58.637+0000 7f02cfe0e640 1 -- start start 2024-09-21T03:33:58.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:58.638+0000 7f02cfe0e640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f02b4004d90 0x7f02c80c8880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:58.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:58.638+0000 7f02cfe0e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f02c81f8c00 0x7f02c81f9010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:58.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:58.638+0000 7f02cfe0e640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f02c81f2400 0x7f02c81f2810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:58.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:58.638+0000 7f02cfe0e640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f02c80b50f0 con 0x7f02c81f8c00 2024-09-21T03:33:58.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:58.638+0000 7f02cfe0e640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f02c80b4f70 con 0x7f02c81f2400 2024-09-21T03:33:58.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:58.638+0000 7f02cfe0e640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f02c80b5270 con 0x7f02b4004d90 2024-09-21T03:33:58.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:58.639+0000 7f02cd3a9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f02c81f8c00 0x7f02c81f9010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:58.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:58.639+0000 7f02cd3a9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f02c81f8c00 0x7f02c81f9010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41994/0 (socket says 172.21.15.38:41994) 2024-09-21T03:33:58.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:58.639+0000 7f02cd3a9640 1 -- 172.21.15.38:0/810904462 learned_addr learned my addr 172.21.15.38:0/810904462 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:33:58.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:58.639+0000 7f02cdbaa640 1 --2- 172.21.15.38:0/810904462 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f02b4004d90 0x7f02c80c8880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:58.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:58.639+0000 7f02ce3ab640 1 --2- 172.21.15.38:0/810904462 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f02c81f2400 0x7f02c81f2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:58.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:58.639+0000 7f02cd3a9640 1 -- 172.21.15.38:0/810904462 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f02b4004d90 msgr2=0x7f02c80c8880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:58.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:58.639+0000 7f02cd3a9640 1 --2- 172.21.15.38:0/810904462 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f02b4004d90 0x7f02c80c8880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:58.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:58.639+0000 7f02cd3a9640 1 -- 172.21.15.38:0/810904462 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f02c81f2400 msgr2=0x7f02c81f2810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:58.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:58.639+0000 7f02cd3a9640 1 --2- 172.21.15.38:0/810904462 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f02c81f2400 0x7f02c81f2810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:58.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:58.639+0000 7f02cd3a9640 1 -- 172.21.15.38:0/810904462 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02c81f9850 con 0x7f02c81f8c00 2024-09-21T03:33:58.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:58.639+0000 7f02ce3ab640 1 --2- 172.21.15.38:0/810904462 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f02c81f2400 0x7f02c81f2810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:33:58.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:58.639+0000 7f02cd3a9640 1 --2- 172.21.15.38:0/810904462 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f02c81f8c00 0x7f02c81f9010 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f02b8004d60 tx=0x7f02b8001800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:58.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:58.639+0000 7f02be7fc640 1 -- 172.21.15.38:0/810904462 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02b8017070 con 0x7f02c81f8c00 2024-09-21T03:33:58.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:58.639+0000 7f02be7fc640 1 -- 172.21.15.38:0/810904462 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f02b80108c0 con 0x7f02c81f8c00 2024-09-21T03:33:58.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:58.639+0000 7f02be7fc640 1 -- 172.21.15.38:0/810904462 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02b8010d10 con 0x7f02c81f8c00 2024-09-21T03:33:58.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:58.639+0000 7f02cfe0e640 1 -- 172.21.15.38:0/810904462 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02c815b820 con 0x7f02c81f8c00 2024-09-21T03:33:58.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:58.639+0000 7f02cfe0e640 1 -- 172.21.15.38:0/810904462 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02c815bde0 con 0x7f02c81f8c00 2024-09-21T03:33:58.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:58.640+0000 7f02cfe0e640 1 -- 172.21.15.38:0/810904462 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02c80d2d10 con 0x7f02c81f8c00 2024-09-21T03:33:58.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:58.641+0000 7f02be7fc640 1 -- 172.21.15.38:0/810904462 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f02b8011460 con 0x7f02c81f8c00 2024-09-21T03:33:58.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:58.642+0000 7f02be7fc640 1 --2- 172.21.15.38:0/810904462 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f02ac078b90 0x7f02ac07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:58.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:58.642+0000 7f02be7fc640 1 -- 172.21.15.38:0/810904462 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f02b809ae60 con 0x7f02c81f8c00 2024-09-21T03:33:58.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:58.642+0000 7f02cdbaa640 1 --2- 172.21.15.38:0/810904462 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f02ac078b90 0x7f02ac07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:58.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:58.642+0000 7f02cdbaa640 1 --2- 172.21.15.38:0/810904462 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f02ac078b90 0x7f02ac07b0a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f02c4005fd0 tx=0x7f02c4009500 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:58.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:58.646+0000 7f02be7fc640 1 -- 172.21.15.38:0/810904462 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f02b8062880 con 0x7f02c81f8c00 2024-09-21T03:33:58.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:58.834+0000 7f02cfe0e640 1 -- 172.21.15.38:0/810904462 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f02c80ce5e0 con 0x7f02c81f8c00 2024-09-21T03:33:58.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:58.835+0000 7f02be7fc640 1 -- 172.21.15.38:0/810904462 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18626 (secure 0 0 0) 0x7f02b80672b0 con 0x7f02c81f8c00 2024-09-21T03:33:58.838 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:33:58.838 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":26,"fsid":"9d50d66f-656b-4fe2-8e3b-fe53b79c4d43","created":"2024-09-21T03:33:10.677865+0000","modified":"2024-09-21T03:33:40.665991+0000","last_up_change":"2024-09-21T03:33:24.023421+0000","last_in_change":"2024-09-21T03:33:14.296193+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T03:33:22.992573+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T03:33:33.530201+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T03:33:34.585875+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T03:33:38.268346+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"dbab183f-7e63-48b8-960a-ae55fd5bf565","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6801","nonce":2381639433}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6803","nonce":2381639433}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6807","nonce":2381639433}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6805","nonce":2381639433}]},"public_addr":"172.21.15.38:6801/2381639433","cluster_addr":"172.21.15.38:6803/2381639433","heartbeat_back_addr":"172.21.15.38:6807/2381639433","heartbeat_front_addr":"172.21.15.38:6805/2381639433","state":["exists","up"]},{"osd":1,"uuid":"fa1c0e8a-ebc1-4359-8146-bd5342abbc16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6817","nonce":1491818732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6819","nonce":1491818732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6823","nonce":1491818732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6821","nonce":1491818732}]},"public_addr":"172.21.15.38:6817/1491818732","cluster_addr":"172.21.15.38:6819/1491818732","heartbeat_back_addr":"172.21.15.38:6823/1491818732","heartbeat_front_addr":"172.21.15.38:6821/1491818732","state":["exists","up"]},{"osd":2,"uuid":"2ed3cd2d-385d-4fc6-9869-214552a84803","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6825","nonce":2577858902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6827","nonce":2577858902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6831","nonce":2577858902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6829","nonce":2577858902}]},"public_addr":"172.21.15.38:6825/2577858902","cluster_addr":"172.21.15.38:6827/2577858902","heartbeat_back_addr":"172.21.15.38:6831/2577858902","heartbeat_front_addr":"172.21.15.38:6829/2577858902","state":["exists","up"]},{"osd":3,"uuid":"62d1d95e-919d-4250-be71-1d34cfa70dbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6809","nonce":1286810294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6811","nonce":1286810294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6815","nonce":1286810294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6813","nonce":1286810294}]},"public_addr":"172.21.15.38:6809/1286810294","cluster_addr":"172.21.15.38:6811/1286810294","heartbeat_back_addr":"172.21.15.38:6815/1286810294","heartbeat_front_addr":"172.21.15.38:6813/1286810294","state":["exists","up"]},{"osd":4,"uuid":"7c9f9314-99d7-4d73-8392-37809a9f54b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6803","nonce":3401837571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6805","nonce":3401837571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6809","nonce":3401837571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6807","nonce":3401837571}]},"public_addr":"172.21.15.195:6803/3401837571","cluster_addr":"172.21.15.195:6805/3401837571","heartbeat_back_addr":"172.21.15.195:6809/3401837571","heartbeat_front_addr":"172.21.15.195:6807/3401837571","state":["exists","up"]},{"osd":5,"uuid":"bde4fe9f-8414-4bba-bf4a-43973d1976bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6811","nonce":2080960180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6813","nonce":2080960180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6817","nonce":2080960180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6815","nonce":2080960180}]},"public_addr":"172.21.15.195:6811/2080960180","cluster_addr":"172.21.15.195:6813/2080960180","heartbeat_back_addr":"172.21.15.195:6817/2080960180","heartbeat_front_addr":"172.21.15.195:6815/2080960180","state":["exists","up"]},{"osd":6,"uuid":"6ee6d7f4-1a5f-4a4c-ac16-48fe05f6a002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6827","nonce":4055825675}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6829","nonce":4055825675}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6833","nonce":4055825675}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6831","nonce":4055825675}]},"public_addr":"172.21.15.195:6827/4055825675","cluster_addr":"172.21.15.195:6829/4055825675","heartbeat_back_addr":"172.21.15.195:6833/4055825675","heartbeat_front_addr":"172.21.15.195:6831/4055825675","state":["exists","up"]},{"osd":7,"uuid":"efeb5f51-329a-4484-8290-92e7d3503d05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6819","nonce":1606106342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6821","nonce":1606106342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6825","nonce":1606106342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6823","nonce":1606106342}]},"public_addr":"172.21.15.195:6819/1606106342","cluster_addr":"172.21.15.195:6821/1606106342","heartbeat_back_addr":"172.21.15.195:6825/1606106342","heartbeat_front_addr":"172.21.15.195:6823/1606106342","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.901343+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:21.115003+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.901623+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.835238+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.132199+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:19.817560+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.829690+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:21.087300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T03:33:58.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:58.837+0000 7f02cfe0e640 1 -- 172.21.15.38:0/810904462 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f02ac078b90 msgr2=0x7f02ac07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:58.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:58.837+0000 7f02cfe0e640 1 --2- 172.21.15.38:0/810904462 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f02ac078b90 0x7f02ac07b0a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f02c4005fd0 tx=0x7f02c4009500 comp rx=0 tx=0).stop 2024-09-21T03:33:58.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:58.837+0000 7f02cfe0e640 1 -- 172.21.15.38:0/810904462 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f02c81f8c00 msgr2=0x7f02c81f9010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:58.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:58.837+0000 7f02cfe0e640 1 --2- 172.21.15.38:0/810904462 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f02c81f8c00 0x7f02c81f9010 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f02b8004d60 tx=0x7f02b8001800 comp rx=0 tx=0).stop 2024-09-21T03:33:58.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:58.837+0000 7f02cfe0e640 1 -- 172.21.15.38:0/810904462 shutdown_connections 2024-09-21T03:33:58.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:58.837+0000 7f02cfe0e640 1 --2- 172.21.15.38:0/810904462 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f02c81f2400 0x7f02c81f2810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:58.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:58.837+0000 7f02cfe0e640 1 --2- 172.21.15.38:0/810904462 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f02c81f8c00 0x7f02c81f9010 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:58.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:58.837+0000 7f02cfe0e640 1 --2- 172.21.15.38:0/810904462 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f02ac078b90 0x7f02ac07b0a0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:58.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:58.837+0000 7f02cfe0e640 1 --2- 172.21.15.38:0/810904462 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f02b4004d90 0x7f02c80c8880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:58.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:58.837+0000 7f02cfe0e640 1 -- 172.21.15.38:0/810904462 >> 172.21.15.38:0/810904462 conn(0x7f02c80d4440 msgr2=0x7f02c80d4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:58.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:58.837+0000 7f02cfe0e640 1 -- 172.21.15.38:0/810904462 shutdown_connections 2024-09-21T03:33:58.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:58.837+0000 7f02cfe0e640 1 -- 172.21.15.38:0/810904462 wait complete. 2024-09-21T03:33:58.851 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-21T03:33:58.852 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T03:33:58.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:58.964+0000 7efc02294640 1 -- 172.21.15.38:0/1893022091 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efbfc0cce70 msgr2=0x7efbfc160190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:58.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:58.964+0000 7efc02294640 1 --2- 172.21.15.38:0/1893022091 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efbfc0cce70 0x7efbfc160190 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7efbfc0b4ab0 tx=0x7efbf402aa30 comp rx=0 tx=0).stop 2024-09-21T03:33:58.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:58.964+0000 7efc02294640 1 -- 172.21.15.38:0/1893022091 shutdown_connections 2024-09-21T03:33:58.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:58.964+0000 7efc02294640 1 --2- 172.21.15.38:0/1893022091 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efbfc0cce70 0x7efbfc160190 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:58.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:58.964+0000 7efc02294640 1 --2- 172.21.15.38:0/1893022091 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efbfc0d1d00 0x7efbfc0cc930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:58.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:58.964+0000 7efc02294640 1 --2- 172.21.15.38:0/1893022091 >> v2:172.21.15.195:3300/0 conn(0x7efbfc1536e0 0x7efbfc0d17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:58.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:58.964+0000 7efc02294640 1 -- 172.21.15.38:0/1893022091 >> 172.21.15.38:0/1893022091 conn(0x7efbfc0cef40 msgr2=0x7efbfc0cf350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:58.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:58.965+0000 7efc02294640 1 -- 172.21.15.38:0/1893022091 shutdown_connections 2024-09-21T03:33:58.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:58.965+0000 7efc02294640 1 -- 172.21.15.38:0/1893022091 wait complete. 2024-09-21T03:33:58.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:58.966+0000 7efc02294640 1 Processor -- start 2024-09-21T03:33:58.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:58.966+0000 7efc02294640 1 -- start start 2024-09-21T03:33:58.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:58.968+0000 7efc02294640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efbfc0cce70 0x7efbfc1562c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:58.972 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:58.968+0000 7efc02294640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efbfc0d1d00 0x7efbfc155620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:58.972 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:58.968+0000 7efc02294640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efbfc1536e0 0x7efbfc14d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:58.972 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:58.968+0000 7efc02294640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7efbfc1606d0 con 0x7efbfc1536e0 2024-09-21T03:33:58.972 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:58.968+0000 7efc02294640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7efbfc160850 con 0x7efbfc0cce70 2024-09-21T03:33:58.972 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:58.968+0000 7efc02294640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7efbfc1609d0 con 0x7efbfc0d1d00 2024-09-21T03:33:58.972 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:58.968+0000 7efc00831640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efbfc1536e0 0x7efbfc14d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:58.972 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:58.968+0000 7efc00831640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efbfc1536e0 0x7efbfc14d2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42018/0 (socket says 172.21.15.38:42018) 2024-09-21T03:33:58.972 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:58.968+0000 7efc00831640 1 -- 172.21.15.38:0/694789750 learned_addr learned my addr 172.21.15.38:0/694789750 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:33:58.972 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:58.968+0000 7efbfbfff640 1 --2- 172.21.15.38:0/694789750 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efbfc0cce70 0x7efbfc1562c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:58.972 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:58.968+0000 7efbfb7fe640 1 --2- 172.21.15.38:0/694789750 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efbfc0d1d00 0x7efbfc155620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:58.972 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:58.968+0000 7efc00831640 1 -- 172.21.15.38:0/694789750 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efbfc0d1d00 msgr2=0x7efbfc155620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:58.972 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:58.968+0000 7efc00831640 1 --2- 172.21.15.38:0/694789750 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efbfc0d1d00 0x7efbfc155620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:58.972 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:58.968+0000 7efc00831640 1 -- 172.21.15.38:0/694789750 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efbfc0cce70 msgr2=0x7efbfc1562c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:58.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:58.968+0000 7efc00831640 1 --2- 172.21.15.38:0/694789750 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efbfc0cce70 0x7efbfc1562c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:58.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:58.968+0000 7efc00831640 1 -- 172.21.15.38:0/694789750 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efbfc1f7370 con 0x7efbfc1536e0 2024-09-21T03:33:58.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:58.968+0000 7efc00831640 1 --2- 172.21.15.38:0/694789750 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efbfc1536e0 0x7efbfc14d2e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7efbf4005fd0 tx=0x7efbf4003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:58.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:58.969+0000 7efbf8ff9640 1 -- 172.21.15.38:0/694789750 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbf4033050 con 0x7efbfc1536e0 2024-09-21T03:33:58.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:58.969+0000 7efbf8ff9640 1 -- 172.21.15.38:0/694789750 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efbf4004450 con 0x7efbfc1536e0 2024-09-21T03:33:58.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:58.969+0000 7efc02294640 1 -- 172.21.15.38:0/694789750 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efbfc1f7650 con 0x7efbfc1536e0 2024-09-21T03:33:58.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:58.969+0000 7efbf8ff9640 1 -- 172.21.15.38:0/694789750 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbf4004840 con 0x7efbfc1536e0 2024-09-21T03:33:58.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:58.969+0000 7efc02294640 1 -- 172.21.15.38:0/694789750 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efbfc1f7c10 con 0x7efbfc1536e0 2024-09-21T03:33:58.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:58.970+0000 7efc02294640 1 -- 172.21.15.38:0/694789750 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efbfc1569a0 con 0x7efbfc1536e0 2024-09-21T03:33:58.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:58.970+0000 7efbf8ff9640 1 -- 172.21.15.38:0/694789750 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7efbf4036070 con 0x7efbfc1536e0 2024-09-21T03:33:58.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:58.971+0000 7efbf8ff9640 1 --2- 172.21.15.38:0/694789750 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7efbe8078bc0 0x7efbe807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:58.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:58.971+0000 7efbf8ff9640 1 -- 172.21.15.38:0/694789750 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7efbf40bc450 con 0x7efbfc1536e0 2024-09-21T03:33:58.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:58.971+0000 7efbfbfff640 1 --2- 172.21.15.38:0/694789750 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7efbe8078bc0 0x7efbe807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:58.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:58.972+0000 7efbfbfff640 1 --2- 172.21.15.38:0/694789750 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7efbe8078bc0 0x7efbe807b0d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7efbec00bb50 tx=0x7efbec00b2e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:58.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:58.976+0000 7efbf8ff9640 1 -- 172.21.15.38:0/694789750 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7efbf4083f00 con 0x7efbfc1536e0 2024-09-21T03:33:59.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.163+0000 7efc02294640 1 -- 172.21.15.38:0/694789750 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efbfc0b4280 con 0x7efbfc1536e0 2024-09-21T03:33:59.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.165+0000 7efbf8ff9640 1 -- 172.21.15.38:0/694789750 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18626 (secure 0 0 0) 0x7efbfc0b4280 con 0x7efbfc1536e0 2024-09-21T03:33:59.168 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:33:59.168 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":26,"fsid":"9d50d66f-656b-4fe2-8e3b-fe53b79c4d43","created":"2024-09-21T03:33:10.677865+0000","modified":"2024-09-21T03:33:40.665991+0000","last_up_change":"2024-09-21T03:33:24.023421+0000","last_in_change":"2024-09-21T03:33:14.296193+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T03:33:22.992573+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T03:33:33.530201+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T03:33:34.585875+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T03:33:38.268346+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"dbab183f-7e63-48b8-960a-ae55fd5bf565","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6801","nonce":2381639433}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6803","nonce":2381639433}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6807","nonce":2381639433}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6805","nonce":2381639433}]},"public_addr":"172.21.15.38:6801/2381639433","cluster_addr":"172.21.15.38:6803/2381639433","heartbeat_back_addr":"172.21.15.38:6807/2381639433","heartbeat_front_addr":"172.21.15.38:6805/2381639433","state":["exists","up"]},{"osd":1,"uuid":"fa1c0e8a-ebc1-4359-8146-bd5342abbc16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6817","nonce":1491818732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6819","nonce":1491818732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6823","nonce":1491818732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6821","nonce":1491818732}]},"public_addr":"172.21.15.38:6817/1491818732","cluster_addr":"172.21.15.38:6819/1491818732","heartbeat_back_addr":"172.21.15.38:6823/1491818732","heartbeat_front_addr":"172.21.15.38:6821/1491818732","state":["exists","up"]},{"osd":2,"uuid":"2ed3cd2d-385d-4fc6-9869-214552a84803","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6825","nonce":2577858902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6827","nonce":2577858902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6831","nonce":2577858902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6829","nonce":2577858902}]},"public_addr":"172.21.15.38:6825/2577858902","cluster_addr":"172.21.15.38:6827/2577858902","heartbeat_back_addr":"172.21.15.38:6831/2577858902","heartbeat_front_addr":"172.21.15.38:6829/2577858902","state":["exists","up"]},{"osd":3,"uuid":"62d1d95e-919d-4250-be71-1d34cfa70dbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6809","nonce":1286810294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6811","nonce":1286810294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6815","nonce":1286810294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6813","nonce":1286810294}]},"public_addr":"172.21.15.38:6809/1286810294","cluster_addr":"172.21.15.38:6811/1286810294","heartbeat_back_addr":"172.21.15.38:6815/1286810294","heartbeat_front_addr":"172.21.15.38:6813/1286810294","state":["exists","up"]},{"osd":4,"uuid":"7c9f9314-99d7-4d73-8392-37809a9f54b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6803","nonce":3401837571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6805","nonce":3401837571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6809","nonce":3401837571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6807","nonce":3401837571}]},"public_addr":"172.21.15.195:6803/3401837571","cluster_addr":"172.21.15.195:6805/3401837571","heartbeat_back_addr":"172.21.15.195:6809/3401837571","heartbeat_front_addr":"172.21.15.195:6807/3401837571","state":["exists","up"]},{"osd":5,"uuid":"bde4fe9f-8414-4bba-bf4a-43973d1976bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6811","nonce":2080960180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6813","nonce":2080960180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6817","nonce":2080960180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6815","nonce":2080960180}]},"public_addr":"172.21.15.195:6811/2080960180","cluster_addr":"172.21.15.195:6813/2080960180","heartbeat_back_addr":"172.21.15.195:6817/2080960180","heartbeat_front_addr":"172.21.15.195:6815/2080960180","state":["exists","up"]},{"osd":6,"uuid":"6ee6d7f4-1a5f-4a4c-ac16-48fe05f6a002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6827","nonce":4055825675}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6829","nonce":4055825675}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6833","nonce":4055825675}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6831","nonce":4055825675}]},"public_addr":"172.21.15.195:6827/4055825675","cluster_addr":"172.21.15.195:6829/4055825675","heartbeat_back_addr":"172.21.15.195:6833/4055825675","heartbeat_front_addr":"172.21.15.195:6831/4055825675","state":["exists","up"]},{"osd":7,"uuid":"efeb5f51-329a-4484-8290-92e7d3503d05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6819","nonce":1606106342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6821","nonce":1606106342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6825","nonce":1606106342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6823","nonce":1606106342}]},"public_addr":"172.21.15.195:6819/1606106342","cluster_addr":"172.21.15.195:6821/1606106342","heartbeat_back_addr":"172.21.15.195:6825/1606106342","heartbeat_front_addr":"172.21.15.195:6823/1606106342","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.901343+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:21.115003+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.901623+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.835238+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.132199+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:19.817560+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.829690+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:21.087300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T03:33:59.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.167+0000 7efc02294640 1 -- 172.21.15.38:0/694789750 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7efbe8078bc0 msgr2=0x7efbe807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:59.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.167+0000 7efc02294640 1 --2- 172.21.15.38:0/694789750 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7efbe8078bc0 0x7efbe807b0d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7efbec00bb50 tx=0x7efbec00b2e0 comp rx=0 tx=0).stop 2024-09-21T03:33:59.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.167+0000 7efc02294640 1 -- 172.21.15.38:0/694789750 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efbfc1536e0 msgr2=0x7efbfc14d2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:59.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.167+0000 7efc02294640 1 --2- 172.21.15.38:0/694789750 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efbfc1536e0 0x7efbfc14d2e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7efbf4005fd0 tx=0x7efbf4003f10 comp rx=0 tx=0).stop 2024-09-21T03:33:59.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.167+0000 7efc02294640 1 -- 172.21.15.38:0/694789750 shutdown_connections 2024-09-21T03:33:59.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.167+0000 7efc02294640 1 --2- 172.21.15.38:0/694789750 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efbfc1536e0 0x7efbfc14d2e0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:59.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.167+0000 7efc02294640 1 --2- 172.21.15.38:0/694789750 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7efbe8078bc0 0x7efbe807b0d0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:59.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.167+0000 7efc02294640 1 --2- 172.21.15.38:0/694789750 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efbfc0d1d00 0x7efbfc155620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:59.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.167+0000 7efc02294640 1 --2- 172.21.15.38:0/694789750 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efbfc0cce70 0x7efbfc1562c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:59.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.167+0000 7efc02294640 1 -- 172.21.15.38:0/694789750 >> 172.21.15.38:0/694789750 conn(0x7efbfc0cef40 msgr2=0x7efbfc15e200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:59.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.167+0000 7efc02294640 1 -- 172.21.15.38:0/694789750 shutdown_connections 2024-09-21T03:33:59.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.167+0000 7efc02294640 1 -- 172.21.15.38:0/694789750 wait complete. 2024-09-21T03:33:59.181 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-21T03:33:59.182 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-21T03:33:59.182 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-21T03:33:59.182 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-21T03:33:59.183 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-21T03:33:59.183 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-21T03:33:59.184 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-09-21T03:33:59.184 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-09-21T03:33:59.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.404+0000 7faaadd9b640 1 -- 172.21.15.38:0/4039691392 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaa0005260 con 0x7faaa8055ee0 2024-09-21T03:33:59.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.405+0000 7faab48e4640 1 -- 172.21.15.38:0/4039691392 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faaa8055ee0 msgr2=0x7faaa8034d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:59.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.405+0000 7faab48e4640 1 --2- 172.21.15.38:0/4039691392 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faaa8055ee0 0x7faaa8034d50 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7faaa0005fd0 tx=0x7faaa002a610 comp rx=0 tx=0).stop 2024-09-21T03:33:59.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.405+0000 7faab48e4640 1 -- 172.21.15.38:0/4039691392 shutdown_connections 2024-09-21T03:33:59.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.405+0000 7faab48e4640 1 --2- 172.21.15.38:0/4039691392 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faaa8055280 0x7faaa8035290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:59.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.405+0000 7faab48e4640 1 --2- 172.21.15.38:0/4039691392 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faaa8055ee0 0x7faaa8034d50 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:59.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.405+0000 7faab48e4640 1 -- 172.21.15.38:0/4039691392 >> 172.21.15.38:0/4039691392 conn(0x7faaa801eae0 msgr2=0x7faaa803a710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:59.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.406+0000 7faab48e4640 1 -- 172.21.15.38:0/4039691392 shutdown_connections 2024-09-21T03:33:59.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.406+0000 7f043a7fc640 1 -- 172.21.15.38:0/3949873305 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f043c0adcb0 con 0x7f0434055ee0 2024-09-21T03:33:59.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.407+0000 7f0440e98640 1 -- 172.21.15.38:0/3949873305 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0434055ee0 msgr2=0x7f04340dfe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:59.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.407+0000 7f0440e98640 1 --2- 172.21.15.38:0/3949873305 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0434055ee0 0x7f04340dfe80 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f043c06c560 tx=0x7f043c0a6430 comp rx=0 tx=0).stop 2024-09-21T03:33:59.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.407+0000 7ff983b39640 1 -- 172.21.15.38:0/610816253 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff97c06def0 msgr2=0x7ff97c06f410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:59.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.407+0000 7ff983b39640 1 --2- 172.21.15.38:0/610816253 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff97c06def0 0x7ff97c06f410 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ff978009830 tx=0x7ff97802a3d0 comp rx=0 tx=0).stop 2024-09-21T03:33:59.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.407+0000 7ff983b39640 1 -- 172.21.15.38:0/610816253 shutdown_connections 2024-09-21T03:33:59.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.407+0000 7ff983b39640 1 --2- 172.21.15.38:0/610816253 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff97c06def0 0x7ff97c06f410 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:59.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.407+0000 7ff983b39640 1 -- 172.21.15.38:0/610816253 >> 172.21.15.38:0/610816253 conn(0x7ff97c0cdca0 msgr2=0x7ff97c0ce0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:59.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.406+0000 7faab48e4640 1 -- 172.21.15.38:0/4039691392 wait complete. 2024-09-21T03:33:59.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.408+0000 7ff983b39640 1 -- 172.21.15.38:0/610816253 shutdown_connections 2024-09-21T03:33:59.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.407+0000 7f0440e98640 1 -- 172.21.15.38:0/3949873305 shutdown_connections 2024-09-21T03:33:59.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.407+0000 7f0440e98640 1 --2- 172.21.15.38:0/3949873305 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0434055ee0 0x7f04340dfe80 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:59.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.407+0000 7f0440e98640 1 -- 172.21.15.38:0/3949873305 >> 172.21.15.38:0/3949873305 conn(0x7f043401eae0 msgr2=0x7f043403a710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:59.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.408+0000 7f0440e98640 1 -- 172.21.15.38:0/3949873305 shutdown_connections 2024-09-21T03:33:59.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.408+0000 7ff983b39640 1 -- 172.21.15.38:0/610816253 wait complete. 2024-09-21T03:33:59.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.408+0000 7f0440e98640 1 -- 172.21.15.38:0/3949873305 wait complete. 2024-09-21T03:33:59.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.408+0000 7fc7168d8640 1 -- 172.21.15.38:0/70852936 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc708009ec0 msgr2=0x7fc6fc004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:59.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.408+0000 7fc7168d8640 1 --2- 172.21.15.38:0/70852936 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc708009ec0 0x7fc6fc004800 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc70c009b50 tx=0x7fc6fc005830 comp rx=0 tx=0).stop 2024-09-21T03:33:59.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.408+0000 7fc7160d7640 1 -- 172.21.15.38:0/70852936 reap_dead start 2024-09-21T03:33:59.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.408+0000 7fc7168d8640 1 -- 172.21.15.38:0/70852936 shutdown_connections 2024-09-21T03:33:59.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.408+0000 7fc7168d8640 1 -- 172.21.15.38:0/70852936 >> 172.21.15.38:0/70852936 conn(0x7fc708051420 msgr2=0x7fc708051860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:59.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.410+0000 7fc7168d8640 1 -- 172.21.15.38:0/70852936 shutdown_connections 2024-09-21T03:33:59.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.410+0000 7fc7168d8640 1 -- 172.21.15.38:0/70852936 wait complete. 2024-09-21T03:33:59.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.410+0000 7faab48e4640 1 Processor -- start 2024-09-21T03:33:59.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.411+0000 7faab48e4640 1 -- start start 2024-09-21T03:33:59.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.411+0000 7f13869de640 1 -- 172.21.15.38:0/2325876978 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1378009ba0 msgr2=0x7f13780df180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:59.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.411+0000 7f13869de640 1 --2- 172.21.15.38:0/2325876978 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1378009ba0 0x7f13780df180 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f137800ca70 tx=0x7f137002b230 comp rx=0 tx=0).stop 2024-09-21T03:33:59.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.411+0000 7f13869de640 1 -- 172.21.15.38:0/2325876978 shutdown_connections 2024-09-21T03:33:59.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.411+0000 7f13869de640 1 --2- 172.21.15.38:0/2325876978 >> v2:172.21.15.38:3300/0 conn(0x7f1378005c60 0x7f1378036430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:59.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.411+0000 7f13869de640 1 --2- 172.21.15.38:0/2325876978 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1378009ba0 0x7f13780df180 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:59.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.411+0000 7f13869de640 1 -- 172.21.15.38:0/2325876978 >> 172.21.15.38:0/2325876978 conn(0x7f1378051420 msgr2=0x7f1378051860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:59.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.411+0000 7f0440e98640 1 Processor -- start 2024-09-21T03:33:59.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.411+0000 7fbbf500a640 1 -- 172.21.15.38:0/4172026405 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbbf0073a20 msgr2=0x7fbbf0073e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:59.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.411+0000 7fbbf500a640 1 --2- 172.21.15.38:0/4172026405 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbbf0073a20 0x7fbbf0073e30 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fbbf00b4c40 tx=0x7fbbe402a4b0 comp rx=0 tx=0).stop 2024-09-21T03:33:59.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.411+0000 7ff983b39640 1 Processor -- start 2024-09-21T03:33:59.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.411+0000 7ff983b39640 1 -- start start 2024-09-21T03:33:59.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.412+0000 7f13869de640 1 -- 172.21.15.38:0/2325876978 shutdown_connections 2024-09-21T03:33:59.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.412+0000 7f0440e98640 1 -- start start 2024-09-21T03:33:59.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.412+0000 7f13869de640 1 -- 172.21.15.38:0/2325876978 wait complete. 2024-09-21T03:33:59.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.412+0000 7fbbf500a640 1 -- 172.21.15.38:0/4172026405 shutdown_connections 2024-09-21T03:33:59.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.412+0000 7fbbf500a640 1 --2- 172.21.15.38:0/4172026405 >> v2:172.21.15.38:3300/0 conn(0x7fbbf015e300 0x7fbbf015e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:59.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.412+0000 7fbbf500a640 1 --2- 172.21.15.38:0/4172026405 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbbf0073a20 0x7fbbf0073e30 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:59.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.412+0000 7fbbf500a640 1 -- 172.21.15.38:0/4172026405 >> 172.21.15.38:0/4172026405 conn(0x7fbbf00d0590 msgr2=0x7fbbf006cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:59.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.412+0000 7fbbf500a640 1 -- 172.21.15.38:0/4172026405 shutdown_connections 2024-09-21T03:33:59.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.412+0000 7fbbf500a640 1 -- 172.21.15.38:0/4172026405 wait complete. 2024-09-21T03:33:59.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.412+0000 7f1b9ffff640 1 -- 172.21.15.38:0/107066560 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1b880051c0 msgr2=0x7f1b8800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:59.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.412+0000 7f1b9ffff640 1 --2- 172.21.15.38:0/107066560 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1b880051c0 0x7f1b8800b9c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f1ba00521d0 tx=0x7f1b88005830 comp rx=0 tx=0).stop 2024-09-21T03:33:59.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.412+0000 7f1b9f7fe640 1 -- 172.21.15.38:0/107066560 reap_dead start 2024-09-21T03:33:59.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.413+0000 7fc7168d8640 1 Processor -- start 2024-09-21T03:33:59.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.413+0000 7fc7168d8640 1 -- start start 2024-09-21T03:33:59.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.413+0000 7f1b9ffff640 1 -- 172.21.15.38:0/107066560 shutdown_connections 2024-09-21T03:33:59.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.413+0000 7f1b9ffff640 1 -- 172.21.15.38:0/107066560 >> 172.21.15.38:0/107066560 conn(0x7f1b98021df0 msgr2=0x7f1b98058bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:59.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.413+0000 7f1b9ffff640 1 -- 172.21.15.38:0/107066560 shutdown_connections 2024-09-21T03:33:59.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.413+0000 7f1b9ffff640 1 -- 172.21.15.38:0/107066560 wait complete. 2024-09-21T03:33:59.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.414+0000 7faab48e4640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faaa80091c0 0x7faaa803bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:59.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.414+0000 7faab48e4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faaa8055280 0x7faaa80d5a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:59.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.414+0000 7faab48e4640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faaa80e1380 0x7faaa80328b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:59.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.414+0000 7f0440e98640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0434008fa0 0x7f0434051b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:59.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.414+0000 7faab48e4640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7faaa80357d0 con 0x7faaa8055280 2024-09-21T03:33:59.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.415+0000 7faaae59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faaa8055280 0x7faaa80d5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:59.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.415+0000 7f0440e98640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f043403ae90 0x7f04340e1770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:59.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.415+0000 7faaae59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faaa8055280 0x7faaa80d5a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42102/0 (socket says 172.21.15.38:42102) 2024-09-21T03:33:59.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.415+0000 7faaae59c640 1 -- 172.21.15.38:0/1666060122 learned_addr learned my addr 172.21.15.38:0/1666060122 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:33:59.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.415+0000 7f0440e98640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f04340d77d0 0x7f04340e1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:59.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.415+0000 7faaaf59e640 1 --2- 172.21.15.38:0/1666060122 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faaa80e1380 0x7faaa80328b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:59.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.415+0000 7faab48e4640 1 -- 172.21.15.38:0/1666060122 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7faaa8036a70 con 0x7faaa80091c0 2024-09-21T03:33:59.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.415+0000 7f13869de640 1 Processor -- start 2024-09-21T03:33:59.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.415+0000 7f0440e98640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f0434053ca0 con 0x7f04340d77d0 2024-09-21T03:33:59.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.415+0000 7faab48e4640 1 -- 172.21.15.38:0/1666060122 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7faaa8035950 con 0x7faaa80e1380 2024-09-21T03:33:59.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.415+0000 7f13869de640 1 -- start start 2024-09-21T03:33:59.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.415+0000 7f043bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f04340d77d0 0x7f04340e1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:59.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.413+0000 7ff983b39640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff97c06def0 0x7ff97c1fa8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:59.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.413+0000 7ff9818d5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff97c06def0 0x7ff97c1fa8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:59.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.413+0000 7ff9818d5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff97c06def0 0x7ff97c1fa8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42094/0 (socket says 172.21.15.38:42094) 2024-09-21T03:33:59.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.413+0000 7ff9818d5640 1 -- 172.21.15.38:0/774792089 learned_addr learned my addr 172.21.15.38:0/774792089 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:33:59.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.413+0000 7ff983b39640 1 --2- 172.21.15.38:0/774792089 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff97c0b5260 0x7ff97c1e80c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:59.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.413+0000 7ff983b39640 1 --2- 172.21.15.38:0/774792089 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff97c0b5670 0x7ff97c0cccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:59.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.413+0000 7ff983b39640 1 -- 172.21.15.38:0/774792089 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ff97c071480 con 0x7ff97c06def0 2024-09-21T03:33:59.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.413+0000 7ff983b39640 1 -- 172.21.15.38:0/774792089 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ff97c071300 con 0x7ff97c0b5260 2024-09-21T03:33:59.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.413+0000 7ff983b39640 1 -- 172.21.15.38:0/774792089 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7ff97c071600 con 0x7ff97c0b5670 2024-09-21T03:33:59.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.414+0000 7ff9820d6640 1 --2- 172.21.15.38:0/774792089 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff97c0b5670 0x7ff97c0cccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:59.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.414+0000 7ff9818d5640 1 -- 172.21.15.38:0/774792089 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff97c0b5670 msgr2=0x7ff97c0cccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:59.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.414+0000 7ff9818d5640 1 --2- 172.21.15.38:0/774792089 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff97c0b5670 0x7ff97c0cccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:59.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.414+0000 7ff9818d5640 1 -- 172.21.15.38:0/774792089 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff97c0b5260 msgr2=0x7ff97c1e80c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T03:33:59.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.414+0000 7ff9818d5640 1 --2- 172.21.15.38:0/774792089 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff97c0b5260 0x7ff97c1e80c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:59.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.414+0000 7ff9818d5640 1 -- 172.21.15.38:0/774792089 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff97c0b1470 con 0x7ff97c06def0 2024-09-21T03:33:59.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.415+0000 7ff9818d5640 1 --2- 172.21.15.38:0/774792089 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff97c06def0 0x7ff97c1fa8f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff978009950 tx=0x7ff978033ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:59.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.415+0000 7f043bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f04340d77d0 0x7f04340e1cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42104/0 (socket says 172.21.15.38:42104) 2024-09-21T03:33:59.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.415+0000 7f043bfff640 1 -- 172.21.15.38:0/1975697659 learned_addr learned my addr 172.21.15.38:0/1975697659 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:33:59.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.415+0000 7f0440e98640 1 -- 172.21.15.38:0/1975697659 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f0434053b20 con 0x7f043403ae90 2024-09-21T03:33:59.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.415+0000 7f0440e98640 1 -- 172.21.15.38:0/1975697659 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f0434053e20 con 0x7f0434008fa0 2024-09-21T03:33:59.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.416+0000 7fc7168d8640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc708009ec0 0x7fc7080f2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:59.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.416+0000 7fc7168d8640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc7080429a0 0x7fc708182340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:59.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.415+0000 7fbbf500a640 1 Processor -- start 2024-09-21T03:33:59.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.415+0000 7fbbf500a640 1 -- start start 2024-09-21T03:33:59.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.416+0000 7faaaf59e640 1 -- 172.21.15.38:0/1666060122 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faaa80091c0 msgr2=0x7faaa803bbf0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T03:33:59.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.416+0000 7faaaf59e640 1 --2- 172.21.15.38:0/1666060122 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faaa80091c0 0x7faaa803bbf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:59.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.416+0000 7faaaf59e640 1 -- 172.21.15.38:0/1666060122 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faaa8055280 msgr2=0x7faaa80d5a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:59.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.416+0000 7faaaf59e640 1 --2- 172.21.15.38:0/1666060122 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faaa8055280 0x7faaa80d5a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:59.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.416+0000 7faaaf59e640 1 -- 172.21.15.38:0/1666060122 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faaa80d8ed0 con 0x7faaa80e1380 2024-09-21T03:33:59.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.416+0000 7faaae59c640 1 --2- 172.21.15.38:0/1666060122 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faaa8055280 0x7faaa80d5a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:33:59.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.416+0000 7faaaf59e640 1 --2- 172.21.15.38:0/1666060122 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faaa80e1380 0x7faaa80328b0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7faaa400e640 tx=0x7faaa400eb00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:59.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.416+0000 7ff9727fc640 1 -- 172.21.15.38:0/774792089 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff978041030 con 0x7ff97c06def0 2024-09-21T03:33:59.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.416+0000 7ff9727fc640 1 -- 172.21.15.38:0/774792089 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff978005020 con 0x7ff97c06def0 2024-09-21T03:33:59.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.416+0000 7ff9727fc640 1 -- 172.21.15.38:0/774792089 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff978037040 con 0x7ff97c06def0 2024-09-21T03:33:59.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.416+0000 7fc7158d6640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc708009ec0 0x7fc7080f2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:59.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.416+0000 7fc7158d6640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc708009ec0 0x7fc7080f2a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.38:50446/0 (socket says 172.21.15.38:50446) 2024-09-21T03:33:59.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.416+0000 7fc7158d6640 1 -- 172.21.15.38:0/3724372677 learned_addr learned my addr 172.21.15.38:0/3724372677 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:33:59.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.416+0000 7fc7150d5640 1 --2- 172.21.15.38:0/3724372677 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc7080429a0 0x7fc708182340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:59.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.416+0000 7fc7168d8640 1 --2- 172.21.15.38:0/3724372677 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc70817e410 0x7fc708046df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:59.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.416+0000 7fc7168d8640 1 -- 172.21.15.38:0/3724372677 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fc7080064f0 con 0x7fc7080429a0 2024-09-21T03:33:59.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.416+0000 7fc7168d8640 1 -- 172.21.15.38:0/3724372677 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fc708006370 con 0x7fc70817e410 2024-09-21T03:33:59.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.416+0000 7f043b7fe640 1 --2- 172.21.15.38:0/1975697659 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0434008fa0 0x7f0434051b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:59.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.416+0000 7f043affd640 1 --2- 172.21.15.38:0/1975697659 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f043403ae90 0x7f04340e1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:59.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.416+0000 7fc7160d7640 1 --2- 172.21.15.38:0/3724372677 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc70817e410 0x7fc708046df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:59.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.416+0000 7fc7168d8640 1 -- 172.21.15.38:0/3724372677 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fc708006670 con 0x7fc708009ec0 2024-09-21T03:33:59.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.416+0000 7faa977fe640 1 -- 172.21.15.38:0/1666060122 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaa4004220 con 0x7faaa80e1380 2024-09-21T03:33:59.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.417+0000 7faab48e4640 1 -- 172.21.15.38:0/1666060122 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faaa80d91b0 con 0x7faaa80e1380 2024-09-21T03:33:59.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.417+0000 7faab48e4640 1 -- 172.21.15.38:0/1666060122 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7faaa80d65e0 con 0x7faaa80e1380 2024-09-21T03:33:59.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.417+0000 7faa977fe640 1 -- 172.21.15.38:0/1666060122 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faaa4005020 con 0x7faaa80e1380 2024-09-21T03:33:59.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.417+0000 7faa977fe640 1 -- 172.21.15.38:0/1666060122 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaa4005470 con 0x7faaa80e1380 2024-09-21T03:33:59.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.417+0000 7f1b9ffff640 1 Processor -- start 2024-09-21T03:33:59.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.417+0000 7f1b9ffff640 1 -- start start 2024-09-21T03:33:59.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.417+0000 7f043b7fe640 1 -- 172.21.15.38:0/1975697659 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f043403ae90 msgr2=0x7f04340e1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:59.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.417+0000 7f043b7fe640 1 --2- 172.21.15.38:0/1975697659 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f043403ae90 0x7f04340e1770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:59.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.417+0000 7f043b7fe640 1 -- 172.21.15.38:0/1975697659 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f04340d77d0 msgr2=0x7f04340e1cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:59.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.417+0000 7f043b7fe640 1 --2- 172.21.15.38:0/1975697659 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f04340d77d0 0x7f04340e1cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:59.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.417+0000 7f043b7fe640 1 -- 172.21.15.38:0/1975697659 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f043416c5d0 con 0x7f0434008fa0 2024-09-21T03:33:59.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.417+0000 7ff983b39640 1 -- 172.21.15.38:0/774792089 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff97c0b1750 con 0x7ff97c06def0 2024-09-21T03:33:59.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.417+0000 7fc7150d5640 1 -- 172.21.15.38:0/3724372677 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc708009ec0 msgr2=0x7fc7080f2a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:59.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.417+0000 7fc7150d5640 1 --2- 172.21.15.38:0/3724372677 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc708009ec0 0x7fc7080f2a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:59.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.417+0000 7fc7150d5640 1 -- 172.21.15.38:0/3724372677 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc70817e410 msgr2=0x7fc708046df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:59.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.417+0000 7fc7150d5640 1 --2- 172.21.15.38:0/3724372677 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc70817e410 0x7fc708046df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:59.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.417+0000 7fc7150d5640 1 -- 172.21.15.38:0/3724372677 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc708181b80 con 0x7fc7080429a0 2024-09-21T03:33:59.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.417+0000 7ff983b39640 1 -- 172.21.15.38:0/774792089 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff97c1e88c0 con 0x7ff97c06def0 2024-09-21T03:33:59.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.418+0000 7fbbf500a640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbbf0070c80 0x7fbbf0150d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:59.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.417+0000 7f13869de640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1378005c60 0x7f1378043740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:59.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.418+0000 7f13869de640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1378009ba0 0x7f137817d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:59.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.418+0000 7f13869de640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1378036970 0x7f137817dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:59.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.418+0000 7f13869de640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f137800bdb0 con 0x7f1378036970 2024-09-21T03:33:59.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.418+0000 7f13869de640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f137800bc30 con 0x7f1378009ba0 2024-09-21T03:33:59.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.418+0000 7f13869de640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f137801ccc0 con 0x7f1378005c60 2024-09-21T03:33:59.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.418+0000 7f13851db640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1378009ba0 0x7f137817d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:59.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.418+0000 7f13851db640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1378009ba0 0x7f137817d860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.38:54674/0 (socket says 172.21.15.38:54674) 2024-09-21T03:33:59.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.418+0000 7f13851db640 1 -- 172.21.15.38:0/2444085775 learned_addr learned my addr 172.21.15.38:0/2444085775 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:33:59.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.418+0000 7f13859dc640 1 --2- 172.21.15.38:0/2444085775 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1378005c60 0x7f1378043740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:59.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.418+0000 7fbbf500a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbbf0073a20 0x7fbbf0205a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:59.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.418+0000 7fbbeed9d640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbbf0070c80 0x7fbbf0150d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:59.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.418+0000 7fbbeed9d640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbbf0070c80 0x7fbbf0150d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.38:54688/0 (socket says 172.21.15.38:54688) 2024-09-21T03:33:59.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.418+0000 7fbbf500a640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbbf015e300 0x7fbbf0205fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:59.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.418+0000 7fbbeed9d640 1 -- 172.21.15.38:0/1384166255 learned_addr learned my addr 172.21.15.38:0/1384166255 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:33:59.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.418+0000 7fbbf500a640 1 -- 172.21.15.38:0/1384166255 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fbbf00cf2f0 con 0x7fbbf0073a20 2024-09-21T03:33:59.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.418+0000 7fbbf500a640 1 -- 172.21.15.38:0/1384166255 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fbbf00cf170 con 0x7fbbf0070c80 2024-09-21T03:33:59.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.418+0000 7fbbf500a640 1 -- 172.21.15.38:0/1384166255 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fbbf00cf470 con 0x7fbbf015e300 2024-09-21T03:33:59.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.419+0000 7f043b7fe640 1 --2- 172.21.15.38:0/1975697659 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0434008fa0 0x7f0434051b20 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f042800c340 tx=0x7f042800c800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:59.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.418+0000 7f13851db640 1 -- 172.21.15.38:0/2444085775 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1378005c60 msgr2=0x7f1378043740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:59.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.418+0000 7f13851db640 1 --2- 172.21.15.38:0/2444085775 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1378005c60 0x7f1378043740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:59.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.418+0000 7f13851db640 1 -- 172.21.15.38:0/2444085775 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1378036970 msgr2=0x7f137817dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:59.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.418+0000 7f13851db640 1 --2- 172.21.15.38:0/2444085775 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1378036970 0x7f137817dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:59.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.418+0000 7f13851db640 1 -- 172.21.15.38:0/2444085775 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13780442c0 con 0x7f1378009ba0 2024-09-21T03:33:59.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.419+0000 7f13851db640 1 --2- 172.21.15.38:0/2444085775 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1378009ba0 0x7f137817d860 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f137c00b410 tx=0x7f137c00b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:59.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.419+0000 7fc7150d5640 1 --2- 172.21.15.38:0/3724372677 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc7080429a0 0x7fc708182340 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fc6fc00bd90 tx=0x7fc70c0336f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:59.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.418+0000 7fbbef59e640 1 --2- 172.21.15.38:0/1384166255 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbbf015e300 0x7fbbf0205fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:59.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.419+0000 7fbbeed9d640 1 -- 172.21.15.38:0/1384166255 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbbf015e300 msgr2=0x7fbbf0205fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:59.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.419+0000 7fbbeed9d640 1 --2- 172.21.15.38:0/1384166255 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbbf015e300 0x7fbbf0205fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:59.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.419+0000 7fbbeed9d640 1 -- 172.21.15.38:0/1384166255 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbbf0073a20 msgr2=0x7fbbf0205a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:59.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.419+0000 7ff983b39640 1 -- 172.21.15.38:0/774792089 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff97c0b1c70 con 0x7ff97c06def0 2024-09-21T03:33:59.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.419+0000 7fbbeed9d640 1 --2- 172.21.15.38:0/1384166255 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbbf0073a20 0x7fbbf0205a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:59.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.419+0000 7fbbeed9d640 1 -- 172.21.15.38:0/1384166255 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbf016be20 con 0x7fbbf0070c80 2024-09-21T03:33:59.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.419+0000 7fbbef59e640 1 --2- 172.21.15.38:0/1384166255 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbbf015e300 0x7fbbf0205fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:33:59.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.419+0000 7f0427fff640 1 -- 172.21.15.38:0/1975697659 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0428017070 con 0x7f0434008fa0 2024-09-21T03:33:59.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.419+0000 7fbbeed9d640 1 --2- 172.21.15.38:0/1384166255 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbbf0070c80 0x7fbbf0150d50 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fbbe000c730 tx=0x7fbbe000cbf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:59.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.419+0000 7f0427fff640 1 -- 172.21.15.38:0/1975697659 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f04280053c0 con 0x7f0434008fa0 2024-09-21T03:33:59.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.419+0000 7f13767fc640 1 -- 172.21.15.38:0/2444085775 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f137c015020 con 0x7f1378009ba0 2024-09-21T03:33:59.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.419+0000 7f13767fc640 1 -- 172.21.15.38:0/2444085775 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f137c011940 con 0x7f1378009ba0 2024-09-21T03:33:59.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.419+0000 7f0440e98640 1 -- 172.21.15.38:0/1975697659 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f043416c8b0 con 0x7f0434008fa0 2024-09-21T03:33:59.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.419+0000 7f13767fc640 1 -- 172.21.15.38:0/2444085775 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f137c011d90 con 0x7f1378009ba0 2024-09-21T03:33:59.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.419+0000 7f0440e98640 1 -- 172.21.15.38:0/1975697659 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f04340d9a80 con 0x7f0434008fa0 2024-09-21T03:33:59.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.419+0000 7f0427fff640 1 -- 172.21.15.38:0/1975697659 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0428003850 con 0x7f0434008fa0 2024-09-21T03:33:59.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.419+0000 7f1b9ffff640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1b880051c0 0x7f1b980e6be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:59.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.420+0000 7f1b9ffff640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1b980da9b0 0x7f1b980dadc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:59.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.420+0000 7f1b9effd640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1b880051c0 0x7f1b980e6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:59.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.420+0000 7f1b9effd640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1b880051c0 0x7f1b980e6be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.38:54700/0 (socket says 172.21.15.38:54700) 2024-09-21T03:33:59.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.420+0000 7f1b9effd640 1 -- 172.21.15.38:0/3885889993 learned_addr learned my addr 172.21.15.38:0/3885889993 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:33:59.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.420+0000 7faa977fe640 1 -- 172.21.15.38:0/1666060122 <== mon.2 v2:172.21.15.195:3301/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7faaa400c810 con 0x7faaa80e1380 2024-09-21T03:33:59.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.420+0000 7f1b9ffff640 1 --2- 172.21.15.38:0/3885889993 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1b980db300 0x7f1b9804d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:59.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.420+0000 7f1b9ffff640 1 -- 172.21.15.38:0/3885889993 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f1b98054960 con 0x7f1b980db300 2024-09-21T03:33:59.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.421+0000 7f1b9ffff640 1 -- 172.21.15.38:0/3885889993 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f1b980547e0 con 0x7f1b880051c0 2024-09-21T03:33:59.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.421+0000 7f1b9f7fe640 1 --2- 172.21.15.38:0/3885889993 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1b980db300 0x7f1b9804d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:59.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.421+0000 7f1b9e7fc640 1 --2- 172.21.15.38:0/3885889993 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1b980da9b0 0x7f1b980dadc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:59.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.419+0000 7fbbd77fe640 1 -- 172.21.15.38:0/1384166255 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbe0016020 con 0x7fbbf0070c80 2024-09-21T03:33:59.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.420+0000 7fbbd77fe640 1 -- 172.21.15.38:0/1384166255 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbbe000a950 con 0x7fbbf0070c80 2024-09-21T03:33:59.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.421+0000 7fc7067fc640 1 -- 172.21.15.38:0/3724372677 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc70c038040 con 0x7fc7080429a0 2024-09-21T03:33:59.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.421+0000 7f1b9ffff640 1 -- 172.21.15.38:0/3885889993 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f1b98054ae0 con 0x7f1b980da9b0 2024-09-21T03:33:59.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.421+0000 7faab48e4640 1 -- 172.21.15.38:0/1666060122 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7faa90001030 con 0x7faaa80e1380 2024-09-21T03:33:59.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.421+0000 7faa977fe640 1 --2- 172.21.15.38:0/1666060122 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7faa9c078c60 0x7faa9c07b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:59.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.421+0000 7fc7168d8640 1 -- 172.21.15.38:0/3724372677 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc708053680 con 0x7fc7080429a0 2024-09-21T03:33:59.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.421+0000 7fc7168d8640 1 -- 172.21.15.38:0/3724372677 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc708053c40 con 0x7fc7080429a0 2024-09-21T03:33:59.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.420+0000 7f13869de640 1 -- 172.21.15.38:0/2444085775 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1378020810 con 0x7f1378009ba0 2024-09-21T03:33:59.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.420+0000 7f13869de640 1 -- 172.21.15.38:0/2444085775 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f137801f870 con 0x7f1378009ba0 2024-09-21T03:33:59.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.421+0000 7f1363fff640 1 -- 172.21.15.38:0/2444085775 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f135c001030 con 0x7f1378009ba0 2024-09-21T03:33:59.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.421+0000 7fbbd77fe640 1 -- 172.21.15.38:0/1384166255 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbe000ada0 con 0x7fbbf0070c80 2024-09-21T03:33:59.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.421+0000 7fbbf500a640 1 -- 172.21.15.38:0/1384166255 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbf016c100 con 0x7fbbf0070c80 2024-09-21T03:33:59.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.421+0000 7fbbf500a640 1 -- 172.21.15.38:0/1384166255 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbf0168b20 con 0x7fbbf0070c80 2024-09-21T03:33:59.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.421+0000 7fc7067fc640 1 -- 172.21.15.38:0/3724372677 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc70c0338e0 con 0x7fc7080429a0 2024-09-21T03:33:59.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.421+0000 7fc7067fc640 1 -- 172.21.15.38:0/3724372677 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc70c033d10 con 0x7fc7080429a0 2024-09-21T03:33:59.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.421+0000 7f1b9effd640 1 -- 172.21.15.38:0/3885889993 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1b980da9b0 msgr2=0x7f1b980dadc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:59.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.421+0000 7f1b9effd640 1 --2- 172.21.15.38:0/3885889993 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1b980da9b0 0x7f1b980dadc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:59.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.421+0000 7f1b9effd640 1 -- 172.21.15.38:0/3885889993 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1b980db300 msgr2=0x7f1b9804d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:59.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.421+0000 7f1b9effd640 1 --2- 172.21.15.38:0/3885889993 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1b980db300 0x7f1b9804d2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:59.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.421+0000 7f1b9effd640 1 -- 172.21.15.38:0/3885889993 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b980e1240 con 0x7f1b880051c0 2024-09-21T03:33:59.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.422+0000 7f0440e98640 1 -- 172.21.15.38:0/1975697659 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f043416e810 con 0x7f0434008fa0 2024-09-21T03:33:59.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.421+0000 7faa977fe640 1 -- 172.21.15.38:0/1666060122 <== mon.2 v2:172.21.15.195:3301/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7faaa409b3a0 con 0x7faaa80e1380 2024-09-21T03:33:59.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.421+0000 7faa977fe640 1 --2- 172.21.15.38:0/1666060122 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x7faa9c0823b0 0x7faa9c084840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:59.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.422+0000 7faa977fe640 1 -- 172.21.15.38:0/1666060122 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7faaa4004e70 con 0x7faa9c0823b0 2024-09-21T03:33:59.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.422+0000 7faa977fe640 1 -- 172.21.15.38:0/1666060122 <== mon.2 v2:172.21.15.195:3301/0 6 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7faaa40a2050 con 0x7faaa80e1380 2024-09-21T03:33:59.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.422+0000 7faaaed9d640 1 --2- 172.21.15.38:0/1666060122 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7faa9c078c60 0x7faa9c07b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:59.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.422+0000 7faaae59c640 1 --2- 172.21.15.38:0/1666060122 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x7faa9c0823b0 0x7faa9c084840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:59.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.423+0000 7f1b9effd640 1 --2- 172.21.15.38:0/3885889993 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1b880051c0 0x7f1b980e6be0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f1b9000c520 tx=0x7f1b9000c9e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:59.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.423+0000 7faaae59c640 1 --2- 172.21.15.38:0/1666060122 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x7faa9c0823b0 0x7faa9c084840 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:59.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.423+0000 7f1b877fe640 1 -- 172.21.15.38:0/3885889993 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b90016020 con 0x7f1b880051c0 2024-09-21T03:33:59.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.423+0000 7f1b877fe640 1 -- 172.21.15.38:0/3885889993 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1b90012940 con 0x7f1b880051c0 2024-09-21T03:33:59.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.423+0000 7f1b877fe640 1 -- 172.21.15.38:0/3885889993 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b90012d90 con 0x7f1b880051c0 2024-09-21T03:33:59.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.423+0000 7f1b9ffff640 1 -- 172.21.15.38:0/3885889993 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b980e1740 con 0x7f1b880051c0 2024-09-21T03:33:59.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.423+0000 7f1b9ffff640 1 -- 172.21.15.38:0/3885889993 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b980e1d00 con 0x7f1b880051c0 2024-09-21T03:33:59.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.423+0000 7f0427fff640 1 -- 172.21.15.38:0/1975697659 <== mon.2 v2:172.21.15.195:3301/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f042801a070 con 0x7f0434008fa0 2024-09-21T03:33:59.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.423+0000 7ff9727fc640 1 -- 172.21.15.38:0/774792089 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ff978044070 con 0x7ff97c06def0 2024-09-21T03:33:59.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.423+0000 7fbbf500a640 1 -- 172.21.15.38:0/1384166255 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fbbf006ac00 con 0x7fbbf0070c80 2024-09-21T03:33:59.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.423+0000 7fc7168d8640 1 -- 172.21.15.38:0/3724372677 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc6f8001030 con 0x7fc7080429a0 2024-09-21T03:33:59.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.424+0000 7faaaed9d640 1 --2- 172.21.15.38:0/1666060122 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7faa9c078c60 0x7faa9c07b170 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7faab006bbc0 tx=0x7faab00773f0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:59.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.424+0000 7faa977fe640 1 -- 172.21.15.38:0/1666060122 <== osd.0 v2:172.21.15.38:6800/2381639433 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7faaa4004e70 con 0x7faa9c0823b0 2024-09-21T03:33:59.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.424+0000 7ff9727fc640 1 --2- 172.21.15.38:0/774792089 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff960078c60 0x7ff96007b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:59.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.424+0000 7ff9727fc640 1 -- 172.21.15.38:0/774792089 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7ff9780bb5e0 con 0x7ff97c06def0 2024-09-21T03:33:59.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.424+0000 7ff9727fc640 1 --2- 172.21.15.38:0/774792089 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x7ff960082430 0x7ff9600848c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:59.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.424+0000 7ff9727fc640 1 -- 172.21.15.38:0/774792089 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff978049e50 con 0x7ff960082430 2024-09-21T03:33:59.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.424+0000 7ff9727fc640 1 -- 172.21.15.38:0/774792089 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff9780bb9d0 con 0x7ff97c06def0 2024-09-21T03:33:59.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.425+0000 7f0427fff640 1 --2- 172.21.15.38:0/1975697659 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0420078c90 0x7f042007b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:59.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.425+0000 7f0427fff640 1 -- 172.21.15.38:0/1975697659 <== mon.2 v2:172.21.15.195:3301/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f042809a290 con 0x7f0434008fa0 2024-09-21T03:33:59.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.425+0000 7f657ffff640 1 -- 172.21.15.38:0/2371857752 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6580011540 con 0x7f6588151920 2024-09-21T03:33:59.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.425+0000 7f043affd640 1 --2- 172.21.15.38:0/1975697659 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0420078c90 0x7f042007b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:59.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.425+0000 7f0427fff640 1 --2- 172.21.15.38:0/1975697659 >> [v2:172.21.15.195:6810/2080960180,v1:172.21.15.195:6811/2080960180] conn(0x7f0420082450 0x7f04200848e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:59.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.425+0000 7f0427fff640 1 -- 172.21.15.38:0/1975697659 --> [v2:172.21.15.195:6810/2080960180,v1:172.21.15.195:6811/2080960180] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0428016ec0 con 0x7f0420082450 2024-09-21T03:33:59.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.425+0000 7f0427fff640 1 -- 172.21.15.38:0/1975697659 <== mon.2 v2:172.21.15.195:3301/0 6 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f04280a1050 con 0x7f0434008fa0 2024-09-21T03:33:59.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.425+0000 7f658f5c3640 1 -- 172.21.15.38:0/2371857752 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6588151920 msgr2=0x7f6588073960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:59.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.425+0000 7f658f5c3640 1 --2- 172.21.15.38:0/2371857752 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6588151920 0x7f6588073960 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f6588070a40 tx=0x7f6580032bf0 comp rx=0 tx=0).stop 2024-09-21T03:33:59.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.425+0000 7f13767fc640 1 -- 172.21.15.38:0/2444085775 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f137c018020 con 0x7f1378009ba0 2024-09-21T03:33:59.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.425+0000 7ff9810d4640 1 --2- 172.21.15.38:0/774792089 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff960078c60 0x7ff96007b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:59.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.425+0000 7ff9820d6640 1 --2- 172.21.15.38:0/774792089 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x7ff960082430 0x7ff9600848c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:59.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.425+0000 7ff9820d6640 1 --2- 172.21.15.38:0/774792089 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x7ff960082430 0x7ff9600848c0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:59.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.426+0000 7f13767fc640 1 --2- 172.21.15.38:0/2444085775 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1364078c90 0x7f136407b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:59.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.425+0000 7f043bfff640 1 --2- 172.21.15.38:0/1975697659 >> [v2:172.21.15.195:6810/2080960180,v1:172.21.15.195:6811/2080960180] conn(0x7f0420082450 0x7f04200848e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:59.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.426+0000 7f043bfff640 1 --2- 172.21.15.38:0/1975697659 >> [v2:172.21.15.195:6810/2080960180,v1:172.21.15.195:6811/2080960180] conn(0x7f0420082450 0x7f04200848e0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:59.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.427+0000 7f13767fc640 1 -- 172.21.15.38:0/2444085775 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f137c09b590 con 0x7f1378009ba0 2024-09-21T03:33:59.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.427+0000 7ff9810d4640 1 --2- 172.21.15.38:0/774792089 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff960078c60 0x7ff96007b170 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff97c1fe390 tx=0x7ff96c007970 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:59.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.425+0000 7f1b9ffff640 1 -- 172.21.15.38:0/3885889993 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1b9804cec0 con 0x7f1b880051c0 2024-09-21T03:33:59.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.426+0000 7f1b877fe640 1 -- 172.21.15.38:0/3885889993 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f1b90019020 con 0x7f1b880051c0 2024-09-21T03:33:59.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.427+0000 7f13859dc640 1 --2- 172.21.15.38:0/2444085775 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1364078c90 0x7f136407b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:59.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.426+0000 7fbbd77fe640 1 -- 172.21.15.38:0/1384166255 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fbbe0019020 con 0x7fbbf0070c80 2024-09-21T03:33:59.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.427+0000 7f658f5c3640 1 -- 172.21.15.38:0/2371857752 shutdown_connections 2024-09-21T03:33:59.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.427+0000 7f658f5c3640 1 --2- 172.21.15.38:0/2371857752 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6588073ea0 0x7f658815e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:59.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.427+0000 7f658f5c3640 1 --2- 172.21.15.38:0/2371857752 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6588151920 0x7f6588073960 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:59.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.427+0000 7f658f5c3640 1 --2- 172.21.15.38:0/2371857752 >> v2:172.21.15.195:3300/0 conn(0x7f6588070c80 0x7f65881513e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:59.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.427+0000 7f658f5c3640 1 -- 172.21.15.38:0/2371857752 >> 172.21.15.38:0/2371857752 conn(0x7f65880d0590 msgr2=0x7f658806cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:59.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.427+0000 7fc7067fc640 1 -- 172.21.15.38:0/3724372677 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc70c043020 con 0x7fc7080429a0 2024-09-21T03:33:59.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.427+0000 7f0427fff640 1 -- 172.21.15.38:0/1975697659 <== osd.5 v2:172.21.15.195:6810/2080960180 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0428016ec0 con 0x7f0420082450 2024-09-21T03:33:59.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.427+0000 7f1b877fe640 1 --2- 172.21.15.38:0/3885889993 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1b8c078b90 0x7f1b8c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:59.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.427+0000 7f043affd640 1 --2- 172.21.15.38:0/1975697659 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0420078c90 0x7f042007b1a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f043400fa60 tx=0x7f043c0a8000 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:59.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.427+0000 7ff9727fc640 1 -- 172.21.15.38:0/774792089 <== osd.2 v2:172.21.15.38:6824/2577858902 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff978049e50 con 0x7ff960082430 2024-09-21T03:33:59.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.427+0000 7f658f5c3640 1 -- 172.21.15.38:0/2371857752 shutdown_connections 2024-09-21T03:33:59.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.427+0000 7f1b877fe640 1 -- 172.21.15.38:0/3885889993 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f1b9009aeb0 con 0x7f1b880051c0 2024-09-21T03:33:59.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.427+0000 7fbbd77fe640 1 --2- 172.21.15.38:0/1384166255 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fbbdc078be0 0x7fbbdc07b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:59.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.427+0000 7f1b877fe640 1 --2- 172.21.15.38:0/3885889993 >> [v2:172.21.15.195:6802/3401837571,v1:172.21.15.195:6803/3401837571] conn(0x7f1b8c082360 0x7f1b8c0847f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:59.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.428+0000 7f1b877fe640 1 -- 172.21.15.38:0/3885889993 --> [v2:172.21.15.195:6802/3401837571,v1:172.21.15.195:6803/3401837571] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1b9001fe80 con 0x7f1b8c082360 2024-09-21T03:33:59.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.428+0000 7f1b877fe640 1 -- 172.21.15.38:0/3885889993 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1b9009b2a0 con 0x7f1b880051c0 2024-09-21T03:33:59.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.428+0000 7fbbd77fe640 1 -- 172.21.15.38:0/1384166255 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fbbe009b130 con 0x7fbbf0070c80 2024-09-21T03:33:59.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.428+0000 7fbbee59c640 1 --2- 172.21.15.38:0/1384166255 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fbbdc078be0 0x7fbbdc07b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:59.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.428+0000 7f1b9e7fc640 1 --2- 172.21.15.38:0/3885889993 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1b8c078b90 0x7f1b8c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:59.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.428+0000 7fc7067fc640 1 --2- 172.21.15.38:0/3724372677 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc6f4078c60 0x7fc6f407b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:59.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.428+0000 7fbbd77fe640 1 --2- 172.21.15.38:0/1384166255 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x7fbbdc0823a0 0x7fbbdc084830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:59.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.428+0000 7f1b9f7fe640 1 --2- 172.21.15.38:0/3885889993 >> [v2:172.21.15.195:6802/3401837571,v1:172.21.15.195:6803/3401837571] conn(0x7f1b8c082360 0x7f1b8c0847f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:59.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.428+0000 7fbbd77fe640 1 -- 172.21.15.38:0/1384166255 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbbe0021e50 con 0x7fbbdc0823a0 2024-09-21T03:33:59.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.428+0000 7fbbd77fe640 1 -- 172.21.15.38:0/1384166255 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fbbe009b4e0 con 0x7fbbf0070c80 2024-09-21T03:33:59.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.428+0000 7f658f5c3640 1 -- 172.21.15.38:0/2371857752 wait complete. 2024-09-21T03:33:59.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.428+0000 7fc7067fc640 1 -- 172.21.15.38:0/3724372677 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fc70c0bbd10 con 0x7fc7080429a0 2024-09-21T03:33:59.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.428+0000 7fc7067fc640 1 --2- 172.21.15.38:0/3724372677 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x7fc6f4082420 0x7fc6f40848b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:59.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.428+0000 7fbbef59e640 1 --2- 172.21.15.38:0/1384166255 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x7fbbdc0823a0 0x7fbbdc084830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:59.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.428+0000 7fbbee59c640 1 --2- 172.21.15.38:0/1384166255 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fbbdc078be0 0x7fbbdc07b0f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fbbe4034000 tx=0x7fbbe40046c0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:59.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.428+0000 7f1b9f7fe640 1 --2- 172.21.15.38:0/3885889993 >> [v2:172.21.15.195:6802/3401837571,v1:172.21.15.195:6803/3401837571] conn(0x7f1b8c082360 0x7f1b8c0847f0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:59.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.428+0000 7f13767fc640 1 --2- 172.21.15.38:0/2444085775 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x7f1364082450 0x7f13640848e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:59.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.429+0000 7f13767fc640 1 -- 172.21.15.38:0/2444085775 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1364084fb0 con 0x7f1364082450 2024-09-21T03:33:59.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.429+0000 7f13767fc640 1 -- 172.21.15.38:0/2444085775 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f137c09b980 con 0x7f1378009ba0 2024-09-21T03:33:59.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.429+0000 7f13861dd640 1 --2- 172.21.15.38:0/2444085775 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x7f1364082450 0x7f13640848e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:59.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.429+0000 7f1b9e7fc640 1 --2- 172.21.15.38:0/3885889993 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1b8c078b90 0x7f1b8c07b0a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1b94005fd0 tx=0x7f1b9400f530 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:59.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.429+0000 7fbbef59e640 1 --2- 172.21.15.38:0/1384166255 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x7fbbdc0823a0 0x7fbbdc084830 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:59.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.428+0000 7fc7158d6640 1 --2- 172.21.15.38:0/3724372677 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc6f4078c60 0x7fc6f407b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:59.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.429+0000 7fc7160d7640 1 --2- 172.21.15.38:0/3724372677 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x7fc6f4082420 0x7fc6f40848b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:59.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.429+0000 7fc7067fc640 1 -- 172.21.15.38:0/3724372677 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc6f4084f80 con 0x7fc6f4082420 2024-09-21T03:33:59.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.429+0000 7fc7158d6640 1 --2- 172.21.15.38:0/3724372677 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc6f4078c60 0x7fc6f407b170 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fc71007ad70 tx=0x7fc71008d040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:59.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.429+0000 7f13859dc640 1 --2- 172.21.15.38:0/2444085775 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1364078c90 0x7f136407b1a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f1370034000 tx=0x7f1370004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:59.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.429+0000 7f1b877fe640 1 -- 172.21.15.38:0/3885889993 <== osd.4 v2:172.21.15.195:6802/3401837571 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1b9001fe80 con 0x7f1b8c082360 2024-09-21T03:33:59.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.429+0000 7f13861dd640 1 --2- 172.21.15.38:0/2444085775 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x7f1364082450 0x7f13640848e0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:59.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.429+0000 7fc7160d7640 1 --2- 172.21.15.38:0/3724372677 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x7fc6f4082420 0x7fc6f40848b0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:59.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.429+0000 7fc7067fc640 1 -- 172.21.15.38:0/3724372677 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc70c0bc100 con 0x7fc7080429a0 2024-09-21T03:33:59.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.430+0000 7fbbd77fe640 1 -- 172.21.15.38:0/1384166255 <== osd.7 v2:172.21.15.195:6818/1606106342 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbbe0021e50 con 0x7fbbdc0823a0 2024-09-21T03:33:59.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.430+0000 7f13767fc640 1 -- 172.21.15.38:0/2444085775 <== osd.6 v2:172.21.15.195:6826/4055825675 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1364084fb0 con 0x7f1364082450 2024-09-21T03:33:59.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.431+0000 7f658f5c3640 1 Processor -- start 2024-09-21T03:33:59.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.431+0000 7fc7067fc640 1 -- 172.21.15.38:0/3724372677 <== osd.3 v2:172.21.15.38:6808/1286810294 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc6f4084f80 con 0x7fc6f4082420 2024-09-21T03:33:59.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.436+0000 7f658f5c3640 1 -- start start 2024-09-21T03:33:59.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.438+0000 7f658f5c3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6588070c80 0x7f658814dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:59.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.438+0000 7f658f5c3640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6588073ea0 0x7f658814b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:59.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.439+0000 7f658d35f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6588070c80 0x7f658814dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:59.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.439+0000 7f658d35f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6588070c80 0x7f658814dcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42170/0 (socket says 172.21.15.38:42170) 2024-09-21T03:33:59.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.439+0000 7f658f5c3640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6588151920 0x7f658814bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:59.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.439+0000 7f658f5c3640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f65880cf2f0 con 0x7f6588070c80 2024-09-21T03:33:59.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.439+0000 7f658f5c3640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f65880cf170 con 0x7f6588073ea0 2024-09-21T03:33:59.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.439+0000 7f658cb5e640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6588073ea0 0x7f658814b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:59.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.439+0000 7f658d35f640 1 -- 172.21.15.38:0/3508840641 learned_addr learned my addr 172.21.15.38:0/3508840641 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:33:59.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.439+0000 7f658f5c3640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f65880cf470 con 0x7f6588151920 2024-09-21T03:33:59.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.439+0000 7f658db60640 1 --2- 172.21.15.38:0/3508840641 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6588151920 0x7f658814bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:59.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.440+0000 7f658cb5e640 1 -- 172.21.15.38:0/3508840641 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6588151920 msgr2=0x7f658814bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:59.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.440+0000 7f658cb5e640 1 --2- 172.21.15.38:0/3508840641 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6588151920 0x7f658814bba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:59.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.440+0000 7f658cb5e640 1 -- 172.21.15.38:0/3508840641 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6588070c80 msgr2=0x7f658814dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:59.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.440+0000 7f658cb5e640 1 --2- 172.21.15.38:0/3508840641 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6588070c80 0x7f658814dcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:59.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.440+0000 7f658cb5e640 1 -- 172.21.15.38:0/3508840641 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f658814e210 con 0x7f6588073ea0 2024-09-21T03:33:59.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.448+0000 7f658cb5e640 1 --2- 172.21.15.38:0/3508840641 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6588073ea0 0x7f658814b660 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f6588072990 tx=0x7f658000ff90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:59.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.450+0000 7ff983b39640 1 -- 172.21.15.38:0/774792089 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff97c0b1c70 con 0x7ff960082430 2024-09-21T03:33:59.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.449+0000 7f657dffb640 1 -- 172.21.15.38:0/3508840641 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6580003970 con 0x7f6588073ea0 2024-09-21T03:33:59.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.449+0000 7f657dffb640 1 -- 172.21.15.38:0/3508840641 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6580003c20 con 0x7f6588073ea0 2024-09-21T03:33:59.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.449+0000 7f657dffb640 1 -- 172.21.15.38:0/3508840641 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65800339a0 con 0x7f6588073ea0 2024-09-21T03:33:59.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.450+0000 7f658f5c3640 1 -- 172.21.15.38:0/3508840641 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f658814a5c0 con 0x7f6588073ea0 2024-09-21T03:33:59.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.450+0000 7ff9727fc640 1 -- 172.21.15.38:0/774792089 <== osd.2 v2:172.21.15.38:6824/2577858902 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7ff97c0b1c70 con 0x7ff960082430 2024-09-21T03:33:59.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.450+0000 7f658f5c3640 1 -- 172.21.15.38:0/3508840641 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f658814ab80 con 0x7f6588073ea0 2024-09-21T03:33:59.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.451+0000 7ff95ffff640 1 -- 172.21.15.38:0/774792089 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x7ff960082430 msgr2=0x7ff9600848c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:59.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.451+0000 7ff95ffff640 1 --2- 172.21.15.38:0/774792089 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x7ff960082430 0x7ff9600848c0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:59.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.451+0000 7fbbf500a640 1 -- 172.21.15.38:0/1384166255 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fbbf006ac00 con 0x7fbbdc0823a0 2024-09-21T03:33:59.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.451+0000 7ff95ffff640 1 -- 172.21.15.38:0/774792089 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff960078c60 msgr2=0x7ff96007b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:59.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.451+0000 7ff95ffff640 1 --2- 172.21.15.38:0/774792089 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff960078c60 0x7ff96007b170 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff97c1fe390 tx=0x7ff96c007970 comp rx=0 tx=0).stop 2024-09-21T03:33:59.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.451+0000 7ff95ffff640 1 -- 172.21.15.38:0/774792089 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff97c06def0 msgr2=0x7ff97c1fa8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:59.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.452+0000 7ff95ffff640 1 --2- 172.21.15.38:0/774792089 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff97c06def0 0x7ff97c1fa8f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff978009950 tx=0x7ff978033ba0 comp rx=0 tx=0).stop 2024-09-21T03:33:59.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.452+0000 7ff9820d6640 1 -- 172.21.15.38:0/774792089 reap_dead start 2024-09-21T03:33:59.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.452+0000 7fbbd77fe640 1 -- 172.21.15.38:0/1384166255 <== osd.7 v2:172.21.15.195:6818/1606106342 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fbbf006ac00 con 0x7fbbdc0823a0 2024-09-21T03:33:59.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.452+0000 7f658f5c3640 1 -- 172.21.15.38:0/3508840641 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f658814c0e0 con 0x7f6588073ea0 2024-09-21T03:33:59.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.452+0000 7ff95ffff640 1 -- 172.21.15.38:0/774792089 shutdown_connections 2024-09-21T03:33:59.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.452+0000 7ff95ffff640 1 -- 172.21.15.38:0/774792089 >> 172.21.15.38:0/774792089 conn(0x7ff97c0cdca0 msgr2=0x7ff97c06d4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:59.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.452+0000 7ff95ffff640 1 -- 172.21.15.38:0/774792089 shutdown_connections 2024-09-21T03:33:59.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.452+0000 7ff95ffff640 1 -- 172.21.15.38:0/774792089 wait complete. 2024-09-21T03:33:59.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.453+0000 7fbbf500a640 1 -- 172.21.15.38:0/1384166255 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x7fbbdc0823a0 msgr2=0x7fbbdc084830 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:59.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.453+0000 7fbbf500a640 1 --2- 172.21.15.38:0/1384166255 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x7fbbdc0823a0 0x7fbbdc084830 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:59.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.453+0000 7fbbf500a640 1 -- 172.21.15.38:0/1384166255 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fbbdc078be0 msgr2=0x7fbbdc07b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:59.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.453+0000 7fbbf500a640 1 --2- 172.21.15.38:0/1384166255 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fbbdc078be0 0x7fbbdc07b0f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fbbe4034000 tx=0x7fbbe40046c0 comp rx=0 tx=0).stop 2024-09-21T03:33:59.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.453+0000 7fbbf500a640 1 -- 172.21.15.38:0/1384166255 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbbf0070c80 msgr2=0x7fbbf0150d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:59.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.453+0000 7fbbf500a640 1 --2- 172.21.15.38:0/1384166255 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbbf0070c80 0x7fbbf0150d50 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fbbe000c730 tx=0x7fbbe000cbf0 comp rx=0 tx=0).stop 2024-09-21T03:33:59.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.453+0000 7fbbef59e640 1 -- 172.21.15.38:0/1384166255 reap_dead start 2024-09-21T03:33:59.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.453+0000 7f657dffb640 1 -- 172.21.15.38:0/3508840641 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f658004c020 con 0x7f6588073ea0 2024-09-21T03:33:59.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.453+0000 7fbbf500a640 1 -- 172.21.15.38:0/1384166255 shutdown_connections 2024-09-21T03:33:59.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.453+0000 7fbbf500a640 1 -- 172.21.15.38:0/1384166255 >> 172.21.15.38:0/1384166255 conn(0x7fbbf00d0590 msgr2=0x7fbbf00ce9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:59.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.454+0000 7f657dffb640 1 --2- 172.21.15.38:0/3508840641 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f656c078c60 0x7f656c07b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:59.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.454+0000 7fbbf500a640 1 -- 172.21.15.38:0/1384166255 shutdown_connections 2024-09-21T03:33:59.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.454+0000 7fbbf500a640 1 -- 172.21.15.38:0/1384166255 wait complete. 2024-09-21T03:33:59.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.454+0000 7f657dffb640 1 -- 172.21.15.38:0/3508840641 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f65800c3a50 con 0x7f6588073ea0 2024-09-21T03:33:59.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.454+0000 7f657dffb640 1 --2- 172.21.15.38:0/3508840641 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x7f656c082420 0x7f656c0848b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:59.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.454+0000 7f658d35f640 1 --2- 172.21.15.38:0/3508840641 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f656c078c60 0x7f656c07b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:59.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.454+0000 7f657dffb640 1 -- 172.21.15.38:0/3508840641 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6580046ec0 con 0x7f656c082420 2024-09-21T03:33:59.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.454+0000 7f657dffb640 1 -- 172.21.15.38:0/3508840641 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f65800c82a0 con 0x7f6588073ea0 2024-09-21T03:33:59.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.454+0000 7f658db60640 1 --2- 172.21.15.38:0/3508840641 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x7f656c082420 0x7f656c0848b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:59.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.455+0000 7faab48e4640 1 -- 172.21.15.38:0/1666060122 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7faa90002dd0 con 0x7faa9c0823b0 2024-09-21T03:33:59.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.455+0000 7f658db60640 1 --2- 172.21.15.38:0/3508840641 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x7f656c082420 0x7f656c0848b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:59.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.455+0000 7f658d35f640 1 --2- 172.21.15.38:0/3508840641 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f656c078c60 0x7f656c07b170 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6584009b70 tx=0x7f6584005f90 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:59.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.455+0000 7faa977fe640 1 -- 172.21.15.38:0/1666060122 <== osd.0 v2:172.21.15.38:6800/2381639433 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7faa90002dd0 con 0x7faa9c0823b0 2024-09-21T03:33:59.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.456+0000 7faab48e4640 1 -- 172.21.15.38:0/1666060122 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x7faa9c0823b0 msgr2=0x7faa9c084840 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:59.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.456+0000 7faab48e4640 1 --2- 172.21.15.38:0/1666060122 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x7faa9c0823b0 0x7faa9c084840 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:59.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.456+0000 7f657dffb640 1 -- 172.21.15.38:0/3508840641 <== osd.1 v2:172.21.15.38:6816/1491818732 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6580046ec0 con 0x7f656c082420 2024-09-21T03:33:59.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.458+0000 7faab48e4640 1 -- 172.21.15.38:0/1666060122 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7faa9c078c60 msgr2=0x7faa9c07b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:59.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.458+0000 7faab48e4640 1 --2- 172.21.15.38:0/1666060122 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7faa9c078c60 0x7faa9c07b170 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7faab006bbc0 tx=0x7faab00773f0 comp rx=0 tx=0).stop 2024-09-21T03:33:59.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.458+0000 7faab48e4640 1 -- 172.21.15.38:0/1666060122 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faaa80e1380 msgr2=0x7faaa80328b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:59.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.458+0000 7faab48e4640 1 --2- 172.21.15.38:0/1666060122 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faaa80e1380 0x7faaa80328b0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7faaa400e640 tx=0x7faaa400eb00 comp rx=0 tx=0).stop 2024-09-21T03:33:59.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.458+0000 7f0440e98640 1 -- 172.21.15.38:0/1975697659 --> [v2:172.21.15.195:6810/2080960180,v1:172.21.15.195:6811/2080960180] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f04340d9d00 con 0x7f0420082450 2024-09-21T03:33:59.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.458+0000 7faaaf59e640 1 -- 172.21.15.38:0/1666060122 reap_dead start 2024-09-21T03:33:59.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.458+0000 7faab48e4640 1 -- 172.21.15.38:0/1666060122 shutdown_connections 2024-09-21T03:33:59.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.458+0000 7faab48e4640 1 -- 172.21.15.38:0/1666060122 >> 172.21.15.38:0/1666060122 conn(0x7faaa801eae0 msgr2=0x7faaa800ae50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:59.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.459+0000 7faab48e4640 1 -- 172.21.15.38:0/1666060122 shutdown_connections 2024-09-21T03:33:59.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.459+0000 7faab48e4640 1 -- 172.21.15.38:0/1666060122 wait complete. 2024-09-21T03:33:59.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.459+0000 7f0427fff640 1 -- 172.21.15.38:0/1975697659 <== osd.5 v2:172.21.15.195:6810/2080960180 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f04340d9d00 con 0x7f0420082450 2024-09-21T03:33:59.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.460+0000 7f0440e98640 1 -- 172.21.15.38:0/1975697659 >> [v2:172.21.15.195:6810/2080960180,v1:172.21.15.195:6811/2080960180] conn(0x7f0420082450 msgr2=0x7f04200848e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:59.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.460+0000 7f0440e98640 1 --2- 172.21.15.38:0/1975697659 >> [v2:172.21.15.195:6810/2080960180,v1:172.21.15.195:6811/2080960180] conn(0x7f0420082450 0x7f04200848e0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:59.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.460+0000 7f0440e98640 1 -- 172.21.15.38:0/1975697659 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0420078c90 msgr2=0x7f042007b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:59.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.460+0000 7f0440e98640 1 --2- 172.21.15.38:0/1975697659 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0420078c90 0x7f042007b1a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f043400fa60 tx=0x7f043c0a8000 comp rx=0 tx=0).stop 2024-09-21T03:33:59.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.460+0000 7f0440e98640 1 -- 172.21.15.38:0/1975697659 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0434008fa0 msgr2=0x7f0434051b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:59.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.460+0000 7f0440e98640 1 --2- 172.21.15.38:0/1975697659 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0434008fa0 0x7f0434051b20 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f042800c340 tx=0x7f042800c800 comp rx=0 tx=0).stop 2024-09-21T03:33:59.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.460+0000 7f043bfff640 1 -- 172.21.15.38:0/1975697659 reap_dead start 2024-09-21T03:33:59.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.460+0000 7f0440e98640 1 -- 172.21.15.38:0/1975697659 shutdown_connections 2024-09-21T03:33:59.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.460+0000 7f0440e98640 1 -- 172.21.15.38:0/1975697659 >> 172.21.15.38:0/1975697659 conn(0x7f043401eae0 msgr2=0x7f0434007470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:59.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.460+0000 7f0440e98640 1 -- 172.21.15.38:0/1975697659 shutdown_connections 2024-09-21T03:33:59.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.461+0000 7f0440e98640 1 -- 172.21.15.38:0/1975697659 wait complete. 2024-09-21T03:33:59.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.461+0000 7f1b9ffff640 1 -- 172.21.15.38:0/3885889993 --> [v2:172.21.15.195:6802/3401837571,v1:172.21.15.195:6803/3401837571] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1b9803de80 con 0x7f1b8c082360 2024-09-21T03:33:59.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.462+0000 7f1363fff640 1 -- 172.21.15.38:0/2444085775 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f135c002dd0 con 0x7f1364082450 2024-09-21T03:33:59.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.462+0000 7f13767fc640 1 -- 172.21.15.38:0/2444085775 <== osd.6 v2:172.21.15.195:6826/4055825675 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f135c002dd0 con 0x7f1364082450 2024-09-21T03:33:59.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.462+0000 7f1b877fe640 1 -- 172.21.15.38:0/3885889993 <== osd.4 v2:172.21.15.195:6802/3401837571 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f1b9803de80 con 0x7f1b8c082360 2024-09-21T03:33:59.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.462+0000 7fc7168d8640 1 -- 172.21.15.38:0/3724372677 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc6f8002d50 con 0x7fc6f4082420 2024-09-21T03:33:59.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.463+0000 7f1363fff640 1 -- 172.21.15.38:0/2444085775 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x7f1364082450 msgr2=0x7f13640848e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:59.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.463+0000 7f1363fff640 1 --2- 172.21.15.38:0/2444085775 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x7f1364082450 0x7f13640848e0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:59.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.463+0000 7f1b9ffff640 1 -- 172.21.15.38:0/3885889993 >> [v2:172.21.15.195:6802/3401837571,v1:172.21.15.195:6803/3401837571] conn(0x7f1b8c082360 msgr2=0x7f1b8c0847f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:59.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.463+0000 7f1b9ffff640 1 --2- 172.21.15.38:0/3885889993 >> [v2:172.21.15.195:6802/3401837571,v1:172.21.15.195:6803/3401837571] conn(0x7f1b8c082360 0x7f1b8c0847f0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:59.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.463+0000 7f1363fff640 1 -- 172.21.15.38:0/2444085775 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1364078c90 msgr2=0x7f136407b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:59.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.463+0000 7f1363fff640 1 --2- 172.21.15.38:0/2444085775 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1364078c90 0x7f136407b1a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f1370034000 tx=0x7f1370004730 comp rx=0 tx=0).stop 2024-09-21T03:33:59.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.463+0000 7f1363fff640 1 -- 172.21.15.38:0/2444085775 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1378009ba0 msgr2=0x7f137817d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:59.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.463+0000 7f1363fff640 1 --2- 172.21.15.38:0/2444085775 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1378009ba0 0x7f137817d860 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f137c00b410 tx=0x7f137c00b8d0 comp rx=0 tx=0).stop 2024-09-21T03:33:59.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.463+0000 7f1b9ffff640 1 -- 172.21.15.38:0/3885889993 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1b8c078b90 msgr2=0x7f1b8c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:59.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.463+0000 7f1b9ffff640 1 --2- 172.21.15.38:0/3885889993 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1b8c078b90 0x7f1b8c07b0a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1b94005fd0 tx=0x7f1b9400f530 comp rx=0 tx=0).stop 2024-09-21T03:33:59.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.463+0000 7f13861dd640 1 -- 172.21.15.38:0/2444085775 reap_dead start 2024-09-21T03:33:59.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.463+0000 7fc7067fc640 1 -- 172.21.15.38:0/3724372677 <== osd.3 v2:172.21.15.38:6808/1286810294 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fc6f8002d50 con 0x7fc6f4082420 2024-09-21T03:33:59.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.463+0000 7f1b9ffff640 1 -- 172.21.15.38:0/3885889993 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1b880051c0 msgr2=0x7f1b980e6be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:59.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.463+0000 7f1363fff640 1 -- 172.21.15.38:0/2444085775 shutdown_connections 2024-09-21T03:33:59.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.463+0000 7f1b9ffff640 1 --2- 172.21.15.38:0/3885889993 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1b880051c0 0x7f1b980e6be0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f1b9000c520 tx=0x7f1b9000c9e0 comp rx=0 tx=0).stop 2024-09-21T03:33:59.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.463+0000 7f1363fff640 1 -- 172.21.15.38:0/2444085775 >> 172.21.15.38:0/2444085775 conn(0x7f1378051420 msgr2=0x7f1378006db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:59.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.463+0000 7f1b9f7fe640 1 -- 172.21.15.38:0/3885889993 reap_dead start 2024-09-21T03:33:59.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.464+0000 7fc7168d8640 1 -- 172.21.15.38:0/3724372677 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x7fc6f4082420 msgr2=0x7fc6f40848b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:59.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.464+0000 7fc7168d8640 1 --2- 172.21.15.38:0/3724372677 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x7fc6f4082420 0x7fc6f40848b0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:59.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.464+0000 7f1b9ffff640 1 -- 172.21.15.38:0/3885889993 shutdown_connections 2024-09-21T03:33:59.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.464+0000 7f1b9ffff640 1 -- 172.21.15.38:0/3885889993 >> 172.21.15.38:0/3885889993 conn(0x7f1b98021df0 msgr2=0x7f1b98057010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:59.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.464+0000 7fc7168d8640 1 -- 172.21.15.38:0/3724372677 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc6f4078c60 msgr2=0x7fc6f407b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:59.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.464+0000 7fc7168d8640 1 --2- 172.21.15.38:0/3724372677 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc6f4078c60 0x7fc6f407b170 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fc71007ad70 tx=0x7fc71008d040 comp rx=0 tx=0).stop 2024-09-21T03:33:59.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.464+0000 7f1b9ffff640 1 -- 172.21.15.38:0/3885889993 shutdown_connections 2024-09-21T03:33:59.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.464+0000 7fc7168d8640 1 -- 172.21.15.38:0/3724372677 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc7080429a0 msgr2=0x7fc708182340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:59.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.464+0000 7fc7168d8640 1 --2- 172.21.15.38:0/3724372677 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc7080429a0 0x7fc708182340 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fc6fc00bd90 tx=0x7fc70c0336f0 comp rx=0 tx=0).stop 2024-09-21T03:33:59.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.464+0000 7fc7160d7640 1 -- 172.21.15.38:0/3724372677 reap_dead start 2024-09-21T03:33:59.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.464+0000 7f1b9ffff640 1 -- 172.21.15.38:0/3885889993 wait complete. 2024-09-21T03:33:59.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.464+0000 7fc7168d8640 1 -- 172.21.15.38:0/3724372677 shutdown_connections 2024-09-21T03:33:59.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.464+0000 7fc7168d8640 1 -- 172.21.15.38:0/3724372677 >> 172.21.15.38:0/3724372677 conn(0x7fc708051420 msgr2=0x7fc7080d5220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:59.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.465+0000 7fc7168d8640 1 -- 172.21.15.38:0/3724372677 shutdown_connections 2024-09-21T03:33:59.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.465+0000 7fc7168d8640 1 -- 172.21.15.38:0/3724372677 wait complete. 2024-09-21T03:33:59.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.465+0000 7f1363fff640 1 -- 172.21.15.38:0/2444085775 shutdown_connections 2024-09-21T03:33:59.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.466+0000 7f1363fff640 1 -- 172.21.15.38:0/2444085775 wait complete. 2024-09-21T03:33:59.491 INFO:teuthology.orchestra.run.smithi038.stdout:64424509451 2024-09-21T03:33:59.491 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-21T03:33:59.492 INFO:teuthology.orchestra.run.smithi038.stdout:64424509451 2024-09-21T03:33:59.493 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-21T03:33:59.493 INFO:teuthology.orchestra.run.smithi038.stdout:64424509450 2024-09-21T03:33:59.494 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-21T03:33:59.494 INFO:teuthology.orchestra.run.smithi038.stdout:55834574858 2024-09-21T03:33:59.495 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-21T03:33:59.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.484+0000 7f658f5c3640 1 -- 172.21.15.38:0/3508840641 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f658815e300 con 0x7f656c082420 2024-09-21T03:33:59.495 INFO:teuthology.orchestra.run.smithi038.stdout:64424509450 2024-09-21T03:33:59.496 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-21T03:33:59.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.484+0000 7f657dffb640 1 -- 172.21.15.38:0/3508840641 <== osd.1 v2:172.21.15.38:6816/1491818732 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f658815e300 con 0x7f656c082420 2024-09-21T03:33:59.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.485+0000 7f658f5c3640 1 -- 172.21.15.38:0/3508840641 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x7f656c082420 msgr2=0x7f656c0848b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:59.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.485+0000 7f658f5c3640 1 --2- 172.21.15.38:0/3508840641 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x7f656c082420 0x7f656c0848b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:59.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.485+0000 7f658f5c3640 1 -- 172.21.15.38:0/3508840641 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f656c078c60 msgr2=0x7f656c07b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:59.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.485+0000 7f658f5c3640 1 --2- 172.21.15.38:0/3508840641 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f656c078c60 0x7f656c07b170 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6584009b70 tx=0x7f6584005f90 comp rx=0 tx=0).stop 2024-09-21T03:33:59.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.485+0000 7f658f5c3640 1 -- 172.21.15.38:0/3508840641 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6588073ea0 msgr2=0x7f658814b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:59.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.485+0000 7f658f5c3640 1 --2- 172.21.15.38:0/3508840641 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6588073ea0 0x7f658814b660 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f6588072990 tx=0x7f658000ff90 comp rx=0 tx=0).stop 2024-09-21T03:33:59.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.485+0000 7f658db60640 1 -- 172.21.15.38:0/3508840641 reap_dead start 2024-09-21T03:33:59.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.485+0000 7f658f5c3640 1 -- 172.21.15.38:0/3508840641 shutdown_connections 2024-09-21T03:33:59.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.485+0000 7f658f5c3640 1 -- 172.21.15.38:0/3508840641 >> 172.21.15.38:0/3508840641 conn(0x7f65880d0590 msgr2=0x7f65880cde20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:59.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.485+0000 7f658f5c3640 1 -- 172.21.15.38:0/3508840641 shutdown_connections 2024-09-21T03:33:59.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.485+0000 7f658f5c3640 1 -- 172.21.15.38:0/3508840641 wait complete. 2024-09-21T03:33:59.498 INFO:teuthology.orchestra.run.smithi038.stdout:60129542155 2024-09-21T03:33:59.499 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-21T03:33:59.499 INFO:teuthology.orchestra.run.smithi038.stdout:60129542154 2024-09-21T03:33:59.500 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-21T03:33:59.501 INFO:teuthology.orchestra.run.smithi038.stdout:68719476746 2024-09-21T03:33:59.501 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-21T03:33:59.603 DEBUG:teuthology.orchestra.run.smithi038:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T03:33:59.631 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T03:33:59.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.741+0000 7f27f2ffd640 1 -- 172.21.15.38:0/3191502419 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27ec031bf0 con 0x7f27f4073000 2024-09-21T03:33:59.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.741+0000 7f27fa84d640 1 -- 172.21.15.38:0/3191502419 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f27f4073000 msgr2=0x7f27f40cc160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:59.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.741+0000 7f27fa84d640 1 --2- 172.21.15.38:0/3191502419 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f27f4073000 0x7f27f40cc160 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f27ec0098e0 tx=0x7f27ec02a520 comp rx=0 tx=0).stop 2024-09-21T03:33:59.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.743+0000 7f27fa84d640 1 -- 172.21.15.38:0/3191502419 shutdown_connections 2024-09-21T03:33:59.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.743+0000 7f27fa84d640 1 --2- 172.21.15.38:0/3191502419 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f27f40cf6b0 0x7f27f415f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:59.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.743+0000 7f27fa84d640 1 --2- 172.21.15.38:0/3191502419 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f27f4073000 0x7f27f40cc160 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:59.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.743+0000 7f27fa84d640 1 -- 172.21.15.38:0/3191502419 >> 172.21.15.38:0/3191502419 conn(0x7f27f40b1b90 msgr2=0x7f27f40b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:59.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.744+0000 7f27fa84d640 1 -- 172.21.15.38:0/3191502419 shutdown_connections 2024-09-21T03:33:59.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.744+0000 7f27fa84d640 1 -- 172.21.15.38:0/3191502419 wait complete. 2024-09-21T03:33:59.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.746+0000 7f7f4d09b640 1 -- 172.21.15.38:0/1521462164 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7f480aee70 msgr2=0x7f7f48072f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:59.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.746+0000 7f7f4d09b640 1 --2- 172.21.15.38:0/1521462164 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7f480aee70 0x7f7f48072f90 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f7f38009850 tx=0x7f7f3803b330 comp rx=0 tx=0).stop 2024-09-21T03:33:59.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.746+0000 7f7f4d09b640 1 -- 172.21.15.38:0/1521462164 shutdown_connections 2024-09-21T03:33:59.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.746+0000 7f7f4d09b640 1 --2- 172.21.15.38:0/1521462164 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7f480aee70 0x7f7f48072f90 secure :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f7f38009850 tx=0x7f7f3803b330 comp rx=0 tx=0).stop 2024-09-21T03:33:59.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.746+0000 7f7f4d09b640 1 -- 172.21.15.38:0/1521462164 >> 172.21.15.38:0/1521462164 conn(0x7f7f480b1b90 msgr2=0x7f7f480b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:59.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.746+0000 7f27fa84d640 1 Processor -- start 2024-09-21T03:33:59.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.747+0000 7f27fa84d640 1 -- start start 2024-09-21T03:33:59.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.747+0000 7f7f4d09b640 1 -- 172.21.15.38:0/1521462164 shutdown_connections 2024-09-21T03:33:59.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.747+0000 7f7f4d09b640 1 -- 172.21.15.38:0/1521462164 wait complete. 2024-09-21T03:33:59.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.748+0000 7f8cec74a640 1 -- 172.21.15.38:0/3046185220 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8ce40cc160 msgr2=0x7f8ce415f5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:59.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.748+0000 7f8cec74a640 1 --2- 172.21.15.38:0/3046185220 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8ce40cc160 0x7f8ce415f5b0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f8ce4072d20 tx=0x7f8cdc03bc10 comp rx=0 tx=0).stop 2024-09-21T03:33:59.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.749+0000 7f8cec74a640 1 -- 172.21.15.38:0/3046185220 shutdown_connections 2024-09-21T03:33:59.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.749+0000 7f8cec74a640 1 --2- 172.21.15.38:0/3046185220 >> v2:172.21.15.195:3300/0 conn(0x7f8ce415faf0 0x7f8ce4162450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:59.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.749+0000 7f8cec74a640 1 --2- 172.21.15.38:0/3046185220 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8ce40cc160 0x7f8ce415f5b0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:59.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.749+0000 7f8cec74a640 1 --2- 172.21.15.38:0/3046185220 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8ce40aee70 0x7f8ce4072f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:59.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.749+0000 7f8cec74a640 1 -- 172.21.15.38:0/3046185220 >> 172.21.15.38:0/3046185220 conn(0x7f8ce40b1b90 msgr2=0x7f8ce40b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:59.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.749+0000 7f27fa84d640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f27f4073000 0x7f27f415d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:59.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.749+0000 7f27fa84d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f27f40cf6b0 0x7f27f409bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:59.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.750+0000 7f27fa84d640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f27f40cfac0 0x7f27f409c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:59.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.750+0000 7f27fa84d640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f27f415fe80 con 0x7f27f40cf6b0 2024-09-21T03:33:59.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.750+0000 7f27fa84d640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f27f415fd00 con 0x7f27f4073000 2024-09-21T03:33:59.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.750+0000 7f27f8dea640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f27f40cfac0 0x7f27f409c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:59.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.750+0000 7f27f8dea640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f27f40cfac0 0x7f27f409c510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.38:50544/0 (socket says 172.21.15.38:50544) 2024-09-21T03:33:59.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.750+0000 7f27f8dea640 1 -- 172.21.15.38:0/662123424 learned_addr learned my addr 172.21.15.38:0/662123424 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:33:59.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.750+0000 7f27fa84d640 1 -- 172.21.15.38:0/662123424 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f27f40b4480 con 0x7f27f40cfac0 2024-09-21T03:33:59.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.750+0000 7f7f4d09b640 1 Processor -- start 2024-09-21T03:33:59.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.750+0000 7f7f4d09b640 1 -- start start 2024-09-21T03:33:59.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.751+0000 7f27f8dea640 1 -- 172.21.15.38:0/662123424 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f27f4073000 msgr2=0x7f27f415d550 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T03:33:59.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.751+0000 7f27f8dea640 1 --2- 172.21.15.38:0/662123424 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f27f4073000 0x7f27f415d550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:59.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.751+0000 7f27f8dea640 1 -- 172.21.15.38:0/662123424 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f27f40cf6b0 msgr2=0x7f27f409bfd0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T03:33:59.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.751+0000 7f27f8dea640 1 --2- 172.21.15.38:0/662123424 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f27f40cf6b0 0x7f27f409bfd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:59.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.751+0000 7f27f8dea640 1 -- 172.21.15.38:0/662123424 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27f40b5cc0 con 0x7f27f40cfac0 2024-09-21T03:33:59.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.751+0000 7f27f8dea640 1 --2- 172.21.15.38:0/662123424 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f27f40cfac0 0x7f27f409c510 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f27dc00b470 tx=0x7f27dc00b930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:59.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.751+0000 7f8cec74a640 1 -- 172.21.15.38:0/3046185220 shutdown_connections 2024-09-21T03:33:59.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.751+0000 7f8cec74a640 1 -- 172.21.15.38:0/3046185220 wait complete. 2024-09-21T03:33:59.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.751+0000 7f27f0ff9640 1 -- 172.21.15.38:0/662123424 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27dc015020 con 0x7f27f40cfac0 2024-09-21T03:33:59.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.751+0000 7f27f0ff9640 1 -- 172.21.15.38:0/662123424 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f27dc011920 con 0x7f27f40cfac0 2024-09-21T03:33:59.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.751+0000 7f27f0ff9640 1 -- 172.21.15.38:0/662123424 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27dc011d70 con 0x7f27f40cfac0 2024-09-21T03:33:59.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.752+0000 7f27fa84d640 1 -- 172.21.15.38:0/662123424 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27f40a0c90 con 0x7f27f40cfac0 2024-09-21T03:33:59.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.752+0000 7f27fa84d640 1 -- 172.21.15.38:0/662123424 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f27f415da90 con 0x7f27f40cfac0 2024-09-21T03:33:59.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.753+0000 7f7f4d09b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7f480cc160 0x7f7f4809e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:59.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.753+0000 7f7f4d09b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7f48162450 0x7f7f4809f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:59.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.753+0000 7f7f4d09b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7f4814c9d0 0x7f7f481f5440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:59.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.753+0000 7f7f4d09b640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f7f481629e0 con 0x7f7f4814c9d0 2024-09-21T03:33:59.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.753+0000 7f7f4d09b640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f7f48162860 con 0x7f7f48162450 2024-09-21T03:33:59.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.753+0000 7f7f4d09b640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f7f48162b60 con 0x7f7f480cc160 2024-09-21T03:33:59.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.753+0000 7f7f4759e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7f4814c9d0 0x7f7f481f5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:59.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.753+0000 7f7f4759e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7f4814c9d0 0x7f7f481f5440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42236/0 (socket says 172.21.15.38:42236) 2024-09-21T03:33:59.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.753+0000 7f7f4759e640 1 -- 172.21.15.38:0/2368976307 learned_addr learned my addr 172.21.15.38:0/2368976307 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:33:59.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.753+0000 7f7f46d9d640 1 --2- 172.21.15.38:0/2368976307 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7f480cc160 0x7f7f4809e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:59.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.752+0000 7fe2b05c6640 1 -- 172.21.15.38:0/2928311194 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe2a806ddf0 msgr2=0x7fe2a806e200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:59.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.752+0000 7fe2b05c6640 1 --2- 172.21.15.38:0/2928311194 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe2a806ddf0 0x7fe2a806e200 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe2a815b060 tx=0x7fe29802b170 comp rx=0 tx=0).stop 2024-09-21T03:33:59.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.753+0000 7fe2b05c6640 1 -- 172.21.15.38:0/2928311194 shutdown_connections 2024-09-21T03:33:59.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.753+0000 7fe2b05c6640 1 --2- 172.21.15.38:0/2928311194 >> v2:172.21.15.195:3300/0 conn(0x7fe2a8072d90 0x7fe2a80731a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:59.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.753+0000 7fe2b05c6640 1 --2- 172.21.15.38:0/2928311194 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe2a806ddf0 0x7fe2a806e200 secure :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe2a815b060 tx=0x7fe29802b170 comp rx=0 tx=0).stop 2024-09-21T03:33:59.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.753+0000 7fe2b05c6640 1 -- 172.21.15.38:0/2928311194 >> 172.21.15.38:0/2928311194 conn(0x7fe2a80b1b90 msgr2=0x7fe2a80b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:59.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.753+0000 7f7f4759e640 1 -- 172.21.15.38:0/2368976307 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7f480cc160 msgr2=0x7f7f4809e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:59.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.753+0000 7f7f4759e640 1 --2- 172.21.15.38:0/2368976307 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7f480cc160 0x7f7f4809e380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:59.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.753+0000 7f7f4759e640 1 -- 172.21.15.38:0/2368976307 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7f48162450 msgr2=0x7f7f4809f1b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T03:33:59.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.753+0000 7f7f4759e640 1 --2- 172.21.15.38:0/2368976307 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7f48162450 0x7f7f4809f1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:59.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.753+0000 7f7f4759e640 1 -- 172.21.15.38:0/2368976307 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f4814d140 con 0x7f7f4814c9d0 2024-09-21T03:33:59.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.754+0000 7f7f4759e640 1 --2- 172.21.15.38:0/2368976307 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7f4814c9d0 0x7f7f481f5440 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f7f4000c7d0 tx=0x7f7f4000cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:59.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.754+0000 7f8cec74a640 1 Processor -- start 2024-09-21T03:33:59.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.754+0000 7f7f3f7fe640 1 -- 172.21.15.38:0/2368976307 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f40003da0 con 0x7f7f4814c9d0 2024-09-21T03:33:59.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.754+0000 7f7f3f7fe640 1 -- 172.21.15.38:0/2368976307 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7f40005070 con 0x7f7f4814c9d0 2024-09-21T03:33:59.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.754+0000 7f27fa84d640 1 -- 172.21.15.38:0/662123424 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27f40cc160 con 0x7f27f40cfac0 2024-09-21T03:33:59.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.754+0000 7f7f4d09b640 1 -- 172.21.15.38:0/2368976307 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f4814d420 con 0x7f7f4814c9d0 2024-09-21T03:33:59.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.754+0000 7f7f3f7fe640 1 -- 172.21.15.38:0/2368976307 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f400054c0 con 0x7f7f4814c9d0 2024-09-21T03:33:59.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.755+0000 7f767dfbb640 1 -- 172.21.15.38:0/2751390245 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7670056da0 msgr2=0x7f7670057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:59.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.755+0000 7f767dfbb640 1 --2- 172.21.15.38:0/2751390245 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7670056da0 0x7f7670057c80 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f7670011f00 tx=0x7f766c02a320 comp rx=0 tx=0).stop 2024-09-21T03:33:59.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.754+0000 7f7f4d09b640 1 -- 172.21.15.38:0/2368976307 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f4809e9c0 con 0x7f7f4814c9d0 2024-09-21T03:33:59.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.756+0000 7f767dfbb640 1 -- 172.21.15.38:0/2751390245 shutdown_connections 2024-09-21T03:33:59.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.756+0000 7f767dfbb640 1 --2- 172.21.15.38:0/2751390245 >> v2:172.21.15.195:3300/0 conn(0x7f76700e2f40 0x7f767003c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:59.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.756+0000 7f767dfbb640 1 --2- 172.21.15.38:0/2751390245 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7670056da0 0x7f7670057c80 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:59.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.756+0000 7f767dfbb640 1 -- 172.21.15.38:0/2751390245 >> 172.21.15.38:0/2751390245 conn(0x7f767003a5f0 msgr2=0x7f767003aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:59.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.756+0000 7f27f0ff9640 1 -- 172.21.15.38:0/662123424 <== mon.2 v2:172.21.15.195:3301/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f27dc018020 con 0x7f27f40cfac0 2024-09-21T03:33:59.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.757+0000 7f767dfbb640 1 -- 172.21.15.38:0/2751390245 shutdown_connections 2024-09-21T03:33:59.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.757+0000 7f27f0ff9640 1 --2- 172.21.15.38:0/662123424 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f27d4078c10 0x7f27d407b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:59.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.757+0000 7f27f0ff9640 1 -- 172.21.15.38:0/662123424 <== mon.2 v2:172.21.15.195:3301/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f27dc09a840 con 0x7f27f40cfac0 2024-09-21T03:33:59.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.757+0000 7f767dfbb640 1 -- 172.21.15.38:0/2751390245 wait complete. 2024-09-21T03:33:59.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.756+0000 7f8cec74a640 1 -- start start 2024-09-21T03:33:59.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.756+0000 7f7f3d7fa640 1 -- 172.21.15.38:0/2368976307 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f24005290 con 0x7f7f4814c9d0 2024-09-21T03:33:59.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.758+0000 7f27f3fff640 1 --2- 172.21.15.38:0/662123424 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f27d4078c10 0x7f27d407b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:59.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.758+0000 7f8cec74a640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8ce40aee70 0x7f8ce40c2f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:59.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.758+0000 7f8cec74a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8ce40cc160 0x7f8ce409b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:59.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.758+0000 7f8cea4e6640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8ce40aee70 0x7f8ce40c2f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:59.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.758+0000 7f8cea4e6640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8ce40aee70 0x7f8ce40c2f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.38:50562/0 (socket says 172.21.15.38:50562) 2024-09-21T03:33:59.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.756+0000 7fe2b05c6640 1 -- 172.21.15.38:0/2928311194 shutdown_connections 2024-09-21T03:33:59.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.757+0000 7fe2b05c6640 1 -- 172.21.15.38:0/2928311194 wait complete. 2024-09-21T03:33:59.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.759+0000 7f8ce9ce5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8ce40cc160 0x7f8ce409b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:59.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.759+0000 7f8ce9ce5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8ce40cc160 0x7f8ce409b8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42250/0 (socket says 172.21.15.38:42250) 2024-09-21T03:33:59.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.759+0000 7f8cea4e6640 1 -- 172.21.15.38:0/2769756428 learned_addr learned my addr 172.21.15.38:0/2769756428 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:33:59.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.758+0000 7f7f3f7fe640 1 -- 172.21.15.38:0/2368976307 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f7f40005660 con 0x7f7f4814c9d0 2024-09-21T03:33:59.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.759+0000 7f7f3f7fe640 1 --2- 172.21.15.38:0/2368976307 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7f2c078b90 0x7f7f2c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:59.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.759+0000 7f7f3f7fe640 1 -- 172.21.15.38:0/2368976307 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f7f4009afe0 con 0x7f7f4814c9d0 2024-09-21T03:33:59.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.760+0000 7fe2b05c6640 1 Processor -- start 2024-09-21T03:33:59.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.760+0000 7f767dfbb640 1 Processor -- start 2024-09-21T03:33:59.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.760+0000 7f7f46d9d640 1 --2- 172.21.15.38:0/2368976307 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7f2c078b90 0x7f7f2c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:59.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.760+0000 7f767dfbb640 1 -- start start 2024-09-21T03:33:59.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.759+0000 7f8cec74a640 1 --2- 172.21.15.38:0/2769756428 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8ce415faf0 0x7f8ce409be10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:59.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.759+0000 7f8cec74a640 1 -- 172.21.15.38:0/2769756428 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f8ce406ddf0 con 0x7f8ce40cc160 2024-09-21T03:33:59.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.759+0000 7f8cec74a640 1 -- 172.21.15.38:0/2769756428 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f8ce4162d10 con 0x7f8ce415faf0 2024-09-21T03:33:59.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.759+0000 7f8cec74a640 1 -- 172.21.15.38:0/2769756428 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f8ce406df70 con 0x7f8ce40aee70 2024-09-21T03:33:59.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.760+0000 7f27f3fff640 1 --2- 172.21.15.38:0/662123424 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f27d4078c10 0x7f27d407b120 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f27ec034000 tx=0x7f27ec004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:59.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.760+0000 7f8cea4e6640 1 -- 172.21.15.38:0/2769756428 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8ce415faf0 msgr2=0x7f8ce409be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:59.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.760+0000 7f8cea4e6640 1 --2- 172.21.15.38:0/2769756428 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8ce415faf0 0x7f8ce409be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:59.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.760+0000 7f8cea4e6640 1 -- 172.21.15.38:0/2769756428 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8ce40cc160 msgr2=0x7f8ce409b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:59.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.760+0000 7f8cea4e6640 1 --2- 172.21.15.38:0/2769756428 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8ce40cc160 0x7f8ce409b8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:59.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.760+0000 7f8cea4e6640 1 -- 172.21.15.38:0/2769756428 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ce409b000 con 0x7f8ce40aee70 2024-09-21T03:33:59.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.760+0000 7f8ce9ce5640 1 --2- 172.21.15.38:0/2769756428 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8ce40cc160 0x7f8ce409b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:33:59.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.760+0000 7f8cea4e6640 1 --2- 172.21.15.38:0/2769756428 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8ce40aee70 0x7f8ce40c2f90 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f8ce000ea20 tx=0x7f8ce000eee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:59.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.760+0000 7fe2b05c6640 1 -- start start 2024-09-21T03:33:59.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.761+0000 7f7f46d9d640 1 --2- 172.21.15.38:0/2368976307 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7f2c078b90 0x7f7f2c07b0a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f7f38045000 tx=0x7f7f3803d040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:59.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.762+0000 7fe2b05c6640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe2a8072d90 0x7fe2a80b5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:59.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.762+0000 7fe2b05c6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe2a815ac20 0x7fe2a81595f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:59.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.762+0000 7fe2b05c6640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe2a8202dc0 0x7fe2a8159b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:59.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.762+0000 7f8cdaffd640 1 -- 172.21.15.38:0/2769756428 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ce000ccc0 con 0x7f8ce40aee70 2024-09-21T03:33:59.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.762+0000 7f8cdaffd640 1 -- 172.21.15.38:0/2769756428 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8ce0010430 con 0x7f8ce40aee70 2024-09-21T03:33:59.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.762+0000 7f8cec74a640 1 -- 172.21.15.38:0/2769756428 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ce40b3bc0 con 0x7f8ce40aee70 2024-09-21T03:33:59.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.762+0000 7f8cec74a640 1 -- 172.21.15.38:0/2769756428 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ce40b4180 con 0x7f8ce40aee70 2024-09-21T03:33:59.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.762+0000 7f767dfbb640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7670053c50 0x7f767004f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:59.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.762+0000 7f27f0ff9640 1 -- 172.21.15.38:0/662123424 <== mon.2 v2:172.21.15.195:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f27dc01f070 con 0x7f27f40cfac0 2024-09-21T03:33:59.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.762+0000 7f767dfbb640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7670056da0 0x7f76700d83c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:59.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.762+0000 7f767dfbb640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f76700e2f40 0x7f7670039860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:59.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.762+0000 7fe2b05c6640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fe2a80736e0 con 0x7fe2a815ac20 2024-09-21T03:33:59.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.762+0000 7fe2b05c6640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fe2a8071090 con 0x7fe2a8202dc0 2024-09-21T03:33:59.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.762+0000 7fe2b05c6640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fe2a8073860 con 0x7fe2a8072d90 2024-09-21T03:33:59.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.762+0000 7fe2ae362640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe2a8072d90 0x7fe2a80b5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:59.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.762+0000 7fe2ae362640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe2a8072d90 0x7fe2a80b5870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.38:50566/0 (socket says 172.21.15.38:50566) 2024-09-21T03:33:59.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.762+0000 7fe2ae362640 1 -- 172.21.15.38:0/2592595084 learned_addr learned my addr 172.21.15.38:0/2592595084 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:33:59.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.762+0000 7fe2adb61640 1 --2- 172.21.15.38:0/2592595084 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe2a815ac20 0x7fe2a81595f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:59.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.763+0000 7fe2aeb63640 1 --2- 172.21.15.38:0/2592595084 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe2a8202dc0 0x7fe2a8159b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:59.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.763+0000 7fe2ae362640 1 -- 172.21.15.38:0/2592595084 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe2a8202dc0 msgr2=0x7fe2a8159b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:59.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.763+0000 7fe2ae362640 1 --2- 172.21.15.38:0/2592595084 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe2a8202dc0 0x7fe2a8159b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:59.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.763+0000 7fe2ae362640 1 -- 172.21.15.38:0/2592595084 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe2a815ac20 msgr2=0x7fe2a81595f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:59.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.764+0000 7fe2ae362640 1 --2- 172.21.15.38:0/2592595084 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe2a815ac20 0x7fe2a81595f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:59.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.764+0000 7fe2ae362640 1 -- 172.21.15.38:0/2592595084 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2a80b3f20 con 0x7fe2a8072d90 2024-09-21T03:33:59.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.764+0000 7fe2adb61640 1 --2- 172.21.15.38:0/2592595084 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe2a815ac20 0x7fe2a81595f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:33:59.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.763+0000 7f7677fff640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7670056da0 0x7f76700d83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:59.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.763+0000 7f7677fff640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7670056da0 0x7f76700d83c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.38:54792/0 (socket says 172.21.15.38:54792) 2024-09-21T03:33:59.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.763+0000 7f767dfbb640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f7670007100 con 0x7f76700e2f40 2024-09-21T03:33:59.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.763+0000 7f767dfbb640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f7670006f80 con 0x7f7670056da0 2024-09-21T03:33:59.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.763+0000 7f767dfbb640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f7670007280 con 0x7f7670053c50 2024-09-21T03:33:59.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.763+0000 7f767cfb9640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7670053c50 0x7f767004f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:59.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.763+0000 7f767cfb9640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7670053c50 0x7f767004f250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.38:50584/0 (socket says 172.21.15.38:50584) 2024-09-21T03:33:59.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.763+0000 7f767cfb9640 1 -- 172.21.15.38:0/2102433518 learned_addr learned my addr 172.21.15.38:0/2102433518 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:33:59.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.764+0000 7f767d7ba640 1 --2- 172.21.15.38:0/2102433518 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f76700e2f40 0x7f7670039860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:59.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.764+0000 7f767cfb9640 1 -- 172.21.15.38:0/2102433518 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7670056da0 msgr2=0x7f76700d83c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:59.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.764+0000 7f767cfb9640 1 --2- 172.21.15.38:0/2102433518 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7670056da0 0x7f76700d83c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:59.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.764+0000 7f767cfb9640 1 -- 172.21.15.38:0/2102433518 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f76700e2f40 msgr2=0x7f7670039860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:59.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.764+0000 7f8cdaffd640 1 -- 172.21.15.38:0/2769756428 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ce0010820 con 0x7f8ce40aee70 2024-09-21T03:33:59.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.765+0000 7fe2ae362640 1 --2- 172.21.15.38:0/2592595084 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe2a8072d90 0x7fe2a80b5870 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe2a000bab0 tx=0x7fe2a000bf70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:59.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.765+0000 7fe29effd640 1 -- 172.21.15.38:0/2592595084 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2a0019070 con 0x7fe2a8072d90 2024-09-21T03:33:59.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.765+0000 7f8cdaffd640 1 -- 172.21.15.38:0/2769756428 <== mon.2 v2:172.21.15.195:3301/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f8ce00109c0 con 0x7f8ce40aee70 2024-09-21T03:33:59.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.765+0000 7fe29effd640 1 -- 172.21.15.38:0/2592595084 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe2a000cd80 con 0x7fe2a8072d90 2024-09-21T03:33:59.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.766+0000 7fe2b05c6640 1 -- 172.21.15.38:0/2592595084 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2a80b4450 con 0x7fe2a8072d90 2024-09-21T03:33:59.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.766+0000 7f8cec74a640 1 -- 172.21.15.38:0/2769756428 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ccc005290 con 0x7f8ce40aee70 2024-09-21T03:33:59.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.766+0000 7fe2b05c6640 1 -- 172.21.15.38:0/2592595084 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2a80b4a10 con 0x7fe2a8072d90 2024-09-21T03:33:59.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.766+0000 7f767cfb9640 1 --2- 172.21.15.38:0/2102433518 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f76700e2f40 0x7f7670039860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:59.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.766+0000 7f767cfb9640 1 -- 172.21.15.38:0/2102433518 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f767017ed00 con 0x7f7670053c50 2024-09-21T03:33:59.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.766+0000 7f8cdaffd640 1 --2- 172.21.15.38:0/2769756428 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8cc8078c60 0x7f8cc807b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:59.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.767+0000 7f8ce9ce5640 1 --2- 172.21.15.38:0/2769756428 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8cc8078c60 0x7f8cc807b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:59.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.766+0000 7fe29effd640 1 -- 172.21.15.38:0/2592595084 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2a00146a0 con 0x7fe2a8072d90 2024-09-21T03:33:59.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.766+0000 7f767d7ba640 1 --2- 172.21.15.38:0/2102433518 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f76700e2f40 0x7f7670039860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:33:59.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.766+0000 7eff128b6640 1 -- 172.21.15.38:0/4035208601 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7eff0c15ac20 msgr2=0x7eff0c070070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:59.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.766+0000 7eff128b6640 1 --2- 172.21.15.38:0/4035208601 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7eff0c15ac20 0x7eff0c070070 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7eff00009900 tx=0x7eff0002a4d0 comp rx=0 tx=0).stop 2024-09-21T03:33:59.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.766+0000 7eff128b6640 1 -- 172.21.15.38:0/4035208601 shutdown_connections 2024-09-21T03:33:59.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.766+0000 7eff128b6640 1 --2- 172.21.15.38:0/4035208601 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7eff0c072d90 0x7eff0c0731a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:59.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.766+0000 7eff128b6640 1 --2- 172.21.15.38:0/4035208601 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7eff0c15ac20 0x7eff0c070070 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:59.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.766+0000 7eff128b6640 1 -- 172.21.15.38:0/4035208601 >> 172.21.15.38:0/4035208601 conn(0x7eff0c0b1b90 msgr2=0x7eff0c0b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:59.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.766+0000 7eff128b6640 1 -- 172.21.15.38:0/4035208601 shutdown_connections 2024-09-21T03:33:59.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.766+0000 7eff128b6640 1 -- 172.21.15.38:0/4035208601 wait complete. 2024-09-21T03:33:59.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.767+0000 7f8cdaffd640 1 -- 172.21.15.38:0/2769756428 <== mon.2 v2:172.21.15.195:3301/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f8ce009bcc0 con 0x7f8ce40aee70 2024-09-21T03:33:59.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.767+0000 7fbf78a0e640 1 -- 172.21.15.38:0/1091026849 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbf7406ddf0 msgr2=0x7fbf7406e200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:59.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.767+0000 7fbf78a0e640 1 --2- 172.21.15.38:0/1091026849 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbf7406ddf0 0x7fbf7406e200 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fbf74074eb0 tx=0x7fbf6802a560 comp rx=0 tx=0).stop 2024-09-21T03:33:59.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.768+0000 7fbf78a0e640 1 -- 172.21.15.38:0/1091026849 shutdown_connections 2024-09-21T03:33:59.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.768+0000 7fbf78a0e640 1 --2- 172.21.15.38:0/1091026849 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbf740ceca0 0x7fbf740cf0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:59.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.768+0000 7fbf78a0e640 1 --2- 172.21.15.38:0/1091026849 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbf7406ddf0 0x7fbf7406e200 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:59.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.768+0000 7fbf78a0e640 1 --2- 172.21.15.38:0/1091026849 >> v2:172.21.15.38:3300/0 conn(0x7fbf740d02b0 0x7fbf740caf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:59.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.767+0000 7fe29cff9640 1 -- 172.21.15.38:0/2592595084 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2a8073e70 con 0x7fe2a8072d90 2024-09-21T03:33:59.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.767+0000 7f767cfb9640 1 --2- 172.21.15.38:0/2102433518 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7670053c50 0x7f767004f250 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f766c000c00 tx=0x7f766c003b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:59.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.767+0000 7f7f3f7fe640 1 -- 172.21.15.38:0/2368976307 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f7f40062fc0 con 0x7f7f4814c9d0 2024-09-21T03:33:59.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.768+0000 7f8ce9ce5640 1 --2- 172.21.15.38:0/2769756428 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8cc8078c60 0x7f8cc807b170 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8cdc046000 tx=0x7f8cdc03e040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:59.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.768+0000 7fbf78a0e640 1 -- 172.21.15.38:0/1091026849 >> 172.21.15.38:0/1091026849 conn(0x7fbf740735d0 msgr2=0x7fbf740739e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:59.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.768+0000 7fbf78a0e640 1 -- 172.21.15.38:0/1091026849 shutdown_connections 2024-09-21T03:33:59.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.768+0000 7fbf78a0e640 1 -- 172.21.15.38:0/1091026849 wait complete. 2024-09-21T03:33:59.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.768+0000 7fe29effd640 1 -- 172.21.15.38:0/2592595084 <== mon.2 v2:172.21.15.195:3301/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fe2a001c070 con 0x7fe2a8072d90 2024-09-21T03:33:59.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.769+0000 7fe29effd640 1 --2- 172.21.15.38:0/2592595084 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fe28c078c10 0x7fe28c07b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:59.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.769+0000 7fe29effd640 1 -- 172.21.15.38:0/2592595084 <== mon.2 v2:172.21.15.195:3301/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fe2a009ac60 con 0x7fe2a8072d90 2024-09-21T03:33:59.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.769+0000 7fe2adb61640 1 --2- 172.21.15.38:0/2592595084 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fe28c078c10 0x7fe28c07b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:59.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.770+0000 7eff128b6640 1 Processor -- start 2024-09-21T03:33:59.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.770+0000 7eff128b6640 1 -- start start 2024-09-21T03:33:59.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.770+0000 7fe2adb61640 1 --2- 172.21.15.38:0/2592595084 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fe28c078c10 0x7fe28c07b120 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fe298033000 tx=0x7fe298004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:59.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.770+0000 7fbf78a0e640 1 Processor -- start 2024-09-21T03:33:59.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.770+0000 7fbf78a0e640 1 -- start start 2024-09-21T03:33:59.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.772+0000 7fbf78a0e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbf7406ddf0 0x7fbf740c7d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:59.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.772+0000 7fbf78a0e640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbf740ceca0 0x7fbf74159eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:59.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.772+0000 7fbf78a0e640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbf740d02b0 0x7fbf74147820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:59.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.772+0000 7fbf7259c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbf7406ddf0 0x7fbf740c7d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:59.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.772+0000 7fbf78a0e640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fbf740696c0 con 0x7fbf7406ddf0 2024-09-21T03:33:59.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.772+0000 7fbf78a0e640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fbf740cb690 con 0x7fbf740d02b0 2024-09-21T03:33:59.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.772+0000 7f76757fa640 1 -- 172.21.15.38:0/2102433518 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f766c040020 con 0x7f7670053c50 2024-09-21T03:33:59.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.772+0000 7fbf78a0e640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fbf74069840 con 0x7fbf740ceca0 2024-09-21T03:33:59.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.772+0000 7fbf7259c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbf7406ddf0 0x7fbf740c7d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42276/0 (socket says 172.21.15.38:42276) 2024-09-21T03:33:59.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.772+0000 7fbf7259c640 1 -- 172.21.15.38:0/3114749633 learned_addr learned my addr 172.21.15.38:0/3114749633 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:33:59.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.772+0000 7eff128b6640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7eff0c06ddf0 0x7eff0c0a0c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:59.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.772+0000 7eff128b6640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7eff0c072d90 0x7eff0c0c6370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:59.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.772+0000 7eff0bfff640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7eff0c06ddf0 0x7eff0c0a0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:59.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.772+0000 7eff0bfff640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7eff0c06ddf0 0x7eff0c0a0c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.38:50594/0 (socket says 172.21.15.38:50594) 2024-09-21T03:33:59.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.772+0000 7eff128b6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7eff0c15ac20 0x7eff0c1f4910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:59.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.772+0000 7f8cdaffd640 1 -- 172.21.15.38:0/2769756428 <== mon.2 v2:172.21.15.195:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f8ce0063da0 con 0x7f8ce40aee70 2024-09-21T03:33:59.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.773+0000 7eff0b7fe640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7eff0c072d90 0x7eff0c0c6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:59.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.773+0000 7eff0b7fe640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7eff0c072d90 0x7eff0c0c6370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.38:54818/0 (socket says 172.21.15.38:54818) 2024-09-21T03:33:59.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.773+0000 7eff0bfff640 1 -- 172.21.15.38:0/129607063 learned_addr learned my addr 172.21.15.38:0/129607063 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:33:59.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.773+0000 7eff128b6640 1 -- 172.21.15.38:0/129607063 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7eff0c070750 con 0x7eff0c15ac20 2024-09-21T03:33:59.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.773+0000 7eff128b6640 1 -- 172.21.15.38:0/129607063 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7eff0c071090 con 0x7eff0c072d90 2024-09-21T03:33:59.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.773+0000 7eff128b6640 1 -- 172.21.15.38:0/129607063 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7eff0c0736e0 con 0x7eff0c06ddf0 2024-09-21T03:33:59.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.773+0000 7f767dfbb640 1 -- 172.21.15.38:0/2102433518 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f767017efe0 con 0x7f7670053c50 2024-09-21T03:33:59.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.773+0000 7eff10e53640 1 --2- 172.21.15.38:0/129607063 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7eff0c15ac20 0x7eff0c1f4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:59.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.773+0000 7fbf7259c640 1 -- 172.21.15.38:0/3114749633 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbf740ceca0 msgr2=0x7fbf74159eb0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T03:33:59.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.773+0000 7fbf7259c640 1 --2- 172.21.15.38:0/3114749633 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbf740ceca0 0x7fbf74159eb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:59.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.773+0000 7fbf7259c640 1 -- 172.21.15.38:0/3114749633 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbf740d02b0 msgr2=0x7fbf74147820 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T03:33:59.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.773+0000 7fbf7259c640 1 --2- 172.21.15.38:0/3114749633 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbf740d02b0 0x7fbf74147820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:59.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.773+0000 7fbf7259c640 1 -- 172.21.15.38:0/3114749633 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf740c8240 con 0x7fbf7406ddf0 2024-09-21T03:33:59.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.773+0000 7f767dfbb640 1 -- 172.21.15.38:0/2102433518 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f767017f5a0 con 0x7f7670053c50 2024-09-21T03:33:59.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.773+0000 7fbf7259c640 1 --2- 172.21.15.38:0/3114749633 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbf7406ddf0 0x7fbf740c7d00 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fbf6400d740 tx=0x7fbf6400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:59.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.773+0000 7eff0bfff640 1 -- 172.21.15.38:0/129607063 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7eff0c072d90 msgr2=0x7eff0c0c6370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:59.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.773+0000 7eff0bfff640 1 --2- 172.21.15.38:0/129607063 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7eff0c072d90 0x7eff0c0c6370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:59.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.773+0000 7eff0bfff640 1 -- 172.21.15.38:0/129607063 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7eff0c15ac20 msgr2=0x7eff0c1f4910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:59.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.773+0000 7eff0bfff640 1 --2- 172.21.15.38:0/129607063 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7eff0c15ac20 0x7eff0c1f4910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:59.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.773+0000 7eff0bfff640 1 -- 172.21.15.38:0/129607063 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff0c0c7eb0 con 0x7eff0c06ddf0 2024-09-21T03:33:59.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.774+0000 7fbf62ffd640 1 -- 172.21.15.38:0/3114749633 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf64018070 con 0x7fbf7406ddf0 2024-09-21T03:33:59.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.774+0000 7fbf62ffd640 1 -- 172.21.15.38:0/3114749633 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbf64009d20 con 0x7fbf7406ddf0 2024-09-21T03:33:59.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.774+0000 7fbf62ffd640 1 -- 172.21.15.38:0/3114749633 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf640129c0 con 0x7fbf7406ddf0 2024-09-21T03:33:59.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.774+0000 7fbf78a0e640 1 -- 172.21.15.38:0/3114749633 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf740cac90 con 0x7fbf7406ddf0 2024-09-21T03:33:59.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.775+0000 7eff0bfff640 1 --2- 172.21.15.38:0/129607063 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7eff0c06ddf0 0x7eff0c0a0c80 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7eff00034040 tx=0x7eff00003b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:59.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.775+0000 7f767dfbb640 1 -- 172.21.15.38:0/2102433518 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7670007820 con 0x7f7670053c50 2024-09-21T03:33:59.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.775+0000 7fbf78a0e640 1 -- 172.21.15.38:0/3114749633 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf74145cc0 con 0x7fbf7406ddf0 2024-09-21T03:33:59.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.776+0000 7fbf78a0e640 1 -- 172.21.15.38:0/3114749633 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf740cf0b0 con 0x7fbf7406ddf0 2024-09-21T03:33:59.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.777+0000 7fbf62ffd640 1 -- 172.21.15.38:0/3114749633 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fbf64012b60 con 0x7fbf7406ddf0 2024-09-21T03:33:59.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.777+0000 7eff08ff9640 1 -- 172.21.15.38:0/129607063 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff0003e070 con 0x7eff0c06ddf0 2024-09-21T03:33:59.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.777+0000 7fe29effd640 1 -- 172.21.15.38:0/2592595084 <== mon.2 v2:172.21.15.195:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fe2a0062880 con 0x7fe2a8072d90 2024-09-21T03:33:59.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.777+0000 7fbf62ffd640 1 --2- 172.21.15.38:0/3114749633 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fbf54078be0 0x7fbf5407b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:59.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.778+0000 7fbf62ffd640 1 -- 172.21.15.38:0/3114749633 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fbf64062db0 con 0x7fbf7406ddf0 2024-09-21T03:33:59.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.777+0000 7f76757fa640 1 -- 172.21.15.38:0/2102433518 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f766c032440 con 0x7f7670053c50 2024-09-21T03:33:59.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.778+0000 7f76757fa640 1 -- 172.21.15.38:0/2102433518 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f766c040020 con 0x7f7670053c50 2024-09-21T03:33:59.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.779+0000 7eff08ff9640 1 -- 172.21.15.38:0/129607063 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7eff00004ab0 con 0x7eff0c06ddf0 2024-09-21T03:33:59.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.779+0000 7eff08ff9640 1 -- 172.21.15.38:0/129607063 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff00035040 con 0x7eff0c06ddf0 2024-09-21T03:33:59.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.779+0000 7eff128b6640 1 -- 172.21.15.38:0/129607063 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff0c0c68b0 con 0x7eff0c06ddf0 2024-09-21T03:33:59.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.780+0000 7fbf71d9b640 1 --2- 172.21.15.38:0/3114749633 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fbf54078be0 0x7fbf5407b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:59.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.780+0000 7eff128b6640 1 -- 172.21.15.38:0/129607063 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7eff0c0c70f0 con 0x7eff0c06ddf0 2024-09-21T03:33:59.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.780+0000 7f76757fa640 1 -- 172.21.15.38:0/2102433518 <== mon.2 v2:172.21.15.195:3301/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f766c043020 con 0x7f7670053c50 2024-09-21T03:33:59.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.781+0000 7f76757fa640 1 --2- 172.21.15.38:0/2102433518 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f765c078d90 0x7f765c07b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:59.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.781+0000 7f76757fa640 1 -- 172.21.15.38:0/2102433518 <== mon.2 v2:172.21.15.195:3301/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f766c0bb420 con 0x7f7670053c50 2024-09-21T03:33:59.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.782+0000 7f7677fff640 1 --2- 172.21.15.38:0/2102433518 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f765c078d90 0x7f765c07b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:59.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.782+0000 7eff128b6640 1 -- 172.21.15.38:0/129607063 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff0c0af7e0 con 0x7eff0c06ddf0 2024-09-21T03:33:59.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.782+0000 7fbf71d9b640 1 --2- 172.21.15.38:0/3114749633 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fbf54078be0 0x7fbf5407b0f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fbf68033000 tx=0x7fbf68004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:59.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.782+0000 7f76757fa640 1 -- 172.21.15.38:0/2102433518 <== mon.2 v2:172.21.15.195:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f766c04a070 con 0x7f7670053c50 2024-09-21T03:33:59.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.783+0000 7f7677fff640 1 --2- 172.21.15.38:0/2102433518 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f765c078d90 0x7f765c07b2a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f767003a3f0 tx=0x7f7664009060 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:59.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.785+0000 7eff08ff9640 1 -- 172.21.15.38:0/129607063 <== mon.2 v2:172.21.15.195:3301/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7eff00041070 con 0x7eff0c06ddf0 2024-09-21T03:33:59.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.786+0000 7eff08ff9640 1 --2- 172.21.15.38:0/129607063 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7efef8078b90 0x7efef807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:59.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.786+0000 7eff0b7fe640 1 --2- 172.21.15.38:0/129607063 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7efef8078b90 0x7efef807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:59.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.786+0000 7eff08ff9640 1 -- 172.21.15.38:0/129607063 <== mon.2 v2:172.21.15.195:3301/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7eff000bc0b0 con 0x7eff0c06ddf0 2024-09-21T03:33:59.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.787+0000 7eff0b7fe640 1 --2- 172.21.15.38:0/129607063 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7efef8078b90 0x7efef807b0a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7eff0c0cc4c0 tx=0x7efefc009060 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:59.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.787+0000 7fbf62ffd640 1 -- 172.21.15.38:0/3114749633 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fbf64067240 con 0x7fbf7406ddf0 2024-09-21T03:33:59.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.788+0000 7eff08ff9640 1 -- 172.21.15.38:0/129607063 <== mon.2 v2:172.21.15.195:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7eff00083b40 con 0x7eff0c06ddf0 2024-09-21T03:33:59.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.801+0000 7f568359e640 1 -- 172.21.15.38:0/2518866795 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f567c008fa0 msgr2=0x7f567c0dfe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:59.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.801+0000 7f568359e640 1 --2- 172.21.15.38:0/2518866795 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f567c008fa0 0x7f567c0dfe80 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f568406bbc0 tx=0x7f56840aa6e0 comp rx=0 tx=0).stop 2024-09-21T03:33:59.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.801+0000 7f568359e640 1 -- 172.21.15.38:0/2518866795 shutdown_connections 2024-09-21T03:33:59.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.801+0000 7f568359e640 1 --2- 172.21.15.38:0/2518866795 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f567c008fa0 0x7f567c0dfe80 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:59.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.801+0000 7f568359e640 1 -- 172.21.15.38:0/2518866795 >> 172.21.15.38:0/2518866795 conn(0x7f567c01eae0 msgr2=0x7f567c03a710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:59.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.802+0000 7f568359e640 1 -- 172.21.15.38:0/2518866795 shutdown_connections 2024-09-21T03:33:59.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.802+0000 7f568359e640 1 -- 172.21.15.38:0/2518866795 wait complete. 2024-09-21T03:33:59.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.805+0000 7f568359e640 1 Processor -- start 2024-09-21T03:33:59.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.805+0000 7f568359e640 1 -- start start 2024-09-21T03:33:59.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.807+0000 7f568359e640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f567c008fa0 0x7f567c0323d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:59.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.807+0000 7f568359e640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f567c03ae90 0x7f567c1725d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:59.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.807+0000 7f568359e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f567c172b10 0x7f567c0c1770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:59.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.808+0000 7f5682d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f567c172b10 0x7f567c0c1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:59.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.808+0000 7f5682d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f567c172b10 0x7f567c0c1770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42300/0 (socket says 172.21.15.38:42300) 2024-09-21T03:33:59.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.808+0000 7f5682d9d640 1 -- 172.21.15.38:0/1387902564 learned_addr learned my addr 172.21.15.38:0/1387902564 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:33:59.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.808+0000 7f568359e640 1 -- 172.21.15.38:0/1387902564 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f567c053ef0 con 0x7f567c172b10 2024-09-21T03:33:59.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.808+0000 7f568359e640 1 -- 172.21.15.38:0/1387902564 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f567c053d70 con 0x7f567c03ae90 2024-09-21T03:33:59.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.808+0000 7f568259c640 1 --2- 172.21.15.38:0/1387902564 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f567c008fa0 0x7f567c0323d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:59.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.808+0000 7f568359e640 1 -- 172.21.15.38:0/1387902564 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f567c054070 con 0x7f567c008fa0 2024-09-21T03:33:59.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.808+0000 7f5682d9d640 1 -- 172.21.15.38:0/1387902564 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f567c008fa0 msgr2=0x7f567c0323d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:59.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.808+0000 7f5682d9d640 1 --2- 172.21.15.38:0/1387902564 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f567c008fa0 0x7f567c0323d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:59.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.808+0000 7f5682d9d640 1 -- 172.21.15.38:0/1387902564 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f567c03ae90 msgr2=0x7f567c1725d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T03:33:59.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.808+0000 7f5682d9d640 1 --2- 172.21.15.38:0/1387902564 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f567c03ae90 0x7f567c1725d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:59.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.808+0000 7f5682d9d640 1 -- 172.21.15.38:0/1387902564 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f567c034270 con 0x7f567c172b10 2024-09-21T03:33:59.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.809+0000 7f5682d9d640 1 --2- 172.21.15.38:0/1387902564 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f567c172b10 0x7f567c0c1770 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f568407ac10 tx=0x7f56840b0d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:59.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.810+0000 7f5672ffd640 1 -- 172.21.15.38:0/1387902564 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56840b1400 con 0x7f567c172b10 2024-09-21T03:33:59.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.810+0000 7f5672ffd640 1 -- 172.21.15.38:0/1387902564 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f56840bf070 con 0x7f567c172b10 2024-09-21T03:33:59.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.810+0000 7f5672ffd640 1 -- 172.21.15.38:0/1387902564 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56840ba450 con 0x7f567c172b10 2024-09-21T03:33:59.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.810+0000 7f568359e640 1 -- 172.21.15.38:0/1387902564 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f567c034530 con 0x7f567c172b10 2024-09-21T03:33:59.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.813+0000 7f568359e640 1 -- 172.21.15.38:0/1387902564 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f567c034af0 con 0x7f567c172b10 2024-09-21T03:33:59.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.814+0000 7f5672ffd640 1 -- 172.21.15.38:0/1387902564 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f56840ba610 con 0x7f567c172b10 2024-09-21T03:33:59.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.815+0000 7f5672ffd640 1 --2- 172.21.15.38:0/1387902564 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5664078c60 0x7f566407b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:59.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.815+0000 7f5672ffd640 1 -- 172.21.15.38:0/1387902564 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f568413c060 con 0x7f567c172b10 2024-09-21T03:33:59.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.816+0000 7f568359e640 1 -- 172.21.15.38:0/1387902564 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5668005290 con 0x7f567c172b10 2024-09-21T03:33:59.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.820+0000 7f568259c640 1 --2- 172.21.15.38:0/1387902564 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5664078c60 0x7f566407b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:59.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.826+0000 7f5672ffd640 1 -- 172.21.15.38:0/1387902564 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f56840ff840 con 0x7f567c172b10 2024-09-21T03:33:59.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:33:59.833+0000 7f568259c640 1 --2- 172.21.15.38:0/1387902564 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5664078c60 0x7f566407b170 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f56740015e0 tx=0x7f5674009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:00.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:00.190+0000 7f27fa84d640 1 -- 172.21.15.38:0/662123424 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f27f40b6020 con 0x7f27f40cfac0 2024-09-21T03:34:00.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:00.190+0000 7f27f0ff9640 1 -- 172.21.15.38:0/662123424 <== mon.2 v2:172.21.15.195:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f27dc062a20 con 0x7f27f40cfac0 2024-09-21T03:34:00.193 INFO:teuthology.orchestra.run.smithi038.stdout:64424509449 2024-09-21T03:34:00.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:00.194+0000 7f27ea7fc640 1 -- 172.21.15.38:0/662123424 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f27d4078c10 msgr2=0x7f27d407b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:00.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:00.194+0000 7f27ea7fc640 1 --2- 172.21.15.38:0/662123424 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f27d4078c10 0x7f27d407b120 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f27ec034000 tx=0x7f27ec004730 comp rx=0 tx=0).stop 2024-09-21T03:34:00.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:00.194+0000 7f27ea7fc640 1 -- 172.21.15.38:0/662123424 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f27f40cfac0 msgr2=0x7f27f409c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:00.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:00.194+0000 7f27ea7fc640 1 --2- 172.21.15.38:0/662123424 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f27f40cfac0 0x7f27f409c510 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f27dc00b470 tx=0x7f27dc00b930 comp rx=0 tx=0).stop 2024-09-21T03:34:00.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:00.194+0000 7f27ea7fc640 1 -- 172.21.15.38:0/662123424 shutdown_connections 2024-09-21T03:34:00.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:00.195+0000 7f27ea7fc640 1 --2- 172.21.15.38:0/662123424 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f27d4078c10 0x7f27d407b120 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:00.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:00.195+0000 7f27ea7fc640 1 --2- 172.21.15.38:0/662123424 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f27f40cfac0 0x7f27f409c510 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:00.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:00.195+0000 7f27ea7fc640 1 --2- 172.21.15.38:0/662123424 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f27f40cf6b0 0x7f27f409bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:00.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:00.195+0000 7f27ea7fc640 1 --2- 172.21.15.38:0/662123424 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f27f4073000 0x7f27f415d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:00.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:00.195+0000 7f27ea7fc640 1 -- 172.21.15.38:0/662123424 >> 172.21.15.38:0/662123424 conn(0x7f27f40b1b90 msgr2=0x7f27f41601e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:00.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:00.195+0000 7f27ea7fc640 1 -- 172.21.15.38:0/662123424 shutdown_connections 2024-09-21T03:34:00.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:00.195+0000 7f27ea7fc640 1 -- 172.21.15.38:0/662123424 wait complete. 2024-09-21T03:34:00.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:00.206+0000 7fe29cff9640 1 -- 172.21.15.38:0/2592595084 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fe2a80af7e0 con 0x7fe2a8072d90 2024-09-21T03:34:00.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:00.208+0000 7fe29effd640 1 -- 172.21.15.38:0/2592595084 <== mon.2 v2:172.21.15.195:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fe2a00672b0 con 0x7fe2a8072d90 2024-09-21T03:34:00.210 INFO:teuthology.orchestra.run.smithi038.stdout:64424509449 2024-09-21T03:34:00.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:00.210+0000 7f7f3d7fa640 1 -- 172.21.15.38:0/2368976307 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f7f24005580 con 0x7f7f4814c9d0 2024-09-21T03:34:00.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:00.211+0000 7f8cec74a640 1 -- 172.21.15.38:0/2769756428 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f8ccc005580 con 0x7f8ce40aee70 2024-09-21T03:34:00.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:00.211+0000 7fe29cff9640 1 -- 172.21.15.38:0/2592595084 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fe28c078c10 msgr2=0x7fe28c07b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:00.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:00.211+0000 7fe29cff9640 1 --2- 172.21.15.38:0/2592595084 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fe28c078c10 0x7fe28c07b120 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fe298033000 tx=0x7fe298004730 comp rx=0 tx=0).stop 2024-09-21T03:34:00.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:00.211+0000 7fe29cff9640 1 -- 172.21.15.38:0/2592595084 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe2a8072d90 msgr2=0x7fe2a80b5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:00.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:00.211+0000 7fe29cff9640 1 --2- 172.21.15.38:0/2592595084 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe2a8072d90 0x7fe2a80b5870 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe2a000bab0 tx=0x7fe2a000bf70 comp rx=0 tx=0).stop 2024-09-21T03:34:00.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:00.211+0000 7f7f3f7fe640 1 -- 172.21.15.38:0/2368976307 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f7f400628c0 con 0x7f7f4814c9d0 2024-09-21T03:34:00.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:00.212+0000 7fe29cff9640 1 -- 172.21.15.38:0/2592595084 shutdown_connections 2024-09-21T03:34:00.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:00.212+0000 7fe29cff9640 1 --2- 172.21.15.38:0/2592595084 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe2a8202dc0 0x7fe2a8159b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:00.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:00.212+0000 7fe29cff9640 1 --2- 172.21.15.38:0/2592595084 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe2a815ac20 0x7fe2a81595f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:00.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:00.212+0000 7fe29cff9640 1 --2- 172.21.15.38:0/2592595084 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fe28c078c10 0x7fe28c07b120 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:00.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:00.212+0000 7fe29cff9640 1 --2- 172.21.15.38:0/2592595084 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe2a8072d90 0x7fe2a80b5870 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:00.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:00.212+0000 7fe29cff9640 1 -- 172.21.15.38:0/2592595084 >> 172.21.15.38:0/2592595084 conn(0x7fe2a80b1b90 msgr2=0x7fe2a816a820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:00.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:00.212+0000 7fe29cff9640 1 -- 172.21.15.38:0/2592595084 shutdown_connections 2024-09-21T03:34:00.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:00.212+0000 7fe29cff9640 1 -- 172.21.15.38:0/2592595084 wait complete. 2024-09-21T03:34:00.216 INFO:teuthology.orchestra.run.smithi038.stdout:64424509448 2024-09-21T03:34:00.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:00.212+0000 7f8cdaffd640 1 -- 172.21.15.38:0/2769756428 <== mon.2 v2:172.21.15.195:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f8ce00636a0 con 0x7f8ce40aee70 2024-09-21T03:34:00.216 INFO:teuthology.orchestra.run.smithi038.stdout:60129542154 2024-09-21T03:34:00.217 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509451 got 64424509449 for osd.2 2024-09-21T03:34:00.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:00.215+0000 7f7f3d7fa640 1 -- 172.21.15.38:0/2368976307 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7f2c078b90 msgr2=0x7f7f2c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:00.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:00.215+0000 7f7f3d7fa640 1 --2- 172.21.15.38:0/2368976307 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7f2c078b90 0x7f7f2c07b0a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f7f38045000 tx=0x7f7f3803d040 comp rx=0 tx=0).stop 2024-09-21T03:34:00.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:00.215+0000 7f7f3d7fa640 1 -- 172.21.15.38:0/2368976307 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7f4814c9d0 msgr2=0x7f7f481f5440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:00.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:00.215+0000 7f7f3d7fa640 1 --2- 172.21.15.38:0/2368976307 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7f4814c9d0 0x7f7f481f5440 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f7f4000c7d0 tx=0x7f7f4000cc90 comp rx=0 tx=0).stop 2024-09-21T03:34:00.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:00.215+0000 7f7f3d7fa640 1 -- 172.21.15.38:0/2368976307 shutdown_connections 2024-09-21T03:34:00.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:00.215+0000 7f7f3d7fa640 1 --2- 172.21.15.38:0/2368976307 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7f4814c9d0 0x7f7f481f5440 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:00.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:00.215+0000 7f7f3d7fa640 1 --2- 172.21.15.38:0/2368976307 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7f48162450 0x7f7f4809f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:00.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:00.215+0000 7f7f3d7fa640 1 --2- 172.21.15.38:0/2368976307 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7f2c078b90 0x7f7f2c07b0a0 secure :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f7f38045000 tx=0x7f7f3803d040 comp rx=0 tx=0).stop 2024-09-21T03:34:00.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:00.215+0000 7f7f3d7fa640 1 --2- 172.21.15.38:0/2368976307 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7f480cc160 0x7f7f4809e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:00.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:00.215+0000 7f7f3d7fa640 1 -- 172.21.15.38:0/2368976307 >> 172.21.15.38:0/2368976307 conn(0x7f7f480b1b90 msgr2=0x7f7f4806dc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:00.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:00.215+0000 7fbf78a0e640 1 -- 172.21.15.38:0/3114749633 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fbf740cafc0 con 0x7fbf7406ddf0 2024-09-21T03:34:00.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:00.215+0000 7f7f3d7fa640 1 -- 172.21.15.38:0/2368976307 shutdown_connections 2024-09-21T03:34:00.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:00.216+0000 7f7f3d7fa640 1 -- 172.21.15.38:0/2368976307 wait complete. 2024-09-21T03:34:00.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:00.216+0000 7f8cec74a640 1 -- 172.21.15.38:0/2769756428 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8cc8078c60 msgr2=0x7f8cc807b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:00.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:00.216+0000 7f8cec74a640 1 --2- 172.21.15.38:0/2769756428 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8cc8078c60 0x7f8cc807b170 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8cdc046000 tx=0x7f8cdc03e040 comp rx=0 tx=0).stop 2024-09-21T03:34:00.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:00.216+0000 7f8cec74a640 1 -- 172.21.15.38:0/2769756428 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8ce40aee70 msgr2=0x7f8ce40c2f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:00.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:00.216+0000 7f8cec74a640 1 --2- 172.21.15.38:0/2769756428 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8ce40aee70 0x7f8ce40c2f90 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f8ce000ea20 tx=0x7f8ce000eee0 comp rx=0 tx=0).stop 2024-09-21T03:34:00.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:00.216+0000 7f8cec74a640 1 -- 172.21.15.38:0/2769756428 shutdown_connections 2024-09-21T03:34:00.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:00.216+0000 7f8cec74a640 1 --2- 172.21.15.38:0/2769756428 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8ce415faf0 0x7f8ce409be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:00.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:00.216+0000 7f8cec74a640 1 --2- 172.21.15.38:0/2769756428 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8ce40cc160 0x7f8ce409b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:00.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:00.216+0000 7f8cec74a640 1 --2- 172.21.15.38:0/2769756428 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8cc8078c60 0x7f8cc807b170 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:00.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:00.216+0000 7f8cec74a640 1 --2- 172.21.15.38:0/2769756428 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8ce40aee70 0x7f8ce40c2f90 secure :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f8ce000ea20 tx=0x7f8ce000eee0 comp rx=0 tx=0).stop 2024-09-21T03:34:00.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:00.216+0000 7f8cec74a640 1 -- 172.21.15.38:0/2769756428 >> 172.21.15.38:0/2769756428 conn(0x7f8ce40b1b90 msgr2=0x7f8ce40cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:00.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:00.216+0000 7fbf62ffd640 1 -- 172.21.15.38:0/3114749633 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fbf64066b40 con 0x7fbf7406ddf0 2024-09-21T03:34:00.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:00.216+0000 7f8cec74a640 1 -- 172.21.15.38:0/2769756428 shutdown_connections 2024-09-21T03:34:00.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:00.216+0000 7f8cec74a640 1 -- 172.21.15.38:0/2769756428 wait complete. 2024-09-21T03:34:00.221 INFO:teuthology.orchestra.run.smithi038.stdout:60129542153 2024-09-21T03:34:00.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:00.218+0000 7fbf78a0e640 1 -- 172.21.15.38:0/3114749633 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fbf54078be0 msgr2=0x7fbf5407b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:00.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:00.219+0000 7fbf78a0e640 1 --2- 172.21.15.38:0/3114749633 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fbf54078be0 0x7fbf5407b0f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fbf68033000 tx=0x7fbf68004730 comp rx=0 tx=0).stop 2024-09-21T03:34:00.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:00.219+0000 7fbf78a0e640 1 -- 172.21.15.38:0/3114749633 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbf7406ddf0 msgr2=0x7fbf740c7d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:00.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:00.219+0000 7fbf78a0e640 1 --2- 172.21.15.38:0/3114749633 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbf7406ddf0 0x7fbf740c7d00 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fbf6400d740 tx=0x7fbf6400dc00 comp rx=0 tx=0).stop 2024-09-21T03:34:00.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:00.219+0000 7fbf78a0e640 1 -- 172.21.15.38:0/3114749633 shutdown_connections 2024-09-21T03:34:00.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:00.219+0000 7fbf78a0e640 1 --2- 172.21.15.38:0/3114749633 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbf740d02b0 0x7fbf74147820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:00.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:00.219+0000 7fbf78a0e640 1 --2- 172.21.15.38:0/3114749633 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fbf54078be0 0x7fbf5407b0f0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:00.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:00.219+0000 7fbf78a0e640 1 --2- 172.21.15.38:0/3114749633 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbf740ceca0 0x7fbf74159eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:00.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:00.219+0000 7fbf78a0e640 1 --2- 172.21.15.38:0/3114749633 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbf7406ddf0 0x7fbf740c7d00 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:00.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:00.220+0000 7fbf78a0e640 1 -- 172.21.15.38:0/3114749633 >> 172.21.15.38:0/3114749633 conn(0x7fbf740735d0 msgr2=0x7fbf740afa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:00.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:00.220+0000 7fbf78a0e640 1 -- 172.21.15.38:0/3114749633 shutdown_connections 2024-09-21T03:34:00.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:00.220+0000 7fbf78a0e640 1 -- 172.21.15.38:0/3114749633 wait complete. 2024-09-21T03:34:00.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:00.223+0000 7f767dfbb640 1 -- 172.21.15.38:0/2102433518 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f767004c3d0 con 0x7f7670053c50 2024-09-21T03:34:00.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:00.224+0000 7f76757fa640 1 -- 172.21.15.38:0/2102433518 <== mon.2 v2:172.21.15.195:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f766c082ec0 con 0x7f7670053c50 2024-09-21T03:34:00.229 INFO:teuthology.orchestra.run.smithi038.stdout:64424509448 2024-09-21T03:34:00.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:00.227+0000 7eff128b6640 1 -- 172.21.15.38:0/129607063 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7eff0c06fa70 con 0x7eff0c06ddf0 2024-09-21T03:34:00.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:00.228+0000 7f767dfbb640 1 -- 172.21.15.38:0/2102433518 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f765c078d90 msgr2=0x7f765c07b2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:00.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:00.228+0000 7f767dfbb640 1 --2- 172.21.15.38:0/2102433518 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f765c078d90 0x7f765c07b2a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f767003a3f0 tx=0x7f7664009060 comp rx=0 tx=0).stop 2024-09-21T03:34:00.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:00.228+0000 7f767dfbb640 1 -- 172.21.15.38:0/2102433518 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7670053c50 msgr2=0x7f767004f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:00.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:00.228+0000 7f767dfbb640 1 --2- 172.21.15.38:0/2102433518 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7670053c50 0x7f767004f250 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f766c000c00 tx=0x7f766c003b90 comp rx=0 tx=0).stop 2024-09-21T03:34:00.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:00.228+0000 7f767dfbb640 1 -- 172.21.15.38:0/2102433518 shutdown_connections 2024-09-21T03:34:00.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:00.228+0000 7f767dfbb640 1 --2- 172.21.15.38:0/2102433518 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f76700e2f40 0x7f7670039860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:00.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:00.228+0000 7f767dfbb640 1 --2- 172.21.15.38:0/2102433518 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7670056da0 0x7f76700d83c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:00.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:00.228+0000 7f767dfbb640 1 --2- 172.21.15.38:0/2102433518 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f765c078d90 0x7f765c07b2a0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:00.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:00.228+0000 7f767dfbb640 1 --2- 172.21.15.38:0/2102433518 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7670053c50 0x7f767004f250 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:00.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:00.228+0000 7f767dfbb640 1 -- 172.21.15.38:0/2102433518 >> 172.21.15.38:0/2102433518 conn(0x7f767003a5f0 msgr2=0x7f76700e6d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:00.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:00.228+0000 7eff08ff9640 1 -- 172.21.15.38:0/129607063 <== mon.2 v2:172.21.15.195:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7eff00088570 con 0x7eff0c06ddf0 2024-09-21T03:34:00.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:00.228+0000 7f767dfbb640 1 -- 172.21.15.38:0/2102433518 shutdown_connections 2024-09-21T03:34:00.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:00.228+0000 7f767dfbb640 1 -- 172.21.15.38:0/2102433518 wait complete. 2024-09-21T03:34:00.232 INFO:teuthology.orchestra.run.smithi038.stdout:55834574857 2024-09-21T03:34:00.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:00.231+0000 7eff128b6640 1 -- 172.21.15.38:0/129607063 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7efef8078b90 msgr2=0x7efef807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:00.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:00.231+0000 7eff128b6640 1 --2- 172.21.15.38:0/129607063 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7efef8078b90 0x7efef807b0a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7eff0c0cc4c0 tx=0x7efefc009060 comp rx=0 tx=0).stop 2024-09-21T03:34:00.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:00.231+0000 7eff128b6640 1 -- 172.21.15.38:0/129607063 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7eff0c06ddf0 msgr2=0x7eff0c0a0c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:00.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:00.231+0000 7eff128b6640 1 --2- 172.21.15.38:0/129607063 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7eff0c06ddf0 0x7eff0c0a0c80 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7eff00034040 tx=0x7eff00003b90 comp rx=0 tx=0).stop 2024-09-21T03:34:00.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:00.232+0000 7eff128b6640 1 -- 172.21.15.38:0/129607063 shutdown_connections 2024-09-21T03:34:00.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:00.232+0000 7eff128b6640 1 --2- 172.21.15.38:0/129607063 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7eff0c15ac20 0x7eff0c1f4910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:00.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:00.232+0000 7eff128b6640 1 --2- 172.21.15.38:0/129607063 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7eff0c072d90 0x7eff0c0c6370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:00.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:00.232+0000 7eff128b6640 1 --2- 172.21.15.38:0/129607063 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7efef8078b90 0x7efef807b0a0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:00.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:00.232+0000 7eff128b6640 1 --2- 172.21.15.38:0/129607063 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7eff0c06ddf0 0x7eff0c0a0c80 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:00.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:00.232+0000 7eff128b6640 1 -- 172.21.15.38:0/129607063 >> 172.21.15.38:0/129607063 conn(0x7eff0c0b1b90 msgr2=0x7eff0c159e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:00.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:00.232+0000 7eff128b6640 1 -- 172.21.15.38:0/129607063 shutdown_connections 2024-09-21T03:34:00.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:00.232+0000 7eff128b6640 1 -- 172.21.15.38:0/129607063 wait complete. 2024-09-21T03:34:00.239 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509450 got 64424509448 for osd.3 2024-09-21T03:34:00.239 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509451 got 64424509449 for osd.7 2024-09-21T03:34:00.239 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542155 got 60129542154 for osd.4 2024-09-21T03:34:00.242 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542154 got 60129542153 for osd.6 2024-09-21T03:34:00.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:00.243+0000 7f568359e640 1 -- 172.21.15.38:0/1387902564 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f5668005850 con 0x7f567c172b10 2024-09-21T03:34:00.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:00.244+0000 7f5672ffd640 1 -- 172.21.15.38:0/1387902564 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f56840ff140 con 0x7f567c172b10 2024-09-21T03:34:00.247 INFO:teuthology.orchestra.run.smithi038.stdout:68719476745 2024-09-21T03:34:00.247 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509450 got 64424509448 for osd.0 2024-09-21T03:34:00.248 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.5 2024-09-21T03:34:00.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:00.246+0000 7f568359e640 1 -- 172.21.15.38:0/1387902564 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5664078c60 msgr2=0x7f566407b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:00.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:00.246+0000 7f568359e640 1 --2- 172.21.15.38:0/1387902564 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5664078c60 0x7f566407b170 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f56740015e0 tx=0x7f5674009040 comp rx=0 tx=0).stop 2024-09-21T03:34:00.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:00.246+0000 7f568359e640 1 -- 172.21.15.38:0/1387902564 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f567c172b10 msgr2=0x7f567c0c1770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:00.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:00.246+0000 7f568359e640 1 --2- 172.21.15.38:0/1387902564 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f567c172b10 0x7f567c0c1770 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f568407ac10 tx=0x7f56840b0d90 comp rx=0 tx=0).stop 2024-09-21T03:34:00.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:00.246+0000 7f568359e640 1 -- 172.21.15.38:0/1387902564 shutdown_connections 2024-09-21T03:34:00.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:00.246+0000 7f568359e640 1 --2- 172.21.15.38:0/1387902564 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f567c172b10 0x7f567c0c1770 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:00.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:00.246+0000 7f568359e640 1 --2- 172.21.15.38:0/1387902564 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f567c03ae90 0x7f567c1725d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:00.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:00.246+0000 7f568359e640 1 --2- 172.21.15.38:0/1387902564 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5664078c60 0x7f566407b170 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:00.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:00.246+0000 7f568359e640 1 --2- 172.21.15.38:0/1387902564 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f567c008fa0 0x7f567c0323d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:00.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:00.246+0000 7f568359e640 1 -- 172.21.15.38:0/1387902564 >> 172.21.15.38:0/1387902564 conn(0x7f567c01eae0 msgr2=0x7f567c054580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:00.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:00.247+0000 7f568359e640 1 -- 172.21.15.38:0/1387902564 shutdown_connections 2024-09-21T03:34:00.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:00.247+0000 7f568359e640 1 -- 172.21.15.38:0/1387902564 wait complete. 2024-09-21T03:34:00.261 INFO:tasks.ceph.ceph_manager.ceph:need seq 68719476746 got 68719476745 for osd.1 2024-09-21T03:34:01.218 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-21T03:34:01.240 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-21T03:34:01.240 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-21T03:34:01.240 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-21T03:34:01.243 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-21T03:34:01.248 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-21T03:34:01.249 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-21T03:34:01.261 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-21T03:34:01.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.423+0000 7f41f10cc640 1 -- 172.21.15.38:0/1621665127 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f41e40e7030 msgr2=0x7f41e4005e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:01.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.423+0000 7f41f10cc640 1 --2- 172.21.15.38:0/1621665127 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f41e40e7030 0x7f41e4005e10 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f41e4034950 tx=0x7f41dc02a640 comp rx=0 tx=0).stop 2024-09-21T03:34:01.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.423+0000 7f41f10cc640 1 -- 172.21.15.38:0/1621665127 shutdown_connections 2024-09-21T03:34:01.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.423+0000 7f41f10cc640 1 --2- 172.21.15.38:0/1621665127 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f41e40e7030 0x7f41e4005e10 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:01.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.423+0000 7f41f10cc640 1 --2- 172.21.15.38:0/1621665127 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f41e4052390 0x7f41e40e6af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:01.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.423+0000 7f41f10cc640 1 -- 172.21.15.38:0/1621665127 >> 172.21.15.38:0/1621665127 conn(0x7f41e401d9d0 msgr2=0x7f41e4039690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:01.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.423+0000 7f41f10cc640 1 -- 172.21.15.38:0/1621665127 shutdown_connections 2024-09-21T03:34:01.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.423+0000 7f41f10cc640 1 -- 172.21.15.38:0/1621665127 wait complete. 2024-09-21T03:34:01.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.427+0000 7f41f10cc640 1 Processor -- start 2024-09-21T03:34:01.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.427+0000 7f41f10cc640 1 -- start start 2024-09-21T03:34:01.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.430+0000 7f41f10cc640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f41e4052390 0x7f41e40e3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:01.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.430+0000 7f41f10cc640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f41e40e7030 0x7f41e40312c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:01.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.431+0000 7f41ead9d640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f41e4052390 0x7f41e40e3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:01.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.431+0000 7f41ead9d640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f41e4052390 0x7f41e40e3800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.38:54824/0 (socket says 172.21.15.38:54824) 2024-09-21T03:34:01.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.431+0000 7f41ea59c640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f41e40e7030 0x7f41e40312c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:01.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.431+0000 7f41ea59c640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f41e40e7030 0x7f41e40312c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.38:50634/0 (socket says 172.21.15.38:50634) 2024-09-21T03:34:01.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.431+0000 7f41f10cc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f41e40e7440 0x7f41e40e52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:01.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.431+0000 7f41ead9d640 1 -- 172.21.15.38:0/3103553342 learned_addr learned my addr 172.21.15.38:0/3103553342 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:34:01.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.431+0000 7f41f10cc640 1 -- 172.21.15.38:0/3103553342 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f41e4051310 con 0x7f41e40e7440 2024-09-21T03:34:01.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.431+0000 7f41f10cc640 1 -- 172.21.15.38:0/3103553342 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f41e40e3650 con 0x7f41e4052390 2024-09-21T03:34:01.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.431+0000 7f41f10cc640 1 -- 172.21.15.38:0/3103553342 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f41e4051490 con 0x7f41e40e7030 2024-09-21T03:34:01.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.432+0000 7f41eb59e640 1 --2- 172.21.15.38:0/3103553342 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f41e40e7440 0x7f41e40e52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:01.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.433+0000 7f41eb59e640 1 -- 172.21.15.38:0/3103553342 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f41e40e7030 msgr2=0x7f41e40312c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:01.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.433+0000 7f41eb59e640 1 --2- 172.21.15.38:0/3103553342 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f41e40e7030 0x7f41e40312c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:01.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.433+0000 7f41eb59e640 1 -- 172.21.15.38:0/3103553342 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f41e4052390 msgr2=0x7f41e40e3800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:01.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.433+0000 7f41eb59e640 1 --2- 172.21.15.38:0/3103553342 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f41e4052390 0x7f41e40e3800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:01.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.433+0000 7f41eb59e640 1 -- 172.21.15.38:0/3103553342 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41e40e3fc0 con 0x7f41e40e7440 2024-09-21T03:34:01.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.433+0000 7f41eb59e640 1 --2- 172.21.15.38:0/3103553342 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f41e40e7440 0x7f41e40e52d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f41e000c3c0 tx=0x7f41e000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:01.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.434+0000 7f41ead9d640 1 --2- 172.21.15.38:0/3103553342 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f41e4052390 0x7f41e40e3800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:34:01.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.434+0000 7f41ea59c640 1 --2- 172.21.15.38:0/3103553342 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f41e40e7030 0x7f41e40312c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:34:01.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.434+0000 7f41d37fe640 1 -- 172.21.15.38:0/3103553342 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41e0017070 con 0x7f41e40e7440 2024-09-21T03:34:01.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.434+0000 7f41f10cc640 1 -- 172.21.15.38:0/3103553342 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41e40e42a0 con 0x7f41e40e7440 2024-09-21T03:34:01.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.434+0000 7f41f10cc640 1 -- 172.21.15.38:0/3103553342 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41e40e4db0 con 0x7f41e40e7440 2024-09-21T03:34:01.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.440+0000 7f41d37fe640 1 -- 172.21.15.38:0/3103553342 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f41e00053c0 con 0x7f41e40e7440 2024-09-21T03:34:01.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.440+0000 7f41d37fe640 1 -- 172.21.15.38:0/3103553342 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41e0003850 con 0x7f41e40e7440 2024-09-21T03:34:01.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.441+0000 7f41d37fe640 1 -- 172.21.15.38:0/3103553342 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f41e001a070 con 0x7f41e40e7440 2024-09-21T03:34:01.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.442+0000 7f41d37fe640 1 --2- 172.21.15.38:0/3103553342 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f41d8078c90 0x7f41d807b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:01.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.443+0000 7f41d37fe640 1 -- 172.21.15.38:0/3103553342 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f41e009a2c0 con 0x7f41e40e7440 2024-09-21T03:34:01.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.443+0000 7f41ead9d640 1 --2- 172.21.15.38:0/3103553342 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f41d8078c90 0x7f41d807b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:01.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.444+0000 7f41f10cc640 1 -- 172.21.15.38:0/3103553342 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41cc005290 con 0x7f41e40e7440 2024-09-21T03:34:01.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.445+0000 7f41ead9d640 1 --2- 172.21.15.38:0/3103553342 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f41d8078c90 0x7f41d807b1a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f41ec066b30 tx=0x7f41ec06c4c0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:01.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.454+0000 7f41d37fe640 1 -- 172.21.15.38:0/3103553342 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f41e4006a60 con 0x7f41e40e7440 2024-09-21T03:34:01.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.488+0000 7f40dffff640 1 -- 172.21.15.38:0/1788117312 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f40d8007d50 msgr2=0x7f40d80df1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:01.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.488+0000 7f40dffff640 1 --2- 172.21.15.38:0/1788117312 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f40d8007d50 0x7f40d80df1b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f40e007a9a0 tx=0x7f40e00a7740 comp rx=0 tx=0).stop 2024-09-21T03:34:01.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.489+0000 7f40dffff640 1 -- 172.21.15.38:0/1788117312 shutdown_connections 2024-09-21T03:34:01.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.489+0000 7f40dffff640 1 --2- 172.21.15.38:0/1788117312 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f40d8007d50 0x7f40d80df1b0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:01.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.489+0000 7f40dffff640 1 --2- 172.21.15.38:0/1788117312 >> v2:172.21.15.195:3300/0 conn(0x7f40d8035060 0x7f40d80dec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:01.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.489+0000 7f40dffff640 1 -- 172.21.15.38:0/1788117312 >> 172.21.15.38:0/1788117312 conn(0x7f40d801e230 msgr2=0x7f40d8039ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:01.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.489+0000 7f73c00b6640 1 -- 172.21.15.38:0/643596247 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f73b80ceb60 msgr2=0x7f73b80cf020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:01.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.489+0000 7f73c00b6640 1 --2- 172.21.15.38:0/643596247 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f73b80ceb60 0x7f73b80cf020 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f73b4009830 tx=0x7f73b402a3f0 comp rx=0 tx=0).stop 2024-09-21T03:34:01.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.489+0000 7f73c00b6640 1 -- 172.21.15.38:0/643596247 shutdown_connections 2024-09-21T03:34:01.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.489+0000 7f73c00b6640 1 --2- 172.21.15.38:0/643596247 >> v2:172.21.15.195:3300/0 conn(0x7f73b8167410 0x7f73b806ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:01.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.489+0000 7f73c00b6640 1 --2- 172.21.15.38:0/643596247 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f73b80ceb60 0x7f73b80cf020 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:01.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.489+0000 7f73c00b6640 1 -- 172.21.15.38:0/643596247 >> 172.21.15.38:0/643596247 conn(0x7f73b80b1b90 msgr2=0x7f73b80b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:01.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.490+0000 7f4366e83640 1 -- 172.21.15.38:0/3315787458 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f436015b7a0 msgr2=0x7f43600cc6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:01.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.490+0000 7f4366e83640 1 --2- 172.21.15.38:0/3315787458 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f436015b7a0 0x7f43600cc6c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f436006eb80 tx=0x7f435002ae80 comp rx=0 tx=0).stop 2024-09-21T03:34:01.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.490+0000 7f73c00b6640 1 -- 172.21.15.38:0/643596247 shutdown_connections 2024-09-21T03:34:01.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.490+0000 7f73c00b6640 1 -- 172.21.15.38:0/643596247 wait complete. 2024-09-21T03:34:01.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.490+0000 7f40dffff640 1 -- 172.21.15.38:0/1788117312 shutdown_connections 2024-09-21T03:34:01.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.490+0000 7f4366e83640 1 -- 172.21.15.38:0/3315787458 shutdown_connections 2024-09-21T03:34:01.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.490+0000 7f4366e83640 1 --2- 172.21.15.38:0/3315787458 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f436015ecd0 0x7f43600cddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:01.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.490+0000 7f4366e83640 1 --2- 172.21.15.38:0/3315787458 >> v2:172.21.15.195:3300/0 conn(0x7f43600ccc00 0x7f436015e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:01.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.490+0000 7f4366e83640 1 --2- 172.21.15.38:0/3315787458 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f436015b7a0 0x7f43600cc6c0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:01.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.490+0000 7f4366e83640 1 -- 172.21.15.38:0/3315787458 >> 172.21.15.38:0/3315787458 conn(0x7f436006f920 msgr2=0x7f43600b1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:01.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.490+0000 7f4366e83640 1 -- 172.21.15.38:0/3315787458 shutdown_connections 2024-09-21T03:34:01.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.490+0000 7f4366e83640 1 -- 172.21.15.38:0/3315787458 wait complete. 2024-09-21T03:34:01.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.493+0000 7f73c00b6640 1 Processor -- start 2024-09-21T03:34:01.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.493+0000 7f73c00b6640 1 -- start start 2024-09-21T03:34:01.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.491+0000 7f40dffff640 1 -- 172.21.15.38:0/1788117312 wait complete. 2024-09-21T03:34:01.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.494+0000 7f4366e83640 1 Processor -- start 2024-09-21T03:34:01.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.494+0000 7f4366e83640 1 -- start start 2024-09-21T03:34:01.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.494+0000 7f40dffff640 1 Processor -- start 2024-09-21T03:34:01.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.494+0000 7f40dffff640 1 -- start start 2024-09-21T03:34:01.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.495+0000 7f73c00b6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f73b806e330 0x7f73b81618d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:01.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.495+0000 7f73c00b6640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f73b80ceb60 0x7f73b81fd6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:01.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.496+0000 7f73c00b6640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f73b8167410 0x7f73b806fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:01.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.496+0000 7f73c00b6640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f73b8074250 con 0x7f73b806e330 2024-09-21T03:34:01.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.496+0000 7f73c00b6640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f73b8163fa0 con 0x7f73b8167410 2024-09-21T03:34:01.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.496+0000 7f73c00b6640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f73b80743d0 con 0x7f73b80ceb60 2024-09-21T03:34:01.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.496+0000 7f70e8b89640 1 -- 172.21.15.38:0/3769991643 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f70dc056200 msgr2=0x7f70dc034e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:01.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.496+0000 7f70e8b89640 1 --2- 172.21.15.38:0/3769991643 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f70dc056200 0x7f70dc034e00 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f70d8009820 tx=0x7f70d802a530 comp rx=0 tx=0).stop 2024-09-21T03:34:01.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.496+0000 7f70e8b89640 1 -- 172.21.15.38:0/3769991643 shutdown_connections 2024-09-21T03:34:01.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.496+0000 7f70e8b89640 1 --2- 172.21.15.38:0/3769991643 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f70dc056200 0x7f70dc034e00 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:01.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.496+0000 7f70e8b89640 1 -- 172.21.15.38:0/3769991643 >> 172.21.15.38:0/3769991643 conn(0x7f70dc01eac0 msgr2=0x7f70dc03a780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:01.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.496+0000 7f73bd651640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f73b80ceb60 0x7f73b81fd6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:01.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.496+0000 7f73bde52640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f73b806e330 0x7f73b81618d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:01.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.496+0000 7f4366e83640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f43600ccc00 0x7f436014f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:01.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.496+0000 7f73bd651640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f73b80ceb60 0x7f73b81fd6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.38:50688/0 (socket says 172.21.15.38:50688) 2024-09-21T03:34:01.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.496+0000 7f73bde52640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f73b806e330 0x7f73b81618d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42346/0 (socket says 172.21.15.38:42346) 2024-09-21T03:34:01.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.496+0000 7f4366e83640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f436015b7a0 0x7f4360152be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:01.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.496+0000 7f73bd651640 1 -- 172.21.15.38:0/3769005612 learned_addr learned my addr 172.21.15.38:0/3769005612 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:34:01.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.496+0000 7f73be653640 1 --2- 172.21.15.38:0/3769005612 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f73b8167410 0x7f73b806fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:01.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.496+0000 7f4366e83640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f436015ecd0 0x7f4360153120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:01.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.496+0000 7f4366e83640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f4360074c60 con 0x7f436015ecd0 2024-09-21T03:34:01.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.496+0000 7f4365e81640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f43600ccc00 0x7f436014f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:01.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.496+0000 7f40dffff640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f40d8007d50 0x7f40d804ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:01.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.496+0000 7f4365e81640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f43600ccc00 0x7f436014f880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.38:54866/0 (socket says 172.21.15.38:54866) 2024-09-21T03:34:01.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.496+0000 7f70e8b89640 1 -- 172.21.15.38:0/3769991643 shutdown_connections 2024-09-21T03:34:01.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.497+0000 7f73bd651640 1 -- 172.21.15.38:0/3769005612 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f73b8167410 msgr2=0x7f73b806fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:01.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.497+0000 7f73bd651640 1 --2- 172.21.15.38:0/3769005612 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f73b8167410 0x7f73b806fbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:01.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.497+0000 7f73bd651640 1 -- 172.21.15.38:0/3769005612 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f73b806e330 msgr2=0x7f73b81618d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:01.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.497+0000 7f73bd651640 1 --2- 172.21.15.38:0/3769005612 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f73b806e330 0x7f73b81618d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:01.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.497+0000 7f73bd651640 1 -- 172.21.15.38:0/3769005612 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73b81fdc20 con 0x7f73b80ceb60 2024-09-21T03:34:01.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.496+0000 7f4365e81640 1 -- 172.21.15.38:0/522059617 learned_addr learned my addr 172.21.15.38:0/522059617 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:34:01.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.497+0000 7f40dffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f40d8036b10 0x7f40d80d0d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:01.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.497+0000 7f4366e83640 1 -- 172.21.15.38:0/522059617 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f4360074ae0 con 0x7f43600ccc00 2024-09-21T03:34:01.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.497+0000 7f3da724d640 1 -- 172.21.15.38:0/3098978735 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3da006e330 msgr2=0x7f3da0072d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:01.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.497+0000 7f3da724d640 1 --2- 172.21.15.38:0/3098978735 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3da006e330 0x7f3da0072d90 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f3d98005fd0 tx=0x7f3d9802a810 comp rx=0 tx=0).stop 2024-09-21T03:34:01.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.497+0000 7f4366e83640 1 -- 172.21.15.38:0/522059617 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f4360070cf0 con 0x7f436015b7a0 2024-09-21T03:34:01.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.497+0000 7f4366682640 1 --2- 172.21.15.38:0/522059617 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f436015ecd0 0x7f4360153120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:01.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.497+0000 7f70e8b89640 1 -- 172.21.15.38:0/3769991643 wait complete. 2024-09-21T03:34:01.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.497+0000 7f73bd651640 1 --2- 172.21.15.38:0/3769005612 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f73b80ceb60 0x7f73b81fd6e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f73a800e790 tx=0x7f73a800ec50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:01.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.497+0000 7f3da724d640 1 -- 172.21.15.38:0/3098978735 shutdown_connections 2024-09-21T03:34:01.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.497+0000 7f3da724d640 1 --2- 172.21.15.38:0/3098978735 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3da006e330 0x7f3da0072d90 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:01.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.497+0000 7f3da724d640 1 --2- 172.21.15.38:0/3098978735 >> v2:172.21.15.38:3300/0 conn(0x7f3da015aa40 0x7f3da006ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:01.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.497+0000 7f3da724d640 1 -- 172.21.15.38:0/3098978735 >> 172.21.15.38:0/3098978735 conn(0x7f3da00b1b90 msgr2=0x7f3da00b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:01.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.497+0000 7f4365680640 1 --2- 172.21.15.38:0/522059617 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f436015b7a0 0x7f4360152be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:01.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.497+0000 7f73bde52640 1 --2- 172.21.15.38:0/3769005612 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f73b806e330 0x7f73b81618d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:34:01.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.497+0000 7f3da724d640 1 -- 172.21.15.38:0/3098978735 shutdown_connections 2024-09-21T03:34:01.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.497+0000 7f73ae7fc640 1 -- 172.21.15.38:0/3769005612 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73a800eea0 con 0x7f73b80ceb60 2024-09-21T03:34:01.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.497+0000 7f73c00b6640 1 -- 172.21.15.38:0/3769005612 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73b809afc0 con 0x7f73b80ceb60 2024-09-21T03:34:01.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.497+0000 7f40deffd640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f40d8007d50 0x7f40d804ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:01.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.497+0000 7f40deffd640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f40d8007d50 0x7f40d804ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.38:54876/0 (socket says 172.21.15.38:54876) 2024-09-21T03:34:01.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.497+0000 7f73ae7fc640 1 -- 172.21.15.38:0/3769005612 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f73a800ccc0 con 0x7f73b80ceb60 2024-09-21T03:34:01.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.497+0000 7f40de7fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f40d8036b10 0x7f40d80d0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:01.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.498+0000 7f40de7fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f40d8036b10 0x7f40d80d0d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42370/0 (socket says 172.21.15.38:42370) 2024-09-21T03:34:01.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.498+0000 7f40deffd640 1 -- 172.21.15.38:0/2410861400 learned_addr learned my addr 172.21.15.38:0/2410861400 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:34:01.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.498+0000 7f4365e81640 1 -- 172.21.15.38:0/522059617 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f436015b7a0 msgr2=0x7f4360152be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:01.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.498+0000 7f4365e81640 1 --2- 172.21.15.38:0/522059617 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f436015b7a0 0x7f4360152be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:01.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.498+0000 7f40dffff640 1 --2- 172.21.15.38:0/2410861400 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f40d80d4d90 0x7f40d8177aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:01.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.498+0000 7f4365e81640 1 -- 172.21.15.38:0/522059617 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f436015ecd0 msgr2=0x7f4360153120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:01.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.498+0000 7f4365e81640 1 --2- 172.21.15.38:0/522059617 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f436015ecd0 0x7f4360153120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:01.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.498+0000 7f73c00b6640 1 -- 172.21.15.38:0/3769005612 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f73b809b580 con 0x7f73b80ceb60 2024-09-21T03:34:01.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.498+0000 7f73ae7fc640 1 -- 172.21.15.38:0/3769005612 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73a8004280 con 0x7f73b80ceb60 2024-09-21T03:34:01.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.498+0000 7f40dffff640 1 -- 172.21.15.38:0/2410861400 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f40d80df870 con 0x7f40d8036b10 2024-09-21T03:34:01.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.497+0000 7f3da724d640 1 -- 172.21.15.38:0/3098978735 wait complete. 2024-09-21T03:34:01.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.498+0000 7f4365e81640 1 -- 172.21.15.38:0/522059617 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4360150510 con 0x7f43600ccc00 2024-09-21T03:34:01.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.498+0000 7f40dffff640 1 -- 172.21.15.38:0/2410861400 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f40d80df6f0 con 0x7f40d8007d50 2024-09-21T03:34:01.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.498+0000 7f40df7fe640 1 --2- 172.21.15.38:0/2410861400 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f40d80d4d90 0x7f40d8177aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:01.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.498+0000 7f40dffff640 1 -- 172.21.15.38:0/2410861400 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f40d80df9f0 con 0x7f40d80d4d90 2024-09-21T03:34:01.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.498+0000 7fe0077fe640 1 -- 172.21.15.38:0/1382799234 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdffc031600 con 0x7fe0081519d0 2024-09-21T03:34:01.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.498+0000 7f4365e81640 1 --2- 172.21.15.38:0/522059617 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f43600ccc00 0x7f436014f880 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f4350035040 tx=0x7f435002ca00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:01.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.498+0000 7fe00f04d640 1 -- 172.21.15.38:0/1382799234 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe0081519d0 msgr2=0x7fe00815e300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:01.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.498+0000 7fe00f04d640 1 --2- 172.21.15.38:0/1382799234 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe0081519d0 0x7fe00815e300 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fdffc009830 tx=0x7fdffc02a490 comp rx=0 tx=0).stop 2024-09-21T03:34:01.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.499+0000 7f43567fc640 1 -- 172.21.15.38:0/522059617 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4350040020 con 0x7f43600ccc00 2024-09-21T03:34:01.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.499+0000 7f4366e83640 1 -- 172.21.15.38:0/522059617 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43600b3ec0 con 0x7f43600ccc00 2024-09-21T03:34:01.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.499+0000 7f4366e83640 1 -- 172.21.15.38:0/522059617 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43600b4480 con 0x7f43600ccc00 2024-09-21T03:34:01.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.499+0000 7f40de7fc640 1 -- 172.21.15.38:0/2410861400 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f40d80d4d90 msgr2=0x7f40d8177aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:01.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.499+0000 7f40de7fc640 1 --2- 172.21.15.38:0/2410861400 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f40d80d4d90 0x7f40d8177aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:01.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.499+0000 7f40de7fc640 1 -- 172.21.15.38:0/2410861400 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f40d8007d50 msgr2=0x7f40d804ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:01.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.499+0000 7f40de7fc640 1 --2- 172.21.15.38:0/2410861400 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f40d8007d50 0x7f40d804ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:01.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.499+0000 7f40de7fc640 1 -- 172.21.15.38:0/2410861400 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40d817b0f0 con 0x7f40d8036b10 2024-09-21T03:34:01.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.499+0000 7fe00f04d640 1 -- 172.21.15.38:0/1382799234 shutdown_connections 2024-09-21T03:34:01.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.499+0000 7fe00f04d640 1 --2- 172.21.15.38:0/1382799234 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe008070c80 0x7fe0080cfa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:01.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.500+0000 7f70e8b89640 1 Processor -- start 2024-09-21T03:34:01.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.499+0000 7fe00f04d640 1 --2- 172.21.15.38:0/1382799234 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe0081519d0 0x7fe00815e300 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:01.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.499+0000 7fe00f04d640 1 -- 172.21.15.38:0/1382799234 >> 172.21.15.38:0/1382799234 conn(0x7fe0080d0590 msgr2=0x7fe00806cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:01.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.500+0000 7f70e8b89640 1 -- start start 2024-09-21T03:34:01.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.500+0000 7f3da724d640 1 Processor -- start 2024-09-21T03:34:01.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.500+0000 7fe00f04d640 1 -- 172.21.15.38:0/1382799234 shutdown_connections 2024-09-21T03:34:01.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.500+0000 7fe00f04d640 1 -- 172.21.15.38:0/1382799234 wait complete. 2024-09-21T03:34:01.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.500+0000 7f3da724d640 1 -- start start 2024-09-21T03:34:01.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.499+0000 7f43567fc640 1 -- 172.21.15.38:0/522059617 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f43500054d0 con 0x7f43600ccc00 2024-09-21T03:34:01.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.500+0000 7f43567fc640 1 -- 172.21.15.38:0/522059617 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4350040020 con 0x7f43600ccc00 2024-09-21T03:34:01.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.500+0000 7f4366e83640 1 -- 172.21.15.38:0/522059617 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4348005290 con 0x7f43600ccc00 2024-09-21T03:34:01.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.500+0000 7f73c00b6640 1 -- 172.21.15.38:0/3769005612 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73a0005290 con 0x7f73b80ceb60 2024-09-21T03:34:01.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.500+0000 7f73ae7fc640 1 -- 172.21.15.38:0/3769005612 <== mon.2 v2:172.21.15.195:3301/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f73a80035f0 con 0x7f73b80ceb60 2024-09-21T03:34:01.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.501+0000 7f40de7fc640 1 --2- 172.21.15.38:0/2410861400 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f40d8036b10 0x7f40d80d0d20 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f40d0009f40 tx=0x7f40d000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:01.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.501+0000 7f40c77fe640 1 -- 172.21.15.38:0/2410861400 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40d0020070 con 0x7f40d8036b10 2024-09-21T03:34:01.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.501+0000 7f40dffff640 1 -- 172.21.15.38:0/2410861400 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40d80d1660 con 0x7f40d8036b10 2024-09-21T03:34:01.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.501+0000 7f40dffff640 1 -- 172.21.15.38:0/2410861400 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40d80d1c20 con 0x7f40d8036b10 2024-09-21T03:34:01.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.502+0000 7f40c77fe640 1 -- 172.21.15.38:0/2410861400 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f40d000cc60 con 0x7f40d8036b10 2024-09-21T03:34:01.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.502+0000 7f40c77fe640 1 -- 172.21.15.38:0/2410861400 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40d0013a30 con 0x7f40d8036b10 2024-09-21T03:34:01.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.502+0000 7f73ae7fc640 1 --2- 172.21.15.38:0/3769005612 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f739c078b90 0x7f739c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:01.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.502+0000 7f73bde52640 1 --2- 172.21.15.38:0/3769005612 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f739c078b90 0x7f739c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:01.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.502+0000 7f73ae7fc640 1 -- 172.21.15.38:0/3769005612 <== mon.2 v2:172.21.15.195:3301/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f73a809b8b0 con 0x7f73b80ceb60 2024-09-21T03:34:01.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.503+0000 7f73bde52640 1 --2- 172.21.15.38:0/3769005612 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f739c078b90 0x7f739c07b0a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f73b4034000 tx=0x7f73b4004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:01.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.503+0000 7f3da724d640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3da006e330 0x7f3da0156850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:01.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.503+0000 7f70e8b89640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f70dc0081d0 0x7f70dc0cec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:01.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.503+0000 7f70e8b89640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f70dc055250 0x7f70dc0ddf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:01.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.503+0000 7f3da724d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3da0070e80 0x7f3da0156d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:01.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.503+0000 7f70e8b89640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f70dc056200 0x7f70dc0de480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:01.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.503+0000 7f70e8b89640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f70dc035340 con 0x7f70dc0081d0 2024-09-21T03:34:01.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.503+0000 7f70e8b89640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f70dc03b180 con 0x7f70dc055250 2024-09-21T03:34:01.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.503+0000 7f70e8b89640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f70dc0354c0 con 0x7f70dc056200 2024-09-21T03:34:01.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.503+0000 7f3da4fe9640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3da006e330 0x7f3da0156850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:01.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.503+0000 7f3da4fe9640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3da006e330 0x7f3da0156850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.38:50704/0 (socket says 172.21.15.38:50704) 2024-09-21T03:34:01.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.503+0000 7f70e2ffd640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f70dc055250 0x7f70dc0ddf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:01.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.504+0000 7fe00f04d640 1 Processor -- start 2024-09-21T03:34:01.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.504+0000 7f70e2ffd640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f70dc055250 0x7f70dc0ddf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.38:54890/0 (socket says 172.21.15.38:54890) 2024-09-21T03:34:01.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.504+0000 7f70e2ffd640 1 -- 172.21.15.38:0/2178265286 learned_addr learned my addr 172.21.15.38:0/2178265286 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:34:01.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.504+0000 7f3d9ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3da0070e80 0x7f3da0156d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:01.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.504+0000 7f3d9ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3da0070e80 0x7f3da0156d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42374/0 (socket says 172.21.15.38:42374) 2024-09-21T03:34:01.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.504+0000 7f3da4fe9640 1 -- 172.21.15.38:0/2680930232 learned_addr learned my addr 172.21.15.38:0/2680930232 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:34:01.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.504+0000 7fe00f04d640 1 -- start start 2024-09-21T03:34:01.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.504+0000 7f3da724d640 1 --2- 172.21.15.38:0/2680930232 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3da015aa40 0x7f3da009fa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:01.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.504+0000 7f40dffff640 1 -- 172.21.15.38:0/2410861400 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40c0005290 con 0x7f40d8036b10 2024-09-21T03:34:01.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.504+0000 7f3da724d640 1 -- 172.21.15.38:0/2680930232 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f3da0073660 con 0x7f3da0070e80 2024-09-21T03:34:01.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.504+0000 7f70e3fff640 1 --2- 172.21.15.38:0/2178265286 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f70dc056200 0x7f70dc0de480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:01.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.504+0000 7f3da724d640 1 -- 172.21.15.38:0/2680930232 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f3da00734e0 con 0x7f3da015aa40 2024-09-21T03:34:01.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.504+0000 7f3da724d640 1 -- 172.21.15.38:0/2680930232 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f3da00737e0 con 0x7f3da006e330 2024-09-21T03:34:01.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.504+0000 7f70e37fe640 1 --2- 172.21.15.38:0/2178265286 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f70dc0081d0 0x7f70dc0cec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:01.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.504+0000 7f70e2ffd640 1 -- 172.21.15.38:0/2178265286 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f70dc056200 msgr2=0x7f70dc0de480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:01.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.504+0000 7f70e2ffd640 1 --2- 172.21.15.38:0/2178265286 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f70dc056200 0x7f70dc0de480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:01.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.504+0000 7f70e2ffd640 1 -- 172.21.15.38:0/2178265286 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f70dc0081d0 msgr2=0x7f70dc0cec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:01.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.504+0000 7f70e2ffd640 1 --2- 172.21.15.38:0/2178265286 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f70dc0081d0 0x7f70dc0cec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:01.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.504+0000 7f70e2ffd640 1 -- 172.21.15.38:0/2178265286 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70dc0cf2e0 con 0x7f70dc055250 2024-09-21T03:34:01.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.504+0000 7f70e37fe640 1 --2- 172.21.15.38:0/2178265286 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f70dc0081d0 0x7f70dc0cec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:34:01.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.504+0000 7f43567fc640 1 -- 172.21.15.38:0/522059617 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f4350043020 con 0x7f43600ccc00 2024-09-21T03:34:01.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.504+0000 7f3da57ea640 1 --2- 172.21.15.38:0/2680930232 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3da015aa40 0x7f3da009fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:01.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.504+0000 7f40c77fe640 1 -- 172.21.15.38:0/2410861400 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f40d0023070 con 0x7f40d8036b10 2024-09-21T03:34:01.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.505+0000 7f3da4fe9640 1 -- 172.21.15.38:0/2680930232 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3da015aa40 msgr2=0x7f3da009fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:01.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.505+0000 7f3da4fe9640 1 --2- 172.21.15.38:0/2680930232 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3da015aa40 0x7f3da009fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:01.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.505+0000 7f3da4fe9640 1 -- 172.21.15.38:0/2680930232 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3da0070e80 msgr2=0x7f3da0156d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:01.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.505+0000 7f3da4fe9640 1 --2- 172.21.15.38:0/2680930232 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3da0070e80 0x7f3da0156d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:01.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.505+0000 7f3da4fe9640 1 -- 172.21.15.38:0/2680930232 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3da00c9830 con 0x7f3da006e330 2024-09-21T03:34:01.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.505+0000 7f70e2ffd640 1 --2- 172.21.15.38:0/2178265286 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f70dc055250 0x7f70dc0ddf40 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f70d000b420 tx=0x7f70d000b8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:01.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.505+0000 7f3da4fe9640 1 --2- 172.21.15.38:0/2680930232 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3da006e330 0x7f3da0156850 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f3d9400c430 tx=0x7f3d9400c8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:01.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.505+0000 7f43567fc640 1 --2- 172.21.15.38:0/522059617 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4344078f90 0x7f434407b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:01.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.505+0000 7f43567fc640 1 -- 172.21.15.38:0/522059617 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f4350039040 con 0x7f43600ccc00 2024-09-21T03:34:01.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.505+0000 7f40c77fe640 1 --2- 172.21.15.38:0/2410861400 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f40cc078bc0 0x7f40cc07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:01.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.506+0000 7f4365680640 1 --2- 172.21.15.38:0/522059617 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4344078f90 0x7f434407b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:01.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.506+0000 7f40deffd640 1 --2- 172.21.15.38:0/2410861400 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f40cc078bc0 0x7f40cc07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:01.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.506+0000 7f40c77fe640 1 -- 172.21.15.38:0/2410861400 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f40d009f430 con 0x7f40d8036b10 2024-09-21T03:34:01.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.506+0000 7fe00f04d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe008070c80 0x7fe0081f5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:01.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.506+0000 7fe00f04d640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe0081515c0 0x7fe00813f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:01.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.506+0000 7f4365680640 1 --2- 172.21.15.38:0/522059617 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4344078f90 0x7f434407b4a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f43600af620 tx=0x7f435800d040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:01.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.506+0000 7fe00f04d640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe0081519d0 0x7fe00815cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:01.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.506+0000 7fe00cde9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe008070c80 0x7fe0081f5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:01.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.506+0000 7f40deffd640 1 --2- 172.21.15.38:0/2410861400 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f40cc078bc0 0x7f40cc07b0d0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f40d4001960 tx=0x7f40d4009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:01.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.506+0000 7fe00cde9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe008070c80 0x7fe0081f5910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42386/0 (socket says 172.21.15.38:42386) 2024-09-21T03:34:01.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.506+0000 7fe007fff640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe0081515c0 0x7fe00813f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:01.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.506+0000 7fe007fff640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe0081515c0 0x7fe00813f270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.38:50720/0 (socket says 172.21.15.38:50720) 2024-09-21T03:34:01.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.507+0000 7fe00cde9640 1 -- 172.21.15.38:0/3062218490 learned_addr learned my addr 172.21.15.38:0/3062218490 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:34:01.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.507+0000 7fe00f04d640 1 -- 172.21.15.38:0/3062218490 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fe00815e9c0 con 0x7fe008070c80 2024-09-21T03:34:01.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.507+0000 7fe00f04d640 1 -- 172.21.15.38:0/3062218490 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fe00815e840 con 0x7fe0081519d0 2024-09-21T03:34:01.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.507+0000 7fe00f04d640 1 -- 172.21.15.38:0/3062218490 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fe00815eb40 con 0x7fe0081515c0 2024-09-21T03:34:01.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.507+0000 7fe00d5ea640 1 --2- 172.21.15.38:0/3062218490 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe0081519d0 0x7fe00815cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:01.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.507+0000 7fe00d5ea640 1 -- 172.21.15.38:0/3062218490 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe0081515c0 msgr2=0x7fe00813f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:01.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.507+0000 7fe00d5ea640 1 --2- 172.21.15.38:0/3062218490 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe0081515c0 0x7fe00813f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:01.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.507+0000 7fe00d5ea640 1 -- 172.21.15.38:0/3062218490 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe008070c80 msgr2=0x7fe0081f5910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:01.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.507+0000 7fe00d5ea640 1 --2- 172.21.15.38:0/3062218490 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe008070c80 0x7fe0081f5910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:01.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.507+0000 7fe00d5ea640 1 -- 172.21.15.38:0/3062218490 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0081480d0 con 0x7fe0081519d0 2024-09-21T03:34:01.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.507+0000 7fe00cde9640 1 --2- 172.21.15.38:0/3062218490 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe008070c80 0x7fe0081f5910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:34:01.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.507+0000 7f70cffff640 1 -- 172.21.15.38:0/2178265286 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70d0011830 con 0x7f70dc055250 2024-09-21T03:34:01.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.507+0000 7f70e8b89640 1 -- 172.21.15.38:0/2178265286 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70dc0d1fd0 con 0x7f70dc055250 2024-09-21T03:34:01.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.508+0000 7f43567fc640 1 -- 172.21.15.38:0/522059617 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f435004a030 con 0x7f43600ccc00 2024-09-21T03:34:01.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.507+0000 7f70e8b89640 1 -- 172.21.15.38:0/2178265286 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70dc04f780 con 0x7f70dc055250 2024-09-21T03:34:01.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.508+0000 7f70cffff640 1 -- 172.21.15.38:0/2178265286 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f70d0016070 con 0x7f70dc055250 2024-09-21T03:34:01.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.508+0000 7f70cffff640 1 -- 172.21.15.38:0/2178265286 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70d0011c70 con 0x7f70dc055250 2024-09-21T03:34:01.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.508+0000 7f73ae7fc640 1 -- 172.21.15.38:0/3769005612 <== mon.2 v2:172.21.15.195:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f73b81fdc20 con 0x7f73b80ceb60 2024-09-21T03:34:01.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.509+0000 7f9c9becc640 1 -- 172.21.15.38:0/139907061 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9c94072d90 msgr2=0x7f9c940731a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:01.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.508+0000 7fe00d5ea640 1 --2- 172.21.15.38:0/3062218490 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe0081519d0 0x7fe00815cec0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fdff800e640 tx=0x7fdff800eb00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:01.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.509+0000 7f9c9becc640 1 --2- 172.21.15.38:0/139907061 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9c94072d90 0x7f9c940731a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f9c9415b060 tx=0x7f9c8402a830 comp rx=0 tx=0).stop 2024-09-21T03:34:01.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.508+0000 7f3d9d7fa640 1 -- 172.21.15.38:0/2680930232 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d94011790 con 0x7f3da006e330 2024-09-21T03:34:01.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.509+0000 7f9c9becc640 1 -- 172.21.15.38:0/139907061 shutdown_connections 2024-09-21T03:34:01.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.509+0000 7f9c9becc640 1 --2- 172.21.15.38:0/139907061 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9c94072d90 0x7f9c940731a0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:01.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.509+0000 7f9c9becc640 1 --2- 172.21.15.38:0/139907061 >> v2:172.21.15.38:3300/0 conn(0x7f9c9406ddf0 0x7f9c9406e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:01.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.509+0000 7f9c9becc640 1 -- 172.21.15.38:0/139907061 >> 172.21.15.38:0/139907061 conn(0x7f9c940b1b90 msgr2=0x7f9c940b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:01.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.508+0000 7f3d9d7fa640 1 -- 172.21.15.38:0/2680930232 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3d94017070 con 0x7f3da006e330 2024-09-21T03:34:01.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.509+0000 7f3d9d7fa640 1 -- 172.21.15.38:0/2680930232 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d94011cc0 con 0x7f3da006e330 2024-09-21T03:34:01.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.509+0000 7f3da724d640 1 -- 172.21.15.38:0/2680930232 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3da00ad150 con 0x7f3da006e330 2024-09-21T03:34:01.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.509+0000 7f3da724d640 1 -- 172.21.15.38:0/2680930232 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f3da00ad710 con 0x7f3da006e330 2024-09-21T03:34:01.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.511+0000 7fe0057fa640 1 -- 172.21.15.38:0/3062218490 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdff8004220 con 0x7fe0081519d0 2024-09-21T03:34:01.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.511+0000 7fe0057fa640 1 -- 172.21.15.38:0/3062218490 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdff8005020 con 0x7fe0081519d0 2024-09-21T03:34:01.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.511+0000 7f3da724d640 1 -- 172.21.15.38:0/2680930232 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3da0073d00 con 0x7f3da006e330 2024-09-21T03:34:01.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.510+0000 7f70cdffb640 1 -- 172.21.15.38:0/2178265286 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70dc0d79b0 con 0x7f70dc055250 2024-09-21T03:34:01.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.512+0000 7f40c77fe640 1 -- 172.21.15.38:0/2410861400 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f40d0070840 con 0x7f40d8036b10 2024-09-21T03:34:01.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.512+0000 7f9c9becc640 1 -- 172.21.15.38:0/139907061 shutdown_connections 2024-09-21T03:34:01.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.512+0000 7f9c9becc640 1 -- 172.21.15.38:0/139907061 wait complete. 2024-09-21T03:34:01.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.512+0000 7f3d9d7fa640 1 -- 172.21.15.38:0/2680930232 <== mon.2 v2:172.21.15.195:3301/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f3d9401c460 con 0x7f3da006e330 2024-09-21T03:34:01.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.510+0000 7f70cffff640 1 -- 172.21.15.38:0/2178265286 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f70d001b460 con 0x7f70dc055250 2024-09-21T03:34:01.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.511+0000 7f70cffff640 1 --2- 172.21.15.38:0/2178265286 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f70c807cfa0 0x7f70c807f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:01.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.511+0000 7f70cffff640 1 -- 172.21.15.38:0/2178265286 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f70d009ad40 con 0x7f70dc055250 2024-09-21T03:34:01.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.512+0000 7f70e37fe640 1 --2- 172.21.15.38:0/2178265286 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f70c807cfa0 0x7f70c807f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:01.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.512+0000 7fe0057fa640 1 -- 172.21.15.38:0/3062218490 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdff8005470 con 0x7fe0081519d0 2024-09-21T03:34:01.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.512+0000 7fe00f04d640 1 -- 172.21.15.38:0/3062218490 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0081483b0 con 0x7fe0081519d0 2024-09-21T03:34:01.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.512+0000 7fe00f04d640 1 -- 172.21.15.38:0/3062218490 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe008148970 con 0x7fe0081519d0 2024-09-21T03:34:01.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.513+0000 7f3d9d7fa640 1 --2- 172.21.15.38:0/2680930232 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3d84078c90 0x7f3d8407b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:01.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.513+0000 7f3d9d7fa640 1 -- 172.21.15.38:0/2680930232 <== mon.2 v2:172.21.15.195:3301/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f3d9409ad30 con 0x7f3da006e330 2024-09-21T03:34:01.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.513+0000 7f3d9ffff640 1 --2- 172.21.15.38:0/2680930232 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3d84078c90 0x7f3d8407b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:01.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.513+0000 7fe00f04d640 1 -- 172.21.15.38:0/3062218490 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe008147e40 con 0x7fe0081519d0 2024-09-21T03:34:01.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.514+0000 7f70e37fe640 1 --2- 172.21.15.38:0/2178265286 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f70c807cfa0 0x7f70c807f4b0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f70d8034000 tx=0x7f70d8004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:01.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.514+0000 7f3d9ffff640 1 --2- 172.21.15.38:0/2680930232 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3d84078c90 0x7f3d8407b1a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f3da00aeeb0 tx=0x7f3d8c00b2e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:01.541 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.515+0000 7f9c9becc640 1 Processor -- start 2024-09-21T03:34:01.541 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.515+0000 7f9c9becc640 1 -- start start 2024-09-21T03:34:01.541 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.516+0000 7f70cffff640 1 -- 172.21.15.38:0/2178265286 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f70d0062ec0 con 0x7f70dc055250 2024-09-21T03:34:01.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.518+0000 7f9c9becc640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9c9406ddf0 0x7f9c940c95e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:01.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.518+0000 7f9c9becc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9c94072d90 0x7f9c941f5be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:01.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.518+0000 7f3d9d7fa640 1 -- 172.21.15.38:0/2680930232 <== mon.2 v2:172.21.15.195:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f3d940627c0 con 0x7f3da006e330 2024-09-21T03:34:01.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.519+0000 7f9c99c68640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9c9406ddf0 0x7f9c940c95e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:01.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.519+0000 7f9c99c68640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9c9406ddf0 0x7f9c940c95e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.38:50722/0 (socket says 172.21.15.38:50722) 2024-09-21T03:34:01.543 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.519+0000 7fe0057fa640 1 -- 172.21.15.38:0/3062218490 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fdff800c810 con 0x7fe0081519d0 2024-09-21T03:34:01.543 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.520+0000 7f9c99c68640 1 -- 172.21.15.38:0/174796544 learned_addr learned my addr 172.21.15.38:0/174796544 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:34:01.543 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.520+0000 7f9c9becc640 1 --2- 172.21.15.38:0/174796544 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9c9415ac20 0x7f9c9409ff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:01.543 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.520+0000 7f9c9becc640 1 -- 172.21.15.38:0/174796544 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f9c940736e0 con 0x7f9c94072d90 2024-09-21T03:34:01.543 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.520+0000 7f9c9becc640 1 -- 172.21.15.38:0/174796544 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f9c94071090 con 0x7f9c9415ac20 2024-09-21T03:34:01.543 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.520+0000 7f9c9becc640 1 -- 172.21.15.38:0/174796544 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f9c94073860 con 0x7f9c9406ddf0 2024-09-21T03:34:01.544 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.520+0000 7fe0057fa640 1 --2- 172.21.15.38:0/3062218490 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fdff4078c10 0x7fdff407b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:01.544 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.520+0000 7fe0057fa640 1 -- 172.21.15.38:0/3062218490 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fdff809b6b0 con 0x7fe0081519d0 2024-09-21T03:34:01.544 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.521+0000 7f9c99467640 1 --2- 172.21.15.38:0/174796544 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9c94072d90 0x7f9c941f5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:01.544 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.521+0000 7f9c99c68640 1 -- 172.21.15.38:0/174796544 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9c9415ac20 msgr2=0x7f9c9409ff40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T03:34:01.544 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.521+0000 7f9c99c68640 1 --2- 172.21.15.38:0/174796544 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9c9415ac20 0x7f9c9409ff40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:01.544 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.521+0000 7f9c99c68640 1 -- 172.21.15.38:0/174796544 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9c94072d90 msgr2=0x7f9c941f5be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:01.544 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.521+0000 7f9c99c68640 1 --2- 172.21.15.38:0/174796544 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9c94072d90 0x7f9c941f5be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:01.544 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.521+0000 7f9c99c68640 1 -- 172.21.15.38:0/174796544 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c940c4910 con 0x7f9c9406ddf0 2024-09-21T03:34:01.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.521+0000 7fe00cde9640 1 --2- 172.21.15.38:0/3062218490 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fdff4078c10 0x7fdff407b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:01.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.521+0000 7f9c99c68640 1 --2- 172.21.15.38:0/174796544 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9c9406ddf0 0x7f9c940c95e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f9c8c00c3d0 tx=0x7f9c8c00c890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:01.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.521+0000 7fe0057fa640 1 -- 172.21.15.38:0/3062218490 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fdff80632e0 con 0x7fe0081519d0 2024-09-21T03:34:01.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.521+0000 7f9c8a7fc640 1 -- 172.21.15.38:0/174796544 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c8c017070 con 0x7f9c9406ddf0 2024-09-21T03:34:01.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.522+0000 7f9c8a7fc640 1 -- 172.21.15.38:0/174796544 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9c8c0053c0 con 0x7f9c9406ddf0 2024-09-21T03:34:01.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.522+0000 7fe00cde9640 1 --2- 172.21.15.38:0/3062218490 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fdff4078c10 0x7fdff407b120 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe000001d30 tx=0x7fe000001cc0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:01.547 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.522+0000 7f9c8a7fc640 1 -- 172.21.15.38:0/174796544 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c8c003850 con 0x7f9c9406ddf0 2024-09-21T03:34:01.547 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.522+0000 7f9c9becc640 1 -- 172.21.15.38:0/174796544 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c940c4bf0 con 0x7f9c9406ddf0 2024-09-21T03:34:01.547 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.522+0000 7f9c9becc640 1 -- 172.21.15.38:0/174796544 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c940c51b0 con 0x7f9c9406ddf0 2024-09-21T03:34:01.547 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.523+0000 7f9c99467640 1 --2- 172.21.15.38:0/174796544 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9c94072d90 0x7f9c941f5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:34:01.547 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.525+0000 7f9c9becc640 1 -- 172.21.15.38:0/174796544 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c94073d80 con 0x7f9c9406ddf0 2024-09-21T03:34:01.547 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.526+0000 7f9c8a7fc640 1 -- 172.21.15.38:0/174796544 <== mon.2 v2:172.21.15.195:3301/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f9c8c01a070 con 0x7f9c9406ddf0 2024-09-21T03:34:01.547 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.526+0000 7f9c8a7fc640 1 --2- 172.21.15.38:0/174796544 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9c78078b90 0x7f9c7807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:01.547 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.527+0000 7f9c8a7fc640 1 -- 172.21.15.38:0/174796544 <== mon.2 v2:172.21.15.195:3301/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f9c8c09a420 con 0x7f9c9406ddf0 2024-09-21T03:34:01.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.527+0000 7f9c99467640 1 --2- 172.21.15.38:0/174796544 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9c78078b90 0x7f9c7807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:01.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.528+0000 7f9c99467640 1 --2- 172.21.15.38:0/174796544 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9c78078b90 0x7f9c7807b0a0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f9c940b05d0 tx=0x7f9c90009af0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:01.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.533+0000 7f9c8a7fc640 1 -- 172.21.15.38:0/174796544 <== mon.2 v2:172.21.15.195:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f9c8c062520 con 0x7f9c9406ddf0 2024-09-21T03:34:01.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.882+0000 7f41f10cc640 1 -- 172.21.15.38:0/3103553342 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f41cc005580 con 0x7f41e40e7440 2024-09-21T03:34:01.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.883+0000 7f41d37fe640 1 -- 172.21.15.38:0/3103553342 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f41e0061de0 con 0x7f41e40e7440 2024-09-21T03:34:01.885 INFO:teuthology.orchestra.run.smithi038.stdout:64424509451 2024-09-21T03:34:01.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.887+0000 7f41d17fa640 1 -- 172.21.15.38:0/3103553342 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f41d8078c90 msgr2=0x7f41d807b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:01.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.887+0000 7f41d17fa640 1 --2- 172.21.15.38:0/3103553342 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f41d8078c90 0x7f41d807b1a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f41ec066b30 tx=0x7f41ec06c4c0 comp rx=0 tx=0).stop 2024-09-21T03:34:01.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.887+0000 7f41d17fa640 1 -- 172.21.15.38:0/3103553342 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f41e40e7440 msgr2=0x7f41e40e52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:01.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.887+0000 7f41d17fa640 1 --2- 172.21.15.38:0/3103553342 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f41e40e7440 0x7f41e40e52d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f41e000c3c0 tx=0x7f41e000c880 comp rx=0 tx=0).stop 2024-09-21T03:34:01.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.887+0000 7f41d17fa640 1 -- 172.21.15.38:0/3103553342 shutdown_connections 2024-09-21T03:34:01.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.887+0000 7f41d17fa640 1 --2- 172.21.15.38:0/3103553342 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f41e40e7440 0x7f41e40e52d0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:01.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.888+0000 7f41d17fa640 1 --2- 172.21.15.38:0/3103553342 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f41d8078c90 0x7f41d807b1a0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:01.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.888+0000 7f41d17fa640 1 --2- 172.21.15.38:0/3103553342 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f41e40e7030 0x7f41e40312c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:01.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.888+0000 7f41d17fa640 1 --2- 172.21.15.38:0/3103553342 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f41e4052390 0x7f41e40e3800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:01.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.888+0000 7f41d17fa640 1 -- 172.21.15.38:0/3103553342 >> 172.21.15.38:0/3103553342 conn(0x7f41e401d9d0 msgr2=0x7f41e4009ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:01.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.888+0000 7f41d17fa640 1 -- 172.21.15.38:0/3103553342 shutdown_connections 2024-09-21T03:34:01.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.888+0000 7f41d17fa640 1 -- 172.21.15.38:0/3103553342 wait complete. 2024-09-21T03:34:01.907 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509451 got 64424509451 for osd.2 2024-09-21T03:34:01.907 DEBUG:teuthology.parallel:result is None 2024-09-21T03:34:01.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.925+0000 7f3da724d640 1 -- 172.21.15.38:0/2680930232 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f3da00af7e0 con 0x7f3da006e330 2024-09-21T03:34:01.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.926+0000 7f3d9d7fa640 1 -- 172.21.15.38:0/2680930232 <== mon.2 v2:172.21.15.195:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f3d940671f0 con 0x7f3da006e330 2024-09-21T03:34:01.928 INFO:teuthology.orchestra.run.smithi038.stdout:68719476747 2024-09-21T03:34:01.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.929+0000 7f3da724d640 1 -- 172.21.15.38:0/2680930232 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3d84078c90 msgr2=0x7f3d8407b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:01.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.929+0000 7f3da724d640 1 --2- 172.21.15.38:0/2680930232 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3d84078c90 0x7f3d8407b1a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f3da00aeeb0 tx=0x7f3d8c00b2e0 comp rx=0 tx=0).stop 2024-09-21T03:34:01.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.929+0000 7f3da724d640 1 -- 172.21.15.38:0/2680930232 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3da006e330 msgr2=0x7f3da0156850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:01.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.929+0000 7f3da724d640 1 --2- 172.21.15.38:0/2680930232 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3da006e330 0x7f3da0156850 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f3d9400c430 tx=0x7f3d9400c8f0 comp rx=0 tx=0).stop 2024-09-21T03:34:01.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.929+0000 7f3da724d640 1 -- 172.21.15.38:0/2680930232 shutdown_connections 2024-09-21T03:34:01.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.929+0000 7f3da724d640 1 --2- 172.21.15.38:0/2680930232 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3da015aa40 0x7f3da009fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:01.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.929+0000 7f3da724d640 1 --2- 172.21.15.38:0/2680930232 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3da0070e80 0x7f3da0156d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:01.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.929+0000 7f3da724d640 1 --2- 172.21.15.38:0/2680930232 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3d84078c90 0x7f3d8407b1a0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:01.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.929+0000 7f3da724d640 1 --2- 172.21.15.38:0/2680930232 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3da006e330 0x7f3da0156850 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:01.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.929+0000 7f3da724d640 1 -- 172.21.15.38:0/2680930232 >> 172.21.15.38:0/2680930232 conn(0x7f3da00b1b90 msgr2=0x7f3da00b0e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:01.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.930+0000 7f3da724d640 1 -- 172.21.15.38:0/2680930232 shutdown_connections 2024-09-21T03:34:01.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.930+0000 7f3da724d640 1 -- 172.21.15.38:0/2680930232 wait complete. 2024-09-21T03:34:01.944 INFO:tasks.ceph.ceph_manager.ceph:need seq 68719476746 got 68719476747 for osd.1 2024-09-21T03:34:01.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.942+0000 7f4366e83640 1 -- 172.21.15.38:0/522059617 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f4348005850 con 0x7f43600ccc00 2024-09-21T03:34:01.944 DEBUG:teuthology.parallel:result is None 2024-09-21T03:34:01.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.942+0000 7f70cdffb640 1 -- 172.21.15.38:0/2178265286 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f70dc034e00 con 0x7f70dc055250 2024-09-21T03:34:01.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.943+0000 7f70cffff640 1 -- 172.21.15.38:0/2178265286 <== mon.1 v2:172.21.15.195:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f70d00627c0 con 0x7f70dc055250 2024-09-21T03:34:01.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.943+0000 7f43567fc640 1 -- 172.21.15.38:0/522059617 <== mon.1 v2:172.21.15.195:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f43500833f0 con 0x7f43600ccc00 2024-09-21T03:34:01.945 INFO:teuthology.orchestra.run.smithi038.stdout:60129542155 2024-09-21T03:34:01.945 INFO:teuthology.orchestra.run.smithi038.stdout:60129542154 2024-09-21T03:34:01.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.945+0000 7f4366e83640 1 -- 172.21.15.38:0/522059617 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4344078f90 msgr2=0x7f434407b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:01.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.945+0000 7f4366e83640 1 --2- 172.21.15.38:0/522059617 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4344078f90 0x7f434407b4a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f43600af620 tx=0x7f435800d040 comp rx=0 tx=0).stop 2024-09-21T03:34:01.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.945+0000 7f4366e83640 1 -- 172.21.15.38:0/522059617 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f43600ccc00 msgr2=0x7f436014f880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:01.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.945+0000 7f4366e83640 1 --2- 172.21.15.38:0/522059617 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f43600ccc00 0x7f436014f880 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f4350035040 tx=0x7f435002ca00 comp rx=0 tx=0).stop 2024-09-21T03:34:01.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.945+0000 7f4366e83640 1 -- 172.21.15.38:0/522059617 shutdown_connections 2024-09-21T03:34:01.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.945+0000 7f4366e83640 1 --2- 172.21.15.38:0/522059617 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f436015ecd0 0x7f4360153120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:01.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.945+0000 7f4366e83640 1 --2- 172.21.15.38:0/522059617 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4344078f90 0x7f434407b4a0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:01.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.945+0000 7f4366e83640 1 --2- 172.21.15.38:0/522059617 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f436015b7a0 0x7f4360152be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:01.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.945+0000 7f4366e83640 1 --2- 172.21.15.38:0/522059617 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f43600ccc00 0x7f436014f880 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:01.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.945+0000 7f4366e83640 1 -- 172.21.15.38:0/522059617 >> 172.21.15.38:0/522059617 conn(0x7f436006f920 msgr2=0x7f43600cf110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:01.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.945+0000 7f4366e83640 1 -- 172.21.15.38:0/522059617 shutdown_connections 2024-09-21T03:34:01.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.945+0000 7f4366e83640 1 -- 172.21.15.38:0/522059617 wait complete. 2024-09-21T03:34:01.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.947+0000 7f70cdffb640 1 -- 172.21.15.38:0/2178265286 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f70c807cfa0 msgr2=0x7f70c807f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:01.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.947+0000 7f70cdffb640 1 --2- 172.21.15.38:0/2178265286 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f70c807cfa0 0x7f70c807f4b0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f70d8034000 tx=0x7f70d8004730 comp rx=0 tx=0).stop 2024-09-21T03:34:01.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.947+0000 7f70cdffb640 1 -- 172.21.15.38:0/2178265286 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f70dc055250 msgr2=0x7f70dc0ddf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:01.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.947+0000 7f70cdffb640 1 --2- 172.21.15.38:0/2178265286 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f70dc055250 0x7f70dc0ddf40 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f70d000b420 tx=0x7f70d000b8e0 comp rx=0 tx=0).stop 2024-09-21T03:34:01.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.947+0000 7f70cdffb640 1 -- 172.21.15.38:0/2178265286 shutdown_connections 2024-09-21T03:34:01.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.947+0000 7f70cdffb640 1 --2- 172.21.15.38:0/2178265286 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f70c807cfa0 0x7f70c807f4b0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:01.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.947+0000 7f70cdffb640 1 --2- 172.21.15.38:0/2178265286 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f70dc056200 0x7f70dc0de480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:01.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.947+0000 7f70cdffb640 1 --2- 172.21.15.38:0/2178265286 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f70dc055250 0x7f70dc0ddf40 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:01.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.947+0000 7f70cdffb640 1 --2- 172.21.15.38:0/2178265286 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f70dc0081d0 0x7f70dc0cec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:01.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.947+0000 7f70cdffb640 1 -- 172.21.15.38:0/2178265286 >> 172.21.15.38:0/2178265286 conn(0x7f70dc01eac0 msgr2=0x7f70dc0344f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:01.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.947+0000 7f70cdffb640 1 -- 172.21.15.38:0/2178265286 shutdown_connections 2024-09-21T03:34:01.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.948+0000 7f70cdffb640 1 -- 172.21.15.38:0/2178265286 wait complete. 2024-09-21T03:34:01.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.948+0000 7f73c00b6640 1 -- 172.21.15.38:0/3769005612 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f73a0005580 con 0x7f73b80ceb60 2024-09-21T03:34:01.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.949+0000 7f73ae7fc640 1 -- 172.21.15.38:0/3769005612 <== mon.2 v2:172.21.15.195:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f73a8063b00 con 0x7f73b80ceb60 2024-09-21T03:34:01.951 INFO:teuthology.orchestra.run.smithi038.stdout:64424509450 2024-09-21T03:34:01.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.952+0000 7f73c00b6640 1 -- 172.21.15.38:0/3769005612 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f739c078b90 msgr2=0x7f739c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:01.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.952+0000 7f73c00b6640 1 --2- 172.21.15.38:0/3769005612 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f739c078b90 0x7f739c07b0a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f73b4034000 tx=0x7f73b4004730 comp rx=0 tx=0).stop 2024-09-21T03:34:01.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.952+0000 7f73c00b6640 1 -- 172.21.15.38:0/3769005612 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f73b80ceb60 msgr2=0x7f73b81fd6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:01.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.952+0000 7f73c00b6640 1 --2- 172.21.15.38:0/3769005612 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f73b80ceb60 0x7f73b81fd6e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f73a800e790 tx=0x7f73a800ec50 comp rx=0 tx=0).stop 2024-09-21T03:34:01.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.952+0000 7f73c00b6640 1 -- 172.21.15.38:0/3769005612 shutdown_connections 2024-09-21T03:34:01.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.952+0000 7f73c00b6640 1 --2- 172.21.15.38:0/3769005612 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f73b8167410 0x7f73b806fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:01.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.952+0000 7f73c00b6640 1 --2- 172.21.15.38:0/3769005612 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f739c078b90 0x7f739c07b0a0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:01.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.952+0000 7f73c00b6640 1 --2- 172.21.15.38:0/3769005612 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f73b80ceb60 0x7f73b81fd6e0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:01.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.952+0000 7f73c00b6640 1 --2- 172.21.15.38:0/3769005612 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f73b806e330 0x7f73b81618d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:01.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.952+0000 7f73c00b6640 1 -- 172.21.15.38:0/3769005612 >> 172.21.15.38:0/3769005612 conn(0x7f73b80b1b90 msgr2=0x7f73b806ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:01.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.953+0000 7f73c00b6640 1 -- 172.21.15.38:0/3769005612 shutdown_connections 2024-09-21T03:34:01.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.953+0000 7f73c00b6640 1 -- 172.21.15.38:0/3769005612 wait complete. 2024-09-21T03:34:01.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.953+0000 7f40dffff640 1 -- 172.21.15.38:0/2410861400 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f40c0005580 con 0x7f40d8036b10 2024-09-21T03:34:01.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.953+0000 7f40c77fe640 1 -- 172.21.15.38:0/2410861400 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f40d000ce10 con 0x7f40d8036b10 2024-09-21T03:34:01.956 INFO:teuthology.orchestra.run.smithi038.stdout:64424509450 2024-09-21T03:34:01.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.956+0000 7f40dffff640 1 -- 172.21.15.38:0/2410861400 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f40cc078bc0 msgr2=0x7f40cc07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:01.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.956+0000 7f40dffff640 1 --2- 172.21.15.38:0/2410861400 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f40cc078bc0 0x7f40cc07b0d0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f40d4001960 tx=0x7f40d4009040 comp rx=0 tx=0).stop 2024-09-21T03:34:01.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.956+0000 7f40dffff640 1 -- 172.21.15.38:0/2410861400 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f40d8036b10 msgr2=0x7f40d80d0d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:01.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.956+0000 7f40dffff640 1 --2- 172.21.15.38:0/2410861400 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f40d8036b10 0x7f40d80d0d20 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f40d0009f40 tx=0x7f40d000ee80 comp rx=0 tx=0).stop 2024-09-21T03:34:01.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.957+0000 7f40dffff640 1 -- 172.21.15.38:0/2410861400 shutdown_connections 2024-09-21T03:34:01.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.957+0000 7f40dffff640 1 --2- 172.21.15.38:0/2410861400 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f40cc078bc0 0x7f40cc07b0d0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:01.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.957+0000 7f40dffff640 1 --2- 172.21.15.38:0/2410861400 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f40d80d4d90 0x7f40d8177aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:01.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.957+0000 7f40dffff640 1 --2- 172.21.15.38:0/2410861400 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f40d8036b10 0x7f40d80d0d20 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:01.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.957+0000 7f40dffff640 1 --2- 172.21.15.38:0/2410861400 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f40d8007d50 0x7f40d804ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:01.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.957+0000 7f40dffff640 1 -- 172.21.15.38:0/2410861400 >> 172.21.15.38:0/2410861400 conn(0x7f40d801e230 msgr2=0x7f40d800aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:01.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.957+0000 7f40dffff640 1 -- 172.21.15.38:0/2410861400 shutdown_connections 2024-09-21T03:34:01.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.957+0000 7f40dffff640 1 -- 172.21.15.38:0/2410861400 wait complete. 2024-09-21T03:34:01.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.959+0000 7fe00f04d640 1 -- 172.21.15.38:0/3062218490 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fe00815e300 con 0x7fe0081519d0 2024-09-21T03:34:01.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.959+0000 7fe0057fa640 1 -- 172.21.15.38:0/3062218490 <== mon.1 v2:172.21.15.195:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fdff8067d10 con 0x7fe0081519d0 2024-09-21T03:34:01.961 INFO:teuthology.orchestra.run.smithi038.stdout:64424509451 2024-09-21T03:34:01.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.962+0000 7f9c9becc640 1 -- 172.21.15.38:0/174796544 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f9c940af7e0 con 0x7f9c9406ddf0 2024-09-21T03:34:01.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.962+0000 7fe00f04d640 1 -- 172.21.15.38:0/3062218490 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fdff4078c10 msgr2=0x7fdff407b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:01.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.962+0000 7fe00f04d640 1 --2- 172.21.15.38:0/3062218490 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fdff4078c10 0x7fdff407b120 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe000001d30 tx=0x7fe000001cc0 comp rx=0 tx=0).stop 2024-09-21T03:34:01.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.962+0000 7fe00f04d640 1 -- 172.21.15.38:0/3062218490 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe0081519d0 msgr2=0x7fe00815cec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:01.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.962+0000 7fe00f04d640 1 --2- 172.21.15.38:0/3062218490 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe0081519d0 0x7fe00815cec0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fdff800e640 tx=0x7fdff800eb00 comp rx=0 tx=0).stop 2024-09-21T03:34:01.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.962+0000 7fe00f04d640 1 -- 172.21.15.38:0/3062218490 shutdown_connections 2024-09-21T03:34:01.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.962+0000 7fe00f04d640 1 --2- 172.21.15.38:0/3062218490 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe0081519d0 0x7fe00815cec0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:01.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.963+0000 7fe00f04d640 1 --2- 172.21.15.38:0/3062218490 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fdff4078c10 0x7fdff407b120 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:01.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.963+0000 7fe00f04d640 1 --2- 172.21.15.38:0/3062218490 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe0081515c0 0x7fe00813f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:01.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.963+0000 7fe00f04d640 1 --2- 172.21.15.38:0/3062218490 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe008070c80 0x7fe0081f5910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:01.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.963+0000 7fe00f04d640 1 -- 172.21.15.38:0/3062218490 >> 172.21.15.38:0/3062218490 conn(0x7fe0080d0590 msgr2=0x7fe00814d3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:01.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.963+0000 7fe00f04d640 1 -- 172.21.15.38:0/3062218490 shutdown_connections 2024-09-21T03:34:01.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.963+0000 7fe00f04d640 1 -- 172.21.15.38:0/3062218490 wait complete. 2024-09-21T03:34:01.966 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542155 got 60129542155 for osd.4 2024-09-21T03:34:01.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.963+0000 7f9c8a7fc640 1 -- 172.21.15.38:0/174796544 <== mon.2 v2:172.21.15.195:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f9c8c061e20 con 0x7f9c9406ddf0 2024-09-21T03:34:01.967 INFO:teuthology.orchestra.run.smithi038.stdout:55834574858 2024-09-21T03:34:01.967 DEBUG:teuthology.parallel:result is None 2024-09-21T03:34:01.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.966+0000 7f9c9becc640 1 -- 172.21.15.38:0/174796544 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9c78078b90 msgr2=0x7f9c7807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:01.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.966+0000 7f9c9becc640 1 --2- 172.21.15.38:0/174796544 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9c78078b90 0x7f9c7807b0a0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f9c940b05d0 tx=0x7f9c90009af0 comp rx=0 tx=0).stop 2024-09-21T03:34:01.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.966+0000 7f9c9becc640 1 -- 172.21.15.38:0/174796544 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9c9406ddf0 msgr2=0x7f9c940c95e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:01.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.966+0000 7f9c9becc640 1 --2- 172.21.15.38:0/174796544 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9c9406ddf0 0x7f9c940c95e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f9c8c00c3d0 tx=0x7f9c8c00c890 comp rx=0 tx=0).stop 2024-09-21T03:34:01.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.966+0000 7f9c9becc640 1 -- 172.21.15.38:0/174796544 shutdown_connections 2024-09-21T03:34:01.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.966+0000 7f9c9becc640 1 --2- 172.21.15.38:0/174796544 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9c9415ac20 0x7f9c9409ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:01.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.966+0000 7f9c9becc640 1 --2- 172.21.15.38:0/174796544 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9c94072d90 0x7f9c941f5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:01.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.966+0000 7f9c9becc640 1 --2- 172.21.15.38:0/174796544 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9c78078b90 0x7f9c7807b0a0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:01.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.966+0000 7f9c9becc640 1 --2- 172.21.15.38:0/174796544 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9c9406ddf0 0x7f9c940c95e0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:01.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.966+0000 7f9c9becc640 1 -- 172.21.15.38:0/174796544 >> 172.21.15.38:0/174796544 conn(0x7f9c940b1b90 msgr2=0x7f9c9415a110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:01.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.966+0000 7f9c9becc640 1 -- 172.21.15.38:0/174796544 shutdown_connections 2024-09-21T03:34:01.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:01.966+0000 7f9c9becc640 1 -- 172.21.15.38:0/174796544 wait complete. 2024-09-21T03:34:01.970 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542154 got 60129542154 for osd.6 2024-09-21T03:34:01.970 DEBUG:teuthology.parallel:result is None 2024-09-21T03:34:01.976 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509450 got 64424509450 for osd.0 2024-09-21T03:34:01.976 DEBUG:teuthology.parallel:result is None 2024-09-21T03:34:01.978 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509450 got 64424509450 for osd.3 2024-09-21T03:34:01.979 DEBUG:teuthology.parallel:result is None 2024-09-21T03:34:01.979 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509451 got 64424509451 for osd.7 2024-09-21T03:34:01.979 DEBUG:teuthology.parallel:result is None 2024-09-21T03:34:01.981 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.5 2024-09-21T03:34:01.981 DEBUG:teuthology.parallel:result is None 2024-09-21T03:34:01.982 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-21T03:34:01.982 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T03:34:02.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:02.136+0000 7f07dfd13640 1 -- 172.21.15.38:0/1770125316 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f07d8162590 msgr2=0x7f07d81629a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:02.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:02.136+0000 7f07dfd13640 1 --2- 172.21.15.38:0/1770125316 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f07d8162590 0x7f07d81629a0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f07c8009830 tx=0x7f07c802a5c0 comp rx=0 tx=0).stop 2024-09-21T03:34:02.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:02.136+0000 7f07dfd13640 1 -- 172.21.15.38:0/1770125316 shutdown_connections 2024-09-21T03:34:02.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:02.136+0000 7f07dfd13640 1 --2- 172.21.15.38:0/1770125316 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f07d8162590 0x7f07d81629a0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:02.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:02.136+0000 7f07dfd13640 1 -- 172.21.15.38:0/1770125316 >> 172.21.15.38:0/1770125316 conn(0x7f07d80d4440 msgr2=0x7f07d80d4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:02.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:02.136+0000 7f07dfd13640 1 -- 172.21.15.38:0/1770125316 shutdown_connections 2024-09-21T03:34:02.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:02.136+0000 7f07dfd13640 1 -- 172.21.15.38:0/1770125316 wait complete. 2024-09-21T03:34:02.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:02.138+0000 7f07dfd13640 1 Processor -- start 2024-09-21T03:34:02.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:02.138+0000 7f07dfd13640 1 -- start start 2024-09-21T03:34:02.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:02.139+0000 7f07dfd13640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f07d80b2e60 0x7f07d81f9850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:02.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:02.139+0000 7f07dfd13640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f07d80cfe00 0x7f07d81f64f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:02.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:02.139+0000 7f07dfd13640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f07d8162590 0x7f07d81f5430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:02.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:02.139+0000 7f07dfd13640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f07d80b3b00 con 0x7f07d80cfe00 2024-09-21T03:34:02.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:02.139+0000 7f07dfd13640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f07d80b3980 con 0x7f07d8162590 2024-09-21T03:34:02.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:02.139+0000 7f07dfd13640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f07d80b4f70 con 0x7f07d80b2e60 2024-09-21T03:34:02.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:02.140+0000 7f07dd2ae640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f07d80cfe00 0x7f07d81f64f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:02.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:02.140+0000 7f07dd2ae640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f07d80cfe00 0x7f07d81f64f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42398/0 (socket says 172.21.15.38:42398) 2024-09-21T03:34:02.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:02.140+0000 7f07ddaaf640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f07d80b2e60 0x7f07d81f9850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:02.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:02.140+0000 7f07dd2ae640 1 -- 172.21.15.38:0/3076329923 learned_addr learned my addr 172.21.15.38:0/3076329923 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:34:02.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:02.140+0000 7f07de2b0640 1 --2- 172.21.15.38:0/3076329923 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f07d8162590 0x7f07d81f5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:02.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:02.140+0000 7f07dd2ae640 1 -- 172.21.15.38:0/3076329923 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f07d80b2e60 msgr2=0x7f07d81f9850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:02.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:02.140+0000 7f07dd2ae640 1 --2- 172.21.15.38:0/3076329923 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f07d80b2e60 0x7f07d81f9850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:02.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:02.140+0000 7f07dd2ae640 1 -- 172.21.15.38:0/3076329923 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f07d8162590 msgr2=0x7f07d81f5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:02.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:02.140+0000 7f07dd2ae640 1 --2- 172.21.15.38:0/3076329923 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f07d8162590 0x7f07d81f5430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:02.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:02.140+0000 7f07dd2ae640 1 -- 172.21.15.38:0/3076329923 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07d81fa230 con 0x7f07d80cfe00 2024-09-21T03:34:02.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:02.140+0000 7f07dd2ae640 1 --2- 172.21.15.38:0/3076329923 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f07d80cfe00 0x7f07d81f64f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f07c8009950 tx=0x7f07c8033ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:02.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:02.140+0000 7f07ce7fc640 1 -- 172.21.15.38:0/3076329923 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07c8031d60 con 0x7f07d80cfe00 2024-09-21T03:34:02.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:02.140+0000 7f07ce7fc640 1 -- 172.21.15.38:0/3076329923 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f07c8005020 con 0x7f07d80cfe00 2024-09-21T03:34:02.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:02.140+0000 7f07dfd13640 1 -- 172.21.15.38:0/3076329923 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07d81fa510 con 0x7f07d80cfe00 2024-09-21T03:34:02.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:02.140+0000 7f07ce7fc640 1 -- 172.21.15.38:0/3076329923 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07c8005470 con 0x7f07d80cfe00 2024-09-21T03:34:02.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:02.140+0000 7f07dfd13640 1 -- 172.21.15.38:0/3076329923 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07d81ea9b0 con 0x7f07d80cfe00 2024-09-21T03:34:02.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:02.141+0000 7f07dfd13640 1 -- 172.21.15.38:0/3076329923 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07d81f9fa0 con 0x7f07d80cfe00 2024-09-21T03:34:02.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:02.142+0000 7f07ce7fc640 1 -- 172.21.15.38:0/3076329923 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f07c80047a0 con 0x7f07d80cfe00 2024-09-21T03:34:02.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:02.143+0000 7f07ce7fc640 1 --2- 172.21.15.38:0/3076329923 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f07bc078bc0 0x7f07bc07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:02.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:02.143+0000 7f07ce7fc640 1 -- 172.21.15.38:0/3076329923 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f07c80bc1a0 con 0x7f07d80cfe00 2024-09-21T03:34:02.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:02.143+0000 7f07ddaaf640 1 --2- 172.21.15.38:0/3076329923 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f07bc078bc0 0x7f07bc07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:02.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:02.144+0000 7f07ddaaf640 1 --2- 172.21.15.38:0/3076329923 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f07bc078bc0 0x7f07bc07b0d0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f07d4001720 tx=0x7f07d4009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:02.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:02.148+0000 7f07ce7fc640 1 -- 172.21.15.38:0/3076329923 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f07c803f030 con 0x7f07d80cfe00 2024-09-21T03:34:02.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:02.332+0000 7f07dfd13640 1 -- 172.21.15.38:0/3076329923 --> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f07d80d1800 con 0x7f07bc078bc0 2024-09-21T03:34:02.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:02.338+0000 7f07ce7fc640 1 -- 172.21.15.38:0/3076329923 <== mgr.4117 v2:172.21.15.195:6800/1631901344 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332519 (secure 0 0 0) 0x7f07d80d1800 con 0x7f07bc078bc0 2024-09-21T03:34:02.343 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:34:02.345 INFO:teuthology.orchestra.run.smithi038.stderr:dumped all 2024-09-21T03:34:02.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:02.342+0000 7f07dfd13640 1 -- 172.21.15.38:0/3076329923 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f07bc078bc0 msgr2=0x7f07bc07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:02.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:02.342+0000 7f07dfd13640 1 --2- 172.21.15.38:0/3076329923 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f07bc078bc0 0x7f07bc07b0d0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f07d4001720 tx=0x7f07d4009040 comp rx=0 tx=0).stop 2024-09-21T03:34:02.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:02.342+0000 7f07dfd13640 1 -- 172.21.15.38:0/3076329923 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f07d80cfe00 msgr2=0x7f07d81f64f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:02.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:02.342+0000 7f07dfd13640 1 --2- 172.21.15.38:0/3076329923 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f07d80cfe00 0x7f07d81f64f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f07c8009950 tx=0x7f07c8033ce0 comp rx=0 tx=0).stop 2024-09-21T03:34:02.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:02.342+0000 7f07dfd13640 1 -- 172.21.15.38:0/3076329923 shutdown_connections 2024-09-21T03:34:02.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:02.342+0000 7f07dfd13640 1 --2- 172.21.15.38:0/3076329923 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f07d8162590 0x7f07d81f5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:02.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:02.342+0000 7f07dfd13640 1 --2- 172.21.15.38:0/3076329923 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f07d80cfe00 0x7f07d81f64f0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:02.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:02.342+0000 7f07dfd13640 1 --2- 172.21.15.38:0/3076329923 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f07bc078bc0 0x7f07bc07b0d0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:02.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:02.342+0000 7f07dfd13640 1 --2- 172.21.15.38:0/3076329923 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f07d80b2e60 0x7f07d81f9850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:02.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:02.342+0000 7f07dfd13640 1 -- 172.21.15.38:0/3076329923 >> 172.21.15.38:0/3076329923 conn(0x7f07d80d4440 msgr2=0x7f07d80d0c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:02.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:02.342+0000 7f07dfd13640 1 -- 172.21.15.38:0/3076329923 shutdown_connections 2024-09-21T03:34:02.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:02.342+0000 7f07dfd13640 1 -- 172.21.15.38:0/3076329923 wait complete. 2024-09-21T03:34:02.355 INFO:teuthology.orchestra.run.smithi038.stdout:{"pg_ready":true,"pg_map":{"version":38,"stamp":"2024-09-21T03:34:00.983109+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":149,"num_write_kb":1646,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":114,"ondisk_log_size":114,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":252840,"kb_used_data":3336,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754721880,"statfs":{"total":773094113280,"available":772835205120,"internally_reserved":0,"allocated":3416064,"data_stored":2048944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12748,"internal_metadata":255315508},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2230,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003755"},"pg_stats":[{"pgid":"2.1f","version":"21'4","reported_seq":27,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.285664+0000","last_change":"2024-09-21T03:33:35.252626+0000","last_active":"2024-09-21T03:33:41.285664+0000","last_peered":"2024-09-21T03:33:41.285664+0000","last_clean":"2024-09-21T03:33:41.285664+0000","last_became_active":"2024-09-21T03:33:35.252204+0000","last_became_peered":"2024-09-21T03:33:35.252204+0000","last_unstale":"2024-09-21T03:33:41.285664+0000","last_undegraded":"2024-09-21T03:33:41.285664+0000","last_fullsized":"2024-09-21T03:33:41.285664+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_clean_scrub_stamp":"2024-09-21T03:33:34.220881+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:42:59.930617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.920863+0000","last_change":"2024-09-21T03:33:36.262395+0000","last_active":"2024-09-21T03:33:42.920863+0000","last_peered":"2024-09-21T03:33:42.920863+0000","last_clean":"2024-09-21T03:33:42.920863+0000","last_became_active":"2024-09-21T03:33:36.262085+0000","last_became_peered":"2024-09-21T03:33:36.262085+0000","last_unstale":"2024-09-21T03:33:42.920863+0000","last_undegraded":"2024-09-21T03:33:42.920863+0000","last_fullsized":"2024-09-21T03:33:42.920863+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:13:46.036072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.754945+0000","last_change":"2024-09-21T03:33:40.303707+0000","last_active":"2024-09-21T03:33:41.754945+0000","last_peered":"2024-09-21T03:33:41.754945+0000","last_clean":"2024-09-21T03:33:41.754945+0000","last_became_active":"2024-09-21T03:33:40.303328+0000","last_became_peered":"2024-09-21T03:33:40.303328+0000","last_unstale":"2024-09-21T03:33:41.754945+0000","last_undegraded":"2024-09-21T03:33:41.754945+0000","last_fullsized":"2024-09-21T03:33:41.754945+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_clean_scrub_stamp":"2024-09-21T03:33:39.262231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:58:01.084602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4,1],"acting":[7,6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.921084+0000","last_change":"2024-09-21T03:33:35.253147+0000","last_active":"2024-09-21T03:33:42.921084+0000","last_peered":"2024-09-21T03:33:42.921084+0000","last_clean":"2024-09-21T03:33:42.921084+0000","last_became_active":"2024-09-21T03:33:35.251243+0000","last_became_peered":"2024-09-21T03:33:35.251243+0000","last_unstale":"2024-09-21T03:33:42.921084+0000","last_undegraded":"2024-09-21T03:33:42.921084+0000","last_fullsized":"2024-09-21T03:33:42.921084+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_clean_scrub_stamp":"2024-09-21T03:33:34.220881+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:21:52.640936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.920870+0000","last_change":"2024-09-21T03:33:36.255323+0000","last_active":"2024-09-21T03:33:42.920870+0000","last_peered":"2024-09-21T03:33:42.920870+0000","last_clean":"2024-09-21T03:33:42.920870+0000","last_became_active":"2024-09-21T03:33:36.253192+0000","last_became_peered":"2024-09-21T03:33:36.253192+0000","last_unstale":"2024-09-21T03:33:42.920870+0000","last_undegraded":"2024-09-21T03:33:42.920870+0000","last_fullsized":"2024-09-21T03:33:42.920870+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:21:43.036932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.285988+0000","last_change":"2024-09-21T03:33:40.294983+0000","last_active":"2024-09-21T03:33:41.285988+0000","last_peered":"2024-09-21T03:33:41.285988+0000","last_clean":"2024-09-21T03:33:41.285988+0000","last_became_active":"2024-09-21T03:33:40.294528+0000","last_became_peered":"2024-09-21T03:33:40.294528+0000","last_unstale":"2024-09-21T03:33:41.285988+0000","last_undegraded":"2024-09-21T03:33:41.285988+0000","last_fullsized":"2024-09-21T03:33:41.285988+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_clean_scrub_stamp":"2024-09-21T03:33:39.262231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:39:42.878029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4,6],"acting":[2,7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.298804+0000","last_change":"2024-09-21T03:33:35.260455+0000","last_active":"2024-09-21T03:33:41.298804+0000","last_peered":"2024-09-21T03:33:41.298804+0000","last_clean":"2024-09-21T03:33:41.298804+0000","last_became_active":"2024-09-21T03:33:35.260074+0000","last_became_peered":"2024-09-21T03:33:35.260074+0000","last_unstale":"2024-09-21T03:33:41.298804+0000","last_undegraded":"2024-09-21T03:33:41.298804+0000","last_fullsized":"2024-09-21T03:33:41.298804+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_clean_scrub_stamp":"2024-09-21T03:33:34.220881+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:55:02.392036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.920774+0000","last_change":"2024-09-21T03:33:36.274462+0000","last_active":"2024-09-21T03:33:42.920774+0000","last_peered":"2024-09-21T03:33:42.920774+0000","last_clean":"2024-09-21T03:33:42.920774+0000","last_became_active":"2024-09-21T03:33:36.274080+0000","last_became_peered":"2024-09-21T03:33:36.274080+0000","last_unstale":"2024-09-21T03:33:42.920774+0000","last_undegraded":"2024-09-21T03:33:42.920774+0000","last_fullsized":"2024-09-21T03:33:42.920774+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:55:20.815479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.286750+0000","last_change":"2024-09-21T03:33:40.294613+0000","last_active":"2024-09-21T03:33:41.286750+0000","last_peered":"2024-09-21T03:33:41.286750+0000","last_clean":"2024-09-21T03:33:41.286750+0000","last_became_active":"2024-09-21T03:33:40.294138+0000","last_became_peered":"2024-09-21T03:33:40.294138+0000","last_unstale":"2024-09-21T03:33:41.286750+0000","last_undegraded":"2024-09-21T03:33:41.286750+0000","last_fullsized":"2024-09-21T03:33:41.286750+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_clean_scrub_stamp":"2024-09-21T03:33:39.262231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:10:29.758718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3,6],"acting":[2,5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.298963+0000","last_change":"2024-09-21T03:33:35.265118+0000","last_active":"2024-09-21T03:33:41.298963+0000","last_peered":"2024-09-21T03:33:41.298963+0000","last_clean":"2024-09-21T03:33:41.298963+0000","last_became_active":"2024-09-21T03:33:35.262875+0000","last_became_peered":"2024-09-21T03:33:35.262875+0000","last_unstale":"2024-09-21T03:33:41.298963+0000","last_undegraded":"2024-09-21T03:33:41.298963+0000","last_fullsized":"2024-09-21T03:33:41.298963+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_clean_scrub_stamp":"2024-09-21T03:33:34.220881+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:40:22.752363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.921523+0000","last_change":"2024-09-21T03:33:36.266776+0000","last_active":"2024-09-21T03:33:42.921523+0000","last_peered":"2024-09-21T03:33:42.921523+0000","last_clean":"2024-09-21T03:33:42.921523+0000","last_became_active":"2024-09-21T03:33:36.266449+0000","last_became_peered":"2024-09-21T03:33:36.266449+0000","last_unstale":"2024-09-21T03:33:42.921523+0000","last_undegraded":"2024-09-21T03:33:42.921523+0000","last_fullsized":"2024-09-21T03:33:42.921523+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:39:22.951079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.286642+0000","last_change":"2024-09-21T03:33:40.293654+0000","last_active":"2024-09-21T03:33:41.286642+0000","last_peered":"2024-09-21T03:33:41.286642+0000","last_clean":"2024-09-21T03:33:41.286642+0000","last_became_active":"2024-09-21T03:33:40.293236+0000","last_became_peered":"2024-09-21T03:33:40.293236+0000","last_unstale":"2024-09-21T03:33:41.286642+0000","last_undegraded":"2024-09-21T03:33:41.286642+0000","last_fullsized":"2024-09-21T03:33:41.286642+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_clean_scrub_stamp":"2024-09-21T03:33:39.262231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:20:09.214266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,5,7],"acting":[2,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.920921+0000","last_change":"2024-09-21T03:33:36.252854+0000","last_active":"2024-09-21T03:33:42.920921+0000","last_peered":"2024-09-21T03:33:42.920921+0000","last_clean":"2024-09-21T03:33:42.920921+0000","last_became_active":"2024-09-21T03:33:36.252384+0000","last_became_peered":"2024-09-21T03:33:36.252384+0000","last_unstale":"2024-09-21T03:33:42.920921+0000","last_undegraded":"2024-09-21T03:33:42.920921+0000","last_fullsized":"2024-09-21T03:33:42.920921+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:35:32.535895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.921139+0000","last_change":"2024-09-21T03:33:35.252643+0000","last_active":"2024-09-21T03:33:42.921139+0000","last_peered":"2024-09-21T03:33:42.921139+0000","last_clean":"2024-09-21T03:33:42.921139+0000","last_became_active":"2024-09-21T03:33:35.252335+0000","last_became_peered":"2024-09-21T03:33:35.252335+0000","last_unstale":"2024-09-21T03:33:42.921139+0000","last_undegraded":"2024-09-21T03:33:42.921139+0000","last_fullsized":"2024-09-21T03:33:42.921139+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_clean_scrub_stamp":"2024-09-21T03:33:34.220881+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:07:29.037511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.921479+0000","last_change":"2024-09-21T03:33:40.307347+0000","last_active":"2024-09-21T03:33:42.921479+0000","last_peered":"2024-09-21T03:33:42.921479+0000","last_clean":"2024-09-21T03:33:42.921479+0000","last_became_active":"2024-09-21T03:33:40.306899+0000","last_became_peered":"2024-09-21T03:33:40.306899+0000","last_unstale":"2024-09-21T03:33:42.921479+0000","last_undegraded":"2024-09-21T03:33:42.921479+0000","last_fullsized":"2024-09-21T03:33:42.921479+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_clean_scrub_stamp":"2024-09-21T03:33:39.262231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:45:45.403094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,5],"acting":[3,1,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.754733+0000","last_change":"2024-09-21T03:33:36.257562+0000","last_active":"2024-09-21T03:33:41.754733+0000","last_peered":"2024-09-21T03:33:41.754733+0000","last_clean":"2024-09-21T03:33:41.754733+0000","last_became_active":"2024-09-21T03:33:36.255065+0000","last_became_peered":"2024-09-21T03:33:36.255065+0000","last_unstale":"2024-09-21T03:33:41.754733+0000","last_undegraded":"2024-09-21T03:33:41.754733+0000","last_fullsized":"2024-09-21T03:33:41.754733+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:35:07.883056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.284335+0000","last_change":"2024-09-21T03:33:35.254523+0000","last_active":"2024-09-21T03:33:41.284335+0000","last_peered":"2024-09-21T03:33:41.284335+0000","last_clean":"2024-09-21T03:33:41.284335+0000","last_became_active":"2024-09-21T03:33:35.254185+0000","last_became_peered":"2024-09-21T03:33:35.254185+0000","last_unstale":"2024-09-21T03:33:41.284335+0000","last_undegraded":"2024-09-21T03:33:41.284335+0000","last_fullsized":"2024-09-21T03:33:41.284335+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_clean_scrub_stamp":"2024-09-21T03:33:34.220881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:06:37.946009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.298659+0000","last_change":"2024-09-21T03:33:40.304238+0000","last_active":"2024-09-21T03:33:41.298659+0000","last_peered":"2024-09-21T03:33:41.298659+0000","last_clean":"2024-09-21T03:33:41.298659+0000","last_became_active":"2024-09-21T03:33:40.303785+0000","last_became_peered":"2024-09-21T03:33:40.303785+0000","last_unstale":"2024-09-21T03:33:41.298659+0000","last_undegraded":"2024-09-21T03:33:41.298659+0000","last_fullsized":"2024-09-21T03:33:41.298659+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_clean_scrub_stamp":"2024-09-21T03:33:39.262231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:28:07.313548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6,1],"acting":[0,4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.283514+0000","last_change":"2024-09-21T03:33:36.255712+0000","last_active":"2024-09-21T03:33:41.283514+0000","last_peered":"2024-09-21T03:33:41.283514+0000","last_clean":"2024-09-21T03:33:41.283514+0000","last_became_active":"2024-09-21T03:33:36.255386+0000","last_became_peered":"2024-09-21T03:33:36.255386+0000","last_unstale":"2024-09-21T03:33:41.283514+0000","last_undegraded":"2024-09-21T03:33:41.283514+0000","last_fullsized":"2024-09-21T03:33:41.283514+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:49:00.717844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.284629+0000","last_change":"2024-09-21T03:33:35.266345+0000","last_active":"2024-09-21T03:33:41.284629+0000","last_peered":"2024-09-21T03:33:41.284629+0000","last_clean":"2024-09-21T03:33:41.284629+0000","last_became_active":"2024-09-21T03:33:35.266059+0000","last_became_peered":"2024-09-21T03:33:35.266059+0000","last_unstale":"2024-09-21T03:33:41.284629+0000","last_undegraded":"2024-09-21T03:33:41.284629+0000","last_fullsized":"2024-09-21T03:33:41.284629+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_clean_scrub_stamp":"2024-09-21T03:33:34.220881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:54:40.741741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.755284+0000","last_change":"2024-09-21T03:33:40.303927+0000","last_active":"2024-09-21T03:33:41.755284+0000","last_peered":"2024-09-21T03:33:41.755284+0000","last_clean":"2024-09-21T03:33:41.755284+0000","last_became_active":"2024-09-21T03:33:40.303511+0000","last_became_peered":"2024-09-21T03:33:40.303511+0000","last_unstale":"2024-09-21T03:33:41.755284+0000","last_undegraded":"2024-09-21T03:33:41.755284+0000","last_fullsized":"2024-09-21T03:33:41.755284+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_clean_scrub_stamp":"2024-09-21T03:33:39.262231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:27:26.725520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2,3],"acting":[7,4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.920506+0000","last_change":"2024-09-21T03:33:36.249470+0000","last_active":"2024-09-21T03:33:42.920506+0000","last_peered":"2024-09-21T03:33:42.920506+0000","last_clean":"2024-09-21T03:33:42.920506+0000","last_became_active":"2024-09-21T03:33:36.248726+0000","last_became_peered":"2024-09-21T03:33:36.248726+0000","last_unstale":"2024-09-21T03:33:42.920506+0000","last_undegraded":"2024-09-21T03:33:42.920506+0000","last_fullsized":"2024-09-21T03:33:42.920506+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:56:46.691011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.297935+0000","last_change":"2024-09-21T03:33:35.260543+0000","last_active":"2024-09-21T03:33:41.297935+0000","last_peered":"2024-09-21T03:33:41.297935+0000","last_clean":"2024-09-21T03:33:41.297935+0000","last_became_active":"2024-09-21T03:33:35.260041+0000","last_became_peered":"2024-09-21T03:33:35.260041+0000","last_unstale":"2024-09-21T03:33:41.297935+0000","last_undegraded":"2024-09-21T03:33:41.297935+0000","last_fullsized":"2024-09-21T03:33:41.297935+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_clean_scrub_stamp":"2024-09-21T03:33:34.220881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:59:06.265387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.286314+0000","last_change":"2024-09-21T03:33:40.309460+0000","last_active":"2024-09-21T03:33:41.286314+0000","last_peered":"2024-09-21T03:33:41.286314+0000","last_clean":"2024-09-21T03:33:41.286314+0000","last_became_active":"2024-09-21T03:33:40.306652+0000","last_became_peered":"2024-09-21T03:33:40.306652+0000","last_unstale":"2024-09-21T03:33:41.286314+0000","last_undegraded":"2024-09-21T03:33:41.286314+0000","last_fullsized":"2024-09-21T03:33:41.286314+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_clean_scrub_stamp":"2024-09-21T03:33:39.262231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:43:35.273085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,7],"acting":[2,1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.284755+0000","last_change":"2024-09-21T03:33:36.256025+0000","last_active":"2024-09-21T03:33:41.284755+0000","last_peered":"2024-09-21T03:33:41.284755+0000","last_clean":"2024-09-21T03:33:41.284755+0000","last_became_active":"2024-09-21T03:33:36.255648+0000","last_became_peered":"2024-09-21T03:33:36.255648+0000","last_unstale":"2024-09-21T03:33:41.284755+0000","last_undegraded":"2024-09-21T03:33:41.284755+0000","last_fullsized":"2024-09-21T03:33:41.284755+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:48:33.535308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.921585+0000","last_change":"2024-09-21T03:33:35.264889+0000","last_active":"2024-09-21T03:33:42.921585+0000","last_peered":"2024-09-21T03:33:42.921585+0000","last_clean":"2024-09-21T03:33:42.921585+0000","last_became_active":"2024-09-21T03:33:35.264160+0000","last_became_peered":"2024-09-21T03:33:35.264160+0000","last_unstale":"2024-09-21T03:33:42.921585+0000","last_undegraded":"2024-09-21T03:33:42.921585+0000","last_fullsized":"2024-09-21T03:33:42.921585+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_clean_scrub_stamp":"2024-09-21T03:33:34.220881+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:48:59.712004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.286424+0000","last_change":"2024-09-21T03:33:40.304067+0000","last_active":"2024-09-21T03:33:41.286424+0000","last_peered":"2024-09-21T03:33:41.286424+0000","last_clean":"2024-09-21T03:33:41.286424+0000","last_became_active":"2024-09-21T03:33:40.303613+0000","last_became_peered":"2024-09-21T03:33:40.303613+0000","last_unstale":"2024-09-21T03:33:41.286424+0000","last_undegraded":"2024-09-21T03:33:41.286424+0000","last_fullsized":"2024-09-21T03:33:41.286424+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_clean_scrub_stamp":"2024-09-21T03:33:39.262231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:51:21.940594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,3,1],"acting":[2,6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.285607+0000","last_change":"2024-09-21T03:33:36.258150+0000","last_active":"2024-09-21T03:33:41.285607+0000","last_peered":"2024-09-21T03:33:41.285607+0000","last_clean":"2024-09-21T03:33:41.285607+0000","last_became_active":"2024-09-21T03:33:36.257655+0000","last_became_peered":"2024-09-21T03:33:36.257655+0000","last_unstale":"2024-09-21T03:33:41.285607+0000","last_undegraded":"2024-09-21T03:33:41.285607+0000","last_fullsized":"2024-09-21T03:33:41.285607+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:10:32.046638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"22'1","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.285234+0000","last_change":"2024-09-21T03:33:35.264750+0000","last_active":"2024-09-21T03:33:41.285234+0000","last_peered":"2024-09-21T03:33:41.285234+0000","last_clean":"2024-09-21T03:33:41.285234+0000","last_became_active":"2024-09-21T03:33:35.264380+0000","last_became_peered":"2024-09-21T03:33:35.264380+0000","last_unstale":"2024-09-21T03:33:41.285234+0000","last_undegraded":"2024-09-21T03:33:41.285234+0000","last_fullsized":"2024-09-21T03:33:41.285234+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_clean_scrub_stamp":"2024-09-21T03:33:34.220881+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:12:06.105531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.284754+0000","last_change":"2024-09-21T03:33:40.303103+0000","last_active":"2024-09-21T03:33:41.284754+0000","last_peered":"2024-09-21T03:33:41.284754+0000","last_clean":"2024-09-21T03:33:41.284754+0000","last_became_active":"2024-09-21T03:33:40.301739+0000","last_became_peered":"2024-09-21T03:33:40.301739+0000","last_unstale":"2024-09-21T03:33:41.284754+0000","last_undegraded":"2024-09-21T03:33:41.284754+0000","last_fullsized":"2024-09-21T03:33:41.284754+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_clean_scrub_stamp":"2024-09-21T03:33:39.262231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:00:58.389938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6,3],"acting":[4,1,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.285678+0000","last_change":"2024-09-21T03:33:36.268661+0000","last_active":"2024-09-21T03:33:41.285678+0000","last_peered":"2024-09-21T03:33:41.285678+0000","last_clean":"2024-09-21T03:33:41.285678+0000","last_became_active":"2024-09-21T03:33:36.268317+0000","last_became_peered":"2024-09-21T03:33:36.268317+0000","last_unstale":"2024-09-21T03:33:41.285678+0000","last_undegraded":"2024-09-21T03:33:41.285678+0000","last_fullsized":"2024-09-21T03:33:41.285678+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:19:27.002524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.921773+0000","last_change":"2024-09-21T03:33:35.250205+0000","last_active":"2024-09-21T03:33:42.921773+0000","last_peered":"2024-09-21T03:33:42.921773+0000","last_clean":"2024-09-21T03:33:42.921773+0000","last_became_active":"2024-09-21T03:33:35.249883+0000","last_became_peered":"2024-09-21T03:33:35.249883+0000","last_unstale":"2024-09-21T03:33:42.921773+0000","last_undegraded":"2024-09-21T03:33:42.921773+0000","last_fullsized":"2024-09-21T03:33:42.921773+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_clean_scrub_stamp":"2024-09-21T03:33:34.220881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:48:54.078693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.283974+0000","last_change":"2024-09-21T03:33:40.298889+0000","last_active":"2024-09-21T03:33:41.283974+0000","last_peered":"2024-09-21T03:33:41.283974+0000","last_clean":"2024-09-21T03:33:41.283974+0000","last_became_active":"2024-09-21T03:33:40.298435+0000","last_became_peered":"2024-09-21T03:33:40.298435+0000","last_unstale":"2024-09-21T03:33:41.283974+0000","last_undegraded":"2024-09-21T03:33:41.283974+0000","last_fullsized":"2024-09-21T03:33:41.283974+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_clean_scrub_stamp":"2024-09-21T03:33:39.262231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:29:46.742857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,2,7],"acting":[6,0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.286003+0000","last_change":"2024-09-21T03:33:36.258037+0000","last_active":"2024-09-21T03:33:41.286003+0000","last_peered":"2024-09-21T03:33:41.286003+0000","last_clean":"2024-09-21T03:33:41.286003+0000","last_became_active":"2024-09-21T03:33:36.257687+0000","last_became_peered":"2024-09-21T03:33:36.257687+0000","last_unstale":"2024-09-21T03:33:41.286003+0000","last_undegraded":"2024-09-21T03:33:41.286003+0000","last_fullsized":"2024-09-21T03:33:41.286003+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:12:07.111371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.920392+0000","last_change":"2024-09-21T03:33:35.254341+0000","last_active":"2024-09-21T03:33:42.920392+0000","last_peered":"2024-09-21T03:33:42.920392+0000","last_clean":"2024-09-21T03:33:42.920392+0000","last_became_active":"2024-09-21T03:33:35.254031+0000","last_became_peered":"2024-09-21T03:33:35.254031+0000","last_unstale":"2024-09-21T03:33:42.920392+0000","last_undegraded":"2024-09-21T03:33:42.920392+0000","last_fullsized":"2024-09-21T03:33:42.920392+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_clean_scrub_stamp":"2024-09-21T03:33:34.220881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:39:48.189397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.284028+0000","last_change":"2024-09-21T03:33:40.295123+0000","last_active":"2024-09-21T03:33:41.284028+0000","last_peered":"2024-09-21T03:33:41.284028+0000","last_clean":"2024-09-21T03:33:41.284028+0000","last_became_active":"2024-09-21T03:33:40.294672+0000","last_became_peered":"2024-09-21T03:33:40.294672+0000","last_unstale":"2024-09-21T03:33:41.284028+0000","last_undegraded":"2024-09-21T03:33:41.284028+0000","last_fullsized":"2024-09-21T03:33:41.284028+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_clean_scrub_stamp":"2024-09-21T03:33:39.262231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:43:35.273085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5,7],"acting":[6,3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.298299+0000","last_change":"2024-09-21T03:33:36.277694+0000","last_active":"2024-09-21T03:33:41.298299+0000","last_peered":"2024-09-21T03:33:41.298299+0000","last_clean":"2024-09-21T03:33:41.298299+0000","last_became_active":"2024-09-21T03:33:36.277375+0000","last_became_peered":"2024-09-21T03:33:36.277375+0000","last_unstale":"2024-09-21T03:33:41.298299+0000","last_undegraded":"2024-09-21T03:33:41.298299+0000","last_fullsized":"2024-09-21T03:33:41.298299+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:39:38.842518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.921283+0000","last_change":"2024-09-21T03:33:35.250423+0000","last_active":"2024-09-21T03:33:42.921283+0000","last_peered":"2024-09-21T03:33:42.921283+0000","last_clean":"2024-09-21T03:33:42.921283+0000","last_became_active":"2024-09-21T03:33:35.248426+0000","last_became_peered":"2024-09-21T03:33:35.248426+0000","last_unstale":"2024-09-21T03:33:42.921283+0000","last_undegraded":"2024-09-21T03:33:42.921283+0000","last_fullsized":"2024-09-21T03:33:42.921283+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_clean_scrub_stamp":"2024-09-21T03:33:34.220881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:56:24.872394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.285593+0000","last_change":"2024-09-21T03:33:40.302877+0000","last_active":"2024-09-21T03:33:41.285593+0000","last_peered":"2024-09-21T03:33:41.285593+0000","last_clean":"2024-09-21T03:33:41.285593+0000","last_became_active":"2024-09-21T03:33:40.301015+0000","last_became_peered":"2024-09-21T03:33:40.301015+0000","last_unstale":"2024-09-21T03:33:41.285593+0000","last_undegraded":"2024-09-21T03:33:41.285593+0000","last_fullsized":"2024-09-21T03:33:41.285593+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_clean_scrub_stamp":"2024-09-21T03:33:39.262231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:20:09.214266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0,3],"acting":[6,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.919937+0000","last_change":"2024-09-21T03:33:36.255208+0000","last_active":"2024-09-21T03:33:42.919937+0000","last_peered":"2024-09-21T03:33:42.919937+0000","last_clean":"2024-09-21T03:33:42.919937+0000","last_became_active":"2024-09-21T03:33:36.254340+0000","last_became_peered":"2024-09-21T03:33:36.254340+0000","last_unstale":"2024-09-21T03:33:42.919937+0000","last_undegraded":"2024-09-21T03:33:42.919937+0000","last_fullsized":"2024-09-21T03:33:42.919937+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:59:34.251362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.754771+0000","last_change":"2024-09-21T03:33:35.255282+0000","last_active":"2024-09-21T03:33:41.754771+0000","last_peered":"2024-09-21T03:33:41.754771+0000","last_clean":"2024-09-21T03:33:41.754771+0000","last_became_active":"2024-09-21T03:33:35.254934+0000","last_became_peered":"2024-09-21T03:33:35.254934+0000","last_unstale":"2024-09-21T03:33:41.754771+0000","last_undegraded":"2024-09-21T03:33:41.754771+0000","last_fullsized":"2024-09-21T03:33:41.754771+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_clean_scrub_stamp":"2024-09-21T03:33:34.220881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:08:00.579605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.285752+0000","last_change":"2024-09-21T03:33:40.305801+0000","last_active":"2024-09-21T03:33:41.285752+0000","last_peered":"2024-09-21T03:33:41.285752+0000","last_clean":"2024-09-21T03:33:41.285752+0000","last_became_active":"2024-09-21T03:33:40.305316+0000","last_became_peered":"2024-09-21T03:33:40.305316+0000","last_unstale":"2024-09-21T03:33:41.285752+0000","last_undegraded":"2024-09-21T03:33:41.285752+0000","last_fullsized":"2024-09-21T03:33:41.285752+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_clean_scrub_stamp":"2024-09-21T03:33:39.262231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:10:29.758718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,7],"acting":[6,0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.284256+0000","last_change":"2024-09-21T03:33:36.264028+0000","last_active":"2024-09-21T03:33:41.284256+0000","last_peered":"2024-09-21T03:33:41.284256+0000","last_clean":"2024-09-21T03:33:41.284256+0000","last_became_active":"2024-09-21T03:33:36.263725+0000","last_became_peered":"2024-09-21T03:33:36.263725+0000","last_unstale":"2024-09-21T03:33:41.284256+0000","last_undegraded":"2024-09-21T03:33:41.284256+0000","last_fullsized":"2024-09-21T03:33:41.284256+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:48:55.084533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.920078+0000","last_change":"2024-09-21T03:33:35.260963+0000","last_active":"2024-09-21T03:33:42.920078+0000","last_peered":"2024-09-21T03:33:42.920078+0000","last_clean":"2024-09-21T03:33:42.920078+0000","last_became_active":"2024-09-21T03:33:35.257593+0000","last_became_peered":"2024-09-21T03:33:35.257593+0000","last_unstale":"2024-09-21T03:33:42.920078+0000","last_undegraded":"2024-09-21T03:33:42.920078+0000","last_fullsized":"2024-09-21T03:33:42.920078+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_clean_scrub_stamp":"2024-09-21T03:33:34.220881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:35:18.509895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.920658+0000","last_change":"2024-09-21T03:33:40.298927+0000","last_active":"2024-09-21T03:33:42.920658+0000","last_peered":"2024-09-21T03:33:42.920658+0000","last_clean":"2024-09-21T03:33:42.920658+0000","last_became_active":"2024-09-21T03:33:40.298272+0000","last_became_peered":"2024-09-21T03:33:40.298272+0000","last_unstale":"2024-09-21T03:33:42.920658+0000","last_undegraded":"2024-09-21T03:33:42.920658+0000","last_fullsized":"2024-09-21T03:33:42.920658+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_clean_scrub_stamp":"2024-09-21T03:33:39.262231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:44:52.684204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6,1],"acting":[5,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.284686+0000","last_change":"2024-09-21T03:33:36.263187+0000","last_active":"2024-09-21T03:33:41.284686+0000","last_peered":"2024-09-21T03:33:41.284686+0000","last_clean":"2024-09-21T03:33:41.284686+0000","last_became_active":"2024-09-21T03:33:36.262852+0000","last_became_peered":"2024-09-21T03:33:36.262852+0000","last_unstale":"2024-09-21T03:33:41.284686+0000","last_undegraded":"2024-09-21T03:33:41.284686+0000","last_fullsized":"2024-09-21T03:33:41.284686+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:28:30.977653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.921266+0000","last_change":"2024-09-21T03:33:35.249580+0000","last_active":"2024-09-21T03:33:42.921266+0000","last_peered":"2024-09-21T03:33:42.921266+0000","last_clean":"2024-09-21T03:33:42.921266+0000","last_became_active":"2024-09-21T03:33:35.248574+0000","last_became_peered":"2024-09-21T03:33:35.248574+0000","last_unstale":"2024-09-21T03:33:42.921266+0000","last_undegraded":"2024-09-21T03:33:42.921266+0000","last_fullsized":"2024-09-21T03:33:42.921266+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_clean_scrub_stamp":"2024-09-21T03:33:34.220881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:19:25.996685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":119,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.919871+0000","last_change":"2024-09-21T03:33:25.319776+0000","last_active":"2024-09-21T03:33:42.919871+0000","last_peered":"2024-09-21T03:33:42.919871+0000","last_clean":"2024-09-21T03:33:42.919871+0000","last_became_active":"2024-09-21T03:33:25.318399+0000","last_became_peered":"2024-09-21T03:33:25.318399+0000","last_unstale":"2024-09-21T03:33:42.919871+0000","last_undegraded":"2024-09-21T03:33:42.919871+0000","last_fullsized":"2024-09-21T03:33:42.919871+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:23.016672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:23.016672+0000","last_clean_scrub_stamp":"2024-09-21T03:33:23.016672+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:32:35.906114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.921733+0000","last_change":"2024-09-21T03:33:40.304303+0000","last_active":"2024-09-21T03:33:42.921733+0000","last_peered":"2024-09-21T03:33:42.921733+0000","last_clean":"2024-09-21T03:33:42.921733+0000","last_became_active":"2024-09-21T03:33:40.303880+0000","last_became_peered":"2024-09-21T03:33:40.303880+0000","last_unstale":"2024-09-21T03:33:42.921733+0000","last_undegraded":"2024-09-21T03:33:42.921733+0000","last_fullsized":"2024-09-21T03:33:42.921733+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_clean_scrub_stamp":"2024-09-21T03:33:39.262231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:22:59.076888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2,0],"acting":[3,1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.921586+0000","last_change":"2024-09-21T03:33:36.256735+0000","last_active":"2024-09-21T03:33:42.921586+0000","last_peered":"2024-09-21T03:33:42.921586+0000","last_clean":"2024-09-21T03:33:42.921586+0000","last_became_active":"2024-09-21T03:33:36.256296+0000","last_became_peered":"2024-09-21T03:33:36.256296+0000","last_unstale":"2024-09-21T03:33:42.921586+0000","last_undegraded":"2024-09-21T03:33:42.921586+0000","last_fullsized":"2024-09-21T03:33:42.921586+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:20:58.478604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.285704+0000","last_change":"2024-09-21T03:33:35.263469+0000","last_active":"2024-09-21T03:33:41.285704+0000","last_peered":"2024-09-21T03:33:41.285704+0000","last_clean":"2024-09-21T03:33:41.285704+0000","last_became_active":"2024-09-21T03:33:35.263119+0000","last_became_peered":"2024-09-21T03:33:35.263119+0000","last_unstale":"2024-09-21T03:33:41.285704+0000","last_undegraded":"2024-09-21T03:33:41.285704+0000","last_fullsized":"2024-09-21T03:33:41.285704+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_clean_scrub_stamp":"2024-09-21T03:33:34.220881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:54:40.741741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.286176+0000","last_change":"2024-09-21T03:33:40.304199+0000","last_active":"2024-09-21T03:33:41.286176+0000","last_peered":"2024-09-21T03:33:41.286176+0000","last_clean":"2024-09-21T03:33:41.286176+0000","last_became_active":"2024-09-21T03:33:40.303770+0000","last_became_peered":"2024-09-21T03:33:40.303770+0000","last_unstale":"2024-09-21T03:33:41.286176+0000","last_undegraded":"2024-09-21T03:33:41.286176+0000","last_fullsized":"2024-09-21T03:33:41.286176+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_clean_scrub_stamp":"2024-09-21T03:33:39.262231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:41:52.568236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7,4],"acting":[2,3,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.298815+0000","last_change":"2024-09-21T03:33:36.269892+0000","last_active":"2024-09-21T03:33:41.298815+0000","last_peered":"2024-09-21T03:33:41.298815+0000","last_clean":"2024-09-21T03:33:41.298815+0000","last_became_active":"2024-09-21T03:33:36.269521+0000","last_became_peered":"2024-09-21T03:33:36.269521+0000","last_unstale":"2024-09-21T03:33:41.298815+0000","last_undegraded":"2024-09-21T03:33:41.298815+0000","last_fullsized":"2024-09-21T03:33:41.298815+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:43:04.668863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.920621+0000","last_change":"2024-09-21T03:33:35.256681+0000","last_active":"2024-09-21T03:33:42.920621+0000","last_peered":"2024-09-21T03:33:42.920621+0000","last_clean":"2024-09-21T03:33:42.920621+0000","last_became_active":"2024-09-21T03:33:35.256325+0000","last_became_peered":"2024-09-21T03:33:35.256325+0000","last_unstale":"2024-09-21T03:33:42.920621+0000","last_undegraded":"2024-09-21T03:33:42.920621+0000","last_fullsized":"2024-09-21T03:33:42.920621+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_clean_scrub_stamp":"2024-09-21T03:33:34.220881+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:49:36.514638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.920396+0000","last_change":"2024-09-21T03:33:40.298797+0000","last_active":"2024-09-21T03:33:42.920396+0000","last_peered":"2024-09-21T03:33:42.920396+0000","last_clean":"2024-09-21T03:33:42.920396+0000","last_became_active":"2024-09-21T03:33:40.298254+0000","last_became_peered":"2024-09-21T03:33:40.298254+0000","last_unstale":"2024-09-21T03:33:42.920396+0000","last_undegraded":"2024-09-21T03:33:42.920396+0000","last_fullsized":"2024-09-21T03:33:42.920396+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_clean_scrub_stamp":"2024-09-21T03:33:39.262231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:33:49.804176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1,3],"acting":[5,4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.299252+0000","last_change":"2024-09-21T03:33:36.274252+0000","last_active":"2024-09-21T03:33:41.299252+0000","last_peered":"2024-09-21T03:33:41.299252+0000","last_clean":"2024-09-21T03:33:41.299252+0000","last_became_active":"2024-09-21T03:33:36.273890+0000","last_became_peered":"2024-09-21T03:33:36.273890+0000","last_unstale":"2024-09-21T03:33:41.299252+0000","last_undegraded":"2024-09-21T03:33:41.299252+0000","last_fullsized":"2024-09-21T03:33:41.299252+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:55:43.689119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.284896+0000","last_change":"2024-09-21T03:33:35.255320+0000","last_active":"2024-09-21T03:33:41.284896+0000","last_peered":"2024-09-21T03:33:41.284896+0000","last_clean":"2024-09-21T03:33:41.284896+0000","last_became_active":"2024-09-21T03:33:35.254912+0000","last_became_peered":"2024-09-21T03:33:35.254912+0000","last_unstale":"2024-09-21T03:33:41.284896+0000","last_undegraded":"2024-09-21T03:33:41.284896+0000","last_fullsized":"2024-09-21T03:33:41.284896+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_clean_scrub_stamp":"2024-09-21T03:33:34.220881+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:55:02.392036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.919867+0000","last_change":"2024-09-21T03:33:40.304053+0000","last_active":"2024-09-21T03:33:42.919867+0000","last_peered":"2024-09-21T03:33:42.919867+0000","last_clean":"2024-09-21T03:33:42.919867+0000","last_became_active":"2024-09-21T03:33:40.303606+0000","last_became_peered":"2024-09-21T03:33:40.303606+0000","last_unstale":"2024-09-21T03:33:42.919867+0000","last_undegraded":"2024-09-21T03:33:42.919867+0000","last_fullsized":"2024-09-21T03:33:42.919867+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_clean_scrub_stamp":"2024-09-21T03:33:39.262231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:43:29.231059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.286527+0000","last_change":"2024-09-21T03:33:36.270385+0000","last_active":"2024-09-21T03:33:41.286527+0000","last_peered":"2024-09-21T03:33:41.286527+0000","last_clean":"2024-09-21T03:33:41.286527+0000","last_became_active":"2024-09-21T03:33:36.267107+0000","last_became_peered":"2024-09-21T03:33:36.267107+0000","last_unstale":"2024-09-21T03:33:41.286527+0000","last_undegraded":"2024-09-21T03:33:41.286527+0000","last_fullsized":"2024-09-21T03:33:41.286527+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:48:55.084533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.299383+0000","last_change":"2024-09-21T03:33:35.263063+0000","last_active":"2024-09-21T03:33:41.299383+0000","last_peered":"2024-09-21T03:33:41.299383+0000","last_clean":"2024-09-21T03:33:41.299383+0000","last_became_active":"2024-09-21T03:33:35.262324+0000","last_became_peered":"2024-09-21T03:33:35.262324+0000","last_unstale":"2024-09-21T03:33:41.299383+0000","last_undegraded":"2024-09-21T03:33:41.299383+0000","last_fullsized":"2024-09-21T03:33:41.299383+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_clean_scrub_stamp":"2024-09-21T03:33:34.220881+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:35:18.509895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.920221+0000","last_change":"2024-09-21T03:33:40.298024+0000","last_active":"2024-09-21T03:33:42.920221+0000","last_peered":"2024-09-21T03:33:42.920221+0000","last_clean":"2024-09-21T03:33:42.920221+0000","last_became_active":"2024-09-21T03:33:40.297580+0000","last_became_peered":"2024-09-21T03:33:40.297580+0000","last_unstale":"2024-09-21T03:33:42.920221+0000","last_undegraded":"2024-09-21T03:33:42.920221+0000","last_fullsized":"2024-09-21T03:33:42.920221+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_clean_scrub_stamp":"2024-09-21T03:33:39.262231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:34:07.830134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,2],"acting":[1,5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.920389+0000","last_change":"2024-09-21T03:33:36.256567+0000","last_active":"2024-09-21T03:33:42.920389+0000","last_peered":"2024-09-21T03:33:42.920389+0000","last_clean":"2024-09-21T03:33:42.920389+0000","last_became_active":"2024-09-21T03:33:36.252598+0000","last_became_peered":"2024-09-21T03:33:36.252598+0000","last_unstale":"2024-09-21T03:33:42.920389+0000","last_undegraded":"2024-09-21T03:33:42.920389+0000","last_fullsized":"2024-09-21T03:33:42.920389+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:53:59.995385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":31,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.283961+0000","last_change":"2024-09-21T03:33:35.246577+0000","last_active":"2024-09-21T03:33:41.283961+0000","last_peered":"2024-09-21T03:33:41.283961+0000","last_clean":"2024-09-21T03:33:41.283961+0000","last_became_active":"2024-09-21T03:33:35.245954+0000","last_became_peered":"2024-09-21T03:33:35.245954+0000","last_unstale":"2024-09-21T03:33:41.283961+0000","last_undegraded":"2024-09-21T03:33:41.283961+0000","last_fullsized":"2024-09-21T03:33:41.283961+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_clean_scrub_stamp":"2024-09-21T03:33:34.220881+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:59:06.265387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.920916+0000","last_change":"2024-09-21T03:33:40.299966+0000","last_active":"2024-09-21T03:33:42.920916+0000","last_peered":"2024-09-21T03:33:42.920916+0000","last_clean":"2024-09-21T03:33:42.920916+0000","last_became_active":"2024-09-21T03:33:40.298455+0000","last_became_peered":"2024-09-21T03:33:40.298455+0000","last_unstale":"2024-09-21T03:33:42.920916+0000","last_undegraded":"2024-09-21T03:33:42.920916+0000","last_fullsized":"2024-09-21T03:33:42.920916+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_clean_scrub_stamp":"2024-09-21T03:33:39.262231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:59:07.141809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1,6],"acting":[5,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.919715+0000","last_change":"2024-09-21T03:33:36.261762+0000","last_active":"2024-09-21T03:33:42.919715+0000","last_peered":"2024-09-21T03:33:42.919715+0000","last_clean":"2024-09-21T03:33:42.919715+0000","last_became_active":"2024-09-21T03:33:36.261379+0000","last_became_peered":"2024-09-21T03:33:36.261379+0000","last_unstale":"2024-09-21T03:33:42.919715+0000","last_undegraded":"2024-09-21T03:33:42.919715+0000","last_fullsized":"2024-09-21T03:33:42.919715+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:35:07.883056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.298133+0000","last_change":"2024-09-21T03:33:35.264258+0000","last_active":"2024-09-21T03:33:41.298133+0000","last_peered":"2024-09-21T03:33:41.298133+0000","last_clean":"2024-09-21T03:33:41.298133+0000","last_became_active":"2024-09-21T03:33:35.262742+0000","last_became_peered":"2024-09-21T03:33:35.262742+0000","last_unstale":"2024-09-21T03:33:41.298133+0000","last_undegraded":"2024-09-21T03:33:41.298133+0000","last_fullsized":"2024-09-21T03:33:41.298133+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_clean_scrub_stamp":"2024-09-21T03:33:34.220881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:42:59.930617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.920862+0000","last_change":"2024-09-21T03:33:40.298402+0000","last_active":"2024-09-21T03:33:42.920862+0000","last_peered":"2024-09-21T03:33:42.920862+0000","last_clean":"2024-09-21T03:33:42.920862+0000","last_became_active":"2024-09-21T03:33:40.297795+0000","last_became_peered":"2024-09-21T03:33:40.297795+0000","last_unstale":"2024-09-21T03:33:42.920862+0000","last_undegraded":"2024-09-21T03:33:42.920862+0000","last_fullsized":"2024-09-21T03:33:42.920862+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_clean_scrub_stamp":"2024-09-21T03:33:39.262231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:18:24.844623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6,4],"acting":[5,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.920969+0000","last_change":"2024-09-21T03:33:36.269233+0000","last_active":"2024-09-21T03:33:42.920969+0000","last_peered":"2024-09-21T03:33:42.920969+0000","last_clean":"2024-09-21T03:33:42.920969+0000","last_became_active":"2024-09-21T03:33:36.268680+0000","last_became_peered":"2024-09-21T03:33:36.268680+0000","last_unstale":"2024-09-21T03:33:42.920969+0000","last_undegraded":"2024-09-21T03:33:42.920969+0000","last_fullsized":"2024-09-21T03:33:42.920969+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:55:43.689119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.921162+0000","last_change":"2024-09-21T03:33:35.251566+0000","last_active":"2024-09-21T03:33:42.921162+0000","last_peered":"2024-09-21T03:33:42.921162+0000","last_clean":"2024-09-21T03:33:42.921162+0000","last_became_active":"2024-09-21T03:33:35.251252+0000","last_became_peered":"2024-09-21T03:33:35.251252+0000","last_unstale":"2024-09-21T03:33:42.921162+0000","last_undegraded":"2024-09-21T03:33:42.921162+0000","last_fullsized":"2024-09-21T03:33:42.921162+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_clean_scrub_stamp":"2024-09-21T03:33:34.220881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:39:01.410384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.285366+0000","last_change":"2024-09-21T03:33:40.300353+0000","last_active":"2024-09-21T03:33:41.285366+0000","last_peered":"2024-09-21T03:33:41.285366+0000","last_clean":"2024-09-21T03:33:41.285366+0000","last_became_active":"2024-09-21T03:33:40.299481+0000","last_became_peered":"2024-09-21T03:33:40.299481+0000","last_unstale":"2024-09-21T03:33:41.285366+0000","last_undegraded":"2024-09-21T03:33:41.285366+0000","last_fullsized":"2024-09-21T03:33:41.285366+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_clean_scrub_stamp":"2024-09-21T03:33:39.262231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:51:21.940594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0,7],"acting":[6,4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.284147+0000","last_change":"2024-09-21T03:33:36.265957+0000","last_active":"2024-09-21T03:33:41.284147+0000","last_peered":"2024-09-21T03:33:41.284147+0000","last_clean":"2024-09-21T03:33:41.284147+0000","last_became_active":"2024-09-21T03:33:36.265578+0000","last_became_peered":"2024-09-21T03:33:36.265578+0000","last_unstale":"2024-09-21T03:33:41.284147+0000","last_undegraded":"2024-09-21T03:33:41.284147+0000","last_fullsized":"2024-09-21T03:33:41.284147+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:06:36.428086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.754862+0000","last_change":"2024-09-21T03:33:35.263298+0000","last_active":"2024-09-21T03:33:41.754862+0000","last_peered":"2024-09-21T03:33:41.754862+0000","last_clean":"2024-09-21T03:33:41.754862+0000","last_became_active":"2024-09-21T03:33:35.262833+0000","last_became_peered":"2024-09-21T03:33:35.262833+0000","last_unstale":"2024-09-21T03:33:41.754862+0000","last_undegraded":"2024-09-21T03:33:41.754862+0000","last_fullsized":"2024-09-21T03:33:41.754862+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_clean_scrub_stamp":"2024-09-21T03:33:34.220881+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:36:04.049350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.921613+0000","last_change":"2024-09-21T03:33:40.299399+0000","last_active":"2024-09-21T03:33:42.921613+0000","last_peered":"2024-09-21T03:33:42.921613+0000","last_clean":"2024-09-21T03:33:42.921613+0000","last_became_active":"2024-09-21T03:33:40.298855+0000","last_became_peered":"2024-09-21T03:33:40.298855+0000","last_unstale":"2024-09-21T03:33:42.921613+0000","last_undegraded":"2024-09-21T03:33:42.921613+0000","last_fullsized":"2024-09-21T03:33:42.921613+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_clean_scrub_stamp":"2024-09-21T03:33:39.262231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:38:17.611449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5,6],"acting":[3,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.298648+0000","last_change":"2024-09-21T03:33:36.268494+0000","last_active":"2024-09-21T03:33:41.298648+0000","last_peered":"2024-09-21T03:33:41.298648+0000","last_clean":"2024-09-21T03:33:41.298648+0000","last_became_active":"2024-09-21T03:33:36.268126+0000","last_became_peered":"2024-09-21T03:33:36.268126+0000","last_unstale":"2024-09-21T03:33:41.298648+0000","last_undegraded":"2024-09-21T03:33:41.298648+0000","last_fullsized":"2024-09-21T03:33:41.298648+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:00:54.354428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.14","version":"26'9","reported_seq":32,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:47.259231+0000","last_change":"2024-09-21T03:33:35.249689+0000","last_active":"2024-09-21T03:33:47.259231+0000","last_peered":"2024-09-21T03:33:47.259231+0000","last_clean":"2024-09-21T03:33:47.259231+0000","last_became_active":"2024-09-21T03:33:35.249355+0000","last_became_peered":"2024-09-21T03:33:35.249355+0000","last_unstale":"2024-09-21T03:33:47.259231+0000","last_undegraded":"2024-09-21T03:33:47.259231+0000","last_fullsized":"2024-09-21T03:33:47.259231+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_clean_scrub_stamp":"2024-09-21T03:33:34.220881+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:01:23.098006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.284651+0000","last_change":"2024-09-21T03:33:40.295609+0000","last_active":"2024-09-21T03:33:41.284651+0000","last_peered":"2024-09-21T03:33:41.284651+0000","last_clean":"2024-09-21T03:33:41.284651+0000","last_became_active":"2024-09-21T03:33:40.294963+0000","last_became_peered":"2024-09-21T03:33:40.294963+0000","last_unstale":"2024-09-21T03:33:41.284651+0000","last_undegraded":"2024-09-21T03:33:41.284651+0000","last_fullsized":"2024-09-21T03:33:41.284651+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_clean_scrub_stamp":"2024-09-21T03:33:39.262231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:55:47.724630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5,0],"acting":[4,3,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.921108+0000","last_change":"2024-09-21T03:33:36.260379+0000","last_active":"2024-09-21T03:33:42.921108+0000","last_peered":"2024-09-21T03:33:42.921108+0000","last_clean":"2024-09-21T03:33:42.921108+0000","last_became_active":"2024-09-21T03:33:36.260035+0000","last_became_peered":"2024-09-21T03:33:36.260035+0000","last_unstale":"2024-09-21T03:33:42.921108+0000","last_undegraded":"2024-09-21T03:33:42.921108+0000","last_fullsized":"2024-09-21T03:33:42.921108+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:28:03.278038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.755172+0000","last_change":"2024-09-21T03:33:35.249062+0000","last_active":"2024-09-21T03:33:41.755172+0000","last_peered":"2024-09-21T03:33:41.755172+0000","last_clean":"2024-09-21T03:33:41.755172+0000","last_became_active":"2024-09-21T03:33:35.248630+0000","last_became_peered":"2024-09-21T03:33:35.248630+0000","last_unstale":"2024-09-21T03:33:41.755172+0000","last_undegraded":"2024-09-21T03:33:41.755172+0000","last_fullsized":"2024-09-21T03:33:41.755172+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_clean_scrub_stamp":"2024-09-21T03:33:34.220881+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:49:09.367077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.286199+0000","last_change":"2024-09-21T03:33:40.305392+0000","last_active":"2024-09-21T03:33:41.286199+0000","last_peered":"2024-09-21T03:33:41.286199+0000","last_clean":"2024-09-21T03:33:41.286199+0000","last_became_active":"2024-09-21T03:33:40.304899+0000","last_became_peered":"2024-09-21T03:33:40.304899+0000","last_unstale":"2024-09-21T03:33:41.286199+0000","last_undegraded":"2024-09-21T03:33:41.286199+0000","last_fullsized":"2024-09-21T03:33:41.286199+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_clean_scrub_stamp":"2024-09-21T03:33:39.262231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:29:46.742857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1,7],"acting":[2,3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.920967+0000","last_change":"2024-09-21T03:33:36.256188+0000","last_active":"2024-09-21T03:33:42.920967+0000","last_peered":"2024-09-21T03:33:42.920967+0000","last_clean":"2024-09-21T03:33:42.920967+0000","last_became_active":"2024-09-21T03:33:36.254900+0000","last_became_peered":"2024-09-21T03:33:36.254900+0000","last_unstale":"2024-09-21T03:33:42.920967+0000","last_undegraded":"2024-09-21T03:33:42.920967+0000","last_fullsized":"2024-09-21T03:33:42.920967+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:20:54.467912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.299116+0000","last_change":"2024-09-21T03:33:35.259866+0000","last_active":"2024-09-21T03:33:41.299116+0000","last_peered":"2024-09-21T03:33:41.299116+0000","last_clean":"2024-09-21T03:33:41.299116+0000","last_became_active":"2024-09-21T03:33:35.259368+0000","last_became_peered":"2024-09-21T03:33:35.259368+0000","last_unstale":"2024-09-21T03:33:41.299116+0000","last_undegraded":"2024-09-21T03:33:41.299116+0000","last_fullsized":"2024-09-21T03:33:41.299116+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_clean_scrub_stamp":"2024-09-21T03:33:34.220881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:12:06.105531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.754662+0000","last_change":"2024-09-21T03:33:40.304856+0000","last_active":"2024-09-21T03:33:41.754662+0000","last_peered":"2024-09-21T03:33:41.754662+0000","last_clean":"2024-09-21T03:33:41.754662+0000","last_became_active":"2024-09-21T03:33:40.304421+0000","last_became_peered":"2024-09-21T03:33:40.304421+0000","last_unstale":"2024-09-21T03:33:41.754662+0000","last_undegraded":"2024-09-21T03:33:41.754662+0000","last_fullsized":"2024-09-21T03:33:41.754662+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_clean_scrub_stamp":"2024-09-21T03:33:39.262231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:10:13.916812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3,2],"acting":[7,1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.286504+0000","last_change":"2024-09-21T03:33:36.265331+0000","last_active":"2024-09-21T03:33:41.286504+0000","last_peered":"2024-09-21T03:33:41.286504+0000","last_clean":"2024-09-21T03:33:41.286504+0000","last_became_active":"2024-09-21T03:33:36.264928+0000","last_became_peered":"2024-09-21T03:33:36.264928+0000","last_unstale":"2024-09-21T03:33:41.286504+0000","last_undegraded":"2024-09-21T03:33:41.286504+0000","last_fullsized":"2024-09-21T03:33:41.286504+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:56:25.878234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.920745+0000","last_change":"2024-09-21T03:33:35.249377+0000","last_active":"2024-09-21T03:33:42.920745+0000","last_peered":"2024-09-21T03:33:42.920745+0000","last_clean":"2024-09-21T03:33:42.920745+0000","last_became_active":"2024-09-21T03:33:35.249027+0000","last_became_peered":"2024-09-21T03:33:35.249027+0000","last_unstale":"2024-09-21T03:33:42.920745+0000","last_undegraded":"2024-09-21T03:33:42.920745+0000","last_fullsized":"2024-09-21T03:33:42.920745+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_clean_scrub_stamp":"2024-09-21T03:33:34.220881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:48:36.003541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.284509+0000","last_change":"2024-09-21T03:33:40.294482+0000","last_active":"2024-09-21T03:33:41.284509+0000","last_peered":"2024-09-21T03:33:41.284509+0000","last_clean":"2024-09-21T03:33:41.284509+0000","last_became_active":"2024-09-21T03:33:40.293996+0000","last_became_peered":"2024-09-21T03:33:40.293996+0000","last_unstale":"2024-09-21T03:33:41.284509+0000","last_undegraded":"2024-09-21T03:33:41.284509+0000","last_fullsized":"2024-09-21T03:33:41.284509+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_clean_scrub_stamp":"2024-09-21T03:33:39.262231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:43:08.704373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,3,7],"acting":[4,0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.283820+0000","last_change":"2024-09-21T03:33:40.300111+0000","last_active":"2024-09-21T03:33:41.283820+0000","last_peered":"2024-09-21T03:33:41.283820+0000","last_clean":"2024-09-21T03:33:41.283820+0000","last_became_active":"2024-09-21T03:33:40.299645+0000","last_became_peered":"2024-09-21T03:33:40.299645+0000","last_unstale":"2024-09-21T03:33:41.283820+0000","last_undegraded":"2024-09-21T03:33:41.283820+0000","last_fullsized":"2024-09-21T03:33:41.283820+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_clean_scrub_stamp":"2024-09-21T03:33:39.262231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:10:29.758718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,5,3],"acting":[4,1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.284548+0000","last_change":"2024-09-21T03:33:36.248593+0000","last_active":"2024-09-21T03:33:41.284548+0000","last_peered":"2024-09-21T03:33:41.284548+0000","last_clean":"2024-09-21T03:33:41.284548+0000","last_became_active":"2024-09-21T03:33:36.248274+0000","last_became_peered":"2024-09-21T03:33:36.248274+0000","last_unstale":"2024-09-21T03:33:41.284548+0000","last_undegraded":"2024-09-21T03:33:41.284548+0000","last_fullsized":"2024-09-21T03:33:41.284548+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:36:57.787555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.920023+0000","last_change":"2024-09-21T03:33:35.261733+0000","last_active":"2024-09-21T03:33:42.920023+0000","last_peered":"2024-09-21T03:33:42.920023+0000","last_clean":"2024-09-21T03:33:42.920023+0000","last_became_active":"2024-09-21T03:33:35.261427+0000","last_became_peered":"2024-09-21T03:33:35.261427+0000","last_unstale":"2024-09-21T03:33:42.920023+0000","last_undegraded":"2024-09-21T03:33:42.920023+0000","last_fullsized":"2024-09-21T03:33:42.920023+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_clean_scrub_stamp":"2024-09-21T03:33:34.220881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:40:22.752363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.921462+0000","last_change":"2024-09-21T03:33:40.306313+0000","last_active":"2024-09-21T03:33:42.921462+0000","last_peered":"2024-09-21T03:33:42.921462+0000","last_clean":"2024-09-21T03:33:42.921462+0000","last_became_active":"2024-09-21T03:33:40.305650+0000","last_became_peered":"2024-09-21T03:33:40.305650+0000","last_unstale":"2024-09-21T03:33:42.921462+0000","last_undegraded":"2024-09-21T03:33:42.921462+0000","last_fullsized":"2024-09-21T03:33:42.921462+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_clean_scrub_stamp":"2024-09-21T03:33:39.262231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:45:58.941848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5,1],"acting":[3,2,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.920101+0000","last_change":"2024-09-21T03:33:36.255929+0000","last_active":"2024-09-21T03:33:42.920101+0000","last_peered":"2024-09-21T03:33:42.920101+0000","last_clean":"2024-09-21T03:33:42.920101+0000","last_became_active":"2024-09-21T03:33:36.255538+0000","last_became_peered":"2024-09-21T03:33:36.255538+0000","last_unstale":"2024-09-21T03:33:42.920101+0000","last_undegraded":"2024-09-21T03:33:42.920101+0000","last_fullsized":"2024-09-21T03:33:42.920101+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:33:10.543079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.284381+0000","last_change":"2024-09-21T03:33:35.264572+0000","last_active":"2024-09-21T03:33:41.284381+0000","last_peered":"2024-09-21T03:33:41.284381+0000","last_clean":"2024-09-21T03:33:41.284381+0000","last_became_active":"2024-09-21T03:33:35.264251+0000","last_became_peered":"2024-09-21T03:33:35.264251+0000","last_unstale":"2024-09-21T03:33:41.284381+0000","last_undegraded":"2024-09-21T03:33:41.284381+0000","last_fullsized":"2024-09-21T03:33:41.284381+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_clean_scrub_stamp":"2024-09-21T03:33:34.220881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:16:25.260763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.921171+0000","last_change":"2024-09-21T03:33:35.258225+0000","last_active":"2024-09-21T03:33:42.921171+0000","last_peered":"2024-09-21T03:33:42.921171+0000","last_clean":"2024-09-21T03:33:42.921171+0000","last_became_active":"2024-09-21T03:33:35.257903+0000","last_became_peered":"2024-09-21T03:33:35.257903+0000","last_unstale":"2024-09-21T03:33:42.921171+0000","last_undegraded":"2024-09-21T03:33:42.921171+0000","last_fullsized":"2024-09-21T03:33:42.921171+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_clean_scrub_stamp":"2024-09-21T03:33:34.220881+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:12:06.105531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.283808+0000","last_change":"2024-09-21T03:33:36.267650+0000","last_active":"2024-09-21T03:33:41.283808+0000","last_peered":"2024-09-21T03:33:41.283808+0000","last_clean":"2024-09-21T03:33:41.283808+0000","last_became_active":"2024-09-21T03:33:36.267315+0000","last_became_peered":"2024-09-21T03:33:36.267315+0000","last_unstale":"2024-09-21T03:33:41.283808+0000","last_undegraded":"2024-09-21T03:33:41.283808+0000","last_fullsized":"2024-09-21T03:33:41.283808+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:21:56.947372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.285062+0000","last_change":"2024-09-21T03:33:40.290894+0000","last_active":"2024-09-21T03:33:41.285062+0000","last_peered":"2024-09-21T03:33:41.285062+0000","last_clean":"2024-09-21T03:33:41.285062+0000","last_became_active":"2024-09-21T03:33:40.290424+0000","last_became_peered":"2024-09-21T03:33:40.290424+0000","last_unstale":"2024-09-21T03:33:41.285062+0000","last_undegraded":"2024-09-21T03:33:41.285062+0000","last_fullsized":"2024-09-21T03:33:41.285062+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_clean_scrub_stamp":"2024-09-21T03:33:39.262231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:56:50.726521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,7],"acting":[4,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.920464+0000","last_change":"2024-09-21T03:33:35.251276+0000","last_active":"2024-09-21T03:33:42.920464+0000","last_peered":"2024-09-21T03:33:42.920464+0000","last_clean":"2024-09-21T03:33:42.920464+0000","last_became_active":"2024-09-21T03:33:35.250758+0000","last_became_peered":"2024-09-21T03:33:35.250758+0000","last_unstale":"2024-09-21T03:33:42.920464+0000","last_undegraded":"2024-09-21T03:33:42.920464+0000","last_fullsized":"2024-09-21T03:33:42.920464+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_clean_scrub_stamp":"2024-09-21T03:33:34.220881+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:03:09.683999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.920393+0000","last_change":"2024-09-21T03:33:36.261907+0000","last_active":"2024-09-21T03:33:42.920393+0000","last_peered":"2024-09-21T03:33:42.920393+0000","last_clean":"2024-09-21T03:33:42.920393+0000","last_became_active":"2024-09-21T03:33:36.261483+0000","last_became_peered":"2024-09-21T03:33:36.261483+0000","last_unstale":"2024-09-21T03:33:42.920393+0000","last_undegraded":"2024-09-21T03:33:42.920393+0000","last_fullsized":"2024-09-21T03:33:42.920393+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:08:27.276982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.755140+0000","last_change":"2024-09-21T03:33:40.305034+0000","last_active":"2024-09-21T03:33:41.755140+0000","last_peered":"2024-09-21T03:33:41.755140+0000","last_clean":"2024-09-21T03:33:41.755140+0000","last_became_active":"2024-09-21T03:33:40.304555+0000","last_became_peered":"2024-09-21T03:33:40.304555+0000","last_unstale":"2024-09-21T03:33:41.755140+0000","last_undegraded":"2024-09-21T03:33:41.755140+0000","last_fullsized":"2024-09-21T03:33:41.755140+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_clean_scrub_stamp":"2024-09-21T03:33:39.262231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:09:41.615406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1,0],"acting":[7,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.285867+0000","last_change":"2024-09-21T03:33:36.258191+0000","last_active":"2024-09-21T03:33:41.285867+0000","last_peered":"2024-09-21T03:33:41.285867+0000","last_clean":"2024-09-21T03:33:41.285867+0000","last_became_active":"2024-09-21T03:33:36.257866+0000","last_became_peered":"2024-09-21T03:33:36.257866+0000","last_unstale":"2024-09-21T03:33:41.285867+0000","last_undegraded":"2024-09-21T03:33:41.285867+0000","last_fullsized":"2024-09-21T03:33:41.285867+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:07:30.043350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.754532+0000","last_change":"2024-09-21T03:33:36.248020+0000","last_active":"2024-09-21T03:33:41.754532+0000","last_peered":"2024-09-21T03:33:41.754532+0000","last_clean":"2024-09-21T03:33:41.754532+0000","last_became_active":"2024-09-21T03:33:36.247696+0000","last_became_peered":"2024-09-21T03:33:36.247696+0000","last_unstale":"2024-09-21T03:33:41.754532+0000","last_undegraded":"2024-09-21T03:33:41.754532+0000","last_fullsized":"2024-09-21T03:33:41.754532+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:11:14.232403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.920927+0000","last_change":"2024-09-21T03:33:36.264435+0000","last_active":"2024-09-21T03:33:42.920927+0000","last_peered":"2024-09-21T03:33:42.920927+0000","last_clean":"2024-09-21T03:33:42.920927+0000","last_became_active":"2024-09-21T03:33:36.263834+0000","last_became_peered":"2024-09-21T03:33:36.263834+0000","last_unstale":"2024-09-21T03:33:42.920927+0000","last_undegraded":"2024-09-21T03:33:42.920927+0000","last_fullsized":"2024-09-21T03:33:42.920927+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:00:54.354428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.920341+0000","last_change":"2024-09-21T03:33:36.256675+0000","last_active":"2024-09-21T03:33:42.920341+0000","last_peered":"2024-09-21T03:33:42.920341+0000","last_clean":"2024-09-21T03:33:42.920341+0000","last_became_active":"2024-09-21T03:33:36.256184+0000","last_became_peered":"2024-09-21T03:33:36.256184+0000","last_unstale":"2024-09-21T03:33:42.920341+0000","last_undegraded":"2024-09-21T03:33:42.920341+0000","last_fullsized":"2024-09-21T03:33:42.920341+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:10:44.754143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.285553+0000","last_change":"2024-09-21T03:33:36.271076+0000","last_active":"2024-09-21T03:33:41.285553+0000","last_peered":"2024-09-21T03:33:41.285553+0000","last_clean":"2024-09-21T03:33:41.285553+0000","last_became_active":"2024-09-21T03:33:36.270707+0000","last_became_peered":"2024-09-21T03:33:36.270707+0000","last_unstale":"2024-09-21T03:33:41.285553+0000","last_undegraded":"2024-09-21T03:33:41.285553+0000","last_fullsized":"2024-09-21T03:33:41.285553+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:07:30.043350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.285361+0000","last_change":"2024-09-21T03:33:36.261405+0000","last_active":"2024-09-21T03:33:41.285361+0000","last_peered":"2024-09-21T03:33:41.285361+0000","last_clean":"2024-09-21T03:33:41.285361+0000","last_became_active":"2024-09-21T03:33:36.261048+0000","last_became_peered":"2024-09-21T03:33:36.261048+0000","last_unstale":"2024-09-21T03:33:41.285361+0000","last_undegraded":"2024-09-21T03:33:41.285361+0000","last_fullsized":"2024-09-21T03:33:41.285361+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:56:25.878234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.286335+0000","last_change":"2024-09-21T03:33:36.272514+0000","last_active":"2024-09-21T03:33:41.286335+0000","last_peered":"2024-09-21T03:33:41.286335+0000","last_clean":"2024-09-21T03:33:41.286335+0000","last_became_active":"2024-09-21T03:33:36.272186+0000","last_became_peered":"2024-09-21T03:33:36.272186+0000","last_unstale":"2024-09-21T03:33:41.286335+0000","last_undegraded":"2024-09-21T03:33:41.286335+0000","last_fullsized":"2024-09-21T03:33:41.286335+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:39:02.416224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.920780+0000","last_change":"2024-09-21T03:33:36.255650+0000","last_active":"2024-09-21T03:33:42.920780+0000","last_peered":"2024-09-21T03:33:42.920780+0000","last_clean":"2024-09-21T03:33:42.920780+0000","last_became_active":"2024-09-21T03:33:36.253524+0000","last_became_peered":"2024-09-21T03:33:36.253524+0000","last_unstale":"2024-09-21T03:33:42.920780+0000","last_undegraded":"2024-09-21T03:33:42.920780+0000","last_fullsized":"2024-09-21T03:33:42.920780+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:51:37.424690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.919971+0000","last_change":"2024-09-21T03:33:36.262263+0000","last_active":"2024-09-21T03:33:42.919971+0000","last_peered":"2024-09-21T03:33:42.919971+0000","last_clean":"2024-09-21T03:33:42.919971+0000","last_became_active":"2024-09-21T03:33:36.261962+0000","last_became_peered":"2024-09-21T03:33:36.261962+0000","last_unstale":"2024-09-21T03:33:42.919971+0000","last_undegraded":"2024-09-21T03:33:42.919971+0000","last_fullsized":"2024-09-21T03:33:42.919971+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:10:25.723208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.920014+0000","last_change":"2024-09-21T03:33:36.261266+0000","last_active":"2024-09-21T03:33:42.920014+0000","last_peered":"2024-09-21T03:33:42.920014+0000","last_clean":"2024-09-21T03:33:42.920014+0000","last_became_active":"2024-09-21T03:33:36.260938+0000","last_became_peered":"2024-09-21T03:33:36.260938+0000","last_unstale":"2024-09-21T03:33:42.920014+0000","last_undegraded":"2024-09-21T03:33:42.920014+0000","last_fullsized":"2024-09-21T03:33:42.920014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:54:42.460406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.920279+0000","last_change":"2024-09-21T03:33:36.268419+0000","last_active":"2024-09-21T03:33:42.920279+0000","last_peered":"2024-09-21T03:33:42.920279+0000","last_clean":"2024-09-21T03:33:42.920279+0000","last_became_active":"2024-09-21T03:33:36.268118+0000","last_became_peered":"2024-09-21T03:33:36.268118+0000","last_unstale":"2024-09-21T03:33:42.920279+0000","last_undegraded":"2024-09-21T03:33:42.920279+0000","last_fullsized":"2024-09-21T03:33:42.920279+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:15:19.693603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.283681+0000","last_change":"2024-09-21T03:33:36.265448+0000","last_active":"2024-09-21T03:33:41.283681+0000","last_peered":"2024-09-21T03:33:41.283681+0000","last_clean":"2024-09-21T03:33:41.283681+0000","last_became_active":"2024-09-21T03:33:36.263265+0000","last_became_peered":"2024-09-21T03:33:36.263265+0000","last_unstale":"2024-09-21T03:33:41.283681+0000","last_undegraded":"2024-09-21T03:33:41.283681+0000","last_fullsized":"2024-09-21T03:33:41.283681+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:06:36.428086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.298767+0000","last_change":"2024-09-21T03:33:36.274784+0000","last_active":"2024-09-21T03:33:41.298767+0000","last_peered":"2024-09-21T03:33:41.298767+0000","last_clean":"2024-09-21T03:33:41.298767+0000","last_became_active":"2024-09-21T03:33:36.273608+0000","last_became_peered":"2024-09-21T03:33:36.273608+0000","last_unstale":"2024-09-21T03:33:41.298767+0000","last_undegraded":"2024-09-21T03:33:41.298767+0000","last_fullsized":"2024-09-21T03:33:41.298767+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:43:31.237575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.921685+0000","last_change":"2024-09-21T03:33:36.253771+0000","last_active":"2024-09-21T03:33:42.921685+0000","last_peered":"2024-09-21T03:33:42.921685+0000","last_clean":"2024-09-21T03:33:42.921685+0000","last_became_active":"2024-09-21T03:33:36.253068+0000","last_became_peered":"2024-09-21T03:33:36.253068+0000","last_unstale":"2024-09-21T03:33:42.921685+0000","last_undegraded":"2024-09-21T03:33:42.921685+0000","last_fullsized":"2024-09-21T03:33:42.921685+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:33:50.625863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.755254+0000","last_change":"2024-09-21T03:33:36.255996+0000","last_active":"2024-09-21T03:33:41.755254+0000","last_peered":"2024-09-21T03:33:41.755254+0000","last_clean":"2024-09-21T03:33:41.755254+0000","last_became_active":"2024-09-21T03:33:36.255696+0000","last_became_peered":"2024-09-21T03:33:36.255696+0000","last_unstale":"2024-09-21T03:33:41.755254+0000","last_undegraded":"2024-09-21T03:33:41.755254+0000","last_fullsized":"2024-09-21T03:33:41.755254+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:15:19.693603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.921552+0000","last_change":"2024-09-21T03:33:36.271593+0000","last_active":"2024-09-21T03:33:42.921552+0000","last_peered":"2024-09-21T03:33:42.921552+0000","last_clean":"2024-09-21T03:33:42.921552+0000","last_became_active":"2024-09-21T03:33:36.271244+0000","last_became_peered":"2024-09-21T03:33:36.271244+0000","last_unstale":"2024-09-21T03:33:42.921552+0000","last_undegraded":"2024-09-21T03:33:42.921552+0000","last_fullsized":"2024-09-21T03:33:42.921552+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:14:54.862806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.919854+0000","last_change":"2024-09-21T03:33:36.252117+0000","last_active":"2024-09-21T03:33:42.919854+0000","last_peered":"2024-09-21T03:33:42.919854+0000","last_clean":"2024-09-21T03:33:42.919854+0000","last_became_active":"2024-09-21T03:33:36.251757+0000","last_became_peered":"2024-09-21T03:33:36.251757+0000","last_unstale":"2024-09-21T03:33:42.919854+0000","last_undegraded":"2024-09-21T03:33:42.919854+0000","last_fullsized":"2024-09-21T03:33:42.919854+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:59:02.176819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.919914+0000","last_change":"2024-09-21T03:33:36.257727+0000","last_active":"2024-09-21T03:33:42.919914+0000","last_peered":"2024-09-21T03:33:42.919914+0000","last_clean":"2024-09-21T03:33:42.919914+0000","last_became_active":"2024-09-21T03:33:36.257423+0000","last_became_peered":"2024-09-21T03:33:36.257423+0000","last_unstale":"2024-09-21T03:33:42.919914+0000","last_undegraded":"2024-09-21T03:33:42.919914+0000","last_fullsized":"2024-09-21T03:33:42.919914+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:11:43.225156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.284433+0000","last_change":"2024-09-21T03:33:36.257080+0000","last_active":"2024-09-21T03:33:41.284433+0000","last_peered":"2024-09-21T03:33:41.284433+0000","last_clean":"2024-09-21T03:33:41.284433+0000","last_became_active":"2024-09-21T03:33:36.256763+0000","last_became_peered":"2024-09-21T03:33:36.256763+0000","last_unstale":"2024-09-21T03:33:41.284433+0000","last_undegraded":"2024-09-21T03:33:41.284433+0000","last_fullsized":"2024-09-21T03:33:41.284433+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:36:57.787555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.285409+0000","last_change":"2024-09-21T03:33:36.265166+0000","last_active":"2024-09-21T03:33:41.285409+0000","last_peered":"2024-09-21T03:33:41.285409+0000","last_clean":"2024-09-21T03:33:41.285409+0000","last_became_active":"2024-09-21T03:33:36.264841+0000","last_became_peered":"2024-09-21T03:33:36.264841+0000","last_unstale":"2024-09-21T03:33:41.285409+0000","last_undegraded":"2024-09-21T03:33:41.285409+0000","last_fullsized":"2024-09-21T03:33:41.285409+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:49:00.717844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.298659+0000","last_change":"2024-09-21T03:33:36.274919+0000","last_active":"2024-09-21T03:33:41.298659+0000","last_peered":"2024-09-21T03:33:41.298659+0000","last_clean":"2024-09-21T03:33:41.298659+0000","last_became_active":"2024-09-21T03:33:36.274111+0000","last_became_peered":"2024-09-21T03:33:36.274111+0000","last_unstale":"2024-09-21T03:33:41.298659+0000","last_undegraded":"2024-09-21T03:33:41.298659+0000","last_fullsized":"2024-09-21T03:33:41.298659+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:29:42.707347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.298611+0000","last_change":"2024-09-21T03:33:36.272954+0000","last_active":"2024-09-21T03:33:41.298611+0000","last_peered":"2024-09-21T03:33:41.298611+0000","last_clean":"2024-09-21T03:33:41.298611+0000","last_became_active":"2024-09-21T03:33:36.272551+0000","last_became_peered":"2024-09-21T03:33:36.272551+0000","last_unstale":"2024-09-21T03:33:41.298611+0000","last_undegraded":"2024-09-21T03:33:41.298611+0000","last_fullsized":"2024-09-21T03:33:41.298611+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:41:48.532726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.298922+0000","last_change":"2024-09-21T03:33:36.276841+0000","last_active":"2024-09-21T03:33:41.298922+0000","last_peered":"2024-09-21T03:33:41.298922+0000","last_clean":"2024-09-21T03:33:41.298922+0000","last_became_active":"2024-09-21T03:33:36.276522+0000","last_became_peered":"2024-09-21T03:33:36.276522+0000","last_unstale":"2024-09-21T03:33:41.298922+0000","last_undegraded":"2024-09-21T03:33:41.298922+0000","last_fullsized":"2024-09-21T03:33:41.298922+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:20:05.178755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.920682+0000","last_change":"2024-09-21T03:33:36.251561+0000","last_active":"2024-09-21T03:33:42.920682+0000","last_peered":"2024-09-21T03:33:42.920682+0000","last_clean":"2024-09-21T03:33:42.920682+0000","last_became_active":"2024-09-21T03:33:36.251161+0000","last_became_peered":"2024-09-21T03:33:36.251161+0000","last_unstale":"2024-09-21T03:33:42.920682+0000","last_undegraded":"2024-09-21T03:33:42.920682+0000","last_fullsized":"2024-09-21T03:33:42.920682+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:22:37.114294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.919931+0000","last_change":"2024-09-21T03:33:36.258249+0000","last_active":"2024-09-21T03:33:42.919931+0000","last_peered":"2024-09-21T03:33:42.919931+0000","last_clean":"2024-09-21T03:33:42.919931+0000","last_became_active":"2024-09-21T03:33:36.257558+0000","last_became_peered":"2024-09-21T03:33:36.257558+0000","last_unstale":"2024-09-21T03:33:42.919931+0000","last_undegraded":"2024-09-21T03:33:42.919931+0000","last_fullsized":"2024-09-21T03:33:42.919931+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:14:09.890548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.298137+0000","last_change":"2024-09-21T03:33:36.265238+0000","last_active":"2024-09-21T03:33:41.298137+0000","last_peered":"2024-09-21T03:33:41.298137+0000","last_clean":"2024-09-21T03:33:41.298137+0000","last_became_active":"2024-09-21T03:33:36.264904+0000","last_became_peered":"2024-09-21T03:33:36.264904+0000","last_unstale":"2024-09-21T03:33:41.298137+0000","last_undegraded":"2024-09-21T03:33:41.298137+0000","last_fullsized":"2024-09-21T03:33:41.298137+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:10:32.046638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.286379+0000","last_change":"2024-09-21T03:33:36.270243+0000","last_active":"2024-09-21T03:33:41.286379+0000","last_peered":"2024-09-21T03:33:41.286379+0000","last_clean":"2024-09-21T03:33:41.286379+0000","last_became_active":"2024-09-21T03:33:36.267458+0000","last_became_peered":"2024-09-21T03:33:36.267458+0000","last_unstale":"2024-09-21T03:33:41.286379+0000","last_undegraded":"2024-09-21T03:33:41.286379+0000","last_fullsized":"2024-09-21T03:33:41.286379+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:40:23.758202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.298523+0000","last_change":"2024-09-21T03:33:36.273294+0000","last_active":"2024-09-21T03:33:41.298523+0000","last_peered":"2024-09-21T03:33:41.298523+0000","last_clean":"2024-09-21T03:33:41.298523+0000","last_became_active":"2024-09-21T03:33:36.272841+0000","last_became_peered":"2024-09-21T03:33:36.272841+0000","last_unstale":"2024-09-21T03:33:41.298523+0000","last_undegraded":"2024-09-21T03:33:41.298523+0000","last_fullsized":"2024-09-21T03:33:41.298523+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:43:32.821740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.285970+0000","last_change":"2024-09-21T03:33:36.267665+0000","last_active":"2024-09-21T03:33:41.285970+0000","last_peered":"2024-09-21T03:33:41.285970+0000","last_clean":"2024-09-21T03:33:41.285970+0000","last_became_active":"2024-09-21T03:33:36.267339+0000","last_became_peered":"2024-09-21T03:33:36.267339+0000","last_unstale":"2024-09-21T03:33:41.285970+0000","last_undegraded":"2024-09-21T03:33:41.285970+0000","last_fullsized":"2024-09-21T03:33:41.285970+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:35:19.515735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.920418+0000","last_change":"2024-09-21T03:33:36.260290+0000","last_active":"2024-09-21T03:33:42.920418+0000","last_peered":"2024-09-21T03:33:42.920418+0000","last_clean":"2024-09-21T03:33:42.920418+0000","last_became_active":"2024-09-21T03:33:36.259984+0000","last_became_peered":"2024-09-21T03:33:36.259984+0000","last_unstale":"2024-09-21T03:33:42.920418+0000","last_undegraded":"2024-09-21T03:33:42.920418+0000","last_fullsized":"2024-09-21T03:33:42.920418+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:15:12.112267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.920700+0000","last_change":"2024-09-21T03:33:36.256894+0000","last_active":"2024-09-21T03:33:42.920700+0000","last_peered":"2024-09-21T03:33:42.920700+0000","last_clean":"2024-09-21T03:33:42.920700+0000","last_became_active":"2024-09-21T03:33:36.256598+0000","last_became_peered":"2024-09-21T03:33:36.256598+0000","last_unstale":"2024-09-21T03:33:42.920700+0000","last_undegraded":"2024-09-21T03:33:42.920700+0000","last_fullsized":"2024-09-21T03:33:42.920700+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:59:34.604906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.298773+0000","last_change":"2024-09-21T03:33:36.273537+0000","last_active":"2024-09-21T03:33:41.298773+0000","last_peered":"2024-09-21T03:33:41.298773+0000","last_clean":"2024-09-21T03:33:41.298773+0000","last_became_active":"2024-09-21T03:33:36.272386+0000","last_became_peered":"2024-09-21T03:33:36.272386+0000","last_unstale":"2024-09-21T03:33:41.298773+0000","last_undegraded":"2024-09-21T03:33:41.298773+0000","last_fullsized":"2024-09-21T03:33:41.298773+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:10:25.723208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":1,"up_from":16,"seq":68719476747,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32036,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339804,"statfs":{"total":96636764160,"available":96603959296,"internally_reserved":0,"allocated":868352,"data_stored":697804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509451,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31460,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340380,"statfs":{"total":96636764160,"available":96604549120,"internally_reserved":0,"allocated":278528,"data_stored":107490,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509450,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31456,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340384,"statfs":{"total":96636764160,"available":96604553216,"internally_reserved":0,"allocated":274432,"data_stored":107948,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509451,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":282624,"data_stored":111208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509450,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31464,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340376,"statfs":{"total":96636764160,"available":96604545024,"internally_reserved":0,"allocated":278528,"data_stored":107422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542154,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31456,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340384,"statfs":{"total":96636764160,"available":96604553216,"internally_reserved":0,"allocated":270336,"data_stored":107366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574858,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32040,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339800,"statfs":{"total":96636764160,"available":96603955200,"internally_reserved":0,"allocated":872448,"data_stored":697826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542155,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35628,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336212,"statfs":{"total":96636764160,"available":96600281088,"internally_reserved":0,"allocated":290816,"data_stored":111880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":4514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T03:34:02.360 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T03:34:02.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:02.511+0000 7f8c6257b640 1 -- 172.21.15.38:0/3045204144 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8c44003a50 msgr2=0x7f8c44003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:02.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:02.511+0000 7f8c6257b640 1 --2- 172.21.15.38:0/3045204144 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8c44003a50 0x7f8c44003ec0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c0016c0 tx=0x7f8c44005830 comp rx=0 tx=0).stop 2024-09-21T03:34:02.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:02.511+0000 7f8c6257b640 1 -- 172.21.15.38:0/3045204144 shutdown_connections 2024-09-21T03:34:02.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:02.511+0000 7f8c6257b640 1 --2- 172.21.15.38:0/3045204144 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8c44003a50 0x7f8c44003ec0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:02.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:02.511+0000 7f8c6257b640 1 -- 172.21.15.38:0/3045204144 >> 172.21.15.38:0/3045204144 conn(0x7f8c5c0d4440 msgr2=0x7f8c5c0d4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:02.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:02.511+0000 7f8c6257b640 1 -- 172.21.15.38:0/3045204144 shutdown_connections 2024-09-21T03:34:02.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:02.511+0000 7f8c6257b640 1 -- 172.21.15.38:0/3045204144 wait complete. 2024-09-21T03:34:02.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:02.513+0000 7f8c6257b640 1 Processor -- start 2024-09-21T03:34:02.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:02.513+0000 7f8c6257b640 1 -- start start 2024-09-21T03:34:02.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:02.514+0000 7f8c6257b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8c44003a50 0x7f8c5c157750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:02.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:02.514+0000 7f8c6257b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8c5c157d70 0x7f8c5c1e9610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:02.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:02.514+0000 7f8c6257b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8c5c1e9b50 0x7f8c5c1f5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:02.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:02.514+0000 7f8c6257b640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f8c5c162e40 con 0x7f8c5c1e9b50 2024-09-21T03:34:02.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:02.514+0000 7f8c6257b640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f8c5c162cc0 con 0x7f8c5c157d70 2024-09-21T03:34:02.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:02.514+0000 7f8c6257b640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f8c5c0d2a20 con 0x7f8c44003a50 2024-09-21T03:34:02.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:02.514+0000 7f8c60b18640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8c5c1e9b50 0x7f8c5c1f5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:02.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:02.514+0000 7f8c60b18640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8c5c1e9b50 0x7f8c5c1f5870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42868/0 (socket says 172.21.15.38:42868) 2024-09-21T03:34:02.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:02.514+0000 7f8c60b18640 1 -- 172.21.15.38:0/2287310213 learned_addr learned my addr 172.21.15.38:0/2287310213 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:34:02.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:02.514+0000 7f8c5bfff640 1 --2- 172.21.15.38:0/2287310213 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8c44003a50 0x7f8c5c157750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:02.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:02.514+0000 7f8c5b7fe640 1 --2- 172.21.15.38:0/2287310213 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8c5c157d70 0x7f8c5c1e9610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:02.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:02.514+0000 7f8c60b18640 1 -- 172.21.15.38:0/2287310213 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8c44003a50 msgr2=0x7f8c5c157750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:02.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:02.514+0000 7f8c60b18640 1 --2- 172.21.15.38:0/2287310213 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8c44003a50 0x7f8c5c157750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:02.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:02.514+0000 7f8c60b18640 1 -- 172.21.15.38:0/2287310213 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8c5c157d70 msgr2=0x7f8c5c1e9610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:02.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:02.514+0000 7f8c60b18640 1 --2- 172.21.15.38:0/2287310213 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8c5c157d70 0x7f8c5c1e9610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:02.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:02.514+0000 7f8c60b18640 1 -- 172.21.15.38:0/2287310213 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c5c158570 con 0x7f8c5c1e9b50 2024-09-21T03:34:02.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:02.515+0000 7f8c60b18640 1 --2- 172.21.15.38:0/2287310213 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8c5c1e9b50 0x7f8c5c1f5870 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f8c54005fd0 tx=0x7f8c54010f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:02.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:02.515+0000 7f8c58ff9640 1 -- 172.21.15.38:0/2287310213 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c5400eea0 con 0x7f8c5c1e9b50 2024-09-21T03:34:02.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:02.515+0000 7f8c58ff9640 1 -- 172.21.15.38:0/2287310213 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8c54015070 con 0x7f8c5c1e9b50 2024-09-21T03:34:02.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:02.515+0000 7f8c58ff9640 1 -- 172.21.15.38:0/2287310213 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c54003d00 con 0x7f8c5c1e9b50 2024-09-21T03:34:02.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:02.515+0000 7f8c6257b640 1 -- 172.21.15.38:0/2287310213 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c5c1ecc90 con 0x7f8c5c1e9b50 2024-09-21T03:34:02.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:02.515+0000 7f8c6257b640 1 -- 172.21.15.38:0/2287310213 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c5c1ea380 con 0x7f8c5c1e9b50 2024-09-21T03:34:02.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:02.516+0000 7f8c6257b640 1 -- 172.21.15.38:0/2287310213 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c5c1f6860 con 0x7f8c5c1e9b50 2024-09-21T03:34:02.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:02.517+0000 7f8c58ff9640 1 -- 172.21.15.38:0/2287310213 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f8c54004030 con 0x7f8c5c1e9b50 2024-09-21T03:34:02.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:02.517+0000 7f8c58ff9640 1 --2- 172.21.15.38:0/2287310213 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8c48078bc0 0x7f8c4807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:02.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:02.517+0000 7f8c58ff9640 1 -- 172.21.15.38:0/2287310213 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f8c5409b7e0 con 0x7f8c5c1e9b50 2024-09-21T03:34:02.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:02.517+0000 7f8c5bfff640 1 --2- 172.21.15.38:0/2287310213 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8c48078bc0 0x7f8c4807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:02.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:02.518+0000 7f8c5bfff640 1 --2- 172.21.15.38:0/2287310213 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8c48078bc0 0x7f8c4807b0d0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c032000 tx=0x7f8c4c001190 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:02.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:02.522+0000 7f8c58ff9640 1 -- 172.21.15.38:0/2287310213 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f8c540638e0 con 0x7f8c5c1e9b50 2024-09-21T03:34:02.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:02.704+0000 7f8c6257b640 1 -- 172.21.15.38:0/2287310213 --> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8c5c1ebf10 con 0x7f8c48078bc0 2024-09-21T03:34:02.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:02.710+0000 7f8c58ff9640 1 -- 172.21.15.38:0/2287310213 <== mgr.4117 v2:172.21.15.195:6800/1631901344 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332519 (secure 0 0 0) 0x7f8c5c1ebf10 con 0x7f8c48078bc0 2024-09-21T03:34:02.715 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:34:02.717 INFO:teuthology.orchestra.run.smithi038.stderr:dumped all 2024-09-21T03:34:02.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:02.714+0000 7f8c6257b640 1 -- 172.21.15.38:0/2287310213 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8c48078bc0 msgr2=0x7f8c4807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:02.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:02.714+0000 7f8c6257b640 1 --2- 172.21.15.38:0/2287310213 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8c48078bc0 0x7f8c4807b0d0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c032000 tx=0x7f8c4c001190 comp rx=0 tx=0).stop 2024-09-21T03:34:02.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:02.714+0000 7f8c6257b640 1 -- 172.21.15.38:0/2287310213 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8c5c1e9b50 msgr2=0x7f8c5c1f5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:02.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:02.714+0000 7f8c6257b640 1 --2- 172.21.15.38:0/2287310213 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8c5c1e9b50 0x7f8c5c1f5870 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f8c54005fd0 tx=0x7f8c54010f40 comp rx=0 tx=0).stop 2024-09-21T03:34:02.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:02.714+0000 7f8c6257b640 1 -- 172.21.15.38:0/2287310213 shutdown_connections 2024-09-21T03:34:02.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:02.714+0000 7f8c6257b640 1 --2- 172.21.15.38:0/2287310213 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8c5c1e9b50 0x7f8c5c1f5870 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:02.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:02.714+0000 7f8c6257b640 1 --2- 172.21.15.38:0/2287310213 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8c5c157d70 0x7f8c5c1e9610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:02.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:02.714+0000 7f8c6257b640 1 --2- 172.21.15.38:0/2287310213 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8c48078bc0 0x7f8c4807b0d0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:02.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:02.714+0000 7f8c6257b640 1 --2- 172.21.15.38:0/2287310213 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8c44003a50 0x7f8c5c157750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:02.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:02.714+0000 7f8c6257b640 1 -- 172.21.15.38:0/2287310213 >> 172.21.15.38:0/2287310213 conn(0x7f8c5c0d4440 msgr2=0x7f8c5c0b35f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:02.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:02.714+0000 7f8c6257b640 1 -- 172.21.15.38:0/2287310213 shutdown_connections 2024-09-21T03:34:02.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:02.714+0000 7f8c6257b640 1 -- 172.21.15.38:0/2287310213 wait complete. 2024-09-21T03:34:02.727 INFO:teuthology.orchestra.run.smithi038.stdout:{"pg_ready":true,"pg_map":{"version":38,"stamp":"2024-09-21T03:34:00.983109+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":149,"num_write_kb":1646,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":114,"ondisk_log_size":114,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":252840,"kb_used_data":3336,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754721880,"statfs":{"total":773094113280,"available":772835205120,"internally_reserved":0,"allocated":3416064,"data_stored":2048944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12748,"internal_metadata":255315508},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2230,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003755"},"pg_stats":[{"pgid":"2.1f","version":"21'4","reported_seq":27,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.285664+0000","last_change":"2024-09-21T03:33:35.252626+0000","last_active":"2024-09-21T03:33:41.285664+0000","last_peered":"2024-09-21T03:33:41.285664+0000","last_clean":"2024-09-21T03:33:41.285664+0000","last_became_active":"2024-09-21T03:33:35.252204+0000","last_became_peered":"2024-09-21T03:33:35.252204+0000","last_unstale":"2024-09-21T03:33:41.285664+0000","last_undegraded":"2024-09-21T03:33:41.285664+0000","last_fullsized":"2024-09-21T03:33:41.285664+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_clean_scrub_stamp":"2024-09-21T03:33:34.220881+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:42:59.930617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.920863+0000","last_change":"2024-09-21T03:33:36.262395+0000","last_active":"2024-09-21T03:33:42.920863+0000","last_peered":"2024-09-21T03:33:42.920863+0000","last_clean":"2024-09-21T03:33:42.920863+0000","last_became_active":"2024-09-21T03:33:36.262085+0000","last_became_peered":"2024-09-21T03:33:36.262085+0000","last_unstale":"2024-09-21T03:33:42.920863+0000","last_undegraded":"2024-09-21T03:33:42.920863+0000","last_fullsized":"2024-09-21T03:33:42.920863+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:13:46.036072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.754945+0000","last_change":"2024-09-21T03:33:40.303707+0000","last_active":"2024-09-21T03:33:41.754945+0000","last_peered":"2024-09-21T03:33:41.754945+0000","last_clean":"2024-09-21T03:33:41.754945+0000","last_became_active":"2024-09-21T03:33:40.303328+0000","last_became_peered":"2024-09-21T03:33:40.303328+0000","last_unstale":"2024-09-21T03:33:41.754945+0000","last_undegraded":"2024-09-21T03:33:41.754945+0000","last_fullsized":"2024-09-21T03:33:41.754945+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_clean_scrub_stamp":"2024-09-21T03:33:39.262231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:58:01.084602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4,1],"acting":[7,6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.921084+0000","last_change":"2024-09-21T03:33:35.253147+0000","last_active":"2024-09-21T03:33:42.921084+0000","last_peered":"2024-09-21T03:33:42.921084+0000","last_clean":"2024-09-21T03:33:42.921084+0000","last_became_active":"2024-09-21T03:33:35.251243+0000","last_became_peered":"2024-09-21T03:33:35.251243+0000","last_unstale":"2024-09-21T03:33:42.921084+0000","last_undegraded":"2024-09-21T03:33:42.921084+0000","last_fullsized":"2024-09-21T03:33:42.921084+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_clean_scrub_stamp":"2024-09-21T03:33:34.220881+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:21:52.640936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.920870+0000","last_change":"2024-09-21T03:33:36.255323+0000","last_active":"2024-09-21T03:33:42.920870+0000","last_peered":"2024-09-21T03:33:42.920870+0000","last_clean":"2024-09-21T03:33:42.920870+0000","last_became_active":"2024-09-21T03:33:36.253192+0000","last_became_peered":"2024-09-21T03:33:36.253192+0000","last_unstale":"2024-09-21T03:33:42.920870+0000","last_undegraded":"2024-09-21T03:33:42.920870+0000","last_fullsized":"2024-09-21T03:33:42.920870+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:21:43.036932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.285988+0000","last_change":"2024-09-21T03:33:40.294983+0000","last_active":"2024-09-21T03:33:41.285988+0000","last_peered":"2024-09-21T03:33:41.285988+0000","last_clean":"2024-09-21T03:33:41.285988+0000","last_became_active":"2024-09-21T03:33:40.294528+0000","last_became_peered":"2024-09-21T03:33:40.294528+0000","last_unstale":"2024-09-21T03:33:41.285988+0000","last_undegraded":"2024-09-21T03:33:41.285988+0000","last_fullsized":"2024-09-21T03:33:41.285988+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_clean_scrub_stamp":"2024-09-21T03:33:39.262231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:39:42.878029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4,6],"acting":[2,7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.298804+0000","last_change":"2024-09-21T03:33:35.260455+0000","last_active":"2024-09-21T03:33:41.298804+0000","last_peered":"2024-09-21T03:33:41.298804+0000","last_clean":"2024-09-21T03:33:41.298804+0000","last_became_active":"2024-09-21T03:33:35.260074+0000","last_became_peered":"2024-09-21T03:33:35.260074+0000","last_unstale":"2024-09-21T03:33:41.298804+0000","last_undegraded":"2024-09-21T03:33:41.298804+0000","last_fullsized":"2024-09-21T03:33:41.298804+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_clean_scrub_stamp":"2024-09-21T03:33:34.220881+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:55:02.392036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.920774+0000","last_change":"2024-09-21T03:33:36.274462+0000","last_active":"2024-09-21T03:33:42.920774+0000","last_peered":"2024-09-21T03:33:42.920774+0000","last_clean":"2024-09-21T03:33:42.920774+0000","last_became_active":"2024-09-21T03:33:36.274080+0000","last_became_peered":"2024-09-21T03:33:36.274080+0000","last_unstale":"2024-09-21T03:33:42.920774+0000","last_undegraded":"2024-09-21T03:33:42.920774+0000","last_fullsized":"2024-09-21T03:33:42.920774+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:55:20.815479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.286750+0000","last_change":"2024-09-21T03:33:40.294613+0000","last_active":"2024-09-21T03:33:41.286750+0000","last_peered":"2024-09-21T03:33:41.286750+0000","last_clean":"2024-09-21T03:33:41.286750+0000","last_became_active":"2024-09-21T03:33:40.294138+0000","last_became_peered":"2024-09-21T03:33:40.294138+0000","last_unstale":"2024-09-21T03:33:41.286750+0000","last_undegraded":"2024-09-21T03:33:41.286750+0000","last_fullsized":"2024-09-21T03:33:41.286750+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_clean_scrub_stamp":"2024-09-21T03:33:39.262231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:10:29.758718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3,6],"acting":[2,5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.298963+0000","last_change":"2024-09-21T03:33:35.265118+0000","last_active":"2024-09-21T03:33:41.298963+0000","last_peered":"2024-09-21T03:33:41.298963+0000","last_clean":"2024-09-21T03:33:41.298963+0000","last_became_active":"2024-09-21T03:33:35.262875+0000","last_became_peered":"2024-09-21T03:33:35.262875+0000","last_unstale":"2024-09-21T03:33:41.298963+0000","last_undegraded":"2024-09-21T03:33:41.298963+0000","last_fullsized":"2024-09-21T03:33:41.298963+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_clean_scrub_stamp":"2024-09-21T03:33:34.220881+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:40:22.752363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.921523+0000","last_change":"2024-09-21T03:33:36.266776+0000","last_active":"2024-09-21T03:33:42.921523+0000","last_peered":"2024-09-21T03:33:42.921523+0000","last_clean":"2024-09-21T03:33:42.921523+0000","last_became_active":"2024-09-21T03:33:36.266449+0000","last_became_peered":"2024-09-21T03:33:36.266449+0000","last_unstale":"2024-09-21T03:33:42.921523+0000","last_undegraded":"2024-09-21T03:33:42.921523+0000","last_fullsized":"2024-09-21T03:33:42.921523+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:39:22.951079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.286642+0000","last_change":"2024-09-21T03:33:40.293654+0000","last_active":"2024-09-21T03:33:41.286642+0000","last_peered":"2024-09-21T03:33:41.286642+0000","last_clean":"2024-09-21T03:33:41.286642+0000","last_became_active":"2024-09-21T03:33:40.293236+0000","last_became_peered":"2024-09-21T03:33:40.293236+0000","last_unstale":"2024-09-21T03:33:41.286642+0000","last_undegraded":"2024-09-21T03:33:41.286642+0000","last_fullsized":"2024-09-21T03:33:41.286642+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_clean_scrub_stamp":"2024-09-21T03:33:39.262231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:20:09.214266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,5,7],"acting":[2,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.920921+0000","last_change":"2024-09-21T03:33:36.252854+0000","last_active":"2024-09-21T03:33:42.920921+0000","last_peered":"2024-09-21T03:33:42.920921+0000","last_clean":"2024-09-21T03:33:42.920921+0000","last_became_active":"2024-09-21T03:33:36.252384+0000","last_became_peered":"2024-09-21T03:33:36.252384+0000","last_unstale":"2024-09-21T03:33:42.920921+0000","last_undegraded":"2024-09-21T03:33:42.920921+0000","last_fullsized":"2024-09-21T03:33:42.920921+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:35:32.535895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.921139+0000","last_change":"2024-09-21T03:33:35.252643+0000","last_active":"2024-09-21T03:33:42.921139+0000","last_peered":"2024-09-21T03:33:42.921139+0000","last_clean":"2024-09-21T03:33:42.921139+0000","last_became_active":"2024-09-21T03:33:35.252335+0000","last_became_peered":"2024-09-21T03:33:35.252335+0000","last_unstale":"2024-09-21T03:33:42.921139+0000","last_undegraded":"2024-09-21T03:33:42.921139+0000","last_fullsized":"2024-09-21T03:33:42.921139+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_clean_scrub_stamp":"2024-09-21T03:33:34.220881+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:07:29.037511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.921479+0000","last_change":"2024-09-21T03:33:40.307347+0000","last_active":"2024-09-21T03:33:42.921479+0000","last_peered":"2024-09-21T03:33:42.921479+0000","last_clean":"2024-09-21T03:33:42.921479+0000","last_became_active":"2024-09-21T03:33:40.306899+0000","last_became_peered":"2024-09-21T03:33:40.306899+0000","last_unstale":"2024-09-21T03:33:42.921479+0000","last_undegraded":"2024-09-21T03:33:42.921479+0000","last_fullsized":"2024-09-21T03:33:42.921479+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_clean_scrub_stamp":"2024-09-21T03:33:39.262231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:45:45.403094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,5],"acting":[3,1,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.754733+0000","last_change":"2024-09-21T03:33:36.257562+0000","last_active":"2024-09-21T03:33:41.754733+0000","last_peered":"2024-09-21T03:33:41.754733+0000","last_clean":"2024-09-21T03:33:41.754733+0000","last_became_active":"2024-09-21T03:33:36.255065+0000","last_became_peered":"2024-09-21T03:33:36.255065+0000","last_unstale":"2024-09-21T03:33:41.754733+0000","last_undegraded":"2024-09-21T03:33:41.754733+0000","last_fullsized":"2024-09-21T03:33:41.754733+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:35:07.883056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.284335+0000","last_change":"2024-09-21T03:33:35.254523+0000","last_active":"2024-09-21T03:33:41.284335+0000","last_peered":"2024-09-21T03:33:41.284335+0000","last_clean":"2024-09-21T03:33:41.284335+0000","last_became_active":"2024-09-21T03:33:35.254185+0000","last_became_peered":"2024-09-21T03:33:35.254185+0000","last_unstale":"2024-09-21T03:33:41.284335+0000","last_undegraded":"2024-09-21T03:33:41.284335+0000","last_fullsized":"2024-09-21T03:33:41.284335+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_clean_scrub_stamp":"2024-09-21T03:33:34.220881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:06:37.946009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.298659+0000","last_change":"2024-09-21T03:33:40.304238+0000","last_active":"2024-09-21T03:33:41.298659+0000","last_peered":"2024-09-21T03:33:41.298659+0000","last_clean":"2024-09-21T03:33:41.298659+0000","last_became_active":"2024-09-21T03:33:40.303785+0000","last_became_peered":"2024-09-21T03:33:40.303785+0000","last_unstale":"2024-09-21T03:33:41.298659+0000","last_undegraded":"2024-09-21T03:33:41.298659+0000","last_fullsized":"2024-09-21T03:33:41.298659+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_clean_scrub_stamp":"2024-09-21T03:33:39.262231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:28:07.313548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6,1],"acting":[0,4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.283514+0000","last_change":"2024-09-21T03:33:36.255712+0000","last_active":"2024-09-21T03:33:41.283514+0000","last_peered":"2024-09-21T03:33:41.283514+0000","last_clean":"2024-09-21T03:33:41.283514+0000","last_became_active":"2024-09-21T03:33:36.255386+0000","last_became_peered":"2024-09-21T03:33:36.255386+0000","last_unstale":"2024-09-21T03:33:41.283514+0000","last_undegraded":"2024-09-21T03:33:41.283514+0000","last_fullsized":"2024-09-21T03:33:41.283514+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:49:00.717844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.284629+0000","last_change":"2024-09-21T03:33:35.266345+0000","last_active":"2024-09-21T03:33:41.284629+0000","last_peered":"2024-09-21T03:33:41.284629+0000","last_clean":"2024-09-21T03:33:41.284629+0000","last_became_active":"2024-09-21T03:33:35.266059+0000","last_became_peered":"2024-09-21T03:33:35.266059+0000","last_unstale":"2024-09-21T03:33:41.284629+0000","last_undegraded":"2024-09-21T03:33:41.284629+0000","last_fullsized":"2024-09-21T03:33:41.284629+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_clean_scrub_stamp":"2024-09-21T03:33:34.220881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:54:40.741741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.755284+0000","last_change":"2024-09-21T03:33:40.303927+0000","last_active":"2024-09-21T03:33:41.755284+0000","last_peered":"2024-09-21T03:33:41.755284+0000","last_clean":"2024-09-21T03:33:41.755284+0000","last_became_active":"2024-09-21T03:33:40.303511+0000","last_became_peered":"2024-09-21T03:33:40.303511+0000","last_unstale":"2024-09-21T03:33:41.755284+0000","last_undegraded":"2024-09-21T03:33:41.755284+0000","last_fullsized":"2024-09-21T03:33:41.755284+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_clean_scrub_stamp":"2024-09-21T03:33:39.262231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:27:26.725520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2,3],"acting":[7,4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.920506+0000","last_change":"2024-09-21T03:33:36.249470+0000","last_active":"2024-09-21T03:33:42.920506+0000","last_peered":"2024-09-21T03:33:42.920506+0000","last_clean":"2024-09-21T03:33:42.920506+0000","last_became_active":"2024-09-21T03:33:36.248726+0000","last_became_peered":"2024-09-21T03:33:36.248726+0000","last_unstale":"2024-09-21T03:33:42.920506+0000","last_undegraded":"2024-09-21T03:33:42.920506+0000","last_fullsized":"2024-09-21T03:33:42.920506+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:56:46.691011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.297935+0000","last_change":"2024-09-21T03:33:35.260543+0000","last_active":"2024-09-21T03:33:41.297935+0000","last_peered":"2024-09-21T03:33:41.297935+0000","last_clean":"2024-09-21T03:33:41.297935+0000","last_became_active":"2024-09-21T03:33:35.260041+0000","last_became_peered":"2024-09-21T03:33:35.260041+0000","last_unstale":"2024-09-21T03:33:41.297935+0000","last_undegraded":"2024-09-21T03:33:41.297935+0000","last_fullsized":"2024-09-21T03:33:41.297935+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_clean_scrub_stamp":"2024-09-21T03:33:34.220881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:59:06.265387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.286314+0000","last_change":"2024-09-21T03:33:40.309460+0000","last_active":"2024-09-21T03:33:41.286314+0000","last_peered":"2024-09-21T03:33:41.286314+0000","last_clean":"2024-09-21T03:33:41.286314+0000","last_became_active":"2024-09-21T03:33:40.306652+0000","last_became_peered":"2024-09-21T03:33:40.306652+0000","last_unstale":"2024-09-21T03:33:41.286314+0000","last_undegraded":"2024-09-21T03:33:41.286314+0000","last_fullsized":"2024-09-21T03:33:41.286314+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_clean_scrub_stamp":"2024-09-21T03:33:39.262231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:43:35.273085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,7],"acting":[2,1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.284755+0000","last_change":"2024-09-21T03:33:36.256025+0000","last_active":"2024-09-21T03:33:41.284755+0000","last_peered":"2024-09-21T03:33:41.284755+0000","last_clean":"2024-09-21T03:33:41.284755+0000","last_became_active":"2024-09-21T03:33:36.255648+0000","last_became_peered":"2024-09-21T03:33:36.255648+0000","last_unstale":"2024-09-21T03:33:41.284755+0000","last_undegraded":"2024-09-21T03:33:41.284755+0000","last_fullsized":"2024-09-21T03:33:41.284755+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:48:33.535308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.921585+0000","last_change":"2024-09-21T03:33:35.264889+0000","last_active":"2024-09-21T03:33:42.921585+0000","last_peered":"2024-09-21T03:33:42.921585+0000","last_clean":"2024-09-21T03:33:42.921585+0000","last_became_active":"2024-09-21T03:33:35.264160+0000","last_became_peered":"2024-09-21T03:33:35.264160+0000","last_unstale":"2024-09-21T03:33:42.921585+0000","last_undegraded":"2024-09-21T03:33:42.921585+0000","last_fullsized":"2024-09-21T03:33:42.921585+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_clean_scrub_stamp":"2024-09-21T03:33:34.220881+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:48:59.712004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.286424+0000","last_change":"2024-09-21T03:33:40.304067+0000","last_active":"2024-09-21T03:33:41.286424+0000","last_peered":"2024-09-21T03:33:41.286424+0000","last_clean":"2024-09-21T03:33:41.286424+0000","last_became_active":"2024-09-21T03:33:40.303613+0000","last_became_peered":"2024-09-21T03:33:40.303613+0000","last_unstale":"2024-09-21T03:33:41.286424+0000","last_undegraded":"2024-09-21T03:33:41.286424+0000","last_fullsized":"2024-09-21T03:33:41.286424+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_clean_scrub_stamp":"2024-09-21T03:33:39.262231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:51:21.940594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,3,1],"acting":[2,6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.285607+0000","last_change":"2024-09-21T03:33:36.258150+0000","last_active":"2024-09-21T03:33:41.285607+0000","last_peered":"2024-09-21T03:33:41.285607+0000","last_clean":"2024-09-21T03:33:41.285607+0000","last_became_active":"2024-09-21T03:33:36.257655+0000","last_became_peered":"2024-09-21T03:33:36.257655+0000","last_unstale":"2024-09-21T03:33:41.285607+0000","last_undegraded":"2024-09-21T03:33:41.285607+0000","last_fullsized":"2024-09-21T03:33:41.285607+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:10:32.046638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"22'1","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.285234+0000","last_change":"2024-09-21T03:33:35.264750+0000","last_active":"2024-09-21T03:33:41.285234+0000","last_peered":"2024-09-21T03:33:41.285234+0000","last_clean":"2024-09-21T03:33:41.285234+0000","last_became_active":"2024-09-21T03:33:35.264380+0000","last_became_peered":"2024-09-21T03:33:35.264380+0000","last_unstale":"2024-09-21T03:33:41.285234+0000","last_undegraded":"2024-09-21T03:33:41.285234+0000","last_fullsized":"2024-09-21T03:33:41.285234+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_clean_scrub_stamp":"2024-09-21T03:33:34.220881+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:12:06.105531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.284754+0000","last_change":"2024-09-21T03:33:40.303103+0000","last_active":"2024-09-21T03:33:41.284754+0000","last_peered":"2024-09-21T03:33:41.284754+0000","last_clean":"2024-09-21T03:33:41.284754+0000","last_became_active":"2024-09-21T03:33:40.301739+0000","last_became_peered":"2024-09-21T03:33:40.301739+0000","last_unstale":"2024-09-21T03:33:41.284754+0000","last_undegraded":"2024-09-21T03:33:41.284754+0000","last_fullsized":"2024-09-21T03:33:41.284754+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_clean_scrub_stamp":"2024-09-21T03:33:39.262231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:00:58.389938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6,3],"acting":[4,1,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.285678+0000","last_change":"2024-09-21T03:33:36.268661+0000","last_active":"2024-09-21T03:33:41.285678+0000","last_peered":"2024-09-21T03:33:41.285678+0000","last_clean":"2024-09-21T03:33:41.285678+0000","last_became_active":"2024-09-21T03:33:36.268317+0000","last_became_peered":"2024-09-21T03:33:36.268317+0000","last_unstale":"2024-09-21T03:33:41.285678+0000","last_undegraded":"2024-09-21T03:33:41.285678+0000","last_fullsized":"2024-09-21T03:33:41.285678+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:19:27.002524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.921773+0000","last_change":"2024-09-21T03:33:35.250205+0000","last_active":"2024-09-21T03:33:42.921773+0000","last_peered":"2024-09-21T03:33:42.921773+0000","last_clean":"2024-09-21T03:33:42.921773+0000","last_became_active":"2024-09-21T03:33:35.249883+0000","last_became_peered":"2024-09-21T03:33:35.249883+0000","last_unstale":"2024-09-21T03:33:42.921773+0000","last_undegraded":"2024-09-21T03:33:42.921773+0000","last_fullsized":"2024-09-21T03:33:42.921773+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_clean_scrub_stamp":"2024-09-21T03:33:34.220881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:48:54.078693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.283974+0000","last_change":"2024-09-21T03:33:40.298889+0000","last_active":"2024-09-21T03:33:41.283974+0000","last_peered":"2024-09-21T03:33:41.283974+0000","last_clean":"2024-09-21T03:33:41.283974+0000","last_became_active":"2024-09-21T03:33:40.298435+0000","last_became_peered":"2024-09-21T03:33:40.298435+0000","last_unstale":"2024-09-21T03:33:41.283974+0000","last_undegraded":"2024-09-21T03:33:41.283974+0000","last_fullsized":"2024-09-21T03:33:41.283974+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_clean_scrub_stamp":"2024-09-21T03:33:39.262231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:29:46.742857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,2,7],"acting":[6,0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.286003+0000","last_change":"2024-09-21T03:33:36.258037+0000","last_active":"2024-09-21T03:33:41.286003+0000","last_peered":"2024-09-21T03:33:41.286003+0000","last_clean":"2024-09-21T03:33:41.286003+0000","last_became_active":"2024-09-21T03:33:36.257687+0000","last_became_peered":"2024-09-21T03:33:36.257687+0000","last_unstale":"2024-09-21T03:33:41.286003+0000","last_undegraded":"2024-09-21T03:33:41.286003+0000","last_fullsized":"2024-09-21T03:33:41.286003+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:12:07.111371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.920392+0000","last_change":"2024-09-21T03:33:35.254341+0000","last_active":"2024-09-21T03:33:42.920392+0000","last_peered":"2024-09-21T03:33:42.920392+0000","last_clean":"2024-09-21T03:33:42.920392+0000","last_became_active":"2024-09-21T03:33:35.254031+0000","last_became_peered":"2024-09-21T03:33:35.254031+0000","last_unstale":"2024-09-21T03:33:42.920392+0000","last_undegraded":"2024-09-21T03:33:42.920392+0000","last_fullsized":"2024-09-21T03:33:42.920392+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_clean_scrub_stamp":"2024-09-21T03:33:34.220881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:39:48.189397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.284028+0000","last_change":"2024-09-21T03:33:40.295123+0000","last_active":"2024-09-21T03:33:41.284028+0000","last_peered":"2024-09-21T03:33:41.284028+0000","last_clean":"2024-09-21T03:33:41.284028+0000","last_became_active":"2024-09-21T03:33:40.294672+0000","last_became_peered":"2024-09-21T03:33:40.294672+0000","last_unstale":"2024-09-21T03:33:41.284028+0000","last_undegraded":"2024-09-21T03:33:41.284028+0000","last_fullsized":"2024-09-21T03:33:41.284028+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_clean_scrub_stamp":"2024-09-21T03:33:39.262231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:43:35.273085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5,7],"acting":[6,3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.298299+0000","last_change":"2024-09-21T03:33:36.277694+0000","last_active":"2024-09-21T03:33:41.298299+0000","last_peered":"2024-09-21T03:33:41.298299+0000","last_clean":"2024-09-21T03:33:41.298299+0000","last_became_active":"2024-09-21T03:33:36.277375+0000","last_became_peered":"2024-09-21T03:33:36.277375+0000","last_unstale":"2024-09-21T03:33:41.298299+0000","last_undegraded":"2024-09-21T03:33:41.298299+0000","last_fullsized":"2024-09-21T03:33:41.298299+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:39:38.842518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.921283+0000","last_change":"2024-09-21T03:33:35.250423+0000","last_active":"2024-09-21T03:33:42.921283+0000","last_peered":"2024-09-21T03:33:42.921283+0000","last_clean":"2024-09-21T03:33:42.921283+0000","last_became_active":"2024-09-21T03:33:35.248426+0000","last_became_peered":"2024-09-21T03:33:35.248426+0000","last_unstale":"2024-09-21T03:33:42.921283+0000","last_undegraded":"2024-09-21T03:33:42.921283+0000","last_fullsized":"2024-09-21T03:33:42.921283+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_clean_scrub_stamp":"2024-09-21T03:33:34.220881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:56:24.872394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.285593+0000","last_change":"2024-09-21T03:33:40.302877+0000","last_active":"2024-09-21T03:33:41.285593+0000","last_peered":"2024-09-21T03:33:41.285593+0000","last_clean":"2024-09-21T03:33:41.285593+0000","last_became_active":"2024-09-21T03:33:40.301015+0000","last_became_peered":"2024-09-21T03:33:40.301015+0000","last_unstale":"2024-09-21T03:33:41.285593+0000","last_undegraded":"2024-09-21T03:33:41.285593+0000","last_fullsized":"2024-09-21T03:33:41.285593+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_clean_scrub_stamp":"2024-09-21T03:33:39.262231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:20:09.214266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0,3],"acting":[6,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.919937+0000","last_change":"2024-09-21T03:33:36.255208+0000","last_active":"2024-09-21T03:33:42.919937+0000","last_peered":"2024-09-21T03:33:42.919937+0000","last_clean":"2024-09-21T03:33:42.919937+0000","last_became_active":"2024-09-21T03:33:36.254340+0000","last_became_peered":"2024-09-21T03:33:36.254340+0000","last_unstale":"2024-09-21T03:33:42.919937+0000","last_undegraded":"2024-09-21T03:33:42.919937+0000","last_fullsized":"2024-09-21T03:33:42.919937+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:59:34.251362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.754771+0000","last_change":"2024-09-21T03:33:35.255282+0000","last_active":"2024-09-21T03:33:41.754771+0000","last_peered":"2024-09-21T03:33:41.754771+0000","last_clean":"2024-09-21T03:33:41.754771+0000","last_became_active":"2024-09-21T03:33:35.254934+0000","last_became_peered":"2024-09-21T03:33:35.254934+0000","last_unstale":"2024-09-21T03:33:41.754771+0000","last_undegraded":"2024-09-21T03:33:41.754771+0000","last_fullsized":"2024-09-21T03:33:41.754771+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_clean_scrub_stamp":"2024-09-21T03:33:34.220881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:08:00.579605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.285752+0000","last_change":"2024-09-21T03:33:40.305801+0000","last_active":"2024-09-21T03:33:41.285752+0000","last_peered":"2024-09-21T03:33:41.285752+0000","last_clean":"2024-09-21T03:33:41.285752+0000","last_became_active":"2024-09-21T03:33:40.305316+0000","last_became_peered":"2024-09-21T03:33:40.305316+0000","last_unstale":"2024-09-21T03:33:41.285752+0000","last_undegraded":"2024-09-21T03:33:41.285752+0000","last_fullsized":"2024-09-21T03:33:41.285752+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_clean_scrub_stamp":"2024-09-21T03:33:39.262231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:10:29.758718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,7],"acting":[6,0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.284256+0000","last_change":"2024-09-21T03:33:36.264028+0000","last_active":"2024-09-21T03:33:41.284256+0000","last_peered":"2024-09-21T03:33:41.284256+0000","last_clean":"2024-09-21T03:33:41.284256+0000","last_became_active":"2024-09-21T03:33:36.263725+0000","last_became_peered":"2024-09-21T03:33:36.263725+0000","last_unstale":"2024-09-21T03:33:41.284256+0000","last_undegraded":"2024-09-21T03:33:41.284256+0000","last_fullsized":"2024-09-21T03:33:41.284256+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:48:55.084533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.920078+0000","last_change":"2024-09-21T03:33:35.260963+0000","last_active":"2024-09-21T03:33:42.920078+0000","last_peered":"2024-09-21T03:33:42.920078+0000","last_clean":"2024-09-21T03:33:42.920078+0000","last_became_active":"2024-09-21T03:33:35.257593+0000","last_became_peered":"2024-09-21T03:33:35.257593+0000","last_unstale":"2024-09-21T03:33:42.920078+0000","last_undegraded":"2024-09-21T03:33:42.920078+0000","last_fullsized":"2024-09-21T03:33:42.920078+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_clean_scrub_stamp":"2024-09-21T03:33:34.220881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:35:18.509895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.920658+0000","last_change":"2024-09-21T03:33:40.298927+0000","last_active":"2024-09-21T03:33:42.920658+0000","last_peered":"2024-09-21T03:33:42.920658+0000","last_clean":"2024-09-21T03:33:42.920658+0000","last_became_active":"2024-09-21T03:33:40.298272+0000","last_became_peered":"2024-09-21T03:33:40.298272+0000","last_unstale":"2024-09-21T03:33:42.920658+0000","last_undegraded":"2024-09-21T03:33:42.920658+0000","last_fullsized":"2024-09-21T03:33:42.920658+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_clean_scrub_stamp":"2024-09-21T03:33:39.262231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:44:52.684204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6,1],"acting":[5,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.284686+0000","last_change":"2024-09-21T03:33:36.263187+0000","last_active":"2024-09-21T03:33:41.284686+0000","last_peered":"2024-09-21T03:33:41.284686+0000","last_clean":"2024-09-21T03:33:41.284686+0000","last_became_active":"2024-09-21T03:33:36.262852+0000","last_became_peered":"2024-09-21T03:33:36.262852+0000","last_unstale":"2024-09-21T03:33:41.284686+0000","last_undegraded":"2024-09-21T03:33:41.284686+0000","last_fullsized":"2024-09-21T03:33:41.284686+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:28:30.977653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.921266+0000","last_change":"2024-09-21T03:33:35.249580+0000","last_active":"2024-09-21T03:33:42.921266+0000","last_peered":"2024-09-21T03:33:42.921266+0000","last_clean":"2024-09-21T03:33:42.921266+0000","last_became_active":"2024-09-21T03:33:35.248574+0000","last_became_peered":"2024-09-21T03:33:35.248574+0000","last_unstale":"2024-09-21T03:33:42.921266+0000","last_undegraded":"2024-09-21T03:33:42.921266+0000","last_fullsized":"2024-09-21T03:33:42.921266+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_clean_scrub_stamp":"2024-09-21T03:33:34.220881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:19:25.996685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":119,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.919871+0000","last_change":"2024-09-21T03:33:25.319776+0000","last_active":"2024-09-21T03:33:42.919871+0000","last_peered":"2024-09-21T03:33:42.919871+0000","last_clean":"2024-09-21T03:33:42.919871+0000","last_became_active":"2024-09-21T03:33:25.318399+0000","last_became_peered":"2024-09-21T03:33:25.318399+0000","last_unstale":"2024-09-21T03:33:42.919871+0000","last_undegraded":"2024-09-21T03:33:42.919871+0000","last_fullsized":"2024-09-21T03:33:42.919871+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:23.016672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:23.016672+0000","last_clean_scrub_stamp":"2024-09-21T03:33:23.016672+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:32:35.906114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.921733+0000","last_change":"2024-09-21T03:33:40.304303+0000","last_active":"2024-09-21T03:33:42.921733+0000","last_peered":"2024-09-21T03:33:42.921733+0000","last_clean":"2024-09-21T03:33:42.921733+0000","last_became_active":"2024-09-21T03:33:40.303880+0000","last_became_peered":"2024-09-21T03:33:40.303880+0000","last_unstale":"2024-09-21T03:33:42.921733+0000","last_undegraded":"2024-09-21T03:33:42.921733+0000","last_fullsized":"2024-09-21T03:33:42.921733+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_clean_scrub_stamp":"2024-09-21T03:33:39.262231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:22:59.076888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2,0],"acting":[3,1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.921586+0000","last_change":"2024-09-21T03:33:36.256735+0000","last_active":"2024-09-21T03:33:42.921586+0000","last_peered":"2024-09-21T03:33:42.921586+0000","last_clean":"2024-09-21T03:33:42.921586+0000","last_became_active":"2024-09-21T03:33:36.256296+0000","last_became_peered":"2024-09-21T03:33:36.256296+0000","last_unstale":"2024-09-21T03:33:42.921586+0000","last_undegraded":"2024-09-21T03:33:42.921586+0000","last_fullsized":"2024-09-21T03:33:42.921586+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:20:58.478604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.285704+0000","last_change":"2024-09-21T03:33:35.263469+0000","last_active":"2024-09-21T03:33:41.285704+0000","last_peered":"2024-09-21T03:33:41.285704+0000","last_clean":"2024-09-21T03:33:41.285704+0000","last_became_active":"2024-09-21T03:33:35.263119+0000","last_became_peered":"2024-09-21T03:33:35.263119+0000","last_unstale":"2024-09-21T03:33:41.285704+0000","last_undegraded":"2024-09-21T03:33:41.285704+0000","last_fullsized":"2024-09-21T03:33:41.285704+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_clean_scrub_stamp":"2024-09-21T03:33:34.220881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:54:40.741741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.286176+0000","last_change":"2024-09-21T03:33:40.304199+0000","last_active":"2024-09-21T03:33:41.286176+0000","last_peered":"2024-09-21T03:33:41.286176+0000","last_clean":"2024-09-21T03:33:41.286176+0000","last_became_active":"2024-09-21T03:33:40.303770+0000","last_became_peered":"2024-09-21T03:33:40.303770+0000","last_unstale":"2024-09-21T03:33:41.286176+0000","last_undegraded":"2024-09-21T03:33:41.286176+0000","last_fullsized":"2024-09-21T03:33:41.286176+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_clean_scrub_stamp":"2024-09-21T03:33:39.262231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:41:52.568236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7,4],"acting":[2,3,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.298815+0000","last_change":"2024-09-21T03:33:36.269892+0000","last_active":"2024-09-21T03:33:41.298815+0000","last_peered":"2024-09-21T03:33:41.298815+0000","last_clean":"2024-09-21T03:33:41.298815+0000","last_became_active":"2024-09-21T03:33:36.269521+0000","last_became_peered":"2024-09-21T03:33:36.269521+0000","last_unstale":"2024-09-21T03:33:41.298815+0000","last_undegraded":"2024-09-21T03:33:41.298815+0000","last_fullsized":"2024-09-21T03:33:41.298815+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:43:04.668863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.920621+0000","last_change":"2024-09-21T03:33:35.256681+0000","last_active":"2024-09-21T03:33:42.920621+0000","last_peered":"2024-09-21T03:33:42.920621+0000","last_clean":"2024-09-21T03:33:42.920621+0000","last_became_active":"2024-09-21T03:33:35.256325+0000","last_became_peered":"2024-09-21T03:33:35.256325+0000","last_unstale":"2024-09-21T03:33:42.920621+0000","last_undegraded":"2024-09-21T03:33:42.920621+0000","last_fullsized":"2024-09-21T03:33:42.920621+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_clean_scrub_stamp":"2024-09-21T03:33:34.220881+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:49:36.514638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.920396+0000","last_change":"2024-09-21T03:33:40.298797+0000","last_active":"2024-09-21T03:33:42.920396+0000","last_peered":"2024-09-21T03:33:42.920396+0000","last_clean":"2024-09-21T03:33:42.920396+0000","last_became_active":"2024-09-21T03:33:40.298254+0000","last_became_peered":"2024-09-21T03:33:40.298254+0000","last_unstale":"2024-09-21T03:33:42.920396+0000","last_undegraded":"2024-09-21T03:33:42.920396+0000","last_fullsized":"2024-09-21T03:33:42.920396+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_clean_scrub_stamp":"2024-09-21T03:33:39.262231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:33:49.804176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1,3],"acting":[5,4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.299252+0000","last_change":"2024-09-21T03:33:36.274252+0000","last_active":"2024-09-21T03:33:41.299252+0000","last_peered":"2024-09-21T03:33:41.299252+0000","last_clean":"2024-09-21T03:33:41.299252+0000","last_became_active":"2024-09-21T03:33:36.273890+0000","last_became_peered":"2024-09-21T03:33:36.273890+0000","last_unstale":"2024-09-21T03:33:41.299252+0000","last_undegraded":"2024-09-21T03:33:41.299252+0000","last_fullsized":"2024-09-21T03:33:41.299252+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:55:43.689119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.284896+0000","last_change":"2024-09-21T03:33:35.255320+0000","last_active":"2024-09-21T03:33:41.284896+0000","last_peered":"2024-09-21T03:33:41.284896+0000","last_clean":"2024-09-21T03:33:41.284896+0000","last_became_active":"2024-09-21T03:33:35.254912+0000","last_became_peered":"2024-09-21T03:33:35.254912+0000","last_unstale":"2024-09-21T03:33:41.284896+0000","last_undegraded":"2024-09-21T03:33:41.284896+0000","last_fullsized":"2024-09-21T03:33:41.284896+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_clean_scrub_stamp":"2024-09-21T03:33:34.220881+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:55:02.392036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.919867+0000","last_change":"2024-09-21T03:33:40.304053+0000","last_active":"2024-09-21T03:33:42.919867+0000","last_peered":"2024-09-21T03:33:42.919867+0000","last_clean":"2024-09-21T03:33:42.919867+0000","last_became_active":"2024-09-21T03:33:40.303606+0000","last_became_peered":"2024-09-21T03:33:40.303606+0000","last_unstale":"2024-09-21T03:33:42.919867+0000","last_undegraded":"2024-09-21T03:33:42.919867+0000","last_fullsized":"2024-09-21T03:33:42.919867+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_clean_scrub_stamp":"2024-09-21T03:33:39.262231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:43:29.231059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.286527+0000","last_change":"2024-09-21T03:33:36.270385+0000","last_active":"2024-09-21T03:33:41.286527+0000","last_peered":"2024-09-21T03:33:41.286527+0000","last_clean":"2024-09-21T03:33:41.286527+0000","last_became_active":"2024-09-21T03:33:36.267107+0000","last_became_peered":"2024-09-21T03:33:36.267107+0000","last_unstale":"2024-09-21T03:33:41.286527+0000","last_undegraded":"2024-09-21T03:33:41.286527+0000","last_fullsized":"2024-09-21T03:33:41.286527+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:48:55.084533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.299383+0000","last_change":"2024-09-21T03:33:35.263063+0000","last_active":"2024-09-21T03:33:41.299383+0000","last_peered":"2024-09-21T03:33:41.299383+0000","last_clean":"2024-09-21T03:33:41.299383+0000","last_became_active":"2024-09-21T03:33:35.262324+0000","last_became_peered":"2024-09-21T03:33:35.262324+0000","last_unstale":"2024-09-21T03:33:41.299383+0000","last_undegraded":"2024-09-21T03:33:41.299383+0000","last_fullsized":"2024-09-21T03:33:41.299383+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_clean_scrub_stamp":"2024-09-21T03:33:34.220881+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:35:18.509895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.920221+0000","last_change":"2024-09-21T03:33:40.298024+0000","last_active":"2024-09-21T03:33:42.920221+0000","last_peered":"2024-09-21T03:33:42.920221+0000","last_clean":"2024-09-21T03:33:42.920221+0000","last_became_active":"2024-09-21T03:33:40.297580+0000","last_became_peered":"2024-09-21T03:33:40.297580+0000","last_unstale":"2024-09-21T03:33:42.920221+0000","last_undegraded":"2024-09-21T03:33:42.920221+0000","last_fullsized":"2024-09-21T03:33:42.920221+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_clean_scrub_stamp":"2024-09-21T03:33:39.262231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:34:07.830134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,2],"acting":[1,5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.920389+0000","last_change":"2024-09-21T03:33:36.256567+0000","last_active":"2024-09-21T03:33:42.920389+0000","last_peered":"2024-09-21T03:33:42.920389+0000","last_clean":"2024-09-21T03:33:42.920389+0000","last_became_active":"2024-09-21T03:33:36.252598+0000","last_became_peered":"2024-09-21T03:33:36.252598+0000","last_unstale":"2024-09-21T03:33:42.920389+0000","last_undegraded":"2024-09-21T03:33:42.920389+0000","last_fullsized":"2024-09-21T03:33:42.920389+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:53:59.995385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":31,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.283961+0000","last_change":"2024-09-21T03:33:35.246577+0000","last_active":"2024-09-21T03:33:41.283961+0000","last_peered":"2024-09-21T03:33:41.283961+0000","last_clean":"2024-09-21T03:33:41.283961+0000","last_became_active":"2024-09-21T03:33:35.245954+0000","last_became_peered":"2024-09-21T03:33:35.245954+0000","last_unstale":"2024-09-21T03:33:41.283961+0000","last_undegraded":"2024-09-21T03:33:41.283961+0000","last_fullsized":"2024-09-21T03:33:41.283961+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_clean_scrub_stamp":"2024-09-21T03:33:34.220881+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:59:06.265387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.920916+0000","last_change":"2024-09-21T03:33:40.299966+0000","last_active":"2024-09-21T03:33:42.920916+0000","last_peered":"2024-09-21T03:33:42.920916+0000","last_clean":"2024-09-21T03:33:42.920916+0000","last_became_active":"2024-09-21T03:33:40.298455+0000","last_became_peered":"2024-09-21T03:33:40.298455+0000","last_unstale":"2024-09-21T03:33:42.920916+0000","last_undegraded":"2024-09-21T03:33:42.920916+0000","last_fullsized":"2024-09-21T03:33:42.920916+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_clean_scrub_stamp":"2024-09-21T03:33:39.262231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:59:07.141809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1,6],"acting":[5,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.919715+0000","last_change":"2024-09-21T03:33:36.261762+0000","last_active":"2024-09-21T03:33:42.919715+0000","last_peered":"2024-09-21T03:33:42.919715+0000","last_clean":"2024-09-21T03:33:42.919715+0000","last_became_active":"2024-09-21T03:33:36.261379+0000","last_became_peered":"2024-09-21T03:33:36.261379+0000","last_unstale":"2024-09-21T03:33:42.919715+0000","last_undegraded":"2024-09-21T03:33:42.919715+0000","last_fullsized":"2024-09-21T03:33:42.919715+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:35:07.883056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.298133+0000","last_change":"2024-09-21T03:33:35.264258+0000","last_active":"2024-09-21T03:33:41.298133+0000","last_peered":"2024-09-21T03:33:41.298133+0000","last_clean":"2024-09-21T03:33:41.298133+0000","last_became_active":"2024-09-21T03:33:35.262742+0000","last_became_peered":"2024-09-21T03:33:35.262742+0000","last_unstale":"2024-09-21T03:33:41.298133+0000","last_undegraded":"2024-09-21T03:33:41.298133+0000","last_fullsized":"2024-09-21T03:33:41.298133+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_clean_scrub_stamp":"2024-09-21T03:33:34.220881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:42:59.930617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.920862+0000","last_change":"2024-09-21T03:33:40.298402+0000","last_active":"2024-09-21T03:33:42.920862+0000","last_peered":"2024-09-21T03:33:42.920862+0000","last_clean":"2024-09-21T03:33:42.920862+0000","last_became_active":"2024-09-21T03:33:40.297795+0000","last_became_peered":"2024-09-21T03:33:40.297795+0000","last_unstale":"2024-09-21T03:33:42.920862+0000","last_undegraded":"2024-09-21T03:33:42.920862+0000","last_fullsized":"2024-09-21T03:33:42.920862+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_clean_scrub_stamp":"2024-09-21T03:33:39.262231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:18:24.844623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6,4],"acting":[5,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.920969+0000","last_change":"2024-09-21T03:33:36.269233+0000","last_active":"2024-09-21T03:33:42.920969+0000","last_peered":"2024-09-21T03:33:42.920969+0000","last_clean":"2024-09-21T03:33:42.920969+0000","last_became_active":"2024-09-21T03:33:36.268680+0000","last_became_peered":"2024-09-21T03:33:36.268680+0000","last_unstale":"2024-09-21T03:33:42.920969+0000","last_undegraded":"2024-09-21T03:33:42.920969+0000","last_fullsized":"2024-09-21T03:33:42.920969+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:55:43.689119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.921162+0000","last_change":"2024-09-21T03:33:35.251566+0000","last_active":"2024-09-21T03:33:42.921162+0000","last_peered":"2024-09-21T03:33:42.921162+0000","last_clean":"2024-09-21T03:33:42.921162+0000","last_became_active":"2024-09-21T03:33:35.251252+0000","last_became_peered":"2024-09-21T03:33:35.251252+0000","last_unstale":"2024-09-21T03:33:42.921162+0000","last_undegraded":"2024-09-21T03:33:42.921162+0000","last_fullsized":"2024-09-21T03:33:42.921162+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_clean_scrub_stamp":"2024-09-21T03:33:34.220881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:39:01.410384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.285366+0000","last_change":"2024-09-21T03:33:40.300353+0000","last_active":"2024-09-21T03:33:41.285366+0000","last_peered":"2024-09-21T03:33:41.285366+0000","last_clean":"2024-09-21T03:33:41.285366+0000","last_became_active":"2024-09-21T03:33:40.299481+0000","last_became_peered":"2024-09-21T03:33:40.299481+0000","last_unstale":"2024-09-21T03:33:41.285366+0000","last_undegraded":"2024-09-21T03:33:41.285366+0000","last_fullsized":"2024-09-21T03:33:41.285366+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_clean_scrub_stamp":"2024-09-21T03:33:39.262231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:51:21.940594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0,7],"acting":[6,4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.284147+0000","last_change":"2024-09-21T03:33:36.265957+0000","last_active":"2024-09-21T03:33:41.284147+0000","last_peered":"2024-09-21T03:33:41.284147+0000","last_clean":"2024-09-21T03:33:41.284147+0000","last_became_active":"2024-09-21T03:33:36.265578+0000","last_became_peered":"2024-09-21T03:33:36.265578+0000","last_unstale":"2024-09-21T03:33:41.284147+0000","last_undegraded":"2024-09-21T03:33:41.284147+0000","last_fullsized":"2024-09-21T03:33:41.284147+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:06:36.428086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.754862+0000","last_change":"2024-09-21T03:33:35.263298+0000","last_active":"2024-09-21T03:33:41.754862+0000","last_peered":"2024-09-21T03:33:41.754862+0000","last_clean":"2024-09-21T03:33:41.754862+0000","last_became_active":"2024-09-21T03:33:35.262833+0000","last_became_peered":"2024-09-21T03:33:35.262833+0000","last_unstale":"2024-09-21T03:33:41.754862+0000","last_undegraded":"2024-09-21T03:33:41.754862+0000","last_fullsized":"2024-09-21T03:33:41.754862+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_clean_scrub_stamp":"2024-09-21T03:33:34.220881+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:36:04.049350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.921613+0000","last_change":"2024-09-21T03:33:40.299399+0000","last_active":"2024-09-21T03:33:42.921613+0000","last_peered":"2024-09-21T03:33:42.921613+0000","last_clean":"2024-09-21T03:33:42.921613+0000","last_became_active":"2024-09-21T03:33:40.298855+0000","last_became_peered":"2024-09-21T03:33:40.298855+0000","last_unstale":"2024-09-21T03:33:42.921613+0000","last_undegraded":"2024-09-21T03:33:42.921613+0000","last_fullsized":"2024-09-21T03:33:42.921613+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_clean_scrub_stamp":"2024-09-21T03:33:39.262231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:38:17.611449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5,6],"acting":[3,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.298648+0000","last_change":"2024-09-21T03:33:36.268494+0000","last_active":"2024-09-21T03:33:41.298648+0000","last_peered":"2024-09-21T03:33:41.298648+0000","last_clean":"2024-09-21T03:33:41.298648+0000","last_became_active":"2024-09-21T03:33:36.268126+0000","last_became_peered":"2024-09-21T03:33:36.268126+0000","last_unstale":"2024-09-21T03:33:41.298648+0000","last_undegraded":"2024-09-21T03:33:41.298648+0000","last_fullsized":"2024-09-21T03:33:41.298648+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:00:54.354428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.14","version":"26'9","reported_seq":32,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:47.259231+0000","last_change":"2024-09-21T03:33:35.249689+0000","last_active":"2024-09-21T03:33:47.259231+0000","last_peered":"2024-09-21T03:33:47.259231+0000","last_clean":"2024-09-21T03:33:47.259231+0000","last_became_active":"2024-09-21T03:33:35.249355+0000","last_became_peered":"2024-09-21T03:33:35.249355+0000","last_unstale":"2024-09-21T03:33:47.259231+0000","last_undegraded":"2024-09-21T03:33:47.259231+0000","last_fullsized":"2024-09-21T03:33:47.259231+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_clean_scrub_stamp":"2024-09-21T03:33:34.220881+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:01:23.098006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.284651+0000","last_change":"2024-09-21T03:33:40.295609+0000","last_active":"2024-09-21T03:33:41.284651+0000","last_peered":"2024-09-21T03:33:41.284651+0000","last_clean":"2024-09-21T03:33:41.284651+0000","last_became_active":"2024-09-21T03:33:40.294963+0000","last_became_peered":"2024-09-21T03:33:40.294963+0000","last_unstale":"2024-09-21T03:33:41.284651+0000","last_undegraded":"2024-09-21T03:33:41.284651+0000","last_fullsized":"2024-09-21T03:33:41.284651+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_clean_scrub_stamp":"2024-09-21T03:33:39.262231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:55:47.724630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5,0],"acting":[4,3,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.921108+0000","last_change":"2024-09-21T03:33:36.260379+0000","last_active":"2024-09-21T03:33:42.921108+0000","last_peered":"2024-09-21T03:33:42.921108+0000","last_clean":"2024-09-21T03:33:42.921108+0000","last_became_active":"2024-09-21T03:33:36.260035+0000","last_became_peered":"2024-09-21T03:33:36.260035+0000","last_unstale":"2024-09-21T03:33:42.921108+0000","last_undegraded":"2024-09-21T03:33:42.921108+0000","last_fullsized":"2024-09-21T03:33:42.921108+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:28:03.278038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.755172+0000","last_change":"2024-09-21T03:33:35.249062+0000","last_active":"2024-09-21T03:33:41.755172+0000","last_peered":"2024-09-21T03:33:41.755172+0000","last_clean":"2024-09-21T03:33:41.755172+0000","last_became_active":"2024-09-21T03:33:35.248630+0000","last_became_peered":"2024-09-21T03:33:35.248630+0000","last_unstale":"2024-09-21T03:33:41.755172+0000","last_undegraded":"2024-09-21T03:33:41.755172+0000","last_fullsized":"2024-09-21T03:33:41.755172+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_clean_scrub_stamp":"2024-09-21T03:33:34.220881+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:49:09.367077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.286199+0000","last_change":"2024-09-21T03:33:40.305392+0000","last_active":"2024-09-21T03:33:41.286199+0000","last_peered":"2024-09-21T03:33:41.286199+0000","last_clean":"2024-09-21T03:33:41.286199+0000","last_became_active":"2024-09-21T03:33:40.304899+0000","last_became_peered":"2024-09-21T03:33:40.304899+0000","last_unstale":"2024-09-21T03:33:41.286199+0000","last_undegraded":"2024-09-21T03:33:41.286199+0000","last_fullsized":"2024-09-21T03:33:41.286199+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_clean_scrub_stamp":"2024-09-21T03:33:39.262231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:29:46.742857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1,7],"acting":[2,3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.920967+0000","last_change":"2024-09-21T03:33:36.256188+0000","last_active":"2024-09-21T03:33:42.920967+0000","last_peered":"2024-09-21T03:33:42.920967+0000","last_clean":"2024-09-21T03:33:42.920967+0000","last_became_active":"2024-09-21T03:33:36.254900+0000","last_became_peered":"2024-09-21T03:33:36.254900+0000","last_unstale":"2024-09-21T03:33:42.920967+0000","last_undegraded":"2024-09-21T03:33:42.920967+0000","last_fullsized":"2024-09-21T03:33:42.920967+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:20:54.467912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.299116+0000","last_change":"2024-09-21T03:33:35.259866+0000","last_active":"2024-09-21T03:33:41.299116+0000","last_peered":"2024-09-21T03:33:41.299116+0000","last_clean":"2024-09-21T03:33:41.299116+0000","last_became_active":"2024-09-21T03:33:35.259368+0000","last_became_peered":"2024-09-21T03:33:35.259368+0000","last_unstale":"2024-09-21T03:33:41.299116+0000","last_undegraded":"2024-09-21T03:33:41.299116+0000","last_fullsized":"2024-09-21T03:33:41.299116+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_clean_scrub_stamp":"2024-09-21T03:33:34.220881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:12:06.105531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.754662+0000","last_change":"2024-09-21T03:33:40.304856+0000","last_active":"2024-09-21T03:33:41.754662+0000","last_peered":"2024-09-21T03:33:41.754662+0000","last_clean":"2024-09-21T03:33:41.754662+0000","last_became_active":"2024-09-21T03:33:40.304421+0000","last_became_peered":"2024-09-21T03:33:40.304421+0000","last_unstale":"2024-09-21T03:33:41.754662+0000","last_undegraded":"2024-09-21T03:33:41.754662+0000","last_fullsized":"2024-09-21T03:33:41.754662+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_clean_scrub_stamp":"2024-09-21T03:33:39.262231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:10:13.916812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3,2],"acting":[7,1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.286504+0000","last_change":"2024-09-21T03:33:36.265331+0000","last_active":"2024-09-21T03:33:41.286504+0000","last_peered":"2024-09-21T03:33:41.286504+0000","last_clean":"2024-09-21T03:33:41.286504+0000","last_became_active":"2024-09-21T03:33:36.264928+0000","last_became_peered":"2024-09-21T03:33:36.264928+0000","last_unstale":"2024-09-21T03:33:41.286504+0000","last_undegraded":"2024-09-21T03:33:41.286504+0000","last_fullsized":"2024-09-21T03:33:41.286504+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:56:25.878234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.920745+0000","last_change":"2024-09-21T03:33:35.249377+0000","last_active":"2024-09-21T03:33:42.920745+0000","last_peered":"2024-09-21T03:33:42.920745+0000","last_clean":"2024-09-21T03:33:42.920745+0000","last_became_active":"2024-09-21T03:33:35.249027+0000","last_became_peered":"2024-09-21T03:33:35.249027+0000","last_unstale":"2024-09-21T03:33:42.920745+0000","last_undegraded":"2024-09-21T03:33:42.920745+0000","last_fullsized":"2024-09-21T03:33:42.920745+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_clean_scrub_stamp":"2024-09-21T03:33:34.220881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:48:36.003541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.284509+0000","last_change":"2024-09-21T03:33:40.294482+0000","last_active":"2024-09-21T03:33:41.284509+0000","last_peered":"2024-09-21T03:33:41.284509+0000","last_clean":"2024-09-21T03:33:41.284509+0000","last_became_active":"2024-09-21T03:33:40.293996+0000","last_became_peered":"2024-09-21T03:33:40.293996+0000","last_unstale":"2024-09-21T03:33:41.284509+0000","last_undegraded":"2024-09-21T03:33:41.284509+0000","last_fullsized":"2024-09-21T03:33:41.284509+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_clean_scrub_stamp":"2024-09-21T03:33:39.262231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:43:08.704373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,3,7],"acting":[4,0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.283820+0000","last_change":"2024-09-21T03:33:40.300111+0000","last_active":"2024-09-21T03:33:41.283820+0000","last_peered":"2024-09-21T03:33:41.283820+0000","last_clean":"2024-09-21T03:33:41.283820+0000","last_became_active":"2024-09-21T03:33:40.299645+0000","last_became_peered":"2024-09-21T03:33:40.299645+0000","last_unstale":"2024-09-21T03:33:41.283820+0000","last_undegraded":"2024-09-21T03:33:41.283820+0000","last_fullsized":"2024-09-21T03:33:41.283820+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_clean_scrub_stamp":"2024-09-21T03:33:39.262231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:10:29.758718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,5,3],"acting":[4,1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.284548+0000","last_change":"2024-09-21T03:33:36.248593+0000","last_active":"2024-09-21T03:33:41.284548+0000","last_peered":"2024-09-21T03:33:41.284548+0000","last_clean":"2024-09-21T03:33:41.284548+0000","last_became_active":"2024-09-21T03:33:36.248274+0000","last_became_peered":"2024-09-21T03:33:36.248274+0000","last_unstale":"2024-09-21T03:33:41.284548+0000","last_undegraded":"2024-09-21T03:33:41.284548+0000","last_fullsized":"2024-09-21T03:33:41.284548+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:36:57.787555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.920023+0000","last_change":"2024-09-21T03:33:35.261733+0000","last_active":"2024-09-21T03:33:42.920023+0000","last_peered":"2024-09-21T03:33:42.920023+0000","last_clean":"2024-09-21T03:33:42.920023+0000","last_became_active":"2024-09-21T03:33:35.261427+0000","last_became_peered":"2024-09-21T03:33:35.261427+0000","last_unstale":"2024-09-21T03:33:42.920023+0000","last_undegraded":"2024-09-21T03:33:42.920023+0000","last_fullsized":"2024-09-21T03:33:42.920023+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_clean_scrub_stamp":"2024-09-21T03:33:34.220881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:40:22.752363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.921462+0000","last_change":"2024-09-21T03:33:40.306313+0000","last_active":"2024-09-21T03:33:42.921462+0000","last_peered":"2024-09-21T03:33:42.921462+0000","last_clean":"2024-09-21T03:33:42.921462+0000","last_became_active":"2024-09-21T03:33:40.305650+0000","last_became_peered":"2024-09-21T03:33:40.305650+0000","last_unstale":"2024-09-21T03:33:42.921462+0000","last_undegraded":"2024-09-21T03:33:42.921462+0000","last_fullsized":"2024-09-21T03:33:42.921462+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_clean_scrub_stamp":"2024-09-21T03:33:39.262231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:45:58.941848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5,1],"acting":[3,2,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.920101+0000","last_change":"2024-09-21T03:33:36.255929+0000","last_active":"2024-09-21T03:33:42.920101+0000","last_peered":"2024-09-21T03:33:42.920101+0000","last_clean":"2024-09-21T03:33:42.920101+0000","last_became_active":"2024-09-21T03:33:36.255538+0000","last_became_peered":"2024-09-21T03:33:36.255538+0000","last_unstale":"2024-09-21T03:33:42.920101+0000","last_undegraded":"2024-09-21T03:33:42.920101+0000","last_fullsized":"2024-09-21T03:33:42.920101+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:33:10.543079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.284381+0000","last_change":"2024-09-21T03:33:35.264572+0000","last_active":"2024-09-21T03:33:41.284381+0000","last_peered":"2024-09-21T03:33:41.284381+0000","last_clean":"2024-09-21T03:33:41.284381+0000","last_became_active":"2024-09-21T03:33:35.264251+0000","last_became_peered":"2024-09-21T03:33:35.264251+0000","last_unstale":"2024-09-21T03:33:41.284381+0000","last_undegraded":"2024-09-21T03:33:41.284381+0000","last_fullsized":"2024-09-21T03:33:41.284381+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_clean_scrub_stamp":"2024-09-21T03:33:34.220881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:16:25.260763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.921171+0000","last_change":"2024-09-21T03:33:35.258225+0000","last_active":"2024-09-21T03:33:42.921171+0000","last_peered":"2024-09-21T03:33:42.921171+0000","last_clean":"2024-09-21T03:33:42.921171+0000","last_became_active":"2024-09-21T03:33:35.257903+0000","last_became_peered":"2024-09-21T03:33:35.257903+0000","last_unstale":"2024-09-21T03:33:42.921171+0000","last_undegraded":"2024-09-21T03:33:42.921171+0000","last_fullsized":"2024-09-21T03:33:42.921171+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_clean_scrub_stamp":"2024-09-21T03:33:34.220881+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:12:06.105531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.283808+0000","last_change":"2024-09-21T03:33:36.267650+0000","last_active":"2024-09-21T03:33:41.283808+0000","last_peered":"2024-09-21T03:33:41.283808+0000","last_clean":"2024-09-21T03:33:41.283808+0000","last_became_active":"2024-09-21T03:33:36.267315+0000","last_became_peered":"2024-09-21T03:33:36.267315+0000","last_unstale":"2024-09-21T03:33:41.283808+0000","last_undegraded":"2024-09-21T03:33:41.283808+0000","last_fullsized":"2024-09-21T03:33:41.283808+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:21:56.947372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.285062+0000","last_change":"2024-09-21T03:33:40.290894+0000","last_active":"2024-09-21T03:33:41.285062+0000","last_peered":"2024-09-21T03:33:41.285062+0000","last_clean":"2024-09-21T03:33:41.285062+0000","last_became_active":"2024-09-21T03:33:40.290424+0000","last_became_peered":"2024-09-21T03:33:40.290424+0000","last_unstale":"2024-09-21T03:33:41.285062+0000","last_undegraded":"2024-09-21T03:33:41.285062+0000","last_fullsized":"2024-09-21T03:33:41.285062+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_clean_scrub_stamp":"2024-09-21T03:33:39.262231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:56:50.726521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,7],"acting":[4,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.920464+0000","last_change":"2024-09-21T03:33:35.251276+0000","last_active":"2024-09-21T03:33:42.920464+0000","last_peered":"2024-09-21T03:33:42.920464+0000","last_clean":"2024-09-21T03:33:42.920464+0000","last_became_active":"2024-09-21T03:33:35.250758+0000","last_became_peered":"2024-09-21T03:33:35.250758+0000","last_unstale":"2024-09-21T03:33:42.920464+0000","last_undegraded":"2024-09-21T03:33:42.920464+0000","last_fullsized":"2024-09-21T03:33:42.920464+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:34.220881+0000","last_clean_scrub_stamp":"2024-09-21T03:33:34.220881+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:03:09.683999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.920393+0000","last_change":"2024-09-21T03:33:36.261907+0000","last_active":"2024-09-21T03:33:42.920393+0000","last_peered":"2024-09-21T03:33:42.920393+0000","last_clean":"2024-09-21T03:33:42.920393+0000","last_became_active":"2024-09-21T03:33:36.261483+0000","last_became_peered":"2024-09-21T03:33:36.261483+0000","last_unstale":"2024-09-21T03:33:42.920393+0000","last_undegraded":"2024-09-21T03:33:42.920393+0000","last_fullsized":"2024-09-21T03:33:42.920393+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:08:27.276982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.755140+0000","last_change":"2024-09-21T03:33:40.305034+0000","last_active":"2024-09-21T03:33:41.755140+0000","last_peered":"2024-09-21T03:33:41.755140+0000","last_clean":"2024-09-21T03:33:41.755140+0000","last_became_active":"2024-09-21T03:33:40.304555+0000","last_became_peered":"2024-09-21T03:33:40.304555+0000","last_unstale":"2024-09-21T03:33:41.755140+0000","last_undegraded":"2024-09-21T03:33:41.755140+0000","last_fullsized":"2024-09-21T03:33:41.755140+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:39.262231+0000","last_clean_scrub_stamp":"2024-09-21T03:33:39.262231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:09:41.615406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1,0],"acting":[7,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.285867+0000","last_change":"2024-09-21T03:33:36.258191+0000","last_active":"2024-09-21T03:33:41.285867+0000","last_peered":"2024-09-21T03:33:41.285867+0000","last_clean":"2024-09-21T03:33:41.285867+0000","last_became_active":"2024-09-21T03:33:36.257866+0000","last_became_peered":"2024-09-21T03:33:36.257866+0000","last_unstale":"2024-09-21T03:33:41.285867+0000","last_undegraded":"2024-09-21T03:33:41.285867+0000","last_fullsized":"2024-09-21T03:33:41.285867+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:07:30.043350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.754532+0000","last_change":"2024-09-21T03:33:36.248020+0000","last_active":"2024-09-21T03:33:41.754532+0000","last_peered":"2024-09-21T03:33:41.754532+0000","last_clean":"2024-09-21T03:33:41.754532+0000","last_became_active":"2024-09-21T03:33:36.247696+0000","last_became_peered":"2024-09-21T03:33:36.247696+0000","last_unstale":"2024-09-21T03:33:41.754532+0000","last_undegraded":"2024-09-21T03:33:41.754532+0000","last_fullsized":"2024-09-21T03:33:41.754532+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:11:14.232403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.920927+0000","last_change":"2024-09-21T03:33:36.264435+0000","last_active":"2024-09-21T03:33:42.920927+0000","last_peered":"2024-09-21T03:33:42.920927+0000","last_clean":"2024-09-21T03:33:42.920927+0000","last_became_active":"2024-09-21T03:33:36.263834+0000","last_became_peered":"2024-09-21T03:33:36.263834+0000","last_unstale":"2024-09-21T03:33:42.920927+0000","last_undegraded":"2024-09-21T03:33:42.920927+0000","last_fullsized":"2024-09-21T03:33:42.920927+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:00:54.354428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.920341+0000","last_change":"2024-09-21T03:33:36.256675+0000","last_active":"2024-09-21T03:33:42.920341+0000","last_peered":"2024-09-21T03:33:42.920341+0000","last_clean":"2024-09-21T03:33:42.920341+0000","last_became_active":"2024-09-21T03:33:36.256184+0000","last_became_peered":"2024-09-21T03:33:36.256184+0000","last_unstale":"2024-09-21T03:33:42.920341+0000","last_undegraded":"2024-09-21T03:33:42.920341+0000","last_fullsized":"2024-09-21T03:33:42.920341+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:10:44.754143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.285553+0000","last_change":"2024-09-21T03:33:36.271076+0000","last_active":"2024-09-21T03:33:41.285553+0000","last_peered":"2024-09-21T03:33:41.285553+0000","last_clean":"2024-09-21T03:33:41.285553+0000","last_became_active":"2024-09-21T03:33:36.270707+0000","last_became_peered":"2024-09-21T03:33:36.270707+0000","last_unstale":"2024-09-21T03:33:41.285553+0000","last_undegraded":"2024-09-21T03:33:41.285553+0000","last_fullsized":"2024-09-21T03:33:41.285553+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:07:30.043350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.285361+0000","last_change":"2024-09-21T03:33:36.261405+0000","last_active":"2024-09-21T03:33:41.285361+0000","last_peered":"2024-09-21T03:33:41.285361+0000","last_clean":"2024-09-21T03:33:41.285361+0000","last_became_active":"2024-09-21T03:33:36.261048+0000","last_became_peered":"2024-09-21T03:33:36.261048+0000","last_unstale":"2024-09-21T03:33:41.285361+0000","last_undegraded":"2024-09-21T03:33:41.285361+0000","last_fullsized":"2024-09-21T03:33:41.285361+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:56:25.878234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.286335+0000","last_change":"2024-09-21T03:33:36.272514+0000","last_active":"2024-09-21T03:33:41.286335+0000","last_peered":"2024-09-21T03:33:41.286335+0000","last_clean":"2024-09-21T03:33:41.286335+0000","last_became_active":"2024-09-21T03:33:36.272186+0000","last_became_peered":"2024-09-21T03:33:36.272186+0000","last_unstale":"2024-09-21T03:33:41.286335+0000","last_undegraded":"2024-09-21T03:33:41.286335+0000","last_fullsized":"2024-09-21T03:33:41.286335+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:39:02.416224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.920780+0000","last_change":"2024-09-21T03:33:36.255650+0000","last_active":"2024-09-21T03:33:42.920780+0000","last_peered":"2024-09-21T03:33:42.920780+0000","last_clean":"2024-09-21T03:33:42.920780+0000","last_became_active":"2024-09-21T03:33:36.253524+0000","last_became_peered":"2024-09-21T03:33:36.253524+0000","last_unstale":"2024-09-21T03:33:42.920780+0000","last_undegraded":"2024-09-21T03:33:42.920780+0000","last_fullsized":"2024-09-21T03:33:42.920780+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:51:37.424690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.919971+0000","last_change":"2024-09-21T03:33:36.262263+0000","last_active":"2024-09-21T03:33:42.919971+0000","last_peered":"2024-09-21T03:33:42.919971+0000","last_clean":"2024-09-21T03:33:42.919971+0000","last_became_active":"2024-09-21T03:33:36.261962+0000","last_became_peered":"2024-09-21T03:33:36.261962+0000","last_unstale":"2024-09-21T03:33:42.919971+0000","last_undegraded":"2024-09-21T03:33:42.919971+0000","last_fullsized":"2024-09-21T03:33:42.919971+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:10:25.723208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.920014+0000","last_change":"2024-09-21T03:33:36.261266+0000","last_active":"2024-09-21T03:33:42.920014+0000","last_peered":"2024-09-21T03:33:42.920014+0000","last_clean":"2024-09-21T03:33:42.920014+0000","last_became_active":"2024-09-21T03:33:36.260938+0000","last_became_peered":"2024-09-21T03:33:36.260938+0000","last_unstale":"2024-09-21T03:33:42.920014+0000","last_undegraded":"2024-09-21T03:33:42.920014+0000","last_fullsized":"2024-09-21T03:33:42.920014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:54:42.460406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.920279+0000","last_change":"2024-09-21T03:33:36.268419+0000","last_active":"2024-09-21T03:33:42.920279+0000","last_peered":"2024-09-21T03:33:42.920279+0000","last_clean":"2024-09-21T03:33:42.920279+0000","last_became_active":"2024-09-21T03:33:36.268118+0000","last_became_peered":"2024-09-21T03:33:36.268118+0000","last_unstale":"2024-09-21T03:33:42.920279+0000","last_undegraded":"2024-09-21T03:33:42.920279+0000","last_fullsized":"2024-09-21T03:33:42.920279+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:15:19.693603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.283681+0000","last_change":"2024-09-21T03:33:36.265448+0000","last_active":"2024-09-21T03:33:41.283681+0000","last_peered":"2024-09-21T03:33:41.283681+0000","last_clean":"2024-09-21T03:33:41.283681+0000","last_became_active":"2024-09-21T03:33:36.263265+0000","last_became_peered":"2024-09-21T03:33:36.263265+0000","last_unstale":"2024-09-21T03:33:41.283681+0000","last_undegraded":"2024-09-21T03:33:41.283681+0000","last_fullsized":"2024-09-21T03:33:41.283681+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:06:36.428086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.298767+0000","last_change":"2024-09-21T03:33:36.274784+0000","last_active":"2024-09-21T03:33:41.298767+0000","last_peered":"2024-09-21T03:33:41.298767+0000","last_clean":"2024-09-21T03:33:41.298767+0000","last_became_active":"2024-09-21T03:33:36.273608+0000","last_became_peered":"2024-09-21T03:33:36.273608+0000","last_unstale":"2024-09-21T03:33:41.298767+0000","last_undegraded":"2024-09-21T03:33:41.298767+0000","last_fullsized":"2024-09-21T03:33:41.298767+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:43:31.237575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.921685+0000","last_change":"2024-09-21T03:33:36.253771+0000","last_active":"2024-09-21T03:33:42.921685+0000","last_peered":"2024-09-21T03:33:42.921685+0000","last_clean":"2024-09-21T03:33:42.921685+0000","last_became_active":"2024-09-21T03:33:36.253068+0000","last_became_peered":"2024-09-21T03:33:36.253068+0000","last_unstale":"2024-09-21T03:33:42.921685+0000","last_undegraded":"2024-09-21T03:33:42.921685+0000","last_fullsized":"2024-09-21T03:33:42.921685+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:33:50.625863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.755254+0000","last_change":"2024-09-21T03:33:36.255996+0000","last_active":"2024-09-21T03:33:41.755254+0000","last_peered":"2024-09-21T03:33:41.755254+0000","last_clean":"2024-09-21T03:33:41.755254+0000","last_became_active":"2024-09-21T03:33:36.255696+0000","last_became_peered":"2024-09-21T03:33:36.255696+0000","last_unstale":"2024-09-21T03:33:41.755254+0000","last_undegraded":"2024-09-21T03:33:41.755254+0000","last_fullsized":"2024-09-21T03:33:41.755254+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:15:19.693603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.921552+0000","last_change":"2024-09-21T03:33:36.271593+0000","last_active":"2024-09-21T03:33:42.921552+0000","last_peered":"2024-09-21T03:33:42.921552+0000","last_clean":"2024-09-21T03:33:42.921552+0000","last_became_active":"2024-09-21T03:33:36.271244+0000","last_became_peered":"2024-09-21T03:33:36.271244+0000","last_unstale":"2024-09-21T03:33:42.921552+0000","last_undegraded":"2024-09-21T03:33:42.921552+0000","last_fullsized":"2024-09-21T03:33:42.921552+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:14:54.862806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.919854+0000","last_change":"2024-09-21T03:33:36.252117+0000","last_active":"2024-09-21T03:33:42.919854+0000","last_peered":"2024-09-21T03:33:42.919854+0000","last_clean":"2024-09-21T03:33:42.919854+0000","last_became_active":"2024-09-21T03:33:36.251757+0000","last_became_peered":"2024-09-21T03:33:36.251757+0000","last_unstale":"2024-09-21T03:33:42.919854+0000","last_undegraded":"2024-09-21T03:33:42.919854+0000","last_fullsized":"2024-09-21T03:33:42.919854+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:59:02.176819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.919914+0000","last_change":"2024-09-21T03:33:36.257727+0000","last_active":"2024-09-21T03:33:42.919914+0000","last_peered":"2024-09-21T03:33:42.919914+0000","last_clean":"2024-09-21T03:33:42.919914+0000","last_became_active":"2024-09-21T03:33:36.257423+0000","last_became_peered":"2024-09-21T03:33:36.257423+0000","last_unstale":"2024-09-21T03:33:42.919914+0000","last_undegraded":"2024-09-21T03:33:42.919914+0000","last_fullsized":"2024-09-21T03:33:42.919914+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:11:43.225156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.284433+0000","last_change":"2024-09-21T03:33:36.257080+0000","last_active":"2024-09-21T03:33:41.284433+0000","last_peered":"2024-09-21T03:33:41.284433+0000","last_clean":"2024-09-21T03:33:41.284433+0000","last_became_active":"2024-09-21T03:33:36.256763+0000","last_became_peered":"2024-09-21T03:33:36.256763+0000","last_unstale":"2024-09-21T03:33:41.284433+0000","last_undegraded":"2024-09-21T03:33:41.284433+0000","last_fullsized":"2024-09-21T03:33:41.284433+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:36:57.787555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.285409+0000","last_change":"2024-09-21T03:33:36.265166+0000","last_active":"2024-09-21T03:33:41.285409+0000","last_peered":"2024-09-21T03:33:41.285409+0000","last_clean":"2024-09-21T03:33:41.285409+0000","last_became_active":"2024-09-21T03:33:36.264841+0000","last_became_peered":"2024-09-21T03:33:36.264841+0000","last_unstale":"2024-09-21T03:33:41.285409+0000","last_undegraded":"2024-09-21T03:33:41.285409+0000","last_fullsized":"2024-09-21T03:33:41.285409+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:49:00.717844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.298659+0000","last_change":"2024-09-21T03:33:36.274919+0000","last_active":"2024-09-21T03:33:41.298659+0000","last_peered":"2024-09-21T03:33:41.298659+0000","last_clean":"2024-09-21T03:33:41.298659+0000","last_became_active":"2024-09-21T03:33:36.274111+0000","last_became_peered":"2024-09-21T03:33:36.274111+0000","last_unstale":"2024-09-21T03:33:41.298659+0000","last_undegraded":"2024-09-21T03:33:41.298659+0000","last_fullsized":"2024-09-21T03:33:41.298659+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:29:42.707347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.298611+0000","last_change":"2024-09-21T03:33:36.272954+0000","last_active":"2024-09-21T03:33:41.298611+0000","last_peered":"2024-09-21T03:33:41.298611+0000","last_clean":"2024-09-21T03:33:41.298611+0000","last_became_active":"2024-09-21T03:33:36.272551+0000","last_became_peered":"2024-09-21T03:33:36.272551+0000","last_unstale":"2024-09-21T03:33:41.298611+0000","last_undegraded":"2024-09-21T03:33:41.298611+0000","last_fullsized":"2024-09-21T03:33:41.298611+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:41:48.532726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.298922+0000","last_change":"2024-09-21T03:33:36.276841+0000","last_active":"2024-09-21T03:33:41.298922+0000","last_peered":"2024-09-21T03:33:41.298922+0000","last_clean":"2024-09-21T03:33:41.298922+0000","last_became_active":"2024-09-21T03:33:36.276522+0000","last_became_peered":"2024-09-21T03:33:36.276522+0000","last_unstale":"2024-09-21T03:33:41.298922+0000","last_undegraded":"2024-09-21T03:33:41.298922+0000","last_fullsized":"2024-09-21T03:33:41.298922+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:20:05.178755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.920682+0000","last_change":"2024-09-21T03:33:36.251561+0000","last_active":"2024-09-21T03:33:42.920682+0000","last_peered":"2024-09-21T03:33:42.920682+0000","last_clean":"2024-09-21T03:33:42.920682+0000","last_became_active":"2024-09-21T03:33:36.251161+0000","last_became_peered":"2024-09-21T03:33:36.251161+0000","last_unstale":"2024-09-21T03:33:42.920682+0000","last_undegraded":"2024-09-21T03:33:42.920682+0000","last_fullsized":"2024-09-21T03:33:42.920682+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:22:37.114294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.919931+0000","last_change":"2024-09-21T03:33:36.258249+0000","last_active":"2024-09-21T03:33:42.919931+0000","last_peered":"2024-09-21T03:33:42.919931+0000","last_clean":"2024-09-21T03:33:42.919931+0000","last_became_active":"2024-09-21T03:33:36.257558+0000","last_became_peered":"2024-09-21T03:33:36.257558+0000","last_unstale":"2024-09-21T03:33:42.919931+0000","last_undegraded":"2024-09-21T03:33:42.919931+0000","last_fullsized":"2024-09-21T03:33:42.919931+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:14:09.890548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.298137+0000","last_change":"2024-09-21T03:33:36.265238+0000","last_active":"2024-09-21T03:33:41.298137+0000","last_peered":"2024-09-21T03:33:41.298137+0000","last_clean":"2024-09-21T03:33:41.298137+0000","last_became_active":"2024-09-21T03:33:36.264904+0000","last_became_peered":"2024-09-21T03:33:36.264904+0000","last_unstale":"2024-09-21T03:33:41.298137+0000","last_undegraded":"2024-09-21T03:33:41.298137+0000","last_fullsized":"2024-09-21T03:33:41.298137+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:10:32.046638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.286379+0000","last_change":"2024-09-21T03:33:36.270243+0000","last_active":"2024-09-21T03:33:41.286379+0000","last_peered":"2024-09-21T03:33:41.286379+0000","last_clean":"2024-09-21T03:33:41.286379+0000","last_became_active":"2024-09-21T03:33:36.267458+0000","last_became_peered":"2024-09-21T03:33:36.267458+0000","last_unstale":"2024-09-21T03:33:41.286379+0000","last_undegraded":"2024-09-21T03:33:41.286379+0000","last_fullsized":"2024-09-21T03:33:41.286379+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:40:23.758202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.298523+0000","last_change":"2024-09-21T03:33:36.273294+0000","last_active":"2024-09-21T03:33:41.298523+0000","last_peered":"2024-09-21T03:33:41.298523+0000","last_clean":"2024-09-21T03:33:41.298523+0000","last_became_active":"2024-09-21T03:33:36.272841+0000","last_became_peered":"2024-09-21T03:33:36.272841+0000","last_unstale":"2024-09-21T03:33:41.298523+0000","last_undegraded":"2024-09-21T03:33:41.298523+0000","last_fullsized":"2024-09-21T03:33:41.298523+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:43:32.821740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.285970+0000","last_change":"2024-09-21T03:33:36.267665+0000","last_active":"2024-09-21T03:33:41.285970+0000","last_peered":"2024-09-21T03:33:41.285970+0000","last_clean":"2024-09-21T03:33:41.285970+0000","last_became_active":"2024-09-21T03:33:36.267339+0000","last_became_peered":"2024-09-21T03:33:36.267339+0000","last_unstale":"2024-09-21T03:33:41.285970+0000","last_undegraded":"2024-09-21T03:33:41.285970+0000","last_fullsized":"2024-09-21T03:33:41.285970+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:35:19.515735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.920418+0000","last_change":"2024-09-21T03:33:36.260290+0000","last_active":"2024-09-21T03:33:42.920418+0000","last_peered":"2024-09-21T03:33:42.920418+0000","last_clean":"2024-09-21T03:33:42.920418+0000","last_became_active":"2024-09-21T03:33:36.259984+0000","last_became_peered":"2024-09-21T03:33:36.259984+0000","last_unstale":"2024-09-21T03:33:42.920418+0000","last_undegraded":"2024-09-21T03:33:42.920418+0000","last_fullsized":"2024-09-21T03:33:42.920418+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:15:12.112267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:42.920700+0000","last_change":"2024-09-21T03:33:36.256894+0000","last_active":"2024-09-21T03:33:42.920700+0000","last_peered":"2024-09-21T03:33:42.920700+0000","last_clean":"2024-09-21T03:33:42.920700+0000","last_became_active":"2024-09-21T03:33:36.256598+0000","last_became_peered":"2024-09-21T03:33:36.256598+0000","last_unstale":"2024-09-21T03:33:42.920700+0000","last_undegraded":"2024-09-21T03:33:42.920700+0000","last_fullsized":"2024-09-21T03:33:42.920700+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:59:34.604906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T03:33:41.298773+0000","last_change":"2024-09-21T03:33:36.273537+0000","last_active":"2024-09-21T03:33:41.298773+0000","last_peered":"2024-09-21T03:33:41.298773+0000","last_clean":"2024-09-21T03:33:41.298773+0000","last_became_active":"2024-09-21T03:33:36.272386+0000","last_became_peered":"2024-09-21T03:33:36.272386+0000","last_unstale":"2024-09-21T03:33:41.298773+0000","last_undegraded":"2024-09-21T03:33:41.298773+0000","last_fullsized":"2024-09-21T03:33:41.298773+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:35.226721+0000","last_clean_scrub_stamp":"2024-09-21T03:33:35.226721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:10:25.723208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":1,"up_from":16,"seq":68719476747,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32036,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339804,"statfs":{"total":96636764160,"available":96603959296,"internally_reserved":0,"allocated":868352,"data_stored":697804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509451,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31460,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340380,"statfs":{"total":96636764160,"available":96604549120,"internally_reserved":0,"allocated":278528,"data_stored":107490,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509450,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31456,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340384,"statfs":{"total":96636764160,"available":96604553216,"internally_reserved":0,"allocated":274432,"data_stored":107948,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509451,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":282624,"data_stored":111208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509450,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31464,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340376,"statfs":{"total":96636764160,"available":96604545024,"internally_reserved":0,"allocated":278528,"data_stored":107422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542154,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31456,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340384,"statfs":{"total":96636764160,"available":96604553216,"internally_reserved":0,"allocated":270336,"data_stored":107366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574858,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32040,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339800,"statfs":{"total":96636764160,"available":96603955200,"internally_reserved":0,"allocated":872448,"data_stored":697826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542155,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35628,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336212,"statfs":{"total":96636764160,"available":96600281088,"internally_reserved":0,"allocated":290816,"data_stored":111880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":4514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T03:34:02.732 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-21T03:34:02.732 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-21T03:34:02.732 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-09-21T03:34:02.732 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-21T03:34:02.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:02.883+0000 7f41da939640 1 -- 172.21.15.38:0/592153169 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f41d4073d50 msgr2=0x7f41d4074160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:02.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:02.883+0000 7f41da939640 1 --2- 172.21.15.38:0/592153169 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f41d4073d50 0x7f41d4074160 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f41cc005fd0 tx=0x7f41cc02aa90 comp rx=0 tx=0).stop 2024-09-21T03:34:02.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:02.883+0000 7f41da939640 1 -- 172.21.15.38:0/592153169 shutdown_connections 2024-09-21T03:34:02.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:02.883+0000 7f41da939640 1 --2- 172.21.15.38:0/592153169 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f41d4073d50 0x7f41d4074160 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:02.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:02.883+0000 7f41da939640 1 --2- 172.21.15.38:0/592153169 >> v2:172.21.15.195:3300/0 conn(0x7f41d40b3fc0 0x7f41d40b43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:02.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:02.883+0000 7f41da939640 1 -- 172.21.15.38:0/592153169 >> 172.21.15.38:0/592153169 conn(0x7f41d40b2cc0 msgr2=0x7f41d40b30d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:02.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:02.883+0000 7f41da939640 1 -- 172.21.15.38:0/592153169 shutdown_connections 2024-09-21T03:34:02.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:02.883+0000 7f41da939640 1 -- 172.21.15.38:0/592153169 wait complete. 2024-09-21T03:34:02.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:02.884+0000 7f41da939640 1 Processor -- start 2024-09-21T03:34:02.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:02.885+0000 7f41da939640 1 -- start start 2024-09-21T03:34:02.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:02.886+0000 7f41da939640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f41d4073d50 0x7f41d4141df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:02.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:02.886+0000 7f41da939640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f41d40b3fc0 0x7f41d4160bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:02.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:02.886+0000 7f41da939640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f41d40d0b40 0x7f41d4159e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:02.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:02.886+0000 7f41da939640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f41d415b960 con 0x7f41d40b3fc0 2024-09-21T03:34:02.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:02.886+0000 7f41da939640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f41d415b7e0 con 0x7f41d4073d50 2024-09-21T03:34:02.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:02.886+0000 7f41da939640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f41d415bae0 con 0x7f41d40d0b40 2024-09-21T03:34:02.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:02.886+0000 7f41d37fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f41d40b3fc0 0x7f41d4160bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:02.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:02.886+0000 7f41d37fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f41d40b3fc0 0x7f41d4160bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42870/0 (socket says 172.21.15.38:42870) 2024-09-21T03:34:02.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:02.886+0000 7f41d37fe640 1 -- 172.21.15.38:0/2781956918 learned_addr learned my addr 172.21.15.38:0/2781956918 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:34:02.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:02.886+0000 7f41d8ed6640 1 --2- 172.21.15.38:0/2781956918 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f41d40d0b40 0x7f41d4159e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:02.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:02.886+0000 7f41d3fff640 1 --2- 172.21.15.38:0/2781956918 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f41d4073d50 0x7f41d4141df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:02.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:02.886+0000 7f41d37fe640 1 -- 172.21.15.38:0/2781956918 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f41d40d0b40 msgr2=0x7f41d4159e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:02.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:02.886+0000 7f41d37fe640 1 --2- 172.21.15.38:0/2781956918 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f41d40d0b40 0x7f41d4159e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:02.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:02.886+0000 7f41d37fe640 1 -- 172.21.15.38:0/2781956918 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f41d4073d50 msgr2=0x7f41d4141df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:02.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:02.886+0000 7f41d37fe640 1 --2- 172.21.15.38:0/2781956918 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f41d4073d50 0x7f41d4141df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:02.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:02.886+0000 7f41d37fe640 1 -- 172.21.15.38:0/2781956918 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41d413eaa0 con 0x7f41d40b3fc0 2024-09-21T03:34:02.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:02.887+0000 7f41d37fe640 1 --2- 172.21.15.38:0/2781956918 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f41d40b3fc0 0x7f41d4160bd0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f41c800e9f0 tx=0x7f41c800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:02.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:02.887+0000 7f41d0ff9640 1 -- 172.21.15.38:0/2781956918 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41c800cd80 con 0x7f41d40b3fc0 2024-09-21T03:34:02.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:02.887+0000 7f41d0ff9640 1 -- 172.21.15.38:0/2781956918 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f41c8004020 con 0x7f41d40b3fc0 2024-09-21T03:34:02.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:02.887+0000 7f41da939640 1 -- 172.21.15.38:0/2781956918 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41d413ed80 con 0x7f41d40b3fc0 2024-09-21T03:34:02.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:02.887+0000 7f41d0ff9640 1 -- 172.21.15.38:0/2781956918 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41c8004470 con 0x7f41d40b3fc0 2024-09-21T03:34:02.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:02.887+0000 7f41da939640 1 -- 172.21.15.38:0/2781956918 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41d413f340 con 0x7f41d40b3fc0 2024-09-21T03:34:02.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:02.888+0000 7f41da939640 1 -- 172.21.15.38:0/2781956918 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41d415bf30 con 0x7f41d40b3fc0 2024-09-21T03:34:02.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:02.889+0000 7f41d0ff9640 1 -- 172.21.15.38:0/2781956918 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f41c80035f0 con 0x7f41d40b3fc0 2024-09-21T03:34:02.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:02.889+0000 7f41d0ff9640 1 --2- 172.21.15.38:0/2781956918 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f41c0078b90 0x7f41c007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:02.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:02.889+0000 7f41d0ff9640 1 -- 172.21.15.38:0/2781956918 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f41c809bc00 con 0x7f41d40b3fc0 2024-09-21T03:34:02.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:02.889+0000 7f41d3fff640 1 --2- 172.21.15.38:0/2781956918 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f41c0078b90 0x7f41c007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:02.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:02.890+0000 7f41d3fff640 1 --2- 172.21.15.38:0/2781956918 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f41c0078b90 0x7f41c007b0a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f41c4007fd0 tx=0x7f41c4009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:02.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:02.894+0000 7f41d0ff9640 1 -- 172.21.15.38:0/2781956918 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f41c80a2080 con 0x7f41d40b3fc0 2024-09-21T03:34:03.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:03.118+0000 7f41da939640 1 -- 172.21.15.38:0/2781956918 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f41d40b50d0 con 0x7f41d40b3fc0 2024-09-21T03:34:03.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:03.118+0000 7f41d0ff9640 1 -- 172.21.15.38:0/2781956918 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f41c8063d00 con 0x7f41d40b3fc0 2024-09-21T03:34:03.120 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:34:03.120 INFO:teuthology.orchestra.run.smithi038.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-21T03:34:03.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:03.120+0000 7f41da939640 1 -- 172.21.15.38:0/2781956918 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f41c0078b90 msgr2=0x7f41c007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:03.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:03.120+0000 7f41da939640 1 --2- 172.21.15.38:0/2781956918 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f41c0078b90 0x7f41c007b0a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f41c4007fd0 tx=0x7f41c4009040 comp rx=0 tx=0).stop 2024-09-21T03:34:03.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:03.120+0000 7f41da939640 1 -- 172.21.15.38:0/2781956918 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f41d40b3fc0 msgr2=0x7f41d4160bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:03.123 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:03.120+0000 7f41da939640 1 --2- 172.21.15.38:0/2781956918 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f41d40b3fc0 0x7f41d4160bd0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f41c800e9f0 tx=0x7f41c800eeb0 comp rx=0 tx=0).stop 2024-09-21T03:34:03.123 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:03.120+0000 7f41da939640 1 -- 172.21.15.38:0/2781956918 shutdown_connections 2024-09-21T03:34:03.123 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:03.120+0000 7f41da939640 1 --2- 172.21.15.38:0/2781956918 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f41c0078b90 0x7f41c007b0a0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:03.123 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:03.120+0000 7f41da939640 1 --2- 172.21.15.38:0/2781956918 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f41d40d0b40 0x7f41d4159e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:03.123 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:03.120+0000 7f41da939640 1 --2- 172.21.15.38:0/2781956918 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f41d40b3fc0 0x7f41d4160bd0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:03.123 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:03.120+0000 7f41da939640 1 --2- 172.21.15.38:0/2781956918 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f41d4073d50 0x7f41d4141df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:03.123 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:03.120+0000 7f41da939640 1 -- 172.21.15.38:0/2781956918 >> 172.21.15.38:0/2781956918 conn(0x7f41d40b2cc0 msgr2=0x7f41d40cffc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:03.123 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:03.120+0000 7f41da939640 1 -- 172.21.15.38:0/2781956918 shutdown_connections 2024-09-21T03:34:03.123 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:03.121+0000 7f41da939640 1 -- 172.21.15.38:0/2781956918 wait complete. 2024-09-21T03:34:03.133 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-09-21T03:34:03.133 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:34:03.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:03.248+0000 7f305185c640 1 -- 172.21.15.38:0/2068098107 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f304c0cfe00 msgr2=0x7f304c0d2a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:03.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:03.248+0000 7f305185c640 1 --2- 172.21.15.38:0/2068098107 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f304c0cfe00 0x7f304c0d2a20 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f303c009830 tx=0x7f303c02a510 comp rx=0 tx=0).stop 2024-09-21T03:34:03.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:03.248+0000 7f305185c640 1 -- 172.21.15.38:0/2068098107 shutdown_connections 2024-09-21T03:34:03.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:03.248+0000 7f305185c640 1 --2- 172.21.15.38:0/2068098107 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f304c162590 0x7f304c1629a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:03.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:03.248+0000 7f305185c640 1 --2- 172.21.15.38:0/2068098107 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f304c0cfe00 0x7f304c0d2a20 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:03.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:03.248+0000 7f305185c640 1 -- 172.21.15.38:0/2068098107 >> 172.21.15.38:0/2068098107 conn(0x7f304c0d4440 msgr2=0x7f304c0d4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:03.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:03.248+0000 7f305185c640 1 -- 172.21.15.38:0/2068098107 shutdown_connections 2024-09-21T03:34:03.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:03.248+0000 7f305185c640 1 -- 172.21.15.38:0/2068098107 wait complete. 2024-09-21T03:34:03.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:03.249+0000 7f305185c640 1 Processor -- start 2024-09-21T03:34:03.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:03.250+0000 7f305185c640 1 -- start start 2024-09-21T03:34:03.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:03.251+0000 7f305185c640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f304c0b2e60 0x7f304c159a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:03.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:03.251+0000 7f305185c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f304c0cfe00 0x7f304c151a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:03.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:03.251+0000 7f305185c640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f304c162590 0x7f304c155df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:03.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:03.251+0000 7f305185c640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f304c0b3980 con 0x7f304c0cfe00 2024-09-21T03:34:03.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:03.251+0000 7f305185c640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f304c0cf5a0 con 0x7f304c0b2e60 2024-09-21T03:34:03.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:03.251+0000 7f305185c640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f304c0b3b00 con 0x7f304c162590 2024-09-21T03:34:03.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:03.251+0000 7f304a7fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f304c0cfe00 0x7f304c151a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:03.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:03.251+0000 7f304a7fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f304c0cfe00 0x7f304c151a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42890/0 (socket says 172.21.15.38:42890) 2024-09-21T03:34:03.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:03.251+0000 7f304a7fc640 1 -- 172.21.15.38:0/341481593 learned_addr learned my addr 172.21.15.38:0/341481593 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:34:03.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:03.251+0000 7f304affd640 1 --2- 172.21.15.38:0/341481593 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f304c0b2e60 0x7f304c159a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:03.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:03.251+0000 7f304b7fe640 1 --2- 172.21.15.38:0/341481593 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f304c162590 0x7f304c155df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:03.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:03.251+0000 7f304a7fc640 1 -- 172.21.15.38:0/341481593 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f304c162590 msgr2=0x7f304c155df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:03.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:03.251+0000 7f304a7fc640 1 --2- 172.21.15.38:0/341481593 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f304c162590 0x7f304c155df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:03.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:03.251+0000 7f304a7fc640 1 -- 172.21.15.38:0/341481593 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f304c0b2e60 msgr2=0x7f304c159a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:03.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:03.251+0000 7f304a7fc640 1 --2- 172.21.15.38:0/341481593 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f304c0b2e60 0x7f304c159a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:03.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:03.251+0000 7f304a7fc640 1 -- 172.21.15.38:0/341481593 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f304c154d50 con 0x7f304c0cfe00 2024-09-21T03:34:03.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:03.252+0000 7f304a7fc640 1 --2- 172.21.15.38:0/341481593 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f304c0cfe00 0x7f304c151a40 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f304000e7c0 tx=0x7f304000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:03.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:03.252+0000 7f3033fff640 1 -- 172.21.15.38:0/341481593 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f304000ccc0 con 0x7f304c0cfe00 2024-09-21T03:34:03.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:03.252+0000 7f3033fff640 1 -- 172.21.15.38:0/341481593 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3040010430 con 0x7f304c0cfe00 2024-09-21T03:34:03.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:03.252+0000 7f3033fff640 1 -- 172.21.15.38:0/341481593 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3040010860 con 0x7f304c0cfe00 2024-09-21T03:34:03.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:03.252+0000 7f305185c640 1 -- 172.21.15.38:0/341481593 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f304c155030 con 0x7f304c0cfe00 2024-09-21T03:34:03.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:03.252+0000 7f305185c640 1 -- 172.21.15.38:0/341481593 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f304c1555f0 con 0x7f304c0cfe00 2024-09-21T03:34:03.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:03.253+0000 7f305185c640 1 -- 172.21.15.38:0/341481593 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f304c0d2a20 con 0x7f304c0cfe00 2024-09-21T03:34:03.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:03.254+0000 7f3033fff640 1 -- 172.21.15.38:0/341481593 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f3040010a00 con 0x7f304c0cfe00 2024-09-21T03:34:03.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:03.254+0000 7f3033fff640 1 --2- 172.21.15.38:0/341481593 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3038078b90 0x7f303807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:03.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:03.254+0000 7f3033fff640 1 -- 172.21.15.38:0/341481593 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f304009b9e0 con 0x7f304c0cfe00 2024-09-21T03:34:03.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:03.254+0000 7f304affd640 1 --2- 172.21.15.38:0/341481593 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3038078b90 0x7f303807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:03.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:03.255+0000 7f304affd640 1 --2- 172.21.15.38:0/341481593 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3038078b90 0x7f303807b0a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f303c034000 tx=0x7f303c004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:03.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:03.259+0000 7f3033fff640 1 -- 172.21.15.38:0/341481593 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f3040063520 con 0x7f304c0cfe00 2024-09-21T03:34:03.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:03.483+0000 7f305185c640 1 -- 172.21.15.38:0/341481593 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f304c0c7b00 con 0x7f304c0cfe00 2024-09-21T03:34:03.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:03.484+0000 7f3033fff640 1 -- 172.21.15.38:0/341481593 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5219 (secure 0 0 0) 0x7f3040067f50 con 0x7f304c0cfe00 2024-09-21T03:34:03.486 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:34:03.486 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":9,"btime":"2024-09-21T03:33:56:329606+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4322,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/593838590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":593838590},{"type":"v1","addr":"172.21.15.195:6836","nonce":593838590}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4374,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/663402481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":663402481},{"type":"v1","addr":"172.21.15.38:6833","nonce":663402481}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4377,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/2614829035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":2614829035},{"type":"v1","addr":"172.21.15.38:6835","nonce":2614829035}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:33:35.622623+0000","modified":"2024-09-21T03:33:55.704494+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4402},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4402":{"gid":4402,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.195:6837/1420805977","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":1420805977},{"type":"v1","addr":"172.21.15.195:6837","nonce":1420805977}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4402,"qdb_cluster":[4402]},"id":1}]} 2024-09-21T03:34:03.486 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 9 2024-09-21T03:34:03.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:03.486+0000 7f305185c640 1 -- 172.21.15.38:0/341481593 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3038078b90 msgr2=0x7f303807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:03.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:03.486+0000 7f305185c640 1 --2- 172.21.15.38:0/341481593 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3038078b90 0x7f303807b0a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f303c034000 tx=0x7f303c004730 comp rx=0 tx=0).stop 2024-09-21T03:34:03.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:03.486+0000 7f305185c640 1 -- 172.21.15.38:0/341481593 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f304c0cfe00 msgr2=0x7f304c151a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:03.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:03.486+0000 7f305185c640 1 --2- 172.21.15.38:0/341481593 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f304c0cfe00 0x7f304c151a40 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f304000e7c0 tx=0x7f304000ec80 comp rx=0 tx=0).stop 2024-09-21T03:34:03.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:03.486+0000 7f305185c640 1 -- 172.21.15.38:0/341481593 shutdown_connections 2024-09-21T03:34:03.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:03.486+0000 7f305185c640 1 --2- 172.21.15.38:0/341481593 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3038078b90 0x7f303807b0a0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:03.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:03.486+0000 7f305185c640 1 --2- 172.21.15.38:0/341481593 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f304c162590 0x7f304c155df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:03.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:03.486+0000 7f305185c640 1 --2- 172.21.15.38:0/341481593 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f304c0cfe00 0x7f304c151a40 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:03.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:03.486+0000 7f305185c640 1 --2- 172.21.15.38:0/341481593 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f304c0b2e60 0x7f304c159a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:03.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:03.486+0000 7f305185c640 1 -- 172.21.15.38:0/341481593 >> 172.21.15.38:0/341481593 conn(0x7f304c0d4440 msgr2=0x7f304c0d4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:03.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:03.486+0000 7f305185c640 1 -- 172.21.15.38:0/341481593 shutdown_connections 2024-09-21T03:34:03.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:03.486+0000 7f305185c640 1 -- 172.21.15.38:0/341481593 wait complete. 2024-09-21T03:34:03.500 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:34:03.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:03.614+0000 7f2abf74a640 1 -- 172.21.15.38:0/541454965 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2ab806f290 msgr2=0x7f2ab80b40c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:03.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:03.614+0000 7f2abf74a640 1 --2- 172.21.15.38:0/541454965 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2ab806f290 0x7f2ab80b40c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f2ab0005fd0 tx=0x7f2ab002a970 comp rx=0 tx=0).stop 2024-09-21T03:34:03.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:03.614+0000 7f2abf74a640 1 -- 172.21.15.38:0/541454965 shutdown_connections 2024-09-21T03:34:03.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:03.614+0000 7f2abf74a640 1 --2- 172.21.15.38:0/541454965 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2ab806f290 0x7f2ab80b40c0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:03.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:03.614+0000 7f2abf74a640 1 --2- 172.21.15.38:0/541454965 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2ab806e940 0x7f2ab806ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:03.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:03.614+0000 7f2abf74a640 1 -- 172.21.15.38:0/541454965 >> 172.21.15.38:0/541454965 conn(0x7f2ab8073e30 msgr2=0x7f2ab8074240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:03.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:03.615+0000 7f2abf74a640 1 -- 172.21.15.38:0/541454965 shutdown_connections 2024-09-21T03:34:03.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:03.615+0000 7f2abf74a640 1 -- 172.21.15.38:0/541454965 wait complete. 2024-09-21T03:34:03.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:03.616+0000 7f2abf74a640 1 Processor -- start 2024-09-21T03:34:03.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:03.616+0000 7f2abf74a640 1 -- start start 2024-09-21T03:34:03.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:03.618+0000 7f2abf74a640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2ab806e940 0x7f2ab80ad960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:03.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:03.618+0000 7f2abf74a640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2ab806f290 0x7f2ab8146670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:03.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:03.618+0000 7f2abf74a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2ab80cee70 0x7f2ab8159a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:03.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:03.618+0000 7f2abf74a640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f2ab80b4a10 con 0x7f2ab80cee70 2024-09-21T03:34:03.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:03.618+0000 7f2abf74a640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f2ab80b4890 con 0x7f2ab806f290 2024-09-21T03:34:03.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:03.618+0000 7f2abf74a640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f2ab80b4b90 con 0x7f2ab806e940 2024-09-21T03:34:03.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:03.618+0000 7f2abdce7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2ab80cee70 0x7f2ab8159a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:03.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:03.618+0000 7f2abdce7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2ab80cee70 0x7f2ab8159a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42914/0 (socket says 172.21.15.38:42914) 2024-09-21T03:34:03.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:03.618+0000 7f2abdce7640 1 -- 172.21.15.38:0/1036112633 learned_addr learned my addr 172.21.15.38:0/1036112633 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:34:03.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:03.618+0000 7f2abd4e6640 1 --2- 172.21.15.38:0/1036112633 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2ab806e940 0x7f2ab80ad960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:03.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:03.618+0000 7f2abcce5640 1 --2- 172.21.15.38:0/1036112633 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2ab806f290 0x7f2ab8146670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:03.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:03.618+0000 7f2abdce7640 1 -- 172.21.15.38:0/1036112633 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2ab806e940 msgr2=0x7f2ab80ad960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:03.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:03.618+0000 7f2abdce7640 1 --2- 172.21.15.38:0/1036112633 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2ab806e940 0x7f2ab80ad960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:03.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:03.618+0000 7f2abdce7640 1 -- 172.21.15.38:0/1036112633 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2ab806f290 msgr2=0x7f2ab8146670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:03.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:03.618+0000 7f2abdce7640 1 --2- 172.21.15.38:0/1036112633 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2ab806f290 0x7f2ab8146670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:03.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:03.618+0000 7f2abdce7640 1 -- 172.21.15.38:0/1036112633 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ab81f6610 con 0x7f2ab80cee70 2024-09-21T03:34:03.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:03.618+0000 7f2abdce7640 1 --2- 172.21.15.38:0/1036112633 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2ab80cee70 0x7f2ab8159a70 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f2ab0006010 tx=0x7f2ab0003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:03.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:03.619+0000 7f2aadffb640 1 -- 172.21.15.38:0/1036112633 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ab0034040 con 0x7f2ab80cee70 2024-09-21T03:34:03.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:03.619+0000 7f2aadffb640 1 -- 172.21.15.38:0/1036112633 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2ab0004640 con 0x7f2ab80cee70 2024-09-21T03:34:03.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:03.619+0000 7f2abf74a640 1 -- 172.21.15.38:0/1036112633 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ab81f68f0 con 0x7f2ab80cee70 2024-09-21T03:34:03.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:03.619+0000 7f2aadffb640 1 -- 172.21.15.38:0/1036112633 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ab0045480 con 0x7f2ab80cee70 2024-09-21T03:34:03.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:03.619+0000 7f2abf74a640 1 -- 172.21.15.38:0/1036112633 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ab81f6eb0 con 0x7f2ab80cee70 2024-09-21T03:34:03.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:03.620+0000 7f2abf74a640 1 -- 172.21.15.38:0/1036112633 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ab80b5020 con 0x7f2ab80cee70 2024-09-21T03:34:03.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:03.621+0000 7f2aadffb640 1 -- 172.21.15.38:0/1036112633 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f2ab0037020 con 0x7f2ab80cee70 2024-09-21T03:34:03.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:03.621+0000 7f2aadffb640 1 --2- 172.21.15.38:0/1036112633 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2a9c078b90 0x7f2a9c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:03.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:03.621+0000 7f2aadffb640 1 -- 172.21.15.38:0/1036112633 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f2ab00bc840 con 0x7f2ab80cee70 2024-09-21T03:34:03.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:03.622+0000 7f2abd4e6640 1 --2- 172.21.15.38:0/1036112633 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2a9c078b90 0x7f2a9c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:03.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:03.622+0000 7f2abd4e6640 1 --2- 172.21.15.38:0/1036112633 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2a9c078b90 0x7f2a9c07b0a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f2ab4006fd0 tx=0x7f2ab4009060 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:03.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:03.626+0000 7f2aadffb640 1 -- 172.21.15.38:0/1036112633 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f2ab00842f0 con 0x7f2ab80cee70 2024-09-21T03:34:03.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:03.852+0000 7f2abf74a640 1 -- 172.21.15.38:0/1036112633 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2ab80725c0 con 0x7f2ab80cee70 2024-09-21T03:34:03.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:03.853+0000 7f2aadffb640 1 -- 172.21.15.38:0/1036112633 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5219 (secure 0 0 0) 0x7f2ab0088d20 con 0x7f2ab80cee70 2024-09-21T03:34:03.855 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:34:03.855 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":9,"btime":"2024-09-21T03:33:56:329606+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4322,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/593838590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":593838590},{"type":"v1","addr":"172.21.15.195:6836","nonce":593838590}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4374,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/663402481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":663402481},{"type":"v1","addr":"172.21.15.38:6833","nonce":663402481}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4377,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/2614829035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":2614829035},{"type":"v1","addr":"172.21.15.38:6835","nonce":2614829035}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:33:35.622623+0000","modified":"2024-09-21T03:33:55.704494+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4402},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4402":{"gid":4402,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.195:6837/1420805977","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":1420805977},{"type":"v1","addr":"172.21.15.195:6837","nonce":1420805977}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4402,"qdb_cluster":[4402]},"id":1}]} 2024-09-21T03:34:03.855 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 9 2024-09-21T03:34:03.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:03.855+0000 7f2abf74a640 1 -- 172.21.15.38:0/1036112633 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2a9c078b90 msgr2=0x7f2a9c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:03.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:03.855+0000 7f2abf74a640 1 --2- 172.21.15.38:0/1036112633 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2a9c078b90 0x7f2a9c07b0a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f2ab4006fd0 tx=0x7f2ab4009060 comp rx=0 tx=0).stop 2024-09-21T03:34:03.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:03.855+0000 7f2abf74a640 1 -- 172.21.15.38:0/1036112633 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2ab80cee70 msgr2=0x7f2ab8159a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:03.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:03.855+0000 7f2abf74a640 1 --2- 172.21.15.38:0/1036112633 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2ab80cee70 0x7f2ab8159a70 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f2ab0006010 tx=0x7f2ab0003f10 comp rx=0 tx=0).stop 2024-09-21T03:34:03.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:03.855+0000 7f2abf74a640 1 -- 172.21.15.38:0/1036112633 shutdown_connections 2024-09-21T03:34:03.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:03.855+0000 7f2abf74a640 1 --2- 172.21.15.38:0/1036112633 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2ab80cee70 0x7f2ab8159a70 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:03.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:03.855+0000 7f2abf74a640 1 --2- 172.21.15.38:0/1036112633 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2ab806f290 0x7f2ab8146670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:03.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:03.855+0000 7f2abf74a640 1 --2- 172.21.15.38:0/1036112633 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2a9c078b90 0x7f2a9c07b0a0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:03.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:03.855+0000 7f2abf74a640 1 --2- 172.21.15.38:0/1036112633 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2ab806e940 0x7f2ab80ad960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:03.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:03.855+0000 7f2abf74a640 1 -- 172.21.15.38:0/1036112633 >> 172.21.15.38:0/1036112633 conn(0x7f2ab8073e30 msgr2=0x7f2ab80b0a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:03.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:03.855+0000 7f2abf74a640 1 -- 172.21.15.38:0/1036112633 shutdown_connections 2024-09-21T03:34:03.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:03.855+0000 7f2abf74a640 1 -- 172.21.15.38:0/1036112633 wait complete. 2024-09-21T03:34:03.868 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T03:34:03.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:03.983+0000 7ff779871640 1 -- 172.21.15.38:0/3520549072 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff7740cd1d0 msgr2=0x7ff7740b5590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:03.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:03.983+0000 7ff779871640 1 --2- 172.21.15.38:0/3520549072 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff7740cd1d0 0x7ff7740b5590 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7ff774073e10 tx=0x7ff76c02a5f0 comp rx=0 tx=0).stop 2024-09-21T03:34:03.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:03.983+0000 7ff779871640 1 -- 172.21.15.38:0/3520549072 shutdown_connections 2024-09-21T03:34:03.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:03.983+0000 7ff779871640 1 --2- 172.21.15.38:0/3520549072 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff7740cd1d0 0x7ff7740b5590 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:03.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:03.983+0000 7ff779871640 1 --2- 172.21.15.38:0/3520549072 >> v2:172.21.15.195:3300/0 conn(0x7ff7740d19d0 0x7ff77415c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:03.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:03.983+0000 7ff779871640 1 -- 172.21.15.38:0/3520549072 >> 172.21.15.38:0/3520549072 conn(0x7ff7740b41d0 msgr2=0x7ff7740b45e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:03.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:03.983+0000 7ff779871640 1 -- 172.21.15.38:0/3520549072 shutdown_connections 2024-09-21T03:34:03.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:03.983+0000 7ff779871640 1 -- 172.21.15.38:0/3520549072 wait complete. 2024-09-21T03:34:03.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:03.985+0000 7ff779871640 1 Processor -- start 2024-09-21T03:34:03.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:03.985+0000 7ff779871640 1 -- start start 2024-09-21T03:34:03.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:03.986+0000 7ff779871640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff7740b2500 0x7ff7740c8b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:03.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:03.986+0000 7ff779871640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff7740d19d0 0x7ff7740cae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:03.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:03.986+0000 7ff779871640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff7740cb350 0x7ff7741f66a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:03.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:03.986+0000 7ff779871640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ff7740b5ff0 con 0x7ff7740b2500 2024-09-21T03:34:03.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:03.986+0000 7ff779871640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ff7740b5e70 con 0x7ff7740d19d0 2024-09-21T03:34:03.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:03.986+0000 7ff779871640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7ff7740b6170 con 0x7ff7740cb350 2024-09-21T03:34:03.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:03.987+0000 7ff772ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff7740b2500 0x7ff7740c8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:03.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:03.987+0000 7ff772ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff7740b2500 0x7ff7740c8b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42940/0 (socket says 172.21.15.38:42940) 2024-09-21T03:34:03.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:03.987+0000 7ff772ffd640 1 -- 172.21.15.38:0/3832659953 learned_addr learned my addr 172.21.15.38:0/3832659953 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:34:03.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:03.987+0000 7ff7737fe640 1 --2- 172.21.15.38:0/3832659953 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff7740cb350 0x7ff7741f66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:03.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:03.987+0000 7ff7727fc640 1 --2- 172.21.15.38:0/3832659953 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff7740d19d0 0x7ff7740cae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:03.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:03.987+0000 7ff772ffd640 1 -- 172.21.15.38:0/3832659953 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff7740cb350 msgr2=0x7ff7741f66a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:03.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:03.987+0000 7ff772ffd640 1 --2- 172.21.15.38:0/3832659953 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff7740cb350 0x7ff7741f66a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:03.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:03.987+0000 7ff772ffd640 1 -- 172.21.15.38:0/3832659953 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff7740d19d0 msgr2=0x7ff7740cae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:03.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:03.987+0000 7ff772ffd640 1 --2- 172.21.15.38:0/3832659953 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff7740d19d0 0x7ff7740cae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:03.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:03.987+0000 7ff772ffd640 1 -- 172.21.15.38:0/3832659953 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7740cbaf0 con 0x7ff7740b2500 2024-09-21T03:34:03.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:03.987+0000 7ff7727fc640 1 --2- 172.21.15.38:0/3832659953 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff7740d19d0 0x7ff7740cae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:34:03.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:03.987+0000 7ff772ffd640 1 --2- 172.21.15.38:0/3832659953 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff7740b2500 0x7ff7740c8b20 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ff76400eb60 tx=0x7ff76400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:03.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:03.987+0000 7ff76bfff640 1 -- 172.21.15.38:0/3832659953 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff764010630 con 0x7ff7740b2500 2024-09-21T03:34:03.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:03.987+0000 7ff76bfff640 1 -- 172.21.15.38:0/3832659953 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff7640108c0 con 0x7ff7740b2500 2024-09-21T03:34:03.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:03.987+0000 7ff76bfff640 1 -- 172.21.15.38:0/3832659953 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff764010d10 con 0x7ff7740b2500 2024-09-21T03:34:03.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:03.987+0000 7ff779871640 1 -- 172.21.15.38:0/3832659953 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7740cbdd0 con 0x7ff7740b2500 2024-09-21T03:34:03.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:03.987+0000 7ff779871640 1 -- 172.21.15.38:0/3832659953 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7741f4660 con 0x7ff7740b2500 2024-09-21T03:34:03.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:03.988+0000 7ff779871640 1 -- 172.21.15.38:0/3832659953 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff77406ac50 con 0x7ff7740b2500 2024-09-21T03:34:03.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:03.989+0000 7ff76bfff640 1 -- 172.21.15.38:0/3832659953 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ff764018420 con 0x7ff7740b2500 2024-09-21T03:34:03.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:03.990+0000 7ff76bfff640 1 --2- 172.21.15.38:0/3832659953 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff754078bc0 0x7ff75407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:03.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:03.990+0000 7ff76bfff640 1 -- 172.21.15.38:0/3832659953 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7ff76409bd30 con 0x7ff7740b2500 2024-09-21T03:34:03.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:03.990+0000 7ff7727fc640 1 --2- 172.21.15.38:0/3832659953 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff754078bc0 0x7ff75407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:03.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:03.991+0000 7ff7727fc640 1 --2- 172.21.15.38:0/3832659953 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff754078bc0 0x7ff75407b0d0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ff7740a9a90 tx=0x7ff75c009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:03.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:03.994+0000 7ff76bfff640 1 -- 172.21.15.38:0/3832659953 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7ff77406ac50 con 0x7ff7740b2500 2024-09-21T03:34:04.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:04.185+0000 7ff779871640 1 -- 172.21.15.38:0/3832659953 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff7740b3a10 con 0x7ff7740b2500 2024-09-21T03:34:04.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:04.186+0000 7ff76bfff640 1 -- 172.21.15.38:0/3832659953 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18626 (secure 0 0 0) 0x7ff7740b3a10 con 0x7ff7740b2500 2024-09-21T03:34:04.188 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:34:04.189 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":26,"fsid":"9d50d66f-656b-4fe2-8e3b-fe53b79c4d43","created":"2024-09-21T03:33:10.677865+0000","modified":"2024-09-21T03:33:40.665991+0000","last_up_change":"2024-09-21T03:33:24.023421+0000","last_in_change":"2024-09-21T03:33:14.296193+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T03:33:22.992573+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T03:33:33.530201+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T03:33:34.585875+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T03:33:38.268346+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"dbab183f-7e63-48b8-960a-ae55fd5bf565","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6801","nonce":2381639433}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6803","nonce":2381639433}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6807","nonce":2381639433}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6805","nonce":2381639433}]},"public_addr":"172.21.15.38:6801/2381639433","cluster_addr":"172.21.15.38:6803/2381639433","heartbeat_back_addr":"172.21.15.38:6807/2381639433","heartbeat_front_addr":"172.21.15.38:6805/2381639433","state":["exists","up"]},{"osd":1,"uuid":"fa1c0e8a-ebc1-4359-8146-bd5342abbc16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6817","nonce":1491818732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6819","nonce":1491818732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6823","nonce":1491818732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6821","nonce":1491818732}]},"public_addr":"172.21.15.38:6817/1491818732","cluster_addr":"172.21.15.38:6819/1491818732","heartbeat_back_addr":"172.21.15.38:6823/1491818732","heartbeat_front_addr":"172.21.15.38:6821/1491818732","state":["exists","up"]},{"osd":2,"uuid":"2ed3cd2d-385d-4fc6-9869-214552a84803","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6825","nonce":2577858902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6827","nonce":2577858902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6831","nonce":2577858902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6829","nonce":2577858902}]},"public_addr":"172.21.15.38:6825/2577858902","cluster_addr":"172.21.15.38:6827/2577858902","heartbeat_back_addr":"172.21.15.38:6831/2577858902","heartbeat_front_addr":"172.21.15.38:6829/2577858902","state":["exists","up"]},{"osd":3,"uuid":"62d1d95e-919d-4250-be71-1d34cfa70dbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6809","nonce":1286810294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6811","nonce":1286810294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6815","nonce":1286810294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6813","nonce":1286810294}]},"public_addr":"172.21.15.38:6809/1286810294","cluster_addr":"172.21.15.38:6811/1286810294","heartbeat_back_addr":"172.21.15.38:6815/1286810294","heartbeat_front_addr":"172.21.15.38:6813/1286810294","state":["exists","up"]},{"osd":4,"uuid":"7c9f9314-99d7-4d73-8392-37809a9f54b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6803","nonce":3401837571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6805","nonce":3401837571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6809","nonce":3401837571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6807","nonce":3401837571}]},"public_addr":"172.21.15.195:6803/3401837571","cluster_addr":"172.21.15.195:6805/3401837571","heartbeat_back_addr":"172.21.15.195:6809/3401837571","heartbeat_front_addr":"172.21.15.195:6807/3401837571","state":["exists","up"]},{"osd":5,"uuid":"bde4fe9f-8414-4bba-bf4a-43973d1976bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6811","nonce":2080960180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6813","nonce":2080960180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6817","nonce":2080960180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6815","nonce":2080960180}]},"public_addr":"172.21.15.195:6811/2080960180","cluster_addr":"172.21.15.195:6813/2080960180","heartbeat_back_addr":"172.21.15.195:6817/2080960180","heartbeat_front_addr":"172.21.15.195:6815/2080960180","state":["exists","up"]},{"osd":6,"uuid":"6ee6d7f4-1a5f-4a4c-ac16-48fe05f6a002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6827","nonce":4055825675}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6829","nonce":4055825675}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6833","nonce":4055825675}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6831","nonce":4055825675}]},"public_addr":"172.21.15.195:6827/4055825675","cluster_addr":"172.21.15.195:6829/4055825675","heartbeat_back_addr":"172.21.15.195:6833/4055825675","heartbeat_front_addr":"172.21.15.195:6831/4055825675","state":["exists","up"]},{"osd":7,"uuid":"efeb5f51-329a-4484-8290-92e7d3503d05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6819","nonce":1606106342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6821","nonce":1606106342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6825","nonce":1606106342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6823","nonce":1606106342}]},"public_addr":"172.21.15.195:6819/1606106342","cluster_addr":"172.21.15.195:6821/1606106342","heartbeat_back_addr":"172.21.15.195:6825/1606106342","heartbeat_front_addr":"172.21.15.195:6823/1606106342","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.901343+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:21.115003+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.901623+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.835238+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.132199+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:19.817560+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.829690+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:21.087300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T03:34:04.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:04.188+0000 7ff779871640 1 -- 172.21.15.38:0/3832659953 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff754078bc0 msgr2=0x7ff75407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:04.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:04.188+0000 7ff779871640 1 --2- 172.21.15.38:0/3832659953 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff754078bc0 0x7ff75407b0d0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ff7740a9a90 tx=0x7ff75c009040 comp rx=0 tx=0).stop 2024-09-21T03:34:04.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:04.188+0000 7ff779871640 1 -- 172.21.15.38:0/3832659953 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff7740b2500 msgr2=0x7ff7740c8b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:04.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:04.188+0000 7ff779871640 1 --2- 172.21.15.38:0/3832659953 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff7740b2500 0x7ff7740c8b20 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ff76400eb60 tx=0x7ff76400c6a0 comp rx=0 tx=0).stop 2024-09-21T03:34:04.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:04.189+0000 7ff779871640 1 -- 172.21.15.38:0/3832659953 shutdown_connections 2024-09-21T03:34:04.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:04.189+0000 7ff779871640 1 --2- 172.21.15.38:0/3832659953 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff754078bc0 0x7ff75407b0d0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:04.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:04.189+0000 7ff779871640 1 --2- 172.21.15.38:0/3832659953 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff7740cb350 0x7ff7741f66a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:04.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:04.189+0000 7ff779871640 1 --2- 172.21.15.38:0/3832659953 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff7740d19d0 0x7ff7740cae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:04.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:04.189+0000 7ff779871640 1 --2- 172.21.15.38:0/3832659953 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff7740b2500 0x7ff7740c8b20 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:04.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:04.189+0000 7ff779871640 1 -- 172.21.15.38:0/3832659953 >> 172.21.15.38:0/3832659953 conn(0x7ff7740b41d0 msgr2=0x7ff7740701e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:04.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:04.189+0000 7ff779871640 1 -- 172.21.15.38:0/3832659953 shutdown_connections 2024-09-21T03:34:04.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:04.189+0000 7ff779871640 1 -- 172.21.15.38:0/3832659953 wait complete. 2024-09-21T03:34:04.202 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:34:04.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:04.320+0000 7fa0276de640 1 -- 172.21.15.38:0/3852644932 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa0200b2e40 msgr2=0x7fa0200b3250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:04.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:04.320+0000 7fa0276de640 1 --2- 172.21.15.38:0/3852644932 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa0200b2e40 0x7fa0200b3250 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa018005fd0 tx=0x7fa01802aa90 comp rx=0 tx=0).stop 2024-09-21T03:34:04.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:04.320+0000 7fa0276de640 1 -- 172.21.15.38:0/3852644932 shutdown_connections 2024-09-21T03:34:04.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:04.320+0000 7fa0276de640 1 --2- 172.21.15.38:0/3852644932 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa0200b2e40 0x7fa0200b3250 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:04.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:04.320+0000 7fa0276de640 1 --2- 172.21.15.38:0/3852644932 >> v2:172.21.15.195:3300/0 conn(0x7fa020162280 0x7fa020162690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:04.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:04.320+0000 7fa0276de640 1 -- 172.21.15.38:0/3852644932 >> 172.21.15.38:0/3852644932 conn(0x7fa0200b6b20 msgr2=0x7fa0200d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:04.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:04.320+0000 7fa0276de640 1 -- 172.21.15.38:0/3852644932 shutdown_connections 2024-09-21T03:34:04.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:04.320+0000 7fa0276de640 1 -- 172.21.15.38:0/3852644932 wait complete. 2024-09-21T03:34:04.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:04.322+0000 7fa0276de640 1 Processor -- start 2024-09-21T03:34:04.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:04.322+0000 7fa0276de640 1 -- start start 2024-09-21T03:34:04.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:04.323+0000 7fa0276de640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa0200b2e40 0x7fa0200afaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:04.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:04.323+0000 7fa0276de640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa020162280 0x7fa0201f2640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:04.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:04.323+0000 7fa0276de640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa0200b2900 0x7fa0201f5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:04.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:04.323+0000 7fa0276de640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa02015e4b0 con 0x7fa020162280 2024-09-21T03:34:04.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:04.323+0000 7fa0276de640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fa0200b39a0 con 0x7fa0200b2900 2024-09-21T03:34:04.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:04.323+0000 7fa0276de640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fa02015e630 con 0x7fa0200b2e40 2024-09-21T03:34:04.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:04.323+0000 7fa024c79640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa020162280 0x7fa0201f2640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:04.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:04.323+0000 7fa024c79640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa020162280 0x7fa0201f2640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42946/0 (socket says 172.21.15.38:42946) 2024-09-21T03:34:04.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:04.323+0000 7fa02547a640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa0200b2e40 0x7fa0200afaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:04.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:04.323+0000 7fa024c79640 1 -- 172.21.15.38:0/4037496706 learned_addr learned my addr 172.21.15.38:0/4037496706 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:34:04.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:04.323+0000 7fa025c7b640 1 --2- 172.21.15.38:0/4037496706 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa0200b2900 0x7fa0201f5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:04.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:04.324+0000 7fa024c79640 1 -- 172.21.15.38:0/4037496706 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa0200b2e40 msgr2=0x7fa0200afaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:04.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:04.324+0000 7fa024c79640 1 --2- 172.21.15.38:0/4037496706 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa0200b2e40 0x7fa0200afaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:04.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:04.324+0000 7fa024c79640 1 -- 172.21.15.38:0/4037496706 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa0200b2900 msgr2=0x7fa0201f5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:04.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:04.324+0000 7fa024c79640 1 --2- 172.21.15.38:0/4037496706 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa0200b2900 0x7fa0201f5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:04.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:04.324+0000 7fa024c79640 1 -- 172.21.15.38:0/4037496706 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0200b1860 con 0x7fa020162280 2024-09-21T03:34:04.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:04.324+0000 7fa024c79640 1 --2- 172.21.15.38:0/4037496706 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa020162280 0x7fa0201f2640 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fa01000e9f0 tx=0x7fa01000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:04.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:04.324+0000 7fa015ffb640 1 -- 172.21.15.38:0/4037496706 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa01000cd80 con 0x7fa020162280 2024-09-21T03:34:04.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:04.324+0000 7fa015ffb640 1 -- 172.21.15.38:0/4037496706 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa010004020 con 0x7fa020162280 2024-09-21T03:34:04.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:04.324+0000 7fa0276de640 1 -- 172.21.15.38:0/4037496706 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0200b1b40 con 0x7fa020162280 2024-09-21T03:34:04.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:04.324+0000 7fa015ffb640 1 -- 172.21.15.38:0/4037496706 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa010004470 con 0x7fa020162280 2024-09-21T03:34:04.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:04.324+0000 7fa0276de640 1 -- 172.21.15.38:0/4037496706 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0200b2100 con 0x7fa020162280 2024-09-21T03:34:04.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:04.325+0000 7fa0276de640 1 -- 172.21.15.38:0/4037496706 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0200b0180 con 0x7fa020162280 2024-09-21T03:34:04.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:04.326+0000 7fa015ffb640 1 -- 172.21.15.38:0/4037496706 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa0100035f0 con 0x7fa020162280 2024-09-21T03:34:04.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:04.326+0000 7fa015ffb640 1 --2- 172.21.15.38:0/4037496706 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa004078b90 0x7fa00407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:04.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:04.327+0000 7fa015ffb640 1 -- 172.21.15.38:0/4037496706 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fa01009b9f0 con 0x7fa020162280 2024-09-21T03:34:04.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:04.327+0000 7fa02547a640 1 --2- 172.21.15.38:0/4037496706 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa004078b90 0x7fa00407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:04.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:04.327+0000 7fa02547a640 1 --2- 172.21.15.38:0/4037496706 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa004078b90 0x7fa00407b0a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa01c0017e0 tx=0x7fa01c009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:04.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:04.331+0000 7fa015ffb640 1 -- 172.21.15.38:0/4037496706 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fa010063530 con 0x7fa020162280 2024-09-21T03:34:04.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:04.559+0000 7fa0276de640 1 -- 172.21.15.38:0/4037496706 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa020162f00 con 0x7fa020162280 2024-09-21T03:34:04.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:04.559+0000 7fa015ffb640 1 -- 172.21.15.38:0/4037496706 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5219 (secure 0 0 0) 0x7fa010067f60 con 0x7fa020162280 2024-09-21T03:34:04.562 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:34:04.562 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":9,"btime":"2024-09-21T03:33:56:329606+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4322,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/593838590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":593838590},{"type":"v1","addr":"172.21.15.195:6836","nonce":593838590}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4374,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/663402481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":663402481},{"type":"v1","addr":"172.21.15.38:6833","nonce":663402481}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4377,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/2614829035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":2614829035},{"type":"v1","addr":"172.21.15.38:6835","nonce":2614829035}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:33:35.622623+0000","modified":"2024-09-21T03:33:55.704494+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4402},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4402":{"gid":4402,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.195:6837/1420805977","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":1420805977},{"type":"v1","addr":"172.21.15.195:6837","nonce":1420805977}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4402,"qdb_cluster":[4402]},"id":1}]} 2024-09-21T03:34:04.562 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 9 2024-09-21T03:34:04.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:04.562+0000 7fa0276de640 1 -- 172.21.15.38:0/4037496706 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa004078b90 msgr2=0x7fa00407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:04.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:04.562+0000 7fa0276de640 1 --2- 172.21.15.38:0/4037496706 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa004078b90 0x7fa00407b0a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa01c0017e0 tx=0x7fa01c009040 comp rx=0 tx=0).stop 2024-09-21T03:34:04.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:04.562+0000 7fa0276de640 1 -- 172.21.15.38:0/4037496706 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa020162280 msgr2=0x7fa0201f2640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:04.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:04.562+0000 7fa0276de640 1 --2- 172.21.15.38:0/4037496706 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa020162280 0x7fa0201f2640 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fa01000e9f0 tx=0x7fa01000eeb0 comp rx=0 tx=0).stop 2024-09-21T03:34:04.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:04.562+0000 7fa0276de640 1 -- 172.21.15.38:0/4037496706 shutdown_connections 2024-09-21T03:34:04.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:04.562+0000 7fa0276de640 1 --2- 172.21.15.38:0/4037496706 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa0200b2900 0x7fa0201f5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:04.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:04.562+0000 7fa0276de640 1 --2- 172.21.15.38:0/4037496706 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa020162280 0x7fa0201f2640 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:04.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:04.562+0000 7fa0276de640 1 --2- 172.21.15.38:0/4037496706 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa004078b90 0x7fa00407b0a0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:04.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:04.562+0000 7fa0276de640 1 --2- 172.21.15.38:0/4037496706 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa0200b2e40 0x7fa0200afaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:04.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:04.562+0000 7fa0276de640 1 -- 172.21.15.38:0/4037496706 >> 172.21.15.38:0/4037496706 conn(0x7fa0200b6b20 msgr2=0x7fa02006cec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:04.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:04.562+0000 7fa0276de640 1 -- 172.21.15.38:0/4037496706 shutdown_connections 2024-09-21T03:34:04.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:04.562+0000 7fa0276de640 1 -- 172.21.15.38:0/4037496706 wait complete. 2024-09-21T03:34:04.978 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 9, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T03:33:35.622623+0000', 'modified': '2024-09-21T03:33:55.704494+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4402}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4402': {'gid': 4402, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.195:6837/1420805977', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6835', 'nonce': 1420805977}, {'type': 'v1', 'addr': '172.21.15.195:6837', 'nonce': 1420805977}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4402, 'qdb_cluster': [4402]} 2024-09-21T03:34:04.978 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-21T03:34:04.978 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-09-21T03:34:05.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.098+0000 7fdfb05df640 1 -- 172.21.15.38:0/693683661 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdfa815ee80 msgr2=0x7fdfa80d0170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:05.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.098+0000 7fdfb05df640 1 --2- 172.21.15.38:0/693683661 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdfa815ee80 0x7fdfa80d0170 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fdfa80b3c20 tx=0x7fdf9802a790 comp rx=0 tx=0).stop 2024-09-21T03:34:05.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.098+0000 7fdfb05df640 1 -- 172.21.15.38:0/693683661 shutdown_connections 2024-09-21T03:34:05.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.098+0000 7fdfb05df640 1 --2- 172.21.15.38:0/693683661 >> v2:172.21.15.195:3300/0 conn(0x7fdfa80ceb90 0x7fdfa80cefa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:05.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.098+0000 7fdfb05df640 1 --2- 172.21.15.38:0/693683661 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdfa815ee80 0x7fdfa80d0170 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:05.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.098+0000 7fdfb05df640 1 -- 172.21.15.38:0/693683661 >> 172.21.15.38:0/693683661 conn(0x7fdfa809d830 msgr2=0x7fdfa80b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:05.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.099+0000 7fdfb05df640 1 -- 172.21.15.38:0/693683661 shutdown_connections 2024-09-21T03:34:05.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.099+0000 7fdfb05df640 1 -- 172.21.15.38:0/693683661 wait complete. 2024-09-21T03:34:05.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.100+0000 7fdfb05df640 1 Processor -- start 2024-09-21T03:34:05.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.100+0000 7fdfb05df640 1 -- start start 2024-09-21T03:34:05.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.102+0000 7fdfb05df640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdfa80b9ad0 0x7fdfa81f8c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:05.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.102+0000 7fdfb05df640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdfa80ceb90 0x7fdfa8158540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:05.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.102+0000 7fdfb05df640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdfa815ee80 0x7fdfa806cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:05.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.102+0000 7fdfb05df640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fdfa80b6580 con 0x7fdfa815ee80 2024-09-21T03:34:05.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.102+0000 7fdfb05df640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fdfa80ba0f0 con 0x7fdfa80b9ad0 2024-09-21T03:34:05.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.102+0000 7fdfb05df640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fdfa80b6700 con 0x7fdfa80ceb90 2024-09-21T03:34:05.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.102+0000 7fdfaeb7c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdfa815ee80 0x7fdfa806cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:05.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.102+0000 7fdfaeb7c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdfa815ee80 0x7fdfa806cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42960/0 (socket says 172.21.15.38:42960) 2024-09-21T03:34:05.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.102+0000 7fdfaeb7c640 1 -- 172.21.15.38:0/3468791659 learned_addr learned my addr 172.21.15.38:0/3468791659 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:34:05.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.102+0000 7fdfae37b640 1 --2- 172.21.15.38:0/3468791659 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdfa80b9ad0 0x7fdfa81f8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:05.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.102+0000 7fdfadb7a640 1 --2- 172.21.15.38:0/3468791659 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdfa80ceb90 0x7fdfa8158540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:05.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.102+0000 7fdfaeb7c640 1 -- 172.21.15.38:0/3468791659 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdfa80ceb90 msgr2=0x7fdfa8158540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:05.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.102+0000 7fdfaeb7c640 1 --2- 172.21.15.38:0/3468791659 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdfa80ceb90 0x7fdfa8158540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:05.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.102+0000 7fdfaeb7c640 1 -- 172.21.15.38:0/3468791659 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdfa80b9ad0 msgr2=0x7fdfa81f8c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:05.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.102+0000 7fdfaeb7c640 1 --2- 172.21.15.38:0/3468791659 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdfa80b9ad0 0x7fdfa81f8c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:05.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.102+0000 7fdfaeb7c640 1 -- 172.21.15.38:0/3468791659 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfa8158a80 con 0x7fdfa815ee80 2024-09-21T03:34:05.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.102+0000 7fdfaeb7c640 1 --2- 172.21.15.38:0/3468791659 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdfa815ee80 0x7fdfa806cde0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fdfa0005d00 tx=0x7fdfa000dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:05.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.103+0000 7fdfa6ffd640 1 -- 172.21.15.38:0/3468791659 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfa0017070 con 0x7fdfa815ee80 2024-09-21T03:34:05.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.103+0000 7fdfa6ffd640 1 -- 172.21.15.38:0/3468791659 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdfa0005330 con 0x7fdfa815ee80 2024-09-21T03:34:05.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.103+0000 7fdfa6ffd640 1 -- 172.21.15.38:0/3468791659 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfa00047a0 con 0x7fdfa815ee80 2024-09-21T03:34:05.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.103+0000 7fdfb05df640 1 -- 172.21.15.38:0/3468791659 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfa81f9b40 con 0x7fdfa815ee80 2024-09-21T03:34:05.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.103+0000 7fdfb05df640 1 -- 172.21.15.38:0/3468791659 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfa81f9860 con 0x7fdfa815ee80 2024-09-21T03:34:05.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.104+0000 7fdfa6ffd640 1 -- 172.21.15.38:0/3468791659 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fdfa0004940 con 0x7fdfa815ee80 2024-09-21T03:34:05.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.105+0000 7fdfa6ffd640 1 --2- 172.21.15.38:0/3468791659 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fdf94078b90 0x7fdf9407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:05.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.105+0000 7fdfa6ffd640 1 -- 172.21.15.38:0/3468791659 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fdfa009b110 con 0x7fdfa815ee80 2024-09-21T03:34:05.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.105+0000 7fdfae37b640 1 --2- 172.21.15.38:0/3468791659 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fdf94078b90 0x7fdf9407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:05.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.106+0000 7fdfae37b640 1 --2- 172.21.15.38:0/3468791659 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fdf94078b90 0x7fdf9407b0a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fdf98033000 tx=0x7fdf98004ab0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:05.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.106+0000 7fdfb7625740 1 Processor -- start 2024-09-21T03:34:05.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.106+0000 7fdfb7625740 1 -- start start 2024-09-21T03:34:05.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.108+0000 7fdfb7625740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b6c54ac270 0x55b6c587dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:05.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.108+0000 7fdfb7625740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b6c54833b0 0x55b6c587e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:05.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.108+0000 7fdfb7625740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b6c548b590 0x55b6c589b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:05.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.108+0000 7fdfb7625740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55b6c57c6820 con 0x55b6c54833b0 2024-09-21T03:34:05.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.108+0000 7fdfb7625740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55b6c57b5c50 con 0x55b6c54ac270 2024-09-21T03:34:05.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.108+0000 7fdfb7625740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55b6c57b9a60 con 0x55b6c548b590 2024-09-21T03:34:05.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.108+0000 7fdfadb7a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b6c54833b0 0x55b6c587e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:05.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.108+0000 7fdfadb7a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b6c54833b0 0x55b6c587e4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42964/0 (socket says 172.21.15.38:42964) 2024-09-21T03:34:05.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.108+0000 7fdfadb7a640 1 -- 172.21.15.38:0/741986221 learned_addr learned my addr 172.21.15.38:0/741986221 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:34:05.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.108+0000 7fdfaeb7c640 1 --2- 172.21.15.38:0/741986221 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b6c548b590 0x55b6c589b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:05.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.108+0000 7fdfae37b640 1 --2- 172.21.15.38:0/741986221 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b6c54ac270 0x55b6c587dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:05.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.108+0000 7fdfaeb7c640 1 -- 172.21.15.38:0/741986221 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b6c54ac270 msgr2=0x55b6c587dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:05.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.108+0000 7fdfaeb7c640 1 --2- 172.21.15.38:0/741986221 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b6c54ac270 0x55b6c587dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:05.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.108+0000 7fdfaeb7c640 1 -- 172.21.15.38:0/741986221 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b6c54833b0 msgr2=0x55b6c587e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:05.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.108+0000 7fdfaeb7c640 1 --2- 172.21.15.38:0/741986221 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b6c54833b0 0x55b6c587e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:05.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.108+0000 7fdfaeb7c640 1 -- 172.21.15.38:0/741986221 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6c5517850 con 0x55b6c548b590 2024-09-21T03:34:05.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.108+0000 7fdfadb7a640 1 --2- 172.21.15.38:0/741986221 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b6c54833b0 0x55b6c587e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:34:05.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.108+0000 7fdfaeb7c640 1 --2- 172.21.15.38:0/741986221 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b6c548b590 0x55b6c589b3e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fdfa00628b0 tx=0x7fdfa005eb80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e7ba2e5d41a9278 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:05.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.108+0000 7fdf8ffff640 1 -- 172.21.15.38:0/741986221 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfa0020040 con 0x55b6c548b590 2024-09-21T03:34:05.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.109+0000 7fdf8ffff640 1 -- 172.21.15.38:0/741986221 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdfa00675e0 con 0x55b6c548b590 2024-09-21T03:34:05.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.109+0000 7fdf8ffff640 1 -- 172.21.15.38:0/741986221 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfa006c260 con 0x55b6c548b590 2024-09-21T03:34:05.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.109+0000 7fdfb7625740 1 -- 172.21.15.38:0/741986221 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b6c548b590 msgr2=0x55b6c589b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:05.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.109+0000 7fdfb7625740 1 --2- 172.21.15.38:0/741986221 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b6c548b590 0x55b6c589b3e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fdfa00628b0 tx=0x7fdfa005eb80 comp rx=0 tx=0).stop 2024-09-21T03:34:05.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.109+0000 7fdfb7625740 1 -- 172.21.15.38:0/741986221 shutdown_connections 2024-09-21T03:34:05.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.109+0000 7fdfb7625740 1 --2- 172.21.15.38:0/741986221 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b6c548b590 0x55b6c589b3e0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:05.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.109+0000 7fdfb7625740 1 --2- 172.21.15.38:0/741986221 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b6c54833b0 0x55b6c587e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:05.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.109+0000 7fdfb7625740 1 --2- 172.21.15.38:0/741986221 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b6c54ac270 0x55b6c587dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:05.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.109+0000 7fdfb7625740 1 -- 172.21.15.38:0/741986221 >> 172.21.15.38:0/741986221 conn(0x55b6c54a03c0 msgr2=0x55b6c5854260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:05.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.110+0000 7fdfb7625740 1 -- 172.21.15.38:0/741986221 shutdown_connections 2024-09-21T03:34:05.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.110+0000 7fdfb7625740 1 -- 172.21.15.38:0/741986221 wait complete. 2024-09-21T03:34:05.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.110+0000 7fdfb7625740 1 -- start start 2024-09-21T03:34:05.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.110+0000 7fdfb7625740 1 Processor -- start 2024-09-21T03:34:05.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.112+0000 7fdfb7625740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b6c54833b0 0x55b6c5893610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:05.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.112+0000 7fdfb7625740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b6c548b590 0x55b6c586c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:05.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.112+0000 7fdfb7625740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b6c54ac270 0x55b6c5852e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:05.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.112+0000 7fdfb7625740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55b6c5526620 con 0x55b6c54833b0 2024-09-21T03:34:05.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.112+0000 7fdfb7625740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55b6c575b5f0 con 0x55b6c548b590 2024-09-21T03:34:05.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.112+0000 7fdfb7625740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55b6c587ea20 con 0x55b6c54ac270 2024-09-21T03:34:05.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.112+0000 7fdfb7625740 10 client.0 _ll_register_callbacks cb 0x55b6c575f0a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:34:05.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.112+0000 7fdfae37b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b6c54833b0 0x55b6c5893610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:05.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.112+0000 7fdfae37b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b6c54833b0 0x55b6c5893610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42980/0 (socket says 172.21.15.38:42980) 2024-09-21T03:34:05.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.112+0000 7fdfae37b640 1 -- 172.21.15.38:0/678957375 learned_addr learned my addr 172.21.15.38:0/678957375 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:34:05.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.112+0000 7fdfaeb7c640 1 --2- 172.21.15.38:0/678957375 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b6c54ac270 0x55b6c5852e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:05.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.112+0000 7fdfadb7a640 1 --2- 172.21.15.38:0/678957375 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b6c548b590 0x55b6c586c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:05.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.112+0000 7fdfae37b640 1 -- 172.21.15.38:0/678957375 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b6c54ac270 msgr2=0x55b6c5852e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:05.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.112+0000 7fdfae37b640 1 --2- 172.21.15.38:0/678957375 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b6c54ac270 0x55b6c5852e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:05.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.112+0000 7fdfae37b640 1 -- 172.21.15.38:0/678957375 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b6c548b590 msgr2=0x55b6c586c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:05.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.112+0000 7fdfae37b640 1 --2- 172.21.15.38:0/678957375 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b6c548b590 0x55b6c586c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:05.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.112+0000 7fdfae37b640 1 -- 172.21.15.38:0/678957375 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6c558e6b0 con 0x55b6c54833b0 2024-09-21T03:34:05.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.112+0000 7fdfadb7a640 1 --2- 172.21.15.38:0/678957375 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b6c548b590 0x55b6c586c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:34:05.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.113+0000 7fdfae37b640 1 --2- 172.21.15.38:0/678957375 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b6c54833b0 0x55b6c5893610 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fdf98003e90 tx=0x7fdf98009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=167c2f880c3c86d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:05.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.113+0000 7fdf8d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-21T03:34:05.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.113+0000 7fdf8d7fa640 1 -- 172.21.15.38:0/678957375 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf9804f070 con 0x55b6c54833b0 2024-09-21T03:34:05.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.113+0000 7fdf8d7fa640 1 -- 172.21.15.38:0/678957375 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fdf98004240 con 0x55b6c54833b0 2024-09-21T03:34:05.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.113+0000 7fdfb7625740 1 -- 172.21.15.38:0/678957375 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdfa00675e0 con 0x55b6c54833b0 2024-09-21T03:34:05.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.113+0000 7fdf8d7fa640 1 -- 172.21.15.38:0/678957375 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) ==== 24+0+0 (secure 0 0 0) 0x7fdf9805a050 con 0x55b6c54833b0 2024-09-21T03:34:05.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.113+0000 7fdfb7625740 10 client.4794 fetch_fsmap learned FSMap version 9 2024-09-21T03:34:05.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.113+0000 7fdfb7625740 1 -- 172.21.15.38:0/678957375 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) -- 0x55b6c5894620 con 0x55b6c54833b0 2024-09-21T03:34:05.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.113+0000 7fdf8d7fa640 1 -- 172.21.15.38:0/678957375 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdfa00675e0 con 0x55b6c54833b0 2024-09-21T03:34:05.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.113+0000 7fdf8d7fa640 1 -- 172.21.15.38:0/678957375 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 9) ==== 3510+0+0 (secure 0 0 0) 0x7fdf9805d020 con 0x55b6c54833b0 2024-09-21T03:34:05.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.113+0000 7fdfb7625740 10 client.4794 fetch_fsmap finished waiting for FSMap version 9 2024-09-21T03:34:05.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.113+0000 7fdfb7625740 10 client.4794 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-21T03:34:05.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.113+0000 7fdfb7625740 20 client.4794 populate_metadata read hostname 'smithi038' 2024-09-21T03:34:05.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.113+0000 7fdfb7625740 1 --2- 172.21.15.38:0/678957375 >> [v2:172.21.15.195:6835/1420805977,v1:172.21.15.195:6837/1420805977] conn(0x55b6c546ae40 0x55b6c58ab5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:05.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.113+0000 7fdfb7625740 4 client.4794 mds_command: new command op to 4402 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:34:05.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.113+0000 7fdfb7625740 1 -- 172.21.15.38:0/678957375 --> [v2:172.21.15.195:6835/1420805977,v1:172.21.15.195:6837/1420805977] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b6c57c8770 con 0x55b6c546ae40 2024-09-21T03:34:05.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.114+0000 7fdfadb7a640 1 --2- 172.21.15.38:0/678957375 >> [v2:172.21.15.195:6835/1420805977,v1:172.21.15.195:6837/1420805977] conn(0x55b6c546ae40 0x55b6c58ab5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:05.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.114+0000 7fdf8d7fa640 10 client.4794 ms_handle_connect on v2:172.21.15.195:6835/1420805977 2024-09-21T03:34:05.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.114+0000 7fdfadb7a640 1 --2- 172.21.15.38:0/678957375 >> [v2:172.21.15.195:6835/1420805977,v1:172.21.15.195:6837/1420805977] conn(0x55b6c546ae40 0x55b6c58ab5b0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e373b856f6cb8998 server_cookie=29e447beecb4287b in_seq=0 out_seq=0 2024-09-21T03:34:05.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.115+0000 7fdf8d7fa640 1 -- 172.21.15.38:0/678957375 <== mds.0 v2:172.21.15.195:6835/1420805977 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b6c57c8770 con 0x55b6c546ae40 2024-09-21T03:34:05.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.115+0000 7fdf8d7fa640 10 client.4794 handle_command_reply: tid=0 2024-09-21T03:34:05.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.115+0000 7fdfb7625740 1 client.4794 shutdown 2024-09-21T03:34:05.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.115+0000 7fdfb7625740 1 -- 172.21.15.38:0/678957375 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b6c54833b0 msgr2=0x55b6c5893610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:05.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.115+0000 7fdfb7625740 1 --2- 172.21.15.38:0/678957375 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b6c54833b0 0x55b6c5893610 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fdf98003e90 tx=0x7fdf98009500 comp rx=0 tx=0).stop 2024-09-21T03:34:05.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.115+0000 7fdfb7625740 1 -- 172.21.15.38:0/678957375 shutdown_connections 2024-09-21T03:34:05.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.115+0000 7fdfb7625740 1 --2- 172.21.15.38:0/678957375 >> [v2:172.21.15.195:6835/1420805977,v1:172.21.15.195:6837/1420805977] conn(0x55b6c546ae40 0x55b6c58ab5b0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:05.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.115+0000 7fdfb7625740 1 --2- 172.21.15.38:0/678957375 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b6c54ac270 0x55b6c5852e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:05.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.115+0000 7fdfb7625740 1 --2- 172.21.15.38:0/678957375 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b6c548b590 0x55b6c586c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:05.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.115+0000 7fdf8d7fa640 0 client.4794 ms_handle_reset on v2:172.21.15.195:6835/1420805977 2024-09-21T03:34:05.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.115+0000 7fdfb7625740 1 --2- 172.21.15.38:0/678957375 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b6c54833b0 0x55b6c5893610 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:05.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.115+0000 7fdfb7625740 1 -- 172.21.15.38:0/678957375 >> 172.21.15.38:0/678957375 conn(0x55b6c54a03c0 msgr2=0x55b6c5854110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:05.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.115+0000 7fdfb7625740 1 -- 172.21.15.38:0/678957375 shutdown_connections 2024-09-21T03:34:05.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.115+0000 7fdfb7625740 1 -- 172.21.15.38:0/678957375 wait complete. 2024-09-21T03:34:05.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.116+0000 7fdfb7625740 20 client.4794 trim_cache size 0 max 16384 2024-09-21T03:34:05.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.134+0000 7fdfb7625740 1 Processor -- start 2024-09-21T03:34:05.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.134+0000 7fdfb7625740 1 -- start start 2024-09-21T03:34:05.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.135+0000 7fdfb7625740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b6c546ae40 0x55b6c5893950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:05.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.135+0000 7fdfb7625740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b6c54833b0 0x55b6c588eee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:05.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.135+0000 7fdfb7625740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b6c548b590 0x55b6c588f420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:05.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.135+0000 7fdfb7625740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fdfa82026f0 con 0x55b6c546ae40 2024-09-21T03:34:05.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.135+0000 7fdfb7625740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fdfa80cc0d0 con 0x55b6c54833b0 2024-09-21T03:34:05.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.135+0000 7fdfb7625740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fdfa82020a0 con 0x55b6c548b590 2024-09-21T03:34:05.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.135+0000 7fdfae37b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b6c546ae40 0x55b6c5893950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:05.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.135+0000 7fdfae37b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b6c546ae40 0x55b6c5893950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42988/0 (socket says 172.21.15.38:42988) 2024-09-21T03:34:05.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.135+0000 7fdfae37b640 1 -- 172.21.15.38:0/1202455696 learned_addr learned my addr 172.21.15.38:0/1202455696 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:34:05.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.136+0000 7fdfadb7a640 1 --2- 172.21.15.38:0/1202455696 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b6c54833b0 0x55b6c588eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:05.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.136+0000 7fdfaeb7c640 1 --2- 172.21.15.38:0/1202455696 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b6c548b590 0x55b6c588f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:05.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.136+0000 7fdfae37b640 1 -- 172.21.15.38:0/1202455696 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b6c548b590 msgr2=0x55b6c588f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:05.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.136+0000 7fdfae37b640 1 --2- 172.21.15.38:0/1202455696 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b6c548b590 0x55b6c588f420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:05.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.136+0000 7fdfae37b640 1 -- 172.21.15.38:0/1202455696 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b6c54833b0 msgr2=0x55b6c588eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:05.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.136+0000 7fdfae37b640 1 --2- 172.21.15.38:0/1202455696 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b6c54833b0 0x55b6c588eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:05.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.136+0000 7fdfae37b640 1 -- 172.21.15.38:0/1202455696 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6c58a8f00 con 0x55b6c546ae40 2024-09-21T03:34:05.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.136+0000 7fdfae37b640 1 --2- 172.21.15.38:0/1202455696 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b6c546ae40 0x55b6c5893950 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fdf9802ad80 tx=0x7fdf9802af10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=433d2270384c8ff1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:05.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.136+0000 7fdf8f7fe640 1 -- 172.21.15.38:0/1202455696 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf98062070 con 0x55b6c546ae40 2024-09-21T03:34:05.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.136+0000 7fdf8f7fe640 1 -- 172.21.15.38:0/1202455696 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdf98004270 con 0x55b6c546ae40 2024-09-21T03:34:05.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.136+0000 7fdf8f7fe640 1 -- 172.21.15.38:0/1202455696 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf9804f040 con 0x55b6c546ae40 2024-09-21T03:34:05.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.137+0000 7fdfb7625740 1 -- 172.21.15.38:0/1202455696 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b6c546ae40 msgr2=0x55b6c5893950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:05.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.137+0000 7fdfb7625740 1 --2- 172.21.15.38:0/1202455696 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b6c546ae40 0x55b6c5893950 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fdf9802ad80 tx=0x7fdf9802af10 comp rx=0 tx=0).stop 2024-09-21T03:34:05.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.137+0000 7fdfb7625740 1 -- 172.21.15.38:0/1202455696 shutdown_connections 2024-09-21T03:34:05.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.137+0000 7fdfb7625740 1 --2- 172.21.15.38:0/1202455696 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b6c548b590 0x55b6c588f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:05.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.137+0000 7fdfb7625740 1 --2- 172.21.15.38:0/1202455696 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b6c54833b0 0x55b6c588eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:05.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.137+0000 7fdfb7625740 1 --2- 172.21.15.38:0/1202455696 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b6c546ae40 0x55b6c5893950 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:05.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.137+0000 7fdfb7625740 1 -- 172.21.15.38:0/1202455696 >> 172.21.15.38:0/1202455696 conn(0x55b6c54a03c0 msgr2=0x55b6c584dd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:05.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.137+0000 7fdfb7625740 1 -- 172.21.15.38:0/1202455696 shutdown_connections 2024-09-21T03:34:05.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.137+0000 7fdfb7625740 1 -- 172.21.15.38:0/1202455696 wait complete. 2024-09-21T03:34:05.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.138+0000 7fdfb7625740 1 -- start start 2024-09-21T03:34:05.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.138+0000 7fdfb7625740 1 Processor -- start 2024-09-21T03:34:05.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.139+0000 7fdfb7625740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b6c546ae40 0x55b6c58a9350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:05.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.139+0000 7fdfb7625740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b6c54833b0 0x55b6c5852780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:05.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.139+0000 7fdfb7625740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b6c548b590 0x55b6c584dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:05.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.139+0000 7fdfb7625740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55b6c588fda0 con 0x55b6c548b590 2024-09-21T03:34:05.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.139+0000 7fdfb7625740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55b6c58916c0 con 0x55b6c546ae40 2024-09-21T03:34:05.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.139+0000 7fdfb7625740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55b6c5891f00 con 0x55b6c54833b0 2024-09-21T03:34:05.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.139+0000 7fdfb7625740 10 client.0 _ll_register_callbacks cb 0x7fdf98003b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:34:05.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.139+0000 7fdfaeb7c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b6c548b590 0x55b6c584dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:05.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.139+0000 7fdfaeb7c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b6c548b590 0x55b6c584dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42998/0 (socket says 172.21.15.38:42998) 2024-09-21T03:34:05.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.139+0000 7fdfaeb7c640 1 -- 172.21.15.38:0/2301766949 learned_addr learned my addr 172.21.15.38:0/2301766949 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:34:05.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.139+0000 7fdfae37b640 1 --2- 172.21.15.38:0/2301766949 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b6c546ae40 0x55b6c58a9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:05.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.139+0000 7fdfadb7a640 1 --2- 172.21.15.38:0/2301766949 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b6c54833b0 0x55b6c5852780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:05.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.140+0000 7fdfaeb7c640 1 -- 172.21.15.38:0/2301766949 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b6c54833b0 msgr2=0x55b6c5852780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:05.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.140+0000 7fdfaeb7c640 1 --2- 172.21.15.38:0/2301766949 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b6c54833b0 0x55b6c5852780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:05.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.140+0000 7fdfaeb7c640 1 -- 172.21.15.38:0/2301766949 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b6c546ae40 msgr2=0x55b6c58a9350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:05.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.140+0000 7fdfaeb7c640 1 --2- 172.21.15.38:0/2301766949 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b6c546ae40 0x55b6c58a9350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:05.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.140+0000 7fdfaeb7c640 1 -- 172.21.15.38:0/2301766949 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6c5895920 con 0x55b6c548b590 2024-09-21T03:34:05.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.140+0000 7fdfaeb7c640 1 --2- 172.21.15.38:0/2301766949 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b6c548b590 0x55b6c584dd80 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fdfa005fe20 tx=0x7fdfa006c210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7476143f9680188f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:05.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.140+0000 7fdf8d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-21T03:34:05.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.140+0000 7fdf8d7fa640 1 -- 172.21.15.38:0/2301766949 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfa0020040 con 0x55b6c548b590 2024-09-21T03:34:05.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.140+0000 7fdf8d7fa640 1 -- 172.21.15.38:0/2301766949 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fdfa00670c0 con 0x55b6c548b590 2024-09-21T03:34:05.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.140+0000 7fdfb7625740 1 -- 172.21.15.38:0/2301766949 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdf98004270 con 0x55b6c548b590 2024-09-21T03:34:05.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.140+0000 7fdf8d7fa640 1 -- 172.21.15.38:0/2301766949 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) ==== 24+0+0 (secure 0 0 0) 0x7fdfa0084050 con 0x55b6c548b590 2024-09-21T03:34:05.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.140+0000 7fdfb7625740 10 client.4800 fetch_fsmap learned FSMap version 9 2024-09-21T03:34:05.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.140+0000 7fdfb7625740 1 -- 172.21.15.38:0/2301766949 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) -- 0x55b6c5851f70 con 0x55b6c548b590 2024-09-21T03:34:05.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.140+0000 7fdf8d7fa640 1 -- 172.21.15.38:0/2301766949 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdf98004270 con 0x55b6c548b590 2024-09-21T03:34:05.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.140+0000 7fdf8d7fa640 1 -- 172.21.15.38:0/2301766949 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 9) ==== 3510+0+0 (secure 0 0 0) 0x7fdfa0061e10 con 0x55b6c548b590 2024-09-21T03:34:05.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.141+0000 7fdfb7625740 10 client.4800 fetch_fsmap finished waiting for FSMap version 9 2024-09-21T03:34:05.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.141+0000 7fdfb7625740 10 client.4800 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-21T03:34:05.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.141+0000 7fdfb7625740 20 client.4800 populate_metadata read hostname 'smithi038' 2024-09-21T03:34:05.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.141+0000 7fdfb7625740 1 --2- 172.21.15.38:0/2301766949 >> [v2:172.21.15.195:6835/1420805977,v1:172.21.15.195:6837/1420805977] conn(0x55b6c5440280 0x55b6c5918c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:05.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.141+0000 7fdfb7625740 4 client.4800 mds_command: new command op to 4402 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T03:34:05.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.141+0000 7fdfb7625740 1 -- 172.21.15.38:0/2301766949 --> [v2:172.21.15.195:6835/1420805977,v1:172.21.15.195:6837/1420805977] -- command(tid 0: {"prefix": "status"}) -- 0x55b6c5849f80 con 0x55b6c5440280 2024-09-21T03:34:05.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.141+0000 7fdfae37b640 1 --2- 172.21.15.38:0/2301766949 >> [v2:172.21.15.195:6835/1420805977,v1:172.21.15.195:6837/1420805977] conn(0x55b6c5440280 0x55b6c5918c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:05.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.141+0000 7fdf8d7fa640 10 client.4800 ms_handle_connect on v2:172.21.15.195:6835/1420805977 2024-09-21T03:34:05.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.141+0000 7fdfae37b640 1 --2- 172.21.15.38:0/2301766949 >> [v2:172.21.15.195:6835/1420805977,v1:172.21.15.195:6837/1420805977] conn(0x55b6c5440280 0x55b6c5918c90 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac7cf65e4e3450f0 server_cookie=cc9d247a0f1a5ba3 in_seq=0 out_seq=0 2024-09-21T03:34:05.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.141+0000 7fdf8d7fa640 1 -- 172.21.15.38:0/2301766949 <== mds.0 v2:172.21.15.195:6835/1420805977 1 ==== command_reply(tid 0: 0 ) ==== 8+0+330 (crc 0 0 0) 0x55b6c5849f80 con 0x55b6c5440280 2024-09-21T03:34:05.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.141+0000 7fdf8d7fa640 10 client.4800 handle_command_reply: tid=0 2024-09-21T03:34:05.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.141+0000 7fdfb7625740 1 client.4800 shutdown 2024-09-21T03:34:05.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.142+0000 7fdfb7625740 1 -- 172.21.15.38:0/2301766949 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b6c548b590 msgr2=0x55b6c584dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:05.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.142+0000 7fdfb7625740 1 --2- 172.21.15.38:0/2301766949 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b6c548b590 0x55b6c584dd80 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fdfa005fe20 tx=0x7fdfa006c210 comp rx=0 tx=0).stop 2024-09-21T03:34:05.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.142+0000 7fdfb7625740 1 -- 172.21.15.38:0/2301766949 shutdown_connections 2024-09-21T03:34:05.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.142+0000 7fdfb7625740 1 --2- 172.21.15.38:0/2301766949 >> [v2:172.21.15.195:6835/1420805977,v1:172.21.15.195:6837/1420805977] conn(0x55b6c5440280 0x55b6c5918c90 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:05.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.142+0000 7fdfb7625740 1 --2- 172.21.15.38:0/2301766949 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b6c548b590 0x55b6c584dd80 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:05.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.142+0000 7fdfb7625740 1 --2- 172.21.15.38:0/2301766949 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b6c54833b0 0x55b6c5852780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:05.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.142+0000 7fdfb7625740 1 --2- 172.21.15.38:0/2301766949 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b6c546ae40 0x55b6c58a9350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:05.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.142+0000 7fdf8d7fa640 0 client.4800 ms_handle_reset on v2:172.21.15.195:6835/1420805977 2024-09-21T03:34:05.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.142+0000 7fdfb7625740 1 -- 172.21.15.38:0/2301766949 >> 172.21.15.38:0/2301766949 conn(0x55b6c54a03c0 msgr2=0x55b6c584d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:05.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.142+0000 7fdfb7625740 1 -- 172.21.15.38:0/2301766949 shutdown_connections 2024-09-21T03:34:05.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.142+0000 7fdfb7625740 1 -- 172.21.15.38:0/2301766949 wait complete. 2024-09-21T03:34:05.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.142+0000 7fdfb7625740 20 client.4800 trim_cache size 0 max 16384 2024-09-21T03:34:05.149 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-09-21T03:34:05.149 INFO:teuthology.orchestra.run.smithi038.stdout: "cluster_fsid": "9d50d66f-656b-4fe2-8e3b-fe53b79c4d43", 2024-09-21T03:34:05.149 INFO:teuthology.orchestra.run.smithi038.stdout: "whoami": 0, 2024-09-21T03:34:05.149 INFO:teuthology.orchestra.run.smithi038.stdout: "id": 4402, 2024-09-21T03:34:05.149 INFO:teuthology.orchestra.run.smithi038.stdout: "want_state": "up:active", 2024-09-21T03:34:05.149 INFO:teuthology.orchestra.run.smithi038.stdout: "state": "up:active", 2024-09-21T03:34:05.149 INFO:teuthology.orchestra.run.smithi038.stdout: "fs_name": "cephfs", 2024-09-21T03:34:05.149 INFO:teuthology.orchestra.run.smithi038.stdout: "rank_uptime": 28.907586415000001, 2024-09-21T03:34:05.150 INFO:teuthology.orchestra.run.smithi038.stdout: "mdsmap_epoch": 9, 2024-09-21T03:34:05.150 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch": 26, 2024-09-21T03:34:05.150 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch_barrier": 26, 2024-09-21T03:34:05.150 INFO:teuthology.orchestra.run.smithi038.stdout: "uptime": 34.928584366999999 2024-09-21T03:34:05.150 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-09-21T03:34:05.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.142+0000 7fdfb05df640 1 -- 172.21.15.38:0/3468791659 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fdf94078b90 msgr2=0x7fdf9407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:05.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.142+0000 7fdfb05df640 1 --2- 172.21.15.38:0/3468791659 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fdf94078b90 0x7fdf9407b0a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fdf98033000 tx=0x7fdf98004ab0 comp rx=0 tx=0).stop 2024-09-21T03:34:05.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.142+0000 7fdfb05df640 1 -- 172.21.15.38:0/3468791659 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdfa815ee80 msgr2=0x7fdfa806cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:05.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.142+0000 7fdfb05df640 1 --2- 172.21.15.38:0/3468791659 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdfa815ee80 0x7fdfa806cde0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fdfa0005d00 tx=0x7fdfa000dae0 comp rx=0 tx=0).stop 2024-09-21T03:34:05.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.142+0000 7fdfb05df640 1 -- 172.21.15.38:0/3468791659 shutdown_connections 2024-09-21T03:34:05.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.142+0000 7fdfb05df640 1 --2- 172.21.15.38:0/3468791659 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdfa815ee80 0x7fdfa806cde0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:05.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.142+0000 7fdfb05df640 1 --2- 172.21.15.38:0/3468791659 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fdf94078b90 0x7fdf9407b0a0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:05.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.142+0000 7fdfb05df640 1 --2- 172.21.15.38:0/3468791659 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdfa80ceb90 0x7fdfa8158540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:05.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.142+0000 7fdfb05df640 1 --2- 172.21.15.38:0/3468791659 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdfa80b9ad0 0x7fdfa81f8c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:05.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.142+0000 7fdfb05df640 1 -- 172.21.15.38:0/3468791659 >> 172.21.15.38:0/3468791659 conn(0x7fdfa809d830 msgr2=0x7fdfa80b3fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:05.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.143+0000 7fdfb05df640 1 -- 172.21.15.38:0/3468791659 shutdown_connections 2024-09-21T03:34:05.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.143+0000 7fdfb05df640 1 -- 172.21.15.38:0/3468791659 wait complete. 2024-09-21T03:34:05.156 INFO:teuthology.run_tasks:Running task kclient... 2024-09-21T03:34:05.169 INFO:tasks.kclient:Mounting kernel clients... 2024-09-21T03:34:05.169 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-09-21T03:34:05.170 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:34:05.170 DEBUG:teuthology.orchestra.run.smithi038:> ip netns list 2024-09-21T03:34:05.186 INFO:teuthology.orchestra.run.smithi038.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-09-21T03:34:05.186 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:34:05.186 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-21T03:34:05.253 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:34:05.253 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip link delete ceph-brx 2024-09-21T03:34:05.352 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:34:05.353 DEBUG:teuthology.orchestra.run.smithi038:> ip netns list 2024-09-21T03:34:05.369 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:34:05.369 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip link delete ceph-brx 2024-09-21T03:34:05.438 INFO:teuthology.orchestra.run.smithi038.stderr:Cannot find device "ceph-brx" 2024-09-21T03:34:05.439 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T03:34:05.439 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:34:05.439 DEBUG:teuthology.orchestra.run.smithi195:> ip netns list 2024-09-21T03:34:05.458 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:34:05.459 DEBUG:teuthology.orchestra.run.smithi195:> sudo ip link delete ceph-brx 2024-09-21T03:34:05.530 INFO:teuthology.orchestra.run.smithi195.stderr:Cannot find device "ceph-brx" 2024-09-21T03:34:05.531 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T03:34:05.532 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:34:05.532 DEBUG:teuthology.orchestra.run.smithi195:> ip netns list 2024-09-21T03:34:05.589 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:34:05.589 DEBUG:teuthology.orchestra.run.smithi195:> sudo ip link delete ceph-brx 2024-09-21T03:34:05.658 INFO:teuthology.orchestra.run.smithi195.stderr:Cannot find device "ceph-brx" 2024-09-21T03:34:05.659 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T03:34:05.659 INFO:tasks.kclient:client.0 config is {'syntax': 'v1'} 2024-09-21T03:34:05.660 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-21T03:34:05.660 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T03:34:05.660 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T03:34:05.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.776+0000 7feb5460e640 1 -- 172.21.15.38:0/4087663495 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feb4c0d0980 msgr2=0x7feb4c0d0d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:05.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.776+0000 7feb5460e640 1 --2- 172.21.15.38:0/4087663495 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feb4c0d0980 0x7feb4c0d0d90 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7feb4c073e10 tx=0x7feb4402a530 comp rx=0 tx=0).stop 2024-09-21T03:34:05.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.776+0000 7feb5460e640 1 -- 172.21.15.38:0/4087663495 shutdown_connections 2024-09-21T03:34:05.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.776+0000 7feb5460e640 1 --2- 172.21.15.38:0/4087663495 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feb4c0d0980 0x7feb4c0d0d90 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:05.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.776+0000 7feb5460e640 1 --2- 172.21.15.38:0/4087663495 >> v2:172.21.15.195:3300/0 conn(0x7feb4c0d01f0 0x7feb4c15daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:05.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.776+0000 7feb5460e640 1 -- 172.21.15.38:0/4087663495 >> 172.21.15.38:0/4087663495 conn(0x7feb4c072990 msgr2=0x7feb4c072da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:05.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.776+0000 7feb5460e640 1 -- 172.21.15.38:0/4087663495 shutdown_connections 2024-09-21T03:34:05.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.776+0000 7feb5460e640 1 -- 172.21.15.38:0/4087663495 wait complete. 2024-09-21T03:34:05.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.778+0000 7feb5460e640 1 Processor -- start 2024-09-21T03:34:05.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.778+0000 7feb5460e640 1 -- start start 2024-09-21T03:34:05.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.779+0000 7feb5460e640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7feb4c0d01f0 0x7feb4c1f98f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:05.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.779+0000 7feb5460e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feb4c0d0980 0x7feb4c144960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:05.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.779+0000 7feb5460e640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feb4c15a6f0 0x7feb4c141f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:05.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.779+0000 7feb5460e640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7feb4c0cd340 con 0x7feb4c0d0980 2024-09-21T03:34:05.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.779+0000 7feb5460e640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7feb4c0cd1c0 con 0x7feb4c15a6f0 2024-09-21T03:34:05.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.779+0000 7feb5460e640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7feb4c0cd4c0 con 0x7feb4c0d01f0 2024-09-21T03:34:05.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.779+0000 7feb51ba9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feb4c0d0980 0x7feb4c144960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:05.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.779+0000 7feb51ba9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feb4c0d0980 0x7feb4c144960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43018/0 (socket says 172.21.15.38:43018) 2024-09-21T03:34:05.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.779+0000 7feb51ba9640 1 -- 172.21.15.38:0/1782992891 learned_addr learned my addr 172.21.15.38:0/1782992891 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:34:05.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.780+0000 7feb52bab640 1 --2- 172.21.15.38:0/1782992891 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feb4c15a6f0 0x7feb4c141f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:05.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.780+0000 7feb523aa640 1 --2- 172.21.15.38:0/1782992891 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7feb4c0d01f0 0x7feb4c1f98f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:05.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.780+0000 7feb51ba9640 1 -- 172.21.15.38:0/1782992891 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7feb4c0d01f0 msgr2=0x7feb4c1f98f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:05.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.780+0000 7feb51ba9640 1 --2- 172.21.15.38:0/1782992891 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7feb4c0d01f0 0x7feb4c1f98f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:05.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.780+0000 7feb51ba9640 1 -- 172.21.15.38:0/1782992891 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feb4c15a6f0 msgr2=0x7feb4c141f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:05.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.780+0000 7feb51ba9640 1 --2- 172.21.15.38:0/1782992891 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feb4c15a6f0 0x7feb4c141f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:05.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.780+0000 7feb51ba9640 1 -- 172.21.15.38:0/1782992891 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb4c145420 con 0x7feb4c0d0980 2024-09-21T03:34:05.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.780+0000 7feb51ba9640 1 --2- 172.21.15.38:0/1782992891 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feb4c0d0980 0x7feb4c144960 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7feb3c00c3c0 tx=0x7feb3c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:05.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.780+0000 7feb42ffd640 1 -- 172.21.15.38:0/1782992891 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb3c017070 con 0x7feb4c0d0980 2024-09-21T03:34:05.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.780+0000 7feb42ffd640 1 -- 172.21.15.38:0/1782992891 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7feb3c0035f0 con 0x7feb4c0d0980 2024-09-21T03:34:05.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.780+0000 7feb5460e640 1 -- 172.21.15.38:0/1782992891 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb4c145700 con 0x7feb4c0d0980 2024-09-21T03:34:05.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.780+0000 7feb42ffd640 1 -- 172.21.15.38:0/1782992891 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb3c003a40 con 0x7feb4c0d0980 2024-09-21T03:34:05.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.780+0000 7feb5460e640 1 -- 172.21.15.38:0/1782992891 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb4c1f93d0 con 0x7feb4c0d0980 2024-09-21T03:34:05.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.781+0000 7feb5460e640 1 -- 172.21.15.38:0/1782992891 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb4c145190 con 0x7feb4c0d0980 2024-09-21T03:34:05.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.782+0000 7feb42ffd640 1 -- 172.21.15.38:0/1782992891 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7feb3c01a070 con 0x7feb4c0d0980 2024-09-21T03:34:05.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.782+0000 7feb42ffd640 1 --2- 172.21.15.38:0/1782992891 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7feb30078b90 0x7feb3007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:05.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.783+0000 7feb42ffd640 1 -- 172.21.15.38:0/1782992891 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7feb3c09a340 con 0x7feb4c0d0980 2024-09-21T03:34:05.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.783+0000 7feb523aa640 1 --2- 172.21.15.38:0/1782992891 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7feb30078b90 0x7feb3007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:05.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.783+0000 7feb523aa640 1 --2- 172.21.15.38:0/1782992891 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7feb30078b90 0x7feb3007b0a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7feb48005fd0 tx=0x7feb480095e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:05.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:05.787+0000 7feb42ffd640 1 -- 172.21.15.38:0/1782992891 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7feb4c0d12d0 con 0x7feb4c0d0980 2024-09-21T03:34:06.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:06.013+0000 7feb5460e640 1 -- 172.21.15.38:0/1782992891 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7feb4c15daf0 con 0x7feb4c0d0980 2024-09-21T03:34:06.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:06.013+0000 7feb42ffd640 1 -- 172.21.15.38:0/1782992891 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7feb3c061d50 con 0x7feb4c0d0980 2024-09-21T03:34:06.015 INFO:teuthology.orchestra.run.smithi038.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-21T03:34:06.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:06.015+0000 7feb5460e640 1 -- 172.21.15.38:0/1782992891 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7feb30078b90 msgr2=0x7feb3007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:06.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:06.015+0000 7feb5460e640 1 --2- 172.21.15.38:0/1782992891 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7feb30078b90 0x7feb3007b0a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7feb48005fd0 tx=0x7feb480095e0 comp rx=0 tx=0).stop 2024-09-21T03:34:06.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:06.015+0000 7feb5460e640 1 -- 172.21.15.38:0/1782992891 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feb4c0d0980 msgr2=0x7feb4c144960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:06.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:06.015+0000 7feb5460e640 1 --2- 172.21.15.38:0/1782992891 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feb4c0d0980 0x7feb4c144960 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7feb3c00c3c0 tx=0x7feb3c00c880 comp rx=0 tx=0).stop 2024-09-21T03:34:06.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:06.015+0000 7feb5460e640 1 -- 172.21.15.38:0/1782992891 shutdown_connections 2024-09-21T03:34:06.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:06.015+0000 7feb5460e640 1 --2- 172.21.15.38:0/1782992891 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feb4c15a6f0 0x7feb4c141f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:06.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:06.015+0000 7feb5460e640 1 --2- 172.21.15.38:0/1782992891 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feb4c0d0980 0x7feb4c144960 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:06.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:06.015+0000 7feb5460e640 1 --2- 172.21.15.38:0/1782992891 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7feb30078b90 0x7feb3007b0a0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:06.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:06.015+0000 7feb5460e640 1 --2- 172.21.15.38:0/1782992891 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7feb4c0d01f0 0x7feb4c1f98f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:06.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:06.015+0000 7feb5460e640 1 -- 172.21.15.38:0/1782992891 >> 172.21.15.38:0/1782992891 conn(0x7feb4c072990 msgr2=0x7feb4c15e210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:06.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:06.016+0000 7feb5460e640 1 -- 172.21.15.38:0/1782992891 shutdown_connections 2024-09-21T03:34:06.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:06.016+0000 7feb5460e640 1 -- 172.21.15.38:0/1782992891 wait complete. 2024-09-21T03:34:06.029 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-21T03:34:06.029 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T03:34:06.029 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi038.front.sepia.ceph.com 2024-09-21T03:34:06.029 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-21T03:34:06.029 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T03:34:06.029 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-21T03:34:06.029 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-21T03:34:06.029 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T03:34:06.029 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T03:34:06.029 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:34:06.029 DEBUG:teuthology.orchestra.run.smithi038:> ip addr 2024-09-21T03:34:06.045 INFO:teuthology.orchestra.run.smithi038.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T03:34:06.045 INFO:teuthology.orchestra.run.smithi038.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T03:34:06.045 INFO:teuthology.orchestra.run.smithi038.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T03:34:06.045 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:34:06.045 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 ::1/128 scope host 2024-09-21T03:34:06.046 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:34:06.046 INFO:teuthology.orchestra.run.smithi038.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T03:34:06.046 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:6b:fd:a2 brd ff:ff:ff:ff:ff:ff 2024-09-21T03:34:06.046 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp7s0f0 2024-09-21T03:34:06.046 INFO:teuthology.orchestra.run.smithi038.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T03:34:06.046 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:6b:fd:a3 brd ff:ff:ff:ff:ff:ff 2024-09-21T03:34:06.046 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp7s0f1 2024-09-21T03:34:06.046 INFO:teuthology.orchestra.run.smithi038.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T03:34:06.046 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:bd:15:fc brd ff:ff:ff:ff:ff:ff 2024-09-21T03:34:06.046 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp1s0f0 2024-09-21T03:34:06.046 INFO:teuthology.orchestra.run.smithi038.stdout: inet 172.21.15.38/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-21T03:34:06.046 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft 42763sec preferred_lft 42763sec 2024-09-21T03:34:06.046 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 fe80::ec4:7aff:febd:15fc/64 scope link 2024-09-21T03:34:06.047 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:34:06.047 INFO:teuthology.orchestra.run.smithi038.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T03:34:06.047 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:bd:15:fd brd ff:ff:ff:ff:ff:ff 2024-09-21T03:34:06.047 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp1s0f1 2024-09-21T03:34:06.047 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-21T03:34:06.048 DEBUG:teuthology.orchestra.run.smithi038:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T03:34:06.048 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-09-21T03:34:06.048 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip link add name ceph-brx type bridge 2024-09-21T03:34:06.048 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip addr flush dev ceph-brx 2024-09-21T03:34:06.048 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip link set ceph-brx up 2024-09-21T03:34:06.048 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-21T03:34:06.048 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-09-21T03:34:06.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:06 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T03:34:06.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:34:06.187 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:34:06.187 DEBUG:teuthology.orchestra.run.smithi038:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T03:34:06.212 INFO:teuthology.orchestra.run.smithi038.stdout:1 2024-09-21T03:34:06.213 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:34:06.213 DEBUG:teuthology.orchestra.run.smithi038:> route 2024-09-21T03:34:06.270 INFO:teuthology.orchestra.run.smithi038.stdout:Kernel IP routing table 2024-09-21T03:34:06.270 INFO:teuthology.orchestra.run.smithi038.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T03:34:06.270 INFO:teuthology.orchestra.run.smithi038.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-21T03:34:06.270 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-21T03:34:06.271 INFO:teuthology.orchestra.run.smithi038.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T03:34:06.271 DEBUG:teuthology.orchestra.run.smithi038:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T03:34:06.271 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-09-21T03:34:06.271 DEBUG:teuthology.orchestra.run.smithi038:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-21T03:34:06.271 DEBUG:teuthology.orchestra.run.smithi038:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-21T03:34:06.271 DEBUG:teuthology.orchestra.run.smithi038:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-21T03:34:06.271 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-09-21T03:34:06.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:06 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T03:34:06.394 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:34:06.400 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:34:06.400 DEBUG:teuthology.orchestra.run.smithi038:> ip netns list 2024-09-21T03:34:06.416 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:34:06.416 DEBUG:teuthology.orchestra.run.smithi038:> ip netns list-id 2024-09-21T03:34:06.472 DEBUG:teuthology.orchestra.run.smithi038:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T03:34:06.472 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-09-21T03:34:06.472 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T03:34:06.472 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-09-21T03:34:06.472 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-09-21T03:34:06.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:06 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T03:34:06.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:34:06.591 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-09-21T03:34:06.591 DEBUG:teuthology.orchestra.run.smithi038:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T03:34:06.592 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-09-21T03:34:06.592 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-09-21T03:34:06.592 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-21T03:34:06.592 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-09-21T03:34:06.592 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-09-21T03:34:06.592 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-09-21T03:34:06.592 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-09-21T03:34:06.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:06 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T03:34:06.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:34:06.708 DEBUG:teuthology.orchestra.run.smithi038:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T03:34:06.709 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-09-21T03:34:06.709 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip link set brx.0 up 2024-09-21T03:34:06.709 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip link set dev brx.0 master ceph-brx 2024-09-21T03:34:06.709 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-09-21T03:34:06.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:06 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T03:34:06.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:34:06.787 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T03:34:06.787 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-21T03:34:06.803 INFO:teuthology.orchestra.run.smithi038.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-21T03:34:06.803 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T03:34:06.803 DEBUG:teuthology.orchestra.run.smithi038:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-21T03:34:06.858 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-21T03:34:06.858 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:34:06.858 DEBUG:teuthology.orchestra.run.smithi038:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T03:34:06.955 INFO:teuthology.orchestra.run.smithi038.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T03:34:06.955 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T03:34:06.955 INFO:teuthology.orchestra.run.smithi038.stdout:invalid new device string format 2024-09-21T03:34:07.119 INFO:teuthology.orchestra.run.smithi038.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T03:34:07.119 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T03:34:07.119 INFO:teuthology.orchestra.run.smithi038.stdout:invalid new device string format 2024-09-21T03:34:07.119 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: resolved to: "172.21.15.38:3300,172.21.15.195:3300,172.21.15.195:3301" 2024-09-21T03:34:07.119 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T03:34:07.125 INFO:teuthology.orchestra.run.smithi038.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T03:34:07.125 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T03:34:07.126 INFO:teuthology.orchestra.run.smithi038.stdout:invalid new device string format 2024-09-21T03:34:07.126 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: resolved to: "172.21.15.38:3300,172.21.15.195:3300,172.21.15.195:3301" 2024-09-21T03:34:07.126 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T03:34:07.126 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: resolved to: "172.21.15.38:6789,172.21.15.195:6789,172.21.15.195:6790" 2024-09-21T03:34:07.126 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.38:6789,172.21.15.195:6789,172.21.15.195:6790:/ 2024-09-21T03:34:07.126 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=9d50d66f-656b-4fe2-8e3b-fe53b79c4d43" will pass to kernel 2024-09-21T03:34:07.127 INFO:teuthology.orchestra.run.smithi038.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-21T03:34:07.127 INFO:teuthology.orchestra.run.smithi038.stdout: You just mounted a file system that supports labels which does not 2024-09-21T03:34:07.127 INFO:teuthology.orchestra.run.smithi038.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-21T03:34:07.127 INFO:teuthology.orchestra.run.smithi038.stdout: applications will generate AVC messages and not be allowed access to 2024-09-21T03:34:07.127 INFO:teuthology.orchestra.run.smithi038.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-21T03:34:07.129 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T03:34:07.129 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:34:07.129 DEBUG:teuthology.orchestra.run.smithi038:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-21T03:34:07.154 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T03:34:07.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:07 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T03:34:07.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:34:07.254 INFO:teuthology.orchestra.run.smithi038.stdout:client4643 2024-09-21T03:34:07.254 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-21T03:34:07.254 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:34:07.271 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T03:34:07.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:07 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T03:34:07.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:34:07.369 INFO:teuthology.orchestra.run.smithi038.stdout:client4643 2024-09-21T03:34:07.370 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T03:34:07.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:07 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T03:34:07.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:34:07.427 INFO:teuthology.orchestra.run.smithi038.stdout:client4643 2024-09-21T03:34:07.427 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/9d50d66f-656b-4fe2-8e3b-fe53b79c4d43.client4643/status') 2024-09-21T03:34:07.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:07 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T03:34:07.488 INFO:teuthology.orchestra.run.smithi038.stdout:instance: client.4643 (3)172.21.15.38:0/1167345553 2024-09-21T03:34:07.488 INFO:teuthology.orchestra.run.smithi038.stdout:blocklisted: false 2024-09-21T03:34:07.489 INFO:teuthology.orchestra.run.smithi038.stderr:0+1 records in 2024-09-21T03:34:07.489 INFO:teuthology.orchestra.run.smithi038.stderr:0+1 records out 2024-09-21T03:34:07.489 INFO:teuthology.orchestra.run.smithi038.stderr:70 bytes copied, 2.483e-05 s, 2.8 MB/s 2024-09-21T03:34:07.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:34:07.496 INFO:tasks.kclient:client.2 config is {'syntax': 'v1'} 2024-09-21T03:34:07.496 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-21T03:34:07.496 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-09-21T03:34:07.496 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T03:34:07.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:07.612+0000 7f0decbad640 1 -- 172.21.15.38:0/3548594141 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0de815f5a0 msgr2=0x7f0de815f9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:07.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:07.612+0000 7f0decbad640 1 --2- 172.21.15.38:0/3548594141 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0de815f5a0 0x7f0de815f9b0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f0de8072560 tx=0x7f0de002aa30 comp rx=0 tx=0).stop 2024-09-21T03:34:07.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:07.612+0000 7f0decbad640 1 -- 172.21.15.38:0/3548594141 shutdown_connections 2024-09-21T03:34:07.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:07.612+0000 7f0decbad640 1 --2- 172.21.15.38:0/3548594141 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0de815f5a0 0x7f0de815f9b0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:07.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:07.612+0000 7f0decbad640 1 --2- 172.21.15.38:0/3548594141 >> v2:172.21.15.195:3300/0 conn(0x7f0de815b9f0 0x7f0de80cf930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:07.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:07.613+0000 7f0decbad640 1 -- 172.21.15.38:0/3548594141 >> 172.21.15.38:0/3548594141 conn(0x7f0de80cca40 msgr2=0x7f0de80cce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:07.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:07.613+0000 7f0decbad640 1 -- 172.21.15.38:0/3548594141 shutdown_connections 2024-09-21T03:34:07.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:07.613+0000 7f0decbad640 1 -- 172.21.15.38:0/3548594141 wait complete. 2024-09-21T03:34:07.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:07.614+0000 7f0decbad640 1 Processor -- start 2024-09-21T03:34:07.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:07.615+0000 7f0decbad640 1 -- start start 2024-09-21T03:34:07.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:07.616+0000 7f0decbad640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0de80cefa0 0x7f0de815afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:07.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:07.616+0000 7f0decbad640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0de815b9f0 0x7f0de8158070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:07.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:07.616+0000 7f0decbad640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0de815f5a0 0x7f0de80aff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:07.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:07.616+0000 7f0decbad640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f0de806df70 con 0x7f0de815f5a0 2024-09-21T03:34:07.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:07.616+0000 7f0decbad640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f0de806ddf0 con 0x7f0de80cefa0 2024-09-21T03:34:07.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:07.616+0000 7f0decbad640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f0de806e0f0 con 0x7f0de815b9f0 2024-09-21T03:34:07.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:07.616+0000 7f0de6d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0de815f5a0 0x7f0de80aff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:07.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:07.616+0000 7f0de6d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0de815f5a0 0x7f0de80aff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43040/0 (socket says 172.21.15.38:43040) 2024-09-21T03:34:07.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:07.616+0000 7f0de6d9d640 1 -- 172.21.15.38:0/3685254255 learned_addr learned my addr 172.21.15.38:0/3685254255 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:34:07.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:07.616+0000 7f0de659c640 1 --2- 172.21.15.38:0/3685254255 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0de80cefa0 0x7f0de815afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:07.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:07.616+0000 7f0de5d9b640 1 --2- 172.21.15.38:0/3685254255 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0de815b9f0 0x7f0de8158070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:07.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:07.616+0000 7f0de6d9d640 1 -- 172.21.15.38:0/3685254255 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0de815b9f0 msgr2=0x7f0de8158070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:07.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:07.616+0000 7f0de6d9d640 1 --2- 172.21.15.38:0/3685254255 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0de815b9f0 0x7f0de8158070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:07.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:07.616+0000 7f0de6d9d640 1 -- 172.21.15.38:0/3685254255 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0de80cefa0 msgr2=0x7f0de815afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:07.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:07.616+0000 7f0de6d9d640 1 --2- 172.21.15.38:0/3685254255 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0de80cefa0 0x7f0de815afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:07.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:07.616+0000 7f0de6d9d640 1 -- 172.21.15.38:0/3685254255 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0de81f6770 con 0x7f0de815f5a0 2024-09-21T03:34:07.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:07.617+0000 7f0de6d9d640 1 --2- 172.21.15.38:0/3685254255 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0de815f5a0 0x7f0de80aff50 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f0de002b020 tx=0x7f0de0003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:07.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:07.617+0000 7f0dd6ffd640 1 -- 172.21.15.38:0/3685254255 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0de0033050 con 0x7f0de815f5a0 2024-09-21T03:34:07.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:07.617+0000 7f0dd6ffd640 1 -- 172.21.15.38:0/3685254255 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0de0004450 con 0x7f0de815f5a0 2024-09-21T03:34:07.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:07.617+0000 7f0dd6ffd640 1 -- 172.21.15.38:0/3685254255 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0de0038450 con 0x7f0de815f5a0 2024-09-21T03:34:07.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:07.617+0000 7f0decbad640 1 -- 172.21.15.38:0/3685254255 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0de81572b0 con 0x7f0de815f5a0 2024-09-21T03:34:07.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:07.617+0000 7f0decbad640 1 -- 172.21.15.38:0/3685254255 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0de8157870 con 0x7f0de815f5a0 2024-09-21T03:34:07.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:07.618+0000 7f0decbad640 1 -- 172.21.15.38:0/3685254255 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0de806e5a0 con 0x7f0de815f5a0 2024-09-21T03:34:07.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:07.619+0000 7f0dd6ffd640 1 -- 172.21.15.38:0/3685254255 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f0de0036070 con 0x7f0de815f5a0 2024-09-21T03:34:07.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:07.619+0000 7f0dd6ffd640 1 --2- 172.21.15.38:0/3685254255 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0dc8078b90 0x7f0dc807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:07.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:07.619+0000 7f0dd6ffd640 1 -- 172.21.15.38:0/3685254255 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f0de00bc410 con 0x7f0de815f5a0 2024-09-21T03:34:07.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:07.619+0000 7f0de659c640 1 --2- 172.21.15.38:0/3685254255 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0dc8078b90 0x7f0dc807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:07.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:07.620+0000 7f0de659c640 1 --2- 172.21.15.38:0/3685254255 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0dc8078b90 0x7f0dc807b0a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f0dd8004630 tx=0x7f0dd800b2e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:07.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:07.624+0000 7f0dd6ffd640 1 -- 172.21.15.38:0/3685254255 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f0de0083f50 con 0x7f0de815f5a0 2024-09-21T03:34:07.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:07.853+0000 7f0decbad640 1 -- 172.21.15.38:0/3685254255 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f0de815edc0 con 0x7f0de815f5a0 2024-09-21T03:34:07.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:07.853+0000 7f0dd6ffd640 1 -- 172.21.15.38:0/3685254255 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7f0de0088980 con 0x7f0de815f5a0 2024-09-21T03:34:07.855 INFO:teuthology.orchestra.run.smithi038.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-21T03:34:07.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:07.855+0000 7f0decbad640 1 -- 172.21.15.38:0/3685254255 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0dc8078b90 msgr2=0x7f0dc807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:07.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:07.855+0000 7f0decbad640 1 --2- 172.21.15.38:0/3685254255 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0dc8078b90 0x7f0dc807b0a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f0dd8004630 tx=0x7f0dd800b2e0 comp rx=0 tx=0).stop 2024-09-21T03:34:07.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:07.855+0000 7f0decbad640 1 -- 172.21.15.38:0/3685254255 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0de815f5a0 msgr2=0x7f0de80aff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:07.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:07.855+0000 7f0decbad640 1 --2- 172.21.15.38:0/3685254255 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0de815f5a0 0x7f0de80aff50 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f0de002b020 tx=0x7f0de0003f10 comp rx=0 tx=0).stop 2024-09-21T03:34:07.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:07.855+0000 7f0decbad640 1 -- 172.21.15.38:0/3685254255 shutdown_connections 2024-09-21T03:34:07.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:07.855+0000 7f0decbad640 1 --2- 172.21.15.38:0/3685254255 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0de815f5a0 0x7f0de80aff50 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:07.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:07.855+0000 7f0decbad640 1 --2- 172.21.15.38:0/3685254255 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0dc8078b90 0x7f0dc807b0a0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:07.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:07.855+0000 7f0decbad640 1 --2- 172.21.15.38:0/3685254255 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0de815b9f0 0x7f0de8158070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:07.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:07.855+0000 7f0decbad640 1 --2- 172.21.15.38:0/3685254255 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0de80cefa0 0x7f0de815afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:07.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:07.855+0000 7f0decbad640 1 -- 172.21.15.38:0/3685254255 >> 172.21.15.38:0/3685254255 conn(0x7f0de80cca40 msgr2=0x7f0de8073eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:07.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:07.856+0000 7f0decbad640 1 -- 172.21.15.38:0/3685254255 shutdown_connections 2024-09-21T03:34:07.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:07.856+0000 7f0decbad640 1 -- 172.21.15.38:0/3685254255 wait complete. 2024-09-21T03:34:07.869 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-21T03:34:07.869 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T03:34:07.870 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi038.front.sepia.ceph.com 2024-09-21T03:34:07.870 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-09-21T03:34:07.870 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-09-21T03:34:07.870 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-21T03:34:07.870 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-21T03:34:07.870 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T03:34:07.870 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-09-21T03:34:07.870 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:34:07.870 DEBUG:teuthology.orchestra.run.smithi038:> ip addr 2024-09-21T03:34:07.887 INFO:teuthology.orchestra.run.smithi038.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T03:34:07.887 INFO:teuthology.orchestra.run.smithi038.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T03:34:07.887 INFO:teuthology.orchestra.run.smithi038.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T03:34:07.887 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:34:07.887 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 ::1/128 scope host 2024-09-21T03:34:07.887 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:34:07.888 INFO:teuthology.orchestra.run.smithi038.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T03:34:07.888 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:6b:fd:a2 brd ff:ff:ff:ff:ff:ff 2024-09-21T03:34:07.888 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp7s0f0 2024-09-21T03:34:07.888 INFO:teuthology.orchestra.run.smithi038.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T03:34:07.888 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:6b:fd:a3 brd ff:ff:ff:ff:ff:ff 2024-09-21T03:34:07.888 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp7s0f1 2024-09-21T03:34:07.888 INFO:teuthology.orchestra.run.smithi038.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T03:34:07.888 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:bd:15:fc brd ff:ff:ff:ff:ff:ff 2024-09-21T03:34:07.888 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp1s0f0 2024-09-21T03:34:07.888 INFO:teuthology.orchestra.run.smithi038.stdout: inet 172.21.15.38/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-21T03:34:07.888 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft 42761sec preferred_lft 42761sec 2024-09-21T03:34:07.888 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 fe80::ec4:7aff:febd:15fc/64 scope link 2024-09-21T03:34:07.889 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:34:07.889 INFO:teuthology.orchestra.run.smithi038.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T03:34:07.889 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:bd:15:fd brd ff:ff:ff:ff:ff:ff 2024-09-21T03:34:07.889 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp1s0f1 2024-09-21T03:34:07.889 INFO:teuthology.orchestra.run.smithi038.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T03:34:07.889 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether a6:3a:83:ad:db:d5 brd ff:ff:ff:ff:ff:ff 2024-09-21T03:34:07.889 INFO:teuthology.orchestra.run.smithi038.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-21T03:34:07.889 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:34:07.889 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 fe80::c49f:c1ff:fe21:b12a/64 scope link tentative 2024-09-21T03:34:07.889 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:34:07.889 INFO:teuthology.orchestra.run.smithi038.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T03:34:07.889 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether a6:3a:83:ad:db:d5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T03:34:07.889 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 fe80::a43a:83ff:fead:dbd5/64 scope link tentative 2024-09-21T03:34:07.889 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:34:07.890 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:34:07.890 DEBUG:teuthology.orchestra.run.smithi038:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T03:34:07.961 INFO:teuthology.orchestra.run.smithi038.stdout:1 2024-09-21T03:34:07.962 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:34:07.962 DEBUG:teuthology.orchestra.run.smithi038:> route 2024-09-21T03:34:08.022 INFO:teuthology.orchestra.run.smithi038.stdout:Kernel IP routing table 2024-09-21T03:34:08.022 INFO:teuthology.orchestra.run.smithi038.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T03:34:08.022 INFO:teuthology.orchestra.run.smithi038.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-21T03:34:08.022 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-21T03:34:08.023 INFO:teuthology.orchestra.run.smithi038.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T03:34:08.023 DEBUG:teuthology.orchestra.run.smithi038:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T03:34:08.023 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-09-21T03:34:08.023 DEBUG:teuthology.orchestra.run.smithi038:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-21T03:34:08.023 DEBUG:teuthology.orchestra.run.smithi038:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-21T03:34:08.023 DEBUG:teuthology.orchestra.run.smithi038:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-21T03:34:08.023 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-09-21T03:34:08.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:08 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T03:34:08.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:34:08.158 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:34:08.158 DEBUG:teuthology.orchestra.run.smithi038:> ip netns list 2024-09-21T03:34:08.179 INFO:teuthology.orchestra.run.smithi038.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-21T03:34:08.179 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:34:08.179 DEBUG:teuthology.orchestra.run.smithi038:> ip netns list-id 2024-09-21T03:34:08.239 INFO:teuthology.orchestra.run.smithi038.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-09-21T03:34:08.239 DEBUG:teuthology.orchestra.run.smithi038:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T03:34:08.239 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-09-21T03:34:08.240 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T03:34:08.240 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-09-21T03:34:08.240 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-09-21T03:34:08.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:08 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T03:34:08.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:34:08.362 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:34:08.362 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-09-21T03:34:08.388 INFO:teuthology.orchestra.run.smithi038.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T03:34:08.388 INFO:teuthology.orchestra.run.smithi038.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T03:34:08.389 INFO:teuthology.orchestra.run.smithi038.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T03:34:08.389 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:34:08.389 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 ::1/128 scope host 2024-09-21T03:34:08.389 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:34:08.389 INFO:teuthology.orchestra.run.smithi038.stdout:2: veth0@if9: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T03:34:08.389 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 7e:3a:0d:5c:28:00 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-21T03:34:08.389 INFO:teuthology.orchestra.run.smithi038.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-21T03:34:08.389 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:34:08.389 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 fe80::7c3a:dff:fe5c:2800/64 scope link tentative 2024-09-21T03:34:08.389 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:34:08.390 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:34:08.390 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-09-21T03:34:08.456 INFO:teuthology.orchestra.run.smithi038.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T03:34:08.456 INFO:teuthology.orchestra.run.smithi038.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T03:34:08.456 INFO:teuthology.orchestra.run.smithi038.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T03:34:08.456 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:34:08.456 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 ::1/128 scope host 2024-09-21T03:34:08.456 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:34:08.456 INFO:teuthology.orchestra.run.smithi038.stdout:2: veth0@if9: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T03:34:08.456 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 7e:3a:0d:5c:28:00 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-21T03:34:08.456 INFO:teuthology.orchestra.run.smithi038.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-21T03:34:08.456 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:34:08.456 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 fe80::7c3a:dff:fe5c:2800/64 scope link tentative 2024-09-21T03:34:08.457 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:34:08.457 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-09-21T03:34:08.458 DEBUG:teuthology.orchestra.run.smithi038:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T03:34:08.458 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-09-21T03:34:08.458 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-09-21T03:34:08.458 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-09-21T03:34:08.458 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-09-21T03:34:08.458 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-09-21T03:34:08.458 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-09-21T03:34:08.458 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-09-21T03:34:08.547 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:08 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T03:34:08.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:34:08.614 DEBUG:teuthology.orchestra.run.smithi038:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T03:34:08.614 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-09-21T03:34:08.614 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip link set brx.1 up 2024-09-21T03:34:08.614 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip link set dev brx.1 master ceph-brx 2024-09-21T03:34:08.614 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-09-21T03:34:08.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:08 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T03:34:08.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:34:08.693 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T03:34:08.694 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-09-21T03:34:08.709 INFO:teuthology.orchestra.run.smithi038.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-09-21T03:34:08.710 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T03:34:08.710 DEBUG:teuthology.orchestra.run.smithi038:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-09-21T03:34:08.765 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-21T03:34:08.765 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:34:08.765 DEBUG:teuthology.orchestra.run.smithi038:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-09-21T03:34:08.859 INFO:teuthology.orchestra.run.smithi038.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-09-21T03:34:08.860 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-09-21T03:34:08.860 INFO:teuthology.orchestra.run.smithi038.stdout:invalid new device string format 2024-09-21T03:34:08.871 INFO:teuthology.orchestra.run.smithi038.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-09-21T03:34:08.871 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-09-21T03:34:08.871 INFO:teuthology.orchestra.run.smithi038.stdout:invalid new device string format 2024-09-21T03:34:08.871 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: resolved to: "172.21.15.38:3300,172.21.15.195:3300,172.21.15.195:3301" 2024-09-21T03:34:08.871 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T03:34:08.877 INFO:teuthology.orchestra.run.smithi038.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-09-21T03:34:08.877 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-09-21T03:34:08.877 INFO:teuthology.orchestra.run.smithi038.stdout:invalid new device string format 2024-09-21T03:34:08.877 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: resolved to: "172.21.15.38:3300,172.21.15.195:3300,172.21.15.195:3301" 2024-09-21T03:34:08.877 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T03:34:08.877 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: resolved to: "172.21.15.38:6789,172.21.15.195:6789,172.21.15.195:6790" 2024-09-21T03:34:08.877 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.38:6789,172.21.15.195:6789,172.21.15.195:6790:/ 2024-09-21T03:34:08.877 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs,key=2,fsid=9d50d66f-656b-4fe2-8e3b-fe53b79c4d43" will pass to kernel 2024-09-21T03:34:08.878 INFO:teuthology.orchestra.run.smithi038.stdout:mount: /home/ubuntu/cephtest/mnt.2 does not contain SELinux labels. 2024-09-21T03:34:08.879 INFO:teuthology.orchestra.run.smithi038.stdout: You just mounted a file system that supports labels which does not 2024-09-21T03:34:08.879 INFO:teuthology.orchestra.run.smithi038.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-21T03:34:08.879 INFO:teuthology.orchestra.run.smithi038.stdout: applications will generate AVC messages and not be allowed access to 2024-09-21T03:34:08.879 INFO:teuthology.orchestra.run.smithi038.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-21T03:34:08.880 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T03:34:08.880 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:34:08.880 DEBUG:teuthology.orchestra.run.smithi038:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-09-21T03:34:08.905 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T03:34:08.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:08 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T03:34:08.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:34:09.003 INFO:teuthology.orchestra.run.smithi038.stdout:client4774 2024-09-21T03:34:09.003 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-21T03:34:09.004 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:34:09.020 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T03:34:09.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:09 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T03:34:09.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:34:09.117 INFO:teuthology.orchestra.run.smithi038.stdout:client4774 2024-09-21T03:34:09.117 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T03:34:09.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:09 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T03:34:09.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:34:09.174 INFO:teuthology.orchestra.run.smithi038.stdout:client4774 2024-09-21T03:34:09.174 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/9d50d66f-656b-4fe2-8e3b-fe53b79c4d43.client4774/status') 2024-09-21T03:34:09.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:09 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T03:34:09.233 INFO:teuthology.orchestra.run.smithi038.stdout:instance: client.4774 (3)172.21.15.38:0/3227177893 2024-09-21T03:34:09.233 INFO:teuthology.orchestra.run.smithi038.stdout:blocklisted: false 2024-09-21T03:34:09.234 INFO:teuthology.orchestra.run.smithi038.stderr:0+1 records in 2024-09-21T03:34:09.234 INFO:teuthology.orchestra.run.smithi038.stderr:0+1 records out 2024-09-21T03:34:09.234 INFO:teuthology.orchestra.run.smithi038.stderr:70 bytes copied, 2.4869e-05 s, 2.8 MB/s 2024-09-21T03:34:09.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:34:09.241 INFO:tasks.kclient:client.1 config is {'syntax': 'v1'} 2024-09-21T03:34:09.241 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-21T03:34:09.241 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-21T03:34:09.241 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T03:34:09.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:09.356+0000 7f4870b19640 1 -- 172.21.15.38:0/3654442745 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f486c0ca610 msgr2=0x7f486c0cdb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:09.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:09.356+0000 7f4870b19640 1 --2- 172.21.15.38:0/3654442745 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f486c0ca610 0x7f486c0cdb90 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f485c005fd0 tx=0x7f485c02a5d0 comp rx=0 tx=0).stop 2024-09-21T03:34:09.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:09.356+0000 7f4870b19640 1 -- 172.21.15.38:0/3654442745 shutdown_connections 2024-09-21T03:34:09.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:09.356+0000 7f4870b19640 1 --2- 172.21.15.38:0/3654442745 >> v2:172.21.15.195:3300/0 conn(0x7f486c074530 0x7f486c15e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:09.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:09.356+0000 7f4870b19640 1 --2- 172.21.15.38:0/3654442745 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f486c0ca610 0x7f486c0cdb90 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:09.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:09.356+0000 7f4870b19640 1 -- 172.21.15.38:0/3654442745 >> 172.21.15.38:0/3654442745 conn(0x7f486c0cca40 msgr2=0x7f486c0cce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:09.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:09.356+0000 7f4870b19640 1 -- 172.21.15.38:0/3654442745 shutdown_connections 2024-09-21T03:34:09.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:09.356+0000 7f4870b19640 1 -- 172.21.15.38:0/3654442745 wait complete. 2024-09-21T03:34:09.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:09.358+0000 7f4870b19640 1 Processor -- start 2024-09-21T03:34:09.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:09.358+0000 7f4870b19640 1 -- start start 2024-09-21T03:34:09.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:09.359+0000 7f4870b19640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f486c074530 0x7f486c0a09d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:09.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:09.359+0000 7f4870b19640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f486c0ca610 0x7f486c154a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:09.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:09.359+0000 7f4870b19640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f486c0ce0d0 0x7f486c151510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:09.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:09.359+0000 7f4870b19640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f486c06e230 con 0x7f486c074530 2024-09-21T03:34:09.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:09.359+0000 7f4870b19640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f486c06e0b0 con 0x7f486c0ce0d0 2024-09-21T03:34:09.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:09.359+0000 7f4870b19640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f486c06e3b0 con 0x7f486c0ca610 2024-09-21T03:34:09.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:09.360+0000 7f486a59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f486c074530 0x7f486c0a09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:09.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:09.360+0000 7f486a59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f486c074530 0x7f486c0a09d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43046/0 (socket says 172.21.15.38:43046) 2024-09-21T03:34:09.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:09.360+0000 7f486a59c640 1 -- 172.21.15.38:0/3675982593 learned_addr learned my addr 172.21.15.38:0/3675982593 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:34:09.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:09.360+0000 7f486ad9d640 1 --2- 172.21.15.38:0/3675982593 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f486c0ce0d0 0x7f486c151510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:09.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:09.360+0000 7f4869d9b640 1 --2- 172.21.15.38:0/3675982593 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f486c0ca610 0x7f486c154a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:09.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:09.360+0000 7f486a59c640 1 -- 172.21.15.38:0/3675982593 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f486c0ca610 msgr2=0x7f486c154a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:09.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:09.360+0000 7f486a59c640 1 --2- 172.21.15.38:0/3675982593 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f486c0ca610 0x7f486c154a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:09.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:09.360+0000 7f486a59c640 1 -- 172.21.15.38:0/3675982593 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f486c0ce0d0 msgr2=0x7f486c151510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:09.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:09.360+0000 7f486a59c640 1 --2- 172.21.15.38:0/3675982593 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f486c0ce0d0 0x7f486c151510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:09.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:09.360+0000 7f486a59c640 1 -- 172.21.15.38:0/3675982593 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f486c166c40 con 0x7f486c074530 2024-09-21T03:34:09.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:09.360+0000 7f4869d9b640 1 --2- 172.21.15.38:0/3675982593 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f486c0ca610 0x7f486c154a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:34:09.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:09.360+0000 7f486a59c640 1 --2- 172.21.15.38:0/3675982593 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f486c074530 0x7f486c0a09d0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f485c034040 tx=0x7f485c0035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:09.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:09.360+0000 7f485affd640 1 -- 172.21.15.38:0/3675982593 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f485c03e030 con 0x7f486c074530 2024-09-21T03:34:09.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:09.360+0000 7f485affd640 1 -- 172.21.15.38:0/3675982593 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f485c003ae0 con 0x7f486c074530 2024-09-21T03:34:09.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:09.360+0000 7f4870b19640 1 -- 172.21.15.38:0/3675982593 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f486c166f20 con 0x7f486c074530 2024-09-21T03:34:09.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:09.360+0000 7f485affd640 1 -- 172.21.15.38:0/3675982593 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f485c035040 con 0x7f486c074530 2024-09-21T03:34:09.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:09.360+0000 7f4870b19640 1 -- 172.21.15.38:0/3675982593 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f486c1674e0 con 0x7f486c074530 2024-09-21T03:34:09.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:09.362+0000 7f4870b19640 1 -- 172.21.15.38:0/3675982593 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f486c074bb0 con 0x7f486c074530 2024-09-21T03:34:09.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:09.362+0000 7f485affd640 1 -- 172.21.15.38:0/3675982593 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f485c041030 con 0x7f486c074530 2024-09-21T03:34:09.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:09.363+0000 7f485affd640 1 --2- 172.21.15.38:0/3675982593 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f484c078b90 0x7f484c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:09.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:09.363+0000 7f485affd640 1 -- 172.21.15.38:0/3675982593 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f485c0bb540 con 0x7f486c074530 2024-09-21T03:34:09.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:09.363+0000 7f4869d9b640 1 --2- 172.21.15.38:0/3675982593 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f484c078b90 0x7f484c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:09.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:09.364+0000 7f4869d9b640 1 --2- 172.21.15.38:0/3675982593 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f484c078b90 0x7f484c07b0a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f4860000f60 tx=0x7f4860009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:09.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:09.368+0000 7f485affd640 1 -- 172.21.15.38:0/3675982593 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f486c074bb0 con 0x7f486c074530 2024-09-21T03:34:09.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:09.594+0000 7f4870b19640 1 -- 172.21.15.38:0/3675982593 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f486c166980 con 0x7f486c074530 2024-09-21T03:34:09.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:09.595+0000 7f485affd640 1 -- 172.21.15.38:0/3675982593 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7f485c083650 con 0x7f486c074530 2024-09-21T03:34:09.597 INFO:teuthology.orchestra.run.smithi038.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-21T03:34:09.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:09.597+0000 7f4870b19640 1 -- 172.21.15.38:0/3675982593 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f484c078b90 msgr2=0x7f484c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:09.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:09.597+0000 7f4870b19640 1 --2- 172.21.15.38:0/3675982593 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f484c078b90 0x7f484c07b0a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f4860000f60 tx=0x7f4860009040 comp rx=0 tx=0).stop 2024-09-21T03:34:09.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:09.597+0000 7f4870b19640 1 -- 172.21.15.38:0/3675982593 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f486c074530 msgr2=0x7f486c0a09d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:09.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:09.597+0000 7f4870b19640 1 --2- 172.21.15.38:0/3675982593 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f486c074530 0x7f486c0a09d0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f485c034040 tx=0x7f485c0035f0 comp rx=0 tx=0).stop 2024-09-21T03:34:09.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:09.597+0000 7f4870b19640 1 -- 172.21.15.38:0/3675982593 shutdown_connections 2024-09-21T03:34:09.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:09.597+0000 7f4870b19640 1 --2- 172.21.15.38:0/3675982593 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f486c0ce0d0 0x7f486c151510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:09.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:09.597+0000 7f4870b19640 1 --2- 172.21.15.38:0/3675982593 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f484c078b90 0x7f484c07b0a0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:09.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:09.597+0000 7f4870b19640 1 --2- 172.21.15.38:0/3675982593 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f486c0ca610 0x7f486c154a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:09.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:09.597+0000 7f4870b19640 1 --2- 172.21.15.38:0/3675982593 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f486c074530 0x7f486c0a09d0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:09.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:09.597+0000 7f4870b19640 1 -- 172.21.15.38:0/3675982593 >> 172.21.15.38:0/3675982593 conn(0x7f486c0cca40 msgr2=0x7f486c1520b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:09.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:09.597+0000 7f4870b19640 1 -- 172.21.15.38:0/3675982593 shutdown_connections 2024-09-21T03:34:09.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:09.598+0000 7f4870b19640 1 -- 172.21.15.38:0/3675982593 wait complete. 2024-09-21T03:34:09.611 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-21T03:34:09.611 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T03:34:09.611 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi195.front.sepia.ceph.com 2024-09-21T03:34:09.611 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-21T03:34:09.611 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-21T03:34:09.611 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-21T03:34:09.611 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-21T03:34:09.611 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T03:34:09.611 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-21T03:34:09.611 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:34:09.611 DEBUG:teuthology.orchestra.run.smithi195:> ip addr 2024-09-21T03:34:09.630 INFO:teuthology.orchestra.run.smithi195.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T03:34:09.630 INFO:teuthology.orchestra.run.smithi195.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T03:34:09.631 INFO:teuthology.orchestra.run.smithi195.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T03:34:09.631 INFO:teuthology.orchestra.run.smithi195.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:34:09.631 INFO:teuthology.orchestra.run.smithi195.stdout: inet6 ::1/128 scope host 2024-09-21T03:34:09.631 INFO:teuthology.orchestra.run.smithi195.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:34:09.631 INFO:teuthology.orchestra.run.smithi195.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T03:34:09.631 INFO:teuthology.orchestra.run.smithi195.stdout: link/ether 0c:c4:7a:d9:36:f0 brd ff:ff:ff:ff:ff:ff 2024-09-21T03:34:09.631 INFO:teuthology.orchestra.run.smithi195.stdout: altname enp7s0f0 2024-09-21T03:34:09.631 INFO:teuthology.orchestra.run.smithi195.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T03:34:09.631 INFO:teuthology.orchestra.run.smithi195.stdout: link/ether 0c:c4:7a:d9:36:f1 brd ff:ff:ff:ff:ff:ff 2024-09-21T03:34:09.631 INFO:teuthology.orchestra.run.smithi195.stdout: altname enp7s0f1 2024-09-21T03:34:09.631 INFO:teuthology.orchestra.run.smithi195.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T03:34:09.631 INFO:teuthology.orchestra.run.smithi195.stdout: link/ether 0c:c4:7a:88:80:80 brd ff:ff:ff:ff:ff:ff 2024-09-21T03:34:09.631 INFO:teuthology.orchestra.run.smithi195.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T03:34:09.631 INFO:teuthology.orchestra.run.smithi195.stdout: link/ether 0c:c4:7a:88:80:81 brd ff:ff:ff:ff:ff:ff 2024-09-21T03:34:09.632 INFO:teuthology.orchestra.run.smithi195.stdout: inet 172.21.15.195/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-21T03:34:09.632 INFO:teuthology.orchestra.run.smithi195.stdout: valid_lft 42745sec preferred_lft 42745sec 2024-09-21T03:34:09.632 INFO:teuthology.orchestra.run.smithi195.stdout: inet6 fe80::ec4:7aff:fe88:8081/64 scope link noprefixroute 2024-09-21T03:34:09.632 INFO:teuthology.orchestra.run.smithi195.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:34:09.632 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-21T03:34:09.632 DEBUG:teuthology.orchestra.run.smithi195:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T03:34:09.632 DEBUG:teuthology.orchestra.run.smithi195:> set -e 2024-09-21T03:34:09.632 DEBUG:teuthology.orchestra.run.smithi195:> sudo ip link add name ceph-brx type bridge 2024-09-21T03:34:09.632 DEBUG:teuthology.orchestra.run.smithi195:> sudo ip addr flush dev ceph-brx 2024-09-21T03:34:09.632 DEBUG:teuthology.orchestra.run.smithi195:> sudo ip link set ceph-brx up 2024-09-21T03:34:09.633 DEBUG:teuthology.orchestra.run.smithi195:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-21T03:34:09.633 DEBUG:teuthology.orchestra.run.smithi195:> ') 2024-09-21T03:34:09.726 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:34:09 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T03:34:09.914 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:34:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:34:09.920 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:34:09.920 DEBUG:teuthology.orchestra.run.smithi195:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T03:34:09.944 INFO:teuthology.orchestra.run.smithi195.stdout:1 2024-09-21T03:34:09.946 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:34:09.946 DEBUG:teuthology.orchestra.run.smithi195:> route 2024-09-21T03:34:10.015 INFO:teuthology.orchestra.run.smithi195.stdout:Kernel IP routing table 2024-09-21T03:34:10.016 INFO:teuthology.orchestra.run.smithi195.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T03:34:10.016 INFO:teuthology.orchestra.run.smithi195.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-21T03:34:10.016 INFO:teuthology.orchestra.run.smithi195.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-21T03:34:10.016 INFO:teuthology.orchestra.run.smithi195.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T03:34:10.016 DEBUG:teuthology.orchestra.run.smithi195:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T03:34:10.016 DEBUG:teuthology.orchestra.run.smithi195:> set -e 2024-09-21T03:34:10.016 DEBUG:teuthology.orchestra.run.smithi195:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-21T03:34:10.016 DEBUG:teuthology.orchestra.run.smithi195:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-21T03:34:10.016 DEBUG:teuthology.orchestra.run.smithi195:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-21T03:34:10.017 DEBUG:teuthology.orchestra.run.smithi195:> ') 2024-09-21T03:34:10.105 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:34:10 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T03:34:10.299 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:34:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:34:10.305 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:34:10.305 DEBUG:teuthology.orchestra.run.smithi195:> ip netns list 2024-09-21T03:34:10.321 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:34:10.321 DEBUG:teuthology.orchestra.run.smithi195:> ip netns list-id 2024-09-21T03:34:10.376 DEBUG:teuthology.orchestra.run.smithi195:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T03:34:10.376 DEBUG:teuthology.orchestra.run.smithi195:> set -e 2024-09-21T03:34:10.377 DEBUG:teuthology.orchestra.run.smithi195:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-21T03:34:10.377 DEBUG:teuthology.orchestra.run.smithi195:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-09-21T03:34:10.377 DEBUG:teuthology.orchestra.run.smithi195:> ') 2024-09-21T03:34:10.465 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:34:10 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T03:34:10.489 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:34:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:34:10.495 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-09-21T03:34:10.496 DEBUG:teuthology.orchestra.run.smithi195:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T03:34:10.496 DEBUG:teuthology.orchestra.run.smithi195:> set -e 2024-09-21T03:34:10.496 DEBUG:teuthology.orchestra.run.smithi195:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-09-21T03:34:10.496 DEBUG:teuthology.orchestra.run.smithi195:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-21T03:34:10.496 DEBUG:teuthology.orchestra.run.smithi195:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-09-21T03:34:10.496 DEBUG:teuthology.orchestra.run.smithi195:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-09-21T03:34:10.496 DEBUG:teuthology.orchestra.run.smithi195:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-09-21T03:34:10.496 DEBUG:teuthology.orchestra.run.smithi195:> ') 2024-09-21T03:34:10.545 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:34:10 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T03:34:10.626 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:34:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:34:10.632 DEBUG:teuthology.orchestra.run.smithi195:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T03:34:10.632 DEBUG:teuthology.orchestra.run.smithi195:> set -e 2024-09-21T03:34:10.632 DEBUG:teuthology.orchestra.run.smithi195:> sudo ip link set brx.0 up 2024-09-21T03:34:10.633 DEBUG:teuthology.orchestra.run.smithi195:> sudo ip link set dev brx.0 master ceph-brx 2024-09-21T03:34:10.633 DEBUG:teuthology.orchestra.run.smithi195:> ') 2024-09-21T03:34:10.680 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:34:10 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T03:34:10.703 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:34:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:34:10.709 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T03:34:10.709 DEBUG:teuthology.orchestra.run.smithi195:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-21T03:34:10.724 INFO:teuthology.orchestra.run.smithi195.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-21T03:34:10.724 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T03:34:10.724 DEBUG:teuthology.orchestra.run.smithi195:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-21T03:34:10.779 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-21T03:34:10.780 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:34:10.780 DEBUG:teuthology.orchestra.run.smithi195:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-21T03:34:10.883 INFO:teuthology.orchestra.run.smithi195.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-21T03:34:10.883 INFO:teuthology.orchestra.run.smithi195.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-21T03:34:10.883 INFO:teuthology.orchestra.run.smithi195.stdout:invalid new device string format 2024-09-21T03:34:11.022 INFO:teuthology.orchestra.run.smithi195.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-21T03:34:11.022 INFO:teuthology.orchestra.run.smithi195.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-21T03:34:11.022 INFO:teuthology.orchestra.run.smithi195.stdout:invalid new device string format 2024-09-21T03:34:11.022 INFO:teuthology.orchestra.run.smithi195.stdout:mount.ceph: resolved to: "172.21.15.38:3300,172.21.15.195:3300,172.21.15.195:3301" 2024-09-21T03:34:11.022 INFO:teuthology.orchestra.run.smithi195.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T03:34:11.029 INFO:teuthology.orchestra.run.smithi195.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-21T03:34:11.029 INFO:teuthology.orchestra.run.smithi195.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-21T03:34:11.029 INFO:teuthology.orchestra.run.smithi195.stdout:invalid new device string format 2024-09-21T03:34:11.030 INFO:teuthology.orchestra.run.smithi195.stdout:mount.ceph: resolved to: "172.21.15.38:3300,172.21.15.195:3300,172.21.15.195:3301" 2024-09-21T03:34:11.030 INFO:teuthology.orchestra.run.smithi195.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T03:34:11.030 INFO:teuthology.orchestra.run.smithi195.stdout:mount.ceph: resolved to: "172.21.15.38:6789,172.21.15.195:6789,172.21.15.195:6790" 2024-09-21T03:34:11.030 INFO:teuthology.orchestra.run.smithi195.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.38:6789,172.21.15.195:6789,172.21.15.195:6790:/ 2024-09-21T03:34:11.030 INFO:teuthology.orchestra.run.smithi195.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=9d50d66f-656b-4fe2-8e3b-fe53b79c4d43" will pass to kernel 2024-09-21T03:34:11.030 INFO:teuthology.orchestra.run.smithi195.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-09-21T03:34:11.031 INFO:teuthology.orchestra.run.smithi195.stdout: You just mounted a file system that supports labels which does not 2024-09-21T03:34:11.031 INFO:teuthology.orchestra.run.smithi195.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-21T03:34:11.031 INFO:teuthology.orchestra.run.smithi195.stdout: applications will generate AVC messages and not be allowed access to 2024-09-21T03:34:11.031 INFO:teuthology.orchestra.run.smithi195.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-21T03:34:11.033 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T03:34:11.033 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:34:11.033 DEBUG:teuthology.orchestra.run.smithi195:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-21T03:34:11.057 DEBUG:teuthology.orchestra.run.smithi195:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T03:34:11.146 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:34:11 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T03:34:11.148 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:34:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:34:11.154 INFO:teuthology.orchestra.run.smithi195.stdout:client4821 2024-09-21T03:34:11.154 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-09-21T03:34:11.154 DEBUG:teuthology.orchestra.run.smithi195:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:34:11.170 DEBUG:teuthology.orchestra.run.smithi195:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T03:34:11.258 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:34:11 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T03:34:11.259 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:34:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:34:11.265 INFO:teuthology.orchestra.run.smithi195.stdout:client4821 2024-09-21T03:34:11.266 DEBUG:teuthology.orchestra.run.smithi195:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T03:34:11.313 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:34:11 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T03:34:11.315 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:34:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:34:11.321 INFO:teuthology.orchestra.run.smithi195.stdout:client4821 2024-09-21T03:34:11.321 DEBUG:teuthology.orchestra.run.smithi195:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/9d50d66f-656b-4fe2-8e3b-fe53b79c4d43.client4821/status') 2024-09-21T03:34:11.370 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:34:11 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T03:34:11.380 INFO:teuthology.orchestra.run.smithi195.stdout:instance: client.4821 (3)172.21.15.195:0/1439602498 2024-09-21T03:34:11.380 INFO:teuthology.orchestra.run.smithi195.stdout:blocklisted: false 2024-09-21T03:34:11.380 INFO:teuthology.orchestra.run.smithi195.stderr:0+1 records in 2024-09-21T03:34:11.380 INFO:teuthology.orchestra.run.smithi195.stderr:0+1 records out 2024-09-21T03:34:11.380 INFO:teuthology.orchestra.run.smithi195.stderr:71 bytes copied, 2.4663e-05 s, 2.9 MB/s 2024-09-21T03:34:11.381 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:34:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:34:11.387 INFO:tasks.kclient:client.3 config is {'syntax': 'v1'} 2024-09-21T03:34:11.387 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-21T03:34:11.387 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-09-21T03:34:11.387 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T03:34:11.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:11.517+0000 7fde2c920640 1 -- 172.21.15.38:0/1339598605 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fde28162590 msgr2=0x7fde281629a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:11.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:11.517+0000 7fde2c920640 1 --2- 172.21.15.38:0/1339598605 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fde28162590 0x7fde281629a0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fde1c009830 tx=0x7fde1c02a510 comp rx=0 tx=0).stop 2024-09-21T03:34:11.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:11.517+0000 7fde2c920640 1 -- 172.21.15.38:0/1339598605 shutdown_connections 2024-09-21T03:34:11.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:11.517+0000 7fde2c920640 1 --2- 172.21.15.38:0/1339598605 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fde28162590 0x7fde281629a0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:11.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:11.517+0000 7fde2c920640 1 --2- 172.21.15.38:0/1339598605 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fde280cfe00 0x7fde280d2a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:11.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:11.517+0000 7fde2c920640 1 -- 172.21.15.38:0/1339598605 >> 172.21.15.38:0/1339598605 conn(0x7fde280d4440 msgr2=0x7fde280d4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:11.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:11.517+0000 7fde2c920640 1 -- 172.21.15.38:0/1339598605 shutdown_connections 2024-09-21T03:34:11.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:11.517+0000 7fde2c920640 1 -- 172.21.15.38:0/1339598605 wait complete. 2024-09-21T03:34:11.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:11.519+0000 7fde2c920640 1 Processor -- start 2024-09-21T03:34:11.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:11.519+0000 7fde2c920640 1 -- start start 2024-09-21T03:34:11.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:11.520+0000 7fde2c920640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fde280b2e60 0x7fde281565b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:11.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:11.520+0000 7fde2c920640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fde280cfe00 0x7fde2815a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:11.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:11.520+0000 7fde2c920640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fde28162590 0x7fde2815ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:11.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:11.520+0000 7fde2c920640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fde280b3b00 con 0x7fde280b2e60 2024-09-21T03:34:11.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:11.520+0000 7fde2c920640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fde280b3980 con 0x7fde280cfe00 2024-09-21T03:34:11.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:11.520+0000 7fde2c920640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fde280b3f60 con 0x7fde28162590 2024-09-21T03:34:11.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:11.521+0000 7fde2659c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fde280b2e60 0x7fde281565b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:11.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:11.521+0000 7fde2659c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fde280b2e60 0x7fde281565b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43062/0 (socket says 172.21.15.38:43062) 2024-09-21T03:34:11.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:11.521+0000 7fde2659c640 1 -- 172.21.15.38:0/76315429 learned_addr learned my addr 172.21.15.38:0/76315429 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:34:11.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:11.521+0000 7fde25d9b640 1 --2- 172.21.15.38:0/76315429 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fde280cfe00 0x7fde2815a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:11.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:11.521+0000 7fde26d9d640 1 --2- 172.21.15.38:0/76315429 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fde28162590 0x7fde2815ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:11.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:11.521+0000 7fde2659c640 1 -- 172.21.15.38:0/76315429 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fde28162590 msgr2=0x7fde2815ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:11.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:11.521+0000 7fde2659c640 1 --2- 172.21.15.38:0/76315429 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fde28162590 0x7fde2815ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:11.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:11.521+0000 7fde2659c640 1 -- 172.21.15.38:0/76315429 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fde280cfe00 msgr2=0x7fde2815a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:11.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:11.521+0000 7fde2659c640 1 --2- 172.21.15.38:0/76315429 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fde280cfe00 0x7fde2815a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:11.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:11.521+0000 7fde2659c640 1 -- 172.21.15.38:0/76315429 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde280ce480 con 0x7fde280b2e60 2024-09-21T03:34:11.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:11.521+0000 7fde2659c640 1 --2- 172.21.15.38:0/76315429 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fde280b2e60 0x7fde281565b0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fde1800e790 tx=0x7fde1800ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:11.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:11.521+0000 7fde16ffd640 1 -- 172.21.15.38:0/76315429 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde1800eea0 con 0x7fde280b2e60 2024-09-21T03:34:11.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:11.521+0000 7fde16ffd640 1 -- 172.21.15.38:0/76315429 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fde1800cdb0 con 0x7fde280b2e60 2024-09-21T03:34:11.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:11.521+0000 7fde2c920640 1 -- 172.21.15.38:0/76315429 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde280ce760 con 0x7fde280b2e60 2024-09-21T03:34:11.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:11.522+0000 7fde16ffd640 1 -- 172.21.15.38:0/76315429 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde18010690 con 0x7fde280b2e60 2024-09-21T03:34:11.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:11.522+0000 7fde2c920640 1 -- 172.21.15.38:0/76315429 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde280c8c00 con 0x7fde280b2e60 2024-09-21T03:34:11.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:11.522+0000 7fde2c920640 1 -- 172.21.15.38:0/76315429 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde280c7b80 con 0x7fde280b2e60 2024-09-21T03:34:11.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:11.524+0000 7fde16ffd640 1 -- 172.21.15.38:0/76315429 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fde18010830 con 0x7fde280b2e60 2024-09-21T03:34:11.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:11.524+0000 7fde16ffd640 1 --2- 172.21.15.38:0/76315429 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fde08078b90 0x7fde0807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:11.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:11.524+0000 7fde16ffd640 1 -- 172.21.15.38:0/76315429 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fde1809ba10 con 0x7fde280b2e60 2024-09-21T03:34:11.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:11.524+0000 7fde25d9b640 1 --2- 172.21.15.38:0/76315429 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fde08078b90 0x7fde0807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:11.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:11.525+0000 7fde25d9b640 1 --2- 172.21.15.38:0/76315429 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fde08078b90 0x7fde0807b0a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fde1c034000 tx=0x7fde1c004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:11.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:11.529+0000 7fde16ffd640 1 -- 172.21.15.38:0/76315429 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fde280c7b80 con 0x7fde280b2e60 2024-09-21T03:34:11.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:11.753+0000 7fde2c920640 1 -- 172.21.15.38:0/76315429 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fde280d2a20 con 0x7fde280b2e60 2024-09-21T03:34:11.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:11.753+0000 7fde16ffd640 1 -- 172.21.15.38:0/76315429 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7fde180634d0 con 0x7fde280b2e60 2024-09-21T03:34:11.755 INFO:teuthology.orchestra.run.smithi038.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-21T03:34:11.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:11.755+0000 7fde2c920640 1 -- 172.21.15.38:0/76315429 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fde08078b90 msgr2=0x7fde0807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:11.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:11.755+0000 7fde2c920640 1 --2- 172.21.15.38:0/76315429 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fde08078b90 0x7fde0807b0a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fde1c034000 tx=0x7fde1c004730 comp rx=0 tx=0).stop 2024-09-21T03:34:11.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:11.755+0000 7fde2c920640 1 -- 172.21.15.38:0/76315429 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fde280b2e60 msgr2=0x7fde281565b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:11.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:11.755+0000 7fde2c920640 1 --2- 172.21.15.38:0/76315429 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fde280b2e60 0x7fde281565b0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fde1800e790 tx=0x7fde1800ec50 comp rx=0 tx=0).stop 2024-09-21T03:34:11.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:11.755+0000 7fde2c920640 1 -- 172.21.15.38:0/76315429 shutdown_connections 2024-09-21T03:34:11.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:11.755+0000 7fde2c920640 1 --2- 172.21.15.38:0/76315429 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fde08078b90 0x7fde0807b0a0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:11.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:11.755+0000 7fde2c920640 1 --2- 172.21.15.38:0/76315429 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fde28162590 0x7fde2815ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:11.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:11.755+0000 7fde2c920640 1 --2- 172.21.15.38:0/76315429 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fde280cfe00 0x7fde2815a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:11.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:11.755+0000 7fde2c920640 1 --2- 172.21.15.38:0/76315429 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fde280b2e60 0x7fde281565b0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:11.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:11.755+0000 7fde2c920640 1 -- 172.21.15.38:0/76315429 >> 172.21.15.38:0/76315429 conn(0x7fde280d4440 msgr2=0x7fde28161a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:11.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:11.755+0000 7fde2c920640 1 -- 172.21.15.38:0/76315429 shutdown_connections 2024-09-21T03:34:11.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:11.755+0000 7fde2c920640 1 -- 172.21.15.38:0/76315429 wait complete. 2024-09-21T03:34:11.768 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-21T03:34:11.768 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T03:34:11.769 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi195.front.sepia.ceph.com 2024-09-21T03:34:11.769 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-09-21T03:34:11.769 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-09-21T03:34:11.769 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-21T03:34:11.769 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-21T03:34:11.769 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T03:34:11.769 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-09-21T03:34:11.769 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:34:11.769 DEBUG:teuthology.orchestra.run.smithi195:> ip addr 2024-09-21T03:34:11.785 INFO:teuthology.orchestra.run.smithi195.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T03:34:11.786 INFO:teuthology.orchestra.run.smithi195.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T03:34:11.786 INFO:teuthology.orchestra.run.smithi195.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T03:34:11.786 INFO:teuthology.orchestra.run.smithi195.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:34:11.786 INFO:teuthology.orchestra.run.smithi195.stdout: inet6 ::1/128 scope host 2024-09-21T03:34:11.786 INFO:teuthology.orchestra.run.smithi195.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:34:11.786 INFO:teuthology.orchestra.run.smithi195.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T03:34:11.786 INFO:teuthology.orchestra.run.smithi195.stdout: link/ether 0c:c4:7a:d9:36:f0 brd ff:ff:ff:ff:ff:ff 2024-09-21T03:34:11.786 INFO:teuthology.orchestra.run.smithi195.stdout: altname enp7s0f0 2024-09-21T03:34:11.786 INFO:teuthology.orchestra.run.smithi195.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T03:34:11.786 INFO:teuthology.orchestra.run.smithi195.stdout: link/ether 0c:c4:7a:d9:36:f1 brd ff:ff:ff:ff:ff:ff 2024-09-21T03:34:11.786 INFO:teuthology.orchestra.run.smithi195.stdout: altname enp7s0f1 2024-09-21T03:34:11.787 INFO:teuthology.orchestra.run.smithi195.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T03:34:11.787 INFO:teuthology.orchestra.run.smithi195.stdout: link/ether 0c:c4:7a:88:80:80 brd ff:ff:ff:ff:ff:ff 2024-09-21T03:34:11.787 INFO:teuthology.orchestra.run.smithi195.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T03:34:11.787 INFO:teuthology.orchestra.run.smithi195.stdout: link/ether 0c:c4:7a:88:80:81 brd ff:ff:ff:ff:ff:ff 2024-09-21T03:34:11.787 INFO:teuthology.orchestra.run.smithi195.stdout: inet 172.21.15.195/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-21T03:34:11.787 INFO:teuthology.orchestra.run.smithi195.stdout: valid_lft 42742sec preferred_lft 42742sec 2024-09-21T03:34:11.787 INFO:teuthology.orchestra.run.smithi195.stdout: inet6 fe80::ec4:7aff:fe88:8081/64 scope link noprefixroute 2024-09-21T03:34:11.787 INFO:teuthology.orchestra.run.smithi195.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:34:11.787 INFO:teuthology.orchestra.run.smithi195.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T03:34:11.787 INFO:teuthology.orchestra.run.smithi195.stdout: link/ether 12:0d:7f:fa:9f:49 brd ff:ff:ff:ff:ff:ff 2024-09-21T03:34:11.787 INFO:teuthology.orchestra.run.smithi195.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-21T03:34:11.787 INFO:teuthology.orchestra.run.smithi195.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:34:11.787 INFO:teuthology.orchestra.run.smithi195.stdout: inet6 fe80::7865:27ff:fe66:1744/64 scope link 2024-09-21T03:34:11.787 INFO:teuthology.orchestra.run.smithi195.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:34:11.788 INFO:teuthology.orchestra.run.smithi195.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T03:34:11.788 INFO:teuthology.orchestra.run.smithi195.stdout: link/ether 12:0d:7f:fa:9f:49 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-21T03:34:11.788 INFO:teuthology.orchestra.run.smithi195.stdout: inet6 fe80::100d:7fff:fefa:9f49/64 scope link tentative 2024-09-21T03:34:11.788 INFO:teuthology.orchestra.run.smithi195.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:34:11.788 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:34:11.788 DEBUG:teuthology.orchestra.run.smithi195:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T03:34:11.850 INFO:teuthology.orchestra.run.smithi195.stdout:1 2024-09-21T03:34:11.851 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:34:11.851 DEBUG:teuthology.orchestra.run.smithi195:> route 2024-09-21T03:34:11.908 INFO:teuthology.orchestra.run.smithi195.stdout:Kernel IP routing table 2024-09-21T03:34:11.908 INFO:teuthology.orchestra.run.smithi195.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T03:34:11.909 INFO:teuthology.orchestra.run.smithi195.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-21T03:34:11.909 INFO:teuthology.orchestra.run.smithi195.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-21T03:34:11.909 INFO:teuthology.orchestra.run.smithi195.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T03:34:11.909 DEBUG:teuthology.orchestra.run.smithi195:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T03:34:11.909 DEBUG:teuthology.orchestra.run.smithi195:> set -e 2024-09-21T03:34:11.909 DEBUG:teuthology.orchestra.run.smithi195:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-21T03:34:11.909 DEBUG:teuthology.orchestra.run.smithi195:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-21T03:34:11.909 DEBUG:teuthology.orchestra.run.smithi195:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-21T03:34:11.909 DEBUG:teuthology.orchestra.run.smithi195:> ') 2024-09-21T03:34:11.995 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:34:11 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T03:34:12.027 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:34:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:34:12.033 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:34:12.033 DEBUG:teuthology.orchestra.run.smithi195:> ip netns list 2024-09-21T03:34:12.048 INFO:teuthology.orchestra.run.smithi195.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-21T03:34:12.049 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:34:12.049 DEBUG:teuthology.orchestra.run.smithi195:> ip netns list-id 2024-09-21T03:34:12.104 INFO:teuthology.orchestra.run.smithi195.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-09-21T03:34:12.104 DEBUG:teuthology.orchestra.run.smithi195:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T03:34:12.104 DEBUG:teuthology.orchestra.run.smithi195:> set -e 2024-09-21T03:34:12.104 DEBUG:teuthology.orchestra.run.smithi195:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-21T03:34:12.104 DEBUG:teuthology.orchestra.run.smithi195:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-09-21T03:34:12.104 DEBUG:teuthology.orchestra.run.smithi195:> ') 2024-09-21T03:34:12.191 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:34:12 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T03:34:12.214 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:34:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:34:12.220 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:34:12.220 DEBUG:teuthology.orchestra.run.smithi195:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-09-21T03:34:12.244 INFO:teuthology.orchestra.run.smithi195.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T03:34:12.244 INFO:teuthology.orchestra.run.smithi195.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T03:34:12.245 INFO:teuthology.orchestra.run.smithi195.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T03:34:12.245 INFO:teuthology.orchestra.run.smithi195.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:34:12.245 INFO:teuthology.orchestra.run.smithi195.stdout: inet6 ::1/128 scope host 2024-09-21T03:34:12.245 INFO:teuthology.orchestra.run.smithi195.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:34:12.245 INFO:teuthology.orchestra.run.smithi195.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T03:34:12.245 INFO:teuthology.orchestra.run.smithi195.stdout: link/ether 36:1d:da:5a:1b:7d brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-21T03:34:12.245 INFO:teuthology.orchestra.run.smithi195.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-21T03:34:12.245 INFO:teuthology.orchestra.run.smithi195.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:34:12.245 INFO:teuthology.orchestra.run.smithi195.stdout: inet6 fe80::341d:daff:fe5a:1b7d/64 scope link 2024-09-21T03:34:12.245 INFO:teuthology.orchestra.run.smithi195.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:34:12.246 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:34:12.246 DEBUG:teuthology.orchestra.run.smithi195:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-09-21T03:34:12.313 INFO:teuthology.orchestra.run.smithi195.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T03:34:12.313 INFO:teuthology.orchestra.run.smithi195.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T03:34:12.313 INFO:teuthology.orchestra.run.smithi195.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T03:34:12.313 INFO:teuthology.orchestra.run.smithi195.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:34:12.313 INFO:teuthology.orchestra.run.smithi195.stdout: inet6 ::1/128 scope host 2024-09-21T03:34:12.314 INFO:teuthology.orchestra.run.smithi195.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:34:12.314 INFO:teuthology.orchestra.run.smithi195.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T03:34:12.314 INFO:teuthology.orchestra.run.smithi195.stdout: link/ether 36:1d:da:5a:1b:7d brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-21T03:34:12.314 INFO:teuthology.orchestra.run.smithi195.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-21T03:34:12.314 INFO:teuthology.orchestra.run.smithi195.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:34:12.314 INFO:teuthology.orchestra.run.smithi195.stdout: inet6 fe80::341d:daff:fe5a:1b7d/64 scope link 2024-09-21T03:34:12.314 INFO:teuthology.orchestra.run.smithi195.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:34:12.315 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-09-21T03:34:12.315 DEBUG:teuthology.orchestra.run.smithi195:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T03:34:12.315 DEBUG:teuthology.orchestra.run.smithi195:> set -e 2024-09-21T03:34:12.315 DEBUG:teuthology.orchestra.run.smithi195:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-09-21T03:34:12.315 DEBUG:teuthology.orchestra.run.smithi195:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-09-21T03:34:12.316 DEBUG:teuthology.orchestra.run.smithi195:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-09-21T03:34:12.316 DEBUG:teuthology.orchestra.run.smithi195:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-09-21T03:34:12.316 DEBUG:teuthology.orchestra.run.smithi195:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-09-21T03:34:12.316 DEBUG:teuthology.orchestra.run.smithi195:> ') 2024-09-21T03:34:12.404 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:34:12 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T03:34:12.461 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:34:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:34:12.467 DEBUG:teuthology.orchestra.run.smithi195:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T03:34:12.467 DEBUG:teuthology.orchestra.run.smithi195:> set -e 2024-09-21T03:34:12.467 DEBUG:teuthology.orchestra.run.smithi195:> sudo ip link set brx.1 up 2024-09-21T03:34:12.467 DEBUG:teuthology.orchestra.run.smithi195:> sudo ip link set dev brx.1 master ceph-brx 2024-09-21T03:34:12.467 DEBUG:teuthology.orchestra.run.smithi195:> ') 2024-09-21T03:34:12.514 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:34:12 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T03:34:12.537 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:34:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:34:12.544 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T03:34:12.544 DEBUG:teuthology.orchestra.run.smithi195:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-09-21T03:34:12.559 INFO:teuthology.orchestra.run.smithi195.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-09-21T03:34:12.560 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T03:34:12.560 DEBUG:teuthology.orchestra.run.smithi195:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-09-21T03:34:12.614 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-21T03:34:12.614 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:34:12.615 DEBUG:teuthology.orchestra.run.smithi195:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-09-21T03:34:12.709 INFO:teuthology.orchestra.run.smithi195.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-09-21T03:34:12.709 INFO:teuthology.orchestra.run.smithi195.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-09-21T03:34:12.709 INFO:teuthology.orchestra.run.smithi195.stdout:invalid new device string format 2024-09-21T03:34:12.720 INFO:teuthology.orchestra.run.smithi195.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-09-21T03:34:12.720 INFO:teuthology.orchestra.run.smithi195.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-09-21T03:34:12.720 INFO:teuthology.orchestra.run.smithi195.stdout:invalid new device string format 2024-09-21T03:34:12.720 INFO:teuthology.orchestra.run.smithi195.stdout:mount.ceph: resolved to: "172.21.15.38:3300,172.21.15.195:3300,172.21.15.195:3301" 2024-09-21T03:34:12.720 INFO:teuthology.orchestra.run.smithi195.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T03:34:12.725 INFO:teuthology.orchestra.run.smithi195.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-09-21T03:34:12.725 INFO:teuthology.orchestra.run.smithi195.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-09-21T03:34:12.726 INFO:teuthology.orchestra.run.smithi195.stdout:invalid new device string format 2024-09-21T03:34:12.726 INFO:teuthology.orchestra.run.smithi195.stdout:mount.ceph: resolved to: "172.21.15.38:3300,172.21.15.195:3300,172.21.15.195:3301" 2024-09-21T03:34:12.726 INFO:teuthology.orchestra.run.smithi195.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T03:34:12.726 INFO:teuthology.orchestra.run.smithi195.stdout:mount.ceph: resolved to: "172.21.15.38:6789,172.21.15.195:6789,172.21.15.195:6790" 2024-09-21T03:34:12.726 INFO:teuthology.orchestra.run.smithi195.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.38:6789,172.21.15.195:6789,172.21.15.195:6790:/ 2024-09-21T03:34:12.726 INFO:teuthology.orchestra.run.smithi195.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs,key=3,fsid=9d50d66f-656b-4fe2-8e3b-fe53b79c4d43" will pass to kernel 2024-09-21T03:34:12.726 INFO:teuthology.orchestra.run.smithi195.stdout:mount: /home/ubuntu/cephtest/mnt.3 does not contain SELinux labels. 2024-09-21T03:34:12.727 INFO:teuthology.orchestra.run.smithi195.stdout: You just mounted a file system that supports labels which does not 2024-09-21T03:34:12.727 INFO:teuthology.orchestra.run.smithi195.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-21T03:34:12.727 INFO:teuthology.orchestra.run.smithi195.stdout: applications will generate AVC messages and not be allowed access to 2024-09-21T03:34:12.727 INFO:teuthology.orchestra.run.smithi195.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-21T03:34:12.728 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T03:34:12.728 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:34:12.728 DEBUG:teuthology.orchestra.run.smithi195:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-09-21T03:34:12.752 DEBUG:teuthology.orchestra.run.smithi195:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T03:34:12.841 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:34:12 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T03:34:12.843 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:34:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:34:12.848 INFO:teuthology.orchestra.run.smithi195.stdout:client4792 2024-09-21T03:34:12.848 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-09-21T03:34:12.849 DEBUG:teuthology.orchestra.run.smithi195:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:34:12.864 DEBUG:teuthology.orchestra.run.smithi195:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T03:34:12.952 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:34:12 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T03:34:12.954 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:34:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:34:12.960 INFO:teuthology.orchestra.run.smithi195.stdout:client4792 2024-09-21T03:34:12.960 DEBUG:teuthology.orchestra.run.smithi195:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T03:34:13.008 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:34:13 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T03:34:13.010 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:34:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:34:13.015 INFO:teuthology.orchestra.run.smithi195.stdout:client4792 2024-09-21T03:34:13.016 DEBUG:teuthology.orchestra.run.smithi195:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/9d50d66f-656b-4fe2-8e3b-fe53b79c4d43.client4792/status') 2024-09-21T03:34:13.063 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:34:13 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T03:34:13.073 INFO:teuthology.orchestra.run.smithi195.stdout:instance: client.4792 (3)192.168.0.2:0/2898392597 2024-09-21T03:34:13.073 INFO:teuthology.orchestra.run.smithi195.stdout:blocklisted: false 2024-09-21T03:34:13.073 INFO:teuthology.orchestra.run.smithi195.stderr:0+1 records in 2024-09-21T03:34:13.073 INFO:teuthology.orchestra.run.smithi195.stderr:0+1 records out 2024-09-21T03:34:13.074 INFO:teuthology.orchestra.run.smithi195.stderr:69 bytes copied, 2.2248e-05 s, 3.1 MB/s 2024-09-21T03:34:13.074 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T03:34:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:34:13.080 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-09-21T03:34:14.875 INFO:tasks.cephfs_test_runner:Starting test: test_damaged_dentry (tasks.cephfs.test_damage.TestDamage) 2024-09-21T03:34:14.876 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T03:34:15.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:15.008+0000 7f8a8ab4a640 1 -- 172.21.15.38:0/3289716522 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8a6c004800 msgr2=0x7f8a6c004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:15.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:15.008+0000 7f8a8ab4a640 1 --2- 172.21.15.38:0/3289716522 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8a6c004800 0x7f8a6c004c10 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f8a78009d00 tx=0x7f8a6c005830 comp rx=0 tx=0).stop 2024-09-21T03:34:15.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:15.008+0000 7f8a8ab4a640 1 -- 172.21.15.38:0/3289716522 shutdown_connections 2024-09-21T03:34:15.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:15.008+0000 7f8a8ab4a640 1 --2- 172.21.15.38:0/3289716522 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8a6c004800 0x7f8a6c004c10 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:15.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:15.008+0000 7f8a8ab4a640 1 -- 172.21.15.38:0/3289716522 >> 172.21.15.38:0/3289716522 conn(0x7f8a840d4440 msgr2=0x7f8a840d4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:15.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:15.008+0000 7f8a8ab4a640 1 -- 172.21.15.38:0/3289716522 shutdown_connections 2024-09-21T03:34:15.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:15.008+0000 7f8a8ab4a640 1 -- 172.21.15.38:0/3289716522 wait complete. 2024-09-21T03:34:15.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:15.010+0000 7f8a8ab4a640 1 Processor -- start 2024-09-21T03:34:15.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:15.010+0000 7f8a8ab4a640 1 -- start start 2024-09-21T03:34:15.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:15.011+0000 7f8a8ab4a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8a6c004800 0x7f8a841f8f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:15.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:15.011+0000 7f8a8ab4a640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8a8415b700 0x7f8a840c96c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:15.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:15.011+0000 7f8a8ab4a640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8a841f9450 0x7f8a840c9c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:15.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:15.011+0000 7f8a8ab4a640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f8a84162d30 con 0x7f8a6c004800 2024-09-21T03:34:15.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:15.011+0000 7f8a8ab4a640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f8a84162bb0 con 0x7f8a8415b700 2024-09-21T03:34:15.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:15.011+0000 7f8a8ab4a640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f8a840d2a20 con 0x7f8a841f9450 2024-09-21T03:34:15.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:15.011+0000 7f8a888e6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8a6c004800 0x7f8a841f8f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:15.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:15.011+0000 7f8a888e6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8a6c004800 0x7f8a841f8f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54264/0 (socket says 172.21.15.38:54264) 2024-09-21T03:34:15.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:15.011+0000 7f8a888e6640 1 -- 172.21.15.38:0/2201006405 learned_addr learned my addr 172.21.15.38:0/2201006405 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:34:15.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:15.012+0000 7f8a890e7640 1 --2- 172.21.15.38:0/2201006405 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8a841f9450 0x7f8a840c9c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:15.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:15.012+0000 7f8a83fff640 1 --2- 172.21.15.38:0/2201006405 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8a8415b700 0x7f8a840c96c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:15.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:15.012+0000 7f8a888e6640 1 -- 172.21.15.38:0/2201006405 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8a841f9450 msgr2=0x7f8a840c9c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:15.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:15.012+0000 7f8a888e6640 1 --2- 172.21.15.38:0/2201006405 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8a841f9450 0x7f8a840c9c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:15.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:15.012+0000 7f8a888e6640 1 -- 172.21.15.38:0/2201006405 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8a8415b700 msgr2=0x7f8a840c96c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:15.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:15.012+0000 7f8a888e6640 1 --2- 172.21.15.38:0/2201006405 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8a8415b700 0x7f8a840c96c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:15.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:15.012+0000 7f8a888e6640 1 -- 172.21.15.38:0/2201006405 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a841f5670 con 0x7f8a6c004800 2024-09-21T03:34:15.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:15.012+0000 7f8a890e7640 1 --2- 172.21.15.38:0/2201006405 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8a841f9450 0x7f8a840c9c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:34:15.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:15.012+0000 7f8a888e6640 1 --2- 172.21.15.38:0/2201006405 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8a6c004800 0x7f8a841f8f10 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f8a74001920 tx=0x7f8a740039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:15.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:15.012+0000 7f8a817fa640 1 -- 172.21.15.38:0/2201006405 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a74015070 con 0x7f8a6c004800 2024-09-21T03:34:15.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:15.012+0000 7f8a817fa640 1 -- 172.21.15.38:0/2201006405 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8a74011900 con 0x7f8a6c004800 2024-09-21T03:34:15.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:15.012+0000 7f8a817fa640 1 -- 172.21.15.38:0/2201006405 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a74011d50 con 0x7f8a6c004800 2024-09-21T03:34:15.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:15.012+0000 7f8a8ab4a640 1 -- 172.21.15.38:0/2201006405 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a841f5950 con 0x7f8a6c004800 2024-09-21T03:34:15.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:15.012+0000 7f8a8ab4a640 1 -- 172.21.15.38:0/2201006405 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a841f5f10 con 0x7f8a6c004800 2024-09-21T03:34:15.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:15.013+0000 7f8a8ab4a640 1 -- 172.21.15.38:0/2201006405 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a840d2f80 con 0x7f8a6c004800 2024-09-21T03:34:15.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:15.014+0000 7f8a817fa640 1 -- 172.21.15.38:0/2201006405 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f8a74018070 con 0x7f8a6c004800 2024-09-21T03:34:15.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:15.014+0000 7f8a817fa640 1 --2- 172.21.15.38:0/2201006405 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8a70078b90 0x7f8a7007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:15.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:15.015+0000 7f8a817fa640 1 -- 172.21.15.38:0/2201006405 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f8a7409bc80 con 0x7f8a6c004800 2024-09-21T03:34:15.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:15.015+0000 7f8a83fff640 1 --2- 172.21.15.38:0/2201006405 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8a70078b90 0x7f8a7007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:15.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:15.015+0000 7f8a83fff640 1 --2- 172.21.15.38:0/2201006405 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8a70078b90 0x7f8a7007b0a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f8a78035000 tx=0x7f8a780046c0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:15.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:15.019+0000 7f8a817fa640 1 -- 172.21.15.38:0/2201006405 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f8a84162ee0 con 0x7f8a6c004800 2024-09-21T03:34:15.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:15.209+0000 7f8a8ab4a640 1 -- 172.21.15.38:0/2201006405 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8a840ce2f0 con 0x7f8a6c004800 2024-09-21T03:34:15.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:15.210+0000 7f8a817fa640 1 -- 172.21.15.38:0/2201006405 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18626 (secure 0 0 0) 0x7f8a74063e20 con 0x7f8a6c004800 2024-09-21T03:34:15.212 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:34:15.212 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":26,"fsid":"9d50d66f-656b-4fe2-8e3b-fe53b79c4d43","created":"2024-09-21T03:33:10.677865+0000","modified":"2024-09-21T03:33:40.665991+0000","last_up_change":"2024-09-21T03:33:24.023421+0000","last_in_change":"2024-09-21T03:33:14.296193+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T03:33:22.992573+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T03:33:33.530201+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T03:33:34.585875+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T03:33:38.268346+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"dbab183f-7e63-48b8-960a-ae55fd5bf565","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6801","nonce":2381639433}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6803","nonce":2381639433}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6807","nonce":2381639433}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6805","nonce":2381639433}]},"public_addr":"172.21.15.38:6801/2381639433","cluster_addr":"172.21.15.38:6803/2381639433","heartbeat_back_addr":"172.21.15.38:6807/2381639433","heartbeat_front_addr":"172.21.15.38:6805/2381639433","state":["exists","up"]},{"osd":1,"uuid":"fa1c0e8a-ebc1-4359-8146-bd5342abbc16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6817","nonce":1491818732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6819","nonce":1491818732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6823","nonce":1491818732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6821","nonce":1491818732}]},"public_addr":"172.21.15.38:6817/1491818732","cluster_addr":"172.21.15.38:6819/1491818732","heartbeat_back_addr":"172.21.15.38:6823/1491818732","heartbeat_front_addr":"172.21.15.38:6821/1491818732","state":["exists","up"]},{"osd":2,"uuid":"2ed3cd2d-385d-4fc6-9869-214552a84803","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6825","nonce":2577858902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6827","nonce":2577858902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6831","nonce":2577858902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6829","nonce":2577858902}]},"public_addr":"172.21.15.38:6825/2577858902","cluster_addr":"172.21.15.38:6827/2577858902","heartbeat_back_addr":"172.21.15.38:6831/2577858902","heartbeat_front_addr":"172.21.15.38:6829/2577858902","state":["exists","up"]},{"osd":3,"uuid":"62d1d95e-919d-4250-be71-1d34cfa70dbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6809","nonce":1286810294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6811","nonce":1286810294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6815","nonce":1286810294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6813","nonce":1286810294}]},"public_addr":"172.21.15.38:6809/1286810294","cluster_addr":"172.21.15.38:6811/1286810294","heartbeat_back_addr":"172.21.15.38:6815/1286810294","heartbeat_front_addr":"172.21.15.38:6813/1286810294","state":["exists","up"]},{"osd":4,"uuid":"7c9f9314-99d7-4d73-8392-37809a9f54b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6803","nonce":3401837571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6805","nonce":3401837571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6809","nonce":3401837571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6807","nonce":3401837571}]},"public_addr":"172.21.15.195:6803/3401837571","cluster_addr":"172.21.15.195:6805/3401837571","heartbeat_back_addr":"172.21.15.195:6809/3401837571","heartbeat_front_addr":"172.21.15.195:6807/3401837571","state":["exists","up"]},{"osd":5,"uuid":"bde4fe9f-8414-4bba-bf4a-43973d1976bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6811","nonce":2080960180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6813","nonce":2080960180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6817","nonce":2080960180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6815","nonce":2080960180}]},"public_addr":"172.21.15.195:6811/2080960180","cluster_addr":"172.21.15.195:6813/2080960180","heartbeat_back_addr":"172.21.15.195:6817/2080960180","heartbeat_front_addr":"172.21.15.195:6815/2080960180","state":["exists","up"]},{"osd":6,"uuid":"6ee6d7f4-1a5f-4a4c-ac16-48fe05f6a002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6827","nonce":4055825675}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6829","nonce":4055825675}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6833","nonce":4055825675}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6831","nonce":4055825675}]},"public_addr":"172.21.15.195:6827/4055825675","cluster_addr":"172.21.15.195:6829/4055825675","heartbeat_back_addr":"172.21.15.195:6833/4055825675","heartbeat_front_addr":"172.21.15.195:6831/4055825675","state":["exists","up"]},{"osd":7,"uuid":"efeb5f51-329a-4484-8290-92e7d3503d05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6819","nonce":1606106342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6821","nonce":1606106342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6825","nonce":1606106342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6823","nonce":1606106342}]},"public_addr":"172.21.15.195:6819/1606106342","cluster_addr":"172.21.15.195:6821/1606106342","heartbeat_back_addr":"172.21.15.195:6825/1606106342","heartbeat_front_addr":"172.21.15.195:6823/1606106342","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.901343+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:21.115003+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.901623+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.835238+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.132199+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:19.817560+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.829690+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:21.087300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T03:34:15.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:15.213+0000 7f8a8ab4a640 1 -- 172.21.15.38:0/2201006405 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8a70078b90 msgr2=0x7f8a7007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:15.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:15.213+0000 7f8a8ab4a640 1 --2- 172.21.15.38:0/2201006405 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8a70078b90 0x7f8a7007b0a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f8a78035000 tx=0x7f8a780046c0 comp rx=0 tx=0).stop 2024-09-21T03:34:15.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:15.213+0000 7f8a8ab4a640 1 -- 172.21.15.38:0/2201006405 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8a6c004800 msgr2=0x7f8a841f8f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:15.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:15.213+0000 7f8a8ab4a640 1 --2- 172.21.15.38:0/2201006405 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8a6c004800 0x7f8a841f8f10 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f8a74001920 tx=0x7f8a740039c0 comp rx=0 tx=0).stop 2024-09-21T03:34:15.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:15.213+0000 7f8a8ab4a640 1 -- 172.21.15.38:0/2201006405 shutdown_connections 2024-09-21T03:34:15.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:15.213+0000 7f8a8ab4a640 1 --2- 172.21.15.38:0/2201006405 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8a70078b90 0x7f8a7007b0a0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:15.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:15.213+0000 7f8a8ab4a640 1 --2- 172.21.15.38:0/2201006405 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8a841f9450 0x7f8a840c9c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:15.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:15.213+0000 7f8a8ab4a640 1 --2- 172.21.15.38:0/2201006405 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8a8415b700 0x7f8a840c96c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:15.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:15.213+0000 7f8a8ab4a640 1 --2- 172.21.15.38:0/2201006405 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8a6c004800 0x7f8a841f8f10 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:15.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:15.213+0000 7f8a8ab4a640 1 -- 172.21.15.38:0/2201006405 >> 172.21.15.38:0/2201006405 conn(0x7f8a840d4440 msgr2=0x7f8a840d4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:15.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:15.213+0000 7f8a8ab4a640 1 -- 172.21.15.38:0/2201006405 shutdown_connections 2024-09-21T03:34:15.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:15.213+0000 7f8a8ab4a640 1 -- 172.21.15.38:0/2201006405 wait complete. 2024-09-21T03:34:15.226 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T03:33:22.992573+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-21T03:33:33.530201+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-21T03:33:34.585875+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-09-21T03:33:38.268346+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-21T03:34:15.227 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-21T03:34:15.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:15.346+0000 7fc2561ba640 1 -- 172.21.15.38:0/1140180677 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc25015ee80 msgr2=0x7fc2500d0170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:15.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:15.346+0000 7fc2561ba640 1 --2- 172.21.15.38:0/1140180677 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc25015ee80 0x7fc2500d0170 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fc2500d06b0 tx=0x7fc24002a410 comp rx=0 tx=0).stop 2024-09-21T03:34:15.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:15.346+0000 7fc2561ba640 1 -- 172.21.15.38:0/1140180677 shutdown_connections 2024-09-21T03:34:15.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:15.346+0000 7fc2561ba640 1 --2- 172.21.15.38:0/1140180677 >> v2:172.21.15.195:3300/0 conn(0x7fc2500b9ad0 0x7fc2500d0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:15.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:15.346+0000 7fc2561ba640 1 --2- 172.21.15.38:0/1140180677 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc25015ee80 0x7fc2500d0170 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:15.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:15.346+0000 7fc2561ba640 1 -- 172.21.15.38:0/1140180677 >> 172.21.15.38:0/1140180677 conn(0x7fc25009d830 msgr2=0x7fc2500b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:15.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:15.346+0000 7fc2561ba640 1 -- 172.21.15.38:0/1140180677 shutdown_connections 2024-09-21T03:34:15.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:15.346+0000 7fc2561ba640 1 -- 172.21.15.38:0/1140180677 wait complete. 2024-09-21T03:34:15.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:15.348+0000 7fc2561ba640 1 Processor -- start 2024-09-21T03:34:15.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:15.348+0000 7fc2561ba640 1 -- start start 2024-09-21T03:34:15.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:15.349+0000 7fc2561ba640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc2500b9ad0 0x7fc25006cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:15.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:15.349+0000 7fc2561ba640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc2500d1120 0x7fc25015bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:15.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:15.349+0000 7fc2561ba640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc25015ee80 0x7fc2501578f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:15.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:15.349+0000 7fc2561ba640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fc2500b6700 con 0x7fc25015ee80 2024-09-21T03:34:15.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:15.349+0000 7fc2561ba640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fc2500b6580 con 0x7fc2500b9ad0 2024-09-21T03:34:15.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:15.349+0000 7fc2561ba640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fc2500b6880 con 0x7fc2500d1120 2024-09-21T03:34:15.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:15.349+0000 7fc24ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc25015ee80 0x7fc2501578f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:15.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:15.349+0000 7fc24ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc25015ee80 0x7fc2501578f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54284/0 (socket says 172.21.15.38:54284) 2024-09-21T03:34:15.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:15.349+0000 7fc24ffff640 1 -- 172.21.15.38:0/232689436 learned_addr learned my addr 172.21.15.38:0/232689436 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:34:15.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:15.350+0000 7fc24f7fe640 1 --2- 172.21.15.38:0/232689436 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc2500b9ad0 0x7fc25006cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:15.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:15.350+0000 7fc24effd640 1 --2- 172.21.15.38:0/232689436 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc2500d1120 0x7fc25015bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:15.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:15.350+0000 7fc24ffff640 1 -- 172.21.15.38:0/232689436 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc2500d1120 msgr2=0x7fc25015bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:15.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:15.350+0000 7fc24ffff640 1 --2- 172.21.15.38:0/232689436 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc2500d1120 0x7fc25015bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:15.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:15.350+0000 7fc24ffff640 1 -- 172.21.15.38:0/232689436 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc2500b9ad0 msgr2=0x7fc25006cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:15.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:15.350+0000 7fc24ffff640 1 --2- 172.21.15.38:0/232689436 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc2500b9ad0 0x7fc25006cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:15.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:15.350+0000 7fc24ffff640 1 -- 172.21.15.38:0/232689436 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2501f6a20 con 0x7fc25015ee80 2024-09-21T03:34:15.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:15.350+0000 7fc24effd640 1 --2- 172.21.15.38:0/232689436 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc2500d1120 0x7fc25015bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:34:15.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:15.350+0000 7fc24ffff640 1 --2- 172.21.15.38:0/232689436 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc25015ee80 0x7fc2501578f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fc24800c760 tx=0x7fc24800cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:15.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:15.350+0000 7fc237fff640 1 -- 172.21.15.38:0/232689436 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc248003da0 con 0x7fc25015ee80 2024-09-21T03:34:15.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:15.350+0000 7fc237fff640 1 -- 172.21.15.38:0/232689436 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc24800ce10 con 0x7fc25015ee80 2024-09-21T03:34:15.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:15.350+0000 7fc2561ba640 1 -- 172.21.15.38:0/232689436 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2501f6d00 con 0x7fc25015ee80 2024-09-21T03:34:15.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:15.350+0000 7fc237fff640 1 -- 172.21.15.38:0/232689436 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2480052d0 con 0x7fc25015ee80 2024-09-21T03:34:15.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:15.350+0000 7fc2561ba640 1 -- 172.21.15.38:0/232689436 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2501f72c0 con 0x7fc25015ee80 2024-09-21T03:34:15.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:15.351+0000 7fc2561ba640 1 -- 172.21.15.38:0/232689436 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2501f6790 con 0x7fc25015ee80 2024-09-21T03:34:15.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:15.352+0000 7fc237fff640 1 -- 172.21.15.38:0/232689436 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc248005470 con 0x7fc25015ee80 2024-09-21T03:34:15.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:15.352+0000 7fc237fff640 1 --2- 172.21.15.38:0/232689436 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc23c078bc0 0x7fc23c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:15.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:15.353+0000 7fc237fff640 1 -- 172.21.15.38:0/232689436 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fc24809ace0 con 0x7fc25015ee80 2024-09-21T03:34:15.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:15.353+0000 7fc24f7fe640 1 --2- 172.21.15.38:0/232689436 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc23c078bc0 0x7fc23c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:15.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:15.353+0000 7fc24f7fe640 1 --2- 172.21.15.38:0/232689436 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc23c078bc0 0x7fc23c07b0d0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fc240034000 tx=0x7fc240004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:15.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:15.357+0000 7fc237fff640 1 -- 172.21.15.38:0/232689436 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fc2501592d0 con 0x7fc25015ee80 2024-09-21T03:34:15.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:15.549+0000 7fc2561ba640 1 -- 172.21.15.38:0/232689436 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc2500b6d50 con 0x7fc25015ee80 2024-09-21T03:34:15.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:15.549+0000 7fc237fff640 1 -- 172.21.15.38:0/232689436 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) ==== 93+0+10 (secure 0 0 0) 0x7fc248062e70 con 0x7fc25015ee80 2024-09-21T03:34:15.551 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 1 2024-09-21T03:34:15.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:15.551+0000 7fc2561ba640 1 -- 172.21.15.38:0/232689436 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc23c078bc0 msgr2=0x7fc23c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:15.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:15.551+0000 7fc2561ba640 1 --2- 172.21.15.38:0/232689436 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc23c078bc0 0x7fc23c07b0d0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fc240034000 tx=0x7fc240004730 comp rx=0 tx=0).stop 2024-09-21T03:34:15.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:15.551+0000 7fc2561ba640 1 -- 172.21.15.38:0/232689436 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc25015ee80 msgr2=0x7fc2501578f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:15.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:15.551+0000 7fc2561ba640 1 --2- 172.21.15.38:0/232689436 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc25015ee80 0x7fc2501578f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fc24800c760 tx=0x7fc24800cc20 comp rx=0 tx=0).stop 2024-09-21T03:34:15.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:15.551+0000 7fc2561ba640 1 -- 172.21.15.38:0/232689436 shutdown_connections 2024-09-21T03:34:15.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:15.551+0000 7fc2561ba640 1 --2- 172.21.15.38:0/232689436 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc25015ee80 0x7fc2501578f0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:15.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:15.551+0000 7fc2561ba640 1 --2- 172.21.15.38:0/232689436 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc23c078bc0 0x7fc23c07b0d0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:15.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:15.551+0000 7fc2561ba640 1 --2- 172.21.15.38:0/232689436 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc2500d1120 0x7fc25015bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:15.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:15.551+0000 7fc2561ba640 1 --2- 172.21.15.38:0/232689436 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc2500b9ad0 0x7fc25006cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:15.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:15.551+0000 7fc2561ba640 1 -- 172.21.15.38:0/232689436 >> 172.21.15.38:0/232689436 conn(0x7fc25009d830 msgr2=0x7fc25015ca10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:15.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:15.552+0000 7fc2561ba640 1 -- 172.21.15.38:0/232689436 shutdown_connections 2024-09-21T03:34:15.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:15.552+0000 7fc2561ba640 1 -- 172.21.15.38:0/232689436 wait complete. 2024-09-21T03:34:15.565 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-21T03:34:15.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:15.682+0000 7f45ff0d5640 1 -- 172.21.15.38:0/1106473762 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f45f815ee80 msgr2=0x7f45f80d0170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:15.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:15.682+0000 7f45ff0d5640 1 --2- 172.21.15.38:0/1106473762 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f45f815ee80 0x7f45f80d0170 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f45e8009830 tx=0x7f45e802a480 comp rx=0 tx=0).stop 2024-09-21T03:34:15.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:15.682+0000 7f45ff0d5640 1 -- 172.21.15.38:0/1106473762 shutdown_connections 2024-09-21T03:34:15.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:15.682+0000 7f45ff0d5640 1 --2- 172.21.15.38:0/1106473762 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f45f80ceb90 0x7f45f80cefa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:15.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:15.682+0000 7f45ff0d5640 1 --2- 172.21.15.38:0/1106473762 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f45f815ee80 0x7f45f80d0170 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:15.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:15.682+0000 7f45ff0d5640 1 -- 172.21.15.38:0/1106473762 >> 172.21.15.38:0/1106473762 conn(0x7f45f809d830 msgr2=0x7f45f80b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:15.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:15.682+0000 7f45ff0d5640 1 -- 172.21.15.38:0/1106473762 shutdown_connections 2024-09-21T03:34:15.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:15.682+0000 7f45ff0d5640 1 -- 172.21.15.38:0/1106473762 wait complete. 2024-09-21T03:34:15.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:15.684+0000 7f45ff0d5640 1 Processor -- start 2024-09-21T03:34:15.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:15.684+0000 7f45ff0d5640 1 -- start start 2024-09-21T03:34:15.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:15.686+0000 7f45ff0d5640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f45f80b9ad0 0x7f45f81594b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:15.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:15.686+0000 7f45ff0d5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f45f80ceb90 0x7f45f81578f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:15.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:15.686+0000 7f45ff0d5640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f45f815ee80 0x7f45f806d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:15.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:15.686+0000 7f45ff0d5640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f45f80b6700 con 0x7f45f80ceb90 2024-09-21T03:34:15.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:15.686+0000 7f45ff0d5640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f45f80b6580 con 0x7f45f815ee80 2024-09-21T03:34:15.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:15.686+0000 7f45ff0d5640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f45f80b6880 con 0x7f45f80b9ad0 2024-09-21T03:34:15.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:15.686+0000 7f45f7fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f45f80ceb90 0x7f45f81578f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:15.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:15.686+0000 7f45f7fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f45f80ceb90 0x7f45f81578f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54286/0 (socket says 172.21.15.38:54286) 2024-09-21T03:34:15.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:15.686+0000 7f45f7fff640 1 -- 172.21.15.38:0/3989779823 learned_addr learned my addr 172.21.15.38:0/3989779823 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:34:15.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:15.686+0000 7f45fce71640 1 --2- 172.21.15.38:0/3989779823 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f45f80b9ad0 0x7f45f81594b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:15.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:15.686+0000 7f45fd672640 1 --2- 172.21.15.38:0/3989779823 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f45f815ee80 0x7f45f806d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:15.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:15.686+0000 7f45f7fff640 1 -- 172.21.15.38:0/3989779823 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f45f80b9ad0 msgr2=0x7f45f81594b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:15.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:15.686+0000 7f45f7fff640 1 --2- 172.21.15.38:0/3989779823 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f45f80b9ad0 0x7f45f81594b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:15.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:15.686+0000 7f45f7fff640 1 -- 172.21.15.38:0/3989779823 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f45f815ee80 msgr2=0x7f45f806d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:15.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:15.686+0000 7f45f7fff640 1 --2- 172.21.15.38:0/3989779823 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f45f815ee80 0x7f45f806d510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:15.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:15.686+0000 7f45f7fff640 1 -- 172.21.15.38:0/3989779823 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45f81f9890 con 0x7f45f80ceb90 2024-09-21T03:34:15.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:15.687+0000 7f45f7fff640 1 --2- 172.21.15.38:0/3989779823 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f45f80ceb90 0x7f45f81578f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f45ec00c3c0 tx=0x7f45ec00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:15.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:15.687+0000 7f45f57fa640 1 -- 172.21.15.38:0/3989779823 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45ec017070 con 0x7f45f80ceb90 2024-09-21T03:34:15.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:15.687+0000 7f45f57fa640 1 -- 172.21.15.38:0/3989779823 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f45ec0053c0 con 0x7f45f80ceb90 2024-09-21T03:34:15.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:15.687+0000 7f45ff0d5640 1 -- 172.21.15.38:0/3989779823 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45f81f9b70 con 0x7f45f80ceb90 2024-09-21T03:34:15.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:15.687+0000 7f45f57fa640 1 -- 172.21.15.38:0/3989779823 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45ec003850 con 0x7f45f80ceb90 2024-09-21T03:34:15.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:15.687+0000 7f45ff0d5640 1 -- 172.21.15.38:0/3989779823 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45f806cff0 con 0x7f45f80ceb90 2024-09-21T03:34:15.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:15.688+0000 7f45ff0d5640 1 -- 172.21.15.38:0/3989779823 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45f81f9600 con 0x7f45f80ceb90 2024-09-21T03:34:15.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:15.689+0000 7f45f57fa640 1 -- 172.21.15.38:0/3989779823 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f45ec01a070 con 0x7f45f80ceb90 2024-09-21T03:34:15.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:15.689+0000 7f45f57fa640 1 --2- 172.21.15.38:0/3989779823 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f45e4078b90 0x7f45e407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:15.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:15.689+0000 7f45f57fa640 1 -- 172.21.15.38:0/3989779823 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6619+0+0 (secure 0 0 0) 0x7f45ec09a2f0 con 0x7f45f80ceb90 2024-09-21T03:34:15.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:15.690+0000 7f45fce71640 1 --2- 172.21.15.38:0/3989779823 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f45e4078b90 0x7f45e407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:15.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:15.690+0000 7f45fce71640 1 --2- 172.21.15.38:0/3989779823 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f45e4078b90 0x7f45e407b0a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f45e8034000 tx=0x7f45e8004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:15.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:15.694+0000 7f45f57fa640 1 -- 172.21.15.38:0/3989779823 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f45f81592d0 con 0x7f45f80ceb90 2024-09-21T03:34:15.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:15.880+0000 7f45ff0d5640 1 -- 172.21.15.38:0/3989779823 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f45f806cde0 con 0x7f45f80ceb90 2024-09-21T03:34:15.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:15.881+0000 7f45f57fa640 1 -- 172.21.15.38:0/3989779823 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) ==== 104+0+11 (secure 0 0 0) 0x7f45ec061d50 con 0x7f45f80ceb90 2024-09-21T03:34:15.883 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 32 2024-09-21T03:34:15.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:15.883+0000 7f45ff0d5640 1 -- 172.21.15.38:0/3989779823 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f45e4078b90 msgr2=0x7f45e407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:15.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:15.883+0000 7f45ff0d5640 1 --2- 172.21.15.38:0/3989779823 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f45e4078b90 0x7f45e407b0a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f45e8034000 tx=0x7f45e8004730 comp rx=0 tx=0).stop 2024-09-21T03:34:15.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:15.883+0000 7f45ff0d5640 1 -- 172.21.15.38:0/3989779823 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f45f80ceb90 msgr2=0x7f45f81578f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:15.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:15.883+0000 7f45ff0d5640 1 --2- 172.21.15.38:0/3989779823 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f45f80ceb90 0x7f45f81578f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f45ec00c3c0 tx=0x7f45ec00c880 comp rx=0 tx=0).stop 2024-09-21T03:34:15.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:15.883+0000 7f45ff0d5640 1 -- 172.21.15.38:0/3989779823 shutdown_connections 2024-09-21T03:34:15.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:15.883+0000 7f45ff0d5640 1 --2- 172.21.15.38:0/3989779823 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f45f815ee80 0x7f45f806d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:15.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:15.883+0000 7f45ff0d5640 1 --2- 172.21.15.38:0/3989779823 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f45f80ceb90 0x7f45f81578f0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:15.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:15.883+0000 7f45ff0d5640 1 --2- 172.21.15.38:0/3989779823 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f45e4078b90 0x7f45e407b0a0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:15.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:15.883+0000 7f45ff0d5640 1 --2- 172.21.15.38:0/3989779823 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f45f80b9ad0 0x7f45f81594b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:15.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:15.883+0000 7f45ff0d5640 1 -- 172.21.15.38:0/3989779823 >> 172.21.15.38:0/3989779823 conn(0x7f45f809d830 msgr2=0x7f45f80b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:15.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:15.883+0000 7f45ff0d5640 1 -- 172.21.15.38:0/3989779823 shutdown_connections 2024-09-21T03:34:15.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:15.883+0000 7f45ff0d5640 1 -- 172.21.15.38:0/3989779823 wait complete. 2024-09-21T03:34:15.896 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-21T03:34:16.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:16.014+0000 7f4e25069640 1 -- 172.21.15.38:0/2335366944 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4e200d1120 msgr2=0x7f4e200ceb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:16.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:16.014+0000 7f4e25069640 1 --2- 172.21.15.38:0/2335366944 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4e200d1120 0x7f4e200ceb90 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f4e18005fd0 tx=0x7f4e1802a820 comp rx=0 tx=0).stop 2024-09-21T03:34:16.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:16.014+0000 7f4e25069640 1 -- 172.21.15.38:0/2335366944 shutdown_connections 2024-09-21T03:34:16.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:16.014+0000 7f4e25069640 1 --2- 172.21.15.38:0/2335366944 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4e200d1120 0x7f4e200ceb90 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:16.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:16.014+0000 7f4e25069640 1 --2- 172.21.15.38:0/2335366944 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4e200d0170 0x7f4e200d0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:16.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:16.014+0000 7f4e25069640 1 -- 172.21.15.38:0/2335366944 >> 172.21.15.38:0/2335366944 conn(0x7f4e2009d830 msgr2=0x7f4e200b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:16.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:16.014+0000 7f4e25069640 1 -- 172.21.15.38:0/2335366944 shutdown_connections 2024-09-21T03:34:16.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:16.014+0000 7f4e25069640 1 -- 172.21.15.38:0/2335366944 wait complete. 2024-09-21T03:34:16.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:16.016+0000 7f4e25069640 1 Processor -- start 2024-09-21T03:34:16.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:16.016+0000 7f4e25069640 1 -- start start 2024-09-21T03:34:16.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:16.017+0000 7f4e25069640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4e200d0170 0x7f4e2014fa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:16.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:16.017+0000 7f4e25069640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4e200d1120 0x7f4e2006efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:16.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:16.017+0000 7f4e25069640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4e2015ee80 0x7f4e2006f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:16.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:16.017+0000 7f4e25069640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f4e200b6700 con 0x7f4e200d0170 2024-09-21T03:34:16.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:16.017+0000 7f4e25069640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f4e200b6580 con 0x7f4e2015ee80 2024-09-21T03:34:16.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:16.017+0000 7f4e25069640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f4e200b6880 con 0x7f4e200d1120 2024-09-21T03:34:16.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:16.018+0000 7f4e1ed9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4e200d0170 0x7f4e2014fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:16.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:16.018+0000 7f4e1ed9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4e200d0170 0x7f4e2014fa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54310/0 (socket says 172.21.15.38:54310) 2024-09-21T03:34:16.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:16.018+0000 7f4e1ed9d640 1 -- 172.21.15.38:0/1355315995 learned_addr learned my addr 172.21.15.38:0/1355315995 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:34:16.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:16.018+0000 7f4e1f59e640 1 --2- 172.21.15.38:0/1355315995 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4e2015ee80 0x7f4e2006f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:16.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:16.018+0000 7f4e1e59c640 1 --2- 172.21.15.38:0/1355315995 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4e200d1120 0x7f4e2006efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:16.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:16.018+0000 7f4e1ed9d640 1 -- 172.21.15.38:0/1355315995 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4e200d1120 msgr2=0x7f4e2006efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:16.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:16.018+0000 7f4e1ed9d640 1 --2- 172.21.15.38:0/1355315995 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4e200d1120 0x7f4e2006efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:16.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:16.018+0000 7f4e1ed9d640 1 -- 172.21.15.38:0/1355315995 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4e2015ee80 msgr2=0x7f4e2006f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:16.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:16.018+0000 7f4e1ed9d640 1 --2- 172.21.15.38:0/1355315995 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4e2015ee80 0x7f4e2006f4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:16.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:16.018+0000 7f4e1ed9d640 1 -- 172.21.15.38:0/1355315995 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e200a1720 con 0x7f4e200d0170 2024-09-21T03:34:16.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:16.018+0000 7f4e1e59c640 1 --2- 172.21.15.38:0/1355315995 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4e200d1120 0x7f4e2006efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:34:16.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:16.018+0000 7f4e1ed9d640 1 --2- 172.21.15.38:0/1355315995 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4e200d0170 0x7f4e2014fa20 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f4e1000c760 tx=0x7f4e1000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:16.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:16.018+0000 7f4e077fe640 1 -- 172.21.15.38:0/1355315995 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e10016020 con 0x7f4e200d0170 2024-09-21T03:34:16.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:16.018+0000 7f4e077fe640 1 -- 172.21.15.38:0/1355315995 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4e1000a950 con 0x7f4e200d0170 2024-09-21T03:34:16.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:16.018+0000 7f4e25069640 1 -- 172.21.15.38:0/1355315995 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e200a1a00 con 0x7f4e200d0170 2024-09-21T03:34:16.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:16.018+0000 7f4e077fe640 1 -- 172.21.15.38:0/1355315995 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e1000ada0 con 0x7f4e200d0170 2024-09-21T03:34:16.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:16.018+0000 7f4e25069640 1 -- 172.21.15.38:0/1355315995 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e201f6fc0 con 0x7f4e200d0170 2024-09-21T03:34:16.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:16.019+0000 7f4e25069640 1 -- 172.21.15.38:0/1355315995 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e200cfcb0 con 0x7f4e200d0170 2024-09-21T03:34:16.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:16.020+0000 7f4e077fe640 1 -- 172.21.15.38:0/1355315995 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f4e10019020 con 0x7f4e200d0170 2024-09-21T03:34:16.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:16.021+0000 7f4e077fe640 1 --2- 172.21.15.38:0/1355315995 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4e0c078b90 0x7f4e0c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:16.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:16.021+0000 7f4e077fe640 1 -- 172.21.15.38:0/1355315995 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6619+0+0 (secure 0 0 0) 0x7f4e1009aff0 con 0x7f4e200d0170 2024-09-21T03:34:16.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:16.021+0000 7f4e1e59c640 1 --2- 172.21.15.38:0/1355315995 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4e0c078b90 0x7f4e0c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:16.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:16.022+0000 7f4e1e59c640 1 --2- 172.21.15.38:0/1355315995 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4e0c078b90 0x7f4e0c07b0a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f4e200cafb0 tx=0x7f4e14005e10 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:16.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:16.026+0000 7f4e077fe640 1 -- 172.21.15.38:0/1355315995 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f4e200cfcb0 con 0x7f4e200d0170 2024-09-21T03:34:16.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:16.210+0000 7f4e25069640 1 -- 172.21.15.38:0/1355315995 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f4e201f7220 con 0x7f4e200d0170 2024-09-21T03:34:16.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:16.211+0000 7f4e077fe640 1 -- 172.21.15.38:0/1355315995 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) ==== 100+0+11 (secure 0 0 0) 0x7f4e10020020 con 0x7f4e200d0170 2024-09-21T03:34:16.213 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 64 2024-09-21T03:34:16.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:16.213+0000 7f4e25069640 1 -- 172.21.15.38:0/1355315995 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4e0c078b90 msgr2=0x7f4e0c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:16.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:16.213+0000 7f4e25069640 1 --2- 172.21.15.38:0/1355315995 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4e0c078b90 0x7f4e0c07b0a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f4e200cafb0 tx=0x7f4e14005e10 comp rx=0 tx=0).stop 2024-09-21T03:34:16.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:16.213+0000 7f4e25069640 1 -- 172.21.15.38:0/1355315995 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4e200d0170 msgr2=0x7f4e2014fa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:16.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:16.213+0000 7f4e25069640 1 --2- 172.21.15.38:0/1355315995 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4e200d0170 0x7f4e2014fa20 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f4e1000c760 tx=0x7f4e1000cc20 comp rx=0 tx=0).stop 2024-09-21T03:34:16.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:16.213+0000 7f4e25069640 1 -- 172.21.15.38:0/1355315995 shutdown_connections 2024-09-21T03:34:16.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:16.213+0000 7f4e25069640 1 --2- 172.21.15.38:0/1355315995 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4e2015ee80 0x7f4e2006f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:16.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:16.213+0000 7f4e25069640 1 --2- 172.21.15.38:0/1355315995 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4e0c078b90 0x7f4e0c07b0a0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:16.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:16.213+0000 7f4e25069640 1 --2- 172.21.15.38:0/1355315995 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4e200d1120 0x7f4e2006efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:16.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:16.213+0000 7f4e25069640 1 --2- 172.21.15.38:0/1355315995 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4e200d0170 0x7f4e2014fa20 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:16.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:16.213+0000 7f4e25069640 1 -- 172.21.15.38:0/1355315995 >> 172.21.15.38:0/1355315995 conn(0x7f4e2009d830 msgr2=0x7f4e201608a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:16.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:16.213+0000 7f4e25069640 1 -- 172.21.15.38:0/1355315995 shutdown_connections 2024-09-21T03:34:16.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:16.213+0000 7f4e25069640 1 -- 172.21.15.38:0/1355315995 wait complete. 2024-09-21T03:34:16.226 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-09-21T03:34:16.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:16.349+0000 7f6dfc069640 1 -- 172.21.15.38:0/356786465 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6de0004800 msgr2=0x7f6de0004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:16.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:16.349+0000 7f6dfc069640 1 --2- 172.21.15.38:0/356786465 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6de0004800 0x7f6de0004c10 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f6de4009b50 tx=0x7f6de0005830 comp rx=0 tx=0).stop 2024-09-21T03:34:16.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:16.349+0000 7f6dfa606640 1 -- 172.21.15.38:0/356786465 reap_dead start 2024-09-21T03:34:16.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:16.349+0000 7f6dfc069640 1 -- 172.21.15.38:0/356786465 shutdown_connections 2024-09-21T03:34:16.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:16.349+0000 7f6dfc069640 1 -- 172.21.15.38:0/356786465 >> 172.21.15.38:0/356786465 conn(0x7f6df40b1b90 msgr2=0x7f6df40b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:16.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:16.349+0000 7f6dfc069640 1 -- 172.21.15.38:0/356786465 shutdown_connections 2024-09-21T03:34:16.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:16.349+0000 7f6dfc069640 1 -- 172.21.15.38:0/356786465 wait complete. 2024-09-21T03:34:16.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:16.351+0000 7f6dfc069640 1 Processor -- start 2024-09-21T03:34:16.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:16.351+0000 7f6dfc069640 1 -- start start 2024-09-21T03:34:16.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:16.352+0000 7f6dfc069640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6de0004800 0x7f6df415e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:16.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:16.352+0000 7f6dfc069640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6df415af50 0x7f6df40b4270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:16.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:16.352+0000 7f6dfc069640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6df415b360 0x7f6df40b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:16.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:16.352+0000 7f6dfc069640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f6df4070d20 con 0x7f6de0004800 2024-09-21T03:34:16.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:16.352+0000 7f6dfc069640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f6df4070ba0 con 0x7f6df415b360 2024-09-21T03:34:16.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:16.352+0000 7f6dfc069640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f6df40cf290 con 0x7f6df415af50 2024-09-21T03:34:16.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:16.352+0000 7f6df9e05640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6de0004800 0x7f6df415e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:16.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:16.352+0000 7f6df9e05640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6de0004800 0x7f6df415e740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54336/0 (socket says 172.21.15.38:54336) 2024-09-21T03:34:16.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:16.352+0000 7f6df9e05640 1 -- 172.21.15.38:0/3240979039 learned_addr learned my addr 172.21.15.38:0/3240979039 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:34:16.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:16.352+0000 7f6df9604640 1 --2- 172.21.15.38:0/3240979039 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6df415af50 0x7f6df40b4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:16.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:16.352+0000 7f6dfa606640 1 --2- 172.21.15.38:0/3240979039 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6df415b360 0x7f6df40b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:16.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:16.352+0000 7f6df9e05640 1 -- 172.21.15.38:0/3240979039 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6df415af50 msgr2=0x7f6df40b4270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:16.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:16.352+0000 7f6df9e05640 1 --2- 172.21.15.38:0/3240979039 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6df415af50 0x7f6df40b4270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:16.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:16.352+0000 7f6df9e05640 1 -- 172.21.15.38:0/3240979039 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6df415b360 msgr2=0x7f6df40b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:16.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:16.352+0000 7f6df9e05640 1 --2- 172.21.15.38:0/3240979039 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6df415b360 0x7f6df40b47b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:16.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:16.352+0000 7f6df9e05640 1 -- 172.21.15.38:0/3240979039 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6df41f4200 con 0x7f6de0004800 2024-09-21T03:34:16.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:16.352+0000 7f6df9604640 1 --2- 172.21.15.38:0/3240979039 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6df415af50 0x7f6df40b4270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:34:16.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:16.352+0000 7f6dfa606640 1 --2- 172.21.15.38:0/3240979039 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6df415b360 0x7f6df40b47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:34:16.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:16.353+0000 7f6df9e05640 1 --2- 172.21.15.38:0/3240979039 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6de0004800 0x7f6df415e740 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f6df0011ae0 tx=0x7f6df0009eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:16.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:16.353+0000 7f6dea7fc640 1 -- 172.21.15.38:0/3240979039 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6df001e070 con 0x7f6de0004800 2024-09-21T03:34:16.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:16.353+0000 7f6dea7fc640 1 -- 172.21.15.38:0/3240979039 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6df000fc70 con 0x7f6de0004800 2024-09-21T03:34:16.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:16.353+0000 7f6dfc069640 1 -- 172.21.15.38:0/3240979039 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6df415ec80 con 0x7f6de0004800 2024-09-21T03:34:16.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:16.353+0000 7f6dea7fc640 1 -- 172.21.15.38:0/3240979039 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6df0019410 con 0x7f6de0004800 2024-09-21T03:34:16.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:16.353+0000 7f6dfc069640 1 -- 172.21.15.38:0/3240979039 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6df40c3f50 con 0x7f6de0004800 2024-09-21T03:34:16.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:16.354+0000 7f6dfc069640 1 -- 172.21.15.38:0/3240979039 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6df406ddf0 con 0x7f6de0004800 2024-09-21T03:34:16.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:16.355+0000 7f6dea7fc640 1 -- 172.21.15.38:0/3240979039 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f6df0021070 con 0x7f6de0004800 2024-09-21T03:34:16.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:16.355+0000 7f6dea7fc640 1 --2- 172.21.15.38:0/3240979039 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6dd8078b90 0x7f6dd807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:16.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:16.356+0000 7f6dea7fc640 1 -- 172.21.15.38:0/3240979039 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6619+0+0 (secure 0 0 0) 0x7f6df009b880 con 0x7f6de0004800 2024-09-21T03:34:16.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:16.356+0000 7f6df9604640 1 --2- 172.21.15.38:0/3240979039 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6dd8078b90 0x7f6dd807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:16.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:16.356+0000 7f6df9604640 1 --2- 172.21.15.38:0/3240979039 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6dd8078b90 0x7f6dd807b0a0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f6de4035000 tx=0x7f6de402d040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:16.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:16.360+0000 7f6dea7fc640 1 -- 172.21.15.38:0/3240979039 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f6df0063400 con 0x7f6de0004800 2024-09-21T03:34:16.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:16.555+0000 7f6dfc069640 1 -- 172.21.15.38:0/3240979039 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f6df4070150 con 0x7f6de0004800 2024-09-21T03:34:16.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:16.556+0000 7f6dea7fc640 1 -- 172.21.15.38:0/3240979039 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) ==== 103+0+11 (secure 0 0 0) 0x7f6df0067e30 con 0x7f6de0004800 2024-09-21T03:34:16.558 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 32 2024-09-21T03:34:16.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:16.558+0000 7f6dfc069640 1 -- 172.21.15.38:0/3240979039 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6dd8078b90 msgr2=0x7f6dd807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:16.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:16.558+0000 7f6dfc069640 1 --2- 172.21.15.38:0/3240979039 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6dd8078b90 0x7f6dd807b0a0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f6de4035000 tx=0x7f6de402d040 comp rx=0 tx=0).stop 2024-09-21T03:34:16.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:16.558+0000 7f6dfc069640 1 -- 172.21.15.38:0/3240979039 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6de0004800 msgr2=0x7f6df415e740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:16.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:16.558+0000 7f6dfc069640 1 --2- 172.21.15.38:0/3240979039 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6de0004800 0x7f6df415e740 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f6df0011ae0 tx=0x7f6df0009eb0 comp rx=0 tx=0).stop 2024-09-21T03:34:16.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:16.558+0000 7f6dfc069640 1 -- 172.21.15.38:0/3240979039 shutdown_connections 2024-09-21T03:34:16.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:16.558+0000 7f6dfc069640 1 --2- 172.21.15.38:0/3240979039 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6df415b360 0x7f6df40b47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:16.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:16.558+0000 7f6dfc069640 1 --2- 172.21.15.38:0/3240979039 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6dd8078b90 0x7f6dd807b0a0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:16.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:16.558+0000 7f6dfc069640 1 --2- 172.21.15.38:0/3240979039 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6df415af50 0x7f6df40b4270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:16.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:16.558+0000 7f6dfc069640 1 --2- 172.21.15.38:0/3240979039 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6de0004800 0x7f6df415e740 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:16.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:16.558+0000 7f6dfc069640 1 -- 172.21.15.38:0/3240979039 >> 172.21.15.38:0/3240979039 conn(0x7f6df40b1b90 msgr2=0x7f6df40cb230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:16.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:16.558+0000 7f6dfc069640 1 -- 172.21.15.38:0/3240979039 shutdown_connections 2024-09-21T03:34:16.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:16.558+0000 7f6dfc069640 1 -- 172.21.15.38:0/3240979039 wait complete. 2024-09-21T03:34:16.571 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_damage.TestDamage.test_damaged_dentry' 2024-09-21T03:34:16.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:16.686+0000 7f65479c6640 1 -- 172.21.15.38:0/2834450975 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f654015e930 msgr2=0x7f65400b5fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:16.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:16.686+0000 7f65479c6640 1 --2- 172.21.15.38:0/2834450975 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f654015e930 0x7f65400b5fb0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f653c009830 tx=0x7f653c02a510 comp rx=0 tx=0).stop 2024-09-21T03:34:16.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:16.686+0000 7f65479c6640 1 -- 172.21.15.38:0/2834450975 shutdown_connections 2024-09-21T03:34:16.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:16.686+0000 7f65479c6640 1 --2- 172.21.15.38:0/2834450975 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f65400b64f0 0x7f6540162b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:16.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:16.686+0000 7f65479c6640 1 --2- 172.21.15.38:0/2834450975 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f654015e930 0x7f65400b5fb0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:16.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:16.686+0000 7f65479c6640 1 -- 172.21.15.38:0/2834450975 >> 172.21.15.38:0/2834450975 conn(0x7f654008bf70 msgr2=0x7f654009d840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:16.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:16.686+0000 7f65479c6640 1 -- 172.21.15.38:0/2834450975 shutdown_connections 2024-09-21T03:34:16.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:16.686+0000 7f65479c6640 1 -- 172.21.15.38:0/2834450975 wait complete. 2024-09-21T03:34:16.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:16.688+0000 7f65479c6640 1 Processor -- start 2024-09-21T03:34:16.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:16.688+0000 7f65479c6640 1 -- start start 2024-09-21T03:34:16.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:16.689+0000 7f65479c6640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f65400b64f0 0x7f65401f99a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:16.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:16.689+0000 7f65479c6640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f654015e930 0x7f6540152d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:16.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:16.689+0000 7f65479c6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6540163040 0x7f65401ea6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:16.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:16.689+0000 7f65479c6640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f65400b9060 con 0x7f6540163040 2024-09-21T03:34:16.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:16.689+0000 7f65479c6640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f65400d0340 con 0x7f65400b64f0 2024-09-21T03:34:16.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:16.689+0000 7f65479c6640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f65400b91e0 con 0x7f654015e930 2024-09-21T03:34:16.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:16.690+0000 7f6545f63640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6540163040 0x7f65401ea6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:16.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:16.690+0000 7f6545f63640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6540163040 0x7f65401ea6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54356/0 (socket says 172.21.15.38:54356) 2024-09-21T03:34:16.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:16.690+0000 7f6545f63640 1 -- 172.21.15.38:0/356607912 learned_addr learned my addr 172.21.15.38:0/356607912 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:34:16.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:16.690+0000 7f6544f61640 1 --2- 172.21.15.38:0/356607912 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f654015e930 0x7f6540152d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:16.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:16.690+0000 7f6545762640 1 --2- 172.21.15.38:0/356607912 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f65400b64f0 0x7f65401f99a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:16.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:16.690+0000 7f6545f63640 1 -- 172.21.15.38:0/356607912 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f654015e930 msgr2=0x7f6540152d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:16.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:16.690+0000 7f6545f63640 1 --2- 172.21.15.38:0/356607912 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f654015e930 0x7f6540152d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:16.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:16.690+0000 7f6545f63640 1 -- 172.21.15.38:0/356607912 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f65400b64f0 msgr2=0x7f65401f99a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:16.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:16.690+0000 7f6545f63640 1 --2- 172.21.15.38:0/356607912 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f65400b64f0 0x7f65401f99a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:16.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:16.690+0000 7f6545f63640 1 -- 172.21.15.38:0/356607912 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65400ca8a0 con 0x7f6540163040 2024-09-21T03:34:16.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:16.690+0000 7f6545f63640 1 --2- 172.21.15.38:0/356607912 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6540163040 0x7f65401ea6a0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f653800c730 tx=0x7f653800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:16.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:16.690+0000 7f6535ffb640 1 -- 172.21.15.38:0/356607912 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6538003b70 con 0x7f6540163040 2024-09-21T03:34:16.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:16.690+0000 7f6535ffb640 1 -- 172.21.15.38:0/356607912 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6538003d30 con 0x7f6540163040 2024-09-21T03:34:16.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:16.690+0000 7f65479c6640 1 -- 172.21.15.38:0/356607912 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65400cab80 con 0x7f6540163040 2024-09-21T03:34:16.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:16.690+0000 7f6535ffb640 1 -- 172.21.15.38:0/356607912 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65380052d0 con 0x7f6540163040 2024-09-21T03:34:16.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:16.691+0000 7f65479c6640 1 -- 172.21.15.38:0/356607912 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65400b3080 con 0x7f6540163040 2024-09-21T03:34:16.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:16.691+0000 7f65479c6640 1 -- 172.21.15.38:0/356607912 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6540162b00 con 0x7f6540163040 2024-09-21T03:34:16.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:16.692+0000 7f6535ffb640 1 -- 172.21.15.38:0/356607912 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f6538005680 con 0x7f6540163040 2024-09-21T03:34:16.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:16.693+0000 7f6535ffb640 1 --2- 172.21.15.38:0/356607912 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6524078bc0 0x7f652407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:16.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:16.693+0000 7f6535ffb640 1 -- 172.21.15.38:0/356607912 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6619+0+0 (secure 0 0 0) 0x7f653809af40 con 0x7f6540163040 2024-09-21T03:34:16.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:16.693+0000 7f6545762640 1 --2- 172.21.15.38:0/356607912 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6524078bc0 0x7f652407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:16.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:16.693+0000 7f6545762640 1 --2- 172.21.15.38:0/356607912 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6524078bc0 0x7f652407b0d0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f653c034000 tx=0x7f653c004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:16.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:16.697+0000 7f6535ffb640 1 -- 172.21.15.38:0/356607912 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f65380629a0 con 0x7f6540163040 2024-09-21T03:34:16.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:16.956+0000 7f65479c6640 1 -- 172.21.15.38:0/356607912 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_damage.TestDamage.test_damaged_dentry"]} v 0) -- 0x7f65401e9170 con 0x7f6540163040 2024-09-21T03:34:17.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:17.415+0000 7f6535ffb640 1 -- 172.21.15.38:0/356607912 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_damage.TestDamage.test_damaged_dentry"]}]=0 v57) ==== 137+0+0 (secure 0 0 0) 0x7f65380673d0 con 0x7f6540163040 2024-09-21T03:34:17.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:17.418+0000 7f65479c6640 1 -- 172.21.15.38:0/356607912 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6524078bc0 msgr2=0x7f652407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:17.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:17.418+0000 7f65479c6640 1 --2- 172.21.15.38:0/356607912 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6524078bc0 0x7f652407b0d0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f653c034000 tx=0x7f653c004730 comp rx=0 tx=0).stop 2024-09-21T03:34:17.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:17.418+0000 7f65479c6640 1 -- 172.21.15.38:0/356607912 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6540163040 msgr2=0x7f65401ea6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:17.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:17.418+0000 7f65479c6640 1 --2- 172.21.15.38:0/356607912 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6540163040 0x7f65401ea6a0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f653800c730 tx=0x7f653800cbf0 comp rx=0 tx=0).stop 2024-09-21T03:34:17.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:17.418+0000 7f65479c6640 1 -- 172.21.15.38:0/356607912 shutdown_connections 2024-09-21T03:34:17.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:17.418+0000 7f65479c6640 1 --2- 172.21.15.38:0/356607912 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6540163040 0x7f65401ea6a0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:17.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:17.418+0000 7f65479c6640 1 --2- 172.21.15.38:0/356607912 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6524078bc0 0x7f652407b0d0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:17.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:17.418+0000 7f65479c6640 1 --2- 172.21.15.38:0/356607912 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f654015e930 0x7f6540152d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:17.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:17.418+0000 7f65479c6640 1 --2- 172.21.15.38:0/356607912 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f65400b64f0 0x7f65401f99a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:17.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:17.418+0000 7f65479c6640 1 -- 172.21.15.38:0/356607912 >> 172.21.15.38:0/356607912 conn(0x7f654008bf70 msgr2=0x7f65400cf920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:17.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:17.418+0000 7f65479c6640 1 -- 172.21.15.38:0/356607912 shutdown_connections 2024-09-21T03:34:17.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:17.418+0000 7f65479c6640 1 -- 172.21.15.38:0/356607912 wait complete. 2024-09-21T03:34:17.433 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-21T03:34:17.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:17.555+0000 7f94a4aa2640 1 -- 172.21.15.38:0/3707015349 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f948c005150 msgr2=0x7f948c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:17.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:17.555+0000 7f94a4aa2640 1 --2- 172.21.15.38:0/3707015349 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f948c005150 0x7f948c00b9c0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f9498001690 tx=0x7f948c005830 comp rx=0 tx=0).stop 2024-09-21T03:34:17.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:17.555+0000 7f949ed9d640 1 -- 172.21.15.38:0/3707015349 reap_dead start 2024-09-21T03:34:17.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:17.555+0000 7f94a4aa2640 1 -- 172.21.15.38:0/3707015349 shutdown_connections 2024-09-21T03:34:17.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:17.555+0000 7f94a4aa2640 1 -- 172.21.15.38:0/3707015349 >> 172.21.15.38:0/3707015349 conn(0x7f94a00b6b20 msgr2=0x7f94a00d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:17.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:17.555+0000 7f94a4aa2640 1 -- 172.21.15.38:0/3707015349 shutdown_connections 2024-09-21T03:34:17.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:17.555+0000 7f94a4aa2640 1 -- 172.21.15.38:0/3707015349 wait complete. 2024-09-21T03:34:17.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:17.557+0000 7f94a4aa2640 1 Processor -- start 2024-09-21T03:34:17.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:17.557+0000 7f94a4aa2640 1 -- start start 2024-09-21T03:34:17.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:17.558+0000 7f94a4aa2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f948c005150 0x7f94a01527c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:17.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:17.558+0000 7f94a4aa2640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f94a014f4a0 0x7f94a00ca1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:17.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:17.558+0000 7f94a4aa2640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f94a00b24f0 0x7f94a00b2900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:17.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:17.558+0000 7f94a4aa2640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f94a015f640 con 0x7f948c005150 2024-09-21T03:34:17.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:17.558+0000 7f94a4aa2640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f94a015f4c0 con 0x7f94a00b24f0 2024-09-21T03:34:17.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:17.558+0000 7f94a4aa2640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f94a015f7c0 con 0x7f94a014f4a0 2024-09-21T03:34:17.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:17.558+0000 7f949e59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f948c005150 0x7f94a01527c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:17.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:17.558+0000 7f949e59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f948c005150 0x7f94a01527c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54368/0 (socket says 172.21.15.38:54368) 2024-09-21T03:34:17.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:17.558+0000 7f949e59c640 1 -- 172.21.15.38:0/3927744836 learned_addr learned my addr 172.21.15.38:0/3927744836 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:34:17.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:17.559+0000 7f949dd9b640 1 --2- 172.21.15.38:0/3927744836 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f94a014f4a0 0x7f94a00ca1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:17.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:17.559+0000 7f949e59c640 1 -- 172.21.15.38:0/3927744836 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f94a014f4a0 msgr2=0x7f94a00ca1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:17.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:17.559+0000 7f949ed9d640 1 --2- 172.21.15.38:0/3927744836 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f94a00b24f0 0x7f94a00b2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:17.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:17.559+0000 7f949e59c640 1 --2- 172.21.15.38:0/3927744836 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f94a014f4a0 0x7f94a00ca1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:17.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:17.559+0000 7f949e59c640 1 -- 172.21.15.38:0/3927744836 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f94a00b24f0 msgr2=0x7f94a00b2900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:17.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:17.559+0000 7f949e59c640 1 --2- 172.21.15.38:0/3927744836 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f94a00b24f0 0x7f94a00b2900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:17.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:17.559+0000 7f949e59c640 1 -- 172.21.15.38:0/3927744836 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94a00c9150 con 0x7f948c005150 2024-09-21T03:34:17.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:17.559+0000 7f949dd9b640 1 --2- 172.21.15.38:0/3927744836 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f94a014f4a0 0x7f94a00ca1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:34:17.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:17.559+0000 7f949e59c640 1 --2- 172.21.15.38:0/3927744836 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f948c005150 0x7f94a01527c0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f9490011d60 tx=0x7f949000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:17.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:17.559+0000 7f9496ffd640 1 -- 172.21.15.38:0/3927744836 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9490003d80 con 0x7f948c005150 2024-09-21T03:34:17.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:17.559+0000 7f9496ffd640 1 -- 172.21.15.38:0/3927744836 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9490011e50 con 0x7f948c005150 2024-09-21T03:34:17.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:17.559+0000 7f94a4aa2640 1 -- 172.21.15.38:0/3927744836 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94a00c9430 con 0x7f948c005150 2024-09-21T03:34:17.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:17.559+0000 7f9496ffd640 1 -- 172.21.15.38:0/3927744836 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9490004d40 con 0x7f948c005150 2024-09-21T03:34:17.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:17.559+0000 7f94a4aa2640 1 -- 172.21.15.38:0/3927744836 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94a00c99f0 con 0x7f948c005150 2024-09-21T03:34:17.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:17.560+0000 7f94a4aa2640 1 -- 172.21.15.38:0/3927744836 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94a00b3250 con 0x7f948c005150 2024-09-21T03:34:17.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:17.561+0000 7f9496ffd640 1 -- 172.21.15.38:0/3927744836 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f9490004050 con 0x7f948c005150 2024-09-21T03:34:17.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:17.562+0000 7f9496ffd640 1 --2- 172.21.15.38:0/3927744836 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9480078b90 0x7f948007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:17.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:17.562+0000 7f9496ffd640 1 -- 172.21.15.38:0/3927744836 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6619+0+0 (secure 0 0 0) 0x7f949009c1b0 con 0x7f948c005150 2024-09-21T03:34:17.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:17.562+0000 7f949dd9b640 1 --2- 172.21.15.38:0/3927744836 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9480078b90 0x7f948007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:17.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:17.563+0000 7f949dd9b640 1 --2- 172.21.15.38:0/3927744836 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9480078b90 0x7f948007b0a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f9488000c00 tx=0x7f94880067a0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:17.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:17.567+0000 7f9496ffd640 1 -- 172.21.15.38:0/3927744836 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f9490064250 con 0x7f948c005150 2024-09-21T03:34:17.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:17.755+0000 7f94a4aa2640 1 -- 172.21.15.38:0/3927744836 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f94a006ebe0 con 0x7f948c005150 2024-09-21T03:34:17.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:17.756+0000 7f9496ffd640 1 -- 172.21.15.38:0/3927744836 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v31) ==== 59+0+2294 (secure 0 0 0) 0x7f9490063b50 con 0x7f948c005150 2024-09-21T03:34:17.758 INFO:teuthology.orchestra.run.smithi038.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-21T03:34:17.758 INFO:teuthology.orchestra.run.smithi038.stdout:mon advanced mon_op_complaint_time 120 2024-09-21T03:34:17.758 INFO:teuthology.orchestra.run.smithi038.stdout:mgr advanced client_mount_timeout 30 2024-09-21T03:34:17.758 INFO:teuthology.orchestra.run.smithi038.stdout:mgr advanced debug_client 20/20 2024-09-21T03:34:17.758 INFO:teuthology.orchestra.run.smithi038.stdout:mgr advanced debug_mgr 20/20 2024-09-21T03:34:17.758 INFO:teuthology.orchestra.run.smithi038.stdout:mgr advanced debug_ms 1/1 2024-09-21T03:34:17.758 INFO:teuthology.orchestra.run.smithi038.stdout:mgr dev mon_warn_on_pool_no_app false 2024-09-21T03:34:17.758 INFO:teuthology.orchestra.run.smithi038.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-21T03:34:17.759 INFO:teuthology.orchestra.run.smithi038.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 3844.962174 2024-09-21T03:34:17.759 INFO:teuthology.orchestra.run.smithi038.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 4916.015278 2024-09-21T03:34:17.759 INFO:teuthology.orchestra.run.smithi038.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 4165.407418 2024-09-21T03:34:17.759 INFO:teuthology.orchestra.run.smithi038.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 3760.481604 2024-09-21T03:34:17.759 INFO:teuthology.orchestra.run.smithi038.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 6052.882314 2024-09-21T03:34:17.759 INFO:teuthology.orchestra.run.smithi038.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 8967.576248 2024-09-21T03:34:17.759 INFO:teuthology.orchestra.run.smithi038.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 6129.316416 2024-09-21T03:34:17.759 INFO:teuthology.orchestra.run.smithi038.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 8751.950197 2024-09-21T03:34:17.759 INFO:teuthology.orchestra.run.smithi038.stdout:mds advanced debug_mds 20/20 2024-09-21T03:34:17.759 INFO:teuthology.orchestra.run.smithi038.stdout:mds advanced debug_mds_balancer 20/20 2024-09-21T03:34:17.759 INFO:teuthology.orchestra.run.smithi038.stdout:mds advanced debug_ms 2/2 2024-09-21T03:34:17.759 INFO:teuthology.orchestra.run.smithi038.stdout:mds dev mds_debug_frag true 2024-09-21T03:34:17.759 INFO:teuthology.orchestra.run.smithi038.stdout:mds dev mds_debug_scatterstat true 2024-09-21T03:34:17.759 INFO:teuthology.orchestra.run.smithi038.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-21T03:34:17.760 INFO:teuthology.orchestra.run.smithi038.stdout:mds dev mds_verify_scatter true 2024-09-21T03:34:17.760 INFO:teuthology.orchestra.run.smithi038.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-21T03:34:17.760 INFO:teuthology.orchestra.run.smithi038.stdout:mds advanced rados_mon_op_timeout 900 2024-09-21T03:34:17.760 INFO:teuthology.orchestra.run.smithi038.stdout:mds advanced rados_osd_op_timeout 900 2024-09-21T03:34:17.760 INFO:teuthology.orchestra.run.smithi038.stdout:client advanced client_mount_timeout 600 2024-09-21T03:34:17.760 INFO:teuthology.orchestra.run.smithi038.stdout:client advanced debug_client 20/20 2024-09-21T03:34:17.760 INFO:teuthology.orchestra.run.smithi038.stdout:client advanced debug_ms 1/1 2024-09-21T03:34:17.760 INFO:teuthology.orchestra.run.smithi038.stdout:client advanced rados_mon_op_timeout 900 2024-09-21T03:34:17.760 INFO:teuthology.orchestra.run.smithi038.stdout:client advanced rados_osd_op_timeout 900 2024-09-21T03:34:17.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:17.758+0000 7f94a4aa2640 1 -- 172.21.15.38:0/3927744836 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9480078b90 msgr2=0x7f948007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:17.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:17.758+0000 7f94a4aa2640 1 --2- 172.21.15.38:0/3927744836 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9480078b90 0x7f948007b0a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f9488000c00 tx=0x7f94880067a0 comp rx=0 tx=0).stop 2024-09-21T03:34:17.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:17.758+0000 7f94a4aa2640 1 -- 172.21.15.38:0/3927744836 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f948c005150 msgr2=0x7f94a01527c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:17.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:17.758+0000 7f94a4aa2640 1 --2- 172.21.15.38:0/3927744836 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f948c005150 0x7f94a01527c0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f9490011d60 tx=0x7f949000f790 comp rx=0 tx=0).stop 2024-09-21T03:34:17.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:17.758+0000 7f94a4aa2640 1 -- 172.21.15.38:0/3927744836 shutdown_connections 2024-09-21T03:34:17.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:17.758+0000 7f94a4aa2640 1 --2- 172.21.15.38:0/3927744836 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f94a00b24f0 0x7f94a00b2900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:17.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:17.758+0000 7f94a4aa2640 1 --2- 172.21.15.38:0/3927744836 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9480078b90 0x7f948007b0a0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:17.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:17.758+0000 7f94a4aa2640 1 --2- 172.21.15.38:0/3927744836 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f94a014f4a0 0x7f94a00ca1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:17.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:17.758+0000 7f94a4aa2640 1 --2- 172.21.15.38:0/3927744836 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f948c005150 0x7f94a01527c0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:17.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:17.758+0000 7f94a4aa2640 1 -- 172.21.15.38:0/3927744836 >> 172.21.15.38:0/3927744836 conn(0x7f94a00b6b20 msgr2=0x7f94a00cf5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:17.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:17.758+0000 7f94a4aa2640 1 -- 172.21.15.38:0/3927744836 shutdown_connections 2024-09-21T03:34:17.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:17.758+0000 7f94a4aa2640 1 -- 172.21.15.38:0/3927744836 wait complete. 2024-09-21T03:34:17.772 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-21T03:34:17.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:17.890+0000 7fd6fab8b640 1 -- 172.21.15.38:0/3947888024 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd6f40d0170 msgr2=0x7fd6f40d0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:17.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:17.890+0000 7fd6fab8b640 1 --2- 172.21.15.38:0/3947888024 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd6f40d0170 0x7fd6f40d0be0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd6e80098e0 tx=0x7fd6e802a530 comp rx=0 tx=0).stop 2024-09-21T03:34:17.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:17.890+0000 7fd6fab8b640 1 -- 172.21.15.38:0/3947888024 shutdown_connections 2024-09-21T03:34:17.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:17.890+0000 7fd6fab8b640 1 --2- 172.21.15.38:0/3947888024 >> v2:172.21.15.195:3300/0 conn(0x7fd6f40d1120 0x7fd6f40ceb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:17.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:17.890+0000 7fd6fab8b640 1 --2- 172.21.15.38:0/3947888024 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd6f40d0170 0x7fd6f40d0be0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:17.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:17.890+0000 7fd6fab8b640 1 -- 172.21.15.38:0/3947888024 >> 172.21.15.38:0/3947888024 conn(0x7fd6f409d830 msgr2=0x7fd6f40b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:17.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:17.890+0000 7fd6fab8b640 1 -- 172.21.15.38:0/3947888024 shutdown_connections 2024-09-21T03:34:17.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:17.890+0000 7fd6fab8b640 1 -- 172.21.15.38:0/3947888024 wait complete. 2024-09-21T03:34:17.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:17.892+0000 7fd6fab8b640 1 Processor -- start 2024-09-21T03:34:17.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:17.892+0000 7fd6fab8b640 1 -- start start 2024-09-21T03:34:17.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:17.893+0000 7fd6fab8b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd6f40d0170 0x7fd6f406cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:17.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:17.893+0000 7fd6fab8b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd6f40d1120 0x7fd6f414c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:17.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:17.893+0000 7fd6fab8b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd6f415ee80 0x7fd6f40d3c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:17.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:17.894+0000 7fd6fab8b640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fd6f40b6700 con 0x7fd6f40d1120 2024-09-21T03:34:17.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:17.894+0000 7fd6fab8b640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd6f40b6580 con 0x7fd6f40d0170 2024-09-21T03:34:17.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:17.894+0000 7fd6fab8b640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fd6f40b6880 con 0x7fd6f415ee80 2024-09-21T03:34:17.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:17.894+0000 7fd6f3fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd6f40d1120 0x7fd6f414c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:17.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:17.894+0000 7fd6f3fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd6f40d1120 0x7fd6f414c9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54376/0 (socket says 172.21.15.38:54376) 2024-09-21T03:34:17.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:17.894+0000 7fd6f3fff640 1 -- 172.21.15.38:0/209631147 learned_addr learned my addr 172.21.15.38:0/209631147 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:34:17.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:17.894+0000 7fd6f9128640 1 --2- 172.21.15.38:0/209631147 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd6f415ee80 0x7fd6f40d3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:17.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:17.894+0000 7fd6f8927640 1 --2- 172.21.15.38:0/209631147 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd6f40d0170 0x7fd6f406cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:17.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:17.894+0000 7fd6f3fff640 1 -- 172.21.15.38:0/209631147 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd6f415ee80 msgr2=0x7fd6f40d3c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:17.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:17.894+0000 7fd6f3fff640 1 --2- 172.21.15.38:0/209631147 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd6f415ee80 0x7fd6f40d3c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:17.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:17.894+0000 7fd6f3fff640 1 -- 172.21.15.38:0/209631147 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd6f40d0170 msgr2=0x7fd6f406cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:17.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:17.894+0000 7fd6f3fff640 1 --2- 172.21.15.38:0/209631147 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd6f40d0170 0x7fd6f406cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:17.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:17.894+0000 7fd6f3fff640 1 -- 172.21.15.38:0/209631147 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6f414cf10 con 0x7fd6f40d1120 2024-09-21T03:34:17.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:17.894+0000 7fd6f3fff640 1 --2- 172.21.15.38:0/209631147 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd6f40d1120 0x7fd6f414c9d0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fd6e8009a00 tx=0x7fd6e80052f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:17.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:17.894+0000 7fd6f17fa640 1 -- 172.21.15.38:0/209631147 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6e803f070 con 0x7fd6f40d1120 2024-09-21T03:34:17.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:17.894+0000 7fd6fab8b640 1 -- 172.21.15.38:0/209631147 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6f406d7c0 con 0x7fd6f40d1120 2024-09-21T03:34:17.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:17.894+0000 7fd6f17fa640 1 -- 172.21.15.38:0/209631147 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd6e8004250 con 0x7fd6f40d1120 2024-09-21T03:34:17.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:17.894+0000 7fd6f17fa640 1 -- 172.21.15.38:0/209631147 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6e8005cf0 con 0x7fd6f40d1120 2024-09-21T03:34:17.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:17.895+0000 7fd6fab8b640 1 -- 172.21.15.38:0/209631147 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6f406e9f0 con 0x7fd6f40d1120 2024-09-21T03:34:17.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:17.895+0000 7fd6fab8b640 1 -- 172.21.15.38:0/209631147 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6f406d530 con 0x7fd6f40d1120 2024-09-21T03:34:17.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:17.896+0000 7fd6f17fa640 1 -- 172.21.15.38:0/209631147 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fd6e8042070 con 0x7fd6f40d1120 2024-09-21T03:34:17.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:17.897+0000 7fd6f17fa640 1 --2- 172.21.15.38:0/209631147 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd6e0078bc0 0x7fd6e007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:17.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:17.897+0000 7fd6f17fa640 1 -- 172.21.15.38:0/209631147 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6619+0+0 (secure 0 0 0) 0x7fd6e80bbe70 con 0x7fd6f40d1120 2024-09-21T03:34:17.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:17.897+0000 7fd6f8927640 1 --2- 172.21.15.38:0/209631147 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd6e0078bc0 0x7fd6e007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:17.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:17.897+0000 7fd6f8927640 1 --2- 172.21.15.38:0/209631147 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd6e0078bc0 0x7fd6e007b0d0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fd6e4001720 tx=0x7fd6e4009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:17.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:17.901+0000 7fd6f17fa640 1 -- 172.21.15.38:0/209631147 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fd6e80838d0 con 0x7fd6f40d1120 2024-09-21T03:34:18.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:18.092+0000 7fd6fab8b640 1 -- 172.21.15.38:0/209631147 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fd6f406dce0 con 0x7fd6f40d1120 2024-09-21T03:34:18.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:18.096+0000 7fd6f17fa640 1 -- 172.21.15.38:0/209631147 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v32)=0 v32) ==== 122+0+0 (secure 0 0 0) 0x7fd6e8088300 con 0x7fd6f40d1120 2024-09-21T03:34:18.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:18.098+0000 7fd6fab8b640 1 -- 172.21.15.38:0/209631147 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd6e0078bc0 msgr2=0x7fd6e007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:18.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:18.098+0000 7fd6fab8b640 1 --2- 172.21.15.38:0/209631147 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd6e0078bc0 0x7fd6e007b0d0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fd6e4001720 tx=0x7fd6e4009040 comp rx=0 tx=0).stop 2024-09-21T03:34:18.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:18.098+0000 7fd6fab8b640 1 -- 172.21.15.38:0/209631147 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd6f40d1120 msgr2=0x7fd6f414c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:18.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:18.098+0000 7fd6fab8b640 1 --2- 172.21.15.38:0/209631147 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd6f40d1120 0x7fd6f414c9d0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fd6e8009a00 tx=0x7fd6e80052f0 comp rx=0 tx=0).stop 2024-09-21T03:34:18.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:18.098+0000 7fd6fab8b640 1 -- 172.21.15.38:0/209631147 shutdown_connections 2024-09-21T03:34:18.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:18.098+0000 7fd6fab8b640 1 --2- 172.21.15.38:0/209631147 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd6e0078bc0 0x7fd6e007b0d0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:18.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:18.098+0000 7fd6fab8b640 1 --2- 172.21.15.38:0/209631147 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd6f415ee80 0x7fd6f40d3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:18.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:18.098+0000 7fd6fab8b640 1 --2- 172.21.15.38:0/209631147 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd6f40d1120 0x7fd6f414c9d0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:18.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:18.098+0000 7fd6fab8b640 1 --2- 172.21.15.38:0/209631147 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd6f40d0170 0x7fd6f406cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:18.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:18.098+0000 7fd6fab8b640 1 -- 172.21.15.38:0/209631147 >> 172.21.15.38:0/209631147 conn(0x7fd6f409d830 msgr2=0x7fd6f415dd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:18.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:18.098+0000 7fd6fab8b640 1 -- 172.21.15.38:0/209631147 shutdown_connections 2024-09-21T03:34:18.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:18.099+0000 7fd6fab8b640 1 -- 172.21.15.38:0/209631147 wait complete. 2024-09-21T03:34:18.112 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-21T03:34:18.113 DEBUG:teuthology.orchestra.run.smithi038:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T03:34:18.152 DEBUG:teuthology.orchestra.run.smithi195:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T03:34:18.197 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-21T03:34:18.197 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-21T03:34:18.197 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:34:18.217 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-21T03:34:18.217 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:34:18.276 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-21T03:34:18.276 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:34:18.333 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T03:34:18.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:18.492+0000 7f4282969640 1 -- 172.21.15.38:0/499911984 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f427c0ceb90 msgr2=0x7f427c0cefa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:18.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:18.492+0000 7f4282969640 1 --2- 172.21.15.38:0/499911984 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f427c0ceb90 0x7f427c0cefa0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f4274005fd0 tx=0x7f427402aa10 comp rx=0 tx=0).stop 2024-09-21T03:34:18.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:18.492+0000 7f4282969640 1 -- 172.21.15.38:0/499911984 shutdown_connections 2024-09-21T03:34:18.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:18.492+0000 7f4282969640 1 --2- 172.21.15.38:0/499911984 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f427c0ceb90 0x7f427c0cefa0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:18.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:18.492+0000 7f4282969640 1 -- 172.21.15.38:0/499911984 >> 172.21.15.38:0/499911984 conn(0x7f427c09d830 msgr2=0x7f427c0b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:18.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:18.492+0000 7f4282969640 1 -- 172.21.15.38:0/499911984 shutdown_connections 2024-09-21T03:34:18.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:18.492+0000 7f4282969640 1 -- 172.21.15.38:0/499911984 wait complete. 2024-09-21T03:34:18.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:18.494+0000 7f4282969640 1 Processor -- start 2024-09-21T03:34:18.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:18.494+0000 7f4282969640 1 -- start start 2024-09-21T03:34:18.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:18.495+0000 7f4282969640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f427c0ceb90 0x7f427c06f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:18.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:18.495+0000 7f4282969640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f427c0d0170 0x7f427c06f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:18.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:18.495+0000 7f4282969640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f427c15ee80 0x7f427c14ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:18.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:18.495+0000 7f4282969640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f427c0b6910 con 0x7f427c0ceb90 2024-09-21T03:34:18.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:18.495+0000 7f4282969640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f427c0b6790 con 0x7f427c15ee80 2024-09-21T03:34:18.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:18.495+0000 7f4282969640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f427c0b6a90 con 0x7f427c0d0170 2024-09-21T03:34:18.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:18.495+0000 7f427bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f427c0ceb90 0x7f427c06f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:18.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:18.495+0000 7f427bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f427c0ceb90 0x7f427c06f250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54390/0 (socket says 172.21.15.38:54390) 2024-09-21T03:34:18.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:18.496+0000 7f427bfff640 1 -- 172.21.15.38:0/1015034636 learned_addr learned my addr 172.21.15.38:0/1015034636 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:34:18.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:18.496+0000 7f4280f06640 1 --2- 172.21.15.38:0/1015034636 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f427c15ee80 0x7f427c14ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:18.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:18.496+0000 7f427b7fe640 1 --2- 172.21.15.38:0/1015034636 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f427c0d0170 0x7f427c06f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:18.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:18.496+0000 7f427bfff640 1 -- 172.21.15.38:0/1015034636 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f427c0d0170 msgr2=0x7f427c06f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:18.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:18.496+0000 7f427bfff640 1 --2- 172.21.15.38:0/1015034636 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f427c0d0170 0x7f427c06f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:18.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:18.496+0000 7f427bfff640 1 -- 172.21.15.38:0/1015034636 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f427c15ee80 msgr2=0x7f427c14ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:18.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:18.496+0000 7f427bfff640 1 --2- 172.21.15.38:0/1015034636 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f427c15ee80 0x7f427c14ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:18.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:18.496+0000 7f427bfff640 1 -- 172.21.15.38:0/1015034636 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f427c0cdd90 con 0x7f427c0ceb90 2024-09-21T03:34:18.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:18.496+0000 7f427bfff640 1 --2- 172.21.15.38:0/1015034636 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f427c0ceb90 0x7f427c06f250 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f426c00c400 tx=0x7f426c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:18.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:18.496+0000 7f4278ff9640 1 -- 172.21.15.38:0/1015034636 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f426c017070 con 0x7f427c0ceb90 2024-09-21T03:34:18.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:18.496+0000 7f4278ff9640 1 -- 172.21.15.38:0/1015034636 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f426c0053c0 con 0x7f427c0ceb90 2024-09-21T03:34:18.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:18.496+0000 7f4282969640 1 -- 172.21.15.38:0/1015034636 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f427c0ce070 con 0x7f427c0ceb90 2024-09-21T03:34:18.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:18.496+0000 7f4278ff9640 1 -- 172.21.15.38:0/1015034636 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f426c003850 con 0x7f427c0ceb90 2024-09-21T03:34:18.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:18.496+0000 7f4282969640 1 -- 172.21.15.38:0/1015034636 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f427c0b1330 con 0x7f427c0ceb90 2024-09-21T03:34:18.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:18.497+0000 7f4282969640 1 -- 172.21.15.38:0/1015034636 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f427c0cefa0 con 0x7f427c0ceb90 2024-09-21T03:34:18.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:18.498+0000 7f4278ff9640 1 -- 172.21.15.38:0/1015034636 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f426c01a070 con 0x7f427c0ceb90 2024-09-21T03:34:18.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:18.499+0000 7f4278ff9640 1 --2- 172.21.15.38:0/1015034636 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4268078b90 0x7f426807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:18.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:18.499+0000 7f4278ff9640 1 -- 172.21.15.38:0/1015034636 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6619+0+0 (secure 0 0 0) 0x7f426c09a4c0 con 0x7f427c0ceb90 2024-09-21T03:34:18.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:18.499+0000 7f427b7fe640 1 --2- 172.21.15.38:0/1015034636 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4268078b90 0x7f426807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:18.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:18.500+0000 7f427b7fe640 1 --2- 172.21.15.38:0/1015034636 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4268078b90 0x7f426807b0a0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f427c15ec30 tx=0x7f4270009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:18.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:18.503+0000 7f4278ff9640 1 -- 172.21.15.38:0/1015034636 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f427c0cf4e0 con 0x7f427c0ceb90 2024-09-21T03:34:18.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:18.706+0000 7f4282969640 1 -- 172.21.15.38:0/1015034636 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f427c15c7a0 con 0x7f427c0ceb90 2024-09-21T03:34:18.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:18.707+0000 7f4278ff9640 1 -- 172.21.15.38:0/1015034636 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v28) ==== 80+0+0 (secure 0 0 0) 0x7f426c061e80 con 0x7f427c0ceb90 2024-09-21T03:34:18.709 INFO:teuthology.orchestra.run.smithi038.stderr:listed 0 entries 2024-09-21T03:34:18.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:18.709+0000 7f4282969640 1 -- 172.21.15.38:0/1015034636 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4268078b90 msgr2=0x7f426807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:18.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:18.709+0000 7f4282969640 1 --2- 172.21.15.38:0/1015034636 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4268078b90 0x7f426807b0a0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f427c15ec30 tx=0x7f4270009040 comp rx=0 tx=0).stop 2024-09-21T03:34:18.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:18.709+0000 7f4282969640 1 -- 172.21.15.38:0/1015034636 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f427c0ceb90 msgr2=0x7f427c06f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:18.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:18.709+0000 7f4282969640 1 --2- 172.21.15.38:0/1015034636 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f427c0ceb90 0x7f427c06f250 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f426c00c400 tx=0x7f426c00c8c0 comp rx=0 tx=0).stop 2024-09-21T03:34:18.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:18.709+0000 7f4282969640 1 -- 172.21.15.38:0/1015034636 shutdown_connections 2024-09-21T03:34:18.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:18.709+0000 7f4282969640 1 --2- 172.21.15.38:0/1015034636 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f427c15ee80 0x7f427c14ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:18.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:18.709+0000 7f4282969640 1 --2- 172.21.15.38:0/1015034636 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4268078b90 0x7f426807b0a0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:18.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:18.709+0000 7f4282969640 1 --2- 172.21.15.38:0/1015034636 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f427c0d0170 0x7f427c06f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:18.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:18.709+0000 7f4282969640 1 --2- 172.21.15.38:0/1015034636 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f427c0ceb90 0x7f427c06f250 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:18.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:18.709+0000 7f4282969640 1 -- 172.21.15.38:0/1015034636 >> 172.21.15.38:0/1015034636 conn(0x7f427c09d830 msgr2=0x7f427c1571b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:18.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:18.709+0000 7f4282969640 1 -- 172.21.15.38:0/1015034636 shutdown_connections 2024-09-21T03:34:18.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:18.709+0000 7f4282969640 1 -- 172.21.15.38:0/1015034636 wait complete. 2024-09-21T03:34:18.723 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-21T03:34:18.723 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:34:18.723 DEBUG:teuthology.orchestra.run.smithi038:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-09-21T03:34:18.754 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi038.front.sepia.ceph.com 2024-09-21T03:34:18.754 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:34:18.755 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T03:34:18.811 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-09-21T03:34:18.811 DEBUG:teuthology.orchestra.run.smithi195:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:34:18.831 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-09-21T03:34:18.831 DEBUG:teuthology.orchestra.run.smithi195:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:34:18.889 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-09-21T03:34:18.889 DEBUG:teuthology.orchestra.run.smithi195:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:34:18.947 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T03:34:19.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:19.063+0000 7f4308369640 1 -- 172.21.15.38:0/2811232919 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f43000ceb90 msgr2=0x7f43000cefa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:19.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:19.063+0000 7f4308369640 1 --2- 172.21.15.38:0/2811232919 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f43000ceb90 0x7f43000cefa0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f43000cfba0 tx=0x7f42f802a8f0 comp rx=0 tx=0).stop 2024-09-21T03:34:19.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:19.064+0000 7f4308369640 1 -- 172.21.15.38:0/2811232919 shutdown_connections 2024-09-21T03:34:19.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:19.064+0000 7f4308369640 1 --2- 172.21.15.38:0/2811232919 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f43000ceb90 0x7f43000cefa0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:19.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:19.064+0000 7f4308369640 1 --2- 172.21.15.38:0/2811232919 >> v2:172.21.15.195:3300/0 conn(0x7f430015ee80 0x7f43000d0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:19.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:19.064+0000 7f4308369640 1 -- 172.21.15.38:0/2811232919 >> 172.21.15.38:0/2811232919 conn(0x7f430009d830 msgr2=0x7f43000b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:19.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:19.064+0000 7f4308369640 1 -- 172.21.15.38:0/2811232919 shutdown_connections 2024-09-21T03:34:19.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:19.064+0000 7f4308369640 1 -- 172.21.15.38:0/2811232919 wait complete. 2024-09-21T03:34:19.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:19.065+0000 7f4308369640 1 Processor -- start 2024-09-21T03:34:19.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:19.065+0000 7f4308369640 1 -- start start 2024-09-21T03:34:19.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:19.067+0000 7f4308369640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f43000b9ad0 0x7f430014f990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:19.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:19.067+0000 7f4308369640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f43000ceb90 0x7f43001f8c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:19.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:19.067+0000 7f4308369640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f430015ee80 0x7f43000b2780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:19.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:19.067+0000 7f4308369640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f43000b6700 con 0x7f43000b9ad0 2024-09-21T03:34:19.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:19.067+0000 7f4308369640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f43000b6580 con 0x7f43000ceb90 2024-09-21T03:34:19.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:19.067+0000 7f4308369640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f43000b6880 con 0x7f430015ee80 2024-09-21T03:34:19.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:19.067+0000 7f4306105640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f43000b9ad0 0x7f430014f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:19.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:19.067+0000 7f4306105640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f43000b9ad0 0x7f430014f990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54406/0 (socket says 172.21.15.38:54406) 2024-09-21T03:34:19.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:19.067+0000 7f4306105640 1 -- 172.21.15.38:0/937577612 learned_addr learned my addr 172.21.15.38:0/937577612 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:34:19.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:19.067+0000 7f4305904640 1 --2- 172.21.15.38:0/937577612 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f43000ceb90 0x7f43001f8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:19.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:19.067+0000 7f4306906640 1 --2- 172.21.15.38:0/937577612 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f430015ee80 0x7f43000b2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:19.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:19.067+0000 7f4306105640 1 -- 172.21.15.38:0/937577612 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f430015ee80 msgr2=0x7f43000b2780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:19.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:19.067+0000 7f4306105640 1 --2- 172.21.15.38:0/937577612 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f430015ee80 0x7f43000b2780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:19.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:19.067+0000 7f4306105640 1 -- 172.21.15.38:0/937577612 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f43000ceb90 msgr2=0x7f43001f8c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:19.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:19.067+0000 7f4306105640 1 --2- 172.21.15.38:0/937577612 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f43000ceb90 0x7f43001f8c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:19.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:19.067+0000 7f4306105640 1 -- 172.21.15.38:0/937577612 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f430014fed0 con 0x7f43000b9ad0 2024-09-21T03:34:19.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:19.067+0000 7f4306105640 1 --2- 172.21.15.38:0/937577612 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f43000b9ad0 0x7f430014f990 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f42fc00d6d0 tx=0x7f42fc00db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:19.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:19.068+0000 7f42f6ffd640 1 -- 172.21.15.38:0/937577612 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42fc017070 con 0x7f43000b9ad0 2024-09-21T03:34:19.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:19.068+0000 7f42f6ffd640 1 -- 172.21.15.38:0/937577612 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f42fc005330 con 0x7f43000b9ad0 2024-09-21T03:34:19.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:19.068+0000 7f42f6ffd640 1 -- 172.21.15.38:0/937577612 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42fc009820 con 0x7f43000b9ad0 2024-09-21T03:34:19.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:19.068+0000 7f4308369640 1 -- 172.21.15.38:0/937577612 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43001f96d0 con 0x7f43000b9ad0 2024-09-21T03:34:19.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:19.068+0000 7f4308369640 1 -- 172.21.15.38:0/937577612 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43000cb860 con 0x7f43000b9ad0 2024-09-21T03:34:19.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:19.069+0000 7f4308369640 1 -- 172.21.15.38:0/937577612 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43000b6e10 con 0x7f43000b9ad0 2024-09-21T03:34:19.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:19.070+0000 7f42f6ffd640 1 -- 172.21.15.38:0/937577612 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f42fc009bd0 con 0x7f43000b9ad0 2024-09-21T03:34:19.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:19.070+0000 7f42f6ffd640 1 --2- 172.21.15.38:0/937577612 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f42e4078b90 0x7f42e407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:19.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:19.070+0000 7f42f6ffd640 1 -- 172.21.15.38:0/937577612 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6619+0+0 (secure 0 0 0) 0x7f42fc09b290 con 0x7f43000b9ad0 2024-09-21T03:34:19.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:19.070+0000 7f4305904640 1 --2- 172.21.15.38:0/937577612 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f42e4078b90 0x7f42e407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:19.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:19.071+0000 7f4305904640 1 --2- 172.21.15.38:0/937577612 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f42e4078b90 0x7f42e407b0a0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f4300158cb0 tx=0x7f42f0007e50 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:19.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:19.075+0000 7f42f6ffd640 1 -- 172.21.15.38:0/937577612 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f430014fed0 con 0x7f43000b9ad0 2024-09-21T03:34:19.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:19.273+0000 7f4308369640 1 -- 172.21.15.38:0/937577612 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f4300158ce0 con 0x7f43000b9ad0 2024-09-21T03:34:19.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:19.274+0000 7f42f6ffd640 1 -- 172.21.15.38:0/937577612 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v28) ==== 80+0+0 (secure 0 0 0) 0x7f42fc062d80 con 0x7f43000b9ad0 2024-09-21T03:34:19.276 INFO:teuthology.orchestra.run.smithi038.stderr:listed 0 entries 2024-09-21T03:34:19.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:19.276+0000 7f4308369640 1 -- 172.21.15.38:0/937577612 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f42e4078b90 msgr2=0x7f42e407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:19.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:19.276+0000 7f4308369640 1 --2- 172.21.15.38:0/937577612 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f42e4078b90 0x7f42e407b0a0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f4300158cb0 tx=0x7f42f0007e50 comp rx=0 tx=0).stop 2024-09-21T03:34:19.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:19.276+0000 7f4308369640 1 -- 172.21.15.38:0/937577612 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f43000b9ad0 msgr2=0x7f430014f990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:19.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:19.276+0000 7f4308369640 1 --2- 172.21.15.38:0/937577612 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f43000b9ad0 0x7f430014f990 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f42fc00d6d0 tx=0x7f42fc00db90 comp rx=0 tx=0).stop 2024-09-21T03:34:19.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:19.276+0000 7f4308369640 1 -- 172.21.15.38:0/937577612 shutdown_connections 2024-09-21T03:34:19.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:19.276+0000 7f4308369640 1 --2- 172.21.15.38:0/937577612 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f42e4078b90 0x7f42e407b0a0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:19.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:19.276+0000 7f4308369640 1 --2- 172.21.15.38:0/937577612 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f430015ee80 0x7f43000b2780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:19.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:19.276+0000 7f4308369640 1 --2- 172.21.15.38:0/937577612 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f43000ceb90 0x7f43001f8c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:19.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:19.276+0000 7f4308369640 1 --2- 172.21.15.38:0/937577612 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f43000b9ad0 0x7f430014f990 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:19.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:19.276+0000 7f4308369640 1 -- 172.21.15.38:0/937577612 >> 172.21.15.38:0/937577612 conn(0x7f430009d830 msgr2=0x7f4300161420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:19.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:19.276+0000 7f4308369640 1 -- 172.21.15.38:0/937577612 shutdown_connections 2024-09-21T03:34:19.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:19.276+0000 7f4308369640 1 -- 172.21.15.38:0/937577612 wait complete. 2024-09-21T03:34:19.289 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-09-21T03:34:19.289 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:34:19.289 DEBUG:teuthology.orchestra.run.smithi195:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2024-09-21T03:34:19.322 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi195.front.sepia.ceph.com 2024-09-21T03:34:19.322 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:34:19.322 DEBUG:teuthology.orchestra.run.smithi195:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-21T03:34:19.528 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-21T03:34:19.528 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:34:19.545 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-21T03:34:19.545 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:34:19.602 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-21T03:34:19.602 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:34:19.659 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T03:34:19.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:19.814+0000 7f02686b2640 1 -- 172.21.15.38:0/2960423003 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f026015ee80 msgr2=0x7f02600d0170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:19.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:19.814+0000 7f02686b2640 1 --2- 172.21.15.38:0/2960423003 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f026015ee80 0x7f02600d0170 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f02601623b0 tx=0x7f025002a980 comp rx=0 tx=0).stop 2024-09-21T03:34:19.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:19.814+0000 7f02686b2640 1 -- 172.21.15.38:0/2960423003 shutdown_connections 2024-09-21T03:34:19.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:19.814+0000 7f02686b2640 1 --2- 172.21.15.38:0/2960423003 >> v2:172.21.15.195:3300/0 conn(0x7f02600d1120 0x7f02600ceb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:19.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:19.814+0000 7f02686b2640 1 --2- 172.21.15.38:0/2960423003 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f02600b9ad0 0x7f02600d0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:19.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:19.814+0000 7f02686b2640 1 --2- 172.21.15.38:0/2960423003 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f026015ee80 0x7f02600d0170 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:19.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:19.814+0000 7f02686b2640 1 -- 172.21.15.38:0/2960423003 >> 172.21.15.38:0/2960423003 conn(0x7f026009d830 msgr2=0x7f02600b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:19.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:19.814+0000 7f02686b2640 1 -- 172.21.15.38:0/2960423003 shutdown_connections 2024-09-21T03:34:19.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:19.814+0000 7f02686b2640 1 -- 172.21.15.38:0/2960423003 wait complete. 2024-09-21T03:34:19.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:19.816+0000 7f02686b2640 1 Processor -- start 2024-09-21T03:34:19.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:19.816+0000 7f02686b2640 1 -- start start 2024-09-21T03:34:19.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:19.817+0000 7f02686b2640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f02600b9ad0 0x7f0260157b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:19.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:19.817+0000 7f02686b2640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f02600d1120 0x7f02600a07d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:19.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:19.817+0000 7f02686b2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f026015ee80 0x7f026006e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:19.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:19.817+0000 7f02686b2640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f02600cf510 con 0x7f026015ee80 2024-09-21T03:34:19.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:19.817+0000 7f02686b2640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f02600cf390 con 0x7f02600b9ad0 2024-09-21T03:34:19.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:19.817+0000 7f02686b2640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f02600b6580 con 0x7f02600d1120 2024-09-21T03:34:19.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:19.818+0000 7f0266c4f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f026015ee80 0x7f026006e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:19.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:19.818+0000 7f0266c4f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f026015ee80 0x7f026006e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54426/0 (socket says 172.21.15.38:54426) 2024-09-21T03:34:19.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:19.818+0000 7f0266c4f640 1 -- 172.21.15.38:0/2910854288 learned_addr learned my addr 172.21.15.38:0/2910854288 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:34:19.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:19.818+0000 7f026644e640 1 --2- 172.21.15.38:0/2910854288 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f02600b9ad0 0x7f0260157b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:19.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:19.818+0000 7f0265c4d640 1 --2- 172.21.15.38:0/2910854288 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f02600d1120 0x7f02600a07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:19.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:19.818+0000 7f0266c4f640 1 -- 172.21.15.38:0/2910854288 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f02600d1120 msgr2=0x7f02600a07d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:19.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:19.818+0000 7f0266c4f640 1 --2- 172.21.15.38:0/2910854288 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f02600d1120 0x7f02600a07d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:19.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:19.818+0000 7f0266c4f640 1 -- 172.21.15.38:0/2910854288 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f02600b9ad0 msgr2=0x7f0260157b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:19.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:19.818+0000 7f0266c4f640 1 --2- 172.21.15.38:0/2910854288 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f02600b9ad0 0x7f0260157b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:19.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:19.818+0000 7f0266c4f640 1 -- 172.21.15.38:0/2910854288 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0260158050 con 0x7f026015ee80 2024-09-21T03:34:19.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:19.818+0000 7f0266c4f640 1 --2- 172.21.15.38:0/2910854288 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f026015ee80 0x7f026006e400 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f0258005d00 tx=0x7f025800dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:19.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:19.818+0000 7f025effd640 1 -- 172.21.15.38:0/2910854288 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0258017070 con 0x7f026015ee80 2024-09-21T03:34:19.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:19.818+0000 7f025effd640 1 -- 172.21.15.38:0/2910854288 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0258005330 con 0x7f026015ee80 2024-09-21T03:34:19.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:19.818+0000 7f025effd640 1 -- 172.21.15.38:0/2910854288 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02580047a0 con 0x7f026015ee80 2024-09-21T03:34:19.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:19.818+0000 7f02686b2640 1 -- 172.21.15.38:0/2910854288 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02600a1200 con 0x7f026015ee80 2024-09-21T03:34:19.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:19.819+0000 7f02686b2640 1 -- 172.21.15.38:0/2910854288 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02600a17c0 con 0x7f026015ee80 2024-09-21T03:34:19.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:19.819+0000 7f02686b2640 1 -- 172.21.15.38:0/2910854288 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02600ceb90 con 0x7f026015ee80 2024-09-21T03:34:19.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:19.820+0000 7f025effd640 1 -- 172.21.15.38:0/2910854288 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f0258004940 con 0x7f026015ee80 2024-09-21T03:34:19.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:19.821+0000 7f025effd640 1 --2- 172.21.15.38:0/2910854288 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f024c078bc0 0x7f024c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:19.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:19.821+0000 7f025effd640 1 -- 172.21.15.38:0/2910854288 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6619+0+0 (secure 0 0 0) 0x7f025809af20 con 0x7f026015ee80 2024-09-21T03:34:19.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:19.821+0000 7f026644e640 1 --2- 172.21.15.38:0/2910854288 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f024c078bc0 0x7f024c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:19.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:19.822+0000 7f026644e640 1 --2- 172.21.15.38:0/2910854288 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f024c078bc0 0x7f024c07b0d0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f0250035000 tx=0x7f0250004ab0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:19.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:19.825+0000 7f025effd640 1 -- 172.21.15.38:0/2910854288 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f0258062aa0 con 0x7f026015ee80 2024-09-21T03:34:20.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:20.023+0000 7f02686b2640 1 -- 172.21.15.38:0/2910854288 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f02600b31a0 con 0x7f026015ee80 2024-09-21T03:34:20.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:20.024+0000 7f025effd640 1 -- 172.21.15.38:0/2910854288 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v28) ==== 80+0+0 (secure 0 0 0) 0x7f02580674d0 con 0x7f026015ee80 2024-09-21T03:34:20.026 INFO:teuthology.orchestra.run.smithi038.stderr:listed 0 entries 2024-09-21T03:34:20.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:20.026+0000 7f02686b2640 1 -- 172.21.15.38:0/2910854288 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f024c078bc0 msgr2=0x7f024c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:20.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:20.026+0000 7f02686b2640 1 --2- 172.21.15.38:0/2910854288 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f024c078bc0 0x7f024c07b0d0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f0250035000 tx=0x7f0250004ab0 comp rx=0 tx=0).stop 2024-09-21T03:34:20.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:20.026+0000 7f02686b2640 1 -- 172.21.15.38:0/2910854288 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f026015ee80 msgr2=0x7f026006e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:20.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:20.026+0000 7f02686b2640 1 --2- 172.21.15.38:0/2910854288 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f026015ee80 0x7f026006e400 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f0258005d00 tx=0x7f025800dae0 comp rx=0 tx=0).stop 2024-09-21T03:34:20.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:20.026+0000 7f02686b2640 1 -- 172.21.15.38:0/2910854288 shutdown_connections 2024-09-21T03:34:20.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:20.026+0000 7f02686b2640 1 --2- 172.21.15.38:0/2910854288 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f026015ee80 0x7f026006e400 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:20.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:20.026+0000 7f02686b2640 1 --2- 172.21.15.38:0/2910854288 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f024c078bc0 0x7f024c07b0d0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:20.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:20.026+0000 7f02686b2640 1 --2- 172.21.15.38:0/2910854288 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f02600d1120 0x7f02600a07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:20.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:20.026+0000 7f02686b2640 1 --2- 172.21.15.38:0/2910854288 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f02600b9ad0 0x7f0260157b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:20.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:20.026+0000 7f02686b2640 1 -- 172.21.15.38:0/2910854288 >> 172.21.15.38:0/2910854288 conn(0x7f026009d830 msgr2=0x7f02600b57f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:20.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:20.026+0000 7f02686b2640 1 -- 172.21.15.38:0/2910854288 shutdown_connections 2024-09-21T03:34:20.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:20.026+0000 7f02686b2640 1 -- 172.21.15.38:0/2910854288 wait complete. 2024-09-21T03:34:20.039 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2024-09-21T03:34:20.039 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:34:20.039 DEBUG:teuthology.orchestra.run.smithi038:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2024-09-21T03:34:20.066 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi038.front.sepia.ceph.com 2024-09-21T03:34:20.066 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:34:20.066 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-09-21T03:34:20.122 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-09-21T03:34:20.123 DEBUG:teuthology.orchestra.run.smithi195:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:34:20.142 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-09-21T03:34:20.142 DEBUG:teuthology.orchestra.run.smithi195:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:34:20.199 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-09-21T03:34:20.200 DEBUG:teuthology.orchestra.run.smithi195:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:34:20.257 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T03:34:20.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:20.376+0000 7fe38f7ba640 1 -- 172.21.15.38:0/2256146206 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe374004800 msgr2=0x7fe374004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:20.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:20.376+0000 7fe38f7ba640 1 --2- 172.21.15.38:0/2256146206 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe374004800 0x7fe374004c10 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fe378001180 tx=0x7fe374005830 comp rx=0 tx=0).stop 2024-09-21T03:34:20.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:20.376+0000 7fe38dd57640 1 -- 172.21.15.38:0/2256146206 reap_dead start 2024-09-21T03:34:20.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:20.376+0000 7fe38f7ba640 1 -- 172.21.15.38:0/2256146206 shutdown_connections 2024-09-21T03:34:20.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:20.376+0000 7fe38f7ba640 1 -- 172.21.15.38:0/2256146206 >> 172.21.15.38:0/2256146206 conn(0x7fe38809d830 msgr2=0x7fe3880b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:20.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:20.377+0000 7fe38f7ba640 1 -- 172.21.15.38:0/2256146206 shutdown_connections 2024-09-21T03:34:20.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:20.377+0000 7fe38f7ba640 1 -- 172.21.15.38:0/2256146206 wait complete. 2024-09-21T03:34:20.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:20.378+0000 7fe38f7ba640 1 Processor -- start 2024-09-21T03:34:20.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:20.378+0000 7fe38f7ba640 1 -- start start 2024-09-21T03:34:20.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:20.380+0000 7fe38f7ba640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe374004800 0x7fe38806d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:20.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:20.380+0000 7fe38f7ba640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe38806eb40 0x7fe388157880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:20.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:20.380+0000 7fe38f7ba640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe3880d3e50 0x7fe3880d4260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:20.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:20.380+0000 7fe38f7ba640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fe3880cf330 con 0x7fe3880d3e50 2024-09-21T03:34:20.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:20.380+0000 7fe38f7ba640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fe3880cf1b0 con 0x7fe374004800 2024-09-21T03:34:20.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:20.380+0000 7fe38f7ba640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fe3880b6580 con 0x7fe38806eb40 2024-09-21T03:34:20.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:20.380+0000 7fe38dd57640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe3880d3e50 0x7fe3880d4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:20.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:20.380+0000 7fe38d556640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe374004800 0x7fe38806d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:20.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:20.380+0000 7fe38dd57640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe3880d3e50 0x7fe3880d4260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54440/0 (socket says 172.21.15.38:54440) 2024-09-21T03:34:20.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:20.380+0000 7fe38dd57640 1 -- 172.21.15.38:0/1319617008 learned_addr learned my addr 172.21.15.38:0/1319617008 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:34:20.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:20.380+0000 7fe38cd55640 1 --2- 172.21.15.38:0/1319617008 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe38806eb40 0x7fe388157880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:20.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:20.380+0000 7fe38dd57640 1 -- 172.21.15.38:0/1319617008 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe38806eb40 msgr2=0x7fe388157880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:20.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:20.380+0000 7fe38dd57640 1 --2- 172.21.15.38:0/1319617008 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe38806eb40 0x7fe388157880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:20.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:20.380+0000 7fe38dd57640 1 -- 172.21.15.38:0/1319617008 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe374004800 msgr2=0x7fe38806d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:20.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:20.380+0000 7fe38dd57640 1 --2- 172.21.15.38:0/1319617008 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe374004800 0x7fe38806d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:20.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:20.380+0000 7fe38dd57640 1 -- 172.21.15.38:0/1319617008 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3880d47a0 con 0x7fe3880d3e50 2024-09-21T03:34:20.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:20.380+0000 7fe38dd57640 1 --2- 172.21.15.38:0/1319617008 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe3880d3e50 0x7fe3880d4260 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fe38815c9c0 tx=0x7fe38000f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:20.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:20.381+0000 7fe37dffb640 1 -- 172.21.15.38:0/1319617008 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe380004b20 con 0x7fe3880d3e50 2024-09-21T03:34:20.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:20.381+0000 7fe37dffb640 1 -- 172.21.15.38:0/1319617008 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe380004cc0 con 0x7fe3880d3e50 2024-09-21T03:34:20.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:20.381+0000 7fe38f7ba640 1 -- 172.21.15.38:0/1319617008 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe388159700 con 0x7fe3880d3e50 2024-09-21T03:34:20.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:20.381+0000 7fe37dffb640 1 -- 172.21.15.38:0/1319617008 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe380003bd0 con 0x7fe3880d3e50 2024-09-21T03:34:20.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:20.381+0000 7fe38f7ba640 1 -- 172.21.15.38:0/1319617008 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe388159cc0 con 0x7fe3880d3e50 2024-09-21T03:34:20.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:20.382+0000 7fe38f7ba640 1 -- 172.21.15.38:0/1319617008 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3880b6ae0 con 0x7fe3880d3e50 2024-09-21T03:34:20.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:20.382+0000 7fe37dffb640 1 -- 172.21.15.38:0/1319617008 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fe380004030 con 0x7fe3880d3e50 2024-09-21T03:34:20.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:20.383+0000 7fe37dffb640 1 --2- 172.21.15.38:0/1319617008 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fe36c078b90 0x7fe36c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:20.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:20.383+0000 7fe37dffb640 1 -- 172.21.15.38:0/1319617008 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6619+0+0 (secure 0 0 0) 0x7fe38009b8a0 con 0x7fe3880d3e50 2024-09-21T03:34:20.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:20.383+0000 7fe38d556640 1 --2- 172.21.15.38:0/1319617008 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fe36c078b90 0x7fe36c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:20.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:20.384+0000 7fe38d556640 1 --2- 172.21.15.38:0/1319617008 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fe36c078b90 0x7fe36c07b0a0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fe384000c00 tx=0x7fe384006b70 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:20.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:20.388+0000 7fe37dffb640 1 -- 172.21.15.38:0/1319617008 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fe3880d47a0 con 0x7fe3880d3e50 2024-09-21T03:34:20.589 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:20.587+0000 7fe38f7ba640 1 -- 172.21.15.38:0/1319617008 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fe38806dd40 con 0x7fe3880d3e50 2024-09-21T03:34:20.589 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:20.587+0000 7fe37dffb640 1 -- 172.21.15.38:0/1319617008 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v28) ==== 80+0+0 (secure 0 0 0) 0x7fe380063300 con 0x7fe3880d3e50 2024-09-21T03:34:20.589 INFO:teuthology.orchestra.run.smithi038.stderr:listed 0 entries 2024-09-21T03:34:20.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:20.589+0000 7fe38f7ba640 1 -- 172.21.15.38:0/1319617008 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fe36c078b90 msgr2=0x7fe36c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:20.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:20.589+0000 7fe38f7ba640 1 --2- 172.21.15.38:0/1319617008 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fe36c078b90 0x7fe36c07b0a0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fe384000c00 tx=0x7fe384006b70 comp rx=0 tx=0).stop 2024-09-21T03:34:20.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:20.589+0000 7fe38f7ba640 1 -- 172.21.15.38:0/1319617008 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe3880d3e50 msgr2=0x7fe3880d4260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:20.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:20.589+0000 7fe38f7ba640 1 --2- 172.21.15.38:0/1319617008 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe3880d3e50 0x7fe3880d4260 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fe38815c9c0 tx=0x7fe38000f3c0 comp rx=0 tx=0).stop 2024-09-21T03:34:20.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:20.589+0000 7fe38f7ba640 1 -- 172.21.15.38:0/1319617008 shutdown_connections 2024-09-21T03:34:20.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:20.589+0000 7fe38f7ba640 1 --2- 172.21.15.38:0/1319617008 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe3880d3e50 0x7fe3880d4260 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:20.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:20.589+0000 7fe38f7ba640 1 --2- 172.21.15.38:0/1319617008 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fe36c078b90 0x7fe36c07b0a0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:20.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:20.589+0000 7fe38f7ba640 1 --2- 172.21.15.38:0/1319617008 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe38806eb40 0x7fe388157880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:20.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:20.589+0000 7fe38f7ba640 1 --2- 172.21.15.38:0/1319617008 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe374004800 0x7fe38806d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:20.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:20.589+0000 7fe38f7ba640 1 -- 172.21.15.38:0/1319617008 >> 172.21.15.38:0/1319617008 conn(0x7fe38809d830 msgr2=0x7fe388161b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:20.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:20.589+0000 7fe38f7ba640 1 -- 172.21.15.38:0/1319617008 shutdown_connections 2024-09-21T03:34:20.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:20.590+0000 7fe38f7ba640 1 -- 172.21.15.38:0/1319617008 wait complete. 2024-09-21T03:34:20.602 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2024-09-21T03:34:20.603 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:34:20.603 DEBUG:teuthology.orchestra.run.smithi195:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2024-09-21T03:34:20.634 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi195.front.sepia.ceph.com 2024-09-21T03:34:20.634 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:34:20.634 DEBUG:teuthology.orchestra.run.smithi195:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-09-21T03:34:20.691 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-21T03:34:20.691 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:34:20.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:20.805+0000 7f1b692bd640 1 -- 172.21.15.38:0/131428409 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1b640cee70 msgr2=0x7f1b6415b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:20.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:20.805+0000 7f1b692bd640 1 --2- 172.21.15.38:0/131428409 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1b640cee70 0x7f1b6415b350 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f1b640710f0 tx=0x7f1b5402a560 comp rx=0 tx=0).stop 2024-09-21T03:34:20.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:20.806+0000 7f1b692bd640 1 -- 172.21.15.38:0/131428409 shutdown_connections 2024-09-21T03:34:20.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:20.806+0000 7f1b692bd640 1 --2- 172.21.15.38:0/131428409 >> v2:172.21.15.195:3300/0 conn(0x7f1b6415f150 0x7f1b6415f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:20.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:20.806+0000 7f1b692bd640 1 --2- 172.21.15.38:0/131428409 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1b640cee70 0x7f1b6415b350 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:20.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:20.806+0000 7f1b692bd640 1 -- 172.21.15.38:0/131428409 >> 172.21.15.38:0/131428409 conn(0x7f1b64073e30 msgr2=0x7f1b64074240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:20.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:20.806+0000 7f1b692bd640 1 -- 172.21.15.38:0/131428409 shutdown_connections 2024-09-21T03:34:20.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:20.806+0000 7f1b692bd640 1 -- 172.21.15.38:0/131428409 wait complete. 2024-09-21T03:34:20.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:20.807+0000 7f1b692bd640 1 Processor -- start 2024-09-21T03:34:20.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:20.808+0000 7f1b692bd640 1 -- start start 2024-09-21T03:34:20.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:20.809+0000 7f1b692bd640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1b640cee70 0x7f1b64150760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:20.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:20.809+0000 7f1b692bd640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1b6415f150 0x7f1b6414a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:20.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:20.809+0000 7f1b692bd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1b6415faa0 0x7f1b6414a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:20.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:20.809+0000 7f1b692bd640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f1b6406f050 con 0x7f1b6415faa0 2024-09-21T03:34:20.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:20.809+0000 7f1b692bd640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f1b640b52e0 con 0x7f1b640cee70 2024-09-21T03:34:20.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:20.809+0000 7f1b692bd640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f1b6406f1d0 con 0x7f1b6415f150 2024-09-21T03:34:20.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:20.809+0000 7f1b637fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1b6415faa0 0x7f1b6414a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:20.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:20.809+0000 7f1b637fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1b6415faa0 0x7f1b6414a7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54462/0 (socket says 172.21.15.38:54462) 2024-09-21T03:34:20.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:20.809+0000 7f1b637fe640 1 -- 172.21.15.38:0/1705947434 learned_addr learned my addr 172.21.15.38:0/1705947434 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:34:20.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:20.809+0000 7f1b627fc640 1 --2- 172.21.15.38:0/1705947434 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1b6415f150 0x7f1b6414a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:20.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:20.809+0000 7f1b62ffd640 1 --2- 172.21.15.38:0/1705947434 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1b640cee70 0x7f1b64150760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:20.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:20.809+0000 7f1b637fe640 1 -- 172.21.15.38:0/1705947434 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1b6415f150 msgr2=0x7f1b6414a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:20.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:20.809+0000 7f1b637fe640 1 --2- 172.21.15.38:0/1705947434 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1b6415f150 0x7f1b6414a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:20.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:20.809+0000 7f1b637fe640 1 -- 172.21.15.38:0/1705947434 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1b640cee70 msgr2=0x7f1b64150760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:20.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:20.809+0000 7f1b637fe640 1 --2- 172.21.15.38:0/1705947434 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1b640cee70 0x7f1b64150760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:20.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:20.809+0000 7f1b637fe640 1 -- 172.21.15.38:0/1705947434 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b640ad850 con 0x7f1b6415faa0 2024-09-21T03:34:20.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:20.810+0000 7f1b637fe640 1 --2- 172.21.15.38:0/1705947434 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1b6415faa0 0x7f1b6414a7e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f1b5c002230 tx=0x7f1b5c00cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:20.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:20.810+0000 7f1b4b7fe640 1 -- 172.21.15.38:0/1705947434 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b5c003b70 con 0x7f1b6415faa0 2024-09-21T03:34:20.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:20.810+0000 7f1b4b7fe640 1 -- 172.21.15.38:0/1705947434 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1b5c003d10 con 0x7f1b6415faa0 2024-09-21T03:34:20.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:20.810+0000 7f1b692bd640 1 -- 172.21.15.38:0/1705947434 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b640b25b0 con 0x7f1b6415faa0 2024-09-21T03:34:20.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:20.810+0000 7f1b4b7fe640 1 -- 172.21.15.38:0/1705947434 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b5c005330 con 0x7f1b6415faa0 2024-09-21T03:34:20.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:20.810+0000 7f1b692bd640 1 -- 172.21.15.38:0/1705947434 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b64158b00 con 0x7f1b6415faa0 2024-09-21T03:34:20.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:20.811+0000 7f1b692bd640 1 -- 172.21.15.38:0/1705947434 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b640ada20 con 0x7f1b6415faa0 2024-09-21T03:34:20.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:20.812+0000 7f1b4b7fe640 1 -- 172.21.15.38:0/1705947434 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f1b5c0054d0 con 0x7f1b6415faa0 2024-09-21T03:34:20.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:20.812+0000 7f1b4b7fe640 1 --2- 172.21.15.38:0/1705947434 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1b50078bc0 0x7f1b5007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:20.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:20.812+0000 7f1b4b7fe640 1 -- 172.21.15.38:0/1705947434 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6619+0+0 (secure 0 0 0) 0x7f1b5c09ad50 con 0x7f1b6415faa0 2024-09-21T03:34:20.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:20.812+0000 7f1b62ffd640 1 --2- 172.21.15.38:0/1705947434 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1b50078bc0 0x7f1b5007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:20.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:20.813+0000 7f1b62ffd640 1 --2- 172.21.15.38:0/1705947434 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1b50078bc0 0x7f1b5007b0d0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f1b54033000 tx=0x7f1b54004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:20.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:20.817+0000 7f1b4b7fe640 1 -- 172.21.15.38:0/1705947434 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f1b5c0627c0 con 0x7f1b6415faa0 2024-09-21T03:34:21.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:21.041+0000 7f1b692bd640 1 -- 172.21.15.38:0/1705947434 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1b6406f680 con 0x7f1b6415faa0 2024-09-21T03:34:21.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:21.042+0000 7f1b4b7fe640 1 -- 172.21.15.38:0/1705947434 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5219 (secure 0 0 0) 0x7f1b5c0671f0 con 0x7f1b6415faa0 2024-09-21T03:34:21.044 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:34:21.044 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":9,"btime":"2024-09-21T03:33:56:329606+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4322,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/593838590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":593838590},{"type":"v1","addr":"172.21.15.195:6836","nonce":593838590}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4374,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/663402481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":663402481},{"type":"v1","addr":"172.21.15.38:6833","nonce":663402481}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4377,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/2614829035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":2614829035},{"type":"v1","addr":"172.21.15.38:6835","nonce":2614829035}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:33:35.622623+0000","modified":"2024-09-21T03:33:55.704494+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4402},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4402":{"gid":4402,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.195:6837/1420805977","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":1420805977},{"type":"v1","addr":"172.21.15.195:6837","nonce":1420805977}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4402,"qdb_cluster":[4402]},"id":1}]} 2024-09-21T03:34:21.044 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 9 2024-09-21T03:34:21.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:21.044+0000 7f1b692bd640 1 -- 172.21.15.38:0/1705947434 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1b50078bc0 msgr2=0x7f1b5007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:21.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:21.044+0000 7f1b692bd640 1 --2- 172.21.15.38:0/1705947434 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1b50078bc0 0x7f1b5007b0d0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f1b54033000 tx=0x7f1b54004730 comp rx=0 tx=0).stop 2024-09-21T03:34:21.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:21.044+0000 7f1b692bd640 1 -- 172.21.15.38:0/1705947434 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1b6415faa0 msgr2=0x7f1b6414a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:21.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:21.044+0000 7f1b692bd640 1 --2- 172.21.15.38:0/1705947434 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1b6415faa0 0x7f1b6414a7e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f1b5c002230 tx=0x7f1b5c00cb90 comp rx=0 tx=0).stop 2024-09-21T03:34:21.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:21.044+0000 7f1b692bd640 1 -- 172.21.15.38:0/1705947434 shutdown_connections 2024-09-21T03:34:21.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:21.044+0000 7f1b692bd640 1 --2- 172.21.15.38:0/1705947434 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1b6415faa0 0x7f1b6414a7e0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:21.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:21.044+0000 7f1b692bd640 1 --2- 172.21.15.38:0/1705947434 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1b50078bc0 0x7f1b5007b0d0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:21.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:21.044+0000 7f1b692bd640 1 --2- 172.21.15.38:0/1705947434 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1b6415f150 0x7f1b6414a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:21.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:21.044+0000 7f1b692bd640 1 --2- 172.21.15.38:0/1705947434 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1b640cee70 0x7f1b64150760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:21.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:21.044+0000 7f1b692bd640 1 -- 172.21.15.38:0/1705947434 >> 172.21.15.38:0/1705947434 conn(0x7f1b64073e30 msgr2=0x7f1b6415e110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:21.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:21.044+0000 7f1b692bd640 1 -- 172.21.15.38:0/1705947434 shutdown_connections 2024-09-21T03:34:21.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:21.044+0000 7f1b692bd640 1 -- 172.21.15.38:0/1705947434 wait complete. 2024-09-21T03:34:21.058 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:34:21.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:21.172+0000 7f119cfad640 1 -- 172.21.15.38:0/453035027 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f119806e160 msgr2=0x7f11981673c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:21.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:21.172+0000 7f119cfad640 1 --2- 172.21.15.38:0/453035027 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f119806e160 0x7f11981673c0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f1188005fd0 tx=0x7f118802a8b0 comp rx=0 tx=0).stop 2024-09-21T03:34:21.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:21.172+0000 7f119cfad640 1 -- 172.21.15.38:0/453035027 shutdown_connections 2024-09-21T03:34:21.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:21.172+0000 7f119cfad640 1 --2- 172.21.15.38:0/453035027 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f119806e160 0x7f11981673c0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:21.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:21.172+0000 7f119cfad640 1 -- 172.21.15.38:0/453035027 >> 172.21.15.38:0/453035027 conn(0x7f11980d0a30 msgr2=0x7f11980d0e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:21.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:21.172+0000 7f119cfad640 1 -- 172.21.15.38:0/453035027 shutdown_connections 2024-09-21T03:34:21.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:21.172+0000 7f119cfad640 1 -- 172.21.15.38:0/453035027 wait complete. 2024-09-21T03:34:21.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:21.174+0000 7f119cfad640 1 Processor -- start 2024-09-21T03:34:21.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:21.174+0000 7f119cfad640 1 -- start start 2024-09-21T03:34:21.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:21.175+0000 7f119cfad640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f119806e160 0x7f1198159b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:21.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:21.175+0000 7f119cfad640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f11980cf700 0x7f1198155e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:21.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:21.175+0000 7f119cfad640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1198153410 0x7f11980b4130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:21.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:21.175+0000 7f119cfad640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f1198167ac0 con 0x7f11980cf700 2024-09-21T03:34:21.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:21.175+0000 7f119cfad640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f1198167940 con 0x7f119806e160 2024-09-21T03:34:21.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:21.175+0000 7f119cfad640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f1198167c40 con 0x7f1198153410 2024-09-21T03:34:21.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:21.175+0000 7f1195d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f11980cf700 0x7f1198155e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:21.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:21.175+0000 7f1195d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f11980cf700 0x7f1198155e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54468/0 (socket says 172.21.15.38:54468) 2024-09-21T03:34:21.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:21.175+0000 7f1195d9b640 1 -- 172.21.15.38:0/2593925637 learned_addr learned my addr 172.21.15.38:0/2593925637 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:34:21.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:21.176+0000 7f119659c640 1 --2- 172.21.15.38:0/2593925637 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f119806e160 0x7f1198159b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:21.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:21.176+0000 7f1196d9d640 1 --2- 172.21.15.38:0/2593925637 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1198153410 0x7f11980b4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:21.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:21.176+0000 7f1195d9b640 1 -- 172.21.15.38:0/2593925637 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1198153410 msgr2=0x7f11980b4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:21.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:21.176+0000 7f1195d9b640 1 --2- 172.21.15.38:0/2593925637 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1198153410 0x7f11980b4130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:21.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:21.176+0000 7f1195d9b640 1 -- 172.21.15.38:0/2593925637 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f119806e160 msgr2=0x7f1198159b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:21.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:21.176+0000 7f1195d9b640 1 --2- 172.21.15.38:0/2593925637 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f119806e160 0x7f1198159b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:21.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:21.176+0000 7f1195d9b640 1 -- 172.21.15.38:0/2593925637 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11981588a0 con 0x7f11980cf700 2024-09-21T03:34:21.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:21.176+0000 7f119659c640 1 --2- 172.21.15.38:0/2593925637 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f119806e160 0x7f1198159b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:34:21.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:21.176+0000 7f1195d9b640 1 --2- 172.21.15.38:0/2593925637 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f11980cf700 0x7f1198155e50 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f118000c370 tx=0x7f118000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:21.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:21.176+0000 7f118effd640 1 -- 172.21.15.38:0/2593925637 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1180017070 con 0x7f11980cf700 2024-09-21T03:34:21.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:21.176+0000 7f118effd640 1 -- 172.21.15.38:0/2593925637 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f11800035f0 con 0x7f11980cf700 2024-09-21T03:34:21.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:21.176+0000 7f119cfad640 1 -- 172.21.15.38:0/2593925637 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1198158b80 con 0x7f11980cf700 2024-09-21T03:34:21.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:21.176+0000 7f118effd640 1 -- 172.21.15.38:0/2593925637 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1180003a40 con 0x7f11980cf700 2024-09-21T03:34:21.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:21.176+0000 7f119cfad640 1 -- 172.21.15.38:0/2593925637 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11981552c0 con 0x7f11980cf700 2024-09-21T03:34:21.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:21.177+0000 7f119cfad640 1 -- 172.21.15.38:0/2593925637 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11980cfb10 con 0x7f11980cf700 2024-09-21T03:34:21.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:21.178+0000 7f118effd640 1 -- 172.21.15.38:0/2593925637 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f118001a070 con 0x7f11980cf700 2024-09-21T03:34:21.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:21.179+0000 7f118effd640 1 --2- 172.21.15.38:0/2593925637 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1178078b90 0x7f117807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:21.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:21.179+0000 7f118effd640 1 -- 172.21.15.38:0/2593925637 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6619+0+0 (secure 0 0 0) 0x7f118009a2f0 con 0x7f11980cf700 2024-09-21T03:34:21.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:21.179+0000 7f119659c640 1 --2- 172.21.15.38:0/2593925637 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1178078b90 0x7f117807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:21.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:21.179+0000 7f119659c640 1 --2- 172.21.15.38:0/2593925637 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1178078b90 0x7f117807b0a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f1190007fd0 tx=0x7f1190009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:21.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:21.183+0000 7f118effd640 1 -- 172.21.15.38:0/2593925637 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f1180062430 con 0x7f11980cf700 2024-09-21T03:34:21.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:21.408+0000 7f119cfad640 1 -- 172.21.15.38:0/2593925637 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1198069d30 con 0x7f11980cf700 2024-09-21T03:34:21.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:21.408+0000 7f118effd640 1 -- 172.21.15.38:0/2593925637 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5219 (secure 0 0 0) 0x7f1180061d30 con 0x7f11980cf700 2024-09-21T03:34:21.410 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:34:21.410 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":9,"btime":"2024-09-21T03:33:56:329606+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4322,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/593838590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":593838590},{"type":"v1","addr":"172.21.15.195:6836","nonce":593838590}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4374,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/663402481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":663402481},{"type":"v1","addr":"172.21.15.38:6833","nonce":663402481}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4377,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/2614829035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":2614829035},{"type":"v1","addr":"172.21.15.38:6835","nonce":2614829035}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:33:35.622623+0000","modified":"2024-09-21T03:33:55.704494+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4402},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4402":{"gid":4402,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.195:6837/1420805977","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":1420805977},{"type":"v1","addr":"172.21.15.195:6837","nonce":1420805977}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4402,"qdb_cluster":[4402]},"id":1}]} 2024-09-21T03:34:21.411 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 9 2024-09-21T03:34:21.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:21.410+0000 7f119cfad640 1 -- 172.21.15.38:0/2593925637 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1178078b90 msgr2=0x7f117807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:21.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:21.410+0000 7f119cfad640 1 --2- 172.21.15.38:0/2593925637 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1178078b90 0x7f117807b0a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f1190007fd0 tx=0x7f1190009040 comp rx=0 tx=0).stop 2024-09-21T03:34:21.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:21.410+0000 7f119cfad640 1 -- 172.21.15.38:0/2593925637 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f11980cf700 msgr2=0x7f1198155e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:21.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:21.410+0000 7f119cfad640 1 --2- 172.21.15.38:0/2593925637 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f11980cf700 0x7f1198155e50 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f118000c370 tx=0x7f118000c830 comp rx=0 tx=0).stop 2024-09-21T03:34:21.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:21.410+0000 7f119cfad640 1 -- 172.21.15.38:0/2593925637 shutdown_connections 2024-09-21T03:34:21.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:21.410+0000 7f119cfad640 1 --2- 172.21.15.38:0/2593925637 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1178078b90 0x7f117807b0a0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:21.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:21.410+0000 7f119cfad640 1 --2- 172.21.15.38:0/2593925637 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1198153410 0x7f11980b4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:21.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:21.410+0000 7f119cfad640 1 --2- 172.21.15.38:0/2593925637 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f11980cf700 0x7f1198155e50 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:21.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:21.410+0000 7f119cfad640 1 --2- 172.21.15.38:0/2593925637 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f119806e160 0x7f1198159b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:21.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:21.410+0000 7f119cfad640 1 -- 172.21.15.38:0/2593925637 >> 172.21.15.38:0/2593925637 conn(0x7f11980d0a30 msgr2=0x7f1198073d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:21.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:21.410+0000 7f119cfad640 1 -- 172.21.15.38:0/2593925637 shutdown_connections 2024-09-21T03:34:21.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:21.411+0000 7f119cfad640 1 -- 172.21.15.38:0/2593925637 wait complete. 2024-09-21T03:34:21.424 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T03:34:21.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:21.540+0000 7f0bbf98b640 1 -- 172.21.15.38:0/3233328932 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0bb81627b0 msgr2=0x7f0bb8162bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:21.543 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:21.540+0000 7f0bbf98b640 1 --2- 172.21.15.38:0/3233328932 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0bb81627b0 0x7f0bb8162bc0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f0bb808d3a0 tx=0x7f0bac02a530 comp rx=0 tx=0).stop 2024-09-21T03:34:21.543 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:21.540+0000 7f0bbf98b640 1 -- 172.21.15.38:0/3233328932 shutdown_connections 2024-09-21T03:34:21.543 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:21.540+0000 7f0bbf98b640 1 --2- 172.21.15.38:0/3233328932 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0bb81627b0 0x7f0bb8162bc0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:21.543 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:21.540+0000 7f0bbf98b640 1 --2- 172.21.15.38:0/3233328932 >> v2:172.21.15.195:3300/0 conn(0x7f0bb815ecf0 0x7f0bb80cf480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:21.543 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:21.540+0000 7f0bbf98b640 1 -- 172.21.15.38:0/3233328932 >> 172.21.15.38:0/3233328932 conn(0x7f0bb80d4600 msgr2=0x7f0bb80d4a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:21.543 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:21.540+0000 7f0bbf98b640 1 -- 172.21.15.38:0/3233328932 shutdown_connections 2024-09-21T03:34:21.543 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:21.540+0000 7f0bbf98b640 1 -- 172.21.15.38:0/3233328932 wait complete. 2024-09-21T03:34:21.544 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:21.542+0000 7f0bbf98b640 1 Processor -- start 2024-09-21T03:34:21.544 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:21.542+0000 7f0bbf98b640 1 -- start start 2024-09-21T03:34:21.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:21.543+0000 7f0bbf98b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0bb80d31e0 0x7f0bb81558b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:21.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:21.543+0000 7f0bbf98b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0bb815ecf0 0x7f0bb815f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:21.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:21.543+0000 7f0bbf98b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0bb81627b0 0x7f0bb815f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:21.547 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:21.543+0000 7f0bbf98b640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f0bb80b5530 con 0x7f0bb80d31e0 2024-09-21T03:34:21.547 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:21.543+0000 7f0bbf98b640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f0bb80b53b0 con 0x7f0bb815ecf0 2024-09-21T03:34:21.547 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:21.543+0000 7f0bbf98b640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f0bb80b56b0 con 0x7f0bb81627b0 2024-09-21T03:34:21.547 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:21.543+0000 7f0bbd727640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0bb80d31e0 0x7f0bb81558b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:21.547 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:21.543+0000 7f0bbd727640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0bb80d31e0 0x7f0bb81558b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54482/0 (socket says 172.21.15.38:54482) 2024-09-21T03:34:21.547 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:21.543+0000 7f0bbd727640 1 -- 172.21.15.38:0/3469067850 learned_addr learned my addr 172.21.15.38:0/3469067850 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:34:21.547 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:21.544+0000 7f0bbcf26640 1 --2- 172.21.15.38:0/3469067850 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0bb815ecf0 0x7f0bb815f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:21.547 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:21.544+0000 7f0bbdf28640 1 --2- 172.21.15.38:0/3469067850 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0bb81627b0 0x7f0bb815f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:21.547 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:21.544+0000 7f0bbd727640 1 -- 172.21.15.38:0/3469067850 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0bb81627b0 msgr2=0x7f0bb815f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:21.547 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:21.544+0000 7f0bbd727640 1 --2- 172.21.15.38:0/3469067850 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0bb81627b0 0x7f0bb815f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:21.547 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:21.544+0000 7f0bbd727640 1 -- 172.21.15.38:0/3469067850 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0bb815ecf0 msgr2=0x7f0bb815f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:21.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:21.544+0000 7f0bbd727640 1 --2- 172.21.15.38:0/3469067850 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0bb815ecf0 0x7f0bb815f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:21.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:21.544+0000 7f0bbd727640 1 -- 172.21.15.38:0/3469067850 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bb80c8250 con 0x7f0bb80d31e0 2024-09-21T03:34:21.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:21.544+0000 7f0bbcf26640 1 --2- 172.21.15.38:0/3469067850 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0bb815ecf0 0x7f0bb815f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:34:21.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:21.544+0000 7f0bbdf28640 1 --2- 172.21.15.38:0/3469067850 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0bb81627b0 0x7f0bb815f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:34:21.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:21.544+0000 7f0bbd727640 1 --2- 172.21.15.38:0/3469067850 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0bb80d31e0 0x7f0bb81558b0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f0ba800ea20 tx=0x7f0ba800eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:21.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:21.544+0000 7f0bb5ffb640 1 -- 172.21.15.38:0/3469067850 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ba800cde0 con 0x7f0bb80d31e0 2024-09-21T03:34:21.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:21.544+0000 7f0bb5ffb640 1 -- 172.21.15.38:0/3469067850 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0ba8011560 con 0x7f0bb80d31e0 2024-09-21T03:34:21.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:21.544+0000 7f0bb5ffb640 1 -- 172.21.15.38:0/3469067850 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ba8011990 con 0x7f0bb80d31e0 2024-09-21T03:34:21.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:21.544+0000 7f0bbf98b640 1 -- 172.21.15.38:0/3469067850 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bb80c8530 con 0x7f0bb80d31e0 2024-09-21T03:34:21.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:21.544+0000 7f0bbf98b640 1 -- 172.21.15.38:0/3469067850 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bb80c8af0 con 0x7f0bb80d31e0 2024-09-21T03:34:21.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:21.545+0000 7f0bbf98b640 1 -- 172.21.15.38:0/3469067850 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0bb80d29a0 con 0x7f0bb80d31e0 2024-09-21T03:34:21.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:21.546+0000 7f0bb5ffb640 1 -- 172.21.15.38:0/3469067850 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f0ba8011b30 con 0x7f0bb80d31e0 2024-09-21T03:34:21.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:21.547+0000 7f0bb5ffb640 1 --2- 172.21.15.38:0/3469067850 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0ba4078bc0 0x7f0ba407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:21.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:21.547+0000 7f0bb5ffb640 1 -- 172.21.15.38:0/3469067850 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6619+0+0 (secure 0 0 0) 0x7f0ba809bd10 con 0x7f0bb80d31e0 2024-09-21T03:34:21.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:21.547+0000 7f0bbcf26640 1 --2- 172.21.15.38:0/3469067850 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0ba4078bc0 0x7f0ba407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:21.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:21.547+0000 7f0bbcf26640 1 --2- 172.21.15.38:0/3469067850 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0ba4078bc0 0x7f0ba407b0d0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f0bac034000 tx=0x7f0bac004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:21.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:21.551+0000 7f0bb5ffb640 1 -- 172.21.15.38:0/3469067850 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f0bb80d29a0 con 0x7f0bb80d31e0 2024-09-21T03:34:21.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:21.740+0000 7f0bbf98b640 1 -- 172.21.15.38:0/3469067850 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0bb80cf480 con 0x7f0bb80d31e0 2024-09-21T03:34:21.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:21.741+0000 7f0bb5ffb640 1 -- 172.21.15.38:0/3469067850 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) ==== 74+0+18768 (secure 0 0 0) 0x7f0ba80a1020 con 0x7f0bb80d31e0 2024-09-21T03:34:21.743 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:34:21.743 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":28,"fsid":"9d50d66f-656b-4fe2-8e3b-fe53b79c4d43","created":"2024-09-21T03:33:10.677865+0000","modified":"2024-09-21T03:34:16.401838+0000","last_up_change":"2024-09-21T03:33:24.023421+0000","last_in_change":"2024-09-21T03:33:14.296193+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T03:33:22.992573+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T03:33:33.530201+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T03:33:34.585875+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T03:33:38.268346+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"dbab183f-7e63-48b8-960a-ae55fd5bf565","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6801","nonce":2381639433}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6803","nonce":2381639433}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6807","nonce":2381639433}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6805","nonce":2381639433}]},"public_addr":"172.21.15.38:6801/2381639433","cluster_addr":"172.21.15.38:6803/2381639433","heartbeat_back_addr":"172.21.15.38:6807/2381639433","heartbeat_front_addr":"172.21.15.38:6805/2381639433","state":["exists","up"]},{"osd":1,"uuid":"fa1c0e8a-ebc1-4359-8146-bd5342abbc16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6817","nonce":1491818732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6819","nonce":1491818732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6823","nonce":1491818732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6821","nonce":1491818732}]},"public_addr":"172.21.15.38:6817/1491818732","cluster_addr":"172.21.15.38:6819/1491818732","heartbeat_back_addr":"172.21.15.38:6823/1491818732","heartbeat_front_addr":"172.21.15.38:6821/1491818732","state":["exists","up"]},{"osd":2,"uuid":"2ed3cd2d-385d-4fc6-9869-214552a84803","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6825","nonce":2577858902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6827","nonce":2577858902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6831","nonce":2577858902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6829","nonce":2577858902}]},"public_addr":"172.21.15.38:6825/2577858902","cluster_addr":"172.21.15.38:6827/2577858902","heartbeat_back_addr":"172.21.15.38:6831/2577858902","heartbeat_front_addr":"172.21.15.38:6829/2577858902","state":["exists","up"]},{"osd":3,"uuid":"62d1d95e-919d-4250-be71-1d34cfa70dbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6809","nonce":1286810294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6811","nonce":1286810294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6815","nonce":1286810294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6813","nonce":1286810294}]},"public_addr":"172.21.15.38:6809/1286810294","cluster_addr":"172.21.15.38:6811/1286810294","heartbeat_back_addr":"172.21.15.38:6815/1286810294","heartbeat_front_addr":"172.21.15.38:6813/1286810294","state":["exists","up"]},{"osd":4,"uuid":"7c9f9314-99d7-4d73-8392-37809a9f54b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6803","nonce":3401837571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6805","nonce":3401837571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6809","nonce":3401837571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6807","nonce":3401837571}]},"public_addr":"172.21.15.195:6803/3401837571","cluster_addr":"172.21.15.195:6805/3401837571","heartbeat_back_addr":"172.21.15.195:6809/3401837571","heartbeat_front_addr":"172.21.15.195:6807/3401837571","state":["exists","up"]},{"osd":5,"uuid":"bde4fe9f-8414-4bba-bf4a-43973d1976bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6811","nonce":2080960180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6813","nonce":2080960180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6817","nonce":2080960180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6815","nonce":2080960180}]},"public_addr":"172.21.15.195:6811/2080960180","cluster_addr":"172.21.15.195:6813/2080960180","heartbeat_back_addr":"172.21.15.195:6817/2080960180","heartbeat_front_addr":"172.21.15.195:6815/2080960180","state":["exists","up"]},{"osd":6,"uuid":"6ee6d7f4-1a5f-4a4c-ac16-48fe05f6a002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6827","nonce":4055825675}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6829","nonce":4055825675}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6833","nonce":4055825675}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6831","nonce":4055825675}]},"public_addr":"172.21.15.195:6827/4055825675","cluster_addr":"172.21.15.195:6829/4055825675","heartbeat_back_addr":"172.21.15.195:6833/4055825675","heartbeat_front_addr":"172.21.15.195:6831/4055825675","state":["exists","up"]},{"osd":7,"uuid":"efeb5f51-329a-4484-8290-92e7d3503d05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6819","nonce":1606106342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6821","nonce":1606106342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6825","nonce":1606106342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6823","nonce":1606106342}]},"public_addr":"172.21.15.195:6819/1606106342","cluster_addr":"172.21.15.195:6821/1606106342","heartbeat_back_addr":"172.21.15.195:6825/1606106342","heartbeat_front_addr":"172.21.15.195:6823/1606106342","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.901343+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:21.115003+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.901623+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.835238+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.132199+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:19.817560+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.829690+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:21.087300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.34","mappings":[{"from":0,"to":7}]},{"pgid":"4.a","mappings":[{"from":3,"to":0}]},{"pgid":"4.1d","mappings":[{"from":7,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T03:34:21.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:21.743+0000 7f0bbf98b640 1 -- 172.21.15.38:0/3469067850 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0ba4078bc0 msgr2=0x7f0ba407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:21.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:21.743+0000 7f0bbf98b640 1 --2- 172.21.15.38:0/3469067850 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0ba4078bc0 0x7f0ba407b0d0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f0bac034000 tx=0x7f0bac004730 comp rx=0 tx=0).stop 2024-09-21T03:34:21.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:21.743+0000 7f0bbf98b640 1 -- 172.21.15.38:0/3469067850 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0bb80d31e0 msgr2=0x7f0bb81558b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:21.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:21.743+0000 7f0bbf98b640 1 --2- 172.21.15.38:0/3469067850 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0bb80d31e0 0x7f0bb81558b0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f0ba800ea20 tx=0x7f0ba800eee0 comp rx=0 tx=0).stop 2024-09-21T03:34:21.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:21.743+0000 7f0bbf98b640 1 -- 172.21.15.38:0/3469067850 shutdown_connections 2024-09-21T03:34:21.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:21.743+0000 7f0bbf98b640 1 --2- 172.21.15.38:0/3469067850 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0ba4078bc0 0x7f0ba407b0d0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:21.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:21.743+0000 7f0bbf98b640 1 --2- 172.21.15.38:0/3469067850 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0bb81627b0 0x7f0bb815f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:21.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:21.743+0000 7f0bbf98b640 1 --2- 172.21.15.38:0/3469067850 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0bb815ecf0 0x7f0bb815f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:21.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:21.743+0000 7f0bbf98b640 1 --2- 172.21.15.38:0/3469067850 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0bb80d31e0 0x7f0bb81558b0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:21.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:21.743+0000 7f0bbf98b640 1 -- 172.21.15.38:0/3469067850 >> 172.21.15.38:0/3469067850 conn(0x7f0bb80d4600 msgr2=0x7f0bb80d35f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:21.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:21.743+0000 7f0bbf98b640 1 -- 172.21.15.38:0/3469067850 shutdown_connections 2024-09-21T03:34:21.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:21.743+0000 7f0bbf98b640 1 -- 172.21.15.38:0/3469067850 wait complete. 2024-09-21T03:34:21.757 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-21T03:34:21.757 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:34:21.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:21.871+0000 7f2323eb6640 1 -- 172.21.15.38:0/321794907 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f231c15cf20 msgr2=0x7f231c0b6960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:21.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:21.871+0000 7f2323eb6640 1 --2- 172.21.15.38:0/321794907 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f231c15cf20 0x7f231c0b6960 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f230c005fd0 tx=0x7f230c02a940 comp rx=0 tx=0).stop 2024-09-21T03:34:21.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:21.871+0000 7f2323eb6640 1 -- 172.21.15.38:0/321794907 shutdown_connections 2024-09-21T03:34:21.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:21.871+0000 7f2323eb6640 1 --2- 172.21.15.38:0/321794907 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f231c15cf20 0x7f231c0b6960 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:21.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:21.871+0000 7f2323eb6640 1 -- 172.21.15.38:0/321794907 >> 172.21.15.38:0/321794907 conn(0x7f231c0d2350 msgr2=0x7f231c0d2760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:21.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:21.871+0000 7f2323eb6640 1 -- 172.21.15.38:0/321794907 shutdown_connections 2024-09-21T03:34:21.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:21.871+0000 7f2323eb6640 1 -- 172.21.15.38:0/321794907 wait complete. 2024-09-21T03:34:21.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:21.873+0000 7f2323eb6640 1 Processor -- start 2024-09-21T03:34:21.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:21.873+0000 7f2323eb6640 1 -- start start 2024-09-21T03:34:21.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:21.874+0000 7f2323eb6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f231c0d19f0 0x7f231c1f4330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:21.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:21.874+0000 7f2323eb6640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f231c15cb10 0x7f231c14af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:21.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:21.874+0000 7f2323eb6640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f231c15cf20 0x7f231c1f6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:21.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:21.874+0000 7f2323eb6640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f231c0b7020 con 0x7f231c0d19f0 2024-09-21T03:34:21.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:21.874+0000 7f2323eb6640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f231c0b6ea0 con 0x7f231c15cf20 2024-09-21T03:34:21.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:21.874+0000 7f2323eb6640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f231c0b71a0 con 0x7f231c15cb10 2024-09-21T03:34:21.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:21.874+0000 7f2321c52640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f231c0d19f0 0x7f231c1f4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:21.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:21.874+0000 7f2321c52640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f231c0d19f0 0x7f231c1f4330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54510/0 (socket says 172.21.15.38:54510) 2024-09-21T03:34:21.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:21.874+0000 7f2321c52640 1 -- 172.21.15.38:0/1827130255 learned_addr learned my addr 172.21.15.38:0/1827130255 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:34:21.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:21.874+0000 7f2322453640 1 --2- 172.21.15.38:0/1827130255 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f231c15cf20 0x7f231c1f6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:21.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:21.874+0000 7f2321451640 1 --2- 172.21.15.38:0/1827130255 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f231c15cb10 0x7f231c14af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:21.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:21.875+0000 7f2321c52640 1 -- 172.21.15.38:0/1827130255 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f231c15cb10 msgr2=0x7f231c14af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:21.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:21.875+0000 7f2321c52640 1 --2- 172.21.15.38:0/1827130255 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f231c15cb10 0x7f231c14af00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:21.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:21.875+0000 7f2321c52640 1 -- 172.21.15.38:0/1827130255 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f231c15cf20 msgr2=0x7f231c1f6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:21.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:21.875+0000 7f2321c52640 1 --2- 172.21.15.38:0/1827130255 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f231c15cf20 0x7f231c1f6dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:21.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:21.875+0000 7f2321c52640 1 -- 172.21.15.38:0/1827130255 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f231c1f7940 con 0x7f231c0d19f0 2024-09-21T03:34:21.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:21.875+0000 7f2321c52640 1 --2- 172.21.15.38:0/1827130255 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f231c0d19f0 0x7f231c1f4330 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f2314001eb0 tx=0x7f231400c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:21.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:21.875+0000 7f231a7fc640 1 -- 172.21.15.38:0/1827130255 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2314011790 con 0x7f231c0d19f0 2024-09-21T03:34:21.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:21.875+0000 7f231a7fc640 1 -- 172.21.15.38:0/1827130255 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2314017070 con 0x7f231c0d19f0 2024-09-21T03:34:21.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:21.875+0000 7f2323eb6640 1 -- 172.21.15.38:0/1827130255 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f231c1f7c20 con 0x7f231c0d19f0 2024-09-21T03:34:21.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:21.875+0000 7f231a7fc640 1 -- 172.21.15.38:0/1827130255 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2314011ca0 con 0x7f231c0d19f0 2024-09-21T03:34:21.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:21.875+0000 7f2323eb6640 1 -- 172.21.15.38:0/1827130255 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f231c14bbc0 con 0x7f231c0d19f0 2024-09-21T03:34:21.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:21.876+0000 7f2323eb6640 1 -- 172.21.15.38:0/1827130255 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f231c0d12e0 con 0x7f231c0d19f0 2024-09-21T03:34:21.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:21.877+0000 7f231a7fc640 1 -- 172.21.15.38:0/1827130255 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f231401c460 con 0x7f231c0d19f0 2024-09-21T03:34:21.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:21.877+0000 7f231a7fc640 1 --2- 172.21.15.38:0/1827130255 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2308078bf0 0x7f230807b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:21.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:21.878+0000 7f231a7fc640 1 -- 172.21.15.38:0/1827130255 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6619+0+0 (secure 0 0 0) 0x7f231409aca0 con 0x7f231c0d19f0 2024-09-21T03:34:21.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:21.878+0000 7f2321451640 1 --2- 172.21.15.38:0/1827130255 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2308078bf0 0x7f230807b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:21.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:21.878+0000 7f2321451640 1 --2- 172.21.15.38:0/1827130255 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2308078bf0 0x7f230807b100 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f231c0d19c0 tx=0x7f2310009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:21.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:21.882+0000 7f231a7fc640 1 -- 172.21.15.38:0/1827130255 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f2314062790 con 0x7f231c0d19f0 2024-09-21T03:34:22.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:22.109+0000 7f2323eb6640 1 -- 172.21.15.38:0/1827130255 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f231c0b56b0 con 0x7f231c0d19f0 2024-09-21T03:34:22.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:22.109+0000 7f231a7fc640 1 -- 172.21.15.38:0/1827130255 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5219 (secure 0 0 0) 0x7f23140671c0 con 0x7f231c0d19f0 2024-09-21T03:34:22.111 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:34:22.111 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":9,"btime":"2024-09-21T03:33:56:329606+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4322,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/593838590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":593838590},{"type":"v1","addr":"172.21.15.195:6836","nonce":593838590}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4374,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/663402481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":663402481},{"type":"v1","addr":"172.21.15.38:6833","nonce":663402481}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4377,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/2614829035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":2614829035},{"type":"v1","addr":"172.21.15.38:6835","nonce":2614829035}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:33:35.622623+0000","modified":"2024-09-21T03:33:55.704494+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4402},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4402":{"gid":4402,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.195:6837/1420805977","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":1420805977},{"type":"v1","addr":"172.21.15.195:6837","nonce":1420805977}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4402,"qdb_cluster":[4402]},"id":1}]} 2024-09-21T03:34:22.112 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 9 2024-09-21T03:34:22.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:22.111+0000 7f2323eb6640 1 -- 172.21.15.38:0/1827130255 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2308078bf0 msgr2=0x7f230807b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:22.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:22.111+0000 7f2323eb6640 1 --2- 172.21.15.38:0/1827130255 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2308078bf0 0x7f230807b100 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f231c0d19c0 tx=0x7f2310009040 comp rx=0 tx=0).stop 2024-09-21T03:34:22.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:22.111+0000 7f2323eb6640 1 -- 172.21.15.38:0/1827130255 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f231c0d19f0 msgr2=0x7f231c1f4330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:22.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:22.111+0000 7f2323eb6640 1 --2- 172.21.15.38:0/1827130255 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f231c0d19f0 0x7f231c1f4330 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f2314001eb0 tx=0x7f231400c890 comp rx=0 tx=0).stop 2024-09-21T03:34:22.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:22.111+0000 7f2323eb6640 1 -- 172.21.15.38:0/1827130255 shutdown_connections 2024-09-21T03:34:22.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:22.111+0000 7f2323eb6640 1 --2- 172.21.15.38:0/1827130255 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f231c15cf20 0x7f231c1f6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:22.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:22.112+0000 7f2323eb6640 1 --2- 172.21.15.38:0/1827130255 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2308078bf0 0x7f230807b100 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:22.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:22.112+0000 7f2323eb6640 1 --2- 172.21.15.38:0/1827130255 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f231c15cb10 0x7f231c14af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:22.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:22.112+0000 7f2323eb6640 1 --2- 172.21.15.38:0/1827130255 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f231c0d19f0 0x7f231c1f4330 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:22.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:22.112+0000 7f2323eb6640 1 -- 172.21.15.38:0/1827130255 >> 172.21.15.38:0/1827130255 conn(0x7f231c0d2350 msgr2=0x7f231c160070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:22.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:22.112+0000 7f2323eb6640 1 -- 172.21.15.38:0/1827130255 shutdown_connections 2024-09-21T03:34:22.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:22.112+0000 7f2323eb6640 1 -- 172.21.15.38:0/1827130255 wait complete. 2024-09-21T03:34:22.125 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:34:22.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:22.240+0000 7f8a068bd640 1 -- 172.21.15.38:0/2796248724 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8a000cea90 msgr2=0x7f8a000d00d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:22.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:22.240+0000 7f8a068bd640 1 --2- 172.21.15.38:0/2796248724 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8a000cea90 0x7f8a000d00d0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f89f00098e0 tx=0x7f89f002a530 comp rx=0 tx=0).stop 2024-09-21T03:34:22.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:22.240+0000 7f8a068bd640 1 -- 172.21.15.38:0/2796248724 shutdown_connections 2024-09-21T03:34:22.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:22.240+0000 7f8a068bd640 1 --2- 172.21.15.38:0/2796248724 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8a000d3360 0x7f8a000d3770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:22.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:22.240+0000 7f8a068bd640 1 --2- 172.21.15.38:0/2796248724 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8a000cea90 0x7f8a000d00d0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:22.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:22.240+0000 7f8a068bd640 1 -- 172.21.15.38:0/2796248724 >> 172.21.15.38:0/2796248724 conn(0x7f8a000d4620 msgr2=0x7f8a000d4a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:22.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:22.240+0000 7f8a068bd640 1 -- 172.21.15.38:0/2796248724 shutdown_connections 2024-09-21T03:34:22.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:22.240+0000 7f8a068bd640 1 -- 172.21.15.38:0/2796248724 wait complete. 2024-09-21T03:34:22.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:22.241+0000 7f8a068bd640 1 Processor -- start 2024-09-21T03:34:22.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:22.242+0000 7f8a068bd640 1 -- start start 2024-09-21T03:34:22.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:22.243+0000 7f8a068bd640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8a000cea90 0x7f8a001f9840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:22.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:22.243+0000 7f8a068bd640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8a000d3360 0x7f8a00158290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:22.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:22.243+0000 7f8a068bd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8a00162790 0x7f8a001f2880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:22.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:22.243+0000 7f8a068bd640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f8a000b4180 con 0x7f8a00162790 2024-09-21T03:34:22.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:22.243+0000 7f8a068bd640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f8a000b4000 con 0x7f8a000d3360 2024-09-21T03:34:22.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:22.243+0000 7f8a068bd640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f8a000b4300 con 0x7f8a000cea90 2024-09-21T03:34:22.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:22.243+0000 7f8a04e5a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8a00162790 0x7f8a001f2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:22.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:22.243+0000 7f8a04e5a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8a00162790 0x7f8a001f2880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54528/0 (socket says 172.21.15.38:54528) 2024-09-21T03:34:22.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:22.243+0000 7f8a04e5a640 1 -- 172.21.15.38:0/3177024502 learned_addr learned my addr 172.21.15.38:0/3177024502 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:34:22.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:22.243+0000 7f89fffff640 1 --2- 172.21.15.38:0/3177024502 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8a000cea90 0x7f8a001f9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:22.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:22.243+0000 7f89ff7fe640 1 --2- 172.21.15.38:0/3177024502 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8a000d3360 0x7f8a00158290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:22.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:22.244+0000 7f8a04e5a640 1 -- 172.21.15.38:0/3177024502 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8a000cea90 msgr2=0x7f8a001f9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:22.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:22.244+0000 7f8a04e5a640 1 --2- 172.21.15.38:0/3177024502 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8a000cea90 0x7f8a001f9840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:22.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:22.244+0000 7f8a04e5a640 1 -- 172.21.15.38:0/3177024502 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8a000d3360 msgr2=0x7f8a00158290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:22.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:22.244+0000 7f8a04e5a640 1 --2- 172.21.15.38:0/3177024502 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8a000d3360 0x7f8a00158290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:22.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:22.244+0000 7f8a04e5a640 1 -- 172.21.15.38:0/3177024502 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a001f5c90 con 0x7f8a00162790 2024-09-21T03:34:22.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:22.244+0000 7f89fffff640 1 --2- 172.21.15.38:0/3177024502 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8a000cea90 0x7f8a001f9840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:34:22.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:22.244+0000 7f89ff7fe640 1 --2- 172.21.15.38:0/3177024502 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8a000d3360 0x7f8a00158290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:34:22.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:22.244+0000 7f8a04e5a640 1 --2- 172.21.15.38:0/3177024502 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8a00162790 0x7f8a001f2880 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f89f800d9e0 tx=0x7f89f800dea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:22.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:22.244+0000 7f89fcff9640 1 -- 172.21.15.38:0/3177024502 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89f8017070 con 0x7f8a00162790 2024-09-21T03:34:22.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:22.244+0000 7f89fcff9640 1 -- 172.21.15.38:0/3177024502 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f89f800be30 con 0x7f8a00162790 2024-09-21T03:34:22.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:22.244+0000 7f8a068bd640 1 -- 172.21.15.38:0/3177024502 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a001f5f70 con 0x7f8a00162790 2024-09-21T03:34:22.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:22.244+0000 7f89fcff9640 1 -- 172.21.15.38:0/3177024502 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89f8003bb0 con 0x7f8a00162790 2024-09-21T03:34:22.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:22.244+0000 7f8a068bd640 1 -- 172.21.15.38:0/3177024502 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a001f6530 con 0x7f8a00162790 2024-09-21T03:34:22.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:22.245+0000 7f8a068bd640 1 -- 172.21.15.38:0/3177024502 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a000b47b0 con 0x7f8a00162790 2024-09-21T03:34:22.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:22.246+0000 7f89fcff9640 1 -- 172.21.15.38:0/3177024502 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f89f8010460 con 0x7f8a00162790 2024-09-21T03:34:22.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:22.246+0000 7f89fcff9640 1 --2- 172.21.15.38:0/3177024502 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f89ec078b90 0x7f89ec07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:22.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:22.246+0000 7f89fcff9640 1 -- 172.21.15.38:0/3177024502 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6619+0+0 (secure 0 0 0) 0x7f89f809b510 con 0x7f8a00162790 2024-09-21T03:34:22.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:22.247+0000 7f89fffff640 1 --2- 172.21.15.38:0/3177024502 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f89ec078b90 0x7f89ec07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:22.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:22.247+0000 7f89fffff640 1 --2- 172.21.15.38:0/3177024502 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f89ec078b90 0x7f89ec07b0a0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f89f0034000 tx=0x7f89f0004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:22.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:22.251+0000 7f89fcff9640 1 -- 172.21.15.38:0/3177024502 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f89f80635c0 con 0x7f8a00162790 2024-09-21T03:34:22.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:22.477+0000 7f8a068bd640 1 -- 172.21.15.38:0/3177024502 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8a001f5a20 con 0x7f8a00162790 2024-09-21T03:34:22.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:22.477+0000 7f89fcff9640 1 -- 172.21.15.38:0/3177024502 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5219 (secure 0 0 0) 0x7f89f8062ec0 con 0x7f8a00162790 2024-09-21T03:34:22.479 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:34:22.479 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":9,"btime":"2024-09-21T03:33:56:329606+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4322,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/593838590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":593838590},{"type":"v1","addr":"172.21.15.195:6836","nonce":593838590}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4374,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/663402481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":663402481},{"type":"v1","addr":"172.21.15.38:6833","nonce":663402481}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4377,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/2614829035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":2614829035},{"type":"v1","addr":"172.21.15.38:6835","nonce":2614829035}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:33:35.622623+0000","modified":"2024-09-21T03:33:55.704494+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4402},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4402":{"gid":4402,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.195:6837/1420805977","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":1420805977},{"type":"v1","addr":"172.21.15.195:6837","nonce":1420805977}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4402,"qdb_cluster":[4402]},"id":1}]} 2024-09-21T03:34:22.479 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 9 2024-09-21T03:34:22.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:22.479+0000 7f8a068bd640 1 -- 172.21.15.38:0/3177024502 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f89ec078b90 msgr2=0x7f89ec07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:22.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:22.479+0000 7f8a068bd640 1 --2- 172.21.15.38:0/3177024502 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f89ec078b90 0x7f89ec07b0a0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f89f0034000 tx=0x7f89f0004730 comp rx=0 tx=0).stop 2024-09-21T03:34:22.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:22.479+0000 7f8a068bd640 1 -- 172.21.15.38:0/3177024502 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8a00162790 msgr2=0x7f8a001f2880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:22.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:22.479+0000 7f8a068bd640 1 --2- 172.21.15.38:0/3177024502 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8a00162790 0x7f8a001f2880 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f89f800d9e0 tx=0x7f89f800dea0 comp rx=0 tx=0).stop 2024-09-21T03:34:22.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:22.479+0000 7f8a068bd640 1 -- 172.21.15.38:0/3177024502 shutdown_connections 2024-09-21T03:34:22.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:22.479+0000 7f8a068bd640 1 --2- 172.21.15.38:0/3177024502 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8a00162790 0x7f8a001f2880 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:22.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:22.479+0000 7f8a068bd640 1 --2- 172.21.15.38:0/3177024502 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8a000d3360 0x7f8a00158290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:22.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:22.479+0000 7f8a068bd640 1 --2- 172.21.15.38:0/3177024502 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f89ec078b90 0x7f89ec07b0a0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:22.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:22.479+0000 7f8a068bd640 1 --2- 172.21.15.38:0/3177024502 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8a000cea90 0x7f8a001f9840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:22.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:22.479+0000 7f8a068bd640 1 -- 172.21.15.38:0/3177024502 >> 172.21.15.38:0/3177024502 conn(0x7f8a000d4620 msgr2=0x7f8a000d0c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:22.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:22.480+0000 7f8a068bd640 1 -- 172.21.15.38:0/3177024502 shutdown_connections 2024-09-21T03:34:22.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:22.480+0000 7f8a068bd640 1 -- 172.21.15.38:0/3177024502 wait complete. 2024-09-21T03:34:22.493 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T03:34:22.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:22.608+0000 7fbe0bb71640 1 -- 172.21.15.38:0/4097502004 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbdec004d40 msgr2=0x7fbdec005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:22.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:22.608+0000 7fbe0bb71640 1 --2- 172.21.15.38:0/4097502004 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbdec004d40 0x7fbdec005150 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fbdfc00c470 tx=0x7fbdec005830 comp rx=0 tx=0).stop 2024-09-21T03:34:22.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:22.608+0000 7fbe0a10e640 1 -- 172.21.15.38:0/4097502004 reap_dead start 2024-09-21T03:34:22.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:22.608+0000 7fbe0bb71640 1 -- 172.21.15.38:0/4097502004 shutdown_connections 2024-09-21T03:34:22.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:22.608+0000 7fbe0bb71640 1 -- 172.21.15.38:0/4097502004 >> 172.21.15.38:0/4097502004 conn(0x7fbe040d4440 msgr2=0x7fbe040d4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:22.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:22.608+0000 7fbe0bb71640 1 -- 172.21.15.38:0/4097502004 shutdown_connections 2024-09-21T03:34:22.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:22.608+0000 7fbe0bb71640 1 -- 172.21.15.38:0/4097502004 wait complete. 2024-09-21T03:34:22.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:22.610+0000 7fbe0bb71640 1 Processor -- start 2024-09-21T03:34:22.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:22.610+0000 7fbe0bb71640 1 -- start start 2024-09-21T03:34:22.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:22.611+0000 7fbe0bb71640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbe041f5110 0x7fbe04151fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:22.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:22.611+0000 7fbe0bb71640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbe04152510 0x7fbe041f8750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:22.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:22.611+0000 7fbe0bb71640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbe041f8c90 0x7fbe041e9400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:22.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:22.611+0000 7fbe0bb71640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fbe040d2a20 con 0x7fbe04152510 2024-09-21T03:34:22.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:22.611+0000 7fbe0bb71640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fbe040b3b90 con 0x7fbe041f8c90 2024-09-21T03:34:22.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:22.611+0000 7fbe0bb71640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fbe040d2ba0 con 0x7fbe041f5110 2024-09-21T03:34:22.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:22.611+0000 7fbe0910c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbe04152510 0x7fbe041f8750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:22.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:22.611+0000 7fbe0910c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbe04152510 0x7fbe041f8750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:52644/0 (socket says 172.21.15.38:52644) 2024-09-21T03:34:22.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:22.611+0000 7fbe0910c640 1 -- 172.21.15.38:0/2365383544 learned_addr learned my addr 172.21.15.38:0/2365383544 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:34:22.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:22.611+0000 7fbe0990d640 1 --2- 172.21.15.38:0/2365383544 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbe041f5110 0x7fbe04151fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:22.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:22.611+0000 7fbe0a10e640 1 --2- 172.21.15.38:0/2365383544 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbe041f8c90 0x7fbe041e9400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:22.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:22.611+0000 7fbe0910c640 1 -- 172.21.15.38:0/2365383544 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbe041f5110 msgr2=0x7fbe04151fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:22.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:22.611+0000 7fbe0910c640 1 --2- 172.21.15.38:0/2365383544 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbe041f5110 0x7fbe04151fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:22.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:22.611+0000 7fbe0910c640 1 -- 172.21.15.38:0/2365383544 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbe041f8c90 msgr2=0x7fbe041e9400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:22.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:22.611+0000 7fbe0910c640 1 --2- 172.21.15.38:0/2365383544 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbe041f8c90 0x7fbe041e9400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:22.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:22.611+0000 7fbe0910c640 1 -- 172.21.15.38:0/2365383544 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe041f90a0 con 0x7fbe04152510 2024-09-21T03:34:22.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:22.611+0000 7fbe0990d640 1 --2- 172.21.15.38:0/2365383544 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbe041f5110 0x7fbe04151fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:34:22.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:22.612+0000 7fbe0910c640 1 --2- 172.21.15.38:0/2365383544 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbe04152510 0x7fbe041f8750 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fbdf8011a30 tx=0x7fbdf8011ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:22.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:22.612+0000 7fbe027fc640 1 -- 172.21.15.38:0/2365383544 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdf801f070 con 0x7fbe04152510 2024-09-21T03:34:22.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:22.612+0000 7fbe027fc640 1 -- 172.21.15.38:0/2365383544 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbdf8012450 con 0x7fbe04152510 2024-09-21T03:34:22.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:22.612+0000 7fbe0bb71640 1 -- 172.21.15.38:0/2365383544 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe041f9580 con 0x7fbe04152510 2024-09-21T03:34:22.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:22.612+0000 7fbe027fc640 1 -- 172.21.15.38:0/2365383544 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdf8012840 con 0x7fbe04152510 2024-09-21T03:34:22.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:22.612+0000 7fbe0bb71640 1 -- 172.21.15.38:0/2365383544 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe041e8920 con 0x7fbe04152510 2024-09-21T03:34:22.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:22.613+0000 7fbe0bb71640 1 -- 172.21.15.38:0/2365383544 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe040d30a0 con 0x7fbe04152510 2024-09-21T03:34:22.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:22.614+0000 7fbe027fc640 1 -- 172.21.15.38:0/2365383544 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fbdf8022070 con 0x7fbe04152510 2024-09-21T03:34:22.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:22.614+0000 7fbe027fc640 1 --2- 172.21.15.38:0/2365383544 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fbdf0078b90 0x7fbdf007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:22.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:22.614+0000 7fbe027fc640 1 -- 172.21.15.38:0/2365383544 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6619+0+0 (secure 0 0 0) 0x7fbdf809bc50 con 0x7fbe04152510 2024-09-21T03:34:22.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:22.614+0000 7fbe0990d640 1 --2- 172.21.15.38:0/2365383544 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fbdf0078b90 0x7fbdf007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:22.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:22.615+0000 7fbe0990d640 1 --2- 172.21.15.38:0/2365383544 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fbdf0078b90 0x7fbdf007b0a0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fbe0415c9c0 tx=0x7fbdf4009500 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:22.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:22.619+0000 7fbe027fc640 1 -- 172.21.15.38:0/2365383544 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fbdf80a1020 con 0x7fbe04152510 2024-09-21T03:34:22.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:22.806+0000 7fbe0bb71640 1 -- 172.21.15.38:0/2365383544 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbe0406f4f0 con 0x7fbe04152510 2024-09-21T03:34:22.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:22.807+0000 7fbe027fc640 1 -- 172.21.15.38:0/2365383544 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) ==== 74+0+18768 (secure 0 0 0) 0x7fbe041f9580 con 0x7fbe04152510 2024-09-21T03:34:22.809 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:34:22.809 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":28,"fsid":"9d50d66f-656b-4fe2-8e3b-fe53b79c4d43","created":"2024-09-21T03:33:10.677865+0000","modified":"2024-09-21T03:34:16.401838+0000","last_up_change":"2024-09-21T03:33:24.023421+0000","last_in_change":"2024-09-21T03:33:14.296193+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T03:33:22.992573+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T03:33:33.530201+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T03:33:34.585875+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T03:33:38.268346+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"dbab183f-7e63-48b8-960a-ae55fd5bf565","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6801","nonce":2381639433}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6803","nonce":2381639433}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6807","nonce":2381639433}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6805","nonce":2381639433}]},"public_addr":"172.21.15.38:6801/2381639433","cluster_addr":"172.21.15.38:6803/2381639433","heartbeat_back_addr":"172.21.15.38:6807/2381639433","heartbeat_front_addr":"172.21.15.38:6805/2381639433","state":["exists","up"]},{"osd":1,"uuid":"fa1c0e8a-ebc1-4359-8146-bd5342abbc16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6817","nonce":1491818732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6819","nonce":1491818732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6823","nonce":1491818732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6821","nonce":1491818732}]},"public_addr":"172.21.15.38:6817/1491818732","cluster_addr":"172.21.15.38:6819/1491818732","heartbeat_back_addr":"172.21.15.38:6823/1491818732","heartbeat_front_addr":"172.21.15.38:6821/1491818732","state":["exists","up"]},{"osd":2,"uuid":"2ed3cd2d-385d-4fc6-9869-214552a84803","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6825","nonce":2577858902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6827","nonce":2577858902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6831","nonce":2577858902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6829","nonce":2577858902}]},"public_addr":"172.21.15.38:6825/2577858902","cluster_addr":"172.21.15.38:6827/2577858902","heartbeat_back_addr":"172.21.15.38:6831/2577858902","heartbeat_front_addr":"172.21.15.38:6829/2577858902","state":["exists","up"]},{"osd":3,"uuid":"62d1d95e-919d-4250-be71-1d34cfa70dbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6809","nonce":1286810294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6811","nonce":1286810294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6815","nonce":1286810294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6813","nonce":1286810294}]},"public_addr":"172.21.15.38:6809/1286810294","cluster_addr":"172.21.15.38:6811/1286810294","heartbeat_back_addr":"172.21.15.38:6815/1286810294","heartbeat_front_addr":"172.21.15.38:6813/1286810294","state":["exists","up"]},{"osd":4,"uuid":"7c9f9314-99d7-4d73-8392-37809a9f54b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6803","nonce":3401837571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6805","nonce":3401837571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6809","nonce":3401837571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6807","nonce":3401837571}]},"public_addr":"172.21.15.195:6803/3401837571","cluster_addr":"172.21.15.195:6805/3401837571","heartbeat_back_addr":"172.21.15.195:6809/3401837571","heartbeat_front_addr":"172.21.15.195:6807/3401837571","state":["exists","up"]},{"osd":5,"uuid":"bde4fe9f-8414-4bba-bf4a-43973d1976bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6811","nonce":2080960180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6813","nonce":2080960180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6817","nonce":2080960180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6815","nonce":2080960180}]},"public_addr":"172.21.15.195:6811/2080960180","cluster_addr":"172.21.15.195:6813/2080960180","heartbeat_back_addr":"172.21.15.195:6817/2080960180","heartbeat_front_addr":"172.21.15.195:6815/2080960180","state":["exists","up"]},{"osd":6,"uuid":"6ee6d7f4-1a5f-4a4c-ac16-48fe05f6a002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6827","nonce":4055825675}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6829","nonce":4055825675}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6833","nonce":4055825675}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6831","nonce":4055825675}]},"public_addr":"172.21.15.195:6827/4055825675","cluster_addr":"172.21.15.195:6829/4055825675","heartbeat_back_addr":"172.21.15.195:6833/4055825675","heartbeat_front_addr":"172.21.15.195:6831/4055825675","state":["exists","up"]},{"osd":7,"uuid":"efeb5f51-329a-4484-8290-92e7d3503d05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6819","nonce":1606106342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6821","nonce":1606106342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6825","nonce":1606106342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6823","nonce":1606106342}]},"public_addr":"172.21.15.195:6819/1606106342","cluster_addr":"172.21.15.195:6821/1606106342","heartbeat_back_addr":"172.21.15.195:6825/1606106342","heartbeat_front_addr":"172.21.15.195:6823/1606106342","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.901343+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:21.115003+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.901623+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.835238+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.132199+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:19.817560+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.829690+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:21.087300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.34","mappings":[{"from":0,"to":7}]},{"pgid":"4.a","mappings":[{"from":3,"to":0}]},{"pgid":"4.1d","mappings":[{"from":7,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T03:34:22.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:22.809+0000 7fbe0bb71640 1 -- 172.21.15.38:0/2365383544 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fbdf0078b90 msgr2=0x7fbdf007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:22.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:22.809+0000 7fbe0bb71640 1 --2- 172.21.15.38:0/2365383544 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fbdf0078b90 0x7fbdf007b0a0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fbe0415c9c0 tx=0x7fbdf4009500 comp rx=0 tx=0).stop 2024-09-21T03:34:22.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:22.809+0000 7fbe0bb71640 1 -- 172.21.15.38:0/2365383544 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbe04152510 msgr2=0x7fbe041f8750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:22.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:22.809+0000 7fbe0bb71640 1 --2- 172.21.15.38:0/2365383544 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbe04152510 0x7fbe041f8750 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fbdf8011a30 tx=0x7fbdf8011ef0 comp rx=0 tx=0).stop 2024-09-21T03:34:22.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:22.809+0000 7fbe0bb71640 1 -- 172.21.15.38:0/2365383544 shutdown_connections 2024-09-21T03:34:22.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:22.809+0000 7fbe0bb71640 1 --2- 172.21.15.38:0/2365383544 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbe041f8c90 0x7fbe041e9400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:22.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:22.809+0000 7fbe0bb71640 1 --2- 172.21.15.38:0/2365383544 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbe04152510 0x7fbe041f8750 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:22.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:22.809+0000 7fbe0bb71640 1 --2- 172.21.15.38:0/2365383544 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fbdf0078b90 0x7fbdf007b0a0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:22.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:22.809+0000 7fbe0bb71640 1 --2- 172.21.15.38:0/2365383544 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbe041f5110 0x7fbe04151fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:22.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:22.809+0000 7fbe0bb71640 1 -- 172.21.15.38:0/2365383544 >> 172.21.15.38:0/2365383544 conn(0x7fbe040d4440 msgr2=0x7fbe040b3270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:22.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:22.809+0000 7fbe0bb71640 1 -- 172.21.15.38:0/2365383544 shutdown_connections 2024-09-21T03:34:22.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:22.809+0000 7fbe0bb71640 1 -- 172.21.15.38:0/2365383544 wait complete. 2024-09-21T03:34:22.823 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-21T03:34:22.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:22.939+0000 7f3ecdb20640 1 -- 172.21.15.38:0/10118992 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3eb0004800 msgr2=0x7f3eb0004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:22.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:22.939+0000 7f3ecdb20640 1 --2- 172.21.15.38:0/10118992 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3eb0004800 0x7f3eb0004c10 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f3ec80d28f0 tx=0x7f3eb0005830 comp rx=0 tx=0).stop 2024-09-21T03:34:22.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:22.939+0000 7f3ecdb20640 1 -- 172.21.15.38:0/10118992 shutdown_connections 2024-09-21T03:34:22.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:22.939+0000 7f3ecdb20640 1 --2- 172.21.15.38:0/10118992 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3eb0004800 0x7f3eb0004c10 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:22.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:22.939+0000 7f3ecdb20640 1 -- 172.21.15.38:0/10118992 >> 172.21.15.38:0/10118992 conn(0x7f3ec809d830 msgr2=0x7f3ec80b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:22.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:22.939+0000 7f3ecdb20640 1 -- 172.21.15.38:0/10118992 shutdown_connections 2024-09-21T03:34:22.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:22.939+0000 7f3ecdb20640 1 -- 172.21.15.38:0/10118992 wait complete. 2024-09-21T03:34:22.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:22.941+0000 7f3ecdb20640 1 Processor -- start 2024-09-21T03:34:22.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:22.941+0000 7f3ecdb20640 1 -- start start 2024-09-21T03:34:22.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:22.942+0000 7f3ecdb20640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3eb0004800 0x7f3ec806eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:22.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:22.942+0000 7f3ecdb20640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3ec80cbad0 0x7f3ec8157270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:22.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:22.942+0000 7f3ecdb20640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3ec80b2760 0x7f3ec80b2b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:22.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:22.942+0000 7f3ecdb20640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f3ec80b9c50 con 0x7f3ec80b2760 2024-09-21T03:34:22.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:22.942+0000 7f3ecdb20640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f3ec80b9ad0 con 0x7f3eb0004800 2024-09-21T03:34:22.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:22.942+0000 7f3ecdb20640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f3ec80b9dd0 con 0x7f3ec80cbad0 2024-09-21T03:34:22.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:22.942+0000 7f3ec7fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3ec80b2760 0x7f3ec80b2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:22.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:22.942+0000 7f3ec7fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3ec80b2760 0x7f3ec80b2b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:52664/0 (socket says 172.21.15.38:52664) 2024-09-21T03:34:22.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:22.942+0000 7f3ec7fff640 1 -- 172.21.15.38:0/1609323503 learned_addr learned my addr 172.21.15.38:0/1609323503 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:34:22.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:22.942+0000 7f3ec77fe640 1 --2- 172.21.15.38:0/1609323503 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3eb0004800 0x7f3ec806eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:22.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:22.942+0000 7f3ec6ffd640 1 --2- 172.21.15.38:0/1609323503 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3ec80cbad0 0x7f3ec8157270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:22.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:22.943+0000 7f3ec7fff640 1 -- 172.21.15.38:0/1609323503 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3ec80cbad0 msgr2=0x7f3ec8157270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:22.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:22.943+0000 7f3ec7fff640 1 --2- 172.21.15.38:0/1609323503 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3ec80cbad0 0x7f3ec8157270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:22.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:22.943+0000 7f3ec7fff640 1 -- 172.21.15.38:0/1609323503 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3eb0004800 msgr2=0x7f3ec806eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:22.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:22.943+0000 7f3ec7fff640 1 --2- 172.21.15.38:0/1609323503 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3eb0004800 0x7f3ec806eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:22.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:22.943+0000 7f3ec7fff640 1 -- 172.21.15.38:0/1609323503 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ec80b30b0 con 0x7f3ec80b2760 2024-09-21T03:34:22.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:22.943+0000 7f3ec7fff640 1 --2- 172.21.15.38:0/1609323503 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3ec80b2760 0x7f3ec80b2b70 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f3ec815c9c0 tx=0x7f3ec00064f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:22.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:22.943+0000 7f3eaffff640 1 -- 172.21.15.38:0/1609323503 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ec00119c0 con 0x7f3ec80b2760 2024-09-21T03:34:22.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:22.943+0000 7f3eaffff640 1 -- 172.21.15.38:0/1609323503 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3ec0017070 con 0x7f3ec80b2760 2024-09-21T03:34:22.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:22.943+0000 7f3ecdb20640 1 -- 172.21.15.38:0/1609323503 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ec806f080 con 0x7f3ec80b2760 2024-09-21T03:34:22.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:22.943+0000 7f3eaffff640 1 -- 172.21.15.38:0/1609323503 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ec0011dc0 con 0x7f3ec80b2760 2024-09-21T03:34:22.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:22.943+0000 7f3ecdb20640 1 -- 172.21.15.38:0/1609323503 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ec806f870 con 0x7f3ec80b2760 2024-09-21T03:34:22.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:22.944+0000 7f3ecdb20640 1 -- 172.21.15.38:0/1609323503 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ec806f300 con 0x7f3ec80b2760 2024-09-21T03:34:22.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:22.945+0000 7f3eaffff640 1 -- 172.21.15.38:0/1609323503 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f3ec001c460 con 0x7f3ec80b2760 2024-09-21T03:34:22.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:22.945+0000 7f3eaffff640 1 --2- 172.21.15.38:0/1609323503 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3eb4078b90 0x7f3eb407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:22.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:22.945+0000 7f3eaffff640 1 -- 172.21.15.38:0/1609323503 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6619+0+0 (secure 0 0 0) 0x7f3ec009b760 con 0x7f3ec80b2760 2024-09-21T03:34:22.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:22.946+0000 7f3ec77fe640 1 --2- 172.21.15.38:0/1609323503 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3eb4078b90 0x7f3eb407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:22.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:22.946+0000 7f3ec77fe640 1 --2- 172.21.15.38:0/1609323503 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3eb4078b90 0x7f3eb407b0a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3eb8000c00 tx=0x7f3eb8006b70 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:22.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:22.950+0000 7f3eaffff640 1 -- 172.21.15.38:0/1609323503 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f3ec80b30b0 con 0x7f3ec80b2760 2024-09-21T03:34:23.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:23.174+0000 7f3ecdb20640 1 -- 172.21.15.38:0/1609323503 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f3ec80cfca0 con 0x7f3ec80b2760 2024-09-21T03:34:23.176 INFO:tasks.ceph.mon.a.smithi038.stderr:2024-09-21T03:34:23.175+0000 7f3f45713640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T03:34:23.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:23.180+0000 7f3eaffff640 1 -- 172.21.15.38:0/1609323503 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v10) ==== 161+0+0 (secure 0 0 0) 0x7f3ec81592d0 con 0x7f3ec80b2760 2024-09-21T03:34:23.182 INFO:teuthology.orchestra.run.smithi038.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-21T03:34:23.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:23.183+0000 7f3ecdb20640 1 -- 172.21.15.38:0/1609323503 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3eb4078b90 msgr2=0x7f3eb407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:23.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:23.183+0000 7f3ecdb20640 1 --2- 172.21.15.38:0/1609323503 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3eb4078b90 0x7f3eb407b0a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3eb8000c00 tx=0x7f3eb8006b70 comp rx=0 tx=0).stop 2024-09-21T03:34:23.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:23.183+0000 7f3ecdb20640 1 -- 172.21.15.38:0/1609323503 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3ec80b2760 msgr2=0x7f3ec80b2b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:23.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:23.183+0000 7f3ecdb20640 1 --2- 172.21.15.38:0/1609323503 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3ec80b2760 0x7f3ec80b2b70 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f3ec815c9c0 tx=0x7f3ec00064f0 comp rx=0 tx=0).stop 2024-09-21T03:34:23.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:23.185+0000 7f3ecdb20640 1 -- 172.21.15.38:0/1609323503 shutdown_connections 2024-09-21T03:34:23.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:23.185+0000 7f3ecdb20640 1 --2- 172.21.15.38:0/1609323503 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3ec80b2760 0x7f3ec80b2b70 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:23.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:23.185+0000 7f3ecdb20640 1 --2- 172.21.15.38:0/1609323503 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3eb4078b90 0x7f3eb407b0a0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:23.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:23.185+0000 7f3ecdb20640 1 --2- 172.21.15.38:0/1609323503 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3ec80cbad0 0x7f3ec8157270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:23.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:23.185+0000 7f3ecdb20640 1 --2- 172.21.15.38:0/1609323503 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3eb0004800 0x7f3ec806eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:23.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:23.185+0000 7f3ecdb20640 1 -- 172.21.15.38:0/1609323503 >> 172.21.15.38:0/1609323503 conn(0x7f3ec809d830 msgr2=0x7f3ec80b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:23.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:23.185+0000 7f3ecdb20640 1 -- 172.21.15.38:0/1609323503 shutdown_connections 2024-09-21T03:34:23.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:23.185+0000 7f3ecdb20640 1 -- 172.21.15.38:0/1609323503 wait complete. 2024-09-21T03:34:23.189 INFO:tasks.ceph.mds.d.smithi195.stderr: -4669> 2024-09-21T03:33:41.214+0000 7fdfaef82640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T03:34:23.198 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-21T03:34:23.247 INFO:tasks.ceph.mds.d.smithi195.stdout:starting mds.d at 2024-09-21T03:34:23.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:23.316+0000 7fd1c7ac5640 1 -- 172.21.15.38:0/1516837211 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd1c00b9ad0 msgr2=0x7fd1c00d0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:23.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:23.316+0000 7fd1c7ac5640 1 --2- 172.21.15.38:0/1516837211 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd1c00b9ad0 0x7fd1c00d0be0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fd1b40098e0 tx=0x7fd1b402a530 comp rx=0 tx=0).stop 2024-09-21T03:34:23.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:23.316+0000 7fd1c7ac5640 1 -- 172.21.15.38:0/1516837211 shutdown_connections 2024-09-21T03:34:23.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:23.316+0000 7fd1c7ac5640 1 --2- 172.21.15.38:0/1516837211 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd1c00b9ad0 0x7fd1c00d0be0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:23.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:23.316+0000 7fd1c7ac5640 1 --2- 172.21.15.38:0/1516837211 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd1c015ee80 0x7fd1c00d0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:23.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:23.316+0000 7fd1c7ac5640 1 -- 172.21.15.38:0/1516837211 >> 172.21.15.38:0/1516837211 conn(0x7fd1c009d830 msgr2=0x7fd1c00b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:23.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:23.316+0000 7fd1c7ac5640 1 -- 172.21.15.38:0/1516837211 shutdown_connections 2024-09-21T03:34:23.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:23.316+0000 7fd1c7ac5640 1 -- 172.21.15.38:0/1516837211 wait complete. 2024-09-21T03:34:23.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:23.318+0000 7fd1c7ac5640 1 Processor -- start 2024-09-21T03:34:23.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:23.318+0000 7fd1c7ac5640 1 -- start start 2024-09-21T03:34:23.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:23.319+0000 7fd1c7ac5640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd1c00b9ad0 0x7fd1c01465b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:23.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:23.319+0000 7fd1c7ac5640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd1c00d1120 0x7fd1c01f8d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:23.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:23.319+0000 7fd1c7ac5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd1c015ee80 0x7fd1c01f92c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:23.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:23.319+0000 7fd1c7ac5640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fd1c00b6700 con 0x7fd1c015ee80 2024-09-21T03:34:23.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:23.319+0000 7fd1c7ac5640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd1c00b6580 con 0x7fd1c00b9ad0 2024-09-21T03:34:23.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:23.319+0000 7fd1c7ac5640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fd1c00b6880 con 0x7fd1c00d1120 2024-09-21T03:34:23.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:23.319+0000 7fd1c6062640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd1c015ee80 0x7fd1c01f92c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:23.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:23.319+0000 7fd1c6062640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd1c015ee80 0x7fd1c01f92c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:52670/0 (socket says 172.21.15.38:52670) 2024-09-21T03:34:23.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:23.320+0000 7fd1c6062640 1 -- 172.21.15.38:0/576072449 learned_addr learned my addr 172.21.15.38:0/576072449 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:34:23.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:23.320+0000 7fd1c5861640 1 --2- 172.21.15.38:0/576072449 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd1c00b9ad0 0x7fd1c01465b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:23.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:23.320+0000 7fd1c5060640 1 --2- 172.21.15.38:0/576072449 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd1c00d1120 0x7fd1c01f8d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:23.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:23.320+0000 7fd1c6062640 1 -- 172.21.15.38:0/576072449 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd1c00d1120 msgr2=0x7fd1c01f8d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:23.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:23.320+0000 7fd1c6062640 1 --2- 172.21.15.38:0/576072449 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd1c00d1120 0x7fd1c01f8d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:23.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:23.320+0000 7fd1c6062640 1 -- 172.21.15.38:0/576072449 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd1c00b9ad0 msgr2=0x7fd1c01465b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:23.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:23.320+0000 7fd1c6062640 1 --2- 172.21.15.38:0/576072449 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd1c00b9ad0 0x7fd1c01465b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:23.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:23.320+0000 7fd1c6062640 1 -- 172.21.15.38:0/576072449 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1c01588e0 con 0x7fd1c015ee80 2024-09-21T03:34:23.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:23.320+0000 7fd1c6062640 1 --2- 172.21.15.38:0/576072449 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd1c015ee80 0x7fd1c01f92c0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fd1b800c440 tx=0x7fd1b800c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:23.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:23.320+0000 7fd1be7fc640 1 -- 172.21.15.38:0/576072449 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1b8017070 con 0x7fd1c015ee80 2024-09-21T03:34:23.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:23.320+0000 7fd1be7fc640 1 -- 172.21.15.38:0/576072449 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd1b80035f0 con 0x7fd1c015ee80 2024-09-21T03:34:23.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:23.320+0000 7fd1c7ac5640 1 -- 172.21.15.38:0/576072449 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1c0157d60 con 0x7fd1c015ee80 2024-09-21T03:34:23.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:23.320+0000 7fd1be7fc640 1 -- 172.21.15.38:0/576072449 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1b8003a40 con 0x7fd1c015ee80 2024-09-21T03:34:23.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:23.320+0000 7fd1c7ac5640 1 -- 172.21.15.38:0/576072449 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1c0158320 con 0x7fd1c015ee80 2024-09-21T03:34:23.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:23.321+0000 7fd1c7ac5640 1 -- 172.21.15.38:0/576072449 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1c00b6d50 con 0x7fd1c015ee80 2024-09-21T03:34:23.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:23.322+0000 7fd1be7fc640 1 -- 172.21.15.38:0/576072449 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fd1b801a070 con 0x7fd1c015ee80 2024-09-21T03:34:23.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:23.323+0000 7fd1be7fc640 1 --2- 172.21.15.38:0/576072449 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd1ac078b90 0x7fd1ac07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:23.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:23.323+0000 7fd1be7fc640 1 -- 172.21.15.38:0/576072449 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6705+0+0 (secure 0 0 0) 0x7fd1b809a190 con 0x7fd1c015ee80 2024-09-21T03:34:23.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:23.323+0000 7fd1c5861640 1 --2- 172.21.15.38:0/576072449 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd1ac078b90 0x7fd1ac07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:23.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:23.323+0000 7fd1c5861640 1 --2- 172.21.15.38:0/576072449 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd1ac078b90 0x7fd1ac07b0a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fd1b0005fd0 tx=0x7fd1b0009500 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:23.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:23.327+0000 7fd1be7fc640 1 -- 172.21.15.38:0/576072449 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fd1c01592d0 con 0x7fd1c015ee80 2024-09-21T03:34:23.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:23.557+0000 7fd1c7ac5640 1 -- 172.21.15.38:0/576072449 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fd1c0157af0 con 0x7fd1c015ee80 2024-09-21T03:34:24.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:24.182+0000 7fd1be7fc640 1 -- 172.21.15.38:0/576072449 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v11) ==== 104+0+0 (secure 0 0 0) 0x7fd1b8061d70 con 0x7fd1c015ee80 2024-09-21T03:34:24.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:24.185+0000 7fd1c7ac5640 1 -- 172.21.15.38:0/576072449 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd1ac078b90 msgr2=0x7fd1ac07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:24.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:24.185+0000 7fd1c7ac5640 1 --2- 172.21.15.38:0/576072449 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd1ac078b90 0x7fd1ac07b0a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fd1b0005fd0 tx=0x7fd1b0009500 comp rx=0 tx=0).stop 2024-09-21T03:34:24.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:24.185+0000 7fd1c7ac5640 1 -- 172.21.15.38:0/576072449 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd1c015ee80 msgr2=0x7fd1c01f92c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:24.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:24.185+0000 7fd1c7ac5640 1 --2- 172.21.15.38:0/576072449 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd1c015ee80 0x7fd1c01f92c0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fd1b800c440 tx=0x7fd1b800c900 comp rx=0 tx=0).stop 2024-09-21T03:34:24.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:24.185+0000 7fd1c7ac5640 1 -- 172.21.15.38:0/576072449 shutdown_connections 2024-09-21T03:34:24.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:24.185+0000 7fd1c7ac5640 1 --2- 172.21.15.38:0/576072449 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd1c015ee80 0x7fd1c01f92c0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:24.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:24.185+0000 7fd1c7ac5640 1 --2- 172.21.15.38:0/576072449 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd1ac078b90 0x7fd1ac07b0a0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:24.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:24.185+0000 7fd1c7ac5640 1 --2- 172.21.15.38:0/576072449 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd1c00d1120 0x7fd1c01f8d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:24.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:24.185+0000 7fd1c7ac5640 1 --2- 172.21.15.38:0/576072449 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd1c00b9ad0 0x7fd1c01465b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:24.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:24.185+0000 7fd1c7ac5640 1 -- 172.21.15.38:0/576072449 >> 172.21.15.38:0/576072449 conn(0x7fd1c009d830 msgr2=0x7fd1c00b57f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:24.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:24.186+0000 7fd1c7ac5640 1 -- 172.21.15.38:0/576072449 shutdown_connections 2024-09-21T03:34:24.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:24.186+0000 7fd1c7ac5640 1 -- 172.21.15.38:0/576072449 wait complete. 2024-09-21T03:34:24.202 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-21T03:34:24.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:24.330+0000 7f3cbf430640 1 -- 172.21.15.38:0/3543511498 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3cb80d0e40 msgr2=0x7f3cb81627b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:24.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:24.330+0000 7f3cbf430640 1 --2- 172.21.15.38:0/3543511498 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3cb80d0e40 0x7f3cb81627b0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f3cb806dc80 tx=0x7f3ca802a460 comp rx=0 tx=0).stop 2024-09-21T03:34:24.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:24.330+0000 7f3cbf430640 1 -- 172.21.15.38:0/3543511498 shutdown_connections 2024-09-21T03:34:24.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:24.330+0000 7f3cbf430640 1 --2- 172.21.15.38:0/3543511498 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3cb8162cf0 0x7f3cb80b3000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:24.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:24.330+0000 7f3cbf430640 1 --2- 172.21.15.38:0/3543511498 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3cb80d0e40 0x7f3cb81627b0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:24.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:24.330+0000 7f3cbf430640 1 --2- 172.21.15.38:0/3543511498 >> v2:172.21.15.195:3300/0 conn(0x7f3cb8163350 0x7f3cb815c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:24.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:24.330+0000 7f3cbf430640 1 -- 172.21.15.38:0/3543511498 >> 172.21.15.38:0/3543511498 conn(0x7f3cb806cde0 msgr2=0x7f3cb806d1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:24.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:24.330+0000 7f3cbf430640 1 -- 172.21.15.38:0/3543511498 shutdown_connections 2024-09-21T03:34:24.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:24.330+0000 7f3cbf430640 1 -- 172.21.15.38:0/3543511498 wait complete. 2024-09-21T03:34:24.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:24.332+0000 7f3cbf430640 1 Processor -- start 2024-09-21T03:34:24.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:24.332+0000 7f3cbf430640 1 -- start start 2024-09-21T03:34:24.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:24.333+0000 7f3cbf430640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3cb80d0e40 0x7f3cb8151650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:24.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:24.334+0000 7f3cbf430640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3cb8162cf0 0x7f3cb81f91b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:24.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:24.334+0000 7f3cbf430640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3cb8163350 0x7f3cb81f96f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:24.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:24.334+0000 7f3cbf430640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f3cb80b38d0 con 0x7f3cb80d0e40 2024-09-21T03:34:24.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:24.334+0000 7f3cbf430640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f3cb80b3750 con 0x7f3cb8163350 2024-09-21T03:34:24.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:24.334+0000 7f3cbf430640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f3cb80b3a50 con 0x7f3cb8162cf0 2024-09-21T03:34:24.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:24.334+0000 7f3cbd1cc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3cb80d0e40 0x7f3cb8151650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:24.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:24.334+0000 7f3cbd1cc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3cb80d0e40 0x7f3cb8151650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:52698/0 (socket says 172.21.15.38:52698) 2024-09-21T03:34:24.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:24.334+0000 7f3cbd1cc640 1 -- 172.21.15.38:0/3418883354 learned_addr learned my addr 172.21.15.38:0/3418883354 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:34:24.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:24.334+0000 7f3cbc9cb640 1 --2- 172.21.15.38:0/3418883354 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3cb8162cf0 0x7f3cb81f91b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:24.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:24.334+0000 7f3cbd9cd640 1 --2- 172.21.15.38:0/3418883354 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3cb8163350 0x7f3cb81f96f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:24.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:24.334+0000 7f3cbd1cc640 1 -- 172.21.15.38:0/3418883354 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3cb8162cf0 msgr2=0x7f3cb81f91b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:24.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:24.334+0000 7f3cbd1cc640 1 --2- 172.21.15.38:0/3418883354 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3cb8162cf0 0x7f3cb81f91b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:24.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:24.334+0000 7f3cbd1cc640 1 -- 172.21.15.38:0/3418883354 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3cb8163350 msgr2=0x7f3cb81f96f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:24.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:24.334+0000 7f3cbd1cc640 1 --2- 172.21.15.38:0/3418883354 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3cb8163350 0x7f3cb81f96f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:24.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:24.334+0000 7f3cbd1cc640 1 -- 172.21.15.38:0/3418883354 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cb80b6bd0 con 0x7f3cb80d0e40 2024-09-21T03:34:24.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:24.334+0000 7f3cbd1cc640 1 --2- 172.21.15.38:0/3418883354 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3cb80d0e40 0x7f3cb8151650 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f3cb400b410 tx=0x7f3cb400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:24.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:24.334+0000 7f3cadffb640 1 -- 172.21.15.38:0/3418883354 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cb4015020 con 0x7f3cb80d0e40 2024-09-21T03:34:24.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:24.334+0000 7f3cadffb640 1 -- 172.21.15.38:0/3418883354 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3cb4011940 con 0x7f3cb80d0e40 2024-09-21T03:34:24.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:24.334+0000 7f3cbf430640 1 -- 172.21.15.38:0/3418883354 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3cb80ca1b0 con 0x7f3cb80d0e40 2024-09-21T03:34:24.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:24.334+0000 7f3cadffb640 1 -- 172.21.15.38:0/3418883354 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cb4011d90 con 0x7f3cb80d0e40 2024-09-21T03:34:24.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:24.335+0000 7f3cbf430640 1 -- 172.21.15.38:0/3418883354 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3cb80c91c0 con 0x7f3cb80d0e40 2024-09-21T03:34:24.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:24.336+0000 7f3cbf430640 1 -- 172.21.15.38:0/3418883354 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3cb80c7f40 con 0x7f3cb80d0e40 2024-09-21T03:34:24.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:24.336+0000 7f3cadffb640 1 -- 172.21.15.38:0/3418883354 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f3cb4018020 con 0x7f3cb80d0e40 2024-09-21T03:34:24.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:24.337+0000 7f3cadffb640 1 --2- 172.21.15.38:0/3418883354 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3c9c078b90 0x7f3c9c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:24.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:24.337+0000 7f3cadffb640 1 -- 172.21.15.38:0/3418883354 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6705+0+0 (secure 0 0 0) 0x7f3cb409aef0 con 0x7f3cb80d0e40 2024-09-21T03:34:24.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:24.337+0000 7f3cbc9cb640 1 --2- 172.21.15.38:0/3418883354 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3c9c078b90 0x7f3c9c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:24.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:24.338+0000 7f3cbc9cb640 1 --2- 172.21.15.38:0/3418883354 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3c9c078b90 0x7f3c9c07b0a0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f3ca8033000 tx=0x7f3ca8004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:24.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:24.342+0000 7f3cadffb640 1 -- 172.21.15.38:0/3418883354 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f3cb80c7f40 con 0x7f3cb80d0e40 2024-09-21T03:34:24.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:24.531+0000 7f3cbf430640 1 -- 172.21.15.38:0/3418883354 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f3cb80b3000 con 0x7f3cb80d0e40 2024-09-21T03:34:25.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:25.466+0000 7f3cadffb640 1 -- 172.21.15.38:0/3418883354 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v30) ==== 181+0+0 (secure 0 0 0) 0x7f3cb401f020 con 0x7f3cb80d0e40 2024-09-21T03:34:25.468 INFO:teuthology.orchestra.run.smithi038.stderr:pool 'cephfs_metadata' removed 2024-09-21T03:34:25.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:25.469+0000 7f3cbf430640 1 -- 172.21.15.38:0/3418883354 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3c9c078b90 msgr2=0x7f3c9c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:25.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:25.469+0000 7f3cbf430640 1 --2- 172.21.15.38:0/3418883354 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3c9c078b90 0x7f3c9c07b0a0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f3ca8033000 tx=0x7f3ca8004730 comp rx=0 tx=0).stop 2024-09-21T03:34:25.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:25.469+0000 7f3cbf430640 1 -- 172.21.15.38:0/3418883354 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3cb80d0e40 msgr2=0x7f3cb8151650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:25.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:25.469+0000 7f3cbf430640 1 --2- 172.21.15.38:0/3418883354 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3cb80d0e40 0x7f3cb8151650 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f3cb400b410 tx=0x7f3cb400b8d0 comp rx=0 tx=0).stop 2024-09-21T03:34:25.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:25.469+0000 7f3cbf430640 1 -- 172.21.15.38:0/3418883354 shutdown_connections 2024-09-21T03:34:25.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:25.469+0000 7f3cbf430640 1 --2- 172.21.15.38:0/3418883354 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3cb8163350 0x7f3cb81f96f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:25.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:25.469+0000 7f3cbf430640 1 --2- 172.21.15.38:0/3418883354 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3c9c078b90 0x7f3c9c07b0a0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:25.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:25.469+0000 7f3cbf430640 1 --2- 172.21.15.38:0/3418883354 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3cb8162cf0 0x7f3cb81f91b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:25.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:25.469+0000 7f3cbf430640 1 --2- 172.21.15.38:0/3418883354 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3cb80d0e40 0x7f3cb8151650 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:25.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:25.469+0000 7f3cbf430640 1 -- 172.21.15.38:0/3418883354 >> 172.21.15.38:0/3418883354 conn(0x7f3cb806cde0 msgr2=0x7f3cb8160a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:25.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:25.470+0000 7f3cbf430640 1 -- 172.21.15.38:0/3418883354 shutdown_connections 2024-09-21T03:34:25.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:25.470+0000 7f3cbf430640 1 -- 172.21.15.38:0/3418883354 wait complete. 2024-09-21T03:34:25.490 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-21T03:34:25.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:25.619+0000 7fd532455640 1 -- 172.21.15.38:0/94051213 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd52c162cf0 msgr2=0x7fd52c0b3000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:25.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:25.619+0000 7fd532455640 1 --2- 172.21.15.38:0/94051213 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd52c162cf0 0x7fd52c0b3000 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fd52c0cb620 tx=0x7fd52402a7a0 comp rx=0 tx=0).stop 2024-09-21T03:34:25.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:25.619+0000 7fd532455640 1 -- 172.21.15.38:0/94051213 shutdown_connections 2024-09-21T03:34:25.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:25.619+0000 7fd532455640 1 --2- 172.21.15.38:0/94051213 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd52c162cf0 0x7fd52c0b3000 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:25.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:25.619+0000 7fd532455640 1 --2- 172.21.15.38:0/94051213 >> v2:172.21.15.195:3300/0 conn(0x7fd52c15c650 0x7fd52c1627b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:25.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:25.619+0000 7fd532455640 1 -- 172.21.15.38:0/94051213 >> 172.21.15.38:0/94051213 conn(0x7fd52c06cde0 msgr2=0x7fd52c06d1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:25.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:25.619+0000 7fd532455640 1 -- 172.21.15.38:0/94051213 shutdown_connections 2024-09-21T03:34:25.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:25.619+0000 7fd532455640 1 -- 172.21.15.38:0/94051213 wait complete. 2024-09-21T03:34:25.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:25.621+0000 7fd532455640 1 Processor -- start 2024-09-21T03:34:25.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:25.621+0000 7fd532455640 1 -- start start 2024-09-21T03:34:25.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:25.623+0000 7fd532455640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd52c15c650 0x7fd52c0c8710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:25.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:25.623+0000 7fd532455640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd52c162cf0 0x7fd52c0cca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:25.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:25.623+0000 7fd532455640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd52c163350 0x7fd52c0cbfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:25.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:25.623+0000 7fd532455640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fd52c0b3a20 con 0x7fd52c15c650 2024-09-21T03:34:25.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:25.623+0000 7fd532455640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd52c0b38a0 con 0x7fd52c163350 2024-09-21T03:34:25.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:25.623+0000 7fd532455640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fd52c0b3ba0 con 0x7fd52c162cf0 2024-09-21T03:34:25.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:25.623+0000 7fd52bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd52c15c650 0x7fd52c0c8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:25.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:25.623+0000 7fd52bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd52c15c650 0x7fd52c0c8710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:52722/0 (socket says 172.21.15.38:52722) 2024-09-21T03:34:25.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:25.623+0000 7fd52bfff640 1 -- 172.21.15.38:0/1001870222 learned_addr learned my addr 172.21.15.38:0/1001870222 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:34:25.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:25.623+0000 7fd5309f2640 1 --2- 172.21.15.38:0/1001870222 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd52c163350 0x7fd52c0cbfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:25.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:25.623+0000 7fd52b7fe640 1 --2- 172.21.15.38:0/1001870222 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd52c162cf0 0x7fd52c0cca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:25.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:25.623+0000 7fd52bfff640 1 -- 172.21.15.38:0/1001870222 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd52c162cf0 msgr2=0x7fd52c0cca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:25.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:25.623+0000 7fd52bfff640 1 --2- 172.21.15.38:0/1001870222 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd52c162cf0 0x7fd52c0cca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:25.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:25.623+0000 7fd52bfff640 1 -- 172.21.15.38:0/1001870222 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd52c163350 msgr2=0x7fd52c0cbfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:25.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:25.623+0000 7fd52bfff640 1 --2- 172.21.15.38:0/1001870222 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd52c163350 0x7fd52c0cbfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:25.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:25.623+0000 7fd52bfff640 1 -- 172.21.15.38:0/1001870222 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd52c0c9080 con 0x7fd52c15c650 2024-09-21T03:34:25.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:25.623+0000 7fd52bfff640 1 --2- 172.21.15.38:0/1001870222 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd52c15c650 0x7fd52c0c8710 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fd51c00c3c0 tx=0x7fd51c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:25.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:25.623+0000 7fd528ff9640 1 -- 172.21.15.38:0/1001870222 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd51c017070 con 0x7fd52c15c650 2024-09-21T03:34:25.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:25.623+0000 7fd528ff9640 1 -- 172.21.15.38:0/1001870222 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd51c0035f0 con 0x7fd52c15c650 2024-09-21T03:34:25.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:25.623+0000 7fd532455640 1 -- 172.21.15.38:0/1001870222 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd52c0c9360 con 0x7fd52c15c650 2024-09-21T03:34:25.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:25.624+0000 7fd528ff9640 1 -- 172.21.15.38:0/1001870222 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd51c003a40 con 0x7fd52c15c650 2024-09-21T03:34:25.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:25.624+0000 7fd532455640 1 -- 172.21.15.38:0/1001870222 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd52c0cbaa0 con 0x7fd52c15c650 2024-09-21T03:34:25.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:25.625+0000 7fd532455640 1 -- 172.21.15.38:0/1001870222 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd52c1627b0 con 0x7fd52c15c650 2024-09-21T03:34:25.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:25.625+0000 7fd528ff9640 1 -- 172.21.15.38:0/1001870222 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fd51c01a070 con 0x7fd52c15c650 2024-09-21T03:34:25.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:25.626+0000 7fd528ff9640 1 --2- 172.21.15.38:0/1001870222 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd50c078bc0 0x7fd50c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:25.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:25.626+0000 7fd528ff9640 1 -- 172.21.15.38:0/1001870222 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6267+0+0 (secure 0 0 0) 0x7fd51c09a220 con 0x7fd52c15c650 2024-09-21T03:34:25.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:25.626+0000 7fd52b7fe640 1 --2- 172.21.15.38:0/1001870222 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd50c078bc0 0x7fd50c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:25.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:25.627+0000 7fd52b7fe640 1 --2- 172.21.15.38:0/1001870222 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd50c078bc0 0x7fd50c07b0d0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fd5140058c0 tx=0x7fd514005850 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:25.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:25.630+0000 7fd528ff9640 1 -- 172.21.15.38:0/1001870222 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fd51c062430 con 0x7fd52c15c650 2024-09-21T03:34:25.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:25.818+0000 7fd532455640 1 -- 172.21.15.38:0/1001870222 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fd52c068ea0 con 0x7fd52c15c650 2024-09-21T03:34:26.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:26.473+0000 7fd528ff9640 1 -- 172.21.15.38:0/1001870222 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v31) ==== 169+0+0 (secure 0 0 0) 0x7fd51c061d30 con 0x7fd52c15c650 2024-09-21T03:34:26.475 INFO:teuthology.orchestra.run.smithi038.stderr:pool 'cephfs_data' removed 2024-09-21T03:34:26.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:26.486+0000 7fd532455640 1 -- 172.21.15.38:0/1001870222 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd50c078bc0 msgr2=0x7fd50c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:26.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:26.486+0000 7fd532455640 1 --2- 172.21.15.38:0/1001870222 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd50c078bc0 0x7fd50c07b0d0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fd5140058c0 tx=0x7fd514005850 comp rx=0 tx=0).stop 2024-09-21T03:34:26.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:26.486+0000 7fd532455640 1 -- 172.21.15.38:0/1001870222 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd52c15c650 msgr2=0x7fd52c0c8710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:26.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:26.486+0000 7fd532455640 1 --2- 172.21.15.38:0/1001870222 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd52c15c650 0x7fd52c0c8710 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fd51c00c3c0 tx=0x7fd51c00c880 comp rx=0 tx=0).stop 2024-09-21T03:34:26.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:26.487+0000 7fd532455640 1 -- 172.21.15.38:0/1001870222 shutdown_connections 2024-09-21T03:34:26.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:26.487+0000 7fd532455640 1 --2- 172.21.15.38:0/1001870222 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd52c163350 0x7fd52c0cbfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:26.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:26.487+0000 7fd532455640 1 --2- 172.21.15.38:0/1001870222 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd50c078bc0 0x7fd50c07b0d0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:26.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:26.487+0000 7fd532455640 1 --2- 172.21.15.38:0/1001870222 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd52c162cf0 0x7fd52c0cca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:26.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:26.487+0000 7fd532455640 1 --2- 172.21.15.38:0/1001870222 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd52c15c650 0x7fd52c0c8710 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:26.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:26.487+0000 7fd532455640 1 -- 172.21.15.38:0/1001870222 >> 172.21.15.38:0/1001870222 conn(0x7fd52c06cde0 msgr2=0x7fd52c0b9f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:26.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:26.487+0000 7fd532455640 1 -- 172.21.15.38:0/1001870222 shutdown_connections 2024-09-21T03:34:26.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:26.488+0000 7fd532455640 1 -- 172.21.15.38:0/1001870222 wait complete. 2024-09-21T03:34:26.506 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-09-21T03:34:26.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:26.631+0000 7f4c301ba640 1 -- 172.21.15.38:0/2408209059 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4c280cf740 msgr2=0x7f4c28162600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:26.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:26.631+0000 7f4c301ba640 1 --2- 172.21.15.38:0/2408209059 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4c280cf740 0x7f4c28162600 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f4c20005fd0 tx=0x7f4c2002a5d0 comp rx=0 tx=0).stop 2024-09-21T03:34:26.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:26.631+0000 7f4c301ba640 1 -- 172.21.15.38:0/2408209059 shutdown_connections 2024-09-21T03:34:26.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:26.631+0000 7f4c301ba640 1 --2- 172.21.15.38:0/2408209059 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4c280cf740 0x7f4c28162600 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:26.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:26.631+0000 7f4c301ba640 1 --2- 172.21.15.38:0/2408209059 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4c28161ed0 0x7f4c280cf200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:26.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:26.631+0000 7f4c301ba640 1 -- 172.21.15.38:0/2408209059 >> 172.21.15.38:0/2408209059 conn(0x7f4c280d4510 msgr2=0x7f4c280d4920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:26.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:26.631+0000 7f4c301ba640 1 -- 172.21.15.38:0/2408209059 shutdown_connections 2024-09-21T03:34:26.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:26.631+0000 7f4c301ba640 1 -- 172.21.15.38:0/2408209059 wait complete. 2024-09-21T03:34:26.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:26.633+0000 7f4c301ba640 1 Processor -- start 2024-09-21T03:34:26.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:26.633+0000 7f4c301ba640 1 -- start start 2024-09-21T03:34:26.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:26.634+0000 7f4c301ba640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4c280cf740 0x7f4c281f6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:26.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:26.634+0000 7f4c301ba640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4c280d0460 0x7f4c28160fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:26.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:26.634+0000 7f4c301ba640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4c28161ed0 0x7f4c281f8d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:26.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:26.634+0000 7f4c301ba640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f4c28162b40 con 0x7f4c280cf740 2024-09-21T03:34:26.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:26.634+0000 7f4c301ba640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f4c28162cc0 con 0x7f4c28161ed0 2024-09-21T03:34:26.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:26.634+0000 7f4c301ba640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f4c28162e40 con 0x7f4c280d0460 2024-09-21T03:34:26.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:26.634+0000 7f4c2df56640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4c280cf740 0x7f4c281f6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:26.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:26.635+0000 7f4c2df56640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4c280cf740 0x7f4c281f6260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:52746/0 (socket says 172.21.15.38:52746) 2024-09-21T03:34:26.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:26.635+0000 7f4c2df56640 1 -- 172.21.15.38:0/1871379362 learned_addr learned my addr 172.21.15.38:0/1871379362 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:34:26.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:26.635+0000 7f4c2d755640 1 --2- 172.21.15.38:0/1871379362 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4c280d0460 0x7f4c28160fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:26.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:26.635+0000 7f4c2df56640 1 -- 172.21.15.38:0/1871379362 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4c280d0460 msgr2=0x7f4c28160fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:26.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:26.635+0000 7f4c2e757640 1 --2- 172.21.15.38:0/1871379362 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4c28161ed0 0x7f4c281f8d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:26.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:26.635+0000 7f4c2df56640 1 --2- 172.21.15.38:0/1871379362 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4c280d0460 0x7f4c28160fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:26.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:26.635+0000 7f4c2df56640 1 -- 172.21.15.38:0/1871379362 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4c28161ed0 msgr2=0x7f4c281f8d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:26.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:26.635+0000 7f4c2df56640 1 --2- 172.21.15.38:0/1871379362 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4c28161ed0 0x7f4c281f8d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:26.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:26.635+0000 7f4c2df56640 1 -- 172.21.15.38:0/1871379362 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c2815af30 con 0x7f4c280cf740 2024-09-21T03:34:26.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:26.635+0000 7f4c2df56640 1 --2- 172.21.15.38:0/1871379362 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4c280cf740 0x7f4c281f6260 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f4c2400c730 tx=0x7f4c2400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:26.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:26.635+0000 7f4c1e7fc640 1 -- 172.21.15.38:0/1871379362 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c24003b70 con 0x7f4c280cf740 2024-09-21T03:34:26.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:26.635+0000 7f4c1e7fc640 1 -- 172.21.15.38:0/1871379362 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4c24003d30 con 0x7f4c280cf740 2024-09-21T03:34:26.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:26.635+0000 7f4c301ba640 1 -- 172.21.15.38:0/1871379362 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c281f9870 con 0x7f4c280cf740 2024-09-21T03:34:26.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:26.635+0000 7f4c1e7fc640 1 -- 172.21.15.38:0/1871379362 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c2400aa50 con 0x7f4c280cf740 2024-09-21T03:34:26.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:26.635+0000 7f4c301ba640 1 -- 172.21.15.38:0/1871379362 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c281f5a60 con 0x7f4c280cf740 2024-09-21T03:34:26.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:26.636+0000 7f4c301ba640 1 -- 172.21.15.38:0/1871379362 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c281632f0 con 0x7f4c280cf740 2024-09-21T03:34:26.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:26.637+0000 7f4c1e7fc640 1 -- 172.21.15.38:0/1871379362 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f4c2400abf0 con 0x7f4c280cf740 2024-09-21T03:34:26.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:26.638+0000 7f4c1e7fc640 1 --2- 172.21.15.38:0/1871379362 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4c0c078b90 0x7f4c0c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:26.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:26.638+0000 7f4c1e7fc640 1 -- 172.21.15.38:0/1871379362 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5824+0+0 (secure 0 0 0) 0x7f4c2409ab00 con 0x7f4c280cf740 2024-09-21T03:34:26.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:26.638+0000 7f4c2d755640 1 --2- 172.21.15.38:0/1871379362 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4c0c078b90 0x7f4c0c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:26.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:26.639+0000 7f4c2d755640 1 --2- 172.21.15.38:0/1871379362 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4c0c078b90 0x7f4c0c07b0a0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f4c180058c0 tx=0x7f4c18005850 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:26.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:26.642+0000 7f4c1e7fc640 1 -- 172.21.15.38:0/1871379362 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f4c24062f60 con 0x7f4c280cf740 2024-09-21T03:34:26.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:26.829+0000 7f4c301ba640 1 -- 172.21.15.38:0/1871379362 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7f4c280d3450 con 0x7f4c280cf740 2024-09-21T03:34:27.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:27.480+0000 7f4c1e7fc640 1 -- 172.21.15.38:0/1871379362 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v32) ==== 178+0+0 (secure 0 0 0) 0x7f4c24062860 con 0x7f4c280cf740 2024-09-21T03:34:27.482 INFO:teuthology.orchestra.run.smithi038.stderr:pool 'cephfs_data_ec' removed 2024-09-21T03:34:27.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:27.488+0000 7f4c301ba640 1 -- 172.21.15.38:0/1871379362 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4c0c078b90 msgr2=0x7f4c0c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:27.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:27.488+0000 7f4c301ba640 1 --2- 172.21.15.38:0/1871379362 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4c0c078b90 0x7f4c0c07b0a0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f4c180058c0 tx=0x7f4c18005850 comp rx=0 tx=0).stop 2024-09-21T03:34:27.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:27.488+0000 7f4c301ba640 1 -- 172.21.15.38:0/1871379362 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4c280cf740 msgr2=0x7f4c281f6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:27.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:27.488+0000 7f4c301ba640 1 --2- 172.21.15.38:0/1871379362 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4c280cf740 0x7f4c281f6260 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f4c2400c730 tx=0x7f4c2400cbf0 comp rx=0 tx=0).stop 2024-09-21T03:34:27.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:27.488+0000 7f4c301ba640 1 -- 172.21.15.38:0/1871379362 shutdown_connections 2024-09-21T03:34:27.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:27.488+0000 7f4c301ba640 1 --2- 172.21.15.38:0/1871379362 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4c28161ed0 0x7f4c281f8d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:27.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:27.488+0000 7f4c301ba640 1 --2- 172.21.15.38:0/1871379362 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4c0c078b90 0x7f4c0c07b0a0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:27.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:27.488+0000 7f4c301ba640 1 --2- 172.21.15.38:0/1871379362 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4c280d0460 0x7f4c28160fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:27.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:27.488+0000 7f4c301ba640 1 --2- 172.21.15.38:0/1871379362 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4c280cf740 0x7f4c281f6260 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:27.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:27.488+0000 7f4c301ba640 1 -- 172.21.15.38:0/1871379362 >> 172.21.15.38:0/1871379362 conn(0x7f4c280d4510 msgr2=0x7f4c280b8480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:27.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:27.489+0000 7f4c301ba640 1 -- 172.21.15.38:0/1871379362 shutdown_connections 2024-09-21T03:34:27.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:27.489+0000 7f4c301ba640 1 -- 172.21.15.38:0/1871379362 wait complete. 2024-09-21T03:34:27.507 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-21T03:34:27.507 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-21T03:34:27.508 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-21T03:34:27.508 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T03:34:27.508 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-21T03:34:27.508 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-21T03:34:27.508 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-21T03:34:27.508 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T03:34:27.508 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-21T03:34:27.508 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-21T03:34:27.509 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-21T03:34:27.509 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T03:34:27.509 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-21T03:34:27.509 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-21T03:34:27.509 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-21T03:34:27.509 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T03:34:27.519 INFO:tasks.ceph.mds.a:Stopped 2024-09-21T03:34:27.519 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-21T03:34:27.520 INFO:tasks.ceph.mds.c:Stopped 2024-09-21T03:34:27.520 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-21T03:34:27.521 INFO:tasks.ceph.mds.b:Stopped 2024-09-21T03:34:27.522 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-21T03:34:27.523 INFO:tasks.ceph.mds.d:Stopped 2024-09-21T03:34:27.523 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-21T03:34:27.562 INFO:tasks.ceph.mds.a:Started 2024-09-21T03:34:27.563 DEBUG:teuthology.parallel:result is None 2024-09-21T03:34:27.563 INFO:tasks.ceph.mds.c:Started 2024-09-21T03:34:27.563 DEBUG:teuthology.parallel:result is None 2024-09-21T03:34:27.564 INFO:tasks.ceph.mds.b:Started 2024-09-21T03:34:27.565 DEBUG:teuthology.parallel:result is None 2024-09-21T03:34:27.566 INFO:tasks.ceph.mds.d:Started 2024-09-21T03:34:27.566 DEBUG:teuthology.parallel:result is None 2024-09-21T03:34:27.566 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-21T03:34:27.656 INFO:tasks.ceph.mds.b.smithi195.stdout:starting mds.b at 2024-09-21T03:34:27.656 INFO:tasks.ceph.mds.d.smithi195.stdout:starting mds.d at 2024-09-21T03:34:27.662 INFO:tasks.ceph.mds.a.smithi038.stdout:starting mds.a at 2024-09-21T03:34:27.662 INFO:tasks.ceph.mds.c.smithi038.stdout:starting mds.c at 2024-09-21T03:34:27.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:27.695+0000 7f8443d58640 1 -- 172.21.15.38:0/3760017292 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f843c0b9ad0 msgr2=0x7f843c0d0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:27.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:27.695+0000 7f8443d58640 1 --2- 172.21.15.38:0/3760017292 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f843c0b9ad0 0x7f843c0d0be0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f843c0cfba0 tx=0x7f842c02a4b0 comp rx=0 tx=0).stop 2024-09-21T03:34:27.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:27.696+0000 7f8443d58640 1 -- 172.21.15.38:0/3760017292 shutdown_connections 2024-09-21T03:34:27.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:27.696+0000 7f8443d58640 1 --2- 172.21.15.38:0/3760017292 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f843c0d1120 0x7f843c0ceb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:27.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:27.696+0000 7f8443d58640 1 --2- 172.21.15.38:0/3760017292 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f843c0b9ad0 0x7f843c0d0be0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:27.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:27.696+0000 7f8443d58640 1 --2- 172.21.15.38:0/3760017292 >> v2:172.21.15.195:3300/0 conn(0x7f843c15ee80 0x7f843c0d0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:27.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:27.696+0000 7f8443d58640 1 -- 172.21.15.38:0/3760017292 >> 172.21.15.38:0/3760017292 conn(0x7f843c09d830 msgr2=0x7f843c0b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:27.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:27.696+0000 7f8443d58640 1 -- 172.21.15.38:0/3760017292 shutdown_connections 2024-09-21T03:34:27.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:27.696+0000 7f8443d58640 1 -- 172.21.15.38:0/3760017292 wait complete. 2024-09-21T03:34:27.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:27.697+0000 7f8443d58640 1 Processor -- start 2024-09-21T03:34:27.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:27.697+0000 7f8443d58640 1 -- start start 2024-09-21T03:34:27.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:27.699+0000 7f8443d58640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f843c0b9ad0 0x7f843c15c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:27.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:27.699+0000 7f8443d58640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f843c0d1120 0x7f843c1577a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:27.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:27.699+0000 7f8443d58640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f843c15ee80 0x7f843c0b27a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:27.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:27.699+0000 7f8443d58640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f843c0cf520 con 0x7f843c15ee80 2024-09-21T03:34:27.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:27.699+0000 7f8443d58640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f843c0cf3a0 con 0x7f843c0b9ad0 2024-09-21T03:34:27.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:27.699+0000 7f8443d58640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f843c0b6580 con 0x7f843c0d1120 2024-09-21T03:34:27.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:27.699+0000 7f84422f5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f843c15ee80 0x7f843c0b27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:27.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:27.699+0000 7f84422f5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f843c15ee80 0x7f843c0b27a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:52808/0 (socket says 172.21.15.38:52808) 2024-09-21T03:34:27.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:27.699+0000 7f84422f5640 1 -- 172.21.15.38:0/4175740372 learned_addr learned my addr 172.21.15.38:0/4175740372 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:34:27.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:27.699+0000 7f84412f3640 1 --2- 172.21.15.38:0/4175740372 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f843c0d1120 0x7f843c1577a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:27.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:27.699+0000 7f8441af4640 1 --2- 172.21.15.38:0/4175740372 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f843c0b9ad0 0x7f843c15c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:27.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:27.699+0000 7f84422f5640 1 -- 172.21.15.38:0/4175740372 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f843c0d1120 msgr2=0x7f843c1577a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:27.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:27.699+0000 7f84422f5640 1 --2- 172.21.15.38:0/4175740372 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f843c0d1120 0x7f843c1577a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:27.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:27.699+0000 7f84422f5640 1 -- 172.21.15.38:0/4175740372 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f843c0b9ad0 msgr2=0x7f843c15c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:27.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:27.699+0000 7f84422f5640 1 --2- 172.21.15.38:0/4175740372 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f843c0b9ad0 0x7f843c15c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:27.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:27.699+0000 7f84422f5640 1 -- 172.21.15.38:0/4175740372 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f843c1f5e00 con 0x7f843c15ee80 2024-09-21T03:34:27.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:27.699+0000 7f84422f5640 1 --2- 172.21.15.38:0/4175740372 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f843c15ee80 0x7f843c0b27a0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f843400ea80 tx=0x7f843400ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:27.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:27.700+0000 7f84327fc640 1 -- 172.21.15.38:0/4175740372 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f843400ce40 con 0x7f843c15ee80 2024-09-21T03:34:27.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:27.700+0000 7f84327fc640 1 -- 172.21.15.38:0/4175740372 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8434004c10 con 0x7f843c15ee80 2024-09-21T03:34:27.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:27.700+0000 7f8443d58640 1 -- 172.21.15.38:0/4175740372 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f843c1f60e0 con 0x7f843c15ee80 2024-09-21T03:34:27.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:27.700+0000 7f84327fc640 1 -- 172.21.15.38:0/4175740372 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8434005020 con 0x7f843c15ee80 2024-09-21T03:34:27.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:27.700+0000 7f8443d58640 1 -- 172.21.15.38:0/4175740372 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f843c1f66a0 con 0x7f843c15ee80 2024-09-21T03:34:27.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:27.701+0000 7f8443d58640 1 -- 172.21.15.38:0/4175740372 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f843c06acf0 con 0x7f843c15ee80 2024-09-21T03:34:27.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:27.702+0000 7f84327fc640 1 -- 172.21.15.38:0/4175740372 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f84340051c0 con 0x7f843c15ee80 2024-09-21T03:34:27.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:27.702+0000 7f84327fc640 1 --2- 172.21.15.38:0/4175740372 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8420078b90 0x7f842007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:27.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:27.702+0000 7f84327fc640 1 -- 172.21.15.38:0/4175740372 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5335+0+0 (secure 0 0 0) 0x7f843409b780 con 0x7f843c15ee80 2024-09-21T03:34:27.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:27.702+0000 7f8441af4640 1 --2- 172.21.15.38:0/4175740372 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8420078b90 0x7f842007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:27.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:27.703+0000 7f8441af4640 1 --2- 172.21.15.38:0/4175740372 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8420078b90 0x7f842007b0a0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f8438009b30 tx=0x7f8438005f90 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:27.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:27.707+0000 7f84327fc640 1 -- 172.21.15.38:0/4175740372 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f843c06acf0 con 0x7f843c15ee80 2024-09-21T03:34:27.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:27.910+0000 7f8443d58640 1 -- 172.21.15.38:0/4175740372 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f843c0ceb90 con 0x7f843c15ee80 2024-09-21T03:34:28.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:28.499+0000 7f84327fc640 1 -- 172.21.15.38:0/4175740372 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v34) ==== 97+0+0 (secure 0 0 0) 0x7f8434063800 con 0x7f843c15ee80 2024-09-21T03:34:28.501 INFO:teuthology.orchestra.run.smithi038.stderr: removed all blocklist entries 2024-09-21T03:34:28.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:28.502+0000 7f8443d58640 1 -- 172.21.15.38:0/4175740372 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8420078b90 msgr2=0x7f842007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:28.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:28.502+0000 7f8443d58640 1 --2- 172.21.15.38:0/4175740372 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8420078b90 0x7f842007b0a0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f8438009b30 tx=0x7f8438005f90 comp rx=0 tx=0).stop 2024-09-21T03:34:28.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:28.502+0000 7f8443d58640 1 -- 172.21.15.38:0/4175740372 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f843c15ee80 msgr2=0x7f843c0b27a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:28.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:28.502+0000 7f8443d58640 1 --2- 172.21.15.38:0/4175740372 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f843c15ee80 0x7f843c0b27a0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f843400ea80 tx=0x7f843400ef40 comp rx=0 tx=0).stop 2024-09-21T03:34:28.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:28.502+0000 7f8443d58640 1 -- 172.21.15.38:0/4175740372 shutdown_connections 2024-09-21T03:34:28.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:28.502+0000 7f8443d58640 1 --2- 172.21.15.38:0/4175740372 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f843c15ee80 0x7f843c0b27a0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:28.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:28.502+0000 7f8443d58640 1 --2- 172.21.15.38:0/4175740372 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8420078b90 0x7f842007b0a0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:28.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:28.502+0000 7f8443d58640 1 --2- 172.21.15.38:0/4175740372 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f843c0d1120 0x7f843c1577a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:28.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:28.502+0000 7f8443d58640 1 --2- 172.21.15.38:0/4175740372 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f843c0b9ad0 0x7f843c15c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:28.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:28.502+0000 7f8443d58640 1 -- 172.21.15.38:0/4175740372 >> 172.21.15.38:0/4175740372 conn(0x7f843c09d830 msgr2=0x7f843c0d2ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:28.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:28.502+0000 7f8443d58640 1 -- 172.21.15.38:0/4175740372 shutdown_connections 2024-09-21T03:34:28.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:28.502+0000 7f8443d58640 1 -- 172.21.15.38:0/4175740372 wait complete. 2024-09-21T03:34:28.521 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-21T03:34:28.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:28.642+0000 7f9dbc2a2640 1 -- 172.21.15.38:0/2969844552 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9db4162f00 msgr2=0x7f9db40d24d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:28.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:28.642+0000 7f9dbc2a2640 1 --2- 172.21.15.38:0/2969844552 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9db4162f00 0x7f9db40d24d0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f9db40b7620 tx=0x7f9db002a560 comp rx=0 tx=0).stop 2024-09-21T03:34:28.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:28.642+0000 7f9dbc2a2640 1 -- 172.21.15.38:0/2969844552 shutdown_connections 2024-09-21T03:34:28.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:28.642+0000 7f9dbc2a2640 1 --2- 172.21.15.38:0/2969844552 >> v2:172.21.15.195:3300/0 conn(0x7f9db40b2e40 0x7f9db40b3250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:28.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:28.642+0000 7f9dbc2a2640 1 --2- 172.21.15.38:0/2969844552 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9db4162280 0x7f9db4162690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:28.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:28.642+0000 7f9dbc2a2640 1 --2- 172.21.15.38:0/2969844552 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9db4162f00 0x7f9db40d24d0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:28.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:28.642+0000 7f9dbc2a2640 1 -- 172.21.15.38:0/2969844552 >> 172.21.15.38:0/2969844552 conn(0x7f9db40b6b20 msgr2=0x7f9db40d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:28.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:28.642+0000 7f9dbc2a2640 1 -- 172.21.15.38:0/2969844552 shutdown_connections 2024-09-21T03:34:28.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:28.642+0000 7f9dbc2a2640 1 -- 172.21.15.38:0/2969844552 wait complete. 2024-09-21T03:34:28.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:28.644+0000 7f9dbc2a2640 1 Processor -- start 2024-09-21T03:34:28.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:28.644+0000 7f9dbc2a2640 1 -- start start 2024-09-21T03:34:28.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:28.645+0000 7f9dbc2a2640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9db40b2e40 0x7f9db415b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:28.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:28.645+0000 7f9dbc2a2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9db4162280 0x7f9db40acaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:28.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:28.645+0000 7f9dbc2a2640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9db40b2060 0x7f9db40b2470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:28.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:28.645+0000 7f9dbc2a2640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f9db40b3dd0 con 0x7f9db4162280 2024-09-21T03:34:28.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:28.645+0000 7f9dbc2a2640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f9db40b39a0 con 0x7f9db40b2060 2024-09-21T03:34:28.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:28.645+0000 7f9dbc2a2640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f9db40b3f50 con 0x7f9db40b2e40 2024-09-21T03:34:28.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:28.645+0000 7f9db983d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9db4162280 0x7f9db40acaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:28.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:28.645+0000 7f9db983d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9db4162280 0x7f9db40acaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:52820/0 (socket says 172.21.15.38:52820) 2024-09-21T03:34:28.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:28.645+0000 7f9db983d640 1 -- 172.21.15.38:0/2778600867 learned_addr learned my addr 172.21.15.38:0/2778600867 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:34:28.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:28.645+0000 7f9dba83f640 1 --2- 172.21.15.38:0/2778600867 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9db40b2060 0x7f9db40b2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:28.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:28.645+0000 7f9dba03e640 1 --2- 172.21.15.38:0/2778600867 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9db40b2e40 0x7f9db415b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:28.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:28.645+0000 7f9db983d640 1 -- 172.21.15.38:0/2778600867 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9db40b2e40 msgr2=0x7f9db415b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:28.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:28.645+0000 7f9db983d640 1 --2- 172.21.15.38:0/2778600867 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9db40b2e40 0x7f9db415b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:28.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:28.645+0000 7f9db983d640 1 -- 172.21.15.38:0/2778600867 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9db40b2060 msgr2=0x7f9db40b2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:28.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:28.645+0000 7f9db983d640 1 --2- 172.21.15.38:0/2778600867 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9db40b2060 0x7f9db40b2470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:28.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:28.645+0000 7f9db983d640 1 -- 172.21.15.38:0/2778600867 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9db40ad410 con 0x7f9db4162280 2024-09-21T03:34:28.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:28.646+0000 7f9db983d640 1 --2- 172.21.15.38:0/2778600867 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9db4162280 0x7f9db40acaf0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f9da400d750 tx=0x7f9da400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:28.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:28.646+0000 7f9daaffd640 1 -- 172.21.15.38:0/2778600867 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9da4017070 con 0x7f9db4162280 2024-09-21T03:34:28.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:28.646+0000 7f9daaffd640 1 -- 172.21.15.38:0/2778600867 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9da4009b20 con 0x7f9db4162280 2024-09-21T03:34:28.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:28.646+0000 7f9dbc2a2640 1 -- 172.21.15.38:0/2778600867 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9db40b29b0 con 0x7f9db4162280 2024-09-21T03:34:28.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:28.646+0000 7f9daaffd640 1 -- 172.21.15.38:0/2778600867 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9da4010460 con 0x7f9db4162280 2024-09-21T03:34:28.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:28.646+0000 7f9dbc2a2640 1 -- 172.21.15.38:0/2778600867 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9db41f5390 con 0x7f9db4162280 2024-09-21T03:34:28.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:28.647+0000 7f9dbc2a2640 1 -- 172.21.15.38:0/2778600867 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9db4162f00 con 0x7f9db4162280 2024-09-21T03:34:28.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:28.648+0000 7f9daaffd640 1 -- 172.21.15.38:0/2778600867 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f9da401a070 con 0x7f9db4162280 2024-09-21T03:34:28.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:28.648+0000 7f9daaffd640 1 --2- 172.21.15.38:0/2778600867 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9d98078b90 0x7f9d9807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:28.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:28.648+0000 7f9daaffd640 1 -- 172.21.15.38:0/2778600867 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5249+0+0 (secure 0 0 0) 0x7f9da409add0 con 0x7f9db4162280 2024-09-21T03:34:28.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:28.648+0000 7f9dba03e640 1 --2- 172.21.15.38:0/2778600867 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9d98078b90 0x7f9d9807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:28.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:28.649+0000 7f9dba03e640 1 --2- 172.21.15.38:0/2778600867 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9d98078b90 0x7f9d9807b0a0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f9db0033000 tx=0x7f9db0004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:28.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:28.653+0000 7f9daaffd640 1 -- 172.21.15.38:0/2778600867 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f9db40b29b0 con 0x7f9db4162280 2024-09-21T03:34:28.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:28.911+0000 7f9dbc2a2640 1 -- 172.21.15.38:0/2778600867 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f9db40b51f0 con 0x7f9db4162280 2024-09-21T03:34:28.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:28.911+0000 7f9daaffd640 1 -- 172.21.15.38:0/2778600867 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v11) ==== 80+0+6645 (secure 0 0 0) 0x7f9da40633d0 con 0x7f9db4162280 2024-09-21T03:34:28.913 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:34:28.913 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-09-21T03:34:28.913 INFO:teuthology.orchestra.run.smithi038.stdout: "auth_dump": [ 2024-09-21T03:34:28.913 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-21T03:34:28.914 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "mds.a", 2024-09-21T03:34:28.914 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQBPPu5mfzGIMRAAmq0TaC3J1VU3GxzhTgJ/Zw==", 2024-09-21T03:34:28.914 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-09-21T03:34:28.914 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow", 2024-09-21T03:34:28.914 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow *", 2024-09-21T03:34:28.914 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow *", 2024-09-21T03:34:28.914 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-09-21T03:34:28.914 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-21T03:34:28.914 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-21T03:34:28.914 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-21T03:34:28.914 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "mds.b", 2024-09-21T03:34:28.914 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQBQPu5m/fzuAhAApyigRlQ8OrV/sf0ap6Fn6A==", 2024-09-21T03:34:28.915 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-09-21T03:34:28.915 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow", 2024-09-21T03:34:28.915 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow *", 2024-09-21T03:34:28.915 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow *", 2024-09-21T03:34:28.915 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-09-21T03:34:28.915 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-21T03:34:28.915 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-21T03:34:28.915 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-21T03:34:28.915 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "mds.c", 2024-09-21T03:34:28.915 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQBPPu5mQipIORAArRn8k8SvvpG8x03F9I2/gQ==", 2024-09-21T03:34:28.915 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-09-21T03:34:28.915 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow", 2024-09-21T03:34:28.916 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow *", 2024-09-21T03:34:28.916 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow *", 2024-09-21T03:34:28.916 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-09-21T03:34:28.916 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-21T03:34:28.916 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-21T03:34:28.916 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-21T03:34:28.916 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "mds.d", 2024-09-21T03:34:28.916 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQBQPu5mzlyDChAAqbPKXS3EtnOqqeHhA1ehIw==", 2024-09-21T03:34:28.916 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-09-21T03:34:28.916 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow", 2024-09-21T03:34:28.916 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow *", 2024-09-21T03:34:28.916 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow *", 2024-09-21T03:34:28.916 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-09-21T03:34:28.917 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-21T03:34:28.917 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-21T03:34:28.917 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-21T03:34:28.917 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "osd.0", 2024-09-21T03:34:28.917 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQBXPu5mMZggAhAAtMPdJipheizZhpby9APj4A==", 2024-09-21T03:34:28.917 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-09-21T03:34:28.917 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow profile osd", 2024-09-21T03:34:28.917 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile osd", 2024-09-21T03:34:28.917 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-09-21T03:34:28.917 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-21T03:34:28.917 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-21T03:34:28.917 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-21T03:34:28.917 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "osd.1", 2024-09-21T03:34:28.918 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQBaPu5mm9TFIhAAWPJCvBYMgMbKr7Z7Diq58A==", 2024-09-21T03:34:28.918 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-09-21T03:34:28.918 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow profile osd", 2024-09-21T03:34:28.918 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile osd", 2024-09-21T03:34:28.918 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-09-21T03:34:28.918 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-21T03:34:28.918 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-21T03:34:28.918 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-21T03:34:28.918 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "osd.2", 2024-09-21T03:34:28.918 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQBePu5mqjljCBAAcdEpzJq6D8e1L+q1s0GjWQ==", 2024-09-21T03:34:28.918 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-09-21T03:34:28.918 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow profile osd", 2024-09-21T03:34:28.919 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile osd", 2024-09-21T03:34:28.919 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-09-21T03:34:28.919 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-21T03:34:28.919 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-21T03:34:28.919 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-21T03:34:28.919 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "osd.3", 2024-09-21T03:34:28.919 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQBhPu5mRjfdKBAAJ6+PlEiRp/xvxny6SjHV8Q==", 2024-09-21T03:34:28.919 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-09-21T03:34:28.919 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow profile osd", 2024-09-21T03:34:28.919 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile osd", 2024-09-21T03:34:28.919 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-09-21T03:34:28.920 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-21T03:34:28.920 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-21T03:34:28.920 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-21T03:34:28.920 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "osd.4", 2024-09-21T03:34:28.920 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQBlPu5m8bNyNxAA2rERruGojEDKmcXN2K6uog==", 2024-09-21T03:34:28.920 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-09-21T03:34:28.920 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow profile osd", 2024-09-21T03:34:28.920 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile osd", 2024-09-21T03:34:28.920 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-09-21T03:34:28.920 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-21T03:34:28.920 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-21T03:34:28.920 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-21T03:34:28.920 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "osd.5", 2024-09-21T03:34:28.921 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQBpPu5mBB2iDBAAvsvhAPMvNWII2Zr9Je45YQ==", 2024-09-21T03:34:28.921 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-09-21T03:34:28.921 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow profile osd", 2024-09-21T03:34:28.921 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile osd", 2024-09-21T03:34:28.921 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-09-21T03:34:28.921 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-21T03:34:28.921 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-21T03:34:28.921 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-21T03:34:28.921 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "osd.6", 2024-09-21T03:34:28.921 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQBsPu5mMQMaLRAAznzYdoNGHWOI2p2mNN/rQw==", 2024-09-21T03:34:28.921 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-09-21T03:34:28.921 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow profile osd", 2024-09-21T03:34:28.921 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile osd", 2024-09-21T03:34:28.922 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-09-21T03:34:28.922 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-21T03:34:28.922 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-21T03:34:28.922 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-21T03:34:28.922 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "osd.7", 2024-09-21T03:34:28.922 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQBwPu5mHu0WERAAir8bmtuNOOj/2D4hLaoYOA==", 2024-09-21T03:34:28.922 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-09-21T03:34:28.922 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow profile osd", 2024-09-21T03:34:28.922 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile osd", 2024-09-21T03:34:28.922 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-09-21T03:34:28.922 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-21T03:34:28.922 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-21T03:34:28.922 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-21T03:34:28.922 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "client.0", 2024-09-21T03:34:28.923 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQBQPu5m6ydBDxAAlWZ2XzQTPBcYPriPGRJVeA==", 2024-09-21T03:34:28.923 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-09-21T03:34:28.923 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow", 2024-09-21T03:34:28.923 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow r", 2024-09-21T03:34:28.923 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow rw", 2024-09-21T03:34:28.923 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow rwx" 2024-09-21T03:34:28.923 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-21T03:34:28.923 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-21T03:34:28.923 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-21T03:34:28.923 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "client.1", 2024-09-21T03:34:28.923 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQBQPu5mlWP0FhAAkNwfaJ6rPr3r4ZRVbPxX8g==", 2024-09-21T03:34:28.923 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-09-21T03:34:28.923 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow", 2024-09-21T03:34:28.924 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow r", 2024-09-21T03:34:28.924 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow rw", 2024-09-21T03:34:28.924 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow rwx" 2024-09-21T03:34:28.924 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-21T03:34:28.924 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-21T03:34:28.924 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-21T03:34:28.924 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "client.2", 2024-09-21T03:34:28.924 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQBQPu5m48oaExAAOzYrR/SsykpzH2TkWHs39Q==", 2024-09-21T03:34:28.924 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-09-21T03:34:28.924 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow", 2024-09-21T03:34:28.924 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow r", 2024-09-21T03:34:28.924 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow rw", 2024-09-21T03:34:28.924 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow rwx" 2024-09-21T03:34:28.925 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-21T03:34:28.925 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-21T03:34:28.925 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-21T03:34:28.925 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "client.3", 2024-09-21T03:34:28.925 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQBQPu5mVQSzGhAAAo6vizLc7GOnenQRsgtn1A==", 2024-09-21T03:34:28.925 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-09-21T03:34:28.925 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow", 2024-09-21T03:34:28.925 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow r", 2024-09-21T03:34:28.925 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow rw", 2024-09-21T03:34:28.925 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow rwx" 2024-09-21T03:34:28.925 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-21T03:34:28.925 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-21T03:34:28.925 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-21T03:34:28.925 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "client.admin", 2024-09-21T03:34:28.926 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQBPPu5mPHphExAA7D+/V7qhoNyoc6h0GnNMlQ==", 2024-09-21T03:34:28.926 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-09-21T03:34:28.926 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow *", 2024-09-21T03:34:28.926 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow *", 2024-09-21T03:34:28.926 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow *", 2024-09-21T03:34:28.926 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-09-21T03:34:28.926 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-21T03:34:28.926 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-21T03:34:28.926 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-21T03:34:28.926 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "client.bootstrap-mds", 2024-09-21T03:34:28.926 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQB2Pu5mhYtzKBAADJyp10nEW7YH2OCPcxVdaA==", 2024-09-21T03:34:28.926 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-09-21T03:34:28.926 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile bootstrap-mds" 2024-09-21T03:34:28.927 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-21T03:34:28.927 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-21T03:34:28.927 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-21T03:34:28.927 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "client.bootstrap-mgr", 2024-09-21T03:34:28.927 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQB2Pu5mc5VzKBAATCRAw+lAO68Ddyt+b9TjlQ==", 2024-09-21T03:34:28.927 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-09-21T03:34:28.927 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile bootstrap-mgr" 2024-09-21T03:34:28.927 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-21T03:34:28.927 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-21T03:34:28.927 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-21T03:34:28.927 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "client.bootstrap-osd", 2024-09-21T03:34:28.927 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQB2Pu5mzpxzKBAArrKp9l86NDqEn1Y6dQkacA==", 2024-09-21T03:34:28.927 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-09-21T03:34:28.928 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile bootstrap-osd" 2024-09-21T03:34:28.928 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-21T03:34:28.928 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-21T03:34:28.928 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-21T03:34:28.928 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "client.bootstrap-rbd", 2024-09-21T03:34:28.928 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQB2Pu5mDqRzKBAAWPlJgJoXDgUjguX2yTDQmA==", 2024-09-21T03:34:28.928 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-09-21T03:34:28.928 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile bootstrap-rbd" 2024-09-21T03:34:28.928 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-21T03:34:28.928 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-21T03:34:28.928 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-21T03:34:28.928 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-09-21T03:34:28.929 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQB2Pu5mm6tzKBAA79qdCStciTU9qCMaYGf2Hg==", 2024-09-21T03:34:28.929 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-09-21T03:34:28.929 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-09-21T03:34:28.929 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-21T03:34:28.929 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-21T03:34:28.929 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-21T03:34:28.929 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "client.bootstrap-rgw", 2024-09-21T03:34:28.929 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQB2Pu5mwbVzKBAAuOoYUOKp9X6f8RFHliVuwQ==", 2024-09-21T03:34:28.929 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-09-21T03:34:28.929 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile bootstrap-rgw" 2024-09-21T03:34:28.929 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-21T03:34:28.929 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-21T03:34:28.930 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-21T03:34:28.930 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "mgr.x", 2024-09-21T03:34:28.930 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQBPPu5m6K2oLRAAzldoBXAMSnWhxVFwPZkHOw==", 2024-09-21T03:34:28.930 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-09-21T03:34:28.930 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow *", 2024-09-21T03:34:28.930 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile mgr", 2024-09-21T03:34:28.930 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-09-21T03:34:28.930 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-21T03:34:28.930 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-21T03:34:28.930 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-21T03:34:28.930 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "mgr.y", 2024-09-21T03:34:28.930 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQBPPu5mWEz+KBAASx2KzDril3lD9cEawy56jA==", 2024-09-21T03:34:28.931 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-09-21T03:34:28.931 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow *", 2024-09-21T03:34:28.931 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile mgr", 2024-09-21T03:34:28.931 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-09-21T03:34:28.931 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-21T03:34:28.931 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-21T03:34:28.931 INFO:teuthology.orchestra.run.smithi038.stdout: ] 2024-09-21T03:34:28.931 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-09-21T03:34:28.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:28.913+0000 7f9dbc2a2640 1 -- 172.21.15.38:0/2778600867 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9d98078b90 msgr2=0x7f9d9807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:28.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:28.913+0000 7f9dbc2a2640 1 --2- 172.21.15.38:0/2778600867 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9d98078b90 0x7f9d9807b0a0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f9db0033000 tx=0x7f9db0004730 comp rx=0 tx=0).stop 2024-09-21T03:34:28.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:28.913+0000 7f9dbc2a2640 1 -- 172.21.15.38:0/2778600867 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9db4162280 msgr2=0x7f9db40acaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:28.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:28.913+0000 7f9dbc2a2640 1 --2- 172.21.15.38:0/2778600867 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9db4162280 0x7f9db40acaf0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f9da400d750 tx=0x7f9da400dc10 comp rx=0 tx=0).stop 2024-09-21T03:34:28.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:28.913+0000 7f9dbc2a2640 1 -- 172.21.15.38:0/2778600867 shutdown_connections 2024-09-21T03:34:28.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:28.913+0000 7f9dbc2a2640 1 --2- 172.21.15.38:0/2778600867 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9db40b2060 0x7f9db40b2470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:28.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:28.913+0000 7f9dbc2a2640 1 --2- 172.21.15.38:0/2778600867 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9db4162280 0x7f9db40acaf0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:28.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:28.913+0000 7f9dbc2a2640 1 --2- 172.21.15.38:0/2778600867 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9d98078b90 0x7f9d9807b0a0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:28.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:28.913+0000 7f9dbc2a2640 1 --2- 172.21.15.38:0/2778600867 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9db40b2e40 0x7f9db415b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:28.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:28.913+0000 7f9dbc2a2640 1 -- 172.21.15.38:0/2778600867 >> 172.21.15.38:0/2778600867 conn(0x7f9db40b6b20 msgr2=0x7f9db415dba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:28.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:28.913+0000 7f9dbc2a2640 1 -- 172.21.15.38:0/2778600867 shutdown_connections 2024-09-21T03:34:28.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:28.913+0000 7f9dbc2a2640 1 -- 172.21.15.38:0/2778600867 wait complete. 2024-09-21T03:34:28.933 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-09-21T03:34:29.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:29.049+0000 7f0ec80de640 1 -- 172.21.15.38:0/1540379560 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0ec00d1120 msgr2=0x7f0ec00ceb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:29.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:29.049+0000 7f0ec80de640 1 --2- 172.21.15.38:0/1540379560 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0ec00d1120 0x7f0ec00ceb90 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f0ec00d2360 tx=0x7f0eb002a9a0 comp rx=0 tx=0).stop 2024-09-21T03:34:29.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:29.049+0000 7f0ec80de640 1 -- 172.21.15.38:0/1540379560 shutdown_connections 2024-09-21T03:34:29.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:29.049+0000 7f0ec80de640 1 --2- 172.21.15.38:0/1540379560 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0ec00d1120 0x7f0ec00ceb90 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:29.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:29.049+0000 7f0ec80de640 1 --2- 172.21.15.38:0/1540379560 >> v2:172.21.15.195:3300/0 conn(0x7f0ec00d0170 0x7f0ec00d0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:29.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:29.049+0000 7f0ec80de640 1 -- 172.21.15.38:0/1540379560 >> 172.21.15.38:0/1540379560 conn(0x7f0ec009d830 msgr2=0x7f0ec00b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:29.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:29.049+0000 7f0ec80de640 1 -- 172.21.15.38:0/1540379560 shutdown_connections 2024-09-21T03:34:29.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:29.049+0000 7f0ec80de640 1 -- 172.21.15.38:0/1540379560 wait complete. 2024-09-21T03:34:29.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:29.051+0000 7f0ec80de640 1 Processor -- start 2024-09-21T03:34:29.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:29.051+0000 7f0ec80de640 1 -- start start 2024-09-21T03:34:29.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:29.052+0000 7f0ec80de640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0ec00d0170 0x7f0ec006ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:29.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:29.052+0000 7f0ec80de640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0ec00d1120 0x7f0ec01575a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:29.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:29.052+0000 7f0ec80de640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0ec015ee80 0x7f0ec00b2620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:29.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:29.052+0000 7f0ec80de640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f0ec00b6700 con 0x7f0ec00d1120 2024-09-21T03:34:29.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:29.052+0000 7f0ec80de640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f0ec00b6580 con 0x7f0ec00d0170 2024-09-21T03:34:29.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:29.052+0000 7f0ec80de640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f0ec00b6880 con 0x7f0ec015ee80 2024-09-21T03:34:29.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:29.052+0000 7f0ec5679640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0ec00d1120 0x7f0ec01575a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:29.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:29.052+0000 7f0ec5679640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0ec00d1120 0x7f0ec01575a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:52834/0 (socket says 172.21.15.38:52834) 2024-09-21T03:34:29.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:29.052+0000 7f0ec5679640 1 -- 172.21.15.38:0/2106333994 learned_addr learned my addr 172.21.15.38:0/2106333994 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:34:29.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:29.052+0000 7f0ec667b640 1 --2- 172.21.15.38:0/2106333994 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0ec015ee80 0x7f0ec00b2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:29.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:29.052+0000 7f0ec5e7a640 1 --2- 172.21.15.38:0/2106333994 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0ec00d0170 0x7f0ec006ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:29.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:29.053+0000 7f0ec5679640 1 -- 172.21.15.38:0/2106333994 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0ec015ee80 msgr2=0x7f0ec00b2620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:29.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:29.053+0000 7f0ec5679640 1 --2- 172.21.15.38:0/2106333994 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0ec015ee80 0x7f0ec00b2620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:29.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:29.053+0000 7f0ec5679640 1 -- 172.21.15.38:0/2106333994 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0ec00d0170 msgr2=0x7f0ec006ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:29.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:29.053+0000 7f0ec5679640 1 --2- 172.21.15.38:0/2106333994 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0ec00d0170 0x7f0ec006ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:29.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:29.053+0000 7f0ec5679640 1 -- 172.21.15.38:0/2106333994 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ec006f310 con 0x7f0ec00d1120 2024-09-21T03:34:29.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:29.053+0000 7f0ec5679640 1 --2- 172.21.15.38:0/2106333994 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0ec00d1120 0x7f0ec01575a0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f0eb400ea20 tx=0x7f0eb400eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:29.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:29.053+0000 7f0ebe7fc640 1 -- 172.21.15.38:0/2106333994 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eb400ccc0 con 0x7f0ec00d1120 2024-09-21T03:34:29.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:29.053+0000 7f0ebe7fc640 1 -- 172.21.15.38:0/2106333994 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0eb400ce60 con 0x7f0ec00d1120 2024-09-21T03:34:29.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:29.053+0000 7f0ec80de640 1 -- 172.21.15.38:0/2106333994 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ec01f5ce0 con 0x7f0ec00d1120 2024-09-21T03:34:29.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:29.053+0000 7f0ebe7fc640 1 -- 172.21.15.38:0/2106333994 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eb4004280 con 0x7f0ec00d1120 2024-09-21T03:34:29.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:29.053+0000 7f0ec80de640 1 -- 172.21.15.38:0/2106333994 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ec01f62a0 con 0x7f0ec00d1120 2024-09-21T03:34:29.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:29.054+0000 7f0ec80de640 1 -- 172.21.15.38:0/2106333994 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ec006f080 con 0x7f0ec00d1120 2024-09-21T03:34:29.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:29.055+0000 7f0ebe7fc640 1 -- 172.21.15.38:0/2106333994 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f0eb40035f0 con 0x7f0ec00d1120 2024-09-21T03:34:29.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:29.056+0000 7f0ebe7fc640 1 --2- 172.21.15.38:0/2106333994 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0eac078b90 0x7f0eac07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:29.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:29.056+0000 7f0ebe7fc640 1 -- 172.21.15.38:0/2106333994 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5249+0+0 (secure 0 0 0) 0x7f0eb409b860 con 0x7f0ec00d1120 2024-09-21T03:34:29.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:29.056+0000 7f0ec5e7a640 1 --2- 172.21.15.38:0/2106333994 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0eac078b90 0x7f0eac07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:29.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:29.057+0000 7f0ec5e7a640 1 --2- 172.21.15.38:0/2106333994 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0eac078b90 0x7f0eac07b0a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f0eb80015e0 tx=0x7f0eb8009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:29.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:29.060+0000 7f0ebe7fc640 1 -- 172.21.15.38:0/2106333994 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f0eb4063dc0 con 0x7f0ec00d1120 2024-09-21T03:34:29.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:29.328+0000 7f0ec80de640 1 -- 172.21.15.38:0/2106333994 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) -- 0x7f0ec01f5ad0 con 0x7f0ec00d1120 2024-09-21T03:34:29.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:29.331+0000 7f0ebe7fc640 1 -- 172.21.15.38:0/2106333994 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v12) ==== 90+0+0 (secure 0 0 0) 0x7f0eb40636c0 con 0x7f0ec00d1120 2024-09-21T03:34:29.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:29.333+0000 7f0ec80de640 1 -- 172.21.15.38:0/2106333994 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0eac078b90 msgr2=0x7f0eac07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:29.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:29.333+0000 7f0ec80de640 1 --2- 172.21.15.38:0/2106333994 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0eac078b90 0x7f0eac07b0a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f0eb80015e0 tx=0x7f0eb8009040 comp rx=0 tx=0).stop 2024-09-21T03:34:29.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:29.333+0000 7f0ec80de640 1 -- 172.21.15.38:0/2106333994 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0ec00d1120 msgr2=0x7f0ec01575a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:29.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:29.333+0000 7f0ec80de640 1 --2- 172.21.15.38:0/2106333994 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0ec00d1120 0x7f0ec01575a0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f0eb400ea20 tx=0x7f0eb400eee0 comp rx=0 tx=0).stop 2024-09-21T03:34:29.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:29.333+0000 7f0ec80de640 1 -- 172.21.15.38:0/2106333994 shutdown_connections 2024-09-21T03:34:29.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:29.333+0000 7f0ec80de640 1 --2- 172.21.15.38:0/2106333994 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0eac078b90 0x7f0eac07b0a0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:29.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:29.333+0000 7f0ec80de640 1 --2- 172.21.15.38:0/2106333994 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0ec015ee80 0x7f0ec00b2620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:29.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:29.333+0000 7f0ec80de640 1 --2- 172.21.15.38:0/2106333994 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0ec00d1120 0x7f0ec01575a0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:29.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:29.333+0000 7f0ec80de640 1 --2- 172.21.15.38:0/2106333994 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0ec00d0170 0x7f0ec006ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:29.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:29.333+0000 7f0ec80de640 1 -- 172.21.15.38:0/2106333994 >> 172.21.15.38:0/2106333994 conn(0x7f0ec009d830 msgr2=0x7f0ec0160950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:29.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:29.333+0000 7f0ec80de640 1 -- 172.21.15.38:0/2106333994 shutdown_connections 2024-09-21T03:34:29.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:29.333+0000 7f0ec80de640 1 -- 172.21.15.38:0/2106333994 wait complete. 2024-09-21T03:34:29.346 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-09-21T03:34:29.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:29.461+0000 7ffb1b0d3640 1 -- 172.21.15.38:0/3831119112 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffb140b5670 msgr2=0x7ffb1415f630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:29.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:29.461+0000 7ffb1b0d3640 1 --2- 172.21.15.38:0/3831119112 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffb140b5670 0x7ffb1415f630 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7ffb14073190 tx=0x7ffb0c02a560 comp rx=0 tx=0).stop 2024-09-21T03:34:29.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:29.462+0000 7ffb1b0d3640 1 -- 172.21.15.38:0/3831119112 shutdown_connections 2024-09-21T03:34:29.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:29.462+0000 7ffb1b0d3640 1 --2- 172.21.15.38:0/3831119112 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffb140b5670 0x7ffb1415f630 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:29.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:29.462+0000 7ffb1b0d3640 1 --2- 172.21.15.38:0/3831119112 >> v2:172.21.15.195:3300/0 conn(0x7ffb1406def0 0x7ffb1406f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:29.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:29.462+0000 7ffb1b0d3640 1 -- 172.21.15.38:0/3831119112 >> 172.21.15.38:0/3831119112 conn(0x7ffb140cdca0 msgr2=0x7ffb140ce0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:29.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:29.462+0000 7ffb1b0d3640 1 -- 172.21.15.38:0/3831119112 shutdown_connections 2024-09-21T03:34:29.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:29.462+0000 7ffb1b0d3640 1 -- 172.21.15.38:0/3831119112 wait complete. 2024-09-21T03:34:29.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:29.463+0000 7ffb1b0d3640 1 Processor -- start 2024-09-21T03:34:29.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:29.463+0000 7ffb1b0d3640 1 -- start start 2024-09-21T03:34:29.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:29.465+0000 7ffb1b0d3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffb1406def0 0x7ffb140ca300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:29.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:29.465+0000 7ffb1b0d3640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffb140b5260 0x7ffb140b2340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:29.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:29.465+0000 7ffb1b0d3640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffb140b5670 0x7ffb1406cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:29.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:29.465+0000 7ffb1b0d3640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ffb1415ff00 con 0x7ffb1406def0 2024-09-21T03:34:29.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:29.465+0000 7ffb1b0d3640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ffb1415fd80 con 0x7ffb140b5260 2024-09-21T03:34:29.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:29.465+0000 7ffb1b0d3640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7ffb14071300 con 0x7ffb140b5670 2024-09-21T03:34:29.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:29.465+0000 7ffb18e6f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffb1406def0 0x7ffb140ca300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:29.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:29.465+0000 7ffb18e6f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffb1406def0 0x7ffb140ca300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:52854/0 (socket says 172.21.15.38:52854) 2024-09-21T03:34:29.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:29.465+0000 7ffb18e6f640 1 -- 172.21.15.38:0/2078524301 learned_addr learned my addr 172.21.15.38:0/2078524301 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:34:29.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:29.465+0000 7ffb19670640 1 --2- 172.21.15.38:0/2078524301 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffb140b5670 0x7ffb1406cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:29.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:29.465+0000 7ffb0bfff640 1 --2- 172.21.15.38:0/2078524301 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffb140b5260 0x7ffb140b2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:29.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:29.465+0000 7ffb18e6f640 1 -- 172.21.15.38:0/2078524301 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffb140b5670 msgr2=0x7ffb1406cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:29.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:29.465+0000 7ffb18e6f640 1 --2- 172.21.15.38:0/2078524301 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffb140b5670 0x7ffb1406cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:29.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:29.465+0000 7ffb18e6f640 1 -- 172.21.15.38:0/2078524301 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffb140b5260 msgr2=0x7ffb140b2340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:29.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:29.465+0000 7ffb18e6f640 1 --2- 172.21.15.38:0/2078524301 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffb140b5260 0x7ffb140b2340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:29.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:29.465+0000 7ffb18e6f640 1 -- 172.21.15.38:0/2078524301 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb14155c40 con 0x7ffb1406def0 2024-09-21T03:34:29.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:29.465+0000 7ffb0bfff640 1 --2- 172.21.15.38:0/2078524301 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffb140b5260 0x7ffb140b2340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:34:29.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:29.466+0000 7ffb18e6f640 1 --2- 172.21.15.38:0/2078524301 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffb1406def0 0x7ffb140ca300 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7ffb1000e9e0 tx=0x7ffb1000eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:29.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:29.466+0000 7ffb097fa640 1 -- 172.21.15.38:0/2078524301 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb1000cd50 con 0x7ffb1406def0 2024-09-21T03:34:29.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:29.466+0000 7ffb097fa640 1 -- 172.21.15.38:0/2078524301 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffb10011470 con 0x7ffb1406def0 2024-09-21T03:34:29.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:29.466+0000 7ffb1b0d3640 1 -- 172.21.15.38:0/2078524301 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb14156370 con 0x7ffb1406def0 2024-09-21T03:34:29.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:29.466+0000 7ffb097fa640 1 -- 172.21.15.38:0/2078524301 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb10011860 con 0x7ffb1406def0 2024-09-21T03:34:29.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:29.466+0000 7ffb1b0d3640 1 -- 172.21.15.38:0/2078524301 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb14156930 con 0x7ffb1406def0 2024-09-21T03:34:29.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:29.467+0000 7ffb1b0d3640 1 -- 172.21.15.38:0/2078524301 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb140af250 con 0x7ffb1406def0 2024-09-21T03:34:29.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:29.467+0000 7ffb097fa640 1 -- 172.21.15.38:0/2078524301 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ffb1001a400 con 0x7ffb1406def0 2024-09-21T03:34:29.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:29.468+0000 7ffb097fa640 1 --2- 172.21.15.38:0/2078524301 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ffb00078bc0 0x7ffb0007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:29.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:29.468+0000 7ffb097fa640 1 -- 172.21.15.38:0/2078524301 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5249+0+0 (secure 0 0 0) 0x7ffb1009b6a0 con 0x7ffb1406def0 2024-09-21T03:34:29.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:29.469+0000 7ffb0bfff640 1 --2- 172.21.15.38:0/2078524301 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ffb00078bc0 0x7ffb0007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:29.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:29.469+0000 7ffb0bfff640 1 --2- 172.21.15.38:0/2078524301 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ffb00078bc0 0x7ffb0007b0d0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ffb04006fd0 tx=0x7ffb04008040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:29.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:29.473+0000 7ffb097fa640 1 -- 172.21.15.38:0/2078524301 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7ffb140af250 con 0x7ffb1406def0 2024-09-21T03:34:29.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:29.735+0000 7ffb1b0d3640 1 -- 172.21.15.38:0/2078524301 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) -- 0x7ffb1406f410 con 0x7ffb1406def0 2024-09-21T03:34:29.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:29.741+0000 7ffb097fa640 1 -- 172.21.15.38:0/2078524301 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v13) ==== 90+0+0 (secure 0 0 0) 0x7ffb14155c40 con 0x7ffb1406def0 2024-09-21T03:34:29.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:29.743+0000 7ffb1b0d3640 1 -- 172.21.15.38:0/2078524301 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ffb00078bc0 msgr2=0x7ffb0007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:29.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:29.743+0000 7ffb1b0d3640 1 --2- 172.21.15.38:0/2078524301 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ffb00078bc0 0x7ffb0007b0d0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ffb04006fd0 tx=0x7ffb04008040 comp rx=0 tx=0).stop 2024-09-21T03:34:29.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:29.743+0000 7ffb1b0d3640 1 -- 172.21.15.38:0/2078524301 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffb1406def0 msgr2=0x7ffb140ca300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:29.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:29.743+0000 7ffb1b0d3640 1 --2- 172.21.15.38:0/2078524301 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffb1406def0 0x7ffb140ca300 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7ffb1000e9e0 tx=0x7ffb1000eea0 comp rx=0 tx=0).stop 2024-09-21T03:34:29.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:29.744+0000 7ffb1b0d3640 1 -- 172.21.15.38:0/2078524301 shutdown_connections 2024-09-21T03:34:29.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:29.744+0000 7ffb1b0d3640 1 --2- 172.21.15.38:0/2078524301 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ffb00078bc0 0x7ffb0007b0d0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:29.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:29.744+0000 7ffb1b0d3640 1 --2- 172.21.15.38:0/2078524301 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffb140b5670 0x7ffb1406cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:29.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:29.744+0000 7ffb1b0d3640 1 --2- 172.21.15.38:0/2078524301 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffb140b5260 0x7ffb140b2340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:29.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:29.744+0000 7ffb1b0d3640 1 --2- 172.21.15.38:0/2078524301 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffb1406def0 0x7ffb140ca300 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:29.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:29.744+0000 7ffb1b0d3640 1 -- 172.21.15.38:0/2078524301 >> 172.21.15.38:0/2078524301 conn(0x7ffb140cdca0 msgr2=0x7ffb140ce0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:29.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:29.744+0000 7ffb1b0d3640 1 -- 172.21.15.38:0/2078524301 shutdown_connections 2024-09-21T03:34:29.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:29.744+0000 7ffb1b0d3640 1 -- 172.21.15.38:0/2078524301 wait complete. 2024-09-21T03:34:29.757 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-09-21T03:34:29.774 DEBUG:teuthology.orchestra.run.smithi038:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T03:34:29.802 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T03:34:29.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:29.873+0000 7f9fc8ab2640 1 -- 172.21.15.38:0/4119994585 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9fc40d0170 msgr2=0x7f9fc40d0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:29.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:29.873+0000 7f9fc8ab2640 1 --2- 172.21.15.38:0/4119994585 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9fc40d0170 0x7f9fc40d0be0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f9fc40d2080 tx=0x7f9fb802a410 comp rx=0 tx=0).stop 2024-09-21T03:34:29.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:29.873+0000 7f9fc8ab2640 1 -- 172.21.15.38:0/4119994585 shutdown_connections 2024-09-21T03:34:29.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:29.873+0000 7f9fc8ab2640 1 --2- 172.21.15.38:0/4119994585 >> v2:172.21.15.195:3300/0 conn(0x7f9fc40d1120 0x7f9fc40ceb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:29.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:29.873+0000 7f9fc8ab2640 1 --2- 172.21.15.38:0/4119994585 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9fc40d0170 0x7f9fc40d0be0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:29.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:29.873+0000 7f9fc8ab2640 1 -- 172.21.15.38:0/4119994585 >> 172.21.15.38:0/4119994585 conn(0x7f9fc409d830 msgr2=0x7f9fc40b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:29.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:29.873+0000 7f9fc8ab2640 1 -- 172.21.15.38:0/4119994585 shutdown_connections 2024-09-21T03:34:29.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:29.873+0000 7f9fc8ab2640 1 -- 172.21.15.38:0/4119994585 wait complete. 2024-09-21T03:34:29.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:29.875+0000 7f9fc8ab2640 1 Processor -- start 2024-09-21T03:34:29.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:29.875+0000 7f9fc8ab2640 1 -- start start 2024-09-21T03:34:29.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:29.876+0000 7f9fc8ab2640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9fc40d0170 0x7f9fc4157da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:29.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:29.876+0000 7f9fc8ab2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9fc40d1120 0x7f9fc40d3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:29.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:29.876+0000 7f9fc8ab2640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9fc415ee80 0x7f9fc40d3a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:29.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:29.876+0000 7f9fc8ab2640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f9fc40b6700 con 0x7f9fc40d1120 2024-09-21T03:34:29.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:29.876+0000 7f9fc8ab2640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f9fc40b6580 con 0x7f9fc40d0170 2024-09-21T03:34:29.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:29.876+0000 7f9fc8ab2640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f9fc40b6880 con 0x7f9fc415ee80 2024-09-21T03:34:29.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:29.876+0000 7f9fc1d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9fc40d1120 0x7f9fc40d3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:29.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:29.876+0000 7f9fc1d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9fc40d1120 0x7f9fc40d3550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:52872/0 (socket says 172.21.15.38:52872) 2024-09-21T03:34:29.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:29.876+0000 7f9fc1d9b640 1 -- 172.21.15.38:0/3808462065 learned_addr learned my addr 172.21.15.38:0/3808462065 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:34:29.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:29.876+0000 7f9fc259c640 1 --2- 172.21.15.38:0/3808462065 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9fc40d0170 0x7f9fc4157da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:29.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:29.876+0000 7f9fc2d9d640 1 --2- 172.21.15.38:0/3808462065 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9fc415ee80 0x7f9fc40d3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:29.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:29.877+0000 7f9fc1d9b640 1 -- 172.21.15.38:0/3808462065 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9fc415ee80 msgr2=0x7f9fc40d3a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:29.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:29.877+0000 7f9fc1d9b640 1 --2- 172.21.15.38:0/3808462065 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9fc415ee80 0x7f9fc40d3a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:29.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:29.877+0000 7f9fc1d9b640 1 -- 172.21.15.38:0/3808462065 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9fc40d0170 msgr2=0x7f9fc4157da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:29.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:29.877+0000 7f9fc1d9b640 1 --2- 172.21.15.38:0/3808462065 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9fc40d0170 0x7f9fc4157da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:29.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:29.877+0000 7f9fc1d9b640 1 -- 172.21.15.38:0/3808462065 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fc40d42d0 con 0x7f9fc40d1120 2024-09-21T03:34:29.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:29.877+0000 7f9fc259c640 1 --2- 172.21.15.38:0/3808462065 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9fc40d0170 0x7f9fc4157da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:34:29.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:29.877+0000 7f9fc1d9b640 1 --2- 172.21.15.38:0/3808462065 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9fc40d1120 0x7f9fc40d3550 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f9fb802aa00 tx=0x7f9fb8004a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:29.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:29.877+0000 7f9fb2ffd640 1 -- 172.21.15.38:0/3808462065 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fb8031b50 con 0x7f9fc40d1120 2024-09-21T03:34:29.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:29.877+0000 7f9fb2ffd640 1 -- 172.21.15.38:0/3808462065 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9fb8004e50 con 0x7f9fc40d1120 2024-09-21T03:34:29.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:29.877+0000 7f9fc8ab2640 1 -- 172.21.15.38:0/3808462065 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fc40d45b0 con 0x7f9fc40d1120 2024-09-21T03:34:29.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:29.877+0000 7f9fb2ffd640 1 -- 172.21.15.38:0/3808462065 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fb80330d0 con 0x7f9fc40d1120 2024-09-21T03:34:29.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:29.877+0000 7f9fc8ab2640 1 -- 172.21.15.38:0/3808462065 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fc40d4b70 con 0x7f9fc40d1120 2024-09-21T03:34:29.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:29.878+0000 7f9fc8ab2640 1 -- 172.21.15.38:0/3808462065 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9fc40b6da0 con 0x7f9fc40d1120 2024-09-21T03:34:29.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:29.879+0000 7f9fb2ffd640 1 -- 172.21.15.38:0/3808462065 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f9fb8033270 con 0x7f9fc40d1120 2024-09-21T03:34:29.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:29.879+0000 7f9fb2ffd640 1 --2- 172.21.15.38:0/3808462065 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9fa4078bc0 0x7f9fa407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:29.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:29.879+0000 7f9fb2ffd640 1 -- 172.21.15.38:0/3808462065 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5249+0+0 (secure 0 0 0) 0x7f9fb80bc0b0 con 0x7f9fc40d1120 2024-09-21T03:34:29.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:29.879+0000 7f9fc259c640 1 --2- 172.21.15.38:0/3808462065 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9fa4078bc0 0x7f9fa407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:29.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:29.880+0000 7f9fc259c640 1 --2- 172.21.15.38:0/3808462065 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9fa4078bc0 0x7f9fa407b0d0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f9fb4000f60 tx=0x7f9fb4009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:29.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:29.884+0000 7f9fb2ffd640 1 -- 172.21.15.38:0/3808462065 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f9fb803f030 con 0x7f9fc40d1120 2024-09-21T03:34:30.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:30.144+0000 7f9fc8ab2640 1 -- 172.21.15.38:0/3808462065 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) -- 0x7f9fc41ea3e0 con 0x7f9fc40d1120 2024-09-21T03:34:30.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:30.147+0000 7f9fb2ffd640 1 -- 172.21.15.38:0/3808462065 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v14) ==== 90+0+0 (secure 0 0 0) 0x7f9fb80846c0 con 0x7f9fc40d1120 2024-09-21T03:34:30.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:30.149+0000 7f9fc8ab2640 1 -- 172.21.15.38:0/3808462065 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9fa4078bc0 msgr2=0x7f9fa407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:30.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:30.149+0000 7f9fc8ab2640 1 --2- 172.21.15.38:0/3808462065 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9fa4078bc0 0x7f9fa407b0d0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f9fb4000f60 tx=0x7f9fb4009040 comp rx=0 tx=0).stop 2024-09-21T03:34:30.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:30.149+0000 7f9fc8ab2640 1 -- 172.21.15.38:0/3808462065 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9fc40d1120 msgr2=0x7f9fc40d3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:30.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:30.149+0000 7f9fc8ab2640 1 --2- 172.21.15.38:0/3808462065 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9fc40d1120 0x7f9fc40d3550 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f9fb802aa00 tx=0x7f9fb8004a50 comp rx=0 tx=0).stop 2024-09-21T03:34:30.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:30.149+0000 7f9fc8ab2640 1 -- 172.21.15.38:0/3808462065 shutdown_connections 2024-09-21T03:34:30.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:30.149+0000 7f9fc8ab2640 1 --2- 172.21.15.38:0/3808462065 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9fa4078bc0 0x7f9fa407b0d0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:30.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:30.149+0000 7f9fc8ab2640 1 --2- 172.21.15.38:0/3808462065 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9fc415ee80 0x7f9fc40d3a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:30.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:30.149+0000 7f9fc8ab2640 1 --2- 172.21.15.38:0/3808462065 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9fc40d1120 0x7f9fc40d3550 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:30.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:30.149+0000 7f9fc8ab2640 1 --2- 172.21.15.38:0/3808462065 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9fc40d0170 0x7f9fc4157da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:30.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:30.149+0000 7f9fc8ab2640 1 -- 172.21.15.38:0/3808462065 >> 172.21.15.38:0/3808462065 conn(0x7f9fc409d830 msgr2=0x7f9fc40b58f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:30.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:30.149+0000 7f9fc8ab2640 1 -- 172.21.15.38:0/3808462065 shutdown_connections 2024-09-21T03:34:30.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:30.149+0000 7f9fc8ab2640 1 -- 172.21.15.38:0/3808462065 wait complete. 2024-09-21T03:34:30.163 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-09-21T03:34:30.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:30.320+0000 7fc1407a9640 1 -- 172.21.15.38:0/3096778997 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc1200053a0 msgr2=0x7fc12000ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:30.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:30.320+0000 7fc1407a9640 1 --2- 172.21.15.38:0/3096778997 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc1200053a0 0x7fc12000ba50 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fc13000c3f0 tx=0x7fc120005830 comp rx=0 tx=0).stop 2024-09-21T03:34:30.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:30.320+0000 7fc1407a9640 1 -- 172.21.15.38:0/3096778997 shutdown_connections 2024-09-21T03:34:30.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:30.320+0000 7fc1407a9640 1 --2- 172.21.15.38:0/3096778997 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc1200053a0 0x7fc12000ba50 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:30.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:30.320+0000 7fc1407a9640 1 -- 172.21.15.38:0/3096778997 >> 172.21.15.38:0/3096778997 conn(0x7fc13809d830 msgr2=0x7fc1380b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:30.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:30.320+0000 7fc1407a9640 1 -- 172.21.15.38:0/3096778997 shutdown_connections 2024-09-21T03:34:30.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:30.320+0000 7fc1407a9640 1 -- 172.21.15.38:0/3096778997 wait complete. 2024-09-21T03:34:30.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:30.321+0000 7fc1407a9640 1 Processor -- start 2024-09-21T03:34:30.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:30.322+0000 7fc1407a9640 1 -- start start 2024-09-21T03:34:30.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:30.323+0000 7fc1407a9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc1200053a0 0x7fc138158a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:30.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:30.323+0000 7fc1407a9640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc1380cb9b0 0x7fc1380cbdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:30.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:30.323+0000 7fc1407a9640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc1380cc300 0x7fc1381f8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:30.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:30.323+0000 7fc1407a9640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fc1380b9c50 con 0x7fc1200053a0 2024-09-21T03:34:30.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:30.323+0000 7fc1407a9640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fc1380b9ad0 con 0x7fc1380cb9b0 2024-09-21T03:34:30.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:30.323+0000 7fc1407a9640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fc1380b9dd0 con 0x7fc1380cc300 2024-09-21T03:34:30.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:30.323+0000 7fc13e545640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc1200053a0 0x7fc138158a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:30.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:30.323+0000 7fc13e545640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc1200053a0 0x7fc138158a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:52888/0 (socket says 172.21.15.38:52888) 2024-09-21T03:34:30.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:30.323+0000 7fc13e545640 1 -- 172.21.15.38:0/3233594079 learned_addr learned my addr 172.21.15.38:0/3233594079 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:34:30.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:30.323+0000 7fc13dd44640 1 --2- 172.21.15.38:0/3233594079 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc1380cb9b0 0x7fc1380cbdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:30.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:30.323+0000 7fc13ed46640 1 --2- 172.21.15.38:0/3233594079 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc1380cc300 0x7fc1381f8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:30.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:30.323+0000 7fc13e545640 1 -- 172.21.15.38:0/3233594079 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc1380cc300 msgr2=0x7fc1381f8bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:30.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:30.323+0000 7fc13e545640 1 --2- 172.21.15.38:0/3233594079 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc1380cc300 0x7fc1381f8bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:30.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:30.323+0000 7fc13e545640 1 -- 172.21.15.38:0/3233594079 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc1380cb9b0 msgr2=0x7fc1380cbdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:30.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:30.323+0000 7fc13e545640 1 --2- 172.21.15.38:0/3233594079 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc1380cb9b0 0x7fc1380cbdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:30.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:30.323+0000 7fc13e545640 1 -- 172.21.15.38:0/3233594079 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc13806e730 con 0x7fc1200053a0 2024-09-21T03:34:30.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:30.323+0000 7fc13ed46640 1 --2- 172.21.15.38:0/3233594079 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc1380cc300 0x7fc1381f8bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:34:30.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:30.323+0000 7fc13dd44640 1 --2- 172.21.15.38:0/3233594079 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc1380cb9b0 0x7fc1380cbdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:34:30.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:30.323+0000 7fc13e545640 1 --2- 172.21.15.38:0/3233594079 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc1200053a0 0x7fc138158a70 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fc128003fd0 tx=0x7fc1280064f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:30.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:30.324+0000 7fc136ffd640 1 -- 172.21.15.38:0/3233594079 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc128015070 con 0x7fc1200053a0 2024-09-21T03:34:30.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:30.324+0000 7fc136ffd640 1 -- 172.21.15.38:0/3233594079 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc12800fa60 con 0x7fc1200053a0 2024-09-21T03:34:30.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:30.324+0000 7fc1407a9640 1 -- 172.21.15.38:0/3233594079 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc138158fb0 con 0x7fc1200053a0 2024-09-21T03:34:30.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:30.324+0000 7fc136ffd640 1 -- 172.21.15.38:0/3233594079 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc128011430 con 0x7fc1200053a0 2024-09-21T03:34:30.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:30.324+0000 7fc1407a9640 1 -- 172.21.15.38:0/3233594079 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc13806d970 con 0x7fc1200053a0 2024-09-21T03:34:30.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:30.325+0000 7fc1407a9640 1 -- 172.21.15.38:0/3233594079 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1380cec70 con 0x7fc1200053a0 2024-09-21T03:34:30.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:30.325+0000 7fc136ffd640 1 -- 172.21.15.38:0/3233594079 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc1280115d0 con 0x7fc1200053a0 2024-09-21T03:34:30.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:30.326+0000 7fc136ffd640 1 --2- 172.21.15.38:0/3233594079 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc124078b90 0x7fc12407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:30.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:30.326+0000 7fc136ffd640 1 -- 172.21.15.38:0/3233594079 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5249+0+0 (secure 0 0 0) 0x7fc12809ba40 con 0x7fc1200053a0 2024-09-21T03:34:30.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:30.326+0000 7fc13dd44640 1 --2- 172.21.15.38:0/3233594079 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc124078b90 0x7fc12407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:30.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:30.327+0000 7fc13dd44640 1 --2- 172.21.15.38:0/3233594079 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc124078b90 0x7fc12407b0a0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fc13806f470 tx=0x7fc12c005f30 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:30.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:30.330+0000 7fc136ffd640 1 -- 172.21.15.38:0/3233594079 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fc128064040 con 0x7fc1200053a0 2024-09-21T03:34:30.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:30.593+0000 7fc1407a9640 1 -- 172.21.15.38:0/3233594079 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) -- 0x7fc1381617e0 con 0x7fc1200053a0 2024-09-21T03:34:30.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:30.595+0000 7fc136ffd640 1 -- 172.21.15.38:0/3233594079 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v15) ==== 90+0+0 (secure 0 0 0) 0x7fc12809be20 con 0x7fc1200053a0 2024-09-21T03:34:30.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:30.597+0000 7fc1407a9640 1 -- 172.21.15.38:0/3233594079 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc124078b90 msgr2=0x7fc12407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:30.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:30.597+0000 7fc1407a9640 1 --2- 172.21.15.38:0/3233594079 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc124078b90 0x7fc12407b0a0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fc13806f470 tx=0x7fc12c005f30 comp rx=0 tx=0).stop 2024-09-21T03:34:30.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:30.597+0000 7fc1407a9640 1 -- 172.21.15.38:0/3233594079 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc1200053a0 msgr2=0x7fc138158a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:30.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:30.597+0000 7fc1407a9640 1 --2- 172.21.15.38:0/3233594079 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc1200053a0 0x7fc138158a70 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fc128003fd0 tx=0x7fc1280064f0 comp rx=0 tx=0).stop 2024-09-21T03:34:30.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:30.597+0000 7fc1407a9640 1 -- 172.21.15.38:0/3233594079 shutdown_connections 2024-09-21T03:34:30.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:30.597+0000 7fc1407a9640 1 --2- 172.21.15.38:0/3233594079 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc124078b90 0x7fc12407b0a0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:30.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:30.597+0000 7fc1407a9640 1 --2- 172.21.15.38:0/3233594079 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc1380cc300 0x7fc1381f8bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:30.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:30.597+0000 7fc1407a9640 1 --2- 172.21.15.38:0/3233594079 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc1380cb9b0 0x7fc1380cbdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:30.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:30.597+0000 7fc1407a9640 1 --2- 172.21.15.38:0/3233594079 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc1200053a0 0x7fc138158a70 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:30.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:30.597+0000 7fc1407a9640 1 -- 172.21.15.38:0/3233594079 >> 172.21.15.38:0/3233594079 conn(0x7fc13809d830 msgr2=0x7fc13815d880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:30.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:30.598+0000 7fc1407a9640 1 -- 172.21.15.38:0/3233594079 shutdown_connections 2024-09-21T03:34:30.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:30.598+0000 7fc1407a9640 1 -- 172.21.15.38:0/3233594079 wait complete. 2024-09-21T03:34:30.611 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-09-21T03:34:30.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:30.726+0000 7f4ca4d0b640 1 -- 172.21.15.38:0/402756470 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4ca015ee80 msgr2=0x7f4ca00d0170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:30.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:30.726+0000 7f4ca4d0b640 1 --2- 172.21.15.38:0/402756470 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4ca015ee80 0x7f4ca00d0170 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f4c90009830 tx=0x7f4c9002a470 comp rx=0 tx=0).stop 2024-09-21T03:34:30.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:30.727+0000 7f4ca4d0b640 1 -- 172.21.15.38:0/402756470 shutdown_connections 2024-09-21T03:34:30.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:30.727+0000 7f4ca4d0b640 1 --2- 172.21.15.38:0/402756470 >> v2:172.21.15.195:3300/0 conn(0x7f4ca00ceb90 0x7f4ca00cefa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:30.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:30.727+0000 7f4ca4d0b640 1 --2- 172.21.15.38:0/402756470 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4ca015ee80 0x7f4ca00d0170 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:30.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:30.727+0000 7f4ca4d0b640 1 -- 172.21.15.38:0/402756470 >> 172.21.15.38:0/402756470 conn(0x7f4ca009d830 msgr2=0x7f4ca00b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:30.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:30.727+0000 7f4ca4d0b640 1 -- 172.21.15.38:0/402756470 shutdown_connections 2024-09-21T03:34:30.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:30.727+0000 7f4ca4d0b640 1 -- 172.21.15.38:0/402756470 wait complete. 2024-09-21T03:34:30.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:30.729+0000 7f4ca4d0b640 1 Processor -- start 2024-09-21T03:34:30.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:30.729+0000 7f4ca4d0b640 1 -- start start 2024-09-21T03:34:30.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:30.730+0000 7f4ca4d0b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4ca00b9ad0 0x7f4ca0157c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:30.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:30.730+0000 7f4ca4d0b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4ca00ceb90 0x7f4ca009e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:30.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:30.730+0000 7f4ca4d0b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4ca015ee80 0x7f4ca009e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:30.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:30.730+0000 7f4ca4d0b640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f4ca00b6700 con 0x7f4ca00b9ad0 2024-09-21T03:34:30.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:30.730+0000 7f4ca4d0b640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f4ca00b6580 con 0x7f4ca00ceb90 2024-09-21T03:34:30.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:30.730+0000 7f4ca4d0b640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f4ca00b6880 con 0x7f4ca015ee80 2024-09-21T03:34:30.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:30.730+0000 7f4c9e59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4ca00b9ad0 0x7f4ca0157c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:30.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:30.730+0000 7f4c9e59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4ca00b9ad0 0x7f4ca0157c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:52896/0 (socket says 172.21.15.38:52896) 2024-09-21T03:34:30.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:30.730+0000 7f4c9e59c640 1 -- 172.21.15.38:0/1079019532 learned_addr learned my addr 172.21.15.38:0/1079019532 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:34:30.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:30.730+0000 7f4c9ed9d640 1 --2- 172.21.15.38:0/1079019532 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4ca015ee80 0x7f4ca009e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:30.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:30.730+0000 7f4c9dd9b640 1 --2- 172.21.15.38:0/1079019532 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4ca00ceb90 0x7f4ca009e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:30.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:30.731+0000 7f4c9e59c640 1 -- 172.21.15.38:0/1079019532 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4ca015ee80 msgr2=0x7f4ca009e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:30.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:30.731+0000 7f4c9e59c640 1 --2- 172.21.15.38:0/1079019532 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4ca015ee80 0x7f4ca009e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:30.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:30.731+0000 7f4c9e59c640 1 -- 172.21.15.38:0/1079019532 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4ca00ceb90 msgr2=0x7f4ca009e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:30.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:30.731+0000 7f4c9e59c640 1 --2- 172.21.15.38:0/1079019532 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4ca00ceb90 0x7f4ca009e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:30.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:30.731+0000 7f4c9e59c640 1 -- 172.21.15.38:0/1079019532 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ca00cdbd0 con 0x7f4ca00b9ad0 2024-09-21T03:34:30.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:30.731+0000 7f4c9e59c640 1 --2- 172.21.15.38:0/1079019532 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4ca00b9ad0 0x7f4ca0157c30 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f4c90009950 tx=0x7f4c90033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:30.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:30.731+0000 7f4c8effd640 1 -- 172.21.15.38:0/1079019532 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c90031d60 con 0x7f4ca00b9ad0 2024-09-21T03:34:30.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:30.731+0000 7f4c8effd640 1 -- 172.21.15.38:0/1079019532 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4c90033e10 con 0x7f4ca00b9ad0 2024-09-21T03:34:30.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:30.731+0000 7f4ca4d0b640 1 -- 172.21.15.38:0/1079019532 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ca006e200 con 0x7f4ca00b9ad0 2024-09-21T03:34:30.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:30.731+0000 7f4c8effd640 1 -- 172.21.15.38:0/1079019532 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c90005260 con 0x7f4ca00b9ad0 2024-09-21T03:34:30.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:30.731+0000 7f4ca4d0b640 1 -- 172.21.15.38:0/1079019532 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ca006e4a0 con 0x7f4ca00b9ad0 2024-09-21T03:34:30.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:30.732+0000 7f4ca4d0b640 1 -- 172.21.15.38:0/1079019532 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ca00cf660 con 0x7f4ca00b9ad0 2024-09-21T03:34:30.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:30.733+0000 7f4c8effd640 1 -- 172.21.15.38:0/1079019532 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f4c90005400 con 0x7f4ca00b9ad0 2024-09-21T03:34:30.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:30.734+0000 7f4c8effd640 1 --2- 172.21.15.38:0/1079019532 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4c80078b90 0x7f4c8007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:30.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:30.734+0000 7f4c8effd640 1 -- 172.21.15.38:0/1079019532 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5249+0+0 (secure 0 0 0) 0x7f4c900bbc60 con 0x7f4ca00b9ad0 2024-09-21T03:34:30.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:30.734+0000 7f4c9dd9b640 1 --2- 172.21.15.38:0/1079019532 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4c80078b90 0x7f4c8007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:30.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:30.734+0000 7f4c9dd9b640 1 --2- 172.21.15.38:0/1079019532 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4c80078b90 0x7f4c8007b0a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f4ca00cb120 tx=0x7f4c94009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:30.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:30.738+0000 7f4c8effd640 1 -- 172.21.15.38:0/1079019532 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f4c9003f030 con 0x7f4ca00b9ad0 2024-09-21T03:34:30.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:30.997+0000 7f4ca4d0b640 1 -- 172.21.15.38:0/1079019532 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) -- 0x7f4ca00b6da0 con 0x7f4ca00b9ad0 2024-09-21T03:34:31.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:31.000+0000 7f4c8effd640 1 -- 172.21.15.38:0/1079019532 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v16) ==== 97+0+0 (secure 0 0 0) 0x7f4c90083c90 con 0x7f4ca00b9ad0 2024-09-21T03:34:31.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:31.002+0000 7f4ca4d0b640 1 -- 172.21.15.38:0/1079019532 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4c80078b90 msgr2=0x7f4c8007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:31.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:31.002+0000 7f4ca4d0b640 1 --2- 172.21.15.38:0/1079019532 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4c80078b90 0x7f4c8007b0a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f4ca00cb120 tx=0x7f4c94009040 comp rx=0 tx=0).stop 2024-09-21T03:34:31.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:31.002+0000 7f4ca4d0b640 1 -- 172.21.15.38:0/1079019532 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4ca00b9ad0 msgr2=0x7f4ca0157c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:31.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:31.002+0000 7f4ca4d0b640 1 --2- 172.21.15.38:0/1079019532 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4ca00b9ad0 0x7f4ca0157c30 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f4c90009950 tx=0x7f4c90033c20 comp rx=0 tx=0).stop 2024-09-21T03:34:31.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:31.002+0000 7f4ca4d0b640 1 -- 172.21.15.38:0/1079019532 shutdown_connections 2024-09-21T03:34:31.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:31.002+0000 7f4ca4d0b640 1 --2- 172.21.15.38:0/1079019532 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4c80078b90 0x7f4c8007b0a0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:31.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:31.002+0000 7f4ca4d0b640 1 --2- 172.21.15.38:0/1079019532 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4ca015ee80 0x7f4ca009e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:31.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:31.002+0000 7f4ca4d0b640 1 --2- 172.21.15.38:0/1079019532 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4ca00ceb90 0x7f4ca009e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:31.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:31.002+0000 7f4ca4d0b640 1 --2- 172.21.15.38:0/1079019532 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4ca00b9ad0 0x7f4ca0157c30 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:31.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:31.002+0000 7f4ca4d0b640 1 -- 172.21.15.38:0/1079019532 >> 172.21.15.38:0/1079019532 conn(0x7f4ca009d830 msgr2=0x7f4ca00d32e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:31.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:31.002+0000 7f4ca4d0b640 1 -- 172.21.15.38:0/1079019532 shutdown_connections 2024-09-21T03:34:31.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:31.002+0000 7f4ca4d0b640 1 -- 172.21.15.38:0/1079019532 wait complete. 2024-09-21T03:34:31.015 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-09-21T03:34:31.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:31.131+0000 7f1f5303f640 1 -- 172.21.15.38:0/4094426141 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1f4c0ccc40 msgr2=0x7f1f4c15ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:31.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:31.131+0000 7f1f5303f640 1 --2- 172.21.15.38:0/4094426141 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1f4c0ccc40 0x7f1f4c15ca90 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f1f40005fd0 tx=0x7f1f4002a540 comp rx=0 tx=0).stop 2024-09-21T03:34:31.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:31.131+0000 7f1f5303f640 1 -- 172.21.15.38:0/4094426141 shutdown_connections 2024-09-21T03:34:31.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:31.131+0000 7f1f5303f640 1 --2- 172.21.15.38:0/4094426141 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1f4c0d0d80 0x7f1f4c0b5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:31.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:31.131+0000 7f1f5303f640 1 --2- 172.21.15.38:0/4094426141 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1f4c0ccc40 0x7f1f4c15ca90 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:31.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:31.131+0000 7f1f5303f640 1 -- 172.21.15.38:0/4094426141 >> 172.21.15.38:0/4094426141 conn(0x7f1f4c0b46e0 msgr2=0x7f1f4c0d23e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:31.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:31.131+0000 7f1f5303f640 1 -- 172.21.15.38:0/4094426141 shutdown_connections 2024-09-21T03:34:31.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:31.131+0000 7f1f5303f640 1 -- 172.21.15.38:0/4094426141 wait complete. 2024-09-21T03:34:31.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:31.133+0000 7f1f5303f640 1 Processor -- start 2024-09-21T03:34:31.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:31.133+0000 7f1f5303f640 1 -- start start 2024-09-21T03:34:31.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:31.135+0000 7f1f5303f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1f4c0ccc40 0x7f1f4c15ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:31.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:31.135+0000 7f1f5303f640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1f4c0cd440 0x7f1f4c15faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:31.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:31.135+0000 7f1f5303f640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1f4c15e2e0 0x7f1f4c1589a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:31.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:31.135+0000 7f1f5303f640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f1f4c0b6650 con 0x7f1f4c0ccc40 2024-09-21T03:34:31.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:31.135+0000 7f1f5303f640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f1f4c0b64d0 con 0x7f1f4c0cd440 2024-09-21T03:34:31.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:31.135+0000 7f1f5303f640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f1f4c0b67d0 con 0x7f1f4c15e2e0 2024-09-21T03:34:31.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:31.135+0000 7f1f50ddb640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1f4c0ccc40 0x7f1f4c15ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:31.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:31.135+0000 7f1f50ddb640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1f4c0ccc40 0x7f1f4c15ae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:52920/0 (socket says 172.21.15.38:52920) 2024-09-21T03:34:31.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:31.135+0000 7f1f50ddb640 1 -- 172.21.15.38:0/2592561616 learned_addr learned my addr 172.21.15.38:0/2592561616 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:34:31.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:31.135+0000 7f1f515dc640 1 --2- 172.21.15.38:0/2592561616 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1f4c15e2e0 0x7f1f4c1589a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:31.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:31.135+0000 7f1f4bfff640 1 --2- 172.21.15.38:0/2592561616 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1f4c0cd440 0x7f1f4c15faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:31.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:31.135+0000 7f1f50ddb640 1 -- 172.21.15.38:0/2592561616 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1f4c15e2e0 msgr2=0x7f1f4c1589a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:31.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:31.135+0000 7f1f50ddb640 1 --2- 172.21.15.38:0/2592561616 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1f4c15e2e0 0x7f1f4c1589a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:31.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:31.135+0000 7f1f50ddb640 1 -- 172.21.15.38:0/2592561616 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1f4c0cd440 msgr2=0x7f1f4c15faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:31.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:31.135+0000 7f1f50ddb640 1 --2- 172.21.15.38:0/2592561616 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1f4c0cd440 0x7f1f4c15faf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:31.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:31.135+0000 7f1f50ddb640 1 -- 172.21.15.38:0/2592561616 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f4c160030 con 0x7f1f4c0ccc40 2024-09-21T03:34:31.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:31.135+0000 7f1f50ddb640 1 --2- 172.21.15.38:0/2592561616 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1f4c0ccc40 0x7f1f4c15ae80 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f1f40034040 tx=0x7f1f400038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:31.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:31.135+0000 7f1f497fa640 1 -- 172.21.15.38:0/2592561616 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f4003e030 con 0x7f1f4c0ccc40 2024-09-21T03:34:31.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:31.135+0000 7f1f497fa640 1 -- 172.21.15.38:0/2592561616 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1f40003d90 con 0x7f1f4c0ccc40 2024-09-21T03:34:31.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:31.135+0000 7f1f497fa640 1 -- 172.21.15.38:0/2592561616 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f40035040 con 0x7f1f4c0ccc40 2024-09-21T03:34:31.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:31.135+0000 7f1f5303f640 1 -- 172.21.15.38:0/2592561616 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f4c15ea50 con 0x7f1f4c0ccc40 2024-09-21T03:34:31.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:31.136+0000 7f1f5303f640 1 -- 172.21.15.38:0/2592561616 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f4c15f010 con 0x7f1f4c0ccc40 2024-09-21T03:34:31.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:31.137+0000 7f1f5303f640 1 -- 172.21.15.38:0/2592561616 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f4c158780 con 0x7f1f4c0ccc40 2024-09-21T03:34:31.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:31.137+0000 7f1f497fa640 1 -- 172.21.15.38:0/2592561616 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f1f40041030 con 0x7f1f4c0ccc40 2024-09-21T03:34:31.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:31.138+0000 7f1f497fa640 1 --2- 172.21.15.38:0/2592561616 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1f30078b90 0x7f1f3007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:31.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:31.138+0000 7f1f497fa640 1 -- 172.21.15.38:0/2592561616 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5249+0+0 (secure 0 0 0) 0x7f1f400bb5b0 con 0x7f1f4c0ccc40 2024-09-21T03:34:31.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:31.138+0000 7f1f4bfff640 1 --2- 172.21.15.38:0/2592561616 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1f30078b90 0x7f1f3007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:31.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:31.139+0000 7f1f4bfff640 1 --2- 172.21.15.38:0/2592561616 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1f30078b90 0x7f1f3007b0a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f1f4c15ba00 tx=0x7f1f38009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:31.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:31.143+0000 7f1f497fa640 1 -- 172.21.15.38:0/2592561616 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f1f40083560 con 0x7f1f4c0ccc40 2024-09-21T03:34:31.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:31.406+0000 7f1f5303f640 1 -- 172.21.15.38:0/2592561616 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) -- 0x7f1f4c0717d0 con 0x7f1f4c0ccc40 2024-09-21T03:34:31.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:31.413+0000 7f1f497fa640 1 -- 172.21.15.38:0/2592561616 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v17) ==== 90+0+0 (secure 0 0 0) 0x7f1f40087f90 con 0x7f1f4c0ccc40 2024-09-21T03:34:31.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:31.415+0000 7f1f5303f640 1 -- 172.21.15.38:0/2592561616 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1f30078b90 msgr2=0x7f1f3007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:31.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:31.415+0000 7f1f5303f640 1 --2- 172.21.15.38:0/2592561616 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1f30078b90 0x7f1f3007b0a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f1f4c15ba00 tx=0x7f1f38009040 comp rx=0 tx=0).stop 2024-09-21T03:34:31.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:31.415+0000 7f1f5303f640 1 -- 172.21.15.38:0/2592561616 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1f4c0ccc40 msgr2=0x7f1f4c15ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:31.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:31.415+0000 7f1f5303f640 1 --2- 172.21.15.38:0/2592561616 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1f4c0ccc40 0x7f1f4c15ae80 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f1f40034040 tx=0x7f1f400038a0 comp rx=0 tx=0).stop 2024-09-21T03:34:31.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:31.415+0000 7f1f5303f640 1 -- 172.21.15.38:0/2592561616 shutdown_connections 2024-09-21T03:34:31.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:31.415+0000 7f1f5303f640 1 --2- 172.21.15.38:0/2592561616 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1f30078b90 0x7f1f3007b0a0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:31.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:31.415+0000 7f1f5303f640 1 --2- 172.21.15.38:0/2592561616 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1f4c15e2e0 0x7f1f4c1589a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:31.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:31.415+0000 7f1f5303f640 1 --2- 172.21.15.38:0/2592561616 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1f4c0cd440 0x7f1f4c15faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:31.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:31.415+0000 7f1f5303f640 1 --2- 172.21.15.38:0/2592561616 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1f4c0ccc40 0x7f1f4c15ae80 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:31.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:31.415+0000 7f1f5303f640 1 -- 172.21.15.38:0/2592561616 >> 172.21.15.38:0/2592561616 conn(0x7f1f4c0b46e0 msgr2=0x7f1f4c070ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:31.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:31.415+0000 7f1f5303f640 1 -- 172.21.15.38:0/2592561616 shutdown_connections 2024-09-21T03:34:31.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:31.415+0000 7f1f5303f640 1 -- 172.21.15.38:0/2592561616 wait complete. 2024-09-21T03:34:31.429 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-21T03:34:31.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:31.544+0000 7f548c13a640 1 -- 172.21.15.38:0/1053578677 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5484162f00 msgr2=0x7f54840d24d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:31.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:31.544+0000 7f548c13a640 1 --2- 172.21.15.38:0/1053578677 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5484162f00 0x7f54840d24d0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f5474005fd0 tx=0x7f547402aa90 comp rx=0 tx=0).stop 2024-09-21T03:34:31.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:31.544+0000 7f548c13a640 1 -- 172.21.15.38:0/1053578677 shutdown_connections 2024-09-21T03:34:31.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:31.544+0000 7f548c13a640 1 --2- 172.21.15.38:0/1053578677 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f54840b2e40 0x7f54840b3250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:31.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:31.544+0000 7f548c13a640 1 --2- 172.21.15.38:0/1053578677 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5484162f00 0x7f54840d24d0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:31.547 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:31.544+0000 7f548c13a640 1 -- 172.21.15.38:0/1053578677 >> 172.21.15.38:0/1053578677 conn(0x7f54840b6b20 msgr2=0x7f54840d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:31.547 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:31.544+0000 7f548c13a640 1 -- 172.21.15.38:0/1053578677 shutdown_connections 2024-09-21T03:34:31.547 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:31.544+0000 7f548c13a640 1 -- 172.21.15.38:0/1053578677 wait complete. 2024-09-21T03:34:31.547 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:31.546+0000 7f548c13a640 1 Processor -- start 2024-09-21T03:34:31.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:31.546+0000 7f548c13a640 1 -- start start 2024-09-21T03:34:31.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:31.547+0000 7f548c13a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f54840b2e40 0x7f548415a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:31.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:31.547+0000 7f548c13a640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5484162280 0x7f54840ce3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:31.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:31.547+0000 7f548c13a640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f54841f98f0 0x7f5484158260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:31.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:31.547+0000 7f548c13a640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f548415e630 con 0x7f54840b2e40 2024-09-21T03:34:31.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:31.547+0000 7f548c13a640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f548415e4b0 con 0x7f54841f98f0 2024-09-21T03:34:31.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:31.547+0000 7f548c13a640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f548415e7b0 con 0x7f5484162280 2024-09-21T03:34:31.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:31.547+0000 7f5489ed6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f54840b2e40 0x7f548415a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:31.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:31.547+0000 7f5489ed6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f54840b2e40 0x7f548415a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:52926/0 (socket says 172.21.15.38:52926) 2024-09-21T03:34:31.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:31.547+0000 7f5489ed6640 1 -- 172.21.15.38:0/2536588787 learned_addr learned my addr 172.21.15.38:0/2536588787 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:34:31.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:31.547+0000 7f54896d5640 1 --2- 172.21.15.38:0/2536588787 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5484162280 0x7f54840ce3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:31.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:31.547+0000 7f548a6d7640 1 --2- 172.21.15.38:0/2536588787 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f54841f98f0 0x7f5484158260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:31.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:31.548+0000 7f5489ed6640 1 -- 172.21.15.38:0/2536588787 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5484162280 msgr2=0x7f54840ce3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:31.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:31.548+0000 7f5489ed6640 1 --2- 172.21.15.38:0/2536588787 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5484162280 0x7f54840ce3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:31.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:31.548+0000 7f5489ed6640 1 -- 172.21.15.38:0/2536588787 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f54841f98f0 msgr2=0x7f5484158260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:31.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:31.548+0000 7f5489ed6640 1 --2- 172.21.15.38:0/2536588787 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f54841f98f0 0x7f5484158260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:31.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:31.548+0000 7f5489ed6640 1 -- 172.21.15.38:0/2536588787 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f548415aec0 con 0x7f54840b2e40 2024-09-21T03:34:31.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:31.548+0000 7f5489ed6640 1 --2- 172.21.15.38:0/2536588787 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f54840b2e40 0x7f548415a590 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f5474000f80 tx=0x7f5474003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:31.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:31.548+0000 7f54827fc640 1 -- 172.21.15.38:0/2536588787 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5474044070 con 0x7f54840b2e40 2024-09-21T03:34:31.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:31.548+0000 7f54827fc640 1 -- 172.21.15.38:0/2536588787 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5474004250 con 0x7f54840b2e40 2024-09-21T03:34:31.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:31.548+0000 7f548c13a640 1 -- 172.21.15.38:0/2536588787 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f548415b1a0 con 0x7f54840b2e40 2024-09-21T03:34:31.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:31.548+0000 7f54827fc640 1 -- 172.21.15.38:0/2536588787 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5474004660 con 0x7f54840b2e40 2024-09-21T03:34:31.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:31.548+0000 7f548c13a640 1 -- 172.21.15.38:0/2536588787 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54840aadc0 con 0x7f54840b2e40 2024-09-21T03:34:31.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:31.549+0000 7f548c13a640 1 -- 172.21.15.38:0/2536588787 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5484162f00 con 0x7f54840b2e40 2024-09-21T03:34:31.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:31.550+0000 7f54827fc640 1 -- 172.21.15.38:0/2536588787 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f5474047070 con 0x7f54840b2e40 2024-09-21T03:34:31.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:31.550+0000 7f54827fc640 1 --2- 172.21.15.38:0/2536588787 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5470078b90 0x7f547007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:31.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:31.551+0000 7f54827fc640 1 -- 172.21.15.38:0/2536588787 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5249+0+0 (secure 0 0 0) 0x7f54740bbb60 con 0x7f54840b2e40 2024-09-21T03:34:31.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:31.551+0000 7f54896d5640 1 --2- 172.21.15.38:0/2536588787 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5470078b90 0x7f547007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:31.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:31.551+0000 7f54896d5640 1 --2- 172.21.15.38:0/2536588787 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5470078b90 0x7f547007b0a0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f5484158210 tx=0x7f5478009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:31.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:31.555+0000 7f54827fc640 1 -- 172.21.15.38:0/2536588787 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f5474032030 con 0x7f54840b2e40 2024-09-21T03:34:31.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:31.744+0000 7f548c13a640 1 -- 172.21.15.38:0/2536588787 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f548415dc40 con 0x7f54840b2e40 2024-09-21T03:34:31.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:31.744+0000 7f54827fc640 1 -- 172.21.15.38:0/2536588787 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v36) ==== 84+0+65 (secure 0 0 0) 0x7f5474083b00 con 0x7f54840b2e40 2024-09-21T03:34:31.746 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:34:31.747 INFO:teuthology.orchestra.run.smithi038.stdout:[ 2024-09-21T03:34:31.747 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-21T03:34:31.747 INFO:teuthology.orchestra.run.smithi038.stdout: "poolnum": 1, 2024-09-21T03:34:31.747 INFO:teuthology.orchestra.run.smithi038.stdout: "poolname": ".mgr" 2024-09-21T03:34:31.747 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-21T03:34:31.747 INFO:teuthology.orchestra.run.smithi038.stdout:] 2024-09-21T03:34:31.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:31.747+0000 7f548c13a640 1 -- 172.21.15.38:0/2536588787 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5470078b90 msgr2=0x7f547007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:31.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:31.747+0000 7f548c13a640 1 --2- 172.21.15.38:0/2536588787 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5470078b90 0x7f547007b0a0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f5484158210 tx=0x7f5478009040 comp rx=0 tx=0).stop 2024-09-21T03:34:31.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:31.747+0000 7f548c13a640 1 -- 172.21.15.38:0/2536588787 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f54840b2e40 msgr2=0x7f548415a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:31.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:31.747+0000 7f548c13a640 1 --2- 172.21.15.38:0/2536588787 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f54840b2e40 0x7f548415a590 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f5474000f80 tx=0x7f5474003f10 comp rx=0 tx=0).stop 2024-09-21T03:34:31.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:31.747+0000 7f548c13a640 1 -- 172.21.15.38:0/2536588787 shutdown_connections 2024-09-21T03:34:31.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:31.747+0000 7f548c13a640 1 --2- 172.21.15.38:0/2536588787 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f54841f98f0 0x7f5484158260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:31.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:31.747+0000 7f548c13a640 1 --2- 172.21.15.38:0/2536588787 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5470078b90 0x7f547007b0a0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:31.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:31.747+0000 7f548c13a640 1 --2- 172.21.15.38:0/2536588787 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5484162280 0x7f54840ce3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:31.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:31.747+0000 7f548c13a640 1 --2- 172.21.15.38:0/2536588787 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f54840b2e40 0x7f548415a590 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:31.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:31.747+0000 7f548c13a640 1 -- 172.21.15.38:0/2536588787 >> 172.21.15.38:0/2536588787 conn(0x7f54840b6b20 msgr2=0x7f548406ddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:31.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:31.747+0000 7f548c13a640 1 -- 172.21.15.38:0/2536588787 shutdown_connections 2024-09-21T03:34:31.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:31.747+0000 7f548c13a640 1 -- 172.21.15.38:0/2536588787 wait complete. 2024-09-21T03:34:31.760 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-21T03:34:31.760 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-21T03:34:31.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:31.878+0000 7f59ceaec640 1 -- 172.21.15.38:0/1665309560 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f59c80cf910 msgr2=0x7f59c80b7f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:31.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:31.878+0000 7f59ceaec640 1 --2- 172.21.15.38:0/1665309560 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f59c80cf910 0x7f59c80b7f70 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f59c0005fd0 tx=0x7f59c002a5d0 comp rx=0 tx=0).stop 2024-09-21T03:34:31.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:31.879+0000 7f59ceaec640 1 -- 172.21.15.38:0/1665309560 shutdown_connections 2024-09-21T03:34:31.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:31.879+0000 7f59ceaec640 1 --2- 172.21.15.38:0/1665309560 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f59c80cf910 0x7f59c80b7f70 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:31.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:31.879+0000 7f59ceaec640 1 --2- 172.21.15.38:0/1665309560 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f59c80d0460 0x7f59c8161ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:31.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:31.879+0000 7f59ceaec640 1 -- 172.21.15.38:0/1665309560 >> 172.21.15.38:0/1665309560 conn(0x7f59c80d4510 msgr2=0x7f59c80d4920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:31.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:31.879+0000 7f59ceaec640 1 -- 172.21.15.38:0/1665309560 shutdown_connections 2024-09-21T03:34:31.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:31.879+0000 7f59ceaec640 1 -- 172.21.15.38:0/1665309560 wait complete. 2024-09-21T03:34:31.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:31.880+0000 7f59ceaec640 1 Processor -- start 2024-09-21T03:34:31.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:31.880+0000 7f59ceaec640 1 -- start start 2024-09-21T03:34:31.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:31.882+0000 7f59ceaec640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f59c80b7b60 0x7f59c81f5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:31.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:31.882+0000 7f59ceaec640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f59c80cf910 0x7f59c81e84c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:31.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:31.882+0000 7f59ceaec640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f59c80d0460 0x7f59c81e8a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:31.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:31.882+0000 7f59ceaec640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f59c8162ad0 con 0x7f59c80cf910 2024-09-21T03:34:31.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:31.882+0000 7f59ceaec640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f59c8162950 con 0x7f59c80d0460 2024-09-21T03:34:31.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:31.882+0000 7f59ceaec640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f59c8162c50 con 0x7f59c80b7b60 2024-09-21T03:34:31.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:31.882+0000 7f59c7fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f59c80cf910 0x7f59c81e84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:31.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:31.882+0000 7f59c7fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f59c80cf910 0x7f59c81e84c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:52942/0 (socket says 172.21.15.38:52942) 2024-09-21T03:34:31.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:31.882+0000 7f59c7fff640 1 -- 172.21.15.38:0/2800724954 learned_addr learned my addr 172.21.15.38:0/2800724954 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:34:31.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:31.882+0000 7f59cc888640 1 --2- 172.21.15.38:0/2800724954 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f59c80b7b60 0x7f59c81f5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:31.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:31.882+0000 7f59cd089640 1 --2- 172.21.15.38:0/2800724954 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f59c80d0460 0x7f59c81e8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:31.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:31.882+0000 7f59c7fff640 1 -- 172.21.15.38:0/2800724954 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f59c80b7b60 msgr2=0x7f59c81f5460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:31.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:31.882+0000 7f59c7fff640 1 --2- 172.21.15.38:0/2800724954 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f59c80b7b60 0x7f59c81f5460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:31.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:31.882+0000 7f59c7fff640 1 -- 172.21.15.38:0/2800724954 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f59c80d0460 msgr2=0x7f59c81e8a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:31.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:31.882+0000 7f59c7fff640 1 --2- 172.21.15.38:0/2800724954 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f59c80d0460 0x7f59c81e8a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:31.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:31.882+0000 7f59c7fff640 1 -- 172.21.15.38:0/2800724954 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59c81f9210 con 0x7f59c80cf910 2024-09-21T03:34:31.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:31.883+0000 7f59c7fff640 1 --2- 172.21.15.38:0/2800724954 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f59c80cf910 0x7f59c81e84c0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f59b000c400 tx=0x7f59b000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:31.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:31.883+0000 7f59c57fa640 1 -- 172.21.15.38:0/2800724954 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59b0017070 con 0x7f59c80cf910 2024-09-21T03:34:31.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:31.883+0000 7f59c57fa640 1 -- 172.21.15.38:0/2800724954 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f59b00053c0 con 0x7f59c80cf910 2024-09-21T03:34:31.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:31.883+0000 7f59ceaec640 1 -- 172.21.15.38:0/2800724954 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59c81f94f0 con 0x7f59c80cf910 2024-09-21T03:34:31.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:31.883+0000 7f59c57fa640 1 -- 172.21.15.38:0/2800724954 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59b0003830 con 0x7f59c80cf910 2024-09-21T03:34:31.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:31.883+0000 7f59ceaec640 1 -- 172.21.15.38:0/2800724954 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59c81f5aa0 con 0x7f59c80cf910 2024-09-21T03:34:31.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:31.884+0000 7f59ceaec640 1 -- 172.21.15.38:0/2800724954 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59c80b7f70 con 0x7f59c80cf910 2024-09-21T03:34:31.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:31.884+0000 7f59c57fa640 1 -- 172.21.15.38:0/2800724954 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f59b001a070 con 0x7f59c80cf910 2024-09-21T03:34:31.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:31.885+0000 7f59c57fa640 1 --2- 172.21.15.38:0/2800724954 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f59a8078b90 0x7f59a807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:31.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:31.885+0000 7f59c57fa640 1 -- 172.21.15.38:0/2800724954 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5249+0+0 (secure 0 0 0) 0x7f59b0099eb0 con 0x7f59c80cf910 2024-09-21T03:34:31.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:31.885+0000 7f59cc888640 1 --2- 172.21.15.38:0/2800724954 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f59a8078b90 0x7f59a807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:31.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:31.886+0000 7f59cc888640 1 --2- 172.21.15.38:0/2800724954 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f59a8078b90 0x7f59a807b0a0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f59b80099a0 tx=0x7f59b8009500 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:31.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:31.890+0000 7f59c57fa640 1 -- 172.21.15.38:0/2800724954 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f59b0061e60 con 0x7f59c80cf910 2024-09-21T03:34:32.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:32.076+0000 7f59ceaec640 1 -- 172.21.15.38:0/2800724954 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f59c80cfe20 con 0x7f59c80cf910 2024-09-21T03:34:32.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:32.603+0000 7f59c57fa640 1 -- 172.21.15.38:0/2800724954 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v37) ==== 138+0+0 (secure 0 0 0) 0x7f59b0066890 con 0x7f59c80cf910 2024-09-21T03:34:32.605 INFO:teuthology.orchestra.run.smithi038.stderr:pool 'cephfs_metadata' created 2024-09-21T03:34:32.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:32.606+0000 7f59ceaec640 1 -- 172.21.15.38:0/2800724954 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f59a8078b90 msgr2=0x7f59a807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:32.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:32.606+0000 7f59ceaec640 1 --2- 172.21.15.38:0/2800724954 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f59a8078b90 0x7f59a807b0a0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f59b80099a0 tx=0x7f59b8009500 comp rx=0 tx=0).stop 2024-09-21T03:34:32.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:32.606+0000 7f59ceaec640 1 -- 172.21.15.38:0/2800724954 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f59c80cf910 msgr2=0x7f59c81e84c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:32.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:32.606+0000 7f59ceaec640 1 --2- 172.21.15.38:0/2800724954 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f59c80cf910 0x7f59c81e84c0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f59b000c400 tx=0x7f59b000c8c0 comp rx=0 tx=0).stop 2024-09-21T03:34:32.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:32.606+0000 7f59ceaec640 1 -- 172.21.15.38:0/2800724954 shutdown_connections 2024-09-21T03:34:32.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:32.606+0000 7f59ceaec640 1 --2- 172.21.15.38:0/2800724954 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f59c80d0460 0x7f59c81e8a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:32.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:32.606+0000 7f59ceaec640 1 --2- 172.21.15.38:0/2800724954 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f59c80cf910 0x7f59c81e84c0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:32.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:32.606+0000 7f59ceaec640 1 --2- 172.21.15.38:0/2800724954 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f59a8078b90 0x7f59a807b0a0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:32.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:32.606+0000 7f59ceaec640 1 --2- 172.21.15.38:0/2800724954 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f59c80b7b60 0x7f59c81f5460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:32.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:32.606+0000 7f59ceaec640 1 -- 172.21.15.38:0/2800724954 >> 172.21.15.38:0/2800724954 conn(0x7f59c80d4510 msgr2=0x7f59c80b8560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:32.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:32.607+0000 7f59ceaec640 1 -- 172.21.15.38:0/2800724954 shutdown_connections 2024-09-21T03:34:32.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:32.607+0000 7f59ceaec640 1 -- 172.21.15.38:0/2800724954 wait complete. 2024-09-21T03:34:32.626 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-21T03:34:32.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:32.754+0000 7f5183e63640 1 -- 172.21.15.38:0/2529809892 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f517c0cea90 msgr2=0x7f517c0d00d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:32.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:32.754+0000 7f5183e63640 1 --2- 172.21.15.38:0/2529809892 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f517c0cea90 0x7f517c0d00d0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f517c0c7f40 tx=0x7f517802a4a0 comp rx=0 tx=0).stop 2024-09-21T03:34:32.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:32.754+0000 7f5183e63640 1 -- 172.21.15.38:0/2529809892 shutdown_connections 2024-09-21T03:34:32.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:32.754+0000 7f5183e63640 1 --2- 172.21.15.38:0/2529809892 >> v2:172.21.15.195:3300/0 conn(0x7f517c0d3360 0x7f517c0d3770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:32.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:32.754+0000 7f5183e63640 1 --2- 172.21.15.38:0/2529809892 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f517c0cea90 0x7f517c0d00d0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:32.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:32.754+0000 7f5183e63640 1 -- 172.21.15.38:0/2529809892 >> 172.21.15.38:0/2529809892 conn(0x7f517c0d4620 msgr2=0x7f517c0d4a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:32.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:32.754+0000 7f5183e63640 1 -- 172.21.15.38:0/2529809892 shutdown_connections 2024-09-21T03:34:32.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:32.754+0000 7f5183e63640 1 -- 172.21.15.38:0/2529809892 wait complete. 2024-09-21T03:34:32.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:32.756+0000 7f5183e63640 1 Processor -- start 2024-09-21T03:34:32.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:32.756+0000 7f5183e63640 1 -- start start 2024-09-21T03:34:32.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:32.757+0000 7f5183e63640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f517c0cea90 0x7f517c0cd810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:32.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:32.757+0000 7f5183e63640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f517c0d3360 0x7f517c157280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:32.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:32.757+0000 7f5183e63640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f517c162790 0x7f517c1577c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:32.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:32.757+0000 7f5183e63640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f517c0b4120 con 0x7f517c0d3360 2024-09-21T03:34:32.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:32.757+0000 7f5183e63640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f517c0b3fa0 con 0x7f517c0cea90 2024-09-21T03:34:32.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:32.757+0000 7f5183e63640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f517c0b42a0 con 0x7f517c162790 2024-09-21T03:34:32.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:32.757+0000 7f51813fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f517c0d3360 0x7f517c157280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:32.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:32.757+0000 7f51813fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f517c0d3360 0x7f517c157280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:33510/0 (socket says 172.21.15.38:33510) 2024-09-21T03:34:32.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:32.757+0000 7f51813fe640 1 -- 172.21.15.38:0/3737730645 learned_addr learned my addr 172.21.15.38:0/3737730645 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:34:32.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:32.758+0000 7f5182400640 1 --2- 172.21.15.38:0/3737730645 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f517c162790 0x7f517c1577c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:32.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:32.758+0000 7f5181bff640 1 --2- 172.21.15.38:0/3737730645 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f517c0cea90 0x7f517c0cd810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:32.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:32.758+0000 7f51813fe640 1 -- 172.21.15.38:0/3737730645 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f517c162790 msgr2=0x7f517c1577c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:32.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:32.758+0000 7f51813fe640 1 --2- 172.21.15.38:0/3737730645 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f517c162790 0x7f517c1577c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:32.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:32.758+0000 7f51813fe640 1 -- 172.21.15.38:0/3737730645 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f517c0cea90 msgr2=0x7f517c0cd810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:32.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:32.758+0000 7f51813fe640 1 --2- 172.21.15.38:0/3737730645 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f517c0cea90 0x7f517c0cd810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:32.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:32.758+0000 7f51813fe640 1 -- 172.21.15.38:0/3737730645 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f517c14a5f0 con 0x7f517c0d3360 2024-09-21T03:34:32.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:32.758+0000 7f51813fe640 1 --2- 172.21.15.38:0/3737730645 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f517c0d3360 0x7f517c157280 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f516c00c3a0 tx=0x7f516c00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:32.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:32.758+0000 7f51727fc640 1 -- 172.21.15.38:0/3737730645 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f516c011790 con 0x7f517c0d3360 2024-09-21T03:34:32.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:32.758+0000 7f51727fc640 1 -- 172.21.15.38:0/3737730645 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f516c017070 con 0x7f517c0d3360 2024-09-21T03:34:32.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:32.758+0000 7f5183e63640 1 -- 172.21.15.38:0/3737730645 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f517c14a8d0 con 0x7f517c0d3360 2024-09-21T03:34:32.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:32.758+0000 7f51727fc640 1 -- 172.21.15.38:0/3737730645 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f516c011c80 con 0x7f517c0d3360 2024-09-21T03:34:32.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:32.758+0000 7f5183e63640 1 -- 172.21.15.38:0/3737730645 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f517c14ae90 con 0x7f517c0d3360 2024-09-21T03:34:32.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:32.759+0000 7f5183e63640 1 -- 172.21.15.38:0/3737730645 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f517c0d1b00 con 0x7f517c0d3360 2024-09-21T03:34:32.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:32.760+0000 7f51727fc640 1 -- 172.21.15.38:0/3737730645 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f516c01c460 con 0x7f517c0d3360 2024-09-21T03:34:32.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:32.761+0000 7f51727fc640 1 --2- 172.21.15.38:0/3737730645 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5160078bc0 0x7f516007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:32.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:32.761+0000 7f51727fc640 1 -- 172.21.15.38:0/3737730645 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 5619+0+0 (secure 0 0 0) 0x7f516c09aa40 con 0x7f517c0d3360 2024-09-21T03:34:32.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:32.761+0000 7f5181bff640 1 --2- 172.21.15.38:0/3737730645 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5160078bc0 0x7f516007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:32.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:32.762+0000 7f5181bff640 1 --2- 172.21.15.38:0/3737730645 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5160078bc0 0x7f516007b0d0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f5178034000 tx=0x7f5178004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:32.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:32.766+0000 7f51727fc640 1 -- 172.21.15.38:0/3737730645 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f517c0d1b00 con 0x7f517c0d3360 2024-09-21T03:34:32.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:32.954+0000 7f5183e63640 1 -- 172.21.15.38:0/3737730645 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f517c0b4750 con 0x7f517c0d3360 2024-09-21T03:34:33.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:33.609+0000 7f51727fc640 1 -- 172.21.15.38:0/3737730645 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v38) ==== 170+0+0 (secure 0 0 0) 0x7f516c062ee0 con 0x7f517c0d3360 2024-09-21T03:34:33.611 INFO:teuthology.orchestra.run.smithi038.stderr:pool 'cephfs_data' created 2024-09-21T03:34:33.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:33.613+0000 7f5183e63640 1 -- 172.21.15.38:0/3737730645 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5160078bc0 msgr2=0x7f516007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:33.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:33.613+0000 7f5183e63640 1 --2- 172.21.15.38:0/3737730645 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5160078bc0 0x7f516007b0d0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f5178034000 tx=0x7f5178004730 comp rx=0 tx=0).stop 2024-09-21T03:34:33.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:33.613+0000 7f5183e63640 1 -- 172.21.15.38:0/3737730645 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f517c0d3360 msgr2=0x7f517c157280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:33.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:33.613+0000 7f5183e63640 1 --2- 172.21.15.38:0/3737730645 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f517c0d3360 0x7f517c157280 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f516c00c3a0 tx=0x7f516c00c860 comp rx=0 tx=0).stop 2024-09-21T03:34:33.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:33.613+0000 7f5183e63640 1 -- 172.21.15.38:0/3737730645 shutdown_connections 2024-09-21T03:34:33.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:33.613+0000 7f5183e63640 1 --2- 172.21.15.38:0/3737730645 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5160078bc0 0x7f516007b0d0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:33.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:33.613+0000 7f5183e63640 1 --2- 172.21.15.38:0/3737730645 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f517c162790 0x7f517c1577c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:33.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:33.613+0000 7f5183e63640 1 --2- 172.21.15.38:0/3737730645 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f517c0d3360 0x7f517c157280 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:33.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:33.613+0000 7f5183e63640 1 --2- 172.21.15.38:0/3737730645 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f517c0cea90 0x7f517c0cd810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:33.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:33.613+0000 7f5183e63640 1 -- 172.21.15.38:0/3737730645 >> 172.21.15.38:0/3737730645 conn(0x7f517c0d4620 msgr2=0x7f517c0d0c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:33.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:33.613+0000 7f5183e63640 1 -- 172.21.15.38:0/3737730645 shutdown_connections 2024-09-21T03:34:33.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:33.614+0000 7f5183e63640 1 -- 172.21.15.38:0/3737730645 wait complete. 2024-09-21T03:34:33.639 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-21T03:34:33.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:33.764+0000 7f650f65c640 1 -- 172.21.15.38:0/931889789 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f65080d1120 msgr2=0x7f65080ceb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:33.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:33.764+0000 7f650f65c640 1 --2- 172.21.15.38:0/931889789 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f65080d1120 0x7f65080ceb90 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f65080757e0 tx=0x7f650002a980 comp rx=0 tx=0).stop 2024-09-21T03:34:33.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:33.764+0000 7f650f65c640 1 -- 172.21.15.38:0/931889789 shutdown_connections 2024-09-21T03:34:33.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:33.764+0000 7f650f65c640 1 --2- 172.21.15.38:0/931889789 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f65080d1120 0x7f65080ceb90 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:33.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:33.764+0000 7f650f65c640 1 --2- 172.21.15.38:0/931889789 >> v2:172.21.15.195:3300/0 conn(0x7f65080b9ad0 0x7f65080d0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:33.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:33.764+0000 7f650f65c640 1 --2- 172.21.15.38:0/931889789 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f650815ee80 0x7f65080d0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:33.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:33.764+0000 7f650f65c640 1 -- 172.21.15.38:0/931889789 >> 172.21.15.38:0/931889789 conn(0x7f650809d830 msgr2=0x7f65080b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:33.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:33.764+0000 7f650f65c640 1 -- 172.21.15.38:0/931889789 shutdown_connections 2024-09-21T03:34:33.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:33.764+0000 7f650f65c640 1 -- 172.21.15.38:0/931889789 wait complete. 2024-09-21T03:34:33.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:33.766+0000 7f650f65c640 1 Processor -- start 2024-09-21T03:34:33.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:33.766+0000 7f650f65c640 1 -- start start 2024-09-21T03:34:33.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:33.767+0000 7f650f65c640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f65080b9ad0 0x7f650809e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:33.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:33.767+0000 7f650f65c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f65080d1120 0x7f65080ce160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:33.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:33.767+0000 7f650f65c640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f650815ee80 0x7f65080a0940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:33.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:33.767+0000 7f650f65c640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f65080cf460 con 0x7f65080d1120 2024-09-21T03:34:33.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:33.767+0000 7f650f65c640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f65080cf2e0 con 0x7f65080b9ad0 2024-09-21T03:34:33.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:33.767+0000 7f650f65c640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f65080b6580 con 0x7f650815ee80 2024-09-21T03:34:33.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:33.767+0000 7f650cbf7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f65080d1120 0x7f65080ce160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:33.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:33.767+0000 7f650cbf7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f65080d1120 0x7f65080ce160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:33534/0 (socket says 172.21.15.38:33534) 2024-09-21T03:34:33.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:33.767+0000 7f650cbf7640 1 -- 172.21.15.38:0/3290299704 learned_addr learned my addr 172.21.15.38:0/3290299704 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:34:33.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:33.767+0000 7f650dbf9640 1 --2- 172.21.15.38:0/3290299704 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f650815ee80 0x7f65080a0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:33.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:33.768+0000 7f650d3f8640 1 --2- 172.21.15.38:0/3290299704 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f65080b9ad0 0x7f650809e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:33.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:33.768+0000 7f650cbf7640 1 -- 172.21.15.38:0/3290299704 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f650815ee80 msgr2=0x7f65080a0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:33.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:33.768+0000 7f650cbf7640 1 --2- 172.21.15.38:0/3290299704 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f650815ee80 0x7f65080a0940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:33.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:33.768+0000 7f650cbf7640 1 -- 172.21.15.38:0/3290299704 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f65080b9ad0 msgr2=0x7f650809e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:33.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:33.768+0000 7f650cbf7640 1 --2- 172.21.15.38:0/3290299704 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f65080b9ad0 0x7f650809e740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:33.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:33.768+0000 7f650cbf7640 1 -- 172.21.15.38:0/3290299704 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65080a0e80 con 0x7f65080d1120 2024-09-21T03:34:33.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:33.768+0000 7f650dbf9640 1 --2- 172.21.15.38:0/3290299704 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f650815ee80 0x7f65080a0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:34:33.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:33.768+0000 7f650cbf7640 1 --2- 172.21.15.38:0/3290299704 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f65080d1120 0x7f65080ce160 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f64f800d660 tx=0x7f64f800db20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:33.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:33.768+0000 7f64fdffb640 1 -- 172.21.15.38:0/3290299704 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64f8017070 con 0x7f65080d1120 2024-09-21T03:34:33.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:33.768+0000 7f64fdffb640 1 -- 172.21.15.38:0/3290299704 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f64f8009b10 con 0x7f65080d1120 2024-09-21T03:34:33.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:33.768+0000 7f64fdffb640 1 -- 172.21.15.38:0/3290299704 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64f8010460 con 0x7f65080d1120 2024-09-21T03:34:33.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:33.768+0000 7f650f65c640 1 -- 172.21.15.38:0/3290299704 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65080a12c0 con 0x7f65080d1120 2024-09-21T03:34:33.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:33.768+0000 7f650f65c640 1 -- 172.21.15.38:0/3290299704 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65080a1880 con 0x7f65080d1120 2024-09-21T03:34:33.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:33.769+0000 7f650f65c640 1 -- 172.21.15.38:0/3290299704 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65080ceb70 con 0x7f65080d1120 2024-09-21T03:34:33.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:33.770+0000 7f64fdffb640 1 -- 172.21.15.38:0/3290299704 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f64f801a070 con 0x7f65080d1120 2024-09-21T03:34:33.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:33.771+0000 7f64fdffb640 1 --2- 172.21.15.38:0/3290299704 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f64ec078b90 0x7f64ec07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:33.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:33.771+0000 7f64fdffb640 1 -- 172.21.15.38:0/3290299704 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6001+0+0 (secure 0 0 0) 0x7f64f809af40 con 0x7f65080d1120 2024-09-21T03:34:33.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:33.771+0000 7f650d3f8640 1 --2- 172.21.15.38:0/3290299704 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f64ec078b90 0x7f64ec07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:33.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:33.772+0000 7f650d3f8640 1 --2- 172.21.15.38:0/3290299704 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f64ec078b90 0x7f64ec07b0a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f65040058c0 tx=0x7f6504005850 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:33.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:33.776+0000 7f64fdffb640 1 -- 172.21.15.38:0/3290299704 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f64f8063250 con 0x7f65080d1120 2024-09-21T03:34:34.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:34.008+0000 7f650f65c640 1 -- 172.21.15.38:0/3290299704 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f65080a1050 con 0x7f65080d1120 2024-09-21T03:34:34.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:34.611+0000 7f64fdffb640 1 -- 172.21.15.38:0/3290299704 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 5 and data pool 6 v14) ==== 172+0+0 (secure 0 0 0) 0x7f64f8062b50 con 0x7f65080d1120 2024-09-21T03:34:34.613 INFO:teuthology.orchestra.run.smithi038.stderr:new fs with metadata pool 5 and data pool 6 2024-09-21T03:34:34.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:34.616+0000 7f650f65c640 1 -- 172.21.15.38:0/3290299704 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f64ec078b90 msgr2=0x7f64ec07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:34.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:34.616+0000 7f650f65c640 1 --2- 172.21.15.38:0/3290299704 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f64ec078b90 0x7f64ec07b0a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f65040058c0 tx=0x7f6504005850 comp rx=0 tx=0).stop 2024-09-21T03:34:34.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:34.616+0000 7f650f65c640 1 -- 172.21.15.38:0/3290299704 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f65080d1120 msgr2=0x7f65080ce160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:34.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:34.616+0000 7f650f65c640 1 --2- 172.21.15.38:0/3290299704 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f65080d1120 0x7f65080ce160 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f64f800d660 tx=0x7f64f800db20 comp rx=0 tx=0).stop 2024-09-21T03:34:34.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:34.617+0000 7f650f65c640 1 -- 172.21.15.38:0/3290299704 shutdown_connections 2024-09-21T03:34:34.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:34.617+0000 7f650f65c640 1 --2- 172.21.15.38:0/3290299704 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f64ec078b90 0x7f64ec07b0a0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:34.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:34.617+0000 7f650f65c640 1 --2- 172.21.15.38:0/3290299704 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f650815ee80 0x7f65080a0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:34.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:34.617+0000 7f650f65c640 1 --2- 172.21.15.38:0/3290299704 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f65080d1120 0x7f65080ce160 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:34.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:34.617+0000 7f650f65c640 1 --2- 172.21.15.38:0/3290299704 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f65080b9ad0 0x7f650809e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:34.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:34.618+0000 7f650f65c640 1 -- 172.21.15.38:0/3290299704 >> 172.21.15.38:0/3290299704 conn(0x7f650809d830 msgr2=0x7f65080d4610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:34.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:34.624+0000 7f650f65c640 1 -- 172.21.15.38:0/3290299704 shutdown_connections 2024-09-21T03:34:34.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:34.624+0000 7f650f65c640 1 -- 172.21.15.38:0/3290299704 wait complete. 2024-09-21T03:34:34.657 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T03:34:34.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:34.774+0000 7f523fb39640 1 -- 172.21.15.38:0/3325596497 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f52380cfe00 msgr2=0x7f52380d2a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:34.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:34.774+0000 7f523fb39640 1 --2- 172.21.15.38:0/3325596497 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f52380cfe00 0x7f52380d2a20 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f5234005fd0 tx=0x7f523402a5d0 comp rx=0 tx=0).stop 2024-09-21T03:34:34.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:34.774+0000 7f523fb39640 1 -- 172.21.15.38:0/3325596497 shutdown_connections 2024-09-21T03:34:34.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:34.774+0000 7f523fb39640 1 --2- 172.21.15.38:0/3325596497 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f52380b2e60 0x7f52380b3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:34.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:34.774+0000 7f523fb39640 1 --2- 172.21.15.38:0/3325596497 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f52380cfe00 0x7f52380d2a20 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:34.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:34.774+0000 7f523fb39640 1 -- 172.21.15.38:0/3325596497 >> 172.21.15.38:0/3325596497 conn(0x7f52380d4440 msgr2=0x7f52380d4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:34.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:34.775+0000 7f523fb39640 1 -- 172.21.15.38:0/3325596497 shutdown_connections 2024-09-21T03:34:34.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:34.775+0000 7f523fb39640 1 -- 172.21.15.38:0/3325596497 wait complete. 2024-09-21T03:34:34.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:34.776+0000 7f523fb39640 1 Processor -- start 2024-09-21T03:34:34.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:34.776+0000 7f523fb39640 1 -- start start 2024-09-21T03:34:34.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:34.778+0000 7f523fb39640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f52380b2e60 0x7f523815daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:34.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:34.778+0000 7f523fb39640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f52380cfe00 0x7f523815e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:34.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:34.778+0000 7f523fb39640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5238162590 0x7f52381f8e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:34.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:34.778+0000 7f523fb39640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f52380b3930 con 0x7f52380b2e60 2024-09-21T03:34:34.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:34.778+0000 7f523fb39640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f52380b37b0 con 0x7f5238162590 2024-09-21T03:34:34.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:34.778+0000 7f523fb39640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f52380b3ab0 con 0x7f52380cfe00 2024-09-21T03:34:34.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:34.778+0000 7f523d8d5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f52380b2e60 0x7f523815daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:34.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:34.778+0000 7f523d8d5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f52380b2e60 0x7f523815daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:33554/0 (socket says 172.21.15.38:33554) 2024-09-21T03:34:34.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:34.778+0000 7f523d8d5640 1 -- 172.21.15.38:0/681013446 learned_addr learned my addr 172.21.15.38:0/681013446 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:34:34.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:34.778+0000 7f523d0d4640 1 --2- 172.21.15.38:0/681013446 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f52380cfe00 0x7f523815e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:34.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:34.778+0000 7f523e0d6640 1 --2- 172.21.15.38:0/681013446 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5238162590 0x7f52381f8e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:34.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:34.778+0000 7f523d8d5640 1 -- 172.21.15.38:0/681013446 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f52380cfe00 msgr2=0x7f523815e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:34.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:34.778+0000 7f523d8d5640 1 --2- 172.21.15.38:0/681013446 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f52380cfe00 0x7f523815e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:34.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:34.778+0000 7f523d8d5640 1 -- 172.21.15.38:0/681013446 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5238162590 msgr2=0x7f52381f8e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:34.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:34.778+0000 7f523d8d5640 1 --2- 172.21.15.38:0/681013446 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5238162590 0x7f52381f8e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:34.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:34.778+0000 7f523d8d5640 1 -- 172.21.15.38:0/681013446 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52381e9920 con 0x7f52380b2e60 2024-09-21T03:34:34.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:34.778+0000 7f523e0d6640 1 --2- 172.21.15.38:0/681013446 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5238162590 0x7f52381f8e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:34:34.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:34.778+0000 7f523d8d5640 1 --2- 172.21.15.38:0/681013446 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f52380b2e60 0x7f523815daf0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f5234006010 tx=0x7f52340035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:34.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:34.778+0000 7f522e7fc640 1 -- 172.21.15.38:0/681013446 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f523403e030 con 0x7f52380b2e60 2024-09-21T03:34:34.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:34.778+0000 7f522e7fc640 1 -- 172.21.15.38:0/681013446 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5234003ae0 con 0x7f52380b2e60 2024-09-21T03:34:34.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:34.778+0000 7f523fb39640 1 -- 172.21.15.38:0/681013446 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52381e8da0 con 0x7f52380b2e60 2024-09-21T03:34:34.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:34.779+0000 7f522e7fc640 1 -- 172.21.15.38:0/681013446 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5234035040 con 0x7f52380b2e60 2024-09-21T03:34:34.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:34.779+0000 7f523fb39640 1 -- 172.21.15.38:0/681013446 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52381e9360 con 0x7f52380b2e60 2024-09-21T03:34:34.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:34.779+0000 7f523fb39640 1 -- 172.21.15.38:0/681013446 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52380b3270 con 0x7f52380b2e60 2024-09-21T03:34:34.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:34.780+0000 7f522e7fc640 1 -- 172.21.15.38:0/681013446 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f5234041030 con 0x7f52380b2e60 2024-09-21T03:34:34.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:34.781+0000 7f522e7fc640 1 --2- 172.21.15.38:0/681013446 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f521c078bc0 0x7f521c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:34.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:34.781+0000 7f522e7fc640 1 -- 172.21.15.38:0/681013446 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f52340bb750 con 0x7f52380b2e60 2024-09-21T03:34:34.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:34.781+0000 7f523d0d4640 1 --2- 172.21.15.38:0/681013446 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f521c078bc0 0x7f521c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:34.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:34.782+0000 7f523d0d4640 1 --2- 172.21.15.38:0/681013446 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f521c078bc0 0x7f521c07b0d0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f52380d2510 tx=0x7f5228009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:34.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:34.785+0000 7f522e7fc640 1 -- 172.21.15.38:0/681013446 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f5234083b70 con 0x7f52380b2e60 2024-09-21T03:34:34.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:34.972+0000 7f523fb39640 1 -- 172.21.15.38:0/681013446 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f523815ea70 con 0x7f52380b2e60 2024-09-21T03:34:34.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:34.974+0000 7f522e7fc640 1 -- 172.21.15.38:0/681013446 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) ==== 74+0+17039 (secure 0 0 0) 0x7f5234083470 con 0x7f52380b2e60 2024-09-21T03:34:34.976 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:34:34.976 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":39,"fsid":"9d50d66f-656b-4fe2-8e3b-fe53b79c4d43","created":"2024-09-21T03:33:10.677865+0000","modified":"2024-09-21T03:34:34.009311+0000","last_up_change":"2024-09-21T03:33:24.023421+0000","last_in_change":"2024-09-21T03:33:14.296193+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T03:33:22.992573+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-09-21T03:34:32.080636+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-09-21T03:34:32.959110+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dbab183f-7e63-48b8-960a-ae55fd5bf565","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6801","nonce":2381639433}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6803","nonce":2381639433}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6807","nonce":2381639433}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6805","nonce":2381639433}]},"public_addr":"172.21.15.38:6801/2381639433","cluster_addr":"172.21.15.38:6803/2381639433","heartbeat_back_addr":"172.21.15.38:6807/2381639433","heartbeat_front_addr":"172.21.15.38:6805/2381639433","state":["exists","up"]},{"osd":1,"uuid":"fa1c0e8a-ebc1-4359-8146-bd5342abbc16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6817","nonce":1491818732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6819","nonce":1491818732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6823","nonce":1491818732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6821","nonce":1491818732}]},"public_addr":"172.21.15.38:6817/1491818732","cluster_addr":"172.21.15.38:6819/1491818732","heartbeat_back_addr":"172.21.15.38:6823/1491818732","heartbeat_front_addr":"172.21.15.38:6821/1491818732","state":["exists","up"]},{"osd":2,"uuid":"2ed3cd2d-385d-4fc6-9869-214552a84803","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6825","nonce":2577858902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6827","nonce":2577858902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6831","nonce":2577858902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6829","nonce":2577858902}]},"public_addr":"172.21.15.38:6825/2577858902","cluster_addr":"172.21.15.38:6827/2577858902","heartbeat_back_addr":"172.21.15.38:6831/2577858902","heartbeat_front_addr":"172.21.15.38:6829/2577858902","state":["exists","up"]},{"osd":3,"uuid":"62d1d95e-919d-4250-be71-1d34cfa70dbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6809","nonce":1286810294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6811","nonce":1286810294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6815","nonce":1286810294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6813","nonce":1286810294}]},"public_addr":"172.21.15.38:6809/1286810294","cluster_addr":"172.21.15.38:6811/1286810294","heartbeat_back_addr":"172.21.15.38:6815/1286810294","heartbeat_front_addr":"172.21.15.38:6813/1286810294","state":["exists","up"]},{"osd":4,"uuid":"7c9f9314-99d7-4d73-8392-37809a9f54b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6803","nonce":3401837571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6805","nonce":3401837571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6809","nonce":3401837571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6807","nonce":3401837571}]},"public_addr":"172.21.15.195:6803/3401837571","cluster_addr":"172.21.15.195:6805/3401837571","heartbeat_back_addr":"172.21.15.195:6809/3401837571","heartbeat_front_addr":"172.21.15.195:6807/3401837571","state":["exists","up"]},{"osd":5,"uuid":"bde4fe9f-8414-4bba-bf4a-43973d1976bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6811","nonce":2080960180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6813","nonce":2080960180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6817","nonce":2080960180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6815","nonce":2080960180}]},"public_addr":"172.21.15.195:6811/2080960180","cluster_addr":"172.21.15.195:6813/2080960180","heartbeat_back_addr":"172.21.15.195:6817/2080960180","heartbeat_front_addr":"172.21.15.195:6815/2080960180","state":["exists","up"]},{"osd":6,"uuid":"6ee6d7f4-1a5f-4a4c-ac16-48fe05f6a002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6827","nonce":4055825675}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6829","nonce":4055825675}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6833","nonce":4055825675}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6831","nonce":4055825675}]},"public_addr":"172.21.15.195:6827/4055825675","cluster_addr":"172.21.15.195:6829/4055825675","heartbeat_back_addr":"172.21.15.195:6833/4055825675","heartbeat_front_addr":"172.21.15.195:6831/4055825675","state":["exists","up"]},{"osd":7,"uuid":"efeb5f51-329a-4484-8290-92e7d3503d05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6819","nonce":1606106342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6821","nonce":1606106342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6825","nonce":1606106342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6823","nonce":1606106342}]},"public_addr":"172.21.15.195:6819/1606106342","cluster_addr":"172.21.15.195:6821/1606106342","heartbeat_back_addr":"172.21.15.195:6825/1606106342","heartbeat_front_addr":"172.21.15.195:6823/1606106342","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.901343+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:21.115003+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.901623+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.835238+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.132199+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:19.817560+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.829690+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:21.087300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T03:34:34.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:34.976+0000 7f523fb39640 1 -- 172.21.15.38:0/681013446 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f521c078bc0 msgr2=0x7f521c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:34.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:34.976+0000 7f523fb39640 1 --2- 172.21.15.38:0/681013446 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f521c078bc0 0x7f521c07b0d0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f52380d2510 tx=0x7f5228009040 comp rx=0 tx=0).stop 2024-09-21T03:34:34.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:34.976+0000 7f523fb39640 1 -- 172.21.15.38:0/681013446 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f52380b2e60 msgr2=0x7f523815daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:34.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:34.976+0000 7f523fb39640 1 --2- 172.21.15.38:0/681013446 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f52380b2e60 0x7f523815daf0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f5234006010 tx=0x7f52340035f0 comp rx=0 tx=0).stop 2024-09-21T03:34:34.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:34.976+0000 7f523fb39640 1 -- 172.21.15.38:0/681013446 shutdown_connections 2024-09-21T03:34:34.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:34.976+0000 7f523fb39640 1 --2- 172.21.15.38:0/681013446 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5238162590 0x7f52381f8e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:34.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:34.976+0000 7f523fb39640 1 --2- 172.21.15.38:0/681013446 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f521c078bc0 0x7f521c07b0d0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:34.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:34.976+0000 7f523fb39640 1 --2- 172.21.15.38:0/681013446 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f52380cfe00 0x7f523815e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:34.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:34.976+0000 7f523fb39640 1 --2- 172.21.15.38:0/681013446 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f52380b2e60 0x7f523815daf0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:34.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:34.976+0000 7f523fb39640 1 -- 172.21.15.38:0/681013446 >> 172.21.15.38:0/681013446 conn(0x7f52380d4440 msgr2=0x7f52380b4610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:34.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:34.976+0000 7f523fb39640 1 -- 172.21.15.38:0/681013446 shutdown_connections 2024-09-21T03:34:34.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:34.976+0000 7f523fb39640 1 -- 172.21.15.38:0/681013446 wait complete. 2024-09-21T03:34:34.989 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T03:34:35.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:35.104+0000 7f848d294640 1 -- 172.21.15.38:0/3794385518 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f84880b2e40 msgr2=0x7f84880b3250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:35.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:35.104+0000 7f848d294640 1 --2- 172.21.15.38:0/3794385518 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f84880b2e40 0x7f84880b3250 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f8478005fd0 tx=0x7f847802aa00 comp rx=0 tx=0).stop 2024-09-21T03:34:35.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:35.104+0000 7f848d294640 1 -- 172.21.15.38:0/3794385518 shutdown_connections 2024-09-21T03:34:35.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:35.104+0000 7f848d294640 1 --2- 172.21.15.38:0/3794385518 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f84880b2e40 0x7f84880b3250 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:35.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:35.104+0000 7f848d294640 1 --2- 172.21.15.38:0/3794385518 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8488162280 0x7f8488162690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:35.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:35.104+0000 7f848d294640 1 -- 172.21.15.38:0/3794385518 >> 172.21.15.38:0/3794385518 conn(0x7f84880b6b20 msgr2=0x7f84880d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:35.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:35.104+0000 7f848d294640 1 -- 172.21.15.38:0/3794385518 shutdown_connections 2024-09-21T03:34:35.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:35.104+0000 7f848d294640 1 -- 172.21.15.38:0/3794385518 wait complete. 2024-09-21T03:34:35.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:35.106+0000 7f848d294640 1 Processor -- start 2024-09-21T03:34:35.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:35.106+0000 7f848d294640 1 -- start start 2024-09-21T03:34:35.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:35.107+0000 7f848d294640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f84880b2e40 0x7f84880cf040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:35.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:35.107+0000 7f848d294640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8488162280 0x7f848814d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:35.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:35.107+0000 7f848d294640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f848814de20 0x7f848814e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:35.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:35.107+0000 7f848d294640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f848815e4b0 con 0x7f8488162280 2024-09-21T03:34:35.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:35.107+0000 7f848d294640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f84880b39a0 con 0x7f848814de20 2024-09-21T03:34:35.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:35.107+0000 7f848d294640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f848815e630 con 0x7f84880b2e40 2024-09-21T03:34:35.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:35.107+0000 7f84867fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8488162280 0x7f848814d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:35.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:35.107+0000 7f84867fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8488162280 0x7f848814d8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:33580/0 (socket says 172.21.15.38:33580) 2024-09-21T03:34:35.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:35.107+0000 7f84867fc640 1 -- 172.21.15.38:0/750217840 learned_addr learned my addr 172.21.15.38:0/750217840 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:34:35.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:35.107+0000 7f8486ffd640 1 --2- 172.21.15.38:0/750217840 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f84880b2e40 0x7f84880cf040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:35.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:35.107+0000 7f84877fe640 1 --2- 172.21.15.38:0/750217840 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f848814de20 0x7f848814e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:35.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:35.108+0000 7f84867fc640 1 -- 172.21.15.38:0/750217840 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f84880b2e40 msgr2=0x7f84880cf040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:35.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:35.108+0000 7f84867fc640 1 --2- 172.21.15.38:0/750217840 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f84880b2e40 0x7f84880cf040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:35.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:35.108+0000 7f84867fc640 1 -- 172.21.15.38:0/750217840 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f848814de20 msgr2=0x7f848814e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:35.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:35.108+0000 7f84867fc640 1 --2- 172.21.15.38:0/750217840 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f848814de20 0x7f848814e230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:35.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:35.108+0000 7f84867fc640 1 -- 172.21.15.38:0/750217840 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f848814e770 con 0x7f8488162280 2024-09-21T03:34:35.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:35.108+0000 7f84867fc640 1 --2- 172.21.15.38:0/750217840 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8488162280 0x7f848814d8e0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f847000d750 tx=0x7f847000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:35.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:35.108+0000 7f847f7fe640 1 -- 172.21.15.38:0/750217840 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8470017070 con 0x7f8488162280 2024-09-21T03:34:35.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:35.108+0000 7f847f7fe640 1 -- 172.21.15.38:0/750217840 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8470009b20 con 0x7f8488162280 2024-09-21T03:34:35.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:35.108+0000 7f848d294640 1 -- 172.21.15.38:0/750217840 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84880ad4b0 con 0x7f8488162280 2024-09-21T03:34:35.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:35.108+0000 7f847f7fe640 1 -- 172.21.15.38:0/750217840 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8470010460 con 0x7f8488162280 2024-09-21T03:34:35.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:35.108+0000 7f848d294640 1 -- 172.21.15.38:0/750217840 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f848814ce00 con 0x7f8488162280 2024-09-21T03:34:35.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:35.109+0000 7f848d294640 1 -- 172.21.15.38:0/750217840 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84880ad220 con 0x7f8488162280 2024-09-21T03:34:35.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:35.110+0000 7f847f7fe640 1 -- 172.21.15.38:0/750217840 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f847001a070 con 0x7f8488162280 2024-09-21T03:34:35.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:35.110+0000 7f847f7fe640 1 --2- 172.21.15.38:0/750217840 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8468078bc0 0x7f846807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:35.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:35.110+0000 7f847f7fe640 1 -- 172.21.15.38:0/750217840 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f847009af30 con 0x7f8488162280 2024-09-21T03:34:35.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:35.111+0000 7f8486ffd640 1 --2- 172.21.15.38:0/750217840 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8468078bc0 0x7f846807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:35.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:35.111+0000 7f8486ffd640 1 --2- 172.21.15.38:0/750217840 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8468078bc0 0x7f846807b0d0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f8480006fd0 tx=0x7f8480009060 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:35.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:35.115+0000 7f847f7fe640 1 -- 172.21.15.38:0/750217840 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f8470062c20 con 0x7f8488162280 2024-09-21T03:34:35.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:35.301+0000 7f848d294640 1 -- 172.21.15.38:0/750217840 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8488162f00 con 0x7f8488162280 2024-09-21T03:34:35.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:35.302+0000 7f847f7fe640 1 -- 172.21.15.38:0/750217840 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) ==== 74+0+17039 (secure 0 0 0) 0x7f8470067650 con 0x7f8488162280 2024-09-21T03:34:35.304 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:34:35.304 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":39,"fsid":"9d50d66f-656b-4fe2-8e3b-fe53b79c4d43","created":"2024-09-21T03:33:10.677865+0000","modified":"2024-09-21T03:34:34.009311+0000","last_up_change":"2024-09-21T03:33:24.023421+0000","last_in_change":"2024-09-21T03:33:14.296193+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T03:33:22.992573+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-09-21T03:34:32.080636+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-09-21T03:34:32.959110+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dbab183f-7e63-48b8-960a-ae55fd5bf565","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6801","nonce":2381639433}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6803","nonce":2381639433}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6807","nonce":2381639433}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6805","nonce":2381639433}]},"public_addr":"172.21.15.38:6801/2381639433","cluster_addr":"172.21.15.38:6803/2381639433","heartbeat_back_addr":"172.21.15.38:6807/2381639433","heartbeat_front_addr":"172.21.15.38:6805/2381639433","state":["exists","up"]},{"osd":1,"uuid":"fa1c0e8a-ebc1-4359-8146-bd5342abbc16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6817","nonce":1491818732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6819","nonce":1491818732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6823","nonce":1491818732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6821","nonce":1491818732}]},"public_addr":"172.21.15.38:6817/1491818732","cluster_addr":"172.21.15.38:6819/1491818732","heartbeat_back_addr":"172.21.15.38:6823/1491818732","heartbeat_front_addr":"172.21.15.38:6821/1491818732","state":["exists","up"]},{"osd":2,"uuid":"2ed3cd2d-385d-4fc6-9869-214552a84803","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6825","nonce":2577858902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6827","nonce":2577858902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6831","nonce":2577858902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6829","nonce":2577858902}]},"public_addr":"172.21.15.38:6825/2577858902","cluster_addr":"172.21.15.38:6827/2577858902","heartbeat_back_addr":"172.21.15.38:6831/2577858902","heartbeat_front_addr":"172.21.15.38:6829/2577858902","state":["exists","up"]},{"osd":3,"uuid":"62d1d95e-919d-4250-be71-1d34cfa70dbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6809","nonce":1286810294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6811","nonce":1286810294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6815","nonce":1286810294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6813","nonce":1286810294}]},"public_addr":"172.21.15.38:6809/1286810294","cluster_addr":"172.21.15.38:6811/1286810294","heartbeat_back_addr":"172.21.15.38:6815/1286810294","heartbeat_front_addr":"172.21.15.38:6813/1286810294","state":["exists","up"]},{"osd":4,"uuid":"7c9f9314-99d7-4d73-8392-37809a9f54b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6803","nonce":3401837571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6805","nonce":3401837571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6809","nonce":3401837571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6807","nonce":3401837571}]},"public_addr":"172.21.15.195:6803/3401837571","cluster_addr":"172.21.15.195:6805/3401837571","heartbeat_back_addr":"172.21.15.195:6809/3401837571","heartbeat_front_addr":"172.21.15.195:6807/3401837571","state":["exists","up"]},{"osd":5,"uuid":"bde4fe9f-8414-4bba-bf4a-43973d1976bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6811","nonce":2080960180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6813","nonce":2080960180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6817","nonce":2080960180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6815","nonce":2080960180}]},"public_addr":"172.21.15.195:6811/2080960180","cluster_addr":"172.21.15.195:6813/2080960180","heartbeat_back_addr":"172.21.15.195:6817/2080960180","heartbeat_front_addr":"172.21.15.195:6815/2080960180","state":["exists","up"]},{"osd":6,"uuid":"6ee6d7f4-1a5f-4a4c-ac16-48fe05f6a002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6827","nonce":4055825675}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6829","nonce":4055825675}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6833","nonce":4055825675}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6831","nonce":4055825675}]},"public_addr":"172.21.15.195:6827/4055825675","cluster_addr":"172.21.15.195:6829/4055825675","heartbeat_back_addr":"172.21.15.195:6833/4055825675","heartbeat_front_addr":"172.21.15.195:6831/4055825675","state":["exists","up"]},{"osd":7,"uuid":"efeb5f51-329a-4484-8290-92e7d3503d05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6819","nonce":1606106342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6821","nonce":1606106342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6825","nonce":1606106342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6823","nonce":1606106342}]},"public_addr":"172.21.15.195:6819/1606106342","cluster_addr":"172.21.15.195:6821/1606106342","heartbeat_back_addr":"172.21.15.195:6825/1606106342","heartbeat_front_addr":"172.21.15.195:6823/1606106342","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.901343+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:21.115003+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.901623+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.835238+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.132199+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:19.817560+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.829690+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:21.087300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T03:34:35.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:35.304+0000 7f848d294640 1 -- 172.21.15.38:0/750217840 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8468078bc0 msgr2=0x7f846807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:35.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:35.304+0000 7f848d294640 1 --2- 172.21.15.38:0/750217840 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8468078bc0 0x7f846807b0d0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f8480006fd0 tx=0x7f8480009060 comp rx=0 tx=0).stop 2024-09-21T03:34:35.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:35.304+0000 7f848d294640 1 -- 172.21.15.38:0/750217840 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8488162280 msgr2=0x7f848814d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:35.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:35.304+0000 7f848d294640 1 --2- 172.21.15.38:0/750217840 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8488162280 0x7f848814d8e0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f847000d750 tx=0x7f847000dc10 comp rx=0 tx=0).stop 2024-09-21T03:34:35.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:35.304+0000 7f848d294640 1 -- 172.21.15.38:0/750217840 shutdown_connections 2024-09-21T03:34:35.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:35.304+0000 7f848d294640 1 --2- 172.21.15.38:0/750217840 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f848814de20 0x7f848814e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:35.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:35.304+0000 7f848d294640 1 --2- 172.21.15.38:0/750217840 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8488162280 0x7f848814d8e0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:35.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:35.304+0000 7f848d294640 1 --2- 172.21.15.38:0/750217840 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8468078bc0 0x7f846807b0d0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:35.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:35.304+0000 7f848d294640 1 --2- 172.21.15.38:0/750217840 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f84880b2e40 0x7f84880cf040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:35.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:35.304+0000 7f848d294640 1 -- 172.21.15.38:0/750217840 >> 172.21.15.38:0/750217840 conn(0x7f84880b6b20 msgr2=0x7f848806cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:35.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:35.304+0000 7f848d294640 1 -- 172.21.15.38:0/750217840 shutdown_connections 2024-09-21T03:34:35.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:35.304+0000 7f848d294640 1 -- 172.21.15.38:0/750217840 wait complete. 2024-09-21T03:34:35.318 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-21T03:34:35.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:35.435+0000 7effb30b4640 1 -- 172.21.15.38:0/3743545507 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7effac0d0bb0 msgr2=0x7effac0b5be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:35.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:35.435+0000 7effb30b4640 1 --2- 172.21.15.38:0/3743545507 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7effac0d0bb0 0x7effac0b5be0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7eff94005fd0 tx=0x7eff9402a4a0 comp rx=0 tx=0).stop 2024-09-21T03:34:35.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:35.435+0000 7effb30b4640 1 -- 172.21.15.38:0/3743545507 shutdown_connections 2024-09-21T03:34:35.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:35.435+0000 7effb30b4640 1 --2- 172.21.15.38:0/3743545507 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7effac0d0bb0 0x7effac0b5be0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:35.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:35.435+0000 7effb30b4640 1 --2- 172.21.15.38:0/3743545507 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7effac15ca90 0x7effac0d0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:35.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:35.435+0000 7effb30b4640 1 -- 172.21.15.38:0/3743545507 >> 172.21.15.38:0/3743545507 conn(0x7effac0b46e0 msgr2=0x7effac0d23e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:35.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:35.435+0000 7effb30b4640 1 -- 172.21.15.38:0/3743545507 shutdown_connections 2024-09-21T03:34:35.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:35.435+0000 7effb30b4640 1 -- 172.21.15.38:0/3743545507 wait complete. 2024-09-21T03:34:35.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:35.437+0000 7effb30b4640 1 Processor -- start 2024-09-21T03:34:35.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:35.437+0000 7effb30b4640 1 -- start start 2024-09-21T03:34:35.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:35.439+0000 7effb30b4640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7effac0ccc40 0x7effac0b3840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:35.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:35.439+0000 7effb30b4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7effac0d0bb0 0x7effac149780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:35.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:35.439+0000 7effb30b4640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7effac15ca90 0x7effac149cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:35.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:35.439+0000 7effb30b4640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7effac0b6620 con 0x7effac0d0bb0 2024-09-21T03:34:35.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:35.439+0000 7effb30b4640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7effac0b64a0 con 0x7effac15ca90 2024-09-21T03:34:35.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:35.439+0000 7effb30b4640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7effac0b67a0 con 0x7effac0ccc40 2024-09-21T03:34:35.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:35.439+0000 7effabfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7effac0d0bb0 0x7effac149780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:35.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:35.439+0000 7effabfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7effac0d0bb0 0x7effac149780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:33596/0 (socket says 172.21.15.38:33596) 2024-09-21T03:34:35.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:35.439+0000 7effabfff640 1 -- 172.21.15.38:0/3558048566 learned_addr learned my addr 172.21.15.38:0/3558048566 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:34:35.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:35.439+0000 7effb0e50640 1 --2- 172.21.15.38:0/3558048566 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7effac0ccc40 0x7effac0b3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:35.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:35.439+0000 7effb1651640 1 --2- 172.21.15.38:0/3558048566 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7effac15ca90 0x7effac149cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:35.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:35.439+0000 7effabfff640 1 -- 172.21.15.38:0/3558048566 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7effac0ccc40 msgr2=0x7effac0b3840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:35.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:35.439+0000 7effabfff640 1 --2- 172.21.15.38:0/3558048566 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7effac0ccc40 0x7effac0b3840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:35.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:35.439+0000 7effabfff640 1 -- 172.21.15.38:0/3558048566 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7effac15ca90 msgr2=0x7effac149cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:35.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:35.439+0000 7effabfff640 1 --2- 172.21.15.38:0/3558048566 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7effac15ca90 0x7effac149cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:35.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:35.439+0000 7effabfff640 1 -- 172.21.15.38:0/3558048566 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effac0b3d80 con 0x7effac0d0bb0 2024-09-21T03:34:35.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:35.439+0000 7effabfff640 1 --2- 172.21.15.38:0/3558048566 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7effac0d0bb0 0x7effac149780 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7eff9c00e9f0 tx=0x7eff9c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:35.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:35.440+0000 7effa97fa640 1 -- 172.21.15.38:0/3558048566 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff9c016070 con 0x7effac0d0bb0 2024-09-21T03:34:35.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:35.440+0000 7effa97fa640 1 -- 172.21.15.38:0/3558048566 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7eff9c00c970 con 0x7effac0d0bb0 2024-09-21T03:34:35.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:35.440+0000 7effb30b4640 1 -- 172.21.15.38:0/3558048566 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effac147890 con 0x7effac0d0bb0 2024-09-21T03:34:35.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:35.440+0000 7effa97fa640 1 -- 172.21.15.38:0/3558048566 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff9c00cdc0 con 0x7effac0d0bb0 2024-09-21T03:34:35.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:35.440+0000 7effb30b4640 1 -- 172.21.15.38:0/3558048566 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effac147e50 con 0x7effac0d0bb0 2024-09-21T03:34:35.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:35.441+0000 7effb30b4640 1 -- 172.21.15.38:0/3558048566 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7effac0b6d30 con 0x7effac0d0bb0 2024-09-21T03:34:35.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:35.442+0000 7effa97fa640 1 -- 172.21.15.38:0/3558048566 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7eff9c019070 con 0x7effac0d0bb0 2024-09-21T03:34:35.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:35.442+0000 7effa97fa640 1 --2- 172.21.15.38:0/3558048566 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7eff8c078bc0 0x7eff8c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:35.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:35.442+0000 7effa97fa640 1 -- 172.21.15.38:0/3558048566 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7eff9c09b660 con 0x7effac0d0bb0 2024-09-21T03:34:35.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:35.442+0000 7effb0e50640 1 --2- 172.21.15.38:0/3558048566 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7eff8c078bc0 0x7eff8c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:35.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:35.443+0000 7effb0e50640 1 --2- 172.21.15.38:0/3558048566 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7eff8c078bc0 0x7eff8c07b0d0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7effa4006fd0 tx=0x7effa4009060 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:35.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:35.447+0000 7effa97fa640 1 -- 172.21.15.38:0/3558048566 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7eff9c063240 con 0x7effac0d0bb0 2024-09-21T03:34:35.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:35.678+0000 7effb30b4640 1 -- 172.21.15.38:0/3558048566 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7effac1e5130 con 0x7effac0d0bb0 2024-09-21T03:34:36.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:36.625+0000 7effa97fa640 1 -- 172.21.15.38:0/3558048566 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v16) ==== 118+0+0 (secure 0 0 0) 0x7eff9c099e20 con 0x7effac0d0bb0 2024-09-21T03:34:36.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:36.628+0000 7effb30b4640 1 -- 172.21.15.38:0/3558048566 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7eff8c078bc0 msgr2=0x7eff8c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:36.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:36.628+0000 7effb30b4640 1 --2- 172.21.15.38:0/3558048566 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7eff8c078bc0 0x7eff8c07b0d0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7effa4006fd0 tx=0x7effa4009060 comp rx=0 tx=0).stop 2024-09-21T03:34:36.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:36.628+0000 7effb30b4640 1 -- 172.21.15.38:0/3558048566 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7effac0d0bb0 msgr2=0x7effac149780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:36.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:36.628+0000 7effb30b4640 1 --2- 172.21.15.38:0/3558048566 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7effac0d0bb0 0x7effac149780 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7eff9c00e9f0 tx=0x7eff9c00eeb0 comp rx=0 tx=0).stop 2024-09-21T03:34:36.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:36.628+0000 7effb30b4640 1 -- 172.21.15.38:0/3558048566 shutdown_connections 2024-09-21T03:34:36.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:36.628+0000 7effb30b4640 1 --2- 172.21.15.38:0/3558048566 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7effac15ca90 0x7effac149cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:36.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:36.628+0000 7effb30b4640 1 --2- 172.21.15.38:0/3558048566 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7effac0d0bb0 0x7effac149780 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:36.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:36.628+0000 7effb30b4640 1 --2- 172.21.15.38:0/3558048566 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7eff8c078bc0 0x7eff8c07b0d0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:36.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:36.628+0000 7effb30b4640 1 --2- 172.21.15.38:0/3558048566 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7effac0ccc40 0x7effac0b3840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:36.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:36.628+0000 7effb30b4640 1 -- 172.21.15.38:0/3558048566 >> 172.21.15.38:0/3558048566 conn(0x7effac0b46e0 msgr2=0x7effac0cd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:36.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:36.628+0000 7effb30b4640 1 -- 172.21.15.38:0/3558048566 shutdown_connections 2024-09-21T03:34:36.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:36.628+0000 7effb30b4640 1 -- 172.21.15.38:0/3558048566 wait complete. 2024-09-21T03:34:36.644 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-21T03:34:36.645 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-21T03:34:36.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:36.769+0000 7f1fd1fcc640 1 -- 172.21.15.38:0/2844199758 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1fcc0d0170 msgr2=0x7f1fcc0d0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:36.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:36.769+0000 7f1fd1fcc640 1 --2- 172.21.15.38:0/2844199758 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1fcc0d0170 0x7f1fcc0d0be0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f1fb4009820 tx=0x7f1fb402a410 comp rx=0 tx=0).stop 2024-09-21T03:34:36.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:36.770+0000 7f1fd1fcc640 1 -- 172.21.15.38:0/2844199758 shutdown_connections 2024-09-21T03:34:36.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:36.770+0000 7f1fd1fcc640 1 --2- 172.21.15.38:0/2844199758 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1fcc0d0170 0x7f1fcc0d0be0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:36.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:36.770+0000 7f1fd1fcc640 1 -- 172.21.15.38:0/2844199758 >> 172.21.15.38:0/2844199758 conn(0x7f1fcc09d830 msgr2=0x7f1fcc0b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:36.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:36.770+0000 7f1fd1fcc640 1 -- 172.21.15.38:0/2844199758 shutdown_connections 2024-09-21T03:34:36.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:36.770+0000 7f1fd1fcc640 1 -- 172.21.15.38:0/2844199758 wait complete. 2024-09-21T03:34:36.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:36.771+0000 7f1fd1fcc640 1 Processor -- start 2024-09-21T03:34:36.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:36.772+0000 7f1fd1fcc640 1 -- start start 2024-09-21T03:34:36.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:36.773+0000 7f1fd1fcc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1fcc0d0170 0x7f1fcc0b2780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:36.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:36.773+0000 7f1fd1fcc640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1fcc0d1120 0x7f1fcc0b2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:36.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:36.773+0000 7f1fd1fcc640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1fcc15ee80 0x7f1fcc06ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:36.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:36.773+0000 7f1fd1fcc640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f1fcc0b6910 con 0x7f1fcc0d0170 2024-09-21T03:34:36.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:36.773+0000 7f1fd1fcc640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f1fcc0b6790 con 0x7f1fcc15ee80 2024-09-21T03:34:36.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:36.773+0000 7f1fd1fcc640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f1fcc0b6a90 con 0x7f1fcc0d1120 2024-09-21T03:34:36.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:36.773+0000 7f1fcb7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1fcc0d0170 0x7f1fcc0b2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:36.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:36.773+0000 7f1fcb7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1fcc0d0170 0x7f1fcc0b2780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:33608/0 (socket says 172.21.15.38:33608) 2024-09-21T03:34:36.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:36.773+0000 7f1fcb7fe640 1 -- 172.21.15.38:0/2141027464 learned_addr learned my addr 172.21.15.38:0/2141027464 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:34:36.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:36.773+0000 7f1fc2ffd640 1 --2- 172.21.15.38:0/2141027464 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1fcc0d1120 0x7f1fcc0b2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:36.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:36.773+0000 7f1fcbfff640 1 --2- 172.21.15.38:0/2141027464 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1fcc15ee80 0x7f1fcc06ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:36.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:36.773+0000 7f1fcb7fe640 1 -- 172.21.15.38:0/2141027464 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1fcc0d1120 msgr2=0x7f1fcc0b2cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:36.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:36.773+0000 7f1fcb7fe640 1 --2- 172.21.15.38:0/2141027464 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1fcc0d1120 0x7f1fcc0b2cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:36.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:36.773+0000 7f1fcb7fe640 1 -- 172.21.15.38:0/2141027464 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1fcc15ee80 msgr2=0x7f1fcc06ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:36.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:36.773+0000 7f1fcb7fe640 1 --2- 172.21.15.38:0/2141027464 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1fcc15ee80 0x7f1fcc06ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:36.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:36.773+0000 7f1fcb7fe640 1 -- 172.21.15.38:0/2141027464 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fcc06e360 con 0x7f1fcc0d0170 2024-09-21T03:34:36.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:36.773+0000 7f1fc2ffd640 1 --2- 172.21.15.38:0/2141027464 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1fcc0d1120 0x7f1fcc0b2cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:34:36.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:36.773+0000 7f1fcb7fe640 1 --2- 172.21.15.38:0/2141027464 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1fcc0d0170 0x7f1fcc0b2780 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f1fbc00c500 tx=0x7f1fbc00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:36.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:36.774+0000 7f1fc8ff9640 1 -- 172.21.15.38:0/2141027464 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fbc016020 con 0x7f1fcc0d0170 2024-09-21T03:34:36.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:36.774+0000 7f1fc8ff9640 1 -- 172.21.15.38:0/2141027464 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1fbc012920 con 0x7f1fcc0d0170 2024-09-21T03:34:36.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:36.774+0000 7f1fd1fcc640 1 -- 172.21.15.38:0/2141027464 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fcc15c230 con 0x7f1fcc0d0170 2024-09-21T03:34:36.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:36.774+0000 7f1fc8ff9640 1 -- 172.21.15.38:0/2141027464 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fbc012d50 con 0x7f1fcc0d0170 2024-09-21T03:34:36.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:36.774+0000 7f1fd1fcc640 1 -- 172.21.15.38:0/2141027464 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fcc06e620 con 0x7f1fcc0d0170 2024-09-21T03:34:36.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:36.775+0000 7f1fd1fcc640 1 -- 172.21.15.38:0/2141027464 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1fcc15bd50 con 0x7f1fcc0d0170 2024-09-21T03:34:36.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:36.776+0000 7f1fc8ff9640 1 -- 172.21.15.38:0/2141027464 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f1fbc019020 con 0x7f1fcc0d0170 2024-09-21T03:34:36.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:36.776+0000 7f1fc8ff9640 1 --2- 172.21.15.38:0/2141027464 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1fac078b90 0x7f1fac07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:36.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:36.776+0000 7f1fc8ff9640 1 -- 172.21.15.38:0/2141027464 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7f1fbc09aca0 con 0x7f1fcc0d0170 2024-09-21T03:34:36.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:36.776+0000 7f1fc2ffd640 1 --2- 172.21.15.38:0/2141027464 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1fac078b90 0x7f1fac07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:36.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:36.777+0000 7f1fc2ffd640 1 --2- 172.21.15.38:0/2141027464 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1fac078b90 0x7f1fac07b0a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f1fb4034000 tx=0x7f1fb4004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:36.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:36.781+0000 7f1fc8ff9640 1 -- 172.21.15.38:0/2141027464 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f1fcc15bd50 con 0x7f1fcc0d0170 2024-09-21T03:34:37.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:37.008+0000 7f1fd1fcc640 1 -- 172.21.15.38:0/2141027464 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f1fcc0d0be0 con 0x7f1fcc0d0170 2024-09-21T03:34:37.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:37.635+0000 7f1fc8ff9640 1 -- 172.21.15.38:0/2141027464 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v17) ==== 122+0+0 (secure 0 0 0) 0x7f1fbc020030 con 0x7f1fcc0d0170 2024-09-21T03:34:37.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:37.638+0000 7f1fd1fcc640 1 -- 172.21.15.38:0/2141027464 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1fac078b90 msgr2=0x7f1fac07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:37.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:37.638+0000 7f1fd1fcc640 1 --2- 172.21.15.38:0/2141027464 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1fac078b90 0x7f1fac07b0a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f1fb4034000 tx=0x7f1fb4004730 comp rx=0 tx=0).stop 2024-09-21T03:34:37.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:37.638+0000 7f1fd1fcc640 1 -- 172.21.15.38:0/2141027464 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1fcc0d0170 msgr2=0x7f1fcc0b2780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:37.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:37.638+0000 7f1fd1fcc640 1 --2- 172.21.15.38:0/2141027464 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1fcc0d0170 0x7f1fcc0b2780 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f1fbc00c500 tx=0x7f1fbc00c9c0 comp rx=0 tx=0).stop 2024-09-21T03:34:37.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:37.638+0000 7f1fd1fcc640 1 -- 172.21.15.38:0/2141027464 shutdown_connections 2024-09-21T03:34:37.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:37.638+0000 7f1fd1fcc640 1 --2- 172.21.15.38:0/2141027464 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1fcc15ee80 0x7f1fcc06ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:37.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:37.638+0000 7f1fd1fcc640 1 --2- 172.21.15.38:0/2141027464 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1fac078b90 0x7f1fac07b0a0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:37.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:37.638+0000 7f1fd1fcc640 1 --2- 172.21.15.38:0/2141027464 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1fcc0d1120 0x7f1fcc0b2cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:37.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:37.638+0000 7f1fd1fcc640 1 --2- 172.21.15.38:0/2141027464 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1fcc0d0170 0x7f1fcc0b2780 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:37.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:37.638+0000 7f1fd1fcc640 1 -- 172.21.15.38:0/2141027464 >> 172.21.15.38:0/2141027464 conn(0x7f1fcc09d830 msgr2=0x7f1fcc15d320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:37.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:37.639+0000 7f1fd1fcc640 1 -- 172.21.15.38:0/2141027464 shutdown_connections 2024-09-21T03:34:37.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:37.639+0000 7f1fd1fcc640 1 -- 172.21.15.38:0/2141027464 wait complete. 2024-09-21T03:34:37.654 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-21T03:34:37.655 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:34:37.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:37.777+0000 7eff49dda640 1 -- 172.21.15.38:0/1630003351 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7eff4415fc70 msgr2=0x7eff440b40c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:37.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:37.777+0000 7eff49dda640 1 --2- 172.21.15.38:0/1630003351 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7eff4415fc70 0x7eff440b40c0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7eff3c005fd0 tx=0x7eff3c02aa10 comp rx=0 tx=0).stop 2024-09-21T03:34:37.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:37.778+0000 7eff49dda640 1 -- 172.21.15.38:0/1630003351 shutdown_connections 2024-09-21T03:34:37.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:37.778+0000 7eff49dda640 1 --2- 172.21.15.38:0/1630003351 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7eff4415fc70 0x7eff440b40c0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:37.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:37.778+0000 7eff49dda640 1 --2- 172.21.15.38:0/1630003351 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7eff440cee70 0x7eff4415b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:37.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:37.778+0000 7eff49dda640 1 -- 172.21.15.38:0/1630003351 >> 172.21.15.38:0/1630003351 conn(0x7eff44073e30 msgr2=0x7eff44074240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:37.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:37.778+0000 7eff49dda640 1 -- 172.21.15.38:0/1630003351 shutdown_connections 2024-09-21T03:34:37.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:37.778+0000 7eff49dda640 1 -- 172.21.15.38:0/1630003351 wait complete. 2024-09-21T03:34:37.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:37.779+0000 7eff49dda640 1 Processor -- start 2024-09-21T03:34:37.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:37.780+0000 7eff49dda640 1 -- start start 2024-09-21T03:34:37.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:37.781+0000 7eff49dda640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7eff440cee70 0x7eff441f9510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:37.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:37.781+0000 7eff49dda640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7eff4415f150 0x7eff4415e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:37.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:37.781+0000 7eff49dda640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7eff4415fc70 0x7eff4406cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:37.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:37.781+0000 7eff49dda640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7eff440b4a70 con 0x7eff440cee70 2024-09-21T03:34:37.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:37.781+0000 7eff49dda640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7eff440b48f0 con 0x7eff4415fc70 2024-09-21T03:34:37.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:37.781+0000 7eff49dda640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7eff440b4bf0 con 0x7eff4415f150 2024-09-21T03:34:37.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:37.781+0000 7eff437fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7eff440cee70 0x7eff441f9510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:37.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:37.781+0000 7eff437fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7eff440cee70 0x7eff441f9510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:33624/0 (socket says 172.21.15.38:33624) 2024-09-21T03:34:37.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:37.781+0000 7eff437fe640 1 -- 172.21.15.38:0/4127681720 learned_addr learned my addr 172.21.15.38:0/4127681720 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:34:37.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:37.781+0000 7eff42ffd640 1 --2- 172.21.15.38:0/4127681720 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7eff4415f150 0x7eff4415e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:37.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:37.781+0000 7eff43fff640 1 --2- 172.21.15.38:0/4127681720 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7eff4415fc70 0x7eff4406cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:37.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:37.781+0000 7eff437fe640 1 -- 172.21.15.38:0/4127681720 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7eff4415f150 msgr2=0x7eff4415e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:37.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:37.781+0000 7eff437fe640 1 --2- 172.21.15.38:0/4127681720 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7eff4415f150 0x7eff4415e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:37.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:37.781+0000 7eff437fe640 1 -- 172.21.15.38:0/4127681720 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7eff4415fc70 msgr2=0x7eff4406cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:37.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:37.781+0000 7eff437fe640 1 --2- 172.21.15.38:0/4127681720 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7eff4415fc70 0x7eff4406cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:37.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:37.782+0000 7eff437fe640 1 -- 172.21.15.38:0/4127681720 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff4415d430 con 0x7eff440cee70 2024-09-21T03:34:37.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:37.782+0000 7eff437fe640 1 --2- 172.21.15.38:0/4127681720 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7eff440cee70 0x7eff441f9510 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7eff3400d700 tx=0x7eff3400dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:37.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:37.782+0000 7eff2bfff640 1 -- 172.21.15.38:0/4127681720 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff34017070 con 0x7eff440cee70 2024-09-21T03:34:37.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:37.782+0000 7eff2bfff640 1 -- 172.21.15.38:0/4127681720 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7eff34005310 con 0x7eff440cee70 2024-09-21T03:34:37.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:37.782+0000 7eff2bfff640 1 -- 172.21.15.38:0/4127681720 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff340097c0 con 0x7eff440cee70 2024-09-21T03:34:37.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:37.782+0000 7eff49dda640 1 -- 172.21.15.38:0/4127681720 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff4415d710 con 0x7eff440cee70 2024-09-21T03:34:37.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:37.782+0000 7eff49dda640 1 -- 172.21.15.38:0/4127681720 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff4415dcd0 con 0x7eff440cee70 2024-09-21T03:34:37.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:37.783+0000 7eff49dda640 1 -- 172.21.15.38:0/4127681720 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff440b50a0 con 0x7eff440cee70 2024-09-21T03:34:37.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:37.784+0000 7eff2bfff640 1 -- 172.21.15.38:0/4127681720 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7eff34009960 con 0x7eff440cee70 2024-09-21T03:34:37.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:37.784+0000 7eff2bfff640 1 --2- 172.21.15.38:0/4127681720 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7eff30078b90 0x7eff3007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:37.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:37.784+0000 7eff2bfff640 1 -- 172.21.15.38:0/4127681720 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7eff3409aee0 con 0x7eff440cee70 2024-09-21T03:34:37.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:37.784+0000 7eff42ffd640 1 --2- 172.21.15.38:0/4127681720 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7eff30078b90 0x7eff3007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:37.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:37.785+0000 7eff42ffd640 1 --2- 172.21.15.38:0/4127681720 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7eff30078b90 0x7eff3007b0a0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7eff440b0480 tx=0x7eff38009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:37.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:37.789+0000 7eff2bfff640 1 -- 172.21.15.38:0/4127681720 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7eff34062b40 con 0x7eff440cee70 2024-09-21T03:34:38.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:38.014+0000 7eff49dda640 1 -- 172.21.15.38:0/4127681720 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7eff440b5850 con 0x7eff440cee70 2024-09-21T03:34:38.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:38.015+0000 7eff2bfff640 1 -- 172.21.15.38:0/4127681720 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5229 (secure 0 0 0) 0x7eff34067570 con 0x7eff440cee70 2024-09-21T03:34:38.016 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:34:38.017 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":17,"btime":"2024-09-21T03:34:37:631111+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4757,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/1905955399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":1905955399},{"type":"v1","addr":"172.21.15.195:6836","nonce":1905955399}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4936,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/2989374027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":2989374027},{"type":"v1","addr":"172.21.15.195:6837","nonce":2989374027}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4942,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/3531731501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6833","nonce":3531731501},{"type":"v1","addr":"172.21.15.38:6835","nonce":3531731501}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:34:34.009269+0000","modified":"2024-09-21T03:34:37.009380+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5013},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5013":{"gid":5013,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.38:6834/1865253656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":1865253656},{"type":"v1","addr":"172.21.15.38:6834","nonce":1865253656}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5013,"qdb_cluster":[5013]},"id":2}]} 2024-09-21T03:34:38.017 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 17 2024-09-21T03:34:38.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:38.017+0000 7eff49dda640 1 -- 172.21.15.38:0/4127681720 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7eff30078b90 msgr2=0x7eff3007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:38.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:38.017+0000 7eff49dda640 1 --2- 172.21.15.38:0/4127681720 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7eff30078b90 0x7eff3007b0a0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7eff440b0480 tx=0x7eff38009040 comp rx=0 tx=0).stop 2024-09-21T03:34:38.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:38.017+0000 7eff49dda640 1 -- 172.21.15.38:0/4127681720 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7eff440cee70 msgr2=0x7eff441f9510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:38.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:38.017+0000 7eff49dda640 1 --2- 172.21.15.38:0/4127681720 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7eff440cee70 0x7eff441f9510 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7eff3400d700 tx=0x7eff3400dbc0 comp rx=0 tx=0).stop 2024-09-21T03:34:38.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:38.017+0000 7eff49dda640 1 -- 172.21.15.38:0/4127681720 shutdown_connections 2024-09-21T03:34:38.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:38.017+0000 7eff49dda640 1 --2- 172.21.15.38:0/4127681720 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7eff4415fc70 0x7eff4406cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:38.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:38.017+0000 7eff49dda640 1 --2- 172.21.15.38:0/4127681720 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7eff30078b90 0x7eff3007b0a0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:38.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:38.017+0000 7eff49dda640 1 --2- 172.21.15.38:0/4127681720 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7eff4415f150 0x7eff4415e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:38.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:38.017+0000 7eff49dda640 1 --2- 172.21.15.38:0/4127681720 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7eff440cee70 0x7eff441f9510 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:38.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:38.017+0000 7eff49dda640 1 -- 172.21.15.38:0/4127681720 >> 172.21.15.38:0/4127681720 conn(0x7eff44073e30 msgr2=0x7eff440b0aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:38.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:38.017+0000 7eff49dda640 1 -- 172.21.15.38:0/4127681720 shutdown_connections 2024-09-21T03:34:38.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:38.017+0000 7eff49dda640 1 -- 172.21.15.38:0/4127681720 wait complete. 2024-09-21T03:34:38.030 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T03:34:38.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:38.147+0000 7f091bf94640 1 -- 172.21.15.38:0/1970448606 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f09140b45d0 msgr2=0x7f09140cdfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:38.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:38.147+0000 7f091bf94640 1 --2- 172.21.15.38:0/1970448606 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f09140b45d0 0x7f09140cdfe0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f09140b4ab0 tx=0x7f090c02aa30 comp rx=0 tx=0).stop 2024-09-21T03:34:38.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:38.147+0000 7f091bf94640 1 -- 172.21.15.38:0/1970448606 shutdown_connections 2024-09-21T03:34:38.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:38.147+0000 7f091bf94640 1 --2- 172.21.15.38:0/1970448606 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f09140b45d0 0x7f09140cdfe0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:38.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:38.147+0000 7f091bf94640 1 --2- 172.21.15.38:0/1970448606 >> v2:172.21.15.195:3300/0 conn(0x7f09140cc930 0x7f0914155570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:38.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:38.147+0000 7f091bf94640 1 -- 172.21.15.38:0/1970448606 >> 172.21.15.38:0/1970448606 conn(0x7f09140cef40 msgr2=0x7f09140cf350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:38.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:38.147+0000 7f091bf94640 1 -- 172.21.15.38:0/1970448606 shutdown_connections 2024-09-21T03:34:38.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:38.147+0000 7f091bf94640 1 -- 172.21.15.38:0/1970448606 wait complete. 2024-09-21T03:34:38.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:38.149+0000 7f091bf94640 1 Processor -- start 2024-09-21T03:34:38.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:38.149+0000 7f091bf94640 1 -- start start 2024-09-21T03:34:38.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:38.151+0000 7f091bf94640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f091406f7b0 0x7f09140c8930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:38.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:38.151+0000 7f091bf94640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f09140cc930 0x7f09141f26c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:38.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:38.151+0000 7f091bf94640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f09140c9340 0x7f09140c9750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:38.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:38.151+0000 7f091bf94640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f09140ce520 con 0x7f09140c9340 2024-09-21T03:34:38.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:38.151+0000 7f091bf94640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f091406fdd0 con 0x7f09140cc930 2024-09-21T03:34:38.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:38.151+0000 7f091bf94640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f09140ce6a0 con 0x7f091406f7b0 2024-09-21T03:34:38.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:38.151+0000 7f091a531640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f09140c9340 0x7f09140c9750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:38.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:38.151+0000 7f091a531640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f09140c9340 0x7f09140c9750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:33640/0 (socket says 172.21.15.38:33640) 2024-09-21T03:34:38.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:38.151+0000 7f091a531640 1 -- 172.21.15.38:0/631732662 learned_addr learned my addr 172.21.15.38:0/631732662 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:34:38.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:38.151+0000 7f091952f640 1 --2- 172.21.15.38:0/631732662 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f09140cc930 0x7f09141f26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:38.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:38.151+0000 7f0919d30640 1 --2- 172.21.15.38:0/631732662 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f091406f7b0 0x7f09140c8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:38.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:38.151+0000 7f091a531640 1 -- 172.21.15.38:0/631732662 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f091406f7b0 msgr2=0x7f09140c8930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:38.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:38.151+0000 7f091a531640 1 --2- 172.21.15.38:0/631732662 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f091406f7b0 0x7f09140c8930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:38.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:38.151+0000 7f091a531640 1 -- 172.21.15.38:0/631732662 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f09140cc930 msgr2=0x7f09141f26c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:38.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:38.151+0000 7f091a531640 1 --2- 172.21.15.38:0/631732662 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f09140cc930 0x7f09141f26c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:38.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:38.151+0000 7f091a531640 1 -- 172.21.15.38:0/631732662 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09140cb2e0 con 0x7f09140c9340 2024-09-21T03:34:38.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:38.151+0000 7f091a531640 1 --2- 172.21.15.38:0/631732662 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f09140c9340 0x7f09140c9750 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f090c009d60 tx=0x7f090c003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:38.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:38.152+0000 7f090a7fc640 1 -- 172.21.15.38:0/631732662 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f090c033050 con 0x7f09140c9340 2024-09-21T03:34:38.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:38.152+0000 7f090a7fc640 1 -- 172.21.15.38:0/631732662 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f090c004450 con 0x7f09140c9340 2024-09-21T03:34:38.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:38.152+0000 7f091bf94640 1 -- 172.21.15.38:0/631732662 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09140c9c90 con 0x7f09140c9340 2024-09-21T03:34:38.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:38.152+0000 7f090a7fc640 1 -- 172.21.15.38:0/631732662 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f090c038450 con 0x7f09140c9340 2024-09-21T03:34:38.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:38.152+0000 7f091bf94640 1 -- 172.21.15.38:0/631732662 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09141f59f0 con 0x7f09140c9340 2024-09-21T03:34:38.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:38.153+0000 7f091bf94640 1 -- 172.21.15.38:0/631732662 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09140ceaf0 con 0x7f09140c9340 2024-09-21T03:34:38.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:38.153+0000 7f090a7fc640 1 -- 172.21.15.38:0/631732662 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f090c036070 con 0x7f09140c9340 2024-09-21T03:34:38.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:38.154+0000 7f090a7fc640 1 --2- 172.21.15.38:0/631732662 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f08f8078bc0 0x7f08f807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:38.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:38.154+0000 7f090a7fc640 1 -- 172.21.15.38:0/631732662 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7f090c0bc1b0 con 0x7f09140c9340 2024-09-21T03:34:38.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:38.154+0000 7f0919d30640 1 --2- 172.21.15.38:0/631732662 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f08f8078bc0 0x7f08f807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:38.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:38.155+0000 7f0919d30640 1 --2- 172.21.15.38:0/631732662 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f08f8078bc0 0x7f08f807b0d0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f0910009b30 tx=0x7f0910005f90 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:38.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:38.159+0000 7f090a7fc640 1 -- 172.21.15.38:0/631732662 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f09140c9c90 con 0x7f09140c9340 2024-09-21T03:34:38.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:38.347+0000 7f091bf94640 1 -- 172.21.15.38:0/631732662 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f09140b8260 con 0x7f09140c9340 2024-09-21T03:34:38.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:38.348+0000 7f090a7fc640 1 -- 172.21.15.38:0/631732662 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) ==== 74+0+17026 (secure 0 0 0) 0x7f09140cb2e0 con 0x7f09140c9340 2024-09-21T03:34:38.350 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:34:38.350 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":40,"fsid":"9d50d66f-656b-4fe2-8e3b-fe53b79c4d43","created":"2024-09-21T03:33:10.677865+0000","modified":"2024-09-21T03:34:35.620283+0000","last_up_change":"2024-09-21T03:33:24.023421+0000","last_in_change":"2024-09-21T03:33:14.296193+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T03:33:22.992573+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-09-21T03:34:32.080636+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-09-21T03:34:32.959110+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dbab183f-7e63-48b8-960a-ae55fd5bf565","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6801","nonce":2381639433}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6803","nonce":2381639433}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6807","nonce":2381639433}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6805","nonce":2381639433}]},"public_addr":"172.21.15.38:6801/2381639433","cluster_addr":"172.21.15.38:6803/2381639433","heartbeat_back_addr":"172.21.15.38:6807/2381639433","heartbeat_front_addr":"172.21.15.38:6805/2381639433","state":["exists","up"]},{"osd":1,"uuid":"fa1c0e8a-ebc1-4359-8146-bd5342abbc16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6817","nonce":1491818732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6819","nonce":1491818732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6823","nonce":1491818732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6821","nonce":1491818732}]},"public_addr":"172.21.15.38:6817/1491818732","cluster_addr":"172.21.15.38:6819/1491818732","heartbeat_back_addr":"172.21.15.38:6823/1491818732","heartbeat_front_addr":"172.21.15.38:6821/1491818732","state":["exists","up"]},{"osd":2,"uuid":"2ed3cd2d-385d-4fc6-9869-214552a84803","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6825","nonce":2577858902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6827","nonce":2577858902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6831","nonce":2577858902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6829","nonce":2577858902}]},"public_addr":"172.21.15.38:6825/2577858902","cluster_addr":"172.21.15.38:6827/2577858902","heartbeat_back_addr":"172.21.15.38:6831/2577858902","heartbeat_front_addr":"172.21.15.38:6829/2577858902","state":["exists","up"]},{"osd":3,"uuid":"62d1d95e-919d-4250-be71-1d34cfa70dbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6809","nonce":1286810294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6811","nonce":1286810294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6815","nonce":1286810294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6813","nonce":1286810294}]},"public_addr":"172.21.15.38:6809/1286810294","cluster_addr":"172.21.15.38:6811/1286810294","heartbeat_back_addr":"172.21.15.38:6815/1286810294","heartbeat_front_addr":"172.21.15.38:6813/1286810294","state":["exists","up"]},{"osd":4,"uuid":"7c9f9314-99d7-4d73-8392-37809a9f54b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6803","nonce":3401837571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6805","nonce":3401837571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6809","nonce":3401837571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6807","nonce":3401837571}]},"public_addr":"172.21.15.195:6803/3401837571","cluster_addr":"172.21.15.195:6805/3401837571","heartbeat_back_addr":"172.21.15.195:6809/3401837571","heartbeat_front_addr":"172.21.15.195:6807/3401837571","state":["exists","up"]},{"osd":5,"uuid":"bde4fe9f-8414-4bba-bf4a-43973d1976bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6811","nonce":2080960180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6813","nonce":2080960180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6817","nonce":2080960180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6815","nonce":2080960180}]},"public_addr":"172.21.15.195:6811/2080960180","cluster_addr":"172.21.15.195:6813/2080960180","heartbeat_back_addr":"172.21.15.195:6817/2080960180","heartbeat_front_addr":"172.21.15.195:6815/2080960180","state":["exists","up"]},{"osd":6,"uuid":"6ee6d7f4-1a5f-4a4c-ac16-48fe05f6a002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6827","nonce":4055825675}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6829","nonce":4055825675}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6833","nonce":4055825675}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6831","nonce":4055825675}]},"public_addr":"172.21.15.195:6827/4055825675","cluster_addr":"172.21.15.195:6829/4055825675","heartbeat_back_addr":"172.21.15.195:6833/4055825675","heartbeat_front_addr":"172.21.15.195:6831/4055825675","state":["exists","up"]},{"osd":7,"uuid":"efeb5f51-329a-4484-8290-92e7d3503d05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6819","nonce":1606106342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6821","nonce":1606106342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6825","nonce":1606106342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6823","nonce":1606106342}]},"public_addr":"172.21.15.195:6819/1606106342","cluster_addr":"172.21.15.195:6821/1606106342","heartbeat_back_addr":"172.21.15.195:6825/1606106342","heartbeat_front_addr":"172.21.15.195:6823/1606106342","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.901343+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:21.115003+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.901623+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.835238+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.132199+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:19.817560+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.829690+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:21.087300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T03:34:38.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:38.350+0000 7f091bf94640 1 -- 172.21.15.38:0/631732662 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f08f8078bc0 msgr2=0x7f08f807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:38.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:38.350+0000 7f091bf94640 1 --2- 172.21.15.38:0/631732662 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f08f8078bc0 0x7f08f807b0d0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f0910009b30 tx=0x7f0910005f90 comp rx=0 tx=0).stop 2024-09-21T03:34:38.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:38.350+0000 7f091bf94640 1 -- 172.21.15.38:0/631732662 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f09140c9340 msgr2=0x7f09140c9750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:38.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:38.350+0000 7f091bf94640 1 --2- 172.21.15.38:0/631732662 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f09140c9340 0x7f09140c9750 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f090c009d60 tx=0x7f090c003f10 comp rx=0 tx=0).stop 2024-09-21T03:34:38.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:38.350+0000 7f091bf94640 1 -- 172.21.15.38:0/631732662 shutdown_connections 2024-09-21T03:34:38.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:38.350+0000 7f091bf94640 1 --2- 172.21.15.38:0/631732662 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f09140c9340 0x7f09140c9750 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:38.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:38.350+0000 7f091bf94640 1 --2- 172.21.15.38:0/631732662 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f09140cc930 0x7f09141f26c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:38.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:38.350+0000 7f091bf94640 1 --2- 172.21.15.38:0/631732662 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f08f8078bc0 0x7f08f807b0d0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:38.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:38.350+0000 7f091bf94640 1 --2- 172.21.15.38:0/631732662 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f091406f7b0 0x7f09140c8930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:38.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:38.350+0000 7f091bf94640 1 -- 172.21.15.38:0/631732662 >> 172.21.15.38:0/631732662 conn(0x7f09140cef40 msgr2=0x7f09141603b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:38.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:38.350+0000 7f091bf94640 1 -- 172.21.15.38:0/631732662 shutdown_connections 2024-09-21T03:34:38.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:38.350+0000 7f091bf94640 1 -- 172.21.15.38:0/631732662 wait complete. 2024-09-21T03:34:38.364 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-21T03:34:38.364 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T03:34:38.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:38.479+0000 7f9e320a4640 1 -- 172.21.15.38:0/3260914752 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9e2c06ec30 msgr2=0x7f9e2c0b3fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:38.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:38.479+0000 7f9e320a4640 1 --2- 172.21.15.38:0/3260914752 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9e2c06ec30 0x7f9e2c0b3fc0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f9e2c0b3af0 tx=0x7f9e2002a560 comp rx=0 tx=0).stop 2024-09-21T03:34:38.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:38.479+0000 7f9e320a4640 1 -- 172.21.15.38:0/3260914752 shutdown_connections 2024-09-21T03:34:38.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:38.479+0000 7f9e320a4640 1 --2- 172.21.15.38:0/3260914752 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9e2c0b4500 0x7f9e2c073d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:38.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:38.479+0000 7f9e320a4640 1 --2- 172.21.15.38:0/3260914752 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9e2c06ec30 0x7f9e2c0b3fc0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:38.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:38.479+0000 7f9e320a4640 1 --2- 172.21.15.38:0/3260914752 >> v2:172.21.15.195:3300/0 conn(0x7f9e2c0d0b40 0x7f9e2c06e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:38.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:38.479+0000 7f9e320a4640 1 -- 172.21.15.38:0/3260914752 >> 172.21.15.38:0/3260914752 conn(0x7f9e2c0b2cc0 msgr2=0x7f9e2c0b30d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:38.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:38.479+0000 7f9e320a4640 1 -- 172.21.15.38:0/3260914752 shutdown_connections 2024-09-21T03:34:38.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:38.479+0000 7f9e320a4640 1 -- 172.21.15.38:0/3260914752 wait complete. 2024-09-21T03:34:38.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:38.481+0000 7f9e320a4640 1 Processor -- start 2024-09-21T03:34:38.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:38.481+0000 7f9e320a4640 1 -- start start 2024-09-21T03:34:38.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:38.483+0000 7f9e320a4640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9e2c06ec30 0x7f9e2c0b0040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:38.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:38.483+0000 7f9e320a4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9e2c0b4500 0x7f9e2c1f5560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:38.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:38.483+0000 7f9e320a4640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9e2c0d0b40 0x7f9e2c14f960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:38.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:38.483+0000 7f9e320a4640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f9e2c15b5d0 con 0x7f9e2c0b4500 2024-09-21T03:34:38.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:38.483+0000 7f9e320a4640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f9e2c0745b0 con 0x7f9e2c0d0b40 2024-09-21T03:34:38.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:38.483+0000 7f9e320a4640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f9e2c15b750 con 0x7f9e2c06ec30 2024-09-21T03:34:38.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:38.483+0000 7f9e2affd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9e2c0b4500 0x7f9e2c1f5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:38.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:38.483+0000 7f9e2affd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9e2c0b4500 0x7f9e2c1f5560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:33658/0 (socket says 172.21.15.38:33658) 2024-09-21T03:34:38.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:38.483+0000 7f9e2affd640 1 -- 172.21.15.38:0/1746066528 learned_addr learned my addr 172.21.15.38:0/1746066528 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:34:38.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:38.483+0000 7f9e2b7fe640 1 --2- 172.21.15.38:0/1746066528 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9e2c06ec30 0x7f9e2c0b0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:38.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:38.483+0000 7f9e2bfff640 1 --2- 172.21.15.38:0/1746066528 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9e2c0d0b40 0x7f9e2c14f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:38.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:38.483+0000 7f9e2affd640 1 -- 172.21.15.38:0/1746066528 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9e2c06ec30 msgr2=0x7f9e2c0b0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:38.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:38.483+0000 7f9e2affd640 1 --2- 172.21.15.38:0/1746066528 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9e2c06ec30 0x7f9e2c0b0040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:38.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:38.483+0000 7f9e2affd640 1 -- 172.21.15.38:0/1746066528 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9e2c0d0b40 msgr2=0x7f9e2c14f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:38.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:38.483+0000 7f9e2affd640 1 --2- 172.21.15.38:0/1746066528 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9e2c0d0b40 0x7f9e2c14f960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:38.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:38.483+0000 7f9e2affd640 1 -- 172.21.15.38:0/1746066528 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e2c1f6070 con 0x7f9e2c0b4500 2024-09-21T03:34:38.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:38.483+0000 7f9e2affd640 1 --2- 172.21.15.38:0/1746066528 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9e2c0b4500 0x7f9e2c1f5560 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f9e200099c0 tx=0x7f9e200038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:38.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:38.484+0000 7f9e13fff640 1 -- 172.21.15.38:0/1746066528 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e2003e030 con 0x7f9e2c0b4500 2024-09-21T03:34:38.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:38.484+0000 7f9e13fff640 1 -- 172.21.15.38:0/1746066528 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9e20003ca0 con 0x7f9e2c0b4500 2024-09-21T03:34:38.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:38.484+0000 7f9e320a4640 1 -- 172.21.15.38:0/1746066528 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e2c0c6140 con 0x7f9e2c0b4500 2024-09-21T03:34:38.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:38.484+0000 7f9e13fff640 1 -- 172.21.15.38:0/1746066528 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e20035040 con 0x7f9e2c0b4500 2024-09-21T03:34:38.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:38.484+0000 7f9e320a4640 1 -- 172.21.15.38:0/1746066528 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e2c1f5aa0 con 0x7f9e2c0b4500 2024-09-21T03:34:38.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:38.485+0000 7f9e320a4640 1 -- 172.21.15.38:0/1746066528 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e2c0c5f30 con 0x7f9e2c0b4500 2024-09-21T03:34:38.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:38.486+0000 7f9e13fff640 1 -- 172.21.15.38:0/1746066528 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f9e20041030 con 0x7f9e2c0b4500 2024-09-21T03:34:38.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:38.486+0000 7f9e13fff640 1 --2- 172.21.15.38:0/1746066528 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9e18078b90 0x7f9e1807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:38.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:38.486+0000 7f9e13fff640 1 -- 172.21.15.38:0/1746066528 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7f9e200bb8d0 con 0x7f9e2c0b4500 2024-09-21T03:34:38.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:38.486+0000 7f9e2b7fe640 1 --2- 172.21.15.38:0/1746066528 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9e18078b90 0x7f9e1807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:38.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:38.487+0000 7f9e2b7fe640 1 --2- 172.21.15.38:0/1746066528 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9e18078b90 0x7f9e1807b0a0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f9e1c009a40 tx=0x7f9e1c005f90 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:38.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:38.494+0000 7f9e13fff640 1 -- 172.21.15.38:0/1746066528 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f9e20083c60 con 0x7f9e2c0b4500 2024-09-21T03:34:38.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:38.681+0000 7f9e320a4640 1 -- 172.21.15.38:0/1746066528 --> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9e2c1f5e00 con 0x7f9e18078b90 2024-09-21T03:34:38.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:38.686+0000 7f9e13fff640 1 -- 172.21.15.38:0/1746066528 <== mgr.4117 v2:172.21.15.195:6800/1631901344 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+244556 (secure 0 0 0) 0x7f9e2c1f5e00 con 0x7f9e18078b90 2024-09-21T03:34:38.690 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:34:38.691 INFO:teuthology.orchestra.run.smithi038.stderr:dumped all 2024-09-21T03:34:38.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:38.689+0000 7f9e320a4640 1 -- 172.21.15.38:0/1746066528 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9e18078b90 msgr2=0x7f9e1807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:38.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:38.689+0000 7f9e320a4640 1 --2- 172.21.15.38:0/1746066528 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9e18078b90 0x7f9e1807b0a0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f9e1c009a40 tx=0x7f9e1c005f90 comp rx=0 tx=0).stop 2024-09-21T03:34:38.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:38.689+0000 7f9e320a4640 1 -- 172.21.15.38:0/1746066528 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9e2c0b4500 msgr2=0x7f9e2c1f5560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:38.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:38.689+0000 7f9e320a4640 1 --2- 172.21.15.38:0/1746066528 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9e2c0b4500 0x7f9e2c1f5560 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f9e200099c0 tx=0x7f9e200038a0 comp rx=0 tx=0).stop 2024-09-21T03:34:38.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:38.689+0000 7f9e320a4640 1 -- 172.21.15.38:0/1746066528 shutdown_connections 2024-09-21T03:34:38.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:38.689+0000 7f9e320a4640 1 --2- 172.21.15.38:0/1746066528 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9e2c0d0b40 0x7f9e2c14f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:38.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:38.689+0000 7f9e320a4640 1 --2- 172.21.15.38:0/1746066528 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9e2c0b4500 0x7f9e2c1f5560 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:38.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:38.689+0000 7f9e320a4640 1 --2- 172.21.15.38:0/1746066528 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9e18078b90 0x7f9e1807b0a0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:38.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:38.689+0000 7f9e320a4640 1 --2- 172.21.15.38:0/1746066528 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9e2c06ec30 0x7f9e2c0b0040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:38.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:38.689+0000 7f9e320a4640 1 -- 172.21.15.38:0/1746066528 >> 172.21.15.38:0/1746066528 conn(0x7f9e2c0b2cc0 msgr2=0x7f9e2c0cefb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:38.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:38.689+0000 7f9e320a4640 1 -- 172.21.15.38:0/1746066528 shutdown_connections 2024-09-21T03:34:38.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:38.689+0000 7f9e320a4640 1 -- 172.21.15.38:0/1746066528 wait complete. 2024-09-21T03:34:38.702 INFO:teuthology.orchestra.run.smithi038.stdout:{"pg_ready":true,"pg_map":{"version":70,"stamp":"2024-09-21T03:34:36.992621+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590390,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":102,"num_write_kb":1629,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":142,"acting":142,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":50,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":253960,"kb_used_data":4440,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754720760,"statfs":{"total":773094113280,"available":772834058240,"internally_reserved":0,"allocated":4546560,"data_stored":2596193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12748,"internal_metadata":255315508},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.387727"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T03:34:33.604595+0000","last_change":"2024-09-21T03:34:33.604595+0000","last_active":"2024-09-21T03:34:33.604595+0000","last_peered":"2024-09-21T03:34:33.604595+0000","last_clean":"2024-09-21T03:34:33.604595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:34:33.604595+0000","last_undegraded":"2024-09-21T03:34:33.604595+0000","last_fullsized":"2024-09-21T03:34:33.604595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T03:34:33.604595+0000","last_change":"2024-09-21T03:34:33.604595+0000","last_active":"2024-09-21T03:34:33.604595+0000","last_peered":"2024-09-21T03:34:33.604595+0000","last_clean":"2024-09-21T03:34:33.604595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:34:33.604595+0000","last_undegraded":"2024-09-21T03:34:33.604595+0000","last_fullsized":"2024-09-21T03:34:33.604595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T03:34:33.604595+0000","last_change":"2024-09-21T03:34:33.604595+0000","last_active":"2024-09-21T03:34:33.604595+0000","last_peered":"2024-09-21T03:34:33.604595+0000","last_clean":"2024-09-21T03:34:33.604595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:34:33.604595+0000","last_undegraded":"2024-09-21T03:34:33.604595+0000","last_fullsized":"2024-09-21T03:34:33.604595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T03:34:33.604595+0000","last_change":"2024-09-21T03:34:33.604595+0000","last_active":"2024-09-21T03:34:33.604595+0000","last_peered":"2024-09-21T03:34:33.604595+0000","last_clean":"2024-09-21T03:34:33.604595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:34:33.604595+0000","last_undegraded":"2024-09-21T03:34:33.604595+0000","last_fullsized":"2024-09-21T03:34:33.604595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-21T03:34:33.614399+0000","last_change":"2024-09-21T03:34:33.614399+0000","last_active":"2024-09-21T03:34:33.604595+0000","last_peered":"2024-09-21T03:34:33.604595+0000","last_clean":"2024-09-21T03:34:33.604595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:34:33.614399+0000","last_undegraded":"2024-09-21T03:34:33.614399+0000","last_fullsized":"2024-09-21T03:34:33.614399+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T03:34:33.604595+0000","last_change":"2024-09-21T03:34:33.604595+0000","last_active":"2024-09-21T03:34:33.604595+0000","last_peered":"2024-09-21T03:34:33.604595+0000","last_clean":"2024-09-21T03:34:33.604595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:34:33.604595+0000","last_undegraded":"2024-09-21T03:34:33.604595+0000","last_fullsized":"2024-09-21T03:34:33.604595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-21T03:34:34.638247+0000","last_change":"2024-09-21T03:34:34.629606+0000","last_active":"2024-09-21T03:34:34.638247+0000","last_peered":"2024-09-21T03:34:34.638247+0000","last_clean":"2024-09-21T03:34:34.638247+0000","last_became_active":"2024-09-21T03:34:34.628992+0000","last_became_peered":"2024-09-21T03:34:34.628992+0000","last_unstale":"2024-09-21T03:34:34.638247+0000","last_undegraded":"2024-09-21T03:34:34.638247+0000","last_fullsized":"2024-09-21T03:34:34.638247+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:34:28.501979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T03:34:33.604595+0000","last_change":"2024-09-21T03:34:33.604595+0000","last_active":"2024-09-21T03:34:33.604595+0000","last_peered":"2024-09-21T03:34:33.604595+0000","last_clean":"2024-09-21T03:34:33.604595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:34:33.604595+0000","last_undegraded":"2024-09-21T03:34:33.604595+0000","last_fullsized":"2024-09-21T03:34:33.604595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-21T03:34:33.614285+0000","last_change":"2024-09-21T03:34:33.614285+0000","last_active":"2024-09-21T03:34:33.604595+0000","last_peered":"2024-09-21T03:34:33.604595+0000","last_clean":"2024-09-21T03:34:33.604595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:34:33.614285+0000","last_undegraded":"2024-09-21T03:34:33.614285+0000","last_fullsized":"2024-09-21T03:34:33.614285+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-21T03:34:34.642546+0000","last_change":"2024-09-21T03:34:34.637080+0000","last_active":"2024-09-21T03:34:34.642546+0000","last_peered":"2024-09-21T03:34:34.642546+0000","last_clean":"2024-09-21T03:34:34.642546+0000","last_became_active":"2024-09-21T03:34:34.636777+0000","last_became_peered":"2024-09-21T03:34:34.636777+0000","last_unstale":"2024-09-21T03:34:34.642546+0000","last_undegraded":"2024-09-21T03:34:34.642546+0000","last_fullsized":"2024-09-21T03:34:34.642546+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:03:32.166555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-21T03:34:33.622983+0000","last_change":"2024-09-21T03:34:33.622983+0000","last_active":"2024-09-21T03:34:33.604595+0000","last_peered":"2024-09-21T03:34:33.604595+0000","last_clean":"2024-09-21T03:34:33.604595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:34:33.622983+0000","last_undegraded":"2024-09-21T03:34:33.622983+0000","last_fullsized":"2024-09-21T03:34:33.622983+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-21T03:34:33.614478+0000","last_change":"2024-09-21T03:34:33.614478+0000","last_active":"2024-09-21T03:34:33.604595+0000","last_peered":"2024-09-21T03:34:33.604595+0000","last_clean":"2024-09-21T03:34:33.604595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:34:33.614478+0000","last_undegraded":"2024-09-21T03:34:33.614478+0000","last_fullsized":"2024-09-21T03:34:33.614478+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-21T03:34:34.642073+0000","last_change":"2024-09-21T03:34:34.629186+0000","last_active":"2024-09-21T03:34:34.642073+0000","last_peered":"2024-09-21T03:34:34.642073+0000","last_clean":"2024-09-21T03:34:34.642073+0000","last_became_active":"2024-09-21T03:34:34.628801+0000","last_became_peered":"2024-09-21T03:34:34.628801+0000","last_unstale":"2024-09-21T03:34:34.642073+0000","last_undegraded":"2024-09-21T03:34:34.642073+0000","last_fullsized":"2024-09-21T03:34:34.642073+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:34:03.702144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-21T03:34:33.616107+0000","last_change":"2024-09-21T03:34:33.616107+0000","last_active":"2024-09-21T03:34:33.604595+0000","last_peered":"2024-09-21T03:34:33.604595+0000","last_clean":"2024-09-21T03:34:33.604595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:34:33.616107+0000","last_undegraded":"2024-09-21T03:34:33.616107+0000","last_fullsized":"2024-09-21T03:34:33.616107+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-21T03:34:33.625487+0000","last_change":"2024-09-21T03:34:33.625487+0000","last_active":"2024-09-21T03:34:33.604595+0000","last_peered":"2024-09-21T03:34:33.604595+0000","last_clean":"2024-09-21T03:34:33.604595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:34:33.625487+0000","last_undegraded":"2024-09-21T03:34:33.625487+0000","last_fullsized":"2024-09-21T03:34:33.625487+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-21T03:34:34.644136+0000","last_change":"2024-09-21T03:34:34.635135+0000","last_active":"2024-09-21T03:34:34.644136+0000","last_peered":"2024-09-21T03:34:34.644136+0000","last_clean":"2024-09-21T03:34:34.644136+0000","last_became_active":"2024-09-21T03:34:34.634811+0000","last_became_peered":"2024-09-21T03:34:34.634811+0000","last_unstale":"2024-09-21T03:34:34.644136+0000","last_undegraded":"2024-09-21T03:34:34.644136+0000","last_fullsized":"2024-09-21T03:34:34.644136+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:44:10.495963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-21T03:34:33.624716+0000","last_change":"2024-09-21T03:34:33.624716+0000","last_active":"2024-09-21T03:34:33.604595+0000","last_peered":"2024-09-21T03:34:33.604595+0000","last_clean":"2024-09-21T03:34:33.604595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:34:33.624716+0000","last_undegraded":"2024-09-21T03:34:33.624716+0000","last_fullsized":"2024-09-21T03:34:33.624716+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-21T03:34:33.623146+0000","last_change":"2024-09-21T03:34:33.623146+0000","last_active":"2024-09-21T03:34:33.604595+0000","last_peered":"2024-09-21T03:34:33.604595+0000","last_clean":"2024-09-21T03:34:33.604595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:34:33.623146+0000","last_undegraded":"2024-09-21T03:34:33.623146+0000","last_fullsized":"2024-09-21T03:34:33.623146+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T03:34:33.604595+0000","last_change":"2024-09-21T03:34:33.604595+0000","last_active":"2024-09-21T03:34:33.604595+0000","last_peered":"2024-09-21T03:34:33.604595+0000","last_clean":"2024-09-21T03:34:33.604595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:34:33.604595+0000","last_undegraded":"2024-09-21T03:34:33.604595+0000","last_fullsized":"2024-09-21T03:34:33.604595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-21T03:34:33.620475+0000","last_change":"2024-09-21T03:34:33.620475+0000","last_active":"2024-09-21T03:34:33.604595+0000","last_peered":"2024-09-21T03:34:33.604595+0000","last_clean":"2024-09-21T03:34:33.604595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:34:33.620475+0000","last_undegraded":"2024-09-21T03:34:33.620475+0000","last_fullsized":"2024-09-21T03:34:33.620475+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-21T03:34:33.621111+0000","last_change":"2024-09-21T03:34:33.621111+0000","last_active":"2024-09-21T03:34:33.604595+0000","last_peered":"2024-09-21T03:34:33.604595+0000","last_clean":"2024-09-21T03:34:33.604595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:34:33.621111+0000","last_undegraded":"2024-09-21T03:34:33.621111+0000","last_fullsized":"2024-09-21T03:34:33.621111+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-21T03:34:33.621590+0000","last_change":"2024-09-21T03:34:33.621590+0000","last_active":"2024-09-21T03:34:33.604595+0000","last_peered":"2024-09-21T03:34:33.604595+0000","last_clean":"2024-09-21T03:34:33.604595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:34:33.621590+0000","last_undegraded":"2024-09-21T03:34:33.621590+0000","last_fullsized":"2024-09-21T03:34:33.621590+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-21T03:34:33.624872+0000","last_change":"2024-09-21T03:34:33.624872+0000","last_active":"2024-09-21T03:34:33.604595+0000","last_peered":"2024-09-21T03:34:33.604595+0000","last_clean":"2024-09-21T03:34:33.604595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:34:33.624872+0000","last_undegraded":"2024-09-21T03:34:33.624872+0000","last_fullsized":"2024-09-21T03:34:33.624872+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T03:34:33.604595+0000","last_change":"2024-09-21T03:34:33.604595+0000","last_active":"2024-09-21T03:34:33.604595+0000","last_peered":"2024-09-21T03:34:33.604595+0000","last_clean":"2024-09-21T03:34:33.604595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:34:33.604595+0000","last_undegraded":"2024-09-21T03:34:33.604595+0000","last_fullsized":"2024-09-21T03:34:33.604595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-21T03:34:33.616131+0000","last_change":"2024-09-21T03:34:33.616131+0000","last_active":"2024-09-21T03:34:33.604595+0000","last_peered":"2024-09-21T03:34:33.604595+0000","last_clean":"2024-09-21T03:34:33.604595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:34:33.616131+0000","last_undegraded":"2024-09-21T03:34:33.616131+0000","last_fullsized":"2024-09-21T03:34:33.616131+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T03:34:33.604595+0000","last_change":"2024-09-21T03:34:33.604595+0000","last_active":"2024-09-21T03:34:33.604595+0000","last_peered":"2024-09-21T03:34:33.604595+0000","last_clean":"2024-09-21T03:34:33.604595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:34:33.604595+0000","last_undegraded":"2024-09-21T03:34:33.604595+0000","last_fullsized":"2024-09-21T03:34:33.604595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-21T03:34:33.622462+0000","last_change":"2024-09-21T03:34:33.622462+0000","last_active":"2024-09-21T03:34:33.604595+0000","last_peered":"2024-09-21T03:34:33.604595+0000","last_clean":"2024-09-21T03:34:33.604595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:34:33.622462+0000","last_undegraded":"2024-09-21T03:34:33.622462+0000","last_fullsized":"2024-09-21T03:34:33.622462+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-21T03:34:34.639470+0000","last_change":"2024-09-21T03:34:34.633013+0000","last_active":"2024-09-21T03:34:34.639470+0000","last_peered":"2024-09-21T03:34:34.639470+0000","last_clean":"2024-09-21T03:34:34.639470+0000","last_became_active":"2024-09-21T03:34:34.631701+0000","last_became_peered":"2024-09-21T03:34:34.631701+0000","last_unstale":"2024-09-21T03:34:34.639470+0000","last_undegraded":"2024-09-21T03:34:34.639470+0000","last_fullsized":"2024-09-21T03:34:34.639470+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:58:27.086990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-21T03:34:33.628158+0000","last_change":"2024-09-21T03:34:33.628158+0000","last_active":"2024-09-21T03:34:33.604595+0000","last_peered":"2024-09-21T03:34:33.604595+0000","last_clean":"2024-09-21T03:34:33.604595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:34:33.628158+0000","last_undegraded":"2024-09-21T03:34:33.628158+0000","last_fullsized":"2024-09-21T03:34:33.628158+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-21T03:34:33.621235+0000","last_change":"2024-09-21T03:34:33.621235+0000","last_active":"2024-09-21T03:34:33.604595+0000","last_peered":"2024-09-21T03:34:33.604595+0000","last_clean":"2024-09-21T03:34:33.604595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:34:33.621235+0000","last_undegraded":"2024-09-21T03:34:33.621235+0000","last_fullsized":"2024-09-21T03:34:33.621235+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-21T03:34:34.639102+0000","last_change":"2024-09-21T03:34:34.629415+0000","last_active":"2024-09-21T03:34:34.639102+0000","last_peered":"2024-09-21T03:34:34.639102+0000","last_clean":"2024-09-21T03:34:34.639102+0000","last_became_active":"2024-09-21T03:34:34.628524+0000","last_became_peered":"2024-09-21T03:34:34.628524+0000","last_unstale":"2024-09-21T03:34:34.639102+0000","last_undegraded":"2024-09-21T03:34:34.639102+0000","last_fullsized":"2024-09-21T03:34:34.639102+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:48:30.216711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T03:34:33.604595+0000","last_change":"2024-09-21T03:34:33.604595+0000","last_active":"2024-09-21T03:34:33.604595+0000","last_peered":"2024-09-21T03:34:33.604595+0000","last_clean":"2024-09-21T03:34:33.604595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:34:33.604595+0000","last_undegraded":"2024-09-21T03:34:33.604595+0000","last_fullsized":"2024-09-21T03:34:33.604595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-21T03:34:33.621969+0000","last_change":"2024-09-21T03:34:33.621969+0000","last_active":"2024-09-21T03:34:33.604595+0000","last_peered":"2024-09-21T03:34:33.604595+0000","last_clean":"2024-09-21T03:34:33.604595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:34:33.621969+0000","last_undegraded":"2024-09-21T03:34:33.621969+0000","last_fullsized":"2024-09-21T03:34:33.621969+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T03:34:33.620882+0000","last_change":"2024-09-21T03:34:33.618066+0000","last_active":"2024-09-21T03:34:33.620882+0000","last_peered":"2024-09-21T03:34:33.620882+0000","last_clean":"2024-09-21T03:34:33.620882+0000","last_became_active":"2024-09-21T03:34:33.617685+0000","last_became_peered":"2024-09-21T03:34:33.617685+0000","last_unstale":"2024-09-21T03:34:33.620882+0000","last_undegraded":"2024-09-21T03:34:33.620882+0000","last_fullsized":"2024-09-21T03:34:33.620882+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_clean_scrub_stamp":"2024-09-21T03:34:32.598217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:34:47.997359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T03:34:33.604595+0000","last_change":"2024-09-21T03:34:33.604595+0000","last_active":"2024-09-21T03:34:33.604595+0000","last_peered":"2024-09-21T03:34:33.604595+0000","last_clean":"2024-09-21T03:34:33.604595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:34:33.604595+0000","last_undegraded":"2024-09-21T03:34:33.604595+0000","last_fullsized":"2024-09-21T03:34:33.604595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-21T03:34:32.610882+0000","last_change":"2024-09-21T03:34:32.610882+0000","last_active":"2024-09-21T03:34:32.598217+0000","last_peered":"2024-09-21T03:34:32.598217+0000","last_clean":"2024-09-21T03:34:32.598217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:34:32.610882+0000","last_undegraded":"2024-09-21T03:34:32.610882+0000","last_fullsized":"2024-09-21T03:34:32.610882+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_clean_scrub_stamp":"2024-09-21T03:34:32.598217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-21T03:34:33.622277+0000","last_change":"2024-09-21T03:34:33.622277+0000","last_active":"2024-09-21T03:34:33.604595+0000","last_peered":"2024-09-21T03:34:33.604595+0000","last_clean":"2024-09-21T03:34:33.604595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:34:33.622277+0000","last_undegraded":"2024-09-21T03:34:33.622277+0000","last_fullsized":"2024-09-21T03:34:33.622277+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-21T03:34:32.607162+0000","last_change":"2024-09-21T03:34:32.607162+0000","last_active":"2024-09-21T03:34:32.598217+0000","last_peered":"2024-09-21T03:34:32.598217+0000","last_clean":"2024-09-21T03:34:32.598217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:34:32.607162+0000","last_undegraded":"2024-09-21T03:34:32.607162+0000","last_fullsized":"2024-09-21T03:34:32.607162+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_clean_scrub_stamp":"2024-09-21T03:34:32.598217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T03:34:33.604595+0000","last_change":"2024-09-21T03:34:33.604595+0000","last_active":"2024-09-21T03:34:33.604595+0000","last_peered":"2024-09-21T03:34:33.604595+0000","last_clean":"2024-09-21T03:34:33.604595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:34:33.604595+0000","last_undegraded":"2024-09-21T03:34:33.604595+0000","last_fullsized":"2024-09-21T03:34:33.604595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T03:34:33.638430+0000","last_change":"2024-09-21T03:34:33.631010+0000","last_active":"2024-09-21T03:34:33.638430+0000","last_peered":"2024-09-21T03:34:33.638430+0000","last_clean":"2024-09-21T03:34:33.638430+0000","last_became_active":"2024-09-21T03:34:33.630576+0000","last_became_peered":"2024-09-21T03:34:33.630576+0000","last_unstale":"2024-09-21T03:34:33.638430+0000","last_undegraded":"2024-09-21T03:34:33.638430+0000","last_fullsized":"2024-09-21T03:34:33.638430+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_clean_scrub_stamp":"2024-09-21T03:34:32.598217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:50:07.223556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T03:34:33.604595+0000","last_change":"2024-09-21T03:34:33.604595+0000","last_active":"2024-09-21T03:34:33.604595+0000","last_peered":"2024-09-21T03:34:33.604595+0000","last_clean":"2024-09-21T03:34:33.604595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:34:33.604595+0000","last_undegraded":"2024-09-21T03:34:33.604595+0000","last_fullsized":"2024-09-21T03:34:33.604595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T03:34:33.635417+0000","last_change":"2024-09-21T03:34:33.619056+0000","last_active":"2024-09-21T03:34:33.635417+0000","last_peered":"2024-09-21T03:34:33.635417+0000","last_clean":"2024-09-21T03:34:33.635417+0000","last_became_active":"2024-09-21T03:34:33.618698+0000","last_became_peered":"2024-09-21T03:34:33.618698+0000","last_unstale":"2024-09-21T03:34:33.635417+0000","last_undegraded":"2024-09-21T03:34:33.635417+0000","last_fullsized":"2024-09-21T03:34:33.635417+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_clean_scrub_stamp":"2024-09-21T03:34:32.598217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:58:31.223579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-21T03:34:33.618669+0000","last_change":"2024-09-21T03:34:33.618669+0000","last_active":"2024-09-21T03:34:33.604595+0000","last_peered":"2024-09-21T03:34:33.604595+0000","last_clean":"2024-09-21T03:34:33.604595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:34:33.618669+0000","last_undegraded":"2024-09-21T03:34:33.618669+0000","last_fullsized":"2024-09-21T03:34:33.618669+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1a","version":"39'1","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-21T03:34:34.633517+0000","last_change":"2024-09-21T03:34:33.634501+0000","last_active":"2024-09-21T03:34:34.633517+0000","last_peered":"2024-09-21T03:34:34.633517+0000","last_clean":"2024-09-21T03:34:34.633517+0000","last_became_active":"2024-09-21T03:34:33.633827+0000","last_became_peered":"2024-09-21T03:34:33.633827+0000","last_unstale":"2024-09-21T03:34:34.633517+0000","last_undegraded":"2024-09-21T03:34:34.633517+0000","last_fullsized":"2024-09-21T03:34:34.633517+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_clean_scrub_stamp":"2024-09-21T03:34:32.598217+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:33:52.483902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-21T03:34:33.620634+0000","last_change":"2024-09-21T03:34:33.620634+0000","last_active":"2024-09-21T03:34:33.604595+0000","last_peered":"2024-09-21T03:34:33.604595+0000","last_clean":"2024-09-21T03:34:33.604595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:34:33.620634+0000","last_undegraded":"2024-09-21T03:34:33.620634+0000","last_fullsized":"2024-09-21T03:34:33.620634+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-21T03:34:32.607309+0000","last_change":"2024-09-21T03:34:32.607309+0000","last_active":"2024-09-21T03:34:32.598217+0000","last_peered":"2024-09-21T03:34:32.598217+0000","last_clean":"2024-09-21T03:34:32.598217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:34:32.607309+0000","last_undegraded":"2024-09-21T03:34:32.607309+0000","last_fullsized":"2024-09-21T03:34:32.607309+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_clean_scrub_stamp":"2024-09-21T03:34:32.598217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T03:34:33.604595+0000","last_change":"2024-09-21T03:34:33.604595+0000","last_active":"2024-09-21T03:34:33.604595+0000","last_peered":"2024-09-21T03:34:33.604595+0000","last_clean":"2024-09-21T03:34:33.604595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:34:33.604595+0000","last_undegraded":"2024-09-21T03:34:33.604595+0000","last_fullsized":"2024-09-21T03:34:33.604595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-21T03:34:34.615067+0000","last_change":"2024-09-21T03:34:33.638401+0000","last_active":"2024-09-21T03:34:34.615067+0000","last_peered":"2024-09-21T03:34:34.615067+0000","last_clean":"2024-09-21T03:34:34.615067+0000","last_became_active":"2024-09-21T03:34:33.637364+0000","last_became_peered":"2024-09-21T03:34:33.637364+0000","last_unstale":"2024-09-21T03:34:34.615067+0000","last_undegraded":"2024-09-21T03:34:34.615067+0000","last_fullsized":"2024-09-21T03:34:34.615067+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_clean_scrub_stamp":"2024-09-21T03:34:32.598217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:37:36.532174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T03:34:33.604595+0000","last_change":"2024-09-21T03:34:33.604595+0000","last_active":"2024-09-21T03:34:33.604595+0000","last_peered":"2024-09-21T03:34:33.604595+0000","last_clean":"2024-09-21T03:34:33.604595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:34:33.604595+0000","last_undegraded":"2024-09-21T03:34:33.604595+0000","last_fullsized":"2024-09-21T03:34:33.604595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-21T03:34:32.608162+0000","last_change":"2024-09-21T03:34:32.608162+0000","last_active":"2024-09-21T03:34:32.598217+0000","last_peered":"2024-09-21T03:34:32.598217+0000","last_clean":"2024-09-21T03:34:32.598217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:34:32.608162+0000","last_undegraded":"2024-09-21T03:34:32.608162+0000","last_fullsized":"2024-09-21T03:34:32.608162+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_clean_scrub_stamp":"2024-09-21T03:34:32.598217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-21T03:34:33.616082+0000","last_change":"2024-09-21T03:34:33.616082+0000","last_active":"2024-09-21T03:34:33.604595+0000","last_peered":"2024-09-21T03:34:33.604595+0000","last_clean":"2024-09-21T03:34:33.604595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:34:33.616082+0000","last_undegraded":"2024-09-21T03:34:33.616082+0000","last_fullsized":"2024-09-21T03:34:33.616082+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T03:34:33.638093+0000","last_change":"2024-09-21T03:34:33.633888+0000","last_active":"2024-09-21T03:34:33.638093+0000","last_peered":"2024-09-21T03:34:33.638093+0000","last_clean":"2024-09-21T03:34:33.638093+0000","last_became_active":"2024-09-21T03:34:33.633372+0000","last_became_peered":"2024-09-21T03:34:33.633372+0000","last_unstale":"2024-09-21T03:34:33.638093+0000","last_undegraded":"2024-09-21T03:34:33.638093+0000","last_fullsized":"2024-09-21T03:34:33.638093+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_clean_scrub_stamp":"2024-09-21T03:34:32.598217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:58:21.314833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T03:34:33.604595+0000","last_change":"2024-09-21T03:34:33.604595+0000","last_active":"2024-09-21T03:34:33.604595+0000","last_peered":"2024-09-21T03:34:33.604595+0000","last_clean":"2024-09-21T03:34:33.604595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:34:33.604595+0000","last_undegraded":"2024-09-21T03:34:33.604595+0000","last_fullsized":"2024-09-21T03:34:33.604595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T03:34:33.638035+0000","last_change":"2024-09-21T03:34:33.632451+0000","last_active":"2024-09-21T03:34:33.638035+0000","last_peered":"2024-09-21T03:34:33.638035+0000","last_clean":"2024-09-21T03:34:33.638035+0000","last_became_active":"2024-09-21T03:34:33.632103+0000","last_became_peered":"2024-09-21T03:34:33.632103+0000","last_unstale":"2024-09-21T03:34:33.638035+0000","last_undegraded":"2024-09-21T03:34:33.638035+0000","last_fullsized":"2024-09-21T03:34:33.638035+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_clean_scrub_stamp":"2024-09-21T03:34:32.598217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:22:08.770678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-21T03:34:34.639059+0000","last_change":"2024-09-21T03:34:34.628419+0000","last_active":"2024-09-21T03:34:34.639059+0000","last_peered":"2024-09-21T03:34:34.639059+0000","last_clean":"2024-09-21T03:34:34.639059+0000","last_became_active":"2024-09-21T03:34:34.628087+0000","last_became_peered":"2024-09-21T03:34:34.628087+0000","last_unstale":"2024-09-21T03:34:34.639059+0000","last_undegraded":"2024-09-21T03:34:34.639059+0000","last_fullsized":"2024-09-21T03:34:34.639059+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:28:03.240497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-21T03:34:32.607600+0000","last_change":"2024-09-21T03:34:32.607600+0000","last_active":"2024-09-21T03:34:32.598217+0000","last_peered":"2024-09-21T03:34:32.598217+0000","last_clean":"2024-09-21T03:34:32.598217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:34:32.607600+0000","last_undegraded":"2024-09-21T03:34:32.607600+0000","last_fullsized":"2024-09-21T03:34:32.607600+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_clean_scrub_stamp":"2024-09-21T03:34:32.598217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T03:34:33.604595+0000","last_change":"2024-09-21T03:34:33.604595+0000","last_active":"2024-09-21T03:34:33.604595+0000","last_peered":"2024-09-21T03:34:33.604595+0000","last_clean":"2024-09-21T03:34:33.604595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:34:33.604595+0000","last_undegraded":"2024-09-21T03:34:33.604595+0000","last_fullsized":"2024-09-21T03:34:33.604595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-21T03:34:32.607718+0000","last_change":"2024-09-21T03:34:32.607718+0000","last_active":"2024-09-21T03:34:32.598217+0000","last_peered":"2024-09-21T03:34:32.598217+0000","last_clean":"2024-09-21T03:34:32.598217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:34:32.607718+0000","last_undegraded":"2024-09-21T03:34:32.607718+0000","last_fullsized":"2024-09-21T03:34:32.607718+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_clean_scrub_stamp":"2024-09-21T03:34:32.598217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T03:34:33.604595+0000","last_change":"2024-09-21T03:34:33.604595+0000","last_active":"2024-09-21T03:34:33.604595+0000","last_peered":"2024-09-21T03:34:33.604595+0000","last_clean":"2024-09-21T03:34:33.604595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:34:33.604595+0000","last_undegraded":"2024-09-21T03:34:33.604595+0000","last_fullsized":"2024-09-21T03:34:33.604595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-21T03:34:32.611193+0000","last_change":"2024-09-21T03:34:32.611193+0000","last_active":"2024-09-21T03:34:32.598217+0000","last_peered":"2024-09-21T03:34:32.598217+0000","last_clean":"2024-09-21T03:34:32.598217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:34:32.611193+0000","last_undegraded":"2024-09-21T03:34:32.611193+0000","last_fullsized":"2024-09-21T03:34:32.611193+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_clean_scrub_stamp":"2024-09-21T03:34:32.598217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-21T03:34:33.619516+0000","last_change":"2024-09-21T03:34:33.619516+0000","last_active":"2024-09-21T03:34:33.604595+0000","last_peered":"2024-09-21T03:34:33.604595+0000","last_clean":"2024-09-21T03:34:33.604595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:34:33.619516+0000","last_undegraded":"2024-09-21T03:34:33.619516+0000","last_fullsized":"2024-09-21T03:34:33.619516+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T03:34:33.644077+0000","last_change":"2024-09-21T03:34:33.639290+0000","last_active":"2024-09-21T03:34:33.644077+0000","last_peered":"2024-09-21T03:34:33.644077+0000","last_clean":"2024-09-21T03:34:33.644077+0000","last_became_active":"2024-09-21T03:34:33.638941+0000","last_became_peered":"2024-09-21T03:34:33.638941+0000","last_unstale":"2024-09-21T03:34:33.644077+0000","last_undegraded":"2024-09-21T03:34:33.644077+0000","last_fullsized":"2024-09-21T03:34:33.644077+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_clean_scrub_stamp":"2024-09-21T03:34:32.598217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:46:42.303637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-21T03:34:33.613436+0000","last_change":"2024-09-21T03:34:33.613436+0000","last_active":"2024-09-21T03:34:33.604595+0000","last_peered":"2024-09-21T03:34:33.604595+0000","last_clean":"2024-09-21T03:34:33.604595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:34:33.613436+0000","last_undegraded":"2024-09-21T03:34:33.613436+0000","last_fullsized":"2024-09-21T03:34:33.613436+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T03:34:33.638439+0000","last_change":"2024-09-21T03:34:33.632101+0000","last_active":"2024-09-21T03:34:33.638439+0000","last_peered":"2024-09-21T03:34:33.638439+0000","last_clean":"2024-09-21T03:34:33.638439+0000","last_became_active":"2024-09-21T03:34:33.631744+0000","last_became_peered":"2024-09-21T03:34:33.631744+0000","last_unstale":"2024-09-21T03:34:33.638439+0000","last_undegraded":"2024-09-21T03:34:33.638439+0000","last_fullsized":"2024-09-21T03:34:33.638439+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_clean_scrub_stamp":"2024-09-21T03:34:32.598217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:17:30.670560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T03:34:33.604595+0000","last_change":"2024-09-21T03:34:33.604595+0000","last_active":"2024-09-21T03:34:33.604595+0000","last_peered":"2024-09-21T03:34:33.604595+0000","last_clean":"2024-09-21T03:34:33.604595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:34:33.604595+0000","last_undegraded":"2024-09-21T03:34:33.604595+0000","last_fullsized":"2024-09-21T03:34:33.604595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.f","version":"39'1","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-21T03:34:34.638168+0000","last_change":"2024-09-21T03:34:33.637659+0000","last_active":"2024-09-21T03:34:34.638168+0000","last_peered":"2024-09-21T03:34:34.638168+0000","last_clean":"2024-09-21T03:34:34.638168+0000","last_became_active":"2024-09-21T03:34:33.637317+0000","last_became_peered":"2024-09-21T03:34:33.637317+0000","last_unstale":"2024-09-21T03:34:34.638168+0000","last_undegraded":"2024-09-21T03:34:34.638168+0000","last_fullsized":"2024-09-21T03:34:34.638168+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_clean_scrub_stamp":"2024-09-21T03:34:32.598217+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:12:23.510532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-21T03:34:34.642113+0000","last_change":"2024-09-21T03:34:34.634278+0000","last_active":"2024-09-21T03:34:34.642113+0000","last_peered":"2024-09-21T03:34:34.642113+0000","last_clean":"2024-09-21T03:34:34.642113+0000","last_became_active":"2024-09-21T03:34:34.633950+0000","last_became_peered":"2024-09-21T03:34:34.633950+0000","last_unstale":"2024-09-21T03:34:34.642113+0000","last_undegraded":"2024-09-21T03:34:34.642113+0000","last_fullsized":"2024-09-21T03:34:34.642113+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:43:22.066515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T03:34:33.638123+0000","last_change":"2024-09-21T03:34:33.634743+0000","last_active":"2024-09-21T03:34:33.638123+0000","last_peered":"2024-09-21T03:34:33.638123+0000","last_clean":"2024-09-21T03:34:33.638123+0000","last_became_active":"2024-09-21T03:34:33.634259+0000","last_became_peered":"2024-09-21T03:34:33.634259+0000","last_unstale":"2024-09-21T03:34:33.638123+0000","last_undegraded":"2024-09-21T03:34:33.638123+0000","last_fullsized":"2024-09-21T03:34:33.638123+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_clean_scrub_stamp":"2024-09-21T03:34:32.598217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:14:11.700926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-21T03:34:34.637883+0000","last_change":"2024-09-21T03:34:34.628247+0000","last_active":"2024-09-21T03:34:34.637883+0000","last_peered":"2024-09-21T03:34:34.637883+0000","last_clean":"2024-09-21T03:34:34.637883+0000","last_became_active":"2024-09-21T03:34:34.627925+0000","last_became_peered":"2024-09-21T03:34:34.627925+0000","last_unstale":"2024-09-21T03:34:34.637883+0000","last_undegraded":"2024-09-21T03:34:34.637883+0000","last_fullsized":"2024-09-21T03:34:34.637883+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:26:59.051709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T03:34:33.636065+0000","last_change":"2024-09-21T03:34:33.635141+0000","last_active":"2024-09-21T03:34:33.636065+0000","last_peered":"2024-09-21T03:34:33.636065+0000","last_clean":"2024-09-21T03:34:33.636065+0000","last_became_active":"2024-09-21T03:34:33.634825+0000","last_became_peered":"2024-09-21T03:34:33.634825+0000","last_unstale":"2024-09-21T03:34:33.636065+0000","last_undegraded":"2024-09-21T03:34:33.636065+0000","last_fullsized":"2024-09-21T03:34:33.636065+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_clean_scrub_stamp":"2024-09-21T03:34:32.598217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:12:23.510532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T03:34:33.604595+0000","last_change":"2024-09-21T03:34:33.604595+0000","last_active":"2024-09-21T03:34:33.604595+0000","last_peered":"2024-09-21T03:34:33.604595+0000","last_clean":"2024-09-21T03:34:33.604595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:34:33.604595+0000","last_undegraded":"2024-09-21T03:34:33.604595+0000","last_fullsized":"2024-09-21T03:34:33.604595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T03:34:32.598217+0000","last_change":"2024-09-21T03:34:32.598217+0000","last_active":"2024-09-21T03:34:32.598217+0000","last_peered":"2024-09-21T03:34:32.598217+0000","last_clean":"2024-09-21T03:34:32.598217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:34:32.598217+0000","last_undegraded":"2024-09-21T03:34:32.598217+0000","last_fullsized":"2024-09-21T03:34:32.598217+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_clean_scrub_stamp":"2024-09-21T03:34:32.598217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-21T03:34:33.620205+0000","last_change":"2024-09-21T03:34:33.620205+0000","last_active":"2024-09-21T03:34:33.604595+0000","last_peered":"2024-09-21T03:34:33.604595+0000","last_clean":"2024-09-21T03:34:33.604595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:34:33.620205+0000","last_undegraded":"2024-09-21T03:34:33.620205+0000","last_fullsized":"2024-09-21T03:34:33.620205+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T03:34:32.598217+0000","last_change":"2024-09-21T03:34:32.598217+0000","last_active":"2024-09-21T03:34:32.598217+0000","last_peered":"2024-09-21T03:34:32.598217+0000","last_clean":"2024-09-21T03:34:32.598217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:34:32.598217+0000","last_undegraded":"2024-09-21T03:34:32.598217+0000","last_fullsized":"2024-09-21T03:34:32.598217+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_clean_scrub_stamp":"2024-09-21T03:34:32.598217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T03:34:33.604595+0000","last_change":"2024-09-21T03:34:33.604595+0000","last_active":"2024-09-21T03:34:33.604595+0000","last_peered":"2024-09-21T03:34:33.604595+0000","last_clean":"2024-09-21T03:34:33.604595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:34:33.604595+0000","last_undegraded":"2024-09-21T03:34:33.604595+0000","last_fullsized":"2024-09-21T03:34:33.604595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-21T03:34:32.610664+0000","last_change":"2024-09-21T03:34:32.610664+0000","last_active":"2024-09-21T03:34:32.598217+0000","last_peered":"2024-09-21T03:34:32.598217+0000","last_clean":"2024-09-21T03:34:32.598217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:34:32.610664+0000","last_undegraded":"2024-09-21T03:34:32.610664+0000","last_fullsized":"2024-09-21T03:34:32.610664+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_clean_scrub_stamp":"2024-09-21T03:34:32.598217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-21T03:34:33.621061+0000","last_change":"2024-09-21T03:34:33.621061+0000","last_active":"2024-09-21T03:34:33.604595+0000","last_peered":"2024-09-21T03:34:33.604595+0000","last_clean":"2024-09-21T03:34:33.604595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:34:33.621061+0000","last_undegraded":"2024-09-21T03:34:33.621061+0000","last_fullsized":"2024-09-21T03:34:33.621061+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-21T03:34:34.616134+0000","last_change":"2024-09-21T03:34:33.637630+0000","last_active":"2024-09-21T03:34:34.616134+0000","last_peered":"2024-09-21T03:34:34.616134+0000","last_clean":"2024-09-21T03:34:34.616134+0000","last_became_active":"2024-09-21T03:34:33.637290+0000","last_became_peered":"2024-09-21T03:34:33.637290+0000","last_unstale":"2024-09-21T03:34:34.616134+0000","last_undegraded":"2024-09-21T03:34:34.616134+0000","last_fullsized":"2024-09-21T03:34:34.616134+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_clean_scrub_stamp":"2024-09-21T03:34:32.598217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:00:46.535619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T03:34:33.604595+0000","last_change":"2024-09-21T03:34:33.604595+0000","last_active":"2024-09-21T03:34:33.604595+0000","last_peered":"2024-09-21T03:34:33.604595+0000","last_clean":"2024-09-21T03:34:33.604595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:34:33.604595+0000","last_undegraded":"2024-09-21T03:34:33.604595+0000","last_fullsized":"2024-09-21T03:34:33.604595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-21T03:34:32.611150+0000","last_change":"2024-09-21T03:34:32.611150+0000","last_active":"2024-09-21T03:34:32.598217+0000","last_peered":"2024-09-21T03:34:32.598217+0000","last_clean":"2024-09-21T03:34:32.598217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:34:32.611150+0000","last_undegraded":"2024-09-21T03:34:32.611150+0000","last_fullsized":"2024-09-21T03:34:32.611150+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_clean_scrub_stamp":"2024-09-21T03:34:32.598217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-21T03:34:33.613904+0000","last_change":"2024-09-21T03:34:33.613904+0000","last_active":"2024-09-21T03:34:33.604595+0000","last_peered":"2024-09-21T03:34:33.604595+0000","last_clean":"2024-09-21T03:34:33.604595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:34:33.613904+0000","last_undegraded":"2024-09-21T03:34:33.613904+0000","last_fullsized":"2024-09-21T03:34:33.613904+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T03:34:33.638065+0000","last_change":"2024-09-21T03:34:33.633316+0000","last_active":"2024-09-21T03:34:33.638065+0000","last_peered":"2024-09-21T03:34:33.638065+0000","last_clean":"2024-09-21T03:34:33.638065+0000","last_became_active":"2024-09-21T03:34:33.632983+0000","last_became_peered":"2024-09-21T03:34:33.632983+0000","last_unstale":"2024-09-21T03:34:33.638065+0000","last_undegraded":"2024-09-21T03:34:33.638065+0000","last_fullsized":"2024-09-21T03:34:33.638065+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_clean_scrub_stamp":"2024-09-21T03:34:32.598217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:48:01.802931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-21T03:34:33.621976+0000","last_change":"2024-09-21T03:34:33.621976+0000","last_active":"2024-09-21T03:34:33.604595+0000","last_peered":"2024-09-21T03:34:33.604595+0000","last_clean":"2024-09-21T03:34:33.604595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:34:33.621976+0000","last_undegraded":"2024-09-21T03:34:33.621976+0000","last_fullsized":"2024-09-21T03:34:33.621976+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T03:34:33.632105+0000","last_change":"2024-09-21T03:34:33.630944+0000","last_active":"2024-09-21T03:34:33.632105+0000","last_peered":"2024-09-21T03:34:33.632105+0000","last_clean":"2024-09-21T03:34:33.632105+0000","last_became_active":"2024-09-21T03:34:33.630396+0000","last_became_peered":"2024-09-21T03:34:33.630396+0000","last_unstale":"2024-09-21T03:34:33.632105+0000","last_undegraded":"2024-09-21T03:34:33.632105+0000","last_fullsized":"2024-09-21T03:34:33.632105+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_clean_scrub_stamp":"2024-09-21T03:34:32.598217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:37:36.532174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T03:34:33.604595+0000","last_change":"2024-09-21T03:34:33.604595+0000","last_active":"2024-09-21T03:34:33.604595+0000","last_peered":"2024-09-21T03:34:33.604595+0000","last_clean":"2024-09-21T03:34:33.604595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:34:33.604595+0000","last_undegraded":"2024-09-21T03:34:33.604595+0000","last_fullsized":"2024-09-21T03:34:33.604595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T03:34:33.627809+0000","last_change":"2024-09-21T03:34:33.626093+0000","last_active":"2024-09-21T03:34:33.627809+0000","last_peered":"2024-09-21T03:34:33.627809+0000","last_clean":"2024-09-21T03:34:33.627809+0000","last_became_active":"2024-09-21T03:34:33.625771+0000","last_became_peered":"2024-09-21T03:34:33.625771+0000","last_unstale":"2024-09-21T03:34:33.627809+0000","last_undegraded":"2024-09-21T03:34:33.627809+0000","last_fullsized":"2024-09-21T03:34:33.627809+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_clean_scrub_stamp":"2024-09-21T03:34:32.598217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:05:08.028564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-21T03:34:33.613418+0000","last_change":"2024-09-21T03:34:33.613418+0000","last_active":"2024-09-21T03:34:33.604595+0000","last_peered":"2024-09-21T03:34:33.604595+0000","last_clean":"2024-09-21T03:34:33.604595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:34:33.613418+0000","last_undegraded":"2024-09-21T03:34:33.613418+0000","last_fullsized":"2024-09-21T03:34:33.613418+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-21T03:34:32.607115+0000","last_change":"2024-09-21T03:34:32.607115+0000","last_active":"2024-09-21T03:34:32.598217+0000","last_peered":"2024-09-21T03:34:32.598217+0000","last_clean":"2024-09-21T03:34:32.598217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:34:32.607115+0000","last_undegraded":"2024-09-21T03:34:32.607115+0000","last_fullsized":"2024-09-21T03:34:32.607115+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_clean_scrub_stamp":"2024-09-21T03:34:32.598217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T03:34:33.604595+0000","last_change":"2024-09-21T03:34:33.604595+0000","last_active":"2024-09-21T03:34:33.604595+0000","last_peered":"2024-09-21T03:34:33.604595+0000","last_clean":"2024-09-21T03:34:33.604595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:34:33.604595+0000","last_undegraded":"2024-09-21T03:34:33.604595+0000","last_fullsized":"2024-09-21T03:34:33.604595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T03:34:33.636106+0000","last_change":"2024-09-21T03:34:33.631273+0000","last_active":"2024-09-21T03:34:33.636106+0000","last_peered":"2024-09-21T03:34:33.636106+0000","last_clean":"2024-09-21T03:34:33.636106+0000","last_became_active":"2024-09-21T03:34:33.628561+0000","last_became_peered":"2024-09-21T03:34:33.628561+0000","last_unstale":"2024-09-21T03:34:33.636106+0000","last_undegraded":"2024-09-21T03:34:33.636106+0000","last_fullsized":"2024-09-21T03:34:33.636106+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_clean_scrub_stamp":"2024-09-21T03:34:32.598217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:49:16.438096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-21T03:34:33.620026+0000","last_change":"2024-09-21T03:34:33.620026+0000","last_active":"2024-09-21T03:34:33.604595+0000","last_peered":"2024-09-21T03:34:33.604595+0000","last_clean":"2024-09-21T03:34:33.604595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:34:33.620026+0000","last_undegraded":"2024-09-21T03:34:33.620026+0000","last_fullsized":"2024-09-21T03:34:33.620026+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T03:34:33.638150+0000","last_change":"2024-09-21T03:34:33.633175+0000","last_active":"2024-09-21T03:34:33.638150+0000","last_peered":"2024-09-21T03:34:33.638150+0000","last_clean":"2024-09-21T03:34:33.638150+0000","last_became_active":"2024-09-21T03:34:33.632872+0000","last_became_peered":"2024-09-21T03:34:33.632872+0000","last_unstale":"2024-09-21T03:34:33.638150+0000","last_undegraded":"2024-09-21T03:34:33.638150+0000","last_fullsized":"2024-09-21T03:34:33.638150+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_clean_scrub_stamp":"2024-09-21T03:34:32.598217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:33:03.801458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-21T03:34:33.619685+0000","last_change":"2024-09-21T03:34:33.619685+0000","last_active":"2024-09-21T03:34:33.604595+0000","last_peered":"2024-09-21T03:34:33.604595+0000","last_clean":"2024-09-21T03:34:33.604595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:34:33.619685+0000","last_undegraded":"2024-09-21T03:34:33.619685+0000","last_fullsized":"2024-09-21T03:34:33.619685+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-21T03:34:32.607048+0000","last_change":"2024-09-21T03:34:32.607048+0000","last_active":"2024-09-21T03:34:32.598217+0000","last_peered":"2024-09-21T03:34:32.598217+0000","last_clean":"2024-09-21T03:34:32.598217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:34:32.607048+0000","last_undegraded":"2024-09-21T03:34:32.607048+0000","last_fullsized":"2024-09-21T03:34:32.607048+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_clean_scrub_stamp":"2024-09-21T03:34:32.598217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-21T03:34:33.625230+0000","last_change":"2024-09-21T03:34:33.625230+0000","last_active":"2024-09-21T03:34:33.604595+0000","last_peered":"2024-09-21T03:34:33.604595+0000","last_clean":"2024-09-21T03:34:33.604595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:34:33.625230+0000","last_undegraded":"2024-09-21T03:34:33.625230+0000","last_fullsized":"2024-09-21T03:34:33.625230+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T03:34:33.637827+0000","last_change":"2024-09-21T03:34:33.635753+0000","last_active":"2024-09-21T03:34:33.637827+0000","last_peered":"2024-09-21T03:34:33.637827+0000","last_clean":"2024-09-21T03:34:33.637827+0000","last_became_active":"2024-09-21T03:34:33.631914+0000","last_became_peered":"2024-09-21T03:34:33.631914+0000","last_unstale":"2024-09-21T03:34:33.637827+0000","last_undegraded":"2024-09-21T03:34:33.637827+0000","last_fullsized":"2024-09-21T03:34:33.637827+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_clean_scrub_stamp":"2024-09-21T03:34:32.598217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:57:01.944255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":143,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-21T03:34:34.615053+0000","last_change":"2024-09-21T03:33:25.319776+0000","last_active":"2024-09-21T03:34:34.615053+0000","last_peered":"2024-09-21T03:34:34.615053+0000","last_clean":"2024-09-21T03:34:34.615053+0000","last_became_active":"2024-09-21T03:33:25.318399+0000","last_became_peered":"2024-09-21T03:33:25.318399+0000","last_unstale":"2024-09-21T03:34:34.615053+0000","last_undegraded":"2024-09-21T03:34:34.615053+0000","last_fullsized":"2024-09-21T03:34:34.615053+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:23.016672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:23.016672+0000","last_clean_scrub_stamp":"2024-09-21T03:33:23.016672+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:32:35.906114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":80,"acting":80,"num_store_stats":5},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":60,"acting":60,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":1,"up_from":16,"seq":68719476754,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32212,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339628,"statfs":{"total":96636764160,"available":96603779072,"internally_reserved":0,"allocated":1040384,"data_stored":780143,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509458,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31596,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340244,"statfs":{"total":96636764160,"available":96604409856,"internally_reserved":0,"allocated":417792,"data_stored":175919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509457,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31612,"kb_used_data":420,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340228,"statfs":{"total":96636764160,"available":96604393472,"internally_reserved":0,"allocated":430080,"data_stored":182586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509458,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27436,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344404,"statfs":{"total":96636764160,"available":96608669696,"internally_reserved":0,"allocated":417792,"data_stored":175919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509457,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31596,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340244,"statfs":{"total":96636764160,"available":96604409856,"internally_reserved":0,"allocated":417792,"data_stored":175919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542161,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31584,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340256,"statfs":{"total":96636764160,"available":96604422144,"internally_reserved":0,"allocated":405504,"data_stored":169710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574865,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32168,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339672,"statfs":{"total":96636764160,"available":96603824128,"internally_reserved":0,"allocated":999424,"data_stored":760078,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542162,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35756,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336084,"statfs":{"total":96636764160,"available":96600150016,"internally_reserved":0,"allocated":417792,"data_stored":175919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T03:34:38.706 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T03:34:38.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:38.860+0000 7fd379de6640 1 -- 172.21.15.38:0/83355789 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd35c004dd0 msgr2=0x7fd35c0052a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:38.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:38.860+0000 7fd379de6640 1 --2- 172.21.15.38:0/83355789 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd35c004dd0 0x7fd35c0052a0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fd36c001f30 tx=0x7fd35c005830 comp rx=0 tx=0).stop 2024-09-21T03:34:38.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:38.860+0000 7fd373fff640 1 -- 172.21.15.38:0/83355789 reap_dead start 2024-09-21T03:34:38.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:38.860+0000 7fd379de6640 1 -- 172.21.15.38:0/83355789 shutdown_connections 2024-09-21T03:34:38.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:38.860+0000 7fd379de6640 1 -- 172.21.15.38:0/83355789 >> 172.21.15.38:0/83355789 conn(0x7fd3740b2cc0 msgr2=0x7fd3740b30d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:38.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:38.860+0000 7fd379de6640 1 -- 172.21.15.38:0/83355789 shutdown_connections 2024-09-21T03:34:38.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:38.860+0000 7fd379de6640 1 -- 172.21.15.38:0/83355789 wait complete. 2024-09-21T03:34:38.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:38.862+0000 7fd379de6640 1 Processor -- start 2024-09-21T03:34:38.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:38.862+0000 7fd379de6640 1 -- start start 2024-09-21T03:34:38.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:38.863+0000 7fd379de6640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd3741f4850 0x7fd374153400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:38.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:38.863+0000 7fd379de6640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd3741ee620 0x7fd374153b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:38.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:38.863+0000 7fd379de6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd3741540b0 0x7fd3741f1890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:38.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:38.863+0000 7fd379de6640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fd37415b750 con 0x7fd3741540b0 2024-09-21T03:34:38.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:38.863+0000 7fd379de6640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd37415b5d0 con 0x7fd3741ee620 2024-09-21T03:34:38.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:38.863+0000 7fd379de6640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fd37415b8d0 con 0x7fd3741f4850 2024-09-21T03:34:38.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:38.863+0000 7fd373fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd3741540b0 0x7fd3741f1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:38.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:38.863+0000 7fd373fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd3741540b0 0x7fd3741f1890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:33678/0 (socket says 172.21.15.38:33678) 2024-09-21T03:34:38.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:38.863+0000 7fd373fff640 1 -- 172.21.15.38:0/2081127185 learned_addr learned my addr 172.21.15.38:0/2081127185 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:34:38.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:38.864+0000 7fd3737fe640 1 --2- 172.21.15.38:0/2081127185 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd3741f4850 0x7fd374153400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:38.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:38.864+0000 7fd372ffd640 1 --2- 172.21.15.38:0/2081127185 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd3741ee620 0x7fd374153b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:38.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:38.864+0000 7fd373fff640 1 -- 172.21.15.38:0/2081127185 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd3741f4850 msgr2=0x7fd374153400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:38.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:38.864+0000 7fd373fff640 1 --2- 172.21.15.38:0/2081127185 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd3741f4850 0x7fd374153400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:38.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:38.864+0000 7fd373fff640 1 -- 172.21.15.38:0/2081127185 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd3741ee620 msgr2=0x7fd374153b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:38.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:38.864+0000 7fd373fff640 1 --2- 172.21.15.38:0/2081127185 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd3741ee620 0x7fd374153b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:38.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:38.864+0000 7fd373fff640 1 -- 172.21.15.38:0/2081127185 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd374153940 con 0x7fd3741540b0 2024-09-21T03:34:38.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:38.864+0000 7fd3737fe640 1 --2- 172.21.15.38:0/2081127185 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd3741f4850 0x7fd374153400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:34:38.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:38.864+0000 7fd372ffd640 1 --2- 172.21.15.38:0/2081127185 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd3741ee620 0x7fd374153b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:34:38.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:38.864+0000 7fd373fff640 1 --2- 172.21.15.38:0/2081127185 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd3741540b0 0x7fd3741f1890 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fd36c02ade0 tx=0x7fd36c006880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:38.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:38.864+0000 7fd35bfff640 1 -- 172.21.15.38:0/2081127185 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd36c001b50 con 0x7fd3741540b0 2024-09-21T03:34:38.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:38.864+0000 7fd35bfff640 1 -- 172.21.15.38:0/2081127185 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd36c003970 con 0x7fd3741540b0 2024-09-21T03:34:38.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:38.864+0000 7fd35bfff640 1 -- 172.21.15.38:0/2081127185 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd36c006c60 con 0x7fd3741540b0 2024-09-21T03:34:38.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:38.864+0000 7fd379de6640 1 -- 172.21.15.38:0/2081127185 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3741f4e70 con 0x7fd3741540b0 2024-09-21T03:34:38.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:38.864+0000 7fd379de6640 1 -- 172.21.15.38:0/2081127185 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3741f5430 con 0x7fd3741540b0 2024-09-21T03:34:38.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:38.865+0000 7fd379de6640 1 -- 172.21.15.38:0/2081127185 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3741f5770 con 0x7fd3741540b0 2024-09-21T03:34:38.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:38.866+0000 7fd35bfff640 1 -- 172.21.15.38:0/2081127185 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fd36c0346c0 con 0x7fd3741540b0 2024-09-21T03:34:38.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:38.867+0000 7fd35bfff640 1 --2- 172.21.15.38:0/2081127185 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd360078b90 0x7fd36007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:38.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:38.867+0000 7fd35bfff640 1 -- 172.21.15.38:0/2081127185 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7fd36c0ba6c0 con 0x7fd3741540b0 2024-09-21T03:34:38.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:38.867+0000 7fd3737fe640 1 --2- 172.21.15.38:0/2081127185 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd360078b90 0x7fd36007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:38.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:38.868+0000 7fd3737fe640 1 --2- 172.21.15.38:0/2081127185 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd360078b90 0x7fd36007b0a0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fd364000fd0 tx=0x7fd3640035f0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:38.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:38.871+0000 7fd35bfff640 1 -- 172.21.15.38:0/2081127185 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fd3741f5770 con 0x7fd3741540b0 2024-09-21T03:34:39.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:39.056+0000 7fd379de6640 1 -- 172.21.15.38:0/2081127185 --> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd37415be60 con 0x7fd360078b90 2024-09-21T03:34:39.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:39.061+0000 7fd35bfff640 1 -- 172.21.15.38:0/2081127185 <== mgr.4117 v2:172.21.15.195:6800/1631901344 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+249387 (secure 0 0 0) 0x7fd37415be60 con 0x7fd360078b90 2024-09-21T03:34:39.065 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:34:39.067 INFO:teuthology.orchestra.run.smithi038.stderr:dumped all 2024-09-21T03:34:39.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:39.064+0000 7fd379de6640 1 -- 172.21.15.38:0/2081127185 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd360078b90 msgr2=0x7fd36007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:39.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:39.064+0000 7fd379de6640 1 --2- 172.21.15.38:0/2081127185 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd360078b90 0x7fd36007b0a0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fd364000fd0 tx=0x7fd3640035f0 comp rx=0 tx=0).stop 2024-09-21T03:34:39.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:39.064+0000 7fd379de6640 1 -- 172.21.15.38:0/2081127185 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd3741540b0 msgr2=0x7fd3741f1890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:39.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:39.064+0000 7fd379de6640 1 --2- 172.21.15.38:0/2081127185 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd3741540b0 0x7fd3741f1890 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fd36c02ade0 tx=0x7fd36c006880 comp rx=0 tx=0).stop 2024-09-21T03:34:39.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:39.065+0000 7fd379de6640 1 -- 172.21.15.38:0/2081127185 shutdown_connections 2024-09-21T03:34:39.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:39.065+0000 7fd379de6640 1 --2- 172.21.15.38:0/2081127185 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd3741540b0 0x7fd3741f1890 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:39.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:39.065+0000 7fd379de6640 1 --2- 172.21.15.38:0/2081127185 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd3741ee620 0x7fd374153b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:39.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:39.065+0000 7fd379de6640 1 --2- 172.21.15.38:0/2081127185 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd360078b90 0x7fd36007b0a0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:39.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:39.065+0000 7fd379de6640 1 --2- 172.21.15.38:0/2081127185 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd3741f4850 0x7fd374153400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:39.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:39.065+0000 7fd379de6640 1 -- 172.21.15.38:0/2081127185 >> 172.21.15.38:0/2081127185 conn(0x7fd3740b2cc0 msgr2=0x7fd3740cf5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:39.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:39.065+0000 7fd379de6640 1 -- 172.21.15.38:0/2081127185 shutdown_connections 2024-09-21T03:34:39.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:39.065+0000 7fd379de6640 1 -- 172.21.15.38:0/2081127185 wait complete. 2024-09-21T03:34:39.078 INFO:teuthology.orchestra.run.smithi038.stdout:{"pg_ready":true,"pg_map":{"version":71,"stamp":"2024-09-21T03:34:38.993186+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592036,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":115,"num_write_kb":1639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":87,"ondisk_log_size":87,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":76,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":254116,"kb_used_data":4588,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754720604,"statfs":{"total":773094113280,"available":772833898496,"internally_reserved":0,"allocated":4698112,"data_stored":2665849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12748,"internal_metadata":255315508},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1668,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001614"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:35.628233+0000","last_change":"2024-09-21T03:34:34.631062+0000","last_active":"2024-09-21T03:34:35.628233+0000","last_peered":"2024-09-21T03:34:35.628233+0000","last_clean":"2024-09-21T03:34:35.628233+0000","last_became_active":"2024-09-21T03:34:34.630468+0000","last_became_peered":"2024-09-21T03:34:34.630468+0000","last_unstale":"2024-09-21T03:34:35.628233+0000","last_undegraded":"2024-09-21T03:34:35.628233+0000","last_fullsized":"2024-09-21T03:34:35.628233+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:24:06.812940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:36.646828+0000","last_change":"2024-09-21T03:34:34.626744+0000","last_active":"2024-09-21T03:34:36.646828+0000","last_peered":"2024-09-21T03:34:36.646828+0000","last_clean":"2024-09-21T03:34:36.646828+0000","last_became_active":"2024-09-21T03:34:34.626445+0000","last_became_peered":"2024-09-21T03:34:34.626445+0000","last_unstale":"2024-09-21T03:34:36.646828+0000","last_undegraded":"2024-09-21T03:34:36.646828+0000","last_fullsized":"2024-09-21T03:34:36.646828+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:28:02.542581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:35.628793+0000","last_change":"2024-09-21T03:34:34.636439+0000","last_active":"2024-09-21T03:34:35.628793+0000","last_peered":"2024-09-21T03:34:35.628793+0000","last_clean":"2024-09-21T03:34:35.628793+0000","last_became_active":"2024-09-21T03:34:34.635938+0000","last_became_peered":"2024-09-21T03:34:34.635938+0000","last_unstale":"2024-09-21T03:34:35.628793+0000","last_undegraded":"2024-09-21T03:34:35.628793+0000","last_fullsized":"2024-09-21T03:34:35.628793+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:13:58.235543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:35.628343+0000","last_change":"2024-09-21T03:34:34.634523+0000","last_active":"2024-09-21T03:34:35.628343+0000","last_peered":"2024-09-21T03:34:35.628343+0000","last_clean":"2024-09-21T03:34:35.628343+0000","last_became_active":"2024-09-21T03:34:34.633777+0000","last_became_peered":"2024-09-21T03:34:34.633777+0000","last_unstale":"2024-09-21T03:34:35.628343+0000","last_undegraded":"2024-09-21T03:34:35.628343+0000","last_fullsized":"2024-09-21T03:34:35.628343+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:18:03.263504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-21T03:34:33.614399+0000","last_change":"2024-09-21T03:34:33.614399+0000","last_active":"2024-09-21T03:34:33.604595+0000","last_peered":"2024-09-21T03:34:33.604595+0000","last_clean":"2024-09-21T03:34:33.604595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:34:33.614399+0000","last_undegraded":"2024-09-21T03:34:33.614399+0000","last_fullsized":"2024-09-21T03:34:33.614399+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:36.646409+0000","last_change":"2024-09-21T03:34:34.634322+0000","last_active":"2024-09-21T03:34:36.646409+0000","last_peered":"2024-09-21T03:34:36.646409+0000","last_clean":"2024-09-21T03:34:36.646409+0000","last_became_active":"2024-09-21T03:34:34.634023+0000","last_became_peered":"2024-09-21T03:34:34.634023+0000","last_unstale":"2024-09-21T03:34:36.646409+0000","last_undegraded":"2024-09-21T03:34:36.646409+0000","last_fullsized":"2024-09-21T03:34:36.646409+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:32:33.650331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-21T03:34:34.638247+0000","last_change":"2024-09-21T03:34:34.629606+0000","last_active":"2024-09-21T03:34:34.638247+0000","last_peered":"2024-09-21T03:34:34.638247+0000","last_clean":"2024-09-21T03:34:34.638247+0000","last_became_active":"2024-09-21T03:34:34.628992+0000","last_became_peered":"2024-09-21T03:34:34.628992+0000","last_unstale":"2024-09-21T03:34:34.638247+0000","last_undegraded":"2024-09-21T03:34:34.638247+0000","last_fullsized":"2024-09-21T03:34:34.638247+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:34:28.501979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:35.627528+0000","last_change":"2024-09-21T03:34:34.637695+0000","last_active":"2024-09-21T03:34:35.627528+0000","last_peered":"2024-09-21T03:34:35.627528+0000","last_clean":"2024-09-21T03:34:35.627528+0000","last_became_active":"2024-09-21T03:34:34.637320+0000","last_became_peered":"2024-09-21T03:34:34.637320+0000","last_unstale":"2024-09-21T03:34:35.627528+0000","last_undegraded":"2024-09-21T03:34:35.627528+0000","last_fullsized":"2024-09-21T03:34:35.627528+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:20:18.794303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-21T03:34:33.614285+0000","last_change":"2024-09-21T03:34:33.614285+0000","last_active":"2024-09-21T03:34:33.604595+0000","last_peered":"2024-09-21T03:34:33.604595+0000","last_clean":"2024-09-21T03:34:33.604595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:34:33.614285+0000","last_undegraded":"2024-09-21T03:34:33.614285+0000","last_fullsized":"2024-09-21T03:34:33.614285+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-21T03:34:34.642546+0000","last_change":"2024-09-21T03:34:34.637080+0000","last_active":"2024-09-21T03:34:34.642546+0000","last_peered":"2024-09-21T03:34:34.642546+0000","last_clean":"2024-09-21T03:34:34.642546+0000","last_became_active":"2024-09-21T03:34:34.636777+0000","last_became_peered":"2024-09-21T03:34:34.636777+0000","last_unstale":"2024-09-21T03:34:34.642546+0000","last_undegraded":"2024-09-21T03:34:34.642546+0000","last_fullsized":"2024-09-21T03:34:34.642546+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:03:32.166555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-21T03:34:33.622983+0000","last_change":"2024-09-21T03:34:33.622983+0000","last_active":"2024-09-21T03:34:33.604595+0000","last_peered":"2024-09-21T03:34:33.604595+0000","last_clean":"2024-09-21T03:34:33.604595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:34:33.622983+0000","last_undegraded":"2024-09-21T03:34:33.622983+0000","last_fullsized":"2024-09-21T03:34:33.622983+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-21T03:34:33.614478+0000","last_change":"2024-09-21T03:34:33.614478+0000","last_active":"2024-09-21T03:34:33.604595+0000","last_peered":"2024-09-21T03:34:33.604595+0000","last_clean":"2024-09-21T03:34:33.604595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:34:33.614478+0000","last_undegraded":"2024-09-21T03:34:33.614478+0000","last_fullsized":"2024-09-21T03:34:33.614478+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-21T03:34:34.642073+0000","last_change":"2024-09-21T03:34:34.629186+0000","last_active":"2024-09-21T03:34:34.642073+0000","last_peered":"2024-09-21T03:34:34.642073+0000","last_clean":"2024-09-21T03:34:34.642073+0000","last_became_active":"2024-09-21T03:34:34.628801+0000","last_became_peered":"2024-09-21T03:34:34.628801+0000","last_unstale":"2024-09-21T03:34:34.642073+0000","last_undegraded":"2024-09-21T03:34:34.642073+0000","last_fullsized":"2024-09-21T03:34:34.642073+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:34:03.702144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-21T03:34:33.616107+0000","last_change":"2024-09-21T03:34:33.616107+0000","last_active":"2024-09-21T03:34:33.604595+0000","last_peered":"2024-09-21T03:34:33.604595+0000","last_clean":"2024-09-21T03:34:33.604595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:34:33.616107+0000","last_undegraded":"2024-09-21T03:34:33.616107+0000","last_fullsized":"2024-09-21T03:34:33.616107+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-21T03:34:33.625487+0000","last_change":"2024-09-21T03:34:33.625487+0000","last_active":"2024-09-21T03:34:33.604595+0000","last_peered":"2024-09-21T03:34:33.604595+0000","last_clean":"2024-09-21T03:34:33.604595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:34:33.625487+0000","last_undegraded":"2024-09-21T03:34:33.625487+0000","last_fullsized":"2024-09-21T03:34:33.625487+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-21T03:34:34.644136+0000","last_change":"2024-09-21T03:34:34.635135+0000","last_active":"2024-09-21T03:34:34.644136+0000","last_peered":"2024-09-21T03:34:34.644136+0000","last_clean":"2024-09-21T03:34:34.644136+0000","last_became_active":"2024-09-21T03:34:34.634811+0000","last_became_peered":"2024-09-21T03:34:34.634811+0000","last_unstale":"2024-09-21T03:34:34.644136+0000","last_undegraded":"2024-09-21T03:34:34.644136+0000","last_fullsized":"2024-09-21T03:34:34.644136+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:44:10.495963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-21T03:34:33.624716+0000","last_change":"2024-09-21T03:34:33.624716+0000","last_active":"2024-09-21T03:34:33.604595+0000","last_peered":"2024-09-21T03:34:33.604595+0000","last_clean":"2024-09-21T03:34:33.604595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:34:33.624716+0000","last_undegraded":"2024-09-21T03:34:33.624716+0000","last_fullsized":"2024-09-21T03:34:33.624716+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-21T03:34:33.623146+0000","last_change":"2024-09-21T03:34:33.623146+0000","last_active":"2024-09-21T03:34:33.604595+0000","last_peered":"2024-09-21T03:34:33.604595+0000","last_clean":"2024-09-21T03:34:33.604595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:34:33.623146+0000","last_undegraded":"2024-09-21T03:34:33.623146+0000","last_fullsized":"2024-09-21T03:34:33.623146+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-21T03:34:34.639249+0000","last_change":"2024-09-21T03:34:34.637926+0000","last_active":"2024-09-21T03:34:34.639249+0000","last_peered":"2024-09-21T03:34:34.639249+0000","last_clean":"2024-09-21T03:34:34.639249+0000","last_became_active":"2024-09-21T03:34:34.637546+0000","last_became_peered":"2024-09-21T03:34:34.637546+0000","last_unstale":"2024-09-21T03:34:34.639249+0000","last_undegraded":"2024-09-21T03:34:34.639249+0000","last_fullsized":"2024-09-21T03:34:34.639249+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:39:27.102702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-21T03:34:33.620475+0000","last_change":"2024-09-21T03:34:33.620475+0000","last_active":"2024-09-21T03:34:33.604595+0000","last_peered":"2024-09-21T03:34:33.604595+0000","last_clean":"2024-09-21T03:34:33.604595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:34:33.620475+0000","last_undegraded":"2024-09-21T03:34:33.620475+0000","last_fullsized":"2024-09-21T03:34:33.620475+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-21T03:34:33.621111+0000","last_change":"2024-09-21T03:34:33.621111+0000","last_active":"2024-09-21T03:34:33.604595+0000","last_peered":"2024-09-21T03:34:33.604595+0000","last_clean":"2024-09-21T03:34:33.604595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:34:33.621111+0000","last_undegraded":"2024-09-21T03:34:33.621111+0000","last_fullsized":"2024-09-21T03:34:33.621111+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-21T03:34:33.621590+0000","last_change":"2024-09-21T03:34:33.621590+0000","last_active":"2024-09-21T03:34:33.604595+0000","last_peered":"2024-09-21T03:34:33.604595+0000","last_clean":"2024-09-21T03:34:33.604595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:34:33.621590+0000","last_undegraded":"2024-09-21T03:34:33.621590+0000","last_fullsized":"2024-09-21T03:34:33.621590+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-21T03:34:33.624872+0000","last_change":"2024-09-21T03:34:33.624872+0000","last_active":"2024-09-21T03:34:33.604595+0000","last_peered":"2024-09-21T03:34:33.604595+0000","last_clean":"2024-09-21T03:34:33.604595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:34:33.624872+0000","last_undegraded":"2024-09-21T03:34:33.624872+0000","last_fullsized":"2024-09-21T03:34:33.624872+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:35.627534+0000","last_change":"2024-09-21T03:34:34.641334+0000","last_active":"2024-09-21T03:34:35.627534+0000","last_peered":"2024-09-21T03:34:35.627534+0000","last_clean":"2024-09-21T03:34:35.627534+0000","last_became_active":"2024-09-21T03:34:34.640922+0000","last_became_peered":"2024-09-21T03:34:34.640922+0000","last_unstale":"2024-09-21T03:34:35.627534+0000","last_undegraded":"2024-09-21T03:34:35.627534+0000","last_fullsized":"2024-09-21T03:34:35.627534+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:12:37.794061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-21T03:34:33.616131+0000","last_change":"2024-09-21T03:34:33.616131+0000","last_active":"2024-09-21T03:34:33.604595+0000","last_peered":"2024-09-21T03:34:33.604595+0000","last_clean":"2024-09-21T03:34:33.604595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:34:33.616131+0000","last_undegraded":"2024-09-21T03:34:33.616131+0000","last_fullsized":"2024-09-21T03:34:33.616131+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:35.628400+0000","last_change":"2024-09-21T03:34:34.640146+0000","last_active":"2024-09-21T03:34:35.628400+0000","last_peered":"2024-09-21T03:34:35.628400+0000","last_clean":"2024-09-21T03:34:35.628400+0000","last_became_active":"2024-09-21T03:34:34.639791+0000","last_became_peered":"2024-09-21T03:34:34.639791+0000","last_unstale":"2024-09-21T03:34:35.628400+0000","last_undegraded":"2024-09-21T03:34:35.628400+0000","last_fullsized":"2024-09-21T03:34:35.628400+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:23:41.788750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-21T03:34:33.622462+0000","last_change":"2024-09-21T03:34:33.622462+0000","last_active":"2024-09-21T03:34:33.604595+0000","last_peered":"2024-09-21T03:34:33.604595+0000","last_clean":"2024-09-21T03:34:33.604595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:34:33.622462+0000","last_undegraded":"2024-09-21T03:34:33.622462+0000","last_fullsized":"2024-09-21T03:34:33.622462+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-21T03:34:34.639470+0000","last_change":"2024-09-21T03:34:34.633013+0000","last_active":"2024-09-21T03:34:34.639470+0000","last_peered":"2024-09-21T03:34:34.639470+0000","last_clean":"2024-09-21T03:34:34.639470+0000","last_became_active":"2024-09-21T03:34:34.631701+0000","last_became_peered":"2024-09-21T03:34:34.631701+0000","last_unstale":"2024-09-21T03:34:34.639470+0000","last_undegraded":"2024-09-21T03:34:34.639470+0000","last_fullsized":"2024-09-21T03:34:34.639470+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:58:27.086990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-21T03:34:33.628158+0000","last_change":"2024-09-21T03:34:33.628158+0000","last_active":"2024-09-21T03:34:33.604595+0000","last_peered":"2024-09-21T03:34:33.604595+0000","last_clean":"2024-09-21T03:34:33.604595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:34:33.628158+0000","last_undegraded":"2024-09-21T03:34:33.628158+0000","last_fullsized":"2024-09-21T03:34:33.628158+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-21T03:34:33.621235+0000","last_change":"2024-09-21T03:34:33.621235+0000","last_active":"2024-09-21T03:34:33.604595+0000","last_peered":"2024-09-21T03:34:33.604595+0000","last_clean":"2024-09-21T03:34:33.604595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:34:33.621235+0000","last_undegraded":"2024-09-21T03:34:33.621235+0000","last_fullsized":"2024-09-21T03:34:33.621235+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-21T03:34:34.639102+0000","last_change":"2024-09-21T03:34:34.629415+0000","last_active":"2024-09-21T03:34:34.639102+0000","last_peered":"2024-09-21T03:34:34.639102+0000","last_clean":"2024-09-21T03:34:34.639102+0000","last_became_active":"2024-09-21T03:34:34.628524+0000","last_became_peered":"2024-09-21T03:34:34.628524+0000","last_unstale":"2024-09-21T03:34:34.639102+0000","last_undegraded":"2024-09-21T03:34:34.639102+0000","last_fullsized":"2024-09-21T03:34:34.639102+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:48:30.216711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:35.627792+0000","last_change":"2024-09-21T03:34:34.639100+0000","last_active":"2024-09-21T03:34:35.627792+0000","last_peered":"2024-09-21T03:34:35.627792+0000","last_clean":"2024-09-21T03:34:35.627792+0000","last_became_active":"2024-09-21T03:34:34.636285+0000","last_became_peered":"2024-09-21T03:34:34.636285+0000","last_unstale":"2024-09-21T03:34:35.627792+0000","last_undegraded":"2024-09-21T03:34:35.627792+0000","last_fullsized":"2024-09-21T03:34:35.627792+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:19:39.778069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-21T03:34:33.621969+0000","last_change":"2024-09-21T03:34:33.621969+0000","last_active":"2024-09-21T03:34:33.604595+0000","last_peered":"2024-09-21T03:34:33.604595+0000","last_clean":"2024-09-21T03:34:33.604595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:34:33.621969+0000","last_undegraded":"2024-09-21T03:34:33.621969+0000","last_fullsized":"2024-09-21T03:34:33.621969+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T03:34:33.620882+0000","last_change":"2024-09-21T03:34:33.618066+0000","last_active":"2024-09-21T03:34:33.620882+0000","last_peered":"2024-09-21T03:34:33.620882+0000","last_clean":"2024-09-21T03:34:33.620882+0000","last_became_active":"2024-09-21T03:34:33.617685+0000","last_became_peered":"2024-09-21T03:34:33.617685+0000","last_unstale":"2024-09-21T03:34:33.620882+0000","last_undegraded":"2024-09-21T03:34:33.620882+0000","last_fullsized":"2024-09-21T03:34:33.620882+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_clean_scrub_stamp":"2024-09-21T03:34:32.598217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:34:47.997359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:36.646764+0000","last_change":"2024-09-21T03:34:34.635480+0000","last_active":"2024-09-21T03:34:36.646764+0000","last_peered":"2024-09-21T03:34:36.646764+0000","last_clean":"2024-09-21T03:34:36.646764+0000","last_became_active":"2024-09-21T03:34:34.634979+0000","last_became_peered":"2024-09-21T03:34:34.634979+0000","last_unstale":"2024-09-21T03:34:36.646764+0000","last_undegraded":"2024-09-21T03:34:36.646764+0000","last_fullsized":"2024-09-21T03:34:36.646764+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:25:43.132670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"39'1","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:36.646633+0000","last_change":"2024-09-21T03:34:33.636543+0000","last_active":"2024-09-21T03:34:36.646633+0000","last_peered":"2024-09-21T03:34:36.646633+0000","last_clean":"2024-09-21T03:34:36.646633+0000","last_became_active":"2024-09-21T03:34:33.636151+0000","last_became_peered":"2024-09-21T03:34:33.636151+0000","last_unstale":"2024-09-21T03:34:36.646633+0000","last_undegraded":"2024-09-21T03:34:36.646633+0000","last_fullsized":"2024-09-21T03:34:36.646633+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_clean_scrub_stamp":"2024-09-21T03:34:32.598217+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:17:30.670560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-21T03:34:33.622277+0000","last_change":"2024-09-21T03:34:33.622277+0000","last_active":"2024-09-21T03:34:33.604595+0000","last_peered":"2024-09-21T03:34:33.604595+0000","last_clean":"2024-09-21T03:34:33.604595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:34:33.622277+0000","last_undegraded":"2024-09-21T03:34:33.622277+0000","last_fullsized":"2024-09-21T03:34:33.622277+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"39'2","reported_seq":17,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:35.628849+0000","last_change":"2024-09-21T03:34:33.629398+0000","last_active":"2024-09-21T03:34:35.628849+0000","last_peered":"2024-09-21T03:34:35.628849+0000","last_clean":"2024-09-21T03:34:35.628849+0000","last_became_active":"2024-09-21T03:34:33.629051+0000","last_became_peered":"2024-09-21T03:34:33.629051+0000","last_unstale":"2024-09-21T03:34:35.628849+0000","last_undegraded":"2024-09-21T03:34:35.628849+0000","last_fullsized":"2024-09-21T03:34:35.628849+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_clean_scrub_stamp":"2024-09-21T03:34:32.598217+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:46:38.739080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:36.646761+0000","last_change":"2024-09-21T03:34:34.629283+0000","last_active":"2024-09-21T03:34:36.646761+0000","last_peered":"2024-09-21T03:34:36.646761+0000","last_clean":"2024-09-21T03:34:36.646761+0000","last_became_active":"2024-09-21T03:34:34.628942+0000","last_became_peered":"2024-09-21T03:34:34.628942+0000","last_unstale":"2024-09-21T03:34:36.646761+0000","last_undegraded":"2024-09-21T03:34:36.646761+0000","last_fullsized":"2024-09-21T03:34:36.646761+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:15:48.821673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T03:34:33.638430+0000","last_change":"2024-09-21T03:34:33.631010+0000","last_active":"2024-09-21T03:34:33.638430+0000","last_peered":"2024-09-21T03:34:33.638430+0000","last_clean":"2024-09-21T03:34:33.638430+0000","last_became_active":"2024-09-21T03:34:33.630576+0000","last_became_peered":"2024-09-21T03:34:33.630576+0000","last_unstale":"2024-09-21T03:34:33.638430+0000","last_undegraded":"2024-09-21T03:34:33.638430+0000","last_fullsized":"2024-09-21T03:34:33.638430+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_clean_scrub_stamp":"2024-09-21T03:34:32.598217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:50:07.223556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:35.627813+0000","last_change":"2024-09-21T03:34:34.629518+0000","last_active":"2024-09-21T03:34:35.627813+0000","last_peered":"2024-09-21T03:34:35.627813+0000","last_clean":"2024-09-21T03:34:35.627813+0000","last_became_active":"2024-09-21T03:34:34.629216+0000","last_became_peered":"2024-09-21T03:34:34.629216+0000","last_unstale":"2024-09-21T03:34:35.627813+0000","last_undegraded":"2024-09-21T03:34:35.627813+0000","last_fullsized":"2024-09-21T03:34:35.627813+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:31:59.133888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T03:34:33.635417+0000","last_change":"2024-09-21T03:34:33.619056+0000","last_active":"2024-09-21T03:34:33.635417+0000","last_peered":"2024-09-21T03:34:33.635417+0000","last_clean":"2024-09-21T03:34:33.635417+0000","last_became_active":"2024-09-21T03:34:33.618698+0000","last_became_peered":"2024-09-21T03:34:33.618698+0000","last_unstale":"2024-09-21T03:34:33.635417+0000","last_undegraded":"2024-09-21T03:34:33.635417+0000","last_fullsized":"2024-09-21T03:34:33.635417+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_clean_scrub_stamp":"2024-09-21T03:34:32.598217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:58:31.223579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-21T03:34:33.618669+0000","last_change":"2024-09-21T03:34:33.618669+0000","last_active":"2024-09-21T03:34:33.604595+0000","last_peered":"2024-09-21T03:34:33.604595+0000","last_clean":"2024-09-21T03:34:33.604595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:34:33.618669+0000","last_undegraded":"2024-09-21T03:34:33.618669+0000","last_fullsized":"2024-09-21T03:34:33.618669+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1a","version":"39'1","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-21T03:34:34.633517+0000","last_change":"2024-09-21T03:34:33.634501+0000","last_active":"2024-09-21T03:34:34.633517+0000","last_peered":"2024-09-21T03:34:34.633517+0000","last_clean":"2024-09-21T03:34:34.633517+0000","last_became_active":"2024-09-21T03:34:33.633827+0000","last_became_peered":"2024-09-21T03:34:33.633827+0000","last_unstale":"2024-09-21T03:34:34.633517+0000","last_undegraded":"2024-09-21T03:34:34.633517+0000","last_fullsized":"2024-09-21T03:34:34.633517+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_clean_scrub_stamp":"2024-09-21T03:34:32.598217+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:33:52.483902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-21T03:34:33.620634+0000","last_change":"2024-09-21T03:34:33.620634+0000","last_active":"2024-09-21T03:34:33.604595+0000","last_peered":"2024-09-21T03:34:33.604595+0000","last_clean":"2024-09-21T03:34:33.604595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:34:33.620634+0000","last_undegraded":"2024-09-21T03:34:33.620634+0000","last_fullsized":"2024-09-21T03:34:33.620634+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:35.627409+0000","last_change":"2024-09-21T03:34:33.636797+0000","last_active":"2024-09-21T03:34:35.627409+0000","last_peered":"2024-09-21T03:34:35.627409+0000","last_clean":"2024-09-21T03:34:35.627409+0000","last_became_active":"2024-09-21T03:34:33.635927+0000","last_became_peered":"2024-09-21T03:34:33.635927+0000","last_unstale":"2024-09-21T03:34:35.627409+0000","last_undegraded":"2024-09-21T03:34:35.627409+0000","last_fullsized":"2024-09-21T03:34:35.627409+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_clean_scrub_stamp":"2024-09-21T03:34:32.598217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:23:52.412874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:36.646794+0000","last_change":"2024-09-21T03:34:34.640266+0000","last_active":"2024-09-21T03:34:36.646794+0000","last_peered":"2024-09-21T03:34:36.646794+0000","last_clean":"2024-09-21T03:34:36.646794+0000","last_became_active":"2024-09-21T03:34:34.639922+0000","last_became_peered":"2024-09-21T03:34:34.639922+0000","last_unstale":"2024-09-21T03:34:36.646794+0000","last_undegraded":"2024-09-21T03:34:36.646794+0000","last_fullsized":"2024-09-21T03:34:36.646794+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:38:18.167710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-21T03:34:34.615067+0000","last_change":"2024-09-21T03:34:33.638401+0000","last_active":"2024-09-21T03:34:34.615067+0000","last_peered":"2024-09-21T03:34:34.615067+0000","last_clean":"2024-09-21T03:34:34.615067+0000","last_became_active":"2024-09-21T03:34:33.637364+0000","last_became_peered":"2024-09-21T03:34:33.637364+0000","last_unstale":"2024-09-21T03:34:34.615067+0000","last_undegraded":"2024-09-21T03:34:34.615067+0000","last_fullsized":"2024-09-21T03:34:34.615067+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_clean_scrub_stamp":"2024-09-21T03:34:32.598217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:37:36.532174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:35.628541+0000","last_change":"2024-09-21T03:34:34.633927+0000","last_active":"2024-09-21T03:34:35.628541+0000","last_peered":"2024-09-21T03:34:35.628541+0000","last_clean":"2024-09-21T03:34:35.628541+0000","last_became_active":"2024-09-21T03:34:34.633294+0000","last_became_peered":"2024-09-21T03:34:34.633294+0000","last_unstale":"2024-09-21T03:34:35.628541+0000","last_undegraded":"2024-09-21T03:34:35.628541+0000","last_fullsized":"2024-09-21T03:34:35.628541+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:41:47.832177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:35.628042+0000","last_change":"2024-09-21T03:34:33.626093+0000","last_active":"2024-09-21T03:34:35.628042+0000","last_peered":"2024-09-21T03:34:35.628042+0000","last_clean":"2024-09-21T03:34:35.628042+0000","last_became_active":"2024-09-21T03:34:33.625795+0000","last_became_peered":"2024-09-21T03:34:33.625795+0000","last_unstale":"2024-09-21T03:34:35.628042+0000","last_undegraded":"2024-09-21T03:34:35.628042+0000","last_fullsized":"2024-09-21T03:34:35.628042+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_clean_scrub_stamp":"2024-09-21T03:34:32.598217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:39:10.947435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-21T03:34:33.616082+0000","last_change":"2024-09-21T03:34:33.616082+0000","last_active":"2024-09-21T03:34:33.604595+0000","last_peered":"2024-09-21T03:34:33.604595+0000","last_clean":"2024-09-21T03:34:33.604595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:34:33.616082+0000","last_undegraded":"2024-09-21T03:34:33.616082+0000","last_fullsized":"2024-09-21T03:34:33.616082+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T03:34:33.638093+0000","last_change":"2024-09-21T03:34:33.633888+0000","last_active":"2024-09-21T03:34:33.638093+0000","last_peered":"2024-09-21T03:34:33.638093+0000","last_clean":"2024-09-21T03:34:33.638093+0000","last_became_active":"2024-09-21T03:34:33.633372+0000","last_became_peered":"2024-09-21T03:34:33.633372+0000","last_unstale":"2024-09-21T03:34:33.638093+0000","last_undegraded":"2024-09-21T03:34:33.638093+0000","last_fullsized":"2024-09-21T03:34:33.638093+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_clean_scrub_stamp":"2024-09-21T03:34:32.598217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:58:21.314833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:35.628444+0000","last_change":"2024-09-21T03:34:34.644095+0000","last_active":"2024-09-21T03:34:35.628444+0000","last_peered":"2024-09-21T03:34:35.628444+0000","last_clean":"2024-09-21T03:34:35.628444+0000","last_became_active":"2024-09-21T03:34:34.643713+0000","last_became_peered":"2024-09-21T03:34:34.643713+0000","last_unstale":"2024-09-21T03:34:35.628444+0000","last_undegraded":"2024-09-21T03:34:35.628444+0000","last_fullsized":"2024-09-21T03:34:35.628444+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:40:54.924468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T03:34:33.638035+0000","last_change":"2024-09-21T03:34:33.632451+0000","last_active":"2024-09-21T03:34:33.638035+0000","last_peered":"2024-09-21T03:34:33.638035+0000","last_clean":"2024-09-21T03:34:33.638035+0000","last_became_active":"2024-09-21T03:34:33.632103+0000","last_became_peered":"2024-09-21T03:34:33.632103+0000","last_unstale":"2024-09-21T03:34:33.638035+0000","last_undegraded":"2024-09-21T03:34:33.638035+0000","last_fullsized":"2024-09-21T03:34:33.638035+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_clean_scrub_stamp":"2024-09-21T03:34:32.598217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:22:08.770678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-21T03:34:34.639059+0000","last_change":"2024-09-21T03:34:34.628419+0000","last_active":"2024-09-21T03:34:34.639059+0000","last_peered":"2024-09-21T03:34:34.639059+0000","last_clean":"2024-09-21T03:34:34.639059+0000","last_became_active":"2024-09-21T03:34:34.628087+0000","last_became_peered":"2024-09-21T03:34:34.628087+0000","last_unstale":"2024-09-21T03:34:34.639059+0000","last_undegraded":"2024-09-21T03:34:34.639059+0000","last_fullsized":"2024-09-21T03:34:34.639059+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:28:03.240497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.14","version":"39'6","reported_seq":21,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:35.628508+0000","last_change":"2024-09-21T03:34:33.639043+0000","last_active":"2024-09-21T03:34:35.628508+0000","last_peered":"2024-09-21T03:34:35.628508+0000","last_clean":"2024-09-21T03:34:35.628508+0000","last_became_active":"2024-09-21T03:34:33.638761+0000","last_became_peered":"2024-09-21T03:34:33.638761+0000","last_unstale":"2024-09-21T03:34:35.628508+0000","last_undegraded":"2024-09-21T03:34:35.628508+0000","last_fullsized":"2024-09-21T03:34:35.628508+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_clean_scrub_stamp":"2024-09-21T03:34:32.598217+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:44:53.992927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:35.628094+0000","last_change":"2024-09-21T03:34:34.640167+0000","last_active":"2024-09-21T03:34:35.628094+0000","last_peered":"2024-09-21T03:34:35.628094+0000","last_clean":"2024-09-21T03:34:35.628094+0000","last_became_active":"2024-09-21T03:34:34.639859+0000","last_became_peered":"2024-09-21T03:34:34.639859+0000","last_unstale":"2024-09-21T03:34:35.628094+0000","last_undegraded":"2024-09-21T03:34:35.628094+0000","last_fullsized":"2024-09-21T03:34:35.628094+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:36:48.213957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"39'2","reported_seq":17,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:35.627656+0000","last_change":"2024-09-21T03:34:33.628173+0000","last_active":"2024-09-21T03:34:35.627656+0000","last_peered":"2024-09-21T03:34:35.627656+0000","last_clean":"2024-09-21T03:34:35.627656+0000","last_became_active":"2024-09-21T03:34:33.627559+0000","last_became_peered":"2024-09-21T03:34:33.627559+0000","last_unstale":"2024-09-21T03:34:35.627656+0000","last_undegraded":"2024-09-21T03:34:35.627656+0000","last_fullsized":"2024-09-21T03:34:35.627656+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_clean_scrub_stamp":"2024-09-21T03:34:32.598217+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:34:22.474063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:36.646395+0000","last_change":"2024-09-21T03:34:34.626145+0000","last_active":"2024-09-21T03:34:36.646395+0000","last_peered":"2024-09-21T03:34:36.646395+0000","last_clean":"2024-09-21T03:34:36.646395+0000","last_became_active":"2024-09-21T03:34:34.625819+0000","last_became_peered":"2024-09-21T03:34:34.625819+0000","last_unstale":"2024-09-21T03:34:36.646395+0000","last_undegraded":"2024-09-21T03:34:36.646395+0000","last_fullsized":"2024-09-21T03:34:36.646395+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:44:09.663791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:36.646276+0000","last_change":"2024-09-21T03:34:33.630029+0000","last_active":"2024-09-21T03:34:36.646276+0000","last_peered":"2024-09-21T03:34:36.646276+0000","last_clean":"2024-09-21T03:34:36.646276+0000","last_became_active":"2024-09-21T03:34:33.629699+0000","last_became_peered":"2024-09-21T03:34:33.629699+0000","last_unstale":"2024-09-21T03:34:36.646276+0000","last_undegraded":"2024-09-21T03:34:36.646276+0000","last_fullsized":"2024-09-21T03:34:36.646276+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_clean_scrub_stamp":"2024-09-21T03:34:32.598217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:49:16.438096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-21T03:34:33.619516+0000","last_change":"2024-09-21T03:34:33.619516+0000","last_active":"2024-09-21T03:34:33.604595+0000","last_peered":"2024-09-21T03:34:33.604595+0000","last_clean":"2024-09-21T03:34:33.604595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:34:33.619516+0000","last_undegraded":"2024-09-21T03:34:33.619516+0000","last_fullsized":"2024-09-21T03:34:33.619516+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T03:34:33.644077+0000","last_change":"2024-09-21T03:34:33.639290+0000","last_active":"2024-09-21T03:34:33.644077+0000","last_peered":"2024-09-21T03:34:33.644077+0000","last_clean":"2024-09-21T03:34:33.644077+0000","last_became_active":"2024-09-21T03:34:33.638941+0000","last_became_peered":"2024-09-21T03:34:33.638941+0000","last_unstale":"2024-09-21T03:34:33.644077+0000","last_undegraded":"2024-09-21T03:34:33.644077+0000","last_fullsized":"2024-09-21T03:34:33.644077+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_clean_scrub_stamp":"2024-09-21T03:34:32.598217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:46:42.303637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-21T03:34:33.613436+0000","last_change":"2024-09-21T03:34:33.613436+0000","last_active":"2024-09-21T03:34:33.604595+0000","last_peered":"2024-09-21T03:34:33.604595+0000","last_clean":"2024-09-21T03:34:33.604595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:34:33.613436+0000","last_undegraded":"2024-09-21T03:34:33.613436+0000","last_fullsized":"2024-09-21T03:34:33.613436+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T03:34:33.638439+0000","last_change":"2024-09-21T03:34:33.632101+0000","last_active":"2024-09-21T03:34:33.638439+0000","last_peered":"2024-09-21T03:34:33.638439+0000","last_clean":"2024-09-21T03:34:33.638439+0000","last_became_active":"2024-09-21T03:34:33.631744+0000","last_became_peered":"2024-09-21T03:34:33.631744+0000","last_unstale":"2024-09-21T03:34:33.638439+0000","last_undegraded":"2024-09-21T03:34:33.638439+0000","last_fullsized":"2024-09-21T03:34:33.638439+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_clean_scrub_stamp":"2024-09-21T03:34:32.598217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:17:30.670560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:36.646635+0000","last_change":"2024-09-21T03:34:34.626189+0000","last_active":"2024-09-21T03:34:36.646635+0000","last_peered":"2024-09-21T03:34:36.646635+0000","last_clean":"2024-09-21T03:34:36.646635+0000","last_became_active":"2024-09-21T03:34:34.625865+0000","last_became_peered":"2024-09-21T03:34:34.625865+0000","last_unstale":"2024-09-21T03:34:36.646635+0000","last_undegraded":"2024-09-21T03:34:36.646635+0000","last_fullsized":"2024-09-21T03:34:36.646635+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:13:44.244299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.f","version":"39'1","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-21T03:34:34.638168+0000","last_change":"2024-09-21T03:34:33.637659+0000","last_active":"2024-09-21T03:34:34.638168+0000","last_peered":"2024-09-21T03:34:34.638168+0000","last_clean":"2024-09-21T03:34:34.638168+0000","last_became_active":"2024-09-21T03:34:33.637317+0000","last_became_peered":"2024-09-21T03:34:33.637317+0000","last_unstale":"2024-09-21T03:34:34.638168+0000","last_undegraded":"2024-09-21T03:34:34.638168+0000","last_fullsized":"2024-09-21T03:34:34.638168+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_clean_scrub_stamp":"2024-09-21T03:34:32.598217+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:12:23.510532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-21T03:34:34.642113+0000","last_change":"2024-09-21T03:34:34.634278+0000","last_active":"2024-09-21T03:34:34.642113+0000","last_peered":"2024-09-21T03:34:34.642113+0000","last_clean":"2024-09-21T03:34:34.642113+0000","last_became_active":"2024-09-21T03:34:34.633950+0000","last_became_peered":"2024-09-21T03:34:34.633950+0000","last_unstale":"2024-09-21T03:34:34.642113+0000","last_undegraded":"2024-09-21T03:34:34.642113+0000","last_fullsized":"2024-09-21T03:34:34.642113+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:43:22.066515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T03:34:33.638123+0000","last_change":"2024-09-21T03:34:33.634743+0000","last_active":"2024-09-21T03:34:33.638123+0000","last_peered":"2024-09-21T03:34:33.638123+0000","last_clean":"2024-09-21T03:34:33.638123+0000","last_became_active":"2024-09-21T03:34:33.634259+0000","last_became_peered":"2024-09-21T03:34:33.634259+0000","last_unstale":"2024-09-21T03:34:33.638123+0000","last_undegraded":"2024-09-21T03:34:33.638123+0000","last_fullsized":"2024-09-21T03:34:33.638123+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_clean_scrub_stamp":"2024-09-21T03:34:32.598217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:14:11.700926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-21T03:34:34.637883+0000","last_change":"2024-09-21T03:34:34.628247+0000","last_active":"2024-09-21T03:34:34.637883+0000","last_peered":"2024-09-21T03:34:34.637883+0000","last_clean":"2024-09-21T03:34:34.637883+0000","last_became_active":"2024-09-21T03:34:34.627925+0000","last_became_peered":"2024-09-21T03:34:34.627925+0000","last_unstale":"2024-09-21T03:34:34.637883+0000","last_undegraded":"2024-09-21T03:34:34.637883+0000","last_fullsized":"2024-09-21T03:34:34.637883+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:26:59.051709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T03:34:33.636065+0000","last_change":"2024-09-21T03:34:33.635141+0000","last_active":"2024-09-21T03:34:33.636065+0000","last_peered":"2024-09-21T03:34:33.636065+0000","last_clean":"2024-09-21T03:34:33.636065+0000","last_became_active":"2024-09-21T03:34:33.634825+0000","last_became_peered":"2024-09-21T03:34:33.634825+0000","last_unstale":"2024-09-21T03:34:33.636065+0000","last_undegraded":"2024-09-21T03:34:33.636065+0000","last_fullsized":"2024-09-21T03:34:33.636065+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_clean_scrub_stamp":"2024-09-21T03:34:32.598217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:12:23.510532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-21T03:34:34.631249+0000","last_change":"2024-09-21T03:34:34.630174+0000","last_active":"2024-09-21T03:34:34.631249+0000","last_peered":"2024-09-21T03:34:34.631249+0000","last_clean":"2024-09-21T03:34:34.631249+0000","last_became_active":"2024-09-21T03:34:34.629879+0000","last_became_peered":"2024-09-21T03:34:34.629879+0000","last_unstale":"2024-09-21T03:34:34.631249+0000","last_undegraded":"2024-09-21T03:34:34.631249+0000","last_fullsized":"2024-09-21T03:34:34.631249+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:03:28.897356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-21T03:34:34.624860+0000","last_change":"2024-09-21T03:34:33.630648+0000","last_active":"2024-09-21T03:34:34.624860+0000","last_peered":"2024-09-21T03:34:34.624860+0000","last_clean":"2024-09-21T03:34:34.624860+0000","last_became_active":"2024-09-21T03:34:33.630288+0000","last_became_peered":"2024-09-21T03:34:33.630288+0000","last_unstale":"2024-09-21T03:34:34.624860+0000","last_undegraded":"2024-09-21T03:34:34.624860+0000","last_fullsized":"2024-09-21T03:34:34.624860+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_clean_scrub_stamp":"2024-09-21T03:34:32.598217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:03:25.007146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-21T03:34:33.620205+0000","last_change":"2024-09-21T03:34:33.620205+0000","last_active":"2024-09-21T03:34:33.604595+0000","last_peered":"2024-09-21T03:34:33.604595+0000","last_clean":"2024-09-21T03:34:33.604595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:34:33.620205+0000","last_undegraded":"2024-09-21T03:34:33.620205+0000","last_fullsized":"2024-09-21T03:34:33.620205+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"39'2","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-21T03:34:34.634012+0000","last_change":"2024-09-21T03:34:33.623824+0000","last_active":"2024-09-21T03:34:34.634012+0000","last_peered":"2024-09-21T03:34:34.634012+0000","last_clean":"2024-09-21T03:34:34.634012+0000","last_became_active":"2024-09-21T03:34:33.623465+0000","last_became_peered":"2024-09-21T03:34:33.623465+0000","last_unstale":"2024-09-21T03:34:34.634012+0000","last_undegraded":"2024-09-21T03:34:34.634012+0000","last_fullsized":"2024-09-21T03:34:34.634012+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_clean_scrub_stamp":"2024-09-21T03:34:32.598217+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:07:51.182124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:35.627438+0000","last_change":"2024-09-21T03:34:34.635616+0000","last_active":"2024-09-21T03:34:35.627438+0000","last_peered":"2024-09-21T03:34:35.627438+0000","last_clean":"2024-09-21T03:34:35.627438+0000","last_became_active":"2024-09-21T03:34:34.632739+0000","last_became_peered":"2024-09-21T03:34:34.632739+0000","last_unstale":"2024-09-21T03:34:35.627438+0000","last_undegraded":"2024-09-21T03:34:35.627438+0000","last_fullsized":"2024-09-21T03:34:35.627438+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:49:01.779676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:36.646422+0000","last_change":"2024-09-21T03:34:33.633593+0000","last_active":"2024-09-21T03:34:36.646422+0000","last_peered":"2024-09-21T03:34:36.646422+0000","last_clean":"2024-09-21T03:34:36.646422+0000","last_became_active":"2024-09-21T03:34:33.633254+0000","last_became_peered":"2024-09-21T03:34:33.633254+0000","last_unstale":"2024-09-21T03:34:36.646422+0000","last_undegraded":"2024-09-21T03:34:36.646422+0000","last_fullsized":"2024-09-21T03:34:36.646422+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_clean_scrub_stamp":"2024-09-21T03:34:32.598217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:57:01.944255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-21T03:34:33.621061+0000","last_change":"2024-09-21T03:34:33.621061+0000","last_active":"2024-09-21T03:34:33.604595+0000","last_peered":"2024-09-21T03:34:33.604595+0000","last_clean":"2024-09-21T03:34:33.604595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:34:33.621061+0000","last_undegraded":"2024-09-21T03:34:33.621061+0000","last_fullsized":"2024-09-21T03:34:33.621061+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-21T03:34:34.616134+0000","last_change":"2024-09-21T03:34:33.637630+0000","last_active":"2024-09-21T03:34:34.616134+0000","last_peered":"2024-09-21T03:34:34.616134+0000","last_clean":"2024-09-21T03:34:34.616134+0000","last_became_active":"2024-09-21T03:34:33.637290+0000","last_became_peered":"2024-09-21T03:34:33.637290+0000","last_unstale":"2024-09-21T03:34:34.616134+0000","last_undegraded":"2024-09-21T03:34:34.616134+0000","last_fullsized":"2024-09-21T03:34:34.616134+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_clean_scrub_stamp":"2024-09-21T03:34:32.598217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:00:46.535619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:35.627723+0000","last_change":"2024-09-21T03:34:34.644149+0000","last_active":"2024-09-21T03:34:35.627723+0000","last_peered":"2024-09-21T03:34:35.627723+0000","last_clean":"2024-09-21T03:34:35.627723+0000","last_became_active":"2024-09-21T03:34:34.643689+0000","last_became_peered":"2024-09-21T03:34:34.643689+0000","last_unstale":"2024-09-21T03:34:35.627723+0000","last_undegraded":"2024-09-21T03:34:35.627723+0000","last_fullsized":"2024-09-21T03:34:35.627723+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:51:48.433744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:36.646016+0000","last_change":"2024-09-21T03:34:33.626688+0000","last_active":"2024-09-21T03:34:36.646016+0000","last_peered":"2024-09-21T03:34:36.646016+0000","last_clean":"2024-09-21T03:34:36.646016+0000","last_became_active":"2024-09-21T03:34:33.626325+0000","last_became_peered":"2024-09-21T03:34:33.626325+0000","last_unstale":"2024-09-21T03:34:36.646016+0000","last_undegraded":"2024-09-21T03:34:36.646016+0000","last_fullsized":"2024-09-21T03:34:36.646016+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_clean_scrub_stamp":"2024-09-21T03:34:32.598217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:50:07.223556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-21T03:34:33.613904+0000","last_change":"2024-09-21T03:34:33.613904+0000","last_active":"2024-09-21T03:34:33.604595+0000","last_peered":"2024-09-21T03:34:33.604595+0000","last_clean":"2024-09-21T03:34:33.604595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:34:33.613904+0000","last_undegraded":"2024-09-21T03:34:33.613904+0000","last_fullsized":"2024-09-21T03:34:33.613904+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T03:34:33.638065+0000","last_change":"2024-09-21T03:34:33.633316+0000","last_active":"2024-09-21T03:34:33.638065+0000","last_peered":"2024-09-21T03:34:33.638065+0000","last_clean":"2024-09-21T03:34:33.638065+0000","last_became_active":"2024-09-21T03:34:33.632983+0000","last_became_peered":"2024-09-21T03:34:33.632983+0000","last_unstale":"2024-09-21T03:34:33.638065+0000","last_undegraded":"2024-09-21T03:34:33.638065+0000","last_fullsized":"2024-09-21T03:34:33.638065+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_clean_scrub_stamp":"2024-09-21T03:34:32.598217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:48:01.802931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-21T03:34:33.621976+0000","last_change":"2024-09-21T03:34:33.621976+0000","last_active":"2024-09-21T03:34:33.604595+0000","last_peered":"2024-09-21T03:34:33.604595+0000","last_clean":"2024-09-21T03:34:33.604595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:34:33.621976+0000","last_undegraded":"2024-09-21T03:34:33.621976+0000","last_fullsized":"2024-09-21T03:34:33.621976+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T03:34:33.632105+0000","last_change":"2024-09-21T03:34:33.630944+0000","last_active":"2024-09-21T03:34:33.632105+0000","last_peered":"2024-09-21T03:34:33.632105+0000","last_clean":"2024-09-21T03:34:33.632105+0000","last_became_active":"2024-09-21T03:34:33.630396+0000","last_became_peered":"2024-09-21T03:34:33.630396+0000","last_unstale":"2024-09-21T03:34:33.632105+0000","last_undegraded":"2024-09-21T03:34:33.632105+0000","last_fullsized":"2024-09-21T03:34:33.632105+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_clean_scrub_stamp":"2024-09-21T03:34:32.598217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:37:36.532174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-21T03:34:34.639837+0000","last_change":"2024-09-21T03:34:34.638755+0000","last_active":"2024-09-21T03:34:34.639837+0000","last_peered":"2024-09-21T03:34:34.639837+0000","last_clean":"2024-09-21T03:34:34.639837+0000","last_became_active":"2024-09-21T03:34:34.638432+0000","last_became_peered":"2024-09-21T03:34:34.638432+0000","last_unstale":"2024-09-21T03:34:34.639837+0000","last_undegraded":"2024-09-21T03:34:34.639837+0000","last_fullsized":"2024-09-21T03:34:34.639837+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:40:56.624795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T03:34:33.627809+0000","last_change":"2024-09-21T03:34:33.626093+0000","last_active":"2024-09-21T03:34:33.627809+0000","last_peered":"2024-09-21T03:34:33.627809+0000","last_clean":"2024-09-21T03:34:33.627809+0000","last_became_active":"2024-09-21T03:34:33.625771+0000","last_became_peered":"2024-09-21T03:34:33.625771+0000","last_unstale":"2024-09-21T03:34:33.627809+0000","last_undegraded":"2024-09-21T03:34:33.627809+0000","last_fullsized":"2024-09-21T03:34:33.627809+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_clean_scrub_stamp":"2024-09-21T03:34:32.598217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:05:08.028564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-21T03:34:33.613418+0000","last_change":"2024-09-21T03:34:33.613418+0000","last_active":"2024-09-21T03:34:33.604595+0000","last_peered":"2024-09-21T03:34:33.604595+0000","last_clean":"2024-09-21T03:34:33.604595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:34:33.613418+0000","last_undegraded":"2024-09-21T03:34:33.613418+0000","last_fullsized":"2024-09-21T03:34:33.613418+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:35.627816+0000","last_change":"2024-09-21T03:34:33.625164+0000","last_active":"2024-09-21T03:34:35.627816+0000","last_peered":"2024-09-21T03:34:35.627816+0000","last_clean":"2024-09-21T03:34:35.627816+0000","last_became_active":"2024-09-21T03:34:33.624299+0000","last_became_peered":"2024-09-21T03:34:33.624299+0000","last_unstale":"2024-09-21T03:34:35.627816+0000","last_undegraded":"2024-09-21T03:34:35.627816+0000","last_fullsized":"2024-09-21T03:34:35.627816+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_clean_scrub_stamp":"2024-09-21T03:34:32.598217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:32:32.643952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:36.646586+0000","last_change":"2024-09-21T03:34:34.637737+0000","last_active":"2024-09-21T03:34:36.646586+0000","last_peered":"2024-09-21T03:34:36.646586+0000","last_clean":"2024-09-21T03:34:36.646586+0000","last_became_active":"2024-09-21T03:34:34.637441+0000","last_became_peered":"2024-09-21T03:34:34.637441+0000","last_unstale":"2024-09-21T03:34:36.646586+0000","last_undegraded":"2024-09-21T03:34:36.646586+0000","last_fullsized":"2024-09-21T03:34:36.646586+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:12:23.136125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T03:34:33.636106+0000","last_change":"2024-09-21T03:34:33.631273+0000","last_active":"2024-09-21T03:34:33.636106+0000","last_peered":"2024-09-21T03:34:33.636106+0000","last_clean":"2024-09-21T03:34:33.636106+0000","last_became_active":"2024-09-21T03:34:33.628561+0000","last_became_peered":"2024-09-21T03:34:33.628561+0000","last_unstale":"2024-09-21T03:34:33.636106+0000","last_undegraded":"2024-09-21T03:34:33.636106+0000","last_fullsized":"2024-09-21T03:34:33.636106+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_clean_scrub_stamp":"2024-09-21T03:34:32.598217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:49:16.438096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-21T03:34:33.620026+0000","last_change":"2024-09-21T03:34:33.620026+0000","last_active":"2024-09-21T03:34:33.604595+0000","last_peered":"2024-09-21T03:34:33.604595+0000","last_clean":"2024-09-21T03:34:33.604595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:34:33.620026+0000","last_undegraded":"2024-09-21T03:34:33.620026+0000","last_fullsized":"2024-09-21T03:34:33.620026+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T03:34:33.638150+0000","last_change":"2024-09-21T03:34:33.633175+0000","last_active":"2024-09-21T03:34:33.638150+0000","last_peered":"2024-09-21T03:34:33.638150+0000","last_clean":"2024-09-21T03:34:33.638150+0000","last_became_active":"2024-09-21T03:34:33.632872+0000","last_became_peered":"2024-09-21T03:34:33.632872+0000","last_unstale":"2024-09-21T03:34:33.638150+0000","last_undegraded":"2024-09-21T03:34:33.638150+0000","last_fullsized":"2024-09-21T03:34:33.638150+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_clean_scrub_stamp":"2024-09-21T03:34:32.598217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:33:03.801458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-21T03:34:33.619685+0000","last_change":"2024-09-21T03:34:33.619685+0000","last_active":"2024-09-21T03:34:33.604595+0000","last_peered":"2024-09-21T03:34:33.604595+0000","last_clean":"2024-09-21T03:34:33.604595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:34:33.619685+0000","last_undegraded":"2024-09-21T03:34:33.619685+0000","last_fullsized":"2024-09-21T03:34:33.619685+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:35.628165+0000","last_change":"2024-09-21T03:34:33.639171+0000","last_active":"2024-09-21T03:34:35.628165+0000","last_peered":"2024-09-21T03:34:35.628165+0000","last_clean":"2024-09-21T03:34:35.628165+0000","last_became_active":"2024-09-21T03:34:33.638819+0000","last_became_peered":"2024-09-21T03:34:33.638819+0000","last_unstale":"2024-09-21T03:34:35.628165+0000","last_undegraded":"2024-09-21T03:34:35.628165+0000","last_fullsized":"2024-09-21T03:34:35.628165+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_clean_scrub_stamp":"2024-09-21T03:34:32.598217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:44:08.657412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-21T03:34:33.625230+0000","last_change":"2024-09-21T03:34:33.625230+0000","last_active":"2024-09-21T03:34:33.604595+0000","last_peered":"2024-09-21T03:34:33.604595+0000","last_clean":"2024-09-21T03:34:33.604595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:34:33.625230+0000","last_undegraded":"2024-09-21T03:34:33.625230+0000","last_fullsized":"2024-09-21T03:34:33.625230+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T03:34:33.637827+0000","last_change":"2024-09-21T03:34:33.635753+0000","last_active":"2024-09-21T03:34:33.637827+0000","last_peered":"2024-09-21T03:34:33.637827+0000","last_clean":"2024-09-21T03:34:33.637827+0000","last_became_active":"2024-09-21T03:34:33.631914+0000","last_became_peered":"2024-09-21T03:34:33.631914+0000","last_unstale":"2024-09-21T03:34:33.637827+0000","last_undegraded":"2024-09-21T03:34:33.637827+0000","last_fullsized":"2024-09-21T03:34:33.637827+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_clean_scrub_stamp":"2024-09-21T03:34:32.598217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:57:01.944255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":143,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-21T03:34:34.615053+0000","last_change":"2024-09-21T03:33:25.319776+0000","last_active":"2024-09-21T03:34:34.615053+0000","last_peered":"2024-09-21T03:34:34.615053+0000","last_clean":"2024-09-21T03:34:34.615053+0000","last_became_active":"2024-09-21T03:33:25.318399+0000","last_became_peered":"2024-09-21T03:33:25.318399+0000","last_unstale":"2024-09-21T03:34:34.615053+0000","last_undegraded":"2024-09-21T03:34:34.615053+0000","last_fullsized":"2024-09-21T03:34:34.615053+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:23.016672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:23.016672+0000","last_clean_scrub_stamp":"2024-09-21T03:33:23.016672+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:32:35.906114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":1668,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":15,"ondisk_log_size":15,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":1,"up_from":16,"seq":68719476754,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32212,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339628,"statfs":{"total":96636764160,"available":96603779072,"internally_reserved":0,"allocated":1040384,"data_stored":780143,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509458,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31596,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340244,"statfs":{"total":96636764160,"available":96604409856,"internally_reserved":0,"allocated":417792,"data_stored":175919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509457,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31612,"kb_used_data":420,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340228,"statfs":{"total":96636764160,"available":96604393472,"internally_reserved":0,"allocated":430080,"data_stored":182586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509458,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27436,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344404,"statfs":{"total":96636764160,"available":96608669696,"internally_reserved":0,"allocated":417792,"data_stored":175919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509457,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31596,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340244,"statfs":{"total":96636764160,"available":96604409856,"internally_reserved":0,"allocated":417792,"data_stored":175919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542162,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31640,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340200,"statfs":{"total":96636764160,"available":96604364800,"internally_reserved":0,"allocated":458752,"data_stored":196479,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574866,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32208,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339632,"statfs":{"total":96636764160,"available":96603783168,"internally_reserved":0,"allocated":1040384,"data_stored":780703,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542163,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35816,"kb_used_data":464,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336024,"statfs":{"total":96636764160,"available":96600088576,"internally_reserved":0,"allocated":475136,"data_stored":198181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T03:34:39.081 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T03:34:39.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:39.242+0000 7f1d983a8640 1 -- 172.21.15.38:0/729026217 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1d900b2e40 msgr2=0x7f1d900b3250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:39.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:39.242+0000 7f1d983a8640 1 --2- 172.21.15.38:0/729026217 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1d900b2e40 0x7f1d900b3250 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f1d900b7460 tx=0x7f1d8802a4a0 comp rx=0 tx=0).stop 2024-09-21T03:34:39.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:39.242+0000 7f1d983a8640 1 -- 172.21.15.38:0/729026217 shutdown_connections 2024-09-21T03:34:39.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:39.242+0000 7f1d983a8640 1 --2- 172.21.15.38:0/729026217 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1d900b2e40 0x7f1d900b3250 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:39.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:39.242+0000 7f1d983a8640 1 --2- 172.21.15.38:0/729026217 >> v2:172.21.15.195:3300/0 conn(0x7f1d90162280 0x7f1d90162690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:39.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:39.242+0000 7f1d983a8640 1 -- 172.21.15.38:0/729026217 >> 172.21.15.38:0/729026217 conn(0x7f1d900b6b20 msgr2=0x7f1d900d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:39.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:39.242+0000 7f1d983a8640 1 -- 172.21.15.38:0/729026217 shutdown_connections 2024-09-21T03:34:39.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:39.242+0000 7f1d983a8640 1 -- 172.21.15.38:0/729026217 wait complete. 2024-09-21T03:34:39.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:39.244+0000 7f1d983a8640 1 Processor -- start 2024-09-21T03:34:39.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:39.244+0000 7f1d983a8640 1 -- start start 2024-09-21T03:34:39.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:39.245+0000 7f1d983a8640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1d900b2e40 0x7f1d900ceee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:39.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:39.245+0000 7f1d983a8640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1d90162280 0x7f1d900b00c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:39.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:39.245+0000 7f1d983a8640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1d900b1d00 0x7f1d900b2110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:39.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:39.245+0000 7f1d983a8640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f1d9015e4b0 con 0x7f1d90162280 2024-09-21T03:34:39.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:39.245+0000 7f1d983a8640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f1d900b39a0 con 0x7f1d900b1d00 2024-09-21T03:34:39.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:39.245+0000 7f1d983a8640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f1d9015e630 con 0x7f1d900b2e40 2024-09-21T03:34:39.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:39.245+0000 7f1d95943640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1d90162280 0x7f1d900b00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:39.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:39.245+0000 7f1d95943640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1d90162280 0x7f1d900b00c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:33690/0 (socket says 172.21.15.38:33690) 2024-09-21T03:34:39.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:39.245+0000 7f1d95943640 1 -- 172.21.15.38:0/2782438636 learned_addr learned my addr 172.21.15.38:0/2782438636 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:34:39.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:39.246+0000 7f1d96144640 1 --2- 172.21.15.38:0/2782438636 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1d900b2e40 0x7f1d900ceee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:39.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:39.246+0000 7f1d96945640 1 --2- 172.21.15.38:0/2782438636 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1d900b1d00 0x7f1d900b2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:39.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:39.246+0000 7f1d95943640 1 -- 172.21.15.38:0/2782438636 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1d900b2e40 msgr2=0x7f1d900ceee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:39.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:39.246+0000 7f1d95943640 1 --2- 172.21.15.38:0/2782438636 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1d900b2e40 0x7f1d900ceee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:39.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:39.246+0000 7f1d95943640 1 -- 172.21.15.38:0/2782438636 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1d900b1d00 msgr2=0x7f1d900b2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:39.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:39.246+0000 7f1d95943640 1 --2- 172.21.15.38:0/2782438636 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1d900b1d00 0x7f1d900b2110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:39.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:39.246+0000 7f1d95943640 1 -- 172.21.15.38:0/2782438636 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d900b2650 con 0x7f1d90162280 2024-09-21T03:34:39.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:39.246+0000 7f1d95943640 1 --2- 172.21.15.38:0/2782438636 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1d90162280 0x7f1d900b00c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f1d8400d740 tx=0x7f1d8400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:39.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:39.246+0000 7f1d8effd640 1 -- 172.21.15.38:0/2782438636 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d84017070 con 0x7f1d90162280 2024-09-21T03:34:39.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:39.246+0000 7f1d8effd640 1 -- 172.21.15.38:0/2782438636 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1d84009d20 con 0x7f1d90162280 2024-09-21T03:34:39.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:39.246+0000 7f1d8effd640 1 -- 172.21.15.38:0/2782438636 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d840106c0 con 0x7f1d90162280 2024-09-21T03:34:39.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:39.246+0000 7f1d983a8640 1 -- 172.21.15.38:0/2782438636 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d900b0600 con 0x7f1d90162280 2024-09-21T03:34:39.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:39.246+0000 7f1d983a8640 1 -- 172.21.15.38:0/2782438636 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d9014dbb0 con 0x7f1d90162280 2024-09-21T03:34:39.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:39.247+0000 7f1d983a8640 1 -- 172.21.15.38:0/2782438636 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d90162f00 con 0x7f1d90162280 2024-09-21T03:34:39.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:39.248+0000 7f1d8effd640 1 -- 172.21.15.38:0/2782438636 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f1d8401a070 con 0x7f1d90162280 2024-09-21T03:34:39.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:39.248+0000 7f1d8effd640 1 --2- 172.21.15.38:0/2782438636 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1d7c078bc0 0x7f1d7c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:39.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:39.249+0000 7f1d8effd640 1 -- 172.21.15.38:0/2782438636 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7f1d8409af70 con 0x7f1d90162280 2024-09-21T03:34:39.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:39.249+0000 7f1d96144640 1 --2- 172.21.15.38:0/2782438636 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1d7c078bc0 0x7f1d7c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:39.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:39.249+0000 7f1d96144640 1 --2- 172.21.15.38:0/2782438636 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1d7c078bc0 0x7f1d7c07b0d0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f1d800012e0 tx=0x7f1d80009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:39.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:39.253+0000 7f1d8effd640 1 -- 172.21.15.38:0/2782438636 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f1d900b2650 con 0x7f1d90162280 2024-09-21T03:34:39.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:39.436+0000 7f1d983a8640 1 -- 172.21.15.38:0/2782438636 --> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1d900b07d0 con 0x7f1d7c078bc0 2024-09-21T03:34:39.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:39.441+0000 7f1d8effd640 1 -- 172.21.15.38:0/2782438636 <== mgr.4117 v2:172.21.15.195:6800/1631901344 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+249387 (secure 0 0 0) 0x7f1d900b07d0 con 0x7f1d7c078bc0 2024-09-21T03:34:39.445 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:34:39.446 INFO:teuthology.orchestra.run.smithi038.stderr:dumped all 2024-09-21T03:34:39.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:39.444+0000 7f1d983a8640 1 -- 172.21.15.38:0/2782438636 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1d7c078bc0 msgr2=0x7f1d7c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:39.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:39.444+0000 7f1d983a8640 1 --2- 172.21.15.38:0/2782438636 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1d7c078bc0 0x7f1d7c07b0d0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f1d800012e0 tx=0x7f1d80009040 comp rx=0 tx=0).stop 2024-09-21T03:34:39.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:39.444+0000 7f1d983a8640 1 -- 172.21.15.38:0/2782438636 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1d90162280 msgr2=0x7f1d900b00c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:39.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:39.444+0000 7f1d983a8640 1 --2- 172.21.15.38:0/2782438636 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1d90162280 0x7f1d900b00c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f1d8400d740 tx=0x7f1d8400dc00 comp rx=0 tx=0).stop 2024-09-21T03:34:39.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:39.444+0000 7f1d983a8640 1 -- 172.21.15.38:0/2782438636 shutdown_connections 2024-09-21T03:34:39.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:39.444+0000 7f1d983a8640 1 --2- 172.21.15.38:0/2782438636 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1d900b1d00 0x7f1d900b2110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:39.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:39.444+0000 7f1d983a8640 1 --2- 172.21.15.38:0/2782438636 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1d90162280 0x7f1d900b00c0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:39.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:39.444+0000 7f1d983a8640 1 --2- 172.21.15.38:0/2782438636 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1d7c078bc0 0x7f1d7c07b0d0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:39.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:39.444+0000 7f1d983a8640 1 --2- 172.21.15.38:0/2782438636 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1d900b2e40 0x7f1d900ceee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:39.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:39.444+0000 7f1d983a8640 1 -- 172.21.15.38:0/2782438636 >> 172.21.15.38:0/2782438636 conn(0x7f1d900b6b20 msgr2=0x7f1d9006d8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:39.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:39.444+0000 7f1d983a8640 1 -- 172.21.15.38:0/2782438636 shutdown_connections 2024-09-21T03:34:39.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:39.444+0000 7f1d983a8640 1 -- 172.21.15.38:0/2782438636 wait complete. 2024-09-21T03:34:39.457 INFO:teuthology.orchestra.run.smithi038.stdout:{"pg_ready":true,"pg_map":{"version":71,"stamp":"2024-09-21T03:34:38.993186+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592036,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":115,"num_write_kb":1639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":87,"ondisk_log_size":87,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":76,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":254116,"kb_used_data":4588,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754720604,"statfs":{"total":773094113280,"available":772833898496,"internally_reserved":0,"allocated":4698112,"data_stored":2665849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12748,"internal_metadata":255315508},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1668,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001614"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:35.628233+0000","last_change":"2024-09-21T03:34:34.631062+0000","last_active":"2024-09-21T03:34:35.628233+0000","last_peered":"2024-09-21T03:34:35.628233+0000","last_clean":"2024-09-21T03:34:35.628233+0000","last_became_active":"2024-09-21T03:34:34.630468+0000","last_became_peered":"2024-09-21T03:34:34.630468+0000","last_unstale":"2024-09-21T03:34:35.628233+0000","last_undegraded":"2024-09-21T03:34:35.628233+0000","last_fullsized":"2024-09-21T03:34:35.628233+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:24:06.812940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:36.646828+0000","last_change":"2024-09-21T03:34:34.626744+0000","last_active":"2024-09-21T03:34:36.646828+0000","last_peered":"2024-09-21T03:34:36.646828+0000","last_clean":"2024-09-21T03:34:36.646828+0000","last_became_active":"2024-09-21T03:34:34.626445+0000","last_became_peered":"2024-09-21T03:34:34.626445+0000","last_unstale":"2024-09-21T03:34:36.646828+0000","last_undegraded":"2024-09-21T03:34:36.646828+0000","last_fullsized":"2024-09-21T03:34:36.646828+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:28:02.542581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:35.628793+0000","last_change":"2024-09-21T03:34:34.636439+0000","last_active":"2024-09-21T03:34:35.628793+0000","last_peered":"2024-09-21T03:34:35.628793+0000","last_clean":"2024-09-21T03:34:35.628793+0000","last_became_active":"2024-09-21T03:34:34.635938+0000","last_became_peered":"2024-09-21T03:34:34.635938+0000","last_unstale":"2024-09-21T03:34:35.628793+0000","last_undegraded":"2024-09-21T03:34:35.628793+0000","last_fullsized":"2024-09-21T03:34:35.628793+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:13:58.235543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:35.628343+0000","last_change":"2024-09-21T03:34:34.634523+0000","last_active":"2024-09-21T03:34:35.628343+0000","last_peered":"2024-09-21T03:34:35.628343+0000","last_clean":"2024-09-21T03:34:35.628343+0000","last_became_active":"2024-09-21T03:34:34.633777+0000","last_became_peered":"2024-09-21T03:34:34.633777+0000","last_unstale":"2024-09-21T03:34:35.628343+0000","last_undegraded":"2024-09-21T03:34:35.628343+0000","last_fullsized":"2024-09-21T03:34:35.628343+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:18:03.263504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-21T03:34:33.614399+0000","last_change":"2024-09-21T03:34:33.614399+0000","last_active":"2024-09-21T03:34:33.604595+0000","last_peered":"2024-09-21T03:34:33.604595+0000","last_clean":"2024-09-21T03:34:33.604595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:34:33.614399+0000","last_undegraded":"2024-09-21T03:34:33.614399+0000","last_fullsized":"2024-09-21T03:34:33.614399+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:36.646409+0000","last_change":"2024-09-21T03:34:34.634322+0000","last_active":"2024-09-21T03:34:36.646409+0000","last_peered":"2024-09-21T03:34:36.646409+0000","last_clean":"2024-09-21T03:34:36.646409+0000","last_became_active":"2024-09-21T03:34:34.634023+0000","last_became_peered":"2024-09-21T03:34:34.634023+0000","last_unstale":"2024-09-21T03:34:36.646409+0000","last_undegraded":"2024-09-21T03:34:36.646409+0000","last_fullsized":"2024-09-21T03:34:36.646409+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:32:33.650331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-21T03:34:34.638247+0000","last_change":"2024-09-21T03:34:34.629606+0000","last_active":"2024-09-21T03:34:34.638247+0000","last_peered":"2024-09-21T03:34:34.638247+0000","last_clean":"2024-09-21T03:34:34.638247+0000","last_became_active":"2024-09-21T03:34:34.628992+0000","last_became_peered":"2024-09-21T03:34:34.628992+0000","last_unstale":"2024-09-21T03:34:34.638247+0000","last_undegraded":"2024-09-21T03:34:34.638247+0000","last_fullsized":"2024-09-21T03:34:34.638247+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:34:28.501979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:35.627528+0000","last_change":"2024-09-21T03:34:34.637695+0000","last_active":"2024-09-21T03:34:35.627528+0000","last_peered":"2024-09-21T03:34:35.627528+0000","last_clean":"2024-09-21T03:34:35.627528+0000","last_became_active":"2024-09-21T03:34:34.637320+0000","last_became_peered":"2024-09-21T03:34:34.637320+0000","last_unstale":"2024-09-21T03:34:35.627528+0000","last_undegraded":"2024-09-21T03:34:35.627528+0000","last_fullsized":"2024-09-21T03:34:35.627528+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:20:18.794303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-21T03:34:33.614285+0000","last_change":"2024-09-21T03:34:33.614285+0000","last_active":"2024-09-21T03:34:33.604595+0000","last_peered":"2024-09-21T03:34:33.604595+0000","last_clean":"2024-09-21T03:34:33.604595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:34:33.614285+0000","last_undegraded":"2024-09-21T03:34:33.614285+0000","last_fullsized":"2024-09-21T03:34:33.614285+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-21T03:34:34.642546+0000","last_change":"2024-09-21T03:34:34.637080+0000","last_active":"2024-09-21T03:34:34.642546+0000","last_peered":"2024-09-21T03:34:34.642546+0000","last_clean":"2024-09-21T03:34:34.642546+0000","last_became_active":"2024-09-21T03:34:34.636777+0000","last_became_peered":"2024-09-21T03:34:34.636777+0000","last_unstale":"2024-09-21T03:34:34.642546+0000","last_undegraded":"2024-09-21T03:34:34.642546+0000","last_fullsized":"2024-09-21T03:34:34.642546+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:03:32.166555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-21T03:34:33.622983+0000","last_change":"2024-09-21T03:34:33.622983+0000","last_active":"2024-09-21T03:34:33.604595+0000","last_peered":"2024-09-21T03:34:33.604595+0000","last_clean":"2024-09-21T03:34:33.604595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:34:33.622983+0000","last_undegraded":"2024-09-21T03:34:33.622983+0000","last_fullsized":"2024-09-21T03:34:33.622983+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-21T03:34:33.614478+0000","last_change":"2024-09-21T03:34:33.614478+0000","last_active":"2024-09-21T03:34:33.604595+0000","last_peered":"2024-09-21T03:34:33.604595+0000","last_clean":"2024-09-21T03:34:33.604595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:34:33.614478+0000","last_undegraded":"2024-09-21T03:34:33.614478+0000","last_fullsized":"2024-09-21T03:34:33.614478+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-21T03:34:34.642073+0000","last_change":"2024-09-21T03:34:34.629186+0000","last_active":"2024-09-21T03:34:34.642073+0000","last_peered":"2024-09-21T03:34:34.642073+0000","last_clean":"2024-09-21T03:34:34.642073+0000","last_became_active":"2024-09-21T03:34:34.628801+0000","last_became_peered":"2024-09-21T03:34:34.628801+0000","last_unstale":"2024-09-21T03:34:34.642073+0000","last_undegraded":"2024-09-21T03:34:34.642073+0000","last_fullsized":"2024-09-21T03:34:34.642073+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:34:03.702144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-21T03:34:33.616107+0000","last_change":"2024-09-21T03:34:33.616107+0000","last_active":"2024-09-21T03:34:33.604595+0000","last_peered":"2024-09-21T03:34:33.604595+0000","last_clean":"2024-09-21T03:34:33.604595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:34:33.616107+0000","last_undegraded":"2024-09-21T03:34:33.616107+0000","last_fullsized":"2024-09-21T03:34:33.616107+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-21T03:34:33.625487+0000","last_change":"2024-09-21T03:34:33.625487+0000","last_active":"2024-09-21T03:34:33.604595+0000","last_peered":"2024-09-21T03:34:33.604595+0000","last_clean":"2024-09-21T03:34:33.604595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:34:33.625487+0000","last_undegraded":"2024-09-21T03:34:33.625487+0000","last_fullsized":"2024-09-21T03:34:33.625487+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-21T03:34:34.644136+0000","last_change":"2024-09-21T03:34:34.635135+0000","last_active":"2024-09-21T03:34:34.644136+0000","last_peered":"2024-09-21T03:34:34.644136+0000","last_clean":"2024-09-21T03:34:34.644136+0000","last_became_active":"2024-09-21T03:34:34.634811+0000","last_became_peered":"2024-09-21T03:34:34.634811+0000","last_unstale":"2024-09-21T03:34:34.644136+0000","last_undegraded":"2024-09-21T03:34:34.644136+0000","last_fullsized":"2024-09-21T03:34:34.644136+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:44:10.495963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-21T03:34:33.624716+0000","last_change":"2024-09-21T03:34:33.624716+0000","last_active":"2024-09-21T03:34:33.604595+0000","last_peered":"2024-09-21T03:34:33.604595+0000","last_clean":"2024-09-21T03:34:33.604595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:34:33.624716+0000","last_undegraded":"2024-09-21T03:34:33.624716+0000","last_fullsized":"2024-09-21T03:34:33.624716+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-21T03:34:33.623146+0000","last_change":"2024-09-21T03:34:33.623146+0000","last_active":"2024-09-21T03:34:33.604595+0000","last_peered":"2024-09-21T03:34:33.604595+0000","last_clean":"2024-09-21T03:34:33.604595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:34:33.623146+0000","last_undegraded":"2024-09-21T03:34:33.623146+0000","last_fullsized":"2024-09-21T03:34:33.623146+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-21T03:34:34.639249+0000","last_change":"2024-09-21T03:34:34.637926+0000","last_active":"2024-09-21T03:34:34.639249+0000","last_peered":"2024-09-21T03:34:34.639249+0000","last_clean":"2024-09-21T03:34:34.639249+0000","last_became_active":"2024-09-21T03:34:34.637546+0000","last_became_peered":"2024-09-21T03:34:34.637546+0000","last_unstale":"2024-09-21T03:34:34.639249+0000","last_undegraded":"2024-09-21T03:34:34.639249+0000","last_fullsized":"2024-09-21T03:34:34.639249+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:39:27.102702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-21T03:34:33.620475+0000","last_change":"2024-09-21T03:34:33.620475+0000","last_active":"2024-09-21T03:34:33.604595+0000","last_peered":"2024-09-21T03:34:33.604595+0000","last_clean":"2024-09-21T03:34:33.604595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:34:33.620475+0000","last_undegraded":"2024-09-21T03:34:33.620475+0000","last_fullsized":"2024-09-21T03:34:33.620475+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-21T03:34:33.621111+0000","last_change":"2024-09-21T03:34:33.621111+0000","last_active":"2024-09-21T03:34:33.604595+0000","last_peered":"2024-09-21T03:34:33.604595+0000","last_clean":"2024-09-21T03:34:33.604595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:34:33.621111+0000","last_undegraded":"2024-09-21T03:34:33.621111+0000","last_fullsized":"2024-09-21T03:34:33.621111+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-21T03:34:33.621590+0000","last_change":"2024-09-21T03:34:33.621590+0000","last_active":"2024-09-21T03:34:33.604595+0000","last_peered":"2024-09-21T03:34:33.604595+0000","last_clean":"2024-09-21T03:34:33.604595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:34:33.621590+0000","last_undegraded":"2024-09-21T03:34:33.621590+0000","last_fullsized":"2024-09-21T03:34:33.621590+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-21T03:34:33.624872+0000","last_change":"2024-09-21T03:34:33.624872+0000","last_active":"2024-09-21T03:34:33.604595+0000","last_peered":"2024-09-21T03:34:33.604595+0000","last_clean":"2024-09-21T03:34:33.604595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:34:33.624872+0000","last_undegraded":"2024-09-21T03:34:33.624872+0000","last_fullsized":"2024-09-21T03:34:33.624872+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:35.627534+0000","last_change":"2024-09-21T03:34:34.641334+0000","last_active":"2024-09-21T03:34:35.627534+0000","last_peered":"2024-09-21T03:34:35.627534+0000","last_clean":"2024-09-21T03:34:35.627534+0000","last_became_active":"2024-09-21T03:34:34.640922+0000","last_became_peered":"2024-09-21T03:34:34.640922+0000","last_unstale":"2024-09-21T03:34:35.627534+0000","last_undegraded":"2024-09-21T03:34:35.627534+0000","last_fullsized":"2024-09-21T03:34:35.627534+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:12:37.794061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-21T03:34:33.616131+0000","last_change":"2024-09-21T03:34:33.616131+0000","last_active":"2024-09-21T03:34:33.604595+0000","last_peered":"2024-09-21T03:34:33.604595+0000","last_clean":"2024-09-21T03:34:33.604595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:34:33.616131+0000","last_undegraded":"2024-09-21T03:34:33.616131+0000","last_fullsized":"2024-09-21T03:34:33.616131+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:35.628400+0000","last_change":"2024-09-21T03:34:34.640146+0000","last_active":"2024-09-21T03:34:35.628400+0000","last_peered":"2024-09-21T03:34:35.628400+0000","last_clean":"2024-09-21T03:34:35.628400+0000","last_became_active":"2024-09-21T03:34:34.639791+0000","last_became_peered":"2024-09-21T03:34:34.639791+0000","last_unstale":"2024-09-21T03:34:35.628400+0000","last_undegraded":"2024-09-21T03:34:35.628400+0000","last_fullsized":"2024-09-21T03:34:35.628400+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:23:41.788750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-21T03:34:33.622462+0000","last_change":"2024-09-21T03:34:33.622462+0000","last_active":"2024-09-21T03:34:33.604595+0000","last_peered":"2024-09-21T03:34:33.604595+0000","last_clean":"2024-09-21T03:34:33.604595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:34:33.622462+0000","last_undegraded":"2024-09-21T03:34:33.622462+0000","last_fullsized":"2024-09-21T03:34:33.622462+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-21T03:34:34.639470+0000","last_change":"2024-09-21T03:34:34.633013+0000","last_active":"2024-09-21T03:34:34.639470+0000","last_peered":"2024-09-21T03:34:34.639470+0000","last_clean":"2024-09-21T03:34:34.639470+0000","last_became_active":"2024-09-21T03:34:34.631701+0000","last_became_peered":"2024-09-21T03:34:34.631701+0000","last_unstale":"2024-09-21T03:34:34.639470+0000","last_undegraded":"2024-09-21T03:34:34.639470+0000","last_fullsized":"2024-09-21T03:34:34.639470+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:58:27.086990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-21T03:34:33.628158+0000","last_change":"2024-09-21T03:34:33.628158+0000","last_active":"2024-09-21T03:34:33.604595+0000","last_peered":"2024-09-21T03:34:33.604595+0000","last_clean":"2024-09-21T03:34:33.604595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:34:33.628158+0000","last_undegraded":"2024-09-21T03:34:33.628158+0000","last_fullsized":"2024-09-21T03:34:33.628158+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-21T03:34:33.621235+0000","last_change":"2024-09-21T03:34:33.621235+0000","last_active":"2024-09-21T03:34:33.604595+0000","last_peered":"2024-09-21T03:34:33.604595+0000","last_clean":"2024-09-21T03:34:33.604595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:34:33.621235+0000","last_undegraded":"2024-09-21T03:34:33.621235+0000","last_fullsized":"2024-09-21T03:34:33.621235+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-21T03:34:34.639102+0000","last_change":"2024-09-21T03:34:34.629415+0000","last_active":"2024-09-21T03:34:34.639102+0000","last_peered":"2024-09-21T03:34:34.639102+0000","last_clean":"2024-09-21T03:34:34.639102+0000","last_became_active":"2024-09-21T03:34:34.628524+0000","last_became_peered":"2024-09-21T03:34:34.628524+0000","last_unstale":"2024-09-21T03:34:34.639102+0000","last_undegraded":"2024-09-21T03:34:34.639102+0000","last_fullsized":"2024-09-21T03:34:34.639102+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:48:30.216711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:35.627792+0000","last_change":"2024-09-21T03:34:34.639100+0000","last_active":"2024-09-21T03:34:35.627792+0000","last_peered":"2024-09-21T03:34:35.627792+0000","last_clean":"2024-09-21T03:34:35.627792+0000","last_became_active":"2024-09-21T03:34:34.636285+0000","last_became_peered":"2024-09-21T03:34:34.636285+0000","last_unstale":"2024-09-21T03:34:35.627792+0000","last_undegraded":"2024-09-21T03:34:35.627792+0000","last_fullsized":"2024-09-21T03:34:35.627792+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:19:39.778069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-21T03:34:33.621969+0000","last_change":"2024-09-21T03:34:33.621969+0000","last_active":"2024-09-21T03:34:33.604595+0000","last_peered":"2024-09-21T03:34:33.604595+0000","last_clean":"2024-09-21T03:34:33.604595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:34:33.621969+0000","last_undegraded":"2024-09-21T03:34:33.621969+0000","last_fullsized":"2024-09-21T03:34:33.621969+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T03:34:33.620882+0000","last_change":"2024-09-21T03:34:33.618066+0000","last_active":"2024-09-21T03:34:33.620882+0000","last_peered":"2024-09-21T03:34:33.620882+0000","last_clean":"2024-09-21T03:34:33.620882+0000","last_became_active":"2024-09-21T03:34:33.617685+0000","last_became_peered":"2024-09-21T03:34:33.617685+0000","last_unstale":"2024-09-21T03:34:33.620882+0000","last_undegraded":"2024-09-21T03:34:33.620882+0000","last_fullsized":"2024-09-21T03:34:33.620882+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_clean_scrub_stamp":"2024-09-21T03:34:32.598217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:34:47.997359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:36.646764+0000","last_change":"2024-09-21T03:34:34.635480+0000","last_active":"2024-09-21T03:34:36.646764+0000","last_peered":"2024-09-21T03:34:36.646764+0000","last_clean":"2024-09-21T03:34:36.646764+0000","last_became_active":"2024-09-21T03:34:34.634979+0000","last_became_peered":"2024-09-21T03:34:34.634979+0000","last_unstale":"2024-09-21T03:34:36.646764+0000","last_undegraded":"2024-09-21T03:34:36.646764+0000","last_fullsized":"2024-09-21T03:34:36.646764+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:25:43.132670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"39'1","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:36.646633+0000","last_change":"2024-09-21T03:34:33.636543+0000","last_active":"2024-09-21T03:34:36.646633+0000","last_peered":"2024-09-21T03:34:36.646633+0000","last_clean":"2024-09-21T03:34:36.646633+0000","last_became_active":"2024-09-21T03:34:33.636151+0000","last_became_peered":"2024-09-21T03:34:33.636151+0000","last_unstale":"2024-09-21T03:34:36.646633+0000","last_undegraded":"2024-09-21T03:34:36.646633+0000","last_fullsized":"2024-09-21T03:34:36.646633+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_clean_scrub_stamp":"2024-09-21T03:34:32.598217+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:17:30.670560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-21T03:34:33.622277+0000","last_change":"2024-09-21T03:34:33.622277+0000","last_active":"2024-09-21T03:34:33.604595+0000","last_peered":"2024-09-21T03:34:33.604595+0000","last_clean":"2024-09-21T03:34:33.604595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:34:33.622277+0000","last_undegraded":"2024-09-21T03:34:33.622277+0000","last_fullsized":"2024-09-21T03:34:33.622277+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"39'2","reported_seq":17,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:35.628849+0000","last_change":"2024-09-21T03:34:33.629398+0000","last_active":"2024-09-21T03:34:35.628849+0000","last_peered":"2024-09-21T03:34:35.628849+0000","last_clean":"2024-09-21T03:34:35.628849+0000","last_became_active":"2024-09-21T03:34:33.629051+0000","last_became_peered":"2024-09-21T03:34:33.629051+0000","last_unstale":"2024-09-21T03:34:35.628849+0000","last_undegraded":"2024-09-21T03:34:35.628849+0000","last_fullsized":"2024-09-21T03:34:35.628849+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_clean_scrub_stamp":"2024-09-21T03:34:32.598217+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:46:38.739080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:36.646761+0000","last_change":"2024-09-21T03:34:34.629283+0000","last_active":"2024-09-21T03:34:36.646761+0000","last_peered":"2024-09-21T03:34:36.646761+0000","last_clean":"2024-09-21T03:34:36.646761+0000","last_became_active":"2024-09-21T03:34:34.628942+0000","last_became_peered":"2024-09-21T03:34:34.628942+0000","last_unstale":"2024-09-21T03:34:36.646761+0000","last_undegraded":"2024-09-21T03:34:36.646761+0000","last_fullsized":"2024-09-21T03:34:36.646761+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:15:48.821673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T03:34:33.638430+0000","last_change":"2024-09-21T03:34:33.631010+0000","last_active":"2024-09-21T03:34:33.638430+0000","last_peered":"2024-09-21T03:34:33.638430+0000","last_clean":"2024-09-21T03:34:33.638430+0000","last_became_active":"2024-09-21T03:34:33.630576+0000","last_became_peered":"2024-09-21T03:34:33.630576+0000","last_unstale":"2024-09-21T03:34:33.638430+0000","last_undegraded":"2024-09-21T03:34:33.638430+0000","last_fullsized":"2024-09-21T03:34:33.638430+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_clean_scrub_stamp":"2024-09-21T03:34:32.598217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:50:07.223556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:35.627813+0000","last_change":"2024-09-21T03:34:34.629518+0000","last_active":"2024-09-21T03:34:35.627813+0000","last_peered":"2024-09-21T03:34:35.627813+0000","last_clean":"2024-09-21T03:34:35.627813+0000","last_became_active":"2024-09-21T03:34:34.629216+0000","last_became_peered":"2024-09-21T03:34:34.629216+0000","last_unstale":"2024-09-21T03:34:35.627813+0000","last_undegraded":"2024-09-21T03:34:35.627813+0000","last_fullsized":"2024-09-21T03:34:35.627813+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:31:59.133888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T03:34:33.635417+0000","last_change":"2024-09-21T03:34:33.619056+0000","last_active":"2024-09-21T03:34:33.635417+0000","last_peered":"2024-09-21T03:34:33.635417+0000","last_clean":"2024-09-21T03:34:33.635417+0000","last_became_active":"2024-09-21T03:34:33.618698+0000","last_became_peered":"2024-09-21T03:34:33.618698+0000","last_unstale":"2024-09-21T03:34:33.635417+0000","last_undegraded":"2024-09-21T03:34:33.635417+0000","last_fullsized":"2024-09-21T03:34:33.635417+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_clean_scrub_stamp":"2024-09-21T03:34:32.598217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:58:31.223579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-21T03:34:33.618669+0000","last_change":"2024-09-21T03:34:33.618669+0000","last_active":"2024-09-21T03:34:33.604595+0000","last_peered":"2024-09-21T03:34:33.604595+0000","last_clean":"2024-09-21T03:34:33.604595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:34:33.618669+0000","last_undegraded":"2024-09-21T03:34:33.618669+0000","last_fullsized":"2024-09-21T03:34:33.618669+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1a","version":"39'1","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-21T03:34:34.633517+0000","last_change":"2024-09-21T03:34:33.634501+0000","last_active":"2024-09-21T03:34:34.633517+0000","last_peered":"2024-09-21T03:34:34.633517+0000","last_clean":"2024-09-21T03:34:34.633517+0000","last_became_active":"2024-09-21T03:34:33.633827+0000","last_became_peered":"2024-09-21T03:34:33.633827+0000","last_unstale":"2024-09-21T03:34:34.633517+0000","last_undegraded":"2024-09-21T03:34:34.633517+0000","last_fullsized":"2024-09-21T03:34:34.633517+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_clean_scrub_stamp":"2024-09-21T03:34:32.598217+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:33:52.483902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-21T03:34:33.620634+0000","last_change":"2024-09-21T03:34:33.620634+0000","last_active":"2024-09-21T03:34:33.604595+0000","last_peered":"2024-09-21T03:34:33.604595+0000","last_clean":"2024-09-21T03:34:33.604595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:34:33.620634+0000","last_undegraded":"2024-09-21T03:34:33.620634+0000","last_fullsized":"2024-09-21T03:34:33.620634+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:35.627409+0000","last_change":"2024-09-21T03:34:33.636797+0000","last_active":"2024-09-21T03:34:35.627409+0000","last_peered":"2024-09-21T03:34:35.627409+0000","last_clean":"2024-09-21T03:34:35.627409+0000","last_became_active":"2024-09-21T03:34:33.635927+0000","last_became_peered":"2024-09-21T03:34:33.635927+0000","last_unstale":"2024-09-21T03:34:35.627409+0000","last_undegraded":"2024-09-21T03:34:35.627409+0000","last_fullsized":"2024-09-21T03:34:35.627409+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_clean_scrub_stamp":"2024-09-21T03:34:32.598217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:23:52.412874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:36.646794+0000","last_change":"2024-09-21T03:34:34.640266+0000","last_active":"2024-09-21T03:34:36.646794+0000","last_peered":"2024-09-21T03:34:36.646794+0000","last_clean":"2024-09-21T03:34:36.646794+0000","last_became_active":"2024-09-21T03:34:34.639922+0000","last_became_peered":"2024-09-21T03:34:34.639922+0000","last_unstale":"2024-09-21T03:34:36.646794+0000","last_undegraded":"2024-09-21T03:34:36.646794+0000","last_fullsized":"2024-09-21T03:34:36.646794+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:38:18.167710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-21T03:34:34.615067+0000","last_change":"2024-09-21T03:34:33.638401+0000","last_active":"2024-09-21T03:34:34.615067+0000","last_peered":"2024-09-21T03:34:34.615067+0000","last_clean":"2024-09-21T03:34:34.615067+0000","last_became_active":"2024-09-21T03:34:33.637364+0000","last_became_peered":"2024-09-21T03:34:33.637364+0000","last_unstale":"2024-09-21T03:34:34.615067+0000","last_undegraded":"2024-09-21T03:34:34.615067+0000","last_fullsized":"2024-09-21T03:34:34.615067+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_clean_scrub_stamp":"2024-09-21T03:34:32.598217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:37:36.532174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:35.628541+0000","last_change":"2024-09-21T03:34:34.633927+0000","last_active":"2024-09-21T03:34:35.628541+0000","last_peered":"2024-09-21T03:34:35.628541+0000","last_clean":"2024-09-21T03:34:35.628541+0000","last_became_active":"2024-09-21T03:34:34.633294+0000","last_became_peered":"2024-09-21T03:34:34.633294+0000","last_unstale":"2024-09-21T03:34:35.628541+0000","last_undegraded":"2024-09-21T03:34:35.628541+0000","last_fullsized":"2024-09-21T03:34:35.628541+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:41:47.832177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:35.628042+0000","last_change":"2024-09-21T03:34:33.626093+0000","last_active":"2024-09-21T03:34:35.628042+0000","last_peered":"2024-09-21T03:34:35.628042+0000","last_clean":"2024-09-21T03:34:35.628042+0000","last_became_active":"2024-09-21T03:34:33.625795+0000","last_became_peered":"2024-09-21T03:34:33.625795+0000","last_unstale":"2024-09-21T03:34:35.628042+0000","last_undegraded":"2024-09-21T03:34:35.628042+0000","last_fullsized":"2024-09-21T03:34:35.628042+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_clean_scrub_stamp":"2024-09-21T03:34:32.598217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:39:10.947435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-21T03:34:33.616082+0000","last_change":"2024-09-21T03:34:33.616082+0000","last_active":"2024-09-21T03:34:33.604595+0000","last_peered":"2024-09-21T03:34:33.604595+0000","last_clean":"2024-09-21T03:34:33.604595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:34:33.616082+0000","last_undegraded":"2024-09-21T03:34:33.616082+0000","last_fullsized":"2024-09-21T03:34:33.616082+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T03:34:33.638093+0000","last_change":"2024-09-21T03:34:33.633888+0000","last_active":"2024-09-21T03:34:33.638093+0000","last_peered":"2024-09-21T03:34:33.638093+0000","last_clean":"2024-09-21T03:34:33.638093+0000","last_became_active":"2024-09-21T03:34:33.633372+0000","last_became_peered":"2024-09-21T03:34:33.633372+0000","last_unstale":"2024-09-21T03:34:33.638093+0000","last_undegraded":"2024-09-21T03:34:33.638093+0000","last_fullsized":"2024-09-21T03:34:33.638093+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_clean_scrub_stamp":"2024-09-21T03:34:32.598217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:58:21.314833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:35.628444+0000","last_change":"2024-09-21T03:34:34.644095+0000","last_active":"2024-09-21T03:34:35.628444+0000","last_peered":"2024-09-21T03:34:35.628444+0000","last_clean":"2024-09-21T03:34:35.628444+0000","last_became_active":"2024-09-21T03:34:34.643713+0000","last_became_peered":"2024-09-21T03:34:34.643713+0000","last_unstale":"2024-09-21T03:34:35.628444+0000","last_undegraded":"2024-09-21T03:34:35.628444+0000","last_fullsized":"2024-09-21T03:34:35.628444+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:40:54.924468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T03:34:33.638035+0000","last_change":"2024-09-21T03:34:33.632451+0000","last_active":"2024-09-21T03:34:33.638035+0000","last_peered":"2024-09-21T03:34:33.638035+0000","last_clean":"2024-09-21T03:34:33.638035+0000","last_became_active":"2024-09-21T03:34:33.632103+0000","last_became_peered":"2024-09-21T03:34:33.632103+0000","last_unstale":"2024-09-21T03:34:33.638035+0000","last_undegraded":"2024-09-21T03:34:33.638035+0000","last_fullsized":"2024-09-21T03:34:33.638035+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_clean_scrub_stamp":"2024-09-21T03:34:32.598217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:22:08.770678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-21T03:34:34.639059+0000","last_change":"2024-09-21T03:34:34.628419+0000","last_active":"2024-09-21T03:34:34.639059+0000","last_peered":"2024-09-21T03:34:34.639059+0000","last_clean":"2024-09-21T03:34:34.639059+0000","last_became_active":"2024-09-21T03:34:34.628087+0000","last_became_peered":"2024-09-21T03:34:34.628087+0000","last_unstale":"2024-09-21T03:34:34.639059+0000","last_undegraded":"2024-09-21T03:34:34.639059+0000","last_fullsized":"2024-09-21T03:34:34.639059+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:28:03.240497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.14","version":"39'6","reported_seq":21,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:35.628508+0000","last_change":"2024-09-21T03:34:33.639043+0000","last_active":"2024-09-21T03:34:35.628508+0000","last_peered":"2024-09-21T03:34:35.628508+0000","last_clean":"2024-09-21T03:34:35.628508+0000","last_became_active":"2024-09-21T03:34:33.638761+0000","last_became_peered":"2024-09-21T03:34:33.638761+0000","last_unstale":"2024-09-21T03:34:35.628508+0000","last_undegraded":"2024-09-21T03:34:35.628508+0000","last_fullsized":"2024-09-21T03:34:35.628508+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_clean_scrub_stamp":"2024-09-21T03:34:32.598217+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:44:53.992927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:35.628094+0000","last_change":"2024-09-21T03:34:34.640167+0000","last_active":"2024-09-21T03:34:35.628094+0000","last_peered":"2024-09-21T03:34:35.628094+0000","last_clean":"2024-09-21T03:34:35.628094+0000","last_became_active":"2024-09-21T03:34:34.639859+0000","last_became_peered":"2024-09-21T03:34:34.639859+0000","last_unstale":"2024-09-21T03:34:35.628094+0000","last_undegraded":"2024-09-21T03:34:35.628094+0000","last_fullsized":"2024-09-21T03:34:35.628094+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:36:48.213957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"39'2","reported_seq":17,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:35.627656+0000","last_change":"2024-09-21T03:34:33.628173+0000","last_active":"2024-09-21T03:34:35.627656+0000","last_peered":"2024-09-21T03:34:35.627656+0000","last_clean":"2024-09-21T03:34:35.627656+0000","last_became_active":"2024-09-21T03:34:33.627559+0000","last_became_peered":"2024-09-21T03:34:33.627559+0000","last_unstale":"2024-09-21T03:34:35.627656+0000","last_undegraded":"2024-09-21T03:34:35.627656+0000","last_fullsized":"2024-09-21T03:34:35.627656+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_clean_scrub_stamp":"2024-09-21T03:34:32.598217+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:34:22.474063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:36.646395+0000","last_change":"2024-09-21T03:34:34.626145+0000","last_active":"2024-09-21T03:34:36.646395+0000","last_peered":"2024-09-21T03:34:36.646395+0000","last_clean":"2024-09-21T03:34:36.646395+0000","last_became_active":"2024-09-21T03:34:34.625819+0000","last_became_peered":"2024-09-21T03:34:34.625819+0000","last_unstale":"2024-09-21T03:34:36.646395+0000","last_undegraded":"2024-09-21T03:34:36.646395+0000","last_fullsized":"2024-09-21T03:34:36.646395+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:44:09.663791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:36.646276+0000","last_change":"2024-09-21T03:34:33.630029+0000","last_active":"2024-09-21T03:34:36.646276+0000","last_peered":"2024-09-21T03:34:36.646276+0000","last_clean":"2024-09-21T03:34:36.646276+0000","last_became_active":"2024-09-21T03:34:33.629699+0000","last_became_peered":"2024-09-21T03:34:33.629699+0000","last_unstale":"2024-09-21T03:34:36.646276+0000","last_undegraded":"2024-09-21T03:34:36.646276+0000","last_fullsized":"2024-09-21T03:34:36.646276+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_clean_scrub_stamp":"2024-09-21T03:34:32.598217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:49:16.438096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-21T03:34:33.619516+0000","last_change":"2024-09-21T03:34:33.619516+0000","last_active":"2024-09-21T03:34:33.604595+0000","last_peered":"2024-09-21T03:34:33.604595+0000","last_clean":"2024-09-21T03:34:33.604595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:34:33.619516+0000","last_undegraded":"2024-09-21T03:34:33.619516+0000","last_fullsized":"2024-09-21T03:34:33.619516+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T03:34:33.644077+0000","last_change":"2024-09-21T03:34:33.639290+0000","last_active":"2024-09-21T03:34:33.644077+0000","last_peered":"2024-09-21T03:34:33.644077+0000","last_clean":"2024-09-21T03:34:33.644077+0000","last_became_active":"2024-09-21T03:34:33.638941+0000","last_became_peered":"2024-09-21T03:34:33.638941+0000","last_unstale":"2024-09-21T03:34:33.644077+0000","last_undegraded":"2024-09-21T03:34:33.644077+0000","last_fullsized":"2024-09-21T03:34:33.644077+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_clean_scrub_stamp":"2024-09-21T03:34:32.598217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:46:42.303637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-21T03:34:33.613436+0000","last_change":"2024-09-21T03:34:33.613436+0000","last_active":"2024-09-21T03:34:33.604595+0000","last_peered":"2024-09-21T03:34:33.604595+0000","last_clean":"2024-09-21T03:34:33.604595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:34:33.613436+0000","last_undegraded":"2024-09-21T03:34:33.613436+0000","last_fullsized":"2024-09-21T03:34:33.613436+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T03:34:33.638439+0000","last_change":"2024-09-21T03:34:33.632101+0000","last_active":"2024-09-21T03:34:33.638439+0000","last_peered":"2024-09-21T03:34:33.638439+0000","last_clean":"2024-09-21T03:34:33.638439+0000","last_became_active":"2024-09-21T03:34:33.631744+0000","last_became_peered":"2024-09-21T03:34:33.631744+0000","last_unstale":"2024-09-21T03:34:33.638439+0000","last_undegraded":"2024-09-21T03:34:33.638439+0000","last_fullsized":"2024-09-21T03:34:33.638439+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_clean_scrub_stamp":"2024-09-21T03:34:32.598217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:17:30.670560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:36.646635+0000","last_change":"2024-09-21T03:34:34.626189+0000","last_active":"2024-09-21T03:34:36.646635+0000","last_peered":"2024-09-21T03:34:36.646635+0000","last_clean":"2024-09-21T03:34:36.646635+0000","last_became_active":"2024-09-21T03:34:34.625865+0000","last_became_peered":"2024-09-21T03:34:34.625865+0000","last_unstale":"2024-09-21T03:34:36.646635+0000","last_undegraded":"2024-09-21T03:34:36.646635+0000","last_fullsized":"2024-09-21T03:34:36.646635+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:13:44.244299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.f","version":"39'1","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-21T03:34:34.638168+0000","last_change":"2024-09-21T03:34:33.637659+0000","last_active":"2024-09-21T03:34:34.638168+0000","last_peered":"2024-09-21T03:34:34.638168+0000","last_clean":"2024-09-21T03:34:34.638168+0000","last_became_active":"2024-09-21T03:34:33.637317+0000","last_became_peered":"2024-09-21T03:34:33.637317+0000","last_unstale":"2024-09-21T03:34:34.638168+0000","last_undegraded":"2024-09-21T03:34:34.638168+0000","last_fullsized":"2024-09-21T03:34:34.638168+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_clean_scrub_stamp":"2024-09-21T03:34:32.598217+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:12:23.510532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-21T03:34:34.642113+0000","last_change":"2024-09-21T03:34:34.634278+0000","last_active":"2024-09-21T03:34:34.642113+0000","last_peered":"2024-09-21T03:34:34.642113+0000","last_clean":"2024-09-21T03:34:34.642113+0000","last_became_active":"2024-09-21T03:34:34.633950+0000","last_became_peered":"2024-09-21T03:34:34.633950+0000","last_unstale":"2024-09-21T03:34:34.642113+0000","last_undegraded":"2024-09-21T03:34:34.642113+0000","last_fullsized":"2024-09-21T03:34:34.642113+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:43:22.066515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T03:34:33.638123+0000","last_change":"2024-09-21T03:34:33.634743+0000","last_active":"2024-09-21T03:34:33.638123+0000","last_peered":"2024-09-21T03:34:33.638123+0000","last_clean":"2024-09-21T03:34:33.638123+0000","last_became_active":"2024-09-21T03:34:33.634259+0000","last_became_peered":"2024-09-21T03:34:33.634259+0000","last_unstale":"2024-09-21T03:34:33.638123+0000","last_undegraded":"2024-09-21T03:34:33.638123+0000","last_fullsized":"2024-09-21T03:34:33.638123+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_clean_scrub_stamp":"2024-09-21T03:34:32.598217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:14:11.700926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-21T03:34:34.637883+0000","last_change":"2024-09-21T03:34:34.628247+0000","last_active":"2024-09-21T03:34:34.637883+0000","last_peered":"2024-09-21T03:34:34.637883+0000","last_clean":"2024-09-21T03:34:34.637883+0000","last_became_active":"2024-09-21T03:34:34.627925+0000","last_became_peered":"2024-09-21T03:34:34.627925+0000","last_unstale":"2024-09-21T03:34:34.637883+0000","last_undegraded":"2024-09-21T03:34:34.637883+0000","last_fullsized":"2024-09-21T03:34:34.637883+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:26:59.051709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T03:34:33.636065+0000","last_change":"2024-09-21T03:34:33.635141+0000","last_active":"2024-09-21T03:34:33.636065+0000","last_peered":"2024-09-21T03:34:33.636065+0000","last_clean":"2024-09-21T03:34:33.636065+0000","last_became_active":"2024-09-21T03:34:33.634825+0000","last_became_peered":"2024-09-21T03:34:33.634825+0000","last_unstale":"2024-09-21T03:34:33.636065+0000","last_undegraded":"2024-09-21T03:34:33.636065+0000","last_fullsized":"2024-09-21T03:34:33.636065+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_clean_scrub_stamp":"2024-09-21T03:34:32.598217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:12:23.510532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-21T03:34:34.631249+0000","last_change":"2024-09-21T03:34:34.630174+0000","last_active":"2024-09-21T03:34:34.631249+0000","last_peered":"2024-09-21T03:34:34.631249+0000","last_clean":"2024-09-21T03:34:34.631249+0000","last_became_active":"2024-09-21T03:34:34.629879+0000","last_became_peered":"2024-09-21T03:34:34.629879+0000","last_unstale":"2024-09-21T03:34:34.631249+0000","last_undegraded":"2024-09-21T03:34:34.631249+0000","last_fullsized":"2024-09-21T03:34:34.631249+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:03:28.897356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-21T03:34:34.624860+0000","last_change":"2024-09-21T03:34:33.630648+0000","last_active":"2024-09-21T03:34:34.624860+0000","last_peered":"2024-09-21T03:34:34.624860+0000","last_clean":"2024-09-21T03:34:34.624860+0000","last_became_active":"2024-09-21T03:34:33.630288+0000","last_became_peered":"2024-09-21T03:34:33.630288+0000","last_unstale":"2024-09-21T03:34:34.624860+0000","last_undegraded":"2024-09-21T03:34:34.624860+0000","last_fullsized":"2024-09-21T03:34:34.624860+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_clean_scrub_stamp":"2024-09-21T03:34:32.598217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:03:25.007146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-21T03:34:33.620205+0000","last_change":"2024-09-21T03:34:33.620205+0000","last_active":"2024-09-21T03:34:33.604595+0000","last_peered":"2024-09-21T03:34:33.604595+0000","last_clean":"2024-09-21T03:34:33.604595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:34:33.620205+0000","last_undegraded":"2024-09-21T03:34:33.620205+0000","last_fullsized":"2024-09-21T03:34:33.620205+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"39'2","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-21T03:34:34.634012+0000","last_change":"2024-09-21T03:34:33.623824+0000","last_active":"2024-09-21T03:34:34.634012+0000","last_peered":"2024-09-21T03:34:34.634012+0000","last_clean":"2024-09-21T03:34:34.634012+0000","last_became_active":"2024-09-21T03:34:33.623465+0000","last_became_peered":"2024-09-21T03:34:33.623465+0000","last_unstale":"2024-09-21T03:34:34.634012+0000","last_undegraded":"2024-09-21T03:34:34.634012+0000","last_fullsized":"2024-09-21T03:34:34.634012+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_clean_scrub_stamp":"2024-09-21T03:34:32.598217+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:07:51.182124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:35.627438+0000","last_change":"2024-09-21T03:34:34.635616+0000","last_active":"2024-09-21T03:34:35.627438+0000","last_peered":"2024-09-21T03:34:35.627438+0000","last_clean":"2024-09-21T03:34:35.627438+0000","last_became_active":"2024-09-21T03:34:34.632739+0000","last_became_peered":"2024-09-21T03:34:34.632739+0000","last_unstale":"2024-09-21T03:34:35.627438+0000","last_undegraded":"2024-09-21T03:34:35.627438+0000","last_fullsized":"2024-09-21T03:34:35.627438+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:49:01.779676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:36.646422+0000","last_change":"2024-09-21T03:34:33.633593+0000","last_active":"2024-09-21T03:34:36.646422+0000","last_peered":"2024-09-21T03:34:36.646422+0000","last_clean":"2024-09-21T03:34:36.646422+0000","last_became_active":"2024-09-21T03:34:33.633254+0000","last_became_peered":"2024-09-21T03:34:33.633254+0000","last_unstale":"2024-09-21T03:34:36.646422+0000","last_undegraded":"2024-09-21T03:34:36.646422+0000","last_fullsized":"2024-09-21T03:34:36.646422+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_clean_scrub_stamp":"2024-09-21T03:34:32.598217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:57:01.944255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-21T03:34:33.621061+0000","last_change":"2024-09-21T03:34:33.621061+0000","last_active":"2024-09-21T03:34:33.604595+0000","last_peered":"2024-09-21T03:34:33.604595+0000","last_clean":"2024-09-21T03:34:33.604595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:34:33.621061+0000","last_undegraded":"2024-09-21T03:34:33.621061+0000","last_fullsized":"2024-09-21T03:34:33.621061+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-21T03:34:34.616134+0000","last_change":"2024-09-21T03:34:33.637630+0000","last_active":"2024-09-21T03:34:34.616134+0000","last_peered":"2024-09-21T03:34:34.616134+0000","last_clean":"2024-09-21T03:34:34.616134+0000","last_became_active":"2024-09-21T03:34:33.637290+0000","last_became_peered":"2024-09-21T03:34:33.637290+0000","last_unstale":"2024-09-21T03:34:34.616134+0000","last_undegraded":"2024-09-21T03:34:34.616134+0000","last_fullsized":"2024-09-21T03:34:34.616134+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_clean_scrub_stamp":"2024-09-21T03:34:32.598217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:00:46.535619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:35.627723+0000","last_change":"2024-09-21T03:34:34.644149+0000","last_active":"2024-09-21T03:34:35.627723+0000","last_peered":"2024-09-21T03:34:35.627723+0000","last_clean":"2024-09-21T03:34:35.627723+0000","last_became_active":"2024-09-21T03:34:34.643689+0000","last_became_peered":"2024-09-21T03:34:34.643689+0000","last_unstale":"2024-09-21T03:34:35.627723+0000","last_undegraded":"2024-09-21T03:34:35.627723+0000","last_fullsized":"2024-09-21T03:34:35.627723+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:51:48.433744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:36.646016+0000","last_change":"2024-09-21T03:34:33.626688+0000","last_active":"2024-09-21T03:34:36.646016+0000","last_peered":"2024-09-21T03:34:36.646016+0000","last_clean":"2024-09-21T03:34:36.646016+0000","last_became_active":"2024-09-21T03:34:33.626325+0000","last_became_peered":"2024-09-21T03:34:33.626325+0000","last_unstale":"2024-09-21T03:34:36.646016+0000","last_undegraded":"2024-09-21T03:34:36.646016+0000","last_fullsized":"2024-09-21T03:34:36.646016+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_clean_scrub_stamp":"2024-09-21T03:34:32.598217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:50:07.223556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-21T03:34:33.613904+0000","last_change":"2024-09-21T03:34:33.613904+0000","last_active":"2024-09-21T03:34:33.604595+0000","last_peered":"2024-09-21T03:34:33.604595+0000","last_clean":"2024-09-21T03:34:33.604595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:34:33.613904+0000","last_undegraded":"2024-09-21T03:34:33.613904+0000","last_fullsized":"2024-09-21T03:34:33.613904+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T03:34:33.638065+0000","last_change":"2024-09-21T03:34:33.633316+0000","last_active":"2024-09-21T03:34:33.638065+0000","last_peered":"2024-09-21T03:34:33.638065+0000","last_clean":"2024-09-21T03:34:33.638065+0000","last_became_active":"2024-09-21T03:34:33.632983+0000","last_became_peered":"2024-09-21T03:34:33.632983+0000","last_unstale":"2024-09-21T03:34:33.638065+0000","last_undegraded":"2024-09-21T03:34:33.638065+0000","last_fullsized":"2024-09-21T03:34:33.638065+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_clean_scrub_stamp":"2024-09-21T03:34:32.598217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:48:01.802931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-21T03:34:33.621976+0000","last_change":"2024-09-21T03:34:33.621976+0000","last_active":"2024-09-21T03:34:33.604595+0000","last_peered":"2024-09-21T03:34:33.604595+0000","last_clean":"2024-09-21T03:34:33.604595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:34:33.621976+0000","last_undegraded":"2024-09-21T03:34:33.621976+0000","last_fullsized":"2024-09-21T03:34:33.621976+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T03:34:33.632105+0000","last_change":"2024-09-21T03:34:33.630944+0000","last_active":"2024-09-21T03:34:33.632105+0000","last_peered":"2024-09-21T03:34:33.632105+0000","last_clean":"2024-09-21T03:34:33.632105+0000","last_became_active":"2024-09-21T03:34:33.630396+0000","last_became_peered":"2024-09-21T03:34:33.630396+0000","last_unstale":"2024-09-21T03:34:33.632105+0000","last_undegraded":"2024-09-21T03:34:33.632105+0000","last_fullsized":"2024-09-21T03:34:33.632105+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_clean_scrub_stamp":"2024-09-21T03:34:32.598217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:37:36.532174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-21T03:34:34.639837+0000","last_change":"2024-09-21T03:34:34.638755+0000","last_active":"2024-09-21T03:34:34.639837+0000","last_peered":"2024-09-21T03:34:34.639837+0000","last_clean":"2024-09-21T03:34:34.639837+0000","last_became_active":"2024-09-21T03:34:34.638432+0000","last_became_peered":"2024-09-21T03:34:34.638432+0000","last_unstale":"2024-09-21T03:34:34.639837+0000","last_undegraded":"2024-09-21T03:34:34.639837+0000","last_fullsized":"2024-09-21T03:34:34.639837+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:40:56.624795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T03:34:33.627809+0000","last_change":"2024-09-21T03:34:33.626093+0000","last_active":"2024-09-21T03:34:33.627809+0000","last_peered":"2024-09-21T03:34:33.627809+0000","last_clean":"2024-09-21T03:34:33.627809+0000","last_became_active":"2024-09-21T03:34:33.625771+0000","last_became_peered":"2024-09-21T03:34:33.625771+0000","last_unstale":"2024-09-21T03:34:33.627809+0000","last_undegraded":"2024-09-21T03:34:33.627809+0000","last_fullsized":"2024-09-21T03:34:33.627809+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_clean_scrub_stamp":"2024-09-21T03:34:32.598217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:05:08.028564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-21T03:34:33.613418+0000","last_change":"2024-09-21T03:34:33.613418+0000","last_active":"2024-09-21T03:34:33.604595+0000","last_peered":"2024-09-21T03:34:33.604595+0000","last_clean":"2024-09-21T03:34:33.604595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:34:33.613418+0000","last_undegraded":"2024-09-21T03:34:33.613418+0000","last_fullsized":"2024-09-21T03:34:33.613418+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:35.627816+0000","last_change":"2024-09-21T03:34:33.625164+0000","last_active":"2024-09-21T03:34:35.627816+0000","last_peered":"2024-09-21T03:34:35.627816+0000","last_clean":"2024-09-21T03:34:35.627816+0000","last_became_active":"2024-09-21T03:34:33.624299+0000","last_became_peered":"2024-09-21T03:34:33.624299+0000","last_unstale":"2024-09-21T03:34:35.627816+0000","last_undegraded":"2024-09-21T03:34:35.627816+0000","last_fullsized":"2024-09-21T03:34:35.627816+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_clean_scrub_stamp":"2024-09-21T03:34:32.598217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:32:32.643952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:36.646586+0000","last_change":"2024-09-21T03:34:34.637737+0000","last_active":"2024-09-21T03:34:36.646586+0000","last_peered":"2024-09-21T03:34:36.646586+0000","last_clean":"2024-09-21T03:34:36.646586+0000","last_became_active":"2024-09-21T03:34:34.637441+0000","last_became_peered":"2024-09-21T03:34:34.637441+0000","last_unstale":"2024-09-21T03:34:36.646586+0000","last_undegraded":"2024-09-21T03:34:36.646586+0000","last_fullsized":"2024-09-21T03:34:36.646586+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:12:23.136125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T03:34:33.636106+0000","last_change":"2024-09-21T03:34:33.631273+0000","last_active":"2024-09-21T03:34:33.636106+0000","last_peered":"2024-09-21T03:34:33.636106+0000","last_clean":"2024-09-21T03:34:33.636106+0000","last_became_active":"2024-09-21T03:34:33.628561+0000","last_became_peered":"2024-09-21T03:34:33.628561+0000","last_unstale":"2024-09-21T03:34:33.636106+0000","last_undegraded":"2024-09-21T03:34:33.636106+0000","last_fullsized":"2024-09-21T03:34:33.636106+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_clean_scrub_stamp":"2024-09-21T03:34:32.598217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:49:16.438096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-21T03:34:33.620026+0000","last_change":"2024-09-21T03:34:33.620026+0000","last_active":"2024-09-21T03:34:33.604595+0000","last_peered":"2024-09-21T03:34:33.604595+0000","last_clean":"2024-09-21T03:34:33.604595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:34:33.620026+0000","last_undegraded":"2024-09-21T03:34:33.620026+0000","last_fullsized":"2024-09-21T03:34:33.620026+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T03:34:33.638150+0000","last_change":"2024-09-21T03:34:33.633175+0000","last_active":"2024-09-21T03:34:33.638150+0000","last_peered":"2024-09-21T03:34:33.638150+0000","last_clean":"2024-09-21T03:34:33.638150+0000","last_became_active":"2024-09-21T03:34:33.632872+0000","last_became_peered":"2024-09-21T03:34:33.632872+0000","last_unstale":"2024-09-21T03:34:33.638150+0000","last_undegraded":"2024-09-21T03:34:33.638150+0000","last_fullsized":"2024-09-21T03:34:33.638150+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_clean_scrub_stamp":"2024-09-21T03:34:32.598217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:33:03.801458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-21T03:34:33.619685+0000","last_change":"2024-09-21T03:34:33.619685+0000","last_active":"2024-09-21T03:34:33.604595+0000","last_peered":"2024-09-21T03:34:33.604595+0000","last_clean":"2024-09-21T03:34:33.604595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:34:33.619685+0000","last_undegraded":"2024-09-21T03:34:33.619685+0000","last_fullsized":"2024-09-21T03:34:33.619685+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:35.628165+0000","last_change":"2024-09-21T03:34:33.639171+0000","last_active":"2024-09-21T03:34:35.628165+0000","last_peered":"2024-09-21T03:34:35.628165+0000","last_clean":"2024-09-21T03:34:35.628165+0000","last_became_active":"2024-09-21T03:34:33.638819+0000","last_became_peered":"2024-09-21T03:34:33.638819+0000","last_unstale":"2024-09-21T03:34:35.628165+0000","last_undegraded":"2024-09-21T03:34:35.628165+0000","last_fullsized":"2024-09-21T03:34:35.628165+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_clean_scrub_stamp":"2024-09-21T03:34:32.598217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:44:08.657412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-21T03:34:33.625230+0000","last_change":"2024-09-21T03:34:33.625230+0000","last_active":"2024-09-21T03:34:33.604595+0000","last_peered":"2024-09-21T03:34:33.604595+0000","last_clean":"2024-09-21T03:34:33.604595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:34:33.625230+0000","last_undegraded":"2024-09-21T03:34:33.625230+0000","last_fullsized":"2024-09-21T03:34:33.625230+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T03:34:33.637827+0000","last_change":"2024-09-21T03:34:33.635753+0000","last_active":"2024-09-21T03:34:33.637827+0000","last_peered":"2024-09-21T03:34:33.637827+0000","last_clean":"2024-09-21T03:34:33.637827+0000","last_became_active":"2024-09-21T03:34:33.631914+0000","last_became_peered":"2024-09-21T03:34:33.631914+0000","last_unstale":"2024-09-21T03:34:33.637827+0000","last_undegraded":"2024-09-21T03:34:33.637827+0000","last_fullsized":"2024-09-21T03:34:33.637827+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_clean_scrub_stamp":"2024-09-21T03:34:32.598217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:57:01.944255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":143,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-21T03:34:34.615053+0000","last_change":"2024-09-21T03:33:25.319776+0000","last_active":"2024-09-21T03:34:34.615053+0000","last_peered":"2024-09-21T03:34:34.615053+0000","last_clean":"2024-09-21T03:34:34.615053+0000","last_became_active":"2024-09-21T03:33:25.318399+0000","last_became_peered":"2024-09-21T03:33:25.318399+0000","last_unstale":"2024-09-21T03:34:34.615053+0000","last_undegraded":"2024-09-21T03:34:34.615053+0000","last_fullsized":"2024-09-21T03:34:34.615053+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:23.016672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:23.016672+0000","last_clean_scrub_stamp":"2024-09-21T03:33:23.016672+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:32:35.906114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":1668,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":15,"ondisk_log_size":15,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":1,"up_from":16,"seq":68719476754,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32212,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339628,"statfs":{"total":96636764160,"available":96603779072,"internally_reserved":0,"allocated":1040384,"data_stored":780143,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509458,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31596,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340244,"statfs":{"total":96636764160,"available":96604409856,"internally_reserved":0,"allocated":417792,"data_stored":175919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509457,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31612,"kb_used_data":420,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340228,"statfs":{"total":96636764160,"available":96604393472,"internally_reserved":0,"allocated":430080,"data_stored":182586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509458,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27436,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344404,"statfs":{"total":96636764160,"available":96608669696,"internally_reserved":0,"allocated":417792,"data_stored":175919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509457,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31596,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340244,"statfs":{"total":96636764160,"available":96604409856,"internally_reserved":0,"allocated":417792,"data_stored":175919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542162,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31640,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340200,"statfs":{"total":96636764160,"available":96604364800,"internally_reserved":0,"allocated":458752,"data_stored":196479,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574866,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32208,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339632,"statfs":{"total":96636764160,"available":96603783168,"internally_reserved":0,"allocated":1040384,"data_stored":780703,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542163,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35816,"kb_used_data":464,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336024,"statfs":{"total":96636764160,"available":96600088576,"internally_reserved":0,"allocated":475136,"data_stored":198181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T03:34:39.460 INFO:tasks.ceph.ceph_manager.ceph:PG 6.3b is not active+clean 2024-09-21T03:34:39.461 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-09-21T03:34:33.614399+0000', 'last_change': '2024-09-21T03:34:33.614399+0000', 'last_active': '2024-09-21T03:34:33.604595+0000', 'last_peered': '2024-09-21T03:34:33.604595+0000', 'last_clean': '2024-09-21T03:34:33.604595+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T03:34:33.614399+0000', 'last_undegraded': '2024-09-21T03:34:33.614399+0000', 'last_fullsized': '2024-09-21T03:34:33.614399+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T03:34:33.604595+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T03:34:33.604595+0000', 'last_clean_scrub_stamp': '2024-09-21T03:34:33.604595+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-21T03:34:39.461 INFO:tasks.ceph.ceph_manager.ceph:PG 6.37 is not active+clean 2024-09-21T03:34:39.461 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-09-21T03:34:33.614285+0000', 'last_change': '2024-09-21T03:34:33.614285+0000', 'last_active': '2024-09-21T03:34:33.604595+0000', 'last_peered': '2024-09-21T03:34:33.604595+0000', 'last_clean': '2024-09-21T03:34:33.604595+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T03:34:33.614285+0000', 'last_undegraded': '2024-09-21T03:34:33.614285+0000', 'last_fullsized': '2024-09-21T03:34:33.614285+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T03:34:33.604595+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T03:34:33.604595+0000', 'last_clean_scrub_stamp': '2024-09-21T03:34:33.604595+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-21T03:34:39.461 INFO:tasks.ceph.ceph_manager.ceph:PG 6.35 is not active+clean 2024-09-21T03:34:39.461 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-09-21T03:34:33.622983+0000', 'last_change': '2024-09-21T03:34:33.622983+0000', 'last_active': '2024-09-21T03:34:33.604595+0000', 'last_peered': '2024-09-21T03:34:33.604595+0000', 'last_clean': '2024-09-21T03:34:33.604595+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T03:34:33.622983+0000', 'last_undegraded': '2024-09-21T03:34:33.622983+0000', 'last_fullsized': '2024-09-21T03:34:33.622983+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T03:34:33.604595+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T03:34:33.604595+0000', 'last_clean_scrub_stamp': '2024-09-21T03:34:33.604595+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-21T03:34:39.461 INFO:tasks.ceph.ceph_manager.ceph:PG 6.34 is not active+clean 2024-09-21T03:34:39.461 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-09-21T03:34:33.614478+0000', 'last_change': '2024-09-21T03:34:33.614478+0000', 'last_active': '2024-09-21T03:34:33.604595+0000', 'last_peered': '2024-09-21T03:34:33.604595+0000', 'last_clean': '2024-09-21T03:34:33.604595+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T03:34:33.614478+0000', 'last_undegraded': '2024-09-21T03:34:33.614478+0000', 'last_fullsized': '2024-09-21T03:34:33.614478+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T03:34:33.604595+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T03:34:33.604595+0000', 'last_clean_scrub_stamp': '2024-09-21T03:34:33.604595+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-21T03:34:39.461 INFO:tasks.ceph.ceph_manager.ceph:PG 6.32 is not active+clean 2024-09-21T03:34:39.461 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-09-21T03:34:33.616107+0000', 'last_change': '2024-09-21T03:34:33.616107+0000', 'last_active': '2024-09-21T03:34:33.604595+0000', 'last_peered': '2024-09-21T03:34:33.604595+0000', 'last_clean': '2024-09-21T03:34:33.604595+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T03:34:33.616107+0000', 'last_undegraded': '2024-09-21T03:34:33.616107+0000', 'last_fullsized': '2024-09-21T03:34:33.616107+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T03:34:33.604595+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T03:34:33.604595+0000', 'last_clean_scrub_stamp': '2024-09-21T03:34:33.604595+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-21T03:34:39.461 INFO:tasks.ceph.ceph_manager.ceph:PG 6.31 is not active+clean 2024-09-21T03:34:39.461 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-09-21T03:34:33.625487+0000', 'last_change': '2024-09-21T03:34:33.625487+0000', 'last_active': '2024-09-21T03:34:33.604595+0000', 'last_peered': '2024-09-21T03:34:33.604595+0000', 'last_clean': '2024-09-21T03:34:33.604595+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T03:34:33.625487+0000', 'last_undegraded': '2024-09-21T03:34:33.625487+0000', 'last_fullsized': '2024-09-21T03:34:33.625487+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T03:34:33.604595+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T03:34:33.604595+0000', 'last_clean_scrub_stamp': '2024-09-21T03:34:33.604595+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-21T03:34:39.462 INFO:tasks.ceph.ceph_manager.ceph:PG 6.2f is not active+clean 2024-09-21T03:34:39.462 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-09-21T03:34:33.624716+0000', 'last_change': '2024-09-21T03:34:33.624716+0000', 'last_active': '2024-09-21T03:34:33.604595+0000', 'last_peered': '2024-09-21T03:34:33.604595+0000', 'last_clean': '2024-09-21T03:34:33.604595+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T03:34:33.624716+0000', 'last_undegraded': '2024-09-21T03:34:33.624716+0000', 'last_fullsized': '2024-09-21T03:34:33.624716+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T03:34:33.604595+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T03:34:33.604595+0000', 'last_clean_scrub_stamp': '2024-09-21T03:34:33.604595+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-21T03:34:39.462 INFO:tasks.ceph.ceph_manager.ceph:PG 6.2e is not active+clean 2024-09-21T03:34:39.462 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-09-21T03:34:33.623146+0000', 'last_change': '2024-09-21T03:34:33.623146+0000', 'last_active': '2024-09-21T03:34:33.604595+0000', 'last_peered': '2024-09-21T03:34:33.604595+0000', 'last_clean': '2024-09-21T03:34:33.604595+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T03:34:33.623146+0000', 'last_undegraded': '2024-09-21T03:34:33.623146+0000', 'last_fullsized': '2024-09-21T03:34:33.623146+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T03:34:33.604595+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T03:34:33.604595+0000', 'last_clean_scrub_stamp': '2024-09-21T03:34:33.604595+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-21T03:34:39.462 INFO:tasks.ceph.ceph_manager.ceph:PG 6.2c is not active+clean 2024-09-21T03:34:39.462 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-09-21T03:34:33.620475+0000', 'last_change': '2024-09-21T03:34:33.620475+0000', 'last_active': '2024-09-21T03:34:33.604595+0000', 'last_peered': '2024-09-21T03:34:33.604595+0000', 'last_clean': '2024-09-21T03:34:33.604595+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T03:34:33.620475+0000', 'last_undegraded': '2024-09-21T03:34:33.620475+0000', 'last_fullsized': '2024-09-21T03:34:33.620475+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T03:34:33.604595+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T03:34:33.604595+0000', 'last_clean_scrub_stamp': '2024-09-21T03:34:33.604595+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-21T03:34:39.462 INFO:tasks.ceph.ceph_manager.ceph:PG 6.2b is not active+clean 2024-09-21T03:34:39.462 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-09-21T03:34:33.621111+0000', 'last_change': '2024-09-21T03:34:33.621111+0000', 'last_active': '2024-09-21T03:34:33.604595+0000', 'last_peered': '2024-09-21T03:34:33.604595+0000', 'last_clean': '2024-09-21T03:34:33.604595+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T03:34:33.621111+0000', 'last_undegraded': '2024-09-21T03:34:33.621111+0000', 'last_fullsized': '2024-09-21T03:34:33.621111+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T03:34:33.604595+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T03:34:33.604595+0000', 'last_clean_scrub_stamp': '2024-09-21T03:34:33.604595+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-21T03:34:39.462 INFO:tasks.ceph.ceph_manager.ceph:PG 6.2a is not active+clean 2024-09-21T03:34:39.462 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-09-21T03:34:33.621590+0000', 'last_change': '2024-09-21T03:34:33.621590+0000', 'last_active': '2024-09-21T03:34:33.604595+0000', 'last_peered': '2024-09-21T03:34:33.604595+0000', 'last_clean': '2024-09-21T03:34:33.604595+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T03:34:33.621590+0000', 'last_undegraded': '2024-09-21T03:34:33.621590+0000', 'last_fullsized': '2024-09-21T03:34:33.621590+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T03:34:33.604595+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T03:34:33.604595+0000', 'last_clean_scrub_stamp': '2024-09-21T03:34:33.604595+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-21T03:34:39.462 INFO:tasks.ceph.ceph_manager.ceph:PG 6.29 is not active+clean 2024-09-21T03:34:39.463 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-09-21T03:34:33.624872+0000', 'last_change': '2024-09-21T03:34:33.624872+0000', 'last_active': '2024-09-21T03:34:33.604595+0000', 'last_peered': '2024-09-21T03:34:33.604595+0000', 'last_clean': '2024-09-21T03:34:33.604595+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T03:34:33.624872+0000', 'last_undegraded': '2024-09-21T03:34:33.624872+0000', 'last_fullsized': '2024-09-21T03:34:33.624872+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T03:34:33.604595+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T03:34:33.604595+0000', 'last_clean_scrub_stamp': '2024-09-21T03:34:33.604595+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-21T03:34:39.463 INFO:tasks.ceph.ceph_manager.ceph:PG 6.27 is not active+clean 2024-09-21T03:34:39.463 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-09-21T03:34:33.616131+0000', 'last_change': '2024-09-21T03:34:33.616131+0000', 'last_active': '2024-09-21T03:34:33.604595+0000', 'last_peered': '2024-09-21T03:34:33.604595+0000', 'last_clean': '2024-09-21T03:34:33.604595+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T03:34:33.616131+0000', 'last_undegraded': '2024-09-21T03:34:33.616131+0000', 'last_fullsized': '2024-09-21T03:34:33.616131+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T03:34:33.604595+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T03:34:33.604595+0000', 'last_clean_scrub_stamp': '2024-09-21T03:34:33.604595+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-21T03:34:39.463 INFO:tasks.ceph.ceph_manager.ceph:PG 6.25 is not active+clean 2024-09-21T03:34:39.463 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-09-21T03:34:33.622462+0000', 'last_change': '2024-09-21T03:34:33.622462+0000', 'last_active': '2024-09-21T03:34:33.604595+0000', 'last_peered': '2024-09-21T03:34:33.604595+0000', 'last_clean': '2024-09-21T03:34:33.604595+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T03:34:33.622462+0000', 'last_undegraded': '2024-09-21T03:34:33.622462+0000', 'last_fullsized': '2024-09-21T03:34:33.622462+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T03:34:33.604595+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T03:34:33.604595+0000', 'last_clean_scrub_stamp': '2024-09-21T03:34:33.604595+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-21T03:34:39.463 INFO:tasks.ceph.ceph_manager.ceph:PG 6.23 is not active+clean 2024-09-21T03:34:39.463 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-09-21T03:34:33.628158+0000', 'last_change': '2024-09-21T03:34:33.628158+0000', 'last_active': '2024-09-21T03:34:33.604595+0000', 'last_peered': '2024-09-21T03:34:33.604595+0000', 'last_clean': '2024-09-21T03:34:33.604595+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T03:34:33.628158+0000', 'last_undegraded': '2024-09-21T03:34:33.628158+0000', 'last_fullsized': '2024-09-21T03:34:33.628158+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T03:34:33.604595+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T03:34:33.604595+0000', 'last_clean_scrub_stamp': '2024-09-21T03:34:33.604595+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-21T03:34:39.463 INFO:tasks.ceph.ceph_manager.ceph:PG 6.22 is not active+clean 2024-09-21T03:34:39.463 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-09-21T03:34:33.621235+0000', 'last_change': '2024-09-21T03:34:33.621235+0000', 'last_active': '2024-09-21T03:34:33.604595+0000', 'last_peered': '2024-09-21T03:34:33.604595+0000', 'last_clean': '2024-09-21T03:34:33.604595+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T03:34:33.621235+0000', 'last_undegraded': '2024-09-21T03:34:33.621235+0000', 'last_fullsized': '2024-09-21T03:34:33.621235+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T03:34:33.604595+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T03:34:33.604595+0000', 'last_clean_scrub_stamp': '2024-09-21T03:34:33.604595+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-21T03:34:39.463 INFO:tasks.ceph.ceph_manager.ceph:PG 6.1c is not active+clean 2024-09-21T03:34:39.463 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-09-21T03:34:33.621969+0000', 'last_change': '2024-09-21T03:34:33.621969+0000', 'last_active': '2024-09-21T03:34:33.604595+0000', 'last_peered': '2024-09-21T03:34:33.604595+0000', 'last_clean': '2024-09-21T03:34:33.604595+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T03:34:33.621969+0000', 'last_undegraded': '2024-09-21T03:34:33.621969+0000', 'last_fullsized': '2024-09-21T03:34:33.621969+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T03:34:33.604595+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T03:34:33.604595+0000', 'last_clean_scrub_stamp': '2024-09-21T03:34:33.604595+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-21T03:34:39.464 INFO:tasks.ceph.ceph_manager.ceph:PG 6.1e is not active+clean 2024-09-21T03:34:39.464 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-09-21T03:34:33.622277+0000', 'last_change': '2024-09-21T03:34:33.622277+0000', 'last_active': '2024-09-21T03:34:33.604595+0000', 'last_peered': '2024-09-21T03:34:33.604595+0000', 'last_clean': '2024-09-21T03:34:33.604595+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T03:34:33.622277+0000', 'last_undegraded': '2024-09-21T03:34:33.622277+0000', 'last_fullsized': '2024-09-21T03:34:33.622277+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T03:34:33.604595+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T03:34:33.604595+0000', 'last_clean_scrub_stamp': '2024-09-21T03:34:33.604595+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-21T03:34:39.464 INFO:tasks.ceph.ceph_manager.ceph:PG 6.19 is not active+clean 2024-09-21T03:34:39.464 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-09-21T03:34:33.618669+0000', 'last_change': '2024-09-21T03:34:33.618669+0000', 'last_active': '2024-09-21T03:34:33.604595+0000', 'last_peered': '2024-09-21T03:34:33.604595+0000', 'last_clean': '2024-09-21T03:34:33.604595+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T03:34:33.618669+0000', 'last_undegraded': '2024-09-21T03:34:33.618669+0000', 'last_fullsized': '2024-09-21T03:34:33.618669+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T03:34:33.604595+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T03:34:33.604595+0000', 'last_clean_scrub_stamp': '2024-09-21T03:34:33.604595+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-21T03:34:39.464 INFO:tasks.ceph.ceph_manager.ceph:PG 6.1a is not active+clean 2024-09-21T03:34:39.464 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-09-21T03:34:33.620634+0000', 'last_change': '2024-09-21T03:34:33.620634+0000', 'last_active': '2024-09-21T03:34:33.604595+0000', 'last_peered': '2024-09-21T03:34:33.604595+0000', 'last_clean': '2024-09-21T03:34:33.604595+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T03:34:33.620634+0000', 'last_undegraded': '2024-09-21T03:34:33.620634+0000', 'last_fullsized': '2024-09-21T03:34:33.620634+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T03:34:33.604595+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T03:34:33.604595+0000', 'last_clean_scrub_stamp': '2024-09-21T03:34:33.604595+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-21T03:34:39.464 INFO:tasks.ceph.ceph_manager.ceph:PG 6.15 is not active+clean 2024-09-21T03:34:39.464 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-09-21T03:34:33.616082+0000', 'last_change': '2024-09-21T03:34:33.616082+0000', 'last_active': '2024-09-21T03:34:33.604595+0000', 'last_peered': '2024-09-21T03:34:33.604595+0000', 'last_clean': '2024-09-21T03:34:33.604595+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T03:34:33.616082+0000', 'last_undegraded': '2024-09-21T03:34:33.616082+0000', 'last_fullsized': '2024-09-21T03:34:33.616082+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T03:34:33.604595+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T03:34:33.604595+0000', 'last_clean_scrub_stamp': '2024-09-21T03:34:33.604595+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-21T03:34:39.464 INFO:tasks.ceph.ceph_manager.ceph:PG 6.12 is not active+clean 2024-09-21T03:34:39.464 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-09-21T03:34:33.619516+0000', 'last_change': '2024-09-21T03:34:33.619516+0000', 'last_active': '2024-09-21T03:34:33.604595+0000', 'last_peered': '2024-09-21T03:34:33.604595+0000', 'last_clean': '2024-09-21T03:34:33.604595+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T03:34:33.619516+0000', 'last_undegraded': '2024-09-21T03:34:33.619516+0000', 'last_fullsized': '2024-09-21T03:34:33.619516+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T03:34:33.604595+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T03:34:33.604595+0000', 'last_clean_scrub_stamp': '2024-09-21T03:34:33.604595+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-21T03:34:39.464 INFO:tasks.ceph.ceph_manager.ceph:PG 6.13 is not active+clean 2024-09-21T03:34:39.464 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-09-21T03:34:33.613436+0000', 'last_change': '2024-09-21T03:34:33.613436+0000', 'last_active': '2024-09-21T03:34:33.604595+0000', 'last_peered': '2024-09-21T03:34:33.604595+0000', 'last_clean': '2024-09-21T03:34:33.604595+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T03:34:33.613436+0000', 'last_undegraded': '2024-09-21T03:34:33.613436+0000', 'last_fullsized': '2024-09-21T03:34:33.613436+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T03:34:33.604595+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T03:34:33.604595+0000', 'last_clean_scrub_stamp': '2024-09-21T03:34:33.604595+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-21T03:34:39.465 INFO:tasks.ceph.ceph_manager.ceph:PG 6.8 is not active+clean 2024-09-21T03:34:39.465 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-09-21T03:34:33.620205+0000', 'last_change': '2024-09-21T03:34:33.620205+0000', 'last_active': '2024-09-21T03:34:33.604595+0000', 'last_peered': '2024-09-21T03:34:33.604595+0000', 'last_clean': '2024-09-21T03:34:33.604595+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T03:34:33.620205+0000', 'last_undegraded': '2024-09-21T03:34:33.620205+0000', 'last_fullsized': '2024-09-21T03:34:33.620205+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T03:34:33.604595+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T03:34:33.604595+0000', 'last_clean_scrub_stamp': '2024-09-21T03:34:33.604595+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-21T03:34:39.465 INFO:tasks.ceph.ceph_manager.ceph:PG 6.a is not active+clean 2024-09-21T03:34:39.465 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-09-21T03:34:33.621061+0000', 'last_change': '2024-09-21T03:34:33.621061+0000', 'last_active': '2024-09-21T03:34:33.604595+0000', 'last_peered': '2024-09-21T03:34:33.604595+0000', 'last_clean': '2024-09-21T03:34:33.604595+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T03:34:33.621061+0000', 'last_undegraded': '2024-09-21T03:34:33.621061+0000', 'last_fullsized': '2024-09-21T03:34:33.621061+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T03:34:33.604595+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T03:34:33.604595+0000', 'last_clean_scrub_stamp': '2024-09-21T03:34:33.604595+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-21T03:34:39.465 INFO:tasks.ceph.ceph_manager.ceph:PG 6.4 is not active+clean 2024-09-21T03:34:39.465 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-09-21T03:34:33.613904+0000', 'last_change': '2024-09-21T03:34:33.613904+0000', 'last_active': '2024-09-21T03:34:33.604595+0000', 'last_peered': '2024-09-21T03:34:33.604595+0000', 'last_clean': '2024-09-21T03:34:33.604595+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T03:34:33.613904+0000', 'last_undegraded': '2024-09-21T03:34:33.613904+0000', 'last_fullsized': '2024-09-21T03:34:33.613904+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T03:34:33.604595+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T03:34:33.604595+0000', 'last_clean_scrub_stamp': '2024-09-21T03:34:33.604595+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-21T03:34:39.465 INFO:tasks.ceph.ceph_manager.ceph:PG 6.5 is not active+clean 2024-09-21T03:34:39.465 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-09-21T03:34:33.621976+0000', 'last_change': '2024-09-21T03:34:33.621976+0000', 'last_active': '2024-09-21T03:34:33.604595+0000', 'last_peered': '2024-09-21T03:34:33.604595+0000', 'last_clean': '2024-09-21T03:34:33.604595+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T03:34:33.621976+0000', 'last_undegraded': '2024-09-21T03:34:33.621976+0000', 'last_fullsized': '2024-09-21T03:34:33.621976+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T03:34:33.604595+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T03:34:33.604595+0000', 'last_clean_scrub_stamp': '2024-09-21T03:34:33.604595+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-21T03:34:39.465 INFO:tasks.ceph.ceph_manager.ceph:PG 6.0 is not active+clean 2024-09-21T03:34:39.465 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-09-21T03:34:33.613418+0000', 'last_change': '2024-09-21T03:34:33.613418+0000', 'last_active': '2024-09-21T03:34:33.604595+0000', 'last_peered': '2024-09-21T03:34:33.604595+0000', 'last_clean': '2024-09-21T03:34:33.604595+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T03:34:33.613418+0000', 'last_undegraded': '2024-09-21T03:34:33.613418+0000', 'last_fullsized': '2024-09-21T03:34:33.613418+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T03:34:33.604595+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T03:34:33.604595+0000', 'last_clean_scrub_stamp': '2024-09-21T03:34:33.604595+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-21T03:34:39.465 INFO:tasks.ceph.ceph_manager.ceph:PG 6.2 is not active+clean 2024-09-21T03:34:39.465 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-09-21T03:34:33.620026+0000', 'last_change': '2024-09-21T03:34:33.620026+0000', 'last_active': '2024-09-21T03:34:33.604595+0000', 'last_peered': '2024-09-21T03:34:33.604595+0000', 'last_clean': '2024-09-21T03:34:33.604595+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T03:34:33.620026+0000', 'last_undegraded': '2024-09-21T03:34:33.620026+0000', 'last_fullsized': '2024-09-21T03:34:33.620026+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T03:34:33.604595+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T03:34:33.604595+0000', 'last_clean_scrub_stamp': '2024-09-21T03:34:33.604595+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-21T03:34:39.466 INFO:tasks.ceph.ceph_manager.ceph:PG 6.3 is not active+clean 2024-09-21T03:34:39.466 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-09-21T03:34:33.619685+0000', 'last_change': '2024-09-21T03:34:33.619685+0000', 'last_active': '2024-09-21T03:34:33.604595+0000', 'last_peered': '2024-09-21T03:34:33.604595+0000', 'last_clean': '2024-09-21T03:34:33.604595+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T03:34:33.619685+0000', 'last_undegraded': '2024-09-21T03:34:33.619685+0000', 'last_fullsized': '2024-09-21T03:34:33.619685+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T03:34:33.604595+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T03:34:33.604595+0000', 'last_clean_scrub_stamp': '2024-09-21T03:34:33.604595+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-21T03:34:39.466 INFO:tasks.ceph.ceph_manager.ceph:PG 6.7 is not active+clean 2024-09-21T03:34:39.466 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-09-21T03:34:33.625230+0000', 'last_change': '2024-09-21T03:34:33.625230+0000', 'last_active': '2024-09-21T03:34:33.604595+0000', 'last_peered': '2024-09-21T03:34:33.604595+0000', 'last_clean': '2024-09-21T03:34:33.604595+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T03:34:33.625230+0000', 'last_undegraded': '2024-09-21T03:34:33.625230+0000', 'last_fullsized': '2024-09-21T03:34:33.625230+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T03:34:33.604595+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T03:34:33.604595+0000', 'last_clean_scrub_stamp': '2024-09-21T03:34:33.604595+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-21T03:34:39.466 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-21T03:34:39.512 INFO:tasks.ceph.mds.c.smithi038.stderr:2024-09-21T03:34:39.510+0000 7ff0d8d9a640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T03:34:39.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:39.612+0000 7fd81947e640 1 -- 172.21.15.38:0/4209845142 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd814162590 msgr2=0x7fd8141629a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:39.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:39.612+0000 7fd81947e640 1 --2- 172.21.15.38:0/4209845142 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd814162590 0x7fd8141629a0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fd7fc005fd0 tx=0x7fd7fc02a5d0 comp rx=0 tx=0).stop 2024-09-21T03:34:39.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:39.612+0000 7fd81947e640 1 -- 172.21.15.38:0/4209845142 shutdown_connections 2024-09-21T03:34:39.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:39.612+0000 7fd81947e640 1 --2- 172.21.15.38:0/4209845142 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd814162590 0x7fd8141629a0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:39.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:39.612+0000 7fd81947e640 1 -- 172.21.15.38:0/4209845142 >> 172.21.15.38:0/4209845142 conn(0x7fd8140d4440 msgr2=0x7fd8140d4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:39.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:39.612+0000 7fd81947e640 1 -- 172.21.15.38:0/4209845142 shutdown_connections 2024-09-21T03:34:39.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:39.612+0000 7fd81947e640 1 -- 172.21.15.38:0/4209845142 wait complete. 2024-09-21T03:34:39.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:39.613+0000 7fd81947e640 1 Processor -- start 2024-09-21T03:34:39.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:39.614+0000 7fd81947e640 1 -- start start 2024-09-21T03:34:39.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:39.615+0000 7fd81947e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd8140b2e60 0x7fd8140c7430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:39.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:39.615+0000 7fd81947e640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd8140cfe00 0x7fd81415dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:39.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:39.615+0000 7fd81947e640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd814162590 0x7fd81415e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:39.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:39.615+0000 7fd81947e640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fd8140b3b00 con 0x7fd8140b2e60 2024-09-21T03:34:39.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:39.615+0000 7fd81947e640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd8140b3980 con 0x7fd8140cfe00 2024-09-21T03:34:39.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:39.615+0000 7fd81947e640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fd8140b4f70 con 0x7fd814162590 2024-09-21T03:34:39.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:39.615+0000 7fd812ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd8140b2e60 0x7fd8140c7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:39.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:39.615+0000 7fd812ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd8140b2e60 0x7fd8140c7430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:33710/0 (socket says 172.21.15.38:33710) 2024-09-21T03:34:39.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:39.615+0000 7fd812ffd640 1 -- 172.21.15.38:0/1088796941 learned_addr learned my addr 172.21.15.38:0/1088796941 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:34:39.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:39.615+0000 7fd80bfff640 1 --2- 172.21.15.38:0/1088796941 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd8140cfe00 0x7fd81415dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:39.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:39.615+0000 7fd8137fe640 1 --2- 172.21.15.38:0/1088796941 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd814162590 0x7fd81415e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:39.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:39.615+0000 7fd812ffd640 1 -- 172.21.15.38:0/1088796941 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd814162590 msgr2=0x7fd81415e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:39.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:39.615+0000 7fd812ffd640 1 --2- 172.21.15.38:0/1088796941 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd814162590 0x7fd81415e180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:39.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:39.615+0000 7fd812ffd640 1 -- 172.21.15.38:0/1088796941 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd8140cfe00 msgr2=0x7fd81415dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:39.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:39.615+0000 7fd812ffd640 1 --2- 172.21.15.38:0/1088796941 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd8140cfe00 0x7fd81415dc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:39.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:39.615+0000 7fd812ffd640 1 -- 172.21.15.38:0/1088796941 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8141eb170 con 0x7fd8140b2e60 2024-09-21T03:34:39.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:39.615+0000 7fd812ffd640 1 --2- 172.21.15.38:0/1088796941 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd8140b2e60 0x7fd8140c7430 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fd80400c3c0 tx=0x7fd80400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:39.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:39.616+0000 7fd80b7fe640 1 -- 172.21.15.38:0/1088796941 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd804017070 con 0x7fd8140b2e60 2024-09-21T03:34:39.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:39.616+0000 7fd80b7fe640 1 -- 172.21.15.38:0/1088796941 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd8040035f0 con 0x7fd8140b2e60 2024-09-21T03:34:39.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:39.616+0000 7fd81947e640 1 -- 172.21.15.38:0/1088796941 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd814159670 con 0x7fd8140b2e60 2024-09-21T03:34:39.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:39.616+0000 7fd80b7fe640 1 -- 172.21.15.38:0/1088796941 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd804003a40 con 0x7fd8140b2e60 2024-09-21T03:34:39.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:39.616+0000 7fd81947e640 1 -- 172.21.15.38:0/1088796941 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd814159c30 con 0x7fd8140b2e60 2024-09-21T03:34:39.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:39.617+0000 7fd81947e640 1 -- 172.21.15.38:0/1088796941 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8141629a0 con 0x7fd8140b2e60 2024-09-21T03:34:39.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:39.617+0000 7fd80b7fe640 1 -- 172.21.15.38:0/1088796941 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fd80401a070 con 0x7fd8140b2e60 2024-09-21T03:34:39.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:39.618+0000 7fd80b7fe640 1 --2- 172.21.15.38:0/1088796941 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd7f4078b90 0x7fd7f407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:39.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:39.618+0000 7fd80b7fe640 1 -- 172.21.15.38:0/1088796941 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7fd80409a1a0 con 0x7fd8140b2e60 2024-09-21T03:34:39.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:39.618+0000 7fd80bfff640 1 --2- 172.21.15.38:0/1088796941 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd7f4078b90 0x7fd7f407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:39.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:39.619+0000 7fd80bfff640 1 --2- 172.21.15.38:0/1088796941 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd7f4078b90 0x7fd7f407b0a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fd7fc033000 tx=0x7fd7fc004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:39.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:39.623+0000 7fd80b7fe640 1 -- 172.21.15.38:0/1088796941 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fd814162ee0 con 0x7fd8140b2e60 2024-09-21T03:34:39.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:39.884+0000 7fd81947e640 1 -- 172.21.15.38:0/1088796941 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7fd814076180 con 0x7fd8140b2e60 2024-09-21T03:34:39.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:39.884+0000 7fd80b7fe640 1 -- 172.21.15.38:0/1088796941 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1688 (secure 0 0 0) 0x7fd804062450 con 0x7fd8140b2e60 2024-09-21T03:34:39.886 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:34:39.886 INFO:teuthology.orchestra.run.smithi038.stdout:{"fsid":"9d50d66f-656b-4fe2-8e3b-fe53b79c4d43","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":89,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":40,"num_osds":8,"num_up_osds":8,"osd_up_since":1726889604,"num_in_osds":8,"osd_in_since":1726889594,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":66},{"state_name":"creating+peering","count":31}],"num_pgs":97,"num_pools":3,"num_objects":11,"data_bytes":592036,"bytes_used":260214784,"bytes_avail":772833898496,"bytes_total":773094113280,"inactive_pgs_ratio":0.31958761811256409,"write_bytes_sec":1876,"read_op_per_sec":0,"write_op_per_sec":2},"fsmap":{"epoch":17,"btime":"2024-09-21T03:34:37:631111+0000","id":2,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":2,"rank":0,"name":"c","status":"up:active","gid":5013}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":5,"modified":"2024-09-21T03:34:26.990546+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-21T03:34:39.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:39.886+0000 7fd81947e640 1 -- 172.21.15.38:0/1088796941 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd7f4078b90 msgr2=0x7fd7f407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:39.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:39.886+0000 7fd81947e640 1 --2- 172.21.15.38:0/1088796941 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd7f4078b90 0x7fd7f407b0a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fd7fc033000 tx=0x7fd7fc004730 comp rx=0 tx=0).stop 2024-09-21T03:34:39.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:39.886+0000 7fd81947e640 1 -- 172.21.15.38:0/1088796941 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd8140b2e60 msgr2=0x7fd8140c7430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:39.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:39.886+0000 7fd81947e640 1 --2- 172.21.15.38:0/1088796941 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd8140b2e60 0x7fd8140c7430 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fd80400c3c0 tx=0x7fd80400c880 comp rx=0 tx=0).stop 2024-09-21T03:34:39.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:39.886+0000 7fd81947e640 1 -- 172.21.15.38:0/1088796941 shutdown_connections 2024-09-21T03:34:39.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:39.886+0000 7fd81947e640 1 --2- 172.21.15.38:0/1088796941 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd7f4078b90 0x7fd7f407b0a0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:39.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:39.886+0000 7fd81947e640 1 --2- 172.21.15.38:0/1088796941 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd814162590 0x7fd81415e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:39.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:39.886+0000 7fd81947e640 1 --2- 172.21.15.38:0/1088796941 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd8140cfe00 0x7fd81415dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:39.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:39.886+0000 7fd81947e640 1 --2- 172.21.15.38:0/1088796941 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd8140b2e60 0x7fd8140c7430 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:39.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:39.886+0000 7fd81947e640 1 -- 172.21.15.38:0/1088796941 >> 172.21.15.38:0/1088796941 conn(0x7fd8140d4440 msgr2=0x7fd8140b3270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:39.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:39.886+0000 7fd81947e640 1 -- 172.21.15.38:0/1088796941 shutdown_connections 2024-09-21T03:34:39.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:39.886+0000 7fd81947e640 1 -- 172.21.15.38:0/1088796941 wait complete. 2024-09-21T03:34:39.899 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-21T03:34:39.899 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T03:34:40.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:40.053+0000 7fd0e4913640 1 -- 172.21.15.38:0/1400697491 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd0e0073d50 msgr2=0x7fd0e0074160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:40.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:40.053+0000 7fd0e4913640 1 --2- 172.21.15.38:0/1400697491 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd0e0073d50 0x7fd0e0074160 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fd0e00b3af0 tx=0x7fd0d802aac0 comp rx=0 tx=0).stop 2024-09-21T03:34:40.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:40.054+0000 7fd0e4913640 1 -- 172.21.15.38:0/1400697491 shutdown_connections 2024-09-21T03:34:40.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:40.054+0000 7fd0e4913640 1 --2- 172.21.15.38:0/1400697491 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd0e0073d50 0x7fd0e0074160 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:40.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:40.054+0000 7fd0e4913640 1 --2- 172.21.15.38:0/1400697491 >> v2:172.21.15.195:3300/0 conn(0x7fd0e00d0b40 0x7fd0e006e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:40.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:40.054+0000 7fd0e4913640 1 -- 172.21.15.38:0/1400697491 >> 172.21.15.38:0/1400697491 conn(0x7fd0e00b2cc0 msgr2=0x7fd0e00b30d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:40.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:40.054+0000 7fd0e4913640 1 -- 172.21.15.38:0/1400697491 shutdown_connections 2024-09-21T03:34:40.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:40.054+0000 7fd0e4913640 1 -- 172.21.15.38:0/1400697491 wait complete. 2024-09-21T03:34:40.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:40.056+0000 7fd0e4913640 1 Processor -- start 2024-09-21T03:34:40.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:40.056+0000 7fd0e4913640 1 -- start start 2024-09-21T03:34:40.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:40.057+0000 7fd0e4913640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd0e006ec30 0x7fd0e014a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:40.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:40.057+0000 7fd0e4913640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd0e0073d50 0x7fd0e014bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:40.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:40.057+0000 7fd0e4913640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd0e00d0b40 0x7fd0e014c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:40.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:40.057+0000 7fd0e4913640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fd0e015b750 con 0x7fd0e006ec30 2024-09-21T03:34:40.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:40.057+0000 7fd0e4913640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd0e015b5d0 con 0x7fd0e00d0b40 2024-09-21T03:34:40.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:40.057+0000 7fd0e4913640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fd0e015b8d0 con 0x7fd0e0073d50 2024-09-21T03:34:40.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:40.057+0000 7fd0de59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd0e006ec30 0x7fd0e014a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:40.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:40.057+0000 7fd0de59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd0e006ec30 0x7fd0e014a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:33722/0 (socket says 172.21.15.38:33722) 2024-09-21T03:34:40.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:40.057+0000 7fd0de59c640 1 -- 172.21.15.38:0/492255507 learned_addr learned my addr 172.21.15.38:0/492255507 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:34:40.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:40.057+0000 7fd0de59c640 1 -- 172.21.15.38:0/492255507 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd0e0073d50 msgr2=0x7fd0e014bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:40.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:40.057+0000 7fd0de59c640 1 --2- 172.21.15.38:0/492255507 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd0e0073d50 0x7fd0e014bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:40.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:40.057+0000 7fd0de59c640 1 -- 172.21.15.38:0/492255507 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd0e00d0b40 msgr2=0x7fd0e014c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:40.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:40.057+0000 7fd0de59c640 1 --2- 172.21.15.38:0/492255507 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd0e00d0b40 0x7fd0e014c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:40.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:40.057+0000 7fd0de59c640 1 -- 172.21.15.38:0/492255507 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0e014acc0 con 0x7fd0e006ec30 2024-09-21T03:34:40.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:40.058+0000 7fd0de59c640 1 --2- 172.21.15.38:0/492255507 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd0e006ec30 0x7fd0e014a780 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fd0d000e7f0 tx=0x7fd0d000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:40.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:40.058+0000 7fd0d6ffd640 1 -- 172.21.15.38:0/492255507 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0d000ccc0 con 0x7fd0e006ec30 2024-09-21T03:34:40.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:40.058+0000 7fd0d6ffd640 1 -- 172.21.15.38:0/492255507 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd0d000ce80 con 0x7fd0e006ec30 2024-09-21T03:34:40.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:40.058+0000 7fd0e4913640 1 -- 172.21.15.38:0/492255507 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0e00cb430 con 0x7fd0e006ec30 2024-09-21T03:34:40.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:40.058+0000 7fd0d6ffd640 1 -- 172.21.15.38:0/492255507 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0d0005280 con 0x7fd0e006ec30 2024-09-21T03:34:40.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:40.058+0000 7fd0e4913640 1 -- 172.21.15.38:0/492255507 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0e014b3b0 con 0x7fd0e006ec30 2024-09-21T03:34:40.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:40.059+0000 7fd0e4913640 1 -- 172.21.15.38:0/492255507 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0e015bda0 con 0x7fd0e006ec30 2024-09-21T03:34:40.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:40.060+0000 7fd0d6ffd640 1 -- 172.21.15.38:0/492255507 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fd0d0005420 con 0x7fd0e006ec30 2024-09-21T03:34:40.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:40.060+0000 7fd0d6ffd640 1 --2- 172.21.15.38:0/492255507 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd0c0078bc0 0x7fd0c007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:40.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:40.060+0000 7fd0d6ffd640 1 -- 172.21.15.38:0/492255507 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7fd0d009b870 con 0x7fd0e006ec30 2024-09-21T03:34:40.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:40.061+0000 7fd0d7fff640 1 --2- 172.21.15.38:0/492255507 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd0c0078bc0 0x7fd0c007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:40.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:40.061+0000 7fd0d7fff640 1 --2- 172.21.15.38:0/492255507 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd0c0078bc0 0x7fd0c007b0d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd0c8000f60 tx=0x7fd0c80079b0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:40.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:40.065+0000 7fd0d6ffd640 1 -- 172.21.15.38:0/492255507 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fd0e014acc0 con 0x7fd0e006ec30 2024-09-21T03:34:40.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:40.249+0000 7fd0e4913640 1 -- 172.21.15.38:0/492255507 --> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd0e00cb710 con 0x7fd0c0078bc0 2024-09-21T03:34:40.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:40.254+0000 7fd0d6ffd640 1 -- 172.21.15.38:0/492255507 <== mgr.4117 v2:172.21.15.195:6800/1631901344 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+249387 (secure 0 0 0) 0x7fd0e00cb710 con 0x7fd0c0078bc0 2024-09-21T03:34:40.258 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:34:40.259 INFO:teuthology.orchestra.run.smithi038.stderr:dumped all 2024-09-21T03:34:40.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:40.257+0000 7fd0e4913640 1 -- 172.21.15.38:0/492255507 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd0c0078bc0 msgr2=0x7fd0c007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:40.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:40.257+0000 7fd0e4913640 1 --2- 172.21.15.38:0/492255507 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd0c0078bc0 0x7fd0c007b0d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd0c8000f60 tx=0x7fd0c80079b0 comp rx=0 tx=0).stop 2024-09-21T03:34:40.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:40.257+0000 7fd0e4913640 1 -- 172.21.15.38:0/492255507 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd0e006ec30 msgr2=0x7fd0e014a780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:40.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:40.257+0000 7fd0e4913640 1 --2- 172.21.15.38:0/492255507 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd0e006ec30 0x7fd0e014a780 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fd0d000e7f0 tx=0x7fd0d000ecb0 comp rx=0 tx=0).stop 2024-09-21T03:34:40.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:40.257+0000 7fd0e4913640 1 -- 172.21.15.38:0/492255507 shutdown_connections 2024-09-21T03:34:40.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:40.257+0000 7fd0e4913640 1 --2- 172.21.15.38:0/492255507 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd0e00d0b40 0x7fd0e014c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:40.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:40.257+0000 7fd0e4913640 1 --2- 172.21.15.38:0/492255507 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd0c0078bc0 0x7fd0c007b0d0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:40.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:40.257+0000 7fd0e4913640 1 --2- 172.21.15.38:0/492255507 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd0e0073d50 0x7fd0e014bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:40.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:40.257+0000 7fd0e4913640 1 --2- 172.21.15.38:0/492255507 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd0e006ec30 0x7fd0e014a780 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:40.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:40.257+0000 7fd0e4913640 1 -- 172.21.15.38:0/492255507 >> 172.21.15.38:0/492255507 conn(0x7fd0e00b2cc0 msgr2=0x7fd0e015ac50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:40.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:40.257+0000 7fd0e4913640 1 -- 172.21.15.38:0/492255507 shutdown_connections 2024-09-21T03:34:40.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:40.257+0000 7fd0e4913640 1 -- 172.21.15.38:0/492255507 wait complete. 2024-09-21T03:34:40.270 INFO:teuthology.orchestra.run.smithi038.stdout:{"pg_ready":true,"pg_map":{"version":71,"stamp":"2024-09-21T03:34:38.993186+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592036,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":115,"num_write_kb":1639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":87,"ondisk_log_size":87,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":76,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":254116,"kb_used_data":4588,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754720604,"statfs":{"total":773094113280,"available":772833898496,"internally_reserved":0,"allocated":4698112,"data_stored":2665849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12748,"internal_metadata":255315508},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1668,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001614"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:35.628233+0000","last_change":"2024-09-21T03:34:34.631062+0000","last_active":"2024-09-21T03:34:35.628233+0000","last_peered":"2024-09-21T03:34:35.628233+0000","last_clean":"2024-09-21T03:34:35.628233+0000","last_became_active":"2024-09-21T03:34:34.630468+0000","last_became_peered":"2024-09-21T03:34:34.630468+0000","last_unstale":"2024-09-21T03:34:35.628233+0000","last_undegraded":"2024-09-21T03:34:35.628233+0000","last_fullsized":"2024-09-21T03:34:35.628233+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:24:06.812940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:36.646828+0000","last_change":"2024-09-21T03:34:34.626744+0000","last_active":"2024-09-21T03:34:36.646828+0000","last_peered":"2024-09-21T03:34:36.646828+0000","last_clean":"2024-09-21T03:34:36.646828+0000","last_became_active":"2024-09-21T03:34:34.626445+0000","last_became_peered":"2024-09-21T03:34:34.626445+0000","last_unstale":"2024-09-21T03:34:36.646828+0000","last_undegraded":"2024-09-21T03:34:36.646828+0000","last_fullsized":"2024-09-21T03:34:36.646828+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:28:02.542581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:35.628793+0000","last_change":"2024-09-21T03:34:34.636439+0000","last_active":"2024-09-21T03:34:35.628793+0000","last_peered":"2024-09-21T03:34:35.628793+0000","last_clean":"2024-09-21T03:34:35.628793+0000","last_became_active":"2024-09-21T03:34:34.635938+0000","last_became_peered":"2024-09-21T03:34:34.635938+0000","last_unstale":"2024-09-21T03:34:35.628793+0000","last_undegraded":"2024-09-21T03:34:35.628793+0000","last_fullsized":"2024-09-21T03:34:35.628793+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:13:58.235543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:35.628343+0000","last_change":"2024-09-21T03:34:34.634523+0000","last_active":"2024-09-21T03:34:35.628343+0000","last_peered":"2024-09-21T03:34:35.628343+0000","last_clean":"2024-09-21T03:34:35.628343+0000","last_became_active":"2024-09-21T03:34:34.633777+0000","last_became_peered":"2024-09-21T03:34:34.633777+0000","last_unstale":"2024-09-21T03:34:35.628343+0000","last_undegraded":"2024-09-21T03:34:35.628343+0000","last_fullsized":"2024-09-21T03:34:35.628343+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:18:03.263504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-21T03:34:33.614399+0000","last_change":"2024-09-21T03:34:33.614399+0000","last_active":"2024-09-21T03:34:33.604595+0000","last_peered":"2024-09-21T03:34:33.604595+0000","last_clean":"2024-09-21T03:34:33.604595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:34:33.614399+0000","last_undegraded":"2024-09-21T03:34:33.614399+0000","last_fullsized":"2024-09-21T03:34:33.614399+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:36.646409+0000","last_change":"2024-09-21T03:34:34.634322+0000","last_active":"2024-09-21T03:34:36.646409+0000","last_peered":"2024-09-21T03:34:36.646409+0000","last_clean":"2024-09-21T03:34:36.646409+0000","last_became_active":"2024-09-21T03:34:34.634023+0000","last_became_peered":"2024-09-21T03:34:34.634023+0000","last_unstale":"2024-09-21T03:34:36.646409+0000","last_undegraded":"2024-09-21T03:34:36.646409+0000","last_fullsized":"2024-09-21T03:34:36.646409+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:32:33.650331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-21T03:34:34.638247+0000","last_change":"2024-09-21T03:34:34.629606+0000","last_active":"2024-09-21T03:34:34.638247+0000","last_peered":"2024-09-21T03:34:34.638247+0000","last_clean":"2024-09-21T03:34:34.638247+0000","last_became_active":"2024-09-21T03:34:34.628992+0000","last_became_peered":"2024-09-21T03:34:34.628992+0000","last_unstale":"2024-09-21T03:34:34.638247+0000","last_undegraded":"2024-09-21T03:34:34.638247+0000","last_fullsized":"2024-09-21T03:34:34.638247+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:34:28.501979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:35.627528+0000","last_change":"2024-09-21T03:34:34.637695+0000","last_active":"2024-09-21T03:34:35.627528+0000","last_peered":"2024-09-21T03:34:35.627528+0000","last_clean":"2024-09-21T03:34:35.627528+0000","last_became_active":"2024-09-21T03:34:34.637320+0000","last_became_peered":"2024-09-21T03:34:34.637320+0000","last_unstale":"2024-09-21T03:34:35.627528+0000","last_undegraded":"2024-09-21T03:34:35.627528+0000","last_fullsized":"2024-09-21T03:34:35.627528+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:20:18.794303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-21T03:34:33.614285+0000","last_change":"2024-09-21T03:34:33.614285+0000","last_active":"2024-09-21T03:34:33.604595+0000","last_peered":"2024-09-21T03:34:33.604595+0000","last_clean":"2024-09-21T03:34:33.604595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:34:33.614285+0000","last_undegraded":"2024-09-21T03:34:33.614285+0000","last_fullsized":"2024-09-21T03:34:33.614285+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-21T03:34:34.642546+0000","last_change":"2024-09-21T03:34:34.637080+0000","last_active":"2024-09-21T03:34:34.642546+0000","last_peered":"2024-09-21T03:34:34.642546+0000","last_clean":"2024-09-21T03:34:34.642546+0000","last_became_active":"2024-09-21T03:34:34.636777+0000","last_became_peered":"2024-09-21T03:34:34.636777+0000","last_unstale":"2024-09-21T03:34:34.642546+0000","last_undegraded":"2024-09-21T03:34:34.642546+0000","last_fullsized":"2024-09-21T03:34:34.642546+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:03:32.166555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-21T03:34:33.622983+0000","last_change":"2024-09-21T03:34:33.622983+0000","last_active":"2024-09-21T03:34:33.604595+0000","last_peered":"2024-09-21T03:34:33.604595+0000","last_clean":"2024-09-21T03:34:33.604595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:34:33.622983+0000","last_undegraded":"2024-09-21T03:34:33.622983+0000","last_fullsized":"2024-09-21T03:34:33.622983+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-21T03:34:33.614478+0000","last_change":"2024-09-21T03:34:33.614478+0000","last_active":"2024-09-21T03:34:33.604595+0000","last_peered":"2024-09-21T03:34:33.604595+0000","last_clean":"2024-09-21T03:34:33.604595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:34:33.614478+0000","last_undegraded":"2024-09-21T03:34:33.614478+0000","last_fullsized":"2024-09-21T03:34:33.614478+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-21T03:34:34.642073+0000","last_change":"2024-09-21T03:34:34.629186+0000","last_active":"2024-09-21T03:34:34.642073+0000","last_peered":"2024-09-21T03:34:34.642073+0000","last_clean":"2024-09-21T03:34:34.642073+0000","last_became_active":"2024-09-21T03:34:34.628801+0000","last_became_peered":"2024-09-21T03:34:34.628801+0000","last_unstale":"2024-09-21T03:34:34.642073+0000","last_undegraded":"2024-09-21T03:34:34.642073+0000","last_fullsized":"2024-09-21T03:34:34.642073+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:34:03.702144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-21T03:34:33.616107+0000","last_change":"2024-09-21T03:34:33.616107+0000","last_active":"2024-09-21T03:34:33.604595+0000","last_peered":"2024-09-21T03:34:33.604595+0000","last_clean":"2024-09-21T03:34:33.604595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:34:33.616107+0000","last_undegraded":"2024-09-21T03:34:33.616107+0000","last_fullsized":"2024-09-21T03:34:33.616107+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-21T03:34:33.625487+0000","last_change":"2024-09-21T03:34:33.625487+0000","last_active":"2024-09-21T03:34:33.604595+0000","last_peered":"2024-09-21T03:34:33.604595+0000","last_clean":"2024-09-21T03:34:33.604595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:34:33.625487+0000","last_undegraded":"2024-09-21T03:34:33.625487+0000","last_fullsized":"2024-09-21T03:34:33.625487+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-21T03:34:34.644136+0000","last_change":"2024-09-21T03:34:34.635135+0000","last_active":"2024-09-21T03:34:34.644136+0000","last_peered":"2024-09-21T03:34:34.644136+0000","last_clean":"2024-09-21T03:34:34.644136+0000","last_became_active":"2024-09-21T03:34:34.634811+0000","last_became_peered":"2024-09-21T03:34:34.634811+0000","last_unstale":"2024-09-21T03:34:34.644136+0000","last_undegraded":"2024-09-21T03:34:34.644136+0000","last_fullsized":"2024-09-21T03:34:34.644136+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:44:10.495963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-21T03:34:33.624716+0000","last_change":"2024-09-21T03:34:33.624716+0000","last_active":"2024-09-21T03:34:33.604595+0000","last_peered":"2024-09-21T03:34:33.604595+0000","last_clean":"2024-09-21T03:34:33.604595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:34:33.624716+0000","last_undegraded":"2024-09-21T03:34:33.624716+0000","last_fullsized":"2024-09-21T03:34:33.624716+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-21T03:34:33.623146+0000","last_change":"2024-09-21T03:34:33.623146+0000","last_active":"2024-09-21T03:34:33.604595+0000","last_peered":"2024-09-21T03:34:33.604595+0000","last_clean":"2024-09-21T03:34:33.604595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:34:33.623146+0000","last_undegraded":"2024-09-21T03:34:33.623146+0000","last_fullsized":"2024-09-21T03:34:33.623146+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-21T03:34:34.639249+0000","last_change":"2024-09-21T03:34:34.637926+0000","last_active":"2024-09-21T03:34:34.639249+0000","last_peered":"2024-09-21T03:34:34.639249+0000","last_clean":"2024-09-21T03:34:34.639249+0000","last_became_active":"2024-09-21T03:34:34.637546+0000","last_became_peered":"2024-09-21T03:34:34.637546+0000","last_unstale":"2024-09-21T03:34:34.639249+0000","last_undegraded":"2024-09-21T03:34:34.639249+0000","last_fullsized":"2024-09-21T03:34:34.639249+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:39:27.102702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-21T03:34:33.620475+0000","last_change":"2024-09-21T03:34:33.620475+0000","last_active":"2024-09-21T03:34:33.604595+0000","last_peered":"2024-09-21T03:34:33.604595+0000","last_clean":"2024-09-21T03:34:33.604595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:34:33.620475+0000","last_undegraded":"2024-09-21T03:34:33.620475+0000","last_fullsized":"2024-09-21T03:34:33.620475+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-21T03:34:33.621111+0000","last_change":"2024-09-21T03:34:33.621111+0000","last_active":"2024-09-21T03:34:33.604595+0000","last_peered":"2024-09-21T03:34:33.604595+0000","last_clean":"2024-09-21T03:34:33.604595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:34:33.621111+0000","last_undegraded":"2024-09-21T03:34:33.621111+0000","last_fullsized":"2024-09-21T03:34:33.621111+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-21T03:34:33.621590+0000","last_change":"2024-09-21T03:34:33.621590+0000","last_active":"2024-09-21T03:34:33.604595+0000","last_peered":"2024-09-21T03:34:33.604595+0000","last_clean":"2024-09-21T03:34:33.604595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:34:33.621590+0000","last_undegraded":"2024-09-21T03:34:33.621590+0000","last_fullsized":"2024-09-21T03:34:33.621590+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-21T03:34:33.624872+0000","last_change":"2024-09-21T03:34:33.624872+0000","last_active":"2024-09-21T03:34:33.604595+0000","last_peered":"2024-09-21T03:34:33.604595+0000","last_clean":"2024-09-21T03:34:33.604595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:34:33.624872+0000","last_undegraded":"2024-09-21T03:34:33.624872+0000","last_fullsized":"2024-09-21T03:34:33.624872+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:35.627534+0000","last_change":"2024-09-21T03:34:34.641334+0000","last_active":"2024-09-21T03:34:35.627534+0000","last_peered":"2024-09-21T03:34:35.627534+0000","last_clean":"2024-09-21T03:34:35.627534+0000","last_became_active":"2024-09-21T03:34:34.640922+0000","last_became_peered":"2024-09-21T03:34:34.640922+0000","last_unstale":"2024-09-21T03:34:35.627534+0000","last_undegraded":"2024-09-21T03:34:35.627534+0000","last_fullsized":"2024-09-21T03:34:35.627534+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:12:37.794061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-21T03:34:33.616131+0000","last_change":"2024-09-21T03:34:33.616131+0000","last_active":"2024-09-21T03:34:33.604595+0000","last_peered":"2024-09-21T03:34:33.604595+0000","last_clean":"2024-09-21T03:34:33.604595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:34:33.616131+0000","last_undegraded":"2024-09-21T03:34:33.616131+0000","last_fullsized":"2024-09-21T03:34:33.616131+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:35.628400+0000","last_change":"2024-09-21T03:34:34.640146+0000","last_active":"2024-09-21T03:34:35.628400+0000","last_peered":"2024-09-21T03:34:35.628400+0000","last_clean":"2024-09-21T03:34:35.628400+0000","last_became_active":"2024-09-21T03:34:34.639791+0000","last_became_peered":"2024-09-21T03:34:34.639791+0000","last_unstale":"2024-09-21T03:34:35.628400+0000","last_undegraded":"2024-09-21T03:34:35.628400+0000","last_fullsized":"2024-09-21T03:34:35.628400+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:23:41.788750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-21T03:34:33.622462+0000","last_change":"2024-09-21T03:34:33.622462+0000","last_active":"2024-09-21T03:34:33.604595+0000","last_peered":"2024-09-21T03:34:33.604595+0000","last_clean":"2024-09-21T03:34:33.604595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:34:33.622462+0000","last_undegraded":"2024-09-21T03:34:33.622462+0000","last_fullsized":"2024-09-21T03:34:33.622462+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-21T03:34:34.639470+0000","last_change":"2024-09-21T03:34:34.633013+0000","last_active":"2024-09-21T03:34:34.639470+0000","last_peered":"2024-09-21T03:34:34.639470+0000","last_clean":"2024-09-21T03:34:34.639470+0000","last_became_active":"2024-09-21T03:34:34.631701+0000","last_became_peered":"2024-09-21T03:34:34.631701+0000","last_unstale":"2024-09-21T03:34:34.639470+0000","last_undegraded":"2024-09-21T03:34:34.639470+0000","last_fullsized":"2024-09-21T03:34:34.639470+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:58:27.086990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-21T03:34:33.628158+0000","last_change":"2024-09-21T03:34:33.628158+0000","last_active":"2024-09-21T03:34:33.604595+0000","last_peered":"2024-09-21T03:34:33.604595+0000","last_clean":"2024-09-21T03:34:33.604595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:34:33.628158+0000","last_undegraded":"2024-09-21T03:34:33.628158+0000","last_fullsized":"2024-09-21T03:34:33.628158+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-21T03:34:33.621235+0000","last_change":"2024-09-21T03:34:33.621235+0000","last_active":"2024-09-21T03:34:33.604595+0000","last_peered":"2024-09-21T03:34:33.604595+0000","last_clean":"2024-09-21T03:34:33.604595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:34:33.621235+0000","last_undegraded":"2024-09-21T03:34:33.621235+0000","last_fullsized":"2024-09-21T03:34:33.621235+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-21T03:34:34.639102+0000","last_change":"2024-09-21T03:34:34.629415+0000","last_active":"2024-09-21T03:34:34.639102+0000","last_peered":"2024-09-21T03:34:34.639102+0000","last_clean":"2024-09-21T03:34:34.639102+0000","last_became_active":"2024-09-21T03:34:34.628524+0000","last_became_peered":"2024-09-21T03:34:34.628524+0000","last_unstale":"2024-09-21T03:34:34.639102+0000","last_undegraded":"2024-09-21T03:34:34.639102+0000","last_fullsized":"2024-09-21T03:34:34.639102+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:48:30.216711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:35.627792+0000","last_change":"2024-09-21T03:34:34.639100+0000","last_active":"2024-09-21T03:34:35.627792+0000","last_peered":"2024-09-21T03:34:35.627792+0000","last_clean":"2024-09-21T03:34:35.627792+0000","last_became_active":"2024-09-21T03:34:34.636285+0000","last_became_peered":"2024-09-21T03:34:34.636285+0000","last_unstale":"2024-09-21T03:34:35.627792+0000","last_undegraded":"2024-09-21T03:34:35.627792+0000","last_fullsized":"2024-09-21T03:34:35.627792+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:19:39.778069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-21T03:34:33.621969+0000","last_change":"2024-09-21T03:34:33.621969+0000","last_active":"2024-09-21T03:34:33.604595+0000","last_peered":"2024-09-21T03:34:33.604595+0000","last_clean":"2024-09-21T03:34:33.604595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:34:33.621969+0000","last_undegraded":"2024-09-21T03:34:33.621969+0000","last_fullsized":"2024-09-21T03:34:33.621969+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T03:34:33.620882+0000","last_change":"2024-09-21T03:34:33.618066+0000","last_active":"2024-09-21T03:34:33.620882+0000","last_peered":"2024-09-21T03:34:33.620882+0000","last_clean":"2024-09-21T03:34:33.620882+0000","last_became_active":"2024-09-21T03:34:33.617685+0000","last_became_peered":"2024-09-21T03:34:33.617685+0000","last_unstale":"2024-09-21T03:34:33.620882+0000","last_undegraded":"2024-09-21T03:34:33.620882+0000","last_fullsized":"2024-09-21T03:34:33.620882+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_clean_scrub_stamp":"2024-09-21T03:34:32.598217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:34:47.997359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:36.646764+0000","last_change":"2024-09-21T03:34:34.635480+0000","last_active":"2024-09-21T03:34:36.646764+0000","last_peered":"2024-09-21T03:34:36.646764+0000","last_clean":"2024-09-21T03:34:36.646764+0000","last_became_active":"2024-09-21T03:34:34.634979+0000","last_became_peered":"2024-09-21T03:34:34.634979+0000","last_unstale":"2024-09-21T03:34:36.646764+0000","last_undegraded":"2024-09-21T03:34:36.646764+0000","last_fullsized":"2024-09-21T03:34:36.646764+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:25:43.132670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"39'1","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:36.646633+0000","last_change":"2024-09-21T03:34:33.636543+0000","last_active":"2024-09-21T03:34:36.646633+0000","last_peered":"2024-09-21T03:34:36.646633+0000","last_clean":"2024-09-21T03:34:36.646633+0000","last_became_active":"2024-09-21T03:34:33.636151+0000","last_became_peered":"2024-09-21T03:34:33.636151+0000","last_unstale":"2024-09-21T03:34:36.646633+0000","last_undegraded":"2024-09-21T03:34:36.646633+0000","last_fullsized":"2024-09-21T03:34:36.646633+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_clean_scrub_stamp":"2024-09-21T03:34:32.598217+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:17:30.670560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-21T03:34:33.622277+0000","last_change":"2024-09-21T03:34:33.622277+0000","last_active":"2024-09-21T03:34:33.604595+0000","last_peered":"2024-09-21T03:34:33.604595+0000","last_clean":"2024-09-21T03:34:33.604595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:34:33.622277+0000","last_undegraded":"2024-09-21T03:34:33.622277+0000","last_fullsized":"2024-09-21T03:34:33.622277+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"39'2","reported_seq":17,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:35.628849+0000","last_change":"2024-09-21T03:34:33.629398+0000","last_active":"2024-09-21T03:34:35.628849+0000","last_peered":"2024-09-21T03:34:35.628849+0000","last_clean":"2024-09-21T03:34:35.628849+0000","last_became_active":"2024-09-21T03:34:33.629051+0000","last_became_peered":"2024-09-21T03:34:33.629051+0000","last_unstale":"2024-09-21T03:34:35.628849+0000","last_undegraded":"2024-09-21T03:34:35.628849+0000","last_fullsized":"2024-09-21T03:34:35.628849+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_clean_scrub_stamp":"2024-09-21T03:34:32.598217+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:46:38.739080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:36.646761+0000","last_change":"2024-09-21T03:34:34.629283+0000","last_active":"2024-09-21T03:34:36.646761+0000","last_peered":"2024-09-21T03:34:36.646761+0000","last_clean":"2024-09-21T03:34:36.646761+0000","last_became_active":"2024-09-21T03:34:34.628942+0000","last_became_peered":"2024-09-21T03:34:34.628942+0000","last_unstale":"2024-09-21T03:34:36.646761+0000","last_undegraded":"2024-09-21T03:34:36.646761+0000","last_fullsized":"2024-09-21T03:34:36.646761+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:15:48.821673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T03:34:33.638430+0000","last_change":"2024-09-21T03:34:33.631010+0000","last_active":"2024-09-21T03:34:33.638430+0000","last_peered":"2024-09-21T03:34:33.638430+0000","last_clean":"2024-09-21T03:34:33.638430+0000","last_became_active":"2024-09-21T03:34:33.630576+0000","last_became_peered":"2024-09-21T03:34:33.630576+0000","last_unstale":"2024-09-21T03:34:33.638430+0000","last_undegraded":"2024-09-21T03:34:33.638430+0000","last_fullsized":"2024-09-21T03:34:33.638430+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_clean_scrub_stamp":"2024-09-21T03:34:32.598217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:50:07.223556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:35.627813+0000","last_change":"2024-09-21T03:34:34.629518+0000","last_active":"2024-09-21T03:34:35.627813+0000","last_peered":"2024-09-21T03:34:35.627813+0000","last_clean":"2024-09-21T03:34:35.627813+0000","last_became_active":"2024-09-21T03:34:34.629216+0000","last_became_peered":"2024-09-21T03:34:34.629216+0000","last_unstale":"2024-09-21T03:34:35.627813+0000","last_undegraded":"2024-09-21T03:34:35.627813+0000","last_fullsized":"2024-09-21T03:34:35.627813+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:31:59.133888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T03:34:33.635417+0000","last_change":"2024-09-21T03:34:33.619056+0000","last_active":"2024-09-21T03:34:33.635417+0000","last_peered":"2024-09-21T03:34:33.635417+0000","last_clean":"2024-09-21T03:34:33.635417+0000","last_became_active":"2024-09-21T03:34:33.618698+0000","last_became_peered":"2024-09-21T03:34:33.618698+0000","last_unstale":"2024-09-21T03:34:33.635417+0000","last_undegraded":"2024-09-21T03:34:33.635417+0000","last_fullsized":"2024-09-21T03:34:33.635417+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_clean_scrub_stamp":"2024-09-21T03:34:32.598217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:58:31.223579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-21T03:34:33.618669+0000","last_change":"2024-09-21T03:34:33.618669+0000","last_active":"2024-09-21T03:34:33.604595+0000","last_peered":"2024-09-21T03:34:33.604595+0000","last_clean":"2024-09-21T03:34:33.604595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:34:33.618669+0000","last_undegraded":"2024-09-21T03:34:33.618669+0000","last_fullsized":"2024-09-21T03:34:33.618669+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1a","version":"39'1","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-21T03:34:34.633517+0000","last_change":"2024-09-21T03:34:33.634501+0000","last_active":"2024-09-21T03:34:34.633517+0000","last_peered":"2024-09-21T03:34:34.633517+0000","last_clean":"2024-09-21T03:34:34.633517+0000","last_became_active":"2024-09-21T03:34:33.633827+0000","last_became_peered":"2024-09-21T03:34:33.633827+0000","last_unstale":"2024-09-21T03:34:34.633517+0000","last_undegraded":"2024-09-21T03:34:34.633517+0000","last_fullsized":"2024-09-21T03:34:34.633517+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_clean_scrub_stamp":"2024-09-21T03:34:32.598217+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:33:52.483902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-21T03:34:33.620634+0000","last_change":"2024-09-21T03:34:33.620634+0000","last_active":"2024-09-21T03:34:33.604595+0000","last_peered":"2024-09-21T03:34:33.604595+0000","last_clean":"2024-09-21T03:34:33.604595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:34:33.620634+0000","last_undegraded":"2024-09-21T03:34:33.620634+0000","last_fullsized":"2024-09-21T03:34:33.620634+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:35.627409+0000","last_change":"2024-09-21T03:34:33.636797+0000","last_active":"2024-09-21T03:34:35.627409+0000","last_peered":"2024-09-21T03:34:35.627409+0000","last_clean":"2024-09-21T03:34:35.627409+0000","last_became_active":"2024-09-21T03:34:33.635927+0000","last_became_peered":"2024-09-21T03:34:33.635927+0000","last_unstale":"2024-09-21T03:34:35.627409+0000","last_undegraded":"2024-09-21T03:34:35.627409+0000","last_fullsized":"2024-09-21T03:34:35.627409+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_clean_scrub_stamp":"2024-09-21T03:34:32.598217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:23:52.412874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:36.646794+0000","last_change":"2024-09-21T03:34:34.640266+0000","last_active":"2024-09-21T03:34:36.646794+0000","last_peered":"2024-09-21T03:34:36.646794+0000","last_clean":"2024-09-21T03:34:36.646794+0000","last_became_active":"2024-09-21T03:34:34.639922+0000","last_became_peered":"2024-09-21T03:34:34.639922+0000","last_unstale":"2024-09-21T03:34:36.646794+0000","last_undegraded":"2024-09-21T03:34:36.646794+0000","last_fullsized":"2024-09-21T03:34:36.646794+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:38:18.167710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-21T03:34:34.615067+0000","last_change":"2024-09-21T03:34:33.638401+0000","last_active":"2024-09-21T03:34:34.615067+0000","last_peered":"2024-09-21T03:34:34.615067+0000","last_clean":"2024-09-21T03:34:34.615067+0000","last_became_active":"2024-09-21T03:34:33.637364+0000","last_became_peered":"2024-09-21T03:34:33.637364+0000","last_unstale":"2024-09-21T03:34:34.615067+0000","last_undegraded":"2024-09-21T03:34:34.615067+0000","last_fullsized":"2024-09-21T03:34:34.615067+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_clean_scrub_stamp":"2024-09-21T03:34:32.598217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:37:36.532174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:35.628541+0000","last_change":"2024-09-21T03:34:34.633927+0000","last_active":"2024-09-21T03:34:35.628541+0000","last_peered":"2024-09-21T03:34:35.628541+0000","last_clean":"2024-09-21T03:34:35.628541+0000","last_became_active":"2024-09-21T03:34:34.633294+0000","last_became_peered":"2024-09-21T03:34:34.633294+0000","last_unstale":"2024-09-21T03:34:35.628541+0000","last_undegraded":"2024-09-21T03:34:35.628541+0000","last_fullsized":"2024-09-21T03:34:35.628541+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:41:47.832177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:35.628042+0000","last_change":"2024-09-21T03:34:33.626093+0000","last_active":"2024-09-21T03:34:35.628042+0000","last_peered":"2024-09-21T03:34:35.628042+0000","last_clean":"2024-09-21T03:34:35.628042+0000","last_became_active":"2024-09-21T03:34:33.625795+0000","last_became_peered":"2024-09-21T03:34:33.625795+0000","last_unstale":"2024-09-21T03:34:35.628042+0000","last_undegraded":"2024-09-21T03:34:35.628042+0000","last_fullsized":"2024-09-21T03:34:35.628042+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_clean_scrub_stamp":"2024-09-21T03:34:32.598217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:39:10.947435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-21T03:34:33.616082+0000","last_change":"2024-09-21T03:34:33.616082+0000","last_active":"2024-09-21T03:34:33.604595+0000","last_peered":"2024-09-21T03:34:33.604595+0000","last_clean":"2024-09-21T03:34:33.604595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:34:33.616082+0000","last_undegraded":"2024-09-21T03:34:33.616082+0000","last_fullsized":"2024-09-21T03:34:33.616082+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T03:34:33.638093+0000","last_change":"2024-09-21T03:34:33.633888+0000","last_active":"2024-09-21T03:34:33.638093+0000","last_peered":"2024-09-21T03:34:33.638093+0000","last_clean":"2024-09-21T03:34:33.638093+0000","last_became_active":"2024-09-21T03:34:33.633372+0000","last_became_peered":"2024-09-21T03:34:33.633372+0000","last_unstale":"2024-09-21T03:34:33.638093+0000","last_undegraded":"2024-09-21T03:34:33.638093+0000","last_fullsized":"2024-09-21T03:34:33.638093+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_clean_scrub_stamp":"2024-09-21T03:34:32.598217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:58:21.314833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:35.628444+0000","last_change":"2024-09-21T03:34:34.644095+0000","last_active":"2024-09-21T03:34:35.628444+0000","last_peered":"2024-09-21T03:34:35.628444+0000","last_clean":"2024-09-21T03:34:35.628444+0000","last_became_active":"2024-09-21T03:34:34.643713+0000","last_became_peered":"2024-09-21T03:34:34.643713+0000","last_unstale":"2024-09-21T03:34:35.628444+0000","last_undegraded":"2024-09-21T03:34:35.628444+0000","last_fullsized":"2024-09-21T03:34:35.628444+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:40:54.924468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T03:34:33.638035+0000","last_change":"2024-09-21T03:34:33.632451+0000","last_active":"2024-09-21T03:34:33.638035+0000","last_peered":"2024-09-21T03:34:33.638035+0000","last_clean":"2024-09-21T03:34:33.638035+0000","last_became_active":"2024-09-21T03:34:33.632103+0000","last_became_peered":"2024-09-21T03:34:33.632103+0000","last_unstale":"2024-09-21T03:34:33.638035+0000","last_undegraded":"2024-09-21T03:34:33.638035+0000","last_fullsized":"2024-09-21T03:34:33.638035+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_clean_scrub_stamp":"2024-09-21T03:34:32.598217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:22:08.770678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-21T03:34:34.639059+0000","last_change":"2024-09-21T03:34:34.628419+0000","last_active":"2024-09-21T03:34:34.639059+0000","last_peered":"2024-09-21T03:34:34.639059+0000","last_clean":"2024-09-21T03:34:34.639059+0000","last_became_active":"2024-09-21T03:34:34.628087+0000","last_became_peered":"2024-09-21T03:34:34.628087+0000","last_unstale":"2024-09-21T03:34:34.639059+0000","last_undegraded":"2024-09-21T03:34:34.639059+0000","last_fullsized":"2024-09-21T03:34:34.639059+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:28:03.240497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.14","version":"39'6","reported_seq":21,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:35.628508+0000","last_change":"2024-09-21T03:34:33.639043+0000","last_active":"2024-09-21T03:34:35.628508+0000","last_peered":"2024-09-21T03:34:35.628508+0000","last_clean":"2024-09-21T03:34:35.628508+0000","last_became_active":"2024-09-21T03:34:33.638761+0000","last_became_peered":"2024-09-21T03:34:33.638761+0000","last_unstale":"2024-09-21T03:34:35.628508+0000","last_undegraded":"2024-09-21T03:34:35.628508+0000","last_fullsized":"2024-09-21T03:34:35.628508+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_clean_scrub_stamp":"2024-09-21T03:34:32.598217+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:44:53.992927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:35.628094+0000","last_change":"2024-09-21T03:34:34.640167+0000","last_active":"2024-09-21T03:34:35.628094+0000","last_peered":"2024-09-21T03:34:35.628094+0000","last_clean":"2024-09-21T03:34:35.628094+0000","last_became_active":"2024-09-21T03:34:34.639859+0000","last_became_peered":"2024-09-21T03:34:34.639859+0000","last_unstale":"2024-09-21T03:34:35.628094+0000","last_undegraded":"2024-09-21T03:34:35.628094+0000","last_fullsized":"2024-09-21T03:34:35.628094+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:36:48.213957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"39'2","reported_seq":17,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:35.627656+0000","last_change":"2024-09-21T03:34:33.628173+0000","last_active":"2024-09-21T03:34:35.627656+0000","last_peered":"2024-09-21T03:34:35.627656+0000","last_clean":"2024-09-21T03:34:35.627656+0000","last_became_active":"2024-09-21T03:34:33.627559+0000","last_became_peered":"2024-09-21T03:34:33.627559+0000","last_unstale":"2024-09-21T03:34:35.627656+0000","last_undegraded":"2024-09-21T03:34:35.627656+0000","last_fullsized":"2024-09-21T03:34:35.627656+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_clean_scrub_stamp":"2024-09-21T03:34:32.598217+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:34:22.474063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:36.646395+0000","last_change":"2024-09-21T03:34:34.626145+0000","last_active":"2024-09-21T03:34:36.646395+0000","last_peered":"2024-09-21T03:34:36.646395+0000","last_clean":"2024-09-21T03:34:36.646395+0000","last_became_active":"2024-09-21T03:34:34.625819+0000","last_became_peered":"2024-09-21T03:34:34.625819+0000","last_unstale":"2024-09-21T03:34:36.646395+0000","last_undegraded":"2024-09-21T03:34:36.646395+0000","last_fullsized":"2024-09-21T03:34:36.646395+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:44:09.663791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:36.646276+0000","last_change":"2024-09-21T03:34:33.630029+0000","last_active":"2024-09-21T03:34:36.646276+0000","last_peered":"2024-09-21T03:34:36.646276+0000","last_clean":"2024-09-21T03:34:36.646276+0000","last_became_active":"2024-09-21T03:34:33.629699+0000","last_became_peered":"2024-09-21T03:34:33.629699+0000","last_unstale":"2024-09-21T03:34:36.646276+0000","last_undegraded":"2024-09-21T03:34:36.646276+0000","last_fullsized":"2024-09-21T03:34:36.646276+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_clean_scrub_stamp":"2024-09-21T03:34:32.598217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:49:16.438096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-21T03:34:33.619516+0000","last_change":"2024-09-21T03:34:33.619516+0000","last_active":"2024-09-21T03:34:33.604595+0000","last_peered":"2024-09-21T03:34:33.604595+0000","last_clean":"2024-09-21T03:34:33.604595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:34:33.619516+0000","last_undegraded":"2024-09-21T03:34:33.619516+0000","last_fullsized":"2024-09-21T03:34:33.619516+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T03:34:33.644077+0000","last_change":"2024-09-21T03:34:33.639290+0000","last_active":"2024-09-21T03:34:33.644077+0000","last_peered":"2024-09-21T03:34:33.644077+0000","last_clean":"2024-09-21T03:34:33.644077+0000","last_became_active":"2024-09-21T03:34:33.638941+0000","last_became_peered":"2024-09-21T03:34:33.638941+0000","last_unstale":"2024-09-21T03:34:33.644077+0000","last_undegraded":"2024-09-21T03:34:33.644077+0000","last_fullsized":"2024-09-21T03:34:33.644077+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_clean_scrub_stamp":"2024-09-21T03:34:32.598217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:46:42.303637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-21T03:34:33.613436+0000","last_change":"2024-09-21T03:34:33.613436+0000","last_active":"2024-09-21T03:34:33.604595+0000","last_peered":"2024-09-21T03:34:33.604595+0000","last_clean":"2024-09-21T03:34:33.604595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:34:33.613436+0000","last_undegraded":"2024-09-21T03:34:33.613436+0000","last_fullsized":"2024-09-21T03:34:33.613436+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T03:34:33.638439+0000","last_change":"2024-09-21T03:34:33.632101+0000","last_active":"2024-09-21T03:34:33.638439+0000","last_peered":"2024-09-21T03:34:33.638439+0000","last_clean":"2024-09-21T03:34:33.638439+0000","last_became_active":"2024-09-21T03:34:33.631744+0000","last_became_peered":"2024-09-21T03:34:33.631744+0000","last_unstale":"2024-09-21T03:34:33.638439+0000","last_undegraded":"2024-09-21T03:34:33.638439+0000","last_fullsized":"2024-09-21T03:34:33.638439+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_clean_scrub_stamp":"2024-09-21T03:34:32.598217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:17:30.670560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:36.646635+0000","last_change":"2024-09-21T03:34:34.626189+0000","last_active":"2024-09-21T03:34:36.646635+0000","last_peered":"2024-09-21T03:34:36.646635+0000","last_clean":"2024-09-21T03:34:36.646635+0000","last_became_active":"2024-09-21T03:34:34.625865+0000","last_became_peered":"2024-09-21T03:34:34.625865+0000","last_unstale":"2024-09-21T03:34:36.646635+0000","last_undegraded":"2024-09-21T03:34:36.646635+0000","last_fullsized":"2024-09-21T03:34:36.646635+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:13:44.244299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.f","version":"39'1","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-21T03:34:34.638168+0000","last_change":"2024-09-21T03:34:33.637659+0000","last_active":"2024-09-21T03:34:34.638168+0000","last_peered":"2024-09-21T03:34:34.638168+0000","last_clean":"2024-09-21T03:34:34.638168+0000","last_became_active":"2024-09-21T03:34:33.637317+0000","last_became_peered":"2024-09-21T03:34:33.637317+0000","last_unstale":"2024-09-21T03:34:34.638168+0000","last_undegraded":"2024-09-21T03:34:34.638168+0000","last_fullsized":"2024-09-21T03:34:34.638168+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_clean_scrub_stamp":"2024-09-21T03:34:32.598217+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:12:23.510532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-21T03:34:34.642113+0000","last_change":"2024-09-21T03:34:34.634278+0000","last_active":"2024-09-21T03:34:34.642113+0000","last_peered":"2024-09-21T03:34:34.642113+0000","last_clean":"2024-09-21T03:34:34.642113+0000","last_became_active":"2024-09-21T03:34:34.633950+0000","last_became_peered":"2024-09-21T03:34:34.633950+0000","last_unstale":"2024-09-21T03:34:34.642113+0000","last_undegraded":"2024-09-21T03:34:34.642113+0000","last_fullsized":"2024-09-21T03:34:34.642113+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:43:22.066515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T03:34:33.638123+0000","last_change":"2024-09-21T03:34:33.634743+0000","last_active":"2024-09-21T03:34:33.638123+0000","last_peered":"2024-09-21T03:34:33.638123+0000","last_clean":"2024-09-21T03:34:33.638123+0000","last_became_active":"2024-09-21T03:34:33.634259+0000","last_became_peered":"2024-09-21T03:34:33.634259+0000","last_unstale":"2024-09-21T03:34:33.638123+0000","last_undegraded":"2024-09-21T03:34:33.638123+0000","last_fullsized":"2024-09-21T03:34:33.638123+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_clean_scrub_stamp":"2024-09-21T03:34:32.598217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:14:11.700926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-21T03:34:34.637883+0000","last_change":"2024-09-21T03:34:34.628247+0000","last_active":"2024-09-21T03:34:34.637883+0000","last_peered":"2024-09-21T03:34:34.637883+0000","last_clean":"2024-09-21T03:34:34.637883+0000","last_became_active":"2024-09-21T03:34:34.627925+0000","last_became_peered":"2024-09-21T03:34:34.627925+0000","last_unstale":"2024-09-21T03:34:34.637883+0000","last_undegraded":"2024-09-21T03:34:34.637883+0000","last_fullsized":"2024-09-21T03:34:34.637883+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:26:59.051709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T03:34:33.636065+0000","last_change":"2024-09-21T03:34:33.635141+0000","last_active":"2024-09-21T03:34:33.636065+0000","last_peered":"2024-09-21T03:34:33.636065+0000","last_clean":"2024-09-21T03:34:33.636065+0000","last_became_active":"2024-09-21T03:34:33.634825+0000","last_became_peered":"2024-09-21T03:34:33.634825+0000","last_unstale":"2024-09-21T03:34:33.636065+0000","last_undegraded":"2024-09-21T03:34:33.636065+0000","last_fullsized":"2024-09-21T03:34:33.636065+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_clean_scrub_stamp":"2024-09-21T03:34:32.598217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:12:23.510532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-21T03:34:34.631249+0000","last_change":"2024-09-21T03:34:34.630174+0000","last_active":"2024-09-21T03:34:34.631249+0000","last_peered":"2024-09-21T03:34:34.631249+0000","last_clean":"2024-09-21T03:34:34.631249+0000","last_became_active":"2024-09-21T03:34:34.629879+0000","last_became_peered":"2024-09-21T03:34:34.629879+0000","last_unstale":"2024-09-21T03:34:34.631249+0000","last_undegraded":"2024-09-21T03:34:34.631249+0000","last_fullsized":"2024-09-21T03:34:34.631249+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:03:28.897356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-21T03:34:34.624860+0000","last_change":"2024-09-21T03:34:33.630648+0000","last_active":"2024-09-21T03:34:34.624860+0000","last_peered":"2024-09-21T03:34:34.624860+0000","last_clean":"2024-09-21T03:34:34.624860+0000","last_became_active":"2024-09-21T03:34:33.630288+0000","last_became_peered":"2024-09-21T03:34:33.630288+0000","last_unstale":"2024-09-21T03:34:34.624860+0000","last_undegraded":"2024-09-21T03:34:34.624860+0000","last_fullsized":"2024-09-21T03:34:34.624860+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_clean_scrub_stamp":"2024-09-21T03:34:32.598217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:03:25.007146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-21T03:34:33.620205+0000","last_change":"2024-09-21T03:34:33.620205+0000","last_active":"2024-09-21T03:34:33.604595+0000","last_peered":"2024-09-21T03:34:33.604595+0000","last_clean":"2024-09-21T03:34:33.604595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:34:33.620205+0000","last_undegraded":"2024-09-21T03:34:33.620205+0000","last_fullsized":"2024-09-21T03:34:33.620205+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"39'2","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-21T03:34:34.634012+0000","last_change":"2024-09-21T03:34:33.623824+0000","last_active":"2024-09-21T03:34:34.634012+0000","last_peered":"2024-09-21T03:34:34.634012+0000","last_clean":"2024-09-21T03:34:34.634012+0000","last_became_active":"2024-09-21T03:34:33.623465+0000","last_became_peered":"2024-09-21T03:34:33.623465+0000","last_unstale":"2024-09-21T03:34:34.634012+0000","last_undegraded":"2024-09-21T03:34:34.634012+0000","last_fullsized":"2024-09-21T03:34:34.634012+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_clean_scrub_stamp":"2024-09-21T03:34:32.598217+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:07:51.182124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:35.627438+0000","last_change":"2024-09-21T03:34:34.635616+0000","last_active":"2024-09-21T03:34:35.627438+0000","last_peered":"2024-09-21T03:34:35.627438+0000","last_clean":"2024-09-21T03:34:35.627438+0000","last_became_active":"2024-09-21T03:34:34.632739+0000","last_became_peered":"2024-09-21T03:34:34.632739+0000","last_unstale":"2024-09-21T03:34:35.627438+0000","last_undegraded":"2024-09-21T03:34:35.627438+0000","last_fullsized":"2024-09-21T03:34:35.627438+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:49:01.779676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:36.646422+0000","last_change":"2024-09-21T03:34:33.633593+0000","last_active":"2024-09-21T03:34:36.646422+0000","last_peered":"2024-09-21T03:34:36.646422+0000","last_clean":"2024-09-21T03:34:36.646422+0000","last_became_active":"2024-09-21T03:34:33.633254+0000","last_became_peered":"2024-09-21T03:34:33.633254+0000","last_unstale":"2024-09-21T03:34:36.646422+0000","last_undegraded":"2024-09-21T03:34:36.646422+0000","last_fullsized":"2024-09-21T03:34:36.646422+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_clean_scrub_stamp":"2024-09-21T03:34:32.598217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:57:01.944255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-21T03:34:33.621061+0000","last_change":"2024-09-21T03:34:33.621061+0000","last_active":"2024-09-21T03:34:33.604595+0000","last_peered":"2024-09-21T03:34:33.604595+0000","last_clean":"2024-09-21T03:34:33.604595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:34:33.621061+0000","last_undegraded":"2024-09-21T03:34:33.621061+0000","last_fullsized":"2024-09-21T03:34:33.621061+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-21T03:34:34.616134+0000","last_change":"2024-09-21T03:34:33.637630+0000","last_active":"2024-09-21T03:34:34.616134+0000","last_peered":"2024-09-21T03:34:34.616134+0000","last_clean":"2024-09-21T03:34:34.616134+0000","last_became_active":"2024-09-21T03:34:33.637290+0000","last_became_peered":"2024-09-21T03:34:33.637290+0000","last_unstale":"2024-09-21T03:34:34.616134+0000","last_undegraded":"2024-09-21T03:34:34.616134+0000","last_fullsized":"2024-09-21T03:34:34.616134+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_clean_scrub_stamp":"2024-09-21T03:34:32.598217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:00:46.535619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:35.627723+0000","last_change":"2024-09-21T03:34:34.644149+0000","last_active":"2024-09-21T03:34:35.627723+0000","last_peered":"2024-09-21T03:34:35.627723+0000","last_clean":"2024-09-21T03:34:35.627723+0000","last_became_active":"2024-09-21T03:34:34.643689+0000","last_became_peered":"2024-09-21T03:34:34.643689+0000","last_unstale":"2024-09-21T03:34:35.627723+0000","last_undegraded":"2024-09-21T03:34:35.627723+0000","last_fullsized":"2024-09-21T03:34:35.627723+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:51:48.433744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:36.646016+0000","last_change":"2024-09-21T03:34:33.626688+0000","last_active":"2024-09-21T03:34:36.646016+0000","last_peered":"2024-09-21T03:34:36.646016+0000","last_clean":"2024-09-21T03:34:36.646016+0000","last_became_active":"2024-09-21T03:34:33.626325+0000","last_became_peered":"2024-09-21T03:34:33.626325+0000","last_unstale":"2024-09-21T03:34:36.646016+0000","last_undegraded":"2024-09-21T03:34:36.646016+0000","last_fullsized":"2024-09-21T03:34:36.646016+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_clean_scrub_stamp":"2024-09-21T03:34:32.598217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:50:07.223556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-21T03:34:33.613904+0000","last_change":"2024-09-21T03:34:33.613904+0000","last_active":"2024-09-21T03:34:33.604595+0000","last_peered":"2024-09-21T03:34:33.604595+0000","last_clean":"2024-09-21T03:34:33.604595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:34:33.613904+0000","last_undegraded":"2024-09-21T03:34:33.613904+0000","last_fullsized":"2024-09-21T03:34:33.613904+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T03:34:33.638065+0000","last_change":"2024-09-21T03:34:33.633316+0000","last_active":"2024-09-21T03:34:33.638065+0000","last_peered":"2024-09-21T03:34:33.638065+0000","last_clean":"2024-09-21T03:34:33.638065+0000","last_became_active":"2024-09-21T03:34:33.632983+0000","last_became_peered":"2024-09-21T03:34:33.632983+0000","last_unstale":"2024-09-21T03:34:33.638065+0000","last_undegraded":"2024-09-21T03:34:33.638065+0000","last_fullsized":"2024-09-21T03:34:33.638065+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_clean_scrub_stamp":"2024-09-21T03:34:32.598217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:48:01.802931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-21T03:34:33.621976+0000","last_change":"2024-09-21T03:34:33.621976+0000","last_active":"2024-09-21T03:34:33.604595+0000","last_peered":"2024-09-21T03:34:33.604595+0000","last_clean":"2024-09-21T03:34:33.604595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:34:33.621976+0000","last_undegraded":"2024-09-21T03:34:33.621976+0000","last_fullsized":"2024-09-21T03:34:33.621976+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T03:34:33.632105+0000","last_change":"2024-09-21T03:34:33.630944+0000","last_active":"2024-09-21T03:34:33.632105+0000","last_peered":"2024-09-21T03:34:33.632105+0000","last_clean":"2024-09-21T03:34:33.632105+0000","last_became_active":"2024-09-21T03:34:33.630396+0000","last_became_peered":"2024-09-21T03:34:33.630396+0000","last_unstale":"2024-09-21T03:34:33.632105+0000","last_undegraded":"2024-09-21T03:34:33.632105+0000","last_fullsized":"2024-09-21T03:34:33.632105+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_clean_scrub_stamp":"2024-09-21T03:34:32.598217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:37:36.532174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-21T03:34:34.639837+0000","last_change":"2024-09-21T03:34:34.638755+0000","last_active":"2024-09-21T03:34:34.639837+0000","last_peered":"2024-09-21T03:34:34.639837+0000","last_clean":"2024-09-21T03:34:34.639837+0000","last_became_active":"2024-09-21T03:34:34.638432+0000","last_became_peered":"2024-09-21T03:34:34.638432+0000","last_unstale":"2024-09-21T03:34:34.639837+0000","last_undegraded":"2024-09-21T03:34:34.639837+0000","last_fullsized":"2024-09-21T03:34:34.639837+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:40:56.624795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T03:34:33.627809+0000","last_change":"2024-09-21T03:34:33.626093+0000","last_active":"2024-09-21T03:34:33.627809+0000","last_peered":"2024-09-21T03:34:33.627809+0000","last_clean":"2024-09-21T03:34:33.627809+0000","last_became_active":"2024-09-21T03:34:33.625771+0000","last_became_peered":"2024-09-21T03:34:33.625771+0000","last_unstale":"2024-09-21T03:34:33.627809+0000","last_undegraded":"2024-09-21T03:34:33.627809+0000","last_fullsized":"2024-09-21T03:34:33.627809+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_clean_scrub_stamp":"2024-09-21T03:34:32.598217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:05:08.028564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-21T03:34:33.613418+0000","last_change":"2024-09-21T03:34:33.613418+0000","last_active":"2024-09-21T03:34:33.604595+0000","last_peered":"2024-09-21T03:34:33.604595+0000","last_clean":"2024-09-21T03:34:33.604595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:34:33.613418+0000","last_undegraded":"2024-09-21T03:34:33.613418+0000","last_fullsized":"2024-09-21T03:34:33.613418+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:35.627816+0000","last_change":"2024-09-21T03:34:33.625164+0000","last_active":"2024-09-21T03:34:35.627816+0000","last_peered":"2024-09-21T03:34:35.627816+0000","last_clean":"2024-09-21T03:34:35.627816+0000","last_became_active":"2024-09-21T03:34:33.624299+0000","last_became_peered":"2024-09-21T03:34:33.624299+0000","last_unstale":"2024-09-21T03:34:35.627816+0000","last_undegraded":"2024-09-21T03:34:35.627816+0000","last_fullsized":"2024-09-21T03:34:35.627816+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_clean_scrub_stamp":"2024-09-21T03:34:32.598217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:32:32.643952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:36.646586+0000","last_change":"2024-09-21T03:34:34.637737+0000","last_active":"2024-09-21T03:34:36.646586+0000","last_peered":"2024-09-21T03:34:36.646586+0000","last_clean":"2024-09-21T03:34:36.646586+0000","last_became_active":"2024-09-21T03:34:34.637441+0000","last_became_peered":"2024-09-21T03:34:34.637441+0000","last_unstale":"2024-09-21T03:34:36.646586+0000","last_undegraded":"2024-09-21T03:34:36.646586+0000","last_fullsized":"2024-09-21T03:34:36.646586+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:12:23.136125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T03:34:33.636106+0000","last_change":"2024-09-21T03:34:33.631273+0000","last_active":"2024-09-21T03:34:33.636106+0000","last_peered":"2024-09-21T03:34:33.636106+0000","last_clean":"2024-09-21T03:34:33.636106+0000","last_became_active":"2024-09-21T03:34:33.628561+0000","last_became_peered":"2024-09-21T03:34:33.628561+0000","last_unstale":"2024-09-21T03:34:33.636106+0000","last_undegraded":"2024-09-21T03:34:33.636106+0000","last_fullsized":"2024-09-21T03:34:33.636106+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_clean_scrub_stamp":"2024-09-21T03:34:32.598217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:49:16.438096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-21T03:34:33.620026+0000","last_change":"2024-09-21T03:34:33.620026+0000","last_active":"2024-09-21T03:34:33.604595+0000","last_peered":"2024-09-21T03:34:33.604595+0000","last_clean":"2024-09-21T03:34:33.604595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:34:33.620026+0000","last_undegraded":"2024-09-21T03:34:33.620026+0000","last_fullsized":"2024-09-21T03:34:33.620026+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T03:34:33.638150+0000","last_change":"2024-09-21T03:34:33.633175+0000","last_active":"2024-09-21T03:34:33.638150+0000","last_peered":"2024-09-21T03:34:33.638150+0000","last_clean":"2024-09-21T03:34:33.638150+0000","last_became_active":"2024-09-21T03:34:33.632872+0000","last_became_peered":"2024-09-21T03:34:33.632872+0000","last_unstale":"2024-09-21T03:34:33.638150+0000","last_undegraded":"2024-09-21T03:34:33.638150+0000","last_fullsized":"2024-09-21T03:34:33.638150+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_clean_scrub_stamp":"2024-09-21T03:34:32.598217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:33:03.801458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-21T03:34:33.619685+0000","last_change":"2024-09-21T03:34:33.619685+0000","last_active":"2024-09-21T03:34:33.604595+0000","last_peered":"2024-09-21T03:34:33.604595+0000","last_clean":"2024-09-21T03:34:33.604595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:34:33.619685+0000","last_undegraded":"2024-09-21T03:34:33.619685+0000","last_fullsized":"2024-09-21T03:34:33.619685+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:35.628165+0000","last_change":"2024-09-21T03:34:33.639171+0000","last_active":"2024-09-21T03:34:35.628165+0000","last_peered":"2024-09-21T03:34:35.628165+0000","last_clean":"2024-09-21T03:34:35.628165+0000","last_became_active":"2024-09-21T03:34:33.638819+0000","last_became_peered":"2024-09-21T03:34:33.638819+0000","last_unstale":"2024-09-21T03:34:35.628165+0000","last_undegraded":"2024-09-21T03:34:35.628165+0000","last_fullsized":"2024-09-21T03:34:35.628165+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_clean_scrub_stamp":"2024-09-21T03:34:32.598217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:44:08.657412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-21T03:34:33.625230+0000","last_change":"2024-09-21T03:34:33.625230+0000","last_active":"2024-09-21T03:34:33.604595+0000","last_peered":"2024-09-21T03:34:33.604595+0000","last_clean":"2024-09-21T03:34:33.604595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:34:33.625230+0000","last_undegraded":"2024-09-21T03:34:33.625230+0000","last_fullsized":"2024-09-21T03:34:33.625230+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T03:34:33.637827+0000","last_change":"2024-09-21T03:34:33.635753+0000","last_active":"2024-09-21T03:34:33.637827+0000","last_peered":"2024-09-21T03:34:33.637827+0000","last_clean":"2024-09-21T03:34:33.637827+0000","last_became_active":"2024-09-21T03:34:33.631914+0000","last_became_peered":"2024-09-21T03:34:33.631914+0000","last_unstale":"2024-09-21T03:34:33.637827+0000","last_undegraded":"2024-09-21T03:34:33.637827+0000","last_fullsized":"2024-09-21T03:34:33.637827+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_clean_scrub_stamp":"2024-09-21T03:34:32.598217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:57:01.944255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":143,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-21T03:34:34.615053+0000","last_change":"2024-09-21T03:33:25.319776+0000","last_active":"2024-09-21T03:34:34.615053+0000","last_peered":"2024-09-21T03:34:34.615053+0000","last_clean":"2024-09-21T03:34:34.615053+0000","last_became_active":"2024-09-21T03:33:25.318399+0000","last_became_peered":"2024-09-21T03:33:25.318399+0000","last_unstale":"2024-09-21T03:34:34.615053+0000","last_undegraded":"2024-09-21T03:34:34.615053+0000","last_fullsized":"2024-09-21T03:34:34.615053+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:23.016672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:23.016672+0000","last_clean_scrub_stamp":"2024-09-21T03:33:23.016672+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:32:35.906114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":1668,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":15,"ondisk_log_size":15,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":1,"up_from":16,"seq":68719476754,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32212,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339628,"statfs":{"total":96636764160,"available":96603779072,"internally_reserved":0,"allocated":1040384,"data_stored":780143,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509458,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31596,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340244,"statfs":{"total":96636764160,"available":96604409856,"internally_reserved":0,"allocated":417792,"data_stored":175919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509457,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31612,"kb_used_data":420,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340228,"statfs":{"total":96636764160,"available":96604393472,"internally_reserved":0,"allocated":430080,"data_stored":182586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509458,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27436,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344404,"statfs":{"total":96636764160,"available":96608669696,"internally_reserved":0,"allocated":417792,"data_stored":175919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509457,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31596,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340244,"statfs":{"total":96636764160,"available":96604409856,"internally_reserved":0,"allocated":417792,"data_stored":175919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542162,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31640,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340200,"statfs":{"total":96636764160,"available":96604364800,"internally_reserved":0,"allocated":458752,"data_stored":196479,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574866,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32208,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339632,"statfs":{"total":96636764160,"available":96603783168,"internally_reserved":0,"allocated":1040384,"data_stored":780703,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542163,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35816,"kb_used_data":464,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336024,"statfs":{"total":96636764160,"available":96600088576,"internally_reserved":0,"allocated":475136,"data_stored":198181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T03:34:43.274 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T03:34:43.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:43.406+0000 7fb5c4177640 1 -- 172.21.15.38:0/3103918892 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb5a8004d40 msgr2=0x7fb5a8005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:43.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:43.406+0000 7fb5c4177640 1 --2- 172.21.15.38:0/3103918892 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb5a8004d40 0x7fb5a8005150 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fb5b4009ed0 tx=0x7fb5a8005830 comp rx=0 tx=0).stop 2024-09-21T03:34:43.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:43.406+0000 7fb5c2714640 1 -- 172.21.15.38:0/3103918892 reap_dead start 2024-09-21T03:34:43.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:43.406+0000 7fb5c4177640 1 -- 172.21.15.38:0/3103918892 shutdown_connections 2024-09-21T03:34:43.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:43.406+0000 7fb5c4177640 1 -- 172.21.15.38:0/3103918892 >> 172.21.15.38:0/3103918892 conn(0x7fb5bc0d4440 msgr2=0x7fb5bc0d4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:43.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:43.406+0000 7fb5c4177640 1 -- 172.21.15.38:0/3103918892 shutdown_connections 2024-09-21T03:34:43.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:43.406+0000 7fb5c4177640 1 -- 172.21.15.38:0/3103918892 wait complete. 2024-09-21T03:34:43.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:43.408+0000 7fb5c4177640 1 Processor -- start 2024-09-21T03:34:43.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:43.408+0000 7fb5c4177640 1 -- start start 2024-09-21T03:34:43.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:43.409+0000 7fb5c4177640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb5bc1589a0 0x7fb5bc15c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:43.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:43.409+0000 7fb5c4177640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb5bc160f70 0x7fb5bc161380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:43.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:43.409+0000 7fb5c4177640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb5bc14f380 0x7fb5bc14f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:43.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:43.409+0000 7fb5c4177640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fb5bc0b33f0 con 0x7fb5bc160f70 2024-09-21T03:34:43.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:43.409+0000 7fb5c4177640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fb5bc0b3270 con 0x7fb5bc1589a0 2024-09-21T03:34:43.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:43.409+0000 7fb5c4177640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fb5bc0b3570 con 0x7fb5bc14f380 2024-09-21T03:34:43.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:43.410+0000 7fb5c1712640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb5bc160f70 0x7fb5bc161380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:43.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:43.410+0000 7fb5c1712640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb5bc160f70 0x7fb5bc161380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50064/0 (socket says 172.21.15.38:50064) 2024-09-21T03:34:43.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:43.410+0000 7fb5c1712640 1 -- 172.21.15.38:0/1982725722 learned_addr learned my addr 172.21.15.38:0/1982725722 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:34:43.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:43.410+0000 7fb5c1f13640 1 --2- 172.21.15.38:0/1982725722 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb5bc1589a0 0x7fb5bc15c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:43.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:43.410+0000 7fb5c2714640 1 --2- 172.21.15.38:0/1982725722 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb5bc14f380 0x7fb5bc14f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:43.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:43.410+0000 7fb5c1712640 1 -- 172.21.15.38:0/1982725722 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb5bc14f380 msgr2=0x7fb5bc14f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:43.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:43.410+0000 7fb5c1712640 1 --2- 172.21.15.38:0/1982725722 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb5bc14f380 0x7fb5bc14f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:43.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:43.410+0000 7fb5c1712640 1 -- 172.21.15.38:0/1982725722 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb5bc1589a0 msgr2=0x7fb5bc15c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:43.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:43.410+0000 7fb5c1712640 1 --2- 172.21.15.38:0/1982725722 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb5bc1589a0 0x7fb5bc15c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:43.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:43.410+0000 7fb5c1712640 1 -- 172.21.15.38:0/1982725722 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5bc14fcd0 con 0x7fb5bc160f70 2024-09-21T03:34:43.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:43.410+0000 7fb5c1f13640 1 --2- 172.21.15.38:0/1982725722 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb5bc1589a0 0x7fb5bc15c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:34:43.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:43.410+0000 7fb5c1712640 1 --2- 172.21.15.38:0/1982725722 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb5bc160f70 0x7fb5bc161380 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fb5ac011bf0 tx=0x7fb5ac00f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:43.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:43.410+0000 7fb5b27fc640 1 -- 172.21.15.38:0/1982725722 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5ac01e070 con 0x7fb5bc160f70 2024-09-21T03:34:43.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:43.410+0000 7fb5b27fc640 1 -- 172.21.15.38:0/1982725722 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb5ac00e7c0 con 0x7fb5bc160f70 2024-09-21T03:34:43.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:43.410+0000 7fb5c4177640 1 -- 172.21.15.38:0/1982725722 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5bc1f8f00 con 0x7fb5bc160f70 2024-09-21T03:34:43.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:43.410+0000 7fb5b27fc640 1 -- 172.21.15.38:0/1982725722 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5ac00ebd0 con 0x7fb5bc160f70 2024-09-21T03:34:43.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:43.411+0000 7fb5c4177640 1 -- 172.21.15.38:0/1982725722 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5bc1f94c0 con 0x7fb5bc160f70 2024-09-21T03:34:43.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:43.411+0000 7fb5c4177640 1 -- 172.21.15.38:0/1982725722 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5bc0b3a90 con 0x7fb5bc160f70 2024-09-21T03:34:43.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:43.412+0000 7fb5b27fc640 1 -- 172.21.15.38:0/1982725722 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb5ac021070 con 0x7fb5bc160f70 2024-09-21T03:34:43.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:43.413+0000 7fb5b27fc640 1 --2- 172.21.15.38:0/1982725722 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb5a0078bc0 0x7fb5a007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:43.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:43.413+0000 7fb5b27fc640 1 -- 172.21.15.38:0/1982725722 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7fb5ac09bd40 con 0x7fb5bc160f70 2024-09-21T03:34:43.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:43.413+0000 7fb5c1f13640 1 --2- 172.21.15.38:0/1982725722 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb5a0078bc0 0x7fb5a007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:43.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:43.414+0000 7fb5c1f13640 1 --2- 172.21.15.38:0/1982725722 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb5a0078bc0 0x7fb5a007b0d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb5b8001330 tx=0x7fb5b8006950 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:43.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:43.417+0000 7fb5b27fc640 1 -- 172.21.15.38:0/1982725722 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fb5ac063ac0 con 0x7fb5bc160f70 2024-09-21T03:34:43.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:43.601+0000 7fb5c4177640 1 -- 172.21.15.38:0/1982725722 --> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb5bc0d4850 con 0x7fb5a0078bc0 2024-09-21T03:34:43.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:43.606+0000 7fb5b27fc640 1 -- 172.21.15.38:0/1982725722 <== mgr.4117 v2:172.21.15.195:6800/1631901344 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251946 (secure 0 0 0) 0x7fb5bc0d4850 con 0x7fb5a0078bc0 2024-09-21T03:34:43.610 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:34:43.611 INFO:teuthology.orchestra.run.smithi038.stderr:dumped all 2024-09-21T03:34:43.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:43.609+0000 7fb5c4177640 1 -- 172.21.15.38:0/1982725722 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb5a0078bc0 msgr2=0x7fb5a007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:43.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:43.609+0000 7fb5c4177640 1 --2- 172.21.15.38:0/1982725722 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb5a0078bc0 0x7fb5a007b0d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb5b8001330 tx=0x7fb5b8006950 comp rx=0 tx=0).stop 2024-09-21T03:34:43.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:43.609+0000 7fb5c4177640 1 -- 172.21.15.38:0/1982725722 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb5bc160f70 msgr2=0x7fb5bc161380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:43.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:43.609+0000 7fb5c4177640 1 --2- 172.21.15.38:0/1982725722 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb5bc160f70 0x7fb5bc161380 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fb5ac011bf0 tx=0x7fb5ac00f790 comp rx=0 tx=0).stop 2024-09-21T03:34:43.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:43.609+0000 7fb5c4177640 1 -- 172.21.15.38:0/1982725722 shutdown_connections 2024-09-21T03:34:43.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:43.609+0000 7fb5c4177640 1 --2- 172.21.15.38:0/1982725722 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb5a0078bc0 0x7fb5a007b0d0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:43.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:43.609+0000 7fb5c4177640 1 --2- 172.21.15.38:0/1982725722 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb5bc14f380 0x7fb5bc14f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:43.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:43.609+0000 7fb5c4177640 1 --2- 172.21.15.38:0/1982725722 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb5bc160f70 0x7fb5bc161380 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:43.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:43.609+0000 7fb5c4177640 1 --2- 172.21.15.38:0/1982725722 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb5bc1589a0 0x7fb5bc15c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:43.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:43.609+0000 7fb5c4177640 1 -- 172.21.15.38:0/1982725722 >> 172.21.15.38:0/1982725722 conn(0x7fb5bc0d4440 msgr2=0x7fb5bc0d0c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:43.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:43.609+0000 7fb5c4177640 1 -- 172.21.15.38:0/1982725722 shutdown_connections 2024-09-21T03:34:43.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:43.609+0000 7fb5c4177640 1 -- 172.21.15.38:0/1982725722 wait complete. 2024-09-21T03:34:43.622 INFO:teuthology.orchestra.run.smithi038.stdout:{"pg_ready":true,"pg_map":{"version":73,"stamp":"2024-09-21T03:34:42.994501+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":254296,"kb_used_data":4776,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754720424,"statfs":{"total":773094113280,"available":772833714176,"internally_reserved":0,"allocated":4890624,"data_stored":2757116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12748,"internal_metadata":255315508},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.381321"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:35.628233+0000","last_change":"2024-09-21T03:34:34.631062+0000","last_active":"2024-09-21T03:34:35.628233+0000","last_peered":"2024-09-21T03:34:35.628233+0000","last_clean":"2024-09-21T03:34:35.628233+0000","last_became_active":"2024-09-21T03:34:34.630468+0000","last_became_peered":"2024-09-21T03:34:34.630468+0000","last_unstale":"2024-09-21T03:34:35.628233+0000","last_undegraded":"2024-09-21T03:34:35.628233+0000","last_fullsized":"2024-09-21T03:34:35.628233+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:24:06.812940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:36.646828+0000","last_change":"2024-09-21T03:34:34.626744+0000","last_active":"2024-09-21T03:34:36.646828+0000","last_peered":"2024-09-21T03:34:36.646828+0000","last_clean":"2024-09-21T03:34:36.646828+0000","last_became_active":"2024-09-21T03:34:34.626445+0000","last_became_peered":"2024-09-21T03:34:34.626445+0000","last_unstale":"2024-09-21T03:34:36.646828+0000","last_undegraded":"2024-09-21T03:34:36.646828+0000","last_fullsized":"2024-09-21T03:34:36.646828+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:28:02.542581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:35.628793+0000","last_change":"2024-09-21T03:34:34.636439+0000","last_active":"2024-09-21T03:34:35.628793+0000","last_peered":"2024-09-21T03:34:35.628793+0000","last_clean":"2024-09-21T03:34:35.628793+0000","last_became_active":"2024-09-21T03:34:34.635938+0000","last_became_peered":"2024-09-21T03:34:34.635938+0000","last_unstale":"2024-09-21T03:34:35.628793+0000","last_undegraded":"2024-09-21T03:34:35.628793+0000","last_fullsized":"2024-09-21T03:34:35.628793+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:13:58.235543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:35.628343+0000","last_change":"2024-09-21T03:34:34.634523+0000","last_active":"2024-09-21T03:34:35.628343+0000","last_peered":"2024-09-21T03:34:35.628343+0000","last_clean":"2024-09-21T03:34:35.628343+0000","last_became_active":"2024-09-21T03:34:34.633777+0000","last_became_peered":"2024-09-21T03:34:34.633777+0000","last_unstale":"2024-09-21T03:34:35.628343+0000","last_undegraded":"2024-09-21T03:34:35.628343+0000","last_fullsized":"2024-09-21T03:34:35.628343+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:18:03.263504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:37.532422+0000","last_change":"2024-09-21T03:34:34.646883+0000","last_active":"2024-09-21T03:34:37.532422+0000","last_peered":"2024-09-21T03:34:37.532422+0000","last_clean":"2024-09-21T03:34:37.532422+0000","last_became_active":"2024-09-21T03:34:34.646529+0000","last_became_peered":"2024-09-21T03:34:34.646529+0000","last_unstale":"2024-09-21T03:34:37.532422+0000","last_undegraded":"2024-09-21T03:34:37.532422+0000","last_fullsized":"2024-09-21T03:34:37.532422+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:03:32.166555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:36.646409+0000","last_change":"2024-09-21T03:34:34.634322+0000","last_active":"2024-09-21T03:34:36.646409+0000","last_peered":"2024-09-21T03:34:36.646409+0000","last_clean":"2024-09-21T03:34:36.646409+0000","last_became_active":"2024-09-21T03:34:34.634023+0000","last_became_peered":"2024-09-21T03:34:34.634023+0000","last_unstale":"2024-09-21T03:34:36.646409+0000","last_undegraded":"2024-09-21T03:34:36.646409+0000","last_fullsized":"2024-09-21T03:34:36.646409+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:32:33.650331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:35.626755+0000","last_change":"2024-09-21T03:34:34.629606+0000","last_active":"2024-09-21T03:34:35.626755+0000","last_peered":"2024-09-21T03:34:35.626755+0000","last_clean":"2024-09-21T03:34:35.626755+0000","last_became_active":"2024-09-21T03:34:34.628992+0000","last_became_peered":"2024-09-21T03:34:34.628992+0000","last_unstale":"2024-09-21T03:34:35.626755+0000","last_undegraded":"2024-09-21T03:34:35.626755+0000","last_fullsized":"2024-09-21T03:34:35.626755+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:58:32.907811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:35.627528+0000","last_change":"2024-09-21T03:34:34.637695+0000","last_active":"2024-09-21T03:34:35.627528+0000","last_peered":"2024-09-21T03:34:35.627528+0000","last_clean":"2024-09-21T03:34:35.627528+0000","last_became_active":"2024-09-21T03:34:34.637320+0000","last_became_peered":"2024-09-21T03:34:34.637320+0000","last_unstale":"2024-09-21T03:34:35.627528+0000","last_undegraded":"2024-09-21T03:34:35.627528+0000","last_fullsized":"2024-09-21T03:34:35.627528+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:20:18.794303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:37.532793+0000","last_change":"2024-09-21T03:34:34.641687+0000","last_active":"2024-09-21T03:34:37.532793+0000","last_peered":"2024-09-21T03:34:37.532793+0000","last_clean":"2024-09-21T03:34:37.532793+0000","last_became_active":"2024-09-21T03:34:34.641363+0000","last_became_peered":"2024-09-21T03:34:34.641363+0000","last_unstale":"2024-09-21T03:34:37.532793+0000","last_undegraded":"2024-09-21T03:34:37.532793+0000","last_fullsized":"2024-09-21T03:34:37.532793+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:05:42.992720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:35.627526+0000","last_change":"2024-09-21T03:34:34.637080+0000","last_active":"2024-09-21T03:34:35.627526+0000","last_peered":"2024-09-21T03:34:35.627526+0000","last_clean":"2024-09-21T03:34:35.627526+0000","last_became_active":"2024-09-21T03:34:34.636777+0000","last_became_peered":"2024-09-21T03:34:34.636777+0000","last_unstale":"2024-09-21T03:34:35.627526+0000","last_undegraded":"2024-09-21T03:34:35.627526+0000","last_fullsized":"2024-09-21T03:34:35.627526+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:52:37.966204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:37.532900+0000","last_change":"2024-09-21T03:34:34.639704+0000","last_active":"2024-09-21T03:34:37.532900+0000","last_peered":"2024-09-21T03:34:37.532900+0000","last_clean":"2024-09-21T03:34:37.532900+0000","last_became_active":"2024-09-21T03:34:34.639379+0000","last_became_peered":"2024-09-21T03:34:34.639379+0000","last_unstale":"2024-09-21T03:34:37.532900+0000","last_undegraded":"2024-09-21T03:34:37.532900+0000","last_fullsized":"2024-09-21T03:34:37.532900+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:34:29.456653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:37.532505+0000","last_change":"2024-09-21T03:34:34.630276+0000","last_active":"2024-09-21T03:34:37.532505+0000","last_peered":"2024-09-21T03:34:37.532505+0000","last_clean":"2024-09-21T03:34:37.532505+0000","last_became_active":"2024-09-21T03:34:34.629954+0000","last_became_peered":"2024-09-21T03:34:34.629954+0000","last_unstale":"2024-09-21T03:34:37.532505+0000","last_undegraded":"2024-09-21T03:34:37.532505+0000","last_fullsized":"2024-09-21T03:34:37.532505+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:27:57.477444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:35.627108+0000","last_change":"2024-09-21T03:34:34.629186+0000","last_active":"2024-09-21T03:34:35.627108+0000","last_peered":"2024-09-21T03:34:35.627108+0000","last_clean":"2024-09-21T03:34:35.627108+0000","last_became_active":"2024-09-21T03:34:34.628801+0000","last_became_peered":"2024-09-21T03:34:34.628801+0000","last_unstale":"2024-09-21T03:34:35.627108+0000","last_undegraded":"2024-09-21T03:34:35.627108+0000","last_fullsized":"2024-09-21T03:34:35.627108+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:28:44.813949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:36.558469+0000","last_change":"2024-09-21T03:34:34.650750+0000","last_active":"2024-09-21T03:34:36.558469+0000","last_peered":"2024-09-21T03:34:36.558469+0000","last_clean":"2024-09-21T03:34:36.558469+0000","last_became_active":"2024-09-21T03:34:34.650436+0000","last_became_peered":"2024-09-21T03:34:34.650436+0000","last_unstale":"2024-09-21T03:34:36.558469+0000","last_undegraded":"2024-09-21T03:34:36.558469+0000","last_fullsized":"2024-09-21T03:34:36.558469+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:44:54.999306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:35.627864+0000","last_change":"2024-09-21T03:34:34.640410+0000","last_active":"2024-09-21T03:34:35.627864+0000","last_peered":"2024-09-21T03:34:35.627864+0000","last_clean":"2024-09-21T03:34:35.627864+0000","last_became_active":"2024-09-21T03:34:34.640093+0000","last_became_peered":"2024-09-21T03:34:34.640093+0000","last_unstale":"2024-09-21T03:34:35.627864+0000","last_undegraded":"2024-09-21T03:34:35.627864+0000","last_fullsized":"2024-09-21T03:34:35.627864+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:12:36.155895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:35.626584+0000","last_change":"2024-09-21T03:34:34.635135+0000","last_active":"2024-09-21T03:34:35.626584+0000","last_peered":"2024-09-21T03:34:35.626584+0000","last_clean":"2024-09-21T03:34:35.626584+0000","last_became_active":"2024-09-21T03:34:34.634811+0000","last_became_peered":"2024-09-21T03:34:34.634811+0000","last_unstale":"2024-09-21T03:34:35.626584+0000","last_undegraded":"2024-09-21T03:34:35.626584+0000","last_fullsized":"2024-09-21T03:34:35.626584+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:34:03.559814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:35.628380+0000","last_change":"2024-09-21T03:34:34.640669+0000","last_active":"2024-09-21T03:34:35.628380+0000","last_peered":"2024-09-21T03:34:35.628380+0000","last_clean":"2024-09-21T03:34:35.628380+0000","last_became_active":"2024-09-21T03:34:34.640283+0000","last_became_peered":"2024-09-21T03:34:34.640283+0000","last_unstale":"2024-09-21T03:34:35.628380+0000","last_undegraded":"2024-09-21T03:34:35.628380+0000","last_fullsized":"2024-09-21T03:34:35.628380+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:36:07.311200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:35.628400+0000","last_change":"2024-09-21T03:34:34.641925+0000","last_active":"2024-09-21T03:34:35.628400+0000","last_peered":"2024-09-21T03:34:35.628400+0000","last_clean":"2024-09-21T03:34:35.628400+0000","last_became_active":"2024-09-21T03:34:34.641309+0000","last_became_peered":"2024-09-21T03:34:34.641309+0000","last_unstale":"2024-09-21T03:34:35.628400+0000","last_undegraded":"2024-09-21T03:34:35.628400+0000","last_fullsized":"2024-09-21T03:34:35.628400+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:49:51.021825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:37.170015+0000","last_change":"2024-09-21T03:34:34.637926+0000","last_active":"2024-09-21T03:34:37.170015+0000","last_peered":"2024-09-21T03:34:37.170015+0000","last_clean":"2024-09-21T03:34:37.170015+0000","last_became_active":"2024-09-21T03:34:34.637546+0000","last_became_peered":"2024-09-21T03:34:34.637546+0000","last_unstale":"2024-09-21T03:34:37.170015+0000","last_undegraded":"2024-09-21T03:34:37.170015+0000","last_fullsized":"2024-09-21T03:34:37.170015+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:06:24.504373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:37.532666+0000","last_change":"2024-09-21T03:34:34.645556+0000","last_active":"2024-09-21T03:34:37.532666+0000","last_peered":"2024-09-21T03:34:37.532666+0000","last_clean":"2024-09-21T03:34:37.532666+0000","last_became_active":"2024-09-21T03:34:34.645228+0000","last_became_peered":"2024-09-21T03:34:34.645228+0000","last_unstale":"2024-09-21T03:34:37.532666+0000","last_undegraded":"2024-09-21T03:34:37.532666+0000","last_fullsized":"2024-09-21T03:34:37.532666+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:38:18.167710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:37.532718+0000","last_change":"2024-09-21T03:34:34.644996+0000","last_active":"2024-09-21T03:34:37.532718+0000","last_peered":"2024-09-21T03:34:37.532718+0000","last_clean":"2024-09-21T03:34:37.532718+0000","last_became_active":"2024-09-21T03:34:34.644662+0000","last_became_peered":"2024-09-21T03:34:34.644662+0000","last_unstale":"2024-09-21T03:34:37.532718+0000","last_undegraded":"2024-09-21T03:34:37.532718+0000","last_fullsized":"2024-09-21T03:34:37.532718+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:22:55.976167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:37.532623+0000","last_change":"2024-09-21T03:34:34.639985+0000","last_active":"2024-09-21T03:34:37.532623+0000","last_peered":"2024-09-21T03:34:37.532623+0000","last_clean":"2024-09-21T03:34:37.532623+0000","last_became_active":"2024-09-21T03:34:34.639595+0000","last_became_peered":"2024-09-21T03:34:34.639595+0000","last_unstale":"2024-09-21T03:34:37.532623+0000","last_undegraded":"2024-09-21T03:34:37.532623+0000","last_fullsized":"2024-09-21T03:34:37.532623+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:45:31.701500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:35.627866+0000","last_change":"2024-09-21T03:34:34.637986+0000","last_active":"2024-09-21T03:34:35.627866+0000","last_peered":"2024-09-21T03:34:35.627866+0000","last_clean":"2024-09-21T03:34:35.627866+0000","last_became_active":"2024-09-21T03:34:34.637664+0000","last_became_peered":"2024-09-21T03:34:34.637664+0000","last_unstale":"2024-09-21T03:34:35.627866+0000","last_undegraded":"2024-09-21T03:34:35.627866+0000","last_fullsized":"2024-09-21T03:34:35.627866+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:08:05.719197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:35.627534+0000","last_change":"2024-09-21T03:34:34.641334+0000","last_active":"2024-09-21T03:34:35.627534+0000","last_peered":"2024-09-21T03:34:35.627534+0000","last_clean":"2024-09-21T03:34:35.627534+0000","last_became_active":"2024-09-21T03:34:34.640922+0000","last_became_peered":"2024-09-21T03:34:34.640922+0000","last_unstale":"2024-09-21T03:34:35.627534+0000","last_undegraded":"2024-09-21T03:34:35.627534+0000","last_fullsized":"2024-09-21T03:34:35.627534+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:12:37.794061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:36.558763+0000","last_change":"2024-09-21T03:34:34.638909+0000","last_active":"2024-09-21T03:34:36.558763+0000","last_peered":"2024-09-21T03:34:36.558763+0000","last_clean":"2024-09-21T03:34:36.558763+0000","last_became_active":"2024-09-21T03:34:34.638591+0000","last_became_peered":"2024-09-21T03:34:34.638591+0000","last_unstale":"2024-09-21T03:34:36.558763+0000","last_undegraded":"2024-09-21T03:34:36.558763+0000","last_fullsized":"2024-09-21T03:34:36.558763+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:44:09.663791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:35.628400+0000","last_change":"2024-09-21T03:34:34.640146+0000","last_active":"2024-09-21T03:34:35.628400+0000","last_peered":"2024-09-21T03:34:35.628400+0000","last_clean":"2024-09-21T03:34:35.628400+0000","last_became_active":"2024-09-21T03:34:34.639791+0000","last_became_peered":"2024-09-21T03:34:34.639791+0000","last_unstale":"2024-09-21T03:34:35.628400+0000","last_undegraded":"2024-09-21T03:34:35.628400+0000","last_fullsized":"2024-09-21T03:34:35.628400+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:23:41.788750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:37.532986+0000","last_change":"2024-09-21T03:34:34.637904+0000","last_active":"2024-09-21T03:34:37.532986+0000","last_peered":"2024-09-21T03:34:37.532986+0000","last_clean":"2024-09-21T03:34:37.532986+0000","last_became_active":"2024-09-21T03:34:34.636017+0000","last_became_peered":"2024-09-21T03:34:34.636017+0000","last_unstale":"2024-09-21T03:34:37.532986+0000","last_undegraded":"2024-09-21T03:34:37.532986+0000","last_fullsized":"2024-09-21T03:34:37.532986+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:47:42.135848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:35.627281+0000","last_change":"2024-09-21T03:34:34.633013+0000","last_active":"2024-09-21T03:34:35.627281+0000","last_peered":"2024-09-21T03:34:35.627281+0000","last_clean":"2024-09-21T03:34:35.627281+0000","last_became_active":"2024-09-21T03:34:34.631701+0000","last_became_peered":"2024-09-21T03:34:34.631701+0000","last_unstale":"2024-09-21T03:34:35.627281+0000","last_undegraded":"2024-09-21T03:34:35.627281+0000","last_fullsized":"2024-09-21T03:34:35.627281+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:35:49.735410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:35.628066+0000","last_change":"2024-09-21T03:34:34.634585+0000","last_active":"2024-09-21T03:34:35.628066+0000","last_peered":"2024-09-21T03:34:35.628066+0000","last_clean":"2024-09-21T03:34:35.628066+0000","last_became_active":"2024-09-21T03:34:34.634274+0000","last_became_peered":"2024-09-21T03:34:34.634274+0000","last_unstale":"2024-09-21T03:34:35.628066+0000","last_undegraded":"2024-09-21T03:34:35.628066+0000","last_fullsized":"2024-09-21T03:34:35.628066+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:00:57.309452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:35.627775+0000","last_change":"2024-09-21T03:34:34.646715+0000","last_active":"2024-09-21T03:34:35.627775+0000","last_peered":"2024-09-21T03:34:35.627775+0000","last_clean":"2024-09-21T03:34:35.627775+0000","last_became_active":"2024-09-21T03:34:34.645628+0000","last_became_peered":"2024-09-21T03:34:34.645628+0000","last_unstale":"2024-09-21T03:34:35.627775+0000","last_undegraded":"2024-09-21T03:34:35.627775+0000","last_fullsized":"2024-09-21T03:34:35.627775+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:15:53.125585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:35.626995+0000","last_change":"2024-09-21T03:34:34.629415+0000","last_active":"2024-09-21T03:34:35.626995+0000","last_peered":"2024-09-21T03:34:35.626995+0000","last_clean":"2024-09-21T03:34:35.626995+0000","last_became_active":"2024-09-21T03:34:34.628524+0000","last_became_peered":"2024-09-21T03:34:34.628524+0000","last_unstale":"2024-09-21T03:34:35.626995+0000","last_undegraded":"2024-09-21T03:34:35.626995+0000","last_fullsized":"2024-09-21T03:34:35.626995+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:42:10.041741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:35.627792+0000","last_change":"2024-09-21T03:34:34.639100+0000","last_active":"2024-09-21T03:34:35.627792+0000","last_peered":"2024-09-21T03:34:35.627792+0000","last_clean":"2024-09-21T03:34:35.627792+0000","last_became_active":"2024-09-21T03:34:34.636285+0000","last_became_peered":"2024-09-21T03:34:34.636285+0000","last_unstale":"2024-09-21T03:34:35.627792+0000","last_undegraded":"2024-09-21T03:34:35.627792+0000","last_fullsized":"2024-09-21T03:34:35.627792+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:19:39.778069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:35.627089+0000","last_change":"2024-09-21T03:34:34.638331+0000","last_active":"2024-09-21T03:34:35.627089+0000","last_peered":"2024-09-21T03:34:35.627089+0000","last_clean":"2024-09-21T03:34:35.627089+0000","last_became_active":"2024-09-21T03:34:34.638007+0000","last_became_peered":"2024-09-21T03:34:34.638007+0000","last_unstale":"2024-09-21T03:34:35.627089+0000","last_undegraded":"2024-09-21T03:34:35.627089+0000","last_fullsized":"2024-09-21T03:34:35.627089+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:52:20.843781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"39'4","reported_seq":19,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:36.558773+0000","last_change":"2024-09-21T03:34:33.618066+0000","last_active":"2024-09-21T03:34:36.558773+0000","last_peered":"2024-09-21T03:34:36.558773+0000","last_clean":"2024-09-21T03:34:36.558773+0000","last_became_active":"2024-09-21T03:34:33.617685+0000","last_became_peered":"2024-09-21T03:34:33.617685+0000","last_unstale":"2024-09-21T03:34:36.558773+0000","last_undegraded":"2024-09-21T03:34:36.558773+0000","last_fullsized":"2024-09-21T03:34:36.558773+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_clean_scrub_stamp":"2024-09-21T03:34:32.598217+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:44:22.567045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:36.646764+0000","last_change":"2024-09-21T03:34:34.635480+0000","last_active":"2024-09-21T03:34:36.646764+0000","last_peered":"2024-09-21T03:34:36.646764+0000","last_clean":"2024-09-21T03:34:36.646764+0000","last_became_active":"2024-09-21T03:34:34.634979+0000","last_became_peered":"2024-09-21T03:34:34.634979+0000","last_unstale":"2024-09-21T03:34:36.646764+0000","last_undegraded":"2024-09-21T03:34:36.646764+0000","last_fullsized":"2024-09-21T03:34:36.646764+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:25:43.132670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"39'1","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:36.646633+0000","last_change":"2024-09-21T03:34:33.636543+0000","last_active":"2024-09-21T03:34:36.646633+0000","last_peered":"2024-09-21T03:34:36.646633+0000","last_clean":"2024-09-21T03:34:36.646633+0000","last_became_active":"2024-09-21T03:34:33.636151+0000","last_became_peered":"2024-09-21T03:34:33.636151+0000","last_unstale":"2024-09-21T03:34:36.646633+0000","last_undegraded":"2024-09-21T03:34:36.646633+0000","last_fullsized":"2024-09-21T03:34:36.646633+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_clean_scrub_stamp":"2024-09-21T03:34:32.598217+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:17:30.670560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:35.628483+0000","last_change":"2024-09-21T03:34:34.642691+0000","last_active":"2024-09-21T03:34:35.628483+0000","last_peered":"2024-09-21T03:34:35.628483+0000","last_clean":"2024-09-21T03:34:35.628483+0000","last_became_active":"2024-09-21T03:34:34.639326+0000","last_became_peered":"2024-09-21T03:34:34.639326+0000","last_unstale":"2024-09-21T03:34:35.628483+0000","last_undegraded":"2024-09-21T03:34:35.628483+0000","last_fullsized":"2024-09-21T03:34:35.628483+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:25:51.673095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"39'2","reported_seq":17,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:35.628849+0000","last_change":"2024-09-21T03:34:33.629398+0000","last_active":"2024-09-21T03:34:35.628849+0000","last_peered":"2024-09-21T03:34:35.628849+0000","last_clean":"2024-09-21T03:34:35.628849+0000","last_became_active":"2024-09-21T03:34:33.629051+0000","last_became_peered":"2024-09-21T03:34:33.629051+0000","last_unstale":"2024-09-21T03:34:35.628849+0000","last_undegraded":"2024-09-21T03:34:35.628849+0000","last_fullsized":"2024-09-21T03:34:35.628849+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_clean_scrub_stamp":"2024-09-21T03:34:32.598217+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:46:38.739080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:36.646761+0000","last_change":"2024-09-21T03:34:34.629283+0000","last_active":"2024-09-21T03:34:36.646761+0000","last_peered":"2024-09-21T03:34:36.646761+0000","last_clean":"2024-09-21T03:34:36.646761+0000","last_became_active":"2024-09-21T03:34:34.628942+0000","last_became_peered":"2024-09-21T03:34:34.628942+0000","last_unstale":"2024-09-21T03:34:36.646761+0000","last_undegraded":"2024-09-21T03:34:36.646761+0000","last_fullsized":"2024-09-21T03:34:36.646761+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:15:48.821673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1c","version":"39'2","reported_seq":17,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:36.558669+0000","last_change":"2024-09-21T03:34:33.631010+0000","last_active":"2024-09-21T03:34:36.558669+0000","last_peered":"2024-09-21T03:34:36.558669+0000","last_clean":"2024-09-21T03:34:36.558669+0000","last_became_active":"2024-09-21T03:34:33.630576+0000","last_became_peered":"2024-09-21T03:34:33.630576+0000","last_unstale":"2024-09-21T03:34:36.558669+0000","last_undegraded":"2024-09-21T03:34:36.558669+0000","last_fullsized":"2024-09-21T03:34:36.558669+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_clean_scrub_stamp":"2024-09-21T03:34:32.598217+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:19:10.406674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:35.627813+0000","last_change":"2024-09-21T03:34:34.629518+0000","last_active":"2024-09-21T03:34:35.627813+0000","last_peered":"2024-09-21T03:34:35.627813+0000","last_clean":"2024-09-21T03:34:35.627813+0000","last_became_active":"2024-09-21T03:34:34.629216+0000","last_became_peered":"2024-09-21T03:34:34.629216+0000","last_unstale":"2024-09-21T03:34:35.627813+0000","last_undegraded":"2024-09-21T03:34:35.627813+0000","last_fullsized":"2024-09-21T03:34:35.627813+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:31:59.133888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"39'1","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:36.558409+0000","last_change":"2024-09-21T03:34:33.619056+0000","last_active":"2024-09-21T03:34:36.558409+0000","last_peered":"2024-09-21T03:34:36.558409+0000","last_clean":"2024-09-21T03:34:36.558409+0000","last_became_active":"2024-09-21T03:34:33.618698+0000","last_became_peered":"2024-09-21T03:34:33.618698+0000","last_unstale":"2024-09-21T03:34:36.558409+0000","last_undegraded":"2024-09-21T03:34:36.558409+0000","last_fullsized":"2024-09-21T03:34:36.558409+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_clean_scrub_stamp":"2024-09-21T03:34:32.598217+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:58:45.880258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:37.532478+0000","last_change":"2024-09-21T03:34:34.632177+0000","last_active":"2024-09-21T03:34:37.532478+0000","last_peered":"2024-09-21T03:34:37.532478+0000","last_clean":"2024-09-21T03:34:37.532478+0000","last_became_active":"2024-09-21T03:34:34.631838+0000","last_became_peered":"2024-09-21T03:34:34.631838+0000","last_unstale":"2024-09-21T03:34:37.532478+0000","last_undegraded":"2024-09-21T03:34:37.532478+0000","last_fullsized":"2024-09-21T03:34:37.532478+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:15:48.821673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1a","version":"39'1","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:35.626780+0000","last_change":"2024-09-21T03:34:33.634501+0000","last_active":"2024-09-21T03:34:35.626780+0000","last_peered":"2024-09-21T03:34:35.626780+0000","last_clean":"2024-09-21T03:34:35.626780+0000","last_became_active":"2024-09-21T03:34:33.633827+0000","last_became_peered":"2024-09-21T03:34:33.633827+0000","last_unstale":"2024-09-21T03:34:35.626780+0000","last_undegraded":"2024-09-21T03:34:35.626780+0000","last_fullsized":"2024-09-21T03:34:35.626780+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_clean_scrub_stamp":"2024-09-21T03:34:32.598217+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:33:52.483902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:35.627886+0000","last_change":"2024-09-21T03:34:34.647398+0000","last_active":"2024-09-21T03:34:35.627886+0000","last_peered":"2024-09-21T03:34:35.627886+0000","last_clean":"2024-09-21T03:34:35.627886+0000","last_became_active":"2024-09-21T03:34:34.646411+0000","last_became_peered":"2024-09-21T03:34:34.646411+0000","last_unstale":"2024-09-21T03:34:35.627886+0000","last_undegraded":"2024-09-21T03:34:35.627886+0000","last_fullsized":"2024-09-21T03:34:35.627886+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:09:12.141326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:35.627409+0000","last_change":"2024-09-21T03:34:33.636797+0000","last_active":"2024-09-21T03:34:35.627409+0000","last_peered":"2024-09-21T03:34:35.627409+0000","last_clean":"2024-09-21T03:34:35.627409+0000","last_became_active":"2024-09-21T03:34:33.635927+0000","last_became_peered":"2024-09-21T03:34:33.635927+0000","last_unstale":"2024-09-21T03:34:35.627409+0000","last_undegraded":"2024-09-21T03:34:35.627409+0000","last_fullsized":"2024-09-21T03:34:35.627409+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_clean_scrub_stamp":"2024-09-21T03:34:32.598217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:23:52.412874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:36.646794+0000","last_change":"2024-09-21T03:34:34.640266+0000","last_active":"2024-09-21T03:34:36.646794+0000","last_peered":"2024-09-21T03:34:36.646794+0000","last_clean":"2024-09-21T03:34:36.646794+0000","last_became_active":"2024-09-21T03:34:34.639922+0000","last_became_peered":"2024-09-21T03:34:34.639922+0000","last_unstale":"2024-09-21T03:34:36.646794+0000","last_undegraded":"2024-09-21T03:34:36.646794+0000","last_fullsized":"2024-09-21T03:34:36.646794+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:38:18.167710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:35.626852+0000","last_change":"2024-09-21T03:34:33.638401+0000","last_active":"2024-09-21T03:34:35.626852+0000","last_peered":"2024-09-21T03:34:35.626852+0000","last_clean":"2024-09-21T03:34:35.626852+0000","last_became_active":"2024-09-21T03:34:33.637364+0000","last_became_peered":"2024-09-21T03:34:33.637364+0000","last_unstale":"2024-09-21T03:34:35.626852+0000","last_undegraded":"2024-09-21T03:34:35.626852+0000","last_fullsized":"2024-09-21T03:34:35.626852+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_clean_scrub_stamp":"2024-09-21T03:34:32.598217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:37:36.532174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:35.628541+0000","last_change":"2024-09-21T03:34:34.633927+0000","last_active":"2024-09-21T03:34:35.628541+0000","last_peered":"2024-09-21T03:34:35.628541+0000","last_clean":"2024-09-21T03:34:35.628541+0000","last_became_active":"2024-09-21T03:34:34.633294+0000","last_became_peered":"2024-09-21T03:34:34.633294+0000","last_unstale":"2024-09-21T03:34:35.628541+0000","last_undegraded":"2024-09-21T03:34:35.628541+0000","last_fullsized":"2024-09-21T03:34:35.628541+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:41:47.832177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:35.628042+0000","last_change":"2024-09-21T03:34:33.626093+0000","last_active":"2024-09-21T03:34:35.628042+0000","last_peered":"2024-09-21T03:34:35.628042+0000","last_clean":"2024-09-21T03:34:35.628042+0000","last_became_active":"2024-09-21T03:34:33.625795+0000","last_became_peered":"2024-09-21T03:34:33.625795+0000","last_unstale":"2024-09-21T03:34:35.628042+0000","last_undegraded":"2024-09-21T03:34:35.628042+0000","last_fullsized":"2024-09-21T03:34:35.628042+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_clean_scrub_stamp":"2024-09-21T03:34:32.598217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:39:10.947435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:36.558568+0000","last_change":"2024-09-21T03:34:34.641036+0000","last_active":"2024-09-21T03:34:36.558568+0000","last_peered":"2024-09-21T03:34:36.558568+0000","last_clean":"2024-09-21T03:34:36.558568+0000","last_became_active":"2024-09-21T03:34:34.640653+0000","last_became_peered":"2024-09-21T03:34:34.640653+0000","last_unstale":"2024-09-21T03:34:36.558568+0000","last_undegraded":"2024-09-21T03:34:36.558568+0000","last_fullsized":"2024-09-21T03:34:36.558568+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:34:23.480441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:35.628283+0000","last_change":"2024-09-21T03:34:33.633888+0000","last_active":"2024-09-21T03:34:35.628283+0000","last_peered":"2024-09-21T03:34:35.628283+0000","last_clean":"2024-09-21T03:34:35.628283+0000","last_became_active":"2024-09-21T03:34:33.633372+0000","last_became_peered":"2024-09-21T03:34:33.633372+0000","last_unstale":"2024-09-21T03:34:35.628283+0000","last_undegraded":"2024-09-21T03:34:35.628283+0000","last_fullsized":"2024-09-21T03:34:35.628283+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_clean_scrub_stamp":"2024-09-21T03:34:32.598217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:33:02.644464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:35.628444+0000","last_change":"2024-09-21T03:34:34.644095+0000","last_active":"2024-09-21T03:34:35.628444+0000","last_peered":"2024-09-21T03:34:35.628444+0000","last_clean":"2024-09-21T03:34:35.628444+0000","last_became_active":"2024-09-21T03:34:34.643713+0000","last_became_peered":"2024-09-21T03:34:34.643713+0000","last_unstale":"2024-09-21T03:34:35.628444+0000","last_undegraded":"2024-09-21T03:34:35.628444+0000","last_fullsized":"2024-09-21T03:34:35.628444+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:40:54.924468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.15","version":"39'3","reported_seq":18,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:35.627266+0000","last_change":"2024-09-21T03:34:33.632451+0000","last_active":"2024-09-21T03:34:35.627266+0000","last_peered":"2024-09-21T03:34:35.627266+0000","last_clean":"2024-09-21T03:34:35.627266+0000","last_became_active":"2024-09-21T03:34:33.632103+0000","last_became_peered":"2024-09-21T03:34:33.632103+0000","last_unstale":"2024-09-21T03:34:35.627266+0000","last_undegraded":"2024-09-21T03:34:35.627266+0000","last_fullsized":"2024-09-21T03:34:35.627266+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_clean_scrub_stamp":"2024-09-21T03:34:32.598217+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:12:36.787683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:35.627520+0000","last_change":"2024-09-21T03:34:34.628419+0000","last_active":"2024-09-21T03:34:35.627520+0000","last_peered":"2024-09-21T03:34:35.627520+0000","last_clean":"2024-09-21T03:34:35.627520+0000","last_became_active":"2024-09-21T03:34:34.628087+0000","last_became_peered":"2024-09-21T03:34:34.628087+0000","last_unstale":"2024-09-21T03:34:35.627520+0000","last_undegraded":"2024-09-21T03:34:35.627520+0000","last_fullsized":"2024-09-21T03:34:35.627520+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:35:03.526533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.14","version":"39'6","reported_seq":21,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:35.628508+0000","last_change":"2024-09-21T03:34:33.639043+0000","last_active":"2024-09-21T03:34:35.628508+0000","last_peered":"2024-09-21T03:34:35.628508+0000","last_clean":"2024-09-21T03:34:35.628508+0000","last_became_active":"2024-09-21T03:34:33.638761+0000","last_became_peered":"2024-09-21T03:34:33.638761+0000","last_unstale":"2024-09-21T03:34:35.628508+0000","last_undegraded":"2024-09-21T03:34:35.628508+0000","last_fullsized":"2024-09-21T03:34:35.628508+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_clean_scrub_stamp":"2024-09-21T03:34:32.598217+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:44:53.992927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:35.628094+0000","last_change":"2024-09-21T03:34:34.640167+0000","last_active":"2024-09-21T03:34:35.628094+0000","last_peered":"2024-09-21T03:34:35.628094+0000","last_clean":"2024-09-21T03:34:35.628094+0000","last_became_active":"2024-09-21T03:34:34.639859+0000","last_became_peered":"2024-09-21T03:34:34.639859+0000","last_unstale":"2024-09-21T03:34:35.628094+0000","last_undegraded":"2024-09-21T03:34:35.628094+0000","last_fullsized":"2024-09-21T03:34:35.628094+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:36:48.213957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"39'2","reported_seq":17,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:35.627656+0000","last_change":"2024-09-21T03:34:33.628173+0000","last_active":"2024-09-21T03:34:35.627656+0000","last_peered":"2024-09-21T03:34:35.627656+0000","last_clean":"2024-09-21T03:34:35.627656+0000","last_became_active":"2024-09-21T03:34:33.627559+0000","last_became_peered":"2024-09-21T03:34:33.627559+0000","last_unstale":"2024-09-21T03:34:35.627656+0000","last_undegraded":"2024-09-21T03:34:35.627656+0000","last_fullsized":"2024-09-21T03:34:35.627656+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_clean_scrub_stamp":"2024-09-21T03:34:32.598217+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:34:22.474063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:36.646395+0000","last_change":"2024-09-21T03:34:34.626145+0000","last_active":"2024-09-21T03:34:36.646395+0000","last_peered":"2024-09-21T03:34:36.646395+0000","last_clean":"2024-09-21T03:34:36.646395+0000","last_became_active":"2024-09-21T03:34:34.625819+0000","last_became_peered":"2024-09-21T03:34:34.625819+0000","last_unstale":"2024-09-21T03:34:36.646395+0000","last_undegraded":"2024-09-21T03:34:36.646395+0000","last_fullsized":"2024-09-21T03:34:36.646395+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:44:09.663791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:36.646276+0000","last_change":"2024-09-21T03:34:33.630029+0000","last_active":"2024-09-21T03:34:36.646276+0000","last_peered":"2024-09-21T03:34:36.646276+0000","last_clean":"2024-09-21T03:34:36.646276+0000","last_became_active":"2024-09-21T03:34:33.629699+0000","last_became_peered":"2024-09-21T03:34:33.629699+0000","last_unstale":"2024-09-21T03:34:36.646276+0000","last_undegraded":"2024-09-21T03:34:36.646276+0000","last_fullsized":"2024-09-21T03:34:36.646276+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_clean_scrub_stamp":"2024-09-21T03:34:32.598217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:49:16.438096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:37.532958+0000","last_change":"2024-09-21T03:34:34.631316+0000","last_active":"2024-09-21T03:34:37.532958+0000","last_peered":"2024-09-21T03:34:37.532958+0000","last_clean":"2024-09-21T03:34:37.532958+0000","last_became_active":"2024-09-21T03:34:34.630729+0000","last_became_peered":"2024-09-21T03:34:34.630729+0000","last_unstale":"2024-09-21T03:34:37.532958+0000","last_undegraded":"2024-09-21T03:34:37.532958+0000","last_fullsized":"2024-09-21T03:34:37.532958+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:58:22.321211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:35.628054+0000","last_change":"2024-09-21T03:34:33.639290+0000","last_active":"2024-09-21T03:34:35.628054+0000","last_peered":"2024-09-21T03:34:35.628054+0000","last_clean":"2024-09-21T03:34:35.628054+0000","last_became_active":"2024-09-21T03:34:33.638941+0000","last_became_peered":"2024-09-21T03:34:33.638941+0000","last_unstale":"2024-09-21T03:34:35.628054+0000","last_undegraded":"2024-09-21T03:34:35.628054+0000","last_fullsized":"2024-09-21T03:34:35.628054+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_clean_scrub_stamp":"2024-09-21T03:34:32.598217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:39:24.714082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:37.532515+0000","last_change":"2024-09-21T03:34:34.631462+0000","last_active":"2024-09-21T03:34:37.532515+0000","last_peered":"2024-09-21T03:34:37.532515+0000","last_clean":"2024-09-21T03:34:37.532515+0000","last_became_active":"2024-09-21T03:34:34.630986+0000","last_became_peered":"2024-09-21T03:34:34.630986+0000","last_unstale":"2024-09-21T03:34:37.532515+0000","last_undegraded":"2024-09-21T03:34:37.532515+0000","last_fullsized":"2024-09-21T03:34:37.532515+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:39:06.107764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.10","version":"39'8","reported_seq":23,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:36.558526+0000","last_change":"2024-09-21T03:34:33.632101+0000","last_active":"2024-09-21T03:34:36.558526+0000","last_peered":"2024-09-21T03:34:36.558526+0000","last_clean":"2024-09-21T03:34:36.558526+0000","last_became_active":"2024-09-21T03:34:33.631744+0000","last_became_peered":"2024-09-21T03:34:33.631744+0000","last_unstale":"2024-09-21T03:34:36.558526+0000","last_undegraded":"2024-09-21T03:34:36.558526+0000","last_fullsized":"2024-09-21T03:34:36.558526+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_clean_scrub_stamp":"2024-09-21T03:34:32.598217+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:54:21.167047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:36.646635+0000","last_change":"2024-09-21T03:34:34.626189+0000","last_active":"2024-09-21T03:34:36.646635+0000","last_peered":"2024-09-21T03:34:36.646635+0000","last_clean":"2024-09-21T03:34:36.646635+0000","last_became_active":"2024-09-21T03:34:34.625865+0000","last_became_peered":"2024-09-21T03:34:34.625865+0000","last_unstale":"2024-09-21T03:34:36.646635+0000","last_undegraded":"2024-09-21T03:34:36.646635+0000","last_fullsized":"2024-09-21T03:34:36.646635+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:13:44.244299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.f","version":"39'1","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:35.627544+0000","last_change":"2024-09-21T03:34:33.637659+0000","last_active":"2024-09-21T03:34:35.627544+0000","last_peered":"2024-09-21T03:34:35.627544+0000","last_clean":"2024-09-21T03:34:35.627544+0000","last_became_active":"2024-09-21T03:34:33.637317+0000","last_became_peered":"2024-09-21T03:34:33.637317+0000","last_unstale":"2024-09-21T03:34:35.627544+0000","last_undegraded":"2024-09-21T03:34:35.627544+0000","last_fullsized":"2024-09-21T03:34:35.627544+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_clean_scrub_stamp":"2024-09-21T03:34:32.598217+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:12:23.510532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:35.627237+0000","last_change":"2024-09-21T03:34:34.634278+0000","last_active":"2024-09-21T03:34:35.627237+0000","last_peered":"2024-09-21T03:34:35.627237+0000","last_clean":"2024-09-21T03:34:35.627237+0000","last_became_active":"2024-09-21T03:34:34.633950+0000","last_became_peered":"2024-09-21T03:34:34.633950+0000","last_unstale":"2024-09-21T03:34:35.627237+0000","last_undegraded":"2024-09-21T03:34:35.627237+0000","last_fullsized":"2024-09-21T03:34:35.627237+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:05:02.878405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"39'2","reported_seq":17,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:35.628583+0000","last_change":"2024-09-21T03:34:33.634743+0000","last_active":"2024-09-21T03:34:35.628583+0000","last_peered":"2024-09-21T03:34:35.628583+0000","last_clean":"2024-09-21T03:34:35.628583+0000","last_became_active":"2024-09-21T03:34:33.634259+0000","last_became_peered":"2024-09-21T03:34:33.634259+0000","last_unstale":"2024-09-21T03:34:35.628583+0000","last_undegraded":"2024-09-21T03:34:35.628583+0000","last_fullsized":"2024-09-21T03:34:35.628583+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_clean_scrub_stamp":"2024-09-21T03:34:32.598217+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:57:43.452397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:35.627546+0000","last_change":"2024-09-21T03:34:34.628247+0000","last_active":"2024-09-21T03:34:35.627546+0000","last_peered":"2024-09-21T03:34:35.627546+0000","last_clean":"2024-09-21T03:34:35.627546+0000","last_became_active":"2024-09-21T03:34:34.627925+0000","last_became_peered":"2024-09-21T03:34:34.627925+0000","last_unstale":"2024-09-21T03:34:35.627546+0000","last_undegraded":"2024-09-21T03:34:35.627546+0000","last_fullsized":"2024-09-21T03:34:35.627546+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:52:32.863568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"39'1","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:37.532400+0000","last_change":"2024-09-21T03:34:33.635141+0000","last_active":"2024-09-21T03:34:37.532400+0000","last_peered":"2024-09-21T03:34:37.532400+0000","last_clean":"2024-09-21T03:34:37.532400+0000","last_became_active":"2024-09-21T03:34:33.634825+0000","last_became_peered":"2024-09-21T03:34:33.634825+0000","last_unstale":"2024-09-21T03:34:37.532400+0000","last_undegraded":"2024-09-21T03:34:37.532400+0000","last_fullsized":"2024-09-21T03:34:37.532400+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_clean_scrub_stamp":"2024-09-21T03:34:32.598217+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:33:52.483902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:37.170144+0000","last_change":"2024-09-21T03:34:34.630174+0000","last_active":"2024-09-21T03:34:37.170144+0000","last_peered":"2024-09-21T03:34:37.170144+0000","last_clean":"2024-09-21T03:34:37.170144+0000","last_became_active":"2024-09-21T03:34:34.629879+0000","last_became_peered":"2024-09-21T03:34:34.629879+0000","last_unstale":"2024-09-21T03:34:37.170144+0000","last_undegraded":"2024-09-21T03:34:37.170144+0000","last_fullsized":"2024-09-21T03:34:37.170144+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:33:33.810623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:37.169868+0000","last_change":"2024-09-21T03:34:33.630648+0000","last_active":"2024-09-21T03:34:37.169868+0000","last_peered":"2024-09-21T03:34:37.169868+0000","last_clean":"2024-09-21T03:34:37.169868+0000","last_became_active":"2024-09-21T03:34:33.630288+0000","last_became_peered":"2024-09-21T03:34:33.630288+0000","last_unstale":"2024-09-21T03:34:37.169868+0000","last_undegraded":"2024-09-21T03:34:37.169868+0000","last_fullsized":"2024-09-21T03:34:37.169868+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_clean_scrub_stamp":"2024-09-21T03:34:32.598217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:03:25.007146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:35.627463+0000","last_change":"2024-09-21T03:34:34.630957+0000","last_active":"2024-09-21T03:34:35.627463+0000","last_peered":"2024-09-21T03:34:35.627463+0000","last_clean":"2024-09-21T03:34:35.627463+0000","last_became_active":"2024-09-21T03:34:34.630549+0000","last_became_peered":"2024-09-21T03:34:34.630549+0000","last_unstale":"2024-09-21T03:34:35.627463+0000","last_undegraded":"2024-09-21T03:34:35.627463+0000","last_fullsized":"2024-09-21T03:34:35.627463+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:31:25.712062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"39'2","reported_seq":17,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:37.169923+0000","last_change":"2024-09-21T03:34:33.623824+0000","last_active":"2024-09-21T03:34:37.169923+0000","last_peered":"2024-09-21T03:34:37.169923+0000","last_clean":"2024-09-21T03:34:37.169923+0000","last_became_active":"2024-09-21T03:34:33.623465+0000","last_became_peered":"2024-09-21T03:34:33.623465+0000","last_unstale":"2024-09-21T03:34:37.169923+0000","last_undegraded":"2024-09-21T03:34:37.169923+0000","last_fullsized":"2024-09-21T03:34:37.169923+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_clean_scrub_stamp":"2024-09-21T03:34:32.598217+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:07:51.182124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:35.627438+0000","last_change":"2024-09-21T03:34:34.635616+0000","last_active":"2024-09-21T03:34:35.627438+0000","last_peered":"2024-09-21T03:34:35.627438+0000","last_clean":"2024-09-21T03:34:35.627438+0000","last_became_active":"2024-09-21T03:34:34.632739+0000","last_became_peered":"2024-09-21T03:34:34.632739+0000","last_unstale":"2024-09-21T03:34:35.627438+0000","last_undegraded":"2024-09-21T03:34:35.627438+0000","last_fullsized":"2024-09-21T03:34:35.627438+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:49:01.779676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:36.646422+0000","last_change":"2024-09-21T03:34:33.633593+0000","last_active":"2024-09-21T03:34:36.646422+0000","last_peered":"2024-09-21T03:34:36.646422+0000","last_clean":"2024-09-21T03:34:36.646422+0000","last_became_active":"2024-09-21T03:34:33.633254+0000","last_became_peered":"2024-09-21T03:34:33.633254+0000","last_unstale":"2024-09-21T03:34:36.646422+0000","last_undegraded":"2024-09-21T03:34:36.646422+0000","last_fullsized":"2024-09-21T03:34:36.646422+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_clean_scrub_stamp":"2024-09-21T03:34:32.598217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:57:01.944255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:37.532622+0000","last_change":"2024-09-21T03:34:34.647019+0000","last_active":"2024-09-21T03:34:37.532622+0000","last_peered":"2024-09-21T03:34:37.532622+0000","last_clean":"2024-09-21T03:34:37.532622+0000","last_became_active":"2024-09-21T03:34:34.646611+0000","last_became_peered":"2024-09-21T03:34:34.646611+0000","last_unstale":"2024-09-21T03:34:37.532622+0000","last_undegraded":"2024-09-21T03:34:37.532622+0000","last_fullsized":"2024-09-21T03:34:37.532622+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:45:10.948679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:35.626999+0000","last_change":"2024-09-21T03:34:33.637630+0000","last_active":"2024-09-21T03:34:35.626999+0000","last_peered":"2024-09-21T03:34:35.626999+0000","last_clean":"2024-09-21T03:34:35.626999+0000","last_became_active":"2024-09-21T03:34:33.637290+0000","last_became_peered":"2024-09-21T03:34:33.637290+0000","last_unstale":"2024-09-21T03:34:35.626999+0000","last_undegraded":"2024-09-21T03:34:35.626999+0000","last_fullsized":"2024-09-21T03:34:35.626999+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_clean_scrub_stamp":"2024-09-21T03:34:32.598217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:00:46.535619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:35.627723+0000","last_change":"2024-09-21T03:34:34.644149+0000","last_active":"2024-09-21T03:34:35.627723+0000","last_peered":"2024-09-21T03:34:35.627723+0000","last_clean":"2024-09-21T03:34:35.627723+0000","last_became_active":"2024-09-21T03:34:34.643689+0000","last_became_peered":"2024-09-21T03:34:34.643689+0000","last_unstale":"2024-09-21T03:34:35.627723+0000","last_undegraded":"2024-09-21T03:34:35.627723+0000","last_fullsized":"2024-09-21T03:34:35.627723+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:51:48.433744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:36.646016+0000","last_change":"2024-09-21T03:34:33.626688+0000","last_active":"2024-09-21T03:34:36.646016+0000","last_peered":"2024-09-21T03:34:36.646016+0000","last_clean":"2024-09-21T03:34:36.646016+0000","last_became_active":"2024-09-21T03:34:33.626325+0000","last_became_peered":"2024-09-21T03:34:33.626325+0000","last_unstale":"2024-09-21T03:34:36.646016+0000","last_undegraded":"2024-09-21T03:34:36.646016+0000","last_fullsized":"2024-09-21T03:34:36.646016+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_clean_scrub_stamp":"2024-09-21T03:34:32.598217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:50:07.223556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:37.532563+0000","last_change":"2024-09-21T03:34:34.638492+0000","last_active":"2024-09-21T03:34:37.532563+0000","last_peered":"2024-09-21T03:34:37.532563+0000","last_clean":"2024-09-21T03:34:37.532563+0000","last_became_active":"2024-09-21T03:34:34.638188+0000","last_became_peered":"2024-09-21T03:34:34.638188+0000","last_unstale":"2024-09-21T03:34:37.532563+0000","last_undegraded":"2024-09-21T03:34:37.532563+0000","last_fullsized":"2024-09-21T03:34:37.532563+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:50:53.901614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"39'2","reported_seq":17,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:35.628356+0000","last_change":"2024-09-21T03:34:33.633316+0000","last_active":"2024-09-21T03:34:35.628356+0000","last_peered":"2024-09-21T03:34:35.628356+0000","last_clean":"2024-09-21T03:34:35.628356+0000","last_became_active":"2024-09-21T03:34:33.632983+0000","last_became_peered":"2024-09-21T03:34:33.632983+0000","last_unstale":"2024-09-21T03:34:35.628356+0000","last_undegraded":"2024-09-21T03:34:35.628356+0000","last_fullsized":"2024-09-21T03:34:35.628356+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_clean_scrub_stamp":"2024-09-21T03:34:32.598217+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:28:29.243636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:37.533231+0000","last_change":"2024-09-21T03:34:34.638896+0000","last_active":"2024-09-21T03:34:37.533231+0000","last_peered":"2024-09-21T03:34:37.533231+0000","last_clean":"2024-09-21T03:34:37.533231+0000","last_became_active":"2024-09-21T03:34:34.638412+0000","last_became_peered":"2024-09-21T03:34:34.638412+0000","last_unstale":"2024-09-21T03:34:37.533231+0000","last_undegraded":"2024-09-21T03:34:37.533231+0000","last_fullsized":"2024-09-21T03:34:37.533231+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:24:34.102015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"39'1","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:37.532556+0000","last_change":"2024-09-21T03:34:33.630944+0000","last_active":"2024-09-21T03:34:37.532556+0000","last_peered":"2024-09-21T03:34:37.532556+0000","last_clean":"2024-09-21T03:34:37.532556+0000","last_became_active":"2024-09-21T03:34:33.630396+0000","last_became_peered":"2024-09-21T03:34:33.630396+0000","last_unstale":"2024-09-21T03:34:37.532556+0000","last_undegraded":"2024-09-21T03:34:37.532556+0000","last_fullsized":"2024-09-21T03:34:37.532556+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_clean_scrub_stamp":"2024-09-21T03:34:32.598217+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:26:58.045331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:37.170031+0000","last_change":"2024-09-21T03:34:34.638755+0000","last_active":"2024-09-21T03:34:37.170031+0000","last_peered":"2024-09-21T03:34:37.170031+0000","last_clean":"2024-09-21T03:34:37.170031+0000","last_became_active":"2024-09-21T03:34:34.638432+0000","last_became_peered":"2024-09-21T03:34:34.638432+0000","last_unstale":"2024-09-21T03:34:37.170031+0000","last_undegraded":"2024-09-21T03:34:37.170031+0000","last_fullsized":"2024-09-21T03:34:37.170031+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:38:35.915944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:37.532432+0000","last_change":"2024-09-21T03:34:33.626093+0000","last_active":"2024-09-21T03:34:37.532432+0000","last_peered":"2024-09-21T03:34:37.532432+0000","last_clean":"2024-09-21T03:34:37.532432+0000","last_became_active":"2024-09-21T03:34:33.625771+0000","last_became_peered":"2024-09-21T03:34:33.625771+0000","last_unstale":"2024-09-21T03:34:37.532432+0000","last_undegraded":"2024-09-21T03:34:37.532432+0000","last_fullsized":"2024-09-21T03:34:37.532432+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_clean_scrub_stamp":"2024-09-21T03:34:32.598217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:00:46.535619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:37.532236+0000","last_change":"2024-09-21T03:34:34.637581+0000","last_active":"2024-09-21T03:34:37.532236+0000","last_peered":"2024-09-21T03:34:37.532236+0000","last_clean":"2024-09-21T03:34:37.532236+0000","last_became_active":"2024-09-21T03:34:34.637064+0000","last_became_peered":"2024-09-21T03:34:34.637064+0000","last_unstale":"2024-09-21T03:34:37.532236+0000","last_undegraded":"2024-09-21T03:34:37.532236+0000","last_fullsized":"2024-09-21T03:34:37.532236+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:44:10.495963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:35.627816+0000","last_change":"2024-09-21T03:34:33.625164+0000","last_active":"2024-09-21T03:34:35.627816+0000","last_peered":"2024-09-21T03:34:35.627816+0000","last_clean":"2024-09-21T03:34:35.627816+0000","last_became_active":"2024-09-21T03:34:33.624299+0000","last_became_peered":"2024-09-21T03:34:33.624299+0000","last_unstale":"2024-09-21T03:34:35.627816+0000","last_undegraded":"2024-09-21T03:34:35.627816+0000","last_fullsized":"2024-09-21T03:34:35.627816+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_clean_scrub_stamp":"2024-09-21T03:34:32.598217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:32:32.643952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:36.646586+0000","last_change":"2024-09-21T03:34:34.637737+0000","last_active":"2024-09-21T03:34:36.646586+0000","last_peered":"2024-09-21T03:34:36.646586+0000","last_clean":"2024-09-21T03:34:36.646586+0000","last_became_active":"2024-09-21T03:34:34.637441+0000","last_became_peered":"2024-09-21T03:34:34.637441+0000","last_unstale":"2024-09-21T03:34:36.646586+0000","last_undegraded":"2024-09-21T03:34:36.646586+0000","last_fullsized":"2024-09-21T03:34:36.646586+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:12:23.136125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:37.533187+0000","last_change":"2024-09-21T03:34:33.631273+0000","last_active":"2024-09-21T03:34:37.533187+0000","last_peered":"2024-09-21T03:34:37.533187+0000","last_clean":"2024-09-21T03:34:37.533187+0000","last_became_active":"2024-09-21T03:34:33.628561+0000","last_became_peered":"2024-09-21T03:34:33.628561+0000","last_unstale":"2024-09-21T03:34:37.533187+0000","last_undegraded":"2024-09-21T03:34:37.533187+0000","last_fullsized":"2024-09-21T03:34:37.533187+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_clean_scrub_stamp":"2024-09-21T03:34:32.598217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:19:10.406674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:35.628484+0000","last_change":"2024-09-21T03:34:34.647069+0000","last_active":"2024-09-21T03:34:35.628484+0000","last_peered":"2024-09-21T03:34:35.628484+0000","last_clean":"2024-09-21T03:34:35.628484+0000","last_became_active":"2024-09-21T03:34:34.646036+0000","last_became_peered":"2024-09-21T03:34:34.646036+0000","last_unstale":"2024-09-21T03:34:35.628484+0000","last_undegraded":"2024-09-21T03:34:35.628484+0000","last_fullsized":"2024-09-21T03:34:35.628484+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:03:44.297556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:35.628184+0000","last_change":"2024-09-21T03:34:33.633175+0000","last_active":"2024-09-21T03:34:35.628184+0000","last_peered":"2024-09-21T03:34:35.628184+0000","last_clean":"2024-09-21T03:34:35.628184+0000","last_became_active":"2024-09-21T03:34:33.632872+0000","last_became_peered":"2024-09-21T03:34:33.632872+0000","last_unstale":"2024-09-21T03:34:35.628184+0000","last_undegraded":"2024-09-21T03:34:35.628184+0000","last_fullsized":"2024-09-21T03:34:35.628184+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_clean_scrub_stamp":"2024-09-21T03:34:32.598217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:22:17.742210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:37.532767+0000","last_change":"2024-09-21T03:34:34.635479+0000","last_active":"2024-09-21T03:34:37.532767+0000","last_peered":"2024-09-21T03:34:37.532767+0000","last_clean":"2024-09-21T03:34:37.532767+0000","last_became_active":"2024-09-21T03:34:34.635163+0000","last_became_peered":"2024-09-21T03:34:34.635163+0000","last_unstale":"2024-09-21T03:34:37.532767+0000","last_undegraded":"2024-09-21T03:34:37.532767+0000","last_fullsized":"2024-09-21T03:34:37.532767+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:37:52.511409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:35.628165+0000","last_change":"2024-09-21T03:34:33.639171+0000","last_active":"2024-09-21T03:34:35.628165+0000","last_peered":"2024-09-21T03:34:35.628165+0000","last_clean":"2024-09-21T03:34:35.628165+0000","last_became_active":"2024-09-21T03:34:33.638819+0000","last_became_peered":"2024-09-21T03:34:33.638819+0000","last_unstale":"2024-09-21T03:34:35.628165+0000","last_undegraded":"2024-09-21T03:34:35.628165+0000","last_fullsized":"2024-09-21T03:34:35.628165+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_clean_scrub_stamp":"2024-09-21T03:34:32.598217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:44:08.657412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:35.628188+0000","last_change":"2024-09-21T03:34:34.639859+0000","last_active":"2024-09-21T03:34:35.628188+0000","last_peered":"2024-09-21T03:34:35.628188+0000","last_clean":"2024-09-21T03:34:35.628188+0000","last_became_active":"2024-09-21T03:34:34.639397+0000","last_became_peered":"2024-09-21T03:34:34.639397+0000","last_unstale":"2024-09-21T03:34:35.628188+0000","last_undegraded":"2024-09-21T03:34:35.628188+0000","last_fullsized":"2024-09-21T03:34:35.628188+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:33.604595+0000","last_clean_scrub_stamp":"2024-09-21T03:34:33.604595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:55:01.479418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:37.532568+0000","last_change":"2024-09-21T03:34:33.635753+0000","last_active":"2024-09-21T03:34:37.532568+0000","last_peered":"2024-09-21T03:34:37.532568+0000","last_clean":"2024-09-21T03:34:37.532568+0000","last_became_active":"2024-09-21T03:34:33.631914+0000","last_became_peered":"2024-09-21T03:34:33.631914+0000","last_unstale":"2024-09-21T03:34:37.532568+0000","last_undegraded":"2024-09-21T03:34:37.532568+0000","last_fullsized":"2024-09-21T03:34:37.532568+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:34:32.598217+0000","last_clean_scrub_stamp":"2024-09-21T03:34:32.598217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:44:22.567045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":145,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T03:34:35.627255+0000","last_change":"2024-09-21T03:33:25.319776+0000","last_active":"2024-09-21T03:34:35.627255+0000","last_peered":"2024-09-21T03:34:35.627255+0000","last_clean":"2024-09-21T03:34:35.627255+0000","last_became_active":"2024-09-21T03:33:25.318399+0000","last_became_peered":"2024-09-21T03:33:25.318399+0000","last_unstale":"2024-09-21T03:34:35.627255+0000","last_undegraded":"2024-09-21T03:34:35.627255+0000","last_fullsized":"2024-09-21T03:34:35.627255+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:23.016672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:23.016672+0000","last_clean_scrub_stamp":"2024-09-21T03:33:23.016672+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:32:35.906114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":1,"up_from":16,"seq":68719476755,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32216,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339624,"statfs":{"total":96636764160,"available":96603774976,"internally_reserved":0,"allocated":1052672,"data_stored":786835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509459,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31636,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340204,"statfs":{"total":96636764160,"available":96604368896,"internally_reserved":0,"allocated":458752,"data_stored":196515,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509458,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31640,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340200,"statfs":{"total":96636764160,"available":96604364800,"internally_reserved":0,"allocated":458752,"data_stored":196535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509459,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27492,"kb_used_data":460,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344348,"statfs":{"total":96636764160,"available":96608612352,"internally_reserved":0,"allocated":471040,"data_stored":198639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509458,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31640,"kb_used_data":452,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340200,"statfs":{"total":96636764160,"available":96604364800,"internally_reserved":0,"allocated":462848,"data_stored":196537,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542162,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31640,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340200,"statfs":{"total":96636764160,"available":96604364800,"internally_reserved":0,"allocated":458752,"data_stored":196479,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574867,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32216,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339624,"statfs":{"total":96636764160,"available":96603774976,"internally_reserved":0,"allocated":1052672,"data_stored":787395,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542163,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35816,"kb_used_data":464,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336024,"statfs":{"total":96636764160,"available":96600088576,"internally_reserved":0,"allocated":475136,"data_stored":198181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T03:34:43.626 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-21T03:34:43.626 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-21T03:34:43.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:43.784+0000 7f61c4071640 1 -- 172.21.15.38:0/2504357996 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f61bc162280 msgr2=0x7f61bc162690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:43.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:43.784+0000 7f61c4071640 1 --2- 172.21.15.38:0/2504357996 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f61bc162280 0x7f61bc162690 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f61b0009830 tx=0x7f61b002a480 comp rx=0 tx=0).stop 2024-09-21T03:34:43.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:43.784+0000 7f61c4071640 1 -- 172.21.15.38:0/2504357996 shutdown_connections 2024-09-21T03:34:43.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:43.784+0000 7f61c4071640 1 --2- 172.21.15.38:0/2504357996 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f61bc162280 0x7f61bc162690 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:43.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:43.784+0000 7f61c4071640 1 -- 172.21.15.38:0/2504357996 >> 172.21.15.38:0/2504357996 conn(0x7f61bc0b6b20 msgr2=0x7f61bc0d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:43.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:43.785+0000 7f61c4071640 1 -- 172.21.15.38:0/2504357996 shutdown_connections 2024-09-21T03:34:43.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:43.785+0000 7f61c4071640 1 -- 172.21.15.38:0/2504357996 wait complete. 2024-09-21T03:34:43.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:43.786+0000 7f61c4071640 1 Processor -- start 2024-09-21T03:34:43.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:43.786+0000 7f61c4071640 1 -- start start 2024-09-21T03:34:43.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:43.788+0000 7f61c4071640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f61bc0b2e40 0x7f61bc1f89a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:43.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:43.788+0000 7f61c4071640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f61bc162280 0x7f61bc1f8ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:43.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:43.788+0000 7f61c4071640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f61bc158b40 0x7f61bc158f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:43.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:43.788+0000 7f61c4071640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f61bc15f640 con 0x7f61bc158b40 2024-09-21T03:34:43.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:43.788+0000 7f61c4071640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f61bc15f4c0 con 0x7f61bc0b2e40 2024-09-21T03:34:43.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:43.788+0000 7f61c4071640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f61bc15f7c0 con 0x7f61bc162280 2024-09-21T03:34:43.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:43.788+0000 7f61c260e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f61bc158b40 0x7f61bc158f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:43.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:43.788+0000 7f61c260e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f61bc158b40 0x7f61bc158f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50092/0 (socket says 172.21.15.38:50092) 2024-09-21T03:34:43.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:43.788+0000 7f61c160c640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f61bc162280 0x7f61bc1f8ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:43.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:43.788+0000 7f61c260e640 1 -- 172.21.15.38:0/268881925 learned_addr learned my addr 172.21.15.38:0/268881925 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:34:43.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:43.788+0000 7f61c1e0d640 1 --2- 172.21.15.38:0/268881925 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f61bc0b2e40 0x7f61bc1f89a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:43.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:43.788+0000 7f61c260e640 1 -- 172.21.15.38:0/268881925 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f61bc162280 msgr2=0x7f61bc1f8ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:43.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:43.788+0000 7f61c260e640 1 --2- 172.21.15.38:0/268881925 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f61bc162280 0x7f61bc1f8ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:43.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:43.788+0000 7f61c260e640 1 -- 172.21.15.38:0/268881925 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f61bc0b2e40 msgr2=0x7f61bc1f89a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:43.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:43.788+0000 7f61c260e640 1 --2- 172.21.15.38:0/268881925 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f61bc0b2e40 0x7f61bc1f89a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:43.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:43.788+0000 7f61c260e640 1 -- 172.21.15.38:0/268881925 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61bc15c430 con 0x7f61bc158b40 2024-09-21T03:34:43.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:43.788+0000 7f61c260e640 1 --2- 172.21.15.38:0/268881925 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f61bc158b40 0x7f61bc158f50 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f61b400ba10 tx=0x7f61b400bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:43.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:43.789+0000 7f61ae7fc640 1 -- 172.21.15.38:0/268881925 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61b4019070 con 0x7f61bc158b40 2024-09-21T03:34:43.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:43.789+0000 7f61ae7fc640 1 -- 172.21.15.38:0/268881925 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f61b400cc30 con 0x7f61bc158b40 2024-09-21T03:34:43.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:43.789+0000 7f61c4071640 1 -- 172.21.15.38:0/268881925 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61bc15c710 con 0x7f61bc158b40 2024-09-21T03:34:43.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:43.789+0000 7f61ae7fc640 1 -- 172.21.15.38:0/268881925 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61b400ce60 con 0x7f61bc158b40 2024-09-21T03:34:43.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:43.789+0000 7f61c4071640 1 -- 172.21.15.38:0/268881925 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61bc15ccd0 con 0x7f61bc158b40 2024-09-21T03:34:43.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:43.790+0000 7f61c4071640 1 -- 172.21.15.38:0/268881925 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61bc1e8a10 con 0x7f61bc158b40 2024-09-21T03:34:43.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:43.790+0000 7f61ae7fc640 1 -- 172.21.15.38:0/268881925 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f61b401c070 con 0x7f61bc158b40 2024-09-21T03:34:43.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:43.791+0000 7f61ae7fc640 1 --2- 172.21.15.38:0/268881925 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f61a0078b90 0x7f61a007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:43.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:43.791+0000 7f61ae7fc640 1 -- 172.21.15.38:0/268881925 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7f61b409a990 con 0x7f61bc158b40 2024-09-21T03:34:43.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:43.791+0000 7f61c1e0d640 1 --2- 172.21.15.38:0/268881925 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f61a0078b90 0x7f61a007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:43.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:43.792+0000 7f61c1e0d640 1 --2- 172.21.15.38:0/268881925 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f61a0078b90 0x7f61a007b0a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f61b8000f60 tx=0x7f61b8009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:43.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:43.796+0000 7f61ae7fc640 1 -- 172.21.15.38:0/268881925 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f61b4062680 con 0x7f61bc158b40 2024-09-21T03:34:44.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.055+0000 7f61c4071640 1 -- 172.21.15.38:0/268881925 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f61bc15eff0 con 0x7f61bc158b40 2024-09-21T03:34:44.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.058+0000 7f61ae7fc640 1 -- 172.21.15.38:0/268881925 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v18) ==== 190+0+147 (secure 0 0 0) 0x7f61b40670b0 con 0x7f61bc158b40 2024-09-21T03:34:44.060 INFO:teuthology.orchestra.run.smithi038.stdout:[client.0] 2024-09-21T03:34:44.060 INFO:teuthology.orchestra.run.smithi038.stdout: key = AQBQPu5m6ydBDxAAlWZ2XzQTPBcYPriPGRJVeA== 2024-09-21T03:34:44.060 INFO:teuthology.orchestra.run.smithi038.stdout: caps mds = "allow" 2024-09-21T03:34:44.060 INFO:teuthology.orchestra.run.smithi038.stdout: caps mon = "allow r" 2024-09-21T03:34:44.060 INFO:teuthology.orchestra.run.smithi038.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-21T03:34:44.060 INFO:teuthology.orchestra.run.smithi038.stderr:updated caps for client.0 2024-09-21T03:34:44.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.060+0000 7f61c4071640 1 -- 172.21.15.38:0/268881925 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f61a0078b90 msgr2=0x7f61a007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:44.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.060+0000 7f61c4071640 1 --2- 172.21.15.38:0/268881925 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f61a0078b90 0x7f61a007b0a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f61b8000f60 tx=0x7f61b8009040 comp rx=0 tx=0).stop 2024-09-21T03:34:44.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.060+0000 7f61c4071640 1 -- 172.21.15.38:0/268881925 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f61bc158b40 msgr2=0x7f61bc158f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:44.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.060+0000 7f61c4071640 1 --2- 172.21.15.38:0/268881925 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f61bc158b40 0x7f61bc158f50 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f61b400ba10 tx=0x7f61b400bed0 comp rx=0 tx=0).stop 2024-09-21T03:34:44.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.060+0000 7f61c4071640 1 -- 172.21.15.38:0/268881925 shutdown_connections 2024-09-21T03:34:44.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.060+0000 7f61c4071640 1 --2- 172.21.15.38:0/268881925 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f61bc158b40 0x7f61bc158f50 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:44.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.060+0000 7f61c4071640 1 --2- 172.21.15.38:0/268881925 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f61a0078b90 0x7f61a007b0a0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:44.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.060+0000 7f61c4071640 1 --2- 172.21.15.38:0/268881925 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f61bc162280 0x7f61bc1f8ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:44.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.060+0000 7f61c4071640 1 --2- 172.21.15.38:0/268881925 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f61bc0b2e40 0x7f61bc1f89a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:44.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.060+0000 7f61c4071640 1 -- 172.21.15.38:0/268881925 >> 172.21.15.38:0/268881925 conn(0x7f61bc0b6b20 msgr2=0x7f61bc15e4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:44.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.060+0000 7f61c4071640 1 -- 172.21.15.38:0/268881925 shutdown_connections 2024-09-21T03:34:44.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.060+0000 7f61c4071640 1 -- 172.21.15.38:0/268881925 wait complete. 2024-09-21T03:34:44.073 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:34:44.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.189+0000 7fc59a7c3640 1 -- 172.21.15.38:0/126491719 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc594162280 msgr2=0x7fc594162690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:44.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.189+0000 7fc59a7c3640 1 --2- 172.21.15.38:0/126491719 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc594162280 0x7fc594162690 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fc5940d5040 tx=0x7fc58802a560 comp rx=0 tx=0).stop 2024-09-21T03:34:44.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.189+0000 7fc59a7c3640 1 -- 172.21.15.38:0/126491719 shutdown_connections 2024-09-21T03:34:44.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.189+0000 7fc59a7c3640 1 --2- 172.21.15.38:0/126491719 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc594162280 0x7fc594162690 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:44.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.189+0000 7fc59a7c3640 1 --2- 172.21.15.38:0/126491719 >> v2:172.21.15.195:3300/0 conn(0x7fc594162f00 0x7fc5940d24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:44.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.189+0000 7fc59a7c3640 1 -- 172.21.15.38:0/126491719 >> 172.21.15.38:0/126491719 conn(0x7fc5940b6b20 msgr2=0x7fc5940d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:44.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.189+0000 7fc59a7c3640 1 -- 172.21.15.38:0/126491719 shutdown_connections 2024-09-21T03:34:44.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.189+0000 7fc59a7c3640 1 -- 172.21.15.38:0/126491719 wait complete. 2024-09-21T03:34:44.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.191+0000 7fc59a7c3640 1 Processor -- start 2024-09-21T03:34:44.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.191+0000 7fc59a7c3640 1 -- start start 2024-09-21T03:34:44.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.192+0000 7fc59a7c3640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc5940b2e40 0x7fc5940b16d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:44.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.192+0000 7fc59a7c3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc594162280 0x7fc5940ce940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:44.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.192+0000 7fc59a7c3640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc5941f9010 0x7fc5940b7ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:44.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.192+0000 7fc59a7c3640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fc59415e630 con 0x7fc594162280 2024-09-21T03:34:44.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.192+0000 7fc59a7c3640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fc59415e4b0 con 0x7fc5940b2e40 2024-09-21T03:34:44.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.192+0000 7fc59a7c3640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fc59415e7b0 con 0x7fc5941f9010 2024-09-21T03:34:44.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.192+0000 7fc5937fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc594162280 0x7fc5940ce940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:44.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.192+0000 7fc5937fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc594162280 0x7fc5940ce940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50102/0 (socket says 172.21.15.38:50102) 2024-09-21T03:34:44.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.192+0000 7fc5937fe640 1 -- 172.21.15.38:0/3245973292 learned_addr learned my addr 172.21.15.38:0/3245973292 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:34:44.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.192+0000 7fc598d60640 1 --2- 172.21.15.38:0/3245973292 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc5941f9010 0x7fc5940b7ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:44.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.192+0000 7fc593fff640 1 --2- 172.21.15.38:0/3245973292 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc5940b2e40 0x7fc5940b16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:44.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.192+0000 7fc5937fe640 1 -- 172.21.15.38:0/3245973292 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc5941f9010 msgr2=0x7fc5940b7ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:44.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.192+0000 7fc5937fe640 1 --2- 172.21.15.38:0/3245973292 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc5941f9010 0x7fc5940b7ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:44.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.193+0000 7fc5937fe640 1 -- 172.21.15.38:0/3245973292 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc5940b2e40 msgr2=0x7fc5940b16d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:44.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.193+0000 7fc5937fe640 1 --2- 172.21.15.38:0/3245973292 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc5940b2e40 0x7fc5940b16d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:44.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.193+0000 7fc5937fe640 1 -- 172.21.15.38:0/3245973292 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc59414d930 con 0x7fc594162280 2024-09-21T03:34:44.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.193+0000 7fc593fff640 1 --2- 172.21.15.38:0/3245973292 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc5940b2e40 0x7fc5940b16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:34:44.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.193+0000 7fc5937fe640 1 --2- 172.21.15.38:0/3245973292 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc594162280 0x7fc5940ce940 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fc588034040 tx=0x7fc5880039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:44.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.193+0000 7fc590ff9640 1 -- 172.21.15.38:0/3245973292 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc58803e030 con 0x7fc594162280 2024-09-21T03:34:44.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.193+0000 7fc590ff9640 1 -- 172.21.15.38:0/3245973292 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc5880047a0 con 0x7fc594162280 2024-09-21T03:34:44.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.193+0000 7fc59a7c3640 1 -- 172.21.15.38:0/3245973292 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5940ac360 con 0x7fc594162280 2024-09-21T03:34:44.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.193+0000 7fc590ff9640 1 -- 172.21.15.38:0/3245973292 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc588035040 con 0x7fc594162280 2024-09-21T03:34:44.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.193+0000 7fc59a7c3640 1 -- 172.21.15.38:0/3245973292 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5940ac920 con 0x7fc594162280 2024-09-21T03:34:44.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.194+0000 7fc59a7c3640 1 -- 172.21.15.38:0/3245973292 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc594162f00 con 0x7fc594162280 2024-09-21T03:34:44.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.195+0000 7fc590ff9640 1 -- 172.21.15.38:0/3245973292 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc588041030 con 0x7fc594162280 2024-09-21T03:34:44.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.195+0000 7fc590ff9640 1 --2- 172.21.15.38:0/3245973292 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc580078bc0 0x7fc58007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:44.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.196+0000 7fc590ff9640 1 -- 172.21.15.38:0/3245973292 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7fc5880bb640 con 0x7fc594162280 2024-09-21T03:34:44.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.196+0000 7fc593fff640 1 --2- 172.21.15.38:0/3245973292 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc580078bc0 0x7fc58007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:44.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.196+0000 7fc593fff640 1 --2- 172.21.15.38:0/3245973292 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc580078bc0 0x7fc58007b0d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc584009aa0 tx=0x7fc584005db0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:44.202 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.201+0000 7fc590ff9640 1 -- 172.21.15.38:0/3245973292 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fc588083900 con 0x7fc594162280 2024-09-21T03:34:44.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.425+0000 7fc59a7c3640 1 -- 172.21.15.38:0/3245973292 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc5940cfc90 con 0x7fc594162280 2024-09-21T03:34:44.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.426+0000 7fc590ff9640 1 -- 172.21.15.38:0/3245973292 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5229 (secure 0 0 0) 0x7fc588083200 con 0x7fc594162280 2024-09-21T03:34:44.428 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:34:44.428 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":17,"btime":"2024-09-21T03:34:37:631111+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4757,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/1905955399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":1905955399},{"type":"v1","addr":"172.21.15.195:6836","nonce":1905955399}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4936,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/2989374027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":2989374027},{"type":"v1","addr":"172.21.15.195:6837","nonce":2989374027}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4942,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/3531731501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6833","nonce":3531731501},{"type":"v1","addr":"172.21.15.38:6835","nonce":3531731501}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:34:34.009269+0000","modified":"2024-09-21T03:34:37.009380+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5013},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5013":{"gid":5013,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.38:6834/1865253656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":1865253656},{"type":"v1","addr":"172.21.15.38:6834","nonce":1865253656}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5013,"qdb_cluster":[5013]},"id":2}]} 2024-09-21T03:34:44.428 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 17 2024-09-21T03:34:44.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.428+0000 7fc59a7c3640 1 -- 172.21.15.38:0/3245973292 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc580078bc0 msgr2=0x7fc58007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:44.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.428+0000 7fc59a7c3640 1 --2- 172.21.15.38:0/3245973292 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc580078bc0 0x7fc58007b0d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc584009aa0 tx=0x7fc584005db0 comp rx=0 tx=0).stop 2024-09-21T03:34:44.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.428+0000 7fc59a7c3640 1 -- 172.21.15.38:0/3245973292 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc594162280 msgr2=0x7fc5940ce940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:44.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.428+0000 7fc59a7c3640 1 --2- 172.21.15.38:0/3245973292 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc594162280 0x7fc5940ce940 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fc588034040 tx=0x7fc5880039e0 comp rx=0 tx=0).stop 2024-09-21T03:34:44.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.428+0000 7fc59a7c3640 1 -- 172.21.15.38:0/3245973292 shutdown_connections 2024-09-21T03:34:44.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.428+0000 7fc59a7c3640 1 --2- 172.21.15.38:0/3245973292 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc580078bc0 0x7fc58007b0d0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:44.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.428+0000 7fc59a7c3640 1 --2- 172.21.15.38:0/3245973292 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc5941f9010 0x7fc5940b7ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:44.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.428+0000 7fc59a7c3640 1 --2- 172.21.15.38:0/3245973292 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc594162280 0x7fc5940ce940 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:44.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.428+0000 7fc59a7c3640 1 --2- 172.21.15.38:0/3245973292 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc5940b2e40 0x7fc5940b16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:44.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.428+0000 7fc59a7c3640 1 -- 172.21.15.38:0/3245973292 >> 172.21.15.38:0/3245973292 conn(0x7fc5940b6b20 msgr2=0x7fc59415da30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:44.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.428+0000 7fc59a7c3640 1 -- 172.21.15.38:0/3245973292 shutdown_connections 2024-09-21T03:34:44.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.428+0000 7fc59a7c3640 1 -- 172.21.15.38:0/3245973292 wait complete. 2024-09-21T03:34:44.843 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T03:34:34.009269+0000', 'modified': '2024-09-21T03:34:37.009380+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5013}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5013': {'gid': 5013, 'name': 'c', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.38:6834/1865253656', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.38:6832', 'nonce': 1865253656}, {'type': 'v1', 'addr': '172.21.15.38:6834', 'nonce': 1865253656}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5013, 'qdb_cluster': [5013]} 2024-09-21T03:34:44.843 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-21T03:34:44.844 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-21T03:34:44.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.965+0000 7f4fa88ec640 1 -- 172.21.15.38:0/679682809 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4fa40cd450 msgr2=0x7f4fa40cd860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:44.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.965+0000 7f4fa88ec640 1 --2- 172.21.15.38:0/679682809 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4fa40cd450 0x7f4fa40cd860 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f4f9c005fd0 tx=0x7f4f9c02a4a0 comp rx=0 tx=0).stop 2024-09-21T03:34:44.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.965+0000 7f4fa88ec640 1 -- 172.21.15.38:0/679682809 shutdown_connections 2024-09-21T03:34:44.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.965+0000 7f4fa88ec640 1 --2- 172.21.15.38:0/679682809 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4fa40cd450 0x7f4fa40cd860 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:44.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.965+0000 7f4fa88ec640 1 --2- 172.21.15.38:0/679682809 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4fa415ead0 0x7f4fa4162e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:44.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.965+0000 7f4fa88ec640 1 -- 172.21.15.38:0/679682809 >> 172.21.15.38:0/679682809 conn(0x7f4fa40b46e0 msgr2=0x7f4fa40d23e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:44.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.965+0000 7f4fa88ec640 1 -- 172.21.15.38:0/679682809 shutdown_connections 2024-09-21T03:34:44.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.965+0000 7f4fa88ec640 1 -- 172.21.15.38:0/679682809 wait complete. 2024-09-21T03:34:44.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.967+0000 7f4fa88ec640 1 Processor -- start 2024-09-21T03:34:44.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.967+0000 7f4fa88ec640 1 -- start start 2024-09-21T03:34:44.972 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.969+0000 7f4fa88ec640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4fa40b5bd0 0x7f4fa414da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:44.972 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.969+0000 7f4fa88ec640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4fa40cd450 0x7f4fa414c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:44.972 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.969+0000 7f4fa88ec640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4fa415ead0 0x7f4fa414ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:44.972 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.969+0000 7f4fa88ec640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f4fa40cdf20 con 0x7f4fa40cd450 2024-09-21T03:34:44.972 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.969+0000 7f4fa88ec640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f4fa40cdda0 con 0x7f4fa40b5bd0 2024-09-21T03:34:44.972 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.969+0000 7f4fa88ec640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f4fa40ce0a0 con 0x7f4fa415ead0 2024-09-21T03:34:44.972 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.969+0000 7f4fa1d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4fa40cd450 0x7f4fa414c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:44.972 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.969+0000 7f4fa1d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4fa40cd450 0x7f4fa414c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50118/0 (socket says 172.21.15.38:50118) 2024-09-21T03:34:44.972 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.969+0000 7f4fa1d9b640 1 -- 172.21.15.38:0/2805463454 learned_addr learned my addr 172.21.15.38:0/2805463454 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:34:44.972 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.969+0000 7f4fa259c640 1 --2- 172.21.15.38:0/2805463454 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4fa40b5bd0 0x7f4fa414da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:44.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.969+0000 7f4fa2d9d640 1 --2- 172.21.15.38:0/2805463454 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4fa415ead0 0x7f4fa414ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:44.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.969+0000 7f4fa1d9b640 1 -- 172.21.15.38:0/2805463454 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4fa415ead0 msgr2=0x7f4fa414ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:44.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.969+0000 7f4fa1d9b640 1 --2- 172.21.15.38:0/2805463454 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4fa415ead0 0x7f4fa414ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:44.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.969+0000 7f4fa1d9b640 1 -- 172.21.15.38:0/2805463454 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4fa40b5bd0 msgr2=0x7f4fa414da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:44.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.969+0000 7f4fa1d9b640 1 --2- 172.21.15.38:0/2805463454 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4fa40b5bd0 0x7f4fa414da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:44.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.969+0000 7f4fa1d9b640 1 -- 172.21.15.38:0/2805463454 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fa40ca1a0 con 0x7f4fa40cd450 2024-09-21T03:34:44.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.969+0000 7f4fa1d9b640 1 --2- 172.21.15.38:0/2805463454 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4fa40cd450 0x7f4fa414c800 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f4f8c00d720 tx=0x7f4f8c00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:44.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.969+0000 7f4f9affd640 1 -- 172.21.15.38:0/2805463454 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f8c017070 con 0x7f4fa40cd450 2024-09-21T03:34:44.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.970+0000 7f4fa88ec640 1 -- 172.21.15.38:0/2805463454 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fa414e370 con 0x7f4fa40cd450 2024-09-21T03:34:44.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.970+0000 7f4f9affd640 1 -- 172.21.15.38:0/2805463454 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4f8c009b10 con 0x7f4fa40cd450 2024-09-21T03:34:44.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.970+0000 7f4f9affd640 1 -- 172.21.15.38:0/2805463454 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f8c010460 con 0x7f4fa40cd450 2024-09-21T03:34:44.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.970+0000 7f4fa88ec640 1 -- 172.21.15.38:0/2805463454 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fa415f510 con 0x7f4fa40cd450 2024-09-21T03:34:44.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.972+0000 7f4f9affd640 1 -- 172.21.15.38:0/2805463454 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f4f8c01a070 con 0x7f4fa40cd450 2024-09-21T03:34:44.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.972+0000 7f4f9affd640 1 --2- 172.21.15.38:0/2805463454 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4f84078b90 0x7f4f8407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:44.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.972+0000 7f4f9affd640 1 -- 172.21.15.38:0/2805463454 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7f4f8c09b050 con 0x7f4fa40cd450 2024-09-21T03:34:44.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.972+0000 7f4fa259c640 1 --2- 172.21.15.38:0/2805463454 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4f84078b90 0x7f4f8407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:44.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.973+0000 7f4faf858740 1 Processor -- start 2024-09-21T03:34:44.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.973+0000 7f4fa259c640 1 --2- 172.21.15.38:0/2805463454 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4f84078b90 0x7f4f8407b0a0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f4f9400b450 tx=0x7f4f9400d040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:44.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.973+0000 7f4faf858740 1 -- start start 2024-09-21T03:34:44.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.974+0000 7f4faf858740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5627414b5270 0x562741886fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:44.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.974+0000 7f4faf858740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56274148c3b0 0x562741887500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:44.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.974+0000 7f4faf858740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562741494590 0x5627418a4400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:44.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.974+0000 7f4faf858740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5627417cf820 con 0x56274148c3b0 2024-09-21T03:34:44.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.974+0000 7f4faf858740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5627417bec50 con 0x5627414b5270 2024-09-21T03:34:44.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.974+0000 7f4faf858740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x5627417c2a60 con 0x562741494590 2024-09-21T03:34:44.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.974+0000 7f4fa1d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56274148c3b0 0x562741887500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:44.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.974+0000 7f4fa1d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56274148c3b0 0x562741887500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50122/0 (socket says 172.21.15.38:50122) 2024-09-21T03:34:44.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.974+0000 7f4fa1d9b640 1 -- 172.21.15.38:0/951956284 learned_addr learned my addr 172.21.15.38:0/951956284 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:34:44.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.975+0000 7f4fa2d9d640 1 --2- 172.21.15.38:0/951956284 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562741494590 0x5627418a4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:44.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.975+0000 7f4fa259c640 1 --2- 172.21.15.38:0/951956284 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5627414b5270 0x562741886fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:44.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.975+0000 7f4fa1d9b640 1 -- 172.21.15.38:0/951956284 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562741494590 msgr2=0x5627418a4400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:44.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.975+0000 7f4fa1d9b640 1 --2- 172.21.15.38:0/951956284 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562741494590 0x5627418a4400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:44.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.975+0000 7f4fa1d9b640 1 -- 172.21.15.38:0/951956284 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5627414b5270 msgr2=0x562741886fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:44.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.975+0000 7f4fa1d9b640 1 --2- 172.21.15.38:0/951956284 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5627414b5270 0x562741886fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:44.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.975+0000 7f4fa1d9b640 1 -- 172.21.15.38:0/951956284 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562741520850 con 0x56274148c3b0 2024-09-21T03:34:44.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.975+0000 7f4fa1d9b640 1 --2- 172.21.15.38:0/951956284 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56274148c3b0 0x562741887500 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f4f8c00bfd0 tx=0x7f4f8c062800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8a2a41198fb2c23 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:44.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.975+0000 7f4f83fff640 1 -- 172.21.15.38:0/951956284 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f8c021070 con 0x56274148c3b0 2024-09-21T03:34:44.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.975+0000 7f4f83fff640 1 -- 172.21.15.38:0/951956284 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4f8c06c5a0 con 0x56274148c3b0 2024-09-21T03:34:44.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.975+0000 7f4f83fff640 1 -- 172.21.15.38:0/951956284 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f8c061e90 con 0x56274148c3b0 2024-09-21T03:34:44.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.976+0000 7f4faf858740 1 -- 172.21.15.38:0/951956284 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56274148c3b0 msgr2=0x562741887500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:44.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.976+0000 7f4faf858740 1 --2- 172.21.15.38:0/951956284 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56274148c3b0 0x562741887500 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f4f8c00bfd0 tx=0x7f4f8c062800 comp rx=0 tx=0).stop 2024-09-21T03:34:44.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.976+0000 7f4faf858740 1 -- 172.21.15.38:0/951956284 shutdown_connections 2024-09-21T03:34:44.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.976+0000 7f4faf858740 1 --2- 172.21.15.38:0/951956284 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562741494590 0x5627418a4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:44.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.976+0000 7f4faf858740 1 --2- 172.21.15.38:0/951956284 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56274148c3b0 0x562741887500 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:44.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.976+0000 7f4faf858740 1 --2- 172.21.15.38:0/951956284 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5627414b5270 0x562741886fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:44.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.976+0000 7f4faf858740 1 -- 172.21.15.38:0/951956284 >> 172.21.15.38:0/951956284 conn(0x5627414a93c0 msgr2=0x56274185d280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:44.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.976+0000 7f4faf858740 1 -- 172.21.15.38:0/951956284 shutdown_connections 2024-09-21T03:34:44.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.976+0000 7f4faf858740 1 -- 172.21.15.38:0/951956284 wait complete. 2024-09-21T03:34:44.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.977+0000 7f4faf858740 1 -- start start 2024-09-21T03:34:44.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.977+0000 7f4faf858740 1 Processor -- start 2024-09-21T03:34:44.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.978+0000 7f4faf858740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56274148c3b0 0x562741871150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:44.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.978+0000 7f4faf858740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562741494590 0x5627418b4730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:44.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.979+0000 7f4faf858740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5627414b5270 0x562741873f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:44.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.979+0000 7f4faf858740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x56274152f620 con 0x56274148c3b0 2024-09-21T03:34:44.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.979+0000 7f4faf858740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5627417645f0 con 0x562741494590 2024-09-21T03:34:44.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.979+0000 7f4faf858740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x562741887a40 con 0x5627414b5270 2024-09-21T03:34:44.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.979+0000 7f4faf858740 10 client.0 _ll_register_callbacks cb 0x5627417680a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:34:44.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.979+0000 7f4fa259c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56274148c3b0 0x562741871150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:44.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.979+0000 7f4fa259c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56274148c3b0 0x562741871150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50132/0 (socket says 172.21.15.38:50132) 2024-09-21T03:34:44.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.979+0000 7f4fa259c640 1 -- 172.21.15.38:0/2001027909 learned_addr learned my addr 172.21.15.38:0/2001027909 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:34:44.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.979+0000 7f4fa2d9d640 1 --2- 172.21.15.38:0/2001027909 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5627414b5270 0x562741873f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:44.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.979+0000 7f4fa1d9b640 1 --2- 172.21.15.38:0/2001027909 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562741494590 0x5627418b4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:44.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.979+0000 7f4fa259c640 1 -- 172.21.15.38:0/2001027909 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5627414b5270 msgr2=0x562741873f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:44.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.979+0000 7f4fa259c640 1 --2- 172.21.15.38:0/2001027909 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5627414b5270 0x562741873f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:44.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.979+0000 7f4fa259c640 1 -- 172.21.15.38:0/2001027909 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562741494590 msgr2=0x5627418b4730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:44.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.979+0000 7f4fa259c640 1 --2- 172.21.15.38:0/2001027909 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562741494590 0x5627418b4730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:44.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.979+0000 7f4fa259c640 1 -- 172.21.15.38:0/2001027909 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5627415976b0 con 0x56274148c3b0 2024-09-21T03:34:44.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.979+0000 7f4fa259c640 1 --2- 172.21.15.38:0/2001027909 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56274148c3b0 0x562741871150 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f4f94016690 tx=0x7f4f94004a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f07a10d4f9e9a560 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:44.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.979+0000 7f4f817fa640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-21T03:34:44.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.979+0000 7f4f817fa640 1 -- 172.21.15.38:0/2001027909 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f94003970 con 0x56274148c3b0 2024-09-21T03:34:44.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.979+0000 7f4f817fa640 1 -- 172.21.15.38:0/2001027909 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7f4f94003b10 con 0x56274148c3b0 2024-09-21T03:34:44.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.979+0000 7f4faf858740 1 -- 172.21.15.38:0/2001027909 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f8c06c5a0 con 0x56274148c3b0 2024-09-21T03:34:44.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.979+0000 7f4f817fa640 1 -- 172.21.15.38:0/2001027909 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f4f9402f050 con 0x56274148c3b0 2024-09-21T03:34:44.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.979+0000 7f4faf858740 10 client.5193 fetch_fsmap learned FSMap version 17 2024-09-21T03:34:44.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.979+0000 7f4faf858740 1 -- 172.21.15.38:0/2001027909 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x56274189aff0 con 0x56274148c3b0 2024-09-21T03:34:44.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.980+0000 7f4f817fa640 1 -- 172.21.15.38:0/2001027909 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4f8c06c5a0 con 0x56274148c3b0 2024-09-21T03:34:44.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.980+0000 7f4f817fa640 1 -- 172.21.15.38:0/2001027909 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 17) ==== 3502+0+0 (secure 0 0 0) 0x7f4f94032020 con 0x56274148c3b0 2024-09-21T03:34:44.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.980+0000 7f4faf858740 10 client.5193 fetch_fsmap finished waiting for FSMap version 17 2024-09-21T03:34:44.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.980+0000 7f4faf858740 10 client.5193 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T03:34:44.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.980+0000 7f4faf858740 20 client.5193 populate_metadata read hostname 'smithi038' 2024-09-21T03:34:44.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.980+0000 7f4faf858740 1 --2- 172.21.15.38:0/2001027909 >> [v2:172.21.15.38:6832/1865253656,v1:172.21.15.38:6834/1865253656] conn(0x562741473e40 0x5627418746d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:44.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.980+0000 7f4faf858740 4 client.5193 mds_command: new command op to 5013 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:34:44.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.980+0000 7f4faf858740 1 -- 172.21.15.38:0/2001027909 --> [v2:172.21.15.38:6832/1865253656,v1:172.21.15.38:6834/1865253656] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5627417d1770 con 0x562741473e40 2024-09-21T03:34:44.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.980+0000 7f4fa1d9b640 1 --2- 172.21.15.38:0/2001027909 >> [v2:172.21.15.38:6832/1865253656,v1:172.21.15.38:6834/1865253656] conn(0x562741473e40 0x5627418746d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:44.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.980+0000 7f4f817fa640 10 client.5193 ms_handle_connect on v2:172.21.15.38:6832/1865253656 2024-09-21T03:34:44.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.980+0000 7f4fa1d9b640 1 --2- 172.21.15.38:0/2001027909 >> [v2:172.21.15.38:6832/1865253656,v1:172.21.15.38:6834/1865253656] conn(0x562741473e40 0x5627418746d0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2efcf5238690ba42 server_cookie=c2a138fe42c7cb8d in_seq=0 out_seq=0 2024-09-21T03:34:44.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.981+0000 7f4f817fa640 1 -- 172.21.15.38:0/2001027909 <== mds.0 v2:172.21.15.38:6832/1865253656 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5627417d1770 con 0x562741473e40 2024-09-21T03:34:44.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.981+0000 7f4f817fa640 10 client.5193 handle_command_reply: tid=0 2024-09-21T03:34:44.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.981+0000 7f4faf858740 1 client.5193 shutdown 2024-09-21T03:34:44.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.981+0000 7f4faf858740 1 -- 172.21.15.38:0/2001027909 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56274148c3b0 msgr2=0x562741871150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:44.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.981+0000 7f4faf858740 1 --2- 172.21.15.38:0/2001027909 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56274148c3b0 0x562741871150 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f4f94016690 tx=0x7f4f94004a00 comp rx=0 tx=0).stop 2024-09-21T03:34:44.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.981+0000 7f4faf858740 1 -- 172.21.15.38:0/2001027909 shutdown_connections 2024-09-21T03:34:44.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.981+0000 7f4faf858740 1 --2- 172.21.15.38:0/2001027909 >> [v2:172.21.15.38:6832/1865253656,v1:172.21.15.38:6834/1865253656] conn(0x562741473e40 0x5627418746d0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:44.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.981+0000 7f4faf858740 1 --2- 172.21.15.38:0/2001027909 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5627414b5270 0x562741873f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:44.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.981+0000 7f4faf858740 1 --2- 172.21.15.38:0/2001027909 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562741494590 0x5627418b4730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:44.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.981+0000 7f4f817fa640 0 client.5193 ms_handle_reset on v2:172.21.15.38:6832/1865253656 2024-09-21T03:34:44.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.981+0000 7f4faf858740 1 --2- 172.21.15.38:0/2001027909 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56274148c3b0 0x562741871150 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:44.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.981+0000 7f4faf858740 1 -- 172.21.15.38:0/2001027909 >> 172.21.15.38:0/2001027909 conn(0x5627414a93c0 msgr2=0x562741857b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:44.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.981+0000 7f4faf858740 1 -- 172.21.15.38:0/2001027909 shutdown_connections 2024-09-21T03:34:44.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.981+0000 7f4faf858740 1 -- 172.21.15.38:0/2001027909 wait complete. 2024-09-21T03:34:44.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:44.982+0000 7f4faf858740 20 client.5193 trim_cache size 0 max 16384 2024-09-21T03:34:45.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:45.000+0000 7f4faf858740 1 Processor -- start 2024-09-21T03:34:45.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:45.000+0000 7f4faf858740 1 -- start start 2024-09-21T03:34:45.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:45.001+0000 7f4faf858740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56274148c3b0 0x56274185d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:45.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:45.001+0000 7f4faf858740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562741494590 0x5627418b29d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:45.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:45.001+0000 7f4faf858740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5627414b5270 0x5627418b2f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:45.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:45.001+0000 7f4faf858740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f4fa42016b0 con 0x56274148c3b0 2024-09-21T03:34:45.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:45.001+0000 7f4faf858740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f4fa4160aa0 con 0x5627414b5270 2024-09-21T03:34:45.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:45.001+0000 7f4faf858740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f4fa4201060 con 0x562741494590 2024-09-21T03:34:45.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:45.002+0000 7f4fa259c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56274148c3b0 0x56274185d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:45.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:45.002+0000 7f4fa259c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56274148c3b0 0x56274185d300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50146/0 (socket says 172.21.15.38:50146) 2024-09-21T03:34:45.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:45.002+0000 7f4fa259c640 1 -- 172.21.15.38:0/2460142906 learned_addr learned my addr 172.21.15.38:0/2460142906 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:34:45.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:45.002+0000 7f4fa1d9b640 1 --2- 172.21.15.38:0/2460142906 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562741494590 0x5627418b29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:45.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:45.002+0000 7f4fa2d9d640 1 --2- 172.21.15.38:0/2460142906 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5627414b5270 0x5627418b2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:45.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:45.002+0000 7f4fa259c640 1 -- 172.21.15.38:0/2460142906 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562741494590 msgr2=0x5627418b29d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:45.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:45.002+0000 7f4fa259c640 1 --2- 172.21.15.38:0/2460142906 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562741494590 0x5627418b29d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:45.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:45.002+0000 7f4fa259c640 1 -- 172.21.15.38:0/2460142906 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5627414b5270 msgr2=0x5627418b2f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:45.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:45.002+0000 7f4fa259c640 1 --2- 172.21.15.38:0/2460142906 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5627414b5270 0x5627418b2f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:45.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:45.002+0000 7f4fa259c640 1 -- 172.21.15.38:0/2460142906 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5627418b41a0 con 0x56274148c3b0 2024-09-21T03:34:45.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:45.002+0000 7f4fa259c640 1 --2- 172.21.15.38:0/2460142906 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56274148c3b0 0x56274185d300 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f4f94017fe0 tx=0x7f4f94003ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=917dd01fefe0ea08 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:45.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:45.002+0000 7f4f837fe640 1 -- 172.21.15.38:0/2460142906 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f94003970 con 0x56274148c3b0 2024-09-21T03:34:45.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:45.002+0000 7f4f837fe640 1 -- 172.21.15.38:0/2460142906 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4f94038070 con 0x56274148c3b0 2024-09-21T03:34:45.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:45.002+0000 7f4f837fe640 1 -- 172.21.15.38:0/2460142906 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f94004230 con 0x56274148c3b0 2024-09-21T03:34:45.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:45.003+0000 7f4faf858740 1 -- 172.21.15.38:0/2460142906 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56274148c3b0 msgr2=0x56274185d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:45.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:45.003+0000 7f4faf858740 1 --2- 172.21.15.38:0/2460142906 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56274148c3b0 0x56274185d300 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f4f94017fe0 tx=0x7f4f94003ed0 comp rx=0 tx=0).stop 2024-09-21T03:34:45.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:45.003+0000 7f4faf858740 1 -- 172.21.15.38:0/2460142906 shutdown_connections 2024-09-21T03:34:45.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:45.003+0000 7f4faf858740 1 --2- 172.21.15.38:0/2460142906 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5627414b5270 0x5627418b2f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:45.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:45.003+0000 7f4faf858740 1 --2- 172.21.15.38:0/2460142906 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562741494590 0x5627418b29d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:45.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:45.003+0000 7f4faf858740 1 --2- 172.21.15.38:0/2460142906 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56274148c3b0 0x56274185d300 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:45.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:45.003+0000 7f4faf858740 1 -- 172.21.15.38:0/2460142906 >> 172.21.15.38:0/2460142906 conn(0x5627414a93c0 msgr2=0x5627418a5350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:45.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:45.003+0000 7f4faf858740 1 -- 172.21.15.38:0/2460142906 shutdown_connections 2024-09-21T03:34:45.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:45.003+0000 7f4faf858740 1 -- 172.21.15.38:0/2460142906 wait complete. 2024-09-21T03:34:45.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:45.004+0000 7f4faf858740 1 -- start start 2024-09-21T03:34:45.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:45.004+0000 7f4faf858740 1 Processor -- start 2024-09-21T03:34:45.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:45.005+0000 7f4faf858740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56274148c3b0 0x562741874690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:45.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:45.005+0000 7f4faf858740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562741494590 0x562741874bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:45.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:45.005+0000 7f4faf858740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5627414b5270 0x5627418ad780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:45.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:45.005+0000 7f4faf858740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x56274189bcc0 con 0x562741494590 2024-09-21T03:34:45.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:45.005+0000 7f4faf858740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5627418717d0 con 0x56274148c3b0 2024-09-21T03:34:45.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:45.005+0000 7f4faf858740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x562741898d60 con 0x5627414b5270 2024-09-21T03:34:45.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:45.006+0000 7f4faf858740 10 client.0 _ll_register_callbacks cb 0x7f4f9400a4a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:34:45.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:45.006+0000 7f4fa1d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562741494590 0x562741874bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:45.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:45.006+0000 7f4fa1d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562741494590 0x562741874bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50156/0 (socket says 172.21.15.38:50156) 2024-09-21T03:34:45.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:45.006+0000 7f4fa1d9b640 1 -- 172.21.15.38:0/1192886017 learned_addr learned my addr 172.21.15.38:0/1192886017 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:34:45.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:45.006+0000 7f4fa259c640 1 --2- 172.21.15.38:0/1192886017 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56274148c3b0 0x562741874690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:45.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:45.006+0000 7f4fa2d9d640 1 --2- 172.21.15.38:0/1192886017 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5627414b5270 0x5627418ad780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:45.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:45.006+0000 7f4fa1d9b640 1 -- 172.21.15.38:0/1192886017 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5627414b5270 msgr2=0x5627418ad780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:45.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:45.006+0000 7f4fa1d9b640 1 --2- 172.21.15.38:0/1192886017 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5627414b5270 0x5627418ad780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:45.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:45.006+0000 7f4fa1d9b640 1 -- 172.21.15.38:0/1192886017 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56274148c3b0 msgr2=0x562741874690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:45.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:45.006+0000 7f4fa1d9b640 1 --2- 172.21.15.38:0/1192886017 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56274148c3b0 0x562741874690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:45.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:45.006+0000 7f4fa1d9b640 1 -- 172.21.15.38:0/1192886017 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56274185e620 con 0x562741494590 2024-09-21T03:34:45.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:45.006+0000 7f4fa259c640 1 --2- 172.21.15.38:0/1192886017 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56274148c3b0 0x562741874690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:34:45.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:45.006+0000 7f4fa2d9d640 1 --2- 172.21.15.38:0/1192886017 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5627414b5270 0x5627418ad780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:34:45.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:45.006+0000 7f4fa1d9b640 1 --2- 172.21.15.38:0/1192886017 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562741494590 0x562741874bd0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f4f8c062df0 tx=0x7f4f8c005bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5857348a2952a06 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:45.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:45.006+0000 7f4f817fa640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-21T03:34:45.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:45.006+0000 7f4f817fa640 1 -- 172.21.15.38:0/1192886017 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f8c021040 con 0x562741494590 2024-09-21T03:34:45.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:45.006+0000 7f4f817fa640 1 -- 172.21.15.38:0/1192886017 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7f4f8c062090 con 0x562741494590 2024-09-21T03:34:45.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:45.006+0000 7f4faf858740 1 -- 172.21.15.38:0/1192886017 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f94038070 con 0x562741494590 2024-09-21T03:34:45.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:45.006+0000 7f4f817fa640 1 -- 172.21.15.38:0/1192886017 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f4f8c05ead0 con 0x562741494590 2024-09-21T03:34:45.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:45.006+0000 7f4faf858740 10 client.5199 fetch_fsmap learned FSMap version 17 2024-09-21T03:34:45.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:45.006+0000 7f4faf858740 1 -- 172.21.15.38:0/1192886017 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x5627418a9ed0 con 0x562741494590 2024-09-21T03:34:45.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:45.007+0000 7f4f817fa640 1 -- 172.21.15.38:0/1192886017 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4f94038070 con 0x562741494590 2024-09-21T03:34:45.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:45.007+0000 7f4f817fa640 1 -- 172.21.15.38:0/1192886017 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 17) ==== 3502+0+0 (secure 0 0 0) 0x7f4f8c083080 con 0x562741494590 2024-09-21T03:34:45.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:45.007+0000 7f4faf858740 10 client.5199 fetch_fsmap finished waiting for FSMap version 17 2024-09-21T03:34:45.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:45.007+0000 7f4faf858740 10 client.5199 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T03:34:45.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:45.007+0000 7f4faf858740 20 client.5199 populate_metadata read hostname 'smithi038' 2024-09-21T03:34:45.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:45.007+0000 7f4faf858740 1 --2- 172.21.15.38:0/1192886017 >> [v2:172.21.15.38:6832/1865253656,v1:172.21.15.38:6834/1865253656] conn(0x562741921250 0x562741921660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:45.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:45.007+0000 7f4faf858740 4 client.5199 mds_command: new command op to 5013 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T03:34:45.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:45.007+0000 7f4faf858740 1 -- 172.21.15.38:0/1192886017 --> [v2:172.21.15.38:6832/1865253656,v1:172.21.15.38:6834/1865253656] -- command(tid 0: {"prefix": "status"}) -- 0x5627417cd220 con 0x562741921250 2024-09-21T03:34:45.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:45.007+0000 7f4fa259c640 1 --2- 172.21.15.38:0/1192886017 >> [v2:172.21.15.38:6832/1865253656,v1:172.21.15.38:6834/1865253656] conn(0x562741921250 0x562741921660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:45.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:45.007+0000 7f4f817fa640 10 client.5199 ms_handle_connect on v2:172.21.15.38:6832/1865253656 2024-09-21T03:34:45.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:45.007+0000 7f4fa259c640 1 --2- 172.21.15.38:0/1192886017 >> [v2:172.21.15.38:6832/1865253656,v1:172.21.15.38:6834/1865253656] conn(0x562741921250 0x562741921660 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=22ccea896b408d98 server_cookie=c12c8b9a22537f27 in_seq=0 out_seq=0 2024-09-21T03:34:45.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:45.008+0000 7f4f817fa640 1 -- 172.21.15.38:0/1192886017 <== mds.0 v2:172.21.15.38:6832/1865253656 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x5627417cd220 con 0x562741921250 2024-09-21T03:34:45.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:45.008+0000 7f4f817fa640 10 client.5199 handle_command_reply: tid=0 2024-09-21T03:34:45.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:45.008+0000 7f4faf858740 1 client.5199 shutdown 2024-09-21T03:34:45.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:45.008+0000 7f4faf858740 1 -- 172.21.15.38:0/1192886017 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562741494590 msgr2=0x562741874bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:45.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:45.008+0000 7f4faf858740 1 --2- 172.21.15.38:0/1192886017 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562741494590 0x562741874bd0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f4f8c062df0 tx=0x7f4f8c005bb0 comp rx=0 tx=0).stop 2024-09-21T03:34:45.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:45.008+0000 7f4faf858740 1 -- 172.21.15.38:0/1192886017 shutdown_connections 2024-09-21T03:34:45.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:45.008+0000 7f4faf858740 1 --2- 172.21.15.38:0/1192886017 >> [v2:172.21.15.38:6832/1865253656,v1:172.21.15.38:6834/1865253656] conn(0x562741921250 0x562741921660 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:45.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:45.008+0000 7f4faf858740 1 --2- 172.21.15.38:0/1192886017 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5627414b5270 0x5627418ad780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:45.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:45.008+0000 7f4faf858740 1 --2- 172.21.15.38:0/1192886017 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562741494590 0x562741874bd0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:45.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:45.008+0000 7f4f817fa640 0 client.5199 ms_handle_reset on v2:172.21.15.38:6832/1865253656 2024-09-21T03:34:45.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:45.008+0000 7f4faf858740 1 --2- 172.21.15.38:0/1192886017 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56274148c3b0 0x562741874690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:45.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:45.008+0000 7f4faf858740 1 -- 172.21.15.38:0/1192886017 >> 172.21.15.38:0/1192886017 conn(0x5627414a93c0 msgr2=0x56274185d300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:45.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:45.008+0000 7f4faf858740 1 -- 172.21.15.38:0/1192886017 shutdown_connections 2024-09-21T03:34:45.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:45.008+0000 7f4faf858740 1 -- 172.21.15.38:0/1192886017 wait complete. 2024-09-21T03:34:45.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:45.008+0000 7f4faf858740 20 client.5199 trim_cache size 0 max 16384 2024-09-21T03:34:45.016 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-09-21T03:34:45.016 INFO:teuthology.orchestra.run.smithi038.stdout: "cluster_fsid": "9d50d66f-656b-4fe2-8e3b-fe53b79c4d43", 2024-09-21T03:34:45.016 INFO:teuthology.orchestra.run.smithi038.stdout: "whoami": 0, 2024-09-21T03:34:45.016 INFO:teuthology.orchestra.run.smithi038.stdout: "id": 5013, 2024-09-21T03:34:45.016 INFO:teuthology.orchestra.run.smithi038.stdout: "want_state": "up:active", 2024-09-21T03:34:45.016 INFO:teuthology.orchestra.run.smithi038.stdout: "state": "up:active", 2024-09-21T03:34:45.016 INFO:teuthology.orchestra.run.smithi038.stdout: "fs_name": "cephfs", 2024-09-21T03:34:45.016 INFO:teuthology.orchestra.run.smithi038.stdout: "rank_uptime": 10.396261911, 2024-09-21T03:34:45.016 INFO:teuthology.orchestra.run.smithi038.stdout: "mdsmap_epoch": 17, 2024-09-21T03:34:45.016 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch": 40, 2024-09-21T03:34:45.016 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch_barrier": 40, 2024-09-21T03:34:45.016 INFO:teuthology.orchestra.run.smithi038.stdout: "uptime": 17.345969376999999 2024-09-21T03:34:45.016 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-09-21T03:34:45.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:45.009+0000 7f4fa88ec640 1 -- 172.21.15.38:0/2805463454 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4f84078b90 msgr2=0x7f4f8407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:45.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:45.009+0000 7f4fa88ec640 1 --2- 172.21.15.38:0/2805463454 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4f84078b90 0x7f4f8407b0a0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f4f9400b450 tx=0x7f4f9400d040 comp rx=0 tx=0).stop 2024-09-21T03:34:45.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:45.009+0000 7f4fa88ec640 1 -- 172.21.15.38:0/2805463454 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4fa40cd450 msgr2=0x7f4fa414c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:45.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:45.009+0000 7f4fa88ec640 1 --2- 172.21.15.38:0/2805463454 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4fa40cd450 0x7f4fa414c800 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f4f8c00d720 tx=0x7f4f8c00dbe0 comp rx=0 tx=0).stop 2024-09-21T03:34:45.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:45.009+0000 7f4fa88ec640 1 -- 172.21.15.38:0/2805463454 shutdown_connections 2024-09-21T03:34:45.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:45.009+0000 7f4fa88ec640 1 --2- 172.21.15.38:0/2805463454 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4f84078b90 0x7f4f8407b0a0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:45.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:45.009+0000 7f4fa88ec640 1 --2- 172.21.15.38:0/2805463454 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4fa415ead0 0x7f4fa414ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:45.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:45.009+0000 7f4fa88ec640 1 --2- 172.21.15.38:0/2805463454 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4fa40cd450 0x7f4fa414c800 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:45.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:45.009+0000 7f4fa88ec640 1 --2- 172.21.15.38:0/2805463454 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4fa40b5bd0 0x7f4fa414da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:45.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:45.009+0000 7f4fa88ec640 1 -- 172.21.15.38:0/2805463454 >> 172.21.15.38:0/2805463454 conn(0x7f4fa40b46e0 msgr2=0x7f4fa4162250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:45.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:45.009+0000 7f4fa88ec640 1 -- 172.21.15.38:0/2805463454 shutdown_connections 2024-09-21T03:34:45.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:45.009+0000 7f4fa88ec640 1 -- 172.21.15.38:0/2805463454 wait complete. 2024-09-21T03:34:45.022 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T03:34:45.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:45.137+0000 7f78c76a9640 1 -- 172.21.15.38:0/991785503 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f78c00cd040 msgr2=0x7f78c00d04c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:45.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:45.137+0000 7f78c76a9640 1 --2- 172.21.15.38:0/991785503 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f78c00cd040 0x7f78c00d04c0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f78c00b4ab0 tx=0x7f78b802a8b0 comp rx=0 tx=0).stop 2024-09-21T03:34:45.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:45.137+0000 7f78c76a9640 1 -- 172.21.15.38:0/991785503 shutdown_connections 2024-09-21T03:34:45.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:45.137+0000 7f78c76a9640 1 --2- 172.21.15.38:0/991785503 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f78c00cd040 0x7f78c00d04c0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:45.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:45.137+0000 7f78c76a9640 1 --2- 172.21.15.38:0/991785503 >> v2:172.21.15.195:3300/0 conn(0x7f78c01536e0 0x7f78c00d17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:45.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:45.137+0000 7f78c76a9640 1 -- 172.21.15.38:0/991785503 >> 172.21.15.38:0/991785503 conn(0x7f78c00cef40 msgr2=0x7f78c00cf350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:45.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:45.138+0000 7f78c76a9640 1 -- 172.21.15.38:0/991785503 shutdown_connections 2024-09-21T03:34:45.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:45.138+0000 7f78c76a9640 1 -- 172.21.15.38:0/991785503 wait complete. 2024-09-21T03:34:45.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:45.139+0000 7f78c76a9640 1 Processor -- start 2024-09-21T03:34:45.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:45.139+0000 7f78c76a9640 1 -- start start 2024-09-21T03:34:45.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:45.141+0000 7f78c76a9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f78c00d1d00 0x7f78c01f6e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:45.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:45.141+0000 7f78c76a9640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f78c01536e0 0x7f78c00cc1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:45.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:45.141+0000 7f78c76a9640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f78c014d790 0x7f78c014dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:45.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:45.141+0000 7f78c76a9640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f78c00d0d90 con 0x7f78c00d1d00 2024-09-21T03:34:45.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:45.141+0000 7f78c76a9640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f78c00d0c10 con 0x7f78c014d790 2024-09-21T03:34:45.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:45.141+0000 7f78c76a9640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f78c00d0f10 con 0x7f78c01536e0 2024-09-21T03:34:45.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:45.141+0000 7f78c5445640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f78c00d1d00 0x7f78c01f6e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:45.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:45.141+0000 7f78c5445640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f78c00d1d00 0x7f78c01f6e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50176/0 (socket says 172.21.15.38:50176) 2024-09-21T03:34:45.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:45.141+0000 7f78c5445640 1 -- 172.21.15.38:0/67575842 learned_addr learned my addr 172.21.15.38:0/67575842 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:34:45.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:45.141+0000 7f78c5c46640 1 --2- 172.21.15.38:0/67575842 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f78c014d790 0x7f78c014dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:45.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:45.141+0000 7f78c4c44640 1 --2- 172.21.15.38:0/67575842 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f78c01536e0 0x7f78c00cc1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:45.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:45.141+0000 7f78c5445640 1 -- 172.21.15.38:0/67575842 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f78c01536e0 msgr2=0x7f78c00cc1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:45.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:45.141+0000 7f78c5445640 1 --2- 172.21.15.38:0/67575842 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f78c01536e0 0x7f78c00cc1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:45.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:45.141+0000 7f78c5445640 1 -- 172.21.15.38:0/67575842 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f78c014d790 msgr2=0x7f78c014dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:45.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:45.141+0000 7f78c5445640 1 --2- 172.21.15.38:0/67575842 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f78c014d790 0x7f78c014dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:45.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:45.141+0000 7f78c5445640 1 -- 172.21.15.38:0/67575842 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78c014e0e0 con 0x7f78c00d1d00 2024-09-21T03:34:45.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:45.141+0000 7f78c4c44640 1 --2- 172.21.15.38:0/67575842 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f78c01536e0 0x7f78c00cc1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:34:45.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:45.141+0000 7f78c5445640 1 --2- 172.21.15.38:0/67575842 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f78c00d1d00 0x7f78c01f6e20 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f78bc00c9a0 tx=0x7f78bc00ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:45.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:45.141+0000 7f78b5ffb640 1 -- 172.21.15.38:0/67575842 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78bc007bb0 con 0x7f78c00d1d00 2024-09-21T03:34:45.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:45.141+0000 7f78b5ffb640 1 -- 172.21.15.38:0/67575842 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f78bc007e40 con 0x7f78c00d1d00 2024-09-21T03:34:45.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:45.141+0000 7f78c76a9640 1 -- 172.21.15.38:0/67575842 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78c01f7360 con 0x7f78c00d1d00 2024-09-21T03:34:45.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:45.141+0000 7f78b5ffb640 1 -- 172.21.15.38:0/67575842 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78bc00f690 con 0x7f78c00d1d00 2024-09-21T03:34:45.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:45.142+0000 7f78c76a9640 1 -- 172.21.15.38:0/67575842 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78c01e4cd0 con 0x7f78c00d1d00 2024-09-21T03:34:45.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:45.143+0000 7f78c76a9640 1 -- 172.21.15.38:0/67575842 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78c00d04c0 con 0x7f78c00d1d00 2024-09-21T03:34:45.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:45.143+0000 7f78b5ffb640 1 -- 172.21.15.38:0/67575842 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f78bc00f830 con 0x7f78c00d1d00 2024-09-21T03:34:45.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:45.144+0000 7f78b5ffb640 1 --2- 172.21.15.38:0/67575842 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f78a4078b90 0x7f78a407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:45.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:45.144+0000 7f78b5ffb640 1 -- 172.21.15.38:0/67575842 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7f78bc09b760 con 0x7f78c00d1d00 2024-09-21T03:34:45.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:45.144+0000 7f78c4c44640 1 --2- 172.21.15.38:0/67575842 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f78a4078b90 0x7f78a407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:45.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:45.145+0000 7f78c4c44640 1 --2- 172.21.15.38:0/67575842 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f78a4078b90 0x7f78a407b0a0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f78c014d6c0 tx=0x7f78b0009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:45.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:45.149+0000 7f78b5ffb640 1 -- 172.21.15.38:0/67575842 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f78c014e0e0 con 0x7f78c00d1d00 2024-09-21T03:34:45.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:45.375+0000 7f78c76a9640 1 -- 172.21.15.38:0/67575842 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f78c00b52b0 con 0x7f78c00d1d00 2024-09-21T03:34:45.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:45.375+0000 7f78b5ffb640 1 -- 172.21.15.38:0/67575842 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+73 (secure 0 0 0) 0x7f78bc014030 con 0x7f78c00d1d00 2024-09-21T03:34:45.377 INFO:teuthology.orchestra.run.smithi038.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T03:34:45.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:45.377+0000 7f78c76a9640 1 -- 172.21.15.38:0/67575842 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f78a4078b90 msgr2=0x7f78a407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:45.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:45.377+0000 7f78c76a9640 1 --2- 172.21.15.38:0/67575842 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f78a4078b90 0x7f78a407b0a0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f78c014d6c0 tx=0x7f78b0009040 comp rx=0 tx=0).stop 2024-09-21T03:34:45.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:45.377+0000 7f78c76a9640 1 -- 172.21.15.38:0/67575842 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f78c00d1d00 msgr2=0x7f78c01f6e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:45.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:45.377+0000 7f78c76a9640 1 --2- 172.21.15.38:0/67575842 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f78c00d1d00 0x7f78c01f6e20 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f78bc00c9a0 tx=0x7f78bc00ce60 comp rx=0 tx=0).stop 2024-09-21T03:34:45.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:45.377+0000 7f78c76a9640 1 -- 172.21.15.38:0/67575842 shutdown_connections 2024-09-21T03:34:45.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:45.377+0000 7f78c76a9640 1 --2- 172.21.15.38:0/67575842 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f78c014d790 0x7f78c014dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:45.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:45.377+0000 7f78c76a9640 1 --2- 172.21.15.38:0/67575842 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f78a4078b90 0x7f78a407b0a0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:45.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:45.377+0000 7f78c76a9640 1 --2- 172.21.15.38:0/67575842 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f78c01536e0 0x7f78c00cc1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:45.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:45.377+0000 7f78c76a9640 1 --2- 172.21.15.38:0/67575842 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f78c00d1d00 0x7f78c01f6e20 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:45.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:45.377+0000 7f78c76a9640 1 -- 172.21.15.38:0/67575842 >> 172.21.15.38:0/67575842 conn(0x7f78c00cef40 msgr2=0x7f78c00d2ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:45.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:45.377+0000 7f78c76a9640 1 -- 172.21.15.38:0/67575842 shutdown_connections 2024-09-21T03:34:45.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:45.377+0000 7f78c76a9640 1 -- 172.21.15.38:0/67575842 wait complete. 2024-09-21T03:34:45.390 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-21T03:34:45.390 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T03:34:45.390 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi038.front.sepia.ceph.com 2024-09-21T03:34:45.391 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-21T03:34:45.391 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T03:34:45.391 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-21T03:34:45.391 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-21T03:34:45.391 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T03:34:45.391 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T03:34:45.391 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:34:45.391 DEBUG:teuthology.orchestra.run.smithi038:> ip addr 2024-09-21T03:34:45.407 INFO:teuthology.orchestra.run.smithi038.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T03:34:45.408 INFO:teuthology.orchestra.run.smithi038.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T03:34:45.408 INFO:teuthology.orchestra.run.smithi038.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T03:34:45.408 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:34:45.408 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 ::1/128 scope host 2024-09-21T03:34:45.408 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:34:45.408 INFO:teuthology.orchestra.run.smithi038.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T03:34:45.408 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:6b:fd:a2 brd ff:ff:ff:ff:ff:ff 2024-09-21T03:34:45.408 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp7s0f0 2024-09-21T03:34:45.408 INFO:teuthology.orchestra.run.smithi038.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T03:34:45.408 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:6b:fd:a3 brd ff:ff:ff:ff:ff:ff 2024-09-21T03:34:45.408 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp7s0f1 2024-09-21T03:34:45.409 INFO:teuthology.orchestra.run.smithi038.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T03:34:45.409 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:bd:15:fc brd ff:ff:ff:ff:ff:ff 2024-09-21T03:34:45.409 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp1s0f0 2024-09-21T03:34:45.409 INFO:teuthology.orchestra.run.smithi038.stdout: inet 172.21.15.38/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-21T03:34:45.409 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft 42724sec preferred_lft 42724sec 2024-09-21T03:34:45.409 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 fe80::ec4:7aff:febd:15fc/64 scope link 2024-09-21T03:34:45.409 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:34:45.409 INFO:teuthology.orchestra.run.smithi038.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T03:34:45.409 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:bd:15:fd brd ff:ff:ff:ff:ff:ff 2024-09-21T03:34:45.409 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp1s0f1 2024-09-21T03:34:45.409 INFO:teuthology.orchestra.run.smithi038.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T03:34:45.409 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 92:55:c7:97:6f:b5 brd ff:ff:ff:ff:ff:ff 2024-09-21T03:34:45.409 INFO:teuthology.orchestra.run.smithi038.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-21T03:34:45.410 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:34:45.410 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 fe80::c49f:c1ff:fe21:b12a/64 scope link 2024-09-21T03:34:45.410 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:34:45.410 INFO:teuthology.orchestra.run.smithi038.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T03:34:45.410 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether a6:3a:83:ad:db:d5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T03:34:45.410 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 fe80::a43a:83ff:fead:dbd5/64 scope link 2024-09-21T03:34:45.410 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:34:45.410 INFO:teuthology.orchestra.run.smithi038.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T03:34:45.410 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 92:55:c7:97:6f:b5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T03:34:45.410 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 fe80::9055:c7ff:fe97:6fb5/64 scope link 2024-09-21T03:34:45.410 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:34:45.411 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:34:45.411 DEBUG:teuthology.orchestra.run.smithi038:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T03:34:45.472 INFO:teuthology.orchestra.run.smithi038.stdout:1 2024-09-21T03:34:45.473 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:34:45.473 DEBUG:teuthology.orchestra.run.smithi038:> route 2024-09-21T03:34:45.531 INFO:teuthology.orchestra.run.smithi038.stdout:Kernel IP routing table 2024-09-21T03:34:45.531 INFO:teuthology.orchestra.run.smithi038.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T03:34:45.531 INFO:teuthology.orchestra.run.smithi038.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-21T03:34:45.531 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-21T03:34:45.531 INFO:teuthology.orchestra.run.smithi038.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T03:34:45.532 DEBUG:teuthology.orchestra.run.smithi038:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T03:34:45.532 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-09-21T03:34:45.532 DEBUG:teuthology.orchestra.run.smithi038:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-21T03:34:45.532 DEBUG:teuthology.orchestra.run.smithi038:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-21T03:34:45.532 DEBUG:teuthology.orchestra.run.smithi038:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-21T03:34:45.532 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-09-21T03:34:45.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:45 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T03:34:45.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:34:45.662 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:34:45.662 DEBUG:teuthology.orchestra.run.smithi038:> ip netns list 2024-09-21T03:34:45.677 INFO:teuthology.orchestra.run.smithi038.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-21T03:34:45.678 INFO:teuthology.orchestra.run.smithi038.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-21T03:34:45.678 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T03:34:45.678 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:34:45.678 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip link set brx.0 up 2024-09-21T03:34:45.744 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T03:34:45.744 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-21T03:34:45.799 INFO:teuthology.orchestra.run.smithi038.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-21T03:34:45.799 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T03:34:45.800 DEBUG:teuthology.orchestra.run.smithi038:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-21T03:34:45.854 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-21T03:34:45.855 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:34:45.855 DEBUG:teuthology.orchestra.run.smithi038:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T03:34:45.951 INFO:teuthology.orchestra.run.smithi038.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T03:34:45.951 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T03:34:45.951 INFO:teuthology.orchestra.run.smithi038.stdout:invalid new device string format 2024-09-21T03:34:45.962 INFO:teuthology.orchestra.run.smithi038.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T03:34:45.962 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T03:34:45.962 INFO:teuthology.orchestra.run.smithi038.stdout:invalid new device string format 2024-09-21T03:34:45.962 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: resolved to: "172.21.15.38:3300,172.21.15.195:3300,172.21.15.195:3301" 2024-09-21T03:34:45.962 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T03:34:45.968 INFO:teuthology.orchestra.run.smithi038.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T03:34:45.969 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T03:34:45.969 INFO:teuthology.orchestra.run.smithi038.stdout:invalid new device string format 2024-09-21T03:34:45.969 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: resolved to: "172.21.15.38:3300,172.21.15.195:3300,172.21.15.195:3301" 2024-09-21T03:34:45.969 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T03:34:45.969 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: resolved to: "172.21.15.38:6789,172.21.15.195:6789,172.21.15.195:6790" 2024-09-21T03:34:45.969 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.38:6789,172.21.15.195:6789,172.21.15.195:6790:/ 2024-09-21T03:34:45.970 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=9d50d66f-656b-4fe2-8e3b-fe53b79c4d43" will pass to kernel 2024-09-21T03:34:45.970 INFO:teuthology.orchestra.run.smithi038.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-21T03:34:45.970 INFO:teuthology.orchestra.run.smithi038.stdout: You just mounted a file system that supports labels which does not 2024-09-21T03:34:45.971 INFO:teuthology.orchestra.run.smithi038.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-21T03:34:45.971 INFO:teuthology.orchestra.run.smithi038.stdout: applications will generate AVC messages and not be allowed access to 2024-09-21T03:34:45.971 INFO:teuthology.orchestra.run.smithi038.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-21T03:34:45.972 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T03:34:45.972 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:34:45.973 DEBUG:teuthology.orchestra.run.smithi038:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-21T03:34:45.999 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T03:34:46.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:46 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T03:34:46.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:34:46.102 INFO:teuthology.orchestra.run.smithi038.stdout:client4898 2024-09-21T03:34:46.102 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-21T03:34:46.102 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:34:46.119 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:34:46.119 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir subdir/) 2024-09-21T03:34:46.179 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:34:46.179 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch subdir/file_undamaged) 2024-09-21T03:34:46.237 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:34:46.237 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch subdir/file_to_be_damaged) 2024-09-21T03:34:46.294 DEBUG:teuthology.orchestra.run.smithi038:> stdin-killer -- python3 -c ' 2024-09-21T03:34:46.294 DEBUG:teuthology.orchestra.run.smithi038:> import os 2024-09-21T03:34:46.294 DEBUG:teuthology.orchestra.run.smithi038:> import stat 2024-09-21T03:34:46.294 DEBUG:teuthology.orchestra.run.smithi038:> 2024-09-21T03:34:46.295 DEBUG:teuthology.orchestra.run.smithi038:> print(os.stat("/home/ubuntu/cephtest/mnt.0/subdir").st_ino) 2024-09-21T03:34:46.295 DEBUG:teuthology.orchestra.run.smithi038:> ' 2024-09-21T03:34:46.400 INFO:teuthology.orchestra.run.smithi038.stdout:1099511627776 2024-09-21T03:34:46.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:34:46.409 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-21T03:34:46.409 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:34:46.465 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-21T03:34:46.465 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:34:46.522 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T03:34:46.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:46.678+0000 7f80ceddf640 1 -- 172.21.15.38:0/1711379187 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f80b0003a50 msgr2=0x7f80b0003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:46.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:46.678+0000 7f80ceddf640 1 --2- 172.21.15.38:0/1711379187 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f80b0003a50 0x7f80b0003ec0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f80c4006fd0 tx=0x7f80b0005830 comp rx=0 tx=0).stop 2024-09-21T03:34:46.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:46.678+0000 7f80ceddf640 1 -- 172.21.15.38:0/1711379187 shutdown_connections 2024-09-21T03:34:46.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:46.678+0000 7f80ceddf640 1 --2- 172.21.15.38:0/1711379187 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f80b0003a50 0x7f80b0003ec0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:46.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:46.678+0000 7f80ceddf640 1 -- 172.21.15.38:0/1711379187 >> 172.21.15.38:0/1711379187 conn(0x7f80c80b1b90 msgr2=0x7f80c80b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:46.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:46.678+0000 7f80ceddf640 1 -- 172.21.15.38:0/1711379187 shutdown_connections 2024-09-21T03:34:46.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:46.678+0000 7f80ceddf640 1 -- 172.21.15.38:0/1711379187 wait complete. 2024-09-21T03:34:46.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:46.680+0000 7f80ceddf640 1 Processor -- start 2024-09-21T03:34:46.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:46.680+0000 7f80ceddf640 1 -- start start 2024-09-21T03:34:46.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:46.681+0000 7f80ceddf640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f80b0003a50 0x7f80c81f6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:46.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:46.681+0000 7f80ceddf640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f80c81587b0 0x7f80c81e4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:46.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:46.681+0000 7f80ceddf640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f80c81e54a0 0x7f80c81f4870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:46.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:46.681+0000 7f80ceddf640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f80c80cf410 con 0x7f80b0003a50 2024-09-21T03:34:46.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:46.681+0000 7f80ceddf640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f80c80cf290 con 0x7f80c81587b0 2024-09-21T03:34:46.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:46.681+0000 7f80ceddf640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f80c80cf590 con 0x7f80c81e54a0 2024-09-21T03:34:46.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:46.681+0000 7f80ccb7b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f80b0003a50 0x7f80c81f6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:46.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:46.681+0000 7f80ccb7b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f80b0003a50 0x7f80c81f6260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50190/0 (socket says 172.21.15.38:50190) 2024-09-21T03:34:46.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:46.681+0000 7f80ccb7b640 1 -- 172.21.15.38:0/1410744700 learned_addr learned my addr 172.21.15.38:0/1410744700 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:34:46.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:46.681+0000 7f80bffff640 1 --2- 172.21.15.38:0/1410744700 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f80c81587b0 0x7f80c81e4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:46.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:46.681+0000 7f80cd37c640 1 --2- 172.21.15.38:0/1410744700 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f80c81e54a0 0x7f80c81f4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:46.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:46.682+0000 7f80ccb7b640 1 -- 172.21.15.38:0/1410744700 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f80c81e54a0 msgr2=0x7f80c81f4870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:46.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:46.682+0000 7f80ccb7b640 1 --2- 172.21.15.38:0/1410744700 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f80c81e54a0 0x7f80c81f4870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:46.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:46.682+0000 7f80ccb7b640 1 -- 172.21.15.38:0/1410744700 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f80c81587b0 msgr2=0x7f80c81e4f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:46.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:46.682+0000 7f80ccb7b640 1 --2- 172.21.15.38:0/1410744700 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f80c81587b0 0x7f80c81e4f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:46.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:46.682+0000 7f80ccb7b640 1 -- 172.21.15.38:0/1410744700 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80c81e58b0 con 0x7f80b0003a50 2024-09-21T03:34:46.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:46.682+0000 7f80bffff640 1 --2- 172.21.15.38:0/1410744700 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f80c81587b0 0x7f80c81e4f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:34:46.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:46.682+0000 7f80ccb7b640 1 --2- 172.21.15.38:0/1410744700 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f80b0003a50 0x7f80c81f6260 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f80c4007010 tx=0x7f80c4001cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:46.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:46.682+0000 7f80bd7fa640 1 -- 172.21.15.38:0/1410744700 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80c4003a00 con 0x7f80b0003a50 2024-09-21T03:34:46.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:46.682+0000 7f80bd7fa640 1 -- 172.21.15.38:0/1410744700 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f80c4003cb0 con 0x7f80b0003a50 2024-09-21T03:34:46.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:46.682+0000 7f80ceddf640 1 -- 172.21.15.38:0/1410744700 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80c81552a0 con 0x7f80b0003a50 2024-09-21T03:34:46.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:46.682+0000 7f80bd7fa640 1 -- 172.21.15.38:0/1410744700 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80c402e6e0 con 0x7f80b0003a50 2024-09-21T03:34:46.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:46.682+0000 7f80ceddf640 1 -- 172.21.15.38:0/1410744700 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80c8154b10 con 0x7f80b0003a50 2024-09-21T03:34:46.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:46.683+0000 7f80ceddf640 1 -- 172.21.15.38:0/1410744700 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80c806ddf0 con 0x7f80b0003a50 2024-09-21T03:34:46.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:46.684+0000 7f80bd7fa640 1 -- 172.21.15.38:0/1410744700 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f80c4035020 con 0x7f80b0003a50 2024-09-21T03:34:46.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:46.684+0000 7f80bd7fa640 1 --2- 172.21.15.38:0/1410744700 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f80b4078bc0 0x7f80b407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:46.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:46.684+0000 7f80bd7fa640 1 -- 172.21.15.38:0/1410744700 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7f80c40bbbe0 con 0x7f80b0003a50 2024-09-21T03:34:46.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:46.684+0000 7f80bffff640 1 --2- 172.21.15.38:0/1410744700 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f80b4078bc0 0x7f80b407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:46.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:46.685+0000 7f80bffff640 1 --2- 172.21.15.38:0/1410744700 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f80b4078bc0 0x7f80b407b0d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f80c806dd60 tx=0x7f80b8006ba0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:46.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:46.689+0000 7f80bd7fa640 1 -- 172.21.15.38:0/1410744700 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f80c4047060 con 0x7f80b0003a50 2024-09-21T03:34:46.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:46.886+0000 7f80ceddf640 1 -- 172.21.15.38:0/1410744700 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f80c80b23e0 con 0x7f80b0003a50 2024-09-21T03:34:46.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:46.886+0000 7f80bd7fa640 1 -- 172.21.15.38:0/1410744700 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v40) ==== 80+0+0 (secure 0 0 0) 0x7f80c81e58b0 con 0x7f80b0003a50 2024-09-21T03:34:46.889 INFO:teuthology.orchestra.run.smithi038.stderr:listed 0 entries 2024-09-21T03:34:46.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:46.888+0000 7f80ceddf640 1 -- 172.21.15.38:0/1410744700 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f80b4078bc0 msgr2=0x7f80b407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:46.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:46.888+0000 7f80ceddf640 1 --2- 172.21.15.38:0/1410744700 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f80b4078bc0 0x7f80b407b0d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f80c806dd60 tx=0x7f80b8006ba0 comp rx=0 tx=0).stop 2024-09-21T03:34:46.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:46.888+0000 7f80ceddf640 1 -- 172.21.15.38:0/1410744700 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f80b0003a50 msgr2=0x7f80c81f6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:46.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:46.888+0000 7f80ceddf640 1 --2- 172.21.15.38:0/1410744700 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f80b0003a50 0x7f80c81f6260 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f80c4007010 tx=0x7f80c4001cf0 comp rx=0 tx=0).stop 2024-09-21T03:34:46.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:46.888+0000 7f80ceddf640 1 -- 172.21.15.38:0/1410744700 shutdown_connections 2024-09-21T03:34:46.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:46.888+0000 7f80ceddf640 1 --2- 172.21.15.38:0/1410744700 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f80b4078bc0 0x7f80b407b0d0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:46.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:46.888+0000 7f80ceddf640 1 --2- 172.21.15.38:0/1410744700 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f80c81e54a0 0x7f80c81f4870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:46.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:46.888+0000 7f80ceddf640 1 --2- 172.21.15.38:0/1410744700 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f80c81587b0 0x7f80c81e4f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:46.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:46.888+0000 7f80ceddf640 1 --2- 172.21.15.38:0/1410744700 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f80b0003a50 0x7f80c81f6260 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:46.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:46.888+0000 7f80ceddf640 1 -- 172.21.15.38:0/1410744700 >> 172.21.15.38:0/1410744700 conn(0x7f80c80b1b90 msgr2=0x7f80c80b1050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:46.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:46.888+0000 7f80ceddf640 1 -- 172.21.15.38:0/1410744700 shutdown_connections 2024-09-21T03:34:46.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:46.888+0000 7f80ceddf640 1 -- 172.21.15.38:0/1410744700 wait complete. 2024-09-21T03:34:46.902 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-21T03:34:46.902 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:34:46.902 DEBUG:teuthology.orchestra.run.smithi038:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-21T03:34:46.963 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi038.front.sepia.ceph.com 2024-09-21T03:34:46.963 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:34:46.964 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T03:34:46.994 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:34:47.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.148+0000 7fcc4fb89640 1 -- 172.21.15.38:0/1862173349 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcc480b2e40 msgr2=0x7fcc480b3250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:47.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.148+0000 7fcc4fb89640 1 --2- 172.21.15.38:0/1862173349 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcc480b2e40 0x7fcc480b3250 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fcc480b7460 tx=0x7fcc3802aa30 comp rx=0 tx=0).stop 2024-09-21T03:34:47.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.148+0000 7fcc4fb89640 1 -- 172.21.15.38:0/1862173349 shutdown_connections 2024-09-21T03:34:47.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.148+0000 7fcc4fb89640 1 --2- 172.21.15.38:0/1862173349 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcc480b2e40 0x7fcc480b3250 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:47.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.148+0000 7fcc4fb89640 1 --2- 172.21.15.38:0/1862173349 >> v2:172.21.15.195:3300/0 conn(0x7fcc48162280 0x7fcc48162690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:47.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.148+0000 7fcc4fb89640 1 -- 172.21.15.38:0/1862173349 >> 172.21.15.38:0/1862173349 conn(0x7fcc480b6b20 msgr2=0x7fcc480d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:47.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.148+0000 7fcc4fb89640 1 -- 172.21.15.38:0/1862173349 shutdown_connections 2024-09-21T03:34:47.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.148+0000 7fcc4fb89640 1 -- 172.21.15.38:0/1862173349 wait complete. 2024-09-21T03:34:47.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.150+0000 7fcc4fb89640 1 Processor -- start 2024-09-21T03:34:47.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.150+0000 7fcc4fb89640 1 -- start start 2024-09-21T03:34:47.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.151+0000 7fcc4fb89640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcc480b2e40 0x7fcc480cf3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:47.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.151+0000 7fcc4fb89640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcc48162280 0x7fcc48156290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:47.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.151+0000 7fcc4fb89640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcc4814a3a0 0x7fcc481567d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:47.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.151+0000 7fcc4fb89640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fcc4815e4b0 con 0x7fcc4814a3a0 2024-09-21T03:34:47.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.151+0000 7fcc4fb89640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fcc480b39a0 con 0x7fcc48162280 2024-09-21T03:34:47.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.151+0000 7fcc4fb89640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fcc4815e630 con 0x7fcc480b2e40 2024-09-21T03:34:47.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.152+0000 7fcc4e126640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcc4814a3a0 0x7fcc481567d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:47.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.152+0000 7fcc4e126640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcc4814a3a0 0x7fcc481567d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50222/0 (socket says 172.21.15.38:50222) 2024-09-21T03:34:47.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.152+0000 7fcc4e126640 1 -- 172.21.15.38:0/3813232062 learned_addr learned my addr 172.21.15.38:0/3813232062 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:34:47.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.152+0000 7fcc4d124640 1 --2- 172.21.15.38:0/3813232062 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcc48162280 0x7fcc48156290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:47.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.152+0000 7fcc4d925640 1 --2- 172.21.15.38:0/3813232062 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcc480b2e40 0x7fcc480cf3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:47.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.152+0000 7fcc4e126640 1 -- 172.21.15.38:0/3813232062 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcc480b2e40 msgr2=0x7fcc480cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:47.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.152+0000 7fcc4e126640 1 --2- 172.21.15.38:0/3813232062 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcc480b2e40 0x7fcc480cf3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:47.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.152+0000 7fcc4e126640 1 -- 172.21.15.38:0/3813232062 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcc48162280 msgr2=0x7fcc48156290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:47.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.152+0000 7fcc4e126640 1 --2- 172.21.15.38:0/3813232062 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcc48162280 0x7fcc48156290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:47.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.152+0000 7fcc4e126640 1 -- 172.21.15.38:0/3813232062 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc480cf900 con 0x7fcc4814a3a0 2024-09-21T03:34:47.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.152+0000 7fcc4e126640 1 --2- 172.21.15.38:0/3813232062 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcc4814a3a0 0x7fcc481567d0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fcc3802b020 tx=0x7fcc38003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:47.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.152+0000 7fcc467fc640 1 -- 172.21.15.38:0/3813232062 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc38033050 con 0x7fcc4814a3a0 2024-09-21T03:34:47.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.152+0000 7fcc467fc640 1 -- 172.21.15.38:0/3813232062 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcc38004540 con 0x7fcc4814a3a0 2024-09-21T03:34:47.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.152+0000 7fcc4fb89640 1 -- 172.21.15.38:0/3813232062 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc48154d50 con 0x7fcc4814a3a0 2024-09-21T03:34:47.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.152+0000 7fcc467fc640 1 -- 172.21.15.38:0/3813232062 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc38038450 con 0x7fcc4814a3a0 2024-09-21T03:34:47.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.152+0000 7fcc4fb89640 1 -- 172.21.15.38:0/3813232062 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc48155310 con 0x7fcc4814a3a0 2024-09-21T03:34:47.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.154+0000 7fcc4fb89640 1 -- 172.21.15.38:0/3813232062 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc48162f00 con 0x7fcc4814a3a0 2024-09-21T03:34:47.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.154+0000 7fcc467fc640 1 -- 172.21.15.38:0/3813232062 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fcc38036070 con 0x7fcc4814a3a0 2024-09-21T03:34:47.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.155+0000 7fcc467fc640 1 --2- 172.21.15.38:0/3813232062 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fcc34078b90 0x7fcc3407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:47.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.155+0000 7fcc467fc640 1 -- 172.21.15.38:0/3813232062 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7fcc380bc1d0 con 0x7fcc4814a3a0 2024-09-21T03:34:47.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.155+0000 7fcc4d925640 1 --2- 172.21.15.38:0/3813232062 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fcc34078b90 0x7fcc3407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:47.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.155+0000 7fcc4d925640 1 --2- 172.21.15.38:0/3813232062 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fcc34078b90 0x7fcc3407b0a0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fcc400015e0 tx=0x7fcc40009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:47.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.159+0000 7fcc467fc640 1 -- 172.21.15.38:0/3813232062 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fcc38084470 con 0x7fcc4814a3a0 2024-09-21T03:34:47.390 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.388+0000 7fcc4fb89640 1 -- 172.21.15.38:0/3813232062 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcc481559d0 con 0x7fcc4814a3a0 2024-09-21T03:34:47.391 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.388+0000 7fcc467fc640 1 -- 172.21.15.38:0/3813232062 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5229 (secure 0 0 0) 0x7fcc38083d70 con 0x7fcc4814a3a0 2024-09-21T03:34:47.391 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:34:47.391 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":17,"btime":"2024-09-21T03:34:37:631111+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4757,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/1905955399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":1905955399},{"type":"v1","addr":"172.21.15.195:6836","nonce":1905955399}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4936,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/2989374027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":2989374027},{"type":"v1","addr":"172.21.15.195:6837","nonce":2989374027}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4942,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/3531731501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6833","nonce":3531731501},{"type":"v1","addr":"172.21.15.38:6835","nonce":3531731501}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:34:34.009269+0000","modified":"2024-09-21T03:34:37.009380+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5013},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5013":{"gid":5013,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.38:6834/1865253656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":1865253656},{"type":"v1","addr":"172.21.15.38:6834","nonce":1865253656}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5013,"qdb_cluster":[5013]},"id":2}]} 2024-09-21T03:34:47.391 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 17 2024-09-21T03:34:47.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.390+0000 7fcc4fb89640 1 -- 172.21.15.38:0/3813232062 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fcc34078b90 msgr2=0x7fcc3407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:47.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.390+0000 7fcc4fb89640 1 --2- 172.21.15.38:0/3813232062 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fcc34078b90 0x7fcc3407b0a0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fcc400015e0 tx=0x7fcc40009040 comp rx=0 tx=0).stop 2024-09-21T03:34:47.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.390+0000 7fcc4fb89640 1 -- 172.21.15.38:0/3813232062 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcc4814a3a0 msgr2=0x7fcc481567d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:47.394 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.390+0000 7fcc4fb89640 1 --2- 172.21.15.38:0/3813232062 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcc4814a3a0 0x7fcc481567d0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fcc3802b020 tx=0x7fcc38003f10 comp rx=0 tx=0).stop 2024-09-21T03:34:47.394 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.390+0000 7fcc4fb89640 1 -- 172.21.15.38:0/3813232062 shutdown_connections 2024-09-21T03:34:47.394 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.390+0000 7fcc4fb89640 1 --2- 172.21.15.38:0/3813232062 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcc4814a3a0 0x7fcc481567d0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:47.394 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.390+0000 7fcc4fb89640 1 --2- 172.21.15.38:0/3813232062 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcc48162280 0x7fcc48156290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:47.394 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.390+0000 7fcc4fb89640 1 --2- 172.21.15.38:0/3813232062 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fcc34078b90 0x7fcc3407b0a0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:47.394 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.390+0000 7fcc4fb89640 1 --2- 172.21.15.38:0/3813232062 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcc480b2e40 0x7fcc480cf3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:47.394 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.390+0000 7fcc4fb89640 1 -- 172.21.15.38:0/3813232062 >> 172.21.15.38:0/3813232062 conn(0x7fcc480b6b20 msgr2=0x7fcc4806e1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:47.394 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.390+0000 7fcc4fb89640 1 -- 172.21.15.38:0/3813232062 shutdown_connections 2024-09-21T03:34:47.394 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.390+0000 7fcc4fb89640 1 -- 172.21.15.38:0/3813232062 wait complete. 2024-09-21T03:34:47.405 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2024-09-21T03:34:47.539 INFO:teuthology.orchestra.run.smithi038.stdout:{"message":"","return_code":0} 2024-09-21T03:34:47.540 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-09-21T03:34:47.540 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-21T03:34:47.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.655+0000 7fad9b0de640 1 -- 172.21.15.38:0/3344745212 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fad940d0bb0 msgr2=0x7fad940b5be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:47.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.655+0000 7fad9b0de640 1 --2- 172.21.15.38:0/3344745212 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fad940d0bb0 0x7fad940b5be0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fad8c005fd0 tx=0x7fad8c02a820 comp rx=0 tx=0).stop 2024-09-21T03:34:47.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.655+0000 7fad9b0de640 1 -- 172.21.15.38:0/3344745212 shutdown_connections 2024-09-21T03:34:47.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.655+0000 7fad9b0de640 1 --2- 172.21.15.38:0/3344745212 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fad940d0bb0 0x7fad940b5be0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:47.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.655+0000 7fad9b0de640 1 --2- 172.21.15.38:0/3344745212 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fad9415ca90 0x7fad940d0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:47.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.655+0000 7fad9b0de640 1 -- 172.21.15.38:0/3344745212 >> 172.21.15.38:0/3344745212 conn(0x7fad940b46e0 msgr2=0x7fad940d23e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:47.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.655+0000 7fad9b0de640 1 -- 172.21.15.38:0/3344745212 shutdown_connections 2024-09-21T03:34:47.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.655+0000 7fad9b0de640 1 -- 172.21.15.38:0/3344745212 wait complete. 2024-09-21T03:34:47.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.657+0000 7fad9b0de640 1 Processor -- start 2024-09-21T03:34:47.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.657+0000 7fad9b0de640 1 -- start start 2024-09-21T03:34:47.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.658+0000 7fad9b0de640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fad940ccc40 0x7fad941506a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:47.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.658+0000 7fad9b0de640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fad940d0bb0 0x7fad94152fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:47.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.658+0000 7fad9b0de640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fad9415ca90 0x7fad94153510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:47.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.658+0000 7fad9b0de640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fad940b65c0 con 0x7fad940d0bb0 2024-09-21T03:34:47.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.658+0000 7fad9b0de640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fad940b6440 con 0x7fad940ccc40 2024-09-21T03:34:47.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.658+0000 7fad9b0de640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fad940b6740 con 0x7fad9415ca90 2024-09-21T03:34:47.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.658+0000 7fad93fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fad940d0bb0 0x7fad94152fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:47.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.658+0000 7fad93fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fad940d0bb0 0x7fad94152fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50228/0 (socket says 172.21.15.38:50228) 2024-09-21T03:34:47.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.658+0000 7fad93fff640 1 -- 172.21.15.38:0/658716453 learned_addr learned my addr 172.21.15.38:0/658716453 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:34:47.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.658+0000 7fad9967b640 1 --2- 172.21.15.38:0/658716453 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fad9415ca90 0x7fad94153510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:47.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.658+0000 7fad98e7a640 1 --2- 172.21.15.38:0/658716453 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fad940ccc40 0x7fad941506a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:47.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.659+0000 7fad93fff640 1 -- 172.21.15.38:0/658716453 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fad9415ca90 msgr2=0x7fad94153510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:47.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.659+0000 7fad93fff640 1 --2- 172.21.15.38:0/658716453 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fad9415ca90 0x7fad94153510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:47.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.659+0000 7fad93fff640 1 -- 172.21.15.38:0/658716453 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fad940ccc40 msgr2=0x7fad941506a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:47.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.659+0000 7fad93fff640 1 --2- 172.21.15.38:0/658716453 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fad940ccc40 0x7fad941506a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:47.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.659+0000 7fad93fff640 1 -- 172.21.15.38:0/658716453 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad94150fe0 con 0x7fad940d0bb0 2024-09-21T03:34:47.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.659+0000 7fad93fff640 1 --2- 172.21.15.38:0/658716453 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fad940d0bb0 0x7fad94152fd0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fad8400e9f0 tx=0x7fad8400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:47.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.659+0000 7fad917fa640 1 -- 172.21.15.38:0/658716453 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad8400cd80 con 0x7fad940d0bb0 2024-09-21T03:34:47.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.659+0000 7fad917fa640 1 -- 172.21.15.38:0/658716453 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fad84010430 con 0x7fad940d0bb0 2024-09-21T03:34:47.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.659+0000 7fad9b0de640 1 -- 172.21.15.38:0/658716453 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad940af2a0 con 0x7fad940d0bb0 2024-09-21T03:34:47.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.659+0000 7fad917fa640 1 -- 172.21.15.38:0/658716453 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad84010860 con 0x7fad940d0bb0 2024-09-21T03:34:47.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.659+0000 7fad9b0de640 1 -- 172.21.15.38:0/658716453 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad94151560 con 0x7fad940d0bb0 2024-09-21T03:34:47.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.660+0000 7fad9b0de640 1 -- 172.21.15.38:0/658716453 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad940b6cd0 con 0x7fad940d0bb0 2024-09-21T03:34:47.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.661+0000 7fad917fa640 1 -- 172.21.15.38:0/658716453 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fad8401f020 con 0x7fad940d0bb0 2024-09-21T03:34:47.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.662+0000 7fad917fa640 1 --2- 172.21.15.38:0/658716453 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fad74078b90 0x7fad7407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:47.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.662+0000 7fad917fa640 1 -- 172.21.15.38:0/658716453 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7fad8409b560 con 0x7fad940d0bb0 2024-09-21T03:34:47.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.662+0000 7fad98e7a640 1 --2- 172.21.15.38:0/658716453 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fad74078b90 0x7fad7407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:47.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.663+0000 7fad98e7a640 1 --2- 172.21.15.38:0/658716453 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fad74078b90 0x7fad7407b0a0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fad7c001720 tx=0x7fad7c009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:47.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.667+0000 7fad917fa640 1 -- 172.21.15.38:0/658716453 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fad840631d0 con 0x7fad940d0bb0 2024-09-21T03:34:47.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.893+0000 7fad9b0de640 1 -- 172.21.15.38:0/658716453 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fad940d27b0 con 0x7fad940d0bb0 2024-09-21T03:34:47.896 INFO:tasks.ceph.mon.a.smithi038.stderr:2024-09-21T03:34:47.893+0000 7f3f45713640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T03:34:47.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.895+0000 7fad917fa640 1 -- 172.21.15.38:0/658716453 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v18) ==== 161+0+0 (secure 0 0 0) 0x7fad84067c00 con 0x7fad940d0bb0 2024-09-21T03:34:47.898 INFO:teuthology.orchestra.run.smithi038.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-21T03:34:47.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.898+0000 7fad9b0de640 1 -- 172.21.15.38:0/658716453 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fad74078b90 msgr2=0x7fad7407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:47.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.898+0000 7fad9b0de640 1 --2- 172.21.15.38:0/658716453 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fad74078b90 0x7fad7407b0a0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fad7c001720 tx=0x7fad7c009040 comp rx=0 tx=0).stop 2024-09-21T03:34:47.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.898+0000 7fad9b0de640 1 -- 172.21.15.38:0/658716453 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fad940d0bb0 msgr2=0x7fad94152fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:47.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.898+0000 7fad9b0de640 1 --2- 172.21.15.38:0/658716453 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fad940d0bb0 0x7fad94152fd0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fad8400e9f0 tx=0x7fad8400edf0 comp rx=0 tx=0).stop 2024-09-21T03:34:47.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.898+0000 7fad9b0de640 1 -- 172.21.15.38:0/658716453 shutdown_connections 2024-09-21T03:34:47.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.898+0000 7fad9b0de640 1 --2- 172.21.15.38:0/658716453 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fad74078b90 0x7fad7407b0a0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:47.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.898+0000 7fad9b0de640 1 --2- 172.21.15.38:0/658716453 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fad9415ca90 0x7fad94153510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:47.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.898+0000 7fad9b0de640 1 --2- 172.21.15.38:0/658716453 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fad940d0bb0 0x7fad94152fd0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:47.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.898+0000 7fad9b0de640 1 --2- 172.21.15.38:0/658716453 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fad940ccc40 0x7fad941506a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:47.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.898+0000 7fad9b0de640 1 -- 172.21.15.38:0/658716453 >> 172.21.15.38:0/658716453 conn(0x7fad940b46e0 msgr2=0x7fad940c8440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:47.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.898+0000 7fad9b0de640 1 -- 172.21.15.38:0/658716453 shutdown_connections 2024-09-21T03:34:47.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.898+0000 7fad9b0de640 1 -- 172.21.15.38:0/658716453 wait complete. 2024-09-21T03:34:47.904 INFO:tasks.ceph.mds.c.smithi038.stderr: -5250> 2024-09-21T03:34:39.510+0000 7ff0d8d9a640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T03:34:47.913 DEBUG:teuthology.orchestra.run.smithi038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata setomapval 10000000000.00000000 file_to_be_damaged_head deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef 2024-09-21T03:34:47.962 INFO:tasks.ceph.mds.c.smithi038.stdout:starting mds.c at 2024-09-21T03:34:47.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.969+0000 7fc245020f40 1 Processor -- start 2024-09-21T03:34:47.972 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.969+0000 7fc245020f40 1 -- start start 2024-09-21T03:34:47.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.970+0000 7fc245020f40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a3d3bb8d00 0x55a3d3be1610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:47.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.970+0000 7fc245020f40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a3d3be5e40 0x55a3d3c70080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:47.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.970+0000 7fc245020f40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a3d3c73950 0x55a3d3bb5080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:47.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.970+0000 7fc245020f40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55a3d3b65430 con 0x55a3d3c73950 2024-09-21T03:34:47.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.970+0000 7fc245020f40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55a3d3b721f0 con 0x55a3d3be5e40 2024-09-21T03:34:47.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.970+0000 7fc245020f40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55a3d3b06d60 con 0x55a3d3bb8d00 2024-09-21T03:34:47.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.970+0000 7fc23e668640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a3d3c73950 0x55a3d3bb5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:47.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.970+0000 7fc23e668640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a3d3c73950 0x55a3d3bb5080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50252/0 (socket says 172.21.15.38:50252) 2024-09-21T03:34:47.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.970+0000 7fc23e668640 1 -- 172.21.15.38:0/2855894587 learned_addr learned my addr 172.21.15.38:0/2855894587 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:34:47.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.970+0000 7fc23de67640 1 --2- 172.21.15.38:0/2855894587 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a3d3bb8d00 0x55a3d3be1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:47.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.970+0000 7fc23d666640 1 --2- 172.21.15.38:0/2855894587 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a3d3be5e40 0x55a3d3c70080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:47.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.971+0000 7fc23e668640 1 -- 172.21.15.38:0/2855894587 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a3d3bb8d00 msgr2=0x55a3d3be1610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:47.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.971+0000 7fc23e668640 1 --2- 172.21.15.38:0/2855894587 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a3d3bb8d00 0x55a3d3be1610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:47.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.971+0000 7fc23e668640 1 -- 172.21.15.38:0/2855894587 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a3d3be5e40 msgr2=0x55a3d3c70080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:47.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.971+0000 7fc23e668640 1 --2- 172.21.15.38:0/2855894587 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a3d3be5e40 0x55a3d3c70080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:47.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.971+0000 7fc23e668640 1 -- 172.21.15.38:0/2855894587 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a3d3be1b50 con 0x55a3d3c73950 2024-09-21T03:34:47.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.971+0000 7fc23d666640 1 --2- 172.21.15.38:0/2855894587 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a3d3be5e40 0x55a3d3c70080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:34:47.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.971+0000 7fc23e668640 1 --2- 172.21.15.38:0/2855894587 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a3d3c73950 0x55a3d3bb5080 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fc23800a550 tx=0x7fc238003330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5063809027d601 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:47.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.971+0000 7fc22ffff640 1 -- 172.21.15.38:0/2855894587 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc238064070 con 0x55a3d3c73950 2024-09-21T03:34:47.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.971+0000 7fc22ffff640 1 -- 172.21.15.38:0/2855894587 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc238004020 con 0x55a3d3c73950 2024-09-21T03:34:47.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.971+0000 7fc22ffff640 1 -- 172.21.15.38:0/2855894587 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc238060430 con 0x55a3d3c73950 2024-09-21T03:34:47.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.972+0000 7fc245020f40 1 -- 172.21.15.38:0/2855894587 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a3d3c73950 msgr2=0x55a3d3bb5080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:47.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.972+0000 7fc245020f40 1 --2- 172.21.15.38:0/2855894587 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a3d3c73950 0x55a3d3bb5080 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fc23800a550 tx=0x7fc238003330 comp rx=0 tx=0).stop 2024-09-21T03:34:47.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.972+0000 7fc245020f40 1 -- 172.21.15.38:0/2855894587 shutdown_connections 2024-09-21T03:34:47.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.972+0000 7fc245020f40 1 --2- 172.21.15.38:0/2855894587 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a3d3c73950 0x55a3d3bb5080 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:47.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.972+0000 7fc245020f40 1 --2- 172.21.15.38:0/2855894587 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a3d3be5e40 0x55a3d3c70080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:47.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.972+0000 7fc245020f40 1 --2- 172.21.15.38:0/2855894587 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a3d3bb8d00 0x55a3d3be1610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:47.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.972+0000 7fc245020f40 1 -- 172.21.15.38:0/2855894587 >> 172.21.15.38:0/2855894587 conn(0x55a3d3ad7ac0 msgr2=0x55a3d3c8a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:47.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.972+0000 7fc245020f40 1 -- 172.21.15.38:0/2855894587 shutdown_connections 2024-09-21T03:34:47.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.972+0000 7fc245020f40 1 -- 172.21.15.38:0/2855894587 wait complete. 2024-09-21T03:34:47.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.973+0000 7fc245020f40 1 Processor -- start 2024-09-21T03:34:47.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.973+0000 7fc245020f40 1 -- start start 2024-09-21T03:34:47.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.974+0000 7fc245020f40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a3d3bb8d00 0x55a3d3c70270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:47.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.974+0000 7fc245020f40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a3d3be5e40 0x55a3d3c866a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:47.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.974+0000 7fc245020f40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a3d3c73950 0x55a3d3c86be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:47.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.974+0000 7fc245020f40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55a3d3c714d0 con 0x55a3d3be5e40 2024-09-21T03:34:47.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.974+0000 7fc245020f40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55a3d3c71770 con 0x55a3d3c73950 2024-09-21T03:34:47.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.974+0000 7fc245020f40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55a3d3c71a10 con 0x55a3d3bb8d00 2024-09-21T03:34:47.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.974+0000 7fc23d666640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a3d3be5e40 0x55a3d3c866a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:47.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.974+0000 7fc23d666640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a3d3be5e40 0x55a3d3c866a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50258/0 (socket says 172.21.15.38:50258) 2024-09-21T03:34:47.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.974+0000 7fc23d666640 1 -- 172.21.15.38:0/2740079821 learned_addr learned my addr 172.21.15.38:0/2740079821 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:34:47.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.975+0000 7fc23d666640 1 -- 172.21.15.38:0/2740079821 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a3d3bb8d00 msgr2=0x55a3d3c70270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:47.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.975+0000 7fc23d666640 1 --2- 172.21.15.38:0/2740079821 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a3d3bb8d00 0x55a3d3c70270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:47.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.975+0000 7fc23d666640 1 -- 172.21.15.38:0/2740079821 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a3d3c73950 msgr2=0x55a3d3c86be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:47.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.975+0000 7fc23e668640 1 --2- 172.21.15.38:0/2740079821 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a3d3c73950 0x55a3d3c86be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:47.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.975+0000 7fc23d666640 1 --2- 172.21.15.38:0/2740079821 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a3d3c73950 0x55a3d3c86be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:47.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.975+0000 7fc23d666640 1 -- 172.21.15.38:0/2740079821 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a3d3c85220 con 0x55a3d3be5e40 2024-09-21T03:34:47.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.975+0000 7fc23e668640 1 --2- 172.21.15.38:0/2740079821 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a3d3c73950 0x55a3d3c86be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:34:47.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.975+0000 7fc23d666640 1 --2- 172.21.15.38:0/2740079821 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a3d3be5e40 0x55a3d3c866a0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fc234002da0 tx=0x7fc23400c620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:47.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.975+0000 7fc22effd640 1 -- 172.21.15.38:0/2740079821 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc234015020 con 0x55a3d3be5e40 2024-09-21T03:34:47.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.975+0000 7fc22effd640 1 -- 172.21.15.38:0/2740079821 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc234011650 con 0x55a3d3be5e40 2024-09-21T03:34:47.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.975+0000 7fc245020f40 1 -- 172.21.15.38:0/2740079821 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55a3d3c74570 con 0x55a3d3be5e40 2024-09-21T03:34:47.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.975+0000 7fc22effd640 1 -- 172.21.15.38:0/2740079821 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc234011a40 con 0x55a3d3be5e40 2024-09-21T03:34:47.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.975+0000 7fc245020f40 1 -- 172.21.15.38:0/2740079821 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a3d3c8a840 con 0x55a3d3be5e40 2024-09-21T03:34:47.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.977+0000 7fc22effd640 1 -- 172.21.15.38:0/2740079821 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc234018020 con 0x55a3d3be5e40 2024-09-21T03:34:47.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.977+0000 7fc22effd640 1 --2- 172.21.15.38:0/2740079821 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc220077570 0x7fc220079a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:47.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.977+0000 7fc22effd640 1 -- 172.21.15.38:0/2740079821 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6187+0+0 (secure 0 0 0) 0x7fc234099c90 con 0x55a3d3be5e40 2024-09-21T03:34:47.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.977+0000 7fc23de67640 1 --2- 172.21.15.38:0/2740079821 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc220077570 0x7fc220079a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:47.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.978+0000 7fc245020f40 1 --2- 172.21.15.38:0/2740079821 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x55a3d3d29b60 0x55a3d3d32470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:47.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.978+0000 7fc245020f40 1 -- 172.21.15.38:0/2740079821 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:1 5.10 5:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=115b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41) -- 0x55a3d3c87120 con 0x55a3d3d29b60 2024-09-21T03:34:47.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.978+0000 7fc23e668640 1 --2- 172.21.15.38:0/2740079821 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x55a3d3d29b60 0x55a3d3d32470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:47.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.978+0000 7fc23de67640 1 --2- 172.21.15.38:0/2740079821 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc220077570 0x7fc220079a80 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fc230002fa0 tx=0x7fc230002aa0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:47.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.978+0000 7fc23e668640 1 --2- 172.21.15.38:0/2740079821 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x55a3d3d29b60 0x55a3d3d32470 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:47.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.983+0000 7fc23e668640 1 -- 172.21.15.38:0/2740079821 <== osd.2 v2:172.21.15.38:6824/2577858902 1 ==== osd_op_reply(1 10000000000.00000000 [omap-set-vals] v41'11 uv11 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fc23800b040 con 0x55a3d3d29b60 2024-09-21T03:34:47.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.983+0000 7fc245020f40 1 -- 172.21.15.38:0/2740079821 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x55a3d3d29b60 msgr2=0x55a3d3d32470 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:47.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.983+0000 7fc245020f40 1 --2- 172.21.15.38:0/2740079821 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x55a3d3d29b60 0x55a3d3d32470 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:47.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.983+0000 7fc245020f40 1 -- 172.21.15.38:0/2740079821 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc220077570 msgr2=0x7fc220079a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:47.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.983+0000 7fc245020f40 1 --2- 172.21.15.38:0/2740079821 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc220077570 0x7fc220079a80 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fc230002fa0 tx=0x7fc230002aa0 comp rx=0 tx=0).stop 2024-09-21T03:34:47.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.983+0000 7fc245020f40 1 -- 172.21.15.38:0/2740079821 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a3d3be5e40 msgr2=0x55a3d3c866a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:47.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.983+0000 7fc245020f40 1 --2- 172.21.15.38:0/2740079821 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a3d3be5e40 0x55a3d3c866a0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fc234002da0 tx=0x7fc23400c620 comp rx=0 tx=0).stop 2024-09-21T03:34:47.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.983+0000 7fc23e668640 1 -- 172.21.15.38:0/2740079821 reap_dead start 2024-09-21T03:34:47.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.983+0000 7fc245020f40 1 -- 172.21.15.38:0/2740079821 shutdown_connections 2024-09-21T03:34:47.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.983+0000 7fc245020f40 1 -- 172.21.15.38:0/2740079821 >> 172.21.15.38:0/2740079821 conn(0x55a3d3ad7ac0 msgr2=0x55a3d3bb4ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:47.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.983+0000 7fc245020f40 1 -- 172.21.15.38:0/2740079821 shutdown_connections 2024-09-21T03:34:47.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:47.983+0000 7fc245020f40 1 -- 172.21.15.38:0/2740079821 wait complete. 2024-09-21T03:34:47.989 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-21T03:34:48.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:48.105+0000 7fb601320640 1 -- 172.21.15.38:0/63089432 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb5e4004d40 msgr2=0x7fb5e4005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:48.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:48.105+0000 7fb601320640 1 --2- 172.21.15.38:0/63089432 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb5e4004d40 0x7fb5e4005190 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fb5ec009ed0 tx=0x7fb5e4005830 comp rx=0 tx=0).stop 2024-09-21T03:34:48.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:48.105+0000 7fb5fb7fe640 1 -- 172.21.15.38:0/63089432 reap_dead start 2024-09-21T03:34:48.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:48.105+0000 7fb601320640 1 -- 172.21.15.38:0/63089432 shutdown_connections 2024-09-21T03:34:48.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:48.105+0000 7fb601320640 1 -- 172.21.15.38:0/63089432 >> 172.21.15.38:0/63089432 conn(0x7fb5fc09d830 msgr2=0x7fb5fc0b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:48.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:48.105+0000 7fb601320640 1 -- 172.21.15.38:0/63089432 shutdown_connections 2024-09-21T03:34:48.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:48.105+0000 7fb601320640 1 -- 172.21.15.38:0/63089432 wait complete. 2024-09-21T03:34:48.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:48.107+0000 7fb601320640 1 Processor -- start 2024-09-21T03:34:48.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:48.107+0000 7fb601320640 1 -- start start 2024-09-21T03:34:48.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:48.109+0000 7fb601320640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb5e4004d40 0x7fb5fc0ca3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:48.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:48.109+0000 7fb601320640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb5fc09f190 0x7fb5fc0a1750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:48.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:48.109+0000 7fb601320640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb5fc0cb610 0x7fb5fc0cba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:48.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:48.109+0000 7fb601320640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fb5fc0cf330 con 0x7fb5fc0cb610 2024-09-21T03:34:48.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:48.109+0000 7fb601320640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fb5fc0cf1b0 con 0x7fb5e4004d40 2024-09-21T03:34:48.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:48.109+0000 7fb601320640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fb5fc0d1480 con 0x7fb5fc09f190 2024-09-21T03:34:48.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:48.109+0000 7fb5fb7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb5fc0cb610 0x7fb5fc0cba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:48.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:48.109+0000 7fb5fb7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb5fc0cb610 0x7fb5fc0cba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50268/0 (socket says 172.21.15.38:50268) 2024-09-21T03:34:48.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:48.109+0000 7fb5fb7fe640 1 -- 172.21.15.38:0/128483326 learned_addr learned my addr 172.21.15.38:0/128483326 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:34:48.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:48.109+0000 7fb5faffd640 1 --2- 172.21.15.38:0/128483326 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb5e4004d40 0x7fb5fc0ca3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:48.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:48.109+0000 7fb5fa7fc640 1 --2- 172.21.15.38:0/128483326 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb5fc09f190 0x7fb5fc0a1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:48.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:48.109+0000 7fb5fb7fe640 1 -- 172.21.15.38:0/128483326 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb5fc09f190 msgr2=0x7fb5fc0a1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:48.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:48.109+0000 7fb5fb7fe640 1 --2- 172.21.15.38:0/128483326 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb5fc09f190 0x7fb5fc0a1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:48.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:48.109+0000 7fb5fb7fe640 1 -- 172.21.15.38:0/128483326 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb5e4004d40 msgr2=0x7fb5fc0ca3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:48.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:48.109+0000 7fb5fb7fe640 1 --2- 172.21.15.38:0/128483326 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb5e4004d40 0x7fb5fc0ca3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:48.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:48.109+0000 7fb5fb7fe640 1 -- 172.21.15.38:0/128483326 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5fc0a06b0 con 0x7fb5fc0cb610 2024-09-21T03:34:48.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:48.109+0000 7fb5fb7fe640 1 --2- 172.21.15.38:0/128483326 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb5fc0cb610 0x7fb5fc0cba20 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fb5ec02aed0 tx=0x7fb5ec033bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:48.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:48.109+0000 7fb5e37fe640 1 -- 172.21.15.38:0/128483326 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5ec042070 con 0x7fb5fc0cb610 2024-09-21T03:34:48.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:48.110+0000 7fb5e37fe640 1 -- 172.21.15.38:0/128483326 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb5ec003970 con 0x7fb5fc0cb610 2024-09-21T03:34:48.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:48.110+0000 7fb5e37fe640 1 -- 172.21.15.38:0/128483326 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5ec038040 con 0x7fb5fc0cb610 2024-09-21T03:34:48.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:48.110+0000 7fb601320640 1 -- 172.21.15.38:0/128483326 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5fc0a0990 con 0x7fb5fc0cb610 2024-09-21T03:34:48.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:48.110+0000 7fb601320640 1 -- 172.21.15.38:0/128483326 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5fc0a0f50 con 0x7fb5fc0cb610 2024-09-21T03:34:48.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:48.111+0000 7fb601320640 1 -- 172.21.15.38:0/128483326 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5fc0d0be0 con 0x7fb5fc0cb610 2024-09-21T03:34:48.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:48.111+0000 7fb5e37fe640 1 -- 172.21.15.38:0/128483326 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb5ec004030 con 0x7fb5fc0cb610 2024-09-21T03:34:48.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:48.112+0000 7fb5e37fe640 1 --2- 172.21.15.38:0/128483326 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb5e8078bc0 0x7fb5e807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:48.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:48.112+0000 7fb5e37fe640 1 -- 172.21.15.38:0/128483326 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6187+0+0 (secure 0 0 0) 0x7fb5ec0bb300 con 0x7fb5fc0cb610 2024-09-21T03:34:48.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:48.112+0000 7fb5faffd640 1 --2- 172.21.15.38:0/128483326 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb5e8078bc0 0x7fb5e807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:48.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:48.113+0000 7fb5faffd640 1 --2- 172.21.15.38:0/128483326 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb5e8078bc0 0x7fb5e807b0d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fb5f4001a30 tx=0x7fb5f40012e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:48.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:48.117+0000 7fb5e37fe640 1 -- 172.21.15.38:0/128483326 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fb5ec082f10 con 0x7fb5fc0cb610 2024-09-21T03:34:48.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:48.343+0000 7fb601320640 1 -- 172.21.15.38:0/128483326 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7fb5fc1620e0 con 0x7fb5fc0cb610 2024-09-21T03:34:48.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:48.898+0000 7fb5e37fe640 1 -- 172.21.15.38:0/128483326 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v19) ==== 162+0+0 (secure 0 0 0) 0x7fb5ec087940 con 0x7fb5fc0cb610 2024-09-21T03:34:48.901 INFO:teuthology.orchestra.run.smithi038.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-21T03:34:48.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:48.900+0000 7fb601320640 1 -- 172.21.15.38:0/128483326 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb5e8078bc0 msgr2=0x7fb5e807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:48.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:48.900+0000 7fb601320640 1 --2- 172.21.15.38:0/128483326 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb5e8078bc0 0x7fb5e807b0d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fb5f4001a30 tx=0x7fb5f40012e0 comp rx=0 tx=0).stop 2024-09-21T03:34:48.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:48.900+0000 7fb601320640 1 -- 172.21.15.38:0/128483326 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb5fc0cb610 msgr2=0x7fb5fc0cba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:48.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:48.900+0000 7fb601320640 1 --2- 172.21.15.38:0/128483326 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb5fc0cb610 0x7fb5fc0cba20 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fb5ec02aed0 tx=0x7fb5ec033bf0 comp rx=0 tx=0).stop 2024-09-21T03:34:48.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:48.900+0000 7fb601320640 1 -- 172.21.15.38:0/128483326 shutdown_connections 2024-09-21T03:34:48.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:48.900+0000 7fb601320640 1 --2- 172.21.15.38:0/128483326 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb5fc0cb610 0x7fb5fc0cba20 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:48.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:48.900+0000 7fb601320640 1 --2- 172.21.15.38:0/128483326 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb5e8078bc0 0x7fb5e807b0d0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:48.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:48.900+0000 7fb601320640 1 --2- 172.21.15.38:0/128483326 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb5fc09f190 0x7fb5fc0a1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:48.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:48.900+0000 7fb601320640 1 --2- 172.21.15.38:0/128483326 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb5e4004d40 0x7fb5fc0ca3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:48.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:48.900+0000 7fb601320640 1 -- 172.21.15.38:0/128483326 >> 172.21.15.38:0/128483326 conn(0x7fb5fc09d830 msgr2=0x7fb5fc0d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:48.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:48.900+0000 7fb601320640 1 -- 172.21.15.38:0/128483326 shutdown_connections 2024-09-21T03:34:48.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:48.900+0000 7fb601320640 1 -- 172.21.15.38:0/128483326 wait complete. 2024-09-21T03:34:48.918 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:34:49.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:49.033+0000 7f15404b6640 1 -- 172.21.15.38:0/1930847599 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f15380b2e40 msgr2=0x7f15380b3250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:49.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:49.033+0000 7f15404b6640 1 --2- 172.21.15.38:0/1930847599 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f15380b2e40 0x7f15380b3250 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f15380d5250 tx=0x7f153002aa30 comp rx=0 tx=0).stop 2024-09-21T03:34:49.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:49.033+0000 7f15404b6640 1 -- 172.21.15.38:0/1930847599 shutdown_connections 2024-09-21T03:34:49.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:49.033+0000 7f15404b6640 1 --2- 172.21.15.38:0/1930847599 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f15380b2e40 0x7f15380b3250 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:49.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:49.033+0000 7f15404b6640 1 --2- 172.21.15.38:0/1930847599 >> v2:172.21.15.195:3300/0 conn(0x7f1538162280 0x7f1538162690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:49.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:49.033+0000 7f15404b6640 1 --2- 172.21.15.38:0/1930847599 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1538162f00 0x7f15380d24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:49.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:49.033+0000 7f15404b6640 1 -- 172.21.15.38:0/1930847599 >> 172.21.15.38:0/1930847599 conn(0x7f15380b6b20 msgr2=0x7f15380d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:49.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:49.033+0000 7f15404b6640 1 -- 172.21.15.38:0/1930847599 shutdown_connections 2024-09-21T03:34:49.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:49.033+0000 7f15404b6640 1 -- 172.21.15.38:0/1930847599 wait complete. 2024-09-21T03:34:49.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:49.035+0000 7f15404b6640 1 Processor -- start 2024-09-21T03:34:49.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:49.035+0000 7f15404b6640 1 -- start start 2024-09-21T03:34:49.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:49.036+0000 7f15404b6640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f15380b2e40 0x7f15380aadc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:49.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:49.036+0000 7f15404b6640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1538162280 0x7f153814e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:49.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:49.037+0000 7f15404b6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f15380ce880 0x7f15380cec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:49.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:49.037+0000 7f15404b6640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f15380b3dd0 con 0x7f15380ce880 2024-09-21T03:34:49.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:49.037+0000 7f15404b6640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f15380b39a0 con 0x7f1538162280 2024-09-21T03:34:49.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:49.037+0000 7f15404b6640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f15380b3f50 con 0x7f15380b2e40 2024-09-21T03:34:49.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:49.037+0000 7f153ea53640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f15380ce880 0x7f15380cec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:49.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:49.037+0000 7f153ea53640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f15380ce880 0x7f15380cec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50278/0 (socket says 172.21.15.38:50278) 2024-09-21T03:34:49.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:49.037+0000 7f153ea53640 1 -- 172.21.15.38:0/1756258273 learned_addr learned my addr 172.21.15.38:0/1756258273 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:34:49.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:49.037+0000 7f153da51640 1 --2- 172.21.15.38:0/1756258273 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1538162280 0x7f153814e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:49.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:49.037+0000 7f153e252640 1 --2- 172.21.15.38:0/1756258273 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f15380b2e40 0x7f15380aadc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:49.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:49.037+0000 7f153ea53640 1 -- 172.21.15.38:0/1756258273 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f15380b2e40 msgr2=0x7f15380aadc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:49.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:49.037+0000 7f153ea53640 1 --2- 172.21.15.38:0/1756258273 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f15380b2e40 0x7f15380aadc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:49.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:49.037+0000 7f153ea53640 1 -- 172.21.15.38:0/1756258273 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1538162280 msgr2=0x7f153814e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:49.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:49.037+0000 7f153ea53640 1 --2- 172.21.15.38:0/1756258273 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1538162280 0x7f153814e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:49.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:49.037+0000 7f153ea53640 1 -- 172.21.15.38:0/1756258273 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f153814edb0 con 0x7f15380ce880 2024-09-21T03:34:49.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:49.037+0000 7f153e252640 1 --2- 172.21.15.38:0/1756258273 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f15380b2e40 0x7f15380aadc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:34:49.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:49.037+0000 7f153ea53640 1 --2- 172.21.15.38:0/1756258273 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f15380ce880 0x7f15380cec90 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f153002b020 tx=0x7f1530003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:49.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:49.037+0000 7f152affd640 1 -- 172.21.15.38:0/1756258273 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1530033050 con 0x7f15380ce880 2024-09-21T03:34:49.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:49.037+0000 7f152affd640 1 -- 172.21.15.38:0/1756258273 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1530004550 con 0x7f15380ce880 2024-09-21T03:34:49.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:49.037+0000 7f15404b6640 1 -- 172.21.15.38:0/1756258273 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15380cd7c0 con 0x7f15380ce880 2024-09-21T03:34:49.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:49.037+0000 7f152affd640 1 -- 172.21.15.38:0/1756258273 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1530038450 con 0x7f15380ce880 2024-09-21T03:34:49.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:49.038+0000 7f15404b6640 1 -- 172.21.15.38:0/1756258273 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15380ce080 con 0x7f15380ce880 2024-09-21T03:34:49.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:49.038+0000 7f15404b6640 1 -- 172.21.15.38:0/1756258273 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f153806ac50 con 0x7f15380ce880 2024-09-21T03:34:49.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:49.039+0000 7f152affd640 1 -- 172.21.15.38:0/1756258273 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f1530036070 con 0x7f15380ce880 2024-09-21T03:34:49.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:49.040+0000 7f152affd640 1 --2- 172.21.15.38:0/1756258273 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f151c078b90 0x7f151c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:49.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:49.040+0000 7f152affd640 1 -- 172.21.15.38:0/1756258273 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6187+0+0 (secure 0 0 0) 0x7f15300bc4a0 con 0x7f15380ce880 2024-09-21T03:34:49.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:49.040+0000 7f153e252640 1 --2- 172.21.15.38:0/1756258273 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f151c078b90 0x7f151c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:49.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:49.041+0000 7f153e252640 1 --2- 172.21.15.38:0/1756258273 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f151c078b90 0x7f151c07b0a0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f152c005fd0 tx=0x7f152c005e00 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:49.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:49.044+0000 7f152affd640 1 -- 172.21.15.38:0/1756258273 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f153806ac50 con 0x7f15380ce880 2024-09-21T03:34:49.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:49.271+0000 7f15404b6640 1 -- 172.21.15.38:0/1756258273 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1538162f00 con 0x7f15380ce880 2024-09-21T03:34:49.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:49.272+0000 7f152affd640 1 -- 172.21.15.38:0/1756258273 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+5223 (secure 0 0 0) 0x7f1530084790 con 0x7f15380ce880 2024-09-21T03:34:49.275 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:34:49.275 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":20,"btime":"2024-09-21T03:34:48:902694+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4757,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/1905955399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":1905955399},{"type":"v1","addr":"172.21.15.195:6836","nonce":1905955399}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4936,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/2989374027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":2989374027},{"type":"v1","addr":"172.21.15.195:6837","nonce":2989374027}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4942,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/3531731501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6833","nonce":3531731501},{"type":"v1","addr":"172.21.15.38:6835","nonce":3531731501}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:34:34.009269+0000","modified":"2024-09-21T03:34:48.902672+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":41,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5226},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5226":{"gid":5226,"name":"c","rank":0,"incarnation":20,"state":"up:replay","state_seq":1,"addr":"172.21.15.38:6834/3533947700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":3533947700},{"type":"v1","addr":"172.21.15.38:6834","nonce":3533947700}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":2}]} 2024-09-21T03:34:49.276 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 20 2024-09-21T03:34:49.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:49.274+0000 7f15404b6640 1 -- 172.21.15.38:0/1756258273 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f151c078b90 msgr2=0x7f151c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:49.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:49.274+0000 7f15404b6640 1 --2- 172.21.15.38:0/1756258273 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f151c078b90 0x7f151c07b0a0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f152c005fd0 tx=0x7f152c005e00 comp rx=0 tx=0).stop 2024-09-21T03:34:49.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:49.274+0000 7f15404b6640 1 -- 172.21.15.38:0/1756258273 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f15380ce880 msgr2=0x7f15380cec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:49.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:49.274+0000 7f15404b6640 1 --2- 172.21.15.38:0/1756258273 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f15380ce880 0x7f15380cec90 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f153002b020 tx=0x7f1530003f10 comp rx=0 tx=0).stop 2024-09-21T03:34:49.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:49.274+0000 7f15404b6640 1 -- 172.21.15.38:0/1756258273 shutdown_connections 2024-09-21T03:34:49.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:49.274+0000 7f15404b6640 1 --2- 172.21.15.38:0/1756258273 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f15380ce880 0x7f15380cec90 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:49.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:49.274+0000 7f15404b6640 1 --2- 172.21.15.38:0/1756258273 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1538162280 0x7f153814e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:49.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:49.274+0000 7f15404b6640 1 --2- 172.21.15.38:0/1756258273 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f151c078b90 0x7f151c07b0a0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:49.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:49.274+0000 7f15404b6640 1 --2- 172.21.15.38:0/1756258273 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f15380b2e40 0x7f15380aadc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:49.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:49.274+0000 7f15404b6640 1 -- 172.21.15.38:0/1756258273 >> 172.21.15.38:0/1756258273 conn(0x7f15380b6b20 msgr2=0x7f15380b7950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:49.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:49.274+0000 7f15404b6640 1 -- 172.21.15.38:0/1756258273 shutdown_connections 2024-09-21T03:34:49.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:49.274+0000 7f15404b6640 1 -- 172.21.15.38:0/1756258273 wait complete. 2024-09-21T03:34:49.691 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 20, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T03:34:34.009269+0000', 'modified': '2024-09-21T03:34:48.902672+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 41, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5226}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5226': {'gid': 5226, 'name': 'c', 'rank': 0, 'incarnation': 20, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.38:6834/3533947700', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.38:6832', 'nonce': 3533947700}, {'type': 'v1', 'addr': '172.21.15.38:6834', 'nonce': 3533947700}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-09-21T03:34:49.691 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_5226:up:replay 2024-09-21T03:34:50.692 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:34:50.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:50.818+0000 7f8f5cd58640 1 -- 172.21.15.38:0/2298229466 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8f580cfe00 msgr2=0x7f8f580d2a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:50.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:50.818+0000 7f8f5cd58640 1 --2- 172.21.15.38:0/2298229466 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8f580cfe00 0x7f8f580d2a20 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f8f48009830 tx=0x7f8f4802a510 comp rx=0 tx=0).stop 2024-09-21T03:34:50.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:50.818+0000 7f8f5cd58640 1 -- 172.21.15.38:0/2298229466 shutdown_connections 2024-09-21T03:34:50.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:50.818+0000 7f8f5cd58640 1 --2- 172.21.15.38:0/2298229466 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8f58162590 0x7f8f581629a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:50.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:50.818+0000 7f8f5cd58640 1 --2- 172.21.15.38:0/2298229466 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8f580cfe00 0x7f8f580d2a20 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:50.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:50.818+0000 7f8f5cd58640 1 -- 172.21.15.38:0/2298229466 >> 172.21.15.38:0/2298229466 conn(0x7f8f580d4440 msgr2=0x7f8f580d4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:50.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:50.819+0000 7f8f5cd58640 1 -- 172.21.15.38:0/2298229466 shutdown_connections 2024-09-21T03:34:50.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:50.819+0000 7f8f5cd58640 1 -- 172.21.15.38:0/2298229466 wait complete. 2024-09-21T03:34:50.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:50.820+0000 7f8f5cd58640 1 Processor -- start 2024-09-21T03:34:50.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:50.821+0000 7f8f5cd58640 1 -- start start 2024-09-21T03:34:50.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:50.822+0000 7f8f5cd58640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8f580b2e60 0x7f8f5814e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:50.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:50.822+0000 7f8f5cd58640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8f580cfe00 0x7f8f580c8930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:50.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:50.822+0000 7f8f5cd58640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8f58162590 0x7f8f580c8e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:50.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:50.822+0000 7f8f5cd58640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f8f580b3980 con 0x7f8f580b2e60 2024-09-21T03:34:50.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:50.822+0000 7f8f5cd58640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f8f580cf5a0 con 0x7f8f58162590 2024-09-21T03:34:50.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:50.822+0000 7f8f5cd58640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f8f580b3b00 con 0x7f8f580cfe00 2024-09-21T03:34:50.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:50.822+0000 7f8f5659c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8f580b2e60 0x7f8f5814e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:50.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:50.822+0000 7f8f5659c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8f580b2e60 0x7f8f5814e780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50286/0 (socket says 172.21.15.38:50286) 2024-09-21T03:34:50.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:50.822+0000 7f8f5659c640 1 -- 172.21.15.38:0/3439017312 learned_addr learned my addr 172.21.15.38:0/3439017312 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:34:50.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:50.822+0000 7f8f55d9b640 1 --2- 172.21.15.38:0/3439017312 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8f580cfe00 0x7f8f580c8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:50.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:50.822+0000 7f8f56d9d640 1 --2- 172.21.15.38:0/3439017312 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8f58162590 0x7f8f580c8e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:50.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:50.822+0000 7f8f5659c640 1 -- 172.21.15.38:0/3439017312 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8f580cfe00 msgr2=0x7f8f580c8930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:50.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:50.822+0000 7f8f5659c640 1 --2- 172.21.15.38:0/3439017312 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8f580cfe00 0x7f8f580c8930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:50.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:50.822+0000 7f8f5659c640 1 -- 172.21.15.38:0/3439017312 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8f58162590 msgr2=0x7f8f580c8e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:50.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:50.822+0000 7f8f5659c640 1 --2- 172.21.15.38:0/3439017312 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8f58162590 0x7f8f580c8e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:50.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:50.822+0000 7f8f5659c640 1 -- 172.21.15.38:0/3439017312 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f5814ecc0 con 0x7f8f580b2e60 2024-09-21T03:34:50.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:50.822+0000 7f8f55d9b640 1 --2- 172.21.15.38:0/3439017312 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8f580cfe00 0x7f8f580c8930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:34:50.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:50.822+0000 7f8f56d9d640 1 --2- 172.21.15.38:0/3439017312 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8f58162590 0x7f8f580c8e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:34:50.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:50.823+0000 7f8f5659c640 1 --2- 172.21.15.38:0/3439017312 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8f580b2e60 0x7f8f5814e780 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f8f48000c00 tx=0x7f8f48033c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:50.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:50.823+0000 7f8f46ffd640 1 -- 172.21.15.38:0/3439017312 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f48005c90 con 0x7f8f580b2e60 2024-09-21T03:34:50.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:50.823+0000 7f8f46ffd640 1 -- 172.21.15.38:0/3439017312 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8f48031e50 con 0x7f8f580b2e60 2024-09-21T03:34:50.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:50.823+0000 7f8f5cd58640 1 -- 172.21.15.38:0/3439017312 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f5815dbf0 con 0x7f8f580b2e60 2024-09-21T03:34:50.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:50.823+0000 7f8f46ffd640 1 -- 172.21.15.38:0/3439017312 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f480332f0 con 0x7f8f580b2e60 2024-09-21T03:34:50.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:50.823+0000 7f8f5cd58640 1 -- 172.21.15.38:0/3439017312 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f5815e1b0 con 0x7f8f580b2e60 2024-09-21T03:34:50.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:50.824+0000 7f8f5cd58640 1 -- 172.21.15.38:0/3439017312 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f580d2a20 con 0x7f8f580b2e60 2024-09-21T03:34:50.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:50.825+0000 7f8f46ffd640 1 -- 172.21.15.38:0/3439017312 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f8f48033490 con 0x7f8f580b2e60 2024-09-21T03:34:50.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:50.825+0000 7f8f46ffd640 1 --2- 172.21.15.38:0/3439017312 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8f38078bc0 0x7f8f3807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:50.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:50.825+0000 7f8f46ffd640 1 -- 172.21.15.38:0/3439017312 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6187+0+0 (secure 0 0 0) 0x7f8f480bc310 con 0x7f8f580b2e60 2024-09-21T03:34:50.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:50.826+0000 7f8f55d9b640 1 --2- 172.21.15.38:0/3439017312 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8f38078bc0 0x7f8f3807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:50.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:50.826+0000 7f8f55d9b640 1 --2- 172.21.15.38:0/3439017312 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8f38078bc0 0x7f8f3807b0d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f8f580d4ca0 tx=0x7f8f4c005f10 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:50.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:50.830+0000 7f8f46ffd640 1 -- 172.21.15.38:0/3439017312 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f8f4803f030 con 0x7f8f580b2e60 2024-09-21T03:34:51.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:51.060+0000 7f8f5cd58640 1 -- 172.21.15.38:0/3439017312 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8f58158b60 con 0x7f8f580b2e60 2024-09-21T03:34:51.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:51.060+0000 7f8f46ffd640 1 -- 172.21.15.38:0/3439017312 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) ==== 94+0+5223 (secure 0 0 0) 0x7f8f48084690 con 0x7f8f580b2e60 2024-09-21T03:34:51.063 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:34:51.063 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":22,"btime":"2024-09-21T03:34:50:912076+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4757,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/1905955399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":1905955399},{"type":"v1","addr":"172.21.15.195:6836","nonce":1905955399}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4936,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/2989374027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":2989374027},{"type":"v1","addr":"172.21.15.195:6837","nonce":2989374027}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4942,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/3531731501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6833","nonce":3531731501},{"type":"v1","addr":"172.21.15.38:6835","nonce":3531731501}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:34:34.009269+0000","modified":"2024-09-21T03:34:49.913663+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":41,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5226},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5226":{"gid":5226,"name":"c","rank":0,"incarnation":20,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.38:6834/3533947700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":3533947700},{"type":"v1","addr":"172.21.15.38:6834","nonce":3533947700}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":2}]} 2024-09-21T03:34:51.064 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 22 2024-09-21T03:34:51.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:51.062+0000 7f8f5cd58640 1 -- 172.21.15.38:0/3439017312 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8f38078bc0 msgr2=0x7f8f3807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:51.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:51.062+0000 7f8f5cd58640 1 --2- 172.21.15.38:0/3439017312 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8f38078bc0 0x7f8f3807b0d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f8f580d4ca0 tx=0x7f8f4c005f10 comp rx=0 tx=0).stop 2024-09-21T03:34:51.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:51.062+0000 7f8f5cd58640 1 -- 172.21.15.38:0/3439017312 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8f580b2e60 msgr2=0x7f8f5814e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:51.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:51.062+0000 7f8f5cd58640 1 --2- 172.21.15.38:0/3439017312 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8f580b2e60 0x7f8f5814e780 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f8f48000c00 tx=0x7f8f48033c90 comp rx=0 tx=0).stop 2024-09-21T03:34:51.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:51.062+0000 7f8f5cd58640 1 -- 172.21.15.38:0/3439017312 shutdown_connections 2024-09-21T03:34:51.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:51.062+0000 7f8f5cd58640 1 --2- 172.21.15.38:0/3439017312 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8f58162590 0x7f8f580c8e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:51.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:51.062+0000 7f8f5cd58640 1 --2- 172.21.15.38:0/3439017312 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8f38078bc0 0x7f8f3807b0d0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:51.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:51.062+0000 7f8f5cd58640 1 --2- 172.21.15.38:0/3439017312 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8f580cfe00 0x7f8f580c8930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:51.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:51.062+0000 7f8f5cd58640 1 --2- 172.21.15.38:0/3439017312 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8f580b2e60 0x7f8f5814e780 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:51.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:51.062+0000 7f8f5cd58640 1 -- 172.21.15.38:0/3439017312 >> 172.21.15.38:0/3439017312 conn(0x7f8f580d4440 msgr2=0x7f8f580b3270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:51.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:51.063+0000 7f8f5cd58640 1 -- 172.21.15.38:0/3439017312 shutdown_connections 2024-09-21T03:34:51.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:51.063+0000 7f8f5cd58640 1 -- 172.21.15.38:0/3439017312 wait complete. 2024-09-21T03:34:51.479 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 22, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T03:34:34.009269+0000', 'modified': '2024-09-21T03:34:49.913663+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 41, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5226}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5226': {'gid': 5226, 'name': 'c', 'rank': 0, 'incarnation': 20, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.38:6834/3533947700', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.38:6832', 'nonce': 3533947700}, {'type': 'v1', 'addr': '172.21.15.38:6834', 'nonce': 3533947700}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-09-21T03:34:51.479 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_5226:up:rejoin 2024-09-21T03:34:52.480 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:34:52.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:52.608+0000 7ff8ad19d640 1 -- 172.21.15.38:0/1983465435 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff8a81627f0 msgr2=0x7ff8a806d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:52.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:52.608+0000 7ff8ad19d640 1 --2- 172.21.15.38:0/1983465435 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff8a81627f0 0x7ff8a806d2b0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7ff8a80b6de0 tx=0x7ff89c02a560 comp rx=0 tx=0).stop 2024-09-21T03:34:52.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:52.608+0000 7ff8ad19d640 1 -- 172.21.15.38:0/1983465435 shutdown_connections 2024-09-21T03:34:52.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:52.608+0000 7ff8ad19d640 1 --2- 172.21.15.38:0/1983465435 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff8a806d7f0 0x7ff8a8154ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:52.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:52.608+0000 7ff8ad19d640 1 --2- 172.21.15.38:0/1983465435 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff8a81627f0 0x7ff8a806d2b0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:52.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:52.608+0000 7ff8ad19d640 1 --2- 172.21.15.38:0/1983465435 >> v2:172.21.15.195:3300/0 conn(0x7ff8a80b9e40 0x7ff8a81622b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:52.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:52.608+0000 7ff8ad19d640 1 -- 172.21.15.38:0/1983465435 >> 172.21.15.38:0/1983465435 conn(0x7ff8a80d1930 msgr2=0x7ff8a80d42e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:52.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:52.608+0000 7ff8ad19d640 1 -- 172.21.15.38:0/1983465435 shutdown_connections 2024-09-21T03:34:52.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:52.608+0000 7ff8ad19d640 1 -- 172.21.15.38:0/1983465435 wait complete. 2024-09-21T03:34:52.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:52.610+0000 7ff8ad19d640 1 Processor -- start 2024-09-21T03:34:52.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:52.610+0000 7ff8ad19d640 1 -- start start 2024-09-21T03:34:52.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:52.613+0000 7ff8ad19d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff8a806d7f0 0x7ff8a81587f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:52.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:52.613+0000 7ff8ad19d640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff8a80b9e40 0x7ff8a81602d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:52.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:52.613+0000 7ff8ad19d640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff8a81627f0 0x7ff8a81615a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:52.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:52.613+0000 7ff8ad19d640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ff8a81555f0 con 0x7ff8a806d7f0 2024-09-21T03:34:52.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:52.613+0000 7ff8ad19d640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ff8a806e7f0 con 0x7ff8a81627f0 2024-09-21T03:34:52.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:52.613+0000 7ff8ad19d640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7ff8a8155770 con 0x7ff8a80b9e40 2024-09-21T03:34:52.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:52.613+0000 7ff8a7fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff8a806d7f0 0x7ff8a81587f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:52.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:52.613+0000 7ff8a7fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff8a806d7f0 0x7ff8a81587f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35416/0 (socket says 172.21.15.38:35416) 2024-09-21T03:34:52.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:52.613+0000 7ff8a7fff640 1 -- 172.21.15.38:0/3215668274 learned_addr learned my addr 172.21.15.38:0/3215668274 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:34:52.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:52.613+0000 7ff8a77fe640 1 --2- 172.21.15.38:0/3215668274 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff8a80b9e40 0x7ff8a81602d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:52.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:52.613+0000 7ff8ac99c640 1 --2- 172.21.15.38:0/3215668274 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff8a81627f0 0x7ff8a81615a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:52.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:52.613+0000 7ff8a7fff640 1 -- 172.21.15.38:0/3215668274 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff8a80b9e40 msgr2=0x7ff8a81602d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:52.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:52.613+0000 7ff8a7fff640 1 --2- 172.21.15.38:0/3215668274 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff8a80b9e40 0x7ff8a81602d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:52.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:52.613+0000 7ff8a7fff640 1 -- 172.21.15.38:0/3215668274 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff8a81627f0 msgr2=0x7ff8a81615a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:52.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:52.613+0000 7ff8a7fff640 1 --2- 172.21.15.38:0/3215668274 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff8a81627f0 0x7ff8a81615a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:52.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:52.613+0000 7ff8a7fff640 1 -- 172.21.15.38:0/3215668274 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8a81f9820 con 0x7ff8a806d7f0 2024-09-21T03:34:52.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:52.613+0000 7ff8a7fff640 1 --2- 172.21.15.38:0/3215668274 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff8a806d7f0 0x7ff8a81587f0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7ff89800d770 tx=0x7ff89800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:52.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:52.613+0000 7ff8a4ff9640 1 -- 172.21.15.38:0/3215668274 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff898017070 con 0x7ff8a806d7f0 2024-09-21T03:34:52.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:52.613+0000 7ff8a4ff9640 1 -- 172.21.15.38:0/3215668274 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff898009d20 con 0x7ff8a806d7f0 2024-09-21T03:34:52.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:52.613+0000 7ff8ad19d640 1 -- 172.21.15.38:0/3215668274 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8a8161ae0 con 0x7ff8a806d7f0 2024-09-21T03:34:52.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:52.613+0000 7ff8a4ff9640 1 -- 172.21.15.38:0/3215668274 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8980106c0 con 0x7ff8a806d7f0 2024-09-21T03:34:52.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:52.613+0000 7ff8ad19d640 1 -- 172.21.15.38:0/3215668274 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8a81f9260 con 0x7ff8a806d7f0 2024-09-21T03:34:52.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:52.614+0000 7ff8ad19d640 1 -- 172.21.15.38:0/3215668274 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8a81622b0 con 0x7ff8a806d7f0 2024-09-21T03:34:52.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:52.615+0000 7ff8a4ff9640 1 -- 172.21.15.38:0/3215668274 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ff89801a070 con 0x7ff8a806d7f0 2024-09-21T03:34:52.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:52.615+0000 7ff8a4ff9640 1 --2- 172.21.15.38:0/3215668274 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff894078b90 0x7ff89407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:52.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:52.615+0000 7ff8a4ff9640 1 -- 172.21.15.38:0/3215668274 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6187+0+0 (secure 0 0 0) 0x7ff89809b460 con 0x7ff8a806d7f0 2024-09-21T03:34:52.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:52.616+0000 7ff8a77fe640 1 --2- 172.21.15.38:0/3215668274 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff894078b90 0x7ff89407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:52.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:52.616+0000 7ff8a77fe640 1 --2- 172.21.15.38:0/3215668274 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff894078b90 0x7ff89407b0a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ff89c033000 tx=0x7ff89c004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:52.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:52.619+0000 7ff8a4ff9640 1 -- 172.21.15.38:0/3215668274 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7ff898063150 con 0x7ff8a806d7f0 2024-09-21T03:34:52.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:52.845+0000 7ff8ad19d640 1 -- 172.21.15.38:0/3215668274 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff8a81f5330 con 0x7ff8a806d7f0 2024-09-21T03:34:52.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:52.845+0000 7ff8a4ff9640 1 -- 172.21.15.38:0/3215668274 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+5230 (secure 0 0 0) 0x7ff898067b80 con 0x7ff8a806d7f0 2024-09-21T03:34:52.848 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:34:52.848 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":23,"btime":"2024-09-21T03:34:51:918500+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4757,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/1905955399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":1905955399},{"type":"v1","addr":"172.21.15.195:6836","nonce":1905955399}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4936,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/2989374027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":2989374027},{"type":"v1","addr":"172.21.15.195:6837","nonce":2989374027}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4942,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/3531731501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6833","nonce":3531731501},{"type":"v1","addr":"172.21.15.38:6835","nonce":3531731501}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:34:34.009269+0000","modified":"2024-09-21T03:34:51.918491+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":41,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5226},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5226":{"gid":5226,"name":"c","rank":0,"incarnation":20,"state":"up:active","state_seq":4,"addr":"172.21.15.38:6834/3533947700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":3533947700},{"type":"v1","addr":"172.21.15.38:6834","nonce":3533947700}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5226,"qdb_cluster":[5226]},"id":2}]} 2024-09-21T03:34:52.848 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 23 2024-09-21T03:34:52.851 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:52.848+0000 7ff8ad19d640 1 -- 172.21.15.38:0/3215668274 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff894078b90 msgr2=0x7ff89407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:52.851 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:52.848+0000 7ff8ad19d640 1 --2- 172.21.15.38:0/3215668274 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff894078b90 0x7ff89407b0a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ff89c033000 tx=0x7ff89c004730 comp rx=0 tx=0).stop 2024-09-21T03:34:52.851 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:52.848+0000 7ff8ad19d640 1 -- 172.21.15.38:0/3215668274 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff8a806d7f0 msgr2=0x7ff8a81587f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:52.851 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:52.848+0000 7ff8ad19d640 1 --2- 172.21.15.38:0/3215668274 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff8a806d7f0 0x7ff8a81587f0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7ff89800d770 tx=0x7ff89800dc30 comp rx=0 tx=0).stop 2024-09-21T03:34:52.851 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:52.848+0000 7ff8ad19d640 1 -- 172.21.15.38:0/3215668274 shutdown_connections 2024-09-21T03:34:52.851 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:52.848+0000 7ff8ad19d640 1 --2- 172.21.15.38:0/3215668274 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff8a81627f0 0x7ff8a81615a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:52.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:52.848+0000 7ff8ad19d640 1 --2- 172.21.15.38:0/3215668274 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff894078b90 0x7ff89407b0a0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:52.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:52.848+0000 7ff8ad19d640 1 --2- 172.21.15.38:0/3215668274 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff8a80b9e40 0x7ff8a81602d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:52.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:52.848+0000 7ff8ad19d640 1 --2- 172.21.15.38:0/3215668274 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff8a806d7f0 0x7ff8a81587f0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:52.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:52.848+0000 7ff8ad19d640 1 -- 172.21.15.38:0/3215668274 >> 172.21.15.38:0/3215668274 conn(0x7ff8a80d1930 msgr2=0x7ff8a80cc560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:52.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:52.848+0000 7ff8ad19d640 1 -- 172.21.15.38:0/3215668274 shutdown_connections 2024-09-21T03:34:52.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:52.848+0000 7ff8ad19d640 1 -- 172.21.15.38:0/3215668274 wait complete. 2024-09-21T03:34:53.265 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 23, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T03:34:34.009269+0000', 'modified': '2024-09-21T03:34:51.918491+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 41, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5226}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5226': {'gid': 5226, 'name': 'c', 'rank': 0, 'incarnation': 20, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.38:6834/3533947700', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.38:6832', 'nonce': 3533947700}, {'type': 'v1', 'addr': '172.21.15.38:6834', 'nonce': 3533947700}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5226, 'qdb_cluster': [5226]} 2024-09-21T03:34:53.266 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-21T03:34:53.266 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-21T03:34:53.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.384+0000 7fc2e7238640 1 -- 172.21.15.38:0/2390545862 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc2e00ceb90 msgr2=0x7fc2e00cefa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:53.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.384+0000 7fc2e7238640 1 --2- 172.21.15.38:0/2390545862 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc2e00ceb90 0x7fc2e00cefa0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fc2d8005fd0 tx=0x7fc2d802aa10 comp rx=0 tx=0).stop 2024-09-21T03:34:53.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.384+0000 7fc2e7238640 1 -- 172.21.15.38:0/2390545862 shutdown_connections 2024-09-21T03:34:53.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.384+0000 7fc2e7238640 1 --2- 172.21.15.38:0/2390545862 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc2e00ceb90 0x7fc2e00cefa0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:53.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.384+0000 7fc2e7238640 1 --2- 172.21.15.38:0/2390545862 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc2e015ee80 0x7fc2e00d0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:53.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.384+0000 7fc2e7238640 1 -- 172.21.15.38:0/2390545862 >> 172.21.15.38:0/2390545862 conn(0x7fc2e009d830 msgr2=0x7fc2e00b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:53.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.384+0000 7fc2e7238640 1 -- 172.21.15.38:0/2390545862 shutdown_connections 2024-09-21T03:34:53.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.384+0000 7fc2e7238640 1 -- 172.21.15.38:0/2390545862 wait complete. 2024-09-21T03:34:53.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.386+0000 7fc2e7238640 1 Processor -- start 2024-09-21T03:34:53.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.386+0000 7fc2e7238640 1 -- start start 2024-09-21T03:34:53.391 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.388+0000 7fc2e7238640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc2e00b9ad0 0x7fc2e006efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:53.391 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.388+0000 7fc2e7238640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc2e00ceb90 0x7fc2e006f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:53.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.388+0000 7fc2e7238640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc2e015ee80 0x7fc2e0148680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:53.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.388+0000 7fc2e7238640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fc2e00b6700 con 0x7fc2e00ceb90 2024-09-21T03:34:53.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.388+0000 7fc2e7238640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fc2e00b6580 con 0x7fc2e015ee80 2024-09-21T03:34:53.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.388+0000 7fc2e7238640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fc2e00b6880 con 0x7fc2e00b9ad0 2024-09-21T03:34:53.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.388+0000 7fc2d7fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc2e00ceb90 0x7fc2e006f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:53.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.388+0000 7fc2d7fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc2e00ceb90 0x7fc2e006f4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35426/0 (socket says 172.21.15.38:35426) 2024-09-21T03:34:53.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.388+0000 7fc2d7fff640 1 -- 172.21.15.38:0/4223788724 learned_addr learned my addr 172.21.15.38:0/4223788724 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:34:53.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.388+0000 7fc2e4fd4640 1 --2- 172.21.15.38:0/4223788724 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc2e00b9ad0 0x7fc2e006efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:53.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.388+0000 7fc2e57d5640 1 --2- 172.21.15.38:0/4223788724 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc2e015ee80 0x7fc2e0148680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:53.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.388+0000 7fc2d7fff640 1 -- 172.21.15.38:0/4223788724 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc2e00b9ad0 msgr2=0x7fc2e006efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:53.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.388+0000 7fc2d7fff640 1 --2- 172.21.15.38:0/4223788724 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc2e00b9ad0 0x7fc2e006efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:53.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.388+0000 7fc2d7fff640 1 -- 172.21.15.38:0/4223788724 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc2e015ee80 msgr2=0x7fc2e0148680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:53.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.388+0000 7fc2d7fff640 1 --2- 172.21.15.38:0/4223788724 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc2e015ee80 0x7fc2e0148680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:53.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.388+0000 7fc2d7fff640 1 -- 172.21.15.38:0/4223788724 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2e0155990 con 0x7fc2e00ceb90 2024-09-21T03:34:53.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.388+0000 7fc2d7fff640 1 --2- 172.21.15.38:0/4223788724 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc2e00ceb90 0x7fc2e006f4f0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fc2d000c440 tx=0x7fc2d000c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:53.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.389+0000 7fc2d57fa640 1 -- 172.21.15.38:0/4223788724 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2d0017070 con 0x7fc2e00ceb90 2024-09-21T03:34:53.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.389+0000 7fc2e7238640 1 -- 172.21.15.38:0/4223788724 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2e01f55e0 con 0x7fc2e00ceb90 2024-09-21T03:34:53.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.389+0000 7fc2d57fa640 1 -- 172.21.15.38:0/4223788724 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc2d00053c0 con 0x7fc2e00ceb90 2024-09-21T03:34:53.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.389+0000 7fc2d57fa640 1 -- 172.21.15.38:0/4223788724 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2d00038a0 con 0x7fc2e00ceb90 2024-09-21T03:34:53.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.389+0000 7fc2e7238640 1 -- 172.21.15.38:0/4223788724 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2e01f5ba0 con 0x7fc2e00ceb90 2024-09-21T03:34:53.394 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.391+0000 7fc2d57fa640 1 -- 172.21.15.38:0/4223788724 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc2d001a070 con 0x7fc2e00ceb90 2024-09-21T03:34:53.394 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.391+0000 7fc2d57fa640 1 --2- 172.21.15.38:0/4223788724 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc2cc078b90 0x7fc2cc07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:53.394 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.391+0000 7fc2d57fa640 1 -- 172.21.15.38:0/4223788724 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6187+0+0 (secure 0 0 0) 0x7fc2d009a150 con 0x7fc2e00ceb90 2024-09-21T03:34:53.394 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.391+0000 7fc2e4fd4640 1 --2- 172.21.15.38:0/4223788724 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc2cc078b90 0x7fc2cc07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:53.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.392+0000 7fc2e4fd4640 1 --2- 172.21.15.38:0/4223788724 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc2cc078b90 0x7fc2cc07b0a0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fc2dc0095f0 tx=0x7fc2dc005f90 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:53.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.392+0000 7fc2ee283740 1 Processor -- start 2024-09-21T03:34:53.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.392+0000 7fc2ee283740 1 -- start start 2024-09-21T03:34:53.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.393+0000 7fc2ee283740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a7c750e270 0x55a7c78dffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:53.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.393+0000 7fc2ee283740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a7c74e53b0 0x55a7c78e04e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:53.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.393+0000 7fc2ee283740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a7c74ed590 0x55a7c78fd3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:53.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.393+0000 7fc2ee283740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55a7c7828820 con 0x55a7c74e53b0 2024-09-21T03:34:53.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.393+0000 7fc2ee283740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55a7c7817c50 con 0x55a7c74ed590 2024-09-21T03:34:53.398 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.393+0000 7fc2ee283740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55a7c781ba60 con 0x55a7c750e270 2024-09-21T03:34:53.398 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.394+0000 7fc2d7fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a7c74e53b0 0x55a7c78e04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:53.398 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.394+0000 7fc2d7fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a7c74e53b0 0x55a7c78e04e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35440/0 (socket says 172.21.15.38:35440) 2024-09-21T03:34:53.398 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.394+0000 7fc2d7fff640 1 -- 172.21.15.38:0/2088548175 learned_addr learned my addr 172.21.15.38:0/2088548175 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:34:53.398 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.394+0000 7fc2e4fd4640 1 --2- 172.21.15.38:0/2088548175 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a7c750e270 0x55a7c78dffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:53.398 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.394+0000 7fc2e57d5640 1 --2- 172.21.15.38:0/2088548175 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a7c74ed590 0x55a7c78fd3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:53.398 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.394+0000 7fc2d7fff640 1 -- 172.21.15.38:0/2088548175 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a7c750e270 msgr2=0x55a7c78dffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:53.398 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.394+0000 7fc2d7fff640 1 --2- 172.21.15.38:0/2088548175 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a7c750e270 0x55a7c78dffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:53.398 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.394+0000 7fc2d7fff640 1 -- 172.21.15.38:0/2088548175 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a7c74ed590 msgr2=0x55a7c78fd3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:53.398 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.394+0000 7fc2d7fff640 1 --2- 172.21.15.38:0/2088548175 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a7c74ed590 0x55a7c78fd3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:53.398 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.394+0000 7fc2d7fff640 1 -- 172.21.15.38:0/2088548175 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7c7579850 con 0x55a7c74e53b0 2024-09-21T03:34:53.398 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.394+0000 7fc2d7fff640 1 --2- 172.21.15.38:0/2088548175 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a7c74e53b0 0x55a7c78e04e0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fc2d000ce00 tx=0x7fc2d0061570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8695e3079a82cc43 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:53.398 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.394+0000 7fc2c67fc640 1 -- 172.21.15.38:0/2088548175 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2d0066830 con 0x55a7c74e53b0 2024-09-21T03:34:53.399 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.394+0000 7fc2c67fc640 1 -- 172.21.15.38:0/2088548175 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc2d006b6e0 con 0x55a7c74e53b0 2024-09-21T03:34:53.399 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.394+0000 7fc2c67fc640 1 -- 172.21.15.38:0/2088548175 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2d0082bd0 con 0x55a7c74e53b0 2024-09-21T03:34:53.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.395+0000 7fc2ee283740 1 -- 172.21.15.38:0/2088548175 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a7c74e53b0 msgr2=0x55a7c78e04e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:53.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.395+0000 7fc2ee283740 1 --2- 172.21.15.38:0/2088548175 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a7c74e53b0 0x55a7c78e04e0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fc2d000ce00 tx=0x7fc2d0061570 comp rx=0 tx=0).stop 2024-09-21T03:34:53.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.395+0000 7fc2ee283740 1 -- 172.21.15.38:0/2088548175 shutdown_connections 2024-09-21T03:34:53.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.395+0000 7fc2ee283740 1 --2- 172.21.15.38:0/2088548175 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a7c74ed590 0x55a7c78fd3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:53.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.395+0000 7fc2ee283740 1 --2- 172.21.15.38:0/2088548175 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a7c74e53b0 0x55a7c78e04e0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:53.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.395+0000 7fc2ee283740 1 --2- 172.21.15.38:0/2088548175 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a7c750e270 0x55a7c78dffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:53.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.395+0000 7fc2ee283740 1 -- 172.21.15.38:0/2088548175 >> 172.21.15.38:0/2088548175 conn(0x55a7c75023c0 msgr2=0x55a7c78b6260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:53.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.395+0000 7fc2ee283740 1 -- 172.21.15.38:0/2088548175 shutdown_connections 2024-09-21T03:34:53.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.395+0000 7fc2ee283740 1 -- 172.21.15.38:0/2088548175 wait complete. 2024-09-21T03:34:53.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.396+0000 7fc2ee283740 1 -- start start 2024-09-21T03:34:53.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.396+0000 7fc2ee283740 1 Processor -- start 2024-09-21T03:34:53.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.398+0000 7fc2ee283740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a7c74e53b0 0x55a7c7909e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:53.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.398+0000 7fc2ee283740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a7c74ed590 0x55a7c78f2670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:53.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.398+0000 7fc2ee283740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a7c750e270 0x55a7c78f6630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:53.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.398+0000 7fc2ee283740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55a7c7588620 con 0x55a7c750e270 2024-09-21T03:34:53.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.398+0000 7fc2ee283740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55a7c77bd5f0 con 0x55a7c74e53b0 2024-09-21T03:34:53.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.398+0000 7fc2ee283740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55a7c78e0a20 con 0x55a7c74ed590 2024-09-21T03:34:53.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.398+0000 7fc2ee283740 10 client.0 _ll_register_callbacks cb 0x55a7c77c10a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:34:53.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.398+0000 7fc2e57d5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a7c750e270 0x55a7c78f6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:53.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.398+0000 7fc2e57d5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a7c750e270 0x55a7c78f6630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35456/0 (socket says 172.21.15.38:35456) 2024-09-21T03:34:53.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.398+0000 7fc2e57d5640 1 -- 172.21.15.38:0/241053173 learned_addr learned my addr 172.21.15.38:0/241053173 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:34:53.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.398+0000 7fc2d7fff640 1 --2- 172.21.15.38:0/241053173 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a7c74ed590 0x55a7c78f2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:53.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.398+0000 7fc2e4fd4640 1 --2- 172.21.15.38:0/241053173 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a7c74e53b0 0x55a7c7909e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:53.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.398+0000 7fc2e57d5640 1 -- 172.21.15.38:0/241053173 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a7c74ed590 msgr2=0x55a7c78f2670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:53.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.398+0000 7fc2e57d5640 1 --2- 172.21.15.38:0/241053173 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a7c74ed590 0x55a7c78f2670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:53.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.398+0000 7fc2e57d5640 1 -- 172.21.15.38:0/241053173 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a7c74e53b0 msgr2=0x55a7c7909e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:53.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.398+0000 7fc2e57d5640 1 --2- 172.21.15.38:0/241053173 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a7c74e53b0 0x55a7c7909e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:53.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.398+0000 7fc2e57d5640 1 -- 172.21.15.38:0/241053173 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a7c75f06b0 con 0x55a7c750e270 2024-09-21T03:34:53.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.398+0000 7fc2e57d5640 1 --2- 172.21.15.38:0/241053173 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a7c750e270 0x55a7c78f6630 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fc2d802b000 tx=0x7fc2d802af90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=783683fe254f269f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:53.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.398+0000 7fc2bf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-21T03:34:53.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.398+0000 7fc2bf7fe640 1 -- 172.21.15.38:0/241053173 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2d8033040 con 0x55a7c750e270 2024-09-21T03:34:53.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.398+0000 7fc2bf7fe640 1 -- 172.21.15.38:0/241053173 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(41..41 src has 1..41) ==== 6187+0+0 (secure 0 0 0) 0x7fc2d8003d80 con 0x55a7c750e270 2024-09-21T03:34:53.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.398+0000 7fc2ee283740 1 -- 172.21.15.38:0/241053173 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc2d006b6e0 con 0x55a7c750e270 2024-09-21T03:34:53.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.399+0000 7fc2bf7fe640 1 -- 172.21.15.38:0/241053173 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fc2d8050050 con 0x55a7c750e270 2024-09-21T03:34:53.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.399+0000 7fc2ee283740 10 client.5274 fetch_fsmap learned FSMap version 23 2024-09-21T03:34:53.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.399+0000 7fc2ee283740 1 -- 172.21.15.38:0/241053173 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) -- 0x55a7c78f2c00 con 0x55a7c750e270 2024-09-21T03:34:53.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.399+0000 7fc2bf7fe640 1 -- 172.21.15.38:0/241053173 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc2d006b6e0 con 0x55a7c750e270 2024-09-21T03:34:53.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.399+0000 7fc2bf7fe640 1 -- 172.21.15.38:0/241053173 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 23) ==== 3502+0+0 (secure 0 0 0) 0x7fc2d8053020 con 0x55a7c750e270 2024-09-21T03:34:53.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.399+0000 7fc2ee283740 10 client.5274 fetch_fsmap finished waiting for FSMap version 23 2024-09-21T03:34:53.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.399+0000 7fc2ee283740 10 client.5274 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T03:34:53.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.399+0000 7fc2ee283740 20 client.5274 populate_metadata read hostname 'smithi038' 2024-09-21T03:34:53.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.399+0000 7fc2ee283740 1 --2- 172.21.15.38:0/241053173 >> [v2:172.21.15.38:6832/3533947700,v1:172.21.15.38:6834/3533947700] conn(0x55a7c74cce40 0x55a7c790d470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:53.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.399+0000 7fc2ee283740 4 client.5274 mds_command: new command op to 5226 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:34:53.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.399+0000 7fc2ee283740 1 -- 172.21.15.38:0/241053173 --> [v2:172.21.15.38:6832/3533947700,v1:172.21.15.38:6834/3533947700] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a7c782a770 con 0x55a7c74cce40 2024-09-21T03:34:53.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.399+0000 7fc2e4fd4640 1 --2- 172.21.15.38:0/241053173 >> [v2:172.21.15.38:6832/3533947700,v1:172.21.15.38:6834/3533947700] conn(0x55a7c74cce40 0x55a7c790d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:53.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.399+0000 7fc2bf7fe640 10 client.5274 ms_handle_connect on v2:172.21.15.38:6832/3533947700 2024-09-21T03:34:53.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.400+0000 7fc2e4fd4640 1 --2- 172.21.15.38:0/241053173 >> [v2:172.21.15.38:6832/3533947700,v1:172.21.15.38:6834/3533947700] conn(0x55a7c74cce40 0x55a7c790d470 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7cd21d6499c8f67a server_cookie=d81764626f24da1e in_seq=0 out_seq=0 2024-09-21T03:34:53.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.400+0000 7fc2bf7fe640 1 -- 172.21.15.38:0/241053173 <== mds.0 v2:172.21.15.38:6832/3533947700 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a7c782a770 con 0x55a7c74cce40 2024-09-21T03:34:53.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.400+0000 7fc2bf7fe640 10 client.5274 handle_command_reply: tid=0 2024-09-21T03:34:53.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.400+0000 7fc2ee283740 1 client.5274 shutdown 2024-09-21T03:34:53.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.400+0000 7fc2ee283740 1 -- 172.21.15.38:0/241053173 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a7c750e270 msgr2=0x55a7c78f6630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:53.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.400+0000 7fc2ee283740 1 --2- 172.21.15.38:0/241053173 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a7c750e270 0x55a7c78f6630 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fc2d802b000 tx=0x7fc2d802af90 comp rx=0 tx=0).stop 2024-09-21T03:34:53.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.400+0000 7fc2ee283740 1 -- 172.21.15.38:0/241053173 shutdown_connections 2024-09-21T03:34:53.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.400+0000 7fc2ee283740 1 --2- 172.21.15.38:0/241053173 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a7c750e270 0x55a7c78f6630 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:53.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.401+0000 7fc2ee283740 1 --2- 172.21.15.38:0/241053173 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a7c74ed590 0x55a7c78f2670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:53.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.401+0000 7fc2ee283740 1 --2- 172.21.15.38:0/241053173 >> [v2:172.21.15.38:6832/3533947700,v1:172.21.15.38:6834/3533947700] conn(0x55a7c74cce40 0x55a7c790d470 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:53.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.401+0000 7fc2ee283740 1 --2- 172.21.15.38:0/241053173 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a7c74e53b0 0x55a7c7909e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:53.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.401+0000 7fc2bf7fe640 0 client.5274 ms_handle_reset on v2:172.21.15.38:6832/3533947700 2024-09-21T03:34:53.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.401+0000 7fc2ee283740 1 -- 172.21.15.38:0/241053173 >> 172.21.15.38:0/241053173 conn(0x55a7c75023c0 msgr2=0x55a7c78b0b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:53.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.401+0000 7fc2ee283740 1 -- 172.21.15.38:0/241053173 shutdown_connections 2024-09-21T03:34:53.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.401+0000 7fc2ee283740 1 -- 172.21.15.38:0/241053173 wait complete. 2024-09-21T03:34:53.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.401+0000 7fc2ee283740 20 client.5274 trim_cache size 0 max 16384 2024-09-21T03:34:53.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.419+0000 7fc2ee283740 1 Processor -- start 2024-09-21T03:34:53.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.419+0000 7fc2ee283740 1 -- start start 2024-09-21T03:34:53.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.420+0000 7fc2ee283740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a7c74e53b0 0x55a7c78fb500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:53.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.420+0000 7fc2ee283740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a7c74ed590 0x55a7c78f5a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:53.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.420+0000 7fc2ee283740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a7c750e270 0x55a7c78f5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:53.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.420+0000 7fc2ee283740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fc2e0201800 con 0x55a7c750e270 2024-09-21T03:34:53.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.420+0000 7fc2ee283740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fc2e0200c30 con 0x55a7c74ed590 2024-09-21T03:34:53.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.420+0000 7fc2ee283740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fc2e02011b0 con 0x55a7c74e53b0 2024-09-21T03:34:53.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.421+0000 7fc2e57d5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a7c750e270 0x55a7c78f5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:53.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.421+0000 7fc2e57d5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a7c750e270 0x55a7c78f5fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35462/0 (socket says 172.21.15.38:35462) 2024-09-21T03:34:53.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.421+0000 7fc2e57d5640 1 -- 172.21.15.38:0/4165236619 learned_addr learned my addr 172.21.15.38:0/4165236619 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:34:53.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.421+0000 7fc2e4fd4640 1 --2- 172.21.15.38:0/4165236619 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a7c74e53b0 0x55a7c78fb500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:53.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.421+0000 7fc2d7fff640 1 --2- 172.21.15.38:0/4165236619 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a7c74ed590 0x55a7c78f5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:53.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.421+0000 7fc2e57d5640 1 -- 172.21.15.38:0/4165236619 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a7c74e53b0 msgr2=0x55a7c78fb500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:53.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.421+0000 7fc2e57d5640 1 --2- 172.21.15.38:0/4165236619 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a7c74e53b0 0x55a7c78fb500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:53.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.421+0000 7fc2e57d5640 1 -- 172.21.15.38:0/4165236619 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a7c74ed590 msgr2=0x55a7c78f5a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:53.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.421+0000 7fc2e57d5640 1 --2- 172.21.15.38:0/4165236619 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a7c74ed590 0x55a7c78f5a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:53.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.421+0000 7fc2e57d5640 1 -- 172.21.15.38:0/4165236619 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7c790a1f0 con 0x55a7c750e270 2024-09-21T03:34:53.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.421+0000 7fc2e57d5640 1 --2- 172.21.15.38:0/4165236619 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a7c750e270 0x55a7c78f5fa0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fc2d8005bd0 tx=0x7fc2d8005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d3bbd98a54a8947 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:53.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.421+0000 7fc2c5ffb640 1 -- 172.21.15.38:0/4165236619 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2d8033070 con 0x55a7c750e270 2024-09-21T03:34:53.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.421+0000 7fc2c5ffb640 1 -- 172.21.15.38:0/4165236619 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc2d8004b40 con 0x55a7c750e270 2024-09-21T03:34:53.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.421+0000 7fc2c5ffb640 1 -- 172.21.15.38:0/4165236619 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2d8004390 con 0x55a7c750e270 2024-09-21T03:34:53.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.422+0000 7fc2ee283740 1 -- 172.21.15.38:0/4165236619 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a7c750e270 msgr2=0x55a7c78f5fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:53.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.422+0000 7fc2ee283740 1 --2- 172.21.15.38:0/4165236619 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a7c750e270 0x55a7c78f5fa0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fc2d8005bd0 tx=0x7fc2d8005c00 comp rx=0 tx=0).stop 2024-09-21T03:34:53.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.422+0000 7fc2ee283740 1 -- 172.21.15.38:0/4165236619 shutdown_connections 2024-09-21T03:34:53.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.422+0000 7fc2ee283740 1 --2- 172.21.15.38:0/4165236619 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a7c750e270 0x55a7c78f5fa0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:53.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.422+0000 7fc2ee283740 1 --2- 172.21.15.38:0/4165236619 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a7c74ed590 0x55a7c78f5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:53.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.422+0000 7fc2ee283740 1 --2- 172.21.15.38:0/4165236619 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a7c74e53b0 0x55a7c78fb500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:53.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.422+0000 7fc2ee283740 1 -- 172.21.15.38:0/4165236619 >> 172.21.15.38:0/4165236619 conn(0x55a7c75023c0 msgr2=0x55a7c78b0b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:53.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.422+0000 7fc2ee283740 1 -- 172.21.15.38:0/4165236619 shutdown_connections 2024-09-21T03:34:53.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.422+0000 7fc2ee283740 1 -- 172.21.15.38:0/4165236619 wait complete. 2024-09-21T03:34:53.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.423+0000 7fc2ee283740 1 -- start start 2024-09-21T03:34:53.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.423+0000 7fc2ee283740 1 Processor -- start 2024-09-21T03:34:53.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.424+0000 7fc2ee283740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a7c74e53b0 0x55a7c7970840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:53.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.424+0000 7fc2ee283740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a7c74ed590 0x55a7c78b60f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:53.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.424+0000 7fc2ee283740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a7c750e270 0x55a7c78b6630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:53.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.424+0000 7fc2ee283740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55a7c78f5660 con 0x55a7c74ed590 2024-09-21T03:34:53.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.424+0000 7fc2ee283740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55a7c78f6c30 con 0x55a7c74e53b0 2024-09-21T03:34:53.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.424+0000 7fc2ee283740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55a7c78f40f0 con 0x55a7c750e270 2024-09-21T03:34:53.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.424+0000 7fc2ee283740 10 client.0 _ll_register_callbacks cb 0x7fc2d8009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:34:53.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.424+0000 7fc2d7fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a7c74ed590 0x55a7c78b60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:53.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.425+0000 7fc2d7fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a7c74ed590 0x55a7c78b60f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35468/0 (socket says 172.21.15.38:35468) 2024-09-21T03:34:53.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.425+0000 7fc2d7fff640 1 -- 172.21.15.38:0/2542547679 learned_addr learned my addr 172.21.15.38:0/2542547679 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:34:53.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.425+0000 7fc2e57d5640 1 --2- 172.21.15.38:0/2542547679 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a7c750e270 0x55a7c78b6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:53.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.425+0000 7fc2e4fd4640 1 --2- 172.21.15.38:0/2542547679 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a7c74e53b0 0x55a7c7970840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:53.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.425+0000 7fc2d7fff640 1 -- 172.21.15.38:0/2542547679 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a7c750e270 msgr2=0x55a7c78b6630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:53.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.425+0000 7fc2d7fff640 1 --2- 172.21.15.38:0/2542547679 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a7c750e270 0x55a7c78b6630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:53.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.425+0000 7fc2d7fff640 1 -- 172.21.15.38:0/2542547679 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a7c74e53b0 msgr2=0x55a7c7970840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:53.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.425+0000 7fc2d7fff640 1 --2- 172.21.15.38:0/2542547679 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a7c74e53b0 0x55a7c7970840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:53.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.425+0000 7fc2d7fff640 1 -- 172.21.15.38:0/2542547679 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a7c78fdf80 con 0x55a7c74ed590 2024-09-21T03:34:53.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.425+0000 7fc2e4fd4640 1 --2- 172.21.15.38:0/2542547679 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a7c74e53b0 0x55a7c7970840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:34:53.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.425+0000 7fc2d7fff640 1 --2- 172.21.15.38:0/2542547679 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a7c74ed590 0x55a7c78b60f0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fc2d0001f00 tx=0x7fc2d0084320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b95c79a1e6fbda27 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:53.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.425+0000 7fc2bf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-21T03:34:53.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.425+0000 7fc2bf7fe640 1 -- 172.21.15.38:0/2542547679 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2d0082dd0 con 0x55a7c74ed590 2024-09-21T03:34:53.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.425+0000 7fc2bf7fe640 1 -- 172.21.15.38:0/2542547679 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(41..41 src has 1..41) ==== 6187+0+0 (secure 0 0 0) 0x7fc2d0061060 con 0x55a7c74ed590 2024-09-21T03:34:53.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.425+0000 7fc2ee283740 1 -- 172.21.15.38:0/2542547679 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc2d8004b40 con 0x55a7c74ed590 2024-09-21T03:34:53.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.425+0000 7fc2bf7fe640 1 -- 172.21.15.38:0/2542547679 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fc2d006b6e0 con 0x55a7c74ed590 2024-09-21T03:34:53.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.425+0000 7fc2ee283740 10 client.5280 fetch_fsmap learned FSMap version 23 2024-09-21T03:34:53.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.425+0000 7fc2ee283740 1 -- 172.21.15.38:0/2542547679 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) -- 0x55a7c78fda30 con 0x55a7c74ed590 2024-09-21T03:34:53.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.426+0000 7fc2bf7fe640 1 -- 172.21.15.38:0/2542547679 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc2d8004b40 con 0x55a7c74ed590 2024-09-21T03:34:53.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.426+0000 7fc2bf7fe640 1 -- 172.21.15.38:0/2542547679 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 23) ==== 3502+0+0 (secure 0 0 0) 0x7fc2d007f030 con 0x55a7c74ed590 2024-09-21T03:34:53.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.426+0000 7fc2ee283740 10 client.5280 fetch_fsmap finished waiting for FSMap version 23 2024-09-21T03:34:53.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.426+0000 7fc2ee283740 10 client.5280 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T03:34:53.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.426+0000 7fc2ee283740 20 client.5280 populate_metadata read hostname 'smithi038' 2024-09-21T03:34:53.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.426+0000 7fc2ee283740 1 --2- 172.21.15.38:0/2542547679 >> [v2:172.21.15.38:6832/3533947700,v1:172.21.15.38:6834/3533947700] conn(0x55a7c791d000 0x55a7c797d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:53.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.426+0000 7fc2ee283740 4 client.5280 mds_command: new command op to 5226 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T03:34:53.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.426+0000 7fc2ee283740 1 -- 172.21.15.38:0/2542547679 --> [v2:172.21.15.38:6832/3533947700,v1:172.21.15.38:6834/3533947700] -- command(tid 0: {"prefix": "status"}) -- 0x55a7c7826220 con 0x55a7c791d000 2024-09-21T03:34:53.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.426+0000 7fc2e4fd4640 1 --2- 172.21.15.38:0/2542547679 >> [v2:172.21.15.38:6832/3533947700,v1:172.21.15.38:6834/3533947700] conn(0x55a7c791d000 0x55a7c797d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:53.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.426+0000 7fc2bf7fe640 10 client.5280 ms_handle_connect on v2:172.21.15.38:6832/3533947700 2024-09-21T03:34:53.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.426+0000 7fc2e4fd4640 1 --2- 172.21.15.38:0/2542547679 >> [v2:172.21.15.38:6832/3533947700,v1:172.21.15.38:6834/3533947700] conn(0x55a7c791d000 0x55a7c797d550 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3707860f41c76aba server_cookie=9d96d5233cad33f4 in_seq=0 out_seq=0 2024-09-21T03:34:53.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.427+0000 7fc2bf7fe640 1 -- 172.21.15.38:0/2542547679 <== mds.0 v2:172.21.15.38:6832/3533947700 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x55a7c7826220 con 0x55a7c791d000 2024-09-21T03:34:53.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.427+0000 7fc2bf7fe640 10 client.5280 handle_command_reply: tid=0 2024-09-21T03:34:53.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.427+0000 7fc2ee283740 1 client.5280 shutdown 2024-09-21T03:34:53.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.427+0000 7fc2ee283740 1 -- 172.21.15.38:0/2542547679 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a7c74ed590 msgr2=0x55a7c78b60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:53.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.427+0000 7fc2ee283740 1 --2- 172.21.15.38:0/2542547679 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a7c74ed590 0x55a7c78b60f0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fc2d0001f00 tx=0x7fc2d0084320 comp rx=0 tx=0).stop 2024-09-21T03:34:53.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.427+0000 7fc2ee283740 1 -- 172.21.15.38:0/2542547679 shutdown_connections 2024-09-21T03:34:53.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.427+0000 7fc2ee283740 1 --2- 172.21.15.38:0/2542547679 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a7c750e270 0x55a7c78b6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:53.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.427+0000 7fc2ee283740 1 --2- 172.21.15.38:0/2542547679 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a7c74ed590 0x55a7c78b60f0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:53.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.427+0000 7fc2ee283740 1 --2- 172.21.15.38:0/2542547679 >> [v2:172.21.15.38:6832/3533947700,v1:172.21.15.38:6834/3533947700] conn(0x55a7c791d000 0x55a7c797d550 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:53.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.427+0000 7fc2ee283740 1 --2- 172.21.15.38:0/2542547679 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a7c74e53b0 0x55a7c7970840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:53.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.427+0000 7fc2bf7fe640 0 client.5280 ms_handle_reset on v2:172.21.15.38:6832/3533947700 2024-09-21T03:34:53.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.427+0000 7fc2ee283740 1 -- 172.21.15.38:0/2542547679 >> 172.21.15.38:0/2542547679 conn(0x55a7c75023c0 msgr2=0x55a7c791db40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:53.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.427+0000 7fc2ee283740 1 -- 172.21.15.38:0/2542547679 shutdown_connections 2024-09-21T03:34:53.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.427+0000 7fc2ee283740 1 -- 172.21.15.38:0/2542547679 wait complete. 2024-09-21T03:34:53.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.427+0000 7fc2ee283740 20 client.5280 trim_cache size 0 max 16384 2024-09-21T03:34:53.435 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-09-21T03:34:53.435 INFO:teuthology.orchestra.run.smithi038.stdout: "cluster_fsid": "9d50d66f-656b-4fe2-8e3b-fe53b79c4d43", 2024-09-21T03:34:53.435 INFO:teuthology.orchestra.run.smithi038.stdout: "whoami": 0, 2024-09-21T03:34:53.435 INFO:teuthology.orchestra.run.smithi038.stdout: "id": 5226, 2024-09-21T03:34:53.435 INFO:teuthology.orchestra.run.smithi038.stdout: "want_state": "up:active", 2024-09-21T03:34:53.435 INFO:teuthology.orchestra.run.smithi038.stdout: "state": "up:active", 2024-09-21T03:34:53.435 INFO:teuthology.orchestra.run.smithi038.stdout: "fs_name": "cephfs", 2024-09-21T03:34:53.436 INFO:teuthology.orchestra.run.smithi038.stdout: "rank_uptime": 4.5208112920000003, 2024-09-21T03:34:53.436 INFO:teuthology.orchestra.run.smithi038.stdout: "mdsmap_epoch": 23, 2024-09-21T03:34:53.436 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch": 41, 2024-09-21T03:34:53.436 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch_barrier": 41, 2024-09-21T03:34:53.436 INFO:teuthology.orchestra.run.smithi038.stdout: "uptime": 5.4656802759999996 2024-09-21T03:34:53.436 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-09-21T03:34:53.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.428+0000 7fc2e7238640 1 -- 172.21.15.38:0/4223788724 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc2cc078b90 msgr2=0x7fc2cc07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:53.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.428+0000 7fc2e7238640 1 --2- 172.21.15.38:0/4223788724 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc2cc078b90 0x7fc2cc07b0a0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fc2dc0095f0 tx=0x7fc2dc005f90 comp rx=0 tx=0).stop 2024-09-21T03:34:53.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.428+0000 7fc2e7238640 1 -- 172.21.15.38:0/4223788724 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc2e00ceb90 msgr2=0x7fc2e006f4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:53.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.428+0000 7fc2e7238640 1 --2- 172.21.15.38:0/4223788724 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc2e00ceb90 0x7fc2e006f4f0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fc2d000c440 tx=0x7fc2d000c900 comp rx=0 tx=0).stop 2024-09-21T03:34:53.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.428+0000 7fc2e7238640 1 -- 172.21.15.38:0/4223788724 shutdown_connections 2024-09-21T03:34:53.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.428+0000 7fc2e7238640 1 --2- 172.21.15.38:0/4223788724 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc2e015ee80 0x7fc2e0148680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:53.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.428+0000 7fc2e7238640 1 --2- 172.21.15.38:0/4223788724 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc2e00ceb90 0x7fc2e006f4f0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:53.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.428+0000 7fc2e7238640 1 --2- 172.21.15.38:0/4223788724 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc2cc078b90 0x7fc2cc07b0a0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:53.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.428+0000 7fc2e7238640 1 --2- 172.21.15.38:0/4223788724 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc2e00b9ad0 0x7fc2e006efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:53.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.428+0000 7fc2e7238640 1 -- 172.21.15.38:0/4223788724 >> 172.21.15.38:0/4223788724 conn(0x7fc2e009d830 msgr2=0x7fc2e00b48d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:53.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.428+0000 7fc2e7238640 1 -- 172.21.15.38:0/4223788724 shutdown_connections 2024-09-21T03:34:53.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.428+0000 7fc2e7238640 1 -- 172.21.15.38:0/4223788724 wait complete. 2024-09-21T03:34:53.442 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T03:34:53.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.555+0000 7f44338c5640 1 -- 172.21.15.38:0/1786993708 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4418004800 msgr2=0x7f4418004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:53.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.555+0000 7f44338c5640 1 --2- 172.21.15.38:0/1786993708 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4418004800 0x7f4418004c10 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f442800c260 tx=0x7f4418005830 comp rx=0 tx=0).stop 2024-09-21T03:34:53.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.555+0000 7f4431e62640 1 -- 172.21.15.38:0/1786993708 reap_dead start 2024-09-21T03:34:53.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.555+0000 7f44338c5640 1 -- 172.21.15.38:0/1786993708 shutdown_connections 2024-09-21T03:34:53.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.555+0000 7f44338c5640 1 -- 172.21.15.38:0/1786993708 >> 172.21.15.38:0/1786993708 conn(0x7f442c0d4440 msgr2=0x7f442c0d4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:53.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.555+0000 7f44338c5640 1 -- 172.21.15.38:0/1786993708 shutdown_connections 2024-09-21T03:34:53.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.555+0000 7f44338c5640 1 -- 172.21.15.38:0/1786993708 wait complete. 2024-09-21T03:34:53.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.557+0000 7f44338c5640 1 Processor -- start 2024-09-21T03:34:53.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.557+0000 7f44338c5640 1 -- start start 2024-09-21T03:34:53.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.558+0000 7f44338c5640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4418004800 0x7f442c15b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:53.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.558+0000 7f44338c5640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f442c1f9590 0x7f442c1588c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:53.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.558+0000 7f44338c5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f442c156a10 0x7f442c1f89f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:53.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.558+0000 7f44338c5640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f442c0b3bf0 con 0x7f442c156a10 2024-09-21T03:34:53.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.558+0000 7f44338c5640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f442c0b3a70 con 0x7f4418004800 2024-09-21T03:34:53.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.558+0000 7f44338c5640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f442c0d2a20 con 0x7f442c1f9590 2024-09-21T03:34:53.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.558+0000 7f4431e62640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f442c156a10 0x7f442c1f89f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:53.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.558+0000 7f4431e62640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f442c156a10 0x7f442c1f89f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35482/0 (socket says 172.21.15.38:35482) 2024-09-21T03:34:53.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.558+0000 7f4431e62640 1 -- 172.21.15.38:0/198814120 learned_addr learned my addr 172.21.15.38:0/198814120 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:34:53.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.558+0000 7f4431661640 1 --2- 172.21.15.38:0/198814120 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4418004800 0x7f442c15b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:53.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.558+0000 7f4430e60640 1 --2- 172.21.15.38:0/198814120 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f442c1f9590 0x7f442c1588c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:53.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.559+0000 7f4431e62640 1 -- 172.21.15.38:0/198814120 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f442c1f9590 msgr2=0x7f442c1588c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:53.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.559+0000 7f4431e62640 1 --2- 172.21.15.38:0/198814120 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f442c1f9590 0x7f442c1588c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:53.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.559+0000 7f4431e62640 1 -- 172.21.15.38:0/198814120 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4418004800 msgr2=0x7f442c15b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:53.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.559+0000 7f4431e62640 1 --2- 172.21.15.38:0/198814120 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4418004800 0x7f442c15b5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:53.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.559+0000 7f4431e62640 1 -- 172.21.15.38:0/198814120 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f442c159540 con 0x7f442c156a10 2024-09-21T03:34:53.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.559+0000 7f4431e62640 1 --2- 172.21.15.38:0/198814120 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f442c156a10 0x7f442c1f89f0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f442c0cea40 tx=0x7f4424011f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:53.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.559+0000 7f441dffb640 1 -- 172.21.15.38:0/198814120 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4424004b20 con 0x7f442c156a10 2024-09-21T03:34:53.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.559+0000 7f441dffb640 1 -- 172.21.15.38:0/198814120 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4424004cc0 con 0x7f442c156a10 2024-09-21T03:34:53.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.559+0000 7f44338c5640 1 -- 172.21.15.38:0/198814120 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f442c159820 con 0x7f442c156a10 2024-09-21T03:34:53.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.559+0000 7f441dffb640 1 -- 172.21.15.38:0/198814120 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4424003bd0 con 0x7f442c156a10 2024-09-21T03:34:53.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.559+0000 7f44338c5640 1 -- 172.21.15.38:0/198814120 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f442c159de0 con 0x7f442c156a10 2024-09-21T03:34:53.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.560+0000 7f44338c5640 1 -- 172.21.15.38:0/198814120 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f442c0d2f40 con 0x7f442c156a10 2024-09-21T03:34:53.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.561+0000 7f441dffb640 1 -- 172.21.15.38:0/198814120 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f4424004030 con 0x7f442c156a10 2024-09-21T03:34:53.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.562+0000 7f441dffb640 1 --2- 172.21.15.38:0/198814120 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4410078b90 0x7f441007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:53.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.562+0000 7f441dffb640 1 -- 172.21.15.38:0/198814120 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6187+0+0 (secure 0 0 0) 0x7f442409bf60 con 0x7f442c156a10 2024-09-21T03:34:53.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.562+0000 7f4431661640 1 --2- 172.21.15.38:0/198814120 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4410078b90 0x7f441007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:53.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.563+0000 7f4431661640 1 --2- 172.21.15.38:0/198814120 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4410078b90 0x7f441007b0a0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f4420009b30 tx=0x7f4420005f90 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:53.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.566+0000 7f441dffb640 1 -- 172.21.15.38:0/198814120 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f442401f060 con 0x7f442c156a10 2024-09-21T03:34:53.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.796+0000 7f44338c5640 1 -- 172.21.15.38:0/198814120 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f442c15c630 con 0x7f442c156a10 2024-09-21T03:34:53.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.796+0000 7f441dffb640 1 -- 172.21.15.38:0/198814120 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v23) ==== 53+0+73 (secure 0 0 0) 0x7f44240641b0 con 0x7f442c156a10 2024-09-21T03:34:53.799 INFO:teuthology.orchestra.run.smithi038.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T03:34:53.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.798+0000 7f44338c5640 1 -- 172.21.15.38:0/198814120 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4410078b90 msgr2=0x7f441007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:53.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.798+0000 7f44338c5640 1 --2- 172.21.15.38:0/198814120 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4410078b90 0x7f441007b0a0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f4420009b30 tx=0x7f4420005f90 comp rx=0 tx=0).stop 2024-09-21T03:34:53.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.798+0000 7f44338c5640 1 -- 172.21.15.38:0/198814120 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f442c156a10 msgr2=0x7f442c1f89f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:53.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.798+0000 7f44338c5640 1 --2- 172.21.15.38:0/198814120 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f442c156a10 0x7f442c1f89f0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f442c0cea40 tx=0x7f4424011f80 comp rx=0 tx=0).stop 2024-09-21T03:34:53.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.798+0000 7f44338c5640 1 -- 172.21.15.38:0/198814120 shutdown_connections 2024-09-21T03:34:53.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.798+0000 7f44338c5640 1 --2- 172.21.15.38:0/198814120 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f442c156a10 0x7f442c1f89f0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:53.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.798+0000 7f44338c5640 1 --2- 172.21.15.38:0/198814120 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4410078b90 0x7f441007b0a0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:53.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.798+0000 7f44338c5640 1 --2- 172.21.15.38:0/198814120 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f442c1f9590 0x7f442c1588c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:53.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.798+0000 7f44338c5640 1 --2- 172.21.15.38:0/198814120 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4418004800 0x7f442c15b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:53.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.798+0000 7f44338c5640 1 -- 172.21.15.38:0/198814120 >> 172.21.15.38:0/198814120 conn(0x7f442c0d4440 msgr2=0x7f442c0d4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:53.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.798+0000 7f44338c5640 1 -- 172.21.15.38:0/198814120 shutdown_connections 2024-09-21T03:34:53.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:53.798+0000 7f44338c5640 1 -- 172.21.15.38:0/198814120 wait complete. 2024-09-21T03:34:53.812 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-21T03:34:53.812 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T03:34:53.813 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi038.front.sepia.ceph.com 2024-09-21T03:34:53.813 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-21T03:34:53.813 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T03:34:53.813 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-21T03:34:53.813 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-21T03:34:53.813 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T03:34:53.813 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T03:34:53.813 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:34:53.814 DEBUG:teuthology.orchestra.run.smithi038:> ip addr 2024-09-21T03:34:53.830 INFO:teuthology.orchestra.run.smithi038.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T03:34:53.830 INFO:teuthology.orchestra.run.smithi038.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T03:34:53.830 INFO:teuthology.orchestra.run.smithi038.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T03:34:53.830 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:34:53.830 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 ::1/128 scope host 2024-09-21T03:34:53.830 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:34:53.830 INFO:teuthology.orchestra.run.smithi038.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T03:34:53.830 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:6b:fd:a2 brd ff:ff:ff:ff:ff:ff 2024-09-21T03:34:53.830 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp7s0f0 2024-09-21T03:34:53.831 INFO:teuthology.orchestra.run.smithi038.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T03:34:53.831 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:6b:fd:a3 brd ff:ff:ff:ff:ff:ff 2024-09-21T03:34:53.831 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp7s0f1 2024-09-21T03:34:53.831 INFO:teuthology.orchestra.run.smithi038.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T03:34:53.831 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:bd:15:fc brd ff:ff:ff:ff:ff:ff 2024-09-21T03:34:53.831 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp1s0f0 2024-09-21T03:34:53.831 INFO:teuthology.orchestra.run.smithi038.stdout: inet 172.21.15.38/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-21T03:34:53.831 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft 42715sec preferred_lft 42715sec 2024-09-21T03:34:53.831 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 fe80::ec4:7aff:febd:15fc/64 scope link 2024-09-21T03:34:53.831 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:34:53.831 INFO:teuthology.orchestra.run.smithi038.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T03:34:53.831 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:bd:15:fd brd ff:ff:ff:ff:ff:ff 2024-09-21T03:34:53.831 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp1s0f1 2024-09-21T03:34:53.831 INFO:teuthology.orchestra.run.smithi038.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T03:34:53.832 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 92:55:c7:97:6f:b5 brd ff:ff:ff:ff:ff:ff 2024-09-21T03:34:53.832 INFO:teuthology.orchestra.run.smithi038.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-21T03:34:53.832 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:34:53.832 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 fe80::c49f:c1ff:fe21:b12a/64 scope link 2024-09-21T03:34:53.832 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:34:53.832 INFO:teuthology.orchestra.run.smithi038.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T03:34:53.832 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether a6:3a:83:ad:db:d5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T03:34:53.832 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 fe80::a43a:83ff:fead:dbd5/64 scope link 2024-09-21T03:34:53.832 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:34:53.832 INFO:teuthology.orchestra.run.smithi038.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T03:34:53.832 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 92:55:c7:97:6f:b5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T03:34:53.832 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 fe80::9055:c7ff:fe97:6fb5/64 scope link 2024-09-21T03:34:53.832 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:34:53.833 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:34:53.833 DEBUG:teuthology.orchestra.run.smithi038:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T03:34:53.895 INFO:teuthology.orchestra.run.smithi038.stdout:1 2024-09-21T03:34:53.897 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:34:53.897 DEBUG:teuthology.orchestra.run.smithi038:> route 2024-09-21T03:34:53.910 INFO:tasks.ceph.mds.c.smithi038.stderr:2024-09-21T03:34:53.908+0000 7feb38379640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T03:34:53.926 INFO:teuthology.orchestra.run.smithi038.stdout:Kernel IP routing table 2024-09-21T03:34:53.927 INFO:teuthology.orchestra.run.smithi038.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T03:34:53.927 INFO:teuthology.orchestra.run.smithi038.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-21T03:34:53.927 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-21T03:34:53.927 INFO:teuthology.orchestra.run.smithi038.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T03:34:53.927 DEBUG:teuthology.orchestra.run.smithi038:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T03:34:53.927 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-09-21T03:34:53.927 DEBUG:teuthology.orchestra.run.smithi038:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-21T03:34:53.927 DEBUG:teuthology.orchestra.run.smithi038:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-21T03:34:53.928 DEBUG:teuthology.orchestra.run.smithi038:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-21T03:34:53.928 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-09-21T03:34:54.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:54 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T03:34:54.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:34:54.056 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:34:54.056 DEBUG:teuthology.orchestra.run.smithi038:> ip netns list 2024-09-21T03:34:54.072 INFO:teuthology.orchestra.run.smithi038.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-21T03:34:54.072 INFO:teuthology.orchestra.run.smithi038.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-21T03:34:54.072 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T03:34:54.072 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:34:54.072 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip link set brx.0 up 2024-09-21T03:34:54.138 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T03:34:54.139 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-21T03:34:54.194 INFO:teuthology.orchestra.run.smithi038.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-21T03:34:54.195 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T03:34:54.195 DEBUG:teuthology.orchestra.run.smithi038:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-21T03:34:54.250 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-21T03:34:54.250 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:34:54.250 DEBUG:teuthology.orchestra.run.smithi038:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T03:34:54.346 INFO:teuthology.orchestra.run.smithi038.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T03:34:54.346 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T03:34:54.346 INFO:teuthology.orchestra.run.smithi038.stdout:invalid new device string format 2024-09-21T03:34:54.357 INFO:teuthology.orchestra.run.smithi038.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T03:34:54.357 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T03:34:54.357 INFO:teuthology.orchestra.run.smithi038.stdout:invalid new device string format 2024-09-21T03:34:54.357 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: resolved to: "172.21.15.38:3300,172.21.15.195:3300,172.21.15.195:3301" 2024-09-21T03:34:54.358 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T03:34:54.364 INFO:teuthology.orchestra.run.smithi038.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T03:34:54.364 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T03:34:54.364 INFO:teuthology.orchestra.run.smithi038.stdout:invalid new device string format 2024-09-21T03:34:54.364 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: resolved to: "172.21.15.38:3300,172.21.15.195:3300,172.21.15.195:3301" 2024-09-21T03:34:54.364 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T03:34:54.364 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: resolved to: "172.21.15.38:6789,172.21.15.195:6789,172.21.15.195:6790" 2024-09-21T03:34:54.364 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.38:6789,172.21.15.195:6789,172.21.15.195:6790:/ 2024-09-21T03:34:54.364 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=9d50d66f-656b-4fe2-8e3b-fe53b79c4d43" will pass to kernel 2024-09-21T03:34:54.365 INFO:teuthology.orchestra.run.smithi038.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-21T03:34:54.365 INFO:teuthology.orchestra.run.smithi038.stdout: You just mounted a file system that supports labels which does not 2024-09-21T03:34:54.365 INFO:teuthology.orchestra.run.smithi038.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-21T03:34:54.365 INFO:teuthology.orchestra.run.smithi038.stdout: applications will generate AVC messages and not be allowed access to 2024-09-21T03:34:54.365 INFO:teuthology.orchestra.run.smithi038.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-21T03:34:54.366 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T03:34:54.367 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:34:54.367 DEBUG:teuthology.orchestra.run.smithi038:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-21T03:34:54.392 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T03:34:54.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:54 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T03:34:54.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:34:54.489 INFO:teuthology.orchestra.run.smithi038.stdout:client4964 2024-09-21T03:34:54.490 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-21T03:34:54.490 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:34:54.506 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:34:54.506 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls subdir/) 2024-09-21T03:34:54.563 INFO:teuthology.orchestra.run.smithi038.stdout:file_undamaged 2024-09-21T03:34:54.564 DEBUG:teuthology.orchestra.run.smithi038:> stdin-killer -- python3 -c ' 2024-09-21T03:34:54.564 DEBUG:teuthology.orchestra.run.smithi038:> import os 2024-09-21T03:34:54.564 DEBUG:teuthology.orchestra.run.smithi038:> import stat 2024-09-21T03:34:54.564 DEBUG:teuthology.orchestra.run.smithi038:> import json 2024-09-21T03:34:54.564 DEBUG:teuthology.orchestra.run.smithi038:> import sys 2024-09-21T03:34:54.564 DEBUG:teuthology.orchestra.run.smithi038:> 2024-09-21T03:34:54.564 DEBUG:teuthology.orchestra.run.smithi038:> try: 2024-09-21T03:34:54.565 DEBUG:teuthology.orchestra.run.smithi038:> s = os.stat('"'"'/home/ubuntu/cephtest/mnt.0/subdir/file_to_be_damaged'"'"') 2024-09-21T03:34:54.565 DEBUG:teuthology.orchestra.run.smithi038:> except OSError as e: 2024-09-21T03:34:54.565 DEBUG:teuthology.orchestra.run.smithi038:> sys.exit(e.errno) 2024-09-21T03:34:54.565 DEBUG:teuthology.orchestra.run.smithi038:> 2024-09-21T03:34:54.565 DEBUG:teuthology.orchestra.run.smithi038:> attrs = ["st_mode", "st_ino", "st_dev", "st_nlink", "st_uid", "st_gid", "st_size", "st_atime", "st_mtime", "st_ctime"] 2024-09-21T03:34:54.565 DEBUG:teuthology.orchestra.run.smithi038:> print(json.dumps( 2024-09-21T03:34:54.565 DEBUG:teuthology.orchestra.run.smithi038:> dict([(a, getattr(s, a)) for a in attrs]), 2024-09-21T03:34:54.565 DEBUG:teuthology.orchestra.run.smithi038:> indent=2)) 2024-09-21T03:34:54.565 DEBUG:teuthology.orchestra.run.smithi038:> ' 2024-09-21T03:34:54.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:54 stdin-killer INFO: command exited with status 2: exiting normally with same code! 2024-09-21T03:34:54.681 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-21T03:34:54.682 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:34:54.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:54.837+0000 7eff24d8a640 1 -- 172.21.15.38:0/566272782 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7eff0c004d40 msgr2=0x7eff0c005170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:54.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:54.837+0000 7eff24d8a640 1 --2- 172.21.15.38:0/566272782 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7eff0c004d40 0x7eff0c005170 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7eff1000c560 tx=0x7eff0c005830 comp rx=0 tx=0).stop 2024-09-21T03:34:54.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:54.837+0000 7eff1ed9d640 1 -- 172.21.15.38:0/566272782 reap_dead start 2024-09-21T03:34:54.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:54.837+0000 7eff24d8a640 1 -- 172.21.15.38:0/566272782 shutdown_connections 2024-09-21T03:34:54.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:54.837+0000 7eff24d8a640 1 -- 172.21.15.38:0/566272782 >> 172.21.15.38:0/566272782 conn(0x7eff200cef40 msgr2=0x7eff200cf350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:54.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:54.837+0000 7eff24d8a640 1 -- 172.21.15.38:0/566272782 shutdown_connections 2024-09-21T03:34:54.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:54.837+0000 7eff24d8a640 1 -- 172.21.15.38:0/566272782 wait complete. 2024-09-21T03:34:54.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:54.839+0000 7eff24d8a640 1 Processor -- start 2024-09-21T03:34:54.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:54.839+0000 7eff24d8a640 1 -- start start 2024-09-21T03:34:54.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:54.840+0000 7eff24d8a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7eff0c004d40 0x7eff20142360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:54.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:54.840+0000 7eff24d8a640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7eff200cc240 0x7eff201f6480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:54.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:54.840+0000 7eff24d8a640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7eff201f69c0 0x7eff201e2da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:54.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:54.840+0000 7eff24d8a640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7eff200d1b50 con 0x7eff0c004d40 2024-09-21T03:34:54.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:54.841+0000 7eff24d8a640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7eff200d19d0 con 0x7eff201f69c0 2024-09-21T03:34:54.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:54.841+0000 7eff24d8a640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7eff200d0bd0 con 0x7eff200cc240 2024-09-21T03:34:54.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:54.841+0000 7eff1e59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7eff0c004d40 0x7eff20142360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:54.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:54.841+0000 7eff1e59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7eff0c004d40 0x7eff20142360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35500/0 (socket says 172.21.15.38:35500) 2024-09-21T03:34:54.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:54.841+0000 7eff1e59c640 1 -- 172.21.15.38:0/199586943 learned_addr learned my addr 172.21.15.38:0/199586943 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:34:54.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:54.841+0000 7eff1ed9d640 1 --2- 172.21.15.38:0/199586943 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7eff201f69c0 0x7eff201e2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:54.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:54.841+0000 7eff1dd9b640 1 --2- 172.21.15.38:0/199586943 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7eff200cc240 0x7eff201f6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:54.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:54.841+0000 7eff1e59c640 1 -- 172.21.15.38:0/199586943 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7eff200cc240 msgr2=0x7eff201f6480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:54.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:54.841+0000 7eff1e59c640 1 --2- 172.21.15.38:0/199586943 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7eff200cc240 0x7eff201f6480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:54.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:54.841+0000 7eff1e59c640 1 -- 172.21.15.38:0/199586943 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7eff201f69c0 msgr2=0x7eff201e2da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:54.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:54.841+0000 7eff1e59c640 1 --2- 172.21.15.38:0/199586943 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7eff201f69c0 0x7eff201e2da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:54.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:54.841+0000 7eff1e59c640 1 -- 172.21.15.38:0/199586943 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff201f6dd0 con 0x7eff0c004d40 2024-09-21T03:34:54.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:54.841+0000 7eff1dd9b640 1 --2- 172.21.15.38:0/199586943 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7eff200cc240 0x7eff201f6480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:34:54.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:54.841+0000 7eff1ed9d640 1 --2- 172.21.15.38:0/199586943 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7eff201f69c0 0x7eff201e2da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:34:54.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:54.841+0000 7eff1e59c640 1 --2- 172.21.15.38:0/199586943 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7eff0c004d40 0x7eff20142360 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7eff18011ae0 tx=0x7eff18009e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:54.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:54.841+0000 7eff16ffd640 1 -- 172.21.15.38:0/199586943 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff1801e070 con 0x7eff0c004d40 2024-09-21T03:34:54.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:54.841+0000 7eff16ffd640 1 -- 172.21.15.38:0/199586943 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7eff1800fe70 con 0x7eff0c004d40 2024-09-21T03:34:54.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:54.841+0000 7eff24d8a640 1 -- 172.21.15.38:0/199586943 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff201428a0 con 0x7eff0c004d40 2024-09-21T03:34:54.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:54.841+0000 7eff16ffd640 1 -- 172.21.15.38:0/199586943 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff18004290 con 0x7eff0c004d40 2024-09-21T03:34:54.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:54.841+0000 7eff24d8a640 1 -- 172.21.15.38:0/199586943 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff20143090 con 0x7eff0c004d40 2024-09-21T03:34:54.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:54.843+0000 7eff24d8a640 1 -- 172.21.15.38:0/199586943 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff200d10d0 con 0x7eff0c004d40 2024-09-21T03:34:54.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:54.843+0000 7eff16ffd640 1 -- 172.21.15.38:0/199586943 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7eff18021070 con 0x7eff0c004d40 2024-09-21T03:34:54.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:54.844+0000 7eff16ffd640 1 --2- 172.21.15.38:0/199586943 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7eff00078bc0 0x7eff0007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:54.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:54.844+0000 7eff16ffd640 1 -- 172.21.15.38:0/199586943 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6187+0+0 (secure 0 0 0) 0x7eff1809b970 con 0x7eff0c004d40 2024-09-21T03:34:54.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:54.844+0000 7eff1dd9b640 1 --2- 172.21.15.38:0/199586943 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7eff00078bc0 0x7eff0007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:54.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:54.845+0000 7eff1dd9b640 1 --2- 172.21.15.38:0/199586943 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7eff00078bc0 0x7eff0007b0d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7eff08006850 tx=0x7eff080067e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:54.850 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:54.848+0000 7eff16ffd640 1 -- 172.21.15.38:0/199586943 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7eff201f6dd0 con 0x7eff0c004d40 2024-09-21T03:34:55.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.074+0000 7eff24d8a640 1 -- 172.21.15.38:0/199586943 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7eff20071250 con 0x7eff0c004d40 2024-09-21T03:34:55.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.074+0000 7eff16ffd640 1 -- 172.21.15.38:0/199586943 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+5230 (secure 0 0 0) 0x7eff18063650 con 0x7eff0c004d40 2024-09-21T03:34:55.076 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:34:55.076 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":23,"btime":"2024-09-21T03:34:51:918500+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4757,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/1905955399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":1905955399},{"type":"v1","addr":"172.21.15.195:6836","nonce":1905955399}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4936,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/2989374027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":2989374027},{"type":"v1","addr":"172.21.15.195:6837","nonce":2989374027}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4942,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/3531731501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6833","nonce":3531731501},{"type":"v1","addr":"172.21.15.38:6835","nonce":3531731501}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:34:34.009269+0000","modified":"2024-09-21T03:34:51.918491+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":41,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5226},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5226":{"gid":5226,"name":"c","rank":0,"incarnation":20,"state":"up:active","state_seq":4,"addr":"172.21.15.38:6834/3533947700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":3533947700},{"type":"v1","addr":"172.21.15.38:6834","nonce":3533947700}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5226,"qdb_cluster":[5226]},"id":2}]} 2024-09-21T03:34:55.076 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 23 2024-09-21T03:34:55.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.076+0000 7eff24d8a640 1 -- 172.21.15.38:0/199586943 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7eff00078bc0 msgr2=0x7eff0007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:55.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.076+0000 7eff24d8a640 1 --2- 172.21.15.38:0/199586943 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7eff00078bc0 0x7eff0007b0d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7eff08006850 tx=0x7eff080067e0 comp rx=0 tx=0).stop 2024-09-21T03:34:55.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.076+0000 7eff24d8a640 1 -- 172.21.15.38:0/199586943 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7eff0c004d40 msgr2=0x7eff20142360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:55.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.076+0000 7eff24d8a640 1 --2- 172.21.15.38:0/199586943 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7eff0c004d40 0x7eff20142360 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7eff18011ae0 tx=0x7eff18009e90 comp rx=0 tx=0).stop 2024-09-21T03:34:55.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.076+0000 7eff24d8a640 1 -- 172.21.15.38:0/199586943 shutdown_connections 2024-09-21T03:34:55.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.076+0000 7eff24d8a640 1 --2- 172.21.15.38:0/199586943 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7eff201f69c0 0x7eff201e2da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:55.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.076+0000 7eff24d8a640 1 --2- 172.21.15.38:0/199586943 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7eff00078bc0 0x7eff0007b0d0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:55.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.076+0000 7eff24d8a640 1 --2- 172.21.15.38:0/199586943 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7eff200cc240 0x7eff201f6480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:55.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.076+0000 7eff24d8a640 1 --2- 172.21.15.38:0/199586943 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7eff0c004d40 0x7eff20142360 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:55.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.076+0000 7eff24d8a640 1 -- 172.21.15.38:0/199586943 >> 172.21.15.38:0/199586943 conn(0x7eff200cef40 msgr2=0x7eff20071770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:55.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.076+0000 7eff24d8a640 1 -- 172.21.15.38:0/199586943 shutdown_connections 2024-09-21T03:34:55.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.076+0000 7eff24d8a640 1 -- 172.21.15.38:0/199586943 wait complete. 2024-09-21T03:34:55.089 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c damage ls --format=json-pretty 2024-09-21T03:34:55.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.206+0000 7f24666f4640 1 -- 172.21.15.38:0/1747390670 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f24600ceb90 msgr2=0x7f24600cefa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:55.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.206+0000 7f24666f4640 1 --2- 172.21.15.38:0/1747390670 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f24600ceb90 0x7f24600cefa0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f2458005fd0 tx=0x7f245802a780 comp rx=0 tx=0).stop 2024-09-21T03:34:55.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.206+0000 7f24666f4640 1 -- 172.21.15.38:0/1747390670 shutdown_connections 2024-09-21T03:34:55.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.206+0000 7f24666f4640 1 --2- 172.21.15.38:0/1747390670 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f24600ceb90 0x7f24600cefa0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:55.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.206+0000 7f24666f4640 1 -- 172.21.15.38:0/1747390670 >> 172.21.15.38:0/1747390670 conn(0x7f246009d830 msgr2=0x7f24600b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:55.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.206+0000 7f24666f4640 1 -- 172.21.15.38:0/1747390670 shutdown_connections 2024-09-21T03:34:55.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.206+0000 7f24666f4640 1 -- 172.21.15.38:0/1747390670 wait complete. 2024-09-21T03:34:55.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.208+0000 7f24666f4640 1 Processor -- start 2024-09-21T03:34:55.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.208+0000 7f24666f4640 1 -- start start 2024-09-21T03:34:55.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.210+0000 7f24666f4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f24600ceb90 0x7f24601575a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:55.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.210+0000 7f24666f4640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f24600d0170 0x7f24600cb610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:55.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.210+0000 7f24666f4640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f246015ee80 0x7f24600d3c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:55.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.210+0000 7f24666f4640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f24600b6910 con 0x7f24600ceb90 2024-09-21T03:34:55.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.210+0000 7f24666f4640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f24600b6790 con 0x7f246015ee80 2024-09-21T03:34:55.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.210+0000 7f24666f4640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f24600b6a90 con 0x7f24600d0170 2024-09-21T03:34:55.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.210+0000 7f245ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f24600ceb90 0x7f24601575a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:55.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.210+0000 7f245ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f24600ceb90 0x7f24601575a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35518/0 (socket says 172.21.15.38:35518) 2024-09-21T03:34:55.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.210+0000 7f245ffff640 1 -- 172.21.15.38:0/1443781291 learned_addr learned my addr 172.21.15.38:0/1443781291 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:34:55.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.210+0000 7f245f7fe640 1 --2- 172.21.15.38:0/1443781291 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f24600d0170 0x7f24600cb610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:55.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.210+0000 7f2464c91640 1 --2- 172.21.15.38:0/1443781291 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f246015ee80 0x7f24600d3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:55.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.210+0000 7f245ffff640 1 -- 172.21.15.38:0/1443781291 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f24600d0170 msgr2=0x7f24600cb610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:55.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.210+0000 7f245ffff640 1 --2- 172.21.15.38:0/1443781291 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f24600d0170 0x7f24600cb610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:55.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.210+0000 7f245ffff640 1 -- 172.21.15.38:0/1443781291 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f246015ee80 msgr2=0x7f24600d3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:55.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.210+0000 7f245ffff640 1 --2- 172.21.15.38:0/1443781291 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f246015ee80 0x7f24600d3c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:55.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.210+0000 7f245ffff640 1 -- 172.21.15.38:0/1443781291 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24600cc0d0 con 0x7f24600ceb90 2024-09-21T03:34:55.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.210+0000 7f245ffff640 1 --2- 172.21.15.38:0/1443781291 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f24600ceb90 0x7f24601575a0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f245000b3f0 tx=0x7f245000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:55.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.210+0000 7f245cff9640 1 -- 172.21.15.38:0/1443781291 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2450015020 con 0x7f24600ceb90 2024-09-21T03:34:55.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.210+0000 7f245cff9640 1 -- 172.21.15.38:0/1443781291 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2450011920 con 0x7f24600ceb90 2024-09-21T03:34:55.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.210+0000 7f245cff9640 1 -- 172.21.15.38:0/1443781291 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2450011d70 con 0x7f24600ceb90 2024-09-21T03:34:55.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.210+0000 7f24666f4640 1 -- 172.21.15.38:0/1443781291 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24600cc3b0 con 0x7f24600ceb90 2024-09-21T03:34:55.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.211+0000 7f24666f4640 1 -- 172.21.15.38:0/1443781291 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f246006d340 con 0x7f24600ceb90 2024-09-21T03:34:55.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.212+0000 7f245cff9640 1 -- 172.21.15.38:0/1443781291 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f2450018020 con 0x7f24600ceb90 2024-09-21T03:34:55.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.213+0000 7f245cff9640 1 --2- 172.21.15.38:0/1443781291 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f244c078bc0 0x7f244c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:55.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.213+0000 7f245cff9640 1 -- 172.21.15.38:0/1443781291 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6187+0+0 (secure 0 0 0) 0x7f245009aaa0 con 0x7f24600ceb90 2024-09-21T03:34:55.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.213+0000 7f245f7fe640 1 --2- 172.21.15.38:0/1443781291 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f244c078bc0 0x7f244c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:55.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.214+0000 7f246d65d740 1 Processor -- start 2024-09-21T03:34:55.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.214+0000 7f246d65d740 1 -- start start 2024-09-21T03:34:55.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.214+0000 7f245f7fe640 1 --2- 172.21.15.38:0/1443781291 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f244c078bc0 0x7f244c07b0d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f24600ce780 tx=0x7f2454009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:55.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.215+0000 7f246d65d740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e56197d350 0x55e561d4ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:55.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.215+0000 7f246d65d740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e561954490 0x55e561d4f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:55.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.215+0000 7f246d65d740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e56195c670 0x55e561d6c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:55.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.215+0000 7f246d65d740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55e561c979a0 con 0x55e56195c670 2024-09-21T03:34:55.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.215+0000 7f246d65d740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55e561c86dd0 con 0x55e56197d350 2024-09-21T03:34:55.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.215+0000 7f246d65d740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55e561c8abe0 con 0x55e561954490 2024-09-21T03:34:55.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.215+0000 7f2464c91640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e56195c670 0x55e561d6c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:55.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.215+0000 7f2464c91640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e56195c670 0x55e561d6c1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35530/0 (socket says 172.21.15.38:35530) 2024-09-21T03:34:55.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.215+0000 7f2464c91640 1 -- 172.21.15.38:0/409346530 learned_addr learned my addr 172.21.15.38:0/409346530 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:34:55.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.215+0000 7f245ffff640 1 --2- 172.21.15.38:0/409346530 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e56197d350 0x55e561d4ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:55.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.215+0000 7f245f7fe640 1 --2- 172.21.15.38:0/409346530 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e561954490 0x55e561d4f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:55.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.216+0000 7f2464c91640 1 -- 172.21.15.38:0/409346530 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e561954490 msgr2=0x55e561d4f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:55.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.216+0000 7f2464c91640 1 --2- 172.21.15.38:0/409346530 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e561954490 0x55e561d4f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:55.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.216+0000 7f2464c91640 1 -- 172.21.15.38:0/409346530 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e56197d350 msgr2=0x55e561d4ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:55.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.216+0000 7f2464c91640 1 --2- 172.21.15.38:0/409346530 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e56197d350 0x55e561d4ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:55.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.216+0000 7f2464c91640 1 -- 172.21.15.38:0/409346530 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e561c1f700 con 0x55e56195c670 2024-09-21T03:34:55.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.216+0000 7f2464c91640 1 --2- 172.21.15.38:0/409346530 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e56195c670 0x55e561d6c1c0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f2458034040 tx=0x7f2458004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=849c14ccb3c4fc2e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:55.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.216+0000 7f2445ffb640 1 -- 172.21.15.38:0/409346530 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2458041070 con 0x55e56195c670 2024-09-21T03:34:55.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.216+0000 7f2445ffb640 1 -- 172.21.15.38:0/409346530 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2458003c60 con 0x55e56195c670 2024-09-21T03:34:55.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.216+0000 7f2445ffb640 1 -- 172.21.15.38:0/409346530 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2458032b20 con 0x55e56195c670 2024-09-21T03:34:55.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.217+0000 7f246d65d740 1 -- 172.21.15.38:0/409346530 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e56195c670 msgr2=0x55e561d6c1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:55.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.217+0000 7f246d65d740 1 --2- 172.21.15.38:0/409346530 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e56195c670 0x55e561d6c1c0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f2458034040 tx=0x7f2458004a40 comp rx=0 tx=0).stop 2024-09-21T03:34:55.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.217+0000 7f246d65d740 1 -- 172.21.15.38:0/409346530 shutdown_connections 2024-09-21T03:34:55.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.217+0000 7f246d65d740 1 --2- 172.21.15.38:0/409346530 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e56195c670 0x55e561d6c1c0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:55.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.217+0000 7f246d65d740 1 --2- 172.21.15.38:0/409346530 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e561954490 0x55e561d4f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:55.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.217+0000 7f246d65d740 1 --2- 172.21.15.38:0/409346530 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e56197d350 0x55e561d4ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:55.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.217+0000 7f246d65d740 1 -- 172.21.15.38:0/409346530 >> 172.21.15.38:0/409346530 conn(0x55e5619714a0 msgr2=0x55e561d25040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:55.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.217+0000 7f246d65d740 1 -- 172.21.15.38:0/409346530 shutdown_connections 2024-09-21T03:34:55.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.217+0000 7f246d65d740 1 -- 172.21.15.38:0/409346530 wait complete. 2024-09-21T03:34:55.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.218+0000 7f246d65d740 1 -- start start 2024-09-21T03:34:55.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.218+0000 7f246d65d740 1 Processor -- start 2024-09-21T03:34:55.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.219+0000 7f246d65d740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e561954490 0x55e561d611a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:55.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.219+0000 7f246d65d740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e56195c670 0x55e561d38640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:55.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.219+0000 7f246d65d740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e56197d350 0x55e561d630c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:55.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.219+0000 7f246d65d740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55e5619f7700 con 0x55e561954490 2024-09-21T03:34:55.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.219+0000 7f246d65d740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55e561c2c340 con 0x55e56197d350 2024-09-21T03:34:55.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.219+0000 7f246d65d740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55e561d4f800 con 0x55e56195c670 2024-09-21T03:34:55.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.219+0000 7f246d65d740 10 client.0 _ll_register_callbacks cb 0x55e561c2aee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:34:55.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.219+0000 7f245ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e561954490 0x55e561d611a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:55.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.219+0000 7f245ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e561954490 0x55e561d611a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35542/0 (socket says 172.21.15.38:35542) 2024-09-21T03:34:55.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.219+0000 7f245ffff640 1 -- 172.21.15.38:0/2020458040 learned_addr learned my addr 172.21.15.38:0/2020458040 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:34:55.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.220+0000 7f245f7fe640 1 --2- 172.21.15.38:0/2020458040 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e56195c670 0x55e561d38640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:55.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.220+0000 7f2464c91640 1 --2- 172.21.15.38:0/2020458040 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e56197d350 0x55e561d630c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:55.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.220+0000 7f245ffff640 1 -- 172.21.15.38:0/2020458040 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e56195c670 msgr2=0x55e561d38640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:55.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.220+0000 7f245ffff640 1 --2- 172.21.15.38:0/2020458040 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e56195c670 0x55e561d38640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:55.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.220+0000 7f245ffff640 1 -- 172.21.15.38:0/2020458040 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e56197d350 msgr2=0x55e561d630c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:55.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.220+0000 7f245ffff640 1 --2- 172.21.15.38:0/2020458040 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e56197d350 0x55e561d630c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:55.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.220+0000 7f245ffff640 1 -- 172.21.15.38:0/2020458040 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e561a617c0 con 0x55e561954490 2024-09-21T03:34:55.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.220+0000 7f2464c91640 1 --2- 172.21.15.38:0/2020458040 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e56197d350 0x55e561d630c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:34:55.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.220+0000 7f245f7fe640 1 --2- 172.21.15.38:0/2020458040 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e56195c670 0x55e561d38640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:34:55.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.220+0000 7f245ffff640 1 --2- 172.21.15.38:0/2020458040 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e561954490 0x55e561d611a0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f245000bdb0 tx=0x7f24500622f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f18c7f87b7e2fd92 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:55.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.220+0000 7f243effd640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-21T03:34:55.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.220+0000 7f243effd640 1 -- 172.21.15.38:0/2020458040 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f245006c020 con 0x55e561954490 2024-09-21T03:34:55.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.220+0000 7f243effd640 1 -- 172.21.15.38:0/2020458040 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(41..41 src has 1..41) ==== 6187+0+0 (secure 0 0 0) 0x7f2450061b20 con 0x55e561954490 2024-09-21T03:34:55.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.220+0000 7f246d65d740 1 -- 172.21.15.38:0/2020458040 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2458003c60 con 0x55e561954490 2024-09-21T03:34:55.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.220+0000 7f243effd640 1 -- 172.21.15.38:0/2020458040 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f2450083050 con 0x55e561954490 2024-09-21T03:34:55.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.220+0000 7f246d65d740 10 client.5307 fetch_fsmap learned FSMap version 23 2024-09-21T03:34:55.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.220+0000 7f246d65d740 1 -- 172.21.15.38:0/2020458040 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) -- 0x55e561ac1f50 con 0x55e561954490 2024-09-21T03:34:55.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.220+0000 7f243effd640 1 -- 172.21.15.38:0/2020458040 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2458003c60 con 0x55e561954490 2024-09-21T03:34:55.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.220+0000 7f243effd640 1 -- 172.21.15.38:0/2020458040 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 23) ==== 3502+0+0 (secure 0 0 0) 0x7f245007cc40 con 0x55e561954490 2024-09-21T03:34:55.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.221+0000 7f246d65d740 10 client.5307 fetch_fsmap finished waiting for FSMap version 23 2024-09-21T03:34:55.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.221+0000 7f246d65d740 10 client.5307 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T03:34:55.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.221+0000 7f246d65d740 20 client.5307 populate_metadata read hostname 'smithi038' 2024-09-21T03:34:55.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.221+0000 7f246d65d740 1 --2- 172.21.15.38:0/2020458040 >> [v2:172.21.15.38:6832/3533947700,v1:172.21.15.38:6834/3533947700] conn(0x55e56193c130 0x55e561d3b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:55.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.221+0000 7f246d65d740 4 client.5307 mds_command: new command op to 5226 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:34:55.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.221+0000 7f246d65d740 1 -- 172.21.15.38:0/2020458040 --> [v2:172.21.15.38:6832/3533947700,v1:172.21.15.38:6834/3533947700] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e561c998f0 con 0x55e56193c130 2024-09-21T03:34:55.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.221+0000 7f245f7fe640 1 --2- 172.21.15.38:0/2020458040 >> [v2:172.21.15.38:6832/3533947700,v1:172.21.15.38:6834/3533947700] conn(0x55e56193c130 0x55e561d3b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:55.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.221+0000 7f243effd640 10 client.5307 ms_handle_connect on v2:172.21.15.38:6832/3533947700 2024-09-21T03:34:55.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.221+0000 7f245f7fe640 1 --2- 172.21.15.38:0/2020458040 >> [v2:172.21.15.38:6832/3533947700,v1:172.21.15.38:6834/3533947700] conn(0x55e56193c130 0x55e561d3b4d0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=645ae7280691f8bf server_cookie=7c39de9785a3a5f6 in_seq=0 out_seq=0 2024-09-21T03:34:55.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.222+0000 7f243effd640 1 -- 172.21.15.38:0/2020458040 <== mds.0 v2:172.21.15.38:6832/3533947700 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e561c998f0 con 0x55e56193c130 2024-09-21T03:34:55.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.222+0000 7f243effd640 10 client.5307 handle_command_reply: tid=0 2024-09-21T03:34:55.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.222+0000 7f246d65d740 1 client.5307 shutdown 2024-09-21T03:34:55.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.222+0000 7f246d65d740 1 -- 172.21.15.38:0/2020458040 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e561954490 msgr2=0x55e561d611a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:55.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.222+0000 7f246d65d740 1 --2- 172.21.15.38:0/2020458040 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e561954490 0x55e561d611a0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f245000bdb0 tx=0x7f24500622f0 comp rx=0 tx=0).stop 2024-09-21T03:34:55.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.222+0000 7f246d65d740 1 -- 172.21.15.38:0/2020458040 shutdown_connections 2024-09-21T03:34:55.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.222+0000 7f246d65d740 1 --2- 172.21.15.38:0/2020458040 >> [v2:172.21.15.38:6832/3533947700,v1:172.21.15.38:6834/3533947700] conn(0x55e56193c130 0x55e561d3b4d0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:55.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.222+0000 7f246d65d740 1 --2- 172.21.15.38:0/2020458040 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e56197d350 0x55e561d630c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:55.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.222+0000 7f246d65d740 1 --2- 172.21.15.38:0/2020458040 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e56195c670 0x55e561d38640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:55.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.222+0000 7f243effd640 0 client.5307 ms_handle_reset on v2:172.21.15.38:6832/3533947700 2024-09-21T03:34:55.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.222+0000 7f246d65d740 1 --2- 172.21.15.38:0/2020458040 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e561954490 0x55e561d611a0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:55.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.222+0000 7f246d65d740 1 -- 172.21.15.38:0/2020458040 >> 172.21.15.38:0/2020458040 conn(0x55e5619714a0 msgr2=0x55e561d1e4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:55.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.222+0000 7f246d65d740 1 -- 172.21.15.38:0/2020458040 shutdown_connections 2024-09-21T03:34:55.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.222+0000 7f246d65d740 1 -- 172.21.15.38:0/2020458040 wait complete. 2024-09-21T03:34:55.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.222+0000 7f246d65d740 20 client.5307 trim_cache size 0 max 16384 2024-09-21T03:34:55.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.236+0000 7f246d65d740 1 Processor -- start 2024-09-21T03:34:55.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.236+0000 7f246d65d740 1 -- start start 2024-09-21T03:34:55.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.237+0000 7f246d65d740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e5619714a0 0x55e561d8d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:55.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.237+0000 7f246d65d740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e561954490 0x55e561d1f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:55.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.237+0000 7f246d65d740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e56195c670 0x55e561d1fc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:55.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.237+0000 7f246d65d740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f24602019c0 con 0x55e561954490 2024-09-21T03:34:55.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.237+0000 7f246d65d740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f24601f99d0 con 0x55e56195c670 2024-09-21T03:34:55.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.237+0000 7f246d65d740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f24602013a0 con 0x55e5619714a0 2024-09-21T03:34:55.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.237+0000 7f245f7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e561954490 0x55e561d1f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:55.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.237+0000 7f245f7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e561954490 0x55e561d1f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35554/0 (socket says 172.21.15.38:35554) 2024-09-21T03:34:55.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.237+0000 7f245f7fe640 1 -- 172.21.15.38:0/2161459641 learned_addr learned my addr 172.21.15.38:0/2161459641 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:34:55.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.238+0000 7f245f7fe640 1 -- 172.21.15.38:0/2161459641 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e5619714a0 msgr2=0x55e561d8d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:55.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.238+0000 7f245f7fe640 1 --2- 172.21.15.38:0/2161459641 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e5619714a0 0x55e561d8d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:55.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.238+0000 7f245f7fe640 1 -- 172.21.15.38:0/2161459641 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e56195c670 msgr2=0x55e561d1fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:55.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.238+0000 7f2464c91640 1 --2- 172.21.15.38:0/2161459641 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e56195c670 0x55e561d1fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:55.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.238+0000 7f245f7fe640 1 --2- 172.21.15.38:0/2161459641 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e56195c670 0x55e561d1fc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:55.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.238+0000 7f245f7fe640 1 -- 172.21.15.38:0/2161459641 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5619c6660 con 0x55e561954490 2024-09-21T03:34:55.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.238+0000 7f245f7fe640 1 --2- 172.21.15.38:0/2161459641 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e561954490 0x55e561d1f6d0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f24600d28f0 tx=0x7f245400a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6d70c886597484a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:55.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.238+0000 7f24457fa640 1 -- 172.21.15.38:0/2161459641 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f245402f070 con 0x55e561954490 2024-09-21T03:34:55.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.238+0000 7f24457fa640 1 -- 172.21.15.38:0/2161459641 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f245401b040 con 0x55e561954490 2024-09-21T03:34:55.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.238+0000 7f24457fa640 1 -- 172.21.15.38:0/2161459641 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24540160d0 con 0x55e561954490 2024-09-21T03:34:55.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.239+0000 7f246d65d740 1 -- 172.21.15.38:0/2161459641 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e561954490 msgr2=0x55e561d1f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:55.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.239+0000 7f246d65d740 1 --2- 172.21.15.38:0/2161459641 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e561954490 0x55e561d1f6d0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f24600d28f0 tx=0x7f245400a000 comp rx=0 tx=0).stop 2024-09-21T03:34:55.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.239+0000 7f246d65d740 1 -- 172.21.15.38:0/2161459641 shutdown_connections 2024-09-21T03:34:55.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.239+0000 7f246d65d740 1 --2- 172.21.15.38:0/2161459641 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e56195c670 0x55e561d1fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:55.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.239+0000 7f246d65d740 1 --2- 172.21.15.38:0/2161459641 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e561954490 0x55e561d1f6d0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:55.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.239+0000 7f246d65d740 1 --2- 172.21.15.38:0/2161459641 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e5619714a0 0x55e561d8d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:55.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.239+0000 7f246d65d740 1 -- 172.21.15.38:0/2161459641 >> 172.21.15.38:0/2161459641 conn(0x55e561d1af80 msgr2=0x55e561d20380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:55.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.239+0000 7f246d65d740 1 -- 172.21.15.38:0/2161459641 shutdown_connections 2024-09-21T03:34:55.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.239+0000 7f246d65d740 1 -- 172.21.15.38:0/2161459641 wait complete. 2024-09-21T03:34:55.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.240+0000 7f246d65d740 1 -- start start 2024-09-21T03:34:55.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.240+0000 7f246d65d740 1 Processor -- start 2024-09-21T03:34:55.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.241+0000 7f246d65d740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e561954490 0x55e561de9ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:55.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.241+0000 7f246d65d740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e56195c670 0x55e561d4ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:55.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.241+0000 7f246d65d740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e5619714a0 0x55e561d77f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:55.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.241+0000 7f246d65d740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55e561d61870 con 0x55e56195c670 2024-09-21T03:34:55.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.241+0000 7f246d65d740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55e561d3c550 con 0x55e5619714a0 2024-09-21T03:34:55.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.241+0000 7f246d65d740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55e561d6d1f0 con 0x55e561954490 2024-09-21T03:34:55.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.242+0000 7f246d65d740 10 client.0 _ll_register_callbacks cb 0x7f2450061eb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:34:55.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.242+0000 7f245f7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e56195c670 0x55e561d4ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:55.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.242+0000 7f245f7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e56195c670 0x55e561d4ea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35566/0 (socket says 172.21.15.38:35566) 2024-09-21T03:34:55.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.242+0000 7f245f7fe640 1 -- 172.21.15.38:0/3257444142 learned_addr learned my addr 172.21.15.38:0/3257444142 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:34:55.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.242+0000 7f2464c91640 1 --2- 172.21.15.38:0/3257444142 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e5619714a0 0x55e561d77f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:55.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.242+0000 7f245ffff640 1 --2- 172.21.15.38:0/3257444142 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e561954490 0x55e561de9ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:55.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.242+0000 7f245f7fe640 1 -- 172.21.15.38:0/3257444142 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e561954490 msgr2=0x55e561de9ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:55.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.242+0000 7f245f7fe640 1 --2- 172.21.15.38:0/3257444142 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e561954490 0x55e561de9ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:55.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.242+0000 7f245f7fe640 1 -- 172.21.15.38:0/3257444142 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e5619714a0 msgr2=0x55e561d77f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:55.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.242+0000 7f245f7fe640 1 --2- 172.21.15.38:0/3257444142 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e5619714a0 0x55e561d77f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:55.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.242+0000 7f245f7fe640 1 -- 172.21.15.38:0/3257444142 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e561d3bab0 con 0x55e56195c670 2024-09-21T03:34:55.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.242+0000 7f245f7fe640 1 --2- 172.21.15.38:0/3257444142 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e56195c670 0x55e561d4ea20 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f2454016840 tx=0x7f2454016970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf679f4f7a7d20b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:55.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.242+0000 7f243effd640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-21T03:34:55.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.242+0000 7f243effd640 1 -- 172.21.15.38:0/3257444142 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f245402f030 con 0x55e56195c670 2024-09-21T03:34:55.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.242+0000 7f243effd640 1 -- 172.21.15.38:0/3257444142 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(41..41 src has 1..41) ==== 6187+0+0 (secure 0 0 0) 0x7f2454029030 con 0x55e56195c670 2024-09-21T03:34:55.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.242+0000 7f246d65d740 1 -- 172.21.15.38:0/3257444142 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f245401b040 con 0x55e56195c670 2024-09-21T03:34:55.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.242+0000 7f243effd640 1 -- 172.21.15.38:0/3257444142 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f245403b050 con 0x55e56195c670 2024-09-21T03:34:55.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.242+0000 7f246d65d740 10 client.5313 fetch_fsmap learned FSMap version 23 2024-09-21T03:34:55.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.242+0000 7f246d65d740 1 -- 172.21.15.38:0/3257444142 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) -- 0x55e561d3d7c0 con 0x55e56195c670 2024-09-21T03:34:55.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.243+0000 7f243effd640 1 -- 172.21.15.38:0/3257444142 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f245401b040 con 0x55e56195c670 2024-09-21T03:34:55.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.243+0000 7f243effd640 1 -- 172.21.15.38:0/3257444142 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 23) ==== 3502+0+0 (secure 0 0 0) 0x7f245403e020 con 0x55e56195c670 2024-09-21T03:34:55.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.243+0000 7f246d65d740 10 client.5313 fetch_fsmap finished waiting for FSMap version 23 2024-09-21T03:34:55.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.243+0000 7f246d65d740 10 client.5313 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T03:34:55.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.243+0000 7f246d65d740 20 client.5313 populate_metadata read hostname 'smithi038' 2024-09-21T03:34:55.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.243+0000 7f246d65d740 1 --2- 172.21.15.38:0/3257444142 >> [v2:172.21.15.38:6832/3533947700,v1:172.21.15.38:6834/3533947700] conn(0x55e561911570 0x55e561d78490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:55.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.243+0000 7f246d65d740 4 client.5313 mds_command: new command op to 5226 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-09-21T03:34:55.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.243+0000 7f246d65d740 1 -- 172.21.15.38:0/3257444142 --> [v2:172.21.15.38:6832/3533947700,v1:172.21.15.38:6834/3533947700] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) -- 0x55e561c953a0 con 0x55e561911570 2024-09-21T03:34:55.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.243+0000 7f245ffff640 1 --2- 172.21.15.38:0/3257444142 >> [v2:172.21.15.38:6832/3533947700,v1:172.21.15.38:6834/3533947700] conn(0x55e561911570 0x55e561d78490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:55.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.243+0000 7f243effd640 10 client.5313 ms_handle_connect on v2:172.21.15.38:6832/3533947700 2024-09-21T03:34:55.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.243+0000 7f245ffff640 1 --2- 172.21.15.38:0/3257444142 >> [v2:172.21.15.38:6832/3533947700,v1:172.21.15.38:6834/3533947700] conn(0x55e561911570 0x55e561d78490 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74fe20cddc40a5cc server_cookie=6c90d1f77d500fa6 in_seq=0 out_seq=0 2024-09-21T03:34:55.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.244+0000 7f243effd640 1 -- 172.21.15.38:0/3257444142 <== mds.0 v2:172.21.15.38:6832/3533947700 1 ==== command_reply(tid 0: 0 ) ==== 8+0+237 (crc 0 0 0) 0x55e561c953a0 con 0x55e561911570 2024-09-21T03:34:55.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.244+0000 7f243effd640 10 client.5313 handle_command_reply: tid=0 2024-09-21T03:34:55.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.244+0000 7f246d65d740 1 client.5313 shutdown 2024-09-21T03:34:55.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.244+0000 7f246d65d740 1 -- 172.21.15.38:0/3257444142 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e56195c670 msgr2=0x55e561d4ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:55.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.244+0000 7f246d65d740 1 --2- 172.21.15.38:0/3257444142 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e56195c670 0x55e561d4ea20 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f2454016840 tx=0x7f2454016970 comp rx=0 tx=0).stop 2024-09-21T03:34:55.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.244+0000 7f246d65d740 1 -- 172.21.15.38:0/3257444142 shutdown_connections 2024-09-21T03:34:55.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.244+0000 7f246d65d740 1 --2- 172.21.15.38:0/3257444142 >> [v2:172.21.15.38:6832/3533947700,v1:172.21.15.38:6834/3533947700] conn(0x55e561911570 0x55e561d78490 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:55.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.244+0000 7f246d65d740 1 --2- 172.21.15.38:0/3257444142 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e5619714a0 0x55e561d77f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:55.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.244+0000 7f246d65d740 1 --2- 172.21.15.38:0/3257444142 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e56195c670 0x55e561d4ea20 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:55.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.244+0000 7f243effd640 0 client.5313 ms_handle_reset on v2:172.21.15.38:6832/3533947700 2024-09-21T03:34:55.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.244+0000 7f246d65d740 1 --2- 172.21.15.38:0/3257444142 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e561954490 0x55e561de9ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:55.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.244+0000 7f246d65d740 1 -- 172.21.15.38:0/3257444142 >> 172.21.15.38:0/3257444142 conn(0x55e561d1af80 msgr2=0x55e561d60e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:55.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.244+0000 7f246d65d740 1 -- 172.21.15.38:0/3257444142 shutdown_connections 2024-09-21T03:34:55.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.244+0000 7f246d65d740 1 -- 172.21.15.38:0/3257444142 wait complete. 2024-09-21T03:34:55.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.244+0000 7f246d65d740 20 client.5313 trim_cache size 0 max 16384 2024-09-21T03:34:55.253 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:34:55.253 INFO:teuthology.orchestra.run.smithi038.stdout:[ 2024-09-21T03:34:55.253 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-21T03:34:55.253 INFO:teuthology.orchestra.run.smithi038.stdout: "damage_type": "dentry", 2024-09-21T03:34:55.253 INFO:teuthology.orchestra.run.smithi038.stdout: "id": 2742033437, 2024-09-21T03:34:55.253 INFO:teuthology.orchestra.run.smithi038.stdout: "ino": 1099511627776, 2024-09-21T03:34:55.253 INFO:teuthology.orchestra.run.smithi038.stdout: "frag": "*", 2024-09-21T03:34:55.253 INFO:teuthology.orchestra.run.smithi038.stdout: "dname": "file_to_be_damaged", 2024-09-21T03:34:55.253 INFO:teuthology.orchestra.run.smithi038.stdout: "snap_id": "head", 2024-09-21T03:34:55.253 INFO:teuthology.orchestra.run.smithi038.stdout: "path": "/subdir/file_to_be_damaged" 2024-09-21T03:34:55.253 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-21T03:34:55.253 INFO:teuthology.orchestra.run.smithi038.stdout:] 2024-09-21T03:34:55.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.245+0000 7f24666f4640 1 -- 172.21.15.38:0/1443781291 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f244c078bc0 msgr2=0x7f244c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:55.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.245+0000 7f24666f4640 1 --2- 172.21.15.38:0/1443781291 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f244c078bc0 0x7f244c07b0d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f24600ce780 tx=0x7f2454009040 comp rx=0 tx=0).stop 2024-09-21T03:34:55.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.245+0000 7f24666f4640 1 -- 172.21.15.38:0/1443781291 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f24600ceb90 msgr2=0x7f24601575a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:55.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.245+0000 7f24666f4640 1 --2- 172.21.15.38:0/1443781291 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f24600ceb90 0x7f24601575a0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f245000b3f0 tx=0x7f245000b8b0 comp rx=0 tx=0).stop 2024-09-21T03:34:55.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.245+0000 7f24666f4640 1 -- 172.21.15.38:0/1443781291 shutdown_connections 2024-09-21T03:34:55.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.245+0000 7f24666f4640 1 --2- 172.21.15.38:0/1443781291 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f246015ee80 0x7f24600d3c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:55.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.245+0000 7f24666f4640 1 --2- 172.21.15.38:0/1443781291 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f244c078bc0 0x7f244c07b0d0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:55.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.245+0000 7f24666f4640 1 --2- 172.21.15.38:0/1443781291 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f24600d0170 0x7f24600cb610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:55.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.245+0000 7f24666f4640 1 --2- 172.21.15.38:0/1443781291 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f24600ceb90 0x7f24601575a0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:55.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.245+0000 7f24666f4640 1 -- 172.21.15.38:0/1443781291 >> 172.21.15.38:0/1443781291 conn(0x7f246009d830 msgr2=0x7f24600b5720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:55.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.245+0000 7f24666f4640 1 -- 172.21.15.38:0/1443781291 shutdown_connections 2024-09-21T03:34:55.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.245+0000 7f24666f4640 1 -- 172.21.15.38:0/1443781291 wait complete. 2024-09-21T03:34:55.258 DEBUG:teuthology.orchestra.run.smithi038:> stdin-killer -- python3 -c ' 2024-09-21T03:34:55.258 DEBUG:teuthology.orchestra.run.smithi038:> import sys 2024-09-21T03:34:55.258 DEBUG:teuthology.orchestra.run.smithi038:> import errno 2024-09-21T03:34:55.258 DEBUG:teuthology.orchestra.run.smithi038:> 2024-09-21T03:34:55.259 DEBUG:teuthology.orchestra.run.smithi038:> try: 2024-09-21T03:34:55.259 DEBUG:teuthology.orchestra.run.smithi038:> f = open("/home/ubuntu/cephtest/mnt.0/subdir/file_to_be_damaged", "w") 2024-09-21T03:34:55.259 DEBUG:teuthology.orchestra.run.smithi038:> f.close() 2024-09-21T03:34:55.259 DEBUG:teuthology.orchestra.run.smithi038:> except IOError as e: 2024-09-21T03:34:55.259 DEBUG:teuthology.orchestra.run.smithi038:> sys.exit(errno.EIO) 2024-09-21T03:34:55.259 DEBUG:teuthology.orchestra.run.smithi038:> ' 2024-09-21T03:34:55.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55 stdin-killer INFO: command exited with status 5: exiting normally with same code! 2024-09-21T03:34:55.335 DEBUG:teuthology.orchestra.run:got remote process result: 5 2024-09-21T03:34:55.336 DEBUG:teuthology.orchestra.run.smithi038:> stdin-killer -- python3 -c ' 2024-09-21T03:34:55.336 DEBUG:teuthology.orchestra.run.smithi038:> import os 2024-09-21T03:34:55.336 DEBUG:teuthology.orchestra.run.smithi038:> import stat 2024-09-21T03:34:55.336 DEBUG:teuthology.orchestra.run.smithi038:> import json 2024-09-21T03:34:55.336 DEBUG:teuthology.orchestra.run.smithi038:> import sys 2024-09-21T03:34:55.336 DEBUG:teuthology.orchestra.run.smithi038:> 2024-09-21T03:34:55.336 DEBUG:teuthology.orchestra.run.smithi038:> try: 2024-09-21T03:34:55.336 DEBUG:teuthology.orchestra.run.smithi038:> s = os.stat('"'"'/home/ubuntu/cephtest/mnt.0/subdir/file_to_be_damaged'"'"') 2024-09-21T03:34:55.337 DEBUG:teuthology.orchestra.run.smithi038:> except OSError as e: 2024-09-21T03:34:55.337 DEBUG:teuthology.orchestra.run.smithi038:> sys.exit(e.errno) 2024-09-21T03:34:55.337 DEBUG:teuthology.orchestra.run.smithi038:> 2024-09-21T03:34:55.337 DEBUG:teuthology.orchestra.run.smithi038:> attrs = ["st_mode", "st_ino", "st_dev", "st_nlink", "st_uid", "st_gid", "st_size", "st_atime", "st_mtime", "st_ctime"] 2024-09-21T03:34:55.337 DEBUG:teuthology.orchestra.run.smithi038:> print(json.dumps( 2024-09-21T03:34:55.337 DEBUG:teuthology.orchestra.run.smithi038:> dict([(a, getattr(s, a)) for a in attrs]), 2024-09-21T03:34:55.337 DEBUG:teuthology.orchestra.run.smithi038:> indent=2)) 2024-09-21T03:34:55.337 DEBUG:teuthology.orchestra.run.smithi038:> ' 2024-09-21T03:34:55.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55 stdin-killer INFO: command exited with status 5: exiting normally with same code! 2024-09-21T03:34:55.454 DEBUG:teuthology.orchestra.run:got remote process result: 5 2024-09-21T03:34:55.454 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:34:55.455 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.files ./subdir) 2024-09-21T03:34:55.511 INFO:teuthology.orchestra.run.smithi038.stdout:2 2024-09-21T03:34:55.511 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-21T03:34:55.512 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:34:55.567 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-21T03:34:55.568 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:34:55.623 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T03:34:55.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.779+0000 7fc94b00e640 1 -- 172.21.15.38:0/913961365 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc9440d0170 msgr2=0x7fc9440d0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:55.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.779+0000 7fc94b00e640 1 --2- 172.21.15.38:0/913961365 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc9440d0170 0x7fc9440d0be0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fc9440d2080 tx=0x7fc92c02a420 comp rx=0 tx=0).stop 2024-09-21T03:34:55.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.779+0000 7fc94b00e640 1 -- 172.21.15.38:0/913961365 shutdown_connections 2024-09-21T03:34:55.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.779+0000 7fc94b00e640 1 --2- 172.21.15.38:0/913961365 >> v2:172.21.15.195:3300/0 conn(0x7fc9440d1120 0x7fc9440ceb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:55.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.779+0000 7fc94b00e640 1 --2- 172.21.15.38:0/913961365 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc9440d0170 0x7fc9440d0be0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:55.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.779+0000 7fc94b00e640 1 -- 172.21.15.38:0/913961365 >> 172.21.15.38:0/913961365 conn(0x7fc94409d830 msgr2=0x7fc9440b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:55.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.779+0000 7fc94b00e640 1 -- 172.21.15.38:0/913961365 shutdown_connections 2024-09-21T03:34:55.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.779+0000 7fc94b00e640 1 -- 172.21.15.38:0/913961365 wait complete. 2024-09-21T03:34:55.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.781+0000 7fc94b00e640 1 Processor -- start 2024-09-21T03:34:55.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.781+0000 7fc94b00e640 1 -- start start 2024-09-21T03:34:55.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.782+0000 7fc94b00e640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc9440d0170 0x7fc94406f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:55.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.782+0000 7fc94b00e640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc9440d1120 0x7fc94414f970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:55.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.782+0000 7fc94b00e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc94415ee80 0x7fc9440b2630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:55.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.782+0000 7fc94b00e640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fc9440b6700 con 0x7fc94415ee80 2024-09-21T03:34:55.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.782+0000 7fc94b00e640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fc9440b6580 con 0x7fc9440d1120 2024-09-21T03:34:55.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.782+0000 7fc94b00e640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fc9440b6880 con 0x7fc9440d0170 2024-09-21T03:34:55.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.783+0000 7fc9495ab640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc94415ee80 0x7fc9440b2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:55.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.783+0000 7fc9495ab640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc94415ee80 0x7fc9440b2630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35584/0 (socket says 172.21.15.38:35584) 2024-09-21T03:34:55.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.783+0000 7fc9495ab640 1 -- 172.21.15.38:0/893503083 learned_addr learned my addr 172.21.15.38:0/893503083 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:34:55.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.783+0000 7fc943fff640 1 --2- 172.21.15.38:0/893503083 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc9440d1120 0x7fc94414f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:55.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.783+0000 7fc948daa640 1 --2- 172.21.15.38:0/893503083 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc9440d0170 0x7fc94406f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:55.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.783+0000 7fc9495ab640 1 -- 172.21.15.38:0/893503083 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc9440d0170 msgr2=0x7fc94406f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:55.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.783+0000 7fc9495ab640 1 --2- 172.21.15.38:0/893503083 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc9440d0170 0x7fc94406f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:55.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.783+0000 7fc9495ab640 1 -- 172.21.15.38:0/893503083 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc9440d1120 msgr2=0x7fc94414f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:55.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.783+0000 7fc9495ab640 1 --2- 172.21.15.38:0/893503083 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc9440d1120 0x7fc94414f970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:55.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.783+0000 7fc9495ab640 1 -- 172.21.15.38:0/893503083 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc94414feb0 con 0x7fc94415ee80 2024-09-21T03:34:55.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.783+0000 7fc9495ab640 1 --2- 172.21.15.38:0/893503083 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc94415ee80 0x7fc9440b2630 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fc93c00e990 tx=0x7fc93c00ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:55.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.783+0000 7fc9417fa640 1 -- 172.21.15.38:0/893503083 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc93c004d20 con 0x7fc94415ee80 2024-09-21T03:34:55.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.783+0000 7fc9417fa640 1 -- 172.21.15.38:0/893503083 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc93c00caa0 con 0x7fc94415ee80 2024-09-21T03:34:55.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.783+0000 7fc94b00e640 1 -- 172.21.15.38:0/893503083 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc94406d330 con 0x7fc94415ee80 2024-09-21T03:34:55.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.783+0000 7fc9417fa640 1 -- 172.21.15.38:0/893503083 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc93c005020 con 0x7fc94415ee80 2024-09-21T03:34:55.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.783+0000 7fc94b00e640 1 -- 172.21.15.38:0/893503083 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc94406d8f0 con 0x7fc94415ee80 2024-09-21T03:34:55.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.784+0000 7fc94b00e640 1 -- 172.21.15.38:0/893503083 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9440b6da0 con 0x7fc94415ee80 2024-09-21T03:34:55.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.785+0000 7fc9417fa640 1 -- 172.21.15.38:0/893503083 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc93c0051c0 con 0x7fc94415ee80 2024-09-21T03:34:55.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.786+0000 7fc9417fa640 1 --2- 172.21.15.38:0/893503083 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc924078b90 0x7fc92407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:55.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.786+0000 7fc9417fa640 1 -- 172.21.15.38:0/893503083 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6187+0+0 (secure 0 0 0) 0x7fc93c09b750 con 0x7fc94415ee80 2024-09-21T03:34:55.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.786+0000 7fc948daa640 1 --2- 172.21.15.38:0/893503083 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc924078b90 0x7fc92407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:55.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.787+0000 7fc948daa640 1 --2- 172.21.15.38:0/893503083 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc924078b90 0x7fc92407b0a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc934006fd0 tx=0x7fc934009060 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:55.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.790+0000 7fc9417fa640 1 -- 172.21.15.38:0/893503083 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fc93c016030 con 0x7fc94415ee80 2024-09-21T03:34:55.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.990+0000 7fc94b00e640 1 -- 172.21.15.38:0/893503083 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fc944158b40 con 0x7fc94415ee80 2024-09-21T03:34:55.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.990+0000 7fc9417fa640 1 -- 172.21.15.38:0/893503083 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v41) ==== 80+0+122 (secure 0 0 0) 0x7fc93c0633f0 con 0x7fc94415ee80 2024-09-21T03:34:55.992 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6834/1865253656 2024-09-22T03:34:47.895300+0000 2024-09-21T03:34:55.993 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6832/1865253656 2024-09-22T03:34:47.895300+0000 2024-09-21T03:34:55.993 INFO:teuthology.orchestra.run.smithi038.stderr:listed 2 entries 2024-09-21T03:34:55.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.993+0000 7fc94b00e640 1 -- 172.21.15.38:0/893503083 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc924078b90 msgr2=0x7fc92407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:55.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.993+0000 7fc94b00e640 1 --2- 172.21.15.38:0/893503083 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc924078b90 0x7fc92407b0a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc934006fd0 tx=0x7fc934009060 comp rx=0 tx=0).stop 2024-09-21T03:34:55.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.993+0000 7fc94b00e640 1 -- 172.21.15.38:0/893503083 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc94415ee80 msgr2=0x7fc9440b2630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:55.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.993+0000 7fc94b00e640 1 --2- 172.21.15.38:0/893503083 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc94415ee80 0x7fc9440b2630 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fc93c00e990 tx=0x7fc93c00ee50 comp rx=0 tx=0).stop 2024-09-21T03:34:55.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.993+0000 7fc94b00e640 1 -- 172.21.15.38:0/893503083 shutdown_connections 2024-09-21T03:34:55.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.993+0000 7fc94b00e640 1 --2- 172.21.15.38:0/893503083 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc94415ee80 0x7fc9440b2630 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:55.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.993+0000 7fc94b00e640 1 --2- 172.21.15.38:0/893503083 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc9440d1120 0x7fc94414f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:55.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.993+0000 7fc94b00e640 1 --2- 172.21.15.38:0/893503083 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc924078b90 0x7fc92407b0a0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:55.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.993+0000 7fc94b00e640 1 --2- 172.21.15.38:0/893503083 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc9440d0170 0x7fc94406f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:55.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.993+0000 7fc94b00e640 1 -- 172.21.15.38:0/893503083 >> 172.21.15.38:0/893503083 conn(0x7fc94409d830 msgr2=0x7fc9440b3fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:55.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.993+0000 7fc94b00e640 1 -- 172.21.15.38:0/893503083 shutdown_connections 2024-09-21T03:34:55.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:55.993+0000 7fc94b00e640 1 -- 172.21.15.38:0/893503083 wait complete. 2024-09-21T03:34:56.006 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-21T03:34:56.006 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:34:56.006 DEBUG:teuthology.orchestra.run.smithi038:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-21T03:34:56.035 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi038.front.sepia.ceph.com 2024-09-21T03:34:56.035 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:34:56.036 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T03:34:56.093 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:34:56.093 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.2:0 scrub start /subdir repair 2024-09-21T03:34:56.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.253+0000 7fec08fe6640 1 -- 172.21.15.38:0/571824593 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7febec004d40 msgr2=0x7febec005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:56.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.253+0000 7fec08fe6640 1 --2- 172.21.15.38:0/571824593 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7febec004d40 0x7febec005150 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fec040cf440 tx=0x7febec005830 comp rx=0 tx=0).stop 2024-09-21T03:34:56.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.253+0000 7fec02d9d640 1 -- 172.21.15.38:0/571824593 reap_dead start 2024-09-21T03:34:56.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.253+0000 7fec08fe6640 1 -- 172.21.15.38:0/571824593 shutdown_connections 2024-09-21T03:34:56.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.253+0000 7fec08fe6640 1 -- 172.21.15.38:0/571824593 >> 172.21.15.38:0/571824593 conn(0x7fec0406cde0 msgr2=0x7fec0406d1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:56.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.253+0000 7fec08fe6640 1 -- 172.21.15.38:0/571824593 shutdown_connections 2024-09-21T03:34:56.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.253+0000 7fec08fe6640 1 -- 172.21.15.38:0/571824593 wait complete. 2024-09-21T03:34:56.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.255+0000 7fec08fe6640 1 Processor -- start 2024-09-21T03:34:56.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.255+0000 7fec08fe6640 1 -- start start 2024-09-21T03:34:56.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.256+0000 7fec08fe6640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7febec004d40 0x7fec04150a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:56.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.256+0000 7fec08fe6640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fec04151190 0x7fec0415de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:56.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.256+0000 7fec08fe6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fec041516e0 0x7fec0415e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:56.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.256+0000 7fec08fe6640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fec040b3540 con 0x7fec041516e0 2024-09-21T03:34:56.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.256+0000 7fec08fe6640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fec0415c8a0 con 0x7febec004d40 2024-09-21T03:34:56.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.256+0000 7fec08fe6640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fec040b36c0 con 0x7fec04151190 2024-09-21T03:34:56.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.256+0000 7fec02d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fec041516e0 0x7fec0415e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:56.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.256+0000 7fec02d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fec041516e0 0x7fec0415e3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35596/0 (socket says 172.21.15.38:35596) 2024-09-21T03:34:56.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.256+0000 7fec02d9d640 1 -- 172.21.15.38:0/1937122390 learned_addr learned my addr 172.21.15.38:0/1937122390 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:34:56.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.256+0000 7fec01d9b640 1 --2- 172.21.15.38:0/1937122390 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fec04151190 0x7fec0415de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:56.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.257+0000 7fec0259c640 1 --2- 172.21.15.38:0/1937122390 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7febec004d40 0x7fec04150a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:56.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.257+0000 7fec02d9d640 1 -- 172.21.15.38:0/1937122390 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fec04151190 msgr2=0x7fec0415de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:56.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.257+0000 7fec02d9d640 1 --2- 172.21.15.38:0/1937122390 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fec04151190 0x7fec0415de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:56.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.257+0000 7fec02d9d640 1 -- 172.21.15.38:0/1937122390 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7febec004d40 msgr2=0x7fec04150a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:56.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.257+0000 7fec02d9d640 1 --2- 172.21.15.38:0/1937122390 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7febec004d40 0x7fec04150a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:56.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.257+0000 7fec02d9d640 1 -- 172.21.15.38:0/1937122390 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec04153160 con 0x7fec041516e0 2024-09-21T03:34:56.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.257+0000 7fec02d9d640 1 --2- 172.21.15.38:0/1937122390 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fec041516e0 0x7fec0415e3d0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7febfc009a70 tx=0x7febfc034660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:56.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.257+0000 7febf2ffd640 1 -- 172.21.15.38:0/1937122390 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febfc038040 con 0x7fec041516e0 2024-09-21T03:34:56.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.257+0000 7febf2ffd640 1 -- 172.21.15.38:0/1937122390 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7febfc042070 con 0x7fec041516e0 2024-09-21T03:34:56.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.257+0000 7fec08fe6640 1 -- 172.21.15.38:0/1937122390 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec04153420 con 0x7fec041516e0 2024-09-21T03:34:56.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.257+0000 7febf2ffd640 1 -- 172.21.15.38:0/1937122390 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febfc034b20 con 0x7fec041516e0 2024-09-21T03:34:56.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.257+0000 7fec08fe6640 1 -- 172.21.15.38:0/1937122390 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec041616b0 con 0x7fec041516e0 2024-09-21T03:34:56.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.259+0000 7febf2ffd640 1 -- 172.21.15.38:0/1937122390 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7febfc034cc0 con 0x7fec041516e0 2024-09-21T03:34:56.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.260+0000 7febf2ffd640 1 --2- 172.21.15.38:0/1937122390 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7febe4078b90 0x7febe407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:56.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.260+0000 7febf2ffd640 1 -- 172.21.15.38:0/1937122390 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6187+0+0 (secure 0 0 0) 0x7febfc0bb7e0 con 0x7fec041516e0 2024-09-21T03:34:56.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.260+0000 7fec0259c640 1 --2- 172.21.15.38:0/1937122390 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7febe4078b90 0x7febe407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:56.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.260+0000 7fec1002d740 1 Processor -- start 2024-09-21T03:34:56.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.260+0000 7fec0259c640 1 --2- 172.21.15.38:0/1937122390 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7febe4078b90 0x7febe407b0a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7febf4005fd0 tx=0x7febf4009500 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:56.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.261+0000 7fec1002d740 1 -- start start 2024-09-21T03:34:56.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.262+0000 7fec1002d740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55756f40b5b0 0x55756f7fa070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:56.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.262+0000 7fec1002d740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55756f3e26f0 0x55756f7dcd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:56.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.262+0000 7fec1002d740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55756f3ea8d0 0x55756f7dd280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:56.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.262+0000 7fec1002d740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55756f725a80 con 0x55756f40b5b0 2024-09-21T03:34:56.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.262+0000 7fec1002d740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55756f714eb0 con 0x55756f3e26f0 2024-09-21T03:34:56.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.262+0000 7fec1002d740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55756f718cc0 con 0x55756f3ea8d0 2024-09-21T03:34:56.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.262+0000 7fec0259c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55756f40b5b0 0x55756f7fa070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:56.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.262+0000 7fec0259c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55756f40b5b0 0x55756f7fa070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35612/0 (socket says 172.21.15.38:35612) 2024-09-21T03:34:56.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.262+0000 7fec0259c640 1 -- 172.21.15.38:0/728250050 learned_addr learned my addr 172.21.15.38:0/728250050 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:34:56.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.262+0000 7fec02d9d640 1 --2- 172.21.15.38:0/728250050 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55756f3ea8d0 0x55756f7dd280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:56.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.262+0000 7fec01d9b640 1 --2- 172.21.15.38:0/728250050 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55756f3e26f0 0x55756f7dcd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:56.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.262+0000 7fec0259c640 1 -- 172.21.15.38:0/728250050 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55756f3ea8d0 msgr2=0x55756f7dd280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:56.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.262+0000 7fec0259c640 1 --2- 172.21.15.38:0/728250050 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55756f3ea8d0 0x55756f7dd280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:56.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.262+0000 7fec0259c640 1 -- 172.21.15.38:0/728250050 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55756f3e26f0 msgr2=0x55756f7dcd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:56.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.262+0000 7fec0259c640 1 --2- 172.21.15.38:0/728250050 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55756f3e26f0 0x55756f7dcd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:56.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.262+0000 7fec0259c640 1 -- 172.21.15.38:0/728250050 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55756f4efa20 con 0x55756f40b5b0 2024-09-21T03:34:56.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.262+0000 7fec0259c640 1 --2- 172.21.15.38:0/728250050 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55756f40b5b0 0x55756f7fa070 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7febf4003d30 tx=0x7febf4016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ecaa13df716ad57 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:56.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.263+0000 7febe3fff640 1 -- 172.21.15.38:0/728250050 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febf4003e20 con 0x55756f40b5b0 2024-09-21T03:34:56.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.263+0000 7febe3fff640 1 -- 172.21.15.38:0/728250050 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7febf4028440 con 0x55756f40b5b0 2024-09-21T03:34:56.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.263+0000 7febe3fff640 1 -- 172.21.15.38:0/728250050 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febf4028810 con 0x55756f40b5b0 2024-09-21T03:34:56.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.264+0000 7fec1002d740 1 -- 172.21.15.38:0/728250050 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55756f40b5b0 msgr2=0x55756f7fa070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:56.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.264+0000 7fec1002d740 1 --2- 172.21.15.38:0/728250050 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55756f40b5b0 0x55756f7fa070 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7febf4003d30 tx=0x7febf4016000 comp rx=0 tx=0).stop 2024-09-21T03:34:56.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.264+0000 7fec1002d740 1 -- 172.21.15.38:0/728250050 shutdown_connections 2024-09-21T03:34:56.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.264+0000 7fec1002d740 1 --2- 172.21.15.38:0/728250050 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55756f3ea8d0 0x55756f7dd280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:56.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.264+0000 7fec1002d740 1 --2- 172.21.15.38:0/728250050 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55756f3e26f0 0x55756f7dcd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:56.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.264+0000 7fec1002d740 1 --2- 172.21.15.38:0/728250050 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55756f40b5b0 0x55756f7fa070 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:56.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.264+0000 7fec1002d740 1 -- 172.21.15.38:0/728250050 >> 172.21.15.38:0/728250050 conn(0x55756f3ff700 msgr2=0x55756f7b2f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:56.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.264+0000 7fec1002d740 1 -- 172.21.15.38:0/728250050 shutdown_connections 2024-09-21T03:34:56.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.264+0000 7fec1002d740 1 -- 172.21.15.38:0/728250050 wait complete. 2024-09-21T03:34:56.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.265+0000 7fec1002d740 1 -- start start 2024-09-21T03:34:56.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.265+0000 7fec1002d740 1 Processor -- start 2024-09-21T03:34:56.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.266+0000 7fec1002d740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55756f3e26f0 0x55756f7ee7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:56.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.266+0000 7fec1002d740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55756f3ea8d0 0x55756f7d9ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:56.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.266+0000 7fec1002d740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55756f40b5b0 0x55756f807290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:56.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.266+0000 7fec1002d740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55756f485960 con 0x55756f40b5b0 2024-09-21T03:34:56.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.266+0000 7fec1002d740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55756f69b670 con 0x55756f3ea8d0 2024-09-21T03:34:56.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.266+0000 7fec1002d740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55756f7ef260 con 0x55756f3e26f0 2024-09-21T03:34:56.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.266+0000 7fec1002d740 10 client.0 _ll_register_callbacks cb 0x55756f70e3e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:34:56.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.266+0000 7fec02d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55756f40b5b0 0x55756f807290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:56.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.266+0000 7fec02d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55756f40b5b0 0x55756f807290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35628/0 (socket says 172.21.15.38:35628) 2024-09-21T03:34:56.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.266+0000 7fec02d9d640 1 -- 172.21.15.38:0/242119848 learned_addr learned my addr 172.21.15.38:0/242119848 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:34:56.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.266+0000 7fec01d9b640 1 --2- 172.21.15.38:0/242119848 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55756f3ea8d0 0x55756f7d9ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:56.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.266+0000 7fec0259c640 1 --2- 172.21.15.38:0/242119848 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55756f3e26f0 0x55756f7ee7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:56.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.267+0000 7fec02d9d640 1 -- 172.21.15.38:0/242119848 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55756f3e26f0 msgr2=0x55756f7ee7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:56.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.267+0000 7fec02d9d640 1 --2- 172.21.15.38:0/242119848 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55756f3e26f0 0x55756f7ee7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:56.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.267+0000 7fec02d9d640 1 -- 172.21.15.38:0/242119848 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55756f3ea8d0 msgr2=0x55756f7d9ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:56.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.267+0000 7fec02d9d640 1 --2- 172.21.15.38:0/242119848 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55756f3ea8d0 0x55756f7d9ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:56.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.267+0000 7fec02d9d640 1 -- 172.21.15.38:0/242119848 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55756f4548c0 con 0x55756f40b5b0 2024-09-21T03:34:56.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.267+0000 7fec02d9d640 1 --2- 172.21.15.38:0/242119848 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55756f40b5b0 0x55756f807290 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7febfc02bfd0 tx=0x7febfc0839a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ef05e2891aaa4a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:56.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.267+0000 7febe17fa640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-21T03:34:56.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.267+0000 7febe17fa640 1 -- 172.21.15.38:0/242119848 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febfc082830 con 0x55756f40b5b0 2024-09-21T03:34:56.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.267+0000 7fec1002d740 1 -- 172.21.15.38:0/242119848 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7febf4028440 con 0x55756f40b5b0 2024-09-21T03:34:56.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.267+0000 7febe17fa640 1 -- 172.21.15.38:0/242119848 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(41..41 src has 1..41) ==== 6187+0+0 (secure 0 0 0) 0x7febfc08a0b0 con 0x55756f40b5b0 2024-09-21T03:34:56.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.267+0000 7febe17fa640 1 -- 172.21.15.38:0/242119848 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7febfc0833d0 con 0x55756f40b5b0 2024-09-21T03:34:56.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.267+0000 7fec1002d740 10 client.5331 fetch_fsmap learned FSMap version 23 2024-09-21T03:34:56.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.267+0000 7fec1002d740 1 -- 172.21.15.38:0/242119848 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) -- 0x55756f5501b0 con 0x55756f40b5b0 2024-09-21T03:34:56.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.267+0000 7febe17fa640 1 -- 172.21.15.38:0/242119848 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7febf4028440 con 0x55756f40b5b0 2024-09-21T03:34:56.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.267+0000 7febe17fa640 1 -- 172.21.15.38:0/242119848 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 23) ==== 3502+0+0 (secure 0 0 0) 0x7febfc03b050 con 0x55756f40b5b0 2024-09-21T03:34:56.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.267+0000 7fec1002d740 10 client.5331 fetch_fsmap finished waiting for FSMap version 23 2024-09-21T03:34:56.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.267+0000 7fec1002d740 10 client.5331 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-09-21T03:34:56.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.267+0000 7fec1002d740 20 client.5331 populate_metadata read hostname 'smithi038' 2024-09-21T03:34:56.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.267+0000 7fec1002d740 1 --2- 172.21.15.38:0/242119848 >> [v2:172.21.15.38:6832/3533947700,v1:172.21.15.38:6834/3533947700] conn(0x55756f3ca180 0x55756f8077d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:56.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.267+0000 7fec1002d740 4 client.5331 mds_command: new command op to 5226 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:34:56.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.267+0000 7fec1002d740 1 -- 172.21.15.38:0/242119848 --> [v2:172.21.15.38:6832/3533947700,v1:172.21.15.38:6834/3533947700] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55756f7279d0 con 0x55756f3ca180 2024-09-21T03:34:56.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.268+0000 7fec0259c640 1 --2- 172.21.15.38:0/242119848 >> [v2:172.21.15.38:6832/3533947700,v1:172.21.15.38:6834/3533947700] conn(0x55756f3ca180 0x55756f8077d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:56.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.268+0000 7febe17fa640 10 client.5331 ms_handle_connect on v2:172.21.15.38:6832/3533947700 2024-09-21T03:34:56.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.268+0000 7fec0259c640 1 --2- 172.21.15.38:0/242119848 >> [v2:172.21.15.38:6832/3533947700,v1:172.21.15.38:6834/3533947700] conn(0x55756f3ca180 0x55756f8077d0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b45f056c4340dd12 server_cookie=88adafefe33f031e in_seq=0 out_seq=0 2024-09-21T03:34:56.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.269+0000 7febe17fa640 1 -- 172.21.15.38:0/242119848 <== mds.0 v2:172.21.15.38:6832/3533947700 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55756f7279d0 con 0x55756f3ca180 2024-09-21T03:34:56.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.269+0000 7febe17fa640 10 client.5331 handle_command_reply: tid=0 2024-09-21T03:34:56.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.269+0000 7fec1002d740 1 client.5331 shutdown 2024-09-21T03:34:56.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.269+0000 7fec1002d740 1 -- 172.21.15.38:0/242119848 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55756f40b5b0 msgr2=0x55756f807290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:56.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.269+0000 7fec1002d740 1 --2- 172.21.15.38:0/242119848 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55756f40b5b0 0x55756f807290 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7febfc02bfd0 tx=0x7febfc0839a0 comp rx=0 tx=0).stop 2024-09-21T03:34:56.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.269+0000 7fec1002d740 1 -- 172.21.15.38:0/242119848 shutdown_connections 2024-09-21T03:34:56.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.269+0000 7fec1002d740 1 --2- 172.21.15.38:0/242119848 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55756f40b5b0 0x55756f807290 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:56.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.269+0000 7fec1002d740 1 --2- 172.21.15.38:0/242119848 >> [v2:172.21.15.38:6832/3533947700,v1:172.21.15.38:6834/3533947700] conn(0x55756f3ca180 0x55756f8077d0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:56.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.269+0000 7fec1002d740 1 --2- 172.21.15.38:0/242119848 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55756f3ea8d0 0x55756f7d9ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:56.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.269+0000 7fec1002d740 1 --2- 172.21.15.38:0/242119848 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55756f3e26f0 0x55756f7ee7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:56.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.269+0000 7febe17fa640 0 client.5331 ms_handle_reset on v2:172.21.15.38:6832/3533947700 2024-09-21T03:34:56.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.269+0000 7fec1002d740 1 -- 172.21.15.38:0/242119848 >> 172.21.15.38:0/242119848 conn(0x55756f3ff700 msgr2=0x55756f7dcd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:56.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.269+0000 7fec1002d740 1 -- 172.21.15.38:0/242119848 shutdown_connections 2024-09-21T03:34:56.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.269+0000 7fec1002d740 1 -- 172.21.15.38:0/242119848 wait complete. 2024-09-21T03:34:56.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.269+0000 7fec1002d740 20 client.5331 trim_cache size 0 max 16384 2024-09-21T03:34:56.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.287+0000 7fec1002d740 1 Processor -- start 2024-09-21T03:34:56.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.287+0000 7fec1002d740 1 -- start start 2024-09-21T03:34:56.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.289+0000 7fec1002d740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55756f3e26f0 0x55756f81b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:56.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.289+0000 7fec1002d740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55756f3ea8d0 0x55756f7d98a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:56.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.289+0000 7fec1002d740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55756f40b5b0 0x55756f7d9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:56.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.289+0000 7fec1002d740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55756f7ae7f0 con 0x55756f3ea8d0 2024-09-21T03:34:56.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.289+0000 7fec1002d740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fec04202860 con 0x55756f40b5b0 2024-09-21T03:34:56.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.289+0000 7fec1002d740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fec042018d0 con 0x55756f3e26f0 2024-09-21T03:34:56.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.289+0000 7fec01d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55756f3ea8d0 0x55756f7d98a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:56.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.289+0000 7fec01d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55756f3ea8d0 0x55756f7d98a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35644/0 (socket says 172.21.15.38:35644) 2024-09-21T03:34:56.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.289+0000 7fec01d9b640 1 -- 172.21.15.38:0/2580844241 learned_addr learned my addr 172.21.15.38:0/2580844241 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:34:56.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.289+0000 7fec0259c640 1 --2- 172.21.15.38:0/2580844241 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55756f3e26f0 0x55756f81b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:56.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.289+0000 7fec02d9d640 1 --2- 172.21.15.38:0/2580844241 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55756f40b5b0 0x55756f7d9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:56.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.289+0000 7fec01d9b640 1 -- 172.21.15.38:0/2580844241 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55756f3e26f0 msgr2=0x55756f81b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:56.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.289+0000 7fec01d9b640 1 --2- 172.21.15.38:0/2580844241 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55756f3e26f0 0x55756f81b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:56.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.289+0000 7fec01d9b640 1 -- 172.21.15.38:0/2580844241 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55756f40b5b0 msgr2=0x55756f7d9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:56.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.289+0000 7fec01d9b640 1 --2- 172.21.15.38:0/2580844241 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55756f40b5b0 0x55756f7d9de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:56.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.289+0000 7fec01d9b640 1 -- 172.21.15.38:0/2580844241 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55756f7b2a40 con 0x55756f3ea8d0 2024-09-21T03:34:56.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.289+0000 7fec01d9b640 1 --2- 172.21.15.38:0/2580844241 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55756f3ea8d0 0x55756f7d98a0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fec041e9880 tx=0x7febf80067a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d39121966c272282 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:56.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.289+0000 7febe37fe640 1 -- 172.21.15.38:0/2580844241 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febf8023070 con 0x55756f3ea8d0 2024-09-21T03:34:56.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.289+0000 7febe37fe640 1 -- 172.21.15.38:0/2580844241 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7febf8003d20 con 0x55756f3ea8d0 2024-09-21T03:34:56.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.289+0000 7febe37fe640 1 -- 172.21.15.38:0/2580844241 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febf8012660 con 0x55756f3ea8d0 2024-09-21T03:34:56.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.290+0000 7fec1002d740 1 -- 172.21.15.38:0/2580844241 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55756f3ea8d0 msgr2=0x55756f7d98a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:56.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.290+0000 7fec1002d740 1 --2- 172.21.15.38:0/2580844241 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55756f3ea8d0 0x55756f7d98a0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fec041e9880 tx=0x7febf80067a0 comp rx=0 tx=0).stop 2024-09-21T03:34:56.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.290+0000 7fec1002d740 1 -- 172.21.15.38:0/2580844241 shutdown_connections 2024-09-21T03:34:56.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.290+0000 7fec1002d740 1 --2- 172.21.15.38:0/2580844241 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55756f40b5b0 0x55756f7d9de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:56.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.290+0000 7fec1002d740 1 --2- 172.21.15.38:0/2580844241 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55756f3ea8d0 0x55756f7d98a0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:56.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.290+0000 7fec1002d740 1 --2- 172.21.15.38:0/2580844241 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55756f3e26f0 0x55756f81b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:56.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.290+0000 7fec1002d740 1 -- 172.21.15.38:0/2580844241 >> 172.21.15.38:0/2580844241 conn(0x55756f3ff700 msgr2=0x55756f7d6e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:56.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.290+0000 7fec1002d740 1 -- 172.21.15.38:0/2580844241 shutdown_connections 2024-09-21T03:34:56.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.290+0000 7fec1002d740 1 -- 172.21.15.38:0/2580844241 wait complete. 2024-09-21T03:34:56.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.291+0000 7fec1002d740 1 -- start start 2024-09-21T03:34:56.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.291+0000 7fec1002d740 1 Processor -- start 2024-09-21T03:34:56.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.293+0000 7fec1002d740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55756f3e26f0 0x55756f877cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:56.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.293+0000 7fec1002d740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55756f3ea8d0 0x55756f86e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:56.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.293+0000 7fec1002d740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55756f40b5b0 0x55756f7b09c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:56.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.293+0000 7fec1002d740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fec04201280 con 0x55756f40b5b0 2024-09-21T03:34:56.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.293+0000 7fec1002d740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fec040d45a0 con 0x55756f3e26f0 2024-09-21T03:34:56.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.293+0000 7fec1002d740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55756f80a360 con 0x55756f3ea8d0 2024-09-21T03:34:56.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.293+0000 7fec1002d740 10 client.0 _ll_register_callbacks cb 0x7febfc0b9f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:34:56.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.293+0000 7fec02d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55756f40b5b0 0x55756f7b09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:56.299 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.293+0000 7fec02d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55756f40b5b0 0x55756f7b09c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35646/0 (socket says 172.21.15.38:35646) 2024-09-21T03:34:56.299 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.293+0000 7fec02d9d640 1 -- 172.21.15.38:0/1551275732 learned_addr learned my addr 172.21.15.38:0/1551275732 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:34:56.299 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.293+0000 7fec0259c640 1 --2- 172.21.15.38:0/1551275732 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55756f3e26f0 0x55756f877cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:56.299 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.293+0000 7fec01d9b640 1 --2- 172.21.15.38:0/1551275732 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55756f3ea8d0 0x55756f86e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:56.299 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.293+0000 7fec02d9d640 1 -- 172.21.15.38:0/1551275732 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55756f3ea8d0 msgr2=0x55756f86e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:56.299 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.293+0000 7fec02d9d640 1 --2- 172.21.15.38:0/1551275732 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55756f3ea8d0 0x55756f86e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:56.299 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.293+0000 7fec02d9d640 1 -- 172.21.15.38:0/1551275732 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55756f3e26f0 msgr2=0x55756f877cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:56.299 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.293+0000 7fec02d9d640 1 --2- 172.21.15.38:0/1551275732 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55756f3e26f0 0x55756f877cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:56.299 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.293+0000 7fec02d9d640 1 -- 172.21.15.38:0/1551275732 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55756f4edd20 con 0x55756f40b5b0 2024-09-21T03:34:56.299 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.293+0000 7fec02d9d640 1 --2- 172.21.15.38:0/1551275732 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55756f40b5b0 0x55756f7b09c0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7febfc083a30 tx=0x7febfc07f3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11c9dfa2f1cb6204 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:56.299 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.293+0000 7febe17fa640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-21T03:34:56.299 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.294+0000 7febe17fa640 1 -- 172.21.15.38:0/1551275732 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febfc082d20 con 0x55756f40b5b0 2024-09-21T03:34:56.300 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.294+0000 7fec1002d740 1 -- 172.21.15.38:0/1551275732 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7febf8003d20 con 0x55756f40b5b0 2024-09-21T03:34:56.300 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.294+0000 7febe17fa640 1 -- 172.21.15.38:0/1551275732 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(41..41 src has 1..41) ==== 6187+0+0 (secure 0 0 0) 0x7febfc0820a0 con 0x55756f40b5b0 2024-09-21T03:34:56.300 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.294+0000 7febe17fa640 1 -- 172.21.15.38:0/1551275732 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7febfc08cd60 con 0x55756f40b5b0 2024-09-21T03:34:56.300 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.294+0000 7fec1002d740 10 client.5337 fetch_fsmap learned FSMap version 23 2024-09-21T03:34:56.300 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.294+0000 7fec1002d740 1 -- 172.21.15.38:0/1551275732 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) -- 0x55756f4edd20 con 0x55756f40b5b0 2024-09-21T03:34:56.300 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.294+0000 7febe17fa640 1 -- 172.21.15.38:0/1551275732 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7febf8003d20 con 0x55756f40b5b0 2024-09-21T03:34:56.300 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.294+0000 7febe17fa640 1 -- 172.21.15.38:0/1551275732 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 23) ==== 3502+0+0 (secure 0 0 0) 0x7febfc08c2a0 con 0x55756f40b5b0 2024-09-21T03:34:56.300 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.294+0000 7fec1002d740 10 client.5337 fetch_fsmap finished waiting for FSMap version 23 2024-09-21T03:34:56.300 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.294+0000 7fec1002d740 10 client.5337 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-09-21T03:34:56.300 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.294+0000 7fec1002d740 20 client.5337 populate_metadata read hostname 'smithi038' 2024-09-21T03:34:56.300 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.294+0000 7fec1002d740 1 --2- 172.21.15.38:0/1551275732 >> [v2:172.21.15.38:6832/3533947700,v1:172.21.15.38:6834/3533947700] conn(0x55756f7f8150 0x55756f7d98a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:56.300 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.294+0000 7fec1002d740 4 client.5337 mds_command: new command op to 5226 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/subdir", "scrubops": ["repair"]}] 2024-09-21T03:34:56.300 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.294+0000 7fec1002d740 1 -- 172.21.15.38:0/1551275732 --> [v2:172.21.15.38:6832/3533947700,v1:172.21.15.38:6834/3533947700] -- command(tid 0: {"prefix": "scrub start", "path": "/subdir", "scrubops": ["repair"]}) -- 0x55756f723480 con 0x55756f7f8150 2024-09-21T03:34:56.300 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.294+0000 7fec0259c640 1 --2- 172.21.15.38:0/1551275732 >> [v2:172.21.15.38:6832/3533947700,v1:172.21.15.38:6834/3533947700] conn(0x55756f7f8150 0x55756f7d98a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:56.301 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.295+0000 7febe17fa640 10 client.5337 ms_handle_connect on v2:172.21.15.38:6832/3533947700 2024-09-21T03:34:56.301 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.295+0000 7fec0259c640 1 --2- 172.21.15.38:0/1551275732 >> [v2:172.21.15.38:6832/3533947700,v1:172.21.15.38:6834/3533947700] conn(0x55756f7f8150 0x55756f7d98a0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6aff7ed1abc1930e server_cookie=8bc8a43d140d0b8f in_seq=0 out_seq=0 2024-09-21T03:34:56.301 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.295+0000 7febe17fa640 1 -- 172.21.15.38:0/1551275732 <== mds.0 v2:172.21.15.38:6832/3533947700 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55756f723480 con 0x55756f7f8150 2024-09-21T03:34:56.301 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.295+0000 7febe17fa640 10 client.5337 handle_command_reply: tid=0 2024-09-21T03:34:56.301 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.295+0000 7fec1002d740 1 client.5337 shutdown 2024-09-21T03:34:56.301 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.295+0000 7fec1002d740 1 -- 172.21.15.38:0/1551275732 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55756f40b5b0 msgr2=0x55756f7b09c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:56.301 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.295+0000 7fec1002d740 1 --2- 172.21.15.38:0/1551275732 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55756f40b5b0 0x55756f7b09c0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7febfc083a30 tx=0x7febfc07f3a0 comp rx=0 tx=0).stop 2024-09-21T03:34:56.301 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.296+0000 7fec1002d740 1 -- 172.21.15.38:0/1551275732 shutdown_connections 2024-09-21T03:34:56.301 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.296+0000 7fec1002d740 1 --2- 172.21.15.38:0/1551275732 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55756f40b5b0 0x55756f7b09c0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:56.301 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.296+0000 7fec1002d740 1 --2- 172.21.15.38:0/1551275732 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55756f3ea8d0 0x55756f86e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:56.301 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.296+0000 7fec1002d740 1 --2- 172.21.15.38:0/1551275732 >> [v2:172.21.15.38:6832/3533947700,v1:172.21.15.38:6834/3533947700] conn(0x55756f7f8150 0x55756f7d98a0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:56.301 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.296+0000 7fec1002d740 1 --2- 172.21.15.38:0/1551275732 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55756f3e26f0 0x55756f877cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:56.301 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.296+0000 7febe17fa640 0 client.5337 ms_handle_reset on v2:172.21.15.38:6832/3533947700 2024-09-21T03:34:56.302 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.296+0000 7fec1002d740 1 -- 172.21.15.38:0/1551275732 >> 172.21.15.38:0/1551275732 conn(0x55756f3ff700 msgr2=0x55756f7dd220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:56.302 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.296+0000 7fec1002d740 1 -- 172.21.15.38:0/1551275732 shutdown_connections 2024-09-21T03:34:56.302 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.296+0000 7fec1002d740 1 -- 172.21.15.38:0/1551275732 wait complete. 2024-09-21T03:34:56.302 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.296+0000 7fec1002d740 20 client.5337 trim_cache size 0 max 16384 2024-09-21T03:34:56.302 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-09-21T03:34:56.302 INFO:teuthology.orchestra.run.smithi038.stdout: "return_code": 0, 2024-09-21T03:34:56.302 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_tag": "b01c44cd-ffb1-4d13-99ae-d02cf55036eb", 2024-09-21T03:34:56.302 INFO:teuthology.orchestra.run.smithi038.stdout: "mode": "asynchronous" 2024-09-21T03:34:56.302 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-09-21T03:34:56.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.296+0000 7fec08fe6640 1 -- 172.21.15.38:0/1937122390 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7febe4078b90 msgr2=0x7febe407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:56.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.296+0000 7fec08fe6640 1 --2- 172.21.15.38:0/1937122390 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7febe4078b90 0x7febe407b0a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7febf4005fd0 tx=0x7febf4009500 comp rx=0 tx=0).stop 2024-09-21T03:34:56.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.296+0000 7fec08fe6640 1 -- 172.21.15.38:0/1937122390 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fec041516e0 msgr2=0x7fec0415e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:56.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.296+0000 7fec08fe6640 1 --2- 172.21.15.38:0/1937122390 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fec041516e0 0x7fec0415e3d0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7febfc009a70 tx=0x7febfc034660 comp rx=0 tx=0).stop 2024-09-21T03:34:56.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.296+0000 7fec08fe6640 1 -- 172.21.15.38:0/1937122390 shutdown_connections 2024-09-21T03:34:56.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.296+0000 7fec08fe6640 1 --2- 172.21.15.38:0/1937122390 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fec041516e0 0x7fec0415e3d0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:56.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.296+0000 7fec08fe6640 1 --2- 172.21.15.38:0/1937122390 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7febe4078b90 0x7febe407b0a0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:56.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.296+0000 7fec08fe6640 1 --2- 172.21.15.38:0/1937122390 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fec04151190 0x7fec0415de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:56.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.296+0000 7fec08fe6640 1 --2- 172.21.15.38:0/1937122390 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7febec004d40 0x7fec04150a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:56.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.296+0000 7fec08fe6640 1 -- 172.21.15.38:0/1937122390 >> 172.21.15.38:0/1937122390 conn(0x7fec0406cde0 msgr2=0x7fec041e8560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:56.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.297+0000 7fec08fe6640 1 -- 172.21.15.38:0/1937122390 shutdown_connections 2024-09-21T03:34:56.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.297+0000 7fec08fe6640 1 -- 172.21.15.38:0/1937122390 wait complete. 2024-09-21T03:34:56.309 INFO:tasks.cephfs.test_damage:{ "return_code": 0, "scrub_tag": "b01c44cd-ffb1-4d13-99ae-d02cf55036eb", "mode": "asynchronous" } 2024-09-21T03:34:56.309 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:34:56.310 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.2:0 scrub status 2024-09-21T03:34:56.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.426+0000 7f4b2e59d640 1 -- 172.21.15.38:0/1423014035 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4b2806d2b0 msgr2=0x7f4b281621d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:56.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.426+0000 7f4b2e59d640 1 --2- 172.21.15.38:0/1423014035 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4b2806d2b0 0x7f4b281621d0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f4b24005fd0 tx=0x7f4b2402a4a0 comp rx=0 tx=0).stop 2024-09-21T03:34:56.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.426+0000 7f4b2e59d640 1 -- 172.21.15.38:0/1423014035 shutdown_connections 2024-09-21T03:34:56.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.426+0000 7f4b2e59d640 1 --2- 172.21.15.38:0/1423014035 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4b28162710 0x7f4b280cf300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:56.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.426+0000 7f4b2e59d640 1 --2- 172.21.15.38:0/1423014035 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4b2806d2b0 0x7f4b281621d0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:56.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.426+0000 7f4b2e59d640 1 -- 172.21.15.38:0/1423014035 >> 172.21.15.38:0/1423014035 conn(0x7f4b280d1960 msgr2=0x7f4b280d42e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:56.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.426+0000 7f4b2e59d640 1 -- 172.21.15.38:0/1423014035 shutdown_connections 2024-09-21T03:34:56.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.426+0000 7f4b2e59d640 1 -- 172.21.15.38:0/1423014035 wait complete. 2024-09-21T03:34:56.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.428+0000 7f4b2e59d640 1 Processor -- start 2024-09-21T03:34:56.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.428+0000 7f4b2e59d640 1 -- start start 2024-09-21T03:34:56.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.429+0000 7f4b2e59d640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4b2806d2b0 0x7f4b280ca4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:56.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.429+0000 7f4b2e59d640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4b280cf840 0x7f4b28158130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:56.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.429+0000 7f4b2e59d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4b28162710 0x7f4b281518c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:56.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.429+0000 7f4b2e59d640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f4b28155940 con 0x7f4b28162710 2024-09-21T03:34:56.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.429+0000 7f4b2e59d640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f4b281557c0 con 0x7f4b280cf840 2024-09-21T03:34:56.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.429+0000 7f4b2e59d640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f4b28155ac0 con 0x7f4b2806d2b0 2024-09-21T03:34:56.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.429+0000 7f4b2dd9c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4b28162710 0x7f4b281518c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:56.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.429+0000 7f4b2dd9c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4b28162710 0x7f4b281518c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35660/0 (socket says 172.21.15.38:35660) 2024-09-21T03:34:56.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.429+0000 7f4b2dd9c640 1 -- 172.21.15.38:0/4065226600 learned_addr learned my addr 172.21.15.38:0/4065226600 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:34:56.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.429+0000 7f4b2d59b640 1 --2- 172.21.15.38:0/4065226600 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4b2806d2b0 0x7f4b280ca4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:56.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.429+0000 7f4b2cd9a640 1 --2- 172.21.15.38:0/4065226600 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4b280cf840 0x7f4b28158130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:56.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.430+0000 7f4b2dd9c640 1 -- 172.21.15.38:0/4065226600 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4b2806d2b0 msgr2=0x7f4b280ca4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:56.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.430+0000 7f4b2dd9c640 1 --2- 172.21.15.38:0/4065226600 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4b2806d2b0 0x7f4b280ca4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:56.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.430+0000 7f4b2dd9c640 1 -- 172.21.15.38:0/4065226600 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4b280cf840 msgr2=0x7f4b28158130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:56.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.430+0000 7f4b2dd9c640 1 --2- 172.21.15.38:0/4065226600 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4b280cf840 0x7f4b28158130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:56.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.430+0000 7f4b2dd9c640 1 -- 172.21.15.38:0/4065226600 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b280cac90 con 0x7f4b28162710 2024-09-21T03:34:56.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.430+0000 7f4b2dd9c640 1 --2- 172.21.15.38:0/4065226600 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4b28162710 0x7f4b281518c0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f4b2000b9e0 tx=0x7f4b2000bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:56.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.430+0000 7f4b1dffb640 1 -- 172.21.15.38:0/4065226600 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b20003b70 con 0x7f4b28162710 2024-09-21T03:34:56.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.430+0000 7f4b1dffb640 1 -- 172.21.15.38:0/4065226600 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4b20003e00 con 0x7f4b28162710 2024-09-21T03:34:56.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.430+0000 7f4b2e59d640 1 -- 172.21.15.38:0/4065226600 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b281f6170 con 0x7f4b28162710 2024-09-21T03:34:56.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.430+0000 7f4b1dffb640 1 -- 172.21.15.38:0/4065226600 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b2000cb80 con 0x7f4b28162710 2024-09-21T03:34:56.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.430+0000 7f4b2e59d640 1 -- 172.21.15.38:0/4065226600 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b281f5690 con 0x7f4b28162710 2024-09-21T03:34:56.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.432+0000 7f4b1dffb640 1 -- 172.21.15.38:0/4065226600 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f4b20004000 con 0x7f4b28162710 2024-09-21T03:34:56.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.433+0000 7f4b1dffb640 1 --2- 172.21.15.38:0/4065226600 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4b0c078bc0 0x7f4b0c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:56.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.433+0000 7f4b1dffb640 1 -- 172.21.15.38:0/4065226600 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6187+0+0 (secure 0 0 0) 0x7f4b2009ad70 con 0x7f4b28162710 2024-09-21T03:34:56.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.433+0000 7f4b2d59b640 1 --2- 172.21.15.38:0/4065226600 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4b0c078bc0 0x7f4b0c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:56.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.433+0000 7f4b36845740 1 Processor -- start 2024-09-21T03:34:56.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.434+0000 7f4b36845740 1 -- start start 2024-09-21T03:34:56.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.434+0000 7f4b2d59b640 1 --2- 172.21.15.38:0/4065226600 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4b0c078bc0 0x7f4b0c07b0d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f4b24033000 tx=0x7f4b24004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:56.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.435+0000 7f4b36845740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56303e36f390 0x56303e75bc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:56.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.435+0000 7f4b36845740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56303e3464d0 0x56303e730a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:56.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.435+0000 7f4b36845740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56303e34e6b0 0x56303e730fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:56.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.435+0000 7f4b36845740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x56303e689a60 con 0x56303e36f390 2024-09-21T03:34:56.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.435+0000 7f4b36845740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x56303e678e90 con 0x56303e34e6b0 2024-09-21T03:34:56.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.435+0000 7f4b36845740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x56303e67cca0 con 0x56303e3464d0 2024-09-21T03:34:56.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.435+0000 7f4b2d59b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56303e36f390 0x56303e75bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:56.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.435+0000 7f4b2d59b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56303e36f390 0x56303e75bc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35676/0 (socket says 172.21.15.38:35676) 2024-09-21T03:34:56.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.435+0000 7f4b2d59b640 1 -- 172.21.15.38:0/390671791 learned_addr learned my addr 172.21.15.38:0/390671791 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:34:56.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.435+0000 7f4b2dd9c640 1 --2- 172.21.15.38:0/390671791 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56303e34e6b0 0x56303e730fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:56.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.435+0000 7f4b2cd9a640 1 --2- 172.21.15.38:0/390671791 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56303e3464d0 0x56303e730a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:56.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.435+0000 7f4b2d59b640 1 -- 172.21.15.38:0/390671791 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56303e3464d0 msgr2=0x56303e730a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:56.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.435+0000 7f4b2d59b640 1 --2- 172.21.15.38:0/390671791 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56303e3464d0 0x56303e730a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:56.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.435+0000 7f4b2d59b640 1 -- 172.21.15.38:0/390671791 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56303e34e6b0 msgr2=0x56303e730fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:56.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.435+0000 7f4b2d59b640 1 --2- 172.21.15.38:0/390671791 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56303e34e6b0 0x56303e730fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:56.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.435+0000 7f4b2d59b640 1 -- 172.21.15.38:0/390671791 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56303e4517d0 con 0x56303e36f390 2024-09-21T03:34:56.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.435+0000 7f4b2cd9a640 1 --2- 172.21.15.38:0/390671791 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56303e3464d0 0x56303e730a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:34:56.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.435+0000 7f4b2dd9c640 1 --2- 172.21.15.38:0/390671791 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56303e34e6b0 0x56303e730fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:34:56.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.435+0000 7f4b2d59b640 1 --2- 172.21.15.38:0/390671791 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56303e36f390 0x56303e75bc10 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f4b240099c0 tx=0x7f4b2403d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37b8ef233dd50738 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:56.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.436+0000 7f4b12ffd640 1 -- 172.21.15.38:0/390671791 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b24035040 con 0x56303e36f390 2024-09-21T03:34:56.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.436+0000 7f4b12ffd640 1 -- 172.21.15.38:0/390671791 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4b2404d070 con 0x56303e36f390 2024-09-21T03:34:56.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.436+0000 7f4b12ffd640 1 -- 172.21.15.38:0/390671791 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b24003dd0 con 0x56303e36f390 2024-09-21T03:34:56.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.436+0000 7f4b36845740 1 -- 172.21.15.38:0/390671791 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56303e36f390 msgr2=0x56303e75bc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:56.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.436+0000 7f4b36845740 1 --2- 172.21.15.38:0/390671791 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56303e36f390 0x56303e75bc10 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f4b240099c0 tx=0x7f4b2403d000 comp rx=0 tx=0).stop 2024-09-21T03:34:56.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.436+0000 7f4b36845740 1 -- 172.21.15.38:0/390671791 shutdown_connections 2024-09-21T03:34:56.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.436+0000 7f4b36845740 1 --2- 172.21.15.38:0/390671791 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56303e34e6b0 0x56303e730fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:56.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.436+0000 7f4b36845740 1 --2- 172.21.15.38:0/390671791 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56303e3464d0 0x56303e730a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:56.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.436+0000 7f4b36845740 1 --2- 172.21.15.38:0/390671791 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56303e36f390 0x56303e75bc10 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:56.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.436+0000 7f4b36845740 1 -- 172.21.15.38:0/390671791 >> 172.21.15.38:0/390671791 conn(0x56303e3634e0 msgr2=0x56303e716f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:56.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.436+0000 7f4b36845740 1 -- 172.21.15.38:0/390671791 shutdown_connections 2024-09-21T03:34:56.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.436+0000 7f4b36845740 1 -- 172.21.15.38:0/390671791 wait complete. 2024-09-21T03:34:56.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.437+0000 7f4b36845740 1 -- start start 2024-09-21T03:34:56.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.437+0000 7f4b36845740 1 Processor -- start 2024-09-21T03:34:56.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.439+0000 7f4b36845740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56303e3464d0 0x56303e76ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:56.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.439+0000 7f4b36845740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56303e34e6b0 0x56303e76a900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:56.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.439+0000 7f4b36845740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56303e36f390 0x56303e754e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:56.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.439+0000 7f4b36845740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x56303e3e9740 con 0x56303e34e6b0 2024-09-21T03:34:56.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.439+0000 7f4b36845740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x56303e61e970 con 0x56303e36f390 2024-09-21T03:34:56.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.439+0000 7f4b36845740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x56303e757310 con 0x56303e3464d0 2024-09-21T03:34:56.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.439+0000 7f4b36845740 10 client.0 _ll_register_callbacks cb 0x56303e62d780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:34:56.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.439+0000 7f4b2cd9a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56303e34e6b0 0x56303e76a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:56.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.439+0000 7f4b2cd9a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56303e34e6b0 0x56303e76a900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35688/0 (socket says 172.21.15.38:35688) 2024-09-21T03:34:56.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.439+0000 7f4b2cd9a640 1 -- 172.21.15.38:0/428223581 learned_addr learned my addr 172.21.15.38:0/428223581 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:34:56.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.439+0000 7f4b2d59b640 1 --2- 172.21.15.38:0/428223581 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56303e3464d0 0x56303e76ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:56.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.439+0000 7f4b2dd9c640 1 --2- 172.21.15.38:0/428223581 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56303e36f390 0x56303e754e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:56.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.439+0000 7f4b2cd9a640 1 -- 172.21.15.38:0/428223581 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56303e3464d0 msgr2=0x56303e76ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:56.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.439+0000 7f4b2cd9a640 1 --2- 172.21.15.38:0/428223581 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56303e3464d0 0x56303e76ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:56.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.439+0000 7f4b2cd9a640 1 -- 172.21.15.38:0/428223581 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56303e36f390 msgr2=0x56303e754e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:56.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.439+0000 7f4b2cd9a640 1 --2- 172.21.15.38:0/428223581 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56303e36f390 0x56303e754e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:56.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.439+0000 7f4b2cd9a640 1 -- 172.21.15.38:0/428223581 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56303e3da970 con 0x56303e34e6b0 2024-09-21T03:34:56.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.440+0000 7f4b2cd9a640 1 --2- 172.21.15.38:0/428223581 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56303e34e6b0 0x56303e76a900 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f4b18012700 tx=0x7f4b18012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36c52f1d3c1c7282 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:56.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.440+0000 7f4b07fff640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-21T03:34:56.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.440+0000 7f4b07fff640 1 -- 172.21.15.38:0/428223581 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b18010dc0 con 0x56303e34e6b0 2024-09-21T03:34:56.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.440+0000 7f4b07fff640 1 -- 172.21.15.38:0/428223581 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(41..41 src has 1..41) ==== 6187+0+0 (secure 0 0 0) 0x7f4b18004020 con 0x56303e34e6b0 2024-09-21T03:34:56.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.440+0000 7f4b36845740 1 -- 172.21.15.38:0/428223581 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b2404d070 con 0x56303e34e6b0 2024-09-21T03:34:56.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.440+0000 7f4b07fff640 1 -- 172.21.15.38:0/428223581 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f4b18023050 con 0x56303e34e6b0 2024-09-21T03:34:56.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.440+0000 7f4b36845740 10 client.5349 fetch_fsmap learned FSMap version 23 2024-09-21T03:34:56.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.440+0000 7f4b36845740 1 -- 172.21.15.38:0/428223581 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) -- 0x56303e3b86a0 con 0x56303e34e6b0 2024-09-21T03:34:56.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.440+0000 7f4b07fff640 1 -- 172.21.15.38:0/428223581 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4b2404d070 con 0x56303e34e6b0 2024-09-21T03:34:56.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.440+0000 7f4b07fff640 1 -- 172.21.15.38:0/428223581 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 23) ==== 3502+0+0 (secure 0 0 0) 0x7f4b18026020 con 0x56303e34e6b0 2024-09-21T03:34:56.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.440+0000 7f4b36845740 10 client.5349 fetch_fsmap finished waiting for FSMap version 23 2024-09-21T03:34:56.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.440+0000 7f4b36845740 10 client.5349 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-09-21T03:34:56.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.440+0000 7f4b36845740 20 client.5349 populate_metadata read hostname 'smithi038' 2024-09-21T03:34:56.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.440+0000 7f4b36845740 1 --2- 172.21.15.38:0/428223581 >> [v2:172.21.15.38:6832/3533947700,v1:172.21.15.38:6834/3533947700] conn(0x56303e32df60 0x56303e756150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:56.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.440+0000 7f4b36845740 4 client.5349 mds_command: new command op to 5226 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:34:56.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.440+0000 7f4b36845740 1 -- 172.21.15.38:0/428223581 --> [v2:172.21.15.38:6832/3533947700,v1:172.21.15.38:6834/3533947700] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56303e68b9b0 con 0x56303e32df60 2024-09-21T03:34:56.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.440+0000 7f4b2d59b640 1 --2- 172.21.15.38:0/428223581 >> [v2:172.21.15.38:6832/3533947700,v1:172.21.15.38:6834/3533947700] conn(0x56303e32df60 0x56303e756150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:56.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.441+0000 7f4b07fff640 10 client.5349 ms_handle_connect on v2:172.21.15.38:6832/3533947700 2024-09-21T03:34:56.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.441+0000 7f4b2d59b640 1 --2- 172.21.15.38:0/428223581 >> [v2:172.21.15.38:6832/3533947700,v1:172.21.15.38:6834/3533947700] conn(0x56303e32df60 0x56303e756150 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=692e59ba1f51d1e6 server_cookie=461ee6b714ffa46c in_seq=0 out_seq=0 2024-09-21T03:34:56.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.441+0000 7f4b07fff640 1 -- 172.21.15.38:0/428223581 <== mds.0 v2:172.21.15.38:6832/3533947700 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56303e68b9b0 con 0x56303e32df60 2024-09-21T03:34:56.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.441+0000 7f4b07fff640 10 client.5349 handle_command_reply: tid=0 2024-09-21T03:34:56.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.441+0000 7f4b36845740 1 client.5349 shutdown 2024-09-21T03:34:56.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.442+0000 7f4b36845740 1 -- 172.21.15.38:0/428223581 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56303e34e6b0 msgr2=0x56303e76a900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:56.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.442+0000 7f4b36845740 1 --2- 172.21.15.38:0/428223581 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56303e34e6b0 0x56303e76a900 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f4b18012700 tx=0x7f4b18012bc0 comp rx=0 tx=0).stop 2024-09-21T03:34:56.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.442+0000 7f4b36845740 1 -- 172.21.15.38:0/428223581 shutdown_connections 2024-09-21T03:34:56.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.442+0000 7f4b36845740 1 --2- 172.21.15.38:0/428223581 >> [v2:172.21.15.38:6832/3533947700,v1:172.21.15.38:6834/3533947700] conn(0x56303e32df60 0x56303e756150 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:56.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.442+0000 7f4b36845740 1 --2- 172.21.15.38:0/428223581 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56303e36f390 0x56303e754e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:56.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.442+0000 7f4b36845740 1 --2- 172.21.15.38:0/428223581 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56303e34e6b0 0x56303e76a900 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:56.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.442+0000 7f4b36845740 1 --2- 172.21.15.38:0/428223581 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56303e3464d0 0x56303e76ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:56.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.442+0000 7f4b07fff640 0 client.5349 ms_handle_reset on v2:172.21.15.38:6832/3533947700 2024-09-21T03:34:56.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.442+0000 7f4b36845740 1 -- 172.21.15.38:0/428223581 >> 172.21.15.38:0/428223581 conn(0x56303e3634e0 msgr2=0x56303e716ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:56.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.442+0000 7f4b36845740 1 -- 172.21.15.38:0/428223581 shutdown_connections 2024-09-21T03:34:56.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.442+0000 7f4b36845740 1 -- 172.21.15.38:0/428223581 wait complete. 2024-09-21T03:34:56.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.442+0000 7f4b36845740 20 client.5349 trim_cache size 0 max 16384 2024-09-21T03:34:56.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.460+0000 7f4b36845740 1 Processor -- start 2024-09-21T03:34:56.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.460+0000 7f4b36845740 1 -- start start 2024-09-21T03:34:56.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.461+0000 7f4b36845740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56303e3464d0 0x56303e756950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:56.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.461+0000 7f4b36845740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56303e34e6b0 0x56303e77ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:56.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.461+0000 7f4b36845740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56303e36f390 0x56303e77f390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:56.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.461+0000 7f4b36845740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f4b282025e0 con 0x56303e3464d0 2024-09-21T03:34:56.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.461+0000 7f4b36845740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f4b282015e0 con 0x56303e34e6b0 2024-09-21T03:34:56.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.461+0000 7f4b36845740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f4b281f9690 con 0x56303e36f390 2024-09-21T03:34:56.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.461+0000 7f4b2d59b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56303e3464d0 0x56303e756950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:56.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.461+0000 7f4b2d59b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56303e3464d0 0x56303e756950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35704/0 (socket says 172.21.15.38:35704) 2024-09-21T03:34:56.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.461+0000 7f4b2d59b640 1 -- 172.21.15.38:0/150638936 learned_addr learned my addr 172.21.15.38:0/150638936 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:34:56.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.461+0000 7f4b2d59b640 1 -- 172.21.15.38:0/150638936 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56303e36f390 msgr2=0x56303e77f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:56.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.461+0000 7f4b2d59b640 1 --2- 172.21.15.38:0/150638936 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56303e36f390 0x56303e77f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:56.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.461+0000 7f4b2d59b640 1 -- 172.21.15.38:0/150638936 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56303e34e6b0 msgr2=0x56303e77ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:56.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.461+0000 7f4b2d59b640 1 --2- 172.21.15.38:0/150638936 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56303e34e6b0 0x56303e77ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:56.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.461+0000 7f4b2d59b640 1 -- 172.21.15.38:0/150638936 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56303e756e90 con 0x56303e3464d0 2024-09-21T03:34:56.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.462+0000 7f4b2d59b640 1 --2- 172.21.15.38:0/150638936 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56303e3464d0 0x56303e756950 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f4b2404a040 tx=0x7f4b240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=341365d77ab96f82 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:56.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.462+0000 7f4b127fc640 1 -- 172.21.15.38:0/150638936 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b24061070 con 0x56303e3464d0 2024-09-21T03:34:56.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.462+0000 7f4b127fc640 1 -- 172.21.15.38:0/150638936 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4b2404d040 con 0x56303e3464d0 2024-09-21T03:34:56.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.462+0000 7f4b127fc640 1 -- 172.21.15.38:0/150638936 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b24040040 con 0x56303e3464d0 2024-09-21T03:34:56.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.462+0000 7f4b36845740 1 -- 172.21.15.38:0/150638936 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56303e3464d0 msgr2=0x56303e756950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:56.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.462+0000 7f4b36845740 1 --2- 172.21.15.38:0/150638936 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56303e3464d0 0x56303e756950 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f4b2404a040 tx=0x7f4b240046c0 comp rx=0 tx=0).stop 2024-09-21T03:34:56.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.463+0000 7f4b36845740 1 -- 172.21.15.38:0/150638936 shutdown_connections 2024-09-21T03:34:56.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.463+0000 7f4b36845740 1 --2- 172.21.15.38:0/150638936 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56303e36f390 0x56303e77f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:56.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.463+0000 7f4b36845740 1 --2- 172.21.15.38:0/150638936 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56303e34e6b0 0x56303e77ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:56.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.463+0000 7f4b36845740 1 --2- 172.21.15.38:0/150638936 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56303e3464d0 0x56303e756950 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:56.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.463+0000 7f4b36845740 1 -- 172.21.15.38:0/150638936 >> 172.21.15.38:0/150638936 conn(0x56303e3634e0 msgr2=0x56303e7574a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:56.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.463+0000 7f4b36845740 1 -- 172.21.15.38:0/150638936 shutdown_connections 2024-09-21T03:34:56.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.463+0000 7f4b36845740 1 -- 172.21.15.38:0/150638936 wait complete. 2024-09-21T03:34:56.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.463+0000 7f4b36845740 1 -- start start 2024-09-21T03:34:56.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.464+0000 7f4b36845740 1 Processor -- start 2024-09-21T03:34:56.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.465+0000 7f4b36845740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56303e3464d0 0x56303e754c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:56.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.465+0000 7f4b36845740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56303e34e6b0 0x56303e7ddaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:56.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.465+0000 7f4b36845740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56303e36f390 0x56303e72e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:56.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.465+0000 7f4b36845740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f4b28200fc0 con 0x56303e3464d0 2024-09-21T03:34:56.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.465+0000 7f4b36845740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x56303e7126d0 con 0x56303e36f390 2024-09-21T03:34:56.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.465+0000 7f4b36845740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x56303e72ef70 con 0x56303e34e6b0 2024-09-21T03:34:56.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.465+0000 7f4b36845740 10 client.0 _ll_register_callbacks cb 0x7f4b1800a500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:34:56.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.465+0000 7f4b2d59b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56303e3464d0 0x56303e754c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:56.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.465+0000 7f4b2d59b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56303e3464d0 0x56303e754c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35718/0 (socket says 172.21.15.38:35718) 2024-09-21T03:34:56.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.465+0000 7f4b2d59b640 1 -- 172.21.15.38:0/2181396005 learned_addr learned my addr 172.21.15.38:0/2181396005 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:34:56.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.465+0000 7f4b2cd9a640 1 --2- 172.21.15.38:0/2181396005 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56303e34e6b0 0x56303e7ddaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:56.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.465+0000 7f4b2dd9c640 1 --2- 172.21.15.38:0/2181396005 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56303e36f390 0x56303e72e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:56.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.465+0000 7f4b2d59b640 1 -- 172.21.15.38:0/2181396005 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56303e34e6b0 msgr2=0x56303e7ddaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:56.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.465+0000 7f4b2d59b640 1 --2- 172.21.15.38:0/2181396005 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56303e34e6b0 0x56303e7ddaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:56.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.465+0000 7f4b2d59b640 1 -- 172.21.15.38:0/2181396005 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56303e36f390 msgr2=0x56303e72e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:56.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.465+0000 7f4b2d59b640 1 --2- 172.21.15.38:0/2181396005 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56303e36f390 0x56303e72e3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:56.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.465+0000 7f4b2d59b640 1 -- 172.21.15.38:0/2181396005 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56303e7551a0 con 0x56303e3464d0 2024-09-21T03:34:56.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.465+0000 7f4b2dd9c640 1 --2- 172.21.15.38:0/2181396005 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56303e36f390 0x56303e72e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:34:56.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.466+0000 7f4b2d59b640 1 --2- 172.21.15.38:0/2181396005 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56303e3464d0 0x56303e754c60 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f4b240039c0 tx=0x7f4b2402ac20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d05bf64fc9a38957 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:56.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.466+0000 7f4b07fff640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-21T03:34:56.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.466+0000 7f4b07fff640 1 -- 172.21.15.38:0/2181396005 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b24040070 con 0x56303e3464d0 2024-09-21T03:34:56.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.466+0000 7f4b07fff640 1 -- 172.21.15.38:0/2181396005 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(41..41 src has 1..41) ==== 6187+0+0 (secure 0 0 0) 0x7f4b240049b0 con 0x56303e3464d0 2024-09-21T03:34:56.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.466+0000 7f4b36845740 1 -- 172.21.15.38:0/2181396005 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b2404d040 con 0x56303e3464d0 2024-09-21T03:34:56.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.466+0000 7f4b07fff640 1 -- 172.21.15.38:0/2181396005 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f4b24073050 con 0x56303e3464d0 2024-09-21T03:34:56.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.466+0000 7f4b36845740 10 client.5355 fetch_fsmap learned FSMap version 23 2024-09-21T03:34:56.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.466+0000 7f4b36845740 1 -- 172.21.15.38:0/2181396005 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) -- 0x56303e7ddfe0 con 0x56303e3464d0 2024-09-21T03:34:56.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.466+0000 7f4b07fff640 1 -- 172.21.15.38:0/2181396005 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4b2404d040 con 0x56303e3464d0 2024-09-21T03:34:56.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.466+0000 7f4b07fff640 1 -- 172.21.15.38:0/2181396005 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 23) ==== 3502+0+0 (secure 0 0 0) 0x7f4b24076020 con 0x56303e3464d0 2024-09-21T03:34:56.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.466+0000 7f4b36845740 10 client.5355 fetch_fsmap finished waiting for FSMap version 23 2024-09-21T03:34:56.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.466+0000 7f4b36845740 10 client.5355 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-09-21T03:34:56.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.466+0000 7f4b36845740 20 client.5355 populate_metadata read hostname 'smithi038' 2024-09-21T03:34:56.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.466+0000 7f4b36845740 1 --2- 172.21.15.38:0/2181396005 >> [v2:172.21.15.38:6832/3533947700,v1:172.21.15.38:6834/3533947700] conn(0x56303e756250 0x56303e72e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:56.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.466+0000 7f4b36845740 4 client.5355 mds_command: new command op to 5226 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T03:34:56.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.466+0000 7f4b36845740 1 -- 172.21.15.38:0/2181396005 --> [v2:172.21.15.38:6832/3533947700,v1:172.21.15.38:6834/3533947700] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56303e687460 con 0x56303e756250 2024-09-21T03:34:56.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.467+0000 7f4b2cd9a640 1 --2- 172.21.15.38:0/2181396005 >> [v2:172.21.15.38:6832/3533947700,v1:172.21.15.38:6834/3533947700] conn(0x56303e756250 0x56303e72e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:56.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.467+0000 7f4b07fff640 10 client.5355 ms_handle_connect on v2:172.21.15.38:6832/3533947700 2024-09-21T03:34:56.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.467+0000 7f4b2cd9a640 1 --2- 172.21.15.38:0/2181396005 >> [v2:172.21.15.38:6832/3533947700,v1:172.21.15.38:6834/3533947700] conn(0x56303e756250 0x56303e72e910 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=30e1d9fd33d62f92 server_cookie=a5640fdb030c5d25 in_seq=0 out_seq=0 2024-09-21T03:34:56.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.467+0000 7f4b07fff640 1 -- 172.21.15.38:0/2181396005 <== mds.0 v2:172.21.15.38:6832/3533947700 1 ==== command_reply(tid 0: 0 ) ==== 8+0+573 (crc 0 0 0) 0x56303e687460 con 0x56303e756250 2024-09-21T03:34:56.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.467+0000 7f4b07fff640 10 client.5355 handle_command_reply: tid=0 2024-09-21T03:34:56.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.467+0000 7f4b36845740 1 client.5355 shutdown 2024-09-21T03:34:56.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.467+0000 7f4b36845740 1 -- 172.21.15.38:0/2181396005 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56303e3464d0 msgr2=0x56303e754c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:56.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.467+0000 7f4b36845740 1 --2- 172.21.15.38:0/2181396005 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56303e3464d0 0x56303e754c60 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f4b240039c0 tx=0x7f4b2402ac20 comp rx=0 tx=0).stop 2024-09-21T03:34:56.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.467+0000 7f4b36845740 1 -- 172.21.15.38:0/2181396005 shutdown_connections 2024-09-21T03:34:56.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.467+0000 7f4b36845740 1 --2- 172.21.15.38:0/2181396005 >> [v2:172.21.15.38:6832/3533947700,v1:172.21.15.38:6834/3533947700] conn(0x56303e756250 0x56303e72e910 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:56.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.467+0000 7f4b36845740 1 --2- 172.21.15.38:0/2181396005 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56303e36f390 0x56303e72e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:56.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.467+0000 7f4b07fff640 0 client.5355 ms_handle_reset on v2:172.21.15.38:6832/3533947700 2024-09-21T03:34:56.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.467+0000 7f4b36845740 1 --2- 172.21.15.38:0/2181396005 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56303e34e6b0 0x56303e7ddaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:56.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.467+0000 7f4b36845740 1 --2- 172.21.15.38:0/2181396005 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56303e3464d0 0x56303e754c60 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:56.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.467+0000 7f4b36845740 1 -- 172.21.15.38:0/2181396005 >> 172.21.15.38:0/2181396005 conn(0x56303e3634e0 msgr2=0x56303e716bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:56.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.467+0000 7f4b36845740 1 -- 172.21.15.38:0/2181396005 shutdown_connections 2024-09-21T03:34:56.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.467+0000 7f4b36845740 1 -- 172.21.15.38:0/2181396005 wait complete. 2024-09-21T03:34:56.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.468+0000 7f4b36845740 20 client.5355 trim_cache size 0 max 16384 2024-09-21T03:34:56.474 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-09-21T03:34:56.475 INFO:teuthology.orchestra.run.smithi038.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T03:34:56.475 INFO:teuthology.orchestra.run.smithi038.stdout: "scrubs": { 2024-09-21T03:34:56.475 INFO:teuthology.orchestra.run.smithi038.stdout: "b01c44cd-ffb1-4d13-99ae-d02cf55036eb": { 2024-09-21T03:34:56.475 INFO:teuthology.orchestra.run.smithi038.stdout: "path": "/subdir", 2024-09-21T03:34:56.475 INFO:teuthology.orchestra.run.smithi038.stdout: "tag": "b01c44cd-ffb1-4d13-99ae-d02cf55036eb", 2024-09-21T03:34:56.475 INFO:teuthology.orchestra.run.smithi038.stdout: "options": "repair" 2024-09-21T03:34:56.475 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-21T03:34:56.475 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-21T03:34:56.475 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_stats": { 2024-09-21T03:34:56.475 INFO:teuthology.orchestra.run.smithi038.stdout: "b01c44cd-ffb1-4d13-99ae-d02cf55036eb": { 2024-09-21T03:34:56.476 INFO:teuthology.orchestra.run.smithi038.stdout: "start_time": "2024-09-21T03:34:56.295808+0000", 2024-09-21T03:34:56.476 INFO:teuthology.orchestra.run.smithi038.stdout: "path": "/subdir", 2024-09-21T03:34:56.476 INFO:teuthology.orchestra.run.smithi038.stdout: "uninline_started": 0, 2024-09-21T03:34:56.476 INFO:teuthology.orchestra.run.smithi038.stdout: "uninline_passed": 0, 2024-09-21T03:34:56.476 INFO:teuthology.orchestra.run.smithi038.stdout: "uninline_failed": 0, 2024-09-21T03:34:56.476 INFO:teuthology.orchestra.run.smithi038.stdout: "uninline_skipped": 0 2024-09-21T03:34:56.476 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-21T03:34:56.476 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-21T03:34:56.476 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-09-21T03:34:56.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.468+0000 7f4b2e59d640 1 -- 172.21.15.38:0/4065226600 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4b0c078bc0 msgr2=0x7f4b0c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:56.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.468+0000 7f4b2e59d640 1 --2- 172.21.15.38:0/4065226600 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4b0c078bc0 0x7f4b0c07b0d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f4b24033000 tx=0x7f4b24004730 comp rx=0 tx=0).stop 2024-09-21T03:34:56.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.468+0000 7f4b2e59d640 1 -- 172.21.15.38:0/4065226600 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4b28162710 msgr2=0x7f4b281518c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:56.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.468+0000 7f4b2e59d640 1 --2- 172.21.15.38:0/4065226600 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4b28162710 0x7f4b281518c0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f4b2000b9e0 tx=0x7f4b2000bea0 comp rx=0 tx=0).stop 2024-09-21T03:34:56.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.468+0000 7f4b2e59d640 1 -- 172.21.15.38:0/4065226600 shutdown_connections 2024-09-21T03:34:56.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.468+0000 7f4b2e59d640 1 --2- 172.21.15.38:0/4065226600 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4b28162710 0x7f4b281518c0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:56.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.468+0000 7f4b2e59d640 1 --2- 172.21.15.38:0/4065226600 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4b280cf840 0x7f4b28158130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:56.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.468+0000 7f4b2e59d640 1 --2- 172.21.15.38:0/4065226600 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4b0c078bc0 0x7f4b0c07b0d0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:56.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.468+0000 7f4b2e59d640 1 --2- 172.21.15.38:0/4065226600 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4b2806d2b0 0x7f4b280ca4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:56.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.468+0000 7f4b2e59d640 1 -- 172.21.15.38:0/4065226600 >> 172.21.15.38:0/4065226600 conn(0x7f4b280d1960 msgr2=0x7f4b280b8230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:56.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.468+0000 7f4b2e59d640 1 -- 172.21.15.38:0/4065226600 shutdown_connections 2024-09-21T03:34:56.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:34:56.468+0000 7f4b2e59d640 1 -- 172.21.15.38:0/4065226600 wait complete. 2024-09-21T03:34:56.481 INFO:tasks.cephfs.filesystem:scrub status for tag:b01c44cd-ffb1-4d13-99ae-d02cf55036eb - {'path': '/subdir', 'tag': 'b01c44cd-ffb1-4d13-99ae-d02cf55036eb', 'options': 'repair'} 2024-09-21T03:35:00.043 DEBUG:teuthology.orchestra.run.smithi038:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T03:35:00.046 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T03:35:26.482 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:35:26.483 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.2:0 scrub status 2024-09-21T03:35:26.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.614+0000 7f9090de6640 1 -- 172.21.15.38:0/1764859653 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f908c15bfc0 msgr2=0x7f908c15f630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:26.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.614+0000 7f9090de6640 1 --2- 172.21.15.38:0/1764859653 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f908c15bfc0 0x7f908c15f630 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f9084005fd0 tx=0x7f908402a8b0 comp rx=0 tx=0).stop 2024-09-21T03:35:26.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.614+0000 7f9090de6640 1 -- 172.21.15.38:0/1764859653 shutdown_connections 2024-09-21T03:35:26.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.614+0000 7f9090de6640 1 --2- 172.21.15.38:0/1764859653 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f908c15bfc0 0x7f908c15f630 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:26.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.614+0000 7f9090de6640 1 --2- 172.21.15.38:0/1764859653 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f908c0b5260 0x7f908c15ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:26.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.614+0000 7f9090de6640 1 -- 172.21.15.38:0/1764859653 >> 172.21.15.38:0/1764859653 conn(0x7f908c0cdca0 msgr2=0x7f908c0ce0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:26.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.614+0000 7f9090de6640 1 -- 172.21.15.38:0/1764859653 shutdown_connections 2024-09-21T03:35:26.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.614+0000 7f9090de6640 1 -- 172.21.15.38:0/1764859653 wait complete. 2024-09-21T03:35:26.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.616+0000 7f9090de6640 1 Processor -- start 2024-09-21T03:35:26.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.616+0000 7f9090de6640 1 -- start start 2024-09-21T03:35:26.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.617+0000 7f9090de6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f908c06def0 0x7f908c1f6750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:26.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.617+0000 7f9090de6640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f908c0b5260 0x7f908c09e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:26.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.617+0000 7f9090de6640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f908c15bfc0 0x7f908c0cce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:26.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.617+0000 7f9090de6640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f908c15ff40 con 0x7f908c06def0 2024-09-21T03:35:26.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.617+0000 7f9090de6640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f908c15fdc0 con 0x7f908c15bfc0 2024-09-21T03:35:26.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.617+0000 7f9090de6640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f908c071300 con 0x7f908c0b5260 2024-09-21T03:35:26.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.617+0000 7f908a59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f908c06def0 0x7f908c1f6750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:26.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.618+0000 7f908a59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f908c06def0 0x7f908c1f6750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44560/0 (socket says 172.21.15.38:44560) 2024-09-21T03:35:26.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.618+0000 7f908a59c640 1 -- 172.21.15.38:0/858992839 learned_addr learned my addr 172.21.15.38:0/858992839 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:35:26.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.618+0000 7f9089d9b640 1 --2- 172.21.15.38:0/858992839 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f908c0b5260 0x7f908c09e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:26.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.618+0000 7f908ad9d640 1 --2- 172.21.15.38:0/858992839 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f908c15bfc0 0x7f908c0cce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:26.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.618+0000 7f908a59c640 1 -- 172.21.15.38:0/858992839 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f908c0b5260 msgr2=0x7f908c09e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:26.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.618+0000 7f908a59c640 1 --2- 172.21.15.38:0/858992839 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f908c0b5260 0x7f908c09e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:26.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.618+0000 7f908a59c640 1 -- 172.21.15.38:0/858992839 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f908c15bfc0 msgr2=0x7f908c0cce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:26.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.618+0000 7f908a59c640 1 --2- 172.21.15.38:0/858992839 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f908c15bfc0 0x7f908c0cce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:26.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.618+0000 7f908a59c640 1 -- 172.21.15.38:0/858992839 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f908c1f7330 con 0x7f908c06def0 2024-09-21T03:35:26.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.618+0000 7f908a59c640 1 --2- 172.21.15.38:0/858992839 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f908c06def0 0x7f908c1f6750 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f907c00c370 tx=0x7f907c00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:26.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.618+0000 7f907affd640 1 -- 172.21.15.38:0/858992839 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f907c011790 con 0x7f908c06def0 2024-09-21T03:35:26.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.618+0000 7f907affd640 1 -- 172.21.15.38:0/858992839 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f907c017070 con 0x7f908c06def0 2024-09-21T03:35:26.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.618+0000 7f907affd640 1 -- 172.21.15.38:0/858992839 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f907c011ce0 con 0x7f908c06def0 2024-09-21T03:35:26.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.618+0000 7f9090de6640 1 -- 172.21.15.38:0/858992839 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f908c1f3300 con 0x7f908c06def0 2024-09-21T03:35:26.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.618+0000 7f9090de6640 1 -- 172.21.15.38:0/858992839 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f908c1f4e80 con 0x7f908c06def0 2024-09-21T03:35:26.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.620+0000 7f907affd640 1 -- 172.21.15.38:0/858992839 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f907c01c460 con 0x7f908c06def0 2024-09-21T03:35:26.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.621+0000 7f907affd640 1 --2- 172.21.15.38:0/858992839 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f906c078b90 0x7f906c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:26.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.621+0000 7f907affd640 1 -- 172.21.15.38:0/858992839 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6245+0+0 (secure 0 0 0) 0x7f907c09abe0 con 0x7f908c06def0 2024-09-21T03:35:26.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.621+0000 7f9089d9b640 1 --2- 172.21.15.38:0/858992839 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f906c078b90 0x7f906c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:26.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.622+0000 7f9097e2a740 1 Processor -- start 2024-09-21T03:35:26.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.622+0000 7f9097e2a740 1 -- start start 2024-09-21T03:35:26.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.622+0000 7f9089d9b640 1 --2- 172.21.15.38:0/858992839 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f906c078b90 0x7f906c07b0a0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f908c0cb570 tx=0x7f908000b2e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:26.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.623+0000 7f9097e2a740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563e70b3f390 0x563e70f2ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:26.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.623+0000 7f9097e2a740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563e70b164d0 0x563e70f00860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:26.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.623+0000 7f9097e2a740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x563e70b1e6b0 0x563e70f00da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:26.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.623+0000 7f9097e2a740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x563e70e59a60 con 0x563e70b164d0 2024-09-21T03:35:26.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.623+0000 7f9097e2a740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x563e70e48e90 con 0x563e70b3f390 2024-09-21T03:35:26.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.623+0000 7f9097e2a740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x563e70e4cca0 con 0x563e70b1e6b0 2024-09-21T03:35:26.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.623+0000 7f9089d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563e70b164d0 0x563e70f00860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:26.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.623+0000 7f9089d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563e70b164d0 0x563e70f00860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44562/0 (socket says 172.21.15.38:44562) 2024-09-21T03:35:26.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.623+0000 7f9089d9b640 1 -- 172.21.15.38:0/1394244397 learned_addr learned my addr 172.21.15.38:0/1394244397 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:35:26.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.623+0000 7f908ad9d640 1 --2- 172.21.15.38:0/1394244397 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x563e70b1e6b0 0x563e70f00da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:26.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.623+0000 7f908a59c640 1 --2- 172.21.15.38:0/1394244397 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563e70b3f390 0x563e70f2ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:26.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.624+0000 7f908a59c640 1 -- 172.21.15.38:0/1394244397 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x563e70b1e6b0 msgr2=0x563e70f00da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:35:26.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.624+0000 7f908a59c640 1 --2- 172.21.15.38:0/1394244397 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x563e70b1e6b0 0x563e70f00da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:26.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.624+0000 7f908a59c640 1 -- 172.21.15.38:0/1394244397 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563e70b164d0 msgr2=0x563e70f00860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:35:26.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.624+0000 7f908a59c640 1 --2- 172.21.15.38:0/1394244397 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563e70b164d0 0x563e70f00860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:26.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.624+0000 7f908a59c640 1 -- 172.21.15.38:0/1394244397 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e70c217d0 con 0x563e70b3f390 2024-09-21T03:35:26.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.624+0000 7f9089d9b640 1 --2- 172.21.15.38:0/1394244397 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563e70b164d0 0x563e70f00860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:35:26.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.624+0000 7f908a59c640 1 --2- 172.21.15.38:0/1394244397 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563e70b3f390 0x563e70f2ba10 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f907c098e20 tx=0x7f907c0621e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=62f5766ab68e6809 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:26.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.624+0000 7f906bfff640 1 -- 172.21.15.38:0/1394244397 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f907c084e60 con 0x563e70b3f390 2024-09-21T03:35:26.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.624+0000 7f906bfff640 1 -- 172.21.15.38:0/1394244397 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f907c0671d0 con 0x563e70b3f390 2024-09-21T03:35:26.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.624+0000 7f906bfff640 1 -- 172.21.15.38:0/1394244397 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f907c061c00 con 0x563e70b3f390 2024-09-21T03:35:26.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.625+0000 7f9097e2a740 1 -- 172.21.15.38:0/1394244397 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563e70b3f390 msgr2=0x563e70f2ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:26.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.625+0000 7f9097e2a740 1 --2- 172.21.15.38:0/1394244397 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563e70b3f390 0x563e70f2ba10 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f907c098e20 tx=0x7f907c0621e0 comp rx=0 tx=0).stop 2024-09-21T03:35:26.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.625+0000 7f9097e2a740 1 -- 172.21.15.38:0/1394244397 shutdown_connections 2024-09-21T03:35:26.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.625+0000 7f9097e2a740 1 --2- 172.21.15.38:0/1394244397 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x563e70b1e6b0 0x563e70f00da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:26.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.625+0000 7f9097e2a740 1 --2- 172.21.15.38:0/1394244397 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563e70b164d0 0x563e70f00860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:26.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.625+0000 7f9097e2a740 1 --2- 172.21.15.38:0/1394244397 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563e70b3f390 0x563e70f2ba10 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:26.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.625+0000 7f9097e2a740 1 -- 172.21.15.38:0/1394244397 >> 172.21.15.38:0/1394244397 conn(0x563e70b334e0 msgr2=0x563e70ee6d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:26.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.625+0000 7f9097e2a740 1 -- 172.21.15.38:0/1394244397 shutdown_connections 2024-09-21T03:35:26.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.625+0000 7f9097e2a740 1 -- 172.21.15.38:0/1394244397 wait complete. 2024-09-21T03:35:26.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.626+0000 7f9097e2a740 1 -- start start 2024-09-21T03:35:26.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.626+0000 7f9097e2a740 1 Processor -- start 2024-09-21T03:35:26.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.628+0000 7f9097e2a740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x563e70b164d0 0x563e70f25800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:26.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.628+0000 7f9097e2a740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563e70b1e6b0 0x563e70f22870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:26.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.628+0000 7f9097e2a740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563e70b3f390 0x563e70f3bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:26.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.628+0000 7f9097e2a740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x563e70bb9740 con 0x563e70b1e6b0 2024-09-21T03:35:26.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.628+0000 7f9097e2a740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x563e70dee970 con 0x563e70b3f390 2024-09-21T03:35:26.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.628+0000 7f9097e2a740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x563e70f3ca60 con 0x563e70b164d0 2024-09-21T03:35:26.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.628+0000 7f9097e2a740 10 client.0 _ll_register_callbacks cb 0x563e70dfd780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:35:26.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.628+0000 7f9089d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563e70b1e6b0 0x563e70f22870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:26.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.628+0000 7f9089d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563e70b1e6b0 0x563e70f22870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44568/0 (socket says 172.21.15.38:44568) 2024-09-21T03:35:26.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.628+0000 7f9089d9b640 1 -- 172.21.15.38:0/1030201570 learned_addr learned my addr 172.21.15.38:0/1030201570 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:35:26.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.628+0000 7f908ad9d640 1 --2- 172.21.15.38:0/1030201570 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563e70b3f390 0x563e70f3bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:26.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.628+0000 7f908a59c640 1 --2- 172.21.15.38:0/1030201570 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x563e70b164d0 0x563e70f25800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:26.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.628+0000 7f9089d9b640 1 -- 172.21.15.38:0/1030201570 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x563e70b164d0 msgr2=0x563e70f25800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:35:26.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.628+0000 7f9089d9b640 1 --2- 172.21.15.38:0/1030201570 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x563e70b164d0 0x563e70f25800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:26.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.628+0000 7f9089d9b640 1 -- 172.21.15.38:0/1030201570 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563e70b3f390 msgr2=0x563e70f3bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:35:26.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.628+0000 7f9089d9b640 1 --2- 172.21.15.38:0/1030201570 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563e70b3f390 0x563e70f3bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:26.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.628+0000 7f9089d9b640 1 -- 172.21.15.38:0/1030201570 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e70baa970 con 0x563e70b1e6b0 2024-09-21T03:35:26.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.628+0000 7f908ad9d640 1 --2- 172.21.15.38:0/1030201570 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563e70b3f390 0x563e70f3bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:35:26.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.628+0000 7f9089d9b640 1 --2- 172.21.15.38:0/1030201570 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563e70b1e6b0 0x563e70f22870 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f90800116f0 tx=0x7f9080004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=232e94a48187cc70 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:26.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.628+0000 7f90697fa640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-21T03:35:26.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.629+0000 7f90697fa640 1 -- 172.21.15.38:0/1030201570 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f908001fa00 con 0x563e70b1e6b0 2024-09-21T03:35:26.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.629+0000 7f90697fa640 1 -- 172.21.15.38:0/1030201570 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(43..43 src has 1..43) ==== 6245+0+0 (secure 0 0 0) 0x7f9080029d10 con 0x563e70b1e6b0 2024-09-21T03:35:26.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.629+0000 7f9097e2a740 1 -- 172.21.15.38:0/1030201570 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f907c0671d0 con 0x563e70b1e6b0 2024-09-21T03:35:26.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.629+0000 7f90697fa640 1 -- 172.21.15.38:0/1030201570 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f908001fba0 con 0x563e70b1e6b0 2024-09-21T03:35:26.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.629+0000 7f9097e2a740 10 client.5370 fetch_fsmap learned FSMap version 23 2024-09-21T03:35:26.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.629+0000 7f9097e2a740 1 -- 172.21.15.38:0/1030201570 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) -- 0x563e70b886a0 con 0x563e70b1e6b0 2024-09-21T03:35:26.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.629+0000 7f90697fa640 1 -- 172.21.15.38:0/1030201570 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f907c0671d0 con 0x563e70b1e6b0 2024-09-21T03:35:26.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.629+0000 7f90697fa640 1 -- 172.21.15.38:0/1030201570 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 23) ==== 3502+0+0 (secure 0 0 0) 0x7f9080030050 con 0x563e70b1e6b0 2024-09-21T03:35:26.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.629+0000 7f9097e2a740 10 client.5370 fetch_fsmap finished waiting for FSMap version 23 2024-09-21T03:35:26.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.629+0000 7f9097e2a740 10 client.5370 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-09-21T03:35:26.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.629+0000 7f9097e2a740 20 client.5370 populate_metadata read hostname 'smithi038' 2024-09-21T03:35:26.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.629+0000 7f9097e2a740 1 --2- 172.21.15.38:0/1030201570 >> [v2:172.21.15.38:6832/3533947700,v1:172.21.15.38:6834/3533947700] conn(0x563e70afdf60 0x563e70f3b390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:26.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.629+0000 7f9097e2a740 4 client.5370 mds_command: new command op to 5226 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:35:26.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.629+0000 7f9097e2a740 1 -- 172.21.15.38:0/1030201570 --> [v2:172.21.15.38:6832/3533947700,v1:172.21.15.38:6834/3533947700] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563e70e5b9b0 con 0x563e70afdf60 2024-09-21T03:35:26.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.629+0000 7f908a59c640 1 --2- 172.21.15.38:0/1030201570 >> [v2:172.21.15.38:6832/3533947700,v1:172.21.15.38:6834/3533947700] conn(0x563e70afdf60 0x563e70f3b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:26.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.630+0000 7f90697fa640 10 client.5370 ms_handle_connect on v2:172.21.15.38:6832/3533947700 2024-09-21T03:35:26.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.630+0000 7f908a59c640 1 --2- 172.21.15.38:0/1030201570 >> [v2:172.21.15.38:6832/3533947700,v1:172.21.15.38:6834/3533947700] conn(0x563e70afdf60 0x563e70f3b390 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a053b7c32f3e9903 server_cookie=e1dca84db3ec08b7 in_seq=0 out_seq=0 2024-09-21T03:35:26.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.630+0000 7f90697fa640 1 -- 172.21.15.38:0/1030201570 <== mds.0 v2:172.21.15.38:6832/3533947700 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x563e70e5b9b0 con 0x563e70afdf60 2024-09-21T03:35:26.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.630+0000 7f90697fa640 10 client.5370 handle_command_reply: tid=0 2024-09-21T03:35:26.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.630+0000 7f9097e2a740 1 client.5370 shutdown 2024-09-21T03:35:26.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.631+0000 7f9097e2a740 1 -- 172.21.15.38:0/1030201570 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563e70b1e6b0 msgr2=0x563e70f22870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:26.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.631+0000 7f9097e2a740 1 --2- 172.21.15.38:0/1030201570 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563e70b1e6b0 0x563e70f22870 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f90800116f0 tx=0x7f9080004730 comp rx=0 tx=0).stop 2024-09-21T03:35:26.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.631+0000 7f9097e2a740 1 -- 172.21.15.38:0/1030201570 shutdown_connections 2024-09-21T03:35:26.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.631+0000 7f9097e2a740 1 --2- 172.21.15.38:0/1030201570 >> [v2:172.21.15.38:6832/3533947700,v1:172.21.15.38:6834/3533947700] conn(0x563e70afdf60 0x563e70f3b390 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:26.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.631+0000 7f9097e2a740 1 --2- 172.21.15.38:0/1030201570 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563e70b3f390 0x563e70f3bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:26.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.631+0000 7f9097e2a740 1 --2- 172.21.15.38:0/1030201570 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563e70b1e6b0 0x563e70f22870 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:26.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.631+0000 7f90697fa640 0 client.5370 ms_handle_reset on v2:172.21.15.38:6832/3533947700 2024-09-21T03:35:26.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.631+0000 7f9097e2a740 1 --2- 172.21.15.38:0/1030201570 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x563e70b164d0 0x563e70f25800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:26.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.631+0000 7f9097e2a740 1 -- 172.21.15.38:0/1030201570 >> 172.21.15.38:0/1030201570 conn(0x563e70b334e0 msgr2=0x563e70f007d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:26.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.631+0000 7f9097e2a740 1 -- 172.21.15.38:0/1030201570 shutdown_connections 2024-09-21T03:35:26.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.631+0000 7f9097e2a740 1 -- 172.21.15.38:0/1030201570 wait complete. 2024-09-21T03:35:26.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.631+0000 7f9097e2a740 20 client.5370 trim_cache size 0 max 16384 2024-09-21T03:35:26.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.649+0000 7f9097e2a740 1 Processor -- start 2024-09-21T03:35:26.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.649+0000 7f9097e2a740 1 -- start start 2024-09-21T03:35:26.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.650+0000 7f9097e2a740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563e70b164d0 0x563e70ee6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:26.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.650+0000 7f9097e2a740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x563e70b1e6b0 0x563e70f28130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:26.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.650+0000 7f9097e2a740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563e70b3f390 0x563e70f4eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:26.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.650+0000 7f9097e2a740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f908c2004f0 con 0x563e70b164d0 2024-09-21T03:35:26.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.650+0000 7f9097e2a740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f908c1ff550 con 0x563e70b3f390 2024-09-21T03:35:26.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.650+0000 7f9097e2a740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f908c0b1ac0 con 0x563e70b1e6b0 2024-09-21T03:35:26.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.650+0000 7f908a59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563e70b164d0 0x563e70ee6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:26.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.650+0000 7f908a59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563e70b164d0 0x563e70ee6dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44582/0 (socket says 172.21.15.38:44582) 2024-09-21T03:35:26.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.650+0000 7f908a59c640 1 -- 172.21.15.38:0/3060660662 learned_addr learned my addr 172.21.15.38:0/3060660662 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:35:26.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.651+0000 7f908a59c640 1 -- 172.21.15.38:0/3060660662 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x563e70b1e6b0 msgr2=0x563e70f28130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:35:26.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.651+0000 7f908a59c640 1 --2- 172.21.15.38:0/3060660662 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x563e70b1e6b0 0x563e70f28130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:26.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.651+0000 7f908a59c640 1 -- 172.21.15.38:0/3060660662 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563e70b3f390 msgr2=0x563e70f4eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:35:26.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.651+0000 7f908a59c640 1 --2- 172.21.15.38:0/3060660662 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563e70b3f390 0x563e70f4eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:26.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.651+0000 7f908a59c640 1 -- 172.21.15.38:0/3060660662 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e70ee7c50 con 0x563e70b164d0 2024-09-21T03:35:26.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.651+0000 7f908a59c640 1 --2- 172.21.15.38:0/3060660662 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563e70b164d0 0x563e70ee6dc0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f907c0624a0 tx=0x7f907c065040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81abef9d7752eaef server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:26.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.651+0000 7f906b7fe640 1 -- 172.21.15.38:0/3060660662 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f907c0671d0 con 0x563e70b164d0 2024-09-21T03:35:26.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.651+0000 7f906b7fe640 1 -- 172.21.15.38:0/3060660662 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f907c011e80 con 0x563e70b164d0 2024-09-21T03:35:26.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.651+0000 7f906b7fe640 1 -- 172.21.15.38:0/3060660662 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f907c07ce10 con 0x563e70b164d0 2024-09-21T03:35:26.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.652+0000 7f9097e2a740 1 -- 172.21.15.38:0/3060660662 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563e70b164d0 msgr2=0x563e70ee6dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:26.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.652+0000 7f9097e2a740 1 --2- 172.21.15.38:0/3060660662 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563e70b164d0 0x563e70ee6dc0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f907c0624a0 tx=0x7f907c065040 comp rx=0 tx=0).stop 2024-09-21T03:35:26.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.652+0000 7f9097e2a740 1 -- 172.21.15.38:0/3060660662 shutdown_connections 2024-09-21T03:35:26.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.652+0000 7f9097e2a740 1 --2- 172.21.15.38:0/3060660662 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563e70b3f390 0x563e70f4eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:26.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.652+0000 7f9097e2a740 1 --2- 172.21.15.38:0/3060660662 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x563e70b1e6b0 0x563e70f28130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:26.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.652+0000 7f9097e2a740 1 --2- 172.21.15.38:0/3060660662 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563e70b164d0 0x563e70ee6dc0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:26.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.652+0000 7f9097e2a740 1 -- 172.21.15.38:0/3060660662 >> 172.21.15.38:0/3060660662 conn(0x563e70b334e0 msgr2=0x563e70f109e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:26.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.652+0000 7f9097e2a740 1 -- 172.21.15.38:0/3060660662 shutdown_connections 2024-09-21T03:35:26.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.652+0000 7f9097e2a740 1 -- 172.21.15.38:0/3060660662 wait complete. 2024-09-21T03:35:26.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.653+0000 7f9097e2a740 1 -- start start 2024-09-21T03:35:26.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.653+0000 7f9097e2a740 1 Processor -- start 2024-09-21T03:35:26.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.654+0000 7f9097e2a740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563e70b164d0 0x563e70faf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:26.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.654+0000 7f9097e2a740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x563e70b1e6b0 0x563e70f28130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:26.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.654+0000 7f9097e2a740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563e70b3f390 0x563e70f28670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:26.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.654+0000 7f9097e2a740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x563e70e5b9b0 con 0x563e70b164d0 2024-09-21T03:35:26.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.654+0000 7f9097e2a740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f908c1fef30 con 0x563e70b3f390 2024-09-21T03:35:26.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.654+0000 7f9097e2a740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x563e70ee3860 con 0x563e70b1e6b0 2024-09-21T03:35:26.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.654+0000 7f9097e2a740 10 client.0 _ll_register_callbacks cb 0x7f9080010b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:35:26.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.654+0000 7f908a59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563e70b164d0 0x563e70faf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:26.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.654+0000 7f908a59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563e70b164d0 0x563e70faf280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44596/0 (socket says 172.21.15.38:44596) 2024-09-21T03:35:26.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.654+0000 7f908a59c640 1 -- 172.21.15.38:0/3329648790 learned_addr learned my addr 172.21.15.38:0/3329648790 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:35:26.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.654+0000 7f9089d9b640 1 --2- 172.21.15.38:0/3329648790 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x563e70b1e6b0 0x563e70f28130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:26.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.654+0000 7f908ad9d640 1 --2- 172.21.15.38:0/3329648790 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563e70b3f390 0x563e70f28670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:26.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.654+0000 7f908a59c640 1 -- 172.21.15.38:0/3329648790 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x563e70b1e6b0 msgr2=0x563e70f28130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:35:26.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.654+0000 7f908a59c640 1 --2- 172.21.15.38:0/3329648790 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x563e70b1e6b0 0x563e70f28130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:26.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.654+0000 7f908a59c640 1 -- 172.21.15.38:0/3329648790 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563e70b3f390 msgr2=0x563e70f28670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:35:26.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.654+0000 7f908a59c640 1 --2- 172.21.15.38:0/3329648790 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563e70b3f390 0x563e70f28670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:26.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.654+0000 7f908a59c640 1 -- 172.21.15.38:0/3329648790 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e70fb3080 con 0x563e70b164d0 2024-09-21T03:35:26.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.655+0000 7f9089d9b640 1 --2- 172.21.15.38:0/3329648790 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x563e70b1e6b0 0x563e70f28130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:35:26.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.655+0000 7f908ad9d640 1 --2- 172.21.15.38:0/3329648790 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563e70b3f390 0x563e70f28670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:35:26.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.655+0000 7f908a59c640 1 --2- 172.21.15.38:0/3329648790 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563e70b164d0 0x563e70faf280 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f907c00cd30 tx=0x7f907c05e6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=897081fc4d0c22 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:26.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.655+0000 7f90697fa640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-21T03:35:26.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.655+0000 7f90697fa640 1 -- 172.21.15.38:0/3329648790 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f907c0671d0 con 0x563e70b164d0 2024-09-21T03:35:26.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.655+0000 7f90697fa640 1 -- 172.21.15.38:0/3329648790 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(43..43 src has 1..43) ==== 6245+0+0 (secure 0 0 0) 0x7f907c061c10 con 0x563e70b164d0 2024-09-21T03:35:26.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.655+0000 7f9097e2a740 1 -- 172.21.15.38:0/3329648790 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f907c011e80 con 0x563e70b164d0 2024-09-21T03:35:26.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.655+0000 7f90697fa640 1 -- 172.21.15.38:0/3329648790 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f907c07cbb0 con 0x563e70b164d0 2024-09-21T03:35:26.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.655+0000 7f9097e2a740 10 client.5376 fetch_fsmap learned FSMap version 23 2024-09-21T03:35:26.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.655+0000 7f9097e2a740 1 -- 172.21.15.38:0/3329648790 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) -- 0x563e70fb3290 con 0x563e70b164d0 2024-09-21T03:35:26.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.655+0000 7f90697fa640 1 -- 172.21.15.38:0/3329648790 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f907c011e80 con 0x563e70b164d0 2024-09-21T03:35:26.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.655+0000 7f90697fa640 1 -- 172.21.15.38:0/3329648790 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 23) ==== 3502+0+0 (secure 0 0 0) 0x7f907c0693d0 con 0x563e70b164d0 2024-09-21T03:35:26.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.655+0000 7f9097e2a740 10 client.5376 fetch_fsmap finished waiting for FSMap version 23 2024-09-21T03:35:26.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.655+0000 7f9097e2a740 10 client.5376 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-09-21T03:35:26.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.655+0000 7f9097e2a740 20 client.5376 populate_metadata read hostname 'smithi038' 2024-09-21T03:35:26.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.655+0000 7f9097e2a740 1 --2- 172.21.15.38:0/3329648790 >> [v2:172.21.15.38:6832/3533947700,v1:172.21.15.38:6834/3533947700] conn(0x563e70ee1d70 0x563e70eff190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:26.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.655+0000 7f9097e2a740 4 client.5376 mds_command: new command op to 5226 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T03:35:26.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.655+0000 7f9097e2a740 1 -- 172.21.15.38:0/3329648790 --> [v2:172.21.15.38:6832/3533947700,v1:172.21.15.38:6834/3533947700] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563e70e57460 con 0x563e70ee1d70 2024-09-21T03:35:26.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.655+0000 7f9089d9b640 1 --2- 172.21.15.38:0/3329648790 >> [v2:172.21.15.38:6832/3533947700,v1:172.21.15.38:6834/3533947700] conn(0x563e70ee1d70 0x563e70eff190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:26.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.656+0000 7f90697fa640 10 client.5376 ms_handle_connect on v2:172.21.15.38:6832/3533947700 2024-09-21T03:35:26.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.656+0000 7f9089d9b640 1 --2- 172.21.15.38:0/3329648790 >> [v2:172.21.15.38:6832/3533947700,v1:172.21.15.38:6834/3533947700] conn(0x563e70ee1d70 0x563e70eff190 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3f1de00c888ff0 server_cookie=93210b0df037be82 in_seq=0 out_seq=0 2024-09-21T03:35:26.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.656+0000 7f90697fa640 1 -- 172.21.15.38:0/3329648790 <== mds.0 v2:172.21.15.38:6832/3533947700 1 ==== command_reply(tid 0: 0 ) ==== 8+0+380 (crc 0 0 0) 0x563e70e57460 con 0x563e70ee1d70 2024-09-21T03:35:26.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.656+0000 7f90697fa640 10 client.5376 handle_command_reply: tid=0 2024-09-21T03:35:26.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.656+0000 7f9097e2a740 1 client.5376 shutdown 2024-09-21T03:35:26.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.656+0000 7f9097e2a740 1 -- 172.21.15.38:0/3329648790 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563e70b164d0 msgr2=0x563e70faf280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:26.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.656+0000 7f9097e2a740 1 --2- 172.21.15.38:0/3329648790 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563e70b164d0 0x563e70faf280 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f907c00cd30 tx=0x7f907c05e6d0 comp rx=0 tx=0).stop 2024-09-21T03:35:26.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.656+0000 7f9097e2a740 1 -- 172.21.15.38:0/3329648790 shutdown_connections 2024-09-21T03:35:26.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.656+0000 7f9097e2a740 1 --2- 172.21.15.38:0/3329648790 >> [v2:172.21.15.38:6832/3533947700,v1:172.21.15.38:6834/3533947700] conn(0x563e70ee1d70 0x563e70eff190 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:26.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.656+0000 7f9097e2a740 1 --2- 172.21.15.38:0/3329648790 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563e70b3f390 0x563e70f28670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:26.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.656+0000 7f9097e2a740 1 --2- 172.21.15.38:0/3329648790 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x563e70b1e6b0 0x563e70f28130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:26.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.656+0000 7f9097e2a740 1 --2- 172.21.15.38:0/3329648790 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563e70b164d0 0x563e70faf280 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:26.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.656+0000 7f90697fa640 0 client.5376 ms_handle_reset on v2:172.21.15.38:6832/3533947700 2024-09-21T03:35:26.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.656+0000 7f9097e2a740 1 -- 172.21.15.38:0/3329648790 >> 172.21.15.38:0/3329648790 conn(0x563e70b334e0 msgr2=0x563e70f112f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:26.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.656+0000 7f9097e2a740 1 -- 172.21.15.38:0/3329648790 shutdown_connections 2024-09-21T03:35:26.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.657+0000 7f9097e2a740 1 -- 172.21.15.38:0/3329648790 wait complete. 2024-09-21T03:35:26.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.657+0000 7f9097e2a740 20 client.5376 trim_cache size 0 max 16384 2024-09-21T03:35:26.663 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-09-21T03:35:26.663 INFO:teuthology.orchestra.run.smithi038.stdout: "status": "no active scrubs running", 2024-09-21T03:35:26.663 INFO:teuthology.orchestra.run.smithi038.stdout: "scrubs": {}, 2024-09-21T03:35:26.664 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_stats": { 2024-09-21T03:35:26.664 INFO:teuthology.orchestra.run.smithi038.stdout: "b01c44cd-ffb1-4d13-99ae-d02cf55036eb": { 2024-09-21T03:35:26.664 INFO:teuthology.orchestra.run.smithi038.stdout: "start_time": "2024-09-21T03:34:56.295808+0000", 2024-09-21T03:35:26.664 INFO:teuthology.orchestra.run.smithi038.stdout: "path": "/subdir", 2024-09-21T03:35:26.664 INFO:teuthology.orchestra.run.smithi038.stdout: "uninline_started": 0, 2024-09-21T03:35:26.664 INFO:teuthology.orchestra.run.smithi038.stdout: "uninline_passed": 0, 2024-09-21T03:35:26.664 INFO:teuthology.orchestra.run.smithi038.stdout: "uninline_failed": 0, 2024-09-21T03:35:26.664 INFO:teuthology.orchestra.run.smithi038.stdout: "uninline_skipped": 0 2024-09-21T03:35:26.664 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-21T03:35:26.664 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-21T03:35:26.664 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-09-21T03:35:26.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.657+0000 7f9090de6640 1 -- 172.21.15.38:0/858992839 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f906c078b90 msgr2=0x7f906c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:26.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.657+0000 7f9090de6640 1 --2- 172.21.15.38:0/858992839 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f906c078b90 0x7f906c07b0a0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f908c0cb570 tx=0x7f908000b2e0 comp rx=0 tx=0).stop 2024-09-21T03:35:26.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.657+0000 7f9090de6640 1 -- 172.21.15.38:0/858992839 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f908c06def0 msgr2=0x7f908c1f6750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:26.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.657+0000 7f9090de6640 1 --2- 172.21.15.38:0/858992839 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f908c06def0 0x7f908c1f6750 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f907c00c370 tx=0x7f907c00c830 comp rx=0 tx=0).stop 2024-09-21T03:35:26.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.657+0000 7f9090de6640 1 -- 172.21.15.38:0/858992839 shutdown_connections 2024-09-21T03:35:26.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.657+0000 7f9090de6640 1 --2- 172.21.15.38:0/858992839 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f908c15bfc0 0x7f908c0cce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:26.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.657+0000 7f9090de6640 1 --2- 172.21.15.38:0/858992839 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f906c078b90 0x7f906c07b0a0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:26.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.657+0000 7f9090de6640 1 --2- 172.21.15.38:0/858992839 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f908c0b5260 0x7f908c09e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:26.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.657+0000 7f9090de6640 1 --2- 172.21.15.38:0/858992839 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f908c06def0 0x7f908c1f6750 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:26.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.657+0000 7f9090de6640 1 -- 172.21.15.38:0/858992839 >> 172.21.15.38:0/858992839 conn(0x7f908c0cdca0 msgr2=0x7f908c06cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:26.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.657+0000 7f9090de6640 1 -- 172.21.15.38:0/858992839 shutdown_connections 2024-09-21T03:35:26.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.657+0000 7f9090de6640 1 -- 172.21.15.38:0/858992839 wait complete. 2024-09-21T03:35:26.670 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T03:35:26.670 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T03:35:26.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.785+0000 7fc6d8b90640 1 -- 172.21.15.38:0/1317527955 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc6d4162280 msgr2=0x7fc6d4162690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:26.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.785+0000 7fc6d8b90640 1 --2- 172.21.15.38:0/1317527955 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc6d4162280 0x7fc6d4162690 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fc6c8009830 tx=0x7fc6c802a480 comp rx=0 tx=0).stop 2024-09-21T03:35:26.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.785+0000 7fc6d8b90640 1 -- 172.21.15.38:0/1317527955 shutdown_connections 2024-09-21T03:35:26.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.785+0000 7fc6d8b90640 1 --2- 172.21.15.38:0/1317527955 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc6d4162280 0x7fc6d4162690 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:26.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.785+0000 7fc6d8b90640 1 -- 172.21.15.38:0/1317527955 >> 172.21.15.38:0/1317527955 conn(0x7fc6d40b6b20 msgr2=0x7fc6d40d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:26.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.785+0000 7fc6d8b90640 1 -- 172.21.15.38:0/1317527955 shutdown_connections 2024-09-21T03:35:26.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.785+0000 7fc6d8b90640 1 -- 172.21.15.38:0/1317527955 wait complete. 2024-09-21T03:35:26.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.787+0000 7fc6d8b90640 1 Processor -- start 2024-09-21T03:35:26.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.787+0000 7fc6d8b90640 1 -- start start 2024-09-21T03:35:26.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.788+0000 7fc6d8b90640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc6d40b2e40 0x7fc6d41f89a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:26.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.788+0000 7fc6d8b90640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc6d4162280 0x7fc6d41f8ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:26.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.788+0000 7fc6d8b90640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc6d4158b40 0x7fc6d4158f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:26.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.788+0000 7fc6d8b90640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fc6d415f640 con 0x7fc6d40b2e40 2024-09-21T03:35:26.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.788+0000 7fc6d8b90640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fc6d415f4c0 con 0x7fc6d4158b40 2024-09-21T03:35:26.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.788+0000 7fc6d8b90640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fc6d415f7c0 con 0x7fc6d4162280 2024-09-21T03:35:26.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.788+0000 7fc6d259c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc6d40b2e40 0x7fc6d41f89a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:26.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.788+0000 7fc6d259c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc6d40b2e40 0x7fc6d41f89a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44612/0 (socket says 172.21.15.38:44612) 2024-09-21T03:35:26.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.788+0000 7fc6d259c640 1 -- 172.21.15.38:0/1298494053 learned_addr learned my addr 172.21.15.38:0/1298494053 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:35:26.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.788+0000 7fc6d1d9b640 1 --2- 172.21.15.38:0/1298494053 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc6d4162280 0x7fc6d41f8ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:26.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.788+0000 7fc6d2d9d640 1 --2- 172.21.15.38:0/1298494053 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc6d4158b40 0x7fc6d4158f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:26.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.789+0000 7fc6d259c640 1 -- 172.21.15.38:0/1298494053 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc6d4162280 msgr2=0x7fc6d41f8ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:26.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.789+0000 7fc6d259c640 1 --2- 172.21.15.38:0/1298494053 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc6d4162280 0x7fc6d41f8ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:26.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.789+0000 7fc6d259c640 1 -- 172.21.15.38:0/1298494053 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc6d4158b40 msgr2=0x7fc6d4158f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:26.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.789+0000 7fc6d259c640 1 --2- 172.21.15.38:0/1298494053 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc6d4158b40 0x7fc6d4158f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:26.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.789+0000 7fc6d259c640 1 -- 172.21.15.38:0/1298494053 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6d415c430 con 0x7fc6d40b2e40 2024-09-21T03:35:26.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.789+0000 7fc6d259c640 1 --2- 172.21.15.38:0/1298494053 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc6d40b2e40 0x7fc6d41f89a0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fc6c400c3f0 tx=0x7fc6c400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:26.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.789+0000 7fc6c2ffd640 1 -- 172.21.15.38:0/1298494053 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6c4017070 con 0x7fc6d40b2e40 2024-09-21T03:35:26.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.789+0000 7fc6c2ffd640 1 -- 172.21.15.38:0/1298494053 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc6c40035f0 con 0x7fc6d40b2e40 2024-09-21T03:35:26.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.789+0000 7fc6d8b90640 1 -- 172.21.15.38:0/1298494053 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6d415c710 con 0x7fc6d40b2e40 2024-09-21T03:35:26.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.789+0000 7fc6c2ffd640 1 -- 172.21.15.38:0/1298494053 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6c4003a40 con 0x7fc6d40b2e40 2024-09-21T03:35:26.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.789+0000 7fc6d8b90640 1 -- 172.21.15.38:0/1298494053 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6d415ccd0 con 0x7fc6d40b2e40 2024-09-21T03:35:26.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.790+0000 7fc6d8b90640 1 -- 172.21.15.38:0/1298494053 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6d4162f00 con 0x7fc6d40b2e40 2024-09-21T03:35:26.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.791+0000 7fc6c2ffd640 1 -- 172.21.15.38:0/1298494053 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc6c401a070 con 0x7fc6d40b2e40 2024-09-21T03:35:26.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.792+0000 7fc6c2ffd640 1 --2- 172.21.15.38:0/1298494053 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc6b4078b90 0x7fc6b407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:26.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.792+0000 7fc6c2ffd640 1 -- 172.21.15.38:0/1298494053 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6245+0+0 (secure 0 0 0) 0x7fc6c409a190 con 0x7fc6d40b2e40 2024-09-21T03:35:26.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.792+0000 7fc6d1d9b640 1 --2- 172.21.15.38:0/1298494053 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc6b4078b90 0x7fc6b407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:26.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.793+0000 7fc6d1d9b640 1 --2- 172.21.15.38:0/1298494053 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc6b4078b90 0x7fc6b407b0a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fc6c8034000 tx=0x7fc6c8004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:26.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:26.796+0000 7fc6c2ffd640 1 -- 172.21.15.38:0/1298494053 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fc6d4162bd0 con 0x7fc6d40b2e40 2024-09-21T03:35:27.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:27.026+0000 7fc6d8b90640 1 -- 172.21.15.38:0/1298494053 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fc6d415eff0 con 0x7fc6d40b2e40 2024-09-21T03:35:27.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:27.027+0000 7fc6c2ffd640 1 -- 172.21.15.38:0/1298494053 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v23) ==== 53+0+73 (secure 0 0 0) 0x7fc6c4061df0 con 0x7fc6d40b2e40 2024-09-21T03:35:27.028 INFO:teuthology.orchestra.run.smithi038.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T03:35:27.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:27.029+0000 7fc6d8b90640 1 -- 172.21.15.38:0/1298494053 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc6b4078b90 msgr2=0x7fc6b407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:27.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:27.029+0000 7fc6d8b90640 1 --2- 172.21.15.38:0/1298494053 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc6b4078b90 0x7fc6b407b0a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fc6c8034000 tx=0x7fc6c8004730 comp rx=0 tx=0).stop 2024-09-21T03:35:27.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:27.029+0000 7fc6d8b90640 1 -- 172.21.15.38:0/1298494053 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc6d40b2e40 msgr2=0x7fc6d41f89a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:27.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:27.029+0000 7fc6d8b90640 1 --2- 172.21.15.38:0/1298494053 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc6d40b2e40 0x7fc6d41f89a0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fc6c400c3f0 tx=0x7fc6c400c8b0 comp rx=0 tx=0).stop 2024-09-21T03:35:27.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:27.029+0000 7fc6d8b90640 1 -- 172.21.15.38:0/1298494053 shutdown_connections 2024-09-21T03:35:27.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:27.029+0000 7fc6d8b90640 1 --2- 172.21.15.38:0/1298494053 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc6d4158b40 0x7fc6d4158f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:27.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:27.029+0000 7fc6d8b90640 1 --2- 172.21.15.38:0/1298494053 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc6b4078b90 0x7fc6b407b0a0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:27.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:27.029+0000 7fc6d8b90640 1 --2- 172.21.15.38:0/1298494053 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc6d4162280 0x7fc6d41f8ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:27.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:27.029+0000 7fc6d8b90640 1 --2- 172.21.15.38:0/1298494053 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc6d40b2e40 0x7fc6d41f89a0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:27.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:27.029+0000 7fc6d8b90640 1 -- 172.21.15.38:0/1298494053 >> 172.21.15.38:0/1298494053 conn(0x7fc6d40b6b20 msgr2=0x7fc6d415e4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:27.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:27.029+0000 7fc6d8b90640 1 -- 172.21.15.38:0/1298494053 shutdown_connections 2024-09-21T03:35:27.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:27.029+0000 7fc6d8b90640 1 -- 172.21.15.38:0/1298494053 wait complete. 2024-09-21T03:35:27.042 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-21T03:35:27.042 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T03:35:27.042 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi038.front.sepia.ceph.com 2024-09-21T03:35:27.042 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-21T03:35:27.042 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T03:35:27.042 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-21T03:35:27.042 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-21T03:35:27.042 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T03:35:27.042 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T03:35:27.043 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:35:27.043 DEBUG:teuthology.orchestra.run.smithi038:> ip addr 2024-09-21T03:35:27.059 INFO:teuthology.orchestra.run.smithi038.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T03:35:27.059 INFO:teuthology.orchestra.run.smithi038.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T03:35:27.059 INFO:teuthology.orchestra.run.smithi038.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T03:35:27.059 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:35:27.059 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 ::1/128 scope host 2024-09-21T03:35:27.059 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:35:27.059 INFO:teuthology.orchestra.run.smithi038.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T03:35:27.059 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:6b:fd:a2 brd ff:ff:ff:ff:ff:ff 2024-09-21T03:35:27.060 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp7s0f0 2024-09-21T03:35:27.060 INFO:teuthology.orchestra.run.smithi038.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T03:35:27.060 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:6b:fd:a3 brd ff:ff:ff:ff:ff:ff 2024-09-21T03:35:27.060 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp7s0f1 2024-09-21T03:35:27.060 INFO:teuthology.orchestra.run.smithi038.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T03:35:27.060 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:bd:15:fc brd ff:ff:ff:ff:ff:ff 2024-09-21T03:35:27.060 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp1s0f0 2024-09-21T03:35:27.060 INFO:teuthology.orchestra.run.smithi038.stdout: inet 172.21.15.38/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-21T03:35:27.060 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft 42682sec preferred_lft 42682sec 2024-09-21T03:35:27.060 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 fe80::ec4:7aff:febd:15fc/64 scope link 2024-09-21T03:35:27.060 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:35:27.060 INFO:teuthology.orchestra.run.smithi038.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T03:35:27.060 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:bd:15:fd brd ff:ff:ff:ff:ff:ff 2024-09-21T03:35:27.060 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp1s0f1 2024-09-21T03:35:27.061 INFO:teuthology.orchestra.run.smithi038.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T03:35:27.061 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 92:55:c7:97:6f:b5 brd ff:ff:ff:ff:ff:ff 2024-09-21T03:35:27.061 INFO:teuthology.orchestra.run.smithi038.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-21T03:35:27.061 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:35:27.061 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 fe80::c49f:c1ff:fe21:b12a/64 scope link 2024-09-21T03:35:27.061 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:35:27.061 INFO:teuthology.orchestra.run.smithi038.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T03:35:27.061 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether a6:3a:83:ad:db:d5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T03:35:27.061 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 fe80::a43a:83ff:fead:dbd5/64 scope link 2024-09-21T03:35:27.061 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:35:27.061 INFO:teuthology.orchestra.run.smithi038.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T03:35:27.061 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 92:55:c7:97:6f:b5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T03:35:27.061 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 fe80::9055:c7ff:fe97:6fb5/64 scope link 2024-09-21T03:35:27.061 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:35:27.062 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:35:27.062 DEBUG:teuthology.orchestra.run.smithi038:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T03:35:27.123 INFO:teuthology.orchestra.run.smithi038.stdout:1 2024-09-21T03:35:27.125 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:35:27.125 DEBUG:teuthology.orchestra.run.smithi038:> route 2024-09-21T03:35:27.182 INFO:teuthology.orchestra.run.smithi038.stdout:Kernel IP routing table 2024-09-21T03:35:27.182 INFO:teuthology.orchestra.run.smithi038.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T03:35:27.182 INFO:teuthology.orchestra.run.smithi038.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-21T03:35:27.182 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-21T03:35:27.182 INFO:teuthology.orchestra.run.smithi038.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T03:35:27.183 DEBUG:teuthology.orchestra.run.smithi038:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T03:35:27.183 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-09-21T03:35:27.183 DEBUG:teuthology.orchestra.run.smithi038:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-21T03:35:27.183 DEBUG:teuthology.orchestra.run.smithi038:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-21T03:35:27.183 DEBUG:teuthology.orchestra.run.smithi038:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-21T03:35:27.183 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-09-21T03:35:27.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:27 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T03:35:27.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:35:27.311 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:35:27.311 DEBUG:teuthology.orchestra.run.smithi038:> ip netns list 2024-09-21T03:35:27.327 INFO:teuthology.orchestra.run.smithi038.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-21T03:35:27.327 INFO:teuthology.orchestra.run.smithi038.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-21T03:35:27.327 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T03:35:27.327 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:35:27.327 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip link set brx.0 up 2024-09-21T03:35:27.392 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T03:35:27.393 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-21T03:35:27.448 INFO:teuthology.orchestra.run.smithi038.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-21T03:35:27.448 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T03:35:27.448 DEBUG:teuthology.orchestra.run.smithi038:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-21T03:35:27.504 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-21T03:35:27.504 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:35:27.504 DEBUG:teuthology.orchestra.run.smithi038:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T03:35:27.600 INFO:teuthology.orchestra.run.smithi038.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T03:35:27.600 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T03:35:27.600 INFO:teuthology.orchestra.run.smithi038.stdout:invalid new device string format 2024-09-21T03:35:27.611 INFO:teuthology.orchestra.run.smithi038.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T03:35:27.611 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T03:35:27.611 INFO:teuthology.orchestra.run.smithi038.stdout:invalid new device string format 2024-09-21T03:35:27.611 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: resolved to: "172.21.15.38:3300,172.21.15.195:3300,172.21.15.195:3301" 2024-09-21T03:35:27.611 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T03:35:27.617 INFO:teuthology.orchestra.run.smithi038.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T03:35:27.617 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T03:35:27.618 INFO:teuthology.orchestra.run.smithi038.stdout:invalid new device string format 2024-09-21T03:35:27.618 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: resolved to: "172.21.15.38:3300,172.21.15.195:3300,172.21.15.195:3301" 2024-09-21T03:35:27.618 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T03:35:27.618 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: resolved to: "172.21.15.38:6789,172.21.15.195:6789,172.21.15.195:6790" 2024-09-21T03:35:27.618 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.38:6789,172.21.15.195:6789,172.21.15.195:6790:/ 2024-09-21T03:35:27.618 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=9d50d66f-656b-4fe2-8e3b-fe53b79c4d43" will pass to kernel 2024-09-21T03:35:27.618 INFO:teuthology.orchestra.run.smithi038.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-21T03:35:27.619 INFO:teuthology.orchestra.run.smithi038.stdout: You just mounted a file system that supports labels which does not 2024-09-21T03:35:27.619 INFO:teuthology.orchestra.run.smithi038.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-21T03:35:27.619 INFO:teuthology.orchestra.run.smithi038.stdout: applications will generate AVC messages and not be allowed access to 2024-09-21T03:35:27.619 INFO:teuthology.orchestra.run.smithi038.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-21T03:35:27.620 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T03:35:27.620 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:35:27.620 DEBUG:teuthology.orchestra.run.smithi038:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-21T03:35:27.646 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T03:35:27.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:27 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T03:35:27.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:35:27.744 INFO:teuthology.orchestra.run.smithi038.stdout:client5030 2024-09-21T03:35:27.744 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-21T03:35:27.744 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:35:27.760 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:35:27.761 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.files ./subdir) 2024-09-21T03:35:27.817 INFO:teuthology.orchestra.run.smithi038.stdout:1 2024-09-21T03:35:27.817 DEBUG:teuthology.orchestra.run.smithi038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata setomapval 10000000000.00000000 file_to_be_damaged_head deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef 2024-09-21T03:35:27.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:27.913+0000 7f62f2873f40 1 Processor -- start 2024-09-21T03:35:27.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:27.913+0000 7f62f2873f40 1 -- start start 2024-09-21T03:35:27.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:27.914+0000 7f62f2873f40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5560527d5240 0x55605288ded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:27.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:27.914+0000 7f62f2873f40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55605281c090 0x55605288e410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:27.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:27.914+0000 7f62f2873f40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556052891520 0x55605293b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:27.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:27.914+0000 7f62f2873f40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x556052722d60 con 0x5560527d5240 2024-09-21T03:35:27.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:27.914+0000 7f62f2873f40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x556052781430 con 0x556052891520 2024-09-21T03:35:27.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:27.914+0000 7f62f2873f40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x5560528a9350 con 0x55605281c090 2024-09-21T03:35:27.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:27.915+0000 7f62ec32f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5560527d5240 0x55605288ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:27.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:27.915+0000 7f62ec32f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5560527d5240 0x55605288ded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44626/0 (socket says 172.21.15.38:44626) 2024-09-21T03:35:27.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:27.915+0000 7f62ec32f640 1 -- 172.21.15.38:0/2417951562 learned_addr learned my addr 172.21.15.38:0/2417951562 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:35:27.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:27.915+0000 7f62ebb2e640 1 --2- 172.21.15.38:0/2417951562 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55605281c090 0x55605288e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:27.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:27.915+0000 7f62ecb30640 1 --2- 172.21.15.38:0/2417951562 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556052891520 0x55605293b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:27.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:27.915+0000 7f62ec32f640 1 -- 172.21.15.38:0/2417951562 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55605281c090 msgr2=0x55605288e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:35:27.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:27.915+0000 7f62ec32f640 1 --2- 172.21.15.38:0/2417951562 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55605281c090 0x55605288e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:27.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:27.915+0000 7f62ec32f640 1 -- 172.21.15.38:0/2417951562 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556052891520 msgr2=0x55605293b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:35:27.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:27.915+0000 7f62ec32f640 1 --2- 172.21.15.38:0/2417951562 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556052891520 0x55605293b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:27.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:27.915+0000 7f62ec32f640 1 -- 172.21.15.38:0/2417951562 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55605287c7b0 con 0x5560527d5240 2024-09-21T03:35:27.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:27.915+0000 7f62ebb2e640 1 --2- 172.21.15.38:0/2417951562 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55605281c090 0x55605288e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:35:27.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:27.915+0000 7f62ecb30640 1 --2- 172.21.15.38:0/2417951562 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556052891520 0x55605293b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:35:27.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:27.915+0000 7f62ec32f640 1 --2- 172.21.15.38:0/2417951562 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5560527d5240 0x55605288ded0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f62dc00a460 tx=0x7f62dc00a920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40b2060dcc90331 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:27.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:27.915+0000 7f62eab2c640 1 -- 172.21.15.38:0/2417951562 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62dc00f7b0 con 0x5560527d5240 2024-09-21T03:35:27.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:27.915+0000 7f62eab2c640 1 -- 172.21.15.38:0/2417951562 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f62dc015070 con 0x5560527d5240 2024-09-21T03:35:27.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:27.915+0000 7f62eab2c640 1 -- 172.21.15.38:0/2417951562 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62dc00fd00 con 0x5560527d5240 2024-09-21T03:35:27.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:27.916+0000 7f62f2873f40 1 -- 172.21.15.38:0/2417951562 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5560527d5240 msgr2=0x55605288ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:27.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:27.916+0000 7f62f2873f40 1 --2- 172.21.15.38:0/2417951562 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5560527d5240 0x55605288ded0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f62dc00a460 tx=0x7f62dc00a920 comp rx=0 tx=0).stop 2024-09-21T03:35:27.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:27.916+0000 7f62f2873f40 1 -- 172.21.15.38:0/2417951562 shutdown_connections 2024-09-21T03:35:27.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:27.916+0000 7f62f2873f40 1 --2- 172.21.15.38:0/2417951562 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556052891520 0x55605293b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:27.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:27.916+0000 7f62f2873f40 1 --2- 172.21.15.38:0/2417951562 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55605281c090 0x55605288e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:27.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:27.916+0000 7f62f2873f40 1 --2- 172.21.15.38:0/2417951562 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5560527d5240 0x55605288ded0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:27.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:27.916+0000 7f62f2873f40 1 -- 172.21.15.38:0/2417951562 >> 172.21.15.38:0/2417951562 conn(0x5560526f3ac0 msgr2=0x55605281a7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:27.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:27.916+0000 7f62f2873f40 1 -- 172.21.15.38:0/2417951562 shutdown_connections 2024-09-21T03:35:27.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:27.916+0000 7f62f2873f40 1 -- 172.21.15.38:0/2417951562 wait complete. 2024-09-21T03:35:27.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:27.917+0000 7f62f2873f40 1 Processor -- start 2024-09-21T03:35:27.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:27.917+0000 7f62f2873f40 1 -- start start 2024-09-21T03:35:27.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:27.918+0000 7f62f2873f40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5560527d5240 0x5560528a0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:27.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:27.918+0000 7f62f2873f40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55605281c090 0x5560527fe620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:27.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:27.918+0000 7f62f2873f40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x556052891520 0x5560527feb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:27.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:27.918+0000 7f62f2873f40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55605288db80 con 0x5560527d5240 2024-09-21T03:35:27.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:27.918+0000 7f62f2873f40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x556052890d30 con 0x55605281c090 2024-09-21T03:35:27.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:27.918+0000 7f62f2873f40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x556052890650 con 0x556052891520 2024-09-21T03:35:27.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:27.918+0000 7f62ec32f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5560527d5240 0x5560528a0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:27.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:27.918+0000 7f62ec32f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5560527d5240 0x5560528a0ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44632/0 (socket says 172.21.15.38:44632) 2024-09-21T03:35:27.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:27.918+0000 7f62ec32f640 1 -- 172.21.15.38:0/611770189 learned_addr learned my addr 172.21.15.38:0/611770189 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:35:27.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:27.919+0000 7f62ebb2e640 1 --2- 172.21.15.38:0/611770189 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55605281c090 0x5560527fe620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:27.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:27.919+0000 7f62ecb30640 1 --2- 172.21.15.38:0/611770189 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x556052891520 0x5560527feb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:27.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:27.919+0000 7f62ec32f640 1 -- 172.21.15.38:0/611770189 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x556052891520 msgr2=0x5560527feb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:27.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:27.919+0000 7f62ec32f640 1 --2- 172.21.15.38:0/611770189 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x556052891520 0x5560527feb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:27.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:27.919+0000 7f62ec32f640 1 -- 172.21.15.38:0/611770189 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55605281c090 msgr2=0x5560527fe620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:27.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:27.919+0000 7f62ec32f640 1 --2- 172.21.15.38:0/611770189 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55605281c090 0x5560527fe620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:27.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:27.919+0000 7f62ec32f640 1 -- 172.21.15.38:0/611770189 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5560528aa940 con 0x5560527d5240 2024-09-21T03:35:27.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:27.919+0000 7f62ec32f640 1 --2- 172.21.15.38:0/611770189 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5560527d5240 0x5560528a0ad0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f62dc00f920 tx=0x7f62dc00fcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:27.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:27.919+0000 7f62e9b2a640 1 -- 172.21.15.38:0/611770189 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62dc010850 con 0x5560527d5240 2024-09-21T03:35:27.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:27.919+0000 7f62e9b2a640 1 -- 172.21.15.38:0/611770189 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f62dc01d070 con 0x5560527d5240 2024-09-21T03:35:27.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:27.919+0000 7f62e9b2a640 1 -- 172.21.15.38:0/611770189 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62dc010d60 con 0x5560527d5240 2024-09-21T03:35:27.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:27.919+0000 7f62f2873f40 1 -- 172.21.15.38:0/611770189 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5560528a4f40 con 0x5560527d5240 2024-09-21T03:35:27.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:27.919+0000 7f62f2873f40 1 -- 172.21.15.38:0/611770189 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556052890330 con 0x5560527d5240 2024-09-21T03:35:27.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:27.921+0000 7f62e9b2a640 1 -- 172.21.15.38:0/611770189 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f62dc0193f0 con 0x5560527d5240 2024-09-21T03:35:27.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:27.921+0000 7f62e9b2a640 1 --2- 172.21.15.38:0/611770189 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f62d80775a0 0x7f62d8079ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:27.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:27.921+0000 7f62e9b2a640 1 -- 172.21.15.38:0/611770189 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6245+0+0 (secure 0 0 0) 0x7f62dc09b120 con 0x5560527d5240 2024-09-21T03:35:27.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:27.922+0000 7f62f2873f40 1 --2- 172.21.15.38:0/611770189 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x5560528a1b10 0x5560528a1f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:27.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:27.922+0000 7f62ebb2e640 1 --2- 172.21.15.38:0/611770189 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f62d80775a0 0x7f62d8079ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:27.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:27.922+0000 7f62f2873f40 1 -- 172.21.15.38:0/611770189 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:1 5.10 5:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=115b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43) -- 0x55605294bab0 con 0x5560528a1b10 2024-09-21T03:35:27.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:27.922+0000 7f62ecb30640 1 --2- 172.21.15.38:0/611770189 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x5560528a1b10 0x5560528a1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:27.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:27.922+0000 7f62ecb30640 1 --2- 172.21.15.38:0/611770189 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x5560528a1b10 0x5560528a1f20 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:27.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:27.922+0000 7f62ebb2e640 1 --2- 172.21.15.38:0/611770189 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f62d80775a0 0x7f62d8079ab0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f62e0055000 tx=0x7f62e0058040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:27.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:27.923+0000 7f62ecb30640 1 -- 172.21.15.38:0/611770189 <== osd.2 v2:172.21.15.38:6824/2577858902 1 ==== osd_op_reply(1 10000000000.00000000 [omap-set-vals] v43'12 uv12 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f62e400b640 con 0x5560528a1b10 2024-09-21T03:35:27.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:27.923+0000 7f62f2873f40 1 -- 172.21.15.38:0/611770189 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x5560528a1b10 msgr2=0x5560528a1f20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:27.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:27.923+0000 7f62f2873f40 1 --2- 172.21.15.38:0/611770189 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x5560528a1b10 0x5560528a1f20 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:27.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:27.924+0000 7f62f2873f40 1 -- 172.21.15.38:0/611770189 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f62d80775a0 msgr2=0x7f62d8079ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:27.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:27.924+0000 7f62f2873f40 1 --2- 172.21.15.38:0/611770189 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f62d80775a0 0x7f62d8079ab0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f62e0055000 tx=0x7f62e0058040 comp rx=0 tx=0).stop 2024-09-21T03:35:27.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:27.924+0000 7f62f2873f40 1 -- 172.21.15.38:0/611770189 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5560527d5240 msgr2=0x5560528a0ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:27.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:27.924+0000 7f62f2873f40 1 --2- 172.21.15.38:0/611770189 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5560527d5240 0x5560528a0ad0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f62dc00f920 tx=0x7f62dc00fcf0 comp rx=0 tx=0).stop 2024-09-21T03:35:27.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:27.924+0000 7f62ecb30640 1 -- 172.21.15.38:0/611770189 reap_dead start 2024-09-21T03:35:27.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:27.924+0000 7f62f2873f40 1 -- 172.21.15.38:0/611770189 shutdown_connections 2024-09-21T03:35:27.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:27.924+0000 7f62f2873f40 1 -- 172.21.15.38:0/611770189 >> 172.21.15.38:0/611770189 conn(0x5560526f3ac0 msgr2=0x55605288de60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:27.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:27.924+0000 7f62f2873f40 1 -- 172.21.15.38:0/611770189 shutdown_connections 2024-09-21T03:35:27.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:27.924+0000 7f62f2873f40 1 -- 172.21.15.38:0/611770189 wait complete. 2024-09-21T03:35:27.930 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:35:28.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.045+0000 7f570f3df640 1 -- 172.21.15.38:0/3174629643 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f570815cb10 msgr2=0x7f570815cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:28.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.045+0000 7f570f3df640 1 --2- 172.21.15.38:0/3174629643 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f570815cb10 0x7f570815cf20 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f57080d2320 tx=0x7f570002a530 comp rx=0 tx=0).stop 2024-09-21T03:35:28.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.045+0000 7f570f3df640 1 -- 172.21.15.38:0/3174629643 shutdown_connections 2024-09-21T03:35:28.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.045+0000 7f570f3df640 1 --2- 172.21.15.38:0/3174629643 >> v2:172.21.15.195:3300/0 conn(0x7f57080d12e0 0x7f57080d16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:28.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.045+0000 7f570f3df640 1 --2- 172.21.15.38:0/3174629643 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f570815cb10 0x7f570815cf20 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:28.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.045+0000 7f570f3df640 1 -- 172.21.15.38:0/3174629643 >> 172.21.15.38:0/3174629643 conn(0x7f57080d2350 msgr2=0x7f57080d2760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:28.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.045+0000 7f570f3df640 1 -- 172.21.15.38:0/3174629643 shutdown_connections 2024-09-21T03:35:28.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.045+0000 7f570f3df640 1 -- 172.21.15.38:0/3174629643 wait complete. 2024-09-21T03:35:28.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.047+0000 7f570f3df640 1 Processor -- start 2024-09-21T03:35:28.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.047+0000 7f570f3df640 1 -- start start 2024-09-21T03:35:28.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.048+0000 7f570f3df640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f57080d12e0 0x7f57081f9840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:28.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.048+0000 7f570f3df640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f570815cb10 0x7f570814f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:28.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.048+0000 7f570f3df640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5708160100 0x7f570806f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:28.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.048+0000 7f570f3df640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f57080b73e0 con 0x7f57080d12e0 2024-09-21T03:35:28.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.048+0000 7f570f3df640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f57080b7260 con 0x7f5708160100 2024-09-21T03:35:28.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.048+0000 7f570f3df640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f5708160c20 con 0x7f570815cb10 2024-09-21T03:35:28.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.048+0000 7f570d17b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f57080d12e0 0x7f57081f9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:28.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.048+0000 7f570d17b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f57080d12e0 0x7f57081f9840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44644/0 (socket says 172.21.15.38:44644) 2024-09-21T03:35:28.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.048+0000 7f570d17b640 1 -- 172.21.15.38:0/3731698808 learned_addr learned my addr 172.21.15.38:0/3731698808 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:35:28.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.048+0000 7f570c97a640 1 --2- 172.21.15.38:0/3731698808 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f570815cb10 0x7f570814f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:28.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.048+0000 7f570d17b640 1 -- 172.21.15.38:0/3731698808 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f570815cb10 msgr2=0x7f570814f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:28.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.048+0000 7f570d17b640 1 --2- 172.21.15.38:0/3731698808 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f570815cb10 0x7f570814f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:28.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.048+0000 7f570d17b640 1 -- 172.21.15.38:0/3731698808 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5708160100 msgr2=0x7f570806f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:28.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.048+0000 7f570d17b640 1 --2- 172.21.15.38:0/3731698808 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5708160100 0x7f570806f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:28.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.048+0000 7f570d17b640 1 -- 172.21.15.38:0/3731698808 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f570815b5e0 con 0x7f57080d12e0 2024-09-21T03:35:28.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.048+0000 7f570d17b640 1 --2- 172.21.15.38:0/3731698808 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f57080d12e0 0x7f57081f9840 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f5700009780 tx=0x7f5700033d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:28.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.049+0000 7f56fdffb640 1 -- 172.21.15.38:0/3731698808 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5700031d60 con 0x7f57080d12e0 2024-09-21T03:35:28.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.049+0000 7f56fdffb640 1 -- 172.21.15.38:0/3731698808 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5700005020 con 0x7f57080d12e0 2024-09-21T03:35:28.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.049+0000 7f56fdffb640 1 -- 172.21.15.38:0/3731698808 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5700005470 con 0x7f57080d12e0 2024-09-21T03:35:28.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.049+0000 7f570f3df640 1 -- 172.21.15.38:0/3731698808 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f570815bae0 con 0x7f57080d12e0 2024-09-21T03:35:28.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.049+0000 7f570f3df640 1 -- 172.21.15.38:0/3731698808 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57081f9040 con 0x7f57080d12e0 2024-09-21T03:35:28.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.050+0000 7f570f3df640 1 -- 172.21.15.38:0/3731698808 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57080d16f0 con 0x7f57080d12e0 2024-09-21T03:35:28.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.051+0000 7f56fdffb640 1 -- 172.21.15.38:0/3731698808 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f57000047a0 con 0x7f57080d12e0 2024-09-21T03:35:28.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.051+0000 7f56fdffb640 1 --2- 172.21.15.38:0/3731698808 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f56ec078b90 0x7f56ec07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:28.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.051+0000 7f56fdffb640 1 -- 172.21.15.38:0/3731698808 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6245+0+0 (secure 0 0 0) 0x7f57000bc230 con 0x7f57080d12e0 2024-09-21T03:35:28.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.051+0000 7f570c97a640 1 --2- 172.21.15.38:0/3731698808 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f56ec078b90 0x7f56ec07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:28.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.052+0000 7f570c97a640 1 --2- 172.21.15.38:0/3731698808 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f56ec078b90 0x7f56ec07b0a0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f56f80058c0 tx=0x7f56f8005be0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:28.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.056+0000 7f56fdffb640 1 -- 172.21.15.38:0/3731698808 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f570003f030 con 0x7f57080d12e0 2024-09-21T03:35:28.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.280+0000 7f570f3df640 1 -- 172.21.15.38:0/3731698808 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f57080b56b0 con 0x7f57080d12e0 2024-09-21T03:35:28.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.281+0000 7f56fdffb640 1 -- 172.21.15.38:0/3731698808 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+5230 (secure 0 0 0) 0x7f5700084450 con 0x7f57080d12e0 2024-09-21T03:35:28.283 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:35:28.283 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":23,"btime":"2024-09-21T03:34:51:918500+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4757,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/1905955399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":1905955399},{"type":"v1","addr":"172.21.15.195:6836","nonce":1905955399}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4936,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/2989374027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":2989374027},{"type":"v1","addr":"172.21.15.195:6837","nonce":2989374027}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4942,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/3531731501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6833","nonce":3531731501},{"type":"v1","addr":"172.21.15.38:6835","nonce":3531731501}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:34:34.009269+0000","modified":"2024-09-21T03:34:51.918491+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":41,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5226},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5226":{"gid":5226,"name":"c","rank":0,"incarnation":20,"state":"up:active","state_seq":4,"addr":"172.21.15.38:6834/3533947700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":3533947700},{"type":"v1","addr":"172.21.15.38:6834","nonce":3533947700}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5226,"qdb_cluster":[5226]},"id":2}]} 2024-09-21T03:35:28.283 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 23 2024-09-21T03:35:28.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.283+0000 7f570f3df640 1 -- 172.21.15.38:0/3731698808 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f56ec078b90 msgr2=0x7f56ec07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:28.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.283+0000 7f570f3df640 1 --2- 172.21.15.38:0/3731698808 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f56ec078b90 0x7f56ec07b0a0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f56f80058c0 tx=0x7f56f8005be0 comp rx=0 tx=0).stop 2024-09-21T03:35:28.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.283+0000 7f570f3df640 1 -- 172.21.15.38:0/3731698808 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f57080d12e0 msgr2=0x7f57081f9840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:28.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.283+0000 7f570f3df640 1 --2- 172.21.15.38:0/3731698808 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f57080d12e0 0x7f57081f9840 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f5700009780 tx=0x7f5700033d60 comp rx=0 tx=0).stop 2024-09-21T03:35:28.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.283+0000 7f570f3df640 1 -- 172.21.15.38:0/3731698808 shutdown_connections 2024-09-21T03:35:28.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.283+0000 7f570f3df640 1 --2- 172.21.15.38:0/3731698808 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5708160100 0x7f570806f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:28.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.283+0000 7f570f3df640 1 --2- 172.21.15.38:0/3731698808 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f56ec078b90 0x7f56ec07b0a0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:28.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.283+0000 7f570f3df640 1 --2- 172.21.15.38:0/3731698808 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f570815cb10 0x7f570814f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:28.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.283+0000 7f570f3df640 1 --2- 172.21.15.38:0/3731698808 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f57080d12e0 0x7f57081f9840 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:28.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.283+0000 7f570f3df640 1 -- 172.21.15.38:0/3731698808 >> 172.21.15.38:0/3731698808 conn(0x7f57080d2350 msgr2=0x7f57080714f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:28.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.283+0000 7f570f3df640 1 -- 172.21.15.38:0/3731698808 shutdown_connections 2024-09-21T03:35:28.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.283+0000 7f570f3df640 1 -- 172.21.15.38:0/3731698808 wait complete. 2024-09-21T03:35:28.296 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c damage rm 2742033437 2024-09-21T03:35:28.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.413+0000 7f9351638640 1 -- 172.21.15.38:0/253367418 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f934c0b31e0 msgr2=0x7f934c073ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:28.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.413+0000 7f9351638640 1 --2- 172.21.15.38:0/253367418 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f934c0b31e0 0x7f934c073ea0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f934c073950 tx=0x7f934402a4d0 comp rx=0 tx=0).stop 2024-09-21T03:35:28.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.413+0000 7f9351638640 1 -- 172.21.15.38:0/253367418 shutdown_connections 2024-09-21T03:35:28.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.413+0000 7f9351638640 1 --2- 172.21.15.38:0/253367418 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f934c0b31e0 0x7f934c073ea0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:28.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.413+0000 7f9351638640 1 --2- 172.21.15.38:0/253367418 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f934c0aeeb0 0x7f934c0b2ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:28.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.413+0000 7f9351638640 1 --2- 172.21.15.38:0/253367418 >> v2:172.21.15.195:3300/0 conn(0x7f934c15c750 0x7f934c0ae970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:28.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.413+0000 7f9351638640 1 -- 172.21.15.38:0/253367418 >> 172.21.15.38:0/253367418 conn(0x7f934c0cdca0 msgr2=0x7f934c0ce0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:28.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.413+0000 7f9351638640 1 -- 172.21.15.38:0/253367418 shutdown_connections 2024-09-21T03:35:28.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.413+0000 7f9351638640 1 -- 172.21.15.38:0/253367418 wait complete. 2024-09-21T03:35:28.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.415+0000 7f9351638640 1 Processor -- start 2024-09-21T03:35:28.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.415+0000 7f9351638640 1 -- start start 2024-09-21T03:35:28.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.416+0000 7f9351638640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f934c0aeeb0 0x7f934c157600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:28.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.416+0000 7f9351638640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f934c0b31e0 0x7f934c153750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:28.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.416+0000 7f9351638640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f934c15c750 0x7f934c15f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:28.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.416+0000 7f9351638640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f934c074560 con 0x7f934c0b31e0 2024-09-21T03:35:28.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.416+0000 7f9351638640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f934c0743e0 con 0x7f934c0aeeb0 2024-09-21T03:35:28.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.416+0000 7f9351638640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f934c0746e0 con 0x7f934c15c750 2024-09-21T03:35:28.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.417+0000 7f934a7fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f934c0b31e0 0x7f934c153750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:28.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.417+0000 7f934a7fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f934c0b31e0 0x7f934c153750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44672/0 (socket says 172.21.15.38:44672) 2024-09-21T03:35:28.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.417+0000 7f934a7fc640 1 -- 172.21.15.38:0/865482 learned_addr learned my addr 172.21.15.38:0/865482 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:35:28.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.417+0000 7f934affd640 1 --2- 172.21.15.38:0/865482 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f934c0aeeb0 0x7f934c157600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:28.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.417+0000 7f934b7fe640 1 --2- 172.21.15.38:0/865482 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f934c15c750 0x7f934c15f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:28.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.417+0000 7f934a7fc640 1 -- 172.21.15.38:0/865482 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f934c15c750 msgr2=0x7f934c15f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:28.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.417+0000 7f934a7fc640 1 --2- 172.21.15.38:0/865482 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f934c15c750 0x7f934c15f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:28.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.417+0000 7f934a7fc640 1 -- 172.21.15.38:0/865482 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f934c0aeeb0 msgr2=0x7f934c157600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:28.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.417+0000 7f934a7fc640 1 --2- 172.21.15.38:0/865482 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f934c0aeeb0 0x7f934c157600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:28.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.417+0000 7f934a7fc640 1 -- 172.21.15.38:0/865482 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f934c0cff70 con 0x7f934c0b31e0 2024-09-21T03:35:28.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.417+0000 7f934a7fc640 1 --2- 172.21.15.38:0/865482 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f934c0b31e0 0x7f934c153750 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f933400b440 tx=0x7f933400b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:28.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.417+0000 7f93437fe640 1 -- 172.21.15.38:0/865482 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9334015020 con 0x7f934c0b31e0 2024-09-21T03:35:28.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.417+0000 7f93437fe640 1 -- 172.21.15.38:0/865482 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9334011980 con 0x7f934c0b31e0 2024-09-21T03:35:28.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.417+0000 7f93437fe640 1 -- 172.21.15.38:0/865482 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9334011dd0 con 0x7f934c0b31e0 2024-09-21T03:35:28.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.417+0000 7f9351638640 1 -- 172.21.15.38:0/865482 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f934c0d0250 con 0x7f934c0b31e0 2024-09-21T03:35:28.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.417+0000 7f9351638640 1 -- 172.21.15.38:0/865482 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f934c0d0810 con 0x7f934c0b31e0 2024-09-21T03:35:28.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.419+0000 7f93437fe640 1 -- 172.21.15.38:0/865482 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f9334018020 con 0x7f934c0b31e0 2024-09-21T03:35:28.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.420+0000 7f93437fe640 1 --2- 172.21.15.38:0/865482 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f932c078b90 0x7f932c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:28.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.420+0000 7f93437fe640 1 -- 172.21.15.38:0/865482 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6245+0+0 (secure 0 0 0) 0x7f933409aef0 con 0x7f934c0b31e0 2024-09-21T03:35:28.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.420+0000 7f934affd640 1 --2- 172.21.15.38:0/865482 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f932c078b90 0x7f932c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:28.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.421+0000 7f9358687740 1 Processor -- start 2024-09-21T03:35:28.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.421+0000 7f934affd640 1 --2- 172.21.15.38:0/865482 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f932c078b90 0x7f932c07b0a0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f933c00a400 tx=0x7f933c00d040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:28.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.421+0000 7f9358687740 1 -- start start 2024-09-21T03:35:28.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.422+0000 7f9358687740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a4694282a0 0x55a4698175b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:28.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.422+0000 7f9358687740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a4693ff3e0 0x55a4697fa280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:28.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.422+0000 7f9358687740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a4694075c0 0x55a4697fa7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:28.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.422+0000 7f9358687740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55a469742910 con 0x55a4694075c0 2024-09-21T03:35:28.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.422+0000 7f9358687740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55a469731d40 con 0x55a4693ff3e0 2024-09-21T03:35:28.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.422+0000 7f9358687740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55a469735b50 con 0x55a4694282a0 2024-09-21T03:35:28.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.422+0000 7f934b7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a4694075c0 0x55a4697fa7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:28.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.422+0000 7f934b7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a4694075c0 0x55a4697fa7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44674/0 (socket says 172.21.15.38:44674) 2024-09-21T03:35:28.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.422+0000 7f934b7fe640 1 -- 172.21.15.38:0/1728149777 learned_addr learned my addr 172.21.15.38:0/1728149777 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:35:28.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.422+0000 7f934affd640 1 --2- 172.21.15.38:0/1728149777 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a4694282a0 0x55a4698175b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:28.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.422+0000 7f934a7fc640 1 --2- 172.21.15.38:0/1728149777 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a4693ff3e0 0x55a4697fa280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:28.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.423+0000 7f934b7fe640 1 -- 172.21.15.38:0/1728149777 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a4694282a0 msgr2=0x55a4698175b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:35:28.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.423+0000 7f934b7fe640 1 --2- 172.21.15.38:0/1728149777 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a4694282a0 0x55a4698175b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:28.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.423+0000 7f934b7fe640 1 -- 172.21.15.38:0/1728149777 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a4693ff3e0 msgr2=0x55a4697fa280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:35:28.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.423+0000 7f934b7fe640 1 --2- 172.21.15.38:0/1728149777 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a4693ff3e0 0x55a4697fa280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:28.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.423+0000 7f934b7fe640 1 -- 172.21.15.38:0/1728149777 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a46950c710 con 0x55a4694075c0 2024-09-21T03:35:28.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.423+0000 7f934b7fe640 1 --2- 172.21.15.38:0/1728149777 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a4694075c0 0x55a4697fa7c0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f934402aac0 tx=0x7f93440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb0ec1e09f7cfb1b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:28.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.423+0000 7f9340ff9640 1 -- 172.21.15.38:0/1728149777 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9344035040 con 0x55a4694075c0 2024-09-21T03:35:28.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.423+0000 7f9340ff9640 1 -- 172.21.15.38:0/1728149777 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9344004330 con 0x55a4694075c0 2024-09-21T03:35:28.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.423+0000 7f9340ff9640 1 -- 172.21.15.38:0/1728149777 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93440044e0 con 0x55a4694075c0 2024-09-21T03:35:28.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.424+0000 7f9358687740 1 -- 172.21.15.38:0/1728149777 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a4694075c0 msgr2=0x55a4697fa7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:28.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.424+0000 7f9358687740 1 --2- 172.21.15.38:0/1728149777 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a4694075c0 0x55a4697fa7c0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f934402aac0 tx=0x7f93440046c0 comp rx=0 tx=0).stop 2024-09-21T03:35:28.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.424+0000 7f9358687740 1 -- 172.21.15.38:0/1728149777 shutdown_connections 2024-09-21T03:35:28.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.424+0000 7f9358687740 1 --2- 172.21.15.38:0/1728149777 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a4694075c0 0x55a4697fa7c0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:28.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.424+0000 7f9358687740 1 --2- 172.21.15.38:0/1728149777 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a4693ff3e0 0x55a4697fa280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:28.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.424+0000 7f9358687740 1 --2- 172.21.15.38:0/1728149777 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a4694282a0 0x55a4698175b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:28.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.424+0000 7f9358687740 1 -- 172.21.15.38:0/1728149777 >> 172.21.15.38:0/1728149777 conn(0x55a46941c3f0 msgr2=0x55a4697d04a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:28.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.424+0000 7f9358687740 1 -- 172.21.15.38:0/1728149777 shutdown_connections 2024-09-21T03:35:28.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.424+0000 7f9358687740 1 -- 172.21.15.38:0/1728149777 wait complete. 2024-09-21T03:35:28.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.425+0000 7f9358687740 1 -- start start 2024-09-21T03:35:28.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.425+0000 7f9358687740 1 Processor -- start 2024-09-21T03:35:28.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.426+0000 7f9358687740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a4693ff3e0 0x55a469815bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:28.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.426+0000 7f9358687740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a4694075c0 0x55a469811430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:28.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.426+0000 7f9358687740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a4694282a0 0x55a4697e80a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:28.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.426+0000 7f9358687740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55a4694a2650 con 0x55a4694075c0 2024-09-21T03:35:28.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.426+0000 7f9358687740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55a4696b85f0 con 0x55a4694282a0 2024-09-21T03:35:28.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.426+0000 7f9358687740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55a469811bf0 con 0x55a4693ff3e0 2024-09-21T03:35:28.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.426+0000 7f9358687740 10 client.0 _ll_register_callbacks cb 0x55a46972c060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:35:28.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.426+0000 7f934a7fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a4694075c0 0x55a469811430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:28.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.426+0000 7f934a7fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a4694075c0 0x55a469811430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44680/0 (socket says 172.21.15.38:44680) 2024-09-21T03:35:28.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.426+0000 7f934a7fc640 1 -- 172.21.15.38:0/3307445175 learned_addr learned my addr 172.21.15.38:0/3307445175 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:35:28.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.426+0000 7f934b7fe640 1 --2- 172.21.15.38:0/3307445175 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a4694282a0 0x55a4697e80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:28.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.427+0000 7f934affd640 1 --2- 172.21.15.38:0/3307445175 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a4693ff3e0 0x55a469815bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:28.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.427+0000 7f934a7fc640 1 -- 172.21.15.38:0/3307445175 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a4693ff3e0 msgr2=0x55a469815bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:35:28.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.427+0000 7f934a7fc640 1 --2- 172.21.15.38:0/3307445175 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a4693ff3e0 0x55a469815bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:28.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.427+0000 7f934a7fc640 1 -- 172.21.15.38:0/3307445175 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a4694282a0 msgr2=0x55a4697e80a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:35:28.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.427+0000 7f934a7fc640 1 --2- 172.21.15.38:0/3307445175 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a4694282a0 0x55a4697e80a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:28.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.427+0000 7f934a7fc640 1 -- 172.21.15.38:0/3307445175 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a46956cea0 con 0x55a4694075c0 2024-09-21T03:35:28.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.427+0000 7f934affd640 1 --2- 172.21.15.38:0/3307445175 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a4693ff3e0 0x55a469815bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:35:28.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.427+0000 7f934a7fc640 1 --2- 172.21.15.38:0/3307445175 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a4694075c0 0x55a469811430 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f933400b410 tx=0x7f9334062780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea7d2cd8d625c464 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:28.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.427+0000 7f9329ffb640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-21T03:35:28.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.427+0000 7f9329ffb640 1 -- 172.21.15.38:0/3307445175 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9334078dc0 con 0x55a4694075c0 2024-09-21T03:35:28.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.427+0000 7f9329ffb640 1 -- 172.21.15.38:0/3307445175 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(43..43 src has 1..43) ==== 6245+0+0 (secure 0 0 0) 0x7f933405e4a0 con 0x55a4694075c0 2024-09-21T03:35:28.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.427+0000 7f9358687740 1 -- 172.21.15.38:0/3307445175 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a46970fb60 con 0x55a4694075c0 2024-09-21T03:35:28.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.427+0000 7f9329ffb640 1 -- 172.21.15.38:0/3307445175 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f9334067580 con 0x55a4694075c0 2024-09-21T03:35:28.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.427+0000 7f9358687740 10 client.5415 fetch_fsmap learned FSMap version 23 2024-09-21T03:35:28.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.427+0000 7f9358687740 1 -- 172.21.15.38:0/3307445175 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) -- 0x55a4694715b0 con 0x55a4694075c0 2024-09-21T03:35:28.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.427+0000 7f9329ffb640 1 -- 172.21.15.38:0/3307445175 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55a46970fb60 con 0x55a4694075c0 2024-09-21T03:35:28.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.427+0000 7f9329ffb640 1 -- 172.21.15.38:0/3307445175 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 23) ==== 3502+0+0 (secure 0 0 0) 0x7f933405e6a0 con 0x55a4694075c0 2024-09-21T03:35:28.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.427+0000 7f9358687740 10 client.5415 fetch_fsmap finished waiting for FSMap version 23 2024-09-21T03:35:28.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.427+0000 7f9358687740 10 client.5415 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T03:35:28.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.427+0000 7f9358687740 20 client.5415 populate_metadata read hostname 'smithi038' 2024-09-21T03:35:28.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.428+0000 7f9358687740 1 --2- 172.21.15.38:0/3307445175 >> [v2:172.21.15.38:6832/3533947700,v1:172.21.15.38:6834/3533947700] conn(0x55a4693e7080 0x55a46980f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:28.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.428+0000 7f9358687740 4 client.5415 mds_command: new command op to 5226 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:35:28.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.428+0000 7f9358687740 1 -- 172.21.15.38:0/3307445175 --> [v2:172.21.15.38:6832/3533947700,v1:172.21.15.38:6834/3533947700] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a469744860 con 0x55a4693e7080 2024-09-21T03:35:28.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.428+0000 7f934affd640 1 --2- 172.21.15.38:0/3307445175 >> [v2:172.21.15.38:6832/3533947700,v1:172.21.15.38:6834/3533947700] conn(0x55a4693e7080 0x55a46980f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:28.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.428+0000 7f9329ffb640 10 client.5415 ms_handle_connect on v2:172.21.15.38:6832/3533947700 2024-09-21T03:35:28.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.428+0000 7f934affd640 1 --2- 172.21.15.38:0/3307445175 >> [v2:172.21.15.38:6832/3533947700,v1:172.21.15.38:6834/3533947700] conn(0x55a4693e7080 0x55a46980f650 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=129de2cf2cb3d195 server_cookie=ba55719195d314e1 in_seq=0 out_seq=0 2024-09-21T03:35:28.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.429+0000 7f9329ffb640 1 -- 172.21.15.38:0/3307445175 <== mds.0 v2:172.21.15.38:6832/3533947700 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a469744860 con 0x55a4693e7080 2024-09-21T03:35:28.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.429+0000 7f9329ffb640 10 client.5415 handle_command_reply: tid=0 2024-09-21T03:35:28.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.429+0000 7f9358687740 1 client.5415 shutdown 2024-09-21T03:35:28.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.429+0000 7f9358687740 1 -- 172.21.15.38:0/3307445175 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a4694075c0 msgr2=0x55a469811430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:28.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.429+0000 7f9358687740 1 --2- 172.21.15.38:0/3307445175 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a4694075c0 0x55a469811430 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f933400b410 tx=0x7f9334062780 comp rx=0 tx=0).stop 2024-09-21T03:35:28.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.429+0000 7f9358687740 1 -- 172.21.15.38:0/3307445175 shutdown_connections 2024-09-21T03:35:28.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.429+0000 7f9358687740 1 --2- 172.21.15.38:0/3307445175 >> [v2:172.21.15.38:6832/3533947700,v1:172.21.15.38:6834/3533947700] conn(0x55a4693e7080 0x55a46980f650 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:28.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.429+0000 7f9358687740 1 --2- 172.21.15.38:0/3307445175 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a4694282a0 0x55a4697e80a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:28.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.429+0000 7f9358687740 1 --2- 172.21.15.38:0/3307445175 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a4694075c0 0x55a469811430 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:28.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.429+0000 7f9358687740 1 --2- 172.21.15.38:0/3307445175 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a4693ff3e0 0x55a469815bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:28.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.429+0000 7f9358687740 1 -- 172.21.15.38:0/3307445175 >> 172.21.15.38:0/3307445175 conn(0x55a46941c3f0 msgr2=0x55a4697fa280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:28.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.429+0000 7f9329ffb640 0 client.5415 ms_handle_reset on v2:172.21.15.38:6832/3533947700 2024-09-21T03:35:28.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.429+0000 7f9358687740 1 -- 172.21.15.38:0/3307445175 shutdown_connections 2024-09-21T03:35:28.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.429+0000 7f9358687740 1 -- 172.21.15.38:0/3307445175 wait complete. 2024-09-21T03:35:28.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.429+0000 7f9358687740 20 client.5415 trim_cache size 0 max 16384 2024-09-21T03:35:28.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.443+0000 7f9358687740 1 Processor -- start 2024-09-21T03:35:28.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.443+0000 7f9358687740 1 -- start start 2024-09-21T03:35:28.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.444+0000 7f9358687740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a4693ff3e0 0x55a469813e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:28.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.444+0000 7f9358687740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a4694075c0 0x55a4697fa4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:28.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.444+0000 7f9358687740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a4694282a0 0x55a4697ca230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:28.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.444+0000 7f9358687740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f934c201730 con 0x55a4693ff3e0 2024-09-21T03:35:28.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.444+0000 7f9358687740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f934c158980 con 0x55a4694282a0 2024-09-21T03:35:28.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.444+0000 7f9358687740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f934c2010e0 con 0x55a4694075c0 2024-09-21T03:35:28.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.444+0000 7f934affd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a4693ff3e0 0x55a469813e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:28.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.444+0000 7f934affd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a4693ff3e0 0x55a469813e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44696/0 (socket says 172.21.15.38:44696) 2024-09-21T03:35:28.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.444+0000 7f934affd640 1 -- 172.21.15.38:0/3408212067 learned_addr learned my addr 172.21.15.38:0/3408212067 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:35:28.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.444+0000 7f934a7fc640 1 --2- 172.21.15.38:0/3408212067 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a4694075c0 0x55a4697fa4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:28.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.444+0000 7f934b7fe640 1 --2- 172.21.15.38:0/3408212067 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a4694282a0 0x55a4697ca230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:28.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.444+0000 7f934affd640 1 -- 172.21.15.38:0/3408212067 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a4694075c0 msgr2=0x55a4697fa4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:35:28.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.444+0000 7f934affd640 1 --2- 172.21.15.38:0/3408212067 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a4694075c0 0x55a4697fa4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:28.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.444+0000 7f934affd640 1 -- 172.21.15.38:0/3408212067 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a4694282a0 msgr2=0x55a4697ca230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:35:28.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.444+0000 7f934affd640 1 --2- 172.21.15.38:0/3408212067 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a4694282a0 0x55a4697ca230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:28.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.445+0000 7f934affd640 1 -- 172.21.15.38:0/3408212067 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4697ccdb0 con 0x55a4693ff3e0 2024-09-21T03:35:28.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.445+0000 7f934affd640 1 --2- 172.21.15.38:0/3408212067 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a4693ff3e0 0x55a469813e80 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f933c016a40 tx=0x7f933c011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=856e6ded942ea390 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:28.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.445+0000 7f932bfff640 1 -- 172.21.15.38:0/3408212067 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f933c030070 con 0x55a4693ff3e0 2024-09-21T03:35:28.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.445+0000 7f932bfff640 1 -- 172.21.15.38:0/3408212067 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f933c024a50 con 0x55a4693ff3e0 2024-09-21T03:35:28.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.445+0000 7f932bfff640 1 -- 172.21.15.38:0/3408212067 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f933c02b460 con 0x55a4693ff3e0 2024-09-21T03:35:28.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.446+0000 7f9358687740 1 -- 172.21.15.38:0/3408212067 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a4693ff3e0 msgr2=0x55a469813e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:28.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.446+0000 7f9358687740 1 --2- 172.21.15.38:0/3408212067 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a4693ff3e0 0x55a469813e80 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f933c016a40 tx=0x7f933c011000 comp rx=0 tx=0).stop 2024-09-21T03:35:28.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.446+0000 7f9358687740 1 -- 172.21.15.38:0/3408212067 shutdown_connections 2024-09-21T03:35:28.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.446+0000 7f9358687740 1 --2- 172.21.15.38:0/3408212067 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a4694282a0 0x55a4697ca230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:28.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.446+0000 7f9358687740 1 --2- 172.21.15.38:0/3408212067 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a4694075c0 0x55a4697fa4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:28.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.446+0000 7f9358687740 1 --2- 172.21.15.38:0/3408212067 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a4693ff3e0 0x55a469813e80 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:28.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.446+0000 7f9358687740 1 -- 172.21.15.38:0/3408212067 >> 172.21.15.38:0/3408212067 conn(0x55a46941c3f0 msgr2=0x55a4697c6010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:28.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.446+0000 7f9358687740 1 -- 172.21.15.38:0/3408212067 shutdown_connections 2024-09-21T03:35:28.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.446+0000 7f9358687740 1 -- 172.21.15.38:0/3408212067 wait complete. 2024-09-21T03:35:28.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.446+0000 7f9358687740 1 -- start start 2024-09-21T03:35:28.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.447+0000 7f9358687740 1 Processor -- start 2024-09-21T03:35:28.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.448+0000 7f9358687740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a4693ff3e0 0x55a469893bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:28.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.448+0000 7f9358687740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a4694075c0 0x55a4697ca230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:28.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.448+0000 7f9358687740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a4694282a0 0x55a4697ca770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:28.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.448+0000 7f9358687740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55a469827670 con 0x55a4694075c0 2024-09-21T03:35:28.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.448+0000 7f9358687740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55a4697e8880 con 0x55a4693ff3e0 2024-09-21T03:35:28.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.448+0000 7f9358687740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55a469813c40 con 0x55a4694282a0 2024-09-21T03:35:28.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.448+0000 7f9358687740 10 client.0 _ll_register_callbacks cb 0x7f93340629e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:35:28.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.448+0000 7f934a7fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a4694075c0 0x55a4697ca230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:28.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.448+0000 7f934a7fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a4694075c0 0x55a4697ca230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44710/0 (socket says 172.21.15.38:44710) 2024-09-21T03:35:28.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.448+0000 7f934a7fc640 1 -- 172.21.15.38:0/4090634277 learned_addr learned my addr 172.21.15.38:0/4090634277 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:35:28.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.448+0000 7f934b7fe640 1 --2- 172.21.15.38:0/4090634277 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a4694282a0 0x55a4697ca770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:28.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.448+0000 7f934affd640 1 --2- 172.21.15.38:0/4090634277 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a4693ff3e0 0x55a469893bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:28.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.448+0000 7f934a7fc640 1 -- 172.21.15.38:0/4090634277 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a4694282a0 msgr2=0x55a4697ca770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:35:28.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.448+0000 7f934a7fc640 1 --2- 172.21.15.38:0/4090634277 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a4694282a0 0x55a4697ca770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:28.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.448+0000 7f934a7fc640 1 -- 172.21.15.38:0/4090634277 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a4693ff3e0 msgr2=0x55a469893bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:35:28.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.448+0000 7f934a7fc640 1 --2- 172.21.15.38:0/4090634277 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a4693ff3e0 0x55a469893bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:28.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.448+0000 7f934a7fc640 1 -- 172.21.15.38:0/4090634277 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a4697cd000 con 0x55a4694075c0 2024-09-21T03:35:28.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.449+0000 7f934a7fc640 1 --2- 172.21.15.38:0/4090634277 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a4694075c0 0x55a4697ca230 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f933400be00 tx=0x7f9334078df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cffe87a35eff459a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:28.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.449+0000 7f9329ffb640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-21T03:35:28.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.449+0000 7f9329ffb640 1 -- 172.21.15.38:0/4090634277 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f933406c480 con 0x55a4694075c0 2024-09-21T03:35:28.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.449+0000 7f9329ffb640 1 -- 172.21.15.38:0/4090634277 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(43..43 src has 1..43) ==== 6245+0+0 (secure 0 0 0) 0x7f933401bcd0 con 0x55a4694075c0 2024-09-21T03:35:28.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.449+0000 7f9358687740 1 -- 172.21.15.38:0/4090634277 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f933c024a50 con 0x55a4694075c0 2024-09-21T03:35:28.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.449+0000 7f9329ffb640 1 -- 172.21.15.38:0/4090634277 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f9334072050 con 0x55a4694075c0 2024-09-21T03:35:28.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.449+0000 7f9358687740 10 client.5421 fetch_fsmap learned FSMap version 23 2024-09-21T03:35:28.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.449+0000 7f9358687740 1 -- 172.21.15.38:0/4090634277 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) -- 0x55a4697e8bd0 con 0x55a4694075c0 2024-09-21T03:35:28.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.449+0000 7f9329ffb640 1 -- 172.21.15.38:0/4090634277 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f933c024a50 con 0x55a4694075c0 2024-09-21T03:35:28.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.449+0000 7f9329ffb640 1 -- 172.21.15.38:0/4090634277 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 23) ==== 3502+0+0 (secure 0 0 0) 0x7f9334075020 con 0x55a4694075c0 2024-09-21T03:35:28.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.449+0000 7f9358687740 10 client.5421 fetch_fsmap finished waiting for FSMap version 23 2024-09-21T03:35:28.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.449+0000 7f9358687740 10 client.5421 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T03:35:28.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.449+0000 7f9358687740 20 client.5421 populate_metadata read hostname 'smithi038' 2024-09-21T03:35:28.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.449+0000 7f9358687740 1 --2- 172.21.15.38:0/4090634277 >> [v2:172.21.15.38:6832/3533947700,v1:172.21.15.38:6834/3533947700] conn(0x55a469822660 0x55a469822a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:28.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.449+0000 7f9358687740 4 client.5421 mds_command: new command op to 5226 tid=0 multi_id=0 [{"prefix": "damage rm", "damage_id": 2742033437}] 2024-09-21T03:35:28.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.449+0000 7f9358687740 1 -- 172.21.15.38:0/4090634277 --> [v2:172.21.15.38:6832/3533947700,v1:172.21.15.38:6834/3533947700] -- command(tid 0: {"prefix": "damage rm", "damage_id": 2742033437}) -- 0x55a469815860 con 0x55a469822660 2024-09-21T03:35:28.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.449+0000 7f934affd640 1 --2- 172.21.15.38:0/4090634277 >> [v2:172.21.15.38:6832/3533947700,v1:172.21.15.38:6834/3533947700] conn(0x55a469822660 0x55a469822a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:28.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.450+0000 7f9329ffb640 10 client.5421 ms_handle_connect on v2:172.21.15.38:6832/3533947700 2024-09-21T03:35:28.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.450+0000 7f934affd640 1 --2- 172.21.15.38:0/4090634277 >> [v2:172.21.15.38:6832/3533947700,v1:172.21.15.38:6834/3533947700] conn(0x55a469822660 0x55a469822a70 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14c26b91fd3f2526 server_cookie=e967b0aa87641256 in_seq=0 out_seq=0 2024-09-21T03:35:28.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.450+0000 7f9329ffb640 1 -- 172.21.15.38:0/4090634277 <== mds.0 v2:172.21.15.38:6832/3533947700 1 ==== command_reply(tid 0: 0 ) ==== 8+0+0 (crc 0 0 0) 0x55a469815860 con 0x55a469822660 2024-09-21T03:35:28.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.450+0000 7f9329ffb640 10 client.5421 handle_command_reply: tid=0 2024-09-21T03:35:28.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.450+0000 7f9358687740 1 client.5421 shutdown 2024-09-21T03:35:28.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.450+0000 7f9358687740 1 -- 172.21.15.38:0/4090634277 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a4694075c0 msgr2=0x55a4697ca230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:28.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.450+0000 7f9358687740 1 --2- 172.21.15.38:0/4090634277 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a4694075c0 0x55a4697ca230 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f933400be00 tx=0x7f9334078df0 comp rx=0 tx=0).stop 2024-09-21T03:35:28.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.450+0000 7f9358687740 1 -- 172.21.15.38:0/4090634277 shutdown_connections 2024-09-21T03:35:28.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.450+0000 7f9358687740 1 --2- 172.21.15.38:0/4090634277 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a4694282a0 0x55a4697ca770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:28.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.450+0000 7f9358687740 1 --2- 172.21.15.38:0/4090634277 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a4694075c0 0x55a4697ca230 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:28.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.450+0000 7f9358687740 1 --2- 172.21.15.38:0/4090634277 >> [v2:172.21.15.38:6832/3533947700,v1:172.21.15.38:6834/3533947700] conn(0x55a469822660 0x55a469822a70 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:28.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.450+0000 7f9358687740 1 --2- 172.21.15.38:0/4090634277 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a4693ff3e0 0x55a469893bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:28.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.450+0000 7f9329ffb640 0 client.5421 ms_handle_reset on v2:172.21.15.38:6832/3533947700 2024-09-21T03:35:28.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.450+0000 7f9358687740 1 -- 172.21.15.38:0/4090634277 >> 172.21.15.38:0/4090634277 conn(0x55a46941c3f0 msgr2=0x55a469817dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:28.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.450+0000 7f9358687740 1 -- 172.21.15.38:0/4090634277 shutdown_connections 2024-09-21T03:35:28.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.450+0000 7f9358687740 1 -- 172.21.15.38:0/4090634277 wait complete. 2024-09-21T03:35:28.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.450+0000 7f9358687740 20 client.5421 trim_cache size 0 max 16384 2024-09-21T03:35:28.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.451+0000 7f9351638640 1 -- 172.21.15.38:0/865482 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f932c078b90 msgr2=0x7f932c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:28.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.451+0000 7f9351638640 1 --2- 172.21.15.38:0/865482 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f932c078b90 0x7f932c07b0a0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f933c00a400 tx=0x7f933c00d040 comp rx=0 tx=0).stop 2024-09-21T03:35:28.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.451+0000 7f9351638640 1 -- 172.21.15.38:0/865482 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f934c0b31e0 msgr2=0x7f934c153750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:28.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.451+0000 7f9351638640 1 --2- 172.21.15.38:0/865482 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f934c0b31e0 0x7f934c153750 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f933400b440 tx=0x7f933400b900 comp rx=0 tx=0).stop 2024-09-21T03:35:28.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.451+0000 7f9351638640 1 -- 172.21.15.38:0/865482 shutdown_connections 2024-09-21T03:35:28.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.451+0000 7f9351638640 1 --2- 172.21.15.38:0/865482 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f932c078b90 0x7f932c07b0a0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:28.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.451+0000 7f9351638640 1 --2- 172.21.15.38:0/865482 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f934c15c750 0x7f934c15f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:28.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.451+0000 7f9351638640 1 --2- 172.21.15.38:0/865482 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f934c0b31e0 0x7f934c153750 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:28.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.451+0000 7f9351638640 1 --2- 172.21.15.38:0/865482 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f934c0aeeb0 0x7f934c157600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:28.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.451+0000 7f9351638640 1 -- 172.21.15.38:0/865482 >> 172.21.15.38:0/865482 conn(0x7f934c0cdca0 msgr2=0x7f934c06eb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:28.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.451+0000 7f9351638640 1 -- 172.21.15.38:0/865482 shutdown_connections 2024-09-21T03:35:28.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.451+0000 7f9351638640 1 -- 172.21.15.38:0/865482 wait complete. 2024-09-21T03:35:28.464 DEBUG:teuthology.orchestra.run.smithi038:> stdin-killer -- python3 -c ' 2024-09-21T03:35:28.464 DEBUG:teuthology.orchestra.run.smithi038:> import sys 2024-09-21T03:35:28.464 DEBUG:teuthology.orchestra.run.smithi038:> import errno 2024-09-21T03:35:28.464 DEBUG:teuthology.orchestra.run.smithi038:> 2024-09-21T03:35:28.464 DEBUG:teuthology.orchestra.run.smithi038:> try: 2024-09-21T03:35:28.464 DEBUG:teuthology.orchestra.run.smithi038:> f = open("/home/ubuntu/cephtest/mnt.0/subdir/file_to_be_damaged", "w") 2024-09-21T03:35:28.464 DEBUG:teuthology.orchestra.run.smithi038:> f.close() 2024-09-21T03:35:28.464 DEBUG:teuthology.orchestra.run.smithi038:> except IOError as e: 2024-09-21T03:35:28.464 DEBUG:teuthology.orchestra.run.smithi038:> sys.exit(errno.EIO) 2024-09-21T03:35:28.464 DEBUG:teuthology.orchestra.run.smithi038:> ' 2024-09-21T03:35:28.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:35:28.544 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-21T03:35:28.545 DEBUG:teuthology.orchestra.run.smithi038:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T03:35:28.638 DEBUG:teuthology.orchestra.run.smithi195:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T03:35:28.687 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-21T03:35:28.688 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T03:35:28.688 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-21T03:35:28.688 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:35:28.704 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-21T03:35:28.704 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:35:28.761 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T03:35:28.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.918+0000 7f96b36f2640 1 -- 172.21.15.38:0/3611428519 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9694004800 msgr2=0x7f9694004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:28.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.918+0000 7f96b36f2640 1 --2- 172.21.15.38:0/3611428519 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9694004800 0x7f9694004c10 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f96ac0d28f0 tx=0x7f9694005830 comp rx=0 tx=0).stop 2024-09-21T03:35:28.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.918+0000 7f96b36f2640 1 -- 172.21.15.38:0/3611428519 shutdown_connections 2024-09-21T03:35:28.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.918+0000 7f96b36f2640 1 --2- 172.21.15.38:0/3611428519 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9694004800 0x7f9694004c10 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:28.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.918+0000 7f96b36f2640 1 -- 172.21.15.38:0/3611428519 >> 172.21.15.38:0/3611428519 conn(0x7f96ac09d830 msgr2=0x7f96ac0b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:28.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.918+0000 7f96b36f2640 1 -- 172.21.15.38:0/3611428519 shutdown_connections 2024-09-21T03:35:28.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.918+0000 7f96b36f2640 1 -- 172.21.15.38:0/3611428519 wait complete. 2024-09-21T03:35:28.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.920+0000 7f96b36f2640 1 Processor -- start 2024-09-21T03:35:28.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.920+0000 7f96b36f2640 1 -- start start 2024-09-21T03:35:28.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.921+0000 7f96b36f2640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9694004800 0x7f96ac1f5fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:28.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.921+0000 7f96b36f2640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f96ac1f2860 0x7f96ac1f89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:28.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.921+0000 7f96b36f2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f96ac1594b0 0x7f96ac1598c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:28.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.921+0000 7f96b36f2640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f96ac0b9c50 con 0x7f96ac1594b0 2024-09-21T03:35:28.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.921+0000 7f96b36f2640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f96ac0b9ad0 con 0x7f9694004800 2024-09-21T03:35:28.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.921+0000 7f96b36f2640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f96ac0b9dd0 con 0x7f96ac1f2860 2024-09-21T03:35:28.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.921+0000 7f96b1c8f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f96ac1594b0 0x7f96ac1598c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:28.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.922+0000 7f96b1c8f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f96ac1594b0 0x7f96ac1598c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44732/0 (socket says 172.21.15.38:44732) 2024-09-21T03:35:28.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.922+0000 7f96b1c8f640 1 -- 172.21.15.38:0/2945634899 learned_addr learned my addr 172.21.15.38:0/2945634899 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:35:28.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.922+0000 7f96b0c8d640 1 --2- 172.21.15.38:0/2945634899 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f96ac1f2860 0x7f96ac1f89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:28.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.922+0000 7f96b148e640 1 --2- 172.21.15.38:0/2945634899 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9694004800 0x7f96ac1f5fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:28.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.922+0000 7f96b1c8f640 1 -- 172.21.15.38:0/2945634899 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f96ac1f2860 msgr2=0x7f96ac1f89e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:28.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.922+0000 7f96b1c8f640 1 --2- 172.21.15.38:0/2945634899 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f96ac1f2860 0x7f96ac1f89e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:28.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.922+0000 7f96b1c8f640 1 -- 172.21.15.38:0/2945634899 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9694004800 msgr2=0x7f96ac1f5fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:28.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.922+0000 7f96b1c8f640 1 --2- 172.21.15.38:0/2945634899 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9694004800 0x7f96ac1f5fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:28.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.922+0000 7f96b1c8f640 1 -- 172.21.15.38:0/2945634899 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96ac06fb70 con 0x7f96ac1594b0 2024-09-21T03:35:28.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.922+0000 7f96b148e640 1 --2- 172.21.15.38:0/2945634899 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9694004800 0x7f96ac1f5fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:35:28.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.922+0000 7f96b1c8f640 1 --2- 172.21.15.38:0/2945634899 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f96ac1594b0 0x7f96ac1598c0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f96a4000fb0 tx=0x7f96a4006710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:28.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.922+0000 7f96a9ffb640 1 -- 172.21.15.38:0/2945634899 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96a40107d0 con 0x7f96ac1594b0 2024-09-21T03:35:28.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.922+0000 7f96a9ffb640 1 -- 172.21.15.38:0/2945634899 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f96a4016070 con 0x7f96ac1594b0 2024-09-21T03:35:28.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.922+0000 7f96b36f2640 1 -- 172.21.15.38:0/2945634899 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96ac1f95c0 con 0x7f96ac1594b0 2024-09-21T03:35:28.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.922+0000 7f96a9ffb640 1 -- 172.21.15.38:0/2945634899 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96a4010d40 con 0x7f96ac1594b0 2024-09-21T03:35:28.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.922+0000 7f96b36f2640 1 -- 172.21.15.38:0/2945634899 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96ac1f5500 con 0x7f96ac1594b0 2024-09-21T03:35:28.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.923+0000 7f96b36f2640 1 -- 172.21.15.38:0/2945634899 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96ac1f9330 con 0x7f96ac1594b0 2024-09-21T03:35:28.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.924+0000 7f96a9ffb640 1 -- 172.21.15.38:0/2945634899 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f96a4004d10 con 0x7f96ac1594b0 2024-09-21T03:35:28.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.925+0000 7f96a9ffb640 1 --2- 172.21.15.38:0/2945634899 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9698078bc0 0x7f969807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:28.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.925+0000 7f96a9ffb640 1 -- 172.21.15.38:0/2945634899 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6245+0+0 (secure 0 0 0) 0x7f96a409b500 con 0x7f96ac1594b0 2024-09-21T03:35:28.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.925+0000 7f96b148e640 1 --2- 172.21.15.38:0/2945634899 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9698078bc0 0x7f969807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:28.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.925+0000 7f96b148e640 1 --2- 172.21.15.38:0/2945634899 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9698078bc0 0x7f969807b0d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f969c006fd0 tx=0x7f969c006f20 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:28.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:28.929+0000 7f96a9ffb640 1 -- 172.21.15.38:0/2945634899 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f96ac06fb70 con 0x7f96ac1594b0 2024-09-21T03:35:29.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:29.129+0000 7f96b36f2640 1 -- 172.21.15.38:0/2945634899 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f96ac0cec70 con 0x7f96ac1594b0 2024-09-21T03:35:29.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:29.129+0000 7f96a9ffb640 1 -- 172.21.15.38:0/2945634899 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v43) ==== 80+0+122 (secure 0 0 0) 0x7f96ac1592d0 con 0x7f96ac1594b0 2024-09-21T03:35:29.131 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6834/1865253656 2024-09-22T03:34:47.895300+0000 2024-09-21T03:35:29.132 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6832/1865253656 2024-09-22T03:34:47.895300+0000 2024-09-21T03:35:29.132 INFO:teuthology.orchestra.run.smithi038.stderr:listed 2 entries 2024-09-21T03:35:29.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:29.132+0000 7f96b36f2640 1 -- 172.21.15.38:0/2945634899 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9698078bc0 msgr2=0x7f969807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:29.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:29.132+0000 7f96b36f2640 1 --2- 172.21.15.38:0/2945634899 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9698078bc0 0x7f969807b0d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f969c006fd0 tx=0x7f969c006f20 comp rx=0 tx=0).stop 2024-09-21T03:35:29.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:29.132+0000 7f96b36f2640 1 -- 172.21.15.38:0/2945634899 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f96ac1594b0 msgr2=0x7f96ac1598c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:29.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:29.132+0000 7f96b36f2640 1 --2- 172.21.15.38:0/2945634899 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f96ac1594b0 0x7f96ac1598c0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f96a4000fb0 tx=0x7f96a4006710 comp rx=0 tx=0).stop 2024-09-21T03:35:29.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:29.132+0000 7f96b36f2640 1 -- 172.21.15.38:0/2945634899 shutdown_connections 2024-09-21T03:35:29.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:29.132+0000 7f96b36f2640 1 --2- 172.21.15.38:0/2945634899 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f96ac1594b0 0x7f96ac1598c0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:29.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:29.132+0000 7f96b36f2640 1 --2- 172.21.15.38:0/2945634899 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9698078bc0 0x7f969807b0d0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:29.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:29.132+0000 7f96b36f2640 1 --2- 172.21.15.38:0/2945634899 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f96ac1f2860 0x7f96ac1f89e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:29.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:29.132+0000 7f96b36f2640 1 --2- 172.21.15.38:0/2945634899 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9694004800 0x7f96ac1f5fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:29.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:29.132+0000 7f96b36f2640 1 -- 172.21.15.38:0/2945634899 >> 172.21.15.38:0/2945634899 conn(0x7f96ac09d830 msgr2=0x7f96ac15e030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:29.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:29.132+0000 7f96b36f2640 1 -- 172.21.15.38:0/2945634899 shutdown_connections 2024-09-21T03:35:29.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:29.132+0000 7f96b36f2640 1 -- 172.21.15.38:0/2945634899 wait complete. 2024-09-21T03:35:29.145 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-21T03:35:29.145 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:35:29.145 DEBUG:teuthology.orchestra.run.smithi038:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-21T03:35:29.174 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi038.front.sepia.ceph.com 2024-09-21T03:35:29.175 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:35:29.175 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T03:35:29.231 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T03:35:29.231 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-09-21T03:35:29.231 DEBUG:teuthology.orchestra.run.smithi195:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:35:29.250 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056014,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497444k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=8978 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248720k,nr_inodes=812180,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497444k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T03:35:29.251 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T03:35:29.251 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-21T03:35:29.251 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:35:29.286 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056795,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2354 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T03:35:29.287 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T03:35:29.287 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-09-21T03:35:29.287 DEBUG:teuthology.orchestra.run.smithi195:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:35:29.310 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056014,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497444k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=8978 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248720k,nr_inodes=812180,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497444k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T03:35:29.310 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:35:29.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:29.441+0000 7fa94455e640 1 -- 172.21.15.38:0/3802611676 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa93c0b2e40 msgr2=0x7fa93c0b3250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:29.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:29.441+0000 7fa94455e640 1 --2- 172.21.15.38:0/3802611676 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa93c0b2e40 0x7fa93c0b3250 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fa93c0d5040 tx=0x7fa93402a560 comp rx=0 tx=0).stop 2024-09-21T03:35:29.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:29.441+0000 7fa94455e640 1 -- 172.21.15.38:0/3802611676 shutdown_connections 2024-09-21T03:35:29.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:29.441+0000 7fa94455e640 1 --2- 172.21.15.38:0/3802611676 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa93c0b2e40 0x7fa93c0b3250 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:29.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:29.441+0000 7fa94455e640 1 --2- 172.21.15.38:0/3802611676 >> v2:172.21.15.195:3300/0 conn(0x7fa93c162f00 0x7fa93c0d24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:29.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:29.441+0000 7fa94455e640 1 -- 172.21.15.38:0/3802611676 >> 172.21.15.38:0/3802611676 conn(0x7fa93c0b6b20 msgr2=0x7fa93c0d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:29.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:29.441+0000 7fa94455e640 1 -- 172.21.15.38:0/3802611676 shutdown_connections 2024-09-21T03:35:29.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:29.441+0000 7fa94455e640 1 -- 172.21.15.38:0/3802611676 wait complete. 2024-09-21T03:35:29.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:29.443+0000 7fa94455e640 1 Processor -- start 2024-09-21T03:35:29.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:29.443+0000 7fa94455e640 1 -- start start 2024-09-21T03:35:29.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:29.444+0000 7fa94455e640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa93c0b2e40 0x7fa93c0ced50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:29.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:29.444+0000 7fa94455e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa93c162280 0x7fa93c0b0b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:29.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:29.444+0000 7fa94455e640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa93c06cde0 0x7fa93c06d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:29.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:29.444+0000 7fa94455e640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa93c15e4b0 con 0x7fa93c162280 2024-09-21T03:35:29.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:29.444+0000 7fa94455e640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fa93c0b39a0 con 0x7fa93c0b2e40 2024-09-21T03:35:29.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:29.444+0000 7fa94455e640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fa93c15e630 con 0x7fa93c06cde0 2024-09-21T03:35:29.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:29.444+0000 7fa941af9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa93c162280 0x7fa93c0b0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:29.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:29.444+0000 7fa941af9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa93c162280 0x7fa93c0b0b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44746/0 (socket says 172.21.15.38:44746) 2024-09-21T03:35:29.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:29.444+0000 7fa941af9640 1 -- 172.21.15.38:0/628129236 learned_addr learned my addr 172.21.15.38:0/628129236 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:35:29.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:29.445+0000 7fa9422fa640 1 --2- 172.21.15.38:0/628129236 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa93c0b2e40 0x7fa93c0ced50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:29.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:29.444+0000 7fa942afb640 1 --2- 172.21.15.38:0/628129236 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa93c06cde0 0x7fa93c06d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:29.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:29.445+0000 7fa941af9640 1 -- 172.21.15.38:0/628129236 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa93c06cde0 msgr2=0x7fa93c06d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:29.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:29.445+0000 7fa941af9640 1 --2- 172.21.15.38:0/628129236 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa93c06cde0 0x7fa93c06d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:29.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:29.445+0000 7fa941af9640 1 -- 172.21.15.38:0/628129236 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa93c0b2e40 msgr2=0x7fa93c0ced50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:29.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:29.445+0000 7fa941af9640 1 --2- 172.21.15.38:0/628129236 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa93c0b2e40 0x7fa93c0ced50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:29.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:29.445+0000 7fa941af9640 1 -- 172.21.15.38:0/628129236 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa93c06d730 con 0x7fa93c162280 2024-09-21T03:35:29.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:29.445+0000 7fa942afb640 1 --2- 172.21.15.38:0/628129236 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa93c06cde0 0x7fa93c06d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:35:29.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:29.445+0000 7fa9422fa640 1 --2- 172.21.15.38:0/628129236 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa93c0b2e40 0x7fa93c0ced50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:35:29.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:29.445+0000 7fa941af9640 1 --2- 172.21.15.38:0/628129236 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa93c162280 0x7fa93c0b0b10 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fa93000c3c0 tx=0x7fa93000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:29.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:29.445+0000 7fa93affd640 1 -- 172.21.15.38:0/628129236 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa930017070 con 0x7fa93c162280 2024-09-21T03:35:29.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:29.445+0000 7fa93affd640 1 -- 172.21.15.38:0/628129236 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa9300035f0 con 0x7fa93c162280 2024-09-21T03:35:29.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:29.445+0000 7fa93affd640 1 -- 172.21.15.38:0/628129236 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa930003a80 con 0x7fa93c162280 2024-09-21T03:35:29.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:29.445+0000 7fa94455e640 1 -- 172.21.15.38:0/628129236 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa93c0ad030 con 0x7fa93c162280 2024-09-21T03:35:29.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:29.445+0000 7fa94455e640 1 -- 172.21.15.38:0/628129236 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa93c0ad5f0 con 0x7fa93c162280 2024-09-21T03:35:29.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:29.446+0000 7fa94455e640 1 -- 172.21.15.38:0/628129236 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa93c162f00 con 0x7fa93c162280 2024-09-21T03:35:29.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:29.447+0000 7fa93affd640 1 -- 172.21.15.38:0/628129236 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa93001a070 con 0x7fa93c162280 2024-09-21T03:35:29.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:29.447+0000 7fa93affd640 1 --2- 172.21.15.38:0/628129236 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa928078b90 0x7fa92807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:29.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:29.447+0000 7fa93affd640 1 -- 172.21.15.38:0/628129236 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6245+0+0 (secure 0 0 0) 0x7fa93009a270 con 0x7fa93c162280 2024-09-21T03:35:29.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:29.448+0000 7fa9422fa640 1 --2- 172.21.15.38:0/628129236 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa928078b90 0x7fa92807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:29.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:29.448+0000 7fa9422fa640 1 --2- 172.21.15.38:0/628129236 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa928078b90 0x7fa92807b0a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fa92c005fd0 tx=0x7fa92c009340 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:29.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:29.452+0000 7fa93affd640 1 -- 172.21.15.38:0/628129236 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fa93c06d730 con 0x7fa93c162280 2024-09-21T03:35:29.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:29.680+0000 7fa94455e640 1 -- 172.21.15.38:0/628129236 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa93c076230 con 0x7fa93c162280 2024-09-21T03:35:29.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:29.681+0000 7fa93affd640 1 -- 172.21.15.38:0/628129236 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+5230 (secure 0 0 0) 0x7fa930062490 con 0x7fa93c162280 2024-09-21T03:35:29.683 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:35:29.683 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":23,"btime":"2024-09-21T03:34:51:918500+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4757,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/1905955399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":1905955399},{"type":"v1","addr":"172.21.15.195:6836","nonce":1905955399}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4936,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/2989374027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":2989374027},{"type":"v1","addr":"172.21.15.195:6837","nonce":2989374027}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4942,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/3531731501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6833","nonce":3531731501},{"type":"v1","addr":"172.21.15.38:6835","nonce":3531731501}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:34:34.009269+0000","modified":"2024-09-21T03:34:51.918491+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":41,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5226},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5226":{"gid":5226,"name":"c","rank":0,"incarnation":20,"state":"up:active","state_seq":4,"addr":"172.21.15.38:6834/3533947700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":3533947700},{"type":"v1","addr":"172.21.15.38:6834","nonce":3533947700}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5226,"qdb_cluster":[5226]},"id":2}]} 2024-09-21T03:35:29.683 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 23 2024-09-21T03:35:29.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:29.683+0000 7fa94455e640 1 -- 172.21.15.38:0/628129236 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa928078b90 msgr2=0x7fa92807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:29.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:29.683+0000 7fa94455e640 1 --2- 172.21.15.38:0/628129236 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa928078b90 0x7fa92807b0a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fa92c005fd0 tx=0x7fa92c009340 comp rx=0 tx=0).stop 2024-09-21T03:35:29.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:29.683+0000 7fa94455e640 1 -- 172.21.15.38:0/628129236 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa93c162280 msgr2=0x7fa93c0b0b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:29.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:29.683+0000 7fa94455e640 1 --2- 172.21.15.38:0/628129236 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa93c162280 0x7fa93c0b0b10 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fa93000c3c0 tx=0x7fa93000c880 comp rx=0 tx=0).stop 2024-09-21T03:35:29.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:29.683+0000 7fa94455e640 1 -- 172.21.15.38:0/628129236 shutdown_connections 2024-09-21T03:35:29.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:29.683+0000 7fa94455e640 1 --2- 172.21.15.38:0/628129236 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa928078b90 0x7fa92807b0a0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:29.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:29.683+0000 7fa94455e640 1 --2- 172.21.15.38:0/628129236 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa93c06cde0 0x7fa93c06d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:29.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:29.683+0000 7fa94455e640 1 --2- 172.21.15.38:0/628129236 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa93c162280 0x7fa93c0b0b10 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:29.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:29.683+0000 7fa94455e640 1 --2- 172.21.15.38:0/628129236 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa93c0b2e40 0x7fa93c0ced50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:29.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:29.683+0000 7fa94455e640 1 -- 172.21.15.38:0/628129236 >> 172.21.15.38:0/628129236 conn(0x7fa93c0b6b20 msgr2=0x7fa93c06da30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:29.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:29.683+0000 7fa94455e640 1 -- 172.21.15.38:0/628129236 shutdown_connections 2024-09-21T03:35:29.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:29.683+0000 7fa94455e640 1 -- 172.21.15.38:0/628129236 wait complete. 2024-09-21T03:35:29.697 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:35:29.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:29.812+0000 7fe1eb343640 1 -- 172.21.15.38:0/3914992139 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe1d0003a00 msgr2=0x7fe1d0003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:29.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:29.812+0000 7fe1eb343640 1 --2- 172.21.15.38:0/3914992139 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe1d0003a00 0x7fe1d0003e70 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fe1e000c470 tx=0x7fe1d0005830 comp rx=0 tx=0).stop 2024-09-21T03:35:29.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:29.812+0000 7fe1e98e0640 1 -- 172.21.15.38:0/3914992139 reap_dead start 2024-09-21T03:35:29.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:29.812+0000 7fe1eb343640 1 -- 172.21.15.38:0/3914992139 shutdown_connections 2024-09-21T03:35:29.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:29.812+0000 7fe1eb343640 1 -- 172.21.15.38:0/3914992139 >> 172.21.15.38:0/3914992139 conn(0x7fe1e40b6b20 msgr2=0x7fe1e40d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:29.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:29.812+0000 7fe1eb343640 1 -- 172.21.15.38:0/3914992139 shutdown_connections 2024-09-21T03:35:29.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:29.812+0000 7fe1eb343640 1 -- 172.21.15.38:0/3914992139 wait complete. 2024-09-21T03:35:29.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:29.814+0000 7fe1eb343640 1 Processor -- start 2024-09-21T03:35:29.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:29.814+0000 7fe1eb343640 1 -- start start 2024-09-21T03:35:29.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:29.815+0000 7fe1eb343640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe1d0003a00 0x7fe1e4158360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:29.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:29.815+0000 7fe1eb343640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe1e415b130 0x7fe1e41f9250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:29.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:29.815+0000 7fe1eb343640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe1e40ca0c0 0x7fe1e414ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:29.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:29.815+0000 7fe1eb343640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fe1e4162690 con 0x7fe1e40ca0c0 2024-09-21T03:35:29.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:29.815+0000 7fe1eb343640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fe1e40b39a0 con 0x7fe1e415b130 2024-09-21T03:35:29.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:29.815+0000 7fe1eb343640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fe1e4162810 con 0x7fe1d0003a00 2024-09-21T03:35:29.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:29.815+0000 7fe1e98e0640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe1e40ca0c0 0x7fe1e414ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:29.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:29.815+0000 7fe1e98e0640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe1e40ca0c0 0x7fe1e414ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44778/0 (socket says 172.21.15.38:44778) 2024-09-21T03:35:29.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:29.815+0000 7fe1e98e0640 1 -- 172.21.15.38:0/3200678684 learned_addr learned my addr 172.21.15.38:0/3200678684 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:35:29.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:29.816+0000 7fe1e88de640 1 --2- 172.21.15.38:0/3200678684 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe1e415b130 0x7fe1e41f9250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:29.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:29.816+0000 7fe1e90df640 1 --2- 172.21.15.38:0/3200678684 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe1d0003a00 0x7fe1e4158360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:29.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:29.816+0000 7fe1e98e0640 1 -- 172.21.15.38:0/3200678684 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe1d0003a00 msgr2=0x7fe1e4158360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:29.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:29.816+0000 7fe1e98e0640 1 --2- 172.21.15.38:0/3200678684 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe1d0003a00 0x7fe1e4158360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:29.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:29.816+0000 7fe1e98e0640 1 -- 172.21.15.38:0/3200678684 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe1e415b130 msgr2=0x7fe1e41f9250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:29.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:29.816+0000 7fe1e98e0640 1 --2- 172.21.15.38:0/3200678684 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe1e415b130 0x7fe1e41f9250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:29.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:29.816+0000 7fe1e98e0640 1 -- 172.21.15.38:0/3200678684 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1e41588a0 con 0x7fe1e40ca0c0 2024-09-21T03:35:29.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:29.816+0000 7fe1e90df640 1 --2- 172.21.15.38:0/3200678684 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe1d0003a00 0x7fe1e4158360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:35:29.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:29.816+0000 7fe1e98e0640 1 --2- 172.21.15.38:0/3200678684 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe1e40ca0c0 0x7fe1e414ab70 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fe1dc005fd0 tx=0x7fe1dc00f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:29.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:29.816+0000 7fe1d9ffb640 1 -- 172.21.15.38:0/3200678684 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1dc004b20 con 0x7fe1e40ca0c0 2024-09-21T03:35:29.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:29.816+0000 7fe1d9ffb640 1 -- 172.21.15.38:0/3200678684 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe1dc004cc0 con 0x7fe1e40ca0c0 2024-09-21T03:35:29.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:29.816+0000 7fe1eb343640 1 -- 172.21.15.38:0/3200678684 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1e40cecc0 con 0x7fe1e40ca0c0 2024-09-21T03:35:29.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:29.816+0000 7fe1d9ffb640 1 -- 172.21.15.38:0/3200678684 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1dc003bd0 con 0x7fe1e40ca0c0 2024-09-21T03:35:29.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:29.816+0000 7fe1eb343640 1 -- 172.21.15.38:0/3200678684 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1e40cf280 con 0x7fe1e40ca0c0 2024-09-21T03:35:29.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:29.817+0000 7fe1eb343640 1 -- 172.21.15.38:0/3200678684 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1e4162f00 con 0x7fe1e40ca0c0 2024-09-21T03:35:29.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:29.818+0000 7fe1d9ffb640 1 -- 172.21.15.38:0/3200678684 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fe1dc004030 con 0x7fe1e40ca0c0 2024-09-21T03:35:29.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:29.819+0000 7fe1d9ffb640 1 --2- 172.21.15.38:0/3200678684 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fe1c8078b90 0x7fe1c807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:29.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:29.819+0000 7fe1d9ffb640 1 -- 172.21.15.38:0/3200678684 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6245+0+0 (secure 0 0 0) 0x7fe1dc09b790 con 0x7fe1e40ca0c0 2024-09-21T03:35:29.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:29.819+0000 7fe1e90df640 1 --2- 172.21.15.38:0/3200678684 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fe1c8078b90 0x7fe1c807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:29.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:29.819+0000 7fe1e90df640 1 --2- 172.21.15.38:0/3200678684 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fe1c8078b90 0x7fe1c807b0a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fe1e0033000 tx=0x7fe1e0000fe0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:29.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:29.823+0000 7fe1d9ffb640 1 -- 172.21.15.38:0/3200678684 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fe1dc063a30 con 0x7fe1e40ca0c0 2024-09-21T03:35:30.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:30.048+0000 7fe1eb343640 1 -- 172.21.15.38:0/3200678684 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe1e40b9820 con 0x7fe1e40ca0c0 2024-09-21T03:35:30.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:30.048+0000 7fe1d9ffb640 1 -- 172.21.15.38:0/3200678684 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+5230 (secure 0 0 0) 0x7fe1dc063330 con 0x7fe1e40ca0c0 2024-09-21T03:35:30.051 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:35:30.052 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":23,"btime":"2024-09-21T03:34:51:918500+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4757,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/1905955399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":1905955399},{"type":"v1","addr":"172.21.15.195:6836","nonce":1905955399}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4936,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/2989374027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":2989374027},{"type":"v1","addr":"172.21.15.195:6837","nonce":2989374027}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4942,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/3531731501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6833","nonce":3531731501},{"type":"v1","addr":"172.21.15.38:6835","nonce":3531731501}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:34:34.009269+0000","modified":"2024-09-21T03:34:51.918491+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":41,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5226},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5226":{"gid":5226,"name":"c","rank":0,"incarnation":20,"state":"up:active","state_seq":4,"addr":"172.21.15.38:6834/3533947700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":3533947700},{"type":"v1","addr":"172.21.15.38:6834","nonce":3533947700}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5226,"qdb_cluster":[5226]},"id":2}]} 2024-09-21T03:35:30.052 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 23 2024-09-21T03:35:30.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:30.051+0000 7fe1eb343640 1 -- 172.21.15.38:0/3200678684 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fe1c8078b90 msgr2=0x7fe1c807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:30.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:30.051+0000 7fe1eb343640 1 --2- 172.21.15.38:0/3200678684 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fe1c8078b90 0x7fe1c807b0a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fe1e0033000 tx=0x7fe1e0000fe0 comp rx=0 tx=0).stop 2024-09-21T03:35:30.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:30.051+0000 7fe1eb343640 1 -- 172.21.15.38:0/3200678684 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe1e40ca0c0 msgr2=0x7fe1e414ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:30.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:30.051+0000 7fe1eb343640 1 --2- 172.21.15.38:0/3200678684 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe1e40ca0c0 0x7fe1e414ab70 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fe1dc005fd0 tx=0x7fe1dc00f3c0 comp rx=0 tx=0).stop 2024-09-21T03:35:30.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:30.051+0000 7fe1eb343640 1 -- 172.21.15.38:0/3200678684 shutdown_connections 2024-09-21T03:35:30.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:30.051+0000 7fe1eb343640 1 --2- 172.21.15.38:0/3200678684 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe1e40ca0c0 0x7fe1e414ab70 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:30.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:30.051+0000 7fe1eb343640 1 --2- 172.21.15.38:0/3200678684 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe1e415b130 0x7fe1e41f9250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:30.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:30.051+0000 7fe1eb343640 1 --2- 172.21.15.38:0/3200678684 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fe1c8078b90 0x7fe1c807b0a0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:30.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:30.051+0000 7fe1eb343640 1 --2- 172.21.15.38:0/3200678684 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe1d0003a00 0x7fe1e4158360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:30.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:30.051+0000 7fe1eb343640 1 -- 172.21.15.38:0/3200678684 >> 172.21.15.38:0/3200678684 conn(0x7fe1e40b6b20 msgr2=0x7fe1e4068d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:30.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:30.051+0000 7fe1eb343640 1 -- 172.21.15.38:0/3200678684 shutdown_connections 2024-09-21T03:35:30.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:30.051+0000 7fe1eb343640 1 -- 172.21.15.38:0/3200678684 wait complete. 2024-09-21T03:35:30.065 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T03:35:30.106 DEBUG:teuthology.orchestra.run.smithi038:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T03:35:30.111 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T03:35:30.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:30.182+0000 7f664afe4640 1 -- 172.21.15.38:0/1620874808 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f662c005250 msgr2=0x7f662c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:30.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:30.182+0000 7f664afe4640 1 --2- 172.21.15.38:0/1620874808 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f662c005250 0x7f662c00b9c0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f663c001dc0 tx=0x7f662c005830 comp rx=0 tx=0).stop 2024-09-21T03:35:30.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:30.182+0000 7f664afe4640 1 -- 172.21.15.38:0/1620874808 shutdown_connections 2024-09-21T03:35:30.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:30.182+0000 7f664afe4640 1 --2- 172.21.15.38:0/1620874808 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f662c005250 0x7f662c00b9c0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:30.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:30.182+0000 7f664afe4640 1 -- 172.21.15.38:0/1620874808 >> 172.21.15.38:0/1620874808 conn(0x7f6644072990 msgr2=0x7f6644072da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:30.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:30.182+0000 7f664afe4640 1 -- 172.21.15.38:0/1620874808 shutdown_connections 2024-09-21T03:35:30.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:30.182+0000 7f664afe4640 1 -- 172.21.15.38:0/1620874808 wait complete. 2024-09-21T03:35:30.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:30.184+0000 7f664afe4640 1 Processor -- start 2024-09-21T03:35:30.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:30.184+0000 7f664afe4640 1 -- start start 2024-09-21T03:35:30.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:30.185+0000 7f664afe4640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f662c005250 0x7f6644157200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:30.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:30.185+0000 7f664afe4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f66441f66e0 0x7f66440b2810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:30.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:30.185+0000 7f664afe4640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f66440b2d50 0x7f66441f92a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:30.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:30.185+0000 7f664afe4640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f664415aa80 con 0x7f66441f66e0 2024-09-21T03:35:30.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:30.185+0000 7f664afe4640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f664415a900 con 0x7f66440b2d50 2024-09-21T03:35:30.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:30.185+0000 7f664afe4640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f664415e230 con 0x7f662c005250 2024-09-21T03:35:30.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:30.185+0000 7f6643fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f66441f66e0 0x7f66440b2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:30.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:30.185+0000 7f6643fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f66441f66e0 0x7f66440b2810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44796/0 (socket says 172.21.15.38:44796) 2024-09-21T03:35:30.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:30.185+0000 7f6643fff640 1 -- 172.21.15.38:0/3776620603 learned_addr learned my addr 172.21.15.38:0/3776620603 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:35:30.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:30.185+0000 7f6648d80640 1 --2- 172.21.15.38:0/3776620603 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f662c005250 0x7f6644157200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:30.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:30.185+0000 7f6649581640 1 --2- 172.21.15.38:0/3776620603 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f66440b2d50 0x7f66441f92a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:30.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:30.185+0000 7f6643fff640 1 -- 172.21.15.38:0/3776620603 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f662c005250 msgr2=0x7f6644157200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:30.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:30.185+0000 7f6643fff640 1 --2- 172.21.15.38:0/3776620603 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f662c005250 0x7f6644157200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:30.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:30.185+0000 7f6643fff640 1 -- 172.21.15.38:0/3776620603 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f66440b2d50 msgr2=0x7f66441f92a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:30.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:30.185+0000 7f6643fff640 1 --2- 172.21.15.38:0/3776620603 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f66440b2d50 0x7f66441f92a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:30.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:30.185+0000 7f6643fff640 1 -- 172.21.15.38:0/3776620603 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66440b3160 con 0x7f66441f66e0 2024-09-21T03:35:30.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:30.186+0000 7f6643fff640 1 --2- 172.21.15.38:0/3776620603 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f66441f66e0 0x7f66440b2810 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f6638011ab0 tx=0x7f6638011f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:30.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:30.186+0000 7f66417fa640 1 -- 172.21.15.38:0/3776620603 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f663801e070 con 0x7f66441f66e0 2024-09-21T03:35:30.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:30.186+0000 7f66417fa640 1 -- 172.21.15.38:0/3776620603 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f663800fd90 con 0x7f66441f66e0 2024-09-21T03:35:30.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:30.186+0000 7f664afe4640 1 -- 172.21.15.38:0/3776620603 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66440ada70 con 0x7f66441f66e0 2024-09-21T03:35:30.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:30.186+0000 7f66417fa640 1 -- 172.21.15.38:0/3776620603 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6638019670 con 0x7f66441f66e0 2024-09-21T03:35:30.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:30.186+0000 7f664afe4640 1 -- 172.21.15.38:0/3776620603 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66440ae030 con 0x7f66441f66e0 2024-09-21T03:35:30.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:30.187+0000 7f664afe4640 1 -- 172.21.15.38:0/3776620603 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f664415e790 con 0x7f66441f66e0 2024-09-21T03:35:30.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:30.188+0000 7f66417fa640 1 -- 172.21.15.38:0/3776620603 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f6638021070 con 0x7f66441f66e0 2024-09-21T03:35:30.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:30.188+0000 7f66417fa640 1 --2- 172.21.15.38:0/3776620603 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6630078b90 0x7f663007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:30.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:30.188+0000 7f66417fa640 1 -- 172.21.15.38:0/3776620603 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6245+0+0 (secure 0 0 0) 0x7f663809b8e0 con 0x7f66441f66e0 2024-09-21T03:35:30.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:30.188+0000 7f6648d80640 1 --2- 172.21.15.38:0/3776620603 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6630078b90 0x7f663007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:30.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:30.189+0000 7f6648d80640 1 --2- 172.21.15.38:0/3776620603 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6630078b90 0x7f663007b0a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f66440cec40 tx=0x7f66340067a0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:30.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:30.193+0000 7f66417fa640 1 -- 172.21.15.38:0/3776620603 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f66440b3160 con 0x7f66441f66e0 2024-09-21T03:35:30.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:30.383+0000 7f664afe4640 1 -- 172.21.15.38:0/3776620603 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f66440b74c0 con 0x7f66441f66e0 2024-09-21T03:35:30.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:30.384+0000 7f66417fa640 1 -- 172.21.15.38:0/3776620603 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) ==== 74+0+17190 (secure 0 0 0) 0x7f6638063540 con 0x7f66441f66e0 2024-09-21T03:35:30.386 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:35:30.386 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":43,"fsid":"9d50d66f-656b-4fe2-8e3b-fe53b79c4d43","created":"2024-09-21T03:33:10.677865+0000","modified":"2024-09-21T03:35:17.098344+0000","last_up_change":"2024-09-21T03:33:24.023421+0000","last_in_change":"2024-09-21T03:33:14.296193+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T03:33:22.992573+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-09-21T03:34:32.080636+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-09-21T03:34:32.959110+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dbab183f-7e63-48b8-960a-ae55fd5bf565","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6801","nonce":2381639433}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6803","nonce":2381639433}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6807","nonce":2381639433}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6805","nonce":2381639433}]},"public_addr":"172.21.15.38:6801/2381639433","cluster_addr":"172.21.15.38:6803/2381639433","heartbeat_back_addr":"172.21.15.38:6807/2381639433","heartbeat_front_addr":"172.21.15.38:6805/2381639433","state":["exists","up"]},{"osd":1,"uuid":"fa1c0e8a-ebc1-4359-8146-bd5342abbc16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6817","nonce":1491818732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6819","nonce":1491818732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6823","nonce":1491818732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6821","nonce":1491818732}]},"public_addr":"172.21.15.38:6817/1491818732","cluster_addr":"172.21.15.38:6819/1491818732","heartbeat_back_addr":"172.21.15.38:6823/1491818732","heartbeat_front_addr":"172.21.15.38:6821/1491818732","state":["exists","up"]},{"osd":2,"uuid":"2ed3cd2d-385d-4fc6-9869-214552a84803","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6825","nonce":2577858902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6827","nonce":2577858902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6831","nonce":2577858902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6829","nonce":2577858902}]},"public_addr":"172.21.15.38:6825/2577858902","cluster_addr":"172.21.15.38:6827/2577858902","heartbeat_back_addr":"172.21.15.38:6831/2577858902","heartbeat_front_addr":"172.21.15.38:6829/2577858902","state":["exists","up"]},{"osd":3,"uuid":"62d1d95e-919d-4250-be71-1d34cfa70dbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6809","nonce":1286810294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6811","nonce":1286810294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6815","nonce":1286810294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6813","nonce":1286810294}]},"public_addr":"172.21.15.38:6809/1286810294","cluster_addr":"172.21.15.38:6811/1286810294","heartbeat_back_addr":"172.21.15.38:6815/1286810294","heartbeat_front_addr":"172.21.15.38:6813/1286810294","state":["exists","up"]},{"osd":4,"uuid":"7c9f9314-99d7-4d73-8392-37809a9f54b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6803","nonce":3401837571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6805","nonce":3401837571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6809","nonce":3401837571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6807","nonce":3401837571}]},"public_addr":"172.21.15.195:6803/3401837571","cluster_addr":"172.21.15.195:6805/3401837571","heartbeat_back_addr":"172.21.15.195:6809/3401837571","heartbeat_front_addr":"172.21.15.195:6807/3401837571","state":["exists","up"]},{"osd":5,"uuid":"bde4fe9f-8414-4bba-bf4a-43973d1976bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6811","nonce":2080960180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6813","nonce":2080960180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6817","nonce":2080960180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6815","nonce":2080960180}]},"public_addr":"172.21.15.195:6811/2080960180","cluster_addr":"172.21.15.195:6813/2080960180","heartbeat_back_addr":"172.21.15.195:6817/2080960180","heartbeat_front_addr":"172.21.15.195:6815/2080960180","state":["exists","up"]},{"osd":6,"uuid":"6ee6d7f4-1a5f-4a4c-ac16-48fe05f6a002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6827","nonce":4055825675}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6829","nonce":4055825675}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6833","nonce":4055825675}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6831","nonce":4055825675}]},"public_addr":"172.21.15.195:6827/4055825675","cluster_addr":"172.21.15.195:6829/4055825675","heartbeat_back_addr":"172.21.15.195:6833/4055825675","heartbeat_front_addr":"172.21.15.195:6831/4055825675","state":["exists","up"]},{"osd":7,"uuid":"efeb5f51-329a-4484-8290-92e7d3503d05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6819","nonce":1606106342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6821","nonce":1606106342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6825","nonce":1606106342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6823","nonce":1606106342}]},"public_addr":"172.21.15.195:6819/1606106342","cluster_addr":"172.21.15.195:6821/1606106342","heartbeat_back_addr":"172.21.15.195:6825/1606106342","heartbeat_front_addr":"172.21.15.195:6823/1606106342","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.901343+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:21.115003+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.901623+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.835238+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.132199+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:19.817560+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.829690+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:21.087300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.b","mappings":[{"from":4,"to":2}]},{"pgid":"6.2d","mappings":[{"from":4,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.38:6834/1865253656":"2024-09-22T03:34:47.895300+0000","172.21.15.38:6832/1865253656":"2024-09-22T03:34:47.895300+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T03:35:30.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:30.387+0000 7f664afe4640 1 -- 172.21.15.38:0/3776620603 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6630078b90 msgr2=0x7f663007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:30.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:30.387+0000 7f664afe4640 1 --2- 172.21.15.38:0/3776620603 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6630078b90 0x7f663007b0a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f66440cec40 tx=0x7f66340067a0 comp rx=0 tx=0).stop 2024-09-21T03:35:30.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:30.387+0000 7f664afe4640 1 -- 172.21.15.38:0/3776620603 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f66441f66e0 msgr2=0x7f66440b2810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:30.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:30.387+0000 7f664afe4640 1 --2- 172.21.15.38:0/3776620603 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f66441f66e0 0x7f66440b2810 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f6638011ab0 tx=0x7f6638011f70 comp rx=0 tx=0).stop 2024-09-21T03:35:30.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:30.387+0000 7f664afe4640 1 -- 172.21.15.38:0/3776620603 shutdown_connections 2024-09-21T03:35:30.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:30.387+0000 7f664afe4640 1 --2- 172.21.15.38:0/3776620603 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f66440b2d50 0x7f66441f92a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:30.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:30.387+0000 7f664afe4640 1 --2- 172.21.15.38:0/3776620603 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f66441f66e0 0x7f66440b2810 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:30.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:30.387+0000 7f664afe4640 1 --2- 172.21.15.38:0/3776620603 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6630078b90 0x7f663007b0a0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:30.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:30.387+0000 7f664afe4640 1 --2- 172.21.15.38:0/3776620603 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f662c005250 0x7f6644157200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:30.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:30.387+0000 7f664afe4640 1 -- 172.21.15.38:0/3776620603 >> 172.21.15.38:0/3776620603 conn(0x7f6644072990 msgr2=0x7f664415ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:30.390 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:30.387+0000 7f664afe4640 1 -- 172.21.15.38:0/3776620603 shutdown_connections 2024-09-21T03:35:30.390 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:30.387+0000 7f664afe4640 1 -- 172.21.15.38:0/3776620603 wait complete. 2024-09-21T03:35:30.400 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-21T03:35:30.401 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:35:30.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:30.555+0000 7f403f313640 1 -- 172.21.15.38:0/1083622927 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4038162280 msgr2=0x7f4038162690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:30.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:30.555+0000 7f403f313640 1 --2- 172.21.15.38:0/1083622927 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4038162280 0x7f4038162690 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f40380d0c00 tx=0x7f402c02a4d0 comp rx=0 tx=0).stop 2024-09-21T03:35:30.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:30.555+0000 7f403f313640 1 -- 172.21.15.38:0/1083622927 shutdown_connections 2024-09-21T03:35:30.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:30.555+0000 7f403f313640 1 --2- 172.21.15.38:0/1083622927 >> v2:172.21.15.195:3300/0 conn(0x7f40380b2e40 0x7f40380b3250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:30.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:30.555+0000 7f403f313640 1 --2- 172.21.15.38:0/1083622927 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4038162280 0x7f4038162690 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:30.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:30.555+0000 7f403f313640 1 -- 172.21.15.38:0/1083622927 >> 172.21.15.38:0/1083622927 conn(0x7f40380b6b20 msgr2=0x7f40380d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:30.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:30.555+0000 7f403f313640 1 -- 172.21.15.38:0/1083622927 shutdown_connections 2024-09-21T03:35:30.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:30.555+0000 7f403f313640 1 -- 172.21.15.38:0/1083622927 wait complete. 2024-09-21T03:35:30.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:30.557+0000 7f403f313640 1 Processor -- start 2024-09-21T03:35:30.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:30.557+0000 7f403f313640 1 -- start start 2024-09-21T03:35:30.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:30.558+0000 7f403f313640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f40380b2e40 0x7f40380ce940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:30.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:30.558+0000 7f403f313640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4038162280 0x7f403815a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:30.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:30.558+0000 7f403f313640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f40380aadc0 0x7f403815ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:30.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:30.558+0000 7f403f313640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f403815e4b0 con 0x7f40380b2e40 2024-09-21T03:35:30.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:30.558+0000 7f403f313640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f40380b39a0 con 0x7f40380aadc0 2024-09-21T03:35:30.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:30.558+0000 7f403f313640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f403815e630 con 0x7f4038162280 2024-09-21T03:35:30.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:30.558+0000 7f403d0af640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f40380b2e40 0x7f40380ce940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:30.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:30.558+0000 7f403d0af640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f40380b2e40 0x7f40380ce940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44810/0 (socket says 172.21.15.38:44810) 2024-09-21T03:35:30.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:30.558+0000 7f403d0af640 1 -- 172.21.15.38:0/2824554031 learned_addr learned my addr 172.21.15.38:0/2824554031 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:35:30.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:30.558+0000 7f403d8b0640 1 --2- 172.21.15.38:0/2824554031 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f40380aadc0 0x7f403815ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:30.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:30.558+0000 7f403c8ae640 1 --2- 172.21.15.38:0/2824554031 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4038162280 0x7f403815a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:30.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:30.559+0000 7f403d0af640 1 -- 172.21.15.38:0/2824554031 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4038162280 msgr2=0x7f403815a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:30.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:30.559+0000 7f403d0af640 1 --2- 172.21.15.38:0/2824554031 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4038162280 0x7f403815a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:30.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:30.559+0000 7f403d0af640 1 -- 172.21.15.38:0/2824554031 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f40380aadc0 msgr2=0x7f403815ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:30.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:30.559+0000 7f403d0af640 1 --2- 172.21.15.38:0/2824554031 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f40380aadc0 0x7f403815ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:30.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:30.559+0000 7f403d0af640 1 -- 172.21.15.38:0/2824554031 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40380ab5a0 con 0x7f40380b2e40 2024-09-21T03:35:30.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:30.559+0000 7f403d0af640 1 --2- 172.21.15.38:0/2824554031 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f40380b2e40 0x7f40380ce940 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f403000b3f0 tx=0x7f403000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:30.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:30.559+0000 7f4035ffb640 1 -- 172.21.15.38:0/2824554031 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4030015020 con 0x7f40380b2e40 2024-09-21T03:35:30.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:30.559+0000 7f4035ffb640 1 -- 172.21.15.38:0/2824554031 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4030011920 con 0x7f40380b2e40 2024-09-21T03:35:30.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:30.559+0000 7f403f313640 1 -- 172.21.15.38:0/2824554031 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40380ab880 con 0x7f40380b2e40 2024-09-21T03:35:30.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:30.559+0000 7f4035ffb640 1 -- 172.21.15.38:0/2824554031 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4030011d70 con 0x7f40380b2e40 2024-09-21T03:35:30.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:30.559+0000 7f403f313640 1 -- 172.21.15.38:0/2824554031 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4038158300 con 0x7f40380b2e40 2024-09-21T03:35:30.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:30.560+0000 7f403f313640 1 -- 172.21.15.38:0/2824554031 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4038162f00 con 0x7f40380b2e40 2024-09-21T03:35:30.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:30.561+0000 7f4035ffb640 1 -- 172.21.15.38:0/2824554031 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f4030018020 con 0x7f40380b2e40 2024-09-21T03:35:30.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:30.561+0000 7f4035ffb640 1 --2- 172.21.15.38:0/2824554031 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4024078b90 0x7f402407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:30.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:30.561+0000 7f4035ffb640 1 -- 172.21.15.38:0/2824554031 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6245+0+0 (secure 0 0 0) 0x7f403009a6b0 con 0x7f40380b2e40 2024-09-21T03:35:30.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:30.562+0000 7f403c8ae640 1 --2- 172.21.15.38:0/2824554031 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4024078b90 0x7f402407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:30.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:30.562+0000 7f403c8ae640 1 --2- 172.21.15.38:0/2824554031 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4024078b90 0x7f402407b0a0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f402c033000 tx=0x7f402c004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:30.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:30.566+0000 7f4035ffb640 1 -- 172.21.15.38:0/2824554031 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f403001f070 con 0x7f40380b2e40 2024-09-21T03:35:30.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:30.792+0000 7f403f313640 1 -- 172.21.15.38:0/2824554031 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f403806dce0 con 0x7f40380b2e40 2024-09-21T03:35:30.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:30.792+0000 7f4035ffb640 1 -- 172.21.15.38:0/2824554031 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+5230 (secure 0 0 0) 0x7f40300623a0 con 0x7f40380b2e40 2024-09-21T03:35:30.795 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:35:30.795 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":23,"btime":"2024-09-21T03:34:51:918500+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4757,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/1905955399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":1905955399},{"type":"v1","addr":"172.21.15.195:6836","nonce":1905955399}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4936,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/2989374027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":2989374027},{"type":"v1","addr":"172.21.15.195:6837","nonce":2989374027}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4942,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/3531731501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6833","nonce":3531731501},{"type":"v1","addr":"172.21.15.38:6835","nonce":3531731501}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:34:34.009269+0000","modified":"2024-09-21T03:34:51.918491+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":41,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5226},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5226":{"gid":5226,"name":"c","rank":0,"incarnation":20,"state":"up:active","state_seq":4,"addr":"172.21.15.38:6834/3533947700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":3533947700},{"type":"v1","addr":"172.21.15.38:6834","nonce":3533947700}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5226,"qdb_cluster":[5226]},"id":2}]} 2024-09-21T03:35:30.795 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 23 2024-09-21T03:35:30.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:30.795+0000 7f403f313640 1 -- 172.21.15.38:0/2824554031 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4024078b90 msgr2=0x7f402407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:30.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:30.795+0000 7f403f313640 1 --2- 172.21.15.38:0/2824554031 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4024078b90 0x7f402407b0a0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f402c033000 tx=0x7f402c004730 comp rx=0 tx=0).stop 2024-09-21T03:35:30.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:30.795+0000 7f403f313640 1 -- 172.21.15.38:0/2824554031 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f40380b2e40 msgr2=0x7f40380ce940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:30.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:30.795+0000 7f403f313640 1 --2- 172.21.15.38:0/2824554031 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f40380b2e40 0x7f40380ce940 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f403000b3f0 tx=0x7f403000b8b0 comp rx=0 tx=0).stop 2024-09-21T03:35:30.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:30.795+0000 7f403f313640 1 -- 172.21.15.38:0/2824554031 shutdown_connections 2024-09-21T03:35:30.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:30.795+0000 7f403f313640 1 --2- 172.21.15.38:0/2824554031 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f40380aadc0 0x7f403815ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:30.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:30.795+0000 7f403f313640 1 --2- 172.21.15.38:0/2824554031 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4024078b90 0x7f402407b0a0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:30.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:30.795+0000 7f403f313640 1 --2- 172.21.15.38:0/2824554031 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4038162280 0x7f403815a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:30.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:30.795+0000 7f403f313640 1 --2- 172.21.15.38:0/2824554031 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f40380b2e40 0x7f40380ce940 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:30.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:30.795+0000 7f403f313640 1 -- 172.21.15.38:0/2824554031 >> 172.21.15.38:0/2824554031 conn(0x7f40380b6b20 msgr2=0x7f403806e470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:30.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:30.795+0000 7f403f313640 1 -- 172.21.15.38:0/2824554031 shutdown_connections 2024-09-21T03:35:30.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:30.795+0000 7f403f313640 1 -- 172.21.15.38:0/2824554031 wait complete. 2024-09-21T03:35:30.808 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:35:30.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:30.930+0000 7f6e12357640 1 -- 172.21.15.38:0/3419069163 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e0c0b2e60 msgr2=0x7f6e0c0b3270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:30.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:30.930+0000 7f6e12357640 1 --2- 172.21.15.38:0/3419069163 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e0c0b2e60 0x7f6e0c0b3270 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f6e0c08d3b0 tx=0x7f6e0402a8b0 comp rx=0 tx=0).stop 2024-09-21T03:35:30.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:30.930+0000 7f6e12357640 1 -- 172.21.15.38:0/3419069163 shutdown_connections 2024-09-21T03:35:30.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:30.930+0000 7f6e12357640 1 --2- 172.21.15.38:0/3419069163 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e0c0b2e60 0x7f6e0c0b3270 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:30.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:30.930+0000 7f6e12357640 1 --2- 172.21.15.38:0/3419069163 >> v2:172.21.15.195:3300/0 conn(0x7f6e0c0cfe00 0x7f6e0c0d2a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:30.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:30.930+0000 7f6e12357640 1 -- 172.21.15.38:0/3419069163 >> 172.21.15.38:0/3419069163 conn(0x7f6e0c0d4440 msgr2=0x7f6e0c0d4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:30.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:30.930+0000 7f6e12357640 1 -- 172.21.15.38:0/3419069163 shutdown_connections 2024-09-21T03:35:30.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:30.930+0000 7f6e12357640 1 -- 172.21.15.38:0/3419069163 wait complete. 2024-09-21T03:35:30.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:30.932+0000 7f6e12357640 1 Processor -- start 2024-09-21T03:35:30.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:30.932+0000 7f6e12357640 1 -- start start 2024-09-21T03:35:30.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:30.933+0000 7f6e12357640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6e0c0b2e60 0x7f6e0c1539e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:30.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:30.933+0000 7f6e12357640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6e0c0cfe00 0x7f6e0c155730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:30.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:30.933+0000 7f6e12357640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e0c162590 0x7f6e0c160000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:30.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:30.933+0000 7f6e12357640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f6e0c0b3930 con 0x7f6e0c162590 2024-09-21T03:35:30.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:30.933+0000 7f6e12357640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f6e0c0b37b0 con 0x7f6e0c0b2e60 2024-09-21T03:35:30.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:30.933+0000 7f6e12357640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f6e0c0b3ab0 con 0x7f6e0c0cfe00 2024-09-21T03:35:30.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:30.933+0000 7f6e108f4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e0c162590 0x7f6e0c160000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:30.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:30.933+0000 7f6e108f4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e0c162590 0x7f6e0c160000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44816/0 (socket says 172.21.15.38:44816) 2024-09-21T03:35:30.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:30.933+0000 7f6e108f4640 1 -- 172.21.15.38:0/4160197817 learned_addr learned my addr 172.21.15.38:0/4160197817 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:35:30.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:30.933+0000 7f6e0bfff640 1 --2- 172.21.15.38:0/4160197817 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6e0c0b2e60 0x7f6e0c1539e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:30.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:30.933+0000 7f6e0b7fe640 1 --2- 172.21.15.38:0/4160197817 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6e0c0cfe00 0x7f6e0c155730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:30.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:30.934+0000 7f6e108f4640 1 -- 172.21.15.38:0/4160197817 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6e0c0cfe00 msgr2=0x7f6e0c155730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:30.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:30.934+0000 7f6e108f4640 1 --2- 172.21.15.38:0/4160197817 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6e0c0cfe00 0x7f6e0c155730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:30.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:30.934+0000 7f6e108f4640 1 -- 172.21.15.38:0/4160197817 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6e0c0b2e60 msgr2=0x7f6e0c1539e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:30.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:30.934+0000 7f6e108f4640 1 --2- 172.21.15.38:0/4160197817 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6e0c0b2e60 0x7f6e0c1539e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:30.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:30.934+0000 7f6e108f4640 1 -- 172.21.15.38:0/4160197817 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e0c15cf30 con 0x7f6e0c162590 2024-09-21T03:35:30.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:30.934+0000 7f6e0b7fe640 1 --2- 172.21.15.38:0/4160197817 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6e0c0cfe00 0x7f6e0c155730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:35:30.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:30.934+0000 7f6e108f4640 1 --2- 172.21.15.38:0/4160197817 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e0c162590 0x7f6e0c160000 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f6e0402aea0 tx=0x7f6e04003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:30.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:30.934+0000 7f6e08ff9640 1 -- 172.21.15.38:0/4160197817 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e04031b90 con 0x7f6e0c162590 2024-09-21T03:35:30.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:30.934+0000 7f6e08ff9640 1 -- 172.21.15.38:0/4160197817 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6e04031e20 con 0x7f6e0c162590 2024-09-21T03:35:30.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:30.934+0000 7f6e12357640 1 -- 172.21.15.38:0/4160197817 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e0c1f91e0 con 0x7f6e0c162590 2024-09-21T03:35:30.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:30.934+0000 7f6e08ff9640 1 -- 172.21.15.38:0/4160197817 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e040332f0 con 0x7f6e0c162590 2024-09-21T03:35:30.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:30.934+0000 7f6e12357640 1 -- 172.21.15.38:0/4160197817 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e0c1f97a0 con 0x7f6e0c162590 2024-09-21T03:35:30.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:30.935+0000 7f6e12357640 1 -- 172.21.15.38:0/4160197817 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e0c0b3270 con 0x7f6e0c162590 2024-09-21T03:35:30.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:30.936+0000 7f6e08ff9640 1 -- 172.21.15.38:0/4160197817 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f6e040316a0 con 0x7f6e0c162590 2024-09-21T03:35:30.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:30.936+0000 7f6e08ff9640 1 --2- 172.21.15.38:0/4160197817 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6df8078b90 0x7f6df807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:30.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:30.936+0000 7f6e08ff9640 1 -- 172.21.15.38:0/4160197817 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6245+0+0 (secure 0 0 0) 0x7f6e040bc520 con 0x7f6e0c162590 2024-09-21T03:35:30.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:30.937+0000 7f6e0bfff640 1 --2- 172.21.15.38:0/4160197817 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6df8078b90 0x7f6df807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:30.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:30.937+0000 7f6e0bfff640 1 --2- 172.21.15.38:0/4160197817 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6df8078b90 0x7f6df807b0a0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f6dfc0058c0 tx=0x7f6dfc00c440 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:30.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:30.941+0000 7f6e08ff9640 1 -- 172.21.15.38:0/4160197817 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f6e0403f030 con 0x7f6e0c162590 2024-09-21T03:35:31.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:31.166+0000 7f6e12357640 1 -- 172.21.15.38:0/4160197817 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6e0c15ea50 con 0x7f6e0c162590 2024-09-21T03:35:31.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:31.167+0000 7f6e08ff9640 1 -- 172.21.15.38:0/4160197817 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+5230 (secure 0 0 0) 0x7f6e040847d0 con 0x7f6e0c162590 2024-09-21T03:35:31.169 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:35:31.170 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":23,"btime":"2024-09-21T03:34:51:918500+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4757,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/1905955399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":1905955399},{"type":"v1","addr":"172.21.15.195:6836","nonce":1905955399}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4936,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/2989374027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":2989374027},{"type":"v1","addr":"172.21.15.195:6837","nonce":2989374027}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4942,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/3531731501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6833","nonce":3531731501},{"type":"v1","addr":"172.21.15.38:6835","nonce":3531731501}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:34:34.009269+0000","modified":"2024-09-21T03:34:51.918491+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":41,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5226},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5226":{"gid":5226,"name":"c","rank":0,"incarnation":20,"state":"up:active","state_seq":4,"addr":"172.21.15.38:6834/3533947700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":3533947700},{"type":"v1","addr":"172.21.15.38:6834","nonce":3533947700}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5226,"qdb_cluster":[5226]},"id":2}]} 2024-09-21T03:35:31.170 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 23 2024-09-21T03:35:31.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:31.169+0000 7f6e12357640 1 -- 172.21.15.38:0/4160197817 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6df8078b90 msgr2=0x7f6df807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:31.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:31.169+0000 7f6e12357640 1 --2- 172.21.15.38:0/4160197817 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6df8078b90 0x7f6df807b0a0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f6dfc0058c0 tx=0x7f6dfc00c440 comp rx=0 tx=0).stop 2024-09-21T03:35:31.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:31.169+0000 7f6e12357640 1 -- 172.21.15.38:0/4160197817 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e0c162590 msgr2=0x7f6e0c160000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:31.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:31.169+0000 7f6e12357640 1 --2- 172.21.15.38:0/4160197817 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e0c162590 0x7f6e0c160000 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f6e0402aea0 tx=0x7f6e04003ba0 comp rx=0 tx=0).stop 2024-09-21T03:35:31.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:31.169+0000 7f6e12357640 1 -- 172.21.15.38:0/4160197817 shutdown_connections 2024-09-21T03:35:31.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:31.169+0000 7f6e12357640 1 --2- 172.21.15.38:0/4160197817 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e0c162590 0x7f6e0c160000 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:31.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:31.169+0000 7f6e12357640 1 --2- 172.21.15.38:0/4160197817 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6df8078b90 0x7f6df807b0a0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:31.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:31.169+0000 7f6e12357640 1 --2- 172.21.15.38:0/4160197817 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6e0c0cfe00 0x7f6e0c155730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:31.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:31.169+0000 7f6e12357640 1 --2- 172.21.15.38:0/4160197817 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6e0c0b2e60 0x7f6e0c1539e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:31.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:31.169+0000 7f6e12357640 1 -- 172.21.15.38:0/4160197817 >> 172.21.15.38:0/4160197817 conn(0x7f6e0c0d4440 msgr2=0x7f6e0c157ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:31.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:31.170+0000 7f6e12357640 1 -- 172.21.15.38:0/4160197817 shutdown_connections 2024-09-21T03:35:31.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:31.170+0000 7f6e12357640 1 -- 172.21.15.38:0/4160197817 wait complete. 2024-09-21T03:35:31.184 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T03:35:31.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:31.304+0000 7f1eecb62640 1 -- 172.21.15.38:0/2132925995 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1ed4004d40 msgr2=0x7f1ed4005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:31.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:31.304+0000 7f1eecb62640 1 --2- 172.21.15.38:0/2132925995 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1ed4004d40 0x7f1ed4005150 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f1ee0005fd0 tx=0x7f1ed4005830 comp rx=0 tx=0).stop 2024-09-21T03:35:31.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:31.304+0000 7f1ee6d9d640 1 -- 172.21.15.38:0/2132925995 reap_dead start 2024-09-21T03:35:31.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:31.304+0000 7f1eecb62640 1 -- 172.21.15.38:0/2132925995 shutdown_connections 2024-09-21T03:35:31.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:31.304+0000 7f1eecb62640 1 -- 172.21.15.38:0/2132925995 >> 172.21.15.38:0/2132925995 conn(0x7f1ee80b6b20 msgr2=0x7f1ee80d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:31.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:31.304+0000 7f1eecb62640 1 -- 172.21.15.38:0/2132925995 shutdown_connections 2024-09-21T03:35:31.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:31.304+0000 7f1eecb62640 1 -- 172.21.15.38:0/2132925995 wait complete. 2024-09-21T03:35:31.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:31.306+0000 7f1eecb62640 1 Processor -- start 2024-09-21T03:35:31.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:31.306+0000 7f1eecb62640 1 -- start start 2024-09-21T03:35:31.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:31.307+0000 7f1eecb62640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1ed4004d40 0x7f1ee81f59f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:31.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:31.307+0000 7f1eecb62640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1ee80c9640 0x7f1ee81f8380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:31.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:31.308+0000 7f1eecb62640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1ee81f88c0 0x7f1ee80c7f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:31.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:31.308+0000 7f1eecb62640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f1ee8162690 con 0x7f1ee80c9640 2024-09-21T03:35:31.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:31.308+0000 7f1eecb62640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f1ee80b39a0 con 0x7f1ee81f88c0 2024-09-21T03:35:31.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:31.308+0000 7f1eecb62640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f1ee8162810 con 0x7f1ed4004d40 2024-09-21T03:35:31.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:31.308+0000 7f1edffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1ee80c9640 0x7f1ee81f8380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:31.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:31.308+0000 7f1edffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1ee80c9640 0x7f1ee81f8380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44838/0 (socket says 172.21.15.38:44838) 2024-09-21T03:35:31.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:31.308+0000 7f1edffff640 1 -- 172.21.15.38:0/4069949209 learned_addr learned my addr 172.21.15.38:0/4069949209 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:35:31.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:31.308+0000 7f1ee659c640 1 --2- 172.21.15.38:0/4069949209 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1ed4004d40 0x7f1ee81f59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:31.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:31.308+0000 7f1ee6d9d640 1 --2- 172.21.15.38:0/4069949209 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1ee81f88c0 0x7f1ee80c7f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:31.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:31.308+0000 7f1edffff640 1 -- 172.21.15.38:0/4069949209 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1ed4004d40 msgr2=0x7f1ee81f59f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:31.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:31.308+0000 7f1edffff640 1 --2- 172.21.15.38:0/4069949209 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1ed4004d40 0x7f1ee81f59f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:31.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:31.308+0000 7f1edffff640 1 -- 172.21.15.38:0/4069949209 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1ee81f88c0 msgr2=0x7f1ee80c7f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:31.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:31.308+0000 7f1edffff640 1 --2- 172.21.15.38:0/4069949209 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1ee81f88c0 0x7f1ee80c7f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:31.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:31.308+0000 7f1edffff640 1 -- 172.21.15.38:0/4069949209 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ee81f92e0 con 0x7f1ee80c9640 2024-09-21T03:35:31.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:31.308+0000 7f1edffff640 1 --2- 172.21.15.38:0/4069949209 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1ee80c9640 0x7f1ee81f8380 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f1ed0011800 tx=0x7f1ed0011cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:31.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:31.308+0000 7f1edeffd640 1 -- 172.21.15.38:0/4069949209 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ed0004460 con 0x7f1ee80c9640 2024-09-21T03:35:31.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:31.308+0000 7f1edeffd640 1 -- 172.21.15.38:0/4069949209 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1ed0016070 con 0x7f1ee80c9640 2024-09-21T03:35:31.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:31.308+0000 7f1eecb62640 1 -- 172.21.15.38:0/4069949209 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ee81f9000 con 0x7f1ee80c9640 2024-09-21T03:35:31.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:31.308+0000 7f1edeffd640 1 -- 172.21.15.38:0/4069949209 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ed000fb80 con 0x7f1ee80c9640 2024-09-21T03:35:31.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:31.309+0000 7f1eecb62640 1 -- 172.21.15.38:0/4069949209 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ee81f51f0 con 0x7f1ee80c9640 2024-09-21T03:35:31.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:31.310+0000 7f1eecb62640 1 -- 172.21.15.38:0/4069949209 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ee8162f00 con 0x7f1ee80c9640 2024-09-21T03:35:31.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:31.310+0000 7f1edeffd640 1 -- 172.21.15.38:0/4069949209 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f1ed0018030 con 0x7f1ee80c9640 2024-09-21T03:35:31.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:31.311+0000 7f1edeffd640 1 --2- 172.21.15.38:0/4069949209 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1ec8078b90 0x7f1ec807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:31.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:31.311+0000 7f1edeffd640 1 -- 172.21.15.38:0/4069949209 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6245+0+0 (secure 0 0 0) 0x7f1ed009b880 con 0x7f1ee80c9640 2024-09-21T03:35:31.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:31.311+0000 7f1ee659c640 1 --2- 172.21.15.38:0/4069949209 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1ec8078b90 0x7f1ec807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:31.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:31.312+0000 7f1ee659c640 1 --2- 172.21.15.38:0/4069949209 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1ec8078b90 0x7f1ec807b0a0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f1ed8001010 tx=0x7f1ed80011c0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:31.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:31.315+0000 7f1edeffd640 1 -- 172.21.15.38:0/4069949209 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f1ed0063330 con 0x7f1ee80c9640 2024-09-21T03:35:31.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:31.503+0000 7f1eecb62640 1 -- 172.21.15.38:0/4069949209 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1ee81f8d60 con 0x7f1ee80c9640 2024-09-21T03:35:31.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:31.504+0000 7f1edeffd640 1 -- 172.21.15.38:0/4069949209 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) ==== 74+0+17190 (secure 0 0 0) 0x7f1ed0067d60 con 0x7f1ee80c9640 2024-09-21T03:35:31.506 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:35:31.507 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":43,"fsid":"9d50d66f-656b-4fe2-8e3b-fe53b79c4d43","created":"2024-09-21T03:33:10.677865+0000","modified":"2024-09-21T03:35:17.098344+0000","last_up_change":"2024-09-21T03:33:24.023421+0000","last_in_change":"2024-09-21T03:33:14.296193+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T03:33:22.992573+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-09-21T03:34:32.080636+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-09-21T03:34:32.959110+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dbab183f-7e63-48b8-960a-ae55fd5bf565","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6801","nonce":2381639433}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6803","nonce":2381639433}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6807","nonce":2381639433}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6805","nonce":2381639433}]},"public_addr":"172.21.15.38:6801/2381639433","cluster_addr":"172.21.15.38:6803/2381639433","heartbeat_back_addr":"172.21.15.38:6807/2381639433","heartbeat_front_addr":"172.21.15.38:6805/2381639433","state":["exists","up"]},{"osd":1,"uuid":"fa1c0e8a-ebc1-4359-8146-bd5342abbc16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6817","nonce":1491818732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6819","nonce":1491818732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6823","nonce":1491818732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6821","nonce":1491818732}]},"public_addr":"172.21.15.38:6817/1491818732","cluster_addr":"172.21.15.38:6819/1491818732","heartbeat_back_addr":"172.21.15.38:6823/1491818732","heartbeat_front_addr":"172.21.15.38:6821/1491818732","state":["exists","up"]},{"osd":2,"uuid":"2ed3cd2d-385d-4fc6-9869-214552a84803","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6825","nonce":2577858902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6827","nonce":2577858902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6831","nonce":2577858902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6829","nonce":2577858902}]},"public_addr":"172.21.15.38:6825/2577858902","cluster_addr":"172.21.15.38:6827/2577858902","heartbeat_back_addr":"172.21.15.38:6831/2577858902","heartbeat_front_addr":"172.21.15.38:6829/2577858902","state":["exists","up"]},{"osd":3,"uuid":"62d1d95e-919d-4250-be71-1d34cfa70dbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6809","nonce":1286810294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6811","nonce":1286810294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6815","nonce":1286810294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6813","nonce":1286810294}]},"public_addr":"172.21.15.38:6809/1286810294","cluster_addr":"172.21.15.38:6811/1286810294","heartbeat_back_addr":"172.21.15.38:6815/1286810294","heartbeat_front_addr":"172.21.15.38:6813/1286810294","state":["exists","up"]},{"osd":4,"uuid":"7c9f9314-99d7-4d73-8392-37809a9f54b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6803","nonce":3401837571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6805","nonce":3401837571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6809","nonce":3401837571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6807","nonce":3401837571}]},"public_addr":"172.21.15.195:6803/3401837571","cluster_addr":"172.21.15.195:6805/3401837571","heartbeat_back_addr":"172.21.15.195:6809/3401837571","heartbeat_front_addr":"172.21.15.195:6807/3401837571","state":["exists","up"]},{"osd":5,"uuid":"bde4fe9f-8414-4bba-bf4a-43973d1976bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6811","nonce":2080960180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6813","nonce":2080960180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6817","nonce":2080960180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6815","nonce":2080960180}]},"public_addr":"172.21.15.195:6811/2080960180","cluster_addr":"172.21.15.195:6813/2080960180","heartbeat_back_addr":"172.21.15.195:6817/2080960180","heartbeat_front_addr":"172.21.15.195:6815/2080960180","state":["exists","up"]},{"osd":6,"uuid":"6ee6d7f4-1a5f-4a4c-ac16-48fe05f6a002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6827","nonce":4055825675}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6829","nonce":4055825675}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6833","nonce":4055825675}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6831","nonce":4055825675}]},"public_addr":"172.21.15.195:6827/4055825675","cluster_addr":"172.21.15.195:6829/4055825675","heartbeat_back_addr":"172.21.15.195:6833/4055825675","heartbeat_front_addr":"172.21.15.195:6831/4055825675","state":["exists","up"]},{"osd":7,"uuid":"efeb5f51-329a-4484-8290-92e7d3503d05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6819","nonce":1606106342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6821","nonce":1606106342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6825","nonce":1606106342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6823","nonce":1606106342}]},"public_addr":"172.21.15.195:6819/1606106342","cluster_addr":"172.21.15.195:6821/1606106342","heartbeat_back_addr":"172.21.15.195:6825/1606106342","heartbeat_front_addr":"172.21.15.195:6823/1606106342","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.901343+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:21.115003+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.901623+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.835238+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.132199+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:19.817560+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.829690+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:21.087300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.b","mappings":[{"from":4,"to":2}]},{"pgid":"6.2d","mappings":[{"from":4,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.38:6834/1865253656":"2024-09-22T03:34:47.895300+0000","172.21.15.38:6832/1865253656":"2024-09-22T03:34:47.895300+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T03:35:31.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:31.506+0000 7f1eecb62640 1 -- 172.21.15.38:0/4069949209 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1ec8078b90 msgr2=0x7f1ec807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:31.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:31.506+0000 7f1eecb62640 1 --2- 172.21.15.38:0/4069949209 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1ec8078b90 0x7f1ec807b0a0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f1ed8001010 tx=0x7f1ed80011c0 comp rx=0 tx=0).stop 2024-09-21T03:35:31.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:31.506+0000 7f1eecb62640 1 -- 172.21.15.38:0/4069949209 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1ee80c9640 msgr2=0x7f1ee81f8380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:31.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:31.506+0000 7f1eecb62640 1 --2- 172.21.15.38:0/4069949209 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1ee80c9640 0x7f1ee81f8380 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f1ed0011800 tx=0x7f1ed0011cc0 comp rx=0 tx=0).stop 2024-09-21T03:35:31.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:31.506+0000 7f1eecb62640 1 -- 172.21.15.38:0/4069949209 shutdown_connections 2024-09-21T03:35:31.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:31.506+0000 7f1eecb62640 1 --2- 172.21.15.38:0/4069949209 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1ee81f88c0 0x7f1ee80c7f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:31.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:31.506+0000 7f1eecb62640 1 --2- 172.21.15.38:0/4069949209 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1ee80c9640 0x7f1ee81f8380 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:31.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:31.506+0000 7f1eecb62640 1 --2- 172.21.15.38:0/4069949209 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1ec8078b90 0x7f1ec807b0a0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:31.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:31.506+0000 7f1eecb62640 1 --2- 172.21.15.38:0/4069949209 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1ed4004d40 0x7f1ee81f59f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:31.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:31.506+0000 7f1eecb62640 1 -- 172.21.15.38:0/4069949209 >> 172.21.15.38:0/4069949209 conn(0x7f1ee80b6b20 msgr2=0x7f1ee8068d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:31.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:31.506+0000 7f1eecb62640 1 -- 172.21.15.38:0/4069949209 shutdown_connections 2024-09-21T03:35:31.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:31.506+0000 7f1eecb62640 1 -- 172.21.15.38:0/4069949209 wait complete. 2024-09-21T03:35:31.520 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-21T03:35:31.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:31.636+0000 7f0d12030640 1 -- 172.21.15.38:0/3718756255 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0d0c15ee80 msgr2=0x7f0d0c0d0170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:31.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:31.636+0000 7f0d12030640 1 --2- 172.21.15.38:0/3718756255 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0d0c15ee80 0x7f0d0c0d0170 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc009860 tx=0x7f0cfc02a510 comp rx=0 tx=0).stop 2024-09-21T03:35:31.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:31.636+0000 7f0d12030640 1 -- 172.21.15.38:0/3718756255 shutdown_connections 2024-09-21T03:35:31.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:31.636+0000 7f0d12030640 1 --2- 172.21.15.38:0/3718756255 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0d0c15ee80 0x7f0d0c0d0170 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:31.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:31.636+0000 7f0d12030640 1 -- 172.21.15.38:0/3718756255 >> 172.21.15.38:0/3718756255 conn(0x7f0d0c09d830 msgr2=0x7f0d0c0b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:31.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:31.636+0000 7f0d12030640 1 -- 172.21.15.38:0/3718756255 shutdown_connections 2024-09-21T03:35:31.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:31.636+0000 7f0d12030640 1 -- 172.21.15.38:0/3718756255 wait complete. 2024-09-21T03:35:31.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:31.638+0000 7f0d12030640 1 Processor -- start 2024-09-21T03:35:31.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:31.638+0000 7f0d12030640 1 -- start start 2024-09-21T03:35:31.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:31.640+0000 7f0d12030640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0d0c0b9ad0 0x7f0d0c1f51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:31.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:31.640+0000 7f0d12030640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0d0c0ceb90 0x7f0d0c06dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:31.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:31.640+0000 7f0d12030640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0d0c15ee80 0x7f0d0c06d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:31.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:31.640+0000 7f0d12030640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f0d0c0b6910 con 0x7f0d0c0ceb90 2024-09-21T03:35:31.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:31.640+0000 7f0d12030640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f0d0c0b6790 con 0x7f0d0c15ee80 2024-09-21T03:35:31.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:31.640+0000 7f0d12030640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f0d0c0b6a90 con 0x7f0d0c0b9ad0 2024-09-21T03:35:31.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:31.640+0000 7f0d0affd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0d0c0ceb90 0x7f0d0c06dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:31.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:31.640+0000 7f0d0affd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0d0c0ceb90 0x7f0d0c06dfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44848/0 (socket says 172.21.15.38:44848) 2024-09-21T03:35:31.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:31.640+0000 7f0d0affd640 1 -- 172.21.15.38:0/384888276 learned_addr learned my addr 172.21.15.38:0/384888276 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:35:31.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:31.640+0000 7f0d0b7fe640 1 --2- 172.21.15.38:0/384888276 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0d0c0b9ad0 0x7f0d0c1f51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:31.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:31.640+0000 7f0d0bfff640 1 --2- 172.21.15.38:0/384888276 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0d0c15ee80 0x7f0d0c06d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:31.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:31.640+0000 7f0d0affd640 1 -- 172.21.15.38:0/384888276 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0d0c0b9ad0 msgr2=0x7f0d0c1f51c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:31.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:31.640+0000 7f0d0affd640 1 --2- 172.21.15.38:0/384888276 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0d0c0b9ad0 0x7f0d0c1f51c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:31.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:31.640+0000 7f0d0affd640 1 -- 172.21.15.38:0/384888276 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0d0c15ee80 msgr2=0x7f0d0c06d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:31.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:31.640+0000 7f0d0affd640 1 --2- 172.21.15.38:0/384888276 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0d0c15ee80 0x7f0d0c06d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:31.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:31.640+0000 7f0d0affd640 1 -- 172.21.15.38:0/384888276 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d0c06e500 con 0x7f0d0c0ceb90 2024-09-21T03:35:31.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:31.640+0000 7f0d0affd640 1 --2- 172.21.15.38:0/384888276 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0d0c0ceb90 0x7f0d0c06dfc0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f0d0000c730 tx=0x7f0d0000cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:31.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:31.640+0000 7f0cf3fff640 1 -- 172.21.15.38:0/384888276 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d00005270 con 0x7f0d0c0ceb90 2024-09-21T03:35:31.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:31.640+0000 7f0cf3fff640 1 -- 172.21.15.38:0/384888276 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0d00005430 con 0x7f0d0c0ceb90 2024-09-21T03:35:31.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:31.640+0000 7f0d12030640 1 -- 172.21.15.38:0/384888276 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d0c1f5b80 con 0x7f0d0c0ceb90 2024-09-21T03:35:31.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:31.640+0000 7f0cf3fff640 1 -- 172.21.15.38:0/384888276 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d00003850 con 0x7f0d0c0ceb90 2024-09-21T03:35:31.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:31.641+0000 7f0d12030640 1 -- 172.21.15.38:0/384888276 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d0c1f6140 con 0x7f0d0c0ceb90 2024-09-21T03:35:31.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:31.642+0000 7f0d12030640 1 -- 172.21.15.38:0/384888276 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d0c0d0170 con 0x7f0d0c0ceb90 2024-09-21T03:35:31.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:31.642+0000 7f0cf3fff640 1 -- 172.21.15.38:0/384888276 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f0d000039f0 con 0x7f0d0c0ceb90 2024-09-21T03:35:31.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:31.643+0000 7f0cf3fff640 1 --2- 172.21.15.38:0/384888276 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0cf8078bc0 0x7f0cf807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:31.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:31.643+0000 7f0cf3fff640 1 -- 172.21.15.38:0/384888276 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6245+0+0 (secure 0 0 0) 0x7f0d0009aae0 con 0x7f0d0c0ceb90 2024-09-21T03:35:31.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:31.643+0000 7f0d0b7fe640 1 --2- 172.21.15.38:0/384888276 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0cf8078bc0 0x7f0cf807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:31.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:31.644+0000 7f0d0b7fe640 1 --2- 172.21.15.38:0/384888276 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0cf8078bc0 0x7f0cf807b0d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc034000 tx=0x7f0cfc004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:31.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:31.647+0000 7f0cf3fff640 1 -- 172.21.15.38:0/384888276 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f0d0c1592d0 con 0x7f0d0c0ceb90 2024-09-21T03:35:31.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:31.873+0000 7f0d12030640 1 -- 172.21.15.38:0/384888276 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f0d0c1f5910 con 0x7f0d0c0ceb90 2024-09-21T03:35:31.875 INFO:tasks.ceph.mon.a.smithi038.stderr:2024-09-21T03:35:31.873+0000 7f3f45713640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T03:35:31.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:31.875+0000 7f0cf3fff640 1 -- 172.21.15.38:0/384888276 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v24) ==== 161+0+0 (secure 0 0 0) 0x7f0d000626b0 con 0x7f0d0c0ceb90 2024-09-21T03:35:31.877 INFO:teuthology.orchestra.run.smithi038.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-21T03:35:31.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:31.879+0000 7f0d12030640 1 -- 172.21.15.38:0/384888276 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0cf8078bc0 msgr2=0x7f0cf807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:31.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:31.879+0000 7f0d12030640 1 --2- 172.21.15.38:0/384888276 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0cf8078bc0 0x7f0cf807b0d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc034000 tx=0x7f0cfc004730 comp rx=0 tx=0).stop 2024-09-21T03:35:31.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:31.879+0000 7f0d12030640 1 -- 172.21.15.38:0/384888276 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0d0c0ceb90 msgr2=0x7f0d0c06dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:31.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:31.879+0000 7f0d12030640 1 --2- 172.21.15.38:0/384888276 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0d0c0ceb90 0x7f0d0c06dfc0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f0d0000c730 tx=0x7f0d0000cb30 comp rx=0 tx=0).stop 2024-09-21T03:35:31.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:31.879+0000 7f0d12030640 1 -- 172.21.15.38:0/384888276 shutdown_connections 2024-09-21T03:35:31.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:31.879+0000 7f0d12030640 1 --2- 172.21.15.38:0/384888276 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0d0c15ee80 0x7f0d0c06d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:31.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:31.879+0000 7f0d12030640 1 --2- 172.21.15.38:0/384888276 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0d0c0ceb90 0x7f0d0c06dfc0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:31.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:31.879+0000 7f0d12030640 1 --2- 172.21.15.38:0/384888276 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0cf8078bc0 0x7f0cf807b0d0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:31.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:31.879+0000 7f0d12030640 1 --2- 172.21.15.38:0/384888276 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0d0c0b9ad0 0x7f0d0c1f51c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:31.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:31.879+0000 7f0d12030640 1 -- 172.21.15.38:0/384888276 >> 172.21.15.38:0/384888276 conn(0x7f0d0c09d830 msgr2=0x7f0d0c161b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:31.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:31.880+0000 7f0d12030640 1 -- 172.21.15.38:0/384888276 shutdown_connections 2024-09-21T03:35:31.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:31.880+0000 7f0d12030640 1 -- 172.21.15.38:0/384888276 wait complete. 2024-09-21T03:35:31.883 INFO:tasks.ceph.mds.c.smithi038.stderr: -3473> 2024-09-21T03:34:53.908+0000 7feb38379640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T03:35:31.895 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-21T03:35:31.943 INFO:tasks.ceph.mds.c.smithi038.stdout:starting mds.c at 2024-09-21T03:35:32.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:32.016+0000 7f8b9b520640 1 -- 172.21.15.38:0/1257706037 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8b80004d40 msgr2=0x7f8b800051f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:32.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:32.016+0000 7f8b9b520640 1 --2- 172.21.15.38:0/1257706037 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8b80004d40 0x7f8b800051f0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c009ed0 tx=0x7f8b80005830 comp rx=0 tx=0).stop 2024-09-21T03:35:32.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:32.016+0000 7f8b99abd640 1 -- 172.21.15.38:0/1257706037 reap_dead start 2024-09-21T03:35:32.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:32.016+0000 7f8b9b520640 1 -- 172.21.15.38:0/1257706037 shutdown_connections 2024-09-21T03:35:32.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:32.016+0000 7f8b9b520640 1 -- 172.21.15.38:0/1257706037 >> 172.21.15.38:0/1257706037 conn(0x7f8b9409d830 msgr2=0x7f8b940b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:32.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:32.016+0000 7f8b9b520640 1 -- 172.21.15.38:0/1257706037 shutdown_connections 2024-09-21T03:35:32.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:32.016+0000 7f8b9b520640 1 -- 172.21.15.38:0/1257706037 wait complete. 2024-09-21T03:35:32.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:32.018+0000 7f8b9b520640 1 Processor -- start 2024-09-21T03:35:32.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:32.018+0000 7f8b9b520640 1 -- start start 2024-09-21T03:35:32.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:32.019+0000 7f8b9b520640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8b80004d40 0x7f8b940b1ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:32.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:32.019+0000 7f8b9b520640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8b940b3130 0x7f8b940ca3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:32.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:32.019+0000 7f8b9b520640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8b9406d7b0 0x7f8b9406dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:32.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:32.019+0000 7f8b9b520640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f8b940cf330 con 0x7f8b940b3130 2024-09-21T03:35:32.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:32.019+0000 7f8b9b520640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f8b940cf1b0 con 0x7f8b80004d40 2024-09-21T03:35:32.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:32.019+0000 7f8b9b520640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f8b940b6580 con 0x7f8b9406d7b0 2024-09-21T03:35:32.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:32.020+0000 7f8b98abb640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8b940b3130 0x7f8b940ca3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:32.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:32.020+0000 7f8b98abb640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8b940b3130 0x7f8b940ca3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44896/0 (socket says 172.21.15.38:44896) 2024-09-21T03:35:32.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:32.020+0000 7f8b98abb640 1 -- 172.21.15.38:0/3298527958 learned_addr learned my addr 172.21.15.38:0/3298527958 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:35:32.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:32.020+0000 7f8b992bc640 1 --2- 172.21.15.38:0/3298527958 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8b80004d40 0x7f8b940b1ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:32.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:32.020+0000 7f8b99abd640 1 --2- 172.21.15.38:0/3298527958 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8b9406d7b0 0x7f8b9406dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:32.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:32.020+0000 7f8b98abb640 1 -- 172.21.15.38:0/3298527958 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8b9406d7b0 msgr2=0x7f8b9406dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:32.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:32.020+0000 7f8b98abb640 1 --2- 172.21.15.38:0/3298527958 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8b9406d7b0 0x7f8b9406dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:32.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:32.020+0000 7f8b98abb640 1 -- 172.21.15.38:0/3298527958 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8b80004d40 msgr2=0x7f8b940b1ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:32.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:32.020+0000 7f8b98abb640 1 --2- 172.21.15.38:0/3298527958 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8b80004d40 0x7f8b940b1ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:32.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:32.020+0000 7f8b98abb640 1 -- 172.21.15.38:0/3298527958 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b9406d2b0 con 0x7f8b940b3130 2024-09-21T03:35:32.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:32.020+0000 7f8b98abb640 1 --2- 172.21.15.38:0/3298527958 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8b940b3130 0x7f8b940ca3d0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f8b84001ba0 tx=0x7f8b840039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:32.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:32.020+0000 7f8b89ffb640 1 -- 172.21.15.38:0/3298527958 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b84015070 con 0x7f8b940b3130 2024-09-21T03:35:32.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:32.020+0000 7f8b89ffb640 1 -- 172.21.15.38:0/3298527958 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8b84011840 con 0x7f8b940b3130 2024-09-21T03:35:32.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:32.020+0000 7f8b89ffb640 1 -- 172.21.15.38:0/3298527958 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b84011c70 con 0x7f8b940b3130 2024-09-21T03:35:32.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:32.020+0000 7f8b9b520640 1 -- 172.21.15.38:0/3298527958 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b940b2360 con 0x7f8b940b3130 2024-09-21T03:35:32.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:32.021+0000 7f8b9b520640 1 -- 172.21.15.38:0/3298527958 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b940b2920 con 0x7f8b940b3130 2024-09-21T03:35:32.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:32.022+0000 7f8b9b520640 1 -- 172.21.15.38:0/3298527958 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b940b6ae0 con 0x7f8b940b3130 2024-09-21T03:35:32.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:32.022+0000 7f8b89ffb640 1 -- 172.21.15.38:0/3298527958 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f8b84018070 con 0x7f8b940b3130 2024-09-21T03:35:32.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:32.023+0000 7f8b89ffb640 1 --2- 172.21.15.38:0/3298527958 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8b78078b90 0x7f8b7807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:32.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:32.023+0000 7f8b89ffb640 1 -- 172.21.15.38:0/3298527958 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 6331+0+0 (secure 0 0 0) 0x7f8b8409bd70 con 0x7f8b940b3130 2024-09-21T03:35:32.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:32.023+0000 7f8b992bc640 1 --2- 172.21.15.38:0/3298527958 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8b78078b90 0x7f8b7807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:32.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:32.023+0000 7f8b992bc640 1 --2- 172.21.15.38:0/3298527958 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8b78078b90 0x7f8b7807b0a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f8b90000c00 tx=0x7f8b90006b70 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:32.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:32.027+0000 7f8b89ffb640 1 -- 172.21.15.38:0/3298527958 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f8b941592d0 con 0x7f8b940b3130 2024-09-21T03:35:32.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:32.251+0000 7f8b9b520640 1 -- 172.21.15.38:0/3298527958 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f8b940b20f0 con 0x7f8b940b3130 2024-09-21T03:35:32.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:32.876+0000 7f8b89ffb640 1 -- 172.21.15.38:0/3298527958 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v25) ==== 104+0+0 (secure 0 0 0) 0x7f8b84063860 con 0x7f8b940b3130 2024-09-21T03:35:32.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:32.879+0000 7f8b9b520640 1 -- 172.21.15.38:0/3298527958 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8b78078b90 msgr2=0x7f8b7807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:32.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:32.879+0000 7f8b9b520640 1 --2- 172.21.15.38:0/3298527958 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8b78078b90 0x7f8b7807b0a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f8b90000c00 tx=0x7f8b90006b70 comp rx=0 tx=0).stop 2024-09-21T03:35:32.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:32.879+0000 7f8b9b520640 1 -- 172.21.15.38:0/3298527958 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8b940b3130 msgr2=0x7f8b940ca3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:32.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:32.879+0000 7f8b9b520640 1 --2- 172.21.15.38:0/3298527958 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8b940b3130 0x7f8b940ca3d0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f8b84001ba0 tx=0x7f8b840039c0 comp rx=0 tx=0).stop 2024-09-21T03:35:32.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:32.879+0000 7f8b9b520640 1 -- 172.21.15.38:0/3298527958 shutdown_connections 2024-09-21T03:35:32.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:32.879+0000 7f8b9b520640 1 --2- 172.21.15.38:0/3298527958 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8b78078b90 0x7f8b7807b0a0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:32.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:32.879+0000 7f8b9b520640 1 --2- 172.21.15.38:0/3298527958 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8b9406d7b0 0x7f8b9406dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:32.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:32.879+0000 7f8b9b520640 1 --2- 172.21.15.38:0/3298527958 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8b940b3130 0x7f8b940ca3d0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:32.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:32.879+0000 7f8b9b520640 1 --2- 172.21.15.38:0/3298527958 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8b80004d40 0x7f8b940b1ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:32.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:32.879+0000 7f8b9b520640 1 -- 172.21.15.38:0/3298527958 >> 172.21.15.38:0/3298527958 conn(0x7f8b9409d830 msgr2=0x7f8b94157340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:32.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:32.879+0000 7f8b9b520640 1 -- 172.21.15.38:0/3298527958 shutdown_connections 2024-09-21T03:35:32.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:32.879+0000 7f8b9b520640 1 -- 172.21.15.38:0/3298527958 wait complete. 2024-09-21T03:35:32.896 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-21T03:35:33.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:33.022+0000 7f11cab58640 1 -- 172.21.15.38:0/4200442486 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f11c40cf190 msgr2=0x7f11c40cf5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:33.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:33.022+0000 7f11cab58640 1 --2- 172.21.15.38:0/4200442486 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f11c40cf190 0x7f11c40cf5a0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f11bc005fd0 tx=0x7f11bc02a8b0 comp rx=0 tx=0).stop 2024-09-21T03:35:33.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:33.023+0000 7f11cab58640 1 -- 172.21.15.38:0/4200442486 shutdown_connections 2024-09-21T03:35:33.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:33.023+0000 7f11cab58640 1 --2- 172.21.15.38:0/4200442486 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f11c40cf190 0x7f11c40cf5a0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:33.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:33.023+0000 7f11cab58640 1 --2- 172.21.15.38:0/4200442486 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f11c40d0f50 0x7f11c40d1360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:33.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:33.023+0000 7f11cab58640 1 -- 172.21.15.38:0/4200442486 >> 172.21.15.38:0/4200442486 conn(0x7f11c406cde0 msgr2=0x7f11c406d1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:33.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:33.023+0000 7f11cab58640 1 -- 172.21.15.38:0/4200442486 shutdown_connections 2024-09-21T03:35:33.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:33.023+0000 7f11cab58640 1 -- 172.21.15.38:0/4200442486 wait complete. 2024-09-21T03:35:33.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:33.024+0000 7f11cab58640 1 Processor -- start 2024-09-21T03:35:33.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:33.024+0000 7f11cab58640 1 -- start start 2024-09-21T03:35:33.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:33.026+0000 7f11cab58640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f11c40cf190 0x7f11c4152070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:33.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:33.026+0000 7f11cab58640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f11c40d0f50 0x7f11c41f8090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:33.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:33.026+0000 7f11cab58640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f11c4155910 0x7f11c41f8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:33.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:33.026+0000 7f11cab58640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f11c40cfc60 con 0x7f11c40cf190 2024-09-21T03:35:33.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:33.026+0000 7f11cab58640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f11c40cfae0 con 0x7f11c40d0f50 2024-09-21T03:35:33.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:33.026+0000 7f11cab58640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f11c40cfde0 con 0x7f11c4155910 2024-09-21T03:35:33.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:33.026+0000 7f11c88f4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f11c40cf190 0x7f11c4152070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:33.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:33.026+0000 7f11c88f4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f11c40cf190 0x7f11c4152070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:39462/0 (socket says 172.21.15.38:39462) 2024-09-21T03:35:33.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:33.026+0000 7f11c88f4640 1 -- 172.21.15.38:0/1945730118 learned_addr learned my addr 172.21.15.38:0/1945730118 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:35:33.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:33.026+0000 7f11c90f5640 1 --2- 172.21.15.38:0/1945730118 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f11c4155910 0x7f11c41f8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:33.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:33.026+0000 7f11c3fff640 1 --2- 172.21.15.38:0/1945730118 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f11c40d0f50 0x7f11c41f8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:33.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:33.026+0000 7f11c88f4640 1 -- 172.21.15.38:0/1945730118 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f11c4155910 msgr2=0x7f11c41f8a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:33.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:33.026+0000 7f11c88f4640 1 --2- 172.21.15.38:0/1945730118 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f11c4155910 0x7f11c41f8a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:33.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:33.026+0000 7f11c88f4640 1 -- 172.21.15.38:0/1945730118 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f11c40d0f50 msgr2=0x7f11c41f8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:33.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:33.026+0000 7f11c88f4640 1 --2- 172.21.15.38:0/1945730118 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f11c40d0f50 0x7f11c41f8090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:33.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:33.026+0000 7f11c88f4640 1 -- 172.21.15.38:0/1945730118 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11c4156f00 con 0x7f11c40cf190 2024-09-21T03:35:33.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:33.027+0000 7f11c88f4640 1 --2- 172.21.15.38:0/1945730118 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f11c40cf190 0x7f11c4152070 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f11b400c3c0 tx=0x7f11b400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:33.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:33.027+0000 7f11c17fa640 1 -- 172.21.15.38:0/1945730118 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11b4017070 con 0x7f11c40cf190 2024-09-21T03:35:33.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:33.027+0000 7f11c17fa640 1 -- 172.21.15.38:0/1945730118 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f11b40035f0 con 0x7f11c40cf190 2024-09-21T03:35:33.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:33.027+0000 7f11cab58640 1 -- 172.21.15.38:0/1945730118 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11c41571a0 con 0x7f11c40cf190 2024-09-21T03:35:33.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:33.027+0000 7f11c17fa640 1 -- 172.21.15.38:0/1945730118 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11b4003a80 con 0x7f11c40cf190 2024-09-21T03:35:33.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:33.027+0000 7f11cab58640 1 -- 172.21.15.38:0/1945730118 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11c4152b70 con 0x7f11c40cf190 2024-09-21T03:35:33.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:33.028+0000 7f11cab58640 1 -- 172.21.15.38:0/1945730118 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11c40d0290 con 0x7f11c40cf190 2024-09-21T03:35:33.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:33.029+0000 7f11c17fa640 1 -- 172.21.15.38:0/1945730118 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f11b401a070 con 0x7f11c40cf190 2024-09-21T03:35:33.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:33.029+0000 7f11c17fa640 1 --2- 172.21.15.38:0/1945730118 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f11b0078b90 0x7f11b007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:33.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:33.029+0000 7f11c17fa640 1 -- 172.21.15.38:0/1945730118 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 6331+0+0 (secure 0 0 0) 0x7f11b409a340 con 0x7f11c40cf190 2024-09-21T03:35:33.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:33.029+0000 7f11c3fff640 1 --2- 172.21.15.38:0/1945730118 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f11b0078b90 0x7f11b007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:33.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:33.030+0000 7f11c3fff640 1 --2- 172.21.15.38:0/1945730118 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f11b0078b90 0x7f11b007b0a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f11b8005d70 tx=0x7f11b8005d00 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:33.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:33.034+0000 7f11c17fa640 1 -- 172.21.15.38:0/1945730118 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f11c40d18a0 con 0x7f11c40cf190 2024-09-21T03:35:33.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:33.223+0000 7f11cab58640 1 -- 172.21.15.38:0/1945730118 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f11c40b9c30 con 0x7f11c40cf190 2024-09-21T03:35:34.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:34.169+0000 7f11c17fa640 1 -- 172.21.15.38:0/1945730118 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v45) ==== 181+0+0 (secure 0 0 0) 0x7f11b4062470 con 0x7f11c40cf190 2024-09-21T03:35:34.171 INFO:teuthology.orchestra.run.smithi038.stderr:pool 'cephfs_metadata' removed 2024-09-21T03:35:34.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:34.173+0000 7f11cab58640 1 -- 172.21.15.38:0/1945730118 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f11b0078b90 msgr2=0x7f11b007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:34.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:34.173+0000 7f11cab58640 1 --2- 172.21.15.38:0/1945730118 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f11b0078b90 0x7f11b007b0a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f11b8005d70 tx=0x7f11b8005d00 comp rx=0 tx=0).stop 2024-09-21T03:35:34.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:34.173+0000 7f11cab58640 1 -- 172.21.15.38:0/1945730118 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f11c40cf190 msgr2=0x7f11c4152070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:34.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:34.173+0000 7f11cab58640 1 --2- 172.21.15.38:0/1945730118 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f11c40cf190 0x7f11c4152070 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f11b400c3c0 tx=0x7f11b400c880 comp rx=0 tx=0).stop 2024-09-21T03:35:34.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:34.174+0000 7f11cab58640 1 -- 172.21.15.38:0/1945730118 shutdown_connections 2024-09-21T03:35:34.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:34.174+0000 7f11cab58640 1 --2- 172.21.15.38:0/1945730118 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f11b0078b90 0x7f11b007b0a0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:34.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:34.174+0000 7f11cab58640 1 --2- 172.21.15.38:0/1945730118 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f11c4155910 0x7f11c41f8a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:34.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:34.174+0000 7f11cab58640 1 --2- 172.21.15.38:0/1945730118 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f11c40d0f50 0x7f11c41f8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:34.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:34.174+0000 7f11cab58640 1 --2- 172.21.15.38:0/1945730118 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f11c40cf190 0x7f11c4152070 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:34.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:34.174+0000 7f11cab58640 1 -- 172.21.15.38:0/1945730118 >> 172.21.15.38:0/1945730118 conn(0x7f11c406cde0 msgr2=0x7f11c40b5f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:34.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:34.174+0000 7f11cab58640 1 -- 172.21.15.38:0/1945730118 shutdown_connections 2024-09-21T03:35:34.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:34.174+0000 7f11cab58640 1 -- 172.21.15.38:0/1945730118 wait complete. 2024-09-21T03:35:34.191 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-21T03:35:34.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:34.319+0000 7fe3f44a9640 1 -- 172.21.15.38:0/263384037 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe3d8004d40 msgr2=0x7fe3d8005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:34.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:34.319+0000 7fe3f44a9640 1 --2- 172.21.15.38:0/263384037 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe3d8004d40 0x7fe3d8005150 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fe3ec0b1c80 tx=0x7fe3d8005830 comp rx=0 tx=0).stop 2024-09-21T03:35:34.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:34.319+0000 7fe3f2a46640 1 -- 172.21.15.38:0/263384037 reap_dead start 2024-09-21T03:35:34.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:34.319+0000 7fe3f44a9640 1 -- 172.21.15.38:0/263384037 shutdown_connections 2024-09-21T03:35:34.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:34.319+0000 7fe3f44a9640 1 -- 172.21.15.38:0/263384037 >> 172.21.15.38:0/263384037 conn(0x7fe3ec0b3ec0 msgr2=0x7fe3ec0b42d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:34.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:34.319+0000 7fe3f44a9640 1 -- 172.21.15.38:0/263384037 shutdown_connections 2024-09-21T03:35:34.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:34.319+0000 7fe3f44a9640 1 -- 172.21.15.38:0/263384037 wait complete. 2024-09-21T03:35:34.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:34.321+0000 7fe3f44a9640 1 Processor -- start 2024-09-21T03:35:34.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:34.321+0000 7fe3f44a9640 1 -- start start 2024-09-21T03:35:34.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:34.322+0000 7fe3f44a9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe3d8004d40 0x7fe3ec154b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:34.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:34.322+0000 7fe3f44a9640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe3ec0c7ce0 0x7fe3ec0b6500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:34.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:34.322+0000 7fe3f44a9640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe3ec0c80f0 0x7fe3ec0b6a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:34.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:34.322+0000 7fe3f44a9640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fe3ec0b3940 con 0x7fe3d8004d40 2024-09-21T03:35:34.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:34.322+0000 7fe3f44a9640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fe3ec0b37c0 con 0x7fe3ec0c7ce0 2024-09-21T03:35:34.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:34.322+0000 7fe3f44a9640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fe3ec0d0410 con 0x7fe3ec0c80f0 2024-09-21T03:35:34.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:34.322+0000 7fe3f2245640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe3d8004d40 0x7fe3ec154b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:34.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:34.322+0000 7fe3f2245640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe3d8004d40 0x7fe3ec154b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:39488/0 (socket says 172.21.15.38:39488) 2024-09-21T03:35:34.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:34.322+0000 7fe3f2245640 1 -- 172.21.15.38:0/3349943597 learned_addr learned my addr 172.21.15.38:0/3349943597 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:35:34.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:34.323+0000 7fe3f2a46640 1 --2- 172.21.15.38:0/3349943597 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe3ec0c80f0 0x7fe3ec0b6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:34.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:34.323+0000 7fe3f1a44640 1 --2- 172.21.15.38:0/3349943597 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe3ec0c7ce0 0x7fe3ec0b6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:34.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:34.323+0000 7fe3f2245640 1 -- 172.21.15.38:0/3349943597 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe3ec0c80f0 msgr2=0x7fe3ec0b6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:34.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:34.323+0000 7fe3f2245640 1 --2- 172.21.15.38:0/3349943597 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe3ec0c80f0 0x7fe3ec0b6a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:34.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:34.323+0000 7fe3f2245640 1 -- 172.21.15.38:0/3349943597 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe3ec0c7ce0 msgr2=0x7fe3ec0b6500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:34.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:34.323+0000 7fe3f2245640 1 --2- 172.21.15.38:0/3349943597 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe3ec0c7ce0 0x7fe3ec0b6500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:34.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:34.323+0000 7fe3f2245640 1 -- 172.21.15.38:0/3349943597 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3ec1f3340 con 0x7fe3d8004d40 2024-09-21T03:35:34.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:34.323+0000 7fe3f2245640 1 --2- 172.21.15.38:0/3349943597 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe3d8004d40 0x7fe3ec154b00 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fe3e8011a80 tx=0x7fe3e8011f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:34.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:34.323+0000 7fe3e2ffd640 1 -- 172.21.15.38:0/3349943597 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3e800fe10 con 0x7fe3d8004d40 2024-09-21T03:35:34.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:34.323+0000 7fe3e2ffd640 1 -- 172.21.15.38:0/3349943597 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe3e8016070 con 0x7fe3d8004d40 2024-09-21T03:35:34.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:34.323+0000 7fe3f44a9640 1 -- 172.21.15.38:0/3349943597 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3ec155040 con 0x7fe3d8004d40 2024-09-21T03:35:34.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:34.323+0000 7fe3e2ffd640 1 -- 172.21.15.38:0/3349943597 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3e8012690 con 0x7fe3d8004d40 2024-09-21T03:35:34.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:34.323+0000 7fe3f44a9640 1 -- 172.21.15.38:0/3349943597 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3ec1f6e30 con 0x7fe3d8004d40 2024-09-21T03:35:34.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:34.324+0000 7fe3f44a9640 1 -- 172.21.15.38:0/3349943597 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3ec15c1c0 con 0x7fe3d8004d40 2024-09-21T03:35:34.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:34.325+0000 7fe3e2ffd640 1 -- 172.21.15.38:0/3349943597 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fe3e8018030 con 0x7fe3d8004d40 2024-09-21T03:35:34.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:34.326+0000 7fe3e2ffd640 1 --2- 172.21.15.38:0/3349943597 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fe3d0078b90 0x7fe3d007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:34.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:34.326+0000 7fe3e2ffd640 1 -- 172.21.15.38:0/3349943597 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5893+0+0 (secure 0 0 0) 0x7fe3e809b6f0 con 0x7fe3d8004d40 2024-09-21T03:35:34.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:34.326+0000 7fe3f1a44640 1 --2- 172.21.15.38:0/3349943597 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fe3d0078b90 0x7fe3d007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:34.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:34.326+0000 7fe3f1a44640 1 --2- 172.21.15.38:0/3349943597 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fe3d0078b90 0x7fe3d007b0a0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fe3dc0018a0 tx=0x7fe3dc000ed0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:34.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:34.330+0000 7fe3e2ffd640 1 -- 172.21.15.38:0/3349943597 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fe3ec15c1c0 con 0x7fe3d8004d40 2024-09-21T03:35:34.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:34.519+0000 7fe3f44a9640 1 -- 172.21.15.38:0/3349943597 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fe3ec0d0910 con 0x7fe3d8004d40 2024-09-21T03:35:35.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:35.174+0000 7fe3e2ffd640 1 -- 172.21.15.38:0/3349943597 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v46) ==== 169+0+0 (secure 0 0 0) 0x7fe3ec155040 con 0x7fe3d8004d40 2024-09-21T03:35:35.176 INFO:teuthology.orchestra.run.smithi038.stderr:pool 'cephfs_data' removed 2024-09-21T03:35:35.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:35.178+0000 7fe3f44a9640 1 -- 172.21.15.38:0/3349943597 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fe3d0078b90 msgr2=0x7fe3d007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:35.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:35.178+0000 7fe3f44a9640 1 --2- 172.21.15.38:0/3349943597 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fe3d0078b90 0x7fe3d007b0a0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fe3dc0018a0 tx=0x7fe3dc000ed0 comp rx=0 tx=0).stop 2024-09-21T03:35:35.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:35.178+0000 7fe3f44a9640 1 -- 172.21.15.38:0/3349943597 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe3d8004d40 msgr2=0x7fe3ec154b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:35.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:35.178+0000 7fe3f44a9640 1 --2- 172.21.15.38:0/3349943597 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe3d8004d40 0x7fe3ec154b00 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fe3e8011a80 tx=0x7fe3e8011f40 comp rx=0 tx=0).stop 2024-09-21T03:35:35.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:35.178+0000 7fe3f44a9640 1 -- 172.21.15.38:0/3349943597 shutdown_connections 2024-09-21T03:35:35.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:35.178+0000 7fe3f44a9640 1 --2- 172.21.15.38:0/3349943597 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fe3d0078b90 0x7fe3d007b0a0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:35.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:35.178+0000 7fe3f44a9640 1 --2- 172.21.15.38:0/3349943597 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe3ec0c80f0 0x7fe3ec0b6a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:35.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:35.178+0000 7fe3f44a9640 1 --2- 172.21.15.38:0/3349943597 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe3ec0c7ce0 0x7fe3ec0b6500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:35.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:35.178+0000 7fe3f44a9640 1 --2- 172.21.15.38:0/3349943597 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe3d8004d40 0x7fe3ec154b00 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:35.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:35.178+0000 7fe3f44a9640 1 -- 172.21.15.38:0/3349943597 >> 172.21.15.38:0/3349943597 conn(0x7fe3ec0b3ec0 msgr2=0x7fe3ec0b7550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:35.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:35.179+0000 7fe3f44a9640 1 -- 172.21.15.38:0/3349943597 shutdown_connections 2024-09-21T03:35:35.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:35.179+0000 7fe3f44a9640 1 -- 172.21.15.38:0/3349943597 wait complete. 2024-09-21T03:35:35.200 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 31 2024-09-21T03:35:35.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:35.322+0000 7f6588713640 1 -- 172.21.15.38:0/1457447375 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f65800ccc40 msgr2=0x7f658015ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:35.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:35.322+0000 7f6588713640 1 --2- 172.21.15.38:0/1457447375 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f65800ccc40 0x7f658015ca90 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f657c005fd0 tx=0x7f657c02a540 comp rx=0 tx=0).stop 2024-09-21T03:35:35.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:35.322+0000 7f6588713640 1 -- 172.21.15.38:0/1457447375 shutdown_connections 2024-09-21T03:35:35.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:35.322+0000 7f6588713640 1 --2- 172.21.15.38:0/1457447375 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f65800d0d80 0x7f65800b5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:35.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:35.322+0000 7f6588713640 1 --2- 172.21.15.38:0/1457447375 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f65800ccc40 0x7f658015ca90 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:35.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:35.322+0000 7f6588713640 1 -- 172.21.15.38:0/1457447375 >> 172.21.15.38:0/1457447375 conn(0x7f65800b46e0 msgr2=0x7f65800d23e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:35.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:35.322+0000 7f6588713640 1 -- 172.21.15.38:0/1457447375 shutdown_connections 2024-09-21T03:35:35.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:35.323+0000 7f6588713640 1 -- 172.21.15.38:0/1457447375 wait complete. 2024-09-21T03:35:35.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:35.324+0000 7f6588713640 1 Processor -- start 2024-09-21T03:35:35.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:35.324+0000 7f6588713640 1 -- start start 2024-09-21T03:35:35.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:35.325+0000 7f6588713640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f65800ccc40 0x7f658015ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:35.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:35.325+0000 7f6588713640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f65800cd440 0x7f658015faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:35.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:35.326+0000 7f6588713640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f658015e2e0 0x7f65801589a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:35.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:35.326+0000 7f6588713640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f65800b6650 con 0x7f65800ccc40 2024-09-21T03:35:35.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:35.326+0000 7f6588713640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f65800b64d0 con 0x7f65800cd440 2024-09-21T03:35:35.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:35.326+0000 7f6588713640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f65800b67d0 con 0x7f658015e2e0 2024-09-21T03:35:35.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:35.326+0000 7f65864af640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f65800ccc40 0x7f658015ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:35.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:35.326+0000 7f65864af640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f65800ccc40 0x7f658015ae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:39500/0 (socket says 172.21.15.38:39500) 2024-09-21T03:35:35.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:35.326+0000 7f65864af640 1 -- 172.21.15.38:0/2367309632 learned_addr learned my addr 172.21.15.38:0/2367309632 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:35:35.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:35.326+0000 7f6585cae640 1 --2- 172.21.15.38:0/2367309632 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f65800cd440 0x7f658015faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:35.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:35.326+0000 7f6586cb0640 1 --2- 172.21.15.38:0/2367309632 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f658015e2e0 0x7f65801589a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:35.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:35.326+0000 7f65864af640 1 -- 172.21.15.38:0/2367309632 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f658015e2e0 msgr2=0x7f65801589a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:35.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:35.326+0000 7f65864af640 1 --2- 172.21.15.38:0/2367309632 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f658015e2e0 0x7f65801589a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:35.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:35.326+0000 7f65864af640 1 -- 172.21.15.38:0/2367309632 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f65800cd440 msgr2=0x7f658015faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:35.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:35.326+0000 7f65864af640 1 --2- 172.21.15.38:0/2367309632 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f65800cd440 0x7f658015faf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:35.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:35.326+0000 7f65864af640 1 -- 172.21.15.38:0/2367309632 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6580160030 con 0x7f65800ccc40 2024-09-21T03:35:35.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:35.326+0000 7f65864af640 1 --2- 172.21.15.38:0/2367309632 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f65800ccc40 0x7f658015ae80 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f657c006010 tx=0x7f657c0038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:35.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:35.326+0000 7f6576ffd640 1 -- 172.21.15.38:0/2367309632 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f657c03e030 con 0x7f65800ccc40 2024-09-21T03:35:35.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:35.327+0000 7f6576ffd640 1 -- 172.21.15.38:0/2367309632 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f657c003d90 con 0x7f65800ccc40 2024-09-21T03:35:35.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:35.327+0000 7f6576ffd640 1 -- 172.21.15.38:0/2367309632 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f657c035040 con 0x7f65800ccc40 2024-09-21T03:35:35.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:35.327+0000 7f6588713640 1 -- 172.21.15.38:0/2367309632 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f658015ea50 con 0x7f65800ccc40 2024-09-21T03:35:35.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:35.327+0000 7f6588713640 1 -- 172.21.15.38:0/2367309632 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f658015f010 con 0x7f65800ccc40 2024-09-21T03:35:35.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:35.328+0000 7f6588713640 1 -- 172.21.15.38:0/2367309632 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6580158780 con 0x7f65800ccc40 2024-09-21T03:35:35.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:35.328+0000 7f6576ffd640 1 -- 172.21.15.38:0/2367309632 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f657c041030 con 0x7f65800ccc40 2024-09-21T03:35:35.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:35.329+0000 7f6576ffd640 1 --2- 172.21.15.38:0/2367309632 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6564078b90 0x7f656407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:35.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:35.329+0000 7f6576ffd640 1 -- 172.21.15.38:0/2367309632 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5421+0+0 (secure 0 0 0) 0x7f657c0bb360 con 0x7f65800ccc40 2024-09-21T03:35:35.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:35.330+0000 7f6585cae640 1 --2- 172.21.15.38:0/2367309632 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6564078b90 0x7f656407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:35.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:35.330+0000 7f6585cae640 1 --2- 172.21.15.38:0/2367309632 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6564078b90 0x7f656407b0a0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f658015ba00 tx=0x7f6570009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:35.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:35.334+0000 7f6576ffd640 1 -- 172.21.15.38:0/2367309632 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f657c0831f0 con 0x7f65800ccc40 2024-09-21T03:35:35.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:35.522+0000 7f6588713640 1 -- 172.21.15.38:0/2367309632 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "config reset", "num": 31} v 0) -- 0x7f65800717d0 con 0x7f65800ccc40 2024-09-21T03:35:35.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:35.525+0000 7f6576ffd640 1 -- 172.21.15.38:0/2367309632 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 31}]=0 v33) ==== 71+0+0 (secure 0 0 0) 0x7f657c087c20 con 0x7f65800ccc40 2024-09-21T03:35:35.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:35.528+0000 7f6588713640 1 -- 172.21.15.38:0/2367309632 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6564078b90 msgr2=0x7f656407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:35.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:35.528+0000 7f6588713640 1 --2- 172.21.15.38:0/2367309632 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6564078b90 0x7f656407b0a0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f658015ba00 tx=0x7f6570009040 comp rx=0 tx=0).stop 2024-09-21T03:35:35.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:35.528+0000 7f6588713640 1 -- 172.21.15.38:0/2367309632 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f65800ccc40 msgr2=0x7f658015ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:35.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:35.528+0000 7f6588713640 1 --2- 172.21.15.38:0/2367309632 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f65800ccc40 0x7f658015ae80 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f657c006010 tx=0x7f657c0038a0 comp rx=0 tx=0).stop 2024-09-21T03:35:35.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:35.528+0000 7f6588713640 1 -- 172.21.15.38:0/2367309632 shutdown_connections 2024-09-21T03:35:35.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:35.528+0000 7f6588713640 1 --2- 172.21.15.38:0/2367309632 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6564078b90 0x7f656407b0a0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:35.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:35.528+0000 7f6588713640 1 --2- 172.21.15.38:0/2367309632 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f658015e2e0 0x7f65801589a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:35.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:35.528+0000 7f6588713640 1 --2- 172.21.15.38:0/2367309632 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f65800cd440 0x7f658015faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:35.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:35.528+0000 7f6588713640 1 --2- 172.21.15.38:0/2367309632 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f65800ccc40 0x7f658015ae80 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:35.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:35.528+0000 7f6588713640 1 -- 172.21.15.38:0/2367309632 >> 172.21.15.38:0/2367309632 conn(0x7f65800b46e0 msgr2=0x7f6580070ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:35.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:35.528+0000 7f6588713640 1 -- 172.21.15.38:0/2367309632 shutdown_connections 2024-09-21T03:35:35.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:35.528+0000 7f6588713640 1 -- 172.21.15.38:0/2367309632 wait complete. 2024-09-21T03:35:35.541 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_damage.TestDamage.test_damaged_dentry' 2024-09-21T03:35:35.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:35.660+0000 7f3dd1f8b640 1 -- 172.21.15.38:0/2901415287 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3dcc073e10 msgr2=0x7f3dcc074220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:35.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:35.660+0000 7f3dd1f8b640 1 --2- 172.21.15.38:0/2901415287 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3dcc073e10 0x7f3dcc074220 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f3dcc0735e0 tx=0x7f3dbc02a560 comp rx=0 tx=0).stop 2024-09-21T03:35:35.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:35.660+0000 7f3dd1f8b640 1 -- 172.21.15.38:0/2901415287 shutdown_connections 2024-09-21T03:35:35.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:35.660+0000 7f3dd1f8b640 1 --2- 172.21.15.38:0/2901415287 >> v2:172.21.15.195:3300/0 conn(0x7f3dcc15bcd0 0x7f3dcc15c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:35.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:35.660+0000 7f3dd1f8b640 1 --2- 172.21.15.38:0/2901415287 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3dcc073e10 0x7f3dcc074220 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:35.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:35.660+0000 7f3dd1f8b640 1 -- 172.21.15.38:0/2901415287 >> 172.21.15.38:0/2901415287 conn(0x7f3dcc0cdc10 msgr2=0x7f3dcc0ce020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:35.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:35.660+0000 7f3dd1f8b640 1 -- 172.21.15.38:0/2901415287 shutdown_connections 2024-09-21T03:35:35.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:35.660+0000 7f3dd1f8b640 1 -- 172.21.15.38:0/2901415287 wait complete. 2024-09-21T03:35:35.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:35.662+0000 7f3dd1f8b640 1 Processor -- start 2024-09-21T03:35:35.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:35.662+0000 7f3dd1f8b640 1 -- start start 2024-09-21T03:35:35.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:35.663+0000 7f3dd1f8b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3dcc073e10 0x7f3dcc1f7500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:35.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:35.664+0000 7f3dd1f8b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3dcc0cffc0 0x7f3dcc1412d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:35.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:35.664+0000 7f3dd1f8b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3dcc15bcd0 0x7f3dcc0b62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:35.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:35.664+0000 7f3dd1f8b640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f3dcc1527f0 con 0x7f3dcc073e10 2024-09-21T03:35:35.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:35.664+0000 7f3dd1f8b640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f3dcc06ef60 con 0x7f3dcc0cffc0 2024-09-21T03:35:35.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:35.664+0000 7f3dd1f8b640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f3dcc152970 con 0x7f3dcc15bcd0 2024-09-21T03:35:35.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:35.664+0000 7f3dcb7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3dcc073e10 0x7f3dcc1f7500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:35.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:35.664+0000 7f3dcb7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3dcc073e10 0x7f3dcc1f7500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:39516/0 (socket says 172.21.15.38:39516) 2024-09-21T03:35:35.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:35.664+0000 7f3dcb7fe640 1 -- 172.21.15.38:0/1477395608 learned_addr learned my addr 172.21.15.38:0/1477395608 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:35:35.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:35.664+0000 7f3dcaffd640 1 --2- 172.21.15.38:0/1477395608 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3dcc0cffc0 0x7f3dcc1412d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:35.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:35.664+0000 7f3dcbfff640 1 --2- 172.21.15.38:0/1477395608 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3dcc15bcd0 0x7f3dcc0b62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:35.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:35.664+0000 7f3dcb7fe640 1 -- 172.21.15.38:0/1477395608 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3dcc15bcd0 msgr2=0x7f3dcc0b62d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:35.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:35.664+0000 7f3dcb7fe640 1 --2- 172.21.15.38:0/1477395608 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3dcc15bcd0 0x7f3dcc0b62d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:35.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:35.664+0000 7f3dcb7fe640 1 -- 172.21.15.38:0/1477395608 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3dcc0cffc0 msgr2=0x7f3dcc1412d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:35.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:35.664+0000 7f3dcb7fe640 1 --2- 172.21.15.38:0/1477395608 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3dcc0cffc0 0x7f3dcc1412d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:35.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:35.664+0000 7f3dcb7fe640 1 -- 172.21.15.38:0/1477395608 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3dcc1f7a40 con 0x7f3dcc073e10 2024-09-21T03:35:35.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:35.664+0000 7f3dcbfff640 1 --2- 172.21.15.38:0/1477395608 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3dcc15bcd0 0x7f3dcc0b62d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:35:35.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:35.664+0000 7f3dcb7fe640 1 --2- 172.21.15.38:0/1477395608 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3dcc073e10 0x7f3dcc1f7500 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f3dbc034040 tx=0x7f3dbc0039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:35.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:35.664+0000 7f3db3fff640 1 -- 172.21.15.38:0/1477395608 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dbc03e030 con 0x7f3dcc073e10 2024-09-21T03:35:35.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:35.664+0000 7f3db3fff640 1 -- 172.21.15.38:0/1477395608 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3dbc0047a0 con 0x7f3dcc073e10 2024-09-21T03:35:35.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:35.664+0000 7f3dd1f8b640 1 -- 172.21.15.38:0/1477395608 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3dcc1442a0 con 0x7f3dcc073e10 2024-09-21T03:35:35.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:35.664+0000 7f3db3fff640 1 -- 172.21.15.38:0/1477395608 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dbc035040 con 0x7f3dcc073e10 2024-09-21T03:35:35.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:35.665+0000 7f3dd1f8b640 1 -- 172.21.15.38:0/1477395608 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3dcc144860 con 0x7f3dcc073e10 2024-09-21T03:35:35.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:35.665+0000 7f3dd1f8b640 1 -- 172.21.15.38:0/1477395608 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3dcc0d0df0 con 0x7f3dcc073e10 2024-09-21T03:35:35.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:35.666+0000 7f3db3fff640 1 -- 172.21.15.38:0/1477395608 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f3dbc041030 con 0x7f3dcc073e10 2024-09-21T03:35:35.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:35.667+0000 7f3db3fff640 1 --2- 172.21.15.38:0/1477395608 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3db8078bc0 0x7f3db807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:35.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:35.667+0000 7f3db3fff640 1 -- 172.21.15.38:0/1477395608 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5421+0+0 (secure 0 0 0) 0x7f3dbc0bb3a0 con 0x7f3dcc073e10 2024-09-21T03:35:35.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:35.667+0000 7f3dcaffd640 1 --2- 172.21.15.38:0/1477395608 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3db8078bc0 0x7f3db807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:35.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:35.668+0000 7f3dcaffd640 1 --2- 172.21.15.38:0/1477395608 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3db8078bc0 0x7f3db807b0d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f3dcc1514b0 tx=0x7f3dc000a250 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:35.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:35.672+0000 7f3db3fff640 1 -- 172.21.15.38:0/1477395608 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f3dcc074760 con 0x7f3dcc073e10 2024-09-21T03:35:35.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:35.932+0000 7f3dd1f8b640 1 -- 172.21.15.38:0/1477395608 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_damage.TestDamage.test_damaged_dentry"]} v 0) -- 0x7f3dcc151200 con 0x7f3dcc073e10 2024-09-21T03:35:36.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:36.176+0000 7f3db3fff640 1 -- 172.21.15.38:0/1477395608 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_damage.TestDamage.test_damaged_dentry"]}]=0 v121) ==== 134+0+0 (secure 0 0 0) 0x7f3dbc083340 con 0x7f3dcc073e10 2024-09-21T03:35:36.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:36.178+0000 7f3dd1f8b640 1 -- 172.21.15.38:0/1477395608 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3db8078bc0 msgr2=0x7f3db807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:36.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:36.178+0000 7f3dd1f8b640 1 --2- 172.21.15.38:0/1477395608 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3db8078bc0 0x7f3db807b0d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f3dcc1514b0 tx=0x7f3dc000a250 comp rx=0 tx=0).stop 2024-09-21T03:35:36.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:36.178+0000 7f3dd1f8b640 1 -- 172.21.15.38:0/1477395608 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3dcc073e10 msgr2=0x7f3dcc1f7500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:36.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:36.178+0000 7f3dd1f8b640 1 --2- 172.21.15.38:0/1477395608 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3dcc073e10 0x7f3dcc1f7500 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f3dbc034040 tx=0x7f3dbc0039e0 comp rx=0 tx=0).stop 2024-09-21T03:35:36.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:36.178+0000 7f3dd1f8b640 1 -- 172.21.15.38:0/1477395608 shutdown_connections 2024-09-21T03:35:36.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:36.178+0000 7f3dd1f8b640 1 --2- 172.21.15.38:0/1477395608 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3db8078bc0 0x7f3db807b0d0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:36.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:36.178+0000 7f3dd1f8b640 1 --2- 172.21.15.38:0/1477395608 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3dcc15bcd0 0x7f3dcc0b62d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:36.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:36.178+0000 7f3dd1f8b640 1 --2- 172.21.15.38:0/1477395608 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3dcc0cffc0 0x7f3dcc1412d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:36.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:36.178+0000 7f3dd1f8b640 1 --2- 172.21.15.38:0/1477395608 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3dcc073e10 0x7f3dcc1f7500 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:36.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:36.178+0000 7f3dd1f8b640 1 -- 172.21.15.38:0/1477395608 >> 172.21.15.38:0/1477395608 conn(0x7f3dcc0cdc10 msgr2=0x7f3dcc0b11f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:36.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:36.179+0000 7f3dd1f8b640 1 -- 172.21.15.38:0/1477395608 shutdown_connections 2024-09-21T03:35:36.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:36.179+0000 7f3dd1f8b640 1 -- 172.21.15.38:0/1477395608 wait complete. 2024-09-21T03:35:36.192 INFO:tasks.cephfs_test_runner:test_damaged_dentry (tasks.cephfs.test_damage.TestDamage) ... ok 2024-09-21T03:35:36.192 INFO:tasks.cephfs_test_runner:Starting test: test_dentry_first_existing (tasks.cephfs.test_damage.TestDamage) That the MDS won't abort when the dentry is already known to be damaged. 2024-09-21T03:35:36.192 INFO:tasks.cephfs_test_runner:test_dentry_first_existing (tasks.cephfs.test_damage.TestDamage) 2024-09-21T03:35:36.193 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T03:35:36.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:36.309+0000 7f568b2bd640 1 -- 172.21.15.38:0/2906880230 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5670005250 msgr2=0x7f567000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:36.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:36.309+0000 7f568b2bd640 1 --2- 172.21.15.38:0/2906880230 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5670005250 0x7f567000b9c0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f56841571a0 tx=0x7f5670005830 comp rx=0 tx=0).stop 2024-09-21T03:35:36.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:36.309+0000 7f568b2bd640 1 -- 172.21.15.38:0/2906880230 shutdown_connections 2024-09-21T03:35:36.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:36.309+0000 7f568b2bd640 1 --2- 172.21.15.38:0/2906880230 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5670005250 0x7f567000b9c0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:36.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:36.309+0000 7f568b2bd640 1 -- 172.21.15.38:0/2906880230 >> 172.21.15.38:0/2906880230 conn(0x7f56840cef40 msgr2=0x7f56840cf350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:36.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:36.309+0000 7f568b2bd640 1 -- 172.21.15.38:0/2906880230 shutdown_connections 2024-09-21T03:35:36.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:36.309+0000 7f568b2bd640 1 -- 172.21.15.38:0/2906880230 wait complete. 2024-09-21T03:35:36.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:36.311+0000 7f568b2bd640 1 Processor -- start 2024-09-21T03:35:36.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:36.311+0000 7f568b2bd640 1 -- start start 2024-09-21T03:35:36.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:36.312+0000 7f568b2bd640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5670005250 0x7f56841560b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:36.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:36.312+0000 7f568b2bd640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5684156a00 0x7f56840b6f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:36.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:36.312+0000 7f568b2bd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f56840b7460 0x7f568414f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:36.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:36.312+0000 7f568b2bd640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f56840d1b50 con 0x7f56840b7460 2024-09-21T03:35:36.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:36.312+0000 7f568b2bd640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f56840d19d0 con 0x7f5670005250 2024-09-21T03:35:36.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:36.312+0000 7f568b2bd640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f56841566b0 con 0x7f5684156a00 2024-09-21T03:35:36.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:36.312+0000 7f568985a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f56840b7460 0x7f568414f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:36.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:36.312+0000 7f568985a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f56840b7460 0x7f568414f4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:39530/0 (socket says 172.21.15.38:39530) 2024-09-21T03:35:36.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:36.312+0000 7f568985a640 1 -- 172.21.15.38:0/3789689011 learned_addr learned my addr 172.21.15.38:0/3789689011 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:35:36.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:36.313+0000 7f5688858640 1 --2- 172.21.15.38:0/3789689011 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5684156a00 0x7f56840b6f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:36.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:36.313+0000 7f5689059640 1 --2- 172.21.15.38:0/3789689011 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5670005250 0x7f56841560b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:36.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:36.313+0000 7f568985a640 1 -- 172.21.15.38:0/3789689011 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5684156a00 msgr2=0x7f56840b6f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:36.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:36.313+0000 7f568985a640 1 --2- 172.21.15.38:0/3789689011 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5684156a00 0x7f56840b6f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:36.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:36.313+0000 7f568985a640 1 -- 172.21.15.38:0/3789689011 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5670005250 msgr2=0x7f56841560b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:36.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:36.313+0000 7f568985a640 1 --2- 172.21.15.38:0/3789689011 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5670005250 0x7f56841560b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:36.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:36.313+0000 7f568985a640 1 -- 172.21.15.38:0/3789689011 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56840b7870 con 0x7f56840b7460 2024-09-21T03:35:36.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:36.313+0000 7f568985a640 1 --2- 172.21.15.38:0/3789689011 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f56840b7460 0x7f568414f4c0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f567c00c6d0 tx=0x7f567c004bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:36.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:36.313+0000 7f5679ffb640 1 -- 172.21.15.38:0/3789689011 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f567c039070 con 0x7f56840b7460 2024-09-21T03:35:36.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:36.313+0000 7f5679ffb640 1 -- 172.21.15.38:0/3789689011 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f567c02bca0 con 0x7f56840b7460 2024-09-21T03:35:36.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:36.313+0000 7f568b2bd640 1 -- 172.21.15.38:0/3789689011 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56841f66f0 con 0x7f56840b7460 2024-09-21T03:35:36.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:36.313+0000 7f5679ffb640 1 -- 172.21.15.38:0/3789689011 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f567c006290 con 0x7f56840b7460 2024-09-21T03:35:36.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:36.313+0000 7f568b2bd640 1 -- 172.21.15.38:0/3789689011 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56841f6cb0 con 0x7f56840b7460 2024-09-21T03:35:36.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:36.314+0000 7f568b2bd640 1 -- 172.21.15.38:0/3789689011 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56840ccd10 con 0x7f56840b7460 2024-09-21T03:35:36.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:36.315+0000 7f5679ffb640 1 -- 172.21.15.38:0/3789689011 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f567c006430 con 0x7f56840b7460 2024-09-21T03:35:36.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:36.316+0000 7f5679ffb640 1 --2- 172.21.15.38:0/3789689011 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5668078bc0 0x7f566807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:36.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:36.316+0000 7f5679ffb640 1 -- 172.21.15.38:0/3789689011 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5421+0+0 (secure 0 0 0) 0x7f567c0baec0 con 0x7f56840b7460 2024-09-21T03:35:36.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:36.316+0000 7f5689059640 1 --2- 172.21.15.38:0/3789689011 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5668078bc0 0x7f566807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:36.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:36.317+0000 7f5689059640 1 --2- 172.21.15.38:0/3789689011 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5668078bc0 0x7f566807b0d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f5680005e00 tx=0x7f5680005f90 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:36.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:36.320+0000 7f5679ffb640 1 -- 172.21.15.38:0/3789689011 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f56840b7870 con 0x7f56840b7460 2024-09-21T03:35:36.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:36.508+0000 7f568b2bd640 1 -- 172.21.15.38:0/3789689011 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f56840b6460 con 0x7f56840b7460 2024-09-21T03:35:36.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:36.509+0000 7f5679ffb640 1 -- 172.21.15.38:0/3789689011 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) ==== 74+0+13459 (secure 0 0 0) 0x7f567c082e60 con 0x7f56840b7460 2024-09-21T03:35:36.511 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:35:36.512 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":46,"fsid":"9d50d66f-656b-4fe2-8e3b-fe53b79c4d43","created":"2024-09-21T03:33:10.677865+0000","modified":"2024-09-21T03:35:35.170408+0000","last_up_change":"2024-09-21T03:33:24.023421+0000","last_in_change":"2024-09-21T03:33:14.296193+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T03:33:22.992573+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dbab183f-7e63-48b8-960a-ae55fd5bf565","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6801","nonce":2381639433}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6803","nonce":2381639433}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6807","nonce":2381639433}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6805","nonce":2381639433}]},"public_addr":"172.21.15.38:6801/2381639433","cluster_addr":"172.21.15.38:6803/2381639433","heartbeat_back_addr":"172.21.15.38:6807/2381639433","heartbeat_front_addr":"172.21.15.38:6805/2381639433","state":["exists","up"]},{"osd":1,"uuid":"fa1c0e8a-ebc1-4359-8146-bd5342abbc16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6817","nonce":1491818732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6819","nonce":1491818732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6823","nonce":1491818732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6821","nonce":1491818732}]},"public_addr":"172.21.15.38:6817/1491818732","cluster_addr":"172.21.15.38:6819/1491818732","heartbeat_back_addr":"172.21.15.38:6823/1491818732","heartbeat_front_addr":"172.21.15.38:6821/1491818732","state":["exists","up"]},{"osd":2,"uuid":"2ed3cd2d-385d-4fc6-9869-214552a84803","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6825","nonce":2577858902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6827","nonce":2577858902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6831","nonce":2577858902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6829","nonce":2577858902}]},"public_addr":"172.21.15.38:6825/2577858902","cluster_addr":"172.21.15.38:6827/2577858902","heartbeat_back_addr":"172.21.15.38:6831/2577858902","heartbeat_front_addr":"172.21.15.38:6829/2577858902","state":["exists","up"]},{"osd":3,"uuid":"62d1d95e-919d-4250-be71-1d34cfa70dbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6809","nonce":1286810294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6811","nonce":1286810294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6815","nonce":1286810294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6813","nonce":1286810294}]},"public_addr":"172.21.15.38:6809/1286810294","cluster_addr":"172.21.15.38:6811/1286810294","heartbeat_back_addr":"172.21.15.38:6815/1286810294","heartbeat_front_addr":"172.21.15.38:6813/1286810294","state":["exists","up"]},{"osd":4,"uuid":"7c9f9314-99d7-4d73-8392-37809a9f54b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6803","nonce":3401837571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6805","nonce":3401837571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6809","nonce":3401837571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6807","nonce":3401837571}]},"public_addr":"172.21.15.195:6803/3401837571","cluster_addr":"172.21.15.195:6805/3401837571","heartbeat_back_addr":"172.21.15.195:6809/3401837571","heartbeat_front_addr":"172.21.15.195:6807/3401837571","state":["exists","up"]},{"osd":5,"uuid":"bde4fe9f-8414-4bba-bf4a-43973d1976bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6811","nonce":2080960180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6813","nonce":2080960180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6817","nonce":2080960180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6815","nonce":2080960180}]},"public_addr":"172.21.15.195:6811/2080960180","cluster_addr":"172.21.15.195:6813/2080960180","heartbeat_back_addr":"172.21.15.195:6817/2080960180","heartbeat_front_addr":"172.21.15.195:6815/2080960180","state":["exists","up"]},{"osd":6,"uuid":"6ee6d7f4-1a5f-4a4c-ac16-48fe05f6a002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6827","nonce":4055825675}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6829","nonce":4055825675}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6833","nonce":4055825675}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6831","nonce":4055825675}]},"public_addr":"172.21.15.195:6827/4055825675","cluster_addr":"172.21.15.195:6829/4055825675","heartbeat_back_addr":"172.21.15.195:6833/4055825675","heartbeat_front_addr":"172.21.15.195:6831/4055825675","state":["exists","up"]},{"osd":7,"uuid":"efeb5f51-329a-4484-8290-92e7d3503d05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6819","nonce":1606106342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6821","nonce":1606106342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6825","nonce":1606106342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6823","nonce":1606106342}]},"public_addr":"172.21.15.195:6819/1606106342","cluster_addr":"172.21.15.195:6821/1606106342","heartbeat_back_addr":"172.21.15.195:6825/1606106342","heartbeat_front_addr":"172.21.15.195:6823/1606106342","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.901343+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:21.115003+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.901623+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.835238+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.132199+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:19.817560+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.829690+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:21.087300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.38:6832/3533947700":"2024-09-22T03:35:31.874255+0000","172.21.15.38:6834/3533947700":"2024-09-22T03:35:31.874255+0000","172.21.15.38:6834/1865253656":"2024-09-22T03:34:47.895300+0000","172.21.15.38:6832/1865253656":"2024-09-22T03:34:47.895300+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T03:35:36.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:36.511+0000 7f568b2bd640 1 -- 172.21.15.38:0/3789689011 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5668078bc0 msgr2=0x7f566807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:36.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:36.511+0000 7f568b2bd640 1 --2- 172.21.15.38:0/3789689011 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5668078bc0 0x7f566807b0d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f5680005e00 tx=0x7f5680005f90 comp rx=0 tx=0).stop 2024-09-21T03:35:36.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:36.511+0000 7f568b2bd640 1 -- 172.21.15.38:0/3789689011 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f56840b7460 msgr2=0x7f568414f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:36.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:36.511+0000 7f568b2bd640 1 --2- 172.21.15.38:0/3789689011 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f56840b7460 0x7f568414f4c0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f567c00c6d0 tx=0x7f567c004bb0 comp rx=0 tx=0).stop 2024-09-21T03:35:36.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:36.512+0000 7f568b2bd640 1 -- 172.21.15.38:0/3789689011 shutdown_connections 2024-09-21T03:35:36.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:36.512+0000 7f568b2bd640 1 --2- 172.21.15.38:0/3789689011 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f56840b7460 0x7f568414f4c0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:36.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:36.512+0000 7f568b2bd640 1 --2- 172.21.15.38:0/3789689011 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5668078bc0 0x7f566807b0d0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:36.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:36.512+0000 7f568b2bd640 1 --2- 172.21.15.38:0/3789689011 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5684156a00 0x7f56840b6f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:36.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:36.512+0000 7f568b2bd640 1 --2- 172.21.15.38:0/3789689011 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5670005250 0x7f56841560b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:36.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:36.512+0000 7f568b2bd640 1 -- 172.21.15.38:0/3789689011 >> 172.21.15.38:0/3789689011 conn(0x7f56840cef40 msgr2=0x7f5684070100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:36.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:36.512+0000 7f568b2bd640 1 -- 172.21.15.38:0/3789689011 shutdown_connections 2024-09-21T03:35:36.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:36.512+0000 7f568b2bd640 1 -- 172.21.15.38:0/3789689011 wait complete. 2024-09-21T03:35:36.526 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T03:33:22.992573+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-21T03:35:36.526 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-21T03:35:36.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:36.644+0000 7f65a084d640 1 -- 172.21.15.38:0/449718517 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f659c0ccc40 msgr2=0x7f659c15ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:36.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:36.644+0000 7f65a084d640 1 --2- 172.21.15.38:0/449718517 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f659c0ccc40 0x7f659c15ca90 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f658c005fd0 tx=0x7f658c02a540 comp rx=0 tx=0).stop 2024-09-21T03:35:36.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:36.644+0000 7f65a084d640 1 -- 172.21.15.38:0/449718517 shutdown_connections 2024-09-21T03:35:36.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:36.644+0000 7f65a084d640 1 --2- 172.21.15.38:0/449718517 >> v2:172.21.15.195:3300/0 conn(0x7f659c0cd440 0x7f659c0d0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:36.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:36.644+0000 7f65a084d640 1 --2- 172.21.15.38:0/449718517 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f659c0ccc40 0x7f659c15ca90 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:36.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:36.644+0000 7f65a084d640 1 -- 172.21.15.38:0/449718517 >> 172.21.15.38:0/449718517 conn(0x7f659c0b46e0 msgr2=0x7f659c0d23e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:36.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:36.644+0000 7f65a084d640 1 -- 172.21.15.38:0/449718517 shutdown_connections 2024-09-21T03:35:36.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:36.644+0000 7f65a084d640 1 -- 172.21.15.38:0/449718517 wait complete. 2024-09-21T03:35:36.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:36.646+0000 7f65a084d640 1 Processor -- start 2024-09-21T03:35:36.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:36.646+0000 7f65a084d640 1 -- start start 2024-09-21T03:35:36.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:36.647+0000 7f65a084d640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f659c0ccc40 0x7f659c15f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:36.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:36.647+0000 7f65a084d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f659c0cd440 0x7f659c15f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:36.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:36.647+0000 7f65a084d640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f659c0d0bb0 0x7f659c0c9960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:36.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:36.647+0000 7f65a084d640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f659c070810 con 0x7f659c0cd440 2024-09-21T03:35:36.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:36.647+0000 7f65a084d640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f659c0b6e00 con 0x7f659c0d0bb0 2024-09-21T03:35:36.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:36.647+0000 7f65a084d640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f659c070990 con 0x7f659c0ccc40 2024-09-21T03:35:36.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:36.647+0000 7f6599d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f659c0cd440 0x7f659c15f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:36.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:36.647+0000 7f6599d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f659c0cd440 0x7f659c15f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:39532/0 (socket says 172.21.15.38:39532) 2024-09-21T03:35:36.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:36.647+0000 7f6599d9b640 1 -- 172.21.15.38:0/3057258403 learned_addr learned my addr 172.21.15.38:0/3057258403 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:35:36.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:36.647+0000 7f659a59c640 1 --2- 172.21.15.38:0/3057258403 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f659c0ccc40 0x7f659c15f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:36.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:36.647+0000 7f659ad9d640 1 --2- 172.21.15.38:0/3057258403 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f659c0d0bb0 0x7f659c0c9960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:36.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:36.647+0000 7f6599d9b640 1 -- 172.21.15.38:0/3057258403 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f659c0ccc40 msgr2=0x7f659c15f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:36.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:36.647+0000 7f6599d9b640 1 --2- 172.21.15.38:0/3057258403 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f659c0ccc40 0x7f659c15f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:36.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:36.647+0000 7f6599d9b640 1 -- 172.21.15.38:0/3057258403 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f659c0d0bb0 msgr2=0x7f659c0c9960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:36.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:36.647+0000 7f6599d9b640 1 --2- 172.21.15.38:0/3057258403 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f659c0d0bb0 0x7f659c0c9960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:36.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:36.648+0000 7f6599d9b640 1 -- 172.21.15.38:0/3057258403 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f659c1f8c60 con 0x7f659c0cd440 2024-09-21T03:35:36.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:36.648+0000 7f6599d9b640 1 --2- 172.21.15.38:0/3057258403 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f659c0cd440 0x7f659c15f720 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f658400e790 tx=0x7f658400ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:36.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:36.648+0000 7f6592ffd640 1 -- 172.21.15.38:0/3057258403 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f658400eea0 con 0x7f659c0cd440 2024-09-21T03:35:36.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:36.648+0000 7f6592ffd640 1 -- 172.21.15.38:0/3057258403 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f658400cdb0 con 0x7f659c0cd440 2024-09-21T03:35:36.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:36.648+0000 7f65a084d640 1 -- 172.21.15.38:0/3057258403 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f659c1f8f40 con 0x7f659c0cd440 2024-09-21T03:35:36.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:36.648+0000 7f6592ffd640 1 -- 172.21.15.38:0/3057258403 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6584010690 con 0x7f659c0cd440 2024-09-21T03:35:36.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:36.648+0000 7f65a084d640 1 -- 172.21.15.38:0/3057258403 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f659c1f9500 con 0x7f659c0cd440 2024-09-21T03:35:36.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:36.649+0000 7f65a084d640 1 -- 172.21.15.38:0/3057258403 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f659c070ed0 con 0x7f659c0cd440 2024-09-21T03:35:36.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:36.650+0000 7f6592ffd640 1 -- 172.21.15.38:0/3057258403 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f6584010830 con 0x7f659c0cd440 2024-09-21T03:35:36.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:36.650+0000 7f6592ffd640 1 --2- 172.21.15.38:0/3057258403 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f657c078b90 0x7f657c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:36.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:36.651+0000 7f6592ffd640 1 -- 172.21.15.38:0/3057258403 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5421+0+0 (secure 0 0 0) 0x7f658409b5f0 con 0x7f659c0cd440 2024-09-21T03:35:36.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:36.651+0000 7f659a59c640 1 --2- 172.21.15.38:0/3057258403 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f657c078b90 0x7f657c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:36.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:36.651+0000 7f659a59c640 1 --2- 172.21.15.38:0/3057258403 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f657c078b90 0x7f657c07b0a0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f658c033000 tx=0x7f658c004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:36.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:36.655+0000 7f6592ffd640 1 -- 172.21.15.38:0/3057258403 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f6584063480 con 0x7f659c0cd440 2024-09-21T03:35:36.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:36.841+0000 7f65a084d640 1 -- 172.21.15.38:0/3057258403 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f659c06fd10 con 0x7f659c0cd440 2024-09-21T03:35:36.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:36.842+0000 7f6592ffd640 1 -- 172.21.15.38:0/3057258403 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v46) ==== 93+0+10 (secure 0 0 0) 0x7f6584067eb0 con 0x7f659c0cd440 2024-09-21T03:35:36.844 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 1 2024-09-21T03:35:36.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:36.844+0000 7f65a084d640 1 -- 172.21.15.38:0/3057258403 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f657c078b90 msgr2=0x7f657c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:36.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:36.844+0000 7f65a084d640 1 --2- 172.21.15.38:0/3057258403 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f657c078b90 0x7f657c07b0a0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f658c033000 tx=0x7f658c004730 comp rx=0 tx=0).stop 2024-09-21T03:35:36.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:36.844+0000 7f65a084d640 1 -- 172.21.15.38:0/3057258403 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f659c0cd440 msgr2=0x7f659c15f720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:36.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:36.844+0000 7f65a084d640 1 --2- 172.21.15.38:0/3057258403 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f659c0cd440 0x7f659c15f720 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f658400e790 tx=0x7f658400ec50 comp rx=0 tx=0).stop 2024-09-21T03:35:36.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:36.844+0000 7f65a084d640 1 -- 172.21.15.38:0/3057258403 shutdown_connections 2024-09-21T03:35:36.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:36.844+0000 7f65a084d640 1 --2- 172.21.15.38:0/3057258403 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f659c0d0bb0 0x7f659c0c9960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:36.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:36.844+0000 7f65a084d640 1 --2- 172.21.15.38:0/3057258403 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f659c0cd440 0x7f659c15f720 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:36.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:36.844+0000 7f65a084d640 1 --2- 172.21.15.38:0/3057258403 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f657c078b90 0x7f657c07b0a0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:36.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:36.844+0000 7f65a084d640 1 --2- 172.21.15.38:0/3057258403 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f659c0ccc40 0x7f659c15f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:36.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:36.844+0000 7f65a084d640 1 -- 172.21.15.38:0/3057258403 >> 172.21.15.38:0/3057258403 conn(0x7f659c0b46e0 msgr2=0x7f659c0c5ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:36.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:36.844+0000 7f65a084d640 1 -- 172.21.15.38:0/3057258403 shutdown_connections 2024-09-21T03:35:36.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:36.844+0000 7f65a084d640 1 -- 172.21.15.38:0/3057258403 wait complete. 2024-09-21T03:35:36.857 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_damage.TestDamage.test_dentry_first_existing' 2024-09-21T03:35:36.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:36.971+0000 7fd56fd32640 1 -- 172.21.15.38:0/1737386277 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd56815ee30 msgr2=0x7fd5680b1b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:36.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:36.971+0000 7fd56fd32640 1 --2- 172.21.15.38:0/1737386277 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd56815ee30 0x7fd5680b1b40 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fd564005fd0 tx=0x7fd56402a540 comp rx=0 tx=0).stop 2024-09-21T03:35:36.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:36.972+0000 7fd56fd32640 1 -- 172.21.15.38:0/1737386277 shutdown_connections 2024-09-21T03:35:36.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:36.972+0000 7fd56fd32640 1 --2- 172.21.15.38:0/1737386277 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd56806e0e0 0x7fd56806e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:36.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:36.972+0000 7fd56fd32640 1 --2- 172.21.15.38:0/1737386277 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd56815ee30 0x7fd5680b1b40 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:36.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:36.972+0000 7fd56fd32640 1 -- 172.21.15.38:0/1737386277 >> 172.21.15.38:0/1737386277 conn(0x7fd568072990 msgr2=0x7fd568072da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:36.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:36.972+0000 7fd56fd32640 1 -- 172.21.15.38:0/1737386277 shutdown_connections 2024-09-21T03:35:36.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:36.972+0000 7fd56fd32640 1 -- 172.21.15.38:0/1737386277 wait complete. 2024-09-21T03:35:36.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:36.973+0000 7fd56fd32640 1 Processor -- start 2024-09-21T03:35:36.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:36.973+0000 7fd56fd32640 1 -- start start 2024-09-21T03:35:36.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:36.975+0000 7fd56fd32640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd56806e0e0 0x7fd568161620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:36.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:36.975+0000 7fd56fd32640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd5680d04c0 0x7fd5681603e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:36.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:36.975+0000 7fd56fd32640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd56815ee30 0x7fd568160920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:36.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:36.975+0000 7fd56fd32640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fd5680cd860 con 0x7fd56815ee30 2024-09-21T03:35:36.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:36.975+0000 7fd56fd32640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd5680cd6e0 con 0x7fd56806e0e0 2024-09-21T03:35:36.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:36.975+0000 7fd56fd32640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fd5680cd9e0 con 0x7fd5680d04c0 2024-09-21T03:35:36.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:36.975+0000 7fd56e2cf640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd56815ee30 0x7fd568160920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:36.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:36.975+0000 7fd56e2cf640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd56815ee30 0x7fd568160920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:39548/0 (socket says 172.21.15.38:39548) 2024-09-21T03:35:36.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:36.975+0000 7fd56e2cf640 1 -- 172.21.15.38:0/2884066847 learned_addr learned my addr 172.21.15.38:0/2884066847 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:35:36.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:36.975+0000 7fd56dace640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd56806e0e0 0x7fd568161620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:36.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:36.975+0000 7fd56d2cd640 1 --2- 172.21.15.38:0/2884066847 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd5680d04c0 0x7fd5681603e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:36.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:36.975+0000 7fd56e2cf640 1 -- 172.21.15.38:0/2884066847 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd5680d04c0 msgr2=0x7fd5681603e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:36.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:36.975+0000 7fd56e2cf640 1 --2- 172.21.15.38:0/2884066847 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd5680d04c0 0x7fd5681603e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:36.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:36.975+0000 7fd56e2cf640 1 -- 172.21.15.38:0/2884066847 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd56806e0e0 msgr2=0x7fd568161620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:36.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:36.975+0000 7fd56e2cf640 1 --2- 172.21.15.38:0/2884066847 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd56806e0e0 0x7fd568161620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:36.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:36.975+0000 7fd56e2cf640 1 -- 172.21.15.38:0/2884066847 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd568149870 con 0x7fd56815ee30 2024-09-21T03:35:36.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:36.975+0000 7fd56e2cf640 1 --2- 172.21.15.38:0/2884066847 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd56815ee30 0x7fd568160920 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fd56000c730 tx=0x7fd56000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:36.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:36.975+0000 7fd55e7fc640 1 -- 172.21.15.38:0/2884066847 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd560016020 con 0x7fd56815ee30 2024-09-21T03:35:36.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:36.976+0000 7fd55e7fc640 1 -- 172.21.15.38:0/2884066847 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd560003970 con 0x7fd56815ee30 2024-09-21T03:35:36.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:36.976+0000 7fd55e7fc640 1 -- 172.21.15.38:0/2884066847 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd560006e60 con 0x7fd56815ee30 2024-09-21T03:35:36.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:36.976+0000 7fd56fd32640 1 -- 172.21.15.38:0/2884066847 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd56815e6f0 con 0x7fd56815ee30 2024-09-21T03:35:36.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:36.976+0000 7fd56fd32640 1 -- 172.21.15.38:0/2884066847 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd56815e9d0 con 0x7fd56815ee30 2024-09-21T03:35:36.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:36.977+0000 7fd56fd32640 1 -- 172.21.15.38:0/2884066847 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5680cdf00 con 0x7fd56815ee30 2024-09-21T03:35:36.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:36.977+0000 7fd55e7fc640 1 -- 172.21.15.38:0/2884066847 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fd5600050d0 con 0x7fd56815ee30 2024-09-21T03:35:36.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:36.978+0000 7fd55e7fc640 1 --2- 172.21.15.38:0/2884066847 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd54c078bc0 0x7fd54c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:36.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:36.978+0000 7fd55e7fc640 1 -- 172.21.15.38:0/2884066847 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5421+0+0 (secure 0 0 0) 0x7fd56009a800 con 0x7fd56815ee30 2024-09-21T03:35:36.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:36.978+0000 7fd56dace640 1 --2- 172.21.15.38:0/2884066847 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd54c078bc0 0x7fd54c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:36.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:36.979+0000 7fd56dace640 1 --2- 172.21.15.38:0/2884066847 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd54c078bc0 0x7fd54c07b0d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fd564033000 tx=0x7fd564004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:36.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:36.983+0000 7fd55e7fc640 1 -- 172.21.15.38:0/2884066847 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fd560020020 con 0x7fd56815ee30 2024-09-21T03:35:37.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:37.250+0000 7fd56fd32640 1 -- 172.21.15.38:0/2884066847 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_damage.TestDamage.test_dentry_first_existing"]} v 0) -- 0x7fd568070460 con 0x7fd56815ee30 2024-09-21T03:35:38.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:38.183+0000 7fd55e7fc640 1 -- 172.21.15.38:0/2884066847 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_damage.TestDamage.test_dentry_first_existing"]}]=0 v123) ==== 144+0+0 (secure 0 0 0) 0x7fd5600627a0 con 0x7fd56815ee30 2024-09-21T03:35:38.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:38.186+0000 7fd56fd32640 1 -- 172.21.15.38:0/2884066847 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd54c078bc0 msgr2=0x7fd54c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:38.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:38.186+0000 7fd56fd32640 1 --2- 172.21.15.38:0/2884066847 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd54c078bc0 0x7fd54c07b0d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fd564033000 tx=0x7fd564004730 comp rx=0 tx=0).stop 2024-09-21T03:35:38.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:38.186+0000 7fd56fd32640 1 -- 172.21.15.38:0/2884066847 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd56815ee30 msgr2=0x7fd568160920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:38.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:38.186+0000 7fd56fd32640 1 --2- 172.21.15.38:0/2884066847 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd56815ee30 0x7fd568160920 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fd56000c730 tx=0x7fd56000cbf0 comp rx=0 tx=0).stop 2024-09-21T03:35:38.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:38.186+0000 7fd56fd32640 1 -- 172.21.15.38:0/2884066847 shutdown_connections 2024-09-21T03:35:38.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:38.186+0000 7fd56fd32640 1 --2- 172.21.15.38:0/2884066847 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd56815ee30 0x7fd568160920 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:38.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:38.186+0000 7fd56fd32640 1 --2- 172.21.15.38:0/2884066847 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd54c078bc0 0x7fd54c07b0d0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:38.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:38.186+0000 7fd56fd32640 1 --2- 172.21.15.38:0/2884066847 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd5680d04c0 0x7fd5681603e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:38.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:38.186+0000 7fd56fd32640 1 --2- 172.21.15.38:0/2884066847 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd56806e0e0 0x7fd568161620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:38.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:38.186+0000 7fd56fd32640 1 -- 172.21.15.38:0/2884066847 >> 172.21.15.38:0/2884066847 conn(0x7fd568072990 msgr2=0x7fd5680b7050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:38.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:38.187+0000 7fd56fd32640 1 -- 172.21.15.38:0/2884066847 shutdown_connections 2024-09-21T03:35:38.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:38.187+0000 7fd56fd32640 1 -- 172.21.15.38:0/2884066847 wait complete. 2024-09-21T03:35:38.201 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-21T03:35:38.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:38.324+0000 7fc33549b640 1 -- 172.21.15.38:0/2545653882 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc3300afb20 msgr2=0x7fc33015f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:38.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:38.324+0000 7fc33549b640 1 --2- 172.21.15.38:0/2545653882 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc3300afb20 0x7fc33015f130 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fc3300ce150 tx=0x7fc32402a560 comp rx=0 tx=0).stop 2024-09-21T03:35:38.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:38.324+0000 7fc33549b640 1 -- 172.21.15.38:0/2545653882 shutdown_connections 2024-09-21T03:35:38.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:38.324+0000 7fc33549b640 1 --2- 172.21.15.38:0/2545653882 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc3300afb20 0x7fc33015f130 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:38.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:38.324+0000 7fc33549b640 1 --2- 172.21.15.38:0/2545653882 >> v2:172.21.15.195:3300/0 conn(0x7fc33006def0 0x7fc3300cd650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:38.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:38.324+0000 7fc33549b640 1 -- 172.21.15.38:0/2545653882 >> 172.21.15.38:0/2545653882 conn(0x7fc3300d0c00 msgr2=0x7fc3300ccf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:38.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:38.324+0000 7fc33549b640 1 -- 172.21.15.38:0/2545653882 shutdown_connections 2024-09-21T03:35:38.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:38.324+0000 7fc33549b640 1 -- 172.21.15.38:0/2545653882 wait complete. 2024-09-21T03:35:38.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:38.326+0000 7fc33549b640 1 Processor -- start 2024-09-21T03:35:38.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:38.326+0000 7fc33549b640 1 -- start start 2024-09-21T03:35:38.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:38.327+0000 7fc33549b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc33006def0 0x7fc330158f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:38.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:38.327+0000 7fc33549b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc3300afb20 0x7fc330155030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:38.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:38.327+0000 7fc33549b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc33015f670 0x7fc33014ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:38.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:38.327+0000 7fc33549b640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fc33015c250 con 0x7fc3300afb20 2024-09-21T03:35:38.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:38.327+0000 7fc33549b640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fc3300cf7c0 con 0x7fc33006def0 2024-09-21T03:35:38.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:38.327+0000 7fc33549b640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fc33015c3d0 con 0x7fc33015f670 2024-09-21T03:35:38.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:38.327+0000 7fc32e7fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc3300afb20 0x7fc330155030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:38.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:38.327+0000 7fc32e7fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc3300afb20 0x7fc330155030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:39560/0 (socket says 172.21.15.38:39560) 2024-09-21T03:35:38.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:38.327+0000 7fc32e7fc640 1 -- 172.21.15.38:0/942382687 learned_addr learned my addr 172.21.15.38:0/942382687 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:35:38.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:38.327+0000 7fc32effd640 1 --2- 172.21.15.38:0/942382687 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc33006def0 0x7fc330158f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:38.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:38.327+0000 7fc32f7fe640 1 --2- 172.21.15.38:0/942382687 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc33015f670 0x7fc33014ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:38.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:38.328+0000 7fc32e7fc640 1 -- 172.21.15.38:0/942382687 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc33015f670 msgr2=0x7fc33014ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:38.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:38.328+0000 7fc32e7fc640 1 --2- 172.21.15.38:0/942382687 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc33015f670 0x7fc33014ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:38.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:38.328+0000 7fc32e7fc640 1 -- 172.21.15.38:0/942382687 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc33006def0 msgr2=0x7fc330158f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:38.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:38.328+0000 7fc32e7fc640 1 --2- 172.21.15.38:0/942382687 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc33006def0 0x7fc330158f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:38.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:38.328+0000 7fc32e7fc640 1 -- 172.21.15.38:0/942382687 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3301f6940 con 0x7fc3300afb20 2024-09-21T03:35:38.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:38.328+0000 7fc32effd640 1 --2- 172.21.15.38:0/942382687 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc33006def0 0x7fc330158f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:35:38.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:38.328+0000 7fc32f7fe640 1 --2- 172.21.15.38:0/942382687 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc33015f670 0x7fc33014ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:35:38.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:38.328+0000 7fc32e7fc640 1 --2- 172.21.15.38:0/942382687 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc3300afb20 0x7fc330155030 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fc324034040 tx=0x7fc324003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:38.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:38.328+0000 7fc3177fe640 1 -- 172.21.15.38:0/942382687 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc32403e030 con 0x7fc3300afb20 2024-09-21T03:35:38.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:38.328+0000 7fc3177fe640 1 -- 172.21.15.38:0/942382687 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc324003d10 con 0x7fc3300afb20 2024-09-21T03:35:38.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:38.328+0000 7fc33549b640 1 -- 172.21.15.38:0/942382687 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3301f6e40 con 0x7fc3300afb20 2024-09-21T03:35:38.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:38.328+0000 7fc3177fe640 1 -- 172.21.15.38:0/942382687 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc324035040 con 0x7fc3300afb20 2024-09-21T03:35:38.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:38.328+0000 7fc33549b640 1 -- 172.21.15.38:0/942382687 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3301f7400 con 0x7fc3300afb20 2024-09-21T03:35:38.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:38.329+0000 7fc33549b640 1 -- 172.21.15.38:0/942382687 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc33014e800 con 0x7fc3300afb20 2024-09-21T03:35:38.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:38.330+0000 7fc3177fe640 1 -- 172.21.15.38:0/942382687 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc324041030 con 0x7fc3300afb20 2024-09-21T03:35:38.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:38.331+0000 7fc3177fe640 1 --2- 172.21.15.38:0/942382687 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc31c078bc0 0x7fc31c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:38.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:38.331+0000 7fc3177fe640 1 -- 172.21.15.38:0/942382687 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5421+0+0 (secure 0 0 0) 0x7fc3240bb400 con 0x7fc3300afb20 2024-09-21T03:35:38.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:38.331+0000 7fc32effd640 1 --2- 172.21.15.38:0/942382687 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc31c078bc0 0x7fc31c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:38.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:38.331+0000 7fc32effd640 1 --2- 172.21.15.38:0/942382687 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc31c078bc0 0x7fc31c07b0d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fc320009b80 tx=0x7fc320005f90 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:38.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:38.335+0000 7fc3177fe640 1 -- 172.21.15.38:0/942382687 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fc3240838c0 con 0x7fc3300afb20 2024-09-21T03:35:38.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:38.524+0000 7fc33549b640 1 -- 172.21.15.38:0/942382687 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7fc3301f6bd0 con 0x7fc3300afb20 2024-09-21T03:35:38.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:38.524+0000 7fc3177fe640 1 -- 172.21.15.38:0/942382687 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v33) ==== 59+0+2294 (secure 0 0 0) 0x7fc3240831c0 con 0x7fc3300afb20 2024-09-21T03:35:38.526 INFO:teuthology.orchestra.run.smithi038.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-21T03:35:38.526 INFO:teuthology.orchestra.run.smithi038.stdout:mon advanced mon_op_complaint_time 120 2024-09-21T03:35:38.526 INFO:teuthology.orchestra.run.smithi038.stdout:mgr advanced client_mount_timeout 30 2024-09-21T03:35:38.527 INFO:teuthology.orchestra.run.smithi038.stdout:mgr advanced debug_client 20/20 2024-09-21T03:35:38.527 INFO:teuthology.orchestra.run.smithi038.stdout:mgr advanced debug_mgr 20/20 2024-09-21T03:35:38.527 INFO:teuthology.orchestra.run.smithi038.stdout:mgr advanced debug_ms 1/1 2024-09-21T03:35:38.527 INFO:teuthology.orchestra.run.smithi038.stdout:mgr dev mon_warn_on_pool_no_app false 2024-09-21T03:35:38.527 INFO:teuthology.orchestra.run.smithi038.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-21T03:35:38.527 INFO:teuthology.orchestra.run.smithi038.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 3844.962174 2024-09-21T03:35:38.527 INFO:teuthology.orchestra.run.smithi038.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 4916.015278 2024-09-21T03:35:38.527 INFO:teuthology.orchestra.run.smithi038.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 4165.407418 2024-09-21T03:35:38.527 INFO:teuthology.orchestra.run.smithi038.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 3760.481604 2024-09-21T03:35:38.527 INFO:teuthology.orchestra.run.smithi038.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 6052.882314 2024-09-21T03:35:38.527 INFO:teuthology.orchestra.run.smithi038.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 8967.576248 2024-09-21T03:35:38.527 INFO:teuthology.orchestra.run.smithi038.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 6129.316416 2024-09-21T03:35:38.527 INFO:teuthology.orchestra.run.smithi038.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 8751.950197 2024-09-21T03:35:38.527 INFO:teuthology.orchestra.run.smithi038.stdout:mds advanced debug_mds 20/20 2024-09-21T03:35:38.528 INFO:teuthology.orchestra.run.smithi038.stdout:mds advanced debug_mds_balancer 20/20 2024-09-21T03:35:38.528 INFO:teuthology.orchestra.run.smithi038.stdout:mds advanced debug_ms 2/2 2024-09-21T03:35:38.528 INFO:teuthology.orchestra.run.smithi038.stdout:mds dev mds_debug_frag true 2024-09-21T03:35:38.528 INFO:teuthology.orchestra.run.smithi038.stdout:mds dev mds_debug_scatterstat true 2024-09-21T03:35:38.528 INFO:teuthology.orchestra.run.smithi038.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-21T03:35:38.528 INFO:teuthology.orchestra.run.smithi038.stdout:mds dev mds_verify_scatter true 2024-09-21T03:35:38.528 INFO:teuthology.orchestra.run.smithi038.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-21T03:35:38.528 INFO:teuthology.orchestra.run.smithi038.stdout:mds advanced rados_mon_op_timeout 900 2024-09-21T03:35:38.528 INFO:teuthology.orchestra.run.smithi038.stdout:mds advanced rados_osd_op_timeout 900 2024-09-21T03:35:38.528 INFO:teuthology.orchestra.run.smithi038.stdout:client advanced client_mount_timeout 600 2024-09-21T03:35:38.528 INFO:teuthology.orchestra.run.smithi038.stdout:client advanced debug_client 20/20 2024-09-21T03:35:38.528 INFO:teuthology.orchestra.run.smithi038.stdout:client advanced debug_ms 1/1 2024-09-21T03:35:38.528 INFO:teuthology.orchestra.run.smithi038.stdout:client advanced rados_mon_op_timeout 900 2024-09-21T03:35:38.528 INFO:teuthology.orchestra.run.smithi038.stdout:client advanced rados_osd_op_timeout 900 2024-09-21T03:35:38.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:38.526+0000 7fc33549b640 1 -- 172.21.15.38:0/942382687 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc31c078bc0 msgr2=0x7fc31c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:38.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:38.526+0000 7fc33549b640 1 --2- 172.21.15.38:0/942382687 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc31c078bc0 0x7fc31c07b0d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fc320009b80 tx=0x7fc320005f90 comp rx=0 tx=0).stop 2024-09-21T03:35:38.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:38.526+0000 7fc33549b640 1 -- 172.21.15.38:0/942382687 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc3300afb20 msgr2=0x7fc330155030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:38.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:38.526+0000 7fc33549b640 1 --2- 172.21.15.38:0/942382687 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc3300afb20 0x7fc330155030 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fc324034040 tx=0x7fc324003820 comp rx=0 tx=0).stop 2024-09-21T03:35:38.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:38.527+0000 7fc33549b640 1 -- 172.21.15.38:0/942382687 shutdown_connections 2024-09-21T03:35:38.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:38.527+0000 7fc33549b640 1 --2- 172.21.15.38:0/942382687 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc31c078bc0 0x7fc31c07b0d0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:38.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:38.527+0000 7fc33549b640 1 --2- 172.21.15.38:0/942382687 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc33015f670 0x7fc33014ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:38.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:38.527+0000 7fc33549b640 1 --2- 172.21.15.38:0/942382687 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc3300afb20 0x7fc330155030 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:38.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:38.527+0000 7fc33549b640 1 --2- 172.21.15.38:0/942382687 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc33006def0 0x7fc330158f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:38.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:38.527+0000 7fc33549b640 1 -- 172.21.15.38:0/942382687 >> 172.21.15.38:0/942382687 conn(0x7fc3300d0c00 msgr2=0x7fc330151df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:38.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:38.527+0000 7fc33549b640 1 -- 172.21.15.38:0/942382687 shutdown_connections 2024-09-21T03:35:38.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:38.527+0000 7fc33549b640 1 -- 172.21.15.38:0/942382687 wait complete. 2024-09-21T03:35:38.540 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-21T03:35:38.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:38.657+0000 7f5b197ba640 1 -- 172.21.15.38:0/3122108762 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5b1406def0 msgr2=0x7f5b1406f410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:38.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:38.657+0000 7f5b197ba640 1 --2- 172.21.15.38:0/3122108762 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5b1406def0 0x7f5b1406f410 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f5b1406e6e0 tx=0x7f5b0402aa00 comp rx=0 tx=0).stop 2024-09-21T03:35:38.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:38.657+0000 7f5b197ba640 1 -- 172.21.15.38:0/3122108762 shutdown_connections 2024-09-21T03:35:38.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:38.657+0000 7f5b197ba640 1 --2- 172.21.15.38:0/3122108762 >> v2:172.21.15.195:3300/0 conn(0x7f5b1415bfc0 0x7f5b1415f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:38.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:38.657+0000 7f5b197ba640 1 --2- 172.21.15.38:0/3122108762 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5b140b5260 0x7f5b1415ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:38.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:38.657+0000 7f5b197ba640 1 --2- 172.21.15.38:0/3122108762 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5b1406def0 0x7f5b1406f410 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:38.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:38.657+0000 7f5b197ba640 1 -- 172.21.15.38:0/3122108762 >> 172.21.15.38:0/3122108762 conn(0x7f5b140cdca0 msgr2=0x7f5b140ce0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:38.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:38.657+0000 7f5b197ba640 1 -- 172.21.15.38:0/3122108762 shutdown_connections 2024-09-21T03:35:38.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:38.657+0000 7f5b197ba640 1 -- 172.21.15.38:0/3122108762 wait complete. 2024-09-21T03:35:38.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:38.659+0000 7f5b197ba640 1 Processor -- start 2024-09-21T03:35:38.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:38.659+0000 7f5b197ba640 1 -- start start 2024-09-21T03:35:38.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:38.661+0000 7f5b197ba640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5b1406def0 0x7f5b140b16f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:38.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:38.661+0000 7f5b197ba640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5b140b5260 0x7f5b141f4110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:38.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:38.661+0000 7f5b197ba640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5b1415bfc0 0x7f5b140d0150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:38.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:38.661+0000 7f5b197ba640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f5b1415fcf0 con 0x7f5b140b5260 2024-09-21T03:35:38.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:38.661+0000 7f5b197ba640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f5b1415fb70 con 0x7f5b1415bfc0 2024-09-21T03:35:38.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:38.661+0000 7f5b197ba640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f5b1415fe70 con 0x7f5b1406def0 2024-09-21T03:35:38.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:38.661+0000 7f5b127fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5b140b5260 0x7f5b141f4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:38.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:38.661+0000 7f5b127fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5b140b5260 0x7f5b141f4110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:39570/0 (socket says 172.21.15.38:39570) 2024-09-21T03:35:38.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:38.661+0000 7f5b127fc640 1 -- 172.21.15.38:0/3709322245 learned_addr learned my addr 172.21.15.38:0/3709322245 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:35:38.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:38.661+0000 7f5b12ffd640 1 --2- 172.21.15.38:0/3709322245 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5b1406def0 0x7f5b140b16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:38.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:38.661+0000 7f5b137fe640 1 --2- 172.21.15.38:0/3709322245 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5b1415bfc0 0x7f5b140d0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:38.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:38.661+0000 7f5b127fc640 1 -- 172.21.15.38:0/3709322245 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5b1406def0 msgr2=0x7f5b140b16f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:38.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:38.661+0000 7f5b127fc640 1 --2- 172.21.15.38:0/3709322245 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5b1406def0 0x7f5b140b16f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:38.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:38.661+0000 7f5b127fc640 1 -- 172.21.15.38:0/3709322245 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5b1415bfc0 msgr2=0x7f5b140d0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:38.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:38.661+0000 7f5b127fc640 1 --2- 172.21.15.38:0/3709322245 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5b1415bfc0 0x7f5b140d0150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:38.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:38.661+0000 7f5b127fc640 1 -- 172.21.15.38:0/3709322245 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b141f3670 con 0x7f5b140b5260 2024-09-21T03:35:38.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:38.661+0000 7f5b127fc640 1 --2- 172.21.15.38:0/3709322245 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5b140b5260 0x7f5b141f4110 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f5afc00c970 tx=0x7f5afc00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:38.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:38.661+0000 7f5b0b7fe640 1 -- 172.21.15.38:0/3709322245 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5afc007bb0 con 0x7f5b140b5260 2024-09-21T03:35:38.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:38.661+0000 7f5b0b7fe640 1 -- 172.21.15.38:0/3709322245 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5afc005020 con 0x7f5b140b5260 2024-09-21T03:35:38.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:38.661+0000 7f5b0b7fe640 1 -- 172.21.15.38:0/3709322245 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5afc007e60 con 0x7f5b140b5260 2024-09-21T03:35:38.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:38.661+0000 7f5b197ba640 1 -- 172.21.15.38:0/3709322245 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b14154b40 con 0x7f5b140b5260 2024-09-21T03:35:38.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:38.662+0000 7f5b197ba640 1 -- 172.21.15.38:0/3709322245 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b141f3bf0 con 0x7f5b140b5260 2024-09-21T03:35:38.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:38.662+0000 7f5b197ba640 1 -- 172.21.15.38:0/3709322245 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b1415ba80 con 0x7f5b140b5260 2024-09-21T03:35:38.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:38.663+0000 7f5b0b7fe640 1 -- 172.21.15.38:0/3709322245 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f5afc005470 con 0x7f5b140b5260 2024-09-21T03:35:38.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:38.664+0000 7f5b0b7fe640 1 --2- 172.21.15.38:0/3709322245 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5af4078b90 0x7f5af407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:38.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:38.664+0000 7f5b0b7fe640 1 -- 172.21.15.38:0/3709322245 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5421+0+0 (secure 0 0 0) 0x7f5afc09b050 con 0x7f5b140b5260 2024-09-21T03:35:38.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:38.664+0000 7f5b12ffd640 1 --2- 172.21.15.38:0/3709322245 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5af4078b90 0x7f5af407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:38.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:38.665+0000 7f5b12ffd640 1 --2- 172.21.15.38:0/3709322245 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5af4078b90 0x7f5af407b0a0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f5b04035000 tx=0x7f5b0403b040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:38.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:38.668+0000 7f5b0b7fe640 1 -- 172.21.15.38:0/3709322245 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f5afc014030 con 0x7f5b140b5260 2024-09-21T03:35:38.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:38.859+0000 7f5b197ba640 1 -- 172.21.15.38:0/3709322245 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f5b1406cf60 con 0x7f5b140b5260 2024-09-21T03:35:38.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:38.863+0000 7f5b0b7fe640 1 -- 172.21.15.38:0/3709322245 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v34)=0 v34) ==== 122+0+0 (secure 0 0 0) 0x7f5afc062f60 con 0x7f5b140b5260 2024-09-21T03:35:38.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:38.865+0000 7f5b197ba640 1 -- 172.21.15.38:0/3709322245 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5af4078b90 msgr2=0x7f5af407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:38.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:38.865+0000 7f5b197ba640 1 --2- 172.21.15.38:0/3709322245 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5af4078b90 0x7f5af407b0a0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f5b04035000 tx=0x7f5b0403b040 comp rx=0 tx=0).stop 2024-09-21T03:35:38.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:38.865+0000 7f5b197ba640 1 -- 172.21.15.38:0/3709322245 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5b140b5260 msgr2=0x7f5b141f4110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:38.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:38.865+0000 7f5b197ba640 1 --2- 172.21.15.38:0/3709322245 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5b140b5260 0x7f5b141f4110 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f5afc00c970 tx=0x7f5afc00ce30 comp rx=0 tx=0).stop 2024-09-21T03:35:38.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:38.865+0000 7f5b197ba640 1 -- 172.21.15.38:0/3709322245 shutdown_connections 2024-09-21T03:35:38.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:38.865+0000 7f5b197ba640 1 --2- 172.21.15.38:0/3709322245 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5b1415bfc0 0x7f5b140d0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:38.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:38.865+0000 7f5b197ba640 1 --2- 172.21.15.38:0/3709322245 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5b140b5260 0x7f5b141f4110 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:38.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:38.865+0000 7f5b197ba640 1 --2- 172.21.15.38:0/3709322245 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5af4078b90 0x7f5af407b0a0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:38.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:38.865+0000 7f5b197ba640 1 --2- 172.21.15.38:0/3709322245 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5b1406def0 0x7f5b140b16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:38.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:38.865+0000 7f5b197ba640 1 -- 172.21.15.38:0/3709322245 >> 172.21.15.38:0/3709322245 conn(0x7f5b140cdca0 msgr2=0x7f5b1406d7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:38.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:38.865+0000 7f5b197ba640 1 -- 172.21.15.38:0/3709322245 shutdown_connections 2024-09-21T03:35:38.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:38.865+0000 7f5b197ba640 1 -- 172.21.15.38:0/3709322245 wait complete. 2024-09-21T03:35:38.879 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-21T03:35:38.879 DEBUG:teuthology.orchestra.run.smithi038:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T03:35:38.934 DEBUG:teuthology.orchestra.run.smithi195:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T03:35:38.976 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-21T03:35:38.977 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-21T03:35:38.977 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:35:38.993 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056795,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2354 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T03:35:38.993 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-09-21T03:35:38.994 DEBUG:teuthology.orchestra.run.smithi195:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:35:39.035 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056014,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497444k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=8978 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248720k,nr_inodes=812180,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497444k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T03:35:39.035 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-21T03:35:39.035 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:35:39.051 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056795,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2354 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T03:35:39.052 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-09-21T03:35:39.052 DEBUG:teuthology.orchestra.run.smithi195:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:35:39.093 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056014,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497444k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=8978 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248720k,nr_inodes=812180,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497444k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T03:35:39.093 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-21T03:35:39.094 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:35:39.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:39.209+0000 7f73775c6640 1 -- 172.21.15.38:0/3431548195 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f73700b2e40 msgr2=0x7f73700b3250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:39.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:39.209+0000 7f73775c6640 1 --2- 172.21.15.38:0/3431548195 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f73700b2e40 0x7f73700b3250 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f7368005fd0 tx=0x7f736802a8b0 comp rx=0 tx=0).stop 2024-09-21T03:35:39.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:39.209+0000 7f73775c6640 1 -- 172.21.15.38:0/3431548195 shutdown_connections 2024-09-21T03:35:39.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:39.209+0000 7f73775c6640 1 --2- 172.21.15.38:0/3431548195 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f73700b2e40 0x7f73700b3250 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:39.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:39.209+0000 7f73775c6640 1 --2- 172.21.15.38:0/3431548195 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7370162280 0x7f7370162690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:39.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:39.209+0000 7f73775c6640 1 -- 172.21.15.38:0/3431548195 >> 172.21.15.38:0/3431548195 conn(0x7f73700b6b20 msgr2=0x7f73700d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:39.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:39.209+0000 7f73775c6640 1 -- 172.21.15.38:0/3431548195 shutdown_connections 2024-09-21T03:35:39.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:39.209+0000 7f73775c6640 1 -- 172.21.15.38:0/3431548195 wait complete. 2024-09-21T03:35:39.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:39.211+0000 7f73775c6640 1 Processor -- start 2024-09-21T03:35:39.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:39.211+0000 7f73775c6640 1 -- start start 2024-09-21T03:35:39.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:39.212+0000 7f73775c6640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f73700b2e40 0x7f73700cee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:39.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:39.212+0000 7f73775c6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7370162280 0x7f73700b1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:39.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:39.212+0000 7f73775c6640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f73701f9580 0x7f73700b86f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:39.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:39.212+0000 7f73775c6640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f737015e4b0 con 0x7f7370162280 2024-09-21T03:35:39.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:39.212+0000 7f73775c6640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f73700b39a0 con 0x7f73701f9580 2024-09-21T03:35:39.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:39.212+0000 7f73775c6640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f737015e630 con 0x7f73700b2e40 2024-09-21T03:35:39.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:39.212+0000 7f7374b61640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7370162280 0x7f73700b1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:39.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:39.212+0000 7f7374b61640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7370162280 0x7f73700b1ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:39576/0 (socket says 172.21.15.38:39576) 2024-09-21T03:35:39.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:39.212+0000 7f7374b61640 1 -- 172.21.15.38:0/475715851 learned_addr learned my addr 172.21.15.38:0/475715851 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:35:39.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:39.212+0000 7f7375b63640 1 --2- 172.21.15.38:0/475715851 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f73701f9580 0x7f73700b86f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:39.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:39.212+0000 7f7375362640 1 --2- 172.21.15.38:0/475715851 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f73700b2e40 0x7f73700cee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:39.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:39.212+0000 7f7374b61640 1 -- 172.21.15.38:0/475715851 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f73700b2e40 msgr2=0x7f73700cee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:39.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:39.212+0000 7f7374b61640 1 --2- 172.21.15.38:0/475715851 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f73700b2e40 0x7f73700cee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:39.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:39.212+0000 7f7374b61640 1 -- 172.21.15.38:0/475715851 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f73701f9580 msgr2=0x7f73700b86f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:39.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:39.212+0000 7f7374b61640 1 --2- 172.21.15.38:0/475715851 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f73701f9580 0x7f73700b86f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:39.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:39.212+0000 7f7374b61640 1 -- 172.21.15.38:0/475715851 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f737015ac60 con 0x7f7370162280 2024-09-21T03:35:39.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:39.212+0000 7f7375362640 1 --2- 172.21.15.38:0/475715851 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f73700b2e40 0x7f73700cee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:35:39.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:39.213+0000 7f7374b61640 1 --2- 172.21.15.38:0/475715851 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7370162280 0x7f73700b1ed0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f736000d700 tx=0x7f736000dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:39.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:39.213+0000 7f7365ffb640 1 -- 172.21.15.38:0/475715851 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7360017070 con 0x7f7370162280 2024-09-21T03:35:39.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:39.213+0000 7f7365ffb640 1 -- 172.21.15.38:0/475715851 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7360004310 con 0x7f7370162280 2024-09-21T03:35:39.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:39.213+0000 7f7365ffb640 1 -- 172.21.15.38:0/475715851 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7360009a80 con 0x7f7370162280 2024-09-21T03:35:39.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:39.213+0000 7f73775c6640 1 -- 172.21.15.38:0/475715851 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73700cf3b0 con 0x7f7370162280 2024-09-21T03:35:39.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:39.213+0000 7f73775c6640 1 -- 172.21.15.38:0/475715851 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73701f8aa0 con 0x7f7370162280 2024-09-21T03:35:39.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:39.214+0000 7f73775c6640 1 -- 172.21.15.38:0/475715851 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73701f8890 con 0x7f7370162280 2024-09-21T03:35:39.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:39.215+0000 7f7365ffb640 1 -- 172.21.15.38:0/475715851 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f7360009c20 con 0x7f7370162280 2024-09-21T03:35:39.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:39.215+0000 7f7365ffb640 1 --2- 172.21.15.38:0/475715851 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7354078bc0 0x7f735407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:39.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:39.215+0000 7f7365ffb640 1 -- 172.21.15.38:0/475715851 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5421+0+0 (secure 0 0 0) 0x7f736009ac50 con 0x7f7370162280 2024-09-21T03:35:39.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:39.215+0000 7f7375362640 1 --2- 172.21.15.38:0/475715851 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7354078bc0 0x7f735407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:39.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:39.216+0000 7f7375362640 1 --2- 172.21.15.38:0/475715851 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7354078bc0 0x7f735407b0d0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f736c0013f0 tx=0x7f736c009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:39.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:39.220+0000 7f7365ffb640 1 -- 172.21.15.38:0/475715851 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f73600a0020 con 0x7f7370162280 2024-09-21T03:35:39.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:39.445+0000 7f73775c6640 1 -- 172.21.15.38:0/475715851 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f73700b2840 con 0x7f7370162280 2024-09-21T03:35:39.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:39.446+0000 7f7365ffb640 1 -- 172.21.15.38:0/475715851 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+3917 (secure 0 0 0) 0x7f7360062b60 con 0x7f7370162280 2024-09-21T03:35:39.448 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:35:39.449 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":25,"btime":"2024-09-21T03:35:32:875986+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4757,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/1905955399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":1905955399},{"type":"v1","addr":"172.21.15.195:6836","nonce":1905955399}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4936,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/2989374027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":2989374027},{"type":"v1","addr":"172.21.15.195:6837","nonce":2989374027}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4942,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/3531731501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6833","nonce":3531731501},{"type":"v1","addr":"172.21.15.38:6835","nonce":3531731501}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5481,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6834/3068296067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":3068296067},{"type":"v1","addr":"172.21.15.38:6834","nonce":3068296067}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":25}],"filesystems":[]} 2024-09-21T03:35:39.449 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 25 2024-09-21T03:35:39.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:39.448+0000 7f73775c6640 1 -- 172.21.15.38:0/475715851 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7354078bc0 msgr2=0x7f735407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:39.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:39.448+0000 7f73775c6640 1 --2- 172.21.15.38:0/475715851 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7354078bc0 0x7f735407b0d0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f736c0013f0 tx=0x7f736c009040 comp rx=0 tx=0).stop 2024-09-21T03:35:39.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:39.448+0000 7f73775c6640 1 -- 172.21.15.38:0/475715851 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7370162280 msgr2=0x7f73700b1ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:39.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:39.448+0000 7f73775c6640 1 --2- 172.21.15.38:0/475715851 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7370162280 0x7f73700b1ed0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f736000d700 tx=0x7f736000dbc0 comp rx=0 tx=0).stop 2024-09-21T03:35:39.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:39.448+0000 7f73775c6640 1 -- 172.21.15.38:0/475715851 shutdown_connections 2024-09-21T03:35:39.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:39.448+0000 7f73775c6640 1 --2- 172.21.15.38:0/475715851 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f73701f9580 0x7f73700b86f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:39.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:39.448+0000 7f73775c6640 1 --2- 172.21.15.38:0/475715851 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7370162280 0x7f73700b1ed0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:39.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:39.448+0000 7f73775c6640 1 --2- 172.21.15.38:0/475715851 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7354078bc0 0x7f735407b0d0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:39.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:39.448+0000 7f73775c6640 1 --2- 172.21.15.38:0/475715851 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f73700b2e40 0x7f73700cee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:39.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:39.448+0000 7f73775c6640 1 -- 172.21.15.38:0/475715851 >> 172.21.15.38:0/475715851 conn(0x7f73700b6b20 msgr2=0x7f737015d900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:39.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:39.448+0000 7f73775c6640 1 -- 172.21.15.38:0/475715851 shutdown_connections 2024-09-21T03:35:39.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:39.448+0000 7f73775c6640 1 -- 172.21.15.38:0/475715851 wait complete. 2024-09-21T03:35:39.463 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-21T03:35:39.463 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-21T03:35:39.463 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-21T03:35:39.463 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T03:35:39.463 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-21T03:35:39.463 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-21T03:35:39.464 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-21T03:35:39.464 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T03:35:39.464 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-21T03:35:39.464 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-21T03:35:39.464 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-21T03:35:39.464 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T03:35:39.464 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-21T03:35:39.465 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-21T03:35:39.465 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-21T03:35:39.465 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T03:35:39.473 INFO:tasks.ceph.mds.a:Stopped 2024-09-21T03:35:39.473 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-21T03:35:39.476 INFO:tasks.ceph.mds.b:Stopped 2024-09-21T03:35:39.476 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-21T03:35:39.477 INFO:tasks.ceph.mds.d:Stopped 2024-09-21T03:35:39.477 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-21T03:35:39.478 INFO:tasks.ceph.mds.c:Stopped 2024-09-21T03:35:39.478 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-21T03:35:39.519 INFO:tasks.ceph.mds.b:Started 2024-09-21T03:35:39.519 DEBUG:teuthology.parallel:result is None 2024-09-21T03:35:39.520 INFO:tasks.ceph.mds.d:Started 2024-09-21T03:35:39.520 DEBUG:teuthology.parallel:result is None 2024-09-21T03:35:39.521 INFO:tasks.ceph.mds.a:Started 2024-09-21T03:35:39.521 DEBUG:teuthology.parallel:result is None 2024-09-21T03:35:39.522 INFO:tasks.ceph.mds.c:Started 2024-09-21T03:35:39.522 DEBUG:teuthology.parallel:result is None 2024-09-21T03:35:39.522 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-21T03:35:39.611 INFO:tasks.ceph.mds.c.smithi038.stdout:starting mds.c at 2024-09-21T03:35:39.611 INFO:tasks.ceph.mds.a.smithi038.stdout:starting mds.a at 2024-09-21T03:35:39.612 INFO:tasks.ceph.mds.b.smithi195.stdout:starting mds.b at 2024-09-21T03:35:39.612 INFO:tasks.ceph.mds.d.smithi195.stdout:starting mds.d at 2024-09-21T03:35:39.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:39.643+0000 7effe337b640 1 -- 172.21.15.38:0/3860912897 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7effdc0ae970 msgr2=0x7effdc0b5b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:39.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:39.643+0000 7effe337b640 1 --2- 172.21.15.38:0/3860912897 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7effdc0ae970 0x7effdc0b5b20 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7effd8005fd0 tx=0x7effd802a5d0 comp rx=0 tx=0).stop 2024-09-21T03:35:39.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:39.643+0000 7effe337b640 1 -- 172.21.15.38:0/3860912897 shutdown_connections 2024-09-21T03:35:39.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:39.643+0000 7effe337b640 1 --2- 172.21.15.38:0/3860912897 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7effdc0ae970 0x7effdc0b5b20 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:39.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:39.643+0000 7effe337b640 1 --2- 172.21.15.38:0/3860912897 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7effdc15a300 0x7effdc06f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:39.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:39.643+0000 7effe337b640 1 -- 172.21.15.38:0/3860912897 >> 172.21.15.38:0/3860912897 conn(0x7effdc0cdca0 msgr2=0x7effdc0ce0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:39.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:39.643+0000 7effe337b640 1 -- 172.21.15.38:0/3860912897 shutdown_connections 2024-09-21T03:35:39.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:39.643+0000 7effe337b640 1 -- 172.21.15.38:0/3860912897 wait complete. 2024-09-21T03:35:39.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:39.645+0000 7effe337b640 1 Processor -- start 2024-09-21T03:35:39.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:39.645+0000 7effe337b640 1 -- start start 2024-09-21T03:35:39.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:39.646+0000 7effe337b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7effdc0ae970 0x7effdc14c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:39.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:39.646+0000 7effe337b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7effdc0b2ca0 0x7effdc0c63d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:39.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:39.646+0000 7effe337b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7effdc15a300 0x7effdc0c6910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:39.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:39.646+0000 7effe337b640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7effdc160530 con 0x7effdc15a300 2024-09-21T03:35:39.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:39.646+0000 7effe337b640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7effdc1603b0 con 0x7effdc0ae970 2024-09-21T03:35:39.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:39.646+0000 7effe337b640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7effdc1606b0 con 0x7effdc0b2ca0 2024-09-21T03:35:39.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:39.646+0000 7effe1918640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7effdc15a300 0x7effdc0c6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:39.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:39.647+0000 7effe1918640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7effdc15a300 0x7effdc0c6910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:39630/0 (socket says 172.21.15.38:39630) 2024-09-21T03:35:39.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:39.647+0000 7effe1918640 1 -- 172.21.15.38:0/3038601289 learned_addr learned my addr 172.21.15.38:0/3038601289 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:35:39.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:39.647+0000 7effe0916640 1 --2- 172.21.15.38:0/3038601289 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7effdc0b2ca0 0x7effdc0c63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:39.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:39.647+0000 7effe1117640 1 --2- 172.21.15.38:0/3038601289 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7effdc0ae970 0x7effdc14c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:39.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:39.647+0000 7effe1918640 1 -- 172.21.15.38:0/3038601289 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7effdc0b2ca0 msgr2=0x7effdc0c63d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:39.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:39.647+0000 7effe1918640 1 --2- 172.21.15.38:0/3038601289 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7effdc0b2ca0 0x7effdc0c63d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:39.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:39.647+0000 7effe1918640 1 -- 172.21.15.38:0/3038601289 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7effdc0ae970 msgr2=0x7effdc14c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:39.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:39.647+0000 7effe1918640 1 --2- 172.21.15.38:0/3038601289 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7effdc0ae970 0x7effdc14c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:39.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:39.647+0000 7effe1918640 1 -- 172.21.15.38:0/3038601289 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effdc09c750 con 0x7effdc15a300 2024-09-21T03:35:39.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:39.647+0000 7effe0916640 1 --2- 172.21.15.38:0/3038601289 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7effdc0b2ca0 0x7effdc0c63d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:35:39.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:39.647+0000 7effe1918640 1 --2- 172.21.15.38:0/3038601289 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7effdc15a300 0x7effdc0c6910 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7effd400c700 tx=0x7effd400cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:39.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:39.647+0000 7effcdffb640 1 -- 172.21.15.38:0/3038601289 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effd4003b70 con 0x7effdc15a300 2024-09-21T03:35:39.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:39.647+0000 7effcdffb640 1 -- 172.21.15.38:0/3038601289 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7effd4003d10 con 0x7effdc15a300 2024-09-21T03:35:39.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:39.647+0000 7effe337b640 1 -- 172.21.15.38:0/3038601289 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effdc1f3af0 con 0x7effdc15a300 2024-09-21T03:35:39.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:39.647+0000 7effcdffb640 1 -- 172.21.15.38:0/3038601289 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effd400aab0 con 0x7effdc15a300 2024-09-21T03:35:39.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:39.648+0000 7effe337b640 1 -- 172.21.15.38:0/3038601289 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effdc1f40b0 con 0x7effdc15a300 2024-09-21T03:35:39.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:39.649+0000 7effe337b640 1 -- 172.21.15.38:0/3038601289 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7effdc160c40 con 0x7effdc15a300 2024-09-21T03:35:39.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:39.649+0000 7effcdffb640 1 -- 172.21.15.38:0/3038601289 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7effd400ac50 con 0x7effdc15a300 2024-09-21T03:35:39.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:39.650+0000 7effcdffb640 1 --2- 172.21.15.38:0/3038601289 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7effc0078b90 0x7effc007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:39.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:39.650+0000 7effcdffb640 1 -- 172.21.15.38:0/3038601289 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5421+0+0 (secure 0 0 0) 0x7effd409ab40 con 0x7effdc15a300 2024-09-21T03:35:39.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:39.650+0000 7effe1117640 1 --2- 172.21.15.38:0/3038601289 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7effc0078b90 0x7effc007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:39.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:39.650+0000 7effe1117640 1 --2- 172.21.15.38:0/3038601289 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7effc0078b90 0x7effc007b0a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7effd000bb00 tx=0x7effd0005e30 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:39.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:39.654+0000 7effcdffb640 1 -- 172.21.15.38:0/3038601289 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7effd4063000 con 0x7effdc15a300 2024-09-21T03:35:39.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:39.856+0000 7effe337b640 1 -- 172.21.15.38:0/3038601289 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7effdc0c4ad0 con 0x7effdc15a300 2024-09-21T03:35:40.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:40.874+0000 7effcdffb640 1 -- 172.21.15.38:0/3038601289 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v51) ==== 97+0+0 (secure 0 0 0) 0x7effd4062900 con 0x7effdc15a300 2024-09-21T03:35:40.878 INFO:teuthology.orchestra.run.smithi038.stderr: removed all blocklist entries 2024-09-21T03:35:40.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:40.878+0000 7effe337b640 1 -- 172.21.15.38:0/3038601289 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7effc0078b90 msgr2=0x7effc007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:40.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:40.878+0000 7effe337b640 1 --2- 172.21.15.38:0/3038601289 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7effc0078b90 0x7effc007b0a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7effd000bb00 tx=0x7effd0005e30 comp rx=0 tx=0).stop 2024-09-21T03:35:40.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:40.878+0000 7effe337b640 1 -- 172.21.15.38:0/3038601289 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7effdc15a300 msgr2=0x7effdc0c6910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:40.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:40.878+0000 7effe337b640 1 --2- 172.21.15.38:0/3038601289 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7effdc15a300 0x7effdc0c6910 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7effd400c700 tx=0x7effd400cbc0 comp rx=0 tx=0).stop 2024-09-21T03:35:40.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:40.878+0000 7effe337b640 1 -- 172.21.15.38:0/3038601289 shutdown_connections 2024-09-21T03:35:40.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:40.878+0000 7effe337b640 1 --2- 172.21.15.38:0/3038601289 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7effdc15a300 0x7effdc0c6910 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:40.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:40.878+0000 7effe337b640 1 --2- 172.21.15.38:0/3038601289 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7effc0078b90 0x7effc007b0a0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:40.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:40.878+0000 7effe337b640 1 --2- 172.21.15.38:0/3038601289 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7effdc0b2ca0 0x7effdc0c63d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:40.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:40.878+0000 7effe337b640 1 --2- 172.21.15.38:0/3038601289 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7effdc0ae970 0x7effdc14c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:40.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:40.878+0000 7effe337b640 1 -- 172.21.15.38:0/3038601289 >> 172.21.15.38:0/3038601289 conn(0x7effdc0cdca0 msgr2=0x7effdc0d1130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:40.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:40.878+0000 7effe337b640 1 -- 172.21.15.38:0/3038601289 shutdown_connections 2024-09-21T03:35:40.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:40.878+0000 7effe337b640 1 -- 172.21.15.38:0/3038601289 wait complete. 2024-09-21T03:35:40.895 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-21T03:35:41.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:41.016+0000 7f0261e7e640 1 -- 172.21.15.38:0/34104282 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0244004d90 msgr2=0x7f0244005220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:41.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:41.016+0000 7f0261e7e640 1 --2- 172.21.15.38:0/34104282 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0244004d90 0x7f0244005220 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f0254001f30 tx=0x7f0244005830 comp rx=0 tx=0).stop 2024-09-21T03:35:41.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:41.016+0000 7f025bfff640 1 -- 172.21.15.38:0/34104282 reap_dead start 2024-09-21T03:35:41.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:41.016+0000 7f0261e7e640 1 -- 172.21.15.38:0/34104282 shutdown_connections 2024-09-21T03:35:41.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:41.016+0000 7f0261e7e640 1 -- 172.21.15.38:0/34104282 >> 172.21.15.38:0/34104282 conn(0x7f025c0cef40 msgr2=0x7f025c0cf350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:41.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:41.016+0000 7f0261e7e640 1 -- 172.21.15.38:0/34104282 shutdown_connections 2024-09-21T03:35:41.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:41.016+0000 7f0261e7e640 1 -- 172.21.15.38:0/34104282 wait complete. 2024-09-21T03:35:41.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:41.018+0000 7f0261e7e640 1 Processor -- start 2024-09-21T03:35:41.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:41.018+0000 7f0261e7e640 1 -- start start 2024-09-21T03:35:41.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:41.019+0000 7f0261e7e640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f025c14f990 0x7f025c156a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:41.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:41.019+0000 7f0261e7e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f025c14c360 0x7f025c14c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:41.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:41.019+0000 7f0261e7e640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f025c1f6b70 0x7f025c1f6f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:41.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:41.019+0000 7f0261e7e640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f025c0d1b50 con 0x7f025c14c360 2024-09-21T03:35:41.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:41.019+0000 7f0261e7e640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f025c0d19d0 con 0x7f025c1f6b70 2024-09-21T03:35:41.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:41.019+0000 7f0261e7e640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f025c0d0bd0 con 0x7f025c14f990 2024-09-21T03:35:41.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:41.020+0000 7f025affd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f025c14c360 0x7f025c14c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:41.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:41.020+0000 7f025affd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f025c14c360 0x7f025c14c770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:39660/0 (socket says 172.21.15.38:39660) 2024-09-21T03:35:41.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:41.020+0000 7f025affd640 1 -- 172.21.15.38:0/929403115 learned_addr learned my addr 172.21.15.38:0/929403115 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:35:41.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:41.020+0000 7f025b7fe640 1 --2- 172.21.15.38:0/929403115 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f025c14f990 0x7f025c156a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:41.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:41.020+0000 7f025bfff640 1 --2- 172.21.15.38:0/929403115 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f025c1f6b70 0x7f025c1f6f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:41.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:41.020+0000 7f025affd640 1 -- 172.21.15.38:0/929403115 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f025c14f990 msgr2=0x7f025c156a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:41.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:41.020+0000 7f025affd640 1 --2- 172.21.15.38:0/929403115 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f025c14f990 0x7f025c156a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:41.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:41.020+0000 7f025affd640 1 -- 172.21.15.38:0/929403115 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f025c1f6b70 msgr2=0x7f025c1f6f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:41.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:41.020+0000 7f025affd640 1 --2- 172.21.15.38:0/929403115 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f025c1f6b70 0x7f025c1f6f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:41.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:41.020+0000 7f025affd640 1 -- 172.21.15.38:0/929403115 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f025c156fa0 con 0x7f025c14c360 2024-09-21T03:35:41.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:41.020+0000 7f025affd640 1 --2- 172.21.15.38:0/929403115 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f025c14c360 0x7f025c14c770 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f0250006600 tx=0x7f0250003aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:41.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:41.020+0000 7f0243fff640 1 -- 172.21.15.38:0/929403115 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0250015070 con 0x7f025c14c360 2024-09-21T03:35:41.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:41.020+0000 7f0243fff640 1 -- 172.21.15.38:0/929403115 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0250011880 con 0x7f025c14c360 2024-09-21T03:35:41.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:41.020+0000 7f0261e7e640 1 -- 172.21.15.38:0/929403115 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f025c152bf0 con 0x7f025c14c360 2024-09-21T03:35:41.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:41.020+0000 7f0243fff640 1 -- 172.21.15.38:0/929403115 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0250011cd0 con 0x7f025c14c360 2024-09-21T03:35:41.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:41.020+0000 7f0261e7e640 1 -- 172.21.15.38:0/929403115 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f025c152630 con 0x7f025c14c360 2024-09-21T03:35:41.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:41.022+0000 7f0261e7e640 1 -- 172.21.15.38:0/929403115 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f025c0d10c0 con 0x7f025c14c360 2024-09-21T03:35:41.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:41.022+0000 7f0243fff640 1 -- 172.21.15.38:0/929403115 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f0250018070 con 0x7f025c14c360 2024-09-21T03:35:41.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:41.023+0000 7f0243fff640 1 --2- 172.21.15.38:0/929403115 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0248078b90 0x7f024807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:41.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:41.023+0000 7f0243fff640 1 -- 172.21.15.38:0/929403115 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 5249+0+0 (secure 0 0 0) 0x7f025009b6d0 con 0x7f025c14c360 2024-09-21T03:35:41.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:41.023+0000 7f025b7fe640 1 --2- 172.21.15.38:0/929403115 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0248078b90 0x7f024807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:41.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:41.024+0000 7f025b7fe640 1 --2- 172.21.15.38:0/929403115 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0248078b90 0x7f024807b0a0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f024c009c50 tx=0x7f024c005f90 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:41.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:41.027+0000 7f0243fff640 1 -- 172.21.15.38:0/929403115 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f025c156fa0 con 0x7f025c14c360 2024-09-21T03:35:41.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:41.289+0000 7f0261e7e640 1 -- 172.21.15.38:0/929403115 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f025c069290 con 0x7f025c14c360 2024-09-21T03:35:41.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:41.289+0000 7f0243fff640 1 -- 172.21.15.38:0/929403115 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v18) ==== 80+0+5304 (secure 0 0 0) 0x7f02500637a0 con 0x7f025c14c360 2024-09-21T03:35:41.292 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:35:41.292 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-09-21T03:35:41.292 INFO:teuthology.orchestra.run.smithi038.stdout: "auth_dump": [ 2024-09-21T03:35:41.292 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-21T03:35:41.292 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "mds.a", 2024-09-21T03:35:41.292 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQBPPu5mfzGIMRAAmq0TaC3J1VU3GxzhTgJ/Zw==", 2024-09-21T03:35:41.293 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-09-21T03:35:41.293 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow", 2024-09-21T03:35:41.293 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow *", 2024-09-21T03:35:41.293 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow *", 2024-09-21T03:35:41.293 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-09-21T03:35:41.293 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-21T03:35:41.293 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-21T03:35:41.293 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-21T03:35:41.293 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "mds.b", 2024-09-21T03:35:41.293 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQBQPu5m/fzuAhAApyigRlQ8OrV/sf0ap6Fn6A==", 2024-09-21T03:35:41.293 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-09-21T03:35:41.293 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow", 2024-09-21T03:35:41.293 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow *", 2024-09-21T03:35:41.294 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow *", 2024-09-21T03:35:41.294 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-09-21T03:35:41.294 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-21T03:35:41.294 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-21T03:35:41.294 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-21T03:35:41.294 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "mds.c", 2024-09-21T03:35:41.294 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQBPPu5mQipIORAArRn8k8SvvpG8x03F9I2/gQ==", 2024-09-21T03:35:41.294 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-09-21T03:35:41.294 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow", 2024-09-21T03:35:41.294 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow *", 2024-09-21T03:35:41.294 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow *", 2024-09-21T03:35:41.294 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-09-21T03:35:41.295 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-21T03:35:41.295 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-21T03:35:41.295 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-21T03:35:41.295 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "mds.d", 2024-09-21T03:35:41.295 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQBQPu5mzlyDChAAqbPKXS3EtnOqqeHhA1ehIw==", 2024-09-21T03:35:41.295 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-09-21T03:35:41.295 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow", 2024-09-21T03:35:41.295 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow *", 2024-09-21T03:35:41.295 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow *", 2024-09-21T03:35:41.295 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-09-21T03:35:41.295 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-21T03:35:41.295 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-21T03:35:41.295 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-21T03:35:41.295 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "osd.0", 2024-09-21T03:35:41.296 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQBXPu5mMZggAhAAtMPdJipheizZhpby9APj4A==", 2024-09-21T03:35:41.296 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-09-21T03:35:41.296 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow profile osd", 2024-09-21T03:35:41.296 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile osd", 2024-09-21T03:35:41.296 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-09-21T03:35:41.296 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-21T03:35:41.296 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-21T03:35:41.296 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-21T03:35:41.296 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "osd.1", 2024-09-21T03:35:41.296 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQBaPu5mm9TFIhAAWPJCvBYMgMbKr7Z7Diq58A==", 2024-09-21T03:35:41.296 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-09-21T03:35:41.296 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow profile osd", 2024-09-21T03:35:41.296 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile osd", 2024-09-21T03:35:41.296 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-09-21T03:35:41.297 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-21T03:35:41.297 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-21T03:35:41.297 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-21T03:35:41.297 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "osd.2", 2024-09-21T03:35:41.297 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQBePu5mqjljCBAAcdEpzJq6D8e1L+q1s0GjWQ==", 2024-09-21T03:35:41.297 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-09-21T03:35:41.297 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow profile osd", 2024-09-21T03:35:41.297 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile osd", 2024-09-21T03:35:41.297 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-09-21T03:35:41.297 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-21T03:35:41.297 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-21T03:35:41.297 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-21T03:35:41.297 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "osd.3", 2024-09-21T03:35:41.297 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQBhPu5mRjfdKBAAJ6+PlEiRp/xvxny6SjHV8Q==", 2024-09-21T03:35:41.298 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-09-21T03:35:41.298 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow profile osd", 2024-09-21T03:35:41.298 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile osd", 2024-09-21T03:35:41.298 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-09-21T03:35:41.298 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-21T03:35:41.298 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-21T03:35:41.298 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-21T03:35:41.298 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "osd.4", 2024-09-21T03:35:41.298 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQBlPu5m8bNyNxAA2rERruGojEDKmcXN2K6uog==", 2024-09-21T03:35:41.298 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-09-21T03:35:41.298 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow profile osd", 2024-09-21T03:35:41.298 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile osd", 2024-09-21T03:35:41.298 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-09-21T03:35:41.299 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-21T03:35:41.299 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-21T03:35:41.299 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-21T03:35:41.299 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "osd.5", 2024-09-21T03:35:41.299 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQBpPu5mBB2iDBAAvsvhAPMvNWII2Zr9Je45YQ==", 2024-09-21T03:35:41.299 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-09-21T03:35:41.299 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow profile osd", 2024-09-21T03:35:41.299 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile osd", 2024-09-21T03:35:41.299 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-09-21T03:35:41.299 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-21T03:35:41.299 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-21T03:35:41.299 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-21T03:35:41.299 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "osd.6", 2024-09-21T03:35:41.299 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQBsPu5mMQMaLRAAznzYdoNGHWOI2p2mNN/rQw==", 2024-09-21T03:35:41.300 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-09-21T03:35:41.300 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow profile osd", 2024-09-21T03:35:41.300 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile osd", 2024-09-21T03:35:41.300 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-09-21T03:35:41.300 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-21T03:35:41.300 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-21T03:35:41.300 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-21T03:35:41.300 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "osd.7", 2024-09-21T03:35:41.300 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQBwPu5mHu0WERAAir8bmtuNOOj/2D4hLaoYOA==", 2024-09-21T03:35:41.300 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-09-21T03:35:41.300 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow profile osd", 2024-09-21T03:35:41.300 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile osd", 2024-09-21T03:35:41.301 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-09-21T03:35:41.301 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-21T03:35:41.301 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-21T03:35:41.301 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-21T03:35:41.301 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "client.0", 2024-09-21T03:35:41.301 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQBQPu5m6ydBDxAAlWZ2XzQTPBcYPriPGRJVeA==", 2024-09-21T03:35:41.301 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-09-21T03:35:41.301 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow", 2024-09-21T03:35:41.301 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow r", 2024-09-21T03:35:41.301 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-21T03:35:41.301 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-21T03:35:41.301 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-21T03:35:41.301 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-21T03:35:41.301 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "client.1", 2024-09-21T03:35:41.302 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQBQPu5mlWP0FhAAkNwfaJ6rPr3r4ZRVbPxX8g==", 2024-09-21T03:35:41.302 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-09-21T03:35:41.302 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow", 2024-09-21T03:35:41.302 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow r", 2024-09-21T03:35:41.302 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow rw", 2024-09-21T03:35:41.302 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow rwx" 2024-09-21T03:35:41.302 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-21T03:35:41.302 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-21T03:35:41.302 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-21T03:35:41.302 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "client.2", 2024-09-21T03:35:41.302 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQBQPu5m48oaExAAOzYrR/SsykpzH2TkWHs39Q==", 2024-09-21T03:35:41.302 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-09-21T03:35:41.302 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow", 2024-09-21T03:35:41.303 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow r", 2024-09-21T03:35:41.303 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow rw", 2024-09-21T03:35:41.303 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow rwx" 2024-09-21T03:35:41.303 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-21T03:35:41.303 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-21T03:35:41.303 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-21T03:35:41.303 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "client.3", 2024-09-21T03:35:41.303 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQBQPu5mVQSzGhAAAo6vizLc7GOnenQRsgtn1A==", 2024-09-21T03:35:41.303 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-09-21T03:35:41.303 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow", 2024-09-21T03:35:41.303 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow r", 2024-09-21T03:35:41.303 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow rw", 2024-09-21T03:35:41.303 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow rwx" 2024-09-21T03:35:41.303 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-21T03:35:41.304 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-21T03:35:41.304 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-21T03:35:41.304 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "client.admin", 2024-09-21T03:35:41.304 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQBPPu5mPHphExAA7D+/V7qhoNyoc6h0GnNMlQ==", 2024-09-21T03:35:41.304 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-09-21T03:35:41.304 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow *", 2024-09-21T03:35:41.304 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow *", 2024-09-21T03:35:41.304 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow *", 2024-09-21T03:35:41.304 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-09-21T03:35:41.304 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-21T03:35:41.304 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-21T03:35:41.304 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-21T03:35:41.304 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "mgr.x", 2024-09-21T03:35:41.305 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQBPPu5m6K2oLRAAzldoBXAMSnWhxVFwPZkHOw==", 2024-09-21T03:35:41.305 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-09-21T03:35:41.305 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow *", 2024-09-21T03:35:41.305 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile mgr", 2024-09-21T03:35:41.305 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-09-21T03:35:41.305 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-21T03:35:41.305 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-21T03:35:41.305 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-21T03:35:41.305 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "mgr.y", 2024-09-21T03:35:41.305 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQBPPu5mWEz+KBAASx2KzDril3lD9cEawy56jA==", 2024-09-21T03:35:41.305 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-09-21T03:35:41.305 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow *", 2024-09-21T03:35:41.305 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile mgr", 2024-09-21T03:35:41.306 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-09-21T03:35:41.306 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-21T03:35:41.306 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-21T03:35:41.306 INFO:teuthology.orchestra.run.smithi038.stdout: ] 2024-09-21T03:35:41.306 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-09-21T03:35:41.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:41.291+0000 7f0261e7e640 1 -- 172.21.15.38:0/929403115 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0248078b90 msgr2=0x7f024807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:41.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:41.291+0000 7f0261e7e640 1 --2- 172.21.15.38:0/929403115 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0248078b90 0x7f024807b0a0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f024c009c50 tx=0x7f024c005f90 comp rx=0 tx=0).stop 2024-09-21T03:35:41.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:41.292+0000 7f0261e7e640 1 -- 172.21.15.38:0/929403115 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f025c14c360 msgr2=0x7f025c14c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:41.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:41.292+0000 7f0261e7e640 1 --2- 172.21.15.38:0/929403115 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f025c14c360 0x7f025c14c770 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f0250006600 tx=0x7f0250003aa0 comp rx=0 tx=0).stop 2024-09-21T03:35:41.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:41.292+0000 7f0261e7e640 1 -- 172.21.15.38:0/929403115 shutdown_connections 2024-09-21T03:35:41.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:41.292+0000 7f0261e7e640 1 --2- 172.21.15.38:0/929403115 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f025c1f6b70 0x7f025c1f6f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:41.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:41.292+0000 7f0261e7e640 1 --2- 172.21.15.38:0/929403115 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f025c14c360 0x7f025c14c770 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:41.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:41.292+0000 7f0261e7e640 1 --2- 172.21.15.38:0/929403115 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0248078b90 0x7f024807b0a0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:41.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:41.292+0000 7f0261e7e640 1 --2- 172.21.15.38:0/929403115 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f025c14f990 0x7f025c156a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:41.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:41.292+0000 7f0261e7e640 1 -- 172.21.15.38:0/929403115 >> 172.21.15.38:0/929403115 conn(0x7f025c0cef40 msgr2=0x7f025c0cc860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:41.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:41.292+0000 7f0261e7e640 1 -- 172.21.15.38:0/929403115 shutdown_connections 2024-09-21T03:35:41.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:41.292+0000 7f0261e7e640 1 -- 172.21.15.38:0/929403115 wait complete. 2024-09-21T03:35:41.309 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-21T03:35:41.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:41.422+0000 7fc3b73ba640 1 -- 172.21.15.38:0/1664878448 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc3b00cce70 msgr2=0x7fc3b00d04c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:41.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:41.422+0000 7fc3b73ba640 1 --2- 172.21.15.38:0/1664878448 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc3b00cce70 0x7fc3b00d04c0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fc3a8005fd0 tx=0x7fc3a802a5d0 comp rx=0 tx=0).stop 2024-09-21T03:35:41.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:41.422+0000 7fc3b73ba640 1 -- 172.21.15.38:0/1664878448 shutdown_connections 2024-09-21T03:35:41.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:41.422+0000 7fc3b73ba640 1 --2- 172.21.15.38:0/1664878448 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc3b00cce70 0x7fc3b00d04c0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:41.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:41.422+0000 7fc3b73ba640 1 --2- 172.21.15.38:0/1664878448 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc3b00d1ed0 0x7fc3b00cc930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:41.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:41.422+0000 7fc3b73ba640 1 -- 172.21.15.38:0/1664878448 >> 172.21.15.38:0/1664878448 conn(0x7fc3b00cef40 msgr2=0x7fc3b00cf350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:41.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:41.422+0000 7fc3b73ba640 1 -- 172.21.15.38:0/1664878448 shutdown_connections 2024-09-21T03:35:41.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:41.422+0000 7fc3b73ba640 1 -- 172.21.15.38:0/1664878448 wait complete. 2024-09-21T03:35:41.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:41.424+0000 7fc3b73ba640 1 Processor -- start 2024-09-21T03:35:41.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:41.424+0000 7fc3b73ba640 1 -- start start 2024-09-21T03:35:41.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:41.425+0000 7fc3b73ba640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc3b00cce70 0x7fc3b014fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:41.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:41.425+0000 7fc3b73ba640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc3b00d1ed0 0x7fc3b01569a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:41.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:41.425+0000 7fc3b73ba640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc3b01536e0 0x7fc3b0151e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:41.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:41.425+0000 7fc3b73ba640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fc3b00d0d90 con 0x7fc3b01536e0 2024-09-21T03:35:41.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:41.425+0000 7fc3b73ba640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fc3b00d0c10 con 0x7fc3b00d1ed0 2024-09-21T03:35:41.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:41.425+0000 7fc3b73ba640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fc3b00d0f10 con 0x7fc3b00cce70 2024-09-21T03:35:41.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:41.425+0000 7fc3b5957640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc3b01536e0 0x7fc3b0151e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:41.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:41.425+0000 7fc3b5957640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc3b01536e0 0x7fc3b0151e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:39688/0 (socket says 172.21.15.38:39688) 2024-09-21T03:35:41.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:41.425+0000 7fc3b5957640 1 -- 172.21.15.38:0/2489971072 learned_addr learned my addr 172.21.15.38:0/2489971072 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:35:41.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:41.425+0000 7fc3b5156640 1 --2- 172.21.15.38:0/2489971072 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc3b00cce70 0x7fc3b014fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:41.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:41.425+0000 7fc3b4955640 1 --2- 172.21.15.38:0/2489971072 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc3b00d1ed0 0x7fc3b01569a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:41.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:41.426+0000 7fc3b5957640 1 -- 172.21.15.38:0/2489971072 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc3b00cce70 msgr2=0x7fc3b014fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:41.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:41.426+0000 7fc3b5957640 1 --2- 172.21.15.38:0/2489971072 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc3b00cce70 0x7fc3b014fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:41.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:41.426+0000 7fc3b5957640 1 -- 172.21.15.38:0/2489971072 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc3b00d1ed0 msgr2=0x7fc3b01569a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:41.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:41.426+0000 7fc3b5957640 1 --2- 172.21.15.38:0/2489971072 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc3b00d1ed0 0x7fc3b01569a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:41.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:41.426+0000 7fc3b5957640 1 -- 172.21.15.38:0/2489971072 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3b0152780 con 0x7fc3b01536e0 2024-09-21T03:35:41.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:41.426+0000 7fc3b5957640 1 --2- 172.21.15.38:0/2489971072 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc3b01536e0 0x7fc3b0151e10 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fc3a8034040 tx=0x7fc3a8005750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:41.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:41.426+0000 7fc3adffb640 1 -- 172.21.15.38:0/2489971072 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3a803e030 con 0x7fc3b01536e0 2024-09-21T03:35:41.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:41.426+0000 7fc3adffb640 1 -- 172.21.15.38:0/2489971072 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc3a8004d10 con 0x7fc3b01536e0 2024-09-21T03:35:41.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:41.426+0000 7fc3b73ba640 1 -- 172.21.15.38:0/2489971072 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3b0152a60 con 0x7fc3b01536e0 2024-09-21T03:35:41.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:41.426+0000 7fc3adffb640 1 -- 172.21.15.38:0/2489971072 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3a8035040 con 0x7fc3b01536e0 2024-09-21T03:35:41.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:41.426+0000 7fc3b73ba640 1 -- 172.21.15.38:0/2489971072 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3b0156ee0 con 0x7fc3b01536e0 2024-09-21T03:35:41.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:41.427+0000 7fc3b73ba640 1 -- 172.21.15.38:0/2489971072 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3b00d04c0 con 0x7fc3b01536e0 2024-09-21T03:35:41.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:41.428+0000 7fc3adffb640 1 -- 172.21.15.38:0/2489971072 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc3a8041030 con 0x7fc3b01536e0 2024-09-21T03:35:41.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:41.429+0000 7fc3adffb640 1 --2- 172.21.15.38:0/2489971072 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc39c078bc0 0x7fc39c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:41.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:41.429+0000 7fc3adffb640 1 -- 172.21.15.38:0/2489971072 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 5249+0+0 (secure 0 0 0) 0x7fc3a80bb020 con 0x7fc3b01536e0 2024-09-21T03:35:41.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:41.429+0000 7fc3b5156640 1 --2- 172.21.15.38:0/2489971072 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc39c078bc0 0x7fc39c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:41.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:41.429+0000 7fc3b5156640 1 --2- 172.21.15.38:0/2489971072 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc39c078bc0 0x7fc39c07b0d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fc3a0001ce0 tx=0x7fc3a0009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:41.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:41.433+0000 7fc3adffb640 1 -- 172.21.15.38:0/2489971072 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fc3a8083740 con 0x7fc3b01536e0 2024-09-21T03:35:41.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:41.621+0000 7fc3b73ba640 1 -- 172.21.15.38:0/2489971072 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fc3b00752a0 con 0x7fc3b01536e0 2024-09-21T03:35:41.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:41.622+0000 7fc3adffb640 1 -- 172.21.15.38:0/2489971072 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v51) ==== 84+0+65 (secure 0 0 0) 0x7fc3a8083040 con 0x7fc3b01536e0 2024-09-21T03:35:41.625 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:35:41.625 INFO:teuthology.orchestra.run.smithi038.stdout:[ 2024-09-21T03:35:41.625 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-21T03:35:41.625 INFO:teuthology.orchestra.run.smithi038.stdout: "poolnum": 1, 2024-09-21T03:35:41.625 INFO:teuthology.orchestra.run.smithi038.stdout: "poolname": ".mgr" 2024-09-21T03:35:41.625 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-21T03:35:41.625 INFO:teuthology.orchestra.run.smithi038.stdout:] 2024-09-21T03:35:41.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:41.624+0000 7fc3b73ba640 1 -- 172.21.15.38:0/2489971072 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc39c078bc0 msgr2=0x7fc39c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:41.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:41.624+0000 7fc3b73ba640 1 --2- 172.21.15.38:0/2489971072 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc39c078bc0 0x7fc39c07b0d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fc3a0001ce0 tx=0x7fc3a0009040 comp rx=0 tx=0).stop 2024-09-21T03:35:41.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:41.624+0000 7fc3b73ba640 1 -- 172.21.15.38:0/2489971072 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc3b01536e0 msgr2=0x7fc3b0151e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:41.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:41.624+0000 7fc3b73ba640 1 --2- 172.21.15.38:0/2489971072 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc3b01536e0 0x7fc3b0151e10 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fc3a8034040 tx=0x7fc3a8005750 comp rx=0 tx=0).stop 2024-09-21T03:35:41.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:41.624+0000 7fc3b73ba640 1 -- 172.21.15.38:0/2489971072 shutdown_connections 2024-09-21T03:35:41.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:41.624+0000 7fc3b73ba640 1 --2- 172.21.15.38:0/2489971072 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc3b01536e0 0x7fc3b0151e10 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:41.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:41.624+0000 7fc3b73ba640 1 --2- 172.21.15.38:0/2489971072 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc3b00d1ed0 0x7fc3b01569a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:41.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:41.624+0000 7fc3b73ba640 1 --2- 172.21.15.38:0/2489971072 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc39c078bc0 0x7fc39c07b0d0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:41.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:41.624+0000 7fc3b73ba640 1 --2- 172.21.15.38:0/2489971072 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc3b00cce70 0x7fc3b014fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:41.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:41.624+0000 7fc3b73ba640 1 -- 172.21.15.38:0/2489971072 >> 172.21.15.38:0/2489971072 conn(0x7fc3b00cef40 msgr2=0x7fc3b00b5eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:41.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:41.624+0000 7fc3b73ba640 1 -- 172.21.15.38:0/2489971072 shutdown_connections 2024-09-21T03:35:41.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:41.624+0000 7fc3b73ba640 1 -- 172.21.15.38:0/2489971072 wait complete. 2024-09-21T03:35:41.638 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-21T03:35:41.638 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-21T03:35:41.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:41.756+0000 7fa2d9a0e640 1 -- 172.21.15.38:0/2723531344 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa2d41624f0 msgr2=0x7fa2d40b39a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:41.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:41.756+0000 7fa2d9a0e640 1 --2- 172.21.15.38:0/2723531344 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa2d41624f0 0x7fa2d40b39a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa2c4009bb0 tx=0x7fa2c402a820 comp rx=0 tx=0).stop 2024-09-21T03:35:41.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:41.756+0000 7fa2d9a0e640 1 -- 172.21.15.38:0/2723531344 shutdown_connections 2024-09-21T03:35:41.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:41.756+0000 7fa2d9a0e640 1 --2- 172.21.15.38:0/2723531344 >> v2:172.21.15.195:3300/0 conn(0x7fa2d40d34c0 0x7fa2d415f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:41.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:41.756+0000 7fa2d9a0e640 1 --2- 172.21.15.38:0/2723531344 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa2d41624f0 0x7fa2d40b39a0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:41.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:41.756+0000 7fa2d9a0e640 1 -- 172.21.15.38:0/2723531344 >> 172.21.15.38:0/2723531344 conn(0x7fa2d409d800 msgr2=0x7fa2d40b94c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:41.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:41.756+0000 7fa2d9a0e640 1 -- 172.21.15.38:0/2723531344 shutdown_connections 2024-09-21T03:35:41.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:41.756+0000 7fa2d9a0e640 1 -- 172.21.15.38:0/2723531344 wait complete. 2024-09-21T03:35:41.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:41.758+0000 7fa2d9a0e640 1 Processor -- start 2024-09-21T03:35:41.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:41.758+0000 7fa2d9a0e640 1 -- start start 2024-09-21T03:35:41.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:41.760+0000 7fa2d9a0e640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa2d40d34c0 0x7fa2d4156d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:41.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:41.760+0000 7fa2d9a0e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa2d415fb50 0x7fa2d415ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:41.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:41.760+0000 7fa2d9a0e640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa2d41624f0 0x7fa2d415bf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:41.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:41.760+0000 7fa2d9a0e640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d40d44c0 con 0x7fa2d415fb50 2024-09-21T03:35:41.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:41.760+0000 7fa2d9a0e640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d40b69f0 con 0x7fa2d41624f0 2024-09-21T03:35:41.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:41.760+0000 7fa2d9a0e640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fa2d40d4640 con 0x7fa2d40d34c0 2024-09-21T03:35:41.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:41.760+0000 7fa2d27fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa2d415fb50 0x7fa2d415ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:41.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:41.760+0000 7fa2d27fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa2d415fb50 0x7fa2d415ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:39698/0 (socket says 172.21.15.38:39698) 2024-09-21T03:35:41.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:41.760+0000 7fa2d27fc640 1 -- 172.21.15.38:0/815471134 learned_addr learned my addr 172.21.15.38:0/815471134 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:35:41.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:41.760+0000 7fa2d2ffd640 1 --2- 172.21.15.38:0/815471134 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa2d40d34c0 0x7fa2d4156d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:41.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:41.760+0000 7fa2d37fe640 1 --2- 172.21.15.38:0/815471134 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa2d41624f0 0x7fa2d415bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:41.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:41.760+0000 7fa2d27fc640 1 -- 172.21.15.38:0/815471134 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa2d40d34c0 msgr2=0x7fa2d4156d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:41.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:41.760+0000 7fa2d27fc640 1 --2- 172.21.15.38:0/815471134 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa2d40d34c0 0x7fa2d4156d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:41.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:41.760+0000 7fa2d27fc640 1 -- 172.21.15.38:0/815471134 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa2d41624f0 msgr2=0x7fa2d415bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:41.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:41.760+0000 7fa2d27fc640 1 --2- 172.21.15.38:0/815471134 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa2d41624f0 0x7fa2d415bf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:41.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:41.760+0000 7fa2d27fc640 1 -- 172.21.15.38:0/815471134 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2d415ac60 con 0x7fa2d415fb50 2024-09-21T03:35:41.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:41.760+0000 7fa2d2ffd640 1 --2- 172.21.15.38:0/815471134 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa2d40d34c0 0x7fa2d4156d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:35:41.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:41.760+0000 7fa2d27fc640 1 --2- 172.21.15.38:0/815471134 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa2d415fb50 0x7fa2d415ba20 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fa2c800e7c0 tx=0x7fa2c800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:41.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:41.761+0000 7fa2bbfff640 1 -- 172.21.15.38:0/815471134 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2c800ccc0 con 0x7fa2d415fb50 2024-09-21T03:35:41.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:41.761+0000 7fa2bbfff640 1 -- 172.21.15.38:0/815471134 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa2c800ce60 con 0x7fa2d415fb50 2024-09-21T03:35:41.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:41.761+0000 7fa2d9a0e640 1 -- 172.21.15.38:0/815471134 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2d415af40 con 0x7fa2d415fb50 2024-09-21T03:35:41.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:41.761+0000 7fa2bbfff640 1 -- 172.21.15.38:0/815471134 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2c8005260 con 0x7fa2d415fb50 2024-09-21T03:35:41.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:41.761+0000 7fa2d9a0e640 1 -- 172.21.15.38:0/815471134 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2d415b500 con 0x7fa2d415fb50 2024-09-21T03:35:41.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:41.762+0000 7fa2d9a0e640 1 -- 172.21.15.38:0/815471134 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2d415f610 con 0x7fa2d415fb50 2024-09-21T03:35:41.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:41.763+0000 7fa2bbfff640 1 -- 172.21.15.38:0/815471134 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa2c8005400 con 0x7fa2d415fb50 2024-09-21T03:35:41.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:41.763+0000 7fa2bbfff640 1 --2- 172.21.15.38:0/815471134 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa2c0078bc0 0x7fa2c007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:41.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:41.763+0000 7fa2bbfff640 1 -- 172.21.15.38:0/815471134 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 5249+0+0 (secure 0 0 0) 0x7fa2c809b3c0 con 0x7fa2d415fb50 2024-09-21T03:35:41.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:41.764+0000 7fa2d2ffd640 1 --2- 172.21.15.38:0/815471134 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa2c0078bc0 0x7fa2c007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:41.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:41.764+0000 7fa2d2ffd640 1 --2- 172.21.15.38:0/815471134 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa2c0078bc0 0x7fa2c007b0d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fa2d40d5360 tx=0x7fa2c4004ab0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:41.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:41.768+0000 7fa2bbfff640 1 -- 172.21.15.38:0/815471134 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fa2c80632f0 con 0x7fa2d415fb50 2024-09-21T03:35:41.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:41.956+0000 7fa2d9a0e640 1 -- 172.21.15.38:0/815471134 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fa2d40d4bc0 con 0x7fa2d415fb50 2024-09-21T03:35:42.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:42.878+0000 7fa2bbfff640 1 -- 172.21.15.38:0/815471134 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v52) ==== 138+0+0 (secure 0 0 0) 0x7fa2c8067d20 con 0x7fa2d415fb50 2024-09-21T03:35:42.881 INFO:teuthology.orchestra.run.smithi038.stderr:pool 'cephfs_metadata' created 2024-09-21T03:35:42.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:42.882+0000 7fa2d9a0e640 1 -- 172.21.15.38:0/815471134 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa2c0078bc0 msgr2=0x7fa2c007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:42.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:42.882+0000 7fa2d9a0e640 1 --2- 172.21.15.38:0/815471134 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa2c0078bc0 0x7fa2c007b0d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fa2d40d5360 tx=0x7fa2c4004ab0 comp rx=0 tx=0).stop 2024-09-21T03:35:42.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:42.882+0000 7fa2d9a0e640 1 -- 172.21.15.38:0/815471134 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa2d415fb50 msgr2=0x7fa2d415ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:42.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:42.882+0000 7fa2d9a0e640 1 --2- 172.21.15.38:0/815471134 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa2d415fb50 0x7fa2d415ba20 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fa2c800e7c0 tx=0x7fa2c800ec80 comp rx=0 tx=0).stop 2024-09-21T03:35:42.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:42.882+0000 7fa2d9a0e640 1 -- 172.21.15.38:0/815471134 shutdown_connections 2024-09-21T03:35:42.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:42.883+0000 7fa2d9a0e640 1 --2- 172.21.15.38:0/815471134 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa2d41624f0 0x7fa2d415bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:42.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:42.883+0000 7fa2d9a0e640 1 --2- 172.21.15.38:0/815471134 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa2d415fb50 0x7fa2d415ba20 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:42.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:42.883+0000 7fa2d9a0e640 1 --2- 172.21.15.38:0/815471134 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa2c0078bc0 0x7fa2c007b0d0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:42.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:42.883+0000 7fa2d9a0e640 1 --2- 172.21.15.38:0/815471134 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa2d40d34c0 0x7fa2d4156d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:42.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:42.883+0000 7fa2d9a0e640 1 -- 172.21.15.38:0/815471134 >> 172.21.15.38:0/815471134 conn(0x7fa2d409d800 msgr2=0x7fa2d4158970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:42.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:42.883+0000 7fa2d9a0e640 1 -- 172.21.15.38:0/815471134 shutdown_connections 2024-09-21T03:35:42.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:42.883+0000 7fa2d9a0e640 1 -- 172.21.15.38:0/815471134 wait complete. 2024-09-21T03:35:42.903 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-21T03:35:43.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:43.032+0000 7f5133430640 1 -- 172.21.15.38:0/1928119684 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5118005250 msgr2=0x7f511800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:43.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:43.032+0000 7f5133430640 1 --2- 172.21.15.38:0/1928119684 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5118005250 0x7f511800b9c0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f5124001dc0 tx=0x7f5118005830 comp rx=0 tx=0).stop 2024-09-21T03:35:43.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:43.032+0000 7f5133430640 1 -- 172.21.15.38:0/1928119684 shutdown_connections 2024-09-21T03:35:43.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:43.032+0000 7f5133430640 1 --2- 172.21.15.38:0/1928119684 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5118005250 0x7f511800b9c0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:43.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:43.032+0000 7f5133430640 1 -- 172.21.15.38:0/1928119684 >> 172.21.15.38:0/1928119684 conn(0x7f512c0d4600 msgr2=0x7f512c0d4a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:43.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:43.032+0000 7f5133430640 1 -- 172.21.15.38:0/1928119684 shutdown_connections 2024-09-21T03:35:43.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:43.032+0000 7f5133430640 1 -- 172.21.15.38:0/1928119684 wait complete. 2024-09-21T03:35:43.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:43.034+0000 7f5133430640 1 Processor -- start 2024-09-21T03:35:43.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:43.034+0000 7f5133430640 1 -- start start 2024-09-21T03:35:43.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:43.035+0000 7f5133430640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5118005250 0x7f512c1f9e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:43.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:43.035+0000 7f5133430640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f512c1527d0 0x7f512c149780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:43.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:43.035+0000 7f5133430640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f512c159030 0x7f512c160350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:43.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:43.035+0000 7f5133430640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f512c162f50 con 0x7f5118005250 2024-09-21T03:35:43.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:43.035+0000 7f5133430640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f512c162dd0 con 0x7f512c1527d0 2024-09-21T03:35:43.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:43.035+0000 7f5133430640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f512c1635d0 con 0x7f512c159030 2024-09-21T03:35:43.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:43.035+0000 7f51311cc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5118005250 0x7f512c1f9e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:43.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:43.035+0000 7f51311cc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5118005250 0x7f512c1f9e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40014/0 (socket says 172.21.15.38:40014) 2024-09-21T03:35:43.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:43.035+0000 7f51311cc640 1 -- 172.21.15.38:0/1557625349 learned_addr learned my addr 172.21.15.38:0/1557625349 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:35:43.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:43.035+0000 7f51319cd640 1 --2- 172.21.15.38:0/1557625349 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f512c159030 0x7f512c160350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:43.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:43.035+0000 7f51309cb640 1 --2- 172.21.15.38:0/1557625349 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f512c1527d0 0x7f512c149780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:43.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:43.035+0000 7f51311cc640 1 -- 172.21.15.38:0/1557625349 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f512c159030 msgr2=0x7f512c160350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:43.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:43.035+0000 7f51311cc640 1 --2- 172.21.15.38:0/1557625349 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f512c159030 0x7f512c160350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:43.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:43.035+0000 7f51311cc640 1 -- 172.21.15.38:0/1557625349 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f512c1527d0 msgr2=0x7f512c149780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:43.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:43.035+0000 7f51311cc640 1 --2- 172.21.15.38:0/1557625349 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f512c1527d0 0x7f512c149780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:43.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:43.035+0000 7f51311cc640 1 -- 172.21.15.38:0/1557625349 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f512c156b10 con 0x7f5118005250 2024-09-21T03:35:43.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:43.035+0000 7f51319cd640 1 --2- 172.21.15.38:0/1557625349 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f512c159030 0x7f512c160350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:35:43.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:43.035+0000 7f51309cb640 1 --2- 172.21.15.38:0/1557625349 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f512c1527d0 0x7f512c149780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:35:43.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:43.036+0000 7f51311cc640 1 --2- 172.21.15.38:0/1557625349 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5118005250 0x7f512c1f9e80 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f5128001d70 tx=0x7f51280039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:43.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:43.036+0000 7f5121ffb640 1 -- 172.21.15.38:0/1557625349 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5128015070 con 0x7f5118005250 2024-09-21T03:35:43.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:43.036+0000 7f5121ffb640 1 -- 172.21.15.38:0/1557625349 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5128011900 con 0x7f5118005250 2024-09-21T03:35:43.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:43.036+0000 7f5133430640 1 -- 172.21.15.38:0/1557625349 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f512c156df0 con 0x7f5118005250 2024-09-21T03:35:43.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:43.036+0000 7f5121ffb640 1 -- 172.21.15.38:0/1557625349 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5128011d50 con 0x7f5118005250 2024-09-21T03:35:43.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:43.036+0000 7f5133430640 1 -- 172.21.15.38:0/1557625349 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f512c158b10 con 0x7f5118005250 2024-09-21T03:35:43.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:43.037+0000 7f5133430640 1 -- 172.21.15.38:0/1557625349 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f512c157030 con 0x7f5118005250 2024-09-21T03:35:43.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:43.038+0000 7f5121ffb640 1 -- 172.21.15.38:0/1557625349 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f5128010420 con 0x7f5118005250 2024-09-21T03:35:43.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:43.038+0000 7f5121ffb640 1 --2- 172.21.15.38:0/1557625349 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5110078b90 0x7f511007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:43.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:43.038+0000 7f5121ffb640 1 -- 172.21.15.38:0/1557625349 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5619+0+0 (secure 0 0 0) 0x7f512809b8d0 con 0x7f5118005250 2024-09-21T03:35:43.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:43.039+0000 7f51309cb640 1 --2- 172.21.15.38:0/1557625349 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5110078b90 0x7f511007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:43.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:43.039+0000 7f51309cb640 1 --2- 172.21.15.38:0/1557625349 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5110078b90 0x7f511007b0a0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f511c009460 tx=0x7f511c005f90 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:43.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:43.043+0000 7f5121ffb640 1 -- 172.21.15.38:0/1557625349 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f512c157030 con 0x7f5118005250 2024-09-21T03:35:43.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:43.231+0000 7f5133430640 1 -- 172.21.15.38:0/1557625349 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f512c0cf790 con 0x7f5118005250 2024-09-21T03:35:43.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:43.884+0000 7f5121ffb640 1 -- 172.21.15.38:0/1557625349 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v53) ==== 170+0+0 (secure 0 0 0) 0x7f512c163100 con 0x7f5118005250 2024-09-21T03:35:43.887 INFO:teuthology.orchestra.run.smithi038.stderr:pool 'cephfs_data' created 2024-09-21T03:35:43.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:43.887+0000 7f5133430640 1 -- 172.21.15.38:0/1557625349 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5110078b90 msgr2=0x7f511007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:43.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:43.887+0000 7f5133430640 1 --2- 172.21.15.38:0/1557625349 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5110078b90 0x7f511007b0a0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f511c009460 tx=0x7f511c005f90 comp rx=0 tx=0).stop 2024-09-21T03:35:43.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:43.887+0000 7f5133430640 1 -- 172.21.15.38:0/1557625349 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5118005250 msgr2=0x7f512c1f9e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:43.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:43.887+0000 7f5133430640 1 --2- 172.21.15.38:0/1557625349 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5118005250 0x7f512c1f9e80 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f5128001d70 tx=0x7f51280039c0 comp rx=0 tx=0).stop 2024-09-21T03:35:43.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:43.887+0000 7f5133430640 1 -- 172.21.15.38:0/1557625349 shutdown_connections 2024-09-21T03:35:43.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:43.887+0000 7f5133430640 1 --2- 172.21.15.38:0/1557625349 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5110078b90 0x7f511007b0a0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:43.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:43.887+0000 7f5133430640 1 --2- 172.21.15.38:0/1557625349 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f512c159030 0x7f512c160350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:43.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:43.887+0000 7f5133430640 1 --2- 172.21.15.38:0/1557625349 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f512c1527d0 0x7f512c149780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:43.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:43.887+0000 7f5133430640 1 --2- 172.21.15.38:0/1557625349 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5118005250 0x7f512c1f9e80 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:43.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:43.887+0000 7f5133430640 1 -- 172.21.15.38:0/1557625349 >> 172.21.15.38:0/1557625349 conn(0x7f512c0d4600 msgr2=0x7f512c15d440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:43.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:43.887+0000 7f5133430640 1 -- 172.21.15.38:0/1557625349 shutdown_connections 2024-09-21T03:35:43.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:43.887+0000 7f5133430640 1 -- 172.21.15.38:0/1557625349 wait complete. 2024-09-21T03:35:43.912 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-21T03:35:44.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:44.032+0000 7fb080869640 1 -- 172.21.15.38:0/4166694805 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb07c0cd440 msgr2=0x7fb07c0d0670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:44.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:44.032+0000 7fb080869640 1 --2- 172.21.15.38:0/4166694805 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb07c0cd440 0x7fb07c0d0670 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fb070005fd0 tx=0x7fb07002a540 comp rx=0 tx=0).stop 2024-09-21T03:35:44.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:44.032+0000 7fb080869640 1 -- 172.21.15.38:0/4166694805 shutdown_connections 2024-09-21T03:35:44.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:44.032+0000 7fb080869640 1 --2- 172.21.15.38:0/4166694805 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb07c0cd440 0x7fb07c0d0670 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:44.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:44.032+0000 7fb080869640 1 --2- 172.21.15.38:0/4166694805 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb07c0ccc40 0x7fb07c15ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:44.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:44.032+0000 7fb080869640 1 -- 172.21.15.38:0/4166694805 >> 172.21.15.38:0/4166694805 conn(0x7fb07c0b46e0 msgr2=0x7fb07c0d23e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:44.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:44.032+0000 7fb080869640 1 -- 172.21.15.38:0/4166694805 shutdown_connections 2024-09-21T03:35:44.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:44.032+0000 7fb080869640 1 -- 172.21.15.38:0/4166694805 wait complete. 2024-09-21T03:35:44.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:44.034+0000 7fb080869640 1 Processor -- start 2024-09-21T03:35:44.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:44.034+0000 7fb080869640 1 -- start start 2024-09-21T03:35:44.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:44.035+0000 7fb080869640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb07c0ccc40 0x7fb07c0b0c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:44.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:44.035+0000 7fb080869640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb07c0cd440 0x7fb07c0af5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:44.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:44.035+0000 7fb080869640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb07c0d0bb0 0x7fb07c0afb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:44.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:44.035+0000 7fb080869640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fb07c070810 con 0x7fb07c0d0bb0 2024-09-21T03:35:44.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:44.035+0000 7fb080869640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fb07c0b6e00 con 0x7fb07c0ccc40 2024-09-21T03:35:44.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:44.035+0000 7fb080869640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fb07c070990 con 0x7fb07c0cd440 2024-09-21T03:35:44.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:44.035+0000 7fb07ad9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb07c0d0bb0 0x7fb07c0afb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:44.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:44.036+0000 7fb07ad9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb07c0d0bb0 0x7fb07c0afb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40034/0 (socket says 172.21.15.38:40034) 2024-09-21T03:35:44.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:44.036+0000 7fb07ad9d640 1 -- 172.21.15.38:0/2285659023 learned_addr learned my addr 172.21.15.38:0/2285659023 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:35:44.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:44.036+0000 7fb07a59c640 1 --2- 172.21.15.38:0/2285659023 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb07c0ccc40 0x7fb07c0b0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:44.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:44.036+0000 7fb079d9b640 1 --2- 172.21.15.38:0/2285659023 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb07c0cd440 0x7fb07c0af5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:44.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:44.036+0000 7fb07ad9d640 1 -- 172.21.15.38:0/2285659023 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb07c0cd440 msgr2=0x7fb07c0af5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:44.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:44.036+0000 7fb07ad9d640 1 --2- 172.21.15.38:0/2285659023 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb07c0cd440 0x7fb07c0af5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:44.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:44.036+0000 7fb07ad9d640 1 -- 172.21.15.38:0/2285659023 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb07c0ccc40 msgr2=0x7fb07c0b0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:44.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:44.036+0000 7fb07ad9d640 1 --2- 172.21.15.38:0/2285659023 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb07c0ccc40 0x7fb07c0b0c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:44.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:44.036+0000 7fb07ad9d640 1 -- 172.21.15.38:0/2285659023 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb07c0addd0 con 0x7fb07c0d0bb0 2024-09-21T03:35:44.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:44.036+0000 7fb07ad9d640 1 --2- 172.21.15.38:0/2285659023 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb07c0d0bb0 0x7fb07c0afb30 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fb07400c760 tx=0x7fb07400cb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:44.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:44.036+0000 7fb06affd640 1 -- 172.21.15.38:0/2285659023 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb074003b70 con 0x7fb07c0d0bb0 2024-09-21T03:35:44.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:44.036+0000 7fb06affd640 1 -- 172.21.15.38:0/2285659023 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb074003d10 con 0x7fb07c0d0bb0 2024-09-21T03:35:44.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:44.036+0000 7fb080869640 1 -- 172.21.15.38:0/2285659023 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb07c0c7b10 con 0x7fb07c0d0bb0 2024-09-21T03:35:44.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:44.036+0000 7fb06affd640 1 -- 172.21.15.38:0/2285659023 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb074005330 con 0x7fb07c0d0bb0 2024-09-21T03:35:44.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:44.036+0000 7fb080869640 1 -- 172.21.15.38:0/2285659023 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb07c15bd70 con 0x7fb07c0d0bb0 2024-09-21T03:35:44.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:44.037+0000 7fb080869640 1 -- 172.21.15.38:0/2285659023 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb07c0adb40 con 0x7fb07c0d0bb0 2024-09-21T03:35:44.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:44.038+0000 7fb06affd640 1 -- 172.21.15.38:0/2285659023 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb0740054d0 con 0x7fb07c0d0bb0 2024-09-21T03:35:44.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:44.039+0000 7fb06affd640 1 --2- 172.21.15.38:0/2285659023 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb05c078bc0 0x7fb05c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:44.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:44.039+0000 7fb06affd640 1 -- 172.21.15.38:0/2285659023 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6001+0+0 (secure 0 0 0) 0x7fb07409b510 con 0x7fb07c0d0bb0 2024-09-21T03:35:44.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:44.039+0000 7fb07a59c640 1 --2- 172.21.15.38:0/2285659023 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb05c078bc0 0x7fb05c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:44.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:44.040+0000 7fb07a59c640 1 --2- 172.21.15.38:0/2285659023 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb05c078bc0 0x7fb05c07b0d0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fb06c009b30 tx=0x7fb06c005f90 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:44.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:44.043+0000 7fb06affd640 1 -- 172.21.15.38:0/2285659023 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fb07c0d0fc0 con 0x7fb07c0d0bb0 2024-09-21T03:35:44.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:44.268+0000 7fb080869640 1 -- 172.21.15.38:0/2285659023 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fb07c0c7900 con 0x7fb07c0d0bb0 2024-09-21T03:35:44.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:44.886+0000 7fb06affd640 1 -- 172.21.15.38:0/2285659023 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 7 and data pool 8 v27) ==== 172+0+0 (secure 0 0 0) 0x7fb074068e20 con 0x7fb07c0d0bb0 2024-09-21T03:35:44.889 INFO:teuthology.orchestra.run.smithi038.stderr:new fs with metadata pool 7 and data pool 8 2024-09-21T03:35:44.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:44.889+0000 7fb080869640 1 -- 172.21.15.38:0/2285659023 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb05c078bc0 msgr2=0x7fb05c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:44.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:44.889+0000 7fb080869640 1 --2- 172.21.15.38:0/2285659023 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb05c078bc0 0x7fb05c07b0d0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fb06c009b30 tx=0x7fb06c005f90 comp rx=0 tx=0).stop 2024-09-21T03:35:44.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:44.889+0000 7fb080869640 1 -- 172.21.15.38:0/2285659023 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb07c0d0bb0 msgr2=0x7fb07c0afb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:44.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:44.889+0000 7fb080869640 1 --2- 172.21.15.38:0/2285659023 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb07c0d0bb0 0x7fb07c0afb30 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fb07400c760 tx=0x7fb07400cb60 comp rx=0 tx=0).stop 2024-09-21T03:35:44.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:44.890+0000 7fb080869640 1 -- 172.21.15.38:0/2285659023 shutdown_connections 2024-09-21T03:35:44.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:44.890+0000 7fb080869640 1 --2- 172.21.15.38:0/2285659023 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb07c0d0bb0 0x7fb07c0afb30 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:44.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:44.890+0000 7fb080869640 1 --2- 172.21.15.38:0/2285659023 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb05c078bc0 0x7fb05c07b0d0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:44.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:44.890+0000 7fb080869640 1 --2- 172.21.15.38:0/2285659023 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb07c0cd440 0x7fb07c0af5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:44.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:44.890+0000 7fb080869640 1 --2- 172.21.15.38:0/2285659023 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb07c0ccc40 0x7fb07c0b0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:44.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:44.890+0000 7fb080869640 1 -- 172.21.15.38:0/2285659023 >> 172.21.15.38:0/2285659023 conn(0x7fb07c0b46e0 msgr2=0x7fb07c0b36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:44.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:44.890+0000 7fb080869640 1 -- 172.21.15.38:0/2285659023 shutdown_connections 2024-09-21T03:35:44.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:44.890+0000 7fb080869640 1 -- 172.21.15.38:0/2285659023 wait complete. 2024-09-21T03:35:44.921 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T03:35:45.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:45.042+0000 7f0ee8894640 1 -- 172.21.15.38:0/2645115100 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0ee40cea90 msgr2=0x7f0ee40d00d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:45.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:45.042+0000 7f0ee8894640 1 --2- 172.21.15.38:0/2645115100 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0ee40cea90 0x7f0ee40d00d0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f0ed40098e0 tx=0x7f0ed402a530 comp rx=0 tx=0).stop 2024-09-21T03:35:45.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:45.043+0000 7f0ee8894640 1 -- 172.21.15.38:0/2645115100 shutdown_connections 2024-09-21T03:35:45.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:45.043+0000 7f0ee8894640 1 --2- 172.21.15.38:0/2645115100 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0ee40d3360 0x7f0ee40d3770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:45.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:45.043+0000 7f0ee8894640 1 --2- 172.21.15.38:0/2645115100 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0ee40cea90 0x7f0ee40d00d0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:45.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:45.043+0000 7f0ee8894640 1 -- 172.21.15.38:0/2645115100 >> 172.21.15.38:0/2645115100 conn(0x7f0ee40d4620 msgr2=0x7f0ee40d4a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:45.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:45.043+0000 7f0ee8894640 1 -- 172.21.15.38:0/2645115100 shutdown_connections 2024-09-21T03:35:45.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:45.043+0000 7f0ee8894640 1 -- 172.21.15.38:0/2645115100 wait complete. 2024-09-21T03:35:45.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:45.045+0000 7f0ee8894640 1 Processor -- start 2024-09-21T03:35:45.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:45.045+0000 7f0ee8894640 1 -- start start 2024-09-21T03:35:45.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:45.046+0000 7f0ee8894640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0ee40cea90 0x7f0ee415afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:45.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:45.046+0000 7f0ee8894640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0ee40d3360 0x7f0ee4159a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:45.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:45.046+0000 7f0ee8894640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0ee4162790 0x7f0ee4159fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:45.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:45.046+0000 7f0ee8894640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f0ee40b4180 con 0x7f0ee4162790 2024-09-21T03:35:45.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:45.046+0000 7f0ee8894640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f0ee40b4000 con 0x7f0ee40cea90 2024-09-21T03:35:45.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:45.046+0000 7f0ee8894640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f0ee40b4300 con 0x7f0ee40d3360 2024-09-21T03:35:45.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:45.046+0000 7f0ee2d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0ee4162790 0x7f0ee4159fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:45.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:45.046+0000 7f0ee2d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0ee4162790 0x7f0ee4159fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40058/0 (socket says 172.21.15.38:40058) 2024-09-21T03:35:45.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:45.046+0000 7f0ee2d9d640 1 -- 172.21.15.38:0/2242762574 learned_addr learned my addr 172.21.15.38:0/2242762574 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:35:45.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:45.046+0000 7f0ee259c640 1 --2- 172.21.15.38:0/2242762574 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0ee40cea90 0x7f0ee415afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:45.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:45.046+0000 7f0ee1d9b640 1 --2- 172.21.15.38:0/2242762574 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0ee40d3360 0x7f0ee4159a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:45.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:45.046+0000 7f0ee2d9d640 1 -- 172.21.15.38:0/2242762574 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0ee40d3360 msgr2=0x7f0ee4159a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:45.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:45.046+0000 7f0ee2d9d640 1 --2- 172.21.15.38:0/2242762574 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0ee40d3360 0x7f0ee4159a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:45.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:45.046+0000 7f0ee2d9d640 1 -- 172.21.15.38:0/2242762574 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0ee40cea90 msgr2=0x7f0ee415afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:45.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:45.046+0000 7f0ee2d9d640 1 --2- 172.21.15.38:0/2242762574 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0ee40cea90 0x7f0ee415afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:45.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:45.046+0000 7f0ee2d9d640 1 -- 172.21.15.38:0/2242762574 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ee4156810 con 0x7f0ee4162790 2024-09-21T03:35:45.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:45.047+0000 7f0ee2d9d640 1 --2- 172.21.15.38:0/2242762574 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0ee4162790 0x7f0ee4159fb0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f0edc00cb00 tx=0x7f0edc00a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:45.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:45.047+0000 7f0ed2ffd640 1 -- 172.21.15.38:0/2242762574 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0edc00f450 con 0x7f0ee4162790 2024-09-21T03:35:45.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:45.047+0000 7f0ed2ffd640 1 -- 172.21.15.38:0/2242762574 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0edc00f700 con 0x7f0ee4162790 2024-09-21T03:35:45.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:45.047+0000 7f0ee8894640 1 -- 172.21.15.38:0/2242762574 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ee4156af0 con 0x7f0ee4162790 2024-09-21T03:35:45.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:45.047+0000 7f0ed2ffd640 1 -- 172.21.15.38:0/2242762574 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0edc00fb50 con 0x7f0ee4162790 2024-09-21T03:35:45.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:45.047+0000 7f0ee8894640 1 -- 172.21.15.38:0/2242762574 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ee41570b0 con 0x7f0ee4162790 2024-09-21T03:35:45.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:45.048+0000 7f0ee8894640 1 -- 172.21.15.38:0/2242762574 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ee4156580 con 0x7f0ee4162790 2024-09-21T03:35:45.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:45.049+0000 7f0ed2ffd640 1 -- 172.21.15.38:0/2242762574 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f0edc00fcf0 con 0x7f0ee4162790 2024-09-21T03:35:45.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:45.049+0000 7f0ed2ffd640 1 --2- 172.21.15.38:0/2242762574 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0ec4078bc0 0x7f0ec407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:45.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:45.049+0000 7f0ed2ffd640 1 -- 172.21.15.38:0/2242762574 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f0edc09b8c0 con 0x7f0ee4162790 2024-09-21T03:35:45.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:45.050+0000 7f0ee259c640 1 --2- 172.21.15.38:0/2242762574 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0ec4078bc0 0x7f0ec407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:45.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:45.050+0000 7f0ee259c640 1 --2- 172.21.15.38:0/2242762574 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0ec4078bc0 0x7f0ec407b0d0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f0ed4034000 tx=0x7f0ed4004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:45.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:45.054+0000 7f0ed2ffd640 1 -- 172.21.15.38:0/2242762574 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f0edc014030 con 0x7f0ee4162790 2024-09-21T03:35:45.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:45.242+0000 7f0ee8894640 1 -- 172.21.15.38:0/2242762574 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0ee40b47b0 con 0x7f0ee4162790 2024-09-21T03:35:45.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:45.243+0000 7f0ed2ffd640 1 -- 172.21.15.38:0/2242762574 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) ==== 74+0+16979 (secure 0 0 0) 0x7f0edc063530 con 0x7f0ee4162790 2024-09-21T03:35:45.246 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:35:45.247 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":54,"fsid":"9d50d66f-656b-4fe2-8e3b-fe53b79c4d43","created":"2024-09-21T03:33:10.677865+0000","modified":"2024-09-21T03:35:44.271212+0000","last_up_change":"2024-09-21T03:33:24.023421+0000","last_in_change":"2024-09-21T03:33:14.296193+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T03:33:22.992573+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-09-21T03:35:41.962122+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-09-21T03:35:43.237360+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dbab183f-7e63-48b8-960a-ae55fd5bf565","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6801","nonce":2381639433}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6803","nonce":2381639433}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6807","nonce":2381639433}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6805","nonce":2381639433}]},"public_addr":"172.21.15.38:6801/2381639433","cluster_addr":"172.21.15.38:6803/2381639433","heartbeat_back_addr":"172.21.15.38:6807/2381639433","heartbeat_front_addr":"172.21.15.38:6805/2381639433","state":["exists","up"]},{"osd":1,"uuid":"fa1c0e8a-ebc1-4359-8146-bd5342abbc16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6817","nonce":1491818732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6819","nonce":1491818732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6823","nonce":1491818732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6821","nonce":1491818732}]},"public_addr":"172.21.15.38:6817/1491818732","cluster_addr":"172.21.15.38:6819/1491818732","heartbeat_back_addr":"172.21.15.38:6823/1491818732","heartbeat_front_addr":"172.21.15.38:6821/1491818732","state":["exists","up"]},{"osd":2,"uuid":"2ed3cd2d-385d-4fc6-9869-214552a84803","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6825","nonce":2577858902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6827","nonce":2577858902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6831","nonce":2577858902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6829","nonce":2577858902}]},"public_addr":"172.21.15.38:6825/2577858902","cluster_addr":"172.21.15.38:6827/2577858902","heartbeat_back_addr":"172.21.15.38:6831/2577858902","heartbeat_front_addr":"172.21.15.38:6829/2577858902","state":["exists","up"]},{"osd":3,"uuid":"62d1d95e-919d-4250-be71-1d34cfa70dbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6809","nonce":1286810294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6811","nonce":1286810294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6815","nonce":1286810294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6813","nonce":1286810294}]},"public_addr":"172.21.15.38:6809/1286810294","cluster_addr":"172.21.15.38:6811/1286810294","heartbeat_back_addr":"172.21.15.38:6815/1286810294","heartbeat_front_addr":"172.21.15.38:6813/1286810294","state":["exists","up"]},{"osd":4,"uuid":"7c9f9314-99d7-4d73-8392-37809a9f54b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6803","nonce":3401837571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6805","nonce":3401837571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6809","nonce":3401837571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6807","nonce":3401837571}]},"public_addr":"172.21.15.195:6803/3401837571","cluster_addr":"172.21.15.195:6805/3401837571","heartbeat_back_addr":"172.21.15.195:6809/3401837571","heartbeat_front_addr":"172.21.15.195:6807/3401837571","state":["exists","up"]},{"osd":5,"uuid":"bde4fe9f-8414-4bba-bf4a-43973d1976bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6811","nonce":2080960180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6813","nonce":2080960180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6817","nonce":2080960180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6815","nonce":2080960180}]},"public_addr":"172.21.15.195:6811/2080960180","cluster_addr":"172.21.15.195:6813/2080960180","heartbeat_back_addr":"172.21.15.195:6817/2080960180","heartbeat_front_addr":"172.21.15.195:6815/2080960180","state":["exists","up"]},{"osd":6,"uuid":"6ee6d7f4-1a5f-4a4c-ac16-48fe05f6a002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6827","nonce":4055825675}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6829","nonce":4055825675}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6833","nonce":4055825675}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6831","nonce":4055825675}]},"public_addr":"172.21.15.195:6827/4055825675","cluster_addr":"172.21.15.195:6829/4055825675","heartbeat_back_addr":"172.21.15.195:6833/4055825675","heartbeat_front_addr":"172.21.15.195:6831/4055825675","state":["exists","up"]},{"osd":7,"uuid":"efeb5f51-329a-4484-8290-92e7d3503d05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6819","nonce":1606106342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6821","nonce":1606106342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6825","nonce":1606106342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6823","nonce":1606106342}]},"public_addr":"172.21.15.195:6819/1606106342","cluster_addr":"172.21.15.195:6821/1606106342","heartbeat_back_addr":"172.21.15.195:6825/1606106342","heartbeat_front_addr":"172.21.15.195:6823/1606106342","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.901343+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:21.115003+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.901623+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.835238+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.132199+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:19.817560+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.829690+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:21.087300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T03:35:45.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:45.245+0000 7f0ee8894640 1 -- 172.21.15.38:0/2242762574 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0ec4078bc0 msgr2=0x7f0ec407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:45.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:45.245+0000 7f0ee8894640 1 --2- 172.21.15.38:0/2242762574 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0ec4078bc0 0x7f0ec407b0d0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f0ed4034000 tx=0x7f0ed4004730 comp rx=0 tx=0).stop 2024-09-21T03:35:45.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:45.246+0000 7f0ee8894640 1 -- 172.21.15.38:0/2242762574 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0ee4162790 msgr2=0x7f0ee4159fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:45.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:45.246+0000 7f0ee8894640 1 --2- 172.21.15.38:0/2242762574 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0ee4162790 0x7f0ee4159fb0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f0edc00cb00 tx=0x7f0edc00a250 comp rx=0 tx=0).stop 2024-09-21T03:35:45.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:45.246+0000 7f0ee8894640 1 -- 172.21.15.38:0/2242762574 shutdown_connections 2024-09-21T03:35:45.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:45.246+0000 7f0ee8894640 1 --2- 172.21.15.38:0/2242762574 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0ee4162790 0x7f0ee4159fb0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:45.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:45.246+0000 7f0ee8894640 1 --2- 172.21.15.38:0/2242762574 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0ec4078bc0 0x7f0ec407b0d0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:45.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:45.246+0000 7f0ee8894640 1 --2- 172.21.15.38:0/2242762574 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0ee40d3360 0x7f0ee4159a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:45.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:45.246+0000 7f0ee8894640 1 --2- 172.21.15.38:0/2242762574 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0ee40cea90 0x7f0ee415afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:45.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:45.246+0000 7f0ee8894640 1 -- 172.21.15.38:0/2242762574 >> 172.21.15.38:0/2242762574 conn(0x7f0ee40d4620 msgr2=0x7f0ee406ce70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:45.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:45.246+0000 7f0ee8894640 1 -- 172.21.15.38:0/2242762574 shutdown_connections 2024-09-21T03:35:45.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:45.246+0000 7f0ee8894640 1 -- 172.21.15.38:0/2242762574 wait complete. 2024-09-21T03:35:45.260 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T03:35:45.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:45.374+0000 7f2466d3a640 1 -- 172.21.15.38:0/3920729613 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2460162280 msgr2=0x7f2460162690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:45.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:45.374+0000 7f2466d3a640 1 --2- 172.21.15.38:0/3920729613 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2460162280 0x7f2460162690 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f24600d0c00 tx=0x7f244c02a4d0 comp rx=0 tx=0).stop 2024-09-21T03:35:45.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:45.374+0000 7f2466d3a640 1 -- 172.21.15.38:0/3920729613 shutdown_connections 2024-09-21T03:35:45.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:45.374+0000 7f2466d3a640 1 --2- 172.21.15.38:0/3920729613 >> v2:172.21.15.195:3300/0 conn(0x7f24600b2e40 0x7f24600b3250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:45.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:45.374+0000 7f2466d3a640 1 --2- 172.21.15.38:0/3920729613 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2460162280 0x7f2460162690 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:45.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:45.374+0000 7f2466d3a640 1 -- 172.21.15.38:0/3920729613 >> 172.21.15.38:0/3920729613 conn(0x7f24600b6b20 msgr2=0x7f24600d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:45.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:45.374+0000 7f2466d3a640 1 -- 172.21.15.38:0/3920729613 shutdown_connections 2024-09-21T03:35:45.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:45.374+0000 7f2466d3a640 1 -- 172.21.15.38:0/3920729613 wait complete. 2024-09-21T03:35:45.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:45.376+0000 7f2466d3a640 1 Processor -- start 2024-09-21T03:35:45.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:45.376+0000 7f2466d3a640 1 -- start start 2024-09-21T03:35:45.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:45.377+0000 7f2466d3a640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f24600b2e40 0x7f24600ce940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:45.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:45.377+0000 7f2466d3a640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2460162280 0x7f24600ad7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:45.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:45.377+0000 7f2466d3a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f24600cee80 0x7f24600add10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:45.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:45.377+0000 7f2466d3a640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f246015e4b0 con 0x7f24600cee80 2024-09-21T03:35:45.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:45.377+0000 7f2466d3a640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f24600b39a0 con 0x7f2460162280 2024-09-21T03:35:45.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:45.377+0000 7f2466d3a640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f246015e630 con 0x7f24600b2e40 2024-09-21T03:35:45.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:45.377+0000 7f24652d7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f24600cee80 0x7f24600add10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:45.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:45.378+0000 7f24652d7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f24600cee80 0x7f24600add10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40074/0 (socket says 172.21.15.38:40074) 2024-09-21T03:35:45.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:45.378+0000 7f24652d7640 1 -- 172.21.15.38:0/2965219193 learned_addr learned my addr 172.21.15.38:0/2965219193 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:35:45.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:45.378+0000 7f245ffff640 1 --2- 172.21.15.38:0/2965219193 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2460162280 0x7f24600ad7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:45.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:45.378+0000 7f2464ad6640 1 --2- 172.21.15.38:0/2965219193 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f24600b2e40 0x7f24600ce940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:45.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:45.378+0000 7f24652d7640 1 -- 172.21.15.38:0/2965219193 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f24600b2e40 msgr2=0x7f24600ce940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:45.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:45.378+0000 7f24652d7640 1 --2- 172.21.15.38:0/2965219193 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f24600b2e40 0x7f24600ce940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:45.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:45.378+0000 7f24652d7640 1 -- 172.21.15.38:0/2965219193 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2460162280 msgr2=0x7f24600ad7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:45.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:45.378+0000 7f24652d7640 1 --2- 172.21.15.38:0/2965219193 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2460162280 0x7f24600ad7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:45.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:45.378+0000 7f24652d7640 1 -- 172.21.15.38:0/2965219193 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24601616b0 con 0x7f24600cee80 2024-09-21T03:35:45.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:45.378+0000 7f24652d7640 1 --2- 172.21.15.38:0/2965219193 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f24600cee80 0x7f24600add10 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f245800ca80 tx=0x7f245800cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:45.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:45.378+0000 7f245d7fa640 1 -- 172.21.15.38:0/2965219193 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2458007d90 con 0x7f24600cee80 2024-09-21T03:35:45.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:45.378+0000 7f245d7fa640 1 -- 172.21.15.38:0/2965219193 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2458004c30 con 0x7f24600cee80 2024-09-21T03:35:45.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:45.378+0000 7f2466d3a640 1 -- 172.21.15.38:0/2965219193 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2460161990 con 0x7f24600cee80 2024-09-21T03:35:45.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:45.378+0000 7f245d7fa640 1 -- 172.21.15.38:0/2965219193 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2458005280 con 0x7f24600cee80 2024-09-21T03:35:45.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:45.378+0000 7f2466d3a640 1 -- 172.21.15.38:0/2965219193 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24600abf70 con 0x7f24600cee80 2024-09-21T03:35:45.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:45.379+0000 7f2466d3a640 1 -- 172.21.15.38:0/2965219193 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2460161420 con 0x7f24600cee80 2024-09-21T03:35:45.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:45.380+0000 7f245d7fa640 1 -- 172.21.15.38:0/2965219193 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f2458005420 con 0x7f24600cee80 2024-09-21T03:35:45.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:45.381+0000 7f245d7fa640 1 --2- 172.21.15.38:0/2965219193 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2444078bc0 0x7f244407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:45.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:45.381+0000 7f245d7fa640 1 -- 172.21.15.38:0/2965219193 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f245809b940 con 0x7f24600cee80 2024-09-21T03:35:45.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:45.381+0000 7f2464ad6640 1 --2- 172.21.15.38:0/2965219193 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2444078bc0 0x7f244407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:45.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:45.382+0000 7f2464ad6640 1 --2- 172.21.15.38:0/2965219193 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2444078bc0 0x7f244407b0d0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f2454006fd0 tx=0x7f2454008040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:45.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:45.386+0000 7f245d7fa640 1 -- 172.21.15.38:0/2965219193 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f2458014070 con 0x7f24600cee80 2024-09-21T03:35:45.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:45.574+0000 7f2466d3a640 1 -- 172.21.15.38:0/2965219193 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f24600b51f0 con 0x7f24600cee80 2024-09-21T03:35:45.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:45.575+0000 7f245d7fa640 1 -- 172.21.15.38:0/2965219193 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) ==== 74+0+16979 (secure 0 0 0) 0x7f2458063bf0 con 0x7f24600cee80 2024-09-21T03:35:45.578 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:35:45.578 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":54,"fsid":"9d50d66f-656b-4fe2-8e3b-fe53b79c4d43","created":"2024-09-21T03:33:10.677865+0000","modified":"2024-09-21T03:35:44.271212+0000","last_up_change":"2024-09-21T03:33:24.023421+0000","last_in_change":"2024-09-21T03:33:14.296193+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T03:33:22.992573+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-09-21T03:35:41.962122+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-09-21T03:35:43.237360+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dbab183f-7e63-48b8-960a-ae55fd5bf565","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6801","nonce":2381639433}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6803","nonce":2381639433}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6807","nonce":2381639433}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6805","nonce":2381639433}]},"public_addr":"172.21.15.38:6801/2381639433","cluster_addr":"172.21.15.38:6803/2381639433","heartbeat_back_addr":"172.21.15.38:6807/2381639433","heartbeat_front_addr":"172.21.15.38:6805/2381639433","state":["exists","up"]},{"osd":1,"uuid":"fa1c0e8a-ebc1-4359-8146-bd5342abbc16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6817","nonce":1491818732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6819","nonce":1491818732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6823","nonce":1491818732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6821","nonce":1491818732}]},"public_addr":"172.21.15.38:6817/1491818732","cluster_addr":"172.21.15.38:6819/1491818732","heartbeat_back_addr":"172.21.15.38:6823/1491818732","heartbeat_front_addr":"172.21.15.38:6821/1491818732","state":["exists","up"]},{"osd":2,"uuid":"2ed3cd2d-385d-4fc6-9869-214552a84803","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6825","nonce":2577858902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6827","nonce":2577858902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6831","nonce":2577858902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6829","nonce":2577858902}]},"public_addr":"172.21.15.38:6825/2577858902","cluster_addr":"172.21.15.38:6827/2577858902","heartbeat_back_addr":"172.21.15.38:6831/2577858902","heartbeat_front_addr":"172.21.15.38:6829/2577858902","state":["exists","up"]},{"osd":3,"uuid":"62d1d95e-919d-4250-be71-1d34cfa70dbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6809","nonce":1286810294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6811","nonce":1286810294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6815","nonce":1286810294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6813","nonce":1286810294}]},"public_addr":"172.21.15.38:6809/1286810294","cluster_addr":"172.21.15.38:6811/1286810294","heartbeat_back_addr":"172.21.15.38:6815/1286810294","heartbeat_front_addr":"172.21.15.38:6813/1286810294","state":["exists","up"]},{"osd":4,"uuid":"7c9f9314-99d7-4d73-8392-37809a9f54b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6803","nonce":3401837571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6805","nonce":3401837571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6809","nonce":3401837571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6807","nonce":3401837571}]},"public_addr":"172.21.15.195:6803/3401837571","cluster_addr":"172.21.15.195:6805/3401837571","heartbeat_back_addr":"172.21.15.195:6809/3401837571","heartbeat_front_addr":"172.21.15.195:6807/3401837571","state":["exists","up"]},{"osd":5,"uuid":"bde4fe9f-8414-4bba-bf4a-43973d1976bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6811","nonce":2080960180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6813","nonce":2080960180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6817","nonce":2080960180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6815","nonce":2080960180}]},"public_addr":"172.21.15.195:6811/2080960180","cluster_addr":"172.21.15.195:6813/2080960180","heartbeat_back_addr":"172.21.15.195:6817/2080960180","heartbeat_front_addr":"172.21.15.195:6815/2080960180","state":["exists","up"]},{"osd":6,"uuid":"6ee6d7f4-1a5f-4a4c-ac16-48fe05f6a002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6827","nonce":4055825675}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6829","nonce":4055825675}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6833","nonce":4055825675}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6831","nonce":4055825675}]},"public_addr":"172.21.15.195:6827/4055825675","cluster_addr":"172.21.15.195:6829/4055825675","heartbeat_back_addr":"172.21.15.195:6833/4055825675","heartbeat_front_addr":"172.21.15.195:6831/4055825675","state":["exists","up"]},{"osd":7,"uuid":"efeb5f51-329a-4484-8290-92e7d3503d05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6819","nonce":1606106342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6821","nonce":1606106342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6825","nonce":1606106342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6823","nonce":1606106342}]},"public_addr":"172.21.15.195:6819/1606106342","cluster_addr":"172.21.15.195:6821/1606106342","heartbeat_back_addr":"172.21.15.195:6825/1606106342","heartbeat_front_addr":"172.21.15.195:6823/1606106342","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.901343+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:21.115003+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.901623+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.835238+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.132199+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:19.817560+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.829690+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:21.087300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T03:35:45.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:45.577+0000 7f2466d3a640 1 -- 172.21.15.38:0/2965219193 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2444078bc0 msgr2=0x7f244407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:45.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:45.577+0000 7f2466d3a640 1 --2- 172.21.15.38:0/2965219193 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2444078bc0 0x7f244407b0d0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f2454006fd0 tx=0x7f2454008040 comp rx=0 tx=0).stop 2024-09-21T03:35:45.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:45.577+0000 7f2466d3a640 1 -- 172.21.15.38:0/2965219193 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f24600cee80 msgr2=0x7f24600add10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:45.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:45.577+0000 7f2466d3a640 1 --2- 172.21.15.38:0/2965219193 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f24600cee80 0x7f24600add10 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f245800ca80 tx=0x7f245800cf40 comp rx=0 tx=0).stop 2024-09-21T03:35:45.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:45.577+0000 7f2466d3a640 1 -- 172.21.15.38:0/2965219193 shutdown_connections 2024-09-21T03:35:45.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:45.577+0000 7f2466d3a640 1 --2- 172.21.15.38:0/2965219193 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f24600cee80 0x7f24600add10 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:45.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:45.577+0000 7f2466d3a640 1 --2- 172.21.15.38:0/2965219193 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2460162280 0x7f24600ad7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:45.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:45.577+0000 7f2466d3a640 1 --2- 172.21.15.38:0/2965219193 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2444078bc0 0x7f244407b0d0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:45.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:45.577+0000 7f2466d3a640 1 --2- 172.21.15.38:0/2965219193 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f24600b2e40 0x7f24600ce940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:45.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:45.577+0000 7f2466d3a640 1 -- 172.21.15.38:0/2965219193 >> 172.21.15.38:0/2965219193 conn(0x7f24600b6b20 msgr2=0x7f246006da30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:45.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:45.577+0000 7f2466d3a640 1 -- 172.21.15.38:0/2965219193 shutdown_connections 2024-09-21T03:35:45.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:45.577+0000 7f2466d3a640 1 -- 172.21.15.38:0/2965219193 wait complete. 2024-09-21T03:35:45.593 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-21T03:35:45.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:45.711+0000 7f58e9a38640 1 -- 172.21.15.38:0/2597041658 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f58cc004d40 msgr2=0x7f58cc005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:45.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:45.711+0000 7f58e9a38640 1 --2- 172.21.15.38:0/2597041658 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f58cc004d40 0x7f58cc005150 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f58dc009ed0 tx=0x7f58cc005830 comp rx=0 tx=0).stop 2024-09-21T03:35:45.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:45.711+0000 7f58e37fe640 1 -- 172.21.15.38:0/2597041658 reap_dead start 2024-09-21T03:35:45.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:45.711+0000 7f58e9a38640 1 -- 172.21.15.38:0/2597041658 shutdown_connections 2024-09-21T03:35:45.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:45.711+0000 7f58e9a38640 1 -- 172.21.15.38:0/2597041658 >> 172.21.15.38:0/2597041658 conn(0x7f58e409d830 msgr2=0x7f58e40b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:45.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:45.711+0000 7f58e9a38640 1 -- 172.21.15.38:0/2597041658 shutdown_connections 2024-09-21T03:35:45.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:45.711+0000 7f58e9a38640 1 -- 172.21.15.38:0/2597041658 wait complete. 2024-09-21T03:35:45.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:45.713+0000 7f58e9a38640 1 Processor -- start 2024-09-21T03:35:45.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:45.713+0000 7f58e9a38640 1 -- start start 2024-09-21T03:35:45.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:45.714+0000 7f58e9a38640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f58cc004d40 0x7f58e4158310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:45.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:45.714+0000 7f58e9a38640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f58e409e2a0 0x7f58e406efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:45.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:45.714+0000 7f58e9a38640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f58e409e6b0 0x7f58e406f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:45.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:45.714+0000 7f58e9a38640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f58e40cefa0 con 0x7f58e409e6b0 2024-09-21T03:35:45.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:45.714+0000 7f58e9a38640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f58e40ba0f0 con 0x7f58e409e2a0 2024-09-21T03:35:45.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:45.714+0000 7f58e9a38640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f58e40cf120 con 0x7f58cc004d40 2024-09-21T03:35:45.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:45.715+0000 7f58e37fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f58e409e6b0 0x7f58e406f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:45.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:45.715+0000 7f58e37fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f58e409e6b0 0x7f58e406f4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40084/0 (socket says 172.21.15.38:40084) 2024-09-21T03:35:45.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:45.715+0000 7f58e37fe640 1 -- 172.21.15.38:0/1031215909 learned_addr learned my addr 172.21.15.38:0/1031215909 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:35:45.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:45.715+0000 7f58e2ffd640 1 --2- 172.21.15.38:0/1031215909 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f58cc004d40 0x7f58e4158310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:45.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:45.715+0000 7f58e27fc640 1 --2- 172.21.15.38:0/1031215909 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f58e409e2a0 0x7f58e406efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:45.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:45.715+0000 7f58e37fe640 1 -- 172.21.15.38:0/1031215909 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f58cc004d40 msgr2=0x7f58e4158310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:45.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:45.715+0000 7f58e37fe640 1 --2- 172.21.15.38:0/1031215909 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f58cc004d40 0x7f58e4158310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:45.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:45.715+0000 7f58e37fe640 1 -- 172.21.15.38:0/1031215909 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f58e409e2a0 msgr2=0x7f58e406efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:45.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:45.715+0000 7f58e37fe640 1 --2- 172.21.15.38:0/1031215909 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f58e409e2a0 0x7f58e406efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:45.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:45.715+0000 7f58e37fe640 1 -- 172.21.15.38:0/1031215909 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58e41f5400 con 0x7f58e409e6b0 2024-09-21T03:35:45.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:45.715+0000 7f58e27fc640 1 --2- 172.21.15.38:0/1031215909 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f58e409e2a0 0x7f58e406efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:35:45.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:45.715+0000 7f58e37fe640 1 --2- 172.21.15.38:0/1031215909 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f58e409e6b0 0x7f58e406f4f0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f58dc036040 tx=0x7f58dc033bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:45.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:45.715+0000 7f58cbfff640 1 -- 172.21.15.38:0/1031215909 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58dc042070 con 0x7f58e409e6b0 2024-09-21T03:35:45.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:45.715+0000 7f58cbfff640 1 -- 172.21.15.38:0/1031215909 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f58dc02bc40 con 0x7f58e409e6b0 2024-09-21T03:35:45.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:45.715+0000 7f58cbfff640 1 -- 172.21.15.38:0/1031215909 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58dc038040 con 0x7f58e409e6b0 2024-09-21T03:35:45.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:45.715+0000 7f58e9a38640 1 -- 172.21.15.38:0/1031215909 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58e41f56e0 con 0x7f58e409e6b0 2024-09-21T03:35:45.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:45.716+0000 7f58e9a38640 1 -- 172.21.15.38:0/1031215909 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58e41f5ca0 con 0x7f58e409e6b0 2024-09-21T03:35:45.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:45.717+0000 7f58e9a38640 1 -- 172.21.15.38:0/1031215909 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58e409fec0 con 0x7f58e409e6b0 2024-09-21T03:35:45.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:45.717+0000 7f58cbfff640 1 -- 172.21.15.38:0/1031215909 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f58dc045070 con 0x7f58e409e6b0 2024-09-21T03:35:45.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:45.718+0000 7f58cbfff640 1 --2- 172.21.15.38:0/1031215909 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f58d0078b90 0x7f58d007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:45.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:45.718+0000 7f58cbfff640 1 -- 172.21.15.38:0/1031215909 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f58dc0baa40 con 0x7f58e409e6b0 2024-09-21T03:35:45.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:45.718+0000 7f58e2ffd640 1 --2- 172.21.15.38:0/1031215909 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f58d0078b90 0x7f58d007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:45.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:45.719+0000 7f58e2ffd640 1 --2- 172.21.15.38:0/1031215909 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f58d0078b90 0x7f58d007b0a0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f58d4009cd0 tx=0x7f58d4005f90 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:45.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:45.723+0000 7f58cbfff640 1 -- 172.21.15.38:0/1031215909 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f58dc0826a0 con 0x7f58e409e6b0 2024-09-21T03:35:45.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:45.951+0000 7f58e9a38640 1 -- 172.21.15.38:0/1031215909 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f58e40d0170 con 0x7f58e409e6b0 2024-09-21T03:35:46.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:46.910+0000 7f58cbfff640 1 -- 172.21.15.38:0/1031215909 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v29) ==== 118+0+0 (secure 0 0 0) 0x7f58dc0bae20 con 0x7f58e409e6b0 2024-09-21T03:35:46.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:46.915+0000 7f58e9a38640 1 -- 172.21.15.38:0/1031215909 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f58d0078b90 msgr2=0x7f58d007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:46.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:46.915+0000 7f58e9a38640 1 --2- 172.21.15.38:0/1031215909 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f58d0078b90 0x7f58d007b0a0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f58d4009cd0 tx=0x7f58d4005f90 comp rx=0 tx=0).stop 2024-09-21T03:35:46.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:46.915+0000 7f58e9a38640 1 -- 172.21.15.38:0/1031215909 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f58e409e6b0 msgr2=0x7f58e406f4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:46.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:46.915+0000 7f58e9a38640 1 --2- 172.21.15.38:0/1031215909 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f58e409e6b0 0x7f58e406f4f0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f58dc036040 tx=0x7f58dc033bf0 comp rx=0 tx=0).stop 2024-09-21T03:35:46.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:46.915+0000 7f58e9a38640 1 -- 172.21.15.38:0/1031215909 shutdown_connections 2024-09-21T03:35:46.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:46.915+0000 7f58e9a38640 1 --2- 172.21.15.38:0/1031215909 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f58e409e6b0 0x7f58e406f4f0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:46.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:46.915+0000 7f58e9a38640 1 --2- 172.21.15.38:0/1031215909 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f58e409e2a0 0x7f58e406efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:46.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:46.915+0000 7f58e9a38640 1 --2- 172.21.15.38:0/1031215909 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f58d0078b90 0x7f58d007b0a0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:46.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:46.915+0000 7f58e9a38640 1 --2- 172.21.15.38:0/1031215909 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f58cc004d40 0x7f58e4158310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:46.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:46.915+0000 7f58e9a38640 1 -- 172.21.15.38:0/1031215909 >> 172.21.15.38:0/1031215909 conn(0x7f58e409d830 msgr2=0x7f58e40b31a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:46.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:46.915+0000 7f58e9a38640 1 -- 172.21.15.38:0/1031215909 shutdown_connections 2024-09-21T03:35:46.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:46.915+0000 7f58e9a38640 1 -- 172.21.15.38:0/1031215909 wait complete. 2024-09-21T03:35:46.931 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-21T03:35:46.931 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-21T03:35:47.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:47.056+0000 7fc6b2390640 1 -- 172.21.15.38:0/4122485280 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc6940053a0 msgr2=0x7fc69400ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:47.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:47.056+0000 7fc6b2390640 1 --2- 172.21.15.38:0/4122485280 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc6940053a0 0x7fc69400ba50 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fc6a4001d90 tx=0x7fc694005830 comp rx=0 tx=0).stop 2024-09-21T03:35:47.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:47.056+0000 7fc6b2390640 1 -- 172.21.15.38:0/4122485280 shutdown_connections 2024-09-21T03:35:47.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:47.056+0000 7fc6b2390640 1 --2- 172.21.15.38:0/4122485280 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc6940053a0 0x7fc69400ba50 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:47.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:47.056+0000 7fc6b2390640 1 -- 172.21.15.38:0/4122485280 >> 172.21.15.38:0/4122485280 conn(0x7fc6ac09d830 msgr2=0x7fc6ac0b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:47.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:47.056+0000 7fc6b2390640 1 -- 172.21.15.38:0/4122485280 shutdown_connections 2024-09-21T03:35:47.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:47.056+0000 7fc6b2390640 1 -- 172.21.15.38:0/4122485280 wait complete. 2024-09-21T03:35:47.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:47.058+0000 7fc6b2390640 1 Processor -- start 2024-09-21T03:35:47.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:47.058+0000 7fc6b2390640 1 -- start start 2024-09-21T03:35:47.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:47.059+0000 7fc6b2390640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc6940053a0 0x7fc6ac0caea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:47.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:47.059+0000 7fc6b2390640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc6ac15b270 0x7fc6ac06f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:47.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:47.059+0000 7fc6b2390640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc6ac06f5e0 0x7fc6ac159b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:47.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:47.059+0000 7fc6b2390640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fc6ac0cf120 con 0x7fc6ac15b270 2024-09-21T03:35:47.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:47.059+0000 7fc6b2390640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fc6ac0cefa0 con 0x7fc6ac06f5e0 2024-09-21T03:35:47.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:47.059+0000 7fc6b2390640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fc6ac0cf2a0 con 0x7fc6940053a0 2024-09-21T03:35:47.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:47.059+0000 7fc6ab7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc6ac15b270 0x7fc6ac06f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:47.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:47.059+0000 7fc6ab7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc6ac15b270 0x7fc6ac06f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40096/0 (socket says 172.21.15.38:40096) 2024-09-21T03:35:47.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:47.059+0000 7fc6ab7fe640 1 -- 172.21.15.38:0/4090101514 learned_addr learned my addr 172.21.15.38:0/4090101514 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:35:47.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:47.060+0000 7fc6b092d640 1 --2- 172.21.15.38:0/4090101514 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc6ac06f5e0 0x7fc6ac159b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:47.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:47.060+0000 7fc6abfff640 1 --2- 172.21.15.38:0/4090101514 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc6940053a0 0x7fc6ac0caea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:47.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:47.060+0000 7fc6ab7fe640 1 -- 172.21.15.38:0/4090101514 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc6940053a0 msgr2=0x7fc6ac0caea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:47.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:47.060+0000 7fc6ab7fe640 1 --2- 172.21.15.38:0/4090101514 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc6940053a0 0x7fc6ac0caea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:47.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:47.060+0000 7fc6ab7fe640 1 -- 172.21.15.38:0/4090101514 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc6ac06f5e0 msgr2=0x7fc6ac159b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:47.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:47.060+0000 7fc6ab7fe640 1 --2- 172.21.15.38:0/4090101514 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc6ac06f5e0 0x7fc6ac159b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:47.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:47.060+0000 7fc6ab7fe640 1 -- 172.21.15.38:0/4090101514 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6ac0cb3e0 con 0x7fc6ac15b270 2024-09-21T03:35:47.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:47.060+0000 7fc6ab7fe640 1 --2- 172.21.15.38:0/4090101514 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc6ac15b270 0x7fc6ac06f0a0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fc6a0006450 tx=0x7fc6a00039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:47.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:47.060+0000 7fc6a8ff9640 1 -- 172.21.15.38:0/4090101514 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6a0015070 con 0x7fc6ac15b270 2024-09-21T03:35:47.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:47.060+0000 7fc6a8ff9640 1 -- 172.21.15.38:0/4090101514 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc6a0011840 con 0x7fc6ac15b270 2024-09-21T03:35:47.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:47.060+0000 7fc6b2390640 1 -- 172.21.15.38:0/4090101514 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6ac06f9f0 con 0x7fc6ac15b270 2024-09-21T03:35:47.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:47.060+0000 7fc6a8ff9640 1 -- 172.21.15.38:0/4090101514 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6a0011c70 con 0x7fc6ac15b270 2024-09-21T03:35:47.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:47.060+0000 7fc6b2390640 1 -- 172.21.15.38:0/4090101514 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6ac1f60a0 con 0x7fc6ac15b270 2024-09-21T03:35:47.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:47.061+0000 7fc6b2390640 1 -- 172.21.15.38:0/4090101514 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6ac0d0c30 con 0x7fc6ac15b270 2024-09-21T03:35:47.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:47.062+0000 7fc6a8ff9640 1 -- 172.21.15.38:0/4090101514 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc6a0018070 con 0x7fc6ac15b270 2024-09-21T03:35:47.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:47.062+0000 7fc6a8ff9640 1 --2- 172.21.15.38:0/4090101514 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc698078b90 0x7fc69807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:47.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:47.063+0000 7fc6a8ff9640 1 -- 172.21.15.38:0/4090101514 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 6101+0+0 (secure 0 0 0) 0x7fc6a009bc40 con 0x7fc6ac15b270 2024-09-21T03:35:47.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:47.063+0000 7fc6abfff640 1 --2- 172.21.15.38:0/4090101514 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc698078b90 0x7fc69807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:47.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:47.063+0000 7fc6abfff640 1 --2- 172.21.15.38:0/4090101514 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc698078b90 0x7fc69807b0a0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fc69c009a10 tx=0x7fc69c005f90 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:47.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:47.068+0000 7fc6a8ff9640 1 -- 172.21.15.38:0/4090101514 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fc6ac06f9f0 con 0x7fc6ac15b270 2024-09-21T03:35:47.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:47.292+0000 7fc6b2390640 1 -- 172.21.15.38:0/4090101514 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fc6ac1609a0 con 0x7fc6ac15b270 2024-09-21T03:35:47.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:47.918+0000 7fc6a8ff9640 1 -- 172.21.15.38:0/4090101514 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v30) ==== 122+0+0 (secure 0 0 0) 0x7fc6ac0cb3e0 con 0x7fc6ac15b270 2024-09-21T03:35:47.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:47.920+0000 7fc6b2390640 1 -- 172.21.15.38:0/4090101514 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc698078b90 msgr2=0x7fc69807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:47.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:47.920+0000 7fc6b2390640 1 --2- 172.21.15.38:0/4090101514 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc698078b90 0x7fc69807b0a0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fc69c009a10 tx=0x7fc69c005f90 comp rx=0 tx=0).stop 2024-09-21T03:35:47.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:47.920+0000 7fc6b2390640 1 -- 172.21.15.38:0/4090101514 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc6ac15b270 msgr2=0x7fc6ac06f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:47.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:47.920+0000 7fc6b2390640 1 --2- 172.21.15.38:0/4090101514 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc6ac15b270 0x7fc6ac06f0a0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fc6a0006450 tx=0x7fc6a00039c0 comp rx=0 tx=0).stop 2024-09-21T03:35:47.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:47.920+0000 7fc6b2390640 1 -- 172.21.15.38:0/4090101514 shutdown_connections 2024-09-21T03:35:47.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:47.920+0000 7fc6b2390640 1 --2- 172.21.15.38:0/4090101514 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc6ac06f5e0 0x7fc6ac159b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:47.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:47.920+0000 7fc6b2390640 1 --2- 172.21.15.38:0/4090101514 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc6ac15b270 0x7fc6ac06f0a0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:47.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:47.920+0000 7fc6b2390640 1 --2- 172.21.15.38:0/4090101514 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc698078b90 0x7fc69807b0a0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:47.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:47.921+0000 7fc6b2390640 1 --2- 172.21.15.38:0/4090101514 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc6940053a0 0x7fc6ac0caea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:47.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:47.921+0000 7fc6b2390640 1 -- 172.21.15.38:0/4090101514 >> 172.21.15.38:0/4090101514 conn(0x7fc6ac09d830 msgr2=0x7fc6ac161820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:47.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:47.921+0000 7fc6b2390640 1 -- 172.21.15.38:0/4090101514 shutdown_connections 2024-09-21T03:35:47.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:47.921+0000 7fc6b2390640 1 -- 172.21.15.38:0/4090101514 wait complete. 2024-09-21T03:35:47.936 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-21T03:35:47.936 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:35:48.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:48.058+0000 7f2292ccc640 1 -- 172.21.15.38:0/2775748196 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2274004800 msgr2=0x7f2274004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:48.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:48.058+0000 7f2292ccc640 1 --2- 172.21.15.38:0/2775748196 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2274004800 0x7f2274004c10 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f228000c290 tx=0x7f2274005830 comp rx=0 tx=0).stop 2024-09-21T03:35:48.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:48.058+0000 7f2291269640 1 -- 172.21.15.38:0/2775748196 reap_dead start 2024-09-21T03:35:48.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:48.058+0000 7f2292ccc640 1 -- 172.21.15.38:0/2775748196 shutdown_connections 2024-09-21T03:35:48.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:48.058+0000 7f2292ccc640 1 -- 172.21.15.38:0/2775748196 >> 172.21.15.38:0/2775748196 conn(0x7f228c0b6b20 msgr2=0x7f228c0d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:48.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:48.059+0000 7f2292ccc640 1 -- 172.21.15.38:0/2775748196 shutdown_connections 2024-09-21T03:35:48.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:48.059+0000 7f2292ccc640 1 -- 172.21.15.38:0/2775748196 wait complete. 2024-09-21T03:35:48.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:48.060+0000 7f2292ccc640 1 Processor -- start 2024-09-21T03:35:48.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:48.060+0000 7f2292ccc640 1 -- start start 2024-09-21T03:35:48.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:48.062+0000 7f2292ccc640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2274004800 0x7f228c0ca9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:48.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:48.062+0000 7f2292ccc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f228c1f92e0 0x7f228c0cd680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:48.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:48.062+0000 7f2292ccc640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f228c0cbac0 0x7f228c1f5b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:48.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:48.062+0000 7f2292ccc640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f228c163170 con 0x7f228c1f92e0 2024-09-21T03:35:48.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:48.062+0000 7f2292ccc640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f228c0b39a0 con 0x7f2274004800 2024-09-21T03:35:48.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:48.062+0000 7f2292ccc640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f228c0d24d0 con 0x7f228c0cbac0 2024-09-21T03:35:48.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:48.062+0000 7f228bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f228c1f92e0 0x7f228c0cd680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:48.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:48.062+0000 7f228bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f228c1f92e0 0x7f228c0cd680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40112/0 (socket says 172.21.15.38:40112) 2024-09-21T03:35:48.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:48.062+0000 7f228bfff640 1 -- 172.21.15.38:0/2736522980 learned_addr learned my addr 172.21.15.38:0/2736522980 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:35:48.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:48.062+0000 7f2290a68640 1 --2- 172.21.15.38:0/2736522980 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2274004800 0x7f228c0ca9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:48.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:48.062+0000 7f2291269640 1 --2- 172.21.15.38:0/2736522980 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f228c0cbac0 0x7f228c1f5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:48.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:48.062+0000 7f228bfff640 1 -- 172.21.15.38:0/2736522980 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f228c0cbac0 msgr2=0x7f228c1f5b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:48.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:48.062+0000 7f228bfff640 1 --2- 172.21.15.38:0/2736522980 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f228c0cbac0 0x7f228c1f5b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:48.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:48.062+0000 7f228bfff640 1 -- 172.21.15.38:0/2736522980 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2274004800 msgr2=0x7f228c0ca9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:48.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:48.062+0000 7f228bfff640 1 --2- 172.21.15.38:0/2736522980 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2274004800 0x7f228c0ca9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:48.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:48.062+0000 7f228bfff640 1 -- 172.21.15.38:0/2736522980 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f228c1f53f0 con 0x7f228c1f92e0 2024-09-21T03:35:48.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:48.063+0000 7f228bfff640 1 --2- 172.21.15.38:0/2736522980 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f228c1f92e0 0x7f228c0cd680 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f228000c380 tx=0x7f2280006c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:48.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:48.063+0000 7f22897fa640 1 -- 172.21.15.38:0/2736522980 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2280031040 con 0x7f228c1f92e0 2024-09-21T03:35:48.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:48.063+0000 7f22897fa640 1 -- 172.21.15.38:0/2736522980 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f22800035f0 con 0x7f228c1f92e0 2024-09-21T03:35:48.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:48.063+0000 7f22897fa640 1 -- 172.21.15.38:0/2736522980 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2280003a40 con 0x7f228c1f92e0 2024-09-21T03:35:48.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:48.063+0000 7f2292ccc640 1 -- 172.21.15.38:0/2736522980 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f228c0cb150 con 0x7f228c1f92e0 2024-09-21T03:35:48.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:48.063+0000 7f2292ccc640 1 -- 172.21.15.38:0/2736522980 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f228c0cb430 con 0x7f228c1f92e0 2024-09-21T03:35:48.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:48.064+0000 7f2292ccc640 1 -- 172.21.15.38:0/2736522980 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f228c0d2a30 con 0x7f228c1f92e0 2024-09-21T03:35:48.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:48.065+0000 7f22897fa640 1 -- 172.21.15.38:0/2736522980 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f2280034070 con 0x7f228c1f92e0 2024-09-21T03:35:48.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:48.065+0000 7f22897fa640 1 --2- 172.21.15.38:0/2736522980 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2278078b90 0x7f227807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:48.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:48.065+0000 7f22897fa640 1 -- 172.21.15.38:0/2736522980 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 6101+0+0 (secure 0 0 0) 0x7f22800bac90 con 0x7f228c1f92e0 2024-09-21T03:35:48.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:48.065+0000 7f2290a68640 1 --2- 172.21.15.38:0/2736522980 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2278078b90 0x7f227807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:48.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:48.066+0000 7f2290a68640 1 --2- 172.21.15.38:0/2736522980 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2278078b90 0x7f227807b0a0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f227c009a30 tx=0x7f227c005f90 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:48.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:48.070+0000 7f22897fa640 1 -- 172.21.15.38:0/2736522980 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f22800829d0 con 0x7f228c1f92e0 2024-09-21T03:35:48.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:48.293+0000 7f2292ccc640 1 -- 172.21.15.38:0/2736522980 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f228c1f97e0 con 0x7f228c1f92e0 2024-09-21T03:35:48.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:48.293+0000 7f22897fa640 1 -- 172.21.15.38:0/2736522980 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+5229 (secure 0 0 0) 0x7f2280087400 con 0x7f228c1f92e0 2024-09-21T03:35:48.295 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:35:48.296 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":30,"btime":"2024-09-21T03:35:47:913013+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5147,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/2858412699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":2858412699},{"type":"v1","addr":"172.21.15.195:6837","nonce":2858412699}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":5368,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/2907144575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":2907144575},{"type":"v1","addr":"172.21.15.195:6836","nonce":2907144575}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":5574,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6834/1911661184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":1911661184},{"type":"v1","addr":"172.21.15.38:6834","nonce":1911661184}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:35:44.271168+0000","modified":"2024-09-21T03:35:47.294703+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5577},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5577":{"gid":5577,"name":"c","rank":0,"incarnation":27,"state":"up:active","state_seq":3,"addr":"172.21.15.38:6835/3184989541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6833","nonce":3184989541},{"type":"v1","addr":"172.21.15.38:6835","nonce":3184989541}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5577,"qdb_cluster":[5577]},"id":3}]} 2024-09-21T03:35:48.296 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 30 2024-09-21T03:35:48.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:48.296+0000 7f2292ccc640 1 -- 172.21.15.38:0/2736522980 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2278078b90 msgr2=0x7f227807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:48.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:48.296+0000 7f2292ccc640 1 --2- 172.21.15.38:0/2736522980 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2278078b90 0x7f227807b0a0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f227c009a30 tx=0x7f227c005f90 comp rx=0 tx=0).stop 2024-09-21T03:35:48.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:48.296+0000 7f2292ccc640 1 -- 172.21.15.38:0/2736522980 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f228c1f92e0 msgr2=0x7f228c0cd680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:48.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:48.296+0000 7f2292ccc640 1 --2- 172.21.15.38:0/2736522980 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f228c1f92e0 0x7f228c0cd680 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f228000c380 tx=0x7f2280006c50 comp rx=0 tx=0).stop 2024-09-21T03:35:48.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:48.296+0000 7f2292ccc640 1 -- 172.21.15.38:0/2736522980 shutdown_connections 2024-09-21T03:35:48.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:48.296+0000 7f2292ccc640 1 --2- 172.21.15.38:0/2736522980 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2278078b90 0x7f227807b0a0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:48.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:48.296+0000 7f2292ccc640 1 --2- 172.21.15.38:0/2736522980 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f228c0cbac0 0x7f228c1f5b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:48.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:48.296+0000 7f2292ccc640 1 --2- 172.21.15.38:0/2736522980 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f228c1f92e0 0x7f228c0cd680 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:48.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:48.296+0000 7f2292ccc640 1 --2- 172.21.15.38:0/2736522980 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2274004800 0x7f228c0ca9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:48.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:48.296+0000 7f2292ccc640 1 -- 172.21.15.38:0/2736522980 >> 172.21.15.38:0/2736522980 conn(0x7f228c0b6b20 msgr2=0x7f228c06cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:48.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:48.296+0000 7f2292ccc640 1 -- 172.21.15.38:0/2736522980 shutdown_connections 2024-09-21T03:35:48.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:48.296+0000 7f2292ccc640 1 -- 172.21.15.38:0/2736522980 wait complete. 2024-09-21T03:35:48.309 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T03:35:48.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:48.425+0000 7f0d66532640 1 -- 172.21.15.38:0/977503467 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0d48004d40 msgr2=0x7f0d48005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:48.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:48.425+0000 7f0d66532640 1 --2- 172.21.15.38:0/977503467 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0d48004d40 0x7f0d48005150 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f0d5800c750 tx=0x7f0d48005830 comp rx=0 tx=0).stop 2024-09-21T03:35:48.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:48.425+0000 7f0d64acf640 1 -- 172.21.15.38:0/977503467 reap_dead start 2024-09-21T03:35:48.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:48.425+0000 7f0d66532640 1 -- 172.21.15.38:0/977503467 shutdown_connections 2024-09-21T03:35:48.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:48.425+0000 7f0d66532640 1 -- 172.21.15.38:0/977503467 >> 172.21.15.38:0/977503467 conn(0x7f0d600d4440 msgr2=0x7f0d600d4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:48.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:48.425+0000 7f0d66532640 1 -- 172.21.15.38:0/977503467 shutdown_connections 2024-09-21T03:35:48.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:48.425+0000 7f0d66532640 1 -- 172.21.15.38:0/977503467 wait complete. 2024-09-21T03:35:48.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:48.427+0000 7f0d66532640 1 Processor -- start 2024-09-21T03:35:48.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:48.427+0000 7f0d66532640 1 -- start start 2024-09-21T03:35:48.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:48.428+0000 7f0d66532640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0d48004d40 0x7f0d60156010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:48.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:48.428+0000 7f0d66532640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0d6015c180 0x7f0d601f88b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:48.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:48.428+0000 7f0d66532640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0d601f8df0 0x7f0d601f9200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:48.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:48.428+0000 7f0d66532640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f0d600d2a20 con 0x7f0d48004d40 2024-09-21T03:35:48.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:48.428+0000 7f0d66532640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f0d600b3b90 con 0x7f0d601f8df0 2024-09-21T03:35:48.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:48.428+0000 7f0d66532640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f0d600d2ba0 con 0x7f0d6015c180 2024-09-21T03:35:48.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:48.429+0000 7f0d5ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0d48004d40 0x7f0d60156010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:48.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:48.429+0000 7f0d5ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0d48004d40 0x7f0d60156010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40124/0 (socket says 172.21.15.38:40124) 2024-09-21T03:35:48.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:48.429+0000 7f0d5ffff640 1 -- 172.21.15.38:0/2949980493 learned_addr learned my addr 172.21.15.38:0/2949980493 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:35:48.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:48.429+0000 7f0d5f7fe640 1 --2- 172.21.15.38:0/2949980493 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0d6015c180 0x7f0d601f88b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:48.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:48.429+0000 7f0d64acf640 1 --2- 172.21.15.38:0/2949980493 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0d601f8df0 0x7f0d601f9200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:48.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:48.429+0000 7f0d5ffff640 1 -- 172.21.15.38:0/2949980493 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0d6015c180 msgr2=0x7f0d601f88b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:48.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:48.429+0000 7f0d5ffff640 1 --2- 172.21.15.38:0/2949980493 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0d6015c180 0x7f0d601f88b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:48.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:48.429+0000 7f0d5ffff640 1 -- 172.21.15.38:0/2949980493 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0d601f8df0 msgr2=0x7f0d601f9200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:48.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:48.429+0000 7f0d5ffff640 1 --2- 172.21.15.38:0/2949980493 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0d601f8df0 0x7f0d601f9200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:48.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:48.429+0000 7f0d5ffff640 1 -- 172.21.15.38:0/2949980493 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d600c9cc0 con 0x7f0d48004d40 2024-09-21T03:35:48.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:48.429+0000 7f0d5ffff640 1 --2- 172.21.15.38:0/2949980493 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0d48004d40 0x7f0d60156010 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f0d50011ab0 tx=0x7f0d50011eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:48.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:48.429+0000 7f0d5cff9640 1 -- 172.21.15.38:0/2949980493 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d5001e070 con 0x7f0d48004d40 2024-09-21T03:35:48.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:48.429+0000 7f0d5cff9640 1 -- 172.21.15.38:0/2949980493 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0d50004460 con 0x7f0d48004d40 2024-09-21T03:35:48.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:48.429+0000 7f0d66532640 1 -- 172.21.15.38:0/2949980493 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d60156550 con 0x7f0d48004d40 2024-09-21T03:35:48.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:48.429+0000 7f0d5cff9640 1 -- 172.21.15.38:0/2949980493 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d5000fb80 con 0x7f0d48004d40 2024-09-21T03:35:48.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:48.429+0000 7f0d66532640 1 -- 172.21.15.38:0/2949980493 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d6015b680 con 0x7f0d48004d40 2024-09-21T03:35:48.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:48.430+0000 7f0d66532640 1 -- 172.21.15.38:0/2949980493 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d600d30a0 con 0x7f0d48004d40 2024-09-21T03:35:48.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:48.431+0000 7f0d5cff9640 1 -- 172.21.15.38:0/2949980493 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f0d50021070 con 0x7f0d48004d40 2024-09-21T03:35:48.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:48.432+0000 7f0d5cff9640 1 --2- 172.21.15.38:0/2949980493 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0d4c078b90 0x7f0d4c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:48.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:48.432+0000 7f0d5cff9640 1 -- 172.21.15.38:0/2949980493 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 6101+0+0 (secure 0 0 0) 0x7f0d5009b890 con 0x7f0d48004d40 2024-09-21T03:35:48.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:48.432+0000 7f0d5f7fe640 1 --2- 172.21.15.38:0/2949980493 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0d4c078b90 0x7f0d4c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:48.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:48.433+0000 7f0d5f7fe640 1 --2- 172.21.15.38:0/2949980493 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0d4c078b90 0x7f0d4c07b0a0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f0d600cdda0 tx=0x7f0d54005f90 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:48.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:48.436+0000 7f0d5cff9640 1 -- 172.21.15.38:0/2949980493 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f0d50063450 con 0x7f0d48004d40 2024-09-21T03:35:48.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:48.624+0000 7f0d66532640 1 -- 172.21.15.38:0/2949980493 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0d60076180 con 0x7f0d48004d40 2024-09-21T03:35:48.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:48.625+0000 7f0d5cff9640 1 -- 172.21.15.38:0/2949980493 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v55) ==== 74+0+16966 (secure 0 0 0) 0x7f0d50067e80 con 0x7f0d48004d40 2024-09-21T03:35:48.627 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:35:48.627 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":55,"fsid":"9d50d66f-656b-4fe2-8e3b-fe53b79c4d43","created":"2024-09-21T03:33:10.677865+0000","modified":"2024-09-21T03:35:45.900439+0000","last_up_change":"2024-09-21T03:33:24.023421+0000","last_in_change":"2024-09-21T03:33:14.296193+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T03:33:22.992573+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-09-21T03:35:41.962122+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-09-21T03:35:43.237360+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dbab183f-7e63-48b8-960a-ae55fd5bf565","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6801","nonce":2381639433}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6803","nonce":2381639433}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6807","nonce":2381639433}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6805","nonce":2381639433}]},"public_addr":"172.21.15.38:6801/2381639433","cluster_addr":"172.21.15.38:6803/2381639433","heartbeat_back_addr":"172.21.15.38:6807/2381639433","heartbeat_front_addr":"172.21.15.38:6805/2381639433","state":["exists","up"]},{"osd":1,"uuid":"fa1c0e8a-ebc1-4359-8146-bd5342abbc16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6817","nonce":1491818732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6819","nonce":1491818732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6823","nonce":1491818732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6821","nonce":1491818732}]},"public_addr":"172.21.15.38:6817/1491818732","cluster_addr":"172.21.15.38:6819/1491818732","heartbeat_back_addr":"172.21.15.38:6823/1491818732","heartbeat_front_addr":"172.21.15.38:6821/1491818732","state":["exists","up"]},{"osd":2,"uuid":"2ed3cd2d-385d-4fc6-9869-214552a84803","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6825","nonce":2577858902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6827","nonce":2577858902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6831","nonce":2577858902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6829","nonce":2577858902}]},"public_addr":"172.21.15.38:6825/2577858902","cluster_addr":"172.21.15.38:6827/2577858902","heartbeat_back_addr":"172.21.15.38:6831/2577858902","heartbeat_front_addr":"172.21.15.38:6829/2577858902","state":["exists","up"]},{"osd":3,"uuid":"62d1d95e-919d-4250-be71-1d34cfa70dbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6809","nonce":1286810294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6811","nonce":1286810294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6815","nonce":1286810294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6813","nonce":1286810294}]},"public_addr":"172.21.15.38:6809/1286810294","cluster_addr":"172.21.15.38:6811/1286810294","heartbeat_back_addr":"172.21.15.38:6815/1286810294","heartbeat_front_addr":"172.21.15.38:6813/1286810294","state":["exists","up"]},{"osd":4,"uuid":"7c9f9314-99d7-4d73-8392-37809a9f54b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6803","nonce":3401837571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6805","nonce":3401837571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6809","nonce":3401837571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6807","nonce":3401837571}]},"public_addr":"172.21.15.195:6803/3401837571","cluster_addr":"172.21.15.195:6805/3401837571","heartbeat_back_addr":"172.21.15.195:6809/3401837571","heartbeat_front_addr":"172.21.15.195:6807/3401837571","state":["exists","up"]},{"osd":5,"uuid":"bde4fe9f-8414-4bba-bf4a-43973d1976bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6811","nonce":2080960180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6813","nonce":2080960180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6817","nonce":2080960180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6815","nonce":2080960180}]},"public_addr":"172.21.15.195:6811/2080960180","cluster_addr":"172.21.15.195:6813/2080960180","heartbeat_back_addr":"172.21.15.195:6817/2080960180","heartbeat_front_addr":"172.21.15.195:6815/2080960180","state":["exists","up"]},{"osd":6,"uuid":"6ee6d7f4-1a5f-4a4c-ac16-48fe05f6a002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6827","nonce":4055825675}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6829","nonce":4055825675}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6833","nonce":4055825675}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6831","nonce":4055825675}]},"public_addr":"172.21.15.195:6827/4055825675","cluster_addr":"172.21.15.195:6829/4055825675","heartbeat_back_addr":"172.21.15.195:6833/4055825675","heartbeat_front_addr":"172.21.15.195:6831/4055825675","state":["exists","up"]},{"osd":7,"uuid":"efeb5f51-329a-4484-8290-92e7d3503d05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6819","nonce":1606106342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6821","nonce":1606106342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6825","nonce":1606106342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6823","nonce":1606106342}]},"public_addr":"172.21.15.195:6819/1606106342","cluster_addr":"172.21.15.195:6821/1606106342","heartbeat_back_addr":"172.21.15.195:6825/1606106342","heartbeat_front_addr":"172.21.15.195:6823/1606106342","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.901343+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:21.115003+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.901623+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.835238+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.132199+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:19.817560+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.829690+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:21.087300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T03:35:48.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:48.627+0000 7f0d66532640 1 -- 172.21.15.38:0/2949980493 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0d4c078b90 msgr2=0x7f0d4c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:48.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:48.627+0000 7f0d66532640 1 --2- 172.21.15.38:0/2949980493 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0d4c078b90 0x7f0d4c07b0a0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f0d600cdda0 tx=0x7f0d54005f90 comp rx=0 tx=0).stop 2024-09-21T03:35:48.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:48.627+0000 7f0d66532640 1 -- 172.21.15.38:0/2949980493 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0d48004d40 msgr2=0x7f0d60156010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:48.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:48.627+0000 7f0d66532640 1 --2- 172.21.15.38:0/2949980493 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0d48004d40 0x7f0d60156010 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f0d50011ab0 tx=0x7f0d50011eb0 comp rx=0 tx=0).stop 2024-09-21T03:35:48.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:48.627+0000 7f0d66532640 1 -- 172.21.15.38:0/2949980493 shutdown_connections 2024-09-21T03:35:48.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:48.627+0000 7f0d66532640 1 --2- 172.21.15.38:0/2949980493 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0d601f8df0 0x7f0d601f9200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:48.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:48.627+0000 7f0d66532640 1 --2- 172.21.15.38:0/2949980493 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0d4c078b90 0x7f0d4c07b0a0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:48.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:48.627+0000 7f0d66532640 1 --2- 172.21.15.38:0/2949980493 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0d6015c180 0x7f0d601f88b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:48.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:48.627+0000 7f0d66532640 1 --2- 172.21.15.38:0/2949980493 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0d48004d40 0x7f0d60156010 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:48.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:48.627+0000 7f0d66532640 1 -- 172.21.15.38:0/2949980493 >> 172.21.15.38:0/2949980493 conn(0x7f0d600d4440 msgr2=0x7f0d601619d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:48.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:48.627+0000 7f0d66532640 1 -- 172.21.15.38:0/2949980493 shutdown_connections 2024-09-21T03:35:48.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:48.627+0000 7f0d66532640 1 -- 172.21.15.38:0/2949980493 wait complete. 2024-09-21T03:35:48.641 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-21T03:35:48.641 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T03:35:48.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:48.756+0000 7f0527739640 1 -- 172.21.15.38:0/1269774113 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f05200cfe00 msgr2=0x7f05200d2a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:48.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:48.756+0000 7f0527739640 1 --2- 172.21.15.38:0/1269774113 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f05200cfe00 0x7f05200d2a20 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f0510009830 tx=0x7f051002a510 comp rx=0 tx=0).stop 2024-09-21T03:35:48.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:48.756+0000 7f0527739640 1 -- 172.21.15.38:0/1269774113 shutdown_connections 2024-09-21T03:35:48.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:48.756+0000 7f0527739640 1 --2- 172.21.15.38:0/1269774113 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f05200b2e60 0x7f05200b3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:48.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:48.756+0000 7f0527739640 1 --2- 172.21.15.38:0/1269774113 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f05200cfe00 0x7f05200d2a20 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:48.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:48.756+0000 7f0527739640 1 -- 172.21.15.38:0/1269774113 >> 172.21.15.38:0/1269774113 conn(0x7f05200d4440 msgr2=0x7f05200d4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:48.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:48.756+0000 7f0527739640 1 -- 172.21.15.38:0/1269774113 shutdown_connections 2024-09-21T03:35:48.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:48.756+0000 7f0527739640 1 -- 172.21.15.38:0/1269774113 wait complete. 2024-09-21T03:35:48.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:48.757+0000 7f0527739640 1 Processor -- start 2024-09-21T03:35:48.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:48.758+0000 7f0527739640 1 -- start start 2024-09-21T03:35:48.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:48.759+0000 7f0527739640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f05200b2e60 0x7f05201f6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:48.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:48.759+0000 7f0527739640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f05200cfe00 0x7f05201f2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:48.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:48.759+0000 7f0527739640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0520162590 0x7f05200c8b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:48.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:48.759+0000 7f0527739640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f05200b3930 con 0x7f05200cfe00 2024-09-21T03:35:48.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:48.759+0000 7f0527739640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f05200b37b0 con 0x7f0520162590 2024-09-21T03:35:48.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:48.759+0000 7f0527739640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f05200b3ab0 con 0x7f05200b2e60 2024-09-21T03:35:48.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:48.759+0000 7f0524cd4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f05200cfe00 0x7f05201f2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:48.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:48.759+0000 7f0524cd4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f05200cfe00 0x7f05201f2ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40138/0 (socket says 172.21.15.38:40138) 2024-09-21T03:35:48.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:48.759+0000 7f0524cd4640 1 -- 172.21.15.38:0/3128065457 learned_addr learned my addr 172.21.15.38:0/3128065457 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:35:48.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:48.759+0000 7f0525cd6640 1 --2- 172.21.15.38:0/3128065457 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0520162590 0x7f05200c8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:48.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:48.759+0000 7f05254d5640 1 --2- 172.21.15.38:0/3128065457 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f05200b2e60 0x7f05201f6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:48.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:48.759+0000 7f0524cd4640 1 -- 172.21.15.38:0/3128065457 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f05200b2e60 msgr2=0x7f05201f6160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:48.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:48.759+0000 7f0524cd4640 1 --2- 172.21.15.38:0/3128065457 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f05200b2e60 0x7f05201f6160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:48.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:48.759+0000 7f0524cd4640 1 -- 172.21.15.38:0/3128065457 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0520162590 msgr2=0x7f05200c8b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:48.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:48.759+0000 7f0524cd4640 1 --2- 172.21.15.38:0/3128065457 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0520162590 0x7f05200c8b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:48.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:48.759+0000 7f0524cd4640 1 -- 172.21.15.38:0/3128065457 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f052015d290 con 0x7f05200cfe00 2024-09-21T03:35:48.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:48.759+0000 7f0524cd4640 1 --2- 172.21.15.38:0/3128065457 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f05200cfe00 0x7f05201f2ac0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f051c00c430 tx=0x7f051c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:48.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:48.760+0000 7f0515ffb640 1 -- 172.21.15.38:0/3128065457 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f051c011790 con 0x7f05200cfe00 2024-09-21T03:35:48.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:48.760+0000 7f0527739640 1 -- 172.21.15.38:0/3128065457 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f052015d570 con 0x7f05200cfe00 2024-09-21T03:35:48.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:48.760+0000 7f0515ffb640 1 -- 172.21.15.38:0/3128065457 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f051c017070 con 0x7f05200cfe00 2024-09-21T03:35:48.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:48.760+0000 7f0515ffb640 1 -- 172.21.15.38:0/3128065457 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f051c011ce0 con 0x7f05200cfe00 2024-09-21T03:35:48.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:48.760+0000 7f0527739640 1 -- 172.21.15.38:0/3128065457 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f052015db30 con 0x7f05200cfe00 2024-09-21T03:35:48.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:48.761+0000 7f0527739640 1 -- 172.21.15.38:0/3128065457 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05200b3270 con 0x7f05200cfe00 2024-09-21T03:35:48.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:48.761+0000 7f0515ffb640 1 -- 172.21.15.38:0/3128065457 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f051c01c460 con 0x7f05200cfe00 2024-09-21T03:35:48.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:48.762+0000 7f0515ffb640 1 --2- 172.21.15.38:0/3128065457 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0504078b90 0x7f050407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:48.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:48.762+0000 7f0515ffb640 1 -- 172.21.15.38:0/3128065457 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 6101+0+0 (secure 0 0 0) 0x7f051c09ac40 con 0x7f05200cfe00 2024-09-21T03:35:48.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:48.762+0000 7f05254d5640 1 --2- 172.21.15.38:0/3128065457 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0504078b90 0x7f050407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:48.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:48.763+0000 7f05254d5640 1 --2- 172.21.15.38:0/3128065457 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0504078b90 0x7f050407b0a0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f0510034000 tx=0x7f0510004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:48.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:48.766+0000 7f0515ffb640 1 -- 172.21.15.38:0/3128065457 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f051c062820 con 0x7f05200cfe00 2024-09-21T03:35:48.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:48.953+0000 7f0527739640 1 -- 172.21.15.38:0/3128065457 --> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f052015cfe0 con 0x7f0504078b90 2024-09-21T03:35:48.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:48.958+0000 7f0515ffb640 1 -- 172.21.15.38:0/3128065457 <== mgr.4117 v2:172.21.15.195:6800/1631901344 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+244252 (secure 0 0 0) 0x7f052015cfe0 con 0x7f0504078b90 2024-09-21T03:35:48.962 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:35:48.963 INFO:teuthology.orchestra.run.smithi038.stderr:dumped all 2024-09-21T03:35:48.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:48.961+0000 7f0527739640 1 -- 172.21.15.38:0/3128065457 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0504078b90 msgr2=0x7f050407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:48.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:48.961+0000 7f0527739640 1 --2- 172.21.15.38:0/3128065457 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0504078b90 0x7f050407b0a0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f0510034000 tx=0x7f0510004730 comp rx=0 tx=0).stop 2024-09-21T03:35:48.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:48.961+0000 7f0527739640 1 -- 172.21.15.38:0/3128065457 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f05200cfe00 msgr2=0x7f05201f2ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:48.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:48.961+0000 7f0527739640 1 --2- 172.21.15.38:0/3128065457 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f05200cfe00 0x7f05201f2ac0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f051c00c430 tx=0x7f051c00c8f0 comp rx=0 tx=0).stop 2024-09-21T03:35:48.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:48.961+0000 7f0527739640 1 -- 172.21.15.38:0/3128065457 shutdown_connections 2024-09-21T03:35:48.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:48.961+0000 7f0527739640 1 --2- 172.21.15.38:0/3128065457 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0520162590 0x7f05200c8b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:48.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:48.961+0000 7f0527739640 1 --2- 172.21.15.38:0/3128065457 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f05200cfe00 0x7f05201f2ac0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:48.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:48.961+0000 7f0527739640 1 --2- 172.21.15.38:0/3128065457 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0504078b90 0x7f050407b0a0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:48.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:48.961+0000 7f0527739640 1 --2- 172.21.15.38:0/3128065457 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f05200b2e60 0x7f05201f6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:48.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:48.961+0000 7f0527739640 1 -- 172.21.15.38:0/3128065457 >> 172.21.15.38:0/3128065457 conn(0x7f05200d4440 msgr2=0x7f052006dbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:48.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:48.962+0000 7f0527739640 1 -- 172.21.15.38:0/3128065457 shutdown_connections 2024-09-21T03:35:48.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:48.962+0000 7f0527739640 1 -- 172.21.15.38:0/3128065457 wait complete. 2024-09-21T03:35:48.975 INFO:teuthology.orchestra.run.smithi038.stdout:{"pg_ready":true,"pg_map":{"version":120,"stamp":"2024-09-21T03:35:47.012998+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590438,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":102,"num_write_kb":1629,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":73,"ondisk_log_size":73,"up":126,"acting":126,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":263700,"kb_used_data":5856,"kb_used_omap":12,"kb_used_meta":257651,"kb_avail":754711020,"statfs":{"total":773094113280,"available":772824084480,"internally_reserved":0,"allocated":5996544,"data_stored":3301500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12748,"internal_metadata":263835188},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.131881"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T03:35:43.880613+0000","last_change":"2024-09-21T03:35:43.880613+0000","last_active":"2024-09-21T03:35:43.880613+0000","last_peered":"2024-09-21T03:35:43.880613+0000","last_clean":"2024-09-21T03:35:43.880613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:35:43.880613+0000","last_undegraded":"2024-09-21T03:35:43.880613+0000","last_fullsized":"2024-09-21T03:35:43.880613+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-09-21T03:35:43.900135+0000","last_change":"2024-09-21T03:35:43.900135+0000","last_active":"2024-09-21T03:35:43.880613+0000","last_peered":"2024-09-21T03:35:43.880613+0000","last_clean":"2024-09-21T03:35:43.880613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:35:43.900135+0000","last_undegraded":"2024-09-21T03:35:43.900135+0000","last_fullsized":"2024-09-21T03:35:43.900135+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T03:35:43.880613+0000","last_change":"2024-09-21T03:35:43.880613+0000","last_active":"2024-09-21T03:35:43.880613+0000","last_peered":"2024-09-21T03:35:43.880613+0000","last_clean":"2024-09-21T03:35:43.880613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:35:43.880613+0000","last_undegraded":"2024-09-21T03:35:43.880613+0000","last_fullsized":"2024-09-21T03:35:43.880613+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T03:35:44.927798+0000","last_change":"2024-09-21T03:35:44.921404+0000","last_active":"2024-09-21T03:35:44.927798+0000","last_peered":"2024-09-21T03:35:44.927798+0000","last_clean":"2024-09-21T03:35:44.927798+0000","last_became_active":"2024-09-21T03:35:44.921059+0000","last_became_peered":"2024-09-21T03:35:44.921059+0000","last_unstale":"2024-09-21T03:35:44.927798+0000","last_undegraded":"2024-09-21T03:35:44.927798+0000","last_fullsized":"2024-09-21T03:35:44.927798+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:54:00.361227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-09-21T03:35:43.902970+0000","last_change":"2024-09-21T03:35:43.902970+0000","last_active":"2024-09-21T03:35:43.880613+0000","last_peered":"2024-09-21T03:35:43.880613+0000","last_clean":"2024-09-21T03:35:43.880613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:35:43.902970+0000","last_undegraded":"2024-09-21T03:35:43.902970+0000","last_fullsized":"2024-09-21T03:35:43.902970+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-09-21T03:35:43.889762+0000","last_change":"2024-09-21T03:35:43.889762+0000","last_active":"2024-09-21T03:35:43.880613+0000","last_peered":"2024-09-21T03:35:43.880613+0000","last_clean":"2024-09-21T03:35:43.880613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:35:43.889762+0000","last_undegraded":"2024-09-21T03:35:43.889762+0000","last_fullsized":"2024-09-21T03:35:43.889762+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T03:35:43.880613+0000","last_change":"2024-09-21T03:35:43.880613+0000","last_active":"2024-09-21T03:35:43.880613+0000","last_peered":"2024-09-21T03:35:43.880613+0000","last_clean":"2024-09-21T03:35:43.880613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:35:43.880613+0000","last_undegraded":"2024-09-21T03:35:43.880613+0000","last_fullsized":"2024-09-21T03:35:43.880613+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-09-21T03:35:43.889893+0000","last_change":"2024-09-21T03:35:43.889893+0000","last_active":"2024-09-21T03:35:43.880613+0000","last_peered":"2024-09-21T03:35:43.880613+0000","last_clean":"2024-09-21T03:35:43.880613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:35:43.889893+0000","last_undegraded":"2024-09-21T03:35:43.889893+0000","last_fullsized":"2024-09-21T03:35:43.889893+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T03:35:43.880613+0000","last_change":"2024-09-21T03:35:43.880613+0000","last_active":"2024-09-21T03:35:43.880613+0000","last_peered":"2024-09-21T03:35:43.880613+0000","last_clean":"2024-09-21T03:35:43.880613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:35:43.880613+0000","last_undegraded":"2024-09-21T03:35:43.880613+0000","last_fullsized":"2024-09-21T03:35:43.880613+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T03:35:43.880613+0000","last_change":"2024-09-21T03:35:43.880613+0000","last_active":"2024-09-21T03:35:43.880613+0000","last_peered":"2024-09-21T03:35:43.880613+0000","last_clean":"2024-09-21T03:35:43.880613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:35:43.880613+0000","last_undegraded":"2024-09-21T03:35:43.880613+0000","last_fullsized":"2024-09-21T03:35:43.880613+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T03:35:43.880613+0000","last_change":"2024-09-21T03:35:43.880613+0000","last_active":"2024-09-21T03:35:43.880613+0000","last_peered":"2024-09-21T03:35:43.880613+0000","last_clean":"2024-09-21T03:35:43.880613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:35:43.880613+0000","last_undegraded":"2024-09-21T03:35:43.880613+0000","last_fullsized":"2024-09-21T03:35:43.880613+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-09-21T03:35:43.889542+0000","last_change":"2024-09-21T03:35:43.889542+0000","last_active":"2024-09-21T03:35:43.880613+0000","last_peered":"2024-09-21T03:35:43.880613+0000","last_clean":"2024-09-21T03:35:43.880613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:35:43.889542+0000","last_undegraded":"2024-09-21T03:35:43.889542+0000","last_fullsized":"2024-09-21T03:35:43.889542+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-09-21T03:35:43.890378+0000","last_change":"2024-09-21T03:35:43.890378+0000","last_active":"2024-09-21T03:35:43.880613+0000","last_peered":"2024-09-21T03:35:43.880613+0000","last_clean":"2024-09-21T03:35:43.880613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:35:43.890378+0000","last_undegraded":"2024-09-21T03:35:43.890378+0000","last_fullsized":"2024-09-21T03:35:43.890378+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T03:35:43.880613+0000","last_change":"2024-09-21T03:35:43.880613+0000","last_active":"2024-09-21T03:35:43.880613+0000","last_peered":"2024-09-21T03:35:43.880613+0000","last_clean":"2024-09-21T03:35:43.880613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:35:43.880613+0000","last_undegraded":"2024-09-21T03:35:43.880613+0000","last_fullsized":"2024-09-21T03:35:43.880613+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T03:35:44.922283+0000","last_change":"2024-09-21T03:35:44.919326+0000","last_active":"2024-09-21T03:35:44.922283+0000","last_peered":"2024-09-21T03:35:44.922283+0000","last_clean":"2024-09-21T03:35:44.922283+0000","last_became_active":"2024-09-21T03:35:44.919004+0000","last_became_peered":"2024-09-21T03:35:44.919004+0000","last_unstale":"2024-09-21T03:35:44.922283+0000","last_undegraded":"2024-09-21T03:35:44.922283+0000","last_fullsized":"2024-09-21T03:35:44.922283+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:31:52.864107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-09-21T03:35:43.889143+0000","last_change":"2024-09-21T03:35:43.889143+0000","last_active":"2024-09-21T03:35:43.880613+0000","last_peered":"2024-09-21T03:35:43.880613+0000","last_clean":"2024-09-21T03:35:43.880613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:35:43.889143+0000","last_undegraded":"2024-09-21T03:35:43.889143+0000","last_fullsized":"2024-09-21T03:35:43.889143+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T03:35:43.880613+0000","last_change":"2024-09-21T03:35:43.880613+0000","last_active":"2024-09-21T03:35:43.880613+0000","last_peered":"2024-09-21T03:35:43.880613+0000","last_clean":"2024-09-21T03:35:43.880613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:35:43.880613+0000","last_undegraded":"2024-09-21T03:35:43.880613+0000","last_fullsized":"2024-09-21T03:35:43.880613+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T03:35:44.924513+0000","last_change":"2024-09-21T03:35:44.920057+0000","last_active":"2024-09-21T03:35:44.924513+0000","last_peered":"2024-09-21T03:35:44.924513+0000","last_clean":"2024-09-21T03:35:44.924513+0000","last_became_active":"2024-09-21T03:35:44.919683+0000","last_became_peered":"2024-09-21T03:35:44.919683+0000","last_unstale":"2024-09-21T03:35:44.924513+0000","last_undegraded":"2024-09-21T03:35:44.924513+0000","last_fullsized":"2024-09-21T03:35:44.924513+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:05:30.105550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T03:35:43.880613+0000","last_change":"2024-09-21T03:35:43.880613+0000","last_active":"2024-09-21T03:35:43.880613+0000","last_peered":"2024-09-21T03:35:43.880613+0000","last_clean":"2024-09-21T03:35:43.880613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:35:43.880613+0000","last_undegraded":"2024-09-21T03:35:43.880613+0000","last_fullsized":"2024-09-21T03:35:43.880613+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-09-21T03:35:43.902374+0000","last_change":"2024-09-21T03:35:43.902374+0000","last_active":"2024-09-21T03:35:43.880613+0000","last_peered":"2024-09-21T03:35:43.880613+0000","last_clean":"2024-09-21T03:35:43.880613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:35:43.902374+0000","last_undegraded":"2024-09-21T03:35:43.902374+0000","last_fullsized":"2024-09-21T03:35:43.902374+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T03:35:44.920375+0000","last_change":"2024-09-21T03:35:44.915023+0000","last_active":"2024-09-21T03:35:44.920375+0000","last_peered":"2024-09-21T03:35:44.920375+0000","last_clean":"2024-09-21T03:35:44.920375+0000","last_became_active":"2024-09-21T03:35:44.914677+0000","last_became_peered":"2024-09-21T03:35:44.914677+0000","last_unstale":"2024-09-21T03:35:44.920375+0000","last_undegraded":"2024-09-21T03:35:44.920375+0000","last_fullsized":"2024-09-21T03:35:44.920375+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:41:13.888920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T03:35:43.880613+0000","last_change":"2024-09-21T03:35:43.880613+0000","last_active":"2024-09-21T03:35:43.880613+0000","last_peered":"2024-09-21T03:35:43.880613+0000","last_clean":"2024-09-21T03:35:43.880613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:35:43.880613+0000","last_undegraded":"2024-09-21T03:35:43.880613+0000","last_fullsized":"2024-09-21T03:35:43.880613+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T03:35:43.880613+0000","last_change":"2024-09-21T03:35:43.880613+0000","last_active":"2024-09-21T03:35:43.880613+0000","last_peered":"2024-09-21T03:35:43.880613+0000","last_clean":"2024-09-21T03:35:43.880613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:35:43.880613+0000","last_undegraded":"2024-09-21T03:35:43.880613+0000","last_fullsized":"2024-09-21T03:35:43.880613+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T03:35:43.880613+0000","last_change":"2024-09-21T03:35:43.880613+0000","last_active":"2024-09-21T03:35:43.880613+0000","last_peered":"2024-09-21T03:35:43.880613+0000","last_clean":"2024-09-21T03:35:43.880613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:35:43.880613+0000","last_undegraded":"2024-09-21T03:35:43.880613+0000","last_fullsized":"2024-09-21T03:35:43.880613+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T03:35:43.880613+0000","last_change":"2024-09-21T03:35:43.880613+0000","last_active":"2024-09-21T03:35:43.880613+0000","last_peered":"2024-09-21T03:35:43.880613+0000","last_clean":"2024-09-21T03:35:43.880613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:35:43.880613+0000","last_undegraded":"2024-09-21T03:35:43.880613+0000","last_fullsized":"2024-09-21T03:35:43.880613+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T03:35:43.880613+0000","last_change":"2024-09-21T03:35:43.880613+0000","last_active":"2024-09-21T03:35:43.880613+0000","last_peered":"2024-09-21T03:35:43.880613+0000","last_clean":"2024-09-21T03:35:43.880613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:35:43.880613+0000","last_undegraded":"2024-09-21T03:35:43.880613+0000","last_fullsized":"2024-09-21T03:35:43.880613+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-09-21T03:35:43.892475+0000","last_change":"2024-09-21T03:35:43.892475+0000","last_active":"2024-09-21T03:35:43.880613+0000","last_peered":"2024-09-21T03:35:43.880613+0000","last_clean":"2024-09-21T03:35:43.880613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:35:43.892475+0000","last_undegraded":"2024-09-21T03:35:43.892475+0000","last_fullsized":"2024-09-21T03:35:43.892475+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-09-21T03:35:43.901876+0000","last_change":"2024-09-21T03:35:43.901876+0000","last_active":"2024-09-21T03:35:43.880613+0000","last_peered":"2024-09-21T03:35:43.880613+0000","last_clean":"2024-09-21T03:35:43.880613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:35:43.901876+0000","last_undegraded":"2024-09-21T03:35:43.901876+0000","last_fullsized":"2024-09-21T03:35:43.901876+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T03:35:43.880613+0000","last_change":"2024-09-21T03:35:43.880613+0000","last_active":"2024-09-21T03:35:43.880613+0000","last_peered":"2024-09-21T03:35:43.880613+0000","last_clean":"2024-09-21T03:35:43.880613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:35:43.880613+0000","last_undegraded":"2024-09-21T03:35:43.880613+0000","last_fullsized":"2024-09-21T03:35:43.880613+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-09-21T03:35:43.896302+0000","last_change":"2024-09-21T03:35:43.896302+0000","last_active":"2024-09-21T03:35:43.880613+0000","last_peered":"2024-09-21T03:35:43.880613+0000","last_clean":"2024-09-21T03:35:43.880613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:35:43.896302+0000","last_undegraded":"2024-09-21T03:35:43.896302+0000","last_fullsized":"2024-09-21T03:35:43.896302+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T03:35:43.880613+0000","last_change":"2024-09-21T03:35:43.880613+0000","last_active":"2024-09-21T03:35:43.880613+0000","last_peered":"2024-09-21T03:35:43.880613+0000","last_clean":"2024-09-21T03:35:43.880613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:35:43.880613+0000","last_undegraded":"2024-09-21T03:35:43.880613+0000","last_fullsized":"2024-09-21T03:35:43.880613+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-09-21T03:35:43.899921+0000","last_change":"2024-09-21T03:35:43.899921+0000","last_active":"2024-09-21T03:35:43.880613+0000","last_peered":"2024-09-21T03:35:43.880613+0000","last_clean":"2024-09-21T03:35:43.880613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:35:43.899921+0000","last_undegraded":"2024-09-21T03:35:43.899921+0000","last_fullsized":"2024-09-21T03:35:43.899921+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-09-21T03:35:43.897228+0000","last_change":"2024-09-21T03:35:43.897228+0000","last_active":"2024-09-21T03:35:43.880613+0000","last_peered":"2024-09-21T03:35:43.880613+0000","last_clean":"2024-09-21T03:35:43.880613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:35:43.897228+0000","last_undegraded":"2024-09-21T03:35:43.897228+0000","last_fullsized":"2024-09-21T03:35:43.897228+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-21T03:35:43.911675+0000","last_change":"2024-09-21T03:35:43.909828+0000","last_active":"2024-09-21T03:35:43.911675+0000","last_peered":"2024-09-21T03:35:43.911675+0000","last_clean":"2024-09-21T03:35:43.911675+0000","last_became_active":"2024-09-21T03:35:43.909503+0000","last_became_peered":"2024-09-21T03:35:43.909503+0000","last_unstale":"2024-09-21T03:35:43.911675+0000","last_undegraded":"2024-09-21T03:35:43.911675+0000","last_fullsized":"2024-09-21T03:35:43.911675+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_clean_scrub_stamp":"2024-09-21T03:35:42.877874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:35:28.406113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-09-21T03:35:43.895909+0000","last_change":"2024-09-21T03:35:43.895909+0000","last_active":"2024-09-21T03:35:43.880613+0000","last_peered":"2024-09-21T03:35:43.880613+0000","last_clean":"2024-09-21T03:35:43.880613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:35:43.895909+0000","last_undegraded":"2024-09-21T03:35:43.895909+0000","last_fullsized":"2024-09-21T03:35:43.895909+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-21T03:35:43.902645+0000","last_change":"2024-09-21T03:35:43.901432+0000","last_active":"2024-09-21T03:35:43.902645+0000","last_peered":"2024-09-21T03:35:43.902645+0000","last_clean":"2024-09-21T03:35:43.902645+0000","last_became_active":"2024-09-21T03:35:43.901134+0000","last_became_peered":"2024-09-21T03:35:43.901134+0000","last_unstale":"2024-09-21T03:35:43.902645+0000","last_undegraded":"2024-09-21T03:35:43.902645+0000","last_fullsized":"2024-09-21T03:35:43.902645+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_clean_scrub_stamp":"2024-09-21T03:35:42.877874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:42:45.225890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-09-21T03:35:43.897697+0000","last_change":"2024-09-21T03:35:43.897697+0000","last_active":"2024-09-21T03:35:43.880613+0000","last_peered":"2024-09-21T03:35:43.880613+0000","last_clean":"2024-09-21T03:35:43.880613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:35:43.897697+0000","last_undegraded":"2024-09-21T03:35:43.897697+0000","last_fullsized":"2024-09-21T03:35:43.897697+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T03:35:42.877874+0000","last_change":"2024-09-21T03:35:42.877874+0000","last_active":"2024-09-21T03:35:42.877874+0000","last_peered":"2024-09-21T03:35:42.877874+0000","last_clean":"2024-09-21T03:35:42.877874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:35:42.877874+0000","last_undegraded":"2024-09-21T03:35:42.877874+0000","last_fullsized":"2024-09-21T03:35:42.877874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_clean_scrub_stamp":"2024-09-21T03:35:42.877874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T03:35:43.880613+0000","last_change":"2024-09-21T03:35:43.880613+0000","last_active":"2024-09-21T03:35:43.880613+0000","last_peered":"2024-09-21T03:35:43.880613+0000","last_clean":"2024-09-21T03:35:43.880613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:35:43.880613+0000","last_undegraded":"2024-09-21T03:35:43.880613+0000","last_fullsized":"2024-09-21T03:35:43.880613+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-21T03:35:43.916772+0000","last_change":"2024-09-21T03:35:43.911405+0000","last_active":"2024-09-21T03:35:43.916772+0000","last_peered":"2024-09-21T03:35:43.916772+0000","last_clean":"2024-09-21T03:35:43.916772+0000","last_became_active":"2024-09-21T03:35:43.911082+0000","last_became_peered":"2024-09-21T03:35:43.911082+0000","last_unstale":"2024-09-21T03:35:43.916772+0000","last_undegraded":"2024-09-21T03:35:43.916772+0000","last_fullsized":"2024-09-21T03:35:43.916772+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_clean_scrub_stamp":"2024-09-21T03:35:42.877874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:40:34.993739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-09-21T03:35:43.895551+0000","last_change":"2024-09-21T03:35:43.895551+0000","last_active":"2024-09-21T03:35:43.880613+0000","last_peered":"2024-09-21T03:35:43.880613+0000","last_clean":"2024-09-21T03:35:43.880613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:35:43.895551+0000","last_undegraded":"2024-09-21T03:35:43.895551+0000","last_fullsized":"2024-09-21T03:35:43.895551+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-09-21T03:35:42.887583+0000","last_change":"2024-09-21T03:35:42.887583+0000","last_active":"2024-09-21T03:35:42.877874+0000","last_peered":"2024-09-21T03:35:42.877874+0000","last_clean":"2024-09-21T03:35:42.877874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:35:42.887583+0000","last_undegraded":"2024-09-21T03:35:42.887583+0000","last_fullsized":"2024-09-21T03:35:42.887583+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_clean_scrub_stamp":"2024-09-21T03:35:42.877874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-09-21T03:35:43.897379+0000","last_change":"2024-09-21T03:35:43.897379+0000","last_active":"2024-09-21T03:35:43.880613+0000","last_peered":"2024-09-21T03:35:43.880613+0000","last_clean":"2024-09-21T03:35:43.880613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:35:43.897379+0000","last_undegraded":"2024-09-21T03:35:43.897379+0000","last_fullsized":"2024-09-21T03:35:43.897379+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-21T03:35:43.915560+0000","last_change":"2024-09-21T03:35:43.910078+0000","last_active":"2024-09-21T03:35:43.915560+0000","last_peered":"2024-09-21T03:35:43.915560+0000","last_clean":"2024-09-21T03:35:43.915560+0000","last_became_active":"2024-09-21T03:35:43.909315+0000","last_became_peered":"2024-09-21T03:35:43.909315+0000","last_unstale":"2024-09-21T03:35:43.915560+0000","last_undegraded":"2024-09-21T03:35:43.915560+0000","last_fullsized":"2024-09-21T03:35:43.915560+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_clean_scrub_stamp":"2024-09-21T03:35:42.877874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:48:07.945139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T03:35:43.880613+0000","last_change":"2024-09-21T03:35:43.880613+0000","last_active":"2024-09-21T03:35:43.880613+0000","last_peered":"2024-09-21T03:35:43.880613+0000","last_clean":"2024-09-21T03:35:43.880613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:35:43.880613+0000","last_undegraded":"2024-09-21T03:35:43.880613+0000","last_fullsized":"2024-09-21T03:35:43.880613+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-21T03:35:43.915132+0000","last_change":"2024-09-21T03:35:43.909144+0000","last_active":"2024-09-21T03:35:43.915132+0000","last_peered":"2024-09-21T03:35:43.915132+0000","last_clean":"2024-09-21T03:35:43.915132+0000","last_became_active":"2024-09-21T03:35:43.908808+0000","last_became_peered":"2024-09-21T03:35:43.908808+0000","last_unstale":"2024-09-21T03:35:43.915132+0000","last_undegraded":"2024-09-21T03:35:43.915132+0000","last_fullsized":"2024-09-21T03:35:43.915132+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_clean_scrub_stamp":"2024-09-21T03:35:42.877874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:51:41.557674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T03:35:44.913427+0000","last_change":"2024-09-21T03:35:44.907438+0000","last_active":"2024-09-21T03:35:44.913427+0000","last_peered":"2024-09-21T03:35:44.913427+0000","last_clean":"2024-09-21T03:35:44.913427+0000","last_became_active":"2024-09-21T03:35:44.907114+0000","last_became_peered":"2024-09-21T03:35:44.907114+0000","last_unstale":"2024-09-21T03:35:44.913427+0000","last_undegraded":"2024-09-21T03:35:44.913427+0000","last_fullsized":"2024-09-21T03:35:44.913427+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:44:34.145008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-09-21T03:35:42.888245+0000","last_change":"2024-09-21T03:35:42.888245+0000","last_active":"2024-09-21T03:35:42.877874+0000","last_peered":"2024-09-21T03:35:42.877874+0000","last_clean":"2024-09-21T03:35:42.877874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:35:42.888245+0000","last_undegraded":"2024-09-21T03:35:42.888245+0000","last_fullsized":"2024-09-21T03:35:42.888245+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_clean_scrub_stamp":"2024-09-21T03:35:42.877874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-09-21T03:35:43.895500+0000","last_change":"2024-09-21T03:35:43.895500+0000","last_active":"2024-09-21T03:35:43.880613+0000","last_peered":"2024-09-21T03:35:43.880613+0000","last_clean":"2024-09-21T03:35:43.880613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:35:43.895500+0000","last_undegraded":"2024-09-21T03:35:43.895500+0000","last_fullsized":"2024-09-21T03:35:43.895500+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T03:35:44.896174+0000","last_change":"2024-09-21T03:35:43.905541+0000","last_active":"2024-09-21T03:35:44.896174+0000","last_peered":"2024-09-21T03:35:44.896174+0000","last_clean":"2024-09-21T03:35:44.896174+0000","last_became_active":"2024-09-21T03:35:43.905230+0000","last_became_peered":"2024-09-21T03:35:43.905230+0000","last_unstale":"2024-09-21T03:35:44.896174+0000","last_undegraded":"2024-09-21T03:35:44.896174+0000","last_fullsized":"2024-09-21T03:35:44.896174+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_clean_scrub_stamp":"2024-09-21T03:35:42.877874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:46:28.427572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T03:35:44.927406+0000","last_change":"2024-09-21T03:35:44.920481+0000","last_active":"2024-09-21T03:35:44.927406+0000","last_peered":"2024-09-21T03:35:44.927406+0000","last_clean":"2024-09-21T03:35:44.927406+0000","last_became_active":"2024-09-21T03:35:44.919780+0000","last_became_peered":"2024-09-21T03:35:44.919780+0000","last_unstale":"2024-09-21T03:35:44.927406+0000","last_undegraded":"2024-09-21T03:35:44.927406+0000","last_fullsized":"2024-09-21T03:35:44.927406+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:33:00.469260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T03:35:44.898827+0000","last_change":"2024-09-21T03:35:43.898522+0000","last_active":"2024-09-21T03:35:44.898827+0000","last_peered":"2024-09-21T03:35:44.898827+0000","last_clean":"2024-09-21T03:35:44.898827+0000","last_became_active":"2024-09-21T03:35:43.898171+0000","last_became_peered":"2024-09-21T03:35:43.898171+0000","last_unstale":"2024-09-21T03:35:44.898827+0000","last_undegraded":"2024-09-21T03:35:44.898827+0000","last_fullsized":"2024-09-21T03:35:44.898827+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_clean_scrub_stamp":"2024-09-21T03:35:42.877874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:14:03.947289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T03:35:43.880613+0000","last_change":"2024-09-21T03:35:43.880613+0000","last_active":"2024-09-21T03:35:43.880613+0000","last_peered":"2024-09-21T03:35:43.880613+0000","last_clean":"2024-09-21T03:35:43.880613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:35:43.880613+0000","last_undegraded":"2024-09-21T03:35:43.880613+0000","last_fullsized":"2024-09-21T03:35:43.880613+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-21T03:35:43.915078+0000","last_change":"2024-09-21T03:35:43.907104+0000","last_active":"2024-09-21T03:35:43.915078+0000","last_peered":"2024-09-21T03:35:43.915078+0000","last_clean":"2024-09-21T03:35:43.915078+0000","last_became_active":"2024-09-21T03:35:43.906758+0000","last_became_peered":"2024-09-21T03:35:43.906758+0000","last_unstale":"2024-09-21T03:35:43.915078+0000","last_undegraded":"2024-09-21T03:35:43.915078+0000","last_fullsized":"2024-09-21T03:35:43.915078+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_clean_scrub_stamp":"2024-09-21T03:35:42.877874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:53:55.235170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-09-21T03:35:43.903519+0000","last_change":"2024-09-21T03:35:43.903519+0000","last_active":"2024-09-21T03:35:43.880613+0000","last_peered":"2024-09-21T03:35:43.880613+0000","last_clean":"2024-09-21T03:35:43.880613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:35:43.903519+0000","last_undegraded":"2024-09-21T03:35:43.903519+0000","last_fullsized":"2024-09-21T03:35:43.903519+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T03:35:42.877874+0000","last_change":"2024-09-21T03:35:42.877874+0000","last_active":"2024-09-21T03:35:42.877874+0000","last_peered":"2024-09-21T03:35:42.877874+0000","last_clean":"2024-09-21T03:35:42.877874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:35:42.877874+0000","last_undegraded":"2024-09-21T03:35:42.877874+0000","last_fullsized":"2024-09-21T03:35:42.877874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_clean_scrub_stamp":"2024-09-21T03:35:42.877874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-09-21T03:35:43.896352+0000","last_change":"2024-09-21T03:35:43.896352+0000","last_active":"2024-09-21T03:35:43.880613+0000","last_peered":"2024-09-21T03:35:43.880613+0000","last_clean":"2024-09-21T03:35:43.880613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:35:43.896352+0000","last_undegraded":"2024-09-21T03:35:43.896352+0000","last_fullsized":"2024-09-21T03:35:43.896352+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-21T03:35:43.913708+0000","last_change":"2024-09-21T03:35:43.913254+0000","last_active":"2024-09-21T03:35:43.913708+0000","last_peered":"2024-09-21T03:35:43.913708+0000","last_clean":"2024-09-21T03:35:43.913708+0000","last_became_active":"2024-09-21T03:35:43.912392+0000","last_became_peered":"2024-09-21T03:35:43.912392+0000","last_unstale":"2024-09-21T03:35:43.913708+0000","last_undegraded":"2024-09-21T03:35:43.913708+0000","last_fullsized":"2024-09-21T03:35:43.913708+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_clean_scrub_stamp":"2024-09-21T03:35:42.877874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:53:16.732262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T03:35:43.880613+0000","last_change":"2024-09-21T03:35:43.880613+0000","last_active":"2024-09-21T03:35:43.880613+0000","last_peered":"2024-09-21T03:35:43.880613+0000","last_clean":"2024-09-21T03:35:43.880613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:35:43.880613+0000","last_undegraded":"2024-09-21T03:35:43.880613+0000","last_fullsized":"2024-09-21T03:35:43.880613+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T03:35:44.895492+0000","last_change":"2024-09-21T03:35:43.897065+0000","last_active":"2024-09-21T03:35:44.895492+0000","last_peered":"2024-09-21T03:35:44.895492+0000","last_clean":"2024-09-21T03:35:44.895492+0000","last_became_active":"2024-09-21T03:35:43.896727+0000","last_became_peered":"2024-09-21T03:35:43.896727+0000","last_unstale":"2024-09-21T03:35:44.895492+0000","last_undegraded":"2024-09-21T03:35:44.895492+0000","last_fullsized":"2024-09-21T03:35:44.895492+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_clean_scrub_stamp":"2024-09-21T03:35:42.877874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:57:57.601786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-09-21T03:35:43.892997+0000","last_change":"2024-09-21T03:35:43.892997+0000","last_active":"2024-09-21T03:35:43.880613+0000","last_peered":"2024-09-21T03:35:43.880613+0000","last_clean":"2024-09-21T03:35:43.880613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:35:43.892997+0000","last_undegraded":"2024-09-21T03:35:43.892997+0000","last_fullsized":"2024-09-21T03:35:43.892997+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-21T03:35:43.915446+0000","last_change":"2024-09-21T03:35:43.910589+0000","last_active":"2024-09-21T03:35:43.915446+0000","last_peered":"2024-09-21T03:35:43.915446+0000","last_clean":"2024-09-21T03:35:43.915446+0000","last_became_active":"2024-09-21T03:35:43.910150+0000","last_became_peered":"2024-09-21T03:35:43.910150+0000","last_unstale":"2024-09-21T03:35:43.915446+0000","last_undegraded":"2024-09-21T03:35:43.915446+0000","last_fullsized":"2024-09-21T03:35:43.915446+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_clean_scrub_stamp":"2024-09-21T03:35:42.877874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:47:52.583294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T03:35:43.880613+0000","last_change":"2024-09-21T03:35:43.880613+0000","last_active":"2024-09-21T03:35:43.880613+0000","last_peered":"2024-09-21T03:35:43.880613+0000","last_clean":"2024-09-21T03:35:43.880613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:35:43.880613+0000","last_undegraded":"2024-09-21T03:35:43.880613+0000","last_fullsized":"2024-09-21T03:35:43.880613+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-21T03:35:43.909227+0000","last_change":"2024-09-21T03:35:43.905763+0000","last_active":"2024-09-21T03:35:43.909227+0000","last_peered":"2024-09-21T03:35:43.909227+0000","last_clean":"2024-09-21T03:35:43.909227+0000","last_became_active":"2024-09-21T03:35:43.905453+0000","last_became_peered":"2024-09-21T03:35:43.905453+0000","last_unstale":"2024-09-21T03:35:43.909227+0000","last_undegraded":"2024-09-21T03:35:43.909227+0000","last_fullsized":"2024-09-21T03:35:43.909227+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_clean_scrub_stamp":"2024-09-21T03:35:42.877874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:12:58.212405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T03:35:43.880613+0000","last_change":"2024-09-21T03:35:43.880613+0000","last_active":"2024-09-21T03:35:43.880613+0000","last_peered":"2024-09-21T03:35:43.880613+0000","last_clean":"2024-09-21T03:35:43.880613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:35:43.880613+0000","last_undegraded":"2024-09-21T03:35:43.880613+0000","last_fullsized":"2024-09-21T03:35:43.880613+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-21T03:35:43.901848+0000","last_change":"2024-09-21T03:35:43.900447+0000","last_active":"2024-09-21T03:35:43.901848+0000","last_peered":"2024-09-21T03:35:43.901848+0000","last_clean":"2024-09-21T03:35:43.901848+0000","last_became_active":"2024-09-21T03:35:43.900127+0000","last_became_peered":"2024-09-21T03:35:43.900127+0000","last_unstale":"2024-09-21T03:35:43.901848+0000","last_undegraded":"2024-09-21T03:35:43.901848+0000","last_fullsized":"2024-09-21T03:35:43.901848+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_clean_scrub_stamp":"2024-09-21T03:35:42.877874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:20:29.206321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T03:35:43.880613+0000","last_change":"2024-09-21T03:35:43.880613+0000","last_active":"2024-09-21T03:35:43.880613+0000","last_peered":"2024-09-21T03:35:43.880613+0000","last_clean":"2024-09-21T03:35:43.880613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:35:43.880613+0000","last_undegraded":"2024-09-21T03:35:43.880613+0000","last_fullsized":"2024-09-21T03:35:43.880613+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-21T03:35:43.909265+0000","last_change":"2024-09-21T03:35:43.906381+0000","last_active":"2024-09-21T03:35:43.909265+0000","last_peered":"2024-09-21T03:35:43.909265+0000","last_clean":"2024-09-21T03:35:43.909265+0000","last_became_active":"2024-09-21T03:35:43.905982+0000","last_became_peered":"2024-09-21T03:35:43.905982+0000","last_unstale":"2024-09-21T03:35:43.909265+0000","last_undegraded":"2024-09-21T03:35:43.909265+0000","last_fullsized":"2024-09-21T03:35:43.909265+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_clean_scrub_stamp":"2024-09-21T03:35:42.877874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:32:38.146616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-09-21T03:35:43.900034+0000","last_change":"2024-09-21T03:35:43.900034+0000","last_active":"2024-09-21T03:35:43.880613+0000","last_peered":"2024-09-21T03:35:43.880613+0000","last_clean":"2024-09-21T03:35:43.880613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:35:43.900034+0000","last_undegraded":"2024-09-21T03:35:43.900034+0000","last_fullsized":"2024-09-21T03:35:43.900034+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.d","version":"54'1","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T03:35:44.915599+0000","last_change":"2024-09-21T03:35:43.899938+0000","last_active":"2024-09-21T03:35:44.915599+0000","last_peered":"2024-09-21T03:35:44.915599+0000","last_clean":"2024-09-21T03:35:44.915599+0000","last_became_active":"2024-09-21T03:35:43.899617+0000","last_became_peered":"2024-09-21T03:35:43.899617+0000","last_unstale":"2024-09-21T03:35:44.915599+0000","last_undegraded":"2024-09-21T03:35:44.915599+0000","last_fullsized":"2024-09-21T03:35:44.915599+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_clean_scrub_stamp":"2024-09-21T03:35:42.877874+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:54:01.776464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-09-21T03:35:43.890997+0000","last_change":"2024-09-21T03:35:43.890997+0000","last_active":"2024-09-21T03:35:43.880613+0000","last_peered":"2024-09-21T03:35:43.880613+0000","last_clean":"2024-09-21T03:35:43.880613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:35:43.890997+0000","last_undegraded":"2024-09-21T03:35:43.890997+0000","last_fullsized":"2024-09-21T03:35:43.890997+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-21T03:35:43.916804+0000","last_change":"2024-09-21T03:35:43.911542+0000","last_active":"2024-09-21T03:35:43.916804+0000","last_peered":"2024-09-21T03:35:43.916804+0000","last_clean":"2024-09-21T03:35:43.916804+0000","last_became_active":"2024-09-21T03:35:43.911246+0000","last_became_peered":"2024-09-21T03:35:43.911246+0000","last_unstale":"2024-09-21T03:35:43.916804+0000","last_undegraded":"2024-09-21T03:35:43.916804+0000","last_fullsized":"2024-09-21T03:35:43.916804+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_clean_scrub_stamp":"2024-09-21T03:35:42.877874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:23:28.021867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-09-21T03:35:43.901415+0000","last_change":"2024-09-21T03:35:43.901415+0000","last_active":"2024-09-21T03:35:43.880613+0000","last_peered":"2024-09-21T03:35:43.880613+0000","last_clean":"2024-09-21T03:35:43.880613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:35:43.901415+0000","last_undegraded":"2024-09-21T03:35:43.901415+0000","last_fullsized":"2024-09-21T03:35:43.901415+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-09-21T03:35:42.887282+0000","last_change":"2024-09-21T03:35:42.887282+0000","last_active":"2024-09-21T03:35:42.877874+0000","last_peered":"2024-09-21T03:35:42.877874+0000","last_clean":"2024-09-21T03:35:42.877874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:35:42.887282+0000","last_undegraded":"2024-09-21T03:35:42.887282+0000","last_fullsized":"2024-09-21T03:35:42.887282+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_clean_scrub_stamp":"2024-09-21T03:35:42.877874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T03:35:44.915374+0000","last_change":"2024-09-21T03:35:44.911441+0000","last_active":"2024-09-21T03:35:44.915374+0000","last_peered":"2024-09-21T03:35:44.915374+0000","last_clean":"2024-09-21T03:35:44.915374+0000","last_became_active":"2024-09-21T03:35:44.911116+0000","last_became_peered":"2024-09-21T03:35:44.911116+0000","last_unstale":"2024-09-21T03:35:44.915374+0000","last_undegraded":"2024-09-21T03:35:44.915374+0000","last_fullsized":"2024-09-21T03:35:44.915374+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:00:19.050385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-09-21T03:35:42.888463+0000","last_change":"2024-09-21T03:35:42.888463+0000","last_active":"2024-09-21T03:35:42.877874+0000","last_peered":"2024-09-21T03:35:42.877874+0000","last_clean":"2024-09-21T03:35:42.877874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:35:42.888463+0000","last_undegraded":"2024-09-21T03:35:42.888463+0000","last_fullsized":"2024-09-21T03:35:42.888463+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_clean_scrub_stamp":"2024-09-21T03:35:42.877874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T03:35:43.880613+0000","last_change":"2024-09-21T03:35:43.880613+0000","last_active":"2024-09-21T03:35:43.880613+0000","last_peered":"2024-09-21T03:35:43.880613+0000","last_clean":"2024-09-21T03:35:43.880613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:35:43.880613+0000","last_undegraded":"2024-09-21T03:35:43.880613+0000","last_fullsized":"2024-09-21T03:35:43.880613+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T03:35:42.877874+0000","last_change":"2024-09-21T03:35:42.877874+0000","last_active":"2024-09-21T03:35:42.877874+0000","last_peered":"2024-09-21T03:35:42.877874+0000","last_clean":"2024-09-21T03:35:42.877874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:35:42.877874+0000","last_undegraded":"2024-09-21T03:35:42.877874+0000","last_fullsized":"2024-09-21T03:35:42.877874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_clean_scrub_stamp":"2024-09-21T03:35:42.877874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-09-21T03:35:43.891647+0000","last_change":"2024-09-21T03:35:43.891647+0000","last_active":"2024-09-21T03:35:43.880613+0000","last_peered":"2024-09-21T03:35:43.880613+0000","last_clean":"2024-09-21T03:35:43.880613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:35:43.891647+0000","last_undegraded":"2024-09-21T03:35:43.891647+0000","last_fullsized":"2024-09-21T03:35:43.891647+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T03:35:42.877874+0000","last_change":"2024-09-21T03:35:42.877874+0000","last_active":"2024-09-21T03:35:42.877874+0000","last_peered":"2024-09-21T03:35:42.877874+0000","last_clean":"2024-09-21T03:35:42.877874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:35:42.877874+0000","last_undegraded":"2024-09-21T03:35:42.877874+0000","last_fullsized":"2024-09-21T03:35:42.877874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_clean_scrub_stamp":"2024-09-21T03:35:42.877874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T03:35:44.908986+0000","last_change":"2024-09-21T03:35:44.907182+0000","last_active":"2024-09-21T03:35:44.908986+0000","last_peered":"2024-09-21T03:35:44.908986+0000","last_clean":"2024-09-21T03:35:44.908986+0000","last_became_active":"2024-09-21T03:35:44.906783+0000","last_became_peered":"2024-09-21T03:35:44.906783+0000","last_unstale":"2024-09-21T03:35:44.908986+0000","last_undegraded":"2024-09-21T03:35:44.908986+0000","last_fullsized":"2024-09-21T03:35:44.908986+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:00:17.389518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-21T03:35:43.915543+0000","last_change":"2024-09-21T03:35:43.910068+0000","last_active":"2024-09-21T03:35:43.915543+0000","last_peered":"2024-09-21T03:35:43.915543+0000","last_clean":"2024-09-21T03:35:43.915543+0000","last_became_active":"2024-09-21T03:35:43.908871+0000","last_became_peered":"2024-09-21T03:35:43.908871+0000","last_unstale":"2024-09-21T03:35:43.915543+0000","last_undegraded":"2024-09-21T03:35:43.915543+0000","last_fullsized":"2024-09-21T03:35:43.915543+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_clean_scrub_stamp":"2024-09-21T03:35:42.877874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:42:59.626174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T03:35:43.880613+0000","last_change":"2024-09-21T03:35:43.880613+0000","last_active":"2024-09-21T03:35:43.880613+0000","last_peered":"2024-09-21T03:35:43.880613+0000","last_clean":"2024-09-21T03:35:43.880613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:35:43.880613+0000","last_undegraded":"2024-09-21T03:35:43.880613+0000","last_fullsized":"2024-09-21T03:35:43.880613+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-21T03:35:43.915020+0000","last_change":"2024-09-21T03:35:43.904698+0000","last_active":"2024-09-21T03:35:43.915020+0000","last_peered":"2024-09-21T03:35:43.915020+0000","last_clean":"2024-09-21T03:35:43.915020+0000","last_became_active":"2024-09-21T03:35:43.904398+0000","last_became_peered":"2024-09-21T03:35:43.904398+0000","last_unstale":"2024-09-21T03:35:43.915020+0000","last_undegraded":"2024-09-21T03:35:43.915020+0000","last_fullsized":"2024-09-21T03:35:43.915020+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_clean_scrub_stamp":"2024-09-21T03:35:42.877874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:37:46.304367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T03:35:43.880613+0000","last_change":"2024-09-21T03:35:43.880613+0000","last_active":"2024-09-21T03:35:43.880613+0000","last_peered":"2024-09-21T03:35:43.880613+0000","last_clean":"2024-09-21T03:35:43.880613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:35:43.880613+0000","last_undegraded":"2024-09-21T03:35:43.880613+0000","last_fullsized":"2024-09-21T03:35:43.880613+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-09-21T03:35:42.888297+0000","last_change":"2024-09-21T03:35:42.888297+0000","last_active":"2024-09-21T03:35:42.877874+0000","last_peered":"2024-09-21T03:35:42.877874+0000","last_clean":"2024-09-21T03:35:42.877874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:35:42.888297+0000","last_undegraded":"2024-09-21T03:35:42.888297+0000","last_fullsized":"2024-09-21T03:35:42.888297+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_clean_scrub_stamp":"2024-09-21T03:35:42.877874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T03:35:43.880613+0000","last_change":"2024-09-21T03:35:43.880613+0000","last_active":"2024-09-21T03:35:43.880613+0000","last_peered":"2024-09-21T03:35:43.880613+0000","last_clean":"2024-09-21T03:35:43.880613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:35:43.880613+0000","last_undegraded":"2024-09-21T03:35:43.880613+0000","last_fullsized":"2024-09-21T03:35:43.880613+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T03:35:42.877874+0000","last_change":"2024-09-21T03:35:42.877874+0000","last_active":"2024-09-21T03:35:42.877874+0000","last_peered":"2024-09-21T03:35:42.877874+0000","last_clean":"2024-09-21T03:35:42.877874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:35:42.877874+0000","last_undegraded":"2024-09-21T03:35:42.877874+0000","last_fullsized":"2024-09-21T03:35:42.877874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_clean_scrub_stamp":"2024-09-21T03:35:42.877874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-09-21T03:35:43.900410+0000","last_change":"2024-09-21T03:35:43.900410+0000","last_active":"2024-09-21T03:35:43.880613+0000","last_peered":"2024-09-21T03:35:43.880613+0000","last_clean":"2024-09-21T03:35:43.880613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:35:43.900410+0000","last_undegraded":"2024-09-21T03:35:43.900410+0000","last_fullsized":"2024-09-21T03:35:43.900410+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-09-21T03:35:42.887152+0000","last_change":"2024-09-21T03:35:42.887152+0000","last_active":"2024-09-21T03:35:42.877874+0000","last_peered":"2024-09-21T03:35:42.877874+0000","last_clean":"2024-09-21T03:35:42.877874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:35:42.887152+0000","last_undegraded":"2024-09-21T03:35:42.887152+0000","last_fullsized":"2024-09-21T03:35:42.887152+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_clean_scrub_stamp":"2024-09-21T03:35:42.877874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T03:35:43.880613+0000","last_change":"2024-09-21T03:35:43.880613+0000","last_active":"2024-09-21T03:35:43.880613+0000","last_peered":"2024-09-21T03:35:43.880613+0000","last_clean":"2024-09-21T03:35:43.880613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:35:43.880613+0000","last_undegraded":"2024-09-21T03:35:43.880613+0000","last_fullsized":"2024-09-21T03:35:43.880613+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-21T03:35:43.916795+0000","last_change":"2024-09-21T03:35:43.912067+0000","last_active":"2024-09-21T03:35:43.916795+0000","last_peered":"2024-09-21T03:35:43.916795+0000","last_clean":"2024-09-21T03:35:43.916795+0000","last_became_active":"2024-09-21T03:35:43.911556+0000","last_became_peered":"2024-09-21T03:35:43.911556+0000","last_unstale":"2024-09-21T03:35:43.916795+0000","last_undegraded":"2024-09-21T03:35:43.916795+0000","last_fullsized":"2024-09-21T03:35:43.916795+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_clean_scrub_stamp":"2024-09-21T03:35:42.877874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:01:33.365289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-09-21T03:35:43.892156+0000","last_change":"2024-09-21T03:35:43.892156+0000","last_active":"2024-09-21T03:35:43.880613+0000","last_peered":"2024-09-21T03:35:43.880613+0000","last_clean":"2024-09-21T03:35:43.880613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:35:43.892156+0000","last_undegraded":"2024-09-21T03:35:43.892156+0000","last_fullsized":"2024-09-21T03:35:43.892156+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T03:35:44.896884+0000","last_change":"2024-09-21T03:35:43.898720+0000","last_active":"2024-09-21T03:35:44.896884+0000","last_peered":"2024-09-21T03:35:44.896884+0000","last_clean":"2024-09-21T03:35:44.896884+0000","last_became_active":"2024-09-21T03:35:43.898067+0000","last_became_peered":"2024-09-21T03:35:43.898067+0000","last_unstale":"2024-09-21T03:35:44.896884+0000","last_undegraded":"2024-09-21T03:35:44.896884+0000","last_fullsized":"2024-09-21T03:35:44.896884+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_clean_scrub_stamp":"2024-09-21T03:35:42.877874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:45:41.645966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-09-21T03:35:43.895794+0000","last_change":"2024-09-21T03:35:43.895794+0000","last_active":"2024-09-21T03:35:43.880613+0000","last_peered":"2024-09-21T03:35:43.880613+0000","last_clean":"2024-09-21T03:35:43.880613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:35:43.895794+0000","last_undegraded":"2024-09-21T03:35:43.895794+0000","last_fullsized":"2024-09-21T03:35:43.895794+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T03:35:42.877874+0000","last_change":"2024-09-21T03:35:42.877874+0000","last_active":"2024-09-21T03:35:42.877874+0000","last_peered":"2024-09-21T03:35:42.877874+0000","last_clean":"2024-09-21T03:35:42.877874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:35:42.877874+0000","last_undegraded":"2024-09-21T03:35:42.877874+0000","last_fullsized":"2024-09-21T03:35:42.877874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_clean_scrub_stamp":"2024-09-21T03:35:42.877874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":174,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T03:35:44.896975+0000","last_change":"2024-09-21T03:33:25.319776+0000","last_active":"2024-09-21T03:35:44.896975+0000","last_peered":"2024-09-21T03:35:44.896975+0000","last_clean":"2024-09-21T03:35:44.896975+0000","last_became_active":"2024-09-21T03:33:25.318399+0000","last_became_peered":"2024-09-21T03:33:25.318399+0000","last_unstale":"2024-09-21T03:35:44.896975+0000","last_undegraded":"2024-09-21T03:35:44.896975+0000","last_fullsized":"2024-09-21T03:35:44.896975+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:23.016672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:23.016672+0000","last_clean_scrub_stamp":"2024-09-21T03:33:23.016672+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:32:35.906114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":72,"acting":72,"num_store_stats":5},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":52,"acting":52,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":1,"up_from":16,"seq":68719476768,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32380,"kb_used_data":1188,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339460,"statfs":{"total":96636764160,"available":96603607040,"internally_reserved":0,"allocated":1216512,"data_stored":866432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509472,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31788,"kb_used_data":596,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340052,"statfs":{"total":96636764160,"available":96604213248,"internally_reserved":0,"allocated":610304,"data_stored":269397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509471,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31788,"kb_used_data":596,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340052,"statfs":{"total":96636764160,"available":96604213248,"internally_reserved":0,"allocated":610304,"data_stored":269397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509472,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31788,"kb_used_data":596,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340052,"statfs":{"total":96636764160,"available":96604213248,"internally_reserved":0,"allocated":610304,"data_stored":269397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509471,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31788,"kb_used_data":596,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340052,"statfs":{"total":96636764160,"available":96604213248,"internally_reserved":0,"allocated":610304,"data_stored":269397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542175,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35932,"kb_used_data":584,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335908,"statfs":{"total":96636764160,"available":96599969792,"internally_reserved":0,"allocated":598016,"data_stored":263188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574879,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32304,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339536,"statfs":{"total":96636764160,"available":96603684864,"internally_reserved":0,"allocated":1142784,"data_stored":831104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542176,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35932,"kb_used_data":584,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335908,"statfs":{"total":96636764160,"available":96599969792,"internally_reserved":0,"allocated":598016,"data_stored":263188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T03:35:48.979 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T03:35:49.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:49.130+0000 7f6c178ec640 1 -- 172.21.15.38:0/4294309137 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6c1015b5f0 msgr2=0x7f6c1006ff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:49.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:49.130+0000 7f6c178ec640 1 --2- 172.21.15.38:0/4294309137 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6c1015b5f0 0x7f6c1006ff00 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f6c04005fd0 tx=0x7f6c0402a540 comp rx=0 tx=0).stop 2024-09-21T03:35:49.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:49.130+0000 7f6c178ec640 1 -- 172.21.15.38:0/4294309137 shutdown_connections 2024-09-21T03:35:49.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:49.130+0000 7f6c178ec640 1 --2- 172.21.15.38:0/4294309137 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6c1015efb0 0x7f6c10073ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:49.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:49.130+0000 7f6c178ec640 1 --2- 172.21.15.38:0/4294309137 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6c1015b5f0 0x7f6c1006ff00 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:49.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:49.130+0000 7f6c178ec640 1 -- 172.21.15.38:0/4294309137 >> 172.21.15.38:0/4294309137 conn(0x7f6c100d1140 msgr2=0x7f6c100d1550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:49.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:49.130+0000 7f6c178ec640 1 -- 172.21.15.38:0/4294309137 shutdown_connections 2024-09-21T03:35:49.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:49.130+0000 7f6c178ec640 1 -- 172.21.15.38:0/4294309137 wait complete. 2024-09-21T03:35:49.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:49.132+0000 7f6c178ec640 1 Processor -- start 2024-09-21T03:35:49.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:49.132+0000 7f6c178ec640 1 -- start start 2024-09-21T03:35:49.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:49.133+0000 7f6c178ec640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6c100cf130 0x7f6c100ccf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:49.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:49.133+0000 7f6c178ec640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6c1015b5f0 0x7f6c100b6140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:49.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:49.133+0000 7f6c178ec640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6c1015efb0 0x7f6c100b6680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:49.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:49.133+0000 7f6c178ec640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f6c100ae770 con 0x7f6c1015efb0 2024-09-21T03:35:49.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:49.133+0000 7f6c178ec640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f6c100ae5f0 con 0x7f6c100cf130 2024-09-21T03:35:49.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:49.133+0000 7f6c178ec640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f6c100ae8f0 con 0x7f6c1015b5f0 2024-09-21T03:35:49.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:49.134+0000 7f6c15e89640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6c1015efb0 0x7f6c100b6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:49.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:49.134+0000 7f6c15e89640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6c1015efb0 0x7f6c100b6680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40148/0 (socket says 172.21.15.38:40148) 2024-09-21T03:35:49.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:49.134+0000 7f6c15e89640 1 -- 172.21.15.38:0/3723479500 learned_addr learned my addr 172.21.15.38:0/3723479500 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:35:49.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:49.134+0000 7f6c15688640 1 --2- 172.21.15.38:0/3723479500 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6c100cf130 0x7f6c100ccf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:49.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:49.134+0000 7f6c14e87640 1 --2- 172.21.15.38:0/3723479500 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6c1015b5f0 0x7f6c100b6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:49.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:49.134+0000 7f6c15e89640 1 -- 172.21.15.38:0/3723479500 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6c1015b5f0 msgr2=0x7f6c100b6140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:49.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:49.134+0000 7f6c15e89640 1 --2- 172.21.15.38:0/3723479500 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6c1015b5f0 0x7f6c100b6140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:49.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:49.134+0000 7f6c15e89640 1 -- 172.21.15.38:0/3723479500 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6c100cf130 msgr2=0x7f6c100ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:49.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:49.134+0000 7f6c15e89640 1 --2- 172.21.15.38:0/3723479500 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6c100cf130 0x7f6c100ccf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:49.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:49.134+0000 7f6c15e89640 1 -- 172.21.15.38:0/3723479500 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c100cd470 con 0x7f6c1015efb0 2024-09-21T03:35:49.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:49.134+0000 7f6c15688640 1 --2- 172.21.15.38:0/3723479500 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6c100cf130 0x7f6c100ccf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:35:49.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:49.134+0000 7f6c15e89640 1 --2- 172.21.15.38:0/3723479500 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6c1015efb0 0x7f6c100b6680 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f6c0800c3f0 tx=0x7f6c0800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:49.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:49.134+0000 7f6c01ffb640 1 -- 172.21.15.38:0/3723479500 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c08017070 con 0x7f6c1015efb0 2024-09-21T03:35:49.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:49.134+0000 7f6c01ffb640 1 -- 172.21.15.38:0/3723479500 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6c080053c0 con 0x7f6c1015efb0 2024-09-21T03:35:49.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:49.134+0000 7f6c178ec640 1 -- 172.21.15.38:0/3723479500 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c1015bbe0 con 0x7f6c1015efb0 2024-09-21T03:35:49.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:49.134+0000 7f6c01ffb640 1 -- 172.21.15.38:0/3723479500 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c08003850 con 0x7f6c1015efb0 2024-09-21T03:35:49.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:49.135+0000 7f6c178ec640 1 -- 172.21.15.38:0/3723479500 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c1015c1a0 con 0x7f6c1015efb0 2024-09-21T03:35:49.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:49.136+0000 7f6c178ec640 1 -- 172.21.15.38:0/3723479500 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c10074f20 con 0x7f6c1015efb0 2024-09-21T03:35:49.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:49.136+0000 7f6c01ffb640 1 -- 172.21.15.38:0/3723479500 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f6c0801a070 con 0x7f6c1015efb0 2024-09-21T03:35:49.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:49.137+0000 7f6c01ffb640 1 --2- 172.21.15.38:0/3723479500 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6bf4078bc0 0x7f6bf407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:49.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:49.137+0000 7f6c01ffb640 1 -- 172.21.15.38:0/3723479500 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 6101+0+0 (secure 0 0 0) 0x7f6c0809a150 con 0x7f6c1015efb0 2024-09-21T03:35:49.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:49.137+0000 7f6c15688640 1 --2- 172.21.15.38:0/3723479500 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6bf4078bc0 0x7f6bf407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:49.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:49.138+0000 7f6c15688640 1 --2- 172.21.15.38:0/3723479500 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6bf4078bc0 0x7f6bf407b0d0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f6c04033000 tx=0x7f6c04004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:49.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:49.141+0000 7f6c01ffb640 1 -- 172.21.15.38:0/3723479500 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f6c08062490 con 0x7f6c1015efb0 2024-09-21T03:35:49.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:49.325+0000 7f6c178ec640 1 -- 172.21.15.38:0/3723479500 --> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6c1009ede0 con 0x7f6bf4078bc0 2024-09-21T03:35:49.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:49.330+0000 7f6c01ffb640 1 -- 172.21.15.38:0/3723479500 <== mgr.4117 v2:172.21.15.195:6800/1631901344 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+249630 (secure 0 0 0) 0x7f6c1009ede0 con 0x7f6bf4078bc0 2024-09-21T03:35:49.335 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:35:49.336 INFO:teuthology.orchestra.run.smithi038.stderr:dumped all 2024-09-21T03:35:49.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:49.333+0000 7f6c178ec640 1 -- 172.21.15.38:0/3723479500 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6bf4078bc0 msgr2=0x7f6bf407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:49.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:49.333+0000 7f6c178ec640 1 --2- 172.21.15.38:0/3723479500 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6bf4078bc0 0x7f6bf407b0d0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f6c04033000 tx=0x7f6c04004730 comp rx=0 tx=0).stop 2024-09-21T03:35:49.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:49.333+0000 7f6c178ec640 1 -- 172.21.15.38:0/3723479500 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6c1015efb0 msgr2=0x7f6c100b6680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:49.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:49.333+0000 7f6c178ec640 1 --2- 172.21.15.38:0/3723479500 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6c1015efb0 0x7f6c100b6680 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f6c0800c3f0 tx=0x7f6c0800c8b0 comp rx=0 tx=0).stop 2024-09-21T03:35:49.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:49.333+0000 7f6c178ec640 1 -- 172.21.15.38:0/3723479500 shutdown_connections 2024-09-21T03:35:49.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:49.333+0000 7f6c178ec640 1 --2- 172.21.15.38:0/3723479500 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6c1015efb0 0x7f6c100b6680 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:49.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:49.333+0000 7f6c178ec640 1 --2- 172.21.15.38:0/3723479500 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6bf4078bc0 0x7f6bf407b0d0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:49.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:49.333+0000 7f6c178ec640 1 --2- 172.21.15.38:0/3723479500 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6c1015b5f0 0x7f6c100b6140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:49.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:49.333+0000 7f6c178ec640 1 --2- 172.21.15.38:0/3723479500 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6c100cf130 0x7f6c100ccf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:49.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:49.333+0000 7f6c178ec640 1 -- 172.21.15.38:0/3723479500 >> 172.21.15.38:0/3723479500 conn(0x7f6c100d1140 msgr2=0x7f6c100d1550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:49.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:49.333+0000 7f6c178ec640 1 -- 172.21.15.38:0/3723479500 shutdown_connections 2024-09-21T03:35:49.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:49.333+0000 7f6c178ec640 1 -- 172.21.15.38:0/3723479500 wait complete. 2024-09-21T03:35:49.346 INFO:teuthology.orchestra.run.smithi038.stdout:{"pg_ready":true,"pg_map":{"version":121,"stamp":"2024-09-21T03:35:49.013480+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592050,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":113,"num_write_kb":1632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":85,"ondisk_log_size":85,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":99,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":263928,"kb_used_data":6076,"kb_used_omap":12,"kb_used_meta":257651,"kb_avail":754710792,"statfs":{"total":773094113280,"available":772823851008,"internally_reserved":0,"allocated":6221824,"data_stored":3401453,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12748,"internal_metadata":263835188},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1682,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001635"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.869704+0000","last_change":"2024-09-21T03:35:44.918690+0000","last_active":"2024-09-21T03:35:46.869704+0000","last_peered":"2024-09-21T03:35:46.869704+0000","last_clean":"2024-09-21T03:35:46.869704+0000","last_became_active":"2024-09-21T03:35:44.917722+0000","last_became_peered":"2024-09-21T03:35:44.917722+0000","last_unstale":"2024-09-21T03:35:46.869704+0000","last_undegraded":"2024-09-21T03:35:46.869704+0000","last_fullsized":"2024-09-21T03:35:46.869704+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:17:03.401604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-09-21T03:35:43.900135+0000","last_change":"2024-09-21T03:35:43.900135+0000","last_active":"2024-09-21T03:35:43.880613+0000","last_peered":"2024-09-21T03:35:43.880613+0000","last_clean":"2024-09-21T03:35:43.880613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:35:43.900135+0000","last_undegraded":"2024-09-21T03:35:43.900135+0000","last_fullsized":"2024-09-21T03:35:43.900135+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.869706+0000","last_change":"2024-09-21T03:35:44.914485+0000","last_active":"2024-09-21T03:35:46.869706+0000","last_peered":"2024-09-21T03:35:46.869706+0000","last_clean":"2024-09-21T03:35:46.869706+0000","last_became_active":"2024-09-21T03:35:44.914150+0000","last_became_peered":"2024-09-21T03:35:44.914150+0000","last_unstale":"2024-09-21T03:35:46.869706+0000","last_undegraded":"2024-09-21T03:35:46.869706+0000","last_fullsized":"2024-09-21T03:35:46.869706+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:34:38.258070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T03:35:44.927798+0000","last_change":"2024-09-21T03:35:44.921404+0000","last_active":"2024-09-21T03:35:44.927798+0000","last_peered":"2024-09-21T03:35:44.927798+0000","last_clean":"2024-09-21T03:35:44.927798+0000","last_became_active":"2024-09-21T03:35:44.921059+0000","last_became_peered":"2024-09-21T03:35:44.921059+0000","last_unstale":"2024-09-21T03:35:44.927798+0000","last_undegraded":"2024-09-21T03:35:44.927798+0000","last_fullsized":"2024-09-21T03:35:44.927798+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:54:00.361227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-09-21T03:35:43.902970+0000","last_change":"2024-09-21T03:35:43.902970+0000","last_active":"2024-09-21T03:35:43.880613+0000","last_peered":"2024-09-21T03:35:43.880613+0000","last_clean":"2024-09-21T03:35:43.880613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:35:43.902970+0000","last_undegraded":"2024-09-21T03:35:43.902970+0000","last_fullsized":"2024-09-21T03:35:43.902970+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-09-21T03:35:43.889762+0000","last_change":"2024-09-21T03:35:43.889762+0000","last_active":"2024-09-21T03:35:43.880613+0000","last_peered":"2024-09-21T03:35:43.880613+0000","last_clean":"2024-09-21T03:35:43.880613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:35:43.889762+0000","last_undegraded":"2024-09-21T03:35:43.889762+0000","last_fullsized":"2024-09-21T03:35:43.889762+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.680170+0000","last_change":"2024-09-21T03:35:44.900285+0000","last_active":"2024-09-21T03:35:46.680170+0000","last_peered":"2024-09-21T03:35:46.680170+0000","last_clean":"2024-09-21T03:35:46.680170+0000","last_became_active":"2024-09-21T03:35:44.899955+0000","last_became_peered":"2024-09-21T03:35:44.899955+0000","last_unstale":"2024-09-21T03:35:46.680170+0000","last_undegraded":"2024-09-21T03:35:46.680170+0000","last_fullsized":"2024-09-21T03:35:46.680170+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:44:47.487941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-09-21T03:35:43.889893+0000","last_change":"2024-09-21T03:35:43.889893+0000","last_active":"2024-09-21T03:35:43.880613+0000","last_peered":"2024-09-21T03:35:43.880613+0000","last_clean":"2024-09-21T03:35:43.880613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:35:43.889893+0000","last_undegraded":"2024-09-21T03:35:43.889893+0000","last_fullsized":"2024-09-21T03:35:43.889893+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.869989+0000","last_change":"2024-09-21T03:35:44.923776+0000","last_active":"2024-09-21T03:35:46.869989+0000","last_peered":"2024-09-21T03:35:46.869989+0000","last_clean":"2024-09-21T03:35:46.869989+0000","last_became_active":"2024-09-21T03:35:44.923315+0000","last_became_peered":"2024-09-21T03:35:44.923315+0000","last_unstale":"2024-09-21T03:35:46.869989+0000","last_undegraded":"2024-09-21T03:35:46.869989+0000","last_fullsized":"2024-09-21T03:35:46.869989+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:05:17.646813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.680010+0000","last_change":"2024-09-21T03:35:44.906104+0000","last_active":"2024-09-21T03:35:46.680010+0000","last_peered":"2024-09-21T03:35:46.680010+0000","last_clean":"2024-09-21T03:35:46.680010+0000","last_became_active":"2024-09-21T03:35:44.905786+0000","last_became_peered":"2024-09-21T03:35:44.905786+0000","last_unstale":"2024-09-21T03:35:46.680010+0000","last_undegraded":"2024-09-21T03:35:46.680010+0000","last_fullsized":"2024-09-21T03:35:46.680010+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:56:36.052470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.869616+0000","last_change":"2024-09-21T03:35:44.913177+0000","last_active":"2024-09-21T03:35:46.869616+0000","last_peered":"2024-09-21T03:35:46.869616+0000","last_clean":"2024-09-21T03:35:46.869616+0000","last_became_active":"2024-09-21T03:35:44.912856+0000","last_became_peered":"2024-09-21T03:35:44.912856+0000","last_unstale":"2024-09-21T03:35:46.869616+0000","last_undegraded":"2024-09-21T03:35:46.869616+0000","last_fullsized":"2024-09-21T03:35:46.869616+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:53:31.119799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-09-21T03:35:43.889542+0000","last_change":"2024-09-21T03:35:43.889542+0000","last_active":"2024-09-21T03:35:43.880613+0000","last_peered":"2024-09-21T03:35:43.880613+0000","last_clean":"2024-09-21T03:35:43.880613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:35:43.889542+0000","last_undegraded":"2024-09-21T03:35:43.889542+0000","last_fullsized":"2024-09-21T03:35:43.889542+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-09-21T03:35:43.890378+0000","last_change":"2024-09-21T03:35:43.890378+0000","last_active":"2024-09-21T03:35:43.880613+0000","last_peered":"2024-09-21T03:35:43.880613+0000","last_clean":"2024-09-21T03:35:43.880613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:35:43.890378+0000","last_undegraded":"2024-09-21T03:35:43.890378+0000","last_fullsized":"2024-09-21T03:35:43.890378+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.868924+0000","last_change":"2024-09-21T03:35:44.916342+0000","last_active":"2024-09-21T03:35:46.868924+0000","last_peered":"2024-09-21T03:35:46.868924+0000","last_clean":"2024-09-21T03:35:46.868924+0000","last_became_active":"2024-09-21T03:35:44.916047+0000","last_became_peered":"2024-09-21T03:35:44.916047+0000","last_unstale":"2024-09-21T03:35:46.868924+0000","last_undegraded":"2024-09-21T03:35:46.868924+0000","last_fullsized":"2024-09-21T03:35:46.868924+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:12:22.459838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T03:35:44.922283+0000","last_change":"2024-09-21T03:35:44.919326+0000","last_active":"2024-09-21T03:35:44.922283+0000","last_peered":"2024-09-21T03:35:44.922283+0000","last_clean":"2024-09-21T03:35:44.922283+0000","last_became_active":"2024-09-21T03:35:44.919004+0000","last_became_peered":"2024-09-21T03:35:44.919004+0000","last_unstale":"2024-09-21T03:35:44.922283+0000","last_undegraded":"2024-09-21T03:35:44.922283+0000","last_fullsized":"2024-09-21T03:35:44.922283+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:31:52.864107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-09-21T03:35:43.889143+0000","last_change":"2024-09-21T03:35:43.889143+0000","last_active":"2024-09-21T03:35:43.880613+0000","last_peered":"2024-09-21T03:35:43.880613+0000","last_clean":"2024-09-21T03:35:43.880613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:35:43.889143+0000","last_undegraded":"2024-09-21T03:35:43.889143+0000","last_fullsized":"2024-09-21T03:35:43.889143+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.870146+0000","last_change":"2024-09-21T03:35:44.918344+0000","last_active":"2024-09-21T03:35:46.870146+0000","last_peered":"2024-09-21T03:35:46.870146+0000","last_clean":"2024-09-21T03:35:46.870146+0000","last_became_active":"2024-09-21T03:35:44.917344+0000","last_became_peered":"2024-09-21T03:35:44.917344+0000","last_unstale":"2024-09-21T03:35:46.870146+0000","last_undegraded":"2024-09-21T03:35:46.870146+0000","last_fullsized":"2024-09-21T03:35:46.870146+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:10:22.417344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T03:35:44.924513+0000","last_change":"2024-09-21T03:35:44.920057+0000","last_active":"2024-09-21T03:35:44.924513+0000","last_peered":"2024-09-21T03:35:44.924513+0000","last_clean":"2024-09-21T03:35:44.924513+0000","last_became_active":"2024-09-21T03:35:44.919683+0000","last_became_peered":"2024-09-21T03:35:44.919683+0000","last_unstale":"2024-09-21T03:35:44.924513+0000","last_undegraded":"2024-09-21T03:35:44.924513+0000","last_fullsized":"2024-09-21T03:35:44.924513+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:05:30.105550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.869952+0000","last_change":"2024-09-21T03:35:44.919698+0000","last_active":"2024-09-21T03:35:46.869952+0000","last_peered":"2024-09-21T03:35:46.869952+0000","last_clean":"2024-09-21T03:35:46.869952+0000","last_became_active":"2024-09-21T03:35:44.919400+0000","last_became_peered":"2024-09-21T03:35:44.919400+0000","last_unstale":"2024-09-21T03:35:46.869952+0000","last_undegraded":"2024-09-21T03:35:46.869952+0000","last_fullsized":"2024-09-21T03:35:46.869952+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:18:19.307626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-09-21T03:35:43.902374+0000","last_change":"2024-09-21T03:35:43.902374+0000","last_active":"2024-09-21T03:35:43.880613+0000","last_peered":"2024-09-21T03:35:43.880613+0000","last_clean":"2024-09-21T03:35:43.880613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:35:43.902374+0000","last_undegraded":"2024-09-21T03:35:43.902374+0000","last_fullsized":"2024-09-21T03:35:43.902374+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T03:35:44.920375+0000","last_change":"2024-09-21T03:35:44.915023+0000","last_active":"2024-09-21T03:35:44.920375+0000","last_peered":"2024-09-21T03:35:44.920375+0000","last_clean":"2024-09-21T03:35:44.920375+0000","last_became_active":"2024-09-21T03:35:44.914677+0000","last_became_peered":"2024-09-21T03:35:44.914677+0000","last_unstale":"2024-09-21T03:35:44.920375+0000","last_undegraded":"2024-09-21T03:35:44.920375+0000","last_fullsized":"2024-09-21T03:35:44.920375+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:41:13.888920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.868808+0000","last_change":"2024-09-21T03:35:44.922185+0000","last_active":"2024-09-21T03:35:46.868808+0000","last_peered":"2024-09-21T03:35:46.868808+0000","last_clean":"2024-09-21T03:35:46.868808+0000","last_became_active":"2024-09-21T03:35:44.921473+0000","last_became_peered":"2024-09-21T03:35:44.921473+0000","last_unstale":"2024-09-21T03:35:46.868808+0000","last_undegraded":"2024-09-21T03:35:46.868808+0000","last_fullsized":"2024-09-21T03:35:46.868808+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:10:57.276638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.869035+0000","last_change":"2024-09-21T03:35:44.916166+0000","last_active":"2024-09-21T03:35:46.869035+0000","last_peered":"2024-09-21T03:35:46.869035+0000","last_clean":"2024-09-21T03:35:46.869035+0000","last_became_active":"2024-09-21T03:35:44.915841+0000","last_became_peered":"2024-09-21T03:35:44.915841+0000","last_unstale":"2024-09-21T03:35:46.869035+0000","last_undegraded":"2024-09-21T03:35:46.869035+0000","last_fullsized":"2024-09-21T03:35:46.869035+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:58:35.540873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.868743+0000","last_change":"2024-09-21T03:35:44.915458+0000","last_active":"2024-09-21T03:35:46.868743+0000","last_peered":"2024-09-21T03:35:46.868743+0000","last_clean":"2024-09-21T03:35:46.868743+0000","last_became_active":"2024-09-21T03:35:44.915163+0000","last_became_peered":"2024-09-21T03:35:44.915163+0000","last_unstale":"2024-09-21T03:35:46.868743+0000","last_undegraded":"2024-09-21T03:35:46.868743+0000","last_fullsized":"2024-09-21T03:35:46.868743+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:03:55.703336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.870005+0000","last_change":"2024-09-21T03:35:44.918018+0000","last_active":"2024-09-21T03:35:46.870005+0000","last_peered":"2024-09-21T03:35:46.870005+0000","last_clean":"2024-09-21T03:35:46.870005+0000","last_became_active":"2024-09-21T03:35:44.916939+0000","last_became_peered":"2024-09-21T03:35:44.916939+0000","last_unstale":"2024-09-21T03:35:46.870005+0000","last_undegraded":"2024-09-21T03:35:46.870005+0000","last_fullsized":"2024-09-21T03:35:46.870005+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:32:35.988080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.869597+0000","last_change":"2024-09-21T03:35:44.904810+0000","last_active":"2024-09-21T03:35:46.869597+0000","last_peered":"2024-09-21T03:35:46.869597+0000","last_clean":"2024-09-21T03:35:46.869597+0000","last_became_active":"2024-09-21T03:35:44.904499+0000","last_became_peered":"2024-09-21T03:35:44.904499+0000","last_unstale":"2024-09-21T03:35:46.869597+0000","last_undegraded":"2024-09-21T03:35:46.869597+0000","last_fullsized":"2024-09-21T03:35:46.869597+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:37:49.682407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-09-21T03:35:43.892475+0000","last_change":"2024-09-21T03:35:43.892475+0000","last_active":"2024-09-21T03:35:43.880613+0000","last_peered":"2024-09-21T03:35:43.880613+0000","last_clean":"2024-09-21T03:35:43.880613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:35:43.892475+0000","last_undegraded":"2024-09-21T03:35:43.892475+0000","last_fullsized":"2024-09-21T03:35:43.892475+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-09-21T03:35:43.901876+0000","last_change":"2024-09-21T03:35:43.901876+0000","last_active":"2024-09-21T03:35:43.880613+0000","last_peered":"2024-09-21T03:35:43.880613+0000","last_clean":"2024-09-21T03:35:43.880613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:35:43.901876+0000","last_undegraded":"2024-09-21T03:35:43.901876+0000","last_fullsized":"2024-09-21T03:35:43.901876+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.869246+0000","last_change":"2024-09-21T03:35:44.913781+0000","last_active":"2024-09-21T03:35:46.869246+0000","last_peered":"2024-09-21T03:35:46.869246+0000","last_clean":"2024-09-21T03:35:46.869246+0000","last_became_active":"2024-09-21T03:35:44.913469+0000","last_became_peered":"2024-09-21T03:35:44.913469+0000","last_unstale":"2024-09-21T03:35:46.869246+0000","last_undegraded":"2024-09-21T03:35:46.869246+0000","last_fullsized":"2024-09-21T03:35:46.869246+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:52:54.152973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-09-21T03:35:43.896302+0000","last_change":"2024-09-21T03:35:43.896302+0000","last_active":"2024-09-21T03:35:43.880613+0000","last_peered":"2024-09-21T03:35:43.880613+0000","last_clean":"2024-09-21T03:35:43.880613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:35:43.896302+0000","last_undegraded":"2024-09-21T03:35:43.896302+0000","last_fullsized":"2024-09-21T03:35:43.896302+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.868885+0000","last_change":"2024-09-21T03:35:44.920858+0000","last_active":"2024-09-21T03:35:46.868885+0000","last_peered":"2024-09-21T03:35:46.868885+0000","last_clean":"2024-09-21T03:35:46.868885+0000","last_became_active":"2024-09-21T03:35:44.920539+0000","last_became_peered":"2024-09-21T03:35:44.920539+0000","last_unstale":"2024-09-21T03:35:46.868885+0000","last_undegraded":"2024-09-21T03:35:46.868885+0000","last_fullsized":"2024-09-21T03:35:46.868885+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:09:20.328763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-09-21T03:35:43.899921+0000","last_change":"2024-09-21T03:35:43.899921+0000","last_active":"2024-09-21T03:35:43.880613+0000","last_peered":"2024-09-21T03:35:43.880613+0000","last_clean":"2024-09-21T03:35:43.880613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:35:43.899921+0000","last_undegraded":"2024-09-21T03:35:43.899921+0000","last_fullsized":"2024-09-21T03:35:43.899921+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-09-21T03:35:43.897228+0000","last_change":"2024-09-21T03:35:43.897228+0000","last_active":"2024-09-21T03:35:43.880613+0000","last_peered":"2024-09-21T03:35:43.880613+0000","last_clean":"2024-09-21T03:35:43.880613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:35:43.897228+0000","last_undegraded":"2024-09-21T03:35:43.897228+0000","last_fullsized":"2024-09-21T03:35:43.897228+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-21T03:35:43.911675+0000","last_change":"2024-09-21T03:35:43.909828+0000","last_active":"2024-09-21T03:35:43.911675+0000","last_peered":"2024-09-21T03:35:43.911675+0000","last_clean":"2024-09-21T03:35:43.911675+0000","last_became_active":"2024-09-21T03:35:43.909503+0000","last_became_peered":"2024-09-21T03:35:43.909503+0000","last_unstale":"2024-09-21T03:35:43.911675+0000","last_undegraded":"2024-09-21T03:35:43.911675+0000","last_fullsized":"2024-09-21T03:35:43.911675+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_clean_scrub_stamp":"2024-09-21T03:35:42.877874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:35:28.406113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-09-21T03:35:43.895909+0000","last_change":"2024-09-21T03:35:43.895909+0000","last_active":"2024-09-21T03:35:43.880613+0000","last_peered":"2024-09-21T03:35:43.880613+0000","last_clean":"2024-09-21T03:35:43.880613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:35:43.895909+0000","last_undegraded":"2024-09-21T03:35:43.895909+0000","last_fullsized":"2024-09-21T03:35:43.895909+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-21T03:35:43.902645+0000","last_change":"2024-09-21T03:35:43.901432+0000","last_active":"2024-09-21T03:35:43.902645+0000","last_peered":"2024-09-21T03:35:43.902645+0000","last_clean":"2024-09-21T03:35:43.902645+0000","last_became_active":"2024-09-21T03:35:43.901134+0000","last_became_peered":"2024-09-21T03:35:43.901134+0000","last_unstale":"2024-09-21T03:35:43.902645+0000","last_undegraded":"2024-09-21T03:35:43.902645+0000","last_fullsized":"2024-09-21T03:35:43.902645+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_clean_scrub_stamp":"2024-09-21T03:35:42.877874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:42:45.225890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-09-21T03:35:43.897697+0000","last_change":"2024-09-21T03:35:43.897697+0000","last_active":"2024-09-21T03:35:43.880613+0000","last_peered":"2024-09-21T03:35:43.880613+0000","last_clean":"2024-09-21T03:35:43.880613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:35:43.897697+0000","last_undegraded":"2024-09-21T03:35:43.897697+0000","last_fullsized":"2024-09-21T03:35:43.897697+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1d","version":"54'2","reported_seq":17,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.680073+0000","last_change":"2024-09-21T03:35:43.902853+0000","last_active":"2024-09-21T03:35:46.680073+0000","last_peered":"2024-09-21T03:35:46.680073+0000","last_clean":"2024-09-21T03:35:46.680073+0000","last_became_active":"2024-09-21T03:35:43.902514+0000","last_became_peered":"2024-09-21T03:35:43.902514+0000","last_unstale":"2024-09-21T03:35:46.680073+0000","last_undegraded":"2024-09-21T03:35:46.680073+0000","last_fullsized":"2024-09-21T03:35:46.680073+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_clean_scrub_stamp":"2024-09-21T03:35:42.877874+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:40:01.918835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.680012+0000","last_change":"2024-09-21T03:35:44.912392+0000","last_active":"2024-09-21T03:35:46.680012+0000","last_peered":"2024-09-21T03:35:46.680012+0000","last_clean":"2024-09-21T03:35:46.680012+0000","last_became_active":"2024-09-21T03:35:44.911332+0000","last_became_peered":"2024-09-21T03:35:44.911332+0000","last_unstale":"2024-09-21T03:35:46.680012+0000","last_undegraded":"2024-09-21T03:35:46.680012+0000","last_fullsized":"2024-09-21T03:35:46.680012+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:16:36.470418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-21T03:35:43.916772+0000","last_change":"2024-09-21T03:35:43.911405+0000","last_active":"2024-09-21T03:35:43.916772+0000","last_peered":"2024-09-21T03:35:43.916772+0000","last_clean":"2024-09-21T03:35:43.916772+0000","last_became_active":"2024-09-21T03:35:43.911082+0000","last_became_peered":"2024-09-21T03:35:43.911082+0000","last_unstale":"2024-09-21T03:35:43.916772+0000","last_undegraded":"2024-09-21T03:35:43.916772+0000","last_fullsized":"2024-09-21T03:35:43.916772+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_clean_scrub_stamp":"2024-09-21T03:35:42.877874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:40:34.993739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-09-21T03:35:43.895551+0000","last_change":"2024-09-21T03:35:43.895551+0000","last_active":"2024-09-21T03:35:43.880613+0000","last_peered":"2024-09-21T03:35:43.880613+0000","last_clean":"2024-09-21T03:35:43.880613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:35:43.895551+0000","last_undegraded":"2024-09-21T03:35:43.895551+0000","last_fullsized":"2024-09-21T03:35:43.895551+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1b","version":"54'1","reported_seq":16,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.868936+0000","last_change":"2024-09-21T03:35:43.909887+0000","last_active":"2024-09-21T03:35:46.868936+0000","last_peered":"2024-09-21T03:35:46.868936+0000","last_clean":"2024-09-21T03:35:46.868936+0000","last_became_active":"2024-09-21T03:35:43.909558+0000","last_became_peered":"2024-09-21T03:35:43.909558+0000","last_unstale":"2024-09-21T03:35:46.868936+0000","last_undegraded":"2024-09-21T03:35:46.868936+0000","last_fullsized":"2024-09-21T03:35:46.868936+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_clean_scrub_stamp":"2024-09-21T03:35:42.877874+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:03:39.525599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-09-21T03:35:43.897379+0000","last_change":"2024-09-21T03:35:43.897379+0000","last_active":"2024-09-21T03:35:43.880613+0000","last_peered":"2024-09-21T03:35:43.880613+0000","last_clean":"2024-09-21T03:35:43.880613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:35:43.897379+0000","last_undegraded":"2024-09-21T03:35:43.897379+0000","last_fullsized":"2024-09-21T03:35:43.897379+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-21T03:35:43.915560+0000","last_change":"2024-09-21T03:35:43.910078+0000","last_active":"2024-09-21T03:35:43.915560+0000","last_peered":"2024-09-21T03:35:43.915560+0000","last_clean":"2024-09-21T03:35:43.915560+0000","last_became_active":"2024-09-21T03:35:43.909315+0000","last_became_peered":"2024-09-21T03:35:43.909315+0000","last_unstale":"2024-09-21T03:35:43.915560+0000","last_undegraded":"2024-09-21T03:35:43.915560+0000","last_fullsized":"2024-09-21T03:35:43.915560+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_clean_scrub_stamp":"2024-09-21T03:35:42.877874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:48:07.945139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.680104+0000","last_change":"2024-09-21T03:35:44.910050+0000","last_active":"2024-09-21T03:35:46.680104+0000","last_peered":"2024-09-21T03:35:46.680104+0000","last_clean":"2024-09-21T03:35:46.680104+0000","last_became_active":"2024-09-21T03:35:44.909755+0000","last_became_peered":"2024-09-21T03:35:44.909755+0000","last_unstale":"2024-09-21T03:35:46.680104+0000","last_undegraded":"2024-09-21T03:35:46.680104+0000","last_fullsized":"2024-09-21T03:35:46.680104+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:30:38.153649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-21T03:35:43.915132+0000","last_change":"2024-09-21T03:35:43.909144+0000","last_active":"2024-09-21T03:35:43.915132+0000","last_peered":"2024-09-21T03:35:43.915132+0000","last_clean":"2024-09-21T03:35:43.915132+0000","last_became_active":"2024-09-21T03:35:43.908808+0000","last_became_peered":"2024-09-21T03:35:43.908808+0000","last_unstale":"2024-09-21T03:35:43.915132+0000","last_undegraded":"2024-09-21T03:35:43.915132+0000","last_fullsized":"2024-09-21T03:35:43.915132+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_clean_scrub_stamp":"2024-09-21T03:35:42.877874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:51:41.557674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T03:35:44.913427+0000","last_change":"2024-09-21T03:35:44.907438+0000","last_active":"2024-09-21T03:35:44.913427+0000","last_peered":"2024-09-21T03:35:44.913427+0000","last_clean":"2024-09-21T03:35:44.913427+0000","last_became_active":"2024-09-21T03:35:44.907114+0000","last_became_peered":"2024-09-21T03:35:44.907114+0000","last_unstale":"2024-09-21T03:35:44.913427+0000","last_undegraded":"2024-09-21T03:35:44.913427+0000","last_fullsized":"2024-09-21T03:35:44.913427+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:44:34.145008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":15,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.868893+0000","last_change":"2024-09-21T03:35:43.903257+0000","last_active":"2024-09-21T03:35:46.868893+0000","last_peered":"2024-09-21T03:35:46.868893+0000","last_clean":"2024-09-21T03:35:46.868893+0000","last_became_active":"2024-09-21T03:35:43.902926+0000","last_became_peered":"2024-09-21T03:35:43.902926+0000","last_unstale":"2024-09-21T03:35:46.868893+0000","last_undegraded":"2024-09-21T03:35:46.868893+0000","last_fullsized":"2024-09-21T03:35:46.868893+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_clean_scrub_stamp":"2024-09-21T03:35:42.877874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:30:24.926072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-09-21T03:35:43.895500+0000","last_change":"2024-09-21T03:35:43.895500+0000","last_active":"2024-09-21T03:35:43.880613+0000","last_peered":"2024-09-21T03:35:43.880613+0000","last_clean":"2024-09-21T03:35:43.880613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:35:43.895500+0000","last_undegraded":"2024-09-21T03:35:43.895500+0000","last_fullsized":"2024-09-21T03:35:43.895500+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T03:35:44.896174+0000","last_change":"2024-09-21T03:35:43.905541+0000","last_active":"2024-09-21T03:35:44.896174+0000","last_peered":"2024-09-21T03:35:44.896174+0000","last_clean":"2024-09-21T03:35:44.896174+0000","last_became_active":"2024-09-21T03:35:43.905230+0000","last_became_peered":"2024-09-21T03:35:43.905230+0000","last_unstale":"2024-09-21T03:35:44.896174+0000","last_undegraded":"2024-09-21T03:35:44.896174+0000","last_fullsized":"2024-09-21T03:35:44.896174+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_clean_scrub_stamp":"2024-09-21T03:35:42.877874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:46:28.427572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T03:35:44.927406+0000","last_change":"2024-09-21T03:35:44.920481+0000","last_active":"2024-09-21T03:35:44.927406+0000","last_peered":"2024-09-21T03:35:44.927406+0000","last_clean":"2024-09-21T03:35:44.927406+0000","last_became_active":"2024-09-21T03:35:44.919780+0000","last_became_peered":"2024-09-21T03:35:44.919780+0000","last_unstale":"2024-09-21T03:35:44.927406+0000","last_undegraded":"2024-09-21T03:35:44.927406+0000","last_fullsized":"2024-09-21T03:35:44.927406+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:33:00.469260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T03:35:44.898827+0000","last_change":"2024-09-21T03:35:43.898522+0000","last_active":"2024-09-21T03:35:44.898827+0000","last_peered":"2024-09-21T03:35:44.898827+0000","last_clean":"2024-09-21T03:35:44.898827+0000","last_became_active":"2024-09-21T03:35:43.898171+0000","last_became_peered":"2024-09-21T03:35:43.898171+0000","last_unstale":"2024-09-21T03:35:44.898827+0000","last_undegraded":"2024-09-21T03:35:44.898827+0000","last_fullsized":"2024-09-21T03:35:44.898827+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_clean_scrub_stamp":"2024-09-21T03:35:42.877874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:14:03.947289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.869721+0000","last_change":"2024-09-21T03:35:44.921020+0000","last_active":"2024-09-21T03:35:46.869721+0000","last_peered":"2024-09-21T03:35:46.869721+0000","last_clean":"2024-09-21T03:35:46.869721+0000","last_became_active":"2024-09-21T03:35:44.920573+0000","last_became_peered":"2024-09-21T03:35:44.920573+0000","last_unstale":"2024-09-21T03:35:46.869721+0000","last_undegraded":"2024-09-21T03:35:46.869721+0000","last_fullsized":"2024-09-21T03:35:46.869721+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:17:42.161554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-21T03:35:43.915078+0000","last_change":"2024-09-21T03:35:43.907104+0000","last_active":"2024-09-21T03:35:43.915078+0000","last_peered":"2024-09-21T03:35:43.915078+0000","last_clean":"2024-09-21T03:35:43.915078+0000","last_became_active":"2024-09-21T03:35:43.906758+0000","last_became_peered":"2024-09-21T03:35:43.906758+0000","last_unstale":"2024-09-21T03:35:43.915078+0000","last_undegraded":"2024-09-21T03:35:43.915078+0000","last_fullsized":"2024-09-21T03:35:43.915078+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_clean_scrub_stamp":"2024-09-21T03:35:42.877874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:53:55.235170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-09-21T03:35:43.903519+0000","last_change":"2024-09-21T03:35:43.903519+0000","last_active":"2024-09-21T03:35:43.880613+0000","last_peered":"2024-09-21T03:35:43.880613+0000","last_clean":"2024-09-21T03:35:43.880613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:35:43.903519+0000","last_undegraded":"2024-09-21T03:35:43.903519+0000","last_fullsized":"2024-09-21T03:35:43.903519+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.14","version":"54'6","reported_seq":21,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.679907+0000","last_change":"2024-09-21T03:35:43.905841+0000","last_active":"2024-09-21T03:35:46.679907+0000","last_peered":"2024-09-21T03:35:46.679907+0000","last_clean":"2024-09-21T03:35:46.679907+0000","last_became_active":"2024-09-21T03:35:43.905505+0000","last_became_peered":"2024-09-21T03:35:43.905505+0000","last_unstale":"2024-09-21T03:35:46.679907+0000","last_undegraded":"2024-09-21T03:35:46.679907+0000","last_fullsized":"2024-09-21T03:35:46.679907+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_clean_scrub_stamp":"2024-09-21T03:35:42.877874+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:40:06.133533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-09-21T03:35:43.896352+0000","last_change":"2024-09-21T03:35:43.896352+0000","last_active":"2024-09-21T03:35:43.880613+0000","last_peered":"2024-09-21T03:35:43.880613+0000","last_clean":"2024-09-21T03:35:43.880613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:35:43.896352+0000","last_undegraded":"2024-09-21T03:35:43.896352+0000","last_fullsized":"2024-09-21T03:35:43.896352+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-21T03:35:43.913708+0000","last_change":"2024-09-21T03:35:43.913254+0000","last_active":"2024-09-21T03:35:43.913708+0000","last_peered":"2024-09-21T03:35:43.913708+0000","last_clean":"2024-09-21T03:35:43.913708+0000","last_became_active":"2024-09-21T03:35:43.912392+0000","last_became_peered":"2024-09-21T03:35:43.912392+0000","last_unstale":"2024-09-21T03:35:43.913708+0000","last_undegraded":"2024-09-21T03:35:43.913708+0000","last_fullsized":"2024-09-21T03:35:43.913708+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_clean_scrub_stamp":"2024-09-21T03:35:42.877874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:53:16.732262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.869864+0000","last_change":"2024-09-21T03:35:44.906977+0000","last_active":"2024-09-21T03:35:46.869864+0000","last_peered":"2024-09-21T03:35:46.869864+0000","last_clean":"2024-09-21T03:35:46.869864+0000","last_became_active":"2024-09-21T03:35:44.906651+0000","last_became_peered":"2024-09-21T03:35:44.906651+0000","last_unstale":"2024-09-21T03:35:46.869864+0000","last_undegraded":"2024-09-21T03:35:46.869864+0000","last_fullsized":"2024-09-21T03:35:46.869864+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:13:46.431913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T03:35:44.895492+0000","last_change":"2024-09-21T03:35:43.897065+0000","last_active":"2024-09-21T03:35:44.895492+0000","last_peered":"2024-09-21T03:35:44.895492+0000","last_clean":"2024-09-21T03:35:44.895492+0000","last_became_active":"2024-09-21T03:35:43.896727+0000","last_became_peered":"2024-09-21T03:35:43.896727+0000","last_unstale":"2024-09-21T03:35:44.895492+0000","last_undegraded":"2024-09-21T03:35:44.895492+0000","last_fullsized":"2024-09-21T03:35:44.895492+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_clean_scrub_stamp":"2024-09-21T03:35:42.877874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:57:57.601786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-09-21T03:35:43.892997+0000","last_change":"2024-09-21T03:35:43.892997+0000","last_active":"2024-09-21T03:35:43.880613+0000","last_peered":"2024-09-21T03:35:43.880613+0000","last_clean":"2024-09-21T03:35:43.880613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:35:43.892997+0000","last_undegraded":"2024-09-21T03:35:43.892997+0000","last_fullsized":"2024-09-21T03:35:43.892997+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-21T03:35:43.915446+0000","last_change":"2024-09-21T03:35:43.910589+0000","last_active":"2024-09-21T03:35:43.915446+0000","last_peered":"2024-09-21T03:35:43.915446+0000","last_clean":"2024-09-21T03:35:43.915446+0000","last_became_active":"2024-09-21T03:35:43.910150+0000","last_became_peered":"2024-09-21T03:35:43.910150+0000","last_unstale":"2024-09-21T03:35:43.915446+0000","last_undegraded":"2024-09-21T03:35:43.915446+0000","last_fullsized":"2024-09-21T03:35:43.915446+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_clean_scrub_stamp":"2024-09-21T03:35:42.877874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:47:52.583294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.870055+0000","last_change":"2024-09-21T03:35:44.916344+0000","last_active":"2024-09-21T03:35:46.870055+0000","last_peered":"2024-09-21T03:35:46.870055+0000","last_clean":"2024-09-21T03:35:46.870055+0000","last_became_active":"2024-09-21T03:35:44.916019+0000","last_became_peered":"2024-09-21T03:35:44.916019+0000","last_unstale":"2024-09-21T03:35:46.870055+0000","last_undegraded":"2024-09-21T03:35:46.870055+0000","last_fullsized":"2024-09-21T03:35:46.870055+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:58:47.408064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-21T03:35:43.909227+0000","last_change":"2024-09-21T03:35:43.905763+0000","last_active":"2024-09-21T03:35:43.909227+0000","last_peered":"2024-09-21T03:35:43.909227+0000","last_clean":"2024-09-21T03:35:43.909227+0000","last_became_active":"2024-09-21T03:35:43.905453+0000","last_became_peered":"2024-09-21T03:35:43.905453+0000","last_unstale":"2024-09-21T03:35:43.909227+0000","last_undegraded":"2024-09-21T03:35:43.909227+0000","last_fullsized":"2024-09-21T03:35:43.909227+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_clean_scrub_stamp":"2024-09-21T03:35:42.877874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:12:58.212405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.679983+0000","last_change":"2024-09-21T03:35:44.917548+0000","last_active":"2024-09-21T03:35:46.679983+0000","last_peered":"2024-09-21T03:35:46.679983+0000","last_clean":"2024-09-21T03:35:46.679983+0000","last_became_active":"2024-09-21T03:35:44.916979+0000","last_became_peered":"2024-09-21T03:35:44.916979+0000","last_unstale":"2024-09-21T03:35:46.679983+0000","last_undegraded":"2024-09-21T03:35:46.679983+0000","last_fullsized":"2024-09-21T03:35:46.679983+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:25:12.342791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-21T03:35:43.901848+0000","last_change":"2024-09-21T03:35:43.900447+0000","last_active":"2024-09-21T03:35:43.901848+0000","last_peered":"2024-09-21T03:35:43.901848+0000","last_clean":"2024-09-21T03:35:43.901848+0000","last_became_active":"2024-09-21T03:35:43.900127+0000","last_became_peered":"2024-09-21T03:35:43.900127+0000","last_unstale":"2024-09-21T03:35:43.901848+0000","last_undegraded":"2024-09-21T03:35:43.901848+0000","last_fullsized":"2024-09-21T03:35:43.901848+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_clean_scrub_stamp":"2024-09-21T03:35:42.877874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:20:29.206321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.869128+0000","last_change":"2024-09-21T03:35:44.910826+0000","last_active":"2024-09-21T03:35:46.869128+0000","last_peered":"2024-09-21T03:35:46.869128+0000","last_clean":"2024-09-21T03:35:46.869128+0000","last_became_active":"2024-09-21T03:35:44.910528+0000","last_became_peered":"2024-09-21T03:35:44.910528+0000","last_unstale":"2024-09-21T03:35:46.869128+0000","last_undegraded":"2024-09-21T03:35:46.869128+0000","last_fullsized":"2024-09-21T03:35:46.869128+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:00:58.755293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-21T03:35:43.909265+0000","last_change":"2024-09-21T03:35:43.906381+0000","last_active":"2024-09-21T03:35:43.909265+0000","last_peered":"2024-09-21T03:35:43.909265+0000","last_clean":"2024-09-21T03:35:43.909265+0000","last_became_active":"2024-09-21T03:35:43.905982+0000","last_became_peered":"2024-09-21T03:35:43.905982+0000","last_unstale":"2024-09-21T03:35:43.909265+0000","last_undegraded":"2024-09-21T03:35:43.909265+0000","last_fullsized":"2024-09-21T03:35:43.909265+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_clean_scrub_stamp":"2024-09-21T03:35:42.877874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:32:38.146616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-09-21T03:35:43.900034+0000","last_change":"2024-09-21T03:35:43.900034+0000","last_active":"2024-09-21T03:35:43.880613+0000","last_peered":"2024-09-21T03:35:43.880613+0000","last_clean":"2024-09-21T03:35:43.880613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:35:43.900034+0000","last_undegraded":"2024-09-21T03:35:43.900034+0000","last_fullsized":"2024-09-21T03:35:43.900034+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.d","version":"54'1","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T03:35:44.915599+0000","last_change":"2024-09-21T03:35:43.899938+0000","last_active":"2024-09-21T03:35:44.915599+0000","last_peered":"2024-09-21T03:35:44.915599+0000","last_clean":"2024-09-21T03:35:44.915599+0000","last_became_active":"2024-09-21T03:35:43.899617+0000","last_became_peered":"2024-09-21T03:35:43.899617+0000","last_unstale":"2024-09-21T03:35:44.915599+0000","last_undegraded":"2024-09-21T03:35:44.915599+0000","last_fullsized":"2024-09-21T03:35:44.915599+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_clean_scrub_stamp":"2024-09-21T03:35:42.877874+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:54:01.776464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-09-21T03:35:43.890997+0000","last_change":"2024-09-21T03:35:43.890997+0000","last_active":"2024-09-21T03:35:43.880613+0000","last_peered":"2024-09-21T03:35:43.880613+0000","last_clean":"2024-09-21T03:35:43.880613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:35:43.890997+0000","last_undegraded":"2024-09-21T03:35:43.890997+0000","last_fullsized":"2024-09-21T03:35:43.890997+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-21T03:35:43.916804+0000","last_change":"2024-09-21T03:35:43.911542+0000","last_active":"2024-09-21T03:35:43.916804+0000","last_peered":"2024-09-21T03:35:43.916804+0000","last_clean":"2024-09-21T03:35:43.916804+0000","last_became_active":"2024-09-21T03:35:43.911246+0000","last_became_peered":"2024-09-21T03:35:43.911246+0000","last_unstale":"2024-09-21T03:35:43.916804+0000","last_undegraded":"2024-09-21T03:35:43.916804+0000","last_fullsized":"2024-09-21T03:35:43.916804+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_clean_scrub_stamp":"2024-09-21T03:35:42.877874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:23:28.021867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-09-21T03:35:43.901415+0000","last_change":"2024-09-21T03:35:43.901415+0000","last_active":"2024-09-21T03:35:43.880613+0000","last_peered":"2024-09-21T03:35:43.880613+0000","last_clean":"2024-09-21T03:35:43.880613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:35:43.901415+0000","last_undegraded":"2024-09-21T03:35:43.901415+0000","last_fullsized":"2024-09-21T03:35:43.901415+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.b","version":"54'2","reported_seq":17,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.869325+0000","last_change":"2024-09-21T03:35:43.912355+0000","last_active":"2024-09-21T03:35:46.869325+0000","last_peered":"2024-09-21T03:35:46.869325+0000","last_clean":"2024-09-21T03:35:46.869325+0000","last_became_active":"2024-09-21T03:35:43.912034+0000","last_became_peered":"2024-09-21T03:35:43.912034+0000","last_unstale":"2024-09-21T03:35:46.869325+0000","last_undegraded":"2024-09-21T03:35:46.869325+0000","last_fullsized":"2024-09-21T03:35:46.869325+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_clean_scrub_stamp":"2024-09-21T03:35:42.877874+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:13:47.067340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T03:35:44.915374+0000","last_change":"2024-09-21T03:35:44.911441+0000","last_active":"2024-09-21T03:35:44.915374+0000","last_peered":"2024-09-21T03:35:44.915374+0000","last_clean":"2024-09-21T03:35:44.915374+0000","last_became_active":"2024-09-21T03:35:44.911116+0000","last_became_peered":"2024-09-21T03:35:44.911116+0000","last_unstale":"2024-09-21T03:35:44.915374+0000","last_undegraded":"2024-09-21T03:35:44.915374+0000","last_fullsized":"2024-09-21T03:35:44.915374+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:00:19.050385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":15,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.868615+0000","last_change":"2024-09-21T03:35:43.909812+0000","last_active":"2024-09-21T03:35:46.868615+0000","last_peered":"2024-09-21T03:35:46.868615+0000","last_clean":"2024-09-21T03:35:46.868615+0000","last_became_active":"2024-09-21T03:35:43.909494+0000","last_became_peered":"2024-09-21T03:35:43.909494+0000","last_unstale":"2024-09-21T03:35:46.868615+0000","last_undegraded":"2024-09-21T03:35:46.868615+0000","last_fullsized":"2024-09-21T03:35:46.868615+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_clean_scrub_stamp":"2024-09-21T03:35:42.877874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:30:08.445515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.869281+0000","last_change":"2024-09-21T03:35:44.906754+0000","last_active":"2024-09-21T03:35:46.869281+0000","last_peered":"2024-09-21T03:35:46.869281+0000","last_clean":"2024-09-21T03:35:46.869281+0000","last_became_active":"2024-09-21T03:35:44.906423+0000","last_became_peered":"2024-09-21T03:35:44.906423+0000","last_unstale":"2024-09-21T03:35:46.869281+0000","last_undegraded":"2024-09-21T03:35:46.869281+0000","last_fullsized":"2024-09-21T03:35:46.869281+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:50:12.528499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":15,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.679811+0000","last_change":"2024-09-21T03:35:43.905895+0000","last_active":"2024-09-21T03:35:46.679811+0000","last_peered":"2024-09-21T03:35:46.679811+0000","last_clean":"2024-09-21T03:35:46.679811+0000","last_became_active":"2024-09-21T03:35:43.905606+0000","last_became_peered":"2024-09-21T03:35:43.905606+0000","last_unstale":"2024-09-21T03:35:46.679811+0000","last_undegraded":"2024-09-21T03:35:46.679811+0000","last_fullsized":"2024-09-21T03:35:46.679811+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_clean_scrub_stamp":"2024-09-21T03:35:42.877874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:50:41.525440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-09-21T03:35:43.891647+0000","last_change":"2024-09-21T03:35:43.891647+0000","last_active":"2024-09-21T03:35:43.880613+0000","last_peered":"2024-09-21T03:35:43.880613+0000","last_clean":"2024-09-21T03:35:43.880613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:35:43.891647+0000","last_undegraded":"2024-09-21T03:35:43.891647+0000","last_fullsized":"2024-09-21T03:35:43.891647+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":15,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.679686+0000","last_change":"2024-09-21T03:35:43.899860+0000","last_active":"2024-09-21T03:35:46.679686+0000","last_peered":"2024-09-21T03:35:46.679686+0000","last_clean":"2024-09-21T03:35:46.679686+0000","last_became_active":"2024-09-21T03:35:43.899517+0000","last_became_peered":"2024-09-21T03:35:43.899517+0000","last_unstale":"2024-09-21T03:35:46.679686+0000","last_undegraded":"2024-09-21T03:35:46.679686+0000","last_fullsized":"2024-09-21T03:35:46.679686+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_clean_scrub_stamp":"2024-09-21T03:35:42.877874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:23:04.412064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T03:35:44.908986+0000","last_change":"2024-09-21T03:35:44.907182+0000","last_active":"2024-09-21T03:35:44.908986+0000","last_peered":"2024-09-21T03:35:44.908986+0000","last_clean":"2024-09-21T03:35:44.908986+0000","last_became_active":"2024-09-21T03:35:44.906783+0000","last_became_peered":"2024-09-21T03:35:44.906783+0000","last_unstale":"2024-09-21T03:35:44.908986+0000","last_undegraded":"2024-09-21T03:35:44.908986+0000","last_fullsized":"2024-09-21T03:35:44.908986+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:00:17.389518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-21T03:35:43.915543+0000","last_change":"2024-09-21T03:35:43.910068+0000","last_active":"2024-09-21T03:35:43.915543+0000","last_peered":"2024-09-21T03:35:43.915543+0000","last_clean":"2024-09-21T03:35:43.915543+0000","last_became_active":"2024-09-21T03:35:43.908871+0000","last_became_peered":"2024-09-21T03:35:43.908871+0000","last_unstale":"2024-09-21T03:35:43.915543+0000","last_undegraded":"2024-09-21T03:35:43.915543+0000","last_fullsized":"2024-09-21T03:35:43.915543+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_clean_scrub_stamp":"2024-09-21T03:35:42.877874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:42:59.626174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.680044+0000","last_change":"2024-09-21T03:35:44.912419+0000","last_active":"2024-09-21T03:35:46.680044+0000","last_peered":"2024-09-21T03:35:46.680044+0000","last_clean":"2024-09-21T03:35:46.680044+0000","last_became_active":"2024-09-21T03:35:44.912135+0000","last_became_peered":"2024-09-21T03:35:44.912135+0000","last_unstale":"2024-09-21T03:35:46.680044+0000","last_undegraded":"2024-09-21T03:35:46.680044+0000","last_fullsized":"2024-09-21T03:35:46.680044+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:00:55.093451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-21T03:35:43.915020+0000","last_change":"2024-09-21T03:35:43.904698+0000","last_active":"2024-09-21T03:35:43.915020+0000","last_peered":"2024-09-21T03:35:43.915020+0000","last_clean":"2024-09-21T03:35:43.915020+0000","last_became_active":"2024-09-21T03:35:43.904398+0000","last_became_peered":"2024-09-21T03:35:43.904398+0000","last_unstale":"2024-09-21T03:35:43.915020+0000","last_undegraded":"2024-09-21T03:35:43.915020+0000","last_fullsized":"2024-09-21T03:35:43.915020+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_clean_scrub_stamp":"2024-09-21T03:35:42.877874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:37:46.304367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.868840+0000","last_change":"2024-09-21T03:35:44.917013+0000","last_active":"2024-09-21T03:35:46.868840+0000","last_peered":"2024-09-21T03:35:46.868840+0000","last_clean":"2024-09-21T03:35:46.868840+0000","last_became_active":"2024-09-21T03:35:44.916698+0000","last_became_peered":"2024-09-21T03:35:44.916698+0000","last_unstale":"2024-09-21T03:35:46.868840+0000","last_undegraded":"2024-09-21T03:35:46.868840+0000","last_fullsized":"2024-09-21T03:35:46.868840+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:11:45.208479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":15,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.869306+0000","last_change":"2024-09-21T03:35:43.911358+0000","last_active":"2024-09-21T03:35:46.869306+0000","last_peered":"2024-09-21T03:35:46.869306+0000","last_clean":"2024-09-21T03:35:46.869306+0000","last_became_active":"2024-09-21T03:35:43.911032+0000","last_became_peered":"2024-09-21T03:35:43.911032+0000","last_unstale":"2024-09-21T03:35:46.869306+0000","last_undegraded":"2024-09-21T03:35:46.869306+0000","last_fullsized":"2024-09-21T03:35:46.869306+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_clean_scrub_stamp":"2024-09-21T03:35:42.877874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:06:46.333955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.679989+0000","last_change":"2024-09-21T03:35:44.902610+0000","last_active":"2024-09-21T03:35:46.679989+0000","last_peered":"2024-09-21T03:35:46.679989+0000","last_clean":"2024-09-21T03:35:46.679989+0000","last_became_active":"2024-09-21T03:35:44.902286+0000","last_became_peered":"2024-09-21T03:35:44.902286+0000","last_unstale":"2024-09-21T03:35:46.679989+0000","last_undegraded":"2024-09-21T03:35:46.679989+0000","last_fullsized":"2024-09-21T03:35:46.679989+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:20:06.254030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":15,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.679843+0000","last_change":"2024-09-21T03:35:43.911600+0000","last_active":"2024-09-21T03:35:46.679843+0000","last_peered":"2024-09-21T03:35:46.679843+0000","last_clean":"2024-09-21T03:35:46.679843+0000","last_became_active":"2024-09-21T03:35:43.911278+0000","last_became_peered":"2024-09-21T03:35:43.911278+0000","last_unstale":"2024-09-21T03:35:46.679843+0000","last_undegraded":"2024-09-21T03:35:46.679843+0000","last_fullsized":"2024-09-21T03:35:46.679843+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_clean_scrub_stamp":"2024-09-21T03:35:42.877874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:32:48.236015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-09-21T03:35:43.900410+0000","last_change":"2024-09-21T03:35:43.900410+0000","last_active":"2024-09-21T03:35:43.880613+0000","last_peered":"2024-09-21T03:35:43.880613+0000","last_clean":"2024-09-21T03:35:43.880613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:35:43.900410+0000","last_undegraded":"2024-09-21T03:35:43.900410+0000","last_fullsized":"2024-09-21T03:35:43.900410+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":15,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.868734+0000","last_change":"2024-09-21T03:35:43.909531+0000","last_active":"2024-09-21T03:35:46.868734+0000","last_peered":"2024-09-21T03:35:46.868734+0000","last_clean":"2024-09-21T03:35:46.868734+0000","last_became_active":"2024-09-21T03:35:43.909013+0000","last_became_peered":"2024-09-21T03:35:43.909013+0000","last_unstale":"2024-09-21T03:35:46.868734+0000","last_undegraded":"2024-09-21T03:35:46.868734+0000","last_fullsized":"2024-09-21T03:35:46.868734+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_clean_scrub_stamp":"2024-09-21T03:35:42.877874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:17:00.326757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.680299+0000","last_change":"2024-09-21T03:35:44.909262+0000","last_active":"2024-09-21T03:35:46.680299+0000","last_peered":"2024-09-21T03:35:46.680299+0000","last_clean":"2024-09-21T03:35:46.680299+0000","last_became_active":"2024-09-21T03:35:44.908944+0000","last_became_peered":"2024-09-21T03:35:44.908944+0000","last_unstale":"2024-09-21T03:35:46.680299+0000","last_undegraded":"2024-09-21T03:35:46.680299+0000","last_fullsized":"2024-09-21T03:35:46.680299+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:17:43.817629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-21T03:35:43.916795+0000","last_change":"2024-09-21T03:35:43.912067+0000","last_active":"2024-09-21T03:35:43.916795+0000","last_peered":"2024-09-21T03:35:43.916795+0000","last_clean":"2024-09-21T03:35:43.916795+0000","last_became_active":"2024-09-21T03:35:43.911556+0000","last_became_peered":"2024-09-21T03:35:43.911556+0000","last_unstale":"2024-09-21T03:35:43.916795+0000","last_undegraded":"2024-09-21T03:35:43.916795+0000","last_fullsized":"2024-09-21T03:35:43.916795+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_clean_scrub_stamp":"2024-09-21T03:35:42.877874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:01:33.365289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-09-21T03:35:43.892156+0000","last_change":"2024-09-21T03:35:43.892156+0000","last_active":"2024-09-21T03:35:43.880613+0000","last_peered":"2024-09-21T03:35:43.880613+0000","last_clean":"2024-09-21T03:35:43.880613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:35:43.892156+0000","last_undegraded":"2024-09-21T03:35:43.892156+0000","last_fullsized":"2024-09-21T03:35:43.892156+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T03:35:44.896884+0000","last_change":"2024-09-21T03:35:43.898720+0000","last_active":"2024-09-21T03:35:44.896884+0000","last_peered":"2024-09-21T03:35:44.896884+0000","last_clean":"2024-09-21T03:35:44.896884+0000","last_became_active":"2024-09-21T03:35:43.898067+0000","last_became_peered":"2024-09-21T03:35:43.898067+0000","last_unstale":"2024-09-21T03:35:44.896884+0000","last_undegraded":"2024-09-21T03:35:44.896884+0000","last_fullsized":"2024-09-21T03:35:44.896884+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_clean_scrub_stamp":"2024-09-21T03:35:42.877874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:45:41.645966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-09-21T03:35:43.895794+0000","last_change":"2024-09-21T03:35:43.895794+0000","last_active":"2024-09-21T03:35:43.880613+0000","last_peered":"2024-09-21T03:35:43.880613+0000","last_clean":"2024-09-21T03:35:43.880613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:35:43.895794+0000","last_undegraded":"2024-09-21T03:35:43.895794+0000","last_fullsized":"2024-09-21T03:35:43.895794+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"54'1","reported_seq":16,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.680261+0000","last_change":"2024-09-21T03:35:43.903173+0000","last_active":"2024-09-21T03:35:46.680261+0000","last_peered":"2024-09-21T03:35:46.680261+0000","last_clean":"2024-09-21T03:35:46.680261+0000","last_became_active":"2024-09-21T03:35:43.902852+0000","last_became_peered":"2024-09-21T03:35:43.902852+0000","last_unstale":"2024-09-21T03:35:46.680261+0000","last_undegraded":"2024-09-21T03:35:46.680261+0000","last_fullsized":"2024-09-21T03:35:46.680261+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_clean_scrub_stamp":"2024-09-21T03:35:42.877874+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:55:49.604908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":174,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T03:35:44.896975+0000","last_change":"2024-09-21T03:33:25.319776+0000","last_active":"2024-09-21T03:35:44.896975+0000","last_peered":"2024-09-21T03:35:44.896975+0000","last_clean":"2024-09-21T03:35:44.896975+0000","last_became_active":"2024-09-21T03:33:25.318399+0000","last_became_peered":"2024-09-21T03:33:25.318399+0000","last_unstale":"2024-09-21T03:35:44.896975+0000","last_undegraded":"2024-09-21T03:35:44.896975+0000","last_fullsized":"2024-09-21T03:35:44.896975+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:23.016672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:23.016672+0000","last_clean_scrub_stamp":"2024-09-21T03:33:23.016672+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:32:35.906114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":1682,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":4058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":13,"ondisk_log_size":13,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":1,"up_from":16,"seq":68719476768,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32380,"kb_used_data":1188,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339460,"statfs":{"total":96636764160,"available":96603607040,"internally_reserved":0,"allocated":1216512,"data_stored":866432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509472,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31788,"kb_used_data":596,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340052,"statfs":{"total":96636764160,"available":96604213248,"internally_reserved":0,"allocated":610304,"data_stored":269397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509471,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31788,"kb_used_data":596,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340052,"statfs":{"total":96636764160,"available":96604213248,"internally_reserved":0,"allocated":610304,"data_stored":269397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509472,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31788,"kb_used_data":596,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340052,"statfs":{"total":96636764160,"available":96604213248,"internally_reserved":0,"allocated":610304,"data_stored":269397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509471,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31788,"kb_used_data":596,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340052,"statfs":{"total":96636764160,"available":96604213248,"internally_reserved":0,"allocated":610304,"data_stored":269397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542176,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35984,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335856,"statfs":{"total":96636764160,"available":96599916544,"internally_reserved":0,"allocated":651264,"data_stored":290505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574880,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32412,"kb_used_data":1216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339428,"statfs":{"total":96636764160,"available":96603574272,"internally_reserved":0,"allocated":1245184,"data_stored":875301,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542177,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36000,"kb_used_data":652,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335840,"statfs":{"total":96636764160,"available":96599900160,"internally_reserved":0,"allocated":667648,"data_stored":291627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T03:35:49.350 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T03:35:49.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:49.506+0000 7f053fef2640 1 -- 172.21.15.38:0/876549984 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0538162f00 msgr2=0x7f05380d24d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:49.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:49.506+0000 7f053fef2640 1 --2- 172.21.15.38:0/876549984 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0538162f00 0x7f05380d24d0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f05380d0c00 tx=0x7f053402a560 comp rx=0 tx=0).stop 2024-09-21T03:35:49.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:49.506+0000 7f053fef2640 1 -- 172.21.15.38:0/876549984 shutdown_connections 2024-09-21T03:35:49.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:49.506+0000 7f053fef2640 1 --2- 172.21.15.38:0/876549984 >> v2:172.21.15.195:3300/0 conn(0x7f05380b2e40 0x7f05380b3250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:49.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:49.506+0000 7f053fef2640 1 --2- 172.21.15.38:0/876549984 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0538162f00 0x7f05380d24d0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:49.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:49.506+0000 7f053fef2640 1 -- 172.21.15.38:0/876549984 >> 172.21.15.38:0/876549984 conn(0x7f05380b6b20 msgr2=0x7f05380d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:49.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:49.506+0000 7f053fef2640 1 -- 172.21.15.38:0/876549984 shutdown_connections 2024-09-21T03:35:49.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:49.506+0000 7f053fef2640 1 -- 172.21.15.38:0/876549984 wait complete. 2024-09-21T03:35:49.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:49.508+0000 7f053fef2640 1 Processor -- start 2024-09-21T03:35:49.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:49.508+0000 7f053fef2640 1 -- start start 2024-09-21T03:35:49.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:49.509+0000 7f053fef2640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f05380b2e40 0x7f053814b710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:49.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:49.509+0000 7f053fef2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0538162280 0x7f053814e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:49.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:49.509+0000 7f053fef2640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f05380ced50 0x7f05380abf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:49.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:49.509+0000 7f053fef2640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f053815e4b0 con 0x7f0538162280 2024-09-21T03:35:49.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:49.509+0000 7f053fef2640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f05380b39a0 con 0x7f05380b2e40 2024-09-21T03:35:49.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:49.509+0000 7f053fef2640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f053815e630 con 0x7f05380ced50 2024-09-21T03:35:49.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:49.509+0000 7f053d48d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0538162280 0x7f053814e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:49.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:49.509+0000 7f053d48d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0538162280 0x7f053814e0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40160/0 (socket says 172.21.15.38:40160) 2024-09-21T03:35:49.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:49.509+0000 7f053d48d640 1 -- 172.21.15.38:0/3273447981 learned_addr learned my addr 172.21.15.38:0/3273447981 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:35:49.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:49.509+0000 7f053dc8e640 1 --2- 172.21.15.38:0/3273447981 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f05380b2e40 0x7f053814b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:49.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:49.509+0000 7f053e48f640 1 --2- 172.21.15.38:0/3273447981 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f05380ced50 0x7f05380abf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:49.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:49.509+0000 7f053d48d640 1 -- 172.21.15.38:0/3273447981 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f05380ced50 msgr2=0x7f05380abf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:49.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:49.509+0000 7f053d48d640 1 --2- 172.21.15.38:0/3273447981 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f05380ced50 0x7f05380abf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:49.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:49.509+0000 7f053d48d640 1 -- 172.21.15.38:0/3273447981 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f05380b2e40 msgr2=0x7f053814b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:49.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:49.509+0000 7f053d48d640 1 --2- 172.21.15.38:0/3273447981 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f05380b2e40 0x7f053814b710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:49.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:49.509+0000 7f053d48d640 1 -- 172.21.15.38:0/3273447981 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f053814e5f0 con 0x7f0538162280 2024-09-21T03:35:49.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:49.509+0000 7f053dc8e640 1 --2- 172.21.15.38:0/3273447981 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f05380b2e40 0x7f053814b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:35:49.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:49.510+0000 7f053d48d640 1 --2- 172.21.15.38:0/3273447981 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0538162280 0x7f053814e0b0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f052800c3a0 tx=0x7f052800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:49.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:49.510+0000 7f052e7fc640 1 -- 172.21.15.38:0/3273447981 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0528017070 con 0x7f0538162280 2024-09-21T03:35:49.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:49.510+0000 7f052e7fc640 1 -- 172.21.15.38:0/3273447981 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f05280053c0 con 0x7f0538162280 2024-09-21T03:35:49.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:49.510+0000 7f053fef2640 1 -- 172.21.15.38:0/3273447981 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05380aae60 con 0x7f0538162280 2024-09-21T03:35:49.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:49.510+0000 7f052e7fc640 1 -- 172.21.15.38:0/3273447981 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0528003850 con 0x7f0538162280 2024-09-21T03:35:49.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:49.510+0000 7f053fef2640 1 -- 172.21.15.38:0/3273447981 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05380ab420 con 0x7f0538162280 2024-09-21T03:35:49.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:49.511+0000 7f053fef2640 1 -- 172.21.15.38:0/3273447981 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0538162f00 con 0x7f0538162280 2024-09-21T03:35:49.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:49.512+0000 7f052e7fc640 1 -- 172.21.15.38:0/3273447981 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f052801a070 con 0x7f0538162280 2024-09-21T03:35:49.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:49.512+0000 7f052e7fc640 1 --2- 172.21.15.38:0/3273447981 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f051c078bc0 0x7f051c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:49.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:49.513+0000 7f052e7fc640 1 -- 172.21.15.38:0/3273447981 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 6101+0+0 (secure 0 0 0) 0x7f052809a310 con 0x7f0538162280 2024-09-21T03:35:49.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:49.513+0000 7f053dc8e640 1 --2- 172.21.15.38:0/3273447981 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f051c078bc0 0x7f051c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:49.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:49.513+0000 7f053dc8e640 1 --2- 172.21.15.38:0/3273447981 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f051c078bc0 0x7f051c07b0d0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f0534033000 tx=0x7f0534004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:49.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:49.517+0000 7f052e7fc640 1 -- 172.21.15.38:0/3273447981 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f0528062520 con 0x7f0538162280 2024-09-21T03:35:49.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:49.700+0000 7f053fef2640 1 -- 172.21.15.38:0/3273447981 --> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f05380b90d0 con 0x7f051c078bc0 2024-09-21T03:35:49.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:49.705+0000 7f052e7fc640 1 -- 172.21.15.38:0/3273447981 <== mgr.4117 v2:172.21.15.195:6800/1631901344 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+249630 (secure 0 0 0) 0x7f05380b90d0 con 0x7f051c078bc0 2024-09-21T03:35:49.708 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:35:49.711 INFO:teuthology.orchestra.run.smithi038.stderr:dumped all 2024-09-21T03:35:49.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:49.708+0000 7f053fef2640 1 -- 172.21.15.38:0/3273447981 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f051c078bc0 msgr2=0x7f051c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:49.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:49.708+0000 7f053fef2640 1 --2- 172.21.15.38:0/3273447981 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f051c078bc0 0x7f051c07b0d0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f0534033000 tx=0x7f0534004730 comp rx=0 tx=0).stop 2024-09-21T03:35:49.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:49.708+0000 7f053fef2640 1 -- 172.21.15.38:0/3273447981 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0538162280 msgr2=0x7f053814e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:49.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:49.708+0000 7f053fef2640 1 --2- 172.21.15.38:0/3273447981 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0538162280 0x7f053814e0b0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f052800c3a0 tx=0x7f052800c860 comp rx=0 tx=0).stop 2024-09-21T03:35:49.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:49.708+0000 7f053fef2640 1 -- 172.21.15.38:0/3273447981 shutdown_connections 2024-09-21T03:35:49.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:49.708+0000 7f053fef2640 1 --2- 172.21.15.38:0/3273447981 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f051c078bc0 0x7f051c07b0d0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:49.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:49.708+0000 7f053fef2640 1 --2- 172.21.15.38:0/3273447981 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f05380ced50 0x7f05380abf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:49.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:49.708+0000 7f053fef2640 1 --2- 172.21.15.38:0/3273447981 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0538162280 0x7f053814e0b0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:49.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:49.708+0000 7f053fef2640 1 --2- 172.21.15.38:0/3273447981 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f05380b2e40 0x7f053814b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:49.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:49.708+0000 7f053fef2640 1 -- 172.21.15.38:0/3273447981 >> 172.21.15.38:0/3273447981 conn(0x7f05380b6b20 msgr2=0x7f053806e100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:49.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:49.708+0000 7f053fef2640 1 -- 172.21.15.38:0/3273447981 shutdown_connections 2024-09-21T03:35:49.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:49.708+0000 7f053fef2640 1 -- 172.21.15.38:0/3273447981 wait complete. 2024-09-21T03:35:49.721 INFO:teuthology.orchestra.run.smithi038.stdout:{"pg_ready":true,"pg_map":{"version":121,"stamp":"2024-09-21T03:35:49.013480+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592050,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":113,"num_write_kb":1632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":85,"ondisk_log_size":85,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":99,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":263928,"kb_used_data":6076,"kb_used_omap":12,"kb_used_meta":257651,"kb_avail":754710792,"statfs":{"total":773094113280,"available":772823851008,"internally_reserved":0,"allocated":6221824,"data_stored":3401453,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12748,"internal_metadata":263835188},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1682,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001635"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.869704+0000","last_change":"2024-09-21T03:35:44.918690+0000","last_active":"2024-09-21T03:35:46.869704+0000","last_peered":"2024-09-21T03:35:46.869704+0000","last_clean":"2024-09-21T03:35:46.869704+0000","last_became_active":"2024-09-21T03:35:44.917722+0000","last_became_peered":"2024-09-21T03:35:44.917722+0000","last_unstale":"2024-09-21T03:35:46.869704+0000","last_undegraded":"2024-09-21T03:35:46.869704+0000","last_fullsized":"2024-09-21T03:35:46.869704+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:17:03.401604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-09-21T03:35:43.900135+0000","last_change":"2024-09-21T03:35:43.900135+0000","last_active":"2024-09-21T03:35:43.880613+0000","last_peered":"2024-09-21T03:35:43.880613+0000","last_clean":"2024-09-21T03:35:43.880613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:35:43.900135+0000","last_undegraded":"2024-09-21T03:35:43.900135+0000","last_fullsized":"2024-09-21T03:35:43.900135+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.869706+0000","last_change":"2024-09-21T03:35:44.914485+0000","last_active":"2024-09-21T03:35:46.869706+0000","last_peered":"2024-09-21T03:35:46.869706+0000","last_clean":"2024-09-21T03:35:46.869706+0000","last_became_active":"2024-09-21T03:35:44.914150+0000","last_became_peered":"2024-09-21T03:35:44.914150+0000","last_unstale":"2024-09-21T03:35:46.869706+0000","last_undegraded":"2024-09-21T03:35:46.869706+0000","last_fullsized":"2024-09-21T03:35:46.869706+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:34:38.258070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T03:35:44.927798+0000","last_change":"2024-09-21T03:35:44.921404+0000","last_active":"2024-09-21T03:35:44.927798+0000","last_peered":"2024-09-21T03:35:44.927798+0000","last_clean":"2024-09-21T03:35:44.927798+0000","last_became_active":"2024-09-21T03:35:44.921059+0000","last_became_peered":"2024-09-21T03:35:44.921059+0000","last_unstale":"2024-09-21T03:35:44.927798+0000","last_undegraded":"2024-09-21T03:35:44.927798+0000","last_fullsized":"2024-09-21T03:35:44.927798+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:54:00.361227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-09-21T03:35:43.902970+0000","last_change":"2024-09-21T03:35:43.902970+0000","last_active":"2024-09-21T03:35:43.880613+0000","last_peered":"2024-09-21T03:35:43.880613+0000","last_clean":"2024-09-21T03:35:43.880613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:35:43.902970+0000","last_undegraded":"2024-09-21T03:35:43.902970+0000","last_fullsized":"2024-09-21T03:35:43.902970+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-09-21T03:35:43.889762+0000","last_change":"2024-09-21T03:35:43.889762+0000","last_active":"2024-09-21T03:35:43.880613+0000","last_peered":"2024-09-21T03:35:43.880613+0000","last_clean":"2024-09-21T03:35:43.880613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:35:43.889762+0000","last_undegraded":"2024-09-21T03:35:43.889762+0000","last_fullsized":"2024-09-21T03:35:43.889762+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.680170+0000","last_change":"2024-09-21T03:35:44.900285+0000","last_active":"2024-09-21T03:35:46.680170+0000","last_peered":"2024-09-21T03:35:46.680170+0000","last_clean":"2024-09-21T03:35:46.680170+0000","last_became_active":"2024-09-21T03:35:44.899955+0000","last_became_peered":"2024-09-21T03:35:44.899955+0000","last_unstale":"2024-09-21T03:35:46.680170+0000","last_undegraded":"2024-09-21T03:35:46.680170+0000","last_fullsized":"2024-09-21T03:35:46.680170+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:44:47.487941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-09-21T03:35:43.889893+0000","last_change":"2024-09-21T03:35:43.889893+0000","last_active":"2024-09-21T03:35:43.880613+0000","last_peered":"2024-09-21T03:35:43.880613+0000","last_clean":"2024-09-21T03:35:43.880613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:35:43.889893+0000","last_undegraded":"2024-09-21T03:35:43.889893+0000","last_fullsized":"2024-09-21T03:35:43.889893+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.869989+0000","last_change":"2024-09-21T03:35:44.923776+0000","last_active":"2024-09-21T03:35:46.869989+0000","last_peered":"2024-09-21T03:35:46.869989+0000","last_clean":"2024-09-21T03:35:46.869989+0000","last_became_active":"2024-09-21T03:35:44.923315+0000","last_became_peered":"2024-09-21T03:35:44.923315+0000","last_unstale":"2024-09-21T03:35:46.869989+0000","last_undegraded":"2024-09-21T03:35:46.869989+0000","last_fullsized":"2024-09-21T03:35:46.869989+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:05:17.646813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.680010+0000","last_change":"2024-09-21T03:35:44.906104+0000","last_active":"2024-09-21T03:35:46.680010+0000","last_peered":"2024-09-21T03:35:46.680010+0000","last_clean":"2024-09-21T03:35:46.680010+0000","last_became_active":"2024-09-21T03:35:44.905786+0000","last_became_peered":"2024-09-21T03:35:44.905786+0000","last_unstale":"2024-09-21T03:35:46.680010+0000","last_undegraded":"2024-09-21T03:35:46.680010+0000","last_fullsized":"2024-09-21T03:35:46.680010+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:56:36.052470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.869616+0000","last_change":"2024-09-21T03:35:44.913177+0000","last_active":"2024-09-21T03:35:46.869616+0000","last_peered":"2024-09-21T03:35:46.869616+0000","last_clean":"2024-09-21T03:35:46.869616+0000","last_became_active":"2024-09-21T03:35:44.912856+0000","last_became_peered":"2024-09-21T03:35:44.912856+0000","last_unstale":"2024-09-21T03:35:46.869616+0000","last_undegraded":"2024-09-21T03:35:46.869616+0000","last_fullsized":"2024-09-21T03:35:46.869616+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:53:31.119799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-09-21T03:35:43.889542+0000","last_change":"2024-09-21T03:35:43.889542+0000","last_active":"2024-09-21T03:35:43.880613+0000","last_peered":"2024-09-21T03:35:43.880613+0000","last_clean":"2024-09-21T03:35:43.880613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:35:43.889542+0000","last_undegraded":"2024-09-21T03:35:43.889542+0000","last_fullsized":"2024-09-21T03:35:43.889542+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-09-21T03:35:43.890378+0000","last_change":"2024-09-21T03:35:43.890378+0000","last_active":"2024-09-21T03:35:43.880613+0000","last_peered":"2024-09-21T03:35:43.880613+0000","last_clean":"2024-09-21T03:35:43.880613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:35:43.890378+0000","last_undegraded":"2024-09-21T03:35:43.890378+0000","last_fullsized":"2024-09-21T03:35:43.890378+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.868924+0000","last_change":"2024-09-21T03:35:44.916342+0000","last_active":"2024-09-21T03:35:46.868924+0000","last_peered":"2024-09-21T03:35:46.868924+0000","last_clean":"2024-09-21T03:35:46.868924+0000","last_became_active":"2024-09-21T03:35:44.916047+0000","last_became_peered":"2024-09-21T03:35:44.916047+0000","last_unstale":"2024-09-21T03:35:46.868924+0000","last_undegraded":"2024-09-21T03:35:46.868924+0000","last_fullsized":"2024-09-21T03:35:46.868924+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:12:22.459838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T03:35:44.922283+0000","last_change":"2024-09-21T03:35:44.919326+0000","last_active":"2024-09-21T03:35:44.922283+0000","last_peered":"2024-09-21T03:35:44.922283+0000","last_clean":"2024-09-21T03:35:44.922283+0000","last_became_active":"2024-09-21T03:35:44.919004+0000","last_became_peered":"2024-09-21T03:35:44.919004+0000","last_unstale":"2024-09-21T03:35:44.922283+0000","last_undegraded":"2024-09-21T03:35:44.922283+0000","last_fullsized":"2024-09-21T03:35:44.922283+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:31:52.864107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-09-21T03:35:43.889143+0000","last_change":"2024-09-21T03:35:43.889143+0000","last_active":"2024-09-21T03:35:43.880613+0000","last_peered":"2024-09-21T03:35:43.880613+0000","last_clean":"2024-09-21T03:35:43.880613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:35:43.889143+0000","last_undegraded":"2024-09-21T03:35:43.889143+0000","last_fullsized":"2024-09-21T03:35:43.889143+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.870146+0000","last_change":"2024-09-21T03:35:44.918344+0000","last_active":"2024-09-21T03:35:46.870146+0000","last_peered":"2024-09-21T03:35:46.870146+0000","last_clean":"2024-09-21T03:35:46.870146+0000","last_became_active":"2024-09-21T03:35:44.917344+0000","last_became_peered":"2024-09-21T03:35:44.917344+0000","last_unstale":"2024-09-21T03:35:46.870146+0000","last_undegraded":"2024-09-21T03:35:46.870146+0000","last_fullsized":"2024-09-21T03:35:46.870146+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:10:22.417344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T03:35:44.924513+0000","last_change":"2024-09-21T03:35:44.920057+0000","last_active":"2024-09-21T03:35:44.924513+0000","last_peered":"2024-09-21T03:35:44.924513+0000","last_clean":"2024-09-21T03:35:44.924513+0000","last_became_active":"2024-09-21T03:35:44.919683+0000","last_became_peered":"2024-09-21T03:35:44.919683+0000","last_unstale":"2024-09-21T03:35:44.924513+0000","last_undegraded":"2024-09-21T03:35:44.924513+0000","last_fullsized":"2024-09-21T03:35:44.924513+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:05:30.105550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.869952+0000","last_change":"2024-09-21T03:35:44.919698+0000","last_active":"2024-09-21T03:35:46.869952+0000","last_peered":"2024-09-21T03:35:46.869952+0000","last_clean":"2024-09-21T03:35:46.869952+0000","last_became_active":"2024-09-21T03:35:44.919400+0000","last_became_peered":"2024-09-21T03:35:44.919400+0000","last_unstale":"2024-09-21T03:35:46.869952+0000","last_undegraded":"2024-09-21T03:35:46.869952+0000","last_fullsized":"2024-09-21T03:35:46.869952+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:18:19.307626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-09-21T03:35:43.902374+0000","last_change":"2024-09-21T03:35:43.902374+0000","last_active":"2024-09-21T03:35:43.880613+0000","last_peered":"2024-09-21T03:35:43.880613+0000","last_clean":"2024-09-21T03:35:43.880613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:35:43.902374+0000","last_undegraded":"2024-09-21T03:35:43.902374+0000","last_fullsized":"2024-09-21T03:35:43.902374+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T03:35:44.920375+0000","last_change":"2024-09-21T03:35:44.915023+0000","last_active":"2024-09-21T03:35:44.920375+0000","last_peered":"2024-09-21T03:35:44.920375+0000","last_clean":"2024-09-21T03:35:44.920375+0000","last_became_active":"2024-09-21T03:35:44.914677+0000","last_became_peered":"2024-09-21T03:35:44.914677+0000","last_unstale":"2024-09-21T03:35:44.920375+0000","last_undegraded":"2024-09-21T03:35:44.920375+0000","last_fullsized":"2024-09-21T03:35:44.920375+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:41:13.888920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.868808+0000","last_change":"2024-09-21T03:35:44.922185+0000","last_active":"2024-09-21T03:35:46.868808+0000","last_peered":"2024-09-21T03:35:46.868808+0000","last_clean":"2024-09-21T03:35:46.868808+0000","last_became_active":"2024-09-21T03:35:44.921473+0000","last_became_peered":"2024-09-21T03:35:44.921473+0000","last_unstale":"2024-09-21T03:35:46.868808+0000","last_undegraded":"2024-09-21T03:35:46.868808+0000","last_fullsized":"2024-09-21T03:35:46.868808+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:10:57.276638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.869035+0000","last_change":"2024-09-21T03:35:44.916166+0000","last_active":"2024-09-21T03:35:46.869035+0000","last_peered":"2024-09-21T03:35:46.869035+0000","last_clean":"2024-09-21T03:35:46.869035+0000","last_became_active":"2024-09-21T03:35:44.915841+0000","last_became_peered":"2024-09-21T03:35:44.915841+0000","last_unstale":"2024-09-21T03:35:46.869035+0000","last_undegraded":"2024-09-21T03:35:46.869035+0000","last_fullsized":"2024-09-21T03:35:46.869035+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:58:35.540873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.868743+0000","last_change":"2024-09-21T03:35:44.915458+0000","last_active":"2024-09-21T03:35:46.868743+0000","last_peered":"2024-09-21T03:35:46.868743+0000","last_clean":"2024-09-21T03:35:46.868743+0000","last_became_active":"2024-09-21T03:35:44.915163+0000","last_became_peered":"2024-09-21T03:35:44.915163+0000","last_unstale":"2024-09-21T03:35:46.868743+0000","last_undegraded":"2024-09-21T03:35:46.868743+0000","last_fullsized":"2024-09-21T03:35:46.868743+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:03:55.703336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.870005+0000","last_change":"2024-09-21T03:35:44.918018+0000","last_active":"2024-09-21T03:35:46.870005+0000","last_peered":"2024-09-21T03:35:46.870005+0000","last_clean":"2024-09-21T03:35:46.870005+0000","last_became_active":"2024-09-21T03:35:44.916939+0000","last_became_peered":"2024-09-21T03:35:44.916939+0000","last_unstale":"2024-09-21T03:35:46.870005+0000","last_undegraded":"2024-09-21T03:35:46.870005+0000","last_fullsized":"2024-09-21T03:35:46.870005+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:32:35.988080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.869597+0000","last_change":"2024-09-21T03:35:44.904810+0000","last_active":"2024-09-21T03:35:46.869597+0000","last_peered":"2024-09-21T03:35:46.869597+0000","last_clean":"2024-09-21T03:35:46.869597+0000","last_became_active":"2024-09-21T03:35:44.904499+0000","last_became_peered":"2024-09-21T03:35:44.904499+0000","last_unstale":"2024-09-21T03:35:46.869597+0000","last_undegraded":"2024-09-21T03:35:46.869597+0000","last_fullsized":"2024-09-21T03:35:46.869597+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:37:49.682407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-09-21T03:35:43.892475+0000","last_change":"2024-09-21T03:35:43.892475+0000","last_active":"2024-09-21T03:35:43.880613+0000","last_peered":"2024-09-21T03:35:43.880613+0000","last_clean":"2024-09-21T03:35:43.880613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:35:43.892475+0000","last_undegraded":"2024-09-21T03:35:43.892475+0000","last_fullsized":"2024-09-21T03:35:43.892475+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-09-21T03:35:43.901876+0000","last_change":"2024-09-21T03:35:43.901876+0000","last_active":"2024-09-21T03:35:43.880613+0000","last_peered":"2024-09-21T03:35:43.880613+0000","last_clean":"2024-09-21T03:35:43.880613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:35:43.901876+0000","last_undegraded":"2024-09-21T03:35:43.901876+0000","last_fullsized":"2024-09-21T03:35:43.901876+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.869246+0000","last_change":"2024-09-21T03:35:44.913781+0000","last_active":"2024-09-21T03:35:46.869246+0000","last_peered":"2024-09-21T03:35:46.869246+0000","last_clean":"2024-09-21T03:35:46.869246+0000","last_became_active":"2024-09-21T03:35:44.913469+0000","last_became_peered":"2024-09-21T03:35:44.913469+0000","last_unstale":"2024-09-21T03:35:46.869246+0000","last_undegraded":"2024-09-21T03:35:46.869246+0000","last_fullsized":"2024-09-21T03:35:46.869246+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:52:54.152973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-09-21T03:35:43.896302+0000","last_change":"2024-09-21T03:35:43.896302+0000","last_active":"2024-09-21T03:35:43.880613+0000","last_peered":"2024-09-21T03:35:43.880613+0000","last_clean":"2024-09-21T03:35:43.880613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:35:43.896302+0000","last_undegraded":"2024-09-21T03:35:43.896302+0000","last_fullsized":"2024-09-21T03:35:43.896302+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.868885+0000","last_change":"2024-09-21T03:35:44.920858+0000","last_active":"2024-09-21T03:35:46.868885+0000","last_peered":"2024-09-21T03:35:46.868885+0000","last_clean":"2024-09-21T03:35:46.868885+0000","last_became_active":"2024-09-21T03:35:44.920539+0000","last_became_peered":"2024-09-21T03:35:44.920539+0000","last_unstale":"2024-09-21T03:35:46.868885+0000","last_undegraded":"2024-09-21T03:35:46.868885+0000","last_fullsized":"2024-09-21T03:35:46.868885+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:09:20.328763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-09-21T03:35:43.899921+0000","last_change":"2024-09-21T03:35:43.899921+0000","last_active":"2024-09-21T03:35:43.880613+0000","last_peered":"2024-09-21T03:35:43.880613+0000","last_clean":"2024-09-21T03:35:43.880613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:35:43.899921+0000","last_undegraded":"2024-09-21T03:35:43.899921+0000","last_fullsized":"2024-09-21T03:35:43.899921+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-09-21T03:35:43.897228+0000","last_change":"2024-09-21T03:35:43.897228+0000","last_active":"2024-09-21T03:35:43.880613+0000","last_peered":"2024-09-21T03:35:43.880613+0000","last_clean":"2024-09-21T03:35:43.880613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:35:43.897228+0000","last_undegraded":"2024-09-21T03:35:43.897228+0000","last_fullsized":"2024-09-21T03:35:43.897228+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-21T03:35:43.911675+0000","last_change":"2024-09-21T03:35:43.909828+0000","last_active":"2024-09-21T03:35:43.911675+0000","last_peered":"2024-09-21T03:35:43.911675+0000","last_clean":"2024-09-21T03:35:43.911675+0000","last_became_active":"2024-09-21T03:35:43.909503+0000","last_became_peered":"2024-09-21T03:35:43.909503+0000","last_unstale":"2024-09-21T03:35:43.911675+0000","last_undegraded":"2024-09-21T03:35:43.911675+0000","last_fullsized":"2024-09-21T03:35:43.911675+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_clean_scrub_stamp":"2024-09-21T03:35:42.877874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:35:28.406113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-09-21T03:35:43.895909+0000","last_change":"2024-09-21T03:35:43.895909+0000","last_active":"2024-09-21T03:35:43.880613+0000","last_peered":"2024-09-21T03:35:43.880613+0000","last_clean":"2024-09-21T03:35:43.880613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:35:43.895909+0000","last_undegraded":"2024-09-21T03:35:43.895909+0000","last_fullsized":"2024-09-21T03:35:43.895909+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-21T03:35:43.902645+0000","last_change":"2024-09-21T03:35:43.901432+0000","last_active":"2024-09-21T03:35:43.902645+0000","last_peered":"2024-09-21T03:35:43.902645+0000","last_clean":"2024-09-21T03:35:43.902645+0000","last_became_active":"2024-09-21T03:35:43.901134+0000","last_became_peered":"2024-09-21T03:35:43.901134+0000","last_unstale":"2024-09-21T03:35:43.902645+0000","last_undegraded":"2024-09-21T03:35:43.902645+0000","last_fullsized":"2024-09-21T03:35:43.902645+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_clean_scrub_stamp":"2024-09-21T03:35:42.877874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:42:45.225890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-09-21T03:35:43.897697+0000","last_change":"2024-09-21T03:35:43.897697+0000","last_active":"2024-09-21T03:35:43.880613+0000","last_peered":"2024-09-21T03:35:43.880613+0000","last_clean":"2024-09-21T03:35:43.880613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:35:43.897697+0000","last_undegraded":"2024-09-21T03:35:43.897697+0000","last_fullsized":"2024-09-21T03:35:43.897697+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1d","version":"54'2","reported_seq":17,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.680073+0000","last_change":"2024-09-21T03:35:43.902853+0000","last_active":"2024-09-21T03:35:46.680073+0000","last_peered":"2024-09-21T03:35:46.680073+0000","last_clean":"2024-09-21T03:35:46.680073+0000","last_became_active":"2024-09-21T03:35:43.902514+0000","last_became_peered":"2024-09-21T03:35:43.902514+0000","last_unstale":"2024-09-21T03:35:46.680073+0000","last_undegraded":"2024-09-21T03:35:46.680073+0000","last_fullsized":"2024-09-21T03:35:46.680073+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_clean_scrub_stamp":"2024-09-21T03:35:42.877874+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:40:01.918835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.680012+0000","last_change":"2024-09-21T03:35:44.912392+0000","last_active":"2024-09-21T03:35:46.680012+0000","last_peered":"2024-09-21T03:35:46.680012+0000","last_clean":"2024-09-21T03:35:46.680012+0000","last_became_active":"2024-09-21T03:35:44.911332+0000","last_became_peered":"2024-09-21T03:35:44.911332+0000","last_unstale":"2024-09-21T03:35:46.680012+0000","last_undegraded":"2024-09-21T03:35:46.680012+0000","last_fullsized":"2024-09-21T03:35:46.680012+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:16:36.470418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-21T03:35:43.916772+0000","last_change":"2024-09-21T03:35:43.911405+0000","last_active":"2024-09-21T03:35:43.916772+0000","last_peered":"2024-09-21T03:35:43.916772+0000","last_clean":"2024-09-21T03:35:43.916772+0000","last_became_active":"2024-09-21T03:35:43.911082+0000","last_became_peered":"2024-09-21T03:35:43.911082+0000","last_unstale":"2024-09-21T03:35:43.916772+0000","last_undegraded":"2024-09-21T03:35:43.916772+0000","last_fullsized":"2024-09-21T03:35:43.916772+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_clean_scrub_stamp":"2024-09-21T03:35:42.877874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:40:34.993739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-09-21T03:35:43.895551+0000","last_change":"2024-09-21T03:35:43.895551+0000","last_active":"2024-09-21T03:35:43.880613+0000","last_peered":"2024-09-21T03:35:43.880613+0000","last_clean":"2024-09-21T03:35:43.880613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:35:43.895551+0000","last_undegraded":"2024-09-21T03:35:43.895551+0000","last_fullsized":"2024-09-21T03:35:43.895551+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1b","version":"54'1","reported_seq":16,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.868936+0000","last_change":"2024-09-21T03:35:43.909887+0000","last_active":"2024-09-21T03:35:46.868936+0000","last_peered":"2024-09-21T03:35:46.868936+0000","last_clean":"2024-09-21T03:35:46.868936+0000","last_became_active":"2024-09-21T03:35:43.909558+0000","last_became_peered":"2024-09-21T03:35:43.909558+0000","last_unstale":"2024-09-21T03:35:46.868936+0000","last_undegraded":"2024-09-21T03:35:46.868936+0000","last_fullsized":"2024-09-21T03:35:46.868936+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_clean_scrub_stamp":"2024-09-21T03:35:42.877874+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:03:39.525599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-09-21T03:35:43.897379+0000","last_change":"2024-09-21T03:35:43.897379+0000","last_active":"2024-09-21T03:35:43.880613+0000","last_peered":"2024-09-21T03:35:43.880613+0000","last_clean":"2024-09-21T03:35:43.880613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:35:43.897379+0000","last_undegraded":"2024-09-21T03:35:43.897379+0000","last_fullsized":"2024-09-21T03:35:43.897379+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-21T03:35:43.915560+0000","last_change":"2024-09-21T03:35:43.910078+0000","last_active":"2024-09-21T03:35:43.915560+0000","last_peered":"2024-09-21T03:35:43.915560+0000","last_clean":"2024-09-21T03:35:43.915560+0000","last_became_active":"2024-09-21T03:35:43.909315+0000","last_became_peered":"2024-09-21T03:35:43.909315+0000","last_unstale":"2024-09-21T03:35:43.915560+0000","last_undegraded":"2024-09-21T03:35:43.915560+0000","last_fullsized":"2024-09-21T03:35:43.915560+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_clean_scrub_stamp":"2024-09-21T03:35:42.877874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:48:07.945139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.680104+0000","last_change":"2024-09-21T03:35:44.910050+0000","last_active":"2024-09-21T03:35:46.680104+0000","last_peered":"2024-09-21T03:35:46.680104+0000","last_clean":"2024-09-21T03:35:46.680104+0000","last_became_active":"2024-09-21T03:35:44.909755+0000","last_became_peered":"2024-09-21T03:35:44.909755+0000","last_unstale":"2024-09-21T03:35:46.680104+0000","last_undegraded":"2024-09-21T03:35:46.680104+0000","last_fullsized":"2024-09-21T03:35:46.680104+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:30:38.153649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-21T03:35:43.915132+0000","last_change":"2024-09-21T03:35:43.909144+0000","last_active":"2024-09-21T03:35:43.915132+0000","last_peered":"2024-09-21T03:35:43.915132+0000","last_clean":"2024-09-21T03:35:43.915132+0000","last_became_active":"2024-09-21T03:35:43.908808+0000","last_became_peered":"2024-09-21T03:35:43.908808+0000","last_unstale":"2024-09-21T03:35:43.915132+0000","last_undegraded":"2024-09-21T03:35:43.915132+0000","last_fullsized":"2024-09-21T03:35:43.915132+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_clean_scrub_stamp":"2024-09-21T03:35:42.877874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:51:41.557674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T03:35:44.913427+0000","last_change":"2024-09-21T03:35:44.907438+0000","last_active":"2024-09-21T03:35:44.913427+0000","last_peered":"2024-09-21T03:35:44.913427+0000","last_clean":"2024-09-21T03:35:44.913427+0000","last_became_active":"2024-09-21T03:35:44.907114+0000","last_became_peered":"2024-09-21T03:35:44.907114+0000","last_unstale":"2024-09-21T03:35:44.913427+0000","last_undegraded":"2024-09-21T03:35:44.913427+0000","last_fullsized":"2024-09-21T03:35:44.913427+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:44:34.145008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":15,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.868893+0000","last_change":"2024-09-21T03:35:43.903257+0000","last_active":"2024-09-21T03:35:46.868893+0000","last_peered":"2024-09-21T03:35:46.868893+0000","last_clean":"2024-09-21T03:35:46.868893+0000","last_became_active":"2024-09-21T03:35:43.902926+0000","last_became_peered":"2024-09-21T03:35:43.902926+0000","last_unstale":"2024-09-21T03:35:46.868893+0000","last_undegraded":"2024-09-21T03:35:46.868893+0000","last_fullsized":"2024-09-21T03:35:46.868893+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_clean_scrub_stamp":"2024-09-21T03:35:42.877874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:30:24.926072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-09-21T03:35:43.895500+0000","last_change":"2024-09-21T03:35:43.895500+0000","last_active":"2024-09-21T03:35:43.880613+0000","last_peered":"2024-09-21T03:35:43.880613+0000","last_clean":"2024-09-21T03:35:43.880613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:35:43.895500+0000","last_undegraded":"2024-09-21T03:35:43.895500+0000","last_fullsized":"2024-09-21T03:35:43.895500+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T03:35:44.896174+0000","last_change":"2024-09-21T03:35:43.905541+0000","last_active":"2024-09-21T03:35:44.896174+0000","last_peered":"2024-09-21T03:35:44.896174+0000","last_clean":"2024-09-21T03:35:44.896174+0000","last_became_active":"2024-09-21T03:35:43.905230+0000","last_became_peered":"2024-09-21T03:35:43.905230+0000","last_unstale":"2024-09-21T03:35:44.896174+0000","last_undegraded":"2024-09-21T03:35:44.896174+0000","last_fullsized":"2024-09-21T03:35:44.896174+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_clean_scrub_stamp":"2024-09-21T03:35:42.877874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:46:28.427572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T03:35:44.927406+0000","last_change":"2024-09-21T03:35:44.920481+0000","last_active":"2024-09-21T03:35:44.927406+0000","last_peered":"2024-09-21T03:35:44.927406+0000","last_clean":"2024-09-21T03:35:44.927406+0000","last_became_active":"2024-09-21T03:35:44.919780+0000","last_became_peered":"2024-09-21T03:35:44.919780+0000","last_unstale":"2024-09-21T03:35:44.927406+0000","last_undegraded":"2024-09-21T03:35:44.927406+0000","last_fullsized":"2024-09-21T03:35:44.927406+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:33:00.469260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T03:35:44.898827+0000","last_change":"2024-09-21T03:35:43.898522+0000","last_active":"2024-09-21T03:35:44.898827+0000","last_peered":"2024-09-21T03:35:44.898827+0000","last_clean":"2024-09-21T03:35:44.898827+0000","last_became_active":"2024-09-21T03:35:43.898171+0000","last_became_peered":"2024-09-21T03:35:43.898171+0000","last_unstale":"2024-09-21T03:35:44.898827+0000","last_undegraded":"2024-09-21T03:35:44.898827+0000","last_fullsized":"2024-09-21T03:35:44.898827+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_clean_scrub_stamp":"2024-09-21T03:35:42.877874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:14:03.947289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.869721+0000","last_change":"2024-09-21T03:35:44.921020+0000","last_active":"2024-09-21T03:35:46.869721+0000","last_peered":"2024-09-21T03:35:46.869721+0000","last_clean":"2024-09-21T03:35:46.869721+0000","last_became_active":"2024-09-21T03:35:44.920573+0000","last_became_peered":"2024-09-21T03:35:44.920573+0000","last_unstale":"2024-09-21T03:35:46.869721+0000","last_undegraded":"2024-09-21T03:35:46.869721+0000","last_fullsized":"2024-09-21T03:35:46.869721+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:17:42.161554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-21T03:35:43.915078+0000","last_change":"2024-09-21T03:35:43.907104+0000","last_active":"2024-09-21T03:35:43.915078+0000","last_peered":"2024-09-21T03:35:43.915078+0000","last_clean":"2024-09-21T03:35:43.915078+0000","last_became_active":"2024-09-21T03:35:43.906758+0000","last_became_peered":"2024-09-21T03:35:43.906758+0000","last_unstale":"2024-09-21T03:35:43.915078+0000","last_undegraded":"2024-09-21T03:35:43.915078+0000","last_fullsized":"2024-09-21T03:35:43.915078+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_clean_scrub_stamp":"2024-09-21T03:35:42.877874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:53:55.235170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-09-21T03:35:43.903519+0000","last_change":"2024-09-21T03:35:43.903519+0000","last_active":"2024-09-21T03:35:43.880613+0000","last_peered":"2024-09-21T03:35:43.880613+0000","last_clean":"2024-09-21T03:35:43.880613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:35:43.903519+0000","last_undegraded":"2024-09-21T03:35:43.903519+0000","last_fullsized":"2024-09-21T03:35:43.903519+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.14","version":"54'6","reported_seq":21,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.679907+0000","last_change":"2024-09-21T03:35:43.905841+0000","last_active":"2024-09-21T03:35:46.679907+0000","last_peered":"2024-09-21T03:35:46.679907+0000","last_clean":"2024-09-21T03:35:46.679907+0000","last_became_active":"2024-09-21T03:35:43.905505+0000","last_became_peered":"2024-09-21T03:35:43.905505+0000","last_unstale":"2024-09-21T03:35:46.679907+0000","last_undegraded":"2024-09-21T03:35:46.679907+0000","last_fullsized":"2024-09-21T03:35:46.679907+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_clean_scrub_stamp":"2024-09-21T03:35:42.877874+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:40:06.133533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-09-21T03:35:43.896352+0000","last_change":"2024-09-21T03:35:43.896352+0000","last_active":"2024-09-21T03:35:43.880613+0000","last_peered":"2024-09-21T03:35:43.880613+0000","last_clean":"2024-09-21T03:35:43.880613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:35:43.896352+0000","last_undegraded":"2024-09-21T03:35:43.896352+0000","last_fullsized":"2024-09-21T03:35:43.896352+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-21T03:35:43.913708+0000","last_change":"2024-09-21T03:35:43.913254+0000","last_active":"2024-09-21T03:35:43.913708+0000","last_peered":"2024-09-21T03:35:43.913708+0000","last_clean":"2024-09-21T03:35:43.913708+0000","last_became_active":"2024-09-21T03:35:43.912392+0000","last_became_peered":"2024-09-21T03:35:43.912392+0000","last_unstale":"2024-09-21T03:35:43.913708+0000","last_undegraded":"2024-09-21T03:35:43.913708+0000","last_fullsized":"2024-09-21T03:35:43.913708+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_clean_scrub_stamp":"2024-09-21T03:35:42.877874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:53:16.732262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.869864+0000","last_change":"2024-09-21T03:35:44.906977+0000","last_active":"2024-09-21T03:35:46.869864+0000","last_peered":"2024-09-21T03:35:46.869864+0000","last_clean":"2024-09-21T03:35:46.869864+0000","last_became_active":"2024-09-21T03:35:44.906651+0000","last_became_peered":"2024-09-21T03:35:44.906651+0000","last_unstale":"2024-09-21T03:35:46.869864+0000","last_undegraded":"2024-09-21T03:35:46.869864+0000","last_fullsized":"2024-09-21T03:35:46.869864+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:13:46.431913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T03:35:44.895492+0000","last_change":"2024-09-21T03:35:43.897065+0000","last_active":"2024-09-21T03:35:44.895492+0000","last_peered":"2024-09-21T03:35:44.895492+0000","last_clean":"2024-09-21T03:35:44.895492+0000","last_became_active":"2024-09-21T03:35:43.896727+0000","last_became_peered":"2024-09-21T03:35:43.896727+0000","last_unstale":"2024-09-21T03:35:44.895492+0000","last_undegraded":"2024-09-21T03:35:44.895492+0000","last_fullsized":"2024-09-21T03:35:44.895492+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_clean_scrub_stamp":"2024-09-21T03:35:42.877874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:57:57.601786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-09-21T03:35:43.892997+0000","last_change":"2024-09-21T03:35:43.892997+0000","last_active":"2024-09-21T03:35:43.880613+0000","last_peered":"2024-09-21T03:35:43.880613+0000","last_clean":"2024-09-21T03:35:43.880613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:35:43.892997+0000","last_undegraded":"2024-09-21T03:35:43.892997+0000","last_fullsized":"2024-09-21T03:35:43.892997+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-21T03:35:43.915446+0000","last_change":"2024-09-21T03:35:43.910589+0000","last_active":"2024-09-21T03:35:43.915446+0000","last_peered":"2024-09-21T03:35:43.915446+0000","last_clean":"2024-09-21T03:35:43.915446+0000","last_became_active":"2024-09-21T03:35:43.910150+0000","last_became_peered":"2024-09-21T03:35:43.910150+0000","last_unstale":"2024-09-21T03:35:43.915446+0000","last_undegraded":"2024-09-21T03:35:43.915446+0000","last_fullsized":"2024-09-21T03:35:43.915446+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_clean_scrub_stamp":"2024-09-21T03:35:42.877874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:47:52.583294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.870055+0000","last_change":"2024-09-21T03:35:44.916344+0000","last_active":"2024-09-21T03:35:46.870055+0000","last_peered":"2024-09-21T03:35:46.870055+0000","last_clean":"2024-09-21T03:35:46.870055+0000","last_became_active":"2024-09-21T03:35:44.916019+0000","last_became_peered":"2024-09-21T03:35:44.916019+0000","last_unstale":"2024-09-21T03:35:46.870055+0000","last_undegraded":"2024-09-21T03:35:46.870055+0000","last_fullsized":"2024-09-21T03:35:46.870055+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:58:47.408064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-21T03:35:43.909227+0000","last_change":"2024-09-21T03:35:43.905763+0000","last_active":"2024-09-21T03:35:43.909227+0000","last_peered":"2024-09-21T03:35:43.909227+0000","last_clean":"2024-09-21T03:35:43.909227+0000","last_became_active":"2024-09-21T03:35:43.905453+0000","last_became_peered":"2024-09-21T03:35:43.905453+0000","last_unstale":"2024-09-21T03:35:43.909227+0000","last_undegraded":"2024-09-21T03:35:43.909227+0000","last_fullsized":"2024-09-21T03:35:43.909227+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_clean_scrub_stamp":"2024-09-21T03:35:42.877874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:12:58.212405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.679983+0000","last_change":"2024-09-21T03:35:44.917548+0000","last_active":"2024-09-21T03:35:46.679983+0000","last_peered":"2024-09-21T03:35:46.679983+0000","last_clean":"2024-09-21T03:35:46.679983+0000","last_became_active":"2024-09-21T03:35:44.916979+0000","last_became_peered":"2024-09-21T03:35:44.916979+0000","last_unstale":"2024-09-21T03:35:46.679983+0000","last_undegraded":"2024-09-21T03:35:46.679983+0000","last_fullsized":"2024-09-21T03:35:46.679983+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:25:12.342791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-21T03:35:43.901848+0000","last_change":"2024-09-21T03:35:43.900447+0000","last_active":"2024-09-21T03:35:43.901848+0000","last_peered":"2024-09-21T03:35:43.901848+0000","last_clean":"2024-09-21T03:35:43.901848+0000","last_became_active":"2024-09-21T03:35:43.900127+0000","last_became_peered":"2024-09-21T03:35:43.900127+0000","last_unstale":"2024-09-21T03:35:43.901848+0000","last_undegraded":"2024-09-21T03:35:43.901848+0000","last_fullsized":"2024-09-21T03:35:43.901848+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_clean_scrub_stamp":"2024-09-21T03:35:42.877874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:20:29.206321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.869128+0000","last_change":"2024-09-21T03:35:44.910826+0000","last_active":"2024-09-21T03:35:46.869128+0000","last_peered":"2024-09-21T03:35:46.869128+0000","last_clean":"2024-09-21T03:35:46.869128+0000","last_became_active":"2024-09-21T03:35:44.910528+0000","last_became_peered":"2024-09-21T03:35:44.910528+0000","last_unstale":"2024-09-21T03:35:46.869128+0000","last_undegraded":"2024-09-21T03:35:46.869128+0000","last_fullsized":"2024-09-21T03:35:46.869128+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:00:58.755293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-21T03:35:43.909265+0000","last_change":"2024-09-21T03:35:43.906381+0000","last_active":"2024-09-21T03:35:43.909265+0000","last_peered":"2024-09-21T03:35:43.909265+0000","last_clean":"2024-09-21T03:35:43.909265+0000","last_became_active":"2024-09-21T03:35:43.905982+0000","last_became_peered":"2024-09-21T03:35:43.905982+0000","last_unstale":"2024-09-21T03:35:43.909265+0000","last_undegraded":"2024-09-21T03:35:43.909265+0000","last_fullsized":"2024-09-21T03:35:43.909265+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_clean_scrub_stamp":"2024-09-21T03:35:42.877874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:32:38.146616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-09-21T03:35:43.900034+0000","last_change":"2024-09-21T03:35:43.900034+0000","last_active":"2024-09-21T03:35:43.880613+0000","last_peered":"2024-09-21T03:35:43.880613+0000","last_clean":"2024-09-21T03:35:43.880613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:35:43.900034+0000","last_undegraded":"2024-09-21T03:35:43.900034+0000","last_fullsized":"2024-09-21T03:35:43.900034+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.d","version":"54'1","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T03:35:44.915599+0000","last_change":"2024-09-21T03:35:43.899938+0000","last_active":"2024-09-21T03:35:44.915599+0000","last_peered":"2024-09-21T03:35:44.915599+0000","last_clean":"2024-09-21T03:35:44.915599+0000","last_became_active":"2024-09-21T03:35:43.899617+0000","last_became_peered":"2024-09-21T03:35:43.899617+0000","last_unstale":"2024-09-21T03:35:44.915599+0000","last_undegraded":"2024-09-21T03:35:44.915599+0000","last_fullsized":"2024-09-21T03:35:44.915599+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_clean_scrub_stamp":"2024-09-21T03:35:42.877874+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:54:01.776464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-09-21T03:35:43.890997+0000","last_change":"2024-09-21T03:35:43.890997+0000","last_active":"2024-09-21T03:35:43.880613+0000","last_peered":"2024-09-21T03:35:43.880613+0000","last_clean":"2024-09-21T03:35:43.880613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:35:43.890997+0000","last_undegraded":"2024-09-21T03:35:43.890997+0000","last_fullsized":"2024-09-21T03:35:43.890997+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-21T03:35:43.916804+0000","last_change":"2024-09-21T03:35:43.911542+0000","last_active":"2024-09-21T03:35:43.916804+0000","last_peered":"2024-09-21T03:35:43.916804+0000","last_clean":"2024-09-21T03:35:43.916804+0000","last_became_active":"2024-09-21T03:35:43.911246+0000","last_became_peered":"2024-09-21T03:35:43.911246+0000","last_unstale":"2024-09-21T03:35:43.916804+0000","last_undegraded":"2024-09-21T03:35:43.916804+0000","last_fullsized":"2024-09-21T03:35:43.916804+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_clean_scrub_stamp":"2024-09-21T03:35:42.877874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:23:28.021867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-09-21T03:35:43.901415+0000","last_change":"2024-09-21T03:35:43.901415+0000","last_active":"2024-09-21T03:35:43.880613+0000","last_peered":"2024-09-21T03:35:43.880613+0000","last_clean":"2024-09-21T03:35:43.880613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:35:43.901415+0000","last_undegraded":"2024-09-21T03:35:43.901415+0000","last_fullsized":"2024-09-21T03:35:43.901415+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.b","version":"54'2","reported_seq":17,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.869325+0000","last_change":"2024-09-21T03:35:43.912355+0000","last_active":"2024-09-21T03:35:46.869325+0000","last_peered":"2024-09-21T03:35:46.869325+0000","last_clean":"2024-09-21T03:35:46.869325+0000","last_became_active":"2024-09-21T03:35:43.912034+0000","last_became_peered":"2024-09-21T03:35:43.912034+0000","last_unstale":"2024-09-21T03:35:46.869325+0000","last_undegraded":"2024-09-21T03:35:46.869325+0000","last_fullsized":"2024-09-21T03:35:46.869325+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_clean_scrub_stamp":"2024-09-21T03:35:42.877874+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:13:47.067340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T03:35:44.915374+0000","last_change":"2024-09-21T03:35:44.911441+0000","last_active":"2024-09-21T03:35:44.915374+0000","last_peered":"2024-09-21T03:35:44.915374+0000","last_clean":"2024-09-21T03:35:44.915374+0000","last_became_active":"2024-09-21T03:35:44.911116+0000","last_became_peered":"2024-09-21T03:35:44.911116+0000","last_unstale":"2024-09-21T03:35:44.915374+0000","last_undegraded":"2024-09-21T03:35:44.915374+0000","last_fullsized":"2024-09-21T03:35:44.915374+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:00:19.050385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":15,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.868615+0000","last_change":"2024-09-21T03:35:43.909812+0000","last_active":"2024-09-21T03:35:46.868615+0000","last_peered":"2024-09-21T03:35:46.868615+0000","last_clean":"2024-09-21T03:35:46.868615+0000","last_became_active":"2024-09-21T03:35:43.909494+0000","last_became_peered":"2024-09-21T03:35:43.909494+0000","last_unstale":"2024-09-21T03:35:46.868615+0000","last_undegraded":"2024-09-21T03:35:46.868615+0000","last_fullsized":"2024-09-21T03:35:46.868615+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_clean_scrub_stamp":"2024-09-21T03:35:42.877874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:30:08.445515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.869281+0000","last_change":"2024-09-21T03:35:44.906754+0000","last_active":"2024-09-21T03:35:46.869281+0000","last_peered":"2024-09-21T03:35:46.869281+0000","last_clean":"2024-09-21T03:35:46.869281+0000","last_became_active":"2024-09-21T03:35:44.906423+0000","last_became_peered":"2024-09-21T03:35:44.906423+0000","last_unstale":"2024-09-21T03:35:46.869281+0000","last_undegraded":"2024-09-21T03:35:46.869281+0000","last_fullsized":"2024-09-21T03:35:46.869281+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:50:12.528499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":15,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.679811+0000","last_change":"2024-09-21T03:35:43.905895+0000","last_active":"2024-09-21T03:35:46.679811+0000","last_peered":"2024-09-21T03:35:46.679811+0000","last_clean":"2024-09-21T03:35:46.679811+0000","last_became_active":"2024-09-21T03:35:43.905606+0000","last_became_peered":"2024-09-21T03:35:43.905606+0000","last_unstale":"2024-09-21T03:35:46.679811+0000","last_undegraded":"2024-09-21T03:35:46.679811+0000","last_fullsized":"2024-09-21T03:35:46.679811+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_clean_scrub_stamp":"2024-09-21T03:35:42.877874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:50:41.525440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-09-21T03:35:43.891647+0000","last_change":"2024-09-21T03:35:43.891647+0000","last_active":"2024-09-21T03:35:43.880613+0000","last_peered":"2024-09-21T03:35:43.880613+0000","last_clean":"2024-09-21T03:35:43.880613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:35:43.891647+0000","last_undegraded":"2024-09-21T03:35:43.891647+0000","last_fullsized":"2024-09-21T03:35:43.891647+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":15,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.679686+0000","last_change":"2024-09-21T03:35:43.899860+0000","last_active":"2024-09-21T03:35:46.679686+0000","last_peered":"2024-09-21T03:35:46.679686+0000","last_clean":"2024-09-21T03:35:46.679686+0000","last_became_active":"2024-09-21T03:35:43.899517+0000","last_became_peered":"2024-09-21T03:35:43.899517+0000","last_unstale":"2024-09-21T03:35:46.679686+0000","last_undegraded":"2024-09-21T03:35:46.679686+0000","last_fullsized":"2024-09-21T03:35:46.679686+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_clean_scrub_stamp":"2024-09-21T03:35:42.877874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:23:04.412064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T03:35:44.908986+0000","last_change":"2024-09-21T03:35:44.907182+0000","last_active":"2024-09-21T03:35:44.908986+0000","last_peered":"2024-09-21T03:35:44.908986+0000","last_clean":"2024-09-21T03:35:44.908986+0000","last_became_active":"2024-09-21T03:35:44.906783+0000","last_became_peered":"2024-09-21T03:35:44.906783+0000","last_unstale":"2024-09-21T03:35:44.908986+0000","last_undegraded":"2024-09-21T03:35:44.908986+0000","last_fullsized":"2024-09-21T03:35:44.908986+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:00:17.389518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-21T03:35:43.915543+0000","last_change":"2024-09-21T03:35:43.910068+0000","last_active":"2024-09-21T03:35:43.915543+0000","last_peered":"2024-09-21T03:35:43.915543+0000","last_clean":"2024-09-21T03:35:43.915543+0000","last_became_active":"2024-09-21T03:35:43.908871+0000","last_became_peered":"2024-09-21T03:35:43.908871+0000","last_unstale":"2024-09-21T03:35:43.915543+0000","last_undegraded":"2024-09-21T03:35:43.915543+0000","last_fullsized":"2024-09-21T03:35:43.915543+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_clean_scrub_stamp":"2024-09-21T03:35:42.877874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:42:59.626174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.680044+0000","last_change":"2024-09-21T03:35:44.912419+0000","last_active":"2024-09-21T03:35:46.680044+0000","last_peered":"2024-09-21T03:35:46.680044+0000","last_clean":"2024-09-21T03:35:46.680044+0000","last_became_active":"2024-09-21T03:35:44.912135+0000","last_became_peered":"2024-09-21T03:35:44.912135+0000","last_unstale":"2024-09-21T03:35:46.680044+0000","last_undegraded":"2024-09-21T03:35:46.680044+0000","last_fullsized":"2024-09-21T03:35:46.680044+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:00:55.093451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-21T03:35:43.915020+0000","last_change":"2024-09-21T03:35:43.904698+0000","last_active":"2024-09-21T03:35:43.915020+0000","last_peered":"2024-09-21T03:35:43.915020+0000","last_clean":"2024-09-21T03:35:43.915020+0000","last_became_active":"2024-09-21T03:35:43.904398+0000","last_became_peered":"2024-09-21T03:35:43.904398+0000","last_unstale":"2024-09-21T03:35:43.915020+0000","last_undegraded":"2024-09-21T03:35:43.915020+0000","last_fullsized":"2024-09-21T03:35:43.915020+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_clean_scrub_stamp":"2024-09-21T03:35:42.877874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:37:46.304367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.868840+0000","last_change":"2024-09-21T03:35:44.917013+0000","last_active":"2024-09-21T03:35:46.868840+0000","last_peered":"2024-09-21T03:35:46.868840+0000","last_clean":"2024-09-21T03:35:46.868840+0000","last_became_active":"2024-09-21T03:35:44.916698+0000","last_became_peered":"2024-09-21T03:35:44.916698+0000","last_unstale":"2024-09-21T03:35:46.868840+0000","last_undegraded":"2024-09-21T03:35:46.868840+0000","last_fullsized":"2024-09-21T03:35:46.868840+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:11:45.208479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":15,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.869306+0000","last_change":"2024-09-21T03:35:43.911358+0000","last_active":"2024-09-21T03:35:46.869306+0000","last_peered":"2024-09-21T03:35:46.869306+0000","last_clean":"2024-09-21T03:35:46.869306+0000","last_became_active":"2024-09-21T03:35:43.911032+0000","last_became_peered":"2024-09-21T03:35:43.911032+0000","last_unstale":"2024-09-21T03:35:46.869306+0000","last_undegraded":"2024-09-21T03:35:46.869306+0000","last_fullsized":"2024-09-21T03:35:46.869306+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_clean_scrub_stamp":"2024-09-21T03:35:42.877874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:06:46.333955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.679989+0000","last_change":"2024-09-21T03:35:44.902610+0000","last_active":"2024-09-21T03:35:46.679989+0000","last_peered":"2024-09-21T03:35:46.679989+0000","last_clean":"2024-09-21T03:35:46.679989+0000","last_became_active":"2024-09-21T03:35:44.902286+0000","last_became_peered":"2024-09-21T03:35:44.902286+0000","last_unstale":"2024-09-21T03:35:46.679989+0000","last_undegraded":"2024-09-21T03:35:46.679989+0000","last_fullsized":"2024-09-21T03:35:46.679989+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:20:06.254030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":15,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.679843+0000","last_change":"2024-09-21T03:35:43.911600+0000","last_active":"2024-09-21T03:35:46.679843+0000","last_peered":"2024-09-21T03:35:46.679843+0000","last_clean":"2024-09-21T03:35:46.679843+0000","last_became_active":"2024-09-21T03:35:43.911278+0000","last_became_peered":"2024-09-21T03:35:43.911278+0000","last_unstale":"2024-09-21T03:35:46.679843+0000","last_undegraded":"2024-09-21T03:35:46.679843+0000","last_fullsized":"2024-09-21T03:35:46.679843+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_clean_scrub_stamp":"2024-09-21T03:35:42.877874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:32:48.236015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-09-21T03:35:43.900410+0000","last_change":"2024-09-21T03:35:43.900410+0000","last_active":"2024-09-21T03:35:43.880613+0000","last_peered":"2024-09-21T03:35:43.880613+0000","last_clean":"2024-09-21T03:35:43.880613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:35:43.900410+0000","last_undegraded":"2024-09-21T03:35:43.900410+0000","last_fullsized":"2024-09-21T03:35:43.900410+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":15,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.868734+0000","last_change":"2024-09-21T03:35:43.909531+0000","last_active":"2024-09-21T03:35:46.868734+0000","last_peered":"2024-09-21T03:35:46.868734+0000","last_clean":"2024-09-21T03:35:46.868734+0000","last_became_active":"2024-09-21T03:35:43.909013+0000","last_became_peered":"2024-09-21T03:35:43.909013+0000","last_unstale":"2024-09-21T03:35:46.868734+0000","last_undegraded":"2024-09-21T03:35:46.868734+0000","last_fullsized":"2024-09-21T03:35:46.868734+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_clean_scrub_stamp":"2024-09-21T03:35:42.877874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:17:00.326757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.680299+0000","last_change":"2024-09-21T03:35:44.909262+0000","last_active":"2024-09-21T03:35:46.680299+0000","last_peered":"2024-09-21T03:35:46.680299+0000","last_clean":"2024-09-21T03:35:46.680299+0000","last_became_active":"2024-09-21T03:35:44.908944+0000","last_became_peered":"2024-09-21T03:35:44.908944+0000","last_unstale":"2024-09-21T03:35:46.680299+0000","last_undegraded":"2024-09-21T03:35:46.680299+0000","last_fullsized":"2024-09-21T03:35:46.680299+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:17:43.817629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-21T03:35:43.916795+0000","last_change":"2024-09-21T03:35:43.912067+0000","last_active":"2024-09-21T03:35:43.916795+0000","last_peered":"2024-09-21T03:35:43.916795+0000","last_clean":"2024-09-21T03:35:43.916795+0000","last_became_active":"2024-09-21T03:35:43.911556+0000","last_became_peered":"2024-09-21T03:35:43.911556+0000","last_unstale":"2024-09-21T03:35:43.916795+0000","last_undegraded":"2024-09-21T03:35:43.916795+0000","last_fullsized":"2024-09-21T03:35:43.916795+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_clean_scrub_stamp":"2024-09-21T03:35:42.877874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:01:33.365289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-09-21T03:35:43.892156+0000","last_change":"2024-09-21T03:35:43.892156+0000","last_active":"2024-09-21T03:35:43.880613+0000","last_peered":"2024-09-21T03:35:43.880613+0000","last_clean":"2024-09-21T03:35:43.880613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:35:43.892156+0000","last_undegraded":"2024-09-21T03:35:43.892156+0000","last_fullsized":"2024-09-21T03:35:43.892156+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T03:35:44.896884+0000","last_change":"2024-09-21T03:35:43.898720+0000","last_active":"2024-09-21T03:35:44.896884+0000","last_peered":"2024-09-21T03:35:44.896884+0000","last_clean":"2024-09-21T03:35:44.896884+0000","last_became_active":"2024-09-21T03:35:43.898067+0000","last_became_peered":"2024-09-21T03:35:43.898067+0000","last_unstale":"2024-09-21T03:35:44.896884+0000","last_undegraded":"2024-09-21T03:35:44.896884+0000","last_fullsized":"2024-09-21T03:35:44.896884+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_clean_scrub_stamp":"2024-09-21T03:35:42.877874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:45:41.645966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-09-21T03:35:43.895794+0000","last_change":"2024-09-21T03:35:43.895794+0000","last_active":"2024-09-21T03:35:43.880613+0000","last_peered":"2024-09-21T03:35:43.880613+0000","last_clean":"2024-09-21T03:35:43.880613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:35:43.895794+0000","last_undegraded":"2024-09-21T03:35:43.895794+0000","last_fullsized":"2024-09-21T03:35:43.895794+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"54'1","reported_seq":16,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.680261+0000","last_change":"2024-09-21T03:35:43.903173+0000","last_active":"2024-09-21T03:35:46.680261+0000","last_peered":"2024-09-21T03:35:46.680261+0000","last_clean":"2024-09-21T03:35:46.680261+0000","last_became_active":"2024-09-21T03:35:43.902852+0000","last_became_peered":"2024-09-21T03:35:43.902852+0000","last_unstale":"2024-09-21T03:35:46.680261+0000","last_undegraded":"2024-09-21T03:35:46.680261+0000","last_fullsized":"2024-09-21T03:35:46.680261+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_clean_scrub_stamp":"2024-09-21T03:35:42.877874+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:55:49.604908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":174,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T03:35:44.896975+0000","last_change":"2024-09-21T03:33:25.319776+0000","last_active":"2024-09-21T03:35:44.896975+0000","last_peered":"2024-09-21T03:35:44.896975+0000","last_clean":"2024-09-21T03:35:44.896975+0000","last_became_active":"2024-09-21T03:33:25.318399+0000","last_became_peered":"2024-09-21T03:33:25.318399+0000","last_unstale":"2024-09-21T03:35:44.896975+0000","last_undegraded":"2024-09-21T03:35:44.896975+0000","last_fullsized":"2024-09-21T03:35:44.896975+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:23.016672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:23.016672+0000","last_clean_scrub_stamp":"2024-09-21T03:33:23.016672+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:32:35.906114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":1682,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":4058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":13,"ondisk_log_size":13,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":1,"up_from":16,"seq":68719476768,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32380,"kb_used_data":1188,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339460,"statfs":{"total":96636764160,"available":96603607040,"internally_reserved":0,"allocated":1216512,"data_stored":866432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509472,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31788,"kb_used_data":596,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340052,"statfs":{"total":96636764160,"available":96604213248,"internally_reserved":0,"allocated":610304,"data_stored":269397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509471,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31788,"kb_used_data":596,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340052,"statfs":{"total":96636764160,"available":96604213248,"internally_reserved":0,"allocated":610304,"data_stored":269397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509472,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31788,"kb_used_data":596,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340052,"statfs":{"total":96636764160,"available":96604213248,"internally_reserved":0,"allocated":610304,"data_stored":269397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509471,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31788,"kb_used_data":596,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340052,"statfs":{"total":96636764160,"available":96604213248,"internally_reserved":0,"allocated":610304,"data_stored":269397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542176,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35984,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335856,"statfs":{"total":96636764160,"available":96599916544,"internally_reserved":0,"allocated":651264,"data_stored":290505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574880,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32412,"kb_used_data":1216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339428,"statfs":{"total":96636764160,"available":96603574272,"internally_reserved":0,"allocated":1245184,"data_stored":875301,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542177,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36000,"kb_used_data":652,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335840,"statfs":{"total":96636764160,"available":96599900160,"internally_reserved":0,"allocated":667648,"data_stored":291627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T03:35:49.724 INFO:tasks.ceph.ceph_manager.ceph:PG 8.3e is not active+clean 2024-09-21T03:35:49.725 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-09-21T03:35:43.900135+0000', 'last_change': '2024-09-21T03:35:43.900135+0000', 'last_active': '2024-09-21T03:35:43.880613+0000', 'last_peered': '2024-09-21T03:35:43.880613+0000', 'last_clean': '2024-09-21T03:35:43.880613+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T03:35:43.900135+0000', 'last_undegraded': '2024-09-21T03:35:43.900135+0000', 'last_fullsized': '2024-09-21T03:35:43.900135+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T03:35:43.880613+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T03:35:43.880613+0000', 'last_clean_scrub_stamp': '2024-09-21T03:35:43.880613+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-21T03:35:49.725 INFO:tasks.ceph.ceph_manager.ceph:PG 8.3b is not active+clean 2024-09-21T03:35:49.725 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-09-21T03:35:43.902970+0000', 'last_change': '2024-09-21T03:35:43.902970+0000', 'last_active': '2024-09-21T03:35:43.880613+0000', 'last_peered': '2024-09-21T03:35:43.880613+0000', 'last_clean': '2024-09-21T03:35:43.880613+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T03:35:43.902970+0000', 'last_undegraded': '2024-09-21T03:35:43.902970+0000', 'last_fullsized': '2024-09-21T03:35:43.902970+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T03:35:43.880613+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T03:35:43.880613+0000', 'last_clean_scrub_stamp': '2024-09-21T03:35:43.880613+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-21T03:35:49.725 INFO:tasks.ceph.ceph_manager.ceph:PG 8.3a is not active+clean 2024-09-21T03:35:49.725 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-09-21T03:35:43.889762+0000', 'last_change': '2024-09-21T03:35:43.889762+0000', 'last_active': '2024-09-21T03:35:43.880613+0000', 'last_peered': '2024-09-21T03:35:43.880613+0000', 'last_clean': '2024-09-21T03:35:43.880613+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T03:35:43.889762+0000', 'last_undegraded': '2024-09-21T03:35:43.889762+0000', 'last_fullsized': '2024-09-21T03:35:43.889762+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T03:35:43.880613+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T03:35:43.880613+0000', 'last_clean_scrub_stamp': '2024-09-21T03:35:43.880613+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-21T03:35:49.725 INFO:tasks.ceph.ceph_manager.ceph:PG 8.38 is not active+clean 2024-09-21T03:35:49.725 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-09-21T03:35:43.889893+0000', 'last_change': '2024-09-21T03:35:43.889893+0000', 'last_active': '2024-09-21T03:35:43.880613+0000', 'last_peered': '2024-09-21T03:35:43.880613+0000', 'last_clean': '2024-09-21T03:35:43.880613+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T03:35:43.889893+0000', 'last_undegraded': '2024-09-21T03:35:43.889893+0000', 'last_fullsized': '2024-09-21T03:35:43.889893+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T03:35:43.880613+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T03:35:43.880613+0000', 'last_clean_scrub_stamp': '2024-09-21T03:35:43.880613+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-21T03:35:49.725 INFO:tasks.ceph.ceph_manager.ceph:PG 8.34 is not active+clean 2024-09-21T03:35:49.725 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-09-21T03:35:43.889542+0000', 'last_change': '2024-09-21T03:35:43.889542+0000', 'last_active': '2024-09-21T03:35:43.880613+0000', 'last_peered': '2024-09-21T03:35:43.880613+0000', 'last_clean': '2024-09-21T03:35:43.880613+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T03:35:43.889542+0000', 'last_undegraded': '2024-09-21T03:35:43.889542+0000', 'last_fullsized': '2024-09-21T03:35:43.889542+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T03:35:43.880613+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T03:35:43.880613+0000', 'last_clean_scrub_stamp': '2024-09-21T03:35:43.880613+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-21T03:35:49.726 INFO:tasks.ceph.ceph_manager.ceph:PG 8.33 is not active+clean 2024-09-21T03:35:49.726 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-09-21T03:35:43.890378+0000', 'last_change': '2024-09-21T03:35:43.890378+0000', 'last_active': '2024-09-21T03:35:43.880613+0000', 'last_peered': '2024-09-21T03:35:43.880613+0000', 'last_clean': '2024-09-21T03:35:43.880613+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T03:35:43.890378+0000', 'last_undegraded': '2024-09-21T03:35:43.890378+0000', 'last_fullsized': '2024-09-21T03:35:43.890378+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T03:35:43.880613+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T03:35:43.880613+0000', 'last_clean_scrub_stamp': '2024-09-21T03:35:43.880613+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-21T03:35:49.726 INFO:tasks.ceph.ceph_manager.ceph:PG 8.30 is not active+clean 2024-09-21T03:35:49.726 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-09-21T03:35:43.889143+0000', 'last_change': '2024-09-21T03:35:43.889143+0000', 'last_active': '2024-09-21T03:35:43.880613+0000', 'last_peered': '2024-09-21T03:35:43.880613+0000', 'last_clean': '2024-09-21T03:35:43.880613+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T03:35:43.889143+0000', 'last_undegraded': '2024-09-21T03:35:43.889143+0000', 'last_fullsized': '2024-09-21T03:35:43.889143+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T03:35:43.880613+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T03:35:43.880613+0000', 'last_clean_scrub_stamp': '2024-09-21T03:35:43.880613+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-21T03:35:49.726 INFO:tasks.ceph.ceph_manager.ceph:PG 8.2c is not active+clean 2024-09-21T03:35:49.726 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-09-21T03:35:43.902374+0000', 'last_change': '2024-09-21T03:35:43.902374+0000', 'last_active': '2024-09-21T03:35:43.880613+0000', 'last_peered': '2024-09-21T03:35:43.880613+0000', 'last_clean': '2024-09-21T03:35:43.880613+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T03:35:43.902374+0000', 'last_undegraded': '2024-09-21T03:35:43.902374+0000', 'last_fullsized': '2024-09-21T03:35:43.902374+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T03:35:43.880613+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T03:35:43.880613+0000', 'last_clean_scrub_stamp': '2024-09-21T03:35:43.880613+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-21T03:35:49.726 INFO:tasks.ceph.ceph_manager.ceph:PG 8.25 is not active+clean 2024-09-21T03:35:49.726 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-09-21T03:35:43.892475+0000', 'last_change': '2024-09-21T03:35:43.892475+0000', 'last_active': '2024-09-21T03:35:43.880613+0000', 'last_peered': '2024-09-21T03:35:43.880613+0000', 'last_clean': '2024-09-21T03:35:43.880613+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T03:35:43.892475+0000', 'last_undegraded': '2024-09-21T03:35:43.892475+0000', 'last_fullsized': '2024-09-21T03:35:43.892475+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T03:35:43.880613+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T03:35:43.880613+0000', 'last_clean_scrub_stamp': '2024-09-21T03:35:43.880613+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-21T03:35:49.726 INFO:tasks.ceph.ceph_manager.ceph:PG 8.24 is not active+clean 2024-09-21T03:35:49.726 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-09-21T03:35:43.901876+0000', 'last_change': '2024-09-21T03:35:43.901876+0000', 'last_active': '2024-09-21T03:35:43.880613+0000', 'last_peered': '2024-09-21T03:35:43.880613+0000', 'last_clean': '2024-09-21T03:35:43.880613+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T03:35:43.901876+0000', 'last_undegraded': '2024-09-21T03:35:43.901876+0000', 'last_fullsized': '2024-09-21T03:35:43.901876+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T03:35:43.880613+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T03:35:43.880613+0000', 'last_clean_scrub_stamp': '2024-09-21T03:35:43.880613+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-21T03:35:49.726 INFO:tasks.ceph.ceph_manager.ceph:PG 8.22 is not active+clean 2024-09-21T03:35:49.727 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-09-21T03:35:43.896302+0000', 'last_change': '2024-09-21T03:35:43.896302+0000', 'last_active': '2024-09-21T03:35:43.880613+0000', 'last_peered': '2024-09-21T03:35:43.880613+0000', 'last_clean': '2024-09-21T03:35:43.880613+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T03:35:43.896302+0000', 'last_undegraded': '2024-09-21T03:35:43.896302+0000', 'last_fullsized': '2024-09-21T03:35:43.896302+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T03:35:43.880613+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T03:35:43.880613+0000', 'last_clean_scrub_stamp': '2024-09-21T03:35:43.880613+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-21T03:35:49.727 INFO:tasks.ceph.ceph_manager.ceph:PG 8.20 is not active+clean 2024-09-21T03:35:49.727 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-09-21T03:35:43.899921+0000', 'last_change': '2024-09-21T03:35:43.899921+0000', 'last_active': '2024-09-21T03:35:43.880613+0000', 'last_peered': '2024-09-21T03:35:43.880613+0000', 'last_clean': '2024-09-21T03:35:43.880613+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T03:35:43.899921+0000', 'last_undegraded': '2024-09-21T03:35:43.899921+0000', 'last_fullsized': '2024-09-21T03:35:43.899921+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T03:35:43.880613+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T03:35:43.880613+0000', 'last_clean_scrub_stamp': '2024-09-21T03:35:43.880613+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-21T03:35:49.727 INFO:tasks.ceph.ceph_manager.ceph:PG 8.10 is not active+clean 2024-09-21T03:35:49.727 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-09-21T03:35:43.897228+0000', 'last_change': '2024-09-21T03:35:43.897228+0000', 'last_active': '2024-09-21T03:35:43.880613+0000', 'last_peered': '2024-09-21T03:35:43.880613+0000', 'last_clean': '2024-09-21T03:35:43.880613+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T03:35:43.897228+0000', 'last_undegraded': '2024-09-21T03:35:43.897228+0000', 'last_fullsized': '2024-09-21T03:35:43.897228+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T03:35:43.880613+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T03:35:43.880613+0000', 'last_clean_scrub_stamp': '2024-09-21T03:35:43.880613+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-21T03:35:49.727 INFO:tasks.ceph.ceph_manager.ceph:PG 8.11 is not active+clean 2024-09-21T03:35:49.727 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-09-21T03:35:43.895909+0000', 'last_change': '2024-09-21T03:35:43.895909+0000', 'last_active': '2024-09-21T03:35:43.880613+0000', 'last_peered': '2024-09-21T03:35:43.880613+0000', 'last_clean': '2024-09-21T03:35:43.880613+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T03:35:43.895909+0000', 'last_undegraded': '2024-09-21T03:35:43.895909+0000', 'last_fullsized': '2024-09-21T03:35:43.895909+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T03:35:43.880613+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T03:35:43.880613+0000', 'last_clean_scrub_stamp': '2024-09-21T03:35:43.880613+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-21T03:35:49.727 INFO:tasks.ceph.ceph_manager.ceph:PG 8.12 is not active+clean 2024-09-21T03:35:49.727 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-09-21T03:35:43.897697+0000', 'last_change': '2024-09-21T03:35:43.897697+0000', 'last_active': '2024-09-21T03:35:43.880613+0000', 'last_peered': '2024-09-21T03:35:43.880613+0000', 'last_clean': '2024-09-21T03:35:43.880613+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T03:35:43.897697+0000', 'last_undegraded': '2024-09-21T03:35:43.897697+0000', 'last_fullsized': '2024-09-21T03:35:43.897697+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T03:35:43.880613+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T03:35:43.880613+0000', 'last_clean_scrub_stamp': '2024-09-21T03:35:43.880613+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-21T03:35:49.727 INFO:tasks.ceph.ceph_manager.ceph:PG 8.14 is not active+clean 2024-09-21T03:35:49.727 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-09-21T03:35:43.895551+0000', 'last_change': '2024-09-21T03:35:43.895551+0000', 'last_active': '2024-09-21T03:35:43.880613+0000', 'last_peered': '2024-09-21T03:35:43.880613+0000', 'last_clean': '2024-09-21T03:35:43.880613+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T03:35:43.895551+0000', 'last_undegraded': '2024-09-21T03:35:43.895551+0000', 'last_fullsized': '2024-09-21T03:35:43.895551+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T03:35:43.880613+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T03:35:43.880613+0000', 'last_clean_scrub_stamp': '2024-09-21T03:35:43.880613+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-21T03:35:49.728 INFO:tasks.ceph.ceph_manager.ceph:PG 8.15 is not active+clean 2024-09-21T03:35:49.728 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-09-21T03:35:43.897379+0000', 'last_change': '2024-09-21T03:35:43.897379+0000', 'last_active': '2024-09-21T03:35:43.880613+0000', 'last_peered': '2024-09-21T03:35:43.880613+0000', 'last_clean': '2024-09-21T03:35:43.880613+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T03:35:43.897379+0000', 'last_undegraded': '2024-09-21T03:35:43.897379+0000', 'last_fullsized': '2024-09-21T03:35:43.897379+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T03:35:43.880613+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T03:35:43.880613+0000', 'last_clean_scrub_stamp': '2024-09-21T03:35:43.880613+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-21T03:35:49.728 INFO:tasks.ceph.ceph_manager.ceph:PG 8.18 is not active+clean 2024-09-21T03:35:49.728 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-09-21T03:35:43.895500+0000', 'last_change': '2024-09-21T03:35:43.895500+0000', 'last_active': '2024-09-21T03:35:43.880613+0000', 'last_peered': '2024-09-21T03:35:43.880613+0000', 'last_clean': '2024-09-21T03:35:43.880613+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T03:35:43.895500+0000', 'last_undegraded': '2024-09-21T03:35:43.895500+0000', 'last_fullsized': '2024-09-21T03:35:43.895500+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T03:35:43.880613+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T03:35:43.880613+0000', 'last_clean_scrub_stamp': '2024-09-21T03:35:43.880613+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-21T03:35:49.728 INFO:tasks.ceph.ceph_manager.ceph:PG 8.1b is not active+clean 2024-09-21T03:35:49.728 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-09-21T03:35:43.903519+0000', 'last_change': '2024-09-21T03:35:43.903519+0000', 'last_active': '2024-09-21T03:35:43.880613+0000', 'last_peered': '2024-09-21T03:35:43.880613+0000', 'last_clean': '2024-09-21T03:35:43.880613+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T03:35:43.903519+0000', 'last_undegraded': '2024-09-21T03:35:43.903519+0000', 'last_fullsized': '2024-09-21T03:35:43.903519+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T03:35:43.880613+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T03:35:43.880613+0000', 'last_clean_scrub_stamp': '2024-09-21T03:35:43.880613+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-21T03:35:49.728 INFO:tasks.ceph.ceph_manager.ceph:PG 8.1c is not active+clean 2024-09-21T03:35:49.728 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-09-21T03:35:43.896352+0000', 'last_change': '2024-09-21T03:35:43.896352+0000', 'last_active': '2024-09-21T03:35:43.880613+0000', 'last_peered': '2024-09-21T03:35:43.880613+0000', 'last_clean': '2024-09-21T03:35:43.880613+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T03:35:43.896352+0000', 'last_undegraded': '2024-09-21T03:35:43.896352+0000', 'last_fullsized': '2024-09-21T03:35:43.896352+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T03:35:43.880613+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T03:35:43.880613+0000', 'last_clean_scrub_stamp': '2024-09-21T03:35:43.880613+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-21T03:35:49.728 INFO:tasks.ceph.ceph_manager.ceph:PG 8.1e is not active+clean 2024-09-21T03:35:49.728 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-09-21T03:35:43.892997+0000', 'last_change': '2024-09-21T03:35:43.892997+0000', 'last_active': '2024-09-21T03:35:43.880613+0000', 'last_peered': '2024-09-21T03:35:43.880613+0000', 'last_clean': '2024-09-21T03:35:43.880613+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T03:35:43.892997+0000', 'last_undegraded': '2024-09-21T03:35:43.892997+0000', 'last_fullsized': '2024-09-21T03:35:43.892997+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T03:35:43.880613+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T03:35:43.880613+0000', 'last_clean_scrub_stamp': '2024-09-21T03:35:43.880613+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-21T03:35:49.728 INFO:tasks.ceph.ceph_manager.ceph:PG 8.2 is not active+clean 2024-09-21T03:35:49.729 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-09-21T03:35:43.900034+0000', 'last_change': '2024-09-21T03:35:43.900034+0000', 'last_active': '2024-09-21T03:35:43.880613+0000', 'last_peered': '2024-09-21T03:35:43.880613+0000', 'last_clean': '2024-09-21T03:35:43.880613+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T03:35:43.900034+0000', 'last_undegraded': '2024-09-21T03:35:43.900034+0000', 'last_fullsized': '2024-09-21T03:35:43.900034+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T03:35:43.880613+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T03:35:43.880613+0000', 'last_clean_scrub_stamp': '2024-09-21T03:35:43.880613+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-21T03:35:49.729 INFO:tasks.ceph.ceph_manager.ceph:PG 8.3 is not active+clean 2024-09-21T03:35:49.729 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-09-21T03:35:43.890997+0000', 'last_change': '2024-09-21T03:35:43.890997+0000', 'last_active': '2024-09-21T03:35:43.880613+0000', 'last_peered': '2024-09-21T03:35:43.880613+0000', 'last_clean': '2024-09-21T03:35:43.880613+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T03:35:43.890997+0000', 'last_undegraded': '2024-09-21T03:35:43.890997+0000', 'last_fullsized': '2024-09-21T03:35:43.890997+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T03:35:43.880613+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T03:35:43.880613+0000', 'last_clean_scrub_stamp': '2024-09-21T03:35:43.880613+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-21T03:35:49.729 INFO:tasks.ceph.ceph_manager.ceph:PG 8.4 is not active+clean 2024-09-21T03:35:49.729 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-09-21T03:35:43.901415+0000', 'last_change': '2024-09-21T03:35:43.901415+0000', 'last_active': '2024-09-21T03:35:43.880613+0000', 'last_peered': '2024-09-21T03:35:43.880613+0000', 'last_clean': '2024-09-21T03:35:43.880613+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T03:35:43.901415+0000', 'last_undegraded': '2024-09-21T03:35:43.901415+0000', 'last_fullsized': '2024-09-21T03:35:43.901415+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T03:35:43.880613+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T03:35:43.880613+0000', 'last_clean_scrub_stamp': '2024-09-21T03:35:43.880613+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-21T03:35:49.729 INFO:tasks.ceph.ceph_manager.ceph:PG 8.7 is not active+clean 2024-09-21T03:35:49.729 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-09-21T03:35:43.891647+0000', 'last_change': '2024-09-21T03:35:43.891647+0000', 'last_active': '2024-09-21T03:35:43.880613+0000', 'last_peered': '2024-09-21T03:35:43.880613+0000', 'last_clean': '2024-09-21T03:35:43.880613+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T03:35:43.891647+0000', 'last_undegraded': '2024-09-21T03:35:43.891647+0000', 'last_fullsized': '2024-09-21T03:35:43.891647+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T03:35:43.880613+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T03:35:43.880613+0000', 'last_clean_scrub_stamp': '2024-09-21T03:35:43.880613+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-21T03:35:49.729 INFO:tasks.ceph.ceph_manager.ceph:PG 8.d is not active+clean 2024-09-21T03:35:49.729 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-09-21T03:35:43.900410+0000', 'last_change': '2024-09-21T03:35:43.900410+0000', 'last_active': '2024-09-21T03:35:43.880613+0000', 'last_peered': '2024-09-21T03:35:43.880613+0000', 'last_clean': '2024-09-21T03:35:43.880613+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T03:35:43.900410+0000', 'last_undegraded': '2024-09-21T03:35:43.900410+0000', 'last_fullsized': '2024-09-21T03:35:43.900410+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T03:35:43.880613+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T03:35:43.880613+0000', 'last_clean_scrub_stamp': '2024-09-21T03:35:43.880613+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-21T03:35:49.729 INFO:tasks.ceph.ceph_manager.ceph:PG 8.f is not active+clean 2024-09-21T03:35:49.729 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-09-21T03:35:43.892156+0000', 'last_change': '2024-09-21T03:35:43.892156+0000', 'last_active': '2024-09-21T03:35:43.880613+0000', 'last_peered': '2024-09-21T03:35:43.880613+0000', 'last_clean': '2024-09-21T03:35:43.880613+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T03:35:43.892156+0000', 'last_undegraded': '2024-09-21T03:35:43.892156+0000', 'last_fullsized': '2024-09-21T03:35:43.892156+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T03:35:43.880613+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T03:35:43.880613+0000', 'last_clean_scrub_stamp': '2024-09-21T03:35:43.880613+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-21T03:35:49.730 INFO:tasks.ceph.ceph_manager.ceph:PG 8.9 is not active+clean 2024-09-21T03:35:49.730 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-09-21T03:35:43.895794+0000', 'last_change': '2024-09-21T03:35:43.895794+0000', 'last_active': '2024-09-21T03:35:43.880613+0000', 'last_peered': '2024-09-21T03:35:43.880613+0000', 'last_clean': '2024-09-21T03:35:43.880613+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T03:35:43.895794+0000', 'last_undegraded': '2024-09-21T03:35:43.895794+0000', 'last_fullsized': '2024-09-21T03:35:43.895794+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T03:35:43.880613+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T03:35:43.880613+0000', 'last_clean_scrub_stamp': '2024-09-21T03:35:43.880613+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-21T03:35:49.730 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-21T03:35:49.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:49.877+0000 7fa6bf790640 1 -- 172.21.15.38:0/1962936916 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa6a0004800 msgr2=0x7fa6a0004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:49.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:49.877+0000 7fa6bf790640 1 --2- 172.21.15.38:0/1962936916 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa6a0004800 0x7fa6a0004c10 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fa6ac0016c0 tx=0x7fa6a0005830 comp rx=0 tx=0).stop 2024-09-21T03:35:49.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:49.877+0000 7fa6bf790640 1 -- 172.21.15.38:0/1962936916 shutdown_connections 2024-09-21T03:35:49.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:49.877+0000 7fa6bf790640 1 --2- 172.21.15.38:0/1962936916 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa6a0004800 0x7fa6a0004c10 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:49.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:49.877+0000 7fa6bf790640 1 -- 172.21.15.38:0/1962936916 >> 172.21.15.38:0/1962936916 conn(0x7fa6b80cef40 msgr2=0x7fa6b80cf350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:49.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:49.877+0000 7fa6bf790640 1 -- 172.21.15.38:0/1962936916 shutdown_connections 2024-09-21T03:35:49.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:49.877+0000 7fa6bf790640 1 -- 172.21.15.38:0/1962936916 wait complete. 2024-09-21T03:35:49.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:49.879+0000 7fa6bf790640 1 Processor -- start 2024-09-21T03:35:49.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:49.879+0000 7fa6bf790640 1 -- start start 2024-09-21T03:35:49.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:49.880+0000 7fa6bf790640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa6a0004800 0x7fa6b8155e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:49.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:49.880+0000 7fa6bf790640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa6b81523a0 0x7fa6b8151c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:49.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:49.880+0000 7fa6bf790640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa6b81e6000 0x7fa6b81e6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:49.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:49.880+0000 7fa6bf790640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa6b80d1b50 con 0x7fa6a0004800 2024-09-21T03:35:49.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:49.880+0000 7fa6bf790640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fa6b80d19d0 con 0x7fa6b81e6000 2024-09-21T03:35:49.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:49.880+0000 7fa6bf790640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fa6b81e6e00 con 0x7fa6b81523a0 2024-09-21T03:35:49.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:49.880+0000 7fa6bd52c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa6a0004800 0x7fa6b8155e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:49.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:49.880+0000 7fa6bd52c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa6a0004800 0x7fa6b8155e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40184/0 (socket says 172.21.15.38:40184) 2024-09-21T03:35:49.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:49.880+0000 7fa6bd52c640 1 -- 172.21.15.38:0/2329328842 learned_addr learned my addr 172.21.15.38:0/2329328842 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:35:49.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:49.881+0000 7fa6bcd2b640 1 --2- 172.21.15.38:0/2329328842 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa6b81523a0 0x7fa6b8151c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:49.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:49.881+0000 7fa6bdd2d640 1 --2- 172.21.15.38:0/2329328842 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa6b81e6000 0x7fa6b81e6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:49.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:49.881+0000 7fa6bd52c640 1 -- 172.21.15.38:0/2329328842 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa6b81523a0 msgr2=0x7fa6b8151c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:49.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:49.881+0000 7fa6bd52c640 1 --2- 172.21.15.38:0/2329328842 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa6b81523a0 0x7fa6b8151c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:49.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:49.881+0000 7fa6bd52c640 1 -- 172.21.15.38:0/2329328842 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa6b81e6000 msgr2=0x7fa6b81e6410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:49.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:49.881+0000 7fa6bd52c640 1 --2- 172.21.15.38:0/2329328842 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa6b81e6000 0x7fa6b81e6410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:49.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:49.881+0000 7fa6bd52c640 1 -- 172.21.15.38:0/2329328842 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6b81e6950 con 0x7fa6a0004800 2024-09-21T03:35:49.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:49.881+0000 7fa6bd52c640 1 --2- 172.21.15.38:0/2329328842 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa6a0004800 0x7fa6b8155e50 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fa6a8011d40 tx=0x7fa6a800f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:49.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:49.881+0000 7fa6b5ffb640 1 -- 172.21.15.38:0/2329328842 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6a8003d80 con 0x7fa6a0004800 2024-09-21T03:35:49.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:49.881+0000 7fa6b5ffb640 1 -- 172.21.15.38:0/2329328842 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa6a8004b20 con 0x7fa6a0004800 2024-09-21T03:35:49.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:49.881+0000 7fa6bf790640 1 -- 172.21.15.38:0/2329328842 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6b8154db0 con 0x7fa6a0004800 2024-09-21T03:35:49.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:49.881+0000 7fa6b5ffb640 1 -- 172.21.15.38:0/2329328842 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6a8004050 con 0x7fa6a0004800 2024-09-21T03:35:49.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:49.881+0000 7fa6bf790640 1 -- 172.21.15.38:0/2329328842 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6b8155370 con 0x7fa6a0004800 2024-09-21T03:35:49.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:49.882+0000 7fa6bf790640 1 -- 172.21.15.38:0/2329328842 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6b80ccd10 con 0x7fa6a0004800 2024-09-21T03:35:49.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:49.883+0000 7fa6b5ffb640 1 -- 172.21.15.38:0/2329328842 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa6a80041f0 con 0x7fa6a0004800 2024-09-21T03:35:49.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:49.884+0000 7fa6b5ffb640 1 --2- 172.21.15.38:0/2329328842 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa6a4078bc0 0x7fa6a407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:49.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:49.884+0000 7fa6b5ffb640 1 -- 172.21.15.38:0/2329328842 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 6101+0+0 (secure 0 0 0) 0x7fa6a809c200 con 0x7fa6a0004800 2024-09-21T03:35:49.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:49.884+0000 7fa6bcd2b640 1 --2- 172.21.15.38:0/2329328842 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa6a4078bc0 0x7fa6a407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:49.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:49.884+0000 7fa6bcd2b640 1 --2- 172.21.15.38:0/2329328842 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa6a4078bc0 0x7fa6a407b0d0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fa6ac032000 tx=0x7fa6ac001190 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:49.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:49.888+0000 7fa6b5ffb640 1 -- 172.21.15.38:0/2329328842 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fa6a8064410 con 0x7fa6a0004800 2024-09-21T03:35:50.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:50.149+0000 7fa6bf790640 1 -- 172.21.15.38:0/2329328842 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7fa6b81e35b0 con 0x7fa6a0004800 2024-09-21T03:35:50.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:50.150+0000 7fa6b5ffb640 1 -- 172.21.15.38:0/2329328842 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1301 (secure 0 0 0) 0x7fa6a8063d10 con 0x7fa6a0004800 2024-09-21T03:35:50.152 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:35:50.152 INFO:teuthology.orchestra.run.smithi038.stdout:{"fsid":"9d50d66f-656b-4fe2-8e3b-fe53b79c4d43","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":159,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":55,"num_osds":8,"num_up_osds":8,"osd_up_since":1726889604,"num_in_osds":8,"osd_in_since":1726889594,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":69},{"state_name":"creating+peering","count":28}],"num_pgs":97,"num_pools":3,"num_objects":10,"data_bytes":592050,"bytes_used":270262272,"bytes_avail":772823851008,"bytes_total":773094113280,"inactive_pgs_ratio":0.28865978121757507,"write_bytes_sec":682,"read_op_per_sec":0,"write_op_per_sec":1},"fsmap":{"epoch":30,"btime":"2024-09-21T03:35:47:913013+0000","id":3,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":3,"rank":0,"name":"c","status":"up:active","gid":5577}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":9,"modified":"2024-09-21T03:35:31.009196+0000","services":{"mds":{"daemons":{"summary":"","c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-21T03:35:50.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:50.152+0000 7fa6bf790640 1 -- 172.21.15.38:0/2329328842 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa6a4078bc0 msgr2=0x7fa6a407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:50.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:50.152+0000 7fa6bf790640 1 --2- 172.21.15.38:0/2329328842 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa6a4078bc0 0x7fa6a407b0d0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fa6ac032000 tx=0x7fa6ac001190 comp rx=0 tx=0).stop 2024-09-21T03:35:50.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:50.152+0000 7fa6bf790640 1 -- 172.21.15.38:0/2329328842 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa6a0004800 msgr2=0x7fa6b8155e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:50.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:50.152+0000 7fa6bf790640 1 --2- 172.21.15.38:0/2329328842 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa6a0004800 0x7fa6b8155e50 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fa6a8011d40 tx=0x7fa6a800f790 comp rx=0 tx=0).stop 2024-09-21T03:35:50.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:50.152+0000 7fa6bf790640 1 -- 172.21.15.38:0/2329328842 shutdown_connections 2024-09-21T03:35:50.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:50.152+0000 7fa6bf790640 1 --2- 172.21.15.38:0/2329328842 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa6b81e6000 0x7fa6b81e6410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:50.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:50.152+0000 7fa6bf790640 1 --2- 172.21.15.38:0/2329328842 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa6a4078bc0 0x7fa6a407b0d0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:50.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:50.152+0000 7fa6bf790640 1 --2- 172.21.15.38:0/2329328842 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa6b81523a0 0x7fa6b8151c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:50.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:50.152+0000 7fa6bf790640 1 --2- 172.21.15.38:0/2329328842 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa6a0004800 0x7fa6b8155e50 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:50.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:50.152+0000 7fa6bf790640 1 -- 172.21.15.38:0/2329328842 >> 172.21.15.38:0/2329328842 conn(0x7fa6b80cef40 msgr2=0x7fa6b80b51d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:50.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:50.153+0000 7fa6bf790640 1 -- 172.21.15.38:0/2329328842 shutdown_connections 2024-09-21T03:35:50.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:50.153+0000 7fa6bf790640 1 -- 172.21.15.38:0/2329328842 wait complete. 2024-09-21T03:35:50.166 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-21T03:35:50.166 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T03:35:50.195 INFO:tasks.ceph.mds.c.smithi038.stderr:2024-09-21T03:35:50.193+0000 7f631f274640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T03:35:50.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:50.281+0000 7f78f5920640 1 -- 172.21.15.38:0/1338739487 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f78f006ec30 msgr2=0x7f78f00b3fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:50.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:50.281+0000 7f78f5920640 1 --2- 172.21.15.38:0/1338739487 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f78f006ec30 0x7f78f00b3fc0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f78d8005fd0 tx=0x7f78d802a5d0 comp rx=0 tx=0).stop 2024-09-21T03:35:50.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:50.281+0000 7f78f5920640 1 -- 172.21.15.38:0/1338739487 shutdown_connections 2024-09-21T03:35:50.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:50.281+0000 7f78f5920640 1 --2- 172.21.15.38:0/1338739487 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f78f006ec30 0x7f78f00b3fc0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:50.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:50.281+0000 7f78f5920640 1 --2- 172.21.15.38:0/1338739487 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f78f00d0b40 0x7f78f006e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:50.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:50.281+0000 7f78f5920640 1 -- 172.21.15.38:0/1338739487 >> 172.21.15.38:0/1338739487 conn(0x7f78f00b2cc0 msgr2=0x7f78f00b30d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:50.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:50.281+0000 7f78f5920640 1 -- 172.21.15.38:0/1338739487 shutdown_connections 2024-09-21T03:35:50.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:50.281+0000 7f78f5920640 1 -- 172.21.15.38:0/1338739487 wait complete. 2024-09-21T03:35:50.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:50.283+0000 7f78f5920640 1 Processor -- start 2024-09-21T03:35:50.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:50.283+0000 7f78f5920640 1 -- start start 2024-09-21T03:35:50.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:50.284+0000 7f78f5920640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f78f006ec30 0x7f78f00ca940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:50.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:50.285+0000 7f78f5920640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f78f00b4500 0x7f78f01f6a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:50.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:50.285+0000 7f78f5920640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f78f00d0b40 0x7f78f01f6fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:50.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:50.285+0000 7f78f5920640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f78f015b750 con 0x7f78f00d0b40 2024-09-21T03:35:50.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:50.285+0000 7f78f5920640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f78f015b5d0 con 0x7f78f00b4500 2024-09-21T03:35:50.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:50.285+0000 7f78f5920640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f78f015b8d0 con 0x7f78f006ec30 2024-09-21T03:35:50.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:50.285+0000 7f78ef7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f78f00d0b40 0x7f78f01f6fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:50.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:50.285+0000 7f78ef7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f78f00d0b40 0x7f78f01f6fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40200/0 (socket says 172.21.15.38:40200) 2024-09-21T03:35:50.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:50.285+0000 7f78ef7fe640 1 -- 172.21.15.38:0/3980995044 learned_addr learned my addr 172.21.15.38:0/3980995044 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:35:50.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:50.285+0000 7f78ee7fc640 1 --2- 172.21.15.38:0/3980995044 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f78f00b4500 0x7f78f01f6a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:50.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:50.285+0000 7f78eeffd640 1 --2- 172.21.15.38:0/3980995044 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f78f006ec30 0x7f78f00ca940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:50.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:50.285+0000 7f78ef7fe640 1 -- 172.21.15.38:0/3980995044 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f78f006ec30 msgr2=0x7f78f00ca940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:50.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:50.285+0000 7f78ef7fe640 1 --2- 172.21.15.38:0/3980995044 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f78f006ec30 0x7f78f00ca940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:50.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:50.285+0000 7f78ef7fe640 1 -- 172.21.15.38:0/3980995044 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f78f00b4500 msgr2=0x7f78f01f6a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:50.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:50.285+0000 7f78ef7fe640 1 --2- 172.21.15.38:0/3980995044 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f78f00b4500 0x7f78f01f6a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:50.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:50.285+0000 7f78ef7fe640 1 -- 172.21.15.38:0/3980995044 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78f00cb170 con 0x7f78f00d0b40 2024-09-21T03:35:50.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:50.285+0000 7f78ef7fe640 1 --2- 172.21.15.38:0/3980995044 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f78f00d0b40 0x7f78f01f6fc0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f78e800c3c0 tx=0x7f78e800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:50.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:50.285+0000 7f78e7fff640 1 -- 172.21.15.38:0/3980995044 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78e8017070 con 0x7f78f00d0b40 2024-09-21T03:35:50.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:50.285+0000 7f78e7fff640 1 -- 172.21.15.38:0/3980995044 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f78e80053c0 con 0x7f78f00d0b40 2024-09-21T03:35:50.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:50.285+0000 7f78f5920640 1 -- 172.21.15.38:0/3980995044 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78f00cb450 con 0x7f78f00d0b40 2024-09-21T03:35:50.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:50.285+0000 7f78e7fff640 1 -- 172.21.15.38:0/3980995044 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78e8003850 con 0x7f78f00d0b40 2024-09-21T03:35:50.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:50.285+0000 7f78f5920640 1 -- 172.21.15.38:0/3980995044 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78f00cba10 con 0x7f78f00d0b40 2024-09-21T03:35:50.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:50.287+0000 7f78f5920640 1 -- 172.21.15.38:0/3980995044 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78f00b3fc0 con 0x7f78f00d0b40 2024-09-21T03:35:50.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:50.287+0000 7f78e7fff640 1 -- 172.21.15.38:0/3980995044 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f78e801a070 con 0x7f78f00d0b40 2024-09-21T03:35:50.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:50.288+0000 7f78e7fff640 1 --2- 172.21.15.38:0/3980995044 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f78d0078b90 0x7f78d007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:50.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:50.288+0000 7f78e7fff640 1 -- 172.21.15.38:0/3980995044 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 6101+0+0 (secure 0 0 0) 0x7f78e809a080 con 0x7f78f00d0b40 2024-09-21T03:35:50.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:50.288+0000 7f78eeffd640 1 --2- 172.21.15.38:0/3980995044 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f78d0078b90 0x7f78d007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:50.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:50.289+0000 7f78eeffd640 1 --2- 172.21.15.38:0/3980995044 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f78d0078b90 0x7f78d007b0a0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f78e0005fd0 tx=0x7f78e00095e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:50.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:50.292+0000 7f78e7fff640 1 -- 172.21.15.38:0/3980995044 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f78e8061d70 con 0x7f78f00d0b40 2024-09-21T03:35:50.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:50.478+0000 7f78f5920640 1 -- 172.21.15.38:0/3980995044 --> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f78f00cd640 con 0x7f78d0078b90 2024-09-21T03:35:50.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:50.482+0000 7f78e7fff640 1 -- 172.21.15.38:0/3980995044 <== mgr.4117 v2:172.21.15.195:6800/1631901344 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+249630 (secure 0 0 0) 0x7f78f00cd640 con 0x7f78d0078b90 2024-09-21T03:35:50.488 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:35:50.489 INFO:teuthology.orchestra.run.smithi038.stderr:dumped all 2024-09-21T03:35:50.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:50.485+0000 7f78f5920640 1 -- 172.21.15.38:0/3980995044 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f78d0078b90 msgr2=0x7f78d007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:50.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:50.485+0000 7f78f5920640 1 --2- 172.21.15.38:0/3980995044 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f78d0078b90 0x7f78d007b0a0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f78e0005fd0 tx=0x7f78e00095e0 comp rx=0 tx=0).stop 2024-09-21T03:35:50.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:50.485+0000 7f78f5920640 1 -- 172.21.15.38:0/3980995044 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f78f00d0b40 msgr2=0x7f78f01f6fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:50.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:50.485+0000 7f78f5920640 1 --2- 172.21.15.38:0/3980995044 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f78f00d0b40 0x7f78f01f6fc0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f78e800c3c0 tx=0x7f78e800c880 comp rx=0 tx=0).stop 2024-09-21T03:35:50.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:50.485+0000 7f78f5920640 1 -- 172.21.15.38:0/3980995044 shutdown_connections 2024-09-21T03:35:50.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:50.485+0000 7f78f5920640 1 --2- 172.21.15.38:0/3980995044 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f78f00d0b40 0x7f78f01f6fc0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:50.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:50.485+0000 7f78f5920640 1 --2- 172.21.15.38:0/3980995044 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f78f00b4500 0x7f78f01f6a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:50.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:50.485+0000 7f78f5920640 1 --2- 172.21.15.38:0/3980995044 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f78d0078b90 0x7f78d007b0a0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:50.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:50.485+0000 7f78f5920640 1 --2- 172.21.15.38:0/3980995044 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f78f006ec30 0x7f78f00ca940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:50.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:50.485+0000 7f78f5920640 1 -- 172.21.15.38:0/3980995044 >> 172.21.15.38:0/3980995044 conn(0x7f78f00b2cc0 msgr2=0x7f78f00d0010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:50.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:50.485+0000 7f78f5920640 1 -- 172.21.15.38:0/3980995044 shutdown_connections 2024-09-21T03:35:50.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:50.485+0000 7f78f5920640 1 -- 172.21.15.38:0/3980995044 wait complete. 2024-09-21T03:35:50.498 INFO:teuthology.orchestra.run.smithi038.stdout:{"pg_ready":true,"pg_map":{"version":121,"stamp":"2024-09-21T03:35:49.013480+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592050,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":113,"num_write_kb":1632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":85,"ondisk_log_size":85,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":99,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":263928,"kb_used_data":6076,"kb_used_omap":12,"kb_used_meta":257651,"kb_avail":754710792,"statfs":{"total":773094113280,"available":772823851008,"internally_reserved":0,"allocated":6221824,"data_stored":3401453,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12748,"internal_metadata":263835188},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1682,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001635"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.869704+0000","last_change":"2024-09-21T03:35:44.918690+0000","last_active":"2024-09-21T03:35:46.869704+0000","last_peered":"2024-09-21T03:35:46.869704+0000","last_clean":"2024-09-21T03:35:46.869704+0000","last_became_active":"2024-09-21T03:35:44.917722+0000","last_became_peered":"2024-09-21T03:35:44.917722+0000","last_unstale":"2024-09-21T03:35:46.869704+0000","last_undegraded":"2024-09-21T03:35:46.869704+0000","last_fullsized":"2024-09-21T03:35:46.869704+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:17:03.401604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-09-21T03:35:43.900135+0000","last_change":"2024-09-21T03:35:43.900135+0000","last_active":"2024-09-21T03:35:43.880613+0000","last_peered":"2024-09-21T03:35:43.880613+0000","last_clean":"2024-09-21T03:35:43.880613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:35:43.900135+0000","last_undegraded":"2024-09-21T03:35:43.900135+0000","last_fullsized":"2024-09-21T03:35:43.900135+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.869706+0000","last_change":"2024-09-21T03:35:44.914485+0000","last_active":"2024-09-21T03:35:46.869706+0000","last_peered":"2024-09-21T03:35:46.869706+0000","last_clean":"2024-09-21T03:35:46.869706+0000","last_became_active":"2024-09-21T03:35:44.914150+0000","last_became_peered":"2024-09-21T03:35:44.914150+0000","last_unstale":"2024-09-21T03:35:46.869706+0000","last_undegraded":"2024-09-21T03:35:46.869706+0000","last_fullsized":"2024-09-21T03:35:46.869706+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:34:38.258070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T03:35:44.927798+0000","last_change":"2024-09-21T03:35:44.921404+0000","last_active":"2024-09-21T03:35:44.927798+0000","last_peered":"2024-09-21T03:35:44.927798+0000","last_clean":"2024-09-21T03:35:44.927798+0000","last_became_active":"2024-09-21T03:35:44.921059+0000","last_became_peered":"2024-09-21T03:35:44.921059+0000","last_unstale":"2024-09-21T03:35:44.927798+0000","last_undegraded":"2024-09-21T03:35:44.927798+0000","last_fullsized":"2024-09-21T03:35:44.927798+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:54:00.361227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-09-21T03:35:43.902970+0000","last_change":"2024-09-21T03:35:43.902970+0000","last_active":"2024-09-21T03:35:43.880613+0000","last_peered":"2024-09-21T03:35:43.880613+0000","last_clean":"2024-09-21T03:35:43.880613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:35:43.902970+0000","last_undegraded":"2024-09-21T03:35:43.902970+0000","last_fullsized":"2024-09-21T03:35:43.902970+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-09-21T03:35:43.889762+0000","last_change":"2024-09-21T03:35:43.889762+0000","last_active":"2024-09-21T03:35:43.880613+0000","last_peered":"2024-09-21T03:35:43.880613+0000","last_clean":"2024-09-21T03:35:43.880613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:35:43.889762+0000","last_undegraded":"2024-09-21T03:35:43.889762+0000","last_fullsized":"2024-09-21T03:35:43.889762+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.680170+0000","last_change":"2024-09-21T03:35:44.900285+0000","last_active":"2024-09-21T03:35:46.680170+0000","last_peered":"2024-09-21T03:35:46.680170+0000","last_clean":"2024-09-21T03:35:46.680170+0000","last_became_active":"2024-09-21T03:35:44.899955+0000","last_became_peered":"2024-09-21T03:35:44.899955+0000","last_unstale":"2024-09-21T03:35:46.680170+0000","last_undegraded":"2024-09-21T03:35:46.680170+0000","last_fullsized":"2024-09-21T03:35:46.680170+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:44:47.487941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-09-21T03:35:43.889893+0000","last_change":"2024-09-21T03:35:43.889893+0000","last_active":"2024-09-21T03:35:43.880613+0000","last_peered":"2024-09-21T03:35:43.880613+0000","last_clean":"2024-09-21T03:35:43.880613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:35:43.889893+0000","last_undegraded":"2024-09-21T03:35:43.889893+0000","last_fullsized":"2024-09-21T03:35:43.889893+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.869989+0000","last_change":"2024-09-21T03:35:44.923776+0000","last_active":"2024-09-21T03:35:46.869989+0000","last_peered":"2024-09-21T03:35:46.869989+0000","last_clean":"2024-09-21T03:35:46.869989+0000","last_became_active":"2024-09-21T03:35:44.923315+0000","last_became_peered":"2024-09-21T03:35:44.923315+0000","last_unstale":"2024-09-21T03:35:46.869989+0000","last_undegraded":"2024-09-21T03:35:46.869989+0000","last_fullsized":"2024-09-21T03:35:46.869989+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:05:17.646813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.680010+0000","last_change":"2024-09-21T03:35:44.906104+0000","last_active":"2024-09-21T03:35:46.680010+0000","last_peered":"2024-09-21T03:35:46.680010+0000","last_clean":"2024-09-21T03:35:46.680010+0000","last_became_active":"2024-09-21T03:35:44.905786+0000","last_became_peered":"2024-09-21T03:35:44.905786+0000","last_unstale":"2024-09-21T03:35:46.680010+0000","last_undegraded":"2024-09-21T03:35:46.680010+0000","last_fullsized":"2024-09-21T03:35:46.680010+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:56:36.052470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.869616+0000","last_change":"2024-09-21T03:35:44.913177+0000","last_active":"2024-09-21T03:35:46.869616+0000","last_peered":"2024-09-21T03:35:46.869616+0000","last_clean":"2024-09-21T03:35:46.869616+0000","last_became_active":"2024-09-21T03:35:44.912856+0000","last_became_peered":"2024-09-21T03:35:44.912856+0000","last_unstale":"2024-09-21T03:35:46.869616+0000","last_undegraded":"2024-09-21T03:35:46.869616+0000","last_fullsized":"2024-09-21T03:35:46.869616+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:53:31.119799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-09-21T03:35:43.889542+0000","last_change":"2024-09-21T03:35:43.889542+0000","last_active":"2024-09-21T03:35:43.880613+0000","last_peered":"2024-09-21T03:35:43.880613+0000","last_clean":"2024-09-21T03:35:43.880613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:35:43.889542+0000","last_undegraded":"2024-09-21T03:35:43.889542+0000","last_fullsized":"2024-09-21T03:35:43.889542+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-09-21T03:35:43.890378+0000","last_change":"2024-09-21T03:35:43.890378+0000","last_active":"2024-09-21T03:35:43.880613+0000","last_peered":"2024-09-21T03:35:43.880613+0000","last_clean":"2024-09-21T03:35:43.880613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:35:43.890378+0000","last_undegraded":"2024-09-21T03:35:43.890378+0000","last_fullsized":"2024-09-21T03:35:43.890378+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.868924+0000","last_change":"2024-09-21T03:35:44.916342+0000","last_active":"2024-09-21T03:35:46.868924+0000","last_peered":"2024-09-21T03:35:46.868924+0000","last_clean":"2024-09-21T03:35:46.868924+0000","last_became_active":"2024-09-21T03:35:44.916047+0000","last_became_peered":"2024-09-21T03:35:44.916047+0000","last_unstale":"2024-09-21T03:35:46.868924+0000","last_undegraded":"2024-09-21T03:35:46.868924+0000","last_fullsized":"2024-09-21T03:35:46.868924+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:12:22.459838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T03:35:44.922283+0000","last_change":"2024-09-21T03:35:44.919326+0000","last_active":"2024-09-21T03:35:44.922283+0000","last_peered":"2024-09-21T03:35:44.922283+0000","last_clean":"2024-09-21T03:35:44.922283+0000","last_became_active":"2024-09-21T03:35:44.919004+0000","last_became_peered":"2024-09-21T03:35:44.919004+0000","last_unstale":"2024-09-21T03:35:44.922283+0000","last_undegraded":"2024-09-21T03:35:44.922283+0000","last_fullsized":"2024-09-21T03:35:44.922283+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:31:52.864107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-09-21T03:35:43.889143+0000","last_change":"2024-09-21T03:35:43.889143+0000","last_active":"2024-09-21T03:35:43.880613+0000","last_peered":"2024-09-21T03:35:43.880613+0000","last_clean":"2024-09-21T03:35:43.880613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:35:43.889143+0000","last_undegraded":"2024-09-21T03:35:43.889143+0000","last_fullsized":"2024-09-21T03:35:43.889143+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.870146+0000","last_change":"2024-09-21T03:35:44.918344+0000","last_active":"2024-09-21T03:35:46.870146+0000","last_peered":"2024-09-21T03:35:46.870146+0000","last_clean":"2024-09-21T03:35:46.870146+0000","last_became_active":"2024-09-21T03:35:44.917344+0000","last_became_peered":"2024-09-21T03:35:44.917344+0000","last_unstale":"2024-09-21T03:35:46.870146+0000","last_undegraded":"2024-09-21T03:35:46.870146+0000","last_fullsized":"2024-09-21T03:35:46.870146+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:10:22.417344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T03:35:44.924513+0000","last_change":"2024-09-21T03:35:44.920057+0000","last_active":"2024-09-21T03:35:44.924513+0000","last_peered":"2024-09-21T03:35:44.924513+0000","last_clean":"2024-09-21T03:35:44.924513+0000","last_became_active":"2024-09-21T03:35:44.919683+0000","last_became_peered":"2024-09-21T03:35:44.919683+0000","last_unstale":"2024-09-21T03:35:44.924513+0000","last_undegraded":"2024-09-21T03:35:44.924513+0000","last_fullsized":"2024-09-21T03:35:44.924513+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:05:30.105550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.869952+0000","last_change":"2024-09-21T03:35:44.919698+0000","last_active":"2024-09-21T03:35:46.869952+0000","last_peered":"2024-09-21T03:35:46.869952+0000","last_clean":"2024-09-21T03:35:46.869952+0000","last_became_active":"2024-09-21T03:35:44.919400+0000","last_became_peered":"2024-09-21T03:35:44.919400+0000","last_unstale":"2024-09-21T03:35:46.869952+0000","last_undegraded":"2024-09-21T03:35:46.869952+0000","last_fullsized":"2024-09-21T03:35:46.869952+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:18:19.307626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-09-21T03:35:43.902374+0000","last_change":"2024-09-21T03:35:43.902374+0000","last_active":"2024-09-21T03:35:43.880613+0000","last_peered":"2024-09-21T03:35:43.880613+0000","last_clean":"2024-09-21T03:35:43.880613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:35:43.902374+0000","last_undegraded":"2024-09-21T03:35:43.902374+0000","last_fullsized":"2024-09-21T03:35:43.902374+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T03:35:44.920375+0000","last_change":"2024-09-21T03:35:44.915023+0000","last_active":"2024-09-21T03:35:44.920375+0000","last_peered":"2024-09-21T03:35:44.920375+0000","last_clean":"2024-09-21T03:35:44.920375+0000","last_became_active":"2024-09-21T03:35:44.914677+0000","last_became_peered":"2024-09-21T03:35:44.914677+0000","last_unstale":"2024-09-21T03:35:44.920375+0000","last_undegraded":"2024-09-21T03:35:44.920375+0000","last_fullsized":"2024-09-21T03:35:44.920375+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:41:13.888920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.868808+0000","last_change":"2024-09-21T03:35:44.922185+0000","last_active":"2024-09-21T03:35:46.868808+0000","last_peered":"2024-09-21T03:35:46.868808+0000","last_clean":"2024-09-21T03:35:46.868808+0000","last_became_active":"2024-09-21T03:35:44.921473+0000","last_became_peered":"2024-09-21T03:35:44.921473+0000","last_unstale":"2024-09-21T03:35:46.868808+0000","last_undegraded":"2024-09-21T03:35:46.868808+0000","last_fullsized":"2024-09-21T03:35:46.868808+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:10:57.276638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.869035+0000","last_change":"2024-09-21T03:35:44.916166+0000","last_active":"2024-09-21T03:35:46.869035+0000","last_peered":"2024-09-21T03:35:46.869035+0000","last_clean":"2024-09-21T03:35:46.869035+0000","last_became_active":"2024-09-21T03:35:44.915841+0000","last_became_peered":"2024-09-21T03:35:44.915841+0000","last_unstale":"2024-09-21T03:35:46.869035+0000","last_undegraded":"2024-09-21T03:35:46.869035+0000","last_fullsized":"2024-09-21T03:35:46.869035+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:58:35.540873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.868743+0000","last_change":"2024-09-21T03:35:44.915458+0000","last_active":"2024-09-21T03:35:46.868743+0000","last_peered":"2024-09-21T03:35:46.868743+0000","last_clean":"2024-09-21T03:35:46.868743+0000","last_became_active":"2024-09-21T03:35:44.915163+0000","last_became_peered":"2024-09-21T03:35:44.915163+0000","last_unstale":"2024-09-21T03:35:46.868743+0000","last_undegraded":"2024-09-21T03:35:46.868743+0000","last_fullsized":"2024-09-21T03:35:46.868743+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:03:55.703336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.870005+0000","last_change":"2024-09-21T03:35:44.918018+0000","last_active":"2024-09-21T03:35:46.870005+0000","last_peered":"2024-09-21T03:35:46.870005+0000","last_clean":"2024-09-21T03:35:46.870005+0000","last_became_active":"2024-09-21T03:35:44.916939+0000","last_became_peered":"2024-09-21T03:35:44.916939+0000","last_unstale":"2024-09-21T03:35:46.870005+0000","last_undegraded":"2024-09-21T03:35:46.870005+0000","last_fullsized":"2024-09-21T03:35:46.870005+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:32:35.988080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.869597+0000","last_change":"2024-09-21T03:35:44.904810+0000","last_active":"2024-09-21T03:35:46.869597+0000","last_peered":"2024-09-21T03:35:46.869597+0000","last_clean":"2024-09-21T03:35:46.869597+0000","last_became_active":"2024-09-21T03:35:44.904499+0000","last_became_peered":"2024-09-21T03:35:44.904499+0000","last_unstale":"2024-09-21T03:35:46.869597+0000","last_undegraded":"2024-09-21T03:35:46.869597+0000","last_fullsized":"2024-09-21T03:35:46.869597+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:37:49.682407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-09-21T03:35:43.892475+0000","last_change":"2024-09-21T03:35:43.892475+0000","last_active":"2024-09-21T03:35:43.880613+0000","last_peered":"2024-09-21T03:35:43.880613+0000","last_clean":"2024-09-21T03:35:43.880613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:35:43.892475+0000","last_undegraded":"2024-09-21T03:35:43.892475+0000","last_fullsized":"2024-09-21T03:35:43.892475+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-09-21T03:35:43.901876+0000","last_change":"2024-09-21T03:35:43.901876+0000","last_active":"2024-09-21T03:35:43.880613+0000","last_peered":"2024-09-21T03:35:43.880613+0000","last_clean":"2024-09-21T03:35:43.880613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:35:43.901876+0000","last_undegraded":"2024-09-21T03:35:43.901876+0000","last_fullsized":"2024-09-21T03:35:43.901876+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.869246+0000","last_change":"2024-09-21T03:35:44.913781+0000","last_active":"2024-09-21T03:35:46.869246+0000","last_peered":"2024-09-21T03:35:46.869246+0000","last_clean":"2024-09-21T03:35:46.869246+0000","last_became_active":"2024-09-21T03:35:44.913469+0000","last_became_peered":"2024-09-21T03:35:44.913469+0000","last_unstale":"2024-09-21T03:35:46.869246+0000","last_undegraded":"2024-09-21T03:35:46.869246+0000","last_fullsized":"2024-09-21T03:35:46.869246+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:52:54.152973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-09-21T03:35:43.896302+0000","last_change":"2024-09-21T03:35:43.896302+0000","last_active":"2024-09-21T03:35:43.880613+0000","last_peered":"2024-09-21T03:35:43.880613+0000","last_clean":"2024-09-21T03:35:43.880613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:35:43.896302+0000","last_undegraded":"2024-09-21T03:35:43.896302+0000","last_fullsized":"2024-09-21T03:35:43.896302+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.868885+0000","last_change":"2024-09-21T03:35:44.920858+0000","last_active":"2024-09-21T03:35:46.868885+0000","last_peered":"2024-09-21T03:35:46.868885+0000","last_clean":"2024-09-21T03:35:46.868885+0000","last_became_active":"2024-09-21T03:35:44.920539+0000","last_became_peered":"2024-09-21T03:35:44.920539+0000","last_unstale":"2024-09-21T03:35:46.868885+0000","last_undegraded":"2024-09-21T03:35:46.868885+0000","last_fullsized":"2024-09-21T03:35:46.868885+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:09:20.328763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-09-21T03:35:43.899921+0000","last_change":"2024-09-21T03:35:43.899921+0000","last_active":"2024-09-21T03:35:43.880613+0000","last_peered":"2024-09-21T03:35:43.880613+0000","last_clean":"2024-09-21T03:35:43.880613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:35:43.899921+0000","last_undegraded":"2024-09-21T03:35:43.899921+0000","last_fullsized":"2024-09-21T03:35:43.899921+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-09-21T03:35:43.897228+0000","last_change":"2024-09-21T03:35:43.897228+0000","last_active":"2024-09-21T03:35:43.880613+0000","last_peered":"2024-09-21T03:35:43.880613+0000","last_clean":"2024-09-21T03:35:43.880613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:35:43.897228+0000","last_undegraded":"2024-09-21T03:35:43.897228+0000","last_fullsized":"2024-09-21T03:35:43.897228+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-21T03:35:43.911675+0000","last_change":"2024-09-21T03:35:43.909828+0000","last_active":"2024-09-21T03:35:43.911675+0000","last_peered":"2024-09-21T03:35:43.911675+0000","last_clean":"2024-09-21T03:35:43.911675+0000","last_became_active":"2024-09-21T03:35:43.909503+0000","last_became_peered":"2024-09-21T03:35:43.909503+0000","last_unstale":"2024-09-21T03:35:43.911675+0000","last_undegraded":"2024-09-21T03:35:43.911675+0000","last_fullsized":"2024-09-21T03:35:43.911675+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_clean_scrub_stamp":"2024-09-21T03:35:42.877874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:35:28.406113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-09-21T03:35:43.895909+0000","last_change":"2024-09-21T03:35:43.895909+0000","last_active":"2024-09-21T03:35:43.880613+0000","last_peered":"2024-09-21T03:35:43.880613+0000","last_clean":"2024-09-21T03:35:43.880613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:35:43.895909+0000","last_undegraded":"2024-09-21T03:35:43.895909+0000","last_fullsized":"2024-09-21T03:35:43.895909+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-21T03:35:43.902645+0000","last_change":"2024-09-21T03:35:43.901432+0000","last_active":"2024-09-21T03:35:43.902645+0000","last_peered":"2024-09-21T03:35:43.902645+0000","last_clean":"2024-09-21T03:35:43.902645+0000","last_became_active":"2024-09-21T03:35:43.901134+0000","last_became_peered":"2024-09-21T03:35:43.901134+0000","last_unstale":"2024-09-21T03:35:43.902645+0000","last_undegraded":"2024-09-21T03:35:43.902645+0000","last_fullsized":"2024-09-21T03:35:43.902645+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_clean_scrub_stamp":"2024-09-21T03:35:42.877874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:42:45.225890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-09-21T03:35:43.897697+0000","last_change":"2024-09-21T03:35:43.897697+0000","last_active":"2024-09-21T03:35:43.880613+0000","last_peered":"2024-09-21T03:35:43.880613+0000","last_clean":"2024-09-21T03:35:43.880613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:35:43.897697+0000","last_undegraded":"2024-09-21T03:35:43.897697+0000","last_fullsized":"2024-09-21T03:35:43.897697+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1d","version":"54'2","reported_seq":17,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.680073+0000","last_change":"2024-09-21T03:35:43.902853+0000","last_active":"2024-09-21T03:35:46.680073+0000","last_peered":"2024-09-21T03:35:46.680073+0000","last_clean":"2024-09-21T03:35:46.680073+0000","last_became_active":"2024-09-21T03:35:43.902514+0000","last_became_peered":"2024-09-21T03:35:43.902514+0000","last_unstale":"2024-09-21T03:35:46.680073+0000","last_undegraded":"2024-09-21T03:35:46.680073+0000","last_fullsized":"2024-09-21T03:35:46.680073+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_clean_scrub_stamp":"2024-09-21T03:35:42.877874+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:40:01.918835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.680012+0000","last_change":"2024-09-21T03:35:44.912392+0000","last_active":"2024-09-21T03:35:46.680012+0000","last_peered":"2024-09-21T03:35:46.680012+0000","last_clean":"2024-09-21T03:35:46.680012+0000","last_became_active":"2024-09-21T03:35:44.911332+0000","last_became_peered":"2024-09-21T03:35:44.911332+0000","last_unstale":"2024-09-21T03:35:46.680012+0000","last_undegraded":"2024-09-21T03:35:46.680012+0000","last_fullsized":"2024-09-21T03:35:46.680012+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:16:36.470418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-21T03:35:43.916772+0000","last_change":"2024-09-21T03:35:43.911405+0000","last_active":"2024-09-21T03:35:43.916772+0000","last_peered":"2024-09-21T03:35:43.916772+0000","last_clean":"2024-09-21T03:35:43.916772+0000","last_became_active":"2024-09-21T03:35:43.911082+0000","last_became_peered":"2024-09-21T03:35:43.911082+0000","last_unstale":"2024-09-21T03:35:43.916772+0000","last_undegraded":"2024-09-21T03:35:43.916772+0000","last_fullsized":"2024-09-21T03:35:43.916772+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_clean_scrub_stamp":"2024-09-21T03:35:42.877874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:40:34.993739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-09-21T03:35:43.895551+0000","last_change":"2024-09-21T03:35:43.895551+0000","last_active":"2024-09-21T03:35:43.880613+0000","last_peered":"2024-09-21T03:35:43.880613+0000","last_clean":"2024-09-21T03:35:43.880613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:35:43.895551+0000","last_undegraded":"2024-09-21T03:35:43.895551+0000","last_fullsized":"2024-09-21T03:35:43.895551+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1b","version":"54'1","reported_seq":16,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.868936+0000","last_change":"2024-09-21T03:35:43.909887+0000","last_active":"2024-09-21T03:35:46.868936+0000","last_peered":"2024-09-21T03:35:46.868936+0000","last_clean":"2024-09-21T03:35:46.868936+0000","last_became_active":"2024-09-21T03:35:43.909558+0000","last_became_peered":"2024-09-21T03:35:43.909558+0000","last_unstale":"2024-09-21T03:35:46.868936+0000","last_undegraded":"2024-09-21T03:35:46.868936+0000","last_fullsized":"2024-09-21T03:35:46.868936+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_clean_scrub_stamp":"2024-09-21T03:35:42.877874+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:03:39.525599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-09-21T03:35:43.897379+0000","last_change":"2024-09-21T03:35:43.897379+0000","last_active":"2024-09-21T03:35:43.880613+0000","last_peered":"2024-09-21T03:35:43.880613+0000","last_clean":"2024-09-21T03:35:43.880613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:35:43.897379+0000","last_undegraded":"2024-09-21T03:35:43.897379+0000","last_fullsized":"2024-09-21T03:35:43.897379+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-21T03:35:43.915560+0000","last_change":"2024-09-21T03:35:43.910078+0000","last_active":"2024-09-21T03:35:43.915560+0000","last_peered":"2024-09-21T03:35:43.915560+0000","last_clean":"2024-09-21T03:35:43.915560+0000","last_became_active":"2024-09-21T03:35:43.909315+0000","last_became_peered":"2024-09-21T03:35:43.909315+0000","last_unstale":"2024-09-21T03:35:43.915560+0000","last_undegraded":"2024-09-21T03:35:43.915560+0000","last_fullsized":"2024-09-21T03:35:43.915560+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_clean_scrub_stamp":"2024-09-21T03:35:42.877874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:48:07.945139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.680104+0000","last_change":"2024-09-21T03:35:44.910050+0000","last_active":"2024-09-21T03:35:46.680104+0000","last_peered":"2024-09-21T03:35:46.680104+0000","last_clean":"2024-09-21T03:35:46.680104+0000","last_became_active":"2024-09-21T03:35:44.909755+0000","last_became_peered":"2024-09-21T03:35:44.909755+0000","last_unstale":"2024-09-21T03:35:46.680104+0000","last_undegraded":"2024-09-21T03:35:46.680104+0000","last_fullsized":"2024-09-21T03:35:46.680104+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:30:38.153649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-21T03:35:43.915132+0000","last_change":"2024-09-21T03:35:43.909144+0000","last_active":"2024-09-21T03:35:43.915132+0000","last_peered":"2024-09-21T03:35:43.915132+0000","last_clean":"2024-09-21T03:35:43.915132+0000","last_became_active":"2024-09-21T03:35:43.908808+0000","last_became_peered":"2024-09-21T03:35:43.908808+0000","last_unstale":"2024-09-21T03:35:43.915132+0000","last_undegraded":"2024-09-21T03:35:43.915132+0000","last_fullsized":"2024-09-21T03:35:43.915132+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_clean_scrub_stamp":"2024-09-21T03:35:42.877874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:51:41.557674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T03:35:44.913427+0000","last_change":"2024-09-21T03:35:44.907438+0000","last_active":"2024-09-21T03:35:44.913427+0000","last_peered":"2024-09-21T03:35:44.913427+0000","last_clean":"2024-09-21T03:35:44.913427+0000","last_became_active":"2024-09-21T03:35:44.907114+0000","last_became_peered":"2024-09-21T03:35:44.907114+0000","last_unstale":"2024-09-21T03:35:44.913427+0000","last_undegraded":"2024-09-21T03:35:44.913427+0000","last_fullsized":"2024-09-21T03:35:44.913427+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:44:34.145008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":15,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.868893+0000","last_change":"2024-09-21T03:35:43.903257+0000","last_active":"2024-09-21T03:35:46.868893+0000","last_peered":"2024-09-21T03:35:46.868893+0000","last_clean":"2024-09-21T03:35:46.868893+0000","last_became_active":"2024-09-21T03:35:43.902926+0000","last_became_peered":"2024-09-21T03:35:43.902926+0000","last_unstale":"2024-09-21T03:35:46.868893+0000","last_undegraded":"2024-09-21T03:35:46.868893+0000","last_fullsized":"2024-09-21T03:35:46.868893+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_clean_scrub_stamp":"2024-09-21T03:35:42.877874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:30:24.926072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-09-21T03:35:43.895500+0000","last_change":"2024-09-21T03:35:43.895500+0000","last_active":"2024-09-21T03:35:43.880613+0000","last_peered":"2024-09-21T03:35:43.880613+0000","last_clean":"2024-09-21T03:35:43.880613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:35:43.895500+0000","last_undegraded":"2024-09-21T03:35:43.895500+0000","last_fullsized":"2024-09-21T03:35:43.895500+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T03:35:44.896174+0000","last_change":"2024-09-21T03:35:43.905541+0000","last_active":"2024-09-21T03:35:44.896174+0000","last_peered":"2024-09-21T03:35:44.896174+0000","last_clean":"2024-09-21T03:35:44.896174+0000","last_became_active":"2024-09-21T03:35:43.905230+0000","last_became_peered":"2024-09-21T03:35:43.905230+0000","last_unstale":"2024-09-21T03:35:44.896174+0000","last_undegraded":"2024-09-21T03:35:44.896174+0000","last_fullsized":"2024-09-21T03:35:44.896174+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_clean_scrub_stamp":"2024-09-21T03:35:42.877874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:46:28.427572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T03:35:44.927406+0000","last_change":"2024-09-21T03:35:44.920481+0000","last_active":"2024-09-21T03:35:44.927406+0000","last_peered":"2024-09-21T03:35:44.927406+0000","last_clean":"2024-09-21T03:35:44.927406+0000","last_became_active":"2024-09-21T03:35:44.919780+0000","last_became_peered":"2024-09-21T03:35:44.919780+0000","last_unstale":"2024-09-21T03:35:44.927406+0000","last_undegraded":"2024-09-21T03:35:44.927406+0000","last_fullsized":"2024-09-21T03:35:44.927406+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:33:00.469260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T03:35:44.898827+0000","last_change":"2024-09-21T03:35:43.898522+0000","last_active":"2024-09-21T03:35:44.898827+0000","last_peered":"2024-09-21T03:35:44.898827+0000","last_clean":"2024-09-21T03:35:44.898827+0000","last_became_active":"2024-09-21T03:35:43.898171+0000","last_became_peered":"2024-09-21T03:35:43.898171+0000","last_unstale":"2024-09-21T03:35:44.898827+0000","last_undegraded":"2024-09-21T03:35:44.898827+0000","last_fullsized":"2024-09-21T03:35:44.898827+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_clean_scrub_stamp":"2024-09-21T03:35:42.877874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:14:03.947289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.869721+0000","last_change":"2024-09-21T03:35:44.921020+0000","last_active":"2024-09-21T03:35:46.869721+0000","last_peered":"2024-09-21T03:35:46.869721+0000","last_clean":"2024-09-21T03:35:46.869721+0000","last_became_active":"2024-09-21T03:35:44.920573+0000","last_became_peered":"2024-09-21T03:35:44.920573+0000","last_unstale":"2024-09-21T03:35:46.869721+0000","last_undegraded":"2024-09-21T03:35:46.869721+0000","last_fullsized":"2024-09-21T03:35:46.869721+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:17:42.161554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-21T03:35:43.915078+0000","last_change":"2024-09-21T03:35:43.907104+0000","last_active":"2024-09-21T03:35:43.915078+0000","last_peered":"2024-09-21T03:35:43.915078+0000","last_clean":"2024-09-21T03:35:43.915078+0000","last_became_active":"2024-09-21T03:35:43.906758+0000","last_became_peered":"2024-09-21T03:35:43.906758+0000","last_unstale":"2024-09-21T03:35:43.915078+0000","last_undegraded":"2024-09-21T03:35:43.915078+0000","last_fullsized":"2024-09-21T03:35:43.915078+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_clean_scrub_stamp":"2024-09-21T03:35:42.877874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:53:55.235170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-09-21T03:35:43.903519+0000","last_change":"2024-09-21T03:35:43.903519+0000","last_active":"2024-09-21T03:35:43.880613+0000","last_peered":"2024-09-21T03:35:43.880613+0000","last_clean":"2024-09-21T03:35:43.880613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:35:43.903519+0000","last_undegraded":"2024-09-21T03:35:43.903519+0000","last_fullsized":"2024-09-21T03:35:43.903519+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.14","version":"54'6","reported_seq":21,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.679907+0000","last_change":"2024-09-21T03:35:43.905841+0000","last_active":"2024-09-21T03:35:46.679907+0000","last_peered":"2024-09-21T03:35:46.679907+0000","last_clean":"2024-09-21T03:35:46.679907+0000","last_became_active":"2024-09-21T03:35:43.905505+0000","last_became_peered":"2024-09-21T03:35:43.905505+0000","last_unstale":"2024-09-21T03:35:46.679907+0000","last_undegraded":"2024-09-21T03:35:46.679907+0000","last_fullsized":"2024-09-21T03:35:46.679907+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_clean_scrub_stamp":"2024-09-21T03:35:42.877874+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:40:06.133533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-09-21T03:35:43.896352+0000","last_change":"2024-09-21T03:35:43.896352+0000","last_active":"2024-09-21T03:35:43.880613+0000","last_peered":"2024-09-21T03:35:43.880613+0000","last_clean":"2024-09-21T03:35:43.880613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:35:43.896352+0000","last_undegraded":"2024-09-21T03:35:43.896352+0000","last_fullsized":"2024-09-21T03:35:43.896352+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-21T03:35:43.913708+0000","last_change":"2024-09-21T03:35:43.913254+0000","last_active":"2024-09-21T03:35:43.913708+0000","last_peered":"2024-09-21T03:35:43.913708+0000","last_clean":"2024-09-21T03:35:43.913708+0000","last_became_active":"2024-09-21T03:35:43.912392+0000","last_became_peered":"2024-09-21T03:35:43.912392+0000","last_unstale":"2024-09-21T03:35:43.913708+0000","last_undegraded":"2024-09-21T03:35:43.913708+0000","last_fullsized":"2024-09-21T03:35:43.913708+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_clean_scrub_stamp":"2024-09-21T03:35:42.877874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:53:16.732262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.869864+0000","last_change":"2024-09-21T03:35:44.906977+0000","last_active":"2024-09-21T03:35:46.869864+0000","last_peered":"2024-09-21T03:35:46.869864+0000","last_clean":"2024-09-21T03:35:46.869864+0000","last_became_active":"2024-09-21T03:35:44.906651+0000","last_became_peered":"2024-09-21T03:35:44.906651+0000","last_unstale":"2024-09-21T03:35:46.869864+0000","last_undegraded":"2024-09-21T03:35:46.869864+0000","last_fullsized":"2024-09-21T03:35:46.869864+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:13:46.431913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T03:35:44.895492+0000","last_change":"2024-09-21T03:35:43.897065+0000","last_active":"2024-09-21T03:35:44.895492+0000","last_peered":"2024-09-21T03:35:44.895492+0000","last_clean":"2024-09-21T03:35:44.895492+0000","last_became_active":"2024-09-21T03:35:43.896727+0000","last_became_peered":"2024-09-21T03:35:43.896727+0000","last_unstale":"2024-09-21T03:35:44.895492+0000","last_undegraded":"2024-09-21T03:35:44.895492+0000","last_fullsized":"2024-09-21T03:35:44.895492+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_clean_scrub_stamp":"2024-09-21T03:35:42.877874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:57:57.601786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-09-21T03:35:43.892997+0000","last_change":"2024-09-21T03:35:43.892997+0000","last_active":"2024-09-21T03:35:43.880613+0000","last_peered":"2024-09-21T03:35:43.880613+0000","last_clean":"2024-09-21T03:35:43.880613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:35:43.892997+0000","last_undegraded":"2024-09-21T03:35:43.892997+0000","last_fullsized":"2024-09-21T03:35:43.892997+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-21T03:35:43.915446+0000","last_change":"2024-09-21T03:35:43.910589+0000","last_active":"2024-09-21T03:35:43.915446+0000","last_peered":"2024-09-21T03:35:43.915446+0000","last_clean":"2024-09-21T03:35:43.915446+0000","last_became_active":"2024-09-21T03:35:43.910150+0000","last_became_peered":"2024-09-21T03:35:43.910150+0000","last_unstale":"2024-09-21T03:35:43.915446+0000","last_undegraded":"2024-09-21T03:35:43.915446+0000","last_fullsized":"2024-09-21T03:35:43.915446+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_clean_scrub_stamp":"2024-09-21T03:35:42.877874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:47:52.583294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.870055+0000","last_change":"2024-09-21T03:35:44.916344+0000","last_active":"2024-09-21T03:35:46.870055+0000","last_peered":"2024-09-21T03:35:46.870055+0000","last_clean":"2024-09-21T03:35:46.870055+0000","last_became_active":"2024-09-21T03:35:44.916019+0000","last_became_peered":"2024-09-21T03:35:44.916019+0000","last_unstale":"2024-09-21T03:35:46.870055+0000","last_undegraded":"2024-09-21T03:35:46.870055+0000","last_fullsized":"2024-09-21T03:35:46.870055+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:58:47.408064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-21T03:35:43.909227+0000","last_change":"2024-09-21T03:35:43.905763+0000","last_active":"2024-09-21T03:35:43.909227+0000","last_peered":"2024-09-21T03:35:43.909227+0000","last_clean":"2024-09-21T03:35:43.909227+0000","last_became_active":"2024-09-21T03:35:43.905453+0000","last_became_peered":"2024-09-21T03:35:43.905453+0000","last_unstale":"2024-09-21T03:35:43.909227+0000","last_undegraded":"2024-09-21T03:35:43.909227+0000","last_fullsized":"2024-09-21T03:35:43.909227+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_clean_scrub_stamp":"2024-09-21T03:35:42.877874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:12:58.212405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.679983+0000","last_change":"2024-09-21T03:35:44.917548+0000","last_active":"2024-09-21T03:35:46.679983+0000","last_peered":"2024-09-21T03:35:46.679983+0000","last_clean":"2024-09-21T03:35:46.679983+0000","last_became_active":"2024-09-21T03:35:44.916979+0000","last_became_peered":"2024-09-21T03:35:44.916979+0000","last_unstale":"2024-09-21T03:35:46.679983+0000","last_undegraded":"2024-09-21T03:35:46.679983+0000","last_fullsized":"2024-09-21T03:35:46.679983+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:25:12.342791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-21T03:35:43.901848+0000","last_change":"2024-09-21T03:35:43.900447+0000","last_active":"2024-09-21T03:35:43.901848+0000","last_peered":"2024-09-21T03:35:43.901848+0000","last_clean":"2024-09-21T03:35:43.901848+0000","last_became_active":"2024-09-21T03:35:43.900127+0000","last_became_peered":"2024-09-21T03:35:43.900127+0000","last_unstale":"2024-09-21T03:35:43.901848+0000","last_undegraded":"2024-09-21T03:35:43.901848+0000","last_fullsized":"2024-09-21T03:35:43.901848+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_clean_scrub_stamp":"2024-09-21T03:35:42.877874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:20:29.206321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.869128+0000","last_change":"2024-09-21T03:35:44.910826+0000","last_active":"2024-09-21T03:35:46.869128+0000","last_peered":"2024-09-21T03:35:46.869128+0000","last_clean":"2024-09-21T03:35:46.869128+0000","last_became_active":"2024-09-21T03:35:44.910528+0000","last_became_peered":"2024-09-21T03:35:44.910528+0000","last_unstale":"2024-09-21T03:35:46.869128+0000","last_undegraded":"2024-09-21T03:35:46.869128+0000","last_fullsized":"2024-09-21T03:35:46.869128+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:00:58.755293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-21T03:35:43.909265+0000","last_change":"2024-09-21T03:35:43.906381+0000","last_active":"2024-09-21T03:35:43.909265+0000","last_peered":"2024-09-21T03:35:43.909265+0000","last_clean":"2024-09-21T03:35:43.909265+0000","last_became_active":"2024-09-21T03:35:43.905982+0000","last_became_peered":"2024-09-21T03:35:43.905982+0000","last_unstale":"2024-09-21T03:35:43.909265+0000","last_undegraded":"2024-09-21T03:35:43.909265+0000","last_fullsized":"2024-09-21T03:35:43.909265+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_clean_scrub_stamp":"2024-09-21T03:35:42.877874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:32:38.146616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-09-21T03:35:43.900034+0000","last_change":"2024-09-21T03:35:43.900034+0000","last_active":"2024-09-21T03:35:43.880613+0000","last_peered":"2024-09-21T03:35:43.880613+0000","last_clean":"2024-09-21T03:35:43.880613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:35:43.900034+0000","last_undegraded":"2024-09-21T03:35:43.900034+0000","last_fullsized":"2024-09-21T03:35:43.900034+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.d","version":"54'1","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T03:35:44.915599+0000","last_change":"2024-09-21T03:35:43.899938+0000","last_active":"2024-09-21T03:35:44.915599+0000","last_peered":"2024-09-21T03:35:44.915599+0000","last_clean":"2024-09-21T03:35:44.915599+0000","last_became_active":"2024-09-21T03:35:43.899617+0000","last_became_peered":"2024-09-21T03:35:43.899617+0000","last_unstale":"2024-09-21T03:35:44.915599+0000","last_undegraded":"2024-09-21T03:35:44.915599+0000","last_fullsized":"2024-09-21T03:35:44.915599+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_clean_scrub_stamp":"2024-09-21T03:35:42.877874+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:54:01.776464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-09-21T03:35:43.890997+0000","last_change":"2024-09-21T03:35:43.890997+0000","last_active":"2024-09-21T03:35:43.880613+0000","last_peered":"2024-09-21T03:35:43.880613+0000","last_clean":"2024-09-21T03:35:43.880613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:35:43.890997+0000","last_undegraded":"2024-09-21T03:35:43.890997+0000","last_fullsized":"2024-09-21T03:35:43.890997+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-21T03:35:43.916804+0000","last_change":"2024-09-21T03:35:43.911542+0000","last_active":"2024-09-21T03:35:43.916804+0000","last_peered":"2024-09-21T03:35:43.916804+0000","last_clean":"2024-09-21T03:35:43.916804+0000","last_became_active":"2024-09-21T03:35:43.911246+0000","last_became_peered":"2024-09-21T03:35:43.911246+0000","last_unstale":"2024-09-21T03:35:43.916804+0000","last_undegraded":"2024-09-21T03:35:43.916804+0000","last_fullsized":"2024-09-21T03:35:43.916804+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_clean_scrub_stamp":"2024-09-21T03:35:42.877874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:23:28.021867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-09-21T03:35:43.901415+0000","last_change":"2024-09-21T03:35:43.901415+0000","last_active":"2024-09-21T03:35:43.880613+0000","last_peered":"2024-09-21T03:35:43.880613+0000","last_clean":"2024-09-21T03:35:43.880613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:35:43.901415+0000","last_undegraded":"2024-09-21T03:35:43.901415+0000","last_fullsized":"2024-09-21T03:35:43.901415+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.b","version":"54'2","reported_seq":17,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.869325+0000","last_change":"2024-09-21T03:35:43.912355+0000","last_active":"2024-09-21T03:35:46.869325+0000","last_peered":"2024-09-21T03:35:46.869325+0000","last_clean":"2024-09-21T03:35:46.869325+0000","last_became_active":"2024-09-21T03:35:43.912034+0000","last_became_peered":"2024-09-21T03:35:43.912034+0000","last_unstale":"2024-09-21T03:35:46.869325+0000","last_undegraded":"2024-09-21T03:35:46.869325+0000","last_fullsized":"2024-09-21T03:35:46.869325+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_clean_scrub_stamp":"2024-09-21T03:35:42.877874+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:13:47.067340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T03:35:44.915374+0000","last_change":"2024-09-21T03:35:44.911441+0000","last_active":"2024-09-21T03:35:44.915374+0000","last_peered":"2024-09-21T03:35:44.915374+0000","last_clean":"2024-09-21T03:35:44.915374+0000","last_became_active":"2024-09-21T03:35:44.911116+0000","last_became_peered":"2024-09-21T03:35:44.911116+0000","last_unstale":"2024-09-21T03:35:44.915374+0000","last_undegraded":"2024-09-21T03:35:44.915374+0000","last_fullsized":"2024-09-21T03:35:44.915374+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:00:19.050385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":15,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.868615+0000","last_change":"2024-09-21T03:35:43.909812+0000","last_active":"2024-09-21T03:35:46.868615+0000","last_peered":"2024-09-21T03:35:46.868615+0000","last_clean":"2024-09-21T03:35:46.868615+0000","last_became_active":"2024-09-21T03:35:43.909494+0000","last_became_peered":"2024-09-21T03:35:43.909494+0000","last_unstale":"2024-09-21T03:35:46.868615+0000","last_undegraded":"2024-09-21T03:35:46.868615+0000","last_fullsized":"2024-09-21T03:35:46.868615+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_clean_scrub_stamp":"2024-09-21T03:35:42.877874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:30:08.445515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.869281+0000","last_change":"2024-09-21T03:35:44.906754+0000","last_active":"2024-09-21T03:35:46.869281+0000","last_peered":"2024-09-21T03:35:46.869281+0000","last_clean":"2024-09-21T03:35:46.869281+0000","last_became_active":"2024-09-21T03:35:44.906423+0000","last_became_peered":"2024-09-21T03:35:44.906423+0000","last_unstale":"2024-09-21T03:35:46.869281+0000","last_undegraded":"2024-09-21T03:35:46.869281+0000","last_fullsized":"2024-09-21T03:35:46.869281+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:50:12.528499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":15,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.679811+0000","last_change":"2024-09-21T03:35:43.905895+0000","last_active":"2024-09-21T03:35:46.679811+0000","last_peered":"2024-09-21T03:35:46.679811+0000","last_clean":"2024-09-21T03:35:46.679811+0000","last_became_active":"2024-09-21T03:35:43.905606+0000","last_became_peered":"2024-09-21T03:35:43.905606+0000","last_unstale":"2024-09-21T03:35:46.679811+0000","last_undegraded":"2024-09-21T03:35:46.679811+0000","last_fullsized":"2024-09-21T03:35:46.679811+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_clean_scrub_stamp":"2024-09-21T03:35:42.877874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:50:41.525440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-09-21T03:35:43.891647+0000","last_change":"2024-09-21T03:35:43.891647+0000","last_active":"2024-09-21T03:35:43.880613+0000","last_peered":"2024-09-21T03:35:43.880613+0000","last_clean":"2024-09-21T03:35:43.880613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:35:43.891647+0000","last_undegraded":"2024-09-21T03:35:43.891647+0000","last_fullsized":"2024-09-21T03:35:43.891647+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":15,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.679686+0000","last_change":"2024-09-21T03:35:43.899860+0000","last_active":"2024-09-21T03:35:46.679686+0000","last_peered":"2024-09-21T03:35:46.679686+0000","last_clean":"2024-09-21T03:35:46.679686+0000","last_became_active":"2024-09-21T03:35:43.899517+0000","last_became_peered":"2024-09-21T03:35:43.899517+0000","last_unstale":"2024-09-21T03:35:46.679686+0000","last_undegraded":"2024-09-21T03:35:46.679686+0000","last_fullsized":"2024-09-21T03:35:46.679686+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_clean_scrub_stamp":"2024-09-21T03:35:42.877874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:23:04.412064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T03:35:44.908986+0000","last_change":"2024-09-21T03:35:44.907182+0000","last_active":"2024-09-21T03:35:44.908986+0000","last_peered":"2024-09-21T03:35:44.908986+0000","last_clean":"2024-09-21T03:35:44.908986+0000","last_became_active":"2024-09-21T03:35:44.906783+0000","last_became_peered":"2024-09-21T03:35:44.906783+0000","last_unstale":"2024-09-21T03:35:44.908986+0000","last_undegraded":"2024-09-21T03:35:44.908986+0000","last_fullsized":"2024-09-21T03:35:44.908986+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:00:17.389518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-21T03:35:43.915543+0000","last_change":"2024-09-21T03:35:43.910068+0000","last_active":"2024-09-21T03:35:43.915543+0000","last_peered":"2024-09-21T03:35:43.915543+0000","last_clean":"2024-09-21T03:35:43.915543+0000","last_became_active":"2024-09-21T03:35:43.908871+0000","last_became_peered":"2024-09-21T03:35:43.908871+0000","last_unstale":"2024-09-21T03:35:43.915543+0000","last_undegraded":"2024-09-21T03:35:43.915543+0000","last_fullsized":"2024-09-21T03:35:43.915543+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_clean_scrub_stamp":"2024-09-21T03:35:42.877874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:42:59.626174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.680044+0000","last_change":"2024-09-21T03:35:44.912419+0000","last_active":"2024-09-21T03:35:46.680044+0000","last_peered":"2024-09-21T03:35:46.680044+0000","last_clean":"2024-09-21T03:35:46.680044+0000","last_became_active":"2024-09-21T03:35:44.912135+0000","last_became_peered":"2024-09-21T03:35:44.912135+0000","last_unstale":"2024-09-21T03:35:46.680044+0000","last_undegraded":"2024-09-21T03:35:46.680044+0000","last_fullsized":"2024-09-21T03:35:46.680044+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:00:55.093451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-21T03:35:43.915020+0000","last_change":"2024-09-21T03:35:43.904698+0000","last_active":"2024-09-21T03:35:43.915020+0000","last_peered":"2024-09-21T03:35:43.915020+0000","last_clean":"2024-09-21T03:35:43.915020+0000","last_became_active":"2024-09-21T03:35:43.904398+0000","last_became_peered":"2024-09-21T03:35:43.904398+0000","last_unstale":"2024-09-21T03:35:43.915020+0000","last_undegraded":"2024-09-21T03:35:43.915020+0000","last_fullsized":"2024-09-21T03:35:43.915020+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_clean_scrub_stamp":"2024-09-21T03:35:42.877874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:37:46.304367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.868840+0000","last_change":"2024-09-21T03:35:44.917013+0000","last_active":"2024-09-21T03:35:46.868840+0000","last_peered":"2024-09-21T03:35:46.868840+0000","last_clean":"2024-09-21T03:35:46.868840+0000","last_became_active":"2024-09-21T03:35:44.916698+0000","last_became_peered":"2024-09-21T03:35:44.916698+0000","last_unstale":"2024-09-21T03:35:46.868840+0000","last_undegraded":"2024-09-21T03:35:46.868840+0000","last_fullsized":"2024-09-21T03:35:46.868840+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:11:45.208479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":15,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.869306+0000","last_change":"2024-09-21T03:35:43.911358+0000","last_active":"2024-09-21T03:35:46.869306+0000","last_peered":"2024-09-21T03:35:46.869306+0000","last_clean":"2024-09-21T03:35:46.869306+0000","last_became_active":"2024-09-21T03:35:43.911032+0000","last_became_peered":"2024-09-21T03:35:43.911032+0000","last_unstale":"2024-09-21T03:35:46.869306+0000","last_undegraded":"2024-09-21T03:35:46.869306+0000","last_fullsized":"2024-09-21T03:35:46.869306+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_clean_scrub_stamp":"2024-09-21T03:35:42.877874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:06:46.333955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.679989+0000","last_change":"2024-09-21T03:35:44.902610+0000","last_active":"2024-09-21T03:35:46.679989+0000","last_peered":"2024-09-21T03:35:46.679989+0000","last_clean":"2024-09-21T03:35:46.679989+0000","last_became_active":"2024-09-21T03:35:44.902286+0000","last_became_peered":"2024-09-21T03:35:44.902286+0000","last_unstale":"2024-09-21T03:35:46.679989+0000","last_undegraded":"2024-09-21T03:35:46.679989+0000","last_fullsized":"2024-09-21T03:35:46.679989+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:20:06.254030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":15,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.679843+0000","last_change":"2024-09-21T03:35:43.911600+0000","last_active":"2024-09-21T03:35:46.679843+0000","last_peered":"2024-09-21T03:35:46.679843+0000","last_clean":"2024-09-21T03:35:46.679843+0000","last_became_active":"2024-09-21T03:35:43.911278+0000","last_became_peered":"2024-09-21T03:35:43.911278+0000","last_unstale":"2024-09-21T03:35:46.679843+0000","last_undegraded":"2024-09-21T03:35:46.679843+0000","last_fullsized":"2024-09-21T03:35:46.679843+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_clean_scrub_stamp":"2024-09-21T03:35:42.877874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:32:48.236015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-09-21T03:35:43.900410+0000","last_change":"2024-09-21T03:35:43.900410+0000","last_active":"2024-09-21T03:35:43.880613+0000","last_peered":"2024-09-21T03:35:43.880613+0000","last_clean":"2024-09-21T03:35:43.880613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:35:43.900410+0000","last_undegraded":"2024-09-21T03:35:43.900410+0000","last_fullsized":"2024-09-21T03:35:43.900410+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":15,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.868734+0000","last_change":"2024-09-21T03:35:43.909531+0000","last_active":"2024-09-21T03:35:46.868734+0000","last_peered":"2024-09-21T03:35:46.868734+0000","last_clean":"2024-09-21T03:35:46.868734+0000","last_became_active":"2024-09-21T03:35:43.909013+0000","last_became_peered":"2024-09-21T03:35:43.909013+0000","last_unstale":"2024-09-21T03:35:46.868734+0000","last_undegraded":"2024-09-21T03:35:46.868734+0000","last_fullsized":"2024-09-21T03:35:46.868734+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_clean_scrub_stamp":"2024-09-21T03:35:42.877874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:17:00.326757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.680299+0000","last_change":"2024-09-21T03:35:44.909262+0000","last_active":"2024-09-21T03:35:46.680299+0000","last_peered":"2024-09-21T03:35:46.680299+0000","last_clean":"2024-09-21T03:35:46.680299+0000","last_became_active":"2024-09-21T03:35:44.908944+0000","last_became_peered":"2024-09-21T03:35:44.908944+0000","last_unstale":"2024-09-21T03:35:46.680299+0000","last_undegraded":"2024-09-21T03:35:46.680299+0000","last_fullsized":"2024-09-21T03:35:46.680299+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:17:43.817629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-21T03:35:43.916795+0000","last_change":"2024-09-21T03:35:43.912067+0000","last_active":"2024-09-21T03:35:43.916795+0000","last_peered":"2024-09-21T03:35:43.916795+0000","last_clean":"2024-09-21T03:35:43.916795+0000","last_became_active":"2024-09-21T03:35:43.911556+0000","last_became_peered":"2024-09-21T03:35:43.911556+0000","last_unstale":"2024-09-21T03:35:43.916795+0000","last_undegraded":"2024-09-21T03:35:43.916795+0000","last_fullsized":"2024-09-21T03:35:43.916795+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_clean_scrub_stamp":"2024-09-21T03:35:42.877874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:01:33.365289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-09-21T03:35:43.892156+0000","last_change":"2024-09-21T03:35:43.892156+0000","last_active":"2024-09-21T03:35:43.880613+0000","last_peered":"2024-09-21T03:35:43.880613+0000","last_clean":"2024-09-21T03:35:43.880613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:35:43.892156+0000","last_undegraded":"2024-09-21T03:35:43.892156+0000","last_fullsized":"2024-09-21T03:35:43.892156+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T03:35:44.896884+0000","last_change":"2024-09-21T03:35:43.898720+0000","last_active":"2024-09-21T03:35:44.896884+0000","last_peered":"2024-09-21T03:35:44.896884+0000","last_clean":"2024-09-21T03:35:44.896884+0000","last_became_active":"2024-09-21T03:35:43.898067+0000","last_became_peered":"2024-09-21T03:35:43.898067+0000","last_unstale":"2024-09-21T03:35:44.896884+0000","last_undegraded":"2024-09-21T03:35:44.896884+0000","last_fullsized":"2024-09-21T03:35:44.896884+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_clean_scrub_stamp":"2024-09-21T03:35:42.877874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:45:41.645966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-09-21T03:35:43.895794+0000","last_change":"2024-09-21T03:35:43.895794+0000","last_active":"2024-09-21T03:35:43.880613+0000","last_peered":"2024-09-21T03:35:43.880613+0000","last_clean":"2024-09-21T03:35:43.880613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:35:43.895794+0000","last_undegraded":"2024-09-21T03:35:43.895794+0000","last_fullsized":"2024-09-21T03:35:43.895794+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"54'1","reported_seq":16,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.680261+0000","last_change":"2024-09-21T03:35:43.903173+0000","last_active":"2024-09-21T03:35:46.680261+0000","last_peered":"2024-09-21T03:35:46.680261+0000","last_clean":"2024-09-21T03:35:46.680261+0000","last_became_active":"2024-09-21T03:35:43.902852+0000","last_became_peered":"2024-09-21T03:35:43.902852+0000","last_unstale":"2024-09-21T03:35:46.680261+0000","last_undegraded":"2024-09-21T03:35:46.680261+0000","last_fullsized":"2024-09-21T03:35:46.680261+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_clean_scrub_stamp":"2024-09-21T03:35:42.877874+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:55:49.604908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":174,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T03:35:44.896975+0000","last_change":"2024-09-21T03:33:25.319776+0000","last_active":"2024-09-21T03:35:44.896975+0000","last_peered":"2024-09-21T03:35:44.896975+0000","last_clean":"2024-09-21T03:35:44.896975+0000","last_became_active":"2024-09-21T03:33:25.318399+0000","last_became_peered":"2024-09-21T03:33:25.318399+0000","last_unstale":"2024-09-21T03:35:44.896975+0000","last_undegraded":"2024-09-21T03:35:44.896975+0000","last_fullsized":"2024-09-21T03:35:44.896975+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:23.016672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:23.016672+0000","last_clean_scrub_stamp":"2024-09-21T03:33:23.016672+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:32:35.906114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":1682,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":4058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":13,"ondisk_log_size":13,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":1,"up_from":16,"seq":68719476768,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32380,"kb_used_data":1188,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339460,"statfs":{"total":96636764160,"available":96603607040,"internally_reserved":0,"allocated":1216512,"data_stored":866432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509472,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31788,"kb_used_data":596,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340052,"statfs":{"total":96636764160,"available":96604213248,"internally_reserved":0,"allocated":610304,"data_stored":269397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509471,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31788,"kb_used_data":596,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340052,"statfs":{"total":96636764160,"available":96604213248,"internally_reserved":0,"allocated":610304,"data_stored":269397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509472,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31788,"kb_used_data":596,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340052,"statfs":{"total":96636764160,"available":96604213248,"internally_reserved":0,"allocated":610304,"data_stored":269397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509471,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31788,"kb_used_data":596,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340052,"statfs":{"total":96636764160,"available":96604213248,"internally_reserved":0,"allocated":610304,"data_stored":269397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542176,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35984,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335856,"statfs":{"total":96636764160,"available":96599916544,"internally_reserved":0,"allocated":651264,"data_stored":290505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574880,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32412,"kb_used_data":1216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339428,"statfs":{"total":96636764160,"available":96603574272,"internally_reserved":0,"allocated":1245184,"data_stored":875301,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542177,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36000,"kb_used_data":652,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335840,"statfs":{"total":96636764160,"available":96599900160,"internally_reserved":0,"allocated":667648,"data_stored":291627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T03:35:53.503 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T03:35:53.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:53.640+0000 7f549a79d640 1 -- 172.21.15.38:0/4013432884 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f54940b4050 msgr2=0x7f549415c430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:53.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:53.640+0000 7f549a79d640 1 --2- 172.21.15.38:0/4013432884 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f54940b4050 0x7f549415c430 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f5494072540 tx=0x7f549002a560 comp rx=0 tx=0).stop 2024-09-21T03:35:53.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:53.640+0000 7f549a79d640 1 -- 172.21.15.38:0/4013432884 shutdown_connections 2024-09-21T03:35:53.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:53.640+0000 7f549a79d640 1 --2- 172.21.15.38:0/4013432884 >> v2:172.21.15.195:3300/0 conn(0x7f5494071160 0x7f5494071570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:53.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:53.640+0000 7f549a79d640 1 --2- 172.21.15.38:0/4013432884 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f54940b4050 0x7f549415c430 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:53.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:53.640+0000 7f549a79d640 1 -- 172.21.15.38:0/4013432884 >> 172.21.15.38:0/4013432884 conn(0x7f54940d10e0 msgr2=0x7f54940d14f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:53.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:53.641+0000 7f549a79d640 1 -- 172.21.15.38:0/4013432884 shutdown_connections 2024-09-21T03:35:53.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:53.641+0000 7f549a79d640 1 -- 172.21.15.38:0/4013432884 wait complete. 2024-09-21T03:35:53.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:53.642+0000 7f549a79d640 1 Processor -- start 2024-09-21T03:35:53.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:53.642+0000 7f549a79d640 1 -- start start 2024-09-21T03:35:53.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:53.644+0000 7f549a79d640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5494071160 0x7f54941587f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:53.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:53.644+0000 7f549a79d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f54940b4050 0x7f549415a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:53.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:53.644+0000 7f549a79d640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f549415efd0 0x7f549415a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:53.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:53.644+0000 7f549a79d640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f549406f550 con 0x7f54940b4050 2024-09-21T03:35:53.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:53.644+0000 7f549a79d640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f5494152950 con 0x7f5494071160 2024-09-21T03:35:53.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:53.644+0000 7f549a79d640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f549406f6d0 con 0x7f549415efd0 2024-09-21T03:35:53.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:53.644+0000 7f5498f9a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f54940b4050 0x7f549415a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:53.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:53.644+0000 7f5498f9a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f54940b4050 0x7f549415a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44344/0 (socket says 172.21.15.38:44344) 2024-09-21T03:35:53.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:53.644+0000 7f5498f9a640 1 -- 172.21.15.38:0/2764660583 learned_addr learned my addr 172.21.15.38:0/2764660583 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:35:53.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:53.644+0000 7f549979b640 1 --2- 172.21.15.38:0/2764660583 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5494071160 0x7f54941587f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:53.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:53.644+0000 7f5499f9c640 1 --2- 172.21.15.38:0/2764660583 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f549415efd0 0x7f549415a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:53.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:53.644+0000 7f5498f9a640 1 -- 172.21.15.38:0/2764660583 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f549415efd0 msgr2=0x7f549415a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:53.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:53.644+0000 7f5498f9a640 1 --2- 172.21.15.38:0/2764660583 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f549415efd0 0x7f549415a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:53.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:53.644+0000 7f5498f9a640 1 -- 172.21.15.38:0/2764660583 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5494071160 msgr2=0x7f54941587f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:53.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:53.644+0000 7f5498f9a640 1 --2- 172.21.15.38:0/2764660583 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5494071160 0x7f54941587f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:53.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:53.644+0000 7f5498f9a640 1 -- 172.21.15.38:0/2764660583 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f549414e9a0 con 0x7f54940b4050 2024-09-21T03:35:53.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:53.645+0000 7f5498f9a640 1 --2- 172.21.15.38:0/2764660583 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f54940b4050 0x7f549415a460 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f548400d740 tx=0x7f548400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:53.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:53.645+0000 7f5489ffb640 1 -- 172.21.15.38:0/2764660583 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5484017070 con 0x7f54940b4050 2024-09-21T03:35:53.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:53.645+0000 7f5489ffb640 1 -- 172.21.15.38:0/2764660583 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5484009d20 con 0x7f54940b4050 2024-09-21T03:35:53.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:53.645+0000 7f5489ffb640 1 -- 172.21.15.38:0/2764660583 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54840106c0 con 0x7f54940b4050 2024-09-21T03:35:53.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:53.645+0000 7f549a79d640 1 -- 172.21.15.38:0/2764660583 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f549414ec80 con 0x7f54940b4050 2024-09-21T03:35:53.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:53.645+0000 7f549a79d640 1 -- 172.21.15.38:0/2764660583 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f549414f240 con 0x7f54940b4050 2024-09-21T03:35:53.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:53.646+0000 7f549a79d640 1 -- 172.21.15.38:0/2764660583 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54940cc9a0 con 0x7f54940b4050 2024-09-21T03:35:53.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:53.647+0000 7f5489ffb640 1 -- 172.21.15.38:0/2764660583 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f548401a070 con 0x7f54940b4050 2024-09-21T03:35:53.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:53.647+0000 7f5489ffb640 1 --2- 172.21.15.38:0/2764660583 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5478078b90 0x7f547807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:53.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:53.647+0000 7f5489ffb640 1 -- 172.21.15.38:0/2764660583 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 6101+0+0 (secure 0 0 0) 0x7f548409afd0 con 0x7f54940b4050 2024-09-21T03:35:53.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:53.648+0000 7f549979b640 1 --2- 172.21.15.38:0/2764660583 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5478078b90 0x7f547807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:53.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:53.648+0000 7f549979b640 1 --2- 172.21.15.38:0/2764660583 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5478078b90 0x7f547807b0a0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f5490033000 tx=0x7f5490004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:53.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:53.652+0000 7f5489ffb640 1 -- 172.21.15.38:0/2764660583 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f54940cc9a0 con 0x7f54940b4050 2024-09-21T03:35:53.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:53.836+0000 7f549a79d640 1 -- 172.21.15.38:0/2764660583 --> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f549415b290 con 0x7f5478078b90 2024-09-21T03:35:53.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:53.841+0000 7f5489ffb640 1 -- 172.21.15.38:0/2764660583 <== mgr.4117 v2:172.21.15.195:6800/1631901344 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251936 (secure 0 0 0) 0x7f549415b290 con 0x7f5478078b90 2024-09-21T03:35:53.843 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:35:53.852 INFO:teuthology.orchestra.run.smithi038.stderr:dumped all 2024-09-21T03:35:53.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:53.844+0000 7f549a79d640 1 -- 172.21.15.38:0/2764660583 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5478078b90 msgr2=0x7f547807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:53.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:53.844+0000 7f549a79d640 1 --2- 172.21.15.38:0/2764660583 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5478078b90 0x7f547807b0a0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f5490033000 tx=0x7f5490004730 comp rx=0 tx=0).stop 2024-09-21T03:35:53.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:53.844+0000 7f549a79d640 1 -- 172.21.15.38:0/2764660583 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f54940b4050 msgr2=0x7f549415a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:53.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:53.844+0000 7f549a79d640 1 --2- 172.21.15.38:0/2764660583 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f54940b4050 0x7f549415a460 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f548400d740 tx=0x7f548400dc00 comp rx=0 tx=0).stop 2024-09-21T03:35:53.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:53.844+0000 7f549a79d640 1 -- 172.21.15.38:0/2764660583 shutdown_connections 2024-09-21T03:35:53.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:53.844+0000 7f549a79d640 1 --2- 172.21.15.38:0/2764660583 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5478078b90 0x7f547807b0a0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:53.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:53.844+0000 7f549a79d640 1 --2- 172.21.15.38:0/2764660583 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f549415efd0 0x7f549415a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:53.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:53.844+0000 7f549a79d640 1 --2- 172.21.15.38:0/2764660583 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f54940b4050 0x7f549415a460 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:53.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:53.844+0000 7f549a79d640 1 --2- 172.21.15.38:0/2764660583 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5494071160 0x7f54941587f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:53.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:53.844+0000 7f549a79d640 1 -- 172.21.15.38:0/2764660583 >> 172.21.15.38:0/2764660583 conn(0x7f54940d10e0 msgr2=0x7f5494068d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:53.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:53.844+0000 7f549a79d640 1 -- 172.21.15.38:0/2764660583 shutdown_connections 2024-09-21T03:35:53.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:53.844+0000 7f549a79d640 1 -- 172.21.15.38:0/2764660583 wait complete. 2024-09-21T03:35:53.857 INFO:teuthology.orchestra.run.smithi038.stdout:{"pg_ready":true,"pg_map":{"version":123,"stamp":"2024-09-21T03:35:53.014649+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":264128,"kb_used_data":6280,"kb_used_omap":12,"kb_used_meta":257651,"kb_avail":754710592,"statfs":{"total":773094113280,"available":772823646208,"internally_reserved":0,"allocated":6430720,"data_stored":3504940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12748,"internal_metadata":263835188},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.125543"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.869704+0000","last_change":"2024-09-21T03:35:44.918690+0000","last_active":"2024-09-21T03:35:46.869704+0000","last_peered":"2024-09-21T03:35:46.869704+0000","last_clean":"2024-09-21T03:35:46.869704+0000","last_became_active":"2024-09-21T03:35:44.917722+0000","last_became_peered":"2024-09-21T03:35:44.917722+0000","last_unstale":"2024-09-21T03:35:46.869704+0000","last_undegraded":"2024-09-21T03:35:46.869704+0000","last_fullsized":"2024-09-21T03:35:46.869704+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:17:03.401604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:45.908591+0000","last_change":"2024-09-21T03:35:44.914114+0000","last_active":"2024-09-21T03:35:45.908591+0000","last_peered":"2024-09-21T03:35:45.908591+0000","last_clean":"2024-09-21T03:35:45.908591+0000","last_became_active":"2024-09-21T03:35:44.913796+0000","last_became_peered":"2024-09-21T03:35:44.913796+0000","last_unstale":"2024-09-21T03:35:45.908591+0000","last_undegraded":"2024-09-21T03:35:45.908591+0000","last_fullsized":"2024-09-21T03:35:45.908591+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:15:26.930354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.869706+0000","last_change":"2024-09-21T03:35:44.914485+0000","last_active":"2024-09-21T03:35:46.869706+0000","last_peered":"2024-09-21T03:35:46.869706+0000","last_clean":"2024-09-21T03:35:46.869706+0000","last_became_active":"2024-09-21T03:35:44.914150+0000","last_became_peered":"2024-09-21T03:35:44.914150+0000","last_unstale":"2024-09-21T03:35:46.869706+0000","last_undegraded":"2024-09-21T03:35:46.869706+0000","last_fullsized":"2024-09-21T03:35:46.869706+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:34:38.258070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.641226+0000","last_change":"2024-09-21T03:35:44.921404+0000","last_active":"2024-09-21T03:35:46.641226+0000","last_peered":"2024-09-21T03:35:46.641226+0000","last_clean":"2024-09-21T03:35:46.641226+0000","last_became_active":"2024-09-21T03:35:44.921059+0000","last_became_peered":"2024-09-21T03:35:44.921059+0000","last_unstale":"2024-09-21T03:35:46.641226+0000","last_undegraded":"2024-09-21T03:35:46.641226+0000","last_fullsized":"2024-09-21T03:35:46.641226+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:33:55.918413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:45.908583+0000","last_change":"2024-09-21T03:35:44.918730+0000","last_active":"2024-09-21T03:35:45.908583+0000","last_peered":"2024-09-21T03:35:45.908583+0000","last_clean":"2024-09-21T03:35:45.908583+0000","last_became_active":"2024-09-21T03:35:44.918430+0000","last_became_peered":"2024-09-21T03:35:44.918430+0000","last_unstale":"2024-09-21T03:35:45.908583+0000","last_undegraded":"2024-09-21T03:35:45.908583+0000","last_fullsized":"2024-09-21T03:35:45.908583+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:52:54.152973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:45.909326+0000","last_change":"2024-09-21T03:35:44.915276+0000","last_active":"2024-09-21T03:35:45.909326+0000","last_peered":"2024-09-21T03:35:45.909326+0000","last_clean":"2024-09-21T03:35:45.909326+0000","last_became_active":"2024-09-21T03:35:44.914949+0000","last_became_peered":"2024-09-21T03:35:44.914949+0000","last_unstale":"2024-09-21T03:35:45.909326+0000","last_undegraded":"2024-09-21T03:35:45.909326+0000","last_fullsized":"2024-09-21T03:35:45.909326+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:33:00.469260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.680170+0000","last_change":"2024-09-21T03:35:44.900285+0000","last_active":"2024-09-21T03:35:46.680170+0000","last_peered":"2024-09-21T03:35:46.680170+0000","last_clean":"2024-09-21T03:35:46.680170+0000","last_became_active":"2024-09-21T03:35:44.899955+0000","last_became_peered":"2024-09-21T03:35:44.899955+0000","last_unstale":"2024-09-21T03:35:46.680170+0000","last_undegraded":"2024-09-21T03:35:46.680170+0000","last_fullsized":"2024-09-21T03:35:46.680170+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:44:47.487941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:45.909087+0000","last_change":"2024-09-21T03:35:44.907006+0000","last_active":"2024-09-21T03:35:45.909087+0000","last_peered":"2024-09-21T03:35:45.909087+0000","last_clean":"2024-09-21T03:35:45.909087+0000","last_became_active":"2024-09-21T03:35:44.906696+0000","last_became_peered":"2024-09-21T03:35:44.906696+0000","last_unstale":"2024-09-21T03:35:45.909087+0000","last_undegraded":"2024-09-21T03:35:45.909087+0000","last_fullsized":"2024-09-21T03:35:45.909087+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:31:52.864107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.869989+0000","last_change":"2024-09-21T03:35:44.923776+0000","last_active":"2024-09-21T03:35:46.869989+0000","last_peered":"2024-09-21T03:35:46.869989+0000","last_clean":"2024-09-21T03:35:46.869989+0000","last_became_active":"2024-09-21T03:35:44.923315+0000","last_became_peered":"2024-09-21T03:35:44.923315+0000","last_unstale":"2024-09-21T03:35:46.869989+0000","last_undegraded":"2024-09-21T03:35:46.869989+0000","last_fullsized":"2024-09-21T03:35:46.869989+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:05:17.646813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.680010+0000","last_change":"2024-09-21T03:35:44.906104+0000","last_active":"2024-09-21T03:35:46.680010+0000","last_peered":"2024-09-21T03:35:46.680010+0000","last_clean":"2024-09-21T03:35:46.680010+0000","last_became_active":"2024-09-21T03:35:44.905786+0000","last_became_peered":"2024-09-21T03:35:44.905786+0000","last_unstale":"2024-09-21T03:35:46.680010+0000","last_undegraded":"2024-09-21T03:35:46.680010+0000","last_fullsized":"2024-09-21T03:35:46.680010+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:56:36.052470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.869616+0000","last_change":"2024-09-21T03:35:44.913177+0000","last_active":"2024-09-21T03:35:46.869616+0000","last_peered":"2024-09-21T03:35:46.869616+0000","last_clean":"2024-09-21T03:35:46.869616+0000","last_became_active":"2024-09-21T03:35:44.912856+0000","last_became_peered":"2024-09-21T03:35:44.912856+0000","last_unstale":"2024-09-21T03:35:46.869616+0000","last_undegraded":"2024-09-21T03:35:46.869616+0000","last_fullsized":"2024-09-21T03:35:46.869616+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:53:31.119799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:45.909249+0000","last_change":"2024-09-21T03:35:44.910618+0000","last_active":"2024-09-21T03:35:45.909249+0000","last_peered":"2024-09-21T03:35:45.909249+0000","last_clean":"2024-09-21T03:35:45.909249+0000","last_became_active":"2024-09-21T03:35:44.910291+0000","last_became_peered":"2024-09-21T03:35:44.910291+0000","last_unstale":"2024-09-21T03:35:45.909249+0000","last_undegraded":"2024-09-21T03:35:45.909249+0000","last_fullsized":"2024-09-21T03:35:45.909249+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:38:36.300449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:45.909363+0000","last_change":"2024-09-21T03:35:44.919405+0000","last_active":"2024-09-21T03:35:45.909363+0000","last_peered":"2024-09-21T03:35:45.909363+0000","last_clean":"2024-09-21T03:35:45.909363+0000","last_became_active":"2024-09-21T03:35:44.919082+0000","last_became_peered":"2024-09-21T03:35:44.919082+0000","last_unstale":"2024-09-21T03:35:45.909363+0000","last_undegraded":"2024-09-21T03:35:45.909363+0000","last_fullsized":"2024-09-21T03:35:45.909363+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:54:00.361227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.868924+0000","last_change":"2024-09-21T03:35:44.916342+0000","last_active":"2024-09-21T03:35:46.868924+0000","last_peered":"2024-09-21T03:35:46.868924+0000","last_clean":"2024-09-21T03:35:46.868924+0000","last_became_active":"2024-09-21T03:35:44.916047+0000","last_became_peered":"2024-09-21T03:35:44.916047+0000","last_unstale":"2024-09-21T03:35:46.868924+0000","last_undegraded":"2024-09-21T03:35:46.868924+0000","last_fullsized":"2024-09-21T03:35:46.868924+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:12:22.459838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.641190+0000","last_change":"2024-09-21T03:35:44.919326+0000","last_active":"2024-09-21T03:35:46.641190+0000","last_peered":"2024-09-21T03:35:46.641190+0000","last_clean":"2024-09-21T03:35:46.641190+0000","last_became_active":"2024-09-21T03:35:44.919004+0000","last_became_peered":"2024-09-21T03:35:44.919004+0000","last_unstale":"2024-09-21T03:35:46.641190+0000","last_undegraded":"2024-09-21T03:35:46.641190+0000","last_fullsized":"2024-09-21T03:35:46.641190+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:00:51.024360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:45.909119+0000","last_change":"2024-09-21T03:35:44.909346+0000","last_active":"2024-09-21T03:35:45.909119+0000","last_peered":"2024-09-21T03:35:45.909119+0000","last_clean":"2024-09-21T03:35:45.909119+0000","last_became_active":"2024-09-21T03:35:44.908743+0000","last_became_peered":"2024-09-21T03:35:44.908743+0000","last_unstale":"2024-09-21T03:35:45.909119+0000","last_undegraded":"2024-09-21T03:35:45.909119+0000","last_fullsized":"2024-09-21T03:35:45.909119+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:05:30.105550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.870146+0000","last_change":"2024-09-21T03:35:44.918344+0000","last_active":"2024-09-21T03:35:46.870146+0000","last_peered":"2024-09-21T03:35:46.870146+0000","last_clean":"2024-09-21T03:35:46.870146+0000","last_became_active":"2024-09-21T03:35:44.917344+0000","last_became_peered":"2024-09-21T03:35:44.917344+0000","last_unstale":"2024-09-21T03:35:46.870146+0000","last_undegraded":"2024-09-21T03:35:46.870146+0000","last_fullsized":"2024-09-21T03:35:46.870146+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:10:22.417344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.641324+0000","last_change":"2024-09-21T03:35:44.920057+0000","last_active":"2024-09-21T03:35:46.641324+0000","last_peered":"2024-09-21T03:35:46.641324+0000","last_clean":"2024-09-21T03:35:46.641324+0000","last_became_active":"2024-09-21T03:35:44.919683+0000","last_became_peered":"2024-09-21T03:35:44.919683+0000","last_unstale":"2024-09-21T03:35:46.641324+0000","last_undegraded":"2024-09-21T03:35:46.641324+0000","last_fullsized":"2024-09-21T03:35:46.641324+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:17:36.663074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.869952+0000","last_change":"2024-09-21T03:35:44.919698+0000","last_active":"2024-09-21T03:35:46.869952+0000","last_peered":"2024-09-21T03:35:46.869952+0000","last_clean":"2024-09-21T03:35:46.869952+0000","last_became_active":"2024-09-21T03:35:44.919400+0000","last_became_peered":"2024-09-21T03:35:44.919400+0000","last_unstale":"2024-09-21T03:35:46.869952+0000","last_undegraded":"2024-09-21T03:35:46.869952+0000","last_fullsized":"2024-09-21T03:35:46.869952+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:18:19.307626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:45.908491+0000","last_change":"2024-09-21T03:35:44.916415+0000","last_active":"2024-09-21T03:35:45.908491+0000","last_peered":"2024-09-21T03:35:45.908491+0000","last_clean":"2024-09-21T03:35:45.908491+0000","last_became_active":"2024-09-21T03:35:44.915990+0000","last_became_peered":"2024-09-21T03:35:44.915990+0000","last_unstale":"2024-09-21T03:35:45.908491+0000","last_undegraded":"2024-09-21T03:35:45.908491+0000","last_fullsized":"2024-09-21T03:35:45.908491+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:03:19.640278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.641174+0000","last_change":"2024-09-21T03:35:44.915023+0000","last_active":"2024-09-21T03:35:46.641174+0000","last_peered":"2024-09-21T03:35:46.641174+0000","last_clean":"2024-09-21T03:35:46.641174+0000","last_became_active":"2024-09-21T03:35:44.914677+0000","last_became_peered":"2024-09-21T03:35:44.914677+0000","last_unstale":"2024-09-21T03:35:46.641174+0000","last_undegraded":"2024-09-21T03:35:46.641174+0000","last_fullsized":"2024-09-21T03:35:46.641174+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:25:53.313219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.868808+0000","last_change":"2024-09-21T03:35:44.922185+0000","last_active":"2024-09-21T03:35:46.868808+0000","last_peered":"2024-09-21T03:35:46.868808+0000","last_clean":"2024-09-21T03:35:46.868808+0000","last_became_active":"2024-09-21T03:35:44.921473+0000","last_became_peered":"2024-09-21T03:35:44.921473+0000","last_unstale":"2024-09-21T03:35:46.868808+0000","last_undegraded":"2024-09-21T03:35:46.868808+0000","last_fullsized":"2024-09-21T03:35:46.868808+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:10:57.276638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.869035+0000","last_change":"2024-09-21T03:35:44.916166+0000","last_active":"2024-09-21T03:35:46.869035+0000","last_peered":"2024-09-21T03:35:46.869035+0000","last_clean":"2024-09-21T03:35:46.869035+0000","last_became_active":"2024-09-21T03:35:44.915841+0000","last_became_peered":"2024-09-21T03:35:44.915841+0000","last_unstale":"2024-09-21T03:35:46.869035+0000","last_undegraded":"2024-09-21T03:35:46.869035+0000","last_fullsized":"2024-09-21T03:35:46.869035+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:58:35.540873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.868743+0000","last_change":"2024-09-21T03:35:44.915458+0000","last_active":"2024-09-21T03:35:46.868743+0000","last_peered":"2024-09-21T03:35:46.868743+0000","last_clean":"2024-09-21T03:35:46.868743+0000","last_became_active":"2024-09-21T03:35:44.915163+0000","last_became_peered":"2024-09-21T03:35:44.915163+0000","last_unstale":"2024-09-21T03:35:46.868743+0000","last_undegraded":"2024-09-21T03:35:46.868743+0000","last_fullsized":"2024-09-21T03:35:46.868743+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:03:55.703336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.870005+0000","last_change":"2024-09-21T03:35:44.918018+0000","last_active":"2024-09-21T03:35:46.870005+0000","last_peered":"2024-09-21T03:35:46.870005+0000","last_clean":"2024-09-21T03:35:46.870005+0000","last_became_active":"2024-09-21T03:35:44.916939+0000","last_became_peered":"2024-09-21T03:35:44.916939+0000","last_unstale":"2024-09-21T03:35:46.870005+0000","last_undegraded":"2024-09-21T03:35:46.870005+0000","last_fullsized":"2024-09-21T03:35:46.870005+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:32:35.988080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.869597+0000","last_change":"2024-09-21T03:35:44.904810+0000","last_active":"2024-09-21T03:35:46.869597+0000","last_peered":"2024-09-21T03:35:46.869597+0000","last_clean":"2024-09-21T03:35:46.869597+0000","last_became_active":"2024-09-21T03:35:44.904499+0000","last_became_peered":"2024-09-21T03:35:44.904499+0000","last_unstale":"2024-09-21T03:35:46.869597+0000","last_undegraded":"2024-09-21T03:35:46.869597+0000","last_fullsized":"2024-09-21T03:35:46.869597+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:37:49.682407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:45.909512+0000","last_change":"2024-09-21T03:35:44.907817+0000","last_active":"2024-09-21T03:35:45.909512+0000","last_peered":"2024-09-21T03:35:45.909512+0000","last_clean":"2024-09-21T03:35:45.909512+0000","last_became_active":"2024-09-21T03:35:44.907520+0000","last_became_peered":"2024-09-21T03:35:44.907520+0000","last_unstale":"2024-09-21T03:35:45.909512+0000","last_undegraded":"2024-09-21T03:35:45.909512+0000","last_fullsized":"2024-09-21T03:35:45.909512+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:23:10.368716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:45.908781+0000","last_change":"2024-09-21T03:35:44.909030+0000","last_active":"2024-09-21T03:35:45.908781+0000","last_peered":"2024-09-21T03:35:45.908781+0000","last_clean":"2024-09-21T03:35:45.908781+0000","last_became_active":"2024-09-21T03:35:44.908688+0000","last_became_peered":"2024-09-21T03:35:44.908688+0000","last_unstale":"2024-09-21T03:35:45.908781+0000","last_undegraded":"2024-09-21T03:35:45.908781+0000","last_fullsized":"2024-09-21T03:35:45.908781+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:14:28.930896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.869246+0000","last_change":"2024-09-21T03:35:44.913781+0000","last_active":"2024-09-21T03:35:46.869246+0000","last_peered":"2024-09-21T03:35:46.869246+0000","last_clean":"2024-09-21T03:35:46.869246+0000","last_became_active":"2024-09-21T03:35:44.913469+0000","last_became_peered":"2024-09-21T03:35:44.913469+0000","last_unstale":"2024-09-21T03:35:46.869246+0000","last_undegraded":"2024-09-21T03:35:46.869246+0000","last_fullsized":"2024-09-21T03:35:46.869246+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:52:54.152973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:45.909641+0000","last_change":"2024-09-21T03:35:44.916829+0000","last_active":"2024-09-21T03:35:45.909641+0000","last_peered":"2024-09-21T03:35:45.909641+0000","last_clean":"2024-09-21T03:35:45.909641+0000","last_became_active":"2024-09-21T03:35:44.916523+0000","last_became_peered":"2024-09-21T03:35:44.916523+0000","last_unstale":"2024-09-21T03:35:45.909641+0000","last_undegraded":"2024-09-21T03:35:45.909641+0000","last_fullsized":"2024-09-21T03:35:45.909641+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:39:53.931510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.868885+0000","last_change":"2024-09-21T03:35:44.920858+0000","last_active":"2024-09-21T03:35:46.868885+0000","last_peered":"2024-09-21T03:35:46.868885+0000","last_clean":"2024-09-21T03:35:46.868885+0000","last_became_active":"2024-09-21T03:35:44.920539+0000","last_became_peered":"2024-09-21T03:35:44.920539+0000","last_unstale":"2024-09-21T03:35:46.868885+0000","last_undegraded":"2024-09-21T03:35:46.868885+0000","last_fullsized":"2024-09-21T03:35:46.868885+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:09:20.328763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:45.908158+0000","last_change":"2024-09-21T03:35:44.920642+0000","last_active":"2024-09-21T03:35:45.908158+0000","last_peered":"2024-09-21T03:35:45.908158+0000","last_clean":"2024-09-21T03:35:45.908158+0000","last_became_active":"2024-09-21T03:35:44.920306+0000","last_became_peered":"2024-09-21T03:35:44.920306+0000","last_unstale":"2024-09-21T03:35:45.908158+0000","last_undegraded":"2024-09-21T03:35:45.908158+0000","last_fullsized":"2024-09-21T03:35:45.908158+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:24:22.041084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:45.909298+0000","last_change":"2024-09-21T03:35:44.921565+0000","last_active":"2024-09-21T03:35:45.909298+0000","last_peered":"2024-09-21T03:35:45.909298+0000","last_clean":"2024-09-21T03:35:45.909298+0000","last_became_active":"2024-09-21T03:35:44.921209+0000","last_became_peered":"2024-09-21T03:35:44.921209+0000","last_unstale":"2024-09-21T03:35:45.909298+0000","last_undegraded":"2024-09-21T03:35:45.909298+0000","last_fullsized":"2024-09-21T03:35:45.909298+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:16:38.503430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"54'4","reported_seq":19,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.868752+0000","last_change":"2024-09-21T03:35:43.909828+0000","last_active":"2024-09-21T03:35:46.868752+0000","last_peered":"2024-09-21T03:35:46.868752+0000","last_clean":"2024-09-21T03:35:46.868752+0000","last_became_active":"2024-09-21T03:35:43.909503+0000","last_became_peered":"2024-09-21T03:35:43.909503+0000","last_unstale":"2024-09-21T03:35:46.868752+0000","last_undegraded":"2024-09-21T03:35:46.868752+0000","last_fullsized":"2024-09-21T03:35:46.868752+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_clean_scrub_stamp":"2024-09-21T03:35:42.877874+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:00:11.751679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:45.909399+0000","last_change":"2024-09-21T03:35:44.926027+0000","last_active":"2024-09-21T03:35:45.909399+0000","last_peered":"2024-09-21T03:35:45.909399+0000","last_clean":"2024-09-21T03:35:45.909399+0000","last_became_active":"2024-09-21T03:35:44.925682+0000","last_became_peered":"2024-09-21T03:35:44.925682+0000","last_unstale":"2024-09-21T03:35:45.909399+0000","last_undegraded":"2024-09-21T03:35:45.909399+0000","last_fullsized":"2024-09-21T03:35:45.909399+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:17:11.792215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1e","version":"54'1","reported_seq":16,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:45.909740+0000","last_change":"2024-09-21T03:35:43.901432+0000","last_active":"2024-09-21T03:35:45.909740+0000","last_peered":"2024-09-21T03:35:45.909740+0000","last_clean":"2024-09-21T03:35:45.909740+0000","last_became_active":"2024-09-21T03:35:43.901134+0000","last_became_peered":"2024-09-21T03:35:43.901134+0000","last_unstale":"2024-09-21T03:35:45.909740+0000","last_undegraded":"2024-09-21T03:35:45.909740+0000","last_fullsized":"2024-09-21T03:35:45.909740+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_clean_scrub_stamp":"2024-09-21T03:35:42.877874+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:01:21.207065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:45.909409+0000","last_change":"2024-09-21T03:35:44.920651+0000","last_active":"2024-09-21T03:35:45.909409+0000","last_peered":"2024-09-21T03:35:45.909409+0000","last_clean":"2024-09-21T03:35:45.909409+0000","last_became_active":"2024-09-21T03:35:44.918874+0000","last_became_peered":"2024-09-21T03:35:44.918874+0000","last_unstale":"2024-09-21T03:35:45.909409+0000","last_undegraded":"2024-09-21T03:35:45.909409+0000","last_fullsized":"2024-09-21T03:35:45.909409+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:35:57.010405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1d","version":"54'2","reported_seq":17,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.680073+0000","last_change":"2024-09-21T03:35:43.902853+0000","last_active":"2024-09-21T03:35:46.680073+0000","last_peered":"2024-09-21T03:35:46.680073+0000","last_clean":"2024-09-21T03:35:46.680073+0000","last_became_active":"2024-09-21T03:35:43.902514+0000","last_became_peered":"2024-09-21T03:35:43.902514+0000","last_unstale":"2024-09-21T03:35:46.680073+0000","last_undegraded":"2024-09-21T03:35:46.680073+0000","last_fullsized":"2024-09-21T03:35:46.680073+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_clean_scrub_stamp":"2024-09-21T03:35:42.877874+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:40:01.918835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.680012+0000","last_change":"2024-09-21T03:35:44.912392+0000","last_active":"2024-09-21T03:35:46.680012+0000","last_peered":"2024-09-21T03:35:46.680012+0000","last_clean":"2024-09-21T03:35:46.680012+0000","last_became_active":"2024-09-21T03:35:44.911332+0000","last_became_peered":"2024-09-21T03:35:44.911332+0000","last_unstale":"2024-09-21T03:35:46.680012+0000","last_undegraded":"2024-09-21T03:35:46.680012+0000","last_fullsized":"2024-09-21T03:35:46.680012+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:16:36.470418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1c","version":"54'2","reported_seq":17,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:45.909186+0000","last_change":"2024-09-21T03:35:43.911405+0000","last_active":"2024-09-21T03:35:45.909186+0000","last_peered":"2024-09-21T03:35:45.909186+0000","last_clean":"2024-09-21T03:35:45.909186+0000","last_became_active":"2024-09-21T03:35:43.911082+0000","last_became_peered":"2024-09-21T03:35:43.911082+0000","last_unstale":"2024-09-21T03:35:45.909186+0000","last_undegraded":"2024-09-21T03:35:45.909186+0000","last_fullsized":"2024-09-21T03:35:45.909186+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_clean_scrub_stamp":"2024-09-21T03:35:42.877874+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:58:53.732054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.868706+0000","last_change":"2024-09-21T03:35:44.926625+0000","last_active":"2024-09-21T03:35:46.868706+0000","last_peered":"2024-09-21T03:35:46.868706+0000","last_clean":"2024-09-21T03:35:46.868706+0000","last_became_active":"2024-09-21T03:35:44.926308+0000","last_became_peered":"2024-09-21T03:35:44.926308+0000","last_unstale":"2024-09-21T03:35:46.868706+0000","last_undegraded":"2024-09-21T03:35:46.868706+0000","last_fullsized":"2024-09-21T03:35:46.868706+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:53:52.547424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1b","version":"54'1","reported_seq":16,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.868936+0000","last_change":"2024-09-21T03:35:43.909887+0000","last_active":"2024-09-21T03:35:46.868936+0000","last_peered":"2024-09-21T03:35:46.868936+0000","last_clean":"2024-09-21T03:35:46.868936+0000","last_became_active":"2024-09-21T03:35:43.909558+0000","last_became_peered":"2024-09-21T03:35:43.909558+0000","last_unstale":"2024-09-21T03:35:46.868936+0000","last_undegraded":"2024-09-21T03:35:46.868936+0000","last_fullsized":"2024-09-21T03:35:46.868936+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_clean_scrub_stamp":"2024-09-21T03:35:42.877874+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:03:39.525599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:45.909861+0000","last_change":"2024-09-21T03:35:44.914357+0000","last_active":"2024-09-21T03:35:45.909861+0000","last_peered":"2024-09-21T03:35:45.909861+0000","last_clean":"2024-09-21T03:35:45.909861+0000","last_became_active":"2024-09-21T03:35:44.914065+0000","last_became_peered":"2024-09-21T03:35:44.914065+0000","last_unstale":"2024-09-21T03:35:45.909861+0000","last_undegraded":"2024-09-21T03:35:45.909861+0000","last_fullsized":"2024-09-21T03:35:45.909861+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:56:27.832354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"54'1","reported_seq":16,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:45.909724+0000","last_change":"2024-09-21T03:35:43.910078+0000","last_active":"2024-09-21T03:35:45.909724+0000","last_peered":"2024-09-21T03:35:45.909724+0000","last_clean":"2024-09-21T03:35:45.909724+0000","last_became_active":"2024-09-21T03:35:43.909315+0000","last_became_peered":"2024-09-21T03:35:43.909315+0000","last_unstale":"2024-09-21T03:35:45.909724+0000","last_undegraded":"2024-09-21T03:35:45.909724+0000","last_fullsized":"2024-09-21T03:35:45.909724+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_clean_scrub_stamp":"2024-09-21T03:35:42.877874+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:09:19.326024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.680104+0000","last_change":"2024-09-21T03:35:44.910050+0000","last_active":"2024-09-21T03:35:46.680104+0000","last_peered":"2024-09-21T03:35:46.680104+0000","last_clean":"2024-09-21T03:35:46.680104+0000","last_became_active":"2024-09-21T03:35:44.909755+0000","last_became_peered":"2024-09-21T03:35:44.909755+0000","last_unstale":"2024-09-21T03:35:46.680104+0000","last_undegraded":"2024-09-21T03:35:46.680104+0000","last_fullsized":"2024-09-21T03:35:46.680104+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:30:38.153649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":15,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:45.909567+0000","last_change":"2024-09-21T03:35:43.909144+0000","last_active":"2024-09-21T03:35:45.909567+0000","last_peered":"2024-09-21T03:35:45.909567+0000","last_clean":"2024-09-21T03:35:45.909567+0000","last_became_active":"2024-09-21T03:35:43.908808+0000","last_became_peered":"2024-09-21T03:35:43.908808+0000","last_unstale":"2024-09-21T03:35:45.909567+0000","last_undegraded":"2024-09-21T03:35:45.909567+0000","last_fullsized":"2024-09-21T03:35:45.909567+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_clean_scrub_stamp":"2024-09-21T03:35:42.877874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:11:44.205740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.641611+0000","last_change":"2024-09-21T03:35:44.907438+0000","last_active":"2024-09-21T03:35:46.641611+0000","last_peered":"2024-09-21T03:35:46.641611+0000","last_clean":"2024-09-21T03:35:46.641611+0000","last_became_active":"2024-09-21T03:35:44.907114+0000","last_became_peered":"2024-09-21T03:35:44.907114+0000","last_unstale":"2024-09-21T03:35:46.641611+0000","last_undegraded":"2024-09-21T03:35:46.641611+0000","last_fullsized":"2024-09-21T03:35:46.641611+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:07:57.982992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":15,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.868893+0000","last_change":"2024-09-21T03:35:43.903257+0000","last_active":"2024-09-21T03:35:46.868893+0000","last_peered":"2024-09-21T03:35:46.868893+0000","last_clean":"2024-09-21T03:35:46.868893+0000","last_became_active":"2024-09-21T03:35:43.902926+0000","last_became_peered":"2024-09-21T03:35:43.902926+0000","last_unstale":"2024-09-21T03:35:46.868893+0000","last_undegraded":"2024-09-21T03:35:46.868893+0000","last_fullsized":"2024-09-21T03:35:46.868893+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_clean_scrub_stamp":"2024-09-21T03:35:42.877874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:30:24.926072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.868357+0000","last_change":"2024-09-21T03:35:44.916545+0000","last_active":"2024-09-21T03:35:46.868357+0000","last_peered":"2024-09-21T03:35:46.868357+0000","last_clean":"2024-09-21T03:35:46.868357+0000","last_became_active":"2024-09-21T03:35:44.916214+0000","last_became_peered":"2024-09-21T03:35:44.916214+0000","last_unstale":"2024-09-21T03:35:46.868357+0000","last_undegraded":"2024-09-21T03:35:46.868357+0000","last_fullsized":"2024-09-21T03:35:46.868357+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:15:26.930354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":15,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.641611+0000","last_change":"2024-09-21T03:35:43.905541+0000","last_active":"2024-09-21T03:35:46.641611+0000","last_peered":"2024-09-21T03:35:46.641611+0000","last_clean":"2024-09-21T03:35:46.641611+0000","last_became_active":"2024-09-21T03:35:43.905230+0000","last_became_peered":"2024-09-21T03:35:43.905230+0000","last_unstale":"2024-09-21T03:35:46.641611+0000","last_undegraded":"2024-09-21T03:35:46.641611+0000","last_fullsized":"2024-09-21T03:35:46.641611+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_clean_scrub_stamp":"2024-09-21T03:35:42.877874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:46:28.427572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.641071+0000","last_change":"2024-09-21T03:35:44.920481+0000","last_active":"2024-09-21T03:35:46.641071+0000","last_peered":"2024-09-21T03:35:46.641071+0000","last_clean":"2024-09-21T03:35:46.641071+0000","last_became_active":"2024-09-21T03:35:44.919780+0000","last_became_peered":"2024-09-21T03:35:44.919780+0000","last_unstale":"2024-09-21T03:35:46.641071+0000","last_undegraded":"2024-09-21T03:35:46.641071+0000","last_fullsized":"2024-09-21T03:35:46.641071+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:44:29.607853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":15,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.641315+0000","last_change":"2024-09-21T03:35:43.898522+0000","last_active":"2024-09-21T03:35:46.641315+0000","last_peered":"2024-09-21T03:35:46.641315+0000","last_clean":"2024-09-21T03:35:46.641315+0000","last_became_active":"2024-09-21T03:35:43.898171+0000","last_became_peered":"2024-09-21T03:35:43.898171+0000","last_unstale":"2024-09-21T03:35:46.641315+0000","last_undegraded":"2024-09-21T03:35:46.641315+0000","last_fullsized":"2024-09-21T03:35:46.641315+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_clean_scrub_stamp":"2024-09-21T03:35:42.877874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:14:03.947289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.869721+0000","last_change":"2024-09-21T03:35:44.921020+0000","last_active":"2024-09-21T03:35:46.869721+0000","last_peered":"2024-09-21T03:35:46.869721+0000","last_clean":"2024-09-21T03:35:46.869721+0000","last_became_active":"2024-09-21T03:35:44.920573+0000","last_became_peered":"2024-09-21T03:35:44.920573+0000","last_unstale":"2024-09-21T03:35:46.869721+0000","last_undegraded":"2024-09-21T03:35:46.869721+0000","last_fullsized":"2024-09-21T03:35:46.869721+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:17:42.161554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.15","version":"54'3","reported_seq":18,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:45.909957+0000","last_change":"2024-09-21T03:35:43.907104+0000","last_active":"2024-09-21T03:35:45.909957+0000","last_peered":"2024-09-21T03:35:45.909957+0000","last_clean":"2024-09-21T03:35:45.909957+0000","last_became_active":"2024-09-21T03:35:43.906758+0000","last_became_peered":"2024-09-21T03:35:43.906758+0000","last_unstale":"2024-09-21T03:35:45.909957+0000","last_undegraded":"2024-09-21T03:35:45.909957+0000","last_fullsized":"2024-09-21T03:35:45.909957+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_clean_scrub_stamp":"2024-09-21T03:35:42.877874+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:58:34.538134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:45.908402+0000","last_change":"2024-09-21T03:35:44.917380+0000","last_active":"2024-09-21T03:35:45.908402+0000","last_peered":"2024-09-21T03:35:45.908402+0000","last_clean":"2024-09-21T03:35:45.908402+0000","last_became_active":"2024-09-21T03:35:44.917041+0000","last_became_peered":"2024-09-21T03:35:44.917041+0000","last_unstale":"2024-09-21T03:35:45.908402+0000","last_undegraded":"2024-09-21T03:35:45.908402+0000","last_fullsized":"2024-09-21T03:35:45.908402+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:10:45.948969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.14","version":"54'6","reported_seq":21,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.679907+0000","last_change":"2024-09-21T03:35:43.905841+0000","last_active":"2024-09-21T03:35:46.679907+0000","last_peered":"2024-09-21T03:35:46.679907+0000","last_clean":"2024-09-21T03:35:46.679907+0000","last_became_active":"2024-09-21T03:35:43.905505+0000","last_became_peered":"2024-09-21T03:35:43.905505+0000","last_unstale":"2024-09-21T03:35:46.679907+0000","last_undegraded":"2024-09-21T03:35:46.679907+0000","last_fullsized":"2024-09-21T03:35:46.679907+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_clean_scrub_stamp":"2024-09-21T03:35:42.877874+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:40:06.133533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:45.909934+0000","last_change":"2024-09-21T03:35:44.913404+0000","last_active":"2024-09-21T03:35:45.909934+0000","last_peered":"2024-09-21T03:35:45.909934+0000","last_clean":"2024-09-21T03:35:45.909934+0000","last_became_active":"2024-09-21T03:35:44.913071+0000","last_became_peered":"2024-09-21T03:35:44.913071+0000","last_unstale":"2024-09-21T03:35:45.909934+0000","last_undegraded":"2024-09-21T03:35:45.909934+0000","last_fullsized":"2024-09-21T03:35:45.909934+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:11:12.619705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"54'2","reported_seq":17,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:45.909554+0000","last_change":"2024-09-21T03:35:43.913254+0000","last_active":"2024-09-21T03:35:45.909554+0000","last_peered":"2024-09-21T03:35:45.909554+0000","last_clean":"2024-09-21T03:35:45.909554+0000","last_became_active":"2024-09-21T03:35:43.912392+0000","last_became_peered":"2024-09-21T03:35:43.912392+0000","last_unstale":"2024-09-21T03:35:45.909554+0000","last_undegraded":"2024-09-21T03:35:45.909554+0000","last_fullsized":"2024-09-21T03:35:45.909554+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_clean_scrub_stamp":"2024-09-21T03:35:42.877874+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:27:19.635650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.869864+0000","last_change":"2024-09-21T03:35:44.906977+0000","last_active":"2024-09-21T03:35:46.869864+0000","last_peered":"2024-09-21T03:35:46.869864+0000","last_clean":"2024-09-21T03:35:46.869864+0000","last_became_active":"2024-09-21T03:35:44.906651+0000","last_became_peered":"2024-09-21T03:35:44.906651+0000","last_unstale":"2024-09-21T03:35:46.869864+0000","last_undegraded":"2024-09-21T03:35:46.869864+0000","last_fullsized":"2024-09-21T03:35:46.869864+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:13:46.431913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":15,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.641005+0000","last_change":"2024-09-21T03:35:43.897065+0000","last_active":"2024-09-21T03:35:46.641005+0000","last_peered":"2024-09-21T03:35:46.641005+0000","last_clean":"2024-09-21T03:35:46.641005+0000","last_became_active":"2024-09-21T03:35:43.896727+0000","last_became_peered":"2024-09-21T03:35:43.896727+0000","last_unstale":"2024-09-21T03:35:46.641005+0000","last_undegraded":"2024-09-21T03:35:46.641005+0000","last_fullsized":"2024-09-21T03:35:46.641005+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_clean_scrub_stamp":"2024-09-21T03:35:42.877874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:57:57.601786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:45.909895+0000","last_change":"2024-09-21T03:35:44.914440+0000","last_active":"2024-09-21T03:35:45.909895+0000","last_peered":"2024-09-21T03:35:45.909895+0000","last_clean":"2024-09-21T03:35:45.909895+0000","last_became_active":"2024-09-21T03:35:44.914127+0000","last_became_peered":"2024-09-21T03:35:44.914127+0000","last_unstale":"2024-09-21T03:35:45.909895+0000","last_undegraded":"2024-09-21T03:35:45.909895+0000","last_fullsized":"2024-09-21T03:35:45.909895+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:15:54.545107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":15,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:45.908306+0000","last_change":"2024-09-21T03:35:43.910589+0000","last_active":"2024-09-21T03:35:45.908306+0000","last_peered":"2024-09-21T03:35:45.908306+0000","last_clean":"2024-09-21T03:35:45.908306+0000","last_became_active":"2024-09-21T03:35:43.910150+0000","last_became_peered":"2024-09-21T03:35:43.910150+0000","last_unstale":"2024-09-21T03:35:45.908306+0000","last_undegraded":"2024-09-21T03:35:45.908306+0000","last_fullsized":"2024-09-21T03:35:45.908306+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_clean_scrub_stamp":"2024-09-21T03:35:42.877874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:13:47.067340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.870055+0000","last_change":"2024-09-21T03:35:44.916344+0000","last_active":"2024-09-21T03:35:46.870055+0000","last_peered":"2024-09-21T03:35:46.870055+0000","last_clean":"2024-09-21T03:35:46.870055+0000","last_became_active":"2024-09-21T03:35:44.916019+0000","last_became_peered":"2024-09-21T03:35:44.916019+0000","last_unstale":"2024-09-21T03:35:46.870055+0000","last_undegraded":"2024-09-21T03:35:46.870055+0000","last_fullsized":"2024-09-21T03:35:46.870055+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:58:47.408064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.10","version":"54'8","reported_seq":23,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:45.909319+0000","last_change":"2024-09-21T03:35:43.905763+0000","last_active":"2024-09-21T03:35:45.909319+0000","last_peered":"2024-09-21T03:35:45.909319+0000","last_clean":"2024-09-21T03:35:45.909319+0000","last_became_active":"2024-09-21T03:35:43.905453+0000","last_became_peered":"2024-09-21T03:35:43.905453+0000","last_unstale":"2024-09-21T03:35:45.909319+0000","last_undegraded":"2024-09-21T03:35:45.909319+0000","last_fullsized":"2024-09-21T03:35:45.909319+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_clean_scrub_stamp":"2024-09-21T03:35:42.877874+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:38:13.497082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.679983+0000","last_change":"2024-09-21T03:35:44.917548+0000","last_active":"2024-09-21T03:35:46.679983+0000","last_peered":"2024-09-21T03:35:46.679983+0000","last_clean":"2024-09-21T03:35:46.679983+0000","last_became_active":"2024-09-21T03:35:44.916979+0000","last_became_peered":"2024-09-21T03:35:44.916979+0000","last_unstale":"2024-09-21T03:35:46.679983+0000","last_undegraded":"2024-09-21T03:35:46.679983+0000","last_fullsized":"2024-09-21T03:35:46.679983+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:25:12.342791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.f","version":"54'1","reported_seq":16,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:45.910087+0000","last_change":"2024-09-21T03:35:43.900447+0000","last_active":"2024-09-21T03:35:45.910087+0000","last_peered":"2024-09-21T03:35:45.910087+0000","last_clean":"2024-09-21T03:35:45.910087+0000","last_became_active":"2024-09-21T03:35:43.900127+0000","last_became_peered":"2024-09-21T03:35:43.900127+0000","last_unstale":"2024-09-21T03:35:45.910087+0000","last_undegraded":"2024-09-21T03:35:45.910087+0000","last_fullsized":"2024-09-21T03:35:45.910087+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_clean_scrub_stamp":"2024-09-21T03:35:42.877874+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:27:00.832452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.869128+0000","last_change":"2024-09-21T03:35:44.910826+0000","last_active":"2024-09-21T03:35:46.869128+0000","last_peered":"2024-09-21T03:35:46.869128+0000","last_clean":"2024-09-21T03:35:46.869128+0000","last_became_active":"2024-09-21T03:35:44.910528+0000","last_became_peered":"2024-09-21T03:35:44.910528+0000","last_unstale":"2024-09-21T03:35:46.869128+0000","last_undegraded":"2024-09-21T03:35:46.869128+0000","last_fullsized":"2024-09-21T03:35:46.869128+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:00:58.755293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.e","version":"54'2","reported_seq":17,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:45.909861+0000","last_change":"2024-09-21T03:35:43.906381+0000","last_active":"2024-09-21T03:35:45.909861+0000","last_peered":"2024-09-21T03:35:45.909861+0000","last_clean":"2024-09-21T03:35:45.909861+0000","last_became_active":"2024-09-21T03:35:43.905982+0000","last_became_peered":"2024-09-21T03:35:43.905982+0000","last_unstale":"2024-09-21T03:35:45.909861+0000","last_undegraded":"2024-09-21T03:35:45.909861+0000","last_fullsized":"2024-09-21T03:35:45.909861+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_clean_scrub_stamp":"2024-09-21T03:35:42.877874+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:55:19.755787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:45.908356+0000","last_change":"2024-09-21T03:35:44.906200+0000","last_active":"2024-09-21T03:35:45.908356+0000","last_peered":"2024-09-21T03:35:45.908356+0000","last_clean":"2024-09-21T03:35:45.908356+0000","last_became_active":"2024-09-21T03:35:44.905224+0000","last_became_peered":"2024-09-21T03:35:44.905224+0000","last_unstale":"2024-09-21T03:35:45.908356+0000","last_undegraded":"2024-09-21T03:35:45.908356+0000","last_fullsized":"2024-09-21T03:35:45.908356+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:53:52.547424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.d","version":"54'1","reported_seq":16,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.641508+0000","last_change":"2024-09-21T03:35:43.899938+0000","last_active":"2024-09-21T03:35:46.641508+0000","last_peered":"2024-09-21T03:35:46.641508+0000","last_clean":"2024-09-21T03:35:46.641508+0000","last_became_active":"2024-09-21T03:35:43.899617+0000","last_became_peered":"2024-09-21T03:35:43.899617+0000","last_unstale":"2024-09-21T03:35:46.641508+0000","last_undegraded":"2024-09-21T03:35:46.641508+0000","last_fullsized":"2024-09-21T03:35:46.641508+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_clean_scrub_stamp":"2024-09-21T03:35:42.877874+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:54:01.776464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:45.909342+0000","last_change":"2024-09-21T03:35:44.909160+0000","last_active":"2024-09-21T03:35:45.909342+0000","last_peered":"2024-09-21T03:35:45.909342+0000","last_clean":"2024-09-21T03:35:45.909342+0000","last_became_active":"2024-09-21T03:35:44.908321+0000","last_became_peered":"2024-09-21T03:35:44.908321+0000","last_unstale":"2024-09-21T03:35:45.909342+0000","last_undegraded":"2024-09-21T03:35:45.909342+0000","last_fullsized":"2024-09-21T03:35:45.909342+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:43:45.130830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":15,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:45.909250+0000","last_change":"2024-09-21T03:35:43.911542+0000","last_active":"2024-09-21T03:35:45.909250+0000","last_peered":"2024-09-21T03:35:45.909250+0000","last_clean":"2024-09-21T03:35:45.909250+0000","last_became_active":"2024-09-21T03:35:43.911246+0000","last_became_peered":"2024-09-21T03:35:43.911246+0000","last_unstale":"2024-09-21T03:35:45.909250+0000","last_undegraded":"2024-09-21T03:35:45.909250+0000","last_fullsized":"2024-09-21T03:35:45.909250+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_clean_scrub_stamp":"2024-09-21T03:35:42.877874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:34:12.924121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:45.909019+0000","last_change":"2024-09-21T03:35:44.916752+0000","last_active":"2024-09-21T03:35:45.909019+0000","last_peered":"2024-09-21T03:35:45.909019+0000","last_clean":"2024-09-21T03:35:45.909019+0000","last_became_active":"2024-09-21T03:35:44.914940+0000","last_became_peered":"2024-09-21T03:35:44.914940+0000","last_unstale":"2024-09-21T03:35:45.909019+0000","last_undegraded":"2024-09-21T03:35:45.909019+0000","last_fullsized":"2024-09-21T03:35:45.909019+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:11:30.604557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.b","version":"54'2","reported_seq":17,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.869325+0000","last_change":"2024-09-21T03:35:43.912355+0000","last_active":"2024-09-21T03:35:46.869325+0000","last_peered":"2024-09-21T03:35:46.869325+0000","last_clean":"2024-09-21T03:35:46.869325+0000","last_became_active":"2024-09-21T03:35:43.912034+0000","last_became_peered":"2024-09-21T03:35:43.912034+0000","last_unstale":"2024-09-21T03:35:46.869325+0000","last_undegraded":"2024-09-21T03:35:46.869325+0000","last_fullsized":"2024-09-21T03:35:46.869325+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_clean_scrub_stamp":"2024-09-21T03:35:42.877874+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:13:47.067340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.641394+0000","last_change":"2024-09-21T03:35:44.911441+0000","last_active":"2024-09-21T03:35:46.641394+0000","last_peered":"2024-09-21T03:35:46.641394+0000","last_clean":"2024-09-21T03:35:46.641394+0000","last_became_active":"2024-09-21T03:35:44.911116+0000","last_became_peered":"2024-09-21T03:35:44.911116+0000","last_unstale":"2024-09-21T03:35:46.641394+0000","last_undegraded":"2024-09-21T03:35:46.641394+0000","last_fullsized":"2024-09-21T03:35:46.641394+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:05:51.728837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":15,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.868615+0000","last_change":"2024-09-21T03:35:43.909812+0000","last_active":"2024-09-21T03:35:46.868615+0000","last_peered":"2024-09-21T03:35:46.868615+0000","last_clean":"2024-09-21T03:35:46.868615+0000","last_became_active":"2024-09-21T03:35:43.909494+0000","last_became_peered":"2024-09-21T03:35:43.909494+0000","last_unstale":"2024-09-21T03:35:46.868615+0000","last_undegraded":"2024-09-21T03:35:46.868615+0000","last_fullsized":"2024-09-21T03:35:46.868615+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_clean_scrub_stamp":"2024-09-21T03:35:42.877874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:30:08.445515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.869281+0000","last_change":"2024-09-21T03:35:44.906754+0000","last_active":"2024-09-21T03:35:46.869281+0000","last_peered":"2024-09-21T03:35:46.869281+0000","last_clean":"2024-09-21T03:35:46.869281+0000","last_became_active":"2024-09-21T03:35:44.906423+0000","last_became_peered":"2024-09-21T03:35:44.906423+0000","last_unstale":"2024-09-21T03:35:46.869281+0000","last_undegraded":"2024-09-21T03:35:46.869281+0000","last_fullsized":"2024-09-21T03:35:46.869281+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:50:12.528499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":15,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.679811+0000","last_change":"2024-09-21T03:35:43.905895+0000","last_active":"2024-09-21T03:35:46.679811+0000","last_peered":"2024-09-21T03:35:46.679811+0000","last_clean":"2024-09-21T03:35:46.679811+0000","last_became_active":"2024-09-21T03:35:43.905606+0000","last_became_peered":"2024-09-21T03:35:43.905606+0000","last_unstale":"2024-09-21T03:35:46.679811+0000","last_undegraded":"2024-09-21T03:35:46.679811+0000","last_fullsized":"2024-09-21T03:35:46.679811+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_clean_scrub_stamp":"2024-09-21T03:35:42.877874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:50:41.525440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:45.909965+0000","last_change":"2024-09-21T03:35:44.897228+0000","last_active":"2024-09-21T03:35:45.909965+0000","last_peered":"2024-09-21T03:35:45.909965+0000","last_clean":"2024-09-21T03:35:45.909965+0000","last_became_active":"2024-09-21T03:35:44.896910+0000","last_became_peered":"2024-09-21T03:35:44.896910+0000","last_unstale":"2024-09-21T03:35:45.909965+0000","last_undegraded":"2024-09-21T03:35:45.909965+0000","last_fullsized":"2024-09-21T03:35:45.909965+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:27:30.659422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":15,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.679686+0000","last_change":"2024-09-21T03:35:43.899860+0000","last_active":"2024-09-21T03:35:46.679686+0000","last_peered":"2024-09-21T03:35:46.679686+0000","last_clean":"2024-09-21T03:35:46.679686+0000","last_became_active":"2024-09-21T03:35:43.899517+0000","last_became_peered":"2024-09-21T03:35:43.899517+0000","last_unstale":"2024-09-21T03:35:46.679686+0000","last_undegraded":"2024-09-21T03:35:46.679686+0000","last_fullsized":"2024-09-21T03:35:46.679686+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_clean_scrub_stamp":"2024-09-21T03:35:42.877874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:23:04.412064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.640856+0000","last_change":"2024-09-21T03:35:44.907182+0000","last_active":"2024-09-21T03:35:46.640856+0000","last_peered":"2024-09-21T03:35:46.640856+0000","last_clean":"2024-09-21T03:35:46.640856+0000","last_became_active":"2024-09-21T03:35:44.906783+0000","last_became_peered":"2024-09-21T03:35:44.906783+0000","last_unstale":"2024-09-21T03:35:46.640856+0000","last_undegraded":"2024-09-21T03:35:46.640856+0000","last_fullsized":"2024-09-21T03:35:46.640856+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:45:37.118982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.7","version":"54'2","reported_seq":17,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:45.909911+0000","last_change":"2024-09-21T03:35:43.910068+0000","last_active":"2024-09-21T03:35:45.909911+0000","last_peered":"2024-09-21T03:35:45.909911+0000","last_clean":"2024-09-21T03:35:45.909911+0000","last_became_active":"2024-09-21T03:35:43.908871+0000","last_became_peered":"2024-09-21T03:35:43.908871+0000","last_unstale":"2024-09-21T03:35:45.909911+0000","last_undegraded":"2024-09-21T03:35:45.909911+0000","last_fullsized":"2024-09-21T03:35:45.909911+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_clean_scrub_stamp":"2024-09-21T03:35:42.877874+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:10:56.273899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.680044+0000","last_change":"2024-09-21T03:35:44.912419+0000","last_active":"2024-09-21T03:35:46.680044+0000","last_peered":"2024-09-21T03:35:46.680044+0000","last_clean":"2024-09-21T03:35:46.680044+0000","last_became_active":"2024-09-21T03:35:44.912135+0000","last_became_peered":"2024-09-21T03:35:44.912135+0000","last_unstale":"2024-09-21T03:35:46.680044+0000","last_undegraded":"2024-09-21T03:35:46.680044+0000","last_fullsized":"2024-09-21T03:35:46.680044+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:00:55.093451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":15,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:45.910079+0000","last_change":"2024-09-21T03:35:43.904698+0000","last_active":"2024-09-21T03:35:45.910079+0000","last_peered":"2024-09-21T03:35:45.910079+0000","last_clean":"2024-09-21T03:35:45.910079+0000","last_became_active":"2024-09-21T03:35:43.904398+0000","last_became_peered":"2024-09-21T03:35:43.904398+0000","last_unstale":"2024-09-21T03:35:45.910079+0000","last_undegraded":"2024-09-21T03:35:45.910079+0000","last_fullsized":"2024-09-21T03:35:45.910079+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_clean_scrub_stamp":"2024-09-21T03:35:42.877874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:37:48.679668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.868840+0000","last_change":"2024-09-21T03:35:44.917013+0000","last_active":"2024-09-21T03:35:46.868840+0000","last_peered":"2024-09-21T03:35:46.868840+0000","last_clean":"2024-09-21T03:35:46.868840+0000","last_became_active":"2024-09-21T03:35:44.916698+0000","last_became_peered":"2024-09-21T03:35:44.916698+0000","last_unstale":"2024-09-21T03:35:46.868840+0000","last_undegraded":"2024-09-21T03:35:46.868840+0000","last_fullsized":"2024-09-21T03:35:46.868840+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:11:45.208479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":15,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.869306+0000","last_change":"2024-09-21T03:35:43.911358+0000","last_active":"2024-09-21T03:35:46.869306+0000","last_peered":"2024-09-21T03:35:46.869306+0000","last_clean":"2024-09-21T03:35:46.869306+0000","last_became_active":"2024-09-21T03:35:43.911032+0000","last_became_peered":"2024-09-21T03:35:43.911032+0000","last_unstale":"2024-09-21T03:35:46.869306+0000","last_undegraded":"2024-09-21T03:35:46.869306+0000","last_fullsized":"2024-09-21T03:35:46.869306+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_clean_scrub_stamp":"2024-09-21T03:35:42.877874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:06:46.333955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.679989+0000","last_change":"2024-09-21T03:35:44.902610+0000","last_active":"2024-09-21T03:35:46.679989+0000","last_peered":"2024-09-21T03:35:46.679989+0000","last_clean":"2024-09-21T03:35:46.679989+0000","last_became_active":"2024-09-21T03:35:44.902286+0000","last_became_peered":"2024-09-21T03:35:44.902286+0000","last_unstale":"2024-09-21T03:35:46.679989+0000","last_undegraded":"2024-09-21T03:35:46.679989+0000","last_fullsized":"2024-09-21T03:35:46.679989+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:20:06.254030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":15,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.679843+0000","last_change":"2024-09-21T03:35:43.911600+0000","last_active":"2024-09-21T03:35:46.679843+0000","last_peered":"2024-09-21T03:35:46.679843+0000","last_clean":"2024-09-21T03:35:46.679843+0000","last_became_active":"2024-09-21T03:35:43.911278+0000","last_became_peered":"2024-09-21T03:35:43.911278+0000","last_unstale":"2024-09-21T03:35:46.679843+0000","last_undegraded":"2024-09-21T03:35:46.679843+0000","last_fullsized":"2024-09-21T03:35:46.679843+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_clean_scrub_stamp":"2024-09-21T03:35:42.877874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:32:48.236015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:45.908704+0000","last_change":"2024-09-21T03:35:44.907941+0000","last_active":"2024-09-21T03:35:45.908704+0000","last_peered":"2024-09-21T03:35:45.908704+0000","last_clean":"2024-09-21T03:35:45.908704+0000","last_became_active":"2024-09-21T03:35:44.905019+0000","last_became_peered":"2024-09-21T03:35:44.905019+0000","last_unstale":"2024-09-21T03:35:45.908704+0000","last_undegraded":"2024-09-21T03:35:45.908704+0000","last_fullsized":"2024-09-21T03:35:45.908704+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:17:03.401604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":15,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.868734+0000","last_change":"2024-09-21T03:35:43.909531+0000","last_active":"2024-09-21T03:35:46.868734+0000","last_peered":"2024-09-21T03:35:46.868734+0000","last_clean":"2024-09-21T03:35:46.868734+0000","last_became_active":"2024-09-21T03:35:43.909013+0000","last_became_peered":"2024-09-21T03:35:43.909013+0000","last_unstale":"2024-09-21T03:35:46.868734+0000","last_undegraded":"2024-09-21T03:35:46.868734+0000","last_fullsized":"2024-09-21T03:35:46.868734+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_clean_scrub_stamp":"2024-09-21T03:35:42.877874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:17:00.326757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.680299+0000","last_change":"2024-09-21T03:35:44.909262+0000","last_active":"2024-09-21T03:35:46.680299+0000","last_peered":"2024-09-21T03:35:46.680299+0000","last_clean":"2024-09-21T03:35:46.680299+0000","last_became_active":"2024-09-21T03:35:44.908944+0000","last_became_peered":"2024-09-21T03:35:44.908944+0000","last_unstale":"2024-09-21T03:35:46.680299+0000","last_undegraded":"2024-09-21T03:35:46.680299+0000","last_fullsized":"2024-09-21T03:35:46.680299+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:17:43.817629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":15,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:45.908478+0000","last_change":"2024-09-21T03:35:43.912067+0000","last_active":"2024-09-21T03:35:45.908478+0000","last_peered":"2024-09-21T03:35:45.908478+0000","last_clean":"2024-09-21T03:35:45.908478+0000","last_became_active":"2024-09-21T03:35:43.911556+0000","last_became_peered":"2024-09-21T03:35:43.911556+0000","last_unstale":"2024-09-21T03:35:45.908478+0000","last_undegraded":"2024-09-21T03:35:45.908478+0000","last_fullsized":"2024-09-21T03:35:45.908478+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_clean_scrub_stamp":"2024-09-21T03:35:42.877874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:29:39.523294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:45.909917+0000","last_change":"2024-09-21T03:35:44.917317+0000","last_active":"2024-09-21T03:35:45.909917+0000","last_peered":"2024-09-21T03:35:45.909917+0000","last_clean":"2024-09-21T03:35:45.909917+0000","last_became_active":"2024-09-21T03:35:44.917002+0000","last_became_peered":"2024-09-21T03:35:44.917002+0000","last_unstale":"2024-09-21T03:35:45.909917+0000","last_undegraded":"2024-09-21T03:35:45.909917+0000","last_fullsized":"2024-09-21T03:35:45.909917+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:59:17.764484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":15,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.641010+0000","last_change":"2024-09-21T03:35:43.898720+0000","last_active":"2024-09-21T03:35:46.641010+0000","last_peered":"2024-09-21T03:35:46.641010+0000","last_clean":"2024-09-21T03:35:46.641010+0000","last_became_active":"2024-09-21T03:35:43.898067+0000","last_became_peered":"2024-09-21T03:35:43.898067+0000","last_unstale":"2024-09-21T03:35:46.641010+0000","last_undegraded":"2024-09-21T03:35:46.641010+0000","last_fullsized":"2024-09-21T03:35:46.641010+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_clean_scrub_stamp":"2024-09-21T03:35:42.877874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:45:41.645966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.868508+0000","last_change":"2024-09-21T03:35:44.925156+0000","last_active":"2024-09-21T03:35:46.868508+0000","last_peered":"2024-09-21T03:35:46.868508+0000","last_clean":"2024-09-21T03:35:46.868508+0000","last_became_active":"2024-09-21T03:35:44.924831+0000","last_became_peered":"2024-09-21T03:35:44.924831+0000","last_unstale":"2024-09-21T03:35:46.868508+0000","last_undegraded":"2024-09-21T03:35:46.868508+0000","last_fullsized":"2024-09-21T03:35:46.868508+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:43.880613+0000","last_clean_scrub_stamp":"2024-09-21T03:35:43.880613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:38:39.342660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"54'1","reported_seq":16,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.680261+0000","last_change":"2024-09-21T03:35:43.903173+0000","last_active":"2024-09-21T03:35:46.680261+0000","last_peered":"2024-09-21T03:35:46.680261+0000","last_clean":"2024-09-21T03:35:46.680261+0000","last_became_active":"2024-09-21T03:35:43.902852+0000","last_became_peered":"2024-09-21T03:35:43.902852+0000","last_unstale":"2024-09-21T03:35:46.680261+0000","last_undegraded":"2024-09-21T03:35:46.680261+0000","last_fullsized":"2024-09-21T03:35:46.680261+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:35:42.877874+0000","last_clean_scrub_stamp":"2024-09-21T03:35:42.877874+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:55:49.604908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":176,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T03:35:46.641131+0000","last_change":"2024-09-21T03:33:25.319776+0000","last_active":"2024-09-21T03:35:46.641131+0000","last_peered":"2024-09-21T03:35:46.641131+0000","last_clean":"2024-09-21T03:35:46.641131+0000","last_became_active":"2024-09-21T03:33:25.318399+0000","last_became_peered":"2024-09-21T03:33:25.318399+0000","last_unstale":"2024-09-21T03:35:46.641131+0000","last_undegraded":"2024-09-21T03:35:46.641131+0000","last_fullsized":"2024-09-21T03:35:46.641131+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:23.016672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:23.016672+0000","last_clean_scrub_stamp":"2024-09-21T03:33:23.016672+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:32:35.906114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":1,"up_from":16,"seq":68719476769,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32408,"kb_used_data":1216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339432,"statfs":{"total":96636764160,"available":96603578368,"internally_reserved":0,"allocated":1245184,"data_stored":880361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509473,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31832,"kb_used_data":640,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340008,"statfs":{"total":96636764160,"available":96604168192,"internally_reserved":0,"allocated":655360,"data_stored":289979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509472,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31832,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340008,"statfs":{"total":96636764160,"available":96604168192,"internally_reserved":0,"allocated":651264,"data_stored":290013,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509473,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31832,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340008,"statfs":{"total":96636764160,"available":96604168192,"internally_reserved":0,"allocated":651264,"data_stored":289957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509472,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31824,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340016,"statfs":{"total":96636764160,"available":96604176384,"internally_reserved":0,"allocated":651264,"data_stored":290505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542176,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35984,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335856,"statfs":{"total":96636764160,"available":96599916544,"internally_reserved":0,"allocated":651264,"data_stored":290505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574881,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32416,"kb_used_data":1228,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339424,"statfs":{"total":96636764160,"available":96603570176,"internally_reserved":0,"allocated":1257472,"data_stored":881993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542177,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36000,"kb_used_data":652,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335840,"statfs":{"total":96636764160,"available":96599900160,"internally_reserved":0,"allocated":667648,"data_stored":291627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T03:35:53.861 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-21T03:35:53.861 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-21T03:35:54.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:54.023+0000 7fcd5760e640 1 -- 172.21.15.38:0/2928891389 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcd500b2e40 msgr2=0x7fcd500b3250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:54.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:54.023+0000 7fcd5760e640 1 --2- 172.21.15.38:0/2928891389 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcd500b2e40 0x7fcd500b3250 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fcd48005fd0 tx=0x7fcd4802aa90 comp rx=0 tx=0).stop 2024-09-21T03:35:54.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:54.023+0000 7fcd5760e640 1 -- 172.21.15.38:0/2928891389 shutdown_connections 2024-09-21T03:35:54.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:54.023+0000 7fcd5760e640 1 --2- 172.21.15.38:0/2928891389 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcd500b2e40 0x7fcd500b3250 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:54.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:54.023+0000 7fcd5760e640 1 --2- 172.21.15.38:0/2928891389 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcd50162280 0x7fcd50162690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:54.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:54.023+0000 7fcd5760e640 1 -- 172.21.15.38:0/2928891389 >> 172.21.15.38:0/2928891389 conn(0x7fcd500b6b20 msgr2=0x7fcd500d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:54.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:54.023+0000 7fcd5760e640 1 -- 172.21.15.38:0/2928891389 shutdown_connections 2024-09-21T03:35:54.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:54.023+0000 7fcd5760e640 1 -- 172.21.15.38:0/2928891389 wait complete. 2024-09-21T03:35:54.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:54.025+0000 7fcd5760e640 1 Processor -- start 2024-09-21T03:35:54.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:54.025+0000 7fcd5760e640 1 -- start start 2024-09-21T03:35:54.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:54.026+0000 7fcd5760e640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcd500b2e40 0x7fcd500aadc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:54.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:54.026+0000 7fcd5760e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcd50162280 0x7fcd500cd370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:54.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:54.026+0000 7fcd5760e640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcd500ceca0 0x7fcd501f88b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:54.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:54.026+0000 7fcd5760e640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fcd5015e4b0 con 0x7fcd50162280 2024-09-21T03:35:54.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:54.026+0000 7fcd5760e640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fcd500b39a0 con 0x7fcd500ceca0 2024-09-21T03:35:54.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:54.026+0000 7fcd5760e640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fcd5015e630 con 0x7fcd500b2e40 2024-09-21T03:35:54.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:54.026+0000 7fcd54ba9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcd50162280 0x7fcd500cd370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:54.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:54.026+0000 7fcd54ba9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcd50162280 0x7fcd500cd370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44356/0 (socket says 172.21.15.38:44356) 2024-09-21T03:35:54.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:54.027+0000 7fcd54ba9640 1 -- 172.21.15.38:0/2894689536 learned_addr learned my addr 172.21.15.38:0/2894689536 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:35:54.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:54.027+0000 7fcd55bab640 1 --2- 172.21.15.38:0/2894689536 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcd500ceca0 0x7fcd501f88b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:54.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:54.027+0000 7fcd553aa640 1 --2- 172.21.15.38:0/2894689536 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcd500b2e40 0x7fcd500aadc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:54.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:54.027+0000 7fcd54ba9640 1 -- 172.21.15.38:0/2894689536 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcd500b2e40 msgr2=0x7fcd500aadc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:54.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:54.027+0000 7fcd54ba9640 1 --2- 172.21.15.38:0/2894689536 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcd500b2e40 0x7fcd500aadc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:54.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:54.027+0000 7fcd54ba9640 1 -- 172.21.15.38:0/2894689536 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcd500ceca0 msgr2=0x7fcd501f88b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:54.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:54.027+0000 7fcd54ba9640 1 --2- 172.21.15.38:0/2894689536 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcd500ceca0 0x7fcd501f88b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:54.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:54.027+0000 7fcd54ba9640 1 -- 172.21.15.38:0/2894689536 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd500cc2d0 con 0x7fcd50162280 2024-09-21T03:35:54.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:54.027+0000 7fcd54ba9640 1 --2- 172.21.15.38:0/2894689536 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcd50162280 0x7fcd500cd370 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fcd4400d780 tx=0x7fcd4400dc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:54.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:54.027+0000 7fcd4dffb640 1 -- 172.21.15.38:0/2894689536 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd44017070 con 0x7fcd50162280 2024-09-21T03:35:54.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:54.027+0000 7fcd4dffb640 1 -- 172.21.15.38:0/2894689536 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcd44009ab0 con 0x7fcd50162280 2024-09-21T03:35:54.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:54.027+0000 7fcd4dffb640 1 -- 172.21.15.38:0/2894689536 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd44010460 con 0x7fcd50162280 2024-09-21T03:35:54.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:54.027+0000 7fcd5760e640 1 -- 172.21.15.38:0/2894689536 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd500cc5b0 con 0x7fcd50162280 2024-09-21T03:35:54.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:54.028+0000 7fcd5760e640 1 -- 172.21.15.38:0/2894689536 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd500ccb70 con 0x7fcd50162280 2024-09-21T03:35:54.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:54.029+0000 7fcd5760e640 1 -- 172.21.15.38:0/2894689536 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd50162f00 con 0x7fcd50162280 2024-09-21T03:35:54.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:54.029+0000 7fcd4dffb640 1 -- 172.21.15.38:0/2894689536 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fcd4401a070 con 0x7fcd50162280 2024-09-21T03:35:54.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:54.030+0000 7fcd4dffb640 1 --2- 172.21.15.38:0/2894689536 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fcd3c078b90 0x7fcd3c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:54.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:54.030+0000 7fcd4dffb640 1 -- 172.21.15.38:0/2894689536 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 6101+0+0 (secure 0 0 0) 0x7fcd4409af60 con 0x7fcd50162280 2024-09-21T03:35:54.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:54.030+0000 7fcd553aa640 1 --2- 172.21.15.38:0/2894689536 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fcd3c078b90 0x7fcd3c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:54.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:54.031+0000 7fcd553aa640 1 --2- 172.21.15.38:0/2894689536 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fcd3c078b90 0x7fcd3c07b0a0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fcd40001720 tx=0x7fcd40009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:54.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:54.035+0000 7fcd4dffb640 1 -- 172.21.15.38:0/2894689536 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fcd44062bc0 con 0x7fcd50162280 2024-09-21T03:35:54.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:54.297+0000 7fcd5760e640 1 -- 172.21.15.38:0/2894689536 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fcd501629b0 con 0x7fcd50162280 2024-09-21T03:35:54.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:54.301+0000 7fcd4dffb640 1 -- 172.21.15.38:0/2894689536 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v19) ==== 190+0+147 (secure 0 0 0) 0x7fcd440675f0 con 0x7fcd50162280 2024-09-21T03:35:54.303 INFO:teuthology.orchestra.run.smithi038.stdout:[client.0] 2024-09-21T03:35:54.303 INFO:teuthology.orchestra.run.smithi038.stdout: key = AQBQPu5m6ydBDxAAlWZ2XzQTPBcYPriPGRJVeA== 2024-09-21T03:35:54.303 INFO:teuthology.orchestra.run.smithi038.stdout: caps mds = "allow" 2024-09-21T03:35:54.303 INFO:teuthology.orchestra.run.smithi038.stdout: caps mon = "allow r" 2024-09-21T03:35:54.304 INFO:teuthology.orchestra.run.smithi038.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-21T03:35:54.304 INFO:teuthology.orchestra.run.smithi038.stderr:updated caps for client.0 2024-09-21T03:35:54.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:54.304+0000 7fcd5760e640 1 -- 172.21.15.38:0/2894689536 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fcd3c078b90 msgr2=0x7fcd3c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:54.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:54.304+0000 7fcd5760e640 1 --2- 172.21.15.38:0/2894689536 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fcd3c078b90 0x7fcd3c07b0a0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fcd40001720 tx=0x7fcd40009040 comp rx=0 tx=0).stop 2024-09-21T03:35:54.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:54.304+0000 7fcd5760e640 1 -- 172.21.15.38:0/2894689536 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcd50162280 msgr2=0x7fcd500cd370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:54.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:54.304+0000 7fcd5760e640 1 --2- 172.21.15.38:0/2894689536 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcd50162280 0x7fcd500cd370 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fcd4400d780 tx=0x7fcd4400dc40 comp rx=0 tx=0).stop 2024-09-21T03:35:54.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:54.304+0000 7fcd5760e640 1 -- 172.21.15.38:0/2894689536 shutdown_connections 2024-09-21T03:35:54.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:54.304+0000 7fcd5760e640 1 --2- 172.21.15.38:0/2894689536 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcd500ceca0 0x7fcd501f88b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:54.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:54.304+0000 7fcd5760e640 1 --2- 172.21.15.38:0/2894689536 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcd50162280 0x7fcd500cd370 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:54.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:54.304+0000 7fcd5760e640 1 --2- 172.21.15.38:0/2894689536 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fcd3c078b90 0x7fcd3c07b0a0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:54.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:54.304+0000 7fcd5760e640 1 --2- 172.21.15.38:0/2894689536 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcd500b2e40 0x7fcd500aadc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:54.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:54.304+0000 7fcd5760e640 1 -- 172.21.15.38:0/2894689536 >> 172.21.15.38:0/2894689536 conn(0x7fcd500b6b20 msgr2=0x7fcd5015da10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:54.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:54.304+0000 7fcd5760e640 1 -- 172.21.15.38:0/2894689536 shutdown_connections 2024-09-21T03:35:54.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:54.304+0000 7fcd5760e640 1 -- 172.21.15.38:0/2894689536 wait complete. 2024-09-21T03:35:54.318 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:35:54.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:54.432+0000 7f0ad4dad640 1 -- 172.21.15.38:0/3343493107 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0ad00b2e40 msgr2=0x7f0ad00b3250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:54.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:54.432+0000 7f0ad4dad640 1 --2- 172.21.15.38:0/3343493107 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0ad00b2e40 0x7f0ad00b3250 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f0ad00b7460 tx=0x7f0ac002aa30 comp rx=0 tx=0).stop 2024-09-21T03:35:54.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:54.432+0000 7f0ad4dad640 1 -- 172.21.15.38:0/3343493107 shutdown_connections 2024-09-21T03:35:54.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:54.432+0000 7f0ad4dad640 1 --2- 172.21.15.38:0/3343493107 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0ad00b2e40 0x7f0ad00b3250 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:54.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:54.432+0000 7f0ad4dad640 1 --2- 172.21.15.38:0/3343493107 >> v2:172.21.15.195:3300/0 conn(0x7f0ad0162280 0x7f0ad0162690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:54.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:54.432+0000 7f0ad4dad640 1 -- 172.21.15.38:0/3343493107 >> 172.21.15.38:0/3343493107 conn(0x7f0ad00b6b20 msgr2=0x7f0ad00d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:54.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:54.432+0000 7f0ad4dad640 1 -- 172.21.15.38:0/3343493107 shutdown_connections 2024-09-21T03:35:54.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:54.432+0000 7f0ad4dad640 1 -- 172.21.15.38:0/3343493107 wait complete. 2024-09-21T03:35:54.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:54.434+0000 7f0ad4dad640 1 Processor -- start 2024-09-21T03:35:54.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:54.434+0000 7f0ad4dad640 1 -- start start 2024-09-21T03:35:54.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:54.436+0000 7f0ad4dad640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0ad00b2e40 0x7f0ad00b27a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:54.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:54.436+0000 7f0ad4dad640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0ad0162280 0x7f0ad00ced50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:54.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:54.436+0000 7f0ad4dad640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0ad014d6c0 0x7f0ad00abe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:54.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:54.436+0000 7f0ad4dad640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f0ad015e4b0 con 0x7f0ad0162280 2024-09-21T03:35:54.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:54.436+0000 7f0ad4dad640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f0ad00b39a0 con 0x7f0ad014d6c0 2024-09-21T03:35:54.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:54.436+0000 7f0ad4dad640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f0ad015e630 con 0x7f0ad00b2e40 2024-09-21T03:35:54.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:54.436+0000 7f0acdd9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0ad0162280 0x7f0ad00ced50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:54.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:54.436+0000 7f0acdd9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0ad0162280 0x7f0ad00ced50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44374/0 (socket says 172.21.15.38:44374) 2024-09-21T03:35:54.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:54.436+0000 7f0acdd9b640 1 -- 172.21.15.38:0/2844811522 learned_addr learned my addr 172.21.15.38:0/2844811522 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:35:54.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:54.436+0000 7f0aced9d640 1 --2- 172.21.15.38:0/2844811522 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0ad014d6c0 0x7f0ad00abe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:54.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:54.436+0000 7f0ace59c640 1 --2- 172.21.15.38:0/2844811522 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0ad00b2e40 0x7f0ad00b27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:54.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:54.436+0000 7f0acdd9b640 1 -- 172.21.15.38:0/2844811522 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0ad00b2e40 msgr2=0x7f0ad00b27a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:54.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:54.436+0000 7f0acdd9b640 1 --2- 172.21.15.38:0/2844811522 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0ad00b2e40 0x7f0ad00b27a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:54.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:54.436+0000 7f0acdd9b640 1 -- 172.21.15.38:0/2844811522 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0ad014d6c0 msgr2=0x7f0ad00abe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:54.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:54.436+0000 7f0acdd9b640 1 --2- 172.21.15.38:0/2844811522 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0ad014d6c0 0x7f0ad00abe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:54.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:54.436+0000 7f0acdd9b640 1 -- 172.21.15.38:0/2844811522 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ad01f52b0 con 0x7f0ad0162280 2024-09-21T03:35:54.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:54.436+0000 7f0acdd9b640 1 --2- 172.21.15.38:0/2844811522 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0ad0162280 0x7f0ad00ced50 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f0ab800e7f0 tx=0x7f0ab800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:54.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:54.436+0000 7f0ac6ffd640 1 -- 172.21.15.38:0/2844811522 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ab800ccc0 con 0x7f0ad0162280 2024-09-21T03:35:54.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:54.437+0000 7f0ac6ffd640 1 -- 172.21.15.38:0/2844811522 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0ab800ce80 con 0x7f0ad0162280 2024-09-21T03:35:54.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:54.437+0000 7f0ad4dad640 1 -- 172.21.15.38:0/2844811522 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ad014e2a0 con 0x7f0ad0162280 2024-09-21T03:35:54.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:54.437+0000 7f0ac6ffd640 1 -- 172.21.15.38:0/2844811522 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ab8005280 con 0x7f0ad0162280 2024-09-21T03:35:54.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:54.437+0000 7f0ad4dad640 1 -- 172.21.15.38:0/2844811522 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ad00b1fa0 con 0x7f0ad0162280 2024-09-21T03:35:54.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:54.438+0000 7f0ad4dad640 1 -- 172.21.15.38:0/2844811522 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ad014e010 con 0x7f0ad0162280 2024-09-21T03:35:54.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:54.438+0000 7f0ac6ffd640 1 -- 172.21.15.38:0/2844811522 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f0ab8005420 con 0x7f0ad0162280 2024-09-21T03:35:54.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:54.439+0000 7f0ac6ffd640 1 --2- 172.21.15.38:0/2844811522 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0ab0078b90 0x7f0ab007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:54.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:54.439+0000 7f0ac6ffd640 1 -- 172.21.15.38:0/2844811522 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 6101+0+0 (secure 0 0 0) 0x7f0ab809b730 con 0x7f0ad0162280 2024-09-21T03:35:54.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:54.439+0000 7f0ace59c640 1 --2- 172.21.15.38:0/2844811522 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0ab0078b90 0x7f0ab007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:54.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:54.440+0000 7f0ace59c640 1 --2- 172.21.15.38:0/2844811522 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0ab0078b90 0x7f0ab007b0a0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f0ac8001720 tx=0x7f0ac8009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:54.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:54.444+0000 7f0ac6ffd640 1 -- 172.21.15.38:0/2844811522 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f0ab80633a0 con 0x7f0ad0162280 2024-09-21T03:35:54.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:54.668+0000 7f0ad4dad640 1 -- 172.21.15.38:0/2844811522 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0ad0162f00 con 0x7f0ad0162280 2024-09-21T03:35:54.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:54.668+0000 7f0ac6ffd640 1 -- 172.21.15.38:0/2844811522 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+5229 (secure 0 0 0) 0x7f0ab8067dd0 con 0x7f0ad0162280 2024-09-21T03:35:54.670 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:35:54.671 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":30,"btime":"2024-09-21T03:35:47:913013+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5147,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/2858412699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":2858412699},{"type":"v1","addr":"172.21.15.195:6837","nonce":2858412699}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":5368,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/2907144575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":2907144575},{"type":"v1","addr":"172.21.15.195:6836","nonce":2907144575}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":5574,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6834/1911661184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":1911661184},{"type":"v1","addr":"172.21.15.38:6834","nonce":1911661184}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:35:44.271168+0000","modified":"2024-09-21T03:35:47.294703+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5577},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5577":{"gid":5577,"name":"c","rank":0,"incarnation":27,"state":"up:active","state_seq":3,"addr":"172.21.15.38:6835/3184989541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6833","nonce":3184989541},{"type":"v1","addr":"172.21.15.38:6835","nonce":3184989541}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5577,"qdb_cluster":[5577]},"id":3}]} 2024-09-21T03:35:54.671 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 30 2024-09-21T03:35:54.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:54.670+0000 7f0ad4dad640 1 -- 172.21.15.38:0/2844811522 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0ab0078b90 msgr2=0x7f0ab007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:54.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:54.670+0000 7f0ad4dad640 1 --2- 172.21.15.38:0/2844811522 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0ab0078b90 0x7f0ab007b0a0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f0ac8001720 tx=0x7f0ac8009040 comp rx=0 tx=0).stop 2024-09-21T03:35:54.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:54.670+0000 7f0ad4dad640 1 -- 172.21.15.38:0/2844811522 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0ad0162280 msgr2=0x7f0ad00ced50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:54.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:54.670+0000 7f0ad4dad640 1 --2- 172.21.15.38:0/2844811522 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0ad0162280 0x7f0ad00ced50 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f0ab800e7f0 tx=0x7f0ab800ecb0 comp rx=0 tx=0).stop 2024-09-21T03:35:54.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:54.670+0000 7f0ad4dad640 1 -- 172.21.15.38:0/2844811522 shutdown_connections 2024-09-21T03:35:54.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:54.670+0000 7f0ad4dad640 1 --2- 172.21.15.38:0/2844811522 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0ad014d6c0 0x7f0ad00abe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:54.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:54.670+0000 7f0ad4dad640 1 --2- 172.21.15.38:0/2844811522 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0ad0162280 0x7f0ad00ced50 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:54.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:54.670+0000 7f0ad4dad640 1 --2- 172.21.15.38:0/2844811522 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0ab0078b90 0x7f0ab007b0a0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:54.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:54.670+0000 7f0ad4dad640 1 --2- 172.21.15.38:0/2844811522 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0ad00b2e40 0x7f0ad00b27a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:54.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:54.670+0000 7f0ad4dad640 1 -- 172.21.15.38:0/2844811522 >> 172.21.15.38:0/2844811522 conn(0x7f0ad00b6b20 msgr2=0x7f0ad006cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:54.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:54.671+0000 7f0ad4dad640 1 -- 172.21.15.38:0/2844811522 shutdown_connections 2024-09-21T03:35:54.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:54.671+0000 7f0ad4dad640 1 -- 172.21.15.38:0/2844811522 wait complete. 2024-09-21T03:35:55.086 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 30, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T03:35:44.271168+0000', 'modified': '2024-09-21T03:35:47.294703+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5577}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5577': {'gid': 5577, 'name': 'c', 'rank': 0, 'incarnation': 27, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.38:6835/3184989541', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.38:6833', 'nonce': 3184989541}, {'type': 'v1', 'addr': '172.21.15.38:6835', 'nonce': 3184989541}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [8], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5577, 'qdb_cluster': [5577]} 2024-09-21T03:35:55.086 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-21T03:35:55.087 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-21T03:35:55.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.212+0000 7f7b1d7a1640 1 -- 172.21.15.38:0/754959258 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7b180d1120 msgr2=0x7f7b180ceb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:55.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.212+0000 7f7b1d7a1640 1 --2- 172.21.15.38:0/754959258 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7b180d1120 0x7f7b180ceb90 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f7b180d2360 tx=0x7f7b1002a9a0 comp rx=0 tx=0).stop 2024-09-21T03:35:55.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.212+0000 7f7b1d7a1640 1 -- 172.21.15.38:0/754959258 shutdown_connections 2024-09-21T03:35:55.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.212+0000 7f7b1d7a1640 1 --2- 172.21.15.38:0/754959258 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7b180d1120 0x7f7b180ceb90 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:55.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.212+0000 7f7b1d7a1640 1 --2- 172.21.15.38:0/754959258 >> v2:172.21.15.195:3300/0 conn(0x7f7b180d0170 0x7f7b180d0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:55.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.212+0000 7f7b1d7a1640 1 -- 172.21.15.38:0/754959258 >> 172.21.15.38:0/754959258 conn(0x7f7b1809d830 msgr2=0x7f7b180b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:55.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.212+0000 7f7b1d7a1640 1 -- 172.21.15.38:0/754959258 shutdown_connections 2024-09-21T03:35:55.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.212+0000 7f7b1d7a1640 1 -- 172.21.15.38:0/754959258 wait complete. 2024-09-21T03:35:55.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.214+0000 7f7b1d7a1640 1 Processor -- start 2024-09-21T03:35:55.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.214+0000 7f7b1d7a1640 1 -- start start 2024-09-21T03:35:55.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.215+0000 7f7b1d7a1640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7b180d0170 0x7f7b1806ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:55.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.215+0000 7f7b1d7a1640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7b180d1120 0x7f7b181575a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:55.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.215+0000 7f7b1d7a1640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7b1815ee80 0x7f7b180b2620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:55.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.215+0000 7f7b1d7a1640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f7b180b6700 con 0x7f7b180d1120 2024-09-21T03:35:55.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.215+0000 7f7b1d7a1640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f7b180b6580 con 0x7f7b1815ee80 2024-09-21T03:35:55.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.215+0000 7f7b1d7a1640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f7b180b6880 con 0x7f7b180d0170 2024-09-21T03:35:55.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.215+0000 7f7b167fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7b180d1120 0x7f7b181575a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:55.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.215+0000 7f7b167fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7b180d1120 0x7f7b181575a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44386/0 (socket says 172.21.15.38:44386) 2024-09-21T03:35:55.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.215+0000 7f7b167fc640 1 -- 172.21.15.38:0/1780607532 learned_addr learned my addr 172.21.15.38:0/1780607532 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:35:55.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.216+0000 7f7b177fe640 1 --2- 172.21.15.38:0/1780607532 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7b1815ee80 0x7f7b180b2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:55.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.216+0000 7f7b16ffd640 1 --2- 172.21.15.38:0/1780607532 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7b180d0170 0x7f7b1806ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:55.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.216+0000 7f7b167fc640 1 -- 172.21.15.38:0/1780607532 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7b180d0170 msgr2=0x7f7b1806ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:55.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.216+0000 7f7b167fc640 1 --2- 172.21.15.38:0/1780607532 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7b180d0170 0x7f7b1806ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:55.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.216+0000 7f7b167fc640 1 -- 172.21.15.38:0/1780607532 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7b1815ee80 msgr2=0x7f7b180b2620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:55.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.216+0000 7f7b167fc640 1 --2- 172.21.15.38:0/1780607532 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7b1815ee80 0x7f7b180b2620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:55.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.216+0000 7f7b167fc640 1 -- 172.21.15.38:0/1780607532 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b1806f310 con 0x7f7b180d1120 2024-09-21T03:35:55.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.216+0000 7f7b167fc640 1 --2- 172.21.15.38:0/1780607532 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7b180d1120 0x7f7b181575a0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f7b0c00d720 tx=0x7f7b0c00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:55.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.216+0000 7f7aff7fe640 1 -- 172.21.15.38:0/1780607532 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b0c017070 con 0x7f7b180d1120 2024-09-21T03:35:55.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.216+0000 7f7aff7fe640 1 -- 172.21.15.38:0/1780607532 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7b0c004310 con 0x7f7b180d1120 2024-09-21T03:35:55.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.216+0000 7f7b1d7a1640 1 -- 172.21.15.38:0/1780607532 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b181f5ce0 con 0x7f7b180d1120 2024-09-21T03:35:55.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.216+0000 7f7aff7fe640 1 -- 172.21.15.38:0/1780607532 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b0c009a20 con 0x7f7b180d1120 2024-09-21T03:35:55.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.216+0000 7f7b1d7a1640 1 -- 172.21.15.38:0/1780607532 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b181f62a0 con 0x7f7b180d1120 2024-09-21T03:35:55.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.219+0000 7f7aff7fe640 1 -- 172.21.15.38:0/1780607532 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f7b0c009bc0 con 0x7f7b180d1120 2024-09-21T03:35:55.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.219+0000 7f7aff7fe640 1 --2- 172.21.15.38:0/1780607532 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7b04078b90 0x7f7b0407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:55.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.219+0000 7f7aff7fe640 1 -- 172.21.15.38:0/1780607532 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 6101+0+0 (secure 0 0 0) 0x7f7b0c09af60 con 0x7f7b180d1120 2024-09-21T03:35:55.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.220+0000 7f7b16ffd640 1 --2- 172.21.15.38:0/1780607532 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7b04078b90 0x7f7b0407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:55.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.220+0000 7f7b247e7740 1 Processor -- start 2024-09-21T03:35:55.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.220+0000 7f7b247e7740 1 -- start start 2024-09-21T03:35:55.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.220+0000 7f7b16ffd640 1 --2- 172.21.15.38:0/1780607532 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7b04078b90 0x7f7b0407b0a0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f7b080015e0 tx=0x7f7b08009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:55.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.221+0000 7f7b247e7740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e5f12e9270 0x55e5f16bafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:55.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.221+0000 7f7b247e7740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e5f12c03b0 0x55e5f16bb4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:55.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.221+0000 7f7b247e7740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e5f12c8590 0x55e5f16d83e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:55.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.221+0000 7f7b247e7740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55e5f1603820 con 0x55e5f12c03b0 2024-09-21T03:35:55.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.221+0000 7f7b247e7740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55e5f15f2c50 con 0x55e5f12e9270 2024-09-21T03:35:55.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.221+0000 7f7b247e7740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55e5f15f6a60 con 0x55e5f12c8590 2024-09-21T03:35:55.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.221+0000 7f7b167fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e5f12c03b0 0x55e5f16bb4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:55.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.222+0000 7f7b167fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e5f12c03b0 0x55e5f16bb4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44402/0 (socket says 172.21.15.38:44402) 2024-09-21T03:35:55.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.222+0000 7f7b167fc640 1 -- 172.21.15.38:0/824660857 learned_addr learned my addr 172.21.15.38:0/824660857 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:35:55.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.222+0000 7f7b177fe640 1 --2- 172.21.15.38:0/824660857 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e5f12c8590 0x55e5f16d83e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:55.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.222+0000 7f7b167fc640 1 -- 172.21.15.38:0/824660857 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e5f12c8590 msgr2=0x55e5f16d83e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:35:55.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.222+0000 7f7b16ffd640 1 --2- 172.21.15.38:0/824660857 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e5f12e9270 0x55e5f16bafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:55.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.222+0000 7f7b167fc640 1 --2- 172.21.15.38:0/824660857 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e5f12c8590 0x55e5f16d83e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:55.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.222+0000 7f7b167fc640 1 -- 172.21.15.38:0/824660857 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e5f12e9270 msgr2=0x55e5f16bafa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:35:55.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.222+0000 7f7b167fc640 1 --2- 172.21.15.38:0/824660857 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e5f12e9270 0x55e5f16bafa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:55.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.222+0000 7f7b167fc640 1 -- 172.21.15.38:0/824660857 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5f1354850 con 0x55e5f12c03b0 2024-09-21T03:35:55.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.222+0000 7f7b167fc640 1 --2- 172.21.15.38:0/824660857 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e5f12c03b0 0x55e5f16bb4e0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f7b0c0628a0 tx=0x7f7b0c0625b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=369a7594d2d1f761 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:55.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.222+0000 7f7afcff9640 1 -- 172.21.15.38:0/824660857 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b0c020070 con 0x55e5f12c03b0 2024-09-21T03:35:55.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.222+0000 7f7afcff9640 1 -- 172.21.15.38:0/824660857 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7b0c067550 con 0x55e5f12c03b0 2024-09-21T03:35:55.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.222+0000 7f7afcff9640 1 -- 172.21.15.38:0/824660857 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b0c067340 con 0x55e5f12c03b0 2024-09-21T03:35:55.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.223+0000 7f7b247e7740 1 -- 172.21.15.38:0/824660857 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e5f12c03b0 msgr2=0x55e5f16bb4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:55.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.223+0000 7f7b247e7740 1 --2- 172.21.15.38:0/824660857 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e5f12c03b0 0x55e5f16bb4e0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f7b0c0628a0 tx=0x7f7b0c0625b0 comp rx=0 tx=0).stop 2024-09-21T03:35:55.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.223+0000 7f7b247e7740 1 -- 172.21.15.38:0/824660857 shutdown_connections 2024-09-21T03:35:55.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.223+0000 7f7b247e7740 1 --2- 172.21.15.38:0/824660857 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e5f12c8590 0x55e5f16d83e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:55.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.223+0000 7f7b247e7740 1 --2- 172.21.15.38:0/824660857 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e5f12c03b0 0x55e5f16bb4e0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:55.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.223+0000 7f7b247e7740 1 --2- 172.21.15.38:0/824660857 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e5f12e9270 0x55e5f16bafa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:55.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.223+0000 7f7b247e7740 1 -- 172.21.15.38:0/824660857 >> 172.21.15.38:0/824660857 conn(0x55e5f12dd3c0 msgr2=0x55e5f1691260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:55.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.223+0000 7f7b247e7740 1 -- 172.21.15.38:0/824660857 shutdown_connections 2024-09-21T03:35:55.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.223+0000 7f7b247e7740 1 -- 172.21.15.38:0/824660857 wait complete. 2024-09-21T03:35:55.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.224+0000 7f7b247e7740 1 -- start start 2024-09-21T03:35:55.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.224+0000 7f7b247e7740 1 Processor -- start 2024-09-21T03:35:55.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.225+0000 7f7b247e7740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e5f12c03b0 0x55e5f16cfc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:55.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.225+0000 7f7b247e7740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e5f12c8590 0x55e5f16b4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:55.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.225+0000 7f7b247e7740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e5f12e9270 0x55e5f16e88f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:55.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.225+0000 7f7b247e7740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55e5f1363620 con 0x55e5f12e9270 2024-09-21T03:35:55.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.225+0000 7f7b247e7740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55e5f15985f0 con 0x55e5f12c03b0 2024-09-21T03:35:55.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.225+0000 7f7b247e7740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55e5f16bba20 con 0x55e5f12c8590 2024-09-21T03:35:55.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.226+0000 7f7b247e7740 10 client.0 _ll_register_callbacks cb 0x55e5f159c0a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:35:55.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.226+0000 7f7b177fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e5f12e9270 0x55e5f16e88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:55.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.226+0000 7f7b177fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e5f12e9270 0x55e5f16e88f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44416/0 (socket says 172.21.15.38:44416) 2024-09-21T03:35:55.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.226+0000 7f7b177fe640 1 -- 172.21.15.38:0/2145473614 learned_addr learned my addr 172.21.15.38:0/2145473614 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:35:55.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.226+0000 7f7b16ffd640 1 --2- 172.21.15.38:0/2145473614 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e5f12c03b0 0x55e5f16cfc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:55.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.226+0000 7f7b167fc640 1 --2- 172.21.15.38:0/2145473614 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e5f12c8590 0x55e5f16b4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:55.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.226+0000 7f7b177fe640 1 -- 172.21.15.38:0/2145473614 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e5f12c8590 msgr2=0x55e5f16b4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:35:55.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.226+0000 7f7b177fe640 1 --2- 172.21.15.38:0/2145473614 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e5f12c8590 0x55e5f16b4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:55.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.226+0000 7f7b177fe640 1 -- 172.21.15.38:0/2145473614 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e5f12c03b0 msgr2=0x55e5f16cfc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:35:55.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.226+0000 7f7b177fe640 1 --2- 172.21.15.38:0/2145473614 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e5f12c03b0 0x55e5f16cfc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:55.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.226+0000 7f7b177fe640 1 -- 172.21.15.38:0/2145473614 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5f13cb6b0 con 0x55e5f12e9270 2024-09-21T03:35:55.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.226+0000 7f7b16ffd640 1 --2- 172.21.15.38:0/2145473614 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e5f12c03b0 0x55e5f16cfc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:35:55.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.226+0000 7f7b177fe640 1 --2- 172.21.15.38:0/2145473614 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e5f12e9270 0x55e5f16e88f0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f7b10005fd0 tx=0x7f7b10003eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1f6d5ee40d5a62e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:55.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.226+0000 7f7af5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-21T03:35:55.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.226+0000 7f7af5ffb640 1 -- 172.21.15.38:0/2145473614 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b10033070 con 0x55e5f12e9270 2024-09-21T03:35:55.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.226+0000 7f7af5ffb640 1 -- 172.21.15.38:0/2145473614 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(55..55 src has 1..55) ==== 6101+0+0 (secure 0 0 0) 0x7f7b1002fd00 con 0x55e5f12e9270 2024-09-21T03:35:55.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.226+0000 7f7b247e7740 1 -- 172.21.15.38:0/2145473614 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b0c067550 con 0x55e5f12e9270 2024-09-21T03:35:55.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.226+0000 7f7af5ffb640 1 -- 172.21.15.38:0/2145473614 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7b10009360 con 0x55e5f12e9270 2024-09-21T03:35:55.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.226+0000 7f7b247e7740 10 client.5718 fetch_fsmap learned FSMap version 30 2024-09-21T03:35:55.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.226+0000 7f7b247e7740 1 -- 172.21.15.38:0/2145473614 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=2+}) -- 0x55e5f16ce260 con 0x55e5f12e9270 2024-09-21T03:35:55.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.227+0000 7f7af5ffb640 1 -- 172.21.15.38:0/2145473614 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7b0c067550 con 0x55e5f12e9270 2024-09-21T03:35:55.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.227+0000 7f7af5ffb640 1 -- 172.21.15.38:0/2145473614 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 30) ==== 3502+0+0 (secure 0 0 0) 0x7f7b1004b050 con 0x55e5f12e9270 2024-09-21T03:35:55.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.227+0000 7f7b247e7740 10 client.5718 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:35:55.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.227+0000 7f7b247e7740 10 client.5718 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T03:35:55.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.227+0000 7f7b247e7740 20 client.5718 populate_metadata read hostname 'smithi038' 2024-09-21T03:35:55.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.227+0000 7f7b247e7740 1 --2- 172.21.15.38:0/2145473614 >> [v2:172.21.15.38:6833/3184989541,v1:172.21.15.38:6835/3184989541] conn(0x55e5f12a7e40 0x55e5f16e5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:55.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.227+0000 7f7b247e7740 4 client.5718 mds_command: new command op to 5577 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:35:55.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.227+0000 7f7b247e7740 1 -- 172.21.15.38:0/2145473614 --> [v2:172.21.15.38:6833/3184989541,v1:172.21.15.38:6835/3184989541] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e5f1605770 con 0x55e5f12a7e40 2024-09-21T03:35:55.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.227+0000 7f7b16ffd640 1 --2- 172.21.15.38:0/2145473614 >> [v2:172.21.15.38:6833/3184989541,v1:172.21.15.38:6835/3184989541] conn(0x55e5f12a7e40 0x55e5f16e5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:55.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.227+0000 7f7af5ffb640 10 client.5718 ms_handle_connect on v2:172.21.15.38:6833/3184989541 2024-09-21T03:35:55.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.227+0000 7f7b16ffd640 1 --2- 172.21.15.38:0/2145473614 >> [v2:172.21.15.38:6833/3184989541,v1:172.21.15.38:6835/3184989541] conn(0x55e5f12a7e40 0x55e5f16e5330 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5acfc85a93979d68 server_cookie=3700f9c9e61b4e1c in_seq=0 out_seq=0 2024-09-21T03:35:55.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.228+0000 7f7af5ffb640 1 -- 172.21.15.38:0/2145473614 <== mds.0 v2:172.21.15.38:6833/3184989541 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e5f1605770 con 0x55e5f12a7e40 2024-09-21T03:35:55.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.228+0000 7f7af5ffb640 10 client.5718 handle_command_reply: tid=0 2024-09-21T03:35:55.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.228+0000 7f7b247e7740 1 client.5718 shutdown 2024-09-21T03:35:55.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.228+0000 7f7b247e7740 1 -- 172.21.15.38:0/2145473614 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e5f12e9270 msgr2=0x55e5f16e88f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:55.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.228+0000 7f7b247e7740 1 --2- 172.21.15.38:0/2145473614 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e5f12e9270 0x55e5f16e88f0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f7b10005fd0 tx=0x7f7b10003eb0 comp rx=0 tx=0).stop 2024-09-21T03:35:55.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.228+0000 7f7b247e7740 1 -- 172.21.15.38:0/2145473614 shutdown_connections 2024-09-21T03:35:55.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.228+0000 7f7b247e7740 1 --2- 172.21.15.38:0/2145473614 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e5f12e9270 0x55e5f16e88f0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:55.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.228+0000 7f7b247e7740 1 --2- 172.21.15.38:0/2145473614 >> [v2:172.21.15.38:6833/3184989541,v1:172.21.15.38:6835/3184989541] conn(0x55e5f12a7e40 0x55e5f16e5330 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:55.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.228+0000 7f7b247e7740 1 --2- 172.21.15.38:0/2145473614 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e5f12c8590 0x55e5f16b4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:55.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.228+0000 7f7b247e7740 1 --2- 172.21.15.38:0/2145473614 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e5f12c03b0 0x55e5f16cfc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:55.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.228+0000 7f7af5ffb640 0 client.5718 ms_handle_reset on v2:172.21.15.38:6833/3184989541 2024-09-21T03:35:55.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.228+0000 7f7b247e7740 1 -- 172.21.15.38:0/2145473614 >> 172.21.15.38:0/2145473614 conn(0x55e5f12dd3c0 msgr2=0x55e5f16d65e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:55.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.228+0000 7f7b247e7740 1 -- 172.21.15.38:0/2145473614 shutdown_connections 2024-09-21T03:35:55.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.228+0000 7f7b247e7740 1 -- 172.21.15.38:0/2145473614 wait complete. 2024-09-21T03:35:55.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.229+0000 7f7b247e7740 20 client.5718 trim_cache size 0 max 16384 2024-09-21T03:35:55.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.248+0000 7f7b247e7740 1 Processor -- start 2024-09-21T03:35:55.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.248+0000 7f7b247e7740 1 -- start start 2024-09-21T03:35:55.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.249+0000 7f7b247e7740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e5f12c03b0 0x55e5f1690000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:55.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.249+0000 7f7b247e7740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e5f12c8590 0x55e5f16cf3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:55.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.249+0000 7f7b247e7740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e5f12e9270 0x55e5f16cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:55.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.249+0000 7f7b247e7740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f7b182016b0 con 0x55e5f12c03b0 2024-09-21T03:35:55.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.249+0000 7f7b247e7740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f7b181f9e00 con 0x55e5f12e9270 2024-09-21T03:35:55.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.249+0000 7f7b247e7740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f7b1806d8a0 con 0x55e5f12c8590 2024-09-21T03:35:55.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.249+0000 7f7b16ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e5f12c03b0 0x55e5f1690000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:55.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.249+0000 7f7b16ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e5f12c03b0 0x55e5f1690000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44418/0 (socket says 172.21.15.38:44418) 2024-09-21T03:35:55.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.249+0000 7f7b16ffd640 1 -- 172.21.15.38:0/3030191979 learned_addr learned my addr 172.21.15.38:0/3030191979 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:35:55.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.249+0000 7f7b16ffd640 1 -- 172.21.15.38:0/3030191979 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e5f12c8590 msgr2=0x55e5f16cf3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:35:55.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.249+0000 7f7b177fe640 1 --2- 172.21.15.38:0/3030191979 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e5f12e9270 0x55e5f16cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:55.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.249+0000 7f7b16ffd640 1 --2- 172.21.15.38:0/3030191979 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e5f12c8590 0x55e5f16cf3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:55.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.249+0000 7f7b16ffd640 1 -- 172.21.15.38:0/3030191979 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e5f12e9270 msgr2=0x55e5f16cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:35:55.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.249+0000 7f7b16ffd640 1 --2- 172.21.15.38:0/3030191979 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e5f12e9270 0x55e5f16cf930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:55.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.249+0000 7f7b16ffd640 1 -- 172.21.15.38:0/3030191979 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5f1692bc0 con 0x55e5f12c03b0 2024-09-21T03:35:55.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.249+0000 7f7b16ffd640 1 --2- 172.21.15.38:0/3030191979 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e5f12c03b0 0x55e5f1690000 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f7b08001e70 tx=0x7f7b08012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7009e0d570e9f97 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:55.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.250+0000 7f7af7fff640 1 -- 172.21.15.38:0/3030191979 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b0800e4b0 con 0x55e5f12c03b0 2024-09-21T03:35:55.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.250+0000 7f7af7fff640 1 -- 172.21.15.38:0/3030191979 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7b0800e650 con 0x55e5f12c03b0 2024-09-21T03:35:55.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.250+0000 7f7af7fff640 1 -- 172.21.15.38:0/3030191979 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b0800e860 con 0x55e5f12c03b0 2024-09-21T03:35:55.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.250+0000 7f7b247e7740 1 -- 172.21.15.38:0/3030191979 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e5f12c03b0 msgr2=0x55e5f1690000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:55.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.250+0000 7f7b247e7740 1 --2- 172.21.15.38:0/3030191979 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e5f12c03b0 0x55e5f1690000 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f7b08001e70 tx=0x7f7b08012000 comp rx=0 tx=0).stop 2024-09-21T03:35:55.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.250+0000 7f7b247e7740 1 -- 172.21.15.38:0/3030191979 shutdown_connections 2024-09-21T03:35:55.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.250+0000 7f7b247e7740 1 --2- 172.21.15.38:0/3030191979 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e5f12e9270 0x55e5f16cf930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:55.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.250+0000 7f7b247e7740 1 --2- 172.21.15.38:0/3030191979 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e5f12c8590 0x55e5f16cf3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:55.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.250+0000 7f7b247e7740 1 --2- 172.21.15.38:0/3030191979 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e5f12c03b0 0x55e5f1690000 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:55.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.250+0000 7f7b247e7740 1 -- 172.21.15.38:0/3030191979 >> 172.21.15.38:0/3030191979 conn(0x55e5f12dd3c0 msgr2=0x55e5f16d8ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:55.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.250+0000 7f7b247e7740 1 -- 172.21.15.38:0/3030191979 shutdown_connections 2024-09-21T03:35:55.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.251+0000 7f7b247e7740 1 -- 172.21.15.38:0/3030191979 wait complete. 2024-09-21T03:35:55.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.251+0000 7f7b247e7740 1 -- start start 2024-09-21T03:35:55.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.251+0000 7f7b247e7740 1 Processor -- start 2024-09-21T03:35:55.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.253+0000 7f7b247e7740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e5f12c03b0 0x55e5f16e3d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:55.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.253+0000 7f7b247e7740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e5f12c8590 0x55e5f174bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:55.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.253+0000 7f7b247e7740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e5f12e9270 0x55e5f1755b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:55.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.253+0000 7f7b247e7740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55e5f16cf1c0 con 0x55e5f12c8590 2024-09-21T03:35:55.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.253+0000 7f7b247e7740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55e5f16cd540 con 0x55e5f12e9270 2024-09-21T03:35:55.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.253+0000 7f7b247e7740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55e5f16b3910 con 0x55e5f12c03b0 2024-09-21T03:35:55.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.253+0000 7f7b247e7740 10 client.0 _ll_register_callbacks cb 0x7f7b10003de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:35:55.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.253+0000 7f7b167fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e5f12c8590 0x55e5f174bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:55.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.253+0000 7f7b167fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e5f12c8590 0x55e5f174bbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44430/0 (socket says 172.21.15.38:44430) 2024-09-21T03:35:55.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.253+0000 7f7b167fc640 1 -- 172.21.15.38:0/105327933 learned_addr learned my addr 172.21.15.38:0/105327933 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:35:55.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.253+0000 7f7b16ffd640 1 --2- 172.21.15.38:0/105327933 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e5f12c03b0 0x55e5f16e3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:55.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.253+0000 7f7b177fe640 1 --2- 172.21.15.38:0/105327933 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e5f12e9270 0x55e5f1755b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:55.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.253+0000 7f7b167fc640 1 -- 172.21.15.38:0/105327933 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e5f12c03b0 msgr2=0x55e5f16e3d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:35:55.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.253+0000 7f7b167fc640 1 --2- 172.21.15.38:0/105327933 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e5f12c03b0 0x55e5f16e3d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:55.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.253+0000 7f7b167fc640 1 -- 172.21.15.38:0/105327933 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e5f12e9270 msgr2=0x55e5f1755b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:35:55.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.253+0000 7f7b167fc640 1 --2- 172.21.15.38:0/105327933 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e5f12e9270 0x55e5f1755b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:55.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.253+0000 7f7b167fc640 1 -- 172.21.15.38:0/105327933 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5f16e3ac0 con 0x55e5f12c8590 2024-09-21T03:35:55.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.253+0000 7f7b16ffd640 1 --2- 172.21.15.38:0/105327933 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e5f12c03b0 0x55e5f16e3d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:35:55.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.253+0000 7f7b167fc640 1 --2- 172.21.15.38:0/105327933 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e5f12c8590 0x55e5f174bbd0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f7b0c0628a0 tx=0x7f7b0c067330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c7cd4a4fce73bc0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:55.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.253+0000 7f7af5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-21T03:35:55.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.253+0000 7f7af5ffb640 1 -- 172.21.15.38:0/105327933 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b0c081060 con 0x55e5f12c8590 2024-09-21T03:35:55.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.253+0000 7f7af5ffb640 1 -- 172.21.15.38:0/105327933 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(55..55 src has 1..55) ==== 6101+0+0 (secure 0 0 0) 0x7f7b0c062010 con 0x55e5f12c8590 2024-09-21T03:35:55.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.253+0000 7f7b247e7740 1 -- 172.21.15.38:0/105327933 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b0800e650 con 0x55e5f12c8590 2024-09-21T03:35:55.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.254+0000 7f7af5ffb640 1 -- 172.21.15.38:0/105327933 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7b0c020040 con 0x55e5f12c8590 2024-09-21T03:35:55.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.254+0000 7f7b247e7740 10 client.5724 fetch_fsmap learned FSMap version 30 2024-09-21T03:35:55.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.254+0000 7f7b247e7740 1 -- 172.21.15.38:0/105327933 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=2+}) -- 0x55e5f1755370 con 0x55e5f12c8590 2024-09-21T03:35:55.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.254+0000 7f7af5ffb640 1 -- 172.21.15.38:0/105327933 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7b0800e650 con 0x55e5f12c8590 2024-09-21T03:35:55.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.254+0000 7f7af5ffb640 1 -- 172.21.15.38:0/105327933 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 30) ==== 3502+0+0 (secure 0 0 0) 0x7f7b0c05e4f0 con 0x55e5f12c8590 2024-09-21T03:35:55.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.254+0000 7f7b247e7740 10 client.5724 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:35:55.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.254+0000 7f7b247e7740 10 client.5724 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T03:35:55.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.254+0000 7f7b247e7740 20 client.5724 populate_metadata read hostname 'smithi038' 2024-09-21T03:35:55.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.254+0000 7f7b247e7740 1 --2- 172.21.15.38:0/105327933 >> [v2:172.21.15.38:6833/3184989541,v1:172.21.15.38:6835/3184989541] conn(0x55e5f16bad30 0x55e5f16bb140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:55.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.254+0000 7f7b247e7740 4 client.5724 mds_command: new command op to 5577 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T03:35:55.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.254+0000 7f7b247e7740 1 -- 172.21.15.38:0/105327933 --> [v2:172.21.15.38:6833/3184989541,v1:172.21.15.38:6835/3184989541] -- command(tid 0: {"prefix": "status"}) -- 0x55e5f1601220 con 0x55e5f16bad30 2024-09-21T03:35:55.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.254+0000 7f7b16ffd640 1 --2- 172.21.15.38:0/105327933 >> [v2:172.21.15.38:6833/3184989541,v1:172.21.15.38:6835/3184989541] conn(0x55e5f16bad30 0x55e5f16bb140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:55.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.254+0000 7f7af5ffb640 10 client.5724 ms_handle_connect on v2:172.21.15.38:6833/3184989541 2024-09-21T03:35:55.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.254+0000 7f7b16ffd640 1 --2- 172.21.15.38:0/105327933 >> [v2:172.21.15.38:6833/3184989541,v1:172.21.15.38:6835/3184989541] conn(0x55e5f16bad30 0x55e5f16bb140 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e7c8ad9c30dcc2fb server_cookie=2154ca1605ec51d9 in_seq=0 out_seq=0 2024-09-21T03:35:55.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.255+0000 7f7af5ffb640 1 -- 172.21.15.38:0/105327933 <== mds.0 v2:172.21.15.38:6833/3184989541 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x55e5f1601220 con 0x55e5f16bad30 2024-09-21T03:35:55.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.255+0000 7f7af5ffb640 10 client.5724 handle_command_reply: tid=0 2024-09-21T03:35:55.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.255+0000 7f7b247e7740 1 client.5724 shutdown 2024-09-21T03:35:55.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.255+0000 7f7b247e7740 1 -- 172.21.15.38:0/105327933 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e5f12c8590 msgr2=0x55e5f174bbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:55.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.255+0000 7f7b247e7740 1 --2- 172.21.15.38:0/105327933 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e5f12c8590 0x55e5f174bbd0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f7b0c0628a0 tx=0x7f7b0c067330 comp rx=0 tx=0).stop 2024-09-21T03:35:55.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.255+0000 7f7b247e7740 1 -- 172.21.15.38:0/105327933 shutdown_connections 2024-09-21T03:35:55.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.255+0000 7f7b247e7740 1 --2- 172.21.15.38:0/105327933 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e5f12e9270 0x55e5f1755b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:55.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.255+0000 7f7b247e7740 1 --2- 172.21.15.38:0/105327933 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e5f12c8590 0x55e5f174bbd0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:55.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.255+0000 7f7b247e7740 1 --2- 172.21.15.38:0/105327933 >> [v2:172.21.15.38:6833/3184989541,v1:172.21.15.38:6835/3184989541] conn(0x55e5f16bad30 0x55e5f16bb140 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:55.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.255+0000 7f7b247e7740 1 --2- 172.21.15.38:0/105327933 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e5f12c03b0 0x55e5f16e3d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:55.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.255+0000 7f7af5ffb640 0 client.5724 ms_handle_reset on v2:172.21.15.38:6833/3184989541 2024-09-21T03:35:55.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.255+0000 7f7b247e7740 1 -- 172.21.15.38:0/105327933 >> 172.21.15.38:0/105327933 conn(0x55e5f12dd3c0 msgr2=0x55e5f1692180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:55.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.255+0000 7f7b247e7740 1 -- 172.21.15.38:0/105327933 shutdown_connections 2024-09-21T03:35:55.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.255+0000 7f7b247e7740 1 -- 172.21.15.38:0/105327933 wait complete. 2024-09-21T03:35:55.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.255+0000 7f7b247e7740 20 client.5724 trim_cache size 0 max 16384 2024-09-21T03:35:55.262 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-09-21T03:35:55.262 INFO:teuthology.orchestra.run.smithi038.stdout: "cluster_fsid": "9d50d66f-656b-4fe2-8e3b-fe53b79c4d43", 2024-09-21T03:35:55.262 INFO:teuthology.orchestra.run.smithi038.stdout: "whoami": 0, 2024-09-21T03:35:55.262 INFO:teuthology.orchestra.run.smithi038.stdout: "id": 5577, 2024-09-21T03:35:55.262 INFO:teuthology.orchestra.run.smithi038.stdout: "want_state": "up:active", 2024-09-21T03:35:55.262 INFO:teuthology.orchestra.run.smithi038.stdout: "state": "up:active", 2024-09-21T03:35:55.262 INFO:teuthology.orchestra.run.smithi038.stdout: "fs_name": "cephfs", 2024-09-21T03:35:55.262 INFO:teuthology.orchestra.run.smithi038.stdout: "rank_uptime": 10.367360595999999, 2024-09-21T03:35:55.262 INFO:teuthology.orchestra.run.smithi038.stdout: "mdsmap_epoch": 30, 2024-09-21T03:35:55.263 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch": 55, 2024-09-21T03:35:55.263 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch_barrier": 55, 2024-09-21T03:35:55.263 INFO:teuthology.orchestra.run.smithi038.stdout: "uptime": 15.643719547 2024-09-21T03:35:55.263 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-09-21T03:35:55.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.256+0000 7f7b1d7a1640 1 -- 172.21.15.38:0/1780607532 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7b04078b90 msgr2=0x7f7b0407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:55.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.256+0000 7f7b1d7a1640 1 --2- 172.21.15.38:0/1780607532 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7b04078b90 0x7f7b0407b0a0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f7b080015e0 tx=0x7f7b08009040 comp rx=0 tx=0).stop 2024-09-21T03:35:55.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.256+0000 7f7b1d7a1640 1 -- 172.21.15.38:0/1780607532 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7b180d1120 msgr2=0x7f7b181575a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:55.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.256+0000 7f7b1d7a1640 1 --2- 172.21.15.38:0/1780607532 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7b180d1120 0x7f7b181575a0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f7b0c00d720 tx=0x7f7b0c00dbe0 comp rx=0 tx=0).stop 2024-09-21T03:35:55.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.256+0000 7f7b1d7a1640 1 -- 172.21.15.38:0/1780607532 shutdown_connections 2024-09-21T03:35:55.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.256+0000 7f7b1d7a1640 1 --2- 172.21.15.38:0/1780607532 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7b1815ee80 0x7f7b180b2620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:55.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.256+0000 7f7b1d7a1640 1 --2- 172.21.15.38:0/1780607532 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7b180d1120 0x7f7b181575a0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:55.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.256+0000 7f7b1d7a1640 1 --2- 172.21.15.38:0/1780607532 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7b04078b90 0x7f7b0407b0a0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:55.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.256+0000 7f7b1d7a1640 1 --2- 172.21.15.38:0/1780607532 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7b180d0170 0x7f7b1806ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:55.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.256+0000 7f7b1d7a1640 1 -- 172.21.15.38:0/1780607532 >> 172.21.15.38:0/1780607532 conn(0x7f7b1809d830 msgr2=0x7f7b18160950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:55.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.256+0000 7f7b1d7a1640 1 -- 172.21.15.38:0/1780607532 shutdown_connections 2024-09-21T03:35:55.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.256+0000 7f7b1d7a1640 1 -- 172.21.15.38:0/1780607532 wait complete. 2024-09-21T03:35:55.269 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T03:35:55.390 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.388+0000 7f069d886640 1 -- 172.21.15.38:0/9840244 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0684004800 msgr2=0x7f0684004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:55.391 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.388+0000 7f069d886640 1 --2- 172.21.15.38:0/9840244 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0684004800 0x7f0684004c10 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f0680009b50 tx=0x7f0684005830 comp rx=0 tx=0).stop 2024-09-21T03:35:55.391 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.388+0000 7f069d886640 1 -- 172.21.15.38:0/9840244 shutdown_connections 2024-09-21T03:35:55.391 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.388+0000 7f069d886640 1 --2- 172.21.15.38:0/9840244 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0684004800 0x7f0684004c10 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:55.391 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.388+0000 7f069d886640 1 -- 172.21.15.38:0/9840244 >> 172.21.15.38:0/9840244 conn(0x7f06980cca40 msgr2=0x7f06980cce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:55.391 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.388+0000 7f069d886640 1 -- 172.21.15.38:0/9840244 shutdown_connections 2024-09-21T03:35:55.391 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.388+0000 7f069d886640 1 -- 172.21.15.38:0/9840244 wait complete. 2024-09-21T03:35:55.391 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.390+0000 7f069d886640 1 Processor -- start 2024-09-21T03:35:55.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.390+0000 7f069d886640 1 -- start start 2024-09-21T03:35:55.394 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.391+0000 7f069d886640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0684004800 0x7f069815e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:55.394 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.391+0000 7f069d886640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f06980c7890 0x7f069815e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:55.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.391+0000 7f069d886640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f06981f1b30 0x7f06981f1f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:55.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.391+0000 7f069d886640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f06980cefa0 con 0x7f06981f1b30 2024-09-21T03:35:55.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.391+0000 7f069d886640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f06980ce640 con 0x7f06980c7890 2024-09-21T03:35:55.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.391+0000 7f069d886640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f06980cf120 con 0x7f0684004800 2024-09-21T03:35:55.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.391+0000 7f06977fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f06981f1b30 0x7f06981f1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:55.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.391+0000 7f06977fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f06981f1b30 0x7f06981f1f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44436/0 (socket says 172.21.15.38:44436) 2024-09-21T03:35:55.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.391+0000 7f06977fe640 1 -- 172.21.15.38:0/2490271929 learned_addr learned my addr 172.21.15.38:0/2490271929 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:35:55.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.392+0000 7f06977fe640 1 -- 172.21.15.38:0/2490271929 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0684004800 msgr2=0x7f069815e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:55.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.392+0000 7f06977fe640 1 --2- 172.21.15.38:0/2490271929 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0684004800 0x7f069815e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:55.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.392+0000 7f06977fe640 1 -- 172.21.15.38:0/2490271929 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f06980c7890 msgr2=0x7f069815e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:55.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.392+0000 7f06977fe640 1 --2- 172.21.15.38:0/2490271929 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f06980c7890 0x7f069815e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:55.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.392+0000 7f06977fe640 1 -- 172.21.15.38:0/2490271929 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06980c7e70 con 0x7f06981f1b30 2024-09-21T03:35:55.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.392+0000 7f06977fe640 1 --2- 172.21.15.38:0/2490271929 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f06981f1b30 0x7f06981f1f40 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f0698152a50 tx=0x7f06900064f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:55.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.392+0000 7f0694ff9640 1 -- 172.21.15.38:0/2490271929 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0690011900 con 0x7f06981f1b30 2024-09-21T03:35:55.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.392+0000 7f0694ff9640 1 -- 172.21.15.38:0/2490271929 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0690018070 con 0x7f06981f1b30 2024-09-21T03:35:55.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.392+0000 7f069d886640 1 -- 172.21.15.38:0/2490271929 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06980c8150 con 0x7f06981f1b30 2024-09-21T03:35:55.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.392+0000 7f0694ff9640 1 -- 172.21.15.38:0/2490271929 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0690011d60 con 0x7f06981f1b30 2024-09-21T03:35:55.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.392+0000 7f069d886640 1 -- 172.21.15.38:0/2490271929 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06980c8710 con 0x7f06981f1b30 2024-09-21T03:35:55.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.393+0000 7f069d886640 1 -- 172.21.15.38:0/2490271929 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f069815f5a0 con 0x7f06981f1b30 2024-09-21T03:35:55.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.394+0000 7f0694ff9640 1 -- 172.21.15.38:0/2490271929 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f069001d460 con 0x7f06981f1b30 2024-09-21T03:35:55.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.394+0000 7f0694ff9640 1 --2- 172.21.15.38:0/2490271929 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0678078b90 0x7f067807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:55.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.394+0000 7f0694ff9640 1 -- 172.21.15.38:0/2490271929 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 6101+0+0 (secure 0 0 0) 0x7f069009b9e0 con 0x7f06981f1b30 2024-09-21T03:35:55.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.395+0000 7f0696ffd640 1 --2- 172.21.15.38:0/2490271929 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0678078b90 0x7f067807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:55.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.395+0000 7f0696ffd640 1 --2- 172.21.15.38:0/2490271929 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0678078b90 0x7f067807b0a0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f069806bb20 tx=0x7f0688006180 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:55.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.399+0000 7f0694ff9640 1 -- 172.21.15.38:0/2490271929 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f0690063bf0 con 0x7f06981f1b30 2024-09-21T03:35:55.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.624+0000 7f069d886640 1 -- 172.21.15.38:0/2490271929 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f069815b1b0 con 0x7f06981f1b30 2024-09-21T03:35:55.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.624+0000 7f0694ff9640 1 -- 172.21.15.38:0/2490271929 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v30) ==== 53+0+73 (secure 0 0 0) 0x7f06900634f0 con 0x7f06981f1b30 2024-09-21T03:35:55.626 INFO:teuthology.orchestra.run.smithi038.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T03:35:55.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.626+0000 7f069d886640 1 -- 172.21.15.38:0/2490271929 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0678078b90 msgr2=0x7f067807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:55.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.626+0000 7f069d886640 1 --2- 172.21.15.38:0/2490271929 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0678078b90 0x7f067807b0a0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f069806bb20 tx=0x7f0688006180 comp rx=0 tx=0).stop 2024-09-21T03:35:55.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.626+0000 7f069d886640 1 -- 172.21.15.38:0/2490271929 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f06981f1b30 msgr2=0x7f06981f1f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:55.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.626+0000 7f069d886640 1 --2- 172.21.15.38:0/2490271929 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f06981f1b30 0x7f06981f1f40 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f0698152a50 tx=0x7f06900064f0 comp rx=0 tx=0).stop 2024-09-21T03:35:55.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.626+0000 7f069d886640 1 -- 172.21.15.38:0/2490271929 shutdown_connections 2024-09-21T03:35:55.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.626+0000 7f069d886640 1 --2- 172.21.15.38:0/2490271929 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f06981f1b30 0x7f06981f1f40 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:55.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.626+0000 7f069d886640 1 --2- 172.21.15.38:0/2490271929 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f06980c7890 0x7f069815e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:55.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.626+0000 7f069d886640 1 --2- 172.21.15.38:0/2490271929 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0678078b90 0x7f067807b0a0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:55.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.626+0000 7f069d886640 1 --2- 172.21.15.38:0/2490271929 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0684004800 0x7f069815e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:55.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.626+0000 7f069d886640 1 -- 172.21.15.38:0/2490271929 >> 172.21.15.38:0/2490271929 conn(0x7f06980cca40 msgr2=0x7f069806d7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:55.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.626+0000 7f069d886640 1 -- 172.21.15.38:0/2490271929 shutdown_connections 2024-09-21T03:35:55.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55.626+0000 7f069d886640 1 -- 172.21.15.38:0/2490271929 wait complete. 2024-09-21T03:35:55.639 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-21T03:35:55.639 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T03:35:55.639 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi038.front.sepia.ceph.com 2024-09-21T03:35:55.640 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-21T03:35:55.640 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T03:35:55.640 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-21T03:35:55.640 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-21T03:35:55.640 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T03:35:55.640 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T03:35:55.640 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:35:55.640 DEBUG:teuthology.orchestra.run.smithi038:> ip addr 2024-09-21T03:35:55.657 INFO:teuthology.orchestra.run.smithi038.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T03:35:55.657 INFO:teuthology.orchestra.run.smithi038.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T03:35:55.657 INFO:teuthology.orchestra.run.smithi038.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T03:35:55.657 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:35:55.657 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 ::1/128 scope host 2024-09-21T03:35:55.657 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:35:55.657 INFO:teuthology.orchestra.run.smithi038.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T03:35:55.657 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:6b:fd:a2 brd ff:ff:ff:ff:ff:ff 2024-09-21T03:35:55.657 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp7s0f0 2024-09-21T03:35:55.657 INFO:teuthology.orchestra.run.smithi038.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T03:35:55.657 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:6b:fd:a3 brd ff:ff:ff:ff:ff:ff 2024-09-21T03:35:55.658 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp7s0f1 2024-09-21T03:35:55.658 INFO:teuthology.orchestra.run.smithi038.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T03:35:55.658 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:bd:15:fc brd ff:ff:ff:ff:ff:ff 2024-09-21T03:35:55.658 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp1s0f0 2024-09-21T03:35:55.658 INFO:teuthology.orchestra.run.smithi038.stdout: inet 172.21.15.38/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-21T03:35:55.658 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft 42653sec preferred_lft 42653sec 2024-09-21T03:35:55.658 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 fe80::ec4:7aff:febd:15fc/64 scope link 2024-09-21T03:35:55.658 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:35:55.658 INFO:teuthology.orchestra.run.smithi038.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T03:35:55.658 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:bd:15:fd brd ff:ff:ff:ff:ff:ff 2024-09-21T03:35:55.658 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp1s0f1 2024-09-21T03:35:55.658 INFO:teuthology.orchestra.run.smithi038.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T03:35:55.658 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 92:55:c7:97:6f:b5 brd ff:ff:ff:ff:ff:ff 2024-09-21T03:35:55.659 INFO:teuthology.orchestra.run.smithi038.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-21T03:35:55.659 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:35:55.659 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 fe80::c49f:c1ff:fe21:b12a/64 scope link 2024-09-21T03:35:55.659 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:35:55.659 INFO:teuthology.orchestra.run.smithi038.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T03:35:55.659 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether a6:3a:83:ad:db:d5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T03:35:55.659 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 fe80::a43a:83ff:fead:dbd5/64 scope link 2024-09-21T03:35:55.659 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:35:55.659 INFO:teuthology.orchestra.run.smithi038.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T03:35:55.659 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 92:55:c7:97:6f:b5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T03:35:55.659 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 fe80::9055:c7ff:fe97:6fb5/64 scope link 2024-09-21T03:35:55.659 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:35:55.660 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:35:55.660 DEBUG:teuthology.orchestra.run.smithi038:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T03:35:55.722 INFO:teuthology.orchestra.run.smithi038.stdout:1 2024-09-21T03:35:55.724 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:35:55.724 DEBUG:teuthology.orchestra.run.smithi038:> route 2024-09-21T03:35:55.780 INFO:teuthology.orchestra.run.smithi038.stdout:Kernel IP routing table 2024-09-21T03:35:55.780 INFO:teuthology.orchestra.run.smithi038.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T03:35:55.780 INFO:teuthology.orchestra.run.smithi038.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-21T03:35:55.781 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-21T03:35:55.781 INFO:teuthology.orchestra.run.smithi038.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T03:35:55.781 DEBUG:teuthology.orchestra.run.smithi038:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T03:35:55.781 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-09-21T03:35:55.781 DEBUG:teuthology.orchestra.run.smithi038:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-21T03:35:55.781 DEBUG:teuthology.orchestra.run.smithi038:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-21T03:35:55.781 DEBUG:teuthology.orchestra.run.smithi038:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-21T03:35:55.781 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-09-21T03:35:55.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T03:35:55.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:35:55.909 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:35:55.909 DEBUG:teuthology.orchestra.run.smithi038:> ip netns list 2024-09-21T03:35:55.925 INFO:teuthology.orchestra.run.smithi038.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-21T03:35:55.925 INFO:teuthology.orchestra.run.smithi038.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-21T03:35:55.925 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T03:35:55.925 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:35:55.925 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip link set brx.0 up 2024-09-21T03:35:55.990 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T03:35:55.990 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-21T03:35:56.046 INFO:teuthology.orchestra.run.smithi038.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-21T03:35:56.047 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T03:35:56.047 DEBUG:teuthology.orchestra.run.smithi038:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-21T03:35:56.102 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-21T03:35:56.102 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:35:56.102 DEBUG:teuthology.orchestra.run.smithi038:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T03:35:56.198 INFO:teuthology.orchestra.run.smithi038.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T03:35:56.199 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T03:35:56.199 INFO:teuthology.orchestra.run.smithi038.stdout:invalid new device string format 2024-09-21T03:35:56.209 INFO:teuthology.orchestra.run.smithi038.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T03:35:56.209 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T03:35:56.210 INFO:teuthology.orchestra.run.smithi038.stdout:invalid new device string format 2024-09-21T03:35:56.210 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: resolved to: "172.21.15.38:3300,172.21.15.195:3300,172.21.15.195:3301" 2024-09-21T03:35:56.210 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T03:35:56.220 INFO:teuthology.orchestra.run.smithi038.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T03:35:56.220 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T03:35:56.220 INFO:teuthology.orchestra.run.smithi038.stdout:invalid new device string format 2024-09-21T03:35:56.220 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: resolved to: "172.21.15.38:3300,172.21.15.195:3300,172.21.15.195:3301" 2024-09-21T03:35:56.220 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T03:35:56.220 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: resolved to: "172.21.15.38:6789,172.21.15.195:6789,172.21.15.195:6790" 2024-09-21T03:35:56.220 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.38:6789,172.21.15.195:6789,172.21.15.195:6790:/ 2024-09-21T03:35:56.220 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=9d50d66f-656b-4fe2-8e3b-fe53b79c4d43" will pass to kernel 2024-09-21T03:35:56.221 INFO:teuthology.orchestra.run.smithi038.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-21T03:35:56.221 INFO:teuthology.orchestra.run.smithi038.stdout: You just mounted a file system that supports labels which does not 2024-09-21T03:35:56.221 INFO:teuthology.orchestra.run.smithi038.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-21T03:35:56.221 INFO:teuthology.orchestra.run.smithi038.stdout: applications will generate AVC messages and not be allowed access to 2024-09-21T03:35:56.221 INFO:teuthology.orchestra.run.smithi038.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-21T03:35:56.222 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T03:35:56.222 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:35:56.223 DEBUG:teuthology.orchestra.run.smithi038:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-21T03:35:56.251 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T03:35:56.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T03:35:56.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:35:56.355 INFO:teuthology.orchestra.run.smithi038.stdout:client5246 2024-09-21T03:35:56.355 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-21T03:35:56.355 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:35:56.372 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'mkdir -p a/b') 2024-09-21T03:35:56.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T03:35:56.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:35:56.478 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 flush journal 2024-09-21T03:35:56.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.595+0000 7f473f44d640 1 -- 172.21.15.38:0/2925485620 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f47380ceb90 msgr2=0x7f47380cefa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:56.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.595+0000 7f473f44d640 1 --2- 172.21.15.38:0/2925485620 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f47380ceb90 0x7f47380cefa0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f4730005fd0 tx=0x7f473002a540 comp rx=0 tx=0).stop 2024-09-21T03:35:56.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.595+0000 7f473f44d640 1 -- 172.21.15.38:0/2925485620 shutdown_connections 2024-09-21T03:35:56.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.595+0000 7f473f44d640 1 --2- 172.21.15.38:0/2925485620 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f47380ceb90 0x7f47380cefa0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:56.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.595+0000 7f473f44d640 1 --2- 172.21.15.38:0/2925485620 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f473815ee80 0x7f47380d0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:56.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.595+0000 7f473f44d640 1 -- 172.21.15.38:0/2925485620 >> 172.21.15.38:0/2925485620 conn(0x7f473809d830 msgr2=0x7f47380b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:56.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.595+0000 7f473f44d640 1 -- 172.21.15.38:0/2925485620 shutdown_connections 2024-09-21T03:35:56.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.595+0000 7f473f44d640 1 -- 172.21.15.38:0/2925485620 wait complete. 2024-09-21T03:35:56.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.597+0000 7f473f44d640 1 Processor -- start 2024-09-21T03:35:56.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.597+0000 7f473f44d640 1 -- start start 2024-09-21T03:35:56.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.599+0000 7f473f44d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47380b9ad0 0x7f47380a1e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:56.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.599+0000 7f473f44d640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f47380ceb90 0x7f4738158990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:56.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.599+0000 7f473f44d640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f473815ee80 0x7f473809eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:56.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.599+0000 7f473f44d640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f47380b6700 con 0x7f47380b9ad0 2024-09-21T03:35:56.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.599+0000 7f473f44d640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f47380b6580 con 0x7f473815ee80 2024-09-21T03:35:56.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.599+0000 7f473f44d640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f47380b6880 con 0x7f47380ceb90 2024-09-21T03:35:56.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.599+0000 7f473d1e9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47380b9ad0 0x7f47380a1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:56.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.599+0000 7f473d1e9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47380b9ad0 0x7f47380a1e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44452/0 (socket says 172.21.15.38:44452) 2024-09-21T03:35:56.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.599+0000 7f473d1e9640 1 -- 172.21.15.38:0/4077130929 learned_addr learned my addr 172.21.15.38:0/4077130929 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:35:56.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.599+0000 7f473d9ea640 1 --2- 172.21.15.38:0/4077130929 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f473815ee80 0x7f473809eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:56.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.599+0000 7f473c9e8640 1 --2- 172.21.15.38:0/4077130929 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f47380ceb90 0x7f4738158990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:56.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.599+0000 7f473d1e9640 1 -- 172.21.15.38:0/4077130929 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f47380ceb90 msgr2=0x7f4738158990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:56.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.599+0000 7f473d1e9640 1 --2- 172.21.15.38:0/4077130929 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f47380ceb90 0x7f4738158990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:56.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.599+0000 7f473d1e9640 1 -- 172.21.15.38:0/4077130929 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f473815ee80 msgr2=0x7f473809eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:56.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.599+0000 7f473d1e9640 1 --2- 172.21.15.38:0/4077130929 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f473815ee80 0x7f473809eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:56.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.599+0000 7f473d1e9640 1 -- 172.21.15.38:0/4077130929 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47380a0dc0 con 0x7f47380b9ad0 2024-09-21T03:35:56.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.599+0000 7f473d1e9640 1 --2- 172.21.15.38:0/4077130929 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47380b9ad0 0x7f47380a1e20 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f472800ece0 tx=0x7f472800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:56.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.599+0000 7f4735ffb640 1 -- 172.21.15.38:0/4077130929 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4728003b70 con 0x7f47380b9ad0 2024-09-21T03:35:56.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.599+0000 7f4735ffb640 1 -- 172.21.15.38:0/4077130929 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4728003d10 con 0x7f47380b9ad0 2024-09-21T03:35:56.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.600+0000 7f473f44d640 1 -- 172.21.15.38:0/4077130929 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47380a1060 con 0x7f47380b9ad0 2024-09-21T03:35:56.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.600+0000 7f4735ffb640 1 -- 172.21.15.38:0/4077130929 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4728004280 con 0x7f47380b9ad0 2024-09-21T03:35:56.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.600+0000 7f473f44d640 1 -- 172.21.15.38:0/4077130929 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47380a1620 con 0x7f47380b9ad0 2024-09-21T03:35:56.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.601+0000 7f4735ffb640 1 -- 172.21.15.38:0/4077130929 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f4728004420 con 0x7f47380b9ad0 2024-09-21T03:35:56.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.602+0000 7f4735ffb640 1 --2- 172.21.15.38:0/4077130929 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4724078b90 0x7f472407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:56.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.602+0000 7f4735ffb640 1 -- 172.21.15.38:0/4077130929 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 6101+0+0 (secure 0 0 0) 0x7f472809bb30 con 0x7f47380b9ad0 2024-09-21T03:35:56.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.602+0000 7f473c9e8640 1 --2- 172.21.15.38:0/4077130929 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4724078b90 0x7f472407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:56.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.603+0000 7f473c9e8640 1 --2- 172.21.15.38:0/4077130929 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4724078b90 0x7f472407b0a0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f473809ea60 tx=0x7f472c009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:56.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.603+0000 7f4745ffd740 1 Processor -- start 2024-09-21T03:35:56.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.603+0000 7f4745ffd740 1 -- start start 2024-09-21T03:35:56.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.604+0000 7f4745ffd740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564f85db9390 0x564f861a5a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:56.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.604+0000 7f4745ffd740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x564f85d904d0 0x564f8617a860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:56.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.604+0000 7f4745ffd740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564f85d986b0 0x564f8617ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:56.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.604+0000 7f4745ffd740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x564f860d3a60 con 0x564f85d986b0 2024-09-21T03:35:56.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.604+0000 7f4745ffd740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x564f860c2e90 con 0x564f85db9390 2024-09-21T03:35:56.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.604+0000 7f4745ffd740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x564f860c6ca0 con 0x564f85d904d0 2024-09-21T03:35:56.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.604+0000 7f473d9ea640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564f85d986b0 0x564f8617ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:56.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.604+0000 7f473d9ea640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564f85d986b0 0x564f8617ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44456/0 (socket says 172.21.15.38:44456) 2024-09-21T03:35:56.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.604+0000 7f473d9ea640 1 -- 172.21.15.38:0/3750925289 learned_addr learned my addr 172.21.15.38:0/3750925289 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:35:56.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.604+0000 7f473c9e8640 1 --2- 172.21.15.38:0/3750925289 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x564f85d904d0 0x564f8617a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:56.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.604+0000 7f473d1e9640 1 --2- 172.21.15.38:0/3750925289 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564f85db9390 0x564f861a5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:56.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.605+0000 7f473d9ea640 1 -- 172.21.15.38:0/3750925289 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x564f85d904d0 msgr2=0x564f8617a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:35:56.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.605+0000 7f473d9ea640 1 --2- 172.21.15.38:0/3750925289 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x564f85d904d0 0x564f8617a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:56.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.605+0000 7f473d9ea640 1 -- 172.21.15.38:0/3750925289 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564f85db9390 msgr2=0x564f861a5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:35:56.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.605+0000 7f473d9ea640 1 --2- 172.21.15.38:0/3750925289 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564f85db9390 0x564f861a5a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:56.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.605+0000 7f473d9ea640 1 -- 172.21.15.38:0/3750925289 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f85e9b7d0 con 0x564f85d986b0 2024-09-21T03:35:56.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.605+0000 7f473d9ea640 1 --2- 172.21.15.38:0/3750925289 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564f85d986b0 0x564f8617ada0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f47300099f0 tx=0x7f47300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcd861c85099b2f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:56.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.605+0000 7f471effd640 1 -- 172.21.15.38:0/3750925289 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4730035040 con 0x564f85d986b0 2024-09-21T03:35:56.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.605+0000 7f471effd640 1 -- 172.21.15.38:0/3750925289 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4730004320 con 0x564f85d986b0 2024-09-21T03:35:56.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.605+0000 7f471effd640 1 -- 172.21.15.38:0/3750925289 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4730003850 con 0x564f85d986b0 2024-09-21T03:35:56.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.606+0000 7f4745ffd740 1 -- 172.21.15.38:0/3750925289 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564f85d986b0 msgr2=0x564f8617ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:56.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.606+0000 7f4745ffd740 1 --2- 172.21.15.38:0/3750925289 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564f85d986b0 0x564f8617ada0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f47300099f0 tx=0x7f47300046c0 comp rx=0 tx=0).stop 2024-09-21T03:35:56.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.606+0000 7f4745ffd740 1 -- 172.21.15.38:0/3750925289 shutdown_connections 2024-09-21T03:35:56.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.606+0000 7f4745ffd740 1 --2- 172.21.15.38:0/3750925289 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564f85d986b0 0x564f8617ada0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:56.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.606+0000 7f4745ffd740 1 --2- 172.21.15.38:0/3750925289 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x564f85d904d0 0x564f8617a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:56.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.606+0000 7f4745ffd740 1 --2- 172.21.15.38:0/3750925289 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564f85db9390 0x564f861a5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:56.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.606+0000 7f4745ffd740 1 -- 172.21.15.38:0/3750925289 >> 172.21.15.38:0/3750925289 conn(0x564f85dad4e0 msgr2=0x564f86160d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:56.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.606+0000 7f4745ffd740 1 -- 172.21.15.38:0/3750925289 shutdown_connections 2024-09-21T03:35:56.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.606+0000 7f4745ffd740 1 -- 172.21.15.38:0/3750925289 wait complete. 2024-09-21T03:35:56.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.607+0000 7f4745ffd740 1 -- start start 2024-09-21T03:35:56.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.607+0000 7f4745ffd740 1 Processor -- start 2024-09-21T03:35:56.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.608+0000 7f4745ffd740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x564f85d904d0 0x564f86178cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:56.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.608+0000 7f4745ffd740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564f85d986b0 0x564f86188f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:56.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.608+0000 7f4745ffd740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564f85db9390 0x564f861894c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:56.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.608+0000 7f4745ffd740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x564f85e33740 con 0x564f85db9390 2024-09-21T03:35:56.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.608+0000 7f4745ffd740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x564f86068970 con 0x564f85d986b0 2024-09-21T03:35:56.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.608+0000 7f4745ffd740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x564f8618a020 con 0x564f85d904d0 2024-09-21T03:35:56.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.608+0000 7f4745ffd740 10 client.0 _ll_register_callbacks cb 0x564f86077780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:35:56.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.608+0000 7f473d9ea640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564f85db9390 0x564f861894c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:56.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.608+0000 7f473d9ea640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564f85db9390 0x564f861894c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44458/0 (socket says 172.21.15.38:44458) 2024-09-21T03:35:56.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.609+0000 7f473d9ea640 1 -- 172.21.15.38:0/2652369658 learned_addr learned my addr 172.21.15.38:0/2652369658 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:35:56.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.609+0000 7f473c9e8640 1 --2- 172.21.15.38:0/2652369658 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564f85d986b0 0x564f86188f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:56.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.609+0000 7f473d1e9640 1 --2- 172.21.15.38:0/2652369658 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x564f85d904d0 0x564f86178cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:56.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.609+0000 7f473d9ea640 1 -- 172.21.15.38:0/2652369658 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x564f85d904d0 msgr2=0x564f86178cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:35:56.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.609+0000 7f473d9ea640 1 --2- 172.21.15.38:0/2652369658 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x564f85d904d0 0x564f86178cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:56.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.609+0000 7f473d9ea640 1 -- 172.21.15.38:0/2652369658 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564f85d986b0 msgr2=0x564f86188f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:35:56.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.609+0000 7f473d9ea640 1 --2- 172.21.15.38:0/2652369658 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564f85d986b0 0x564f86188f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:56.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.609+0000 7f473d9ea640 1 -- 172.21.15.38:0/2652369658 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f85e24970 con 0x564f85db9390 2024-09-21T03:35:56.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.609+0000 7f473c9e8640 1 --2- 172.21.15.38:0/2652369658 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564f85d986b0 0x564f86188f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:35:56.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.609+0000 7f473d9ea640 1 --2- 172.21.15.38:0/2652369658 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564f85db9390 0x564f861894c0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f4730009850 tx=0x7f4730004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1686b4a4f160ff74 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:56.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.609+0000 7f4717fff640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-21T03:35:56.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.609+0000 7f4717fff640 1 -- 172.21.15.38:0/2652369658 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4730035030 con 0x564f85db9390 2024-09-21T03:35:56.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.609+0000 7f4717fff640 1 -- 172.21.15.38:0/2652369658 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(55..55 src has 1..55) ==== 6101+0+0 (secure 0 0 0) 0x7f4730003b30 con 0x564f85db9390 2024-09-21T03:35:56.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.609+0000 7f4745ffd740 1 -- 172.21.15.38:0/2652369658 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4730004320 con 0x564f85db9390 2024-09-21T03:35:56.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.609+0000 7f4717fff640 1 -- 172.21.15.38:0/2652369658 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4730009520 con 0x564f85db9390 2024-09-21T03:35:56.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.609+0000 7f4745ffd740 10 client.5739 fetch_fsmap learned FSMap version 30 2024-09-21T03:35:56.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.609+0000 7f4745ffd740 1 -- 172.21.15.38:0/2652369658 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=2+}) -- 0x564f85e026a0 con 0x564f85db9390 2024-09-21T03:35:56.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.610+0000 7f4717fff640 1 -- 172.21.15.38:0/2652369658 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4730004320 con 0x564f85db9390 2024-09-21T03:35:56.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.610+0000 7f4717fff640 1 -- 172.21.15.38:0/2652369658 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 30) ==== 3502+0+0 (secure 0 0 0) 0x7f47300047a0 con 0x564f85db9390 2024-09-21T03:35:56.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.610+0000 7f4745ffd740 10 client.5739 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:35:56.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.610+0000 7f4745ffd740 10 client.5739 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-09-21T03:35:56.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.610+0000 7f4745ffd740 20 client.5739 populate_metadata read hostname 'smithi038' 2024-09-21T03:35:56.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.610+0000 7f4745ffd740 1 --2- 172.21.15.38:0/2652369658 >> [v2:172.21.15.38:6833/3184989541,v1:172.21.15.38:6835/3184989541] conn(0x564f85d77f60 0x564f861663e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:56.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.610+0000 7f4745ffd740 4 client.5739 mds_command: new command op to 5577 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:35:56.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.610+0000 7f4745ffd740 1 -- 172.21.15.38:0/2652369658 --> [v2:172.21.15.38:6833/3184989541,v1:172.21.15.38:6835/3184989541] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564f860d59b0 con 0x564f85d77f60 2024-09-21T03:35:56.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.610+0000 7f473d1e9640 1 --2- 172.21.15.38:0/2652369658 >> [v2:172.21.15.38:6833/3184989541,v1:172.21.15.38:6835/3184989541] conn(0x564f85d77f60 0x564f861663e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:56.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.610+0000 7f4717fff640 10 client.5739 ms_handle_connect on v2:172.21.15.38:6833/3184989541 2024-09-21T03:35:56.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.610+0000 7f473d1e9640 1 --2- 172.21.15.38:0/2652369658 >> [v2:172.21.15.38:6833/3184989541,v1:172.21.15.38:6835/3184989541] conn(0x564f85d77f60 0x564f861663e0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=466e0c36ac10788 server_cookie=2de9eda77a3c1b50 in_seq=0 out_seq=0 2024-09-21T03:35:56.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.611+0000 7f4717fff640 1 -- 172.21.15.38:0/2652369658 <== mds.0 v2:172.21.15.38:6833/3184989541 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x564f860d59b0 con 0x564f85d77f60 2024-09-21T03:35:56.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.611+0000 7f4717fff640 10 client.5739 handle_command_reply: tid=0 2024-09-21T03:35:56.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.611+0000 7f4745ffd740 1 client.5739 shutdown 2024-09-21T03:35:56.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.611+0000 7f4745ffd740 1 -- 172.21.15.38:0/2652369658 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564f85db9390 msgr2=0x564f861894c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:56.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.611+0000 7f4745ffd740 1 --2- 172.21.15.38:0/2652369658 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564f85db9390 0x564f861894c0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f4730009850 tx=0x7f4730004610 comp rx=0 tx=0).stop 2024-09-21T03:35:56.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.611+0000 7f4745ffd740 1 -- 172.21.15.38:0/2652369658 shutdown_connections 2024-09-21T03:35:56.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.611+0000 7f4745ffd740 1 --2- 172.21.15.38:0/2652369658 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564f85db9390 0x564f861894c0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:56.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.611+0000 7f4745ffd740 1 --2- 172.21.15.38:0/2652369658 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564f85d986b0 0x564f86188f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:56.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.611+0000 7f4745ffd740 1 --2- 172.21.15.38:0/2652369658 >> [v2:172.21.15.38:6833/3184989541,v1:172.21.15.38:6835/3184989541] conn(0x564f85d77f60 0x564f861663e0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:56.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.611+0000 7f4745ffd740 1 --2- 172.21.15.38:0/2652369658 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x564f85d904d0 0x564f86178cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:56.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.611+0000 7f4745ffd740 1 -- 172.21.15.38:0/2652369658 >> 172.21.15.38:0/2652369658 conn(0x564f85dad4e0 msgr2=0x564f861a4e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:56.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.611+0000 7f4717fff640 0 client.5739 ms_handle_reset on v2:172.21.15.38:6833/3184989541 2024-09-21T03:35:56.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.611+0000 7f4745ffd740 1 -- 172.21.15.38:0/2652369658 shutdown_connections 2024-09-21T03:35:56.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.611+0000 7f4745ffd740 1 -- 172.21.15.38:0/2652369658 wait complete. 2024-09-21T03:35:56.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.612+0000 7f4745ffd740 20 client.5739 trim_cache size 0 max 16384 2024-09-21T03:35:56.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.626+0000 7f4745ffd740 1 Processor -- start 2024-09-21T03:35:56.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.627+0000 7f4745ffd740 1 -- start start 2024-09-21T03:35:56.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.628+0000 7f4745ffd740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564f85d904d0 0x564f8615b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:56.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.628+0000 7f4745ffd740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564f85d986b0 0x564f8617a850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:56.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.628+0000 7f4745ffd740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x564f85db9390 0x564f8617ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:56.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.628+0000 7f4745ffd740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f47382026c0 con 0x564f85d986b0 2024-09-21T03:35:56.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.628+0000 7f4745ffd740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f47382016e0 con 0x564f85d904d0 2024-09-21T03:35:56.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.628+0000 7f4745ffd740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f47380cc3f0 con 0x564f85db9390 2024-09-21T03:35:56.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.628+0000 7f473c9e8640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564f85d986b0 0x564f8617a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:56.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.628+0000 7f473c9e8640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564f85d986b0 0x564f8617a850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44466/0 (socket says 172.21.15.38:44466) 2024-09-21T03:35:56.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.628+0000 7f473c9e8640 1 -- 172.21.15.38:0/1200764248 learned_addr learned my addr 172.21.15.38:0/1200764248 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:35:56.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.628+0000 7f473d1e9640 1 --2- 172.21.15.38:0/1200764248 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564f85d904d0 0x564f8615b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:56.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.628+0000 7f473c9e8640 1 -- 172.21.15.38:0/1200764248 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x564f85db9390 msgr2=0x564f8617ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:35:56.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.628+0000 7f473d9ea640 1 --2- 172.21.15.38:0/1200764248 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x564f85db9390 0x564f8617ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:56.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.628+0000 7f473c9e8640 1 --2- 172.21.15.38:0/1200764248 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x564f85db9390 0x564f8617ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:56.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.628+0000 7f473c9e8640 1 -- 172.21.15.38:0/1200764248 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564f85d904d0 msgr2=0x564f8615b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:35:56.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.628+0000 7f473c9e8640 1 --2- 172.21.15.38:0/1200764248 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564f85d904d0 0x564f8615b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:56.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.628+0000 7f473c9e8640 1 -- 172.21.15.38:0/1200764248 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f8615d860 con 0x564f85d986b0 2024-09-21T03:35:56.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.628+0000 7f473d1e9640 1 --2- 172.21.15.38:0/1200764248 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564f85d904d0 0x564f8615b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:35:56.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.628+0000 7f473c9e8640 1 --2- 172.21.15.38:0/1200764248 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564f85d986b0 0x564f8617a850 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f472c006a60 tx=0x7f472c012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15ca9db0fb43802b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:56.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.629+0000 7f471e7fc640 1 -- 172.21.15.38:0/1200764248 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f472c028070 con 0x564f85d986b0 2024-09-21T03:35:56.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.629+0000 7f471e7fc640 1 -- 172.21.15.38:0/1200764248 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f472c007120 con 0x564f85d986b0 2024-09-21T03:35:56.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.629+0000 7f471e7fc640 1 -- 172.21.15.38:0/1200764248 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f472c019070 con 0x564f85d986b0 2024-09-21T03:35:56.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.629+0000 7f4745ffd740 1 -- 172.21.15.38:0/1200764248 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564f85d986b0 msgr2=0x564f8617a850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:56.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.629+0000 7f4745ffd740 1 --2- 172.21.15.38:0/1200764248 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564f85d986b0 0x564f8617a850 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f472c006a60 tx=0x7f472c012000 comp rx=0 tx=0).stop 2024-09-21T03:35:56.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.629+0000 7f4745ffd740 1 -- 172.21.15.38:0/1200764248 shutdown_connections 2024-09-21T03:35:56.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.629+0000 7f4745ffd740 1 --2- 172.21.15.38:0/1200764248 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x564f85db9390 0x564f8617ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:56.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.629+0000 7f4745ffd740 1 --2- 172.21.15.38:0/1200764248 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564f85d986b0 0x564f8617a850 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:56.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.629+0000 7f4745ffd740 1 --2- 172.21.15.38:0/1200764248 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564f85d904d0 0x564f8615b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:56.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.629+0000 7f4745ffd740 1 -- 172.21.15.38:0/1200764248 >> 172.21.15.38:0/1200764248 conn(0x564f85dad4e0 msgr2=0x564f861a5820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:56.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.629+0000 7f4745ffd740 1 -- 172.21.15.38:0/1200764248 shutdown_connections 2024-09-21T03:35:56.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.629+0000 7f4745ffd740 1 -- 172.21.15.38:0/1200764248 wait complete. 2024-09-21T03:35:56.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.630+0000 7f4745ffd740 1 -- start start 2024-09-21T03:35:56.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.630+0000 7f4745ffd740 1 Processor -- start 2024-09-21T03:35:56.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.632+0000 7f4745ffd740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564f85d904d0 0x564f86223e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:56.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.632+0000 7f4745ffd740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564f85d986b0 0x564f861b9ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:56.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.632+0000 7f4745ffd740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x564f85db9390 0x564f861ba0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:56.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.632+0000 7f4745ffd740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f4738201090 con 0x564f85d904d0 2024-09-21T03:35:56.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.632+0000 7f4745ffd740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x564f86225ab0 con 0x564f85d986b0 2024-09-21T03:35:56.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.632+0000 7f4745ffd740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x564f8618a770 con 0x564f85db9390 2024-09-21T03:35:56.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.632+0000 7f4745ffd740 10 client.0 _ll_register_callbacks cb 0x7f47300053a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:35:56.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.632+0000 7f473d1e9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564f85d904d0 0x564f86223e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:56.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.632+0000 7f473d1e9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564f85d904d0 0x564f86223e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44472/0 (socket says 172.21.15.38:44472) 2024-09-21T03:35:56.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.632+0000 7f473d1e9640 1 -- 172.21.15.38:0/426817272 learned_addr learned my addr 172.21.15.38:0/426817272 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:35:56.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.632+0000 7f473c9e8640 1 --2- 172.21.15.38:0/426817272 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564f85d986b0 0x564f861b9ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:56.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.632+0000 7f473d9ea640 1 --2- 172.21.15.38:0/426817272 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x564f85db9390 0x564f861ba0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:56.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.632+0000 7f473d1e9640 1 -- 172.21.15.38:0/426817272 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x564f85db9390 msgr2=0x564f861ba0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:35:56.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.632+0000 7f473d1e9640 1 --2- 172.21.15.38:0/426817272 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x564f85db9390 0x564f861ba0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:56.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.632+0000 7f473d1e9640 1 -- 172.21.15.38:0/426817272 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564f85d986b0 msgr2=0x564f861b9ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:35:56.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.632+0000 7f473d1e9640 1 --2- 172.21.15.38:0/426817272 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564f85d986b0 0x564f861b9ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:56.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.632+0000 7f473d1e9640 1 -- 172.21.15.38:0/426817272 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f86224340 con 0x564f85d904d0 2024-09-21T03:35:56.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.632+0000 7f473c9e8640 1 --2- 172.21.15.38:0/426817272 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564f85d986b0 0x564f861b9ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:35:56.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.632+0000 7f473d9ea640 1 --2- 172.21.15.38:0/426817272 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x564f85db9390 0x564f861ba0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:35:56.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.632+0000 7f473d1e9640 1 --2- 172.21.15.38:0/426817272 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564f85d904d0 0x564f86223e00 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f4728063680 tx=0x7f4728009590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ad860e0bc1fe3b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:56.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.632+0000 7f4717fff640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-21T03:35:56.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.633+0000 7f4717fff640 1 -- 172.21.15.38:0/426817272 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4728082060 con 0x564f85d904d0 2024-09-21T03:35:56.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.633+0000 7f4717fff640 1 -- 172.21.15.38:0/426817272 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(55..55 src has 1..55) ==== 6101+0+0 (secure 0 0 0) 0x7f472806cbd0 con 0x564f85d904d0 2024-09-21T03:35:56.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.633+0000 7f4745ffd740 1 -- 172.21.15.38:0/426817272 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f472c007120 con 0x564f85d904d0 2024-09-21T03:35:56.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.633+0000 7f4717fff640 1 -- 172.21.15.38:0/426817272 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f47280636f0 con 0x564f85d904d0 2024-09-21T03:35:56.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.633+0000 7f4745ffd740 10 client.5745 fetch_fsmap learned FSMap version 30 2024-09-21T03:35:56.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.633+0000 7f4745ffd740 1 -- 172.21.15.38:0/426817272 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=2+}) -- 0x564f861b98a0 con 0x564f85d904d0 2024-09-21T03:35:56.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.633+0000 7f4717fff640 1 -- 172.21.15.38:0/426817272 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f472c007120 con 0x564f85d904d0 2024-09-21T03:35:56.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.633+0000 7f4717fff640 1 -- 172.21.15.38:0/426817272 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 30) ==== 3502+0+0 (secure 0 0 0) 0x7f472807db90 con 0x564f85d904d0 2024-09-21T03:35:56.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.633+0000 7f4745ffd740 10 client.5745 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:35:56.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.633+0000 7f4745ffd740 10 client.5745 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-09-21T03:35:56.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.633+0000 7f4745ffd740 20 client.5745 populate_metadata read hostname 'smithi038' 2024-09-21T03:35:56.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.633+0000 7f4745ffd740 1 --2- 172.21.15.38:0/426817272 >> [v2:172.21.15.38:6833/3184989541,v1:172.21.15.38:6835/3184989541] conn(0x564f861b79a0 0x564f861b7db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:56.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.633+0000 7f4745ffd740 4 client.5745 mds_command: new command op to 5577 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T03:35:56.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.633+0000 7f4745ffd740 1 -- 172.21.15.38:0/426817272 --> [v2:172.21.15.38:6833/3184989541,v1:172.21.15.38:6835/3184989541] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564f860d1460 con 0x564f861b79a0 2024-09-21T03:35:56.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.633+0000 7f473c9e8640 1 --2- 172.21.15.38:0/426817272 >> [v2:172.21.15.38:6833/3184989541,v1:172.21.15.38:6835/3184989541] conn(0x564f861b79a0 0x564f861b7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:56.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.634+0000 7f4717fff640 10 client.5745 ms_handle_connect on v2:172.21.15.38:6833/3184989541 2024-09-21T03:35:56.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.634+0000 7f473c9e8640 1 --2- 172.21.15.38:0/426817272 >> [v2:172.21.15.38:6833/3184989541,v1:172.21.15.38:6835/3184989541] conn(0x564f861b79a0 0x564f861b7db0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=750ca8d1d00e0303 server_cookie=cd1c7e7e5efaaef3 in_seq=0 out_seq=0 2024-09-21T03:35:56.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.649+0000 7f4717fff640 1 -- 172.21.15.38:0/426817272 <== mds.0 v2:172.21.15.38:6833/3184989541 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564f860d1460 con 0x564f861b79a0 2024-09-21T03:35:56.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.649+0000 7f4717fff640 10 client.5745 handle_command_reply: tid=0 2024-09-21T03:35:56.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.649+0000 7f4745ffd740 1 client.5745 shutdown 2024-09-21T03:35:56.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.649+0000 7f4745ffd740 1 -- 172.21.15.38:0/426817272 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564f85d904d0 msgr2=0x564f86223e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:56.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.649+0000 7f4745ffd740 1 --2- 172.21.15.38:0/426817272 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564f85d904d0 0x564f86223e00 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f4728063680 tx=0x7f4728009590 comp rx=0 tx=0).stop 2024-09-21T03:35:56.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.649+0000 7f4745ffd740 1 -- 172.21.15.38:0/426817272 shutdown_connections 2024-09-21T03:35:56.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.649+0000 7f4745ffd740 1 --2- 172.21.15.38:0/426817272 >> [v2:172.21.15.38:6833/3184989541,v1:172.21.15.38:6835/3184989541] conn(0x564f861b79a0 0x564f861b7db0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:56.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.649+0000 7f4745ffd740 1 --2- 172.21.15.38:0/426817272 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x564f85db9390 0x564f861ba0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:56.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.649+0000 7f4745ffd740 1 --2- 172.21.15.38:0/426817272 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564f85d986b0 0x564f861b9ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:56.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.649+0000 7f4717fff640 0 client.5745 ms_handle_reset on v2:172.21.15.38:6833/3184989541 2024-09-21T03:35:56.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.649+0000 7f4745ffd740 1 --2- 172.21.15.38:0/426817272 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564f85d904d0 0x564f86223e00 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:56.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.649+0000 7f4745ffd740 1 -- 172.21.15.38:0/426817272 >> 172.21.15.38:0/426817272 conn(0x564f85dad4e0 msgr2=0x564f86178580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:56.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.649+0000 7f4745ffd740 1 -- 172.21.15.38:0/426817272 shutdown_connections 2024-09-21T03:35:56.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.649+0000 7f4745ffd740 1 -- 172.21.15.38:0/426817272 wait complete. 2024-09-21T03:35:56.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.649+0000 7f4745ffd740 20 client.5745 trim_cache size 0 max 16384 2024-09-21T03:35:56.652 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-09-21T03:35:56.653 INFO:teuthology.orchestra.run.smithi038.stdout: "message": "", 2024-09-21T03:35:56.653 INFO:teuthology.orchestra.run.smithi038.stdout: "return_code": 0 2024-09-21T03:35:56.653 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-09-21T03:35:56.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.650+0000 7f473f44d640 1 -- 172.21.15.38:0/4077130929 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4724078b90 msgr2=0x7f472407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:56.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.650+0000 7f473f44d640 1 --2- 172.21.15.38:0/4077130929 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4724078b90 0x7f472407b0a0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f473809ea60 tx=0x7f472c009040 comp rx=0 tx=0).stop 2024-09-21T03:35:56.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.650+0000 7f473f44d640 1 -- 172.21.15.38:0/4077130929 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47380b9ad0 msgr2=0x7f47380a1e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:56.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.650+0000 7f473f44d640 1 --2- 172.21.15.38:0/4077130929 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47380b9ad0 0x7f47380a1e20 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f472800ece0 tx=0x7f472800c6a0 comp rx=0 tx=0).stop 2024-09-21T03:35:56.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.650+0000 7f473f44d640 1 -- 172.21.15.38:0/4077130929 shutdown_connections 2024-09-21T03:35:56.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.650+0000 7f473f44d640 1 --2- 172.21.15.38:0/4077130929 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f473815ee80 0x7f473809eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:56.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.650+0000 7f473f44d640 1 --2- 172.21.15.38:0/4077130929 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4724078b90 0x7f472407b0a0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:56.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.650+0000 7f473f44d640 1 --2- 172.21.15.38:0/4077130929 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f47380ceb90 0x7f4738158990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:56.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.650+0000 7f473f44d640 1 --2- 172.21.15.38:0/4077130929 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47380b9ad0 0x7f47380a1e20 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:56.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.650+0000 7f473f44d640 1 -- 172.21.15.38:0/4077130929 >> 172.21.15.38:0/4077130929 conn(0x7f473809d830 msgr2=0x7f47380d34f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:56.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.650+0000 7f473f44d640 1 -- 172.21.15.38:0/4077130929 shutdown_connections 2024-09-21T03:35:56.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.650+0000 7f473f44d640 1 -- 172.21.15.38:0/4077130929 wait complete. 2024-09-21T03:35:56.663 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_abort_on_newly_corrupt_dentry False 2024-09-21T03:35:56.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.781+0000 7f348215e640 1 -- 172.21.15.38:0/4177506228 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3464003a50 msgr2=0x7f3464003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:56.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.781+0000 7f348215e640 1 --2- 172.21.15.38:0/4177506228 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3464003a50 0x7f3464003ec0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f346c00c3f0 tx=0x7f3464005830 comp rx=0 tx=0).stop 2024-09-21T03:35:56.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.781+0000 7f348215e640 1 -- 172.21.15.38:0/4177506228 shutdown_connections 2024-09-21T03:35:56.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.781+0000 7f348215e640 1 --2- 172.21.15.38:0/4177506228 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3464003a50 0x7f3464003ec0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:56.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.781+0000 7f348215e640 1 -- 172.21.15.38:0/4177506228 >> 172.21.15.38:0/4177506228 conn(0x7f347c09d830 msgr2=0x7f347c0b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:56.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.782+0000 7f348215e640 1 -- 172.21.15.38:0/4177506228 shutdown_connections 2024-09-21T03:35:56.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.782+0000 7f348215e640 1 -- 172.21.15.38:0/4177506228 wait complete. 2024-09-21T03:35:56.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.783+0000 7f348215e640 1 Processor -- start 2024-09-21T03:35:56.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.783+0000 7f348215e640 1 -- start start 2024-09-21T03:35:56.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.785+0000 7f348215e640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3464003a50 0x7f347c1594b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:56.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.785+0000 7f348215e640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f347c14f990 0x7f347c0b2110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:56.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.785+0000 7f348215e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f347c14fda0 0x7f347c0b2650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:56.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.785+0000 7f348215e640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f347c0b9c50 con 0x7f347c14fda0 2024-09-21T03:35:56.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.785+0000 7f348215e640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f347c0b9ad0 con 0x7f3464003a50 2024-09-21T03:35:56.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.785+0000 7f348215e640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f347c0b9dd0 con 0x7f347c14f990 2024-09-21T03:35:56.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.785+0000 7f347bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f347c14fda0 0x7f347c0b2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:56.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.785+0000 7f347bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f347c14fda0 0x7f347c0b2650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44490/0 (socket says 172.21.15.38:44490) 2024-09-21T03:35:56.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.785+0000 7f347bfff640 1 -- 172.21.15.38:0/242412199 learned_addr learned my addr 172.21.15.38:0/242412199 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:35:56.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.785+0000 7f347b7fe640 1 --2- 172.21.15.38:0/242412199 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3464003a50 0x7f347c1594b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:56.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.785+0000 7f347affd640 1 --2- 172.21.15.38:0/242412199 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f347c14f990 0x7f347c0b2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:56.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.785+0000 7f347bfff640 1 -- 172.21.15.38:0/242412199 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f347c14f990 msgr2=0x7f347c0b2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:56.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.785+0000 7f347bfff640 1 --2- 172.21.15.38:0/242412199 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f347c14f990 0x7f347c0b2110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:56.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.785+0000 7f347bfff640 1 -- 172.21.15.38:0/242412199 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3464003a50 msgr2=0x7f347c1594b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:56.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.785+0000 7f347bfff640 1 --2- 172.21.15.38:0/242412199 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3464003a50 0x7f347c1594b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:56.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.785+0000 7f347bfff640 1 -- 172.21.15.38:0/242412199 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f347c157e60 con 0x7f347c14fda0 2024-09-21T03:35:56.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.785+0000 7f347bfff640 1 --2- 172.21.15.38:0/242412199 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f347c14fda0 0x7f347c0b2650 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f347c0b3c20 tx=0x7f34740019b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:56.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.786+0000 7f3463fff640 1 -- 172.21.15.38:0/242412199 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3474011870 con 0x7f347c14fda0 2024-09-21T03:35:56.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.786+0000 7f3463fff640 1 -- 172.21.15.38:0/242412199 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3474017070 con 0x7f347c14fda0 2024-09-21T03:35:56.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.786+0000 7f348215e640 1 -- 172.21.15.38:0/242412199 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f347c15a020 con 0x7f347c14fda0 2024-09-21T03:35:56.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.786+0000 7f3463fff640 1 -- 172.21.15.38:0/242412199 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3474011da0 con 0x7f347c14fda0 2024-09-21T03:35:56.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.786+0000 7f348215e640 1 -- 172.21.15.38:0/242412199 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f347c06e8a0 con 0x7f347c14fda0 2024-09-21T03:35:56.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.787+0000 7f348215e640 1 -- 172.21.15.38:0/242412199 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f347c159d90 con 0x7f347c14fda0 2024-09-21T03:35:56.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.788+0000 7f3463fff640 1 -- 172.21.15.38:0/242412199 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f347401c460 con 0x7f347c14fda0 2024-09-21T03:35:56.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.788+0000 7f3463fff640 1 --2- 172.21.15.38:0/242412199 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3468078b90 0x7f346807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:56.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.788+0000 7f3463fff640 1 -- 172.21.15.38:0/242412199 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 6101+0+0 (secure 0 0 0) 0x7f347409b460 con 0x7f347c14fda0 2024-09-21T03:35:56.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.788+0000 7f347b7fe640 1 --2- 172.21.15.38:0/242412199 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3468078b90 0x7f346807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:56.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.789+0000 7f347b7fe640 1 --2- 172.21.15.38:0/242412199 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3468078b90 0x7f346807b0a0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f346c033000 tx=0x7f346c000fe0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:56.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.793+0000 7f3463fff640 1 -- 172.21.15.38:0/242412199 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f34740631e0 con 0x7f347c14fda0 2024-09-21T03:35:56.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.979+0000 7f348215e640 1 -- 172.21.15.38:0/242412199 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command([{prefix=config set, name=mds_abort_on_newly_corrupt_dentry}] v 0) -- 0x7f347c0cec70 con 0x7f347c14fda0 2024-09-21T03:35:56.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.983+0000 7f3463fff640 1 -- 172.21.15.38:0/242412199 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_abort_on_newly_corrupt_dentry}]=0 v35)=0 v35) ==== 135+0+0 (secure 0 0 0) 0x7f3474067c10 con 0x7f347c14fda0 2024-09-21T03:35:56.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.985+0000 7f348215e640 1 -- 172.21.15.38:0/242412199 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3468078b90 msgr2=0x7f346807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:56.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.985+0000 7f348215e640 1 --2- 172.21.15.38:0/242412199 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3468078b90 0x7f346807b0a0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f346c033000 tx=0x7f346c000fe0 comp rx=0 tx=0).stop 2024-09-21T03:35:56.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.985+0000 7f348215e640 1 -- 172.21.15.38:0/242412199 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f347c14fda0 msgr2=0x7f347c0b2650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:56.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.985+0000 7f348215e640 1 --2- 172.21.15.38:0/242412199 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f347c14fda0 0x7f347c0b2650 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f347c0b3c20 tx=0x7f34740019b0 comp rx=0 tx=0).stop 2024-09-21T03:35:56.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.985+0000 7f348215e640 1 -- 172.21.15.38:0/242412199 shutdown_connections 2024-09-21T03:35:56.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.985+0000 7f348215e640 1 --2- 172.21.15.38:0/242412199 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f347c14fda0 0x7f347c0b2650 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:56.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.985+0000 7f348215e640 1 --2- 172.21.15.38:0/242412199 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3468078b90 0x7f346807b0a0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:56.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.985+0000 7f348215e640 1 --2- 172.21.15.38:0/242412199 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f347c14f990 0x7f347c0b2110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:56.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.985+0000 7f348215e640 1 --2- 172.21.15.38:0/242412199 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3464003a50 0x7f347c1594b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:56.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.985+0000 7f348215e640 1 -- 172.21.15.38:0/242412199 >> 172.21.15.38:0/242412199 conn(0x7f347c09d830 msgr2=0x7f347c160b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:56.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.985+0000 7f348215e640 1 -- 172.21.15.38:0/242412199 shutdown_connections 2024-09-21T03:35:56.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:56.985+0000 7f348215e640 1 -- 172.21.15.38:0/242412199 wait complete. 2024-09-21T03:35:56.998 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_inject_rename_corrupt_dentry_first 1.0 2024-09-21T03:35:57.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:57.118+0000 7f06ac1a1640 1 -- 172.21.15.38:0/3380054110 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0690004d40 msgr2=0x7f0690005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:57.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:57.118+0000 7f06ac1a1640 1 --2- 172.21.15.38:0/3380054110 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0690004d40 0x7f0690005190 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f069c001e10 tx=0x7f0690005830 comp rx=0 tx=0).stop 2024-09-21T03:35:57.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:57.118+0000 7f06aa73e640 1 -- 172.21.15.38:0/3380054110 reap_dead start 2024-09-21T03:35:57.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:57.118+0000 7f06ac1a1640 1 -- 172.21.15.38:0/3380054110 shutdown_connections 2024-09-21T03:35:57.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:57.118+0000 7f06ac1a1640 1 -- 172.21.15.38:0/3380054110 >> 172.21.15.38:0/3380054110 conn(0x7f06a409d830 msgr2=0x7f06a40b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:57.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:57.118+0000 7f06ac1a1640 1 -- 172.21.15.38:0/3380054110 shutdown_connections 2024-09-21T03:35:57.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:57.118+0000 7f06ac1a1640 1 -- 172.21.15.38:0/3380054110 wait complete. 2024-09-21T03:35:57.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:57.120+0000 7f06ac1a1640 1 Processor -- start 2024-09-21T03:35:57.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:57.120+0000 7f06ac1a1640 1 -- start start 2024-09-21T03:35:57.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:57.121+0000 7f06ac1a1640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0690004d40 0x7f06a40ca7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:57.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:57.121+0000 7f06ac1a1640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f06a406f680 0x7f06a4155100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:57.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:57.121+0000 7f06ac1a1640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f06a4155640 0x7f06a4155a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:57.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:57.121+0000 7f06ac1a1640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f06a40d0500 con 0x7f0690004d40 2024-09-21T03:35:57.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:57.121+0000 7f06ac1a1640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f06a40d0380 con 0x7f06a406f680 2024-09-21T03:35:57.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:57.121+0000 7f06ac1a1640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f06a40ba0f0 con 0x7f06a4155640 2024-09-21T03:35:57.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:57.121+0000 7f06a9f3d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0690004d40 0x7f06a40ca7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:57.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:57.121+0000 7f06a9f3d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0690004d40 0x7f06a40ca7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44512/0 (socket says 172.21.15.38:44512) 2024-09-21T03:35:57.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:57.121+0000 7f06a9f3d640 1 -- 172.21.15.38:0/573685357 learned_addr learned my addr 172.21.15.38:0/573685357 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:35:57.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:57.121+0000 7f06aa73e640 1 --2- 172.21.15.38:0/573685357 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f06a4155640 0x7f06a4155a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:57.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:57.121+0000 7f06a973c640 1 --2- 172.21.15.38:0/573685357 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f06a406f680 0x7f06a4155100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:57.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:57.121+0000 7f06a9f3d640 1 -- 172.21.15.38:0/573685357 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f06a4155640 msgr2=0x7f06a4155a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:57.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:57.121+0000 7f06a9f3d640 1 --2- 172.21.15.38:0/573685357 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f06a4155640 0x7f06a4155a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:57.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:57.121+0000 7f06a9f3d640 1 -- 172.21.15.38:0/573685357 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f06a406f680 msgr2=0x7f06a4155100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:57.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:57.122+0000 7f06a9f3d640 1 --2- 172.21.15.38:0/573685357 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f06a406f680 0x7f06a4155100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:57.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:57.122+0000 7f06a9f3d640 1 -- 172.21.15.38:0/573685357 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06a40b38b0 con 0x7f0690004d40 2024-09-21T03:35:57.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:57.122+0000 7f06a973c640 1 --2- 172.21.15.38:0/573685357 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f06a406f680 0x7f06a4155100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:35:57.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:57.122+0000 7f06a9f3d640 1 --2- 172.21.15.38:0/573685357 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0690004d40 0x7f06a40ca7c0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f06a0011ae0 tx=0x7f06a0009eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:57.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:57.122+0000 7f069a7fc640 1 -- 172.21.15.38:0/573685357 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06a001e070 con 0x7f0690004d40 2024-09-21T03:35:57.126 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:57.122+0000 7f069a7fc640 1 -- 172.21.15.38:0/573685357 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f06a000fe70 con 0x7f0690004d40 2024-09-21T03:35:57.126 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:57.122+0000 7f06ac1a1640 1 -- 172.21.15.38:0/573685357 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06a41f57b0 con 0x7f0690004d40 2024-09-21T03:35:57.126 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:57.122+0000 7f069a7fc640 1 -- 172.21.15.38:0/573685357 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06a0004250 con 0x7f0690004d40 2024-09-21T03:35:57.126 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:57.122+0000 7f06ac1a1640 1 -- 172.21.15.38:0/573685357 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06a41f5d70 con 0x7f0690004d40 2024-09-21T03:35:57.126 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:57.123+0000 7f06ac1a1640 1 -- 172.21.15.38:0/573685357 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06a40d0be0 con 0x7f0690004d40 2024-09-21T03:35:57.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:57.124+0000 7f069a7fc640 1 -- 172.21.15.38:0/573685357 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f06a0021070 con 0x7f0690004d40 2024-09-21T03:35:57.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:57.124+0000 7f069a7fc640 1 --2- 172.21.15.38:0/573685357 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0688078bc0 0x7f068807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:57.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:57.125+0000 7f069a7fc640 1 -- 172.21.15.38:0/573685357 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 6101+0+0 (secure 0 0 0) 0x7f06a009b8e0 con 0x7f0690004d40 2024-09-21T03:35:57.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:57.125+0000 7f06a973c640 1 --2- 172.21.15.38:0/573685357 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0688078bc0 0x7f068807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:57.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:57.125+0000 7f06a973c640 1 --2- 172.21.15.38:0/573685357 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0688078bc0 0x7f068807b0d0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f06a40cb840 tx=0x7f0694000fa0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:57.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:57.129+0000 7f069a7fc640 1 -- 172.21.15.38:0/573685357 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f06a00635e0 con 0x7f0690004d40 2024-09-21T03:35:57.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:57.316+0000 7f06ac1a1640 1 -- 172.21.15.38:0/573685357 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command([{prefix=config set, name=mds_inject_rename_corrupt_dentry_first}] v 0) -- 0x7f06a40d43b0 con 0x7f0690004d40 2024-09-21T03:35:57.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:57.324+0000 7f069a7fc640 1 -- 172.21.15.38:0/573685357 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_inject_rename_corrupt_dentry_first}]=0 v36)=0 v36) ==== 138+0+0 (secure 0 0 0) 0x7f06a0068010 con 0x7f0690004d40 2024-09-21T03:35:57.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:57.326+0000 7f06ac1a1640 1 -- 172.21.15.38:0/573685357 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0688078bc0 msgr2=0x7f068807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:57.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:57.326+0000 7f06ac1a1640 1 --2- 172.21.15.38:0/573685357 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0688078bc0 0x7f068807b0d0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f06a40cb840 tx=0x7f0694000fa0 comp rx=0 tx=0).stop 2024-09-21T03:35:57.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:57.326+0000 7f06ac1a1640 1 -- 172.21.15.38:0/573685357 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0690004d40 msgr2=0x7f06a40ca7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:57.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:57.326+0000 7f06ac1a1640 1 --2- 172.21.15.38:0/573685357 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0690004d40 0x7f06a40ca7c0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f06a0011ae0 tx=0x7f06a0009eb0 comp rx=0 tx=0).stop 2024-09-21T03:35:57.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:57.326+0000 7f06ac1a1640 1 -- 172.21.15.38:0/573685357 shutdown_connections 2024-09-21T03:35:57.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:57.326+0000 7f06ac1a1640 1 --2- 172.21.15.38:0/573685357 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0688078bc0 0x7f068807b0d0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:57.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:57.326+0000 7f06ac1a1640 1 --2- 172.21.15.38:0/573685357 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f06a4155640 0x7f06a4155a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:57.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:57.326+0000 7f06ac1a1640 1 --2- 172.21.15.38:0/573685357 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f06a406f680 0x7f06a4155100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:57.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:57.326+0000 7f06ac1a1640 1 --2- 172.21.15.38:0/573685357 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0690004d40 0x7f06a40ca7c0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:57.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:57.326+0000 7f06ac1a1640 1 -- 172.21.15.38:0/573685357 >> 172.21.15.38:0/573685357 conn(0x7f06a409d830 msgr2=0x7f06a4158930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:57.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:57.326+0000 7f06ac1a1640 1 -- 172.21.15.38:0/573685357 shutdown_connections 2024-09-21T03:35:57.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:35:57.326+0000 7f06ac1a1640 1 -- 172.21.15.38:0/573685357 wait complete. 2024-09-21T03:36:00.170 DEBUG:teuthology.orchestra.run.smithi038:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T03:36:00.173 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T03:36:02.340 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'mv a/b a/c; sync .') 2024-09-21T03:36:02.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:02 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T03:36:02.403 INFO:tasks.ceph.mds.c.smithi038.stderr:2024-09-21T03:36:02.401+0000 7f6321a79640 -1 mds.0.cache.den(0x10000000000 c) newly corrupt dentry to be committed: [dentry #0x1/a/c [fffffffffffffff6,head] auth NULL (dn xlock x=1 by request(client.5246:44 nref=7 cr=0x561425a23c00)) (dversion lock w=1 last_client=5246) pv=4 v=3 ap=2 ino=(nil) state=1073741824 | request=1 lock=2 authpin=1 0x5614258f7680] with next_snap: 2 2024-09-21T03:36:02.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:36:02.411 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-21T03:36:02.411 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:36:02.429 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T03:36:02.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:02.590+0000 7f9437d89640 1 -- 172.21.15.38:0/674270899 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f943015ca90 msgr2=0x7f94300d0670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:02.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:02.590+0000 7f9437d89640 1 --2- 172.21.15.38:0/674270899 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f943015ca90 0x7f94300d0670 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f94300cd050 tx=0x7f942002a560 comp rx=0 tx=0).stop 2024-09-21T03:36:02.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:02.590+0000 7f9437d89640 1 -- 172.21.15.38:0/674270899 shutdown_connections 2024-09-21T03:36:02.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:02.590+0000 7f9437d89640 1 --2- 172.21.15.38:0/674270899 >> v2:172.21.15.195:3300/0 conn(0x7f94300d0bb0 0x7f94300b5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:02.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:02.590+0000 7f9437d89640 1 --2- 172.21.15.38:0/674270899 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f943015ca90 0x7f94300d0670 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:02.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:02.590+0000 7f9437d89640 1 -- 172.21.15.38:0/674270899 >> 172.21.15.38:0/674270899 conn(0x7f94300b46e0 msgr2=0x7f94300d23e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:02.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:02.590+0000 7f9437d89640 1 -- 172.21.15.38:0/674270899 shutdown_connections 2024-09-21T03:36:02.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:02.590+0000 7f9437d89640 1 -- 172.21.15.38:0/674270899 wait complete. 2024-09-21T03:36:02.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:02.592+0000 7f9437d89640 1 Processor -- start 2024-09-21T03:36:02.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:02.592+0000 7f9437d89640 1 -- start start 2024-09-21T03:36:02.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:02.593+0000 7f9437d89640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f94300ccc40 0x7f9430152780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:02.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:02.593+0000 7f9437d89640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f94300d0bb0 0x7f94300c9490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:02.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:02.593+0000 7f9437d89640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f943015ca90 0x7f94300c99d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:02.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:02.593+0000 7f9437d89640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f94300b6620 con 0x7f943015ca90 2024-09-21T03:36:02.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:02.593+0000 7f9437d89640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f94300b64a0 con 0x7f94300d0bb0 2024-09-21T03:36:02.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:02.593+0000 7f9437d89640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f94300b67a0 con 0x7f94300ccc40 2024-09-21T03:36:02.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:02.593+0000 7f9436326640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f943015ca90 0x7f94300c99d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:02.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:02.593+0000 7f9436326640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f943015ca90 0x7f94300c99d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37248/0 (socket says 172.21.15.38:37248) 2024-09-21T03:36:02.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:02.593+0000 7f9436326640 1 -- 172.21.15.38:0/2426708903 learned_addr learned my addr 172.21.15.38:0/2426708903 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:36:02.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:02.594+0000 7f9435324640 1 --2- 172.21.15.38:0/2426708903 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f94300d0bb0 0x7f94300c9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:02.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:02.594+0000 7f9435b25640 1 --2- 172.21.15.38:0/2426708903 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f94300ccc40 0x7f9430152780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:02.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:02.594+0000 7f9436326640 1 -- 172.21.15.38:0/2426708903 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f94300ccc40 msgr2=0x7f9430152780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:02.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:02.594+0000 7f9436326640 1 --2- 172.21.15.38:0/2426708903 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f94300ccc40 0x7f9430152780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:02.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:02.594+0000 7f9436326640 1 -- 172.21.15.38:0/2426708903 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f94300d0bb0 msgr2=0x7f94300c9490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:02.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:02.594+0000 7f9436326640 1 --2- 172.21.15.38:0/2426708903 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f94300d0bb0 0x7f94300c9490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:02.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:02.594+0000 7f9436326640 1 -- 172.21.15.38:0/2426708903 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94300c9f10 con 0x7f943015ca90 2024-09-21T03:36:02.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:02.594+0000 7f9435324640 1 --2- 172.21.15.38:0/2426708903 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f94300d0bb0 0x7f94300c9490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:36:02.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:02.594+0000 7f9436326640 1 --2- 172.21.15.38:0/2426708903 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f943015ca90 0x7f94300c99d0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f942800e990 tx=0x7f942800ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:02.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:02.594+0000 7f94267fc640 1 -- 172.21.15.38:0/2426708903 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9428004d20 con 0x7f943015ca90 2024-09-21T03:36:02.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:02.594+0000 7f94267fc640 1 -- 172.21.15.38:0/2426708903 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f942800caa0 con 0x7f943015ca90 2024-09-21T03:36:02.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:02.594+0000 7f9437d89640 1 -- 172.21.15.38:0/2426708903 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94301f56a0 con 0x7f943015ca90 2024-09-21T03:36:02.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:02.594+0000 7f94267fc640 1 -- 172.21.15.38:0/2426708903 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9428005020 con 0x7f943015ca90 2024-09-21T03:36:02.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:02.594+0000 7f9437d89640 1 -- 172.21.15.38:0/2426708903 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94301f5c60 con 0x7f943015ca90 2024-09-21T03:36:02.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:02.595+0000 7f9437d89640 1 -- 172.21.15.38:0/2426708903 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94301f5410 con 0x7f943015ca90 2024-09-21T03:36:02.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:02.596+0000 7f94267fc640 1 -- 172.21.15.38:0/2426708903 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f94280051c0 con 0x7f943015ca90 2024-09-21T03:36:02.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:02.597+0000 7f94267fc640 1 --2- 172.21.15.38:0/2426708903 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9414078b90 0x7f941407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:02.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:02.597+0000 7f94267fc640 1 -- 172.21.15.38:0/2426708903 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 6101+0+0 (secure 0 0 0) 0x7f942809b7f0 con 0x7f943015ca90 2024-09-21T03:36:02.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:02.597+0000 7f9435b25640 1 --2- 172.21.15.38:0/2426708903 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9414078b90 0x7f941407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:02.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:02.598+0000 7f9435b25640 1 --2- 172.21.15.38:0/2426708903 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9414078b90 0x7f941407b0a0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f942c0015e0 tx=0x7f942c009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:02.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:02.602+0000 7f94267fc640 1 -- 172.21.15.38:0/2426708903 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f9428016030 con 0x7f943015ca90 2024-09-21T03:36:02.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:02.799+0000 7f9437d89640 1 -- 172.21.15.38:0/2426708903 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f94300b6d30 con 0x7f943015ca90 2024-09-21T03:36:02.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:02.799+0000 7f94267fc640 1 -- 172.21.15.38:0/2426708903 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v55) ==== 80+0+0 (secure 0 0 0) 0x7f9428063aa0 con 0x7f943015ca90 2024-09-21T03:36:02.801 INFO:teuthology.orchestra.run.smithi038.stderr:listed 0 entries 2024-09-21T03:36:02.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:02.801+0000 7f9437d89640 1 -- 172.21.15.38:0/2426708903 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9414078b90 msgr2=0x7f941407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:02.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:02.801+0000 7f9437d89640 1 --2- 172.21.15.38:0/2426708903 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9414078b90 0x7f941407b0a0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f942c0015e0 tx=0x7f942c009040 comp rx=0 tx=0).stop 2024-09-21T03:36:02.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:02.801+0000 7f9437d89640 1 -- 172.21.15.38:0/2426708903 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f943015ca90 msgr2=0x7f94300c99d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:02.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:02.802+0000 7f9437d89640 1 --2- 172.21.15.38:0/2426708903 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f943015ca90 0x7f94300c99d0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f942800e990 tx=0x7f942800ee50 comp rx=0 tx=0).stop 2024-09-21T03:36:02.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:02.802+0000 7f9437d89640 1 -- 172.21.15.38:0/2426708903 shutdown_connections 2024-09-21T03:36:02.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:02.802+0000 7f9437d89640 1 --2- 172.21.15.38:0/2426708903 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f943015ca90 0x7f94300c99d0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:02.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:02.802+0000 7f9437d89640 1 --2- 172.21.15.38:0/2426708903 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f94300d0bb0 0x7f94300c9490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:02.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:02.802+0000 7f9437d89640 1 --2- 172.21.15.38:0/2426708903 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9414078b90 0x7f941407b0a0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:02.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:02.802+0000 7f9437d89640 1 --2- 172.21.15.38:0/2426708903 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f94300ccc40 0x7f9430152780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:02.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:02.802+0000 7f9437d89640 1 -- 172.21.15.38:0/2426708903 >> 172.21.15.38:0/2426708903 conn(0x7f94300b46e0 msgr2=0x7f94300cd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:02.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:02.802+0000 7f9437d89640 1 -- 172.21.15.38:0/2426708903 shutdown_connections 2024-09-21T03:36:02.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:02.802+0000 7f9437d89640 1 -- 172.21.15.38:0/2426708903 wait complete. 2024-09-21T03:36:02.815 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-21T03:36:02.815 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:36:02.815 DEBUG:teuthology.orchestra.run.smithi038:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-21T03:36:02.846 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi038.front.sepia.ceph.com 2024-09-21T03:36:02.846 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:36:02.846 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T03:36:02.903 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:36:03.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:03.058+0000 7ff7b9c30640 1 -- 172.21.15.38:0/3933803332 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff7b40b2e40 msgr2=0x7ff7b40b3250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:03.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:03.058+0000 7ff7b9c30640 1 --2- 172.21.15.38:0/3933803332 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff7b40b2e40 0x7ff7b40b3250 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7ff7ac005fd0 tx=0x7ff7ac02aa90 comp rx=0 tx=0).stop 2024-09-21T03:36:03.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:03.058+0000 7ff7b9c30640 1 -- 172.21.15.38:0/3933803332 shutdown_connections 2024-09-21T03:36:03.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:03.058+0000 7ff7b9c30640 1 --2- 172.21.15.38:0/3933803332 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff7b40b2e40 0x7ff7b40b3250 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:03.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:03.058+0000 7ff7b9c30640 1 --2- 172.21.15.38:0/3933803332 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff7b4162f00 0x7ff7b40d24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:03.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:03.058+0000 7ff7b9c30640 1 -- 172.21.15.38:0/3933803332 >> 172.21.15.38:0/3933803332 conn(0x7ff7b40b6b20 msgr2=0x7ff7b40d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:03.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:03.058+0000 7ff7b9c30640 1 -- 172.21.15.38:0/3933803332 shutdown_connections 2024-09-21T03:36:03.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:03.058+0000 7ff7b9c30640 1 -- 172.21.15.38:0/3933803332 wait complete. 2024-09-21T03:36:03.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:03.060+0000 7ff7b9c30640 1 Processor -- start 2024-09-21T03:36:03.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:03.060+0000 7ff7b9c30640 1 -- start start 2024-09-21T03:36:03.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:03.061+0000 7ff7b9c30640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff7b40b2e40 0x7ff7b415b040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:03.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:03.061+0000 7ff7b9c30640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff7b4162280 0x7ff7b4151830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:03.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:03.061+0000 7ff7b9c30640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff7b41f8820 0x7ff7b41f8c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:03.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:03.061+0000 7ff7b9c30640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ff7b415e4b0 con 0x7ff7b40b2e40 2024-09-21T03:36:03.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:03.061+0000 7ff7b9c30640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ff7b40b39a0 con 0x7ff7b4162280 2024-09-21T03:36:03.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:03.062+0000 7ff7b9c30640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7ff7b415e630 con 0x7ff7b41f8820 2024-09-21T03:36:03.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:03.062+0000 7ff7b37fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff7b40b2e40 0x7ff7b415b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:03.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:03.062+0000 7ff7b37fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff7b40b2e40 0x7ff7b415b040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37272/0 (socket says 172.21.15.38:37272) 2024-09-21T03:36:03.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:03.062+0000 7ff7b37fe640 1 -- 172.21.15.38:0/81754948 learned_addr learned my addr 172.21.15.38:0/81754948 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:36:03.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:03.062+0000 7ff7b3fff640 1 --2- 172.21.15.38:0/81754948 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff7b41f8820 0x7ff7b41f8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:03.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:03.062+0000 7ff7b2ffd640 1 --2- 172.21.15.38:0/81754948 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff7b4162280 0x7ff7b4151830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:03.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:03.062+0000 7ff7b37fe640 1 -- 172.21.15.38:0/81754948 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff7b41f8820 msgr2=0x7ff7b41f8c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:03.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:03.062+0000 7ff7b37fe640 1 --2- 172.21.15.38:0/81754948 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff7b41f8820 0x7ff7b41f8c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:03.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:03.062+0000 7ff7b37fe640 1 -- 172.21.15.38:0/81754948 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff7b4162280 msgr2=0x7ff7b4151830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:03.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:03.062+0000 7ff7b37fe640 1 --2- 172.21.15.38:0/81754948 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff7b4162280 0x7ff7b4151830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:03.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:03.062+0000 7ff7b37fe640 1 -- 172.21.15.38:0/81754948 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7b40ce0f0 con 0x7ff7b40b2e40 2024-09-21T03:36:03.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:03.062+0000 7ff7b37fe640 1 --2- 172.21.15.38:0/81754948 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff7b40b2e40 0x7ff7b415b040 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7ff7a400c9a0 tx=0x7ff7a400ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:03.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:03.062+0000 7ff79bfff640 1 -- 172.21.15.38:0/81754948 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7a4007bb0 con 0x7ff7b40b2e40 2024-09-21T03:36:03.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:03.062+0000 7ff79bfff640 1 -- 172.21.15.38:0/81754948 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff7a4007e40 con 0x7ff7b40b2e40 2024-09-21T03:36:03.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:03.062+0000 7ff79bfff640 1 -- 172.21.15.38:0/81754948 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7a4005280 con 0x7ff7b40b2e40 2024-09-21T03:36:03.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:03.062+0000 7ff7b9c30640 1 -- 172.21.15.38:0/81754948 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7b40ce3d0 con 0x7ff7b40b2e40 2024-09-21T03:36:03.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:03.063+0000 7ff7b9c30640 1 -- 172.21.15.38:0/81754948 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7b40ce990 con 0x7ff7b40b2e40 2024-09-21T03:36:03.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:03.063+0000 7ff7b9c30640 1 -- 172.21.15.38:0/81754948 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7b4162f00 con 0x7ff7b40b2e40 2024-09-21T03:36:03.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:03.064+0000 7ff79bfff640 1 -- 172.21.15.38:0/81754948 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ff7a4005420 con 0x7ff7b40b2e40 2024-09-21T03:36:03.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:03.065+0000 7ff79bfff640 1 --2- 172.21.15.38:0/81754948 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff7a0078bc0 0x7ff7a007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:03.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:03.065+0000 7ff79bfff640 1 -- 172.21.15.38:0/81754948 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 6101+0+0 (secure 0 0 0) 0x7ff7a409b280 con 0x7ff7b40b2e40 2024-09-21T03:36:03.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:03.065+0000 7ff7b2ffd640 1 --2- 172.21.15.38:0/81754948 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff7a0078bc0 0x7ff7a007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:03.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:03.066+0000 7ff7b2ffd640 1 --2- 172.21.15.38:0/81754948 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff7a0078bc0 0x7ff7a007b0d0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ff7a8000f60 tx=0x7ff7a8009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:03.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:03.069+0000 7ff79bfff640 1 -- 172.21.15.38:0/81754948 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7ff7a4014030 con 0x7ff7b40b2e40 2024-09-21T03:36:03.301 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:03.299+0000 7ff7b9c30640 1 -- 172.21.15.38:0/81754948 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff7b40b7950 con 0x7ff7b40b2e40 2024-09-21T03:36:03.301 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:03.300+0000 7ff79bfff640 1 -- 172.21.15.38:0/81754948 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+5229 (secure 0 0 0) 0x7ff7a4062f90 con 0x7ff7b40b2e40 2024-09-21T03:36:03.302 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:36:03.302 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":30,"btime":"2024-09-21T03:35:47:913013+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5147,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/2858412699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":2858412699},{"type":"v1","addr":"172.21.15.195:6837","nonce":2858412699}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":5368,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/2907144575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":2907144575},{"type":"v1","addr":"172.21.15.195:6836","nonce":2907144575}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":5574,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6834/1911661184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":1911661184},{"type":"v1","addr":"172.21.15.38:6834","nonce":1911661184}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:35:44.271168+0000","modified":"2024-09-21T03:35:47.294703+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5577},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5577":{"gid":5577,"name":"c","rank":0,"incarnation":27,"state":"up:active","state_seq":3,"addr":"172.21.15.38:6835/3184989541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6833","nonce":3184989541},{"type":"v1","addr":"172.21.15.38:6835","nonce":3184989541}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5577,"qdb_cluster":[5577]},"id":3}]} 2024-09-21T03:36:03.302 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 30 2024-09-21T03:36:03.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:03.302+0000 7ff7b9c30640 1 -- 172.21.15.38:0/81754948 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff7a0078bc0 msgr2=0x7ff7a007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:03.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:03.302+0000 7ff7b9c30640 1 --2- 172.21.15.38:0/81754948 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff7a0078bc0 0x7ff7a007b0d0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ff7a8000f60 tx=0x7ff7a8009040 comp rx=0 tx=0).stop 2024-09-21T03:36:03.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:03.302+0000 7ff7b9c30640 1 -- 172.21.15.38:0/81754948 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff7b40b2e40 msgr2=0x7ff7b415b040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:03.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:03.302+0000 7ff7b9c30640 1 --2- 172.21.15.38:0/81754948 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff7b40b2e40 0x7ff7b415b040 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7ff7a400c9a0 tx=0x7ff7a400ce60 comp rx=0 tx=0).stop 2024-09-21T03:36:03.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:03.302+0000 7ff7b9c30640 1 -- 172.21.15.38:0/81754948 shutdown_connections 2024-09-21T03:36:03.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:03.302+0000 7ff7b9c30640 1 --2- 172.21.15.38:0/81754948 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff7a0078bc0 0x7ff7a007b0d0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:03.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:03.302+0000 7ff7b9c30640 1 --2- 172.21.15.38:0/81754948 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff7b41f8820 0x7ff7b41f8c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:03.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:03.302+0000 7ff7b9c30640 1 --2- 172.21.15.38:0/81754948 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff7b4162280 0x7ff7b4151830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:03.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:03.302+0000 7ff7b9c30640 1 --2- 172.21.15.38:0/81754948 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff7b40b2e40 0x7ff7b415b040 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:03.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:03.302+0000 7ff7b9c30640 1 -- 172.21.15.38:0/81754948 >> 172.21.15.38:0/81754948 conn(0x7ff7b40b6b20 msgr2=0x7ff7b40d0c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:03.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:03.302+0000 7ff7b9c30640 1 -- 172.21.15.38:0/81754948 shutdown_connections 2024-09-21T03:36:03.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:03.302+0000 7ff7b9c30640 1 -- 172.21.15.38:0/81754948 wait complete. 2024-09-21T03:36:03.316 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:36:03.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:03.430+0000 7f8c6f550640 1 -- 172.21.15.38:0/27341153 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8c680d0b40 msgr2=0x7f8c6806e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:03.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:03.430+0000 7f8c6f550640 1 --2- 172.21.15.38:0/27341153 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8c680d0b40 0x7f8c6806e6f0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f8c60005fd0 tx=0x7f8c6002a5d0 comp rx=0 tx=0).stop 2024-09-21T03:36:03.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:03.430+0000 7f8c6f550640 1 -- 172.21.15.38:0/27341153 shutdown_connections 2024-09-21T03:36:03.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:03.430+0000 7f8c6f550640 1 --2- 172.21.15.38:0/27341153 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8c6806ec30 0x7f8c680b3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:03.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:03.430+0000 7f8c6f550640 1 --2- 172.21.15.38:0/27341153 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8c680d0b40 0x7f8c6806e6f0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:03.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:03.430+0000 7f8c6f550640 1 -- 172.21.15.38:0/27341153 >> 172.21.15.38:0/27341153 conn(0x7f8c680b2cc0 msgr2=0x7f8c680b30d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:03.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:03.430+0000 7f8c6f550640 1 -- 172.21.15.38:0/27341153 shutdown_connections 2024-09-21T03:36:03.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:03.430+0000 7f8c6f550640 1 -- 172.21.15.38:0/27341153 wait complete. 2024-09-21T03:36:03.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:03.432+0000 7f8c6f550640 1 Processor -- start 2024-09-21T03:36:03.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:03.432+0000 7f8c6f550640 1 -- start start 2024-09-21T03:36:03.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:03.433+0000 7f8c6f550640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8c6806ec30 0x7f8c681482c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:03.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:03.433+0000 7f8c6f550640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8c680b4500 0x7f8c68148800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:03.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:03.433+0000 7f8c6f550640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8c680d0b40 0x7f8c6814e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:03.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:03.433+0000 7f8c6f550640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f8c6815b750 con 0x7f8c6806ec30 2024-09-21T03:36:03.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:03.433+0000 7f8c6f550640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f8c6815b5d0 con 0x7f8c680d0b40 2024-09-21T03:36:03.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:03.433+0000 7f8c6f550640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f8c6815b8d0 con 0x7f8c680b4500 2024-09-21T03:36:03.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:03.434+0000 7f8c6d2ec640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8c6806ec30 0x7f8c681482c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:03.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:03.434+0000 7f8c6d2ec640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8c6806ec30 0x7f8c681482c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37290/0 (socket says 172.21.15.38:37290) 2024-09-21T03:36:03.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:03.434+0000 7f8c6d2ec640 1 -- 172.21.15.38:0/3772201728 learned_addr learned my addr 172.21.15.38:0/3772201728 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:36:03.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:03.434+0000 7f8c6caeb640 1 --2- 172.21.15.38:0/3772201728 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8c680b4500 0x7f8c68148800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:03.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:03.434+0000 7f8c6daed640 1 --2- 172.21.15.38:0/3772201728 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8c680d0b40 0x7f8c6814e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:03.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:03.434+0000 7f8c6d2ec640 1 -- 172.21.15.38:0/3772201728 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8c680b4500 msgr2=0x7f8c68148800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:03.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:03.434+0000 7f8c6d2ec640 1 --2- 172.21.15.38:0/3772201728 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8c680b4500 0x7f8c68148800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:03.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:03.434+0000 7f8c6d2ec640 1 -- 172.21.15.38:0/3772201728 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8c680d0b40 msgr2=0x7f8c6814e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:03.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:03.434+0000 7f8c6d2ec640 1 --2- 172.21.15.38:0/3772201728 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8c680d0b40 0x7f8c6814e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:03.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:03.434+0000 7f8c6d2ec640 1 -- 172.21.15.38:0/3772201728 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c681564f0 con 0x7f8c6806ec30 2024-09-21T03:36:03.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:03.434+0000 7f8c6d2ec640 1 --2- 172.21.15.38:0/3772201728 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8c6806ec30 0x7f8c681482c0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f8c60034040 tx=0x7f8c600038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:03.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:03.434+0000 7f8c65ffb640 1 -- 172.21.15.38:0/3772201728 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c6003e030 con 0x7f8c6806ec30 2024-09-21T03:36:03.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:03.434+0000 7f8c65ffb640 1 -- 172.21.15.38:0/3772201728 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8c60003ca0 con 0x7f8c6806ec30 2024-09-21T03:36:03.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:03.434+0000 7f8c6f550640 1 -- 172.21.15.38:0/3772201728 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c681567d0 con 0x7f8c6806ec30 2024-09-21T03:36:03.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:03.434+0000 7f8c65ffb640 1 -- 172.21.15.38:0/3772201728 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c60035040 con 0x7f8c6806ec30 2024-09-21T03:36:03.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:03.435+0000 7f8c6f550640 1 -- 172.21.15.38:0/3772201728 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c68149350 con 0x7f8c6806ec30 2024-09-21T03:36:03.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:03.436+0000 7f8c6f550640 1 -- 172.21.15.38:0/3772201728 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c680b3fc0 con 0x7f8c6806ec30 2024-09-21T03:36:03.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:03.437+0000 7f8c65ffb640 1 -- 172.21.15.38:0/3772201728 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f8c60041030 con 0x7f8c6806ec30 2024-09-21T03:36:03.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:03.438+0000 7f8c65ffb640 1 --2- 172.21.15.38:0/3772201728 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8c54078b90 0x7f8c5407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:03.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:03.438+0000 7f8c65ffb640 1 -- 172.21.15.38:0/3772201728 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 6101+0+0 (secure 0 0 0) 0x7f8c600bba50 con 0x7f8c6806ec30 2024-09-21T03:36:03.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:03.438+0000 7f8c6caeb640 1 --2- 172.21.15.38:0/3772201728 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8c54078b90 0x7f8c5407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:03.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:03.439+0000 7f8c6caeb640 1 --2- 172.21.15.38:0/3772201728 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8c54078b90 0x7f8c5407b0a0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f8c6806de90 tx=0x7f8c5c00c440 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:03.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:03.441+0000 7f8c65ffb640 1 -- 172.21.15.38:0/3772201728 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f8c60083ce0 con 0x7f8c6806ec30 2024-09-21T03:36:03.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:03.671+0000 7f8c6f550640 1 -- 172.21.15.38:0/3772201728 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8c680cfcd0 con 0x7f8c6806ec30 2024-09-21T03:36:03.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:03.672+0000 7f8c65ffb640 1 -- 172.21.15.38:0/3772201728 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+5229 (secure 0 0 0) 0x7f8c600835e0 con 0x7f8c6806ec30 2024-09-21T03:36:03.674 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:36:03.674 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":30,"btime":"2024-09-21T03:35:47:913013+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5147,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/2858412699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":2858412699},{"type":"v1","addr":"172.21.15.195:6837","nonce":2858412699}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":5368,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/2907144575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":2907144575},{"type":"v1","addr":"172.21.15.195:6836","nonce":2907144575}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":5574,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6834/1911661184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":1911661184},{"type":"v1","addr":"172.21.15.38:6834","nonce":1911661184}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:35:44.271168+0000","modified":"2024-09-21T03:35:47.294703+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5577},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5577":{"gid":5577,"name":"c","rank":0,"incarnation":27,"state":"up:active","state_seq":3,"addr":"172.21.15.38:6835/3184989541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6833","nonce":3184989541},{"type":"v1","addr":"172.21.15.38:6835","nonce":3184989541}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5577,"qdb_cluster":[5577]},"id":3}]} 2024-09-21T03:36:03.674 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 30 2024-09-21T03:36:03.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:03.674+0000 7f8c6f550640 1 -- 172.21.15.38:0/3772201728 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8c54078b90 msgr2=0x7f8c5407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:03.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:03.674+0000 7f8c6f550640 1 --2- 172.21.15.38:0/3772201728 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8c54078b90 0x7f8c5407b0a0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f8c6806de90 tx=0x7f8c5c00c440 comp rx=0 tx=0).stop 2024-09-21T03:36:03.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:03.674+0000 7f8c6f550640 1 -- 172.21.15.38:0/3772201728 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8c6806ec30 msgr2=0x7f8c681482c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:03.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:03.674+0000 7f8c6f550640 1 --2- 172.21.15.38:0/3772201728 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8c6806ec30 0x7f8c681482c0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f8c60034040 tx=0x7f8c600038a0 comp rx=0 tx=0).stop 2024-09-21T03:36:03.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:03.674+0000 7f8c6f550640 1 -- 172.21.15.38:0/3772201728 shutdown_connections 2024-09-21T03:36:03.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:03.674+0000 7f8c6f550640 1 --2- 172.21.15.38:0/3772201728 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8c680d0b40 0x7f8c6814e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:03.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:03.674+0000 7f8c6f550640 1 --2- 172.21.15.38:0/3772201728 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8c54078b90 0x7f8c5407b0a0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:03.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:03.674+0000 7f8c6f550640 1 --2- 172.21.15.38:0/3772201728 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8c680b4500 0x7f8c68148800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:03.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:03.674+0000 7f8c6f550640 1 --2- 172.21.15.38:0/3772201728 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8c6806ec30 0x7f8c681482c0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:03.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:03.674+0000 7f8c6f550640 1 -- 172.21.15.38:0/3772201728 >> 172.21.15.38:0/3772201728 conn(0x7f8c680b2cc0 msgr2=0x7f8c68073d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:03.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:03.674+0000 7f8c6f550640 1 -- 172.21.15.38:0/3772201728 shutdown_connections 2024-09-21T03:36:03.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:03.674+0000 7f8c6f550640 1 -- 172.21.15.38:0/3772201728 wait complete. 2024-09-21T03:36:03.688 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json dump tree /a 0 2024-09-21T03:36:03.814 INFO:teuthology.orchestra.run.smithi038.stdout:[{"ino":1099511627776,"rdev":0,"ctime":"2024-09-21T03:36:02.400568+0000","btime":"2024-09-21T03:35:56.466679+0000","mode":16877,"uid":1000,"gid":1333,"nlink":1,"dir_layout":{"dir_hash":2,"unused1":0,"unused2":0,"unused3":0},"layout":{"stripe_unit":0,"stripe_count":0,"object_size":0,"pool_id":-1,"pool_ns":""},"old_pools":[],"size":0,"truncate_seq":1,"truncate_size":18446744073709551615,"truncate_from":0,"truncate_pending":0,"mtime":"2024-09-21T03:36:02.400568+0000","atime":"2024-09-21T03:35:56.466679+0000","time_warp_seq":0,"change_attr":3,"export_pin":-1,"export_ephemeral_random_pin":0,"export_ephemeral_distributed_pin":false,"quiesce_block":false,"client_ranges":[],"dirstat":{"version":0,"mtime":"2024-09-21T03:36:02.400568+0000","num_files":0,"num_subdirs":1,"change_attr":3},"rstat":{"version":0,"rbytes":0,"rfiles":0,"rsubdirs":2,"rsnaps":0,"rctime":"2024-09-21T03:36:02.400568+0000"},"accounted_rstat":{"version":0,"rbytes":0,"rfiles":0,"rsubdirs":2,"rsnaps":0,"rctime":"2024-09-21T03:36:02.400568+0000"},"version":9,"file_data_version":0,"xattr_version":1,"backtrace_version":2,"inline_data_version":18446744073709551615,"inline_data_length":0,"stray_prior_path":"","max_size_ever":0,"quota":{"max_bytes":0,"max_files":0},"last_scrub_stamp":"0.000000","last_scrub_version":0,"symlink":"","xattrs":[{"key":"security.selinux","val":"unconfined_u:object_r:unlabeled_t:s0\u0000"}],"dirfragtree":{"splits":[]},"old_inodes":[],"oldest_snap":18446744073709551614,"damage_flags":0,"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"request":0,"lock":0,"dirfrag":1,"caps":0,"dirtyparent":0,"dirty":1,"authpin":0},"nref":2,"versionlock":{"gather_set":[],"state":"lock","type":"iversion","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"authlock":{},"linklock":{},"dirfragtreelock":{},"filelock":{},"xattrlock":{},"snaplock":{},"nestlock":{"gather_set":[],"state":"lock","type":"inest","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"flocklock":{},"policylock":{},"quiescelock":{"gather_set":[],"state":"lock","type":"iquiesce","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"states":["auth","dirty"],"client_caps":[],"loner":-1,"want_loner":-1,"mds_caps_wanted":[],"dirfrags":[{"path":"/a","dirfrag":"0x10000000000","snapid_first":2,"projected_version":"7","version":"7","committing_version":"3","committed_version":"3","is_rep":false,"dir_auth":"","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"dirty":1,"authpin":0},"nref":2,"dentries":[{"path":"a/c","alternate_name":"","path_ino":1,"snap_first":18446744073709551606,"snap_last":18446744073709551614,"is_primary":true,"is_remote":false,"is_null":false,"is_new":false,"inode":1099511627777,"remote_type":"","version":4,"projected_version":4,"auth_pins":0,"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"request":0,"lock":0,"inodepin":1,"dirty":1,"authpin":0},"nref":2,"lock":{},"versionlock":{"gather_set":[],"state":"lock","type":"dversion","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"states":["auth","dirty"]},{"path":"a/b","alternate_name":"","path_ino":1,"snap_first":2,"snap_last":18446744073709551614,"is_primary":false,"is_remote":false,"is_null":true,"is_new":false,"inode":0,"remote_type":"","version":5,"projected_version":5,"auth_pins":0,"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"request":0,"lock":0,"inodepin":0,"dirty":1,"authpin":0},"nref":1,"lock":{},"versionlock":{"gather_set":[],"state":"lock","type":"dversion","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"states":["auth","dirty"]}]}],"is_subvolume":false}] 2024-09-21T03:36:03.815 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "accounted_rstat": { "rbytes": 0, "rctime": "2024-09-21T03:36:02.400568+0000", "rfiles": 0, "rsnaps": 0, "rsubdirs": 2, "version": 0 }, "atime": "2024-09-21T03:35:56.466679+0000", "auth_pins": 0, "auth_state": { "replicas": {} }, "authlock": {}, "backtrace_version": 2, "btime": "2024-09-21T03:35:56.466679+0000", "change_attr": 3, "client_caps": [], "client_ranges": [], "ctime": "2024-09-21T03:36:02.400568+0000", "damage_flags": 0, "dir_layout": { "dir_hash": 2, "unused1": 0, "unused2": 0, "unused3": 0 }, "dirfrags": [ { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "3", "committing_version": "3", "dentries": [ { "alternate_name": "", "auth_pins": 0, "auth_state": { "replicas": {} }, "inode": 1099511627777, "is_auth": true, "is_freezing": false, "is_frozen": false, "is_new": false, "is_null": false, "is_primary": true, "is_remote": false, "lock": {}, "nref": 2, "path": "a/c", "path_ino": 1, "pins": { "authpin": 0, "dirty": 1, "inodepin": 1, "lock": 0, "request": 0 }, "projected_version": 4, "remote_type": "", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snap_first": 18446744073709551606, "snap_last": 18446744073709551614, "states": [ "auth", "dirty" ], "version": 4, "versionlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "type": "dversion", "xlock_by": {} } }, { "alternate_name": "", "auth_pins": 0, "auth_state": { "replicas": {} }, "inode": 0, "is_auth": true, "is_freezing": false, "is_frozen": false, "is_new": false, "is_null": true, "is_primary": false, "is_remote": false, "lock": {}, "nref": 1, "path": "a/b", "path_ino": 1, "pins": { "authpin": 0, "dirty": 1, "inodepin": 0, "lock": 0, "request": 0 }, "projected_version": 5, "remote_type": "", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snap_first": 2, "snap_last": 18446744073709551614, "states": [ "auth", "dirty" ], "version": 5, "versionlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "type": "dversion", "xlock_by": {} } } ], "dir_auth": "", "dirfrag": "0x10000000000", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "/a", "pins": { "authpin": 0, "child": 1, "dirty": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "7" } ], "dirfragtree": { "splits": [] }, "dirfragtreelock": {}, "dirstat": { "change_attr": 3, "mtime": "2024-09-21T03:36:02.400568+0000", "num_files": 0, "num_subdirs": 1, "version": 0 }, "export_ephemeral_distributed_pin": false, "export_ephemeral_random_pin": 0, "export_pin": -1, "file_data_version": 0, "filelock": {}, "flocklock": {}, "gid": 1333, "inline_data_length": 0, "inline_data_version": 18446744073709551615, "ino": 1099511627776, "is_auth": true, "is_freezing": false, "is_frozen": false, "is_subvolume": false, "last_scrub_stamp": "0.000000", "last_scrub_version": 0, "layout": { "object_size": 0, "pool_id": -1, "pool_ns": "", "stripe_count": 0, "stripe_unit": 0 }, "linklock": {}, "loner": -1, "max_size_ever": 0, "mds_caps_wanted": [], "mode": 16877, "mtime": "2024-09-21T03:36:02.400568+0000", "nestlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "type": "inest", "xlock_by": {} }, "nlink": 1, "nref": 2, "old_inodes": [], "old_pools": [], "oldest_snap": 18446744073709551614, "pins": { "authpin": 0, "caps": 0, "dirfrag": 1, "dirty": 1, "dirtyparent": 0, "lock": 0, "request": 0 }, "policylock": {}, "quiesce_block": false, "quiescelock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "type": "iquiesce", "xlock_by": {} }, "quota": { "max_bytes": 0, "max_files": 0 }, "rdev": 0, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "rstat": { "rbytes": 0, "rctime": "2024-09-21T03:36:02.400568+0000", "rfiles": 0, "rsnaps": 0, "rsubdirs": 2, "version": 0 }, "size": 0, "snaplock": {}, "states": [ "auth", "dirty" ], "stray_prior_path": "", "symlink": "", "time_warp_seq": 0, "truncate_from": 0, "truncate_pending": 0, "truncate_seq": 1, "truncate_size": 18446744073709551615, "uid": 1000, "version": 9, "versionlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "type": "iversion", "xlock_by": {} }, "want_loner": -1, "xattr_version": 1, "xattrlock": {}, "xattrs": [ { "key": "security.selinux", "val": "unconfined_u:object_r:unlabeled_t:s0\u0000" } ] } ] 2024-09-21T03:36:03.815 DEBUG:tasks.cephfs.test_damage:[{'ino': 1099511627776, 'rdev': 0, 'ctime': '2024-09-21T03:36:02.400568+0000', 'btime': '2024-09-21T03:35:56.466679+0000', 'mode': 16877, 'uid': 1000, 'gid': 1333, 'nlink': 1, 'dir_layout': {'dir_hash': 2, 'unused1': 0, 'unused2': 0, 'unused3': 0}, 'layout': {'stripe_unit': 0, 'stripe_count': 0, 'object_size': 0, 'pool_id': -1, 'pool_ns': ''}, 'old_pools': [], 'size': 0, 'truncate_seq': 1, 'truncate_size': 18446744073709551615, 'truncate_from': 0, 'truncate_pending': 0, 'mtime': '2024-09-21T03:36:02.400568+0000', 'atime': '2024-09-21T03:35:56.466679+0000', 'time_warp_seq': 0, 'change_attr': 3, 'export_pin': -1, 'export_ephemeral_random_pin': 0, 'export_ephemeral_distributed_pin': False, 'quiesce_block': False, 'client_ranges': [], 'dirstat': {'version': 0, 'mtime': '2024-09-21T03:36:02.400568+0000', 'num_files': 0, 'num_subdirs': 1, 'change_attr': 3}, 'rstat': {'version': 0, 'rbytes': 0, 'rfiles': 0, 'rsubdirs': 2, 'rsnaps': 0, 'rctime': '2024-09-21T03:36:02.400568+0000'}, 'accounted_rstat': {'version': 0, 'rbytes': 0, 'rfiles': 0, 'rsubdirs': 2, 'rsnaps': 0, 'rctime': '2024-09-21T03:36:02.400568+0000'}, 'version': 9, 'file_data_version': 0, 'xattr_version': 1, 'backtrace_version': 2, 'inline_data_version': 18446744073709551615, 'inline_data_length': 0, 'stray_prior_path': '', 'max_size_ever': 0, 'quota': {'max_bytes': 0, 'max_files': 0}, 'last_scrub_stamp': '0.000000', 'last_scrub_version': 0, 'symlink': '', 'xattrs': [{'key': 'security.selinux', 'val': 'unconfined_u:object_r:unlabeled_t:s0\x00'}], 'dirfragtree': {'splits': []}, 'old_inodes': [], 'oldest_snap': 18446744073709551614, 'damage_flags': 0, 'is_auth': True, 'auth_state': {'replicas': {}}, 'replica_state': {'authority': [0, -2], 'replica_nonce': 0}, 'auth_pins': 0, 'is_frozen': False, 'is_freezing': False, 'pins': {'request': 0, 'lock': 0, 'dirfrag': 1, 'caps': 0, 'dirtyparent': 0, 'dirty': 1, 'authpin': 0}, 'nref': 2, 'versionlock': {'gather_set': [], 'state': 'lock', 'type': 'iversion', 'is_leased': False, 'num_rdlocks': 0, 'num_wrlocks': 0, 'num_xlocks': 0, 'xlock_by': {}}, 'authlock': {}, 'linklock': {}, 'dirfragtreelock': {}, 'filelock': {}, 'xattrlock': {}, 'snaplock': {}, 'nestlock': {'gather_set': [], 'state': 'lock', 'type': 'inest', 'is_leased': False, 'num_rdlocks': 0, 'num_wrlocks': 0, 'num_xlocks': 0, 'xlock_by': {}}, 'flocklock': {}, 'policylock': {}, 'quiescelock': {'gather_set': [], 'state': 'lock', 'type': 'iquiesce', 'is_leased': False, 'num_rdlocks': 0, 'num_wrlocks': 0, 'num_xlocks': 0, 'xlock_by': {}}, 'states': ['auth', 'dirty'], 'client_caps': [], 'loner': -1, 'want_loner': -1, 'mds_caps_wanted': [], 'dirfrags': [{'path': '/a', 'dirfrag': '0x10000000000', 'snapid_first': 2, 'projected_version': '7', 'version': '7', 'committing_version': '3', 'committed_version': '3', 'is_rep': False, 'dir_auth': '', 'states': ['auth', 'dirty', 'complete'], 'is_auth': True, 'auth_state': {'replicas': {}}, 'replica_state': {'authority': [0, -2], 'replica_nonce': 0}, 'auth_pins': 0, 'is_frozen': False, 'is_freezing': False, 'pins': {'child': 1, 'dirty': 1, 'authpin': 0}, 'nref': 2, 'dentries': [{'path': 'a/c', 'alternate_name': '', 'path_ino': 1, 'snap_first': 18446744073709551606, 'snap_last': 18446744073709551614, 'is_primary': True, 'is_remote': False, 'is_null': False, 'is_new': False, 'inode': 1099511627777, 'remote_type': '', 'version': 4, 'projected_version': 4, 'auth_pins': 0, 'is_auth': True, 'auth_state': {'replicas': {}}, 'replica_state': {'authority': [0, -2], 'replica_nonce': 0}, 'is_frozen': False, 'is_freezing': False, 'pins': {'request': 0, 'lock': 0, 'inodepin': 1, 'dirty': 1, 'authpin': 0}, 'nref': 2, 'lock': {}, 'versionlock': {'gather_set': [], 'state': 'lock', 'type': 'dversion', 'is_leased': False, 'num_rdlocks': 0, 'num_wrlocks': 0, 'num_xlocks': 0, 'xlock_by': {}}, 'states': ['auth', 'dirty']}, {'path': 'a/b', 'alternate_name': '', 'path_ino': 1, 'snap_first': 2, 'snap_last': 18446744073709551614, 'is_primary': False, 'is_remote': False, 'is_null': True, 'is_new': False, 'inode': 0, 'remote_type': '', 'version': 5, 'projected_version': 5, 'auth_pins': 0, 'is_auth': True, 'auth_state': {'replicas': {}}, 'replica_state': {'authority': [0, -2], 'replica_nonce': 0}, 'is_frozen': False, 'is_freezing': False, 'pins': {'request': 0, 'lock': 0, 'inodepin': 0, 'dirty': 1, 'authpin': 0}, 'nref': 1, 'lock': {}, 'versionlock': {'gather_set': [], 'state': 'lock', 'type': 'dversion', 'is_leased': False, 'num_rdlocks': 0, 'num_wrlocks': 0, 'num_xlocks': 0, 'xlock_by': {}}, 'states': ['auth', 'dirty']}]}], 'is_subvolume': False}] 2024-09-21T03:36:03.815 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-21T03:36:03.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:03.932+0000 7f363825c640 1 -- 172.21.15.38:0/3575337518 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f36300d0bb0 msgr2=0x7f36300b5be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:03.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:03.932+0000 7f363825c640 1 --2- 172.21.15.38:0/3575337518 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f36300d0bb0 0x7f36300b5be0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f36300cc9c0 tx=0x7f362802a980 comp rx=0 tx=0).stop 2024-09-21T03:36:03.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:03.932+0000 7f363825c640 1 -- 172.21.15.38:0/3575337518 shutdown_connections 2024-09-21T03:36:03.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:03.932+0000 7f363825c640 1 --2- 172.21.15.38:0/3575337518 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f36300d0bb0 0x7f36300b5be0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:03.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:03.932+0000 7f363825c640 1 --2- 172.21.15.38:0/3575337518 >> v2:172.21.15.195:3300/0 conn(0x7f363015ca90 0x7f36300d0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:03.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:03.932+0000 7f363825c640 1 -- 172.21.15.38:0/3575337518 >> 172.21.15.38:0/3575337518 conn(0x7f36300b46e0 msgr2=0x7f36300d23e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:03.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:03.932+0000 7f363825c640 1 -- 172.21.15.38:0/3575337518 shutdown_connections 2024-09-21T03:36:03.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:03.932+0000 7f363825c640 1 -- 172.21.15.38:0/3575337518 wait complete. 2024-09-21T03:36:03.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:03.934+0000 7f363825c640 1 Processor -- start 2024-09-21T03:36:03.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:03.934+0000 7f363825c640 1 -- start start 2024-09-21T03:36:03.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:03.935+0000 7f363825c640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f36300ccc40 0x7f36300aa9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:03.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:03.935+0000 7f363825c640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f36300d0bb0 0x7f36300cb8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:03.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:03.935+0000 7f363825c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f363015ca90 0x7f36300cbe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:03.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:03.935+0000 7f363825c640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f36300b6620 con 0x7f363015ca90 2024-09-21T03:36:03.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:03.935+0000 7f363825c640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f36300b64a0 con 0x7f36300d0bb0 2024-09-21T03:36:03.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:03.935+0000 7f363825c640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f36300b67a0 con 0x7f36300ccc40 2024-09-21T03:36:03.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:03.935+0000 7f36367f9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f363015ca90 0x7f36300cbe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:03.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:03.935+0000 7f36367f9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f363015ca90 0x7f36300cbe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37304/0 (socket says 172.21.15.38:37304) 2024-09-21T03:36:03.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:03.935+0000 7f36367f9640 1 -- 172.21.15.38:0/2575987882 learned_addr learned my addr 172.21.15.38:0/2575987882 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:36:03.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:03.935+0000 7f36357f7640 1 --2- 172.21.15.38:0/2575987882 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f36300d0bb0 0x7f36300cb8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:03.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:03.935+0000 7f3635ff8640 1 --2- 172.21.15.38:0/2575987882 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f36300ccc40 0x7f36300aa9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:03.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:03.936+0000 7f36367f9640 1 -- 172.21.15.38:0/2575987882 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f36300ccc40 msgr2=0x7f36300aa9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:03.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:03.936+0000 7f36367f9640 1 --2- 172.21.15.38:0/2575987882 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f36300ccc40 0x7f36300aa9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:03.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:03.936+0000 7f36367f9640 1 -- 172.21.15.38:0/2575987882 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f36300d0bb0 msgr2=0x7f36300cb8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:03.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:03.936+0000 7f36367f9640 1 --2- 172.21.15.38:0/2575987882 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f36300d0bb0 0x7f36300cb8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:03.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:03.936+0000 7f36367f9640 1 -- 172.21.15.38:0/2575987882 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36301f3600 con 0x7f363015ca90 2024-09-21T03:36:03.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:03.936+0000 7f3635ff8640 1 --2- 172.21.15.38:0/2575987882 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f36300ccc40 0x7f36300aa9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:36:03.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:03.936+0000 7f36367f9640 1 --2- 172.21.15.38:0/2575987882 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f363015ca90 0x7f36300cbe00 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f3628009bf0 tx=0x7f3628003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:03.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:03.936+0000 7f36227fc640 1 -- 172.21.15.38:0/2575987882 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3628034040 con 0x7f363015ca90 2024-09-21T03:36:03.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:03.936+0000 7f36227fc640 1 -- 172.21.15.38:0/2575987882 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3628004600 con 0x7f363015ca90 2024-09-21T03:36:03.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:03.936+0000 7f363825c640 1 -- 172.21.15.38:0/2575987882 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36301f38e0 con 0x7f363015ca90 2024-09-21T03:36:03.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:03.936+0000 7f36227fc640 1 -- 172.21.15.38:0/2575987882 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3628045480 con 0x7f363015ca90 2024-09-21T03:36:03.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:03.936+0000 7f363825c640 1 -- 172.21.15.38:0/2575987882 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36301f3ea0 con 0x7f363015ca90 2024-09-21T03:36:03.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:03.937+0000 7f363825c640 1 -- 172.21.15.38:0/2575987882 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36300b6d50 con 0x7f363015ca90 2024-09-21T03:36:03.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:03.938+0000 7f36227fc640 1 -- 172.21.15.38:0/2575987882 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f3628037020 con 0x7f363015ca90 2024-09-21T03:36:03.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:03.939+0000 7f36227fc640 1 --2- 172.21.15.38:0/2575987882 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3614078b90 0x7f361407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:03.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:03.939+0000 7f36227fc640 1 -- 172.21.15.38:0/2575987882 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 6101+0+0 (secure 0 0 0) 0x7f36280bc840 con 0x7f363015ca90 2024-09-21T03:36:03.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:03.939+0000 7f3635ff8640 1 --2- 172.21.15.38:0/2575987882 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3614078b90 0x7f361407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:03.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:03.939+0000 7f3635ff8640 1 --2- 172.21.15.38:0/2575987882 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3614078b90 0x7f361407b0a0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f362c006fd0 tx=0x7f362c008040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:03.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:03.943+0000 7f36227fc640 1 -- 172.21.15.38:0/2575987882 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f36300d0fc0 con 0x7f363015ca90 2024-09-21T03:36:04.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:04.172+0000 7f363825c640 1 -- 172.21.15.38:0/2575987882 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f36300c6510 con 0x7f363015ca90 2024-09-21T03:36:04.174 INFO:tasks.ceph.mon.a.smithi038.stderr:2024-09-21T03:36:04.172+0000 7f3f45713640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T03:36:04.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:04.174+0000 7f36227fc640 1 -- 172.21.15.38:0/2575987882 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v31) ==== 161+0+0 (secure 0 0 0) 0x7f3628084b90 con 0x7f363015ca90 2024-09-21T03:36:04.176 INFO:teuthology.orchestra.run.smithi038.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-21T03:36:04.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:04.178+0000 7f363825c640 1 -- 172.21.15.38:0/2575987882 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3614078b90 msgr2=0x7f361407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:04.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:04.178+0000 7f363825c640 1 --2- 172.21.15.38:0/2575987882 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3614078b90 0x7f361407b0a0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f362c006fd0 tx=0x7f362c008040 comp rx=0 tx=0).stop 2024-09-21T03:36:04.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:04.178+0000 7f363825c640 1 -- 172.21.15.38:0/2575987882 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f363015ca90 msgr2=0x7f36300cbe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:04.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:04.178+0000 7f363825c640 1 --2- 172.21.15.38:0/2575987882 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f363015ca90 0x7f36300cbe00 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f3628009bf0 tx=0x7f3628003f90 comp rx=0 tx=0).stop 2024-09-21T03:36:04.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:04.178+0000 7f363825c640 1 -- 172.21.15.38:0/2575987882 shutdown_connections 2024-09-21T03:36:04.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:04.178+0000 7f363825c640 1 --2- 172.21.15.38:0/2575987882 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f363015ca90 0x7f36300cbe00 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:04.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:04.178+0000 7f363825c640 1 --2- 172.21.15.38:0/2575987882 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f36300d0bb0 0x7f36300cb8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:04.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:04.178+0000 7f363825c640 1 --2- 172.21.15.38:0/2575987882 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3614078b90 0x7f361407b0a0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:04.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:04.178+0000 7f363825c640 1 --2- 172.21.15.38:0/2575987882 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f36300ccc40 0x7f36300aa9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:04.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:04.178+0000 7f363825c640 1 -- 172.21.15.38:0/2575987882 >> 172.21.15.38:0/2575987882 conn(0x7f36300b46e0 msgr2=0x7f36300d23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:04.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:04.178+0000 7f363825c640 1 -- 172.21.15.38:0/2575987882 shutdown_connections 2024-09-21T03:36:04.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:04.178+0000 7f363825c640 1 -- 172.21.15.38:0/2575987882 wait complete. 2024-09-21T03:36:04.182 INFO:tasks.ceph.mds.c.smithi038.stderr: -4269> 2024-09-21T03:35:50.193+0000 7f631f274640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T03:36:04.186 INFO:tasks.ceph.mds.c.smithi038.stderr: -317> 2024-09-21T03:36:02.401+0000 7f6321a79640 -1 mds.0.cache.den(0x10000000000 c) newly corrupt dentry to be committed: [dentry #0x1/a/c [fffffffffffffff6,head] auth NULL (dn xlock x=1 by request(client.5246:44 nref=7 cr=0x561425a23c00)) (dversion lock w=1 last_client=5246) pv=4 v=3 ap=2 ino=(nil) state=1073741824 | request=1 lock=2 authpin=1 0x5614258f7680] with next_snap: 2 2024-09-21T03:36:04.193 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_inject_rename_corrupt_dentry_first 2024-09-21T03:36:04.240 INFO:tasks.ceph.mds.c.smithi038.stdout:starting mds.c at 2024-09-21T03:36:04.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:04.314+0000 7fd6c06f9640 1 -- 172.21.15.38:0/1057335782 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd6b80d0bb0 msgr2=0x7fd6b80b5be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:04.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:04.314+0000 7fd6c06f9640 1 --2- 172.21.15.38:0/1057335782 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd6b80d0bb0 0x7fd6b80b5be0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd6b0005fd0 tx=0x7fd6b002a540 comp rx=0 tx=0).stop 2024-09-21T03:36:04.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:04.314+0000 7fd6c06f9640 1 -- 172.21.15.38:0/1057335782 shutdown_connections 2024-09-21T03:36:04.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:04.314+0000 7fd6c06f9640 1 --2- 172.21.15.38:0/1057335782 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd6b80d0bb0 0x7fd6b80b5be0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:04.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:04.314+0000 7fd6c06f9640 1 --2- 172.21.15.38:0/1057335782 >> v2:172.21.15.195:3300/0 conn(0x7fd6b815ca90 0x7fd6b80d0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:04.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:04.314+0000 7fd6c06f9640 1 -- 172.21.15.38:0/1057335782 >> 172.21.15.38:0/1057335782 conn(0x7fd6b80b46e0 msgr2=0x7fd6b80d23e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:04.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:04.314+0000 7fd6c06f9640 1 -- 172.21.15.38:0/1057335782 shutdown_connections 2024-09-21T03:36:04.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:04.314+0000 7fd6c06f9640 1 -- 172.21.15.38:0/1057335782 wait complete. 2024-09-21T03:36:04.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:04.315+0000 7fd6c06f9640 1 Processor -- start 2024-09-21T03:36:04.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:04.316+0000 7fd6c06f9640 1 -- start start 2024-09-21T03:36:04.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:04.317+0000 7fd6c06f9640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd6b80ccc40 0x7fd6b806e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:04.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:04.317+0000 7fd6c06f9640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd6b80d0bb0 0x7fd6b8150960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:04.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:04.317+0000 7fd6c06f9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd6b815ca90 0x7fd6b81e95d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:04.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:04.317+0000 7fd6c06f9640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fd6b80b66a0 con 0x7fd6b815ca90 2024-09-21T03:36:04.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:04.317+0000 7fd6c06f9640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd6b80b6520 con 0x7fd6b80ccc40 2024-09-21T03:36:04.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:04.317+0000 7fd6c06f9640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fd6b80b6820 con 0x7fd6b80d0bb0 2024-09-21T03:36:04.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:04.317+0000 7fd6bec96640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd6b815ca90 0x7fd6b81e95d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:04.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:04.317+0000 7fd6bec96640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd6b815ca90 0x7fd6b81e95d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37328/0 (socket says 172.21.15.38:37328) 2024-09-21T03:36:04.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:04.317+0000 7fd6bec96640 1 -- 172.21.15.38:0/3439553782 learned_addr learned my addr 172.21.15.38:0/3439553782 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:36:04.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:04.317+0000 7fd6be495640 1 --2- 172.21.15.38:0/3439553782 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd6b80ccc40 0x7fd6b806e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:04.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:04.317+0000 7fd6bdc94640 1 --2- 172.21.15.38:0/3439553782 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd6b80d0bb0 0x7fd6b8150960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:04.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:04.317+0000 7fd6bec96640 1 -- 172.21.15.38:0/3439553782 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd6b80d0bb0 msgr2=0x7fd6b8150960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:04.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:04.317+0000 7fd6bec96640 1 --2- 172.21.15.38:0/3439553782 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd6b80d0bb0 0x7fd6b8150960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:04.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:04.317+0000 7fd6bec96640 1 -- 172.21.15.38:0/3439553782 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd6b80ccc40 msgr2=0x7fd6b806e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:04.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:04.317+0000 7fd6bec96640 1 --2- 172.21.15.38:0/3439553782 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd6b80ccc40 0x7fd6b806e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:04.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:04.317+0000 7fd6bec96640 1 -- 172.21.15.38:0/3439553782 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6b806ee40 con 0x7fd6b815ca90 2024-09-21T03:36:04.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:04.318+0000 7fd6bec96640 1 --2- 172.21.15.38:0/3439553782 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd6b815ca90 0x7fd6b81e95d0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fd6b0034040 tx=0x7fd6b00038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:04.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:04.318+0000 7fd6b6ffd640 1 -- 172.21.15.38:0/3439553782 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6b003e030 con 0x7fd6b815ca90 2024-09-21T03:36:04.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:04.318+0000 7fd6b6ffd640 1 -- 172.21.15.38:0/3439553782 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd6b0003ca0 con 0x7fd6b815ca90 2024-09-21T03:36:04.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:04.318+0000 7fd6c06f9640 1 -- 172.21.15.38:0/3439553782 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6b806f680 con 0x7fd6b815ca90 2024-09-21T03:36:04.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:04.318+0000 7fd6b6ffd640 1 -- 172.21.15.38:0/3439553782 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6b0035040 con 0x7fd6b815ca90 2024-09-21T03:36:04.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:04.318+0000 7fd6c06f9640 1 -- 172.21.15.38:0/3439553782 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6b81f96f0 con 0x7fd6b815ca90 2024-09-21T03:36:04.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:04.319+0000 7fd6c06f9640 1 -- 172.21.15.38:0/3439553782 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6b806f3f0 con 0x7fd6b815ca90 2024-09-21T03:36:04.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:04.320+0000 7fd6b6ffd640 1 -- 172.21.15.38:0/3439553782 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fd6b0041030 con 0x7fd6b815ca90 2024-09-21T03:36:04.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:04.320+0000 7fd6b6ffd640 1 --2- 172.21.15.38:0/3439553782 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd6a4078b90 0x7fd6a407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:04.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:04.320+0000 7fd6b6ffd640 1 -- 172.21.15.38:0/3439553782 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 6187+0+0 (secure 0 0 0) 0x7fd6b00bb5f0 con 0x7fd6b815ca90 2024-09-21T03:36:04.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:04.320+0000 7fd6be495640 1 --2- 172.21.15.38:0/3439553782 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd6a4078b90 0x7fd6a407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:04.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:04.321+0000 7fd6be495640 1 --2- 172.21.15.38:0/3439553782 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd6a4078b90 0x7fd6a407b0a0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fd6a8001e20 tx=0x7fd6a8009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:04.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:04.325+0000 7fd6b6ffd640 1 -- 172.21.15.38:0/3439553782 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fd6b806ee40 con 0x7fd6b815ca90 2024-09-21T03:36:04.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:04.514+0000 7fd6c06f9640 1 -- 172.21.15.38:0/3439553782 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_inject_rename_corrupt_dentry_first"} v 0) -- 0x7fd6b81f9230 con 0x7fd6b815ca90 2024-09-21T03:36:04.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:04.518+0000 7fd6b6ffd640 1 -- 172.21.15.38:0/3439553782 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_inject_rename_corrupt_dentry_first"}]=0 v37) ==== 121+0+0 (secure 0 0 0) 0x7fd6b81f9230 con 0x7fd6b815ca90 2024-09-21T03:36:04.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:04.520+0000 7fd6c06f9640 1 -- 172.21.15.38:0/3439553782 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd6a4078b90 msgr2=0x7fd6a407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:04.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:04.520+0000 7fd6c06f9640 1 --2- 172.21.15.38:0/3439553782 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd6a4078b90 0x7fd6a407b0a0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fd6a8001e20 tx=0x7fd6a8009040 comp rx=0 tx=0).stop 2024-09-21T03:36:04.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:04.520+0000 7fd6c06f9640 1 -- 172.21.15.38:0/3439553782 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd6b815ca90 msgr2=0x7fd6b81e95d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:04.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:04.521+0000 7fd6c06f9640 1 --2- 172.21.15.38:0/3439553782 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd6b815ca90 0x7fd6b81e95d0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fd6b0034040 tx=0x7fd6b00038a0 comp rx=0 tx=0).stop 2024-09-21T03:36:04.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:04.521+0000 7fd6c06f9640 1 -- 172.21.15.38:0/3439553782 shutdown_connections 2024-09-21T03:36:04.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:04.521+0000 7fd6c06f9640 1 --2- 172.21.15.38:0/3439553782 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd6b815ca90 0x7fd6b81e95d0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:04.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:04.521+0000 7fd6c06f9640 1 --2- 172.21.15.38:0/3439553782 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd6a4078b90 0x7fd6a407b0a0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:04.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:04.521+0000 7fd6c06f9640 1 --2- 172.21.15.38:0/3439553782 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd6b80d0bb0 0x7fd6b8150960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:04.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:04.521+0000 7fd6c06f9640 1 --2- 172.21.15.38:0/3439553782 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd6b80ccc40 0x7fd6b806e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:04.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:04.521+0000 7fd6c06f9640 1 -- 172.21.15.38:0/3439553782 >> 172.21.15.38:0/3439553782 conn(0x7fd6b80b46e0 msgr2=0x7fd6b815f0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:04.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:04.521+0000 7fd6c06f9640 1 -- 172.21.15.38:0/3439553782 shutdown_connections 2024-09-21T03:36:04.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:04.521+0000 7fd6c06f9640 1 -- 172.21.15.38:0/3439553782 wait complete. 2024-09-21T03:36:04.534 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_abort_on_newly_corrupt_dentry False 2024-09-21T03:36:04.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:04.651+0000 7fb67724a640 1 -- 172.21.15.38:0/301934217 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb67015ca90 msgr2=0x7fb6700d0670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:04.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:04.651+0000 7fb67724a640 1 --2- 172.21.15.38:0/301934217 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb67015ca90 0x7fb6700d0670 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fb660005fd0 tx=0x7fb66002aa10 comp rx=0 tx=0).stop 2024-09-21T03:36:04.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:04.651+0000 7fb67724a640 1 -- 172.21.15.38:0/301934217 shutdown_connections 2024-09-21T03:36:04.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:04.651+0000 7fb67724a640 1 --2- 172.21.15.38:0/301934217 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb6700d0bb0 0x7fb6700b5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:04.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:04.651+0000 7fb67724a640 1 --2- 172.21.15.38:0/301934217 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb67015ca90 0x7fb6700d0670 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:04.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:04.651+0000 7fb67724a640 1 -- 172.21.15.38:0/301934217 >> 172.21.15.38:0/301934217 conn(0x7fb6700b46e0 msgr2=0x7fb6700d23e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:04.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:04.651+0000 7fb67724a640 1 -- 172.21.15.38:0/301934217 shutdown_connections 2024-09-21T03:36:04.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:04.651+0000 7fb67724a640 1 -- 172.21.15.38:0/301934217 wait complete. 2024-09-21T03:36:04.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:04.653+0000 7fb67724a640 1 Processor -- start 2024-09-21T03:36:04.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:04.653+0000 7fb67724a640 1 -- start start 2024-09-21T03:36:04.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:04.655+0000 7fb67724a640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb6700ccc40 0x7fb6701f7a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:04.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:04.655+0000 7fb67724a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb6700d0bb0 0x7fb6700ab1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:04.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:04.655+0000 7fb67724a640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb67015ca90 0x7fb6700b2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:04.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:04.655+0000 7fb67724a640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fb6700b66a0 con 0x7fb6700d0bb0 2024-09-21T03:36:04.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:04.655+0000 7fb67724a640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fb6700b6520 con 0x7fb67015ca90 2024-09-21T03:36:04.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:04.655+0000 7fb67724a640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fb6700b6820 con 0x7fb6700ccc40 2024-09-21T03:36:04.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:04.655+0000 7fb66ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb6700d0bb0 0x7fb6700ab1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:04.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:04.655+0000 7fb66ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb6700d0bb0 0x7fb6700ab1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37356/0 (socket says 172.21.15.38:37356) 2024-09-21T03:36:04.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:04.655+0000 7fb66ffff640 1 -- 172.21.15.38:0/1207712069 learned_addr learned my addr 172.21.15.38:0/1207712069 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:36:04.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:04.655+0000 7fb674fe6640 1 --2- 172.21.15.38:0/1207712069 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb6700ccc40 0x7fb6701f7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:04.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:04.655+0000 7fb6757e7640 1 --2- 172.21.15.38:0/1207712069 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb67015ca90 0x7fb6700b2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:04.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:04.655+0000 7fb66ffff640 1 -- 172.21.15.38:0/1207712069 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb6700ccc40 msgr2=0x7fb6701f7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:04.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:04.655+0000 7fb66ffff640 1 --2- 172.21.15.38:0/1207712069 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb6700ccc40 0x7fb6701f7a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:04.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:04.655+0000 7fb66ffff640 1 -- 172.21.15.38:0/1207712069 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb67015ca90 msgr2=0x7fb6700b2ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:04.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:04.655+0000 7fb66ffff640 1 --2- 172.21.15.38:0/1207712069 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb67015ca90 0x7fb6700b2ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:04.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:04.655+0000 7fb66ffff640 1 -- 172.21.15.38:0/1207712069 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6700ab8c0 con 0x7fb6700d0bb0 2024-09-21T03:36:04.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:04.655+0000 7fb66ffff640 1 --2- 172.21.15.38:0/1207712069 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb6700d0bb0 0x7fb6700ab1b0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fb660000f80 tx=0x7fb660003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:04.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:04.656+0000 7fb66d7fa640 1 -- 172.21.15.38:0/1207712069 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb660046070 con 0x7fb6700d0bb0 2024-09-21T03:36:04.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:04.656+0000 7fb66d7fa640 1 -- 172.21.15.38:0/1207712069 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb660004540 con 0x7fb6700d0bb0 2024-09-21T03:36:04.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:04.656+0000 7fb67724a640 1 -- 172.21.15.38:0/1207712069 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6700abba0 con 0x7fb6700d0bb0 2024-09-21T03:36:04.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:04.656+0000 7fb66d7fa640 1 -- 172.21.15.38:0/1207712069 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb660033070 con 0x7fb6700d0bb0 2024-09-21T03:36:04.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:04.656+0000 7fb67724a640 1 -- 172.21.15.38:0/1207712069 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6700ac160 con 0x7fb6700d0bb0 2024-09-21T03:36:04.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:04.657+0000 7fb67724a640 1 -- 172.21.15.38:0/1207712069 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb67006a780 con 0x7fb6700d0bb0 2024-09-21T03:36:04.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:04.657+0000 7fb66d7fa640 1 -- 172.21.15.38:0/1207712069 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb660036070 con 0x7fb6700d0bb0 2024-09-21T03:36:04.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:04.658+0000 7fb66d7fa640 1 --2- 172.21.15.38:0/1207712069 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb650078b90 0x7fb65007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:04.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:04.658+0000 7fb66d7fa640 1 -- 172.21.15.38:0/1207712069 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 6187+0+0 (secure 0 0 0) 0x7fb6600bbe30 con 0x7fb6700d0bb0 2024-09-21T03:36:04.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:04.658+0000 7fb674fe6640 1 --2- 172.21.15.38:0/1207712069 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb650078b90 0x7fb65007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:04.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:04.659+0000 7fb674fe6640 1 --2- 172.21.15.38:0/1207712069 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb650078b90 0x7fb65007b0a0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fb658001720 tx=0x7fb658009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:04.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:04.663+0000 7fb66d7fa640 1 -- 172.21.15.38:0/1207712069 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fb67006a780 con 0x7fb6700d0bb0 2024-09-21T03:36:04.850 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:04.848+0000 7fb67724a640 1 -- 172.21.15.38:0/1207712069 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command([{prefix=config set, name=mds_abort_on_newly_corrupt_dentry}] v 0) -- 0x7fb6700b6db0 con 0x7fb6700d0bb0 2024-09-21T03:36:04.851 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:04.849+0000 7fb66d7fa640 1 -- 172.21.15.38:0/1207712069 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_abort_on_newly_corrupt_dentry}]=0 v37)=0 v37) ==== 135+0+0 (secure 0 0 0) 0x7fb66003e070 con 0x7fb6700d0bb0 2024-09-21T03:36:04.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:04.851+0000 7fb67724a640 1 -- 172.21.15.38:0/1207712069 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb650078b90 msgr2=0x7fb65007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:04.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:04.851+0000 7fb67724a640 1 --2- 172.21.15.38:0/1207712069 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb650078b90 0x7fb65007b0a0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fb658001720 tx=0x7fb658009040 comp rx=0 tx=0).stop 2024-09-21T03:36:04.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:04.851+0000 7fb67724a640 1 -- 172.21.15.38:0/1207712069 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb6700d0bb0 msgr2=0x7fb6700ab1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:04.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:04.851+0000 7fb67724a640 1 --2- 172.21.15.38:0/1207712069 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb6700d0bb0 0x7fb6700ab1b0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fb660000f80 tx=0x7fb660003f90 comp rx=0 tx=0).stop 2024-09-21T03:36:04.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:04.851+0000 7fb67724a640 1 -- 172.21.15.38:0/1207712069 shutdown_connections 2024-09-21T03:36:04.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:04.851+0000 7fb67724a640 1 --2- 172.21.15.38:0/1207712069 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb67015ca90 0x7fb6700b2ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:04.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:04.851+0000 7fb67724a640 1 --2- 172.21.15.38:0/1207712069 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb6700d0bb0 0x7fb6700ab1b0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:04.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:04.851+0000 7fb67724a640 1 --2- 172.21.15.38:0/1207712069 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb650078b90 0x7fb65007b0a0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:04.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:04.851+0000 7fb67724a640 1 --2- 172.21.15.38:0/1207712069 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb6700ccc40 0x7fb6701f7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:04.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:04.851+0000 7fb67724a640 1 -- 172.21.15.38:0/1207712069 >> 172.21.15.38:0/1207712069 conn(0x7fb6700b46e0 msgr2=0x7fb6700c6b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:04.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:04.851+0000 7fb67724a640 1 -- 172.21.15.38:0/1207712069 shutdown_connections 2024-09-21T03:36:04.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:04.851+0000 7fb67724a640 1 -- 172.21.15.38:0/1207712069 wait complete. 2024-09-21T03:36:04.864 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-21T03:36:04.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:04.982+0000 7fa5b74b6640 1 -- 172.21.15.38:0/2624989348 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa5b00ceb90 msgr2=0x7fa5b00cefa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:04.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:04.982+0000 7fa5b74b6640 1 --2- 172.21.15.38:0/2624989348 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa5b00ceb90 0x7fa5b00cefa0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fa5a0005fd0 tx=0x7fa5a002a780 comp rx=0 tx=0).stop 2024-09-21T03:36:04.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:04.982+0000 7fa5b74b6640 1 -- 172.21.15.38:0/2624989348 shutdown_connections 2024-09-21T03:36:04.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:04.982+0000 7fa5b74b6640 1 --2- 172.21.15.38:0/2624989348 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa5b00ceb90 0x7fa5b00cefa0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:04.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:04.982+0000 7fa5b74b6640 1 -- 172.21.15.38:0/2624989348 >> 172.21.15.38:0/2624989348 conn(0x7fa5b009d830 msgr2=0x7fa5b00b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:04.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:04.982+0000 7fa5b74b6640 1 -- 172.21.15.38:0/2624989348 shutdown_connections 2024-09-21T03:36:04.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:04.982+0000 7fa5b74b6640 1 -- 172.21.15.38:0/2624989348 wait complete. 2024-09-21T03:36:04.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:04.984+0000 7fa5b74b6640 1 Processor -- start 2024-09-21T03:36:04.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:04.984+0000 7fa5b74b6640 1 -- start start 2024-09-21T03:36:04.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:04.985+0000 7fa5b74b6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa5b00ceb90 0x7fa5b0154720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:04.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:04.985+0000 7fa5b74b6640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa5b00d0170 0x7fa5b0159b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:04.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:04.985+0000 7fa5b74b6640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa5b015ee80 0x7fa5b015a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:04.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:04.985+0000 7fa5b74b6640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa5b00b6910 con 0x7fa5b00ceb90 2024-09-21T03:36:04.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:04.985+0000 7fa5b74b6640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fa5b00b6790 con 0x7fa5b015ee80 2024-09-21T03:36:04.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:04.985+0000 7fa5b74b6640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fa5b00b6a90 con 0x7fa5b00d0170 2024-09-21T03:36:04.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:04.985+0000 7fa5b5252640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa5b00ceb90 0x7fa5b0154720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:04.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:04.985+0000 7fa5b5252640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa5b00ceb90 0x7fa5b0154720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37382/0 (socket says 172.21.15.38:37382) 2024-09-21T03:36:04.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:04.985+0000 7fa5b5252640 1 -- 172.21.15.38:0/3094439400 learned_addr learned my addr 172.21.15.38:0/3094439400 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:36:04.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:04.986+0000 7fa5b4a51640 1 --2- 172.21.15.38:0/3094439400 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa5b00d0170 0x7fa5b0159b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:04.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:04.986+0000 7fa5b5a53640 1 --2- 172.21.15.38:0/3094439400 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa5b015ee80 0x7fa5b015a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:04.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:04.986+0000 7fa5b5252640 1 -- 172.21.15.38:0/3094439400 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa5b00d0170 msgr2=0x7fa5b0159b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:04.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:04.986+0000 7fa5b5252640 1 --2- 172.21.15.38:0/3094439400 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa5b00d0170 0x7fa5b0159b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:04.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:04.986+0000 7fa5b5252640 1 -- 172.21.15.38:0/3094439400 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa5b015ee80 msgr2=0x7fa5b015a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:04.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:04.986+0000 7fa5b5252640 1 --2- 172.21.15.38:0/3094439400 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa5b015ee80 0x7fa5b015a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:04.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:04.986+0000 7fa5b5252640 1 -- 172.21.15.38:0/3094439400 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5b006f6d0 con 0x7fa5b00ceb90 2024-09-21T03:36:04.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:04.986+0000 7fa5b5a53640 1 --2- 172.21.15.38:0/3094439400 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa5b015ee80 0x7fa5b015a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:36:04.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:04.986+0000 7fa5b5252640 1 --2- 172.21.15.38:0/3094439400 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa5b00ceb90 0x7fa5b0154720 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fa5a800b3f0 tx=0x7fa5a800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:04.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:04.986+0000 7fa5adffb640 1 -- 172.21.15.38:0/3094439400 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5a8015020 con 0x7fa5b00ceb90 2024-09-21T03:36:04.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:04.986+0000 7fa5adffb640 1 -- 172.21.15.38:0/3094439400 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa5a8011920 con 0x7fa5b00ceb90 2024-09-21T03:36:04.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:04.986+0000 7fa5b74b6640 1 -- 172.21.15.38:0/3094439400 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5b006f970 con 0x7fa5b00ceb90 2024-09-21T03:36:04.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:04.986+0000 7fa5adffb640 1 -- 172.21.15.38:0/3094439400 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5a8011d70 con 0x7fa5b00ceb90 2024-09-21T03:36:04.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:04.986+0000 7fa5b74b6640 1 -- 172.21.15.38:0/3094439400 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5b006cf10 con 0x7fa5b00ceb90 2024-09-21T03:36:04.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:04.987+0000 7fa5b74b6640 1 -- 172.21.15.38:0/3094439400 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5b00cefa0 con 0x7fa5b00ceb90 2024-09-21T03:36:04.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:04.988+0000 7fa5adffb640 1 -- 172.21.15.38:0/3094439400 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa5a8018020 con 0x7fa5b00ceb90 2024-09-21T03:36:04.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:04.989+0000 7fa5adffb640 1 --2- 172.21.15.38:0/3094439400 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa59c078b90 0x7fa59c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:04.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:04.989+0000 7fa5adffb640 1 -- 172.21.15.38:0/3094439400 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 6187+0+0 (secure 0 0 0) 0x7fa5a809ab50 con 0x7fa5b00ceb90 2024-09-21T03:36:04.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:04.989+0000 7fa5b4a51640 1 --2- 172.21.15.38:0/3094439400 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa59c078b90 0x7fa59c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:04.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:04.990+0000 7fa5b4a51640 1 --2- 172.21.15.38:0/3094439400 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa59c078b90 0x7fa59c07b0a0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fa5b00b3bd0 tx=0x7fa5a4009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:04.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:04.993+0000 7fa5adffb640 1 -- 172.21.15.38:0/3094439400 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fa5a801f030 con 0x7fa5b00ceb90 2024-09-21T03:36:05.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.220+0000 7fa5b74b6640 1 -- 172.21.15.38:0/3094439400 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7fa5b00ce720 con 0x7fa5b00ceb90 2024-09-21T03:36:05.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.480+0000 7fa5adffb640 1 -- 172.21.15.38:0/3094439400 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v33) ==== 162+0+0 (secure 0 0 0) 0x7fa5a8062e40 con 0x7fa5b00ceb90 2024-09-21T03:36:05.483 INFO:teuthology.orchestra.run.smithi038.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-21T03:36:05.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.483+0000 7fa5b74b6640 1 -- 172.21.15.38:0/3094439400 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa59c078b90 msgr2=0x7fa59c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:05.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.483+0000 7fa5b74b6640 1 --2- 172.21.15.38:0/3094439400 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa59c078b90 0x7fa59c07b0a0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fa5b00b3bd0 tx=0x7fa5a4009040 comp rx=0 tx=0).stop 2024-09-21T03:36:05.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.483+0000 7fa5b74b6640 1 -- 172.21.15.38:0/3094439400 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa5b00ceb90 msgr2=0x7fa5b0154720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:05.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.483+0000 7fa5b74b6640 1 --2- 172.21.15.38:0/3094439400 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa5b00ceb90 0x7fa5b0154720 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fa5a800b3f0 tx=0x7fa5a800b8b0 comp rx=0 tx=0).stop 2024-09-21T03:36:05.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.483+0000 7fa5b74b6640 1 -- 172.21.15.38:0/3094439400 shutdown_connections 2024-09-21T03:36:05.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.483+0000 7fa5b74b6640 1 --2- 172.21.15.38:0/3094439400 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa5b015ee80 0x7fa5b015a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:05.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.483+0000 7fa5b74b6640 1 --2- 172.21.15.38:0/3094439400 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa59c078b90 0x7fa59c07b0a0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:05.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.483+0000 7fa5b74b6640 1 --2- 172.21.15.38:0/3094439400 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa5b00d0170 0x7fa5b0159b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:05.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.483+0000 7fa5b74b6640 1 --2- 172.21.15.38:0/3094439400 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa5b00ceb90 0x7fa5b0154720 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:05.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.483+0000 7fa5b74b6640 1 -- 172.21.15.38:0/3094439400 >> 172.21.15.38:0/3094439400 conn(0x7fa5b009d830 msgr2=0x7fa5b0068d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:05.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.483+0000 7fa5b74b6640 1 -- 172.21.15.38:0/3094439400 shutdown_connections 2024-09-21T03:36:05.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.483+0000 7fa5b74b6640 1 -- 172.21.15.38:0/3094439400 wait complete. 2024-09-21T03:36:05.498 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:36:05.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.612+0000 7f068a6f9640 1 -- 172.21.15.38:0/1507072639 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f068406e1d0 msgr2=0x7f06840afd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:05.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.612+0000 7f068a6f9640 1 --2- 172.21.15.38:0/1507072639 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f068406e1d0 0x7f06840afd70 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f0684069870 tx=0x7f067402a560 comp rx=0 tx=0).stop 2024-09-21T03:36:05.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.612+0000 7f068a6f9640 1 -- 172.21.15.38:0/1507072639 shutdown_connections 2024-09-21T03:36:05.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.612+0000 7f068a6f9640 1 --2- 172.21.15.38:0/1507072639 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f06840b4b50 0x7f068406f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:05.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.612+0000 7f068a6f9640 1 --2- 172.21.15.38:0/1507072639 >> v2:172.21.15.195:3300/0 conn(0x7f06840b4740 0x7f068406f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:05.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.612+0000 7f068a6f9640 1 --2- 172.21.15.38:0/1507072639 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f068406e1d0 0x7f06840afd70 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:05.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.612+0000 7f068a6f9640 1 -- 172.21.15.38:0/1507072639 >> 172.21.15.38:0/1507072639 conn(0x7f06840d0a30 msgr2=0x7f06840d0e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:05.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.612+0000 7f068a6f9640 1 -- 172.21.15.38:0/1507072639 shutdown_connections 2024-09-21T03:36:05.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.612+0000 7f068a6f9640 1 -- 172.21.15.38:0/1507072639 wait complete. 2024-09-21T03:36:05.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.614+0000 7f068a6f9640 1 Processor -- start 2024-09-21T03:36:05.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.614+0000 7f068a6f9640 1 -- start start 2024-09-21T03:36:05.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.616+0000 7f068a6f9640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f068406e1d0 0x7f06840ad850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:05.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.616+0000 7f068a6f9640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f06840b4740 0x7f06841f8620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:05.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.616+0000 7f068a6f9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f06840b4b50 0x7f0684158d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:05.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.616+0000 7f068a6f9640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f068415b060 con 0x7f06840b4b50 2024-09-21T03:36:05.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.616+0000 7f068a6f9640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f068415aee0 con 0x7f068406e1d0 2024-09-21T03:36:05.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.616+0000 7f068a6f9640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f068415b1e0 con 0x7f06840b4740 2024-09-21T03:36:05.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.616+0000 7f0688c96640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f06840b4b50 0x7f0684158d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:05.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.616+0000 7f0688c96640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f06840b4b50 0x7f0684158d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37398/0 (socket says 172.21.15.38:37398) 2024-09-21T03:36:05.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.616+0000 7f0688c96640 1 -- 172.21.15.38:0/825577514 learned_addr learned my addr 172.21.15.38:0/825577514 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:36:05.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.616+0000 7f0683fff640 1 --2- 172.21.15.38:0/825577514 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f068406e1d0 0x7f06840ad850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:05.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.616+0000 7f06837fe640 1 --2- 172.21.15.38:0/825577514 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f06840b4740 0x7f06841f8620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:05.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.616+0000 7f0688c96640 1 -- 172.21.15.38:0/825577514 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f06840b4740 msgr2=0x7f06841f8620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:05.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.616+0000 7f0688c96640 1 --2- 172.21.15.38:0/825577514 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f06840b4740 0x7f06841f8620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:05.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.616+0000 7f0688c96640 1 -- 172.21.15.38:0/825577514 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f068406e1d0 msgr2=0x7f06840ad850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:05.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.616+0000 7f0688c96640 1 --2- 172.21.15.38:0/825577514 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f068406e1d0 0x7f06840ad850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:05.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.616+0000 7f0688c96640 1 -- 172.21.15.38:0/825577514 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06841f8b60 con 0x7f06840b4b50 2024-09-21T03:36:05.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.616+0000 7f06837fe640 1 --2- 172.21.15.38:0/825577514 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f06840b4740 0x7f06841f8620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:36:05.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.616+0000 7f0688c96640 1 --2- 172.21.15.38:0/825577514 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f06840b4b50 0x7f0684158d50 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f067c00ba10 tx=0x7f067c00bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:05.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.617+0000 7f0680ff9640 1 -- 172.21.15.38:0/825577514 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f067c019070 con 0x7f06840b4b50 2024-09-21T03:36:05.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.617+0000 7f0680ff9640 1 -- 172.21.15.38:0/825577514 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f067c00cc50 con 0x7f06840b4b50 2024-09-21T03:36:05.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.617+0000 7f068a6f9640 1 -- 172.21.15.38:0/825577514 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06841e3d00 con 0x7f06840b4b50 2024-09-21T03:36:05.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.617+0000 7f0680ff9640 1 -- 172.21.15.38:0/825577514 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f067c0146a0 con 0x7f06840b4b50 2024-09-21T03:36:05.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.617+0000 7f068a6f9640 1 -- 172.21.15.38:0/825577514 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06840ae1d0 con 0x7f06840b4b50 2024-09-21T03:36:05.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.618+0000 7f068a6f9640 1 -- 172.21.15.38:0/825577514 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06840d1120 con 0x7f06840b4b50 2024-09-21T03:36:05.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.619+0000 7f0680ff9640 1 -- 172.21.15.38:0/825577514 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f067c01c070 con 0x7f06840b4b50 2024-09-21T03:36:05.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.619+0000 7f0680ff9640 1 --2- 172.21.15.38:0/825577514 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0664078bc0 0x7f066407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:05.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.619+0000 7f0680ff9640 1 -- 172.21.15.38:0/825577514 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 6187+0+0 (secure 0 0 0) 0x7f067c09ab40 con 0x7f06840b4b50 2024-09-21T03:36:05.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.619+0000 7f0683fff640 1 --2- 172.21.15.38:0/825577514 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0664078bc0 0x7f066407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:05.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.620+0000 7f0683fff640 1 --2- 172.21.15.38:0/825577514 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0664078bc0 0x7f066407b0d0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f0674033000 tx=0x7f0674004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:05.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.624+0000 7f0680ff9640 1 -- 172.21.15.38:0/825577514 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f067c062750 con 0x7f06840b4b50 2024-09-21T03:36:05.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.851+0000 7f068a6f9640 1 -- 172.21.15.38:0/825577514 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f06840d0e40 con 0x7f06840b4b50 2024-09-21T03:36:05.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.852+0000 7f0680ff9640 1 -- 172.21.15.38:0/825577514 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+5223 (secure 0 0 0) 0x7f067c067180 con 0x7f06840b4b50 2024-09-21T03:36:05.854 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:36:05.854 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":33,"btime":"2024-09-21T03:36:05:480416+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5147,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/2858412699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":2858412699},{"type":"v1","addr":"172.21.15.195:6837","nonce":2858412699}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":5368,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/2907144575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":2907144575},{"type":"v1","addr":"172.21.15.195:6836","nonce":2907144575}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":5574,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6834/1911661184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":1911661184},{"type":"v1","addr":"172.21.15.38:6834","nonce":1911661184}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:35:44.271168+0000","modified":"2024-09-21T03:36:05.480402+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":56,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5793},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5793":{"gid":5793,"name":"c","rank":0,"incarnation":33,"state":"up:replay","state_seq":1,"addr":"172.21.15.38:6835/3716808119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6833","nonce":3716808119},{"type":"v1","addr":"172.21.15.38:6835","nonce":3716808119}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":3}]} 2024-09-21T03:36:05.854 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 33 2024-09-21T03:36:05.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.854+0000 7f068a6f9640 1 -- 172.21.15.38:0/825577514 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0664078bc0 msgr2=0x7f066407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:05.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.854+0000 7f068a6f9640 1 --2- 172.21.15.38:0/825577514 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0664078bc0 0x7f066407b0d0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f0674033000 tx=0x7f0674004730 comp rx=0 tx=0).stop 2024-09-21T03:36:05.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.854+0000 7f068a6f9640 1 -- 172.21.15.38:0/825577514 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f06840b4b50 msgr2=0x7f0684158d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:05.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.854+0000 7f068a6f9640 1 --2- 172.21.15.38:0/825577514 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f06840b4b50 0x7f0684158d50 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f067c00ba10 tx=0x7f067c00bed0 comp rx=0 tx=0).stop 2024-09-21T03:36:05.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.854+0000 7f068a6f9640 1 -- 172.21.15.38:0/825577514 shutdown_connections 2024-09-21T03:36:05.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.854+0000 7f068a6f9640 1 --2- 172.21.15.38:0/825577514 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f06840b4b50 0x7f0684158d50 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:05.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.854+0000 7f068a6f9640 1 --2- 172.21.15.38:0/825577514 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0664078bc0 0x7f066407b0d0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:05.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.854+0000 7f068a6f9640 1 --2- 172.21.15.38:0/825577514 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f06840b4740 0x7f06841f8620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:05.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.854+0000 7f068a6f9640 1 --2- 172.21.15.38:0/825577514 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f068406e1d0 0x7f06840ad850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:05.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.854+0000 7f068a6f9640 1 -- 172.21.15.38:0/825577514 >> 172.21.15.38:0/825577514 conn(0x7f06840d0a30 msgr2=0x7f06840cf100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:05.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.854+0000 7f068a6f9640 1 -- 172.21.15.38:0/825577514 shutdown_connections 2024-09-21T03:36:05.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.854+0000 7f068a6f9640 1 -- 172.21.15.38:0/825577514 wait complete. 2024-09-21T03:36:05.867 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 flush journal 2024-09-21T03:36:05.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.984+0000 7f0680cdc640 1 -- 172.21.15.38:0/1759218288 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f067c06def0 msgr2=0x7f067c06f410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:05.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.984+0000 7f0680cdc640 1 --2- 172.21.15.38:0/1759218288 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f067c06def0 0x7f067c06f410 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f0674005fd0 tx=0x7f067402a940 comp rx=0 tx=0).stop 2024-09-21T03:36:05.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.984+0000 7f0680cdc640 1 -- 172.21.15.38:0/1759218288 shutdown_connections 2024-09-21T03:36:05.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.984+0000 7f0680cdc640 1 --2- 172.21.15.38:0/1759218288 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f067c06def0 0x7f067c06f410 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:05.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.984+0000 7f0680cdc640 1 -- 172.21.15.38:0/1759218288 >> 172.21.15.38:0/1759218288 conn(0x7f067c0cdca0 msgr2=0x7f067c0ce0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:05.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.984+0000 7f0680cdc640 1 -- 172.21.15.38:0/1759218288 shutdown_connections 2024-09-21T03:36:05.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.984+0000 7f0680cdc640 1 -- 172.21.15.38:0/1759218288 wait complete. 2024-09-21T03:36:05.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.986+0000 7f0680cdc640 1 Processor -- start 2024-09-21T03:36:05.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.986+0000 7f0680cdc640 1 -- start start 2024-09-21T03:36:05.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.988+0000 7f0680cdc640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f067c06def0 0x7f067c1f2840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:05.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.988+0000 7f0680cdc640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f067c0b5260 0x7f067c1f68a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:05.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.988+0000 7f0680cdc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f067c0b5670 0x7f067c1555c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:05.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.988+0000 7f0680cdc640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f067c071480 con 0x7f067c0b5670 2024-09-21T03:36:05.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.988+0000 7f0680cdc640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f067c071300 con 0x7f067c06def0 2024-09-21T03:36:05.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.988+0000 7f0680cdc640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f067c071600 con 0x7f067c0b5260 2024-09-21T03:36:05.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.988+0000 7f067ad9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f067c0b5670 0x7f067c1555c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:05.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.988+0000 7f067ad9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f067c0b5670 0x7f067c1555c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37406/0 (socket says 172.21.15.38:37406) 2024-09-21T03:36:05.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.988+0000 7f067ad9d640 1 -- 172.21.15.38:0/219850990 learned_addr learned my addr 172.21.15.38:0/219850990 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:36:05.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.988+0000 7f067a59c640 1 --2- 172.21.15.38:0/219850990 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f067c06def0 0x7f067c1f2840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:05.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.988+0000 7f0679d9b640 1 --2- 172.21.15.38:0/219850990 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f067c0b5260 0x7f067c1f68a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:05.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.988+0000 7f067ad9d640 1 -- 172.21.15.38:0/219850990 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f067c0b5260 msgr2=0x7f067c1f68a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:05.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.988+0000 7f067ad9d640 1 --2- 172.21.15.38:0/219850990 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f067c0b5260 0x7f067c1f68a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:05.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.988+0000 7f067ad9d640 1 -- 172.21.15.38:0/219850990 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f067c06def0 msgr2=0x7f067c1f2840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:05.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.988+0000 7f067ad9d640 1 --2- 172.21.15.38:0/219850990 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f067c06def0 0x7f067c1f2840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:05.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.988+0000 7f067ad9d640 1 -- 172.21.15.38:0/219850990 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f067c1f6de0 con 0x7f067c0b5670 2024-09-21T03:36:05.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.988+0000 7f067ad9d640 1 --2- 172.21.15.38:0/219850990 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f067c0b5670 0x7f067c1555c0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f066c00c390 tx=0x7f066c00c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:05.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.988+0000 7f0672ffd640 1 -- 172.21.15.38:0/219850990 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f066c011790 con 0x7f067c0b5670 2024-09-21T03:36:05.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.989+0000 7f0672ffd640 1 -- 172.21.15.38:0/219850990 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f066c017070 con 0x7f067c0b5670 2024-09-21T03:36:05.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.989+0000 7f0680cdc640 1 -- 172.21.15.38:0/219850990 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f067c1f2eb0 con 0x7f067c0b5670 2024-09-21T03:36:05.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.989+0000 7f0672ffd640 1 -- 172.21.15.38:0/219850990 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f066c011c40 con 0x7f067c0b5670 2024-09-21T03:36:05.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.989+0000 7f0680cdc640 1 -- 172.21.15.38:0/219850990 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f067c1f3470 con 0x7f067c0b5670 2024-09-21T03:36:05.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.990+0000 7f0672ffd640 1 -- 172.21.15.38:0/219850990 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f066c01c460 con 0x7f067c0b5670 2024-09-21T03:36:05.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.991+0000 7f0672ffd640 1 --2- 172.21.15.38:0/219850990 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f065c078bc0 0x7f065c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:05.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.991+0000 7f0672ffd640 1 -- 172.21.15.38:0/219850990 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 6187+0+0 (secure 0 0 0) 0x7f066c09ab20 con 0x7f067c0b5670 2024-09-21T03:36:05.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.991+0000 7f067a59c640 1 --2- 172.21.15.38:0/219850990 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f065c078bc0 0x7f065c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:05.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.992+0000 7f067a59c640 1 --2- 172.21.15.38:0/219850990 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f065c078bc0 0x7f065c07b0d0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f0674034000 tx=0x7f0674004ab0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:05.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.992+0000 7f0687c49740 1 Processor -- start 2024-09-21T03:36:05.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.992+0000 7f0687c49740 1 -- start start 2024-09-21T03:36:05.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.993+0000 7f0687c49740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555e38cba390 0x555e390a6a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:05.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.994+0000 7f0687c49740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555e38c914d0 0x555e3907b860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:05.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.994+0000 7f0687c49740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555e38c996b0 0x555e3907bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:05.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.994+0000 7f0687c49740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x555e38fd4a60 con 0x555e38c996b0 2024-09-21T03:36:05.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.994+0000 7f0687c49740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x555e38fc3e90 con 0x555e38c914d0 2024-09-21T03:36:05.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.994+0000 7f0687c49740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x555e38fc7ca0 con 0x555e38cba390 2024-09-21T03:36:05.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.994+0000 7f067ad9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555e38c996b0 0x555e3907bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:05.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.994+0000 7f067ad9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555e38c996b0 0x555e3907bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37412/0 (socket says 172.21.15.38:37412) 2024-09-21T03:36:05.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.994+0000 7f067ad9d640 1 -- 172.21.15.38:0/1584243948 learned_addr learned my addr 172.21.15.38:0/1584243948 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:36:05.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.994+0000 7f067a59c640 1 --2- 172.21.15.38:0/1584243948 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555e38cba390 0x555e390a6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:05.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.994+0000 7f0679d9b640 1 --2- 172.21.15.38:0/1584243948 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555e38c914d0 0x555e3907b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:05.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.994+0000 7f067ad9d640 1 -- 172.21.15.38:0/1584243948 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555e38cba390 msgr2=0x555e390a6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:36:05.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.994+0000 7f067ad9d640 1 --2- 172.21.15.38:0/1584243948 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555e38cba390 0x555e390a6a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:05.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.994+0000 7f067ad9d640 1 -- 172.21.15.38:0/1584243948 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555e38c914d0 msgr2=0x555e3907b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:36:05.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.994+0000 7f067ad9d640 1 --2- 172.21.15.38:0/1584243948 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555e38c914d0 0x555e3907b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:05.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.994+0000 7f067ad9d640 1 -- 172.21.15.38:0/1584243948 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e38d9c7d0 con 0x555e38c996b0 2024-09-21T03:36:05.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.994+0000 7f067a59c640 1 --2- 172.21.15.38:0/1584243948 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555e38cba390 0x555e390a6a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:36:05.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.994+0000 7f0679d9b640 1 --2- 172.21.15.38:0/1584243948 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555e38c914d0 0x555e3907b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:36:05.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.994+0000 7f067ad9d640 1 --2- 172.21.15.38:0/1584243948 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555e38c996b0 0x555e3907bda0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f066c0056e0 tx=0x7f066c061fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec6272a6b5241c3c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:05.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.994+0000 7f065bfff640 1 -- 172.21.15.38:0/1584243948 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f066c067160 con 0x555e38c996b0 2024-09-21T03:36:05.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.994+0000 7f065bfff640 1 -- 172.21.15.38:0/1584243948 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f066c06c0a0 con 0x555e38c996b0 2024-09-21T03:36:05.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.994+0000 7f065bfff640 1 -- 172.21.15.38:0/1584243948 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f066c061990 con 0x555e38c996b0 2024-09-21T03:36:05.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.995+0000 7f0687c49740 1 -- 172.21.15.38:0/1584243948 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555e38c996b0 msgr2=0x555e3907bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:05.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.995+0000 7f0687c49740 1 --2- 172.21.15.38:0/1584243948 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555e38c996b0 0x555e3907bda0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f066c0056e0 tx=0x7f066c061fd0 comp rx=0 tx=0).stop 2024-09-21T03:36:06.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.995+0000 7f0687c49740 1 -- 172.21.15.38:0/1584243948 shutdown_connections 2024-09-21T03:36:06.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.995+0000 7f0687c49740 1 --2- 172.21.15.38:0/1584243948 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555e38c996b0 0x555e3907bda0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:06.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.995+0000 7f0687c49740 1 --2- 172.21.15.38:0/1584243948 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555e38c914d0 0x555e3907b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:06.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.995+0000 7f0687c49740 1 --2- 172.21.15.38:0/1584243948 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555e38cba390 0x555e390a6a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:06.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.995+0000 7f0687c49740 1 -- 172.21.15.38:0/1584243948 >> 172.21.15.38:0/1584243948 conn(0x555e38cae4e0 msgr2=0x555e39061d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:06.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.995+0000 7f0687c49740 1 -- 172.21.15.38:0/1584243948 shutdown_connections 2024-09-21T03:36:06.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.995+0000 7f0687c49740 1 -- 172.21.15.38:0/1584243948 wait complete. 2024-09-21T03:36:06.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.996+0000 7f0687c49740 1 -- start start 2024-09-21T03:36:06.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.996+0000 7f0687c49740 1 Processor -- start 2024-09-21T03:36:06.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.998+0000 7f0687c49740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555e38c914d0 0x555e3909ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:06.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.998+0000 7f0687c49740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555e38c996b0 0x555e390786a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:06.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.998+0000 7f0687c49740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555e38cba390 0x555e3905f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:06.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.998+0000 7f0687c49740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x555e38d34740 con 0x555e38c996b0 2024-09-21T03:36:06.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.998+0000 7f0687c49740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x555e38f69970 con 0x555e38c914d0 2024-09-21T03:36:06.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.998+0000 7f0687c49740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x555e38fd4a60 con 0x555e38cba390 2024-09-21T03:36:06.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.998+0000 7f0687c49740 10 client.0 _ll_register_callbacks cb 0x555e38f78780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:36:06.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.998+0000 7f0679d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555e38c996b0 0x555e390786a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:06.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.998+0000 7f0679d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555e38c996b0 0x555e390786a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37428/0 (socket says 172.21.15.38:37428) 2024-09-21T03:36:06.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.998+0000 7f0679d9b640 1 -- 172.21.15.38:0/1649274174 learned_addr learned my addr 172.21.15.38:0/1649274174 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:36:06.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.998+0000 7f067a59c640 1 --2- 172.21.15.38:0/1649274174 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555e38c914d0 0x555e3909ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:06.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.998+0000 7f067ad9d640 1 --2- 172.21.15.38:0/1649274174 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555e38cba390 0x555e3905f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:06.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.998+0000 7f0679d9b640 1 -- 172.21.15.38:0/1649274174 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555e38cba390 msgr2=0x555e3905f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:36:06.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.998+0000 7f0679d9b640 1 --2- 172.21.15.38:0/1649274174 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555e38cba390 0x555e3905f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:06.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.998+0000 7f0679d9b640 1 -- 172.21.15.38:0/1649274174 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555e38c914d0 msgr2=0x555e3909ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:36:06.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.998+0000 7f0679d9b640 1 --2- 172.21.15.38:0/1649274174 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555e38c914d0 0x555e3909ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:06.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.998+0000 7f0679d9b640 1 -- 172.21.15.38:0/1649274174 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e38d25970 con 0x555e38c996b0 2024-09-21T03:36:06.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.998+0000 7f067a59c640 1 --2- 172.21.15.38:0/1649274174 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555e38c914d0 0x555e3909ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:36:06.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.998+0000 7f0679d9b640 1 --2- 172.21.15.38:0/1649274174 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555e38c996b0 0x555e390786a0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f0664003c70 tx=0x7f0664001590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b47da8f479e109f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:06.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.998+0000 7f06597fa640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-21T03:36:06.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.999+0000 7f06597fa640 1 -- 172.21.15.38:0/1649274174 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0664013540 con 0x555e38c996b0 2024-09-21T03:36:06.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.999+0000 7f06597fa640 1 -- 172.21.15.38:0/1649274174 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(56..56 src has 1..56) ==== 6187+0+0 (secure 0 0 0) 0x7f0664010040 con 0x555e38c996b0 2024-09-21T03:36:06.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.999+0000 7f0687c49740 1 -- 172.21.15.38:0/1649274174 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f066c06c0a0 con 0x555e38c996b0 2024-09-21T03:36:06.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.999+0000 7f06597fa640 1 -- 172.21.15.38:0/1649274174 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f06640136e0 con 0x555e38c996b0 2024-09-21T03:36:06.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.999+0000 7f0687c49740 10 client.5832 fetch_fsmap learned FSMap version 33 2024-09-21T03:36:06.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.999+0000 7f0687c49740 1 -- 172.21.15.38:0/1649274174 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=2+}) -- 0x555e38d036a0 con 0x555e38c996b0 2024-09-21T03:36:06.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.999+0000 7f06597fa640 1 -- 172.21.15.38:0/1649274174 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f066c06c0a0 con 0x555e38c996b0 2024-09-21T03:36:06.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.999+0000 7f06597fa640 1 -- 172.21.15.38:0/1649274174 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 33) ==== 3494+0+0 (secure 0 0 0) 0x7f0664013880 con 0x555e38c996b0 2024-09-21T03:36:06.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.999+0000 7f0687c49740 10 client.5832 fetch_fsmap finished waiting for FSMap version 33 2024-09-21T03:36:06.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.999+0000 7f0687c49740 10 client.5832 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-09-21T03:36:06.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.999+0000 7f0687c49740 20 client.5832 populate_metadata read hostname 'smithi038' 2024-09-21T03:36:06.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.999+0000 7f0687c49740 1 --2- 172.21.15.38:0/1649274174 >> [v2:172.21.15.38:6833/3716808119,v1:172.21.15.38:6835/3716808119] conn(0x555e38c78f60 0x555e3909f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:06.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.999+0000 7f0687c49740 4 client.5832 mds_command: new command op to 5793 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:36:06.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.999+0000 7f0687c49740 1 -- 172.21.15.38:0/1649274174 --> [v2:172.21.15.38:6833/3716808119,v1:172.21.15.38:6835/3716808119] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555e38fd69b0 con 0x555e38c78f60 2024-09-21T03:36:06.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:05.999+0000 7f067a59c640 1 --2- 172.21.15.38:0/1649274174 >> [v2:172.21.15.38:6833/3716808119,v1:172.21.15.38:6835/3716808119] conn(0x555e38c78f60 0x555e3909f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:06.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.000+0000 7f06597fa640 10 client.5832 ms_handle_connect on v2:172.21.15.38:6833/3716808119 2024-09-21T03:36:06.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.000+0000 7f067a59c640 1 --2- 172.21.15.38:0/1649274174 >> [v2:172.21.15.38:6833/3716808119,v1:172.21.15.38:6835/3716808119] conn(0x555e38c78f60 0x555e3909f920 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9b174ac3c036f27 server_cookie=f310ef576d62c141 in_seq=0 out_seq=0 2024-09-21T03:36:06.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.000+0000 7f06597fa640 1 -- 172.21.15.38:0/1649274174 <== mds.0 v2:172.21.15.38:6833/3716808119 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x555e38fd69b0 con 0x555e38c78f60 2024-09-21T03:36:06.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.000+0000 7f06597fa640 10 client.5832 handle_command_reply: tid=0 2024-09-21T03:36:06.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.000+0000 7f0687c49740 1 client.5832 shutdown 2024-09-21T03:36:06.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.001+0000 7f0687c49740 1 -- 172.21.15.38:0/1649274174 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555e38c996b0 msgr2=0x555e390786a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:06.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.001+0000 7f0687c49740 1 --2- 172.21.15.38:0/1649274174 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555e38c996b0 0x555e390786a0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f0664003c70 tx=0x7f0664001590 comp rx=0 tx=0).stop 2024-09-21T03:36:06.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.001+0000 7f0687c49740 1 -- 172.21.15.38:0/1649274174 shutdown_connections 2024-09-21T03:36:06.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.001+0000 7f0687c49740 1 --2- 172.21.15.38:0/1649274174 >> [v2:172.21.15.38:6833/3716808119,v1:172.21.15.38:6835/3716808119] conn(0x555e38c78f60 0x555e3909f920 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:06.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.001+0000 7f0687c49740 1 --2- 172.21.15.38:0/1649274174 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555e38cba390 0x555e3905f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:06.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.001+0000 7f0687c49740 1 --2- 172.21.15.38:0/1649274174 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555e38c996b0 0x555e390786a0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:06.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.001+0000 7f0687c49740 1 --2- 172.21.15.38:0/1649274174 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555e38c914d0 0x555e3909ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:06.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.001+0000 7f06597fa640 0 client.5832 ms_handle_reset on v2:172.21.15.38:6833/3716808119 2024-09-21T03:36:06.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.001+0000 7f0687c49740 1 -- 172.21.15.38:0/1649274174 >> 172.21.15.38:0/1649274174 conn(0x555e38cae4e0 msgr2=0x555e390a2f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:06.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.001+0000 7f0687c49740 1 -- 172.21.15.38:0/1649274174 shutdown_connections 2024-09-21T03:36:06.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.001+0000 7f0687c49740 1 -- 172.21.15.38:0/1649274174 wait complete. 2024-09-21T03:36:06.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.001+0000 7f0687c49740 20 client.5832 trim_cache size 0 max 16384 2024-09-21T03:36:06.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.016+0000 7f0687c49740 1 Processor -- start 2024-09-21T03:36:06.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.016+0000 7f0687c49740 1 -- start start 2024-09-21T03:36:06.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.017+0000 7f0687c49740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555e38c914d0 0x555e390a3540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:06.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.017+0000 7f0687c49740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555e38c996b0 0x555e39088900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:06.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.017+0000 7f0687c49740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555e38cba390 0x555e3905ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:06.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.017+0000 7f0687c49740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f067c1ff610 con 0x555e38c914d0 2024-09-21T03:36:06.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.017+0000 7f0687c49740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f067c1fe640 con 0x555e38cba390 2024-09-21T03:36:06.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.017+0000 7f0687c49740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f067c09b370 con 0x555e38c996b0 2024-09-21T03:36:06.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.017+0000 7f067a59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555e38c914d0 0x555e390a3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:06.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.017+0000 7f067a59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555e38c914d0 0x555e390a3540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37438/0 (socket says 172.21.15.38:37438) 2024-09-21T03:36:06.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.017+0000 7f067a59c640 1 -- 172.21.15.38:0/2361341580 learned_addr learned my addr 172.21.15.38:0/2361341580 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:36:06.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.018+0000 7f0679d9b640 1 --2- 172.21.15.38:0/2361341580 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555e38c996b0 0x555e39088900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:06.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.018+0000 7f067ad9d640 1 --2- 172.21.15.38:0/2361341580 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555e38cba390 0x555e3905ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:06.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.018+0000 7f067a59c640 1 -- 172.21.15.38:0/2361341580 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555e38c996b0 msgr2=0x555e39088900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:36:06.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.018+0000 7f067a59c640 1 --2- 172.21.15.38:0/2361341580 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555e38c996b0 0x555e39088900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:06.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.018+0000 7f067a59c640 1 -- 172.21.15.38:0/2361341580 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555e38cba390 msgr2=0x555e3905ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:36:06.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.018+0000 7f067a59c640 1 --2- 172.21.15.38:0/2361341580 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555e38cba390 0x555e3905ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:06.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.018+0000 7f067a59c640 1 -- 172.21.15.38:0/2361341580 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e3905fad0 con 0x555e38c914d0 2024-09-21T03:36:06.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.018+0000 7f067ad9d640 1 --2- 172.21.15.38:0/2361341580 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555e38cba390 0x555e3905ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:36:06.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.018+0000 7f067a59c640 1 --2- 172.21.15.38:0/2361341580 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555e38c914d0 0x555e390a3540 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f067402af30 tx=0x7f067403d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45d5cb9896fd31d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:06.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.018+0000 7f065b7fe640 1 -- 172.21.15.38:0/2361341580 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0674032d30 con 0x555e38c914d0 2024-09-21T03:36:06.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.018+0000 7f065b7fe640 1 -- 172.21.15.38:0/2361341580 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0674004410 con 0x555e38c914d0 2024-09-21T03:36:06.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.018+0000 7f065b7fe640 1 -- 172.21.15.38:0/2361341580 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0674004840 con 0x555e38c914d0 2024-09-21T03:36:06.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.019+0000 7f0687c49740 1 -- 172.21.15.38:0/2361341580 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555e38c914d0 msgr2=0x555e390a3540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:06.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.019+0000 7f0687c49740 1 --2- 172.21.15.38:0/2361341580 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555e38c914d0 0x555e390a3540 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f067402af30 tx=0x7f067403d040 comp rx=0 tx=0).stop 2024-09-21T03:36:06.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.019+0000 7f0687c49740 1 -- 172.21.15.38:0/2361341580 shutdown_connections 2024-09-21T03:36:06.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.019+0000 7f0687c49740 1 --2- 172.21.15.38:0/2361341580 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555e38cba390 0x555e3905ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:06.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.019+0000 7f0687c49740 1 --2- 172.21.15.38:0/2361341580 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555e38c996b0 0x555e39088900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:06.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.019+0000 7f0687c49740 1 --2- 172.21.15.38:0/2361341580 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555e38c914d0 0x555e390a3540 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:06.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.019+0000 7f0687c49740 1 -- 172.21.15.38:0/2361341580 >> 172.21.15.38:0/2361341580 conn(0x555e38cae4e0 msgr2=0x555e39061870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:06.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.019+0000 7f0687c49740 1 -- 172.21.15.38:0/2361341580 shutdown_connections 2024-09-21T03:36:06.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.019+0000 7f0687c49740 1 -- 172.21.15.38:0/2361341580 wait complete. 2024-09-21T03:36:06.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.020+0000 7f0687c49740 1 -- start start 2024-09-21T03:36:06.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.020+0000 7f0687c49740 1 Processor -- start 2024-09-21T03:36:06.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.021+0000 7f0687c49740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555e38c914d0 0x555e391272a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:06.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.021+0000 7f0687c49740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555e38c996b0 0x555e390a6820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:06.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.021+0000 7f0687c49740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555e38cba390 0x555e390a6d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:06.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.021+0000 7f0687c49740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f067c1fe020 con 0x555e38c996b0 2024-09-21T03:36:06.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.021+0000 7f0687c49740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x555e390c7ac0 con 0x555e38cba390 2024-09-21T03:36:06.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.021+0000 7f0687c49740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x555e390a0290 con 0x555e38c914d0 2024-09-21T03:36:06.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.021+0000 7f0687c49740 10 client.0 _ll_register_callbacks cb 0x7f0664005050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:36:06.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.021+0000 7f0679d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555e38c996b0 0x555e390a6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:06.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.021+0000 7f0679d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555e38c996b0 0x555e390a6820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37450/0 (socket says 172.21.15.38:37450) 2024-09-21T03:36:06.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.021+0000 7f0679d9b640 1 -- 172.21.15.38:0/3988345691 learned_addr learned my addr 172.21.15.38:0/3988345691 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:36:06.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.022+0000 7f067ad9d640 1 --2- 172.21.15.38:0/3988345691 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555e38cba390 0x555e390a6d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:06.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.022+0000 7f067a59c640 1 --2- 172.21.15.38:0/3988345691 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555e38c914d0 0x555e391272a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:06.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.022+0000 7f0679d9b640 1 -- 172.21.15.38:0/3988345691 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555e38c914d0 msgr2=0x555e391272a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:36:06.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.022+0000 7f0679d9b640 1 --2- 172.21.15.38:0/3988345691 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555e38c914d0 0x555e391272a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:06.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.022+0000 7f0679d9b640 1 -- 172.21.15.38:0/3988345691 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555e38cba390 msgr2=0x555e390a6d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:36:06.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.022+0000 7f0679d9b640 1 --2- 172.21.15.38:0/3988345691 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555e38cba390 0x555e390a6d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:06.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.022+0000 7f0679d9b640 1 -- 172.21.15.38:0/3988345691 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e3905d320 con 0x555e38c996b0 2024-09-21T03:36:06.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.022+0000 7f0679d9b640 1 --2- 172.21.15.38:0/3988345691 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555e38c996b0 0x555e390a6820 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f0664001eb0 tx=0x7f06640135c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21291f3720dcd036 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:06.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.022+0000 7f06597fa640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-21T03:36:06.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.022+0000 7f06597fa640 1 -- 172.21.15.38:0/3988345691 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0664013d80 con 0x555e38c996b0 2024-09-21T03:36:06.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.022+0000 7f06597fa640 1 -- 172.21.15.38:0/3988345691 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(56..56 src has 1..56) ==== 6187+0+0 (secure 0 0 0) 0x7f0664015710 con 0x555e38c996b0 2024-09-21T03:36:06.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.022+0000 7f0687c49740 1 -- 172.21.15.38:0/3988345691 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0674004410 con 0x555e38c996b0 2024-09-21T03:36:06.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.022+0000 7f06597fa640 1 -- 172.21.15.38:0/3988345691 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f0664033050 con 0x555e38c996b0 2024-09-21T03:36:06.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.022+0000 7f0687c49740 10 client.5838 fetch_fsmap learned FSMap version 33 2024-09-21T03:36:06.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.022+0000 7f0687c49740 1 -- 172.21.15.38:0/3988345691 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=2+}) -- 0x555e39077d50 con 0x555e38c996b0 2024-09-21T03:36:06.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.022+0000 7f06597fa640 1 -- 172.21.15.38:0/3988345691 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0674004410 con 0x555e38c996b0 2024-09-21T03:36:06.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.022+0000 7f06597fa640 1 -- 172.21.15.38:0/3988345691 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 33) ==== 3494+0+0 (secure 0 0 0) 0x7f0664036020 con 0x555e38c996b0 2024-09-21T03:36:06.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.023+0000 7f0687c49740 10 client.5838 fetch_fsmap finished waiting for FSMap version 33 2024-09-21T03:36:06.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.023+0000 7f0687c49740 10 client.5838 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-09-21T03:36:06.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.023+0000 7f0687c49740 20 client.5838 populate_metadata read hostname 'smithi038' 2024-09-21T03:36:06.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.023+0000 7f0687c49740 1 --2- 172.21.15.38:0/3988345691 >> [v2:172.21.15.38:6833/3716808119,v1:172.21.15.38:6835/3716808119] conn(0x555e3907be20 0x555e390c6ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:06.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.023+0000 7f0687c49740 4 client.5838 mds_command: new command op to 5793 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T03:36:06.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.023+0000 7f0687c49740 1 -- 172.21.15.38:0/3988345691 --> [v2:172.21.15.38:6833/3716808119,v1:172.21.15.38:6835/3716808119] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555e38fd2460 con 0x555e3907be20 2024-09-21T03:36:06.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.023+0000 7f067a59c640 1 --2- 172.21.15.38:0/3988345691 >> [v2:172.21.15.38:6833/3716808119,v1:172.21.15.38:6835/3716808119] conn(0x555e3907be20 0x555e390c6ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:06.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.023+0000 7f06597fa640 10 client.5838 ms_handle_connect on v2:172.21.15.38:6833/3716808119 2024-09-21T03:36:06.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.023+0000 7f067a59c640 1 --2- 172.21.15.38:0/3988345691 >> [v2:172.21.15.38:6833/3716808119,v1:172.21.15.38:6835/3716808119] conn(0x555e3907be20 0x555e390c6ba0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=473b729f02d7cea0 server_cookie=dd51dfc65f30b663 in_seq=0 out_seq=0 2024-09-21T03:36:06.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.023+0000 7f06597fa640 1 -- 172.21.15.38:0/3988345691 <== mds.0 v2:172.21.15.38:6833/3716808119 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555e38fd2460 con 0x555e3907be20 2024-09-21T03:36:06.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.023+0000 7f06597fa640 10 client.5838 handle_command_reply: tid=0 2024-09-21T03:36:06.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.023+0000 7f0687c49740 1 client.5838 shutdown 2024-09-21T03:36:06.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.024+0000 7f0687c49740 1 -- 172.21.15.38:0/3988345691 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555e38c996b0 msgr2=0x555e390a6820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:06.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.024+0000 7f0687c49740 1 --2- 172.21.15.38:0/3988345691 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555e38c996b0 0x555e390a6820 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f0664001eb0 tx=0x7f06640135c0 comp rx=0 tx=0).stop 2024-09-21T03:36:06.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.024+0000 7f0687c49740 1 -- 172.21.15.38:0/3988345691 shutdown_connections 2024-09-21T03:36:06.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.024+0000 7f0687c49740 1 --2- 172.21.15.38:0/3988345691 >> [v2:172.21.15.38:6833/3716808119,v1:172.21.15.38:6835/3716808119] conn(0x555e3907be20 0x555e390c6ba0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:06.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.024+0000 7f0687c49740 1 --2- 172.21.15.38:0/3988345691 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555e38cba390 0x555e390a6d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:06.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.024+0000 7f0687c49740 1 --2- 172.21.15.38:0/3988345691 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555e38c996b0 0x555e390a6820 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:06.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.024+0000 7f0687c49740 1 --2- 172.21.15.38:0/3988345691 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555e38c914d0 0x555e391272a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:06.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.024+0000 7f06597fa640 0 client.5838 ms_handle_reset on v2:172.21.15.38:6833/3716808119 2024-09-21T03:36:06.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.024+0000 7f0687c49740 1 -- 172.21.15.38:0/3988345691 >> 172.21.15.38:0/3988345691 conn(0x555e38cae4e0 msgr2=0x555e39061380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:06.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.024+0000 7f0687c49740 1 -- 172.21.15.38:0/3988345691 shutdown_connections 2024-09-21T03:36:06.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.024+0000 7f0687c49740 1 -- 172.21.15.38:0/3988345691 wait complete. 2024-09-21T03:36:06.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.024+0000 7f0687c49740 20 client.5838 trim_cache size 0 max 16384 2024-09-21T03:36:06.031 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-09-21T03:36:06.031 INFO:teuthology.orchestra.run.smithi038.stdout: "message": "", 2024-09-21T03:36:06.031 INFO:teuthology.orchestra.run.smithi038.stdout: "return_code": 0 2024-09-21T03:36:06.031 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-09-21T03:36:06.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.024+0000 7f0680cdc640 1 -- 172.21.15.38:0/219850990 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f065c078bc0 msgr2=0x7f065c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:06.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.024+0000 7f0680cdc640 1 --2- 172.21.15.38:0/219850990 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f065c078bc0 0x7f065c07b0d0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f0674034000 tx=0x7f0674004ab0 comp rx=0 tx=0).stop 2024-09-21T03:36:06.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.024+0000 7f0680cdc640 1 -- 172.21.15.38:0/219850990 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f067c0b5670 msgr2=0x7f067c1555c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:06.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.024+0000 7f0680cdc640 1 --2- 172.21.15.38:0/219850990 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f067c0b5670 0x7f067c1555c0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f066c00c390 tx=0x7f066c00c850 comp rx=0 tx=0).stop 2024-09-21T03:36:06.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.024+0000 7f0680cdc640 1 -- 172.21.15.38:0/219850990 shutdown_connections 2024-09-21T03:36:06.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.024+0000 7f0680cdc640 1 --2- 172.21.15.38:0/219850990 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f067c0b5670 0x7f067c1555c0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:06.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.024+0000 7f0680cdc640 1 --2- 172.21.15.38:0/219850990 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f065c078bc0 0x7f065c07b0d0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:06.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.024+0000 7f0680cdc640 1 --2- 172.21.15.38:0/219850990 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f067c0b5260 0x7f067c1f68a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:06.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.024+0000 7f0680cdc640 1 --2- 172.21.15.38:0/219850990 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f067c06def0 0x7f067c1f2840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:06.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.024+0000 7f0680cdc640 1 -- 172.21.15.38:0/219850990 >> 172.21.15.38:0/219850990 conn(0x7f067c0cdca0 msgr2=0x7f067c0d11b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:06.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.024+0000 7f0680cdc640 1 -- 172.21.15.38:0/219850990 shutdown_connections 2024-09-21T03:36:06.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.024+0000 7f0680cdc640 1 -- 172.21.15.38:0/219850990 wait complete. 2024-09-21T03:36:06.037 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:36:06.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.151+0000 7f206ebf1640 1 -- 172.21.15.38:0/1333767981 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f206806ec30 msgr2=0x7f20680b3fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:06.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.151+0000 7f206ebf1640 1 --2- 172.21.15.38:0/1333767981 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f206806ec30 0x7f20680b3fc0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f2068075360 tx=0x7f205402a560 comp rx=0 tx=0).stop 2024-09-21T03:36:06.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.151+0000 7f206ebf1640 1 -- 172.21.15.38:0/1333767981 shutdown_connections 2024-09-21T03:36:06.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.151+0000 7f206ebf1640 1 --2- 172.21.15.38:0/1333767981 >> v2:172.21.15.195:3300/0 conn(0x7f20680b4500 0x7f2068073d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:06.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.151+0000 7f206ebf1640 1 --2- 172.21.15.38:0/1333767981 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f206806ec30 0x7f20680b3fc0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:06.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.151+0000 7f206ebf1640 1 --2- 172.21.15.38:0/1333767981 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f20680d0b40 0x7f206806e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:06.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.151+0000 7f206ebf1640 1 -- 172.21.15.38:0/1333767981 >> 172.21.15.38:0/1333767981 conn(0x7f20680b2cc0 msgr2=0x7f20680b30d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:06.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.151+0000 7f206ebf1640 1 -- 172.21.15.38:0/1333767981 shutdown_connections 2024-09-21T03:36:06.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.152+0000 7f206ebf1640 1 -- 172.21.15.38:0/1333767981 wait complete. 2024-09-21T03:36:06.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.153+0000 7f206ebf1640 1 Processor -- start 2024-09-21T03:36:06.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.153+0000 7f206ebf1640 1 -- start start 2024-09-21T03:36:06.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.155+0000 7f206ebf1640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f206806ec30 0x7f2068154340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:06.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.155+0000 7f206ebf1640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f20680b4500 0x7f2068161d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:06.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.155+0000 7f206ebf1640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f20680d0b40 0x7f2068162290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:06.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.155+0000 7f206ebf1640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f206815b750 con 0x7f20680b4500 2024-09-21T03:36:06.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.155+0000 7f206ebf1640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f206815b5d0 con 0x7f20680d0b40 2024-09-21T03:36:06.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.155+0000 7f206ebf1640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f206815b8d0 con 0x7f206806ec30 2024-09-21T03:36:06.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.155+0000 7f2067fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f20680b4500 0x7f2068161d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:06.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.155+0000 7f2067fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f20680b4500 0x7f2068161d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37462/0 (socket says 172.21.15.38:37462) 2024-09-21T03:36:06.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.155+0000 7f2067fff640 1 -- 172.21.15.38:0/3613880245 learned_addr learned my addr 172.21.15.38:0/3613880245 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:36:06.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.155+0000 7f206c98d640 1 --2- 172.21.15.38:0/3613880245 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f206806ec30 0x7f2068154340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:06.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.155+0000 7f206d18e640 1 --2- 172.21.15.38:0/3613880245 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f20680d0b40 0x7f2068162290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:06.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.155+0000 7f2067fff640 1 -- 172.21.15.38:0/3613880245 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f206806ec30 msgr2=0x7f2068154340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:06.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.155+0000 7f2067fff640 1 --2- 172.21.15.38:0/3613880245 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f206806ec30 0x7f2068154340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:06.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.155+0000 7f2067fff640 1 -- 172.21.15.38:0/3613880245 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f20680d0b40 msgr2=0x7f2068162290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:06.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.155+0000 7f2067fff640 1 --2- 172.21.15.38:0/3613880245 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f20680d0b40 0x7f2068162290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:06.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.155+0000 7f2067fff640 1 -- 172.21.15.38:0/3613880245 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20681612b0 con 0x7f20680b4500 2024-09-21T03:36:06.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.156+0000 7f2067fff640 1 --2- 172.21.15.38:0/3613880245 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f20680b4500 0x7f2068161d50 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f2054034040 tx=0x7f2054003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:06.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.156+0000 7f20657fa640 1 -- 172.21.15.38:0/3613880245 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f205403e030 con 0x7f20680b4500 2024-09-21T03:36:06.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.156+0000 7f20657fa640 1 -- 172.21.15.38:0/3613880245 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2054003e00 con 0x7f20680b4500 2024-09-21T03:36:06.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.156+0000 7f206ebf1640 1 -- 172.21.15.38:0/3613880245 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f206815f640 con 0x7f20680b4500 2024-09-21T03:36:06.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.156+0000 7f20657fa640 1 -- 172.21.15.38:0/3613880245 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2054035040 con 0x7f20680b4500 2024-09-21T03:36:06.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.156+0000 7f206ebf1640 1 -- 172.21.15.38:0/3613880245 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2068161830 con 0x7f20680b4500 2024-09-21T03:36:06.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.157+0000 7f206ebf1640 1 -- 172.21.15.38:0/3613880245 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2068073d50 con 0x7f20680b4500 2024-09-21T03:36:06.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.158+0000 7f20657fa640 1 -- 172.21.15.38:0/3613880245 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f2054041030 con 0x7f20680b4500 2024-09-21T03:36:06.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.158+0000 7f20657fa640 1 --2- 172.21.15.38:0/3613880245 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f204c078b90 0x7f204c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:06.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.158+0000 7f20657fa640 1 -- 172.21.15.38:0/3613880245 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 6187+0+0 (secure 0 0 0) 0x7f20540bb670 con 0x7f20680b4500 2024-09-21T03:36:06.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.159+0000 7f206c98d640 1 --2- 172.21.15.38:0/3613880245 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f204c078b90 0x7f204c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:06.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.159+0000 7f206c98d640 1 --2- 172.21.15.38:0/3613880245 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f204c078b90 0x7f204c07b0a0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f205c00a490 tx=0x7f205c00a420 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:06.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.163+0000 7f20657fa640 1 -- 172.21.15.38:0/3613880245 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f2054083280 con 0x7f20680b4500 2024-09-21T03:36:06.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.387+0000 7f206ebf1640 1 -- 172.21.15.38:0/3613880245 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f206806d120 con 0x7f20680b4500 2024-09-21T03:36:06.390 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.388+0000 7f20657fa640 1 -- 172.21.15.38:0/3613880245 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+5223 (secure 0 0 0) 0x7f2054087cb0 con 0x7f20680b4500 2024-09-21T03:36:06.390 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:36:06.390 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":33,"btime":"2024-09-21T03:36:05:480416+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5147,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/2858412699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":2858412699},{"type":"v1","addr":"172.21.15.195:6837","nonce":2858412699}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":5368,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/2907144575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":2907144575},{"type":"v1","addr":"172.21.15.195:6836","nonce":2907144575}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":5574,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6834/1911661184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":1911661184},{"type":"v1","addr":"172.21.15.38:6834","nonce":1911661184}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:35:44.271168+0000","modified":"2024-09-21T03:36:05.480402+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":56,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5793},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5793":{"gid":5793,"name":"c","rank":0,"incarnation":33,"state":"up:replay","state_seq":1,"addr":"172.21.15.38:6835/3716808119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6833","nonce":3716808119},{"type":"v1","addr":"172.21.15.38:6835","nonce":3716808119}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":3}]} 2024-09-21T03:36:06.390 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 33 2024-09-21T03:36:06.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.390+0000 7f206ebf1640 1 -- 172.21.15.38:0/3613880245 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f204c078b90 msgr2=0x7f204c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:06.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.390+0000 7f206ebf1640 1 --2- 172.21.15.38:0/3613880245 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f204c078b90 0x7f204c07b0a0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f205c00a490 tx=0x7f205c00a420 comp rx=0 tx=0).stop 2024-09-21T03:36:06.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.390+0000 7f206ebf1640 1 -- 172.21.15.38:0/3613880245 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f20680b4500 msgr2=0x7f2068161d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:06.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.390+0000 7f206ebf1640 1 --2- 172.21.15.38:0/3613880245 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f20680b4500 0x7f2068161d50 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f2054034040 tx=0x7f2054003910 comp rx=0 tx=0).stop 2024-09-21T03:36:06.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.390+0000 7f206ebf1640 1 -- 172.21.15.38:0/3613880245 shutdown_connections 2024-09-21T03:36:06.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.390+0000 7f206ebf1640 1 --2- 172.21.15.38:0/3613880245 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f20680d0b40 0x7f2068162290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:06.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.390+0000 7f206ebf1640 1 --2- 172.21.15.38:0/3613880245 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f20680b4500 0x7f2068161d50 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:06.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.390+0000 7f206ebf1640 1 --2- 172.21.15.38:0/3613880245 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f204c078b90 0x7f204c07b0a0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:06.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.390+0000 7f206ebf1640 1 --2- 172.21.15.38:0/3613880245 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f206806ec30 0x7f2068154340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:06.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.390+0000 7f206ebf1640 1 -- 172.21.15.38:0/3613880245 >> 172.21.15.38:0/3613880245 conn(0x7f20680b2cc0 msgr2=0x7f20680d0010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:06.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.390+0000 7f206ebf1640 1 -- 172.21.15.38:0/3613880245 shutdown_connections 2024-09-21T03:36:06.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.390+0000 7f206ebf1640 1 -- 172.21.15.38:0/3613880245 wait complete. 2024-09-21T03:36:06.403 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:36:06.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.518+0000 7efc317c6640 1 -- 172.21.15.38:0/3433399433 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efc2c0b2e60 msgr2=0x7efc2c0b3270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:06.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.518+0000 7efc317c6640 1 --2- 172.21.15.38:0/3433399433 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efc2c0b2e60 0x7efc2c0b3270 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7efc2c08d3b0 tx=0x7efc2402a560 comp rx=0 tx=0).stop 2024-09-21T03:36:06.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.518+0000 7efc317c6640 1 -- 172.21.15.38:0/3433399433 shutdown_connections 2024-09-21T03:36:06.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.518+0000 7efc317c6640 1 --2- 172.21.15.38:0/3433399433 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efc2c0b2e60 0x7efc2c0b3270 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:06.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.518+0000 7efc317c6640 1 --2- 172.21.15.38:0/3433399433 >> v2:172.21.15.195:3300/0 conn(0x7efc2c0cfe00 0x7efc2c0d2a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:06.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.518+0000 7efc317c6640 1 -- 172.21.15.38:0/3433399433 >> 172.21.15.38:0/3433399433 conn(0x7efc2c0d4440 msgr2=0x7efc2c0d4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:06.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.518+0000 7efc317c6640 1 -- 172.21.15.38:0/3433399433 shutdown_connections 2024-09-21T03:36:06.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.518+0000 7efc317c6640 1 -- 172.21.15.38:0/3433399433 wait complete. 2024-09-21T03:36:06.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.520+0000 7efc317c6640 1 Processor -- start 2024-09-21T03:36:06.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.520+0000 7efc317c6640 1 -- start start 2024-09-21T03:36:06.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.521+0000 7efc317c6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efc2c0b2e60 0x7efc2c15b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:06.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.521+0000 7efc317c6640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efc2c0cfe00 0x7efc2c1595d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:06.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.521+0000 7efc317c6640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efc2c162590 0x7efc2c155710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:06.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.521+0000 7efc317c6640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7efc2c0b3930 con 0x7efc2c0b2e60 2024-09-21T03:36:06.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.521+0000 7efc317c6640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7efc2c0b37b0 con 0x7efc2c162590 2024-09-21T03:36:06.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.521+0000 7efc317c6640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7efc2c0b3ab0 con 0x7efc2c0cfe00 2024-09-21T03:36:06.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.521+0000 7efc2affd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efc2c0b2e60 0x7efc2c15b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:06.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.521+0000 7efc2affd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efc2c0b2e60 0x7efc2c15b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37474/0 (socket says 172.21.15.38:37474) 2024-09-21T03:36:06.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.521+0000 7efc2affd640 1 -- 172.21.15.38:0/1040565040 learned_addr learned my addr 172.21.15.38:0/1040565040 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:36:06.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.522+0000 7efc2b7fe640 1 --2- 172.21.15.38:0/1040565040 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efc2c162590 0x7efc2c155710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:06.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.522+0000 7efc2affd640 1 -- 172.21.15.38:0/1040565040 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efc2c0cfe00 msgr2=0x7efc2c1595d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:06.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.522+0000 7efc2a7fc640 1 --2- 172.21.15.38:0/1040565040 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efc2c0cfe00 0x7efc2c1595d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:06.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.522+0000 7efc2affd640 1 --2- 172.21.15.38:0/1040565040 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efc2c0cfe00 0x7efc2c1595d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:06.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.522+0000 7efc2affd640 1 -- 172.21.15.38:0/1040565040 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efc2c162590 msgr2=0x7efc2c155710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:06.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.522+0000 7efc2affd640 1 --2- 172.21.15.38:0/1040565040 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efc2c162590 0x7efc2c155710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:06.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.522+0000 7efc2affd640 1 -- 172.21.15.38:0/1040565040 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc2c1eb320 con 0x7efc2c0b2e60 2024-09-21T03:36:06.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.522+0000 7efc2b7fe640 1 --2- 172.21.15.38:0/1040565040 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efc2c162590 0x7efc2c155710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:36:06.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.522+0000 7efc2affd640 1 --2- 172.21.15.38:0/1040565040 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efc2c0b2e60 0x7efc2c15b900 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7efc1c00ba80 tx=0x7efc1c00bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:06.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.522+0000 7efc237fe640 1 -- 172.21.15.38:0/1040565040 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc1c019070 con 0x7efc2c0b2e60 2024-09-21T03:36:06.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.522+0000 7efc237fe640 1 -- 172.21.15.38:0/1040565040 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efc1c00cd10 con 0x7efc2c0b2e60 2024-09-21T03:36:06.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.522+0000 7efc317c6640 1 -- 172.21.15.38:0/1040565040 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc2c1eb600 con 0x7efc2c0b2e60 2024-09-21T03:36:06.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.522+0000 7efc237fe640 1 -- 172.21.15.38:0/1040565040 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc1c0146a0 con 0x7efc2c0b2e60 2024-09-21T03:36:06.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.522+0000 7efc317c6640 1 -- 172.21.15.38:0/1040565040 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc2c1e8890 con 0x7efc2c0b2e60 2024-09-21T03:36:06.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.523+0000 7efc317c6640 1 -- 172.21.15.38:0/1040565040 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc2c0b3270 con 0x7efc2c0b2e60 2024-09-21T03:36:06.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.524+0000 7efc237fe640 1 -- 172.21.15.38:0/1040565040 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7efc1c01c070 con 0x7efc2c0b2e60 2024-09-21T03:36:06.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.525+0000 7efc237fe640 1 --2- 172.21.15.38:0/1040565040 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7efc0c078b90 0x7efc0c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:06.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.525+0000 7efc237fe640 1 -- 172.21.15.38:0/1040565040 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 6187+0+0 (secure 0 0 0) 0x7efc1c09aca0 con 0x7efc2c0b2e60 2024-09-21T03:36:06.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.525+0000 7efc2a7fc640 1 --2- 172.21.15.38:0/1040565040 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7efc0c078b90 0x7efc0c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:06.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.526+0000 7efc2a7fc640 1 --2- 172.21.15.38:0/1040565040 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7efc0c078b90 0x7efc0c07b0a0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7efc2c161bb0 tx=0x7efc14008040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:06.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.529+0000 7efc237fe640 1 -- 172.21.15.38:0/1040565040 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7efc1c063020 con 0x7efc2c0b2e60 2024-09-21T03:36:06.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.755+0000 7efc317c6640 1 -- 172.21.15.38:0/1040565040 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efc2c156080 con 0x7efc2c0b2e60 2024-09-21T03:36:06.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.755+0000 7efc237fe640 1 -- 172.21.15.38:0/1040565040 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+5226 (secure 0 0 0) 0x7efc1c062920 con 0x7efc2c0b2e60 2024-09-21T03:36:06.757 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:36:06.757 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":34,"btime":"2024-09-21T03:36:06:481772+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5147,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/2858412699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":2858412699},{"type":"v1","addr":"172.21.15.195:6837","nonce":2858412699}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":5368,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/2907144575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":2907144575},{"type":"v1","addr":"172.21.15.195:6836","nonce":2907144575}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":5574,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6834/1911661184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":1911661184},{"type":"v1","addr":"172.21.15.38:6834","nonce":1911661184}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:35:44.271168+0000","modified":"2024-09-21T03:36:05.491338+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":56,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5793},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5793":{"gid":5793,"name":"c","rank":0,"incarnation":33,"state":"up:reconnect","state_seq":2,"addr":"172.21.15.38:6835/3716808119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6833","nonce":3716808119},{"type":"v1","addr":"172.21.15.38:6835","nonce":3716808119}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":3}]} 2024-09-21T03:36:06.758 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 34 2024-09-21T03:36:06.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.758+0000 7efc317c6640 1 -- 172.21.15.38:0/1040565040 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7efc0c078b90 msgr2=0x7efc0c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:06.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.758+0000 7efc317c6640 1 --2- 172.21.15.38:0/1040565040 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7efc0c078b90 0x7efc0c07b0a0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7efc2c161bb0 tx=0x7efc14008040 comp rx=0 tx=0).stop 2024-09-21T03:36:06.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.758+0000 7efc317c6640 1 -- 172.21.15.38:0/1040565040 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efc2c0b2e60 msgr2=0x7efc2c15b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:06.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.758+0000 7efc317c6640 1 --2- 172.21.15.38:0/1040565040 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efc2c0b2e60 0x7efc2c15b900 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7efc1c00ba80 tx=0x7efc1c00bf40 comp rx=0 tx=0).stop 2024-09-21T03:36:06.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.758+0000 7efc317c6640 1 -- 172.21.15.38:0/1040565040 shutdown_connections 2024-09-21T03:36:06.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.758+0000 7efc317c6640 1 --2- 172.21.15.38:0/1040565040 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efc2c162590 0x7efc2c155710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:06.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.758+0000 7efc317c6640 1 --2- 172.21.15.38:0/1040565040 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7efc0c078b90 0x7efc0c07b0a0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:06.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.758+0000 7efc317c6640 1 --2- 172.21.15.38:0/1040565040 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efc2c0cfe00 0x7efc2c1595d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:06.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.758+0000 7efc317c6640 1 --2- 172.21.15.38:0/1040565040 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efc2c0b2e60 0x7efc2c15b900 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:06.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.758+0000 7efc317c6640 1 -- 172.21.15.38:0/1040565040 >> 172.21.15.38:0/1040565040 conn(0x7efc2c0d4440 msgr2=0x7efc2c06cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:06.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.758+0000 7efc317c6640 1 -- 172.21.15.38:0/1040565040 shutdown_connections 2024-09-21T03:36:06.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.758+0000 7efc317c6640 1 -- 172.21.15.38:0/1040565040 wait complete. 2024-09-21T03:36:06.771 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:36:06.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.885+0000 7f6d85c86640 1 -- 172.21.15.38:0/1271082218 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6d8015cb10 msgr2=0x7f6d8015cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:06.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.885+0000 7f6d85c86640 1 --2- 172.21.15.38:0/1271082218 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6d8015cb10 0x7f6d8015cf20 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f6d800d2320 tx=0x7f6d7002a560 comp rx=0 tx=0).stop 2024-09-21T03:36:06.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.885+0000 7f6d85c86640 1 -- 172.21.15.38:0/1271082218 shutdown_connections 2024-09-21T03:36:06.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.885+0000 7f6d85c86640 1 --2- 172.21.15.38:0/1271082218 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6d80160100 0x7f6d80160510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:06.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.885+0000 7f6d85c86640 1 --2- 172.21.15.38:0/1271082218 >> v2:172.21.15.195:3300/0 conn(0x7f6d800d12e0 0x7f6d800d16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:06.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.885+0000 7f6d85c86640 1 --2- 172.21.15.38:0/1271082218 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6d8015cb10 0x7f6d8015cf20 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:06.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.885+0000 7f6d85c86640 1 -- 172.21.15.38:0/1271082218 >> 172.21.15.38:0/1271082218 conn(0x7f6d800d2350 msgr2=0x7f6d800d2760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:06.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.885+0000 7f6d85c86640 1 -- 172.21.15.38:0/1271082218 shutdown_connections 2024-09-21T03:36:06.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.885+0000 7f6d85c86640 1 -- 172.21.15.38:0/1271082218 wait complete. 2024-09-21T03:36:06.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.887+0000 7f6d85c86640 1 Processor -- start 2024-09-21T03:36:06.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.887+0000 7f6d85c86640 1 -- start start 2024-09-21T03:36:06.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.889+0000 7f6d85c86640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6d800d12e0 0x7f6d80143d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:06.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.889+0000 7f6d85c86640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6d8015cb10 0x7f6d80150c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:06.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.889+0000 7f6d85c86640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6d80160100 0x7f6d8015b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:06.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.889+0000 7f6d85c86640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f6d80160bd0 con 0x7f6d8015cb10 2024-09-21T03:36:06.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.889+0000 7f6d85c86640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f6d80160a50 con 0x7f6d80160100 2024-09-21T03:36:06.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.889+0000 7f6d85c86640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f6d80160d50 con 0x7f6d800d12e0 2024-09-21T03:36:06.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.889+0000 7f6d7effd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6d8015cb10 0x7f6d80150c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:06.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.889+0000 7f6d7effd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6d8015cb10 0x7f6d80150c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37492/0 (socket says 172.21.15.38:37492) 2024-09-21T03:36:06.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.889+0000 7f6d7effd640 1 -- 172.21.15.38:0/3459572996 learned_addr learned my addr 172.21.15.38:0/3459572996 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:36:06.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.889+0000 7f6d7ffff640 1 --2- 172.21.15.38:0/3459572996 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6d80160100 0x7f6d8015b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:06.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.889+0000 7f6d7f7fe640 1 --2- 172.21.15.38:0/3459572996 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6d800d12e0 0x7f6d80143d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:06.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.889+0000 7f6d7effd640 1 -- 172.21.15.38:0/3459572996 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6d800d12e0 msgr2=0x7f6d80143d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:06.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.889+0000 7f6d7effd640 1 --2- 172.21.15.38:0/3459572996 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6d800d12e0 0x7f6d80143d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:06.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.889+0000 7f6d7effd640 1 -- 172.21.15.38:0/3459572996 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6d80160100 msgr2=0x7f6d8015b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:06.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.889+0000 7f6d7effd640 1 --2- 172.21.15.38:0/3459572996 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6d80160100 0x7f6d8015b5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:06.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.889+0000 7f6d7effd640 1 -- 172.21.15.38:0/3459572996 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d801e65c0 con 0x7f6d8015cb10 2024-09-21T03:36:06.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.889+0000 7f6d7effd640 1 --2- 172.21.15.38:0/3459572996 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6d8015cb10 0x7f6d80150c90 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f6d6800d720 tx=0x7f6d6800dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:06.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.890+0000 7f6d77fff640 1 -- 172.21.15.38:0/3459572996 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d68017070 con 0x7f6d8015cb10 2024-09-21T03:36:06.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.890+0000 7f6d85c86640 1 -- 172.21.15.38:0/3459572996 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d8006e630 con 0x7f6d8015cb10 2024-09-21T03:36:06.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.890+0000 7f6d77fff640 1 -- 172.21.15.38:0/3459572996 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6d68009ab0 con 0x7f6d8015cb10 2024-09-21T03:36:06.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.890+0000 7f6d77fff640 1 -- 172.21.15.38:0/3459572996 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d68010460 con 0x7f6d8015cb10 2024-09-21T03:36:06.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.890+0000 7f6d85c86640 1 -- 172.21.15.38:0/3459572996 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d801e6a30 con 0x7f6d8015cb10 2024-09-21T03:36:06.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.891+0000 7f6d85c86640 1 -- 172.21.15.38:0/3459572996 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d8015cf20 con 0x7f6d8015cb10 2024-09-21T03:36:06.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.892+0000 7f6d77fff640 1 -- 172.21.15.38:0/3459572996 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f6d6801a070 con 0x7f6d8015cb10 2024-09-21T03:36:06.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.892+0000 7f6d77fff640 1 --2- 172.21.15.38:0/3459572996 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6d60078b90 0x7f6d6007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:06.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.892+0000 7f6d77fff640 1 -- 172.21.15.38:0/3459572996 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 6187+0+0 (secure 0 0 0) 0x7f6d6809b030 con 0x7f6d8015cb10 2024-09-21T03:36:06.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.892+0000 7f6d7f7fe640 1 --2- 172.21.15.38:0/3459572996 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6d60078b90 0x7f6d6007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:06.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.893+0000 7f6d7f7fe640 1 --2- 172.21.15.38:0/3459572996 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6d60078b90 0x7f6d6007b0a0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f6d70033000 tx=0x7f6d70004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:06.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:06.897+0000 7f6d77fff640 1 -- 172.21.15.38:0/3459572996 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f6d68062c50 con 0x7f6d8015cb10 2024-09-21T03:36:07.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:07.125+0000 7f6d85c86640 1 -- 172.21.15.38:0/3459572996 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6d801e6790 con 0x7f6d8015cb10 2024-09-21T03:36:07.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:07.126+0000 7f6d77fff640 1 -- 172.21.15.38:0/3459572996 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+5226 (secure 0 0 0) 0x7f6d68067680 con 0x7f6d8015cb10 2024-09-21T03:36:07.128 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:36:07.128 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":34,"btime":"2024-09-21T03:36:06:481772+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5147,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/2858412699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":2858412699},{"type":"v1","addr":"172.21.15.195:6837","nonce":2858412699}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":5368,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/2907144575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":2907144575},{"type":"v1","addr":"172.21.15.195:6836","nonce":2907144575}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":5574,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6834/1911661184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":1911661184},{"type":"v1","addr":"172.21.15.38:6834","nonce":1911661184}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:35:44.271168+0000","modified":"2024-09-21T03:36:05.491338+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":56,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5793},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5793":{"gid":5793,"name":"c","rank":0,"incarnation":33,"state":"up:reconnect","state_seq":2,"addr":"172.21.15.38:6835/3716808119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6833","nonce":3716808119},{"type":"v1","addr":"172.21.15.38:6835","nonce":3716808119}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":3}]} 2024-09-21T03:36:07.128 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 34 2024-09-21T03:36:07.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:07.128+0000 7f6d85c86640 1 -- 172.21.15.38:0/3459572996 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6d60078b90 msgr2=0x7f6d6007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:07.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:07.128+0000 7f6d85c86640 1 --2- 172.21.15.38:0/3459572996 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6d60078b90 0x7f6d6007b0a0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f6d70033000 tx=0x7f6d70004730 comp rx=0 tx=0).stop 2024-09-21T03:36:07.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:07.128+0000 7f6d85c86640 1 -- 172.21.15.38:0/3459572996 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6d8015cb10 msgr2=0x7f6d80150c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:07.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:07.128+0000 7f6d85c86640 1 --2- 172.21.15.38:0/3459572996 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6d8015cb10 0x7f6d80150c90 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f6d6800d720 tx=0x7f6d6800dbe0 comp rx=0 tx=0).stop 2024-09-21T03:36:07.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:07.128+0000 7f6d85c86640 1 -- 172.21.15.38:0/3459572996 shutdown_connections 2024-09-21T03:36:07.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:07.128+0000 7f6d85c86640 1 --2- 172.21.15.38:0/3459572996 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6d80160100 0x7f6d8015b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:07.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:07.128+0000 7f6d85c86640 1 --2- 172.21.15.38:0/3459572996 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6d8015cb10 0x7f6d80150c90 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:07.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:07.128+0000 7f6d85c86640 1 --2- 172.21.15.38:0/3459572996 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6d60078b90 0x7f6d6007b0a0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:07.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:07.128+0000 7f6d85c86640 1 --2- 172.21.15.38:0/3459572996 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6d800d12e0 0x7f6d80143d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:07.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:07.128+0000 7f6d85c86640 1 -- 172.21.15.38:0/3459572996 >> 172.21.15.38:0/3459572996 conn(0x7f6d800d2350 msgr2=0x7f6d80153e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:07.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:07.128+0000 7f6d85c86640 1 -- 172.21.15.38:0/3459572996 shutdown_connections 2024-09-21T03:36:07.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:07.128+0000 7f6d85c86640 1 -- 172.21.15.38:0/3459572996 wait complete. 2024-09-21T03:36:07.142 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json dump tree /a 0 2024-09-21T03:36:07.269 INFO:teuthology.orchestra.run.smithi038.stdout:[{"ino":1099511627776,"rdev":0,"ctime":"2024-09-21T03:36:02.400568+0000","btime":"2024-09-21T03:35:56.466679+0000","mode":16877,"uid":1000,"gid":1333,"nlink":1,"dir_layout":{"dir_hash":2,"unused1":0,"unused2":0,"unused3":0},"layout":{"stripe_unit":0,"stripe_count":0,"object_size":0,"pool_id":-1,"pool_ns":""},"old_pools":[],"size":0,"truncate_seq":1,"truncate_size":18446744073709551615,"truncate_from":0,"truncate_pending":0,"mtime":"2024-09-21T03:36:02.400568+0000","atime":"2024-09-21T03:35:56.466679+0000","time_warp_seq":0,"change_attr":3,"export_pin":-1,"export_ephemeral_random_pin":0,"export_ephemeral_distributed_pin":false,"quiesce_block":false,"client_ranges":[],"dirstat":{"version":0,"mtime":"2024-09-21T03:36:02.400568+0000","num_files":0,"num_subdirs":1,"change_attr":3},"rstat":{"version":0,"rbytes":0,"rfiles":0,"rsubdirs":2,"rsnaps":0,"rctime":"2024-09-21T03:36:02.400568+0000"},"accounted_rstat":{"version":0,"rbytes":0,"rfiles":0,"rsubdirs":2,"rsnaps":0,"rctime":"2024-09-21T03:36:02.400568+0000"},"version":9,"file_data_version":0,"xattr_version":1,"backtrace_version":2,"inline_data_version":18446744073709551615,"inline_data_length":0,"stray_prior_path":"","max_size_ever":0,"quota":{"max_bytes":0,"max_files":0},"last_scrub_stamp":"0.000000","last_scrub_version":0,"symlink":"","xattrs":[{"key":"security.selinux","val":"unconfined_u:object_r:unlabeled_t:s0\u0000"}],"dirfragtree":{"splits":[]},"old_inodes":[],"oldest_snap":18446744073709551614,"damage_flags":0,"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"dirfrag":1,"dirty":1},"nref":2,"versionlock":{"gather_set":[],"state":"lock","type":"iversion","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"authlock":{},"linklock":{},"dirfragtreelock":{},"filelock":{},"xattrlock":{},"snaplock":{},"nestlock":{},"flocklock":{},"policylock":{},"quiescelock":{"gather_set":[],"state":"lock","type":"iquiesce","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"states":["auth","dirty"],"client_caps":[],"loner":-1,"want_loner":-1,"mds_caps_wanted":[],"dirfrags":[{"path":"/a","dirfrag":"0x10000000000","snapid_first":2,"projected_version":"7","version":"7","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"","states":["auth"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1},"nref":1,"dentries":[{"path":"a/c","alternate_name":"","path_ino":1,"snap_first":18446744073709551606,"snap_last":18446744073709551614,"is_primary":true,"is_remote":false,"is_null":false,"is_new":false,"inode":1099511627777,"remote_type":"","version":4,"projected_version":4,"auth_pins":0,"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"inodepin":1,"dirty":1},"nref":2,"lock":{},"versionlock":{"gather_set":[],"state":"lock","type":"dversion","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"states":["auth","dirty"]},{"path":"a/b","alternate_name":"","path_ino":1,"snap_first":2,"snap_last":18446744073709551614,"is_primary":false,"is_remote":false,"is_null":true,"is_new":false,"inode":0,"remote_type":"","version":5,"projected_version":5,"auth_pins":0,"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"dirty":1},"nref":1,"lock":{},"versionlock":{"gather_set":[],"state":"lock","type":"dversion","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"states":["auth","dirty"]}]}],"is_subvolume":false}] 2024-09-21T03:36:07.270 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "accounted_rstat": { "rbytes": 0, "rctime": "2024-09-21T03:36:02.400568+0000", "rfiles": 0, "rsnaps": 0, "rsubdirs": 2, "version": 0 }, "atime": "2024-09-21T03:35:56.466679+0000", "auth_pins": 0, "auth_state": { "replicas": {} }, "authlock": {}, "backtrace_version": 2, "btime": "2024-09-21T03:35:56.466679+0000", "change_attr": 3, "client_caps": [], "client_ranges": [], "ctime": "2024-09-21T03:36:02.400568+0000", "damage_flags": 0, "dir_layout": { "dir_hash": 2, "unused1": 0, "unused2": 0, "unused3": 0 }, "dirfrags": [ { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dentries": [ { "alternate_name": "", "auth_pins": 0, "auth_state": { "replicas": {} }, "inode": 1099511627777, "is_auth": true, "is_freezing": false, "is_frozen": false, "is_new": false, "is_null": false, "is_primary": true, "is_remote": false, "lock": {}, "nref": 2, "path": "a/c", "path_ino": 1, "pins": { "dirty": 1, "inodepin": 1 }, "projected_version": 4, "remote_type": "", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snap_first": 18446744073709551606, "snap_last": 18446744073709551614, "states": [ "auth", "dirty" ], "version": 4, "versionlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "type": "dversion", "xlock_by": {} } }, { "alternate_name": "", "auth_pins": 0, "auth_state": { "replicas": {} }, "inode": 0, "is_auth": true, "is_freezing": false, "is_frozen": false, "is_new": false, "is_null": true, "is_primary": false, "is_remote": false, "lock": {}, "nref": 1, "path": "a/b", "path_ino": 1, "pins": { "dirty": 1 }, "projected_version": 5, "remote_type": "", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snap_first": 2, "snap_last": 18446744073709551614, "states": [ "auth", "dirty" ], "version": 5, "versionlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "type": "dversion", "xlock_by": {} } } ], "dir_auth": "", "dirfrag": "0x10000000000", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/a", "pins": { "child": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth" ], "version": "7" } ], "dirfragtree": { "splits": [] }, "dirfragtreelock": {}, "dirstat": { "change_attr": 3, "mtime": "2024-09-21T03:36:02.400568+0000", "num_files": 0, "num_subdirs": 1, "version": 0 }, "export_ephemeral_distributed_pin": false, "export_ephemeral_random_pin": 0, "export_pin": -1, "file_data_version": 0, "filelock": {}, "flocklock": {}, "gid": 1333, "inline_data_length": 0, "inline_data_version": 18446744073709551615, "ino": 1099511627776, "is_auth": true, "is_freezing": false, "is_frozen": false, "is_subvolume": false, "last_scrub_stamp": "0.000000", "last_scrub_version": 0, "layout": { "object_size": 0, "pool_id": -1, "pool_ns": "", "stripe_count": 0, "stripe_unit": 0 }, "linklock": {}, "loner": -1, "max_size_ever": 0, "mds_caps_wanted": [], "mode": 16877, "mtime": "2024-09-21T03:36:02.400568+0000", "nestlock": {}, "nlink": 1, "nref": 2, "old_inodes": [], "old_pools": [], "oldest_snap": 18446744073709551614, "pins": { "dirfrag": 1, "dirty": 1 }, "policylock": {}, "quiesce_block": false, "quiescelock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "type": "iquiesce", "xlock_by": {} }, "quota": { "max_bytes": 0, "max_files": 0 }, "rdev": 0, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "rstat": { "rbytes": 0, "rctime": "2024-09-21T03:36:02.400568+0000", "rfiles": 0, "rsnaps": 0, "rsubdirs": 2, "version": 0 }, "size": 0, "snaplock": {}, "states": [ "auth", "dirty" ], "stray_prior_path": "", "symlink": "", "time_warp_seq": 0, "truncate_from": 0, "truncate_pending": 0, "truncate_seq": 1, "truncate_size": 18446744073709551615, "uid": 1000, "version": 9, "versionlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "type": "iversion", "xlock_by": {} }, "want_loner": -1, "xattr_version": 1, "xattrlock": {}, "xattrs": [ { "key": "security.selinux", "val": "unconfined_u:object_r:unlabeled_t:s0\u0000" } ] } ] 2024-09-21T03:36:07.270 DEBUG:tasks.cephfs.test_damage:[{'ino': 1099511627776, 'rdev': 0, 'ctime': '2024-09-21T03:36:02.400568+0000', 'btime': '2024-09-21T03:35:56.466679+0000', 'mode': 16877, 'uid': 1000, 'gid': 1333, 'nlink': 1, 'dir_layout': {'dir_hash': 2, 'unused1': 0, 'unused2': 0, 'unused3': 0}, 'layout': {'stripe_unit': 0, 'stripe_count': 0, 'object_size': 0, 'pool_id': -1, 'pool_ns': ''}, 'old_pools': [], 'size': 0, 'truncate_seq': 1, 'truncate_size': 18446744073709551615, 'truncate_from': 0, 'truncate_pending': 0, 'mtime': '2024-09-21T03:36:02.400568+0000', 'atime': '2024-09-21T03:35:56.466679+0000', 'time_warp_seq': 0, 'change_attr': 3, 'export_pin': -1, 'export_ephemeral_random_pin': 0, 'export_ephemeral_distributed_pin': False, 'quiesce_block': False, 'client_ranges': [], 'dirstat': {'version': 0, 'mtime': '2024-09-21T03:36:02.400568+0000', 'num_files': 0, 'num_subdirs': 1, 'change_attr': 3}, 'rstat': {'version': 0, 'rbytes': 0, 'rfiles': 0, 'rsubdirs': 2, 'rsnaps': 0, 'rctime': '2024-09-21T03:36:02.400568+0000'}, 'accounted_rstat': {'version': 0, 'rbytes': 0, 'rfiles': 0, 'rsubdirs': 2, 'rsnaps': 0, 'rctime': '2024-09-21T03:36:02.400568+0000'}, 'version': 9, 'file_data_version': 0, 'xattr_version': 1, 'backtrace_version': 2, 'inline_data_version': 18446744073709551615, 'inline_data_length': 0, 'stray_prior_path': '', 'max_size_ever': 0, 'quota': {'max_bytes': 0, 'max_files': 0}, 'last_scrub_stamp': '0.000000', 'last_scrub_version': 0, 'symlink': '', 'xattrs': [{'key': 'security.selinux', 'val': 'unconfined_u:object_r:unlabeled_t:s0\x00'}], 'dirfragtree': {'splits': []}, 'old_inodes': [], 'oldest_snap': 18446744073709551614, 'damage_flags': 0, 'is_auth': True, 'auth_state': {'replicas': {}}, 'replica_state': {'authority': [0, -2], 'replica_nonce': 0}, 'auth_pins': 0, 'is_frozen': False, 'is_freezing': False, 'pins': {'dirfrag': 1, 'dirty': 1}, 'nref': 2, 'versionlock': {'gather_set': [], 'state': 'lock', 'type': 'iversion', 'is_leased': False, 'num_rdlocks': 0, 'num_wrlocks': 0, 'num_xlocks': 0, 'xlock_by': {}}, 'authlock': {}, 'linklock': {}, 'dirfragtreelock': {}, 'filelock': {}, 'xattrlock': {}, 'snaplock': {}, 'nestlock': {}, 'flocklock': {}, 'policylock': {}, 'quiescelock': {'gather_set': [], 'state': 'lock', 'type': 'iquiesce', 'is_leased': False, 'num_rdlocks': 0, 'num_wrlocks': 0, 'num_xlocks': 0, 'xlock_by': {}}, 'states': ['auth', 'dirty'], 'client_caps': [], 'loner': -1, 'want_loner': -1, 'mds_caps_wanted': [], 'dirfrags': [{'path': '/a', 'dirfrag': '0x10000000000', 'snapid_first': 2, 'projected_version': '7', 'version': '7', 'committing_version': '0', 'committed_version': '0', 'is_rep': False, 'dir_auth': '', 'states': ['auth'], 'is_auth': True, 'auth_state': {'replicas': {}}, 'replica_state': {'authority': [0, -2], 'replica_nonce': 0}, 'auth_pins': 0, 'is_frozen': False, 'is_freezing': False, 'pins': {'child': 1}, 'nref': 1, 'dentries': [{'path': 'a/c', 'alternate_name': '', 'path_ino': 1, 'snap_first': 18446744073709551606, 'snap_last': 18446744073709551614, 'is_primary': True, 'is_remote': False, 'is_null': False, 'is_new': False, 'inode': 1099511627777, 'remote_type': '', 'version': 4, 'projected_version': 4, 'auth_pins': 0, 'is_auth': True, 'auth_state': {'replicas': {}}, 'replica_state': {'authority': [0, -2], 'replica_nonce': 0}, 'is_frozen': False, 'is_freezing': False, 'pins': {'inodepin': 1, 'dirty': 1}, 'nref': 2, 'lock': {}, 'versionlock': {'gather_set': [], 'state': 'lock', 'type': 'dversion', 'is_leased': False, 'num_rdlocks': 0, 'num_wrlocks': 0, 'num_xlocks': 0, 'xlock_by': {}}, 'states': ['auth', 'dirty']}, {'path': 'a/b', 'alternate_name': '', 'path_ino': 1, 'snap_first': 2, 'snap_last': 18446744073709551614, 'is_primary': False, 'is_remote': False, 'is_null': True, 'is_new': False, 'inode': 0, 'remote_type': '', 'version': 5, 'projected_version': 5, 'auth_pins': 0, 'is_auth': True, 'auth_state': {'replicas': {}}, 'replica_state': {'authority': [0, -2], 'replica_nonce': 0}, 'is_frozen': False, 'is_freezing': False, 'pins': {'dirty': 1}, 'nref': 1, 'lock': {}, 'versionlock': {'gather_set': [], 'state': 'lock', 'type': 'dversion', 'is_leased': False, 'num_rdlocks': 0, 'num_wrlocks': 0, 'num_xlocks': 0, 'xlock_by': {}}, 'states': ['auth', 'dirty']}]}], 'is_subvolume': False}] 2024-09-21T03:36:07.270 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-21T03:36:07.271 DEBUG:teuthology.orchestra.run.smithi038:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T03:36:07.320 DEBUG:teuthology.orchestra.run.smithi195:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T03:36:07.363 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-21T03:36:07.364 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T03:36:07.364 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-21T03:36:07.364 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:36:07.380 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056795,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2354 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T03:36:07.380 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T03:36:07.380 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-09-21T03:36:07.380 DEBUG:teuthology.orchestra.run.smithi195:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:36:07.422 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056014,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497444k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=8978 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248720k,nr_inodes=812180,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497444k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T03:36:07.423 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T03:36:07.423 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-21T03:36:07.423 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:36:07.439 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056795,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2354 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T03:36:07.439 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T03:36:07.439 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-09-21T03:36:07.440 DEBUG:teuthology.orchestra.run.smithi195:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:36:07.480 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056014,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497444k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=8978 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248720k,nr_inodes=812180,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497444k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T03:36:07.481 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:36:07.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:07.596+0000 7f1cb8750640 1 -- 172.21.15.38:0/878375156 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1c9c004d40 msgr2=0x7f1c9c005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:07.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:07.596+0000 7f1cb8750640 1 --2- 172.21.15.38:0/878375156 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1c9c004d40 0x7f1c9c005150 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f1ca8009e90 tx=0x7f1c9c005830 comp rx=0 tx=0).stop 2024-09-21T03:36:07.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:07.596+0000 7f1cb6ced640 1 -- 172.21.15.38:0/878375156 reap_dead start 2024-09-21T03:36:07.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:07.596+0000 7f1cb8750640 1 -- 172.21.15.38:0/878375156 shutdown_connections 2024-09-21T03:36:07.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:07.596+0000 7f1cb8750640 1 -- 172.21.15.38:0/878375156 >> 172.21.15.38:0/878375156 conn(0x7f1cb0073510 msgr2=0x7f1cb0073920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:07.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:07.596+0000 7f1cb8750640 1 -- 172.21.15.38:0/878375156 shutdown_connections 2024-09-21T03:36:07.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:07.596+0000 7f1cb8750640 1 -- 172.21.15.38:0/878375156 wait complete. 2024-09-21T03:36:07.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:07.598+0000 7f1cb8750640 1 Processor -- start 2024-09-21T03:36:07.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:07.598+0000 7f1cb8750640 1 -- start start 2024-09-21T03:36:07.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:07.599+0000 7f1cb8750640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1c9c004d40 0x7f1cb014ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:07.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:07.599+0000 7f1cb8750640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1cb014cc70 0x7f1cb01f4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:07.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:07.599+0000 7f1cb8750640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1cb014a390 0x7f1cb01f7d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:07.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:07.599+0000 7f1cb8750640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f1cb00afa40 con 0x7f1c9c004d40 2024-09-21T03:36:07.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:07.599+0000 7f1cb8750640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f1cb00af8c0 con 0x7f1cb014a390 2024-09-21T03:36:07.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:07.599+0000 7f1cb8750640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f1cb00afbc0 con 0x7f1cb014cc70 2024-09-21T03:36:07.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:07.600+0000 7f1cb64ec640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1c9c004d40 0x7f1cb014ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:07.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:07.600+0000 7f1cb64ec640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1c9c004d40 0x7f1cb014ac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37512/0 (socket says 172.21.15.38:37512) 2024-09-21T03:36:07.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:07.600+0000 7f1cb64ec640 1 -- 172.21.15.38:0/1870750972 learned_addr learned my addr 172.21.15.38:0/1870750972 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:36:07.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:07.600+0000 7f1cb5ceb640 1 --2- 172.21.15.38:0/1870750972 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1cb014cc70 0x7f1cb01f4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:07.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:07.600+0000 7f1cb6ced640 1 --2- 172.21.15.38:0/1870750972 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1cb014a390 0x7f1cb01f7d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:07.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:07.600+0000 7f1cb64ec640 1 -- 172.21.15.38:0/1870750972 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1cb014cc70 msgr2=0x7f1cb01f4ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:07.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:07.600+0000 7f1cb64ec640 1 --2- 172.21.15.38:0/1870750972 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1cb014cc70 0x7f1cb01f4ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:07.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:07.600+0000 7f1cb64ec640 1 -- 172.21.15.38:0/1870750972 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1cb014a390 msgr2=0x7f1cb01f7d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:07.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:07.600+0000 7f1cb64ec640 1 --2- 172.21.15.38:0/1870750972 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1cb014a390 0x7f1cb01f7d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:07.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:07.600+0000 7f1cb64ec640 1 -- 172.21.15.38:0/1870750972 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cb014d3a0 con 0x7f1c9c004d40 2024-09-21T03:36:07.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:07.600+0000 7f1cb64ec640 1 --2- 172.21.15.38:0/1870750972 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1c9c004d40 0x7f1cb014ac30 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f1cac006660 tx=0x7f1cac0039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:07.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:07.600+0000 7f1ca6ffd640 1 -- 172.21.15.38:0/1870750972 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cac015070 con 0x7f1c9c004d40 2024-09-21T03:36:07.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:07.600+0000 7f1ca6ffd640 1 -- 172.21.15.38:0/1870750972 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1cac011920 con 0x7f1c9c004d40 2024-09-21T03:36:07.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:07.600+0000 7f1cb8750640 1 -- 172.21.15.38:0/1870750972 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cb014d680 con 0x7f1c9c004d40 2024-09-21T03:36:07.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:07.600+0000 7f1ca6ffd640 1 -- 172.21.15.38:0/1870750972 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cac011d70 con 0x7f1c9c004d40 2024-09-21T03:36:07.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:07.600+0000 7f1cb8750640 1 -- 172.21.15.38:0/1870750972 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cb01f5020 con 0x7f1c9c004d40 2024-09-21T03:36:07.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:07.601+0000 7f1cb8750640 1 -- 172.21.15.38:0/1870750972 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1cb006e200 con 0x7f1c9c004d40 2024-09-21T03:36:07.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:07.602+0000 7f1ca6ffd640 1 -- 172.21.15.38:0/1870750972 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f1cac018070 con 0x7f1c9c004d40 2024-09-21T03:36:07.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:07.603+0000 7f1ca6ffd640 1 --2- 172.21.15.38:0/1870750972 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1c94078b90 0x7f1c9407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:07.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:07.603+0000 7f1ca6ffd640 1 -- 172.21.15.38:0/1870750972 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 6187+0+0 (secure 0 0 0) 0x7f1cac09bc50 con 0x7f1c9c004d40 2024-09-21T03:36:07.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:07.603+0000 7f1cb5ceb640 1 --2- 172.21.15.38:0/1870750972 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1c94078b90 0x7f1c9407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:07.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:07.604+0000 7f1cb5ceb640 1 --2- 172.21.15.38:0/1870750972 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1c94078b90 0x7f1c9407b0a0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f1cb00b09a0 tx=0x7f1ca0005f90 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:07.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:07.608+0000 7f1ca6ffd640 1 -- 172.21.15.38:0/1870750972 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f1cb006e740 con 0x7f1c9c004d40 2024-09-21T03:36:07.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:07.836+0000 7f1cb8750640 1 -- 172.21.15.38:0/1870750972 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1cb015c830 con 0x7f1c9c004d40 2024-09-21T03:36:07.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:07.837+0000 7f1ca6ffd640 1 -- 172.21.15.38:0/1870750972 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+5223 (secure 0 0 0) 0x7f1cac0639a0 con 0x7f1c9c004d40 2024-09-21T03:36:07.839 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:36:07.839 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":35,"btime":"2024-09-21T03:36:07:483629+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5147,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/2858412699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":2858412699},{"type":"v1","addr":"172.21.15.195:6837","nonce":2858412699}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":5368,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/2907144575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":2907144575},{"type":"v1","addr":"172.21.15.195:6836","nonce":2907144575}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":5574,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6834/1911661184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":1911661184},{"type":"v1","addr":"172.21.15.38:6834","nonce":1911661184}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:35:44.271168+0000","modified":"2024-09-21T03:36:06.485795+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":56,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5793},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5793":{"gid":5793,"name":"c","rank":0,"incarnation":33,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.38:6835/3716808119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6833","nonce":3716808119},{"type":"v1","addr":"172.21.15.38:6835","nonce":3716808119}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":3}]} 2024-09-21T03:36:07.839 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 35 2024-09-21T03:36:07.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:07.839+0000 7f1cb8750640 1 -- 172.21.15.38:0/1870750972 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1c94078b90 msgr2=0x7f1c9407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:07.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:07.839+0000 7f1cb8750640 1 --2- 172.21.15.38:0/1870750972 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1c94078b90 0x7f1c9407b0a0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f1cb00b09a0 tx=0x7f1ca0005f90 comp rx=0 tx=0).stop 2024-09-21T03:36:07.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:07.839+0000 7f1cb8750640 1 -- 172.21.15.38:0/1870750972 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1c9c004d40 msgr2=0x7f1cb014ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:07.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:07.839+0000 7f1cb8750640 1 --2- 172.21.15.38:0/1870750972 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1c9c004d40 0x7f1cb014ac30 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f1cac006660 tx=0x7f1cac0039c0 comp rx=0 tx=0).stop 2024-09-21T03:36:07.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:07.839+0000 7f1cb8750640 1 -- 172.21.15.38:0/1870750972 shutdown_connections 2024-09-21T03:36:07.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:07.839+0000 7f1cb8750640 1 --2- 172.21.15.38:0/1870750972 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1cb014a390 0x7f1cb01f7d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:07.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:07.839+0000 7f1cb8750640 1 --2- 172.21.15.38:0/1870750972 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1c94078b90 0x7f1c9407b0a0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:07.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:07.839+0000 7f1cb8750640 1 --2- 172.21.15.38:0/1870750972 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1cb014cc70 0x7f1cb01f4ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:07.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:07.839+0000 7f1cb8750640 1 --2- 172.21.15.38:0/1870750972 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1c9c004d40 0x7f1cb014ac30 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:07.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:07.839+0000 7f1cb8750640 1 -- 172.21.15.38:0/1870750972 >> 172.21.15.38:0/1870750972 conn(0x7f1cb0073510 msgr2=0x7f1cb0159340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:07.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:07.839+0000 7f1cb8750640 1 -- 172.21.15.38:0/1870750972 shutdown_connections 2024-09-21T03:36:07.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:07.839+0000 7f1cb8750640 1 -- 172.21.15.38:0/1870750972 wait complete. 2024-09-21T03:36:07.853 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:36:07.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:07.969+0000 7f94b3bc3640 1 -- 172.21.15.38:0/3480833843 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9498003a00 msgr2=0x7f9498003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:07.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:07.969+0000 7f94b3bc3640 1 --2- 172.21.15.38:0/3480833843 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9498003a00 0x7f9498003e70 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f94a8009b50 tx=0x7f9498005830 comp rx=0 tx=0).stop 2024-09-21T03:36:07.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:07.969+0000 7f94b2160640 1 -- 172.21.15.38:0/3480833843 reap_dead start 2024-09-21T03:36:07.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:07.969+0000 7f94b3bc3640 1 -- 172.21.15.38:0/3480833843 shutdown_connections 2024-09-21T03:36:07.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:07.969+0000 7f94b3bc3640 1 -- 172.21.15.38:0/3480833843 >> 172.21.15.38:0/3480833843 conn(0x7f94ac0d2350 msgr2=0x7f94ac0d2760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:07.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:07.969+0000 7f94b3bc3640 1 -- 172.21.15.38:0/3480833843 shutdown_connections 2024-09-21T03:36:07.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:07.969+0000 7f94b3bc3640 1 -- 172.21.15.38:0/3480833843 wait complete. 2024-09-21T03:36:07.972 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:07.971+0000 7f94b3bc3640 1 Processor -- start 2024-09-21T03:36:07.972 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:07.971+0000 7f94b3bc3640 1 -- start start 2024-09-21T03:36:07.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:07.972+0000 7f94b3bc3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9498003a00 0x7f94ac140a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:07.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:07.972+0000 7f94b3bc3640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f94ac1424b0 0x7f94ac15a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:07.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:07.972+0000 7f94b3bc3640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f94ac151fb0 0x7f94ac1523c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:07.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:07.972+0000 7f94b3bc3640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f94ac160c20 con 0x7f9498003a00 2024-09-21T03:36:07.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:07.972+0000 7f94b3bc3640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f94ac0b7470 con 0x7f94ac151fb0 2024-09-21T03:36:07.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:07.972+0000 7f94b3bc3640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f94ac160da0 con 0x7f94ac1424b0 2024-09-21T03:36:07.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:07.972+0000 7f94b195f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9498003a00 0x7f94ac140a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:07.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:07.972+0000 7f94b195f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9498003a00 0x7f94ac140a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37532/0 (socket says 172.21.15.38:37532) 2024-09-21T03:36:07.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:07.972+0000 7f94b195f640 1 -- 172.21.15.38:0/932836408 learned_addr learned my addr 172.21.15.38:0/932836408 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:36:07.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:07.972+0000 7f94b115e640 1 --2- 172.21.15.38:0/932836408 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f94ac1424b0 0x7f94ac15a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:07.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:07.972+0000 7f94b2160640 1 --2- 172.21.15.38:0/932836408 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f94ac151fb0 0x7f94ac1523c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:07.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:07.972+0000 7f94b195f640 1 -- 172.21.15.38:0/932836408 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f94ac1424b0 msgr2=0x7f94ac15a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:07.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:07.972+0000 7f94b195f640 1 --2- 172.21.15.38:0/932836408 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f94ac1424b0 0x7f94ac15a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:07.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:07.972+0000 7f94b195f640 1 -- 172.21.15.38:0/932836408 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f94ac151fb0 msgr2=0x7f94ac1523c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:07.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:07.972+0000 7f94b195f640 1 --2- 172.21.15.38:0/932836408 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f94ac151fb0 0x7f94ac1523c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:07.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:07.972+0000 7f94b195f640 1 -- 172.21.15.38:0/932836408 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94ac152900 con 0x7f9498003a00 2024-09-21T03:36:07.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:07.972+0000 7f94b2160640 1 --2- 172.21.15.38:0/932836408 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f94ac151fb0 0x7f94ac1523c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:36:07.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:07.972+0000 7f94b195f640 1 --2- 172.21.15.38:0/932836408 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9498003a00 0x7f94ac140a70 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f94a8009c70 tx=0x7f94a8033670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:07.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:07.973+0000 7f94a27fc640 1 -- 172.21.15.38:0/932836408 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94a8038040 con 0x7f9498003a00 2024-09-21T03:36:07.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:07.973+0000 7f94a27fc640 1 -- 172.21.15.38:0/932836408 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f94a8004020 con 0x7f9498003a00 2024-09-21T03:36:07.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:07.973+0000 7f94b3bc3640 1 -- 172.21.15.38:0/932836408 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94ac14fe10 con 0x7f9498003a00 2024-09-21T03:36:07.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:07.973+0000 7f94a27fc640 1 -- 172.21.15.38:0/932836408 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94a8004450 con 0x7f9498003a00 2024-09-21T03:36:07.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:07.973+0000 7f94b3bc3640 1 -- 172.21.15.38:0/932836408 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94ac1430c0 con 0x7f9498003a00 2024-09-21T03:36:07.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:07.974+0000 7f94b3bc3640 1 -- 172.21.15.38:0/932836408 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94ac15cf20 con 0x7f9498003a00 2024-09-21T03:36:07.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:07.974+0000 7f94a27fc640 1 -- 172.21.15.38:0/932836408 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f94a8045020 con 0x7f9498003a00 2024-09-21T03:36:07.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:07.975+0000 7f94a27fc640 1 --2- 172.21.15.38:0/932836408 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9490078b90 0x7f949007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:07.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:07.975+0000 7f94a27fc640 1 -- 172.21.15.38:0/932836408 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 6187+0+0 (secure 0 0 0) 0x7f94a80bb8d0 con 0x7f9498003a00 2024-09-21T03:36:07.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:07.975+0000 7f94b115e640 1 --2- 172.21.15.38:0/932836408 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9490078b90 0x7f949007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:07.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:07.976+0000 7f94b115e640 1 --2- 172.21.15.38:0/932836408 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9490078b90 0x7f949007b0a0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f94ac0c8c00 tx=0x7f949c005f90 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:07.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:07.980+0000 7f94a27fc640 1 -- 172.21.15.38:0/932836408 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f94ac152900 con 0x7f9498003a00 2024-09-21T03:36:08.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:08.202+0000 7f94b3bc3640 1 -- 172.21.15.38:0/932836408 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f94ac0706f0 con 0x7f9498003a00 2024-09-21T03:36:08.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:08.203+0000 7f94a27fc640 1 -- 172.21.15.38:0/932836408 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+5223 (secure 0 0 0) 0x7f94a8083b30 con 0x7f9498003a00 2024-09-21T03:36:08.205 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:36:08.205 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":35,"btime":"2024-09-21T03:36:07:483629+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5147,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/2858412699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":2858412699},{"type":"v1","addr":"172.21.15.195:6837","nonce":2858412699}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":5368,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/2907144575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":2907144575},{"type":"v1","addr":"172.21.15.195:6836","nonce":2907144575}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":5574,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6834/1911661184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":1911661184},{"type":"v1","addr":"172.21.15.38:6834","nonce":1911661184}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:35:44.271168+0000","modified":"2024-09-21T03:36:06.485795+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":56,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5793},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5793":{"gid":5793,"name":"c","rank":0,"incarnation":33,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.38:6835/3716808119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6833","nonce":3716808119},{"type":"v1","addr":"172.21.15.38:6835","nonce":3716808119}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":3}]} 2024-09-21T03:36:08.205 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 35 2024-09-21T03:36:08.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:08.205+0000 7f94b3bc3640 1 -- 172.21.15.38:0/932836408 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9490078b90 msgr2=0x7f949007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:08.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:08.205+0000 7f94b3bc3640 1 --2- 172.21.15.38:0/932836408 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9490078b90 0x7f949007b0a0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f94ac0c8c00 tx=0x7f949c005f90 comp rx=0 tx=0).stop 2024-09-21T03:36:08.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:08.205+0000 7f94b3bc3640 1 -- 172.21.15.38:0/932836408 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9498003a00 msgr2=0x7f94ac140a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:08.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:08.205+0000 7f94b3bc3640 1 --2- 172.21.15.38:0/932836408 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9498003a00 0x7f94ac140a70 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f94a8009c70 tx=0x7f94a8033670 comp rx=0 tx=0).stop 2024-09-21T03:36:08.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:08.205+0000 7f94b3bc3640 1 -- 172.21.15.38:0/932836408 shutdown_connections 2024-09-21T03:36:08.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:08.205+0000 7f94b3bc3640 1 --2- 172.21.15.38:0/932836408 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f94ac151fb0 0x7f94ac1523c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:08.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:08.205+0000 7f94b3bc3640 1 --2- 172.21.15.38:0/932836408 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9490078b90 0x7f949007b0a0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:08.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:08.205+0000 7f94b3bc3640 1 --2- 172.21.15.38:0/932836408 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f94ac1424b0 0x7f94ac15a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:08.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:08.205+0000 7f94b3bc3640 1 --2- 172.21.15.38:0/932836408 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9498003a00 0x7f94ac140a70 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:08.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:08.205+0000 7f94b3bc3640 1 -- 172.21.15.38:0/932836408 >> 172.21.15.38:0/932836408 conn(0x7f94ac0d2350 msgr2=0x7f94ac160510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:08.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:08.205+0000 7f94b3bc3640 1 -- 172.21.15.38:0/932836408 shutdown_connections 2024-09-21T03:36:08.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:08.205+0000 7f94b3bc3640 1 -- 172.21.15.38:0/932836408 wait complete. 2024-09-21T03:36:08.218 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T03:36:08.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:08.334+0000 7f192a863640 1 -- 172.21.15.38:0/4287579433 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f190c003a50 msgr2=0x7f190c003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:08.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:08.334+0000 7f192a863640 1 --2- 172.21.15.38:0/4287579433 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f190c003a50 0x7f190c003ec0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f1914009b50 tx=0x7f190c005830 comp rx=0 tx=0).stop 2024-09-21T03:36:08.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:08.334+0000 7f192a863640 1 -- 172.21.15.38:0/4287579433 shutdown_connections 2024-09-21T03:36:08.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:08.334+0000 7f192a863640 1 --2- 172.21.15.38:0/4287579433 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f190c003a50 0x7f190c003ec0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:08.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:08.334+0000 7f192a863640 1 -- 172.21.15.38:0/4287579433 >> 172.21.15.38:0/4287579433 conn(0x7f19240b6b20 msgr2=0x7f19240d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:08.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:08.334+0000 7f192a863640 1 -- 172.21.15.38:0/4287579433 shutdown_connections 2024-09-21T03:36:08.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:08.334+0000 7f192a863640 1 -- 172.21.15.38:0/4287579433 wait complete. 2024-09-21T03:36:08.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:08.336+0000 7f192a863640 1 Processor -- start 2024-09-21T03:36:08.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:08.336+0000 7f192a863640 1 -- start start 2024-09-21T03:36:08.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:08.337+0000 7f192a863640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f190c003a50 0x7f19241537c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:08.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:08.337+0000 7f192a863640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f192415a480 0x7f192415ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:08.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:08.337+0000 7f192a863640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f192415f450 0x7f192415b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:08.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:08.338+0000 7f192a863640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f1924162b80 con 0x7f192415f450 2024-09-21T03:36:08.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:08.338+0000 7f192a863640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f1924162a00 con 0x7f190c003a50 2024-09-21T03:36:08.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:08.338+0000 7f192a863640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f19240d24d0 con 0x7f192415a480 2024-09-21T03:36:08.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:08.338+0000 7f1928e00640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f192415f450 0x7f192415b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:08.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:08.338+0000 7f1928e00640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f192415f450 0x7f192415b390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37554/0 (socket says 172.21.15.38:37554) 2024-09-21T03:36:08.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:08.338+0000 7f1928e00640 1 -- 172.21.15.38:0/1758956021 learned_addr learned my addr 172.21.15.38:0/1758956021 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:36:08.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:08.338+0000 7f1923fff640 1 --2- 172.21.15.38:0/1758956021 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f190c003a50 0x7f19241537c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:08.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:08.338+0000 7f19237fe640 1 --2- 172.21.15.38:0/1758956021 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f192415a480 0x7f192415ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:08.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:08.338+0000 7f1928e00640 1 -- 172.21.15.38:0/1758956021 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f192415a480 msgr2=0x7f192415ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:08.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:08.338+0000 7f1928e00640 1 --2- 172.21.15.38:0/1758956021 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f192415a480 0x7f192415ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:08.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:08.338+0000 7f1928e00640 1 -- 172.21.15.38:0/1758956021 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f190c003a50 msgr2=0x7f19241537c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:08.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:08.338+0000 7f1928e00640 1 --2- 172.21.15.38:0/1758956021 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f190c003a50 0x7f19241537c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:08.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:08.338+0000 7f1928e00640 1 -- 172.21.15.38:0/1758956021 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1924159440 con 0x7f192415f450 2024-09-21T03:36:08.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:08.338+0000 7f1928e00640 1 --2- 172.21.15.38:0/1758956021 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f192415f450 0x7f192415b390 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f19240b7430 tx=0x7f191c006720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:08.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:08.338+0000 7f1920ff9640 1 -- 172.21.15.38:0/1758956021 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f191c00fa80 con 0x7f192415f450 2024-09-21T03:36:08.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:08.338+0000 7f1920ff9640 1 -- 172.21.15.38:0/1758956021 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f191c018070 con 0x7f192415f450 2024-09-21T03:36:08.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:08.338+0000 7f192a863640 1 -- 172.21.15.38:0/1758956021 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19241596c0 con 0x7f192415f450 2024-09-21T03:36:08.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:08.338+0000 7f1920ff9640 1 -- 172.21.15.38:0/1758956021 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f191c011430 con 0x7f192415f450 2024-09-21T03:36:08.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:08.338+0000 7f192a863640 1 -- 172.21.15.38:0/1758956021 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1924159c80 con 0x7f192415f450 2024-09-21T03:36:08.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:08.339+0000 7f192a863640 1 -- 172.21.15.38:0/1758956021 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19240d2a30 con 0x7f192415f450 2024-09-21T03:36:08.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:08.340+0000 7f1920ff9640 1 -- 172.21.15.38:0/1758956021 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f191c0115d0 con 0x7f192415f450 2024-09-21T03:36:08.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:08.341+0000 7f1920ff9640 1 --2- 172.21.15.38:0/1758956021 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1910078b90 0x7f191007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:08.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:08.341+0000 7f1920ff9640 1 -- 172.21.15.38:0/1758956021 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 6187+0+0 (secure 0 0 0) 0x7f191c09b600 con 0x7f192415f450 2024-09-21T03:36:08.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:08.341+0000 7f1923fff640 1 --2- 172.21.15.38:0/1758956021 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1910078b90 0x7f191007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:08.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:08.342+0000 7f1923fff640 1 --2- 172.21.15.38:0/1758956021 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1910078b90 0x7f191007b0a0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f1914035000 tx=0x7f19140046c0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:08.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:08.345+0000 7f1920ff9640 1 -- 172.21.15.38:0/1758956021 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f1924159440 con 0x7f192415f450 2024-09-21T03:36:08.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:08.534+0000 7f192a863640 1 -- 172.21.15.38:0/1758956021 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f192406eba0 con 0x7f192415f450 2024-09-21T03:36:08.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:08.535+0000 7f1920ff9640 1 -- 172.21.15.38:0/1758956021 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v56) ==== 74+0+17095 (secure 0 0 0) 0x7f191c063210 con 0x7f192415f450 2024-09-21T03:36:08.537 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:36:08.537 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":56,"fsid":"9d50d66f-656b-4fe2-8e3b-fe53b79c4d43","created":"2024-09-21T03:33:10.677865+0000","modified":"2024-09-21T03:36:04.173381+0000","last_up_change":"2024-09-21T03:33:24.023421+0000","last_in_change":"2024-09-21T03:33:14.296193+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T03:33:22.992573+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-09-21T03:35:41.962122+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-09-21T03:35:43.237360+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dbab183f-7e63-48b8-960a-ae55fd5bf565","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6801","nonce":2381639433}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6803","nonce":2381639433}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6807","nonce":2381639433}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6805","nonce":2381639433}]},"public_addr":"172.21.15.38:6801/2381639433","cluster_addr":"172.21.15.38:6803/2381639433","heartbeat_back_addr":"172.21.15.38:6807/2381639433","heartbeat_front_addr":"172.21.15.38:6805/2381639433","state":["exists","up"]},{"osd":1,"uuid":"fa1c0e8a-ebc1-4359-8146-bd5342abbc16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6817","nonce":1491818732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6819","nonce":1491818732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6823","nonce":1491818732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6821","nonce":1491818732}]},"public_addr":"172.21.15.38:6817/1491818732","cluster_addr":"172.21.15.38:6819/1491818732","heartbeat_back_addr":"172.21.15.38:6823/1491818732","heartbeat_front_addr":"172.21.15.38:6821/1491818732","state":["exists","up"]},{"osd":2,"uuid":"2ed3cd2d-385d-4fc6-9869-214552a84803","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6825","nonce":2577858902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6827","nonce":2577858902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6831","nonce":2577858902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6829","nonce":2577858902}]},"public_addr":"172.21.15.38:6825/2577858902","cluster_addr":"172.21.15.38:6827/2577858902","heartbeat_back_addr":"172.21.15.38:6831/2577858902","heartbeat_front_addr":"172.21.15.38:6829/2577858902","state":["exists","up"]},{"osd":3,"uuid":"62d1d95e-919d-4250-be71-1d34cfa70dbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6809","nonce":1286810294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6811","nonce":1286810294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6815","nonce":1286810294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6813","nonce":1286810294}]},"public_addr":"172.21.15.38:6809/1286810294","cluster_addr":"172.21.15.38:6811/1286810294","heartbeat_back_addr":"172.21.15.38:6815/1286810294","heartbeat_front_addr":"172.21.15.38:6813/1286810294","state":["exists","up"]},{"osd":4,"uuid":"7c9f9314-99d7-4d73-8392-37809a9f54b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6803","nonce":3401837571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6805","nonce":3401837571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6809","nonce":3401837571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6807","nonce":3401837571}]},"public_addr":"172.21.15.195:6803/3401837571","cluster_addr":"172.21.15.195:6805/3401837571","heartbeat_back_addr":"172.21.15.195:6809/3401837571","heartbeat_front_addr":"172.21.15.195:6807/3401837571","state":["exists","up"]},{"osd":5,"uuid":"bde4fe9f-8414-4bba-bf4a-43973d1976bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6811","nonce":2080960180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6813","nonce":2080960180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6817","nonce":2080960180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6815","nonce":2080960180}]},"public_addr":"172.21.15.195:6811/2080960180","cluster_addr":"172.21.15.195:6813/2080960180","heartbeat_back_addr":"172.21.15.195:6817/2080960180","heartbeat_front_addr":"172.21.15.195:6815/2080960180","state":["exists","up"]},{"osd":6,"uuid":"6ee6d7f4-1a5f-4a4c-ac16-48fe05f6a002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6827","nonce":4055825675}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6829","nonce":4055825675}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6833","nonce":4055825675}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6831","nonce":4055825675}]},"public_addr":"172.21.15.195:6827/4055825675","cluster_addr":"172.21.15.195:6829/4055825675","heartbeat_back_addr":"172.21.15.195:6833/4055825675","heartbeat_front_addr":"172.21.15.195:6831/4055825675","state":["exists","up"]},{"osd":7,"uuid":"efeb5f51-329a-4484-8290-92e7d3503d05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6819","nonce":1606106342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6821","nonce":1606106342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6825","nonce":1606106342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6823","nonce":1606106342}]},"public_addr":"172.21.15.195:6819/1606106342","cluster_addr":"172.21.15.195:6821/1606106342","heartbeat_back_addr":"172.21.15.195:6825/1606106342","heartbeat_front_addr":"172.21.15.195:6823/1606106342","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.901343+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:21.115003+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.901623+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.835238+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.132199+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:19.817560+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.829690+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:21.087300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.38:6835/3184989541":"2024-09-22T03:36:04.173351+0000","172.21.15.38:6833/3184989541":"2024-09-22T03:36:04.173351+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T03:36:08.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:08.537+0000 7f192a863640 1 -- 172.21.15.38:0/1758956021 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1910078b90 msgr2=0x7f191007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:08.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:08.537+0000 7f192a863640 1 --2- 172.21.15.38:0/1758956021 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1910078b90 0x7f191007b0a0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f1914035000 tx=0x7f19140046c0 comp rx=0 tx=0).stop 2024-09-21T03:36:08.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:08.537+0000 7f192a863640 1 -- 172.21.15.38:0/1758956021 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f192415f450 msgr2=0x7f192415b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:08.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:08.537+0000 7f192a863640 1 --2- 172.21.15.38:0/1758956021 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f192415f450 0x7f192415b390 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f19240b7430 tx=0x7f191c006720 comp rx=0 tx=0).stop 2024-09-21T03:36:08.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:08.537+0000 7f192a863640 1 -- 172.21.15.38:0/1758956021 shutdown_connections 2024-09-21T03:36:08.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:08.537+0000 7f192a863640 1 --2- 172.21.15.38:0/1758956021 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f192415f450 0x7f192415b390 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:08.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:08.537+0000 7f192a863640 1 --2- 172.21.15.38:0/1758956021 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1910078b90 0x7f191007b0a0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:08.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:08.537+0000 7f192a863640 1 --2- 172.21.15.38:0/1758956021 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f192415a480 0x7f192415ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:08.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:08.537+0000 7f192a863640 1 --2- 172.21.15.38:0/1758956021 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f190c003a50 0x7f19241537c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:08.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:08.537+0000 7f192a863640 1 -- 172.21.15.38:0/1758956021 >> 172.21.15.38:0/1758956021 conn(0x7f19240b6b20 msgr2=0x7f19240cf890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:08.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:08.537+0000 7f192a863640 1 -- 172.21.15.38:0/1758956021 shutdown_connections 2024-09-21T03:36:08.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:08.537+0000 7f192a863640 1 -- 172.21.15.38:0/1758956021 wait complete. 2024-09-21T03:36:08.551 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-21T03:36:08.551 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:36:08.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:08.666+0000 7f06ab6dc640 1 -- 172.21.15.38:0/1507554417 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f06a4162280 msgr2=0x7f06a4162690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:08.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:08.666+0000 7f06ab6dc640 1 --2- 172.21.15.38:0/1507554417 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f06a4162280 0x7f06a4162690 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f0698009830 tx=0x7f069802a510 comp rx=0 tx=0).stop 2024-09-21T03:36:08.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:08.666+0000 7f06ab6dc640 1 -- 172.21.15.38:0/1507554417 shutdown_connections 2024-09-21T03:36:08.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:08.666+0000 7f06ab6dc640 1 --2- 172.21.15.38:0/1507554417 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f06a40b2e40 0x7f06a40b3250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:08.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:08.666+0000 7f06ab6dc640 1 --2- 172.21.15.38:0/1507554417 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f06a4162280 0x7f06a4162690 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:08.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:08.666+0000 7f06ab6dc640 1 -- 172.21.15.38:0/1507554417 >> 172.21.15.38:0/1507554417 conn(0x7f06a40b6b20 msgr2=0x7f06a40d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:08.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:08.666+0000 7f06ab6dc640 1 -- 172.21.15.38:0/1507554417 shutdown_connections 2024-09-21T03:36:08.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:08.666+0000 7f06ab6dc640 1 -- 172.21.15.38:0/1507554417 wait complete. 2024-09-21T03:36:08.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:08.668+0000 7f06ab6dc640 1 Processor -- start 2024-09-21T03:36:08.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:08.668+0000 7f06ab6dc640 1 -- start start 2024-09-21T03:36:08.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:08.669+0000 7f06ab6dc640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f06a40b2e40 0x7f06a40cd6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:08.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:08.669+0000 7f06ab6dc640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f06a4162280 0x7f06a40cc100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:08.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:08.669+0000 7f06ab6dc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f06a40cc640 0x7f06a40cca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:08.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:08.669+0000 7f06ab6dc640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f06a415e4b0 con 0x7f06a40cc640 2024-09-21T03:36:08.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:08.669+0000 7f06ab6dc640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f06a40b39d0 con 0x7f06a4162280 2024-09-21T03:36:08.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:08.669+0000 7f06ab6dc640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f06a415e630 con 0x7f06a40b2e40 2024-09-21T03:36:08.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:08.669+0000 7f06a9c79640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f06a40cc640 0x7f06a40cca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:08.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:08.669+0000 7f06a9c79640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f06a40cc640 0x7f06a40cca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37558/0 (socket says 172.21.15.38:37558) 2024-09-21T03:36:08.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:08.669+0000 7f06a9c79640 1 -- 172.21.15.38:0/2541294748 learned_addr learned my addr 172.21.15.38:0/2541294748 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:36:08.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:08.669+0000 7f06a8c77640 1 --2- 172.21.15.38:0/2541294748 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f06a4162280 0x7f06a40cc100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:08.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:08.669+0000 7f06a9478640 1 --2- 172.21.15.38:0/2541294748 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f06a40b2e40 0x7f06a40cd6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:08.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:08.669+0000 7f06a9c79640 1 -- 172.21.15.38:0/2541294748 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f06a40b2e40 msgr2=0x7f06a40cd6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:08.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:08.669+0000 7f06a9c79640 1 --2- 172.21.15.38:0/2541294748 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f06a40b2e40 0x7f06a40cd6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:08.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:08.669+0000 7f06a9c79640 1 -- 172.21.15.38:0/2541294748 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f06a4162280 msgr2=0x7f06a40cc100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:08.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:08.669+0000 7f06a9c79640 1 --2- 172.21.15.38:0/2541294748 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f06a4162280 0x7f06a40cc100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:08.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:08.669+0000 7f06a9c79640 1 -- 172.21.15.38:0/2541294748 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06a4150390 con 0x7f06a40cc640 2024-09-21T03:36:08.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:08.670+0000 7f06a9c79640 1 --2- 172.21.15.38:0/2541294748 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f06a40cc640 0x7f06a40cca50 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f069c00d650 tx=0x7f069c00db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:08.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:08.670+0000 7f06a1ffb640 1 -- 172.21.15.38:0/2541294748 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f069c017070 con 0x7f06a40cc640 2024-09-21T03:36:08.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:08.670+0000 7f06a1ffb640 1 -- 172.21.15.38:0/2541294748 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f069c005310 con 0x7f06a40cc640 2024-09-21T03:36:08.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:08.670+0000 7f06ab6dc640 1 -- 172.21.15.38:0/2541294748 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06a4150670 con 0x7f06a40cc640 2024-09-21T03:36:08.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:08.670+0000 7f06a1ffb640 1 -- 172.21.15.38:0/2541294748 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f069c009820 con 0x7f06a40cc640 2024-09-21T03:36:08.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:08.670+0000 7f06ab6dc640 1 -- 172.21.15.38:0/2541294748 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06a4150c30 con 0x7f06a40cc640 2024-09-21T03:36:08.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:08.671+0000 7f06ab6dc640 1 -- 172.21.15.38:0/2541294748 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06a40cf400 con 0x7f06a40cc640 2024-09-21T03:36:08.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:08.672+0000 7f06a1ffb640 1 -- 172.21.15.38:0/2541294748 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f069c0099c0 con 0x7f06a40cc640 2024-09-21T03:36:08.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:08.672+0000 7f06a1ffb640 1 --2- 172.21.15.38:0/2541294748 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0690078bc0 0x7f069007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:08.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:08.672+0000 7f06a1ffb640 1 -- 172.21.15.38:0/2541294748 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 6187+0+0 (secure 0 0 0) 0x7f069c09ae20 con 0x7f06a40cc640 2024-09-21T03:36:08.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:08.673+0000 7f06a9478640 1 --2- 172.21.15.38:0/2541294748 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0690078bc0 0x7f069007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:08.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:08.673+0000 7f06a9478640 1 --2- 172.21.15.38:0/2541294748 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0690078bc0 0x7f069007b0d0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f06940012e0 tx=0x7f0694009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:08.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:08.677+0000 7f06a1ffb640 1 -- 172.21.15.38:0/2541294748 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f06a40cf400 con 0x7f06a40cc640 2024-09-21T03:36:08.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:08.903+0000 7f06ab6dc640 1 -- 172.21.15.38:0/2541294748 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f06a40cdbf0 con 0x7f06a40cc640 2024-09-21T03:36:08.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:08.904+0000 7f06a1ffb640 1 -- 172.21.15.38:0/2541294748 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+5230 (secure 0 0 0) 0x7f069c062ac0 con 0x7f06a40cc640 2024-09-21T03:36:08.905 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:36:08.906 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":36,"btime":"2024-09-21T03:36:08:488383+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5147,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/2858412699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":2858412699},{"type":"v1","addr":"172.21.15.195:6837","nonce":2858412699}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":5368,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/2907144575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":2907144575},{"type":"v1","addr":"172.21.15.195:6836","nonce":2907144575}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":5574,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6834/1911661184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":1911661184},{"type":"v1","addr":"172.21.15.38:6834","nonce":1911661184}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:35:44.271168+0000","modified":"2024-09-21T03:36:08.488376+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":56,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5793},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5793":{"gid":5793,"name":"c","rank":0,"incarnation":33,"state":"up:active","state_seq":4,"addr":"172.21.15.38:6835/3716808119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6833","nonce":3716808119},{"type":"v1","addr":"172.21.15.38:6835","nonce":3716808119}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5793,"qdb_cluster":[5793]},"id":3}]} 2024-09-21T03:36:08.906 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 36 2024-09-21T03:36:08.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:08.906+0000 7f06ab6dc640 1 -- 172.21.15.38:0/2541294748 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0690078bc0 msgr2=0x7f069007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:08.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:08.906+0000 7f06ab6dc640 1 --2- 172.21.15.38:0/2541294748 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0690078bc0 0x7f069007b0d0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f06940012e0 tx=0x7f0694009040 comp rx=0 tx=0).stop 2024-09-21T03:36:08.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:08.906+0000 7f06ab6dc640 1 -- 172.21.15.38:0/2541294748 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f06a40cc640 msgr2=0x7f06a40cca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:08.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:08.906+0000 7f06ab6dc640 1 --2- 172.21.15.38:0/2541294748 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f06a40cc640 0x7f06a40cca50 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f069c00d650 tx=0x7f069c00db10 comp rx=0 tx=0).stop 2024-09-21T03:36:08.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:08.906+0000 7f06ab6dc640 1 -- 172.21.15.38:0/2541294748 shutdown_connections 2024-09-21T03:36:08.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:08.906+0000 7f06ab6dc640 1 --2- 172.21.15.38:0/2541294748 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f06a40cc640 0x7f06a40cca50 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:08.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:08.906+0000 7f06ab6dc640 1 --2- 172.21.15.38:0/2541294748 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f06a4162280 0x7f06a40cc100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:08.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:08.906+0000 7f06ab6dc640 1 --2- 172.21.15.38:0/2541294748 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0690078bc0 0x7f069007b0d0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:08.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:08.906+0000 7f06ab6dc640 1 --2- 172.21.15.38:0/2541294748 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f06a40b2e40 0x7f06a40cd6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:08.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:08.906+0000 7f06ab6dc640 1 -- 172.21.15.38:0/2541294748 >> 172.21.15.38:0/2541294748 conn(0x7f06a40b6b20 msgr2=0x7f06a415eaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:08.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:08.906+0000 7f06ab6dc640 1 -- 172.21.15.38:0/2541294748 shutdown_connections 2024-09-21T03:36:08.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:08.906+0000 7f06ab6dc640 1 -- 172.21.15.38:0/2541294748 wait complete. 2024-09-21T03:36:08.921 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:36:09.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:09.036+0000 7f29ace0a640 1 -- 172.21.15.38:0/2293268004 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f29a80b2e40 msgr2=0x7f29a80b3250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:09.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:09.036+0000 7f29ace0a640 1 --2- 172.21.15.38:0/2293268004 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f29a80b2e40 0x7f29a80b3250 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f29a80d5250 tx=0x7f29a002aa30 comp rx=0 tx=0).stop 2024-09-21T03:36:09.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:09.036+0000 7f29ace0a640 1 -- 172.21.15.38:0/2293268004 shutdown_connections 2024-09-21T03:36:09.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:09.036+0000 7f29ace0a640 1 --2- 172.21.15.38:0/2293268004 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f29a80b2e40 0x7f29a80b3250 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:09.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:09.036+0000 7f29ace0a640 1 --2- 172.21.15.38:0/2293268004 >> v2:172.21.15.195:3300/0 conn(0x7f29a8162280 0x7f29a8162690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:09.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:09.036+0000 7f29ace0a640 1 --2- 172.21.15.38:0/2293268004 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f29a8162f00 0x7f29a80d24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:09.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:09.036+0000 7f29ace0a640 1 -- 172.21.15.38:0/2293268004 >> 172.21.15.38:0/2293268004 conn(0x7f29a80b6b20 msgr2=0x7f29a80d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:09.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:09.037+0000 7f29ace0a640 1 -- 172.21.15.38:0/2293268004 shutdown_connections 2024-09-21T03:36:09.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:09.037+0000 7f29ace0a640 1 -- 172.21.15.38:0/2293268004 wait complete. 2024-09-21T03:36:09.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:09.038+0000 7f29ace0a640 1 Processor -- start 2024-09-21T03:36:09.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:09.038+0000 7f29ace0a640 1 -- start start 2024-09-21T03:36:09.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:09.040+0000 7f29ace0a640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f29a80b2e40 0x7f29a81f9400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:09.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:09.040+0000 7f29ace0a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f29a8162280 0x7f29a8158df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:09.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:09.040+0000 7f29ace0a640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f29a81595f0 0x7f29a80cb840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:09.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:09.040+0000 7f29ace0a640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f29a80b3dd0 con 0x7f29a8162280 2024-09-21T03:36:09.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:09.040+0000 7f29ace0a640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f29a80b39a0 con 0x7f29a81595f0 2024-09-21T03:36:09.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:09.040+0000 7f29ace0a640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f29a80b3f50 con 0x7f29a80b2e40 2024-09-21T03:36:09.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:09.040+0000 7f299ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f29a8162280 0x7f29a8158df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:09.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:09.040+0000 7f299ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f29a8162280 0x7f29a8158df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37582/0 (socket says 172.21.15.38:37582) 2024-09-21T03:36:09.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:09.040+0000 7f299ffff640 1 -- 172.21.15.38:0/1043521418 learned_addr learned my addr 172.21.15.38:0/1043521418 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:36:09.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:09.040+0000 7f29a659c640 1 --2- 172.21.15.38:0/1043521418 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f29a80b2e40 0x7f29a81f9400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:09.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:09.040+0000 7f29a6d9d640 1 --2- 172.21.15.38:0/1043521418 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f29a81595f0 0x7f29a80cb840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:09.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:09.040+0000 7f299ffff640 1 -- 172.21.15.38:0/1043521418 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f29a80b2e40 msgr2=0x7f29a81f9400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:09.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:09.040+0000 7f299ffff640 1 --2- 172.21.15.38:0/1043521418 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f29a80b2e40 0x7f29a81f9400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:09.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:09.040+0000 7f299ffff640 1 -- 172.21.15.38:0/1043521418 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f29a81595f0 msgr2=0x7f29a80cb840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:09.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:09.040+0000 7f299ffff640 1 --2- 172.21.15.38:0/1043521418 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f29a81595f0 0x7f29a80cb840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:09.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:09.040+0000 7f299ffff640 1 -- 172.21.15.38:0/1043521418 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29a815fdc0 con 0x7f29a8162280 2024-09-21T03:36:09.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:09.041+0000 7f299ffff640 1 --2- 172.21.15.38:0/1043521418 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f29a8162280 0x7f29a8158df0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f299000d650 tx=0x7f299000db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:09.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:09.041+0000 7f299effd640 1 -- 172.21.15.38:0/1043521418 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2990017070 con 0x7f29a8162280 2024-09-21T03:36:09.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:09.041+0000 7f299effd640 1 -- 172.21.15.38:0/1043521418 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2990004310 con 0x7f29a8162280 2024-09-21T03:36:09.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:09.041+0000 7f29ace0a640 1 -- 172.21.15.38:0/1043521418 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29a815a1c0 con 0x7f29a8162280 2024-09-21T03:36:09.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:09.041+0000 7f299effd640 1 -- 172.21.15.38:0/1043521418 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2990005280 con 0x7f29a8162280 2024-09-21T03:36:09.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:09.041+0000 7f29ace0a640 1 -- 172.21.15.38:0/1043521418 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29a815b120 con 0x7f29a8162280 2024-09-21T03:36:09.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:09.042+0000 7f29ace0a640 1 -- 172.21.15.38:0/1043521418 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29a8162f00 con 0x7f29a8162280 2024-09-21T03:36:09.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:09.043+0000 7f299effd640 1 -- 172.21.15.38:0/1043521418 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f2990005420 con 0x7f29a8162280 2024-09-21T03:36:09.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:09.043+0000 7f299effd640 1 --2- 172.21.15.38:0/1043521418 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2988078b90 0x7f298807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:09.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:09.043+0000 7f299effd640 1 -- 172.21.15.38:0/1043521418 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 6187+0+0 (secure 0 0 0) 0x7f299009b180 con 0x7f29a8162280 2024-09-21T03:36:09.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:09.043+0000 7f29a659c640 1 --2- 172.21.15.38:0/1043521418 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2988078b90 0x7f298807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:09.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:09.044+0000 7f29a659c640 1 --2- 172.21.15.38:0/1043521418 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2988078b90 0x7f298807b0a0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f2998004630 tx=0x7f299800b2e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:09.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:09.048+0000 7f299effd640 1 -- 172.21.15.38:0/1043521418 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f2990020020 con 0x7f29a8162280 2024-09-21T03:36:09.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:09.273+0000 7f29ace0a640 1 -- 172.21.15.38:0/1043521418 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f29a80b3570 con 0x7f29a8162280 2024-09-21T03:36:09.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:09.274+0000 7f299effd640 1 -- 172.21.15.38:0/1043521418 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+5230 (secure 0 0 0) 0x7f2990062d80 con 0x7f29a8162280 2024-09-21T03:36:09.276 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:36:09.276 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":36,"btime":"2024-09-21T03:36:08:488383+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5147,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/2858412699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":2858412699},{"type":"v1","addr":"172.21.15.195:6837","nonce":2858412699}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":5368,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/2907144575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":2907144575},{"type":"v1","addr":"172.21.15.195:6836","nonce":2907144575}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":5574,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6834/1911661184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":1911661184},{"type":"v1","addr":"172.21.15.38:6834","nonce":1911661184}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:35:44.271168+0000","modified":"2024-09-21T03:36:08.488376+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":56,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5793},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5793":{"gid":5793,"name":"c","rank":0,"incarnation":33,"state":"up:active","state_seq":4,"addr":"172.21.15.38:6835/3716808119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6833","nonce":3716808119},{"type":"v1","addr":"172.21.15.38:6835","nonce":3716808119}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5793,"qdb_cluster":[5793]},"id":3}]} 2024-09-21T03:36:09.276 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 36 2024-09-21T03:36:09.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:09.276+0000 7f29ace0a640 1 -- 172.21.15.38:0/1043521418 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2988078b90 msgr2=0x7f298807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:09.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:09.276+0000 7f29ace0a640 1 --2- 172.21.15.38:0/1043521418 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2988078b90 0x7f298807b0a0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f2998004630 tx=0x7f299800b2e0 comp rx=0 tx=0).stop 2024-09-21T03:36:09.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:09.276+0000 7f29ace0a640 1 -- 172.21.15.38:0/1043521418 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f29a8162280 msgr2=0x7f29a8158df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:09.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:09.276+0000 7f29ace0a640 1 --2- 172.21.15.38:0/1043521418 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f29a8162280 0x7f29a8158df0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f299000d650 tx=0x7f299000db10 comp rx=0 tx=0).stop 2024-09-21T03:36:09.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:09.276+0000 7f29ace0a640 1 -- 172.21.15.38:0/1043521418 shutdown_connections 2024-09-21T03:36:09.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:09.276+0000 7f29ace0a640 1 --2- 172.21.15.38:0/1043521418 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f29a81595f0 0x7f29a80cb840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:09.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:09.276+0000 7f29ace0a640 1 --2- 172.21.15.38:0/1043521418 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f29a8162280 0x7f29a8158df0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:09.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:09.276+0000 7f29ace0a640 1 --2- 172.21.15.38:0/1043521418 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2988078b90 0x7f298807b0a0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:09.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:09.276+0000 7f29ace0a640 1 --2- 172.21.15.38:0/1043521418 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f29a80b2e40 0x7f29a81f9400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:09.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:09.276+0000 7f29ace0a640 1 -- 172.21.15.38:0/1043521418 >> 172.21.15.38:0/1043521418 conn(0x7f29a80b6b20 msgr2=0x7f29a806cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:09.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:09.276+0000 7f29ace0a640 1 -- 172.21.15.38:0/1043521418 shutdown_connections 2024-09-21T03:36:09.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:09.276+0000 7f29ace0a640 1 -- 172.21.15.38:0/1043521418 wait complete. 2024-09-21T03:36:09.290 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T03:36:09.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:09.404+0000 7f09fc190640 1 -- 172.21.15.38:0/1484239980 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f09dc003fd0 msgr2=0x7f09dc004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:09.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:09.404+0000 7f09fc190640 1 --2- 172.21.15.38:0/1484239980 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f09dc003fd0 0x7f09dc004800 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f09e8009b20 tx=0x7f09dc005830 comp rx=0 tx=0).stop 2024-09-21T03:36:09.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:09.405+0000 7f09fa72d640 1 -- 172.21.15.38:0/1484239980 reap_dead start 2024-09-21T03:36:09.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:09.405+0000 7f09fc190640 1 -- 172.21.15.38:0/1484239980 shutdown_connections 2024-09-21T03:36:09.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:09.405+0000 7f09fc190640 1 -- 172.21.15.38:0/1484239980 >> 172.21.15.38:0/1484239980 conn(0x7f09f40d4440 msgr2=0x7f09f40d4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:09.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:09.405+0000 7f09fc190640 1 -- 172.21.15.38:0/1484239980 shutdown_connections 2024-09-21T03:36:09.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:09.405+0000 7f09fc190640 1 -- 172.21.15.38:0/1484239980 wait complete. 2024-09-21T03:36:09.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:09.406+0000 7f09fc190640 1 Processor -- start 2024-09-21T03:36:09.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:09.406+0000 7f09fc190640 1 -- start start 2024-09-21T03:36:09.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:09.408+0000 7f09fc190640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f09dc003fd0 0x7f09f414c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:09.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:09.408+0000 7f09fc190640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f09f414d540 0x7f09f414d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:09.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:09.408+0000 7f09fc190640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f09f414de90 0x7f09f414eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:09.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:09.408+0000 7f09fc190640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f09f40d2a20 con 0x7f09dc003fd0 2024-09-21T03:36:09.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:09.408+0000 7f09fc190640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f09f40b3b90 con 0x7f09f414de90 2024-09-21T03:36:09.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:09.408+0000 7f09fc190640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f09f40d2ba0 con 0x7f09f414d540 2024-09-21T03:36:09.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:09.408+0000 7f09f9f2c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f09dc003fd0 0x7f09f414c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:09.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:09.408+0000 7f09f9f2c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f09dc003fd0 0x7f09f414c1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37598/0 (socket says 172.21.15.38:37598) 2024-09-21T03:36:09.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:09.408+0000 7f09f9f2c640 1 -- 172.21.15.38:0/2079987904 learned_addr learned my addr 172.21.15.38:0/2079987904 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:36:09.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:09.408+0000 7f09f972b640 1 --2- 172.21.15.38:0/2079987904 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f09f414d540 0x7f09f414d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:09.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:09.408+0000 7f09fa72d640 1 --2- 172.21.15.38:0/2079987904 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f09f414de90 0x7f09f414eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:09.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:09.408+0000 7f09f9f2c640 1 -- 172.21.15.38:0/2079987904 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f09f414d540 msgr2=0x7f09f414d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:09.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:09.408+0000 7f09f9f2c640 1 --2- 172.21.15.38:0/2079987904 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f09f414d540 0x7f09f414d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:09.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:09.408+0000 7f09f9f2c640 1 -- 172.21.15.38:0/2079987904 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f09f414de90 msgr2=0x7f09f414eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:09.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:09.408+0000 7f09f9f2c640 1 --2- 172.21.15.38:0/2079987904 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f09f414de90 0x7f09f414eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:09.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:09.408+0000 7f09f9f2c640 1 -- 172.21.15.38:0/2079987904 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09f414cad0 con 0x7f09dc003fd0 2024-09-21T03:36:09.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:09.408+0000 7f09f9f2c640 1 --2- 172.21.15.38:0/2079987904 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f09dc003fd0 0x7f09f414c1a0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f09e4011ae0 tx=0x7f09e4009e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:09.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:09.408+0000 7f09f27fc640 1 -- 172.21.15.38:0/2079987904 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09e401e070 con 0x7f09dc003fd0 2024-09-21T03:36:09.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:09.409+0000 7f09f27fc640 1 -- 172.21.15.38:0/2079987904 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f09e400fe70 con 0x7f09dc003fd0 2024-09-21T03:36:09.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:09.409+0000 7f09fc190640 1 -- 172.21.15.38:0/2079987904 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09f40c9680 con 0x7f09dc003fd0 2024-09-21T03:36:09.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:09.409+0000 7f09f27fc640 1 -- 172.21.15.38:0/2079987904 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09e4004290 con 0x7f09dc003fd0 2024-09-21T03:36:09.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:09.409+0000 7f09fc190640 1 -- 172.21.15.38:0/2079987904 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09f414c6e0 con 0x7f09dc003fd0 2024-09-21T03:36:09.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:09.410+0000 7f09fc190640 1 -- 172.21.15.38:0/2079987904 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09f41629a0 con 0x7f09dc003fd0 2024-09-21T03:36:09.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:09.410+0000 7f09f27fc640 1 -- 172.21.15.38:0/2079987904 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f09e4021070 con 0x7f09dc003fd0 2024-09-21T03:36:09.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:09.411+0000 7f09f27fc640 1 --2- 172.21.15.38:0/2079987904 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f09e0078bc0 0x7f09e007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:09.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:09.411+0000 7f09f27fc640 1 -- 172.21.15.38:0/2079987904 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 6187+0+0 (secure 0 0 0) 0x7f09e409b8f0 con 0x7f09dc003fd0 2024-09-21T03:36:09.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:09.411+0000 7f09f972b640 1 --2- 172.21.15.38:0/2079987904 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f09e0078bc0 0x7f09e007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:09.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:09.412+0000 7f09f972b640 1 --2- 172.21.15.38:0/2079987904 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f09e0078bc0 0x7f09e007b0d0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f09e8035000 tx=0x7f09e80046c0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:09.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:09.416+0000 7f09f27fc640 1 -- 172.21.15.38:0/2079987904 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f09e4063520 con 0x7f09dc003fd0 2024-09-21T03:36:09.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:09.612+0000 7f09fc190640 1 -- 172.21.15.38:0/2079987904 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f09f40b36d0 con 0x7f09dc003fd0 2024-09-21T03:36:09.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:09.613+0000 7f09f27fc640 1 -- 172.21.15.38:0/2079987904 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v56) ==== 74+0+17095 (secure 0 0 0) 0x7f09e4067f50 con 0x7f09dc003fd0 2024-09-21T03:36:09.615 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:36:09.615 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":56,"fsid":"9d50d66f-656b-4fe2-8e3b-fe53b79c4d43","created":"2024-09-21T03:33:10.677865+0000","modified":"2024-09-21T03:36:04.173381+0000","last_up_change":"2024-09-21T03:33:24.023421+0000","last_in_change":"2024-09-21T03:33:14.296193+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T03:33:22.992573+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-09-21T03:35:41.962122+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-09-21T03:35:43.237360+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dbab183f-7e63-48b8-960a-ae55fd5bf565","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6801","nonce":2381639433}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6803","nonce":2381639433}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6807","nonce":2381639433}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6805","nonce":2381639433}]},"public_addr":"172.21.15.38:6801/2381639433","cluster_addr":"172.21.15.38:6803/2381639433","heartbeat_back_addr":"172.21.15.38:6807/2381639433","heartbeat_front_addr":"172.21.15.38:6805/2381639433","state":["exists","up"]},{"osd":1,"uuid":"fa1c0e8a-ebc1-4359-8146-bd5342abbc16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6817","nonce":1491818732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6819","nonce":1491818732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6823","nonce":1491818732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6821","nonce":1491818732}]},"public_addr":"172.21.15.38:6817/1491818732","cluster_addr":"172.21.15.38:6819/1491818732","heartbeat_back_addr":"172.21.15.38:6823/1491818732","heartbeat_front_addr":"172.21.15.38:6821/1491818732","state":["exists","up"]},{"osd":2,"uuid":"2ed3cd2d-385d-4fc6-9869-214552a84803","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6825","nonce":2577858902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6827","nonce":2577858902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6831","nonce":2577858902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6829","nonce":2577858902}]},"public_addr":"172.21.15.38:6825/2577858902","cluster_addr":"172.21.15.38:6827/2577858902","heartbeat_back_addr":"172.21.15.38:6831/2577858902","heartbeat_front_addr":"172.21.15.38:6829/2577858902","state":["exists","up"]},{"osd":3,"uuid":"62d1d95e-919d-4250-be71-1d34cfa70dbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6809","nonce":1286810294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6811","nonce":1286810294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6815","nonce":1286810294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6813","nonce":1286810294}]},"public_addr":"172.21.15.38:6809/1286810294","cluster_addr":"172.21.15.38:6811/1286810294","heartbeat_back_addr":"172.21.15.38:6815/1286810294","heartbeat_front_addr":"172.21.15.38:6813/1286810294","state":["exists","up"]},{"osd":4,"uuid":"7c9f9314-99d7-4d73-8392-37809a9f54b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6803","nonce":3401837571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6805","nonce":3401837571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6809","nonce":3401837571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6807","nonce":3401837571}]},"public_addr":"172.21.15.195:6803/3401837571","cluster_addr":"172.21.15.195:6805/3401837571","heartbeat_back_addr":"172.21.15.195:6809/3401837571","heartbeat_front_addr":"172.21.15.195:6807/3401837571","state":["exists","up"]},{"osd":5,"uuid":"bde4fe9f-8414-4bba-bf4a-43973d1976bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6811","nonce":2080960180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6813","nonce":2080960180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6817","nonce":2080960180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6815","nonce":2080960180}]},"public_addr":"172.21.15.195:6811/2080960180","cluster_addr":"172.21.15.195:6813/2080960180","heartbeat_back_addr":"172.21.15.195:6817/2080960180","heartbeat_front_addr":"172.21.15.195:6815/2080960180","state":["exists","up"]},{"osd":6,"uuid":"6ee6d7f4-1a5f-4a4c-ac16-48fe05f6a002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6827","nonce":4055825675}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6829","nonce":4055825675}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6833","nonce":4055825675}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6831","nonce":4055825675}]},"public_addr":"172.21.15.195:6827/4055825675","cluster_addr":"172.21.15.195:6829/4055825675","heartbeat_back_addr":"172.21.15.195:6833/4055825675","heartbeat_front_addr":"172.21.15.195:6831/4055825675","state":["exists","up"]},{"osd":7,"uuid":"efeb5f51-329a-4484-8290-92e7d3503d05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6819","nonce":1606106342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6821","nonce":1606106342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6825","nonce":1606106342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6823","nonce":1606106342}]},"public_addr":"172.21.15.195:6819/1606106342","cluster_addr":"172.21.15.195:6821/1606106342","heartbeat_back_addr":"172.21.15.195:6825/1606106342","heartbeat_front_addr":"172.21.15.195:6823/1606106342","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.901343+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:21.115003+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.901623+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.835238+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.132199+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:19.817560+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.829690+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:21.087300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.38:6835/3184989541":"2024-09-22T03:36:04.173351+0000","172.21.15.38:6833/3184989541":"2024-09-22T03:36:04.173351+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T03:36:09.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:09.615+0000 7f09fc190640 1 -- 172.21.15.38:0/2079987904 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f09e0078bc0 msgr2=0x7f09e007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:09.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:09.615+0000 7f09fc190640 1 --2- 172.21.15.38:0/2079987904 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f09e0078bc0 0x7f09e007b0d0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f09e8035000 tx=0x7f09e80046c0 comp rx=0 tx=0).stop 2024-09-21T03:36:09.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:09.615+0000 7f09fc190640 1 -- 172.21.15.38:0/2079987904 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f09dc003fd0 msgr2=0x7f09f414c1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:09.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:09.615+0000 7f09fc190640 1 --2- 172.21.15.38:0/2079987904 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f09dc003fd0 0x7f09f414c1a0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f09e4011ae0 tx=0x7f09e4009e90 comp rx=0 tx=0).stop 2024-09-21T03:36:09.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:09.615+0000 7f09fc190640 1 -- 172.21.15.38:0/2079987904 shutdown_connections 2024-09-21T03:36:09.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:09.615+0000 7f09fc190640 1 --2- 172.21.15.38:0/2079987904 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f09f414de90 0x7f09f414eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:09.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:09.615+0000 7f09fc190640 1 --2- 172.21.15.38:0/2079987904 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f09e0078bc0 0x7f09e007b0d0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:09.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:09.615+0000 7f09fc190640 1 --2- 172.21.15.38:0/2079987904 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f09f414d540 0x7f09f414d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:09.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:09.615+0000 7f09fc190640 1 --2- 172.21.15.38:0/2079987904 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f09dc003fd0 0x7f09f414c1a0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:09.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:09.615+0000 7f09fc190640 1 -- 172.21.15.38:0/2079987904 >> 172.21.15.38:0/2079987904 conn(0x7f09f40d4440 msgr2=0x7f09f40d15f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:09.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:09.615+0000 7f09fc190640 1 -- 172.21.15.38:0/2079987904 shutdown_connections 2024-09-21T03:36:09.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:09.615+0000 7f09fc190640 1 -- 172.21.15.38:0/2079987904 wait complete. 2024-09-21T03:36:09.629 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-21T03:36:09.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:09.744+0000 7fd40948b640 1 -- 172.21.15.38:0/2294696235 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd4040d0bb0 msgr2=0x7fd4040b5be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:09.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:09.744+0000 7fd40948b640 1 --2- 172.21.15.38:0/2294696235 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd4040d0bb0 0x7fd4040b5be0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fd4040cc9c0 tx=0x7fd3fc02a560 comp rx=0 tx=0).stop 2024-09-21T03:36:09.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:09.744+0000 7fd40948b640 1 -- 172.21.15.38:0/2294696235 shutdown_connections 2024-09-21T03:36:09.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:09.744+0000 7fd40948b640 1 --2- 172.21.15.38:0/2294696235 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd4040d0bb0 0x7fd4040b5be0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:09.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:09.744+0000 7fd40948b640 1 --2- 172.21.15.38:0/2294696235 >> v2:172.21.15.195:3300/0 conn(0x7fd40415ca90 0x7fd4040d0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:09.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:09.744+0000 7fd40948b640 1 -- 172.21.15.38:0/2294696235 >> 172.21.15.38:0/2294696235 conn(0x7fd4040b46e0 msgr2=0x7fd4040d23e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:09.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:09.744+0000 7fd40948b640 1 -- 172.21.15.38:0/2294696235 shutdown_connections 2024-09-21T03:36:09.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:09.745+0000 7fd40948b640 1 -- 172.21.15.38:0/2294696235 wait complete. 2024-09-21T03:36:09.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:09.746+0000 7fd40948b640 1 Processor -- start 2024-09-21T03:36:09.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:09.746+0000 7fd40948b640 1 -- start start 2024-09-21T03:36:09.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:09.748+0000 7fd40948b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd4040ccc40 0x7fd4040ca180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:09.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:09.748+0000 7fd40948b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd4040d0bb0 0x7fd40415f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:09.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:09.748+0000 7fd40948b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd40415ca90 0x7fd4040cb130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:09.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:09.748+0000 7fd40948b640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fd4040b6620 con 0x7fd40415ca90 2024-09-21T03:36:09.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:09.748+0000 7fd40948b640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd4040b64a0 con 0x7fd4040d0bb0 2024-09-21T03:36:09.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:09.748+0000 7fd40948b640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fd4040b67a0 con 0x7fd4040ccc40 2024-09-21T03:36:09.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:09.748+0000 7fd4037fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd40415ca90 0x7fd4040cb130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:09.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:09.748+0000 7fd4037fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd40415ca90 0x7fd4040cb130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37626/0 (socket says 172.21.15.38:37626) 2024-09-21T03:36:09.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:09.748+0000 7fd4037fe640 1 -- 172.21.15.38:0/3597563701 learned_addr learned my addr 172.21.15.38:0/3597563701 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:36:09.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:09.748+0000 7fd402ffd640 1 --2- 172.21.15.38:0/3597563701 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd4040ccc40 0x7fd4040ca180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:09.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:09.748+0000 7fd4027fc640 1 --2- 172.21.15.38:0/3597563701 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd4040d0bb0 0x7fd40415f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:09.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:09.748+0000 7fd4037fe640 1 -- 172.21.15.38:0/3597563701 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd4040ccc40 msgr2=0x7fd4040ca180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:09.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:09.748+0000 7fd4037fe640 1 --2- 172.21.15.38:0/3597563701 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd4040ccc40 0x7fd4040ca180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:09.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:09.748+0000 7fd4037fe640 1 -- 172.21.15.38:0/3597563701 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd4040d0bb0 msgr2=0x7fd40415f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:09.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:09.748+0000 7fd4037fe640 1 --2- 172.21.15.38:0/3597563701 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd4040d0bb0 0x7fd40415f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:09.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:09.748+0000 7fd4037fe640 1 -- 172.21.15.38:0/3597563701 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4041f4260 con 0x7fd40415ca90 2024-09-21T03:36:09.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:09.748+0000 7fd4037fe640 1 --2- 172.21.15.38:0/3597563701 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd40415ca90 0x7fd4040cb130 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fd3fc009850 tx=0x7fd3fc0038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:09.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:09.749+0000 7fd3fb7fe640 1 -- 172.21.15.38:0/3597563701 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3fc03e030 con 0x7fd40415ca90 2024-09-21T03:36:09.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:09.749+0000 7fd3fb7fe640 1 -- 172.21.15.38:0/3597563701 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd3fc003d90 con 0x7fd40415ca90 2024-09-21T03:36:09.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:09.749+0000 7fd3fb7fe640 1 -- 172.21.15.38:0/3597563701 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3fc035040 con 0x7fd40415ca90 2024-09-21T03:36:09.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:09.749+0000 7fd40948b640 1 -- 172.21.15.38:0/3597563701 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4040c96a0 con 0x7fd40415ca90 2024-09-21T03:36:09.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:09.749+0000 7fd40948b640 1 -- 172.21.15.38:0/3597563701 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4040c9c60 con 0x7fd40415ca90 2024-09-21T03:36:09.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:09.750+0000 7fd40948b640 1 -- 172.21.15.38:0/3597563701 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4040b6d30 con 0x7fd40415ca90 2024-09-21T03:36:09.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:09.750+0000 7fd3fb7fe640 1 -- 172.21.15.38:0/3597563701 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fd3fc041030 con 0x7fd40415ca90 2024-09-21T03:36:09.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:09.751+0000 7fd3fb7fe640 1 --2- 172.21.15.38:0/3597563701 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd3e4078b90 0x7fd3e407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:09.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:09.751+0000 7fd3fb7fe640 1 -- 172.21.15.38:0/3597563701 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 6187+0+0 (secure 0 0 0) 0x7fd3fc0bb6b0 con 0x7fd40415ca90 2024-09-21T03:36:09.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:09.751+0000 7fd402ffd640 1 --2- 172.21.15.38:0/3597563701 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd3e4078b90 0x7fd3e407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:09.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:09.752+0000 7fd402ffd640 1 --2- 172.21.15.38:0/3597563701 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd3e4078b90 0x7fd3e407b0a0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fd3f4001e20 tx=0x7fd3f4009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:09.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:09.756+0000 7fd3fb7fe640 1 -- 172.21.15.38:0/3597563701 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fd3fc0832c0 con 0x7fd40415ca90 2024-09-21T03:36:09.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:09.984+0000 7fd40948b640 1 -- 172.21.15.38:0/3597563701 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fd4040cfb10 con 0x7fd40415ca90 2024-09-21T03:36:09.986 INFO:tasks.ceph.mon.a.smithi038.stderr:2024-09-21T03:36:09.984+0000 7f3f45713640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T03:36:09.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:09.986+0000 7fd3fb7fe640 1 -- 172.21.15.38:0/3597563701 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v37) ==== 161+0+0 (secure 0 0 0) 0x7fd3fc0b9e20 con 0x7fd40415ca90 2024-09-21T03:36:09.988 INFO:teuthology.orchestra.run.smithi038.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-21T03:36:09.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:09.989+0000 7fd40948b640 1 -- 172.21.15.38:0/3597563701 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd3e4078b90 msgr2=0x7fd3e407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:09.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:09.989+0000 7fd40948b640 1 --2- 172.21.15.38:0/3597563701 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd3e4078b90 0x7fd3e407b0a0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fd3f4001e20 tx=0x7fd3f4009040 comp rx=0 tx=0).stop 2024-09-21T03:36:09.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:09.989+0000 7fd40948b640 1 -- 172.21.15.38:0/3597563701 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd40415ca90 msgr2=0x7fd4040cb130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:09.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:09.989+0000 7fd40948b640 1 --2- 172.21.15.38:0/3597563701 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd40415ca90 0x7fd4040cb130 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fd3fc009850 tx=0x7fd3fc0038a0 comp rx=0 tx=0).stop 2024-09-21T03:36:09.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:09.990+0000 7fd40948b640 1 -- 172.21.15.38:0/3597563701 shutdown_connections 2024-09-21T03:36:09.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:09.990+0000 7fd40948b640 1 --2- 172.21.15.38:0/3597563701 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd40415ca90 0x7fd4040cb130 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:09.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:09.990+0000 7fd40948b640 1 --2- 172.21.15.38:0/3597563701 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd4040d0bb0 0x7fd40415f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:09.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:09.990+0000 7fd40948b640 1 --2- 172.21.15.38:0/3597563701 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd3e4078b90 0x7fd3e407b0a0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:09.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:09.990+0000 7fd40948b640 1 --2- 172.21.15.38:0/3597563701 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd4040ccc40 0x7fd4040ca180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:09.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:09.990+0000 7fd40948b640 1 -- 172.21.15.38:0/3597563701 >> 172.21.15.38:0/3597563701 conn(0x7fd4040b46e0 msgr2=0x7fd4040c8830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:09.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:09.990+0000 7fd40948b640 1 -- 172.21.15.38:0/3597563701 shutdown_connections 2024-09-21T03:36:09.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:09.990+0000 7fd40948b640 1 -- 172.21.15.38:0/3597563701 wait complete. 2024-09-21T03:36:10.004 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-21T03:36:10.048 INFO:tasks.ceph.mds.c.smithi038.stdout:starting mds.c at 2024-09-21T03:36:10.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:10.122+0000 7fd063e8b640 1 -- 172.21.15.38:0/3841102663 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd05c15bfc0 msgr2=0x7fd05c15f630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:10.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:10.122+0000 7fd063e8b640 1 --2- 172.21.15.38:0/3841102663 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd05c15bfc0 0x7fd05c15f630 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fd05c06e6e0 tx=0x7fd05402aa00 comp rx=0 tx=0).stop 2024-09-21T03:36:10.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:10.122+0000 7fd063e8b640 1 -- 172.21.15.38:0/3841102663 shutdown_connections 2024-09-21T03:36:10.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:10.122+0000 7fd063e8b640 1 --2- 172.21.15.38:0/3841102663 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd05c15bfc0 0x7fd05c15f630 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:10.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:10.122+0000 7fd063e8b640 1 --2- 172.21.15.38:0/3841102663 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd05c0b5260 0x7fd05c15ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:10.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:10.122+0000 7fd063e8b640 1 --2- 172.21.15.38:0/3841102663 >> v2:172.21.15.195:3300/0 conn(0x7fd05c06def0 0x7fd05c06f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:10.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:10.123+0000 7fd063e8b640 1 -- 172.21.15.38:0/3841102663 >> 172.21.15.38:0/3841102663 conn(0x7fd05c0cdca0 msgr2=0x7fd05c0ce0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:10.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:10.123+0000 7fd063e8b640 1 -- 172.21.15.38:0/3841102663 shutdown_connections 2024-09-21T03:36:10.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:10.123+0000 7fd063e8b640 1 -- 172.21.15.38:0/3841102663 wait complete. 2024-09-21T03:36:10.126 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:10.124+0000 7fd063e8b640 1 Processor -- start 2024-09-21T03:36:10.126 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:10.124+0000 7fd063e8b640 1 -- start start 2024-09-21T03:36:10.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:10.126+0000 7fd063e8b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd05c06def0 0x7fd05c0b1640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:10.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:10.126+0000 7fd063e8b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd05c0b5260 0x7fd05c15a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:10.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:10.126+0000 7fd063e8b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd05c15bfc0 0x7fd05c1f5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:10.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:10.126+0000 7fd063e8b640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fd05c15fcf0 con 0x7fd05c15bfc0 2024-09-21T03:36:10.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:10.126+0000 7fd063e8b640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd05c15fb70 con 0x7fd05c06def0 2024-09-21T03:36:10.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:10.126+0000 7fd063e8b640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fd05c15fe70 con 0x7fd05c0b5260 2024-09-21T03:36:10.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:10.126+0000 7fd062428640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd05c15bfc0 0x7fd05c1f5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:10.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:10.126+0000 7fd062428640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd05c15bfc0 0x7fd05c1f5300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37660/0 (socket says 172.21.15.38:37660) 2024-09-21T03:36:10.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:10.126+0000 7fd062428640 1 -- 172.21.15.38:0/2513908744 learned_addr learned my addr 172.21.15.38:0/2513908744 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:36:10.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:10.126+0000 7fd061c27640 1 --2- 172.21.15.38:0/2513908744 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd05c06def0 0x7fd05c0b1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:10.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:10.126+0000 7fd061426640 1 --2- 172.21.15.38:0/2513908744 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd05c0b5260 0x7fd05c15a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:10.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:10.126+0000 7fd062428640 1 -- 172.21.15.38:0/2513908744 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd05c0b5260 msgr2=0x7fd05c15a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:10.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:10.126+0000 7fd062428640 1 --2- 172.21.15.38:0/2513908744 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd05c0b5260 0x7fd05c15a950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:10.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:10.126+0000 7fd062428640 1 -- 172.21.15.38:0/2513908744 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd05c06def0 msgr2=0x7fd05c0b1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:10.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:10.126+0000 7fd062428640 1 --2- 172.21.15.38:0/2513908744 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd05c06def0 0x7fd05c0b1640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:10.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:10.126+0000 7fd062428640 1 -- 172.21.15.38:0/2513908744 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd05c0b1e00 con 0x7fd05c15bfc0 2024-09-21T03:36:10.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:10.126+0000 7fd062428640 1 --2- 172.21.15.38:0/2513908744 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd05c15bfc0 0x7fd05c1f5300 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fd05402aff0 tx=0x7fd054003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:10.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:10.126+0000 7fd0527fc640 1 -- 172.21.15.38:0/2513908744 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd054033050 con 0x7fd05c15bfc0 2024-09-21T03:36:10.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:10.127+0000 7fd0527fc640 1 -- 172.21.15.38:0/2513908744 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd054004450 con 0x7fd05c15bfc0 2024-09-21T03:36:10.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:10.127+0000 7fd063e8b640 1 -- 172.21.15.38:0/2513908744 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd05c0b20e0 con 0x7fd05c15bfc0 2024-09-21T03:36:10.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:10.127+0000 7fd0527fc640 1 -- 172.21.15.38:0/2513908744 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd054038450 con 0x7fd05c15bfc0 2024-09-21T03:36:10.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:10.127+0000 7fd063e8b640 1 -- 172.21.15.38:0/2513908744 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd05c1f4de0 con 0x7fd05c15bfc0 2024-09-21T03:36:10.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:10.128+0000 7fd063e8b640 1 -- 172.21.15.38:0/2513908744 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd05c0b2600 con 0x7fd05c15bfc0 2024-09-21T03:36:10.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:10.129+0000 7fd0527fc640 1 -- 172.21.15.38:0/2513908744 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fd054036070 con 0x7fd05c15bfc0 2024-09-21T03:36:10.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:10.129+0000 7fd0527fc640 1 --2- 172.21.15.38:0/2513908744 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd040078bc0 0x7fd04007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:10.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:10.129+0000 7fd0527fc640 1 -- 172.21.15.38:0/2513908744 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 6273+0+0 (secure 0 0 0) 0x7fd0540bc670 con 0x7fd05c15bfc0 2024-09-21T03:36:10.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:10.130+0000 7fd061c27640 1 --2- 172.21.15.38:0/2513908744 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd040078bc0 0x7fd04007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:10.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:10.130+0000 7fd061c27640 1 --2- 172.21.15.38:0/2513908744 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd040078bc0 0x7fd04007b0d0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fd058005fd0 tx=0x7fd058005850 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:10.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:10.133+0000 7fd0527fc640 1 -- 172.21.15.38:0/2513908744 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fd05c0b2600 con 0x7fd05c15bfc0 2024-09-21T03:36:10.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:10.360+0000 7fd063e8b640 1 -- 172.21.15.38:0/2513908744 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fd05c0b1b90 con 0x7fd05c15bfc0 2024-09-21T03:36:10.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:10.988+0000 7fd0527fc640 1 -- 172.21.15.38:0/2513908744 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v38) ==== 104+0+0 (secure 0 0 0) 0x7fd054084850 con 0x7fd05c15bfc0 2024-09-21T03:36:10.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:10.990+0000 7fd063e8b640 1 -- 172.21.15.38:0/2513908744 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd040078bc0 msgr2=0x7fd04007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:10.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:10.990+0000 7fd063e8b640 1 --2- 172.21.15.38:0/2513908744 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd040078bc0 0x7fd04007b0d0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fd058005fd0 tx=0x7fd058005850 comp rx=0 tx=0).stop 2024-09-21T03:36:10.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:10.990+0000 7fd063e8b640 1 -- 172.21.15.38:0/2513908744 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd05c15bfc0 msgr2=0x7fd05c1f5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:10.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:10.990+0000 7fd063e8b640 1 --2- 172.21.15.38:0/2513908744 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd05c15bfc0 0x7fd05c1f5300 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fd05402aff0 tx=0x7fd054003f10 comp rx=0 tx=0).stop 2024-09-21T03:36:10.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:10.990+0000 7fd063e8b640 1 -- 172.21.15.38:0/2513908744 shutdown_connections 2024-09-21T03:36:10.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:10.991+0000 7fd063e8b640 1 --2- 172.21.15.38:0/2513908744 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd05c15bfc0 0x7fd05c1f5300 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:10.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:10.991+0000 7fd063e8b640 1 --2- 172.21.15.38:0/2513908744 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd040078bc0 0x7fd04007b0d0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:10.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:10.991+0000 7fd063e8b640 1 --2- 172.21.15.38:0/2513908744 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd05c0b5260 0x7fd05c15a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:10.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:10.991+0000 7fd063e8b640 1 --2- 172.21.15.38:0/2513908744 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd05c06def0 0x7fd05c0b1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:10.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:10.991+0000 7fd063e8b640 1 -- 172.21.15.38:0/2513908744 >> 172.21.15.38:0/2513908744 conn(0x7fd05c0cdca0 msgr2=0x7fd05c15cd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:10.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:10.991+0000 7fd063e8b640 1 -- 172.21.15.38:0/2513908744 shutdown_connections 2024-09-21T03:36:10.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:10.991+0000 7fd063e8b640 1 -- 172.21.15.38:0/2513908744 wait complete. 2024-09-21T03:36:11.005 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-21T03:36:11.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:11.126+0000 7f61232c5640 1 -- 172.21.15.38:0/247068044 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f611c163350 msgr2=0x7f611c15c650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:11.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:11.126+0000 7f61232c5640 1 --2- 172.21.15.38:0/247068044 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f611c163350 0x7f611c15c650 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f6118005fd0 tx=0x7f611802a4b0 comp rx=0 tx=0).stop 2024-09-21T03:36:11.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:11.126+0000 7f61232c5640 1 -- 172.21.15.38:0/247068044 shutdown_connections 2024-09-21T03:36:11.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:11.126+0000 7f61232c5640 1 --2- 172.21.15.38:0/247068044 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f611c162ec0 0x7f611c0b3000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:11.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:11.126+0000 7f61232c5640 1 --2- 172.21.15.38:0/247068044 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f611c163350 0x7f611c15c650 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:11.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:11.126+0000 7f61232c5640 1 -- 172.21.15.38:0/247068044 >> 172.21.15.38:0/247068044 conn(0x7f611c06cde0 msgr2=0x7f611c06d1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:11.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:11.126+0000 7f61232c5640 1 -- 172.21.15.38:0/247068044 shutdown_connections 2024-09-21T03:36:11.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:11.126+0000 7f61232c5640 1 -- 172.21.15.38:0/247068044 wait complete. 2024-09-21T03:36:11.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:11.128+0000 7f61232c5640 1 Processor -- start 2024-09-21T03:36:11.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:11.128+0000 7f61232c5640 1 -- start start 2024-09-21T03:36:11.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:11.129+0000 7f61232c5640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f611c0d0e40 0x7f611c0c7f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:11.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:11.129+0000 7f61232c5640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f611c163350 0x7f611c1467b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:11.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:11.129+0000 7f61232c5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f611c0ca880 0x7f611c161c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:11.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:11.130+0000 7f61232c5640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f611c0b39e0 con 0x7f611c0ca880 2024-09-21T03:36:11.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:11.130+0000 7f61232c5640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f611c0b3860 con 0x7f611c0d0e40 2024-09-21T03:36:11.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:11.130+0000 7f61232c5640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f611c0b3b60 con 0x7f611c163350 2024-09-21T03:36:11.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:11.130+0000 7f6121862640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f611c0ca880 0x7f611c161c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:11.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:11.130+0000 7f6121862640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f611c0ca880 0x7f611c161c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37682/0 (socket says 172.21.15.38:37682) 2024-09-21T03:36:11.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:11.130+0000 7f6121862640 1 -- 172.21.15.38:0/2652381377 learned_addr learned my addr 172.21.15.38:0/2652381377 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:36:11.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:11.130+0000 7f6121061640 1 --2- 172.21.15.38:0/2652381377 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f611c0d0e40 0x7f611c0c7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:11.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:11.130+0000 7f6120860640 1 --2- 172.21.15.38:0/2652381377 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f611c163350 0x7f611c1467b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:11.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:11.130+0000 7f6121862640 1 -- 172.21.15.38:0/2652381377 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f611c163350 msgr2=0x7f611c1467b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:11.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:11.130+0000 7f6121862640 1 --2- 172.21.15.38:0/2652381377 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f611c163350 0x7f611c1467b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:11.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:11.130+0000 7f6121862640 1 -- 172.21.15.38:0/2652381377 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f611c0d0e40 msgr2=0x7f611c0c7f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:11.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:11.130+0000 7f6121862640 1 --2- 172.21.15.38:0/2652381377 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f611c0d0e40 0x7f611c0c7f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:11.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:11.130+0000 7f6121862640 1 -- 172.21.15.38:0/2652381377 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f611c0b67f0 con 0x7f611c0ca880 2024-09-21T03:36:11.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:11.130+0000 7f6121862640 1 --2- 172.21.15.38:0/2652381377 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f611c0ca880 0x7f611c161c70 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f611400e9c0 tx=0x7f611400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:11.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:11.130+0000 7f6111ffb640 1 -- 172.21.15.38:0/2652381377 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6114004d20 con 0x7f611c0ca880 2024-09-21T03:36:11.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:11.130+0000 7f6111ffb640 1 -- 172.21.15.38:0/2652381377 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f611400cb20 con 0x7f611c0ca880 2024-09-21T03:36:11.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:11.130+0000 7f61232c5640 1 -- 172.21.15.38:0/2652381377 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f611c0b6ad0 con 0x7f611c0ca880 2024-09-21T03:36:11.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:11.130+0000 7f6111ffb640 1 -- 172.21.15.38:0/2652381377 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6114005020 con 0x7f611c0ca880 2024-09-21T03:36:11.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:11.131+0000 7f61232c5640 1 -- 172.21.15.38:0/2652381377 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f611c161750 con 0x7f611c0ca880 2024-09-21T03:36:11.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:11.132+0000 7f61232c5640 1 -- 172.21.15.38:0/2652381377 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f611c0b3000 con 0x7f611c0ca880 2024-09-21T03:36:11.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:11.132+0000 7f6111ffb640 1 -- 172.21.15.38:0/2652381377 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f61140051c0 con 0x7f611c0ca880 2024-09-21T03:36:11.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:11.133+0000 7f6111ffb640 1 --2- 172.21.15.38:0/2652381377 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6100078bc0 0x7f610007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:11.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:11.133+0000 7f6111ffb640 1 -- 172.21.15.38:0/2652381377 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 6273+0+0 (secure 0 0 0) 0x7f611409b740 con 0x7f611c0ca880 2024-09-21T03:36:11.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:11.133+0000 7f6121061640 1 --2- 172.21.15.38:0/2652381377 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6100078bc0 0x7f610007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:11.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:11.134+0000 7f6121061640 1 --2- 172.21.15.38:0/2652381377 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6100078bc0 0x7f610007b0d0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f6118033000 tx=0x7f6118004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:11.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:11.138+0000 7f6111ffb640 1 -- 172.21.15.38:0/2652381377 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f6114016030 con 0x7f611c0ca880 2024-09-21T03:36:11.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:11.325+0000 7f61232c5640 1 -- 172.21.15.38:0/2652381377 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f611c0cadb0 con 0x7f611c0ca880 2024-09-21T03:36:11.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:11.994+0000 7f6111ffb640 1 -- 172.21.15.38:0/2652381377 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v58) ==== 181+0+0 (secure 0 0 0) 0x7f6114063380 con 0x7f611c0ca880 2024-09-21T03:36:11.996 INFO:teuthology.orchestra.run.smithi038.stderr:pool 'cephfs_metadata' removed 2024-09-21T03:36:12.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:11.998+0000 7f61232c5640 1 -- 172.21.15.38:0/2652381377 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6100078bc0 msgr2=0x7f610007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:12.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:11.998+0000 7f61232c5640 1 --2- 172.21.15.38:0/2652381377 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6100078bc0 0x7f610007b0d0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f6118033000 tx=0x7f6118004730 comp rx=0 tx=0).stop 2024-09-21T03:36:12.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:11.998+0000 7f61232c5640 1 -- 172.21.15.38:0/2652381377 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f611c0ca880 msgr2=0x7f611c161c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:12.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:11.998+0000 7f61232c5640 1 --2- 172.21.15.38:0/2652381377 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f611c0ca880 0x7f611c161c70 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f611400e9c0 tx=0x7f611400ee80 comp rx=0 tx=0).stop 2024-09-21T03:36:12.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:11.998+0000 7f61232c5640 1 -- 172.21.15.38:0/2652381377 shutdown_connections 2024-09-21T03:36:12.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:11.998+0000 7f61232c5640 1 --2- 172.21.15.38:0/2652381377 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f611c0ca880 0x7f611c161c70 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:12.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:11.998+0000 7f61232c5640 1 --2- 172.21.15.38:0/2652381377 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6100078bc0 0x7f610007b0d0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:12.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:11.998+0000 7f61232c5640 1 --2- 172.21.15.38:0/2652381377 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f611c163350 0x7f611c1467b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:12.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:11.998+0000 7f61232c5640 1 --2- 172.21.15.38:0/2652381377 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f611c0d0e40 0x7f611c0c7f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:12.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:11.998+0000 7f61232c5640 1 -- 172.21.15.38:0/2652381377 >> 172.21.15.38:0/2652381377 conn(0x7f611c06cde0 msgr2=0x7f611c15cd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:12.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:11.998+0000 7f61232c5640 1 -- 172.21.15.38:0/2652381377 shutdown_connections 2024-09-21T03:36:12.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:11.999+0000 7f61232c5640 1 -- 172.21.15.38:0/2652381377 wait complete. 2024-09-21T03:36:12.015 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-21T03:36:12.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:12.141+0000 7fe9f3d7b640 1 -- 172.21.15.38:0/3602510732 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe9ec0b1d30 msgr2=0x7fe9ec0ccc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:12.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:12.141+0000 7fe9f3d7b640 1 --2- 172.21.15.38:0/3602510732 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe9ec0b1d30 0x7fe9ec0ccc30 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fe9ec075b80 tx=0x7fe9e002a4d0 comp rx=0 tx=0).stop 2024-09-21T03:36:12.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:12.142+0000 7fe9f3d7b640 1 -- 172.21.15.38:0/3602510732 shutdown_connections 2024-09-21T03:36:12.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:12.142+0000 7fe9f3d7b640 1 --2- 172.21.15.38:0/3602510732 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe9ec0b1d30 0x7fe9ec0ccc30 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:12.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:12.142+0000 7fe9f3d7b640 1 --2- 172.21.15.38:0/3602510732 >> v2:172.21.15.195:3300/0 conn(0x7fe9ec160310 0x7fe9ec0b35b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:12.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:12.142+0000 7fe9f3d7b640 1 -- 172.21.15.38:0/3602510732 >> 172.21.15.38:0/3602510732 conn(0x7fe9ec0b3ec0 msgr2=0x7fe9ec0b42d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:12.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:12.142+0000 7fe9f3d7b640 1 -- 172.21.15.38:0/3602510732 shutdown_connections 2024-09-21T03:36:12.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:12.142+0000 7fe9f3d7b640 1 -- 172.21.15.38:0/3602510732 wait complete. 2024-09-21T03:36:12.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:12.143+0000 7fe9f3d7b640 1 Processor -- start 2024-09-21T03:36:12.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:12.143+0000 7fe9f3d7b640 1 -- start start 2024-09-21T03:36:12.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:12.145+0000 7fe9f3d7b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe9ec0b1d30 0x7fe9ec154ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:12.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:12.145+0000 7fe9f3d7b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe9ec0cd170 0x7fe9ec152130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:12.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:12.145+0000 7fe9f3d7b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe9ec160310 0x7fe9ec152670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:12.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:12.145+0000 7fe9f3d7b640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fe9ec15daa0 con 0x7fe9ec0cd170 2024-09-21T03:36:12.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:12.145+0000 7fe9f3d7b640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fe9ec15d920 con 0x7fe9ec0b1d30 2024-09-21T03:36:12.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:12.145+0000 7fe9f3d7b640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fe9ec15dc20 con 0x7fe9ec160310 2024-09-21T03:36:12.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:12.145+0000 7fe9f1316640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe9ec0cd170 0x7fe9ec152130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:12.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:12.145+0000 7fe9f1316640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe9ec0cd170 0x7fe9ec152130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37700/0 (socket says 172.21.15.38:37700) 2024-09-21T03:36:12.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:12.145+0000 7fe9f1316640 1 -- 172.21.15.38:0/1686742087 learned_addr learned my addr 172.21.15.38:0/1686742087 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:36:12.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:12.145+0000 7fe9f1b17640 1 --2- 172.21.15.38:0/1686742087 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe9ec0b1d30 0x7fe9ec154ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:12.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:12.145+0000 7fe9f2318640 1 --2- 172.21.15.38:0/1686742087 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe9ec160310 0x7fe9ec152670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:12.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:12.145+0000 7fe9f1316640 1 -- 172.21.15.38:0/1686742087 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe9ec160310 msgr2=0x7fe9ec152670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:12.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:12.145+0000 7fe9f1316640 1 --2- 172.21.15.38:0/1686742087 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe9ec160310 0x7fe9ec152670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:12.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:12.145+0000 7fe9f1316640 1 -- 172.21.15.38:0/1686742087 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe9ec0b1d30 msgr2=0x7fe9ec154ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:12.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:12.145+0000 7fe9f1316640 1 --2- 172.21.15.38:0/1686742087 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe9ec0b1d30 0x7fe9ec154ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:12.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:12.145+0000 7fe9f1316640 1 -- 172.21.15.38:0/1686742087 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9ec0cab10 con 0x7fe9ec0cd170 2024-09-21T03:36:12.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:12.146+0000 7fe9f1316640 1 --2- 172.21.15.38:0/1686742087 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe9ec0cd170 0x7fe9ec152130 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fe9e0034040 tx=0x7fe9e00039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:12.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:12.146+0000 7fe9ea7fc640 1 -- 172.21.15.38:0/1686742087 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9e003e030 con 0x7fe9ec0cd170 2024-09-21T03:36:12.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:12.146+0000 7fe9ea7fc640 1 -- 172.21.15.38:0/1686742087 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe9e00047a0 con 0x7fe9ec0cd170 2024-09-21T03:36:12.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:12.146+0000 7fe9f3d7b640 1 -- 172.21.15.38:0/1686742087 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9ec0cadf0 con 0x7fe9ec0cd170 2024-09-21T03:36:12.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:12.146+0000 7fe9ea7fc640 1 -- 172.21.15.38:0/1686742087 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9e0035040 con 0x7fe9ec0cd170 2024-09-21T03:36:12.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:12.146+0000 7fe9f3d7b640 1 -- 172.21.15.38:0/1686742087 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9ec0cb3b0 con 0x7fe9ec0cd170 2024-09-21T03:36:12.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:12.147+0000 7fe9f3d7b640 1 -- 172.21.15.38:0/1686742087 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9ec155220 con 0x7fe9ec0cd170 2024-09-21T03:36:12.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:12.148+0000 7fe9ea7fc640 1 -- 172.21.15.38:0/1686742087 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fe9e0041030 con 0x7fe9ec0cd170 2024-09-21T03:36:12.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:12.148+0000 7fe9ea7fc640 1 --2- 172.21.15.38:0/1686742087 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fe9d8078b90 0x7fe9d807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:12.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:12.148+0000 7fe9ea7fc640 1 -- 172.21.15.38:0/1686742087 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5835+0+0 (secure 0 0 0) 0x7fe9e00bb920 con 0x7fe9ec0cd170 2024-09-21T03:36:12.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:12.148+0000 7fe9f1b17640 1 --2- 172.21.15.38:0/1686742087 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fe9d8078b90 0x7fe9d807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:12.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:12.149+0000 7fe9f1b17640 1 --2- 172.21.15.38:0/1686742087 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fe9d8078b90 0x7fe9d807b0a0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fe9e4005fd0 tx=0x7fe9e400a250 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:12.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:12.152+0000 7fe9ea7fc640 1 -- 172.21.15.38:0/1686742087 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fe9ec0cd580 con 0x7fe9ec0cd170 2024-09-21T03:36:12.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:12.337+0000 7fe9f3d7b640 1 -- 172.21.15.38:0/1686742087 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fe9ec0d0ec0 con 0x7fe9ec0cd170 2024-09-21T03:36:12.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:12.996+0000 7fe9ea7fc640 1 -- 172.21.15.38:0/1686742087 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v59) ==== 169+0+0 (secure 0 0 0) 0x7fe9e0083810 con 0x7fe9ec0cd170 2024-09-21T03:36:12.998 INFO:teuthology.orchestra.run.smithi038.stderr:pool 'cephfs_data' removed 2024-09-21T03:36:13.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:12.999+0000 7fe9f3d7b640 1 -- 172.21.15.38:0/1686742087 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fe9d8078b90 msgr2=0x7fe9d807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:13.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:12.999+0000 7fe9f3d7b640 1 --2- 172.21.15.38:0/1686742087 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fe9d8078b90 0x7fe9d807b0a0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fe9e4005fd0 tx=0x7fe9e400a250 comp rx=0 tx=0).stop 2024-09-21T03:36:13.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:12.999+0000 7fe9f3d7b640 1 -- 172.21.15.38:0/1686742087 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe9ec0cd170 msgr2=0x7fe9ec152130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:13.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:12.999+0000 7fe9f3d7b640 1 --2- 172.21.15.38:0/1686742087 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe9ec0cd170 0x7fe9ec152130 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fe9e0034040 tx=0x7fe9e00039e0 comp rx=0 tx=0).stop 2024-09-21T03:36:13.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:13.000+0000 7fe9f3d7b640 1 -- 172.21.15.38:0/1686742087 shutdown_connections 2024-09-21T03:36:13.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:13.000+0000 7fe9f3d7b640 1 --2- 172.21.15.38:0/1686742087 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fe9d8078b90 0x7fe9d807b0a0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:13.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:13.000+0000 7fe9f3d7b640 1 --2- 172.21.15.38:0/1686742087 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe9ec160310 0x7fe9ec152670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:13.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:13.000+0000 7fe9f3d7b640 1 --2- 172.21.15.38:0/1686742087 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe9ec0cd170 0x7fe9ec152130 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:13.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:13.000+0000 7fe9f3d7b640 1 --2- 172.21.15.38:0/1686742087 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe9ec0b1d30 0x7fe9ec154ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:13.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:13.000+0000 7fe9f3d7b640 1 -- 172.21.15.38:0/1686742087 >> 172.21.15.38:0/1686742087 conn(0x7fe9ec0b3ec0 msgr2=0x7fe9ec0b42d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:13.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:13.002+0000 7fe9f3d7b640 1 -- 172.21.15.38:0/1686742087 shutdown_connections 2024-09-21T03:36:13.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:13.002+0000 7fe9f3d7b640 1 -- 172.21.15.38:0/1686742087 wait complete. 2024-09-21T03:36:13.019 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 31 2024-09-21T03:36:13.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:13.138+0000 7f56a130b640 1 -- 172.21.15.38:0/3489753577 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f569c0cf7d0 msgr2=0x7f569c072d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:13.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:13.138+0000 7f56a130b640 1 --2- 172.21.15.38:0/3489753577 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f569c0cf7d0 0x7f569c072d90 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f569c075be0 tx=0x7f569402a560 comp rx=0 tx=0).stop 2024-09-21T03:36:13.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:13.138+0000 7f56a130b640 1 -- 172.21.15.38:0/3489753577 shutdown_connections 2024-09-21T03:36:13.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:13.138+0000 7f56a130b640 1 --2- 172.21.15.38:0/3489753577 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f569c0cf7d0 0x7f569c072d90 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:13.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:13.138+0000 7f56a130b640 1 --2- 172.21.15.38:0/3489753577 >> v2:172.21.15.195:3300/0 conn(0x7f569c06e330 0x7f569c0cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:13.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:13.138+0000 7f56a130b640 1 --2- 172.21.15.38:0/3489753577 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f569c15fc50 0x7f569c06ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:13.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:13.138+0000 7f56a130b640 1 -- 172.21.15.38:0/3489753577 >> 172.21.15.38:0/3489753577 conn(0x7f569c0b1b90 msgr2=0x7f569c0b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:13.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:13.138+0000 7f56a130b640 1 -- 172.21.15.38:0/3489753577 shutdown_connections 2024-09-21T03:36:13.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:13.138+0000 7f56a130b640 1 -- 172.21.15.38:0/3489753577 wait complete. 2024-09-21T03:36:13.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:13.140+0000 7f56a130b640 1 Processor -- start 2024-09-21T03:36:13.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:13.140+0000 7f56a130b640 1 -- start start 2024-09-21T03:36:13.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:13.141+0000 7f56a130b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f569c06e330 0x7f569c1f3a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:13.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:13.141+0000 7f56a130b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f569c0cf7d0 0x7f569c15e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:13.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:13.141+0000 7f56a130b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f569c15fc50 0x7f569c1f6420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:13.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:13.141+0000 7f56a130b640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f569c070ba0 con 0x7f569c15fc50 2024-09-21T03:36:13.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:13.141+0000 7f56a130b640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f569c0cfbe0 con 0x7f569c0cf7d0 2024-09-21T03:36:13.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:13.141+0000 7f56a130b640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f569c070d20 con 0x7f569c06e330 2024-09-21T03:36:13.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:13.141+0000 7f569b7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f569c15fc50 0x7f569c1f6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:13.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:13.141+0000 7f569b7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f569c15fc50 0x7f569c1f6420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42158/0 (socket says 172.21.15.38:42158) 2024-09-21T03:36:13.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:13.141+0000 7f569b7fe640 1 -- 172.21.15.38:0/851955657 learned_addr learned my addr 172.21.15.38:0/851955657 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:36:13.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:13.142+0000 7f569affd640 1 --2- 172.21.15.38:0/851955657 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f569c06e330 0x7f569c1f3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:13.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:13.142+0000 7f569a7fc640 1 --2- 172.21.15.38:0/851955657 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f569c0cf7d0 0x7f569c15e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:13.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:13.142+0000 7f569b7fe640 1 -- 172.21.15.38:0/851955657 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f569c06e330 msgr2=0x7f569c1f3a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:13.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:13.142+0000 7f569b7fe640 1 --2- 172.21.15.38:0/851955657 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f569c06e330 0x7f569c1f3a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:13.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:13.142+0000 7f569b7fe640 1 -- 172.21.15.38:0/851955657 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f569c0cf7d0 msgr2=0x7f569c15e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:13.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:13.142+0000 7f569b7fe640 1 --2- 172.21.15.38:0/851955657 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f569c0cf7d0 0x7f569c15e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:13.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:13.142+0000 7f569b7fe640 1 -- 172.21.15.38:0/851955657 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f569c0a10a0 con 0x7f569c15fc50 2024-09-21T03:36:13.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:13.142+0000 7f569a7fc640 1 --2- 172.21.15.38:0/851955657 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f569c0cf7d0 0x7f569c15e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:36:13.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:13.142+0000 7f569b7fe640 1 --2- 172.21.15.38:0/851955657 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f569c15fc50 0x7f569c1f6420 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f5694034040 tx=0x7f5694003960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:13.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:13.142+0000 7f56937fe640 1 -- 172.21.15.38:0/851955657 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f569403e030 con 0x7f569c15fc50 2024-09-21T03:36:13.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:13.142+0000 7f56937fe640 1 -- 172.21.15.38:0/851955657 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5694003e50 con 0x7f569c15fc50 2024-09-21T03:36:13.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:13.142+0000 7f56a130b640 1 -- 172.21.15.38:0/851955657 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f569c1f7050 con 0x7f569c15fc50 2024-09-21T03:36:13.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:13.142+0000 7f56937fe640 1 -- 172.21.15.38:0/851955657 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5694035040 con 0x7f569c15fc50 2024-09-21T03:36:13.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:13.142+0000 7f56a130b640 1 -- 172.21.15.38:0/851955657 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f569c1f2f90 con 0x7f569c15fc50 2024-09-21T03:36:13.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:13.143+0000 7f56a130b640 1 -- 172.21.15.38:0/851955657 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f569c1f6dc0 con 0x7f569c15fc50 2024-09-21T03:36:13.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:13.144+0000 7f56937fe640 1 -- 172.21.15.38:0/851955657 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f5694041030 con 0x7f569c15fc50 2024-09-21T03:36:13.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:13.144+0000 7f56937fe640 1 --2- 172.21.15.38:0/851955657 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f567c078b90 0x7f567c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:13.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:13.145+0000 7f56937fe640 1 -- 172.21.15.38:0/851955657 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5421+0+0 (secure 0 0 0) 0x7f56940bb4b0 con 0x7f569c15fc50 2024-09-21T03:36:13.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:13.145+0000 7f569affd640 1 --2- 172.21.15.38:0/851955657 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f567c078b90 0x7f567c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:13.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:13.145+0000 7f569affd640 1 --2- 172.21.15.38:0/851955657 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f567c078b90 0x7f567c07b0a0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f568c005fd0 tx=0x7f568c0097d0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:13.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:13.149+0000 7f56937fe640 1 -- 172.21.15.38:0/851955657 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f5694083a10 con 0x7f569c15fc50 2024-09-21T03:36:13.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:13.337+0000 7f56a130b640 1 -- 172.21.15.38:0/851955657 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "config reset", "num": 31} v 0) -- 0x7f569c072d90 con 0x7f569c15fc50 2024-09-21T03:36:13.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:13.341+0000 7f56937fe640 1 -- 172.21.15.38:0/851955657 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 31}]=0 v38) ==== 71+0+0 (secure 0 0 0) 0x7f5694083310 con 0x7f569c15fc50 2024-09-21T03:36:13.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:13.344+0000 7f56a130b640 1 -- 172.21.15.38:0/851955657 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f567c078b90 msgr2=0x7f567c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:13.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:13.344+0000 7f56a130b640 1 --2- 172.21.15.38:0/851955657 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f567c078b90 0x7f567c07b0a0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f568c005fd0 tx=0x7f568c0097d0 comp rx=0 tx=0).stop 2024-09-21T03:36:13.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:13.344+0000 7f56a130b640 1 -- 172.21.15.38:0/851955657 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f569c15fc50 msgr2=0x7f569c1f6420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:13.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:13.344+0000 7f56a130b640 1 --2- 172.21.15.38:0/851955657 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f569c15fc50 0x7f569c1f6420 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f5694034040 tx=0x7f5694003960 comp rx=0 tx=0).stop 2024-09-21T03:36:13.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:13.344+0000 7f56a130b640 1 -- 172.21.15.38:0/851955657 shutdown_connections 2024-09-21T03:36:13.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:13.344+0000 7f56a130b640 1 --2- 172.21.15.38:0/851955657 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f569c15fc50 0x7f569c1f6420 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:13.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:13.344+0000 7f56a130b640 1 --2- 172.21.15.38:0/851955657 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f569c0cf7d0 0x7f569c15e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:13.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:13.344+0000 7f56a130b640 1 --2- 172.21.15.38:0/851955657 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f567c078b90 0x7f567c07b0a0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:13.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:13.344+0000 7f56a130b640 1 --2- 172.21.15.38:0/851955657 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f569c06e330 0x7f569c1f3a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:13.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:13.344+0000 7f56a130b640 1 -- 172.21.15.38:0/851955657 >> 172.21.15.38:0/851955657 conn(0x7f569c0b1b90 msgr2=0x7f569c073600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:13.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:13.344+0000 7f56a130b640 1 -- 172.21.15.38:0/851955657 shutdown_connections 2024-09-21T03:36:13.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:13.344+0000 7f56a130b640 1 -- 172.21.15.38:0/851955657 wait complete. 2024-09-21T03:36:13.357 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_damage.TestDamage.test_dentry_first_existing' 2024-09-21T03:36:13.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:13.472+0000 7fc6a80dc640 1 -- 172.21.15.38:0/2736201241 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc688005150 msgr2=0x7fc68800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:13.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:13.472+0000 7fc6a80dc640 1 --2- 172.21.15.38:0/2736201241 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc688005150 0x7fc68800b9c0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fc698005fd0 tx=0x7fc688005830 comp rx=0 tx=0).stop 2024-09-21T03:36:13.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:13.472+0000 7fc6a6679640 1 -- 172.21.15.38:0/2736201241 reap_dead start 2024-09-21T03:36:13.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:13.472+0000 7fc6a80dc640 1 -- 172.21.15.38:0/2736201241 shutdown_connections 2024-09-21T03:36:13.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:13.472+0000 7fc6a80dc640 1 -- 172.21.15.38:0/2736201241 >> 172.21.15.38:0/2736201241 conn(0x7fc6a00cef40 msgr2=0x7fc6a00cf350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:13.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:13.472+0000 7fc6a80dc640 1 -- 172.21.15.38:0/2736201241 shutdown_connections 2024-09-21T03:36:13.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:13.472+0000 7fc6a80dc640 1 -- 172.21.15.38:0/2736201241 wait complete. 2024-09-21T03:36:13.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:13.474+0000 7fc6a80dc640 1 Processor -- start 2024-09-21T03:36:13.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:13.474+0000 7fc6a80dc640 1 -- start start 2024-09-21T03:36:13.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:13.475+0000 7fc6a80dc640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc688005150 0x7fc6a014a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:13.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:13.475+0000 7fc6a80dc640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc6a015bbc0 0x7fc6a015bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:13.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:13.475+0000 7fc6a80dc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc6a0070ef0 0x7fc6a0071300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:13.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:13.475+0000 7fc6a80dc640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fc6a00cd1c0 con 0x7fc6a0070ef0 2024-09-21T03:36:13.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:13.475+0000 7fc6a80dc640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fc6a00cd040 con 0x7fc6a015bbc0 2024-09-21T03:36:13.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:13.475+0000 7fc6a80dc640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fc6a00d04c0 con 0x7fc688005150 2024-09-21T03:36:13.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:13.475+0000 7fc6a6679640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc6a0070ef0 0x7fc6a0071300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:13.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:13.475+0000 7fc6a6679640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc6a0070ef0 0x7fc6a0071300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42184/0 (socket says 172.21.15.38:42184) 2024-09-21T03:36:13.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:13.475+0000 7fc6a6679640 1 -- 172.21.15.38:0/2114404275 learned_addr learned my addr 172.21.15.38:0/2114404275 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:36:13.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:13.476+0000 7fc6a5677640 1 --2- 172.21.15.38:0/2114404275 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc6a015bbc0 0x7fc6a015bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:13.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:13.476+0000 7fc6a5e78640 1 --2- 172.21.15.38:0/2114404275 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc688005150 0x7fc6a014a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:13.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:13.476+0000 7fc6a6679640 1 -- 172.21.15.38:0/2114404275 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc688005150 msgr2=0x7fc6a014a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:13.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:13.476+0000 7fc6a6679640 1 --2- 172.21.15.38:0/2114404275 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc688005150 0x7fc6a014a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:13.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:13.476+0000 7fc6a6679640 1 -- 172.21.15.38:0/2114404275 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc6a015bbc0 msgr2=0x7fc6a015bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:13.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:13.476+0000 7fc6a6679640 1 --2- 172.21.15.38:0/2114404275 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc6a015bbc0 0x7fc6a015bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:13.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:13.476+0000 7fc6a6679640 1 -- 172.21.15.38:0/2114404275 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6a0071840 con 0x7fc6a0070ef0 2024-09-21T03:36:13.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:13.476+0000 7fc6a5e78640 1 --2- 172.21.15.38:0/2114404275 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc688005150 0x7fc6a014a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:36:13.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:13.476+0000 7fc6a5677640 1 --2- 172.21.15.38:0/2114404275 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc6a015bbc0 0x7fc6a015bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:36:13.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:13.476+0000 7fc6a6679640 1 --2- 172.21.15.38:0/2114404275 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc6a0070ef0 0x7fc6a0071300 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fc698009f10 tx=0x7fc698033670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:13.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:13.476+0000 7fc69e7fc640 1 -- 172.21.15.38:0/2114404275 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc698038040 con 0x7fc6a0070ef0 2024-09-21T03:36:13.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:13.476+0000 7fc69e7fc640 1 -- 172.21.15.38:0/2114404275 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc698042070 con 0x7fc6a0070ef0 2024-09-21T03:36:13.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:13.476+0000 7fc6a80dc640 1 -- 172.21.15.38:0/2114404275 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6a01f60e0 con 0x7fc6a0070ef0 2024-09-21T03:36:13.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:13.476+0000 7fc69e7fc640 1 -- 172.21.15.38:0/2114404275 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc698033940 con 0x7fc6a0070ef0 2024-09-21T03:36:13.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:13.476+0000 7fc6a80dc640 1 -- 172.21.15.38:0/2114404275 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6a01f66a0 con 0x7fc6a0070ef0 2024-09-21T03:36:13.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:13.477+0000 7fc6a80dc640 1 -- 172.21.15.38:0/2114404275 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6a014edf0 con 0x7fc6a0070ef0 2024-09-21T03:36:13.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:13.478+0000 7fc69e7fc640 1 -- 172.21.15.38:0/2114404275 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc698033ae0 con 0x7fc6a0070ef0 2024-09-21T03:36:13.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:13.478+0000 7fc69e7fc640 1 --2- 172.21.15.38:0/2114404275 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc68c078b90 0x7fc68c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:13.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:13.479+0000 7fc69e7fc640 1 -- 172.21.15.38:0/2114404275 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5421+0+0 (secure 0 0 0) 0x7fc6980bb060 con 0x7fc6a0070ef0 2024-09-21T03:36:13.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:13.479+0000 7fc6a5e78640 1 --2- 172.21.15.38:0/2114404275 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc68c078b90 0x7fc68c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:13.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:13.479+0000 7fc6a5e78640 1 --2- 172.21.15.38:0/2114404275 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc68c078b90 0x7fc68c07b0a0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fc690009cd0 tx=0x7fc690005f90 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:13.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:13.483+0000 7fc69e7fc640 1 -- 172.21.15.38:0/2114404275 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fc6a014edf0 con 0x7fc6a0070ef0 2024-09-21T03:36:13.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:13.746+0000 7fc6a80dc640 1 -- 172.21.15.38:0/2114404275 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_damage.TestDamage.test_dentry_first_existing"]} v 0) -- 0x7fc6a00d17c0 con 0x7fc6a0070ef0 2024-09-21T03:36:14.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:14.001+0000 7fc69e7fc640 1 -- 172.21.15.38:0/2114404275 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_damage.TestDamage.test_dentry_first_existing"]}]=0 v153) ==== 141+0+0 (secure 0 0 0) 0x7fc6a0071840 con 0x7fc6a0070ef0 2024-09-21T03:36:14.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:14.003+0000 7fc6a80dc640 1 -- 172.21.15.38:0/2114404275 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc68c078b90 msgr2=0x7fc68c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:14.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:14.004+0000 7fc6a80dc640 1 --2- 172.21.15.38:0/2114404275 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc68c078b90 0x7fc68c07b0a0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fc690009cd0 tx=0x7fc690005f90 comp rx=0 tx=0).stop 2024-09-21T03:36:14.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:14.004+0000 7fc6a80dc640 1 -- 172.21.15.38:0/2114404275 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc6a0070ef0 msgr2=0x7fc6a0071300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:14.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:14.004+0000 7fc6a80dc640 1 --2- 172.21.15.38:0/2114404275 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc6a0070ef0 0x7fc6a0071300 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fc698009f10 tx=0x7fc698033670 comp rx=0 tx=0).stop 2024-09-21T03:36:14.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:14.004+0000 7fc6a80dc640 1 -- 172.21.15.38:0/2114404275 shutdown_connections 2024-09-21T03:36:14.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:14.004+0000 7fc6a80dc640 1 --2- 172.21.15.38:0/2114404275 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc6a0070ef0 0x7fc6a0071300 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:14.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:14.004+0000 7fc6a80dc640 1 --2- 172.21.15.38:0/2114404275 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc6a015bbc0 0x7fc6a015bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:14.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:14.004+0000 7fc6a80dc640 1 --2- 172.21.15.38:0/2114404275 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc68c078b90 0x7fc68c07b0a0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:14.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:14.004+0000 7fc6a80dc640 1 --2- 172.21.15.38:0/2114404275 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc688005150 0x7fc6a014a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:14.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:14.004+0000 7fc6a80dc640 1 -- 172.21.15.38:0/2114404275 >> 172.21.15.38:0/2114404275 conn(0x7fc6a00cef40 msgr2=0x7fc6a014b610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:14.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:14.004+0000 7fc6a80dc640 1 -- 172.21.15.38:0/2114404275 shutdown_connections 2024-09-21T03:36:14.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:14.004+0000 7fc6a80dc640 1 -- 172.21.15.38:0/2114404275 wait complete. 2024-09-21T03:36:14.017 INFO:tasks.cephfs_test_runner:That the MDS won't abort when the dentry is already known to be damaged. ... ok 2024-09-21T03:36:14.017 INFO:tasks.cephfs_test_runner:Starting test: test_dentry_first_precommit (tasks.cephfs.test_damage.TestDamage) That the MDS won't write a dentry with new damage to CDentry::first 2024-09-21T03:36:14.017 INFO:tasks.cephfs_test_runner:test_dentry_first_precommit (tasks.cephfs.test_damage.TestDamage) 2024-09-21T03:36:14.018 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T03:36:14.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:14.134+0000 7f1efb532640 1 -- 172.21.15.38:0/34543161 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1ef40d1d00 msgr2=0x7f1ef40cc930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:14.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:14.134+0000 7f1efb532640 1 --2- 172.21.15.38:0/34543161 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1ef40d1d00 0x7f1ef40cc930 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1ee8009830 tx=0x7f1ee802a500 comp rx=0 tx=0).stop 2024-09-21T03:36:14.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:14.134+0000 7f1efb532640 1 -- 172.21.15.38:0/34543161 shutdown_connections 2024-09-21T03:36:14.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:14.134+0000 7f1efb532640 1 --2- 172.21.15.38:0/34543161 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1ef40d1d00 0x7f1ef40cc930 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:14.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:14.134+0000 7f1efb532640 1 --2- 172.21.15.38:0/34543161 >> v2:172.21.15.195:3300/0 conn(0x7f1ef41536e0 0x7f1ef40d17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:14.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:14.134+0000 7f1efb532640 1 -- 172.21.15.38:0/34543161 >> 172.21.15.38:0/34543161 conn(0x7f1ef40cef40 msgr2=0x7f1ef40cf350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:14.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:14.134+0000 7f1efb532640 1 -- 172.21.15.38:0/34543161 shutdown_connections 2024-09-21T03:36:14.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:14.134+0000 7f1efb532640 1 -- 172.21.15.38:0/34543161 wait complete. 2024-09-21T03:36:14.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:14.136+0000 7f1efb532640 1 Processor -- start 2024-09-21T03:36:14.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:14.136+0000 7f1efb532640 1 -- start start 2024-09-21T03:36:14.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:14.137+0000 7f1efb532640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1ef40cce70 0x7f1ef414f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:14.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:14.137+0000 7f1efb532640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1ef40d1d00 0x7f1ef41f7010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:14.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:14.137+0000 7f1efb532640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1ef41536e0 0x7f1ef41f7550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:14.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:14.137+0000 7f1efb532640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f1ef41608a0 con 0x7f1ef41536e0 2024-09-21T03:36:14.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:14.137+0000 7f1efb532640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f1ef4160a20 con 0x7f1ef40d1d00 2024-09-21T03:36:14.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:14.137+0000 7f1efb532640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f1ef4160ba0 con 0x7f1ef40cce70 2024-09-21T03:36:14.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:14.137+0000 7f1ef9acf640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1ef41536e0 0x7f1ef41f7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:14.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:14.137+0000 7f1ef9acf640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1ef41536e0 0x7f1ef41f7550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42200/0 (socket says 172.21.15.38:42200) 2024-09-21T03:36:14.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:14.137+0000 7f1ef9acf640 1 -- 172.21.15.38:0/3310205566 learned_addr learned my addr 172.21.15.38:0/3310205566 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:36:14.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:14.137+0000 7f1ef92ce640 1 --2- 172.21.15.38:0/3310205566 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1ef40cce70 0x7f1ef414f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:14.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:14.137+0000 7f1ef8acd640 1 --2- 172.21.15.38:0/3310205566 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1ef40d1d00 0x7f1ef41f7010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:14.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:14.138+0000 7f1ef9acf640 1 -- 172.21.15.38:0/3310205566 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1ef40cce70 msgr2=0x7f1ef414f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:14.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:14.138+0000 7f1ef9acf640 1 --2- 172.21.15.38:0/3310205566 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1ef40cce70 0x7f1ef414f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:14.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:14.138+0000 7f1ef9acf640 1 -- 172.21.15.38:0/3310205566 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1ef40d1d00 msgr2=0x7f1ef41f7010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:14.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:14.138+0000 7f1ef9acf640 1 --2- 172.21.15.38:0/3310205566 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1ef40d1d00 0x7f1ef41f7010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:14.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:14.138+0000 7f1ef9acf640 1 -- 172.21.15.38:0/3310205566 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ef41e68f0 con 0x7f1ef41536e0 2024-09-21T03:36:14.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:14.138+0000 7f1ef9acf640 1 --2- 172.21.15.38:0/3310205566 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1ef41536e0 0x7f1ef41f7550 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f1eec00c3c0 tx=0x7f1eec00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:14.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:14.138+0000 7f1ef1ffb640 1 -- 172.21.15.38:0/3310205566 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1eec011790 con 0x7f1ef41536e0 2024-09-21T03:36:14.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:14.138+0000 7f1ef1ffb640 1 -- 172.21.15.38:0/3310205566 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1eec017070 con 0x7f1ef41536e0 2024-09-21T03:36:14.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:14.138+0000 7f1efb532640 1 -- 172.21.15.38:0/3310205566 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ef41e6bd0 con 0x7f1ef41536e0 2024-09-21T03:36:14.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:14.138+0000 7f1ef1ffb640 1 -- 172.21.15.38:0/3310205566 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1eec011ca0 con 0x7f1ef41536e0 2024-09-21T03:36:14.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:14.138+0000 7f1efb532640 1 -- 172.21.15.38:0/3310205566 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ef41e7680 con 0x7f1ef41536e0 2024-09-21T03:36:14.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:14.139+0000 7f1efb532640 1 -- 172.21.15.38:0/3310205566 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ef41e70f0 con 0x7f1ef41536e0 2024-09-21T03:36:14.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:14.140+0000 7f1ef1ffb640 1 -- 172.21.15.38:0/3310205566 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f1eec01c460 con 0x7f1ef41536e0 2024-09-21T03:36:14.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:14.141+0000 7f1ef1ffb640 1 --2- 172.21.15.38:0/3310205566 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1ee0078b90 0x7f1ee007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:14.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:14.141+0000 7f1ef1ffb640 1 -- 172.21.15.38:0/3310205566 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5421+0+0 (secure 0 0 0) 0x7f1eec09a810 con 0x7f1ef41536e0 2024-09-21T03:36:14.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:14.141+0000 7f1ef92ce640 1 --2- 172.21.15.38:0/3310205566 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1ee0078b90 0x7f1ee007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:14.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:14.142+0000 7f1ef92ce640 1 --2- 172.21.15.38:0/3310205566 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1ee0078b90 0x7f1ee007b0a0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f1ee40099a0 tx=0x7f1ee4009500 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:14.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:14.146+0000 7f1ef1ffb640 1 -- 172.21.15.38:0/3310205566 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f1ef41e70f0 con 0x7f1ef41536e0 2024-09-21T03:36:14.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:14.337+0000 7f1efb532640 1 -- 172.21.15.38:0/3310205566 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1ef40cc930 con 0x7f1ef41536e0 2024-09-21T03:36:14.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:14.337+0000 7f1ef1ffb640 1 -- 172.21.15.38:0/3310205566 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v59) ==== 74+0+13459 (secure 0 0 0) 0x7f1eec0627c0 con 0x7f1ef41536e0 2024-09-21T03:36:14.339 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:36:14.340 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":59,"fsid":"9d50d66f-656b-4fe2-8e3b-fe53b79c4d43","created":"2024-09-21T03:33:10.677865+0000","modified":"2024-09-21T03:36:12.995195+0000","last_up_change":"2024-09-21T03:33:24.023421+0000","last_in_change":"2024-09-21T03:33:14.296193+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T03:33:22.992573+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dbab183f-7e63-48b8-960a-ae55fd5bf565","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6801","nonce":2381639433}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6803","nonce":2381639433}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6807","nonce":2381639433}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6805","nonce":2381639433}]},"public_addr":"172.21.15.38:6801/2381639433","cluster_addr":"172.21.15.38:6803/2381639433","heartbeat_back_addr":"172.21.15.38:6807/2381639433","heartbeat_front_addr":"172.21.15.38:6805/2381639433","state":["exists","up"]},{"osd":1,"uuid":"fa1c0e8a-ebc1-4359-8146-bd5342abbc16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6817","nonce":1491818732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6819","nonce":1491818732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6823","nonce":1491818732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6821","nonce":1491818732}]},"public_addr":"172.21.15.38:6817/1491818732","cluster_addr":"172.21.15.38:6819/1491818732","heartbeat_back_addr":"172.21.15.38:6823/1491818732","heartbeat_front_addr":"172.21.15.38:6821/1491818732","state":["exists","up"]},{"osd":2,"uuid":"2ed3cd2d-385d-4fc6-9869-214552a84803","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6825","nonce":2577858902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6827","nonce":2577858902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6831","nonce":2577858902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6829","nonce":2577858902}]},"public_addr":"172.21.15.38:6825/2577858902","cluster_addr":"172.21.15.38:6827/2577858902","heartbeat_back_addr":"172.21.15.38:6831/2577858902","heartbeat_front_addr":"172.21.15.38:6829/2577858902","state":["exists","up"]},{"osd":3,"uuid":"62d1d95e-919d-4250-be71-1d34cfa70dbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6809","nonce":1286810294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6811","nonce":1286810294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6815","nonce":1286810294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6813","nonce":1286810294}]},"public_addr":"172.21.15.38:6809/1286810294","cluster_addr":"172.21.15.38:6811/1286810294","heartbeat_back_addr":"172.21.15.38:6815/1286810294","heartbeat_front_addr":"172.21.15.38:6813/1286810294","state":["exists","up"]},{"osd":4,"uuid":"7c9f9314-99d7-4d73-8392-37809a9f54b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6803","nonce":3401837571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6805","nonce":3401837571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6809","nonce":3401837571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6807","nonce":3401837571}]},"public_addr":"172.21.15.195:6803/3401837571","cluster_addr":"172.21.15.195:6805/3401837571","heartbeat_back_addr":"172.21.15.195:6809/3401837571","heartbeat_front_addr":"172.21.15.195:6807/3401837571","state":["exists","up"]},{"osd":5,"uuid":"bde4fe9f-8414-4bba-bf4a-43973d1976bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6811","nonce":2080960180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6813","nonce":2080960180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6817","nonce":2080960180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6815","nonce":2080960180}]},"public_addr":"172.21.15.195:6811/2080960180","cluster_addr":"172.21.15.195:6813/2080960180","heartbeat_back_addr":"172.21.15.195:6817/2080960180","heartbeat_front_addr":"172.21.15.195:6815/2080960180","state":["exists","up"]},{"osd":6,"uuid":"6ee6d7f4-1a5f-4a4c-ac16-48fe05f6a002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6827","nonce":4055825675}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6829","nonce":4055825675}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6833","nonce":4055825675}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6831","nonce":4055825675}]},"public_addr":"172.21.15.195:6827/4055825675","cluster_addr":"172.21.15.195:6829/4055825675","heartbeat_back_addr":"172.21.15.195:6833/4055825675","heartbeat_front_addr":"172.21.15.195:6831/4055825675","state":["exists","up"]},{"osd":7,"uuid":"efeb5f51-329a-4484-8290-92e7d3503d05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6819","nonce":1606106342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6821","nonce":1606106342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6825","nonce":1606106342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6823","nonce":1606106342}]},"public_addr":"172.21.15.195:6819/1606106342","cluster_addr":"172.21.15.195:6821/1606106342","heartbeat_back_addr":"172.21.15.195:6825/1606106342","heartbeat_front_addr":"172.21.15.195:6823/1606106342","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.901343+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:21.115003+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.901623+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.835238+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.132199+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:19.817560+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.829690+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:21.087300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.38:6835/3716808119":"2024-09-22T03:36:09.985455+0000","172.21.15.38:6833/3716808119":"2024-09-22T03:36:09.985455+0000","172.21.15.38:6835/3184989541":"2024-09-22T03:36:04.173351+0000","172.21.15.38:6833/3184989541":"2024-09-22T03:36:04.173351+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T03:36:14.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:14.340+0000 7f1efb532640 1 -- 172.21.15.38:0/3310205566 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1ee0078b90 msgr2=0x7f1ee007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:14.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:14.340+0000 7f1efb532640 1 --2- 172.21.15.38:0/3310205566 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1ee0078b90 0x7f1ee007b0a0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f1ee40099a0 tx=0x7f1ee4009500 comp rx=0 tx=0).stop 2024-09-21T03:36:14.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:14.340+0000 7f1efb532640 1 -- 172.21.15.38:0/3310205566 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1ef41536e0 msgr2=0x7f1ef41f7550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:14.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:14.340+0000 7f1efb532640 1 --2- 172.21.15.38:0/3310205566 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1ef41536e0 0x7f1ef41f7550 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f1eec00c3c0 tx=0x7f1eec00c880 comp rx=0 tx=0).stop 2024-09-21T03:36:14.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:14.340+0000 7f1efb532640 1 -- 172.21.15.38:0/3310205566 shutdown_connections 2024-09-21T03:36:14.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:14.340+0000 7f1efb532640 1 --2- 172.21.15.38:0/3310205566 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1ef41536e0 0x7f1ef41f7550 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:14.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:14.340+0000 7f1efb532640 1 --2- 172.21.15.38:0/3310205566 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1ef40d1d00 0x7f1ef41f7010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:14.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:14.340+0000 7f1efb532640 1 --2- 172.21.15.38:0/3310205566 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1ee0078b90 0x7f1ee007b0a0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:14.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:14.340+0000 7f1efb532640 1 --2- 172.21.15.38:0/3310205566 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1ef40cce70 0x7f1ef414f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:14.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:14.340+0000 7f1efb532640 1 -- 172.21.15.38:0/3310205566 >> 172.21.15.38:0/3310205566 conn(0x7f1ef40cef40 msgr2=0x7f1ef4152d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:14.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:14.340+0000 7f1efb532640 1 -- 172.21.15.38:0/3310205566 shutdown_connections 2024-09-21T03:36:14.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:14.340+0000 7f1efb532640 1 -- 172.21.15.38:0/3310205566 wait complete. 2024-09-21T03:36:14.353 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T03:33:22.992573+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-21T03:36:14.353 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-21T03:36:14.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:14.470+0000 7fd02261f640 1 -- 172.21.15.38:0/1259490550 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd01c15ee80 msgr2=0x7fd01c0d0170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:14.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:14.470+0000 7fd02261f640 1 --2- 172.21.15.38:0/1259490550 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd01c15ee80 0x7fd01c0d0170 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fd00c009830 tx=0x7fd00c02a4a0 comp rx=0 tx=0).stop 2024-09-21T03:36:14.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:14.470+0000 7fd02261f640 1 -- 172.21.15.38:0/1259490550 shutdown_connections 2024-09-21T03:36:14.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:14.470+0000 7fd02261f640 1 --2- 172.21.15.38:0/1259490550 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd01c15ee80 0x7fd01c0d0170 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:14.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:14.470+0000 7fd02261f640 1 -- 172.21.15.38:0/1259490550 >> 172.21.15.38:0/1259490550 conn(0x7fd01c09d830 msgr2=0x7fd01c0b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:14.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:14.470+0000 7fd02261f640 1 -- 172.21.15.38:0/1259490550 shutdown_connections 2024-09-21T03:36:14.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:14.470+0000 7fd02261f640 1 -- 172.21.15.38:0/1259490550 wait complete. 2024-09-21T03:36:14.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:14.472+0000 7fd02261f640 1 Processor -- start 2024-09-21T03:36:14.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:14.472+0000 7fd02261f640 1 -- start start 2024-09-21T03:36:14.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:14.474+0000 7fd02261f640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd01c0b9ad0 0x7fd01c150910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:14.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:14.474+0000 7fd02261f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd01c0ceb90 0x7fd01c1594b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:14.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:14.474+0000 7fd02261f640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd01c15ee80 0x7fd01c15adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:14.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:14.474+0000 7fd02261f640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fd01c0b6910 con 0x7fd01c0ceb90 2024-09-21T03:36:14.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:14.474+0000 7fd02261f640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd01c0b6790 con 0x7fd01c0b9ad0 2024-09-21T03:36:14.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:14.474+0000 7fd02261f640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fd01c0b6a90 con 0x7fd01c15ee80 2024-09-21T03:36:14.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:14.474+0000 7fd01b7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd01c0ceb90 0x7fd01c1594b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:14.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:14.474+0000 7fd01b7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd01c0ceb90 0x7fd01c1594b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42222/0 (socket says 172.21.15.38:42222) 2024-09-21T03:36:14.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:14.474+0000 7fd01b7fe640 1 -- 172.21.15.38:0/693935648 learned_addr learned my addr 172.21.15.38:0/693935648 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:36:14.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:14.474+0000 7fd01bfff640 1 --2- 172.21.15.38:0/693935648 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd01c0b9ad0 0x7fd01c150910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:14.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:14.474+0000 7fd020bbc640 1 --2- 172.21.15.38:0/693935648 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd01c15ee80 0x7fd01c15adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:14.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:14.474+0000 7fd01b7fe640 1 -- 172.21.15.38:0/693935648 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd01c15ee80 msgr2=0x7fd01c15adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:14.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:14.474+0000 7fd01b7fe640 1 --2- 172.21.15.38:0/693935648 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd01c15ee80 0x7fd01c15adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:14.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:14.474+0000 7fd01b7fe640 1 -- 172.21.15.38:0/693935648 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd01c0b9ad0 msgr2=0x7fd01c150910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:14.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:14.474+0000 7fd01b7fe640 1 --2- 172.21.15.38:0/693935648 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd01c0b9ad0 0x7fd01c150910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:14.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:14.474+0000 7fd01b7fe640 1 -- 172.21.15.38:0/693935648 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd01c06cde0 con 0x7fd01c0ceb90 2024-09-21T03:36:14.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:14.474+0000 7fd01b7fe640 1 --2- 172.21.15.38:0/693935648 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd01c0ceb90 0x7fd01c1594b0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fd00400c390 tx=0x7fd00400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:14.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:14.474+0000 7fd018ff9640 1 -- 172.21.15.38:0/693935648 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd004017070 con 0x7fd01c0ceb90 2024-09-21T03:36:14.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:14.474+0000 7fd018ff9640 1 -- 172.21.15.38:0/693935648 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd0040053c0 con 0x7fd01c0ceb90 2024-09-21T03:36:14.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:14.474+0000 7fd018ff9640 1 -- 172.21.15.38:0/693935648 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0040038a0 con 0x7fd01c0ceb90 2024-09-21T03:36:14.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:14.474+0000 7fd02261f640 1 -- 172.21.15.38:0/693935648 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd01c159c50 con 0x7fd01c0ceb90 2024-09-21T03:36:14.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:14.475+0000 7fd02261f640 1 -- 172.21.15.38:0/693935648 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd01c15a210 con 0x7fd01c0ceb90 2024-09-21T03:36:14.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:14.476+0000 7fd02261f640 1 -- 172.21.15.38:0/693935648 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd01c0d0170 con 0x7fd01c0ceb90 2024-09-21T03:36:14.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:14.476+0000 7fd018ff9640 1 -- 172.21.15.38:0/693935648 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fd00401a070 con 0x7fd01c0ceb90 2024-09-21T03:36:14.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:14.477+0000 7fd018ff9640 1 --2- 172.21.15.38:0/693935648 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fcffc078b90 0x7fcffc07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:14.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:14.477+0000 7fd018ff9640 1 -- 172.21.15.38:0/693935648 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5421+0+0 (secure 0 0 0) 0x7fd00409a030 con 0x7fd01c0ceb90 2024-09-21T03:36:14.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:14.477+0000 7fd01bfff640 1 --2- 172.21.15.38:0/693935648 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fcffc078b90 0x7fcffc07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:14.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:14.478+0000 7fd01bfff640 1 --2- 172.21.15.38:0/693935648 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fcffc078b90 0x7fcffc07b0a0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fd00c034000 tx=0x7fd00c004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:14.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:14.482+0000 7fd018ff9640 1 -- 172.21.15.38:0/693935648 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fd004061ea0 con 0x7fd01c0ceb90 2024-09-21T03:36:14.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:14.666+0000 7fd02261f640 1 -- 172.21.15.38:0/693935648 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd01c1613f0 con 0x7fd01c0ceb90 2024-09-21T03:36:14.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:14.667+0000 7fd018ff9640 1 -- 172.21.15.38:0/693935648 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v59) ==== 93+0+10 (secure 0 0 0) 0x7fd0040668d0 con 0x7fd01c0ceb90 2024-09-21T03:36:14.669 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 1 2024-09-21T03:36:14.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:14.669+0000 7fd02261f640 1 -- 172.21.15.38:0/693935648 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fcffc078b90 msgr2=0x7fcffc07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:14.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:14.669+0000 7fd02261f640 1 --2- 172.21.15.38:0/693935648 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fcffc078b90 0x7fcffc07b0a0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fd00c034000 tx=0x7fd00c004730 comp rx=0 tx=0).stop 2024-09-21T03:36:14.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:14.669+0000 7fd02261f640 1 -- 172.21.15.38:0/693935648 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd01c0ceb90 msgr2=0x7fd01c1594b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:14.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:14.669+0000 7fd02261f640 1 --2- 172.21.15.38:0/693935648 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd01c0ceb90 0x7fd01c1594b0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fd00400c390 tx=0x7fd00400c850 comp rx=0 tx=0).stop 2024-09-21T03:36:14.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:14.669+0000 7fd02261f640 1 -- 172.21.15.38:0/693935648 shutdown_connections 2024-09-21T03:36:14.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:14.669+0000 7fd02261f640 1 --2- 172.21.15.38:0/693935648 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fcffc078b90 0x7fcffc07b0a0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:14.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:14.669+0000 7fd02261f640 1 --2- 172.21.15.38:0/693935648 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd01c15ee80 0x7fd01c15adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:14.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:14.669+0000 7fd02261f640 1 --2- 172.21.15.38:0/693935648 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd01c0ceb90 0x7fd01c1594b0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:14.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:14.669+0000 7fd02261f640 1 --2- 172.21.15.38:0/693935648 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd01c0b9ad0 0x7fd01c150910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:14.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:14.669+0000 7fd02261f640 1 -- 172.21.15.38:0/693935648 >> 172.21.15.38:0/693935648 conn(0x7fd01c09d830 msgr2=0x7fd01c161910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:14.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:14.669+0000 7fd02261f640 1 -- 172.21.15.38:0/693935648 shutdown_connections 2024-09-21T03:36:14.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:14.669+0000 7fd02261f640 1 -- 172.21.15.38:0/693935648 wait complete. 2024-09-21T03:36:14.682 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_damage.TestDamage.test_dentry_first_precommit' 2024-09-21T03:36:14.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:14.797+0000 7f7a7c88b640 1 -- 172.21.15.38:0/2984840720 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7a60005270 msgr2=0x7f7a6000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:14.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:14.797+0000 7f7a7c88b640 1 --2- 172.21.15.38:0/2984840720 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7a60005270 0x7f7a6000b9c0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f7a70001dc0 tx=0x7f7a60005830 comp rx=0 tx=0).stop 2024-09-21T03:36:14.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:14.797+0000 7f7a7c88b640 1 -- 172.21.15.38:0/2984840720 shutdown_connections 2024-09-21T03:36:14.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:14.797+0000 7f7a7c88b640 1 --2- 172.21.15.38:0/2984840720 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7a60005270 0x7f7a6000b9c0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:14.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:14.797+0000 7f7a7c88b640 1 -- 172.21.15.38:0/2984840720 >> 172.21.15.38:0/2984840720 conn(0x7f7a780b2cc0 msgr2=0x7f7a780b30d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:14.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:14.797+0000 7f7a7c88b640 1 -- 172.21.15.38:0/2984840720 shutdown_connections 2024-09-21T03:36:14.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:14.797+0000 7f7a7c88b640 1 -- 172.21.15.38:0/2984840720 wait complete. 2024-09-21T03:36:14.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:14.799+0000 7f7a7c88b640 1 Processor -- start 2024-09-21T03:36:14.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:14.799+0000 7f7a7c88b640 1 -- start start 2024-09-21T03:36:14.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:14.801+0000 7f7a7c88b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7a60005270 0x7f7a78149aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:14.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:14.801+0000 7f7a7c88b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7a78157f70 0x7f7a780cc1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:14.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:14.801+0000 7f7a7c88b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7a7815f060 0x7f7a7815f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:14.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:14.801+0000 7f7a7c88b640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f7a78074160 con 0x7f7a7815f060 2024-09-21T03:36:14.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:14.801+0000 7f7a7c88b640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f7a780742e0 con 0x7f7a78157f70 2024-09-21T03:36:14.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:14.801+0000 7f7a7c88b640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f7a78074460 con 0x7f7a60005270 2024-09-21T03:36:14.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:14.801+0000 7f7a76d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7a7815f060 0x7f7a7815f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:14.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:14.801+0000 7f7a76d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7a7815f060 0x7f7a7815f470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42246/0 (socket says 172.21.15.38:42246) 2024-09-21T03:36:14.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:14.801+0000 7f7a76d9d640 1 -- 172.21.15.38:0/3260398908 learned_addr learned my addr 172.21.15.38:0/3260398908 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:36:14.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:14.801+0000 7f7a75d9b640 1 --2- 172.21.15.38:0/3260398908 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7a78157f70 0x7f7a780cc1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:14.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:14.801+0000 7f7a7659c640 1 --2- 172.21.15.38:0/3260398908 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7a60005270 0x7f7a78149aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:14.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:14.801+0000 7f7a76d9d640 1 -- 172.21.15.38:0/3260398908 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7a60005270 msgr2=0x7f7a78149aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:14.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:14.801+0000 7f7a76d9d640 1 --2- 172.21.15.38:0/3260398908 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7a60005270 0x7f7a78149aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:14.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:14.801+0000 7f7a76d9d640 1 -- 172.21.15.38:0/3260398908 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7a78157f70 msgr2=0x7f7a780cc1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:14.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:14.801+0000 7f7a76d9d640 1 --2- 172.21.15.38:0/3260398908 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7a78157f70 0x7f7a780cc1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:14.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:14.801+0000 7f7a76d9d640 1 -- 172.21.15.38:0/3260398908 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a7815f9b0 con 0x7f7a7815f060 2024-09-21T03:36:14.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:14.801+0000 7f7a75d9b640 1 --2- 172.21.15.38:0/3260398908 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7a78157f70 0x7f7a780cc1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:36:14.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:14.801+0000 7f7a76d9d640 1 --2- 172.21.15.38:0/3260398908 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7a7815f060 0x7f7a7815f470 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f7a70001eb0 tx=0x7f7a700019e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:14.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:14.801+0000 7f7a66ffd640 1 -- 172.21.15.38:0/3260398908 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a70003d80 con 0x7f7a7815f060 2024-09-21T03:36:14.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:14.801+0000 7f7a66ffd640 1 -- 172.21.15.38:0/3260398908 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7a700060d0 con 0x7f7a7815f060 2024-09-21T03:36:14.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:14.801+0000 7f7a7c88b640 1 -- 172.21.15.38:0/3260398908 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a78155700 con 0x7f7a7815f060 2024-09-21T03:36:14.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:14.801+0000 7f7a66ffd640 1 -- 172.21.15.38:0/3260398908 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a700326d0 con 0x7f7a7815f060 2024-09-21T03:36:14.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:14.802+0000 7f7a7c88b640 1 -- 172.21.15.38:0/3260398908 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a781587f0 con 0x7f7a7815f060 2024-09-21T03:36:14.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:14.802+0000 7f7a7c88b640 1 -- 172.21.15.38:0/3260398908 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a7806e860 con 0x7f7a7815f060 2024-09-21T03:36:14.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:14.803+0000 7f7a66ffd640 1 -- 172.21.15.38:0/3260398908 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f7a70032870 con 0x7f7a7815f060 2024-09-21T03:36:14.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:14.804+0000 7f7a66ffd640 1 --2- 172.21.15.38:0/3260398908 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7a58078b90 0x7f7a5807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:14.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:14.804+0000 7f7a66ffd640 1 -- 172.21.15.38:0/3260398908 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5421+0+0 (secure 0 0 0) 0x7f7a700baea0 con 0x7f7a7815f060 2024-09-21T03:36:14.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:14.804+0000 7f7a7659c640 1 --2- 172.21.15.38:0/3260398908 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7a58078b90 0x7f7a5807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:14.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:14.804+0000 7f7a7659c640 1 --2- 172.21.15.38:0/3260398908 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7a58078b90 0x7f7a5807b0a0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f7a68006b50 tx=0x7f7a68006720 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:14.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:14.808+0000 7f7a66ffd640 1 -- 172.21.15.38:0/3260398908 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f7a7815f9b0 con 0x7f7a7815f060 2024-09-21T03:36:15.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:15.070+0000 7f7a7c88b640 1 -- 172.21.15.38:0/3260398908 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_damage.TestDamage.test_dentry_first_precommit"]} v 0) -- 0x7f7a780696c0 con 0x7f7a7815f060 2024-09-21T03:36:16.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:16.009+0000 7f7a66ffd640 1 -- 172.21.15.38:0/3260398908 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_damage.TestDamage.test_dentry_first_precommit"]}]=0 v155) ==== 145+0+0 (secure 0 0 0) 0x7f7a700833f0 con 0x7f7a7815f060 2024-09-21T03:36:16.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:16.012+0000 7f7a7c88b640 1 -- 172.21.15.38:0/3260398908 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7a58078b90 msgr2=0x7f7a5807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:16.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:16.012+0000 7f7a7c88b640 1 --2- 172.21.15.38:0/3260398908 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7a58078b90 0x7f7a5807b0a0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f7a68006b50 tx=0x7f7a68006720 comp rx=0 tx=0).stop 2024-09-21T03:36:16.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:16.012+0000 7f7a7c88b640 1 -- 172.21.15.38:0/3260398908 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7a7815f060 msgr2=0x7f7a7815f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:16.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:16.012+0000 7f7a7c88b640 1 --2- 172.21.15.38:0/3260398908 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7a7815f060 0x7f7a7815f470 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f7a70001eb0 tx=0x7f7a700019e0 comp rx=0 tx=0).stop 2024-09-21T03:36:16.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:16.012+0000 7f7a7c88b640 1 -- 172.21.15.38:0/3260398908 shutdown_connections 2024-09-21T03:36:16.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:16.012+0000 7f7a7c88b640 1 --2- 172.21.15.38:0/3260398908 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7a7815f060 0x7f7a7815f470 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:16.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:16.012+0000 7f7a7c88b640 1 --2- 172.21.15.38:0/3260398908 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7a78157f70 0x7f7a780cc1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:16.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:16.012+0000 7f7a7c88b640 1 --2- 172.21.15.38:0/3260398908 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7a58078b90 0x7f7a5807b0a0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:16.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:16.012+0000 7f7a7c88b640 1 --2- 172.21.15.38:0/3260398908 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7a60005270 0x7f7a78149aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:16.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:16.012+0000 7f7a7c88b640 1 -- 172.21.15.38:0/3260398908 >> 172.21.15.38:0/3260398908 conn(0x7f7a780b2cc0 msgr2=0x7f7a780b5a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:16.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:16.012+0000 7f7a7c88b640 1 -- 172.21.15.38:0/3260398908 shutdown_connections 2024-09-21T03:36:16.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:16.012+0000 7f7a7c88b640 1 -- 172.21.15.38:0/3260398908 wait complete. 2024-09-21T03:36:16.028 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-21T03:36:16.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:16.149+0000 7f277e98b640 1 -- 172.21.15.38:0/4018856155 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f27781627b0 msgr2=0x7f2778162bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:16.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:16.149+0000 7f277e98b640 1 --2- 172.21.15.38:0/4018856155 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f27781627b0 0x7f2778162bc0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f2760009830 tx=0x7f276002a510 comp rx=0 tx=0).stop 2024-09-21T03:36:16.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:16.150+0000 7f277e98b640 1 -- 172.21.15.38:0/4018856155 shutdown_connections 2024-09-21T03:36:16.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:16.150+0000 7f277e98b640 1 --2- 172.21.15.38:0/4018856155 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f27780b43a0 0x7f27780b47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:16.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:16.150+0000 7f277e98b640 1 --2- 172.21.15.38:0/4018856155 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f27781627b0 0x7f2778162bc0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:16.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:16.150+0000 7f277e98b640 1 -- 172.21.15.38:0/4018856155 >> 172.21.15.38:0/4018856155 conn(0x7f27780d4600 msgr2=0x7f27780d4a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:16.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:16.150+0000 7f277e98b640 1 -- 172.21.15.38:0/4018856155 shutdown_connections 2024-09-21T03:36:16.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:16.150+0000 7f277e98b640 1 -- 172.21.15.38:0/4018856155 wait complete. 2024-09-21T03:36:16.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:16.151+0000 7f277e98b640 1 Processor -- start 2024-09-21T03:36:16.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:16.152+0000 7f277e98b640 1 -- start start 2024-09-21T03:36:16.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:16.153+0000 7f277e98b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f27780b43a0 0x7f2778161540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:16.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:16.153+0000 7f277e98b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f277815ecf0 0x7f27780c99f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:16.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:16.153+0000 7f277e98b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f27781627b0 0x7f27780c9f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:16.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:16.153+0000 7f277e98b640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f27780b5080 con 0x7f277815ecf0 2024-09-21T03:36:16.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:16.153+0000 7f277e98b640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f27780b4f00 con 0x7f27781627b0 2024-09-21T03:36:16.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:16.153+0000 7f277e98b640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f27780b5200 con 0x7f27780b43a0 2024-09-21T03:36:16.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:16.153+0000 7f27777fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f277815ecf0 0x7f27780c99f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:16.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:16.153+0000 7f27777fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f277815ecf0 0x7f27780c99f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42264/0 (socket says 172.21.15.38:42264) 2024-09-21T03:36:16.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:16.153+0000 7f27777fe640 1 -- 172.21.15.38:0/1738226567 learned_addr learned my addr 172.21.15.38:0/1738226567 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:36:16.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:16.153+0000 7f277cf28640 1 --2- 172.21.15.38:0/1738226567 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f27781627b0 0x7f27780c9f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:16.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:16.153+0000 7f2777fff640 1 --2- 172.21.15.38:0/1738226567 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f27780b43a0 0x7f2778161540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:16.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:16.153+0000 7f27777fe640 1 -- 172.21.15.38:0/1738226567 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f27780b43a0 msgr2=0x7f2778161540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:16.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:16.153+0000 7f27777fe640 1 --2- 172.21.15.38:0/1738226567 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f27780b43a0 0x7f2778161540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:16.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:16.153+0000 7f27777fe640 1 -- 172.21.15.38:0/1738226567 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f27781627b0 msgr2=0x7f27780c9f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:16.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:16.153+0000 7f27777fe640 1 --2- 172.21.15.38:0/1738226567 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f27781627b0 0x7f27780c9f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:16.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:16.153+0000 7f27777fe640 1 -- 172.21.15.38:0/1738226567 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27781497a0 con 0x7f277815ecf0 2024-09-21T03:36:16.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:16.153+0000 7f277cf28640 1 --2- 172.21.15.38:0/1738226567 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f27781627b0 0x7f27780c9f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:36:16.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:16.154+0000 7f27777fe640 1 --2- 172.21.15.38:0/1738226567 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f277815ecf0 0x7f27780c99f0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f276002ab00 tx=0x7f2760033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:16.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:16.154+0000 7f2774ff9640 1 -- 172.21.15.38:0/1738226567 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2760031d60 con 0x7f277815ecf0 2024-09-21T03:36:16.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:16.154+0000 7f2774ff9640 1 -- 172.21.15.38:0/1738226567 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2760033e10 con 0x7f277815ecf0 2024-09-21T03:36:16.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:16.154+0000 7f277e98b640 1 -- 172.21.15.38:0/1738226567 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27781f9d60 con 0x7f277815ecf0 2024-09-21T03:36:16.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:16.154+0000 7f2774ff9640 1 -- 172.21.15.38:0/1738226567 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2760005260 con 0x7f277815ecf0 2024-09-21T03:36:16.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:16.154+0000 7f277e98b640 1 -- 172.21.15.38:0/1738226567 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27780c91f0 con 0x7f277815ecf0 2024-09-21T03:36:16.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:16.155+0000 7f277e98b640 1 -- 172.21.15.38:0/1738226567 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27780b47b0 con 0x7f277815ecf0 2024-09-21T03:36:16.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:16.156+0000 7f2774ff9640 1 -- 172.21.15.38:0/1738226567 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f2760005400 con 0x7f277815ecf0 2024-09-21T03:36:16.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:16.156+0000 7f2774ff9640 1 --2- 172.21.15.38:0/1738226567 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2758078b90 0x7f275807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:16.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:16.157+0000 7f2774ff9640 1 -- 172.21.15.38:0/1738226567 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5421+0+0 (secure 0 0 0) 0x7f27600bbfb0 con 0x7f277815ecf0 2024-09-21T03:36:16.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:16.157+0000 7f2777fff640 1 --2- 172.21.15.38:0/1738226567 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2758078b90 0x7f275807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:16.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:16.157+0000 7f2777fff640 1 --2- 172.21.15.38:0/1738226567 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2758078b90 0x7f275807b0a0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f2768007fd0 tx=0x7f2768009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:16.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:16.161+0000 7f2774ff9640 1 -- 172.21.15.38:0/1738226567 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f276003f030 con 0x7f277815ecf0 2024-09-21T03:36:16.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:16.349+0000 7f277e98b640 1 -- 172.21.15.38:0/1738226567 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f27780b35d0 con 0x7f277815ecf0 2024-09-21T03:36:16.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:16.349+0000 7f2774ff9640 1 -- 172.21.15.38:0/1738226567 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v38) ==== 59+0+2294 (secure 0 0 0) 0x7f2760084500 con 0x7f277815ecf0 2024-09-21T03:36:16.352 INFO:teuthology.orchestra.run.smithi038.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-21T03:36:16.352 INFO:teuthology.orchestra.run.smithi038.stdout:mon advanced mon_op_complaint_time 120 2024-09-21T03:36:16.352 INFO:teuthology.orchestra.run.smithi038.stdout:mgr advanced client_mount_timeout 30 2024-09-21T03:36:16.352 INFO:teuthology.orchestra.run.smithi038.stdout:mgr advanced debug_client 20/20 2024-09-21T03:36:16.352 INFO:teuthology.orchestra.run.smithi038.stdout:mgr advanced debug_mgr 20/20 2024-09-21T03:36:16.352 INFO:teuthology.orchestra.run.smithi038.stdout:mgr advanced debug_ms 1/1 2024-09-21T03:36:16.353 INFO:teuthology.orchestra.run.smithi038.stdout:mgr dev mon_warn_on_pool_no_app false 2024-09-21T03:36:16.353 INFO:teuthology.orchestra.run.smithi038.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-21T03:36:16.353 INFO:teuthology.orchestra.run.smithi038.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 3844.962174 2024-09-21T03:36:16.353 INFO:teuthology.orchestra.run.smithi038.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 4916.015278 2024-09-21T03:36:16.353 INFO:teuthology.orchestra.run.smithi038.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 4165.407418 2024-09-21T03:36:16.353 INFO:teuthology.orchestra.run.smithi038.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 3760.481604 2024-09-21T03:36:16.354 INFO:teuthology.orchestra.run.smithi038.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 6052.882314 2024-09-21T03:36:16.354 INFO:teuthology.orchestra.run.smithi038.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 8967.576248 2024-09-21T03:36:16.354 INFO:teuthology.orchestra.run.smithi038.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 6129.316416 2024-09-21T03:36:16.354 INFO:teuthology.orchestra.run.smithi038.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 8751.950197 2024-09-21T03:36:16.354 INFO:teuthology.orchestra.run.smithi038.stdout:mds advanced debug_mds 20/20 2024-09-21T03:36:16.354 INFO:teuthology.orchestra.run.smithi038.stdout:mds advanced debug_mds_balancer 20/20 2024-09-21T03:36:16.355 INFO:teuthology.orchestra.run.smithi038.stdout:mds advanced debug_ms 2/2 2024-09-21T03:36:16.355 INFO:teuthology.orchestra.run.smithi038.stdout:mds dev mds_debug_frag true 2024-09-21T03:36:16.355 INFO:teuthology.orchestra.run.smithi038.stdout:mds dev mds_debug_scatterstat true 2024-09-21T03:36:16.355 INFO:teuthology.orchestra.run.smithi038.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-21T03:36:16.355 INFO:teuthology.orchestra.run.smithi038.stdout:mds dev mds_verify_scatter true 2024-09-21T03:36:16.355 INFO:teuthology.orchestra.run.smithi038.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-21T03:36:16.356 INFO:teuthology.orchestra.run.smithi038.stdout:mds advanced rados_mon_op_timeout 900 2024-09-21T03:36:16.356 INFO:teuthology.orchestra.run.smithi038.stdout:mds advanced rados_osd_op_timeout 900 2024-09-21T03:36:16.356 INFO:teuthology.orchestra.run.smithi038.stdout:client advanced client_mount_timeout 600 2024-09-21T03:36:16.356 INFO:teuthology.orchestra.run.smithi038.stdout:client advanced debug_client 20/20 2024-09-21T03:36:16.356 INFO:teuthology.orchestra.run.smithi038.stdout:client advanced debug_ms 1/1 2024-09-21T03:36:16.356 INFO:teuthology.orchestra.run.smithi038.stdout:client advanced rados_mon_op_timeout 900 2024-09-21T03:36:16.357 INFO:teuthology.orchestra.run.smithi038.stdout:client advanced rados_osd_op_timeout 900 2024-09-21T03:36:16.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:16.352+0000 7f277e98b640 1 -- 172.21.15.38:0/1738226567 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2758078b90 msgr2=0x7f275807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:16.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:16.352+0000 7f277e98b640 1 --2- 172.21.15.38:0/1738226567 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2758078b90 0x7f275807b0a0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f2768007fd0 tx=0x7f2768009040 comp rx=0 tx=0).stop 2024-09-21T03:36:16.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:16.352+0000 7f277e98b640 1 -- 172.21.15.38:0/1738226567 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f277815ecf0 msgr2=0x7f27780c99f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:16.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:16.352+0000 7f277e98b640 1 --2- 172.21.15.38:0/1738226567 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f277815ecf0 0x7f27780c99f0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f276002ab00 tx=0x7f2760033c20 comp rx=0 tx=0).stop 2024-09-21T03:36:16.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:16.352+0000 7f277e98b640 1 -- 172.21.15.38:0/1738226567 shutdown_connections 2024-09-21T03:36:16.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:16.352+0000 7f277e98b640 1 --2- 172.21.15.38:0/1738226567 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f27781627b0 0x7f27780c9f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:16.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:16.352+0000 7f277e98b640 1 --2- 172.21.15.38:0/1738226567 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f277815ecf0 0x7f27780c99f0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:16.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:16.352+0000 7f277e98b640 1 --2- 172.21.15.38:0/1738226567 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2758078b90 0x7f275807b0a0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:16.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:16.352+0000 7f277e98b640 1 --2- 172.21.15.38:0/1738226567 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f27780b43a0 0x7f2778161540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:16.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:16.352+0000 7f277e98b640 1 -- 172.21.15.38:0/1738226567 >> 172.21.15.38:0/1738226567 conn(0x7f27780d4600 msgr2=0x7f27780d4a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:16.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:16.352+0000 7f277e98b640 1 -- 172.21.15.38:0/1738226567 shutdown_connections 2024-09-21T03:36:16.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:16.352+0000 7f277e98b640 1 -- 172.21.15.38:0/1738226567 wait complete. 2024-09-21T03:36:16.366 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-21T03:36:16.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:16.483+0000 7f2c484cc640 1 -- 172.21.15.38:0/3880149536 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2c400b9ad0 msgr2=0x7f2c400d0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:16.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:16.483+0000 7f2c484cc640 1 --2- 172.21.15.38:0/3880149536 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2c400b9ad0 0x7f2c400d0be0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f2c400cfba0 tx=0x7f2c3402a410 comp rx=0 tx=0).stop 2024-09-21T03:36:16.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:16.483+0000 7f2c484cc640 1 -- 172.21.15.38:0/3880149536 shutdown_connections 2024-09-21T03:36:16.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:16.483+0000 7f2c484cc640 1 --2- 172.21.15.38:0/3880149536 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2c400b9ad0 0x7f2c400d0be0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:16.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:16.483+0000 7f2c484cc640 1 --2- 172.21.15.38:0/3880149536 >> v2:172.21.15.195:3300/0 conn(0x7f2c4015ee80 0x7f2c400d0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:16.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:16.483+0000 7f2c484cc640 1 -- 172.21.15.38:0/3880149536 >> 172.21.15.38:0/3880149536 conn(0x7f2c4009d830 msgr2=0x7f2c400b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:16.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:16.483+0000 7f2c484cc640 1 -- 172.21.15.38:0/3880149536 shutdown_connections 2024-09-21T03:36:16.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:16.483+0000 7f2c484cc640 1 -- 172.21.15.38:0/3880149536 wait complete. 2024-09-21T03:36:16.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:16.485+0000 7f2c484cc640 1 Processor -- start 2024-09-21T03:36:16.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:16.485+0000 7f2c484cc640 1 -- start start 2024-09-21T03:36:16.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:16.486+0000 7f2c484cc640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2c400b9ad0 0x7f2c401f8800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:16.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:16.486+0000 7f2c484cc640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2c400d1120 0x7f2c4006e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:16.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:16.487+0000 7f2c484cc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2c4015ee80 0x7f2c401f5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:16.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:16.487+0000 7f2c484cc640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f2c400b6700 con 0x7f2c4015ee80 2024-09-21T03:36:16.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:16.487+0000 7f2c484cc640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f2c400b6580 con 0x7f2c400d1120 2024-09-21T03:36:16.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:16.487+0000 7f2c484cc640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f2c400b6880 con 0x7f2c400b9ad0 2024-09-21T03:36:16.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:16.487+0000 7f2c46a69640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2c4015ee80 0x7f2c401f5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:16.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:16.487+0000 7f2c46a69640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2c4015ee80 0x7f2c401f5a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42292/0 (socket says 172.21.15.38:42292) 2024-09-21T03:36:16.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:16.487+0000 7f2c46a69640 1 -- 172.21.15.38:0/3538244296 learned_addr learned my addr 172.21.15.38:0/3538244296 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:36:16.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:16.487+0000 7f2c45a67640 1 --2- 172.21.15.38:0/3538244296 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2c400d1120 0x7f2c4006e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:16.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:16.487+0000 7f2c46268640 1 --2- 172.21.15.38:0/3538244296 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2c400b9ad0 0x7f2c401f8800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:16.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:16.487+0000 7f2c46a69640 1 -- 172.21.15.38:0/3538244296 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2c400b9ad0 msgr2=0x7f2c401f8800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:16.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:16.487+0000 7f2c46a69640 1 --2- 172.21.15.38:0/3538244296 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2c400b9ad0 0x7f2c401f8800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:16.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:16.487+0000 7f2c46a69640 1 -- 172.21.15.38:0/3538244296 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2c400d1120 msgr2=0x7f2c4006e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:16.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:16.487+0000 7f2c46a69640 1 --2- 172.21.15.38:0/3538244296 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2c400d1120 0x7f2c4006e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:16.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:16.487+0000 7f2c46a69640 1 -- 172.21.15.38:0/3538244296 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c4006e850 con 0x7f2c4015ee80 2024-09-21T03:36:16.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:16.487+0000 7f2c46a69640 1 --2- 172.21.15.38:0/3538244296 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2c4015ee80 0x7f2c401f5a80 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f2c3800c3c0 tx=0x7f2c3800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:16.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:16.487+0000 7f2c3effd640 1 -- 172.21.15.38:0/3538244296 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c38017070 con 0x7f2c4015ee80 2024-09-21T03:36:16.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:16.487+0000 7f2c3effd640 1 -- 172.21.15.38:0/3538244296 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2c380035f0 con 0x7f2c4015ee80 2024-09-21T03:36:16.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:16.487+0000 7f2c484cc640 1 -- 172.21.15.38:0/3538244296 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c401f9030 con 0x7f2c4015ee80 2024-09-21T03:36:16.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:16.487+0000 7f2c3effd640 1 -- 172.21.15.38:0/3538244296 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c38003a40 con 0x7f2c4015ee80 2024-09-21T03:36:16.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:16.488+0000 7f2c484cc640 1 -- 172.21.15.38:0/3538244296 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c401f95f0 con 0x7f2c4015ee80 2024-09-21T03:36:16.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:16.488+0000 7f2c484cc640 1 -- 172.21.15.38:0/3538244296 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c400b6d30 con 0x7f2c4015ee80 2024-09-21T03:36:16.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:16.489+0000 7f2c3effd640 1 -- 172.21.15.38:0/3538244296 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f2c3801a070 con 0x7f2c4015ee80 2024-09-21T03:36:16.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:16.490+0000 7f2c3effd640 1 --2- 172.21.15.38:0/3538244296 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2c2c078b90 0x7f2c2c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:16.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:16.490+0000 7f2c3effd640 1 -- 172.21.15.38:0/3538244296 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5421+0+0 (secure 0 0 0) 0x7f2c38099cb0 con 0x7f2c4015ee80 2024-09-21T03:36:16.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:16.490+0000 7f2c46268640 1 --2- 172.21.15.38:0/3538244296 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2c2c078b90 0x7f2c2c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:16.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:16.491+0000 7f2c46268640 1 --2- 172.21.15.38:0/3538244296 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2c2c078b90 0x7f2c2c07b0a0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f2c30005fd0 tx=0x7f2c300095e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:16.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:16.494+0000 7f2c3effd640 1 -- 172.21.15.38:0/3538244296 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f2c401592d0 con 0x7f2c4015ee80 2024-09-21T03:36:16.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:16.682+0000 7f2c484cc640 1 -- 172.21.15.38:0/3538244296 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f2c400b9560 con 0x7f2c4015ee80 2024-09-21T03:36:16.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:16.688+0000 7f2c3effd640 1 -- 172.21.15.38:0/3538244296 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v39)=0 v39) ==== 122+0+0 (secure 0 0 0) 0x7f2c38061d10 con 0x7f2c4015ee80 2024-09-21T03:36:16.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:16.690+0000 7f2c484cc640 1 -- 172.21.15.38:0/3538244296 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2c2c078b90 msgr2=0x7f2c2c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:16.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:16.690+0000 7f2c484cc640 1 --2- 172.21.15.38:0/3538244296 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2c2c078b90 0x7f2c2c07b0a0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f2c30005fd0 tx=0x7f2c300095e0 comp rx=0 tx=0).stop 2024-09-21T03:36:16.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:16.690+0000 7f2c484cc640 1 -- 172.21.15.38:0/3538244296 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2c4015ee80 msgr2=0x7f2c401f5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:16.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:16.690+0000 7f2c484cc640 1 --2- 172.21.15.38:0/3538244296 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2c4015ee80 0x7f2c401f5a80 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f2c3800c3c0 tx=0x7f2c3800c880 comp rx=0 tx=0).stop 2024-09-21T03:36:16.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:16.690+0000 7f2c484cc640 1 -- 172.21.15.38:0/3538244296 shutdown_connections 2024-09-21T03:36:16.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:16.690+0000 7f2c484cc640 1 --2- 172.21.15.38:0/3538244296 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2c4015ee80 0x7f2c401f5a80 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:16.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:16.690+0000 7f2c484cc640 1 --2- 172.21.15.38:0/3538244296 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2c400d1120 0x7f2c4006e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:16.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:16.690+0000 7f2c484cc640 1 --2- 172.21.15.38:0/3538244296 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2c2c078b90 0x7f2c2c07b0a0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:16.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:16.690+0000 7f2c484cc640 1 --2- 172.21.15.38:0/3538244296 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2c400b9ad0 0x7f2c401f8800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:16.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:16.690+0000 7f2c484cc640 1 -- 172.21.15.38:0/3538244296 >> 172.21.15.38:0/3538244296 conn(0x7f2c4009d830 msgr2=0x7f2c4015d880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:16.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:16.690+0000 7f2c484cc640 1 -- 172.21.15.38:0/3538244296 shutdown_connections 2024-09-21T03:36:16.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:16.690+0000 7f2c484cc640 1 -- 172.21.15.38:0/3538244296 wait complete. 2024-09-21T03:36:16.703 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-21T03:36:16.703 DEBUG:teuthology.orchestra.run.smithi038:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T03:36:16.755 DEBUG:teuthology.orchestra.run.smithi195:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T03:36:16.800 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-21T03:36:16.801 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-21T03:36:16.801 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:36:16.817 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056795,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2354 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T03:36:16.817 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-09-21T03:36:16.817 DEBUG:teuthology.orchestra.run.smithi195:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:36:16.835 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056014,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497444k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=8978 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248720k,nr_inodes=812180,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497444k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T03:36:16.835 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-21T03:36:16.835 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:36:16.873 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056795,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2354 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T03:36:16.874 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-09-21T03:36:16.874 DEBUG:teuthology.orchestra.run.smithi195:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:36:16.894 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056014,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497444k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=8978 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248720k,nr_inodes=812180,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497444k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T03:36:16.894 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-21T03:36:16.894 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:36:17.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:17.029+0000 7f5af3640640 1 -- 172.21.15.38:0/2614387068 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5aec162f00 msgr2=0x7f5aec0d24d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:17.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:17.029+0000 7f5af3640640 1 --2- 172.21.15.38:0/2614387068 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5aec162f00 0x7f5aec0d24d0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f5ae4005fd0 tx=0x7f5ae402a5d0 comp rx=0 tx=0).stop 2024-09-21T03:36:17.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:17.029+0000 7f5af3640640 1 -- 172.21.15.38:0/2614387068 shutdown_connections 2024-09-21T03:36:17.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:17.029+0000 7f5af3640640 1 --2- 172.21.15.38:0/2614387068 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5aec162280 0x7f5aec162690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:17.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:17.029+0000 7f5af3640640 1 --2- 172.21.15.38:0/2614387068 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5aec162f00 0x7f5aec0d24d0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:17.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:17.029+0000 7f5af3640640 1 -- 172.21.15.38:0/2614387068 >> 172.21.15.38:0/2614387068 conn(0x7f5aec0b6b20 msgr2=0x7f5aec0d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:17.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:17.029+0000 7f5af3640640 1 -- 172.21.15.38:0/2614387068 shutdown_connections 2024-09-21T03:36:17.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:17.029+0000 7f5af3640640 1 -- 172.21.15.38:0/2614387068 wait complete. 2024-09-21T03:36:17.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:17.031+0000 7f5af3640640 1 Processor -- start 2024-09-21T03:36:17.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:17.031+0000 7f5af3640640 1 -- start start 2024-09-21T03:36:17.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:17.032+0000 7f5af3640640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5aec0b2e40 0x7f5aec1f8620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:17.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:17.032+0000 7f5af3640640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5aec162280 0x7f5aec1f8b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:17.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:17.032+0000 7f5af3640640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5aec14cbd0 0x7f5aec14bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:17.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:17.032+0000 7f5af3640640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f5aec15e630 con 0x7f5aec162280 2024-09-21T03:36:17.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:17.032+0000 7f5af3640640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f5aec15e4b0 con 0x7f5aec0b2e40 2024-09-21T03:36:17.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:17.032+0000 7f5af3640640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f5aec15e7b0 con 0x7f5aec14cbd0 2024-09-21T03:36:17.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:17.033+0000 7f5af0bdb640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5aec162280 0x7f5aec1f8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:17.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:17.033+0000 7f5af0bdb640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5aec162280 0x7f5aec1f8b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42316/0 (socket says 172.21.15.38:42316) 2024-09-21T03:36:17.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:17.033+0000 7f5af0bdb640 1 -- 172.21.15.38:0/2749195116 learned_addr learned my addr 172.21.15.38:0/2749195116 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:36:17.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:17.033+0000 7f5af13dc640 1 --2- 172.21.15.38:0/2749195116 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5aec0b2e40 0x7f5aec1f8620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:17.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:17.033+0000 7f5af1bdd640 1 --2- 172.21.15.38:0/2749195116 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5aec14cbd0 0x7f5aec14bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:17.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:17.033+0000 7f5af0bdb640 1 -- 172.21.15.38:0/2749195116 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5aec14cbd0 msgr2=0x7f5aec14bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:17.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:17.033+0000 7f5af0bdb640 1 --2- 172.21.15.38:0/2749195116 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5aec14cbd0 0x7f5aec14bf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:17.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:17.033+0000 7f5af0bdb640 1 -- 172.21.15.38:0/2749195116 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5aec0b2e40 msgr2=0x7f5aec1f8620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:17.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:17.033+0000 7f5af0bdb640 1 --2- 172.21.15.38:0/2749195116 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5aec0b2e40 0x7f5aec1f8620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:17.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:17.033+0000 7f5af0bdb640 1 -- 172.21.15.38:0/2749195116 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5aec0abfb0 con 0x7f5aec162280 2024-09-21T03:36:17.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:17.033+0000 7f5af0bdb640 1 --2- 172.21.15.38:0/2749195116 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5aec162280 0x7f5aec1f8b60 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f5ae000d6d0 tx=0x7f5ae000db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:17.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:17.033+0000 7f5ae9ffb640 1 -- 172.21.15.38:0/2749195116 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ae0017070 con 0x7f5aec162280 2024-09-21T03:36:17.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:17.033+0000 7f5ae9ffb640 1 -- 172.21.15.38:0/2749195116 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5ae0004310 con 0x7f5aec162280 2024-09-21T03:36:17.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:17.033+0000 7f5ae9ffb640 1 -- 172.21.15.38:0/2749195116 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ae0009a80 con 0x7f5aec162280 2024-09-21T03:36:17.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:17.033+0000 7f5af3640640 1 -- 172.21.15.38:0/2749195116 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5aec0ac290 con 0x7f5aec162280 2024-09-21T03:36:17.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:17.033+0000 7f5af3640640 1 -- 172.21.15.38:0/2749195116 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5aec0ac850 con 0x7f5aec162280 2024-09-21T03:36:17.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:17.034+0000 7f5af3640640 1 -- 172.21.15.38:0/2749195116 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5aec06ac50 con 0x7f5aec162280 2024-09-21T03:36:17.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:17.035+0000 7f5ae9ffb640 1 -- 172.21.15.38:0/2749195116 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f5ae00035f0 con 0x7f5aec162280 2024-09-21T03:36:17.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:17.036+0000 7f5ae9ffb640 1 --2- 172.21.15.38:0/2749195116 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5ad8078b90 0x7f5ad807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:17.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:17.036+0000 7f5ae9ffb640 1 -- 172.21.15.38:0/2749195116 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5421+0+0 (secure 0 0 0) 0x7f5ae009ae70 con 0x7f5aec162280 2024-09-21T03:36:17.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:17.036+0000 7f5af13dc640 1 --2- 172.21.15.38:0/2749195116 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5ad8078b90 0x7f5ad807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:17.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:17.036+0000 7f5af13dc640 1 --2- 172.21.15.38:0/2749195116 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5ad8078b90 0x7f5ad807b0a0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f5ae4033000 tx=0x7f5ae4004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:17.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:17.040+0000 7f5ae9ffb640 1 -- 172.21.15.38:0/2749195116 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f5aec06ac50 con 0x7f5aec162280 2024-09-21T03:36:17.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:17.267+0000 7f5af3640640 1 -- 172.21.15.38:0/2749195116 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5aec162f00 con 0x7f5aec162280 2024-09-21T03:36:17.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:17.267+0000 7f5ae9ffb640 1 -- 172.21.15.38:0/2749195116 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) ==== 94+0+3914 (secure 0 0 0) 0x7f5ae0062d80 con 0x7f5aec162280 2024-09-21T03:36:17.269 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:36:17.269 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":38,"btime":"2024-09-21T03:36:10:987122+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5147,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/2858412699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":2858412699},{"type":"v1","addr":"172.21.15.195:6837","nonce":2858412699}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":5368,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/2907144575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":2907144575},{"type":"v1","addr":"172.21.15.195:6836","nonce":2907144575}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":5574,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6834/1911661184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":1911661184},{"type":"v1","addr":"172.21.15.38:6834","nonce":1911661184}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":5904,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/629196862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6833","nonce":629196862},{"type":"v1","addr":"172.21.15.38:6835","nonce":629196862}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38}],"filesystems":[]} 2024-09-21T03:36:17.269 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 38 2024-09-21T03:36:17.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:17.269+0000 7f5af3640640 1 -- 172.21.15.38:0/2749195116 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5ad8078b90 msgr2=0x7f5ad807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:17.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:17.269+0000 7f5af3640640 1 --2- 172.21.15.38:0/2749195116 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5ad8078b90 0x7f5ad807b0a0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f5ae4033000 tx=0x7f5ae4004730 comp rx=0 tx=0).stop 2024-09-21T03:36:17.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:17.269+0000 7f5af3640640 1 -- 172.21.15.38:0/2749195116 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5aec162280 msgr2=0x7f5aec1f8b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:17.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:17.269+0000 7f5af3640640 1 --2- 172.21.15.38:0/2749195116 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5aec162280 0x7f5aec1f8b60 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f5ae000d6d0 tx=0x7f5ae000db90 comp rx=0 tx=0).stop 2024-09-21T03:36:17.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:17.269+0000 7f5af3640640 1 -- 172.21.15.38:0/2749195116 shutdown_connections 2024-09-21T03:36:17.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:17.269+0000 7f5af3640640 1 --2- 172.21.15.38:0/2749195116 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5ad8078b90 0x7f5ad807b0a0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:17.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:17.269+0000 7f5af3640640 1 --2- 172.21.15.38:0/2749195116 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5aec14cbd0 0x7f5aec14bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:17.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:17.269+0000 7f5af3640640 1 --2- 172.21.15.38:0/2749195116 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5aec162280 0x7f5aec1f8b60 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:17.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:17.269+0000 7f5af3640640 1 --2- 172.21.15.38:0/2749195116 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5aec0b2e40 0x7f5aec1f8620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:17.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:17.269+0000 7f5af3640640 1 -- 172.21.15.38:0/2749195116 >> 172.21.15.38:0/2749195116 conn(0x7f5aec0b6b20 msgr2=0x7f5aec0b3be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:17.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:17.269+0000 7f5af3640640 1 -- 172.21.15.38:0/2749195116 shutdown_connections 2024-09-21T03:36:17.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:17.269+0000 7f5af3640640 1 -- 172.21.15.38:0/2749195116 wait complete. 2024-09-21T03:36:17.283 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-21T03:36:17.283 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-21T03:36:17.283 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-21T03:36:17.283 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T03:36:17.283 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-21T03:36:17.284 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-21T03:36:17.284 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-21T03:36:17.284 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T03:36:17.284 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-21T03:36:17.284 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-21T03:36:17.284 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-21T03:36:17.284 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T03:36:17.285 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-21T03:36:17.285 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-21T03:36:17.285 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-21T03:36:17.285 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T03:36:17.293 INFO:tasks.ceph.mds.a:Stopped 2024-09-21T03:36:17.293 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-21T03:36:17.296 INFO:tasks.ceph.mds.b:Stopped 2024-09-21T03:36:17.296 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-21T03:36:17.297 INFO:tasks.ceph.mds.d:Stopped 2024-09-21T03:36:17.297 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-21T03:36:17.298 INFO:tasks.ceph.mds.c:Stopped 2024-09-21T03:36:17.298 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-21T03:36:17.340 INFO:tasks.ceph.mds.b:Started 2024-09-21T03:36:17.340 DEBUG:teuthology.parallel:result is None 2024-09-21T03:36:17.341 INFO:tasks.ceph.mds.a:Started 2024-09-21T03:36:17.341 INFO:tasks.ceph.mds.d:Started 2024-09-21T03:36:17.341 DEBUG:teuthology.parallel:result is None 2024-09-21T03:36:17.342 DEBUG:teuthology.parallel:result is None 2024-09-21T03:36:17.342 INFO:tasks.ceph.mds.c:Started 2024-09-21T03:36:17.343 DEBUG:teuthology.parallel:result is None 2024-09-21T03:36:17.343 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-21T03:36:17.431 INFO:tasks.ceph.mds.a.smithi038.stdout:starting mds.a at 2024-09-21T03:36:17.431 INFO:tasks.ceph.mds.c.smithi038.stdout:starting mds.c at 2024-09-21T03:36:17.433 INFO:tasks.ceph.mds.b.smithi195.stdout:starting mds.b at 2024-09-21T03:36:17.433 INFO:tasks.ceph.mds.d.smithi195.stdout:starting mds.d at 2024-09-21T03:36:17.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:17.463+0000 7f3125dba640 1 -- 172.21.15.38:0/3044015354 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f31200d0d80 msgr2=0x7f31200b5be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:17.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:17.463+0000 7f3125dba640 1 --2- 172.21.15.38:0/3044015354 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f31200d0d80 0x7f31200b5be0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f3118005fd0 tx=0x7f311802a970 comp rx=0 tx=0).stop 2024-09-21T03:36:17.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:17.463+0000 7f3125dba640 1 -- 172.21.15.38:0/3044015354 shutdown_connections 2024-09-21T03:36:17.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:17.463+0000 7f3125dba640 1 --2- 172.21.15.38:0/3044015354 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f31200d0d80 0x7f31200b5be0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:17.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:17.463+0000 7f3125dba640 1 --2- 172.21.15.38:0/3044015354 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f31200ccc40 0x7f312015ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:17.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:17.463+0000 7f3125dba640 1 -- 172.21.15.38:0/3044015354 >> 172.21.15.38:0/3044015354 conn(0x7f31200b46e0 msgr2=0x7f31200d23e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:17.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:17.463+0000 7f3125dba640 1 -- 172.21.15.38:0/3044015354 shutdown_connections 2024-09-21T03:36:17.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:17.463+0000 7f3125dba640 1 -- 172.21.15.38:0/3044015354 wait complete. 2024-09-21T03:36:17.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:17.465+0000 7f3125dba640 1 Processor -- start 2024-09-21T03:36:17.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:17.465+0000 7f3125dba640 1 -- start start 2024-09-21T03:36:17.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:17.467+0000 7f3125dba640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f31200ccc40 0x7f31200a9450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:17.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:17.467+0000 7f3125dba640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f31200cd440 0x7f31201f3d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:17.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:17.467+0000 7f3125dba640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f31200aa220 0x7f31200aa630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:17.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:17.467+0000 7f3125dba640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f31200b6630 con 0x7f31200cd440 2024-09-21T03:36:17.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:17.467+0000 7f3125dba640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f31200b64b0 con 0x7f31200ccc40 2024-09-21T03:36:17.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:17.467+0000 7f3125dba640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f31200b67b0 con 0x7f31200aa220 2024-09-21T03:36:17.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:17.467+0000 7f311effd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f31200cd440 0x7f31201f3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:17.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:17.467+0000 7f311effd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f31200cd440 0x7f31201f3d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42354/0 (socket says 172.21.15.38:42354) 2024-09-21T03:36:17.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:17.467+0000 7f311effd640 1 -- 172.21.15.38:0/2274146876 learned_addr learned my addr 172.21.15.38:0/2274146876 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:36:17.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:17.467+0000 7f311ffff640 1 --2- 172.21.15.38:0/2274146876 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f31200aa220 0x7f31200aa630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:17.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:17.467+0000 7f311f7fe640 1 --2- 172.21.15.38:0/2274146876 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f31200ccc40 0x7f31200a9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:17.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:17.467+0000 7f311effd640 1 -- 172.21.15.38:0/2274146876 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f31200aa220 msgr2=0x7f31200aa630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:17.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:17.467+0000 7f311effd640 1 --2- 172.21.15.38:0/2274146876 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f31200aa220 0x7f31200aa630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:17.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:17.467+0000 7f311effd640 1 -- 172.21.15.38:0/2274146876 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f31200ccc40 msgr2=0x7f31200a9450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:17.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:17.467+0000 7f311effd640 1 --2- 172.21.15.38:0/2274146876 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f31200ccc40 0x7f31200a9450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:17.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:17.467+0000 7f311effd640 1 -- 172.21.15.38:0/2274146876 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31200cb400 con 0x7f31200cd440 2024-09-21T03:36:17.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:17.467+0000 7f311effd640 1 --2- 172.21.15.38:0/2274146876 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f31200cd440 0x7f31201f3d20 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f311400b420 tx=0x7f311400b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:17.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:17.468+0000 7f3107fff640 1 -- 172.21.15.38:0/2274146876 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3114011830 con 0x7f31200cd440 2024-09-21T03:36:17.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:17.468+0000 7f3107fff640 1 -- 172.21.15.38:0/2274146876 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3114016070 con 0x7f31200cd440 2024-09-21T03:36:17.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:17.468+0000 7f3125dba640 1 -- 172.21.15.38:0/2274146876 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31200aab70 con 0x7f31200cd440 2024-09-21T03:36:17.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:17.468+0000 7f3107fff640 1 -- 172.21.15.38:0/2274146876 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3114011d60 con 0x7f31200cd440 2024-09-21T03:36:17.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:17.468+0000 7f3125dba640 1 -- 172.21.15.38:0/2274146876 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31200cbbb0 con 0x7f31200cd440 2024-09-21T03:36:17.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:17.469+0000 7f3125dba640 1 -- 172.21.15.38:0/2274146876 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31200cb640 con 0x7f31200cd440 2024-09-21T03:36:17.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:17.470+0000 7f3107fff640 1 -- 172.21.15.38:0/2274146876 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f3114004020 con 0x7f31200cd440 2024-09-21T03:36:17.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:17.470+0000 7f3107fff640 1 --2- 172.21.15.38:0/2274146876 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f310c078b90 0x7f310c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:17.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:17.470+0000 7f3107fff640 1 -- 172.21.15.38:0/2274146876 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5421+0+0 (secure 0 0 0) 0x7f311409af10 con 0x7f31200cd440 2024-09-21T03:36:17.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:17.470+0000 7f311f7fe640 1 --2- 172.21.15.38:0/2274146876 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f310c078b90 0x7f310c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:17.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:17.471+0000 7f311f7fe640 1 --2- 172.21.15.38:0/2274146876 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f310c078b90 0x7f310c07b0a0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f31100058c0 tx=0x7f3110005f60 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:17.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:17.474+0000 7f3107fff640 1 -- 172.21.15.38:0/2274146876 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f31200aab70 con 0x7f31200cd440 2024-09-21T03:36:17.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:17.672+0000 7f3125dba640 1 -- 172.21.15.38:0/2274146876 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f31200b6cd0 con 0x7f31200cd440 2024-09-21T03:36:18.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:18.024+0000 7f3107fff640 1 -- 172.21.15.38:0/2274146876 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v61) ==== 97+0+0 (secure 0 0 0) 0x7f3114062eb0 con 0x7f31200cd440 2024-09-21T03:36:18.026 INFO:teuthology.orchestra.run.smithi038.stderr: removed all blocklist entries 2024-09-21T03:36:18.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:18.027+0000 7f3125dba640 1 -- 172.21.15.38:0/2274146876 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f310c078b90 msgr2=0x7f310c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:18.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:18.027+0000 7f3125dba640 1 --2- 172.21.15.38:0/2274146876 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f310c078b90 0x7f310c07b0a0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f31100058c0 tx=0x7f3110005f60 comp rx=0 tx=0).stop 2024-09-21T03:36:18.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:18.027+0000 7f3125dba640 1 -- 172.21.15.38:0/2274146876 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f31200cd440 msgr2=0x7f31201f3d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:18.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:18.027+0000 7f3125dba640 1 --2- 172.21.15.38:0/2274146876 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f31200cd440 0x7f31201f3d20 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f311400b420 tx=0x7f311400b8e0 comp rx=0 tx=0).stop 2024-09-21T03:36:18.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:18.027+0000 7f3125dba640 1 -- 172.21.15.38:0/2274146876 shutdown_connections 2024-09-21T03:36:18.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:18.027+0000 7f3125dba640 1 --2- 172.21.15.38:0/2274146876 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f310c078b90 0x7f310c07b0a0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:18.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:18.027+0000 7f3125dba640 1 --2- 172.21.15.38:0/2274146876 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f31200aa220 0x7f31200aa630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:18.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:18.027+0000 7f3125dba640 1 --2- 172.21.15.38:0/2274146876 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f31200cd440 0x7f31201f3d20 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:18.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:18.027+0000 7f3125dba640 1 --2- 172.21.15.38:0/2274146876 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f31200ccc40 0x7f31200a9450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:18.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:18.027+0000 7f3125dba640 1 -- 172.21.15.38:0/2274146876 >> 172.21.15.38:0/2274146876 conn(0x7f31200b46e0 msgr2=0x7f3120070100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:18.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:18.027+0000 7f3125dba640 1 -- 172.21.15.38:0/2274146876 shutdown_connections 2024-09-21T03:36:18.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:18.027+0000 7f3125dba640 1 -- 172.21.15.38:0/2274146876 wait complete. 2024-09-21T03:36:18.042 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-21T03:36:18.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:18.159+0000 7f9b149a8640 1 -- 172.21.15.38:0/2434022506 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9afc005250 msgr2=0x7f9afc00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:18.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:18.159+0000 7f9b149a8640 1 --2- 172.21.15.38:0/2434022506 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9afc005250 0x7f9afc00b9c0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f9b080016c0 tx=0x7f9afc005830 comp rx=0 tx=0).stop 2024-09-21T03:36:18.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:18.159+0000 7f9b149a8640 1 -- 172.21.15.38:0/2434022506 shutdown_connections 2024-09-21T03:36:18.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:18.159+0000 7f9b149a8640 1 --2- 172.21.15.38:0/2434022506 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9afc005250 0x7f9afc00b9c0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:18.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:18.159+0000 7f9b149a8640 1 -- 172.21.15.38:0/2434022506 >> 172.21.15.38:0/2434022506 conn(0x7f9b10072990 msgr2=0x7f9b10072da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:18.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:18.159+0000 7f9b149a8640 1 -- 172.21.15.38:0/2434022506 shutdown_connections 2024-09-21T03:36:18.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:18.159+0000 7f9b149a8640 1 -- 172.21.15.38:0/2434022506 wait complete. 2024-09-21T03:36:18.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:18.161+0000 7f9b149a8640 1 Processor -- start 2024-09-21T03:36:18.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:18.161+0000 7f9b149a8640 1 -- start start 2024-09-21T03:36:18.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:18.162+0000 7f9b149a8640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9afc005250 0x7f9b10157200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:18.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:18.162+0000 7f9b149a8640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9b101f66e0 0x7f9b100b2810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:18.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:18.162+0000 7f9b149a8640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9b100b2d50 0x7f9b101f92a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:18.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:18.162+0000 7f9b149a8640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f9b1015aa80 con 0x7f9b100b2d50 2024-09-21T03:36:18.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:18.162+0000 7f9b149a8640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f9b1015a900 con 0x7f9afc005250 2024-09-21T03:36:18.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:18.162+0000 7f9b149a8640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f9b1015e230 con 0x7f9b101f66e0 2024-09-21T03:36:18.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:18.163+0000 7f9b0ed9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9b100b2d50 0x7f9b101f92a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:18.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:18.163+0000 7f9b0ed9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9b100b2d50 0x7f9b101f92a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42372/0 (socket says 172.21.15.38:42372) 2024-09-21T03:36:18.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:18.163+0000 7f9b0ed9d640 1 -- 172.21.15.38:0/706858249 learned_addr learned my addr 172.21.15.38:0/706858249 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:36:18.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:18.163+0000 7f9b0e59c640 1 --2- 172.21.15.38:0/706858249 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9afc005250 0x7f9b10157200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:18.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:18.163+0000 7f9b0dd9b640 1 --2- 172.21.15.38:0/706858249 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9b101f66e0 0x7f9b100b2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:18.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:18.163+0000 7f9b0ed9d640 1 -- 172.21.15.38:0/706858249 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9b101f66e0 msgr2=0x7f9b100b2810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:18.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:18.163+0000 7f9b0ed9d640 1 --2- 172.21.15.38:0/706858249 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9b101f66e0 0x7f9b100b2810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:18.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:18.163+0000 7f9b0ed9d640 1 -- 172.21.15.38:0/706858249 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9afc005250 msgr2=0x7f9b10157200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:18.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:18.163+0000 7f9b0ed9d640 1 --2- 172.21.15.38:0/706858249 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9afc005250 0x7f9b10157200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:18.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:18.163+0000 7f9b0ed9d640 1 -- 172.21.15.38:0/706858249 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b100b3160 con 0x7f9b100b2d50 2024-09-21T03:36:18.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:18.163+0000 7f9b0e59c640 1 --2- 172.21.15.38:0/706858249 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9afc005250 0x7f9b10157200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:36:18.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:18.163+0000 7f9b0ed9d640 1 --2- 172.21.15.38:0/706858249 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9b100b2d50 0x7f9b101f92a0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f9b0802a930 tx=0x7f9b08003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:18.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:18.163+0000 7f9b06ffd640 1 -- 172.21.15.38:0/706858249 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b08037070 con 0x7f9b100b2d50 2024-09-21T03:36:18.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:18.163+0000 7f9b06ffd640 1 -- 172.21.15.38:0/706858249 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9b08003ba0 con 0x7f9b100b2d50 2024-09-21T03:36:18.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:18.163+0000 7f9b149a8640 1 -- 172.21.15.38:0/706858249 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b100ada70 con 0x7f9b100b2d50 2024-09-21T03:36:18.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:18.163+0000 7f9b06ffd640 1 -- 172.21.15.38:0/706858249 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b08006920 con 0x7f9b100b2d50 2024-09-21T03:36:18.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:18.163+0000 7f9b149a8640 1 -- 172.21.15.38:0/706858249 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b100ae030 con 0x7f9b100b2d50 2024-09-21T03:36:18.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:18.164+0000 7f9b149a8640 1 -- 172.21.15.38:0/706858249 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b1015e790 con 0x7f9b100b2d50 2024-09-21T03:36:18.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:18.165+0000 7f9b06ffd640 1 -- 172.21.15.38:0/706858249 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f9b08030020 con 0x7f9b100b2d50 2024-09-21T03:36:18.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:18.165+0000 7f9b06ffd640 1 --2- 172.21.15.38:0/706858249 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9af0078b90 0x7f9af007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:18.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:18.166+0000 7f9b06ffd640 1 -- 172.21.15.38:0/706858249 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5249+0+0 (secure 0 0 0) 0x7f9b080ba1a0 con 0x7f9b100b2d50 2024-09-21T03:36:18.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:18.166+0000 7f9b0e59c640 1 --2- 172.21.15.38:0/706858249 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9af0078b90 0x7f9af007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:18.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:18.166+0000 7f9b0e59c640 1 --2- 172.21.15.38:0/706858249 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9af0078b90 0x7f9af007b0a0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f9b000013d0 tx=0x7f9b00006b20 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:18.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:18.170+0000 7f9b06ffd640 1 -- 172.21.15.38:0/706858249 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f9b100b3160 con 0x7f9b100b2d50 2024-09-21T03:36:18.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:18.432+0000 7f9b149a8640 1 -- 172.21.15.38:0/706858249 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f9b100b74c0 con 0x7f9b100b2d50 2024-09-21T03:36:18.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:18.432+0000 7f9b06ffd640 1 -- 172.21.15.38:0/706858249 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v19) ==== 80+0+5304 (secure 0 0 0) 0x7f9b08082830 con 0x7f9b100b2d50 2024-09-21T03:36:18.434 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:36:18.434 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-09-21T03:36:18.434 INFO:teuthology.orchestra.run.smithi038.stdout: "auth_dump": [ 2024-09-21T03:36:18.434 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-21T03:36:18.435 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "mds.a", 2024-09-21T03:36:18.435 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQBPPu5mfzGIMRAAmq0TaC3J1VU3GxzhTgJ/Zw==", 2024-09-21T03:36:18.435 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-09-21T03:36:18.435 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow", 2024-09-21T03:36:18.435 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow *", 2024-09-21T03:36:18.435 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow *", 2024-09-21T03:36:18.435 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-09-21T03:36:18.435 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-21T03:36:18.435 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-21T03:36:18.435 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-21T03:36:18.435 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "mds.b", 2024-09-21T03:36:18.435 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQBQPu5m/fzuAhAApyigRlQ8OrV/sf0ap6Fn6A==", 2024-09-21T03:36:18.435 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-09-21T03:36:18.436 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow", 2024-09-21T03:36:18.436 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow *", 2024-09-21T03:36:18.436 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow *", 2024-09-21T03:36:18.436 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-09-21T03:36:18.436 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-21T03:36:18.436 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-21T03:36:18.436 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-21T03:36:18.436 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "mds.c", 2024-09-21T03:36:18.436 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQBPPu5mQipIORAArRn8k8SvvpG8x03F9I2/gQ==", 2024-09-21T03:36:18.436 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-09-21T03:36:18.436 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow", 2024-09-21T03:36:18.436 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow *", 2024-09-21T03:36:18.436 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow *", 2024-09-21T03:36:18.437 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-09-21T03:36:18.437 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-21T03:36:18.437 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-21T03:36:18.437 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-21T03:36:18.437 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "mds.d", 2024-09-21T03:36:18.437 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQBQPu5mzlyDChAAqbPKXS3EtnOqqeHhA1ehIw==", 2024-09-21T03:36:18.437 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-09-21T03:36:18.437 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow", 2024-09-21T03:36:18.437 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow *", 2024-09-21T03:36:18.437 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow *", 2024-09-21T03:36:18.437 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-09-21T03:36:18.437 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-21T03:36:18.437 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-21T03:36:18.438 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-21T03:36:18.438 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "osd.0", 2024-09-21T03:36:18.438 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQBXPu5mMZggAhAAtMPdJipheizZhpby9APj4A==", 2024-09-21T03:36:18.438 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-09-21T03:36:18.438 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow profile osd", 2024-09-21T03:36:18.438 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile osd", 2024-09-21T03:36:18.438 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-09-21T03:36:18.438 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-21T03:36:18.438 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-21T03:36:18.438 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-21T03:36:18.438 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "osd.1", 2024-09-21T03:36:18.438 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQBaPu5mm9TFIhAAWPJCvBYMgMbKr7Z7Diq58A==", 2024-09-21T03:36:18.438 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-09-21T03:36:18.438 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow profile osd", 2024-09-21T03:36:18.439 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile osd", 2024-09-21T03:36:18.439 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-09-21T03:36:18.439 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-21T03:36:18.439 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-21T03:36:18.439 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-21T03:36:18.439 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "osd.2", 2024-09-21T03:36:18.439 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQBePu5mqjljCBAAcdEpzJq6D8e1L+q1s0GjWQ==", 2024-09-21T03:36:18.439 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-09-21T03:36:18.439 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow profile osd", 2024-09-21T03:36:18.439 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile osd", 2024-09-21T03:36:18.439 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-09-21T03:36:18.439 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-21T03:36:18.439 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-21T03:36:18.440 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-21T03:36:18.440 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "osd.3", 2024-09-21T03:36:18.440 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQBhPu5mRjfdKBAAJ6+PlEiRp/xvxny6SjHV8Q==", 2024-09-21T03:36:18.440 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-09-21T03:36:18.440 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow profile osd", 2024-09-21T03:36:18.440 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile osd", 2024-09-21T03:36:18.440 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-09-21T03:36:18.440 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-21T03:36:18.440 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-21T03:36:18.440 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-21T03:36:18.440 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "osd.4", 2024-09-21T03:36:18.440 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQBlPu5m8bNyNxAA2rERruGojEDKmcXN2K6uog==", 2024-09-21T03:36:18.440 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-09-21T03:36:18.440 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow profile osd", 2024-09-21T03:36:18.441 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile osd", 2024-09-21T03:36:18.441 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-09-21T03:36:18.441 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-21T03:36:18.441 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-21T03:36:18.441 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-21T03:36:18.441 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "osd.5", 2024-09-21T03:36:18.441 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQBpPu5mBB2iDBAAvsvhAPMvNWII2Zr9Je45YQ==", 2024-09-21T03:36:18.441 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-09-21T03:36:18.441 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow profile osd", 2024-09-21T03:36:18.441 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile osd", 2024-09-21T03:36:18.441 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-09-21T03:36:18.441 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-21T03:36:18.441 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-21T03:36:18.442 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-21T03:36:18.442 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "osd.6", 2024-09-21T03:36:18.442 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQBsPu5mMQMaLRAAznzYdoNGHWOI2p2mNN/rQw==", 2024-09-21T03:36:18.442 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-09-21T03:36:18.442 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow profile osd", 2024-09-21T03:36:18.442 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile osd", 2024-09-21T03:36:18.442 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-09-21T03:36:18.442 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-21T03:36:18.442 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-21T03:36:18.442 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-21T03:36:18.442 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "osd.7", 2024-09-21T03:36:18.442 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQBwPu5mHu0WERAAir8bmtuNOOj/2D4hLaoYOA==", 2024-09-21T03:36:18.442 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-09-21T03:36:18.442 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow profile osd", 2024-09-21T03:36:18.443 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile osd", 2024-09-21T03:36:18.443 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-09-21T03:36:18.443 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-21T03:36:18.443 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-21T03:36:18.443 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-21T03:36:18.443 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "client.0", 2024-09-21T03:36:18.443 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQBQPu5m6ydBDxAAlWZ2XzQTPBcYPriPGRJVeA==", 2024-09-21T03:36:18.443 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-09-21T03:36:18.443 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow", 2024-09-21T03:36:18.443 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow r", 2024-09-21T03:36:18.443 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-21T03:36:18.443 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-21T03:36:18.443 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-21T03:36:18.444 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-21T03:36:18.444 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "client.1", 2024-09-21T03:36:18.444 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQBQPu5mlWP0FhAAkNwfaJ6rPr3r4ZRVbPxX8g==", 2024-09-21T03:36:18.444 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-09-21T03:36:18.444 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow", 2024-09-21T03:36:18.444 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow r", 2024-09-21T03:36:18.444 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow rw", 2024-09-21T03:36:18.444 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow rwx" 2024-09-21T03:36:18.444 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-21T03:36:18.444 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-21T03:36:18.444 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-21T03:36:18.444 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "client.2", 2024-09-21T03:36:18.444 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQBQPu5m48oaExAAOzYrR/SsykpzH2TkWHs39Q==", 2024-09-21T03:36:18.445 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-09-21T03:36:18.445 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow", 2024-09-21T03:36:18.445 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow r", 2024-09-21T03:36:18.445 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow rw", 2024-09-21T03:36:18.445 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow rwx" 2024-09-21T03:36:18.445 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-21T03:36:18.445 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-21T03:36:18.445 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-21T03:36:18.445 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "client.3", 2024-09-21T03:36:18.445 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQBQPu5mVQSzGhAAAo6vizLc7GOnenQRsgtn1A==", 2024-09-21T03:36:18.445 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-09-21T03:36:18.445 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow", 2024-09-21T03:36:18.445 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow r", 2024-09-21T03:36:18.445 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow rw", 2024-09-21T03:36:18.446 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow rwx" 2024-09-21T03:36:18.446 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-21T03:36:18.446 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-21T03:36:18.446 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-21T03:36:18.446 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "client.admin", 2024-09-21T03:36:18.446 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQBPPu5mPHphExAA7D+/V7qhoNyoc6h0GnNMlQ==", 2024-09-21T03:36:18.446 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-09-21T03:36:18.446 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow *", 2024-09-21T03:36:18.446 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow *", 2024-09-21T03:36:18.446 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow *", 2024-09-21T03:36:18.446 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-09-21T03:36:18.446 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-21T03:36:18.446 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-21T03:36:18.446 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-21T03:36:18.447 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "mgr.x", 2024-09-21T03:36:18.447 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQBPPu5m6K2oLRAAzldoBXAMSnWhxVFwPZkHOw==", 2024-09-21T03:36:18.447 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-09-21T03:36:18.447 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow *", 2024-09-21T03:36:18.447 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile mgr", 2024-09-21T03:36:18.447 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-09-21T03:36:18.447 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-21T03:36:18.447 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-21T03:36:18.447 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-21T03:36:18.447 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "mgr.y", 2024-09-21T03:36:18.447 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQBPPu5mWEz+KBAASx2KzDril3lD9cEawy56jA==", 2024-09-21T03:36:18.447 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-09-21T03:36:18.447 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow *", 2024-09-21T03:36:18.447 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile mgr", 2024-09-21T03:36:18.448 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-09-21T03:36:18.448 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-21T03:36:18.448 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-21T03:36:18.448 INFO:teuthology.orchestra.run.smithi038.stdout: ] 2024-09-21T03:36:18.448 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-09-21T03:36:18.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:18.435+0000 7f9b149a8640 1 -- 172.21.15.38:0/706858249 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9af0078b90 msgr2=0x7f9af007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:18.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:18.435+0000 7f9b149a8640 1 --2- 172.21.15.38:0/706858249 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9af0078b90 0x7f9af007b0a0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f9b000013d0 tx=0x7f9b00006b20 comp rx=0 tx=0).stop 2024-09-21T03:36:18.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:18.435+0000 7f9b149a8640 1 -- 172.21.15.38:0/706858249 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9b100b2d50 msgr2=0x7f9b101f92a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:18.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:18.435+0000 7f9b149a8640 1 --2- 172.21.15.38:0/706858249 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9b100b2d50 0x7f9b101f92a0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f9b0802a930 tx=0x7f9b08003820 comp rx=0 tx=0).stop 2024-09-21T03:36:18.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:18.435+0000 7f9b149a8640 1 -- 172.21.15.38:0/706858249 shutdown_connections 2024-09-21T03:36:18.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:18.435+0000 7f9b149a8640 1 --2- 172.21.15.38:0/706858249 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9b100b2d50 0x7f9b101f92a0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:18.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:18.435+0000 7f9b149a8640 1 --2- 172.21.15.38:0/706858249 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9af0078b90 0x7f9af007b0a0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:18.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:18.435+0000 7f9b149a8640 1 --2- 172.21.15.38:0/706858249 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9b101f66e0 0x7f9b100b2810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:18.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:18.435+0000 7f9b149a8640 1 --2- 172.21.15.38:0/706858249 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9afc005250 0x7f9b10157200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:18.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:18.435+0000 7f9b149a8640 1 -- 172.21.15.38:0/706858249 >> 172.21.15.38:0/706858249 conn(0x7f9b10072990 msgr2=0x7f9b1015ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:18.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:18.435+0000 7f9b149a8640 1 -- 172.21.15.38:0/706858249 shutdown_connections 2024-09-21T03:36:18.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:18.435+0000 7f9b149a8640 1 -- 172.21.15.38:0/706858249 wait complete. 2024-09-21T03:36:18.451 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-21T03:36:18.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:18.565+0000 7f2d60625640 1 -- 172.21.15.38:0/3883076376 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2d44003a00 msgr2=0x7f2d44003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:18.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:18.565+0000 7f2d60625640 1 --2- 172.21.15.38:0/3883076376 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2d44003a00 0x7f2d44003e70 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f2d5400c4a0 tx=0x7f2d44005830 comp rx=0 tx=0).stop 2024-09-21T03:36:18.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:18.565+0000 7f2d5ebc2640 1 -- 172.21.15.38:0/3883076376 reap_dead start 2024-09-21T03:36:18.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:18.565+0000 7f2d60625640 1 -- 172.21.15.38:0/3883076376 shutdown_connections 2024-09-21T03:36:18.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:18.565+0000 7f2d60625640 1 -- 172.21.15.38:0/3883076376 >> 172.21.15.38:0/3883076376 conn(0x7f2d580b6b20 msgr2=0x7f2d580d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:18.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:18.565+0000 7f2d60625640 1 -- 172.21.15.38:0/3883076376 shutdown_connections 2024-09-21T03:36:18.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:18.565+0000 7f2d60625640 1 -- 172.21.15.38:0/3883076376 wait complete. 2024-09-21T03:36:18.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:18.567+0000 7f2d60625640 1 Processor -- start 2024-09-21T03:36:18.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:18.567+0000 7f2d60625640 1 -- start start 2024-09-21T03:36:18.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:18.568+0000 7f2d60625640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2d44003a00 0x7f2d580abd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:18.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:18.568+0000 7f2d60625640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2d580cb050 0x7f2d581f84a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:18.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:18.568+0000 7f2d60625640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2d580cb460 0x7f2d581f89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:18.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:18.568+0000 7f2d60625640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f2d58162690 con 0x7f2d580cb460 2024-09-21T03:36:18.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:18.568+0000 7f2d60625640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f2d580b39a0 con 0x7f2d44003a00 2024-09-21T03:36:18.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:18.568+0000 7f2d60625640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f2d58162810 con 0x7f2d580cb050 2024-09-21T03:36:18.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:18.568+0000 7f2d5ebc2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2d580cb460 0x7f2d581f89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:18.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:18.568+0000 7f2d5ebc2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2d580cb460 0x7f2d581f89e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42392/0 (socket says 172.21.15.38:42392) 2024-09-21T03:36:18.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:18.568+0000 7f2d5ebc2640 1 -- 172.21.15.38:0/1897824606 learned_addr learned my addr 172.21.15.38:0/1897824606 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:36:18.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:18.568+0000 7f2d5e3c1640 1 --2- 172.21.15.38:0/1897824606 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2d44003a00 0x7f2d580abd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:18.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:18.568+0000 7f2d5dbc0640 1 --2- 172.21.15.38:0/1897824606 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2d580cb050 0x7f2d581f84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:18.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:18.569+0000 7f2d5ebc2640 1 -- 172.21.15.38:0/1897824606 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2d580cb050 msgr2=0x7f2d581f84a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:18.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:18.569+0000 7f2d5ebc2640 1 --2- 172.21.15.38:0/1897824606 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2d580cb050 0x7f2d581f84a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:18.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:18.569+0000 7f2d5ebc2640 1 -- 172.21.15.38:0/1897824606 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2d44003a00 msgr2=0x7f2d580abd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:18.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:18.569+0000 7f2d5ebc2640 1 --2- 172.21.15.38:0/1897824606 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2d44003a00 0x7f2d580abd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:18.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:18.569+0000 7f2d5ebc2640 1 -- 172.21.15.38:0/1897824606 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d580cbbc0 con 0x7f2d580cb460 2024-09-21T03:36:18.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:18.569+0000 7f2d5e3c1640 1 --2- 172.21.15.38:0/1897824606 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2d44003a00 0x7f2d580abd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:36:18.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:18.569+0000 7f2d5ebc2640 1 --2- 172.21.15.38:0/1897824606 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2d580cb460 0x7f2d581f89e0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f2d580d00c0 tx=0x7f2d50010e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:18.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:18.569+0000 7f2d4effd640 1 -- 172.21.15.38:0/1897824606 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d5000edf0 con 0x7f2d580cb460 2024-09-21T03:36:18.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:18.569+0000 7f2d4effd640 1 -- 172.21.15.38:0/1897824606 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2d50015070 con 0x7f2d580cb460 2024-09-21T03:36:18.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:18.569+0000 7f2d60625640 1 -- 172.21.15.38:0/1897824606 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d580cbea0 con 0x7f2d580cb460 2024-09-21T03:36:18.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:18.569+0000 7f2d4effd640 1 -- 172.21.15.38:0/1897824606 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d50003d00 con 0x7f2d580cb460 2024-09-21T03:36:18.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:18.569+0000 7f2d60625640 1 -- 172.21.15.38:0/1897824606 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d580cc460 con 0x7f2d580cb460 2024-09-21T03:36:18.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:18.570+0000 7f2d60625640 1 -- 172.21.15.38:0/1897824606 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d580cb930 con 0x7f2d580cb460 2024-09-21T03:36:18.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:18.571+0000 7f2d4effd640 1 -- 172.21.15.38:0/1897824606 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f2d50004030 con 0x7f2d580cb460 2024-09-21T03:36:18.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:18.571+0000 7f2d4effd640 1 --2- 172.21.15.38:0/1897824606 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2d3c078b90 0x7f2d3c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:18.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:18.571+0000 7f2d4effd640 1 -- 172.21.15.38:0/1897824606 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5249+0+0 (secure 0 0 0) 0x7f2d5009b1c0 con 0x7f2d580cb460 2024-09-21T03:36:18.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:18.572+0000 7f2d5e3c1640 1 --2- 172.21.15.38:0/1897824606 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2d3c078b90 0x7f2d3c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:18.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:18.572+0000 7f2d5e3c1640 1 --2- 172.21.15.38:0/1897824606 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2d3c078b90 0x7f2d3c07b0a0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f2d54033000 tx=0x7f2d54038040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:18.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:18.576+0000 7f2d4effd640 1 -- 172.21.15.38:0/1897824606 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f2d500637d0 con 0x7f2d580cb460 2024-09-21T03:36:18.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:18.768+0000 7f2d60625640 1 -- 172.21.15.38:0/1897824606 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f2d58162f00 con 0x7f2d580cb460 2024-09-21T03:36:18.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:18.768+0000 7f2d4effd640 1 -- 172.21.15.38:0/1897824606 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v63) ==== 84+0+65 (secure 0 0 0) 0x7f2d500630d0 con 0x7f2d580cb460 2024-09-21T03:36:18.771 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:36:18.771 INFO:teuthology.orchestra.run.smithi038.stdout:[ 2024-09-21T03:36:18.771 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-21T03:36:18.771 INFO:teuthology.orchestra.run.smithi038.stdout: "poolnum": 1, 2024-09-21T03:36:18.771 INFO:teuthology.orchestra.run.smithi038.stdout: "poolname": ".mgr" 2024-09-21T03:36:18.771 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-21T03:36:18.771 INFO:teuthology.orchestra.run.smithi038.stdout:] 2024-09-21T03:36:18.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:18.771+0000 7f2d60625640 1 -- 172.21.15.38:0/1897824606 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2d3c078b90 msgr2=0x7f2d3c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:18.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:18.771+0000 7f2d60625640 1 --2- 172.21.15.38:0/1897824606 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2d3c078b90 0x7f2d3c07b0a0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f2d54033000 tx=0x7f2d54038040 comp rx=0 tx=0).stop 2024-09-21T03:36:18.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:18.771+0000 7f2d60625640 1 -- 172.21.15.38:0/1897824606 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2d580cb460 msgr2=0x7f2d581f89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:18.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:18.771+0000 7f2d60625640 1 --2- 172.21.15.38:0/1897824606 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2d580cb460 0x7f2d581f89e0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f2d580d00c0 tx=0x7f2d50010e80 comp rx=0 tx=0).stop 2024-09-21T03:36:18.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:18.771+0000 7f2d60625640 1 -- 172.21.15.38:0/1897824606 shutdown_connections 2024-09-21T03:36:18.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:18.771+0000 7f2d60625640 1 --2- 172.21.15.38:0/1897824606 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2d580cb460 0x7f2d581f89e0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:18.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:18.771+0000 7f2d60625640 1 --2- 172.21.15.38:0/1897824606 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2d3c078b90 0x7f2d3c07b0a0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:18.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:18.771+0000 7f2d60625640 1 --2- 172.21.15.38:0/1897824606 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2d580cb050 0x7f2d581f84a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:18.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:18.771+0000 7f2d60625640 1 --2- 172.21.15.38:0/1897824606 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2d44003a00 0x7f2d580abd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:18.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:18.771+0000 7f2d60625640 1 -- 172.21.15.38:0/1897824606 >> 172.21.15.38:0/1897824606 conn(0x7f2d580b6b20 msgr2=0x7f2d580b8520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:18.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:18.771+0000 7f2d60625640 1 -- 172.21.15.38:0/1897824606 shutdown_connections 2024-09-21T03:36:18.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:18.771+0000 7f2d60625640 1 -- 172.21.15.38:0/1897824606 wait complete. 2024-09-21T03:36:18.784 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-21T03:36:18.784 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-21T03:36:18.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:18.903+0000 7f144ef5f640 1 -- 172.21.15.38:0/409645564 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f14480cd020 msgr2=0x7f14480b1ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:18.906 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:18.903+0000 7f144ef5f640 1 --2- 172.21.15.38:0/409645564 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f14480cd020 0x7f14480b1ca0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f1430009830 tx=0x7f143002a500 comp rx=0 tx=0).stop 2024-09-21T03:36:18.906 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:18.903+0000 7f144ef5f640 1 -- 172.21.15.38:0/409645564 shutdown_connections 2024-09-21T03:36:18.906 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:18.903+0000 7f144ef5f640 1 --2- 172.21.15.38:0/409645564 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f14480cd020 0x7f14480b1ca0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:18.906 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:18.903+0000 7f144ef5f640 1 --2- 172.21.15.38:0/409645564 >> v2:172.21.15.195:3300/0 conn(0x7f1448160bb0 0x7f14480ccae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:18.906 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:18.903+0000 7f144ef5f640 1 -- 172.21.15.38:0/409645564 >> 172.21.15.38:0/409645564 conn(0x7f14480d2280 msgr2=0x7f14480d2690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:18.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:18.903+0000 7f144ef5f640 1 -- 172.21.15.38:0/409645564 shutdown_connections 2024-09-21T03:36:18.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:18.903+0000 7f144ef5f640 1 -- 172.21.15.38:0/409645564 wait complete. 2024-09-21T03:36:18.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:18.905+0000 7f144ef5f640 1 Processor -- start 2024-09-21T03:36:18.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:18.905+0000 7f144ef5f640 1 -- start start 2024-09-21T03:36:18.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:18.906+0000 7f144ef5f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f14480b21e0 0x7f14481f7610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:18.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:18.906+0000 7f144ef5f640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f14480cd020 0x7f14481f3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:18.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:18.906+0000 7f144ef5f640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1448160bb0 0x7f1448148660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:18.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:18.906+0000 7f144ef5f640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f14480b2e80 con 0x7f14480b21e0 2024-09-21T03:36:18.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:18.906+0000 7f144ef5f640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f14480b2d00 con 0x7f1448160bb0 2024-09-21T03:36:18.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:18.906+0000 7f144ef5f640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f144815d510 con 0x7f14480cd020 2024-09-21T03:36:18.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:18.907+0000 7f144ccfb640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f14480b21e0 0x7f14481f7610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:18.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:18.907+0000 7f144ccfb640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f14480b21e0 0x7f14481f7610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42400/0 (socket says 172.21.15.38:42400) 2024-09-21T03:36:18.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:18.907+0000 7f144ccfb640 1 -- 172.21.15.38:0/1848501238 learned_addr learned my addr 172.21.15.38:0/1848501238 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:36:18.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:18.907+0000 7f1447fff640 1 --2- 172.21.15.38:0/1848501238 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f14480cd020 0x7f14481f3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:18.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:18.907+0000 7f144d4fc640 1 --2- 172.21.15.38:0/1848501238 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1448160bb0 0x7f1448148660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:18.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:18.907+0000 7f144ccfb640 1 -- 172.21.15.38:0/1848501238 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f14480cd020 msgr2=0x7f14481f3fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:18.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:18.907+0000 7f144ccfb640 1 --2- 172.21.15.38:0/1848501238 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f14480cd020 0x7f14481f3fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:18.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:18.907+0000 7f144ccfb640 1 -- 172.21.15.38:0/1848501238 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1448160bb0 msgr2=0x7f1448148660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:18.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:18.907+0000 7f144ccfb640 1 --2- 172.21.15.38:0/1848501238 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1448160bb0 0x7f1448148660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:18.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:18.907+0000 7f144ccfb640 1 -- 172.21.15.38:0/1848501238 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14481f44f0 con 0x7f14480b21e0 2024-09-21T03:36:18.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:18.907+0000 7f144d4fc640 1 --2- 172.21.15.38:0/1848501238 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1448160bb0 0x7f1448148660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:36:18.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:18.907+0000 7f1447fff640 1 --2- 172.21.15.38:0/1848501238 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f14480cd020 0x7f14481f3fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:36:18.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:18.907+0000 7f144ccfb640 1 --2- 172.21.15.38:0/1848501238 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f14480b21e0 0x7f14481f7610 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f143800e9f0 tx=0x7f143800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:18.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:18.907+0000 7f14457fa640 1 -- 172.21.15.38:0/1848501238 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f143800cd80 con 0x7f14480b21e0 2024-09-21T03:36:18.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:18.907+0000 7f14457fa640 1 -- 172.21.15.38:0/1848501238 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1438004020 con 0x7f14480b21e0 2024-09-21T03:36:18.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:18.907+0000 7f144ef5f640 1 -- 172.21.15.38:0/1848501238 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14481563b0 con 0x7f14480b21e0 2024-09-21T03:36:18.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:18.907+0000 7f14457fa640 1 -- 172.21.15.38:0/1848501238 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1438004470 con 0x7f14480b21e0 2024-09-21T03:36:18.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:18.908+0000 7f144ef5f640 1 -- 172.21.15.38:0/1848501238 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14481f6b30 con 0x7f14480b21e0 2024-09-21T03:36:18.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:18.908+0000 7f144ef5f640 1 -- 172.21.15.38:0/1848501238 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14481f7250 con 0x7f14480b21e0 2024-09-21T03:36:18.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:18.909+0000 7f14457fa640 1 -- 172.21.15.38:0/1848501238 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f1438005030 con 0x7f14480b21e0 2024-09-21T03:36:18.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:18.910+0000 7f14457fa640 1 --2- 172.21.15.38:0/1848501238 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1428078b90 0x7f142807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:18.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:18.910+0000 7f14457fa640 1 -- 172.21.15.38:0/1848501238 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5249+0+0 (secure 0 0 0) 0x7f143809b4d0 con 0x7f14480b21e0 2024-09-21T03:36:18.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:18.910+0000 7f1447fff640 1 --2- 172.21.15.38:0/1848501238 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1428078b90 0x7f142807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:18.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:18.911+0000 7f1447fff640 1 --2- 172.21.15.38:0/1848501238 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1428078b90 0x7f142807b0a0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f1430034000 tx=0x7f1430004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:18.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:18.915+0000 7f14457fa640 1 -- 172.21.15.38:0/1848501238 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f1438063ae0 con 0x7f14480b21e0 2024-09-21T03:36:19.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:19.105+0000 7f144ef5f640 1 -- 172.21.15.38:0/1848501238 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f144815da30 con 0x7f14480b21e0 2024-09-21T03:36:20.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:20.035+0000 7f14457fa640 1 -- 172.21.15.38:0/1848501238 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v64) ==== 138+0+0 (secure 0 0 0) 0x7f143806c020 con 0x7f14480b21e0 2024-09-21T03:36:20.037 INFO:teuthology.orchestra.run.smithi038.stderr:pool 'cephfs_metadata' created 2024-09-21T03:36:20.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:20.039+0000 7f144ef5f640 1 -- 172.21.15.38:0/1848501238 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1428078b90 msgr2=0x7f142807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:20.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:20.039+0000 7f144ef5f640 1 --2- 172.21.15.38:0/1848501238 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1428078b90 0x7f142807b0a0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f1430034000 tx=0x7f1430004730 comp rx=0 tx=0).stop 2024-09-21T03:36:20.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:20.039+0000 7f144ef5f640 1 -- 172.21.15.38:0/1848501238 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f14480b21e0 msgr2=0x7f14481f7610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:20.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:20.039+0000 7f144ef5f640 1 --2- 172.21.15.38:0/1848501238 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f14480b21e0 0x7f14481f7610 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f143800e9f0 tx=0x7f143800eeb0 comp rx=0 tx=0).stop 2024-09-21T03:36:20.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:20.039+0000 7f144ef5f640 1 -- 172.21.15.38:0/1848501238 shutdown_connections 2024-09-21T03:36:20.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:20.039+0000 7f144ef5f640 1 --2- 172.21.15.38:0/1848501238 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1448160bb0 0x7f1448148660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:20.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:20.039+0000 7f144ef5f640 1 --2- 172.21.15.38:0/1848501238 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1428078b90 0x7f142807b0a0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:20.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:20.039+0000 7f144ef5f640 1 --2- 172.21.15.38:0/1848501238 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f14480cd020 0x7f14481f3fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:20.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:20.039+0000 7f144ef5f640 1 --2- 172.21.15.38:0/1848501238 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f14480b21e0 0x7f14481f7610 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:20.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:20.039+0000 7f144ef5f640 1 -- 172.21.15.38:0/1848501238 >> 172.21.15.38:0/1848501238 conn(0x7f14480d2280 msgr2=0x7f14480ceac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:20.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:20.039+0000 7f144ef5f640 1 -- 172.21.15.38:0/1848501238 shutdown_connections 2024-09-21T03:36:20.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:20.039+0000 7f144ef5f640 1 -- 172.21.15.38:0/1848501238 wait complete. 2024-09-21T03:36:20.059 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-21T03:36:20.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:20.187+0000 7f3a592c4640 1 -- 172.21.15.38:0/1922989126 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3a540cfe00 msgr2=0x7f3a540d2a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:20.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:20.187+0000 7f3a592c4640 1 --2- 172.21.15.38:0/1922989126 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3a540cfe00 0x7f3a540d2a20 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f3a44005fd0 tx=0x7f3a4402a5d0 comp rx=0 tx=0).stop 2024-09-21T03:36:20.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:20.187+0000 7f3a592c4640 1 -- 172.21.15.38:0/1922989126 shutdown_connections 2024-09-21T03:36:20.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:20.187+0000 7f3a592c4640 1 --2- 172.21.15.38:0/1922989126 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3a54162590 0x7f3a541629a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:20.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:20.187+0000 7f3a592c4640 1 --2- 172.21.15.38:0/1922989126 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3a540cfe00 0x7f3a540d2a20 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:20.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:20.187+0000 7f3a592c4640 1 -- 172.21.15.38:0/1922989126 >> 172.21.15.38:0/1922989126 conn(0x7f3a540d4440 msgr2=0x7f3a540d4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:20.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:20.188+0000 7f3a592c4640 1 -- 172.21.15.38:0/1922989126 shutdown_connections 2024-09-21T03:36:20.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:20.188+0000 7f3a592c4640 1 -- 172.21.15.38:0/1922989126 wait complete. 2024-09-21T03:36:20.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:20.189+0000 7f3a592c4640 1 Processor -- start 2024-09-21T03:36:20.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:20.189+0000 7f3a592c4640 1 -- start start 2024-09-21T03:36:20.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:20.191+0000 7f3a592c4640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3a540b2e60 0x7f3a5415d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:20.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:20.191+0000 7f3a592c4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3a540cfe00 0x7f3a5414e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:20.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:20.191+0000 7f3a592c4640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3a54162590 0x7f3a540c8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:20.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:20.191+0000 7f3a592c4640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f3a540b3980 con 0x7f3a540cfe00 2024-09-21T03:36:20.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:20.191+0000 7f3a592c4640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f3a540cf5a0 con 0x7f3a54162590 2024-09-21T03:36:20.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:20.191+0000 7f3a592c4640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f3a540b3b00 con 0x7f3a540b2e60 2024-09-21T03:36:20.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:20.191+0000 7f3a4a7fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3a540cfe00 0x7f3a5414e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:20.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:20.191+0000 7f3a4a7fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3a540cfe00 0x7f3a5414e780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42418/0 (socket says 172.21.15.38:42418) 2024-09-21T03:36:20.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:20.191+0000 7f3a4a7fc640 1 -- 172.21.15.38:0/3950377978 learned_addr learned my addr 172.21.15.38:0/3950377978 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:36:20.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:20.191+0000 7f3a52ffd640 1 --2- 172.21.15.38:0/3950377978 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3a540b2e60 0x7f3a5415d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:20.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:20.191+0000 7f3a537fe640 1 --2- 172.21.15.38:0/3950377978 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3a54162590 0x7f3a540c8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:20.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:20.191+0000 7f3a4a7fc640 1 -- 172.21.15.38:0/3950377978 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3a540b2e60 msgr2=0x7f3a5415d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:20.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:20.191+0000 7f3a4a7fc640 1 --2- 172.21.15.38:0/3950377978 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3a540b2e60 0x7f3a5415d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:20.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:20.191+0000 7f3a4a7fc640 1 -- 172.21.15.38:0/3950377978 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3a54162590 msgr2=0x7f3a540c8af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:20.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:20.191+0000 7f3a4a7fc640 1 --2- 172.21.15.38:0/3950377978 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3a54162590 0x7f3a540c8af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:20.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:20.191+0000 7f3a4a7fc640 1 -- 172.21.15.38:0/3950377978 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a5415a620 con 0x7f3a540cfe00 2024-09-21T03:36:20.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:20.191+0000 7f3a4a7fc640 1 --2- 172.21.15.38:0/3950377978 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3a540cfe00 0x7f3a5414e780 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f3a3c00c970 tx=0x7f3a3c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:20.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:20.192+0000 7f3a4bfff640 1 -- 172.21.15.38:0/3950377978 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a3c007bb0 con 0x7f3a540cfe00 2024-09-21T03:36:20.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:20.192+0000 7f3a4bfff640 1 -- 172.21.15.38:0/3950377978 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3a3c007e40 con 0x7f3a540cfe00 2024-09-21T03:36:20.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:20.192+0000 7f3a592c4640 1 -- 172.21.15.38:0/3950377978 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a5415dde0 con 0x7f3a540cfe00 2024-09-21T03:36:20.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:20.192+0000 7f3a4bfff640 1 -- 172.21.15.38:0/3950377978 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a3c00f690 con 0x7f3a540cfe00 2024-09-21T03:36:20.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:20.192+0000 7f3a592c4640 1 -- 172.21.15.38:0/3950377978 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a5415e3a0 con 0x7f3a540cfe00 2024-09-21T03:36:20.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:20.193+0000 7f3a592c4640 1 -- 172.21.15.38:0/3950377978 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a5415db50 con 0x7f3a540cfe00 2024-09-21T03:36:20.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:20.194+0000 7f3a4bfff640 1 -- 172.21.15.38:0/3950377978 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f3a3c00f830 con 0x7f3a540cfe00 2024-09-21T03:36:20.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:20.194+0000 7f3a4bfff640 1 --2- 172.21.15.38:0/3950377978 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3a34078b90 0x7f3a3407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:20.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:20.195+0000 7f3a4bfff640 1 -- 172.21.15.38:0/3950377978 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5619+0+0 (secure 0 0 0) 0x7f3a3c09b540 con 0x7f3a540cfe00 2024-09-21T03:36:20.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:20.195+0000 7f3a52ffd640 1 --2- 172.21.15.38:0/3950377978 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3a34078b90 0x7f3a3407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:20.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:20.195+0000 7f3a52ffd640 1 --2- 172.21.15.38:0/3950377978 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3a34078b90 0x7f3a3407b0a0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f3a44033000 tx=0x7f3a44004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:20.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:20.199+0000 7f3a4bfff640 1 -- 172.21.15.38:0/3950377978 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f3a3c014030 con 0x7f3a540cfe00 2024-09-21T03:36:20.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:20.387+0000 7f3a592c4640 1 -- 172.21.15.38:0/3950377978 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f3a540d2a20 con 0x7f3a540cfe00 2024-09-21T03:36:21.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:21.045+0000 7f3a4bfff640 1 -- 172.21.15.38:0/3950377978 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v65) ==== 170+0+0 (secure 0 0 0) 0x7f3a3c0639d0 con 0x7f3a540cfe00 2024-09-21T03:36:21.047 INFO:teuthology.orchestra.run.smithi038.stderr:pool 'cephfs_data' created 2024-09-21T03:36:21.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:21.049+0000 7f3a592c4640 1 -- 172.21.15.38:0/3950377978 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3a34078b90 msgr2=0x7f3a3407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:21.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:21.049+0000 7f3a592c4640 1 --2- 172.21.15.38:0/3950377978 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3a34078b90 0x7f3a3407b0a0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f3a44033000 tx=0x7f3a44004730 comp rx=0 tx=0).stop 2024-09-21T03:36:21.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:21.049+0000 7f3a592c4640 1 -- 172.21.15.38:0/3950377978 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3a540cfe00 msgr2=0x7f3a5414e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:21.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:21.049+0000 7f3a592c4640 1 --2- 172.21.15.38:0/3950377978 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3a540cfe00 0x7f3a5414e780 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f3a3c00c970 tx=0x7f3a3c00ce30 comp rx=0 tx=0).stop 2024-09-21T03:36:21.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:21.052+0000 7f3a592c4640 1 -- 172.21.15.38:0/3950377978 shutdown_connections 2024-09-21T03:36:21.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:21.052+0000 7f3a592c4640 1 --2- 172.21.15.38:0/3950377978 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3a54162590 0x7f3a540c8af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:21.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:21.052+0000 7f3a592c4640 1 --2- 172.21.15.38:0/3950377978 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3a540cfe00 0x7f3a5414e780 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:21.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:21.052+0000 7f3a592c4640 1 --2- 172.21.15.38:0/3950377978 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3a34078b90 0x7f3a3407b0a0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:21.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:21.052+0000 7f3a592c4640 1 --2- 172.21.15.38:0/3950377978 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3a540b2e60 0x7f3a5415d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:21.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:21.052+0000 7f3a592c4640 1 -- 172.21.15.38:0/3950377978 >> 172.21.15.38:0/3950377978 conn(0x7f3a540d4440 msgr2=0x7f3a540b3270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:21.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:21.059+0000 7f3a592c4640 1 -- 172.21.15.38:0/3950377978 shutdown_connections 2024-09-21T03:36:21.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:21.062+0000 7f3a592c4640 1 -- 172.21.15.38:0/3950377978 wait complete. 2024-09-21T03:36:21.082 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-21T03:36:21.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:21.204+0000 7f20bf7f1640 1 -- 172.21.15.38:0/3614153581 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f20a4003fe0 msgr2=0x7f20a4004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:21.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:21.204+0000 7f20bf7f1640 1 --2- 172.21.15.38:0/3614153581 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f20a4003fe0 0x7f20a4004800 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f20a8009b10 tx=0x7f20a4005830 comp rx=0 tx=0).stop 2024-09-21T03:36:21.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:21.204+0000 7f20bdd8e640 1 -- 172.21.15.38:0/3614153581 reap_dead start 2024-09-21T03:36:21.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:21.204+0000 7f20bf7f1640 1 -- 172.21.15.38:0/3614153581 shutdown_connections 2024-09-21T03:36:21.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:21.204+0000 7f20bf7f1640 1 -- 172.21.15.38:0/3614153581 >> 172.21.15.38:0/3614153581 conn(0x7f20b809d830 msgr2=0x7f20b80b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:21.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:21.204+0000 7f20bf7f1640 1 -- 172.21.15.38:0/3614153581 shutdown_connections 2024-09-21T03:36:21.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:21.204+0000 7f20bf7f1640 1 -- 172.21.15.38:0/3614153581 wait complete. 2024-09-21T03:36:21.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:21.206+0000 7f20bf7f1640 1 Processor -- start 2024-09-21T03:36:21.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:21.206+0000 7f20bf7f1640 1 -- start start 2024-09-21T03:36:21.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:21.207+0000 7f20bf7f1640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f20a4003fe0 0x7f20b806f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:21.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:21.207+0000 7f20bf7f1640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f20b806e220 0x7f20b814d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:21.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:21.207+0000 7f20bf7f1640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f20b80cb610 0x7f20b80cba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:21.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:21.207+0000 7f20bf7f1640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f20b80d0f70 con 0x7f20b80cb610 2024-09-21T03:36:21.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:21.207+0000 7f20bf7f1640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f20b80d0df0 con 0x7f20b806e220 2024-09-21T03:36:21.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:21.207+0000 7f20bf7f1640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f20b80cf4f0 con 0x7f20a4003fe0 2024-09-21T03:36:21.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:21.207+0000 7f20bdd8e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f20b80cb610 0x7f20b80cba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:21.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:21.207+0000 7f20bd58d640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f20a4003fe0 0x7f20b806f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:21.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:21.207+0000 7f20bdd8e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f20b80cb610 0x7f20b80cba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42440/0 (socket says 172.21.15.38:42440) 2024-09-21T03:36:21.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:21.207+0000 7f20bdd8e640 1 -- 172.21.15.38:0/436578922 learned_addr learned my addr 172.21.15.38:0/436578922 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:36:21.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:21.207+0000 7f20bcd8c640 1 --2- 172.21.15.38:0/436578922 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f20b806e220 0x7f20b814d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:21.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:21.208+0000 7f20bdd8e640 1 -- 172.21.15.38:0/436578922 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f20a4003fe0 msgr2=0x7f20b806f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:21.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:21.208+0000 7f20bdd8e640 1 --2- 172.21.15.38:0/436578922 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f20a4003fe0 0x7f20b806f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:21.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:21.208+0000 7f20bdd8e640 1 -- 172.21.15.38:0/436578922 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f20b806e220 msgr2=0x7f20b814d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:21.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:21.208+0000 7f20bdd8e640 1 --2- 172.21.15.38:0/436578922 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f20b806e220 0x7f20b814d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:21.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:21.208+0000 7f20bdd8e640 1 -- 172.21.15.38:0/436578922 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20b814deb0 con 0x7f20b80cb610 2024-09-21T03:36:21.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:21.208+0000 7f20bdd8e640 1 --2- 172.21.15.38:0/436578922 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f20b80cb610 0x7f20b80cba20 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f20b808cbe0 tx=0x7f20b00064f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:21.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:21.208+0000 7f20adffb640 1 -- 172.21.15.38:0/436578922 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20b000f920 con 0x7f20b80cb610 2024-09-21T03:36:21.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:21.208+0000 7f20bf7f1640 1 -- 172.21.15.38:0/436578922 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20b806d180 con 0x7f20b80cb610 2024-09-21T03:36:21.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:21.208+0000 7f20adffb640 1 -- 172.21.15.38:0/436578922 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f20b0017070 con 0x7f20b80cb610 2024-09-21T03:36:21.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:21.208+0000 7f20adffb640 1 -- 172.21.15.38:0/436578922 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20b000fd20 con 0x7f20b80cb610 2024-09-21T03:36:21.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:21.208+0000 7f20bf7f1640 1 -- 172.21.15.38:0/436578922 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20b806d740 con 0x7f20b80cb610 2024-09-21T03:36:21.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:21.209+0000 7f20bf7f1640 1 -- 172.21.15.38:0/436578922 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20b80ceb90 con 0x7f20b80cb610 2024-09-21T03:36:21.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:21.210+0000 7f20adffb640 1 -- 172.21.15.38:0/436578922 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f20b001c460 con 0x7f20b80cb610 2024-09-21T03:36:21.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:21.211+0000 7f20adffb640 1 --2- 172.21.15.38:0/436578922 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f209c078bc0 0x7f209c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:21.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:21.211+0000 7f20adffb640 1 -- 172.21.15.38:0/436578922 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 6001+0+0 (secure 0 0 0) 0x7f20b009b500 con 0x7f20b80cb610 2024-09-21T03:36:21.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:21.211+0000 7f20bd58d640 1 --2- 172.21.15.38:0/436578922 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f209c078bc0 0x7f209c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:21.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:21.211+0000 7f20bd58d640 1 --2- 172.21.15.38:0/436578922 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f209c078bc0 0x7f209c07b0d0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f20b4001a30 tx=0x7f20b40068a0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:21.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:21.216+0000 7f20adffb640 1 -- 172.21.15.38:0/436578922 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f20b814deb0 con 0x7f20b80cb610 2024-09-21T03:36:21.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:21.441+0000 7f20bf7f1640 1 -- 172.21.15.38:0/436578922 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f20b80b9f70 con 0x7f20b80cb610 2024-09-21T03:36:22.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:22.047+0000 7f20adffb640 1 -- 172.21.15.38:0/436578922 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 9 and data pool 10 v41) ==== 173+0+0 (secure 0 0 0) 0x7f20b00630a0 con 0x7f20b80cb610 2024-09-21T03:36:22.049 INFO:teuthology.orchestra.run.smithi038.stderr:new fs with metadata pool 9 and data pool 10 2024-09-21T03:36:22.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:22.051+0000 7f20bf7f1640 1 -- 172.21.15.38:0/436578922 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f209c078bc0 msgr2=0x7f209c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:22.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:22.051+0000 7f20bf7f1640 1 --2- 172.21.15.38:0/436578922 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f209c078bc0 0x7f209c07b0d0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f20b4001a30 tx=0x7f20b40068a0 comp rx=0 tx=0).stop 2024-09-21T03:36:22.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:22.051+0000 7f20bf7f1640 1 -- 172.21.15.38:0/436578922 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f20b80cb610 msgr2=0x7f20b80cba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:22.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:22.051+0000 7f20bf7f1640 1 --2- 172.21.15.38:0/436578922 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f20b80cb610 0x7f20b80cba20 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f20b808cbe0 tx=0x7f20b00064f0 comp rx=0 tx=0).stop 2024-09-21T03:36:22.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:22.051+0000 7f20bf7f1640 1 -- 172.21.15.38:0/436578922 shutdown_connections 2024-09-21T03:36:22.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:22.051+0000 7f20bf7f1640 1 --2- 172.21.15.38:0/436578922 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f20b80cb610 0x7f20b80cba20 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:22.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:22.051+0000 7f20bf7f1640 1 --2- 172.21.15.38:0/436578922 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f20b806e220 0x7f20b814d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:22.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:22.051+0000 7f20bf7f1640 1 --2- 172.21.15.38:0/436578922 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f209c078bc0 0x7f209c07b0d0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:22.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:22.051+0000 7f20bf7f1640 1 --2- 172.21.15.38:0/436578922 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f20a4003fe0 0x7f20b806f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:22.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:22.051+0000 7f20bf7f1640 1 -- 172.21.15.38:0/436578922 >> 172.21.15.38:0/436578922 conn(0x7f20b809d830 msgr2=0x7f20b8160670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:22.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:22.051+0000 7f20bf7f1640 1 -- 172.21.15.38:0/436578922 shutdown_connections 2024-09-21T03:36:22.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:22.051+0000 7f20bf7f1640 1 -- 172.21.15.38:0/436578922 wait complete. 2024-09-21T03:36:22.079 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T03:36:22.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:22.206+0000 7fc6d230b640 1 -- 172.21.15.38:0/3691223325 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc6b4004800 msgr2=0x7fc6b4004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:22.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:22.206+0000 7fc6d230b640 1 --2- 172.21.15.38:0/3691223325 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc6b4004800 0x7fc6b4004c10 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fc6cc074e00 tx=0x7fc6b4005830 comp rx=0 tx=0).stop 2024-09-21T03:36:22.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:22.206+0000 7fc6d08a8640 1 -- 172.21.15.38:0/3691223325 reap_dead start 2024-09-21T03:36:22.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:22.206+0000 7fc6d230b640 1 -- 172.21.15.38:0/3691223325 shutdown_connections 2024-09-21T03:36:22.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:22.206+0000 7fc6d230b640 1 -- 172.21.15.38:0/3691223325 >> 172.21.15.38:0/3691223325 conn(0x7fc6cc0cef40 msgr2=0x7fc6cc0cf350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:22.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:22.206+0000 7fc6d230b640 1 -- 172.21.15.38:0/3691223325 shutdown_connections 2024-09-21T03:36:22.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:22.206+0000 7fc6d230b640 1 -- 172.21.15.38:0/3691223325 wait complete. 2024-09-21T03:36:22.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:22.208+0000 7fc6d230b640 1 Processor -- start 2024-09-21T03:36:22.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:22.208+0000 7fc6d230b640 1 -- start start 2024-09-21T03:36:22.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:22.209+0000 7fc6d230b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc6b4004800 0x7fc6cc1f8400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:22.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:22.209+0000 7fc6d230b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc6cc15db00 0x7fc6cc15df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:22.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:22.209+0000 7fc6d230b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc6cc1f4300 0x7fc6cc1f4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:22.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:22.209+0000 7fc6d230b640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fc6cc06fd40 con 0x7fc6cc15db00 2024-09-21T03:36:22.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:22.209+0000 7fc6d230b640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fc6cc06fbc0 con 0x7fc6cc1f4300 2024-09-21T03:36:22.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:22.209+0000 7fc6d230b640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fc6cc1f8bd0 con 0x7fc6b4004800 2024-09-21T03:36:22.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:22.209+0000 7fc6cb7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc6cc15db00 0x7fc6cc15df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:22.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:22.210+0000 7fc6cb7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc6cc15db00 0x7fc6cc15df10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42462/0 (socket says 172.21.15.38:42462) 2024-09-21T03:36:22.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:22.210+0000 7fc6cb7fe640 1 -- 172.21.15.38:0/860158947 learned_addr learned my addr 172.21.15.38:0/860158947 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:36:22.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:22.210+0000 7fc6cbfff640 1 --2- 172.21.15.38:0/860158947 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc6b4004800 0x7fc6cc1f8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:22.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:22.210+0000 7fc6d08a8640 1 --2- 172.21.15.38:0/860158947 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc6cc1f4300 0x7fc6cc1f4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:22.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:22.210+0000 7fc6cb7fe640 1 -- 172.21.15.38:0/860158947 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc6b4004800 msgr2=0x7fc6cc1f8400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:22.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:22.210+0000 7fc6cb7fe640 1 --2- 172.21.15.38:0/860158947 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc6b4004800 0x7fc6cc1f8400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:22.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:22.210+0000 7fc6cb7fe640 1 -- 172.21.15.38:0/860158947 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc6cc1f4300 msgr2=0x7fc6cc1f4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:22.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:22.210+0000 7fc6cb7fe640 1 --2- 172.21.15.38:0/860158947 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc6cc1f4300 0x7fc6cc1f4710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:22.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:22.210+0000 7fc6cb7fe640 1 -- 172.21.15.38:0/860158947 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6cc1f8940 con 0x7fc6cc15db00 2024-09-21T03:36:22.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:22.210+0000 7fc6d08a8640 1 --2- 172.21.15.38:0/860158947 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc6cc1f4300 0x7fc6cc1f4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:36:22.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:22.210+0000 7fc6cb7fe640 1 --2- 172.21.15.38:0/860158947 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc6cc15db00 0x7fc6cc15df10 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fc6c002b070 tx=0x7fc6c0001c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:22.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:22.210+0000 7fc6c8ff9640 1 -- 172.21.15.38:0/860158947 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6c00037f0 con 0x7fc6cc15db00 2024-09-21T03:36:22.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:22.210+0000 7fc6c8ff9640 1 -- 172.21.15.38:0/860158947 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc6c0003a80 con 0x7fc6cc15db00 2024-09-21T03:36:22.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:22.210+0000 7fc6d230b640 1 -- 172.21.15.38:0/860158947 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6cc1f8e20 con 0x7fc6cc15db00 2024-09-21T03:36:22.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:22.210+0000 7fc6c8ff9640 1 -- 172.21.15.38:0/860158947 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6c002e450 con 0x7fc6cc15db00 2024-09-21T03:36:22.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:22.210+0000 7fc6d230b640 1 -- 172.21.15.38:0/860158947 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6cc156f60 con 0x7fc6cc15db00 2024-09-21T03:36:22.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:22.211+0000 7fc6d230b640 1 -- 172.21.15.38:0/860158947 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6cc1f9340 con 0x7fc6cc15db00 2024-09-21T03:36:22.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:22.212+0000 7fc6c8ff9640 1 -- 172.21.15.38:0/860158947 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc6c0035020 con 0x7fc6cc15db00 2024-09-21T03:36:22.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:22.213+0000 7fc6c8ff9640 1 --2- 172.21.15.38:0/860158947 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc6b8078b90 0x7fc6b807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:22.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:22.213+0000 7fc6c8ff9640 1 -- 172.21.15.38:0/860158947 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 6101+0+0 (secure 0 0 0) 0x7fc6c00bb700 con 0x7fc6cc15db00 2024-09-21T03:36:22.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:22.213+0000 7fc6cbfff640 1 --2- 172.21.15.38:0/860158947 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc6b8078b90 0x7fc6b807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:22.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:22.214+0000 7fc6cbfff640 1 --2- 172.21.15.38:0/860158947 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc6b8078b90 0x7fc6b807b0a0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fc6bc000c00 tx=0x7fc6bc006b70 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:22.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:22.218+0000 7fc6c8ff9640 1 -- 172.21.15.38:0/860158947 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fc6cc1f9340 con 0x7fc6cc15db00 2024-09-21T03:36:22.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:22.407+0000 7fc6d230b640 1 -- 172.21.15.38:0/860158947 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc6cc0b3ec0 con 0x7fc6cc15db00 2024-09-21T03:36:22.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:22.408+0000 7fc6c8ff9640 1 -- 172.21.15.38:0/860158947 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v66) ==== 74+0+17029 (secure 0 0 0) 0x7fc6c0047020 con 0x7fc6cc15db00 2024-09-21T03:36:22.410 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:36:22.410 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":66,"fsid":"9d50d66f-656b-4fe2-8e3b-fe53b79c4d43","created":"2024-09-21T03:33:10.677865+0000","modified":"2024-09-21T03:36:21.442769+0000","last_up_change":"2024-09-21T03:33:24.023421+0000","last_in_change":"2024-09-21T03:33:14.296193+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T03:33:22.992573+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-09-21T03:36:19.109965+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-09-21T03:36:20.392299+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dbab183f-7e63-48b8-960a-ae55fd5bf565","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6801","nonce":2381639433}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6803","nonce":2381639433}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6807","nonce":2381639433}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6805","nonce":2381639433}]},"public_addr":"172.21.15.38:6801/2381639433","cluster_addr":"172.21.15.38:6803/2381639433","heartbeat_back_addr":"172.21.15.38:6807/2381639433","heartbeat_front_addr":"172.21.15.38:6805/2381639433","state":["exists","up"]},{"osd":1,"uuid":"fa1c0e8a-ebc1-4359-8146-bd5342abbc16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6817","nonce":1491818732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6819","nonce":1491818732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6823","nonce":1491818732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6821","nonce":1491818732}]},"public_addr":"172.21.15.38:6817/1491818732","cluster_addr":"172.21.15.38:6819/1491818732","heartbeat_back_addr":"172.21.15.38:6823/1491818732","heartbeat_front_addr":"172.21.15.38:6821/1491818732","state":["exists","up"]},{"osd":2,"uuid":"2ed3cd2d-385d-4fc6-9869-214552a84803","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6825","nonce":2577858902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6827","nonce":2577858902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6831","nonce":2577858902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6829","nonce":2577858902}]},"public_addr":"172.21.15.38:6825/2577858902","cluster_addr":"172.21.15.38:6827/2577858902","heartbeat_back_addr":"172.21.15.38:6831/2577858902","heartbeat_front_addr":"172.21.15.38:6829/2577858902","state":["exists","up"]},{"osd":3,"uuid":"62d1d95e-919d-4250-be71-1d34cfa70dbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6809","nonce":1286810294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6811","nonce":1286810294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6815","nonce":1286810294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6813","nonce":1286810294}]},"public_addr":"172.21.15.38:6809/1286810294","cluster_addr":"172.21.15.38:6811/1286810294","heartbeat_back_addr":"172.21.15.38:6815/1286810294","heartbeat_front_addr":"172.21.15.38:6813/1286810294","state":["exists","up"]},{"osd":4,"uuid":"7c9f9314-99d7-4d73-8392-37809a9f54b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6803","nonce":3401837571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6805","nonce":3401837571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6809","nonce":3401837571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6807","nonce":3401837571}]},"public_addr":"172.21.15.195:6803/3401837571","cluster_addr":"172.21.15.195:6805/3401837571","heartbeat_back_addr":"172.21.15.195:6809/3401837571","heartbeat_front_addr":"172.21.15.195:6807/3401837571","state":["exists","up"]},{"osd":5,"uuid":"bde4fe9f-8414-4bba-bf4a-43973d1976bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6811","nonce":2080960180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6813","nonce":2080960180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6817","nonce":2080960180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6815","nonce":2080960180}]},"public_addr":"172.21.15.195:6811/2080960180","cluster_addr":"172.21.15.195:6813/2080960180","heartbeat_back_addr":"172.21.15.195:6817/2080960180","heartbeat_front_addr":"172.21.15.195:6815/2080960180","state":["exists","up"]},{"osd":6,"uuid":"6ee6d7f4-1a5f-4a4c-ac16-48fe05f6a002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6827","nonce":4055825675}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6829","nonce":4055825675}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6833","nonce":4055825675}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6831","nonce":4055825675}]},"public_addr":"172.21.15.195:6827/4055825675","cluster_addr":"172.21.15.195:6829/4055825675","heartbeat_back_addr":"172.21.15.195:6833/4055825675","heartbeat_front_addr":"172.21.15.195:6831/4055825675","state":["exists","up"]},{"osd":7,"uuid":"efeb5f51-329a-4484-8290-92e7d3503d05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6819","nonce":1606106342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6821","nonce":1606106342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6825","nonce":1606106342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6823","nonce":1606106342}]},"public_addr":"172.21.15.195:6819/1606106342","cluster_addr":"172.21.15.195:6821/1606106342","heartbeat_back_addr":"172.21.15.195:6825/1606106342","heartbeat_front_addr":"172.21.15.195:6823/1606106342","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.901343+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:21.115003+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.901623+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.835238+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.132199+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:19.817560+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.829690+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:21.087300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T03:36:22.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:22.410+0000 7fc6d230b640 1 -- 172.21.15.38:0/860158947 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc6b8078b90 msgr2=0x7fc6b807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:22.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:22.410+0000 7fc6d230b640 1 --2- 172.21.15.38:0/860158947 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc6b8078b90 0x7fc6b807b0a0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fc6bc000c00 tx=0x7fc6bc006b70 comp rx=0 tx=0).stop 2024-09-21T03:36:22.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:22.410+0000 7fc6d230b640 1 -- 172.21.15.38:0/860158947 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc6cc15db00 msgr2=0x7fc6cc15df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:22.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:22.410+0000 7fc6d230b640 1 --2- 172.21.15.38:0/860158947 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc6cc15db00 0x7fc6cc15df10 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fc6c002b070 tx=0x7fc6c0001c70 comp rx=0 tx=0).stop 2024-09-21T03:36:22.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:22.410+0000 7fc6d230b640 1 -- 172.21.15.38:0/860158947 shutdown_connections 2024-09-21T03:36:22.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:22.410+0000 7fc6d230b640 1 --2- 172.21.15.38:0/860158947 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc6cc1f4300 0x7fc6cc1f4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:22.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:22.410+0000 7fc6d230b640 1 --2- 172.21.15.38:0/860158947 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc6cc15db00 0x7fc6cc15df10 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:22.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:22.410+0000 7fc6d230b640 1 --2- 172.21.15.38:0/860158947 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc6b8078b90 0x7fc6b807b0a0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:22.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:22.410+0000 7fc6d230b640 1 --2- 172.21.15.38:0/860158947 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc6b4004800 0x7fc6cc1f8400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:22.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:22.410+0000 7fc6d230b640 1 -- 172.21.15.38:0/860158947 >> 172.21.15.38:0/860158947 conn(0x7fc6cc0cef40 msgr2=0x7fc6cc0cf350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:22.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:22.410+0000 7fc6d230b640 1 -- 172.21.15.38:0/860158947 shutdown_connections 2024-09-21T03:36:22.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:22.410+0000 7fc6d230b640 1 -- 172.21.15.38:0/860158947 wait complete. 2024-09-21T03:36:22.424 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T03:36:22.541 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:22.538+0000 7fbe93f39640 1 -- 172.21.15.38:0/3385570571 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbe78005150 msgr2=0x7fbe7800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:22.541 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:22.538+0000 7fbe93f39640 1 --2- 172.21.15.38:0/3385570571 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbe78005150 0x7fbe7800b9c0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fbe84001df0 tx=0x7fbe78005830 comp rx=0 tx=0).stop 2024-09-21T03:36:22.541 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:22.538+0000 7fbe924d6640 1 -- 172.21.15.38:0/3385570571 reap_dead start 2024-09-21T03:36:22.541 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:22.538+0000 7fbe93f39640 1 -- 172.21.15.38:0/3385570571 shutdown_connections 2024-09-21T03:36:22.541 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:22.538+0000 7fbe93f39640 1 -- 172.21.15.38:0/3385570571 >> 172.21.15.38:0/3385570571 conn(0x7fbe8c0d4620 msgr2=0x7fbe8c0d4a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:22.541 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:22.539+0000 7fbe93f39640 1 -- 172.21.15.38:0/3385570571 shutdown_connections 2024-09-21T03:36:22.541 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:22.539+0000 7fbe93f39640 1 -- 172.21.15.38:0/3385570571 wait complete. 2024-09-21T03:36:22.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:22.540+0000 7fbe93f39640 1 Processor -- start 2024-09-21T03:36:22.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:22.540+0000 7fbe93f39640 1 -- start start 2024-09-21T03:36:22.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:22.542+0000 7fbe93f39640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbe78005150 0x7fbe8c1602d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:22.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:22.542+0000 7fbe93f39640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbe8c160810 0x7fbe8c15b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:22.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:22.542+0000 7fbe93f39640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbe8c15b870 0x7fbe8c15bc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:22.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:22.542+0000 7fbe93f39640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fbe8c0b3af0 con 0x7fbe78005150 2024-09-21T03:36:22.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:22.542+0000 7fbe93f39640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fbe8c0d03d0 con 0x7fbe8c15b870 2024-09-21T03:36:22.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:22.542+0000 7fbe93f39640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fbe8c0b3c70 con 0x7fbe8c160810 2024-09-21T03:36:22.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:22.542+0000 7fbe91cd5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbe78005150 0x7fbe8c1602d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:22.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:22.542+0000 7fbe91cd5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbe78005150 0x7fbe8c1602d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37254/0 (socket says 172.21.15.38:37254) 2024-09-21T03:36:22.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:22.542+0000 7fbe91cd5640 1 -- 172.21.15.38:0/1199207264 learned_addr learned my addr 172.21.15.38:0/1199207264 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:36:22.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:22.542+0000 7fbe914d4640 1 --2- 172.21.15.38:0/1199207264 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbe8c160810 0x7fbe8c15b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:22.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:22.542+0000 7fbe924d6640 1 --2- 172.21.15.38:0/1199207264 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbe8c15b870 0x7fbe8c15bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:22.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:22.542+0000 7fbe91cd5640 1 -- 172.21.15.38:0/1199207264 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbe8c160810 msgr2=0x7fbe8c15b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:22.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:22.542+0000 7fbe91cd5640 1 --2- 172.21.15.38:0/1199207264 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbe8c160810 0x7fbe8c15b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:22.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:22.542+0000 7fbe91cd5640 1 -- 172.21.15.38:0/1199207264 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbe8c15b870 msgr2=0x7fbe8c15bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:22.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:22.542+0000 7fbe91cd5640 1 --2- 172.21.15.38:0/1199207264 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbe8c15b870 0x7fbe8c15bc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:22.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:22.542+0000 7fbe91cd5640 1 -- 172.21.15.38:0/1199207264 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe8c160c20 con 0x7fbe78005150 2024-09-21T03:36:22.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:22.542+0000 7fbe914d4640 1 --2- 172.21.15.38:0/1199207264 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbe8c160810 0x7fbe8c15b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:36:22.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:22.542+0000 7fbe924d6640 1 --2- 172.21.15.38:0/1199207264 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbe8c15b870 0x7fbe8c15bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:36:22.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:22.542+0000 7fbe91cd5640 1 --2- 172.21.15.38:0/1199207264 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbe78005150 0x7fbe8c1602d0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fbe880018b0 tx=0x7fbe880039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:22.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:22.542+0000 7fbe827fc640 1 -- 172.21.15.38:0/1199207264 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe88011700 con 0x7fbe78005150 2024-09-21T03:36:22.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:22.542+0000 7fbe827fc640 1 -- 172.21.15.38:0/1199207264 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbe88016020 con 0x7fbe78005150 2024-09-21T03:36:22.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:22.542+0000 7fbe93f39640 1 -- 172.21.15.38:0/1199207264 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe8c156560 con 0x7fbe78005150 2024-09-21T03:36:22.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:22.542+0000 7fbe827fc640 1 -- 172.21.15.38:0/1199207264 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe88011b20 con 0x7fbe78005150 2024-09-21T03:36:22.547 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:22.543+0000 7fbe93f39640 1 -- 172.21.15.38:0/1199207264 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe8c156b20 con 0x7fbe78005150 2024-09-21T03:36:22.547 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:22.543+0000 7fbe93f39640 1 -- 172.21.15.38:0/1199207264 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe8c0008d0 con 0x7fbe78005150 2024-09-21T03:36:22.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:22.544+0000 7fbe827fc640 1 -- 172.21.15.38:0/1199207264 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fbe88011cc0 con 0x7fbe78005150 2024-09-21T03:36:22.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:22.545+0000 7fbe827fc640 1 --2- 172.21.15.38:0/1199207264 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fbe70078b90 0x7fbe7007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:22.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:22.545+0000 7fbe827fc640 1 -- 172.21.15.38:0/1199207264 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 6101+0+0 (secure 0 0 0) 0x7fbe8809b6f0 con 0x7fbe78005150 2024-09-21T03:36:22.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:22.545+0000 7fbe914d4640 1 --2- 172.21.15.38:0/1199207264 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fbe70078b90 0x7fbe7007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:22.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:22.546+0000 7fbe914d4640 1 --2- 172.21.15.38:0/1199207264 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fbe70078b90 0x7fbe7007b0a0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c1f86f0 tx=0x7fbe7c005f90 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:22.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:22.549+0000 7fbe827fc640 1 -- 172.21.15.38:0/1199207264 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fbe88020030 con 0x7fbe78005150 2024-09-21T03:36:22.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:22.738+0000 7fbe93f39640 1 -- 172.21.15.38:0/1199207264 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbe8c06f4d0 con 0x7fbe78005150 2024-09-21T03:36:22.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:22.739+0000 7fbe827fc640 1 -- 172.21.15.38:0/1199207264 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v66) ==== 74+0+17029 (secure 0 0 0) 0x7fbe8c160c20 con 0x7fbe78005150 2024-09-21T03:36:22.741 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:36:22.741 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":66,"fsid":"9d50d66f-656b-4fe2-8e3b-fe53b79c4d43","created":"2024-09-21T03:33:10.677865+0000","modified":"2024-09-21T03:36:21.442769+0000","last_up_change":"2024-09-21T03:33:24.023421+0000","last_in_change":"2024-09-21T03:33:14.296193+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T03:33:22.992573+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-09-21T03:36:19.109965+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-09-21T03:36:20.392299+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dbab183f-7e63-48b8-960a-ae55fd5bf565","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6801","nonce":2381639433}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6803","nonce":2381639433}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6807","nonce":2381639433}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6805","nonce":2381639433}]},"public_addr":"172.21.15.38:6801/2381639433","cluster_addr":"172.21.15.38:6803/2381639433","heartbeat_back_addr":"172.21.15.38:6807/2381639433","heartbeat_front_addr":"172.21.15.38:6805/2381639433","state":["exists","up"]},{"osd":1,"uuid":"fa1c0e8a-ebc1-4359-8146-bd5342abbc16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6817","nonce":1491818732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6819","nonce":1491818732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6823","nonce":1491818732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6821","nonce":1491818732}]},"public_addr":"172.21.15.38:6817/1491818732","cluster_addr":"172.21.15.38:6819/1491818732","heartbeat_back_addr":"172.21.15.38:6823/1491818732","heartbeat_front_addr":"172.21.15.38:6821/1491818732","state":["exists","up"]},{"osd":2,"uuid":"2ed3cd2d-385d-4fc6-9869-214552a84803","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6825","nonce":2577858902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6827","nonce":2577858902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6831","nonce":2577858902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6829","nonce":2577858902}]},"public_addr":"172.21.15.38:6825/2577858902","cluster_addr":"172.21.15.38:6827/2577858902","heartbeat_back_addr":"172.21.15.38:6831/2577858902","heartbeat_front_addr":"172.21.15.38:6829/2577858902","state":["exists","up"]},{"osd":3,"uuid":"62d1d95e-919d-4250-be71-1d34cfa70dbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6809","nonce":1286810294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6811","nonce":1286810294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6815","nonce":1286810294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6813","nonce":1286810294}]},"public_addr":"172.21.15.38:6809/1286810294","cluster_addr":"172.21.15.38:6811/1286810294","heartbeat_back_addr":"172.21.15.38:6815/1286810294","heartbeat_front_addr":"172.21.15.38:6813/1286810294","state":["exists","up"]},{"osd":4,"uuid":"7c9f9314-99d7-4d73-8392-37809a9f54b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6803","nonce":3401837571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6805","nonce":3401837571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6809","nonce":3401837571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6807","nonce":3401837571}]},"public_addr":"172.21.15.195:6803/3401837571","cluster_addr":"172.21.15.195:6805/3401837571","heartbeat_back_addr":"172.21.15.195:6809/3401837571","heartbeat_front_addr":"172.21.15.195:6807/3401837571","state":["exists","up"]},{"osd":5,"uuid":"bde4fe9f-8414-4bba-bf4a-43973d1976bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6811","nonce":2080960180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6813","nonce":2080960180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6817","nonce":2080960180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6815","nonce":2080960180}]},"public_addr":"172.21.15.195:6811/2080960180","cluster_addr":"172.21.15.195:6813/2080960180","heartbeat_back_addr":"172.21.15.195:6817/2080960180","heartbeat_front_addr":"172.21.15.195:6815/2080960180","state":["exists","up"]},{"osd":6,"uuid":"6ee6d7f4-1a5f-4a4c-ac16-48fe05f6a002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6827","nonce":4055825675}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6829","nonce":4055825675}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6833","nonce":4055825675}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6831","nonce":4055825675}]},"public_addr":"172.21.15.195:6827/4055825675","cluster_addr":"172.21.15.195:6829/4055825675","heartbeat_back_addr":"172.21.15.195:6833/4055825675","heartbeat_front_addr":"172.21.15.195:6831/4055825675","state":["exists","up"]},{"osd":7,"uuid":"efeb5f51-329a-4484-8290-92e7d3503d05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6819","nonce":1606106342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6821","nonce":1606106342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6825","nonce":1606106342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6823","nonce":1606106342}]},"public_addr":"172.21.15.195:6819/1606106342","cluster_addr":"172.21.15.195:6821/1606106342","heartbeat_back_addr":"172.21.15.195:6825/1606106342","heartbeat_front_addr":"172.21.15.195:6823/1606106342","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.901343+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:21.115003+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.901623+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.835238+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.132199+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:19.817560+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.829690+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:21.087300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T03:36:22.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:22.741+0000 7fbe93f39640 1 -- 172.21.15.38:0/1199207264 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fbe70078b90 msgr2=0x7fbe7007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:22.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:22.741+0000 7fbe93f39640 1 --2- 172.21.15.38:0/1199207264 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fbe70078b90 0x7fbe7007b0a0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c1f86f0 tx=0x7fbe7c005f90 comp rx=0 tx=0).stop 2024-09-21T03:36:22.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:22.741+0000 7fbe93f39640 1 -- 172.21.15.38:0/1199207264 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbe78005150 msgr2=0x7fbe8c1602d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:22.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:22.741+0000 7fbe93f39640 1 --2- 172.21.15.38:0/1199207264 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbe78005150 0x7fbe8c1602d0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fbe880018b0 tx=0x7fbe880039c0 comp rx=0 tx=0).stop 2024-09-21T03:36:22.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:22.741+0000 7fbe93f39640 1 -- 172.21.15.38:0/1199207264 shutdown_connections 2024-09-21T03:36:22.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:22.741+0000 7fbe93f39640 1 --2- 172.21.15.38:0/1199207264 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbe8c15b870 0x7fbe8c15bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:22.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:22.741+0000 7fbe93f39640 1 --2- 172.21.15.38:0/1199207264 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fbe70078b90 0x7fbe7007b0a0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:22.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:22.741+0000 7fbe93f39640 1 --2- 172.21.15.38:0/1199207264 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbe8c160810 0x7fbe8c15b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:22.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:22.741+0000 7fbe93f39640 1 --2- 172.21.15.38:0/1199207264 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbe78005150 0x7fbe8c1602d0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:22.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:22.741+0000 7fbe93f39640 1 -- 172.21.15.38:0/1199207264 >> 172.21.15.38:0/1199207264 conn(0x7fbe8c0d4620 msgr2=0x7fbe8c0d3770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:22.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:22.741+0000 7fbe93f39640 1 -- 172.21.15.38:0/1199207264 shutdown_connections 2024-09-21T03:36:22.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:22.741+0000 7fbe93f39640 1 -- 172.21.15.38:0/1199207264 wait complete. 2024-09-21T03:36:22.755 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-21T03:36:22.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:22.873+0000 7f7aa1313640 1 -- 172.21.15.38:0/1941670107 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7a9c15ee80 msgr2=0x7f7a9c0d0170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:22.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:22.873+0000 7f7aa1313640 1 --2- 172.21.15.38:0/1941670107 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7a9c15ee80 0x7f7a9c0d0170 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f7a8c005fd0 tx=0x7f7a8c02a820 comp rx=0 tx=0).stop 2024-09-21T03:36:22.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:22.873+0000 7f7aa1313640 1 -- 172.21.15.38:0/1941670107 shutdown_connections 2024-09-21T03:36:22.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:22.873+0000 7f7aa1313640 1 --2- 172.21.15.38:0/1941670107 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7a9c0b9ad0 0x7f7a9c0d0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:22.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:22.873+0000 7f7aa1313640 1 --2- 172.21.15.38:0/1941670107 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7a9c15ee80 0x7f7a9c0d0170 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:22.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:22.873+0000 7f7aa1313640 1 -- 172.21.15.38:0/1941670107 >> 172.21.15.38:0/1941670107 conn(0x7f7a9c09d830 msgr2=0x7f7a9c0b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:22.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:22.873+0000 7f7aa1313640 1 -- 172.21.15.38:0/1941670107 shutdown_connections 2024-09-21T03:36:22.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:22.873+0000 7f7aa1313640 1 -- 172.21.15.38:0/1941670107 wait complete. 2024-09-21T03:36:22.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:22.875+0000 7f7aa1313640 1 Processor -- start 2024-09-21T03:36:22.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:22.875+0000 7f7aa1313640 1 -- start start 2024-09-21T03:36:22.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:22.876+0000 7f7aa1313640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7a9c0b9ad0 0x7f7a9c0b49d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:22.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:22.876+0000 7f7aa1313640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7a9c0d1120 0x7f7a9c0b4f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:22.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:22.876+0000 7f7aa1313640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7a9c15ee80 0x7f7a9c1f8e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:22.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:22.876+0000 7f7aa1313640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f7a9c0b6700 con 0x7f7a9c15ee80 2024-09-21T03:36:22.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:22.876+0000 7f7aa1313640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f7a9c0b6580 con 0x7f7a9c0d1120 2024-09-21T03:36:22.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:22.876+0000 7f7aa1313640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f7a9c0b6880 con 0x7f7a9c0b9ad0 2024-09-21T03:36:22.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:22.877+0000 7f7a9b7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7a9c15ee80 0x7f7a9c1f8e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:22.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:22.877+0000 7f7a9b7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7a9c15ee80 0x7f7a9c1f8e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37268/0 (socket says 172.21.15.38:37268) 2024-09-21T03:36:22.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:22.877+0000 7f7a9b7fe640 1 -- 172.21.15.38:0/3284326445 learned_addr learned my addr 172.21.15.38:0/3284326445 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:36:22.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:22.877+0000 7f7a9a7fc640 1 --2- 172.21.15.38:0/3284326445 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7a9c0d1120 0x7f7a9c0b4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:22.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:22.877+0000 7f7a9affd640 1 --2- 172.21.15.38:0/3284326445 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7a9c0b9ad0 0x7f7a9c0b49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:22.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:22.877+0000 7f7a9b7fe640 1 -- 172.21.15.38:0/3284326445 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7a9c0b9ad0 msgr2=0x7f7a9c0b49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:22.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:22.877+0000 7f7a9b7fe640 1 --2- 172.21.15.38:0/3284326445 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7a9c0b9ad0 0x7f7a9c0b49d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:22.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:22.877+0000 7f7a9b7fe640 1 -- 172.21.15.38:0/3284326445 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7a9c0d1120 msgr2=0x7f7a9c0b4f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:22.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:22.877+0000 7f7a9b7fe640 1 --2- 172.21.15.38:0/3284326445 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7a9c0d1120 0x7f7a9c0b4f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:22.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:22.877+0000 7f7a9b7fe640 1 -- 172.21.15.38:0/3284326445 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a9c06f970 con 0x7f7a9c15ee80 2024-09-21T03:36:22.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:22.877+0000 7f7a9b7fe640 1 --2- 172.21.15.38:0/3284326445 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7a9c15ee80 0x7f7a9c1f8e80 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f7a9400c3c0 tx=0x7f7a9400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:22.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:22.877+0000 7f7a937fe640 1 -- 172.21.15.38:0/3284326445 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a94017070 con 0x7f7a9c15ee80 2024-09-21T03:36:22.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:22.877+0000 7f7a937fe640 1 -- 172.21.15.38:0/3284326445 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7a940053e0 con 0x7f7a9c15ee80 2024-09-21T03:36:22.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:22.877+0000 7f7aa1313640 1 -- 172.21.15.38:0/3284326445 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a9c1f5670 con 0x7f7a9c15ee80 2024-09-21T03:36:22.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:22.877+0000 7f7a937fe640 1 -- 172.21.15.38:0/3284326445 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a94003850 con 0x7f7a9c15ee80 2024-09-21T03:36:22.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:22.877+0000 7f7aa1313640 1 -- 172.21.15.38:0/3284326445 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a9c1f5c30 con 0x7f7a9c15ee80 2024-09-21T03:36:22.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:22.878+0000 7f7aa1313640 1 -- 172.21.15.38:0/3284326445 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a9c0b6d30 con 0x7f7a9c15ee80 2024-09-21T03:36:22.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:22.879+0000 7f7a937fe640 1 -- 172.21.15.38:0/3284326445 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f7a9401a070 con 0x7f7a9c15ee80 2024-09-21T03:36:22.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:22.880+0000 7f7a937fe640 1 --2- 172.21.15.38:0/3284326445 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7a7c078bc0 0x7f7a7c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:22.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:22.880+0000 7f7a937fe640 1 -- 172.21.15.38:0/3284326445 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 6101+0+0 (secure 0 0 0) 0x7f7a9409a130 con 0x7f7a9c15ee80 2024-09-21T03:36:22.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:22.880+0000 7f7a9affd640 1 --2- 172.21.15.38:0/3284326445 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7a7c078bc0 0x7f7a7c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:22.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:22.881+0000 7f7a9affd640 1 --2- 172.21.15.38:0/3284326445 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7a7c078bc0 0x7f7a7c07b0d0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f7a8c034000 tx=0x7f7a8c004ab0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:22.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:22.885+0000 7f7a937fe640 1 -- 172.21.15.38:0/3284326445 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f7a9c1592d0 con 0x7f7a9c15ee80 2024-09-21T03:36:23.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:23.113+0000 7f7aa1313640 1 -- 172.21.15.38:0/3284326445 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f7a9c06f700 con 0x7f7a9c15ee80 2024-09-21T03:36:24.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:24.091+0000 7f7a937fe640 1 -- 172.21.15.38:0/3284326445 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v44) ==== 118+0+0 (secure 0 0 0) 0x7f7a94062470 con 0x7f7a9c15ee80 2024-09-21T03:36:24.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:24.094+0000 7f7aa1313640 1 -- 172.21.15.38:0/3284326445 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7a7c078bc0 msgr2=0x7f7a7c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:24.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:24.094+0000 7f7aa1313640 1 --2- 172.21.15.38:0/3284326445 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7a7c078bc0 0x7f7a7c07b0d0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f7a8c034000 tx=0x7f7a8c004ab0 comp rx=0 tx=0).stop 2024-09-21T03:36:24.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:24.094+0000 7f7aa1313640 1 -- 172.21.15.38:0/3284326445 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7a9c15ee80 msgr2=0x7f7a9c1f8e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:24.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:24.094+0000 7f7aa1313640 1 --2- 172.21.15.38:0/3284326445 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7a9c15ee80 0x7f7a9c1f8e80 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f7a9400c3c0 tx=0x7f7a9400c880 comp rx=0 tx=0).stop 2024-09-21T03:36:24.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:24.094+0000 7f7aa1313640 1 -- 172.21.15.38:0/3284326445 shutdown_connections 2024-09-21T03:36:24.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:24.094+0000 7f7aa1313640 1 --2- 172.21.15.38:0/3284326445 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7a9c15ee80 0x7f7a9c1f8e80 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:24.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:24.094+0000 7f7aa1313640 1 --2- 172.21.15.38:0/3284326445 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7a9c0d1120 0x7f7a9c0b4f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:24.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:24.094+0000 7f7aa1313640 1 --2- 172.21.15.38:0/3284326445 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7a7c078bc0 0x7f7a7c07b0d0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:24.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:24.094+0000 7f7aa1313640 1 --2- 172.21.15.38:0/3284326445 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7a9c0b9ad0 0x7f7a9c0b49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:24.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:24.094+0000 7f7aa1313640 1 -- 172.21.15.38:0/3284326445 >> 172.21.15.38:0/3284326445 conn(0x7f7a9c09d830 msgr2=0x7f7a9c0cae40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:24.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:24.094+0000 7f7aa1313640 1 -- 172.21.15.38:0/3284326445 shutdown_connections 2024-09-21T03:36:24.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:24.094+0000 7f7aa1313640 1 -- 172.21.15.38:0/3284326445 wait complete. 2024-09-21T03:36:24.110 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-21T03:36:24.110 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-21T03:36:24.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:24.236+0000 7f3fb66dc640 1 -- 172.21.15.38:0/1395776374 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3fb00ceb90 msgr2=0x7f3fb00cefa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:24.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:24.236+0000 7f3fb66dc640 1 --2- 172.21.15.38:0/1395776374 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3fb00ceb90 0x7f3fb00cefa0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f3fa8005fd0 tx=0x7f3fa802aa10 comp rx=0 tx=0).stop 2024-09-21T03:36:24.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:24.236+0000 7f3fb66dc640 1 -- 172.21.15.38:0/1395776374 shutdown_connections 2024-09-21T03:36:24.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:24.236+0000 7f3fb66dc640 1 --2- 172.21.15.38:0/1395776374 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3fb00ceb90 0x7f3fb00cefa0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:24.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:24.236+0000 7f3fb66dc640 1 -- 172.21.15.38:0/1395776374 >> 172.21.15.38:0/1395776374 conn(0x7f3fb009d830 msgr2=0x7f3fb00b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:24.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:24.236+0000 7f3fb66dc640 1 -- 172.21.15.38:0/1395776374 shutdown_connections 2024-09-21T03:36:24.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:24.236+0000 7f3fb66dc640 1 -- 172.21.15.38:0/1395776374 wait complete. 2024-09-21T03:36:24.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:24.238+0000 7f3fb66dc640 1 Processor -- start 2024-09-21T03:36:24.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:24.238+0000 7f3fb66dc640 1 -- start start 2024-09-21T03:36:24.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:24.239+0000 7f3fb66dc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3fb00ceb90 0x7f3fb014a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:24.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:24.239+0000 7f3fb66dc640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3fb00d0170 0x7f3fb01567c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:24.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:24.239+0000 7f3fb66dc640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3fb015ee80 0x7f3fb00b1190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:24.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:24.239+0000 7f3fb66dc640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f3fb00b6790 con 0x7f3fb00ceb90 2024-09-21T03:36:24.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:24.239+0000 7f3fb66dc640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f3fb00d14b0 con 0x7f3fb00d0170 2024-09-21T03:36:24.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:24.239+0000 7f3fb66dc640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f3fb00b6910 con 0x7f3fb015ee80 2024-09-21T03:36:24.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:24.239+0000 7f3faffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3fb00ceb90 0x7f3fb014a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:24.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:24.239+0000 7f3faffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3fb00ceb90 0x7f3fb014a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37280/0 (socket says 172.21.15.38:37280) 2024-09-21T03:36:24.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:24.239+0000 7f3faffff640 1 -- 172.21.15.38:0/975828098 learned_addr learned my addr 172.21.15.38:0/975828098 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:36:24.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:24.239+0000 7f3fb4c79640 1 --2- 172.21.15.38:0/975828098 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3fb015ee80 0x7f3fb00b1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:24.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:24.239+0000 7f3faf7fe640 1 --2- 172.21.15.38:0/975828098 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3fb00d0170 0x7f3fb01567c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:24.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:24.240+0000 7f3faffff640 1 -- 172.21.15.38:0/975828098 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3fb015ee80 msgr2=0x7f3fb00b1190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:24.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:24.240+0000 7f3faffff640 1 --2- 172.21.15.38:0/975828098 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3fb015ee80 0x7f3fb00b1190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:24.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:24.240+0000 7f3faffff640 1 -- 172.21.15.38:0/975828098 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3fb00d0170 msgr2=0x7f3fb01567c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:24.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:24.240+0000 7f3faffff640 1 --2- 172.21.15.38:0/975828098 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3fb00d0170 0x7f3fb01567c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:24.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:24.240+0000 7f3faffff640 1 -- 172.21.15.38:0/975828098 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fb014af30 con 0x7f3fb00ceb90 2024-09-21T03:36:24.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:24.240+0000 7f3fb4c79640 1 --2- 172.21.15.38:0/975828098 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3fb015ee80 0x7f3fb00b1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:36:24.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:24.240+0000 7f3faf7fe640 1 --2- 172.21.15.38:0/975828098 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3fb00d0170 0x7f3fb01567c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:36:24.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:24.240+0000 7f3faffff640 1 --2- 172.21.15.38:0/975828098 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3fb00ceb90 0x7f3fb014a9f0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f3fa000c3a0 tx=0x7f3fa000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:24.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:24.240+0000 7f3facff9640 1 -- 172.21.15.38:0/975828098 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fa0017070 con 0x7f3fb00ceb90 2024-09-21T03:36:24.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:24.240+0000 7f3facff9640 1 -- 172.21.15.38:0/975828098 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3fa00035f0 con 0x7f3fb00ceb90 2024-09-21T03:36:24.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:24.240+0000 7f3fb66dc640 1 -- 172.21.15.38:0/975828098 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fb00cd690 con 0x7f3fb00ceb90 2024-09-21T03:36:24.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:24.240+0000 7f3facff9640 1 -- 172.21.15.38:0/975828098 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fa0003a40 con 0x7f3fb00ceb90 2024-09-21T03:36:24.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:24.240+0000 7f3fb66dc640 1 -- 172.21.15.38:0/975828098 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fb014b440 con 0x7f3fb00ceb90 2024-09-21T03:36:24.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:24.241+0000 7f3fb66dc640 1 -- 172.21.15.38:0/975828098 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3fb00b6e90 con 0x7f3fb00ceb90 2024-09-21T03:36:24.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:24.242+0000 7f3facff9640 1 -- 172.21.15.38:0/975828098 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f3fa001a070 con 0x7f3fb00ceb90 2024-09-21T03:36:24.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:24.243+0000 7f3facff9640 1 --2- 172.21.15.38:0/975828098 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3f9c078bc0 0x7f3f9c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:24.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:24.243+0000 7f3facff9640 1 -- 172.21.15.38:0/975828098 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6101+0+0 (secure 0 0 0) 0x7f3fa009a0e0 con 0x7f3fb00ceb90 2024-09-21T03:36:24.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:24.243+0000 7f3faf7fe640 1 --2- 172.21.15.38:0/975828098 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3f9c078bc0 0x7f3f9c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:24.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:24.243+0000 7f3faf7fe640 1 --2- 172.21.15.38:0/975828098 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3f9c078bc0 0x7f3f9c07b0d0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f3fb006fde0 tx=0x7f3fa4009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:24.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:24.247+0000 7f3facff9640 1 -- 172.21.15.38:0/975828098 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f3fb01592d0 con 0x7f3fb00ceb90 2024-09-21T03:36:24.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:24.473+0000 7f3fb66dc640 1 -- 172.21.15.38:0/975828098 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f3fb0158080 con 0x7f3fb00ceb90 2024-09-21T03:36:25.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:25.097+0000 7f3facff9640 1 -- 172.21.15.38:0/975828098 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v45) ==== 122+0+0 (secure 0 0 0) 0x7f3fa0062410 con 0x7f3fb00ceb90 2024-09-21T03:36:25.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:25.101+0000 7f3fb66dc640 1 -- 172.21.15.38:0/975828098 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3f9c078bc0 msgr2=0x7f3f9c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:25.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:25.101+0000 7f3fb66dc640 1 --2- 172.21.15.38:0/975828098 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3f9c078bc0 0x7f3f9c07b0d0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f3fb006fde0 tx=0x7f3fa4009040 comp rx=0 tx=0).stop 2024-09-21T03:36:25.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:25.101+0000 7f3fb66dc640 1 -- 172.21.15.38:0/975828098 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3fb00ceb90 msgr2=0x7f3fb014a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:25.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:25.101+0000 7f3fb66dc640 1 --2- 172.21.15.38:0/975828098 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3fb00ceb90 0x7f3fb014a9f0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f3fa000c3a0 tx=0x7f3fa000c860 comp rx=0 tx=0).stop 2024-09-21T03:36:25.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:25.101+0000 7f3fb66dc640 1 -- 172.21.15.38:0/975828098 shutdown_connections 2024-09-21T03:36:25.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:25.101+0000 7f3fb66dc640 1 --2- 172.21.15.38:0/975828098 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3f9c078bc0 0x7f3f9c07b0d0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:25.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:25.101+0000 7f3fb66dc640 1 --2- 172.21.15.38:0/975828098 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3fb015ee80 0x7f3fb00b1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:25.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:25.101+0000 7f3fb66dc640 1 --2- 172.21.15.38:0/975828098 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3fb00d0170 0x7f3fb01567c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:25.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:25.101+0000 7f3fb66dc640 1 --2- 172.21.15.38:0/975828098 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3fb00ceb90 0x7f3fb014a9f0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:25.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:25.101+0000 7f3fb66dc640 1 -- 172.21.15.38:0/975828098 >> 172.21.15.38:0/975828098 conn(0x7f3fb009d830 msgr2=0x7f3fb0157340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:25.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:25.101+0000 7f3fb66dc640 1 -- 172.21.15.38:0/975828098 shutdown_connections 2024-09-21T03:36:25.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:25.101+0000 7f3fb66dc640 1 -- 172.21.15.38:0/975828098 wait complete. 2024-09-21T03:36:25.117 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-21T03:36:25.117 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:36:25.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:25.238+0000 7f4309a4d640 1 -- 172.21.15.38:0/1085523376 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4304162590 msgr2=0x7f43041629a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:25.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:25.238+0000 7f4309a4d640 1 --2- 172.21.15.38:0/1085523376 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4304162590 0x7f43041629a0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f430408d3b0 tx=0x7f42ec02a560 comp rx=0 tx=0).stop 2024-09-21T03:36:25.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:25.238+0000 7f4309a4d640 1 -- 172.21.15.38:0/1085523376 shutdown_connections 2024-09-21T03:36:25.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:25.238+0000 7f4309a4d640 1 --2- 172.21.15.38:0/1085523376 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f43040b2e60 0x7f43040b3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:25.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:25.238+0000 7f4309a4d640 1 --2- 172.21.15.38:0/1085523376 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4304162590 0x7f43041629a0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:25.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:25.238+0000 7f4309a4d640 1 --2- 172.21.15.38:0/1085523376 >> v2:172.21.15.195:3300/0 conn(0x7f43040cfe00 0x7f43040d2a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:25.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:25.238+0000 7f4309a4d640 1 -- 172.21.15.38:0/1085523376 >> 172.21.15.38:0/1085523376 conn(0x7f43040d4440 msgr2=0x7f43040d4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:25.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:25.238+0000 7f4309a4d640 1 -- 172.21.15.38:0/1085523376 shutdown_connections 2024-09-21T03:36:25.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:25.238+0000 7f4309a4d640 1 -- 172.21.15.38:0/1085523376 wait complete. 2024-09-21T03:36:25.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:25.240+0000 7f4309a4d640 1 Processor -- start 2024-09-21T03:36:25.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:25.240+0000 7f4309a4d640 1 -- start start 2024-09-21T03:36:25.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:25.241+0000 7f4309a4d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f43040b2e60 0x7f43041f9430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:25.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:25.242+0000 7f4309a4d640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f43040cfe00 0x7f43041f2910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:25.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:25.242+0000 7f4309a4d640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4304162590 0x7f4304160190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:25.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:25.242+0000 7f4309a4d640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f43040b37b0 con 0x7f43040b2e60 2024-09-21T03:36:25.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:25.242+0000 7f4309a4d640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f43040cf5a0 con 0x7f43040cfe00 2024-09-21T03:36:25.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:25.242+0000 7f4309a4d640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f43040b3930 con 0x7f4304162590 2024-09-21T03:36:25.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:25.242+0000 7f4302ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f43040b2e60 0x7f43041f9430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:25.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:25.242+0000 7f4302ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f43040b2e60 0x7f43041f9430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37288/0 (socket says 172.21.15.38:37288) 2024-09-21T03:36:25.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:25.242+0000 7f4302ffd640 1 -- 172.21.15.38:0/3257979403 learned_addr learned my addr 172.21.15.38:0/3257979403 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:36:25.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:25.242+0000 7f43027fc640 1 --2- 172.21.15.38:0/3257979403 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f43040cfe00 0x7f43041f2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:25.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:25.242+0000 7f43037fe640 1 --2- 172.21.15.38:0/3257979403 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4304162590 0x7f4304160190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:25.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:25.242+0000 7f4302ffd640 1 -- 172.21.15.38:0/3257979403 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4304162590 msgr2=0x7f4304160190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:25.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:25.242+0000 7f4302ffd640 1 --2- 172.21.15.38:0/3257979403 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4304162590 0x7f4304160190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:25.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:25.242+0000 7f4302ffd640 1 -- 172.21.15.38:0/3257979403 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f43040cfe00 msgr2=0x7f43041f2910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:25.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:25.242+0000 7f4302ffd640 1 --2- 172.21.15.38:0/3257979403 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f43040cfe00 0x7f43041f2910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:25.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:25.242+0000 7f4302ffd640 1 -- 172.21.15.38:0/3257979403 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43041f5b10 con 0x7f43040b2e60 2024-09-21T03:36:25.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:25.242+0000 7f4302ffd640 1 --2- 172.21.15.38:0/3257979403 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f43040b2e60 0x7f43041f9430 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f42f400cce0 tx=0x7f42f400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:25.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:25.242+0000 7f42fbfff640 1 -- 172.21.15.38:0/3257979403 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42f400f450 con 0x7f43040b2e60 2024-09-21T03:36:25.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:25.242+0000 7f42fbfff640 1 -- 172.21.15.38:0/3257979403 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f42f400f6e0 con 0x7f43040b2e60 2024-09-21T03:36:25.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:25.242+0000 7f4309a4d640 1 -- 172.21.15.38:0/3257979403 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43041f5df0 con 0x7f43040b2e60 2024-09-21T03:36:25.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:25.242+0000 7f42fbfff640 1 -- 172.21.15.38:0/3257979403 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42f400fb30 con 0x7f43040b2e60 2024-09-21T03:36:25.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:25.243+0000 7f4309a4d640 1 -- 172.21.15.38:0/3257979403 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43041f7ca0 con 0x7f43040b2e60 2024-09-21T03:36:25.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:25.244+0000 7f4309a4d640 1 -- 172.21.15.38:0/3257979403 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43040d2a20 con 0x7f43040b2e60 2024-09-21T03:36:25.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:25.244+0000 7f42fbfff640 1 -- 172.21.15.38:0/3257979403 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f42f400fcd0 con 0x7f43040b2e60 2024-09-21T03:36:25.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:25.245+0000 7f42fbfff640 1 --2- 172.21.15.38:0/3257979403 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f42e4078b90 0x7f42e407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:25.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:25.245+0000 7f42fbfff640 1 -- 172.21.15.38:0/3257979403 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6101+0+0 (secure 0 0 0) 0x7f42f409b930 con 0x7f43040b2e60 2024-09-21T03:36:25.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:25.245+0000 7f43027fc640 1 --2- 172.21.15.38:0/3257979403 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f42e4078b90 0x7f42e407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:25.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:25.246+0000 7f43027fc640 1 --2- 172.21.15.38:0/3257979403 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f42e4078b90 0x7f42e407b0a0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f42ec033000 tx=0x7f42ec004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:25.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:25.250+0000 7f42fbfff640 1 -- 172.21.15.38:0/3257979403 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f42f4014030 con 0x7f43040b2e60 2024-09-21T03:36:25.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:25.476+0000 7f4309a4d640 1 -- 172.21.15.38:0/3257979403 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f43041f7ed0 con 0x7f43040b2e60 2024-09-21T03:36:25.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:25.477+0000 7f42fbfff640 1 -- 172.21.15.38:0/3257979403 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) ==== 94+0+5230 (secure 0 0 0) 0x7f42f4063580 con 0x7f43040b2e60 2024-09-21T03:36:25.479 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:36:25.479 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":45,"btime":"2024-09-21T03:36:25:092622+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5432,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/1042764291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":1042764291},{"type":"v1","addr":"172.21.15.195:6837","nonce":1042764291}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":5728,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/3035019482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6833","nonce":3035019482},{"type":"v1","addr":"172.21.15.38:6835","nonce":3035019482}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":5731,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/2536177898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":2536177898},{"type":"v1","addr":"172.21.15.195:6836","nonce":2536177898}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":45,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:36:21.442724+0000","modified":"2024-09-21T03:36:24.474620+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5991},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5991":{"gid":5991,"name":"a","rank":0,"incarnation":41,"state":"up:active","state_seq":3,"addr":"172.21.15.38:6834/1353335665","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":1353335665},{"type":"v1","addr":"172.21.15.38:6834","nonce":1353335665}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5991,"qdb_cluster":[5991]},"id":4}]} 2024-09-21T03:36:25.479 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 45 2024-09-21T03:36:25.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:25.479+0000 7f4309a4d640 1 -- 172.21.15.38:0/3257979403 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f42e4078b90 msgr2=0x7f42e407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:25.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:25.479+0000 7f4309a4d640 1 --2- 172.21.15.38:0/3257979403 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f42e4078b90 0x7f42e407b0a0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f42ec033000 tx=0x7f42ec004730 comp rx=0 tx=0).stop 2024-09-21T03:36:25.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:25.479+0000 7f4309a4d640 1 -- 172.21.15.38:0/3257979403 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f43040b2e60 msgr2=0x7f43041f9430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:25.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:25.479+0000 7f4309a4d640 1 --2- 172.21.15.38:0/3257979403 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f43040b2e60 0x7f43041f9430 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f42f400cce0 tx=0x7f42f400a620 comp rx=0 tx=0).stop 2024-09-21T03:36:25.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:25.479+0000 7f4309a4d640 1 -- 172.21.15.38:0/3257979403 shutdown_connections 2024-09-21T03:36:25.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:25.479+0000 7f4309a4d640 1 --2- 172.21.15.38:0/3257979403 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f42e4078b90 0x7f42e407b0a0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:25.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:25.479+0000 7f4309a4d640 1 --2- 172.21.15.38:0/3257979403 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4304162590 0x7f4304160190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:25.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:25.479+0000 7f4309a4d640 1 --2- 172.21.15.38:0/3257979403 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f43040cfe00 0x7f43041f2910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:25.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:25.479+0000 7f4309a4d640 1 --2- 172.21.15.38:0/3257979403 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f43040b2e60 0x7f43041f9430 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:25.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:25.479+0000 7f4309a4d640 1 -- 172.21.15.38:0/3257979403 >> 172.21.15.38:0/3257979403 conn(0x7f43040d4440 msgr2=0x7f43040d4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:25.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:25.479+0000 7f4309a4d640 1 -- 172.21.15.38:0/3257979403 shutdown_connections 2024-09-21T03:36:25.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:25.479+0000 7f4309a4d640 1 -- 172.21.15.38:0/3257979403 wait complete. 2024-09-21T03:36:25.493 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T03:36:25.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:25.607+0000 7f6d1fab2640 1 -- 172.21.15.38:0/3416776763 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6d180b2e40 msgr2=0x7f6d180b3250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:25.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:25.607+0000 7f6d1fab2640 1 --2- 172.21.15.38:0/3416776763 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6d180b2e40 0x7f6d180b3250 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f6d180b7460 tx=0x7f6d1002a4d0 comp rx=0 tx=0).stop 2024-09-21T03:36:25.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:25.607+0000 7f6d1fab2640 1 -- 172.21.15.38:0/3416776763 shutdown_connections 2024-09-21T03:36:25.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:25.607+0000 7f6d1fab2640 1 --2- 172.21.15.38:0/3416776763 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6d180b2e40 0x7f6d180b3250 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:25.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:25.607+0000 7f6d1fab2640 1 --2- 172.21.15.38:0/3416776763 >> v2:172.21.15.195:3300/0 conn(0x7f6d18162280 0x7f6d18162690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:25.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:25.607+0000 7f6d1fab2640 1 -- 172.21.15.38:0/3416776763 >> 172.21.15.38:0/3416776763 conn(0x7f6d180b6b20 msgr2=0x7f6d180d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:25.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:25.607+0000 7f6d1fab2640 1 -- 172.21.15.38:0/3416776763 shutdown_connections 2024-09-21T03:36:25.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:25.607+0000 7f6d1fab2640 1 -- 172.21.15.38:0/3416776763 wait complete. 2024-09-21T03:36:25.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:25.609+0000 7f6d1fab2640 1 Processor -- start 2024-09-21T03:36:25.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:25.609+0000 7f6d1fab2640 1 -- start start 2024-09-21T03:36:25.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:25.610+0000 7f6d1fab2640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6d180b2e40 0x7f6d180ceee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:25.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:25.610+0000 7f6d1fab2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6d18162280 0x7f6d1814e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:25.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:25.610+0000 7f6d1fab2640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6d1814ec40 0x7f6d1814f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:25.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:25.610+0000 7f6d1fab2640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f6d1815e4b0 con 0x7f6d18162280 2024-09-21T03:36:25.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:25.610+0000 7f6d1fab2640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f6d180b39a0 con 0x7f6d180b2e40 2024-09-21T03:36:25.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:25.610+0000 7f6d1fab2640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f6d1815e630 con 0x7f6d1814ec40 2024-09-21T03:36:25.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:25.610+0000 7f6d1d04d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6d18162280 0x7f6d1814e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:25.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:25.610+0000 7f6d1d04d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6d18162280 0x7f6d1814e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37306/0 (socket says 172.21.15.38:37306) 2024-09-21T03:36:25.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:25.610+0000 7f6d1d04d640 1 -- 172.21.15.38:0/1161869948 learned_addr learned my addr 172.21.15.38:0/1161869948 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:36:25.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:25.610+0000 7f6d1e04f640 1 --2- 172.21.15.38:0/1161869948 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6d1814ec40 0x7f6d1814f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:25.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:25.610+0000 7f6d1d84e640 1 --2- 172.21.15.38:0/1161869948 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6d180b2e40 0x7f6d180ceee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:25.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:25.611+0000 7f6d1d04d640 1 -- 172.21.15.38:0/1161869948 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6d1814ec40 msgr2=0x7f6d1814f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:25.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:25.611+0000 7f6d1d04d640 1 --2- 172.21.15.38:0/1161869948 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6d1814ec40 0x7f6d1814f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:25.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:25.611+0000 7f6d1d04d640 1 -- 172.21.15.38:0/1161869948 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6d180b2e40 msgr2=0x7f6d180ceee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:25.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:25.611+0000 7f6d1d04d640 1 --2- 172.21.15.38:0/1161869948 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6d180b2e40 0x7f6d180ceee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:25.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:25.611+0000 7f6d1d04d640 1 -- 172.21.15.38:0/1161869948 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d180aba20 con 0x7f6d18162280 2024-09-21T03:36:25.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:25.611+0000 7f6d1e04f640 1 --2- 172.21.15.38:0/1161869948 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6d1814ec40 0x7f6d1814f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:36:25.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:25.611+0000 7f6d1d84e640 1 --2- 172.21.15.38:0/1161869948 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6d180b2e40 0x7f6d180ceee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:36:25.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:25.611+0000 7f6d1d04d640 1 --2- 172.21.15.38:0/1161869948 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6d18162280 0x7f6d1814e700 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f6d0c00d6d0 tx=0x7f6d0c00db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:25.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:25.611+0000 7f6d167fc640 1 -- 172.21.15.38:0/1161869948 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d0c017070 con 0x7f6d18162280 2024-09-21T03:36:25.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:25.611+0000 7f6d167fc640 1 -- 172.21.15.38:0/1161869948 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6d0c005330 con 0x7f6d18162280 2024-09-21T03:36:25.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:25.611+0000 7f6d1fab2640 1 -- 172.21.15.38:0/1161869948 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d180abd00 con 0x7f6d18162280 2024-09-21T03:36:25.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:25.611+0000 7f6d167fc640 1 -- 172.21.15.38:0/1161869948 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d0c009820 con 0x7f6d18162280 2024-09-21T03:36:25.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:25.611+0000 7f6d1fab2640 1 -- 172.21.15.38:0/1161869948 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d1814f600 con 0x7f6d18162280 2024-09-21T03:36:25.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:25.612+0000 7f6d1fab2640 1 -- 172.21.15.38:0/1161869948 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d18162f00 con 0x7f6d18162280 2024-09-21T03:36:25.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:25.613+0000 7f6d167fc640 1 -- 172.21.15.38:0/1161869948 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f6d0c009bd0 con 0x7f6d18162280 2024-09-21T03:36:25.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:25.613+0000 7f6d167fc640 1 --2- 172.21.15.38:0/1161869948 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6d04078b90 0x7f6d0407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:25.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:25.613+0000 7f6d167fc640 1 -- 172.21.15.38:0/1161869948 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6101+0+0 (secure 0 0 0) 0x7f6d0c09b1e0 con 0x7f6d18162280 2024-09-21T03:36:25.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:25.614+0000 7f6d1d84e640 1 --2- 172.21.15.38:0/1161869948 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6d04078b90 0x7f6d0407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:25.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:25.614+0000 7f6d1d84e640 1 --2- 172.21.15.38:0/1161869948 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6d04078b90 0x7f6d0407b0a0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f6d08009fc0 tx=0x7f6d08009550 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:25.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:25.618+0000 7f6d167fc640 1 -- 172.21.15.38:0/1161869948 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f6d0c063480 con 0x7f6d18162280 2024-09-21T03:36:25.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:25.806+0000 7f6d1fab2640 1 -- 172.21.15.38:0/1161869948 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6d180b8fd0 con 0x7f6d18162280 2024-09-21T03:36:25.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:25.807+0000 7f6d167fc640 1 -- 172.21.15.38:0/1161869948 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v67) ==== 74+0+17016 (secure 0 0 0) 0x7f6d0c062d80 con 0x7f6d18162280 2024-09-21T03:36:25.809 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:36:25.809 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":67,"fsid":"9d50d66f-656b-4fe2-8e3b-fe53b79c4d43","created":"2024-09-21T03:33:10.677865+0000","modified":"2024-09-21T03:36:23.067468+0000","last_up_change":"2024-09-21T03:33:24.023421+0000","last_in_change":"2024-09-21T03:33:14.296193+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T03:33:22.992573+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-09-21T03:36:19.109965+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-09-21T03:36:20.392299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dbab183f-7e63-48b8-960a-ae55fd5bf565","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6801","nonce":2381639433}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6803","nonce":2381639433}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6807","nonce":2381639433}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6805","nonce":2381639433}]},"public_addr":"172.21.15.38:6801/2381639433","cluster_addr":"172.21.15.38:6803/2381639433","heartbeat_back_addr":"172.21.15.38:6807/2381639433","heartbeat_front_addr":"172.21.15.38:6805/2381639433","state":["exists","up"]},{"osd":1,"uuid":"fa1c0e8a-ebc1-4359-8146-bd5342abbc16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6817","nonce":1491818732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6819","nonce":1491818732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6823","nonce":1491818732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6821","nonce":1491818732}]},"public_addr":"172.21.15.38:6817/1491818732","cluster_addr":"172.21.15.38:6819/1491818732","heartbeat_back_addr":"172.21.15.38:6823/1491818732","heartbeat_front_addr":"172.21.15.38:6821/1491818732","state":["exists","up"]},{"osd":2,"uuid":"2ed3cd2d-385d-4fc6-9869-214552a84803","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6825","nonce":2577858902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6827","nonce":2577858902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6831","nonce":2577858902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6829","nonce":2577858902}]},"public_addr":"172.21.15.38:6825/2577858902","cluster_addr":"172.21.15.38:6827/2577858902","heartbeat_back_addr":"172.21.15.38:6831/2577858902","heartbeat_front_addr":"172.21.15.38:6829/2577858902","state":["exists","up"]},{"osd":3,"uuid":"62d1d95e-919d-4250-be71-1d34cfa70dbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6809","nonce":1286810294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6811","nonce":1286810294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6815","nonce":1286810294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6813","nonce":1286810294}]},"public_addr":"172.21.15.38:6809/1286810294","cluster_addr":"172.21.15.38:6811/1286810294","heartbeat_back_addr":"172.21.15.38:6815/1286810294","heartbeat_front_addr":"172.21.15.38:6813/1286810294","state":["exists","up"]},{"osd":4,"uuid":"7c9f9314-99d7-4d73-8392-37809a9f54b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6803","nonce":3401837571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6805","nonce":3401837571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6809","nonce":3401837571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6807","nonce":3401837571}]},"public_addr":"172.21.15.195:6803/3401837571","cluster_addr":"172.21.15.195:6805/3401837571","heartbeat_back_addr":"172.21.15.195:6809/3401837571","heartbeat_front_addr":"172.21.15.195:6807/3401837571","state":["exists","up"]},{"osd":5,"uuid":"bde4fe9f-8414-4bba-bf4a-43973d1976bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6811","nonce":2080960180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6813","nonce":2080960180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6817","nonce":2080960180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6815","nonce":2080960180}]},"public_addr":"172.21.15.195:6811/2080960180","cluster_addr":"172.21.15.195:6813/2080960180","heartbeat_back_addr":"172.21.15.195:6817/2080960180","heartbeat_front_addr":"172.21.15.195:6815/2080960180","state":["exists","up"]},{"osd":6,"uuid":"6ee6d7f4-1a5f-4a4c-ac16-48fe05f6a002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6827","nonce":4055825675}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6829","nonce":4055825675}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6833","nonce":4055825675}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6831","nonce":4055825675}]},"public_addr":"172.21.15.195:6827/4055825675","cluster_addr":"172.21.15.195:6829/4055825675","heartbeat_back_addr":"172.21.15.195:6833/4055825675","heartbeat_front_addr":"172.21.15.195:6831/4055825675","state":["exists","up"]},{"osd":7,"uuid":"efeb5f51-329a-4484-8290-92e7d3503d05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6819","nonce":1606106342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6821","nonce":1606106342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6825","nonce":1606106342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6823","nonce":1606106342}]},"public_addr":"172.21.15.195:6819/1606106342","cluster_addr":"172.21.15.195:6821/1606106342","heartbeat_back_addr":"172.21.15.195:6825/1606106342","heartbeat_front_addr":"172.21.15.195:6823/1606106342","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.901343+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:21.115003+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.901623+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.835238+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.132199+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:19.817560+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.829690+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:21.087300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T03:36:25.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:25.809+0000 7f6d1fab2640 1 -- 172.21.15.38:0/1161869948 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6d04078b90 msgr2=0x7f6d0407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:25.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:25.809+0000 7f6d1fab2640 1 --2- 172.21.15.38:0/1161869948 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6d04078b90 0x7f6d0407b0a0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f6d08009fc0 tx=0x7f6d08009550 comp rx=0 tx=0).stop 2024-09-21T03:36:25.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:25.809+0000 7f6d1fab2640 1 -- 172.21.15.38:0/1161869948 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6d18162280 msgr2=0x7f6d1814e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:25.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:25.809+0000 7f6d1fab2640 1 --2- 172.21.15.38:0/1161869948 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6d18162280 0x7f6d1814e700 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f6d0c00d6d0 tx=0x7f6d0c00db90 comp rx=0 tx=0).stop 2024-09-21T03:36:25.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:25.809+0000 7f6d1fab2640 1 -- 172.21.15.38:0/1161869948 shutdown_connections 2024-09-21T03:36:25.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:25.809+0000 7f6d1fab2640 1 --2- 172.21.15.38:0/1161869948 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6d04078b90 0x7f6d0407b0a0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:25.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:25.809+0000 7f6d1fab2640 1 --2- 172.21.15.38:0/1161869948 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6d1814ec40 0x7f6d1814f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:25.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:25.809+0000 7f6d1fab2640 1 --2- 172.21.15.38:0/1161869948 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6d18162280 0x7f6d1814e700 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:25.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:25.809+0000 7f6d1fab2640 1 --2- 172.21.15.38:0/1161869948 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6d180b2e40 0x7f6d180ceee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:25.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:25.809+0000 7f6d1fab2640 1 -- 172.21.15.38:0/1161869948 >> 172.21.15.38:0/1161869948 conn(0x7f6d180b6b20 msgr2=0x7f6d180b3b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:25.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:25.809+0000 7f6d1fab2640 1 -- 172.21.15.38:0/1161869948 shutdown_connections 2024-09-21T03:36:25.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:25.809+0000 7f6d1fab2640 1 -- 172.21.15.38:0/1161869948 wait complete. 2024-09-21T03:36:25.823 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-21T03:36:25.823 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T03:36:25.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:25.938+0000 7f47689c3640 1 -- 172.21.15.38:0/1275729771 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47500051a0 msgr2=0x7f475000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:25.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:25.938+0000 7f47689c3640 1 --2- 172.21.15.38:0/1275729771 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47500051a0 0x7f475000b9c0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f4754005fd0 tx=0x7f4750005830 comp rx=0 tx=0).stop 2024-09-21T03:36:25.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:25.938+0000 7f4762d9d640 1 -- 172.21.15.38:0/1275729771 reap_dead start 2024-09-21T03:36:25.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:25.938+0000 7f47689c3640 1 -- 172.21.15.38:0/1275729771 shutdown_connections 2024-09-21T03:36:25.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:25.938+0000 7f47689c3640 1 -- 172.21.15.38:0/1275729771 >> 172.21.15.38:0/1275729771 conn(0x7f47640cef40 msgr2=0x7f47640cf350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:25.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:25.938+0000 7f47689c3640 1 -- 172.21.15.38:0/1275729771 shutdown_connections 2024-09-21T03:36:25.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:25.938+0000 7f47689c3640 1 -- 172.21.15.38:0/1275729771 wait complete. 2024-09-21T03:36:25.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:25.940+0000 7f47689c3640 1 Processor -- start 2024-09-21T03:36:25.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:25.940+0000 7f47689c3640 1 -- start start 2024-09-21T03:36:25.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:25.941+0000 7f47689c3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47500051a0 0x7f47641f6710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:25.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:25.941+0000 7f47689c3640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f476415ac20 0x7f47641f3120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:25.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:25.941+0000 7f47689c3640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f476414e620 0x7f47641561b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:25.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:25.941+0000 7f47689c3640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f4764165300 con 0x7f47500051a0 2024-09-21T03:36:25.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:25.941+0000 7f47689c3640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f47640cd250 con 0x7f476415ac20 2024-09-21T03:36:25.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:25.941+0000 7f47689c3640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f4764165480 con 0x7f476414e620 2024-09-21T03:36:25.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:25.941+0000 7f476259c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47500051a0 0x7f47641f6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:25.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:25.941+0000 7f476259c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47500051a0 0x7f47641f6710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37310/0 (socket says 172.21.15.38:37310) 2024-09-21T03:36:25.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:25.941+0000 7f476259c640 1 -- 172.21.15.38:0/250437447 learned_addr learned my addr 172.21.15.38:0/250437447 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:36:25.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:25.942+0000 7f4762d9d640 1 --2- 172.21.15.38:0/250437447 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f476414e620 0x7f47641561b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:25.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:25.942+0000 7f4761d9b640 1 --2- 172.21.15.38:0/250437447 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f476415ac20 0x7f47641f3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:25.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:25.942+0000 7f476259c640 1 -- 172.21.15.38:0/250437447 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f476414e620 msgr2=0x7f47641561b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:25.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:25.942+0000 7f476259c640 1 --2- 172.21.15.38:0/250437447 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f476414e620 0x7f47641561b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:25.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:25.942+0000 7f476259c640 1 -- 172.21.15.38:0/250437447 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f476415ac20 msgr2=0x7f47641f3120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:25.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:25.942+0000 7f476259c640 1 --2- 172.21.15.38:0/250437447 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f476415ac20 0x7f47641f3120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:25.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:25.942+0000 7f476259c640 1 -- 172.21.15.38:0/250437447 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47641f6ed0 con 0x7f47500051a0 2024-09-21T03:36:25.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:25.942+0000 7f476259c640 1 --2- 172.21.15.38:0/250437447 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47500051a0 0x7f47641f6710 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f475c003e70 tx=0x7f475c001800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:25.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:25.942+0000 7f475affd640 1 -- 172.21.15.38:0/250437447 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f475c011700 con 0x7f47500051a0 2024-09-21T03:36:25.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:25.942+0000 7f475affd640 1 -- 172.21.15.38:0/250437447 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f475c016020 con 0x7f47500051a0 2024-09-21T03:36:25.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:25.942+0000 7f47689c3640 1 -- 172.21.15.38:0/250437447 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47641f71b0 con 0x7f47500051a0 2024-09-21T03:36:25.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:25.942+0000 7f475affd640 1 -- 172.21.15.38:0/250437447 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f475c011b60 con 0x7f47500051a0 2024-09-21T03:36:25.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:25.942+0000 7f47689c3640 1 -- 172.21.15.38:0/250437447 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f476415b060 con 0x7f47500051a0 2024-09-21T03:36:25.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:25.943+0000 7f47689c3640 1 -- 172.21.15.38:0/250437447 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4764158410 con 0x7f47500051a0 2024-09-21T03:36:25.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:25.944+0000 7f475affd640 1 -- 172.21.15.38:0/250437447 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f475c011d00 con 0x7f47500051a0 2024-09-21T03:36:25.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:25.945+0000 7f475affd640 1 --2- 172.21.15.38:0/250437447 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4744078b90 0x7f474407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:25.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:25.945+0000 7f475affd640 1 -- 172.21.15.38:0/250437447 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6101+0+0 (secure 0 0 0) 0x7f475c09bda0 con 0x7f47500051a0 2024-09-21T03:36:25.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:25.945+0000 7f4761d9b640 1 --2- 172.21.15.38:0/250437447 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4744078b90 0x7f474407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:25.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:25.945+0000 7f4761d9b640 1 --2- 172.21.15.38:0/250437447 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4744078b90 0x7f474407b0a0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f474c001050 tx=0x7f474c001110 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:25.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:25.949+0000 7f475affd640 1 -- 172.21.15.38:0/250437447 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f475c020020 con 0x7f47500051a0 2024-09-21T03:36:26.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:26.132+0000 7f47689c3640 1 -- 172.21.15.38:0/250437447 --> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f47640cc930 con 0x7f4744078b90 2024-09-21T03:36:26.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:26.137+0000 7f475affd640 1 -- 172.21.15.38:0/250437447 <== mgr.4117 v2:172.21.15.195:6800/1631901344 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+249897 (secure 0 0 0) 0x7f47640cc930 con 0x7f4744078b90 2024-09-21T03:36:26.141 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:36:26.143 INFO:teuthology.orchestra.run.smithi038.stderr:dumped all 2024-09-21T03:36:26.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:26.140+0000 7f47689c3640 1 -- 172.21.15.38:0/250437447 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4744078b90 msgr2=0x7f474407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:26.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:26.140+0000 7f47689c3640 1 --2- 172.21.15.38:0/250437447 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4744078b90 0x7f474407b0a0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f474c001050 tx=0x7f474c001110 comp rx=0 tx=0).stop 2024-09-21T03:36:26.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:26.140+0000 7f47689c3640 1 -- 172.21.15.38:0/250437447 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47500051a0 msgr2=0x7f47641f6710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:26.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:26.140+0000 7f47689c3640 1 --2- 172.21.15.38:0/250437447 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47500051a0 0x7f47641f6710 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f475c003e70 tx=0x7f475c001800 comp rx=0 tx=0).stop 2024-09-21T03:36:26.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:26.140+0000 7f47689c3640 1 -- 172.21.15.38:0/250437447 shutdown_connections 2024-09-21T03:36:26.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:26.140+0000 7f47689c3640 1 --2- 172.21.15.38:0/250437447 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4744078b90 0x7f474407b0a0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:26.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:26.140+0000 7f47689c3640 1 --2- 172.21.15.38:0/250437447 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f476414e620 0x7f47641561b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:26.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:26.140+0000 7f47689c3640 1 --2- 172.21.15.38:0/250437447 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f476415ac20 0x7f47641f3120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:26.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:26.140+0000 7f47689c3640 1 --2- 172.21.15.38:0/250437447 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47500051a0 0x7f47641f6710 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:26.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:26.140+0000 7f47689c3640 1 -- 172.21.15.38:0/250437447 >> 172.21.15.38:0/250437447 conn(0x7f47640cef40 msgr2=0x7f47641527a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:26.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:26.140+0000 7f47689c3640 1 -- 172.21.15.38:0/250437447 shutdown_connections 2024-09-21T03:36:26.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:26.140+0000 7f47689c3640 1 -- 172.21.15.38:0/250437447 wait complete. 2024-09-21T03:36:26.154 INFO:teuthology.orchestra.run.smithi038.stdout:{"pg_ready":true,"pg_map":{"version":151,"stamp":"2024-09-21T03:36:25.023385+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591166,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":123,"num_write_kb":1639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":91,"ondisk_log_size":91,"up":174,"acting":174,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":117,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":273532,"kb_used_data":7352,"kb_used_omap":12,"kb_used_meta":265971,"kb_avail":754701188,"statfs":{"total":773094113280,"available":772814016512,"internally_reserved":0,"allocated":7528448,"data_stored":4031012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12748,"internal_metadata":272354868},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":798,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.985644"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-21T03:36:22.088822+0000","last_change":"2024-09-21T03:36:22.088324+0000","last_active":"2024-09-21T03:36:22.088822+0000","last_peered":"2024-09-21T03:36:22.088822+0000","last_clean":"2024-09-21T03:36:22.088822+0000","last_became_active":"2024-09-21T03:36:22.088010+0000","last_became_peered":"2024-09-21T03:36:22.088010+0000","last_unstale":"2024-09-21T03:36:22.088822+0000","last_undegraded":"2024-09-21T03:36:22.088822+0000","last_fullsized":"2024-09-21T03:36:22.088822+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:17:56.797134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-21T03:36:21.054741+0000","last_change":"2024-09-21T03:36:21.054741+0000","last_active":"2024-09-21T03:36:21.036908+0000","last_peered":"2024-09-21T03:36:21.036908+0000","last_clean":"2024-09-21T03:36:21.036908+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:36:21.054741+0000","last_undegraded":"2024-09-21T03:36:21.054741+0000","last_fullsized":"2024-09-21T03:36:21.054741+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.086324+0000","last_change":"2024-09-21T03:36:22.071184+0000","last_active":"2024-09-21T03:36:23.086324+0000","last_peered":"2024-09-21T03:36:23.086324+0000","last_clean":"2024-09-21T03:36:23.086324+0000","last_became_active":"2024-09-21T03:36:22.070772+0000","last_became_peered":"2024-09-21T03:36:22.070772+0000","last_unstale":"2024-09-21T03:36:23.086324+0000","last_undegraded":"2024-09-21T03:36:23.086324+0000","last_fullsized":"2024-09-21T03:36:23.086324+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:17:47.479714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.661468+0000","last_change":"2024-09-21T03:36:22.070400+0000","last_active":"2024-09-21T03:36:23.661468+0000","last_peered":"2024-09-21T03:36:23.661468+0000","last_clean":"2024-09-21T03:36:23.661468+0000","last_became_active":"2024-09-21T03:36:22.068959+0000","last_became_peered":"2024-09-21T03:36:22.068959+0000","last_unstale":"2024-09-21T03:36:23.661468+0000","last_undegraded":"2024-09-21T03:36:23.661468+0000","last_fullsized":"2024-09-21T03:36:23.661468+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:30:41.436799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.086049+0000","last_change":"2024-09-21T03:36:22.081653+0000","last_active":"2024-09-21T03:36:23.086049+0000","last_peered":"2024-09-21T03:36:23.086049+0000","last_clean":"2024-09-21T03:36:23.086049+0000","last_became_active":"2024-09-21T03:36:22.081343+0000","last_became_peered":"2024-09-21T03:36:22.081343+0000","last_unstale":"2024-09-21T03:36:23.086049+0000","last_undegraded":"2024-09-21T03:36:23.086049+0000","last_fullsized":"2024-09-21T03:36:23.086049+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:37:46.632334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-21T03:36:21.055099+0000","last_change":"2024-09-21T03:36:21.055099+0000","last_active":"2024-09-21T03:36:21.036908+0000","last_peered":"2024-09-21T03:36:21.036908+0000","last_clean":"2024-09-21T03:36:21.036908+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:36:21.055099+0000","last_undegraded":"2024-09-21T03:36:21.055099+0000","last_fullsized":"2024-09-21T03:36:21.055099+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-21T03:36:22.075723+0000","last_change":"2024-09-21T03:36:22.074034+0000","last_active":"2024-09-21T03:36:22.075723+0000","last_peered":"2024-09-21T03:36:22.075723+0000","last_clean":"2024-09-21T03:36:22.075723+0000","last_became_active":"2024-09-21T03:36:22.073682+0000","last_became_peered":"2024-09-21T03:36:22.073682+0000","last_unstale":"2024-09-21T03:36:22.075723+0000","last_undegraded":"2024-09-21T03:36:22.075723+0000","last_fullsized":"2024-09-21T03:36:22.075723+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:25:30.522013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.085192+0000","last_change":"2024-09-21T03:36:22.080482+0000","last_active":"2024-09-21T03:36:23.085192+0000","last_peered":"2024-09-21T03:36:23.085192+0000","last_clean":"2024-09-21T03:36:23.085192+0000","last_became_active":"2024-09-21T03:36:22.080177+0000","last_became_peered":"2024-09-21T03:36:22.080177+0000","last_unstale":"2024-09-21T03:36:23.085192+0000","last_undegraded":"2024-09-21T03:36:23.085192+0000","last_fullsized":"2024-09-21T03:36:23.085192+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:56:46.927190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.661540+0000","last_change":"2024-09-21T03:36:22.077361+0000","last_active":"2024-09-21T03:36:23.661540+0000","last_peered":"2024-09-21T03:36:23.661540+0000","last_clean":"2024-09-21T03:36:23.661540+0000","last_became_active":"2024-09-21T03:36:22.077050+0000","last_became_peered":"2024-09-21T03:36:22.077050+0000","last_unstale":"2024-09-21T03:36:23.661540+0000","last_undegraded":"2024-09-21T03:36:23.661540+0000","last_fullsized":"2024-09-21T03:36:23.661540+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:30:24.348269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T03:36:21.036908+0000","last_change":"2024-09-21T03:36:21.036908+0000","last_active":"2024-09-21T03:36:21.036908+0000","last_peered":"2024-09-21T03:36:21.036908+0000","last_clean":"2024-09-21T03:36:21.036908+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:36:21.036908+0000","last_undegraded":"2024-09-21T03:36:21.036908+0000","last_fullsized":"2024-09-21T03:36:21.036908+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.661250+0000","last_change":"2024-09-21T03:36:22.072761+0000","last_active":"2024-09-21T03:36:23.661250+0000","last_peered":"2024-09-21T03:36:23.661250+0000","last_clean":"2024-09-21T03:36:23.661250+0000","last_became_active":"2024-09-21T03:36:22.072450+0000","last_became_peered":"2024-09-21T03:36:22.072450+0000","last_unstale":"2024-09-21T03:36:23.661250+0000","last_undegraded":"2024-09-21T03:36:23.661250+0000","last_fullsized":"2024-09-21T03:36:23.661250+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:04:45.989423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.661340+0000","last_change":"2024-09-21T03:36:22.086026+0000","last_active":"2024-09-21T03:36:23.661340+0000","last_peered":"2024-09-21T03:36:23.661340+0000","last_clean":"2024-09-21T03:36:23.661340+0000","last_became_active":"2024-09-21T03:36:22.085717+0000","last_became_peered":"2024-09-21T03:36:22.085717+0000","last_unstale":"2024-09-21T03:36:23.661340+0000","last_undegraded":"2024-09-21T03:36:23.661340+0000","last_fullsized":"2024-09-21T03:36:23.661340+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:47:51.103846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.660593+0000","last_change":"2024-09-21T03:36:22.090483+0000","last_active":"2024-09-21T03:36:23.660593+0000","last_peered":"2024-09-21T03:36:23.660593+0000","last_clean":"2024-09-21T03:36:23.660593+0000","last_became_active":"2024-09-21T03:36:22.090124+0000","last_became_peered":"2024-09-21T03:36:22.090124+0000","last_unstale":"2024-09-21T03:36:23.660593+0000","last_undegraded":"2024-09-21T03:36:23.660593+0000","last_fullsized":"2024-09-21T03:36:23.660593+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:52:19.716709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.660532+0000","last_change":"2024-09-21T03:36:22.080680+0000","last_active":"2024-09-21T03:36:23.660532+0000","last_peered":"2024-09-21T03:36:23.660532+0000","last_clean":"2024-09-21T03:36:23.660532+0000","last_became_active":"2024-09-21T03:36:22.080377+0000","last_became_peered":"2024-09-21T03:36:22.080377+0000","last_unstale":"2024-09-21T03:36:23.660532+0000","last_undegraded":"2024-09-21T03:36:23.660532+0000","last_fullsized":"2024-09-21T03:36:23.660532+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:54:33.394204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T03:36:21.036908+0000","last_change":"2024-09-21T03:36:21.036908+0000","last_active":"2024-09-21T03:36:21.036908+0000","last_peered":"2024-09-21T03:36:21.036908+0000","last_clean":"2024-09-21T03:36:21.036908+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:36:21.036908+0000","last_undegraded":"2024-09-21T03:36:21.036908+0000","last_fullsized":"2024-09-21T03:36:21.036908+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.660634+0000","last_change":"2024-09-21T03:36:22.085437+0000","last_active":"2024-09-21T03:36:23.660634+0000","last_peered":"2024-09-21T03:36:23.660634+0000","last_clean":"2024-09-21T03:36:23.660634+0000","last_became_active":"2024-09-21T03:36:22.085087+0000","last_became_peered":"2024-09-21T03:36:22.085087+0000","last_unstale":"2024-09-21T03:36:23.660634+0000","last_undegraded":"2024-09-21T03:36:23.660634+0000","last_fullsized":"2024-09-21T03:36:23.660634+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:30:00.925328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-21T03:36:22.082591+0000","last_change":"2024-09-21T03:36:22.081251+0000","last_active":"2024-09-21T03:36:22.082591+0000","last_peered":"2024-09-21T03:36:22.082591+0000","last_clean":"2024-09-21T03:36:22.082591+0000","last_became_active":"2024-09-21T03:36:22.080931+0000","last_became_peered":"2024-09-21T03:36:22.080931+0000","last_unstale":"2024-09-21T03:36:22.082591+0000","last_undegraded":"2024-09-21T03:36:22.082591+0000","last_fullsized":"2024-09-21T03:36:22.082591+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:04:34.873429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.661300+0000","last_change":"2024-09-21T03:36:22.069803+0000","last_active":"2024-09-21T03:36:23.661300+0000","last_peered":"2024-09-21T03:36:23.661300+0000","last_clean":"2024-09-21T03:36:23.661300+0000","last_became_active":"2024-09-21T03:36:22.069481+0000","last_became_peered":"2024-09-21T03:36:22.069481+0000","last_unstale":"2024-09-21T03:36:23.661300+0000","last_undegraded":"2024-09-21T03:36:23.661300+0000","last_fullsized":"2024-09-21T03:36:23.661300+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:15:44.483810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.661639+0000","last_change":"2024-09-21T03:36:22.076014+0000","last_active":"2024-09-21T03:36:23.661639+0000","last_peered":"2024-09-21T03:36:23.661639+0000","last_clean":"2024-09-21T03:36:23.661639+0000","last_became_active":"2024-09-21T03:36:22.075689+0000","last_became_peered":"2024-09-21T03:36:22.075689+0000","last_unstale":"2024-09-21T03:36:23.661639+0000","last_undegraded":"2024-09-21T03:36:23.661639+0000","last_fullsized":"2024-09-21T03:36:23.661639+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:47:25.173315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-21T03:36:22.073135+0000","last_change":"2024-09-21T03:36:22.071709+0000","last_active":"2024-09-21T03:36:22.073135+0000","last_peered":"2024-09-21T03:36:22.073135+0000","last_clean":"2024-09-21T03:36:22.073135+0000","last_became_active":"2024-09-21T03:36:22.070413+0000","last_became_peered":"2024-09-21T03:36:22.070413+0000","last_unstale":"2024-09-21T03:36:22.073135+0000","last_undegraded":"2024-09-21T03:36:22.073135+0000","last_fullsized":"2024-09-21T03:36:22.073135+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:16:55.679529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.660747+0000","last_change":"2024-09-21T03:36:22.086945+0000","last_active":"2024-09-21T03:36:23.660747+0000","last_peered":"2024-09-21T03:36:23.660747+0000","last_clean":"2024-09-21T03:36:23.660747+0000","last_became_active":"2024-09-21T03:36:22.086620+0000","last_became_peered":"2024-09-21T03:36:22.086620+0000","last_unstale":"2024-09-21T03:36:23.660747+0000","last_undegraded":"2024-09-21T03:36:23.660747+0000","last_fullsized":"2024-09-21T03:36:23.660747+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:48:46.104173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-21T03:36:21.053928+0000","last_change":"2024-09-21T03:36:21.053928+0000","last_active":"2024-09-21T03:36:21.036908+0000","last_peered":"2024-09-21T03:36:21.036908+0000","last_clean":"2024-09-21T03:36:21.036908+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:36:21.053928+0000","last_undegraded":"2024-09-21T03:36:21.053928+0000","last_fullsized":"2024-09-21T03:36:21.053928+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.660604+0000","last_change":"2024-09-21T03:36:22.086122+0000","last_active":"2024-09-21T03:36:23.660604+0000","last_peered":"2024-09-21T03:36:23.660604+0000","last_clean":"2024-09-21T03:36:23.660604+0000","last_became_active":"2024-09-21T03:36:22.085783+0000","last_became_peered":"2024-09-21T03:36:22.085783+0000","last_unstale":"2024-09-21T03:36:23.660604+0000","last_undegraded":"2024-09-21T03:36:23.660604+0000","last_fullsized":"2024-09-21T03:36:23.660604+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:05:24.777050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.085420+0000","last_change":"2024-09-21T03:36:22.067412+0000","last_active":"2024-09-21T03:36:23.085420+0000","last_peered":"2024-09-21T03:36:23.085420+0000","last_clean":"2024-09-21T03:36:23.085420+0000","last_became_active":"2024-09-21T03:36:22.067084+0000","last_became_peered":"2024-09-21T03:36:22.067084+0000","last_unstale":"2024-09-21T03:36:23.085420+0000","last_undegraded":"2024-09-21T03:36:23.085420+0000","last_fullsized":"2024-09-21T03:36:23.085420+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:01:59.040840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.661165+0000","last_change":"2024-09-21T03:36:22.082532+0000","last_active":"2024-09-21T03:36:23.661165+0000","last_peered":"2024-09-21T03:36:23.661165+0000","last_clean":"2024-09-21T03:36:23.661165+0000","last_became_active":"2024-09-21T03:36:22.082233+0000","last_became_peered":"2024-09-21T03:36:22.082233+0000","last_unstale":"2024-09-21T03:36:23.661165+0000","last_undegraded":"2024-09-21T03:36:23.661165+0000","last_fullsized":"2024-09-21T03:36:23.661165+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:09:57.485058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.086025+0000","last_change":"2024-09-21T03:36:22.068982+0000","last_active":"2024-09-21T03:36:23.086025+0000","last_peered":"2024-09-21T03:36:23.086025+0000","last_clean":"2024-09-21T03:36:23.086025+0000","last_became_active":"2024-09-21T03:36:22.068584+0000","last_became_peered":"2024-09-21T03:36:22.068584+0000","last_unstale":"2024-09-21T03:36:23.086025+0000","last_undegraded":"2024-09-21T03:36:23.086025+0000","last_fullsized":"2024-09-21T03:36:23.086025+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:02:01.275514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.660817+0000","last_change":"2024-09-21T03:36:22.085467+0000","last_active":"2024-09-21T03:36:23.660817+0000","last_peered":"2024-09-21T03:36:23.660817+0000","last_clean":"2024-09-21T03:36:23.660817+0000","last_became_active":"2024-09-21T03:36:22.085127+0000","last_became_peered":"2024-09-21T03:36:22.085127+0000","last_unstale":"2024-09-21T03:36:23.660817+0000","last_undegraded":"2024-09-21T03:36:23.660817+0000","last_fullsized":"2024-09-21T03:36:23.660817+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:04:32.859631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-21T03:36:22.084080+0000","last_change":"2024-09-21T03:36:22.083174+0000","last_active":"2024-09-21T03:36:22.084080+0000","last_peered":"2024-09-21T03:36:22.084080+0000","last_clean":"2024-09-21T03:36:22.084080+0000","last_became_active":"2024-09-21T03:36:22.082930+0000","last_became_peered":"2024-09-21T03:36:22.082930+0000","last_unstale":"2024-09-21T03:36:22.084080+0000","last_undegraded":"2024-09-21T03:36:22.084080+0000","last_fullsized":"2024-09-21T03:36:22.084080+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:09:50.811454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.661615+0000","last_change":"2024-09-21T03:36:22.088889+0000","last_active":"2024-09-21T03:36:23.661615+0000","last_peered":"2024-09-21T03:36:23.661615+0000","last_clean":"2024-09-21T03:36:23.661615+0000","last_became_active":"2024-09-21T03:36:22.088515+0000","last_became_peered":"2024-09-21T03:36:22.088515+0000","last_unstale":"2024-09-21T03:36:23.661615+0000","last_undegraded":"2024-09-21T03:36:23.661615+0000","last_fullsized":"2024-09-21T03:36:23.661615+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:00:06.850866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.660822+0000","last_change":"2024-09-21T03:36:22.088463+0000","last_active":"2024-09-21T03:36:23.660822+0000","last_peered":"2024-09-21T03:36:23.660822+0000","last_clean":"2024-09-21T03:36:23.660822+0000","last_became_active":"2024-09-21T03:36:22.088149+0000","last_became_peered":"2024-09-21T03:36:22.088149+0000","last_unstale":"2024-09-21T03:36:23.660822+0000","last_undegraded":"2024-09-21T03:36:23.660822+0000","last_fullsized":"2024-09-21T03:36:23.660822+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:03:04.392175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.085608+0000","last_change":"2024-09-21T03:36:22.066107+0000","last_active":"2024-09-21T03:36:23.085608+0000","last_peered":"2024-09-21T03:36:23.085608+0000","last_clean":"2024-09-21T03:36:23.085608+0000","last_became_active":"2024-09-21T03:36:22.064917+0000","last_became_peered":"2024-09-21T03:36:22.064917+0000","last_unstale":"2024-09-21T03:36:23.085608+0000","last_undegraded":"2024-09-21T03:36:23.085608+0000","last_fullsized":"2024-09-21T03:36:23.085608+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:58:06.692992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-21T03:36:22.072148+0000","last_change":"2024-09-21T03:36:22.069000+0000","last_active":"2024-09-21T03:36:22.072148+0000","last_peered":"2024-09-21T03:36:22.072148+0000","last_clean":"2024-09-21T03:36:22.072148+0000","last_became_active":"2024-09-21T03:36:22.068661+0000","last_became_peered":"2024-09-21T03:36:22.068661+0000","last_unstale":"2024-09-21T03:36:22.072148+0000","last_undegraded":"2024-09-21T03:36:22.072148+0000","last_fullsized":"2024-09-21T03:36:22.072148+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:39:01.505213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.661745+0000","last_change":"2024-09-21T03:36:22.071976+0000","last_active":"2024-09-21T03:36:23.661745+0000","last_peered":"2024-09-21T03:36:23.661745+0000","last_clean":"2024-09-21T03:36:23.661745+0000","last_became_active":"2024-09-21T03:36:22.071182+0000","last_became_peered":"2024-09-21T03:36:22.071182+0000","last_unstale":"2024-09-21T03:36:23.661745+0000","last_undegraded":"2024-09-21T03:36:23.661745+0000","last_fullsized":"2024-09-21T03:36:23.661745+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:58:44.343782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1f","version":"66'4","reported_seq":17,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-21T03:36:22.081942+0000","last_change":"2024-09-21T03:36:21.066587+0000","last_active":"2024-09-21T03:36:22.081942+0000","last_peered":"2024-09-21T03:36:22.081942+0000","last_clean":"2024-09-21T03:36:22.081942+0000","last_became_active":"2024-09-21T03:36:21.066299+0000","last_became_peered":"2024-09-21T03:36:21.066299+0000","last_unstale":"2024-09-21T03:36:22.081942+0000","last_undegraded":"2024-09-21T03:36:22.081942+0000","last_fullsized":"2024-09-21T03:36:22.081942+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_clean_scrub_stamp":"2024-09-21T03:36:20.027499+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:06:51.671380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.086591+0000","last_change":"2024-09-21T03:36:22.073629+0000","last_active":"2024-09-21T03:36:23.086591+0000","last_peered":"2024-09-21T03:36:23.086591+0000","last_clean":"2024-09-21T03:36:23.086591+0000","last_became_active":"2024-09-21T03:36:22.073211+0000","last_became_peered":"2024-09-21T03:36:22.073211+0000","last_unstale":"2024-09-21T03:36:23.086591+0000","last_undegraded":"2024-09-21T03:36:23.086591+0000","last_fullsized":"2024-09-21T03:36:23.086591+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:27:32.381253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1e","version":"66'1","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-21T03:36:22.092834+0000","last_change":"2024-09-21T03:36:21.064384+0000","last_active":"2024-09-21T03:36:22.092834+0000","last_peered":"2024-09-21T03:36:22.092834+0000","last_clean":"2024-09-21T03:36:22.092834+0000","last_became_active":"2024-09-21T03:36:21.064042+0000","last_became_peered":"2024-09-21T03:36:21.064042+0000","last_unstale":"2024-09-21T03:36:22.092834+0000","last_undegraded":"2024-09-21T03:36:22.092834+0000","last_fullsized":"2024-09-21T03:36:22.092834+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_clean_scrub_stamp":"2024-09-21T03:36:20.027499+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:17:25.045895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-21T03:36:21.054153+0000","last_change":"2024-09-21T03:36:21.054153+0000","last_active":"2024-09-21T03:36:21.036908+0000","last_peered":"2024-09-21T03:36:21.036908+0000","last_clean":"2024-09-21T03:36:21.036908+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:36:21.054153+0000","last_undegraded":"2024-09-21T03:36:21.054153+0000","last_fullsized":"2024-09-21T03:36:21.054153+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1d","version":"66'2","reported_seq":15,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-21T03:36:22.080793+0000","last_change":"2024-09-21T03:36:21.071133+0000","last_active":"2024-09-21T03:36:22.080793+0000","last_peered":"2024-09-21T03:36:22.080793+0000","last_clean":"2024-09-21T03:36:22.080793+0000","last_became_active":"2024-09-21T03:36:21.070834+0000","last_became_peered":"2024-09-21T03:36:21.070834+0000","last_unstale":"2024-09-21T03:36:22.080793+0000","last_undegraded":"2024-09-21T03:36:22.080793+0000","last_fullsized":"2024-09-21T03:36:22.080793+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_clean_scrub_stamp":"2024-09-21T03:36:20.027499+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:37:25.878136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.661630+0000","last_change":"2024-09-21T03:36:22.070080+0000","last_active":"2024-09-21T03:36:23.661630+0000","last_peered":"2024-09-21T03:36:23.661630+0000","last_clean":"2024-09-21T03:36:23.661630+0000","last_became_active":"2024-09-21T03:36:22.069779+0000","last_became_peered":"2024-09-21T03:36:22.069779+0000","last_unstale":"2024-09-21T03:36:23.661630+0000","last_undegraded":"2024-09-21T03:36:23.661630+0000","last_fullsized":"2024-09-21T03:36:23.661630+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:29:10.577562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-21T03:36:21.066934+0000","last_change":"2024-09-21T03:36:21.065338+0000","last_active":"2024-09-21T03:36:21.066934+0000","last_peered":"2024-09-21T03:36:21.066934+0000","last_clean":"2024-09-21T03:36:21.066934+0000","last_became_active":"2024-09-21T03:36:21.065032+0000","last_became_peered":"2024-09-21T03:36:21.065032+0000","last_unstale":"2024-09-21T03:36:21.066934+0000","last_undegraded":"2024-09-21T03:36:21.066934+0000","last_fullsized":"2024-09-21T03:36:21.066934+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_clean_scrub_stamp":"2024-09-21T03:36:20.027499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:38:09.536350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-21T03:36:22.065482+0000","last_change":"2024-09-21T03:36:22.063213+0000","last_active":"2024-09-21T03:36:22.065482+0000","last_peered":"2024-09-21T03:36:22.065482+0000","last_clean":"2024-09-21T03:36:22.065482+0000","last_became_active":"2024-09-21T03:36:22.062870+0000","last_became_peered":"2024-09-21T03:36:22.062870+0000","last_unstale":"2024-09-21T03:36:22.065482+0000","last_undegraded":"2024-09-21T03:36:22.065482+0000","last_fullsized":"2024-09-21T03:36:22.065482+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:07:32.687989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"66'1","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-21T03:36:22.079299+0000","last_change":"2024-09-21T03:36:21.060507+0000","last_active":"2024-09-21T03:36:22.079299+0000","last_peered":"2024-09-21T03:36:22.079299+0000","last_clean":"2024-09-21T03:36:22.079299+0000","last_became_active":"2024-09-21T03:36:21.060129+0000","last_became_peered":"2024-09-21T03:36:21.060129+0000","last_unstale":"2024-09-21T03:36:22.079299+0000","last_undegraded":"2024-09-21T03:36:22.079299+0000","last_fullsized":"2024-09-21T03:36:22.079299+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_clean_scrub_stamp":"2024-09-21T03:36:20.027499+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:57:03.979240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-21T03:36:22.080880+0000","last_change":"2024-09-21T03:36:22.077206+0000","last_active":"2024-09-21T03:36:22.080880+0000","last_peered":"2024-09-21T03:36:22.080880+0000","last_clean":"2024-09-21T03:36:22.080880+0000","last_became_active":"2024-09-21T03:36:22.076897+0000","last_became_peered":"2024-09-21T03:36:22.076897+0000","last_unstale":"2024-09-21T03:36:22.080880+0000","last_undegraded":"2024-09-21T03:36:22.080880+0000","last_fullsized":"2024-09-21T03:36:22.080880+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:39:58.170006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-21T03:36:21.062625+0000","last_change":"2024-09-21T03:36:21.059505+0000","last_active":"2024-09-21T03:36:21.062625+0000","last_peered":"2024-09-21T03:36:21.062625+0000","last_clean":"2024-09-21T03:36:21.062625+0000","last_became_active":"2024-09-21T03:36:21.059110+0000","last_became_peered":"2024-09-21T03:36:21.059110+0000","last_unstale":"2024-09-21T03:36:21.062625+0000","last_undegraded":"2024-09-21T03:36:21.062625+0000","last_fullsized":"2024-09-21T03:36:21.062625+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_clean_scrub_stamp":"2024-09-21T03:36:20.027499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:47:13.398068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T03:36:21.036908+0000","last_change":"2024-09-21T03:36:21.036908+0000","last_active":"2024-09-21T03:36:21.036908+0000","last_peered":"2024-09-21T03:36:21.036908+0000","last_clean":"2024-09-21T03:36:21.036908+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:36:21.036908+0000","last_undegraded":"2024-09-21T03:36:21.036908+0000","last_fullsized":"2024-09-21T03:36:21.036908+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":13,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-21T03:36:22.065557+0000","last_change":"2024-09-21T03:36:21.065608+0000","last_active":"2024-09-21T03:36:22.065557+0000","last_peered":"2024-09-21T03:36:22.065557+0000","last_clean":"2024-09-21T03:36:22.065557+0000","last_became_active":"2024-09-21T03:36:21.065292+0000","last_became_peered":"2024-09-21T03:36:21.065292+0000","last_unstale":"2024-09-21T03:36:22.065557+0000","last_undegraded":"2024-09-21T03:36:22.065557+0000","last_fullsized":"2024-09-21T03:36:22.065557+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_clean_scrub_stamp":"2024-09-21T03:36:20.027499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:36:50.067003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-21T03:36:22.072312+0000","last_change":"2024-09-21T03:36:22.069640+0000","last_active":"2024-09-21T03:36:22.072312+0000","last_peered":"2024-09-21T03:36:22.072312+0000","last_clean":"2024-09-21T03:36:22.072312+0000","last_became_active":"2024-09-21T03:36:22.068836+0000","last_became_peered":"2024-09-21T03:36:22.068836+0000","last_unstale":"2024-09-21T03:36:22.072312+0000","last_undegraded":"2024-09-21T03:36:22.072312+0000","last_fullsized":"2024-09-21T03:36:22.072312+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:29:10.577562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.660609+0000","last_change":"2024-09-21T03:36:21.062628+0000","last_active":"2024-09-21T03:36:23.660609+0000","last_peered":"2024-09-21T03:36:23.660609+0000","last_clean":"2024-09-21T03:36:23.660609+0000","last_became_active":"2024-09-21T03:36:21.062295+0000","last_became_peered":"2024-09-21T03:36:21.062295+0000","last_unstale":"2024-09-21T03:36:23.660609+0000","last_undegraded":"2024-09-21T03:36:23.660609+0000","last_fullsized":"2024-09-21T03:36:23.660609+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_clean_scrub_stamp":"2024-09-21T03:36:20.027499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:20:15.258373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-21T03:36:22.069673+0000","last_change":"2024-09-21T03:36:22.067826+0000","last_active":"2024-09-21T03:36:22.069673+0000","last_peered":"2024-09-21T03:36:22.069673+0000","last_clean":"2024-09-21T03:36:22.069673+0000","last_became_active":"2024-09-21T03:36:22.067516+0000","last_became_peered":"2024-09-21T03:36:22.067516+0000","last_unstale":"2024-09-21T03:36:22.069673+0000","last_undegraded":"2024-09-21T03:36:22.069673+0000","last_fullsized":"2024-09-21T03:36:22.069673+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:20:16.267783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.661694+0000","last_change":"2024-09-21T03:36:21.063524+0000","last_active":"2024-09-21T03:36:23.661694+0000","last_peered":"2024-09-21T03:36:23.661694+0000","last_clean":"2024-09-21T03:36:23.661694+0000","last_became_active":"2024-09-21T03:36:21.063203+0000","last_became_peered":"2024-09-21T03:36:21.063203+0000","last_unstale":"2024-09-21T03:36:23.661694+0000","last_undegraded":"2024-09-21T03:36:23.661694+0000","last_fullsized":"2024-09-21T03:36:23.661694+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_clean_scrub_stamp":"2024-09-21T03:36:20.027499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:25:29.512603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-21T03:36:22.065852+0000","last_change":"2024-09-21T03:36:22.063211+0000","last_active":"2024-09-21T03:36:22.065852+0000","last_peered":"2024-09-21T03:36:22.065852+0000","last_clean":"2024-09-21T03:36:22.065852+0000","last_became_active":"2024-09-21T03:36:22.062828+0000","last_became_peered":"2024-09-21T03:36:22.062828+0000","last_unstale":"2024-09-21T03:36:22.065852+0000","last_undegraded":"2024-09-21T03:36:22.065852+0000","last_fullsized":"2024-09-21T03:36:22.065852+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:30:41.436799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.661311+0000","last_change":"2024-09-21T03:36:21.064314+0000","last_active":"2024-09-21T03:36:23.661311+0000","last_peered":"2024-09-21T03:36:23.661311+0000","last_clean":"2024-09-21T03:36:23.661311+0000","last_became_active":"2024-09-21T03:36:21.063271+0000","last_became_peered":"2024-09-21T03:36:21.063271+0000","last_unstale":"2024-09-21T03:36:23.661311+0000","last_undegraded":"2024-09-21T03:36:23.661311+0000","last_fullsized":"2024-09-21T03:36:23.661311+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_clean_scrub_stamp":"2024-09-21T03:36:20.027499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:03:09.222194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.660928+0000","last_change":"2024-09-21T03:36:22.092139+0000","last_active":"2024-09-21T03:36:23.660928+0000","last_peered":"2024-09-21T03:36:23.660928+0000","last_clean":"2024-09-21T03:36:23.660928+0000","last_became_active":"2024-09-21T03:36:22.091718+0000","last_became_peered":"2024-09-21T03:36:22.091718+0000","last_unstale":"2024-09-21T03:36:23.660928+0000","last_undegraded":"2024-09-21T03:36:23.660928+0000","last_fullsized":"2024-09-21T03:36:23.660928+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:28:40.036155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.15","version":"66'3","reported_seq":18,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.086559+0000","last_change":"2024-09-21T03:36:21.064387+0000","last_active":"2024-09-21T03:36:23.086559+0000","last_peered":"2024-09-21T03:36:23.086559+0000","last_clean":"2024-09-21T03:36:23.086559+0000","last_became_active":"2024-09-21T03:36:21.064086+0000","last_became_peered":"2024-09-21T03:36:21.064086+0000","last_unstale":"2024-09-21T03:36:23.086559+0000","last_undegraded":"2024-09-21T03:36:23.086559+0000","last_fullsized":"2024-09-21T03:36:23.086559+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_clean_scrub_stamp":"2024-09-21T03:36:20.027499+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:01:58.431275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.086256+0000","last_change":"2024-09-21T03:36:22.070639+0000","last_active":"2024-09-21T03:36:23.086256+0000","last_peered":"2024-09-21T03:36:23.086256+0000","last_clean":"2024-09-21T03:36:23.086256+0000","last_became_active":"2024-09-21T03:36:22.069620+0000","last_became_peered":"2024-09-21T03:36:22.069620+0000","last_unstale":"2024-09-21T03:36:23.086256+0000","last_undegraded":"2024-09-21T03:36:23.086256+0000","last_fullsized":"2024-09-21T03:36:23.086256+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:23:36.147744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-21T03:36:21.066007+0000","last_change":"2024-09-21T03:36:21.065091+0000","last_active":"2024-09-21T03:36:21.066007+0000","last_peered":"2024-09-21T03:36:21.066007+0000","last_clean":"2024-09-21T03:36:21.066007+0000","last_became_active":"2024-09-21T03:36:21.064760+0000","last_became_peered":"2024-09-21T03:36:21.064760+0000","last_unstale":"2024-09-21T03:36:21.066007+0000","last_undegraded":"2024-09-21T03:36:21.066007+0000","last_fullsized":"2024-09-21T03:36:21.066007+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_clean_scrub_stamp":"2024-09-21T03:36:20.027499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:17:34.450050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T03:36:21.036908+0000","last_change":"2024-09-21T03:36:21.036908+0000","last_active":"2024-09-21T03:36:21.036908+0000","last_peered":"2024-09-21T03:36:21.036908+0000","last_clean":"2024-09-21T03:36:21.036908+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:36:21.036908+0000","last_undegraded":"2024-09-21T03:36:21.036908+0000","last_fullsized":"2024-09-21T03:36:21.036908+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.13","version":"66'2","reported_seq":17,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.661866+0000","last_change":"2024-09-21T03:36:21.064320+0000","last_active":"2024-09-21T03:36:23.661866+0000","last_peered":"2024-09-21T03:36:23.661866+0000","last_clean":"2024-09-21T03:36:23.661866+0000","last_became_active":"2024-09-21T03:36:21.064030+0000","last_became_peered":"2024-09-21T03:36:21.064030+0000","last_unstale":"2024-09-21T03:36:23.661866+0000","last_undegraded":"2024-09-21T03:36:23.661866+0000","last_fullsized":"2024-09-21T03:36:23.661866+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_clean_scrub_stamp":"2024-09-21T03:36:20.027499+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:36:50.067003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T03:36:21.036908+0000","last_change":"2024-09-21T03:36:21.036908+0000","last_active":"2024-09-21T03:36:21.036908+0000","last_peered":"2024-09-21T03:36:21.036908+0000","last_clean":"2024-09-21T03:36:21.036908+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:36:21.036908+0000","last_undegraded":"2024-09-21T03:36:21.036908+0000","last_fullsized":"2024-09-21T03:36:21.036908+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T03:36:20.027499+0000","last_change":"2024-09-21T03:36:20.027499+0000","last_active":"2024-09-21T03:36:20.027499+0000","last_peered":"2024-09-21T03:36:20.027499+0000","last_clean":"2024-09-21T03:36:20.027499+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:36:20.027499+0000","last_undegraded":"2024-09-21T03:36:20.027499+0000","last_fullsized":"2024-09-21T03:36:20.027499+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_clean_scrub_stamp":"2024-09-21T03:36:20.027499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T03:36:21.036908+0000","last_change":"2024-09-21T03:36:21.036908+0000","last_active":"2024-09-21T03:36:21.036908+0000","last_peered":"2024-09-21T03:36:21.036908+0000","last_clean":"2024-09-21T03:36:21.036908+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:36:21.036908+0000","last_undegraded":"2024-09-21T03:36:21.036908+0000","last_fullsized":"2024-09-21T03:36:21.036908+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.085748+0000","last_change":"2024-09-21T03:36:21.061510+0000","last_active":"2024-09-21T03:36:23.085748+0000","last_peered":"2024-09-21T03:36:23.085748+0000","last_clean":"2024-09-21T03:36:23.085748+0000","last_became_active":"2024-09-21T03:36:21.061181+0000","last_became_peered":"2024-09-21T03:36:21.061181+0000","last_unstale":"2024-09-21T03:36:23.085748+0000","last_undegraded":"2024-09-21T03:36:23.085748+0000","last_fullsized":"2024-09-21T03:36:23.085748+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_clean_scrub_stamp":"2024-09-21T03:36:20.027499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:39:48.217217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-21T03:36:22.086623+0000","last_change":"2024-09-21T03:36:22.085750+0000","last_active":"2024-09-21T03:36:22.086623+0000","last_peered":"2024-09-21T03:36:22.086623+0000","last_clean":"2024-09-21T03:36:22.086623+0000","last_became_active":"2024-09-21T03:36:22.085426+0000","last_became_peered":"2024-09-21T03:36:22.085426+0000","last_unstale":"2024-09-21T03:36:22.086623+0000","last_undegraded":"2024-09-21T03:36:22.086623+0000","last_fullsized":"2024-09-21T03:36:22.086623+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:43:37.785209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T03:36:20.027499+0000","last_change":"2024-09-21T03:36:20.027499+0000","last_active":"2024-09-21T03:36:20.027499+0000","last_peered":"2024-09-21T03:36:20.027499+0000","last_clean":"2024-09-21T03:36:20.027499+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:36:20.027499+0000","last_undegraded":"2024-09-21T03:36:20.027499+0000","last_fullsized":"2024-09-21T03:36:20.027499+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_clean_scrub_stamp":"2024-09-21T03:36:20.027499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.661909+0000","last_change":"2024-09-21T03:36:22.086802+0000","last_active":"2024-09-21T03:36:23.661909+0000","last_peered":"2024-09-21T03:36:23.661909+0000","last_clean":"2024-09-21T03:36:23.661909+0000","last_became_active":"2024-09-21T03:36:22.086435+0000","last_became_peered":"2024-09-21T03:36:22.086435+0000","last_unstale":"2024-09-21T03:36:23.661909+0000","last_undegraded":"2024-09-21T03:36:23.661909+0000","last_fullsized":"2024-09-21T03:36:23.661909+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:32:26.377539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.f","version":"66'1","reported_seq":16,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.661756+0000","last_change":"2024-09-21T03:36:21.062807+0000","last_active":"2024-09-21T03:36:23.661756+0000","last_peered":"2024-09-21T03:36:23.661756+0000","last_clean":"2024-09-21T03:36:23.661756+0000","last_became_active":"2024-09-21T03:36:21.062513+0000","last_became_peered":"2024-09-21T03:36:21.062513+0000","last_unstale":"2024-09-21T03:36:23.661756+0000","last_undegraded":"2024-09-21T03:36:23.661756+0000","last_fullsized":"2024-09-21T03:36:23.661756+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_clean_scrub_stamp":"2024-09-21T03:36:20.027499+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:39:57.160597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.660925+0000","last_change":"2024-09-21T03:36:22.086393+0000","last_active":"2024-09-21T03:36:23.660925+0000","last_peered":"2024-09-21T03:36:23.660925+0000","last_clean":"2024-09-21T03:36:23.660925+0000","last_became_active":"2024-09-21T03:36:22.086094+0000","last_became_peered":"2024-09-21T03:36:22.086094+0000","last_unstale":"2024-09-21T03:36:23.660925+0000","last_undegraded":"2024-09-21T03:36:23.660925+0000","last_fullsized":"2024-09-21T03:36:23.660925+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:11:34.432933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.e","version":"66'2","reported_seq":17,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.661138+0000","last_change":"2024-09-21T03:36:21.053740+0000","last_active":"2024-09-21T03:36:23.661138+0000","last_peered":"2024-09-21T03:36:23.661138+0000","last_clean":"2024-09-21T03:36:23.661138+0000","last_became_active":"2024-09-21T03:36:21.053383+0000","last_became_peered":"2024-09-21T03:36:21.053383+0000","last_unstale":"2024-09-21T03:36:23.661138+0000","last_undegraded":"2024-09-21T03:36:23.661138+0000","last_fullsized":"2024-09-21T03:36:23.661138+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_clean_scrub_stamp":"2024-09-21T03:36:20.027499+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:37:25.878136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.661181+0000","last_change":"2024-09-21T03:36:22.093865+0000","last_active":"2024-09-21T03:36:23.661181+0000","last_peered":"2024-09-21T03:36:23.661181+0000","last_clean":"2024-09-21T03:36:23.661181+0000","last_became_active":"2024-09-21T03:36:22.093526+0000","last_became_peered":"2024-09-21T03:36:22.093526+0000","last_unstale":"2024-09-21T03:36:23.661181+0000","last_undegraded":"2024-09-21T03:36:23.661181+0000","last_fullsized":"2024-09-21T03:36:23.661181+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:59:12.697168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.d","version":"66'1","reported_seq":16,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.086666+0000","last_change":"2024-09-21T03:36:21.063656+0000","last_active":"2024-09-21T03:36:23.086666+0000","last_peered":"2024-09-21T03:36:23.086666+0000","last_clean":"2024-09-21T03:36:23.086666+0000","last_became_active":"2024-09-21T03:36:21.063355+0000","last_became_peered":"2024-09-21T03:36:21.063355+0000","last_unstale":"2024-09-21T03:36:23.086666+0000","last_undegraded":"2024-09-21T03:36:23.086666+0000","last_fullsized":"2024-09-21T03:36:23.086666+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_clean_scrub_stamp":"2024-09-21T03:36:20.027499+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:05:53.691084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.086342+0000","last_change":"2024-09-21T03:36:22.070297+0000","last_active":"2024-09-21T03:36:23.086342+0000","last_peered":"2024-09-21T03:36:23.086342+0000","last_clean":"2024-09-21T03:36:23.086342+0000","last_became_active":"2024-09-21T03:36:22.069258+0000","last_became_peered":"2024-09-21T03:36:22.069258+0000","last_unstale":"2024-09-21T03:36:23.086342+0000","last_undegraded":"2024-09-21T03:36:23.086342+0000","last_fullsized":"2024-09-21T03:36:23.086342+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:51:57.465792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.660607+0000","last_change":"2024-09-21T03:36:21.067658+0000","last_active":"2024-09-21T03:36:23.660607+0000","last_peered":"2024-09-21T03:36:23.660607+0000","last_clean":"2024-09-21T03:36:23.660607+0000","last_became_active":"2024-09-21T03:36:21.067359+0000","last_became_peered":"2024-09-21T03:36:21.067359+0000","last_unstale":"2024-09-21T03:36:23.660607+0000","last_undegraded":"2024-09-21T03:36:23.660607+0000","last_fullsized":"2024-09-21T03:36:23.660607+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_clean_scrub_stamp":"2024-09-21T03:36:20.027499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:38:26.468485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-21T03:36:21.054221+0000","last_change":"2024-09-21T03:36:21.054221+0000","last_active":"2024-09-21T03:36:21.036908+0000","last_peered":"2024-09-21T03:36:21.036908+0000","last_clean":"2024-09-21T03:36:21.036908+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:36:21.054221+0000","last_undegraded":"2024-09-21T03:36:21.054221+0000","last_fullsized":"2024-09-21T03:36:21.054221+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-21T03:36:21.068841+0000","last_change":"2024-09-21T03:36:21.067047+0000","last_active":"2024-09-21T03:36:21.068841+0000","last_peered":"2024-09-21T03:36:21.068841+0000","last_clean":"2024-09-21T03:36:21.068841+0000","last_became_active":"2024-09-21T03:36:21.066718+0000","last_became_peered":"2024-09-21T03:36:21.066718+0000","last_unstale":"2024-09-21T03:36:21.068841+0000","last_undegraded":"2024-09-21T03:36:21.068841+0000","last_fullsized":"2024-09-21T03:36:21.068841+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_clean_scrub_stamp":"2024-09-21T03:36:20.027499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:34:30.827273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.660351+0000","last_change":"2024-09-21T03:36:22.081020+0000","last_active":"2024-09-21T03:36:23.660351+0000","last_peered":"2024-09-21T03:36:23.660351+0000","last_clean":"2024-09-21T03:36:23.660351+0000","last_became_active":"2024-09-21T03:36:22.080090+0000","last_became_peered":"2024-09-21T03:36:22.080090+0000","last_unstale":"2024-09-21T03:36:23.660351+0000","last_undegraded":"2024-09-21T03:36:23.660351+0000","last_fullsized":"2024-09-21T03:36:23.660351+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:05:02.647190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T03:36:20.027499+0000","last_change":"2024-09-21T03:36:20.027499+0000","last_active":"2024-09-21T03:36:20.027499+0000","last_peered":"2024-09-21T03:36:20.027499+0000","last_clean":"2024-09-21T03:36:20.027499+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:36:20.027499+0000","last_undegraded":"2024-09-21T03:36:20.027499+0000","last_fullsized":"2024-09-21T03:36:20.027499+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_clean_scrub_stamp":"2024-09-21T03:36:20.027499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.660983+0000","last_change":"2024-09-21T03:36:22.071663+0000","last_active":"2024-09-21T03:36:23.660983+0000","last_peered":"2024-09-21T03:36:23.660983+0000","last_clean":"2024-09-21T03:36:23.660983+0000","last_became_active":"2024-09-21T03:36:22.071326+0000","last_became_peered":"2024-09-21T03:36:22.071326+0000","last_unstale":"2024-09-21T03:36:23.660983+0000","last_undegraded":"2024-09-21T03:36:23.660983+0000","last_fullsized":"2024-09-21T03:36:23.660983+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:33:14.339525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-21T03:36:21.064334+0000","last_change":"2024-09-21T03:36:21.062867+0000","last_active":"2024-09-21T03:36:21.064334+0000","last_peered":"2024-09-21T03:36:21.064334+0000","last_clean":"2024-09-21T03:36:21.064334+0000","last_became_active":"2024-09-21T03:36:21.062555+0000","last_became_peered":"2024-09-21T03:36:21.062555+0000","last_unstale":"2024-09-21T03:36:21.064334+0000","last_undegraded":"2024-09-21T03:36:21.064334+0000","last_fullsized":"2024-09-21T03:36:21.064334+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_clean_scrub_stamp":"2024-09-21T03:36:20.027499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:24:33.618928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-21T03:36:22.066694+0000","last_change":"2024-09-21T03:36:22.063923+0000","last_active":"2024-09-21T03:36:22.066694+0000","last_peered":"2024-09-21T03:36:22.066694+0000","last_clean":"2024-09-21T03:36:22.066694+0000","last_became_active":"2024-09-21T03:36:22.063195+0000","last_became_peered":"2024-09-21T03:36:22.063195+0000","last_unstale":"2024-09-21T03:36:22.066694+0000","last_undegraded":"2024-09-21T03:36:22.066694+0000","last_fullsized":"2024-09-21T03:36:22.066694+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:48:30.795859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.7","version":"66'2","reported_seq":17,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.661858+0000","last_change":"2024-09-21T03:36:21.068456+0000","last_active":"2024-09-21T03:36:23.661858+0000","last_peered":"2024-09-21T03:36:23.661858+0000","last_clean":"2024-09-21T03:36:23.661858+0000","last_became_active":"2024-09-21T03:36:21.068101+0000","last_became_peered":"2024-09-21T03:36:21.068101+0000","last_unstale":"2024-09-21T03:36:23.661858+0000","last_undegraded":"2024-09-21T03:36:23.661858+0000","last_fullsized":"2024-09-21T03:36:23.661858+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_clean_scrub_stamp":"2024-09-21T03:36:20.027499+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:09:49.802044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-21T03:36:22.073022+0000","last_change":"2024-09-21T03:36:22.071078+0000","last_active":"2024-09-21T03:36:22.073022+0000","last_peered":"2024-09-21T03:36:22.073022+0000","last_clean":"2024-09-21T03:36:22.073022+0000","last_became_active":"2024-09-21T03:36:22.070566+0000","last_became_peered":"2024-09-21T03:36:22.070566+0000","last_unstale":"2024-09-21T03:36:22.073022+0000","last_undegraded":"2024-09-21T03:36:22.073022+0000","last_fullsized":"2024-09-21T03:36:22.073022+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:09:22.850109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.661366+0000","last_change":"2024-09-21T03:36:21.059377+0000","last_active":"2024-09-21T03:36:23.661366+0000","last_peered":"2024-09-21T03:36:23.661366+0000","last_clean":"2024-09-21T03:36:23.661366+0000","last_became_active":"2024-09-21T03:36:21.059079+0000","last_became_peered":"2024-09-21T03:36:21.059079+0000","last_unstale":"2024-09-21T03:36:23.661366+0000","last_undegraded":"2024-09-21T03:36:23.661366+0000","last_fullsized":"2024-09-21T03:36:23.661366+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_clean_scrub_stamp":"2024-09-21T03:36:20.027499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:39:00.495803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.086122+0000","last_change":"2024-09-21T03:36:22.072209+0000","last_active":"2024-09-21T03:36:23.086122+0000","last_peered":"2024-09-21T03:36:23.086122+0000","last_clean":"2024-09-21T03:36:23.086122+0000","last_became_active":"2024-09-21T03:36:22.071876+0000","last_became_peered":"2024-09-21T03:36:22.071876+0000","last_unstale":"2024-09-21T03:36:23.086122+0000","last_undegraded":"2024-09-21T03:36:23.086122+0000","last_fullsized":"2024-09-21T03:36:23.086122+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:37:07.049180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-21T03:36:21.064294+0000","last_change":"2024-09-21T03:36:21.063184+0000","last_active":"2024-09-21T03:36:21.064294+0000","last_peered":"2024-09-21T03:36:21.064294+0000","last_clean":"2024-09-21T03:36:21.064294+0000","last_became_active":"2024-09-21T03:36:21.062859+0000","last_became_peered":"2024-09-21T03:36:21.062859+0000","last_unstale":"2024-09-21T03:36:21.064294+0000","last_undegraded":"2024-09-21T03:36:21.064294+0000","last_fullsized":"2024-09-21T03:36:21.064294+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_clean_scrub_stamp":"2024-09-21T03:36:20.027499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:18:15.549734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.086389+0000","last_change":"2024-09-21T03:36:22.069954+0000","last_active":"2024-09-21T03:36:23.086389+0000","last_peered":"2024-09-21T03:36:23.086389+0000","last_clean":"2024-09-21T03:36:23.086389+0000","last_became_active":"2024-09-21T03:36:22.068916+0000","last_became_peered":"2024-09-21T03:36:22.068916+0000","last_unstale":"2024-09-21T03:36:23.086389+0000","last_undegraded":"2024-09-21T03:36:23.086389+0000","last_fullsized":"2024-09-21T03:36:23.086389+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:03:46.925197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.660652+0000","last_change":"2024-09-21T03:36:21.071978+0000","last_active":"2024-09-21T03:36:23.660652+0000","last_peered":"2024-09-21T03:36:23.660652+0000","last_clean":"2024-09-21T03:36:23.660652+0000","last_became_active":"2024-09-21T03:36:21.071717+0000","last_became_peered":"2024-09-21T03:36:21.071717+0000","last_unstale":"2024-09-21T03:36:23.660652+0000","last_undegraded":"2024-09-21T03:36:23.660652+0000","last_fullsized":"2024-09-21T03:36:23.660652+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_clean_scrub_stamp":"2024-09-21T03:36:20.027499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:46:05.982882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.661216+0000","last_change":"2024-09-21T03:36:22.072086+0000","last_active":"2024-09-21T03:36:23.661216+0000","last_peered":"2024-09-21T03:36:23.661216+0000","last_clean":"2024-09-21T03:36:23.661216+0000","last_became_active":"2024-09-21T03:36:22.071729+0000","last_became_peered":"2024-09-21T03:36:22.071729+0000","last_unstale":"2024-09-21T03:36:23.661216+0000","last_undegraded":"2024-09-21T03:36:23.661216+0000","last_fullsized":"2024-09-21T03:36:23.661216+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:30:14.015775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.660720+0000","last_change":"2024-09-21T03:36:21.066336+0000","last_active":"2024-09-21T03:36:23.660720+0000","last_peered":"2024-09-21T03:36:23.660720+0000","last_clean":"2024-09-21T03:36:23.660720+0000","last_became_active":"2024-09-21T03:36:21.066043+0000","last_became_peered":"2024-09-21T03:36:21.066043+0000","last_unstale":"2024-09-21T03:36:23.660720+0000","last_undegraded":"2024-09-21T03:36:23.660720+0000","last_fullsized":"2024-09-21T03:36:23.660720+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_clean_scrub_stamp":"2024-09-21T03:36:20.027499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:45:07.961665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.660391+0000","last_change":"2024-09-21T03:36:22.089730+0000","last_active":"2024-09-21T03:36:23.660391+0000","last_peered":"2024-09-21T03:36:23.660391+0000","last_clean":"2024-09-21T03:36:23.660391+0000","last_became_active":"2024-09-21T03:36:22.089403+0000","last_became_peered":"2024-09-21T03:36:22.089403+0000","last_unstale":"2024-09-21T03:36:23.660391+0000","last_undegraded":"2024-09-21T03:36:23.660391+0000","last_fullsized":"2024-09-21T03:36:23.660391+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:38:24.463402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":13,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-21T03:36:22.061045+0000","last_change":"2024-09-21T03:36:21.066672+0000","last_active":"2024-09-21T03:36:22.061045+0000","last_peered":"2024-09-21T03:36:22.061045+0000","last_clean":"2024-09-21T03:36:22.061045+0000","last_became_active":"2024-09-21T03:36:21.066366+0000","last_became_peered":"2024-09-21T03:36:21.066366+0000","last_unstale":"2024-09-21T03:36:22.061045+0000","last_undegraded":"2024-09-21T03:36:22.061045+0000","last_fullsized":"2024-09-21T03:36:22.061045+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_clean_scrub_stamp":"2024-09-21T03:36:20.027499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:28:15.884066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-21T03:36:21.054594+0000","last_change":"2024-09-21T03:36:21.054594+0000","last_active":"2024-09-21T03:36:21.036908+0000","last_peered":"2024-09-21T03:36:21.036908+0000","last_clean":"2024-09-21T03:36:21.036908+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:36:21.054594+0000","last_undegraded":"2024-09-21T03:36:21.054594+0000","last_fullsized":"2024-09-21T03:36:21.054594+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.660737+0000","last_change":"2024-09-21T03:36:21.065843+0000","last_active":"2024-09-21T03:36:23.660737+0000","last_peered":"2024-09-21T03:36:23.660737+0000","last_clean":"2024-09-21T03:36:23.660737+0000","last_became_active":"2024-09-21T03:36:21.065466+0000","last_became_peered":"2024-09-21T03:36:21.065466+0000","last_unstale":"2024-09-21T03:36:23.660737+0000","last_undegraded":"2024-09-21T03:36:23.660737+0000","last_fullsized":"2024-09-21T03:36:23.660737+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_clean_scrub_stamp":"2024-09-21T03:36:20.027499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:10:41.577481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-21T03:36:22.072278+0000","last_change":"2024-09-21T03:36:22.068997+0000","last_active":"2024-09-21T03:36:22.072278+0000","last_peered":"2024-09-21T03:36:22.072278+0000","last_clean":"2024-09-21T03:36:22.072278+0000","last_became_active":"2024-09-21T03:36:22.067174+0000","last_became_peered":"2024-09-21T03:36:22.067174+0000","last_unstale":"2024-09-21T03:36:22.072278+0000","last_undegraded":"2024-09-21T03:36:22.072278+0000","last_fullsized":"2024-09-21T03:36:22.072278+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:42:28.131400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-21T03:36:21.069684+0000","last_change":"2024-09-21T03:36:21.067567+0000","last_active":"2024-09-21T03:36:21.069684+0000","last_peered":"2024-09-21T03:36:21.069684+0000","last_clean":"2024-09-21T03:36:21.069684+0000","last_became_active":"2024-09-21T03:36:21.067235+0000","last_became_peered":"2024-09-21T03:36:21.067235+0000","last_unstale":"2024-09-21T03:36:21.069684+0000","last_undegraded":"2024-09-21T03:36:21.069684+0000","last_fullsized":"2024-09-21T03:36:21.069684+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_clean_scrub_stamp":"2024-09-21T03:36:20.027499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:24:32.368652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T03:36:21.036908+0000","last_change":"2024-09-21T03:36:21.036908+0000","last_active":"2024-09-21T03:36:21.036908+0000","last_peered":"2024-09-21T03:36:21.036908+0000","last_clean":"2024-09-21T03:36:21.036908+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:36:21.036908+0000","last_undegraded":"2024-09-21T03:36:21.036908+0000","last_fullsized":"2024-09-21T03:36:21.036908+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-21T03:36:21.062519+0000","last_change":"2024-09-21T03:36:21.059196+0000","last_active":"2024-09-21T03:36:21.062519+0000","last_peered":"2024-09-21T03:36:21.062519+0000","last_clean":"2024-09-21T03:36:21.062519+0000","last_became_active":"2024-09-21T03:36:21.058881+0000","last_became_peered":"2024-09-21T03:36:21.058881+0000","last_unstale":"2024-09-21T03:36:21.062519+0000","last_undegraded":"2024-09-21T03:36:21.062519+0000","last_fullsized":"2024-09-21T03:36:21.062519+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_clean_scrub_stamp":"2024-09-21T03:36:20.027499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:37:14.541836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":190,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-21T03:36:18.037492+0000","last_change":"2024-09-21T03:33:25.319776+0000","last_active":"2024-09-21T03:36:18.037492+0000","last_peered":"2024-09-21T03:36:18.037492+0000","last_clean":"2024-09-21T03:36:18.037492+0000","last_became_active":"2024-09-21T03:33:25.318399+0000","last_became_peered":"2024-09-21T03:33:25.318399+0000","last_unstale":"2024-09-21T03:36:18.037492+0000","last_undegraded":"2024-09-21T03:36:18.037492+0000","last_fullsized":"2024-09-21T03:36:18.037492+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:23.016672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:23.016672+0000","last_clean_scrub_stamp":"2024-09-21T03:33:23.016672+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:32:35.906114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":114,"acting":114,"num_store_stats":7},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":798,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19,"ondisk_log_size":19,"up":58,"acting":58,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":1,"up_from":16,"seq":68719476775,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32508,"kb_used_data":1316,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339332,"statfs":{"total":96636764160,"available":96603475968,"internally_reserved":0,"allocated":1347584,"data_stored":927955,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509480,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36124,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335716,"statfs":{"total":96636764160,"available":96599773184,"internally_reserved":0,"allocated":790528,"data_stored":357630,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":36174283},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509479,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31980,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339860,"statfs":{"total":96636764160,"available":96604016640,"internally_reserved":0,"allocated":811008,"data_stored":364482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509480,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31980,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339860,"statfs":{"total":96636764160,"available":96604016640,"internally_reserved":0,"allocated":811008,"data_stored":364378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509479,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31968,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339872,"statfs":{"total":96636764160,"available":96604028928,"internally_reserved":0,"allocated":790528,"data_stored":357630,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542183,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36148,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335692,"statfs":{"total":96636764160,"available":96599748608,"internally_reserved":0,"allocated":811008,"data_stored":359894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574887,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36692,"kb_used_data":1340,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335148,"statfs":{"total":96636764160,"available":96599191552,"internally_reserved":0,"allocated":1372160,"data_stored":940831,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542184,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36132,"kb_used_data":776,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335708,"statfs":{"total":96636764160,"available":96599764992,"internally_reserved":0,"allocated":794624,"data_stored":358212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T03:36:26.158 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T03:36:26.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:26.310+0000 7f49220b6640 1 -- 172.21.15.38:0/960702182 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f491c0d1ed0 msgr2=0x7f491c0cc930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:26.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:26.310+0000 7f49220b6640 1 --2- 172.21.15.38:0/960702182 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f491c0d1ed0 0x7f491c0cc930 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f4910009830 tx=0x7f491002a510 comp rx=0 tx=0).stop 2024-09-21T03:36:26.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:26.310+0000 7f49220b6640 1 -- 172.21.15.38:0/960702182 shutdown_connections 2024-09-21T03:36:26.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:26.310+0000 7f49220b6640 1 --2- 172.21.15.38:0/960702182 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f491c0cce70 0x7f491c0d04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:26.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:26.310+0000 7f49220b6640 1 --2- 172.21.15.38:0/960702182 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f491c0d1ed0 0x7f491c0cc930 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:26.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:26.310+0000 7f49220b6640 1 -- 172.21.15.38:0/960702182 >> 172.21.15.38:0/960702182 conn(0x7f491c0cef40 msgr2=0x7f491c0cf350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:26.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:26.310+0000 7f49220b6640 1 -- 172.21.15.38:0/960702182 shutdown_connections 2024-09-21T03:36:26.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:26.310+0000 7f49220b6640 1 -- 172.21.15.38:0/960702182 wait complete. 2024-09-21T03:36:26.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:26.312+0000 7f49220b6640 1 Processor -- start 2024-09-21T03:36:26.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:26.312+0000 7f49220b6640 1 -- start start 2024-09-21T03:36:26.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:26.313+0000 7f49220b6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f491c0cce70 0x7f491c0cbd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:26.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:26.313+0000 7f49220b6640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f491c0d1ed0 0x7f491c1457e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:26.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:26.313+0000 7f49220b6640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f491c1536e0 0x7f491c145d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:26.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:26.313+0000 7f49220b6640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f491c0d0d90 con 0x7f491c0cce70 2024-09-21T03:36:26.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:26.313+0000 7f49220b6640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f491c0d0c10 con 0x7f491c0d1ed0 2024-09-21T03:36:26.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:26.313+0000 7f49220b6640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f491c0d0f10 con 0x7f491c1536e0 2024-09-21T03:36:26.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:26.313+0000 7f491b7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f491c0cce70 0x7f491c0cbd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:26.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:26.313+0000 7f491b7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f491c0cce70 0x7f491c0cbd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37324/0 (socket says 172.21.15.38:37324) 2024-09-21T03:36:26.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:26.313+0000 7f491b7fe640 1 -- 172.21.15.38:0/608553025 learned_addr learned my addr 172.21.15.38:0/608553025 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:36:26.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:26.313+0000 7f491affd640 1 --2- 172.21.15.38:0/608553025 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f491c0d1ed0 0x7f491c1457e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:26.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:26.313+0000 7f491bfff640 1 --2- 172.21.15.38:0/608553025 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f491c1536e0 0x7f491c145d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:26.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:26.314+0000 7f491b7fe640 1 -- 172.21.15.38:0/608553025 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f491c1536e0 msgr2=0x7f491c145d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:26.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:26.314+0000 7f491b7fe640 1 --2- 172.21.15.38:0/608553025 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f491c1536e0 0x7f491c145d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:26.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:26.314+0000 7f491b7fe640 1 -- 172.21.15.38:0/608553025 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f491c0d1ed0 msgr2=0x7f491c1457e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:26.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:26.314+0000 7f491b7fe640 1 --2- 172.21.15.38:0/608553025 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f491c0d1ed0 0x7f491c1457e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:26.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:26.314+0000 7f491b7fe640 1 -- 172.21.15.38:0/608553025 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f491c1f3cc0 con 0x7f491c0cce70 2024-09-21T03:36:26.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:26.314+0000 7f491b7fe640 1 --2- 172.21.15.38:0/608553025 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f491c0cce70 0x7f491c0cbd90 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f490c00c3a0 tx=0x7f490c00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:26.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:26.314+0000 7f4903fff640 1 -- 172.21.15.38:0/608553025 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f490c011790 con 0x7f491c0cce70 2024-09-21T03:36:26.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:26.314+0000 7f4903fff640 1 -- 172.21.15.38:0/608553025 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f490c017070 con 0x7f491c0cce70 2024-09-21T03:36:26.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:26.314+0000 7f49220b6640 1 -- 172.21.15.38:0/608553025 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f491c0a9ab0 con 0x7f491c0cce70 2024-09-21T03:36:26.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:26.314+0000 7f4903fff640 1 -- 172.21.15.38:0/608553025 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f490c011cc0 con 0x7f491c0cce70 2024-09-21T03:36:26.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:26.314+0000 7f49220b6640 1 -- 172.21.15.38:0/608553025 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f491c0a97d0 con 0x7f491c0cce70 2024-09-21T03:36:26.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:26.315+0000 7f49220b6640 1 -- 172.21.15.38:0/608553025 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f491c0d04c0 con 0x7f491c0cce70 2024-09-21T03:36:26.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:26.316+0000 7f4903fff640 1 -- 172.21.15.38:0/608553025 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f490c01c460 con 0x7f491c0cce70 2024-09-21T03:36:26.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:26.316+0000 7f4903fff640 1 --2- 172.21.15.38:0/608553025 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4908078bc0 0x7f490807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:26.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:26.316+0000 7f4903fff640 1 -- 172.21.15.38:0/608553025 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6101+0+0 (secure 0 0 0) 0x7f490c09ac10 con 0x7f491c0cce70 2024-09-21T03:36:26.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:26.317+0000 7f491affd640 1 --2- 172.21.15.38:0/608553025 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4908078bc0 0x7f490807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:26.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:26.317+0000 7f491affd640 1 --2- 172.21.15.38:0/608553025 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4908078bc0 0x7f490807b0d0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f4910034000 tx=0x7f4910004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:26.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:26.321+0000 7f4903fff640 1 -- 172.21.15.38:0/608553025 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f491c0a9ab0 con 0x7f491c0cce70 2024-09-21T03:36:26.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:26.504+0000 7f49220b6640 1 -- 172.21.15.38:0/608553025 --> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f491c0b7b60 con 0x7f4908078bc0 2024-09-21T03:36:26.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:26.508+0000 7f4903fff640 1 -- 172.21.15.38:0/608553025 <== mgr.4117 v2:172.21.15.195:6800/1631901344 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+249897 (secure 0 0 0) 0x7f491c0b7b60 con 0x7f4908078bc0 2024-09-21T03:36:26.512 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:36:26.513 INFO:teuthology.orchestra.run.smithi038.stderr:dumped all 2024-09-21T03:36:26.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:26.511+0000 7f49220b6640 1 -- 172.21.15.38:0/608553025 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4908078bc0 msgr2=0x7f490807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:26.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:26.511+0000 7f49220b6640 1 --2- 172.21.15.38:0/608553025 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4908078bc0 0x7f490807b0d0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f4910034000 tx=0x7f4910004730 comp rx=0 tx=0).stop 2024-09-21T03:36:26.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:26.511+0000 7f49220b6640 1 -- 172.21.15.38:0/608553025 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f491c0cce70 msgr2=0x7f491c0cbd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:26.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:26.511+0000 7f49220b6640 1 --2- 172.21.15.38:0/608553025 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f491c0cce70 0x7f491c0cbd90 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f490c00c3a0 tx=0x7f490c00c860 comp rx=0 tx=0).stop 2024-09-21T03:36:26.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:26.511+0000 7f49220b6640 1 -- 172.21.15.38:0/608553025 shutdown_connections 2024-09-21T03:36:26.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:26.511+0000 7f49220b6640 1 --2- 172.21.15.38:0/608553025 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4908078bc0 0x7f490807b0d0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:26.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:26.511+0000 7f49220b6640 1 --2- 172.21.15.38:0/608553025 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f491c1536e0 0x7f491c145d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:26.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:26.511+0000 7f49220b6640 1 --2- 172.21.15.38:0/608553025 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f491c0d1ed0 0x7f491c1457e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:26.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:26.511+0000 7f49220b6640 1 --2- 172.21.15.38:0/608553025 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f491c0cce70 0x7f491c0cbd90 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:26.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:26.511+0000 7f49220b6640 1 -- 172.21.15.38:0/608553025 >> 172.21.15.38:0/608553025 conn(0x7f491c0cef40 msgr2=0x7f491c0cf350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:26.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:26.511+0000 7f49220b6640 1 -- 172.21.15.38:0/608553025 shutdown_connections 2024-09-21T03:36:26.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:26.511+0000 7f49220b6640 1 -- 172.21.15.38:0/608553025 wait complete. 2024-09-21T03:36:26.526 INFO:teuthology.orchestra.run.smithi038.stdout:{"pg_ready":true,"pg_map":{"version":151,"stamp":"2024-09-21T03:36:25.023385+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591166,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":123,"num_write_kb":1639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":91,"ondisk_log_size":91,"up":174,"acting":174,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":117,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":273532,"kb_used_data":7352,"kb_used_omap":12,"kb_used_meta":265971,"kb_avail":754701188,"statfs":{"total":773094113280,"available":772814016512,"internally_reserved":0,"allocated":7528448,"data_stored":4031012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12748,"internal_metadata":272354868},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":798,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.985644"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-21T03:36:22.088822+0000","last_change":"2024-09-21T03:36:22.088324+0000","last_active":"2024-09-21T03:36:22.088822+0000","last_peered":"2024-09-21T03:36:22.088822+0000","last_clean":"2024-09-21T03:36:22.088822+0000","last_became_active":"2024-09-21T03:36:22.088010+0000","last_became_peered":"2024-09-21T03:36:22.088010+0000","last_unstale":"2024-09-21T03:36:22.088822+0000","last_undegraded":"2024-09-21T03:36:22.088822+0000","last_fullsized":"2024-09-21T03:36:22.088822+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:17:56.797134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-21T03:36:21.054741+0000","last_change":"2024-09-21T03:36:21.054741+0000","last_active":"2024-09-21T03:36:21.036908+0000","last_peered":"2024-09-21T03:36:21.036908+0000","last_clean":"2024-09-21T03:36:21.036908+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:36:21.054741+0000","last_undegraded":"2024-09-21T03:36:21.054741+0000","last_fullsized":"2024-09-21T03:36:21.054741+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.086324+0000","last_change":"2024-09-21T03:36:22.071184+0000","last_active":"2024-09-21T03:36:23.086324+0000","last_peered":"2024-09-21T03:36:23.086324+0000","last_clean":"2024-09-21T03:36:23.086324+0000","last_became_active":"2024-09-21T03:36:22.070772+0000","last_became_peered":"2024-09-21T03:36:22.070772+0000","last_unstale":"2024-09-21T03:36:23.086324+0000","last_undegraded":"2024-09-21T03:36:23.086324+0000","last_fullsized":"2024-09-21T03:36:23.086324+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:17:47.479714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.661468+0000","last_change":"2024-09-21T03:36:22.070400+0000","last_active":"2024-09-21T03:36:23.661468+0000","last_peered":"2024-09-21T03:36:23.661468+0000","last_clean":"2024-09-21T03:36:23.661468+0000","last_became_active":"2024-09-21T03:36:22.068959+0000","last_became_peered":"2024-09-21T03:36:22.068959+0000","last_unstale":"2024-09-21T03:36:23.661468+0000","last_undegraded":"2024-09-21T03:36:23.661468+0000","last_fullsized":"2024-09-21T03:36:23.661468+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:30:41.436799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.086049+0000","last_change":"2024-09-21T03:36:22.081653+0000","last_active":"2024-09-21T03:36:23.086049+0000","last_peered":"2024-09-21T03:36:23.086049+0000","last_clean":"2024-09-21T03:36:23.086049+0000","last_became_active":"2024-09-21T03:36:22.081343+0000","last_became_peered":"2024-09-21T03:36:22.081343+0000","last_unstale":"2024-09-21T03:36:23.086049+0000","last_undegraded":"2024-09-21T03:36:23.086049+0000","last_fullsized":"2024-09-21T03:36:23.086049+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:37:46.632334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-21T03:36:21.055099+0000","last_change":"2024-09-21T03:36:21.055099+0000","last_active":"2024-09-21T03:36:21.036908+0000","last_peered":"2024-09-21T03:36:21.036908+0000","last_clean":"2024-09-21T03:36:21.036908+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:36:21.055099+0000","last_undegraded":"2024-09-21T03:36:21.055099+0000","last_fullsized":"2024-09-21T03:36:21.055099+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-21T03:36:22.075723+0000","last_change":"2024-09-21T03:36:22.074034+0000","last_active":"2024-09-21T03:36:22.075723+0000","last_peered":"2024-09-21T03:36:22.075723+0000","last_clean":"2024-09-21T03:36:22.075723+0000","last_became_active":"2024-09-21T03:36:22.073682+0000","last_became_peered":"2024-09-21T03:36:22.073682+0000","last_unstale":"2024-09-21T03:36:22.075723+0000","last_undegraded":"2024-09-21T03:36:22.075723+0000","last_fullsized":"2024-09-21T03:36:22.075723+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:25:30.522013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.085192+0000","last_change":"2024-09-21T03:36:22.080482+0000","last_active":"2024-09-21T03:36:23.085192+0000","last_peered":"2024-09-21T03:36:23.085192+0000","last_clean":"2024-09-21T03:36:23.085192+0000","last_became_active":"2024-09-21T03:36:22.080177+0000","last_became_peered":"2024-09-21T03:36:22.080177+0000","last_unstale":"2024-09-21T03:36:23.085192+0000","last_undegraded":"2024-09-21T03:36:23.085192+0000","last_fullsized":"2024-09-21T03:36:23.085192+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:56:46.927190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.661540+0000","last_change":"2024-09-21T03:36:22.077361+0000","last_active":"2024-09-21T03:36:23.661540+0000","last_peered":"2024-09-21T03:36:23.661540+0000","last_clean":"2024-09-21T03:36:23.661540+0000","last_became_active":"2024-09-21T03:36:22.077050+0000","last_became_peered":"2024-09-21T03:36:22.077050+0000","last_unstale":"2024-09-21T03:36:23.661540+0000","last_undegraded":"2024-09-21T03:36:23.661540+0000","last_fullsized":"2024-09-21T03:36:23.661540+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:30:24.348269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T03:36:21.036908+0000","last_change":"2024-09-21T03:36:21.036908+0000","last_active":"2024-09-21T03:36:21.036908+0000","last_peered":"2024-09-21T03:36:21.036908+0000","last_clean":"2024-09-21T03:36:21.036908+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:36:21.036908+0000","last_undegraded":"2024-09-21T03:36:21.036908+0000","last_fullsized":"2024-09-21T03:36:21.036908+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.661250+0000","last_change":"2024-09-21T03:36:22.072761+0000","last_active":"2024-09-21T03:36:23.661250+0000","last_peered":"2024-09-21T03:36:23.661250+0000","last_clean":"2024-09-21T03:36:23.661250+0000","last_became_active":"2024-09-21T03:36:22.072450+0000","last_became_peered":"2024-09-21T03:36:22.072450+0000","last_unstale":"2024-09-21T03:36:23.661250+0000","last_undegraded":"2024-09-21T03:36:23.661250+0000","last_fullsized":"2024-09-21T03:36:23.661250+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:04:45.989423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.661340+0000","last_change":"2024-09-21T03:36:22.086026+0000","last_active":"2024-09-21T03:36:23.661340+0000","last_peered":"2024-09-21T03:36:23.661340+0000","last_clean":"2024-09-21T03:36:23.661340+0000","last_became_active":"2024-09-21T03:36:22.085717+0000","last_became_peered":"2024-09-21T03:36:22.085717+0000","last_unstale":"2024-09-21T03:36:23.661340+0000","last_undegraded":"2024-09-21T03:36:23.661340+0000","last_fullsized":"2024-09-21T03:36:23.661340+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:47:51.103846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.660593+0000","last_change":"2024-09-21T03:36:22.090483+0000","last_active":"2024-09-21T03:36:23.660593+0000","last_peered":"2024-09-21T03:36:23.660593+0000","last_clean":"2024-09-21T03:36:23.660593+0000","last_became_active":"2024-09-21T03:36:22.090124+0000","last_became_peered":"2024-09-21T03:36:22.090124+0000","last_unstale":"2024-09-21T03:36:23.660593+0000","last_undegraded":"2024-09-21T03:36:23.660593+0000","last_fullsized":"2024-09-21T03:36:23.660593+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:52:19.716709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.660532+0000","last_change":"2024-09-21T03:36:22.080680+0000","last_active":"2024-09-21T03:36:23.660532+0000","last_peered":"2024-09-21T03:36:23.660532+0000","last_clean":"2024-09-21T03:36:23.660532+0000","last_became_active":"2024-09-21T03:36:22.080377+0000","last_became_peered":"2024-09-21T03:36:22.080377+0000","last_unstale":"2024-09-21T03:36:23.660532+0000","last_undegraded":"2024-09-21T03:36:23.660532+0000","last_fullsized":"2024-09-21T03:36:23.660532+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:54:33.394204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T03:36:21.036908+0000","last_change":"2024-09-21T03:36:21.036908+0000","last_active":"2024-09-21T03:36:21.036908+0000","last_peered":"2024-09-21T03:36:21.036908+0000","last_clean":"2024-09-21T03:36:21.036908+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:36:21.036908+0000","last_undegraded":"2024-09-21T03:36:21.036908+0000","last_fullsized":"2024-09-21T03:36:21.036908+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.660634+0000","last_change":"2024-09-21T03:36:22.085437+0000","last_active":"2024-09-21T03:36:23.660634+0000","last_peered":"2024-09-21T03:36:23.660634+0000","last_clean":"2024-09-21T03:36:23.660634+0000","last_became_active":"2024-09-21T03:36:22.085087+0000","last_became_peered":"2024-09-21T03:36:22.085087+0000","last_unstale":"2024-09-21T03:36:23.660634+0000","last_undegraded":"2024-09-21T03:36:23.660634+0000","last_fullsized":"2024-09-21T03:36:23.660634+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:30:00.925328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-21T03:36:22.082591+0000","last_change":"2024-09-21T03:36:22.081251+0000","last_active":"2024-09-21T03:36:22.082591+0000","last_peered":"2024-09-21T03:36:22.082591+0000","last_clean":"2024-09-21T03:36:22.082591+0000","last_became_active":"2024-09-21T03:36:22.080931+0000","last_became_peered":"2024-09-21T03:36:22.080931+0000","last_unstale":"2024-09-21T03:36:22.082591+0000","last_undegraded":"2024-09-21T03:36:22.082591+0000","last_fullsized":"2024-09-21T03:36:22.082591+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:04:34.873429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.661300+0000","last_change":"2024-09-21T03:36:22.069803+0000","last_active":"2024-09-21T03:36:23.661300+0000","last_peered":"2024-09-21T03:36:23.661300+0000","last_clean":"2024-09-21T03:36:23.661300+0000","last_became_active":"2024-09-21T03:36:22.069481+0000","last_became_peered":"2024-09-21T03:36:22.069481+0000","last_unstale":"2024-09-21T03:36:23.661300+0000","last_undegraded":"2024-09-21T03:36:23.661300+0000","last_fullsized":"2024-09-21T03:36:23.661300+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:15:44.483810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.661639+0000","last_change":"2024-09-21T03:36:22.076014+0000","last_active":"2024-09-21T03:36:23.661639+0000","last_peered":"2024-09-21T03:36:23.661639+0000","last_clean":"2024-09-21T03:36:23.661639+0000","last_became_active":"2024-09-21T03:36:22.075689+0000","last_became_peered":"2024-09-21T03:36:22.075689+0000","last_unstale":"2024-09-21T03:36:23.661639+0000","last_undegraded":"2024-09-21T03:36:23.661639+0000","last_fullsized":"2024-09-21T03:36:23.661639+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:47:25.173315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-21T03:36:22.073135+0000","last_change":"2024-09-21T03:36:22.071709+0000","last_active":"2024-09-21T03:36:22.073135+0000","last_peered":"2024-09-21T03:36:22.073135+0000","last_clean":"2024-09-21T03:36:22.073135+0000","last_became_active":"2024-09-21T03:36:22.070413+0000","last_became_peered":"2024-09-21T03:36:22.070413+0000","last_unstale":"2024-09-21T03:36:22.073135+0000","last_undegraded":"2024-09-21T03:36:22.073135+0000","last_fullsized":"2024-09-21T03:36:22.073135+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:16:55.679529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.660747+0000","last_change":"2024-09-21T03:36:22.086945+0000","last_active":"2024-09-21T03:36:23.660747+0000","last_peered":"2024-09-21T03:36:23.660747+0000","last_clean":"2024-09-21T03:36:23.660747+0000","last_became_active":"2024-09-21T03:36:22.086620+0000","last_became_peered":"2024-09-21T03:36:22.086620+0000","last_unstale":"2024-09-21T03:36:23.660747+0000","last_undegraded":"2024-09-21T03:36:23.660747+0000","last_fullsized":"2024-09-21T03:36:23.660747+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:48:46.104173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-21T03:36:21.053928+0000","last_change":"2024-09-21T03:36:21.053928+0000","last_active":"2024-09-21T03:36:21.036908+0000","last_peered":"2024-09-21T03:36:21.036908+0000","last_clean":"2024-09-21T03:36:21.036908+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:36:21.053928+0000","last_undegraded":"2024-09-21T03:36:21.053928+0000","last_fullsized":"2024-09-21T03:36:21.053928+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.660604+0000","last_change":"2024-09-21T03:36:22.086122+0000","last_active":"2024-09-21T03:36:23.660604+0000","last_peered":"2024-09-21T03:36:23.660604+0000","last_clean":"2024-09-21T03:36:23.660604+0000","last_became_active":"2024-09-21T03:36:22.085783+0000","last_became_peered":"2024-09-21T03:36:22.085783+0000","last_unstale":"2024-09-21T03:36:23.660604+0000","last_undegraded":"2024-09-21T03:36:23.660604+0000","last_fullsized":"2024-09-21T03:36:23.660604+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:05:24.777050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.085420+0000","last_change":"2024-09-21T03:36:22.067412+0000","last_active":"2024-09-21T03:36:23.085420+0000","last_peered":"2024-09-21T03:36:23.085420+0000","last_clean":"2024-09-21T03:36:23.085420+0000","last_became_active":"2024-09-21T03:36:22.067084+0000","last_became_peered":"2024-09-21T03:36:22.067084+0000","last_unstale":"2024-09-21T03:36:23.085420+0000","last_undegraded":"2024-09-21T03:36:23.085420+0000","last_fullsized":"2024-09-21T03:36:23.085420+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:01:59.040840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.661165+0000","last_change":"2024-09-21T03:36:22.082532+0000","last_active":"2024-09-21T03:36:23.661165+0000","last_peered":"2024-09-21T03:36:23.661165+0000","last_clean":"2024-09-21T03:36:23.661165+0000","last_became_active":"2024-09-21T03:36:22.082233+0000","last_became_peered":"2024-09-21T03:36:22.082233+0000","last_unstale":"2024-09-21T03:36:23.661165+0000","last_undegraded":"2024-09-21T03:36:23.661165+0000","last_fullsized":"2024-09-21T03:36:23.661165+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:09:57.485058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.086025+0000","last_change":"2024-09-21T03:36:22.068982+0000","last_active":"2024-09-21T03:36:23.086025+0000","last_peered":"2024-09-21T03:36:23.086025+0000","last_clean":"2024-09-21T03:36:23.086025+0000","last_became_active":"2024-09-21T03:36:22.068584+0000","last_became_peered":"2024-09-21T03:36:22.068584+0000","last_unstale":"2024-09-21T03:36:23.086025+0000","last_undegraded":"2024-09-21T03:36:23.086025+0000","last_fullsized":"2024-09-21T03:36:23.086025+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:02:01.275514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.660817+0000","last_change":"2024-09-21T03:36:22.085467+0000","last_active":"2024-09-21T03:36:23.660817+0000","last_peered":"2024-09-21T03:36:23.660817+0000","last_clean":"2024-09-21T03:36:23.660817+0000","last_became_active":"2024-09-21T03:36:22.085127+0000","last_became_peered":"2024-09-21T03:36:22.085127+0000","last_unstale":"2024-09-21T03:36:23.660817+0000","last_undegraded":"2024-09-21T03:36:23.660817+0000","last_fullsized":"2024-09-21T03:36:23.660817+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:04:32.859631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-21T03:36:22.084080+0000","last_change":"2024-09-21T03:36:22.083174+0000","last_active":"2024-09-21T03:36:22.084080+0000","last_peered":"2024-09-21T03:36:22.084080+0000","last_clean":"2024-09-21T03:36:22.084080+0000","last_became_active":"2024-09-21T03:36:22.082930+0000","last_became_peered":"2024-09-21T03:36:22.082930+0000","last_unstale":"2024-09-21T03:36:22.084080+0000","last_undegraded":"2024-09-21T03:36:22.084080+0000","last_fullsized":"2024-09-21T03:36:22.084080+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:09:50.811454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.661615+0000","last_change":"2024-09-21T03:36:22.088889+0000","last_active":"2024-09-21T03:36:23.661615+0000","last_peered":"2024-09-21T03:36:23.661615+0000","last_clean":"2024-09-21T03:36:23.661615+0000","last_became_active":"2024-09-21T03:36:22.088515+0000","last_became_peered":"2024-09-21T03:36:22.088515+0000","last_unstale":"2024-09-21T03:36:23.661615+0000","last_undegraded":"2024-09-21T03:36:23.661615+0000","last_fullsized":"2024-09-21T03:36:23.661615+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:00:06.850866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.660822+0000","last_change":"2024-09-21T03:36:22.088463+0000","last_active":"2024-09-21T03:36:23.660822+0000","last_peered":"2024-09-21T03:36:23.660822+0000","last_clean":"2024-09-21T03:36:23.660822+0000","last_became_active":"2024-09-21T03:36:22.088149+0000","last_became_peered":"2024-09-21T03:36:22.088149+0000","last_unstale":"2024-09-21T03:36:23.660822+0000","last_undegraded":"2024-09-21T03:36:23.660822+0000","last_fullsized":"2024-09-21T03:36:23.660822+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:03:04.392175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.085608+0000","last_change":"2024-09-21T03:36:22.066107+0000","last_active":"2024-09-21T03:36:23.085608+0000","last_peered":"2024-09-21T03:36:23.085608+0000","last_clean":"2024-09-21T03:36:23.085608+0000","last_became_active":"2024-09-21T03:36:22.064917+0000","last_became_peered":"2024-09-21T03:36:22.064917+0000","last_unstale":"2024-09-21T03:36:23.085608+0000","last_undegraded":"2024-09-21T03:36:23.085608+0000","last_fullsized":"2024-09-21T03:36:23.085608+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:58:06.692992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-21T03:36:22.072148+0000","last_change":"2024-09-21T03:36:22.069000+0000","last_active":"2024-09-21T03:36:22.072148+0000","last_peered":"2024-09-21T03:36:22.072148+0000","last_clean":"2024-09-21T03:36:22.072148+0000","last_became_active":"2024-09-21T03:36:22.068661+0000","last_became_peered":"2024-09-21T03:36:22.068661+0000","last_unstale":"2024-09-21T03:36:22.072148+0000","last_undegraded":"2024-09-21T03:36:22.072148+0000","last_fullsized":"2024-09-21T03:36:22.072148+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:39:01.505213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.661745+0000","last_change":"2024-09-21T03:36:22.071976+0000","last_active":"2024-09-21T03:36:23.661745+0000","last_peered":"2024-09-21T03:36:23.661745+0000","last_clean":"2024-09-21T03:36:23.661745+0000","last_became_active":"2024-09-21T03:36:22.071182+0000","last_became_peered":"2024-09-21T03:36:22.071182+0000","last_unstale":"2024-09-21T03:36:23.661745+0000","last_undegraded":"2024-09-21T03:36:23.661745+0000","last_fullsized":"2024-09-21T03:36:23.661745+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:58:44.343782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1f","version":"66'4","reported_seq":17,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-21T03:36:22.081942+0000","last_change":"2024-09-21T03:36:21.066587+0000","last_active":"2024-09-21T03:36:22.081942+0000","last_peered":"2024-09-21T03:36:22.081942+0000","last_clean":"2024-09-21T03:36:22.081942+0000","last_became_active":"2024-09-21T03:36:21.066299+0000","last_became_peered":"2024-09-21T03:36:21.066299+0000","last_unstale":"2024-09-21T03:36:22.081942+0000","last_undegraded":"2024-09-21T03:36:22.081942+0000","last_fullsized":"2024-09-21T03:36:22.081942+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_clean_scrub_stamp":"2024-09-21T03:36:20.027499+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:06:51.671380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.086591+0000","last_change":"2024-09-21T03:36:22.073629+0000","last_active":"2024-09-21T03:36:23.086591+0000","last_peered":"2024-09-21T03:36:23.086591+0000","last_clean":"2024-09-21T03:36:23.086591+0000","last_became_active":"2024-09-21T03:36:22.073211+0000","last_became_peered":"2024-09-21T03:36:22.073211+0000","last_unstale":"2024-09-21T03:36:23.086591+0000","last_undegraded":"2024-09-21T03:36:23.086591+0000","last_fullsized":"2024-09-21T03:36:23.086591+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:27:32.381253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1e","version":"66'1","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-21T03:36:22.092834+0000","last_change":"2024-09-21T03:36:21.064384+0000","last_active":"2024-09-21T03:36:22.092834+0000","last_peered":"2024-09-21T03:36:22.092834+0000","last_clean":"2024-09-21T03:36:22.092834+0000","last_became_active":"2024-09-21T03:36:21.064042+0000","last_became_peered":"2024-09-21T03:36:21.064042+0000","last_unstale":"2024-09-21T03:36:22.092834+0000","last_undegraded":"2024-09-21T03:36:22.092834+0000","last_fullsized":"2024-09-21T03:36:22.092834+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_clean_scrub_stamp":"2024-09-21T03:36:20.027499+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:17:25.045895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-21T03:36:21.054153+0000","last_change":"2024-09-21T03:36:21.054153+0000","last_active":"2024-09-21T03:36:21.036908+0000","last_peered":"2024-09-21T03:36:21.036908+0000","last_clean":"2024-09-21T03:36:21.036908+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:36:21.054153+0000","last_undegraded":"2024-09-21T03:36:21.054153+0000","last_fullsized":"2024-09-21T03:36:21.054153+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1d","version":"66'2","reported_seq":15,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-21T03:36:22.080793+0000","last_change":"2024-09-21T03:36:21.071133+0000","last_active":"2024-09-21T03:36:22.080793+0000","last_peered":"2024-09-21T03:36:22.080793+0000","last_clean":"2024-09-21T03:36:22.080793+0000","last_became_active":"2024-09-21T03:36:21.070834+0000","last_became_peered":"2024-09-21T03:36:21.070834+0000","last_unstale":"2024-09-21T03:36:22.080793+0000","last_undegraded":"2024-09-21T03:36:22.080793+0000","last_fullsized":"2024-09-21T03:36:22.080793+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_clean_scrub_stamp":"2024-09-21T03:36:20.027499+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:37:25.878136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.661630+0000","last_change":"2024-09-21T03:36:22.070080+0000","last_active":"2024-09-21T03:36:23.661630+0000","last_peered":"2024-09-21T03:36:23.661630+0000","last_clean":"2024-09-21T03:36:23.661630+0000","last_became_active":"2024-09-21T03:36:22.069779+0000","last_became_peered":"2024-09-21T03:36:22.069779+0000","last_unstale":"2024-09-21T03:36:23.661630+0000","last_undegraded":"2024-09-21T03:36:23.661630+0000","last_fullsized":"2024-09-21T03:36:23.661630+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:29:10.577562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-21T03:36:21.066934+0000","last_change":"2024-09-21T03:36:21.065338+0000","last_active":"2024-09-21T03:36:21.066934+0000","last_peered":"2024-09-21T03:36:21.066934+0000","last_clean":"2024-09-21T03:36:21.066934+0000","last_became_active":"2024-09-21T03:36:21.065032+0000","last_became_peered":"2024-09-21T03:36:21.065032+0000","last_unstale":"2024-09-21T03:36:21.066934+0000","last_undegraded":"2024-09-21T03:36:21.066934+0000","last_fullsized":"2024-09-21T03:36:21.066934+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_clean_scrub_stamp":"2024-09-21T03:36:20.027499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:38:09.536350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-21T03:36:22.065482+0000","last_change":"2024-09-21T03:36:22.063213+0000","last_active":"2024-09-21T03:36:22.065482+0000","last_peered":"2024-09-21T03:36:22.065482+0000","last_clean":"2024-09-21T03:36:22.065482+0000","last_became_active":"2024-09-21T03:36:22.062870+0000","last_became_peered":"2024-09-21T03:36:22.062870+0000","last_unstale":"2024-09-21T03:36:22.065482+0000","last_undegraded":"2024-09-21T03:36:22.065482+0000","last_fullsized":"2024-09-21T03:36:22.065482+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:07:32.687989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"66'1","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-21T03:36:22.079299+0000","last_change":"2024-09-21T03:36:21.060507+0000","last_active":"2024-09-21T03:36:22.079299+0000","last_peered":"2024-09-21T03:36:22.079299+0000","last_clean":"2024-09-21T03:36:22.079299+0000","last_became_active":"2024-09-21T03:36:21.060129+0000","last_became_peered":"2024-09-21T03:36:21.060129+0000","last_unstale":"2024-09-21T03:36:22.079299+0000","last_undegraded":"2024-09-21T03:36:22.079299+0000","last_fullsized":"2024-09-21T03:36:22.079299+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_clean_scrub_stamp":"2024-09-21T03:36:20.027499+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:57:03.979240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-21T03:36:22.080880+0000","last_change":"2024-09-21T03:36:22.077206+0000","last_active":"2024-09-21T03:36:22.080880+0000","last_peered":"2024-09-21T03:36:22.080880+0000","last_clean":"2024-09-21T03:36:22.080880+0000","last_became_active":"2024-09-21T03:36:22.076897+0000","last_became_peered":"2024-09-21T03:36:22.076897+0000","last_unstale":"2024-09-21T03:36:22.080880+0000","last_undegraded":"2024-09-21T03:36:22.080880+0000","last_fullsized":"2024-09-21T03:36:22.080880+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:39:58.170006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-21T03:36:21.062625+0000","last_change":"2024-09-21T03:36:21.059505+0000","last_active":"2024-09-21T03:36:21.062625+0000","last_peered":"2024-09-21T03:36:21.062625+0000","last_clean":"2024-09-21T03:36:21.062625+0000","last_became_active":"2024-09-21T03:36:21.059110+0000","last_became_peered":"2024-09-21T03:36:21.059110+0000","last_unstale":"2024-09-21T03:36:21.062625+0000","last_undegraded":"2024-09-21T03:36:21.062625+0000","last_fullsized":"2024-09-21T03:36:21.062625+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_clean_scrub_stamp":"2024-09-21T03:36:20.027499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:47:13.398068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T03:36:21.036908+0000","last_change":"2024-09-21T03:36:21.036908+0000","last_active":"2024-09-21T03:36:21.036908+0000","last_peered":"2024-09-21T03:36:21.036908+0000","last_clean":"2024-09-21T03:36:21.036908+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:36:21.036908+0000","last_undegraded":"2024-09-21T03:36:21.036908+0000","last_fullsized":"2024-09-21T03:36:21.036908+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":13,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-21T03:36:22.065557+0000","last_change":"2024-09-21T03:36:21.065608+0000","last_active":"2024-09-21T03:36:22.065557+0000","last_peered":"2024-09-21T03:36:22.065557+0000","last_clean":"2024-09-21T03:36:22.065557+0000","last_became_active":"2024-09-21T03:36:21.065292+0000","last_became_peered":"2024-09-21T03:36:21.065292+0000","last_unstale":"2024-09-21T03:36:22.065557+0000","last_undegraded":"2024-09-21T03:36:22.065557+0000","last_fullsized":"2024-09-21T03:36:22.065557+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_clean_scrub_stamp":"2024-09-21T03:36:20.027499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:36:50.067003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-21T03:36:22.072312+0000","last_change":"2024-09-21T03:36:22.069640+0000","last_active":"2024-09-21T03:36:22.072312+0000","last_peered":"2024-09-21T03:36:22.072312+0000","last_clean":"2024-09-21T03:36:22.072312+0000","last_became_active":"2024-09-21T03:36:22.068836+0000","last_became_peered":"2024-09-21T03:36:22.068836+0000","last_unstale":"2024-09-21T03:36:22.072312+0000","last_undegraded":"2024-09-21T03:36:22.072312+0000","last_fullsized":"2024-09-21T03:36:22.072312+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:29:10.577562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.660609+0000","last_change":"2024-09-21T03:36:21.062628+0000","last_active":"2024-09-21T03:36:23.660609+0000","last_peered":"2024-09-21T03:36:23.660609+0000","last_clean":"2024-09-21T03:36:23.660609+0000","last_became_active":"2024-09-21T03:36:21.062295+0000","last_became_peered":"2024-09-21T03:36:21.062295+0000","last_unstale":"2024-09-21T03:36:23.660609+0000","last_undegraded":"2024-09-21T03:36:23.660609+0000","last_fullsized":"2024-09-21T03:36:23.660609+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_clean_scrub_stamp":"2024-09-21T03:36:20.027499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:20:15.258373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-21T03:36:22.069673+0000","last_change":"2024-09-21T03:36:22.067826+0000","last_active":"2024-09-21T03:36:22.069673+0000","last_peered":"2024-09-21T03:36:22.069673+0000","last_clean":"2024-09-21T03:36:22.069673+0000","last_became_active":"2024-09-21T03:36:22.067516+0000","last_became_peered":"2024-09-21T03:36:22.067516+0000","last_unstale":"2024-09-21T03:36:22.069673+0000","last_undegraded":"2024-09-21T03:36:22.069673+0000","last_fullsized":"2024-09-21T03:36:22.069673+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:20:16.267783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.661694+0000","last_change":"2024-09-21T03:36:21.063524+0000","last_active":"2024-09-21T03:36:23.661694+0000","last_peered":"2024-09-21T03:36:23.661694+0000","last_clean":"2024-09-21T03:36:23.661694+0000","last_became_active":"2024-09-21T03:36:21.063203+0000","last_became_peered":"2024-09-21T03:36:21.063203+0000","last_unstale":"2024-09-21T03:36:23.661694+0000","last_undegraded":"2024-09-21T03:36:23.661694+0000","last_fullsized":"2024-09-21T03:36:23.661694+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_clean_scrub_stamp":"2024-09-21T03:36:20.027499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:25:29.512603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-21T03:36:22.065852+0000","last_change":"2024-09-21T03:36:22.063211+0000","last_active":"2024-09-21T03:36:22.065852+0000","last_peered":"2024-09-21T03:36:22.065852+0000","last_clean":"2024-09-21T03:36:22.065852+0000","last_became_active":"2024-09-21T03:36:22.062828+0000","last_became_peered":"2024-09-21T03:36:22.062828+0000","last_unstale":"2024-09-21T03:36:22.065852+0000","last_undegraded":"2024-09-21T03:36:22.065852+0000","last_fullsized":"2024-09-21T03:36:22.065852+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:30:41.436799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.661311+0000","last_change":"2024-09-21T03:36:21.064314+0000","last_active":"2024-09-21T03:36:23.661311+0000","last_peered":"2024-09-21T03:36:23.661311+0000","last_clean":"2024-09-21T03:36:23.661311+0000","last_became_active":"2024-09-21T03:36:21.063271+0000","last_became_peered":"2024-09-21T03:36:21.063271+0000","last_unstale":"2024-09-21T03:36:23.661311+0000","last_undegraded":"2024-09-21T03:36:23.661311+0000","last_fullsized":"2024-09-21T03:36:23.661311+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_clean_scrub_stamp":"2024-09-21T03:36:20.027499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:03:09.222194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.660928+0000","last_change":"2024-09-21T03:36:22.092139+0000","last_active":"2024-09-21T03:36:23.660928+0000","last_peered":"2024-09-21T03:36:23.660928+0000","last_clean":"2024-09-21T03:36:23.660928+0000","last_became_active":"2024-09-21T03:36:22.091718+0000","last_became_peered":"2024-09-21T03:36:22.091718+0000","last_unstale":"2024-09-21T03:36:23.660928+0000","last_undegraded":"2024-09-21T03:36:23.660928+0000","last_fullsized":"2024-09-21T03:36:23.660928+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:28:40.036155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.15","version":"66'3","reported_seq":18,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.086559+0000","last_change":"2024-09-21T03:36:21.064387+0000","last_active":"2024-09-21T03:36:23.086559+0000","last_peered":"2024-09-21T03:36:23.086559+0000","last_clean":"2024-09-21T03:36:23.086559+0000","last_became_active":"2024-09-21T03:36:21.064086+0000","last_became_peered":"2024-09-21T03:36:21.064086+0000","last_unstale":"2024-09-21T03:36:23.086559+0000","last_undegraded":"2024-09-21T03:36:23.086559+0000","last_fullsized":"2024-09-21T03:36:23.086559+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_clean_scrub_stamp":"2024-09-21T03:36:20.027499+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:01:58.431275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.086256+0000","last_change":"2024-09-21T03:36:22.070639+0000","last_active":"2024-09-21T03:36:23.086256+0000","last_peered":"2024-09-21T03:36:23.086256+0000","last_clean":"2024-09-21T03:36:23.086256+0000","last_became_active":"2024-09-21T03:36:22.069620+0000","last_became_peered":"2024-09-21T03:36:22.069620+0000","last_unstale":"2024-09-21T03:36:23.086256+0000","last_undegraded":"2024-09-21T03:36:23.086256+0000","last_fullsized":"2024-09-21T03:36:23.086256+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:23:36.147744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-21T03:36:21.066007+0000","last_change":"2024-09-21T03:36:21.065091+0000","last_active":"2024-09-21T03:36:21.066007+0000","last_peered":"2024-09-21T03:36:21.066007+0000","last_clean":"2024-09-21T03:36:21.066007+0000","last_became_active":"2024-09-21T03:36:21.064760+0000","last_became_peered":"2024-09-21T03:36:21.064760+0000","last_unstale":"2024-09-21T03:36:21.066007+0000","last_undegraded":"2024-09-21T03:36:21.066007+0000","last_fullsized":"2024-09-21T03:36:21.066007+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_clean_scrub_stamp":"2024-09-21T03:36:20.027499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:17:34.450050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T03:36:21.036908+0000","last_change":"2024-09-21T03:36:21.036908+0000","last_active":"2024-09-21T03:36:21.036908+0000","last_peered":"2024-09-21T03:36:21.036908+0000","last_clean":"2024-09-21T03:36:21.036908+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:36:21.036908+0000","last_undegraded":"2024-09-21T03:36:21.036908+0000","last_fullsized":"2024-09-21T03:36:21.036908+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.13","version":"66'2","reported_seq":17,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.661866+0000","last_change":"2024-09-21T03:36:21.064320+0000","last_active":"2024-09-21T03:36:23.661866+0000","last_peered":"2024-09-21T03:36:23.661866+0000","last_clean":"2024-09-21T03:36:23.661866+0000","last_became_active":"2024-09-21T03:36:21.064030+0000","last_became_peered":"2024-09-21T03:36:21.064030+0000","last_unstale":"2024-09-21T03:36:23.661866+0000","last_undegraded":"2024-09-21T03:36:23.661866+0000","last_fullsized":"2024-09-21T03:36:23.661866+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_clean_scrub_stamp":"2024-09-21T03:36:20.027499+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:36:50.067003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T03:36:21.036908+0000","last_change":"2024-09-21T03:36:21.036908+0000","last_active":"2024-09-21T03:36:21.036908+0000","last_peered":"2024-09-21T03:36:21.036908+0000","last_clean":"2024-09-21T03:36:21.036908+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:36:21.036908+0000","last_undegraded":"2024-09-21T03:36:21.036908+0000","last_fullsized":"2024-09-21T03:36:21.036908+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T03:36:20.027499+0000","last_change":"2024-09-21T03:36:20.027499+0000","last_active":"2024-09-21T03:36:20.027499+0000","last_peered":"2024-09-21T03:36:20.027499+0000","last_clean":"2024-09-21T03:36:20.027499+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:36:20.027499+0000","last_undegraded":"2024-09-21T03:36:20.027499+0000","last_fullsized":"2024-09-21T03:36:20.027499+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_clean_scrub_stamp":"2024-09-21T03:36:20.027499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T03:36:21.036908+0000","last_change":"2024-09-21T03:36:21.036908+0000","last_active":"2024-09-21T03:36:21.036908+0000","last_peered":"2024-09-21T03:36:21.036908+0000","last_clean":"2024-09-21T03:36:21.036908+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:36:21.036908+0000","last_undegraded":"2024-09-21T03:36:21.036908+0000","last_fullsized":"2024-09-21T03:36:21.036908+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.085748+0000","last_change":"2024-09-21T03:36:21.061510+0000","last_active":"2024-09-21T03:36:23.085748+0000","last_peered":"2024-09-21T03:36:23.085748+0000","last_clean":"2024-09-21T03:36:23.085748+0000","last_became_active":"2024-09-21T03:36:21.061181+0000","last_became_peered":"2024-09-21T03:36:21.061181+0000","last_unstale":"2024-09-21T03:36:23.085748+0000","last_undegraded":"2024-09-21T03:36:23.085748+0000","last_fullsized":"2024-09-21T03:36:23.085748+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_clean_scrub_stamp":"2024-09-21T03:36:20.027499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:39:48.217217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-21T03:36:22.086623+0000","last_change":"2024-09-21T03:36:22.085750+0000","last_active":"2024-09-21T03:36:22.086623+0000","last_peered":"2024-09-21T03:36:22.086623+0000","last_clean":"2024-09-21T03:36:22.086623+0000","last_became_active":"2024-09-21T03:36:22.085426+0000","last_became_peered":"2024-09-21T03:36:22.085426+0000","last_unstale":"2024-09-21T03:36:22.086623+0000","last_undegraded":"2024-09-21T03:36:22.086623+0000","last_fullsized":"2024-09-21T03:36:22.086623+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:43:37.785209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T03:36:20.027499+0000","last_change":"2024-09-21T03:36:20.027499+0000","last_active":"2024-09-21T03:36:20.027499+0000","last_peered":"2024-09-21T03:36:20.027499+0000","last_clean":"2024-09-21T03:36:20.027499+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:36:20.027499+0000","last_undegraded":"2024-09-21T03:36:20.027499+0000","last_fullsized":"2024-09-21T03:36:20.027499+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_clean_scrub_stamp":"2024-09-21T03:36:20.027499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.661909+0000","last_change":"2024-09-21T03:36:22.086802+0000","last_active":"2024-09-21T03:36:23.661909+0000","last_peered":"2024-09-21T03:36:23.661909+0000","last_clean":"2024-09-21T03:36:23.661909+0000","last_became_active":"2024-09-21T03:36:22.086435+0000","last_became_peered":"2024-09-21T03:36:22.086435+0000","last_unstale":"2024-09-21T03:36:23.661909+0000","last_undegraded":"2024-09-21T03:36:23.661909+0000","last_fullsized":"2024-09-21T03:36:23.661909+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:32:26.377539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.f","version":"66'1","reported_seq":16,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.661756+0000","last_change":"2024-09-21T03:36:21.062807+0000","last_active":"2024-09-21T03:36:23.661756+0000","last_peered":"2024-09-21T03:36:23.661756+0000","last_clean":"2024-09-21T03:36:23.661756+0000","last_became_active":"2024-09-21T03:36:21.062513+0000","last_became_peered":"2024-09-21T03:36:21.062513+0000","last_unstale":"2024-09-21T03:36:23.661756+0000","last_undegraded":"2024-09-21T03:36:23.661756+0000","last_fullsized":"2024-09-21T03:36:23.661756+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_clean_scrub_stamp":"2024-09-21T03:36:20.027499+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:39:57.160597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.660925+0000","last_change":"2024-09-21T03:36:22.086393+0000","last_active":"2024-09-21T03:36:23.660925+0000","last_peered":"2024-09-21T03:36:23.660925+0000","last_clean":"2024-09-21T03:36:23.660925+0000","last_became_active":"2024-09-21T03:36:22.086094+0000","last_became_peered":"2024-09-21T03:36:22.086094+0000","last_unstale":"2024-09-21T03:36:23.660925+0000","last_undegraded":"2024-09-21T03:36:23.660925+0000","last_fullsized":"2024-09-21T03:36:23.660925+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:11:34.432933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.e","version":"66'2","reported_seq":17,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.661138+0000","last_change":"2024-09-21T03:36:21.053740+0000","last_active":"2024-09-21T03:36:23.661138+0000","last_peered":"2024-09-21T03:36:23.661138+0000","last_clean":"2024-09-21T03:36:23.661138+0000","last_became_active":"2024-09-21T03:36:21.053383+0000","last_became_peered":"2024-09-21T03:36:21.053383+0000","last_unstale":"2024-09-21T03:36:23.661138+0000","last_undegraded":"2024-09-21T03:36:23.661138+0000","last_fullsized":"2024-09-21T03:36:23.661138+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_clean_scrub_stamp":"2024-09-21T03:36:20.027499+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:37:25.878136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.661181+0000","last_change":"2024-09-21T03:36:22.093865+0000","last_active":"2024-09-21T03:36:23.661181+0000","last_peered":"2024-09-21T03:36:23.661181+0000","last_clean":"2024-09-21T03:36:23.661181+0000","last_became_active":"2024-09-21T03:36:22.093526+0000","last_became_peered":"2024-09-21T03:36:22.093526+0000","last_unstale":"2024-09-21T03:36:23.661181+0000","last_undegraded":"2024-09-21T03:36:23.661181+0000","last_fullsized":"2024-09-21T03:36:23.661181+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:59:12.697168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.d","version":"66'1","reported_seq":16,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.086666+0000","last_change":"2024-09-21T03:36:21.063656+0000","last_active":"2024-09-21T03:36:23.086666+0000","last_peered":"2024-09-21T03:36:23.086666+0000","last_clean":"2024-09-21T03:36:23.086666+0000","last_became_active":"2024-09-21T03:36:21.063355+0000","last_became_peered":"2024-09-21T03:36:21.063355+0000","last_unstale":"2024-09-21T03:36:23.086666+0000","last_undegraded":"2024-09-21T03:36:23.086666+0000","last_fullsized":"2024-09-21T03:36:23.086666+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_clean_scrub_stamp":"2024-09-21T03:36:20.027499+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:05:53.691084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.086342+0000","last_change":"2024-09-21T03:36:22.070297+0000","last_active":"2024-09-21T03:36:23.086342+0000","last_peered":"2024-09-21T03:36:23.086342+0000","last_clean":"2024-09-21T03:36:23.086342+0000","last_became_active":"2024-09-21T03:36:22.069258+0000","last_became_peered":"2024-09-21T03:36:22.069258+0000","last_unstale":"2024-09-21T03:36:23.086342+0000","last_undegraded":"2024-09-21T03:36:23.086342+0000","last_fullsized":"2024-09-21T03:36:23.086342+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:51:57.465792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.660607+0000","last_change":"2024-09-21T03:36:21.067658+0000","last_active":"2024-09-21T03:36:23.660607+0000","last_peered":"2024-09-21T03:36:23.660607+0000","last_clean":"2024-09-21T03:36:23.660607+0000","last_became_active":"2024-09-21T03:36:21.067359+0000","last_became_peered":"2024-09-21T03:36:21.067359+0000","last_unstale":"2024-09-21T03:36:23.660607+0000","last_undegraded":"2024-09-21T03:36:23.660607+0000","last_fullsized":"2024-09-21T03:36:23.660607+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_clean_scrub_stamp":"2024-09-21T03:36:20.027499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:38:26.468485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-21T03:36:21.054221+0000","last_change":"2024-09-21T03:36:21.054221+0000","last_active":"2024-09-21T03:36:21.036908+0000","last_peered":"2024-09-21T03:36:21.036908+0000","last_clean":"2024-09-21T03:36:21.036908+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:36:21.054221+0000","last_undegraded":"2024-09-21T03:36:21.054221+0000","last_fullsized":"2024-09-21T03:36:21.054221+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-21T03:36:21.068841+0000","last_change":"2024-09-21T03:36:21.067047+0000","last_active":"2024-09-21T03:36:21.068841+0000","last_peered":"2024-09-21T03:36:21.068841+0000","last_clean":"2024-09-21T03:36:21.068841+0000","last_became_active":"2024-09-21T03:36:21.066718+0000","last_became_peered":"2024-09-21T03:36:21.066718+0000","last_unstale":"2024-09-21T03:36:21.068841+0000","last_undegraded":"2024-09-21T03:36:21.068841+0000","last_fullsized":"2024-09-21T03:36:21.068841+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_clean_scrub_stamp":"2024-09-21T03:36:20.027499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:34:30.827273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.660351+0000","last_change":"2024-09-21T03:36:22.081020+0000","last_active":"2024-09-21T03:36:23.660351+0000","last_peered":"2024-09-21T03:36:23.660351+0000","last_clean":"2024-09-21T03:36:23.660351+0000","last_became_active":"2024-09-21T03:36:22.080090+0000","last_became_peered":"2024-09-21T03:36:22.080090+0000","last_unstale":"2024-09-21T03:36:23.660351+0000","last_undegraded":"2024-09-21T03:36:23.660351+0000","last_fullsized":"2024-09-21T03:36:23.660351+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:05:02.647190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T03:36:20.027499+0000","last_change":"2024-09-21T03:36:20.027499+0000","last_active":"2024-09-21T03:36:20.027499+0000","last_peered":"2024-09-21T03:36:20.027499+0000","last_clean":"2024-09-21T03:36:20.027499+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:36:20.027499+0000","last_undegraded":"2024-09-21T03:36:20.027499+0000","last_fullsized":"2024-09-21T03:36:20.027499+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_clean_scrub_stamp":"2024-09-21T03:36:20.027499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.660983+0000","last_change":"2024-09-21T03:36:22.071663+0000","last_active":"2024-09-21T03:36:23.660983+0000","last_peered":"2024-09-21T03:36:23.660983+0000","last_clean":"2024-09-21T03:36:23.660983+0000","last_became_active":"2024-09-21T03:36:22.071326+0000","last_became_peered":"2024-09-21T03:36:22.071326+0000","last_unstale":"2024-09-21T03:36:23.660983+0000","last_undegraded":"2024-09-21T03:36:23.660983+0000","last_fullsized":"2024-09-21T03:36:23.660983+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:33:14.339525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-21T03:36:21.064334+0000","last_change":"2024-09-21T03:36:21.062867+0000","last_active":"2024-09-21T03:36:21.064334+0000","last_peered":"2024-09-21T03:36:21.064334+0000","last_clean":"2024-09-21T03:36:21.064334+0000","last_became_active":"2024-09-21T03:36:21.062555+0000","last_became_peered":"2024-09-21T03:36:21.062555+0000","last_unstale":"2024-09-21T03:36:21.064334+0000","last_undegraded":"2024-09-21T03:36:21.064334+0000","last_fullsized":"2024-09-21T03:36:21.064334+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_clean_scrub_stamp":"2024-09-21T03:36:20.027499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:24:33.618928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-21T03:36:22.066694+0000","last_change":"2024-09-21T03:36:22.063923+0000","last_active":"2024-09-21T03:36:22.066694+0000","last_peered":"2024-09-21T03:36:22.066694+0000","last_clean":"2024-09-21T03:36:22.066694+0000","last_became_active":"2024-09-21T03:36:22.063195+0000","last_became_peered":"2024-09-21T03:36:22.063195+0000","last_unstale":"2024-09-21T03:36:22.066694+0000","last_undegraded":"2024-09-21T03:36:22.066694+0000","last_fullsized":"2024-09-21T03:36:22.066694+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:48:30.795859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.7","version":"66'2","reported_seq":17,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.661858+0000","last_change":"2024-09-21T03:36:21.068456+0000","last_active":"2024-09-21T03:36:23.661858+0000","last_peered":"2024-09-21T03:36:23.661858+0000","last_clean":"2024-09-21T03:36:23.661858+0000","last_became_active":"2024-09-21T03:36:21.068101+0000","last_became_peered":"2024-09-21T03:36:21.068101+0000","last_unstale":"2024-09-21T03:36:23.661858+0000","last_undegraded":"2024-09-21T03:36:23.661858+0000","last_fullsized":"2024-09-21T03:36:23.661858+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_clean_scrub_stamp":"2024-09-21T03:36:20.027499+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:09:49.802044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-21T03:36:22.073022+0000","last_change":"2024-09-21T03:36:22.071078+0000","last_active":"2024-09-21T03:36:22.073022+0000","last_peered":"2024-09-21T03:36:22.073022+0000","last_clean":"2024-09-21T03:36:22.073022+0000","last_became_active":"2024-09-21T03:36:22.070566+0000","last_became_peered":"2024-09-21T03:36:22.070566+0000","last_unstale":"2024-09-21T03:36:22.073022+0000","last_undegraded":"2024-09-21T03:36:22.073022+0000","last_fullsized":"2024-09-21T03:36:22.073022+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:09:22.850109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.661366+0000","last_change":"2024-09-21T03:36:21.059377+0000","last_active":"2024-09-21T03:36:23.661366+0000","last_peered":"2024-09-21T03:36:23.661366+0000","last_clean":"2024-09-21T03:36:23.661366+0000","last_became_active":"2024-09-21T03:36:21.059079+0000","last_became_peered":"2024-09-21T03:36:21.059079+0000","last_unstale":"2024-09-21T03:36:23.661366+0000","last_undegraded":"2024-09-21T03:36:23.661366+0000","last_fullsized":"2024-09-21T03:36:23.661366+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_clean_scrub_stamp":"2024-09-21T03:36:20.027499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:39:00.495803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.086122+0000","last_change":"2024-09-21T03:36:22.072209+0000","last_active":"2024-09-21T03:36:23.086122+0000","last_peered":"2024-09-21T03:36:23.086122+0000","last_clean":"2024-09-21T03:36:23.086122+0000","last_became_active":"2024-09-21T03:36:22.071876+0000","last_became_peered":"2024-09-21T03:36:22.071876+0000","last_unstale":"2024-09-21T03:36:23.086122+0000","last_undegraded":"2024-09-21T03:36:23.086122+0000","last_fullsized":"2024-09-21T03:36:23.086122+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:37:07.049180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-21T03:36:21.064294+0000","last_change":"2024-09-21T03:36:21.063184+0000","last_active":"2024-09-21T03:36:21.064294+0000","last_peered":"2024-09-21T03:36:21.064294+0000","last_clean":"2024-09-21T03:36:21.064294+0000","last_became_active":"2024-09-21T03:36:21.062859+0000","last_became_peered":"2024-09-21T03:36:21.062859+0000","last_unstale":"2024-09-21T03:36:21.064294+0000","last_undegraded":"2024-09-21T03:36:21.064294+0000","last_fullsized":"2024-09-21T03:36:21.064294+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_clean_scrub_stamp":"2024-09-21T03:36:20.027499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:18:15.549734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.086389+0000","last_change":"2024-09-21T03:36:22.069954+0000","last_active":"2024-09-21T03:36:23.086389+0000","last_peered":"2024-09-21T03:36:23.086389+0000","last_clean":"2024-09-21T03:36:23.086389+0000","last_became_active":"2024-09-21T03:36:22.068916+0000","last_became_peered":"2024-09-21T03:36:22.068916+0000","last_unstale":"2024-09-21T03:36:23.086389+0000","last_undegraded":"2024-09-21T03:36:23.086389+0000","last_fullsized":"2024-09-21T03:36:23.086389+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:03:46.925197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.660652+0000","last_change":"2024-09-21T03:36:21.071978+0000","last_active":"2024-09-21T03:36:23.660652+0000","last_peered":"2024-09-21T03:36:23.660652+0000","last_clean":"2024-09-21T03:36:23.660652+0000","last_became_active":"2024-09-21T03:36:21.071717+0000","last_became_peered":"2024-09-21T03:36:21.071717+0000","last_unstale":"2024-09-21T03:36:23.660652+0000","last_undegraded":"2024-09-21T03:36:23.660652+0000","last_fullsized":"2024-09-21T03:36:23.660652+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_clean_scrub_stamp":"2024-09-21T03:36:20.027499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:46:05.982882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.661216+0000","last_change":"2024-09-21T03:36:22.072086+0000","last_active":"2024-09-21T03:36:23.661216+0000","last_peered":"2024-09-21T03:36:23.661216+0000","last_clean":"2024-09-21T03:36:23.661216+0000","last_became_active":"2024-09-21T03:36:22.071729+0000","last_became_peered":"2024-09-21T03:36:22.071729+0000","last_unstale":"2024-09-21T03:36:23.661216+0000","last_undegraded":"2024-09-21T03:36:23.661216+0000","last_fullsized":"2024-09-21T03:36:23.661216+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:30:14.015775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.660720+0000","last_change":"2024-09-21T03:36:21.066336+0000","last_active":"2024-09-21T03:36:23.660720+0000","last_peered":"2024-09-21T03:36:23.660720+0000","last_clean":"2024-09-21T03:36:23.660720+0000","last_became_active":"2024-09-21T03:36:21.066043+0000","last_became_peered":"2024-09-21T03:36:21.066043+0000","last_unstale":"2024-09-21T03:36:23.660720+0000","last_undegraded":"2024-09-21T03:36:23.660720+0000","last_fullsized":"2024-09-21T03:36:23.660720+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_clean_scrub_stamp":"2024-09-21T03:36:20.027499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:45:07.961665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.660391+0000","last_change":"2024-09-21T03:36:22.089730+0000","last_active":"2024-09-21T03:36:23.660391+0000","last_peered":"2024-09-21T03:36:23.660391+0000","last_clean":"2024-09-21T03:36:23.660391+0000","last_became_active":"2024-09-21T03:36:22.089403+0000","last_became_peered":"2024-09-21T03:36:22.089403+0000","last_unstale":"2024-09-21T03:36:23.660391+0000","last_undegraded":"2024-09-21T03:36:23.660391+0000","last_fullsized":"2024-09-21T03:36:23.660391+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:38:24.463402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":13,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-21T03:36:22.061045+0000","last_change":"2024-09-21T03:36:21.066672+0000","last_active":"2024-09-21T03:36:22.061045+0000","last_peered":"2024-09-21T03:36:22.061045+0000","last_clean":"2024-09-21T03:36:22.061045+0000","last_became_active":"2024-09-21T03:36:21.066366+0000","last_became_peered":"2024-09-21T03:36:21.066366+0000","last_unstale":"2024-09-21T03:36:22.061045+0000","last_undegraded":"2024-09-21T03:36:22.061045+0000","last_fullsized":"2024-09-21T03:36:22.061045+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_clean_scrub_stamp":"2024-09-21T03:36:20.027499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:28:15.884066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-21T03:36:21.054594+0000","last_change":"2024-09-21T03:36:21.054594+0000","last_active":"2024-09-21T03:36:21.036908+0000","last_peered":"2024-09-21T03:36:21.036908+0000","last_clean":"2024-09-21T03:36:21.036908+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:36:21.054594+0000","last_undegraded":"2024-09-21T03:36:21.054594+0000","last_fullsized":"2024-09-21T03:36:21.054594+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.660737+0000","last_change":"2024-09-21T03:36:21.065843+0000","last_active":"2024-09-21T03:36:23.660737+0000","last_peered":"2024-09-21T03:36:23.660737+0000","last_clean":"2024-09-21T03:36:23.660737+0000","last_became_active":"2024-09-21T03:36:21.065466+0000","last_became_peered":"2024-09-21T03:36:21.065466+0000","last_unstale":"2024-09-21T03:36:23.660737+0000","last_undegraded":"2024-09-21T03:36:23.660737+0000","last_fullsized":"2024-09-21T03:36:23.660737+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_clean_scrub_stamp":"2024-09-21T03:36:20.027499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:10:41.577481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-21T03:36:22.072278+0000","last_change":"2024-09-21T03:36:22.068997+0000","last_active":"2024-09-21T03:36:22.072278+0000","last_peered":"2024-09-21T03:36:22.072278+0000","last_clean":"2024-09-21T03:36:22.072278+0000","last_became_active":"2024-09-21T03:36:22.067174+0000","last_became_peered":"2024-09-21T03:36:22.067174+0000","last_unstale":"2024-09-21T03:36:22.072278+0000","last_undegraded":"2024-09-21T03:36:22.072278+0000","last_fullsized":"2024-09-21T03:36:22.072278+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:42:28.131400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-21T03:36:21.069684+0000","last_change":"2024-09-21T03:36:21.067567+0000","last_active":"2024-09-21T03:36:21.069684+0000","last_peered":"2024-09-21T03:36:21.069684+0000","last_clean":"2024-09-21T03:36:21.069684+0000","last_became_active":"2024-09-21T03:36:21.067235+0000","last_became_peered":"2024-09-21T03:36:21.067235+0000","last_unstale":"2024-09-21T03:36:21.069684+0000","last_undegraded":"2024-09-21T03:36:21.069684+0000","last_fullsized":"2024-09-21T03:36:21.069684+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_clean_scrub_stamp":"2024-09-21T03:36:20.027499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:24:32.368652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T03:36:21.036908+0000","last_change":"2024-09-21T03:36:21.036908+0000","last_active":"2024-09-21T03:36:21.036908+0000","last_peered":"2024-09-21T03:36:21.036908+0000","last_clean":"2024-09-21T03:36:21.036908+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:36:21.036908+0000","last_undegraded":"2024-09-21T03:36:21.036908+0000","last_fullsized":"2024-09-21T03:36:21.036908+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-21T03:36:21.062519+0000","last_change":"2024-09-21T03:36:21.059196+0000","last_active":"2024-09-21T03:36:21.062519+0000","last_peered":"2024-09-21T03:36:21.062519+0000","last_clean":"2024-09-21T03:36:21.062519+0000","last_became_active":"2024-09-21T03:36:21.058881+0000","last_became_peered":"2024-09-21T03:36:21.058881+0000","last_unstale":"2024-09-21T03:36:21.062519+0000","last_undegraded":"2024-09-21T03:36:21.062519+0000","last_fullsized":"2024-09-21T03:36:21.062519+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_clean_scrub_stamp":"2024-09-21T03:36:20.027499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:37:14.541836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":190,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-21T03:36:18.037492+0000","last_change":"2024-09-21T03:33:25.319776+0000","last_active":"2024-09-21T03:36:18.037492+0000","last_peered":"2024-09-21T03:36:18.037492+0000","last_clean":"2024-09-21T03:36:18.037492+0000","last_became_active":"2024-09-21T03:33:25.318399+0000","last_became_peered":"2024-09-21T03:33:25.318399+0000","last_unstale":"2024-09-21T03:36:18.037492+0000","last_undegraded":"2024-09-21T03:36:18.037492+0000","last_fullsized":"2024-09-21T03:36:18.037492+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:23.016672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:23.016672+0000","last_clean_scrub_stamp":"2024-09-21T03:33:23.016672+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:32:35.906114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":114,"acting":114,"num_store_stats":7},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":798,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19,"ondisk_log_size":19,"up":58,"acting":58,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":1,"up_from":16,"seq":68719476775,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32508,"kb_used_data":1316,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339332,"statfs":{"total":96636764160,"available":96603475968,"internally_reserved":0,"allocated":1347584,"data_stored":927955,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509480,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36124,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335716,"statfs":{"total":96636764160,"available":96599773184,"internally_reserved":0,"allocated":790528,"data_stored":357630,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":36174283},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509479,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31980,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339860,"statfs":{"total":96636764160,"available":96604016640,"internally_reserved":0,"allocated":811008,"data_stored":364482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509480,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31980,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339860,"statfs":{"total":96636764160,"available":96604016640,"internally_reserved":0,"allocated":811008,"data_stored":364378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509479,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31968,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339872,"statfs":{"total":96636764160,"available":96604028928,"internally_reserved":0,"allocated":790528,"data_stored":357630,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542183,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36148,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335692,"statfs":{"total":96636764160,"available":96599748608,"internally_reserved":0,"allocated":811008,"data_stored":359894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574887,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36692,"kb_used_data":1340,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335148,"statfs":{"total":96636764160,"available":96599191552,"internally_reserved":0,"allocated":1372160,"data_stored":940831,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542184,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36132,"kb_used_data":776,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335708,"statfs":{"total":96636764160,"available":96599764992,"internally_reserved":0,"allocated":794624,"data_stored":358212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T03:36:26.530 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T03:36:26.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:26.682+0000 7f3c07919640 1 -- 172.21.15.38:0/1789604059 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3be8003a50 msgr2=0x7f3be8003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:26.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:26.682+0000 7f3c07919640 1 --2- 172.21.15.38:0/1789604059 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3be8003a50 0x7f3be8003ec0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f3bf000c640 tx=0x7f3be8005830 comp rx=0 tx=0).stop 2024-09-21T03:36:26.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:26.682+0000 7f3c05eb6640 1 -- 172.21.15.38:0/1789604059 reap_dead start 2024-09-21T03:36:26.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:26.682+0000 7f3c07919640 1 -- 172.21.15.38:0/1789604059 shutdown_connections 2024-09-21T03:36:26.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:26.682+0000 7f3c07919640 1 -- 172.21.15.38:0/1789604059 >> 172.21.15.38:0/1789604059 conn(0x7f3c000b2cc0 msgr2=0x7f3c000b30d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:26.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:26.682+0000 7f3c07919640 1 -- 172.21.15.38:0/1789604059 shutdown_connections 2024-09-21T03:36:26.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:26.682+0000 7f3c07919640 1 -- 172.21.15.38:0/1789604059 wait complete. 2024-09-21T03:36:26.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:26.684+0000 7f3c07919640 1 Processor -- start 2024-09-21T03:36:26.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:26.684+0000 7f3c07919640 1 -- start start 2024-09-21T03:36:26.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:26.685+0000 7f3c07919640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3be8003a50 0x7f3c0015e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:26.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:26.685+0000 7f3c07919640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3c0015d3a0 0x7f3c0015d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:26.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:26.685+0000 7f3c07919640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3c00150ac0 0x7f3c00150ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:26.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:26.686+0000 7f3c07919640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f3c000b4760 con 0x7f3c0015d3a0 2024-09-21T03:36:26.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:26.686+0000 7f3c07919640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f3c000b45e0 con 0x7f3be8003a50 2024-09-21T03:36:26.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:26.686+0000 7f3c07919640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f3c0015b5d0 con 0x7f3c00150ac0 2024-09-21T03:36:26.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:26.686+0000 7f3c04eb4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3c0015d3a0 0x7f3c0015d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:26.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:26.686+0000 7f3c04eb4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3c0015d3a0 0x7f3c0015d7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37332/0 (socket says 172.21.15.38:37332) 2024-09-21T03:36:26.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:26.686+0000 7f3c04eb4640 1 -- 172.21.15.38:0/195471713 learned_addr learned my addr 172.21.15.38:0/195471713 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:36:26.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:26.686+0000 7f3c05eb6640 1 --2- 172.21.15.38:0/195471713 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3c00150ac0 0x7f3c00150ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:26.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:26.686+0000 7f3c056b5640 1 --2- 172.21.15.38:0/195471713 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3be8003a50 0x7f3c0015e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:26.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:26.686+0000 7f3c04eb4640 1 -- 172.21.15.38:0/195471713 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3c00150ac0 msgr2=0x7f3c00150ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:26.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:26.686+0000 7f3c04eb4640 1 --2- 172.21.15.38:0/195471713 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3c00150ac0 0x7f3c00150ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:26.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:26.686+0000 7f3c04eb4640 1 -- 172.21.15.38:0/195471713 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3be8003a50 msgr2=0x7f3c0015e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:26.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:26.686+0000 7f3c04eb4640 1 --2- 172.21.15.38:0/195471713 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3be8003a50 0x7f3c0015e2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:26.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:26.686+0000 7f3c04eb4640 1 -- 172.21.15.38:0/195471713 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c001f5b70 con 0x7f3c0015d3a0 2024-09-21T03:36:26.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:26.686+0000 7f3c05eb6640 1 --2- 172.21.15.38:0/195471713 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3c00150ac0 0x7f3c00150ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:36:26.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:26.686+0000 7f3c056b5640 1 --2- 172.21.15.38:0/195471713 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3be8003a50 0x7f3c0015e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:36:26.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:26.686+0000 7f3c04eb4640 1 --2- 172.21.15.38:0/195471713 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3c0015d3a0 0x7f3c0015d7b0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f3bf4011a30 tx=0x7f3bf4011ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:26.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:26.686+0000 7f3bfdffb640 1 -- 172.21.15.38:0/195471713 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bf401f070 con 0x7f3c0015d3a0 2024-09-21T03:36:26.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:26.686+0000 7f3bfdffb640 1 -- 172.21.15.38:0/195471713 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3bf400fe40 con 0x7f3c0015d3a0 2024-09-21T03:36:26.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:26.686+0000 7f3c07919640 1 -- 172.21.15.38:0/195471713 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c001f56d0 con 0x7f3c0015d3a0 2024-09-21T03:36:26.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:26.686+0000 7f3bfdffb640 1 -- 172.21.15.38:0/195471713 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bf40126b0 con 0x7f3c0015d3a0 2024-09-21T03:36:26.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:26.687+0000 7f3c07919640 1 -- 172.21.15.38:0/195471713 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c001f6130 con 0x7f3c0015d3a0 2024-09-21T03:36:26.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:26.688+0000 7f3c07919640 1 -- 172.21.15.38:0/195471713 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c00074160 con 0x7f3c0015d3a0 2024-09-21T03:36:26.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:26.688+0000 7f3bfdffb640 1 -- 172.21.15.38:0/195471713 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f3bf4022070 con 0x7f3c0015d3a0 2024-09-21T03:36:26.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:26.689+0000 7f3bfdffb640 1 --2- 172.21.15.38:0/195471713 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3bec078b90 0x7f3bec07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:26.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:26.689+0000 7f3bfdffb640 1 -- 172.21.15.38:0/195471713 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6101+0+0 (secure 0 0 0) 0x7f3bf409ba20 con 0x7f3c0015d3a0 2024-09-21T03:36:26.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:26.690+0000 7f3c056b5640 1 --2- 172.21.15.38:0/195471713 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3bec078b90 0x7f3bec07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:26.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:26.690+0000 7f3c056b5640 1 --2- 172.21.15.38:0/195471713 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3bec078b90 0x7f3bec07b0a0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f3bf0033000 tx=0x7f3c0006e1d0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:26.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:26.693+0000 7f3bfdffb640 1 -- 172.21.15.38:0/195471713 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f3bf40635e0 con 0x7f3c0015d3a0 2024-09-21T03:36:26.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:26.878+0000 7f3c07919640 1 -- 172.21.15.38:0/195471713 --> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3c00151410 con 0x7f3bec078b90 2024-09-21T03:36:26.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:26.882+0000 7f3bfdffb640 1 -- 172.21.15.38:0/195471713 <== mgr.4117 v2:172.21.15.195:6800/1631901344 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+249897 (secure 0 0 0) 0x7f3c00151410 con 0x7f3bec078b90 2024-09-21T03:36:26.887 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:36:26.888 INFO:teuthology.orchestra.run.smithi038.stderr:dumped all 2024-09-21T03:36:26.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:26.885+0000 7f3c07919640 1 -- 172.21.15.38:0/195471713 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3bec078b90 msgr2=0x7f3bec07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:26.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:26.885+0000 7f3c07919640 1 --2- 172.21.15.38:0/195471713 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3bec078b90 0x7f3bec07b0a0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f3bf0033000 tx=0x7f3c0006e1d0 comp rx=0 tx=0).stop 2024-09-21T03:36:26.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:26.885+0000 7f3c07919640 1 -- 172.21.15.38:0/195471713 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3c0015d3a0 msgr2=0x7f3c0015d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:26.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:26.885+0000 7f3c07919640 1 --2- 172.21.15.38:0/195471713 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3c0015d3a0 0x7f3c0015d7b0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f3bf4011a30 tx=0x7f3bf4011ef0 comp rx=0 tx=0).stop 2024-09-21T03:36:26.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:26.885+0000 7f3c07919640 1 -- 172.21.15.38:0/195471713 shutdown_connections 2024-09-21T03:36:26.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:26.885+0000 7f3c07919640 1 --2- 172.21.15.38:0/195471713 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3bec078b90 0x7f3bec07b0a0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:26.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:26.885+0000 7f3c07919640 1 --2- 172.21.15.38:0/195471713 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3c00150ac0 0x7f3c00150ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:26.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:26.885+0000 7f3c07919640 1 --2- 172.21.15.38:0/195471713 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3c0015d3a0 0x7f3c0015d7b0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:26.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:26.885+0000 7f3c07919640 1 --2- 172.21.15.38:0/195471713 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3be8003a50 0x7f3c0015e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:26.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:26.885+0000 7f3c07919640 1 -- 172.21.15.38:0/195471713 >> 172.21.15.38:0/195471713 conn(0x7f3c000b2cc0 msgr2=0x7f3c000b30d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:26.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:26.885+0000 7f3c07919640 1 -- 172.21.15.38:0/195471713 shutdown_connections 2024-09-21T03:36:26.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:26.886+0000 7f3c07919640 1 -- 172.21.15.38:0/195471713 wait complete. 2024-09-21T03:36:26.898 INFO:teuthology.orchestra.run.smithi038.stdout:{"pg_ready":true,"pg_map":{"version":151,"stamp":"2024-09-21T03:36:25.023385+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591166,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":123,"num_write_kb":1639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":91,"ondisk_log_size":91,"up":174,"acting":174,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":117,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":273532,"kb_used_data":7352,"kb_used_omap":12,"kb_used_meta":265971,"kb_avail":754701188,"statfs":{"total":773094113280,"available":772814016512,"internally_reserved":0,"allocated":7528448,"data_stored":4031012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12748,"internal_metadata":272354868},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":798,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.985644"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-21T03:36:22.088822+0000","last_change":"2024-09-21T03:36:22.088324+0000","last_active":"2024-09-21T03:36:22.088822+0000","last_peered":"2024-09-21T03:36:22.088822+0000","last_clean":"2024-09-21T03:36:22.088822+0000","last_became_active":"2024-09-21T03:36:22.088010+0000","last_became_peered":"2024-09-21T03:36:22.088010+0000","last_unstale":"2024-09-21T03:36:22.088822+0000","last_undegraded":"2024-09-21T03:36:22.088822+0000","last_fullsized":"2024-09-21T03:36:22.088822+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:17:56.797134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-21T03:36:21.054741+0000","last_change":"2024-09-21T03:36:21.054741+0000","last_active":"2024-09-21T03:36:21.036908+0000","last_peered":"2024-09-21T03:36:21.036908+0000","last_clean":"2024-09-21T03:36:21.036908+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:36:21.054741+0000","last_undegraded":"2024-09-21T03:36:21.054741+0000","last_fullsized":"2024-09-21T03:36:21.054741+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.086324+0000","last_change":"2024-09-21T03:36:22.071184+0000","last_active":"2024-09-21T03:36:23.086324+0000","last_peered":"2024-09-21T03:36:23.086324+0000","last_clean":"2024-09-21T03:36:23.086324+0000","last_became_active":"2024-09-21T03:36:22.070772+0000","last_became_peered":"2024-09-21T03:36:22.070772+0000","last_unstale":"2024-09-21T03:36:23.086324+0000","last_undegraded":"2024-09-21T03:36:23.086324+0000","last_fullsized":"2024-09-21T03:36:23.086324+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:17:47.479714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.661468+0000","last_change":"2024-09-21T03:36:22.070400+0000","last_active":"2024-09-21T03:36:23.661468+0000","last_peered":"2024-09-21T03:36:23.661468+0000","last_clean":"2024-09-21T03:36:23.661468+0000","last_became_active":"2024-09-21T03:36:22.068959+0000","last_became_peered":"2024-09-21T03:36:22.068959+0000","last_unstale":"2024-09-21T03:36:23.661468+0000","last_undegraded":"2024-09-21T03:36:23.661468+0000","last_fullsized":"2024-09-21T03:36:23.661468+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:30:41.436799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.086049+0000","last_change":"2024-09-21T03:36:22.081653+0000","last_active":"2024-09-21T03:36:23.086049+0000","last_peered":"2024-09-21T03:36:23.086049+0000","last_clean":"2024-09-21T03:36:23.086049+0000","last_became_active":"2024-09-21T03:36:22.081343+0000","last_became_peered":"2024-09-21T03:36:22.081343+0000","last_unstale":"2024-09-21T03:36:23.086049+0000","last_undegraded":"2024-09-21T03:36:23.086049+0000","last_fullsized":"2024-09-21T03:36:23.086049+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:37:46.632334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-21T03:36:21.055099+0000","last_change":"2024-09-21T03:36:21.055099+0000","last_active":"2024-09-21T03:36:21.036908+0000","last_peered":"2024-09-21T03:36:21.036908+0000","last_clean":"2024-09-21T03:36:21.036908+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:36:21.055099+0000","last_undegraded":"2024-09-21T03:36:21.055099+0000","last_fullsized":"2024-09-21T03:36:21.055099+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-21T03:36:22.075723+0000","last_change":"2024-09-21T03:36:22.074034+0000","last_active":"2024-09-21T03:36:22.075723+0000","last_peered":"2024-09-21T03:36:22.075723+0000","last_clean":"2024-09-21T03:36:22.075723+0000","last_became_active":"2024-09-21T03:36:22.073682+0000","last_became_peered":"2024-09-21T03:36:22.073682+0000","last_unstale":"2024-09-21T03:36:22.075723+0000","last_undegraded":"2024-09-21T03:36:22.075723+0000","last_fullsized":"2024-09-21T03:36:22.075723+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:25:30.522013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.085192+0000","last_change":"2024-09-21T03:36:22.080482+0000","last_active":"2024-09-21T03:36:23.085192+0000","last_peered":"2024-09-21T03:36:23.085192+0000","last_clean":"2024-09-21T03:36:23.085192+0000","last_became_active":"2024-09-21T03:36:22.080177+0000","last_became_peered":"2024-09-21T03:36:22.080177+0000","last_unstale":"2024-09-21T03:36:23.085192+0000","last_undegraded":"2024-09-21T03:36:23.085192+0000","last_fullsized":"2024-09-21T03:36:23.085192+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:56:46.927190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.661540+0000","last_change":"2024-09-21T03:36:22.077361+0000","last_active":"2024-09-21T03:36:23.661540+0000","last_peered":"2024-09-21T03:36:23.661540+0000","last_clean":"2024-09-21T03:36:23.661540+0000","last_became_active":"2024-09-21T03:36:22.077050+0000","last_became_peered":"2024-09-21T03:36:22.077050+0000","last_unstale":"2024-09-21T03:36:23.661540+0000","last_undegraded":"2024-09-21T03:36:23.661540+0000","last_fullsized":"2024-09-21T03:36:23.661540+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:30:24.348269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T03:36:21.036908+0000","last_change":"2024-09-21T03:36:21.036908+0000","last_active":"2024-09-21T03:36:21.036908+0000","last_peered":"2024-09-21T03:36:21.036908+0000","last_clean":"2024-09-21T03:36:21.036908+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:36:21.036908+0000","last_undegraded":"2024-09-21T03:36:21.036908+0000","last_fullsized":"2024-09-21T03:36:21.036908+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.661250+0000","last_change":"2024-09-21T03:36:22.072761+0000","last_active":"2024-09-21T03:36:23.661250+0000","last_peered":"2024-09-21T03:36:23.661250+0000","last_clean":"2024-09-21T03:36:23.661250+0000","last_became_active":"2024-09-21T03:36:22.072450+0000","last_became_peered":"2024-09-21T03:36:22.072450+0000","last_unstale":"2024-09-21T03:36:23.661250+0000","last_undegraded":"2024-09-21T03:36:23.661250+0000","last_fullsized":"2024-09-21T03:36:23.661250+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:04:45.989423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.661340+0000","last_change":"2024-09-21T03:36:22.086026+0000","last_active":"2024-09-21T03:36:23.661340+0000","last_peered":"2024-09-21T03:36:23.661340+0000","last_clean":"2024-09-21T03:36:23.661340+0000","last_became_active":"2024-09-21T03:36:22.085717+0000","last_became_peered":"2024-09-21T03:36:22.085717+0000","last_unstale":"2024-09-21T03:36:23.661340+0000","last_undegraded":"2024-09-21T03:36:23.661340+0000","last_fullsized":"2024-09-21T03:36:23.661340+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:47:51.103846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.660593+0000","last_change":"2024-09-21T03:36:22.090483+0000","last_active":"2024-09-21T03:36:23.660593+0000","last_peered":"2024-09-21T03:36:23.660593+0000","last_clean":"2024-09-21T03:36:23.660593+0000","last_became_active":"2024-09-21T03:36:22.090124+0000","last_became_peered":"2024-09-21T03:36:22.090124+0000","last_unstale":"2024-09-21T03:36:23.660593+0000","last_undegraded":"2024-09-21T03:36:23.660593+0000","last_fullsized":"2024-09-21T03:36:23.660593+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:52:19.716709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.660532+0000","last_change":"2024-09-21T03:36:22.080680+0000","last_active":"2024-09-21T03:36:23.660532+0000","last_peered":"2024-09-21T03:36:23.660532+0000","last_clean":"2024-09-21T03:36:23.660532+0000","last_became_active":"2024-09-21T03:36:22.080377+0000","last_became_peered":"2024-09-21T03:36:22.080377+0000","last_unstale":"2024-09-21T03:36:23.660532+0000","last_undegraded":"2024-09-21T03:36:23.660532+0000","last_fullsized":"2024-09-21T03:36:23.660532+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:54:33.394204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T03:36:21.036908+0000","last_change":"2024-09-21T03:36:21.036908+0000","last_active":"2024-09-21T03:36:21.036908+0000","last_peered":"2024-09-21T03:36:21.036908+0000","last_clean":"2024-09-21T03:36:21.036908+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:36:21.036908+0000","last_undegraded":"2024-09-21T03:36:21.036908+0000","last_fullsized":"2024-09-21T03:36:21.036908+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.660634+0000","last_change":"2024-09-21T03:36:22.085437+0000","last_active":"2024-09-21T03:36:23.660634+0000","last_peered":"2024-09-21T03:36:23.660634+0000","last_clean":"2024-09-21T03:36:23.660634+0000","last_became_active":"2024-09-21T03:36:22.085087+0000","last_became_peered":"2024-09-21T03:36:22.085087+0000","last_unstale":"2024-09-21T03:36:23.660634+0000","last_undegraded":"2024-09-21T03:36:23.660634+0000","last_fullsized":"2024-09-21T03:36:23.660634+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:30:00.925328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-21T03:36:22.082591+0000","last_change":"2024-09-21T03:36:22.081251+0000","last_active":"2024-09-21T03:36:22.082591+0000","last_peered":"2024-09-21T03:36:22.082591+0000","last_clean":"2024-09-21T03:36:22.082591+0000","last_became_active":"2024-09-21T03:36:22.080931+0000","last_became_peered":"2024-09-21T03:36:22.080931+0000","last_unstale":"2024-09-21T03:36:22.082591+0000","last_undegraded":"2024-09-21T03:36:22.082591+0000","last_fullsized":"2024-09-21T03:36:22.082591+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:04:34.873429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.661300+0000","last_change":"2024-09-21T03:36:22.069803+0000","last_active":"2024-09-21T03:36:23.661300+0000","last_peered":"2024-09-21T03:36:23.661300+0000","last_clean":"2024-09-21T03:36:23.661300+0000","last_became_active":"2024-09-21T03:36:22.069481+0000","last_became_peered":"2024-09-21T03:36:22.069481+0000","last_unstale":"2024-09-21T03:36:23.661300+0000","last_undegraded":"2024-09-21T03:36:23.661300+0000","last_fullsized":"2024-09-21T03:36:23.661300+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:15:44.483810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.661639+0000","last_change":"2024-09-21T03:36:22.076014+0000","last_active":"2024-09-21T03:36:23.661639+0000","last_peered":"2024-09-21T03:36:23.661639+0000","last_clean":"2024-09-21T03:36:23.661639+0000","last_became_active":"2024-09-21T03:36:22.075689+0000","last_became_peered":"2024-09-21T03:36:22.075689+0000","last_unstale":"2024-09-21T03:36:23.661639+0000","last_undegraded":"2024-09-21T03:36:23.661639+0000","last_fullsized":"2024-09-21T03:36:23.661639+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:47:25.173315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-21T03:36:22.073135+0000","last_change":"2024-09-21T03:36:22.071709+0000","last_active":"2024-09-21T03:36:22.073135+0000","last_peered":"2024-09-21T03:36:22.073135+0000","last_clean":"2024-09-21T03:36:22.073135+0000","last_became_active":"2024-09-21T03:36:22.070413+0000","last_became_peered":"2024-09-21T03:36:22.070413+0000","last_unstale":"2024-09-21T03:36:22.073135+0000","last_undegraded":"2024-09-21T03:36:22.073135+0000","last_fullsized":"2024-09-21T03:36:22.073135+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:16:55.679529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.660747+0000","last_change":"2024-09-21T03:36:22.086945+0000","last_active":"2024-09-21T03:36:23.660747+0000","last_peered":"2024-09-21T03:36:23.660747+0000","last_clean":"2024-09-21T03:36:23.660747+0000","last_became_active":"2024-09-21T03:36:22.086620+0000","last_became_peered":"2024-09-21T03:36:22.086620+0000","last_unstale":"2024-09-21T03:36:23.660747+0000","last_undegraded":"2024-09-21T03:36:23.660747+0000","last_fullsized":"2024-09-21T03:36:23.660747+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:48:46.104173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-21T03:36:21.053928+0000","last_change":"2024-09-21T03:36:21.053928+0000","last_active":"2024-09-21T03:36:21.036908+0000","last_peered":"2024-09-21T03:36:21.036908+0000","last_clean":"2024-09-21T03:36:21.036908+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:36:21.053928+0000","last_undegraded":"2024-09-21T03:36:21.053928+0000","last_fullsized":"2024-09-21T03:36:21.053928+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.660604+0000","last_change":"2024-09-21T03:36:22.086122+0000","last_active":"2024-09-21T03:36:23.660604+0000","last_peered":"2024-09-21T03:36:23.660604+0000","last_clean":"2024-09-21T03:36:23.660604+0000","last_became_active":"2024-09-21T03:36:22.085783+0000","last_became_peered":"2024-09-21T03:36:22.085783+0000","last_unstale":"2024-09-21T03:36:23.660604+0000","last_undegraded":"2024-09-21T03:36:23.660604+0000","last_fullsized":"2024-09-21T03:36:23.660604+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:05:24.777050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.085420+0000","last_change":"2024-09-21T03:36:22.067412+0000","last_active":"2024-09-21T03:36:23.085420+0000","last_peered":"2024-09-21T03:36:23.085420+0000","last_clean":"2024-09-21T03:36:23.085420+0000","last_became_active":"2024-09-21T03:36:22.067084+0000","last_became_peered":"2024-09-21T03:36:22.067084+0000","last_unstale":"2024-09-21T03:36:23.085420+0000","last_undegraded":"2024-09-21T03:36:23.085420+0000","last_fullsized":"2024-09-21T03:36:23.085420+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:01:59.040840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.661165+0000","last_change":"2024-09-21T03:36:22.082532+0000","last_active":"2024-09-21T03:36:23.661165+0000","last_peered":"2024-09-21T03:36:23.661165+0000","last_clean":"2024-09-21T03:36:23.661165+0000","last_became_active":"2024-09-21T03:36:22.082233+0000","last_became_peered":"2024-09-21T03:36:22.082233+0000","last_unstale":"2024-09-21T03:36:23.661165+0000","last_undegraded":"2024-09-21T03:36:23.661165+0000","last_fullsized":"2024-09-21T03:36:23.661165+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:09:57.485058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.086025+0000","last_change":"2024-09-21T03:36:22.068982+0000","last_active":"2024-09-21T03:36:23.086025+0000","last_peered":"2024-09-21T03:36:23.086025+0000","last_clean":"2024-09-21T03:36:23.086025+0000","last_became_active":"2024-09-21T03:36:22.068584+0000","last_became_peered":"2024-09-21T03:36:22.068584+0000","last_unstale":"2024-09-21T03:36:23.086025+0000","last_undegraded":"2024-09-21T03:36:23.086025+0000","last_fullsized":"2024-09-21T03:36:23.086025+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:02:01.275514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.660817+0000","last_change":"2024-09-21T03:36:22.085467+0000","last_active":"2024-09-21T03:36:23.660817+0000","last_peered":"2024-09-21T03:36:23.660817+0000","last_clean":"2024-09-21T03:36:23.660817+0000","last_became_active":"2024-09-21T03:36:22.085127+0000","last_became_peered":"2024-09-21T03:36:22.085127+0000","last_unstale":"2024-09-21T03:36:23.660817+0000","last_undegraded":"2024-09-21T03:36:23.660817+0000","last_fullsized":"2024-09-21T03:36:23.660817+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:04:32.859631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-21T03:36:22.084080+0000","last_change":"2024-09-21T03:36:22.083174+0000","last_active":"2024-09-21T03:36:22.084080+0000","last_peered":"2024-09-21T03:36:22.084080+0000","last_clean":"2024-09-21T03:36:22.084080+0000","last_became_active":"2024-09-21T03:36:22.082930+0000","last_became_peered":"2024-09-21T03:36:22.082930+0000","last_unstale":"2024-09-21T03:36:22.084080+0000","last_undegraded":"2024-09-21T03:36:22.084080+0000","last_fullsized":"2024-09-21T03:36:22.084080+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:09:50.811454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.661615+0000","last_change":"2024-09-21T03:36:22.088889+0000","last_active":"2024-09-21T03:36:23.661615+0000","last_peered":"2024-09-21T03:36:23.661615+0000","last_clean":"2024-09-21T03:36:23.661615+0000","last_became_active":"2024-09-21T03:36:22.088515+0000","last_became_peered":"2024-09-21T03:36:22.088515+0000","last_unstale":"2024-09-21T03:36:23.661615+0000","last_undegraded":"2024-09-21T03:36:23.661615+0000","last_fullsized":"2024-09-21T03:36:23.661615+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:00:06.850866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.660822+0000","last_change":"2024-09-21T03:36:22.088463+0000","last_active":"2024-09-21T03:36:23.660822+0000","last_peered":"2024-09-21T03:36:23.660822+0000","last_clean":"2024-09-21T03:36:23.660822+0000","last_became_active":"2024-09-21T03:36:22.088149+0000","last_became_peered":"2024-09-21T03:36:22.088149+0000","last_unstale":"2024-09-21T03:36:23.660822+0000","last_undegraded":"2024-09-21T03:36:23.660822+0000","last_fullsized":"2024-09-21T03:36:23.660822+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:03:04.392175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.085608+0000","last_change":"2024-09-21T03:36:22.066107+0000","last_active":"2024-09-21T03:36:23.085608+0000","last_peered":"2024-09-21T03:36:23.085608+0000","last_clean":"2024-09-21T03:36:23.085608+0000","last_became_active":"2024-09-21T03:36:22.064917+0000","last_became_peered":"2024-09-21T03:36:22.064917+0000","last_unstale":"2024-09-21T03:36:23.085608+0000","last_undegraded":"2024-09-21T03:36:23.085608+0000","last_fullsized":"2024-09-21T03:36:23.085608+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:58:06.692992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-21T03:36:22.072148+0000","last_change":"2024-09-21T03:36:22.069000+0000","last_active":"2024-09-21T03:36:22.072148+0000","last_peered":"2024-09-21T03:36:22.072148+0000","last_clean":"2024-09-21T03:36:22.072148+0000","last_became_active":"2024-09-21T03:36:22.068661+0000","last_became_peered":"2024-09-21T03:36:22.068661+0000","last_unstale":"2024-09-21T03:36:22.072148+0000","last_undegraded":"2024-09-21T03:36:22.072148+0000","last_fullsized":"2024-09-21T03:36:22.072148+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:39:01.505213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.661745+0000","last_change":"2024-09-21T03:36:22.071976+0000","last_active":"2024-09-21T03:36:23.661745+0000","last_peered":"2024-09-21T03:36:23.661745+0000","last_clean":"2024-09-21T03:36:23.661745+0000","last_became_active":"2024-09-21T03:36:22.071182+0000","last_became_peered":"2024-09-21T03:36:22.071182+0000","last_unstale":"2024-09-21T03:36:23.661745+0000","last_undegraded":"2024-09-21T03:36:23.661745+0000","last_fullsized":"2024-09-21T03:36:23.661745+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:58:44.343782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1f","version":"66'4","reported_seq":17,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-21T03:36:22.081942+0000","last_change":"2024-09-21T03:36:21.066587+0000","last_active":"2024-09-21T03:36:22.081942+0000","last_peered":"2024-09-21T03:36:22.081942+0000","last_clean":"2024-09-21T03:36:22.081942+0000","last_became_active":"2024-09-21T03:36:21.066299+0000","last_became_peered":"2024-09-21T03:36:21.066299+0000","last_unstale":"2024-09-21T03:36:22.081942+0000","last_undegraded":"2024-09-21T03:36:22.081942+0000","last_fullsized":"2024-09-21T03:36:22.081942+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_clean_scrub_stamp":"2024-09-21T03:36:20.027499+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:06:51.671380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.086591+0000","last_change":"2024-09-21T03:36:22.073629+0000","last_active":"2024-09-21T03:36:23.086591+0000","last_peered":"2024-09-21T03:36:23.086591+0000","last_clean":"2024-09-21T03:36:23.086591+0000","last_became_active":"2024-09-21T03:36:22.073211+0000","last_became_peered":"2024-09-21T03:36:22.073211+0000","last_unstale":"2024-09-21T03:36:23.086591+0000","last_undegraded":"2024-09-21T03:36:23.086591+0000","last_fullsized":"2024-09-21T03:36:23.086591+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:27:32.381253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1e","version":"66'1","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-21T03:36:22.092834+0000","last_change":"2024-09-21T03:36:21.064384+0000","last_active":"2024-09-21T03:36:22.092834+0000","last_peered":"2024-09-21T03:36:22.092834+0000","last_clean":"2024-09-21T03:36:22.092834+0000","last_became_active":"2024-09-21T03:36:21.064042+0000","last_became_peered":"2024-09-21T03:36:21.064042+0000","last_unstale":"2024-09-21T03:36:22.092834+0000","last_undegraded":"2024-09-21T03:36:22.092834+0000","last_fullsized":"2024-09-21T03:36:22.092834+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_clean_scrub_stamp":"2024-09-21T03:36:20.027499+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:17:25.045895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-21T03:36:21.054153+0000","last_change":"2024-09-21T03:36:21.054153+0000","last_active":"2024-09-21T03:36:21.036908+0000","last_peered":"2024-09-21T03:36:21.036908+0000","last_clean":"2024-09-21T03:36:21.036908+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:36:21.054153+0000","last_undegraded":"2024-09-21T03:36:21.054153+0000","last_fullsized":"2024-09-21T03:36:21.054153+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1d","version":"66'2","reported_seq":15,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-21T03:36:22.080793+0000","last_change":"2024-09-21T03:36:21.071133+0000","last_active":"2024-09-21T03:36:22.080793+0000","last_peered":"2024-09-21T03:36:22.080793+0000","last_clean":"2024-09-21T03:36:22.080793+0000","last_became_active":"2024-09-21T03:36:21.070834+0000","last_became_peered":"2024-09-21T03:36:21.070834+0000","last_unstale":"2024-09-21T03:36:22.080793+0000","last_undegraded":"2024-09-21T03:36:22.080793+0000","last_fullsized":"2024-09-21T03:36:22.080793+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_clean_scrub_stamp":"2024-09-21T03:36:20.027499+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:37:25.878136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.661630+0000","last_change":"2024-09-21T03:36:22.070080+0000","last_active":"2024-09-21T03:36:23.661630+0000","last_peered":"2024-09-21T03:36:23.661630+0000","last_clean":"2024-09-21T03:36:23.661630+0000","last_became_active":"2024-09-21T03:36:22.069779+0000","last_became_peered":"2024-09-21T03:36:22.069779+0000","last_unstale":"2024-09-21T03:36:23.661630+0000","last_undegraded":"2024-09-21T03:36:23.661630+0000","last_fullsized":"2024-09-21T03:36:23.661630+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:29:10.577562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-21T03:36:21.066934+0000","last_change":"2024-09-21T03:36:21.065338+0000","last_active":"2024-09-21T03:36:21.066934+0000","last_peered":"2024-09-21T03:36:21.066934+0000","last_clean":"2024-09-21T03:36:21.066934+0000","last_became_active":"2024-09-21T03:36:21.065032+0000","last_became_peered":"2024-09-21T03:36:21.065032+0000","last_unstale":"2024-09-21T03:36:21.066934+0000","last_undegraded":"2024-09-21T03:36:21.066934+0000","last_fullsized":"2024-09-21T03:36:21.066934+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_clean_scrub_stamp":"2024-09-21T03:36:20.027499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:38:09.536350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-21T03:36:22.065482+0000","last_change":"2024-09-21T03:36:22.063213+0000","last_active":"2024-09-21T03:36:22.065482+0000","last_peered":"2024-09-21T03:36:22.065482+0000","last_clean":"2024-09-21T03:36:22.065482+0000","last_became_active":"2024-09-21T03:36:22.062870+0000","last_became_peered":"2024-09-21T03:36:22.062870+0000","last_unstale":"2024-09-21T03:36:22.065482+0000","last_undegraded":"2024-09-21T03:36:22.065482+0000","last_fullsized":"2024-09-21T03:36:22.065482+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:07:32.687989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"66'1","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-21T03:36:22.079299+0000","last_change":"2024-09-21T03:36:21.060507+0000","last_active":"2024-09-21T03:36:22.079299+0000","last_peered":"2024-09-21T03:36:22.079299+0000","last_clean":"2024-09-21T03:36:22.079299+0000","last_became_active":"2024-09-21T03:36:21.060129+0000","last_became_peered":"2024-09-21T03:36:21.060129+0000","last_unstale":"2024-09-21T03:36:22.079299+0000","last_undegraded":"2024-09-21T03:36:22.079299+0000","last_fullsized":"2024-09-21T03:36:22.079299+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_clean_scrub_stamp":"2024-09-21T03:36:20.027499+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:57:03.979240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-21T03:36:22.080880+0000","last_change":"2024-09-21T03:36:22.077206+0000","last_active":"2024-09-21T03:36:22.080880+0000","last_peered":"2024-09-21T03:36:22.080880+0000","last_clean":"2024-09-21T03:36:22.080880+0000","last_became_active":"2024-09-21T03:36:22.076897+0000","last_became_peered":"2024-09-21T03:36:22.076897+0000","last_unstale":"2024-09-21T03:36:22.080880+0000","last_undegraded":"2024-09-21T03:36:22.080880+0000","last_fullsized":"2024-09-21T03:36:22.080880+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:39:58.170006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-21T03:36:21.062625+0000","last_change":"2024-09-21T03:36:21.059505+0000","last_active":"2024-09-21T03:36:21.062625+0000","last_peered":"2024-09-21T03:36:21.062625+0000","last_clean":"2024-09-21T03:36:21.062625+0000","last_became_active":"2024-09-21T03:36:21.059110+0000","last_became_peered":"2024-09-21T03:36:21.059110+0000","last_unstale":"2024-09-21T03:36:21.062625+0000","last_undegraded":"2024-09-21T03:36:21.062625+0000","last_fullsized":"2024-09-21T03:36:21.062625+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_clean_scrub_stamp":"2024-09-21T03:36:20.027499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:47:13.398068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T03:36:21.036908+0000","last_change":"2024-09-21T03:36:21.036908+0000","last_active":"2024-09-21T03:36:21.036908+0000","last_peered":"2024-09-21T03:36:21.036908+0000","last_clean":"2024-09-21T03:36:21.036908+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:36:21.036908+0000","last_undegraded":"2024-09-21T03:36:21.036908+0000","last_fullsized":"2024-09-21T03:36:21.036908+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":13,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-21T03:36:22.065557+0000","last_change":"2024-09-21T03:36:21.065608+0000","last_active":"2024-09-21T03:36:22.065557+0000","last_peered":"2024-09-21T03:36:22.065557+0000","last_clean":"2024-09-21T03:36:22.065557+0000","last_became_active":"2024-09-21T03:36:21.065292+0000","last_became_peered":"2024-09-21T03:36:21.065292+0000","last_unstale":"2024-09-21T03:36:22.065557+0000","last_undegraded":"2024-09-21T03:36:22.065557+0000","last_fullsized":"2024-09-21T03:36:22.065557+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_clean_scrub_stamp":"2024-09-21T03:36:20.027499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:36:50.067003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-21T03:36:22.072312+0000","last_change":"2024-09-21T03:36:22.069640+0000","last_active":"2024-09-21T03:36:22.072312+0000","last_peered":"2024-09-21T03:36:22.072312+0000","last_clean":"2024-09-21T03:36:22.072312+0000","last_became_active":"2024-09-21T03:36:22.068836+0000","last_became_peered":"2024-09-21T03:36:22.068836+0000","last_unstale":"2024-09-21T03:36:22.072312+0000","last_undegraded":"2024-09-21T03:36:22.072312+0000","last_fullsized":"2024-09-21T03:36:22.072312+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:29:10.577562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.660609+0000","last_change":"2024-09-21T03:36:21.062628+0000","last_active":"2024-09-21T03:36:23.660609+0000","last_peered":"2024-09-21T03:36:23.660609+0000","last_clean":"2024-09-21T03:36:23.660609+0000","last_became_active":"2024-09-21T03:36:21.062295+0000","last_became_peered":"2024-09-21T03:36:21.062295+0000","last_unstale":"2024-09-21T03:36:23.660609+0000","last_undegraded":"2024-09-21T03:36:23.660609+0000","last_fullsized":"2024-09-21T03:36:23.660609+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_clean_scrub_stamp":"2024-09-21T03:36:20.027499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:20:15.258373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-21T03:36:22.069673+0000","last_change":"2024-09-21T03:36:22.067826+0000","last_active":"2024-09-21T03:36:22.069673+0000","last_peered":"2024-09-21T03:36:22.069673+0000","last_clean":"2024-09-21T03:36:22.069673+0000","last_became_active":"2024-09-21T03:36:22.067516+0000","last_became_peered":"2024-09-21T03:36:22.067516+0000","last_unstale":"2024-09-21T03:36:22.069673+0000","last_undegraded":"2024-09-21T03:36:22.069673+0000","last_fullsized":"2024-09-21T03:36:22.069673+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:20:16.267783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.661694+0000","last_change":"2024-09-21T03:36:21.063524+0000","last_active":"2024-09-21T03:36:23.661694+0000","last_peered":"2024-09-21T03:36:23.661694+0000","last_clean":"2024-09-21T03:36:23.661694+0000","last_became_active":"2024-09-21T03:36:21.063203+0000","last_became_peered":"2024-09-21T03:36:21.063203+0000","last_unstale":"2024-09-21T03:36:23.661694+0000","last_undegraded":"2024-09-21T03:36:23.661694+0000","last_fullsized":"2024-09-21T03:36:23.661694+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_clean_scrub_stamp":"2024-09-21T03:36:20.027499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:25:29.512603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-21T03:36:22.065852+0000","last_change":"2024-09-21T03:36:22.063211+0000","last_active":"2024-09-21T03:36:22.065852+0000","last_peered":"2024-09-21T03:36:22.065852+0000","last_clean":"2024-09-21T03:36:22.065852+0000","last_became_active":"2024-09-21T03:36:22.062828+0000","last_became_peered":"2024-09-21T03:36:22.062828+0000","last_unstale":"2024-09-21T03:36:22.065852+0000","last_undegraded":"2024-09-21T03:36:22.065852+0000","last_fullsized":"2024-09-21T03:36:22.065852+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:30:41.436799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.661311+0000","last_change":"2024-09-21T03:36:21.064314+0000","last_active":"2024-09-21T03:36:23.661311+0000","last_peered":"2024-09-21T03:36:23.661311+0000","last_clean":"2024-09-21T03:36:23.661311+0000","last_became_active":"2024-09-21T03:36:21.063271+0000","last_became_peered":"2024-09-21T03:36:21.063271+0000","last_unstale":"2024-09-21T03:36:23.661311+0000","last_undegraded":"2024-09-21T03:36:23.661311+0000","last_fullsized":"2024-09-21T03:36:23.661311+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_clean_scrub_stamp":"2024-09-21T03:36:20.027499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:03:09.222194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.660928+0000","last_change":"2024-09-21T03:36:22.092139+0000","last_active":"2024-09-21T03:36:23.660928+0000","last_peered":"2024-09-21T03:36:23.660928+0000","last_clean":"2024-09-21T03:36:23.660928+0000","last_became_active":"2024-09-21T03:36:22.091718+0000","last_became_peered":"2024-09-21T03:36:22.091718+0000","last_unstale":"2024-09-21T03:36:23.660928+0000","last_undegraded":"2024-09-21T03:36:23.660928+0000","last_fullsized":"2024-09-21T03:36:23.660928+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:28:40.036155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.15","version":"66'3","reported_seq":18,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.086559+0000","last_change":"2024-09-21T03:36:21.064387+0000","last_active":"2024-09-21T03:36:23.086559+0000","last_peered":"2024-09-21T03:36:23.086559+0000","last_clean":"2024-09-21T03:36:23.086559+0000","last_became_active":"2024-09-21T03:36:21.064086+0000","last_became_peered":"2024-09-21T03:36:21.064086+0000","last_unstale":"2024-09-21T03:36:23.086559+0000","last_undegraded":"2024-09-21T03:36:23.086559+0000","last_fullsized":"2024-09-21T03:36:23.086559+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_clean_scrub_stamp":"2024-09-21T03:36:20.027499+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:01:58.431275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.086256+0000","last_change":"2024-09-21T03:36:22.070639+0000","last_active":"2024-09-21T03:36:23.086256+0000","last_peered":"2024-09-21T03:36:23.086256+0000","last_clean":"2024-09-21T03:36:23.086256+0000","last_became_active":"2024-09-21T03:36:22.069620+0000","last_became_peered":"2024-09-21T03:36:22.069620+0000","last_unstale":"2024-09-21T03:36:23.086256+0000","last_undegraded":"2024-09-21T03:36:23.086256+0000","last_fullsized":"2024-09-21T03:36:23.086256+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:23:36.147744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-21T03:36:21.066007+0000","last_change":"2024-09-21T03:36:21.065091+0000","last_active":"2024-09-21T03:36:21.066007+0000","last_peered":"2024-09-21T03:36:21.066007+0000","last_clean":"2024-09-21T03:36:21.066007+0000","last_became_active":"2024-09-21T03:36:21.064760+0000","last_became_peered":"2024-09-21T03:36:21.064760+0000","last_unstale":"2024-09-21T03:36:21.066007+0000","last_undegraded":"2024-09-21T03:36:21.066007+0000","last_fullsized":"2024-09-21T03:36:21.066007+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_clean_scrub_stamp":"2024-09-21T03:36:20.027499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:17:34.450050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T03:36:21.036908+0000","last_change":"2024-09-21T03:36:21.036908+0000","last_active":"2024-09-21T03:36:21.036908+0000","last_peered":"2024-09-21T03:36:21.036908+0000","last_clean":"2024-09-21T03:36:21.036908+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:36:21.036908+0000","last_undegraded":"2024-09-21T03:36:21.036908+0000","last_fullsized":"2024-09-21T03:36:21.036908+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.13","version":"66'2","reported_seq":17,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.661866+0000","last_change":"2024-09-21T03:36:21.064320+0000","last_active":"2024-09-21T03:36:23.661866+0000","last_peered":"2024-09-21T03:36:23.661866+0000","last_clean":"2024-09-21T03:36:23.661866+0000","last_became_active":"2024-09-21T03:36:21.064030+0000","last_became_peered":"2024-09-21T03:36:21.064030+0000","last_unstale":"2024-09-21T03:36:23.661866+0000","last_undegraded":"2024-09-21T03:36:23.661866+0000","last_fullsized":"2024-09-21T03:36:23.661866+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_clean_scrub_stamp":"2024-09-21T03:36:20.027499+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:36:50.067003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T03:36:21.036908+0000","last_change":"2024-09-21T03:36:21.036908+0000","last_active":"2024-09-21T03:36:21.036908+0000","last_peered":"2024-09-21T03:36:21.036908+0000","last_clean":"2024-09-21T03:36:21.036908+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:36:21.036908+0000","last_undegraded":"2024-09-21T03:36:21.036908+0000","last_fullsized":"2024-09-21T03:36:21.036908+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T03:36:20.027499+0000","last_change":"2024-09-21T03:36:20.027499+0000","last_active":"2024-09-21T03:36:20.027499+0000","last_peered":"2024-09-21T03:36:20.027499+0000","last_clean":"2024-09-21T03:36:20.027499+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:36:20.027499+0000","last_undegraded":"2024-09-21T03:36:20.027499+0000","last_fullsized":"2024-09-21T03:36:20.027499+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_clean_scrub_stamp":"2024-09-21T03:36:20.027499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T03:36:21.036908+0000","last_change":"2024-09-21T03:36:21.036908+0000","last_active":"2024-09-21T03:36:21.036908+0000","last_peered":"2024-09-21T03:36:21.036908+0000","last_clean":"2024-09-21T03:36:21.036908+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:36:21.036908+0000","last_undegraded":"2024-09-21T03:36:21.036908+0000","last_fullsized":"2024-09-21T03:36:21.036908+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.085748+0000","last_change":"2024-09-21T03:36:21.061510+0000","last_active":"2024-09-21T03:36:23.085748+0000","last_peered":"2024-09-21T03:36:23.085748+0000","last_clean":"2024-09-21T03:36:23.085748+0000","last_became_active":"2024-09-21T03:36:21.061181+0000","last_became_peered":"2024-09-21T03:36:21.061181+0000","last_unstale":"2024-09-21T03:36:23.085748+0000","last_undegraded":"2024-09-21T03:36:23.085748+0000","last_fullsized":"2024-09-21T03:36:23.085748+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_clean_scrub_stamp":"2024-09-21T03:36:20.027499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:39:48.217217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-21T03:36:22.086623+0000","last_change":"2024-09-21T03:36:22.085750+0000","last_active":"2024-09-21T03:36:22.086623+0000","last_peered":"2024-09-21T03:36:22.086623+0000","last_clean":"2024-09-21T03:36:22.086623+0000","last_became_active":"2024-09-21T03:36:22.085426+0000","last_became_peered":"2024-09-21T03:36:22.085426+0000","last_unstale":"2024-09-21T03:36:22.086623+0000","last_undegraded":"2024-09-21T03:36:22.086623+0000","last_fullsized":"2024-09-21T03:36:22.086623+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:43:37.785209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T03:36:20.027499+0000","last_change":"2024-09-21T03:36:20.027499+0000","last_active":"2024-09-21T03:36:20.027499+0000","last_peered":"2024-09-21T03:36:20.027499+0000","last_clean":"2024-09-21T03:36:20.027499+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:36:20.027499+0000","last_undegraded":"2024-09-21T03:36:20.027499+0000","last_fullsized":"2024-09-21T03:36:20.027499+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_clean_scrub_stamp":"2024-09-21T03:36:20.027499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.661909+0000","last_change":"2024-09-21T03:36:22.086802+0000","last_active":"2024-09-21T03:36:23.661909+0000","last_peered":"2024-09-21T03:36:23.661909+0000","last_clean":"2024-09-21T03:36:23.661909+0000","last_became_active":"2024-09-21T03:36:22.086435+0000","last_became_peered":"2024-09-21T03:36:22.086435+0000","last_unstale":"2024-09-21T03:36:23.661909+0000","last_undegraded":"2024-09-21T03:36:23.661909+0000","last_fullsized":"2024-09-21T03:36:23.661909+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:32:26.377539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.f","version":"66'1","reported_seq":16,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.661756+0000","last_change":"2024-09-21T03:36:21.062807+0000","last_active":"2024-09-21T03:36:23.661756+0000","last_peered":"2024-09-21T03:36:23.661756+0000","last_clean":"2024-09-21T03:36:23.661756+0000","last_became_active":"2024-09-21T03:36:21.062513+0000","last_became_peered":"2024-09-21T03:36:21.062513+0000","last_unstale":"2024-09-21T03:36:23.661756+0000","last_undegraded":"2024-09-21T03:36:23.661756+0000","last_fullsized":"2024-09-21T03:36:23.661756+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_clean_scrub_stamp":"2024-09-21T03:36:20.027499+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:39:57.160597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.660925+0000","last_change":"2024-09-21T03:36:22.086393+0000","last_active":"2024-09-21T03:36:23.660925+0000","last_peered":"2024-09-21T03:36:23.660925+0000","last_clean":"2024-09-21T03:36:23.660925+0000","last_became_active":"2024-09-21T03:36:22.086094+0000","last_became_peered":"2024-09-21T03:36:22.086094+0000","last_unstale":"2024-09-21T03:36:23.660925+0000","last_undegraded":"2024-09-21T03:36:23.660925+0000","last_fullsized":"2024-09-21T03:36:23.660925+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:11:34.432933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.e","version":"66'2","reported_seq":17,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.661138+0000","last_change":"2024-09-21T03:36:21.053740+0000","last_active":"2024-09-21T03:36:23.661138+0000","last_peered":"2024-09-21T03:36:23.661138+0000","last_clean":"2024-09-21T03:36:23.661138+0000","last_became_active":"2024-09-21T03:36:21.053383+0000","last_became_peered":"2024-09-21T03:36:21.053383+0000","last_unstale":"2024-09-21T03:36:23.661138+0000","last_undegraded":"2024-09-21T03:36:23.661138+0000","last_fullsized":"2024-09-21T03:36:23.661138+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_clean_scrub_stamp":"2024-09-21T03:36:20.027499+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:37:25.878136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.661181+0000","last_change":"2024-09-21T03:36:22.093865+0000","last_active":"2024-09-21T03:36:23.661181+0000","last_peered":"2024-09-21T03:36:23.661181+0000","last_clean":"2024-09-21T03:36:23.661181+0000","last_became_active":"2024-09-21T03:36:22.093526+0000","last_became_peered":"2024-09-21T03:36:22.093526+0000","last_unstale":"2024-09-21T03:36:23.661181+0000","last_undegraded":"2024-09-21T03:36:23.661181+0000","last_fullsized":"2024-09-21T03:36:23.661181+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:59:12.697168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.d","version":"66'1","reported_seq":16,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.086666+0000","last_change":"2024-09-21T03:36:21.063656+0000","last_active":"2024-09-21T03:36:23.086666+0000","last_peered":"2024-09-21T03:36:23.086666+0000","last_clean":"2024-09-21T03:36:23.086666+0000","last_became_active":"2024-09-21T03:36:21.063355+0000","last_became_peered":"2024-09-21T03:36:21.063355+0000","last_unstale":"2024-09-21T03:36:23.086666+0000","last_undegraded":"2024-09-21T03:36:23.086666+0000","last_fullsized":"2024-09-21T03:36:23.086666+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_clean_scrub_stamp":"2024-09-21T03:36:20.027499+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:05:53.691084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.086342+0000","last_change":"2024-09-21T03:36:22.070297+0000","last_active":"2024-09-21T03:36:23.086342+0000","last_peered":"2024-09-21T03:36:23.086342+0000","last_clean":"2024-09-21T03:36:23.086342+0000","last_became_active":"2024-09-21T03:36:22.069258+0000","last_became_peered":"2024-09-21T03:36:22.069258+0000","last_unstale":"2024-09-21T03:36:23.086342+0000","last_undegraded":"2024-09-21T03:36:23.086342+0000","last_fullsized":"2024-09-21T03:36:23.086342+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:51:57.465792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.660607+0000","last_change":"2024-09-21T03:36:21.067658+0000","last_active":"2024-09-21T03:36:23.660607+0000","last_peered":"2024-09-21T03:36:23.660607+0000","last_clean":"2024-09-21T03:36:23.660607+0000","last_became_active":"2024-09-21T03:36:21.067359+0000","last_became_peered":"2024-09-21T03:36:21.067359+0000","last_unstale":"2024-09-21T03:36:23.660607+0000","last_undegraded":"2024-09-21T03:36:23.660607+0000","last_fullsized":"2024-09-21T03:36:23.660607+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_clean_scrub_stamp":"2024-09-21T03:36:20.027499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:38:26.468485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-21T03:36:21.054221+0000","last_change":"2024-09-21T03:36:21.054221+0000","last_active":"2024-09-21T03:36:21.036908+0000","last_peered":"2024-09-21T03:36:21.036908+0000","last_clean":"2024-09-21T03:36:21.036908+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:36:21.054221+0000","last_undegraded":"2024-09-21T03:36:21.054221+0000","last_fullsized":"2024-09-21T03:36:21.054221+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-21T03:36:21.068841+0000","last_change":"2024-09-21T03:36:21.067047+0000","last_active":"2024-09-21T03:36:21.068841+0000","last_peered":"2024-09-21T03:36:21.068841+0000","last_clean":"2024-09-21T03:36:21.068841+0000","last_became_active":"2024-09-21T03:36:21.066718+0000","last_became_peered":"2024-09-21T03:36:21.066718+0000","last_unstale":"2024-09-21T03:36:21.068841+0000","last_undegraded":"2024-09-21T03:36:21.068841+0000","last_fullsized":"2024-09-21T03:36:21.068841+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_clean_scrub_stamp":"2024-09-21T03:36:20.027499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:34:30.827273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.660351+0000","last_change":"2024-09-21T03:36:22.081020+0000","last_active":"2024-09-21T03:36:23.660351+0000","last_peered":"2024-09-21T03:36:23.660351+0000","last_clean":"2024-09-21T03:36:23.660351+0000","last_became_active":"2024-09-21T03:36:22.080090+0000","last_became_peered":"2024-09-21T03:36:22.080090+0000","last_unstale":"2024-09-21T03:36:23.660351+0000","last_undegraded":"2024-09-21T03:36:23.660351+0000","last_fullsized":"2024-09-21T03:36:23.660351+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:05:02.647190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T03:36:20.027499+0000","last_change":"2024-09-21T03:36:20.027499+0000","last_active":"2024-09-21T03:36:20.027499+0000","last_peered":"2024-09-21T03:36:20.027499+0000","last_clean":"2024-09-21T03:36:20.027499+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:36:20.027499+0000","last_undegraded":"2024-09-21T03:36:20.027499+0000","last_fullsized":"2024-09-21T03:36:20.027499+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_clean_scrub_stamp":"2024-09-21T03:36:20.027499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.660983+0000","last_change":"2024-09-21T03:36:22.071663+0000","last_active":"2024-09-21T03:36:23.660983+0000","last_peered":"2024-09-21T03:36:23.660983+0000","last_clean":"2024-09-21T03:36:23.660983+0000","last_became_active":"2024-09-21T03:36:22.071326+0000","last_became_peered":"2024-09-21T03:36:22.071326+0000","last_unstale":"2024-09-21T03:36:23.660983+0000","last_undegraded":"2024-09-21T03:36:23.660983+0000","last_fullsized":"2024-09-21T03:36:23.660983+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:33:14.339525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-21T03:36:21.064334+0000","last_change":"2024-09-21T03:36:21.062867+0000","last_active":"2024-09-21T03:36:21.064334+0000","last_peered":"2024-09-21T03:36:21.064334+0000","last_clean":"2024-09-21T03:36:21.064334+0000","last_became_active":"2024-09-21T03:36:21.062555+0000","last_became_peered":"2024-09-21T03:36:21.062555+0000","last_unstale":"2024-09-21T03:36:21.064334+0000","last_undegraded":"2024-09-21T03:36:21.064334+0000","last_fullsized":"2024-09-21T03:36:21.064334+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_clean_scrub_stamp":"2024-09-21T03:36:20.027499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:24:33.618928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-21T03:36:22.066694+0000","last_change":"2024-09-21T03:36:22.063923+0000","last_active":"2024-09-21T03:36:22.066694+0000","last_peered":"2024-09-21T03:36:22.066694+0000","last_clean":"2024-09-21T03:36:22.066694+0000","last_became_active":"2024-09-21T03:36:22.063195+0000","last_became_peered":"2024-09-21T03:36:22.063195+0000","last_unstale":"2024-09-21T03:36:22.066694+0000","last_undegraded":"2024-09-21T03:36:22.066694+0000","last_fullsized":"2024-09-21T03:36:22.066694+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:48:30.795859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.7","version":"66'2","reported_seq":17,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.661858+0000","last_change":"2024-09-21T03:36:21.068456+0000","last_active":"2024-09-21T03:36:23.661858+0000","last_peered":"2024-09-21T03:36:23.661858+0000","last_clean":"2024-09-21T03:36:23.661858+0000","last_became_active":"2024-09-21T03:36:21.068101+0000","last_became_peered":"2024-09-21T03:36:21.068101+0000","last_unstale":"2024-09-21T03:36:23.661858+0000","last_undegraded":"2024-09-21T03:36:23.661858+0000","last_fullsized":"2024-09-21T03:36:23.661858+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_clean_scrub_stamp":"2024-09-21T03:36:20.027499+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:09:49.802044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-21T03:36:22.073022+0000","last_change":"2024-09-21T03:36:22.071078+0000","last_active":"2024-09-21T03:36:22.073022+0000","last_peered":"2024-09-21T03:36:22.073022+0000","last_clean":"2024-09-21T03:36:22.073022+0000","last_became_active":"2024-09-21T03:36:22.070566+0000","last_became_peered":"2024-09-21T03:36:22.070566+0000","last_unstale":"2024-09-21T03:36:22.073022+0000","last_undegraded":"2024-09-21T03:36:22.073022+0000","last_fullsized":"2024-09-21T03:36:22.073022+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:09:22.850109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.661366+0000","last_change":"2024-09-21T03:36:21.059377+0000","last_active":"2024-09-21T03:36:23.661366+0000","last_peered":"2024-09-21T03:36:23.661366+0000","last_clean":"2024-09-21T03:36:23.661366+0000","last_became_active":"2024-09-21T03:36:21.059079+0000","last_became_peered":"2024-09-21T03:36:21.059079+0000","last_unstale":"2024-09-21T03:36:23.661366+0000","last_undegraded":"2024-09-21T03:36:23.661366+0000","last_fullsized":"2024-09-21T03:36:23.661366+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_clean_scrub_stamp":"2024-09-21T03:36:20.027499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:39:00.495803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.086122+0000","last_change":"2024-09-21T03:36:22.072209+0000","last_active":"2024-09-21T03:36:23.086122+0000","last_peered":"2024-09-21T03:36:23.086122+0000","last_clean":"2024-09-21T03:36:23.086122+0000","last_became_active":"2024-09-21T03:36:22.071876+0000","last_became_peered":"2024-09-21T03:36:22.071876+0000","last_unstale":"2024-09-21T03:36:23.086122+0000","last_undegraded":"2024-09-21T03:36:23.086122+0000","last_fullsized":"2024-09-21T03:36:23.086122+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:37:07.049180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-21T03:36:21.064294+0000","last_change":"2024-09-21T03:36:21.063184+0000","last_active":"2024-09-21T03:36:21.064294+0000","last_peered":"2024-09-21T03:36:21.064294+0000","last_clean":"2024-09-21T03:36:21.064294+0000","last_became_active":"2024-09-21T03:36:21.062859+0000","last_became_peered":"2024-09-21T03:36:21.062859+0000","last_unstale":"2024-09-21T03:36:21.064294+0000","last_undegraded":"2024-09-21T03:36:21.064294+0000","last_fullsized":"2024-09-21T03:36:21.064294+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_clean_scrub_stamp":"2024-09-21T03:36:20.027499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:18:15.549734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.086389+0000","last_change":"2024-09-21T03:36:22.069954+0000","last_active":"2024-09-21T03:36:23.086389+0000","last_peered":"2024-09-21T03:36:23.086389+0000","last_clean":"2024-09-21T03:36:23.086389+0000","last_became_active":"2024-09-21T03:36:22.068916+0000","last_became_peered":"2024-09-21T03:36:22.068916+0000","last_unstale":"2024-09-21T03:36:23.086389+0000","last_undegraded":"2024-09-21T03:36:23.086389+0000","last_fullsized":"2024-09-21T03:36:23.086389+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:03:46.925197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.660652+0000","last_change":"2024-09-21T03:36:21.071978+0000","last_active":"2024-09-21T03:36:23.660652+0000","last_peered":"2024-09-21T03:36:23.660652+0000","last_clean":"2024-09-21T03:36:23.660652+0000","last_became_active":"2024-09-21T03:36:21.071717+0000","last_became_peered":"2024-09-21T03:36:21.071717+0000","last_unstale":"2024-09-21T03:36:23.660652+0000","last_undegraded":"2024-09-21T03:36:23.660652+0000","last_fullsized":"2024-09-21T03:36:23.660652+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_clean_scrub_stamp":"2024-09-21T03:36:20.027499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:46:05.982882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.661216+0000","last_change":"2024-09-21T03:36:22.072086+0000","last_active":"2024-09-21T03:36:23.661216+0000","last_peered":"2024-09-21T03:36:23.661216+0000","last_clean":"2024-09-21T03:36:23.661216+0000","last_became_active":"2024-09-21T03:36:22.071729+0000","last_became_peered":"2024-09-21T03:36:22.071729+0000","last_unstale":"2024-09-21T03:36:23.661216+0000","last_undegraded":"2024-09-21T03:36:23.661216+0000","last_fullsized":"2024-09-21T03:36:23.661216+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:30:14.015775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.660720+0000","last_change":"2024-09-21T03:36:21.066336+0000","last_active":"2024-09-21T03:36:23.660720+0000","last_peered":"2024-09-21T03:36:23.660720+0000","last_clean":"2024-09-21T03:36:23.660720+0000","last_became_active":"2024-09-21T03:36:21.066043+0000","last_became_peered":"2024-09-21T03:36:21.066043+0000","last_unstale":"2024-09-21T03:36:23.660720+0000","last_undegraded":"2024-09-21T03:36:23.660720+0000","last_fullsized":"2024-09-21T03:36:23.660720+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_clean_scrub_stamp":"2024-09-21T03:36:20.027499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:45:07.961665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.660391+0000","last_change":"2024-09-21T03:36:22.089730+0000","last_active":"2024-09-21T03:36:23.660391+0000","last_peered":"2024-09-21T03:36:23.660391+0000","last_clean":"2024-09-21T03:36:23.660391+0000","last_became_active":"2024-09-21T03:36:22.089403+0000","last_became_peered":"2024-09-21T03:36:22.089403+0000","last_unstale":"2024-09-21T03:36:23.660391+0000","last_undegraded":"2024-09-21T03:36:23.660391+0000","last_fullsized":"2024-09-21T03:36:23.660391+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:38:24.463402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":13,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-21T03:36:22.061045+0000","last_change":"2024-09-21T03:36:21.066672+0000","last_active":"2024-09-21T03:36:22.061045+0000","last_peered":"2024-09-21T03:36:22.061045+0000","last_clean":"2024-09-21T03:36:22.061045+0000","last_became_active":"2024-09-21T03:36:21.066366+0000","last_became_peered":"2024-09-21T03:36:21.066366+0000","last_unstale":"2024-09-21T03:36:22.061045+0000","last_undegraded":"2024-09-21T03:36:22.061045+0000","last_fullsized":"2024-09-21T03:36:22.061045+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_clean_scrub_stamp":"2024-09-21T03:36:20.027499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:28:15.884066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-21T03:36:21.054594+0000","last_change":"2024-09-21T03:36:21.054594+0000","last_active":"2024-09-21T03:36:21.036908+0000","last_peered":"2024-09-21T03:36:21.036908+0000","last_clean":"2024-09-21T03:36:21.036908+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:36:21.054594+0000","last_undegraded":"2024-09-21T03:36:21.054594+0000","last_fullsized":"2024-09-21T03:36:21.054594+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.660737+0000","last_change":"2024-09-21T03:36:21.065843+0000","last_active":"2024-09-21T03:36:23.660737+0000","last_peered":"2024-09-21T03:36:23.660737+0000","last_clean":"2024-09-21T03:36:23.660737+0000","last_became_active":"2024-09-21T03:36:21.065466+0000","last_became_peered":"2024-09-21T03:36:21.065466+0000","last_unstale":"2024-09-21T03:36:23.660737+0000","last_undegraded":"2024-09-21T03:36:23.660737+0000","last_fullsized":"2024-09-21T03:36:23.660737+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_clean_scrub_stamp":"2024-09-21T03:36:20.027499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:10:41.577481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-21T03:36:22.072278+0000","last_change":"2024-09-21T03:36:22.068997+0000","last_active":"2024-09-21T03:36:22.072278+0000","last_peered":"2024-09-21T03:36:22.072278+0000","last_clean":"2024-09-21T03:36:22.072278+0000","last_became_active":"2024-09-21T03:36:22.067174+0000","last_became_peered":"2024-09-21T03:36:22.067174+0000","last_unstale":"2024-09-21T03:36:22.072278+0000","last_undegraded":"2024-09-21T03:36:22.072278+0000","last_fullsized":"2024-09-21T03:36:22.072278+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:42:28.131400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-21T03:36:21.069684+0000","last_change":"2024-09-21T03:36:21.067567+0000","last_active":"2024-09-21T03:36:21.069684+0000","last_peered":"2024-09-21T03:36:21.069684+0000","last_clean":"2024-09-21T03:36:21.069684+0000","last_became_active":"2024-09-21T03:36:21.067235+0000","last_became_peered":"2024-09-21T03:36:21.067235+0000","last_unstale":"2024-09-21T03:36:21.069684+0000","last_undegraded":"2024-09-21T03:36:21.069684+0000","last_fullsized":"2024-09-21T03:36:21.069684+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_clean_scrub_stamp":"2024-09-21T03:36:20.027499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:24:32.368652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T03:36:21.036908+0000","last_change":"2024-09-21T03:36:21.036908+0000","last_active":"2024-09-21T03:36:21.036908+0000","last_peered":"2024-09-21T03:36:21.036908+0000","last_clean":"2024-09-21T03:36:21.036908+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:36:21.036908+0000","last_undegraded":"2024-09-21T03:36:21.036908+0000","last_fullsized":"2024-09-21T03:36:21.036908+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-21T03:36:21.062519+0000","last_change":"2024-09-21T03:36:21.059196+0000","last_active":"2024-09-21T03:36:21.062519+0000","last_peered":"2024-09-21T03:36:21.062519+0000","last_clean":"2024-09-21T03:36:21.062519+0000","last_became_active":"2024-09-21T03:36:21.058881+0000","last_became_peered":"2024-09-21T03:36:21.058881+0000","last_unstale":"2024-09-21T03:36:21.062519+0000","last_undegraded":"2024-09-21T03:36:21.062519+0000","last_fullsized":"2024-09-21T03:36:21.062519+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_clean_scrub_stamp":"2024-09-21T03:36:20.027499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:37:14.541836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":190,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-21T03:36:18.037492+0000","last_change":"2024-09-21T03:33:25.319776+0000","last_active":"2024-09-21T03:36:18.037492+0000","last_peered":"2024-09-21T03:36:18.037492+0000","last_clean":"2024-09-21T03:36:18.037492+0000","last_became_active":"2024-09-21T03:33:25.318399+0000","last_became_peered":"2024-09-21T03:33:25.318399+0000","last_unstale":"2024-09-21T03:36:18.037492+0000","last_undegraded":"2024-09-21T03:36:18.037492+0000","last_fullsized":"2024-09-21T03:36:18.037492+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:23.016672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:23.016672+0000","last_clean_scrub_stamp":"2024-09-21T03:33:23.016672+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:32:35.906114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":114,"acting":114,"num_store_stats":7},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":798,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19,"ondisk_log_size":19,"up":58,"acting":58,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":1,"up_from":16,"seq":68719476775,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32508,"kb_used_data":1316,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339332,"statfs":{"total":96636764160,"available":96603475968,"internally_reserved":0,"allocated":1347584,"data_stored":927955,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509480,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36124,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335716,"statfs":{"total":96636764160,"available":96599773184,"internally_reserved":0,"allocated":790528,"data_stored":357630,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":36174283},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509479,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31980,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339860,"statfs":{"total":96636764160,"available":96604016640,"internally_reserved":0,"allocated":811008,"data_stored":364482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509480,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31980,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339860,"statfs":{"total":96636764160,"available":96604016640,"internally_reserved":0,"allocated":811008,"data_stored":364378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509479,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31968,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339872,"statfs":{"total":96636764160,"available":96604028928,"internally_reserved":0,"allocated":790528,"data_stored":357630,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542183,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36148,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335692,"statfs":{"total":96636764160,"available":96599748608,"internally_reserved":0,"allocated":811008,"data_stored":359894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574887,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36692,"kb_used_data":1340,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335148,"statfs":{"total":96636764160,"available":96599191552,"internally_reserved":0,"allocated":1372160,"data_stored":940831,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542184,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36132,"kb_used_data":776,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335708,"statfs":{"total":96636764160,"available":96599764992,"internally_reserved":0,"allocated":794624,"data_stored":358212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T03:36:26.902 INFO:tasks.ceph.ceph_manager.ceph:PG 10.3e is not active+clean 2024-09-21T03:36:26.902 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2024-09-21T03:36:21.054741+0000', 'last_change': '2024-09-21T03:36:21.054741+0000', 'last_active': '2024-09-21T03:36:21.036908+0000', 'last_peered': '2024-09-21T03:36:21.036908+0000', 'last_clean': '2024-09-21T03:36:21.036908+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T03:36:21.054741+0000', 'last_undegraded': '2024-09-21T03:36:21.054741+0000', 'last_fullsized': '2024-09-21T03:36:21.054741+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T03:36:21.036908+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T03:36:21.036908+0000', 'last_clean_scrub_stamp': '2024-09-21T03:36:21.036908+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-21T03:36:26.902 INFO:tasks.ceph.ceph_manager.ceph:PG 10.3a is not active+clean 2024-09-21T03:36:26.902 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2024-09-21T03:36:21.055099+0000', 'last_change': '2024-09-21T03:36:21.055099+0000', 'last_active': '2024-09-21T03:36:21.036908+0000', 'last_peered': '2024-09-21T03:36:21.036908+0000', 'last_clean': '2024-09-21T03:36:21.036908+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T03:36:21.055099+0000', 'last_undegraded': '2024-09-21T03:36:21.055099+0000', 'last_fullsized': '2024-09-21T03:36:21.055099+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T03:36:21.036908+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T03:36:21.036908+0000', 'last_clean_scrub_stamp': '2024-09-21T03:36:21.036908+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-21T03:36:26.903 INFO:tasks.ceph.ceph_manager.ceph:PG 10.36 is not active+clean 2024-09-21T03:36:26.903 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T03:36:21.036908+0000', 'last_change': '2024-09-21T03:36:21.036908+0000', 'last_active': '2024-09-21T03:36:21.036908+0000', 'last_peered': '2024-09-21T03:36:21.036908+0000', 'last_clean': '2024-09-21T03:36:21.036908+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T03:36:21.036908+0000', 'last_undegraded': '2024-09-21T03:36:21.036908+0000', 'last_fullsized': '2024-09-21T03:36:21.036908+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T03:36:21.036908+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T03:36:21.036908+0000', 'last_clean_scrub_stamp': '2024-09-21T03:36:21.036908+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T03:36:26.903 INFO:tasks.ceph.ceph_manager.ceph:PG 10.31 is not active+clean 2024-09-21T03:36:26.903 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T03:36:21.036908+0000', 'last_change': '2024-09-21T03:36:21.036908+0000', 'last_active': '2024-09-21T03:36:21.036908+0000', 'last_peered': '2024-09-21T03:36:21.036908+0000', 'last_clean': '2024-09-21T03:36:21.036908+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T03:36:21.036908+0000', 'last_undegraded': '2024-09-21T03:36:21.036908+0000', 'last_fullsized': '2024-09-21T03:36:21.036908+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T03:36:21.036908+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T03:36:21.036908+0000', 'last_clean_scrub_stamp': '2024-09-21T03:36:21.036908+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T03:36:26.903 INFO:tasks.ceph.ceph_manager.ceph:PG 10.2a is not active+clean 2024-09-21T03:36:26.903 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2024-09-21T03:36:21.053928+0000', 'last_change': '2024-09-21T03:36:21.053928+0000', 'last_active': '2024-09-21T03:36:21.036908+0000', 'last_peered': '2024-09-21T03:36:21.036908+0000', 'last_clean': '2024-09-21T03:36:21.036908+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T03:36:21.053928+0000', 'last_undegraded': '2024-09-21T03:36:21.053928+0000', 'last_fullsized': '2024-09-21T03:36:21.053928+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T03:36:21.036908+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T03:36:21.036908+0000', 'last_clean_scrub_stamp': '2024-09-21T03:36:21.036908+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-21T03:36:26.903 INFO:tasks.ceph.ceph_manager.ceph:PG 10.1e is not active+clean 2024-09-21T03:36:26.903 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2024-09-21T03:36:21.054153+0000', 'last_change': '2024-09-21T03:36:21.054153+0000', 'last_active': '2024-09-21T03:36:21.036908+0000', 'last_peered': '2024-09-21T03:36:21.036908+0000', 'last_clean': '2024-09-21T03:36:21.036908+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T03:36:21.054153+0000', 'last_undegraded': '2024-09-21T03:36:21.054153+0000', 'last_fullsized': '2024-09-21T03:36:21.054153+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T03:36:21.036908+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T03:36:21.036908+0000', 'last_clean_scrub_stamp': '2024-09-21T03:36:21.036908+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-21T03:36:26.903 INFO:tasks.ceph.ceph_manager.ceph:PG 10.1a is not active+clean 2024-09-21T03:36:26.903 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T03:36:21.036908+0000', 'last_change': '2024-09-21T03:36:21.036908+0000', 'last_active': '2024-09-21T03:36:21.036908+0000', 'last_peered': '2024-09-21T03:36:21.036908+0000', 'last_clean': '2024-09-21T03:36:21.036908+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T03:36:21.036908+0000', 'last_undegraded': '2024-09-21T03:36:21.036908+0000', 'last_fullsized': '2024-09-21T03:36:21.036908+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T03:36:21.036908+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T03:36:21.036908+0000', 'last_clean_scrub_stamp': '2024-09-21T03:36:21.036908+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T03:36:26.903 INFO:tasks.ceph.ceph_manager.ceph:PG 10.10 is not active+clean 2024-09-21T03:36:26.904 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T03:36:21.036908+0000', 'last_change': '2024-09-21T03:36:21.036908+0000', 'last_active': '2024-09-21T03:36:21.036908+0000', 'last_peered': '2024-09-21T03:36:21.036908+0000', 'last_clean': '2024-09-21T03:36:21.036908+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T03:36:21.036908+0000', 'last_undegraded': '2024-09-21T03:36:21.036908+0000', 'last_fullsized': '2024-09-21T03:36:21.036908+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T03:36:21.036908+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T03:36:21.036908+0000', 'last_clean_scrub_stamp': '2024-09-21T03:36:21.036908+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T03:36:26.904 INFO:tasks.ceph.ceph_manager.ceph:PG 10.11 is not active+clean 2024-09-21T03:36:26.904 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T03:36:21.036908+0000', 'last_change': '2024-09-21T03:36:21.036908+0000', 'last_active': '2024-09-21T03:36:21.036908+0000', 'last_peered': '2024-09-21T03:36:21.036908+0000', 'last_clean': '2024-09-21T03:36:21.036908+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T03:36:21.036908+0000', 'last_undegraded': '2024-09-21T03:36:21.036908+0000', 'last_fullsized': '2024-09-21T03:36:21.036908+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T03:36:21.036908+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T03:36:21.036908+0000', 'last_clean_scrub_stamp': '2024-09-21T03:36:21.036908+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T03:36:26.904 INFO:tasks.ceph.ceph_manager.ceph:PG 9.12 is not active+clean 2024-09-21T03:36:26.904 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T03:36:20.027499+0000', 'last_change': '2024-09-21T03:36:20.027499+0000', 'last_active': '2024-09-21T03:36:20.027499+0000', 'last_peered': '2024-09-21T03:36:20.027499+0000', 'last_clean': '2024-09-21T03:36:20.027499+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T03:36:20.027499+0000', 'last_undegraded': '2024-09-21T03:36:20.027499+0000', 'last_fullsized': '2024-09-21T03:36:20.027499+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T03:36:20.027499+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T03:36:20.027499+0000', 'last_clean_scrub_stamp': '2024-09-21T03:36:20.027499+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T03:36:26.904 INFO:tasks.ceph.ceph_manager.ceph:PG 10.12 is not active+clean 2024-09-21T03:36:26.904 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T03:36:21.036908+0000', 'last_change': '2024-09-21T03:36:21.036908+0000', 'last_active': '2024-09-21T03:36:21.036908+0000', 'last_peered': '2024-09-21T03:36:21.036908+0000', 'last_clean': '2024-09-21T03:36:21.036908+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T03:36:21.036908+0000', 'last_undegraded': '2024-09-21T03:36:21.036908+0000', 'last_fullsized': '2024-09-21T03:36:21.036908+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T03:36:21.036908+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T03:36:21.036908+0000', 'last_clean_scrub_stamp': '2024-09-21T03:36:21.036908+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T03:36:26.904 INFO:tasks.ceph.ceph_manager.ceph:PG 9.10 is not active+clean 2024-09-21T03:36:26.904 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T03:36:20.027499+0000', 'last_change': '2024-09-21T03:36:20.027499+0000', 'last_active': '2024-09-21T03:36:20.027499+0000', 'last_peered': '2024-09-21T03:36:20.027499+0000', 'last_clean': '2024-09-21T03:36:20.027499+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T03:36:20.027499+0000', 'last_undegraded': '2024-09-21T03:36:20.027499+0000', 'last_fullsized': '2024-09-21T03:36:20.027499+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T03:36:20.027499+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T03:36:20.027499+0000', 'last_clean_scrub_stamp': '2024-09-21T03:36:20.027499+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T03:36:26.904 INFO:tasks.ceph.ceph_manager.ceph:PG 10.8 is not active+clean 2024-09-21T03:36:26.904 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2024-09-21T03:36:21.054221+0000', 'last_change': '2024-09-21T03:36:21.054221+0000', 'last_active': '2024-09-21T03:36:21.036908+0000', 'last_peered': '2024-09-21T03:36:21.036908+0000', 'last_clean': '2024-09-21T03:36:21.036908+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T03:36:21.054221+0000', 'last_undegraded': '2024-09-21T03:36:21.054221+0000', 'last_fullsized': '2024-09-21T03:36:21.054221+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T03:36:21.036908+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T03:36:21.036908+0000', 'last_clean_scrub_stamp': '2024-09-21T03:36:21.036908+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-21T03:36:26.905 INFO:tasks.ceph.ceph_manager.ceph:PG 9.a is not active+clean 2024-09-21T03:36:26.905 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T03:36:20.027499+0000', 'last_change': '2024-09-21T03:36:20.027499+0000', 'last_active': '2024-09-21T03:36:20.027499+0000', 'last_peered': '2024-09-21T03:36:20.027499+0000', 'last_clean': '2024-09-21T03:36:20.027499+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T03:36:20.027499+0000', 'last_undegraded': '2024-09-21T03:36:20.027499+0000', 'last_fullsized': '2024-09-21T03:36:20.027499+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T03:36:20.027499+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T03:36:20.027499+0000', 'last_clean_scrub_stamp': '2024-09-21T03:36:20.027499+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T03:36:26.905 INFO:tasks.ceph.ceph_manager.ceph:PG 10.2 is not active+clean 2024-09-21T03:36:26.905 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2024-09-21T03:36:21.054594+0000', 'last_change': '2024-09-21T03:36:21.054594+0000', 'last_active': '2024-09-21T03:36:21.036908+0000', 'last_peered': '2024-09-21T03:36:21.036908+0000', 'last_clean': '2024-09-21T03:36:21.036908+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T03:36:21.054594+0000', 'last_undegraded': '2024-09-21T03:36:21.054594+0000', 'last_fullsized': '2024-09-21T03:36:21.054594+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T03:36:21.036908+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T03:36:21.036908+0000', 'last_clean_scrub_stamp': '2024-09-21T03:36:21.036908+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-21T03:36:26.905 INFO:tasks.ceph.ceph_manager.ceph:PG 10.b is not active+clean 2024-09-21T03:36:26.905 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T03:36:21.036908+0000', 'last_change': '2024-09-21T03:36:21.036908+0000', 'last_active': '2024-09-21T03:36:21.036908+0000', 'last_peered': '2024-09-21T03:36:21.036908+0000', 'last_clean': '2024-09-21T03:36:21.036908+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T03:36:21.036908+0000', 'last_undegraded': '2024-09-21T03:36:21.036908+0000', 'last_fullsized': '2024-09-21T03:36:21.036908+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T03:36:21.036908+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T03:36:21.036908+0000', 'last_clean_scrub_stamp': '2024-09-21T03:36:21.036908+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T03:36:26.906 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-21T03:36:27.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:27.053+0000 7f64ef9c3640 1 -- 172.21.15.38:0/4065073663 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f64e80d0b40 msgr2=0x7f64e806e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:27.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:27.053+0000 7f64ef9c3640 1 --2- 172.21.15.38:0/4065073663 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f64e80d0b40 0x7f64e806e6f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f64e4005fd0 tx=0x7f64e402a5c0 comp rx=0 tx=0).stop 2024-09-21T03:36:27.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:27.053+0000 7f64ef9c3640 1 -- 172.21.15.38:0/4065073663 shutdown_connections 2024-09-21T03:36:27.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:27.053+0000 7f64ef9c3640 1 --2- 172.21.15.38:0/4065073663 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f64e8073d50 0x7f64e8074160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:27.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:27.053+0000 7f64ef9c3640 1 --2- 172.21.15.38:0/4065073663 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f64e80d0b40 0x7f64e806e6f0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:27.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:27.053+0000 7f64ef9c3640 1 -- 172.21.15.38:0/4065073663 >> 172.21.15.38:0/4065073663 conn(0x7f64e80b2cc0 msgr2=0x7f64e80b30d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:27.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:27.053+0000 7f64ef9c3640 1 -- 172.21.15.38:0/4065073663 shutdown_connections 2024-09-21T03:36:27.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:27.053+0000 7f64ef9c3640 1 -- 172.21.15.38:0/4065073663 wait complete. 2024-09-21T03:36:27.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:27.055+0000 7f64ef9c3640 1 Processor -- start 2024-09-21T03:36:27.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:27.055+0000 7f64ef9c3640 1 -- start start 2024-09-21T03:36:27.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:27.056+0000 7f64ef9c3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f64e806ec30 0x7f64e81f6880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:27.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:27.056+0000 7f64ef9c3640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f64e8073d50 0x7f64e8150ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:27.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:27.056+0000 7f64ef9c3640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f64e80d0b40 0x7f64e81f3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:27.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:27.056+0000 7f64ef9c3640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f64e815b750 con 0x7f64e806ec30 2024-09-21T03:36:27.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:27.056+0000 7f64ef9c3640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f64e815b5d0 con 0x7f64e8073d50 2024-09-21T03:36:27.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:27.056+0000 7f64ef9c3640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f64e815b8d0 con 0x7f64e80d0b40 2024-09-21T03:36:27.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:27.056+0000 7f64ed75f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f64e806ec30 0x7f64e81f6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:27.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:27.056+0000 7f64ed75f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f64e806ec30 0x7f64e81f6880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37346/0 (socket says 172.21.15.38:37346) 2024-09-21T03:36:27.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:27.056+0000 7f64ed75f640 1 -- 172.21.15.38:0/3362960215 learned_addr learned my addr 172.21.15.38:0/3362960215 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:36:27.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:27.056+0000 7f64edf60640 1 --2- 172.21.15.38:0/3362960215 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f64e80d0b40 0x7f64e81f3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:27.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:27.056+0000 7f64ecf5e640 1 --2- 172.21.15.38:0/3362960215 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f64e8073d50 0x7f64e8150ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:27.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:27.056+0000 7f64ed75f640 1 -- 172.21.15.38:0/3362960215 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f64e80d0b40 msgr2=0x7f64e81f3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:27.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:27.056+0000 7f64ed75f640 1 --2- 172.21.15.38:0/3362960215 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f64e80d0b40 0x7f64e81f3ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:27.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:27.057+0000 7f64ed75f640 1 -- 172.21.15.38:0/3362960215 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f64e8073d50 msgr2=0x7f64e8150ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:27.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:27.057+0000 7f64ed75f640 1 --2- 172.21.15.38:0/3362960215 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f64e8073d50 0x7f64e8150ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:27.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:27.057+0000 7f64ed75f640 1 -- 172.21.15.38:0/3362960215 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64e81f71f0 con 0x7f64e806ec30 2024-09-21T03:36:27.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:27.057+0000 7f64ed75f640 1 --2- 172.21.15.38:0/3362960215 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f64e806ec30 0x7f64e81f6880 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f64e4000c00 tx=0x7f64e40038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:27.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:27.057+0000 7f64ddffb640 1 -- 172.21.15.38:0/3362960215 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64e403e030 con 0x7f64e806ec30 2024-09-21T03:36:27.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:27.057+0000 7f64ef9c3640 1 -- 172.21.15.38:0/3362960215 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64e81f74d0 con 0x7f64e806ec30 2024-09-21T03:36:27.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:27.057+0000 7f64ddffb640 1 -- 172.21.15.38:0/3362960215 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f64e4003db0 con 0x7f64e806ec30 2024-09-21T03:36:27.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:27.057+0000 7f64ddffb640 1 -- 172.21.15.38:0/3362960215 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64e4035040 con 0x7f64e806ec30 2024-09-21T03:36:27.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:27.057+0000 7f64ef9c3640 1 -- 172.21.15.38:0/3362960215 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64e8154090 con 0x7f64e806ec30 2024-09-21T03:36:27.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:27.058+0000 7f64ef9c3640 1 -- 172.21.15.38:0/3362960215 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64e81f7710 con 0x7f64e806ec30 2024-09-21T03:36:27.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:27.059+0000 7f64ddffb640 1 -- 172.21.15.38:0/3362960215 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f64e4041030 con 0x7f64e806ec30 2024-09-21T03:36:27.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:27.060+0000 7f64ddffb640 1 --2- 172.21.15.38:0/3362960215 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f64cc078b90 0x7f64cc07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:27.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:27.060+0000 7f64ddffb640 1 -- 172.21.15.38:0/3362960215 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6101+0+0 (secure 0 0 0) 0x7f64e40bb5a0 con 0x7f64e806ec30 2024-09-21T03:36:27.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:27.060+0000 7f64ecf5e640 1 --2- 172.21.15.38:0/3362960215 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f64cc078b90 0x7f64cc07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:27.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:27.060+0000 7f64ecf5e640 1 --2- 172.21.15.38:0/3362960215 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f64cc078b90 0x7f64cc07b0a0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f64e81536a0 tx=0x7f64d8009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:27.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:27.064+0000 7f64ddffb640 1 -- 172.21.15.38:0/3362960215 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f64e81f7710 con 0x7f64e806ec30 2024-09-21T03:36:27.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:27.331+0000 7f64ef9c3640 1 -- 172.21.15.38:0/3362960215 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f64e815ae90 con 0x7f64e806ec30 2024-09-21T03:36:27.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:27.331+0000 7f64ddffb640 1 -- 172.21.15.38:0/3362960215 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1561 (secure 0 0 0) 0x7f64e4083170 con 0x7f64e806ec30 2024-09-21T03:36:27.333 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:36:27.334 INFO:teuthology.orchestra.run.smithi038.stdout:{"fsid":"9d50d66f-656b-4fe2-8e3b-fe53b79c4d43","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":196,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":67,"num_osds":8,"num_up_osds":8,"osd_up_since":1726889604,"num_in_osds":8,"osd_in_since":1726889594,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":91},{"state_name":"creating+peering","count":6}],"num_pgs":97,"num_pools":3,"num_objects":17,"data_bytes":591166,"bytes_used":280154112,"bytes_avail":772813959168,"bytes_total":773094113280,"inactive_pgs_ratio":0.061855670064687729,"write_bytes_sec":1876,"read_op_per_sec":0,"write_op_per_sec":5},"fsmap":{"epoch":45,"btime":"2024-09-21T03:36:25:092622+0000","id":4,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":4,"rank":0,"name":"a","status":"up:active","gid":5991}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":10,"modified":"2024-09-21T03:36:05.018410+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-21T03:36:27.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:27.334+0000 7f64ef9c3640 1 -- 172.21.15.38:0/3362960215 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f64cc078b90 msgr2=0x7f64cc07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:27.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:27.334+0000 7f64ef9c3640 1 --2- 172.21.15.38:0/3362960215 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f64cc078b90 0x7f64cc07b0a0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f64e81536a0 tx=0x7f64d8009040 comp rx=0 tx=0).stop 2024-09-21T03:36:27.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:27.334+0000 7f64ef9c3640 1 -- 172.21.15.38:0/3362960215 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f64e806ec30 msgr2=0x7f64e81f6880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:27.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:27.334+0000 7f64ef9c3640 1 --2- 172.21.15.38:0/3362960215 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f64e806ec30 0x7f64e81f6880 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f64e4000c00 tx=0x7f64e40038a0 comp rx=0 tx=0).stop 2024-09-21T03:36:27.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:27.334+0000 7f64ef9c3640 1 -- 172.21.15.38:0/3362960215 shutdown_connections 2024-09-21T03:36:27.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:27.334+0000 7f64ef9c3640 1 --2- 172.21.15.38:0/3362960215 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f64cc078b90 0x7f64cc07b0a0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:27.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:27.334+0000 7f64ef9c3640 1 --2- 172.21.15.38:0/3362960215 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f64e80d0b40 0x7f64e81f3ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:27.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:27.334+0000 7f64ef9c3640 1 --2- 172.21.15.38:0/3362960215 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f64e8073d50 0x7f64e8150ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:27.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:27.334+0000 7f64ef9c3640 1 --2- 172.21.15.38:0/3362960215 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f64e806ec30 0x7f64e81f6880 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:27.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:27.334+0000 7f64ef9c3640 1 -- 172.21.15.38:0/3362960215 >> 172.21.15.38:0/3362960215 conn(0x7f64e80b2cc0 msgr2=0x7f64e80b30d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:27.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:27.334+0000 7f64ef9c3640 1 -- 172.21.15.38:0/3362960215 shutdown_connections 2024-09-21T03:36:27.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:27.334+0000 7f64ef9c3640 1 -- 172.21.15.38:0/3362960215 wait complete. 2024-09-21T03:36:27.347 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-21T03:36:27.347 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T03:36:27.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:27.461+0000 7f8066550640 1 -- 172.21.15.38:0/4018331936 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f80600ce860 msgr2=0x7f806015f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:27.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:27.461+0000 7f8066550640 1 --2- 172.21.15.38:0/4018331936 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f80600ce860 0x7f806015f5a0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f8060072d40 tx=0x7f805802a8b0 comp rx=0 tx=0).stop 2024-09-21T03:36:27.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:27.461+0000 7f8066550640 1 -- 172.21.15.38:0/4018331936 shutdown_connections 2024-09-21T03:36:27.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:27.461+0000 7f8066550640 1 --2- 172.21.15.38:0/4018331936 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f80600ce860 0x7f806015f5a0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:27.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:27.461+0000 7f8066550640 1 --2- 172.21.15.38:0/4018331936 >> v2:172.21.15.195:3300/0 conn(0x7f80600cf930 0x7f80600ce320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:27.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:27.461+0000 7f8066550640 1 -- 172.21.15.38:0/4018331936 >> 172.21.15.38:0/4018331936 conn(0x7f80600cca40 msgr2=0x7f80600cce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:27.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:27.462+0000 7f8066550640 1 -- 172.21.15.38:0/4018331936 shutdown_connections 2024-09-21T03:36:27.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:27.462+0000 7f8066550640 1 -- 172.21.15.38:0/4018331936 wait complete. 2024-09-21T03:36:27.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:27.463+0000 7f8066550640 1 Processor -- start 2024-09-21T03:36:27.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:27.464+0000 7f8066550640 1 -- start start 2024-09-21T03:36:27.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:27.465+0000 7f8066550640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f80600ce860 0x7f80601f3050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:27.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:27.465+0000 7f8066550640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f80600cf930 0x7f80600c4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:27.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:27.465+0000 7f8066550640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f806015b9f0 0x7f80600c53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:27.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:27.465+0000 7f8066550640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f806006ddf0 con 0x7f806015b9f0 2024-09-21T03:36:27.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:27.465+0000 7f8066550640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f806015fe60 con 0x7f80600ce860 2024-09-21T03:36:27.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:27.465+0000 7f8066550640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f806006df70 con 0x7f80600cf930 2024-09-21T03:36:27.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:27.465+0000 7f8064aed640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f806015b9f0 0x7f80600c53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:27.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:27.465+0000 7f8064aed640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f806015b9f0 0x7f80600c53e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37366/0 (socket says 172.21.15.38:37366) 2024-09-21T03:36:27.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:27.465+0000 7f8064aed640 1 -- 172.21.15.38:0/1929802651 learned_addr learned my addr 172.21.15.38:0/1929802651 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:36:27.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:27.465+0000 7f805f7fe640 1 --2- 172.21.15.38:0/1929802651 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f80600cf930 0x7f80600c4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:27.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:27.465+0000 7f805ffff640 1 --2- 172.21.15.38:0/1929802651 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f80600ce860 0x7f80601f3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:27.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:27.465+0000 7f8064aed640 1 -- 172.21.15.38:0/1929802651 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f80600cf930 msgr2=0x7f80600c4ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:27.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:27.465+0000 7f8064aed640 1 --2- 172.21.15.38:0/1929802651 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f80600cf930 0x7f80600c4ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:27.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:27.465+0000 7f8064aed640 1 -- 172.21.15.38:0/1929802651 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f80600ce860 msgr2=0x7f80601f3050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:27.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:27.465+0000 7f8064aed640 1 --2- 172.21.15.38:0/1929802651 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f80600ce860 0x7f80601f3050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:27.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:27.465+0000 7f8064aed640 1 -- 172.21.15.38:0/1929802651 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8060158a70 con 0x7f806015b9f0 2024-09-21T03:36:27.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:27.466+0000 7f8064aed640 1 --2- 172.21.15.38:0/1929802651 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f806015b9f0 0x7f80600c53e0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f805802aea0 tx=0x7f80580039f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:27.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:27.466+0000 7f805cff9640 1 -- 172.21.15.38:0/1929802651 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8058031990 con 0x7f806015b9f0 2024-09-21T03:36:27.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:27.466+0000 7f8066550640 1 -- 172.21.15.38:0/1929802651 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8060158d50 con 0x7f806015b9f0 2024-09-21T03:36:27.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:27.466+0000 7f805cff9640 1 -- 172.21.15.38:0/1929802651 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8058031c20 con 0x7f806015b9f0 2024-09-21T03:36:27.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:27.466+0000 7f805cff9640 1 -- 172.21.15.38:0/1929802651 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8058033330 con 0x7f806015b9f0 2024-09-21T03:36:27.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:27.466+0000 7f8066550640 1 -- 172.21.15.38:0/1929802651 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80600c7590 con 0x7f806015b9f0 2024-09-21T03:36:27.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:27.467+0000 7f8066550640 1 -- 172.21.15.38:0/1929802651 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80600c44d0 con 0x7f806015b9f0 2024-09-21T03:36:27.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:27.467+0000 7f805cff9640 1 -- 172.21.15.38:0/1929802651 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f80580334d0 con 0x7f806015b9f0 2024-09-21T03:36:27.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:27.468+0000 7f805cff9640 1 --2- 172.21.15.38:0/1929802651 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f804c078bc0 0x7f804c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:27.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:27.468+0000 7f805cff9640 1 -- 172.21.15.38:0/1929802651 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6101+0+0 (secure 0 0 0) 0x7f80580bc2e0 con 0x7f806015b9f0 2024-09-21T03:36:27.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:27.468+0000 7f805ffff640 1 --2- 172.21.15.38:0/1929802651 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f804c078bc0 0x7f804c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:27.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:27.469+0000 7f805ffff640 1 --2- 172.21.15.38:0/1929802651 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f804c078bc0 0x7f804c07b0d0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f8050001720 tx=0x7f8050007e90 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:27.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:27.473+0000 7f805cff9640 1 -- 172.21.15.38:0/1929802651 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f805803f030 con 0x7f806015b9f0 2024-09-21T03:36:27.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:27.654+0000 7f8066550640 1 -- 172.21.15.38:0/1929802651 --> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f80600c77f0 con 0x7f804c078bc0 2024-09-21T03:36:27.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:27.659+0000 7f805cff9640 1 -- 172.21.15.38:0/1929802651 <== mgr.4117 v2:172.21.15.195:6800/1631901344 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251506 (secure 0 0 0) 0x7f80600c77f0 con 0x7f804c078bc0 2024-09-21T03:36:27.663 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:36:27.664 INFO:teuthology.orchestra.run.smithi038.stderr:dumped all 2024-09-21T03:36:27.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:27.662+0000 7f8066550640 1 -- 172.21.15.38:0/1929802651 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f804c078bc0 msgr2=0x7f804c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:27.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:27.662+0000 7f8066550640 1 --2- 172.21.15.38:0/1929802651 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f804c078bc0 0x7f804c07b0d0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f8050001720 tx=0x7f8050007e90 comp rx=0 tx=0).stop 2024-09-21T03:36:27.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:27.662+0000 7f8066550640 1 -- 172.21.15.38:0/1929802651 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f806015b9f0 msgr2=0x7f80600c53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:27.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:27.662+0000 7f8066550640 1 --2- 172.21.15.38:0/1929802651 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f806015b9f0 0x7f80600c53e0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f805802aea0 tx=0x7f80580039f0 comp rx=0 tx=0).stop 2024-09-21T03:36:27.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:27.662+0000 7f8066550640 1 -- 172.21.15.38:0/1929802651 shutdown_connections 2024-09-21T03:36:27.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:27.662+0000 7f8066550640 1 --2- 172.21.15.38:0/1929802651 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f806015b9f0 0x7f80600c53e0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:27.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:27.662+0000 7f8066550640 1 --2- 172.21.15.38:0/1929802651 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f804c078bc0 0x7f804c07b0d0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:27.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:27.662+0000 7f8066550640 1 --2- 172.21.15.38:0/1929802651 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f80600cf930 0x7f80600c4ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:27.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:27.662+0000 7f8066550640 1 --2- 172.21.15.38:0/1929802651 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f80600ce860 0x7f80601f3050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:27.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:27.662+0000 7f8066550640 1 -- 172.21.15.38:0/1929802651 >> 172.21.15.38:0/1929802651 conn(0x7f80600cca40 msgr2=0x7f80600b0d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:27.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:27.662+0000 7f8066550640 1 -- 172.21.15.38:0/1929802651 shutdown_connections 2024-09-21T03:36:27.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:27.662+0000 7f8066550640 1 -- 172.21.15.38:0/1929802651 wait complete. 2024-09-21T03:36:27.675 INFO:teuthology.orchestra.run.smithi038.stdout:{"pg_ready":true,"pg_map":{"version":152,"stamp":"2024-09-21T03:36:27.023925+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591166,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":135,"num_write_kb":1639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":99,"ondisk_log_size":99,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":134,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":273588,"kb_used_data":7404,"kb_used_omap":12,"kb_used_meta":265971,"kb_avail":754701132,"statfs":{"total":773094113280,"available":772813959168,"internally_reserved":0,"allocated":7581696,"data_stored":4057781,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12748,"internal_metadata":272354868},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":798,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":34,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001367"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-21T03:36:22.088822+0000","last_change":"2024-09-21T03:36:22.088324+0000","last_active":"2024-09-21T03:36:22.088822+0000","last_peered":"2024-09-21T03:36:22.088822+0000","last_clean":"2024-09-21T03:36:22.088822+0000","last_became_active":"2024-09-21T03:36:22.088010+0000","last_became_peered":"2024-09-21T03:36:22.088010+0000","last_unstale":"2024-09-21T03:36:22.088822+0000","last_undegraded":"2024-09-21T03:36:22.088822+0000","last_fullsized":"2024-09-21T03:36:22.088822+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:17:56.797134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-21T03:36:21.054741+0000","last_change":"2024-09-21T03:36:21.054741+0000","last_active":"2024-09-21T03:36:21.036908+0000","last_peered":"2024-09-21T03:36:21.036908+0000","last_clean":"2024-09-21T03:36:21.036908+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:36:21.054741+0000","last_undegraded":"2024-09-21T03:36:21.054741+0000","last_fullsized":"2024-09-21T03:36:21.054741+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.086324+0000","last_change":"2024-09-21T03:36:22.071184+0000","last_active":"2024-09-21T03:36:23.086324+0000","last_peered":"2024-09-21T03:36:23.086324+0000","last_clean":"2024-09-21T03:36:23.086324+0000","last_became_active":"2024-09-21T03:36:22.070772+0000","last_became_peered":"2024-09-21T03:36:22.070772+0000","last_unstale":"2024-09-21T03:36:23.086324+0000","last_undegraded":"2024-09-21T03:36:23.086324+0000","last_fullsized":"2024-09-21T03:36:23.086324+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:17:47.479714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.661468+0000","last_change":"2024-09-21T03:36:22.070400+0000","last_active":"2024-09-21T03:36:23.661468+0000","last_peered":"2024-09-21T03:36:23.661468+0000","last_clean":"2024-09-21T03:36:23.661468+0000","last_became_active":"2024-09-21T03:36:22.068959+0000","last_became_peered":"2024-09-21T03:36:22.068959+0000","last_unstale":"2024-09-21T03:36:23.661468+0000","last_undegraded":"2024-09-21T03:36:23.661468+0000","last_fullsized":"2024-09-21T03:36:23.661468+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:30:41.436799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.086049+0000","last_change":"2024-09-21T03:36:22.081653+0000","last_active":"2024-09-21T03:36:23.086049+0000","last_peered":"2024-09-21T03:36:23.086049+0000","last_clean":"2024-09-21T03:36:23.086049+0000","last_became_active":"2024-09-21T03:36:22.081343+0000","last_became_peered":"2024-09-21T03:36:22.081343+0000","last_unstale":"2024-09-21T03:36:23.086049+0000","last_undegraded":"2024-09-21T03:36:23.086049+0000","last_fullsized":"2024-09-21T03:36:23.086049+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:37:46.632334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-21T03:36:21.055099+0000","last_change":"2024-09-21T03:36:21.055099+0000","last_active":"2024-09-21T03:36:21.036908+0000","last_peered":"2024-09-21T03:36:21.036908+0000","last_clean":"2024-09-21T03:36:21.036908+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:36:21.055099+0000","last_undegraded":"2024-09-21T03:36:21.055099+0000","last_fullsized":"2024-09-21T03:36:21.055099+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-21T03:36:22.075723+0000","last_change":"2024-09-21T03:36:22.074034+0000","last_active":"2024-09-21T03:36:22.075723+0000","last_peered":"2024-09-21T03:36:22.075723+0000","last_clean":"2024-09-21T03:36:22.075723+0000","last_became_active":"2024-09-21T03:36:22.073682+0000","last_became_peered":"2024-09-21T03:36:22.073682+0000","last_unstale":"2024-09-21T03:36:22.075723+0000","last_undegraded":"2024-09-21T03:36:22.075723+0000","last_fullsized":"2024-09-21T03:36:22.075723+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:25:30.522013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.085192+0000","last_change":"2024-09-21T03:36:22.080482+0000","last_active":"2024-09-21T03:36:23.085192+0000","last_peered":"2024-09-21T03:36:23.085192+0000","last_clean":"2024-09-21T03:36:23.085192+0000","last_became_active":"2024-09-21T03:36:22.080177+0000","last_became_peered":"2024-09-21T03:36:22.080177+0000","last_unstale":"2024-09-21T03:36:23.085192+0000","last_undegraded":"2024-09-21T03:36:23.085192+0000","last_fullsized":"2024-09-21T03:36:23.085192+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:56:46.927190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.661540+0000","last_change":"2024-09-21T03:36:22.077361+0000","last_active":"2024-09-21T03:36:23.661540+0000","last_peered":"2024-09-21T03:36:23.661540+0000","last_clean":"2024-09-21T03:36:23.661540+0000","last_became_active":"2024-09-21T03:36:22.077050+0000","last_became_peered":"2024-09-21T03:36:22.077050+0000","last_unstale":"2024-09-21T03:36:23.661540+0000","last_undegraded":"2024-09-21T03:36:23.661540+0000","last_fullsized":"2024-09-21T03:36:23.661540+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:30:24.348269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.085635+0000","last_change":"2024-09-21T03:36:22.077257+0000","last_active":"2024-09-21T03:36:23.085635+0000","last_peered":"2024-09-21T03:36:23.085635+0000","last_clean":"2024-09-21T03:36:23.085635+0000","last_became_active":"2024-09-21T03:36:22.076951+0000","last_became_peered":"2024-09-21T03:36:22.076951+0000","last_unstale":"2024-09-21T03:36:23.085635+0000","last_undegraded":"2024-09-21T03:36:23.085635+0000","last_fullsized":"2024-09-21T03:36:23.085635+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:23:59.409270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.661250+0000","last_change":"2024-09-21T03:36:22.072761+0000","last_active":"2024-09-21T03:36:23.661250+0000","last_peered":"2024-09-21T03:36:23.661250+0000","last_clean":"2024-09-21T03:36:23.661250+0000","last_became_active":"2024-09-21T03:36:22.072450+0000","last_became_peered":"2024-09-21T03:36:22.072450+0000","last_unstale":"2024-09-21T03:36:23.661250+0000","last_undegraded":"2024-09-21T03:36:23.661250+0000","last_fullsized":"2024-09-21T03:36:23.661250+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:04:45.989423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.661340+0000","last_change":"2024-09-21T03:36:22.086026+0000","last_active":"2024-09-21T03:36:23.661340+0000","last_peered":"2024-09-21T03:36:23.661340+0000","last_clean":"2024-09-21T03:36:23.661340+0000","last_became_active":"2024-09-21T03:36:22.085717+0000","last_became_peered":"2024-09-21T03:36:22.085717+0000","last_unstale":"2024-09-21T03:36:23.661340+0000","last_undegraded":"2024-09-21T03:36:23.661340+0000","last_fullsized":"2024-09-21T03:36:23.661340+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:47:51.103846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.660593+0000","last_change":"2024-09-21T03:36:22.090483+0000","last_active":"2024-09-21T03:36:23.660593+0000","last_peered":"2024-09-21T03:36:23.660593+0000","last_clean":"2024-09-21T03:36:23.660593+0000","last_became_active":"2024-09-21T03:36:22.090124+0000","last_became_peered":"2024-09-21T03:36:22.090124+0000","last_unstale":"2024-09-21T03:36:23.660593+0000","last_undegraded":"2024-09-21T03:36:23.660593+0000","last_fullsized":"2024-09-21T03:36:23.660593+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:52:19.716709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.660532+0000","last_change":"2024-09-21T03:36:22.080680+0000","last_active":"2024-09-21T03:36:23.660532+0000","last_peered":"2024-09-21T03:36:23.660532+0000","last_clean":"2024-09-21T03:36:23.660532+0000","last_became_active":"2024-09-21T03:36:22.080377+0000","last_became_peered":"2024-09-21T03:36:22.080377+0000","last_unstale":"2024-09-21T03:36:23.660532+0000","last_undegraded":"2024-09-21T03:36:23.660532+0000","last_fullsized":"2024-09-21T03:36:23.660532+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:54:33.394204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.085451+0000","last_change":"2024-09-21T03:36:22.073613+0000","last_active":"2024-09-21T03:36:23.085451+0000","last_peered":"2024-09-21T03:36:23.085451+0000","last_clean":"2024-09-21T03:36:23.085451+0000","last_became_active":"2024-09-21T03:36:22.073229+0000","last_became_peered":"2024-09-21T03:36:22.073229+0000","last_unstale":"2024-09-21T03:36:23.085451+0000","last_undegraded":"2024-09-21T03:36:23.085451+0000","last_fullsized":"2024-09-21T03:36:23.085451+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:37:15.857923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.660634+0000","last_change":"2024-09-21T03:36:22.085437+0000","last_active":"2024-09-21T03:36:23.660634+0000","last_peered":"2024-09-21T03:36:23.660634+0000","last_clean":"2024-09-21T03:36:23.660634+0000","last_became_active":"2024-09-21T03:36:22.085087+0000","last_became_peered":"2024-09-21T03:36:22.085087+0000","last_unstale":"2024-09-21T03:36:23.660634+0000","last_undegraded":"2024-09-21T03:36:23.660634+0000","last_fullsized":"2024-09-21T03:36:23.660634+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:30:00.925328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-21T03:36:22.082591+0000","last_change":"2024-09-21T03:36:22.081251+0000","last_active":"2024-09-21T03:36:22.082591+0000","last_peered":"2024-09-21T03:36:22.082591+0000","last_clean":"2024-09-21T03:36:22.082591+0000","last_became_active":"2024-09-21T03:36:22.080931+0000","last_became_peered":"2024-09-21T03:36:22.080931+0000","last_unstale":"2024-09-21T03:36:22.082591+0000","last_undegraded":"2024-09-21T03:36:22.082591+0000","last_fullsized":"2024-09-21T03:36:22.082591+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:04:34.873429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.661300+0000","last_change":"2024-09-21T03:36:22.069803+0000","last_active":"2024-09-21T03:36:23.661300+0000","last_peered":"2024-09-21T03:36:23.661300+0000","last_clean":"2024-09-21T03:36:23.661300+0000","last_became_active":"2024-09-21T03:36:22.069481+0000","last_became_peered":"2024-09-21T03:36:22.069481+0000","last_unstale":"2024-09-21T03:36:23.661300+0000","last_undegraded":"2024-09-21T03:36:23.661300+0000","last_fullsized":"2024-09-21T03:36:23.661300+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:15:44.483810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.661639+0000","last_change":"2024-09-21T03:36:22.076014+0000","last_active":"2024-09-21T03:36:23.661639+0000","last_peered":"2024-09-21T03:36:23.661639+0000","last_clean":"2024-09-21T03:36:23.661639+0000","last_became_active":"2024-09-21T03:36:22.075689+0000","last_became_peered":"2024-09-21T03:36:22.075689+0000","last_unstale":"2024-09-21T03:36:23.661639+0000","last_undegraded":"2024-09-21T03:36:23.661639+0000","last_fullsized":"2024-09-21T03:36:23.661639+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:47:25.173315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-21T03:36:22.073135+0000","last_change":"2024-09-21T03:36:22.071709+0000","last_active":"2024-09-21T03:36:22.073135+0000","last_peered":"2024-09-21T03:36:22.073135+0000","last_clean":"2024-09-21T03:36:22.073135+0000","last_became_active":"2024-09-21T03:36:22.070413+0000","last_became_peered":"2024-09-21T03:36:22.070413+0000","last_unstale":"2024-09-21T03:36:22.073135+0000","last_undegraded":"2024-09-21T03:36:22.073135+0000","last_fullsized":"2024-09-21T03:36:22.073135+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:16:55.679529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.660747+0000","last_change":"2024-09-21T03:36:22.086945+0000","last_active":"2024-09-21T03:36:23.660747+0000","last_peered":"2024-09-21T03:36:23.660747+0000","last_clean":"2024-09-21T03:36:23.660747+0000","last_became_active":"2024-09-21T03:36:22.086620+0000","last_became_peered":"2024-09-21T03:36:22.086620+0000","last_unstale":"2024-09-21T03:36:23.660747+0000","last_undegraded":"2024-09-21T03:36:23.660747+0000","last_fullsized":"2024-09-21T03:36:23.660747+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:48:46.104173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-21T03:36:21.053928+0000","last_change":"2024-09-21T03:36:21.053928+0000","last_active":"2024-09-21T03:36:21.036908+0000","last_peered":"2024-09-21T03:36:21.036908+0000","last_clean":"2024-09-21T03:36:21.036908+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:36:21.053928+0000","last_undegraded":"2024-09-21T03:36:21.053928+0000","last_fullsized":"2024-09-21T03:36:21.053928+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.660604+0000","last_change":"2024-09-21T03:36:22.086122+0000","last_active":"2024-09-21T03:36:23.660604+0000","last_peered":"2024-09-21T03:36:23.660604+0000","last_clean":"2024-09-21T03:36:23.660604+0000","last_became_active":"2024-09-21T03:36:22.085783+0000","last_became_peered":"2024-09-21T03:36:22.085783+0000","last_unstale":"2024-09-21T03:36:23.660604+0000","last_undegraded":"2024-09-21T03:36:23.660604+0000","last_fullsized":"2024-09-21T03:36:23.660604+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:05:24.777050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.085420+0000","last_change":"2024-09-21T03:36:22.067412+0000","last_active":"2024-09-21T03:36:23.085420+0000","last_peered":"2024-09-21T03:36:23.085420+0000","last_clean":"2024-09-21T03:36:23.085420+0000","last_became_active":"2024-09-21T03:36:22.067084+0000","last_became_peered":"2024-09-21T03:36:22.067084+0000","last_unstale":"2024-09-21T03:36:23.085420+0000","last_undegraded":"2024-09-21T03:36:23.085420+0000","last_fullsized":"2024-09-21T03:36:23.085420+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:01:59.040840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.661165+0000","last_change":"2024-09-21T03:36:22.082532+0000","last_active":"2024-09-21T03:36:23.661165+0000","last_peered":"2024-09-21T03:36:23.661165+0000","last_clean":"2024-09-21T03:36:23.661165+0000","last_became_active":"2024-09-21T03:36:22.082233+0000","last_became_peered":"2024-09-21T03:36:22.082233+0000","last_unstale":"2024-09-21T03:36:23.661165+0000","last_undegraded":"2024-09-21T03:36:23.661165+0000","last_fullsized":"2024-09-21T03:36:23.661165+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:09:57.485058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.086025+0000","last_change":"2024-09-21T03:36:22.068982+0000","last_active":"2024-09-21T03:36:23.086025+0000","last_peered":"2024-09-21T03:36:23.086025+0000","last_clean":"2024-09-21T03:36:23.086025+0000","last_became_active":"2024-09-21T03:36:22.068584+0000","last_became_peered":"2024-09-21T03:36:22.068584+0000","last_unstale":"2024-09-21T03:36:23.086025+0000","last_undegraded":"2024-09-21T03:36:23.086025+0000","last_fullsized":"2024-09-21T03:36:23.086025+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:02:01.275514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.660817+0000","last_change":"2024-09-21T03:36:22.085467+0000","last_active":"2024-09-21T03:36:23.660817+0000","last_peered":"2024-09-21T03:36:23.660817+0000","last_clean":"2024-09-21T03:36:23.660817+0000","last_became_active":"2024-09-21T03:36:22.085127+0000","last_became_peered":"2024-09-21T03:36:22.085127+0000","last_unstale":"2024-09-21T03:36:23.660817+0000","last_undegraded":"2024-09-21T03:36:23.660817+0000","last_fullsized":"2024-09-21T03:36:23.660817+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:04:32.859631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-21T03:36:22.084080+0000","last_change":"2024-09-21T03:36:22.083174+0000","last_active":"2024-09-21T03:36:22.084080+0000","last_peered":"2024-09-21T03:36:22.084080+0000","last_clean":"2024-09-21T03:36:22.084080+0000","last_became_active":"2024-09-21T03:36:22.082930+0000","last_became_peered":"2024-09-21T03:36:22.082930+0000","last_unstale":"2024-09-21T03:36:22.084080+0000","last_undegraded":"2024-09-21T03:36:22.084080+0000","last_fullsized":"2024-09-21T03:36:22.084080+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:09:50.811454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.661615+0000","last_change":"2024-09-21T03:36:22.088889+0000","last_active":"2024-09-21T03:36:23.661615+0000","last_peered":"2024-09-21T03:36:23.661615+0000","last_clean":"2024-09-21T03:36:23.661615+0000","last_became_active":"2024-09-21T03:36:22.088515+0000","last_became_peered":"2024-09-21T03:36:22.088515+0000","last_unstale":"2024-09-21T03:36:23.661615+0000","last_undegraded":"2024-09-21T03:36:23.661615+0000","last_fullsized":"2024-09-21T03:36:23.661615+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:00:06.850866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.660822+0000","last_change":"2024-09-21T03:36:22.088463+0000","last_active":"2024-09-21T03:36:23.660822+0000","last_peered":"2024-09-21T03:36:23.660822+0000","last_clean":"2024-09-21T03:36:23.660822+0000","last_became_active":"2024-09-21T03:36:22.088149+0000","last_became_peered":"2024-09-21T03:36:22.088149+0000","last_unstale":"2024-09-21T03:36:23.660822+0000","last_undegraded":"2024-09-21T03:36:23.660822+0000","last_fullsized":"2024-09-21T03:36:23.660822+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:03:04.392175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.085608+0000","last_change":"2024-09-21T03:36:22.066107+0000","last_active":"2024-09-21T03:36:23.085608+0000","last_peered":"2024-09-21T03:36:23.085608+0000","last_clean":"2024-09-21T03:36:23.085608+0000","last_became_active":"2024-09-21T03:36:22.064917+0000","last_became_peered":"2024-09-21T03:36:22.064917+0000","last_unstale":"2024-09-21T03:36:23.085608+0000","last_undegraded":"2024-09-21T03:36:23.085608+0000","last_fullsized":"2024-09-21T03:36:23.085608+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:58:06.692992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-21T03:36:22.072148+0000","last_change":"2024-09-21T03:36:22.069000+0000","last_active":"2024-09-21T03:36:22.072148+0000","last_peered":"2024-09-21T03:36:22.072148+0000","last_clean":"2024-09-21T03:36:22.072148+0000","last_became_active":"2024-09-21T03:36:22.068661+0000","last_became_peered":"2024-09-21T03:36:22.068661+0000","last_unstale":"2024-09-21T03:36:22.072148+0000","last_undegraded":"2024-09-21T03:36:22.072148+0000","last_fullsized":"2024-09-21T03:36:22.072148+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:39:01.505213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.661745+0000","last_change":"2024-09-21T03:36:22.071976+0000","last_active":"2024-09-21T03:36:23.661745+0000","last_peered":"2024-09-21T03:36:23.661745+0000","last_clean":"2024-09-21T03:36:23.661745+0000","last_became_active":"2024-09-21T03:36:22.071182+0000","last_became_peered":"2024-09-21T03:36:22.071182+0000","last_unstale":"2024-09-21T03:36:23.661745+0000","last_undegraded":"2024-09-21T03:36:23.661745+0000","last_fullsized":"2024-09-21T03:36:23.661745+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:58:44.343782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1f","version":"66'4","reported_seq":17,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-21T03:36:22.081942+0000","last_change":"2024-09-21T03:36:21.066587+0000","last_active":"2024-09-21T03:36:22.081942+0000","last_peered":"2024-09-21T03:36:22.081942+0000","last_clean":"2024-09-21T03:36:22.081942+0000","last_became_active":"2024-09-21T03:36:21.066299+0000","last_became_peered":"2024-09-21T03:36:21.066299+0000","last_unstale":"2024-09-21T03:36:22.081942+0000","last_undegraded":"2024-09-21T03:36:22.081942+0000","last_fullsized":"2024-09-21T03:36:22.081942+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_clean_scrub_stamp":"2024-09-21T03:36:20.027499+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:06:51.671380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.086591+0000","last_change":"2024-09-21T03:36:22.073629+0000","last_active":"2024-09-21T03:36:23.086591+0000","last_peered":"2024-09-21T03:36:23.086591+0000","last_clean":"2024-09-21T03:36:23.086591+0000","last_became_active":"2024-09-21T03:36:22.073211+0000","last_became_peered":"2024-09-21T03:36:22.073211+0000","last_unstale":"2024-09-21T03:36:23.086591+0000","last_undegraded":"2024-09-21T03:36:23.086591+0000","last_fullsized":"2024-09-21T03:36:23.086591+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:27:32.381253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1e","version":"66'1","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-21T03:36:22.092834+0000","last_change":"2024-09-21T03:36:21.064384+0000","last_active":"2024-09-21T03:36:22.092834+0000","last_peered":"2024-09-21T03:36:22.092834+0000","last_clean":"2024-09-21T03:36:22.092834+0000","last_became_active":"2024-09-21T03:36:21.064042+0000","last_became_peered":"2024-09-21T03:36:21.064042+0000","last_unstale":"2024-09-21T03:36:22.092834+0000","last_undegraded":"2024-09-21T03:36:22.092834+0000","last_fullsized":"2024-09-21T03:36:22.092834+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_clean_scrub_stamp":"2024-09-21T03:36:20.027499+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:17:25.045895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-21T03:36:21.054153+0000","last_change":"2024-09-21T03:36:21.054153+0000","last_active":"2024-09-21T03:36:21.036908+0000","last_peered":"2024-09-21T03:36:21.036908+0000","last_clean":"2024-09-21T03:36:21.036908+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:36:21.054153+0000","last_undegraded":"2024-09-21T03:36:21.054153+0000","last_fullsized":"2024-09-21T03:36:21.054153+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1d","version":"66'2","reported_seq":15,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-21T03:36:22.080793+0000","last_change":"2024-09-21T03:36:21.071133+0000","last_active":"2024-09-21T03:36:22.080793+0000","last_peered":"2024-09-21T03:36:22.080793+0000","last_clean":"2024-09-21T03:36:22.080793+0000","last_became_active":"2024-09-21T03:36:21.070834+0000","last_became_peered":"2024-09-21T03:36:21.070834+0000","last_unstale":"2024-09-21T03:36:22.080793+0000","last_undegraded":"2024-09-21T03:36:22.080793+0000","last_fullsized":"2024-09-21T03:36:22.080793+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_clean_scrub_stamp":"2024-09-21T03:36:20.027499+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:37:25.878136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.661630+0000","last_change":"2024-09-21T03:36:22.070080+0000","last_active":"2024-09-21T03:36:23.661630+0000","last_peered":"2024-09-21T03:36:23.661630+0000","last_clean":"2024-09-21T03:36:23.661630+0000","last_became_active":"2024-09-21T03:36:22.069779+0000","last_became_peered":"2024-09-21T03:36:22.069779+0000","last_unstale":"2024-09-21T03:36:23.661630+0000","last_undegraded":"2024-09-21T03:36:23.661630+0000","last_fullsized":"2024-09-21T03:36:23.661630+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:29:10.577562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-21T03:36:21.066934+0000","last_change":"2024-09-21T03:36:21.065338+0000","last_active":"2024-09-21T03:36:21.066934+0000","last_peered":"2024-09-21T03:36:21.066934+0000","last_clean":"2024-09-21T03:36:21.066934+0000","last_became_active":"2024-09-21T03:36:21.065032+0000","last_became_peered":"2024-09-21T03:36:21.065032+0000","last_unstale":"2024-09-21T03:36:21.066934+0000","last_undegraded":"2024-09-21T03:36:21.066934+0000","last_fullsized":"2024-09-21T03:36:21.066934+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_clean_scrub_stamp":"2024-09-21T03:36:20.027499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:38:09.536350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-21T03:36:22.065482+0000","last_change":"2024-09-21T03:36:22.063213+0000","last_active":"2024-09-21T03:36:22.065482+0000","last_peered":"2024-09-21T03:36:22.065482+0000","last_clean":"2024-09-21T03:36:22.065482+0000","last_became_active":"2024-09-21T03:36:22.062870+0000","last_became_peered":"2024-09-21T03:36:22.062870+0000","last_unstale":"2024-09-21T03:36:22.065482+0000","last_undegraded":"2024-09-21T03:36:22.065482+0000","last_fullsized":"2024-09-21T03:36:22.065482+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:07:32.687989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"66'1","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-21T03:36:22.079299+0000","last_change":"2024-09-21T03:36:21.060507+0000","last_active":"2024-09-21T03:36:22.079299+0000","last_peered":"2024-09-21T03:36:22.079299+0000","last_clean":"2024-09-21T03:36:22.079299+0000","last_became_active":"2024-09-21T03:36:21.060129+0000","last_became_peered":"2024-09-21T03:36:21.060129+0000","last_unstale":"2024-09-21T03:36:22.079299+0000","last_undegraded":"2024-09-21T03:36:22.079299+0000","last_fullsized":"2024-09-21T03:36:22.079299+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_clean_scrub_stamp":"2024-09-21T03:36:20.027499+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:57:03.979240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-21T03:36:22.080880+0000","last_change":"2024-09-21T03:36:22.077206+0000","last_active":"2024-09-21T03:36:22.080880+0000","last_peered":"2024-09-21T03:36:22.080880+0000","last_clean":"2024-09-21T03:36:22.080880+0000","last_became_active":"2024-09-21T03:36:22.076897+0000","last_became_peered":"2024-09-21T03:36:22.076897+0000","last_unstale":"2024-09-21T03:36:22.080880+0000","last_undegraded":"2024-09-21T03:36:22.080880+0000","last_fullsized":"2024-09-21T03:36:22.080880+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:39:58.170006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-21T03:36:21.062625+0000","last_change":"2024-09-21T03:36:21.059505+0000","last_active":"2024-09-21T03:36:21.062625+0000","last_peered":"2024-09-21T03:36:21.062625+0000","last_clean":"2024-09-21T03:36:21.062625+0000","last_became_active":"2024-09-21T03:36:21.059110+0000","last_became_peered":"2024-09-21T03:36:21.059110+0000","last_unstale":"2024-09-21T03:36:21.062625+0000","last_undegraded":"2024-09-21T03:36:21.062625+0000","last_fullsized":"2024-09-21T03:36:21.062625+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_clean_scrub_stamp":"2024-09-21T03:36:20.027499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:47:13.398068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.085728+0000","last_change":"2024-09-21T03:36:22.086098+0000","last_active":"2024-09-21T03:36:23.085728+0000","last_peered":"2024-09-21T03:36:23.085728+0000","last_clean":"2024-09-21T03:36:23.085728+0000","last_became_active":"2024-09-21T03:36:22.085410+0000","last_became_peered":"2024-09-21T03:36:22.085410+0000","last_unstale":"2024-09-21T03:36:23.085728+0000","last_undegraded":"2024-09-21T03:36:23.085728+0000","last_fullsized":"2024-09-21T03:36:23.085728+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:14:19.660267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":13,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-21T03:36:22.065557+0000","last_change":"2024-09-21T03:36:21.065608+0000","last_active":"2024-09-21T03:36:22.065557+0000","last_peered":"2024-09-21T03:36:22.065557+0000","last_clean":"2024-09-21T03:36:22.065557+0000","last_became_active":"2024-09-21T03:36:21.065292+0000","last_became_peered":"2024-09-21T03:36:21.065292+0000","last_unstale":"2024-09-21T03:36:22.065557+0000","last_undegraded":"2024-09-21T03:36:22.065557+0000","last_fullsized":"2024-09-21T03:36:22.065557+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_clean_scrub_stamp":"2024-09-21T03:36:20.027499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:36:50.067003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-21T03:36:22.072312+0000","last_change":"2024-09-21T03:36:22.069640+0000","last_active":"2024-09-21T03:36:22.072312+0000","last_peered":"2024-09-21T03:36:22.072312+0000","last_clean":"2024-09-21T03:36:22.072312+0000","last_became_active":"2024-09-21T03:36:22.068836+0000","last_became_peered":"2024-09-21T03:36:22.068836+0000","last_unstale":"2024-09-21T03:36:22.072312+0000","last_undegraded":"2024-09-21T03:36:22.072312+0000","last_fullsized":"2024-09-21T03:36:22.072312+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:29:10.577562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.660609+0000","last_change":"2024-09-21T03:36:21.062628+0000","last_active":"2024-09-21T03:36:23.660609+0000","last_peered":"2024-09-21T03:36:23.660609+0000","last_clean":"2024-09-21T03:36:23.660609+0000","last_became_active":"2024-09-21T03:36:21.062295+0000","last_became_peered":"2024-09-21T03:36:21.062295+0000","last_unstale":"2024-09-21T03:36:23.660609+0000","last_undegraded":"2024-09-21T03:36:23.660609+0000","last_fullsized":"2024-09-21T03:36:23.660609+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_clean_scrub_stamp":"2024-09-21T03:36:20.027499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:20:15.258373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-21T03:36:22.069673+0000","last_change":"2024-09-21T03:36:22.067826+0000","last_active":"2024-09-21T03:36:22.069673+0000","last_peered":"2024-09-21T03:36:22.069673+0000","last_clean":"2024-09-21T03:36:22.069673+0000","last_became_active":"2024-09-21T03:36:22.067516+0000","last_became_peered":"2024-09-21T03:36:22.067516+0000","last_unstale":"2024-09-21T03:36:22.069673+0000","last_undegraded":"2024-09-21T03:36:22.069673+0000","last_fullsized":"2024-09-21T03:36:22.069673+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:20:16.267783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.661694+0000","last_change":"2024-09-21T03:36:21.063524+0000","last_active":"2024-09-21T03:36:23.661694+0000","last_peered":"2024-09-21T03:36:23.661694+0000","last_clean":"2024-09-21T03:36:23.661694+0000","last_became_active":"2024-09-21T03:36:21.063203+0000","last_became_peered":"2024-09-21T03:36:21.063203+0000","last_unstale":"2024-09-21T03:36:23.661694+0000","last_undegraded":"2024-09-21T03:36:23.661694+0000","last_fullsized":"2024-09-21T03:36:23.661694+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_clean_scrub_stamp":"2024-09-21T03:36:20.027499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:25:29.512603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-21T03:36:22.065852+0000","last_change":"2024-09-21T03:36:22.063211+0000","last_active":"2024-09-21T03:36:22.065852+0000","last_peered":"2024-09-21T03:36:22.065852+0000","last_clean":"2024-09-21T03:36:22.065852+0000","last_became_active":"2024-09-21T03:36:22.062828+0000","last_became_peered":"2024-09-21T03:36:22.062828+0000","last_unstale":"2024-09-21T03:36:22.065852+0000","last_undegraded":"2024-09-21T03:36:22.065852+0000","last_fullsized":"2024-09-21T03:36:22.065852+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:30:41.436799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.661311+0000","last_change":"2024-09-21T03:36:21.064314+0000","last_active":"2024-09-21T03:36:23.661311+0000","last_peered":"2024-09-21T03:36:23.661311+0000","last_clean":"2024-09-21T03:36:23.661311+0000","last_became_active":"2024-09-21T03:36:21.063271+0000","last_became_peered":"2024-09-21T03:36:21.063271+0000","last_unstale":"2024-09-21T03:36:23.661311+0000","last_undegraded":"2024-09-21T03:36:23.661311+0000","last_fullsized":"2024-09-21T03:36:23.661311+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_clean_scrub_stamp":"2024-09-21T03:36:20.027499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:03:09.222194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.660928+0000","last_change":"2024-09-21T03:36:22.092139+0000","last_active":"2024-09-21T03:36:23.660928+0000","last_peered":"2024-09-21T03:36:23.660928+0000","last_clean":"2024-09-21T03:36:23.660928+0000","last_became_active":"2024-09-21T03:36:22.091718+0000","last_became_peered":"2024-09-21T03:36:22.091718+0000","last_unstale":"2024-09-21T03:36:23.660928+0000","last_undegraded":"2024-09-21T03:36:23.660928+0000","last_fullsized":"2024-09-21T03:36:23.660928+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:28:40.036155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.15","version":"66'3","reported_seq":18,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.086559+0000","last_change":"2024-09-21T03:36:21.064387+0000","last_active":"2024-09-21T03:36:23.086559+0000","last_peered":"2024-09-21T03:36:23.086559+0000","last_clean":"2024-09-21T03:36:23.086559+0000","last_became_active":"2024-09-21T03:36:21.064086+0000","last_became_peered":"2024-09-21T03:36:21.064086+0000","last_unstale":"2024-09-21T03:36:23.086559+0000","last_undegraded":"2024-09-21T03:36:23.086559+0000","last_fullsized":"2024-09-21T03:36:23.086559+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_clean_scrub_stamp":"2024-09-21T03:36:20.027499+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:01:58.431275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.086256+0000","last_change":"2024-09-21T03:36:22.070639+0000","last_active":"2024-09-21T03:36:23.086256+0000","last_peered":"2024-09-21T03:36:23.086256+0000","last_clean":"2024-09-21T03:36:23.086256+0000","last_became_active":"2024-09-21T03:36:22.069620+0000","last_became_peered":"2024-09-21T03:36:22.069620+0000","last_unstale":"2024-09-21T03:36:23.086256+0000","last_undegraded":"2024-09-21T03:36:23.086256+0000","last_fullsized":"2024-09-21T03:36:23.086256+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:23:36.147744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-21T03:36:21.066007+0000","last_change":"2024-09-21T03:36:21.065091+0000","last_active":"2024-09-21T03:36:21.066007+0000","last_peered":"2024-09-21T03:36:21.066007+0000","last_clean":"2024-09-21T03:36:21.066007+0000","last_became_active":"2024-09-21T03:36:21.064760+0000","last_became_peered":"2024-09-21T03:36:21.064760+0000","last_unstale":"2024-09-21T03:36:21.066007+0000","last_undegraded":"2024-09-21T03:36:21.066007+0000","last_fullsized":"2024-09-21T03:36:21.066007+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_clean_scrub_stamp":"2024-09-21T03:36:20.027499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:17:34.450050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.085112+0000","last_change":"2024-09-21T03:36:22.085864+0000","last_active":"2024-09-21T03:36:23.085112+0000","last_peered":"2024-09-21T03:36:23.085112+0000","last_clean":"2024-09-21T03:36:23.085112+0000","last_became_active":"2024-09-21T03:36:22.085557+0000","last_became_peered":"2024-09-21T03:36:22.085557+0000","last_unstale":"2024-09-21T03:36:23.085112+0000","last_undegraded":"2024-09-21T03:36:23.085112+0000","last_fullsized":"2024-09-21T03:36:23.085112+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:25:31.677998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.13","version":"66'2","reported_seq":17,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.661866+0000","last_change":"2024-09-21T03:36:21.064320+0000","last_active":"2024-09-21T03:36:23.661866+0000","last_peered":"2024-09-21T03:36:23.661866+0000","last_clean":"2024-09-21T03:36:23.661866+0000","last_became_active":"2024-09-21T03:36:21.064030+0000","last_became_peered":"2024-09-21T03:36:21.064030+0000","last_unstale":"2024-09-21T03:36:23.661866+0000","last_undegraded":"2024-09-21T03:36:23.661866+0000","last_fullsized":"2024-09-21T03:36:23.661866+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_clean_scrub_stamp":"2024-09-21T03:36:20.027499+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:36:50.067003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.085860+0000","last_change":"2024-09-21T03:36:22.072437+0000","last_active":"2024-09-21T03:36:23.085860+0000","last_peered":"2024-09-21T03:36:23.085860+0000","last_clean":"2024-09-21T03:36:23.085860+0000","last_became_active":"2024-09-21T03:36:22.072130+0000","last_became_peered":"2024-09-21T03:36:22.072130+0000","last_unstale":"2024-09-21T03:36:23.085860+0000","last_undegraded":"2024-09-21T03:36:23.085860+0000","last_fullsized":"2024-09-21T03:36:23.085860+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:32:25.157463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.085572+0000","last_change":"2024-09-21T03:36:21.067895+0000","last_active":"2024-09-21T03:36:23.085572+0000","last_peered":"2024-09-21T03:36:23.085572+0000","last_clean":"2024-09-21T03:36:23.085572+0000","last_became_active":"2024-09-21T03:36:21.067573+0000","last_became_peered":"2024-09-21T03:36:21.067573+0000","last_unstale":"2024-09-21T03:36:23.085572+0000","last_undegraded":"2024-09-21T03:36:23.085572+0000","last_fullsized":"2024-09-21T03:36:23.085572+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_clean_scrub_stamp":"2024-09-21T03:36:20.027499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:25:13.103147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.085382+0000","last_change":"2024-09-21T03:36:22.086475+0000","last_active":"2024-09-21T03:36:23.085382+0000","last_peered":"2024-09-21T03:36:23.085382+0000","last_clean":"2024-09-21T03:36:23.085382+0000","last_became_active":"2024-09-21T03:36:22.085797+0000","last_became_peered":"2024-09-21T03:36:22.085797+0000","last_unstale":"2024-09-21T03:36:23.085382+0000","last_undegraded":"2024-09-21T03:36:23.085382+0000","last_fullsized":"2024-09-21T03:36:23.085382+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:18:08.028552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.085748+0000","last_change":"2024-09-21T03:36:21.061510+0000","last_active":"2024-09-21T03:36:23.085748+0000","last_peered":"2024-09-21T03:36:23.085748+0000","last_clean":"2024-09-21T03:36:23.085748+0000","last_became_active":"2024-09-21T03:36:21.061181+0000","last_became_peered":"2024-09-21T03:36:21.061181+0000","last_unstale":"2024-09-21T03:36:23.085748+0000","last_undegraded":"2024-09-21T03:36:23.085748+0000","last_fullsized":"2024-09-21T03:36:23.085748+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_clean_scrub_stamp":"2024-09-21T03:36:20.027499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:39:48.217217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-21T03:36:22.086623+0000","last_change":"2024-09-21T03:36:22.085750+0000","last_active":"2024-09-21T03:36:22.086623+0000","last_peered":"2024-09-21T03:36:22.086623+0000","last_clean":"2024-09-21T03:36:22.086623+0000","last_became_active":"2024-09-21T03:36:22.085426+0000","last_became_peered":"2024-09-21T03:36:22.085426+0000","last_unstale":"2024-09-21T03:36:22.086623+0000","last_undegraded":"2024-09-21T03:36:22.086623+0000","last_fullsized":"2024-09-21T03:36:22.086623+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:43:37.785209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"66'8","reported_seq":23,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.084997+0000","last_change":"2024-09-21T03:36:21.063882+0000","last_active":"2024-09-21T03:36:23.084997+0000","last_peered":"2024-09-21T03:36:23.084997+0000","last_clean":"2024-09-21T03:36:23.084997+0000","last_became_active":"2024-09-21T03:36:21.063544+0000","last_became_peered":"2024-09-21T03:36:21.063544+0000","last_unstale":"2024-09-21T03:36:23.084997+0000","last_undegraded":"2024-09-21T03:36:23.084997+0000","last_fullsized":"2024-09-21T03:36:23.084997+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_clean_scrub_stamp":"2024-09-21T03:36:20.027499+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:38:51.296443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.661909+0000","last_change":"2024-09-21T03:36:22.086802+0000","last_active":"2024-09-21T03:36:23.661909+0000","last_peered":"2024-09-21T03:36:23.661909+0000","last_clean":"2024-09-21T03:36:23.661909+0000","last_became_active":"2024-09-21T03:36:22.086435+0000","last_became_peered":"2024-09-21T03:36:22.086435+0000","last_unstale":"2024-09-21T03:36:23.661909+0000","last_undegraded":"2024-09-21T03:36:23.661909+0000","last_fullsized":"2024-09-21T03:36:23.661909+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:32:26.377539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.f","version":"66'1","reported_seq":16,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.661756+0000","last_change":"2024-09-21T03:36:21.062807+0000","last_active":"2024-09-21T03:36:23.661756+0000","last_peered":"2024-09-21T03:36:23.661756+0000","last_clean":"2024-09-21T03:36:23.661756+0000","last_became_active":"2024-09-21T03:36:21.062513+0000","last_became_peered":"2024-09-21T03:36:21.062513+0000","last_unstale":"2024-09-21T03:36:23.661756+0000","last_undegraded":"2024-09-21T03:36:23.661756+0000","last_fullsized":"2024-09-21T03:36:23.661756+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_clean_scrub_stamp":"2024-09-21T03:36:20.027499+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:39:57.160597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.660925+0000","last_change":"2024-09-21T03:36:22.086393+0000","last_active":"2024-09-21T03:36:23.660925+0000","last_peered":"2024-09-21T03:36:23.660925+0000","last_clean":"2024-09-21T03:36:23.660925+0000","last_became_active":"2024-09-21T03:36:22.086094+0000","last_became_peered":"2024-09-21T03:36:22.086094+0000","last_unstale":"2024-09-21T03:36:23.660925+0000","last_undegraded":"2024-09-21T03:36:23.660925+0000","last_fullsized":"2024-09-21T03:36:23.660925+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:11:34.432933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.e","version":"66'2","reported_seq":17,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.661138+0000","last_change":"2024-09-21T03:36:21.053740+0000","last_active":"2024-09-21T03:36:23.661138+0000","last_peered":"2024-09-21T03:36:23.661138+0000","last_clean":"2024-09-21T03:36:23.661138+0000","last_became_active":"2024-09-21T03:36:21.053383+0000","last_became_peered":"2024-09-21T03:36:21.053383+0000","last_unstale":"2024-09-21T03:36:23.661138+0000","last_undegraded":"2024-09-21T03:36:23.661138+0000","last_fullsized":"2024-09-21T03:36:23.661138+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_clean_scrub_stamp":"2024-09-21T03:36:20.027499+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:37:25.878136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.661181+0000","last_change":"2024-09-21T03:36:22.093865+0000","last_active":"2024-09-21T03:36:23.661181+0000","last_peered":"2024-09-21T03:36:23.661181+0000","last_clean":"2024-09-21T03:36:23.661181+0000","last_became_active":"2024-09-21T03:36:22.093526+0000","last_became_peered":"2024-09-21T03:36:22.093526+0000","last_unstale":"2024-09-21T03:36:23.661181+0000","last_undegraded":"2024-09-21T03:36:23.661181+0000","last_fullsized":"2024-09-21T03:36:23.661181+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:59:12.697168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.d","version":"66'1","reported_seq":16,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.086666+0000","last_change":"2024-09-21T03:36:21.063656+0000","last_active":"2024-09-21T03:36:23.086666+0000","last_peered":"2024-09-21T03:36:23.086666+0000","last_clean":"2024-09-21T03:36:23.086666+0000","last_became_active":"2024-09-21T03:36:21.063355+0000","last_became_peered":"2024-09-21T03:36:21.063355+0000","last_unstale":"2024-09-21T03:36:23.086666+0000","last_undegraded":"2024-09-21T03:36:23.086666+0000","last_fullsized":"2024-09-21T03:36:23.086666+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_clean_scrub_stamp":"2024-09-21T03:36:20.027499+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:05:53.691084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.086342+0000","last_change":"2024-09-21T03:36:22.070297+0000","last_active":"2024-09-21T03:36:23.086342+0000","last_peered":"2024-09-21T03:36:23.086342+0000","last_clean":"2024-09-21T03:36:23.086342+0000","last_became_active":"2024-09-21T03:36:22.069258+0000","last_became_peered":"2024-09-21T03:36:22.069258+0000","last_unstale":"2024-09-21T03:36:23.086342+0000","last_undegraded":"2024-09-21T03:36:23.086342+0000","last_fullsized":"2024-09-21T03:36:23.086342+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:51:57.465792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.660607+0000","last_change":"2024-09-21T03:36:21.067658+0000","last_active":"2024-09-21T03:36:23.660607+0000","last_peered":"2024-09-21T03:36:23.660607+0000","last_clean":"2024-09-21T03:36:23.660607+0000","last_became_active":"2024-09-21T03:36:21.067359+0000","last_became_peered":"2024-09-21T03:36:21.067359+0000","last_unstale":"2024-09-21T03:36:23.660607+0000","last_undegraded":"2024-09-21T03:36:23.660607+0000","last_fullsized":"2024-09-21T03:36:23.660607+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_clean_scrub_stamp":"2024-09-21T03:36:20.027499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:38:26.468485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-21T03:36:21.054221+0000","last_change":"2024-09-21T03:36:21.054221+0000","last_active":"2024-09-21T03:36:21.036908+0000","last_peered":"2024-09-21T03:36:21.036908+0000","last_clean":"2024-09-21T03:36:21.036908+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:36:21.054221+0000","last_undegraded":"2024-09-21T03:36:21.054221+0000","last_fullsized":"2024-09-21T03:36:21.054221+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-21T03:36:21.068841+0000","last_change":"2024-09-21T03:36:21.067047+0000","last_active":"2024-09-21T03:36:21.068841+0000","last_peered":"2024-09-21T03:36:21.068841+0000","last_clean":"2024-09-21T03:36:21.068841+0000","last_became_active":"2024-09-21T03:36:21.066718+0000","last_became_peered":"2024-09-21T03:36:21.066718+0000","last_unstale":"2024-09-21T03:36:21.068841+0000","last_undegraded":"2024-09-21T03:36:21.068841+0000","last_fullsized":"2024-09-21T03:36:21.068841+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_clean_scrub_stamp":"2024-09-21T03:36:20.027499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:34:30.827273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.660351+0000","last_change":"2024-09-21T03:36:22.081020+0000","last_active":"2024-09-21T03:36:23.660351+0000","last_peered":"2024-09-21T03:36:23.660351+0000","last_clean":"2024-09-21T03:36:23.660351+0000","last_became_active":"2024-09-21T03:36:22.080090+0000","last_became_peered":"2024-09-21T03:36:22.080090+0000","last_unstale":"2024-09-21T03:36:23.660351+0000","last_undegraded":"2024-09-21T03:36:23.660351+0000","last_fullsized":"2024-09-21T03:36:23.660351+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:05:02.647190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.085682+0000","last_change":"2024-09-21T03:36:21.065401+0000","last_active":"2024-09-21T03:36:23.085682+0000","last_peered":"2024-09-21T03:36:23.085682+0000","last_clean":"2024-09-21T03:36:23.085682+0000","last_became_active":"2024-09-21T03:36:21.065075+0000","last_became_peered":"2024-09-21T03:36:21.065075+0000","last_unstale":"2024-09-21T03:36:23.085682+0000","last_undegraded":"2024-09-21T03:36:23.085682+0000","last_fullsized":"2024-09-21T03:36:23.085682+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_clean_scrub_stamp":"2024-09-21T03:36:20.027499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:06:26.032047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.660983+0000","last_change":"2024-09-21T03:36:22.071663+0000","last_active":"2024-09-21T03:36:23.660983+0000","last_peered":"2024-09-21T03:36:23.660983+0000","last_clean":"2024-09-21T03:36:23.660983+0000","last_became_active":"2024-09-21T03:36:22.071326+0000","last_became_peered":"2024-09-21T03:36:22.071326+0000","last_unstale":"2024-09-21T03:36:23.660983+0000","last_undegraded":"2024-09-21T03:36:23.660983+0000","last_fullsized":"2024-09-21T03:36:23.660983+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:33:14.339525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-21T03:36:21.064334+0000","last_change":"2024-09-21T03:36:21.062867+0000","last_active":"2024-09-21T03:36:21.064334+0000","last_peered":"2024-09-21T03:36:21.064334+0000","last_clean":"2024-09-21T03:36:21.064334+0000","last_became_active":"2024-09-21T03:36:21.062555+0000","last_became_peered":"2024-09-21T03:36:21.062555+0000","last_unstale":"2024-09-21T03:36:21.064334+0000","last_undegraded":"2024-09-21T03:36:21.064334+0000","last_fullsized":"2024-09-21T03:36:21.064334+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_clean_scrub_stamp":"2024-09-21T03:36:20.027499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:24:33.618928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-21T03:36:22.066694+0000","last_change":"2024-09-21T03:36:22.063923+0000","last_active":"2024-09-21T03:36:22.066694+0000","last_peered":"2024-09-21T03:36:22.066694+0000","last_clean":"2024-09-21T03:36:22.066694+0000","last_became_active":"2024-09-21T03:36:22.063195+0000","last_became_peered":"2024-09-21T03:36:22.063195+0000","last_unstale":"2024-09-21T03:36:22.066694+0000","last_undegraded":"2024-09-21T03:36:22.066694+0000","last_fullsized":"2024-09-21T03:36:22.066694+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:48:30.795859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.7","version":"66'2","reported_seq":17,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.661858+0000","last_change":"2024-09-21T03:36:21.068456+0000","last_active":"2024-09-21T03:36:23.661858+0000","last_peered":"2024-09-21T03:36:23.661858+0000","last_clean":"2024-09-21T03:36:23.661858+0000","last_became_active":"2024-09-21T03:36:21.068101+0000","last_became_peered":"2024-09-21T03:36:21.068101+0000","last_unstale":"2024-09-21T03:36:23.661858+0000","last_undegraded":"2024-09-21T03:36:23.661858+0000","last_fullsized":"2024-09-21T03:36:23.661858+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_clean_scrub_stamp":"2024-09-21T03:36:20.027499+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:09:49.802044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-21T03:36:22.073022+0000","last_change":"2024-09-21T03:36:22.071078+0000","last_active":"2024-09-21T03:36:22.073022+0000","last_peered":"2024-09-21T03:36:22.073022+0000","last_clean":"2024-09-21T03:36:22.073022+0000","last_became_active":"2024-09-21T03:36:22.070566+0000","last_became_peered":"2024-09-21T03:36:22.070566+0000","last_unstale":"2024-09-21T03:36:22.073022+0000","last_undegraded":"2024-09-21T03:36:22.073022+0000","last_fullsized":"2024-09-21T03:36:22.073022+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:09:22.850109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.661366+0000","last_change":"2024-09-21T03:36:21.059377+0000","last_active":"2024-09-21T03:36:23.661366+0000","last_peered":"2024-09-21T03:36:23.661366+0000","last_clean":"2024-09-21T03:36:23.661366+0000","last_became_active":"2024-09-21T03:36:21.059079+0000","last_became_peered":"2024-09-21T03:36:21.059079+0000","last_unstale":"2024-09-21T03:36:23.661366+0000","last_undegraded":"2024-09-21T03:36:23.661366+0000","last_fullsized":"2024-09-21T03:36:23.661366+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_clean_scrub_stamp":"2024-09-21T03:36:20.027499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:39:00.495803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.086122+0000","last_change":"2024-09-21T03:36:22.072209+0000","last_active":"2024-09-21T03:36:23.086122+0000","last_peered":"2024-09-21T03:36:23.086122+0000","last_clean":"2024-09-21T03:36:23.086122+0000","last_became_active":"2024-09-21T03:36:22.071876+0000","last_became_peered":"2024-09-21T03:36:22.071876+0000","last_unstale":"2024-09-21T03:36:23.086122+0000","last_undegraded":"2024-09-21T03:36:23.086122+0000","last_fullsized":"2024-09-21T03:36:23.086122+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:37:07.049180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-21T03:36:21.064294+0000","last_change":"2024-09-21T03:36:21.063184+0000","last_active":"2024-09-21T03:36:21.064294+0000","last_peered":"2024-09-21T03:36:21.064294+0000","last_clean":"2024-09-21T03:36:21.064294+0000","last_became_active":"2024-09-21T03:36:21.062859+0000","last_became_peered":"2024-09-21T03:36:21.062859+0000","last_unstale":"2024-09-21T03:36:21.064294+0000","last_undegraded":"2024-09-21T03:36:21.064294+0000","last_fullsized":"2024-09-21T03:36:21.064294+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_clean_scrub_stamp":"2024-09-21T03:36:20.027499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:18:15.549734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.086389+0000","last_change":"2024-09-21T03:36:22.069954+0000","last_active":"2024-09-21T03:36:23.086389+0000","last_peered":"2024-09-21T03:36:23.086389+0000","last_clean":"2024-09-21T03:36:23.086389+0000","last_became_active":"2024-09-21T03:36:22.068916+0000","last_became_peered":"2024-09-21T03:36:22.068916+0000","last_unstale":"2024-09-21T03:36:23.086389+0000","last_undegraded":"2024-09-21T03:36:23.086389+0000","last_fullsized":"2024-09-21T03:36:23.086389+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:03:46.925197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.660652+0000","last_change":"2024-09-21T03:36:21.071978+0000","last_active":"2024-09-21T03:36:23.660652+0000","last_peered":"2024-09-21T03:36:23.660652+0000","last_clean":"2024-09-21T03:36:23.660652+0000","last_became_active":"2024-09-21T03:36:21.071717+0000","last_became_peered":"2024-09-21T03:36:21.071717+0000","last_unstale":"2024-09-21T03:36:23.660652+0000","last_undegraded":"2024-09-21T03:36:23.660652+0000","last_fullsized":"2024-09-21T03:36:23.660652+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_clean_scrub_stamp":"2024-09-21T03:36:20.027499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:46:05.982882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.661216+0000","last_change":"2024-09-21T03:36:22.072086+0000","last_active":"2024-09-21T03:36:23.661216+0000","last_peered":"2024-09-21T03:36:23.661216+0000","last_clean":"2024-09-21T03:36:23.661216+0000","last_became_active":"2024-09-21T03:36:22.071729+0000","last_became_peered":"2024-09-21T03:36:22.071729+0000","last_unstale":"2024-09-21T03:36:23.661216+0000","last_undegraded":"2024-09-21T03:36:23.661216+0000","last_fullsized":"2024-09-21T03:36:23.661216+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:30:14.015775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.660720+0000","last_change":"2024-09-21T03:36:21.066336+0000","last_active":"2024-09-21T03:36:23.660720+0000","last_peered":"2024-09-21T03:36:23.660720+0000","last_clean":"2024-09-21T03:36:23.660720+0000","last_became_active":"2024-09-21T03:36:21.066043+0000","last_became_peered":"2024-09-21T03:36:21.066043+0000","last_unstale":"2024-09-21T03:36:23.660720+0000","last_undegraded":"2024-09-21T03:36:23.660720+0000","last_fullsized":"2024-09-21T03:36:23.660720+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_clean_scrub_stamp":"2024-09-21T03:36:20.027499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:45:07.961665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.660391+0000","last_change":"2024-09-21T03:36:22.089730+0000","last_active":"2024-09-21T03:36:23.660391+0000","last_peered":"2024-09-21T03:36:23.660391+0000","last_clean":"2024-09-21T03:36:23.660391+0000","last_became_active":"2024-09-21T03:36:22.089403+0000","last_became_peered":"2024-09-21T03:36:22.089403+0000","last_unstale":"2024-09-21T03:36:23.660391+0000","last_undegraded":"2024-09-21T03:36:23.660391+0000","last_fullsized":"2024-09-21T03:36:23.660391+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:38:24.463402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":13,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-21T03:36:22.061045+0000","last_change":"2024-09-21T03:36:21.066672+0000","last_active":"2024-09-21T03:36:22.061045+0000","last_peered":"2024-09-21T03:36:22.061045+0000","last_clean":"2024-09-21T03:36:22.061045+0000","last_became_active":"2024-09-21T03:36:21.066366+0000","last_became_peered":"2024-09-21T03:36:21.066366+0000","last_unstale":"2024-09-21T03:36:22.061045+0000","last_undegraded":"2024-09-21T03:36:22.061045+0000","last_fullsized":"2024-09-21T03:36:22.061045+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_clean_scrub_stamp":"2024-09-21T03:36:20.027499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:28:15.884066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-21T03:36:21.054594+0000","last_change":"2024-09-21T03:36:21.054594+0000","last_active":"2024-09-21T03:36:21.036908+0000","last_peered":"2024-09-21T03:36:21.036908+0000","last_clean":"2024-09-21T03:36:21.036908+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:36:21.054594+0000","last_undegraded":"2024-09-21T03:36:21.054594+0000","last_fullsized":"2024-09-21T03:36:21.054594+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.660737+0000","last_change":"2024-09-21T03:36:21.065843+0000","last_active":"2024-09-21T03:36:23.660737+0000","last_peered":"2024-09-21T03:36:23.660737+0000","last_clean":"2024-09-21T03:36:23.660737+0000","last_became_active":"2024-09-21T03:36:21.065466+0000","last_became_peered":"2024-09-21T03:36:21.065466+0000","last_unstale":"2024-09-21T03:36:23.660737+0000","last_undegraded":"2024-09-21T03:36:23.660737+0000","last_fullsized":"2024-09-21T03:36:23.660737+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_clean_scrub_stamp":"2024-09-21T03:36:20.027499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:10:41.577481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-21T03:36:22.072278+0000","last_change":"2024-09-21T03:36:22.068997+0000","last_active":"2024-09-21T03:36:22.072278+0000","last_peered":"2024-09-21T03:36:22.072278+0000","last_clean":"2024-09-21T03:36:22.072278+0000","last_became_active":"2024-09-21T03:36:22.067174+0000","last_became_peered":"2024-09-21T03:36:22.067174+0000","last_unstale":"2024-09-21T03:36:22.072278+0000","last_undegraded":"2024-09-21T03:36:22.072278+0000","last_fullsized":"2024-09-21T03:36:22.072278+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:42:28.131400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-21T03:36:21.069684+0000","last_change":"2024-09-21T03:36:21.067567+0000","last_active":"2024-09-21T03:36:21.069684+0000","last_peered":"2024-09-21T03:36:21.069684+0000","last_clean":"2024-09-21T03:36:21.069684+0000","last_became_active":"2024-09-21T03:36:21.067235+0000","last_became_peered":"2024-09-21T03:36:21.067235+0000","last_unstale":"2024-09-21T03:36:21.069684+0000","last_undegraded":"2024-09-21T03:36:21.069684+0000","last_fullsized":"2024-09-21T03:36:21.069684+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_clean_scrub_stamp":"2024-09-21T03:36:20.027499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:24:32.368652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.085908+0000","last_change":"2024-09-21T03:36:22.083766+0000","last_active":"2024-09-21T03:36:23.085908+0000","last_peered":"2024-09-21T03:36:23.085908+0000","last_clean":"2024-09-21T03:36:23.085908+0000","last_became_active":"2024-09-21T03:36:22.083362+0000","last_became_peered":"2024-09-21T03:36:22.083362+0000","last_unstale":"2024-09-21T03:36:23.085908+0000","last_undegraded":"2024-09-21T03:36:23.085908+0000","last_fullsized":"2024-09-21T03:36:23.085908+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:52:59.833307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-21T03:36:21.062519+0000","last_change":"2024-09-21T03:36:21.059196+0000","last_active":"2024-09-21T03:36:21.062519+0000","last_peered":"2024-09-21T03:36:21.062519+0000","last_clean":"2024-09-21T03:36:21.062519+0000","last_became_active":"2024-09-21T03:36:21.058881+0000","last_became_peered":"2024-09-21T03:36:21.058881+0000","last_unstale":"2024-09-21T03:36:21.062519+0000","last_undegraded":"2024-09-21T03:36:21.062519+0000","last_fullsized":"2024-09-21T03:36:21.062519+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_clean_scrub_stamp":"2024-09-21T03:36:20.027499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:37:14.541836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":198,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.085071+0000","last_change":"2024-09-21T03:33:25.319776+0000","last_active":"2024-09-21T03:36:23.085071+0000","last_peered":"2024-09-21T03:36:23.085071+0000","last_clean":"2024-09-21T03:36:23.085071+0000","last_became_active":"2024-09-21T03:33:25.318399+0000","last_became_peered":"2024-09-21T03:33:25.318399+0000","last_unstale":"2024-09-21T03:36:23.085071+0000","last_undegraded":"2024-09-21T03:36:23.085071+0000","last_fullsized":"2024-09-21T03:36:23.085071+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:23.016672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:23.016672+0000","last_clean_scrub_stamp":"2024-09-21T03:33:23.016672+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:32:35.906114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":798,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":34,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":27,"ondisk_log_size":27,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":1,"up_from":16,"seq":68719476776,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32564,"kb_used_data":1368,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339276,"statfs":{"total":96636764160,"available":96603418624,"internally_reserved":0,"allocated":1400832,"data_stored":954724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509480,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36124,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335716,"statfs":{"total":96636764160,"available":96599773184,"internally_reserved":0,"allocated":790528,"data_stored":357630,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":36174283},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509479,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31980,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339860,"statfs":{"total":96636764160,"available":96604016640,"internally_reserved":0,"allocated":811008,"data_stored":364482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509480,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31980,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339860,"statfs":{"total":96636764160,"available":96604016640,"internally_reserved":0,"allocated":811008,"data_stored":364378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509479,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31968,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339872,"statfs":{"total":96636764160,"available":96604028928,"internally_reserved":0,"allocated":790528,"data_stored":357630,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542183,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36148,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335692,"statfs":{"total":96636764160,"available":96599748608,"internally_reserved":0,"allocated":811008,"data_stored":359894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574887,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36692,"kb_used_data":1340,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335148,"statfs":{"total":96636764160,"available":96599191552,"internally_reserved":0,"allocated":1372160,"data_stored":940831,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542184,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36132,"kb_used_data":776,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335708,"statfs":{"total":96636764160,"available":96599764992,"internally_reserved":0,"allocated":794624,"data_stored":358212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T03:36:28.030 INFO:tasks.ceph.mds.a.smithi038.stderr:2024-09-21T03:36:28.029+0000 7fd96d497640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T03:36:30.226 DEBUG:teuthology.orchestra.run.smithi038:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T03:36:30.229 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T03:36:30.679 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T03:36:30.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:30.807+0000 7f0c34d13640 1 -- 172.21.15.38:0/2734375729 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0c30162f00 msgr2=0x7f0c300d24d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:30.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:30.807+0000 7f0c34d13640 1 --2- 172.21.15.38:0/2734375729 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0c30162f00 0x7f0c300d24d0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f0c20005fd0 tx=0x7f0c2002a530 comp rx=0 tx=0).stop 2024-09-21T03:36:30.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:30.807+0000 7f0c34d13640 1 -- 172.21.15.38:0/2734375729 shutdown_connections 2024-09-21T03:36:30.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:30.807+0000 7f0c34d13640 1 --2- 172.21.15.38:0/2734375729 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0c30162280 0x7f0c30162690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:30.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:30.807+0000 7f0c34d13640 1 --2- 172.21.15.38:0/2734375729 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0c30162f00 0x7f0c300d24d0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:30.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:30.807+0000 7f0c34d13640 1 -- 172.21.15.38:0/2734375729 >> 172.21.15.38:0/2734375729 conn(0x7f0c300b6b20 msgr2=0x7f0c300d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:30.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:30.807+0000 7f0c34d13640 1 -- 172.21.15.38:0/2734375729 shutdown_connections 2024-09-21T03:36:30.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:30.807+0000 7f0c34d13640 1 -- 172.21.15.38:0/2734375729 wait complete. 2024-09-21T03:36:30.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:30.809+0000 7f0c34d13640 1 Processor -- start 2024-09-21T03:36:30.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:30.809+0000 7f0c34d13640 1 -- start start 2024-09-21T03:36:30.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:30.810+0000 7f0c34d13640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0c300b2e40 0x7f0c300ced50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:30.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:30.810+0000 7f0c34d13640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0c30162280 0x7f0c300ab660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:30.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:30.810+0000 7f0c34d13640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0c300abba0 0x7f0c300abfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:30.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:30.810+0000 7f0c34d13640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f0c3015e630 con 0x7f0c300abba0 2024-09-21T03:36:30.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:30.810+0000 7f0c34d13640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f0c3015e4b0 con 0x7f0c300b2e40 2024-09-21T03:36:30.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:30.810+0000 7f0c34d13640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f0c3015e7b0 con 0x7f0c30162280 2024-09-21T03:36:30.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:30.811+0000 7f0c2ed9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0c300abba0 0x7f0c300abfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:30.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:30.811+0000 7f0c2ed9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0c300abba0 0x7f0c300abfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37380/0 (socket says 172.21.15.38:37380) 2024-09-21T03:36:30.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:30.811+0000 7f0c2ed9d640 1 -- 172.21.15.38:0/2581518476 learned_addr learned my addr 172.21.15.38:0/2581518476 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:36:30.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:30.811+0000 7f0c2dd9b640 1 --2- 172.21.15.38:0/2581518476 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0c30162280 0x7f0c300ab660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:30.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:30.811+0000 7f0c2e59c640 1 --2- 172.21.15.38:0/2581518476 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0c300b2e40 0x7f0c300ced50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:30.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:30.811+0000 7f0c2ed9d640 1 -- 172.21.15.38:0/2581518476 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0c30162280 msgr2=0x7f0c300ab660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:30.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:30.811+0000 7f0c2ed9d640 1 --2- 172.21.15.38:0/2581518476 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0c30162280 0x7f0c300ab660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:30.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:30.811+0000 7f0c2ed9d640 1 -- 172.21.15.38:0/2581518476 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0c300b2e40 msgr2=0x7f0c300ced50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:30.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:30.811+0000 7f0c2ed9d640 1 --2- 172.21.15.38:0/2581518476 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0c300b2e40 0x7f0c300ced50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:30.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:30.811+0000 7f0c2ed9d640 1 -- 172.21.15.38:0/2581518476 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c300ac4f0 con 0x7f0c300abba0 2024-09-21T03:36:30.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:30.811+0000 7f0c2e59c640 1 --2- 172.21.15.38:0/2581518476 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0c300b2e40 0x7f0c300ced50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:36:30.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:30.811+0000 7f0c2dd9b640 1 --2- 172.21.15.38:0/2581518476 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0c30162280 0x7f0c300ab660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:36:30.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:30.811+0000 7f0c2ed9d640 1 --2- 172.21.15.38:0/2581518476 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0c300abba0 0x7f0c300abfb0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f0c2800b9e0 tx=0x7f0c2800bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:30.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:30.811+0000 7f0c1effd640 1 -- 172.21.15.38:0/2581518476 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c28003b70 con 0x7f0c300abba0 2024-09-21T03:36:30.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:30.811+0000 7f0c1effd640 1 -- 172.21.15.38:0/2581518476 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0c28003e00 con 0x7f0c300abba0 2024-09-21T03:36:30.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:30.811+0000 7f0c34d13640 1 -- 172.21.15.38:0/2581518476 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c301ea680 con 0x7f0c300abba0 2024-09-21T03:36:30.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:30.811+0000 7f0c1effd640 1 -- 172.21.15.38:0/2581518476 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c2800cb80 con 0x7f0c300abba0 2024-09-21T03:36:30.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:30.811+0000 7f0c34d13640 1 -- 172.21.15.38:0/2581518476 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c300aae60 con 0x7f0c300abba0 2024-09-21T03:36:30.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:30.812+0000 7f0c34d13640 1 -- 172.21.15.38:0/2581518476 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c30162f00 con 0x7f0c300abba0 2024-09-21T03:36:30.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:30.813+0000 7f0c1effd640 1 -- 172.21.15.38:0/2581518476 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f0c28004000 con 0x7f0c300abba0 2024-09-21T03:36:30.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:30.814+0000 7f0c1effd640 1 --2- 172.21.15.38:0/2581518476 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0c10078b90 0x7f0c1007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:30.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:30.814+0000 7f0c1effd640 1 -- 172.21.15.38:0/2581518476 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6101+0+0 (secure 0 0 0) 0x7f0c2809ad70 con 0x7f0c300abba0 2024-09-21T03:36:30.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:30.814+0000 7f0c2e59c640 1 --2- 172.21.15.38:0/2581518476 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0c10078b90 0x7f0c1007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:30.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:30.815+0000 7f0c2e59c640 1 --2- 172.21.15.38:0/2581518476 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0c10078b90 0x7f0c1007b0a0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f0c20033000 tx=0x7f0c20004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:30.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:30.818+0000 7f0c1effd640 1 -- 172.21.15.38:0/2581518476 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f0c28063000 con 0x7f0c300abba0 2024-09-21T03:36:31.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:31.002+0000 7f0c34d13640 1 -- 172.21.15.38:0/2581518476 --> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0c300b33f0 con 0x7f0c10078b90 2024-09-21T03:36:31.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:31.007+0000 7f0c1effd640 1 -- 172.21.15.38:0/2581518476 <== mgr.4117 v2:172.21.15.195:6800/1631901344 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252000 (secure 0 0 0) 0x7f0c300b33f0 con 0x7f0c10078b90 2024-09-21T03:36:31.011 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:36:31.012 INFO:teuthology.orchestra.run.smithi038.stderr:dumped all 2024-09-21T03:36:31.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:31.010+0000 7f0c34d13640 1 -- 172.21.15.38:0/2581518476 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0c10078b90 msgr2=0x7f0c1007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:31.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:31.010+0000 7f0c34d13640 1 --2- 172.21.15.38:0/2581518476 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0c10078b90 0x7f0c1007b0a0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f0c20033000 tx=0x7f0c20004730 comp rx=0 tx=0).stop 2024-09-21T03:36:31.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:31.010+0000 7f0c34d13640 1 -- 172.21.15.38:0/2581518476 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0c300abba0 msgr2=0x7f0c300abfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:31.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:31.010+0000 7f0c34d13640 1 --2- 172.21.15.38:0/2581518476 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0c300abba0 0x7f0c300abfb0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f0c2800b9e0 tx=0x7f0c2800bea0 comp rx=0 tx=0).stop 2024-09-21T03:36:31.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:31.010+0000 7f0c34d13640 1 -- 172.21.15.38:0/2581518476 shutdown_connections 2024-09-21T03:36:31.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:31.010+0000 7f0c34d13640 1 --2- 172.21.15.38:0/2581518476 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0c300abba0 0x7f0c300abfb0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:31.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:31.010+0000 7f0c34d13640 1 --2- 172.21.15.38:0/2581518476 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0c10078b90 0x7f0c1007b0a0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:31.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:31.010+0000 7f0c34d13640 1 --2- 172.21.15.38:0/2581518476 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0c30162280 0x7f0c300ab660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:31.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:31.010+0000 7f0c34d13640 1 --2- 172.21.15.38:0/2581518476 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0c300b2e40 0x7f0c300ced50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:31.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:31.010+0000 7f0c34d13640 1 -- 172.21.15.38:0/2581518476 >> 172.21.15.38:0/2581518476 conn(0x7f0c300b6b20 msgr2=0x7f0c300b3be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:31.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:31.010+0000 7f0c34d13640 1 -- 172.21.15.38:0/2581518476 shutdown_connections 2024-09-21T03:36:31.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:31.010+0000 7f0c34d13640 1 -- 172.21.15.38:0/2581518476 wait complete. 2024-09-21T03:36:31.023 INFO:teuthology.orchestra.run.smithi038.stdout:{"pg_ready":true,"pg_map":{"version":153,"stamp":"2024-09-21T03:36:29.024364+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":145,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110,"ondisk_log_size":110,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":273640,"kb_used_data":7472,"kb_used_omap":12,"kb_used_meta":265971,"kb_avail":754701080,"statfs":{"total":773094113280,"available":772813905920,"internally_reserved":0,"allocated":7651328,"data_stored":4086748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12748,"internal_metadata":272354868},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.977192"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.159805+0000","last_change":"2024-09-21T03:36:22.088324+0000","last_active":"2024-09-21T03:36:23.159805+0000","last_peered":"2024-09-21T03:36:23.159805+0000","last_clean":"2024-09-21T03:36:23.159805+0000","last_became_active":"2024-09-21T03:36:22.088010+0000","last_became_peered":"2024-09-21T03:36:22.088010+0000","last_unstale":"2024-09-21T03:36:23.159805+0000","last_undegraded":"2024-09-21T03:36:23.159805+0000","last_fullsized":"2024-09-21T03:36:23.159805+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:38:26.838702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.384982+0000","last_change":"2024-09-21T03:36:22.067778+0000","last_active":"2024-09-21T03:36:23.384982+0000","last_peered":"2024-09-21T03:36:23.384982+0000","last_clean":"2024-09-21T03:36:23.384982+0000","last_became_active":"2024-09-21T03:36:22.067457+0000","last_became_peered":"2024-09-21T03:36:22.067457+0000","last_unstale":"2024-09-21T03:36:23.384982+0000","last_undegraded":"2024-09-21T03:36:23.384982+0000","last_fullsized":"2024-09-21T03:36:23.384982+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:49:41.079295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.086324+0000","last_change":"2024-09-21T03:36:22.071184+0000","last_active":"2024-09-21T03:36:23.086324+0000","last_peered":"2024-09-21T03:36:23.086324+0000","last_clean":"2024-09-21T03:36:23.086324+0000","last_became_active":"2024-09-21T03:36:22.070772+0000","last_became_peered":"2024-09-21T03:36:22.070772+0000","last_unstale":"2024-09-21T03:36:23.086324+0000","last_undegraded":"2024-09-21T03:36:23.086324+0000","last_fullsized":"2024-09-21T03:36:23.086324+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:17:47.479714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.661468+0000","last_change":"2024-09-21T03:36:22.070400+0000","last_active":"2024-09-21T03:36:23.661468+0000","last_peered":"2024-09-21T03:36:23.661468+0000","last_clean":"2024-09-21T03:36:23.661468+0000","last_became_active":"2024-09-21T03:36:22.068959+0000","last_became_peered":"2024-09-21T03:36:22.068959+0000","last_unstale":"2024-09-21T03:36:23.661468+0000","last_undegraded":"2024-09-21T03:36:23.661468+0000","last_fullsized":"2024-09-21T03:36:23.661468+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:30:41.436799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.086049+0000","last_change":"2024-09-21T03:36:22.081653+0000","last_active":"2024-09-21T03:36:23.086049+0000","last_peered":"2024-09-21T03:36:23.086049+0000","last_clean":"2024-09-21T03:36:23.086049+0000","last_became_active":"2024-09-21T03:36:22.081343+0000","last_became_peered":"2024-09-21T03:36:22.081343+0000","last_unstale":"2024-09-21T03:36:23.086049+0000","last_undegraded":"2024-09-21T03:36:23.086049+0000","last_fullsized":"2024-09-21T03:36:23.086049+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:37:46.632334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.384498+0000","last_change":"2024-09-21T03:36:22.084072+0000","last_active":"2024-09-21T03:36:23.384498+0000","last_peered":"2024-09-21T03:36:23.384498+0000","last_clean":"2024-09-21T03:36:23.384498+0000","last_became_active":"2024-09-21T03:36:22.083731+0000","last_became_peered":"2024-09-21T03:36:22.083731+0000","last_unstale":"2024-09-21T03:36:23.384498+0000","last_undegraded":"2024-09-21T03:36:23.384498+0000","last_fullsized":"2024-09-21T03:36:23.384498+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:07:13.763457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.159967+0000","last_change":"2024-09-21T03:36:22.074034+0000","last_active":"2024-09-21T03:36:23.159967+0000","last_peered":"2024-09-21T03:36:23.159967+0000","last_clean":"2024-09-21T03:36:23.159967+0000","last_became_active":"2024-09-21T03:36:22.073682+0000","last_became_peered":"2024-09-21T03:36:22.073682+0000","last_unstale":"2024-09-21T03:36:23.159967+0000","last_undegraded":"2024-09-21T03:36:23.159967+0000","last_fullsized":"2024-09-21T03:36:23.159967+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:18:56.463921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.085192+0000","last_change":"2024-09-21T03:36:22.080482+0000","last_active":"2024-09-21T03:36:23.085192+0000","last_peered":"2024-09-21T03:36:23.085192+0000","last_clean":"2024-09-21T03:36:23.085192+0000","last_became_active":"2024-09-21T03:36:22.080177+0000","last_became_peered":"2024-09-21T03:36:22.080177+0000","last_unstale":"2024-09-21T03:36:23.085192+0000","last_undegraded":"2024-09-21T03:36:23.085192+0000","last_fullsized":"2024-09-21T03:36:23.085192+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:56:46.927190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.661540+0000","last_change":"2024-09-21T03:36:22.077361+0000","last_active":"2024-09-21T03:36:23.661540+0000","last_peered":"2024-09-21T03:36:23.661540+0000","last_clean":"2024-09-21T03:36:23.661540+0000","last_became_active":"2024-09-21T03:36:22.077050+0000","last_became_peered":"2024-09-21T03:36:22.077050+0000","last_unstale":"2024-09-21T03:36:23.661540+0000","last_undegraded":"2024-09-21T03:36:23.661540+0000","last_fullsized":"2024-09-21T03:36:23.661540+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:30:24.348269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.085635+0000","last_change":"2024-09-21T03:36:22.077257+0000","last_active":"2024-09-21T03:36:23.085635+0000","last_peered":"2024-09-21T03:36:23.085635+0000","last_clean":"2024-09-21T03:36:23.085635+0000","last_became_active":"2024-09-21T03:36:22.076951+0000","last_became_peered":"2024-09-21T03:36:22.076951+0000","last_unstale":"2024-09-21T03:36:23.085635+0000","last_undegraded":"2024-09-21T03:36:23.085635+0000","last_fullsized":"2024-09-21T03:36:23.085635+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:23:59.409270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.661250+0000","last_change":"2024-09-21T03:36:22.072761+0000","last_active":"2024-09-21T03:36:23.661250+0000","last_peered":"2024-09-21T03:36:23.661250+0000","last_clean":"2024-09-21T03:36:23.661250+0000","last_became_active":"2024-09-21T03:36:22.072450+0000","last_became_peered":"2024-09-21T03:36:22.072450+0000","last_unstale":"2024-09-21T03:36:23.661250+0000","last_undegraded":"2024-09-21T03:36:23.661250+0000","last_fullsized":"2024-09-21T03:36:23.661250+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:04:45.989423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.661340+0000","last_change":"2024-09-21T03:36:22.086026+0000","last_active":"2024-09-21T03:36:23.661340+0000","last_peered":"2024-09-21T03:36:23.661340+0000","last_clean":"2024-09-21T03:36:23.661340+0000","last_became_active":"2024-09-21T03:36:22.085717+0000","last_became_peered":"2024-09-21T03:36:22.085717+0000","last_unstale":"2024-09-21T03:36:23.661340+0000","last_undegraded":"2024-09-21T03:36:23.661340+0000","last_fullsized":"2024-09-21T03:36:23.661340+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:47:51.103846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.660593+0000","last_change":"2024-09-21T03:36:22.090483+0000","last_active":"2024-09-21T03:36:23.660593+0000","last_peered":"2024-09-21T03:36:23.660593+0000","last_clean":"2024-09-21T03:36:23.660593+0000","last_became_active":"2024-09-21T03:36:22.090124+0000","last_became_peered":"2024-09-21T03:36:22.090124+0000","last_unstale":"2024-09-21T03:36:23.660593+0000","last_undegraded":"2024-09-21T03:36:23.660593+0000","last_fullsized":"2024-09-21T03:36:23.660593+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:52:19.716709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.660532+0000","last_change":"2024-09-21T03:36:22.080680+0000","last_active":"2024-09-21T03:36:23.660532+0000","last_peered":"2024-09-21T03:36:23.660532+0000","last_clean":"2024-09-21T03:36:23.660532+0000","last_became_active":"2024-09-21T03:36:22.080377+0000","last_became_peered":"2024-09-21T03:36:22.080377+0000","last_unstale":"2024-09-21T03:36:23.660532+0000","last_undegraded":"2024-09-21T03:36:23.660532+0000","last_fullsized":"2024-09-21T03:36:23.660532+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:54:33.394204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.085451+0000","last_change":"2024-09-21T03:36:22.073613+0000","last_active":"2024-09-21T03:36:23.085451+0000","last_peered":"2024-09-21T03:36:23.085451+0000","last_clean":"2024-09-21T03:36:23.085451+0000","last_became_active":"2024-09-21T03:36:22.073229+0000","last_became_peered":"2024-09-21T03:36:22.073229+0000","last_unstale":"2024-09-21T03:36:23.085451+0000","last_undegraded":"2024-09-21T03:36:23.085451+0000","last_fullsized":"2024-09-21T03:36:23.085451+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:37:15.857923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.660634+0000","last_change":"2024-09-21T03:36:22.085437+0000","last_active":"2024-09-21T03:36:23.660634+0000","last_peered":"2024-09-21T03:36:23.660634+0000","last_clean":"2024-09-21T03:36:23.660634+0000","last_became_active":"2024-09-21T03:36:22.085087+0000","last_became_peered":"2024-09-21T03:36:22.085087+0000","last_unstale":"2024-09-21T03:36:23.660634+0000","last_undegraded":"2024-09-21T03:36:23.660634+0000","last_fullsized":"2024-09-21T03:36:23.660634+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:30:00.925328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.082433+0000","last_change":"2024-09-21T03:36:22.081251+0000","last_active":"2024-09-21T03:36:23.082433+0000","last_peered":"2024-09-21T03:36:23.082433+0000","last_clean":"2024-09-21T03:36:23.082433+0000","last_became_active":"2024-09-21T03:36:22.080931+0000","last_became_peered":"2024-09-21T03:36:22.080931+0000","last_unstale":"2024-09-21T03:36:23.082433+0000","last_undegraded":"2024-09-21T03:36:23.082433+0000","last_fullsized":"2024-09-21T03:36:23.082433+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:08:27.308913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.661300+0000","last_change":"2024-09-21T03:36:22.069803+0000","last_active":"2024-09-21T03:36:23.661300+0000","last_peered":"2024-09-21T03:36:23.661300+0000","last_clean":"2024-09-21T03:36:23.661300+0000","last_became_active":"2024-09-21T03:36:22.069481+0000","last_became_peered":"2024-09-21T03:36:22.069481+0000","last_unstale":"2024-09-21T03:36:23.661300+0000","last_undegraded":"2024-09-21T03:36:23.661300+0000","last_fullsized":"2024-09-21T03:36:23.661300+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:15:44.483810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.661639+0000","last_change":"2024-09-21T03:36:22.076014+0000","last_active":"2024-09-21T03:36:23.661639+0000","last_peered":"2024-09-21T03:36:23.661639+0000","last_clean":"2024-09-21T03:36:23.661639+0000","last_became_active":"2024-09-21T03:36:22.075689+0000","last_became_peered":"2024-09-21T03:36:22.075689+0000","last_unstale":"2024-09-21T03:36:23.661639+0000","last_undegraded":"2024-09-21T03:36:23.661639+0000","last_fullsized":"2024-09-21T03:36:23.661639+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:47:25.173315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.082057+0000","last_change":"2024-09-21T03:36:22.071709+0000","last_active":"2024-09-21T03:36:23.082057+0000","last_peered":"2024-09-21T03:36:23.082057+0000","last_clean":"2024-09-21T03:36:23.082057+0000","last_became_active":"2024-09-21T03:36:22.070413+0000","last_became_peered":"2024-09-21T03:36:22.070413+0000","last_unstale":"2024-09-21T03:36:23.082057+0000","last_undegraded":"2024-09-21T03:36:23.082057+0000","last_fullsized":"2024-09-21T03:36:23.082057+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:30:33.810045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.660747+0000","last_change":"2024-09-21T03:36:22.086945+0000","last_active":"2024-09-21T03:36:23.660747+0000","last_peered":"2024-09-21T03:36:23.660747+0000","last_clean":"2024-09-21T03:36:23.660747+0000","last_became_active":"2024-09-21T03:36:22.086620+0000","last_became_peered":"2024-09-21T03:36:22.086620+0000","last_unstale":"2024-09-21T03:36:23.660747+0000","last_undegraded":"2024-09-21T03:36:23.660747+0000","last_fullsized":"2024-09-21T03:36:23.660747+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:48:46.104173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.384855+0000","last_change":"2024-09-21T03:36:22.084138+0000","last_active":"2024-09-21T03:36:23.384855+0000","last_peered":"2024-09-21T03:36:23.384855+0000","last_clean":"2024-09-21T03:36:23.384855+0000","last_became_active":"2024-09-21T03:36:22.083810+0000","last_became_peered":"2024-09-21T03:36:22.083810+0000","last_unstale":"2024-09-21T03:36:23.384855+0000","last_undegraded":"2024-09-21T03:36:23.384855+0000","last_fullsized":"2024-09-21T03:36:23.384855+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:22:50.955268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.660604+0000","last_change":"2024-09-21T03:36:22.086122+0000","last_active":"2024-09-21T03:36:23.660604+0000","last_peered":"2024-09-21T03:36:23.660604+0000","last_clean":"2024-09-21T03:36:23.660604+0000","last_became_active":"2024-09-21T03:36:22.085783+0000","last_became_peered":"2024-09-21T03:36:22.085783+0000","last_unstale":"2024-09-21T03:36:23.660604+0000","last_undegraded":"2024-09-21T03:36:23.660604+0000","last_fullsized":"2024-09-21T03:36:23.660604+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:05:24.777050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.085420+0000","last_change":"2024-09-21T03:36:22.067412+0000","last_active":"2024-09-21T03:36:23.085420+0000","last_peered":"2024-09-21T03:36:23.085420+0000","last_clean":"2024-09-21T03:36:23.085420+0000","last_became_active":"2024-09-21T03:36:22.067084+0000","last_became_peered":"2024-09-21T03:36:22.067084+0000","last_unstale":"2024-09-21T03:36:23.085420+0000","last_undegraded":"2024-09-21T03:36:23.085420+0000","last_fullsized":"2024-09-21T03:36:23.085420+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:01:59.040840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.661165+0000","last_change":"2024-09-21T03:36:22.082532+0000","last_active":"2024-09-21T03:36:23.661165+0000","last_peered":"2024-09-21T03:36:23.661165+0000","last_clean":"2024-09-21T03:36:23.661165+0000","last_became_active":"2024-09-21T03:36:22.082233+0000","last_became_peered":"2024-09-21T03:36:22.082233+0000","last_unstale":"2024-09-21T03:36:23.661165+0000","last_undegraded":"2024-09-21T03:36:23.661165+0000","last_fullsized":"2024-09-21T03:36:23.661165+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:09:57.485058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.086025+0000","last_change":"2024-09-21T03:36:22.068982+0000","last_active":"2024-09-21T03:36:23.086025+0000","last_peered":"2024-09-21T03:36:23.086025+0000","last_clean":"2024-09-21T03:36:23.086025+0000","last_became_active":"2024-09-21T03:36:22.068584+0000","last_became_peered":"2024-09-21T03:36:22.068584+0000","last_unstale":"2024-09-21T03:36:23.086025+0000","last_undegraded":"2024-09-21T03:36:23.086025+0000","last_fullsized":"2024-09-21T03:36:23.086025+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:02:01.275514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.660817+0000","last_change":"2024-09-21T03:36:22.085467+0000","last_active":"2024-09-21T03:36:23.660817+0000","last_peered":"2024-09-21T03:36:23.660817+0000","last_clean":"2024-09-21T03:36:23.660817+0000","last_became_active":"2024-09-21T03:36:22.085127+0000","last_became_peered":"2024-09-21T03:36:22.085127+0000","last_unstale":"2024-09-21T03:36:23.660817+0000","last_undegraded":"2024-09-21T03:36:23.660817+0000","last_fullsized":"2024-09-21T03:36:23.660817+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:04:32.859631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.159829+0000","last_change":"2024-09-21T03:36:22.083174+0000","last_active":"2024-09-21T03:36:23.159829+0000","last_peered":"2024-09-21T03:36:23.159829+0000","last_clean":"2024-09-21T03:36:23.159829+0000","last_became_active":"2024-09-21T03:36:22.082930+0000","last_became_peered":"2024-09-21T03:36:22.082930+0000","last_unstale":"2024-09-21T03:36:23.159829+0000","last_undegraded":"2024-09-21T03:36:23.159829+0000","last_fullsized":"2024-09-21T03:36:23.159829+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:35:15.414365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.661615+0000","last_change":"2024-09-21T03:36:22.088889+0000","last_active":"2024-09-21T03:36:23.661615+0000","last_peered":"2024-09-21T03:36:23.661615+0000","last_clean":"2024-09-21T03:36:23.661615+0000","last_became_active":"2024-09-21T03:36:22.088515+0000","last_became_peered":"2024-09-21T03:36:22.088515+0000","last_unstale":"2024-09-21T03:36:23.661615+0000","last_undegraded":"2024-09-21T03:36:23.661615+0000","last_fullsized":"2024-09-21T03:36:23.661615+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:00:06.850866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.660822+0000","last_change":"2024-09-21T03:36:22.088463+0000","last_active":"2024-09-21T03:36:23.660822+0000","last_peered":"2024-09-21T03:36:23.660822+0000","last_clean":"2024-09-21T03:36:23.660822+0000","last_became_active":"2024-09-21T03:36:22.088149+0000","last_became_peered":"2024-09-21T03:36:22.088149+0000","last_unstale":"2024-09-21T03:36:23.660822+0000","last_undegraded":"2024-09-21T03:36:23.660822+0000","last_fullsized":"2024-09-21T03:36:23.660822+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:03:04.392175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.085608+0000","last_change":"2024-09-21T03:36:22.066107+0000","last_active":"2024-09-21T03:36:23.085608+0000","last_peered":"2024-09-21T03:36:23.085608+0000","last_clean":"2024-09-21T03:36:23.085608+0000","last_became_active":"2024-09-21T03:36:22.064917+0000","last_became_peered":"2024-09-21T03:36:22.064917+0000","last_unstale":"2024-09-21T03:36:23.085608+0000","last_undegraded":"2024-09-21T03:36:23.085608+0000","last_fullsized":"2024-09-21T03:36:23.085608+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:58:06.692992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.159573+0000","last_change":"2024-09-21T03:36:22.069000+0000","last_active":"2024-09-21T03:36:23.159573+0000","last_peered":"2024-09-21T03:36:23.159573+0000","last_clean":"2024-09-21T03:36:23.159573+0000","last_became_active":"2024-09-21T03:36:22.068661+0000","last_became_peered":"2024-09-21T03:36:22.068661+0000","last_unstale":"2024-09-21T03:36:23.159573+0000","last_undegraded":"2024-09-21T03:36:23.159573+0000","last_fullsized":"2024-09-21T03:36:23.159573+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:12:22.364774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.661745+0000","last_change":"2024-09-21T03:36:22.071976+0000","last_active":"2024-09-21T03:36:23.661745+0000","last_peered":"2024-09-21T03:36:23.661745+0000","last_clean":"2024-09-21T03:36:23.661745+0000","last_became_active":"2024-09-21T03:36:22.071182+0000","last_became_peered":"2024-09-21T03:36:22.071182+0000","last_unstale":"2024-09-21T03:36:23.661745+0000","last_undegraded":"2024-09-21T03:36:23.661745+0000","last_fullsized":"2024-09-21T03:36:23.661745+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:58:44.343782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1f","version":"66'4","reported_seq":19,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.159755+0000","last_change":"2024-09-21T03:36:21.066587+0000","last_active":"2024-09-21T03:36:23.159755+0000","last_peered":"2024-09-21T03:36:23.159755+0000","last_clean":"2024-09-21T03:36:23.159755+0000","last_became_active":"2024-09-21T03:36:21.066299+0000","last_became_peered":"2024-09-21T03:36:21.066299+0000","last_unstale":"2024-09-21T03:36:23.159755+0000","last_undegraded":"2024-09-21T03:36:23.159755+0000","last_fullsized":"2024-09-21T03:36:23.159755+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_clean_scrub_stamp":"2024-09-21T03:36:20.027499+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:06:51.671380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.086591+0000","last_change":"2024-09-21T03:36:22.073629+0000","last_active":"2024-09-21T03:36:23.086591+0000","last_peered":"2024-09-21T03:36:23.086591+0000","last_clean":"2024-09-21T03:36:23.086591+0000","last_became_active":"2024-09-21T03:36:22.073211+0000","last_became_peered":"2024-09-21T03:36:22.073211+0000","last_unstale":"2024-09-21T03:36:23.086591+0000","last_undegraded":"2024-09-21T03:36:23.086591+0000","last_fullsized":"2024-09-21T03:36:23.086591+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:27:32.381253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1e","version":"66'1","reported_seq":16,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.159761+0000","last_change":"2024-09-21T03:36:21.064384+0000","last_active":"2024-09-21T03:36:23.159761+0000","last_peered":"2024-09-21T03:36:23.159761+0000","last_clean":"2024-09-21T03:36:23.159761+0000","last_became_active":"2024-09-21T03:36:21.064042+0000","last_became_peered":"2024-09-21T03:36:21.064042+0000","last_unstale":"2024-09-21T03:36:23.159761+0000","last_undegraded":"2024-09-21T03:36:23.159761+0000","last_fullsized":"2024-09-21T03:36:23.159761+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_clean_scrub_stamp":"2024-09-21T03:36:20.027499+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:17:25.045895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.385051+0000","last_change":"2024-09-21T03:36:22.066306+0000","last_active":"2024-09-21T03:36:23.385051+0000","last_peered":"2024-09-21T03:36:23.385051+0000","last_clean":"2024-09-21T03:36:23.385051+0000","last_became_active":"2024-09-21T03:36:22.065997+0000","last_became_peered":"2024-09-21T03:36:22.065997+0000","last_unstale":"2024-09-21T03:36:23.385051+0000","last_undegraded":"2024-09-21T03:36:23.385051+0000","last_fullsized":"2024-09-21T03:36:23.385051+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:22:09.609281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1d","version":"66'2","reported_seq":17,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.160128+0000","last_change":"2024-09-21T03:36:21.071133+0000","last_active":"2024-09-21T03:36:23.160128+0000","last_peered":"2024-09-21T03:36:23.160128+0000","last_clean":"2024-09-21T03:36:23.160128+0000","last_became_active":"2024-09-21T03:36:21.070834+0000","last_became_peered":"2024-09-21T03:36:21.070834+0000","last_unstale":"2024-09-21T03:36:23.160128+0000","last_undegraded":"2024-09-21T03:36:23.160128+0000","last_fullsized":"2024-09-21T03:36:23.160128+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_clean_scrub_stamp":"2024-09-21T03:36:20.027499+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:37:25.878136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.661630+0000","last_change":"2024-09-21T03:36:22.070080+0000","last_active":"2024-09-21T03:36:23.661630+0000","last_peered":"2024-09-21T03:36:23.661630+0000","last_clean":"2024-09-21T03:36:23.661630+0000","last_became_active":"2024-09-21T03:36:22.069779+0000","last_became_peered":"2024-09-21T03:36:22.069779+0000","last_unstale":"2024-09-21T03:36:23.661630+0000","last_undegraded":"2024-09-21T03:36:23.661630+0000","last_fullsized":"2024-09-21T03:36:23.661630+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:29:10.577562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1c","version":"66'2","reported_seq":17,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.384652+0000","last_change":"2024-09-21T03:36:21.065338+0000","last_active":"2024-09-21T03:36:23.384652+0000","last_peered":"2024-09-21T03:36:23.384652+0000","last_clean":"2024-09-21T03:36:23.384652+0000","last_became_active":"2024-09-21T03:36:21.065032+0000","last_became_peered":"2024-09-21T03:36:21.065032+0000","last_unstale":"2024-09-21T03:36:23.384652+0000","last_undegraded":"2024-09-21T03:36:23.384652+0000","last_fullsized":"2024-09-21T03:36:23.384652+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_clean_scrub_stamp":"2024-09-21T03:36:20.027499+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:15:54.672395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.081698+0000","last_change":"2024-09-21T03:36:22.063213+0000","last_active":"2024-09-21T03:36:23.081698+0000","last_peered":"2024-09-21T03:36:23.081698+0000","last_clean":"2024-09-21T03:36:23.081698+0000","last_became_active":"2024-09-21T03:36:22.062870+0000","last_became_peered":"2024-09-21T03:36:22.062870+0000","last_unstale":"2024-09-21T03:36:23.081698+0000","last_undegraded":"2024-09-21T03:36:23.081698+0000","last_fullsized":"2024-09-21T03:36:23.081698+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:24:11.515272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"66'1","reported_seq":16,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.083109+0000","last_change":"2024-09-21T03:36:21.060507+0000","last_active":"2024-09-21T03:36:23.083109+0000","last_peered":"2024-09-21T03:36:23.083109+0000","last_clean":"2024-09-21T03:36:23.083109+0000","last_became_active":"2024-09-21T03:36:21.060129+0000","last_became_peered":"2024-09-21T03:36:21.060129+0000","last_unstale":"2024-09-21T03:36:23.083109+0000","last_undegraded":"2024-09-21T03:36:23.083109+0000","last_fullsized":"2024-09-21T03:36:23.083109+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_clean_scrub_stamp":"2024-09-21T03:36:20.027499+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:57:03.979240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.159578+0000","last_change":"2024-09-21T03:36:22.077206+0000","last_active":"2024-09-21T03:36:23.159578+0000","last_peered":"2024-09-21T03:36:23.159578+0000","last_clean":"2024-09-21T03:36:23.159578+0000","last_became_active":"2024-09-21T03:36:22.076897+0000","last_became_peered":"2024-09-21T03:36:22.076897+0000","last_unstale":"2024-09-21T03:36:23.159578+0000","last_undegraded":"2024-09-21T03:36:23.159578+0000","last_fullsized":"2024-09-21T03:36:23.159578+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:59:12.697168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1a","version":"66'1","reported_seq":16,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.384813+0000","last_change":"2024-09-21T03:36:21.059505+0000","last_active":"2024-09-21T03:36:23.384813+0000","last_peered":"2024-09-21T03:36:23.384813+0000","last_clean":"2024-09-21T03:36:23.384813+0000","last_became_active":"2024-09-21T03:36:21.059110+0000","last_became_peered":"2024-09-21T03:36:21.059110+0000","last_unstale":"2024-09-21T03:36:23.384813+0000","last_undegraded":"2024-09-21T03:36:23.384813+0000","last_fullsized":"2024-09-21T03:36:23.384813+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_clean_scrub_stamp":"2024-09-21T03:36:20.027499+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:38:58.183487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.085728+0000","last_change":"2024-09-21T03:36:22.086098+0000","last_active":"2024-09-21T03:36:23.085728+0000","last_peered":"2024-09-21T03:36:23.085728+0000","last_clean":"2024-09-21T03:36:23.085728+0000","last_became_active":"2024-09-21T03:36:22.085410+0000","last_became_peered":"2024-09-21T03:36:22.085410+0000","last_unstale":"2024-09-21T03:36:23.085728+0000","last_undegraded":"2024-09-21T03:36:23.085728+0000","last_fullsized":"2024-09-21T03:36:23.085728+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:14:19.660267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.159983+0000","last_change":"2024-09-21T03:36:21.065608+0000","last_active":"2024-09-21T03:36:23.159983+0000","last_peered":"2024-09-21T03:36:23.159983+0000","last_clean":"2024-09-21T03:36:23.159983+0000","last_became_active":"2024-09-21T03:36:21.065292+0000","last_became_peered":"2024-09-21T03:36:21.065292+0000","last_unstale":"2024-09-21T03:36:23.159983+0000","last_undegraded":"2024-09-21T03:36:23.159983+0000","last_fullsized":"2024-09-21T03:36:23.159983+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_clean_scrub_stamp":"2024-09-21T03:36:20.027499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:36:50.067003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.082703+0000","last_change":"2024-09-21T03:36:22.069640+0000","last_active":"2024-09-21T03:36:23.082703+0000","last_peered":"2024-09-21T03:36:23.082703+0000","last_clean":"2024-09-21T03:36:23.082703+0000","last_became_active":"2024-09-21T03:36:22.068836+0000","last_became_peered":"2024-09-21T03:36:22.068836+0000","last_unstale":"2024-09-21T03:36:23.082703+0000","last_undegraded":"2024-09-21T03:36:23.082703+0000","last_fullsized":"2024-09-21T03:36:23.082703+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:45:32.938463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.660609+0000","last_change":"2024-09-21T03:36:21.062628+0000","last_active":"2024-09-21T03:36:23.660609+0000","last_peered":"2024-09-21T03:36:23.660609+0000","last_clean":"2024-09-21T03:36:23.660609+0000","last_became_active":"2024-09-21T03:36:21.062295+0000","last_became_peered":"2024-09-21T03:36:21.062295+0000","last_unstale":"2024-09-21T03:36:23.660609+0000","last_undegraded":"2024-09-21T03:36:23.660609+0000","last_fullsized":"2024-09-21T03:36:23.660609+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_clean_scrub_stamp":"2024-09-21T03:36:20.027499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:20:15.258373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.160007+0000","last_change":"2024-09-21T03:36:22.067826+0000","last_active":"2024-09-21T03:36:23.160007+0000","last_peered":"2024-09-21T03:36:23.160007+0000","last_clean":"2024-09-21T03:36:23.160007+0000","last_became_active":"2024-09-21T03:36:22.067516+0000","last_became_peered":"2024-09-21T03:36:22.067516+0000","last_unstale":"2024-09-21T03:36:23.160007+0000","last_undegraded":"2024-09-21T03:36:23.160007+0000","last_fullsized":"2024-09-21T03:36:23.160007+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:58:18.713702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.661694+0000","last_change":"2024-09-21T03:36:21.063524+0000","last_active":"2024-09-21T03:36:23.661694+0000","last_peered":"2024-09-21T03:36:23.661694+0000","last_clean":"2024-09-21T03:36:23.661694+0000","last_became_active":"2024-09-21T03:36:21.063203+0000","last_became_peered":"2024-09-21T03:36:21.063203+0000","last_unstale":"2024-09-21T03:36:23.661694+0000","last_undegraded":"2024-09-21T03:36:23.661694+0000","last_fullsized":"2024-09-21T03:36:23.661694+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_clean_scrub_stamp":"2024-09-21T03:36:20.027499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:25:29.512603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.082311+0000","last_change":"2024-09-21T03:36:22.063211+0000","last_active":"2024-09-21T03:36:23.082311+0000","last_peered":"2024-09-21T03:36:23.082311+0000","last_clean":"2024-09-21T03:36:23.082311+0000","last_became_active":"2024-09-21T03:36:22.062828+0000","last_became_peered":"2024-09-21T03:36:22.062828+0000","last_unstale":"2024-09-21T03:36:23.082311+0000","last_undegraded":"2024-09-21T03:36:23.082311+0000","last_fullsized":"2024-09-21T03:36:23.082311+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:01:59.440685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.661311+0000","last_change":"2024-09-21T03:36:21.064314+0000","last_active":"2024-09-21T03:36:23.661311+0000","last_peered":"2024-09-21T03:36:23.661311+0000","last_clean":"2024-09-21T03:36:23.661311+0000","last_became_active":"2024-09-21T03:36:21.063271+0000","last_became_peered":"2024-09-21T03:36:21.063271+0000","last_unstale":"2024-09-21T03:36:23.661311+0000","last_undegraded":"2024-09-21T03:36:23.661311+0000","last_fullsized":"2024-09-21T03:36:23.661311+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_clean_scrub_stamp":"2024-09-21T03:36:20.027499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:03:09.222194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.660928+0000","last_change":"2024-09-21T03:36:22.092139+0000","last_active":"2024-09-21T03:36:23.660928+0000","last_peered":"2024-09-21T03:36:23.660928+0000","last_clean":"2024-09-21T03:36:23.660928+0000","last_became_active":"2024-09-21T03:36:22.091718+0000","last_became_peered":"2024-09-21T03:36:22.091718+0000","last_unstale":"2024-09-21T03:36:23.660928+0000","last_undegraded":"2024-09-21T03:36:23.660928+0000","last_fullsized":"2024-09-21T03:36:23.660928+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:28:40.036155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.15","version":"66'3","reported_seq":18,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.086559+0000","last_change":"2024-09-21T03:36:21.064387+0000","last_active":"2024-09-21T03:36:23.086559+0000","last_peered":"2024-09-21T03:36:23.086559+0000","last_clean":"2024-09-21T03:36:23.086559+0000","last_became_active":"2024-09-21T03:36:21.064086+0000","last_became_peered":"2024-09-21T03:36:21.064086+0000","last_unstale":"2024-09-21T03:36:23.086559+0000","last_undegraded":"2024-09-21T03:36:23.086559+0000","last_fullsized":"2024-09-21T03:36:23.086559+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_clean_scrub_stamp":"2024-09-21T03:36:20.027499+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:01:58.431275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.086256+0000","last_change":"2024-09-21T03:36:22.070639+0000","last_active":"2024-09-21T03:36:23.086256+0000","last_peered":"2024-09-21T03:36:23.086256+0000","last_clean":"2024-09-21T03:36:23.086256+0000","last_became_active":"2024-09-21T03:36:22.069620+0000","last_became_peered":"2024-09-21T03:36:22.069620+0000","last_unstale":"2024-09-21T03:36:23.086256+0000","last_undegraded":"2024-09-21T03:36:23.086256+0000","last_fullsized":"2024-09-21T03:36:23.086256+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:23:36.147744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"66'6","reported_seq":21,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.384724+0000","last_change":"2024-09-21T03:36:21.065091+0000","last_active":"2024-09-21T03:36:23.384724+0000","last_peered":"2024-09-21T03:36:23.384724+0000","last_clean":"2024-09-21T03:36:23.384724+0000","last_became_active":"2024-09-21T03:36:21.064760+0000","last_became_peered":"2024-09-21T03:36:21.064760+0000","last_unstale":"2024-09-21T03:36:23.384724+0000","last_undegraded":"2024-09-21T03:36:23.384724+0000","last_fullsized":"2024-09-21T03:36:23.384724+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_clean_scrub_stamp":"2024-09-21T03:36:20.027499+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:40:26.782003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.085112+0000","last_change":"2024-09-21T03:36:22.085864+0000","last_active":"2024-09-21T03:36:23.085112+0000","last_peered":"2024-09-21T03:36:23.085112+0000","last_clean":"2024-09-21T03:36:23.085112+0000","last_became_active":"2024-09-21T03:36:22.085557+0000","last_became_peered":"2024-09-21T03:36:22.085557+0000","last_unstale":"2024-09-21T03:36:23.085112+0000","last_undegraded":"2024-09-21T03:36:23.085112+0000","last_fullsized":"2024-09-21T03:36:23.085112+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:25:31.677998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.13","version":"66'2","reported_seq":17,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.661866+0000","last_change":"2024-09-21T03:36:21.064320+0000","last_active":"2024-09-21T03:36:23.661866+0000","last_peered":"2024-09-21T03:36:23.661866+0000","last_clean":"2024-09-21T03:36:23.661866+0000","last_became_active":"2024-09-21T03:36:21.064030+0000","last_became_peered":"2024-09-21T03:36:21.064030+0000","last_unstale":"2024-09-21T03:36:23.661866+0000","last_undegraded":"2024-09-21T03:36:23.661866+0000","last_fullsized":"2024-09-21T03:36:23.661866+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_clean_scrub_stamp":"2024-09-21T03:36:20.027499+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:36:50.067003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.085860+0000","last_change":"2024-09-21T03:36:22.072437+0000","last_active":"2024-09-21T03:36:23.085860+0000","last_peered":"2024-09-21T03:36:23.085860+0000","last_clean":"2024-09-21T03:36:23.085860+0000","last_became_active":"2024-09-21T03:36:22.072130+0000","last_became_peered":"2024-09-21T03:36:22.072130+0000","last_unstale":"2024-09-21T03:36:23.085860+0000","last_undegraded":"2024-09-21T03:36:23.085860+0000","last_fullsized":"2024-09-21T03:36:23.085860+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:32:25.157463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.085572+0000","last_change":"2024-09-21T03:36:21.067895+0000","last_active":"2024-09-21T03:36:23.085572+0000","last_peered":"2024-09-21T03:36:23.085572+0000","last_clean":"2024-09-21T03:36:23.085572+0000","last_became_active":"2024-09-21T03:36:21.067573+0000","last_became_peered":"2024-09-21T03:36:21.067573+0000","last_unstale":"2024-09-21T03:36:23.085572+0000","last_undegraded":"2024-09-21T03:36:23.085572+0000","last_fullsized":"2024-09-21T03:36:23.085572+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_clean_scrub_stamp":"2024-09-21T03:36:20.027499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:25:13.103147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.085382+0000","last_change":"2024-09-21T03:36:22.086475+0000","last_active":"2024-09-21T03:36:23.085382+0000","last_peered":"2024-09-21T03:36:23.085382+0000","last_clean":"2024-09-21T03:36:23.085382+0000","last_became_active":"2024-09-21T03:36:22.085797+0000","last_became_peered":"2024-09-21T03:36:22.085797+0000","last_unstale":"2024-09-21T03:36:23.085382+0000","last_undegraded":"2024-09-21T03:36:23.085382+0000","last_fullsized":"2024-09-21T03:36:23.085382+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:18:08.028552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.085748+0000","last_change":"2024-09-21T03:36:21.061510+0000","last_active":"2024-09-21T03:36:23.085748+0000","last_peered":"2024-09-21T03:36:23.085748+0000","last_clean":"2024-09-21T03:36:23.085748+0000","last_became_active":"2024-09-21T03:36:21.061181+0000","last_became_peered":"2024-09-21T03:36:21.061181+0000","last_unstale":"2024-09-21T03:36:23.085748+0000","last_undegraded":"2024-09-21T03:36:23.085748+0000","last_fullsized":"2024-09-21T03:36:23.085748+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_clean_scrub_stamp":"2024-09-21T03:36:20.027499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:39:48.217217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.159669+0000","last_change":"2024-09-21T03:36:22.085750+0000","last_active":"2024-09-21T03:36:23.159669+0000","last_peered":"2024-09-21T03:36:23.159669+0000","last_clean":"2024-09-21T03:36:23.159669+0000","last_became_active":"2024-09-21T03:36:22.085426+0000","last_became_peered":"2024-09-21T03:36:22.085426+0000","last_unstale":"2024-09-21T03:36:23.159669+0000","last_undegraded":"2024-09-21T03:36:23.159669+0000","last_fullsized":"2024-09-21T03:36:23.159669+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:09:57.485058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"66'8","reported_seq":23,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.084997+0000","last_change":"2024-09-21T03:36:21.063882+0000","last_active":"2024-09-21T03:36:23.084997+0000","last_peered":"2024-09-21T03:36:23.084997+0000","last_clean":"2024-09-21T03:36:23.084997+0000","last_became_active":"2024-09-21T03:36:21.063544+0000","last_became_peered":"2024-09-21T03:36:21.063544+0000","last_unstale":"2024-09-21T03:36:23.084997+0000","last_undegraded":"2024-09-21T03:36:23.084997+0000","last_fullsized":"2024-09-21T03:36:23.084997+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_clean_scrub_stamp":"2024-09-21T03:36:20.027499+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:38:51.296443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.661909+0000","last_change":"2024-09-21T03:36:22.086802+0000","last_active":"2024-09-21T03:36:23.661909+0000","last_peered":"2024-09-21T03:36:23.661909+0000","last_clean":"2024-09-21T03:36:23.661909+0000","last_became_active":"2024-09-21T03:36:22.086435+0000","last_became_peered":"2024-09-21T03:36:22.086435+0000","last_unstale":"2024-09-21T03:36:23.661909+0000","last_undegraded":"2024-09-21T03:36:23.661909+0000","last_fullsized":"2024-09-21T03:36:23.661909+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:32:26.377539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.f","version":"66'1","reported_seq":16,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.661756+0000","last_change":"2024-09-21T03:36:21.062807+0000","last_active":"2024-09-21T03:36:23.661756+0000","last_peered":"2024-09-21T03:36:23.661756+0000","last_clean":"2024-09-21T03:36:23.661756+0000","last_became_active":"2024-09-21T03:36:21.062513+0000","last_became_peered":"2024-09-21T03:36:21.062513+0000","last_unstale":"2024-09-21T03:36:23.661756+0000","last_undegraded":"2024-09-21T03:36:23.661756+0000","last_fullsized":"2024-09-21T03:36:23.661756+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_clean_scrub_stamp":"2024-09-21T03:36:20.027499+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:39:57.160597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.660925+0000","last_change":"2024-09-21T03:36:22.086393+0000","last_active":"2024-09-21T03:36:23.660925+0000","last_peered":"2024-09-21T03:36:23.660925+0000","last_clean":"2024-09-21T03:36:23.660925+0000","last_became_active":"2024-09-21T03:36:22.086094+0000","last_became_peered":"2024-09-21T03:36:22.086094+0000","last_unstale":"2024-09-21T03:36:23.660925+0000","last_undegraded":"2024-09-21T03:36:23.660925+0000","last_fullsized":"2024-09-21T03:36:23.660925+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:11:34.432933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.e","version":"66'2","reported_seq":17,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.661138+0000","last_change":"2024-09-21T03:36:21.053740+0000","last_active":"2024-09-21T03:36:23.661138+0000","last_peered":"2024-09-21T03:36:23.661138+0000","last_clean":"2024-09-21T03:36:23.661138+0000","last_became_active":"2024-09-21T03:36:21.053383+0000","last_became_peered":"2024-09-21T03:36:21.053383+0000","last_unstale":"2024-09-21T03:36:23.661138+0000","last_undegraded":"2024-09-21T03:36:23.661138+0000","last_fullsized":"2024-09-21T03:36:23.661138+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_clean_scrub_stamp":"2024-09-21T03:36:20.027499+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:37:25.878136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.661181+0000","last_change":"2024-09-21T03:36:22.093865+0000","last_active":"2024-09-21T03:36:23.661181+0000","last_peered":"2024-09-21T03:36:23.661181+0000","last_clean":"2024-09-21T03:36:23.661181+0000","last_became_active":"2024-09-21T03:36:22.093526+0000","last_became_peered":"2024-09-21T03:36:22.093526+0000","last_unstale":"2024-09-21T03:36:23.661181+0000","last_undegraded":"2024-09-21T03:36:23.661181+0000","last_fullsized":"2024-09-21T03:36:23.661181+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:59:12.697168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.d","version":"66'1","reported_seq":16,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.086666+0000","last_change":"2024-09-21T03:36:21.063656+0000","last_active":"2024-09-21T03:36:23.086666+0000","last_peered":"2024-09-21T03:36:23.086666+0000","last_clean":"2024-09-21T03:36:23.086666+0000","last_became_active":"2024-09-21T03:36:21.063355+0000","last_became_peered":"2024-09-21T03:36:21.063355+0000","last_unstale":"2024-09-21T03:36:23.086666+0000","last_undegraded":"2024-09-21T03:36:23.086666+0000","last_fullsized":"2024-09-21T03:36:23.086666+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_clean_scrub_stamp":"2024-09-21T03:36:20.027499+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:05:53.691084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.086342+0000","last_change":"2024-09-21T03:36:22.070297+0000","last_active":"2024-09-21T03:36:23.086342+0000","last_peered":"2024-09-21T03:36:23.086342+0000","last_clean":"2024-09-21T03:36:23.086342+0000","last_became_active":"2024-09-21T03:36:22.069258+0000","last_became_peered":"2024-09-21T03:36:22.069258+0000","last_unstale":"2024-09-21T03:36:23.086342+0000","last_undegraded":"2024-09-21T03:36:23.086342+0000","last_fullsized":"2024-09-21T03:36:23.086342+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:51:57.465792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.660607+0000","last_change":"2024-09-21T03:36:21.067658+0000","last_active":"2024-09-21T03:36:23.660607+0000","last_peered":"2024-09-21T03:36:23.660607+0000","last_clean":"2024-09-21T03:36:23.660607+0000","last_became_active":"2024-09-21T03:36:21.067359+0000","last_became_peered":"2024-09-21T03:36:21.067359+0000","last_unstale":"2024-09-21T03:36:23.660607+0000","last_undegraded":"2024-09-21T03:36:23.660607+0000","last_fullsized":"2024-09-21T03:36:23.660607+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_clean_scrub_stamp":"2024-09-21T03:36:20.027499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:38:26.468485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.384483+0000","last_change":"2024-09-21T03:36:22.084015+0000","last_active":"2024-09-21T03:36:23.384483+0000","last_peered":"2024-09-21T03:36:23.384483+0000","last_clean":"2024-09-21T03:36:23.384483+0000","last_became_active":"2024-09-21T03:36:22.083570+0000","last_became_peered":"2024-09-21T03:36:22.083570+0000","last_unstale":"2024-09-21T03:36:23.384483+0000","last_undegraded":"2024-09-21T03:36:23.384483+0000","last_fullsized":"2024-09-21T03:36:23.384483+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:47:40.026408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.b","version":"66'2","reported_seq":17,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.384539+0000","last_change":"2024-09-21T03:36:21.067047+0000","last_active":"2024-09-21T03:36:23.384539+0000","last_peered":"2024-09-21T03:36:23.384539+0000","last_clean":"2024-09-21T03:36:23.384539+0000","last_became_active":"2024-09-21T03:36:21.066718+0000","last_became_peered":"2024-09-21T03:36:21.066718+0000","last_unstale":"2024-09-21T03:36:23.384539+0000","last_undegraded":"2024-09-21T03:36:23.384539+0000","last_fullsized":"2024-09-21T03:36:23.384539+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_clean_scrub_stamp":"2024-09-21T03:36:20.027499+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:35:39.732264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.660351+0000","last_change":"2024-09-21T03:36:22.081020+0000","last_active":"2024-09-21T03:36:23.660351+0000","last_peered":"2024-09-21T03:36:23.660351+0000","last_clean":"2024-09-21T03:36:23.660351+0000","last_became_active":"2024-09-21T03:36:22.080090+0000","last_became_peered":"2024-09-21T03:36:22.080090+0000","last_unstale":"2024-09-21T03:36:23.660351+0000","last_undegraded":"2024-09-21T03:36:23.660351+0000","last_fullsized":"2024-09-21T03:36:23.660351+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:05:02.647190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.085682+0000","last_change":"2024-09-21T03:36:21.065401+0000","last_active":"2024-09-21T03:36:23.085682+0000","last_peered":"2024-09-21T03:36:23.085682+0000","last_clean":"2024-09-21T03:36:23.085682+0000","last_became_active":"2024-09-21T03:36:21.065075+0000","last_became_peered":"2024-09-21T03:36:21.065075+0000","last_unstale":"2024-09-21T03:36:23.085682+0000","last_undegraded":"2024-09-21T03:36:23.085682+0000","last_fullsized":"2024-09-21T03:36:23.085682+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_clean_scrub_stamp":"2024-09-21T03:36:20.027499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:06:26.032047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.660983+0000","last_change":"2024-09-21T03:36:22.071663+0000","last_active":"2024-09-21T03:36:23.660983+0000","last_peered":"2024-09-21T03:36:23.660983+0000","last_clean":"2024-09-21T03:36:23.660983+0000","last_became_active":"2024-09-21T03:36:22.071326+0000","last_became_peered":"2024-09-21T03:36:22.071326+0000","last_unstale":"2024-09-21T03:36:23.660983+0000","last_undegraded":"2024-09-21T03:36:23.660983+0000","last_fullsized":"2024-09-21T03:36:23.660983+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:33:14.339525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.384351+0000","last_change":"2024-09-21T03:36:21.062867+0000","last_active":"2024-09-21T03:36:23.384351+0000","last_peered":"2024-09-21T03:36:23.384351+0000","last_clean":"2024-09-21T03:36:23.384351+0000","last_became_active":"2024-09-21T03:36:21.062555+0000","last_became_peered":"2024-09-21T03:36:21.062555+0000","last_unstale":"2024-09-21T03:36:23.384351+0000","last_undegraded":"2024-09-21T03:36:23.384351+0000","last_fullsized":"2024-09-21T03:36:23.384351+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_clean_scrub_stamp":"2024-09-21T03:36:20.027499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:17:21.290496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.082099+0000","last_change":"2024-09-21T03:36:22.063923+0000","last_active":"2024-09-21T03:36:23.082099+0000","last_peered":"2024-09-21T03:36:23.082099+0000","last_clean":"2024-09-21T03:36:23.082099+0000","last_became_active":"2024-09-21T03:36:22.063195+0000","last_became_peered":"2024-09-21T03:36:22.063195+0000","last_unstale":"2024-09-21T03:36:23.082099+0000","last_undegraded":"2024-09-21T03:36:23.082099+0000","last_fullsized":"2024-09-21T03:36:23.082099+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:39:49.226626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.7","version":"66'2","reported_seq":17,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.661858+0000","last_change":"2024-09-21T03:36:21.068456+0000","last_active":"2024-09-21T03:36:23.661858+0000","last_peered":"2024-09-21T03:36:23.661858+0000","last_clean":"2024-09-21T03:36:23.661858+0000","last_became_active":"2024-09-21T03:36:21.068101+0000","last_became_peered":"2024-09-21T03:36:21.068101+0000","last_unstale":"2024-09-21T03:36:23.661858+0000","last_undegraded":"2024-09-21T03:36:23.661858+0000","last_fullsized":"2024-09-21T03:36:23.661858+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_clean_scrub_stamp":"2024-09-21T03:36:20.027499+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:09:49.802044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.082358+0000","last_change":"2024-09-21T03:36:22.071078+0000","last_active":"2024-09-21T03:36:23.082358+0000","last_peered":"2024-09-21T03:36:23.082358+0000","last_clean":"2024-09-21T03:36:23.082358+0000","last_became_active":"2024-09-21T03:36:22.070566+0000","last_became_peered":"2024-09-21T03:36:22.070566+0000","last_unstale":"2024-09-21T03:36:23.082358+0000","last_undegraded":"2024-09-21T03:36:23.082358+0000","last_fullsized":"2024-09-21T03:36:23.082358+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:05:54.700493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.661366+0000","last_change":"2024-09-21T03:36:21.059377+0000","last_active":"2024-09-21T03:36:23.661366+0000","last_peered":"2024-09-21T03:36:23.661366+0000","last_clean":"2024-09-21T03:36:23.661366+0000","last_became_active":"2024-09-21T03:36:21.059079+0000","last_became_peered":"2024-09-21T03:36:21.059079+0000","last_unstale":"2024-09-21T03:36:23.661366+0000","last_undegraded":"2024-09-21T03:36:23.661366+0000","last_fullsized":"2024-09-21T03:36:23.661366+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_clean_scrub_stamp":"2024-09-21T03:36:20.027499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:39:00.495803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.086122+0000","last_change":"2024-09-21T03:36:22.072209+0000","last_active":"2024-09-21T03:36:23.086122+0000","last_peered":"2024-09-21T03:36:23.086122+0000","last_clean":"2024-09-21T03:36:23.086122+0000","last_became_active":"2024-09-21T03:36:22.071876+0000","last_became_peered":"2024-09-21T03:36:22.071876+0000","last_unstale":"2024-09-21T03:36:23.086122+0000","last_undegraded":"2024-09-21T03:36:23.086122+0000","last_fullsized":"2024-09-21T03:36:23.086122+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:37:07.049180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.384831+0000","last_change":"2024-09-21T03:36:21.063184+0000","last_active":"2024-09-21T03:36:23.384831+0000","last_peered":"2024-09-21T03:36:23.384831+0000","last_clean":"2024-09-21T03:36:23.384831+0000","last_became_active":"2024-09-21T03:36:21.062859+0000","last_became_peered":"2024-09-21T03:36:21.062859+0000","last_unstale":"2024-09-21T03:36:23.384831+0000","last_undegraded":"2024-09-21T03:36:23.384831+0000","last_fullsized":"2024-09-21T03:36:23.384831+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_clean_scrub_stamp":"2024-09-21T03:36:20.027499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:34:09.250333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.086389+0000","last_change":"2024-09-21T03:36:22.069954+0000","last_active":"2024-09-21T03:36:23.086389+0000","last_peered":"2024-09-21T03:36:23.086389+0000","last_clean":"2024-09-21T03:36:23.086389+0000","last_became_active":"2024-09-21T03:36:22.068916+0000","last_became_peered":"2024-09-21T03:36:22.068916+0000","last_unstale":"2024-09-21T03:36:23.086389+0000","last_undegraded":"2024-09-21T03:36:23.086389+0000","last_fullsized":"2024-09-21T03:36:23.086389+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:03:46.925197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.660652+0000","last_change":"2024-09-21T03:36:21.071978+0000","last_active":"2024-09-21T03:36:23.660652+0000","last_peered":"2024-09-21T03:36:23.660652+0000","last_clean":"2024-09-21T03:36:23.660652+0000","last_became_active":"2024-09-21T03:36:21.071717+0000","last_became_peered":"2024-09-21T03:36:21.071717+0000","last_unstale":"2024-09-21T03:36:23.660652+0000","last_undegraded":"2024-09-21T03:36:23.660652+0000","last_fullsized":"2024-09-21T03:36:23.660652+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_clean_scrub_stamp":"2024-09-21T03:36:20.027499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:46:05.982882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.661216+0000","last_change":"2024-09-21T03:36:22.072086+0000","last_active":"2024-09-21T03:36:23.661216+0000","last_peered":"2024-09-21T03:36:23.661216+0000","last_clean":"2024-09-21T03:36:23.661216+0000","last_became_active":"2024-09-21T03:36:22.071729+0000","last_became_peered":"2024-09-21T03:36:22.071729+0000","last_unstale":"2024-09-21T03:36:23.661216+0000","last_undegraded":"2024-09-21T03:36:23.661216+0000","last_fullsized":"2024-09-21T03:36:23.661216+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:30:14.015775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.660720+0000","last_change":"2024-09-21T03:36:21.066336+0000","last_active":"2024-09-21T03:36:23.660720+0000","last_peered":"2024-09-21T03:36:23.660720+0000","last_clean":"2024-09-21T03:36:23.660720+0000","last_became_active":"2024-09-21T03:36:21.066043+0000","last_became_peered":"2024-09-21T03:36:21.066043+0000","last_unstale":"2024-09-21T03:36:23.660720+0000","last_undegraded":"2024-09-21T03:36:23.660720+0000","last_fullsized":"2024-09-21T03:36:23.660720+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_clean_scrub_stamp":"2024-09-21T03:36:20.027499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:45:07.961665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.660391+0000","last_change":"2024-09-21T03:36:22.089730+0000","last_active":"2024-09-21T03:36:23.660391+0000","last_peered":"2024-09-21T03:36:23.660391+0000","last_clean":"2024-09-21T03:36:23.660391+0000","last_became_active":"2024-09-21T03:36:22.089403+0000","last_became_peered":"2024-09-21T03:36:22.089403+0000","last_unstale":"2024-09-21T03:36:23.660391+0000","last_undegraded":"2024-09-21T03:36:23.660391+0000","last_fullsized":"2024-09-21T03:36:23.660391+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:38:24.463402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.159550+0000","last_change":"2024-09-21T03:36:21.066672+0000","last_active":"2024-09-21T03:36:23.159550+0000","last_peered":"2024-09-21T03:36:23.159550+0000","last_clean":"2024-09-21T03:36:23.159550+0000","last_became_active":"2024-09-21T03:36:21.066366+0000","last_became_peered":"2024-09-21T03:36:21.066366+0000","last_unstale":"2024-09-21T03:36:23.159550+0000","last_undegraded":"2024-09-21T03:36:23.159550+0000","last_fullsized":"2024-09-21T03:36:23.159550+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_clean_scrub_stamp":"2024-09-21T03:36:20.027499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:28:15.884066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.384578+0000","last_change":"2024-09-21T03:36:22.078128+0000","last_active":"2024-09-21T03:36:23.384578+0000","last_peered":"2024-09-21T03:36:23.384578+0000","last_clean":"2024-09-21T03:36:23.384578+0000","last_became_active":"2024-09-21T03:36:22.077832+0000","last_became_peered":"2024-09-21T03:36:22.077832+0000","last_unstale":"2024-09-21T03:36:23.384578+0000","last_undegraded":"2024-09-21T03:36:23.384578+0000","last_fullsized":"2024-09-21T03:36:23.384578+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:51:59.256548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.660737+0000","last_change":"2024-09-21T03:36:21.065843+0000","last_active":"2024-09-21T03:36:23.660737+0000","last_peered":"2024-09-21T03:36:23.660737+0000","last_clean":"2024-09-21T03:36:23.660737+0000","last_became_active":"2024-09-21T03:36:21.065466+0000","last_became_peered":"2024-09-21T03:36:21.065466+0000","last_unstale":"2024-09-21T03:36:23.660737+0000","last_undegraded":"2024-09-21T03:36:23.660737+0000","last_fullsized":"2024-09-21T03:36:23.660737+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_clean_scrub_stamp":"2024-09-21T03:36:20.027499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:10:41.577481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.082917+0000","last_change":"2024-09-21T03:36:22.068997+0000","last_active":"2024-09-21T03:36:23.082917+0000","last_peered":"2024-09-21T03:36:23.082917+0000","last_clean":"2024-09-21T03:36:23.082917+0000","last_became_active":"2024-09-21T03:36:22.067174+0000","last_became_peered":"2024-09-21T03:36:22.067174+0000","last_unstale":"2024-09-21T03:36:23.082917+0000","last_undegraded":"2024-09-21T03:36:23.082917+0000","last_fullsized":"2024-09-21T03:36:23.082917+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:52:53.506502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.384419+0000","last_change":"2024-09-21T03:36:21.067567+0000","last_active":"2024-09-21T03:36:23.384419+0000","last_peered":"2024-09-21T03:36:23.384419+0000","last_clean":"2024-09-21T03:36:23.384419+0000","last_became_active":"2024-09-21T03:36:21.067235+0000","last_became_peered":"2024-09-21T03:36:21.067235+0000","last_unstale":"2024-09-21T03:36:23.384419+0000","last_undegraded":"2024-09-21T03:36:23.384419+0000","last_fullsized":"2024-09-21T03:36:23.384419+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_clean_scrub_stamp":"2024-09-21T03:36:20.027499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:14:15.007878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.085908+0000","last_change":"2024-09-21T03:36:22.083766+0000","last_active":"2024-09-21T03:36:23.085908+0000","last_peered":"2024-09-21T03:36:23.085908+0000","last_clean":"2024-09-21T03:36:23.085908+0000","last_became_active":"2024-09-21T03:36:22.083362+0000","last_became_peered":"2024-09-21T03:36:22.083362+0000","last_unstale":"2024-09-21T03:36:23.085908+0000","last_undegraded":"2024-09-21T03:36:23.085908+0000","last_fullsized":"2024-09-21T03:36:23.085908+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:21.036908+0000","last_clean_scrub_stamp":"2024-09-21T03:36:21.036908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:52:59.833307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.384570+0000","last_change":"2024-09-21T03:36:21.059196+0000","last_active":"2024-09-21T03:36:23.384570+0000","last_peered":"2024-09-21T03:36:23.384570+0000","last_clean":"2024-09-21T03:36:23.384570+0000","last_became_active":"2024-09-21T03:36:21.058881+0000","last_became_peered":"2024-09-21T03:36:21.058881+0000","last_unstale":"2024-09-21T03:36:23.384570+0000","last_undegraded":"2024-09-21T03:36:23.384570+0000","last_fullsized":"2024-09-21T03:36:23.384570+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:36:20.027499+0000","last_clean_scrub_stamp":"2024-09-21T03:36:20.027499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:16:36.186560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":198,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T03:36:23.085071+0000","last_change":"2024-09-21T03:33:25.319776+0000","last_active":"2024-09-21T03:36:23.085071+0000","last_peered":"2024-09-21T03:36:23.085071+0000","last_clean":"2024-09-21T03:36:23.085071+0000","last_became_active":"2024-09-21T03:33:25.318399+0000","last_became_peered":"2024-09-21T03:33:25.318399+0000","last_unstale":"2024-09-21T03:36:23.085071+0000","last_undegraded":"2024-09-21T03:36:23.085071+0000","last_fullsized":"2024-09-21T03:36:23.085071+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:23.016672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:23.016672+0000","last_clean_scrub_stamp":"2024-09-21T03:33:23.016672+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:32:35.906114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":1,"up_from":16,"seq":68719476776,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32564,"kb_used_data":1368,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339276,"statfs":{"total":96636764160,"available":96603418624,"internally_reserved":0,"allocated":1400832,"data_stored":954724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509480,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36124,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335716,"statfs":{"total":96636764160,"available":96599773184,"internally_reserved":0,"allocated":790528,"data_stored":357630,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":36174283},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509479,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31980,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339860,"statfs":{"total":96636764160,"available":96604016640,"internally_reserved":0,"allocated":811008,"data_stored":364482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509480,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31980,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339860,"statfs":{"total":96636764160,"available":96604016640,"internally_reserved":0,"allocated":811008,"data_stored":364378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509479,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31968,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339872,"statfs":{"total":96636764160,"available":96604028928,"internally_reserved":0,"allocated":790528,"data_stored":357630,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542184,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36152,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335688,"statfs":{"total":96636764160,"available":96599744512,"internally_reserved":0,"allocated":823296,"data_stored":366586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574888,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36736,"kb_used_data":1384,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335104,"statfs":{"total":96636764160,"available":96599146496,"internally_reserved":0,"allocated":1417216,"data_stored":956414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542185,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36136,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335704,"statfs":{"total":96636764160,"available":96599760896,"internally_reserved":0,"allocated":806912,"data_stored":364904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T03:36:31.027 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-21T03:36:31.028 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-21T03:36:31.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:31.187+0000 7f393dcdd640 1 -- 172.21.15.38:0/1639931349 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f39380b2e40 msgr2=0x7f39380b3250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:31.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:31.187+0000 7f393dcdd640 1 --2- 172.21.15.38:0/1639931349 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f39380b2e40 0x7f39380b3250 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f39380b7460 tx=0x7f393002a820 comp rx=0 tx=0).stop 2024-09-21T03:36:31.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:31.187+0000 7f393dcdd640 1 -- 172.21.15.38:0/1639931349 shutdown_connections 2024-09-21T03:36:31.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:31.187+0000 7f393dcdd640 1 --2- 172.21.15.38:0/1639931349 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f39380b2e40 0x7f39380b3250 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:31.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:31.187+0000 7f393dcdd640 1 --2- 172.21.15.38:0/1639931349 >> v2:172.21.15.195:3300/0 conn(0x7f3938162280 0x7f3938162690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:31.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:31.187+0000 7f393dcdd640 1 -- 172.21.15.38:0/1639931349 >> 172.21.15.38:0/1639931349 conn(0x7f39380b6b20 msgr2=0x7f39380d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:31.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:31.187+0000 7f393dcdd640 1 -- 172.21.15.38:0/1639931349 shutdown_connections 2024-09-21T03:36:31.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:31.187+0000 7f393dcdd640 1 -- 172.21.15.38:0/1639931349 wait complete. 2024-09-21T03:36:31.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:31.189+0000 7f393dcdd640 1 Processor -- start 2024-09-21T03:36:31.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:31.189+0000 7f393dcdd640 1 -- start start 2024-09-21T03:36:31.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:31.190+0000 7f393dcdd640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f39380b2e40 0x7f393814e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:31.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:31.190+0000 7f393dcdd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3938162280 0x7f393814e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:31.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:31.190+0000 7f393dcdd640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f393814ead0 0x7f393814f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:31.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:31.190+0000 7f393dcdd640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f393815e4b0 con 0x7f3938162280 2024-09-21T03:36:31.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:31.190+0000 7f393dcdd640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f39380b39a0 con 0x7f393814ead0 2024-09-21T03:36:31.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:31.190+0000 7f393dcdd640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f393815e630 con 0x7f39380b2e40 2024-09-21T03:36:31.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:31.191+0000 7f392effd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3938162280 0x7f393814e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:31.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:31.191+0000 7f392effd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3938162280 0x7f393814e590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37390/0 (socket says 172.21.15.38:37390) 2024-09-21T03:36:31.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:31.191+0000 7f392effd640 1 -- 172.21.15.38:0/3946474241 learned_addr learned my addr 172.21.15.38:0/3946474241 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:36:31.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:31.191+0000 7f3937fff640 1 --2- 172.21.15.38:0/3946474241 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f393814ead0 0x7f393814f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:31.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:31.191+0000 7f39377fe640 1 --2- 172.21.15.38:0/3946474241 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f39380b2e40 0x7f393814e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:31.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:31.191+0000 7f392effd640 1 -- 172.21.15.38:0/3946474241 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f39380b2e40 msgr2=0x7f393814e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:31.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:31.191+0000 7f392effd640 1 --2- 172.21.15.38:0/3946474241 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f39380b2e40 0x7f393814e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:31.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:31.191+0000 7f392effd640 1 -- 172.21.15.38:0/3946474241 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f393814ead0 msgr2=0x7f393814f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:31.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:31.191+0000 7f392effd640 1 --2- 172.21.15.38:0/3946474241 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f393814ead0 0x7f393814f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:31.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:31.191+0000 7f392effd640 1 -- 172.21.15.38:0/3946474241 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f393814eee0 con 0x7f3938162280 2024-09-21T03:36:31.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:31.191+0000 7f39377fe640 1 --2- 172.21.15.38:0/3946474241 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f39380b2e40 0x7f393814e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:36:31.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:31.191+0000 7f392effd640 1 --2- 172.21.15.38:0/3946474241 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3938162280 0x7f393814e590 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f392000c9a0 tx=0x7f392000ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:31.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:31.191+0000 7f3934ff9640 1 -- 172.21.15.38:0/3946474241 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3920007bb0 con 0x7f3938162280 2024-09-21T03:36:31.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:31.191+0000 7f3934ff9640 1 -- 172.21.15.38:0/3946474241 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3920007e40 con 0x7f3938162280 2024-09-21T03:36:31.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:31.191+0000 7f393dcdd640 1 -- 172.21.15.38:0/3946474241 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f393815c9c0 con 0x7f3938162280 2024-09-21T03:36:31.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:31.191+0000 7f3934ff9640 1 -- 172.21.15.38:0/3946474241 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3920005280 con 0x7f3938162280 2024-09-21T03:36:31.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:31.191+0000 7f393dcdd640 1 -- 172.21.15.38:0/3946474241 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39380adc90 con 0x7f3938162280 2024-09-21T03:36:31.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:31.192+0000 7f393dcdd640 1 -- 172.21.15.38:0/3946474241 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39380ad720 con 0x7f3938162280 2024-09-21T03:36:31.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:31.193+0000 7f3934ff9640 1 -- 172.21.15.38:0/3946474241 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f3920005420 con 0x7f3938162280 2024-09-21T03:36:31.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:31.193+0000 7f3934ff9640 1 --2- 172.21.15.38:0/3946474241 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3918078b90 0x7f391807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:31.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:31.194+0000 7f3934ff9640 1 -- 172.21.15.38:0/3946474241 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6101+0+0 (secure 0 0 0) 0x7f392009b1b0 con 0x7f3938162280 2024-09-21T03:36:31.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:31.194+0000 7f39377fe640 1 --2- 172.21.15.38:0/3946474241 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3918078b90 0x7f391807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:31.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:31.194+0000 7f39377fe640 1 --2- 172.21.15.38:0/3946474241 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3918078b90 0x7f391807b0a0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f3928000f60 tx=0x7f3928009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:31.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:31.198+0000 7f3934ff9640 1 -- 172.21.15.38:0/3946474241 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f3920014030 con 0x7f3938162280 2024-09-21T03:36:31.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:31.459+0000 7f393dcdd640 1 -- 172.21.15.38:0/3946474241 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f39380b3580 con 0x7f3938162280 2024-09-21T03:36:31.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:31.462+0000 7f3934ff9640 1 -- 172.21.15.38:0/3946474241 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v20) ==== 190+0+147 (secure 0 0 0) 0x7f39380b3580 con 0x7f3938162280 2024-09-21T03:36:31.464 INFO:teuthology.orchestra.run.smithi038.stdout:[client.0] 2024-09-21T03:36:31.464 INFO:teuthology.orchestra.run.smithi038.stdout: key = AQBQPu5m6ydBDxAAlWZ2XzQTPBcYPriPGRJVeA== 2024-09-21T03:36:31.464 INFO:teuthology.orchestra.run.smithi038.stdout: caps mds = "allow" 2024-09-21T03:36:31.464 INFO:teuthology.orchestra.run.smithi038.stdout: caps mon = "allow r" 2024-09-21T03:36:31.465 INFO:teuthology.orchestra.run.smithi038.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-21T03:36:31.465 INFO:teuthology.orchestra.run.smithi038.stderr:updated caps for client.0 2024-09-21T03:36:31.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:31.465+0000 7f393dcdd640 1 -- 172.21.15.38:0/3946474241 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3918078b90 msgr2=0x7f391807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:31.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:31.465+0000 7f393dcdd640 1 --2- 172.21.15.38:0/3946474241 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3918078b90 0x7f391807b0a0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f3928000f60 tx=0x7f3928009040 comp rx=0 tx=0).stop 2024-09-21T03:36:31.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:31.465+0000 7f393dcdd640 1 -- 172.21.15.38:0/3946474241 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3938162280 msgr2=0x7f393814e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:31.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:31.465+0000 7f393dcdd640 1 --2- 172.21.15.38:0/3946474241 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3938162280 0x7f393814e590 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f392000c9a0 tx=0x7f392000ce60 comp rx=0 tx=0).stop 2024-09-21T03:36:31.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:31.465+0000 7f393dcdd640 1 -- 172.21.15.38:0/3946474241 shutdown_connections 2024-09-21T03:36:31.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:31.465+0000 7f393dcdd640 1 --2- 172.21.15.38:0/3946474241 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f393814ead0 0x7f393814f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:31.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:31.465+0000 7f393dcdd640 1 --2- 172.21.15.38:0/3946474241 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3938162280 0x7f393814e590 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:31.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:31.465+0000 7f393dcdd640 1 --2- 172.21.15.38:0/3946474241 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3918078b90 0x7f391807b0a0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:31.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:31.465+0000 7f393dcdd640 1 --2- 172.21.15.38:0/3946474241 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f39380b2e40 0x7f393814e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:31.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:31.465+0000 7f393dcdd640 1 -- 172.21.15.38:0/3946474241 >> 172.21.15.38:0/3946474241 conn(0x7f39380b6b20 msgr2=0x7f393815eac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:31.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:31.465+0000 7f393dcdd640 1 -- 172.21.15.38:0/3946474241 shutdown_connections 2024-09-21T03:36:31.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:31.465+0000 7f393dcdd640 1 -- 172.21.15.38:0/3946474241 wait complete. 2024-09-21T03:36:31.478 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:36:31.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:31.593+0000 7f053f5ba640 1 -- 172.21.15.38:0/8520523 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0538162f00 msgr2=0x7f05380d24d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:31.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:31.593+0000 7f053f5ba640 1 --2- 172.21.15.38:0/8520523 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0538162f00 0x7f05380d24d0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f0534005fd0 tx=0x7f053402a530 comp rx=0 tx=0).stop 2024-09-21T03:36:31.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:31.593+0000 7f053f5ba640 1 -- 172.21.15.38:0/8520523 shutdown_connections 2024-09-21T03:36:31.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:31.593+0000 7f053f5ba640 1 --2- 172.21.15.38:0/8520523 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0538162280 0x7f0538162690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:31.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:31.593+0000 7f053f5ba640 1 --2- 172.21.15.38:0/8520523 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0538162f00 0x7f05380d24d0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:31.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:31.593+0000 7f053f5ba640 1 -- 172.21.15.38:0/8520523 >> 172.21.15.38:0/8520523 conn(0x7f05380b6b20 msgr2=0x7f05380d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:31.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:31.593+0000 7f053f5ba640 1 -- 172.21.15.38:0/8520523 shutdown_connections 2024-09-21T03:36:31.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:31.593+0000 7f053f5ba640 1 -- 172.21.15.38:0/8520523 wait complete. 2024-09-21T03:36:31.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:31.594+0000 7f053f5ba640 1 Processor -- start 2024-09-21T03:36:31.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:31.595+0000 7f053f5ba640 1 -- start start 2024-09-21T03:36:31.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:31.596+0000 7f053f5ba640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f05380b2e40 0x7f05380b2700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:31.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:31.596+0000 7f053f5ba640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0538162280 0x7f05380ce940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:31.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:31.596+0000 7f053f5ba640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f05381f6310 0x7f05380ab720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:31.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:31.596+0000 7f053f5ba640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f053815e630 con 0x7f05380b2e40 2024-09-21T03:36:31.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:31.596+0000 7f053f5ba640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f053815e4b0 con 0x7f0538162280 2024-09-21T03:36:31.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:31.596+0000 7f053f5ba640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f053815e7b0 con 0x7f05381f6310 2024-09-21T03:36:31.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:31.596+0000 7f053d356640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f05380b2e40 0x7f05380b2700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:31.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:31.596+0000 7f053d356640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f05380b2e40 0x7f05380b2700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37410/0 (socket says 172.21.15.38:37410) 2024-09-21T03:36:31.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:31.596+0000 7f053d356640 1 -- 172.21.15.38:0/3658917006 learned_addr learned my addr 172.21.15.38:0/3658917006 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:36:31.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:31.596+0000 7f053cb55640 1 --2- 172.21.15.38:0/3658917006 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0538162280 0x7f05380ce940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:31.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:31.596+0000 7f053db57640 1 --2- 172.21.15.38:0/3658917006 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f05381f6310 0x7f05380ab720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:31.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:31.596+0000 7f053d356640 1 -- 172.21.15.38:0/3658917006 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f05381f6310 msgr2=0x7f05380ab720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:31.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:31.596+0000 7f053d356640 1 --2- 172.21.15.38:0/3658917006 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f05381f6310 0x7f05380ab720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:31.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:31.596+0000 7f053d356640 1 -- 172.21.15.38:0/3658917006 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0538162280 msgr2=0x7f05380ce940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:31.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:31.596+0000 7f053d356640 1 --2- 172.21.15.38:0/3658917006 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0538162280 0x7f05380ce940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:31.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:31.596+0000 7f053d356640 1 -- 172.21.15.38:0/3658917006 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05380b2c40 con 0x7f05380b2e40 2024-09-21T03:36:31.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:31.596+0000 7f053d356640 1 --2- 172.21.15.38:0/3658917006 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f05380b2e40 0x7f05380b2700 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f0534034040 tx=0x7f0534003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:31.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:31.597+0000 7f052dffb640 1 -- 172.21.15.38:0/3658917006 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f053403e030 con 0x7f05380b2e40 2024-09-21T03:36:31.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:31.597+0000 7f052dffb640 1 -- 172.21.15.38:0/3658917006 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f05340047a0 con 0x7f05380b2e40 2024-09-21T03:36:31.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:31.597+0000 7f053f5ba640 1 -- 172.21.15.38:0/3658917006 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05380b1660 con 0x7f05380b2e40 2024-09-21T03:36:31.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:31.597+0000 7f052dffb640 1 -- 172.21.15.38:0/3658917006 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0534035040 con 0x7f05380b2e40 2024-09-21T03:36:31.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:31.597+0000 7f053f5ba640 1 -- 172.21.15.38:0/3658917006 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05380b1c20 con 0x7f05380b2e40 2024-09-21T03:36:31.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:31.598+0000 7f053f5ba640 1 -- 172.21.15.38:0/3658917006 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0538162f00 con 0x7f05380b2e40 2024-09-21T03:36:31.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:31.599+0000 7f052dffb640 1 -- 172.21.15.38:0/3658917006 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f0534041030 con 0x7f05380b2e40 2024-09-21T03:36:31.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:31.599+0000 7f052dffb640 1 --2- 172.21.15.38:0/3658917006 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f051c078b90 0x7f051c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:31.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:31.599+0000 7f052dffb640 1 -- 172.21.15.38:0/3658917006 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6101+0+0 (secure 0 0 0) 0x7f05340bb7a0 con 0x7f05380b2e40 2024-09-21T03:36:31.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:31.600+0000 7f053cb55640 1 --2- 172.21.15.38:0/3658917006 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f051c078b90 0x7f051c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:31.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:31.600+0000 7f053cb55640 1 --2- 172.21.15.38:0/3658917006 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f051c078b90 0x7f051c07b0a0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f05381f9800 tx=0x7f0528008040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:31.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:31.604+0000 7f052dffb640 1 -- 172.21.15.38:0/3658917006 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f0538162bd0 con 0x7f05380b2e40 2024-09-21T03:36:31.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:31.834+0000 7f053f5ba640 1 -- 172.21.15.38:0/3658917006 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f05380b9170 con 0x7f05380b2e40 2024-09-21T03:36:31.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:31.835+0000 7f052dffb640 1 -- 172.21.15.38:0/3658917006 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) ==== 94+0+5230 (secure 0 0 0) 0x7f053408c020 con 0x7f05380b2e40 2024-09-21T03:36:31.838 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:36:31.838 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":45,"btime":"2024-09-21T03:36:25:092622+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5432,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/1042764291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":1042764291},{"type":"v1","addr":"172.21.15.195:6837","nonce":1042764291}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":5728,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/3035019482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6833","nonce":3035019482},{"type":"v1","addr":"172.21.15.38:6835","nonce":3035019482}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":5731,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/2536177898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":2536177898},{"type":"v1","addr":"172.21.15.195:6836","nonce":2536177898}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":45,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:36:21.442724+0000","modified":"2024-09-21T03:36:24.474620+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5991},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5991":{"gid":5991,"name":"a","rank":0,"incarnation":41,"state":"up:active","state_seq":3,"addr":"172.21.15.38:6834/1353335665","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":1353335665},{"type":"v1","addr":"172.21.15.38:6834","nonce":1353335665}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5991,"qdb_cluster":[5991]},"id":4}]} 2024-09-21T03:36:31.839 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 45 2024-09-21T03:36:31.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:31.837+0000 7f053f5ba640 1 -- 172.21.15.38:0/3658917006 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f051c078b90 msgr2=0x7f051c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:31.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:31.837+0000 7f053f5ba640 1 --2- 172.21.15.38:0/3658917006 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f051c078b90 0x7f051c07b0a0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f05381f9800 tx=0x7f0528008040 comp rx=0 tx=0).stop 2024-09-21T03:36:31.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:31.837+0000 7f053f5ba640 1 -- 172.21.15.38:0/3658917006 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f05380b2e40 msgr2=0x7f05380b2700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:31.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:31.837+0000 7f053f5ba640 1 --2- 172.21.15.38:0/3658917006 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f05380b2e40 0x7f05380b2700 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f0534034040 tx=0x7f0534003970 comp rx=0 tx=0).stop 2024-09-21T03:36:31.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:31.837+0000 7f053f5ba640 1 -- 172.21.15.38:0/3658917006 shutdown_connections 2024-09-21T03:36:31.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:31.837+0000 7f053f5ba640 1 --2- 172.21.15.38:0/3658917006 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f051c078b90 0x7f051c07b0a0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:31.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:31.837+0000 7f053f5ba640 1 --2- 172.21.15.38:0/3658917006 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f05381f6310 0x7f05380ab720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:31.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:31.837+0000 7f053f5ba640 1 --2- 172.21.15.38:0/3658917006 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0538162280 0x7f05380ce940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:31.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:31.837+0000 7f053f5ba640 1 --2- 172.21.15.38:0/3658917006 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f05380b2e40 0x7f05380b2700 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:31.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:31.838+0000 7f053f5ba640 1 -- 172.21.15.38:0/3658917006 >> 172.21.15.38:0/3658917006 conn(0x7f05380b6b20 msgr2=0x7f053815dbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:31.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:31.838+0000 7f053f5ba640 1 -- 172.21.15.38:0/3658917006 shutdown_connections 2024-09-21T03:36:31.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:31.838+0000 7f053f5ba640 1 -- 172.21.15.38:0/3658917006 wait complete. 2024-09-21T03:36:32.254 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 45, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T03:36:21.442724+0000', 'modified': '2024-09-21T03:36:24.474620+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5991}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5991': {'gid': 5991, 'name': 'a', 'rank': 0, 'incarnation': 41, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.38:6834/1353335665', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.38:6832', 'nonce': 1353335665}, {'type': 'v1', 'addr': '172.21.15.38:6834', 'nonce': 1353335665}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5991, 'qdb_cluster': [5991]} 2024-09-21T03:36:32.254 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-21T03:36:32.254 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-21T03:36:32.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.375+0000 7fd6ed294640 1 -- 172.21.15.38:0/3050414987 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd6e806def0 msgr2=0x7fd6e806f410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:32.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.375+0000 7fd6ed294640 1 --2- 172.21.15.38:0/3050414987 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd6e806def0 0x7fd6e806f410 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fd6d8005fd0 tx=0x7fd6d802a940 comp rx=0 tx=0).stop 2024-09-21T03:36:32.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.375+0000 7fd6ed294640 1 -- 172.21.15.38:0/3050414987 shutdown_connections 2024-09-21T03:36:32.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.375+0000 7fd6ed294640 1 --2- 172.21.15.38:0/3050414987 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd6e806def0 0x7fd6e806f410 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:32.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.375+0000 7fd6ed294640 1 -- 172.21.15.38:0/3050414987 >> 172.21.15.38:0/3050414987 conn(0x7fd6e80cdca0 msgr2=0x7fd6e80ce0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:32.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.375+0000 7fd6ed294640 1 -- 172.21.15.38:0/3050414987 shutdown_connections 2024-09-21T03:36:32.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.375+0000 7fd6ed294640 1 -- 172.21.15.38:0/3050414987 wait complete. 2024-09-21T03:36:32.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.377+0000 7fd6ed294640 1 Processor -- start 2024-09-21T03:36:32.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.377+0000 7fd6ed294640 1 -- start start 2024-09-21T03:36:32.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.378+0000 7fd6ed294640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd6e806def0 0x7fd6e81f71a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:32.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.378+0000 7fd6ed294640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd6e80b5260 0x7fd6e8155220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:32.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.378+0000 7fd6ed294640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd6e80b5670 0x7fd6e806cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:32.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.378+0000 7fd6ed294640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fd6e8071480 con 0x7fd6e80b5260 2024-09-21T03:36:32.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.378+0000 7fd6ed294640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd6e8071300 con 0x7fd6e806def0 2024-09-21T03:36:32.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.378+0000 7fd6ed294640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fd6e8071600 con 0x7fd6e80b5670 2024-09-21T03:36:32.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.378+0000 7fd6e67fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd6e80b5260 0x7fd6e8155220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:32.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.378+0000 7fd6e67fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd6e80b5260 0x7fd6e8155220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40088/0 (socket says 172.21.15.38:40088) 2024-09-21T03:36:32.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.378+0000 7fd6e67fc640 1 -- 172.21.15.38:0/1747787826 learned_addr learned my addr 172.21.15.38:0/1747787826 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:36:32.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.378+0000 7fd6e6ffd640 1 --2- 172.21.15.38:0/1747787826 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd6e806def0 0x7fd6e81f71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:32.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.379+0000 7fd6e77fe640 1 --2- 172.21.15.38:0/1747787826 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd6e80b5670 0x7fd6e806cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:32.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.379+0000 7fd6e67fc640 1 -- 172.21.15.38:0/1747787826 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd6e80b5670 msgr2=0x7fd6e806cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:32.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.379+0000 7fd6e67fc640 1 --2- 172.21.15.38:0/1747787826 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd6e80b5670 0x7fd6e806cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:32.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.379+0000 7fd6e67fc640 1 -- 172.21.15.38:0/1747787826 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd6e806def0 msgr2=0x7fd6e81f71a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:32.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.379+0000 7fd6e67fc640 1 --2- 172.21.15.38:0/1747787826 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd6e806def0 0x7fd6e81f71a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:32.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.379+0000 7fd6e67fc640 1 -- 172.21.15.38:0/1747787826 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6e809b570 con 0x7fd6e80b5260 2024-09-21T03:36:32.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.379+0000 7fd6e67fc640 1 --2- 172.21.15.38:0/1747787826 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd6e80b5260 0x7fd6e8155220 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fd6d000c400 tx=0x7fd6d000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:32.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.379+0000 7fd6df7fe640 1 -- 172.21.15.38:0/1747787826 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6d0017070 con 0x7fd6e80b5260 2024-09-21T03:36:32.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.379+0000 7fd6df7fe640 1 -- 172.21.15.38:0/1747787826 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd6d00053c0 con 0x7fd6e80b5260 2024-09-21T03:36:32.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.379+0000 7fd6ed294640 1 -- 172.21.15.38:0/1747787826 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6e809b850 con 0x7fd6e80b5260 2024-09-21T03:36:32.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.379+0000 7fd6df7fe640 1 -- 172.21.15.38:0/1747787826 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6d0003830 con 0x7fd6e80b5260 2024-09-21T03:36:32.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.379+0000 7fd6ed294640 1 -- 172.21.15.38:0/1747787826 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6e809bb30 con 0x7fd6e80b5260 2024-09-21T03:36:32.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.381+0000 7fd6df7fe640 1 -- 172.21.15.38:0/1747787826 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fd6d001a070 con 0x7fd6e80b5260 2024-09-21T03:36:32.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.382+0000 7fd6df7fe640 1 --2- 172.21.15.38:0/1747787826 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd6c8078b90 0x7fd6c807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:32.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.382+0000 7fd6df7fe640 1 -- 172.21.15.38:0/1747787826 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6101+0+0 (secure 0 0 0) 0x7fd6d009a130 con 0x7fd6e80b5260 2024-09-21T03:36:32.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.382+0000 7fd6e6ffd640 1 --2- 172.21.15.38:0/1747787826 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd6c8078b90 0x7fd6c807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:32.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.383+0000 7fd6f4205740 1 Processor -- start 2024-09-21T03:36:32.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.383+0000 7fd6e6ffd640 1 --2- 172.21.15.38:0/1747787826 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd6c8078b90 0x7fd6c807b0a0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fd6d8034000 tx=0x7fd6d8004ab0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:32.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.383+0000 7fd6f4205740 1 -- start start 2024-09-21T03:36:32.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.384+0000 7fd6f4205740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55c467bb4270 0x55c467f85fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:32.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.384+0000 7fd6f4205740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c467b8b3b0 0x55c467f86500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:32.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.384+0000 7fd6f4205740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c467b93590 0x55c467fa3400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:32.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.384+0000 7fd6f4205740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55c467ece820 con 0x55c467b93590 2024-09-21T03:36:32.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.384+0000 7fd6f4205740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55c467ebdc50 con 0x55c467b8b3b0 2024-09-21T03:36:32.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.384+0000 7fd6f4205740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55c467ec1a60 con 0x55c467bb4270 2024-09-21T03:36:32.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.384+0000 7fd6e77fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c467b93590 0x55c467fa3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:32.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.384+0000 7fd6e77fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c467b93590 0x55c467fa3400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40090/0 (socket says 172.21.15.38:40090) 2024-09-21T03:36:32.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.384+0000 7fd6e77fe640 1 -- 172.21.15.38:0/3995897320 learned_addr learned my addr 172.21.15.38:0/3995897320 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:36:32.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.384+0000 7fd6e67fc640 1 --2- 172.21.15.38:0/3995897320 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c467b8b3b0 0x55c467f86500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:32.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.384+0000 7fd6e6ffd640 1 --2- 172.21.15.38:0/3995897320 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55c467bb4270 0x55c467f85fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:32.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.385+0000 7fd6e77fe640 1 -- 172.21.15.38:0/3995897320 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55c467bb4270 msgr2=0x55c467f85fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:36:32.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.385+0000 7fd6e77fe640 1 --2- 172.21.15.38:0/3995897320 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55c467bb4270 0x55c467f85fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:32.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.385+0000 7fd6e77fe640 1 -- 172.21.15.38:0/3995897320 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c467b8b3b0 msgr2=0x55c467f86500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:36:32.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.385+0000 7fd6e77fe640 1 --2- 172.21.15.38:0/3995897320 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c467b8b3b0 0x55c467f86500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:32.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.385+0000 7fd6e77fe640 1 -- 172.21.15.38:0/3995897320 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c467c1f850 con 0x55c467b93590 2024-09-21T03:36:32.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.385+0000 7fd6e77fe640 1 --2- 172.21.15.38:0/3995897320 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c467b93590 0x55c467fa3400 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fd6e00016b0 tx=0x7fd6e0001b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9910f754ed7a661f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:32.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.385+0000 7fd6dcff9640 1 -- 172.21.15.38:0/3995897320 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6e00052e0 con 0x55c467b93590 2024-09-21T03:36:32.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.385+0000 7fd6dcff9640 1 -- 172.21.15.38:0/3995897320 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd6e0018070 con 0x55c467b93590 2024-09-21T03:36:32.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.385+0000 7fd6dcff9640 1 -- 172.21.15.38:0/3995897320 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6e000fa10 con 0x55c467b93590 2024-09-21T03:36:32.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.386+0000 7fd6f4205740 1 -- 172.21.15.38:0/3995897320 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c467b93590 msgr2=0x55c467fa3400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:32.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.386+0000 7fd6f4205740 1 --2- 172.21.15.38:0/3995897320 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c467b93590 0x55c467fa3400 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fd6e00016b0 tx=0x7fd6e0001b70 comp rx=0 tx=0).stop 2024-09-21T03:36:32.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.386+0000 7fd6f4205740 1 -- 172.21.15.38:0/3995897320 shutdown_connections 2024-09-21T03:36:32.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.386+0000 7fd6f4205740 1 --2- 172.21.15.38:0/3995897320 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c467b93590 0x55c467fa3400 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:32.390 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.386+0000 7fd6f4205740 1 --2- 172.21.15.38:0/3995897320 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c467b8b3b0 0x55c467f86500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:32.390 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.386+0000 7fd6f4205740 1 --2- 172.21.15.38:0/3995897320 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55c467bb4270 0x55c467f85fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:32.390 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.386+0000 7fd6f4205740 1 -- 172.21.15.38:0/3995897320 >> 172.21.15.38:0/3995897320 conn(0x55c467ba83c0 msgr2=0x55c467f5c280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:32.390 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.386+0000 7fd6f4205740 1 -- 172.21.15.38:0/3995897320 shutdown_connections 2024-09-21T03:36:32.390 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.386+0000 7fd6f4205740 1 -- 172.21.15.38:0/3995897320 wait complete. 2024-09-21T03:36:32.390 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.387+0000 7fd6f4205740 1 -- start start 2024-09-21T03:36:32.390 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.387+0000 7fd6f4205740 1 Processor -- start 2024-09-21T03:36:32.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.388+0000 7fd6f4205740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c467b8b3b0 0x55c467f99620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:32.394 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.388+0000 7fd6f4205740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c467b93590 0x55c467f97bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:32.394 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.388+0000 7fd6f4205740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55c467bb4270 0x55c467f74400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:32.394 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.388+0000 7fd6f4205740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55c467c2e620 con 0x55c467b8b3b0 2024-09-21T03:36:32.394 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.388+0000 7fd6f4205740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55c467e635f0 con 0x55c467b93590 2024-09-21T03:36:32.394 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.388+0000 7fd6f4205740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55c467f86a40 con 0x55c467bb4270 2024-09-21T03:36:32.394 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.389+0000 7fd6f4205740 10 client.0 _ll_register_callbacks cb 0x55c467e670a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:36:32.394 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.389+0000 7fd6e6ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c467b8b3b0 0x55c467f99620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:32.394 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.389+0000 7fd6e6ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c467b8b3b0 0x55c467f99620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40100/0 (socket says 172.21.15.38:40100) 2024-09-21T03:36:32.394 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.389+0000 7fd6e6ffd640 1 -- 172.21.15.38:0/1342630931 learned_addr learned my addr 172.21.15.38:0/1342630931 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:36:32.394 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.389+0000 7fd6e67fc640 1 --2- 172.21.15.38:0/1342630931 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c467b93590 0x55c467f97bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:32.394 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.389+0000 7fd6e77fe640 1 --2- 172.21.15.38:0/1342630931 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55c467bb4270 0x55c467f74400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:32.394 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.389+0000 7fd6e6ffd640 1 -- 172.21.15.38:0/1342630931 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55c467bb4270 msgr2=0x55c467f74400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:36:32.394 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.389+0000 7fd6e6ffd640 1 --2- 172.21.15.38:0/1342630931 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55c467bb4270 0x55c467f74400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:32.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.389+0000 7fd6e6ffd640 1 -- 172.21.15.38:0/1342630931 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c467b93590 msgr2=0x55c467f97bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:36:32.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.389+0000 7fd6e6ffd640 1 --2- 172.21.15.38:0/1342630931 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c467b93590 0x55c467f97bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:32.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.389+0000 7fd6e6ffd640 1 -- 172.21.15.38:0/1342630931 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c467c966b0 con 0x55c467b8b3b0 2024-09-21T03:36:32.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.389+0000 7fd6e6ffd640 1 --2- 172.21.15.38:0/1342630931 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c467b8b3b0 0x55c467f99620 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fd6d802af30 tx=0x7fd6d8009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3adda3e6eef15574 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:32.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.389+0000 7fd6c5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-21T03:36:32.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.389+0000 7fd6c5ffb640 1 -- 172.21.15.38:0/1342630931 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6d803b650 con 0x55c467b8b3b0 2024-09-21T03:36:32.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.389+0000 7fd6c5ffb640 1 -- 172.21.15.38:0/1342630931 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(67..67 src has 1..67) ==== 6101+0+0 (secure 0 0 0) 0x7fd6d803b7f0 con 0x55c467b8b3b0 2024-09-21T03:36:32.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.389+0000 7fd6f4205740 1 -- 172.21.15.38:0/1342630931 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd6e0018070 con 0x55c467b8b3b0 2024-09-21T03:36:32.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.390+0000 7fd6c5ffb640 1 -- 172.21.15.38:0/1342630931 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fd6d803b9f0 con 0x55c467b8b3b0 2024-09-21T03:36:32.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.390+0000 7fd6f4205740 10 client.6135 fetch_fsmap learned FSMap version 45 2024-09-21T03:36:32.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.390+0000 7fd6f4205740 1 -- 172.21.15.38:0/1342630931 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=2+}) -- 0x55c467f99bf0 con 0x55c467b8b3b0 2024-09-21T03:36:32.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.390+0000 7fd6c5ffb640 1 -- 172.21.15.38:0/1342630931 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd6e0018070 con 0x55c467b8b3b0 2024-09-21T03:36:32.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.390+0000 7fd6c5ffb640 1 -- 172.21.15.38:0/1342630931 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 45) ==== 3502+0+0 (secure 0 0 0) 0x7fd6d804e020 con 0x55c467b8b3b0 2024-09-21T03:36:32.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.390+0000 7fd6f4205740 10 client.6135 fetch_fsmap finished waiting for FSMap version 45 2024-09-21T03:36:32.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.390+0000 7fd6f4205740 10 client.6135 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-21T03:36:32.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.390+0000 7fd6f4205740 20 client.6135 populate_metadata read hostname 'smithi038' 2024-09-21T03:36:32.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.390+0000 7fd6f4205740 1 --2- 172.21.15.38:0/1342630931 >> [v2:172.21.15.38:6832/1353335665,v1:172.21.15.38:6834/1353335665] conn(0x55c467b72e40 0x55c467f99eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:32.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.390+0000 7fd6f4205740 4 client.6135 mds_command: new command op to 5991 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:36:32.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.390+0000 7fd6f4205740 1 -- 172.21.15.38:0/1342630931 --> [v2:172.21.15.38:6832/1353335665,v1:172.21.15.38:6834/1353335665] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c467ed0770 con 0x55c467b72e40 2024-09-21T03:36:32.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.390+0000 7fd6e67fc640 1 --2- 172.21.15.38:0/1342630931 >> [v2:172.21.15.38:6832/1353335665,v1:172.21.15.38:6834/1353335665] conn(0x55c467b72e40 0x55c467f99eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:32.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.390+0000 7fd6c5ffb640 10 client.6135 ms_handle_connect on v2:172.21.15.38:6832/1353335665 2024-09-21T03:36:32.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.390+0000 7fd6e67fc640 1 --2- 172.21.15.38:0/1342630931 >> [v2:172.21.15.38:6832/1353335665,v1:172.21.15.38:6834/1353335665] conn(0x55c467b72e40 0x55c467f99eb0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67437ef8637d8f80 server_cookie=56bb92db0e616dd in_seq=0 out_seq=0 2024-09-21T03:36:32.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.391+0000 7fd6c5ffb640 1 -- 172.21.15.38:0/1342630931 <== mds.0 v2:172.21.15.38:6832/1353335665 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c467ed0770 con 0x55c467b72e40 2024-09-21T03:36:32.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.391+0000 7fd6c5ffb640 10 client.6135 handle_command_reply: tid=0 2024-09-21T03:36:32.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.391+0000 7fd6f4205740 1 client.6135 shutdown 2024-09-21T03:36:32.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.391+0000 7fd6f4205740 1 -- 172.21.15.38:0/1342630931 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c467b8b3b0 msgr2=0x55c467f99620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:32.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.391+0000 7fd6f4205740 1 --2- 172.21.15.38:0/1342630931 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c467b8b3b0 0x55c467f99620 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fd6d802af30 tx=0x7fd6d8009500 comp rx=0 tx=0).stop 2024-09-21T03:36:32.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.391+0000 7fd6f4205740 1 -- 172.21.15.38:0/1342630931 shutdown_connections 2024-09-21T03:36:32.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.391+0000 7fd6f4205740 1 --2- 172.21.15.38:0/1342630931 >> [v2:172.21.15.38:6832/1353335665,v1:172.21.15.38:6834/1353335665] conn(0x55c467b72e40 0x55c467f99eb0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:32.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.391+0000 7fd6f4205740 1 --2- 172.21.15.38:0/1342630931 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55c467bb4270 0x55c467f74400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:32.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.391+0000 7fd6f4205740 1 --2- 172.21.15.38:0/1342630931 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c467b93590 0x55c467f97bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:32.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.391+0000 7fd6c5ffb640 0 client.6135 ms_handle_reset on v2:172.21.15.38:6832/1353335665 2024-09-21T03:36:32.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.391+0000 7fd6f4205740 1 --2- 172.21.15.38:0/1342630931 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c467b8b3b0 0x55c467f99620 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:32.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.392+0000 7fd6f4205740 1 -- 172.21.15.38:0/1342630931 >> 172.21.15.38:0/1342630931 conn(0x55c467ba83c0 msgr2=0x55c467f76190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:32.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.392+0000 7fd6f4205740 1 -- 172.21.15.38:0/1342630931 shutdown_connections 2024-09-21T03:36:32.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.392+0000 7fd6f4205740 1 -- 172.21.15.38:0/1342630931 wait complete. 2024-09-21T03:36:32.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.392+0000 7fd6f4205740 20 client.6135 trim_cache size 0 max 16384 2024-09-21T03:36:32.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.410+0000 7fd6f4205740 1 Processor -- start 2024-09-21T03:36:32.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.410+0000 7fd6f4205740 1 -- start start 2024-09-21T03:36:32.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.411+0000 7fd6f4205740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c467b8b3b0 0x55c467fa4500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:32.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.411+0000 7fd6f4205740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55c467b93590 0x55c467f85b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:32.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.411+0000 7fd6f4205740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c467bb4270 0x55c467f86050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:32.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.411+0000 7fd6f4205740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fd6e81ff2b0 con 0x55c467bb4270 2024-09-21T03:36:32.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.411+0000 7fd6f4205740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd6e80cd230 con 0x55c467b8b3b0 2024-09-21T03:36:32.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.411+0000 7fd6f4205740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fd6e81fec90 con 0x55c467b93590 2024-09-21T03:36:32.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.412+0000 7fd6e77fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c467bb4270 0x55c467f86050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:32.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.412+0000 7fd6e77fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c467bb4270 0x55c467f86050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40116/0 (socket says 172.21.15.38:40116) 2024-09-21T03:36:32.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.412+0000 7fd6e77fe640 1 -- 172.21.15.38:0/663921647 learned_addr learned my addr 172.21.15.38:0/663921647 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:36:32.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.412+0000 7fd6e67fc640 1 --2- 172.21.15.38:0/663921647 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55c467b93590 0x55c467f85b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:32.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.412+0000 7fd6e77fe640 1 -- 172.21.15.38:0/663921647 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55c467b93590 msgr2=0x55c467f85b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:36:32.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.412+0000 7fd6e77fe640 1 --2- 172.21.15.38:0/663921647 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55c467b93590 0x55c467f85b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:32.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.412+0000 7fd6e77fe640 1 -- 172.21.15.38:0/663921647 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c467b8b3b0 msgr2=0x55c467fa4500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:36:32.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.412+0000 7fd6e77fe640 1 --2- 172.21.15.38:0/663921647 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c467b8b3b0 0x55c467fa4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:32.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.412+0000 7fd6e77fe640 1 -- 172.21.15.38:0/663921647 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c467f58b90 con 0x55c467bb4270 2024-09-21T03:36:32.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.412+0000 7fd6e67fc640 1 --2- 172.21.15.38:0/663921647 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55c467b93590 0x55c467f85b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:36:32.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.412+0000 7fd6e77fe640 1 --2- 172.21.15.38:0/663921647 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c467bb4270 0x55c467f86050 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fd6e000f5a0 tx=0x7fd6e0000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98d5b08f07209ea6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:32.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.412+0000 7fd6c7fff640 1 -- 172.21.15.38:0/663921647 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6e000f7b0 con 0x55c467bb4270 2024-09-21T03:36:32.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.412+0000 7fd6c7fff640 1 -- 172.21.15.38:0/663921647 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd6e0018040 con 0x55c467bb4270 2024-09-21T03:36:32.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.412+0000 7fd6c7fff640 1 -- 172.21.15.38:0/663921647 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6e000fd00 con 0x55c467bb4270 2024-09-21T03:36:32.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.413+0000 7fd6f4205740 1 -- 172.21.15.38:0/663921647 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c467bb4270 msgr2=0x55c467f86050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:32.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.413+0000 7fd6f4205740 1 --2- 172.21.15.38:0/663921647 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c467bb4270 0x55c467f86050 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fd6e000f5a0 tx=0x7fd6e0000ed0 comp rx=0 tx=0).stop 2024-09-21T03:36:32.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.413+0000 7fd6f4205740 1 -- 172.21.15.38:0/663921647 shutdown_connections 2024-09-21T03:36:32.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.413+0000 7fd6f4205740 1 --2- 172.21.15.38:0/663921647 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c467bb4270 0x55c467f86050 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:32.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.413+0000 7fd6f4205740 1 --2- 172.21.15.38:0/663921647 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55c467b93590 0x55c467f85b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:32.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.413+0000 7fd6f4205740 1 --2- 172.21.15.38:0/663921647 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c467b8b3b0 0x55c467fa4500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:32.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.413+0000 7fd6f4205740 1 -- 172.21.15.38:0/663921647 >> 172.21.15.38:0/663921647 conn(0x55c467ba83c0 msgr2=0x55c467f96ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:32.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.413+0000 7fd6f4205740 1 -- 172.21.15.38:0/663921647 shutdown_connections 2024-09-21T03:36:32.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.413+0000 7fd6f4205740 1 -- 172.21.15.38:0/663921647 wait complete. 2024-09-21T03:36:32.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.414+0000 7fd6f4205740 1 -- start start 2024-09-21T03:36:32.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.414+0000 7fd6f4205740 1 Processor -- start 2024-09-21T03:36:32.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.415+0000 7fd6f4205740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55c467b8b3b0 0x55c4680251d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:32.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.415+0000 7fd6f4205740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c467b93590 0x55c467fa28b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:32.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.415+0000 7fd6f4205740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c467bb4270 0x55c467fa2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:32.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.415+0000 7fd6f4205740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55c467fb00b0 con 0x55c467b93590 2024-09-21T03:36:32.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.415+0000 7fd6f4205740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55c467f74940 con 0x55c467bb4270 2024-09-21T03:36:32.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.415+0000 7fd6f4205740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55c467f9c800 con 0x55c467b8b3b0 2024-09-21T03:36:32.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.415+0000 7fd6f4205740 10 client.0 _ll_register_callbacks cb 0x7fd6d8003c60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:36:32.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.415+0000 7fd6e67fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c467b93590 0x55c467fa28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:32.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.415+0000 7fd6e67fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c467b93590 0x55c467fa28b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40126/0 (socket says 172.21.15.38:40126) 2024-09-21T03:36:32.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.415+0000 7fd6e67fc640 1 -- 172.21.15.38:0/939822372 learned_addr learned my addr 172.21.15.38:0/939822372 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:36:32.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.416+0000 7fd6e6ffd640 1 --2- 172.21.15.38:0/939822372 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55c467b8b3b0 0x55c4680251d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:32.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.416+0000 7fd6e77fe640 1 --2- 172.21.15.38:0/939822372 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c467bb4270 0x55c467fa2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:32.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.416+0000 7fd6e67fc640 1 -- 172.21.15.38:0/939822372 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55c467b8b3b0 msgr2=0x55c4680251d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:36:32.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.416+0000 7fd6e67fc640 1 --2- 172.21.15.38:0/939822372 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55c467b8b3b0 0x55c4680251d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:32.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.416+0000 7fd6e67fc640 1 -- 172.21.15.38:0/939822372 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c467bb4270 msgr2=0x55c467fa2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:36:32.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.416+0000 7fd6e67fc640 1 --2- 172.21.15.38:0/939822372 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c467bb4270 0x55c467fa2df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:32.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.416+0000 7fd6e67fc640 1 -- 172.21.15.38:0/939822372 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c467f9c9e0 con 0x55c467b93590 2024-09-21T03:36:32.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.416+0000 7fd6e77fe640 1 --2- 172.21.15.38:0/939822372 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c467bb4270 0x55c467fa2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:36:32.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.416+0000 7fd6e6ffd640 1 --2- 172.21.15.38:0/939822372 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55c467b8b3b0 0x55c4680251d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:36:32.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.416+0000 7fd6e67fc640 1 --2- 172.21.15.38:0/939822372 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c467b93590 0x55c467fa28b0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fd6d000cdc0 tx=0x7fd6d001e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab5b3d8438dd111d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:32.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.416+0000 7fd6c5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-21T03:36:32.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.416+0000 7fd6c5ffb640 1 -- 172.21.15.38:0/939822372 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6d006b700 con 0x55c467b93590 2024-09-21T03:36:32.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.416+0000 7fd6c5ffb640 1 -- 172.21.15.38:0/939822372 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(67..67 src has 1..67) ==== 6101+0+0 (secure 0 0 0) 0x7fd6d0081c60 con 0x55c467b93590 2024-09-21T03:36:32.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.416+0000 7fd6f4205740 1 -- 172.21.15.38:0/939822372 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd6e0018040 con 0x55c467b93590 2024-09-21T03:36:32.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.416+0000 7fd6c5ffb640 1 -- 172.21.15.38:0/939822372 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fd6d0081e60 con 0x55c467b93590 2024-09-21T03:36:32.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.416+0000 7fd6f4205740 10 client.6141 fetch_fsmap learned FSMap version 45 2024-09-21T03:36:32.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.416+0000 7fd6f4205740 1 -- 172.21.15.38:0/939822372 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=2+}) -- 0x55c467fbabe0 con 0x55c467b93590 2024-09-21T03:36:32.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.416+0000 7fd6c5ffb640 1 -- 172.21.15.38:0/939822372 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd6e0018040 con 0x55c467b93590 2024-09-21T03:36:32.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.416+0000 7fd6c5ffb640 1 -- 172.21.15.38:0/939822372 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 45) ==== 3502+0+0 (secure 0 0 0) 0x7fd6d0061080 con 0x55c467b93590 2024-09-21T03:36:32.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.417+0000 7fd6f4205740 10 client.6141 fetch_fsmap finished waiting for FSMap version 45 2024-09-21T03:36:32.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.417+0000 7fd6f4205740 10 client.6141 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-21T03:36:32.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.417+0000 7fd6f4205740 20 client.6141 populate_metadata read hostname 'smithi038' 2024-09-21T03:36:32.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.417+0000 7fd6f4205740 1 --2- 172.21.15.38:0/939822372 >> [v2:172.21.15.38:6832/1353335665,v1:172.21.15.38:6834/1353335665] conn(0x55c467f82280 0x55c468028de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:32.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.417+0000 7fd6f4205740 4 client.6141 mds_command: new command op to 5991 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T03:36:32.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.417+0000 7fd6f4205740 1 -- 172.21.15.38:0/939822372 --> [v2:172.21.15.38:6832/1353335665,v1:172.21.15.38:6834/1353335665] -- command(tid 0: {"prefix": "status"}) -- 0x55c467ecc220 con 0x55c467f82280 2024-09-21T03:36:32.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.417+0000 7fd6e6ffd640 1 --2- 172.21.15.38:0/939822372 >> [v2:172.21.15.38:6832/1353335665,v1:172.21.15.38:6834/1353335665] conn(0x55c467f82280 0x55c468028de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:32.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.417+0000 7fd6c5ffb640 10 client.6141 ms_handle_connect on v2:172.21.15.38:6832/1353335665 2024-09-21T03:36:32.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.417+0000 7fd6e6ffd640 1 --2- 172.21.15.38:0/939822372 >> [v2:172.21.15.38:6832/1353335665,v1:172.21.15.38:6834/1353335665] conn(0x55c467f82280 0x55c468028de0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e75282ed16dcdeec server_cookie=dcb5722e25e43fd3 in_seq=0 out_seq=0 2024-09-21T03:36:32.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.417+0000 7fd6c5ffb640 1 -- 172.21.15.38:0/939822372 <== mds.0 v2:172.21.15.38:6832/1353335665 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x55c467ecc220 con 0x55c467f82280 2024-09-21T03:36:32.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.417+0000 7fd6c5ffb640 10 client.6141 handle_command_reply: tid=0 2024-09-21T03:36:32.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.417+0000 7fd6f4205740 1 client.6141 shutdown 2024-09-21T03:36:32.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.417+0000 7fd6f4205740 1 -- 172.21.15.38:0/939822372 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c467b93590 msgr2=0x55c467fa28b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:32.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.417+0000 7fd6f4205740 1 --2- 172.21.15.38:0/939822372 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c467b93590 0x55c467fa28b0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fd6d000cdc0 tx=0x7fd6d001e040 comp rx=0 tx=0).stop 2024-09-21T03:36:32.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.417+0000 7fd6f4205740 1 -- 172.21.15.38:0/939822372 shutdown_connections 2024-09-21T03:36:32.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.417+0000 7fd6f4205740 1 --2- 172.21.15.38:0/939822372 >> [v2:172.21.15.38:6832/1353335665,v1:172.21.15.38:6834/1353335665] conn(0x55c467f82280 0x55c468028de0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:32.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.417+0000 7fd6f4205740 1 --2- 172.21.15.38:0/939822372 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c467bb4270 0x55c467fa2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:32.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.417+0000 7fd6f4205740 1 --2- 172.21.15.38:0/939822372 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c467b93590 0x55c467fa28b0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:32.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.417+0000 7fd6c5ffb640 0 client.6141 ms_handle_reset on v2:172.21.15.38:6832/1353335665 2024-09-21T03:36:32.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.417+0000 7fd6f4205740 1 --2- 172.21.15.38:0/939822372 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55c467b8b3b0 0x55c4680251d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:32.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.417+0000 7fd6f4205740 1 -- 172.21.15.38:0/939822372 >> 172.21.15.38:0/939822372 conn(0x55c467ba83c0 msgr2=0x55c467f56800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:32.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.418+0000 7fd6f4205740 1 -- 172.21.15.38:0/939822372 shutdown_connections 2024-09-21T03:36:32.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.418+0000 7fd6f4205740 1 -- 172.21.15.38:0/939822372 wait complete. 2024-09-21T03:36:32.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.418+0000 7fd6f4205740 20 client.6141 trim_cache size 0 max 16384 2024-09-21T03:36:32.425 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-09-21T03:36:32.425 INFO:teuthology.orchestra.run.smithi038.stdout: "cluster_fsid": "9d50d66f-656b-4fe2-8e3b-fe53b79c4d43", 2024-09-21T03:36:32.426 INFO:teuthology.orchestra.run.smithi038.stdout: "whoami": 0, 2024-09-21T03:36:32.426 INFO:teuthology.orchestra.run.smithi038.stdout: "id": 5991, 2024-09-21T03:36:32.426 INFO:teuthology.orchestra.run.smithi038.stdout: "want_state": "up:active", 2024-09-21T03:36:32.426 INFO:teuthology.orchestra.run.smithi038.stdout: "state": "up:active", 2024-09-21T03:36:32.426 INFO:teuthology.orchestra.run.smithi038.stdout: "fs_name": "cephfs", 2024-09-21T03:36:32.426 INFO:teuthology.orchestra.run.smithi038.stdout: "rank_uptime": 10.369572354000001, 2024-09-21T03:36:32.426 INFO:teuthology.orchestra.run.smithi038.stdout: "mdsmap_epoch": 45, 2024-09-21T03:36:32.426 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch": 67, 2024-09-21T03:36:32.426 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch_barrier": 67, 2024-09-21T03:36:32.426 INFO:teuthology.orchestra.run.smithi038.stdout: "uptime": 14.987135152 2024-09-21T03:36:32.426 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-09-21T03:36:32.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.418+0000 7fd6ed294640 1 -- 172.21.15.38:0/1747787826 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd6c8078b90 msgr2=0x7fd6c807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:32.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.418+0000 7fd6ed294640 1 --2- 172.21.15.38:0/1747787826 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd6c8078b90 0x7fd6c807b0a0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fd6d8034000 tx=0x7fd6d8004ab0 comp rx=0 tx=0).stop 2024-09-21T03:36:32.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.418+0000 7fd6ed294640 1 -- 172.21.15.38:0/1747787826 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd6e80b5260 msgr2=0x7fd6e8155220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:32.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.418+0000 7fd6ed294640 1 --2- 172.21.15.38:0/1747787826 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd6e80b5260 0x7fd6e8155220 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fd6d000c400 tx=0x7fd6d000c8c0 comp rx=0 tx=0).stop 2024-09-21T03:36:32.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.418+0000 7fd6ed294640 1 -- 172.21.15.38:0/1747787826 shutdown_connections 2024-09-21T03:36:32.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.418+0000 7fd6ed294640 1 --2- 172.21.15.38:0/1747787826 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd6c8078b90 0x7fd6c807b0a0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:32.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.418+0000 7fd6ed294640 1 --2- 172.21.15.38:0/1747787826 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd6e80b5670 0x7fd6e806cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:32.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.418+0000 7fd6ed294640 1 --2- 172.21.15.38:0/1747787826 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd6e80b5260 0x7fd6e8155220 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:32.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.418+0000 7fd6ed294640 1 --2- 172.21.15.38:0/1747787826 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd6e806def0 0x7fd6e81f71a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:32.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.418+0000 7fd6ed294640 1 -- 172.21.15.38:0/1747787826 >> 172.21.15.38:0/1747787826 conn(0x7fd6e80cdca0 msgr2=0x7fd6e8068d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:32.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.418+0000 7fd6ed294640 1 -- 172.21.15.38:0/1747787826 shutdown_connections 2024-09-21T03:36:32.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.418+0000 7fd6ed294640 1 -- 172.21.15.38:0/1747787826 wait complete. 2024-09-21T03:36:32.431 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T03:36:32.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.545+0000 7fdf5d132640 1 -- 172.21.15.38:0/2591283406 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdf580d3a70 msgr2=0x7fdf580d00d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:32.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.545+0000 7fdf5d132640 1 --2- 172.21.15.38:0/2591283406 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdf580d3a70 0x7fdf580d00d0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fdf50009830 tx=0x7fdf5002a530 comp rx=0 tx=0).stop 2024-09-21T03:36:32.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.545+0000 7fdf5d132640 1 -- 172.21.15.38:0/2591283406 shutdown_connections 2024-09-21T03:36:32.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.545+0000 7fdf5d132640 1 --2- 172.21.15.38:0/2591283406 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdf580d3a70 0x7fdf580d00d0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:32.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.545+0000 7fdf5d132640 1 -- 172.21.15.38:0/2591283406 >> 172.21.15.38:0/2591283406 conn(0x7fdf580d4620 msgr2=0x7fdf580d4a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:32.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.546+0000 7fdf5d132640 1 -- 172.21.15.38:0/2591283406 shutdown_connections 2024-09-21T03:36:32.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.546+0000 7fdf5d132640 1 -- 172.21.15.38:0/2591283406 wait complete. 2024-09-21T03:36:32.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.548+0000 7fdf5d132640 1 Processor -- start 2024-09-21T03:36:32.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.548+0000 7fdf5d132640 1 -- start start 2024-09-21T03:36:32.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.549+0000 7fdf5d132640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdf580cea90 0x7fdf58157280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:32.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.549+0000 7fdf5d132640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdf580d3a70 0x7fdf581f4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:32.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.549+0000 7fdf5d132640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdf58159d00 0x7fdf580b9070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:32.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.549+0000 7fdf5d132640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fdf580b4360 con 0x7fdf58159d00 2024-09-21T03:36:32.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.549+0000 7fdf5d132640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fdf580b41e0 con 0x7fdf580cea90 2024-09-21T03:36:32.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.549+0000 7fdf5d132640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fdf580b44e0 con 0x7fdf580d3a70 2024-09-21T03:36:32.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.549+0000 7fdf5759e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdf58159d00 0x7fdf580b9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:32.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.549+0000 7fdf5759e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdf58159d00 0x7fdf580b9070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40146/0 (socket says 172.21.15.38:40146) 2024-09-21T03:36:32.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.549+0000 7fdf5759e640 1 -- 172.21.15.38:0/3960884393 learned_addr learned my addr 172.21.15.38:0/3960884393 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:36:32.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.549+0000 7fdf56d9d640 1 --2- 172.21.15.38:0/3960884393 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdf580cea90 0x7fdf58157280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:32.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.549+0000 7fdf5659c640 1 --2- 172.21.15.38:0/3960884393 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdf580d3a70 0x7fdf581f4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:32.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.549+0000 7fdf5759e640 1 -- 172.21.15.38:0/3960884393 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdf580d3a70 msgr2=0x7fdf581f4d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:32.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.549+0000 7fdf5759e640 1 --2- 172.21.15.38:0/3960884393 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdf580d3a70 0x7fdf581f4d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:32.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.549+0000 7fdf5759e640 1 -- 172.21.15.38:0/3960884393 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdf580cea90 msgr2=0x7fdf58157280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:32.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.549+0000 7fdf5759e640 1 --2- 172.21.15.38:0/3960884393 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdf580cea90 0x7fdf58157280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:32.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.549+0000 7fdf5759e640 1 -- 172.21.15.38:0/3960884393 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf58161210 con 0x7fdf58159d00 2024-09-21T03:36:32.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.550+0000 7fdf5759e640 1 --2- 172.21.15.38:0/3960884393 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdf58159d00 0x7fdf580b9070 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fdf50009950 tx=0x7fdf50033c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:32.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.550+0000 7fdf3f7fe640 1 -- 172.21.15.38:0/3960884393 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf50031d60 con 0x7fdf58159d00 2024-09-21T03:36:32.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.550+0000 7fdf3f7fe640 1 -- 172.21.15.38:0/3960884393 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdf500330d0 con 0x7fdf58159d00 2024-09-21T03:36:32.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.550+0000 7fdf5d132640 1 -- 172.21.15.38:0/3960884393 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf5815a360 con 0x7fdf58159d00 2024-09-21T03:36:32.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.550+0000 7fdf3f7fe640 1 -- 172.21.15.38:0/3960884393 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf50033500 con 0x7fdf58159d00 2024-09-21T03:36:32.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.550+0000 7fdf5d132640 1 -- 172.21.15.38:0/3960884393 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf5815a920 con 0x7fdf58159d00 2024-09-21T03:36:32.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.551+0000 7fdf5d132640 1 -- 172.21.15.38:0/3960884393 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf580d00d0 con 0x7fdf58159d00 2024-09-21T03:36:32.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.552+0000 7fdf3f7fe640 1 -- 172.21.15.38:0/3960884393 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fdf500035f0 con 0x7fdf58159d00 2024-09-21T03:36:32.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.552+0000 7fdf3f7fe640 1 --2- 172.21.15.38:0/3960884393 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fdf44078b90 0x7fdf4407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:32.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.552+0000 7fdf3f7fe640 1 -- 172.21.15.38:0/3960884393 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6101+0+0 (secure 0 0 0) 0x7fdf500bc3b0 con 0x7fdf58159d00 2024-09-21T03:36:32.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.553+0000 7fdf56d9d640 1 --2- 172.21.15.38:0/3960884393 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fdf44078b90 0x7fdf4407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:32.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.553+0000 7fdf56d9d640 1 --2- 172.21.15.38:0/3960884393 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fdf44078b90 0x7fdf4407b0a0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fdf48007fd0 tx=0x7fdf48009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:32.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.557+0000 7fdf3f7fe640 1 -- 172.21.15.38:0/3960884393 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fdf5003f030 con 0x7fdf58159d00 2024-09-21T03:36:32.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.784+0000 7fdf5d132640 1 -- 172.21.15.38:0/3960884393 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fdf581577c0 con 0x7fdf58159d00 2024-09-21T03:36:32.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.785+0000 7fdf3f7fe640 1 -- 172.21.15.38:0/3960884393 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v45) ==== 53+0+73 (secure 0 0 0) 0x7fdf50084030 con 0x7fdf58159d00 2024-09-21T03:36:32.788 INFO:teuthology.orchestra.run.smithi038.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T03:36:32.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.787+0000 7fdf5d132640 1 -- 172.21.15.38:0/3960884393 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fdf44078b90 msgr2=0x7fdf4407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:32.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.787+0000 7fdf5d132640 1 --2- 172.21.15.38:0/3960884393 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fdf44078b90 0x7fdf4407b0a0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fdf48007fd0 tx=0x7fdf48009040 comp rx=0 tx=0).stop 2024-09-21T03:36:32.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.787+0000 7fdf5d132640 1 -- 172.21.15.38:0/3960884393 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdf58159d00 msgr2=0x7fdf580b9070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:32.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.787+0000 7fdf5d132640 1 --2- 172.21.15.38:0/3960884393 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdf58159d00 0x7fdf580b9070 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fdf50009950 tx=0x7fdf50033c90 comp rx=0 tx=0).stop 2024-09-21T03:36:32.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.787+0000 7fdf5d132640 1 -- 172.21.15.38:0/3960884393 shutdown_connections 2024-09-21T03:36:32.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.787+0000 7fdf5d132640 1 --2- 172.21.15.38:0/3960884393 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdf58159d00 0x7fdf580b9070 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:32.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.787+0000 7fdf5d132640 1 --2- 172.21.15.38:0/3960884393 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fdf44078b90 0x7fdf4407b0a0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:32.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.787+0000 7fdf5d132640 1 --2- 172.21.15.38:0/3960884393 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdf580d3a70 0x7fdf581f4d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:32.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.787+0000 7fdf5d132640 1 --2- 172.21.15.38:0/3960884393 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdf580cea90 0x7fdf58157280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:32.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.787+0000 7fdf5d132640 1 -- 172.21.15.38:0/3960884393 >> 172.21.15.38:0/3960884393 conn(0x7fdf580d4620 msgr2=0x7fdf58155700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:32.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.787+0000 7fdf5d132640 1 -- 172.21.15.38:0/3960884393 shutdown_connections 2024-09-21T03:36:32.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:32.787+0000 7fdf5d132640 1 -- 172.21.15.38:0/3960884393 wait complete. 2024-09-21T03:36:32.801 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-21T03:36:32.801 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T03:36:32.802 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi038.front.sepia.ceph.com 2024-09-21T03:36:32.802 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-21T03:36:32.802 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T03:36:32.802 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-21T03:36:32.802 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-21T03:36:32.802 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T03:36:32.802 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T03:36:32.802 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:36:32.802 DEBUG:teuthology.orchestra.run.smithi038:> ip addr 2024-09-21T03:36:32.819 INFO:teuthology.orchestra.run.smithi038.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T03:36:32.819 INFO:teuthology.orchestra.run.smithi038.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T03:36:32.819 INFO:teuthology.orchestra.run.smithi038.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T03:36:32.819 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:36:32.819 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 ::1/128 scope host 2024-09-21T03:36:32.819 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:36:32.819 INFO:teuthology.orchestra.run.smithi038.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T03:36:32.819 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:6b:fd:a2 brd ff:ff:ff:ff:ff:ff 2024-09-21T03:36:32.819 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp7s0f0 2024-09-21T03:36:32.819 INFO:teuthology.orchestra.run.smithi038.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T03:36:32.819 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:6b:fd:a3 brd ff:ff:ff:ff:ff:ff 2024-09-21T03:36:32.819 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp7s0f1 2024-09-21T03:36:32.820 INFO:teuthology.orchestra.run.smithi038.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T03:36:32.820 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:bd:15:fc brd ff:ff:ff:ff:ff:ff 2024-09-21T03:36:32.820 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp1s0f0 2024-09-21T03:36:32.820 INFO:teuthology.orchestra.run.smithi038.stdout: inet 172.21.15.38/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-21T03:36:32.820 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft 42616sec preferred_lft 42616sec 2024-09-21T03:36:32.820 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 fe80::ec4:7aff:febd:15fc/64 scope link 2024-09-21T03:36:32.820 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:36:32.820 INFO:teuthology.orchestra.run.smithi038.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T03:36:32.820 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:bd:15:fd brd ff:ff:ff:ff:ff:ff 2024-09-21T03:36:32.820 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp1s0f1 2024-09-21T03:36:32.820 INFO:teuthology.orchestra.run.smithi038.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T03:36:32.820 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 92:55:c7:97:6f:b5 brd ff:ff:ff:ff:ff:ff 2024-09-21T03:36:32.820 INFO:teuthology.orchestra.run.smithi038.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-21T03:36:32.820 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:36:32.821 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 fe80::c49f:c1ff:fe21:b12a/64 scope link 2024-09-21T03:36:32.821 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:36:32.821 INFO:teuthology.orchestra.run.smithi038.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T03:36:32.821 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether a6:3a:83:ad:db:d5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T03:36:32.821 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 fe80::a43a:83ff:fead:dbd5/64 scope link 2024-09-21T03:36:32.821 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:36:32.821 INFO:teuthology.orchestra.run.smithi038.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T03:36:32.821 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 92:55:c7:97:6f:b5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T03:36:32.821 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 fe80::9055:c7ff:fe97:6fb5/64 scope link 2024-09-21T03:36:32.821 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:36:32.821 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:36:32.821 DEBUG:teuthology.orchestra.run.smithi038:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T03:36:32.883 INFO:teuthology.orchestra.run.smithi038.stdout:1 2024-09-21T03:36:32.885 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:36:32.885 DEBUG:teuthology.orchestra.run.smithi038:> route 2024-09-21T03:36:32.941 INFO:teuthology.orchestra.run.smithi038.stdout:Kernel IP routing table 2024-09-21T03:36:32.941 INFO:teuthology.orchestra.run.smithi038.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T03:36:32.942 INFO:teuthology.orchestra.run.smithi038.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-21T03:36:32.942 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-21T03:36:32.942 INFO:teuthology.orchestra.run.smithi038.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T03:36:32.942 DEBUG:teuthology.orchestra.run.smithi038:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T03:36:32.942 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-09-21T03:36:32.942 DEBUG:teuthology.orchestra.run.smithi038:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-21T03:36:32.942 DEBUG:teuthology.orchestra.run.smithi038:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-21T03:36:32.942 DEBUG:teuthology.orchestra.run.smithi038:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-21T03:36:32.942 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-09-21T03:36:33.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:33 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T03:36:33.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:36:33.070 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:36:33.071 DEBUG:teuthology.orchestra.run.smithi038:> ip netns list 2024-09-21T03:36:33.087 INFO:teuthology.orchestra.run.smithi038.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-21T03:36:33.087 INFO:teuthology.orchestra.run.smithi038.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-21T03:36:33.087 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T03:36:33.087 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:36:33.087 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip link set brx.0 up 2024-09-21T03:36:33.154 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T03:36:33.154 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-21T03:36:33.209 INFO:teuthology.orchestra.run.smithi038.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-21T03:36:33.210 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T03:36:33.210 DEBUG:teuthology.orchestra.run.smithi038:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-21T03:36:33.265 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-21T03:36:33.265 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:36:33.265 DEBUG:teuthology.orchestra.run.smithi038:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T03:36:33.360 INFO:teuthology.orchestra.run.smithi038.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T03:36:33.360 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T03:36:33.360 INFO:teuthology.orchestra.run.smithi038.stdout:invalid new device string format 2024-09-21T03:36:33.371 INFO:teuthology.orchestra.run.smithi038.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T03:36:33.371 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T03:36:33.371 INFO:teuthology.orchestra.run.smithi038.stdout:invalid new device string format 2024-09-21T03:36:33.372 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: resolved to: "172.21.15.38:3300,172.21.15.195:3300,172.21.15.195:3301" 2024-09-21T03:36:33.372 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T03:36:33.378 INFO:teuthology.orchestra.run.smithi038.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T03:36:33.378 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T03:36:33.379 INFO:teuthology.orchestra.run.smithi038.stdout:invalid new device string format 2024-09-21T03:36:33.379 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: resolved to: "172.21.15.38:3300,172.21.15.195:3300,172.21.15.195:3301" 2024-09-21T03:36:33.379 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T03:36:33.379 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: resolved to: "172.21.15.38:6789,172.21.15.195:6789,172.21.15.195:6790" 2024-09-21T03:36:33.379 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.38:6789,172.21.15.195:6789,172.21.15.195:6790:/ 2024-09-21T03:36:33.379 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=9d50d66f-656b-4fe2-8e3b-fe53b79c4d43" will pass to kernel 2024-09-21T03:36:33.379 INFO:teuthology.orchestra.run.smithi038.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-21T03:36:33.379 INFO:teuthology.orchestra.run.smithi038.stdout: You just mounted a file system that supports labels which does not 2024-09-21T03:36:33.380 INFO:teuthology.orchestra.run.smithi038.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-21T03:36:33.380 INFO:teuthology.orchestra.run.smithi038.stdout: applications will generate AVC messages and not be allowed access to 2024-09-21T03:36:33.380 INFO:teuthology.orchestra.run.smithi038.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-21T03:36:33.381 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T03:36:33.381 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:36:33.381 DEBUG:teuthology.orchestra.run.smithi038:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-21T03:36:33.406 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T03:36:33.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:33 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T03:36:33.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:36:33.509 INFO:teuthology.orchestra.run.smithi038.stdout:client5531 2024-09-21T03:36:33.510 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-21T03:36:33.510 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:36:33.526 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'mkdir -p a/{b,c}/d; sync .') 2024-09-21T03:36:33.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:33 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T03:36:33.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:36:33.636 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-21T03:36:33.636 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:36:33.653 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T03:36:33.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:33.809+0000 7f47facf4640 1 -- 172.21.15.38:0/1062630464 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47dc0052b0 msgr2=0x7f47dc00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:33.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:33.809+0000 7f47facf4640 1 --2- 172.21.15.38:0/1062630464 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47dc0052b0 0x7f47dc00b9c0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f47f40cd070 tx=0x7f47dc005830 comp rx=0 tx=0).stop 2024-09-21T03:36:33.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:33.809+0000 7f47f9291640 1 -- 172.21.15.38:0/1062630464 reap_dead start 2024-09-21T03:36:33.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:33.809+0000 7f47facf4640 1 -- 172.21.15.38:0/1062630464 shutdown_connections 2024-09-21T03:36:33.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:33.809+0000 7f47facf4640 1 -- 172.21.15.38:0/1062630464 >> 172.21.15.38:0/1062630464 conn(0x7f47f40b46e0 msgr2=0x7f47f40d23e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:33.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:33.809+0000 7f47facf4640 1 -- 172.21.15.38:0/1062630464 shutdown_connections 2024-09-21T03:36:33.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:33.809+0000 7f47facf4640 1 -- 172.21.15.38:0/1062630464 wait complete. 2024-09-21T03:36:33.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:33.811+0000 7f47facf4640 1 Processor -- start 2024-09-21T03:36:33.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:33.811+0000 7f47facf4640 1 -- start start 2024-09-21T03:36:33.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:33.812+0000 7f47facf4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47dc0052b0 0x7f47f41f8ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:33.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:33.812+0000 7f47facf4640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f47f41f9400 0x7f47f40cecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:33.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:33.812+0000 7f47facf4640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f47f40cf230 0x7f47f40c7aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:33.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:33.812+0000 7f47facf4640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f47f4165540 con 0x7f47dc0052b0 2024-09-21T03:36:33.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:33.812+0000 7f47facf4640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f47f40b6e00 con 0x7f47f41f9400 2024-09-21T03:36:33.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:33.812+0000 7f47facf4640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f47f41656c0 con 0x7f47f40cf230 2024-09-21T03:36:33.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:33.812+0000 7f47f8a90640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47dc0052b0 0x7f47f41f8ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:33.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:33.812+0000 7f47f8a90640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47dc0052b0 0x7f47f41f8ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40164/0 (socket says 172.21.15.38:40164) 2024-09-21T03:36:33.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:33.812+0000 7f47f8a90640 1 -- 172.21.15.38:0/2796256657 learned_addr learned my addr 172.21.15.38:0/2796256657 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:36:33.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:33.812+0000 7f47f3fff640 1 --2- 172.21.15.38:0/2796256657 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f47f41f9400 0x7f47f40cecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:33.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:33.812+0000 7f47f9291640 1 --2- 172.21.15.38:0/2796256657 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f47f40cf230 0x7f47f40c7aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:33.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:33.813+0000 7f47f8a90640 1 -- 172.21.15.38:0/2796256657 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f47f40cf230 msgr2=0x7f47f40c7aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:33.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:33.813+0000 7f47f8a90640 1 --2- 172.21.15.38:0/2796256657 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f47f40cf230 0x7f47f40c7aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:33.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:33.813+0000 7f47f8a90640 1 -- 172.21.15.38:0/2796256657 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f47f41f9400 msgr2=0x7f47f40cecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:33.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:33.813+0000 7f47f8a90640 1 --2- 172.21.15.38:0/2796256657 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f47f41f9400 0x7f47f40cecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:33.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:33.813+0000 7f47f8a90640 1 -- 172.21.15.38:0/2796256657 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47f40c59f0 con 0x7f47dc0052b0 2024-09-21T03:36:33.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:33.813+0000 7f47f3fff640 1 --2- 172.21.15.38:0/2796256657 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f47f41f9400 0x7f47f40cecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:36:33.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:33.813+0000 7f47f8a90640 1 --2- 172.21.15.38:0/2796256657 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47dc0052b0 0x7f47f41f8ec0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f47e4003e70 tx=0x7f47e4001800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:33.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:33.813+0000 7f47f17fa640 1 -- 172.21.15.38:0/2796256657 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47e4011700 con 0x7f47dc0052b0 2024-09-21T03:36:33.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:33.813+0000 7f47f17fa640 1 -- 172.21.15.38:0/2796256657 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f47e4016020 con 0x7f47dc0052b0 2024-09-21T03:36:33.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:33.813+0000 7f47facf4640 1 -- 172.21.15.38:0/2796256657 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47f40cf640 con 0x7f47dc0052b0 2024-09-21T03:36:33.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:33.813+0000 7f47f17fa640 1 -- 172.21.15.38:0/2796256657 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47e4011b60 con 0x7f47dc0052b0 2024-09-21T03:36:33.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:33.813+0000 7f47facf4640 1 -- 172.21.15.38:0/2796256657 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47f40c6510 con 0x7f47dc0052b0 2024-09-21T03:36:33.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:33.814+0000 7f47facf4640 1 -- 172.21.15.38:0/2796256657 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47f4156430 con 0x7f47dc0052b0 2024-09-21T03:36:33.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:33.815+0000 7f47f17fa640 1 -- 172.21.15.38:0/2796256657 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f47e4011d00 con 0x7f47dc0052b0 2024-09-21T03:36:33.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:33.815+0000 7f47f17fa640 1 --2- 172.21.15.38:0/2796256657 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f47e0078b90 0x7f47e007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:33.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:33.815+0000 7f47f17fa640 1 -- 172.21.15.38:0/2796256657 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6101+0+0 (secure 0 0 0) 0x7f47e409bc10 con 0x7f47dc0052b0 2024-09-21T03:36:33.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:33.815+0000 7f47f3fff640 1 --2- 172.21.15.38:0/2796256657 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f47e0078b90 0x7f47e007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:33.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:33.816+0000 7f47f3fff640 1 --2- 172.21.15.38:0/2796256657 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f47e0078b90 0x7f47e007b0a0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f47e8001cd0 tx=0x7f47e8001ba0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:33.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:33.820+0000 7f47f17fa640 1 -- 172.21.15.38:0/2796256657 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f47e4020020 con 0x7f47dc0052b0 2024-09-21T03:36:34.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.021+0000 7f47facf4640 1 -- 172.21.15.38:0/2796256657 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f47f41573c0 con 0x7f47dc0052b0 2024-09-21T03:36:34.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.021+0000 7f47f17fa640 1 -- 172.21.15.38:0/2796256657 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v67) ==== 80+0+0 (secure 0 0 0) 0x7f47f40cf640 con 0x7f47dc0052b0 2024-09-21T03:36:34.024 INFO:teuthology.orchestra.run.smithi038.stderr:listed 0 entries 2024-09-21T03:36:34.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.023+0000 7f47facf4640 1 -- 172.21.15.38:0/2796256657 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f47e0078b90 msgr2=0x7f47e007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:34.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.023+0000 7f47facf4640 1 --2- 172.21.15.38:0/2796256657 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f47e0078b90 0x7f47e007b0a0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f47e8001cd0 tx=0x7f47e8001ba0 comp rx=0 tx=0).stop 2024-09-21T03:36:34.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.023+0000 7f47facf4640 1 -- 172.21.15.38:0/2796256657 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47dc0052b0 msgr2=0x7f47f41f8ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:34.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.023+0000 7f47facf4640 1 --2- 172.21.15.38:0/2796256657 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47dc0052b0 0x7f47f41f8ec0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f47e4003e70 tx=0x7f47e4001800 comp rx=0 tx=0).stop 2024-09-21T03:36:34.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.024+0000 7f47facf4640 1 -- 172.21.15.38:0/2796256657 shutdown_connections 2024-09-21T03:36:34.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.024+0000 7f47facf4640 1 --2- 172.21.15.38:0/2796256657 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f47e0078b90 0x7f47e007b0a0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:34.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.024+0000 7f47facf4640 1 --2- 172.21.15.38:0/2796256657 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f47f40cf230 0x7f47f40c7aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:34.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.024+0000 7f47facf4640 1 --2- 172.21.15.38:0/2796256657 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f47f41f9400 0x7f47f40cecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:34.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.024+0000 7f47facf4640 1 --2- 172.21.15.38:0/2796256657 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47dc0052b0 0x7f47f41f8ec0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:34.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.024+0000 7f47facf4640 1 -- 172.21.15.38:0/2796256657 >> 172.21.15.38:0/2796256657 conn(0x7f47f40b46e0 msgr2=0x7f47f415a690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:34.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.024+0000 7f47facf4640 1 -- 172.21.15.38:0/2796256657 shutdown_connections 2024-09-21T03:36:34.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.024+0000 7f47facf4640 1 -- 172.21.15.38:0/2796256657 wait complete. 2024-09-21T03:36:34.038 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-21T03:36:34.038 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:36:34.038 DEBUG:teuthology.orchestra.run.smithi038:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-21T03:36:34.071 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi038.front.sepia.ceph.com 2024-09-21T03:36:34.071 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:36:34.071 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T03:36:34.134 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.4:0 flush journal 2024-09-21T03:36:34.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.293+0000 7f3ef348c640 1 -- 172.21.15.38:0/3440575737 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3eec0d1120 msgr2=0x7f3eec0ceb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:34.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.293+0000 7f3ef348c640 1 --2- 172.21.15.38:0/3440575737 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3eec0d1120 0x7f3eec0ceb90 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f3ee4005fd0 tx=0x7f3ee402aa10 comp rx=0 tx=0).stop 2024-09-21T03:36:34.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.293+0000 7f3ef348c640 1 -- 172.21.15.38:0/3440575737 shutdown_connections 2024-09-21T03:36:34.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.293+0000 7f3ef348c640 1 --2- 172.21.15.38:0/3440575737 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3eec0d1120 0x7f3eec0ceb90 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:34.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.293+0000 7f3ef348c640 1 --2- 172.21.15.38:0/3440575737 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3eec0d0170 0x7f3eec0d0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:34.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.293+0000 7f3ef348c640 1 -- 172.21.15.38:0/3440575737 >> 172.21.15.38:0/3440575737 conn(0x7f3eec09d830 msgr2=0x7f3eec0b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:34.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.293+0000 7f3ef348c640 1 -- 172.21.15.38:0/3440575737 shutdown_connections 2024-09-21T03:36:34.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.293+0000 7f3ef348c640 1 -- 172.21.15.38:0/3440575737 wait complete. 2024-09-21T03:36:34.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.295+0000 7f3ef348c640 1 Processor -- start 2024-09-21T03:36:34.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.295+0000 7f3ef348c640 1 -- start start 2024-09-21T03:36:34.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.296+0000 7f3ef348c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3eec0d0170 0x7f3eec0a1e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:34.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.296+0000 7f3ef348c640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3eec0d1120 0x7f3eec0ce440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:34.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.296+0000 7f3ef348c640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3eec15ee80 0x7f3eec1f87f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:34.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.296+0000 7f3ef348c640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f3eec0b6700 con 0x7f3eec0d0170 2024-09-21T03:36:34.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.296+0000 7f3ef348c640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f3eec0b6580 con 0x7f3eec15ee80 2024-09-21T03:36:34.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.296+0000 7f3ef348c640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f3eec0b6880 con 0x7f3eec0d1120 2024-09-21T03:36:34.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.296+0000 7f3ef1228640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3eec0d0170 0x7f3eec0a1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:34.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.296+0000 7f3ef1228640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3eec0d0170 0x7f3eec0a1e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40172/0 (socket says 172.21.15.38:40172) 2024-09-21T03:36:34.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.296+0000 7f3ef1228640 1 -- 172.21.15.38:0/4253392789 learned_addr learned my addr 172.21.15.38:0/4253392789 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:36:34.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.296+0000 7f3ef0a27640 1 --2- 172.21.15.38:0/4253392789 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3eec0d1120 0x7f3eec0ce440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:34.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.296+0000 7f3ef1a29640 1 --2- 172.21.15.38:0/4253392789 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3eec15ee80 0x7f3eec1f87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:34.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.297+0000 7f3ef1228640 1 -- 172.21.15.38:0/4253392789 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3eec0d1120 msgr2=0x7f3eec0ce440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:34.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.297+0000 7f3ef1228640 1 --2- 172.21.15.38:0/4253392789 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3eec0d1120 0x7f3eec0ce440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:34.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.297+0000 7f3ef1228640 1 -- 172.21.15.38:0/4253392789 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3eec15ee80 msgr2=0x7f3eec1f87f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:34.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.297+0000 7f3ef1228640 1 --2- 172.21.15.38:0/4253392789 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3eec15ee80 0x7f3eec1f87f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:34.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.297+0000 7f3ef1228640 1 -- 172.21.15.38:0/4253392789 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3eec0ce980 con 0x7f3eec0d0170 2024-09-21T03:36:34.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.297+0000 7f3ef1228640 1 --2- 172.21.15.38:0/4253392789 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3eec0d0170 0x7f3eec0a1e10 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f3ee800c3a0 tx=0x7f3ee800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:34.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.297+0000 7f3ee1ffb640 1 -- 172.21.15.38:0/4253392789 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ee8017070 con 0x7f3eec0d0170 2024-09-21T03:36:34.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.297+0000 7f3ee1ffb640 1 -- 172.21.15.38:0/4253392789 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3ee80053c0 con 0x7f3eec0d0170 2024-09-21T03:36:34.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.297+0000 7f3ef348c640 1 -- 172.21.15.38:0/4253392789 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3eec0cbbe0 con 0x7f3eec0d0170 2024-09-21T03:36:34.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.297+0000 7f3ee1ffb640 1 -- 172.21.15.38:0/4253392789 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ee8003850 con 0x7f3eec0d0170 2024-09-21T03:36:34.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.297+0000 7f3ef348c640 1 -- 172.21.15.38:0/4253392789 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3eec0cc1a0 con 0x7f3eec0d0170 2024-09-21T03:36:34.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.299+0000 7f3ee1ffb640 1 -- 172.21.15.38:0/4253392789 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f3ee801a070 con 0x7f3eec0d0170 2024-09-21T03:36:34.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.300+0000 7f3ee1ffb640 1 --2- 172.21.15.38:0/4253392789 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3ed0078b90 0x7f3ed007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:34.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.300+0000 7f3ee1ffb640 1 -- 172.21.15.38:0/4253392789 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6101+0+0 (secure 0 0 0) 0x7f3ee809a290 con 0x7f3eec0d0170 2024-09-21T03:36:34.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.300+0000 7f3ef0a27640 1 --2- 172.21.15.38:0/4253392789 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3ed0078b90 0x7f3ed007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:34.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.301+0000 7f3efa3f3740 1 Processor -- start 2024-09-21T03:36:34.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.301+0000 7f3ef0a27640 1 --2- 172.21.15.38:0/4253392789 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3ed0078b90 0x7f3ed007b0a0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f3eec0caf20 tx=0x7f3edc00b2e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:34.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.301+0000 7f3efa3f3740 1 -- start start 2024-09-21T03:36:34.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.302+0000 7f3efa3f3740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55977795b390 0x559777d47a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:34.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.302+0000 7f3efa3f3740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5597779324d0 0x559777d1c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:34.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.302+0000 7f3efa3f3740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55977793a6b0 0x559777d1cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:34.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.302+0000 7f3efa3f3740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x559777c75a60 con 0x55977795b390 2024-09-21T03:36:34.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.302+0000 7f3efa3f3740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x559777c64e90 con 0x5597779324d0 2024-09-21T03:36:34.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.302+0000 7f3efa3f3740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x559777c68ca0 con 0x55977793a6b0 2024-09-21T03:36:34.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.302+0000 7f3ef1228640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55977795b390 0x559777d47a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:34.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.302+0000 7f3ef1228640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55977795b390 0x559777d47a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40186/0 (socket says 172.21.15.38:40186) 2024-09-21T03:36:34.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.302+0000 7f3ef1228640 1 -- 172.21.15.38:0/725521332 learned_addr learned my addr 172.21.15.38:0/725521332 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:36:34.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.302+0000 7f3ef0a27640 1 --2- 172.21.15.38:0/725521332 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5597779324d0 0x559777d1c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:34.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.302+0000 7f3ef1a29640 1 --2- 172.21.15.38:0/725521332 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55977793a6b0 0x559777d1cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:34.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.303+0000 7f3ef1228640 1 -- 172.21.15.38:0/725521332 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55977793a6b0 msgr2=0x559777d1cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:36:34.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.303+0000 7f3ef1228640 1 --2- 172.21.15.38:0/725521332 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55977793a6b0 0x559777d1cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:34.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.303+0000 7f3ef1228640 1 -- 172.21.15.38:0/725521332 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5597779324d0 msgr2=0x559777d1c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:36:34.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.303+0000 7f3ef1228640 1 --2- 172.21.15.38:0/725521332 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5597779324d0 0x559777d1c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:34.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.303+0000 7f3ef1228640 1 -- 172.21.15.38:0/725521332 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559777a3d7d0 con 0x55977795b390 2024-09-21T03:36:34.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.303+0000 7f3ef1228640 1 --2- 172.21.15.38:0/725521332 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55977795b390 0x559777d47a10 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f3ee80984d0 tx=0x7f3ee8000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18c88f3fcb478393 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:34.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.303+0000 7f3ed6ffd640 1 -- 172.21.15.38:0/725521332 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ee8065070 con 0x55977795b390 2024-09-21T03:36:34.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.303+0000 7f3ed6ffd640 1 -- 172.21.15.38:0/725521332 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3ee809a070 con 0x55977795b390 2024-09-21T03:36:34.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.303+0000 7f3ed6ffd640 1 -- 172.21.15.38:0/725521332 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ee8066880 con 0x55977795b390 2024-09-21T03:36:34.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.304+0000 7f3efa3f3740 1 -- 172.21.15.38:0/725521332 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55977795b390 msgr2=0x559777d47a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:34.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.304+0000 7f3efa3f3740 1 --2- 172.21.15.38:0/725521332 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55977795b390 0x559777d47a10 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f3ee80984d0 tx=0x7f3ee8000ed0 comp rx=0 tx=0).stop 2024-09-21T03:36:34.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.304+0000 7f3efa3f3740 1 -- 172.21.15.38:0/725521332 shutdown_connections 2024-09-21T03:36:34.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.304+0000 7f3efa3f3740 1 --2- 172.21.15.38:0/725521332 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55977793a6b0 0x559777d1cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:34.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.304+0000 7f3efa3f3740 1 --2- 172.21.15.38:0/725521332 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5597779324d0 0x559777d1c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:34.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.304+0000 7f3efa3f3740 1 --2- 172.21.15.38:0/725521332 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55977795b390 0x559777d47a10 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:34.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.304+0000 7f3efa3f3740 1 -- 172.21.15.38:0/725521332 >> 172.21.15.38:0/725521332 conn(0x55977794f4e0 msgr2=0x559777d02d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:34.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.304+0000 7f3efa3f3740 1 -- 172.21.15.38:0/725521332 shutdown_connections 2024-09-21T03:36:34.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.304+0000 7f3efa3f3740 1 -- 172.21.15.38:0/725521332 wait complete. 2024-09-21T03:36:34.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.305+0000 7f3efa3f3740 1 -- start start 2024-09-21T03:36:34.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.305+0000 7f3efa3f3740 1 Processor -- start 2024-09-21T03:36:34.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.306+0000 7f3efa3f3740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5597779324d0 0x559777d3d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:34.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.306+0000 7f3efa3f3740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55977793a6b0 0x559777d41080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:34.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.306+0000 7f3efa3f3740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55977795b390 0x559777d43530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:34.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.306+0000 7f3efa3f3740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5597779d5740 con 0x55977793a6b0 2024-09-21T03:36:34.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.306+0000 7f3efa3f3740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x559777c0a970 con 0x5597779324d0 2024-09-21T03:36:34.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.306+0000 7f3efa3f3740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x559777d41910 con 0x55977795b390 2024-09-21T03:36:34.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.306+0000 7f3efa3f3740 10 client.0 _ll_register_callbacks cb 0x559777c19780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:36:34.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.306+0000 7f3ef0a27640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55977793a6b0 0x559777d41080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:34.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.307+0000 7f3ef0a27640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55977793a6b0 0x559777d41080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40188/0 (socket says 172.21.15.38:40188) 2024-09-21T03:36:34.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.307+0000 7f3ef0a27640 1 -- 172.21.15.38:0/3581364444 learned_addr learned my addr 172.21.15.38:0/3581364444 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:36:34.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.307+0000 7f3ef1228640 1 --2- 172.21.15.38:0/3581364444 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5597779324d0 0x559777d3d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:34.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.307+0000 7f3ef1a29640 1 --2- 172.21.15.38:0/3581364444 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55977795b390 0x559777d43530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:34.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.307+0000 7f3ef0a27640 1 -- 172.21.15.38:0/3581364444 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55977795b390 msgr2=0x559777d43530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:36:34.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.307+0000 7f3ef0a27640 1 --2- 172.21.15.38:0/3581364444 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55977795b390 0x559777d43530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:34.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.307+0000 7f3ef0a27640 1 -- 172.21.15.38:0/3581364444 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5597779324d0 msgr2=0x559777d3d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:36:34.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.307+0000 7f3ef0a27640 1 --2- 172.21.15.38:0/3581364444 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5597779324d0 0x559777d3d7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:34.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.307+0000 7f3ef0a27640 1 -- 172.21.15.38:0/3581364444 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5597779c6970 con 0x55977793a6b0 2024-09-21T03:36:34.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.307+0000 7f3ef0a27640 1 --2- 172.21.15.38:0/3581364444 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55977793a6b0 0x559777d41080 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f3edc011720 tx=0x7f3edc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d920ceb28d55318 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:34.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.307+0000 7f3ecbfff640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-21T03:36:34.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.307+0000 7f3ecbfff640 1 -- 172.21.15.38:0/3581364444 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3edc01f900 con 0x55977793a6b0 2024-09-21T03:36:34.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.307+0000 7f3ecbfff640 1 -- 172.21.15.38:0/3581364444 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(67..67 src has 1..67) ==== 6101+0+0 (secure 0 0 0) 0x7f3edc029c80 con 0x55977793a6b0 2024-09-21T03:36:34.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.307+0000 7f3efa3f3740 1 -- 172.21.15.38:0/3581364444 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x559777c779b0 con 0x55977793a6b0 2024-09-21T03:36:34.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.307+0000 7f3ecbfff640 1 -- 172.21.15.38:0/3581364444 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f3edc01faa0 con 0x55977793a6b0 2024-09-21T03:36:34.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.307+0000 7f3efa3f3740 10 client.6168 fetch_fsmap learned FSMap version 45 2024-09-21T03:36:34.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.307+0000 7f3efa3f3740 1 -- 172.21.15.38:0/3581364444 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=2+}) -- 0x5597779a46a0 con 0x55977793a6b0 2024-09-21T03:36:34.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.308+0000 7f3ecbfff640 1 -- 172.21.15.38:0/3581364444 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x559777c779b0 con 0x55977793a6b0 2024-09-21T03:36:34.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.308+0000 7f3ecbfff640 1 -- 172.21.15.38:0/3581364444 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 45) ==== 3502+0+0 (secure 0 0 0) 0x7f3edc030050 con 0x55977793a6b0 2024-09-21T03:36:34.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.308+0000 7f3efa3f3740 10 client.6168 fetch_fsmap finished waiting for FSMap version 45 2024-09-21T03:36:34.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.308+0000 7f3efa3f3740 10 client.6168 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-09-21T03:36:34.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.308+0000 7f3efa3f3740 20 client.6168 populate_metadata read hostname 'smithi038' 2024-09-21T03:36:34.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.308+0000 7f3efa3f3740 1 --2- 172.21.15.38:0/3581364444 >> [v2:172.21.15.38:6832/1353335665,v1:172.21.15.38:6834/1353335665] conn(0x559777919f60 0x559777d11ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:34.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.308+0000 7f3efa3f3740 4 client.6168 mds_command: new command op to 5991 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:36:34.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.308+0000 7f3efa3f3740 1 -- 172.21.15.38:0/3581364444 --> [v2:172.21.15.38:6832/1353335665,v1:172.21.15.38:6834/1353335665] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559777c73460 con 0x559777919f60 2024-09-21T03:36:34.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.308+0000 7f3ef1228640 1 --2- 172.21.15.38:0/3581364444 >> [v2:172.21.15.38:6832/1353335665,v1:172.21.15.38:6834/1353335665] conn(0x559777919f60 0x559777d11ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:34.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.308+0000 7f3ecbfff640 10 client.6168 ms_handle_connect on v2:172.21.15.38:6832/1353335665 2024-09-21T03:36:34.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.308+0000 7f3ef1228640 1 --2- 172.21.15.38:0/3581364444 >> [v2:172.21.15.38:6832/1353335665,v1:172.21.15.38:6834/1353335665] conn(0x559777919f60 0x559777d11ad0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=559d0a7a9bc31553 server_cookie=d81db2bbab0f4c84 in_seq=0 out_seq=0 2024-09-21T03:36:34.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.309+0000 7f3ecbfff640 1 -- 172.21.15.38:0/3581364444 <== mds.0 v2:172.21.15.38:6832/1353335665 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x559777c73460 con 0x559777919f60 2024-09-21T03:36:34.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.309+0000 7f3ecbfff640 10 client.6168 handle_command_reply: tid=0 2024-09-21T03:36:34.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.309+0000 7f3efa3f3740 1 client.6168 shutdown 2024-09-21T03:36:34.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.309+0000 7f3efa3f3740 1 -- 172.21.15.38:0/3581364444 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55977793a6b0 msgr2=0x559777d41080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:34.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.309+0000 7f3efa3f3740 1 --2- 172.21.15.38:0/3581364444 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55977793a6b0 0x559777d41080 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f3edc011720 tx=0x7f3edc004730 comp rx=0 tx=0).stop 2024-09-21T03:36:34.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.309+0000 7f3efa3f3740 1 -- 172.21.15.38:0/3581364444 shutdown_connections 2024-09-21T03:36:34.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.309+0000 7f3efa3f3740 1 --2- 172.21.15.38:0/3581364444 >> [v2:172.21.15.38:6832/1353335665,v1:172.21.15.38:6834/1353335665] conn(0x559777919f60 0x559777d11ad0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:34.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.309+0000 7f3efa3f3740 1 --2- 172.21.15.38:0/3581364444 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55977795b390 0x559777d43530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:34.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.309+0000 7f3efa3f3740 1 --2- 172.21.15.38:0/3581364444 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55977793a6b0 0x559777d41080 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:34.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.309+0000 7f3ecbfff640 0 client.6168 ms_handle_reset on v2:172.21.15.38:6832/1353335665 2024-09-21T03:36:34.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.309+0000 7f3efa3f3740 1 --2- 172.21.15.38:0/3581364444 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5597779324d0 0x559777d3d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:34.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.309+0000 7f3efa3f3740 1 -- 172.21.15.38:0/3581364444 >> 172.21.15.38:0/3581364444 conn(0x55977794f4e0 msgr2=0x559777d1ce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:34.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.309+0000 7f3efa3f3740 1 -- 172.21.15.38:0/3581364444 shutdown_connections 2024-09-21T03:36:34.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.309+0000 7f3efa3f3740 1 -- 172.21.15.38:0/3581364444 wait complete. 2024-09-21T03:36:34.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.310+0000 7f3efa3f3740 20 client.6168 trim_cache size 0 max 16384 2024-09-21T03:36:34.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.324+0000 7f3efa3f3740 1 Processor -- start 2024-09-21T03:36:34.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.324+0000 7f3efa3f3740 1 -- start start 2024-09-21T03:36:34.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.326+0000 7f3efa3f3740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5597779324d0 0x559777d6a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:34.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.326+0000 7f3efa3f3740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55977793a6b0 0x559777d1ae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:34.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.326+0000 7f3efa3f3740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55977795b390 0x559777d1b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:34.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.326+0000 7f3efa3f3740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f3eec2026c0 con 0x55977795b390 2024-09-21T03:36:34.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.326+0000 7f3efa3f3740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f3eec0d4810 con 0x5597779324d0 2024-09-21T03:36:34.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.326+0000 7f3efa3f3740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f3eec0d3b70 con 0x55977793a6b0 2024-09-21T03:36:34.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.326+0000 7f3ef1a29640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55977795b390 0x559777d1b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:34.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.326+0000 7f3ef1a29640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55977795b390 0x559777d1b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40192/0 (socket says 172.21.15.38:40192) 2024-09-21T03:36:34.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.326+0000 7f3ef1a29640 1 -- 172.21.15.38:0/2732407094 learned_addr learned my addr 172.21.15.38:0/2732407094 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:36:34.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.326+0000 7f3ef1228640 1 --2- 172.21.15.38:0/2732407094 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5597779324d0 0x559777d6a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:34.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.326+0000 7f3ef0a27640 1 --2- 172.21.15.38:0/2732407094 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55977793a6b0 0x559777d1ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:34.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.326+0000 7f3ef1a29640 1 -- 172.21.15.38:0/2732407094 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55977793a6b0 msgr2=0x559777d1ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:36:34.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.326+0000 7f3ef1a29640 1 --2- 172.21.15.38:0/2732407094 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55977793a6b0 0x559777d1ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:34.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.326+0000 7f3ef1a29640 1 -- 172.21.15.38:0/2732407094 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5597779324d0 msgr2=0x559777d6a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:36:34.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.326+0000 7f3ef1a29640 1 --2- 172.21.15.38:0/2732407094 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5597779324d0 0x559777d6a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:34.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.326+0000 7f3ef1a29640 1 -- 172.21.15.38:0/2732407094 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559777d492f0 con 0x55977795b390 2024-09-21T03:36:34.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.326+0000 7f3ef1228640 1 --2- 172.21.15.38:0/2732407094 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5597779324d0 0x559777d6a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:36:34.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.326+0000 7f3ef0a27640 1 --2- 172.21.15.38:0/2732407094 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55977793a6b0 0x559777d1ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:36:34.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.326+0000 7f3ef1a29640 1 --2- 172.21.15.38:0/2732407094 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55977795b390 0x559777d1b370 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f3ee4009d60 tx=0x7f3ee4004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e2bb24654dd658b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:34.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.326+0000 7f3ed67fc640 1 -- 172.21.15.38:0/2732407094 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ee404a070 con 0x55977795b390 2024-09-21T03:36:34.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.326+0000 7f3ed67fc640 1 -- 172.21.15.38:0/2732407094 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3ee402f960 con 0x55977795b390 2024-09-21T03:36:34.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.327+0000 7f3ed67fc640 1 -- 172.21.15.38:0/2732407094 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ee4033040 con 0x55977795b390 2024-09-21T03:36:34.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.327+0000 7f3efa3f3740 1 -- 172.21.15.38:0/2732407094 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55977795b390 msgr2=0x559777d1b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:34.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.327+0000 7f3efa3f3740 1 --2- 172.21.15.38:0/2732407094 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55977795b390 0x559777d1b370 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f3ee4009d60 tx=0x7f3ee4004ab0 comp rx=0 tx=0).stop 2024-09-21T03:36:34.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.327+0000 7f3efa3f3740 1 -- 172.21.15.38:0/2732407094 shutdown_connections 2024-09-21T03:36:34.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.327+0000 7f3efa3f3740 1 --2- 172.21.15.38:0/2732407094 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55977795b390 0x559777d1b370 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:34.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.327+0000 7f3efa3f3740 1 --2- 172.21.15.38:0/2732407094 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55977793a6b0 0x559777d1ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:34.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.327+0000 7f3efa3f3740 1 --2- 172.21.15.38:0/2732407094 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5597779324d0 0x559777d6a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:34.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.327+0000 7f3efa3f3740 1 -- 172.21.15.38:0/2732407094 >> 172.21.15.38:0/2732407094 conn(0x55977794f4e0 msgr2=0x559777d43110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:34.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.327+0000 7f3efa3f3740 1 -- 172.21.15.38:0/2732407094 shutdown_connections 2024-09-21T03:36:34.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.327+0000 7f3efa3f3740 1 -- 172.21.15.38:0/2732407094 wait complete. 2024-09-21T03:36:34.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.328+0000 7f3efa3f3740 1 -- start start 2024-09-21T03:36:34.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.328+0000 7f3efa3f3740 1 Processor -- start 2024-09-21T03:36:34.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.330+0000 7f3efa3f3740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5597779324d0 0x559777d3b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:34.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.330+0000 7f3efa3f3740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55977793a6b0 0x559777dc7790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:34.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.330+0000 7f3efa3f3740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55977795b390 0x559777dbe230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:34.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.330+0000 7f3efa3f3740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f3eec0d41c0 con 0x55977793a6b0 2024-09-21T03:36:34.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.330+0000 7f3efa3f3740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x559777d11890 con 0x5597779324d0 2024-09-21T03:36:34.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.330+0000 7f3efa3f3740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x559777dc7fe0 con 0x55977795b390 2024-09-21T03:36:34.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.330+0000 7f3efa3f3740 10 client.0 _ll_register_callbacks cb 0x7f3edc010b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:36:34.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.330+0000 7f3ef0a27640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55977793a6b0 0x559777dc7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:34.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.330+0000 7f3ef0a27640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55977793a6b0 0x559777dc7790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40198/0 (socket says 172.21.15.38:40198) 2024-09-21T03:36:34.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.330+0000 7f3ef0a27640 1 -- 172.21.15.38:0/2352254313 learned_addr learned my addr 172.21.15.38:0/2352254313 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:36:34.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.330+0000 7f3ef1228640 1 --2- 172.21.15.38:0/2352254313 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5597779324d0 0x559777d3b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:34.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.330+0000 7f3ef1a29640 1 --2- 172.21.15.38:0/2352254313 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55977795b390 0x559777dbe230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:34.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.330+0000 7f3ef0a27640 1 -- 172.21.15.38:0/2352254313 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55977795b390 msgr2=0x559777dbe230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:36:34.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.330+0000 7f3ef0a27640 1 --2- 172.21.15.38:0/2352254313 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55977795b390 0x559777dbe230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:34.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.330+0000 7f3ef0a27640 1 -- 172.21.15.38:0/2352254313 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5597779324d0 msgr2=0x559777d3b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:36:34.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.330+0000 7f3ef0a27640 1 --2- 172.21.15.38:0/2352254313 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5597779324d0 0x559777d3b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:34.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.330+0000 7f3ef0a27640 1 -- 172.21.15.38:0/2352254313 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559777dcae20 con 0x55977793a6b0 2024-09-21T03:36:34.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.330+0000 7f3ef0a27640 1 --2- 172.21.15.38:0/2352254313 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55977793a6b0 0x559777dc7790 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x559777d02db0 tx=0x7f3edc01fd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83a6056602c31411 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:34.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.330+0000 7f3ecbfff640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-21T03:36:34.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.330+0000 7f3ecbfff640 1 -- 172.21.15.38:0/2352254313 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3edc032a00 con 0x55977793a6b0 2024-09-21T03:36:34.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.330+0000 7f3ecbfff640 1 -- 172.21.15.38:0/2352254313 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(67..67 src has 1..67) ==== 6101+0+0 (secure 0 0 0) 0x7f3edc032ba0 con 0x55977793a6b0 2024-09-21T03:36:34.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.330+0000 7f3efa3f3740 1 -- 172.21.15.38:0/2352254313 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ee402f960 con 0x55977793a6b0 2024-09-21T03:36:34.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.331+0000 7f3ecbfff640 1 -- 172.21.15.38:0/2352254313 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f3edc032da0 con 0x55977793a6b0 2024-09-21T03:36:34.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.331+0000 7f3efa3f3740 10 client.6174 fetch_fsmap learned FSMap version 45 2024-09-21T03:36:34.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.331+0000 7f3efa3f3740 1 -- 172.21.15.38:0/2352254313 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=2+}) -- 0x559777cfe5e0 con 0x55977793a6b0 2024-09-21T03:36:34.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.331+0000 7f3ecbfff640 1 -- 172.21.15.38:0/2352254313 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3ee402f960 con 0x55977793a6b0 2024-09-21T03:36:34.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.331+0000 7f3ecbfff640 1 -- 172.21.15.38:0/2352254313 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 45) ==== 3502+0+0 (secure 0 0 0) 0x7f3edc043080 con 0x55977793a6b0 2024-09-21T03:36:34.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.331+0000 7f3efa3f3740 10 client.6174 fetch_fsmap finished waiting for FSMap version 45 2024-09-21T03:36:34.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.331+0000 7f3efa3f3740 10 client.6174 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-09-21T03:36:34.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.331+0000 7f3efa3f3740 20 client.6174 populate_metadata read hostname 'smithi038' 2024-09-21T03:36:34.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.331+0000 7f3efa3f3740 1 --2- 172.21.15.38:0/2352254313 >> [v2:172.21.15.38:6832/1353335665,v1:172.21.15.38:6834/1353335665] conn(0x559777dcb300 0x559777dcb710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:34.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.331+0000 7f3efa3f3740 4 client.6174 mds_command: new command op to 5991 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T03:36:34.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.331+0000 7f3efa3f3740 1 -- 172.21.15.38:0/2352254313 --> [v2:172.21.15.38:6832/1353335665,v1:172.21.15.38:6834/1353335665] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559777cf8a90 con 0x559777dcb300 2024-09-21T03:36:34.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.331+0000 7f3ef1228640 1 --2- 172.21.15.38:0/2352254313 >> [v2:172.21.15.38:6832/1353335665,v1:172.21.15.38:6834/1353335665] conn(0x559777dcb300 0x559777dcb710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:34.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.331+0000 7f3ecbfff640 10 client.6174 ms_handle_connect on v2:172.21.15.38:6832/1353335665 2024-09-21T03:36:34.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.332+0000 7f3ef1228640 1 --2- 172.21.15.38:0/2352254313 >> [v2:172.21.15.38:6832/1353335665,v1:172.21.15.38:6834/1353335665] conn(0x559777dcb300 0x559777dcb710 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d55fbb352a7ba7ad server_cookie=308995f56f82ae8d in_seq=0 out_seq=0 2024-09-21T03:36:34.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.343+0000 7f3ecbfff640 1 -- 172.21.15.38:0/2352254313 <== mds.0 v2:172.21.15.38:6832/1353335665 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559777cf8a90 con 0x559777dcb300 2024-09-21T03:36:34.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.343+0000 7f3ecbfff640 10 client.6174 handle_command_reply: tid=0 2024-09-21T03:36:34.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.343+0000 7f3efa3f3740 1 client.6174 shutdown 2024-09-21T03:36:34.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.343+0000 7f3efa3f3740 1 -- 172.21.15.38:0/2352254313 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55977793a6b0 msgr2=0x559777dc7790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:34.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.343+0000 7f3efa3f3740 1 --2- 172.21.15.38:0/2352254313 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55977793a6b0 0x559777dc7790 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x559777d02db0 tx=0x7f3edc01fd80 comp rx=0 tx=0).stop 2024-09-21T03:36:34.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.343+0000 7f3efa3f3740 1 -- 172.21.15.38:0/2352254313 shutdown_connections 2024-09-21T03:36:34.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.343+0000 7f3efa3f3740 1 --2- 172.21.15.38:0/2352254313 >> [v2:172.21.15.38:6832/1353335665,v1:172.21.15.38:6834/1353335665] conn(0x559777dcb300 0x559777dcb710 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:34.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.343+0000 7f3efa3f3740 1 --2- 172.21.15.38:0/2352254313 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55977795b390 0x559777dbe230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:34.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.343+0000 7f3efa3f3740 1 --2- 172.21.15.38:0/2352254313 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55977793a6b0 0x559777dc7790 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:34.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.343+0000 7f3ecbfff640 0 client.6174 ms_handle_reset on v2:172.21.15.38:6832/1353335665 2024-09-21T03:36:34.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.343+0000 7f3efa3f3740 1 --2- 172.21.15.38:0/2352254313 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5597779324d0 0x559777d3b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:34.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.343+0000 7f3efa3f3740 1 -- 172.21.15.38:0/2352254313 >> 172.21.15.38:0/2352254313 conn(0x55977794f4e0 msgr2=0x559777d3f8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:34.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.343+0000 7f3efa3f3740 1 -- 172.21.15.38:0/2352254313 shutdown_connections 2024-09-21T03:36:34.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.343+0000 7f3efa3f3740 1 -- 172.21.15.38:0/2352254313 wait complete. 2024-09-21T03:36:34.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.344+0000 7f3efa3f3740 20 client.6174 trim_cache size 0 max 16384 2024-09-21T03:36:34.348 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-09-21T03:36:34.348 INFO:teuthology.orchestra.run.smithi038.stdout: "message": "", 2024-09-21T03:36:34.348 INFO:teuthology.orchestra.run.smithi038.stdout: "return_code": 0 2024-09-21T03:36:34.348 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-09-21T03:36:34.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.344+0000 7f3ef348c640 1 -- 172.21.15.38:0/4253392789 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3ed0078b90 msgr2=0x7f3ed007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:34.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.344+0000 7f3ef348c640 1 --2- 172.21.15.38:0/4253392789 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3ed0078b90 0x7f3ed007b0a0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f3eec0caf20 tx=0x7f3edc00b2e0 comp rx=0 tx=0).stop 2024-09-21T03:36:34.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.344+0000 7f3ef348c640 1 -- 172.21.15.38:0/4253392789 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3eec0d0170 msgr2=0x7f3eec0a1e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:34.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.344+0000 7f3ef348c640 1 --2- 172.21.15.38:0/4253392789 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3eec0d0170 0x7f3eec0a1e10 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f3ee800c3a0 tx=0x7f3ee800c860 comp rx=0 tx=0).stop 2024-09-21T03:36:34.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.344+0000 7f3ef348c640 1 -- 172.21.15.38:0/4253392789 shutdown_connections 2024-09-21T03:36:34.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.344+0000 7f3ef348c640 1 --2- 172.21.15.38:0/4253392789 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3eec15ee80 0x7f3eec1f87f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:34.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.344+0000 7f3ef348c640 1 --2- 172.21.15.38:0/4253392789 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3ed0078b90 0x7f3ed007b0a0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:34.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.344+0000 7f3ef348c640 1 --2- 172.21.15.38:0/4253392789 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3eec0d1120 0x7f3eec0ce440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:34.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.344+0000 7f3ef348c640 1 --2- 172.21.15.38:0/4253392789 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3eec0d0170 0x7f3eec0a1e10 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:34.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.344+0000 7f3ef348c640 1 -- 172.21.15.38:0/4253392789 >> 172.21.15.38:0/4253392789 conn(0x7f3eec09d830 msgr2=0x7f3eec0d09e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:34.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.344+0000 7f3ef348c640 1 -- 172.21.15.38:0/4253392789 shutdown_connections 2024-09-21T03:36:34.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.344+0000 7f3ef348c640 1 -- 172.21.15.38:0/4253392789 wait complete. 2024-09-21T03:36:34.358 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T03:36:34.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.473+0000 7fd30327e640 1 -- 172.21.15.38:0/3505988270 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd2e8004800 msgr2=0x7fd2e8004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:34.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.473+0000 7fd30327e640 1 --2- 172.21.15.38:0/3505988270 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd2e8004800 0x7fd2e8004c10 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fd2ec009b50 tx=0x7fd2e8005830 comp rx=0 tx=0).stop 2024-09-21T03:36:34.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.473+0000 7fd30181b640 1 -- 172.21.15.38:0/3505988270 reap_dead start 2024-09-21T03:36:34.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.473+0000 7fd30327e640 1 -- 172.21.15.38:0/3505988270 shutdown_connections 2024-09-21T03:36:34.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.473+0000 7fd30327e640 1 -- 172.21.15.38:0/3505988270 >> 172.21.15.38:0/3505988270 conn(0x7fd2fc0b6b20 msgr2=0x7fd2fc0d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:34.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.473+0000 7fd30327e640 1 -- 172.21.15.38:0/3505988270 shutdown_connections 2024-09-21T03:36:34.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.473+0000 7fd30327e640 1 -- 172.21.15.38:0/3505988270 wait complete. 2024-09-21T03:36:34.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.475+0000 7fd30327e640 1 Processor -- start 2024-09-21T03:36:34.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.475+0000 7fd30327e640 1 -- start start 2024-09-21T03:36:34.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.476+0000 7fd30327e640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd2e8004800 0x7fd2fc0aadc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:34.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.476+0000 7fd30327e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd2fc14d880 0x7fd2fc14dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:34.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.476+0000 7fd30327e640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd2fc0b23c0 0x7fd2fc0b27d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:34.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.476+0000 7fd30327e640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fd2fc0b4ea0 con 0x7fd2fc14d880 2024-09-21T03:36:34.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.476+0000 7fd30327e640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd2fc163110 con 0x7fd2e8004800 2024-09-21T03:36:34.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.476+0000 7fd30327e640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fd2fc0d24d0 con 0x7fd2fc0b23c0 2024-09-21T03:36:34.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.477+0000 7fd300819640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd2fc14d880 0x7fd2fc14dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:34.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.477+0000 7fd300819640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd2fc14d880 0x7fd2fc14dc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40216/0 (socket says 172.21.15.38:40216) 2024-09-21T03:36:34.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.477+0000 7fd300819640 1 -- 172.21.15.38:0/2787634099 learned_addr learned my addr 172.21.15.38:0/2787634099 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:36:34.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.477+0000 7fd30101a640 1 --2- 172.21.15.38:0/2787634099 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd2e8004800 0x7fd2fc0aadc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:34.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.477+0000 7fd30181b640 1 --2- 172.21.15.38:0/2787634099 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd2fc0b23c0 0x7fd2fc0b27d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:34.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.477+0000 7fd300819640 1 -- 172.21.15.38:0/2787634099 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd2fc0b23c0 msgr2=0x7fd2fc0b27d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:34.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.477+0000 7fd300819640 1 --2- 172.21.15.38:0/2787634099 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd2fc0b23c0 0x7fd2fc0b27d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:34.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.477+0000 7fd300819640 1 -- 172.21.15.38:0/2787634099 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd2e8004800 msgr2=0x7fd2fc0aadc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:34.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.477+0000 7fd300819640 1 --2- 172.21.15.38:0/2787634099 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd2e8004800 0x7fd2fc0aadc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:34.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.477+0000 7fd300819640 1 -- 172.21.15.38:0/2787634099 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2fc0b1600 con 0x7fd2fc14d880 2024-09-21T03:36:34.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.477+0000 7fd300819640 1 --2- 172.21.15.38:0/2787634099 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd2fc14d880 0x7fd2fc14dc90 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fd2ec009c70 tx=0x7fd2ec033670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:34.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.477+0000 7fd2f1ffb640 1 -- 172.21.15.38:0/2787634099 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2ec038040 con 0x7fd2fc14d880 2024-09-21T03:36:34.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.477+0000 7fd2f1ffb640 1 -- 172.21.15.38:0/2787634099 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd2ec004020 con 0x7fd2fc14d880 2024-09-21T03:36:34.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.477+0000 7fd30327e640 1 -- 172.21.15.38:0/2787634099 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2fc0b18e0 con 0x7fd2fc14d880 2024-09-21T03:36:34.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.477+0000 7fd2f1ffb640 1 -- 172.21.15.38:0/2787634099 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2ec004450 con 0x7fd2fc14d880 2024-09-21T03:36:34.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.478+0000 7fd30327e640 1 -- 172.21.15.38:0/2787634099 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2fc0b1ea0 con 0x7fd2fc14d880 2024-09-21T03:36:34.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.479+0000 7fd30327e640 1 -- 172.21.15.38:0/2787634099 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2fc0d2980 con 0x7fd2fc14d880 2024-09-21T03:36:34.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.479+0000 7fd2f1ffb640 1 -- 172.21.15.38:0/2787634099 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fd2ec045020 con 0x7fd2fc14d880 2024-09-21T03:36:34.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.480+0000 7fd2f1ffb640 1 --2- 172.21.15.38:0/2787634099 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd2e0078b90 0x7fd2e007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:34.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.480+0000 7fd2f1ffb640 1 -- 172.21.15.38:0/2787634099 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6101+0+0 (secure 0 0 0) 0x7fd2ec0bb830 con 0x7fd2fc14d880 2024-09-21T03:36:34.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.480+0000 7fd30101a640 1 --2- 172.21.15.38:0/2787634099 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd2e0078b90 0x7fd2e007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:34.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.480+0000 7fd30101a640 1 --2- 172.21.15.38:0/2787634099 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd2e0078b90 0x7fd2e007b0a0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fd2f8009b30 tx=0x7fd2f8005f90 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:34.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.484+0000 7fd2f1ffb640 1 -- 172.21.15.38:0/2787634099 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fd2ec083b00 con 0x7fd2fc14d880 2024-09-21T03:36:34.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.708+0000 7fd30327e640 1 -- 172.21.15.38:0/2787634099 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fd2fc14e380 con 0x7fd2fc14d880 2024-09-21T03:36:34.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.709+0000 7fd2f1ffb640 1 -- 172.21.15.38:0/2787634099 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v45) ==== 53+0+73 (secure 0 0 0) 0x7fd2ec083400 con 0x7fd2fc14d880 2024-09-21T03:36:34.712 INFO:teuthology.orchestra.run.smithi038.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T03:36:34.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.711+0000 7fd30327e640 1 -- 172.21.15.38:0/2787634099 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd2e0078b90 msgr2=0x7fd2e007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:34.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.711+0000 7fd30327e640 1 --2- 172.21.15.38:0/2787634099 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd2e0078b90 0x7fd2e007b0a0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fd2f8009b30 tx=0x7fd2f8005f90 comp rx=0 tx=0).stop 2024-09-21T03:36:34.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.711+0000 7fd30327e640 1 -- 172.21.15.38:0/2787634099 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd2fc14d880 msgr2=0x7fd2fc14dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:34.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.711+0000 7fd30327e640 1 --2- 172.21.15.38:0/2787634099 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd2fc14d880 0x7fd2fc14dc90 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fd2ec009c70 tx=0x7fd2ec033670 comp rx=0 tx=0).stop 2024-09-21T03:36:34.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.711+0000 7fd30327e640 1 -- 172.21.15.38:0/2787634099 shutdown_connections 2024-09-21T03:36:34.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.711+0000 7fd30327e640 1 --2- 172.21.15.38:0/2787634099 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd2e0078b90 0x7fd2e007b0a0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:34.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.711+0000 7fd30327e640 1 --2- 172.21.15.38:0/2787634099 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd2fc0b23c0 0x7fd2fc0b27d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:34.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.711+0000 7fd30327e640 1 --2- 172.21.15.38:0/2787634099 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd2fc14d880 0x7fd2fc14dc90 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:34.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.711+0000 7fd30327e640 1 --2- 172.21.15.38:0/2787634099 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd2e8004800 0x7fd2fc0aadc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:34.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.711+0000 7fd30327e640 1 -- 172.21.15.38:0/2787634099 >> 172.21.15.38:0/2787634099 conn(0x7fd2fc0b6b20 msgr2=0x7fd2fc15d8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:34.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.711+0000 7fd30327e640 1 -- 172.21.15.38:0/2787634099 shutdown_connections 2024-09-21T03:36:34.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34.711+0000 7fd30327e640 1 -- 172.21.15.38:0/2787634099 wait complete. 2024-09-21T03:36:34.725 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-21T03:36:34.725 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T03:36:34.725 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi038.front.sepia.ceph.com 2024-09-21T03:36:34.725 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-21T03:36:34.726 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T03:36:34.726 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-21T03:36:34.726 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-21T03:36:34.726 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T03:36:34.726 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T03:36:34.726 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:36:34.726 DEBUG:teuthology.orchestra.run.smithi038:> ip addr 2024-09-21T03:36:34.743 INFO:teuthology.orchestra.run.smithi038.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T03:36:34.743 INFO:teuthology.orchestra.run.smithi038.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T03:36:34.743 INFO:teuthology.orchestra.run.smithi038.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T03:36:34.743 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:36:34.743 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 ::1/128 scope host 2024-09-21T03:36:34.743 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:36:34.743 INFO:teuthology.orchestra.run.smithi038.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T03:36:34.743 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:6b:fd:a2 brd ff:ff:ff:ff:ff:ff 2024-09-21T03:36:34.743 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp7s0f0 2024-09-21T03:36:34.743 INFO:teuthology.orchestra.run.smithi038.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T03:36:34.743 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:6b:fd:a3 brd ff:ff:ff:ff:ff:ff 2024-09-21T03:36:34.744 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp7s0f1 2024-09-21T03:36:34.744 INFO:teuthology.orchestra.run.smithi038.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T03:36:34.744 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:bd:15:fc brd ff:ff:ff:ff:ff:ff 2024-09-21T03:36:34.744 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp1s0f0 2024-09-21T03:36:34.744 INFO:teuthology.orchestra.run.smithi038.stdout: inet 172.21.15.38/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-21T03:36:34.744 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft 42614sec preferred_lft 42614sec 2024-09-21T03:36:34.744 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 fe80::ec4:7aff:febd:15fc/64 scope link 2024-09-21T03:36:34.744 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:36:34.744 INFO:teuthology.orchestra.run.smithi038.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T03:36:34.744 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:bd:15:fd brd ff:ff:ff:ff:ff:ff 2024-09-21T03:36:34.744 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp1s0f1 2024-09-21T03:36:34.744 INFO:teuthology.orchestra.run.smithi038.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T03:36:34.744 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 92:55:c7:97:6f:b5 brd ff:ff:ff:ff:ff:ff 2024-09-21T03:36:34.744 INFO:teuthology.orchestra.run.smithi038.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-21T03:36:34.745 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:36:34.745 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 fe80::c49f:c1ff:fe21:b12a/64 scope link 2024-09-21T03:36:34.745 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:36:34.745 INFO:teuthology.orchestra.run.smithi038.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T03:36:34.745 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether a6:3a:83:ad:db:d5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T03:36:34.745 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 fe80::a43a:83ff:fead:dbd5/64 scope link 2024-09-21T03:36:34.745 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:36:34.745 INFO:teuthology.orchestra.run.smithi038.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T03:36:34.745 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 92:55:c7:97:6f:b5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T03:36:34.745 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 fe80::9055:c7ff:fe97:6fb5/64 scope link 2024-09-21T03:36:34.745 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:36:34.746 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:36:34.746 DEBUG:teuthology.orchestra.run.smithi038:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T03:36:34.807 INFO:teuthology.orchestra.run.smithi038.stdout:1 2024-09-21T03:36:34.808 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:36:34.808 DEBUG:teuthology.orchestra.run.smithi038:> route 2024-09-21T03:36:34.865 INFO:teuthology.orchestra.run.smithi038.stdout:Kernel IP routing table 2024-09-21T03:36:34.865 INFO:teuthology.orchestra.run.smithi038.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T03:36:34.865 INFO:teuthology.orchestra.run.smithi038.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-21T03:36:34.865 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-21T03:36:34.865 INFO:teuthology.orchestra.run.smithi038.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T03:36:34.865 DEBUG:teuthology.orchestra.run.smithi038:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T03:36:34.865 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-09-21T03:36:34.865 DEBUG:teuthology.orchestra.run.smithi038:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-21T03:36:34.866 DEBUG:teuthology.orchestra.run.smithi038:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-21T03:36:34.866 DEBUG:teuthology.orchestra.run.smithi038:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-21T03:36:34.866 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-09-21T03:36:34.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T03:36:34.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:36:34.993 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:36:34.993 DEBUG:teuthology.orchestra.run.smithi038:> ip netns list 2024-09-21T03:36:35.009 INFO:teuthology.orchestra.run.smithi038.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-21T03:36:35.009 INFO:teuthology.orchestra.run.smithi038.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-21T03:36:35.009 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T03:36:35.009 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:36:35.009 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip link set brx.0 up 2024-09-21T03:36:35.075 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T03:36:35.075 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-21T03:36:35.131 INFO:teuthology.orchestra.run.smithi038.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-21T03:36:35.131 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T03:36:35.131 DEBUG:teuthology.orchestra.run.smithi038:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-21T03:36:35.187 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-21T03:36:35.187 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:36:35.187 DEBUG:teuthology.orchestra.run.smithi038:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T03:36:35.283 INFO:teuthology.orchestra.run.smithi038.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T03:36:35.283 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T03:36:35.283 INFO:teuthology.orchestra.run.smithi038.stdout:invalid new device string format 2024-09-21T03:36:35.294 INFO:teuthology.orchestra.run.smithi038.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T03:36:35.294 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T03:36:35.294 INFO:teuthology.orchestra.run.smithi038.stdout:invalid new device string format 2024-09-21T03:36:35.294 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: resolved to: "172.21.15.38:3300,172.21.15.195:3300,172.21.15.195:3301" 2024-09-21T03:36:35.294 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T03:36:35.300 INFO:teuthology.orchestra.run.smithi038.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T03:36:35.300 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T03:36:35.300 INFO:teuthology.orchestra.run.smithi038.stdout:invalid new device string format 2024-09-21T03:36:35.300 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: resolved to: "172.21.15.38:3300,172.21.15.195:3300,172.21.15.195:3301" 2024-09-21T03:36:35.300 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T03:36:35.300 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: resolved to: "172.21.15.38:6789,172.21.15.195:6789,172.21.15.195:6790" 2024-09-21T03:36:35.300 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.38:6789,172.21.15.195:6789,172.21.15.195:6790:/ 2024-09-21T03:36:35.301 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=9d50d66f-656b-4fe2-8e3b-fe53b79c4d43" will pass to kernel 2024-09-21T03:36:35.301 INFO:teuthology.orchestra.run.smithi038.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-21T03:36:35.301 INFO:teuthology.orchestra.run.smithi038.stdout: You just mounted a file system that supports labels which does not 2024-09-21T03:36:35.301 INFO:teuthology.orchestra.run.smithi038.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-21T03:36:35.301 INFO:teuthology.orchestra.run.smithi038.stdout: applications will generate AVC messages and not be allowed access to 2024-09-21T03:36:35.301 INFO:teuthology.orchestra.run.smithi038.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-21T03:36:35.302 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T03:36:35.302 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:36:35.303 DEBUG:teuthology.orchestra.run.smithi038:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-21T03:36:35.328 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T03:36:35.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:35 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T03:36:35.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:36:35.425 INFO:teuthology.orchestra.run.smithi038.stdout:client6183 2024-09-21T03:36:35.425 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-21T03:36:35.425 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:36:35.442 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'chmod 711 a/b/d; sync .') 2024-09-21T03:36:35.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:35 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T03:36:35.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:36:35.540 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-21T03:36:35.540 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:36:35.556 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T03:36:35.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:35.712+0000 7fde9d670640 1 -- 172.21.15.38:0/4207334497 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fde84003fd0 msgr2=0x7fde84004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:35.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:35.712+0000 7fde9d670640 1 --2- 172.21.15.38:0/4207334497 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fde84003fd0 0x7fde84004800 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fde98073e50 tx=0x7fde84005830 comp rx=0 tx=0).stop 2024-09-21T03:36:35.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:35.712+0000 7fde977fe640 1 -- 172.21.15.38:0/4207334497 reap_dead start 2024-09-21T03:36:35.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:35.712+0000 7fde9d670640 1 -- 172.21.15.38:0/4207334497 shutdown_connections 2024-09-21T03:36:35.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:35.712+0000 7fde9d670640 1 -- 172.21.15.38:0/4207334497 >> 172.21.15.38:0/4207334497 conn(0x7fde980cdca0 msgr2=0x7fde980ce0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:35.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:35.712+0000 7fde9d670640 1 -- 172.21.15.38:0/4207334497 shutdown_connections 2024-09-21T03:36:35.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:35.712+0000 7fde9d670640 1 -- 172.21.15.38:0/4207334497 wait complete. 2024-09-21T03:36:35.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:35.713+0000 7fde9d670640 1 Processor -- start 2024-09-21T03:36:35.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:35.713+0000 7fde9d670640 1 -- start start 2024-09-21T03:36:35.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:35.715+0000 7fde9d670640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fde84003fd0 0x7fde981e69a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:35.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:35.715+0000 7fde9d670640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fde98158da0 0x7fde98158500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:35.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:35.715+0000 7fde9d670640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fde980b1de0 0x7fde980b21f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:35.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:35.715+0000 7fde9d670640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fde980aed00 con 0x7fde98158da0 2024-09-21T03:36:35.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:35.715+0000 7fde9d670640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fde980aeb80 con 0x7fde980b1de0 2024-09-21T03:36:35.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:35.715+0000 7fde9d670640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fde980b5ff0 con 0x7fde84003fd0 2024-09-21T03:36:35.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:35.715+0000 7fde8ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fde98158da0 0x7fde98158500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:35.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:35.715+0000 7fde8ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fde98158da0 0x7fde98158500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40226/0 (socket says 172.21.15.38:40226) 2024-09-21T03:36:35.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:35.715+0000 7fde8ffff640 1 -- 172.21.15.38:0/3981183923 learned_addr learned my addr 172.21.15.38:0/3981183923 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:36:35.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:35.715+0000 7fde977fe640 1 --2- 172.21.15.38:0/3981183923 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fde980b1de0 0x7fde980b21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:35.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:35.715+0000 7fde96ffd640 1 --2- 172.21.15.38:0/3981183923 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fde84003fd0 0x7fde981e69a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:35.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:35.715+0000 7fde8ffff640 1 -- 172.21.15.38:0/3981183923 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fde84003fd0 msgr2=0x7fde981e69a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:35.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:35.715+0000 7fde8ffff640 1 --2- 172.21.15.38:0/3981183923 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fde84003fd0 0x7fde981e69a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:35.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:35.715+0000 7fde8ffff640 1 -- 172.21.15.38:0/3981183923 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fde980b1de0 msgr2=0x7fde980b21f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:35.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:35.715+0000 7fde8ffff640 1 --2- 172.21.15.38:0/3981183923 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fde980b1de0 0x7fde980b21f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:35.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:35.715+0000 7fde8ffff640 1 -- 172.21.15.38:0/3981183923 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde980b2730 con 0x7fde98158da0 2024-09-21T03:36:35.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:35.715+0000 7fde8ffff640 1 --2- 172.21.15.38:0/3981183923 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fde98158da0 0x7fde98158500 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fde80031040 tx=0x7fde80006420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:35.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:35.716+0000 7fde8f7fe640 1 -- 172.21.15.38:0/3981183923 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde80032070 con 0x7fde98158da0 2024-09-21T03:36:35.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:35.716+0000 7fde8f7fe640 1 -- 172.21.15.38:0/3981183923 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fde800038e0 con 0x7fde98158da0 2024-09-21T03:36:35.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:35.716+0000 7fde9d670640 1 -- 172.21.15.38:0/3981183923 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde981f2c20 con 0x7fde98158da0 2024-09-21T03:36:35.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:35.716+0000 7fde8f7fe640 1 -- 172.21.15.38:0/3981183923 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde80003d30 con 0x7fde98158da0 2024-09-21T03:36:35.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:35.716+0000 7fde9d670640 1 -- 172.21.15.38:0/3981183923 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde981f31e0 con 0x7fde98158da0 2024-09-21T03:36:35.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:35.717+0000 7fde9d670640 1 -- 172.21.15.38:0/3981183923 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde980ce300 con 0x7fde98158da0 2024-09-21T03:36:35.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:35.718+0000 7fde8f7fe640 1 -- 172.21.15.38:0/3981183923 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fde80035020 con 0x7fde98158da0 2024-09-21T03:36:35.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:35.718+0000 7fde8f7fe640 1 --2- 172.21.15.38:0/3981183923 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fde78078b90 0x7fde7807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:35.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:35.718+0000 7fde8f7fe640 1 -- 172.21.15.38:0/3981183923 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6101+0+0 (secure 0 0 0) 0x7fde800bb8c0 con 0x7fde98158da0 2024-09-21T03:36:35.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:35.719+0000 7fde96ffd640 1 --2- 172.21.15.38:0/3981183923 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fde78078b90 0x7fde7807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:35.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:35.719+0000 7fde96ffd640 1 --2- 172.21.15.38:0/3981183923 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fde78078b90 0x7fde7807b0a0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fde88009af0 tx=0x7fde88005f90 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:35.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:35.723+0000 7fde8f7fe640 1 -- 172.21.15.38:0/3981183923 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fde80046060 con 0x7fde98158da0 2024-09-21T03:36:35.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:35.925+0000 7fde9d670640 1 -- 172.21.15.38:0/3981183923 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fde9806fe80 con 0x7fde98158da0 2024-09-21T03:36:35.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:35.925+0000 7fde8f7fe640 1 -- 172.21.15.38:0/3981183923 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v67) ==== 80+0+0 (secure 0 0 0) 0x7fde980b2730 con 0x7fde98158da0 2024-09-21T03:36:35.928 INFO:teuthology.orchestra.run.smithi038.stderr:listed 0 entries 2024-09-21T03:36:35.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:35.927+0000 7fde9d670640 1 -- 172.21.15.38:0/3981183923 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fde78078b90 msgr2=0x7fde7807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:35.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:35.927+0000 7fde9d670640 1 --2- 172.21.15.38:0/3981183923 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fde78078b90 0x7fde7807b0a0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fde88009af0 tx=0x7fde88005f90 comp rx=0 tx=0).stop 2024-09-21T03:36:35.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:35.927+0000 7fde9d670640 1 -- 172.21.15.38:0/3981183923 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fde98158da0 msgr2=0x7fde98158500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:35.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:35.927+0000 7fde9d670640 1 --2- 172.21.15.38:0/3981183923 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fde98158da0 0x7fde98158500 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fde80031040 tx=0x7fde80006420 comp rx=0 tx=0).stop 2024-09-21T03:36:35.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:35.927+0000 7fde9d670640 1 -- 172.21.15.38:0/3981183923 shutdown_connections 2024-09-21T03:36:35.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:35.927+0000 7fde9d670640 1 --2- 172.21.15.38:0/3981183923 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fde980b1de0 0x7fde980b21f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:35.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:35.927+0000 7fde9d670640 1 --2- 172.21.15.38:0/3981183923 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fde98158da0 0x7fde98158500 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:35.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:35.927+0000 7fde9d670640 1 --2- 172.21.15.38:0/3981183923 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fde78078b90 0x7fde7807b0a0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:35.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:35.927+0000 7fde9d670640 1 --2- 172.21.15.38:0/3981183923 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fde84003fd0 0x7fde981e69a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:35.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:35.927+0000 7fde9d670640 1 -- 172.21.15.38:0/3981183923 >> 172.21.15.38:0/3981183923 conn(0x7fde980cdca0 msgr2=0x7fde9806edf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:35.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:35.927+0000 7fde9d670640 1 -- 172.21.15.38:0/3981183923 shutdown_connections 2024-09-21T03:36:35.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:35.927+0000 7fde9d670640 1 -- 172.21.15.38:0/3981183923 wait complete. 2024-09-21T03:36:35.942 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-21T03:36:35.942 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:36:35.942 DEBUG:teuthology.orchestra.run.smithi038:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-21T03:36:35.973 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi038.front.sepia.ceph.com 2024-09-21T03:36:35.973 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:36:35.973 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T03:36:36.030 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-21T03:36:36.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:36.184+0000 7fc8a8c84640 1 -- 172.21.15.38:0/223019706 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc8a415ee80 msgr2=0x7fc8a40d0170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:36.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:36.184+0000 7fc8a8c84640 1 --2- 172.21.15.38:0/223019706 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc8a415ee80 0x7fc8a40d0170 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fc8a40b3c20 tx=0x7fc89402a410 comp rx=0 tx=0).stop 2024-09-21T03:36:36.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:36.184+0000 7fc8a8c84640 1 -- 172.21.15.38:0/223019706 shutdown_connections 2024-09-21T03:36:36.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:36.184+0000 7fc8a8c84640 1 --2- 172.21.15.38:0/223019706 >> v2:172.21.15.195:3300/0 conn(0x7fc8a40ceb90 0x7fc8a40cefa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:36.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:36.184+0000 7fc8a8c84640 1 --2- 172.21.15.38:0/223019706 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc8a415ee80 0x7fc8a40d0170 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:36.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:36.184+0000 7fc8a8c84640 1 -- 172.21.15.38:0/223019706 >> 172.21.15.38:0/223019706 conn(0x7fc8a409d830 msgr2=0x7fc8a40b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:36.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:36.184+0000 7fc8a8c84640 1 -- 172.21.15.38:0/223019706 shutdown_connections 2024-09-21T03:36:36.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:36.184+0000 7fc8a8c84640 1 -- 172.21.15.38:0/223019706 wait complete. 2024-09-21T03:36:36.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:36.186+0000 7fc8a8c84640 1 Processor -- start 2024-09-21T03:36:36.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:36.186+0000 7fc8a8c84640 1 -- start start 2024-09-21T03:36:36.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:36.187+0000 7fc8a8c84640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc8a40b9ad0 0x7fc8a414fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:36.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:36.187+0000 7fc8a8c84640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc8a40ceb90 0x7fc8a40b21e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:36.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:36.187+0000 7fc8a8c84640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc8a415ee80 0x7fc8a40b2720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:36.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:36.187+0000 7fc8a8c84640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fc8a40b6700 con 0x7fc8a40b9ad0 2024-09-21T03:36:36.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:36.187+0000 7fc8a8c84640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fc8a40b6580 con 0x7fc8a40ceb90 2024-09-21T03:36:36.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:36.187+0000 7fc8a8c84640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fc8a40b6880 con 0x7fc8a415ee80 2024-09-21T03:36:36.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:36.187+0000 7fc8a259c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc8a40b9ad0 0x7fc8a414fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:36.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:36.187+0000 7fc8a259c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc8a40b9ad0 0x7fc8a414fa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40248/0 (socket says 172.21.15.38:40248) 2024-09-21T03:36:36.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:36.187+0000 7fc8a259c640 1 -- 172.21.15.38:0/64306356 learned_addr learned my addr 172.21.15.38:0/64306356 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:36:36.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:36.188+0000 7fc8a2d9d640 1 --2- 172.21.15.38:0/64306356 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc8a415ee80 0x7fc8a40b2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:36.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:36.188+0000 7fc8a1d9b640 1 --2- 172.21.15.38:0/64306356 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc8a40ceb90 0x7fc8a40b21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:36.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:36.188+0000 7fc8a259c640 1 -- 172.21.15.38:0/64306356 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc8a415ee80 msgr2=0x7fc8a40b2720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:36.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:36.188+0000 7fc8a259c640 1 --2- 172.21.15.38:0/64306356 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc8a415ee80 0x7fc8a40b2720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:36.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:36.188+0000 7fc8a259c640 1 -- 172.21.15.38:0/64306356 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc8a40ceb90 msgr2=0x7fc8a40b21e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:36.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:36.188+0000 7fc8a259c640 1 --2- 172.21.15.38:0/64306356 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc8a40ceb90 0x7fc8a40b21e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:36.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:36.188+0000 7fc8a259c640 1 -- 172.21.15.38:0/64306356 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8a415c4e0 con 0x7fc8a40b9ad0 2024-09-21T03:36:36.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:36.188+0000 7fc8a2d9d640 1 --2- 172.21.15.38:0/64306356 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc8a415ee80 0x7fc8a40b2720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:36:36.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:36.188+0000 7fc8a259c640 1 --2- 172.21.15.38:0/64306356 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc8a40b9ad0 0x7fc8a414fa90 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fc89402aa00 tx=0x7fc894033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:36.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:36.188+0000 7fc892ffd640 1 -- 172.21.15.38:0/64306356 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc894031d60 con 0x7fc8a40b9ad0 2024-09-21T03:36:36.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:36.188+0000 7fc892ffd640 1 -- 172.21.15.38:0/64306356 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc894033e10 con 0x7fc8a40b9ad0 2024-09-21T03:36:36.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:36.188+0000 7fc8a8c84640 1 -- 172.21.15.38:0/64306356 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8a406dfb0 con 0x7fc8a40b9ad0 2024-09-21T03:36:36.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:36.188+0000 7fc892ffd640 1 -- 172.21.15.38:0/64306356 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc894005260 con 0x7fc8a40b9ad0 2024-09-21T03:36:36.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:36.188+0000 7fc8a8c84640 1 -- 172.21.15.38:0/64306356 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8a406e570 con 0x7fc8a40b9ad0 2024-09-21T03:36:36.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:36.189+0000 7fc8a8c84640 1 -- 172.21.15.38:0/64306356 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8a40cf260 con 0x7fc8a40b9ad0 2024-09-21T03:36:36.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:36.190+0000 7fc892ffd640 1 -- 172.21.15.38:0/64306356 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc894005400 con 0x7fc8a40b9ad0 2024-09-21T03:36:36.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:36.191+0000 7fc892ffd640 1 --2- 172.21.15.38:0/64306356 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc884078b90 0x7fc88407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:36.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:36.191+0000 7fc892ffd640 1 -- 172.21.15.38:0/64306356 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6101+0+0 (secure 0 0 0) 0x7fc8940bc4b0 con 0x7fc8a40b9ad0 2024-09-21T03:36:36.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:36.191+0000 7fc8a1d9b640 1 --2- 172.21.15.38:0/64306356 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc884078b90 0x7fc88407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:36.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:36.191+0000 7fc8a1d9b640 1 --2- 172.21.15.38:0/64306356 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc884078b90 0x7fc88407b0a0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fc8a4157cf0 tx=0x7fc898009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:36.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:36.195+0000 7fc892ffd640 1 -- 172.21.15.38:0/64306356 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fc8a40cf260 con 0x7fc8a40b9ad0 2024-09-21T03:36:36.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:36.418+0000 7fc8a8c84640 1 -- 172.21.15.38:0/64306356 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fc8a40b6df0 con 0x7fc8a40b9ad0 2024-09-21T03:36:36.421 INFO:tasks.ceph.mon.a.smithi038.stderr:2024-09-21T03:36:36.419+0000 7f3f45713640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T03:36:36.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:36.422+0000 7fc892ffd640 1 -- 172.21.15.38:0/64306356 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v46) ==== 161+0+0 (secure 0 0 0) 0x7fc89403f030 con 0x7fc8a40b9ad0 2024-09-21T03:36:36.425 INFO:teuthology.orchestra.run.smithi038.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-21T03:36:36.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:36.424+0000 7fc8a8c84640 1 -- 172.21.15.38:0/64306356 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc884078b90 msgr2=0x7fc88407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:36.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:36.424+0000 7fc8a8c84640 1 --2- 172.21.15.38:0/64306356 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc884078b90 0x7fc88407b0a0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fc8a4157cf0 tx=0x7fc898009040 comp rx=0 tx=0).stop 2024-09-21T03:36:36.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:36.424+0000 7fc8a8c84640 1 -- 172.21.15.38:0/64306356 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc8a40b9ad0 msgr2=0x7fc8a414fa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:36.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:36.424+0000 7fc8a8c84640 1 --2- 172.21.15.38:0/64306356 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc8a40b9ad0 0x7fc8a414fa90 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fc89402aa00 tx=0x7fc894033c20 comp rx=0 tx=0).stop 2024-09-21T03:36:36.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:36.425+0000 7fc8a8c84640 1 -- 172.21.15.38:0/64306356 shutdown_connections 2024-09-21T03:36:36.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:36.425+0000 7fc8a8c84640 1 --2- 172.21.15.38:0/64306356 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc884078b90 0x7fc88407b0a0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:36.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:36.425+0000 7fc8a8c84640 1 --2- 172.21.15.38:0/64306356 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc8a415ee80 0x7fc8a40b2720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:36.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:36.425+0000 7fc8a8c84640 1 --2- 172.21.15.38:0/64306356 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc8a40ceb90 0x7fc8a40b21e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:36.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:36.425+0000 7fc8a8c84640 1 --2- 172.21.15.38:0/64306356 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc8a40b9ad0 0x7fc8a414fa90 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:36.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:36.425+0000 7fc8a8c84640 1 -- 172.21.15.38:0/64306356 >> 172.21.15.38:0/64306356 conn(0x7fc8a409d830 msgr2=0x7fc8a415e200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:36.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:36.425+0000 7fc8a8c84640 1 -- 172.21.15.38:0/64306356 shutdown_connections 2024-09-21T03:36:36.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:36.425+0000 7fc8a8c84640 1 -- 172.21.15.38:0/64306356 wait complete. 2024-09-21T03:36:36.431 INFO:tasks.ceph.mds.a.smithi038.stderr: -5464> 2024-09-21T03:36:28.029+0000 7fd96d497640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T03:36:36.440 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_inject_journal_corrupt_dentry_first 1.0 2024-09-21T03:36:36.490 INFO:tasks.ceph.mds.a.smithi038.stdout:starting mds.a at 2024-09-21T03:36:36.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:36.560+0000 7f8b2a670640 1 -- 172.21.15.38:0/2467113341 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8b240b9ad0 msgr2=0x7f8b240d0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:36.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:36.560+0000 7f8b2a670640 1 --2- 172.21.15.38:0/2467113341 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8b240b9ad0 0x7f8b240d0be0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f8b240cfba0 tx=0x7f8b0c02a4d0 comp rx=0 tx=0).stop 2024-09-21T03:36:36.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:36.560+0000 7f8b2a670640 1 -- 172.21.15.38:0/2467113341 shutdown_connections 2024-09-21T03:36:36.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:36.560+0000 7f8b2a670640 1 --2- 172.21.15.38:0/2467113341 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8b240b9ad0 0x7f8b240d0be0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:36.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:36.560+0000 7f8b2a670640 1 --2- 172.21.15.38:0/2467113341 >> v2:172.21.15.195:3300/0 conn(0x7f8b2415ee80 0x7f8b240d0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:36.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:36.560+0000 7f8b2a670640 1 -- 172.21.15.38:0/2467113341 >> 172.21.15.38:0/2467113341 conn(0x7f8b2409d830 msgr2=0x7f8b240b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:36.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:36.560+0000 7f8b2a670640 1 -- 172.21.15.38:0/2467113341 shutdown_connections 2024-09-21T03:36:36.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:36.560+0000 7f8b2a670640 1 -- 172.21.15.38:0/2467113341 wait complete. 2024-09-21T03:36:36.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:36.562+0000 7f8b2a670640 1 Processor -- start 2024-09-21T03:36:36.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:36.562+0000 7f8b2a670640 1 -- start start 2024-09-21T03:36:36.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:36.563+0000 7f8b2a670640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8b240b9ad0 0x7f8b24157a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:36.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:36.563+0000 7f8b2a670640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8b240d1120 0x7f8b240cb610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:36.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:36.563+0000 7f8b2a670640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8b2415ee80 0x7f8b240b2c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:36.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:36.563+0000 7f8b2a670640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f8b240b6700 con 0x7f8b2415ee80 2024-09-21T03:36:36.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:36.563+0000 7f8b2a670640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f8b240b6580 con 0x7f8b240b9ad0 2024-09-21T03:36:36.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:36.563+0000 7f8b2a670640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f8b240b6880 con 0x7f8b240d1120 2024-09-21T03:36:36.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:36.563+0000 7f8b28c0d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8b2415ee80 0x7f8b240b2c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:36.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:36.563+0000 7f8b28c0d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8b2415ee80 0x7f8b240b2c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40298/0 (socket says 172.21.15.38:40298) 2024-09-21T03:36:36.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:36.563+0000 7f8b28c0d640 1 -- 172.21.15.38:0/3020992462 learned_addr learned my addr 172.21.15.38:0/3020992462 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:36:36.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:36.563+0000 7f8b23fff640 1 --2- 172.21.15.38:0/3020992462 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8b240b9ad0 0x7f8b24157a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:36.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:36.563+0000 7f8b1bfff640 1 --2- 172.21.15.38:0/3020992462 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8b240d1120 0x7f8b240cb610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:36.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:36.564+0000 7f8b28c0d640 1 -- 172.21.15.38:0/3020992462 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8b240d1120 msgr2=0x7f8b240cb610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:36.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:36.564+0000 7f8b28c0d640 1 --2- 172.21.15.38:0/3020992462 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8b240d1120 0x7f8b240cb610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:36.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:36.564+0000 7f8b28c0d640 1 -- 172.21.15.38:0/3020992462 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8b240b9ad0 msgr2=0x7f8b24157a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:36.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:36.564+0000 7f8b28c0d640 1 --2- 172.21.15.38:0/3020992462 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8b240b9ad0 0x7f8b24157a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:36.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:36.564+0000 7f8b28c0d640 1 -- 172.21.15.38:0/3020992462 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b24157fc0 con 0x7f8b2415ee80 2024-09-21T03:36:36.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:36.564+0000 7f8b23fff640 1 --2- 172.21.15.38:0/3020992462 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8b240b9ad0 0x7f8b24157a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:36:36.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:36.564+0000 7f8b28c0d640 1 --2- 172.21.15.38:0/3020992462 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8b2415ee80 0x7f8b240b2c90 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f8b1c00c700 tx=0x7f8b1c00cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:36.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:36.564+0000 7f8b217fa640 1 -- 172.21.15.38:0/3020992462 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b1c003b70 con 0x7f8b2415ee80 2024-09-21T03:36:36.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:36.564+0000 7f8b217fa640 1 -- 172.21.15.38:0/3020992462 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8b1c003d10 con 0x7f8b2415ee80 2024-09-21T03:36:36.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:36.564+0000 7f8b2a670640 1 -- 172.21.15.38:0/3020992462 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b240ce400 con 0x7f8b2415ee80 2024-09-21T03:36:36.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:36.564+0000 7f8b217fa640 1 -- 172.21.15.38:0/3020992462 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b1c00aab0 con 0x7f8b2415ee80 2024-09-21T03:36:36.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:36.564+0000 7f8b2a670640 1 -- 172.21.15.38:0/3020992462 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b241fd140 con 0x7f8b2415ee80 2024-09-21T03:36:36.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:36.565+0000 7f8b2a670640 1 -- 172.21.15.38:0/3020992462 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b240d4bf0 con 0x7f8b2415ee80 2024-09-21T03:36:36.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:36.566+0000 7f8b217fa640 1 -- 172.21.15.38:0/3020992462 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f8b1c00ac50 con 0x7f8b2415ee80 2024-09-21T03:36:36.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:36.567+0000 7f8b217fa640 1 --2- 172.21.15.38:0/3020992462 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8b04078bc0 0x7f8b0407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:36.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:36.567+0000 7f8b217fa640 1 -- 172.21.15.38:0/3020992462 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6187+0+0 (secure 0 0 0) 0x7f8b1c09ac00 con 0x7f8b2415ee80 2024-09-21T03:36:36.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:36.567+0000 7f8b23fff640 1 --2- 172.21.15.38:0/3020992462 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8b04078bc0 0x7f8b0407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:36.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:36.567+0000 7f8b23fff640 1 --2- 172.21.15.38:0/3020992462 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8b04078bc0 0x7f8b0407b0d0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f8b1400bf40 tx=0x7f8b1400b2e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:36.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:36.572+0000 7f8b217fa640 1 -- 172.21.15.38:0/3020992462 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f8b240d4bf0 con 0x7f8b2415ee80 2024-09-21T03:36:36.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:36.761+0000 7f8b2a670640 1 -- 172.21.15.38:0/3020992462 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command([{prefix=config set, name=mds_inject_journal_corrupt_dentry_first}] v 0) -- 0x7f8b240b6d30 con 0x7f8b2415ee80 2024-09-21T03:36:36.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:36.767+0000 7f8b217fa640 1 -- 172.21.15.38:0/3020992462 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_inject_journal_corrupt_dentry_first}]=0 v40)=0 v40) ==== 139+0+0 (secure 0 0 0) 0x7f8b24157fc0 con 0x7f8b2415ee80 2024-09-21T03:36:36.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:36.769+0000 7f8b2a670640 1 -- 172.21.15.38:0/3020992462 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8b04078bc0 msgr2=0x7f8b0407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:36.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:36.769+0000 7f8b2a670640 1 --2- 172.21.15.38:0/3020992462 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8b04078bc0 0x7f8b0407b0d0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f8b1400bf40 tx=0x7f8b1400b2e0 comp rx=0 tx=0).stop 2024-09-21T03:36:36.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:36.769+0000 7f8b2a670640 1 -- 172.21.15.38:0/3020992462 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8b2415ee80 msgr2=0x7f8b240b2c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:36.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:36.769+0000 7f8b2a670640 1 --2- 172.21.15.38:0/3020992462 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8b2415ee80 0x7f8b240b2c90 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f8b1c00c700 tx=0x7f8b1c00cbc0 comp rx=0 tx=0).stop 2024-09-21T03:36:36.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:36.769+0000 7f8b2a670640 1 -- 172.21.15.38:0/3020992462 shutdown_connections 2024-09-21T03:36:36.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:36.769+0000 7f8b2a670640 1 --2- 172.21.15.38:0/3020992462 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8b2415ee80 0x7f8b240b2c90 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:36.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:36.769+0000 7f8b2a670640 1 --2- 172.21.15.38:0/3020992462 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8b04078bc0 0x7f8b0407b0d0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:36.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:36.769+0000 7f8b2a670640 1 --2- 172.21.15.38:0/3020992462 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8b240d1120 0x7f8b240cb610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:36.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:36.769+0000 7f8b2a670640 1 --2- 172.21.15.38:0/3020992462 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8b240b9ad0 0x7f8b24157a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:36.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:36.769+0000 7f8b2a670640 1 -- 172.21.15.38:0/3020992462 >> 172.21.15.38:0/3020992462 conn(0x7f8b2409d830 msgr2=0x7f8b24160dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:36.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:36.769+0000 7f8b2a670640 1 -- 172.21.15.38:0/3020992462 shutdown_connections 2024-09-21T03:36:36.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:36.769+0000 7f8b2a670640 1 -- 172.21.15.38:0/3020992462 wait complete. 2024-09-21T03:36:41.783 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-21T03:36:41.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:41.917+0000 7f9d366de640 1 -- 172.21.15.38:0/1869678793 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9d18003c10 msgr2=0x7f9d18004060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:41.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:41.917+0000 7f9d366de640 1 --2- 172.21.15.38:0/1869678793 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9d18003c10 0x7f9d18004060 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f9d2000c4a0 tx=0x7f9d18005830 comp rx=0 tx=0).stop 2024-09-21T03:36:41.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:41.917+0000 7f9d34c7b640 1 -- 172.21.15.38:0/1869678793 reap_dead start 2024-09-21T03:36:41.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:41.917+0000 7f9d366de640 1 -- 172.21.15.38:0/1869678793 shutdown_connections 2024-09-21T03:36:41.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:41.917+0000 7f9d366de640 1 -- 172.21.15.38:0/1869678793 >> 172.21.15.38:0/1869678793 conn(0x7f9d3009d830 msgr2=0x7f9d300b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:41.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:41.917+0000 7f9d366de640 1 -- 172.21.15.38:0/1869678793 shutdown_connections 2024-09-21T03:36:41.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:41.917+0000 7f9d366de640 1 -- 172.21.15.38:0/1869678793 wait complete. 2024-09-21T03:36:41.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:41.919+0000 7f9d366de640 1 Processor -- start 2024-09-21T03:36:41.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:41.919+0000 7f9d366de640 1 -- start start 2024-09-21T03:36:41.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:41.921+0000 7f9d366de640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9d18003c10 0x7f9d30157c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:41.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:41.921+0000 7f9d366de640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9d3006cde0 0x7f9d3006d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:41.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:41.921+0000 7f9d366de640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9d300d4580 0x7f9d300d4990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:41.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:41.921+0000 7f9d366de640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f9d300cf330 con 0x7f9d18003c10 2024-09-21T03:36:41.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:41.921+0000 7f9d366de640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f9d300cf1b0 con 0x7f9d3006cde0 2024-09-21T03:36:41.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:41.921+0000 7f9d366de640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f9d300b6580 con 0x7f9d300d4580 2024-09-21T03:36:41.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:41.921+0000 7f9d2ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9d18003c10 0x7f9d30157c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:41.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:41.921+0000 7f9d2ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9d18003c10 0x7f9d30157c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40308/0 (socket says 172.21.15.38:40308) 2024-09-21T03:36:41.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:41.921+0000 7f9d2ffff640 1 -- 172.21.15.38:0/4157501865 learned_addr learned my addr 172.21.15.38:0/4157501865 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:36:41.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:41.921+0000 7f9d34c7b640 1 --2- 172.21.15.38:0/4157501865 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9d300d4580 0x7f9d300d4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:41.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:41.921+0000 7f9d2f7fe640 1 --2- 172.21.15.38:0/4157501865 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9d3006cde0 0x7f9d3006d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:41.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:41.921+0000 7f9d2ffff640 1 -- 172.21.15.38:0/4157501865 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9d300d4580 msgr2=0x7f9d300d4990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:41.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:41.921+0000 7f9d2ffff640 1 --2- 172.21.15.38:0/4157501865 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9d300d4580 0x7f9d300d4990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:41.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:41.921+0000 7f9d2ffff640 1 -- 172.21.15.38:0/4157501865 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9d3006cde0 msgr2=0x7f9d3006d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:41.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:41.921+0000 7f9d2ffff640 1 --2- 172.21.15.38:0/4157501865 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9d3006cde0 0x7f9d3006d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:41.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:41.921+0000 7f9d2ffff640 1 -- 172.21.15.38:0/4157501865 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d300d4ed0 con 0x7f9d18003c10 2024-09-21T03:36:41.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:41.921+0000 7f9d34c7b640 1 --2- 172.21.15.38:0/4157501865 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9d300d4580 0x7f9d300d4990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:36:41.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:41.921+0000 7f9d2f7fe640 1 --2- 172.21.15.38:0/4157501865 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9d3006cde0 0x7f9d3006d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:36:41.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:41.921+0000 7f9d2ffff640 1 --2- 172.21.15.38:0/4157501865 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9d18003c10 0x7f9d30157c30 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f9d2000c4d0 tx=0x7f9d20006310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:41.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:41.922+0000 7f9d2cff9640 1 -- 172.21.15.38:0/4157501865 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d20006da0 con 0x7f9d18003c10 2024-09-21T03:36:41.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:41.922+0000 7f9d2cff9640 1 -- 172.21.15.38:0/4157501865 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9d200038e0 con 0x7f9d18003c10 2024-09-21T03:36:41.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:41.922+0000 7f9d2cff9640 1 -- 172.21.15.38:0/4157501865 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d20003d10 con 0x7f9d18003c10 2024-09-21T03:36:41.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:41.922+0000 7f9d366de640 1 -- 172.21.15.38:0/4157501865 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d3006db10 con 0x7f9d18003c10 2024-09-21T03:36:41.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:41.922+0000 7f9d366de640 1 -- 172.21.15.38:0/4157501865 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d3006f080 con 0x7f9d18003c10 2024-09-21T03:36:41.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:41.923+0000 7f9d366de640 1 -- 172.21.15.38:0/4157501865 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d300b6ae0 con 0x7f9d18003c10 2024-09-21T03:36:41.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:41.924+0000 7f9d2cff9640 1 -- 172.21.15.38:0/4157501865 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f9d2002b5b0 con 0x7f9d18003c10 2024-09-21T03:36:41.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:41.924+0000 7f9d2cff9640 1 --2- 172.21.15.38:0/4157501865 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9d1c078bc0 0x7f9d1c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:41.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:41.925+0000 7f9d2cff9640 1 -- 172.21.15.38:0/4157501865 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6187+0+0 (secure 0 0 0) 0x7f9d200bb3f0 con 0x7f9d18003c10 2024-09-21T03:36:41.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:41.925+0000 7f9d2f7fe640 1 --2- 172.21.15.38:0/4157501865 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9d1c078bc0 0x7f9d1c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:41.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:41.925+0000 7f9d2f7fe640 1 --2- 172.21.15.38:0/4157501865 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9d1c078bc0 0x7f9d1c07b0d0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f9d300cafd0 tx=0x7f9d24005040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:41.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:41.929+0000 7f9d2cff9640 1 -- 172.21.15.38:0/4157501865 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f9d20045060 con 0x7f9d18003c10 2024-09-21T03:36:42.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:42.159+0000 7f9d366de640 1 -- 172.21.15.38:0/4157501865 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f9d300b17f0 con 0x7f9d18003c10 2024-09-21T03:36:42.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:42.437+0000 7f9d2cff9640 1 -- 172.21.15.38:0/4157501865 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v48) ==== 162+0+0 (secure 0 0 0) 0x7f9d300d4ed0 con 0x7f9d18003c10 2024-09-21T03:36:42.439 INFO:teuthology.orchestra.run.smithi038.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-21T03:36:42.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:42.439+0000 7f9d366de640 1 -- 172.21.15.38:0/4157501865 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9d1c078bc0 msgr2=0x7f9d1c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:42.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:42.439+0000 7f9d366de640 1 --2- 172.21.15.38:0/4157501865 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9d1c078bc0 0x7f9d1c07b0d0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f9d300cafd0 tx=0x7f9d24005040 comp rx=0 tx=0).stop 2024-09-21T03:36:42.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:42.439+0000 7f9d366de640 1 -- 172.21.15.38:0/4157501865 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9d18003c10 msgr2=0x7f9d30157c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:42.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:42.439+0000 7f9d366de640 1 --2- 172.21.15.38:0/4157501865 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9d18003c10 0x7f9d30157c30 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f9d2000c4d0 tx=0x7f9d20006310 comp rx=0 tx=0).stop 2024-09-21T03:36:42.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:42.439+0000 7f9d366de640 1 -- 172.21.15.38:0/4157501865 shutdown_connections 2024-09-21T03:36:42.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:42.439+0000 7f9d366de640 1 --2- 172.21.15.38:0/4157501865 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9d1c078bc0 0x7f9d1c07b0d0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:42.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:42.439+0000 7f9d366de640 1 --2- 172.21.15.38:0/4157501865 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9d300d4580 0x7f9d300d4990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:42.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:42.439+0000 7f9d366de640 1 --2- 172.21.15.38:0/4157501865 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9d3006cde0 0x7f9d3006d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:42.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:42.439+0000 7f9d366de640 1 --2- 172.21.15.38:0/4157501865 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9d18003c10 0x7f9d30157c30 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:42.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:42.439+0000 7f9d366de640 1 -- 172.21.15.38:0/4157501865 >> 172.21.15.38:0/4157501865 conn(0x7f9d3009d830 msgr2=0x7f9d30161300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:42.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:42.439+0000 7f9d366de640 1 -- 172.21.15.38:0/4157501865 shutdown_connections 2024-09-21T03:36:42.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:42.439+0000 7f9d366de640 1 -- 172.21.15.38:0/4157501865 wait complete. 2024-09-21T03:36:42.455 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:36:42.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:42.570+0000 7f1e8567e640 1 -- 172.21.15.38:0/1794755835 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1e800d0b40 msgr2=0x7f1e8006e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:42.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:42.570+0000 7f1e8567e640 1 --2- 172.21.15.38:0/1794755835 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1e800d0b40 0x7f1e8006e6f0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f1e8006e3e0 tx=0x7f1e7002a5d0 comp rx=0 tx=0).stop 2024-09-21T03:36:42.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:42.570+0000 7f1e8567e640 1 -- 172.21.15.38:0/1794755835 shutdown_connections 2024-09-21T03:36:42.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:42.570+0000 7f1e8567e640 1 --2- 172.21.15.38:0/1794755835 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1e800d0b40 0x7f1e8006e6f0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:42.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:42.570+0000 7f1e8567e640 1 -- 172.21.15.38:0/1794755835 >> 172.21.15.38:0/1794755835 conn(0x7f1e800b2cc0 msgr2=0x7f1e800b30d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:42.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:42.570+0000 7f1e8567e640 1 -- 172.21.15.38:0/1794755835 shutdown_connections 2024-09-21T03:36:42.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:42.570+0000 7f1e8567e640 1 -- 172.21.15.38:0/1794755835 wait complete. 2024-09-21T03:36:42.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:42.572+0000 7f1e8567e640 1 Processor -- start 2024-09-21T03:36:42.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:42.572+0000 7f1e8567e640 1 -- start start 2024-09-21T03:36:42.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:42.574+0000 7f1e8567e640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1e8006ec30 0x7f1e800ae310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:42.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:42.574+0000 7f1e8567e640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1e80073d50 0x7f1e80151bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:42.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:42.574+0000 7f1e8567e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1e800d0b40 0x7f1e801f57a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:42.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:42.574+0000 7f1e8567e640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f1e8015b960 con 0x7f1e800d0b40 2024-09-21T03:36:42.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:42.574+0000 7f1e8567e640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f1e8015b7e0 con 0x7f1e80073d50 2024-09-21T03:36:42.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:42.574+0000 7f1e8567e640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f1e8015bae0 con 0x7f1e8006ec30 2024-09-21T03:36:42.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:42.574+0000 7f1e7f7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1e800d0b40 0x7f1e801f57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:42.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:42.574+0000 7f1e7f7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1e800d0b40 0x7f1e801f57a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:36556/0 (socket says 172.21.15.38:36556) 2024-09-21T03:36:42.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:42.574+0000 7f1e7f7fe640 1 -- 172.21.15.38:0/2504273013 learned_addr learned my addr 172.21.15.38:0/2504273013 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:36:42.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:42.574+0000 7f1e7e7fc640 1 --2- 172.21.15.38:0/2504273013 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1e80073d50 0x7f1e80151bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:42.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:42.574+0000 7f1e7effd640 1 --2- 172.21.15.38:0/2504273013 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1e8006ec30 0x7f1e800ae310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:42.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:42.574+0000 7f1e7f7fe640 1 -- 172.21.15.38:0/2504273013 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1e8006ec30 msgr2=0x7f1e800ae310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:42.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:42.574+0000 7f1e7f7fe640 1 --2- 172.21.15.38:0/2504273013 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1e8006ec30 0x7f1e800ae310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:42.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:42.574+0000 7f1e7f7fe640 1 -- 172.21.15.38:0/2504273013 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1e80073d50 msgr2=0x7f1e80151bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:42.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:42.574+0000 7f1e7f7fe640 1 --2- 172.21.15.38:0/2504273013 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1e80073d50 0x7f1e80151bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:42.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:42.574+0000 7f1e7f7fe640 1 -- 172.21.15.38:0/2504273013 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e800ad850 con 0x7f1e800d0b40 2024-09-21T03:36:42.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:42.574+0000 7f1e7effd640 1 --2- 172.21.15.38:0/2504273013 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1e8006ec30 0x7f1e800ae310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:36:42.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:42.574+0000 7f1e7f7fe640 1 --2- 172.21.15.38:0/2504273013 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1e800d0b40 0x7f1e801f57a0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f1e7800c6b0 tx=0x7f1e7800cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:42.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:42.574+0000 7f1e677fe640 1 -- 172.21.15.38:0/2504273013 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e78003b70 con 0x7f1e800d0b40 2024-09-21T03:36:42.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:42.574+0000 7f1e677fe640 1 -- 172.21.15.38:0/2504273013 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1e78003d30 con 0x7f1e800d0b40 2024-09-21T03:36:42.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:42.574+0000 7f1e8567e640 1 -- 172.21.15.38:0/2504273013 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e801f62d0 con 0x7f1e800d0b40 2024-09-21T03:36:42.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:42.574+0000 7f1e677fe640 1 -- 172.21.15.38:0/2504273013 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e78005330 con 0x7f1e800d0b40 2024-09-21T03:36:42.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:42.574+0000 7f1e8567e640 1 -- 172.21.15.38:0/2504273013 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e800adb10 con 0x7f1e800d0b40 2024-09-21T03:36:42.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:42.575+0000 7f1e8567e640 1 -- 172.21.15.38:0/2504273013 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e801f6040 con 0x7f1e800d0b40 2024-09-21T03:36:42.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:42.576+0000 7f1e677fe640 1 -- 172.21.15.38:0/2504273013 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f1e780054d0 con 0x7f1e800d0b40 2024-09-21T03:36:42.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:42.577+0000 7f1e677fe640 1 --2- 172.21.15.38:0/2504273013 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1e6c078b90 0x7f1e6c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:42.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:42.577+0000 7f1e677fe640 1 -- 172.21.15.38:0/2504273013 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6187+0+0 (secure 0 0 0) 0x7f1e7809ad40 con 0x7f1e800d0b40 2024-09-21T03:36:42.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:42.577+0000 7f1e7effd640 1 --2- 172.21.15.38:0/2504273013 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1e6c078b90 0x7f1e6c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:42.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:42.578+0000 7f1e7effd640 1 --2- 172.21.15.38:0/2504273013 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1e6c078b90 0x7f1e6c07b0a0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f1e70033000 tx=0x7f1e70004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:42.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:42.581+0000 7f1e677fe640 1 -- 172.21.15.38:0/2504273013 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f1e78062f10 con 0x7f1e800d0b40 2024-09-21T03:36:42.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:42.805+0000 7f1e8567e640 1 -- 172.21.15.38:0/2504273013 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1e8015d3a0 con 0x7f1e800d0b40 2024-09-21T03:36:42.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:42.806+0000 7f1e677fe640 1 -- 172.21.15.38:0/2504273013 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) ==== 94+0+5224 (secure 0 0 0) 0x7f1e78062810 con 0x7f1e800d0b40 2024-09-21T03:36:42.808 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:36:42.808 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":49,"btime":"2024-09-21T03:36:42:440566+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5432,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/1042764291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":1042764291},{"type":"v1","addr":"172.21.15.195:6837","nonce":1042764291}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":5728,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/3035019482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6833","nonce":3035019482},{"type":"v1","addr":"172.21.15.38:6835","nonce":3035019482}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":5731,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/2536177898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":2536177898},{"type":"v1","addr":"172.21.15.195:6836","nonce":2536177898}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:36:21.442724+0000","modified":"2024-09-21T03:36:42.440546+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":68,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6204},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6204":{"gid":6204,"name":"a","rank":0,"incarnation":49,"state":"up:replay","state_seq":1,"addr":"172.21.15.38:6834/1058376326","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":1058376326},{"type":"v1","addr":"172.21.15.38:6834","nonce":1058376326}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":4}]} 2024-09-21T03:36:42.808 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 49 2024-09-21T03:36:42.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:42.808+0000 7f1e8567e640 1 -- 172.21.15.38:0/2504273013 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1e6c078b90 msgr2=0x7f1e6c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:42.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:42.808+0000 7f1e8567e640 1 --2- 172.21.15.38:0/2504273013 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1e6c078b90 0x7f1e6c07b0a0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f1e70033000 tx=0x7f1e70004730 comp rx=0 tx=0).stop 2024-09-21T03:36:42.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:42.808+0000 7f1e8567e640 1 -- 172.21.15.38:0/2504273013 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1e800d0b40 msgr2=0x7f1e801f57a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:42.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:42.808+0000 7f1e8567e640 1 --2- 172.21.15.38:0/2504273013 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1e800d0b40 0x7f1e801f57a0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f1e7800c6b0 tx=0x7f1e7800cb70 comp rx=0 tx=0).stop 2024-09-21T03:36:42.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:42.808+0000 7f1e8567e640 1 -- 172.21.15.38:0/2504273013 shutdown_connections 2024-09-21T03:36:42.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:42.808+0000 7f1e8567e640 1 --2- 172.21.15.38:0/2504273013 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1e800d0b40 0x7f1e801f57a0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:42.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:42.808+0000 7f1e8567e640 1 --2- 172.21.15.38:0/2504273013 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1e80073d50 0x7f1e80151bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:42.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:42.808+0000 7f1e8567e640 1 --2- 172.21.15.38:0/2504273013 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1e6c078b90 0x7f1e6c07b0a0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:42.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:42.808+0000 7f1e8567e640 1 --2- 172.21.15.38:0/2504273013 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1e8006ec30 0x7f1e800ae310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:42.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:42.809+0000 7f1e8567e640 1 -- 172.21.15.38:0/2504273013 >> 172.21.15.38:0/2504273013 conn(0x7f1e800b2cc0 msgr2=0x7f1e800cc960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:42.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:42.809+0000 7f1e8567e640 1 -- 172.21.15.38:0/2504273013 shutdown_connections 2024-09-21T03:36:42.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:42.809+0000 7f1e8567e640 1 -- 172.21.15.38:0/2504273013 wait complete. 2024-09-21T03:36:43.224 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 49, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T03:36:21.442724+0000', 'modified': '2024-09-21T03:36:42.440546+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 68, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6204}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6204': {'gid': 6204, 'name': 'a', 'rank': 0, 'incarnation': 49, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.38:6834/1058376326', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.38:6832', 'nonce': 1058376326}, {'type': 'v1', 'addr': '172.21.15.38:6834', 'nonce': 1058376326}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-09-21T03:36:43.224 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6204:up:replay 2024-09-21T03:36:44.225 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:36:44.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:44.355+0000 7f20d2c8b640 1 -- 172.21.15.38:0/2623940290 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f20cc0cce70 msgr2=0x7f20cc0d04c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:44.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:44.355+0000 7f20d2c8b640 1 --2- 172.21.15.38:0/2623940290 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f20cc0cce70 0x7f20cc0d04c0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f20c4005fd0 tx=0x7f20c402aa90 comp rx=0 tx=0).stop 2024-09-21T03:36:44.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:44.355+0000 7f20d2c8b640 1 -- 172.21.15.38:0/2623940290 shutdown_connections 2024-09-21T03:36:44.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:44.355+0000 7f20d2c8b640 1 --2- 172.21.15.38:0/2623940290 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f20cc0cce70 0x7f20cc0d04c0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:44.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:44.355+0000 7f20d2c8b640 1 --2- 172.21.15.38:0/2623940290 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f20cc0d1ed0 0x7f20cc0cc930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:44.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:44.355+0000 7f20d2c8b640 1 -- 172.21.15.38:0/2623940290 >> 172.21.15.38:0/2623940290 conn(0x7f20cc0cef40 msgr2=0x7f20cc0cf350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:44.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:44.355+0000 7f20d2c8b640 1 -- 172.21.15.38:0/2623940290 shutdown_connections 2024-09-21T03:36:44.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:44.355+0000 7f20d2c8b640 1 -- 172.21.15.38:0/2623940290 wait complete. 2024-09-21T03:36:44.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:44.357+0000 7f20d2c8b640 1 Processor -- start 2024-09-21T03:36:44.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:44.357+0000 7f20d2c8b640 1 -- start start 2024-09-21T03:36:44.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:44.358+0000 7f20d2c8b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f20cc0cce70 0x7f20cc14fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:44.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:44.358+0000 7f20d2c8b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f20cc0d1ed0 0x7f20cc1e7310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:44.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:44.358+0000 7f20d2c8b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f20cc1536e0 0x7f20cc155480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:44.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:44.358+0000 7f20d2c8b640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f20cc0d0d90 con 0x7f20cc0d1ed0 2024-09-21T03:36:44.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:44.358+0000 7f20d2c8b640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f20cc0d0c10 con 0x7f20cc0cce70 2024-09-21T03:36:44.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:44.358+0000 7f20d2c8b640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f20cc0d0f10 con 0x7f20cc1536e0 2024-09-21T03:36:44.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:44.359+0000 7f20cbfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f20cc0d1ed0 0x7f20cc1e7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:44.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:44.359+0000 7f20cbfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f20cc0d1ed0 0x7f20cc1e7310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:36582/0 (socket says 172.21.15.38:36582) 2024-09-21T03:36:44.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:44.359+0000 7f20cbfff640 1 -- 172.21.15.38:0/1481604174 learned_addr learned my addr 172.21.15.38:0/1481604174 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:36:44.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:44.359+0000 7f20d1228640 1 --2- 172.21.15.38:0/1481604174 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f20cc1536e0 0x7f20cc155480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:44.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:44.359+0000 7f20d0a27640 1 --2- 172.21.15.38:0/1481604174 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f20cc0cce70 0x7f20cc14fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:44.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:44.359+0000 7f20cbfff640 1 -- 172.21.15.38:0/1481604174 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f20cc1536e0 msgr2=0x7f20cc155480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:44.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:44.359+0000 7f20cbfff640 1 --2- 172.21.15.38:0/1481604174 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f20cc1536e0 0x7f20cc155480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:44.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:44.359+0000 7f20cbfff640 1 -- 172.21.15.38:0/1481604174 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f20cc0cce70 msgr2=0x7f20cc14fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:44.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:44.359+0000 7f20cbfff640 1 --2- 172.21.15.38:0/1481604174 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f20cc0cce70 0x7f20cc14fbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:44.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:44.359+0000 7f20cbfff640 1 -- 172.21.15.38:0/1481604174 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20cc1546e0 con 0x7f20cc0d1ed0 2024-09-21T03:36:44.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:44.359+0000 7f20d1228640 1 --2- 172.21.15.38:0/1481604174 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f20cc1536e0 0x7f20cc155480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:36:44.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:44.359+0000 7f20d0a27640 1 --2- 172.21.15.38:0/1481604174 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f20cc0cce70 0x7f20cc14fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:36:44.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:44.359+0000 7f20cbfff640 1 --2- 172.21.15.38:0/1481604174 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f20cc0d1ed0 0x7f20cc1e7310 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f20c000d680 tx=0x7f20c000db40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:44.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:44.359+0000 7f20c97fa640 1 -- 172.21.15.38:0/1481604174 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c0017070 con 0x7f20cc0d1ed0 2024-09-21T03:36:44.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:44.359+0000 7f20c97fa640 1 -- 172.21.15.38:0/1481604174 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f20c0009b10 con 0x7f20cc0d1ed0 2024-09-21T03:36:44.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:44.359+0000 7f20d2c8b640 1 -- 172.21.15.38:0/1481604174 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20cc1549c0 con 0x7f20cc0d1ed0 2024-09-21T03:36:44.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:44.359+0000 7f20c97fa640 1 -- 172.21.15.38:0/1481604174 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c0010460 con 0x7f20cc0d1ed0 2024-09-21T03:36:44.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:44.359+0000 7f20d2c8b640 1 -- 172.21.15.38:0/1481604174 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20cc154f80 con 0x7f20cc0d1ed0 2024-09-21T03:36:44.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:44.361+0000 7f20d2c8b640 1 -- 172.21.15.38:0/1481604174 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20cc150230 con 0x7f20cc0d1ed0 2024-09-21T03:36:44.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:44.361+0000 7f20c97fa640 1 -- 172.21.15.38:0/1481604174 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f20c001a070 con 0x7f20cc0d1ed0 2024-09-21T03:36:44.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:44.362+0000 7f20c97fa640 1 --2- 172.21.15.38:0/1481604174 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f20b8078b90 0x7f20b807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:44.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:44.362+0000 7f20c97fa640 1 -- 172.21.15.38:0/1481604174 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6187+0+0 (secure 0 0 0) 0x7f20c009aff0 con 0x7f20cc0d1ed0 2024-09-21T03:36:44.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:44.363+0000 7f20d0a27640 1 --2- 172.21.15.38:0/1481604174 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f20b8078b90 0x7f20b807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:44.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:44.363+0000 7f20d0a27640 1 --2- 172.21.15.38:0/1481604174 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f20b8078b90 0x7f20b807b0a0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f20bc007fd0 tx=0x7f20bc009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:44.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:44.367+0000 7f20c97fa640 1 -- 172.21.15.38:0/1481604174 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f20c0063250 con 0x7f20cc0d1ed0 2024-09-21T03:36:44.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:44.590+0000 7f20d2c8b640 1 -- 172.21.15.38:0/1481604174 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f20cc0d04c0 con 0x7f20cc0d1ed0 2024-09-21T03:36:44.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:44.591+0000 7f20c97fa640 1 -- 172.21.15.38:0/1481604174 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) ==== 94+0+5224 (secure 0 0 0) 0x7f20c0062b50 con 0x7f20cc0d1ed0 2024-09-21T03:36:44.593 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:36:44.593 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":51,"btime":"2024-09-21T03:36:44:452863+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5432,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/1042764291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":1042764291},{"type":"v1","addr":"172.21.15.195:6837","nonce":1042764291}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":5728,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/3035019482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6833","nonce":3035019482},{"type":"v1","addr":"172.21.15.38:6835","nonce":3035019482}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":5731,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/2536177898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":2536177898},{"type":"v1","addr":"172.21.15.195:6836","nonce":2536177898}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":51,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:36:21.442724+0000","modified":"2024-09-21T03:36:43.454690+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":68,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6204},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6204":{"gid":6204,"name":"a","rank":0,"incarnation":49,"state":"up:rejoin","state_seq":4,"addr":"172.21.15.38:6834/1058376326","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":1058376326},{"type":"v1","addr":"172.21.15.38:6834","nonce":1058376326}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":4}]} 2024-09-21T03:36:44.593 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 51 2024-09-21T03:36:44.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:44.593+0000 7f20d2c8b640 1 -- 172.21.15.38:0/1481604174 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f20b8078b90 msgr2=0x7f20b807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:44.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:44.593+0000 7f20d2c8b640 1 --2- 172.21.15.38:0/1481604174 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f20b8078b90 0x7f20b807b0a0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f20bc007fd0 tx=0x7f20bc009040 comp rx=0 tx=0).stop 2024-09-21T03:36:44.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:44.593+0000 7f20d2c8b640 1 -- 172.21.15.38:0/1481604174 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f20cc0d1ed0 msgr2=0x7f20cc1e7310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:44.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:44.593+0000 7f20d2c8b640 1 --2- 172.21.15.38:0/1481604174 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f20cc0d1ed0 0x7f20cc1e7310 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f20c000d680 tx=0x7f20c000db40 comp rx=0 tx=0).stop 2024-09-21T03:36:44.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:44.593+0000 7f20d2c8b640 1 -- 172.21.15.38:0/1481604174 shutdown_connections 2024-09-21T03:36:44.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:44.593+0000 7f20d2c8b640 1 --2- 172.21.15.38:0/1481604174 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f20b8078b90 0x7f20b807b0a0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:44.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:44.593+0000 7f20d2c8b640 1 --2- 172.21.15.38:0/1481604174 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f20cc1536e0 0x7f20cc155480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:44.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:44.593+0000 7f20d2c8b640 1 --2- 172.21.15.38:0/1481604174 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f20cc0d1ed0 0x7f20cc1e7310 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:44.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:44.593+0000 7f20d2c8b640 1 --2- 172.21.15.38:0/1481604174 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f20cc0cce70 0x7f20cc14fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:44.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:44.593+0000 7f20d2c8b640 1 -- 172.21.15.38:0/1481604174 >> 172.21.15.38:0/1481604174 conn(0x7f20cc0cef40 msgr2=0x7f20cc071560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:44.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:44.593+0000 7f20d2c8b640 1 -- 172.21.15.38:0/1481604174 shutdown_connections 2024-09-21T03:36:44.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:44.593+0000 7f20d2c8b640 1 -- 172.21.15.38:0/1481604174 wait complete. 2024-09-21T03:36:45.008 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 51, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T03:36:21.442724+0000', 'modified': '2024-09-21T03:36:43.454690+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 68, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6204}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6204': {'gid': 6204, 'name': 'a', 'rank': 0, 'incarnation': 49, 'state': 'up:rejoin', 'state_seq': 4, 'addr': '172.21.15.38:6834/1058376326', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.38:6832', 'nonce': 1058376326}, {'type': 'v1', 'addr': '172.21.15.38:6834', 'nonce': 1058376326}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-09-21T03:36:45.008 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6204:up:rejoin 2024-09-21T03:36:46.009 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:36:46.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.143+0000 7ff52539d640 1 -- 172.21.15.38:0/2239214746 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff5200b5940 msgr2=0x7ff5200b5d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:46.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.143+0000 7ff52539d640 1 --2- 172.21.15.38:0/2239214746 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff5200b5940 0x7ff5200b5d50 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7ff5200cfbb0 tx=0x7ff51802aa30 comp rx=0 tx=0).stop 2024-09-21T03:36:46.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.143+0000 7ff52539d640 1 -- 172.21.15.38:0/2239214746 shutdown_connections 2024-09-21T03:36:46.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.143+0000 7ff52539d640 1 --2- 172.21.15.38:0/2239214746 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff5200b5940 0x7ff5200b5d50 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:46.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.143+0000 7ff52539d640 1 --2- 172.21.15.38:0/2239214746 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff5200cd0d0 0x7ff5200b5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:46.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.143+0000 7ff52539d640 1 --2- 172.21.15.38:0/2239214746 >> v2:172.21.15.195:3300/0 conn(0x7ff5200b16e0 0x7ff5200b4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:46.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.143+0000 7ff52539d640 1 -- 172.21.15.38:0/2239214746 >> 172.21.15.38:0/2239214746 conn(0x7ff5200b4180 msgr2=0x7ff5200d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:46.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.143+0000 7ff52539d640 1 -- 172.21.15.38:0/2239214746 shutdown_connections 2024-09-21T03:36:46.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.143+0000 7ff52539d640 1 -- 172.21.15.38:0/2239214746 wait complete. 2024-09-21T03:36:46.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.145+0000 7ff52539d640 1 Processor -- start 2024-09-21T03:36:46.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.145+0000 7ff52539d640 1 -- start start 2024-09-21T03:36:46.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.146+0000 7ff52539d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff5200b16e0 0x7ff5201505d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:46.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.146+0000 7ff52539d640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff5200b5940 0x7ff52006f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:46.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.146+0000 7ff52539d640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff5200cd0d0 0x7ff52006f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:46.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.146+0000 7ff52539d640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ff520070670 con 0x7ff5200b16e0 2024-09-21T03:36:46.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.146+0000 7ff52539d640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ff5200704f0 con 0x7ff5200cd0d0 2024-09-21T03:36:46.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.146+0000 7ff52539d640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7ff5200707f0 con 0x7ff5200b5940 2024-09-21T03:36:46.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.146+0000 7ff51ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff5200b16e0 0x7ff5201505d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:46.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.146+0000 7ff51ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff5200b16e0 0x7ff5201505d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:36602/0 (socket says 172.21.15.38:36602) 2024-09-21T03:36:46.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.146+0000 7ff51ffff640 1 -- 172.21.15.38:0/78866294 learned_addr learned my addr 172.21.15.38:0/78866294 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:36:46.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.147+0000 7ff51f7fe640 1 --2- 172.21.15.38:0/78866294 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff5200b5940 0x7ff52006f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:46.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.147+0000 7ff51ffff640 1 -- 172.21.15.38:0/78866294 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff5200b5940 msgr2=0x7ff52006f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:46.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.147+0000 7ff524b9c640 1 --2- 172.21.15.38:0/78866294 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff5200cd0d0 0x7ff52006f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:46.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.147+0000 7ff51ffff640 1 --2- 172.21.15.38:0/78866294 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff5200b5940 0x7ff52006f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:46.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.147+0000 7ff51ffff640 1 -- 172.21.15.38:0/78866294 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff5200cd0d0 msgr2=0x7ff52006f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:46.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.147+0000 7ff51ffff640 1 --2- 172.21.15.38:0/78866294 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff5200cd0d0 0x7ff52006f890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:46.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.147+0000 7ff51ffff640 1 -- 172.21.15.38:0/78866294 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff520150b10 con 0x7ff5200b16e0 2024-09-21T03:36:46.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.147+0000 7ff524b9c640 1 --2- 172.21.15.38:0/78866294 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff5200cd0d0 0x7ff52006f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:36:46.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.147+0000 7ff51f7fe640 1 --2- 172.21.15.38:0/78866294 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff5200b5940 0x7ff52006f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:36:46.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.147+0000 7ff51ffff640 1 --2- 172.21.15.38:0/78866294 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff5200b16e0 0x7ff5201505d0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7ff51000d680 tx=0x7ff51000db40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:46.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.147+0000 7ff51cff9640 1 -- 172.21.15.38:0/78866294 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff510017070 con 0x7ff5200b16e0 2024-09-21T03:36:46.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.147+0000 7ff51cff9640 1 -- 172.21.15.38:0/78866294 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff510004310 con 0x7ff5200b16e0 2024-09-21T03:36:46.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.147+0000 7ff52539d640 1 -- 172.21.15.38:0/78866294 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff52006df90 con 0x7ff5200b16e0 2024-09-21T03:36:46.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.147+0000 7ff51cff9640 1 -- 172.21.15.38:0/78866294 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff510005280 con 0x7ff5200b16e0 2024-09-21T03:36:46.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.147+0000 7ff52539d640 1 -- 172.21.15.38:0/78866294 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff52006e550 con 0x7ff5200b16e0 2024-09-21T03:36:46.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.148+0000 7ff52539d640 1 -- 172.21.15.38:0/78866294 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5200b5d50 con 0x7ff5200b16e0 2024-09-21T03:36:46.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.149+0000 7ff51cff9640 1 -- 172.21.15.38:0/78866294 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ff510005420 con 0x7ff5200b16e0 2024-09-21T03:36:46.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.150+0000 7ff51cff9640 1 --2- 172.21.15.38:0/78866294 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff50c078bc0 0x7ff50c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:46.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.150+0000 7ff51cff9640 1 -- 172.21.15.38:0/78866294 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6187+0+0 (secure 0 0 0) 0x7ff51009b210 con 0x7ff5200b16e0 2024-09-21T03:36:46.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.150+0000 7ff51f7fe640 1 --2- 172.21.15.38:0/78866294 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff50c078bc0 0x7ff50c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:46.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.151+0000 7ff51f7fe640 1 --2- 172.21.15.38:0/78866294 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff50c078bc0 0x7ff50c07b0d0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7ff5200aef50 tx=0x7ff514005f90 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:46.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.155+0000 7ff51cff9640 1 -- 172.21.15.38:0/78866294 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7ff510020020 con 0x7ff5200b16e0 2024-09-21T03:36:46.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.385+0000 7ff52539d640 1 -- 172.21.15.38:0/78866294 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff5200d0d00 con 0x7ff5200b16e0 2024-09-21T03:36:46.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.385+0000 7ff51cff9640 1 -- 172.21.15.38:0/78866294 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) ==== 94+0+5231 (secure 0 0 0) 0x7ff5100634f0 con 0x7ff5200b16e0 2024-09-21T03:36:46.387 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:36:46.387 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":52,"btime":"2024-09-21T03:36:45:465615+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5432,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/1042764291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":1042764291},{"type":"v1","addr":"172.21.15.195:6837","nonce":1042764291}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":5728,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/3035019482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6833","nonce":3035019482},{"type":"v1","addr":"172.21.15.38:6835","nonce":3035019482}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":5731,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/2536177898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":2536177898},{"type":"v1","addr":"172.21.15.195:6836","nonce":2536177898}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":52,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:36:21.442724+0000","modified":"2024-09-21T03:36:45.465606+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":68,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6204},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6204":{"gid":6204,"name":"a","rank":0,"incarnation":49,"state":"up:active","state_seq":5,"addr":"172.21.15.38:6834/1058376326","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":1058376326},{"type":"v1","addr":"172.21.15.38:6834","nonce":1058376326}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6204,"qdb_cluster":[6204]},"id":4}]} 2024-09-21T03:36:46.388 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 52 2024-09-21T03:36:46.390 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.388+0000 7ff52539d640 1 -- 172.21.15.38:0/78866294 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff50c078bc0 msgr2=0x7ff50c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:46.390 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.388+0000 7ff52539d640 1 --2- 172.21.15.38:0/78866294 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff50c078bc0 0x7ff50c07b0d0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7ff5200aef50 tx=0x7ff514005f90 comp rx=0 tx=0).stop 2024-09-21T03:36:46.390 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.388+0000 7ff52539d640 1 -- 172.21.15.38:0/78866294 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff5200b16e0 msgr2=0x7ff5201505d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:46.390 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.388+0000 7ff52539d640 1 --2- 172.21.15.38:0/78866294 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff5200b16e0 0x7ff5201505d0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7ff51000d680 tx=0x7ff51000db40 comp rx=0 tx=0).stop 2024-09-21T03:36:46.390 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.388+0000 7ff52539d640 1 -- 172.21.15.38:0/78866294 shutdown_connections 2024-09-21T03:36:46.390 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.388+0000 7ff52539d640 1 --2- 172.21.15.38:0/78866294 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff5200cd0d0 0x7ff52006f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:46.390 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.388+0000 7ff52539d640 1 --2- 172.21.15.38:0/78866294 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff50c078bc0 0x7ff50c07b0d0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:46.390 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.388+0000 7ff52539d640 1 --2- 172.21.15.38:0/78866294 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff5200b5940 0x7ff52006f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:46.390 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.388+0000 7ff52539d640 1 --2- 172.21.15.38:0/78866294 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff5200b16e0 0x7ff5201505d0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:46.391 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.388+0000 7ff52539d640 1 -- 172.21.15.38:0/78866294 >> 172.21.15.38:0/78866294 conn(0x7ff5200b4180 msgr2=0x7ff5200b2db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:46.391 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.388+0000 7ff52539d640 1 -- 172.21.15.38:0/78866294 shutdown_connections 2024-09-21T03:36:46.391 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.388+0000 7ff52539d640 1 -- 172.21.15.38:0/78866294 wait complete. 2024-09-21T03:36:46.804 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 52, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T03:36:21.442724+0000', 'modified': '2024-09-21T03:36:45.465606+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 68, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6204}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6204': {'gid': 6204, 'name': 'a', 'rank': 0, 'incarnation': 49, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.38:6834/1058376326', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.38:6832', 'nonce': 1058376326}, {'type': 'v1', 'addr': '172.21.15.38:6834', 'nonce': 1058376326}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6204, 'qdb_cluster': [6204]} 2024-09-21T03:36:46.804 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-21T03:36:46.804 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-21T03:36:46.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.925+0000 7f4555dda640 1 -- 172.21.15.38:0/1354005836 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f455015ca90 msgr2=0x7f45500d0670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:46.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.925+0000 7f4555dda640 1 --2- 172.21.15.38:0/1354005836 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f455015ca90 0x7f45500d0670 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f4544009830 tx=0x7f454402a530 comp rx=0 tx=0).stop 2024-09-21T03:36:46.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.925+0000 7f4555dda640 1 -- 172.21.15.38:0/1354005836 shutdown_connections 2024-09-21T03:36:46.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.925+0000 7f4555dda640 1 --2- 172.21.15.38:0/1354005836 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f455015ca90 0x7f45500d0670 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:46.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.925+0000 7f4555dda640 1 -- 172.21.15.38:0/1354005836 >> 172.21.15.38:0/1354005836 conn(0x7f45500b46e0 msgr2=0x7f45500d23e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:46.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.925+0000 7f4555dda640 1 -- 172.21.15.38:0/1354005836 shutdown_connections 2024-09-21T03:36:46.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.925+0000 7f4555dda640 1 -- 172.21.15.38:0/1354005836 wait complete. 2024-09-21T03:36:46.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.927+0000 7f4555dda640 1 Processor -- start 2024-09-21T03:36:46.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.927+0000 7f4555dda640 1 -- start start 2024-09-21T03:36:46.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.928+0000 7f4555dda640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f45500ccc40 0x7f4550150110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:46.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.928+0000 7f4555dda640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f45500d0bb0 0x7f45500cb840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:46.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.928+0000 7f4555dda640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f455015ca90 0x7f455014c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:46.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.928+0000 7f4555dda640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f45500b6840 con 0x7f45500d0bb0 2024-09-21T03:36:46.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.928+0000 7f4555dda640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f45500b66c0 con 0x7f455015ca90 2024-09-21T03:36:46.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.928+0000 7f4555dda640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f45500b69c0 con 0x7f45500ccc40 2024-09-21T03:36:46.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.929+0000 7f454effd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f45500d0bb0 0x7f45500cb840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:46.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.929+0000 7f454effd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f45500d0bb0 0x7f45500cb840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:36618/0 (socket says 172.21.15.38:36618) 2024-09-21T03:36:46.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.929+0000 7f454effd640 1 -- 172.21.15.38:0/829488782 learned_addr learned my addr 172.21.15.38:0/829488782 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:36:46.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.929+0000 7f454ffff640 1 --2- 172.21.15.38:0/829488782 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f455015ca90 0x7f455014c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:46.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.929+0000 7f454f7fe640 1 --2- 172.21.15.38:0/829488782 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f45500ccc40 0x7f4550150110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:46.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.929+0000 7f454effd640 1 -- 172.21.15.38:0/829488782 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f45500ccc40 msgr2=0x7f4550150110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:46.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.929+0000 7f454effd640 1 --2- 172.21.15.38:0/829488782 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f45500ccc40 0x7f4550150110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:46.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.929+0000 7f454effd640 1 -- 172.21.15.38:0/829488782 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f455015ca90 msgr2=0x7f455014c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:46.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.929+0000 7f454effd640 1 --2- 172.21.15.38:0/829488782 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f455015ca90 0x7f455014c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:46.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.929+0000 7f454effd640 1 -- 172.21.15.38:0/829488782 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f455014d710 con 0x7f45500d0bb0 2024-09-21T03:36:46.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.929+0000 7f454f7fe640 1 --2- 172.21.15.38:0/829488782 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f45500ccc40 0x7f4550150110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:36:46.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.929+0000 7f454effd640 1 --2- 172.21.15.38:0/829488782 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f45500d0bb0 0x7f45500cb840 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f4544009950 tx=0x7f4544033c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:46.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.929+0000 7f4537fff640 1 -- 172.21.15.38:0/829488782 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4544031b60 con 0x7f45500d0bb0 2024-09-21T03:36:46.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.929+0000 7f4537fff640 1 -- 172.21.15.38:0/829488782 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4544031df0 con 0x7f45500d0bb0 2024-09-21T03:36:46.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.929+0000 7f4537fff640 1 -- 172.21.15.38:0/829488782 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4544005280 con 0x7f45500d0bb0 2024-09-21T03:36:46.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.929+0000 7f4555dda640 1 -- 172.21.15.38:0/829488782 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4550150650 con 0x7f45500d0bb0 2024-09-21T03:36:46.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.930+0000 7f4555dda640 1 -- 172.21.15.38:0/829488782 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45501f7560 con 0x7f45500d0bb0 2024-09-21T03:36:46.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.931+0000 7f4537fff640 1 -- 172.21.15.38:0/829488782 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f4544005420 con 0x7f45500d0bb0 2024-09-21T03:36:46.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.932+0000 7f4537fff640 1 --2- 172.21.15.38:0/829488782 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f453c078bc0 0x7f453c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:46.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.932+0000 7f4537fff640 1 -- 172.21.15.38:0/829488782 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6187+0+0 (secure 0 0 0) 0x7f45440bc1a0 con 0x7f45500d0bb0 2024-09-21T03:36:46.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.932+0000 7f454f7fe640 1 --2- 172.21.15.38:0/829488782 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f453c078bc0 0x7f453c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:46.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.933+0000 7f455ce20740 1 Processor -- start 2024-09-21T03:36:46.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.933+0000 7f455ce20740 1 -- start start 2024-09-21T03:36:46.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.933+0000 7f454f7fe640 1 --2- 172.21.15.38:0/829488782 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f453c078bc0 0x7f453c07b0d0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f4540007fd0 tx=0x7f4540009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:46.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.934+0000 7f455ce20740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x558afcc79270 0x558afd04afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:46.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.934+0000 7f455ce20740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558afcc503b0 0x558afd04b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:46.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.934+0000 7f455ce20740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558afcc58590 0x558afd0683e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:46.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.934+0000 7f455ce20740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x558afcf93820 con 0x558afcc503b0 2024-09-21T03:36:46.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.934+0000 7f455ce20740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x558afcf82c50 con 0x558afcc58590 2024-09-21T03:36:46.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.934+0000 7f455ce20740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x558afcf86a60 con 0x558afcc79270 2024-09-21T03:36:46.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.934+0000 7f454effd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558afcc503b0 0x558afd04b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:46.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.934+0000 7f454effd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558afcc503b0 0x558afd04b4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:36630/0 (socket says 172.21.15.38:36630) 2024-09-21T03:36:46.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.934+0000 7f454effd640 1 -- 172.21.15.38:0/2060052614 learned_addr learned my addr 172.21.15.38:0/2060052614 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:36:46.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.934+0000 7f454ffff640 1 --2- 172.21.15.38:0/2060052614 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558afcc58590 0x558afd0683e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:46.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.934+0000 7f454f7fe640 1 --2- 172.21.15.38:0/2060052614 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x558afcc79270 0x558afd04afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:46.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.934+0000 7f454effd640 1 -- 172.21.15.38:0/2060052614 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x558afcc79270 msgr2=0x558afd04afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:36:46.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.934+0000 7f454effd640 1 --2- 172.21.15.38:0/2060052614 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x558afcc79270 0x558afd04afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:46.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.934+0000 7f454effd640 1 -- 172.21.15.38:0/2060052614 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558afcc58590 msgr2=0x558afd0683e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:36:46.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.934+0000 7f454effd640 1 --2- 172.21.15.38:0/2060052614 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558afcc58590 0x558afd0683e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:46.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.934+0000 7f454effd640 1 -- 172.21.15.38:0/2060052614 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558afcce4850 con 0x558afcc503b0 2024-09-21T03:36:46.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.935+0000 7f454effd640 1 --2- 172.21.15.38:0/2060052614 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558afcc503b0 0x558afd04b4e0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f4544005eb0 tx=0x7f4544083cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f6578627ff1a304 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:46.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.935+0000 7f45357fa640 1 -- 172.21.15.38:0/2060052614 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4544048050 con 0x558afcc503b0 2024-09-21T03:36:46.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.935+0000 7f45357fa640 1 -- 172.21.15.38:0/2060052614 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4544083950 con 0x558afcc503b0 2024-09-21T03:36:46.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.935+0000 7f45357fa640 1 -- 172.21.15.38:0/2060052614 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4544031770 con 0x558afcc503b0 2024-09-21T03:36:46.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.936+0000 7f455ce20740 1 -- 172.21.15.38:0/2060052614 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558afcc503b0 msgr2=0x558afd04b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:46.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.936+0000 7f455ce20740 1 --2- 172.21.15.38:0/2060052614 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558afcc503b0 0x558afd04b4e0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f4544005eb0 tx=0x7f4544083cd0 comp rx=0 tx=0).stop 2024-09-21T03:36:46.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.936+0000 7f455ce20740 1 -- 172.21.15.38:0/2060052614 shutdown_connections 2024-09-21T03:36:46.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.936+0000 7f455ce20740 1 --2- 172.21.15.38:0/2060052614 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558afcc58590 0x558afd0683e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:46.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.936+0000 7f455ce20740 1 --2- 172.21.15.38:0/2060052614 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558afcc503b0 0x558afd04b4e0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:46.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.936+0000 7f455ce20740 1 --2- 172.21.15.38:0/2060052614 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x558afcc79270 0x558afd04afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:46.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.936+0000 7f455ce20740 1 -- 172.21.15.38:0/2060052614 >> 172.21.15.38:0/2060052614 conn(0x558afcc6d3c0 msgr2=0x558afd021260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:46.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.936+0000 7f455ce20740 1 -- 172.21.15.38:0/2060052614 shutdown_connections 2024-09-21T03:36:46.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.936+0000 7f455ce20740 1 -- 172.21.15.38:0/2060052614 wait complete. 2024-09-21T03:36:46.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.937+0000 7f455ce20740 1 -- start start 2024-09-21T03:36:46.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.937+0000 7f455ce20740 1 Processor -- start 2024-09-21T03:36:46.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.938+0000 7f455ce20740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558afcc503b0 0x558afd0782b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:46.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.938+0000 7f455ce20740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x558afcc58590 0x558afd0787f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:46.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.938+0000 7f455ce20740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558afcc79270 0x558afd038fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:46.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.938+0000 7f455ce20740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x558afccf3620 con 0x558afcc79270 2024-09-21T03:36:46.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.938+0000 7f455ce20740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x558afcf285f0 con 0x558afcc503b0 2024-09-21T03:36:46.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.938+0000 7f455ce20740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x558afd077c10 con 0x558afcc58590 2024-09-21T03:36:46.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.938+0000 7f455ce20740 10 client.0 _ll_register_callbacks cb 0x558afcf2c0a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:36:46.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.938+0000 7f454ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558afcc79270 0x558afd038fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:46.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.938+0000 7f454ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558afcc79270 0x558afd038fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:36644/0 (socket says 172.21.15.38:36644) 2024-09-21T03:36:46.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.938+0000 7f454ffff640 1 -- 172.21.15.38:0/1206275739 learned_addr learned my addr 172.21.15.38:0/1206275739 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:36:46.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.938+0000 7f454f7fe640 1 --2- 172.21.15.38:0/1206275739 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558afcc503b0 0x558afd0782b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:46.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.938+0000 7f454effd640 1 --2- 172.21.15.38:0/1206275739 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x558afcc58590 0x558afd0787f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:46.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.938+0000 7f454ffff640 1 -- 172.21.15.38:0/1206275739 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x558afcc58590 msgr2=0x558afd0787f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:36:46.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.938+0000 7f454ffff640 1 --2- 172.21.15.38:0/1206275739 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x558afcc58590 0x558afd0787f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:46.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.938+0000 7f454ffff640 1 -- 172.21.15.38:0/1206275739 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558afcc503b0 msgr2=0x558afd0782b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:36:46.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.938+0000 7f454ffff640 1 --2- 172.21.15.38:0/1206275739 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558afcc503b0 0x558afd0782b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:46.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.938+0000 7f454ffff640 1 -- 172.21.15.38:0/1206275739 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558afcd5b6b0 con 0x558afcc79270 2024-09-21T03:36:46.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.939+0000 7f454ffff640 1 --2- 172.21.15.38:0/1206275739 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558afcc79270 0x558afd038fc0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f4548005740 tx=0x7f4548001c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d49ed71a3e0027bf server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:46.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.939+0000 7f452e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-21T03:36:46.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.939+0000 7f452e7fc640 1 -- 172.21.15.38:0/1206275739 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4548013540 con 0x558afcc79270 2024-09-21T03:36:46.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.939+0000 7f452e7fc640 1 -- 172.21.15.38:0/1206275739 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(68..68 src has 1..68) ==== 6187+0+0 (secure 0 0 0) 0x7f4548010040 con 0x558afcc79270 2024-09-21T03:36:46.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.939+0000 7f455ce20740 1 -- 172.21.15.38:0/1206275739 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4544083950 con 0x558afcc79270 2024-09-21T03:36:46.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.939+0000 7f452e7fc640 1 -- 172.21.15.38:0/1206275739 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f45480136e0 con 0x558afcc79270 2024-09-21T03:36:46.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.939+0000 7f455ce20740 10 client.6255 fetch_fsmap learned FSMap version 52 2024-09-21T03:36:46.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.939+0000 7f455ce20740 1 -- 172.21.15.38:0/1206275739 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) -- 0x558afd0399d0 con 0x558afcc79270 2024-09-21T03:36:46.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.939+0000 7f452e7fc640 1 -- 172.21.15.38:0/1206275739 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4544083950 con 0x558afcc79270 2024-09-21T03:36:46.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.939+0000 7f452e7fc640 1 -- 172.21.15.38:0/1206275739 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 52) ==== 3502+0+0 (secure 0 0 0) 0x7f4548013970 con 0x558afcc79270 2024-09-21T03:36:46.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.939+0000 7f455ce20740 10 client.6255 fetch_fsmap finished waiting for FSMap version 52 2024-09-21T03:36:46.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.939+0000 7f455ce20740 10 client.6255 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-21T03:36:46.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.939+0000 7f455ce20740 20 client.6255 populate_metadata read hostname 'smithi038' 2024-09-21T03:36:46.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.939+0000 7f455ce20740 1 --2- 172.21.15.38:0/1206275739 >> [v2:172.21.15.38:6832/1058376326,v1:172.21.15.38:6834/1058376326] conn(0x558afcc37e40 0x558afd075070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:46.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.939+0000 7f455ce20740 4 client.6255 mds_command: new command op to 6204 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:36:46.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.939+0000 7f455ce20740 1 -- 172.21.15.38:0/1206275739 --> [v2:172.21.15.38:6832/1058376326,v1:172.21.15.38:6834/1058376326] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558afcf95770 con 0x558afcc37e40 2024-09-21T03:36:46.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.940+0000 7f454f7fe640 1 --2- 172.21.15.38:0/1206275739 >> [v2:172.21.15.38:6832/1058376326,v1:172.21.15.38:6834/1058376326] conn(0x558afcc37e40 0x558afd075070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:46.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.940+0000 7f452e7fc640 10 client.6255 ms_handle_connect on v2:172.21.15.38:6832/1058376326 2024-09-21T03:36:46.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.940+0000 7f454f7fe640 1 --2- 172.21.15.38:0/1206275739 >> [v2:172.21.15.38:6832/1058376326,v1:172.21.15.38:6834/1058376326] conn(0x558afcc37e40 0x558afd075070 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a16a72ee2197b58f server_cookie=625e5641b56a2094 in_seq=0 out_seq=0 2024-09-21T03:36:46.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.941+0000 7f452e7fc640 1 -- 172.21.15.38:0/1206275739 <== mds.0 v2:172.21.15.38:6832/1058376326 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x558afcf95770 con 0x558afcc37e40 2024-09-21T03:36:46.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.941+0000 7f452e7fc640 10 client.6255 handle_command_reply: tid=0 2024-09-21T03:36:46.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.941+0000 7f455ce20740 1 client.6255 shutdown 2024-09-21T03:36:46.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.941+0000 7f455ce20740 1 -- 172.21.15.38:0/1206275739 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558afcc79270 msgr2=0x558afd038fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:46.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.941+0000 7f455ce20740 1 --2- 172.21.15.38:0/1206275739 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558afcc79270 0x558afd038fc0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f4548005740 tx=0x7f4548001c10 comp rx=0 tx=0).stop 2024-09-21T03:36:46.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.941+0000 7f455ce20740 1 -- 172.21.15.38:0/1206275739 shutdown_connections 2024-09-21T03:36:46.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.941+0000 7f455ce20740 1 --2- 172.21.15.38:0/1206275739 >> [v2:172.21.15.38:6832/1058376326,v1:172.21.15.38:6834/1058376326] conn(0x558afcc37e40 0x558afd075070 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:46.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.941+0000 7f455ce20740 1 --2- 172.21.15.38:0/1206275739 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558afcc79270 0x558afd038fc0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:46.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.941+0000 7f455ce20740 1 --2- 172.21.15.38:0/1206275739 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x558afcc58590 0x558afd0787f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:46.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.941+0000 7f452e7fc640 0 client.6255 ms_handle_reset on v2:172.21.15.38:6832/1058376326 2024-09-21T03:36:46.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.941+0000 7f455ce20740 1 --2- 172.21.15.38:0/1206275739 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558afcc503b0 0x558afd0782b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:46.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.941+0000 7f455ce20740 1 -- 172.21.15.38:0/1206275739 >> 172.21.15.38:0/1206275739 conn(0x558afcc6d3c0 msgr2=0x558afd0665e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:46.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.941+0000 7f455ce20740 1 -- 172.21.15.38:0/1206275739 shutdown_connections 2024-09-21T03:36:46.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.941+0000 7f455ce20740 1 -- 172.21.15.38:0/1206275739 wait complete. 2024-09-21T03:36:46.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.941+0000 7f455ce20740 20 client.6255 trim_cache size 0 max 16384 2024-09-21T03:36:46.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.959+0000 7f455ce20740 1 Processor -- start 2024-09-21T03:36:46.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.960+0000 7f455ce20740 1 -- start start 2024-09-21T03:36:46.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.961+0000 7f455ce20740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558afcc503b0 0x558afd078240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:46.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.961+0000 7f455ce20740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558afcc58590 0x558afd078780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:46.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.961+0000 7f455ce20740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x558afcc79270 0x558afd089780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:46.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.961+0000 7f455ce20740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f45501ff5b0 con 0x558afcc503b0 2024-09-21T03:36:46.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.961+0000 7f455ce20740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f455014e520 con 0x558afcc58590 2024-09-21T03:36:46.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.961+0000 7f455ce20740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f45501ff030 con 0x558afcc79270 2024-09-21T03:36:46.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.961+0000 7f454f7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558afcc503b0 0x558afd078240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:46.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.961+0000 7f454f7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558afcc503b0 0x558afd078240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:36648/0 (socket says 172.21.15.38:36648) 2024-09-21T03:36:46.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.961+0000 7f454f7fe640 1 -- 172.21.15.38:0/191269875 learned_addr learned my addr 172.21.15.38:0/191269875 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:36:46.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.961+0000 7f454f7fe640 1 -- 172.21.15.38:0/191269875 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x558afcc79270 msgr2=0x558afd089780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:36:46.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.961+0000 7f454f7fe640 1 --2- 172.21.15.38:0/191269875 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x558afcc79270 0x558afd089780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:46.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.961+0000 7f454f7fe640 1 -- 172.21.15.38:0/191269875 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558afcc58590 msgr2=0x558afd078780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:36:46.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.961+0000 7f454f7fe640 1 --2- 172.21.15.38:0/191269875 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558afcc58590 0x558afd078780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:46.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.961+0000 7f454f7fe640 1 -- 172.21.15.38:0/191269875 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558afd077fd0 con 0x558afcc503b0 2024-09-21T03:36:46.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.962+0000 7f454f7fe640 1 --2- 172.21.15.38:0/191269875 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558afcc503b0 0x558afd078240 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f4540000f60 tx=0x7f4540012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce0fcf11a78b336a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:46.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.962+0000 7f4534ff9640 1 -- 172.21.15.38:0/191269875 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45400035f0 con 0x558afcc503b0 2024-09-21T03:36:46.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.962+0000 7f4534ff9640 1 -- 172.21.15.38:0/191269875 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4540003880 con 0x558afcc503b0 2024-09-21T03:36:46.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.962+0000 7f4534ff9640 1 -- 172.21.15.38:0/191269875 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4540003a70 con 0x558afcc503b0 2024-09-21T03:36:46.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.962+0000 7f455ce20740 1 -- 172.21.15.38:0/191269875 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558afcc503b0 msgr2=0x558afd078240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:46.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.962+0000 7f455ce20740 1 --2- 172.21.15.38:0/191269875 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558afcc503b0 0x558afd078240 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f4540000f60 tx=0x7f4540012000 comp rx=0 tx=0).stop 2024-09-21T03:36:46.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.963+0000 7f455ce20740 1 -- 172.21.15.38:0/191269875 shutdown_connections 2024-09-21T03:36:46.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.963+0000 7f455ce20740 1 --2- 172.21.15.38:0/191269875 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x558afcc79270 0x558afd089780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:46.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.963+0000 7f455ce20740 1 --2- 172.21.15.38:0/191269875 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558afcc58590 0x558afd078780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:46.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.963+0000 7f455ce20740 1 --2- 172.21.15.38:0/191269875 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558afcc503b0 0x558afd078240 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:46.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.963+0000 7f455ce20740 1 -- 172.21.15.38:0/191269875 >> 172.21.15.38:0/191269875 conn(0x558afcc6d3c0 msgr2=0x558afd05eba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:46.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.963+0000 7f455ce20740 1 -- 172.21.15.38:0/191269875 shutdown_connections 2024-09-21T03:36:46.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.963+0000 7f455ce20740 1 -- 172.21.15.38:0/191269875 wait complete. 2024-09-21T03:36:46.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.963+0000 7f455ce20740 1 -- start start 2024-09-21T03:36:46.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.964+0000 7f455ce20740 1 Processor -- start 2024-09-21T03:36:46.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.965+0000 7f455ce20740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x558afcc503b0 0x558afd07fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:46.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.965+0000 7f455ce20740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558afcc58590 0x558afd0ea670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:46.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.965+0000 7f455ce20740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558afcc79270 0x558afd0ed3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:46.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.965+0000 7f455ce20740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x558afd076a10 con 0x558afcc58590 2024-09-21T03:36:46.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.965+0000 7f455ce20740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x558afd039cc0 con 0x558afcc79270 2024-09-21T03:36:46.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.965+0000 7f455ce20740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x558afd077000 con 0x558afcc503b0 2024-09-21T03:36:46.972 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.965+0000 7f455ce20740 10 client.0 _ll_register_callbacks cb 0x7f45480039f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:36:46.972 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.965+0000 7f454effd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558afcc58590 0x558afd0ea670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:46.972 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.965+0000 7f454effd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558afcc58590 0x558afd0ea670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:36652/0 (socket says 172.21.15.38:36652) 2024-09-21T03:36:46.972 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.965+0000 7f454effd640 1 -- 172.21.15.38:0/2178338885 learned_addr learned my addr 172.21.15.38:0/2178338885 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:36:46.972 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.965+0000 7f454f7fe640 1 --2- 172.21.15.38:0/2178338885 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x558afcc503b0 0x558afd07fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:46.972 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.965+0000 7f454ffff640 1 --2- 172.21.15.38:0/2178338885 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558afcc79270 0x558afd0ed3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:46.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.965+0000 7f454effd640 1 -- 172.21.15.38:0/2178338885 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x558afcc503b0 msgr2=0x558afd07fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:36:46.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.965+0000 7f454effd640 1 --2- 172.21.15.38:0/2178338885 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x558afcc503b0 0x558afd07fb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:46.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.965+0000 7f454effd640 1 -- 172.21.15.38:0/2178338885 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558afcc79270 msgr2=0x558afd0ed3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:36:46.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.965+0000 7f454effd640 1 --2- 172.21.15.38:0/2178338885 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558afcc79270 0x558afd0ed3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:46.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.965+0000 7f454effd640 1 -- 172.21.15.38:0/2178338885 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558afd0e9bd0 con 0x558afcc58590 2024-09-21T03:36:46.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.965+0000 7f454ffff640 1 --2- 172.21.15.38:0/2178338885 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558afcc79270 0x558afd0ed3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:36:46.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.966+0000 7f454effd640 1 --2- 172.21.15.38:0/2178338885 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558afcc58590 0x558afd0ea670 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f4544009920 tx=0x7f454402aa70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91c23f10086dcbae server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:46.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.966+0000 7f452e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-21T03:36:46.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.966+0000 7f452e7fc640 1 -- 172.21.15.38:0/2178338885 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4544083950 con 0x558afcc58590 2024-09-21T03:36:46.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.966+0000 7f452e7fc640 1 -- 172.21.15.38:0/2178338885 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(68..68 src has 1..68) ==== 6187+0+0 (secure 0 0 0) 0x7f454408d290 con 0x558afcc58590 2024-09-21T03:36:46.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.966+0000 7f455ce20740 1 -- 172.21.15.38:0/2178338885 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4540003880 con 0x558afcc58590 2024-09-21T03:36:46.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.966+0000 7f452e7fc640 1 -- 172.21.15.38:0/2178338885 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f4544048030 con 0x558afcc58590 2024-09-21T03:36:46.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.966+0000 7f455ce20740 10 client.6261 fetch_fsmap learned FSMap version 52 2024-09-21T03:36:46.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.966+0000 7f455ce20740 1 -- 172.21.15.38:0/2178338885 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) -- 0x558afd0ed930 con 0x558afcc58590 2024-09-21T03:36:46.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.966+0000 7f452e7fc640 1 -- 172.21.15.38:0/2178338885 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4540003880 con 0x558afcc58590 2024-09-21T03:36:46.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.966+0000 7f452e7fc640 1 -- 172.21.15.38:0/2178338885 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 52) ==== 3502+0+0 (secure 0 0 0) 0x7f454408bb60 con 0x558afcc58590 2024-09-21T03:36:46.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.966+0000 7f455ce20740 10 client.6261 fetch_fsmap finished waiting for FSMap version 52 2024-09-21T03:36:46.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.966+0000 7f455ce20740 10 client.6261 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-21T03:36:46.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.966+0000 7f455ce20740 20 client.6261 populate_metadata read hostname 'smithi038' 2024-09-21T03:36:46.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.966+0000 7f455ce20740 1 --2- 172.21.15.38:0/2178338885 >> [v2:172.21.15.38:6832/1058376326,v1:172.21.15.38:6834/1058376326] conn(0x558afd0659e0 0x558afd0ced20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:46.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.966+0000 7f455ce20740 4 client.6261 mds_command: new command op to 6204 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T03:36:46.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.966+0000 7f455ce20740 1 -- 172.21.15.38:0/2178338885 --> [v2:172.21.15.38:6832/1058376326,v1:172.21.15.38:6834/1058376326] -- command(tid 0: {"prefix": "status"}) -- 0x558afcf91220 con 0x558afd0659e0 2024-09-21T03:36:46.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.966+0000 7f454f7fe640 1 --2- 172.21.15.38:0/2178338885 >> [v2:172.21.15.38:6832/1058376326,v1:172.21.15.38:6834/1058376326] conn(0x558afd0659e0 0x558afd0ced20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:46.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.967+0000 7f452e7fc640 10 client.6261 ms_handle_connect on v2:172.21.15.38:6832/1058376326 2024-09-21T03:36:46.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.967+0000 7f454f7fe640 1 --2- 172.21.15.38:0/2178338885 >> [v2:172.21.15.38:6832/1058376326,v1:172.21.15.38:6834/1058376326] conn(0x558afd0659e0 0x558afd0ced20 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a826179e191f81c server_cookie=b8dcb806e1fb91d6 in_seq=0 out_seq=0 2024-09-21T03:36:46.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.967+0000 7f452e7fc640 1 -- 172.21.15.38:0/2178338885 <== mds.0 v2:172.21.15.38:6832/1058376326 1 ==== command_reply(tid 0: 0 ) ==== 8+0+318 (crc 0 0 0) 0x558afcf91220 con 0x558afd0659e0 2024-09-21T03:36:46.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.967+0000 7f452e7fc640 10 client.6261 handle_command_reply: tid=0 2024-09-21T03:36:46.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.967+0000 7f455ce20740 1 client.6261 shutdown 2024-09-21T03:36:46.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.967+0000 7f455ce20740 1 -- 172.21.15.38:0/2178338885 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558afcc58590 msgr2=0x558afd0ea670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:46.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.967+0000 7f455ce20740 1 --2- 172.21.15.38:0/2178338885 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558afcc58590 0x558afd0ea670 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f4544009920 tx=0x7f454402aa70 comp rx=0 tx=0).stop 2024-09-21T03:36:46.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.967+0000 7f455ce20740 1 -- 172.21.15.38:0/2178338885 shutdown_connections 2024-09-21T03:36:46.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.967+0000 7f455ce20740 1 --2- 172.21.15.38:0/2178338885 >> [v2:172.21.15.38:6832/1058376326,v1:172.21.15.38:6834/1058376326] conn(0x558afd0659e0 0x558afd0ced20 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:46.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.967+0000 7f455ce20740 1 --2- 172.21.15.38:0/2178338885 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558afcc79270 0x558afd0ed3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:46.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.967+0000 7f455ce20740 1 --2- 172.21.15.38:0/2178338885 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558afcc58590 0x558afd0ea670 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:46.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.967+0000 7f455ce20740 1 --2- 172.21.15.38:0/2178338885 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x558afcc503b0 0x558afd07fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:46.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.967+0000 7f452e7fc640 0 client.6261 ms_handle_reset on v2:172.21.15.38:6832/1058376326 2024-09-21T03:36:46.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.967+0000 7f455ce20740 1 -- 172.21.15.38:0/2178338885 >> 172.21.15.38:0/2178338885 conn(0x558afcc6d3c0 msgr2=0x558afd05eba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:46.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.967+0000 7f455ce20740 1 -- 172.21.15.38:0/2178338885 shutdown_connections 2024-09-21T03:36:46.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.967+0000 7f455ce20740 1 -- 172.21.15.38:0/2178338885 wait complete. 2024-09-21T03:36:46.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.967+0000 7f455ce20740 20 client.6261 trim_cache size 0 max 16384 2024-09-21T03:36:46.976 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-09-21T03:36:46.976 INFO:teuthology.orchestra.run.smithi038.stdout: "cluster_fsid": "9d50d66f-656b-4fe2-8e3b-fe53b79c4d43", 2024-09-21T03:36:46.976 INFO:teuthology.orchestra.run.smithi038.stdout: "whoami": 0, 2024-09-21T03:36:46.977 INFO:teuthology.orchestra.run.smithi038.stdout: "id": 6204, 2024-09-21T03:36:46.977 INFO:teuthology.orchestra.run.smithi038.stdout: "want_state": "up:active", 2024-09-21T03:36:46.977 INFO:teuthology.orchestra.run.smithi038.stdout: "state": "up:active", 2024-09-21T03:36:46.977 INFO:teuthology.orchestra.run.smithi038.stdout: "fs_name": "cephfs", 2024-09-21T03:36:46.977 INFO:teuthology.orchestra.run.smithi038.stdout: "rank_uptime": 4.523936891, 2024-09-21T03:36:46.977 INFO:teuthology.orchestra.run.smithi038.stdout: "mdsmap_epoch": 52, 2024-09-21T03:36:46.977 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch": 68, 2024-09-21T03:36:46.977 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch_barrier": 68, 2024-09-21T03:36:46.977 INFO:teuthology.orchestra.run.smithi038.stdout: "uptime": 10.476701537 2024-09-21T03:36:46.977 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-09-21T03:36:46.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.968+0000 7f4555dda640 1 -- 172.21.15.38:0/829488782 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f453c078bc0 msgr2=0x7f453c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:46.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.968+0000 7f4555dda640 1 --2- 172.21.15.38:0/829488782 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f453c078bc0 0x7f453c07b0d0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f4540007fd0 tx=0x7f4540009040 comp rx=0 tx=0).stop 2024-09-21T03:36:46.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.968+0000 7f4555dda640 1 -- 172.21.15.38:0/829488782 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f45500d0bb0 msgr2=0x7f45500cb840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:46.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.968+0000 7f4555dda640 1 --2- 172.21.15.38:0/829488782 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f45500d0bb0 0x7f45500cb840 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f4544009950 tx=0x7f4544033c10 comp rx=0 tx=0).stop 2024-09-21T03:36:46.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.968+0000 7f4555dda640 1 -- 172.21.15.38:0/829488782 shutdown_connections 2024-09-21T03:36:46.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.968+0000 7f4555dda640 1 --2- 172.21.15.38:0/829488782 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f455015ca90 0x7f455014c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:46.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.968+0000 7f4555dda640 1 --2- 172.21.15.38:0/829488782 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f45500d0bb0 0x7f45500cb840 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:46.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.968+0000 7f4555dda640 1 --2- 172.21.15.38:0/829488782 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f453c078bc0 0x7f453c07b0d0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:46.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.968+0000 7f4555dda640 1 --2- 172.21.15.38:0/829488782 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f45500ccc40 0x7f4550150110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:46.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.968+0000 7f4555dda640 1 -- 172.21.15.38:0/829488782 >> 172.21.15.38:0/829488782 conn(0x7f45500b46e0 msgr2=0x7f45500b5bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:46.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.968+0000 7f4555dda640 1 -- 172.21.15.38:0/829488782 shutdown_connections 2024-09-21T03:36:46.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:46.968+0000 7f4555dda640 1 -- 172.21.15.38:0/829488782 wait complete. 2024-09-21T03:36:46.981 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:36:47.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:47.096+0000 7f021d6a9640 1 -- 172.21.15.38:0/1378878085 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0200004800 msgr2=0x7f0200004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:47.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:47.096+0000 7f021d6a9640 1 --2- 172.21.15.38:0/1378878085 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0200004800 0x7f0200004c10 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f020c009b50 tx=0x7f0200005830 comp rx=0 tx=0).stop 2024-09-21T03:36:47.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:47.096+0000 7f021d6a9640 1 -- 172.21.15.38:0/1378878085 shutdown_connections 2024-09-21T03:36:47.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:47.096+0000 7f021d6a9640 1 --2- 172.21.15.38:0/1378878085 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0200004800 0x7f0200004c10 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:47.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:47.096+0000 7f021d6a9640 1 -- 172.21.15.38:0/1378878085 >> 172.21.15.38:0/1378878085 conn(0x7f02180b2cc0 msgr2=0x7f02180b30d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:47.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:47.096+0000 7f021d6a9640 1 -- 172.21.15.38:0/1378878085 shutdown_connections 2024-09-21T03:36:47.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:47.096+0000 7f021d6a9640 1 -- 172.21.15.38:0/1378878085 wait complete. 2024-09-21T03:36:47.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:47.098+0000 7f021d6a9640 1 Processor -- start 2024-09-21T03:36:47.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:47.098+0000 7f021d6a9640 1 -- start start 2024-09-21T03:36:47.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:47.099+0000 7f021d6a9640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0200004800 0x7f021813f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:47.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:47.100+0000 7f021d6a9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f02181f65a0 0x7f021813d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:47.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:47.100+0000 7f021d6a9640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f021813d710 0x7f021813b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:47.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:47.100+0000 7f021d6a9640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f021815d140 con 0x7f02181f65a0 2024-09-21T03:36:47.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:47.100+0000 7f021d6a9640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f021815cfc0 con 0x7f0200004800 2024-09-21T03:36:47.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:47.100+0000 7f021d6a9640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f0218073d50 con 0x7f021813d710 2024-09-21T03:36:47.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:47.100+0000 7f02167fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f02181f65a0 0x7f021813d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:47.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:47.100+0000 7f02167fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f02181f65a0 0x7f021813d1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:36680/0 (socket says 172.21.15.38:36680) 2024-09-21T03:36:47.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:47.100+0000 7f02167fc640 1 -- 172.21.15.38:0/3996952892 learned_addr learned my addr 172.21.15.38:0/3996952892 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:36:47.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:47.100+0000 7f02177fe640 1 --2- 172.21.15.38:0/3996952892 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f021813d710 0x7f021813b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:47.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:47.100+0000 7f0216ffd640 1 --2- 172.21.15.38:0/3996952892 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0200004800 0x7f021813f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:47.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:47.100+0000 7f02167fc640 1 -- 172.21.15.38:0/3996952892 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f021813d710 msgr2=0x7f021813b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:47.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:47.100+0000 7f02167fc640 1 --2- 172.21.15.38:0/3996952892 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f021813d710 0x7f021813b6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:47.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:47.100+0000 7f02167fc640 1 -- 172.21.15.38:0/3996952892 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0200004800 msgr2=0x7f021813f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:47.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:47.100+0000 7f02167fc640 1 --2- 172.21.15.38:0/3996952892 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0200004800 0x7f021813f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:47.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:47.100+0000 7f02167fc640 1 -- 172.21.15.38:0/3996952892 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02181f6cf0 con 0x7f02181f65a0 2024-09-21T03:36:47.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:47.100+0000 7f02167fc640 1 --2- 172.21.15.38:0/3996952892 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f02181f65a0 0x7f021813d1d0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f020c036040 tx=0x7f020c0336f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:47.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:47.100+0000 7f01ff7fe640 1 -- 172.21.15.38:0/3996952892 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f020c038040 con 0x7f02181f65a0 2024-09-21T03:36:47.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:47.100+0000 7f01ff7fe640 1 -- 172.21.15.38:0/3996952892 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f020c004230 con 0x7f02181f65a0 2024-09-21T03:36:47.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:47.100+0000 7f01ff7fe640 1 -- 172.21.15.38:0/3996952892 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f020c0338e0 con 0x7f02181f65a0 2024-09-21T03:36:47.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:47.100+0000 7f021d6a9640 1 -- 172.21.15.38:0/3996952892 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02181f6fd0 con 0x7f02181f65a0 2024-09-21T03:36:47.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:47.101+0000 7f021d6a9640 1 -- 172.21.15.38:0/3996952892 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02181e75a0 con 0x7f02181f65a0 2024-09-21T03:36:47.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:47.101+0000 7f021d6a9640 1 -- 172.21.15.38:0/3996952892 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02181f74f0 con 0x7f02181f65a0 2024-09-21T03:36:47.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:47.102+0000 7f01ff7fe640 1 -- 172.21.15.38:0/3996952892 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f020c02b8a0 con 0x7f02181f65a0 2024-09-21T03:36:47.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:47.103+0000 7f01ff7fe640 1 --2- 172.21.15.38:0/3996952892 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0204078b90 0x7f020407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:47.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:47.103+0000 7f01ff7fe640 1 -- 172.21.15.38:0/3996952892 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6187+0+0 (secure 0 0 0) 0x7f020c0bb600 con 0x7f02181f65a0 2024-09-21T03:36:47.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:47.103+0000 7f0216ffd640 1 --2- 172.21.15.38:0/3996952892 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0204078b90 0x7f020407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:47.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:47.104+0000 7f0216ffd640 1 --2- 172.21.15.38:0/3996952892 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0204078b90 0x7f020407b0a0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f0208001220 tx=0x7f02080067a0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:47.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:47.107+0000 7f01ff7fe640 1 -- 172.21.15.38:0/3996952892 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f02181f74f0 con 0x7f02181f65a0 2024-09-21T03:36:47.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:47.334+0000 7f021d6a9640 1 -- 172.21.15.38:0/3996952892 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0218074260 con 0x7f02181f65a0 2024-09-21T03:36:47.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:47.334+0000 7f01ff7fe640 1 -- 172.21.15.38:0/3996952892 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) ==== 94+0+5231 (secure 0 0 0) 0x7f020c0832a0 con 0x7f02181f65a0 2024-09-21T03:36:47.336 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:36:47.336 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":52,"btime":"2024-09-21T03:36:45:465615+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5432,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/1042764291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":1042764291},{"type":"v1","addr":"172.21.15.195:6837","nonce":1042764291}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":5728,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/3035019482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6833","nonce":3035019482},{"type":"v1","addr":"172.21.15.38:6835","nonce":3035019482}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":5731,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/2536177898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":2536177898},{"type":"v1","addr":"172.21.15.195:6836","nonce":2536177898}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":52,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:36:21.442724+0000","modified":"2024-09-21T03:36:45.465606+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":68,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6204},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6204":{"gid":6204,"name":"a","rank":0,"incarnation":49,"state":"up:active","state_seq":5,"addr":"172.21.15.38:6834/1058376326","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":1058376326},{"type":"v1","addr":"172.21.15.38:6834","nonce":1058376326}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6204,"qdb_cluster":[6204]},"id":4}]} 2024-09-21T03:36:47.337 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 52 2024-09-21T03:36:47.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:47.336+0000 7f021d6a9640 1 -- 172.21.15.38:0/3996952892 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0204078b90 msgr2=0x7f020407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:47.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:47.336+0000 7f021d6a9640 1 --2- 172.21.15.38:0/3996952892 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0204078b90 0x7f020407b0a0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f0208001220 tx=0x7f02080067a0 comp rx=0 tx=0).stop 2024-09-21T03:36:47.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:47.336+0000 7f021d6a9640 1 -- 172.21.15.38:0/3996952892 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f02181f65a0 msgr2=0x7f021813d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:47.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:47.336+0000 7f021d6a9640 1 --2- 172.21.15.38:0/3996952892 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f02181f65a0 0x7f021813d1d0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f020c036040 tx=0x7f020c0336f0 comp rx=0 tx=0).stop 2024-09-21T03:36:47.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:47.336+0000 7f021d6a9640 1 -- 172.21.15.38:0/3996952892 shutdown_connections 2024-09-21T03:36:47.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:47.336+0000 7f021d6a9640 1 --2- 172.21.15.38:0/3996952892 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0204078b90 0x7f020407b0a0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:47.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:47.336+0000 7f021d6a9640 1 --2- 172.21.15.38:0/3996952892 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f021813d710 0x7f021813b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:47.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:47.336+0000 7f021d6a9640 1 --2- 172.21.15.38:0/3996952892 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f02181f65a0 0x7f021813d1d0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:47.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:47.336+0000 7f021d6a9640 1 --2- 172.21.15.38:0/3996952892 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0200004800 0x7f021813f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:47.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:47.337+0000 7f021d6a9640 1 -- 172.21.15.38:0/3996952892 >> 172.21.15.38:0/3996952892 conn(0x7f02180b2cc0 msgr2=0x7f021815eca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:47.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:47.337+0000 7f021d6a9640 1 -- 172.21.15.38:0/3996952892 shutdown_connections 2024-09-21T03:36:47.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:47.337+0000 7f021d6a9640 1 -- 172.21.15.38:0/3996952892 wait complete. 2024-09-21T03:36:47.350 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds freeze 4:0 true 2024-09-21T03:36:47.433 INFO:tasks.ceph.mds.a.smithi038.stderr:2024-09-21T03:36:47.432+0000 7f4475adc640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T03:36:47.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:47.467+0000 7fd4f70b6640 1 -- 172.21.15.38:0/2252422719 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd4f015ee80 msgr2=0x7fd4f00d0170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:47.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:47.467+0000 7fd4f70b6640 1 --2- 172.21.15.38:0/2252422719 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd4f015ee80 0x7fd4f00d0170 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fd4f00d06b0 tx=0x7fd4e002a440 comp rx=0 tx=0).stop 2024-09-21T03:36:47.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:47.467+0000 7fd4f70b6640 1 -- 172.21.15.38:0/2252422719 shutdown_connections 2024-09-21T03:36:47.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:47.467+0000 7fd4f70b6640 1 --2- 172.21.15.38:0/2252422719 >> v2:172.21.15.195:3300/0 conn(0x7fd4f00b9ad0 0x7fd4f00d0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:47.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:47.467+0000 7fd4f70b6640 1 --2- 172.21.15.38:0/2252422719 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd4f015ee80 0x7fd4f00d0170 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:47.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:47.467+0000 7fd4f70b6640 1 -- 172.21.15.38:0/2252422719 >> 172.21.15.38:0/2252422719 conn(0x7fd4f009d830 msgr2=0x7fd4f00b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:47.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:47.467+0000 7fd4f70b6640 1 -- 172.21.15.38:0/2252422719 shutdown_connections 2024-09-21T03:36:47.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:47.467+0000 7fd4f70b6640 1 -- 172.21.15.38:0/2252422719 wait complete. 2024-09-21T03:36:47.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:47.469+0000 7fd4f70b6640 1 Processor -- start 2024-09-21T03:36:47.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:47.469+0000 7fd4f70b6640 1 -- start start 2024-09-21T03:36:47.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:47.470+0000 7fd4f70b6640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd4f00b9ad0 0x7fd4f01f5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:47.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:47.470+0000 7fd4f70b6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd4f00d1120 0x7fd4f0157510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:47.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:47.470+0000 7fd4f70b6640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd4f015ee80 0x7fd4f015ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:47.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:47.470+0000 7fd4f70b6640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fd4f00b6700 con 0x7fd4f00d1120 2024-09-21T03:36:47.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:47.470+0000 7fd4f70b6640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd4f00b6580 con 0x7fd4f00b9ad0 2024-09-21T03:36:47.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:47.470+0000 7fd4f70b6640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fd4f00b6880 con 0x7fd4f015ee80 2024-09-21T03:36:47.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:47.471+0000 7fd4effff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd4f00d1120 0x7fd4f0157510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:47.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:47.471+0000 7fd4effff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd4f00d1120 0x7fd4f0157510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:36704/0 (socket says 172.21.15.38:36704) 2024-09-21T03:36:47.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:47.471+0000 7fd4effff640 1 -- 172.21.15.38:0/1665965980 learned_addr learned my addr 172.21.15.38:0/1665965980 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:36:47.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:47.471+0000 7fd4f4e52640 1 --2- 172.21.15.38:0/1665965980 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd4f00b9ad0 0x7fd4f01f5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:47.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:47.471+0000 7fd4f5653640 1 --2- 172.21.15.38:0/1665965980 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd4f015ee80 0x7fd4f015ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:47.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:47.471+0000 7fd4effff640 1 -- 172.21.15.38:0/1665965980 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd4f015ee80 msgr2=0x7fd4f015ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:47.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:47.471+0000 7fd4effff640 1 --2- 172.21.15.38:0/1665965980 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd4f015ee80 0x7fd4f015ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:47.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:47.471+0000 7fd4effff640 1 -- 172.21.15.38:0/1665965980 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd4f00b9ad0 msgr2=0x7fd4f01f5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:47.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:47.471+0000 7fd4effff640 1 --2- 172.21.15.38:0/1665965980 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd4f00b9ad0 0x7fd4f01f5ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:47.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:47.471+0000 7fd4effff640 1 -- 172.21.15.38:0/1665965980 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4f006ef10 con 0x7fd4f00d1120 2024-09-21T03:36:47.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:47.471+0000 7fd4effff640 1 --2- 172.21.15.38:0/1665965980 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd4f00d1120 0x7fd4f0157510 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fd4e400d6b0 tx=0x7fd4e400db70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:47.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:47.471+0000 7fd4ed7fa640 1 -- 172.21.15.38:0/1665965980 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4e4017070 con 0x7fd4f00d1120 2024-09-21T03:36:47.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:47.471+0000 7fd4ed7fa640 1 -- 172.21.15.38:0/1665965980 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd4e4009b10 con 0x7fd4f00d1120 2024-09-21T03:36:47.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:47.471+0000 7fd4f70b6640 1 -- 172.21.15.38:0/1665965980 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4f006f1f0 con 0x7fd4f00d1120 2024-09-21T03:36:47.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:47.471+0000 7fd4ed7fa640 1 -- 172.21.15.38:0/1665965980 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4e4010460 con 0x7fd4f00d1120 2024-09-21T03:36:47.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:47.471+0000 7fd4f70b6640 1 -- 172.21.15.38:0/1665965980 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4f01f6160 con 0x7fd4f00d1120 2024-09-21T03:36:47.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:47.473+0000 7fd4f70b6640 1 -- 172.21.15.38:0/1665965980 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4f00b6d90 con 0x7fd4f00d1120 2024-09-21T03:36:47.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:47.473+0000 7fd4ed7fa640 1 -- 172.21.15.38:0/1665965980 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fd4e401a070 con 0x7fd4f00d1120 2024-09-21T03:36:47.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:47.474+0000 7fd4ed7fa640 1 --2- 172.21.15.38:0/1665965980 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd4dc078b90 0x7fd4dc07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:47.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:47.474+0000 7fd4ed7fa640 1 -- 172.21.15.38:0/1665965980 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6187+0+0 (secure 0 0 0) 0x7fd4e409aea0 con 0x7fd4f00d1120 2024-09-21T03:36:47.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:47.474+0000 7fd4f4e52640 1 --2- 172.21.15.38:0/1665965980 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd4dc078b90 0x7fd4dc07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:47.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:47.475+0000 7fd4f4e52640 1 --2- 172.21.15.38:0/1665965980 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd4dc078b90 0x7fd4dc07b0a0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fd4e0033000 tx=0x7fd4e0004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:47.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:47.478+0000 7fd4ed7fa640 1 -- 172.21.15.38:0/1665965980 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fd4e4062b40 con 0x7fd4f00d1120 2024-09-21T03:36:47.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:47.740+0000 7fd4f70b6640 1 -- 172.21.15.38:0/1665965980 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "mds freeze", "role_or_gid": "4:0", "val": "true"} v 0) -- 0x7fd4f006eca0 con 0x7fd4f00d1120 2024-09-21T03:36:48.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.492+0000 7fd4ed7fa640 1 -- 172.21.15.38:0/1665965980 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "mds freeze", "role_or_gid": "4:0", "val": "true"}]=0 freezing mds.6204 v53) ==== 112+0+0 (secure 0 0 0) 0x7fd4e4067570 con 0x7fd4f00d1120 2024-09-21T03:36:48.494 INFO:teuthology.orchestra.run.smithi038.stderr:freezing mds.6204 2024-09-21T03:36:48.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.495+0000 7fd4f70b6640 1 -- 172.21.15.38:0/1665965980 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd4dc078b90 msgr2=0x7fd4dc07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:48.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.495+0000 7fd4f70b6640 1 --2- 172.21.15.38:0/1665965980 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd4dc078b90 0x7fd4dc07b0a0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fd4e0033000 tx=0x7fd4e0004730 comp rx=0 tx=0).stop 2024-09-21T03:36:48.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.495+0000 7fd4f70b6640 1 -- 172.21.15.38:0/1665965980 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd4f00d1120 msgr2=0x7fd4f0157510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:48.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.495+0000 7fd4f70b6640 1 --2- 172.21.15.38:0/1665965980 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd4f00d1120 0x7fd4f0157510 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fd4e400d6b0 tx=0x7fd4e400db70 comp rx=0 tx=0).stop 2024-09-21T03:36:48.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.495+0000 7fd4f70b6640 1 -- 172.21.15.38:0/1665965980 shutdown_connections 2024-09-21T03:36:48.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.495+0000 7fd4f70b6640 1 --2- 172.21.15.38:0/1665965980 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd4dc078b90 0x7fd4dc07b0a0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:48.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.495+0000 7fd4f70b6640 1 --2- 172.21.15.38:0/1665965980 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd4f015ee80 0x7fd4f015ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:48.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.495+0000 7fd4f70b6640 1 --2- 172.21.15.38:0/1665965980 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd4f00d1120 0x7fd4f0157510 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:48.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.495+0000 7fd4f70b6640 1 --2- 172.21.15.38:0/1665965980 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd4f00b9ad0 0x7fd4f01f5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:48.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.495+0000 7fd4f70b6640 1 -- 172.21.15.38:0/1665965980 >> 172.21.15.38:0/1665965980 conn(0x7fd4f009d830 msgr2=0x7fd4f00b7d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:48.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.495+0000 7fd4f70b6640 1 -- 172.21.15.38:0/1665965980 shutdown_connections 2024-09-21T03:36:48.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.495+0000 7fd4f70b6640 1 -- 172.21.15.38:0/1665965980 wait complete. 2024-09-21T03:36:48.511 DEBUG:teuthology.orchestra.run.smithi038:> sudo daemon-helper kill ceph --cluster ceph -w 2024-09-21T03:36:48.513 DEBUG:teuthology.orchestra.run.smithi038:> timeout 30 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --connect-timeout=60 tell mds.4:0 flush journal 2024-09-21T03:36:48.516 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok --format=json config get mds_beacon_grace 2024-09-21T03:36:48.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.643+0000 7f98b5577640 1 -- 172.21.15.38:0/2682427726 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f98b00d0bb0 msgr2=0x7f98b00b5be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:48.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.643+0000 7f98b5577640 1 --2- 172.21.15.38:0/2682427726 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f98b00d0bb0 0x7f98b00b5be0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f98b00cc9c0 tx=0x7f98a802aa30 comp rx=0 tx=0).stop 2024-09-21T03:36:48.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.643+0000 7f98b5577640 1 -- 172.21.15.38:0/2682427726 shutdown_connections 2024-09-21T03:36:48.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.643+0000 7f98b5577640 1 --2- 172.21.15.38:0/2682427726 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f98b00d0bb0 0x7f98b00b5be0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:48.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.643+0000 7f98b5577640 1 --2- 172.21.15.38:0/2682427726 >> v2:172.21.15.195:3300/0 conn(0x7f98b015ca90 0x7f98b00d0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:48.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.643+0000 7f98b5577640 1 -- 172.21.15.38:0/2682427726 >> 172.21.15.38:0/2682427726 conn(0x7f98b00b46e0 msgr2=0x7f98b00d23e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:48.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.643+0000 7f98b5577640 1 -- 172.21.15.38:0/2682427726 shutdown_connections 2024-09-21T03:36:48.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.643+0000 7f98b5577640 1 -- 172.21.15.38:0/2682427726 wait complete. 2024-09-21T03:36:48.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.647+0000 7f98b5577640 1 Processor -- start 2024-09-21T03:36:48.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.647+0000 7f98b5577640 1 -- start start 2024-09-21T03:36:48.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.649+0000 7f98b5577640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f98b00ccc40 0x7f98b015ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:48.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.649+0000 7f98b5577640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f98b00d0bb0 0x7f98b00aa2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:48.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.649+0000 7f98b5577640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f98b015ca90 0x7f98b00aa7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:48.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.649+0000 7f98b5577640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f98b00b6620 con 0x7f98b00ccc40 2024-09-21T03:36:48.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.649+0000 7f98b5577640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f98b00b64a0 con 0x7f98b00d0bb0 2024-09-21T03:36:48.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.649+0000 7f98b5577640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f98b00b67a0 con 0x7f98b015ca90 2024-09-21T03:36:48.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.649+0000 7f98aeffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f98b00ccc40 0x7f98b015ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:48.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.649+0000 7f98aeffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f98b00ccc40 0x7f98b015ad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:36714/0 (socket says 172.21.15.38:36714) 2024-09-21T03:36:48.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.649+0000 7f98aeffd640 1 -- 172.21.15.38:0/3512184550 learned_addr learned my addr 172.21.15.38:0/3512184550 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:36:48.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.649+0000 7f98af7fe640 1 --2- 172.21.15.38:0/3512184550 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f98b015ca90 0x7f98b00aa7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:48.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.649+0000 7f98ae7fc640 1 --2- 172.21.15.38:0/3512184550 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f98b00d0bb0 0x7f98b00aa2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:48.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.650+0000 7f98aeffd640 1 -- 172.21.15.38:0/3512184550 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f98b015ca90 msgr2=0x7f98b00aa7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:48.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.650+0000 7f98aeffd640 1 --2- 172.21.15.38:0/3512184550 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f98b015ca90 0x7f98b00aa7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:48.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.650+0000 7f98aeffd640 1 -- 172.21.15.38:0/3512184550 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f98b00d0bb0 msgr2=0x7f98b00aa2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:48.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.650+0000 7f98aeffd640 1 --2- 172.21.15.38:0/3512184550 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f98b00d0bb0 0x7f98b00aa2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:48.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.650+0000 7f98aeffd640 1 -- 172.21.15.38:0/3512184550 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98b00cc4d0 con 0x7f98b00ccc40 2024-09-21T03:36:48.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.650+0000 7f98aeffd640 1 --2- 172.21.15.38:0/3512184550 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f98b00ccc40 0x7f98b015ad50 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f98a000c3a0 tx=0x7f98a000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:48.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.650+0000 7f98977fe640 1 -- 172.21.15.38:0/3512184550 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98a0017070 con 0x7f98b00ccc40 2024-09-21T03:36:48.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.650+0000 7f98977fe640 1 -- 172.21.15.38:0/3512184550 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f98a00053c0 con 0x7f98b00ccc40 2024-09-21T03:36:48.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.650+0000 7f98b5577640 1 -- 172.21.15.38:0/3512184550 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98b00c5b90 con 0x7f98b00ccc40 2024-09-21T03:36:48.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.650+0000 7f98977fe640 1 -- 172.21.15.38:0/3512184550 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98a0003850 con 0x7f98b00ccc40 2024-09-21T03:36:48.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.650+0000 7f98b5577640 1 -- 172.21.15.38:0/3512184550 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98b01f3190 con 0x7f98b00ccc40 2024-09-21T03:36:48.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.652+0000 7f98977fe640 1 -- 172.21.15.38:0/3512184550 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f98a001a070 con 0x7f98b00ccc40 2024-09-21T03:36:48.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.653+0000 7f98977fe640 1 --2- 172.21.15.38:0/3512184550 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f989c078b90 0x7f989c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:48.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.653+0000 7f98977fe640 1 -- 172.21.15.38:0/3512184550 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6187+0+0 (secure 0 0 0) 0x7f98a009a230 con 0x7f98b00ccc40 2024-09-21T03:36:48.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.653+0000 7f98bc5bf740 1 Processor -- start 2024-09-21T03:36:48.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.654+0000 7f98ae7fc640 1 --2- 172.21.15.38:0/3512184550 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f989c078b90 0x7f989c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:48.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.654+0000 7f98bc5bf740 1 -- start start 2024-09-21T03:36:48.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.654+0000 7f98ae7fc640 1 --2- 172.21.15.38:0/3512184550 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f989c078b90 0x7f989c07b0a0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f98b00ce3a0 tx=0x7f98a400b2e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:48.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.655+0000 7f98bc5bf740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55c451a7c5c0 0x55c451e3e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:48.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.655+0000 7f98bc5bf740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c451a53700 0x55c451e6a460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:48.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.655+0000 7f98bc5bf740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c451a5b8e0 0x55c451e4b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:48.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.655+0000 7f98bc5bf740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55c451d969a0 con 0x55c451a5b8e0 2024-09-21T03:36:48.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.655+0000 7f98bc5bf740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55c451d85dd0 con 0x55c451a53700 2024-09-21T03:36:48.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.655+0000 7f98bc5bf740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55c451d89be0 con 0x55c451a7c5c0 2024-09-21T03:36:48.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.655+0000 7f98af7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c451a5b8e0 0x55c451e4b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:48.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.655+0000 7f98af7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c451a5b8e0 0x55c451e4b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:36730/0 (socket says 172.21.15.38:36730) 2024-09-21T03:36:48.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.655+0000 7f98af7fe640 1 -- 172.21.15.38:0/2329645016 learned_addr learned my addr 172.21.15.38:0/2329645016 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:36:48.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.655+0000 7f98aeffd640 1 --2- 172.21.15.38:0/2329645016 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55c451a7c5c0 0x55c451e3e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:48.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.655+0000 7f98ae7fc640 1 --2- 172.21.15.38:0/2329645016 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c451a53700 0x55c451e6a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:48.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.656+0000 7f98af7fe640 1 -- 172.21.15.38:0/2329645016 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55c451a7c5c0 msgr2=0x55c451e3e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:36:48.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.656+0000 7f98af7fe640 1 --2- 172.21.15.38:0/2329645016 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55c451a7c5c0 0x55c451e3e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:48.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.656+0000 7f98af7fe640 1 -- 172.21.15.38:0/2329645016 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c451a53700 msgr2=0x55c451e6a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:36:48.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.656+0000 7f98af7fe640 1 --2- 172.21.15.38:0/2329645016 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c451a53700 0x55c451e6a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:48.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.656+0000 7f98af7fe640 1 -- 172.21.15.38:0/2329645016 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c451d52d70 con 0x55c451a5b8e0 2024-09-21T03:36:48.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.656+0000 7f98ae7fc640 1 --2- 172.21.15.38:0/2329645016 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c451a53700 0x55c451e6a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:36:48.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.656+0000 7f98aeffd640 1 --2- 172.21.15.38:0/2329645016 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55c451a7c5c0 0x55c451e3e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:36:48.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.656+0000 7f98af7fe640 1 --2- 172.21.15.38:0/2329645016 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c451a5b8e0 0x55c451e4b4c0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f98a8005fd0 tx=0x7f98a8004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cca639a637a3ff0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:48.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.656+0000 7f9894ff9640 1 -- 172.21.15.38:0/2329645016 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98a8033030 con 0x55c451a5b8e0 2024-09-21T03:36:48.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.656+0000 7f9894ff9640 1 -- 172.21.15.38:0/2329645016 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f98a802fb80 con 0x55c451a5b8e0 2024-09-21T03:36:48.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.656+0000 7f9894ff9640 1 -- 172.21.15.38:0/2329645016 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98a8003970 con 0x55c451a5b8e0 2024-09-21T03:36:48.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.657+0000 7f98bc5bf740 1 -- 172.21.15.38:0/2329645016 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c451a5b8e0 msgr2=0x55c451e4b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:48.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.657+0000 7f98bc5bf740 1 --2- 172.21.15.38:0/2329645016 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c451a5b8e0 0x55c451e4b4c0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f98a8005fd0 tx=0x7f98a8004ab0 comp rx=0 tx=0).stop 2024-09-21T03:36:48.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.657+0000 7f98bc5bf740 1 -- 172.21.15.38:0/2329645016 shutdown_connections 2024-09-21T03:36:48.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.657+0000 7f98bc5bf740 1 --2- 172.21.15.38:0/2329645016 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c451a5b8e0 0x55c451e4b4c0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:48.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.657+0000 7f98bc5bf740 1 --2- 172.21.15.38:0/2329645016 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c451a53700 0x55c451e6a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:48.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.657+0000 7f98bc5bf740 1 --2- 172.21.15.38:0/2329645016 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55c451a7c5c0 0x55c451e3e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:48.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.657+0000 7f98bc5bf740 1 -- 172.21.15.38:0/2329645016 >> 172.21.15.38:0/2329645016 conn(0x55c451a70710 msgr2=0x55c451e247b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:48.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.657+0000 7f98bc5bf740 1 -- 172.21.15.38:0/2329645016 shutdown_connections 2024-09-21T03:36:48.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.657+0000 7f98bc5bf740 1 -- 172.21.15.38:0/2329645016 wait complete. 2024-09-21T03:36:48.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.658+0000 7f98bc5bf740 1 -- start start 2024-09-21T03:36:48.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.658+0000 7f98bc5bf740 1 Processor -- start 2024-09-21T03:36:48.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.659+0000 7f98bc5bf740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55c451a53700 0x55c451e52130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:48.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.659+0000 7f98bc5bf740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c451a5b8e0 0x55c451e7c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:48.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.660+0000 7f98bc5bf740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c451a7c5c0 0x55c451e63960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:48.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.660+0000 7f98bc5bf740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55c451af6970 con 0x55c451a5b8e0 2024-09-21T03:36:48.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.660+0000 7f98bc5bf740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55c451d2b5d0 con 0x55c451a7c5c0 2024-09-21T03:36:48.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.660+0000 7f98bc5bf740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55c451e35840 con 0x55c451a53700 2024-09-21T03:36:48.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.660+0000 7f98bc5bf740 10 client.0 _ll_register_callbacks cb 0x55c451d3abf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:36:48.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.660+0000 7f98ae7fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c451a5b8e0 0x55c451e7c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:48.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.660+0000 7f98ae7fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c451a5b8e0 0x55c451e7c8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:36742/0 (socket says 172.21.15.38:36742) 2024-09-21T03:36:48.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.660+0000 7f98ae7fc640 1 -- 172.21.15.38:0/1246943909 learned_addr learned my addr 172.21.15.38:0/1246943909 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:36:48.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.660+0000 7f98aeffd640 1 --2- 172.21.15.38:0/1246943909 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55c451a53700 0x55c451e52130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:48.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.660+0000 7f98af7fe640 1 --2- 172.21.15.38:0/1246943909 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c451a7c5c0 0x55c451e63960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:48.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.660+0000 7f98ae7fc640 1 -- 172.21.15.38:0/1246943909 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55c451a53700 msgr2=0x55c451e52130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:36:48.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.660+0000 7f98ae7fc640 1 --2- 172.21.15.38:0/1246943909 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55c451a53700 0x55c451e52130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:48.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.660+0000 7f98ae7fc640 1 -- 172.21.15.38:0/1246943909 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c451a7c5c0 msgr2=0x55c451e63960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:36:48.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.660+0000 7f98ae7fc640 1 --2- 172.21.15.38:0/1246943909 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c451a7c5c0 0x55c451e63960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:48.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.660+0000 7f98ae7fc640 1 -- 172.21.15.38:0/1246943909 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c451b60a30 con 0x55c451a5b8e0 2024-09-21T03:36:48.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.660+0000 7f98ae7fc640 1 --2- 172.21.15.38:0/1246943909 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c451a5b8e0 0x55c451e7c8d0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f98a40116f0 tx=0x7f98a4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9ad0cc23990035f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:48.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.660+0000 7f988dffb640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-21T03:36:48.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.660+0000 7f988dffb640 1 -- 172.21.15.38:0/1246943909 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98a401fa40 con 0x55c451a5b8e0 2024-09-21T03:36:48.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.661+0000 7f988dffb640 1 -- 172.21.15.38:0/1246943909 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(68..68 src has 1..68) ==== 6187+0+0 (secure 0 0 0) 0x7f98a4029cd0 con 0x55c451a5b8e0 2024-09-21T03:36:48.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.661+0000 7f98bc5bf740 1 -- 172.21.15.38:0/1246943909 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55c451d9ee60 con 0x55c451a5b8e0 2024-09-21T03:36:48.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.661+0000 7f988dffb640 1 -- 172.21.15.38:0/1246943909 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7f98a401fbe0 con 0x55c451a5b8e0 2024-09-21T03:36:48.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.661+0000 7f98bc5bf740 10 client.6285 fetch_fsmap learned FSMap version 53 2024-09-21T03:36:48.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.661+0000 7f98bc5bf740 1 -- 172.21.15.38:0/1246943909 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=2+}) -- 0x55c451ae7ba0 con 0x55c451a5b8e0 2024-09-21T03:36:48.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.661+0000 7f988dffb640 1 -- 172.21.15.38:0/1246943909 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55c451d9ee60 con 0x55c451a5b8e0 2024-09-21T03:36:48.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.661+0000 7f988dffb640 1 -- 172.21.15.38:0/1246943909 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 53) ==== 3502+0+0 (secure 0 0 0) 0x7f98a4030050 con 0x55c451a5b8e0 2024-09-21T03:36:48.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.661+0000 7f98bc5bf740 10 client.6285 fetch_fsmap finished waiting for FSMap version 53 2024-09-21T03:36:48.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.661+0000 7f98bc5bf740 10 client.6285 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-09-21T03:36:48.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.661+0000 7f98bc5bf740 20 client.6285 populate_metadata read hostname 'smithi038' 2024-09-21T03:36:48.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.661+0000 7f98bc5bf740 1 --2- 172.21.15.38:0/1246943909 >> [v2:172.21.15.38:6832/1058376326,v1:172.21.15.38:6834/1058376326] conn(0x55c451a3b190 0x55c451e80520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:48.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.661+0000 7f98bc5bf740 4 client.6285 mds_command: new command op to 6204 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:36:48.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.661+0000 7f98bc5bf740 1 -- 172.21.15.38:0/1246943909 --> [v2:172.21.15.38:6832/1058376326,v1:172.21.15.38:6834/1058376326] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c451d988f0 con 0x55c451a3b190 2024-09-21T03:36:48.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.661+0000 7f98aeffd640 1 --2- 172.21.15.38:0/1246943909 >> [v2:172.21.15.38:6832/1058376326,v1:172.21.15.38:6834/1058376326] conn(0x55c451a3b190 0x55c451e80520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:48.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.662+0000 7f988dffb640 10 client.6285 ms_handle_connect on v2:172.21.15.38:6832/1058376326 2024-09-21T03:36:48.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.662+0000 7f98aeffd640 1 --2- 172.21.15.38:0/1246943909 >> [v2:172.21.15.38:6832/1058376326,v1:172.21.15.38:6834/1058376326] conn(0x55c451a3b190 0x55c451e80520 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e51cbda3ef62dd3 server_cookie=fa4b3485d5a1939c in_seq=0 out_seq=0 2024-09-21T03:36:48.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.663+0000 7f988dffb640 1 -- 172.21.15.38:0/1246943909 <== mds.0 v2:172.21.15.38:6832/1058376326 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c451d988f0 con 0x55c451a3b190 2024-09-21T03:36:48.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.663+0000 7f988dffb640 10 client.6285 handle_command_reply: tid=0 2024-09-21T03:36:48.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.663+0000 7f98bc5bf740 1 client.6285 shutdown 2024-09-21T03:36:48.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.663+0000 7f98bc5bf740 1 -- 172.21.15.38:0/1246943909 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c451a5b8e0 msgr2=0x55c451e7c8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:48.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.663+0000 7f98bc5bf740 1 --2- 172.21.15.38:0/1246943909 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c451a5b8e0 0x55c451e7c8d0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f98a40116f0 tx=0x7f98a4004730 comp rx=0 tx=0).stop 2024-09-21T03:36:48.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.663+0000 7f98bc5bf740 1 -- 172.21.15.38:0/1246943909 shutdown_connections 2024-09-21T03:36:48.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.663+0000 7f98bc5bf740 1 --2- 172.21.15.38:0/1246943909 >> [v2:172.21.15.38:6832/1058376326,v1:172.21.15.38:6834/1058376326] conn(0x55c451a3b190 0x55c451e80520 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:48.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.663+0000 7f98bc5bf740 1 --2- 172.21.15.38:0/1246943909 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c451a7c5c0 0x55c451e63960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:48.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.663+0000 7f98bc5bf740 1 --2- 172.21.15.38:0/1246943909 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c451a5b8e0 0x55c451e7c8d0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:48.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.663+0000 7f98bc5bf740 1 --2- 172.21.15.38:0/1246943909 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55c451a53700 0x55c451e52130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:48.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.663+0000 7f988dffb640 0 client.6285 ms_handle_reset on v2:172.21.15.38:6832/1058376326 2024-09-21T03:36:48.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.663+0000 7f98bc5bf740 1 -- 172.21.15.38:0/1246943909 >> 172.21.15.38:0/1246943909 conn(0x55c451a70710 msgr2=0x55c451e3dd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:48.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.663+0000 7f98bc5bf740 1 -- 172.21.15.38:0/1246943909 shutdown_connections 2024-09-21T03:36:48.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.663+0000 7f98bc5bf740 1 -- 172.21.15.38:0/1246943909 wait complete. 2024-09-21T03:36:48.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.663+0000 7f98bc5bf740 20 client.6285 trim_cache size 0 max 16384 2024-09-21T03:36:48.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.667+0000 7f0e5db39640 1 -- 172.21.15.38:0/3884109434 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0e580cd300 msgr2=0x7f0e5815c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:48.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.667+0000 7f0e5db39640 1 --2- 172.21.15.38:0/3884109434 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0e580cd300 0x7f0e5815c7c0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f0e48009860 tx=0x7f0e4802a670 comp rx=0 tx=0).stop 2024-09-21T03:36:48.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.667+0000 7f0e5db39640 1 -- 172.21.15.38:0/3884109434 shutdown_connections 2024-09-21T03:36:48.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.667+0000 7f0e5db39640 1 --2- 172.21.15.38:0/3884109434 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0e580cd300 0x7f0e5815c7c0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:48.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.667+0000 7f0e5db39640 1 -- 172.21.15.38:0/3884109434 >> 172.21.15.38:0/3884109434 conn(0x7f0e580d2310 msgr2=0x7f0e580d2720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:48.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.668+0000 7f0e5db39640 1 -- 172.21.15.38:0/3884109434 shutdown_connections 2024-09-21T03:36:48.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.668+0000 7f0e5db39640 1 -- 172.21.15.38:0/3884109434 wait complete. 2024-09-21T03:36:48.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.669+0000 7f0e5db39640 1 Processor -- start 2024-09-21T03:36:48.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.670+0000 7f0e5db39640 1 -- start start 2024-09-21T03:36:48.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.671+0000 7f0e5db39640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0e580cd300 0x7f0e581f5830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:48.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.671+0000 7f0e5db39640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0e580cfee0 0x7f0e580b6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:48.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.671+0000 7f0e5db39640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0e580d14f0 0x7f0e5815ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:48.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.671+0000 7f0e5db39640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f0e580ccb70 con 0x7f0e580d14f0 2024-09-21T03:36:48.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.671+0000 7f0e5db39640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f0e580cc9f0 con 0x7f0e580cfee0 2024-09-21T03:36:48.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.671+0000 7f0e5db39640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f0e580d0a00 con 0x7f0e580cd300 2024-09-21T03:36:48.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.671+0000 7f0e57fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0e580d14f0 0x7f0e5815ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:48.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.671+0000 7f0e57fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0e580d14f0 0x7f0e5815ec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:36748/0 (socket says 172.21.15.38:36748) 2024-09-21T03:36:48.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.671+0000 7f0e57fff640 1 -- 172.21.15.38:0/3736328712 learned_addr learned my addr 172.21.15.38:0/3736328712 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:36:48.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.671+0000 7f0e577fe640 1 --2- 172.21.15.38:0/3736328712 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0e580cd300 0x7f0e581f5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:48.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.671+0000 7f0e56ffd640 1 --2- 172.21.15.38:0/3736328712 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0e580cfee0 0x7f0e580b6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:48.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.671+0000 7f0e57fff640 1 -- 172.21.15.38:0/3736328712 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0e580cd300 msgr2=0x7f0e581f5830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:48.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.671+0000 7f0e57fff640 1 --2- 172.21.15.38:0/3736328712 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0e580cd300 0x7f0e581f5830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:48.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.671+0000 7f0e57fff640 1 -- 172.21.15.38:0/3736328712 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0e580cfee0 msgr2=0x7f0e580b6060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:48.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.671+0000 7f0e57fff640 1 --2- 172.21.15.38:0/3736328712 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0e580cfee0 0x7f0e580b6060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:48.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.672+0000 7f0e57fff640 1 -- 172.21.15.38:0/3736328712 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e580b5520 con 0x7f0e580d14f0 2024-09-21T03:36:48.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.672+0000 7f0e57fff640 1 --2- 172.21.15.38:0/3736328712 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0e580d14f0 0x7f0e5815ec60 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f0e5000ca60 tx=0x7f0e5000cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:48.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.672+0000 7f0e3ffff640 1 -- 172.21.15.38:0/3736328712 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e50016020 con 0x7f0e580d14f0 2024-09-21T03:36:48.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.672+0000 7f0e3ffff640 1 -- 172.21.15.38:0/3736328712 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0e5000acc0 con 0x7f0e580d14f0 2024-09-21T03:36:48.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.672+0000 7f0e5db39640 1 -- 172.21.15.38:0/3736328712 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e5815e180 con 0x7f0e580d14f0 2024-09-21T03:36:48.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.672+0000 7f0e3ffff640 1 -- 172.21.15.38:0/3736328712 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e50012680 con 0x7f0e580d14f0 2024-09-21T03:36:48.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.672+0000 7f0e5db39640 1 -- 172.21.15.38:0/3736328712 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e5815e740 con 0x7f0e580d14f0 2024-09-21T03:36:48.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.673+0000 7f0e5db39640 1 -- 172.21.15.38:0/3736328712 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f0e580b5290 con 0x7f0e580d14f0 2024-09-21T03:36:48.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.674+0000 7f0e3ffff640 1 -- 172.21.15.38:0/3736328712 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f0e50019020 con 0x7f0e580d14f0 2024-09-21T03:36:48.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.675+0000 7f0e3ffff640 1 --2- 172.21.15.38:0/3736328712 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0e44078bc0 0x7f0e4407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:48.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.675+0000 7f0e3ffff640 1 -- 172.21.15.38:0/3736328712 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6187+0+0 (secure 0 0 0) 0x7f0e5009b860 con 0x7f0e580d14f0 2024-09-21T03:36:48.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.675+0000 7f0e577fe640 1 --2- 172.21.15.38:0/3736328712 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0e44078bc0 0x7f0e4407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:48.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.675+0000 7f0e3ffff640 1 -- 172.21.15.38:0/3736328712 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+513 (secure 0 0 0) 0x7f0e50020020 con 0x7f0e580d14f0 2024-09-21T03:36:48.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.675+0000 7f0e5db39640 1 -- 172.21.15.38:0/3736328712 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({log-info=0+}) -- 0x7f0e580d0e60 con 0x7f0e580d14f0 2024-09-21T03:36:48.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.675+0000 7f0e3ffff640 1 -- 172.21.15.38:0/3736328712 <== mon.0 v2:172.21.15.38:3300/0 7 ==== log(1 entries from seq 602 at 2024-09-21T03:36:47.741243+0000) ==== 308+0+0 (secure 0 0 0) 0x7f0e50099cc0 con 0x7f0e580d14f0 2024-09-21T03:36:48.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.676+0000 7f0e577fe640 1 --2- 172.21.15.38:0/3736328712 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0e44078bc0 0x7f0e4407b0d0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f0e48034000 tx=0x7f0e48004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:48.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.680+0000 7f98bc5bf740 1 Processor -- start 2024-09-21T03:36:48.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.680+0000 7f98bc5bf740 1 -- start start 2024-09-21T03:36:48.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.681+0000 7f98bc5bf740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c451a53700 0x55c451e36e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:48.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.681+0000 7f98bc5bf740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c451a5b8e0 0x55c451e1fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:48.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.681+0000 7f98bc5bf740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55c451a7c5c0 0x55c451e20370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:48.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.681+0000 7f98bc5bf740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f98b01fdfe0 con 0x55c451a53700 2024-09-21T03:36:48.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.681+0000 7f98bc5bf740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f98b01fcfe0 con 0x55c451a5b8e0 2024-09-21T03:36:48.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.681+0000 7f98bc5bf740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f98b0070db0 con 0x55c451a7c5c0 2024-09-21T03:36:48.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.681+0000 7f98aeffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c451a53700 0x55c451e36e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:48.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.681+0000 7f98aeffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c451a53700 0x55c451e36e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:36764/0 (socket says 172.21.15.38:36764) 2024-09-21T03:36:48.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.681+0000 7f98aeffd640 1 -- 172.21.15.38:0/1830221351 learned_addr learned my addr 172.21.15.38:0/1830221351 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:36:48.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.681+0000 7f98af7fe640 1 --2- 172.21.15.38:0/1830221351 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55c451a7c5c0 0x55c451e20370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:48.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.682+0000 7f98ae7fc640 1 --2- 172.21.15.38:0/1830221351 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c451a5b8e0 0x55c451e1fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:48.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.682+0000 7f98aeffd640 1 -- 172.21.15.38:0/1830221351 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55c451a7c5c0 msgr2=0x55c451e20370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:36:48.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.682+0000 7f98aeffd640 1 --2- 172.21.15.38:0/1830221351 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55c451a7c5c0 0x55c451e20370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:48.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.682+0000 7f98aeffd640 1 -- 172.21.15.38:0/1830221351 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c451a5b8e0 msgr2=0x55c451e1fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:36:48.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.682+0000 7f98aeffd640 1 --2- 172.21.15.38:0/1830221351 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c451a5b8e0 0x55c451e1fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:48.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.682+0000 7f98aeffd640 1 -- 172.21.15.38:0/1830221351 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c451e63cd0 con 0x55c451a53700 2024-09-21T03:36:48.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.682+0000 7f98ae7fc640 1 --2- 172.21.15.38:0/1830221351 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c451a5b8e0 0x55c451e1fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:36:48.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.682+0000 7f98aeffd640 1 --2- 172.21.15.38:0/1830221351 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c451a53700 0x55c451e36e00 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f98a0061bd0 tx=0x7f98a001e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cc620b0340023b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:48.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.682+0000 7f988ffff640 1 -- 172.21.15.38:0/1830221351 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98a00881e0 con 0x55c451a53700 2024-09-21T03:36:48.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.682+0000 7f988ffff640 1 -- 172.21.15.38:0/1830221351 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f98a0088030 con 0x55c451a53700 2024-09-21T03:36:48.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.682+0000 7f988ffff640 1 -- 172.21.15.38:0/1830221351 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98a006b7b0 con 0x55c451a53700 2024-09-21T03:36:48.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.683+0000 7f98bc5bf740 1 -- 172.21.15.38:0/1830221351 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c451a53700 msgr2=0x55c451e36e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:48.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.683+0000 7f98bc5bf740 1 --2- 172.21.15.38:0/1830221351 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c451a53700 0x55c451e36e00 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f98a0061bd0 tx=0x7f98a001e040 comp rx=0 tx=0).stop 2024-09-21T03:36:48.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.683+0000 7f98bc5bf740 1 -- 172.21.15.38:0/1830221351 shutdown_connections 2024-09-21T03:36:48.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.683+0000 7f98bc5bf740 1 --2- 172.21.15.38:0/1830221351 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55c451a7c5c0 0x55c451e20370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:48.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.683+0000 7f98bc5bf740 1 --2- 172.21.15.38:0/1830221351 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c451a5b8e0 0x55c451e1fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:48.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.683+0000 7f98bc5bf740 1 --2- 172.21.15.38:0/1830221351 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c451a53700 0x55c451e36e00 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:48.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.683+0000 7f98bc5bf740 1 -- 172.21.15.38:0/1830221351 >> 172.21.15.38:0/1830221351 conn(0x55c451a70710 msgr2=0x55c451e1db40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:48.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.683+0000 7f98bc5bf740 1 -- 172.21.15.38:0/1830221351 shutdown_connections 2024-09-21T03:36:48.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.683+0000 7f98bc5bf740 1 -- 172.21.15.38:0/1830221351 wait complete. 2024-09-21T03:36:48.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.684+0000 7f98bc5bf740 1 -- start start 2024-09-21T03:36:48.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.684+0000 7f98bc5bf740 1 Processor -- start 2024-09-21T03:36:48.688 INFO:teuthology.orchestra.run.smithi038.stdout:{"mds_beacon_grace":"15.000000"} 2024-09-21T03:36:48.688 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mds_beacon_grace": "15.000000" } 2024-09-21T03:36:48.688 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:36:48.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.686+0000 7f98bc5bf740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c451a53700 0x55c451e913c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:48.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.686+0000 7f98bc5bf740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55c451a5b8e0 0x55c451eec950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:48.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.686+0000 7f98bc5bf740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c451a7c5c0 0x55c451ef0f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:48.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.686+0000 7f98bc5bf740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55c451e24990 con 0x55c451a7c5c0 2024-09-21T03:36:48.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.686+0000 7f98bc5bf740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f98b01fc990 con 0x55c451a53700 2024-09-21T03:36:48.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.686+0000 7f98bc5bf740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55c451e5fc60 con 0x55c451a5b8e0 2024-09-21T03:36:48.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.686+0000 7f98bc5bf740 10 client.0 _ll_register_callbacks cb 0x7f98a4010e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:36:48.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.686+0000 7f98af7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c451a7c5c0 0x55c451ef0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:48.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.686+0000 7f98af7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c451a7c5c0 0x55c451ef0f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:36778/0 (socket says 172.21.15.38:36778) 2024-09-21T03:36:48.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.686+0000 7f98af7fe640 1 -- 172.21.15.38:0/117103957 learned_addr learned my addr 172.21.15.38:0/117103957 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:36:48.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.686+0000 7f98aeffd640 1 --2- 172.21.15.38:0/117103957 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c451a53700 0x55c451e913c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:48.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.686+0000 7f98ae7fc640 1 --2- 172.21.15.38:0/117103957 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55c451a5b8e0 0x55c451eec950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:48.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.686+0000 7f98af7fe640 1 -- 172.21.15.38:0/117103957 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55c451a5b8e0 msgr2=0x55c451eec950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:36:48.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.686+0000 7f98af7fe640 1 --2- 172.21.15.38:0/117103957 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55c451a5b8e0 0x55c451eec950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:48.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.686+0000 7f98af7fe640 1 -- 172.21.15.38:0/117103957 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c451a53700 msgr2=0x55c451e913c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:36:48.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.686+0000 7f98af7fe640 1 --2- 172.21.15.38:0/117103957 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c451a53700 0x55c451e913c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:48.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.686+0000 7f98af7fe640 1 -- 172.21.15.38:0/117103957 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c451ac58d0 con 0x55c451a7c5c0 2024-09-21T03:36:48.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.686+0000 7f98af7fe640 1 --2- 172.21.15.38:0/117103957 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c451a7c5c0 0x55c451ef0f70 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f98a8000f80 tx=0x7f98a802fbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c78c2c19e9196ab1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:48.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.686+0000 7f988dffb640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-21T03:36:48.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.686+0000 7f988dffb640 1 -- 172.21.15.38:0/117103957 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98a8050020 con 0x55c451a7c5c0 2024-09-21T03:36:48.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.686+0000 7f988dffb640 1 -- 172.21.15.38:0/117103957 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(68..68 src has 1..68) ==== 6187+0+0 (secure 0 0 0) 0x7f98a8003970 con 0x55c451a7c5c0 2024-09-21T03:36:48.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.687+0000 7f98bc5bf740 1 -- 172.21.15.38:0/117103957 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f98a0088030 con 0x55c451a7c5c0 2024-09-21T03:36:48.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.687+0000 7f988dffb640 1 -- 172.21.15.38:0/117103957 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7f98a805a050 con 0x55c451a7c5c0 2024-09-21T03:36:48.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.687+0000 7f98bc5bf740 10 client.6297 fetch_fsmap learned FSMap version 53 2024-09-21T03:36:48.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.687+0000 7f98bc5bf740 1 -- 172.21.15.38:0/117103957 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=2+}) -- 0x55c451ef0770 con 0x55c451a7c5c0 2024-09-21T03:36:48.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.687+0000 7f988dffb640 1 -- 172.21.15.38:0/117103957 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f98a0088030 con 0x55c451a7c5c0 2024-09-21T03:36:48.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.687+0000 7f988dffb640 1 -- 172.21.15.38:0/117103957 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 53) ==== 3502+0+0 (secure 0 0 0) 0x7f98a805d020 con 0x55c451a7c5c0 2024-09-21T03:36:48.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.687+0000 7f98bc5bf740 10 client.6297 fetch_fsmap finished waiting for FSMap version 53 2024-09-21T03:36:48.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.687+0000 7f98bc5bf740 10 client.6297 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-09-21T03:36:48.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.687+0000 7f98bc5bf740 20 client.6297 populate_metadata read hostname 'smithi038' 2024-09-21T03:36:48.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.687+0000 7f98bc5bf740 1 --2- 172.21.15.38:0/117103957 >> [v2:172.21.15.38:6832/1058376326,v1:172.21.15.38:6834/1058376326] conn(0x55c451e5f030 0x55c451e5f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:48.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.687+0000 7f98bc5bf740 4 client.6297 mds_command: new command op to 6204 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T03:36:48.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.687+0000 7f98bc5bf740 1 -- 172.21.15.38:0/117103957 --> [v2:172.21.15.38:6832/1058376326,v1:172.21.15.38:6834/1058376326] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c451d943a0 con 0x55c451e5f030 2024-09-21T03:36:48.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.687+0000 7f98aeffd640 1 --2- 172.21.15.38:0/117103957 >> [v2:172.21.15.38:6832/1058376326,v1:172.21.15.38:6834/1058376326] conn(0x55c451e5f030 0x55c451e5f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:48.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.688+0000 7f988dffb640 10 client.6297 ms_handle_connect on v2:172.21.15.38:6832/1058376326 2024-09-21T03:36:48.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.688+0000 7f98aeffd640 1 --2- 172.21.15.38:0/117103957 >> [v2:172.21.15.38:6832/1058376326,v1:172.21.15.38:6834/1058376326] conn(0x55c451e5f030 0x55c451e5f440 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=591325e40bee01df server_cookie=cde555255ee03107 in_seq=0 out_seq=0 2024-09-21T03:36:48.695 INFO:tasks.ceph.mds.a.smithi038.stderr:2024-09-21T03:36:48.690+0000 7f4471ad4640 -1 mds.0.cache.den(0x1 a) newly corrupt dentry to be committed: [dentry #0x1/a [fffffffffffffff6,head] auth (dversion lock) v=13 ino=0x10000000000 state=1610612736 | inodepin=1 dirty=1 0x55cdef393180] with next_snap: 2 2024-09-21T03:36:48.695 INFO:tasks.ceph.mds.a.smithi038.stderr:2024-09-21T03:36:48.690+0000 7f4471ad4640 -1 log_channel(cluster) log [ERR] : MDS abort because newly corrupt dentry to be committed: [dentry #0x1/a [fffffffffffffff6,head] auth (dversion lock) v=13 ino=0x10000000000 state=1610612736 | inodepin=1 dirty=1 0x55cdef393180] 2024-09-21T03:36:48.695 INFO:tasks.ceph.mds.a.smithi038.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5120-g18432218/rpm/el9/BUILD/ceph-19.3.0-5120-g18432218/src/mds/MDSRank.cc: In function 'void MDSRank::abort(std::string_view)' thread 7f4471ad4640 time 2024-09-21T03:36:48.691727+0000 2024-09-21T03:36:48.695 INFO:tasks.ceph.mds.a.smithi038.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5120-g18432218/rpm/el9/BUILD/ceph-19.3.0-5120-g18432218/src/mds/MDSRank.cc: 913: ceph_abort_msg("abort() called") 2024-09-21T03:36:48.695 INFO:tasks.ceph.mds.a.smithi038.stderr: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T03:36:48.696 INFO:tasks.ceph.mds.a.smithi038.stderr: 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xcf) [0x7f447e057214] 2024-09-21T03:36:48.696 INFO:tasks.ceph.mds.a.smithi038.stderr: 2: (MDSRank::abort(std::basic_string_view >)+0x5f) [0x55cdea265159] 2024-09-21T03:36:48.696 INFO:tasks.ceph.mds.a.smithi038.stderr: 3: (CDentry::check_corruption(bool)+0x54e) [0x55cdea5553ca] 2024-09-21T03:36:48.696 INFO:tasks.ceph.mds.a.smithi038.stderr: 4: ceph-mds(+0x40d845) [0x55cdea565845] 2024-09-21T03:36:48.696 INFO:tasks.ceph.mds.a.smithi038.stderr: 5: (CDir::_omap_commit(int)+0x7ff) [0x55cdea573e29] 2024-09-21T03:36:48.696 INFO:tasks.ceph.mds.a.smithi038.stderr: 6: (CDir::_commit(unsigned long, int)+0x5ba) [0x55cdea574bcc] 2024-09-21T03:36:48.696 INFO:tasks.ceph.mds.a.smithi038.stderr: 7: (CDir::commit(unsigned long, MDSContext*, bool, int)+0x11a) [0x55cdea564bc6] 2024-09-21T03:36:48.696 INFO:tasks.ceph.mds.a.smithi038.stderr: 8: (LogSegment::try_to_expire(MDSRank*, C_GatherBuilderBase >&, int)+0x10c1) [0x55cdea2d3057] 2024-09-21T03:36:48.696 INFO:tasks.ceph.mds.a.smithi038.stderr: 9: (MDLog::try_expire(LogSegment*, int)+0x6f) [0x55cdea6263e1] 2024-09-21T03:36:48.696 INFO:tasks.ceph.mds.a.smithi038.stderr: 10: (MDLog::trim_to(unsigned long)+0x922) [0x55cdea6271f4] 2024-09-21T03:36:48.696 INFO:tasks.ceph.mds.a.smithi038.stderr: 11: ceph-mds(+0x10e5a7) [0x55cdea2665a7] 2024-09-21T03:36:48.696 INFO:tasks.ceph.mds.a.smithi038.stderr: 12: ceph-mds(+0x10e730) [0x55cdea266730] 2024-09-21T03:36:48.696 INFO:tasks.ceph.mds.a.smithi038.stderr: 13: ceph-mds(+0x1574b5) [0x55cdea2af4b5] 2024-09-21T03:36:48.696 INFO:tasks.ceph.mds.a.smithi038.stderr: 14: ceph-mds(+0xed37f) [0x55cdea24537f] 2024-09-21T03:36:48.697 INFO:tasks.ceph.mds.a.smithi038.stderr: 15: (MDSInternalContextWrapper::finish(int)+0x12) [0x55cdea619770] 2024-09-21T03:36:48.697 INFO:tasks.ceph.mds.a.smithi038.stderr: 16: (MDSContext::complete(int)+0x7c) [0x55cdea61f6d8] 2024-09-21T03:36:48.697 INFO:tasks.ceph.mds.a.smithi038.stderr: 17: ceph-mds(+0x4c17c9) [0x55cdea6197c9] 2024-09-21T03:36:48.697 INFO:tasks.ceph.mds.a.smithi038.stderr: 18: (MDSContext::complete(int)+0x7c) [0x55cdea61f6d8] 2024-09-21T03:36:48.697 INFO:tasks.ceph.mds.a.smithi038.stderr: 19: (MDSIOContextBase::complete(int)+0x501) [0x55cdea61fd57] 2024-09-21T03:36:48.697 INFO:tasks.ceph.mds.a.smithi038.stderr: 20: (MDSLogContextBase::complete(int)+0x46) [0x55cdea61fdd4] 2024-09-21T03:36:48.697 INFO:tasks.ceph.mds.a.smithi038.stderr: 21: (Finisher::finisher_thread_entry()+0x495) [0x7f447dfde34b] 2024-09-21T03:36:48.697 INFO:tasks.ceph.mds.a.smithi038.stderr: 22: ceph-mds(+0x1052e1) [0x55cdea25d2e1] 2024-09-21T03:36:48.697 INFO:tasks.ceph.mds.a.smithi038.stderr: 23: (Thread::entry_wrapper()+0x43) [0x7f447e02fd57] 2024-09-21T03:36:48.697 INFO:tasks.ceph.mds.a.smithi038.stderr: 24: (Thread::_entry_func(void*)+0xd) [0x7f447e02fd73] 2024-09-21T03:36:48.697 INFO:tasks.ceph.mds.a.smithi038.stderr: 25: /lib64/libc.so.6(+0x89c52) [0x7f447d689c52] 2024-09-21T03:36:48.697 INFO:tasks.ceph.mds.a.smithi038.stderr: 26: /lib64/libc.so.6(+0x10ec80) [0x7f447d70ec80] 2024-09-21T03:36:48.697 INFO:tasks.ceph.mds.a.smithi038.stderr:2024-09-21T03:36:48.692+0000 7f4471ad4640 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5120-g18432218/rpm/el9/BUILD/ceph-19.3.0-5120-g18432218/src/mds/MDSRank.cc: In function 'void MDSRank::abort(std::string_view)' thread 7f4471ad4640 time 2024-09-21T03:36:48.691727+0000 2024-09-21T03:36:48.698 INFO:tasks.ceph.mds.a.smithi038.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5120-g18432218/rpm/el9/BUILD/ceph-19.3.0-5120-g18432218/src/mds/MDSRank.cc: 913: ceph_abort_msg("abort() called") 2024-09-21T03:36:48.698 INFO:tasks.ceph.mds.a.smithi038.stderr: 2024-09-21T03:36:48.698 INFO:tasks.ceph.mds.a.smithi038.stderr: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T03:36:48.698 INFO:tasks.ceph.mds.a.smithi038.stderr: 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xcf) [0x7f447e057214] 2024-09-21T03:36:48.698 INFO:tasks.ceph.mds.a.smithi038.stderr: 2: (MDSRank::abort(std::basic_string_view >)+0x5f) [0x55cdea265159] 2024-09-21T03:36:48.698 INFO:tasks.ceph.mds.a.smithi038.stderr: 3: (CDentry::check_corruption(bool)+0x54e) [0x55cdea5553ca] 2024-09-21T03:36:48.698 INFO:tasks.ceph.mds.a.smithi038.stderr: 4: ceph-mds(+0x40d845) [0x55cdea565845] 2024-09-21T03:36:48.698 INFO:tasks.ceph.mds.a.smithi038.stderr: 5: (CDir::_omap_commit(int)+0x7ff) [0x55cdea573e29] 2024-09-21T03:36:48.698 INFO:tasks.ceph.mds.a.smithi038.stderr: 6: (CDir::_commit(unsigned long, int)+0x5ba) [0x55cdea574bcc] 2024-09-21T03:36:48.698 INFO:tasks.ceph.mds.a.smithi038.stderr: 7: (CDir::commit(unsigned long, MDSContext*, bool, int)+0x11a) [0x55cdea564bc6] 2024-09-21T03:36:48.698 INFO:tasks.ceph.mds.a.smithi038.stderr: 8: (LogSegment::try_to_expire(MDSRank*, C_GatherBuilderBase >&, int)+0x10c1) [0x55cdea2d3057] 2024-09-21T03:36:48.698 INFO:tasks.ceph.mds.a.smithi038.stderr: 9: (MDLog::try_expire(LogSegment*, int)+0x6f) [0x55cdea6263e1] 2024-09-21T03:36:48.698 INFO:tasks.ceph.mds.a.smithi038.stderr: 10: (MDLog::trim_to(unsigned long)+0x922) [0x55cdea6271f4] 2024-09-21T03:36:48.699 INFO:tasks.ceph.mds.a.smithi038.stderr: 11: ceph-mds(+0x10e5a7) [0x55cdea2665a7] 2024-09-21T03:36:48.699 INFO:tasks.ceph.mds.a.smithi038.stderr: 12: ceph-mds(+0x10e730) [0x55cdea266730] 2024-09-21T03:36:48.699 INFO:tasks.ceph.mds.a.smithi038.stderr: 13: ceph-mds(+0x1574b5) [0x55cdea2af4b5] 2024-09-21T03:36:48.699 INFO:tasks.ceph.mds.a.smithi038.stderr: 14: ceph-mds(+0xed37f) [0x55cdea24537f] 2024-09-21T03:36:48.699 INFO:tasks.ceph.mds.a.smithi038.stderr: 15: (MDSInternalContextWrapper::finish(int)+0x12) [0x55cdea619770] 2024-09-21T03:36:48.699 INFO:tasks.ceph.mds.a.smithi038.stderr: 16: (MDSContext::complete(int)+0x7c) [0x55cdea61f6d8] 2024-09-21T03:36:48.699 INFO:tasks.ceph.mds.a.smithi038.stderr: 17: ceph-mds(+0x4c17c9) [0x55cdea6197c9] 2024-09-21T03:36:48.699 INFO:tasks.ceph.mds.a.smithi038.stderr: 18: (MDSContext::complete(int)+0x7c) [0x55cdea61f6d8] 2024-09-21T03:36:48.699 INFO:tasks.ceph.mds.a.smithi038.stderr: 19: (MDSIOContextBase::complete(int)+0x501) [0x55cdea61fd57] 2024-09-21T03:36:48.699 INFO:tasks.ceph.mds.a.smithi038.stderr: 20: (MDSLogContextBase::complete(int)+0x46) [0x55cdea61fdd4] 2024-09-21T03:36:48.699 INFO:tasks.ceph.mds.a.smithi038.stderr: 21: (Finisher::finisher_thread_entry()+0x495) [0x7f447dfde34b] 2024-09-21T03:36:48.699 INFO:tasks.ceph.mds.a.smithi038.stderr: 22: ceph-mds(+0x1052e1) [0x55cdea25d2e1] 2024-09-21T03:36:48.699 INFO:tasks.ceph.mds.a.smithi038.stderr: 23: (Thread::entry_wrapper()+0x43) [0x7f447e02fd57] 2024-09-21T03:36:48.699 INFO:tasks.ceph.mds.a.smithi038.stderr: 24: (Thread::_entry_func(void*)+0xd) [0x7f447e02fd73] 2024-09-21T03:36:48.700 INFO:tasks.ceph.mds.a.smithi038.stderr: 25: /lib64/libc.so.6(+0x89c52) [0x7f447d689c52] 2024-09-21T03:36:48.700 INFO:tasks.ceph.mds.a.smithi038.stderr: 26: /lib64/libc.so.6(+0x10ec80) [0x7f447d70ec80] 2024-09-21T03:36:48.700 INFO:tasks.ceph.mds.a.smithi038.stderr: 2024-09-21T03:36:48.700 INFO:tasks.ceph.mds.a.smithi038.stderr:*** Caught signal (Aborted) ** 2024-09-21T03:36:48.700 INFO:tasks.ceph.mds.a.smithi038.stderr: in thread 7f4471ad4640 thread_name:MR_Finisher 2024-09-21T03:36:48.700 INFO:tasks.ceph.mds.a.smithi038.stderr: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T03:36:48.700 INFO:tasks.ceph.mds.a.smithi038.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7f447d63e6f0] 2024-09-21T03:36:48.700 INFO:tasks.ceph.mds.a.smithi038.stderr: 2: /lib64/libc.so.6(+0x8b99c) [0x7f447d68b99c] 2024-09-21T03:36:48.700 INFO:tasks.ceph.mds.a.smithi038.stderr: 3: raise() 2024-09-21T03:36:48.700 INFO:tasks.ceph.mds.a.smithi038.stderr: 4: abort() 2024-09-21T03:36:48.700 INFO:tasks.ceph.mds.a.smithi038.stderr: 5: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0x241) [0x7f447e057386] 2024-09-21T03:36:48.700 INFO:tasks.ceph.mds.a.smithi038.stderr: 6: (MDSRank::abort(std::basic_string_view >)+0x5f) [0x55cdea265159] 2024-09-21T03:36:48.700 INFO:tasks.ceph.mds.a.smithi038.stderr: 7: (CDentry::check_corruption(bool)+0x54e) [0x55cdea5553ca] 2024-09-21T03:36:48.701 INFO:tasks.ceph.mds.a.smithi038.stderr: 8: ceph-mds(+0x40d845) [0x55cdea565845] 2024-09-21T03:36:48.701 INFO:tasks.ceph.mds.a.smithi038.stderr: 9: (CDir::_omap_commit(int)+0x7ff) [0x55cdea573e29] 2024-09-21T03:36:48.701 INFO:tasks.ceph.mds.a.smithi038.stderr: 10: (CDir::_commit(unsigned long, int)+0x5ba) [0x55cdea574bcc] 2024-09-21T03:36:48.701 INFO:tasks.ceph.mds.a.smithi038.stderr: 11: (CDir::commit(unsigned long, MDSContext*, bool, int)+0x11a) [0x55cdea564bc6] 2024-09-21T03:36:48.701 INFO:tasks.ceph.mds.a.smithi038.stderr: 12: (LogSegment::try_to_expire(MDSRank*, C_GatherBuilderBase >&, int)+0x10c1) [0x55cdea2d3057] 2024-09-21T03:36:48.701 INFO:tasks.ceph.mds.a.smithi038.stderr: 13: (MDLog::try_expire(LogSegment*, int)+0x6f) [0x55cdea6263e1] 2024-09-21T03:36:48.701 INFO:tasks.ceph.mds.a.smithi038.stderr: 14: (MDLog::trim_to(unsigned long)+0x922) [0x55cdea6271f4] 2024-09-21T03:36:48.701 INFO:tasks.ceph.mds.a.smithi038.stderr: 15: ceph-mds(+0x10e5a7) [0x55cdea2665a7] 2024-09-21T03:36:48.701 INFO:tasks.ceph.mds.a.smithi038.stderr: 16: ceph-mds(+0x10e730) [0x55cdea266730] 2024-09-21T03:36:48.701 INFO:tasks.ceph.mds.a.smithi038.stderr: 17: ceph-mds(+0x1574b5) [0x55cdea2af4b5] 2024-09-21T03:36:48.701 INFO:tasks.ceph.mds.a.smithi038.stderr: 18: ceph-mds(+0xed37f) [0x55cdea24537f] 2024-09-21T03:36:48.701 INFO:tasks.ceph.mds.a.smithi038.stderr: 19: (MDSInternalContextWrapper::finish(int)+0x12) [0x55cdea619770] 2024-09-21T03:36:48.701 INFO:tasks.ceph.mds.a.smithi038.stderr: 20: (MDSContext::complete(int)+0x7c) [0x55cdea61f6d8] 2024-09-21T03:36:48.701 INFO:tasks.ceph.mds.a.smithi038.stderr: 21: ceph-mds(+0x4c17c9) [0x55cdea6197c9] 2024-09-21T03:36:48.702 INFO:tasks.ceph.mds.a.smithi038.stderr: 22: (MDSContext::complete(int)+0x7c) [0x55cdea61f6d8] 2024-09-21T03:36:48.702 INFO:tasks.ceph.mds.a.smithi038.stderr: 23: (MDSIOContextBase::complete(int)+0x501) [0x55cdea61fd57] 2024-09-21T03:36:48.702 INFO:tasks.ceph.mds.a.smithi038.stderr: 24: (MDSLogContextBase::complete(int)+0x46) [0x55cdea61fdd4] 2024-09-21T03:36:48.702 INFO:tasks.ceph.mds.a.smithi038.stderr: 25: (Finisher::finisher_thread_entry()+0x495) [0x7f447dfde34b] 2024-09-21T03:36:48.702 INFO:tasks.ceph.mds.a.smithi038.stderr: 26: ceph-mds(+0x1052e1) [0x55cdea25d2e1] 2024-09-21T03:36:48.702 INFO:tasks.ceph.mds.a.smithi038.stderr: 27: (Thread::entry_wrapper()+0x43) [0x7f447e02fd57] 2024-09-21T03:36:48.702 INFO:tasks.ceph.mds.a.smithi038.stderr: 28: (Thread::_entry_func(void*)+0xd) [0x7f447e02fd73] 2024-09-21T03:36:48.702 INFO:tasks.ceph.mds.a.smithi038.stderr: 29: /lib64/libc.so.6(+0x89c52) [0x7f447d689c52] 2024-09-21T03:36:48.702 INFO:tasks.ceph.mds.a.smithi038.stderr: 30: /lib64/libc.so.6(+0x10ec80) [0x7f447d70ec80] 2024-09-21T03:36:48.702 INFO:tasks.ceph.mds.a.smithi038.stderr:2024-09-21T03:36:48.693+0000 7f4471ad4640 -1 *** Caught signal (Aborted) ** 2024-09-21T03:36:48.702 INFO:tasks.ceph.mds.a.smithi038.stderr: in thread 7f4471ad4640 thread_name:MR_Finisher 2024-09-21T03:36:48.702 INFO:tasks.ceph.mds.a.smithi038.stderr: 2024-09-21T03:36:48.702 INFO:tasks.ceph.mds.a.smithi038.stderr: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T03:36:48.702 INFO:tasks.ceph.mds.a.smithi038.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7f447d63e6f0] 2024-09-21T03:36:48.703 INFO:tasks.ceph.mds.a.smithi038.stderr: 2: /lib64/libc.so.6(+0x8b99c) [0x7f447d68b99c] 2024-09-21T03:36:48.703 INFO:tasks.ceph.mds.a.smithi038.stderr: 3: raise() 2024-09-21T03:36:48.703 INFO:tasks.ceph.mds.a.smithi038.stderr: 4: abort() 2024-09-21T03:36:48.703 INFO:tasks.ceph.mds.a.smithi038.stderr: 5: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0x241) [0x7f447e057386] 2024-09-21T03:36:48.703 INFO:tasks.ceph.mds.a.smithi038.stderr: 6: (MDSRank::abort(std::basic_string_view >)+0x5f) [0x55cdea265159] 2024-09-21T03:36:48.703 INFO:tasks.ceph.mds.a.smithi038.stderr: 7: (CDentry::check_corruption(bool)+0x54e) [0x55cdea5553ca] 2024-09-21T03:36:48.703 INFO:tasks.ceph.mds.a.smithi038.stderr: 8: ceph-mds(+0x40d845) [0x55cdea565845] 2024-09-21T03:36:48.703 INFO:tasks.ceph.mds.a.smithi038.stderr: 9: (CDir::_omap_commit(int)+0x7ff) [0x55cdea573e29] 2024-09-21T03:36:48.703 INFO:tasks.ceph.mds.a.smithi038.stderr: 10: (CDir::_commit(unsigned long, int)+0x5ba) [0x55cdea574bcc] 2024-09-21T03:36:48.703 INFO:tasks.ceph.mds.a.smithi038.stderr: 11: (CDir::commit(unsigned long, MDSContext*, bool, int)+0x11a) [0x55cdea564bc6] 2024-09-21T03:36:48.703 INFO:tasks.ceph.mds.a.smithi038.stderr: 12: (LogSegment::try_to_expire(MDSRank*, C_GatherBuilderBase >&, int)+0x10c1) [0x55cdea2d3057] 2024-09-21T03:36:48.703 INFO:tasks.ceph.mds.a.smithi038.stderr: 13: (MDLog::try_expire(LogSegment*, int)+0x6f) [0x55cdea6263e1] 2024-09-21T03:36:48.703 INFO:tasks.ceph.mds.a.smithi038.stderr: 14: (MDLog::trim_to(unsigned long)+0x922) [0x55cdea6271f4] 2024-09-21T03:36:48.704 INFO:tasks.ceph.mds.a.smithi038.stderr: 15: ceph-mds(+0x10e5a7) [0x55cdea2665a7] 2024-09-21T03:36:48.704 INFO:tasks.ceph.mds.a.smithi038.stderr: 16: ceph-mds(+0x10e730) [0x55cdea266730] 2024-09-21T03:36:48.704 INFO:tasks.ceph.mds.a.smithi038.stderr: 17: ceph-mds(+0x1574b5) [0x55cdea2af4b5] 2024-09-21T03:36:48.704 INFO:tasks.ceph.mds.a.smithi038.stderr: 18: ceph-mds(+0xed37f) [0x55cdea24537f] 2024-09-21T03:36:48.704 INFO:tasks.ceph.mds.a.smithi038.stderr: 19: (MDSInternalContextWrapper::finish(int)+0x12) [0x55cdea619770] 2024-09-21T03:36:48.704 INFO:tasks.ceph.mds.a.smithi038.stderr: 20: (MDSContext::complete(int)+0x7c) [0x55cdea61f6d8] 2024-09-21T03:36:48.704 INFO:tasks.ceph.mds.a.smithi038.stderr: 21: ceph-mds(+0x4c17c9) [0x55cdea6197c9] 2024-09-21T03:36:48.704 INFO:tasks.ceph.mds.a.smithi038.stderr: 22: (MDSContext::complete(int)+0x7c) [0x55cdea61f6d8] 2024-09-21T03:36:48.704 INFO:tasks.ceph.mds.a.smithi038.stderr: 23: (MDSIOContextBase::complete(int)+0x501) [0x55cdea61fd57] 2024-09-21T03:36:48.704 INFO:tasks.ceph.mds.a.smithi038.stderr: 24: (MDSLogContextBase::complete(int)+0x46) [0x55cdea61fdd4] 2024-09-21T03:36:48.704 INFO:tasks.ceph.mds.a.smithi038.stderr: 25: (Finisher::finisher_thread_entry()+0x495) [0x7f447dfde34b] 2024-09-21T03:36:48.704 INFO:tasks.ceph.mds.a.smithi038.stderr: 26: ceph-mds(+0x1052e1) [0x55cdea25d2e1] 2024-09-21T03:36:48.704 INFO:tasks.ceph.mds.a.smithi038.stderr: 27: (Thread::entry_wrapper()+0x43) [0x7f447e02fd57] 2024-09-21T03:36:48.704 INFO:tasks.ceph.mds.a.smithi038.stderr: 28: (Thread::_entry_func(void*)+0xd) [0x7f447e02fd73] 2024-09-21T03:36:48.704 INFO:tasks.ceph.mds.a.smithi038.stderr: 29: /lib64/libc.so.6(+0x89c52) [0x7f447d689c52] 2024-09-21T03:36:48.705 INFO:tasks.ceph.mds.a.smithi038.stderr: 30: /lib64/libc.so.6(+0x10ec80) [0x7f447d70ec80] 2024-09-21T03:36:48.705 INFO:tasks.ceph.mds.a.smithi038.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-09-21T03:36:48.705 INFO:tasks.ceph.mds.a.smithi038.stderr: 2024-09-21T03:36:48.706 INFO:tasks.ceph.mds.a.smithi038.stderr: -208> 2024-09-21T03:36:47.432+0000 7f4475adc640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T03:36:48.706 INFO:tasks.ceph.mds.a.smithi038.stderr: -12> 2024-09-21T03:36:48.690+0000 7f4471ad4640 -1 mds.0.cache.den(0x1 a) newly corrupt dentry to be committed: [dentry #0x1/a [fffffffffffffff6,head] auth (dversion lock) v=13 ino=0x10000000000 state=1610612736 | inodepin=1 dirty=1 0x55cdef393180] with next_snap: 2 2024-09-21T03:36:48.706 INFO:tasks.ceph.mds.a.smithi038.stderr: -10> 2024-09-21T03:36:48.690+0000 7f4471ad4640 -1 log_channel(cluster) log [ERR] : MDS abort because newly corrupt dentry to be committed: [dentry #0x1/a [fffffffffffffff6,head] auth (dversion lock) v=13 ino=0x10000000000 state=1610612736 | inodepin=1 dirty=1 0x55cdef393180] 2024-09-21T03:36:48.706 INFO:tasks.ceph.mds.a.smithi038.stderr: -2> 2024-09-21T03:36:48.692+0000 7f4471ad4640 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5120-g18432218/rpm/el9/BUILD/ceph-19.3.0-5120-g18432218/src/mds/MDSRank.cc: In function 'void MDSRank::abort(std::string_view)' thread 7f4471ad4640 time 2024-09-21T03:36:48.691727+0000 2024-09-21T03:36:48.706 INFO:tasks.ceph.mds.a.smithi038.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5120-g18432218/rpm/el9/BUILD/ceph-19.3.0-5120-g18432218/src/mds/MDSRank.cc: 913: ceph_abort_msg("abort() called") 2024-09-21T03:36:48.706 INFO:tasks.ceph.mds.a.smithi038.stderr: 2024-09-21T03:36:48.706 INFO:tasks.ceph.mds.a.smithi038.stderr: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T03:36:48.706 INFO:tasks.ceph.mds.a.smithi038.stderr: 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xcf) [0x7f447e057214] 2024-09-21T03:36:48.706 INFO:tasks.ceph.mds.a.smithi038.stderr: 2: (MDSRank::abort(std::basic_string_view >)+0x5f) [0x55cdea265159] 2024-09-21T03:36:48.706 INFO:tasks.ceph.mds.a.smithi038.stderr: 3: (CDentry::check_corruption(bool)+0x54e) [0x55cdea5553ca] 2024-09-21T03:36:48.707 INFO:tasks.ceph.mds.a.smithi038.stderr: 4: ceph-mds(+0x40d845) [0x55cdea565845] 2024-09-21T03:36:48.707 INFO:tasks.ceph.mds.a.smithi038.stderr: 5: (CDir::_omap_commit(int)+0x7ff) [0x55cdea573e29] 2024-09-21T03:36:48.707 INFO:tasks.ceph.mds.a.smithi038.stderr: 6: (CDir::_commit(unsigned long, int)+0x5ba) [0x55cdea574bcc] 2024-09-21T03:36:48.707 INFO:tasks.ceph.mds.a.smithi038.stderr: 7: (CDir::commit(unsigned long, MDSContext*, bool, int)+0x11a) [0x55cdea564bc6] 2024-09-21T03:36:48.707 INFO:tasks.ceph.mds.a.smithi038.stderr: 8: (LogSegment::try_to_expire(MDSRank*, C_GatherBuilderBase >&, int)+0x10c1) [0x55cdea2d3057] 2024-09-21T03:36:48.707 INFO:tasks.ceph.mds.a.smithi038.stderr: 9: (MDLog::try_expire(LogSegment*, int)+0x6f) [0x55cdea6263e1] 2024-09-21T03:36:48.707 INFO:tasks.ceph.mds.a.smithi038.stderr: 10: (MDLog::trim_to(unsigned long)+0x922) [0x55cdea6271f4] 2024-09-21T03:36:48.707 INFO:tasks.ceph.mds.a.smithi038.stderr: 11: ceph-mds(+0x10e5a7) [0x55cdea2665a7] 2024-09-21T03:36:48.707 INFO:tasks.ceph.mds.a.smithi038.stderr: 12: ceph-mds(+0x10e730) [0x55cdea266730] 2024-09-21T03:36:48.707 INFO:tasks.ceph.mds.a.smithi038.stderr: 13: ceph-mds(+0x1574b5) [0x55cdea2af4b5] 2024-09-21T03:36:48.707 INFO:tasks.ceph.mds.a.smithi038.stderr: 14: ceph-mds(+0xed37f) [0x55cdea24537f] 2024-09-21T03:36:48.707 INFO:tasks.ceph.mds.a.smithi038.stderr: 15: (MDSInternalContextWrapper::finish(int)+0x12) [0x55cdea619770] 2024-09-21T03:36:48.707 INFO:tasks.ceph.mds.a.smithi038.stderr: 16: (MDSContext::complete(int)+0x7c) [0x55cdea61f6d8] 2024-09-21T03:36:48.707 INFO:tasks.ceph.mds.a.smithi038.stderr: 17: ceph-mds(+0x4c17c9) [0x55cdea6197c9] 2024-09-21T03:36:48.708 INFO:tasks.ceph.mds.a.smithi038.stderr: 18: (MDSContext::complete(int)+0x7c) [0x55cdea61f6d8] 2024-09-21T03:36:48.708 INFO:tasks.ceph.mds.a.smithi038.stderr: 19: (MDSIOContextBase::complete(int)+0x501) [0x55cdea61fd57] 2024-09-21T03:36:48.708 INFO:tasks.ceph.mds.a.smithi038.stderr: 20: (MDSLogContextBase::complete(int)+0x46) [0x55cdea61fdd4] 2024-09-21T03:36:48.708 INFO:tasks.ceph.mds.a.smithi038.stderr: 21: (Finisher::finisher_thread_entry()+0x495) [0x7f447dfde34b] 2024-09-21T03:36:48.708 INFO:tasks.ceph.mds.a.smithi038.stderr: 22: ceph-mds(+0x1052e1) [0x55cdea25d2e1] 2024-09-21T03:36:48.708 INFO:tasks.ceph.mds.a.smithi038.stderr: 23: (Thread::entry_wrapper()+0x43) [0x7f447e02fd57] 2024-09-21T03:36:48.708 INFO:tasks.ceph.mds.a.smithi038.stderr: 24: (Thread::_entry_func(void*)+0xd) [0x7f447e02fd73] 2024-09-21T03:36:48.708 INFO:tasks.ceph.mds.a.smithi038.stderr: 25: /lib64/libc.so.6(+0x89c52) [0x7f447d689c52] 2024-09-21T03:36:48.708 INFO:tasks.ceph.mds.a.smithi038.stderr: 26: /lib64/libc.so.6(+0x10ec80) [0x7f447d70ec80] 2024-09-21T03:36:48.708 INFO:tasks.ceph.mds.a.smithi038.stderr: 2024-09-21T03:36:48.708 INFO:tasks.ceph.mds.a.smithi038.stderr: 0> 2024-09-21T03:36:48.693+0000 7f4471ad4640 -1 *** Caught signal (Aborted) ** 2024-09-21T03:36:48.708 INFO:tasks.ceph.mds.a.smithi038.stderr: in thread 7f4471ad4640 thread_name:MR_Finisher 2024-09-21T03:36:48.708 INFO:tasks.ceph.mds.a.smithi038.stderr: 2024-09-21T03:36:48.708 INFO:tasks.ceph.mds.a.smithi038.stderr: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T03:36:48.709 INFO:tasks.ceph.mds.a.smithi038.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7f447d63e6f0] 2024-09-21T03:36:48.709 INFO:tasks.ceph.mds.a.smithi038.stderr: 2: /lib64/libc.so.6(+0x8b99c) [0x7f447d68b99c] 2024-09-21T03:36:48.709 INFO:tasks.ceph.mds.a.smithi038.stderr: 3: raise() 2024-09-21T03:36:48.709 INFO:tasks.ceph.mds.a.smithi038.stderr: 4: abort() 2024-09-21T03:36:48.709 INFO:tasks.ceph.mds.a.smithi038.stderr: 5: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0x241) [0x7f447e057386] 2024-09-21T03:36:48.709 INFO:tasks.ceph.mds.a.smithi038.stderr: 6: (MDSRank::abort(std::basic_string_view >)+0x5f) [0x55cdea265159] 2024-09-21T03:36:48.709 INFO:tasks.ceph.mds.a.smithi038.stderr: 7: (CDentry::check_corruption(bool)+0x54e) [0x55cdea5553ca] 2024-09-21T03:36:48.709 INFO:tasks.ceph.mds.a.smithi038.stderr: 8: ceph-mds(+0x40d845) [0x55cdea565845] 2024-09-21T03:36:48.709 INFO:tasks.ceph.mds.a.smithi038.stderr: 9: (CDir::_omap_commit(int)+0x7ff) [0x55cdea573e29] 2024-09-21T03:36:48.709 INFO:tasks.ceph.mds.a.smithi038.stderr: 10: (CDir::_commit(unsigned long, int)+0x5ba) [0x55cdea574bcc] 2024-09-21T03:36:48.709 INFO:tasks.ceph.mds.a.smithi038.stderr: 11: (CDir::commit(unsigned long, MDSContext*, bool, int)+0x11a) [0x55cdea564bc6] 2024-09-21T03:36:48.709 INFO:tasks.ceph.mds.a.smithi038.stderr: 12: (LogSegment::try_to_expire(MDSRank*, C_GatherBuilderBase >&, int)+0x10c1) [0x55cdea2d3057] 2024-09-21T03:36:48.709 INFO:tasks.ceph.mds.a.smithi038.stderr: 13: (MDLog::try_expire(LogSegment*, int)+0x6f) [0x55cdea6263e1] 2024-09-21T03:36:48.709 INFO:tasks.ceph.mds.a.smithi038.stderr: 14: (MDLog::trim_to(unsigned long)+0x922) [0x55cdea6271f4] 2024-09-21T03:36:48.710 INFO:tasks.ceph.mds.a.smithi038.stderr: 15: ceph-mds(+0x10e5a7) [0x55cdea2665a7] 2024-09-21T03:36:48.710 INFO:tasks.ceph.mds.a.smithi038.stderr: 16: ceph-mds(+0x10e730) [0x55cdea266730] 2024-09-21T03:36:48.710 INFO:tasks.ceph.mds.a.smithi038.stderr: 17: ceph-mds(+0x1574b5) [0x55cdea2af4b5] 2024-09-21T03:36:48.710 INFO:tasks.ceph.mds.a.smithi038.stderr: 18: ceph-mds(+0xed37f) [0x55cdea24537f] 2024-09-21T03:36:48.710 INFO:tasks.ceph.mds.a.smithi038.stderr: 19: (MDSInternalContextWrapper::finish(int)+0x12) [0x55cdea619770] 2024-09-21T03:36:48.710 INFO:tasks.ceph.mds.a.smithi038.stderr: 20: (MDSContext::complete(int)+0x7c) [0x55cdea61f6d8] 2024-09-21T03:36:48.710 INFO:tasks.ceph.mds.a.smithi038.stderr: 21: ceph-mds(+0x4c17c9) [0x55cdea6197c9] 2024-09-21T03:36:48.710 INFO:tasks.ceph.mds.a.smithi038.stderr: 22: (MDSContext::complete(int)+0x7c) [0x55cdea61f6d8] 2024-09-21T03:36:48.710 INFO:tasks.ceph.mds.a.smithi038.stderr: 23: (MDSIOContextBase::complete(int)+0x501) [0x55cdea61fd57] 2024-09-21T03:36:48.710 INFO:tasks.ceph.mds.a.smithi038.stderr: 24: (MDSLogContextBase::complete(int)+0x46) [0x55cdea61fdd4] 2024-09-21T03:36:48.710 INFO:tasks.ceph.mds.a.smithi038.stderr: 25: (Finisher::finisher_thread_entry()+0x495) [0x7f447dfde34b] 2024-09-21T03:36:48.710 INFO:tasks.ceph.mds.a.smithi038.stderr: 26: ceph-mds(+0x1052e1) [0x55cdea25d2e1] 2024-09-21T03:36:48.710 INFO:tasks.ceph.mds.a.smithi038.stderr: 27: (Thread::entry_wrapper()+0x43) [0x7f447e02fd57] 2024-09-21T03:36:48.711 INFO:tasks.ceph.mds.a.smithi038.stderr: 28: (Thread::_entry_func(void*)+0xd) [0x7f447e02fd73] 2024-09-21T03:36:48.711 INFO:tasks.ceph.mds.a.smithi038.stderr: 29: /lib64/libc.so.6(+0x89c52) [0x7f447d689c52] 2024-09-21T03:36:48.711 INFO:tasks.ceph.mds.a.smithi038.stderr: 30: /lib64/libc.so.6(+0x10ec80) [0x7f447d70ec80] 2024-09-21T03:36:48.711 INFO:tasks.ceph.mds.a.smithi038.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-09-21T03:36:48.711 INFO:tasks.ceph.mds.a.smithi038.stderr: 2024-09-21T03:36:48.711 INFO:tasks.ceph.mds.a.smithi038.stderr: -208> 2024-09-21T03:36:47.432+0000 7f4475adc640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T03:36:48.711 INFO:tasks.ceph.mds.a.smithi038.stderr: -12> 2024-09-21T03:36:48.690+0000 7f4471ad4640 -1 mds.0.cache.den(0x1 a) newly corrupt dentry to be committed: [dentry #0x1/a [fffffffffffffff6,head] auth (dversion lock) v=13 ino=0x10000000000 state=1610612736 | inodepin=1 dirty=1 0x55cdef393180] with next_snap: 2 2024-09-21T03:36:48.711 INFO:tasks.ceph.mds.a.smithi038.stderr: -10> 2024-09-21T03:36:48.690+0000 7f4471ad4640 -1 log_channel(cluster) log [ERR] : MDS abort because newly corrupt dentry to be committed: [dentry #0x1/a [fffffffffffffff6,head] auth (dversion lock) v=13 ino=0x10000000000 state=1610612736 | inodepin=1 dirty=1 0x55cdef393180] 2024-09-21T03:36:48.711 INFO:tasks.ceph.mds.a.smithi038.stderr: -2> 2024-09-21T03:36:48.692+0000 7f4471ad4640 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5120-g18432218/rpm/el9/BUILD/ceph-19.3.0-5120-g18432218/src/mds/MDSRank.cc: In function 'void MDSRank::abort(std::string_view)' thread 7f4471ad4640 time 2024-09-21T03:36:48.691727+0000 2024-09-21T03:36:48.711 INFO:tasks.ceph.mds.a.smithi038.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5120-g18432218/rpm/el9/BUILD/ceph-19.3.0-5120-g18432218/src/mds/MDSRank.cc: 913: ceph_abort_msg("abort() called") 2024-09-21T03:36:48.711 INFO:tasks.ceph.mds.a.smithi038.stderr: 2024-09-21T03:36:48.711 INFO:tasks.ceph.mds.a.smithi038.stderr: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T03:36:48.711 INFO:tasks.ceph.mds.a.smithi038.stderr: 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xcf) [0x7f447e057214] 2024-09-21T03:36:48.712 INFO:tasks.ceph.mds.a.smithi038.stderr: 2: (MDSRank::abort(std::basic_string_view >)+0x5f) [0x55cdea265159] 2024-09-21T03:36:48.712 INFO:tasks.ceph.mds.a.smithi038.stderr: 3: (CDentry::check_corruption(bool)+0x54e) [0x55cdea5553ca] 2024-09-21T03:36:48.712 INFO:tasks.ceph.mds.a.smithi038.stderr: 4: ceph-mds(+0x40d845) [0x55cdea565845] 2024-09-21T03:36:48.712 INFO:tasks.ceph.mds.a.smithi038.stderr: 5: (CDir::_omap_commit(int)+0x7ff) [0x55cdea573e29] 2024-09-21T03:36:48.712 INFO:tasks.ceph.mds.a.smithi038.stderr: 6: (CDir::_commit(unsigned long, int)+0x5ba) [0x55cdea574bcc] 2024-09-21T03:36:48.712 INFO:tasks.ceph.mds.a.smithi038.stderr: 7: (CDir::commit(unsigned long, MDSContext*, bool, int)+0x11a) [0x55cdea564bc6] 2024-09-21T03:36:48.712 INFO:tasks.ceph.mds.a.smithi038.stderr: 8: (LogSegment::try_to_expire(MDSRank*, C_GatherBuilderBase >&, int)+0x10c1) [0x55cdea2d3057] 2024-09-21T03:36:48.712 INFO:tasks.ceph.mds.a.smithi038.stderr: 9: (MDLog::try_expire(LogSegment*, int)+0x6f) [0x55cdea6263e1] 2024-09-21T03:36:48.712 INFO:tasks.ceph.mds.a.smithi038.stderr: 10: (MDLog::trim_to(unsigned long)+0x922) [0x55cdea6271f4] 2024-09-21T03:36:48.712 INFO:tasks.ceph.mds.a.smithi038.stderr: 11: ceph-mds(+0x10e5a7) [0x55cdea2665a7] 2024-09-21T03:36:48.712 INFO:tasks.ceph.mds.a.smithi038.stderr: 12: ceph-mds(+0x10e730) [0x55cdea266730] 2024-09-21T03:36:48.712 INFO:tasks.ceph.mds.a.smithi038.stderr: 13: ceph-mds(+0x1574b5) [0x55cdea2af4b5] 2024-09-21T03:36:48.712 INFO:tasks.ceph.mds.a.smithi038.stderr: 14: ceph-mds(+0xed37f) [0x55cdea24537f] 2024-09-21T03:36:48.712 INFO:tasks.ceph.mds.a.smithi038.stderr: 15: (MDSInternalContextWrapper::finish(int)+0x12) [0x55cdea619770] 2024-09-21T03:36:48.713 INFO:tasks.ceph.mds.a.smithi038.stderr: 16: (MDSContext::complete(int)+0x7c) [0x55cdea61f6d8] 2024-09-21T03:36:48.713 INFO:tasks.ceph.mds.a.smithi038.stderr: 17: ceph-mds(+0x4c17c9) [0x55cdea6197c9] 2024-09-21T03:36:48.713 INFO:tasks.ceph.mds.a.smithi038.stderr: 18: (MDSContext::complete(int)+0x7c) [0x55cdea61f6d8] 2024-09-21T03:36:48.713 INFO:tasks.ceph.mds.a.smithi038.stderr: 19: (MDSIOContextBase::complete(int)+0x501) [0x55cdea61fd57] 2024-09-21T03:36:48.713 INFO:tasks.ceph.mds.a.smithi038.stderr: 20: (MDSLogContextBase::complete(int)+0x46) [0x55cdea61fdd4] 2024-09-21T03:36:48.713 INFO:tasks.ceph.mds.a.smithi038.stderr: 21: (Finisher::finisher_thread_entry()+0x495) [0x7f447dfde34b] 2024-09-21T03:36:48.713 INFO:tasks.ceph.mds.a.smithi038.stderr: 22: ceph-mds(+0x1052e1) [0x55cdea25d2e1] 2024-09-21T03:36:48.713 INFO:tasks.ceph.mds.a.smithi038.stderr: 23: (Thread::entry_wrapper()+0x43) [0x7f447e02fd57] 2024-09-21T03:36:48.713 INFO:tasks.ceph.mds.a.smithi038.stderr: 24: (Thread::_entry_func(void*)+0xd) [0x7f447e02fd73] 2024-09-21T03:36:48.713 INFO:tasks.ceph.mds.a.smithi038.stderr: 25: /lib64/libc.so.6(+0x89c52) [0x7f447d689c52] 2024-09-21T03:36:48.713 INFO:tasks.ceph.mds.a.smithi038.stderr: 26: /lib64/libc.so.6(+0x10ec80) [0x7f447d70ec80] 2024-09-21T03:36:48.713 INFO:tasks.ceph.mds.a.smithi038.stderr: 2024-09-21T03:36:48.713 INFO:tasks.ceph.mds.a.smithi038.stderr: 0> 2024-09-21T03:36:48.693+0000 7f4471ad4640 -1 *** Caught signal (Aborted) ** 2024-09-21T03:36:48.714 INFO:tasks.ceph.mds.a.smithi038.stderr: in thread 7f4471ad4640 thread_name:MR_Finisher 2024-09-21T03:36:48.714 INFO:tasks.ceph.mds.a.smithi038.stderr: 2024-09-21T03:36:48.714 INFO:tasks.ceph.mds.a.smithi038.stderr: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T03:36:48.714 INFO:tasks.ceph.mds.a.smithi038.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7f447d63e6f0] 2024-09-21T03:36:48.714 INFO:tasks.ceph.mds.a.smithi038.stderr: 2: /lib64/libc.so.6(+0x8b99c) [0x7f447d68b99c] 2024-09-21T03:36:48.714 INFO:tasks.ceph.mds.a.smithi038.stderr: 3: raise() 2024-09-21T03:36:48.714 INFO:tasks.ceph.mds.a.smithi038.stderr: 4: abort() 2024-09-21T03:36:48.714 INFO:tasks.ceph.mds.a.smithi038.stderr: 5: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0x241) [0x7f447e057386] 2024-09-21T03:36:48.714 INFO:tasks.ceph.mds.a.smithi038.stderr: 6: (MDSRank::abort(std::basic_string_view >)+0x5f) [0x55cdea265159] 2024-09-21T03:36:48.714 INFO:tasks.ceph.mds.a.smithi038.stderr: 7: (CDentry::check_corruption(bool)+0x54e) [0x55cdea5553ca] 2024-09-21T03:36:48.714 INFO:tasks.ceph.mds.a.smithi038.stderr: 8: ceph-mds(+0x40d845) [0x55cdea565845] 2024-09-21T03:36:48.714 INFO:tasks.ceph.mds.a.smithi038.stderr: 9: (CDir::_omap_commit(int)+0x7ff) [0x55cdea573e29] 2024-09-21T03:36:48.714 INFO:tasks.ceph.mds.a.smithi038.stderr: 10: (CDir::_commit(unsigned long, int)+0x5ba) [0x55cdea574bcc] 2024-09-21T03:36:48.715 INFO:tasks.ceph.mds.a.smithi038.stderr: 11: (CDir::commit(unsigned long, MDSContext*, bool, int)+0x11a) [0x55cdea564bc6] 2024-09-21T03:36:48.715 INFO:tasks.ceph.mds.a.smithi038.stderr: 12: (LogSegment::try_to_expire(MDSRank*, C_GatherBuilderBase >&, int)+0x10c1) [0x55cdea2d3057] 2024-09-21T03:36:48.715 INFO:tasks.ceph.mds.a.smithi038.stderr: 13: (MDLog::try_expire(LogSegment*, int)+0x6f) [0x55cdea6263e1] 2024-09-21T03:36:48.715 INFO:tasks.ceph.mds.a.smithi038.stderr: 14: (MDLog::trim_to(unsigned long)+0x922) [0x55cdea6271f4] 2024-09-21T03:36:48.715 INFO:tasks.ceph.mds.a.smithi038.stderr: 15: ceph-mds(+0x10e5a7) [0x55cdea2665a7] 2024-09-21T03:36:48.715 INFO:tasks.ceph.mds.a.smithi038.stderr: 16: ceph-mds(+0x10e730) [0x55cdea266730] 2024-09-21T03:36:48.715 INFO:tasks.ceph.mds.a.smithi038.stderr: 17: ceph-mds(+0x1574b5) [0x55cdea2af4b5] 2024-09-21T03:36:48.715 INFO:tasks.ceph.mds.a.smithi038.stderr: 18: ceph-mds(+0xed37f) [0x55cdea24537f] 2024-09-21T03:36:48.715 INFO:tasks.ceph.mds.a.smithi038.stderr: 19: (MDSInternalContextWrapper::finish(int)+0x12) [0x55cdea619770] 2024-09-21T03:36:48.715 INFO:tasks.ceph.mds.a.smithi038.stderr: 20: (MDSContext::complete(int)+0x7c) [0x55cdea61f6d8] 2024-09-21T03:36:48.715 INFO:tasks.ceph.mds.a.smithi038.stderr: 21: ceph-mds(+0x4c17c9) [0x55cdea6197c9] 2024-09-21T03:36:48.715 INFO:tasks.ceph.mds.a.smithi038.stderr: 22: (MDSContext::complete(int)+0x7c) [0x55cdea61f6d8] 2024-09-21T03:36:48.715 INFO:tasks.ceph.mds.a.smithi038.stderr: 23: (MDSIOContextBase::complete(int)+0x501) [0x55cdea61fd57] 2024-09-21T03:36:48.715 INFO:tasks.ceph.mds.a.smithi038.stderr: 24: (MDSLogContextBase::complete(int)+0x46) [0x55cdea61fdd4] 2024-09-21T03:36:48.716 INFO:tasks.ceph.mds.a.smithi038.stderr: 25: (Finisher::finisher_thread_entry()+0x495) [0x7f447dfde34b] 2024-09-21T03:36:48.716 INFO:tasks.ceph.mds.a.smithi038.stderr: 26: ceph-mds(+0x1052e1) [0x55cdea25d2e1] 2024-09-21T03:36:48.716 INFO:tasks.ceph.mds.a.smithi038.stderr: 27: (Thread::entry_wrapper()+0x43) [0x7f447e02fd57] 2024-09-21T03:36:48.716 INFO:tasks.ceph.mds.a.smithi038.stderr: 28: (Thread::_entry_func(void*)+0xd) [0x7f447e02fd73] 2024-09-21T03:36:48.716 INFO:tasks.ceph.mds.a.smithi038.stderr: 29: /lib64/libc.so.6(+0x89c52) [0x7f447d689c52] 2024-09-21T03:36:48.716 INFO:tasks.ceph.mds.a.smithi038.stderr: 30: /lib64/libc.so.6(+0x10ec80) [0x7f447d70ec80] 2024-09-21T03:36:48.716 INFO:tasks.ceph.mds.a.smithi038.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-09-21T03:36:48.716 INFO:tasks.ceph.mds.a.smithi038.stderr: 2024-09-21T03:36:48.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.762+0000 7f98aeffd640 1 -- 172.21.15.38:0/117103957 >> [v2:172.21.15.38:6832/1058376326,v1:172.21.15.38:6834/1058376326] conn(0x55c451e5f030 msgr2=0x55c451e5f440 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 18 2024-09-21T03:36:48.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.762+0000 7f98aeffd640 1 -- 172.21.15.38:0/117103957 >> [v2:172.21.15.38:6832/1058376326,v1:172.21.15.38:6834/1058376326] conn(0x55c451e5f030 msgr2=0x55c451e5f440 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-21T03:36:48.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.762+0000 7f98aeffd640 1 --2- 172.21.15.38:0/117103957 >> [v2:172.21.15.38:6832/1058376326,v1:172.21.15.38:6834/1058376326] conn(0x55c451e5f030 0x55c451e5f440 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-21T03:36:48.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.762+0000 7f98aeffd640 1 --2- 172.21.15.38:0/117103957 >> [v2:172.21.15.38:6832/1058376326,v1:172.21.15.38:6834/1058376326] conn(0x55c451e5f030 0x55c451e5f440 unknown :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault initiating reconnect 2024-09-21T03:36:48.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.762+0000 7f98aeffd640 1 -- 172.21.15.38:0/117103957 >> [v2:172.21.15.38:6832/1058376326,v1:172.21.15.38:6834/1058376326] conn(0x55c451e5f030 msgr2=0x55c451e5f440 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.38:6832/1058376326 2024-09-21T03:36:48.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.762+0000 7f98aeffd640 1 --2- 172.21.15.38:0/117103957 >> [v2:172.21.15.38:6832/1058376326,v1:172.21.15.38:6834/1058376326] conn(0x55c451e5f030 0x55c451e5f440 unknown :-1 s=START_CONNECT pgs=15 cs=2 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-21T03:36:48.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.762+0000 7f988dffb640 1 client.6297 ms_handle_refused on v2:172.21.15.38:6832/1058376326 2024-09-21T03:36:48.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.809+0000 7f7ed44c5640 1 -- 172.21.15.38:0/116230372 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7eb8004800 msgr2=0x7f7eb8004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:48.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.809+0000 7f7ed44c5640 1 --2- 172.21.15.38:0/116230372 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7eb8004800 0x7f7eb8004c10 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc00c310 tx=0x7f7eb8005830 comp rx=0 tx=0).stop 2024-09-21T03:36:48.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.809+0000 7f7ed44c5640 1 -- 172.21.15.38:0/116230372 shutdown_connections 2024-09-21T03:36:48.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.809+0000 7f7ed44c5640 1 --2- 172.21.15.38:0/116230372 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7eb8004800 0x7f7eb8004c10 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:48.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.809+0000 7f7ed44c5640 1 -- 172.21.15.38:0/116230372 >> 172.21.15.38:0/116230372 conn(0x7f7ecc0cef40 msgr2=0x7f7ecc0cf350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:48.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.809+0000 7f7ed44c5640 1 -- 172.21.15.38:0/116230372 shutdown_connections 2024-09-21T03:36:48.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.809+0000 7f7ed44c5640 1 -- 172.21.15.38:0/116230372 wait complete. 2024-09-21T03:36:48.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.811+0000 7f7ed44c5640 1 Processor -- start 2024-09-21T03:36:48.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.811+0000 7f7ed44c5640 1 -- start start 2024-09-21T03:36:48.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.812+0000 7f7ed44c5640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7eb8004800 0x7f7ecc1571d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:48.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.812+0000 7f7ed44c5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7ecc0cc0b0 0x7f7ecc1f6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:48.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.812+0000 7f7ed44c5640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7ecc1f6950 0x7f7ecc1f2e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:48.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.812+0000 7f7ed44c5640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f7ecc0d19d0 con 0x7f7ecc0cc0b0 2024-09-21T03:36:48.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.812+0000 7f7ed44c5640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f7ecc0b39b0 con 0x7f7eb8004800 2024-09-21T03:36:48.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.812+0000 7f7ed44c5640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f7ecc0d1b50 con 0x7f7ecc1f6950 2024-09-21T03:36:48.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.813+0000 7f7ed1a60640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7ecc0cc0b0 0x7f7ecc1f6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:48.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.813+0000 7f7ed1a60640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7ecc0cc0b0 0x7f7ecc1f6410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:36802/0 (socket says 172.21.15.38:36802) 2024-09-21T03:36:48.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.813+0000 7f7ed1a60640 1 -- 172.21.15.38:0/322716624 learned_addr learned my addr 172.21.15.38:0/322716624 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:36:48.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.813+0000 7f7ed2261640 1 --2- 172.21.15.38:0/322716624 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7eb8004800 0x7f7ecc1571d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:48.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.813+0000 7f7ed2a62640 1 --2- 172.21.15.38:0/322716624 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7ecc1f6950 0x7f7ecc1f2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:48.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.813+0000 7f7ed1a60640 1 -- 172.21.15.38:0/322716624 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7ecc1f6950 msgr2=0x7f7ecc1f2e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:48.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.813+0000 7f7ed1a60640 1 --2- 172.21.15.38:0/322716624 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7ecc1f6950 0x7f7ecc1f2e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:48.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.813+0000 7f7ed1a60640 1 -- 172.21.15.38:0/322716624 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7eb8004800 msgr2=0x7f7ecc1571d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:48.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.813+0000 7f7ed1a60640 1 --2- 172.21.15.38:0/322716624 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7eb8004800 0x7f7ecc1571d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:48.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.813+0000 7f7ed1a60640 1 -- 172.21.15.38:0/322716624 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ecc1f6d60 con 0x7f7ecc0cc0b0 2024-09-21T03:36:48.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.813+0000 7f7ed2a62640 1 --2- 172.21.15.38:0/322716624 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7ecc1f6950 0x7f7ecc1f2e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:36:48.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.813+0000 7f7ed1a60640 1 --2- 172.21.15.38:0/322716624 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7ecc0cc0b0 0x7f7ecc1f6410 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc00c430 tx=0x7f7ebc001d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:48.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.813+0000 7f7ec2ffd640 1 -- 172.21.15.38:0/322716624 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ebc031040 con 0x7f7ecc0cc0b0 2024-09-21T03:36:48.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.813+0000 7f7ec2ffd640 1 -- 172.21.15.38:0/322716624 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7ebc0065e0 con 0x7f7ecc0cc0b0 2024-09-21T03:36:48.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.813+0000 7f7ed44c5640 1 -- 172.21.15.38:0/322716624 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ecc157b20 con 0x7f7ecc0cc0b0 2024-09-21T03:36:48.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.813+0000 7f7ec2ffd640 1 -- 172.21.15.38:0/322716624 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ebc003a60 con 0x7f7ecc0cc0b0 2024-09-21T03:36:48.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.813+0000 7f7ed44c5640 1 -- 172.21.15.38:0/322716624 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ecc1580e0 con 0x7f7ecc0cc0b0 2024-09-21T03:36:48.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.814+0000 7f7ed44c5640 1 -- 172.21.15.38:0/322716624 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ecc0ccd10 con 0x7f7ecc0cc0b0 2024-09-21T03:36:48.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.815+0000 7f7ec2ffd640 1 -- 172.21.15.38:0/322716624 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f7ebc03e030 con 0x7f7ecc0cc0b0 2024-09-21T03:36:48.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.816+0000 7f7ec2ffd640 1 --2- 172.21.15.38:0/322716624 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7eb0078bc0 0x7f7eb007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:48.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.816+0000 7f7ec2ffd640 1 -- 172.21.15.38:0/322716624 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6187+0+0 (secure 0 0 0) 0x7f7ebc0bb1e0 con 0x7f7ecc0cc0b0 2024-09-21T03:36:48.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.816+0000 7f7ed2261640 1 --2- 172.21.15.38:0/322716624 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7eb0078bc0 0x7f7eb007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:48.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.817+0000 7f7ed2261640 1 --2- 172.21.15.38:0/322716624 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7eb0078bc0 0x7f7eb007b0d0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f7ec8009460 tx=0x7f7ec8005f90 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:48.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.820+0000 7f7ec2ffd640 1 -- 172.21.15.38:0/322716624 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f7ecc1f6d60 con 0x7f7ecc0cc0b0 2024-09-21T03:36:48.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.963+0000 7f98aeffd640 1 -- 172.21.15.38:0/117103957 >> [v2:172.21.15.38:6832/1058376326,v1:172.21.15.38:6834/1058376326] conn(0x55c451e5f030 msgr2=0x55c451e5f440 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.38:6832/1058376326 2024-09-21T03:36:48.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.964+0000 7f988dffb640 1 client.6297 ms_handle_refused on v2:172.21.15.38:6832/1058376326 2024-09-21T03:36:48.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:48.964+0000 7f98aeffd640 1 --2- 172.21.15.38:0/117103957 >> [v2:172.21.15.38:6832/1058376326,v1:172.21.15.38:6834/1058376326] conn(0x55c451e5f030 0x55c451e5f440 unknown :-1 s=START_CONNECT pgs=15 cs=3 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-21T03:36:49.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:49.046+0000 7f7ed44c5640 1 -- 172.21.15.38:0/322716624 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7ecc152ec0 con 0x7f7ecc0cc0b0 2024-09-21T03:36:49.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:49.047+0000 7f7ec2ffd640 1 -- 172.21.15.38:0/322716624 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) ==== 94+0+5231 (secure 0 0 0) 0x7f7ebc082e60 con 0x7f7ecc0cc0b0 2024-09-21T03:36:49.049 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:36:49.049 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":53,"btime":"2024-09-21T03:36:48:487357+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5432,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/1042764291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":1042764291},{"type":"v1","addr":"172.21.15.195:6837","nonce":1042764291}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":5728,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/3035019482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6833","nonce":3035019482},{"type":"v1","addr":"172.21.15.38:6835","nonce":3035019482}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":5731,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/2536177898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":2536177898},{"type":"v1","addr":"172.21.15.195:6836","nonce":2536177898}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:36:21.442724+0000","modified":"2024-09-21T03:36:47.741335+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":68,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6204},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6204":{"gid":6204,"name":"a","rank":0,"incarnation":49,"state":"up:active","state_seq":5,"addr":"172.21.15.38:6834/1058376326","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":1058376326},{"type":"v1","addr":"172.21.15.38:6834","nonce":1058376326}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6204,"qdb_cluster":[6204]},"id":4}]} 2024-09-21T03:36:49.049 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 53 2024-09-21T03:36:49.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:49.049+0000 7f7ed44c5640 1 -- 172.21.15.38:0/322716624 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7eb0078bc0 msgr2=0x7f7eb007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:49.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:49.049+0000 7f7ed44c5640 1 --2- 172.21.15.38:0/322716624 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7eb0078bc0 0x7f7eb007b0d0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f7ec8009460 tx=0x7f7ec8005f90 comp rx=0 tx=0).stop 2024-09-21T03:36:49.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:49.049+0000 7f7ed44c5640 1 -- 172.21.15.38:0/322716624 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7ecc0cc0b0 msgr2=0x7f7ecc1f6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:49.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:49.049+0000 7f7ed44c5640 1 --2- 172.21.15.38:0/322716624 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7ecc0cc0b0 0x7f7ecc1f6410 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc00c430 tx=0x7f7ebc001d90 comp rx=0 tx=0).stop 2024-09-21T03:36:49.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:49.049+0000 7f7ed44c5640 1 -- 172.21.15.38:0/322716624 shutdown_connections 2024-09-21T03:36:49.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:49.050+0000 7f7ed44c5640 1 --2- 172.21.15.38:0/322716624 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7eb0078bc0 0x7f7eb007b0d0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:49.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:49.050+0000 7f7ed44c5640 1 --2- 172.21.15.38:0/322716624 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7ecc1f6950 0x7f7ecc1f2e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:49.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:49.050+0000 7f7ed44c5640 1 --2- 172.21.15.38:0/322716624 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7ecc0cc0b0 0x7f7ecc1f6410 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:49.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:49.050+0000 7f7ed44c5640 1 --2- 172.21.15.38:0/322716624 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7eb8004800 0x7f7ecc1571d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:49.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:49.050+0000 7f7ed44c5640 1 -- 172.21.15.38:0/322716624 >> 172.21.15.38:0/322716624 conn(0x7f7ecc0cef40 msgr2=0x7f7ecc071780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:49.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:49.050+0000 7f7ed44c5640 1 -- 172.21.15.38:0/322716624 shutdown_connections 2024-09-21T03:36:49.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:49.050+0000 7f7ed44c5640 1 -- 172.21.15.38:0/322716624 wait complete. 2024-09-21T03:36:49.063 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2024-09-21T03:36:49.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:49.365+0000 7f98aeffd640 1 -- 172.21.15.38:0/117103957 >> [v2:172.21.15.38:6832/1058376326,v1:172.21.15.38:6834/1058376326] conn(0x55c451e5f030 msgr2=0x55c451e5f440 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.38:6832/1058376326 2024-09-21T03:36:49.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:49.365+0000 7f988dffb640 1 client.6297 ms_handle_refused on v2:172.21.15.38:6832/1058376326 2024-09-21T03:36:49.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:49.365+0000 7f98aeffd640 1 --2- 172.21.15.38:0/117103957 >> [v2:172.21.15.38:6832/1058376326,v1:172.21.15.38:6834/1058376326] conn(0x55c451e5f030 0x55c451e5f440 unknown :-1 s=START_CONNECT pgs=15 cs=4 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-21T03:36:49.439 INFO:tasks.ceph.mds.a.smithi038.stderr:daemon-helper: command crashed with signal 6 2024-09-21T03:36:49.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:49.511+0000 7f0e3ffff640 1 -- 172.21.15.38:0/3736328712 <== mon.0 v2:172.21.15.38:3300/0 8 ==== log(2 entries from seq 603 at 2024-09-21T03:36:48.492870+0000) ==== 635+0+0 (secure 0 0 0) 0x7f0e5009c000 con 0x7f0e580d14f0 2024-09-21T03:36:49.513 INFO:teuthology.orchestra.run.smithi038.stdout: cluster: 2024-09-21T03:36:49.513 INFO:teuthology.orchestra.run.smithi038.stdout: id: 9d50d66f-656b-4fe2-8e3b-fe53b79c4d43 2024-09-21T03:36:49.513 INFO:teuthology.orchestra.run.smithi038.stdout: health: HEALTH_OK 2024-09-21T03:36:49.513 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:36:49.513 INFO:teuthology.orchestra.run.smithi038.stdout: services: 2024-09-21T03:36:49.513 INFO:teuthology.orchestra.run.smithi038.stdout: mon: 3 daemons, quorum a,b,c (age 3m) 2024-09-21T03:36:49.513 INFO:teuthology.orchestra.run.smithi038.stdout: mgr: x(active, since 3m), standbys: y 2024-09-21T03:36:49.513 INFO:teuthology.orchestra.run.smithi038.stdout: mds: 1/1 daemons up, 3 standby 2024-09-21T03:36:49.513 INFO:teuthology.orchestra.run.smithi038.stdout: osd: 8 osds: 8 up (since 3m), 8 in (since 3m) 2024-09-21T03:36:49.514 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:36:49.514 INFO:teuthology.orchestra.run.smithi038.stdout: data: 2024-09-21T03:36:49.514 INFO:teuthology.orchestra.run.smithi038.stdout: volumes: 1/1 healthy 2024-09-21T03:36:49.514 INFO:teuthology.orchestra.run.smithi038.stdout: pools: 3 pools, 97 pgs 2024-09-21T03:36:49.514 INFO:teuthology.orchestra.run.smithi038.stdout: objects: 29 objects, 602 KiB 2024-09-21T03:36:49.514 INFO:teuthology.orchestra.run.smithi038.stdout: usage: 267 MiB used, 720 GiB / 720 GiB avail 2024-09-21T03:36:49.514 INFO:teuthology.orchestra.run.smithi038.stdout: pgs: 97 active+clean 2024-09-21T03:36:49.514 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:36:49.514 INFO:teuthology.orchestra.run.smithi038.stdout: io: 2024-09-21T03:36:49.514 INFO:teuthology.orchestra.run.smithi038.stdout: client: 675 B/s rd, 3.5 KiB/s wr, 0 op/s rd, 3 op/s wr 2024-09-21T03:36:49.514 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:36:49.514 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:36:49.514 INFO:teuthology.orchestra.run.smithi038.stdout:2024-09-21T03:36:48.691701+0000 mds.a [ERR] MDS abort because newly corrupt dentry to be committed: [dentry #0x1/a [fffffffffffffff6,head] auth (dversion lock) v=13 ino=0x10000000000 state=1610612736 | inodepin=1 dirty=1 0x55cdef393180] 2024-09-21T03:36:50.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:50.166+0000 7f98aeffd640 1 -- 172.21.15.38:0/117103957 >> [v2:172.21.15.38:6832/1058376326,v1:172.21.15.38:6834/1058376326] conn(0x55c451e5f030 msgr2=0x55c451e5f440 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.38:6832/1058376326 2024-09-21T03:36:50.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:50.166+0000 7f988dffb640 1 client.6297 ms_handle_refused on v2:172.21.15.38:6832/1058376326 2024-09-21T03:36:50.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:50.166+0000 7f98aeffd640 1 --2- 172.21.15.38:0/117103957 >> [v2:172.21.15.38:6832/1058376326,v1:172.21.15.38:6834/1058376326] conn(0x55c451e5f030 0x55c451e5f440 unknown :-1 s=START_CONNECT pgs=15 cs=5 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-21T03:36:51.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:51.767+0000 7f98aeffd640 1 -- 172.21.15.38:0/117103957 >> [v2:172.21.15.38:6832/1058376326,v1:172.21.15.38:6834/1058376326] conn(0x55c451e5f030 msgr2=0x55c451e5f440 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.38:6832/1058376326 2024-09-21T03:36:51.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:51.767+0000 7f988dffb640 1 client.6297 ms_handle_refused on v2:172.21.15.38:6832/1058376326 2024-09-21T03:36:51.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:51.767+0000 7f98aeffd640 1 --2- 172.21.15.38:0/117103957 >> [v2:172.21.15.38:6832/1058376326,v1:172.21.15.38:6834/1058376326] conn(0x55c451e5f030 0x55c451e5f440 unknown :-1 s=START_CONNECT pgs=15 cs=6 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-09-21T03:36:53.894 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~0s 2024-09-21T03:36:54.064 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:36:54.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:54.196+0000 7f79ebc7e640 1 -- 172.21.15.38:0/2650018720 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f79e4162f00 msgr2=0x7f79e40d24d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:54.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:54.196+0000 7f79ebc7e640 1 --2- 172.21.15.38:0/2650018720 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f79e4162f00 0x7f79e40d24d0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f79e0005fd0 tx=0x7f79e002a5d0 comp rx=0 tx=0).stop 2024-09-21T03:36:54.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:54.196+0000 7f79ebc7e640 1 -- 172.21.15.38:0/2650018720 shutdown_connections 2024-09-21T03:36:54.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:54.196+0000 7f79ebc7e640 1 --2- 172.21.15.38:0/2650018720 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f79e40b2e40 0x7f79e40b3250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:54.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:54.196+0000 7f79ebc7e640 1 --2- 172.21.15.38:0/2650018720 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f79e4162f00 0x7f79e40d24d0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:54.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:54.196+0000 7f79ebc7e640 1 -- 172.21.15.38:0/2650018720 >> 172.21.15.38:0/2650018720 conn(0x7f79e40b6b20 msgr2=0x7f79e40d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:54.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:54.196+0000 7f79ebc7e640 1 -- 172.21.15.38:0/2650018720 shutdown_connections 2024-09-21T03:36:54.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:54.196+0000 7f79ebc7e640 1 -- 172.21.15.38:0/2650018720 wait complete. 2024-09-21T03:36:54.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:54.198+0000 7f79ebc7e640 1 Processor -- start 2024-09-21T03:36:54.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:54.198+0000 7f79ebc7e640 1 -- start start 2024-09-21T03:36:54.202 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:54.199+0000 7f79ebc7e640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f79e40b2e40 0x7f79e414cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:54.202 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:54.199+0000 7f79ebc7e640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f79e4162280 0x7f79e40b26c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:54.202 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:54.199+0000 7f79ebc7e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f79e40ce320 0x7f79e40ad100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:54.202 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:54.199+0000 7f79ebc7e640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f79e415e4b0 con 0x7f79e40ce320 2024-09-21T03:36:54.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:54.199+0000 7f79ebc7e640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f79e40b3a20 con 0x7f79e4162280 2024-09-21T03:36:54.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:54.199+0000 7f79ebc7e640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f79e415e630 con 0x7f79e40b2e40 2024-09-21T03:36:54.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:54.200+0000 7f79ea21b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f79e40ce320 0x7f79e40ad100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:54.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:54.200+0000 7f79ea21b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f79e40ce320 0x7f79e40ad100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:59938/0 (socket says 172.21.15.38:59938) 2024-09-21T03:36:54.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:54.200+0000 7f79ea21b640 1 -- 172.21.15.38:0/889895132 learned_addr learned my addr 172.21.15.38:0/889895132 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:36:54.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:54.200+0000 7f79e9a1a640 1 --2- 172.21.15.38:0/889895132 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f79e40b2e40 0x7f79e414cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:54.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:54.200+0000 7f79e9219640 1 --2- 172.21.15.38:0/889895132 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f79e4162280 0x7f79e40b26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:54.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:54.200+0000 7f79ea21b640 1 -- 172.21.15.38:0/889895132 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f79e40b2e40 msgr2=0x7f79e414cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:54.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:54.200+0000 7f79ea21b640 1 --2- 172.21.15.38:0/889895132 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f79e40b2e40 0x7f79e414cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:54.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:54.200+0000 7f79ea21b640 1 -- 172.21.15.38:0/889895132 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f79e4162280 msgr2=0x7f79e40b26c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:54.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:54.200+0000 7f79ea21b640 1 --2- 172.21.15.38:0/889895132 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f79e4162280 0x7f79e40b26c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:54.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:54.200+0000 7f79ea21b640 1 -- 172.21.15.38:0/889895132 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79e40ac060 con 0x7f79e40ce320 2024-09-21T03:36:54.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:54.200+0000 7f79ea21b640 1 --2- 172.21.15.38:0/889895132 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f79e40ce320 0x7f79e40ad100 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f79dc00e990 tx=0x7f79dc00ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:54.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:54.200+0000 7f79da7fc640 1 -- 172.21.15.38:0/889895132 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79dc004d20 con 0x7f79e40ce320 2024-09-21T03:36:54.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:54.200+0000 7f79da7fc640 1 -- 172.21.15.38:0/889895132 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f79dc00caa0 con 0x7f79e40ce320 2024-09-21T03:36:54.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:54.200+0000 7f79ebc7e640 1 -- 172.21.15.38:0/889895132 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79e40ac340 con 0x7f79e40ce320 2024-09-21T03:36:54.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:54.200+0000 7f79da7fc640 1 -- 172.21.15.38:0/889895132 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79dc005020 con 0x7f79e40ce320 2024-09-21T03:36:54.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:54.200+0000 7f79ebc7e640 1 -- 172.21.15.38:0/889895132 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79e40ac900 con 0x7f79e40ce320 2024-09-21T03:36:54.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:54.202+0000 7f79ebc7e640 1 -- 172.21.15.38:0/889895132 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79e4162f00 con 0x7f79e40ce320 2024-09-21T03:36:54.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:54.202+0000 7f79da7fc640 1 -- 172.21.15.38:0/889895132 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f79dc0051c0 con 0x7f79e40ce320 2024-09-21T03:36:54.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:54.203+0000 7f79da7fc640 1 --2- 172.21.15.38:0/889895132 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f79c8078b90 0x7f79c807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:54.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:54.203+0000 7f79da7fc640 1 -- 172.21.15.38:0/889895132 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6187+0+0 (secure 0 0 0) 0x7f79dc09b750 con 0x7f79e40ce320 2024-09-21T03:36:54.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:54.203+0000 7f79e9a1a640 1 --2- 172.21.15.38:0/889895132 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f79c8078b90 0x7f79c807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:54.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:54.204+0000 7f79e9a1a640 1 --2- 172.21.15.38:0/889895132 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f79c8078b90 0x7f79c807b0a0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f79e0033000 tx=0x7f79e0004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:54.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:54.208+0000 7f79da7fc640 1 -- 172.21.15.38:0/889895132 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f79dc016030 con 0x7f79e40ce320 2024-09-21T03:36:54.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:54.431+0000 7f79ebc7e640 1 -- 172.21.15.38:0/889895132 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f79e40b2c00 con 0x7f79e40ce320 2024-09-21T03:36:54.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:54.432+0000 7f79da7fc640 1 -- 172.21.15.38:0/889895132 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) ==== 94+0+5231 (secure 0 0 0) 0x7f79dc063360 con 0x7f79e40ce320 2024-09-21T03:36:54.434 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:36:54.434 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":53,"btime":"2024-09-21T03:36:48:487357+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5432,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/1042764291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":1042764291},{"type":"v1","addr":"172.21.15.195:6837","nonce":1042764291}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":5728,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/3035019482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6833","nonce":3035019482},{"type":"v1","addr":"172.21.15.38:6835","nonce":3035019482}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":5731,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/2536177898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":2536177898},{"type":"v1","addr":"172.21.15.195:6836","nonce":2536177898}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:36:21.442724+0000","modified":"2024-09-21T03:36:47.741335+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":68,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6204},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6204":{"gid":6204,"name":"a","rank":0,"incarnation":49,"state":"up:active","state_seq":5,"addr":"172.21.15.38:6834/1058376326","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":1058376326},{"type":"v1","addr":"172.21.15.38:6834","nonce":1058376326}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6204,"qdb_cluster":[6204]},"id":4}]} 2024-09-21T03:36:54.434 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 53 2024-09-21T03:36:54.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:54.434+0000 7f79ebc7e640 1 -- 172.21.15.38:0/889895132 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f79c8078b90 msgr2=0x7f79c807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:54.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:54.434+0000 7f79ebc7e640 1 --2- 172.21.15.38:0/889895132 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f79c8078b90 0x7f79c807b0a0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f79e0033000 tx=0x7f79e0004730 comp rx=0 tx=0).stop 2024-09-21T03:36:54.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:54.434+0000 7f79ebc7e640 1 -- 172.21.15.38:0/889895132 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f79e40ce320 msgr2=0x7f79e40ad100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:54.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:54.434+0000 7f79ebc7e640 1 --2- 172.21.15.38:0/889895132 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f79e40ce320 0x7f79e40ad100 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f79dc00e990 tx=0x7f79dc00ee50 comp rx=0 tx=0).stop 2024-09-21T03:36:54.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:54.434+0000 7f79ebc7e640 1 -- 172.21.15.38:0/889895132 shutdown_connections 2024-09-21T03:36:54.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:54.434+0000 7f79ebc7e640 1 --2- 172.21.15.38:0/889895132 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f79e40ce320 0x7f79e40ad100 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:54.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:54.434+0000 7f79ebc7e640 1 --2- 172.21.15.38:0/889895132 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f79e4162280 0x7f79e40b26c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:54.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:54.434+0000 7f79ebc7e640 1 --2- 172.21.15.38:0/889895132 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f79c8078b90 0x7f79c807b0a0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:54.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:54.434+0000 7f79ebc7e640 1 --2- 172.21.15.38:0/889895132 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f79e40b2e40 0x7f79e414cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:54.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:54.434+0000 7f79ebc7e640 1 -- 172.21.15.38:0/889895132 >> 172.21.15.38:0/889895132 conn(0x7f79e40b6b20 msgr2=0x7f79e406cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:54.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:54.434+0000 7f79ebc7e640 1 -- 172.21.15.38:0/889895132 shutdown_connections 2024-09-21T03:36:54.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:54.434+0000 7f79ebc7e640 1 -- 172.21.15.38:0/889895132 wait complete. 2024-09-21T03:36:54.448 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2024-09-21T03:36:54.972 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:54.970+0000 7f98aeffd640 1 -- 172.21.15.38:0/117103957 >> [v2:172.21.15.38:6832/1058376326,v1:172.21.15.38:6834/1058376326] conn(0x55c451e5f030 msgr2=0x55c451e5f440 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.38:6832/1058376326 2024-09-21T03:36:55.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:54.970+0000 7f988dffb640 1 client.6297 ms_handle_refused on v2:172.21.15.38:6832/1058376326 2024-09-21T03:36:55.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:54.970+0000 7f98aeffd640 1 --2- 172.21.15.38:0/117103957 >> [v2:172.21.15.38:6832/1058376326,v1:172.21.15.38:6834/1058376326] conn(0x55c451e5f030 0x55c451e5f440 unknown :-1 s=START_CONNECT pgs=15 cs=7 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2024-09-21T03:36:59.449 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:36:59.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:59.580+0000 7f8ea90f9640 1 -- 172.21.15.38:0/2676488993 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8ea4162280 msgr2=0x7f8ea4162690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:59.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:59.580+0000 7f8ea90f9640 1 --2- 172.21.15.38:0/2676488993 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8ea4162280 0x7f8ea4162690 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f8ea40d0c00 tx=0x7f8e9802a4d0 comp rx=0 tx=0).stop 2024-09-21T03:36:59.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:59.580+0000 7f8ea90f9640 1 -- 172.21.15.38:0/2676488993 shutdown_connections 2024-09-21T03:36:59.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:59.580+0000 7f8ea90f9640 1 --2- 172.21.15.38:0/2676488993 >> v2:172.21.15.195:3300/0 conn(0x7f8ea40b2e40 0x7f8ea40b3250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:59.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:59.580+0000 7f8ea90f9640 1 --2- 172.21.15.38:0/2676488993 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8ea4162280 0x7f8ea4162690 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:59.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:59.580+0000 7f8ea90f9640 1 -- 172.21.15.38:0/2676488993 >> 172.21.15.38:0/2676488993 conn(0x7f8ea40b6b20 msgr2=0x7f8ea40d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:59.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:59.580+0000 7f8ea90f9640 1 -- 172.21.15.38:0/2676488993 shutdown_connections 2024-09-21T03:36:59.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:59.580+0000 7f8ea90f9640 1 -- 172.21.15.38:0/2676488993 wait complete. 2024-09-21T03:36:59.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:59.582+0000 7f8ea90f9640 1 Processor -- start 2024-09-21T03:36:59.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:59.582+0000 7f8ea90f9640 1 -- start start 2024-09-21T03:36:59.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:59.583+0000 7f8ea90f9640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8ea40b2e40 0x7f8ea40ce940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:59.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:59.583+0000 7f8ea90f9640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8ea4162280 0x7f8ea415a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:59.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:59.583+0000 7f8ea90f9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8ea40aadc0 0x7f8ea415ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:59.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:59.583+0000 7f8ea90f9640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f8ea415e4b0 con 0x7f8ea40aadc0 2024-09-21T03:36:59.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:59.583+0000 7f8ea90f9640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f8ea40b39a0 con 0x7f8ea4162280 2024-09-21T03:36:59.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:59.583+0000 7f8ea90f9640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f8ea415e630 con 0x7f8ea40b2e40 2024-09-21T03:36:59.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:59.583+0000 7f8ea359e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8ea40aadc0 0x7f8ea415ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:59.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:59.583+0000 7f8ea359e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8ea40aadc0 0x7f8ea415ae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:59958/0 (socket says 172.21.15.38:59958) 2024-09-21T03:36:59.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:59.583+0000 7f8ea359e640 1 -- 172.21.15.38:0/1230819944 learned_addr learned my addr 172.21.15.38:0/1230819944 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:36:59.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:59.584+0000 7f8ea259c640 1 --2- 172.21.15.38:0/1230819944 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8ea4162280 0x7f8ea415a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:59.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:59.584+0000 7f8ea2d9d640 1 --2- 172.21.15.38:0/1230819944 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8ea40b2e40 0x7f8ea40ce940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:59.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:59.584+0000 7f8ea359e640 1 -- 172.21.15.38:0/1230819944 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8ea40b2e40 msgr2=0x7f8ea40ce940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:59.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:59.584+0000 7f8ea359e640 1 --2- 172.21.15.38:0/1230819944 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8ea40b2e40 0x7f8ea40ce940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:59.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:59.584+0000 7f8ea359e640 1 -- 172.21.15.38:0/1230819944 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8ea4162280 msgr2=0x7f8ea415a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:59.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:59.584+0000 7f8ea359e640 1 --2- 172.21.15.38:0/1230819944 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8ea4162280 0x7f8ea415a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:59.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:59.584+0000 7f8ea359e640 1 -- 172.21.15.38:0/1230819944 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ea40ab5a0 con 0x7f8ea40aadc0 2024-09-21T03:36:59.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:59.584+0000 7f8ea259c640 1 --2- 172.21.15.38:0/1230819944 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8ea4162280 0x7f8ea415a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:36:59.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:59.584+0000 7f8ea359e640 1 --2- 172.21.15.38:0/1230819944 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8ea40aadc0 0x7f8ea415ae50 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f8e9c00e990 tx=0x7f8e9c00ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:59.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:59.584+0000 7f8e8b7fe640 1 -- 172.21.15.38:0/1230819944 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e9c004d20 con 0x7f8ea40aadc0 2024-09-21T03:36:59.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:59.584+0000 7f8e8b7fe640 1 -- 172.21.15.38:0/1230819944 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8e9c00caa0 con 0x7f8ea40aadc0 2024-09-21T03:36:59.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:59.584+0000 7f8ea90f9640 1 -- 172.21.15.38:0/1230819944 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ea40ab880 con 0x7f8ea40aadc0 2024-09-21T03:36:59.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:59.584+0000 7f8e8b7fe640 1 -- 172.21.15.38:0/1230819944 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e9c005020 con 0x7f8ea40aadc0 2024-09-21T03:36:59.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:59.584+0000 7f8ea90f9640 1 -- 172.21.15.38:0/1230819944 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ea4158300 con 0x7f8ea40aadc0 2024-09-21T03:36:59.589 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:59.585+0000 7f8ea90f9640 1 -- 172.21.15.38:0/1230819944 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ea4162f00 con 0x7f8ea40aadc0 2024-09-21T03:36:59.589 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:59.586+0000 7f8e8b7fe640 1 -- 172.21.15.38:0/1230819944 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f8e9c0051c0 con 0x7f8ea40aadc0 2024-09-21T03:36:59.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:59.587+0000 7f8e8b7fe640 1 --2- 172.21.15.38:0/1230819944 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8e90078b90 0x7f8e9007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:59.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:59.587+0000 7f8e8b7fe640 1 -- 172.21.15.38:0/1230819944 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6187+0+0 (secure 0 0 0) 0x7f8e9c09b7e0 con 0x7f8ea40aadc0 2024-09-21T03:36:59.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:59.587+0000 7f8ea2d9d640 1 --2- 172.21.15.38:0/1230819944 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8e90078b90 0x7f8e9007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:59.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:59.587+0000 7f8ea2d9d640 1 --2- 172.21.15.38:0/1230819944 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8e90078b90 0x7f8e9007b0a0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f8e94006fd0 tx=0x7f8e94008040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:59.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:59.591+0000 7f8e8b7fe640 1 -- 172.21.15.38:0/1230819944 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f8e9c016030 con 0x7f8ea40aadc0 2024-09-21T03:36:59.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:59.817+0000 7f8ea90f9640 1 -- 172.21.15.38:0/1230819944 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8ea406dce0 con 0x7f8ea40aadc0 2024-09-21T03:36:59.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:59.818+0000 7f8e8b7fe640 1 -- 172.21.15.38:0/1230819944 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) ==== 94+0+5231 (secure 0 0 0) 0x7f8e9c063ac0 con 0x7f8ea40aadc0 2024-09-21T03:36:59.820 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:36:59.820 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":53,"btime":"2024-09-21T03:36:48:487357+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5432,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/1042764291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":1042764291},{"type":"v1","addr":"172.21.15.195:6837","nonce":1042764291}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":5728,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/3035019482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6833","nonce":3035019482},{"type":"v1","addr":"172.21.15.38:6835","nonce":3035019482}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":5731,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/2536177898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":2536177898},{"type":"v1","addr":"172.21.15.195:6836","nonce":2536177898}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:36:21.442724+0000","modified":"2024-09-21T03:36:47.741335+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":68,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6204},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6204":{"gid":6204,"name":"a","rank":0,"incarnation":49,"state":"up:active","state_seq":5,"addr":"172.21.15.38:6834/1058376326","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":1058376326},{"type":"v1","addr":"172.21.15.38:6834","nonce":1058376326}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6204,"qdb_cluster":[6204]},"id":4}]} 2024-09-21T03:36:59.820 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 53 2024-09-21T03:36:59.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:59.820+0000 7f8ea90f9640 1 -- 172.21.15.38:0/1230819944 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8e90078b90 msgr2=0x7f8e9007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:59.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:59.820+0000 7f8ea90f9640 1 --2- 172.21.15.38:0/1230819944 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8e90078b90 0x7f8e9007b0a0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f8e94006fd0 tx=0x7f8e94008040 comp rx=0 tx=0).stop 2024-09-21T03:36:59.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:59.820+0000 7f8ea90f9640 1 -- 172.21.15.38:0/1230819944 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8ea40aadc0 msgr2=0x7f8ea415ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:59.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:59.820+0000 7f8ea90f9640 1 --2- 172.21.15.38:0/1230819944 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8ea40aadc0 0x7f8ea415ae50 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f8e9c00e990 tx=0x7f8e9c00ee50 comp rx=0 tx=0).stop 2024-09-21T03:36:59.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:59.820+0000 7f8ea90f9640 1 -- 172.21.15.38:0/1230819944 shutdown_connections 2024-09-21T03:36:59.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:59.820+0000 7f8ea90f9640 1 --2- 172.21.15.38:0/1230819944 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8ea40aadc0 0x7f8ea415ae50 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:59.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:59.820+0000 7f8ea90f9640 1 --2- 172.21.15.38:0/1230819944 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8ea4162280 0x7f8ea415a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:59.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:59.820+0000 7f8ea90f9640 1 --2- 172.21.15.38:0/1230819944 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8e90078b90 0x7f8e9007b0a0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:59.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:59.820+0000 7f8ea90f9640 1 --2- 172.21.15.38:0/1230819944 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8ea40b2e40 0x7f8ea40ce940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:59.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:59.820+0000 7f8ea90f9640 1 -- 172.21.15.38:0/1230819944 >> 172.21.15.38:0/1230819944 conn(0x7f8ea40b6b20 msgr2=0x7f8ea406e470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:59.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:59.820+0000 7f8ea90f9640 1 -- 172.21.15.38:0/1230819944 shutdown_connections 2024-09-21T03:36:59.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:36:59.820+0000 7f8ea90f9640 1 -- 172.21.15.38:0/1230819944 wait complete. 2024-09-21T03:36:59.834 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2024-09-21T03:37:00.408 DEBUG:teuthology.orchestra.run.smithi038:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T03:37:00.411 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T03:37:00.506 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~7s 2024-09-21T03:37:01.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:01.377+0000 7f98aeffd640 1 -- 172.21.15.38:0/117103957 >> [v2:172.21.15.38:6832/1058376326,v1:172.21.15.38:6834/1058376326] conn(0x55c451e5f030 msgr2=0x55c451e5f440 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.38:6832/1058376326 2024-09-21T03:37:01.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:01.377+0000 7f988dffb640 1 client.6297 ms_handle_refused on v2:172.21.15.38:6832/1058376326 2024-09-21T03:37:01.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:01.377+0000 7f98aeffd640 1 --2- 172.21.15.38:0/117103957 >> [v2:172.21.15.38:6832/1058376326,v1:172.21.15.38:6834/1058376326] conn(0x55c451e5f030 0x55c451e5f440 unknown :-1 s=START_CONNECT pgs=15 cs=8 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 12.800000 2024-09-21T03:37:04.835 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:37:04.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:04.965+0000 7f77ae44d640 1 -- 172.21.15.38:0/3520140532 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f77a80cea90 msgr2=0x7f77a80d00d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:04.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:04.965+0000 7f77ae44d640 1 --2- 172.21.15.38:0/3520140532 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f77a80cea90 0x7f77a80d00d0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f7798009820 tx=0x7f779802a480 comp rx=0 tx=0).stop 2024-09-21T03:37:04.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:04.965+0000 7f77ae44d640 1 -- 172.21.15.38:0/3520140532 shutdown_connections 2024-09-21T03:37:04.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:04.965+0000 7f77ae44d640 1 --2- 172.21.15.38:0/3520140532 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f77a80cea90 0x7f77a80d00d0 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:04.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:04.965+0000 7f77ae44d640 1 -- 172.21.15.38:0/3520140532 >> 172.21.15.38:0/3520140532 conn(0x7f77a80d4620 msgr2=0x7f77a80d4a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:04.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:04.965+0000 7f77ae44d640 1 -- 172.21.15.38:0/3520140532 shutdown_connections 2024-09-21T03:37:04.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:04.965+0000 7f77ae44d640 1 -- 172.21.15.38:0/3520140532 wait complete. 2024-09-21T03:37:04.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:04.967+0000 7f77ae44d640 1 Processor -- start 2024-09-21T03:37:04.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:04.967+0000 7f77ae44d640 1 -- start start 2024-09-21T03:37:04.972 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:04.968+0000 7f77ae44d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f77a80cea90 0x7f77a815c310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:04.972 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:04.968+0000 7f77ae44d640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f77a80d3360 0x7f77a815a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:04.972 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:04.969+0000 7f77ae44d640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f77a8162790 0x7f77a80b8f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:04.972 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:04.969+0000 7f77ae44d640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f77a80b4390 con 0x7f77a80cea90 2024-09-21T03:37:04.972 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:04.969+0000 7f77ae44d640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f77a80b4210 con 0x7f77a80d3360 2024-09-21T03:37:04.972 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:04.969+0000 7f77ae44d640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f77a80b4510 con 0x7f77a8162790 2024-09-21T03:37:04.972 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:04.969+0000 7f77a7fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f77a80cea90 0x7f77a815c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:04.972 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:04.969+0000 7f77a7fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f77a80cea90 0x7f77a815c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42892/0 (socket says 172.21.15.38:42892) 2024-09-21T03:37:04.972 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:04.969+0000 7f77a7fff640 1 -- 172.21.15.38:0/1785062246 learned_addr learned my addr 172.21.15.38:0/1785062246 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:37:04.972 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:04.969+0000 7f77a77fe640 1 --2- 172.21.15.38:0/1785062246 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f77a80d3360 0x7f77a815a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:04.972 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:04.969+0000 7f77ac9ea640 1 --2- 172.21.15.38:0/1785062246 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f77a8162790 0x7f77a80b8f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:04.972 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:04.969+0000 7f77a7fff640 1 -- 172.21.15.38:0/1785062246 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f77a8162790 msgr2=0x7f77a80b8f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:04.972 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:04.969+0000 7f77a7fff640 1 --2- 172.21.15.38:0/1785062246 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f77a8162790 0x7f77a80b8f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:04.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:04.969+0000 7f77a7fff640 1 -- 172.21.15.38:0/1785062246 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f77a80d3360 msgr2=0x7f77a815a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:04.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:04.969+0000 7f77a7fff640 1 --2- 172.21.15.38:0/1785062246 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f77a80d3360 0x7f77a815a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:04.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:04.969+0000 7f77a7fff640 1 -- 172.21.15.38:0/1785062246 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77a81f59b0 con 0x7f77a80cea90 2024-09-21T03:37:04.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:04.969+0000 7f77ac9ea640 1 --2- 172.21.15.38:0/1785062246 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f77a8162790 0x7f77a80b8f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:37:04.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:04.969+0000 7f77a7fff640 1 --2- 172.21.15.38:0/1785062246 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f77a80cea90 0x7f77a815c310 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f7798009910 tx=0x7f7798004a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:04.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:04.969+0000 7f77a4ff9640 1 -- 172.21.15.38:0/1785062246 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f779803f070 con 0x7f77a80cea90 2024-09-21T03:37:04.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:04.969+0000 7f77a4ff9640 1 -- 172.21.15.38:0/1785062246 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7798004e50 con 0x7f77a80cea90 2024-09-21T03:37:04.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:04.969+0000 7f77ae44d640 1 -- 172.21.15.38:0/1785062246 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77a81f5c90 con 0x7f77a80cea90 2024-09-21T03:37:04.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:04.970+0000 7f77a4ff9640 1 -- 172.21.15.38:0/1785062246 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7798031950 con 0x7f77a80cea90 2024-09-21T03:37:04.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:04.970+0000 7f77ae44d640 1 -- 172.21.15.38:0/1785062246 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77a81f6250 con 0x7f77a80cea90 2024-09-21T03:37:04.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:04.970+0000 7f77ae44d640 1 -- 172.21.15.38:0/1785062246 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77a80d00d0 con 0x7f77a80cea90 2024-09-21T03:37:04.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:04.971+0000 7f77a4ff9640 1 -- 172.21.15.38:0/1785062246 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f7798042070 con 0x7f77a80cea90 2024-09-21T03:37:04.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:04.972+0000 7f77a4ff9640 1 --2- 172.21.15.38:0/1785062246 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7794078b90 0x7f779407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:04.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:04.972+0000 7f77a4ff9640 1 -- 172.21.15.38:0/1785062246 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6187+0+0 (secure 0 0 0) 0x7f77980bbe50 con 0x7f77a80cea90 2024-09-21T03:37:04.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:04.972+0000 7f77a77fe640 1 --2- 172.21.15.38:0/1785062246 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7794078b90 0x7f779407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:04.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:04.973+0000 7f77a77fe640 1 --2- 172.21.15.38:0/1785062246 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7794078b90 0x7f779407b0a0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f77a8160bc0 tx=0x7f779c009060 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:04.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:04.976+0000 7f77a4ff9640 1 -- 172.21.15.38:0/1785062246 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f7798083b80 con 0x7f77a80cea90 2024-09-21T03:37:05.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:05.206+0000 7f77ae44d640 1 -- 172.21.15.38:0/1785062246 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f77a8076180 con 0x7f77a80cea90 2024-09-21T03:37:05.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:05.207+0000 7f77a4ff9640 1 -- 172.21.15.38:0/1785062246 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) ==== 94+0+5231 (secure 0 0 0) 0x7f77980885b0 con 0x7f77a80cea90 2024-09-21T03:37:05.209 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:37:05.209 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":53,"btime":"2024-09-21T03:36:48:487357+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5432,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/1042764291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":1042764291},{"type":"v1","addr":"172.21.15.195:6837","nonce":1042764291}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":5728,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/3035019482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6833","nonce":3035019482},{"type":"v1","addr":"172.21.15.38:6835","nonce":3035019482}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":5731,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/2536177898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":2536177898},{"type":"v1","addr":"172.21.15.195:6836","nonce":2536177898}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:36:21.442724+0000","modified":"2024-09-21T03:36:47.741335+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":68,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6204},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6204":{"gid":6204,"name":"a","rank":0,"incarnation":49,"state":"up:active","state_seq":5,"addr":"172.21.15.38:6834/1058376326","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":1058376326},{"type":"v1","addr":"172.21.15.38:6834","nonce":1058376326}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6204,"qdb_cluster":[6204]},"id":4}]} 2024-09-21T03:37:05.209 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 53 2024-09-21T03:37:05.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:05.209+0000 7f77ae44d640 1 -- 172.21.15.38:0/1785062246 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7794078b90 msgr2=0x7f779407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:05.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:05.209+0000 7f77ae44d640 1 --2- 172.21.15.38:0/1785062246 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7794078b90 0x7f779407b0a0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f77a8160bc0 tx=0x7f779c009060 comp rx=0 tx=0).stop 2024-09-21T03:37:05.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:05.209+0000 7f77ae44d640 1 -- 172.21.15.38:0/1785062246 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f77a80cea90 msgr2=0x7f77a815c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:05.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:05.209+0000 7f77ae44d640 1 --2- 172.21.15.38:0/1785062246 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f77a80cea90 0x7f77a815c310 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f7798009910 tx=0x7f7798004a50 comp rx=0 tx=0).stop 2024-09-21T03:37:05.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:05.209+0000 7f77ae44d640 1 -- 172.21.15.38:0/1785062246 shutdown_connections 2024-09-21T03:37:05.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:05.209+0000 7f77ae44d640 1 --2- 172.21.15.38:0/1785062246 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7794078b90 0x7f779407b0a0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:05.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:05.209+0000 7f77ae44d640 1 --2- 172.21.15.38:0/1785062246 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f77a8162790 0x7f77a80b8f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:05.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:05.209+0000 7f77ae44d640 1 --2- 172.21.15.38:0/1785062246 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f77a80d3360 0x7f77a815a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:05.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:05.209+0000 7f77ae44d640 1 --2- 172.21.15.38:0/1785062246 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f77a80cea90 0x7f77a815c310 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:05.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:05.210+0000 7f77ae44d640 1 -- 172.21.15.38:0/1785062246 >> 172.21.15.38:0/1785062246 conn(0x7f77a80d4620 msgr2=0x7f77a80d3770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:05.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:05.210+0000 7f77ae44d640 1 -- 172.21.15.38:0/1785062246 shutdown_connections 2024-09-21T03:37:05.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:05.210+0000 7f77ae44d640 1 -- 172.21.15.38:0/1785062246 wait complete. 2024-09-21T03:37:05.223 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2024-09-21T03:37:07.118 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~13s 2024-09-21T03:37:10.224 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:37:10.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:10.357+0000 7f1e58bc2640 1 -- 172.21.15.38:0/1187535963 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1e54162790 msgr2=0x7f1e54162ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:10.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:10.357+0000 7f1e58bc2640 1 --2- 172.21.15.38:0/1187535963 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1e54162790 0x7f1e54162ba0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f1e5408d390 tx=0x7f1e4802a4d0 comp rx=0 tx=0).stop 2024-09-21T03:37:10.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:10.357+0000 7f1e58bc2640 1 -- 172.21.15.38:0/1187535963 shutdown_connections 2024-09-21T03:37:10.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:10.357+0000 7f1e58bc2640 1 --2- 172.21.15.38:0/1187535963 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1e540d3360 0x7f1e540d3770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:10.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:10.357+0000 7f1e58bc2640 1 --2- 172.21.15.38:0/1187535963 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1e54162790 0x7f1e54162ba0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:10.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:10.357+0000 7f1e58bc2640 1 --2- 172.21.15.38:0/1187535963 >> v2:172.21.15.195:3300/0 conn(0x7f1e540cea90 0x7f1e540d00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:10.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:10.357+0000 7f1e58bc2640 1 -- 172.21.15.38:0/1187535963 >> 172.21.15.38:0/1187535963 conn(0x7f1e540d4620 msgr2=0x7f1e540d4a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:10.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:10.357+0000 7f1e58bc2640 1 -- 172.21.15.38:0/1187535963 shutdown_connections 2024-09-21T03:37:10.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:10.357+0000 7f1e58bc2640 1 -- 172.21.15.38:0/1187535963 wait complete. 2024-09-21T03:37:10.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:10.359+0000 7f1e58bc2640 1 Processor -- start 2024-09-21T03:37:10.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:10.359+0000 7f1e58bc2640 1 -- start start 2024-09-21T03:37:10.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:10.360+0000 7f1e58bc2640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1e540cea90 0x7f1e541f5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:10.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:10.360+0000 7f1e58bc2640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1e540d3360 0x7f1e54156f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:10.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:10.360+0000 7f1e58bc2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1e54162790 0x7f1e540b8f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:10.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:10.360+0000 7f1e58bc2640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f1e540b3f70 con 0x7f1e54162790 2024-09-21T03:37:10.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:10.360+0000 7f1e58bc2640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f1e540b3df0 con 0x7f1e540cea90 2024-09-21T03:37:10.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:10.360+0000 7f1e58bc2640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f1e540b40f0 con 0x7f1e540d3360 2024-09-21T03:37:10.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:10.361+0000 7f1e52d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1e54162790 0x7f1e540b8f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:10.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:10.361+0000 7f1e52d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1e54162790 0x7f1e540b8f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42900/0 (socket says 172.21.15.38:42900) 2024-09-21T03:37:10.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:10.361+0000 7f1e52d9d640 1 -- 172.21.15.38:0/1881987414 learned_addr learned my addr 172.21.15.38:0/1881987414 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:37:10.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:10.361+0000 7f1e5259c640 1 --2- 172.21.15.38:0/1881987414 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1e540cea90 0x7f1e541f5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:10.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:10.361+0000 7f1e51d9b640 1 --2- 172.21.15.38:0/1881987414 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1e540d3360 0x7f1e54156f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:10.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:10.361+0000 7f1e52d9d640 1 -- 172.21.15.38:0/1881987414 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1e540d3360 msgr2=0x7f1e54156f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:10.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:10.361+0000 7f1e52d9d640 1 --2- 172.21.15.38:0/1881987414 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1e540d3360 0x7f1e54156f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:10.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:10.361+0000 7f1e52d9d640 1 -- 172.21.15.38:0/1881987414 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1e540cea90 msgr2=0x7f1e541f5180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:10.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:10.361+0000 7f1e52d9d640 1 --2- 172.21.15.38:0/1881987414 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1e540cea90 0x7f1e541f5180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:10.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:10.361+0000 7f1e52d9d640 1 -- 172.21.15.38:0/1881987414 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e541f8fa0 con 0x7f1e54162790 2024-09-21T03:37:10.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:10.361+0000 7f1e52d9d640 1 --2- 172.21.15.38:0/1881987414 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1e54162790 0x7f1e540b8f40 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f1e4c00d9e0 tx=0x7f1e4c00dea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:10.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:10.361+0000 7f1e42ffd640 1 -- 172.21.15.38:0/1881987414 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e4c017070 con 0x7f1e54162790 2024-09-21T03:37:10.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:10.361+0000 7f1e58bc2640 1 -- 172.21.15.38:0/1881987414 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e541f9210 con 0x7f1e54162790 2024-09-21T03:37:10.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:10.361+0000 7f1e42ffd640 1 -- 172.21.15.38:0/1881987414 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1e4c00be30 con 0x7f1e54162790 2024-09-21T03:37:10.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:10.361+0000 7f1e42ffd640 1 -- 172.21.15.38:0/1881987414 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e4c003bb0 con 0x7f1e54162790 2024-09-21T03:37:10.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:10.361+0000 7f1e58bc2640 1 -- 172.21.15.38:0/1881987414 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e541f97d0 con 0x7f1e54162790 2024-09-21T03:37:10.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:10.362+0000 7f1e58bc2640 1 -- 172.21.15.38:0/1881987414 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e540b4610 con 0x7f1e54162790 2024-09-21T03:37:10.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:10.363+0000 7f1e42ffd640 1 -- 172.21.15.38:0/1881987414 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f1e4c010460 con 0x7f1e54162790 2024-09-21T03:37:10.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:10.364+0000 7f1e42ffd640 1 --2- 172.21.15.38:0/1881987414 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1e34078b90 0x7f1e3407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:10.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:10.364+0000 7f1e42ffd640 1 -- 172.21.15.38:0/1881987414 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6187+0+0 (secure 0 0 0) 0x7f1e4c09b360 con 0x7f1e54162790 2024-09-21T03:37:10.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:10.364+0000 7f1e5259c640 1 --2- 172.21.15.38:0/1881987414 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1e34078b90 0x7f1e3407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:10.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:10.365+0000 7f1e5259c640 1 --2- 172.21.15.38:0/1881987414 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1e34078b90 0x7f1e3407b0a0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f1e44005fd0 tx=0x7f1e4400b260 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:10.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:10.369+0000 7f1e42ffd640 1 -- 172.21.15.38:0/1881987414 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f1e4c062ef0 con 0x7f1e54162790 2024-09-21T03:37:10.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:10.595+0000 7f1e58bc2640 1 -- 172.21.15.38:0/1881987414 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1e54162cd0 con 0x7f1e54162790 2024-09-21T03:37:10.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:10.595+0000 7f1e42ffd640 1 -- 172.21.15.38:0/1881987414 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) ==== 94+0+5279 (secure 0 0 0) 0x7f1e4c067920 con 0x7f1e54162790 2024-09-21T03:37:10.597 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:37:10.598 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":54,"btime":"2024-09-21T03:37:05:490671+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5432,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/1042764291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":1042764291},{"type":"v1","addr":"172.21.15.195:6837","nonce":1042764291}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":5728,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/3035019482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6833","nonce":3035019482},{"type":"v1","addr":"172.21.15.38:6835","nonce":3035019482}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":5731,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/2536177898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":2536177898},{"type":"v1","addr":"172.21.15.195:6836","nonce":2536177898}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:36:21.442724+0000","modified":"2024-09-21T03:37:05.490625+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":68,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6204},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6204":{"gid":6204,"name":"a","rank":0,"incarnation":49,"state":"up:active","state_seq":5,"addr":"172.21.15.38:6834/1058376326","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":1058376326},{"type":"v1","addr":"172.21.15.38:6834","nonce":1058376326}]},"join_fscid":-1,"laggy_since":"2024-09-21T03:37:05.490624+0000","export_targets":[],"features":4540719139924082687,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6204,"qdb_cluster":[6204]},"id":4}]} 2024-09-21T03:37:10.598 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 54 2024-09-21T03:37:10.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:10.598+0000 7f1e58bc2640 1 -- 172.21.15.38:0/1881987414 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1e34078b90 msgr2=0x7f1e3407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:10.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:10.598+0000 7f1e58bc2640 1 --2- 172.21.15.38:0/1881987414 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1e34078b90 0x7f1e3407b0a0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f1e44005fd0 tx=0x7f1e4400b260 comp rx=0 tx=0).stop 2024-09-21T03:37:10.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:10.598+0000 7f1e58bc2640 1 -- 172.21.15.38:0/1881987414 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1e54162790 msgr2=0x7f1e540b8f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:10.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:10.598+0000 7f1e58bc2640 1 --2- 172.21.15.38:0/1881987414 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1e54162790 0x7f1e540b8f40 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f1e4c00d9e0 tx=0x7f1e4c00dea0 comp rx=0 tx=0).stop 2024-09-21T03:37:10.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:10.598+0000 7f1e58bc2640 1 -- 172.21.15.38:0/1881987414 shutdown_connections 2024-09-21T03:37:10.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:10.598+0000 7f1e58bc2640 1 --2- 172.21.15.38:0/1881987414 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1e54162790 0x7f1e540b8f40 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:10.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:10.598+0000 7f1e58bc2640 1 --2- 172.21.15.38:0/1881987414 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1e34078b90 0x7f1e3407b0a0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:10.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:10.598+0000 7f1e58bc2640 1 --2- 172.21.15.38:0/1881987414 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1e540d3360 0x7f1e54156f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:10.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:10.598+0000 7f1e58bc2640 1 --2- 172.21.15.38:0/1881987414 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1e540cea90 0x7f1e541f5180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:10.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:10.598+0000 7f1e58bc2640 1 -- 172.21.15.38:0/1881987414 >> 172.21.15.38:0/1881987414 conn(0x7f1e540d4620 msgr2=0x7f1e540d4a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:10.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:10.598+0000 7f1e58bc2640 1 -- 172.21.15.38:0/1881987414 shutdown_connections 2024-09-21T03:37:10.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:10.598+0000 7f1e58bc2640 1 -- 172.21.15.38:0/1881987414 wait complete. 2024-09-21T03:37:10.612 DEBUG:tasks.ceph_test_case:wait_until_true: success in 20s and 0 retries 2024-09-21T03:37:10.713 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_inject_journal_corrupt_dentry_first 2024-09-21T03:37:10.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:10.831+0000 7fe6edfc6640 1 -- 172.21.15.38:0/919836862 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe6d0005350 msgr2=0x7fe6d000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:10.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:10.831+0000 7fe6edfc6640 1 --2- 172.21.15.38:0/919836862 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe6d0005350 0x7fe6d000b9c0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fe6e8073ca0 tx=0x7fe6d0005830 comp rx=0 tx=0).stop 2024-09-21T03:37:10.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:10.831+0000 7fe6edfc6640 1 -- 172.21.15.38:0/919836862 shutdown_connections 2024-09-21T03:37:10.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:10.831+0000 7fe6edfc6640 1 --2- 172.21.15.38:0/919836862 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe6d0005350 0x7fe6d000b9c0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:10.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:10.831+0000 7fe6edfc6640 1 -- 172.21.15.38:0/919836862 >> 172.21.15.38:0/919836862 conn(0x7fe6e80cdca0 msgr2=0x7fe6e80ce0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:10.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:10.831+0000 7fe6edfc6640 1 -- 172.21.15.38:0/919836862 shutdown_connections 2024-09-21T03:37:10.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:10.831+0000 7fe6edfc6640 1 -- 172.21.15.38:0/919836862 wait complete. 2024-09-21T03:37:10.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:10.833+0000 7fe6edfc6640 1 Processor -- start 2024-09-21T03:37:10.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:10.833+0000 7fe6edfc6640 1 -- start start 2024-09-21T03:37:10.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:10.834+0000 7fe6edfc6640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe6d0005350 0x7fe6e80ca200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:10.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:10.834+0000 7fe6edfc6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe6e81f5730 0x7fe6e81f5b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:10.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:10.834+0000 7fe6edfc6640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe6e8156140 0x7fe6e8156550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:10.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:10.834+0000 7fe6edfc6640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fe6e815bc00 con 0x7fe6e81f5730 2024-09-21T03:37:10.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:10.834+0000 7fe6edfc6640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fe6e815ba80 con 0x7fe6e8156140 2024-09-21T03:37:10.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:10.834+0000 7fe6edfc6640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fe6e815bd80 con 0x7fe6d0005350 2024-09-21T03:37:10.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:10.834+0000 7fe6e6ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe6e81f5730 0x7fe6e81f5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:10.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:10.834+0000 7fe6e6ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe6e81f5730 0x7fe6e81f5b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42928/0 (socket says 172.21.15.38:42928) 2024-09-21T03:37:10.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:10.834+0000 7fe6e6ffd640 1 -- 172.21.15.38:0/1095101481 learned_addr learned my addr 172.21.15.38:0/1095101481 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:37:10.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:10.835+0000 7fe6e7fff640 1 --2- 172.21.15.38:0/1095101481 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe6e8156140 0x7fe6e8156550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:10.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:10.835+0000 7fe6e77fe640 1 --2- 172.21.15.38:0/1095101481 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe6d0005350 0x7fe6e80ca200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:10.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:10.835+0000 7fe6e6ffd640 1 -- 172.21.15.38:0/1095101481 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe6d0005350 msgr2=0x7fe6e80ca200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:10.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:10.835+0000 7fe6e6ffd640 1 --2- 172.21.15.38:0/1095101481 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe6d0005350 0x7fe6e80ca200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:10.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:10.835+0000 7fe6e6ffd640 1 -- 172.21.15.38:0/1095101481 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe6e8156140 msgr2=0x7fe6e8156550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:10.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:10.835+0000 7fe6e6ffd640 1 --2- 172.21.15.38:0/1095101481 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe6e8156140 0x7fe6e8156550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:10.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:10.835+0000 7fe6e6ffd640 1 -- 172.21.15.38:0/1095101481 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6e81f99e0 con 0x7fe6e81f5730 2024-09-21T03:37:10.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:10.835+0000 7fe6e7fff640 1 --2- 172.21.15.38:0/1095101481 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe6e8156140 0x7fe6e8156550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:37:10.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:10.835+0000 7fe6e77fe640 1 --2- 172.21.15.38:0/1095101481 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe6d0005350 0x7fe6e80ca200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:37:10.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:10.835+0000 7fe6e6ffd640 1 --2- 172.21.15.38:0/1095101481 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe6e81f5730 0x7fe6e81f5b40 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fe6dc006fd0 tx=0x7fe6dc00d830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:10.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:10.835+0000 7fe6cffff640 1 -- 172.21.15.38:0/1095101481 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6dc014070 con 0x7fe6e81f5730 2024-09-21T03:37:10.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:10.835+0000 7fe6cffff640 1 -- 172.21.15.38:0/1095101481 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe6dc00f970 con 0x7fe6e81f5730 2024-09-21T03:37:10.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:10.835+0000 7fe6cffff640 1 -- 172.21.15.38:0/1095101481 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6dc00fdc0 con 0x7fe6e81f5730 2024-09-21T03:37:10.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:10.835+0000 7fe6edfc6640 1 -- 172.21.15.38:0/1095101481 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6e8156a90 con 0x7fe6e81f5730 2024-09-21T03:37:10.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:10.835+0000 7fe6edfc6640 1 -- 172.21.15.38:0/1095101481 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6e814cdf0 con 0x7fe6e81f5730 2024-09-21T03:37:10.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:10.836+0000 7fe6edfc6640 1 -- 172.21.15.38:0/1095101481 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6e81f6210 con 0x7fe6e81f5730 2024-09-21T03:37:10.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:10.837+0000 7fe6cffff640 1 -- 172.21.15.38:0/1095101481 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fe6dc017070 con 0x7fe6e81f5730 2024-09-21T03:37:10.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:10.837+0000 7fe6cffff640 1 --2- 172.21.15.38:0/1095101481 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fe6d4078bf0 0x7fe6d407b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:10.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:10.837+0000 7fe6cffff640 1 -- 172.21.15.38:0/1095101481 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6187+0+0 (secure 0 0 0) 0x7fe6dc09ae80 con 0x7fe6e81f5730 2024-09-21T03:37:10.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:10.838+0000 7fe6e77fe640 1 --2- 172.21.15.38:0/1095101481 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fe6d4078bf0 0x7fe6d407b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:10.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:10.838+0000 7fe6e77fe640 1 --2- 172.21.15.38:0/1095101481 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fe6d4078bf0 0x7fe6d407b100 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fe6d8001860 tx=0x7fe6d8000ed0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:10.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:10.842+0000 7fe6cffff640 1 -- 172.21.15.38:0/1095101481 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fe6e8156a90 con 0x7fe6e81f5730 2024-09-21T03:37:11.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:11.032+0000 7fe6edfc6640 1 -- 172.21.15.38:0/1095101481 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_inject_journal_corrupt_dentry_first"} v 0) -- 0x7fe6e8070610 con 0x7fe6e81f5730 2024-09-21T03:37:11.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:11.036+0000 7fe6cffff640 1 -- 172.21.15.38:0/1095101481 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_inject_journal_corrupt_dentry_first"}]=0 v41) ==== 122+0+0 (secure 0 0 0) 0x7fe6e8070610 con 0x7fe6e81f5730 2024-09-21T03:37:11.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:11.038+0000 7fe6edfc6640 1 -- 172.21.15.38:0/1095101481 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fe6d4078bf0 msgr2=0x7fe6d407b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:11.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:11.038+0000 7fe6edfc6640 1 --2- 172.21.15.38:0/1095101481 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fe6d4078bf0 0x7fe6d407b100 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fe6d8001860 tx=0x7fe6d8000ed0 comp rx=0 tx=0).stop 2024-09-21T03:37:11.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:11.038+0000 7fe6edfc6640 1 -- 172.21.15.38:0/1095101481 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe6e81f5730 msgr2=0x7fe6e81f5b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:11.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:11.038+0000 7fe6edfc6640 1 --2- 172.21.15.38:0/1095101481 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe6e81f5730 0x7fe6e81f5b40 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fe6dc006fd0 tx=0x7fe6dc00d830 comp rx=0 tx=0).stop 2024-09-21T03:37:11.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:11.039+0000 7fe6edfc6640 1 -- 172.21.15.38:0/1095101481 shutdown_connections 2024-09-21T03:37:11.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:11.039+0000 7fe6edfc6640 1 --2- 172.21.15.38:0/1095101481 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe6e8156140 0x7fe6e8156550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:11.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:11.039+0000 7fe6edfc6640 1 --2- 172.21.15.38:0/1095101481 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe6e81f5730 0x7fe6e81f5b40 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:11.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:11.039+0000 7fe6edfc6640 1 --2- 172.21.15.38:0/1095101481 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fe6d4078bf0 0x7fe6d407b100 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:11.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:11.039+0000 7fe6edfc6640 1 --2- 172.21.15.38:0/1095101481 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe6d0005350 0x7fe6e80ca200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:11.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:11.039+0000 7fe6edfc6640 1 -- 172.21.15.38:0/1095101481 >> 172.21.15.38:0/1095101481 conn(0x7fe6e80cdca0 msgr2=0x7fe6e80caae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:11.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:11.039+0000 7fe6edfc6640 1 -- 172.21.15.38:0/1095101481 shutdown_connections 2024-09-21T03:37:11.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:11.039+0000 7fe6edfc6640 1 -- 172.21.15.38:0/1095101481 wait complete. 2024-09-21T03:37:11.052 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds freeze 4:0 false 2024-09-21T03:37:11.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:11.170+0000 7f8a20af2640 1 -- 172.21.15.38:0/2798436047 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8a08004d40 msgr2=0x7f8a08005170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:11.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:11.170+0000 7f8a20af2640 1 --2- 172.21.15.38:0/2798436047 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8a08004d40 0x7f8a08005170 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c001710 tx=0x7f8a08005830 comp rx=0 tx=0).stop 2024-09-21T03:37:11.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:11.170+0000 7f8a1ad9d640 1 -- 172.21.15.38:0/2798436047 reap_dead start 2024-09-21T03:37:11.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:11.170+0000 7f8a20af2640 1 -- 172.21.15.38:0/2798436047 shutdown_connections 2024-09-21T03:37:11.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:11.170+0000 7f8a20af2640 1 -- 172.21.15.38:0/2798436047 >> 172.21.15.38:0/2798436047 conn(0x7f8a1c09d830 msgr2=0x7f8a1c0b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:11.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:11.170+0000 7f8a20af2640 1 -- 172.21.15.38:0/2798436047 shutdown_connections 2024-09-21T03:37:11.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:11.170+0000 7f8a20af2640 1 -- 172.21.15.38:0/2798436047 wait complete. 2024-09-21T03:37:11.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:11.172+0000 7f8a20af2640 1 Processor -- start 2024-09-21T03:37:11.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:11.172+0000 7f8a20af2640 1 -- start start 2024-09-21T03:37:11.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:11.173+0000 7f8a20af2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8a08004d40 0x7f8a1c14f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:11.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:11.173+0000 7f8a20af2640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8a1c157cd0 0x7f8a1c0ccea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:11.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:11.173+0000 7f8a20af2640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8a1c157780 0x7f8a1c0cd3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:11.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:11.173+0000 7f8a20af2640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f8a1c0d0500 con 0x7f8a08004d40 2024-09-21T03:37:11.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:11.173+0000 7f8a20af2640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f8a1c0d0380 con 0x7f8a1c157780 2024-09-21T03:37:11.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:11.173+0000 7f8a20af2640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f8a1c0ba0d0 con 0x7f8a1c157cd0 2024-09-21T03:37:11.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:11.173+0000 7f8a1a59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8a08004d40 0x7f8a1c14f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:11.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:11.173+0000 7f8a1a59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8a08004d40 0x7f8a1c14f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42940/0 (socket says 172.21.15.38:42940) 2024-09-21T03:37:11.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:11.173+0000 7f8a1a59c640 1 -- 172.21.15.38:0/1483406143 learned_addr learned my addr 172.21.15.38:0/1483406143 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:37:11.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:11.174+0000 7f8a1ad9d640 1 --2- 172.21.15.38:0/1483406143 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8a1c157780 0x7f8a1c0cd3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:11.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:11.174+0000 7f8a19d9b640 1 --2- 172.21.15.38:0/1483406143 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8a1c157cd0 0x7f8a1c0ccea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:11.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:11.174+0000 7f8a1a59c640 1 -- 172.21.15.38:0/1483406143 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8a1c157cd0 msgr2=0x7f8a1c0ccea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:11.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:11.174+0000 7f8a1a59c640 1 --2- 172.21.15.38:0/1483406143 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8a1c157cd0 0x7f8a1c0ccea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:11.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:11.174+0000 7f8a1a59c640 1 -- 172.21.15.38:0/1483406143 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8a1c157780 msgr2=0x7f8a1c0cd3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:11.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:11.174+0000 7f8a1a59c640 1 --2- 172.21.15.38:0/1483406143 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8a1c157780 0x7f8a1c0cd3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:11.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:11.174+0000 7f8a1a59c640 1 -- 172.21.15.38:0/1483406143 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a1c0cad80 con 0x7f8a08004d40 2024-09-21T03:37:11.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:11.174+0000 7f8a1ad9d640 1 --2- 172.21.15.38:0/1483406143 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8a1c157780 0x7f8a1c0cd3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:37:11.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:11.174+0000 7f8a1a59c640 1 --2- 172.21.15.38:0/1483406143 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8a08004d40 0x7f8a1c14f740 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f8a14011d90 tx=0x7f8a1400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:11.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:11.174+0000 7f8a12ffd640 1 -- 172.21.15.38:0/1483406143 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a14003d80 con 0x7f8a08004d40 2024-09-21T03:37:11.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:11.174+0000 7f8a12ffd640 1 -- 172.21.15.38:0/1483406143 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8a14004b20 con 0x7f8a08004d40 2024-09-21T03:37:11.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:11.174+0000 7f8a20af2640 1 -- 172.21.15.38:0/1483406143 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a1c0cdb00 con 0x7f8a08004d40 2024-09-21T03:37:11.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:11.174+0000 7f8a12ffd640 1 -- 172.21.15.38:0/1483406143 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a14004050 con 0x7f8a08004d40 2024-09-21T03:37:11.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:11.174+0000 7f8a20af2640 1 -- 172.21.15.38:0/1483406143 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a1c0cb8b0 con 0x7f8a08004d40 2024-09-21T03:37:11.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:11.175+0000 7f8a20af2640 1 -- 172.21.15.38:0/1483406143 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a1c0cd920 con 0x7f8a08004d40 2024-09-21T03:37:11.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:11.176+0000 7f8a12ffd640 1 -- 172.21.15.38:0/1483406143 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f8a140041f0 con 0x7f8a08004d40 2024-09-21T03:37:11.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:11.177+0000 7f8a12ffd640 1 --2- 172.21.15.38:0/1483406143 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f89fc078bc0 0x7f89fc07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:11.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:11.177+0000 7f8a12ffd640 1 -- 172.21.15.38:0/1483406143 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6187+0+0 (secure 0 0 0) 0x7f8a1409c0c0 con 0x7f8a08004d40 2024-09-21T03:37:11.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:11.177+0000 7f8a19d9b640 1 --2- 172.21.15.38:0/1483406143 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f89fc078bc0 0x7f89fc07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:11.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:11.177+0000 7f8a19d9b640 1 --2- 172.21.15.38:0/1483406143 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f89fc078bc0 0x7f89fc07b0d0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c159060 tx=0x7f8a040011b0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:11.183 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:11.181+0000 7f8a12ffd640 1 -- 172.21.15.38:0/1483406143 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f8a1c0cd920 con 0x7f8a08004d40 2024-09-21T03:37:11.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:11.442+0000 7f8a20af2640 1 -- 172.21.15.38:0/1483406143 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "mds freeze", "role_or_gid": "4:0", "val": "false"} v 0) -- 0x7f8a1c0d0be0 con 0x7f8a08004d40 2024-09-21T03:37:12.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:12.041+0000 7f8a12ffd640 1 -- 172.21.15.38:0/1483406143 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "mds freeze", "role_or_gid": "4:0", "val": "false"}]=0 unfreezing mds.6204 v55) ==== 115+0+0 (secure 0 0 0) 0x7f8a14064330 con 0x7f8a08004d40 2024-09-21T03:37:12.043 INFO:teuthology.orchestra.run.smithi038.stderr:unfreezing mds.6204 2024-09-21T03:37:12.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:12.044+0000 7f8a20af2640 1 -- 172.21.15.38:0/1483406143 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f89fc078bc0 msgr2=0x7f89fc07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:12.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:12.044+0000 7f8a20af2640 1 --2- 172.21.15.38:0/1483406143 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f89fc078bc0 0x7f89fc07b0d0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c159060 tx=0x7f8a040011b0 comp rx=0 tx=0).stop 2024-09-21T03:37:12.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:12.044+0000 7f8a20af2640 1 -- 172.21.15.38:0/1483406143 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8a08004d40 msgr2=0x7f8a1c14f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:12.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:12.044+0000 7f8a20af2640 1 --2- 172.21.15.38:0/1483406143 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8a08004d40 0x7f8a1c14f740 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f8a14011d90 tx=0x7f8a1400f790 comp rx=0 tx=0).stop 2024-09-21T03:37:12.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:12.044+0000 7f8a20af2640 1 -- 172.21.15.38:0/1483406143 shutdown_connections 2024-09-21T03:37:12.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:12.044+0000 7f8a20af2640 1 --2- 172.21.15.38:0/1483406143 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8a1c157780 0x7f8a1c0cd3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:12.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:12.044+0000 7f8a20af2640 1 --2- 172.21.15.38:0/1483406143 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f89fc078bc0 0x7f89fc07b0d0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:12.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:12.044+0000 7f8a20af2640 1 --2- 172.21.15.38:0/1483406143 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8a1c157cd0 0x7f8a1c0ccea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:12.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:12.044+0000 7f8a20af2640 1 --2- 172.21.15.38:0/1483406143 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8a08004d40 0x7f8a1c14f740 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:12.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:12.044+0000 7f8a20af2640 1 -- 172.21.15.38:0/1483406143 >> 172.21.15.38:0/1483406143 conn(0x7f8a1c09d830 msgr2=0x7f8a1c0b48f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:12.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:12.044+0000 7f8a20af2640 1 -- 172.21.15.38:0/1483406143 shutdown_connections 2024-09-21T03:37:12.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:12.044+0000 7f8a20af2640 1 -- 172.21.15.38:0/1483406143 wait complete. 2024-09-21T03:37:12.061 DEBUG:teuthology.orchestra.run.smithi038:> sudo sysctl -n kernel.core_pattern 2024-09-21T03:37:12.087 INFO:teuthology.orchestra.run.smithi038.stdout:/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T03:37:12.089 INFO:tasks.cephfs.cephfs_test_case:Clearing core from directory: /home/ubuntu/cephtest/archive/coredump 2024-09-21T03:37:12.089 DEBUG:teuthology.orchestra.run.smithi038:> cd /home/ubuntu/cephtest/archive/coredump && sudo ls | sudo xargs file 2024-09-21T03:37:12.160 INFO:teuthology.orchestra.run.smithi038.stdout:1726889808.55880.core: ELF 64-bit LSB core file, x86-64, version 1 (SYSV), SVR4-style, from 'ceph-mds -f --cluster ceph -i a', real uid: 0, effective uid: 0, real gid: 0, effective gid: 0, execfn: '/proc/self/exe', platform: 'x86_64' 2024-09-21T03:37:12.162 INFO:tasks.cephfs.cephfs_test_case:Enumerated cores: ['1726889808.55880.core'] 2024-09-21T03:37:12.162 INFO:tasks.cephfs.cephfs_test_case:Found core file 1726889808.55880.core, deleting it 2024-09-21T03:37:12.162 DEBUG:teuthology.orchestra.run.smithi038:> cd /home/ubuntu/cephtest/archive/coredump && sudo rm -f 1726889808.55880.core 2024-09-21T03:37:12.238 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-21T03:37:12.239 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-21T03:37:12.239 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-21T03:37:12.239 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T03:37:12.239 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T03:37:12.239 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/daemon/state.py", line 146, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi038 with status 1: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a' 2024-09-21T03:37:12.240 INFO:tasks.ceph.mds.a:Stopped 2024-09-21T03:37:12.240 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-21T03:37:12.281 INFO:tasks.ceph.mds.a:Started 2024-09-21T03:37:12.281 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:37:12.370 INFO:tasks.ceph.mds.a.smithi038.stdout:starting mds.a at 2024-09-21T03:37:12.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:12.399+0000 7f6b4c207640 1 -- 172.21.15.38:0/1402102911 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6b440b2e60 msgr2=0x7f6b440b3270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:12.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:12.399+0000 7f6b4c207640 1 --2- 172.21.15.38:0/1402102911 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6b440b2e60 0x7f6b440b3270 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f6b3c005fd0 tx=0x7f6b3c02aa90 comp rx=0 tx=0).stop 2024-09-21T03:37:12.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:12.399+0000 7f6b4c207640 1 -- 172.21.15.38:0/1402102911 shutdown_connections 2024-09-21T03:37:12.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:12.399+0000 7f6b4c207640 1 --2- 172.21.15.38:0/1402102911 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6b440b2e60 0x7f6b440b3270 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:12.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:12.399+0000 7f6b4c207640 1 --2- 172.21.15.38:0/1402102911 >> v2:172.21.15.195:3300/0 conn(0x7f6b44162590 0x7f6b441629a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:12.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:12.399+0000 7f6b4c207640 1 -- 172.21.15.38:0/1402102911 >> 172.21.15.38:0/1402102911 conn(0x7f6b440d4440 msgr2=0x7f6b440d4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:12.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:12.399+0000 7f6b4c207640 1 -- 172.21.15.38:0/1402102911 shutdown_connections 2024-09-21T03:37:12.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:12.399+0000 7f6b4c207640 1 -- 172.21.15.38:0/1402102911 wait complete. 2024-09-21T03:37:12.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:12.401+0000 7f6b4c207640 1 Processor -- start 2024-09-21T03:37:12.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:12.401+0000 7f6b4c207640 1 -- start start 2024-09-21T03:37:12.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:12.402+0000 7f6b4c207640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6b440b2e60 0x7f6b441f8d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:12.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:12.402+0000 7f6b4c207640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6b440cfe00 0x7f6b441f9250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:12.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:12.402+0000 7f6b4c207640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6b44162590 0x7f6b441f9790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:12.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:12.402+0000 7f6b4c207640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f6b440b3930 con 0x7f6b44162590 2024-09-21T03:37:12.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:12.402+0000 7f6b4c207640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f6b440b37b0 con 0x7f6b440b2e60 2024-09-21T03:37:12.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:12.402+0000 7f6b4c207640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f6b440b3ab0 con 0x7f6b440cfe00 2024-09-21T03:37:12.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:12.402+0000 7f6b4a7a4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6b44162590 0x7f6b441f9790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:12.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:12.402+0000 7f6b4a7a4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6b44162590 0x7f6b441f9790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41006/0 (socket says 172.21.15.38:41006) 2024-09-21T03:37:12.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:12.402+0000 7f6b4a7a4640 1 -- 172.21.15.38:0/2996805048 learned_addr learned my addr 172.21.15.38:0/2996805048 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:37:12.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:12.403+0000 7f6b49fa3640 1 --2- 172.21.15.38:0/2996805048 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6b440b2e60 0x7f6b441f8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:12.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:12.403+0000 7f6b497a2640 1 --2- 172.21.15.38:0/2996805048 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6b440cfe00 0x7f6b441f9250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:12.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:12.403+0000 7f6b4a7a4640 1 -- 172.21.15.38:0/2996805048 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6b440cfe00 msgr2=0x7f6b441f9250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:12.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:12.403+0000 7f6b4a7a4640 1 --2- 172.21.15.38:0/2996805048 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6b440cfe00 0x7f6b441f9250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:12.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:12.403+0000 7f6b4a7a4640 1 -- 172.21.15.38:0/2996805048 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6b440b2e60 msgr2=0x7f6b441f8d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:12.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:12.403+0000 7f6b4a7a4640 1 --2- 172.21.15.38:0/2996805048 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6b440b2e60 0x7f6b441f8d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:12.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:12.403+0000 7f6b4a7a4640 1 -- 172.21.15.38:0/2996805048 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b44159770 con 0x7f6b44162590 2024-09-21T03:37:12.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:12.403+0000 7f6b4a7a4640 1 --2- 172.21.15.38:0/2996805048 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6b44162590 0x7f6b441f9790 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f6b3c009d90 tx=0x7f6b3c003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:12.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:12.403+0000 7f6b3a7fc640 1 -- 172.21.15.38:0/2996805048 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b3c044070 con 0x7f6b44162590 2024-09-21T03:37:12.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:12.403+0000 7f6b3a7fc640 1 -- 172.21.15.38:0/2996805048 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6b3c004040 con 0x7f6b44162590 2024-09-21T03:37:12.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:12.403+0000 7f6b4c207640 1 -- 172.21.15.38:0/2996805048 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b44159a50 con 0x7f6b44162590 2024-09-21T03:37:12.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:12.403+0000 7f6b3a7fc640 1 -- 172.21.15.38:0/2996805048 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b3c004490 con 0x7f6b44162590 2024-09-21T03:37:12.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:12.403+0000 7f6b4c207640 1 -- 172.21.15.38:0/2996805048 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b44150ef0 con 0x7f6b44162590 2024-09-21T03:37:12.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:12.404+0000 7f6b4c207640 1 -- 172.21.15.38:0/2996805048 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b44144090 con 0x7f6b44162590 2024-09-21T03:37:12.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:12.405+0000 7f6b3a7fc640 1 -- 172.21.15.38:0/2996805048 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f6b3c047070 con 0x7f6b44162590 2024-09-21T03:37:12.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:12.406+0000 7f6b3a7fc640 1 --2- 172.21.15.38:0/2996805048 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6b28078b90 0x7f6b2807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:12.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:12.406+0000 7f6b3a7fc640 1 -- 172.21.15.38:0/2996805048 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6273+0+0 (secure 0 0 0) 0x7f6b3c0bbd30 con 0x7f6b44162590 2024-09-21T03:37:12.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:12.406+0000 7f6b49fa3640 1 --2- 172.21.15.38:0/2996805048 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6b28078b90 0x7f6b2807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:12.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:12.407+0000 7f6b49fa3640 1 --2- 172.21.15.38:0/2996805048 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6b28078b90 0x7f6b2807b0a0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f6b400017e0 tx=0x7f6b40009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:12.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:12.410+0000 7f6b3a7fc640 1 -- 172.21.15.38:0/2996805048 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f6b3c032030 con 0x7f6b44162590 2024-09-21T03:37:12.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:12.636+0000 7f6b4c207640 1 -- 172.21.15.38:0/2996805048 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6b440b3270 con 0x7f6b44162590 2024-09-21T03:37:12.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:12.637+0000 7f6b3a7fc640 1 -- 172.21.15.38:0/2996805048 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) ==== 94+0+4399 (secure 0 0 0) 0x7f6b3c083970 con 0x7f6b44162590 2024-09-21T03:37:12.639 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:37:12.639 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":56,"btime":"2024-09-21T03:37:12:045023+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5432,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/1042764291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":1042764291},{"type":"v1","addr":"172.21.15.195:6837","nonce":1042764291}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":5728,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/3035019482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6833","nonce":3035019482},{"type":"v1","addr":"172.21.15.38:6835","nonce":3035019482}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:36:21.442724+0000","modified":"2024-09-21T03:37:12.045018+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":69,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5731},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5731":{"gid":5731,"name":"b","rank":0,"incarnation":56,"state":"up:replay","state_seq":1,"addr":"172.21.15.195:6836/2536177898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":2536177898},{"type":"v1","addr":"172.21.15.195:6836","nonce":2536177898}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":4}]} 2024-09-21T03:37:12.639 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 56 2024-09-21T03:37:12.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:12.639+0000 7f6b4c207640 1 -- 172.21.15.38:0/2996805048 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6b28078b90 msgr2=0x7f6b2807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:12.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:12.639+0000 7f6b4c207640 1 --2- 172.21.15.38:0/2996805048 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6b28078b90 0x7f6b2807b0a0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f6b400017e0 tx=0x7f6b40009040 comp rx=0 tx=0).stop 2024-09-21T03:37:12.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:12.639+0000 7f6b4c207640 1 -- 172.21.15.38:0/2996805048 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6b44162590 msgr2=0x7f6b441f9790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:12.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:12.639+0000 7f6b4c207640 1 --2- 172.21.15.38:0/2996805048 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6b44162590 0x7f6b441f9790 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f6b3c009d90 tx=0x7f6b3c003ba0 comp rx=0 tx=0).stop 2024-09-21T03:37:12.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:12.639+0000 7f6b4c207640 1 -- 172.21.15.38:0/2996805048 shutdown_connections 2024-09-21T03:37:12.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:12.639+0000 7f6b4c207640 1 --2- 172.21.15.38:0/2996805048 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6b44162590 0x7f6b441f9790 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:12.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:12.639+0000 7f6b4c207640 1 --2- 172.21.15.38:0/2996805048 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6b28078b90 0x7f6b2807b0a0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:12.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:12.639+0000 7f6b4c207640 1 --2- 172.21.15.38:0/2996805048 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6b440cfe00 0x7f6b441f9250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:12.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:12.639+0000 7f6b4c207640 1 --2- 172.21.15.38:0/2996805048 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6b440b2e60 0x7f6b441f8d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:12.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:12.639+0000 7f6b4c207640 1 -- 172.21.15.38:0/2996805048 >> 172.21.15.38:0/2996805048 conn(0x7f6b440d4440 msgr2=0x7f6b440b4ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:12.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:12.640+0000 7f6b4c207640 1 -- 172.21.15.38:0/2996805048 shutdown_connections 2024-09-21T03:37:12.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:12.640+0000 7f6b4c207640 1 -- 172.21.15.38:0/2996805048 wait complete. 2024-09-21T03:37:13.056 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 56, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T03:36:21.442724+0000', 'modified': '2024-09-21T03:37:12.045018+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 69, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5731}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5731': {'gid': 5731, 'name': 'b', 'rank': 0, 'incarnation': 56, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.195:6836/2536177898', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6834', 'nonce': 2536177898}, {'type': 'v1', 'addr': '172.21.15.195:6836', 'nonce': 2536177898}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-09-21T03:37:13.056 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_5731:up:replay 2024-09-21T03:37:13.831 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a has been restored 2024-09-21T03:37:14.057 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:37:14.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:14.180+0000 7f98aeffd640 1 --2- 172.21.15.38:0/117103957 >> [v2:172.21.15.38:6832/1058376326,v1:172.21.15.38:6834/1058376326] conn(0x55c451e5f030 0x55c451e5f440 unknown :-1 s=BANNER_CONNECTING pgs=15 cs=8 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:14.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:14.180+0000 7f98aeffd640 1 --2- 172.21.15.38:0/117103957 >> [v2:172.21.15.38:6832/1058376326,v1:172.21.15.38:6834/1058376326] conn(0x55c451e5f030 0x55c451e5f440 crc :-1 s=SESSION_RECONNECTING pgs=15 cs=8 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_session_reset received session reset full=1 2024-09-21T03:37:14.183 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:14.180+0000 7f98aeffd640 1 --2- 172.21.15.38:0/117103957 >> [v2:172.21.15.38:6832/1058376326,v1:172.21.15.38:6834/1058376326] conn(0x55c451e5f030 0x55c451e5f440 crc :-1 s=SESSION_RECONNECTING pgs=15 cs=8 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).reset_session 2024-09-21T03:37:14.183 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:14.180+0000 7f988dffb640 0 client.6297 ms_handle_remote_reset on v2:172.21.15.38:6832/1058376326 2024-09-21T03:37:14.183 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:14.180+0000 7f98aeffd640 1 -- 172.21.15.38:0/117103957 >> [v2:172.21.15.38:6832/1058376326,v1:172.21.15.38:6834/1058376326] conn(0x55c451e5f030 msgr2=0x55c451e5f440 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 18 2024-09-21T03:37:14.183 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:14.180+0000 7f98aeffd640 1 -- 172.21.15.38:0/117103957 >> [v2:172.21.15.38:6832/1058376326,v1:172.21.15.38:6834/1058376326] conn(0x55c451e5f030 msgr2=0x55c451e5f440 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-21T03:37:14.183 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:14.180+0000 7f98aeffd640 1 --2- 172.21.15.38:0/117103957 >> [v2:172.21.15.38:6832/1058376326,v1:172.21.15.38:6834/1058376326] conn(0x55c451e5f030 0x55c451e5f440 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-21T03:37:14.183 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:14.180+0000 7f98aeffd640 1 --2- 172.21.15.38:0/117103957 >> [v2:172.21.15.38:6832/1058376326,v1:172.21.15.38:6834/1058376326] conn(0x55c451e5f030 0x55c451e5f440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-09-21T03:37:14.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:14.185+0000 7f4660a8b640 1 -- 172.21.15.38:0/2680923844 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f465c0d0b40 msgr2=0x7f465c06e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:14.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:14.185+0000 7f4660a8b640 1 --2- 172.21.15.38:0/2680923844 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f465c0d0b40 0x7f465c06e6f0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f465c074d30 tx=0x7f464c02a560 comp rx=0 tx=0).stop 2024-09-21T03:37:14.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:14.185+0000 7f4660a8b640 1 -- 172.21.15.38:0/2680923844 shutdown_connections 2024-09-21T03:37:14.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:14.185+0000 7f4660a8b640 1 --2- 172.21.15.38:0/2680923844 >> v2:172.21.15.195:3300/0 conn(0x7f465c073d50 0x7f465c074160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:14.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:14.185+0000 7f4660a8b640 1 --2- 172.21.15.38:0/2680923844 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f465c0d0b40 0x7f465c06e6f0 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:14.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:14.185+0000 7f4660a8b640 1 -- 172.21.15.38:0/2680923844 >> 172.21.15.38:0/2680923844 conn(0x7f465c0b2cc0 msgr2=0x7f465c0b30d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:14.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:14.185+0000 7f4660a8b640 1 -- 172.21.15.38:0/2680923844 shutdown_connections 2024-09-21T03:37:14.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:14.185+0000 7f4660a8b640 1 -- 172.21.15.38:0/2680923844 wait complete. 2024-09-21T03:37:14.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:14.187+0000 7f4660a8b640 1 Processor -- start 2024-09-21T03:37:14.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:14.187+0000 7f4660a8b640 1 -- start start 2024-09-21T03:37:14.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:14.188+0000 7f4660a8b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f465c06ec30 0x7f465c1f4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:14.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:14.188+0000 7f4660a8b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f465c073d50 0x7f465c159ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:14.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:14.188+0000 7f4660a8b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f465c0d0b40 0x7f465c0ae000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:14.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:14.188+0000 7f4660a8b640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f465c15b750 con 0x7f465c0d0b40 2024-09-21T03:37:14.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:14.188+0000 7f4660a8b640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f465c15b5d0 con 0x7f465c06ec30 2024-09-21T03:37:14.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:14.188+0000 7f4660a8b640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f465c15b8d0 con 0x7f465c073d50 2024-09-21T03:37:14.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:14.188+0000 7f465ad9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f465c0d0b40 0x7f465c0ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:14.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:14.188+0000 7f465ad9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f465c0d0b40 0x7f465c0ae000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41030/0 (socket says 172.21.15.38:41030) 2024-09-21T03:37:14.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:14.188+0000 7f465ad9d640 1 -- 172.21.15.38:0/472132248 learned_addr learned my addr 172.21.15.38:0/472132248 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:37:14.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:14.188+0000 7f465a59c640 1 --2- 172.21.15.38:0/472132248 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f465c06ec30 0x7f465c1f4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:14.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:14.189+0000 7f4659d9b640 1 --2- 172.21.15.38:0/472132248 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f465c073d50 0x7f465c159ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:14.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:14.189+0000 7f465ad9d640 1 -- 172.21.15.38:0/472132248 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f465c073d50 msgr2=0x7f465c159ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:14.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:14.189+0000 7f465ad9d640 1 --2- 172.21.15.38:0/472132248 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f465c073d50 0x7f465c159ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:14.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:14.189+0000 7f465ad9d640 1 -- 172.21.15.38:0/472132248 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f465c06ec30 msgr2=0x7f465c1f4f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:14.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:14.189+0000 7f465ad9d640 1 --2- 172.21.15.38:0/472132248 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f465c06ec30 0x7f465c1f4f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:14.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:14.189+0000 7f465ad9d640 1 -- 172.21.15.38:0/472132248 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f465c15a1e0 con 0x7f465c0d0b40 2024-09-21T03:37:14.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:14.189+0000 7f465ad9d640 1 --2- 172.21.15.38:0/472132248 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f465c0d0b40 0x7f465c0ae000 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f465400cab0 tx=0x7f465400cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:14.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:14.189+0000 7f464affd640 1 -- 172.21.15.38:0/472132248 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4654007d90 con 0x7f465c0d0b40 2024-09-21T03:37:14.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:14.189+0000 7f464affd640 1 -- 172.21.15.38:0/472132248 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f465400f540 con 0x7f465c0d0b40 2024-09-21T03:37:14.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:14.189+0000 7f4660a8b640 1 -- 172.21.15.38:0/472132248 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f465c0cb680 con 0x7f465c0d0b40 2024-09-21T03:37:14.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:14.189+0000 7f464affd640 1 -- 172.21.15.38:0/472132248 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f465400f990 con 0x7f465c0d0b40 2024-09-21T03:37:14.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:14.189+0000 7f4660a8b640 1 -- 172.21.15.38:0/472132248 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f465c0cbc40 con 0x7f465c0d0b40 2024-09-21T03:37:14.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:14.190+0000 7f4660a8b640 1 -- 172.21.15.38:0/472132248 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f465c15bdf0 con 0x7f465c0d0b40 2024-09-21T03:37:14.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:14.191+0000 7f464affd640 1 -- 172.21.15.38:0/472132248 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f465400fb30 con 0x7f465c0d0b40 2024-09-21T03:37:14.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:14.191+0000 7f464affd640 1 --2- 172.21.15.38:0/472132248 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f463c078b90 0x7f463c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:14.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:14.192+0000 7f464affd640 1 -- 172.21.15.38:0/472132248 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6273+0+0 (secure 0 0 0) 0x7f465409b9c0 con 0x7f465c0d0b40 2024-09-21T03:37:14.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:14.192+0000 7f465a59c640 1 --2- 172.21.15.38:0/472132248 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f463c078b90 0x7f463c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:14.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:14.192+0000 7f465a59c640 1 --2- 172.21.15.38:0/472132248 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f463c078b90 0x7f463c07b0a0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f464c033000 tx=0x7f464c004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:14.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:14.196+0000 7f464affd640 1 -- 172.21.15.38:0/472132248 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f465409de20 con 0x7f465c0d0b40 2024-09-21T03:37:14.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:14.422+0000 7f4660a8b640 1 -- 172.21.15.38:0/472132248 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f465c0696c0 con 0x7f465c0d0b40 2024-09-21T03:37:14.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:14.422+0000 7f464affd640 1 -- 172.21.15.38:0/472132248 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) ==== 94+0+5225 (secure 0 0 0) 0x7f4654014070 con 0x7f465c0d0b40 2024-09-21T03:37:14.424 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:37:14.425 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":58,"btime":"2024-09-21T03:37:14:050353+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5432,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/1042764291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":1042764291},{"type":"v1","addr":"172.21.15.195:6837","nonce":1042764291}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":5728,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/3035019482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6833","nonce":3035019482},{"type":"v1","addr":"172.21.15.38:6835","nonce":3035019482}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":6348,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6834/3775275137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":3775275137},{"type":"v1","addr":"172.21.15.38:6834","nonce":3775275137}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":58,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:36:21.442724+0000","modified":"2024-09-21T03:37:13.054000+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":69,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5731},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5731":{"gid":5731,"name":"b","rank":0,"incarnation":56,"state":"up:rejoin","state_seq":16,"addr":"172.21.15.195:6836/2536177898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":2536177898},{"type":"v1","addr":"172.21.15.195:6836","nonce":2536177898}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":4}]} 2024-09-21T03:37:14.425 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 58 2024-09-21T03:37:14.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:14.425+0000 7f4660a8b640 1 -- 172.21.15.38:0/472132248 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f463c078b90 msgr2=0x7f463c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:14.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:14.425+0000 7f4660a8b640 1 --2- 172.21.15.38:0/472132248 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f463c078b90 0x7f463c07b0a0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f464c033000 tx=0x7f464c004730 comp rx=0 tx=0).stop 2024-09-21T03:37:14.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:14.425+0000 7f4660a8b640 1 -- 172.21.15.38:0/472132248 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f465c0d0b40 msgr2=0x7f465c0ae000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:14.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:14.425+0000 7f4660a8b640 1 --2- 172.21.15.38:0/472132248 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f465c0d0b40 0x7f465c0ae000 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f465400cab0 tx=0x7f465400cf70 comp rx=0 tx=0).stop 2024-09-21T03:37:14.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:14.425+0000 7f4660a8b640 1 -- 172.21.15.38:0/472132248 shutdown_connections 2024-09-21T03:37:14.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:14.425+0000 7f4660a8b640 1 --2- 172.21.15.38:0/472132248 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f465c0d0b40 0x7f465c0ae000 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:14.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:14.425+0000 7f4660a8b640 1 --2- 172.21.15.38:0/472132248 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f463c078b90 0x7f463c07b0a0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:14.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:14.425+0000 7f4660a8b640 1 --2- 172.21.15.38:0/472132248 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f465c073d50 0x7f465c159ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:14.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:14.425+0000 7f4660a8b640 1 --2- 172.21.15.38:0/472132248 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f465c06ec30 0x7f465c1f4f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:14.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:14.425+0000 7f4660a8b640 1 -- 172.21.15.38:0/472132248 >> 172.21.15.38:0/472132248 conn(0x7f465c0b2cc0 msgr2=0x7f465c15cc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:14.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:14.425+0000 7f4660a8b640 1 -- 172.21.15.38:0/472132248 shutdown_connections 2024-09-21T03:37:14.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:14.425+0000 7f4660a8b640 1 -- 172.21.15.38:0/472132248 wait complete. 2024-09-21T03:37:14.840 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 58, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T03:36:21.442724+0000', 'modified': '2024-09-21T03:37:13.054000+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 69, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5731}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5731': {'gid': 5731, 'name': 'b', 'rank': 0, 'incarnation': 56, 'state': 'up:rejoin', 'state_seq': 16, 'addr': '172.21.15.195:6836/2536177898', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6834', 'nonce': 2536177898}, {'type': 'v1', 'addr': '172.21.15.195:6836', 'nonce': 2536177898}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-09-21T03:37:14.840 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_5731:up:rejoin 2024-09-21T03:37:15.841 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:37:15.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:15.970+0000 7f4cfb830640 1 -- 172.21.15.38:0/2293047223 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4cf41536e0 msgr2=0x7f4cf40d17c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:15.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:15.970+0000 7f4cfb830640 1 --2- 172.21.15.38:0/2293047223 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4cf41536e0 0x7f4cf40d17c0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f4cf0005fd0 tx=0x7f4cf002a8b0 comp rx=0 tx=0).stop 2024-09-21T03:37:15.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:15.970+0000 7f4cfb830640 1 -- 172.21.15.38:0/2293047223 shutdown_connections 2024-09-21T03:37:15.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:15.970+0000 7f4cfb830640 1 --2- 172.21.15.38:0/2293047223 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4cf41536e0 0x7f4cf40d17c0 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:15.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:15.970+0000 7f4cfb830640 1 -- 172.21.15.38:0/2293047223 >> 172.21.15.38:0/2293047223 conn(0x7f4cf40cef40 msgr2=0x7f4cf40cf350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:15.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:15.971+0000 7f4cfb830640 1 -- 172.21.15.38:0/2293047223 shutdown_connections 2024-09-21T03:37:15.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:15.971+0000 7f4cfb830640 1 -- 172.21.15.38:0/2293047223 wait complete. 2024-09-21T03:37:15.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:15.972+0000 7f4cfb830640 1 Processor -- start 2024-09-21T03:37:15.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:15.972+0000 7f4cfb830640 1 -- start start 2024-09-21T03:37:15.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:15.974+0000 7f4cfb830640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4cf40d1d00 0x7f4cf406f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:15.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:15.974+0000 7f4cfb830640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4cf41536e0 0x7f4cf415b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:15.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:15.974+0000 7f4cfb830640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4cf41e5f70 0x7f4cf415bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:15.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:15.974+0000 7f4cfb830640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f4cf40d0f60 con 0x7f4cf41536e0 2024-09-21T03:37:15.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:15.974+0000 7f4cfb830640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f4cf40d0de0 con 0x7f4cf41e5f70 2024-09-21T03:37:15.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:15.974+0000 7f4cfb830640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f4cf40d10e0 con 0x7f4cf40d1d00 2024-09-21T03:37:15.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:15.974+0000 7f4cf8dcb640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4cf41536e0 0x7f4cf415b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:15.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:15.974+0000 7f4cf8dcb640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4cf41536e0 0x7f4cf415b5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41050/0 (socket says 172.21.15.38:41050) 2024-09-21T03:37:15.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:15.974+0000 7f4cf8dcb640 1 -- 172.21.15.38:0/402197897 learned_addr learned my addr 172.21.15.38:0/402197897 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:37:15.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:15.974+0000 7f4cf95cc640 1 --2- 172.21.15.38:0/402197897 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4cf40d1d00 0x7f4cf406f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:15.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:15.974+0000 7f4cf9dcd640 1 --2- 172.21.15.38:0/402197897 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4cf41e5f70 0x7f4cf415bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:15.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:15.974+0000 7f4cf8dcb640 1 -- 172.21.15.38:0/402197897 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4cf40d1d00 msgr2=0x7f4cf406f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:15.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:15.974+0000 7f4cf8dcb640 1 --2- 172.21.15.38:0/402197897 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4cf40d1d00 0x7f4cf406f9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:15.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:15.974+0000 7f4cf8dcb640 1 -- 172.21.15.38:0/402197897 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4cf41e5f70 msgr2=0x7f4cf415bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:15.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:15.974+0000 7f4cf8dcb640 1 --2- 172.21.15.38:0/402197897 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4cf41e5f70 0x7f4cf415bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:15.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:15.974+0000 7f4cf8dcb640 1 -- 172.21.15.38:0/402197897 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cf4152850 con 0x7f4cf41536e0 2024-09-21T03:37:15.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:15.974+0000 7f4cf9dcd640 1 --2- 172.21.15.38:0/402197897 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4cf41e5f70 0x7f4cf415bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:37:15.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:15.975+0000 7f4cf8dcb640 1 --2- 172.21.15.38:0/402197897 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4cf41536e0 0x7f4cf415b5e0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f4ce400b420 tx=0x7f4ce400b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:15.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:15.975+0000 7f4ce9ffb640 1 -- 172.21.15.38:0/402197897 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ce4011830 con 0x7f4cf41536e0 2024-09-21T03:37:15.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:15.975+0000 7f4ce9ffb640 1 -- 172.21.15.38:0/402197897 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4ce4016070 con 0x7f4cf41536e0 2024-09-21T03:37:15.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:15.975+0000 7f4cfb830640 1 -- 172.21.15.38:0/402197897 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cf41f5960 con 0x7f4cf41536e0 2024-09-21T03:37:15.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:15.975+0000 7f4ce9ffb640 1 -- 172.21.15.38:0/402197897 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ce4011d60 con 0x7f4cf41536e0 2024-09-21T03:37:15.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:15.975+0000 7f4cfb830640 1 -- 172.21.15.38:0/402197897 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cf41f5f10 con 0x7f4cf41536e0 2024-09-21T03:37:15.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:15.976+0000 7f4cfb830640 1 -- 172.21.15.38:0/402197897 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4cf40d3430 con 0x7f4cf41536e0 2024-09-21T03:37:15.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:15.977+0000 7f4ce9ffb640 1 -- 172.21.15.38:0/402197897 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f4ce401b460 con 0x7f4cf41536e0 2024-09-21T03:37:15.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:15.977+0000 7f4ce9ffb640 1 --2- 172.21.15.38:0/402197897 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4cd8078b90 0x7f4cd807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:15.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:15.977+0000 7f4ce9ffb640 1 -- 172.21.15.38:0/402197897 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6273+0+0 (secure 0 0 0) 0x7f4ce409aa10 con 0x7f4cf41536e0 2024-09-21T03:37:15.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:15.978+0000 7f4cf95cc640 1 --2- 172.21.15.38:0/402197897 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4cd8078b90 0x7f4cd807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:15.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:15.978+0000 7f4cf95cc640 1 --2- 172.21.15.38:0/402197897 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4cd8078b90 0x7f4cd807b0a0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f4cf0034000 tx=0x7f4cf0004ab0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:15.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:15.982+0000 7f4ce9ffb640 1 -- 172.21.15.38:0/402197897 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f4cf40d3430 con 0x7f4cf41536e0 2024-09-21T03:37:16.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.210+0000 7f4cfb830640 1 -- 172.21.15.38:0/402197897 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4cf40d17c0 con 0x7f4cf41536e0 2024-09-21T03:37:16.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.211+0000 7f4ce9ffb640 1 -- 172.21.15.38:0/402197897 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) ==== 94+0+5232 (secure 0 0 0) 0x7f4ce4020020 con 0x7f4cf41536e0 2024-09-21T03:37:16.213 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:37:16.213 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":59,"btime":"2024-09-21T03:37:15:051941+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5432,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/1042764291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":1042764291},{"type":"v1","addr":"172.21.15.195:6837","nonce":1042764291}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":5728,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/3035019482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6833","nonce":3035019482},{"type":"v1","addr":"172.21.15.38:6835","nonce":3035019482}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":6348,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6834/3775275137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":3775275137},{"type":"v1","addr":"172.21.15.38:6834","nonce":3775275137}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":59,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:36:21.442724+0000","modified":"2024-09-21T03:37:15.051929+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":69,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5731},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5731":{"gid":5731,"name":"b","rank":0,"incarnation":56,"state":"up:active","state_seq":17,"addr":"172.21.15.195:6836/2536177898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":2536177898},{"type":"v1","addr":"172.21.15.195:6836","nonce":2536177898}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5731,"qdb_cluster":[5731]},"id":4}]} 2024-09-21T03:37:16.213 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 59 2024-09-21T03:37:16.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.213+0000 7f4cfb830640 1 -- 172.21.15.38:0/402197897 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4cd8078b90 msgr2=0x7f4cd807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:16.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.213+0000 7f4cfb830640 1 --2- 172.21.15.38:0/402197897 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4cd8078b90 0x7f4cd807b0a0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f4cf0034000 tx=0x7f4cf0004ab0 comp rx=0 tx=0).stop 2024-09-21T03:37:16.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.213+0000 7f4cfb830640 1 -- 172.21.15.38:0/402197897 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4cf41536e0 msgr2=0x7f4cf415b5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:16.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.213+0000 7f4cfb830640 1 --2- 172.21.15.38:0/402197897 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4cf41536e0 0x7f4cf415b5e0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f4ce400b420 tx=0x7f4ce400b8e0 comp rx=0 tx=0).stop 2024-09-21T03:37:16.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.213+0000 7f4cfb830640 1 -- 172.21.15.38:0/402197897 shutdown_connections 2024-09-21T03:37:16.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.213+0000 7f4cfb830640 1 --2- 172.21.15.38:0/402197897 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4cf41e5f70 0x7f4cf415bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:16.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.213+0000 7f4cfb830640 1 --2- 172.21.15.38:0/402197897 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4cf41536e0 0x7f4cf415b5e0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:16.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.213+0000 7f4cfb830640 1 --2- 172.21.15.38:0/402197897 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4cd8078b90 0x7f4cd807b0a0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:16.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.213+0000 7f4cfb830640 1 --2- 172.21.15.38:0/402197897 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4cf40d1d00 0x7f4cf406f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:16.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.213+0000 7f4cfb830640 1 -- 172.21.15.38:0/402197897 >> 172.21.15.38:0/402197897 conn(0x7f4cf40cef40 msgr2=0x7f4cf4155c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:16.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.213+0000 7f4cfb830640 1 -- 172.21.15.38:0/402197897 shutdown_connections 2024-09-21T03:37:16.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.213+0000 7f4cfb830640 1 -- 172.21.15.38:0/402197897 wait complete. 2024-09-21T03:37:16.629 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 59, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T03:36:21.442724+0000', 'modified': '2024-09-21T03:37:15.051929+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 69, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5731}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5731': {'gid': 5731, 'name': 'b', 'rank': 0, 'incarnation': 56, 'state': 'up:active', 'state_seq': 17, 'addr': '172.21.15.195:6836/2536177898', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6834', 'nonce': 2536177898}, {'type': 'v1', 'addr': '172.21.15.195:6836', 'nonce': 2536177898}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5731, 'qdb_cluster': [5731]} 2024-09-21T03:37:16.629 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-21T03:37:16.630 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-09-21T03:37:16.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.749+0000 7f0545215640 1 -- 172.21.15.38:0/3722493700 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f05400d0bb0 msgr2=0x7f05400b5be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:16.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.749+0000 7f0545215640 1 --2- 172.21.15.38:0/3722493700 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f05400d0bb0 0x7f05400b5be0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f0530005fd0 tx=0x7f053002a4a0 comp rx=0 tx=0).stop 2024-09-21T03:37:16.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.749+0000 7f0545215640 1 -- 172.21.15.38:0/3722493700 shutdown_connections 2024-09-21T03:37:16.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.749+0000 7f0545215640 1 --2- 172.21.15.38:0/3722493700 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f05400d0bb0 0x7f05400b5be0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:16.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.749+0000 7f0545215640 1 --2- 172.21.15.38:0/3722493700 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f054015ca90 0x7f05400d0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:16.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.749+0000 7f0545215640 1 -- 172.21.15.38:0/3722493700 >> 172.21.15.38:0/3722493700 conn(0x7f05400b46e0 msgr2=0x7f05400d23e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:16.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.749+0000 7f0545215640 1 -- 172.21.15.38:0/3722493700 shutdown_connections 2024-09-21T03:37:16.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.749+0000 7f0545215640 1 -- 172.21.15.38:0/3722493700 wait complete. 2024-09-21T03:37:16.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.751+0000 7f0545215640 1 Processor -- start 2024-09-21T03:37:16.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.751+0000 7f0545215640 1 -- start start 2024-09-21T03:37:16.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.753+0000 7f0545215640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f05400ccc40 0x7f054006def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:16.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.753+0000 7f0545215640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f05400d0bb0 0x7f0540147d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:16.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.753+0000 7f0545215640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f054015ca90 0x7f05401e8980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:16.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.753+0000 7f0545215640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f05400b6620 con 0x7f054015ca90 2024-09-21T03:37:16.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.753+0000 7f0545215640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f05400b64a0 con 0x7f05400ccc40 2024-09-21T03:37:16.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.753+0000 7f0545215640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f05400b67a0 con 0x7f05400d0bb0 2024-09-21T03:37:16.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.753+0000 7f053f59e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f054015ca90 0x7f05401e8980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:16.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.753+0000 7f053f59e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f054015ca90 0x7f05401e8980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41062/0 (socket says 172.21.15.38:41062) 2024-09-21T03:37:16.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.753+0000 7f053f59e640 1 -- 172.21.15.38:0/576049833 learned_addr learned my addr 172.21.15.38:0/576049833 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:37:16.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.753+0000 7f053e59c640 1 --2- 172.21.15.38:0/576049833 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f05400d0bb0 0x7f0540147d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:16.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.753+0000 7f053ed9d640 1 --2- 172.21.15.38:0/576049833 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f05400ccc40 0x7f054006def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:16.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.753+0000 7f053f59e640 1 -- 172.21.15.38:0/576049833 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f05400d0bb0 msgr2=0x7f0540147d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:16.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.753+0000 7f053f59e640 1 --2- 172.21.15.38:0/576049833 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f05400d0bb0 0x7f0540147d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:16.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.753+0000 7f053f59e640 1 -- 172.21.15.38:0/576049833 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f05400ccc40 msgr2=0x7f054006def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:16.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.753+0000 7f053f59e640 1 --2- 172.21.15.38:0/576049833 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f05400ccc40 0x7f054006def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:16.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.753+0000 7f053f59e640 1 -- 172.21.15.38:0/576049833 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f054006e430 con 0x7f054015ca90 2024-09-21T03:37:16.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.753+0000 7f053f59e640 1 --2- 172.21.15.38:0/576049833 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f054015ca90 0x7f05401e8980 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f0530006010 tx=0x7f0530003960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:16.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.753+0000 7f05377fe640 1 -- 172.21.15.38:0/576049833 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f053003e030 con 0x7f054015ca90 2024-09-21T03:37:16.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.754+0000 7f05377fe640 1 -- 172.21.15.38:0/576049833 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0530003e50 con 0x7f054015ca90 2024-09-21T03:37:16.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.754+0000 7f05377fe640 1 -- 172.21.15.38:0/576049833 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0530035040 con 0x7f054015ca90 2024-09-21T03:37:16.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.754+0000 7f0545215640 1 -- 172.21.15.38:0/576049833 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f054006e8b0 con 0x7f054015ca90 2024-09-21T03:37:16.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.754+0000 7f0545215640 1 -- 172.21.15.38:0/576049833 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f054006ee70 con 0x7f054015ca90 2024-09-21T03:37:16.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.756+0000 7f05377fe640 1 -- 172.21.15.38:0/576049833 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f0530041030 con 0x7f054015ca90 2024-09-21T03:37:16.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.756+0000 7f05377fe640 1 --2- 172.21.15.38:0/576049833 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0520078b90 0x7f052007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:16.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.756+0000 7f05377fe640 1 -- 172.21.15.38:0/576049833 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 6476+0+0 (secure 0 0 0) 0x7f05300bb840 con 0x7f054015ca90 2024-09-21T03:37:16.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.757+0000 7f053ed9d640 1 --2- 172.21.15.38:0/576049833 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0520078b90 0x7f052007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:16.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.757+0000 7f053ed9d640 1 --2- 172.21.15.38:0/576049833 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0520078b90 0x7f052007b0a0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f053800b450 tx=0x7f053800d040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:16.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.757+0000 7f054c260740 1 Processor -- start 2024-09-21T03:37:16.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.758+0000 7f054c260740 1 -- start start 2024-09-21T03:37:16.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.759+0000 7f054c260740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560ad9572270 0x560ad9943fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:16.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.759+0000 7f054c260740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560ad95493b0 0x560ad9944500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:16.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.759+0000 7f054c260740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560ad9551590 0x560ad9961400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:16.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.759+0000 7f054c260740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x560ad988c820 con 0x560ad9551590 2024-09-21T03:37:16.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.759+0000 7f054c260740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x560ad987bc50 con 0x560ad95493b0 2024-09-21T03:37:16.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.759+0000 7f054c260740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x560ad987fa60 con 0x560ad9572270 2024-09-21T03:37:16.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.759+0000 7f053f59e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560ad9551590 0x560ad9961400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:16.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.759+0000 7f053f59e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560ad9551590 0x560ad9961400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41072/0 (socket says 172.21.15.38:41072) 2024-09-21T03:37:16.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.759+0000 7f053f59e640 1 -- 172.21.15.38:0/3591185779 learned_addr learned my addr 172.21.15.38:0/3591185779 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:37:16.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.759+0000 7f053e59c640 1 --2- 172.21.15.38:0/3591185779 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560ad95493b0 0x560ad9944500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:16.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.759+0000 7f053ed9d640 1 --2- 172.21.15.38:0/3591185779 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560ad9572270 0x560ad9943fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:16.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.759+0000 7f053f59e640 1 -- 172.21.15.38:0/3591185779 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560ad9572270 msgr2=0x560ad9943fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:37:16.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.759+0000 7f053f59e640 1 --2- 172.21.15.38:0/3591185779 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560ad9572270 0x560ad9943fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:16.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.759+0000 7f053f59e640 1 -- 172.21.15.38:0/3591185779 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560ad95493b0 msgr2=0x560ad9944500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:37:16.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.759+0000 7f053f59e640 1 --2- 172.21.15.38:0/3591185779 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560ad95493b0 0x560ad9944500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:16.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.759+0000 7f053f59e640 1 -- 172.21.15.38:0/3591185779 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ad95dd850 con 0x560ad9551590 2024-09-21T03:37:16.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.759+0000 7f053f59e640 1 --2- 172.21.15.38:0/3591185779 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560ad9551590 0x560ad9961400 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f0530005820 tx=0x7f0530082d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d93fc924f0618307 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:16.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.760+0000 7f0534ff9640 1 -- 172.21.15.38:0/3591185779 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0530048040 con 0x560ad9551590 2024-09-21T03:37:16.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.760+0000 7f0534ff9640 1 -- 172.21.15.38:0/3591185779 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f05300833c0 con 0x560ad9551590 2024-09-21T03:37:16.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.760+0000 7f0534ff9640 1 -- 172.21.15.38:0/3591185779 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f053008a330 con 0x560ad9551590 2024-09-21T03:37:16.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.760+0000 7f054c260740 1 -- 172.21.15.38:0/3591185779 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560ad9551590 msgr2=0x560ad9961400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:16.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.760+0000 7f054c260740 1 --2- 172.21.15.38:0/3591185779 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560ad9551590 0x560ad9961400 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f0530005820 tx=0x7f0530082d10 comp rx=0 tx=0).stop 2024-09-21T03:37:16.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.761+0000 7f054c260740 1 -- 172.21.15.38:0/3591185779 shutdown_connections 2024-09-21T03:37:16.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.761+0000 7f054c260740 1 --2- 172.21.15.38:0/3591185779 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560ad9551590 0x560ad9961400 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:16.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.761+0000 7f054c260740 1 --2- 172.21.15.38:0/3591185779 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560ad95493b0 0x560ad9944500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:16.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.761+0000 7f054c260740 1 --2- 172.21.15.38:0/3591185779 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560ad9572270 0x560ad9943fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:16.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.761+0000 7f054c260740 1 -- 172.21.15.38:0/3591185779 >> 172.21.15.38:0/3591185779 conn(0x560ad95663c0 msgr2=0x560ad991a280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:16.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.761+0000 7f054c260740 1 -- 172.21.15.38:0/3591185779 shutdown_connections 2024-09-21T03:37:16.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.761+0000 7f054c260740 1 -- 172.21.15.38:0/3591185779 wait complete. 2024-09-21T03:37:16.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.761+0000 7f054c260740 1 -- start start 2024-09-21T03:37:16.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.762+0000 7f054c260740 1 Processor -- start 2024-09-21T03:37:16.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.763+0000 7f054c260740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560ad95493b0 0x560ad995b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:16.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.763+0000 7f054c260740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560ad9551590 0x560ad9958980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:16.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.763+0000 7f054c260740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560ad9572270 0x560ad9955bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:16.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.763+0000 7f054c260740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x560ad95ec620 con 0x560ad9551590 2024-09-21T03:37:16.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.763+0000 7f054c260740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x560ad98215f0 con 0x560ad95493b0 2024-09-21T03:37:16.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.763+0000 7f054c260740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x560ad9944a40 con 0x560ad9572270 2024-09-21T03:37:16.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.763+0000 7f054c260740 10 client.0 _ll_register_callbacks cb 0x560ad98250a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:37:16.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.763+0000 7f053e59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560ad9551590 0x560ad9958980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:16.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.763+0000 7f053e59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560ad9551590 0x560ad9958980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41078/0 (socket says 172.21.15.38:41078) 2024-09-21T03:37:16.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.763+0000 7f053e59c640 1 -- 172.21.15.38:0/1247892238 learned_addr learned my addr 172.21.15.38:0/1247892238 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:37:16.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.763+0000 7f053ed9d640 1 --2- 172.21.15.38:0/1247892238 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560ad95493b0 0x560ad995b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:16.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.763+0000 7f053f59e640 1 --2- 172.21.15.38:0/1247892238 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560ad9572270 0x560ad9955bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:16.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.763+0000 7f053e59c640 1 -- 172.21.15.38:0/1247892238 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560ad9572270 msgr2=0x560ad9955bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:37:16.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.763+0000 7f053e59c640 1 --2- 172.21.15.38:0/1247892238 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560ad9572270 0x560ad9955bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:16.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.763+0000 7f053e59c640 1 -- 172.21.15.38:0/1247892238 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560ad95493b0 msgr2=0x560ad995b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:37:16.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.763+0000 7f053e59c640 1 --2- 172.21.15.38:0/1247892238 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560ad95493b0 0x560ad995b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:16.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.763+0000 7f053e59c640 1 -- 172.21.15.38:0/1247892238 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ad96546b0 con 0x560ad9551590 2024-09-21T03:37:16.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.763+0000 7f053f59e640 1 --2- 172.21.15.38:0/1247892238 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560ad9572270 0x560ad9955bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:37:16.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.763+0000 7f053ed9d640 1 --2- 172.21.15.38:0/1247892238 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560ad95493b0 0x560ad995b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:37:16.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.763+0000 7f053e59c640 1 --2- 172.21.15.38:0/1247892238 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560ad9551590 0x560ad9958980 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f0528010850 tx=0x7f0528010d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9da9249e20a41e95 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:16.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.763+0000 7f051dffb640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-21T03:37:16.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.764+0000 7f051dffb640 1 -- 172.21.15.38:0/1247892238 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0528004240 con 0x560ad9551590 2024-09-21T03:37:16.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.764+0000 7f051dffb640 1 -- 172.21.15.38:0/1247892238 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(70..70 src has 1..70) ==== 6476+0+0 (secure 0 0 0) 0x7f052801bdd0 con 0x560ad9551590 2024-09-21T03:37:16.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.764+0000 7f054c260740 1 -- 172.21.15.38:0/1247892238 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f05300833c0 con 0x560ad9551590 2024-09-21T03:37:16.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.764+0000 7f051dffb640 1 -- 172.21.15.38:0/1247892238 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7f05280043e0 con 0x560ad9551590 2024-09-21T03:37:16.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.764+0000 7f054c260740 10 client.6378 fetch_fsmap learned FSMap version 59 2024-09-21T03:37:16.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.764+0000 7f054c260740 1 -- 172.21.15.38:0/1247892238 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=2+}) -- 0x560ad9956570 con 0x560ad9551590 2024-09-21T03:37:16.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.764+0000 7f051dffb640 1 -- 172.21.15.38:0/1247892238 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f05300833c0 con 0x560ad9551590 2024-09-21T03:37:16.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.764+0000 7f051dffb640 1 -- 172.21.15.38:0/1247892238 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 59) ==== 3502+0+0 (secure 0 0 0) 0x7f0528022050 con 0x560ad9551590 2024-09-21T03:37:16.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.764+0000 7f054c260740 10 client.6378 fetch_fsmap finished waiting for FSMap version 59 2024-09-21T03:37:16.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.764+0000 7f054c260740 10 client.6378 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-21T03:37:16.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.764+0000 7f054c260740 20 client.6378 populate_metadata read hostname 'smithi038' 2024-09-21T03:37:16.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.764+0000 7f054c260740 1 --2- 172.21.15.38:0/1247892238 >> [v2:172.21.15.195:6834/2536177898,v1:172.21.15.195:6836/2536177898] conn(0x560ad9530e40 0x560ad9931fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:16.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.764+0000 7f054c260740 4 client.6378 mds_command: new command op to 5731 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:37:16.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.764+0000 7f054c260740 1 -- 172.21.15.38:0/1247892238 --> [v2:172.21.15.195:6834/2536177898,v1:172.21.15.195:6836/2536177898] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560ad988e770 con 0x560ad9530e40 2024-09-21T03:37:16.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.764+0000 7f053ed9d640 1 --2- 172.21.15.38:0/1247892238 >> [v2:172.21.15.195:6834/2536177898,v1:172.21.15.195:6836/2536177898] conn(0x560ad9530e40 0x560ad9931fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:16.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.765+0000 7f053ed9d640 1 --2- 172.21.15.38:0/1247892238 >> [v2:172.21.15.195:6834/2536177898,v1:172.21.15.195:6836/2536177898] conn(0x560ad9530e40 0x560ad9931fe0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6cc7ea603eba9793 server_cookie=ceed412bb8f17dd1 in_seq=0 out_seq=0 2024-09-21T03:37:16.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.765+0000 7f051dffb640 10 client.6378 ms_handle_connect on v2:172.21.15.195:6834/2536177898 2024-09-21T03:37:16.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.766+0000 7f051dffb640 1 -- 172.21.15.38:0/1247892238 <== mds.0 v2:172.21.15.195:6834/2536177898 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x560ad988e770 con 0x560ad9530e40 2024-09-21T03:37:16.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.766+0000 7f051dffb640 10 client.6378 handle_command_reply: tid=0 2024-09-21T03:37:16.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.766+0000 7f054c260740 1 client.6378 shutdown 2024-09-21T03:37:16.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.766+0000 7f054c260740 1 -- 172.21.15.38:0/1247892238 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560ad9551590 msgr2=0x560ad9958980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:16.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.766+0000 7f054c260740 1 --2- 172.21.15.38:0/1247892238 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560ad9551590 0x560ad9958980 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f0528010850 tx=0x7f0528010d10 comp rx=0 tx=0).stop 2024-09-21T03:37:16.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.766+0000 7f054c260740 1 -- 172.21.15.38:0/1247892238 shutdown_connections 2024-09-21T03:37:16.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.766+0000 7f054c260740 1 --2- 172.21.15.38:0/1247892238 >> [v2:172.21.15.195:6834/2536177898,v1:172.21.15.195:6836/2536177898] conn(0x560ad9530e40 0x560ad9931fe0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:16.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.766+0000 7f054c260740 1 --2- 172.21.15.38:0/1247892238 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560ad9572270 0x560ad9955bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:16.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.766+0000 7f051dffb640 0 client.6378 ms_handle_reset on v2:172.21.15.195:6834/2536177898 2024-09-21T03:37:16.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.766+0000 7f054c260740 1 --2- 172.21.15.38:0/1247892238 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560ad9551590 0x560ad9958980 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:16.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.766+0000 7f054c260740 1 --2- 172.21.15.38:0/1247892238 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560ad95493b0 0x560ad995b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:16.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.766+0000 7f054c260740 1 -- 172.21.15.38:0/1247892238 >> 172.21.15.38:0/1247892238 conn(0x560ad95663c0 msgr2=0x560ad9913550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:16.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.766+0000 7f054c260740 1 -- 172.21.15.38:0/1247892238 shutdown_connections 2024-09-21T03:37:16.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.766+0000 7f054c260740 1 -- 172.21.15.38:0/1247892238 wait complete. 2024-09-21T03:37:16.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.766+0000 7f054c260740 20 client.6378 trim_cache size 0 max 16384 2024-09-21T03:37:16.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.785+0000 7f054c260740 1 Processor -- start 2024-09-21T03:37:16.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.785+0000 7f054c260740 1 -- start start 2024-09-21T03:37:16.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.786+0000 7f054c260740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560ad95493b0 0x560ad9983040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:16.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.786+0000 7f054c260740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560ad9551590 0x560ad991a110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:16.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.786+0000 7f054c260740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560ad9572270 0x560ad991a650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:16.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.786+0000 7f054c260740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f0540201900 con 0x560ad95493b0 2024-09-21T03:37:16.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.786+0000 7f054c260740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f0540200d10 con 0x560ad9572270 2024-09-21T03:37:16.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.786+0000 7f054c260740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f0540201230 con 0x560ad9551590 2024-09-21T03:37:16.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.786+0000 7f053ed9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560ad95493b0 0x560ad9983040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:16.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.786+0000 7f053ed9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560ad95493b0 0x560ad9983040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41092/0 (socket says 172.21.15.38:41092) 2024-09-21T03:37:16.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.786+0000 7f053ed9d640 1 -- 172.21.15.38:0/2677939053 learned_addr learned my addr 172.21.15.38:0/2677939053 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:37:16.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.786+0000 7f053ed9d640 1 -- 172.21.15.38:0/2677939053 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560ad9551590 msgr2=0x560ad991a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:37:16.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.786+0000 7f053e59c640 1 --2- 172.21.15.38:0/2677939053 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560ad9551590 0x560ad991a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:16.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.786+0000 7f053f59e640 1 --2- 172.21.15.38:0/2677939053 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560ad9572270 0x560ad991a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:16.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.787+0000 7f053ed9d640 1 --2- 172.21.15.38:0/2677939053 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560ad9551590 0x560ad991a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:16.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.787+0000 7f053ed9d640 1 -- 172.21.15.38:0/2677939053 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560ad9572270 msgr2=0x560ad991a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:37:16.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.787+0000 7f053ed9d640 1 --2- 172.21.15.38:0/2677939053 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560ad9572270 0x560ad991a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:16.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.787+0000 7f053ed9d640 1 -- 172.21.15.38:0/2677939053 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ad991ab90 con 0x560ad95493b0 2024-09-21T03:37:16.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.787+0000 7f053f59e640 1 --2- 172.21.15.38:0/2677939053 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560ad9572270 0x560ad991a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:37:16.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.787+0000 7f053e59c640 1 --2- 172.21.15.38:0/2677939053 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560ad9551590 0x560ad991a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:37:16.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.787+0000 7f053ed9d640 1 --2- 172.21.15.38:0/2677939053 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560ad95493b0 0x560ad9983040 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f053800bce0 tx=0x7f0538011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5e78c8051d7ca8c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:16.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.787+0000 7f051ffff640 1 -- 172.21.15.38:0/2677939053 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f053800ad50 con 0x560ad95493b0 2024-09-21T03:37:16.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.787+0000 7f051ffff640 1 -- 172.21.15.38:0/2677939053 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f053802c420 con 0x560ad95493b0 2024-09-21T03:37:16.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.787+0000 7f051ffff640 1 -- 172.21.15.38:0/2677939053 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f053802c850 con 0x560ad95493b0 2024-09-21T03:37:16.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.788+0000 7f054c260740 1 -- 172.21.15.38:0/2677939053 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560ad95493b0 msgr2=0x560ad9983040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:16.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.788+0000 7f054c260740 1 --2- 172.21.15.38:0/2677939053 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560ad95493b0 0x560ad9983040 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f053800bce0 tx=0x7f0538011000 comp rx=0 tx=0).stop 2024-09-21T03:37:16.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.788+0000 7f054c260740 1 -- 172.21.15.38:0/2677939053 shutdown_connections 2024-09-21T03:37:16.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.788+0000 7f054c260740 1 --2- 172.21.15.38:0/2677939053 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560ad9572270 0x560ad991a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:16.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.788+0000 7f054c260740 1 --2- 172.21.15.38:0/2677939053 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560ad9551590 0x560ad991a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:16.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.788+0000 7f054c260740 1 --2- 172.21.15.38:0/2677939053 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560ad95493b0 0x560ad9983040 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:16.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.788+0000 7f054c260740 1 -- 172.21.15.38:0/2677939053 >> 172.21.15.38:0/2677939053 conn(0x560ad95663c0 msgr2=0x560ad990f960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:16.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.788+0000 7f054c260740 1 -- 172.21.15.38:0/2677939053 shutdown_connections 2024-09-21T03:37:16.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.788+0000 7f054c260740 1 -- 172.21.15.38:0/2677939053 wait complete. 2024-09-21T03:37:16.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.788+0000 7f054c260740 1 -- start start 2024-09-21T03:37:16.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.789+0000 7f054c260740 1 Processor -- start 2024-09-21T03:37:16.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.790+0000 7f054c260740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560ad95493b0 0x560ad99df420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:16.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.790+0000 7f054c260740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560ad9551590 0x560ad9913550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:16.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.790+0000 7f054c260740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560ad9572270 0x560ad9913a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:16.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.790+0000 7f054c260740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x560ad99561c0 con 0x560ad9572270 2024-09-21T03:37:16.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.790+0000 7f054c260740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x560ad9958ec0 con 0x560ad9551590 2024-09-21T03:37:16.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.790+0000 7f054c260740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x560ad99570b0 con 0x560ad95493b0 2024-09-21T03:37:16.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.790+0000 7f054c260740 10 client.0 _ll_register_callbacks cb 0x7f052800a6a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:37:16.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.790+0000 7f053f59e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560ad9572270 0x560ad9913a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:16.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.790+0000 7f053f59e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560ad9572270 0x560ad9913a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41098/0 (socket says 172.21.15.38:41098) 2024-09-21T03:37:16.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.790+0000 7f053f59e640 1 -- 172.21.15.38:0/3541347897 learned_addr learned my addr 172.21.15.38:0/3541347897 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:37:16.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.790+0000 7f053e59c640 1 --2- 172.21.15.38:0/3541347897 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560ad9551590 0x560ad9913550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:16.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.790+0000 7f053ed9d640 1 --2- 172.21.15.38:0/3541347897 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560ad95493b0 0x560ad99df420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:16.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.790+0000 7f053f59e640 1 -- 172.21.15.38:0/3541347897 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560ad95493b0 msgr2=0x560ad99df420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:37:16.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.790+0000 7f053f59e640 1 --2- 172.21.15.38:0/3541347897 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560ad95493b0 0x560ad99df420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:16.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.790+0000 7f053f59e640 1 -- 172.21.15.38:0/3541347897 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560ad9551590 msgr2=0x560ad9913550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:37:16.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.790+0000 7f053f59e640 1 --2- 172.21.15.38:0/3541347897 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560ad9551590 0x560ad9913550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:16.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.790+0000 7f053f59e640 1 -- 172.21.15.38:0/3541347897 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ad9961b50 con 0x560ad9572270 2024-09-21T03:37:16.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.790+0000 7f053ed9d640 1 --2- 172.21.15.38:0/3541347897 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560ad95493b0 0x560ad99df420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:37:16.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.791+0000 7f053f59e640 1 --2- 172.21.15.38:0/3541347897 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560ad9572270 0x560ad9913a90 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f0530034040 tx=0x7f053008ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b08322a020db10 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:16.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.791+0000 7f051dffb640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-21T03:37:16.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.791+0000 7f051dffb640 1 -- 172.21.15.38:0/3541347897 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0530048030 con 0x560ad9572270 2024-09-21T03:37:16.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.791+0000 7f051dffb640 1 -- 172.21.15.38:0/3541347897 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(70..70 src has 1..70) ==== 6476+0+0 (secure 0 0 0) 0x7f053008a2c0 con 0x560ad9572270 2024-09-21T03:37:16.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.791+0000 7f054c260740 1 -- 172.21.15.38:0/3541347897 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f053802c420 con 0x560ad9572270 2024-09-21T03:37:16.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.791+0000 7f051dffb640 1 -- 172.21.15.38:0/3541347897 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7f053006d080 con 0x560ad9572270 2024-09-21T03:37:16.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.791+0000 7f054c260740 10 client.6384 fetch_fsmap learned FSMap version 59 2024-09-21T03:37:16.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.791+0000 7f054c260740 1 -- 172.21.15.38:0/3541347897 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=2+}) -- 0x560ad995fae0 con 0x560ad9572270 2024-09-21T03:37:16.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.791+0000 7f051dffb640 1 -- 172.21.15.38:0/3541347897 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f053802c420 con 0x560ad9572270 2024-09-21T03:37:16.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.791+0000 7f051dffb640 1 -- 172.21.15.38:0/3541347897 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 59) ==== 3502+0+0 (secure 0 0 0) 0x7f0530082620 con 0x560ad9572270 2024-09-21T03:37:16.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.791+0000 7f054c260740 10 client.6384 fetch_fsmap finished waiting for FSMap version 59 2024-09-21T03:37:16.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.791+0000 7f054c260740 10 client.6384 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-21T03:37:16.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.791+0000 7f054c260740 20 client.6384 populate_metadata read hostname 'smithi038' 2024-09-21T03:37:16.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.791+0000 7f054c260740 1 --2- 172.21.15.38:0/3541347897 >> [v2:172.21.15.195:6834/2536177898,v1:172.21.15.195:6836/2536177898] conn(0x560ad9960c30 0x560ad9961040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:16.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.791+0000 7f054c260740 4 client.6384 mds_command: new command op to 5731 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T03:37:16.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.791+0000 7f054c260740 1 -- 172.21.15.38:0/3541347897 --> [v2:172.21.15.195:6834/2536177898,v1:172.21.15.195:6836/2536177898] -- command(tid 0: {"prefix": "status"}) -- 0x560ad988a220 con 0x560ad9960c30 2024-09-21T03:37:16.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.792+0000 7f053ed9d640 1 --2- 172.21.15.38:0/3541347897 >> [v2:172.21.15.195:6834/2536177898,v1:172.21.15.195:6836/2536177898] conn(0x560ad9960c30 0x560ad9961040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:16.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.792+0000 7f051dffb640 10 client.6384 ms_handle_connect on v2:172.21.15.195:6834/2536177898 2024-09-21T03:37:16.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.792+0000 7f053ed9d640 1 --2- 172.21.15.38:0/3541347897 >> [v2:172.21.15.195:6834/2536177898,v1:172.21.15.195:6836/2536177898] conn(0x560ad9960c30 0x560ad9961040 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8beea733928e8dd server_cookie=f4ef379b369e72d4 in_seq=0 out_seq=0 2024-09-21T03:37:16.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.792+0000 7f051dffb640 1 -- 172.21.15.38:0/3541347897 <== mds.0 v2:172.21.15.195:6834/2536177898 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x560ad988a220 con 0x560ad9960c30 2024-09-21T03:37:16.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.792+0000 7f051dffb640 10 client.6384 handle_command_reply: tid=0 2024-09-21T03:37:16.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.792+0000 7f054c260740 1 client.6384 shutdown 2024-09-21T03:37:16.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.792+0000 7f054c260740 1 -- 172.21.15.38:0/3541347897 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560ad9572270 msgr2=0x560ad9913a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:16.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.792+0000 7f054c260740 1 --2- 172.21.15.38:0/3541347897 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560ad9572270 0x560ad9913a90 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f0530034040 tx=0x7f053008ca40 comp rx=0 tx=0).stop 2024-09-21T03:37:16.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.792+0000 7f054c260740 1 -- 172.21.15.38:0/3541347897 shutdown_connections 2024-09-21T03:37:16.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.792+0000 7f054c260740 1 --2- 172.21.15.38:0/3541347897 >> [v2:172.21.15.195:6834/2536177898,v1:172.21.15.195:6836/2536177898] conn(0x560ad9960c30 0x560ad9961040 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:16.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.792+0000 7f054c260740 1 --2- 172.21.15.38:0/3541347897 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560ad9572270 0x560ad9913a90 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:16.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.792+0000 7f054c260740 1 --2- 172.21.15.38:0/3541347897 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560ad9551590 0x560ad9913550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:16.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.792+0000 7f051dffb640 0 client.6384 ms_handle_reset on v2:172.21.15.195:6834/2536177898 2024-09-21T03:37:16.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.792+0000 7f054c260740 1 --2- 172.21.15.38:0/3541347897 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560ad95493b0 0x560ad99df420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:16.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.792+0000 7f054c260740 1 -- 172.21.15.38:0/3541347897 >> 172.21.15.38:0/3541347897 conn(0x560ad95663c0 msgr2=0x560ad990f960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:16.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.792+0000 7f054c260740 1 -- 172.21.15.38:0/3541347897 shutdown_connections 2024-09-21T03:37:16.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.792+0000 7f054c260740 1 -- 172.21.15.38:0/3541347897 wait complete. 2024-09-21T03:37:16.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.793+0000 7f054c260740 20 client.6384 trim_cache size 0 max 16384 2024-09-21T03:37:16.800 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-09-21T03:37:16.800 INFO:teuthology.orchestra.run.smithi038.stdout: "cluster_fsid": "9d50d66f-656b-4fe2-8e3b-fe53b79c4d43", 2024-09-21T03:37:16.800 INFO:teuthology.orchestra.run.smithi038.stdout: "whoami": 0, 2024-09-21T03:37:16.800 INFO:teuthology.orchestra.run.smithi038.stdout: "id": 5731, 2024-09-21T03:37:16.800 INFO:teuthology.orchestra.run.smithi038.stdout: "want_state": "up:active", 2024-09-21T03:37:16.800 INFO:teuthology.orchestra.run.smithi038.stdout: "state": "up:active", 2024-09-21T03:37:16.800 INFO:teuthology.orchestra.run.smithi038.stdout: "fs_name": "cephfs", 2024-09-21T03:37:16.800 INFO:teuthology.orchestra.run.smithi038.stdout: "rank_uptime": 4.7441429660000001, 2024-09-21T03:37:16.800 INFO:teuthology.orchestra.run.smithi038.stdout: "mdsmap_epoch": 59, 2024-09-21T03:37:16.800 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch": 70, 2024-09-21T03:37:16.800 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch_barrier": 69, 2024-09-21T03:37:16.801 INFO:teuthology.orchestra.run.smithi038.stdout: "uptime": 59.359439424000001 2024-09-21T03:37:16.801 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-09-21T03:37:16.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.793+0000 7f0545215640 1 -- 172.21.15.38:0/576049833 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0520078b90 msgr2=0x7f052007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:16.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.793+0000 7f0545215640 1 --2- 172.21.15.38:0/576049833 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0520078b90 0x7f052007b0a0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f053800b450 tx=0x7f053800d040 comp rx=0 tx=0).stop 2024-09-21T03:37:16.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.793+0000 7f0545215640 1 -- 172.21.15.38:0/576049833 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f054015ca90 msgr2=0x7f05401e8980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:16.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.793+0000 7f0545215640 1 --2- 172.21.15.38:0/576049833 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f054015ca90 0x7f05401e8980 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f0530006010 tx=0x7f0530003960 comp rx=0 tx=0).stop 2024-09-21T03:37:16.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.793+0000 7f0545215640 1 -- 172.21.15.38:0/576049833 shutdown_connections 2024-09-21T03:37:16.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.793+0000 7f0545215640 1 --2- 172.21.15.38:0/576049833 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f054015ca90 0x7f05401e8980 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:16.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.793+0000 7f0545215640 1 --2- 172.21.15.38:0/576049833 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0520078b90 0x7f052007b0a0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:16.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.793+0000 7f0545215640 1 --2- 172.21.15.38:0/576049833 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f05400d0bb0 0x7f0540147d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:16.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.793+0000 7f0545215640 1 --2- 172.21.15.38:0/576049833 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f05400ccc40 0x7f054006def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:16.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.793+0000 7f0545215640 1 -- 172.21.15.38:0/576049833 >> 172.21.15.38:0/576049833 conn(0x7f05400b46e0 msgr2=0x7f0540070100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:16.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.793+0000 7f0545215640 1 -- 172.21.15.38:0/576049833 shutdown_connections 2024-09-21T03:37:16.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:16.793+0000 7f0545215640 1 -- 172.21.15.38:0/576049833 wait complete. 2024-09-21T03:37:19.038 INFO:tasks.ceph.mds.b.smithi195.stderr:2024-09-21T03:37:19.036+0000 7f225610a640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T03:37:29.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:29.187+0000 7f98aeffd640 1 --2- 172.21.15.38:0/117103957 >> [v2:172.21.15.38:6832/1058376326,v1:172.21.15.38:6834/1058376326] conn(0x55c451e5f030 0x55c451e5f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:29.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:29.187+0000 7f98aeffd640 1 -- 172.21.15.38:0/117103957 >> [v2:172.21.15.38:6832/1058376326,v1:172.21.15.38:6834/1058376326] conn(0x55c451e5f030 msgr2=0x55c451e5f440 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 18 2024-09-21T03:37:29.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:29.187+0000 7f98aeffd640 1 -- 172.21.15.38:0/117103957 >> [v2:172.21.15.38:6832/1058376326,v1:172.21.15.38:6834/1058376326] conn(0x55c451e5f030 msgr2=0x55c451e5f440 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-21T03:37:29.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:29.187+0000 7f98aeffd640 1 --2- 172.21.15.38:0/117103957 >> [v2:172.21.15.38:6832/1058376326,v1:172.21.15.38:6834/1058376326] conn(0x55c451e5f030 0x55c451e5f440 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-21T03:37:29.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:29.187+0000 7f98aeffd640 1 --2- 172.21.15.38:0/117103957 >> [v2:172.21.15.38:6832/1058376326,v1:172.21.15.38:6834/1058376326] conn(0x55c451e5f030 0x55c451e5f440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-09-21T03:37:30.471 DEBUG:teuthology.orchestra.run.smithi038:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T03:37:30.475 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T03:37:44.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:44.190+0000 7f98aeffd640 1 --2- 172.21.15.38:0/117103957 >> [v2:172.21.15.38:6832/1058376326,v1:172.21.15.38:6834/1058376326] conn(0x55c451e5f030 0x55c451e5f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:44.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:44.191+0000 7f98aeffd640 1 -- 172.21.15.38:0/117103957 >> [v2:172.21.15.38:6832/1058376326,v1:172.21.15.38:6834/1058376326] conn(0x55c451e5f030 msgr2=0x55c451e5f440 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 18 2024-09-21T03:37:44.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:44.191+0000 7f98aeffd640 1 -- 172.21.15.38:0/117103957 >> [v2:172.21.15.38:6832/1058376326,v1:172.21.15.38:6834/1058376326] conn(0x55c451e5f030 msgr2=0x55c451e5f440 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-21T03:37:44.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:44.191+0000 7f98aeffd640 1 --2- 172.21.15.38:0/117103957 >> [v2:172.21.15.38:6832/1058376326,v1:172.21.15.38:6834/1058376326] conn(0x55c451e5f030 0x55c451e5f440 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-21T03:37:44.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:44.191+0000 7f98aeffd640 1 --2- 172.21.15.38:0/117103957 >> [v2:172.21.15.38:6832/1058376326,v1:172.21.15.38:6834/1058376326] conn(0x55c451e5f030 0x55c451e5f440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-09-21T03:37:59.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:59.199+0000 7f98aeffd640 1 --2- 172.21.15.38:0/117103957 >> [v2:172.21.15.38:6832/1058376326,v1:172.21.15.38:6834/1058376326] conn(0x55c451e5f030 0x55c451e5f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:59.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:59.199+0000 7f98aeffd640 1 -- 172.21.15.38:0/117103957 >> [v2:172.21.15.38:6832/1058376326,v1:172.21.15.38:6834/1058376326] conn(0x55c451e5f030 msgr2=0x55c451e5f440 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 18 2024-09-21T03:37:59.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:59.199+0000 7f98aeffd640 1 -- 172.21.15.38:0/117103957 >> [v2:172.21.15.38:6832/1058376326,v1:172.21.15.38:6834/1058376326] conn(0x55c451e5f030 msgr2=0x55c451e5f440 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-21T03:37:59.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:59.199+0000 7f98aeffd640 1 --2- 172.21.15.38:0/117103957 >> [v2:172.21.15.38:6832/1058376326,v1:172.21.15.38:6834/1058376326] conn(0x55c451e5f030 0x55c451e5f440 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-21T03:37:59.202 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:37:59.199+0000 7f98aeffd640 1 --2- 172.21.15.38:0/117103957 >> [v2:172.21.15.38:6832/1058376326,v1:172.21.15.38:6834/1058376326] conn(0x55c451e5f030 0x55c451e5f440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-09-21T03:38:00.618 DEBUG:teuthology.orchestra.run.smithi038:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T03:38:00.621 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T03:38:14.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:14.201+0000 7f98aeffd640 1 --2- 172.21.15.38:0/117103957 >> [v2:172.21.15.38:6832/1058376326,v1:172.21.15.38:6834/1058376326] conn(0x55c451e5f030 0x55c451e5f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:14.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:14.201+0000 7f98aeffd640 1 -- 172.21.15.38:0/117103957 >> [v2:172.21.15.38:6832/1058376326,v1:172.21.15.38:6834/1058376326] conn(0x55c451e5f030 msgr2=0x55c451e5f440 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 18 2024-09-21T03:38:14.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:14.201+0000 7f98aeffd640 1 -- 172.21.15.38:0/117103957 >> [v2:172.21.15.38:6832/1058376326,v1:172.21.15.38:6834/1058376326] conn(0x55c451e5f030 msgr2=0x55c451e5f440 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-21T03:38:14.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:14.201+0000 7f98aeffd640 1 --2- 172.21.15.38:0/117103957 >> [v2:172.21.15.38:6832/1058376326,v1:172.21.15.38:6834/1058376326] conn(0x55c451e5f030 0x55c451e5f440 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-21T03:38:14.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:14.201+0000 7f98aeffd640 1 --2- 172.21.15.38:0/117103957 >> [v2:172.21.15.38:6832/1058376326,v1:172.21.15.38:6834/1058376326] conn(0x55c451e5f030 0x55c451e5f440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-09-21T03:38:29.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:29.209+0000 7f98aeffd640 1 --2- 172.21.15.38:0/117103957 >> [v2:172.21.15.38:6832/1058376326,v1:172.21.15.38:6834/1058376326] conn(0x55c451e5f030 0x55c451e5f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:29.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:29.209+0000 7f98aeffd640 1 -- 172.21.15.38:0/117103957 >> [v2:172.21.15.38:6832/1058376326,v1:172.21.15.38:6834/1058376326] conn(0x55c451e5f030 msgr2=0x55c451e5f440 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 18 2024-09-21T03:38:29.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:29.209+0000 7f98aeffd640 1 -- 172.21.15.38:0/117103957 >> [v2:172.21.15.38:6832/1058376326,v1:172.21.15.38:6834/1058376326] conn(0x55c451e5f030 msgr2=0x55c451e5f440 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-21T03:38:29.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:29.209+0000 7f98aeffd640 1 --2- 172.21.15.38:0/117103957 >> [v2:172.21.15.38:6832/1058376326,v1:172.21.15.38:6834/1058376326] conn(0x55c451e5f030 0x55c451e5f440 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-21T03:38:29.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:29.209+0000 7f98aeffd640 1 --2- 172.21.15.38:0/117103957 >> [v2:172.21.15.38:6832/1058376326,v1:172.21.15.38:6834/1058376326] conn(0x55c451e5f030 0x55c451e5f440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-09-21T03:38:30.699 DEBUG:teuthology.orchestra.run.smithi038:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T03:38:30.701 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T03:38:44.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:44.212+0000 7f98aeffd640 1 --2- 172.21.15.38:0/117103957 >> [v2:172.21.15.38:6832/1058376326,v1:172.21.15.38:6834/1058376326] conn(0x55c451e5f030 0x55c451e5f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:44.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:44.212+0000 7f98aeffd640 1 -- 172.21.15.38:0/117103957 >> [v2:172.21.15.38:6832/1058376326,v1:172.21.15.38:6834/1058376326] conn(0x55c451e5f030 msgr2=0x55c451e5f440 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 18 2024-09-21T03:38:44.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:44.212+0000 7f98aeffd640 1 -- 172.21.15.38:0/117103957 >> [v2:172.21.15.38:6832/1058376326,v1:172.21.15.38:6834/1058376326] conn(0x55c451e5f030 msgr2=0x55c451e5f440 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-21T03:38:44.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:44.212+0000 7f98aeffd640 1 --2- 172.21.15.38:0/117103957 >> [v2:172.21.15.38:6832/1058376326,v1:172.21.15.38:6834/1058376326] conn(0x55c451e5f030 0x55c451e5f440 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-21T03:38:44.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:44.212+0000 7f98aeffd640 1 --2- 172.21.15.38:0/117103957 >> [v2:172.21.15.38:6832/1058376326,v1:172.21.15.38:6834/1058376326] conn(0x55c451e5f030 0x55c451e5f440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-09-21T03:38:48.548 DEBUG:teuthology.orchestra.run:got remote process result: 124 2024-09-21T03:38:48.548 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T03:38:48.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:48.677+0000 7f3b39ecc640 1 -- 172.21.15.38:0/2045179658 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3b34162f00 msgr2=0x7f3b340d24d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:48.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:48.677+0000 7f3b39ecc640 1 --2- 172.21.15.38:0/2045179658 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3b34162f00 0x7f3b340d24d0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f3b24005fd0 tx=0x7f3b2402a5d0 comp rx=0 tx=0).stop 2024-09-21T03:38:48.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:48.677+0000 7f3b39ecc640 1 -- 172.21.15.38:0/2045179658 shutdown_connections 2024-09-21T03:38:48.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:48.677+0000 7f3b39ecc640 1 --2- 172.21.15.38:0/2045179658 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3b340b2e40 0x7f3b340b3250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:48.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:48.677+0000 7f3b39ecc640 1 --2- 172.21.15.38:0/2045179658 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3b34162f00 0x7f3b340d24d0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:48.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:48.677+0000 7f3b39ecc640 1 -- 172.21.15.38:0/2045179658 >> 172.21.15.38:0/2045179658 conn(0x7f3b340b6b20 msgr2=0x7f3b340d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:48.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:48.677+0000 7f3b39ecc640 1 -- 172.21.15.38:0/2045179658 shutdown_connections 2024-09-21T03:38:48.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:48.677+0000 7f3b39ecc640 1 -- 172.21.15.38:0/2045179658 wait complete. 2024-09-21T03:38:48.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:48.679+0000 7f3b39ecc640 1 Processor -- start 2024-09-21T03:38:48.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:48.679+0000 7f3b39ecc640 1 -- start start 2024-09-21T03:38:48.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:48.680+0000 7f3b39ecc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3b340b2e40 0x7f3b3415cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:48.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:48.680+0000 7f3b39ecc640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3b34162280 0x7f3b341f4340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:48.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:48.680+0000 7f3b39ecc640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3b341f4880 0x7f3b341f8220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:48.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:48.680+0000 7f3b39ecc640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f3b3415e4b0 con 0x7f3b340b2e40 2024-09-21T03:38:48.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:48.680+0000 7f3b39ecc640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f3b340b3a20 con 0x7f3b341f4880 2024-09-21T03:38:48.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:48.680+0000 7f3b39ecc640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f3b3415e630 con 0x7f3b34162280 2024-09-21T03:38:48.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:48.680+0000 7f3b337fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3b340b2e40 0x7f3b3415cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:48.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:48.680+0000 7f3b337fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3b340b2e40 0x7f3b3415cf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48270/0 (socket says 172.21.15.38:48270) 2024-09-21T03:38:48.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:48.680+0000 7f3b337fe640 1 -- 172.21.15.38:0/1972693999 learned_addr learned my addr 172.21.15.38:0/1972693999 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:38:48.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:48.680+0000 7f3b33fff640 1 --2- 172.21.15.38:0/1972693999 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3b341f4880 0x7f3b341f8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:48.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:48.680+0000 7f3b32ffd640 1 --2- 172.21.15.38:0/1972693999 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3b34162280 0x7f3b341f4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:48.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:48.681+0000 7f3b337fe640 1 -- 172.21.15.38:0/1972693999 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3b34162280 msgr2=0x7f3b341f4340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:48.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:48.681+0000 7f3b337fe640 1 --2- 172.21.15.38:0/1972693999 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3b34162280 0x7f3b341f4340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:48.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:48.681+0000 7f3b337fe640 1 -- 172.21.15.38:0/1972693999 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3b341f4880 msgr2=0x7f3b341f8220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:48.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:48.681+0000 7f3b337fe640 1 --2- 172.21.15.38:0/1972693999 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3b341f4880 0x7f3b341f8220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:48.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:48.681+0000 7f3b337fe640 1 -- 172.21.15.38:0/1972693999 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b341f8fa0 con 0x7f3b340b2e40 2024-09-21T03:38:48.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:48.681+0000 7f3b337fe640 1 --2- 172.21.15.38:0/1972693999 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3b340b2e40 0x7f3b3415cf80 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f3b24034040 tx=0x7f3b240038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:48.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:48.681+0000 7f3b1bfff640 1 -- 172.21.15.38:0/1972693999 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b2403e030 con 0x7f3b340b2e40 2024-09-21T03:38:48.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:48.681+0000 7f3b1bfff640 1 -- 172.21.15.38:0/1972693999 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3b24003ca0 con 0x7f3b340b2e40 2024-09-21T03:38:48.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:48.681+0000 7f3b1bfff640 1 -- 172.21.15.38:0/1972693999 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b24035040 con 0x7f3b340b2e40 2024-09-21T03:38:48.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:48.681+0000 7f3b39ecc640 1 -- 172.21.15.38:0/1972693999 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b3414c910 con 0x7f3b340b2e40 2024-09-21T03:38:48.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:48.681+0000 7f3b39ecc640 1 -- 172.21.15.38:0/1972693999 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b3414e470 con 0x7f3b340b2e40 2024-09-21T03:38:48.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:48.682+0000 7f3b39ecc640 1 -- 172.21.15.38:0/1972693999 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b3414c700 con 0x7f3b340b2e40 2024-09-21T03:38:48.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:48.683+0000 7f3b1bfff640 1 -- 172.21.15.38:0/1972693999 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f3b24041030 con 0x7f3b340b2e40 2024-09-21T03:38:48.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:48.683+0000 7f3b1bfff640 1 --2- 172.21.15.38:0/1972693999 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3b20078b90 0x7f3b2007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:48.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:48.684+0000 7f3b1bfff640 1 -- 172.21.15.38:0/1972693999 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 6476+0+0 (secure 0 0 0) 0x7f3b240bb470 con 0x7f3b340b2e40 2024-09-21T03:38:48.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:48.684+0000 7f3b32ffd640 1 --2- 172.21.15.38:0/1972693999 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3b20078b90 0x7f3b2007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:48.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:48.684+0000 7f3b32ffd640 1 --2- 172.21.15.38:0/1972693999 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3b20078b90 0x7f3b2007b0a0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f3b3415ded0 tx=0x7f3b28009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:48.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:48.688+0000 7f3b1bfff640 1 -- 172.21.15.38:0/1972693999 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f3b24082ff0 con 0x7f3b340b2e40 2024-09-21T03:38:48.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:48.914+0000 7f3b39ecc640 1 -- 172.21.15.38:0/1972693999 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f3b34162f00 con 0x7f3b340b2e40 2024-09-21T03:38:48.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:48.914+0000 7f3b1bfff640 1 -- 172.21.15.38:0/1972693999 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v59) ==== 53+0+73 (secure 0 0 0) 0x7f3b24087a20 con 0x7f3b340b2e40 2024-09-21T03:38:48.916 INFO:teuthology.orchestra.run.smithi038.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T03:38:48.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:48.916+0000 7f3b39ecc640 1 -- 172.21.15.38:0/1972693999 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3b20078b90 msgr2=0x7f3b2007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:48.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:48.916+0000 7f3b39ecc640 1 --2- 172.21.15.38:0/1972693999 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3b20078b90 0x7f3b2007b0a0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f3b3415ded0 tx=0x7f3b28009040 comp rx=0 tx=0).stop 2024-09-21T03:38:48.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:48.916+0000 7f3b39ecc640 1 -- 172.21.15.38:0/1972693999 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3b340b2e40 msgr2=0x7f3b3415cf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:48.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:48.916+0000 7f3b39ecc640 1 --2- 172.21.15.38:0/1972693999 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3b340b2e40 0x7f3b3415cf80 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f3b24034040 tx=0x7f3b240038a0 comp rx=0 tx=0).stop 2024-09-21T03:38:48.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:48.916+0000 7f3b39ecc640 1 -- 172.21.15.38:0/1972693999 shutdown_connections 2024-09-21T03:38:48.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:48.916+0000 7f3b39ecc640 1 --2- 172.21.15.38:0/1972693999 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3b341f4880 0x7f3b341f8220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:48.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:48.916+0000 7f3b39ecc640 1 --2- 172.21.15.38:0/1972693999 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3b20078b90 0x7f3b2007b0a0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:48.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:48.916+0000 7f3b39ecc640 1 --2- 172.21.15.38:0/1972693999 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3b34162280 0x7f3b341f4340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:48.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:48.916+0000 7f3b39ecc640 1 --2- 172.21.15.38:0/1972693999 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3b340b2e40 0x7f3b3415cf80 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:48.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:48.916+0000 7f3b39ecc640 1 -- 172.21.15.38:0/1972693999 >> 172.21.15.38:0/1972693999 conn(0x7f3b340b6b20 msgr2=0x7f3b3406db70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:48.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:48.917+0000 7f3b39ecc640 1 -- 172.21.15.38:0/1972693999 shutdown_connections 2024-09-21T03:38:48.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:48.917+0000 7f3b39ecc640 1 -- 172.21.15.38:0/1972693999 wait complete. 2024-09-21T03:38:48.930 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-21T03:38:48.930 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T03:38:48.930 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi038.front.sepia.ceph.com 2024-09-21T03:38:48.930 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-21T03:38:48.930 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T03:38:48.930 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-21T03:38:48.930 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-21T03:38:48.930 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T03:38:48.930 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T03:38:48.931 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:38:48.931 DEBUG:teuthology.orchestra.run.smithi038:> ip addr 2024-09-21T03:38:48.947 INFO:teuthology.orchestra.run.smithi038.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T03:38:48.947 INFO:teuthology.orchestra.run.smithi038.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T03:38:48.947 INFO:teuthology.orchestra.run.smithi038.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T03:38:48.947 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:38:48.947 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 ::1/128 scope host 2024-09-21T03:38:48.947 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:38:48.948 INFO:teuthology.orchestra.run.smithi038.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T03:38:48.948 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:6b:fd:a2 brd ff:ff:ff:ff:ff:ff 2024-09-21T03:38:48.948 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp7s0f0 2024-09-21T03:38:48.948 INFO:teuthology.orchestra.run.smithi038.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T03:38:48.948 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:6b:fd:a3 brd ff:ff:ff:ff:ff:ff 2024-09-21T03:38:48.948 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp7s0f1 2024-09-21T03:38:48.948 INFO:teuthology.orchestra.run.smithi038.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T03:38:48.948 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:bd:15:fc brd ff:ff:ff:ff:ff:ff 2024-09-21T03:38:48.948 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp1s0f0 2024-09-21T03:38:48.948 INFO:teuthology.orchestra.run.smithi038.stdout: inet 172.21.15.38/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-21T03:38:48.948 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft 42480sec preferred_lft 42480sec 2024-09-21T03:38:48.948 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 fe80::ec4:7aff:febd:15fc/64 scope link 2024-09-21T03:38:48.948 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:38:48.949 INFO:teuthology.orchestra.run.smithi038.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T03:38:48.949 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:bd:15:fd brd ff:ff:ff:ff:ff:ff 2024-09-21T03:38:48.949 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp1s0f1 2024-09-21T03:38:48.949 INFO:teuthology.orchestra.run.smithi038.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T03:38:48.949 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 92:55:c7:97:6f:b5 brd ff:ff:ff:ff:ff:ff 2024-09-21T03:38:48.949 INFO:teuthology.orchestra.run.smithi038.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-21T03:38:48.949 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:38:48.949 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 fe80::c49f:c1ff:fe21:b12a/64 scope link 2024-09-21T03:38:48.949 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:38:48.949 INFO:teuthology.orchestra.run.smithi038.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T03:38:48.949 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether a6:3a:83:ad:db:d5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T03:38:48.949 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 fe80::a43a:83ff:fead:dbd5/64 scope link 2024-09-21T03:38:48.949 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:38:48.950 INFO:teuthology.orchestra.run.smithi038.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T03:38:48.950 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 92:55:c7:97:6f:b5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T03:38:48.950 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 fe80::9055:c7ff:fe97:6fb5/64 scope link 2024-09-21T03:38:48.950 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:38:48.950 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:38:48.950 DEBUG:teuthology.orchestra.run.smithi038:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T03:38:49.012 INFO:teuthology.orchestra.run.smithi038.stdout:1 2024-09-21T03:38:49.013 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:38:49.014 DEBUG:teuthology.orchestra.run.smithi038:> route 2024-09-21T03:38:49.070 INFO:teuthology.orchestra.run.smithi038.stdout:Kernel IP routing table 2024-09-21T03:38:49.070 INFO:teuthology.orchestra.run.smithi038.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T03:38:49.070 INFO:teuthology.orchestra.run.smithi038.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-21T03:38:49.070 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-21T03:38:49.070 INFO:teuthology.orchestra.run.smithi038.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T03:38:49.071 DEBUG:teuthology.orchestra.run.smithi038:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T03:38:49.071 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-09-21T03:38:49.071 DEBUG:teuthology.orchestra.run.smithi038:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-21T03:38:49.071 DEBUG:teuthology.orchestra.run.smithi038:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-21T03:38:49.071 DEBUG:teuthology.orchestra.run.smithi038:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-21T03:38:49.071 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-09-21T03:38:49.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T03:38:49.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:38:49.199 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:38:49.199 DEBUG:teuthology.orchestra.run.smithi038:> ip netns list 2024-09-21T03:38:49.215 INFO:teuthology.orchestra.run.smithi038.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-21T03:38:49.215 INFO:teuthology.orchestra.run.smithi038.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-21T03:38:49.215 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T03:38:49.215 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:38:49.216 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip link set brx.0 up 2024-09-21T03:38:49.281 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T03:38:49.281 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-21T03:38:49.338 INFO:teuthology.orchestra.run.smithi038.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-21T03:38:49.338 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T03:38:49.338 DEBUG:teuthology.orchestra.run.smithi038:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-21T03:38:49.394 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-21T03:38:49.394 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:38:49.394 DEBUG:teuthology.orchestra.run.smithi038:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T03:38:49.490 INFO:teuthology.orchestra.run.smithi038.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T03:38:49.490 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T03:38:49.490 INFO:teuthology.orchestra.run.smithi038.stdout:invalid new device string format 2024-09-21T03:38:49.501 INFO:teuthology.orchestra.run.smithi038.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T03:38:49.501 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T03:38:49.501 INFO:teuthology.orchestra.run.smithi038.stdout:invalid new device string format 2024-09-21T03:38:49.501 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: resolved to: "172.21.15.38:3300,172.21.15.195:3300,172.21.15.195:3301" 2024-09-21T03:38:49.501 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T03:38:49.519 INFO:teuthology.orchestra.run.smithi038.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T03:38:49.520 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T03:38:49.520 INFO:teuthology.orchestra.run.smithi038.stdout:invalid new device string format 2024-09-21T03:38:49.520 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: resolved to: "172.21.15.38:3300,172.21.15.195:3300,172.21.15.195:3301" 2024-09-21T03:38:49.520 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T03:38:49.520 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: resolved to: "172.21.15.38:6789,172.21.15.195:6789,172.21.15.195:6790" 2024-09-21T03:38:49.520 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.38:6789,172.21.15.195:6789,172.21.15.195:6790:/ 2024-09-21T03:38:49.520 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=9d50d66f-656b-4fe2-8e3b-fe53b79c4d43" will pass to kernel 2024-09-21T03:38:49.520 INFO:teuthology.orchestra.run.smithi038.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-21T03:38:49.520 INFO:teuthology.orchestra.run.smithi038.stdout: You just mounted a file system that supports labels which does not 2024-09-21T03:38:49.521 INFO:teuthology.orchestra.run.smithi038.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-21T03:38:49.521 INFO:teuthology.orchestra.run.smithi038.stdout: applications will generate AVC messages and not be allowed access to 2024-09-21T03:38:49.521 INFO:teuthology.orchestra.run.smithi038.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-21T03:38:49.522 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T03:38:49.522 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:38:49.522 DEBUG:teuthology.orchestra.run.smithi038:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-21T03:38:49.547 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T03:38:49.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T03:38:49.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:38:49.647 INFO:teuthology.orchestra.run.smithi038.stdout:client6393 2024-09-21T03:38:49.647 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-21T03:38:49.647 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:38:49.663 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat a/ && find a/') 2024-09-21T03:38:49.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T03:38:49.755 INFO:teuthology.orchestra.run.smithi038.stdout: File: a/ 2024-09-21T03:38:49.755 INFO:teuthology.orchestra.run.smithi038.stdout: Size: 2 Blocks: 0 IO Block: 65536 directory 2024-09-21T03:38:49.755 INFO:teuthology.orchestra.run.smithi038.stdout:Device: 29h/41d Inode: 1099511627776 Links: 4 2024-09-21T03:38:49.755 INFO:teuthology.orchestra.run.smithi038.stdout:Access: (0755/drwxr-xr-x) Uid: ( 1000/ ubuntu) Gid: ( 1333/ ubuntu) 2024-09-21T03:38:49.755 INFO:teuthology.orchestra.run.smithi038.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-09-21T03:38:49.755 INFO:teuthology.orchestra.run.smithi038.stdout:Access: 2024-09-21 03:36:33.619982546 +0000 2024-09-21T03:38:49.755 INFO:teuthology.orchestra.run.smithi038.stdout:Modify: 2024-09-21 03:36:33.624982452 +0000 2024-09-21T03:38:49.756 INFO:teuthology.orchestra.run.smithi038.stdout:Change: 2024-09-21 03:36:33.624982452 +0000 2024-09-21T03:38:49.756 INFO:teuthology.orchestra.run.smithi038.stdout: Birth: 2024-09-21 03:36:33.619982546 +0000 2024-09-21T03:38:49.761 INFO:teuthology.orchestra.run.smithi038.stdout:a/ 2024-09-21T03:38:49.761 INFO:teuthology.orchestra.run.smithi038.stdout:a/c 2024-09-21T03:38:49.761 INFO:teuthology.orchestra.run.smithi038.stdout:a/c/d 2024-09-21T03:38:49.761 INFO:teuthology.orchestra.run.smithi038.stdout:a/b 2024-09-21T03:38:49.762 INFO:teuthology.orchestra.run.smithi038.stdout:a/b/d 2024-09-21T03:38:49.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:38:49.767 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.4:0 flush journal 2024-09-21T03:38:49.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.884+0000 7fc0b552b640 1 -- 172.21.15.38:0/2293933526 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc0b015fc50 msgr2=0x7fc0b006ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:49.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.884+0000 7fc0b552b640 1 --2- 172.21.15.38:0/2293933526 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc0b015fc50 0x7fc0b006ddf0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fc0a0005fd0 tx=0x7fc0a002aa00 comp rx=0 tx=0).stop 2024-09-21T03:38:49.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.884+0000 7fc0b552b640 1 -- 172.21.15.38:0/2293933526 shutdown_connections 2024-09-21T03:38:49.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.884+0000 7fc0b552b640 1 --2- 172.21.15.38:0/2293933526 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc0b00cf9a0 0x7fc0b0072d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:49.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.884+0000 7fc0b552b640 1 --2- 172.21.15.38:0/2293933526 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc0b015fc50 0x7fc0b006ddf0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:49.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.884+0000 7fc0b552b640 1 -- 172.21.15.38:0/2293933526 >> 172.21.15.38:0/2293933526 conn(0x7fc0b00b1b90 msgr2=0x7fc0b00b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:49.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.884+0000 7fc0b552b640 1 -- 172.21.15.38:0/2293933526 shutdown_connections 2024-09-21T03:38:49.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.884+0000 7fc0b552b640 1 -- 172.21.15.38:0/2293933526 wait complete. 2024-09-21T03:38:49.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.886+0000 7fc0b552b640 1 Processor -- start 2024-09-21T03:38:49.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.886+0000 7fc0b552b640 1 -- start start 2024-09-21T03:38:49.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.887+0000 7fc0b552b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc0b006e330 0x7fc0b01f1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:49.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.887+0000 7fc0b552b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc0b00cf9a0 0x7fc0b015cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:49.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.887+0000 7fc0b552b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc0b015fc50 0x7fc0b015d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:49.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.887+0000 7fc0b552b640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fc0b0073450 con 0x7fc0b006e330 2024-09-21T03:38:49.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.887+0000 7fc0b552b640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fc0b00732d0 con 0x7fc0b00cf9a0 2024-09-21T03:38:49.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.887+0000 7fc0b552b640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fc0b00735d0 con 0x7fc0b015fc50 2024-09-21T03:38:49.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.887+0000 7fc0aeffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc0b006e330 0x7fc0b01f1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:49.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.887+0000 7fc0aeffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc0b006e330 0x7fc0b01f1ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48292/0 (socket says 172.21.15.38:48292) 2024-09-21T03:38:49.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.887+0000 7fc0aeffd640 1 -- 172.21.15.38:0/3689336920 learned_addr learned my addr 172.21.15.38:0/3689336920 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:38:49.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.887+0000 7fc0ae7fc640 1 --2- 172.21.15.38:0/3689336920 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc0b00cf9a0 0x7fc0b015cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:49.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.887+0000 7fc0af7fe640 1 --2- 172.21.15.38:0/3689336920 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc0b015fc50 0x7fc0b015d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:49.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.888+0000 7fc0aeffd640 1 -- 172.21.15.38:0/3689336920 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc0b015fc50 msgr2=0x7fc0b015d1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:49.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.888+0000 7fc0aeffd640 1 --2- 172.21.15.38:0/3689336920 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc0b015fc50 0x7fc0b015d1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:49.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.888+0000 7fc0aeffd640 1 -- 172.21.15.38:0/3689336920 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc0b00cf9a0 msgr2=0x7fc0b015cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:49.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.888+0000 7fc0aeffd640 1 --2- 172.21.15.38:0/3689336920 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc0b00cf9a0 0x7fc0b015cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:49.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.888+0000 7fc0aeffd640 1 -- 172.21.15.38:0/3689336920 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0b00c6400 con 0x7fc0b006e330 2024-09-21T03:38:49.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.888+0000 7fc0aeffd640 1 --2- 172.21.15.38:0/3689336920 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc0b006e330 0x7fc0b01f1ef0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fc0a0000f80 tx=0x7fc0a0003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:49.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.888+0000 7fc0977fe640 1 -- 172.21.15.38:0/3689336920 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0a0046070 con 0x7fc0b006e330 2024-09-21T03:38:49.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.888+0000 7fc0977fe640 1 -- 172.21.15.38:0/3689336920 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc0a00045c0 con 0x7fc0b006e330 2024-09-21T03:38:49.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.888+0000 7fc0b552b640 1 -- 172.21.15.38:0/3689336920 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0b015c1a0 con 0x7fc0b006e330 2024-09-21T03:38:49.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.888+0000 7fc0977fe640 1 -- 172.21.15.38:0/3689336920 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0a0033070 con 0x7fc0b006e330 2024-09-21T03:38:49.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.888+0000 7fc0b552b640 1 -- 172.21.15.38:0/3689336920 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0b00c7170 con 0x7fc0b006e330 2024-09-21T03:38:49.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.890+0000 7fc0977fe640 1 -- 172.21.15.38:0/3689336920 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc0a0049070 con 0x7fc0b006e330 2024-09-21T03:38:49.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.890+0000 7fc0977fe640 1 --2- 172.21.15.38:0/3689336920 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc09c078b90 0x7fc09c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:49.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.890+0000 7fc0977fe640 1 -- 172.21.15.38:0/3689336920 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 6476+0+0 (secure 0 0 0) 0x7fc0a00bc130 con 0x7fc0b006e330 2024-09-21T03:38:49.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.891+0000 7fc0ae7fc640 1 --2- 172.21.15.38:0/3689336920 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc09c078b90 0x7fc09c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:49.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.891+0000 7fc0ae7fc640 1 --2- 172.21.15.38:0/3689336920 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc09c078b90 0x7fc09c07b0a0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fc0b015af50 tx=0x7fc0a4009060 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:49.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.891+0000 7fc0bc49a740 1 Processor -- start 2024-09-21T03:38:49.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.892+0000 7fc0bc49a740 1 -- start start 2024-09-21T03:38:49.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.893+0000 7fc0bc49a740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5612fde6c390 0x5612fe258a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:49.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.893+0000 7fc0bc49a740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5612fde434d0 0x5612fe22d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:49.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.893+0000 7fc0bc49a740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5612fde4b6b0 0x5612fe22dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:49.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.893+0000 7fc0bc49a740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5612fe186a60 con 0x5612fde6c390 2024-09-21T03:38:49.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.893+0000 7fc0bc49a740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5612fe175e90 con 0x5612fde434d0 2024-09-21T03:38:49.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.893+0000 7fc0bc49a740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x5612fe179ca0 con 0x5612fde4b6b0 2024-09-21T03:38:49.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.893+0000 7fc0aeffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5612fde6c390 0x5612fe258a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:49.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.893+0000 7fc0aeffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5612fde6c390 0x5612fe258a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48300/0 (socket says 172.21.15.38:48300) 2024-09-21T03:38:49.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.893+0000 7fc0aeffd640 1 -- 172.21.15.38:0/943005686 learned_addr learned my addr 172.21.15.38:0/943005686 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:38:49.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.893+0000 7fc0ae7fc640 1 --2- 172.21.15.38:0/943005686 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5612fde434d0 0x5612fe22d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:49.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.893+0000 7fc0af7fe640 1 --2- 172.21.15.38:0/943005686 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5612fde4b6b0 0x5612fe22dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:49.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.893+0000 7fc0aeffd640 1 -- 172.21.15.38:0/943005686 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5612fde4b6b0 msgr2=0x5612fe22dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:38:49.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.893+0000 7fc0aeffd640 1 --2- 172.21.15.38:0/943005686 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5612fde4b6b0 0x5612fe22dda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:49.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.893+0000 7fc0aeffd640 1 -- 172.21.15.38:0/943005686 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5612fde434d0 msgr2=0x5612fe22d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:38:49.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.893+0000 7fc0aeffd640 1 --2- 172.21.15.38:0/943005686 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5612fde434d0 0x5612fe22d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:49.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.893+0000 7fc0aeffd640 1 -- 172.21.15.38:0/943005686 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5612fdf4e7d0 con 0x5612fde6c390 2024-09-21T03:38:49.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.894+0000 7fc0aeffd640 1 --2- 172.21.15.38:0/943005686 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5612fde6c390 0x5612fe258a10 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fc0a00ba2c0 tx=0x7fc0a0083410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9ba8f24e730bf85 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:49.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.894+0000 7fc094ff9640 1 -- 172.21.15.38:0/943005686 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0a0088030 con 0x5612fde6c390 2024-09-21T03:38:49.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.894+0000 7fc094ff9640 1 -- 172.21.15.38:0/943005686 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc0a00886d0 con 0x5612fde6c390 2024-09-21T03:38:49.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.894+0000 7fc094ff9640 1 -- 172.21.15.38:0/943005686 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0a0088410 con 0x5612fde6c390 2024-09-21T03:38:49.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.894+0000 7fc0bc49a740 1 -- 172.21.15.38:0/943005686 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5612fde6c390 msgr2=0x5612fe258a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:49.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.894+0000 7fc0bc49a740 1 --2- 172.21.15.38:0/943005686 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5612fde6c390 0x5612fe258a10 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fc0a00ba2c0 tx=0x7fc0a0083410 comp rx=0 tx=0).stop 2024-09-21T03:38:49.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.895+0000 7fc0bc49a740 1 -- 172.21.15.38:0/943005686 shutdown_connections 2024-09-21T03:38:49.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.895+0000 7fc0bc49a740 1 --2- 172.21.15.38:0/943005686 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5612fde4b6b0 0x5612fe22dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:49.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.895+0000 7fc0bc49a740 1 --2- 172.21.15.38:0/943005686 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5612fde434d0 0x5612fe22d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:49.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.895+0000 7fc0bc49a740 1 --2- 172.21.15.38:0/943005686 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5612fde6c390 0x5612fe258a10 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:49.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.895+0000 7fc0bc49a740 1 -- 172.21.15.38:0/943005686 >> 172.21.15.38:0/943005686 conn(0x5612fde604e0 msgr2=0x5612fe213d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:49.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.895+0000 7fc0bc49a740 1 -- 172.21.15.38:0/943005686 shutdown_connections 2024-09-21T03:38:49.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.895+0000 7fc0bc49a740 1 -- 172.21.15.38:0/943005686 wait complete. 2024-09-21T03:38:49.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.896+0000 7fc0bc49a740 1 -- start start 2024-09-21T03:38:49.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.896+0000 7fc0bc49a740 1 Processor -- start 2024-09-21T03:38:49.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.897+0000 7fc0bc49a740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5612fde434d0 0x5612fe23e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:49.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.897+0000 7fc0bc49a740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5612fde4b6b0 0x5612fe24c020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:49.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.897+0000 7fc0bc49a740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5612fde6c390 0x5612fe23cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:49.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.897+0000 7fc0bc49a740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5612fdee6740 con 0x5612fde434d0 2024-09-21T03:38:49.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.897+0000 7fc0bc49a740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5612fe11b970 con 0x5612fde6c390 2024-09-21T03:38:49.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.897+0000 7fc0bc49a740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x5612fe175e90 con 0x5612fde4b6b0 2024-09-21T03:38:49.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.897+0000 7fc0bc49a740 10 client.0 _ll_register_callbacks cb 0x5612fe12a780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:38:49.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.897+0000 7fc0aeffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5612fde434d0 0x5612fe23e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:49.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.897+0000 7fc0aeffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5612fde434d0 0x5612fe23e020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48312/0 (socket says 172.21.15.38:48312) 2024-09-21T03:38:49.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.897+0000 7fc0aeffd640 1 -- 172.21.15.38:0/1579358651 learned_addr learned my addr 172.21.15.38:0/1579358651 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:38:49.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.897+0000 7fc0af7fe640 1 --2- 172.21.15.38:0/1579358651 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5612fde6c390 0x5612fe23cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:49.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.897+0000 7fc0ae7fc640 1 --2- 172.21.15.38:0/1579358651 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5612fde4b6b0 0x5612fe24c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:49.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.897+0000 7fc0aeffd640 1 -- 172.21.15.38:0/1579358651 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5612fde4b6b0 msgr2=0x5612fe24c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:38:49.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.897+0000 7fc0aeffd640 1 --2- 172.21.15.38:0/1579358651 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5612fde4b6b0 0x5612fe24c020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:49.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.897+0000 7fc0aeffd640 1 -- 172.21.15.38:0/1579358651 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5612fde6c390 msgr2=0x5612fe23cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:38:49.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.897+0000 7fc0aeffd640 1 --2- 172.21.15.38:0/1579358651 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5612fde6c390 0x5612fe23cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:49.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.897+0000 7fc0aeffd640 1 -- 172.21.15.38:0/1579358651 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5612fded7970 con 0x5612fde434d0 2024-09-21T03:38:49.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.898+0000 7fc0aeffd640 1 --2- 172.21.15.38:0/1579358651 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5612fde434d0 0x5612fe23e020 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fc0a00aa120 tx=0x7fc0a0004d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da5214ff1c7c40ca server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:49.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.898+0000 7fc08dffb640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-21T03:38:49.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.898+0000 7fc08dffb640 1 -- 172.21.15.38:0/1579358651 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0a0083110 con 0x5612fde434d0 2024-09-21T03:38:49.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.898+0000 7fc08dffb640 1 -- 172.21.15.38:0/1579358651 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(71..71 src has 1..71) ==== 6476+0+0 (secure 0 0 0) 0x7fc0a007f5f0 con 0x5612fde434d0 2024-09-21T03:38:49.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.898+0000 7fc0bc49a740 1 -- 172.21.15.38:0/1579358651 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc0a00886d0 con 0x5612fde434d0 2024-09-21T03:38:49.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.898+0000 7fc08dffb640 1 -- 172.21.15.38:0/1579358651 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7fc0a007f810 con 0x5612fde434d0 2024-09-21T03:38:49.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.898+0000 7fc0bc49a740 10 client.6405 fetch_fsmap learned FSMap version 59 2024-09-21T03:38:49.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.898+0000 7fc0bc49a740 1 -- 172.21.15.38:0/1579358651 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=2+}) -- 0x5612fdeb56a0 con 0x5612fde434d0 2024-09-21T03:38:49.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.898+0000 7fc08dffb640 1 -- 172.21.15.38:0/1579358651 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc0a00886d0 con 0x5612fde434d0 2024-09-21T03:38:49.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.898+0000 7fc08dffb640 1 -- 172.21.15.38:0/1579358651 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 59) ==== 3502+0+0 (secure 0 0 0) 0x7fc0a0040030 con 0x5612fde434d0 2024-09-21T03:38:49.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.898+0000 7fc0bc49a740 10 client.6405 fetch_fsmap finished waiting for FSMap version 59 2024-09-21T03:38:49.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.898+0000 7fc0bc49a740 10 client.6405 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.b 2024-09-21T03:38:49.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.898+0000 7fc0bc49a740 20 client.6405 populate_metadata read hostname 'smithi038' 2024-09-21T03:38:49.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.898+0000 7fc0bc49a740 1 --2- 172.21.15.38:0/1579358651 >> [v2:172.21.15.195:6834/2536177898,v1:172.21.15.195:6836/2536177898] conn(0x5612fde2af60 0x5612fe23d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:49.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.898+0000 7fc0bc49a740 4 client.6405 mds_command: new command op to 5731 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:38:49.906 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.898+0000 7fc0bc49a740 1 -- 172.21.15.38:0/1579358651 --> [v2:172.21.15.195:6834/2536177898,v1:172.21.15.195:6836/2536177898] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5612fe1889b0 con 0x5612fde2af60 2024-09-21T03:38:49.906 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.899+0000 7fc0ae7fc640 1 --2- 172.21.15.38:0/1579358651 >> [v2:172.21.15.195:6834/2536177898,v1:172.21.15.195:6836/2536177898] conn(0x5612fde2af60 0x5612fe23d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:49.906 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.899+0000 7fc08dffb640 10 client.6405 ms_handle_connect on v2:172.21.15.195:6834/2536177898 2024-09-21T03:38:49.906 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.899+0000 7fc0ae7fc640 1 --2- 172.21.15.38:0/1579358651 >> [v2:172.21.15.195:6834/2536177898,v1:172.21.15.195:6836/2536177898] conn(0x5612fde2af60 0x5612fe23d4b0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=736f7c416ad12e16 server_cookie=19f68a9caa32c4ab in_seq=0 out_seq=0 2024-09-21T03:38:49.906 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.900+0000 7fc08dffb640 1 -- 172.21.15.38:0/1579358651 <== mds.0 v2:172.21.15.195:6834/2536177898 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5612fe1889b0 con 0x5612fde2af60 2024-09-21T03:38:49.906 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.900+0000 7fc08dffb640 10 client.6405 handle_command_reply: tid=0 2024-09-21T03:38:49.906 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.900+0000 7fc0bc49a740 1 client.6405 shutdown 2024-09-21T03:38:49.906 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.900+0000 7fc0bc49a740 1 -- 172.21.15.38:0/1579358651 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5612fde434d0 msgr2=0x5612fe23e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:49.906 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.900+0000 7fc0bc49a740 1 --2- 172.21.15.38:0/1579358651 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5612fde434d0 0x5612fe23e020 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fc0a00aa120 tx=0x7fc0a0004d70 comp rx=0 tx=0).stop 2024-09-21T03:38:49.906 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.900+0000 7fc0bc49a740 1 -- 172.21.15.38:0/1579358651 shutdown_connections 2024-09-21T03:38:49.906 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.900+0000 7fc0bc49a740 1 --2- 172.21.15.38:0/1579358651 >> [v2:172.21.15.195:6834/2536177898,v1:172.21.15.195:6836/2536177898] conn(0x5612fde2af60 0x5612fe23d4b0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:49.906 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.900+0000 7fc0bc49a740 1 --2- 172.21.15.38:0/1579358651 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5612fde6c390 0x5612fe23cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:49.906 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.900+0000 7fc0bc49a740 1 --2- 172.21.15.38:0/1579358651 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5612fde4b6b0 0x5612fe24c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:49.906 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.900+0000 7fc08dffb640 0 client.6405 ms_handle_reset on v2:172.21.15.195:6834/2536177898 2024-09-21T03:38:49.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.900+0000 7fc0bc49a740 1 --2- 172.21.15.38:0/1579358651 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5612fde434d0 0x5612fe23e020 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:49.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.900+0000 7fc0bc49a740 1 -- 172.21.15.38:0/1579358651 >> 172.21.15.38:0/1579358651 conn(0x5612fde604e0 msgr2=0x5612fe213d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:49.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.900+0000 7fc0bc49a740 1 -- 172.21.15.38:0/1579358651 shutdown_connections 2024-09-21T03:38:49.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.900+0000 7fc0bc49a740 1 -- 172.21.15.38:0/1579358651 wait complete. 2024-09-21T03:38:49.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.900+0000 7fc0bc49a740 20 client.6405 trim_cache size 0 max 16384 2024-09-21T03:38:49.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.915+0000 7fc0bc49a740 1 Processor -- start 2024-09-21T03:38:49.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.915+0000 7fc0bc49a740 1 -- start start 2024-09-21T03:38:49.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.917+0000 7fc0bc49a740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5612fde434d0 0x5612fe267680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:49.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.917+0000 7fc0bc49a740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5612fde4b6b0 0x5612fe23b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:49.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.917+0000 7fc0bc49a740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5612fde6c390 0x5612fe2398c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:49.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.917+0000 7fc0bc49a740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5612fe2154d0 con 0x5612fde434d0 2024-09-21T03:38:49.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.917+0000 7fc0bc49a740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5612fe259530 con 0x5612fde4b6b0 2024-09-21T03:38:49.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.917+0000 7fc0bc49a740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fc0b01fe8b0 con 0x5612fde6c390 2024-09-21T03:38:49.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.917+0000 7fc0aeffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5612fde434d0 0x5612fe267680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:49.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.917+0000 7fc0aeffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5612fde434d0 0x5612fe267680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48316/0 (socket says 172.21.15.38:48316) 2024-09-21T03:38:49.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.917+0000 7fc0aeffd640 1 -- 172.21.15.38:0/1132795163 learned_addr learned my addr 172.21.15.38:0/1132795163 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:38:49.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.917+0000 7fc0aeffd640 1 -- 172.21.15.38:0/1132795163 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5612fde6c390 msgr2=0x5612fe2398c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:38:49.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.917+0000 7fc0aeffd640 1 --2- 172.21.15.38:0/1132795163 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5612fde6c390 0x5612fe2398c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:49.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.917+0000 7fc0aeffd640 1 -- 172.21.15.38:0/1132795163 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5612fde4b6b0 msgr2=0x5612fe23b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:38:49.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.917+0000 7fc0aeffd640 1 --2- 172.21.15.38:0/1132795163 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5612fde4b6b0 0x5612fe23b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:49.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.917+0000 7fc0aeffd640 1 -- 172.21.15.38:0/1132795163 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5612fe24e530 con 0x5612fde434d0 2024-09-21T03:38:49.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.917+0000 7fc0aeffd640 1 --2- 172.21.15.38:0/1132795163 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5612fde434d0 0x5612fe267680 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fc0a00839a0 tx=0x7fc0a0080420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=246a50faee2c408f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:49.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.917+0000 7fc08ffff640 1 -- 172.21.15.38:0/1132795163 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0a0088040 con 0x5612fde434d0 2024-09-21T03:38:49.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.917+0000 7fc08ffff640 1 -- 172.21.15.38:0/1132795163 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc0a00837b0 con 0x5612fde434d0 2024-09-21T03:38:49.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.917+0000 7fc08ffff640 1 -- 172.21.15.38:0/1132795163 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0a00886d0 con 0x5612fde434d0 2024-09-21T03:38:49.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.918+0000 7fc0bc49a740 1 -- 172.21.15.38:0/1132795163 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5612fde434d0 msgr2=0x5612fe267680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:49.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.918+0000 7fc0bc49a740 1 --2- 172.21.15.38:0/1132795163 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5612fde434d0 0x5612fe267680 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fc0a00839a0 tx=0x7fc0a0080420 comp rx=0 tx=0).stop 2024-09-21T03:38:49.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.918+0000 7fc0bc49a740 1 -- 172.21.15.38:0/1132795163 shutdown_connections 2024-09-21T03:38:49.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.918+0000 7fc0bc49a740 1 --2- 172.21.15.38:0/1132795163 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5612fde6c390 0x5612fe2398c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:49.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.918+0000 7fc0bc49a740 1 --2- 172.21.15.38:0/1132795163 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5612fde4b6b0 0x5612fe23b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:49.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.918+0000 7fc0bc49a740 1 --2- 172.21.15.38:0/1132795163 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5612fde434d0 0x5612fe267680 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:49.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.918+0000 7fc0bc49a740 1 -- 172.21.15.38:0/1132795163 >> 172.21.15.38:0/1132795163 conn(0x5612fde604e0 msgr2=0x5612fe23cd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:49.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.918+0000 7fc0bc49a740 1 -- 172.21.15.38:0/1132795163 shutdown_connections 2024-09-21T03:38:49.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.918+0000 7fc0bc49a740 1 -- 172.21.15.38:0/1132795163 wait complete. 2024-09-21T03:38:49.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.919+0000 7fc0bc49a740 1 -- start start 2024-09-21T03:38:49.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.919+0000 7fc0bc49a740 1 Processor -- start 2024-09-21T03:38:49.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.920+0000 7fc0bc49a740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5612fde434d0 0x5612fe2d7c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:49.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.920+0000 7fc0bc49a740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5612fde4b6b0 0x5612fe23b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:49.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.920+0000 7fc0bc49a740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5612fde6c390 0x5612fe23bae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:49.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.920+0000 7fc0bc49a740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fc0b01f6970 con 0x5612fde434d0 2024-09-21T03:38:49.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.920+0000 7fc0bc49a740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fc0b009f380 con 0x5612fde6c390 2024-09-21T03:38:49.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.920+0000 7fc0bc49a740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fc0b009ecf0 con 0x5612fde4b6b0 2024-09-21T03:38:49.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.921+0000 7fc0bc49a740 10 client.0 _ll_register_callbacks cb 0x7fc0a00aa070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:38:49.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.921+0000 7fc0aeffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5612fde434d0 0x5612fe2d7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:49.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.921+0000 7fc0aeffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5612fde434d0 0x5612fe2d7c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48324/0 (socket says 172.21.15.38:48324) 2024-09-21T03:38:49.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.921+0000 7fc0aeffd640 1 -- 172.21.15.38:0/637203491 learned_addr learned my addr 172.21.15.38:0/637203491 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:38:49.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.921+0000 7fc0ae7fc640 1 --2- 172.21.15.38:0/637203491 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5612fde4b6b0 0x5612fe23b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:49.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.921+0000 7fc0af7fe640 1 --2- 172.21.15.38:0/637203491 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5612fde6c390 0x5612fe23bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:49.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.921+0000 7fc0aeffd640 1 -- 172.21.15.38:0/637203491 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5612fde4b6b0 msgr2=0x5612fe23b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:38:49.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.921+0000 7fc0aeffd640 1 --2- 172.21.15.38:0/637203491 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5612fde4b6b0 0x5612fe23b5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:49.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.921+0000 7fc0aeffd640 1 -- 172.21.15.38:0/637203491 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5612fde6c390 msgr2=0x5612fe23bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:38:49.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.921+0000 7fc0aeffd640 1 --2- 172.21.15.38:0/637203491 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5612fde6c390 0x5612fe23bae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:49.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.921+0000 7fc0aeffd640 1 -- 172.21.15.38:0/637203491 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5612fe23c020 con 0x5612fde434d0 2024-09-21T03:38:49.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.921+0000 7fc0af7fe640 1 --2- 172.21.15.38:0/637203491 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5612fde6c390 0x5612fe23bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:38:49.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.921+0000 7fc0ae7fc640 1 --2- 172.21.15.38:0/637203491 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5612fde4b6b0 0x5612fe23b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:38:49.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.921+0000 7fc0aeffd640 1 --2- 172.21.15.38:0/637203491 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5612fde434d0 0x5612fe2d7c40 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fc0a0080f10 tx=0x7fc0a0005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abe3a467acf139ec server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:49.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.921+0000 7fc08dffb640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-21T03:38:49.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.921+0000 7fc08dffb640 1 -- 172.21.15.38:0/637203491 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0a0088050 con 0x5612fde434d0 2024-09-21T03:38:49.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.921+0000 7fc08dffb640 1 -- 172.21.15.38:0/637203491 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(71..71 src has 1..71) ==== 6476+0+0 (secure 0 0 0) 0x7fc0a00881f0 con 0x5612fde434d0 2024-09-21T03:38:49.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.921+0000 7fc0bc49a740 1 -- 172.21.15.38:0/637203491 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc0a00837b0 con 0x5612fde434d0 2024-09-21T03:38:49.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.921+0000 7fc08dffb640 1 -- 172.21.15.38:0/637203491 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7fc0a00886d0 con 0x5612fde434d0 2024-09-21T03:38:49.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.921+0000 7fc0bc49a740 10 client.6411 fetch_fsmap learned FSMap version 59 2024-09-21T03:38:49.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.921+0000 7fc0bc49a740 1 -- 172.21.15.38:0/637203491 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=2+}) -- 0x5612fe23c020 con 0x5612fde434d0 2024-09-21T03:38:49.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.922+0000 7fc08dffb640 1 -- 172.21.15.38:0/637203491 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc0a00837b0 con 0x5612fde434d0 2024-09-21T03:38:49.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.922+0000 7fc08dffb640 1 -- 172.21.15.38:0/637203491 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 59) ==== 3502+0+0 (secure 0 0 0) 0x7fc0a0040030 con 0x5612fde434d0 2024-09-21T03:38:49.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.922+0000 7fc0bc49a740 10 client.6411 fetch_fsmap finished waiting for FSMap version 59 2024-09-21T03:38:49.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.922+0000 7fc0bc49a740 10 client.6411 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.b 2024-09-21T03:38:49.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.922+0000 7fc0bc49a740 20 client.6411 populate_metadata read hostname 'smithi038' 2024-09-21T03:38:49.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.922+0000 7fc0bc49a740 1 --2- 172.21.15.38:0/637203491 >> [v2:172.21.15.195:6834/2536177898,v1:172.21.15.195:6836/2536177898] conn(0x5612fe24a620 0x5612fe24aa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:49.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.922+0000 7fc0bc49a740 4 client.6411 mds_command: new command op to 5731 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T03:38:49.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.922+0000 7fc0bc49a740 1 -- 172.21.15.38:0/637203491 --> [v2:172.21.15.195:6834/2536177898,v1:172.21.15.195:6836/2536177898] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5612fe184460 con 0x5612fe24a620 2024-09-21T03:38:49.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.922+0000 7fc0ae7fc640 1 --2- 172.21.15.38:0/637203491 >> [v2:172.21.15.195:6834/2536177898,v1:172.21.15.195:6836/2536177898] conn(0x5612fe24a620 0x5612fe24aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:49.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.922+0000 7fc08dffb640 10 client.6411 ms_handle_connect on v2:172.21.15.195:6834/2536177898 2024-09-21T03:38:49.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.922+0000 7fc0ae7fc640 1 --2- 172.21.15.38:0/637203491 >> [v2:172.21.15.195:6834/2536177898,v1:172.21.15.195:6836/2536177898] conn(0x5612fe24a620 0x5612fe24aa30 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=acd3481792f99bc2 server_cookie=2151fb001524337a in_seq=0 out_seq=0 2024-09-21T03:38:49.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.944+0000 7fc08dffb640 1 -- 172.21.15.38:0/637203491 <== mds.0 v2:172.21.15.195:6834/2536177898 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5612fe184460 con 0x5612fe24a620 2024-09-21T03:38:49.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.944+0000 7fc08dffb640 10 client.6411 handle_command_reply: tid=0 2024-09-21T03:38:49.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.944+0000 7fc0bc49a740 1 client.6411 shutdown 2024-09-21T03:38:49.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.944+0000 7fc0bc49a740 1 -- 172.21.15.38:0/637203491 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5612fde434d0 msgr2=0x5612fe2d7c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:49.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.944+0000 7fc0bc49a740 1 --2- 172.21.15.38:0/637203491 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5612fde434d0 0x5612fe2d7c40 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fc0a0080f10 tx=0x7fc0a0005fd0 comp rx=0 tx=0).stop 2024-09-21T03:38:49.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.944+0000 7fc0bc49a740 1 -- 172.21.15.38:0/637203491 shutdown_connections 2024-09-21T03:38:49.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.944+0000 7fc0bc49a740 1 --2- 172.21.15.38:0/637203491 >> [v2:172.21.15.195:6834/2536177898,v1:172.21.15.195:6836/2536177898] conn(0x5612fe24a620 0x5612fe24aa30 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:49.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.944+0000 7fc0bc49a740 1 --2- 172.21.15.38:0/637203491 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5612fde6c390 0x5612fe23bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:49.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.944+0000 7fc0bc49a740 1 --2- 172.21.15.38:0/637203491 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5612fde4b6b0 0x5612fe23b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:49.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.944+0000 7fc0bc49a740 1 --2- 172.21.15.38:0/637203491 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5612fde434d0 0x5612fe2d7c40 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:49.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.944+0000 7fc08dffb640 0 client.6411 ms_handle_reset on v2:172.21.15.195:6834/2536177898 2024-09-21T03:38:49.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.944+0000 7fc0bc49a740 1 -- 172.21.15.38:0/637203491 >> 172.21.15.38:0/637203491 conn(0x5612fde604e0 msgr2=0x5612fe266c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:49.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.944+0000 7fc0bc49a740 1 -- 172.21.15.38:0/637203491 shutdown_connections 2024-09-21T03:38:49.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.944+0000 7fc0bc49a740 1 -- 172.21.15.38:0/637203491 wait complete. 2024-09-21T03:38:49.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.945+0000 7fc0bc49a740 20 client.6411 trim_cache size 0 max 16384 2024-09-21T03:38:49.948 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-09-21T03:38:49.948 INFO:teuthology.orchestra.run.smithi038.stdout: "message": "", 2024-09-21T03:38:49.948 INFO:teuthology.orchestra.run.smithi038.stdout: "return_code": 0 2024-09-21T03:38:49.948 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-09-21T03:38:49.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.945+0000 7fc0b552b640 1 -- 172.21.15.38:0/3689336920 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc09c078b90 msgr2=0x7fc09c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:49.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.945+0000 7fc0b552b640 1 --2- 172.21.15.38:0/3689336920 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc09c078b90 0x7fc09c07b0a0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fc0b015af50 tx=0x7fc0a4009060 comp rx=0 tx=0).stop 2024-09-21T03:38:49.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.945+0000 7fc0b552b640 1 -- 172.21.15.38:0/3689336920 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc0b006e330 msgr2=0x7fc0b01f1ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:49.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.945+0000 7fc0b552b640 1 --2- 172.21.15.38:0/3689336920 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc0b006e330 0x7fc0b01f1ef0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fc0a0000f80 tx=0x7fc0a0003f90 comp rx=0 tx=0).stop 2024-09-21T03:38:49.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.945+0000 7fc0b552b640 1 -- 172.21.15.38:0/3689336920 shutdown_connections 2024-09-21T03:38:49.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.945+0000 7fc0b552b640 1 --2- 172.21.15.38:0/3689336920 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc09c078b90 0x7fc09c07b0a0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:49.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.945+0000 7fc0b552b640 1 --2- 172.21.15.38:0/3689336920 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc0b015fc50 0x7fc0b015d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:49.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.945+0000 7fc0b552b640 1 --2- 172.21.15.38:0/3689336920 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc0b00cf9a0 0x7fc0b015cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:49.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.945+0000 7fc0b552b640 1 --2- 172.21.15.38:0/3689336920 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc0b006e330 0x7fc0b01f1ef0 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:49.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.945+0000 7fc0b552b640 1 -- 172.21.15.38:0/3689336920 >> 172.21.15.38:0/3689336920 conn(0x7fc0b00b1b90 msgr2=0x7fc0b00b4ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:49.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.945+0000 7fc0b552b640 1 -- 172.21.15.38:0/3689336920 shutdown_connections 2024-09-21T03:38:49.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:49.945+0000 7fc0b552b640 1 -- 172.21.15.38:0/3689336920 wait complete. 2024-09-21T03:38:49.958 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-21T03:38:49.958 DEBUG:teuthology.orchestra.run.smithi038:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T03:38:50.016 DEBUG:teuthology.orchestra.run.smithi195:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T03:38:50.061 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-21T03:38:50.061 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T03:38:50.061 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-21T03:38:50.061 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:38:50.078 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-21T03:38:50.078 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:38:50.134 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T03:38:50.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:50.296+0000 7f0a8648b640 1 -- 172.21.15.38:0/3390171475 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0a68004800 msgr2=0x7f0a68004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:50.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:50.296+0000 7f0a8648b640 1 --2- 172.21.15.38:0/3390171475 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0a68004800 0x7f0a68004c10 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f0a74009b10 tx=0x7f0a68005830 comp rx=0 tx=0).stop 2024-09-21T03:38:50.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:50.296+0000 7f0a84a28640 1 -- 172.21.15.38:0/3390171475 reap_dead start 2024-09-21T03:38:50.299 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:50.296+0000 7f0a8648b640 1 -- 172.21.15.38:0/3390171475 shutdown_connections 2024-09-21T03:38:50.299 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:50.296+0000 7f0a8648b640 1 -- 172.21.15.38:0/3390171475 >> 172.21.15.38:0/3390171475 conn(0x7f0a8009d830 msgr2=0x7f0a800b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:50.299 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:50.296+0000 7f0a8648b640 1 -- 172.21.15.38:0/3390171475 shutdown_connections 2024-09-21T03:38:50.299 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:50.296+0000 7f0a8648b640 1 -- 172.21.15.38:0/3390171475 wait complete. 2024-09-21T03:38:50.300 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:50.298+0000 7f0a8648b640 1 Processor -- start 2024-09-21T03:38:50.300 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:50.298+0000 7f0a8648b640 1 -- start start 2024-09-21T03:38:50.302 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:50.299+0000 7f0a8648b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0a68004800 0x7f0a8006f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:50.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:50.299+0000 7f0a8648b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0a80157c20 0x7f0a8014f990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:50.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:50.299+0000 7f0a8648b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0a800a00a0 0x7f0a801f56c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:50.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:50.299+0000 7f0a8648b640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f0a800cf250 con 0x7f0a68004800 2024-09-21T03:38:50.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:50.299+0000 7f0a8648b640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f0a800cf0d0 con 0x7f0a80157c20 2024-09-21T03:38:50.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:50.300+0000 7f0a8648b640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f0a800cf3d0 con 0x7f0a800a00a0 2024-09-21T03:38:50.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:50.300+0000 7f0a7ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0a68004800 0x7f0a8006f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:50.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:50.300+0000 7f0a7ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0a68004800 0x7f0a8006f170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48342/0 (socket says 172.21.15.38:48342) 2024-09-21T03:38:50.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:50.300+0000 7f0a7ffff640 1 -- 172.21.15.38:0/2999970381 learned_addr learned my addr 172.21.15.38:0/2999970381 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:38:50.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:50.300+0000 7f0a7f7fe640 1 --2- 172.21.15.38:0/2999970381 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0a80157c20 0x7f0a8014f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:50.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:50.300+0000 7f0a84a28640 1 --2- 172.21.15.38:0/2999970381 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0a800a00a0 0x7f0a801f56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:50.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:50.300+0000 7f0a7ffff640 1 -- 172.21.15.38:0/2999970381 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0a800a00a0 msgr2=0x7f0a801f56c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:50.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:50.300+0000 7f0a7ffff640 1 --2- 172.21.15.38:0/2999970381 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0a800a00a0 0x7f0a801f56c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:50.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:50.300+0000 7f0a7ffff640 1 -- 172.21.15.38:0/2999970381 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0a80157c20 msgr2=0x7f0a8014f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:50.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:50.300+0000 7f0a7ffff640 1 --2- 172.21.15.38:0/2999970381 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0a80157c20 0x7f0a8014f990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:50.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:50.300+0000 7f0a7ffff640 1 -- 172.21.15.38:0/2999970381 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a8009e4b0 con 0x7f0a68004800 2024-09-21T03:38:50.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:50.300+0000 7f0a7f7fe640 1 --2- 172.21.15.38:0/2999970381 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0a80157c20 0x7f0a8014f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:38:50.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:50.300+0000 7f0a7ffff640 1 --2- 172.21.15.38:0/2999970381 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0a68004800 0x7f0a8006f170 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f0a70006670 tx=0x7f0a700039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:50.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:50.300+0000 7f0a7cff9640 1 -- 172.21.15.38:0/2999970381 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a70015070 con 0x7f0a68004800 2024-09-21T03:38:50.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:50.300+0000 7f0a7cff9640 1 -- 172.21.15.38:0/2999970381 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0a70011900 con 0x7f0a68004800 2024-09-21T03:38:50.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:50.300+0000 7f0a8648b640 1 -- 172.21.15.38:0/2999970381 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a8009e790 con 0x7f0a68004800 2024-09-21T03:38:50.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:50.300+0000 7f0a7cff9640 1 -- 172.21.15.38:0/2999970381 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a70011d50 con 0x7f0a68004800 2024-09-21T03:38:50.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:50.300+0000 7f0a8648b640 1 -- 172.21.15.38:0/2999970381 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a8006ec50 con 0x7f0a68004800 2024-09-21T03:38:50.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:50.301+0000 7f0a8648b640 1 -- 172.21.15.38:0/2999970381 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a800b46b0 con 0x7f0a68004800 2024-09-21T03:38:50.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:50.302+0000 7f0a7cff9640 1 -- 172.21.15.38:0/2999970381 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f0a7000f430 con 0x7f0a68004800 2024-09-21T03:38:50.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:50.303+0000 7f0a7cff9640 1 --2- 172.21.15.38:0/2999970381 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0a6c078bc0 0x7f0a6c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:50.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:50.303+0000 7f0a7cff9640 1 -- 172.21.15.38:0/2999970381 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 6476+0+0 (secure 0 0 0) 0x7f0a7009bd50 con 0x7f0a68004800 2024-09-21T03:38:50.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:50.303+0000 7f0a7f7fe640 1 --2- 172.21.15.38:0/2999970381 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0a6c078bc0 0x7f0a6c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:50.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:50.304+0000 7f0a7f7fe640 1 --2- 172.21.15.38:0/2999970381 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0a6c078bc0 0x7f0a6c07b0d0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f0a74009b10 tx=0x7f0a7402d040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:50.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:50.307+0000 7f0a7cff9640 1 -- 172.21.15.38:0/2999970381 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f0a7009f030 con 0x7f0a68004800 2024-09-21T03:38:50.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:50.507+0000 7f0a8648b640 1 -- 172.21.15.38:0/2999970381 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f0a800b43d0 con 0x7f0a68004800 2024-09-21T03:38:50.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:50.508+0000 7f0a7cff9640 1 -- 172.21.15.38:0/2999970381 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 4 entries v71) ==== 80+0+244 (secure 0 0 0) 0x7f0a801592d0 con 0x7f0a68004800 2024-09-21T03:38:50.510 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6834/1058376326 2024-09-22T03:37:12.044790+0000 2024-09-21T03:38:50.510 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6832/1058376326 2024-09-22T03:37:12.044790+0000 2024-09-21T03:38:50.510 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6834/1353335665 2024-09-22T03:36:36.420530+0000 2024-09-21T03:38:50.510 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6832/1353335665 2024-09-22T03:36:36.420530+0000 2024-09-21T03:38:50.510 INFO:teuthology.orchestra.run.smithi038.stderr:listed 4 entries 2024-09-21T03:38:50.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:50.510+0000 7f0a8648b640 1 -- 172.21.15.38:0/2999970381 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0a6c078bc0 msgr2=0x7f0a6c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:50.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:50.510+0000 7f0a8648b640 1 --2- 172.21.15.38:0/2999970381 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0a6c078bc0 0x7f0a6c07b0d0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f0a74009b10 tx=0x7f0a7402d040 comp rx=0 tx=0).stop 2024-09-21T03:38:50.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:50.510+0000 7f0a8648b640 1 -- 172.21.15.38:0/2999970381 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0a68004800 msgr2=0x7f0a8006f170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:50.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:50.510+0000 7f0a8648b640 1 --2- 172.21.15.38:0/2999970381 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0a68004800 0x7f0a8006f170 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f0a70006670 tx=0x7f0a700039c0 comp rx=0 tx=0).stop 2024-09-21T03:38:50.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:50.510+0000 7f0a8648b640 1 -- 172.21.15.38:0/2999970381 shutdown_connections 2024-09-21T03:38:50.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:50.510+0000 7f0a8648b640 1 --2- 172.21.15.38:0/2999970381 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0a6c078bc0 0x7f0a6c07b0d0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:50.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:50.510+0000 7f0a8648b640 1 --2- 172.21.15.38:0/2999970381 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0a800a00a0 0x7f0a801f56c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:50.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:50.510+0000 7f0a8648b640 1 --2- 172.21.15.38:0/2999970381 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0a80157c20 0x7f0a8014f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:50.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:50.510+0000 7f0a8648b640 1 --2- 172.21.15.38:0/2999970381 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0a68004800 0x7f0a8006f170 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:50.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:50.510+0000 7f0a8648b640 1 -- 172.21.15.38:0/2999970381 >> 172.21.15.38:0/2999970381 conn(0x7f0a8009d830 msgr2=0x7f0a800b31a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:50.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:50.510+0000 7f0a8648b640 1 -- 172.21.15.38:0/2999970381 shutdown_connections 2024-09-21T03:38:50.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:50.510+0000 7f0a8648b640 1 -- 172.21.15.38:0/2999970381 wait complete. 2024-09-21T03:38:50.523 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-21T03:38:50.523 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:38:50.524 DEBUG:teuthology.orchestra.run.smithi038:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-21T03:38:50.553 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi038.front.sepia.ceph.com 2024-09-21T03:38:50.553 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:38:50.553 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T03:38:50.610 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T03:38:50.610 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-09-21T03:38:50.610 DEBUG:teuthology.orchestra.run.smithi195:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:38:50.629 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056014,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497444k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=8978 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248720k,nr_inodes=812180,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497444k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T03:38:50.629 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T03:38:50.630 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-21T03:38:50.630 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:38:50.665 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056795,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2354 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T03:38:50.666 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T03:38:50.666 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-09-21T03:38:50.666 DEBUG:teuthology.orchestra.run.smithi195:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:38:50.688 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056014,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497444k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=8978 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248720k,nr_inodes=812180,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497444k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T03:38:50.689 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:38:50.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:50.820+0000 7f6f4f2a2640 1 -- 172.21.15.38:0/3470169622 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6f48162280 msgr2=0x7f6f48162690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:50.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:50.820+0000 7f6f4f2a2640 1 --2- 172.21.15.38:0/3470169622 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6f48162280 0x7f6f48162690 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f6f480d5040 tx=0x7f6f3802a560 comp rx=0 tx=0).stop 2024-09-21T03:38:50.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:50.820+0000 7f6f4f2a2640 1 -- 172.21.15.38:0/3470169622 shutdown_connections 2024-09-21T03:38:50.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:50.820+0000 7f6f4f2a2640 1 --2- 172.21.15.38:0/3470169622 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6f48162280 0x7f6f48162690 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:50.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:50.820+0000 7f6f4f2a2640 1 --2- 172.21.15.38:0/3470169622 >> v2:172.21.15.195:3300/0 conn(0x7f6f48162f00 0x7f6f480d24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:50.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:50.820+0000 7f6f4f2a2640 1 -- 172.21.15.38:0/3470169622 >> 172.21.15.38:0/3470169622 conn(0x7f6f480b6b20 msgr2=0x7f6f480d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:50.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:50.820+0000 7f6f4f2a2640 1 -- 172.21.15.38:0/3470169622 shutdown_connections 2024-09-21T03:38:50.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:50.820+0000 7f6f4f2a2640 1 -- 172.21.15.38:0/3470169622 wait complete. 2024-09-21T03:38:50.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:50.821+0000 7f6f4f2a2640 1 Processor -- start 2024-09-21T03:38:50.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:50.822+0000 7f6f4f2a2640 1 -- start start 2024-09-21T03:38:50.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:50.823+0000 7f6f4f2a2640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6f480b2e40 0x7f6f480b1b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:50.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:50.823+0000 7f6f4f2a2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6f48162280 0x7f6f4814fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:50.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:50.823+0000 7f6f4f2a2640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6f480af7a0 0x7f6f480b0f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:50.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:50.823+0000 7f6f4f2a2640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f6f4815e630 con 0x7f6f48162280 2024-09-21T03:38:50.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:50.823+0000 7f6f4f2a2640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f6f4815e4b0 con 0x7f6f480af7a0 2024-09-21T03:38:50.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:50.823+0000 7f6f4f2a2640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f6f4815e7b0 con 0x7f6f480b2e40 2024-09-21T03:38:50.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:50.823+0000 7f6f4c83d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6f48162280 0x7f6f4814fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:50.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:50.823+0000 7f6f4c83d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6f48162280 0x7f6f4814fba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48372/0 (socket says 172.21.15.38:48372) 2024-09-21T03:38:50.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:50.823+0000 7f6f4c83d640 1 -- 172.21.15.38:0/2080153236 learned_addr learned my addr 172.21.15.38:0/2080153236 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:38:50.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:50.823+0000 7f6f4d03e640 1 --2- 172.21.15.38:0/2080153236 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6f480b2e40 0x7f6f480b1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:50.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:50.823+0000 7f6f4d83f640 1 --2- 172.21.15.38:0/2080153236 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6f480af7a0 0x7f6f480b0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:50.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:50.823+0000 7f6f4c83d640 1 -- 172.21.15.38:0/2080153236 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6f480b2e40 msgr2=0x7f6f480b1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:50.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:50.823+0000 7f6f4c83d640 1 --2- 172.21.15.38:0/2080153236 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6f480b2e40 0x7f6f480b1b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:50.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:50.823+0000 7f6f4c83d640 1 -- 172.21.15.38:0/2080153236 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6f480af7a0 msgr2=0x7f6f480b0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:50.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:50.823+0000 7f6f4c83d640 1 --2- 172.21.15.38:0/2080153236 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6f480af7a0 0x7f6f480b0f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:50.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:50.823+0000 7f6f4c83d640 1 -- 172.21.15.38:0/2080153236 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f481500e0 con 0x7f6f48162280 2024-09-21T03:38:50.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:50.824+0000 7f6f4c83d640 1 --2- 172.21.15.38:0/2080153236 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6f48162280 0x7f6f4814fba0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f6f38034040 tx=0x7f6f380038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:50.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:50.824+0000 7f6f3dffb640 1 -- 172.21.15.38:0/2080153236 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f3803e030 con 0x7f6f48162280 2024-09-21T03:38:50.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:50.824+0000 7f6f3dffb640 1 -- 172.21.15.38:0/2080153236 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6f38003db0 con 0x7f6f48162280 2024-09-21T03:38:50.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:50.824+0000 7f6f4f2a2640 1 -- 172.21.15.38:0/2080153236 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f480afef0 con 0x7f6f48162280 2024-09-21T03:38:50.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:50.824+0000 7f6f3dffb640 1 -- 172.21.15.38:0/2080153236 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f38035040 con 0x7f6f48162280 2024-09-21T03:38:50.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:50.824+0000 7f6f4f2a2640 1 -- 172.21.15.38:0/2080153236 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f480b04b0 con 0x7f6f48162280 2024-09-21T03:38:50.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:50.825+0000 7f6f4f2a2640 1 -- 172.21.15.38:0/2080153236 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f480afc60 con 0x7f6f48162280 2024-09-21T03:38:50.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:50.826+0000 7f6f3dffb640 1 -- 172.21.15.38:0/2080153236 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f6f38041030 con 0x7f6f48162280 2024-09-21T03:38:50.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:50.826+0000 7f6f3dffb640 1 --2- 172.21.15.38:0/2080153236 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6f2c078b90 0x7f6f2c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:50.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:50.826+0000 7f6f3dffb640 1 -- 172.21.15.38:0/2080153236 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 6476+0+0 (secure 0 0 0) 0x7f6f380bba00 con 0x7f6f48162280 2024-09-21T03:38:50.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:50.826+0000 7f6f4d03e640 1 --2- 172.21.15.38:0/2080153236 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6f2c078b90 0x7f6f2c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:50.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:50.827+0000 7f6f4d03e640 1 --2- 172.21.15.38:0/2080153236 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6f2c078b90 0x7f6f2c07b0a0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f6f44009a00 tx=0x7f6f44005f90 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:50.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:50.831+0000 7f6f3dffb640 1 -- 172.21.15.38:0/2080153236 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f6f38083bd0 con 0x7f6f48162280 2024-09-21T03:38:51.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:51.054+0000 7f6f4f2a2640 1 -- 172.21.15.38:0/2080153236 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6f48162f00 con 0x7f6f48162280 2024-09-21T03:38:51.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:51.055+0000 7f6f3dffb640 1 -- 172.21.15.38:0/2080153236 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) ==== 94+0+5232 (secure 0 0 0) 0x7f6f380834d0 con 0x7f6f48162280 2024-09-21T03:38:51.057 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:38:51.057 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":59,"btime":"2024-09-21T03:37:15:051941+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5432,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/1042764291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":1042764291},{"type":"v1","addr":"172.21.15.195:6837","nonce":1042764291}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":5728,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/3035019482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6833","nonce":3035019482},{"type":"v1","addr":"172.21.15.38:6835","nonce":3035019482}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":6348,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6834/3775275137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":3775275137},{"type":"v1","addr":"172.21.15.38:6834","nonce":3775275137}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":59,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:36:21.442724+0000","modified":"2024-09-21T03:37:15.051929+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":69,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5731},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5731":{"gid":5731,"name":"b","rank":0,"incarnation":56,"state":"up:active","state_seq":17,"addr":"172.21.15.195:6836/2536177898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":2536177898},{"type":"v1","addr":"172.21.15.195:6836","nonce":2536177898}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5731,"qdb_cluster":[5731]},"id":4}]} 2024-09-21T03:38:51.057 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 59 2024-09-21T03:38:51.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:51.057+0000 7f6f4f2a2640 1 -- 172.21.15.38:0/2080153236 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6f2c078b90 msgr2=0x7f6f2c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:51.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:51.057+0000 7f6f4f2a2640 1 --2- 172.21.15.38:0/2080153236 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6f2c078b90 0x7f6f2c07b0a0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f6f44009a00 tx=0x7f6f44005f90 comp rx=0 tx=0).stop 2024-09-21T03:38:51.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:51.057+0000 7f6f4f2a2640 1 -- 172.21.15.38:0/2080153236 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6f48162280 msgr2=0x7f6f4814fba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:51.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:51.057+0000 7f6f4f2a2640 1 --2- 172.21.15.38:0/2080153236 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6f48162280 0x7f6f4814fba0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f6f38034040 tx=0x7f6f380038a0 comp rx=0 tx=0).stop 2024-09-21T03:38:51.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:51.057+0000 7f6f4f2a2640 1 -- 172.21.15.38:0/2080153236 shutdown_connections 2024-09-21T03:38:51.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:51.057+0000 7f6f4f2a2640 1 --2- 172.21.15.38:0/2080153236 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6f480af7a0 0x7f6f480b0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:51.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:51.057+0000 7f6f4f2a2640 1 --2- 172.21.15.38:0/2080153236 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6f48162280 0x7f6f4814fba0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:51.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:51.057+0000 7f6f4f2a2640 1 --2- 172.21.15.38:0/2080153236 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6f2c078b90 0x7f6f2c07b0a0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:51.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:51.057+0000 7f6f4f2a2640 1 --2- 172.21.15.38:0/2080153236 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6f480b2e40 0x7f6f480b1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:51.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:51.057+0000 7f6f4f2a2640 1 -- 172.21.15.38:0/2080153236 >> 172.21.15.38:0/2080153236 conn(0x7f6f480b6b20 msgr2=0x7f6f4815ec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:51.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:51.057+0000 7f6f4f2a2640 1 -- 172.21.15.38:0/2080153236 shutdown_connections 2024-09-21T03:38:51.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:51.057+0000 7f6f4f2a2640 1 -- 172.21.15.38:0/2080153236 wait complete. 2024-09-21T03:38:51.071 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:38:51.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:51.185+0000 7f88e2b72640 1 -- 172.21.15.38:0/720475693 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f88c8005270 msgr2=0x7f88c800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:51.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:51.186+0000 7f88e2b72640 1 --2- 172.21.15.38:0/720475693 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f88c8005270 0x7f88c800b9c0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f88c400c310 tx=0x7f88c8005830 comp rx=0 tx=0).stop 2024-09-21T03:38:51.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:51.186+0000 7f88e2b72640 1 -- 172.21.15.38:0/720475693 shutdown_connections 2024-09-21T03:38:51.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:51.186+0000 7f88e2b72640 1 --2- 172.21.15.38:0/720475693 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f88c8005270 0x7f88c800b9c0 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:51.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:51.186+0000 7f88e2b72640 1 -- 172.21.15.38:0/720475693 >> 172.21.15.38:0/720475693 conn(0x7f88dc0b2cc0 msgr2=0x7f88dc0b30d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:51.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:51.186+0000 7f88e2b72640 1 -- 172.21.15.38:0/720475693 shutdown_connections 2024-09-21T03:38:51.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:51.186+0000 7f88e2b72640 1 -- 172.21.15.38:0/720475693 wait complete. 2024-09-21T03:38:51.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:51.187+0000 7f88e2b72640 1 Processor -- start 2024-09-21T03:38:51.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:51.187+0000 7f88e2b72640 1 -- start start 2024-09-21T03:38:51.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:51.189+0000 7f88e2b72640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f88c8005270 0x7f88dc1f5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:51.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:51.189+0000 7f88e2b72640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f88dc152f20 0x7f88dc142820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:51.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:51.189+0000 7f88e2b72640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f88dc06cde0 0x7f88dc06d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:51.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:51.189+0000 7f88e2b72640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f88dc0744f0 con 0x7f88dc06cde0 2024-09-21T03:38:51.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:51.189+0000 7f88e2b72640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f88dc074370 con 0x7f88dc152f20 2024-09-21T03:38:51.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:51.189+0000 7f88e2b72640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f88dc074670 con 0x7f88c8005270 2024-09-21T03:38:51.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:51.189+0000 7f88e110f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f88dc06cde0 0x7f88dc06d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:51.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:51.189+0000 7f88e110f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f88dc06cde0 0x7f88dc06d1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48392/0 (socket says 172.21.15.38:48392) 2024-09-21T03:38:51.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:51.189+0000 7f88e110f640 1 -- 172.21.15.38:0/3455994640 learned_addr learned my addr 172.21.15.38:0/3455994640 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:38:51.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:51.189+0000 7f88dbfff640 1 --2- 172.21.15.38:0/3455994640 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f88dc152f20 0x7f88dc142820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:51.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:51.189+0000 7f88e090e640 1 --2- 172.21.15.38:0/3455994640 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f88c8005270 0x7f88dc1f5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:51.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:51.189+0000 7f88e110f640 1 -- 172.21.15.38:0/3455994640 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f88c8005270 msgr2=0x7f88dc1f5150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:51.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:51.189+0000 7f88e110f640 1 --2- 172.21.15.38:0/3455994640 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f88c8005270 0x7f88dc1f5150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:51.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:51.189+0000 7f88e110f640 1 -- 172.21.15.38:0/3455994640 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f88dc152f20 msgr2=0x7f88dc142820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:51.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:51.189+0000 7f88e110f640 1 --2- 172.21.15.38:0/3455994640 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f88dc152f20 0x7f88dc142820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:51.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:51.189+0000 7f88e110f640 1 -- 172.21.15.38:0/3455994640 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88dc156c40 con 0x7f88dc06cde0 2024-09-21T03:38:51.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:51.190+0000 7f88e110f640 1 --2- 172.21.15.38:0/3455994640 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f88dc06cde0 0x7f88dc06d1f0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f88c402b160 tx=0x7f88c40011d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:51.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:51.190+0000 7f88d97fa640 1 -- 172.21.15.38:0/3455994640 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88c4031050 con 0x7f88dc06cde0 2024-09-21T03:38:51.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:51.190+0000 7f88d97fa640 1 -- 172.21.15.38:0/3455994640 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f88c402d670 con 0x7f88dc06cde0 2024-09-21T03:38:51.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:51.190+0000 7f88e2b72640 1 -- 172.21.15.38:0/3455994640 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88dc06d730 con 0x7f88dc06cde0 2024-09-21T03:38:51.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:51.190+0000 7f88d97fa640 1 -- 172.21.15.38:0/3455994640 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88c402dac0 con 0x7f88dc06cde0 2024-09-21T03:38:51.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:51.190+0000 7f88e2b72640 1 -- 172.21.15.38:0/3455994640 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88dc156960 con 0x7f88dc06cde0 2024-09-21T03:38:51.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:51.191+0000 7f88e2b72640 1 -- 172.21.15.38:0/3455994640 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88dc06e970 con 0x7f88dc06cde0 2024-09-21T03:38:51.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:51.192+0000 7f88d97fa640 1 -- 172.21.15.38:0/3455994640 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f88c4034070 con 0x7f88dc06cde0 2024-09-21T03:38:51.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:51.192+0000 7f88d97fa640 1 --2- 172.21.15.38:0/3455994640 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f88bc078bc0 0x7f88bc07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:51.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:51.192+0000 7f88d97fa640 1 -- 172.21.15.38:0/3455994640 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 6476+0+0 (secure 0 0 0) 0x7f88c40bad10 con 0x7f88dc06cde0 2024-09-21T03:38:51.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:51.193+0000 7f88e090e640 1 --2- 172.21.15.38:0/3455994640 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f88bc078bc0 0x7f88bc07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:51.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:51.193+0000 7f88e090e640 1 --2- 172.21.15.38:0/3455994640 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f88bc078bc0 0x7f88bc07b0d0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f88d4006680 tx=0x7f88d4006b70 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:51.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:51.197+0000 7f88d97fa640 1 -- 172.21.15.38:0/3455994640 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f88c40c0020 con 0x7f88dc06cde0 2024-09-21T03:38:51.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:51.423+0000 7f88e2b72640 1 -- 172.21.15.38:0/3455994640 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f88dc15b950 con 0x7f88dc06cde0 2024-09-21T03:38:51.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:51.423+0000 7f88d97fa640 1 -- 172.21.15.38:0/3455994640 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) ==== 94+0+5232 (secure 0 0 0) 0x7f88c4045020 con 0x7f88dc06cde0 2024-09-21T03:38:51.425 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:38:51.425 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":59,"btime":"2024-09-21T03:37:15:051941+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5432,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/1042764291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":1042764291},{"type":"v1","addr":"172.21.15.195:6837","nonce":1042764291}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":5728,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/3035019482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6833","nonce":3035019482},{"type":"v1","addr":"172.21.15.38:6835","nonce":3035019482}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":6348,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6834/3775275137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":3775275137},{"type":"v1","addr":"172.21.15.38:6834","nonce":3775275137}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":59,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:36:21.442724+0000","modified":"2024-09-21T03:37:15.051929+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":69,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5731},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5731":{"gid":5731,"name":"b","rank":0,"incarnation":56,"state":"up:active","state_seq":17,"addr":"172.21.15.195:6836/2536177898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":2536177898},{"type":"v1","addr":"172.21.15.195:6836","nonce":2536177898}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5731,"qdb_cluster":[5731]},"id":4}]} 2024-09-21T03:38:51.426 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 59 2024-09-21T03:38:51.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:51.425+0000 7f88e2b72640 1 -- 172.21.15.38:0/3455994640 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f88bc078bc0 msgr2=0x7f88bc07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:51.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:51.425+0000 7f88e2b72640 1 --2- 172.21.15.38:0/3455994640 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f88bc078bc0 0x7f88bc07b0d0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f88d4006680 tx=0x7f88d4006b70 comp rx=0 tx=0).stop 2024-09-21T03:38:51.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:51.425+0000 7f88e2b72640 1 -- 172.21.15.38:0/3455994640 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f88dc06cde0 msgr2=0x7f88dc06d1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:51.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:51.425+0000 7f88e2b72640 1 --2- 172.21.15.38:0/3455994640 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f88dc06cde0 0x7f88dc06d1f0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f88c402b160 tx=0x7f88c40011d0 comp rx=0 tx=0).stop 2024-09-21T03:38:51.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:51.426+0000 7f88e2b72640 1 -- 172.21.15.38:0/3455994640 shutdown_connections 2024-09-21T03:38:51.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:51.426+0000 7f88e2b72640 1 --2- 172.21.15.38:0/3455994640 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f88dc06cde0 0x7f88dc06d1f0 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:51.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:51.426+0000 7f88e2b72640 1 --2- 172.21.15.38:0/3455994640 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f88dc152f20 0x7f88dc142820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:51.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:51.426+0000 7f88e2b72640 1 --2- 172.21.15.38:0/3455994640 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f88bc078bc0 0x7f88bc07b0d0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:51.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:51.426+0000 7f88e2b72640 1 --2- 172.21.15.38:0/3455994640 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f88c8005270 0x7f88dc1f5150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:51.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:51.426+0000 7f88e2b72640 1 -- 172.21.15.38:0/3455994640 >> 172.21.15.38:0/3455994640 conn(0x7f88dc0b2cc0 msgr2=0x7f88dc159bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:51.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:51.426+0000 7f88e2b72640 1 -- 172.21.15.38:0/3455994640 shutdown_connections 2024-09-21T03:38:51.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:51.426+0000 7f88e2b72640 1 -- 172.21.15.38:0/3455994640 wait complete. 2024-09-21T03:38:51.439 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T03:38:51.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:51.554+0000 7f0821a7f640 1 -- 172.21.15.38:0/1267292539 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0804003c10 msgr2=0x7f0804004060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:51.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:51.554+0000 7f0821a7f640 1 --2- 172.21.15.38:0/1267292539 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0804003c10 0x7f0804004060 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f081c0d2460 tx=0x7f0804005830 comp rx=0 tx=0).stop 2024-09-21T03:38:51.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:51.554+0000 7f081bfff640 1 -- 172.21.15.38:0/1267292539 reap_dead start 2024-09-21T03:38:51.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:51.554+0000 7f0821a7f640 1 -- 172.21.15.38:0/1267292539 shutdown_connections 2024-09-21T03:38:51.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:51.554+0000 7f0821a7f640 1 -- 172.21.15.38:0/1267292539 >> 172.21.15.38:0/1267292539 conn(0x7f081c0b6b20 msgr2=0x7f081c0d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:51.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:51.554+0000 7f0821a7f640 1 -- 172.21.15.38:0/1267292539 shutdown_connections 2024-09-21T03:38:51.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:51.554+0000 7f0821a7f640 1 -- 172.21.15.38:0/1267292539 wait complete. 2024-09-21T03:38:51.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:51.556+0000 7f0821a7f640 1 Processor -- start 2024-09-21T03:38:51.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:51.556+0000 7f0821a7f640 1 -- start start 2024-09-21T03:38:51.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:51.557+0000 7f0821a7f640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0804003c10 0x7f081c1e8e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:51.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:51.557+0000 7f0821a7f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f081c1f9190 0x7f081c0b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:51.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:51.557+0000 7f0821a7f640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f081c0b9040 0x7f081c146e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:51.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:51.557+0000 7f0821a7f640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f081c15f640 con 0x7f081c1f9190 2024-09-21T03:38:51.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:51.557+0000 7f0821a7f640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f081c15f4c0 con 0x7f081c0b9040 2024-09-21T03:38:51.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:51.557+0000 7f0821a7f640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f081c15f7c0 con 0x7f0804003c10 2024-09-21T03:38:51.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:51.557+0000 7f081affd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f081c1f9190 0x7f081c0b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:51.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:51.557+0000 7f081affd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f081c1f9190 0x7f081c0b8b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48412/0 (socket says 172.21.15.38:48412) 2024-09-21T03:38:51.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:51.557+0000 7f081affd640 1 -- 172.21.15.38:0/1959314430 learned_addr learned my addr 172.21.15.38:0/1959314430 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:38:51.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:51.558+0000 7f081bfff640 1 --2- 172.21.15.38:0/1959314430 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f081c0b9040 0x7f081c146e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:51.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:51.558+0000 7f081b7fe640 1 --2- 172.21.15.38:0/1959314430 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0804003c10 0x7f081c1e8e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:51.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:51.558+0000 7f081affd640 1 -- 172.21.15.38:0/1959314430 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0804003c10 msgr2=0x7f081c1e8e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:51.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:51.558+0000 7f081affd640 1 --2- 172.21.15.38:0/1959314430 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0804003c10 0x7f081c1e8e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:51.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:51.558+0000 7f081affd640 1 -- 172.21.15.38:0/1959314430 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f081c0b9040 msgr2=0x7f081c146e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:51.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:51.558+0000 7f081affd640 1 --2- 172.21.15.38:0/1959314430 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f081c0b9040 0x7f081c146e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:51.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:51.558+0000 7f081affd640 1 -- 172.21.15.38:0/1959314430 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f081c0cb0d0 con 0x7f081c1f9190 2024-09-21T03:38:51.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:51.558+0000 7f081bfff640 1 --2- 172.21.15.38:0/1959314430 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f081c0b9040 0x7f081c146e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:38:51.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:51.558+0000 7f081affd640 1 --2- 172.21.15.38:0/1959314430 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f081c1f9190 0x7f081c0b8b00 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f0810006650 tx=0x7f0810001800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:51.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:51.558+0000 7f0803fff640 1 -- 172.21.15.38:0/1959314430 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0810011700 con 0x7f081c1f9190 2024-09-21T03:38:51.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:51.558+0000 7f0803fff640 1 -- 172.21.15.38:0/1959314430 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0810016020 con 0x7f081c1f9190 2024-09-21T03:38:51.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:51.558+0000 7f0821a7f640 1 -- 172.21.15.38:0/1959314430 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f081c0cb5d0 con 0x7f081c1f9190 2024-09-21T03:38:51.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:51.558+0000 7f0803fff640 1 -- 172.21.15.38:0/1959314430 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0810011b00 con 0x7f081c1f9190 2024-09-21T03:38:51.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:51.558+0000 7f0821a7f640 1 -- 172.21.15.38:0/1959314430 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f081c1e8690 con 0x7f081c1f9190 2024-09-21T03:38:51.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:51.559+0000 7f0821a7f640 1 -- 172.21.15.38:0/1959314430 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f081c0b3250 con 0x7f081c1f9190 2024-09-21T03:38:51.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:51.560+0000 7f0803fff640 1 -- 172.21.15.38:0/1959314430 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f0810011ca0 con 0x7f081c1f9190 2024-09-21T03:38:51.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:51.560+0000 7f0803fff640 1 --2- 172.21.15.38:0/1959314430 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0808078b90 0x7f080807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:51.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:51.560+0000 7f0803fff640 1 -- 172.21.15.38:0/1959314430 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 6476+0+0 (secure 0 0 0) 0x7f081009bcd0 con 0x7f081c1f9190 2024-09-21T03:38:51.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:51.561+0000 7f081b7fe640 1 --2- 172.21.15.38:0/1959314430 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0808078b90 0x7f080807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:51.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:51.561+0000 7f081b7fe640 1 --2- 172.21.15.38:0/1959314430 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0808078b90 0x7f080807b0a0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f080c035000 tx=0x7f080c02d040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:51.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:51.565+0000 7f0803fff640 1 -- 172.21.15.38:0/1959314430 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f0810020020 con 0x7f081c1f9190 2024-09-21T03:38:51.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:51.754+0000 7f0821a7f640 1 -- 172.21.15.38:0/1959314430 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f081c0cb360 con 0x7f081c1f9190 2024-09-21T03:38:51.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:51.755+0000 7f0803fff640 1 -- 172.21.15.38:0/1959314430 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) ==== 74+0+17612 (secure 0 0 0) 0x7f0810063e20 con 0x7f081c1f9190 2024-09-21T03:38:51.757 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:38:51.757 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":71,"fsid":"9d50d66f-656b-4fe2-8e3b-fe53b79c4d43","created":"2024-09-21T03:33:10.677865+0000","modified":"2024-09-21T03:37:17.609008+0000","last_up_change":"2024-09-21T03:33:24.023421+0000","last_in_change":"2024-09-21T03:33:14.296193+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T03:33:22.992573+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-09-21T03:36:19.109965+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-09-21T03:36:20.392299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dbab183f-7e63-48b8-960a-ae55fd5bf565","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6801","nonce":2381639433}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6803","nonce":2381639433}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6807","nonce":2381639433}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6805","nonce":2381639433}]},"public_addr":"172.21.15.38:6801/2381639433","cluster_addr":"172.21.15.38:6803/2381639433","heartbeat_back_addr":"172.21.15.38:6807/2381639433","heartbeat_front_addr":"172.21.15.38:6805/2381639433","state":["exists","up"]},{"osd":1,"uuid":"fa1c0e8a-ebc1-4359-8146-bd5342abbc16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6817","nonce":1491818732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6819","nonce":1491818732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6823","nonce":1491818732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6821","nonce":1491818732}]},"public_addr":"172.21.15.38:6817/1491818732","cluster_addr":"172.21.15.38:6819/1491818732","heartbeat_back_addr":"172.21.15.38:6823/1491818732","heartbeat_front_addr":"172.21.15.38:6821/1491818732","state":["exists","up"]},{"osd":2,"uuid":"2ed3cd2d-385d-4fc6-9869-214552a84803","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6825","nonce":2577858902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6827","nonce":2577858902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6831","nonce":2577858902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6829","nonce":2577858902}]},"public_addr":"172.21.15.38:6825/2577858902","cluster_addr":"172.21.15.38:6827/2577858902","heartbeat_back_addr":"172.21.15.38:6831/2577858902","heartbeat_front_addr":"172.21.15.38:6829/2577858902","state":["exists","up"]},{"osd":3,"uuid":"62d1d95e-919d-4250-be71-1d34cfa70dbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6809","nonce":1286810294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6811","nonce":1286810294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6815","nonce":1286810294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6813","nonce":1286810294}]},"public_addr":"172.21.15.38:6809/1286810294","cluster_addr":"172.21.15.38:6811/1286810294","heartbeat_back_addr":"172.21.15.38:6815/1286810294","heartbeat_front_addr":"172.21.15.38:6813/1286810294","state":["exists","up"]},{"osd":4,"uuid":"7c9f9314-99d7-4d73-8392-37809a9f54b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6803","nonce":3401837571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6805","nonce":3401837571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6809","nonce":3401837571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6807","nonce":3401837571}]},"public_addr":"172.21.15.195:6803/3401837571","cluster_addr":"172.21.15.195:6805/3401837571","heartbeat_back_addr":"172.21.15.195:6809/3401837571","heartbeat_front_addr":"172.21.15.195:6807/3401837571","state":["exists","up"]},{"osd":5,"uuid":"bde4fe9f-8414-4bba-bf4a-43973d1976bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6811","nonce":2080960180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6813","nonce":2080960180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6817","nonce":2080960180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6815","nonce":2080960180}]},"public_addr":"172.21.15.195:6811/2080960180","cluster_addr":"172.21.15.195:6813/2080960180","heartbeat_back_addr":"172.21.15.195:6817/2080960180","heartbeat_front_addr":"172.21.15.195:6815/2080960180","state":["exists","up"]},{"osd":6,"uuid":"6ee6d7f4-1a5f-4a4c-ac16-48fe05f6a002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6827","nonce":4055825675}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6829","nonce":4055825675}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6833","nonce":4055825675}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6831","nonce":4055825675}]},"public_addr":"172.21.15.195:6827/4055825675","cluster_addr":"172.21.15.195:6829/4055825675","heartbeat_back_addr":"172.21.15.195:6833/4055825675","heartbeat_front_addr":"172.21.15.195:6831/4055825675","state":["exists","up"]},{"osd":7,"uuid":"efeb5f51-329a-4484-8290-92e7d3503d05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6819","nonce":1606106342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6821","nonce":1606106342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6825","nonce":1606106342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6823","nonce":1606106342}]},"public_addr":"172.21.15.195:6819/1606106342","cluster_addr":"172.21.15.195:6821/1606106342","heartbeat_back_addr":"172.21.15.195:6825/1606106342","heartbeat_front_addr":"172.21.15.195:6823/1606106342","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.901343+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:21.115003+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.901623+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.835238+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.132199+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:19.817560+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.829690+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:21.087300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"10.3","mappings":[{"from":4,"to":1}]},{"pgid":"10.1a","mappings":[{"from":2,"to":7}]},{"pgid":"10.1c","mappings":[{"from":2,"to":7}]},{"pgid":"10.22","mappings":[{"from":2,"to":5}]},{"pgid":"10.23","mappings":[{"from":2,"to":5}]},{"pgid":"10.2e","mappings":[{"from":2,"to":7}]},{"pgid":"10.33","mappings":[{"from":2,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.38:6834/1058376326":"2024-09-22T03:37:12.044790+0000","172.21.15.38:6832/1058376326":"2024-09-22T03:37:12.044790+0000","172.21.15.38:6834/1353335665":"2024-09-22T03:36:36.420530+0000","172.21.15.38:6832/1353335665":"2024-09-22T03:36:36.420530+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T03:38:51.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:51.757+0000 7f0821a7f640 1 -- 172.21.15.38:0/1959314430 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0808078b90 msgr2=0x7f080807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:51.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:51.757+0000 7f0821a7f640 1 --2- 172.21.15.38:0/1959314430 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0808078b90 0x7f080807b0a0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f080c035000 tx=0x7f080c02d040 comp rx=0 tx=0).stop 2024-09-21T03:38:51.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:51.757+0000 7f0821a7f640 1 -- 172.21.15.38:0/1959314430 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f081c1f9190 msgr2=0x7f081c0b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:51.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:51.757+0000 7f0821a7f640 1 --2- 172.21.15.38:0/1959314430 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f081c1f9190 0x7f081c0b8b00 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f0810006650 tx=0x7f0810001800 comp rx=0 tx=0).stop 2024-09-21T03:38:51.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:51.758+0000 7f0821a7f640 1 -- 172.21.15.38:0/1959314430 shutdown_connections 2024-09-21T03:38:51.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:51.758+0000 7f0821a7f640 1 --2- 172.21.15.38:0/1959314430 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f081c0b9040 0x7f081c146e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:51.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:51.758+0000 7f0821a7f640 1 --2- 172.21.15.38:0/1959314430 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f081c1f9190 0x7f081c0b8b00 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:51.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:51.758+0000 7f0821a7f640 1 --2- 172.21.15.38:0/1959314430 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0808078b90 0x7f080807b0a0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:51.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:51.758+0000 7f0821a7f640 1 --2- 172.21.15.38:0/1959314430 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0804003c10 0x7f081c1e8e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:51.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:51.758+0000 7f0821a7f640 1 -- 172.21.15.38:0/1959314430 >> 172.21.15.38:0/1959314430 conn(0x7f081c0b6b20 msgr2=0x7f081c06cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:51.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:51.758+0000 7f0821a7f640 1 -- 172.21.15.38:0/1959314430 shutdown_connections 2024-09-21T03:38:51.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:51.758+0000 7f0821a7f640 1 -- 172.21.15.38:0/1959314430 wait complete. 2024-09-21T03:38:51.771 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-21T03:38:51.771 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:38:51.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:51.886+0000 7f5ebd101640 1 -- 172.21.15.38:0/2965559485 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5eb80d19d0 msgr2=0x7f5eb815c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:51.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:51.886+0000 7f5ebd101640 1 --2- 172.21.15.38:0/2965559485 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5eb80d19d0 0x7f5eb815c6a0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f5eb8074e90 tx=0x7f5ea802a560 comp rx=0 tx=0).stop 2024-09-21T03:38:51.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:51.886+0000 7f5ebd101640 1 -- 172.21.15.38:0/2965559485 shutdown_connections 2024-09-21T03:38:51.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:51.886+0000 7f5ebd101640 1 --2- 172.21.15.38:0/2965559485 >> v2:172.21.15.195:3300/0 conn(0x7f5eb80b2500 0x7f5eb80ccac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:51.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:51.886+0000 7f5ebd101640 1 --2- 172.21.15.38:0/2965559485 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5eb80d19d0 0x7f5eb815c6a0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:51.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:51.886+0000 7f5ebd101640 1 -- 172.21.15.38:0/2965559485 >> 172.21.15.38:0/2965559485 conn(0x7f5eb80b41d0 msgr2=0x7f5eb80b45e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:51.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:51.886+0000 7f5ebd101640 1 -- 172.21.15.38:0/2965559485 shutdown_connections 2024-09-21T03:38:51.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:51.886+0000 7f5ebd101640 1 -- 172.21.15.38:0/2965559485 wait complete. 2024-09-21T03:38:51.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:51.888+0000 7f5ebd101640 1 Processor -- start 2024-09-21T03:38:51.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:51.888+0000 7f5ebd101640 1 -- start start 2024-09-21T03:38:51.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:51.889+0000 7f5ebd101640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5eb80b2500 0x7f5eb8158e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:51.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:51.889+0000 7f5ebd101640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5eb80cd000 0x7f5eb81f87d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:51.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:51.889+0000 7f5ebd101640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5eb80d19d0 0x7f5eb81f8d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:51.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:51.889+0000 7f5ebd101640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f5eb80d08e0 con 0x7f5eb80d19d0 2024-09-21T03:38:51.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:51.889+0000 7f5ebd101640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f5eb80b67b0 con 0x7f5eb80b2500 2024-09-21T03:38:51.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:51.889+0000 7f5ebd101640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f5eb80d0a60 con 0x7f5eb80cd000 2024-09-21T03:38:51.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:51.890+0000 7f5eb759e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5eb80d19d0 0x7f5eb81f8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:51.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:51.890+0000 7f5eb759e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5eb80d19d0 0x7f5eb81f8d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48430/0 (socket says 172.21.15.38:48430) 2024-09-21T03:38:51.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:51.890+0000 7f5eb759e640 1 -- 172.21.15.38:0/4000170521 learned_addr learned my addr 172.21.15.38:0/4000170521 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:38:51.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:51.890+0000 7f5eb6d9d640 1 --2- 172.21.15.38:0/4000170521 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5eb80b2500 0x7f5eb8158e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:51.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:51.890+0000 7f5eb659c640 1 --2- 172.21.15.38:0/4000170521 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5eb80cd000 0x7f5eb81f87d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:51.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:51.890+0000 7f5eb759e640 1 -- 172.21.15.38:0/4000170521 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5eb80cd000 msgr2=0x7f5eb81f87d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:51.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:51.890+0000 7f5eb759e640 1 --2- 172.21.15.38:0/4000170521 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5eb80cd000 0x7f5eb81f87d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:51.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:51.890+0000 7f5eb759e640 1 -- 172.21.15.38:0/4000170521 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5eb80b2500 msgr2=0x7f5eb8158e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:51.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:51.890+0000 7f5eb759e640 1 --2- 172.21.15.38:0/4000170521 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5eb80b2500 0x7f5eb8158e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:51.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:51.890+0000 7f5eb759e640 1 -- 172.21.15.38:0/4000170521 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5eb80cbdc0 con 0x7f5eb80d19d0 2024-09-21T03:38:51.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:51.890+0000 7f5eb759e640 1 --2- 172.21.15.38:0/4000170521 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5eb80d19d0 0x7f5eb81f8d10 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f5eb000ba10 tx=0x7f5eb000bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:51.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:51.890+0000 7f5e9f7fe640 1 -- 172.21.15.38:0/4000170521 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eb0019070 con 0x7f5eb80d19d0 2024-09-21T03:38:51.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:51.890+0000 7f5e9f7fe640 1 -- 172.21.15.38:0/4000170521 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5eb000cc30 con 0x7f5eb80d19d0 2024-09-21T03:38:51.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:51.890+0000 7f5ebd101640 1 -- 172.21.15.38:0/4000170521 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5eb81f93c0 con 0x7f5eb80d19d0 2024-09-21T03:38:51.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:51.890+0000 7f5e9f7fe640 1 -- 172.21.15.38:0/4000170521 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eb000ce60 con 0x7f5eb80d19d0 2024-09-21T03:38:51.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:51.890+0000 7f5ebd101640 1 -- 172.21.15.38:0/4000170521 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5eb80cc060 con 0x7f5eb80d19d0 2024-09-21T03:38:51.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:51.891+0000 7f5ebd101640 1 -- 172.21.15.38:0/4000170521 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5eb80d0f10 con 0x7f5eb80d19d0 2024-09-21T03:38:51.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:51.892+0000 7f5e9f7fe640 1 -- 172.21.15.38:0/4000170521 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f5eb001c070 con 0x7f5eb80d19d0 2024-09-21T03:38:51.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:51.893+0000 7f5e9f7fe640 1 --2- 172.21.15.38:0/4000170521 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5ea4078bc0 0x7f5ea407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:51.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:51.893+0000 7f5e9f7fe640 1 -- 172.21.15.38:0/4000170521 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 6476+0+0 (secure 0 0 0) 0x7f5eb009ab90 con 0x7f5eb80d19d0 2024-09-21T03:38:51.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:51.893+0000 7f5eb6d9d640 1 --2- 172.21.15.38:0/4000170521 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5ea4078bc0 0x7f5ea407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:51.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:51.893+0000 7f5eb6d9d640 1 --2- 172.21.15.38:0/4000170521 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5ea4078bc0 0x7f5ea407b0d0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f5ea8033000 tx=0x7f5ea8004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:51.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:51.897+0000 7f5e9f7fe640 1 -- 172.21.15.38:0/4000170521 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f5eb0062670 con 0x7f5eb80d19d0 2024-09-21T03:38:52.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:52.125+0000 7f5ebd101640 1 -- 172.21.15.38:0/4000170521 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5eb81f96a0 con 0x7f5eb80d19d0 2024-09-21T03:38:52.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:52.126+0000 7f5e9f7fe640 1 -- 172.21.15.38:0/4000170521 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) ==== 94+0+5232 (secure 0 0 0) 0x7f5eb00670a0 con 0x7f5eb80d19d0 2024-09-21T03:38:52.128 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:38:52.128 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":59,"btime":"2024-09-21T03:37:15:051941+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5432,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/1042764291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":1042764291},{"type":"v1","addr":"172.21.15.195:6837","nonce":1042764291}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":5728,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/3035019482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6833","nonce":3035019482},{"type":"v1","addr":"172.21.15.38:6835","nonce":3035019482}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":6348,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6834/3775275137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":3775275137},{"type":"v1","addr":"172.21.15.38:6834","nonce":3775275137}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":59,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:36:21.442724+0000","modified":"2024-09-21T03:37:15.051929+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":69,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5731},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5731":{"gid":5731,"name":"b","rank":0,"incarnation":56,"state":"up:active","state_seq":17,"addr":"172.21.15.195:6836/2536177898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":2536177898},{"type":"v1","addr":"172.21.15.195:6836","nonce":2536177898}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5731,"qdb_cluster":[5731]},"id":4}]} 2024-09-21T03:38:52.128 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 59 2024-09-21T03:38:52.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:52.128+0000 7f5ebd101640 1 -- 172.21.15.38:0/4000170521 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5ea4078bc0 msgr2=0x7f5ea407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:52.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:52.128+0000 7f5ebd101640 1 --2- 172.21.15.38:0/4000170521 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5ea4078bc0 0x7f5ea407b0d0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f5ea8033000 tx=0x7f5ea8004730 comp rx=0 tx=0).stop 2024-09-21T03:38:52.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:52.128+0000 7f5ebd101640 1 -- 172.21.15.38:0/4000170521 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5eb80d19d0 msgr2=0x7f5eb81f8d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:52.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:52.128+0000 7f5ebd101640 1 --2- 172.21.15.38:0/4000170521 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5eb80d19d0 0x7f5eb81f8d10 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f5eb000ba10 tx=0x7f5eb000bed0 comp rx=0 tx=0).stop 2024-09-21T03:38:52.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:52.128+0000 7f5ebd101640 1 -- 172.21.15.38:0/4000170521 shutdown_connections 2024-09-21T03:38:52.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:52.128+0000 7f5ebd101640 1 --2- 172.21.15.38:0/4000170521 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5eb80d19d0 0x7f5eb81f8d10 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:52.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:52.128+0000 7f5ebd101640 1 --2- 172.21.15.38:0/4000170521 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5ea4078bc0 0x7f5ea407b0d0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:52.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:52.128+0000 7f5ebd101640 1 --2- 172.21.15.38:0/4000170521 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5eb80cd000 0x7f5eb81f87d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:52.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:52.128+0000 7f5ebd101640 1 --2- 172.21.15.38:0/4000170521 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5eb80b2500 0x7f5eb8158e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:52.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:52.128+0000 7f5ebd101640 1 -- 172.21.15.38:0/4000170521 >> 172.21.15.38:0/4000170521 conn(0x7f5eb80b41d0 msgr2=0x7f5eb80b45e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:52.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:52.128+0000 7f5ebd101640 1 -- 172.21.15.38:0/4000170521 shutdown_connections 2024-09-21T03:38:52.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:52.128+0000 7f5ebd101640 1 -- 172.21.15.38:0/4000170521 wait complete. 2024-09-21T03:38:52.141 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:38:52.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:52.255+0000 7f2ee1a70640 1 -- 172.21.15.38:0/1004324801 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2ec4004800 msgr2=0x7f2ec4004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:52.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:52.255+0000 7f2ee1a70640 1 --2- 172.21.15.38:0/1004324801 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2ec4004800 0x7f2ec4004c10 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f2ed00016c0 tx=0x7f2ec4005830 comp rx=0 tx=0).stop 2024-09-21T03:38:52.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:52.255+0000 7f2ee1a70640 1 -- 172.21.15.38:0/1004324801 shutdown_connections 2024-09-21T03:38:52.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:52.255+0000 7f2ee1a70640 1 --2- 172.21.15.38:0/1004324801 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2ec4004800 0x7f2ec4004c10 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:52.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:52.255+0000 7f2ee1a70640 1 -- 172.21.15.38:0/1004324801 >> 172.21.15.38:0/1004324801 conn(0x7f2edc0d2350 msgr2=0x7f2edc0d2760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:52.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:52.256+0000 7f2ee1a70640 1 -- 172.21.15.38:0/1004324801 shutdown_connections 2024-09-21T03:38:52.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:52.256+0000 7f2ee1a70640 1 -- 172.21.15.38:0/1004324801 wait complete. 2024-09-21T03:38:52.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:52.257+0000 7f2ee1a70640 1 Processor -- start 2024-09-21T03:38:52.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:52.258+0000 7f2ee1a70640 1 -- start start 2024-09-21T03:38:52.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:52.259+0000 7f2ee1a70640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2ec4004800 0x7f2edc148b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:52.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:52.259+0000 7f2ee1a70640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2edc0c8790 0x7f2edc0c8ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:52.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:52.259+0000 7f2ee1a70640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2edc15bec0 0x7f2edc15c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:52.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:52.259+0000 7f2ee1a70640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f2edc15d2b0 con 0x7f2edc0c8790 2024-09-21T03:38:52.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:52.259+0000 7f2ee1a70640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f2edc15d130 con 0x7f2edc15bec0 2024-09-21T03:38:52.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:52.259+0000 7f2ee1a70640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f2edc0d16f0 con 0x7f2ec4004800 2024-09-21T03:38:52.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:52.259+0000 7f2edaffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2edc0c8790 0x7f2edc0c8ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:52.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:52.259+0000 7f2edaffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2edc0c8790 0x7f2edc0c8ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48456/0 (socket says 172.21.15.38:48456) 2024-09-21T03:38:52.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:52.259+0000 7f2edaffd640 1 -- 172.21.15.38:0/578946826 learned_addr learned my addr 172.21.15.38:0/578946826 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:38:52.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:52.259+0000 7f2edb7fe640 1 --2- 172.21.15.38:0/578946826 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2ec4004800 0x7f2edc148b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:52.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:52.259+0000 7f2edbfff640 1 --2- 172.21.15.38:0/578946826 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2edc15bec0 0x7f2edc15c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:52.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:52.259+0000 7f2edaffd640 1 -- 172.21.15.38:0/578946826 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2ec4004800 msgr2=0x7f2edc148b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:52.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:52.259+0000 7f2edaffd640 1 --2- 172.21.15.38:0/578946826 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2ec4004800 0x7f2edc148b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:52.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:52.259+0000 7f2edaffd640 1 -- 172.21.15.38:0/578946826 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2edc15bec0 msgr2=0x7f2edc15c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:52.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:52.259+0000 7f2edaffd640 1 --2- 172.21.15.38:0/578946826 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2edc15bec0 0x7f2edc15c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:52.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:52.259+0000 7f2edaffd640 1 -- 172.21.15.38:0/578946826 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2edc15c810 con 0x7f2edc0c8790 2024-09-21T03:38:52.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:52.259+0000 7f2edaffd640 1 --2- 172.21.15.38:0/578946826 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2edc0c8790 0x7f2edc0c8ba0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f2ed002a910 tx=0x7f2ed00063a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:52.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:52.260+0000 7f2ec3fff640 1 -- 172.21.15.38:0/578946826 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ed0003a00 con 0x7f2edc0c8790 2024-09-21T03:38:52.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:52.260+0000 7f2ec3fff640 1 -- 172.21.15.38:0/578946826 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2ed0039070 con 0x7f2edc0c8790 2024-09-21T03:38:52.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:52.260+0000 7f2ee1a70640 1 -- 172.21.15.38:0/578946826 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2edc1f3920 con 0x7f2edc0c8790 2024-09-21T03:38:52.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:52.260+0000 7f2ec3fff640 1 -- 172.21.15.38:0/578946826 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ed0033420 con 0x7f2edc0c8790 2024-09-21T03:38:52.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:52.260+0000 7f2ee1a70640 1 -- 172.21.15.38:0/578946826 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2edc1f3ee0 con 0x7f2edc0c8790 2024-09-21T03:38:52.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:52.261+0000 7f2ee1a70640 1 -- 172.21.15.38:0/578946826 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2edc0d1c50 con 0x7f2edc0c8790 2024-09-21T03:38:52.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:52.261+0000 7f2ec3fff640 1 -- 172.21.15.38:0/578946826 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f2ed0044020 con 0x7f2edc0c8790 2024-09-21T03:38:52.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:52.262+0000 7f2ec3fff640 1 --2- 172.21.15.38:0/578946826 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2ec8078b90 0x7f2ec807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:52.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:52.262+0000 7f2ec3fff640 1 -- 172.21.15.38:0/578946826 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 6476+0+0 (secure 0 0 0) 0x7f2ed00ba9d0 con 0x7f2edc0c8790 2024-09-21T03:38:52.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:52.262+0000 7f2edb7fe640 1 --2- 172.21.15.38:0/578946826 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2ec8078b90 0x7f2ec807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:52.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:52.263+0000 7f2edb7fe640 1 --2- 172.21.15.38:0/578946826 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2ec8078b90 0x7f2ec807b0a0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc009c10 tx=0x7f2ecc005f90 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:52.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:52.267+0000 7f2ec3fff640 1 -- 172.21.15.38:0/578946826 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f2edc15c810 con 0x7f2edc0c8790 2024-09-21T03:38:52.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:52.490+0000 7f2ee1a70640 1 -- 172.21.15.38:0/578946826 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2edc15f310 con 0x7f2edc0c8790 2024-09-21T03:38:52.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:52.491+0000 7f2ec3fff640 1 -- 172.21.15.38:0/578946826 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) ==== 94+0+5232 (secure 0 0 0) 0x7f2ed0082b10 con 0x7f2edc0c8790 2024-09-21T03:38:52.493 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:38:52.493 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":59,"btime":"2024-09-21T03:37:15:051941+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5432,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/1042764291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":1042764291},{"type":"v1","addr":"172.21.15.195:6837","nonce":1042764291}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":5728,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/3035019482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6833","nonce":3035019482},{"type":"v1","addr":"172.21.15.38:6835","nonce":3035019482}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":6348,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6834/3775275137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":3775275137},{"type":"v1","addr":"172.21.15.38:6834","nonce":3775275137}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":59,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:36:21.442724+0000","modified":"2024-09-21T03:37:15.051929+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":69,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5731},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5731":{"gid":5731,"name":"b","rank":0,"incarnation":56,"state":"up:active","state_seq":17,"addr":"172.21.15.195:6836/2536177898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":2536177898},{"type":"v1","addr":"172.21.15.195:6836","nonce":2536177898}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5731,"qdb_cluster":[5731]},"id":4}]} 2024-09-21T03:38:52.493 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 59 2024-09-21T03:38:52.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:52.493+0000 7f2ee1a70640 1 -- 172.21.15.38:0/578946826 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2ec8078b90 msgr2=0x7f2ec807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:52.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:52.493+0000 7f2ee1a70640 1 --2- 172.21.15.38:0/578946826 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2ec8078b90 0x7f2ec807b0a0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc009c10 tx=0x7f2ecc005f90 comp rx=0 tx=0).stop 2024-09-21T03:38:52.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:52.493+0000 7f2ee1a70640 1 -- 172.21.15.38:0/578946826 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2edc0c8790 msgr2=0x7f2edc0c8ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:52.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:52.493+0000 7f2ee1a70640 1 --2- 172.21.15.38:0/578946826 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2edc0c8790 0x7f2edc0c8ba0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f2ed002a910 tx=0x7f2ed00063a0 comp rx=0 tx=0).stop 2024-09-21T03:38:52.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:52.493+0000 7f2ee1a70640 1 -- 172.21.15.38:0/578946826 shutdown_connections 2024-09-21T03:38:52.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:52.493+0000 7f2ee1a70640 1 --2- 172.21.15.38:0/578946826 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2edc15bec0 0x7f2edc15c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:52.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:52.493+0000 7f2ee1a70640 1 --2- 172.21.15.38:0/578946826 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2edc0c8790 0x7f2edc0c8ba0 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:52.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:52.493+0000 7f2ee1a70640 1 --2- 172.21.15.38:0/578946826 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2ec8078b90 0x7f2ec807b0a0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:52.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:52.493+0000 7f2ee1a70640 1 --2- 172.21.15.38:0/578946826 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2ec4004800 0x7f2edc148b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:52.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:52.493+0000 7f2ee1a70640 1 -- 172.21.15.38:0/578946826 >> 172.21.15.38:0/578946826 conn(0x7f2edc0d2350 msgr2=0x7f2edc0728a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:52.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:52.493+0000 7f2ee1a70640 1 -- 172.21.15.38:0/578946826 shutdown_connections 2024-09-21T03:38:52.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:52.493+0000 7f2ee1a70640 1 -- 172.21.15.38:0/578946826 wait complete. 2024-09-21T03:38:52.507 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T03:38:52.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:52.621+0000 7f9c59520640 1 -- 172.21.15.38:0/1589173654 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9c54162280 msgr2=0x7f9c54162690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:52.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:52.621+0000 7f9c59520640 1 --2- 172.21.15.38:0/1589173654 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9c54162280 0x7f9c54162690 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f9c3c009830 tx=0x7f9c3c02a510 comp rx=0 tx=0).stop 2024-09-21T03:38:52.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:52.622+0000 7f9c59520640 1 -- 172.21.15.38:0/1589173654 shutdown_connections 2024-09-21T03:38:52.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:52.622+0000 7f9c59520640 1 --2- 172.21.15.38:0/1589173654 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9c54162280 0x7f9c54162690 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:52.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:52.622+0000 7f9c59520640 1 --2- 172.21.15.38:0/1589173654 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9c54162f00 0x7f9c540d24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:52.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:52.622+0000 7f9c59520640 1 -- 172.21.15.38:0/1589173654 >> 172.21.15.38:0/1589173654 conn(0x7f9c540b6b20 msgr2=0x7f9c540d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:52.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:52.622+0000 7f9c59520640 1 -- 172.21.15.38:0/1589173654 shutdown_connections 2024-09-21T03:38:52.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:52.622+0000 7f9c59520640 1 -- 172.21.15.38:0/1589173654 wait complete. 2024-09-21T03:38:52.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:52.623+0000 7f9c59520640 1 Processor -- start 2024-09-21T03:38:52.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:52.623+0000 7f9c59520640 1 -- start start 2024-09-21T03:38:52.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:52.625+0000 7f9c59520640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9c540b2e40 0x7f9c540b19d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:52.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:52.625+0000 7f9c59520640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9c54162280 0x7f9c540cede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:52.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:52.625+0000 7f9c59520640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9c5414d350 0x7f9c540ad370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:52.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:52.625+0000 7f9c59520640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f9c5415e630 con 0x7f9c540b2e40 2024-09-21T03:38:52.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:52.625+0000 7f9c59520640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f9c5415e4b0 con 0x7f9c54162280 2024-09-21T03:38:52.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:52.625+0000 7f9c59520640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f9c5415e7b0 con 0x7f9c5414d350 2024-09-21T03:38:52.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:52.625+0000 7f9c52ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9c540b2e40 0x7f9c540b19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:52.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:52.625+0000 7f9c52ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9c540b2e40 0x7f9c540b19d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43130/0 (socket says 172.21.15.38:43130) 2024-09-21T03:38:52.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:52.625+0000 7f9c52ffd640 1 -- 172.21.15.38:0/2884752610 learned_addr learned my addr 172.21.15.38:0/2884752610 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:38:52.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:52.625+0000 7f9c4bfff640 1 --2- 172.21.15.38:0/2884752610 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9c54162280 0x7f9c540cede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:52.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:52.625+0000 7f9c537fe640 1 --2- 172.21.15.38:0/2884752610 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9c5414d350 0x7f9c540ad370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:52.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:52.625+0000 7f9c52ffd640 1 -- 172.21.15.38:0/2884752610 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9c5414d350 msgr2=0x7f9c540ad370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:52.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:52.625+0000 7f9c52ffd640 1 --2- 172.21.15.38:0/2884752610 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9c5414d350 0x7f9c540ad370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:52.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:52.625+0000 7f9c52ffd640 1 -- 172.21.15.38:0/2884752610 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9c54162280 msgr2=0x7f9c540cede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:52.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:52.625+0000 7f9c52ffd640 1 --2- 172.21.15.38:0/2884752610 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9c54162280 0x7f9c540cede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:52.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:52.625+0000 7f9c52ffd640 1 -- 172.21.15.38:0/2884752610 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c540ac2d0 con 0x7f9c540b2e40 2024-09-21T03:38:52.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:52.625+0000 7f9c4bfff640 1 --2- 172.21.15.38:0/2884752610 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9c54162280 0x7f9c540cede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:38:52.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:52.625+0000 7f9c537fe640 1 --2- 172.21.15.38:0/2884752610 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9c5414d350 0x7f9c540ad370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:38:52.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:52.625+0000 7f9c52ffd640 1 --2- 172.21.15.38:0/2884752610 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9c540b2e40 0x7f9c540b19d0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f9c4400d780 tx=0x7f9c4400dc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:52.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:52.625+0000 7f9c4b7fe640 1 -- 172.21.15.38:0/2884752610 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c44017070 con 0x7f9c540b2e40 2024-09-21T03:38:52.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:52.626+0000 7f9c4b7fe640 1 -- 172.21.15.38:0/2884752610 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9c4400de90 con 0x7f9c540b2e40 2024-09-21T03:38:52.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:52.626+0000 7f9c59520640 1 -- 172.21.15.38:0/2884752610 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c540ac5b0 con 0x7f9c540b2e40 2024-09-21T03:38:52.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:52.626+0000 7f9c4b7fe640 1 -- 172.21.15.38:0/2884752610 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c440099d0 con 0x7f9c540b2e40 2024-09-21T03:38:52.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:52.626+0000 7f9c59520640 1 -- 172.21.15.38:0/2884752610 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c540acb70 con 0x7f9c540b2e40 2024-09-21T03:38:52.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:52.627+0000 7f9c59520640 1 -- 172.21.15.38:0/2884752610 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c54162f00 con 0x7f9c540b2e40 2024-09-21T03:38:52.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:52.627+0000 7f9c4b7fe640 1 -- 172.21.15.38:0/2884752610 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f9c44009b70 con 0x7f9c540b2e40 2024-09-21T03:38:52.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:52.628+0000 7f9c4b7fe640 1 --2- 172.21.15.38:0/2884752610 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9c34078b90 0x7f9c3407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:52.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:52.628+0000 7f9c4b7fe640 1 -- 172.21.15.38:0/2884752610 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 6476+0+0 (secure 0 0 0) 0x7f9c4409b480 con 0x7f9c540b2e40 2024-09-21T03:38:52.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:52.628+0000 7f9c4bfff640 1 --2- 172.21.15.38:0/2884752610 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9c34078b90 0x7f9c3407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:52.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:52.629+0000 7f9c4bfff640 1 --2- 172.21.15.38:0/2884752610 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9c34078b90 0x7f9c3407b0a0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f9c3c034000 tx=0x7f9c3c004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:52.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:52.632+0000 7f9c4b7fe640 1 -- 172.21.15.38:0/2884752610 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f9c44020020 con 0x7f9c540b2e40 2024-09-21T03:38:52.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:52.821+0000 7f9c59520640 1 -- 172.21.15.38:0/2884752610 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9c540b32f0 con 0x7f9c540b2e40 2024-09-21T03:38:52.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:52.822+0000 7f9c4b7fe640 1 -- 172.21.15.38:0/2884752610 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) ==== 74+0+17612 (secure 0 0 0) 0x7f9c440635c0 con 0x7f9c540b2e40 2024-09-21T03:38:52.824 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:38:52.824 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":71,"fsid":"9d50d66f-656b-4fe2-8e3b-fe53b79c4d43","created":"2024-09-21T03:33:10.677865+0000","modified":"2024-09-21T03:37:17.609008+0000","last_up_change":"2024-09-21T03:33:24.023421+0000","last_in_change":"2024-09-21T03:33:14.296193+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T03:33:22.992573+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-09-21T03:36:19.109965+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-09-21T03:36:20.392299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dbab183f-7e63-48b8-960a-ae55fd5bf565","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6801","nonce":2381639433}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6803","nonce":2381639433}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6807","nonce":2381639433}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6805","nonce":2381639433}]},"public_addr":"172.21.15.38:6801/2381639433","cluster_addr":"172.21.15.38:6803/2381639433","heartbeat_back_addr":"172.21.15.38:6807/2381639433","heartbeat_front_addr":"172.21.15.38:6805/2381639433","state":["exists","up"]},{"osd":1,"uuid":"fa1c0e8a-ebc1-4359-8146-bd5342abbc16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6817","nonce":1491818732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6819","nonce":1491818732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6823","nonce":1491818732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6821","nonce":1491818732}]},"public_addr":"172.21.15.38:6817/1491818732","cluster_addr":"172.21.15.38:6819/1491818732","heartbeat_back_addr":"172.21.15.38:6823/1491818732","heartbeat_front_addr":"172.21.15.38:6821/1491818732","state":["exists","up"]},{"osd":2,"uuid":"2ed3cd2d-385d-4fc6-9869-214552a84803","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6825","nonce":2577858902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6827","nonce":2577858902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6831","nonce":2577858902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6829","nonce":2577858902}]},"public_addr":"172.21.15.38:6825/2577858902","cluster_addr":"172.21.15.38:6827/2577858902","heartbeat_back_addr":"172.21.15.38:6831/2577858902","heartbeat_front_addr":"172.21.15.38:6829/2577858902","state":["exists","up"]},{"osd":3,"uuid":"62d1d95e-919d-4250-be71-1d34cfa70dbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6809","nonce":1286810294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6811","nonce":1286810294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6815","nonce":1286810294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6813","nonce":1286810294}]},"public_addr":"172.21.15.38:6809/1286810294","cluster_addr":"172.21.15.38:6811/1286810294","heartbeat_back_addr":"172.21.15.38:6815/1286810294","heartbeat_front_addr":"172.21.15.38:6813/1286810294","state":["exists","up"]},{"osd":4,"uuid":"7c9f9314-99d7-4d73-8392-37809a9f54b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6803","nonce":3401837571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6805","nonce":3401837571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6809","nonce":3401837571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6807","nonce":3401837571}]},"public_addr":"172.21.15.195:6803/3401837571","cluster_addr":"172.21.15.195:6805/3401837571","heartbeat_back_addr":"172.21.15.195:6809/3401837571","heartbeat_front_addr":"172.21.15.195:6807/3401837571","state":["exists","up"]},{"osd":5,"uuid":"bde4fe9f-8414-4bba-bf4a-43973d1976bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6811","nonce":2080960180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6813","nonce":2080960180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6817","nonce":2080960180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6815","nonce":2080960180}]},"public_addr":"172.21.15.195:6811/2080960180","cluster_addr":"172.21.15.195:6813/2080960180","heartbeat_back_addr":"172.21.15.195:6817/2080960180","heartbeat_front_addr":"172.21.15.195:6815/2080960180","state":["exists","up"]},{"osd":6,"uuid":"6ee6d7f4-1a5f-4a4c-ac16-48fe05f6a002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6827","nonce":4055825675}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6829","nonce":4055825675}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6833","nonce":4055825675}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6831","nonce":4055825675}]},"public_addr":"172.21.15.195:6827/4055825675","cluster_addr":"172.21.15.195:6829/4055825675","heartbeat_back_addr":"172.21.15.195:6833/4055825675","heartbeat_front_addr":"172.21.15.195:6831/4055825675","state":["exists","up"]},{"osd":7,"uuid":"efeb5f51-329a-4484-8290-92e7d3503d05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6819","nonce":1606106342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6821","nonce":1606106342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6825","nonce":1606106342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6823","nonce":1606106342}]},"public_addr":"172.21.15.195:6819/1606106342","cluster_addr":"172.21.15.195:6821/1606106342","heartbeat_back_addr":"172.21.15.195:6825/1606106342","heartbeat_front_addr":"172.21.15.195:6823/1606106342","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.901343+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:21.115003+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.901623+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.835238+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.132199+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:19.817560+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.829690+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:21.087300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"10.3","mappings":[{"from":4,"to":1}]},{"pgid":"10.1a","mappings":[{"from":2,"to":7}]},{"pgid":"10.1c","mappings":[{"from":2,"to":7}]},{"pgid":"10.22","mappings":[{"from":2,"to":5}]},{"pgid":"10.23","mappings":[{"from":2,"to":5}]},{"pgid":"10.2e","mappings":[{"from":2,"to":7}]},{"pgid":"10.33","mappings":[{"from":2,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.38:6834/1058376326":"2024-09-22T03:37:12.044790+0000","172.21.15.38:6832/1058376326":"2024-09-22T03:37:12.044790+0000","172.21.15.38:6834/1353335665":"2024-09-22T03:36:36.420530+0000","172.21.15.38:6832/1353335665":"2024-09-22T03:36:36.420530+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T03:38:52.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:52.824+0000 7f9c59520640 1 -- 172.21.15.38:0/2884752610 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9c34078b90 msgr2=0x7f9c3407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:52.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:52.824+0000 7f9c59520640 1 --2- 172.21.15.38:0/2884752610 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9c34078b90 0x7f9c3407b0a0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f9c3c034000 tx=0x7f9c3c004730 comp rx=0 tx=0).stop 2024-09-21T03:38:52.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:52.824+0000 7f9c59520640 1 -- 172.21.15.38:0/2884752610 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9c540b2e40 msgr2=0x7f9c540b19d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:52.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:52.824+0000 7f9c59520640 1 --2- 172.21.15.38:0/2884752610 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9c540b2e40 0x7f9c540b19d0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f9c4400d780 tx=0x7f9c4400dc40 comp rx=0 tx=0).stop 2024-09-21T03:38:52.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:52.824+0000 7f9c59520640 1 -- 172.21.15.38:0/2884752610 shutdown_connections 2024-09-21T03:38:52.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:52.824+0000 7f9c59520640 1 --2- 172.21.15.38:0/2884752610 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9c34078b90 0x7f9c3407b0a0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:52.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:52.824+0000 7f9c59520640 1 --2- 172.21.15.38:0/2884752610 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9c5414d350 0x7f9c540ad370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:52.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:52.824+0000 7f9c59520640 1 --2- 172.21.15.38:0/2884752610 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9c54162280 0x7f9c540cede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:52.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:52.824+0000 7f9c59520640 1 --2- 172.21.15.38:0/2884752610 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9c540b2e40 0x7f9c540b19d0 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:52.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:52.824+0000 7f9c59520640 1 -- 172.21.15.38:0/2884752610 >> 172.21.15.38:0/2884752610 conn(0x7f9c540b6b20 msgr2=0x7f9c540b7950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:52.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:52.824+0000 7f9c59520640 1 -- 172.21.15.38:0/2884752610 shutdown_connections 2024-09-21T03:38:52.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:52.824+0000 7f9c59520640 1 -- 172.21.15.38:0/2884752610 wait complete. 2024-09-21T03:38:52.838 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-21T03:38:52.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:52.954+0000 7ff33e670640 1 -- 172.21.15.38:0/2151072519 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff3380d1120 msgr2=0x7ff3380ceb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:52.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:52.954+0000 7ff33e670640 1 --2- 172.21.15.38:0/2151072519 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff3380d1120 0x7ff3380ceb90 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7ff3380d2360 tx=0x7ff33002a9a0 comp rx=0 tx=0).stop 2024-09-21T03:38:52.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:52.954+0000 7ff33e670640 1 -- 172.21.15.38:0/2151072519 shutdown_connections 2024-09-21T03:38:52.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:52.954+0000 7ff33e670640 1 --2- 172.21.15.38:0/2151072519 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff3380d1120 0x7ff3380ceb90 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:52.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:52.954+0000 7ff33e670640 1 --2- 172.21.15.38:0/2151072519 >> v2:172.21.15.195:3300/0 conn(0x7ff3380d0170 0x7ff3380d0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:52.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:52.954+0000 7ff33e670640 1 -- 172.21.15.38:0/2151072519 >> 172.21.15.38:0/2151072519 conn(0x7ff33809d830 msgr2=0x7ff3380b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:52.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:52.954+0000 7ff33e670640 1 -- 172.21.15.38:0/2151072519 shutdown_connections 2024-09-21T03:38:52.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:52.954+0000 7ff33e670640 1 -- 172.21.15.38:0/2151072519 wait complete. 2024-09-21T03:38:52.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:52.956+0000 7ff33e670640 1 Processor -- start 2024-09-21T03:38:52.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:52.956+0000 7ff33e670640 1 -- start start 2024-09-21T03:38:52.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:52.957+0000 7ff33e670640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff3380d0170 0x7ff33815be50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:52.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:52.957+0000 7ff33e670640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff3380d1120 0x7ff3380ca460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:52.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:52.957+0000 7ff33e670640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff33815ee80 0x7ff3380b21e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:52.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:52.957+0000 7ff33e670640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ff3380b6700 con 0x7ff33815ee80 2024-09-21T03:38:52.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:52.957+0000 7ff33e670640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ff3380b6580 con 0x7ff3380d1120 2024-09-21T03:38:52.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:52.957+0000 7ff33e670640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7ff3380b6880 con 0x7ff3380d0170 2024-09-21T03:38:52.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:52.957+0000 7ff33cc0d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff33815ee80 0x7ff3380b21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:52.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:52.957+0000 7ff33cc0d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff33815ee80 0x7ff3380b21e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43150/0 (socket says 172.21.15.38:43150) 2024-09-21T03:38:52.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:52.957+0000 7ff33cc0d640 1 -- 172.21.15.38:0/2588581576 learned_addr learned my addr 172.21.15.38:0/2588581576 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:38:52.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:52.957+0000 7ff3377fe640 1 --2- 172.21.15.38:0/2588581576 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff3380d1120 0x7ff3380ca460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:52.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:52.957+0000 7ff337fff640 1 --2- 172.21.15.38:0/2588581576 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff3380d0170 0x7ff33815be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:52.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:52.958+0000 7ff33cc0d640 1 -- 172.21.15.38:0/2588581576 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff3380d0170 msgr2=0x7ff33815be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:52.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:52.958+0000 7ff33cc0d640 1 --2- 172.21.15.38:0/2588581576 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff3380d0170 0x7ff33815be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:52.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:52.958+0000 7ff33cc0d640 1 -- 172.21.15.38:0/2588581576 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff3380d1120 msgr2=0x7ff3380ca460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:52.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:52.958+0000 7ff33cc0d640 1 --2- 172.21.15.38:0/2588581576 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff3380d1120 0x7ff3380ca460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:52.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:52.958+0000 7ff33cc0d640 1 -- 172.21.15.38:0/2588581576 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3381f5c70 con 0x7ff33815ee80 2024-09-21T03:38:52.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:52.958+0000 7ff33cc0d640 1 --2- 172.21.15.38:0/2588581576 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff33815ee80 0x7ff3380b21e0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7ff33002af90 tx=0x7ff330003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:52.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:52.958+0000 7ff334ff9640 1 -- 172.21.15.38:0/2588581576 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff330033050 con 0x7ff33815ee80 2024-09-21T03:38:52.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:52.958+0000 7ff334ff9640 1 -- 172.21.15.38:0/2588581576 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff3300044d0 con 0x7ff33815ee80 2024-09-21T03:38:52.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:52.958+0000 7ff33e670640 1 -- 172.21.15.38:0/2588581576 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3381f5f50 con 0x7ff33815ee80 2024-09-21T03:38:52.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:52.958+0000 7ff334ff9640 1 -- 172.21.15.38:0/2588581576 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff330038450 con 0x7ff33815ee80 2024-09-21T03:38:52.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:52.958+0000 7ff33e670640 1 -- 172.21.15.38:0/2588581576 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3381f6510 con 0x7ff33815ee80 2024-09-21T03:38:52.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:52.959+0000 7ff33e670640 1 -- 172.21.15.38:0/2588581576 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff33815c450 con 0x7ff33815ee80 2024-09-21T03:38:52.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:52.960+0000 7ff334ff9640 1 -- 172.21.15.38:0/2588581576 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ff330036070 con 0x7ff33815ee80 2024-09-21T03:38:52.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:52.960+0000 7ff334ff9640 1 --2- 172.21.15.38:0/2588581576 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff324078bc0 0x7ff32407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:52.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:52.960+0000 7ff334ff9640 1 -- 172.21.15.38:0/2588581576 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 6476+0+0 (secure 0 0 0) 0x7ff3300bc260 con 0x7ff33815ee80 2024-09-21T03:38:52.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:52.961+0000 7ff337fff640 1 --2- 172.21.15.38:0/2588581576 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff324078bc0 0x7ff32407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:52.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:52.961+0000 7ff337fff640 1 --2- 172.21.15.38:0/2588581576 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff324078bc0 0x7ff32407b0d0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7ff328000f60 tx=0x7ff328009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:52.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:52.965+0000 7ff334ff9640 1 -- 172.21.15.38:0/2588581576 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7ff330083d50 con 0x7ff33815ee80 2024-09-21T03:38:53.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:53.191+0000 7ff33e670640 1 -- 172.21.15.38:0/2588581576 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7ff3380b6da0 con 0x7ff33815ee80 2024-09-21T03:38:53.193 INFO:tasks.ceph.mon.a.smithi038.stderr:2024-09-21T03:38:53.192+0000 7f3f45713640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T03:38:53.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:53.193+0000 7ff334ff9640 1 -- 172.21.15.38:0/2588581576 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v60) ==== 161+0+0 (secure 0 0 0) 0x7ff330088780 con 0x7ff33815ee80 2024-09-21T03:38:53.195 INFO:teuthology.orchestra.run.smithi038.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-21T03:38:53.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:53.196+0000 7ff33e670640 1 -- 172.21.15.38:0/2588581576 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff324078bc0 msgr2=0x7ff32407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:53.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:53.196+0000 7ff33e670640 1 --2- 172.21.15.38:0/2588581576 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff324078bc0 0x7ff32407b0d0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7ff328000f60 tx=0x7ff328009040 comp rx=0 tx=0).stop 2024-09-21T03:38:53.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:53.196+0000 7ff33e670640 1 -- 172.21.15.38:0/2588581576 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff33815ee80 msgr2=0x7ff3380b21e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:53.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:53.196+0000 7ff33e670640 1 --2- 172.21.15.38:0/2588581576 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff33815ee80 0x7ff3380b21e0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7ff33002af90 tx=0x7ff330003f10 comp rx=0 tx=0).stop 2024-09-21T03:38:53.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:53.196+0000 7ff33e670640 1 -- 172.21.15.38:0/2588581576 shutdown_connections 2024-09-21T03:38:53.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:53.196+0000 7ff33e670640 1 --2- 172.21.15.38:0/2588581576 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff33815ee80 0x7ff3380b21e0 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:53.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:53.196+0000 7ff33e670640 1 --2- 172.21.15.38:0/2588581576 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff3380d1120 0x7ff3380ca460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:53.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:53.196+0000 7ff33e670640 1 --2- 172.21.15.38:0/2588581576 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff324078bc0 0x7ff32407b0d0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:53.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:53.196+0000 7ff33e670640 1 --2- 172.21.15.38:0/2588581576 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff3380d0170 0x7ff33815be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:53.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:53.196+0000 7ff33e670640 1 -- 172.21.15.38:0/2588581576 >> 172.21.15.38:0/2588581576 conn(0x7ff33809d830 msgr2=0x7ff3380b5740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:53.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:53.196+0000 7ff33e670640 1 -- 172.21.15.38:0/2588581576 shutdown_connections 2024-09-21T03:38:53.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:53.196+0000 7ff33e670640 1 -- 172.21.15.38:0/2588581576 wait complete. 2024-09-21T03:38:53.203 INFO:tasks.ceph.mds.b.smithi195.stderr: -4276> 2024-09-21T03:37:19.036+0000 7f225610a640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T03:38:53.211 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-21T03:38:53.260 INFO:tasks.ceph.mds.b.smithi195.stdout:starting mds.b at 2024-09-21T03:38:53.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:53.329+0000 7fb5834d3640 1 -- 172.21.15.38:0/612734916 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb568003a50 msgr2=0x7fb568003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:53.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:53.329+0000 7fb5834d3640 1 --2- 172.21.15.38:0/612734916 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb568003a50 0x7fb568003ec0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fb578009b20 tx=0x7fb568005830 comp rx=0 tx=0).stop 2024-09-21T03:38:53.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:53.329+0000 7fb5834d3640 1 -- 172.21.15.38:0/612734916 shutdown_connections 2024-09-21T03:38:53.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:53.329+0000 7fb5834d3640 1 --2- 172.21.15.38:0/612734916 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb568003a50 0x7fb568003ec0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:53.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:53.329+0000 7fb5834d3640 1 -- 172.21.15.38:0/612734916 >> 172.21.15.38:0/612734916 conn(0x7fb57c0cdca0 msgr2=0x7fb57c0ce0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:53.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:53.329+0000 7fb5834d3640 1 -- 172.21.15.38:0/612734916 shutdown_connections 2024-09-21T03:38:53.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:53.329+0000 7fb5834d3640 1 -- 172.21.15.38:0/612734916 wait complete. 2024-09-21T03:38:53.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:53.331+0000 7fb5834d3640 1 Processor -- start 2024-09-21T03:38:53.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:53.331+0000 7fb5834d3640 1 -- start start 2024-09-21T03:38:53.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:53.332+0000 7fb5834d3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb568003a50 0x7fb57c1f2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:53.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:53.332+0000 7fb5834d3640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb57c15af30 0x7fb57c156320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:53.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:53.332+0000 7fb5834d3640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb57c156860 0x7fb57c159720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:53.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:53.332+0000 7fb5834d3640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fb57c15bc00 con 0x7fb568003a50 2024-09-21T03:38:53.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:53.332+0000 7fb5834d3640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fb57c15ba80 con 0x7fb57c156860 2024-09-21T03:38:53.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:53.332+0000 7fb5834d3640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fb57c15bd80 con 0x7fb57c15af30 2024-09-21T03:38:53.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:53.332+0000 7fb58126f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb568003a50 0x7fb57c1f2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:53.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:53.332+0000 7fb58126f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb568003a50 0x7fb57c1f2170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43170/0 (socket says 172.21.15.38:43170) 2024-09-21T03:38:53.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:53.332+0000 7fb58126f640 1 -- 172.21.15.38:0/98466665 learned_addr learned my addr 172.21.15.38:0/98466665 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:38:53.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:53.332+0000 7fb581a70640 1 --2- 172.21.15.38:0/98466665 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb57c156860 0x7fb57c159720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:53.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:53.333+0000 7fb580a6e640 1 --2- 172.21.15.38:0/98466665 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb57c15af30 0x7fb57c156320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:53.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:53.333+0000 7fb58126f640 1 -- 172.21.15.38:0/98466665 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb57c15af30 msgr2=0x7fb57c156320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:53.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:53.333+0000 7fb58126f640 1 --2- 172.21.15.38:0/98466665 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb57c15af30 0x7fb57c156320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:53.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:53.333+0000 7fb58126f640 1 -- 172.21.15.38:0/98466665 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb57c156860 msgr2=0x7fb57c159720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:53.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:53.333+0000 7fb58126f640 1 --2- 172.21.15.38:0/98466665 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb57c156860 0x7fb57c159720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:53.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:53.333+0000 7fb58126f640 1 -- 172.21.15.38:0/98466665 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb57c1f26b0 con 0x7fb568003a50 2024-09-21T03:38:53.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:53.333+0000 7fb580a6e640 1 --2- 172.21.15.38:0/98466665 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb57c15af30 0x7fb57c156320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:38:53.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:53.333+0000 7fb58126f640 1 --2- 172.21.15.38:0/98466665 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb568003a50 0x7fb57c1f2170 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fb578009a70 tx=0x7fb578033650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:53.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:53.333+0000 7fb571ffb640 1 -- 172.21.15.38:0/98466665 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb578040020 con 0x7fb568003a50 2024-09-21T03:38:53.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:53.333+0000 7fb5834d3640 1 -- 172.21.15.38:0/98466665 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb57c1f2bc0 con 0x7fb568003a50 2024-09-21T03:38:53.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:53.333+0000 7fb571ffb640 1 -- 172.21.15.38:0/98466665 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb578033b30 con 0x7fb568003a50 2024-09-21T03:38:53.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:53.333+0000 7fb571ffb640 1 -- 172.21.15.38:0/98466665 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb578038040 con 0x7fb568003a50 2024-09-21T03:38:53.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:53.333+0000 7fb5834d3640 1 -- 172.21.15.38:0/98466665 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb57c158f20 con 0x7fb568003a50 2024-09-21T03:38:53.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:53.334+0000 7fb5834d3640 1 -- 172.21.15.38:0/98466665 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb57c15f630 con 0x7fb568003a50 2024-09-21T03:38:53.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:53.335+0000 7fb571ffb640 1 -- 172.21.15.38:0/98466665 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb578043070 con 0x7fb568003a50 2024-09-21T03:38:53.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:53.335+0000 7fb571ffb640 1 --2- 172.21.15.38:0/98466665 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb560078b90 0x7fb56007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:53.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:53.336+0000 7fb571ffb640 1 -- 172.21.15.38:0/98466665 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 6562+0+0 (secure 0 0 0) 0x7fb5780bae70 con 0x7fb568003a50 2024-09-21T03:38:53.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:53.336+0000 7fb580a6e640 1 --2- 172.21.15.38:0/98466665 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb560078b90 0x7fb56007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:53.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:53.336+0000 7fb580a6e640 1 --2- 172.21.15.38:0/98466665 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb560078b90 0x7fb56007b0a0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fb57c09ede0 tx=0x7fb56c006b70 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:53.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:53.340+0000 7fb571ffb640 1 -- 172.21.15.38:0/98466665 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fb578082f50 con 0x7fb568003a50 2024-09-21T03:38:53.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:53.569+0000 7fb5834d3640 1 -- 172.21.15.38:0/98466665 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fb57c069450 con 0x7fb568003a50 2024-09-21T03:38:54.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:54.195+0000 7fb571ffb640 1 -- 172.21.15.38:0/98466665 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v61) ==== 104+0+0 (secure 0 0 0) 0x7fb578082850 con 0x7fb568003a50 2024-09-21T03:38:54.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:54.198+0000 7fb5834d3640 1 -- 172.21.15.38:0/98466665 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb560078b90 msgr2=0x7fb56007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:54.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:54.198+0000 7fb5834d3640 1 --2- 172.21.15.38:0/98466665 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb560078b90 0x7fb56007b0a0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fb57c09ede0 tx=0x7fb56c006b70 comp rx=0 tx=0).stop 2024-09-21T03:38:54.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:54.198+0000 7fb5834d3640 1 -- 172.21.15.38:0/98466665 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb568003a50 msgr2=0x7fb57c1f2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:54.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:54.198+0000 7fb5834d3640 1 --2- 172.21.15.38:0/98466665 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb568003a50 0x7fb57c1f2170 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fb578009a70 tx=0x7fb578033650 comp rx=0 tx=0).stop 2024-09-21T03:38:54.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:54.198+0000 7fb5834d3640 1 -- 172.21.15.38:0/98466665 shutdown_connections 2024-09-21T03:38:54.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:54.198+0000 7fb5834d3640 1 --2- 172.21.15.38:0/98466665 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb57c156860 0x7fb57c159720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:54.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:54.198+0000 7fb5834d3640 1 --2- 172.21.15.38:0/98466665 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb560078b90 0x7fb56007b0a0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:54.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:54.198+0000 7fb5834d3640 1 --2- 172.21.15.38:0/98466665 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb57c15af30 0x7fb57c156320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:54.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:54.198+0000 7fb5834d3640 1 --2- 172.21.15.38:0/98466665 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb568003a50 0x7fb57c1f2170 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:54.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:54.198+0000 7fb5834d3640 1 -- 172.21.15.38:0/98466665 >> 172.21.15.38:0/98466665 conn(0x7fb57c0cdca0 msgr2=0x7fb57c1551f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:54.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:54.198+0000 7fb5834d3640 1 -- 172.21.15.38:0/98466665 shutdown_connections 2024-09-21T03:38:54.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:54.198+0000 7fb5834d3640 1 -- 172.21.15.38:0/98466665 wait complete. 2024-09-21T03:38:54.214 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-21T03:38:54.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:54.339+0000 7f8592139640 1 -- 172.21.15.38:0/3601043325 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f858c155910 msgr2=0x7f858c162a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:54.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:54.339+0000 7f8592139640 1 --2- 172.21.15.38:0/3601043325 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f858c155910 0x7f858c162a30 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f858c06e1e0 tx=0x7f857c02a560 comp rx=0 tx=0).stop 2024-09-21T03:38:54.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:54.339+0000 7f8592139640 1 -- 172.21.15.38:0/3601043325 shutdown_connections 2024-09-21T03:38:54.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:54.339+0000 7f8592139640 1 --2- 172.21.15.38:0/3601043325 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f858c0d1490 0x7f858c0b5fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:54.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:54.339+0000 7f8592139640 1 --2- 172.21.15.38:0/3601043325 >> v2:172.21.15.195:3300/0 conn(0x7f858c162f70 0x7f858c0d0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:54.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:54.339+0000 7f8592139640 1 --2- 172.21.15.38:0/3601043325 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f858c155910 0x7f858c162a30 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:54.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:54.339+0000 7f8592139640 1 -- 172.21.15.38:0/3601043325 >> 172.21.15.38:0/3601043325 conn(0x7f858c06cde0 msgr2=0x7f858c06d1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:54.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:54.339+0000 7f8592139640 1 -- 172.21.15.38:0/3601043325 shutdown_connections 2024-09-21T03:38:54.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:54.339+0000 7f8592139640 1 -- 172.21.15.38:0/3601043325 wait complete. 2024-09-21T03:38:54.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:54.341+0000 7f8592139640 1 Processor -- start 2024-09-21T03:38:54.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:54.341+0000 7f8592139640 1 -- start start 2024-09-21T03:38:54.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:54.343+0000 7f8592139640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f858c0d1490 0x7f858c153b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:54.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:54.343+0000 7f8592139640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f858c155910 0x7f858c1e80f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:54.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:54.343+0000 7f8592139640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f858c162f70 0x7f858c1e8630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:54.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:54.343+0000 7f8592139640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f858c0b66a0 con 0x7f858c162f70 2024-09-21T03:38:54.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:54.343+0000 7f8592139640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f858c0b6520 con 0x7f858c0d1490 2024-09-21T03:38:54.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:54.343+0000 7f8592139640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f858c0b6820 con 0x7f858c155910 2024-09-21T03:38:54.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:54.343+0000 7f858bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f858c162f70 0x7f858c1e8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:54.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:54.343+0000 7f858bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f858c162f70 0x7f858c1e8630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43184/0 (socket says 172.21.15.38:43184) 2024-09-21T03:38:54.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:54.343+0000 7f858bfff640 1 -- 172.21.15.38:0/1102174966 learned_addr learned my addr 172.21.15.38:0/1102174966 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:38:54.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:54.343+0000 7f858b7fe640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f858c0d1490 0x7f858c153b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:54.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:54.343+0000 7f8582ffd640 1 --2- 172.21.15.38:0/1102174966 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f858c155910 0x7f858c1e80f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:54.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:54.343+0000 7f858bfff640 1 -- 172.21.15.38:0/1102174966 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f858c155910 msgr2=0x7f858c1e80f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:54.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:54.343+0000 7f858bfff640 1 --2- 172.21.15.38:0/1102174966 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f858c155910 0x7f858c1e80f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:54.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:54.343+0000 7f858bfff640 1 -- 172.21.15.38:0/1102174966 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f858c0d1490 msgr2=0x7f858c153b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:54.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:54.343+0000 7f858bfff640 1 --2- 172.21.15.38:0/1102174966 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f858c0d1490 0x7f858c153b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:54.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:54.343+0000 7f858bfff640 1 -- 172.21.15.38:0/1102174966 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f858c154ee0 con 0x7f858c162f70 2024-09-21T03:38:54.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:54.343+0000 7f858bfff640 1 --2- 172.21.15.38:0/1102174966 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f858c162f70 0x7f858c1e8630 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f858400e960 tx=0x7f858400ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:54.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:54.343+0000 7f8588ff9640 1 -- 172.21.15.38:0/1102174966 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8584004d20 con 0x7f858c162f70 2024-09-21T03:38:54.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:54.344+0000 7f8588ff9640 1 -- 172.21.15.38:0/1102174966 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f858400ca60 con 0x7f858c162f70 2024-09-21T03:38:54.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:54.344+0000 7f8588ff9640 1 -- 172.21.15.38:0/1102174966 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8584005020 con 0x7f858c162f70 2024-09-21T03:38:54.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:54.344+0000 7f8592139640 1 -- 172.21.15.38:0/1102174966 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f858c1540c0 con 0x7f858c162f70 2024-09-21T03:38:54.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:54.344+0000 7f8592139640 1 -- 172.21.15.38:0/1102174966 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f858c1f9880 con 0x7f858c162f70 2024-09-21T03:38:54.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:54.345+0000 7f8592139640 1 -- 172.21.15.38:0/1102174966 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f858c0d4070 con 0x7f858c162f70 2024-09-21T03:38:54.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:54.345+0000 7f8588ff9640 1 -- 172.21.15.38:0/1102174966 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f85840051c0 con 0x7f858c162f70 2024-09-21T03:38:54.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:54.346+0000 7f8588ff9640 1 --2- 172.21.15.38:0/1102174966 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f856c078b90 0x7f856c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:54.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:54.346+0000 7f8588ff9640 1 -- 172.21.15.38:0/1102174966 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 6562+0+0 (secure 0 0 0) 0x7f858409b8a0 con 0x7f858c162f70 2024-09-21T03:38:54.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:54.346+0000 7f858b7fe640 1 --2- 172.21.15.38:0/1102174966 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f856c078b90 0x7f856c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:54.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:54.347+0000 7f858b7fe640 1 --2- 172.21.15.38:0/1102174966 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f856c078b90 0x7f856c07b0a0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f857c033000 tx=0x7f857c004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:54.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:54.350+0000 7f8588ff9640 1 -- 172.21.15.38:0/1102174966 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f858c0d4070 con 0x7f858c162f70 2024-09-21T03:38:54.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:54.538+0000 7f8592139640 1 -- 172.21.15.38:0/1102174966 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f858c162370 con 0x7f858c162f70 2024-09-21T03:38:55.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:55.338+0000 7f8588ff9640 1 -- 172.21.15.38:0/1102174966 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v73) ==== 181+0+0 (secure 0 0 0) 0x7f858c162370 con 0x7f858c162f70 2024-09-21T03:38:55.340 INFO:teuthology.orchestra.run.smithi038.stderr:pool 'cephfs_metadata' removed 2024-09-21T03:38:55.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:55.342+0000 7f8592139640 1 -- 172.21.15.38:0/1102174966 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f856c078b90 msgr2=0x7f856c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:55.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:55.342+0000 7f8592139640 1 --2- 172.21.15.38:0/1102174966 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f856c078b90 0x7f856c07b0a0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f857c033000 tx=0x7f857c004730 comp rx=0 tx=0).stop 2024-09-21T03:38:55.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:55.342+0000 7f8592139640 1 -- 172.21.15.38:0/1102174966 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f858c162f70 msgr2=0x7f858c1e8630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:55.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:55.342+0000 7f8592139640 1 --2- 172.21.15.38:0/1102174966 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f858c162f70 0x7f858c1e8630 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f858400e960 tx=0x7f858400ee20 comp rx=0 tx=0).stop 2024-09-21T03:38:55.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:55.342+0000 7f8592139640 1 -- 172.21.15.38:0/1102174966 shutdown_connections 2024-09-21T03:38:55.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:55.342+0000 7f8592139640 1 --2- 172.21.15.38:0/1102174966 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f858c162f70 0x7f858c1e8630 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:55.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:55.342+0000 7f8592139640 1 --2- 172.21.15.38:0/1102174966 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f856c078b90 0x7f856c07b0a0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:55.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:55.342+0000 7f8592139640 1 --2- 172.21.15.38:0/1102174966 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f858c155910 0x7f858c1e80f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:55.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:55.342+0000 7f8592139640 1 --2- 172.21.15.38:0/1102174966 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f858c0d1490 0x7f858c153b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:55.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:55.342+0000 7f8592139640 1 -- 172.21.15.38:0/1102174966 >> 172.21.15.38:0/1102174966 conn(0x7f858c06cde0 msgr2=0x7f858c0d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:55.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:55.343+0000 7f8592139640 1 -- 172.21.15.38:0/1102174966 shutdown_connections 2024-09-21T03:38:55.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:55.343+0000 7f8592139640 1 -- 172.21.15.38:0/1102174966 wait complete. 2024-09-21T03:38:55.359 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-21T03:38:55.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:55.490+0000 7f26f1415640 1 -- 172.21.15.38:0/3156589641 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f26ec162c10 msgr2=0x7f26ec06e300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:55.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:55.490+0000 7f26f1415640 1 --2- 172.21.15.38:0/3156589641 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f26ec162c10 0x7f26ec06e300 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f26ec08ca50 tx=0x7f26dc02aa30 comp rx=0 tx=0).stop 2024-09-21T03:38:55.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:55.490+0000 7f26f1415640 1 -- 172.21.15.38:0/3156589641 shutdown_connections 2024-09-21T03:38:55.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:55.490+0000 7f26f1415640 1 --2- 172.21.15.38:0/3156589641 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f26ec162c10 0x7f26ec06e300 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:55.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:55.490+0000 7f26f1415640 1 --2- 172.21.15.38:0/3156589641 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f26ec0b64a0 0x7f26ec1626d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:55.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:55.490+0000 7f26f1415640 1 --2- 172.21.15.38:0/3156589641 >> v2:172.21.15.195:3300/0 conn(0x7f26ec15e7d0 0x7f26ec0b5f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:55.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:55.490+0000 7f26f1415640 1 -- 172.21.15.38:0/3156589641 >> 172.21.15.38:0/3156589641 conn(0x7f26ec0d44e0 msgr2=0x7f26ec0d48f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:55.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:55.490+0000 7f26f1415640 1 -- 172.21.15.38:0/3156589641 shutdown_connections 2024-09-21T03:38:55.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:55.490+0000 7f26f1415640 1 -- 172.21.15.38:0/3156589641 wait complete. 2024-09-21T03:38:55.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:55.492+0000 7f26f1415640 1 Processor -- start 2024-09-21T03:38:55.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:55.492+0000 7f26f1415640 1 -- start start 2024-09-21T03:38:55.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:55.493+0000 7f26f1415640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f26ec0b64a0 0x7f26ec1ea160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:55.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:55.493+0000 7f26f1415640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f26ec15e7d0 0x7f26ec149410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:55.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:55.493+0000 7f26f1415640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f26ec162c10 0x7f26ec149950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:55.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:55.493+0000 7f26f1415640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f26ec06e9c0 con 0x7f26ec0b64a0 2024-09-21T03:38:55.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:55.493+0000 7f26f1415640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f26ec06e840 con 0x7f26ec162c10 2024-09-21T03:38:55.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:55.493+0000 7f26f1415640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f26ec06eb40 con 0x7f26ec15e7d0 2024-09-21T03:38:55.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:55.494+0000 7f26eaffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f26ec0b64a0 0x7f26ec1ea160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:55.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:55.494+0000 7f26eaffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f26ec0b64a0 0x7f26ec1ea160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43196/0 (socket says 172.21.15.38:43196) 2024-09-21T03:38:55.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:55.494+0000 7f26eaffd640 1 -- 172.21.15.38:0/411623949 learned_addr learned my addr 172.21.15.38:0/411623949 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:38:55.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:55.494+0000 7f26eb7fe640 1 --2- 172.21.15.38:0/411623949 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f26ec162c10 0x7f26ec149950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:55.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:55.494+0000 7f26ea7fc640 1 --2- 172.21.15.38:0/411623949 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f26ec15e7d0 0x7f26ec149410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:55.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:55.494+0000 7f26eaffd640 1 -- 172.21.15.38:0/411623949 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f26ec15e7d0 msgr2=0x7f26ec149410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:55.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:55.494+0000 7f26eaffd640 1 --2- 172.21.15.38:0/411623949 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f26ec15e7d0 0x7f26ec149410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:55.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:55.494+0000 7f26eaffd640 1 -- 172.21.15.38:0/411623949 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f26ec162c10 msgr2=0x7f26ec149950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:55.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:55.494+0000 7f26eaffd640 1 --2- 172.21.15.38:0/411623949 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f26ec162c10 0x7f26ec149950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:55.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:55.494+0000 7f26eaffd640 1 -- 172.21.15.38:0/411623949 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26ec1eaa20 con 0x7f26ec0b64a0 2024-09-21T03:38:55.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:55.494+0000 7f26eaffd640 1 --2- 172.21.15.38:0/411623949 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f26ec0b64a0 0x7f26ec1ea160 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f26e400d750 tx=0x7f26e400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:55.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:55.494+0000 7f26e37fe640 1 -- 172.21.15.38:0/411623949 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26e4017070 con 0x7f26ec0b64a0 2024-09-21T03:38:55.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:55.494+0000 7f26e37fe640 1 -- 172.21.15.38:0/411623949 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f26e4004310 con 0x7f26ec0b64a0 2024-09-21T03:38:55.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:55.494+0000 7f26f1415640 1 -- 172.21.15.38:0/411623949 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26ec1ead00 con 0x7f26ec0b64a0 2024-09-21T03:38:55.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:55.494+0000 7f26e37fe640 1 -- 172.21.15.38:0/411623949 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26e400de60 con 0x7f26ec0b64a0 2024-09-21T03:38:55.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:55.495+0000 7f26f1415640 1 -- 172.21.15.38:0/411623949 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26ec1eb2c0 con 0x7f26ec0b64a0 2024-09-21T03:38:55.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:55.495+0000 7f26f1415640 1 -- 172.21.15.38:0/411623949 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26ec06f060 con 0x7f26ec0b64a0 2024-09-21T03:38:55.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:55.496+0000 7f26e37fe640 1 -- 172.21.15.38:0/411623949 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f26e4009c30 con 0x7f26ec0b64a0 2024-09-21T03:38:55.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:55.497+0000 7f26e37fe640 1 --2- 172.21.15.38:0/411623949 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f26cc078b90 0x7f26cc07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:55.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:55.497+0000 7f26e37fe640 1 -- 172.21.15.38:0/411623949 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 6124+0+0 (secure 0 0 0) 0x7f26e409b1e0 con 0x7f26ec0b64a0 2024-09-21T03:38:55.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:55.497+0000 7f26ea7fc640 1 --2- 172.21.15.38:0/411623949 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f26cc078b90 0x7f26cc07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:55.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:55.498+0000 7f26ea7fc640 1 --2- 172.21.15.38:0/411623949 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f26cc078b90 0x7f26cc07b0a0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f26ec15f690 tx=0x7f26d400b260 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:55.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:55.501+0000 7f26e37fe640 1 -- 172.21.15.38:0/411623949 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f26e4020020 con 0x7f26ec0b64a0 2024-09-21T03:38:55.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:55.690+0000 7f26f1415640 1 -- 172.21.15.38:0/411623949 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f26ec0d3cc0 con 0x7f26ec0b64a0 2024-09-21T03:38:56.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:56.341+0000 7f26e37fe640 1 -- 172.21.15.38:0/411623949 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v74) ==== 169+0+0 (secure 0 0 0) 0x7f26e4062e20 con 0x7f26ec0b64a0 2024-09-21T03:38:56.343 INFO:teuthology.orchestra.run.smithi038.stderr:pool 'cephfs_data' removed 2024-09-21T03:38:56.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:56.344+0000 7f26f1415640 1 -- 172.21.15.38:0/411623949 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f26cc078b90 msgr2=0x7f26cc07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:56.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:56.344+0000 7f26f1415640 1 --2- 172.21.15.38:0/411623949 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f26cc078b90 0x7f26cc07b0a0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f26ec15f690 tx=0x7f26d400b260 comp rx=0 tx=0).stop 2024-09-21T03:38:56.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:56.344+0000 7f26f1415640 1 -- 172.21.15.38:0/411623949 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f26ec0b64a0 msgr2=0x7f26ec1ea160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:56.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:56.344+0000 7f26f1415640 1 --2- 172.21.15.38:0/411623949 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f26ec0b64a0 0x7f26ec1ea160 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f26e400d750 tx=0x7f26e400dc10 comp rx=0 tx=0).stop 2024-09-21T03:38:56.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:56.344+0000 7f26f1415640 1 -- 172.21.15.38:0/411623949 shutdown_connections 2024-09-21T03:38:56.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:56.344+0000 7f26f1415640 1 --2- 172.21.15.38:0/411623949 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f26ec162c10 0x7f26ec149950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:56.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:56.344+0000 7f26f1415640 1 --2- 172.21.15.38:0/411623949 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f26cc078b90 0x7f26cc07b0a0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:56.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:56.344+0000 7f26f1415640 1 --2- 172.21.15.38:0/411623949 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f26ec15e7d0 0x7f26ec149410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:56.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:56.344+0000 7f26f1415640 1 --2- 172.21.15.38:0/411623949 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f26ec0b64a0 0x7f26ec1ea160 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:56.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:56.344+0000 7f26f1415640 1 -- 172.21.15.38:0/411623949 >> 172.21.15.38:0/411623949 conn(0x7f26ec0d44e0 msgr2=0x7f26ec0d48f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:56.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:56.344+0000 7f26f1415640 1 -- 172.21.15.38:0/411623949 shutdown_connections 2024-09-21T03:38:56.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:56.344+0000 7f26f1415640 1 -- 172.21.15.38:0/411623949 wait complete. 2024-09-21T03:38:56.362 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 31 2024-09-21T03:38:56.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:56.483+0000 7fa2e2a5c640 1 -- 172.21.15.38:0/1949080076 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa2dc0d0170 msgr2=0x7fa2dc0d0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:56.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:56.483+0000 7fa2e2a5c640 1 --2- 172.21.15.38:0/1949080076 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa2dc0d0170 0x7fa2dc0d0be0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fa2dc0d2080 tx=0x7fa2c402a4d0 comp rx=0 tx=0).stop 2024-09-21T03:38:56.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:56.483+0000 7fa2e2a5c640 1 -- 172.21.15.38:0/1949080076 shutdown_connections 2024-09-21T03:38:56.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:56.483+0000 7fa2e2a5c640 1 --2- 172.21.15.38:0/1949080076 >> v2:172.21.15.195:3300/0 conn(0x7fa2dc0d1120 0x7fa2dc0ceb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:56.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:56.483+0000 7fa2e2a5c640 1 --2- 172.21.15.38:0/1949080076 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa2dc0d0170 0x7fa2dc0d0be0 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:56.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:56.483+0000 7fa2e2a5c640 1 -- 172.21.15.38:0/1949080076 >> 172.21.15.38:0/1949080076 conn(0x7fa2dc09d830 msgr2=0x7fa2dc0b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:56.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:56.483+0000 7fa2e2a5c640 1 -- 172.21.15.38:0/1949080076 shutdown_connections 2024-09-21T03:38:56.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:56.483+0000 7fa2e2a5c640 1 -- 172.21.15.38:0/1949080076 wait complete. 2024-09-21T03:38:56.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:56.485+0000 7fa2e2a5c640 1 Processor -- start 2024-09-21T03:38:56.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:56.485+0000 7fa2e2a5c640 1 -- start start 2024-09-21T03:38:56.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:56.487+0000 7fa2e2a5c640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa2dc0d0170 0x7fa2dc06d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:56.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:56.487+0000 7fa2e2a5c640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa2dc0d1120 0x7fa2dc1578e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:56.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:56.487+0000 7fa2e2a5c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa2dc15ee80 0x7fa2dc0b2780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:56.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:56.487+0000 7fa2e2a5c640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa2dc0b6700 con 0x7fa2dc15ee80 2024-09-21T03:38:56.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:56.487+0000 7fa2e2a5c640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fa2dc0b6580 con 0x7fa2dc0d1120 2024-09-21T03:38:56.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:56.487+0000 7fa2e2a5c640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fa2dc0b6880 con 0x7fa2dc0d0170 2024-09-21T03:38:56.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:56.487+0000 7fa2e0ff9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa2dc15ee80 0x7fa2dc0b2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:56.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:56.487+0000 7fa2e0ff9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa2dc15ee80 0x7fa2dc0b2780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43218/0 (socket says 172.21.15.38:43218) 2024-09-21T03:38:56.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:56.487+0000 7fa2e0ff9640 1 -- 172.21.15.38:0/641430189 learned_addr learned my addr 172.21.15.38:0/641430189 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:38:56.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:56.487+0000 7fa2db7fe640 1 --2- 172.21.15.38:0/641430189 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa2dc0d1120 0x7fa2dc1578e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:56.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:56.487+0000 7fa2dbfff640 1 --2- 172.21.15.38:0/641430189 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa2dc0d0170 0x7fa2dc06d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:56.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:56.487+0000 7fa2e0ff9640 1 -- 172.21.15.38:0/641430189 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa2dc0d0170 msgr2=0x7fa2dc06d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:56.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:56.487+0000 7fa2e0ff9640 1 --2- 172.21.15.38:0/641430189 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa2dc0d0170 0x7fa2dc06d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:56.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:56.487+0000 7fa2e0ff9640 1 -- 172.21.15.38:0/641430189 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa2dc0d1120 msgr2=0x7fa2dc1578e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:56.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:56.487+0000 7fa2e0ff9640 1 --2- 172.21.15.38:0/641430189 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa2dc0d1120 0x7fa2dc1578e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:56.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:56.487+0000 7fa2e0ff9640 1 -- 172.21.15.38:0/641430189 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2dc0ccc20 con 0x7fa2dc15ee80 2024-09-21T03:38:56.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:56.487+0000 7fa2e0ff9640 1 --2- 172.21.15.38:0/641430189 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa2dc15ee80 0x7fa2dc0b2780 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fa2d400e990 tx=0x7fa2d400ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:56.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:56.488+0000 7fa2d8ff9640 1 -- 172.21.15.38:0/641430189 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2d4004d20 con 0x7fa2dc15ee80 2024-09-21T03:38:56.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:56.488+0000 7fa2d8ff9640 1 -- 172.21.15.38:0/641430189 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa2d400cb00 con 0x7fa2dc15ee80 2024-09-21T03:38:56.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:56.488+0000 7fa2e2a5c640 1 -- 172.21.15.38:0/641430189 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2dc0cc660 con 0x7fa2dc15ee80 2024-09-21T03:38:56.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:56.488+0000 7fa2d8ff9640 1 -- 172.21.15.38:0/641430189 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2d4005020 con 0x7fa2dc15ee80 2024-09-21T03:38:56.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:56.488+0000 7fa2e2a5c640 1 -- 172.21.15.38:0/641430189 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2dc06e240 con 0x7fa2dc15ee80 2024-09-21T03:38:56.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:56.489+0000 7fa2e2a5c640 1 -- 172.21.15.38:0/641430189 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2dc06e680 con 0x7fa2dc15ee80 2024-09-21T03:38:56.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:56.489+0000 7fa2d8ff9640 1 -- 172.21.15.38:0/641430189 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa2d40051c0 con 0x7fa2dc15ee80 2024-09-21T03:38:56.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:56.490+0000 7fa2d8ff9640 1 --2- 172.21.15.38:0/641430189 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa2bc078b90 0x7fa2bc07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:56.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:56.490+0000 7fa2d8ff9640 1 -- 172.21.15.38:0/641430189 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5507+0+0 (secure 0 0 0) 0x7fa2d409b420 con 0x7fa2dc15ee80 2024-09-21T03:38:56.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:56.490+0000 7fa2dbfff640 1 --2- 172.21.15.38:0/641430189 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa2bc078b90 0x7fa2bc07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:56.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:56.491+0000 7fa2dbfff640 1 --2- 172.21.15.38:0/641430189 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa2bc078b90 0x7fa2bc07b0a0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc001720 tx=0x7fa2cc009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:56.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:56.495+0000 7fa2d8ff9640 1 -- 172.21.15.38:0/641430189 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fa2d4016030 con 0x7fa2dc15ee80 2024-09-21T03:38:56.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:56.686+0000 7fa2e2a5c640 1 -- 172.21.15.38:0/641430189 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "config reset", "num": 31} v 0) -- 0x7fa2dc0b6da0 con 0x7fa2dc15ee80 2024-09-21T03:38:56.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:56.690+0000 7fa2d8ff9640 1 -- 172.21.15.38:0/641430189 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 31}]=0 v42) ==== 71+0+0 (secure 0 0 0) 0x7fa2d40633f0 con 0x7fa2dc15ee80 2024-09-21T03:38:56.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:56.692+0000 7fa2e2a5c640 1 -- 172.21.15.38:0/641430189 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa2bc078b90 msgr2=0x7fa2bc07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:56.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:56.692+0000 7fa2e2a5c640 1 --2- 172.21.15.38:0/641430189 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa2bc078b90 0x7fa2bc07b0a0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc001720 tx=0x7fa2cc009040 comp rx=0 tx=0).stop 2024-09-21T03:38:56.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:56.692+0000 7fa2e2a5c640 1 -- 172.21.15.38:0/641430189 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa2dc15ee80 msgr2=0x7fa2dc0b2780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:56.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:56.692+0000 7fa2e2a5c640 1 --2- 172.21.15.38:0/641430189 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa2dc15ee80 0x7fa2dc0b2780 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fa2d400e990 tx=0x7fa2d400ee50 comp rx=0 tx=0).stop 2024-09-21T03:38:56.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:56.692+0000 7fa2e2a5c640 1 -- 172.21.15.38:0/641430189 shutdown_connections 2024-09-21T03:38:56.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:56.692+0000 7fa2e2a5c640 1 --2- 172.21.15.38:0/641430189 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa2dc15ee80 0x7fa2dc0b2780 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:56.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:56.692+0000 7fa2e2a5c640 1 --2- 172.21.15.38:0/641430189 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa2dc0d1120 0x7fa2dc1578e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:56.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:56.692+0000 7fa2e2a5c640 1 --2- 172.21.15.38:0/641430189 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa2bc078b90 0x7fa2bc07b0a0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:56.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:56.692+0000 7fa2e2a5c640 1 --2- 172.21.15.38:0/641430189 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa2dc0d0170 0x7fa2dc06d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:56.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:56.692+0000 7fa2e2a5c640 1 -- 172.21.15.38:0/641430189 >> 172.21.15.38:0/641430189 conn(0x7fa2dc09d830 msgr2=0x7fa2dc0b57e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:56.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:56.692+0000 7fa2e2a5c640 1 -- 172.21.15.38:0/641430189 shutdown_connections 2024-09-21T03:38:56.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:56.692+0000 7fa2e2a5c640 1 -- 172.21.15.38:0/641430189 wait complete. 2024-09-21T03:38:56.705 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_damage.TestDamage.test_dentry_first_precommit' 2024-09-21T03:38:56.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:56.820+0000 7fd6ea69b640 1 -- 172.21.15.38:0/2064805096 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd6cc003a50 msgr2=0x7fd6cc003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:56.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:56.820+0000 7fd6ea69b640 1 --2- 172.21.15.38:0/2064805096 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd6cc003a50 0x7fd6cc003ec0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fd6d400bb20 tx=0x7fd6cc005830 comp rx=0 tx=0).stop 2024-09-21T03:38:56.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:56.820+0000 7fd6ea69b640 1 -- 172.21.15.38:0/2064805096 shutdown_connections 2024-09-21T03:38:56.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:56.820+0000 7fd6ea69b640 1 --2- 172.21.15.38:0/2064805096 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd6cc003a50 0x7fd6cc003ec0 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:56.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:56.820+0000 7fd6ea69b640 1 -- 172.21.15.38:0/2064805096 >> 172.21.15.38:0/2064805096 conn(0x7fd6e40d4440 msgr2=0x7fd6e40d4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:56.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:56.820+0000 7fd6ea69b640 1 -- 172.21.15.38:0/2064805096 shutdown_connections 2024-09-21T03:38:56.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:56.820+0000 7fd6ea69b640 1 -- 172.21.15.38:0/2064805096 wait complete. 2024-09-21T03:38:56.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:56.822+0000 7fd6ea69b640 1 Processor -- start 2024-09-21T03:38:56.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:56.822+0000 7fd6ea69b640 1 -- start start 2024-09-21T03:38:56.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:56.823+0000 7fd6ea69b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd6cc003a50 0x7fd6e4155760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:56.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:56.823+0000 7fd6ea69b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd6e41585d0 0x7fd6e415d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:56.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:56.823+0000 7fd6ea69b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd6e415da40 0x7fd6e415de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:56.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:56.823+0000 7fd6ea69b640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fd6e4162e90 con 0x7fd6e415da40 2024-09-21T03:38:56.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:56.823+0000 7fd6ea69b640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd6e4162d10 con 0x7fd6cc003a50 2024-09-21T03:38:56.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:56.823+0000 7fd6ea69b640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fd6e40d2a20 con 0x7fd6e41585d0 2024-09-21T03:38:56.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:56.823+0000 7fd6e8c38640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd6e415da40 0x7fd6e415de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:56.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:56.823+0000 7fd6e8c38640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd6e415da40 0x7fd6e415de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43242/0 (socket says 172.21.15.38:43242) 2024-09-21T03:38:56.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:56.823+0000 7fd6e8c38640 1 -- 172.21.15.38:0/642944967 learned_addr learned my addr 172.21.15.38:0/642944967 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:38:56.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:56.824+0000 7fd6e3fff640 1 --2- 172.21.15.38:0/642944967 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd6cc003a50 0x7fd6e4155760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:56.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:56.824+0000 7fd6e37fe640 1 --2- 172.21.15.38:0/642944967 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd6e41585d0 0x7fd6e415d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:56.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:56.824+0000 7fd6e8c38640 1 -- 172.21.15.38:0/642944967 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd6e41585d0 msgr2=0x7fd6e415d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:56.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:56.824+0000 7fd6e8c38640 1 --2- 172.21.15.38:0/642944967 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd6e41585d0 0x7fd6e415d500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:56.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:56.824+0000 7fd6e8c38640 1 -- 172.21.15.38:0/642944967 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd6cc003a50 msgr2=0x7fd6e4155760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:56.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:56.824+0000 7fd6e8c38640 1 --2- 172.21.15.38:0/642944967 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd6cc003a50 0x7fd6e4155760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:56.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:56.824+0000 7fd6e8c38640 1 -- 172.21.15.38:0/642944967 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6e4156190 con 0x7fd6e415da40 2024-09-21T03:38:56.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:56.824+0000 7fd6e8c38640 1 --2- 172.21.15.38:0/642944967 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd6e415da40 0x7fd6e415de50 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fd6dc001330 tx=0x7fd6dc0019b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:56.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:56.824+0000 7fd6e0ff9640 1 -- 172.21.15.38:0/642944967 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6dc011870 con 0x7fd6e415da40 2024-09-21T03:38:56.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:56.824+0000 7fd6e0ff9640 1 -- 172.21.15.38:0/642944967 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd6dc017070 con 0x7fd6e415da40 2024-09-21T03:38:56.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:56.824+0000 7fd6e0ff9640 1 -- 172.21.15.38:0/642944967 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6dc011dc0 con 0x7fd6e415da40 2024-09-21T03:38:56.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:56.824+0000 7fd6ea69b640 1 -- 172.21.15.38:0/642944967 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6e4156470 con 0x7fd6e415da40 2024-09-21T03:38:56.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:56.824+0000 7fd6ea69b640 1 -- 172.21.15.38:0/642944967 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6e4156a30 con 0x7fd6e415da40 2024-09-21T03:38:56.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:56.825+0000 7fd6ea69b640 1 -- 172.21.15.38:0/642944967 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6e40d2f80 con 0x7fd6e415da40 2024-09-21T03:38:56.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:56.826+0000 7fd6e0ff9640 1 -- 172.21.15.38:0/642944967 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fd6dc01c460 con 0x7fd6e415da40 2024-09-21T03:38:56.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:56.827+0000 7fd6e0ff9640 1 --2- 172.21.15.38:0/642944967 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd6d0078bc0 0x7fd6d007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:56.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:56.827+0000 7fd6e0ff9640 1 -- 172.21.15.38:0/642944967 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5507+0+0 (secure 0 0 0) 0x7fd6dc09b3d0 con 0x7fd6e415da40 2024-09-21T03:38:56.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:56.827+0000 7fd6e3fff640 1 --2- 172.21.15.38:0/642944967 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd6d0078bc0 0x7fd6d007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:56.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:56.828+0000 7fd6e3fff640 1 --2- 172.21.15.38:0/642944967 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd6d0078bc0 0x7fd6d007b0d0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fd6d4036000 tx=0x7fd6d403c040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:56.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:56.831+0000 7fd6e0ff9640 1 -- 172.21.15.38:0/642944967 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fd6dc0631e0 con 0x7fd6e415da40 2024-09-21T03:38:57.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:57.095+0000 7fd6ea69b640 1 -- 172.21.15.38:0/642944967 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_damage.TestDamage.test_dentry_first_precommit"]} v 0) -- 0x7fd6e415e390 con 0x7fd6e415da40 2024-09-21T03:38:57.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:57.350+0000 7fd6e0ff9640 1 -- 172.21.15.38:0/642944967 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_damage.TestDamage.test_dentry_first_precommit"]}]=0 v260) ==== 142+0+0 (secure 0 0 0) 0x7fd6dc067c10 con 0x7fd6e415da40 2024-09-21T03:38:57.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:57.353+0000 7fd6ea69b640 1 -- 172.21.15.38:0/642944967 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd6d0078bc0 msgr2=0x7fd6d007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:57.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:57.353+0000 7fd6ea69b640 1 --2- 172.21.15.38:0/642944967 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd6d0078bc0 0x7fd6d007b0d0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fd6d4036000 tx=0x7fd6d403c040 comp rx=0 tx=0).stop 2024-09-21T03:38:57.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:57.353+0000 7fd6ea69b640 1 -- 172.21.15.38:0/642944967 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd6e415da40 msgr2=0x7fd6e415de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:57.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:57.353+0000 7fd6ea69b640 1 --2- 172.21.15.38:0/642944967 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd6e415da40 0x7fd6e415de50 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fd6dc001330 tx=0x7fd6dc0019b0 comp rx=0 tx=0).stop 2024-09-21T03:38:57.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:57.353+0000 7fd6ea69b640 1 -- 172.21.15.38:0/642944967 shutdown_connections 2024-09-21T03:38:57.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:57.353+0000 7fd6ea69b640 1 --2- 172.21.15.38:0/642944967 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd6e415da40 0x7fd6e415de50 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:57.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:57.353+0000 7fd6ea69b640 1 --2- 172.21.15.38:0/642944967 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd6d0078bc0 0x7fd6d007b0d0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:57.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:57.353+0000 7fd6ea69b640 1 --2- 172.21.15.38:0/642944967 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd6e41585d0 0x7fd6e415d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:57.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:57.353+0000 7fd6ea69b640 1 --2- 172.21.15.38:0/642944967 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd6cc003a50 0x7fd6e4155760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:57.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:57.353+0000 7fd6ea69b640 1 -- 172.21.15.38:0/642944967 >> 172.21.15.38:0/642944967 conn(0x7fd6e40d4440 msgr2=0x7fd6e40d4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:57.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:57.353+0000 7fd6ea69b640 1 -- 172.21.15.38:0/642944967 shutdown_connections 2024-09-21T03:38:57.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:57.353+0000 7fd6ea69b640 1 -- 172.21.15.38:0/642944967 wait complete. 2024-09-21T03:38:57.367 INFO:tasks.cephfs_test_runner:That the MDS won't write a dentry with new damage to CDentry::first ... ok 2024-09-21T03:38:57.367 INFO:tasks.cephfs_test_runner:Starting test: test_dentry_first_preflush (tasks.cephfs.test_damage.TestDamage) That the MDS won't write a dentry with new damage to CDentry::first 2024-09-21T03:38:57.367 INFO:tasks.cephfs_test_runner:test_dentry_first_preflush (tasks.cephfs.test_damage.TestDamage) 2024-09-21T03:38:57.367 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T03:38:57.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:57.484+0000 7f9f2fc0e640 1 -- 172.21.15.38:0/3072732102 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9f140050c0 msgr2=0x7f9f1400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:57.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:57.484+0000 7f9f2fc0e640 1 --2- 172.21.15.38:0/3072732102 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9f140050c0 0x7f9f1400b9c0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f9f20005fd0 tx=0x7f9f14005830 comp rx=0 tx=0).stop 2024-09-21T03:38:57.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:57.484+0000 7f9f2e1ab640 1 -- 172.21.15.38:0/3072732102 reap_dead start 2024-09-21T03:38:57.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:57.484+0000 7f9f2fc0e640 1 -- 172.21.15.38:0/3072732102 shutdown_connections 2024-09-21T03:38:57.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:57.484+0000 7f9f2fc0e640 1 -- 172.21.15.38:0/3072732102 >> 172.21.15.38:0/3072732102 conn(0x7f9f280cef40 msgr2=0x7f9f280cf350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:57.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:57.484+0000 7f9f2fc0e640 1 -- 172.21.15.38:0/3072732102 shutdown_connections 2024-09-21T03:38:57.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:57.484+0000 7f9f2fc0e640 1 -- 172.21.15.38:0/3072732102 wait complete. 2024-09-21T03:38:57.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:57.485+0000 7f9f2fc0e640 1 Processor -- start 2024-09-21T03:38:57.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:57.486+0000 7f9f2fc0e640 1 -- start start 2024-09-21T03:38:57.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:57.487+0000 7f9f2fc0e640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9f140050c0 0x7f9f281485c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:57.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:57.487+0000 7f9f2fc0e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9f28148d30 0x7f9f281f83a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:57.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:57.487+0000 7f9f2fc0e640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9f281f88e0 0x7f9f281f8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:57.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:57.487+0000 7f9f2fc0e640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f9f28165300 con 0x7f9f28148d30 2024-09-21T03:38:57.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:57.487+0000 7f9f2fc0e640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f9f280cd250 con 0x7f9f140050c0 2024-09-21T03:38:57.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:57.487+0000 7f9f2fc0e640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f9f28165480 con 0x7f9f281f88e0 2024-09-21T03:38:57.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:57.487+0000 7f9f2d1a9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9f28148d30 0x7f9f281f83a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:57.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:57.487+0000 7f9f2d1a9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9f28148d30 0x7f9f281f83a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43250/0 (socket says 172.21.15.38:43250) 2024-09-21T03:38:57.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:57.487+0000 7f9f2d1a9640 1 -- 172.21.15.38:0/943864293 learned_addr learned my addr 172.21.15.38:0/943864293 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:38:57.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:57.487+0000 7f9f2d9aa640 1 --2- 172.21.15.38:0/943864293 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9f140050c0 0x7f9f281485c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:57.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:57.487+0000 7f9f2e1ab640 1 --2- 172.21.15.38:0/943864293 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9f281f88e0 0x7f9f281f8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:57.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:57.487+0000 7f9f2d1a9640 1 -- 172.21.15.38:0/943864293 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9f281f88e0 msgr2=0x7f9f281f8cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:57.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:57.487+0000 7f9f2d1a9640 1 --2- 172.21.15.38:0/943864293 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9f281f88e0 0x7f9f281f8cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:57.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:57.487+0000 7f9f2d1a9640 1 -- 172.21.15.38:0/943864293 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9f140050c0 msgr2=0x7f9f281485c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:57.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:57.487+0000 7f9f2d1a9640 1 --2- 172.21.15.38:0/943864293 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9f140050c0 0x7f9f281485c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:57.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:57.487+0000 7f9f2d1a9640 1 -- 172.21.15.38:0/943864293 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f28148b00 con 0x7f9f28148d30 2024-09-21T03:38:57.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:57.488+0000 7f9f2d1a9640 1 --2- 172.21.15.38:0/943864293 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9f28148d30 0x7f9f281f83a0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f9f18003f80 tx=0x7f9f180017d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:57.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:57.488+0000 7f9f1e7fc640 1 -- 172.21.15.38:0/943864293 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f18015070 con 0x7f9f28148d30 2024-09-21T03:38:57.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:57.488+0000 7f9f1e7fc640 1 -- 172.21.15.38:0/943864293 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9f18011900 con 0x7f9f28148d30 2024-09-21T03:38:57.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:57.488+0000 7f9f2fc0e640 1 -- 172.21.15.38:0/943864293 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f2814cc60 con 0x7f9f28148d30 2024-09-21T03:38:57.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:57.488+0000 7f9f1e7fc640 1 -- 172.21.15.38:0/943864293 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f18011d50 con 0x7f9f28148d30 2024-09-21T03:38:57.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:57.488+0000 7f9f2fc0e640 1 -- 172.21.15.38:0/943864293 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f2806e100 con 0x7f9f28148d30 2024-09-21T03:38:57.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:57.489+0000 7f9f2fc0e640 1 -- 172.21.15.38:0/943864293 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f2806def0 con 0x7f9f28148d30 2024-09-21T03:38:57.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:57.490+0000 7f9f1e7fc640 1 -- 172.21.15.38:0/943864293 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f9f18018070 con 0x7f9f28148d30 2024-09-21T03:38:57.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:57.490+0000 7f9f1e7fc640 1 --2- 172.21.15.38:0/943864293 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9f0c078bc0 0x7f9f0c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:57.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:57.491+0000 7f9f1e7fc640 1 -- 172.21.15.38:0/943864293 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5507+0+0 (secure 0 0 0) 0x7f9f1809b9b0 con 0x7f9f28148d30 2024-09-21T03:38:57.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:57.491+0000 7f9f2d9aa640 1 --2- 172.21.15.38:0/943864293 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9f0c078bc0 0x7f9f0c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:57.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:57.491+0000 7f9f2d9aa640 1 --2- 172.21.15.38:0/943864293 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9f0c078bc0 0x7f9f0c07b0d0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f9f24000f60 tx=0x7f9f24000ef0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:57.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:57.495+0000 7f9f1e7fc640 1 -- 172.21.15.38:0/943864293 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f9f18063eb0 con 0x7f9f28148d30 2024-09-21T03:38:57.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:57.682+0000 7f9f2fc0e640 1 -- 172.21.15.38:0/943864293 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9f280cc930 con 0x7f9f28148d30 2024-09-21T03:38:57.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:57.683+0000 7f9f1e7fc640 1 -- 172.21.15.38:0/943864293 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) ==== 74+0+13592 (secure 0 0 0) 0x7f9f180637b0 con 0x7f9f28148d30 2024-09-21T03:38:57.685 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:38:57.685 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":74,"fsid":"9d50d66f-656b-4fe2-8e3b-fe53b79c4d43","created":"2024-09-21T03:33:10.677865+0000","modified":"2024-09-21T03:38:56.339268+0000","last_up_change":"2024-09-21T03:33:24.023421+0000","last_in_change":"2024-09-21T03:33:14.296193+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T03:33:22.992573+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dbab183f-7e63-48b8-960a-ae55fd5bf565","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6801","nonce":2381639433}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6803","nonce":2381639433}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6807","nonce":2381639433}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6805","nonce":2381639433}]},"public_addr":"172.21.15.38:6801/2381639433","cluster_addr":"172.21.15.38:6803/2381639433","heartbeat_back_addr":"172.21.15.38:6807/2381639433","heartbeat_front_addr":"172.21.15.38:6805/2381639433","state":["exists","up"]},{"osd":1,"uuid":"fa1c0e8a-ebc1-4359-8146-bd5342abbc16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6817","nonce":1491818732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6819","nonce":1491818732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6823","nonce":1491818732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6821","nonce":1491818732}]},"public_addr":"172.21.15.38:6817/1491818732","cluster_addr":"172.21.15.38:6819/1491818732","heartbeat_back_addr":"172.21.15.38:6823/1491818732","heartbeat_front_addr":"172.21.15.38:6821/1491818732","state":["exists","up"]},{"osd":2,"uuid":"2ed3cd2d-385d-4fc6-9869-214552a84803","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6825","nonce":2577858902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6827","nonce":2577858902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6831","nonce":2577858902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6829","nonce":2577858902}]},"public_addr":"172.21.15.38:6825/2577858902","cluster_addr":"172.21.15.38:6827/2577858902","heartbeat_back_addr":"172.21.15.38:6831/2577858902","heartbeat_front_addr":"172.21.15.38:6829/2577858902","state":["exists","up"]},{"osd":3,"uuid":"62d1d95e-919d-4250-be71-1d34cfa70dbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6809","nonce":1286810294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6811","nonce":1286810294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6815","nonce":1286810294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6813","nonce":1286810294}]},"public_addr":"172.21.15.38:6809/1286810294","cluster_addr":"172.21.15.38:6811/1286810294","heartbeat_back_addr":"172.21.15.38:6815/1286810294","heartbeat_front_addr":"172.21.15.38:6813/1286810294","state":["exists","up"]},{"osd":4,"uuid":"7c9f9314-99d7-4d73-8392-37809a9f54b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6803","nonce":3401837571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6805","nonce":3401837571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6809","nonce":3401837571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6807","nonce":3401837571}]},"public_addr":"172.21.15.195:6803/3401837571","cluster_addr":"172.21.15.195:6805/3401837571","heartbeat_back_addr":"172.21.15.195:6809/3401837571","heartbeat_front_addr":"172.21.15.195:6807/3401837571","state":["exists","up"]},{"osd":5,"uuid":"bde4fe9f-8414-4bba-bf4a-43973d1976bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6811","nonce":2080960180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6813","nonce":2080960180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6817","nonce":2080960180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6815","nonce":2080960180}]},"public_addr":"172.21.15.195:6811/2080960180","cluster_addr":"172.21.15.195:6813/2080960180","heartbeat_back_addr":"172.21.15.195:6817/2080960180","heartbeat_front_addr":"172.21.15.195:6815/2080960180","state":["exists","up"]},{"osd":6,"uuid":"6ee6d7f4-1a5f-4a4c-ac16-48fe05f6a002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6827","nonce":4055825675}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6829","nonce":4055825675}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6833","nonce":4055825675}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6831","nonce":4055825675}]},"public_addr":"172.21.15.195:6827/4055825675","cluster_addr":"172.21.15.195:6829/4055825675","heartbeat_back_addr":"172.21.15.195:6833/4055825675","heartbeat_front_addr":"172.21.15.195:6831/4055825675","state":["exists","up"]},{"osd":7,"uuid":"efeb5f51-329a-4484-8290-92e7d3503d05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6819","nonce":1606106342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6821","nonce":1606106342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6825","nonce":1606106342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6823","nonce":1606106342}]},"public_addr":"172.21.15.195:6819/1606106342","cluster_addr":"172.21.15.195:6821/1606106342","heartbeat_back_addr":"172.21.15.195:6825/1606106342","heartbeat_front_addr":"172.21.15.195:6823/1606106342","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.901343+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:21.115003+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.901623+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.835238+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.132199+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:19.817560+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.829690+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:21.087300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.195:6836/2536177898":"2024-09-22T03:38:53.192776+0000","172.21.15.195:6834/2536177898":"2024-09-22T03:38:53.192776+0000","172.21.15.38:6834/1058376326":"2024-09-22T03:37:12.044790+0000","172.21.15.38:6832/1058376326":"2024-09-22T03:37:12.044790+0000","172.21.15.38:6834/1353335665":"2024-09-22T03:36:36.420530+0000","172.21.15.38:6832/1353335665":"2024-09-22T03:36:36.420530+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T03:38:57.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:57.685+0000 7f9f2fc0e640 1 -- 172.21.15.38:0/943864293 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9f0c078bc0 msgr2=0x7f9f0c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:57.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:57.685+0000 7f9f2fc0e640 1 --2- 172.21.15.38:0/943864293 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9f0c078bc0 0x7f9f0c07b0d0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f9f24000f60 tx=0x7f9f24000ef0 comp rx=0 tx=0).stop 2024-09-21T03:38:57.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:57.685+0000 7f9f2fc0e640 1 -- 172.21.15.38:0/943864293 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9f28148d30 msgr2=0x7f9f281f83a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:57.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:57.685+0000 7f9f2fc0e640 1 --2- 172.21.15.38:0/943864293 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9f28148d30 0x7f9f281f83a0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f9f18003f80 tx=0x7f9f180017d0 comp rx=0 tx=0).stop 2024-09-21T03:38:57.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:57.685+0000 7f9f2fc0e640 1 -- 172.21.15.38:0/943864293 shutdown_connections 2024-09-21T03:38:57.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:57.685+0000 7f9f2fc0e640 1 --2- 172.21.15.38:0/943864293 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9f0c078bc0 0x7f9f0c07b0d0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:57.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:57.685+0000 7f9f2fc0e640 1 --2- 172.21.15.38:0/943864293 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9f281f88e0 0x7f9f281f8cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:57.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:57.685+0000 7f9f2fc0e640 1 --2- 172.21.15.38:0/943864293 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9f28148d30 0x7f9f281f83a0 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:57.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:57.685+0000 7f9f2fc0e640 1 --2- 172.21.15.38:0/943864293 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9f140050c0 0x7f9f281485c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:57.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:57.685+0000 7f9f2fc0e640 1 -- 172.21.15.38:0/943864293 >> 172.21.15.38:0/943864293 conn(0x7f9f280cef40 msgr2=0x7f9f28070100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:57.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:57.685+0000 7f9f2fc0e640 1 -- 172.21.15.38:0/943864293 shutdown_connections 2024-09-21T03:38:57.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:57.685+0000 7f9f2fc0e640 1 -- 172.21.15.38:0/943864293 wait complete. 2024-09-21T03:38:57.699 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T03:33:22.992573+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-21T03:38:57.699 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-21T03:38:57.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:57.817+0000 7f8112b0b640 1 -- 172.21.15.38:0/1117022764 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f80fc003fd0 msgr2=0x7f80fc004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:57.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:57.817+0000 7f8112b0b640 1 --2- 172.21.15.38:0/1117022764 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f80fc003fd0 0x7f80fc004800 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f810c0700d0 tx=0x7f80fc005830 comp rx=0 tx=0).stop 2024-09-21T03:38:57.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:57.817+0000 7f81110a8640 1 -- 172.21.15.38:0/1117022764 reap_dead start 2024-09-21T03:38:57.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:57.817+0000 7f8112b0b640 1 -- 172.21.15.38:0/1117022764 shutdown_connections 2024-09-21T03:38:57.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:57.817+0000 7f8112b0b640 1 -- 172.21.15.38:0/1117022764 >> 172.21.15.38:0/1117022764 conn(0x7f810c09d830 msgr2=0x7f810c0b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:57.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:57.817+0000 7f8112b0b640 1 -- 172.21.15.38:0/1117022764 shutdown_connections 2024-09-21T03:38:57.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:57.817+0000 7f8112b0b640 1 -- 172.21.15.38:0/1117022764 wait complete. 2024-09-21T03:38:57.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:57.819+0000 7f8112b0b640 1 Processor -- start 2024-09-21T03:38:57.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:57.819+0000 7f8112b0b640 1 -- start start 2024-09-21T03:38:57.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:57.820+0000 7f8112b0b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f80fc003fd0 0x7f810c0a1e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:57.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:57.821+0000 7f8112b0b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f810c0cd2b0 0x7f810c09e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:57.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:57.821+0000 7f8112b0b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f810c0b23c0 0x7f810c0b27d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:57.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:57.821+0000 7f8112b0b640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f810c0d0be0 con 0x7f810c0cd2b0 2024-09-21T03:38:57.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:57.821+0000 7f8112b0b640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f810c0d0d60 con 0x7f810c0b23c0 2024-09-21T03:38:57.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:57.821+0000 7f8112b0b640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f810c0d0ee0 con 0x7f80fc003fd0 2024-09-21T03:38:57.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:57.821+0000 7f810bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f810c0cd2b0 0x7f810c09e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:57.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:57.821+0000 7f810bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f810c0cd2b0 0x7f810c09e2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43262/0 (socket says 172.21.15.38:43262) 2024-09-21T03:38:57.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:57.821+0000 7f810bfff640 1 -- 172.21.15.38:0/379699561 learned_addr learned my addr 172.21.15.38:0/379699561 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:38:57.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:57.821+0000 7f81110a8640 1 --2- 172.21.15.38:0/379699561 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f810c0b23c0 0x7f810c0b27d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:57.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:57.821+0000 7f81108a7640 1 --2- 172.21.15.38:0/379699561 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f80fc003fd0 0x7f810c0a1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:57.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:57.821+0000 7f810bfff640 1 -- 172.21.15.38:0/379699561 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f80fc003fd0 msgr2=0x7f810c0a1e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:57.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:57.821+0000 7f810bfff640 1 --2- 172.21.15.38:0/379699561 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f80fc003fd0 0x7f810c0a1e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:57.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:57.821+0000 7f810bfff640 1 -- 172.21.15.38:0/379699561 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f810c0b23c0 msgr2=0x7f810c0b27d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:57.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:57.821+0000 7f810bfff640 1 --2- 172.21.15.38:0/379699561 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f810c0b23c0 0x7f810c0b27d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:57.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:57.821+0000 7f810bfff640 1 -- 172.21.15.38:0/379699561 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f810c0cda20 con 0x7f810c0cd2b0 2024-09-21T03:38:57.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:57.821+0000 7f810bfff640 1 --2- 172.21.15.38:0/379699561 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f810c0cd2b0 0x7f810c09e2a0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f80f8009c00 tx=0x7f80f80336f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:57.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:57.821+0000 7f81097fa640 1 -- 172.21.15.38:0/379699561 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80f8038040 con 0x7f810c0cd2b0 2024-09-21T03:38:57.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:57.821+0000 7f81097fa640 1 -- 172.21.15.38:0/379699561 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f80f8041070 con 0x7f810c0cd2b0 2024-09-21T03:38:57.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:57.822+0000 7f8112b0b640 1 -- 172.21.15.38:0/379699561 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f810c0cdd00 con 0x7f810c0cd2b0 2024-09-21T03:38:57.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:57.822+0000 7f81097fa640 1 -- 172.21.15.38:0/379699561 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80f803c430 con 0x7f810c0cd2b0 2024-09-21T03:38:57.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:57.822+0000 7f8112b0b640 1 -- 172.21.15.38:0/379699561 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f810c0cb990 con 0x7f810c0cd2b0 2024-09-21T03:38:57.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:57.823+0000 7f8112b0b640 1 -- 172.21.15.38:0/379699561 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f810c0d0170 con 0x7f810c0cd2b0 2024-09-21T03:38:57.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:57.823+0000 7f81097fa640 1 -- 172.21.15.38:0/379699561 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f80f802b7e0 con 0x7f810c0cd2b0 2024-09-21T03:38:57.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:57.824+0000 7f81097fa640 1 --2- 172.21.15.38:0/379699561 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f80f0078bc0 0x7f80f007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:57.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:57.824+0000 7f81097fa640 1 -- 172.21.15.38:0/379699561 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5507+0+0 (secure 0 0 0) 0x7f80f80bb520 con 0x7f810c0cd2b0 2024-09-21T03:38:57.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:57.824+0000 7f81108a7640 1 --2- 172.21.15.38:0/379699561 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f80f0078bc0 0x7f80f007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:57.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:57.825+0000 7f81108a7640 1 --2- 172.21.15.38:0/379699561 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f80f0078bc0 0x7f80f007b0d0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f8100009e50 tx=0x7f8100005f90 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:57.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:57.829+0000 7f81097fa640 1 -- 172.21.15.38:0/379699561 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f80f80833d0 con 0x7f810c0cd2b0 2024-09-21T03:38:58.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:58.015+0000 7f8112b0b640 1 -- 172.21.15.38:0/379699561 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f810c0cd770 con 0x7f810c0cd2b0 2024-09-21T03:38:58.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:58.016+0000 7f81097fa640 1 -- 172.21.15.38:0/379699561 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v74) ==== 93+0+10 (secure 0 0 0) 0x7f80f8087e00 con 0x7f810c0cd2b0 2024-09-21T03:38:58.018 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 1 2024-09-21T03:38:58.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:58.018+0000 7f8112b0b640 1 -- 172.21.15.38:0/379699561 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f80f0078bc0 msgr2=0x7f80f007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:58.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:58.018+0000 7f8112b0b640 1 --2- 172.21.15.38:0/379699561 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f80f0078bc0 0x7f80f007b0d0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f8100009e50 tx=0x7f8100005f90 comp rx=0 tx=0).stop 2024-09-21T03:38:58.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:58.018+0000 7f8112b0b640 1 -- 172.21.15.38:0/379699561 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f810c0cd2b0 msgr2=0x7f810c09e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:58.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:58.018+0000 7f8112b0b640 1 --2- 172.21.15.38:0/379699561 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f810c0cd2b0 0x7f810c09e2a0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f80f8009c00 tx=0x7f80f80336f0 comp rx=0 tx=0).stop 2024-09-21T03:38:58.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:58.018+0000 7f8112b0b640 1 -- 172.21.15.38:0/379699561 shutdown_connections 2024-09-21T03:38:58.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:58.018+0000 7f8112b0b640 1 --2- 172.21.15.38:0/379699561 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f810c0b23c0 0x7f810c0b27d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:58.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:58.018+0000 7f8112b0b640 1 --2- 172.21.15.38:0/379699561 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f810c0cd2b0 0x7f810c09e2a0 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:58.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:58.018+0000 7f8112b0b640 1 --2- 172.21.15.38:0/379699561 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f80f0078bc0 0x7f80f007b0d0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:58.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:58.018+0000 7f8112b0b640 1 --2- 172.21.15.38:0/379699561 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f80fc003fd0 0x7f810c0a1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:58.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:58.018+0000 7f8112b0b640 1 -- 172.21.15.38:0/379699561 >> 172.21.15.38:0/379699561 conn(0x7f810c09d830 msgr2=0x7f810c0b2d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:58.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:58.018+0000 7f8112b0b640 1 -- 172.21.15.38:0/379699561 shutdown_connections 2024-09-21T03:38:58.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:58.018+0000 7f8112b0b640 1 -- 172.21.15.38:0/379699561 wait complete. 2024-09-21T03:38:58.031 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_damage.TestDamage.test_dentry_first_preflush' 2024-09-21T03:38:58.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:58.150+0000 7f5babf69640 1 -- 172.21.15.38:0/1105124189 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5ba40d01f0 msgr2=0x7f5ba415daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:58.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:58.150+0000 7f5babf69640 1 --2- 172.21.15.38:0/1105124189 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5ba40d01f0 0x7f5ba415daf0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f5ba40cec40 tx=0x7f5ba002a560 comp rx=0 tx=0).stop 2024-09-21T03:38:58.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:58.150+0000 7f5babf69640 1 -- 172.21.15.38:0/1105124189 shutdown_connections 2024-09-21T03:38:58.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:58.150+0000 7f5babf69640 1 --2- 172.21.15.38:0/1105124189 >> v2:172.21.15.195:3300/0 conn(0x7f5ba415a6f0 0x7f5ba415e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:58.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:58.150+0000 7f5babf69640 1 --2- 172.21.15.38:0/1105124189 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5ba40d01f0 0x7f5ba415daf0 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:58.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:58.150+0000 7f5babf69640 1 -- 172.21.15.38:0/1105124189 >> 172.21.15.38:0/1105124189 conn(0x7f5ba4072990 msgr2=0x7f5ba4072da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:58.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:58.150+0000 7f5babf69640 1 -- 172.21.15.38:0/1105124189 shutdown_connections 2024-09-21T03:38:58.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:58.150+0000 7f5babf69640 1 -- 172.21.15.38:0/1105124189 wait complete. 2024-09-21T03:38:58.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:58.152+0000 7f5babf69640 1 Processor -- start 2024-09-21T03:38:58.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:58.152+0000 7f5babf69640 1 -- start start 2024-09-21T03:38:58.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:58.153+0000 7f5babf69640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5ba40d01f0 0x7f5ba41467a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:58.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:58.153+0000 7f5babf69640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5ba40d0980 0x7f5ba41f4290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:58.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:58.153+0000 7f5babf69640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5ba415a6f0 0x7f5ba41f3180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:58.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:58.153+0000 7f5babf69640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f5ba40cd130 con 0x7f5ba415a6f0 2024-09-21T03:38:58.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:58.153+0000 7f5babf69640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f5ba40ccfb0 con 0x7f5ba40d0980 2024-09-21T03:38:58.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:58.153+0000 7f5babf69640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f5ba40cd2b0 con 0x7f5ba40d01f0 2024-09-21T03:38:58.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:58.153+0000 7f5baa506640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5ba415a6f0 0x7f5ba41f3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:58.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:58.153+0000 7f5ba9504640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5ba40d0980 0x7f5ba41f4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:58.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:58.153+0000 7f5baa506640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5ba415a6f0 0x7f5ba41f3180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43266/0 (socket says 172.21.15.38:43266) 2024-09-21T03:38:58.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:58.153+0000 7f5baa506640 1 -- 172.21.15.38:0/727603064 learned_addr learned my addr 172.21.15.38:0/727603064 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:38:58.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:58.154+0000 7f5ba9d05640 1 --2- 172.21.15.38:0/727603064 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5ba40d01f0 0x7f5ba41467a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:58.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:58.154+0000 7f5baa506640 1 -- 172.21.15.38:0/727603064 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5ba40d01f0 msgr2=0x7f5ba41467a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:58.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:58.154+0000 7f5baa506640 1 --2- 172.21.15.38:0/727603064 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5ba40d01f0 0x7f5ba41467a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:58.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:58.154+0000 7f5baa506640 1 -- 172.21.15.38:0/727603064 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5ba40d0980 msgr2=0x7f5ba41f4290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:58.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:58.154+0000 7f5baa506640 1 --2- 172.21.15.38:0/727603064 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5ba40d0980 0x7f5ba41f4290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:58.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:58.154+0000 7f5baa506640 1 -- 172.21.15.38:0/727603064 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ba41f7470 con 0x7f5ba415a6f0 2024-09-21T03:38:58.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:58.154+0000 7f5baa506640 1 --2- 172.21.15.38:0/727603064 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5ba415a6f0 0x7f5ba41f3180 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f5b9c00b9e0 tx=0x7f5b9c00bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:58.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:58.154+0000 7f5b9a7fc640 1 -- 172.21.15.38:0/727603064 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b9c003b70 con 0x7f5ba415a6f0 2024-09-21T03:38:58.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:58.154+0000 7f5b9a7fc640 1 -- 172.21.15.38:0/727603064 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5b9c003e00 con 0x7f5ba415a6f0 2024-09-21T03:38:58.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:58.154+0000 7f5b9a7fc640 1 -- 172.21.15.38:0/727603064 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b9c00cb80 con 0x7f5ba415a6f0 2024-09-21T03:38:58.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:58.154+0000 7f5babf69640 1 -- 172.21.15.38:0/727603064 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ba41f7750 con 0x7f5ba415a6f0 2024-09-21T03:38:58.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:58.154+0000 7f5babf69640 1 -- 172.21.15.38:0/727603064 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ba41489d0 con 0x7f5ba415a6f0 2024-09-21T03:38:58.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:58.155+0000 7f5babf69640 1 -- 172.21.15.38:0/727603064 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ba41f71e0 con 0x7f5ba415a6f0 2024-09-21T03:38:58.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:58.156+0000 7f5b9a7fc640 1 -- 172.21.15.38:0/727603064 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f5b9c004000 con 0x7f5ba415a6f0 2024-09-21T03:38:58.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:58.157+0000 7f5b9a7fc640 1 --2- 172.21.15.38:0/727603064 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5b88078b90 0x7f5b8807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:58.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:58.157+0000 7f5b9a7fc640 1 -- 172.21.15.38:0/727603064 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5507+0+0 (secure 0 0 0) 0x7f5b9c09aa60 con 0x7f5ba415a6f0 2024-09-21T03:38:58.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:58.157+0000 7f5ba9d05640 1 --2- 172.21.15.38:0/727603064 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5b88078b90 0x7f5b8807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:58.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:58.158+0000 7f5ba9d05640 1 --2- 172.21.15.38:0/727603064 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5b88078b90 0x7f5b8807b0a0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f5ba0033000 tx=0x7f5ba0004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:58.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:58.161+0000 7f5b9a7fc640 1 -- 172.21.15.38:0/727603064 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f5b9c062900 con 0x7f5ba415a6f0 2024-09-21T03:38:58.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:58.422+0000 7f5babf69640 1 -- 172.21.15.38:0/727603064 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_damage.TestDamage.test_dentry_first_preflush"]} v 0) -- 0x7f5ba41f7c70 con 0x7f5ba415a6f0 2024-09-21T03:38:59.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:59.359+0000 7f5b9a7fc640 1 -- 172.21.15.38:0/727603064 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_damage.TestDamage.test_dentry_first_preflush"]}]=0 v262) ==== 144+0+0 (secure 0 0 0) 0x7f5ba41f7c70 con 0x7f5ba415a6f0 2024-09-21T03:38:59.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:59.362+0000 7f5babf69640 1 -- 172.21.15.38:0/727603064 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5b88078b90 msgr2=0x7f5b8807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:59.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:59.362+0000 7f5babf69640 1 --2- 172.21.15.38:0/727603064 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5b88078b90 0x7f5b8807b0a0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f5ba0033000 tx=0x7f5ba0004730 comp rx=0 tx=0).stop 2024-09-21T03:38:59.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:59.362+0000 7f5babf69640 1 -- 172.21.15.38:0/727603064 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5ba415a6f0 msgr2=0x7f5ba41f3180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:59.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:59.362+0000 7f5babf69640 1 --2- 172.21.15.38:0/727603064 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5ba415a6f0 0x7f5ba41f3180 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f5b9c00b9e0 tx=0x7f5b9c00bea0 comp rx=0 tx=0).stop 2024-09-21T03:38:59.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:59.362+0000 7f5babf69640 1 -- 172.21.15.38:0/727603064 shutdown_connections 2024-09-21T03:38:59.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:59.362+0000 7f5babf69640 1 --2- 172.21.15.38:0/727603064 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5ba415a6f0 0x7f5ba41f3180 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:59.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:59.362+0000 7f5babf69640 1 --2- 172.21.15.38:0/727603064 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5ba40d0980 0x7f5ba41f4290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:59.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:59.362+0000 7f5babf69640 1 --2- 172.21.15.38:0/727603064 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5b88078b90 0x7f5b8807b0a0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:59.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:59.362+0000 7f5babf69640 1 --2- 172.21.15.38:0/727603064 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5ba40d01f0 0x7f5ba41467a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:59.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:59.362+0000 7f5babf69640 1 -- 172.21.15.38:0/727603064 >> 172.21.15.38:0/727603064 conn(0x7f5ba4072990 msgr2=0x7f5ba415c690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:59.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:59.362+0000 7f5babf69640 1 -- 172.21.15.38:0/727603064 shutdown_connections 2024-09-21T03:38:59.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:59.362+0000 7f5babf69640 1 -- 172.21.15.38:0/727603064 wait complete. 2024-09-21T03:38:59.378 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-21T03:38:59.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:59.500+0000 7f39b78cc640 1 -- 172.21.15.38:0/2051254868 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f399c004d90 msgr2=0x7f399c005220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:59.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:59.500+0000 7f39b78cc640 1 --2- 172.21.15.38:0/2051254868 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f399c004d90 0x7f399c005220 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f39a8001f00 tx=0x7f399c005830 comp rx=0 tx=0).stop 2024-09-21T03:38:59.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:59.500+0000 7f39b5e69640 1 -- 172.21.15.38:0/2051254868 reap_dead start 2024-09-21T03:38:59.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:59.500+0000 7f39b78cc640 1 -- 172.21.15.38:0/2051254868 shutdown_connections 2024-09-21T03:38:59.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:59.500+0000 7f39b78cc640 1 -- 172.21.15.38:0/2051254868 >> 172.21.15.38:0/2051254868 conn(0x7f39b0072990 msgr2=0x7f39b0072da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:59.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:59.500+0000 7f39b78cc640 1 -- 172.21.15.38:0/2051254868 shutdown_connections 2024-09-21T03:38:59.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:59.500+0000 7f39b78cc640 1 -- 172.21.15.38:0/2051254868 wait complete. 2024-09-21T03:38:59.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:59.502+0000 7f39b78cc640 1 Processor -- start 2024-09-21T03:38:59.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:59.502+0000 7f39b78cc640 1 -- start start 2024-09-21T03:38:59.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:59.504+0000 7f39b78cc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f399c004d90 0x7f39b01f4090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:59.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:59.504+0000 7f39b78cc640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f39b00aa270 0x7f39b01567e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:59.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:59.504+0000 7f39b78cc640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f39b00aabc0 0x7f39b00aafd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:59.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:59.504+0000 7f39b78cc640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f39b00ccfb0 con 0x7f399c004d90 2024-09-21T03:38:59.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:59.504+0000 7f39b78cc640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f39b015dd90 con 0x7f39b00aabc0 2024-09-21T03:38:59.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:59.504+0000 7f39b78cc640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f39b00cd130 con 0x7f39b00aa270 2024-09-21T03:38:59.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:59.504+0000 7f39b5668640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f399c004d90 0x7f39b01f4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:59.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:59.504+0000 7f39b5668640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f399c004d90 0x7f39b01f4090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43282/0 (socket says 172.21.15.38:43282) 2024-09-21T03:38:59.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:59.504+0000 7f39b5668640 1 -- 172.21.15.38:0/2517826628 learned_addr learned my addr 172.21.15.38:0/2517826628 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:38:59.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:59.504+0000 7f39b4e67640 1 --2- 172.21.15.38:0/2517826628 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f39b00aa270 0x7f39b01567e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:59.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:59.504+0000 7f39b5e69640 1 --2- 172.21.15.38:0/2517826628 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f39b00aabc0 0x7f39b00aafd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:59.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:59.504+0000 7f39b5668640 1 -- 172.21.15.38:0/2517826628 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f39b00aa270 msgr2=0x7f39b01567e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:59.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:59.504+0000 7f39b5668640 1 --2- 172.21.15.38:0/2517826628 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f39b00aa270 0x7f39b01567e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:59.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:59.504+0000 7f39b5668640 1 -- 172.21.15.38:0/2517826628 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f39b00aabc0 msgr2=0x7f39b00aafd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:59.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:59.504+0000 7f39b5668640 1 --2- 172.21.15.38:0/2517826628 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f39b00aabc0 0x7f39b00aafd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:59.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:59.504+0000 7f39b5668640 1 -- 172.21.15.38:0/2517826628 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39b00ab510 con 0x7f399c004d90 2024-09-21T03:38:59.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:59.504+0000 7f39b5668640 1 --2- 172.21.15.38:0/2517826628 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f399c004d90 0x7f39b01f4090 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f39ac011ae0 tx=0x7f39ac009e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:59.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:59.504+0000 7f39a1ffb640 1 -- 172.21.15.38:0/2517826628 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39ac01e070 con 0x7f399c004d90 2024-09-21T03:38:59.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:59.504+0000 7f39a1ffb640 1 -- 172.21.15.38:0/2517826628 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f39ac004460 con 0x7f399c004d90 2024-09-21T03:38:59.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:59.504+0000 7f39b78cc640 1 -- 172.21.15.38:0/2517826628 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39b00a94f0 con 0x7f399c004d90 2024-09-21T03:38:59.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:59.504+0000 7f39a1ffb640 1 -- 172.21.15.38:0/2517826628 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39ac00fc60 con 0x7f399c004d90 2024-09-21T03:38:59.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:59.505+0000 7f39b78cc640 1 -- 172.21.15.38:0/2517826628 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39b00a9a70 con 0x7f399c004d90 2024-09-21T03:38:59.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:59.506+0000 7f39b78cc640 1 -- 172.21.15.38:0/2517826628 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39b00cd690 con 0x7f399c004d90 2024-09-21T03:38:59.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:59.506+0000 7f39a1ffb640 1 -- 172.21.15.38:0/2517826628 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f39ac021070 con 0x7f399c004d90 2024-09-21T03:38:59.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:59.507+0000 7f39a1ffb640 1 --2- 172.21.15.38:0/2517826628 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3994078b90 0x7f399407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:59.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:59.507+0000 7f39a1ffb640 1 -- 172.21.15.38:0/2517826628 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5507+0+0 (secure 0 0 0) 0x7f39ac09b7c0 con 0x7f399c004d90 2024-09-21T03:38:59.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:59.507+0000 7f39b4e67640 1 --2- 172.21.15.38:0/2517826628 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3994078b90 0x7f399407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:59.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:59.508+0000 7f39b4e67640 1 --2- 172.21.15.38:0/2517826628 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3994078b90 0x7f399407b0a0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f39a4000f60 tx=0x7f39a4000ed0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:59.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:59.512+0000 7f39a1ffb640 1 -- 172.21.15.38:0/2517826628 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f39b00ab510 con 0x7f399c004d90 2024-09-21T03:38:59.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:59.700+0000 7f39b78cc640 1 -- 172.21.15.38:0/2517826628 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f39b015a0a0 con 0x7f399c004d90 2024-09-21T03:38:59.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:59.701+0000 7f39a1ffb640 1 -- 172.21.15.38:0/2517826628 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v42) ==== 59+0+2294 (secure 0 0 0) 0x7f39ac063690 con 0x7f399c004d90 2024-09-21T03:38:59.703 INFO:teuthology.orchestra.run.smithi038.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-21T03:38:59.703 INFO:teuthology.orchestra.run.smithi038.stdout:mon advanced mon_op_complaint_time 120 2024-09-21T03:38:59.703 INFO:teuthology.orchestra.run.smithi038.stdout:mgr advanced client_mount_timeout 30 2024-09-21T03:38:59.703 INFO:teuthology.orchestra.run.smithi038.stdout:mgr advanced debug_client 20/20 2024-09-21T03:38:59.703 INFO:teuthology.orchestra.run.smithi038.stdout:mgr advanced debug_mgr 20/20 2024-09-21T03:38:59.703 INFO:teuthology.orchestra.run.smithi038.stdout:mgr advanced debug_ms 1/1 2024-09-21T03:38:59.703 INFO:teuthology.orchestra.run.smithi038.stdout:mgr dev mon_warn_on_pool_no_app false 2024-09-21T03:38:59.703 INFO:teuthology.orchestra.run.smithi038.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-21T03:38:59.703 INFO:teuthology.orchestra.run.smithi038.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 3844.962174 2024-09-21T03:38:59.704 INFO:teuthology.orchestra.run.smithi038.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 4916.015278 2024-09-21T03:38:59.704 INFO:teuthology.orchestra.run.smithi038.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 4165.407418 2024-09-21T03:38:59.704 INFO:teuthology.orchestra.run.smithi038.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 3760.481604 2024-09-21T03:38:59.704 INFO:teuthology.orchestra.run.smithi038.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 6052.882314 2024-09-21T03:38:59.704 INFO:teuthology.orchestra.run.smithi038.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 8967.576248 2024-09-21T03:38:59.704 INFO:teuthology.orchestra.run.smithi038.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 6129.316416 2024-09-21T03:38:59.704 INFO:teuthology.orchestra.run.smithi038.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 8751.950197 2024-09-21T03:38:59.704 INFO:teuthology.orchestra.run.smithi038.stdout:mds advanced debug_mds 20/20 2024-09-21T03:38:59.704 INFO:teuthology.orchestra.run.smithi038.stdout:mds advanced debug_mds_balancer 20/20 2024-09-21T03:38:59.704 INFO:teuthology.orchestra.run.smithi038.stdout:mds advanced debug_ms 2/2 2024-09-21T03:38:59.704 INFO:teuthology.orchestra.run.smithi038.stdout:mds dev mds_debug_frag true 2024-09-21T03:38:59.704 INFO:teuthology.orchestra.run.smithi038.stdout:mds dev mds_debug_scatterstat true 2024-09-21T03:38:59.704 INFO:teuthology.orchestra.run.smithi038.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-21T03:38:59.704 INFO:teuthology.orchestra.run.smithi038.stdout:mds dev mds_verify_scatter true 2024-09-21T03:38:59.704 INFO:teuthology.orchestra.run.smithi038.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-21T03:38:59.705 INFO:teuthology.orchestra.run.smithi038.stdout:mds advanced rados_mon_op_timeout 900 2024-09-21T03:38:59.705 INFO:teuthology.orchestra.run.smithi038.stdout:mds advanced rados_osd_op_timeout 900 2024-09-21T03:38:59.705 INFO:teuthology.orchestra.run.smithi038.stdout:client advanced client_mount_timeout 600 2024-09-21T03:38:59.705 INFO:teuthology.orchestra.run.smithi038.stdout:client advanced debug_client 20/20 2024-09-21T03:38:59.705 INFO:teuthology.orchestra.run.smithi038.stdout:client advanced debug_ms 1/1 2024-09-21T03:38:59.705 INFO:teuthology.orchestra.run.smithi038.stdout:client advanced rados_mon_op_timeout 900 2024-09-21T03:38:59.705 INFO:teuthology.orchestra.run.smithi038.stdout:client advanced rados_osd_op_timeout 900 2024-09-21T03:38:59.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:59.703+0000 7f39b78cc640 1 -- 172.21.15.38:0/2517826628 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3994078b90 msgr2=0x7f399407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:59.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:59.703+0000 7f39b78cc640 1 --2- 172.21.15.38:0/2517826628 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3994078b90 0x7f399407b0a0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f39a4000f60 tx=0x7f39a4000ed0 comp rx=0 tx=0).stop 2024-09-21T03:38:59.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:59.703+0000 7f39b78cc640 1 -- 172.21.15.38:0/2517826628 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f399c004d90 msgr2=0x7f39b01f4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:59.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:59.703+0000 7f39b78cc640 1 --2- 172.21.15.38:0/2517826628 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f399c004d90 0x7f39b01f4090 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f39ac011ae0 tx=0x7f39ac009e90 comp rx=0 tx=0).stop 2024-09-21T03:38:59.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:59.703+0000 7f39b78cc640 1 -- 172.21.15.38:0/2517826628 shutdown_connections 2024-09-21T03:38:59.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:59.703+0000 7f39b78cc640 1 --2- 172.21.15.38:0/2517826628 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f39b00aabc0 0x7f39b00aafd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:59.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:59.703+0000 7f39b78cc640 1 --2- 172.21.15.38:0/2517826628 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3994078b90 0x7f399407b0a0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:59.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:59.703+0000 7f39b78cc640 1 --2- 172.21.15.38:0/2517826628 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f39b00aa270 0x7f39b01567e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:59.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:59.703+0000 7f39b78cc640 1 --2- 172.21.15.38:0/2517826628 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f399c004d90 0x7f39b01f4090 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:59.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:59.703+0000 7f39b78cc640 1 -- 172.21.15.38:0/2517826628 >> 172.21.15.38:0/2517826628 conn(0x7f39b0072990 msgr2=0x7f39b015eb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:59.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:59.703+0000 7f39b78cc640 1 -- 172.21.15.38:0/2517826628 shutdown_connections 2024-09-21T03:38:59.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:59.703+0000 7f39b78cc640 1 -- 172.21.15.38:0/2517826628 wait complete. 2024-09-21T03:38:59.716 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-21T03:38:59.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:59.834+0000 7f1775abd640 1 -- 172.21.15.38:0/2689471908 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f177015bfc0 msgr2=0x7f177015f630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:59.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:59.834+0000 7f1775abd640 1 --2- 172.21.15.38:0/2689471908 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f177015bfc0 0x7f177015f630 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f177006e6e0 tx=0x7f176802aa00 comp rx=0 tx=0).stop 2024-09-21T03:38:59.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:59.834+0000 7f1775abd640 1 -- 172.21.15.38:0/2689471908 shutdown_connections 2024-09-21T03:38:59.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:59.834+0000 7f1775abd640 1 --2- 172.21.15.38:0/2689471908 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f177015bfc0 0x7f177015f630 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:59.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:59.834+0000 7f1775abd640 1 --2- 172.21.15.38:0/2689471908 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f17700b5260 0x7f177015ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:59.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:59.834+0000 7f1775abd640 1 --2- 172.21.15.38:0/2689471908 >> v2:172.21.15.195:3300/0 conn(0x7f177006def0 0x7f177006f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:59.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:59.834+0000 7f1775abd640 1 -- 172.21.15.38:0/2689471908 >> 172.21.15.38:0/2689471908 conn(0x7f17700cdca0 msgr2=0x7f17700ce0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:59.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:59.834+0000 7f1775abd640 1 -- 172.21.15.38:0/2689471908 shutdown_connections 2024-09-21T03:38:59.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:59.834+0000 7f1775abd640 1 -- 172.21.15.38:0/2689471908 wait complete. 2024-09-21T03:38:59.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:59.836+0000 7f1775abd640 1 Processor -- start 2024-09-21T03:38:59.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:59.836+0000 7f1775abd640 1 -- start start 2024-09-21T03:38:59.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:59.837+0000 7f1775abd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f177006def0 0x7f17700b2b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:59.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:59.837+0000 7f1775abd640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f17700b5260 0x7f17700ccf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:59.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:59.837+0000 7f1775abd640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f177015bfc0 0x7f17700cfe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:59.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:59.837+0000 7f1775abd640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f177015fcf0 con 0x7f177006def0 2024-09-21T03:38:59.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:59.837+0000 7f1775abd640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f177015fb70 con 0x7f177015bfc0 2024-09-21T03:38:59.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:59.837+0000 7f1775abd640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f177015fe70 con 0x7f17700b5260 2024-09-21T03:38:59.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:59.838+0000 7f176f7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f177006def0 0x7f17700b2b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:59.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:59.838+0000 7f176f7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f177006def0 0x7f17700b2b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43298/0 (socket says 172.21.15.38:43298) 2024-09-21T03:38:59.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:59.838+0000 7f176f7fe640 1 -- 172.21.15.38:0/2148248705 learned_addr learned my addr 172.21.15.38:0/2148248705 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:38:59.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:59.838+0000 7f176ffff640 1 --2- 172.21.15.38:0/2148248705 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f177015bfc0 0x7f17700cfe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:59.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:59.838+0000 7f176effd640 1 --2- 172.21.15.38:0/2148248705 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f17700b5260 0x7f17700ccf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:59.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:59.838+0000 7f176f7fe640 1 -- 172.21.15.38:0/2148248705 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f17700b5260 msgr2=0x7f17700ccf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:59.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:59.838+0000 7f176f7fe640 1 --2- 172.21.15.38:0/2148248705 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f17700b5260 0x7f17700ccf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:59.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:59.838+0000 7f176f7fe640 1 -- 172.21.15.38:0/2148248705 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f177015bfc0 msgr2=0x7f17700cfe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:59.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:59.838+0000 7f176f7fe640 1 --2- 172.21.15.38:0/2148248705 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f177015bfc0 0x7f17700cfe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:59.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:59.838+0000 7f176f7fe640 1 -- 172.21.15.38:0/2148248705 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17701f89f0 con 0x7f177006def0 2024-09-21T03:38:59.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:59.838+0000 7f176f7fe640 1 --2- 172.21.15.38:0/2148248705 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f177006def0 0x7f17700b2b40 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f176000d700 tx=0x7f176000dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:59.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:59.838+0000 7f1757fff640 1 -- 172.21.15.38:0/2148248705 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1760017070 con 0x7f177006def0 2024-09-21T03:38:59.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:59.838+0000 7f1757fff640 1 -- 172.21.15.38:0/2148248705 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1760004310 con 0x7f177006def0 2024-09-21T03:38:59.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:59.838+0000 7f1775abd640 1 -- 172.21.15.38:0/2148248705 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17700cd840 con 0x7f177006def0 2024-09-21T03:38:59.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:59.838+0000 7f1757fff640 1 -- 172.21.15.38:0/2148248705 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1760009a80 con 0x7f177006def0 2024-09-21T03:38:59.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:59.838+0000 7f1775abd640 1 -- 172.21.15.38:0/2148248705 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17701f8f70 con 0x7f177006def0 2024-09-21T03:38:59.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:59.839+0000 7f1775abd640 1 -- 172.21.15.38:0/2148248705 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17701f8800 con 0x7f177006def0 2024-09-21T03:38:59.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:59.840+0000 7f1757fff640 1 -- 172.21.15.38:0/2148248705 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f1760009c20 con 0x7f177006def0 2024-09-21T03:38:59.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:59.841+0000 7f1757fff640 1 --2- 172.21.15.38:0/2148248705 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f175c078bc0 0x7f175c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:59.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:59.841+0000 7f1757fff640 1 -- 172.21.15.38:0/2148248705 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5507+0+0 (secure 0 0 0) 0x7f176009ac40 con 0x7f177006def0 2024-09-21T03:38:59.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:59.841+0000 7f176effd640 1 --2- 172.21.15.38:0/2148248705 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f175c078bc0 0x7f175c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:59.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:59.842+0000 7f176effd640 1 --2- 172.21.15.38:0/2148248705 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f175c078bc0 0x7f175c07b0d0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f17700c4b50 tx=0x7f176400b2e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:59.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:38:59.846+0000 7f1757fff640 1 -- 172.21.15.38:0/2148248705 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f1760062af0 con 0x7f177006def0 2024-09-21T03:39:00.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:00.032+0000 7f1775abd640 1 -- 172.21.15.38:0/2148248705 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f177015ba80 con 0x7f177006def0 2024-09-21T03:39:00.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:00.037+0000 7f1757fff640 1 -- 172.21.15.38:0/2148248705 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v43)=0 v43) ==== 122+0+0 (secure 0 0 0) 0x7f1760067520 con 0x7f177006def0 2024-09-21T03:39:00.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:00.039+0000 7f1775abd640 1 -- 172.21.15.38:0/2148248705 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f175c078bc0 msgr2=0x7f175c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:00.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:00.039+0000 7f1775abd640 1 --2- 172.21.15.38:0/2148248705 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f175c078bc0 0x7f175c07b0d0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f17700c4b50 tx=0x7f176400b2e0 comp rx=0 tx=0).stop 2024-09-21T03:39:00.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:00.039+0000 7f1775abd640 1 -- 172.21.15.38:0/2148248705 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f177006def0 msgr2=0x7f17700b2b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:00.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:00.039+0000 7f1775abd640 1 --2- 172.21.15.38:0/2148248705 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f177006def0 0x7f17700b2b40 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f176000d700 tx=0x7f176000dbc0 comp rx=0 tx=0).stop 2024-09-21T03:39:00.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:00.039+0000 7f1775abd640 1 -- 172.21.15.38:0/2148248705 shutdown_connections 2024-09-21T03:39:00.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:00.039+0000 7f1775abd640 1 --2- 172.21.15.38:0/2148248705 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f177015bfc0 0x7f17700cfe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:00.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:00.039+0000 7f1775abd640 1 --2- 172.21.15.38:0/2148248705 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f175c078bc0 0x7f175c07b0d0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:00.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:00.039+0000 7f1775abd640 1 --2- 172.21.15.38:0/2148248705 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f17700b5260 0x7f17700ccf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:00.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:00.039+0000 7f1775abd640 1 --2- 172.21.15.38:0/2148248705 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f177006def0 0x7f17700b2b40 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:00.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:00.039+0000 7f1775abd640 1 -- 172.21.15.38:0/2148248705 >> 172.21.15.38:0/2148248705 conn(0x7f17700cdca0 msgr2=0x7f17700b31a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:00.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:00.039+0000 7f1775abd640 1 -- 172.21.15.38:0/2148248705 shutdown_connections 2024-09-21T03:39:00.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:00.039+0000 7f1775abd640 1 -- 172.21.15.38:0/2148248705 wait complete. 2024-09-21T03:39:00.052 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-21T03:39:00.052 DEBUG:teuthology.orchestra.run.smithi038:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T03:39:00.093 DEBUG:teuthology.orchestra.run.smithi195:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T03:39:00.134 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-21T03:39:00.134 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-21T03:39:00.134 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:39:00.150 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056795,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2354 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T03:39:00.150 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-09-21T03:39:00.150 DEBUG:teuthology.orchestra.run.smithi195:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:39:00.192 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056014,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497444k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=8978 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248720k,nr_inodes=812180,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497444k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T03:39:00.193 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-21T03:39:00.193 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:39:00.208 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056795,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2354 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T03:39:00.208 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-09-21T03:39:00.209 DEBUG:teuthology.orchestra.run.smithi195:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:39:00.250 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056014,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497444k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=8978 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248720k,nr_inodes=812180,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497444k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T03:39:00.250 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-21T03:39:00.250 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:39:00.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:00.366+0000 7f7509b8a640 1 -- 172.21.15.38:0/1848952743 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f750415daf0 msgr2=0x7f750415a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:00.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:00.366+0000 7f7509b8a640 1 --2- 172.21.15.38:0/1848952743 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f750415daf0 0x7f750415a6f0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f74f8009830 tx=0x7f74f802a510 comp rx=0 tx=0).stop 2024-09-21T03:39:00.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:00.366+0000 7f7509b8a640 1 -- 172.21.15.38:0/1848952743 shutdown_connections 2024-09-21T03:39:00.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:00.366+0000 7f7509b8a640 1 --2- 172.21.15.38:0/1848952743 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f75040d0980 0x7f75040d0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:00.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:00.366+0000 7f7509b8a640 1 --2- 172.21.15.38:0/1848952743 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f750415daf0 0x7f750415a6f0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:00.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:00.366+0000 7f7509b8a640 1 -- 172.21.15.38:0/1848952743 >> 172.21.15.38:0/1848952743 conn(0x7f7504072990 msgr2=0x7f7504072da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:00.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:00.366+0000 7f7509b8a640 1 -- 172.21.15.38:0/1848952743 shutdown_connections 2024-09-21T03:39:00.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:00.366+0000 7f7509b8a640 1 -- 172.21.15.38:0/1848952743 wait complete. 2024-09-21T03:39:00.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:00.368+0000 7f7509b8a640 1 Processor -- start 2024-09-21T03:39:00.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:00.368+0000 7f7509b8a640 1 -- start start 2024-09-21T03:39:00.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:00.370+0000 7f7509b8a640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f75040d01f0 0x7f75040b1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:00.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:00.370+0000 7f7509b8a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f75040d0980 0x7f75040c9fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:00.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:00.370+0000 7f7509b8a640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f750415daf0 0x7f75040ca510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:00.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:00.370+0000 7f7509b8a640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f75040cd340 con 0x7f75040d0980 2024-09-21T03:39:00.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:00.370+0000 7f7509b8a640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f75040cd1c0 con 0x7f75040d01f0 2024-09-21T03:39:00.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:00.370+0000 7f7509b8a640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f75040cd4c0 con 0x7f750415daf0 2024-09-21T03:39:00.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:00.370+0000 7f7502ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f75040d0980 0x7f75040c9fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:00.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:00.370+0000 7f7502ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f75040d0980 0x7f75040c9fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43306/0 (socket says 172.21.15.38:43306) 2024-09-21T03:39:00.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:00.370+0000 7f7502ffd640 1 -- 172.21.15.38:0/3194354349 learned_addr learned my addr 172.21.15.38:0/3194354349 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:39:00.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:00.370+0000 7f75037fe640 1 --2- 172.21.15.38:0/3194354349 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f75040d01f0 0x7f75040b1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:00.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:00.370+0000 7f7503fff640 1 --2- 172.21.15.38:0/3194354349 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f750415daf0 0x7f75040ca510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:00.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:00.370+0000 7f7502ffd640 1 -- 172.21.15.38:0/3194354349 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f750415daf0 msgr2=0x7f75040ca510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:00.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:00.370+0000 7f7502ffd640 1 --2- 172.21.15.38:0/3194354349 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f750415daf0 0x7f75040ca510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:00.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:00.370+0000 7f7502ffd640 1 -- 172.21.15.38:0/3194354349 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f75040d01f0 msgr2=0x7f75040b1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:00.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:00.370+0000 7f7502ffd640 1 --2- 172.21.15.38:0/3194354349 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f75040d01f0 0x7f75040b1bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:00.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:00.370+0000 7f7502ffd640 1 -- 172.21.15.38:0/3194354349 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7504151250 con 0x7f75040d0980 2024-09-21T03:39:00.374 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:00.370+0000 7f7502ffd640 1 --2- 172.21.15.38:0/3194354349 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f75040d0980 0x7f75040c9fd0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f74f8000c00 tx=0x7f74f8033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:00.374 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:00.371+0000 7f74ebfff640 1 -- 172.21.15.38:0/3194354349 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74f8005c90 con 0x7f75040d0980 2024-09-21T03:39:00.374 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:00.371+0000 7f7509b8a640 1 -- 172.21.15.38:0/3194354349 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7504151530 con 0x7f75040d0980 2024-09-21T03:39:00.374 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:00.371+0000 7f74ebfff640 1 -- 172.21.15.38:0/3194354349 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f74f8031e50 con 0x7f75040d0980 2024-09-21T03:39:00.374 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:00.371+0000 7f74ebfff640 1 -- 172.21.15.38:0/3194354349 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74f8033310 con 0x7f75040d0980 2024-09-21T03:39:00.374 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:00.371+0000 7f7509b8a640 1 -- 172.21.15.38:0/3194354349 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7504151af0 con 0x7f75040d0980 2024-09-21T03:39:00.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:00.372+0000 7f7509b8a640 1 -- 172.21.15.38:0/3194354349 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75040cd970 con 0x7f75040d0980 2024-09-21T03:39:00.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:00.372+0000 7f74ebfff640 1 -- 172.21.15.38:0/3194354349 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f74f80334b0 con 0x7f75040d0980 2024-09-21T03:39:00.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:00.373+0000 7f74ebfff640 1 --2- 172.21.15.38:0/3194354349 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f74f0078b90 0x7f74f007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:00.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:00.373+0000 7f74ebfff640 1 -- 172.21.15.38:0/3194354349 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5507+0+0 (secure 0 0 0) 0x7f74f80bbc60 con 0x7f75040d0980 2024-09-21T03:39:00.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:00.373+0000 7f75037fe640 1 --2- 172.21.15.38:0/3194354349 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f74f0078b90 0x7f74f007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:00.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:00.374+0000 7f75037fe640 1 --2- 172.21.15.38:0/3194354349 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f74f0078b90 0x7f74f007b0a0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f75040cec40 tx=0x7f74f4007e90 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:00.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:00.377+0000 7f74ebfff640 1 -- 172.21.15.38:0/3194354349 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f74f804a030 con 0x7f75040d0980 2024-09-21T03:39:00.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:00.603+0000 7f7509b8a640 1 -- 172.21.15.38:0/3194354349 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f750415f650 con 0x7f75040d0980 2024-09-21T03:39:00.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:00.604+0000 7f74ebfff640 1 -- 172.21.15.38:0/3194354349 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) ==== 94+0+3914 (secure 0 0 0) 0x7f74f803f030 con 0x7f75040d0980 2024-09-21T03:39:00.606 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:39:00.606 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":61,"btime":"2024-09-21T03:38:54:194752+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5432,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/1042764291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":1042764291},{"type":"v1","addr":"172.21.15.195:6837","nonce":1042764291}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":5720,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/444747999","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":444747999},{"type":"v1","addr":"172.21.15.195:6836","nonce":444747999}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":61},{"gid":5728,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/3035019482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6833","nonce":3035019482},{"type":"v1","addr":"172.21.15.38:6835","nonce":3035019482}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":6348,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6834/3775275137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":3775275137},{"type":"v1","addr":"172.21.15.38:6834","nonce":3775275137}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57}],"filesystems":[]} 2024-09-21T03:39:00.606 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 61 2024-09-21T03:39:00.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:00.606+0000 7f7509b8a640 1 -- 172.21.15.38:0/3194354349 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f74f0078b90 msgr2=0x7f74f007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:00.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:00.606+0000 7f7509b8a640 1 --2- 172.21.15.38:0/3194354349 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f74f0078b90 0x7f74f007b0a0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f75040cec40 tx=0x7f74f4007e90 comp rx=0 tx=0).stop 2024-09-21T03:39:00.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:00.606+0000 7f7509b8a640 1 -- 172.21.15.38:0/3194354349 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f75040d0980 msgr2=0x7f75040c9fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:00.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:00.606+0000 7f7509b8a640 1 --2- 172.21.15.38:0/3194354349 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f75040d0980 0x7f75040c9fd0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f74f8000c00 tx=0x7f74f8033c20 comp rx=0 tx=0).stop 2024-09-21T03:39:00.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:00.606+0000 7f7509b8a640 1 -- 172.21.15.38:0/3194354349 shutdown_connections 2024-09-21T03:39:00.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:00.606+0000 7f7509b8a640 1 --2- 172.21.15.38:0/3194354349 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f74f0078b90 0x7f74f007b0a0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:00.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:00.606+0000 7f7509b8a640 1 --2- 172.21.15.38:0/3194354349 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f750415daf0 0x7f75040ca510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:00.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:00.606+0000 7f7509b8a640 1 --2- 172.21.15.38:0/3194354349 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f75040d0980 0x7f75040c9fd0 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:00.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:00.606+0000 7f7509b8a640 1 --2- 172.21.15.38:0/3194354349 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f75040d01f0 0x7f75040b1bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:00.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:00.606+0000 7f7509b8a640 1 -- 172.21.15.38:0/3194354349 >> 172.21.15.38:0/3194354349 conn(0x7f7504072990 msgr2=0x7f75040c7d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:00.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:00.606+0000 7f7509b8a640 1 -- 172.21.15.38:0/3194354349 shutdown_connections 2024-09-21T03:39:00.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:00.606+0000 7f7509b8a640 1 -- 172.21.15.38:0/3194354349 wait complete. 2024-09-21T03:39:00.620 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-21T03:39:00.620 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-21T03:39:00.620 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-21T03:39:00.620 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T03:39:00.621 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-21T03:39:00.621 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-21T03:39:00.621 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-21T03:39:00.621 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T03:39:00.621 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-21T03:39:00.621 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-21T03:39:00.622 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-21T03:39:00.622 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T03:39:00.622 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-21T03:39:00.622 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-21T03:39:00.622 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-21T03:39:00.622 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T03:39:00.630 INFO:tasks.ceph.mds.a:Stopped 2024-09-21T03:39:00.630 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-21T03:39:00.631 INFO:tasks.ceph.mds.c:Stopped 2024-09-21T03:39:00.631 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-21T03:39:00.635 INFO:tasks.ceph.mds.d:Stopped 2024-09-21T03:39:00.635 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-21T03:39:00.636 INFO:tasks.ceph.mds.b:Stopped 2024-09-21T03:39:00.636 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-21T03:39:00.673 INFO:tasks.ceph.mds.a:Started 2024-09-21T03:39:00.673 DEBUG:teuthology.parallel:result is None 2024-09-21T03:39:00.674 INFO:tasks.ceph.mds.c:Started 2024-09-21T03:39:00.674 DEBUG:teuthology.parallel:result is None 2024-09-21T03:39:00.679 INFO:tasks.ceph.mds.d:Started 2024-09-21T03:39:00.679 DEBUG:teuthology.parallel:result is None 2024-09-21T03:39:00.680 INFO:tasks.ceph.mds.b:Started 2024-09-21T03:39:00.681 DEBUG:teuthology.parallel:result is None 2024-09-21T03:39:00.681 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-21T03:39:00.761 INFO:tasks.ceph.mds.c.smithi038.stdout:starting mds.c at 2024-09-21T03:39:00.763 INFO:tasks.ceph.mds.a.smithi038.stdout:starting mds.a at 2024-09-21T03:39:00.774 INFO:tasks.ceph.mds.b.smithi195.stdout:starting mds.b at 2024-09-21T03:39:00.775 INFO:tasks.ceph.mds.d.smithi195.stdout:starting mds.d at 2024-09-21T03:39:00.775 INFO:tasks.ceph.mds.b.smithi195.stderr:2024-09-21T03:39:00.774+0000 7f5969ac7640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2024-09-21T03:39:00.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:00.802+0000 7f6f06372640 1 -- 172.21.15.38:0/2200372637 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6f000b2ca0 msgr2=0x7f6f000b30b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:00.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:00.802+0000 7f6f06372640 1 --2- 172.21.15.38:0/2200372637 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6f000b2ca0 0x7f6f000b30b0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f6ef4009860 tx=0x7f6ef402a4a0 comp rx=0 tx=0).stop 2024-09-21T03:39:00.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:00.802+0000 7f6f06372640 1 -- 172.21.15.38:0/2200372637 shutdown_connections 2024-09-21T03:39:00.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:00.802+0000 7f6f06372640 1 --2- 172.21.15.38:0/2200372637 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6f000b2ca0 0x7f6f000b30b0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:00.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:00.802+0000 7f6f06372640 1 -- 172.21.15.38:0/2200372637 >> 172.21.15.38:0/2200372637 conn(0x7f6f000cdca0 msgr2=0x7f6f000ce0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:00.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:00.802+0000 7f6f06372640 1 -- 172.21.15.38:0/2200372637 shutdown_connections 2024-09-21T03:39:00.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:00.802+0000 7f6f06372640 1 -- 172.21.15.38:0/2200372637 wait complete. 2024-09-21T03:39:00.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:00.804+0000 7f6f06372640 1 Processor -- start 2024-09-21T03:39:00.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:00.804+0000 7f6f06372640 1 -- start start 2024-09-21T03:39:00.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:00.806+0000 7f6f06372640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6f0006ff00 0x7f6f0014f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:00.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:00.806+0000 7f6f06372640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6f000b2ca0 0x7f6f001f6ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:00.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:00.806+0000 7f6f06372640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6f0015c750 0x7f6f001f7420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:00.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:00.806+0000 7f6f06372640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f6f000af200 con 0x7f6f000b2ca0 2024-09-21T03:39:00.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:00.806+0000 7f6f06372640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f6f000af080 con 0x7f6f0015c750 2024-09-21T03:39:00.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:00.806+0000 7f6eff7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6f000b2ca0 0x7f6f001f6ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:00.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:00.806+0000 7f6eff7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6f000b2ca0 0x7f6f001f6ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43336/0 (socket says 172.21.15.38:43336) 2024-09-21T03:39:00.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:00.806+0000 7f6eff7fe640 1 -- 172.21.15.38:0/3904789385 learned_addr learned my addr 172.21.15.38:0/3904789385 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:39:00.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:00.806+0000 7f6f06372640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f6f00070a20 con 0x7f6f0006ff00 2024-09-21T03:39:00.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:00.806+0000 7f6efffff640 1 --2- 172.21.15.38:0/3904789385 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6f0006ff00 0x7f6f0014f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:00.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:00.806+0000 7f6f0490f640 1 --2- 172.21.15.38:0/3904789385 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6f0015c750 0x7f6f001f7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:00.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:00.806+0000 7f6eff7fe640 1 -- 172.21.15.38:0/3904789385 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6f0006ff00 msgr2=0x7f6f0014f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:00.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:00.806+0000 7f6eff7fe640 1 --2- 172.21.15.38:0/3904789385 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6f0006ff00 0x7f6f0014f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:00.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:00.806+0000 7f6eff7fe640 1 -- 172.21.15.38:0/3904789385 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6f0015c750 msgr2=0x7f6f001f7420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:00.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:00.806+0000 7f6eff7fe640 1 --2- 172.21.15.38:0/3904789385 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6f0015c750 0x7f6f001f7420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:00.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:00.806+0000 7f6eff7fe640 1 -- 172.21.15.38:0/3904789385 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f001e4db0 con 0x7f6f000b2ca0 2024-09-21T03:39:00.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:00.806+0000 7f6eff7fe640 1 --2- 172.21.15.38:0/3904789385 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6f000b2ca0 0x7f6f001f6ee0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f6ef4009980 tx=0x7f6ef4003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:00.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:00.807+0000 7f6efcff9640 1 -- 172.21.15.38:0/3904789385 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ef4005c90 con 0x7f6f000b2ca0 2024-09-21T03:39:00.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:00.807+0000 7f6efcff9640 1 -- 172.21.15.38:0/3904789385 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6ef4005020 con 0x7f6f000b2ca0 2024-09-21T03:39:00.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:00.807+0000 7f6efcff9640 1 -- 172.21.15.38:0/3904789385 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ef4005410 con 0x7f6f000b2ca0 2024-09-21T03:39:00.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:00.807+0000 7f6f06372640 1 -- 172.21.15.38:0/3904789385 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f0009e250 con 0x7f6f000b2ca0 2024-09-21T03:39:00.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:00.807+0000 7f6f06372640 1 -- 172.21.15.38:0/3904789385 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f001e5070 con 0x7f6f000b2ca0 2024-09-21T03:39:00.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:00.808+0000 7f6f06372640 1 -- 172.21.15.38:0/3904789385 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f001f7c60 con 0x7f6f000b2ca0 2024-09-21T03:39:00.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:00.809+0000 7f6efcff9640 1 -- 172.21.15.38:0/3904789385 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f6ef40317c0 con 0x7f6f000b2ca0 2024-09-21T03:39:00.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:00.809+0000 7f6efcff9640 1 --2- 172.21.15.38:0/3904789385 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6eec07cfa0 0x7f6eec07f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:00.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:00.809+0000 7f6efcff9640 1 -- 172.21.15.38:0/3904789385 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5507+0+0 (secure 0 0 0) 0x7f6ef40bbc00 con 0x7f6f000b2ca0 2024-09-21T03:39:00.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:00.809+0000 7f6efffff640 1 --2- 172.21.15.38:0/3904789385 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6eec07cfa0 0x7f6eec07f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:00.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:00.810+0000 7f6efffff640 1 --2- 172.21.15.38:0/3904789385 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6eec07cfa0 0x7f6eec07f4b0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f6ef0001120 tx=0x7f6ef0009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:00.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:00.814+0000 7f6efcff9640 1 -- 172.21.15.38:0/3904789385 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f6ef40c2080 con 0x7f6f000b2ca0 2024-09-21T03:39:00.838 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~0s 2024-09-21T03:39:00.838 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~0s 2024-09-21T03:39:00.838 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~0s 2024-09-21T03:39:00.854 DEBUG:teuthology.orchestra.run.smithi038:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T03:39:00.857 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T03:39:01.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:01.015+0000 7f6f06372640 1 -- 172.21.15.38:0/3904789385 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f6f000b30b0 con 0x7f6f000b2ca0 2024-09-21T03:39:01.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:01.373+0000 7f6efcff9640 1 -- 172.21.15.38:0/3904789385 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v76) ==== 97+0+0 (secure 0 0 0) 0x7f6ef403f030 con 0x7f6f000b2ca0 2024-09-21T03:39:01.375 INFO:teuthology.orchestra.run.smithi038.stderr: removed all blocklist entries 2024-09-21T03:39:01.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:01.376+0000 7f6f06372640 1 -- 172.21.15.38:0/3904789385 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6eec07cfa0 msgr2=0x7f6eec07f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:01.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:01.376+0000 7f6f06372640 1 --2- 172.21.15.38:0/3904789385 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6eec07cfa0 0x7f6eec07f4b0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f6ef0001120 tx=0x7f6ef0009040 comp rx=0 tx=0).stop 2024-09-21T03:39:01.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:01.376+0000 7f6f06372640 1 -- 172.21.15.38:0/3904789385 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6f000b2ca0 msgr2=0x7f6f001f6ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:01.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:01.376+0000 7f6f06372640 1 --2- 172.21.15.38:0/3904789385 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6f000b2ca0 0x7f6f001f6ee0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f6ef4009980 tx=0x7f6ef4003c10 comp rx=0 tx=0).stop 2024-09-21T03:39:01.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:01.376+0000 7f6f06372640 1 -- 172.21.15.38:0/3904789385 shutdown_connections 2024-09-21T03:39:01.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:01.376+0000 7f6f06372640 1 --2- 172.21.15.38:0/3904789385 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6f0015c750 0x7f6f001f7420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:01.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:01.376+0000 7f6f06372640 1 --2- 172.21.15.38:0/3904789385 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6f000b2ca0 0x7f6f001f6ee0 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:01.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:01.376+0000 7f6f06372640 1 --2- 172.21.15.38:0/3904789385 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6eec07cfa0 0x7f6eec07f4b0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:01.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:01.376+0000 7f6f06372640 1 --2- 172.21.15.38:0/3904789385 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6f0006ff00 0x7f6f0014f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:01.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:01.376+0000 7f6f06372640 1 -- 172.21.15.38:0/3904789385 >> 172.21.15.38:0/3904789385 conn(0x7f6f000cdca0 msgr2=0x7f6f000b17d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:01.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:01.377+0000 7f6f06372640 1 -- 172.21.15.38:0/3904789385 shutdown_connections 2024-09-21T03:39:01.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:01.377+0000 7f6f06372640 1 -- 172.21.15.38:0/3904789385 wait complete. 2024-09-21T03:39:01.391 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-21T03:39:01.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:01.506+0000 7f187ef6e640 1 -- 172.21.15.38:0/3220430594 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1878162280 msgr2=0x7f1878162690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:01.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:01.506+0000 7f187ef6e640 1 --2- 172.21.15.38:0/3220430594 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1878162280 0x7f1878162690 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f186c009830 tx=0x7f186c02a510 comp rx=0 tx=0).stop 2024-09-21T03:39:01.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:01.506+0000 7f187ef6e640 1 -- 172.21.15.38:0/3220430594 shutdown_connections 2024-09-21T03:39:01.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:01.506+0000 7f187ef6e640 1 --2- 172.21.15.38:0/3220430594 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f18780b2e40 0x7f18780b3250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:01.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:01.506+0000 7f187ef6e640 1 --2- 172.21.15.38:0/3220430594 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1878162280 0x7f1878162690 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:01.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:01.506+0000 7f187ef6e640 1 -- 172.21.15.38:0/3220430594 >> 172.21.15.38:0/3220430594 conn(0x7f18780b6b20 msgr2=0x7f18780d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:01.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:01.506+0000 7f187ef6e640 1 -- 172.21.15.38:0/3220430594 shutdown_connections 2024-09-21T03:39:01.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:01.506+0000 7f187ef6e640 1 -- 172.21.15.38:0/3220430594 wait complete. 2024-09-21T03:39:01.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:01.508+0000 7f187ef6e640 1 Processor -- start 2024-09-21T03:39:01.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:01.508+0000 7f187ef6e640 1 -- start start 2024-09-21T03:39:01.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:01.509+0000 7f187ef6e640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f18780b2e40 0x7f18780ceb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:01.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:01.509+0000 7f187ef6e640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1878162280 0x7f18781f9190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:01.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:01.509+0000 7f187ef6e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f18780cf0b0 0x7f18781f96d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:01.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:01.509+0000 7f187ef6e640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f187815e4b0 con 0x7f18780cf0b0 2024-09-21T03:39:01.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:01.509+0000 7f187ef6e640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f18780b3a20 con 0x7f1878162280 2024-09-21T03:39:01.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:01.509+0000 7f187ef6e640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f187815e630 con 0x7f18780b2e40 2024-09-21T03:39:01.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:01.509+0000 7f187d50b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f18780cf0b0 0x7f18781f96d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:01.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:01.509+0000 7f187d50b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f18780cf0b0 0x7f18781f96d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43358/0 (socket says 172.21.15.38:43358) 2024-09-21T03:39:01.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:01.509+0000 7f187d50b640 1 -- 172.21.15.38:0/2163885555 learned_addr learned my addr 172.21.15.38:0/2163885555 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:39:01.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:01.510+0000 7f187cd0a640 1 --2- 172.21.15.38:0/2163885555 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f18780b2e40 0x7f18780ceb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:01.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:01.510+0000 7f1877fff640 1 --2- 172.21.15.38:0/2163885555 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1878162280 0x7f18781f9190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:01.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:01.510+0000 7f187d50b640 1 -- 172.21.15.38:0/2163885555 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f18780b2e40 msgr2=0x7f18780ceb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:01.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:01.510+0000 7f187d50b640 1 --2- 172.21.15.38:0/2163885555 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f18780b2e40 0x7f18780ceb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:01.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:01.510+0000 7f187d50b640 1 -- 172.21.15.38:0/2163885555 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1878162280 msgr2=0x7f18781f9190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:01.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:01.510+0000 7f187d50b640 1 --2- 172.21.15.38:0/2163885555 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1878162280 0x7f18781f9190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:01.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:01.510+0000 7f187d50b640 1 -- 172.21.15.38:0/2163885555 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f187814ee30 con 0x7f18780cf0b0 2024-09-21T03:39:01.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:01.510+0000 7f1877fff640 1 --2- 172.21.15.38:0/2163885555 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1878162280 0x7f18781f9190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:39:01.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:01.510+0000 7f187cd0a640 1 --2- 172.21.15.38:0/2163885555 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f18780b2e40 0x7f18780ceb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:39:01.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:01.510+0000 7f187d50b640 1 --2- 172.21.15.38:0/2163885555 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f18780cf0b0 0x7f18781f96d0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f187000e9c0 tx=0x7f187000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:01.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:01.510+0000 7f18757fa640 1 -- 172.21.15.38:0/2163885555 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1870004d20 con 0x7f18780cf0b0 2024-09-21T03:39:01.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:01.510+0000 7f18757fa640 1 -- 172.21.15.38:0/2163885555 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f187000cac0 con 0x7f18780cf0b0 2024-09-21T03:39:01.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:01.510+0000 7f187ef6e640 1 -- 172.21.15.38:0/2163885555 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f187814f110 con 0x7f18780cf0b0 2024-09-21T03:39:01.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:01.510+0000 7f18757fa640 1 -- 172.21.15.38:0/2163885555 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1870005020 con 0x7f18780cf0b0 2024-09-21T03:39:01.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:01.510+0000 7f187ef6e640 1 -- 172.21.15.38:0/2163885555 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18780ab8e0 con 0x7f18780cf0b0 2024-09-21T03:39:01.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:01.511+0000 7f187ef6e640 1 -- 172.21.15.38:0/2163885555 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18780ab3f0 con 0x7f18780cf0b0 2024-09-21T03:39:01.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:01.512+0000 7f18757fa640 1 -- 172.21.15.38:0/2163885555 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f18700051c0 con 0x7f18780cf0b0 2024-09-21T03:39:01.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:01.513+0000 7f18757fa640 1 --2- 172.21.15.38:0/2163885555 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1864078b90 0x7f186407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:01.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:01.513+0000 7f18757fa640 1 -- 172.21.15.38:0/2163885555 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5249+0+0 (secure 0 0 0) 0x7f187009b420 con 0x7f18780cf0b0 2024-09-21T03:39:01.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:01.513+0000 7f187cd0a640 1 --2- 172.21.15.38:0/2163885555 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1864078b90 0x7f186407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:01.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:01.513+0000 7f187cd0a640 1 --2- 172.21.15.38:0/2163885555 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1864078b90 0x7f186407b0a0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f18680013f0 tx=0x7f1868009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:01.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:01.517+0000 7f18757fa640 1 -- 172.21.15.38:0/2163885555 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f1870016030 con 0x7f18780cf0b0 2024-09-21T03:39:01.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:01.775+0000 7f187ef6e640 1 -- 172.21.15.38:0/2163885555 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f1878162f00 con 0x7f18780cf0b0 2024-09-21T03:39:01.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:01.776+0000 7f18757fa640 1 -- 172.21.15.38:0/2163885555 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v20) ==== 80+0+5304 (secure 0 0 0) 0x7f1870063a20 con 0x7f18780cf0b0 2024-09-21T03:39:01.778 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:39:01.778 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-09-21T03:39:01.778 INFO:teuthology.orchestra.run.smithi038.stdout: "auth_dump": [ 2024-09-21T03:39:01.778 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-21T03:39:01.778 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "mds.a", 2024-09-21T03:39:01.778 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQBPPu5mfzGIMRAAmq0TaC3J1VU3GxzhTgJ/Zw==", 2024-09-21T03:39:01.778 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-09-21T03:39:01.778 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow", 2024-09-21T03:39:01.778 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow *", 2024-09-21T03:39:01.778 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow *", 2024-09-21T03:39:01.779 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-09-21T03:39:01.779 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-21T03:39:01.779 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-21T03:39:01.779 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-21T03:39:01.779 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "mds.b", 2024-09-21T03:39:01.779 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQBQPu5m/fzuAhAApyigRlQ8OrV/sf0ap6Fn6A==", 2024-09-21T03:39:01.779 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-09-21T03:39:01.779 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow", 2024-09-21T03:39:01.779 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow *", 2024-09-21T03:39:01.779 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow *", 2024-09-21T03:39:01.779 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-09-21T03:39:01.779 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-21T03:39:01.779 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-21T03:39:01.780 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-21T03:39:01.780 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "mds.c", 2024-09-21T03:39:01.780 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQBPPu5mQipIORAArRn8k8SvvpG8x03F9I2/gQ==", 2024-09-21T03:39:01.780 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-09-21T03:39:01.780 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow", 2024-09-21T03:39:01.780 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow *", 2024-09-21T03:39:01.780 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow *", 2024-09-21T03:39:01.780 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-09-21T03:39:01.780 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-21T03:39:01.780 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-21T03:39:01.780 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-21T03:39:01.780 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "mds.d", 2024-09-21T03:39:01.780 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQBQPu5mzlyDChAAqbPKXS3EtnOqqeHhA1ehIw==", 2024-09-21T03:39:01.781 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-09-21T03:39:01.781 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow", 2024-09-21T03:39:01.781 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow *", 2024-09-21T03:39:01.781 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow *", 2024-09-21T03:39:01.781 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-09-21T03:39:01.781 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-21T03:39:01.781 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-21T03:39:01.781 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-21T03:39:01.781 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "osd.0", 2024-09-21T03:39:01.781 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQBXPu5mMZggAhAAtMPdJipheizZhpby9APj4A==", 2024-09-21T03:39:01.781 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-09-21T03:39:01.781 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow profile osd", 2024-09-21T03:39:01.781 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile osd", 2024-09-21T03:39:01.781 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-09-21T03:39:01.782 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-21T03:39:01.782 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-21T03:39:01.782 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-21T03:39:01.782 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "osd.1", 2024-09-21T03:39:01.782 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQBaPu5mm9TFIhAAWPJCvBYMgMbKr7Z7Diq58A==", 2024-09-21T03:39:01.782 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-09-21T03:39:01.782 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow profile osd", 2024-09-21T03:39:01.782 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile osd", 2024-09-21T03:39:01.782 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-09-21T03:39:01.782 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-21T03:39:01.782 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-21T03:39:01.782 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-21T03:39:01.783 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "osd.2", 2024-09-21T03:39:01.783 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQBePu5mqjljCBAAcdEpzJq6D8e1L+q1s0GjWQ==", 2024-09-21T03:39:01.783 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-09-21T03:39:01.783 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow profile osd", 2024-09-21T03:39:01.783 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile osd", 2024-09-21T03:39:01.783 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-09-21T03:39:01.783 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-21T03:39:01.783 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-21T03:39:01.783 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-21T03:39:01.783 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "osd.3", 2024-09-21T03:39:01.783 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQBhPu5mRjfdKBAAJ6+PlEiRp/xvxny6SjHV8Q==", 2024-09-21T03:39:01.783 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-09-21T03:39:01.783 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow profile osd", 2024-09-21T03:39:01.783 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile osd", 2024-09-21T03:39:01.784 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-09-21T03:39:01.784 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-21T03:39:01.784 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-21T03:39:01.784 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-21T03:39:01.784 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "osd.4", 2024-09-21T03:39:01.784 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQBlPu5m8bNyNxAA2rERruGojEDKmcXN2K6uog==", 2024-09-21T03:39:01.784 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-09-21T03:39:01.784 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow profile osd", 2024-09-21T03:39:01.784 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile osd", 2024-09-21T03:39:01.784 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-09-21T03:39:01.784 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-21T03:39:01.784 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-21T03:39:01.784 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-21T03:39:01.784 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "osd.5", 2024-09-21T03:39:01.785 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQBpPu5mBB2iDBAAvsvhAPMvNWII2Zr9Je45YQ==", 2024-09-21T03:39:01.785 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-09-21T03:39:01.785 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow profile osd", 2024-09-21T03:39:01.785 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile osd", 2024-09-21T03:39:01.785 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-09-21T03:39:01.785 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-21T03:39:01.785 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-21T03:39:01.785 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-21T03:39:01.785 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "osd.6", 2024-09-21T03:39:01.785 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQBsPu5mMQMaLRAAznzYdoNGHWOI2p2mNN/rQw==", 2024-09-21T03:39:01.785 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-09-21T03:39:01.785 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow profile osd", 2024-09-21T03:39:01.786 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile osd", 2024-09-21T03:39:01.786 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-09-21T03:39:01.786 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-21T03:39:01.786 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-21T03:39:01.786 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-21T03:39:01.786 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "osd.7", 2024-09-21T03:39:01.786 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQBwPu5mHu0WERAAir8bmtuNOOj/2D4hLaoYOA==", 2024-09-21T03:39:01.786 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-09-21T03:39:01.786 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow profile osd", 2024-09-21T03:39:01.786 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile osd", 2024-09-21T03:39:01.786 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-09-21T03:39:01.786 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-21T03:39:01.786 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-21T03:39:01.787 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-21T03:39:01.787 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "client.0", 2024-09-21T03:39:01.787 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQBQPu5m6ydBDxAAlWZ2XzQTPBcYPriPGRJVeA==", 2024-09-21T03:39:01.787 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-09-21T03:39:01.787 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow", 2024-09-21T03:39:01.787 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow r", 2024-09-21T03:39:01.787 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-21T03:39:01.787 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-21T03:39:01.787 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-21T03:39:01.787 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-21T03:39:01.787 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "client.1", 2024-09-21T03:39:01.787 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQBQPu5mlWP0FhAAkNwfaJ6rPr3r4ZRVbPxX8g==", 2024-09-21T03:39:01.787 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-09-21T03:39:01.787 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow", 2024-09-21T03:39:01.788 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow r", 2024-09-21T03:39:01.788 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow rw", 2024-09-21T03:39:01.788 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow rwx" 2024-09-21T03:39:01.788 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-21T03:39:01.788 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-21T03:39:01.788 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-21T03:39:01.788 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "client.2", 2024-09-21T03:39:01.788 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQBQPu5m48oaExAAOzYrR/SsykpzH2TkWHs39Q==", 2024-09-21T03:39:01.788 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-09-21T03:39:01.788 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow", 2024-09-21T03:39:01.788 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow r", 2024-09-21T03:39:01.788 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow rw", 2024-09-21T03:39:01.788 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow rwx" 2024-09-21T03:39:01.789 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-21T03:39:01.789 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-21T03:39:01.789 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-21T03:39:01.789 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "client.3", 2024-09-21T03:39:01.789 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQBQPu5mVQSzGhAAAo6vizLc7GOnenQRsgtn1A==", 2024-09-21T03:39:01.789 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-09-21T03:39:01.789 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow", 2024-09-21T03:39:01.789 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow r", 2024-09-21T03:39:01.789 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow rw", 2024-09-21T03:39:01.789 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow rwx" 2024-09-21T03:39:01.789 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-21T03:39:01.789 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-21T03:39:01.789 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-21T03:39:01.790 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "client.admin", 2024-09-21T03:39:01.790 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQBPPu5mPHphExAA7D+/V7qhoNyoc6h0GnNMlQ==", 2024-09-21T03:39:01.790 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-09-21T03:39:01.790 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow *", 2024-09-21T03:39:01.790 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow *", 2024-09-21T03:39:01.790 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow *", 2024-09-21T03:39:01.790 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-09-21T03:39:01.790 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-21T03:39:01.790 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-21T03:39:01.790 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-21T03:39:01.790 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "mgr.x", 2024-09-21T03:39:01.790 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQBPPu5m6K2oLRAAzldoBXAMSnWhxVFwPZkHOw==", 2024-09-21T03:39:01.790 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-09-21T03:39:01.790 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow *", 2024-09-21T03:39:01.791 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile mgr", 2024-09-21T03:39:01.791 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-09-21T03:39:01.791 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-21T03:39:01.791 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-21T03:39:01.791 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-21T03:39:01.791 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "mgr.y", 2024-09-21T03:39:01.791 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQBPPu5mWEz+KBAASx2KzDril3lD9cEawy56jA==", 2024-09-21T03:39:01.791 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-09-21T03:39:01.791 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow *", 2024-09-21T03:39:01.791 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile mgr", 2024-09-21T03:39:01.791 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-09-21T03:39:01.791 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-21T03:39:01.791 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-21T03:39:01.791 INFO:teuthology.orchestra.run.smithi038.stdout: ] 2024-09-21T03:39:01.792 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-09-21T03:39:01.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:01.778+0000 7f187ef6e640 1 -- 172.21.15.38:0/2163885555 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1864078b90 msgr2=0x7f186407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:01.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:01.778+0000 7f187ef6e640 1 --2- 172.21.15.38:0/2163885555 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1864078b90 0x7f186407b0a0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f18680013f0 tx=0x7f1868009040 comp rx=0 tx=0).stop 2024-09-21T03:39:01.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:01.778+0000 7f187ef6e640 1 -- 172.21.15.38:0/2163885555 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f18780cf0b0 msgr2=0x7f18781f96d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:01.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:01.778+0000 7f187ef6e640 1 --2- 172.21.15.38:0/2163885555 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f18780cf0b0 0x7f18781f96d0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f187000e9c0 tx=0x7f187000ee80 comp rx=0 tx=0).stop 2024-09-21T03:39:01.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:01.778+0000 7f187ef6e640 1 -- 172.21.15.38:0/2163885555 shutdown_connections 2024-09-21T03:39:01.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:01.778+0000 7f187ef6e640 1 --2- 172.21.15.38:0/2163885555 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f18780cf0b0 0x7f18781f96d0 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:01.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:01.778+0000 7f187ef6e640 1 --2- 172.21.15.38:0/2163885555 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1878162280 0x7f18781f9190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:01.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:01.778+0000 7f187ef6e640 1 --2- 172.21.15.38:0/2163885555 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1864078b90 0x7f186407b0a0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:01.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:01.778+0000 7f187ef6e640 1 --2- 172.21.15.38:0/2163885555 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f18780b2e40 0x7f18780ceb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:01.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:01.778+0000 7f187ef6e640 1 -- 172.21.15.38:0/2163885555 >> 172.21.15.38:0/2163885555 conn(0x7f18780b6b20 msgr2=0x7f187806d7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:01.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:01.778+0000 7f187ef6e640 1 -- 172.21.15.38:0/2163885555 shutdown_connections 2024-09-21T03:39:01.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:01.778+0000 7f187ef6e640 1 -- 172.21.15.38:0/2163885555 wait complete. 2024-09-21T03:39:01.794 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-21T03:39:01.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:01.909+0000 7ff71084d640 1 -- 172.21.15.38:0/856542767 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff70c162280 msgr2=0x7ff70c162690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:01.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:01.909+0000 7ff71084d640 1 --2- 172.21.15.38:0/856542767 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff70c162280 0x7ff70c162690 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7ff700009830 tx=0x7ff70002a510 comp rx=0 tx=0).stop 2024-09-21T03:39:01.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:01.909+0000 7ff71084d640 1 -- 172.21.15.38:0/856542767 shutdown_connections 2024-09-21T03:39:01.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:01.909+0000 7ff71084d640 1 --2- 172.21.15.38:0/856542767 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff70c162280 0x7ff70c162690 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:01.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:01.909+0000 7ff71084d640 1 --2- 172.21.15.38:0/856542767 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff70c162f00 0x7ff70c0d24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:01.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:01.909+0000 7ff71084d640 1 -- 172.21.15.38:0/856542767 >> 172.21.15.38:0/856542767 conn(0x7ff70c0b6b20 msgr2=0x7ff70c0d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:01.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:01.909+0000 7ff71084d640 1 -- 172.21.15.38:0/856542767 shutdown_connections 2024-09-21T03:39:01.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:01.909+0000 7ff71084d640 1 -- 172.21.15.38:0/856542767 wait complete. 2024-09-21T03:39:01.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:01.911+0000 7ff71084d640 1 Processor -- start 2024-09-21T03:39:01.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:01.911+0000 7ff71084d640 1 -- start start 2024-09-21T03:39:01.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:01.912+0000 7ff71084d640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff70c0b2e40 0x7ff70c0cb0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:01.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:01.912+0000 7ff71084d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff70c162280 0x7ff70c159820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:01.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:01.912+0000 7ff71084d640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff70c15a160 0x7ff70c0cdbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:01.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:01.913+0000 7ff71084d640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ff70c15e630 con 0x7ff70c162280 2024-09-21T03:39:01.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:01.913+0000 7ff71084d640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ff70c15e4b0 con 0x7ff70c0b2e40 2024-09-21T03:39:01.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:01.913+0000 7ff71084d640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7ff70c15e7b0 con 0x7ff70c15a160 2024-09-21T03:39:01.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:01.913+0000 7ff709d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff70c162280 0x7ff70c159820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:01.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:01.913+0000 7ff709d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff70c162280 0x7ff70c159820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43362/0 (socket says 172.21.15.38:43362) 2024-09-21T03:39:01.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:01.913+0000 7ff709d9b640 1 -- 172.21.15.38:0/3544893735 learned_addr learned my addr 172.21.15.38:0/3544893735 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:39:01.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:01.913+0000 7ff70a59c640 1 --2- 172.21.15.38:0/3544893735 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff70c0b2e40 0x7ff70c0cb0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:01.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:01.913+0000 7ff70ad9d640 1 --2- 172.21.15.38:0/3544893735 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff70c15a160 0x7ff70c0cdbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:01.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:01.913+0000 7ff709d9b640 1 -- 172.21.15.38:0/3544893735 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff70c15a160 msgr2=0x7ff70c0cdbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:01.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:01.913+0000 7ff709d9b640 1 --2- 172.21.15.38:0/3544893735 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff70c15a160 0x7ff70c0cdbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:01.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:01.913+0000 7ff709d9b640 1 -- 172.21.15.38:0/3544893735 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff70c0b2e40 msgr2=0x7ff70c0cb0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:01.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:01.913+0000 7ff709d9b640 1 --2- 172.21.15.38:0/3544893735 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff70c0b2e40 0x7ff70c0cb0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:01.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:01.913+0000 7ff709d9b640 1 -- 172.21.15.38:0/3544893735 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff70c0cd120 con 0x7ff70c162280 2024-09-21T03:39:01.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:01.913+0000 7ff709d9b640 1 --2- 172.21.15.38:0/3544893735 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff70c162280 0x7ff70c159820 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7ff700009950 tx=0x7ff700033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:01.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:01.913+0000 7ff6faffd640 1 -- 172.21.15.38:0/3544893735 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff700005c90 con 0x7ff70c162280 2024-09-21T03:39:01.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:01.913+0000 7ff6faffd640 1 -- 172.21.15.38:0/3544893735 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff700031e50 con 0x7ff70c162280 2024-09-21T03:39:01.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:01.913+0000 7ff6faffd640 1 -- 172.21.15.38:0/3544893735 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff700005260 con 0x7ff70c162280 2024-09-21T03:39:01.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:01.913+0000 7ff71084d640 1 -- 172.21.15.38:0/3544893735 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff70c0ab760 con 0x7ff70c162280 2024-09-21T03:39:01.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:01.914+0000 7ff71084d640 1 -- 172.21.15.38:0/3544893735 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff70c0cd3c0 con 0x7ff70c162280 2024-09-21T03:39:01.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:01.915+0000 7ff71084d640 1 -- 172.21.15.38:0/3544893735 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff70c0b9860 con 0x7ff70c162280 2024-09-21T03:39:01.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:01.915+0000 7ff6faffd640 1 -- 172.21.15.38:0/3544893735 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ff700005400 con 0x7ff70c162280 2024-09-21T03:39:01.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:01.916+0000 7ff6faffd640 1 --2- 172.21.15.38:0/3544893735 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff6ec078b90 0x7ff6ec07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:01.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:01.916+0000 7ff6faffd640 1 -- 172.21.15.38:0/3544893735 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5249+0+0 (secure 0 0 0) 0x7ff7000bbb90 con 0x7ff70c162280 2024-09-21T03:39:01.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:01.916+0000 7ff70a59c640 1 --2- 172.21.15.38:0/3544893735 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff6ec078b90 0x7ff6ec07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:01.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:01.917+0000 7ff70a59c640 1 --2- 172.21.15.38:0/3544893735 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff6ec078b90 0x7ff6ec07b0a0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7ff6fc009530 tx=0x7ff6fc009260 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:01.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:01.921+0000 7ff6faffd640 1 -- 172.21.15.38:0/3544893735 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7ff70c0b9860 con 0x7ff70c162280 2024-09-21T03:39:02.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:02.109+0000 7ff71084d640 1 -- 172.21.15.38:0/3544893735 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7ff70c1f9970 con 0x7ff70c162280 2024-09-21T03:39:02.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:02.109+0000 7ff6faffd640 1 -- 172.21.15.38:0/3544893735 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v78) ==== 84+0+65 (secure 0 0 0) 0x7ff70003f030 con 0x7ff70c162280 2024-09-21T03:39:02.111 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:39:02.111 INFO:teuthology.orchestra.run.smithi038.stdout:[ 2024-09-21T03:39:02.111 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-21T03:39:02.111 INFO:teuthology.orchestra.run.smithi038.stdout: "poolnum": 1, 2024-09-21T03:39:02.112 INFO:teuthology.orchestra.run.smithi038.stdout: "poolname": ".mgr" 2024-09-21T03:39:02.112 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-21T03:39:02.112 INFO:teuthology.orchestra.run.smithi038.stdout:] 2024-09-21T03:39:02.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:02.111+0000 7ff71084d640 1 -- 172.21.15.38:0/3544893735 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff6ec078b90 msgr2=0x7ff6ec07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:02.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:02.111+0000 7ff71084d640 1 --2- 172.21.15.38:0/3544893735 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff6ec078b90 0x7ff6ec07b0a0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7ff6fc009530 tx=0x7ff6fc009260 comp rx=0 tx=0).stop 2024-09-21T03:39:02.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:02.111+0000 7ff71084d640 1 -- 172.21.15.38:0/3544893735 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff70c162280 msgr2=0x7ff70c159820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:02.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:02.111+0000 7ff71084d640 1 --2- 172.21.15.38:0/3544893735 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff70c162280 0x7ff70c159820 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7ff700009950 tx=0x7ff700033c20 comp rx=0 tx=0).stop 2024-09-21T03:39:02.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:02.111+0000 7ff71084d640 1 -- 172.21.15.38:0/3544893735 shutdown_connections 2024-09-21T03:39:02.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:02.111+0000 7ff71084d640 1 --2- 172.21.15.38:0/3544893735 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff6ec078b90 0x7ff6ec07b0a0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:02.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:02.111+0000 7ff71084d640 1 --2- 172.21.15.38:0/3544893735 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff70c15a160 0x7ff70c0cdbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:02.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:02.111+0000 7ff71084d640 1 --2- 172.21.15.38:0/3544893735 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff70c162280 0x7ff70c159820 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:02.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:02.111+0000 7ff71084d640 1 --2- 172.21.15.38:0/3544893735 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff70c0b2e40 0x7ff70c0cb0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:02.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:02.111+0000 7ff71084d640 1 -- 172.21.15.38:0/3544893735 >> 172.21.15.38:0/3544893735 conn(0x7ff70c0b6b20 msgr2=0x7ff70c15f640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:02.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:02.111+0000 7ff71084d640 1 -- 172.21.15.38:0/3544893735 shutdown_connections 2024-09-21T03:39:02.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:02.111+0000 7ff71084d640 1 -- 172.21.15.38:0/3544893735 wait complete. 2024-09-21T03:39:02.124 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-21T03:39:02.125 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-21T03:39:02.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:02.244+0000 7f0cb3190640 1 -- 172.21.15.38:0/3889157341 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0cac160310 msgr2=0x7f0cac0b35b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:02.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:02.244+0000 7f0cb3190640 1 --2- 172.21.15.38:0/3889157341 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0cac160310 0x7f0cac0b35b0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f0c9c005fd0 tx=0x7f0c9c02a4a0 comp rx=0 tx=0).stop 2024-09-21T03:39:02.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:02.244+0000 7f0cb3190640 1 -- 172.21.15.38:0/3889157341 shutdown_connections 2024-09-21T03:39:02.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:02.244+0000 7f0cb3190640 1 --2- 172.21.15.38:0/3889157341 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0cac0b1d30 0x7f0cac0ccc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:02.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:02.244+0000 7f0cb3190640 1 --2- 172.21.15.38:0/3889157341 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0cac160310 0x7f0cac0b35b0 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:02.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:02.244+0000 7f0cb3190640 1 -- 172.21.15.38:0/3889157341 >> 172.21.15.38:0/3889157341 conn(0x7f0cac0b3ec0 msgr2=0x7f0cac0b42d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:02.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:02.244+0000 7f0cb3190640 1 -- 172.21.15.38:0/3889157341 shutdown_connections 2024-09-21T03:39:02.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:02.244+0000 7f0cb3190640 1 -- 172.21.15.38:0/3889157341 wait complete. 2024-09-21T03:39:02.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:02.246+0000 7f0cb3190640 1 Processor -- start 2024-09-21T03:39:02.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:02.246+0000 7f0cb3190640 1 -- start start 2024-09-21T03:39:02.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:02.248+0000 7f0cb3190640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0cac0b1d30 0x7f0cac0ca5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:02.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:02.248+0000 7f0cb3190640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0cac0cd170 0x7f0cac14aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:02.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:02.248+0000 7f0cb3190640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0cac160310 0x7f0cac1f69d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:02.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:02.248+0000 7f0cb3190640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f0cac15db00 con 0x7f0cac0b1d30 2024-09-21T03:39:02.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:02.248+0000 7f0cb3190640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f0cac15d980 con 0x7f0cac0cd170 2024-09-21T03:39:02.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:02.248+0000 7f0cb3190640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f0cac0d0b20 con 0x7f0cac160310 2024-09-21T03:39:02.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:02.248+0000 7f0cb0f2c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0cac0b1d30 0x7f0cac0ca5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:02.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:02.248+0000 7f0cb0f2c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0cac0b1d30 0x7f0cac0ca5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43366/0 (socket says 172.21.15.38:43366) 2024-09-21T03:39:02.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:02.248+0000 7f0cb0f2c640 1 -- 172.21.15.38:0/3080177043 learned_addr learned my addr 172.21.15.38:0/3080177043 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:39:02.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:02.248+0000 7f0cabfff640 1 --2- 172.21.15.38:0/3080177043 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0cac0cd170 0x7f0cac14aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:02.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:02.248+0000 7f0cb172d640 1 --2- 172.21.15.38:0/3080177043 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0cac160310 0x7f0cac1f69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:02.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:02.248+0000 7f0cb0f2c640 1 -- 172.21.15.38:0/3080177043 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0cac160310 msgr2=0x7f0cac1f69d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:02.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:02.248+0000 7f0cb0f2c640 1 --2- 172.21.15.38:0/3080177043 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0cac160310 0x7f0cac1f69d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:02.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:02.248+0000 7f0cb0f2c640 1 -- 172.21.15.38:0/3080177043 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0cac0cd170 msgr2=0x7f0cac14aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:02.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:02.248+0000 7f0cb0f2c640 1 --2- 172.21.15.38:0/3080177043 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0cac0cd170 0x7f0cac14aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:02.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:02.248+0000 7f0cb0f2c640 1 -- 172.21.15.38:0/3080177043 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cac0cbfc0 con 0x7f0cac0b1d30 2024-09-21T03:39:02.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:02.248+0000 7f0cb0f2c640 1 --2- 172.21.15.38:0/3080177043 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0cac0b1d30 0x7f0cac0ca5b0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f0c9c034040 tx=0x7f0c9c003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:02.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:02.248+0000 7f0ca97fa640 1 -- 172.21.15.38:0/3080177043 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c9c03e030 con 0x7f0cac0b1d30 2024-09-21T03:39:02.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:02.249+0000 7f0ca97fa640 1 -- 172.21.15.38:0/3080177043 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0c9c0047a0 con 0x7f0cac0b1d30 2024-09-21T03:39:02.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:02.249+0000 7f0cb3190640 1 -- 172.21.15.38:0/3080177043 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cac0caf20 con 0x7f0cac0b1d30 2024-09-21T03:39:02.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:02.249+0000 7f0ca97fa640 1 -- 172.21.15.38:0/3080177043 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c9c035040 con 0x7f0cac0b1d30 2024-09-21T03:39:02.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:02.249+0000 7f0cb3190640 1 -- 172.21.15.38:0/3080177043 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cac0cb4e0 con 0x7f0cac0b1d30 2024-09-21T03:39:02.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:02.250+0000 7f0cb3190640 1 -- 172.21.15.38:0/3080177043 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0cac0d1080 con 0x7f0cac0b1d30 2024-09-21T03:39:02.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:02.250+0000 7f0ca97fa640 1 -- 172.21.15.38:0/3080177043 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f0c9c041030 con 0x7f0cac0b1d30 2024-09-21T03:39:02.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:02.251+0000 7f0ca97fa640 1 --2- 172.21.15.38:0/3080177043 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0c8c078b90 0x7f0c8c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:02.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:02.251+0000 7f0ca97fa640 1 -- 172.21.15.38:0/3080177043 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5249+0+0 (secure 0 0 0) 0x7f0c9c0bb400 con 0x7f0cac0b1d30 2024-09-21T03:39:02.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:02.251+0000 7f0cabfff640 1 --2- 172.21.15.38:0/3080177043 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0c8c078b90 0x7f0c8c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:02.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:02.252+0000 7f0cabfff640 1 --2- 172.21.15.38:0/3080177043 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0c8c078b90 0x7f0c8c07b0a0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f0c94005c40 tx=0x7f0c9400a550 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:02.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:02.255+0000 7f0ca97fa640 1 -- 172.21.15.38:0/3080177043 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f0cac0cd580 con 0x7f0cac0b1d30 2024-09-21T03:39:02.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:02.441+0000 7f0cb3190640 1 -- 172.21.15.38:0/3080177043 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f0cac15ec40 con 0x7f0cac0b1d30 2024-09-21T03:39:03.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:03.382+0000 7f0ca97fa640 1 -- 172.21.15.38:0/3080177043 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v79) ==== 138+0+0 (secure 0 0 0) 0x7f0c9c0833b0 con 0x7f0cac0b1d30 2024-09-21T03:39:03.384 INFO:teuthology.orchestra.run.smithi038.stderr:pool 'cephfs_metadata' created 2024-09-21T03:39:03.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:03.385+0000 7f0cb3190640 1 -- 172.21.15.38:0/3080177043 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0c8c078b90 msgr2=0x7f0c8c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:03.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:03.385+0000 7f0cb3190640 1 --2- 172.21.15.38:0/3080177043 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0c8c078b90 0x7f0c8c07b0a0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f0c94005c40 tx=0x7f0c9400a550 comp rx=0 tx=0).stop 2024-09-21T03:39:03.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:03.385+0000 7f0cb3190640 1 -- 172.21.15.38:0/3080177043 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0cac0b1d30 msgr2=0x7f0cac0ca5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:03.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:03.385+0000 7f0cb3190640 1 --2- 172.21.15.38:0/3080177043 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0cac0b1d30 0x7f0cac0ca5b0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f0c9c034040 tx=0x7f0c9c003970 comp rx=0 tx=0).stop 2024-09-21T03:39:03.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:03.386+0000 7f0cb3190640 1 -- 172.21.15.38:0/3080177043 shutdown_connections 2024-09-21T03:39:03.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:03.386+0000 7f0cb3190640 1 --2- 172.21.15.38:0/3080177043 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0c8c078b90 0x7f0c8c07b0a0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:03.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:03.386+0000 7f0cb3190640 1 --2- 172.21.15.38:0/3080177043 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0cac160310 0x7f0cac1f69d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:03.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:03.386+0000 7f0cb3190640 1 --2- 172.21.15.38:0/3080177043 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0cac0cd170 0x7f0cac14aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:03.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:03.386+0000 7f0cb3190640 1 --2- 172.21.15.38:0/3080177043 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0cac0b1d30 0x7f0cac0ca5b0 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:03.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:03.386+0000 7f0cb3190640 1 -- 172.21.15.38:0/3080177043 >> 172.21.15.38:0/3080177043 conn(0x7f0cac0b3ec0 msgr2=0x7f0cac15e210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:03.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:03.386+0000 7f0cb3190640 1 -- 172.21.15.38:0/3080177043 shutdown_connections 2024-09-21T03:39:03.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:03.386+0000 7f0cb3190640 1 -- 172.21.15.38:0/3080177043 wait complete. 2024-09-21T03:39:03.406 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-21T03:39:03.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:03.534+0000 7fd462294640 1 -- 172.21.15.38:0/4014900549 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd45c162f00 msgr2=0x7fd45c0d24d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:03.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:03.534+0000 7fd462294640 1 --2- 172.21.15.38:0/4014900549 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd45c162f00 0x7fd45c0d24d0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fd44c009830 tx=0x7fd44c02a510 comp rx=0 tx=0).stop 2024-09-21T03:39:03.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:03.534+0000 7fd462294640 1 -- 172.21.15.38:0/4014900549 shutdown_connections 2024-09-21T03:39:03.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:03.534+0000 7fd462294640 1 --2- 172.21.15.38:0/4014900549 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd45c0b2e40 0x7fd45c0b3250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:03.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:03.534+0000 7fd462294640 1 --2- 172.21.15.38:0/4014900549 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd45c162f00 0x7fd45c0d24d0 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:03.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:03.534+0000 7fd462294640 1 -- 172.21.15.38:0/4014900549 >> 172.21.15.38:0/4014900549 conn(0x7fd45c0b6b20 msgr2=0x7fd45c0d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:03.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:03.534+0000 7fd462294640 1 -- 172.21.15.38:0/4014900549 shutdown_connections 2024-09-21T03:39:03.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:03.534+0000 7fd462294640 1 -- 172.21.15.38:0/4014900549 wait complete. 2024-09-21T03:39:03.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:03.536+0000 7fd462294640 1 Processor -- start 2024-09-21T03:39:03.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:03.536+0000 7fd462294640 1 -- start start 2024-09-21T03:39:03.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:03.537+0000 7fd462294640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd45c0b2e40 0x7fd45c0cbc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:03.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:03.537+0000 7fd462294640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd45c162280 0x7fd45c0aadc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:03.541 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:03.537+0000 7fd462294640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd45c0cefd0 0x7fd45c0ca900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:03.541 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:03.537+0000 7fd462294640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fd45c15e4b0 con 0x7fd45c162280 2024-09-21T03:39:03.541 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:03.537+0000 7fd462294640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd45c0b3a20 con 0x7fd45c0b2e40 2024-09-21T03:39:03.541 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:03.537+0000 7fd462294640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fd45c15e630 con 0x7fd45c0cefd0 2024-09-21T03:39:03.541 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:03.537+0000 7fd45b7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd45c162280 0x7fd45c0aadc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:03.541 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:03.537+0000 7fd45b7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd45c162280 0x7fd45c0aadc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:52524/0 (socket says 172.21.15.38:52524) 2024-09-21T03:39:03.541 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:03.537+0000 7fd45b7fe640 1 -- 172.21.15.38:0/1377277525 learned_addr learned my addr 172.21.15.38:0/1377277525 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:39:03.541 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:03.538+0000 7fd45bfff640 1 --2- 172.21.15.38:0/1377277525 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd45c0b2e40 0x7fd45c0cbc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:03.541 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:03.538+0000 7fd460831640 1 --2- 172.21.15.38:0/1377277525 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd45c0cefd0 0x7fd45c0ca900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:03.541 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:03.538+0000 7fd45b7fe640 1 -- 172.21.15.38:0/1377277525 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd45c0cefd0 msgr2=0x7fd45c0ca900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:03.541 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:03.538+0000 7fd45b7fe640 1 --2- 172.21.15.38:0/1377277525 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd45c0cefd0 0x7fd45c0ca900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:03.541 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:03.538+0000 7fd45b7fe640 1 -- 172.21.15.38:0/1377277525 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd45c0b2e40 msgr2=0x7fd45c0cbc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:03.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:03.538+0000 7fd45b7fe640 1 --2- 172.21.15.38:0/1377277525 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd45c0b2e40 0x7fd45c0cbc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:03.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:03.538+0000 7fd45b7fe640 1 -- 172.21.15.38:0/1377277525 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd45c0cae40 con 0x7fd45c162280 2024-09-21T03:39:03.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:03.538+0000 7fd45bfff640 1 --2- 172.21.15.38:0/1377277525 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd45c0b2e40 0x7fd45c0cbc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:39:03.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:03.538+0000 7fd460831640 1 --2- 172.21.15.38:0/1377277525 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd45c0cefd0 0x7fd45c0ca900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:39:03.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:03.538+0000 7fd45b7fe640 1 --2- 172.21.15.38:0/1377277525 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd45c162280 0x7fd45c0aadc0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fd45000c3f0 tx=0x7fd45000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:03.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:03.538+0000 7fd458ff9640 1 -- 172.21.15.38:0/1377277525 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd450017070 con 0x7fd45c162280 2024-09-21T03:39:03.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:03.538+0000 7fd458ff9640 1 -- 172.21.15.38:0/1377277525 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd4500035f0 con 0x7fd45c162280 2024-09-21T03:39:03.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:03.538+0000 7fd458ff9640 1 -- 172.21.15.38:0/1377277525 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd450003a40 con 0x7fd45c162280 2024-09-21T03:39:03.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:03.538+0000 7fd462294640 1 -- 172.21.15.38:0/1377277525 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd45c14d6d0 con 0x7fd45c162280 2024-09-21T03:39:03.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:03.538+0000 7fd462294640 1 -- 172.21.15.38:0/1377277525 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd45c14d3f0 con 0x7fd45c162280 2024-09-21T03:39:03.543 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:03.539+0000 7fd462294640 1 -- 172.21.15.38:0/1377277525 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd45c15ceb0 con 0x7fd45c162280 2024-09-21T03:39:03.543 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:03.540+0000 7fd458ff9640 1 -- 172.21.15.38:0/1377277525 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fd45001a070 con 0x7fd45c162280 2024-09-21T03:39:03.543 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:03.541+0000 7fd458ff9640 1 --2- 172.21.15.38:0/1377277525 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd448078bc0 0x7fd44807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:03.543 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:03.541+0000 7fd458ff9640 1 -- 172.21.15.38:0/1377277525 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 5619+0+0 (secure 0 0 0) 0x7fd450099fb0 con 0x7fd45c162280 2024-09-21T03:39:03.543 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:03.541+0000 7fd45bfff640 1 --2- 172.21.15.38:0/1377277525 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd448078bc0 0x7fd44807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:03.544 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:03.542+0000 7fd45bfff640 1 --2- 172.21.15.38:0/1377277525 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd448078bc0 0x7fd44807b0d0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fd44c034000 tx=0x7fd44c004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:03.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:03.546+0000 7fd458ff9640 1 -- 172.21.15.38:0/1377277525 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fd45c15ceb0 con 0x7fd45c162280 2024-09-21T03:39:03.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:03.735+0000 7fd462294640 1 -- 172.21.15.38:0/1377277525 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fd45c162f00 con 0x7fd45c162280 2024-09-21T03:39:04.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:04.385+0000 7fd458ff9640 1 -- 172.21.15.38:0/1377277525 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v80) ==== 170+0+0 (secure 0 0 0) 0x7fd4500624d0 con 0x7fd45c162280 2024-09-21T03:39:04.387 INFO:teuthology.orchestra.run.smithi038.stderr:pool 'cephfs_data' created 2024-09-21T03:39:04.391 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:04.389+0000 7fd462294640 1 -- 172.21.15.38:0/1377277525 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd448078bc0 msgr2=0x7fd44807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:04.391 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:04.389+0000 7fd462294640 1 --2- 172.21.15.38:0/1377277525 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd448078bc0 0x7fd44807b0d0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fd44c034000 tx=0x7fd44c004730 comp rx=0 tx=0).stop 2024-09-21T03:39:04.391 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:04.389+0000 7fd462294640 1 -- 172.21.15.38:0/1377277525 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd45c162280 msgr2=0x7fd45c0aadc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:04.391 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:04.389+0000 7fd462294640 1 --2- 172.21.15.38:0/1377277525 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd45c162280 0x7fd45c0aadc0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fd45000c3f0 tx=0x7fd45000c8b0 comp rx=0 tx=0).stop 2024-09-21T03:39:04.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:04.389+0000 7fd462294640 1 -- 172.21.15.38:0/1377277525 shutdown_connections 2024-09-21T03:39:04.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:04.389+0000 7fd462294640 1 --2- 172.21.15.38:0/1377277525 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd448078bc0 0x7fd44807b0d0 secure :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fd44c034000 tx=0x7fd44c004730 comp rx=0 tx=0).stop 2024-09-21T03:39:04.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:04.389+0000 7fd462294640 1 --2- 172.21.15.38:0/1377277525 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd45c0cefd0 0x7fd45c0ca900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:04.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:04.389+0000 7fd462294640 1 --2- 172.21.15.38:0/1377277525 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd45c162280 0x7fd45c0aadc0 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:04.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:04.389+0000 7fd462294640 1 --2- 172.21.15.38:0/1377277525 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd45c0b2e40 0x7fd45c0cbc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:04.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:04.389+0000 7fd462294640 1 -- 172.21.15.38:0/1377277525 >> 172.21.15.38:0/1377277525 conn(0x7fd45c0b6b20 msgr2=0x7fd45c06da90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:04.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:04.390+0000 7fd462294640 1 -- 172.21.15.38:0/1377277525 shutdown_connections 2024-09-21T03:39:04.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:04.391+0000 7fd462294640 1 -- 172.21.15.38:0/1377277525 wait complete. 2024-09-21T03:39:04.418 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-21T03:39:04.544 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:04.542+0000 7f7a90e0a640 1 -- 172.21.15.38:0/983876380 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7a8c0ceb90 msgr2=0x7f7a8c0cefa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:04.544 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:04.542+0000 7f7a90e0a640 1 --2- 172.21.15.38:0/983876380 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7a8c0ceb90 0x7f7a8c0cefa0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f7a7c005fd0 tx=0x7f7a7c02aa10 comp rx=0 tx=0).stop 2024-09-21T03:39:04.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:04.542+0000 7f7a90e0a640 1 -- 172.21.15.38:0/983876380 shutdown_connections 2024-09-21T03:39:04.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:04.542+0000 7f7a90e0a640 1 --2- 172.21.15.38:0/983876380 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7a8c0ceb90 0x7f7a8c0cefa0 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:04.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:04.542+0000 7f7a90e0a640 1 --2- 172.21.15.38:0/983876380 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7a8c15ee80 0x7f7a8c0d0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:04.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:04.542+0000 7f7a90e0a640 1 -- 172.21.15.38:0/983876380 >> 172.21.15.38:0/983876380 conn(0x7f7a8c09d830 msgr2=0x7f7a8c0b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:04.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:04.542+0000 7f7a90e0a640 1 -- 172.21.15.38:0/983876380 shutdown_connections 2024-09-21T03:39:04.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:04.542+0000 7f7a90e0a640 1 -- 172.21.15.38:0/983876380 wait complete. 2024-09-21T03:39:04.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:04.544+0000 7f7a90e0a640 1 Processor -- start 2024-09-21T03:39:04.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:04.544+0000 7f7a90e0a640 1 -- start start 2024-09-21T03:39:04.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:04.545+0000 7f7a90e0a640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7a8c0b9ad0 0x7f7a8c06f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:04.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:04.545+0000 7f7a90e0a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7a8c0ceb90 0x7f7a8c1f5540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:04.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:04.545+0000 7f7a90e0a640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7a8c15ee80 0x7f7a8c158310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:04.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:04.545+0000 7f7a90e0a640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f7a8c0b6700 con 0x7f7a8c0ceb90 2024-09-21T03:39:04.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:04.545+0000 7f7a90e0a640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f7a8c0b6580 con 0x7f7a8c15ee80 2024-09-21T03:39:04.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:04.545+0000 7f7a90e0a640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f7a8c0b6880 con 0x7f7a8c0b9ad0 2024-09-21T03:39:04.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:04.546+0000 7f7a89d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7a8c0ceb90 0x7f7a8c1f5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:04.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:04.546+0000 7f7a89d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7a8c0ceb90 0x7f7a8c1f5540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:52538/0 (socket says 172.21.15.38:52538) 2024-09-21T03:39:04.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:04.546+0000 7f7a89d9b640 1 -- 172.21.15.38:0/3288889329 learned_addr learned my addr 172.21.15.38:0/3288889329 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:39:04.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:04.546+0000 7f7a8ad9d640 1 --2- 172.21.15.38:0/3288889329 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7a8c15ee80 0x7f7a8c158310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:04.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:04.546+0000 7f7a8a59c640 1 --2- 172.21.15.38:0/3288889329 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7a8c0b9ad0 0x7f7a8c06f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:04.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:04.546+0000 7f7a89d9b640 1 -- 172.21.15.38:0/3288889329 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7a8c0b9ad0 msgr2=0x7f7a8c06f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:04.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:04.546+0000 7f7a89d9b640 1 --2- 172.21.15.38:0/3288889329 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7a8c0b9ad0 0x7f7a8c06f890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:04.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:04.546+0000 7f7a89d9b640 1 -- 172.21.15.38:0/3288889329 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7a8c15ee80 msgr2=0x7f7a8c158310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:04.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:04.546+0000 7f7a89d9b640 1 --2- 172.21.15.38:0/3288889329 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7a8c15ee80 0x7f7a8c158310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:04.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:04.546+0000 7f7a89d9b640 1 -- 172.21.15.38:0/3288889329 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a8c1f6700 con 0x7f7a8c0ceb90 2024-09-21T03:39:04.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:04.546+0000 7f7a89d9b640 1 --2- 172.21.15.38:0/3288889329 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7a8c0ceb90 0x7f7a8c1f5540 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f7a7400c440 tx=0x7f7a7400c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:04.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:04.546+0000 7f7a82ffd640 1 -- 172.21.15.38:0/3288889329 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a74017070 con 0x7f7a8c0ceb90 2024-09-21T03:39:04.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:04.546+0000 7f7a82ffd640 1 -- 172.21.15.38:0/3288889329 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7a740053c0 con 0x7f7a8c0ceb90 2024-09-21T03:39:04.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:04.546+0000 7f7a90e0a640 1 -- 172.21.15.38:0/3288889329 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a8c0a0110 con 0x7f7a8c0ceb90 2024-09-21T03:39:04.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:04.546+0000 7f7a82ffd640 1 -- 172.21.15.38:0/3288889329 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a740038a0 con 0x7f7a8c0ceb90 2024-09-21T03:39:04.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:04.546+0000 7f7a90e0a640 1 -- 172.21.15.38:0/3288889329 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a8c1f60e0 con 0x7f7a8c0ceb90 2024-09-21T03:39:04.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:04.547+0000 7f7a90e0a640 1 -- 172.21.15.38:0/3288889329 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a8c1f5b50 con 0x7f7a8c0ceb90 2024-09-21T03:39:04.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:04.548+0000 7f7a82ffd640 1 -- 172.21.15.38:0/3288889329 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f7a7401a070 con 0x7f7a8c0ceb90 2024-09-21T03:39:04.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:04.549+0000 7f7a82ffd640 1 --2- 172.21.15.38:0/3288889329 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7a6c078b90 0x7f7a6c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:04.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:04.549+0000 7f7a82ffd640 1 -- 172.21.15.38:0/3288889329 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6001+0+0 (secure 0 0 0) 0x7f7a7409a180 con 0x7f7a8c0ceb90 2024-09-21T03:39:04.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:04.549+0000 7f7a8a59c640 1 --2- 172.21.15.38:0/3288889329 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7a6c078b90 0x7f7a6c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:04.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:04.550+0000 7f7a8a59c640 1 --2- 172.21.15.38:0/3288889329 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7a6c078b90 0x7f7a6c07b0a0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f7a84009b30 tx=0x7f7a84005f90 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:04.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:04.553+0000 7f7a82ffd640 1 -- 172.21.15.38:0/3288889329 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f7a8c1592d0 con 0x7f7a8c0ceb90 2024-09-21T03:39:04.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:04.778+0000 7f7a90e0a640 1 -- 172.21.15.38:0/3288889329 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f7a8c0b6da0 con 0x7f7a8c0ceb90 2024-09-21T03:39:05.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:05.407+0000 7f7a82ffd640 1 -- 172.21.15.38:0/3288889329 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 11 and data pool 12 v64) ==== 174+0+0 (secure 0 0 0) 0x7f7a74061ec0 con 0x7f7a8c0ceb90 2024-09-21T03:39:05.409 INFO:teuthology.orchestra.run.smithi038.stderr:new fs with metadata pool 11 and data pool 12 2024-09-21T03:39:05.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:05.418+0000 7f7a90e0a640 1 -- 172.21.15.38:0/3288889329 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7a6c078b90 msgr2=0x7f7a6c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:05.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:05.418+0000 7f7a90e0a640 1 --2- 172.21.15.38:0/3288889329 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7a6c078b90 0x7f7a6c07b0a0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f7a84009b30 tx=0x7f7a84005f90 comp rx=0 tx=0).stop 2024-09-21T03:39:05.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:05.418+0000 7f7a90e0a640 1 -- 172.21.15.38:0/3288889329 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7a8c0ceb90 msgr2=0x7f7a8c1f5540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:05.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:05.418+0000 7f7a90e0a640 1 --2- 172.21.15.38:0/3288889329 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7a8c0ceb90 0x7f7a8c1f5540 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f7a7400c440 tx=0x7f7a7400c900 comp rx=0 tx=0).stop 2024-09-21T03:39:05.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:05.420+0000 7f7a90e0a640 1 -- 172.21.15.38:0/3288889329 shutdown_connections 2024-09-21T03:39:05.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:05.420+0000 7f7a90e0a640 1 --2- 172.21.15.38:0/3288889329 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7a8c15ee80 0x7f7a8c158310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:05.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:05.420+0000 7f7a90e0a640 1 --2- 172.21.15.38:0/3288889329 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7a8c0ceb90 0x7f7a8c1f5540 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:05.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:05.420+0000 7f7a90e0a640 1 --2- 172.21.15.38:0/3288889329 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7a6c078b90 0x7f7a6c07b0a0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:05.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:05.420+0000 7f7a90e0a640 1 --2- 172.21.15.38:0/3288889329 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7a8c0b9ad0 0x7f7a8c06f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:05.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:05.420+0000 7f7a90e0a640 1 -- 172.21.15.38:0/3288889329 >> 172.21.15.38:0/3288889329 conn(0x7f7a8c09d830 msgr2=0x7f7a8c15ff90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:05.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:05.422+0000 7f7a90e0a640 1 -- 172.21.15.38:0/3288889329 shutdown_connections 2024-09-21T03:39:05.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:05.423+0000 7f7a90e0a640 1 -- 172.21.15.38:0/3288889329 wait complete. 2024-09-21T03:39:05.448 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T03:39:05.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:05.566+0000 7feb02bad640 1 -- 172.21.15.38:0/3696018729 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feae8004d40 msgr2=0x7feae8005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:05.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:05.566+0000 7feb02bad640 1 --2- 172.21.15.38:0/3696018729 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feae8004d40 0x7feae8005150 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7feaf4009ed0 tx=0x7feae8005830 comp rx=0 tx=0).stop 2024-09-21T03:39:05.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:05.566+0000 7feb0114a640 1 -- 172.21.15.38:0/3696018729 reap_dead start 2024-09-21T03:39:05.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:05.566+0000 7feb02bad640 1 -- 172.21.15.38:0/3696018729 shutdown_connections 2024-09-21T03:39:05.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:05.566+0000 7feb02bad640 1 -- 172.21.15.38:0/3696018729 >> 172.21.15.38:0/3696018729 conn(0x7feafc0d4440 msgr2=0x7feafc0d4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:05.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:05.567+0000 7feb02bad640 1 -- 172.21.15.38:0/3696018729 shutdown_connections 2024-09-21T03:39:05.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:05.567+0000 7feb02bad640 1 -- 172.21.15.38:0/3696018729 wait complete. 2024-09-21T03:39:05.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:05.568+0000 7feb02bad640 1 Processor -- start 2024-09-21T03:39:05.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:05.568+0000 7feb02bad640 1 -- start start 2024-09-21T03:39:05.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:05.570+0000 7feb02bad640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7feae8004d40 0x7feafc15c230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:05.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:05.570+0000 7feb02bad640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feafc1f58c0 0x7feafc1f5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:05.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:05.570+0000 7feb02bad640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feafc1f8db0 0x7feafc1f91c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:05.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:05.570+0000 7feb02bad640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7feafc0b33f0 con 0x7feafc1f8db0 2024-09-21T03:39:05.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:05.570+0000 7feb02bad640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7feafc0b3270 con 0x7feafc1f58c0 2024-09-21T03:39:05.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:05.570+0000 7feb02bad640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7feafc0b3570 con 0x7feae8004d40 2024-09-21T03:39:05.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:05.570+0000 7feb0114a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feafc1f8db0 0x7feafc1f91c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:05.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:05.570+0000 7feb0114a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feafc1f8db0 0x7feafc1f91c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:52556/0 (socket says 172.21.15.38:52556) 2024-09-21T03:39:05.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:05.570+0000 7feb0114a640 1 -- 172.21.15.38:0/1020563789 learned_addr learned my addr 172.21.15.38:0/1020563789 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:39:05.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:05.570+0000 7feb00949640 1 --2- 172.21.15.38:0/1020563789 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7feae8004d40 0x7feafc15c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:05.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:05.570+0000 7feafbfff640 1 --2- 172.21.15.38:0/1020563789 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feafc1f58c0 0x7feafc1f5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:05.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:05.570+0000 7feb0114a640 1 -- 172.21.15.38:0/1020563789 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7feae8004d40 msgr2=0x7feafc15c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:05.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:05.570+0000 7feb0114a640 1 --2- 172.21.15.38:0/1020563789 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7feae8004d40 0x7feafc15c230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:05.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:05.570+0000 7feb0114a640 1 -- 172.21.15.38:0/1020563789 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feafc1f58c0 msgr2=0x7feafc1f5cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:05.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:05.570+0000 7feb0114a640 1 --2- 172.21.15.38:0/1020563789 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feafc1f58c0 0x7feafc1f5cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:05.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:05.570+0000 7feb0114a640 1 -- 172.21.15.38:0/1020563789 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feafc1f9700 con 0x7feafc1f8db0 2024-09-21T03:39:05.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:05.570+0000 7feb00949640 1 --2- 172.21.15.38:0/1020563789 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7feae8004d40 0x7feafc15c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:39:05.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:05.570+0000 7feafbfff640 1 --2- 172.21.15.38:0/1020563789 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feafc1f58c0 0x7feafc1f5cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:39:05.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:05.570+0000 7feb0114a640 1 --2- 172.21.15.38:0/1020563789 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feafc1f8db0 0x7feafc1f91c0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7feaf402b870 tx=0x7feaf4033bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:05.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:05.571+0000 7feaf97fa640 1 -- 172.21.15.38:0/1020563789 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaf4042070 con 0x7feafc1f8db0 2024-09-21T03:39:05.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:05.571+0000 7feaf97fa640 1 -- 172.21.15.38:0/1020563789 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7feaf4003970 con 0x7feafc1f8db0 2024-09-21T03:39:05.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:05.571+0000 7feaf97fa640 1 -- 172.21.15.38:0/1020563789 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaf4038040 con 0x7feafc1f8db0 2024-09-21T03:39:05.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:05.571+0000 7feb02bad640 1 -- 172.21.15.38:0/1020563789 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feafc1f6680 con 0x7feafc1f8db0 2024-09-21T03:39:05.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:05.571+0000 7feb02bad640 1 -- 172.21.15.38:0/1020563789 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feafc15dde0 con 0x7feafc1f8db0 2024-09-21T03:39:05.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:05.572+0000 7feb02bad640 1 -- 172.21.15.38:0/1020563789 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feafc1f63f0 con 0x7feafc1f8db0 2024-09-21T03:39:05.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:05.572+0000 7feaf97fa640 1 -- 172.21.15.38:0/1020563789 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7feaf4004030 con 0x7feafc1f8db0 2024-09-21T03:39:05.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:05.573+0000 7feaf97fa640 1 --2- 172.21.15.38:0/1020563789 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7feadc078b90 0x7feadc07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:05.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:05.573+0000 7feaf97fa640 1 -- 172.21.15.38:0/1020563789 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 6101+0+0 (secure 0 0 0) 0x7feaf40bb460 con 0x7feafc1f8db0 2024-09-21T03:39:05.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:05.573+0000 7feb00949640 1 --2- 172.21.15.38:0/1020563789 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7feadc078b90 0x7feadc07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:05.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:05.574+0000 7feb00949640 1 --2- 172.21.15.38:0/1020563789 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7feadc078b90 0x7feadc07b0a0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7feaec009bc0 tx=0x7feaec005f90 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:05.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:05.578+0000 7feaf97fa640 1 -- 172.21.15.38:0/1020563789 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7feafc1f9700 con 0x7feafc1f8db0 2024-09-21T03:39:05.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:05.765+0000 7feb02bad640 1 -- 172.21.15.38:0/1020563789 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7feafc162380 con 0x7feafc1f8db0 2024-09-21T03:39:05.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:05.766+0000 7feaf97fa640 1 -- 172.21.15.38:0/1020563789 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+16982 (secure 0 0 0) 0x7feafc162380 con 0x7feafc1f8db0 2024-09-21T03:39:05.768 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:39:05.769 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":81,"fsid":"9d50d66f-656b-4fe2-8e3b-fe53b79c4d43","created":"2024-09-21T03:33:10.677865+0000","modified":"2024-09-21T03:39:04.779987+0000","last_up_change":"2024-09-21T03:33:24.023421+0000","last_in_change":"2024-09-21T03:33:14.296193+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T03:33:22.992573+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-09-21T03:39:02.446050+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-09-21T03:39:03.739922+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dbab183f-7e63-48b8-960a-ae55fd5bf565","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6801","nonce":2381639433}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6803","nonce":2381639433}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6807","nonce":2381639433}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6805","nonce":2381639433}]},"public_addr":"172.21.15.38:6801/2381639433","cluster_addr":"172.21.15.38:6803/2381639433","heartbeat_back_addr":"172.21.15.38:6807/2381639433","heartbeat_front_addr":"172.21.15.38:6805/2381639433","state":["exists","up"]},{"osd":1,"uuid":"fa1c0e8a-ebc1-4359-8146-bd5342abbc16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6817","nonce":1491818732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6819","nonce":1491818732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6823","nonce":1491818732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6821","nonce":1491818732}]},"public_addr":"172.21.15.38:6817/1491818732","cluster_addr":"172.21.15.38:6819/1491818732","heartbeat_back_addr":"172.21.15.38:6823/1491818732","heartbeat_front_addr":"172.21.15.38:6821/1491818732","state":["exists","up"]},{"osd":2,"uuid":"2ed3cd2d-385d-4fc6-9869-214552a84803","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6825","nonce":2577858902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6827","nonce":2577858902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6831","nonce":2577858902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6829","nonce":2577858902}]},"public_addr":"172.21.15.38:6825/2577858902","cluster_addr":"172.21.15.38:6827/2577858902","heartbeat_back_addr":"172.21.15.38:6831/2577858902","heartbeat_front_addr":"172.21.15.38:6829/2577858902","state":["exists","up"]},{"osd":3,"uuid":"62d1d95e-919d-4250-be71-1d34cfa70dbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6809","nonce":1286810294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6811","nonce":1286810294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6815","nonce":1286810294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6813","nonce":1286810294}]},"public_addr":"172.21.15.38:6809/1286810294","cluster_addr":"172.21.15.38:6811/1286810294","heartbeat_back_addr":"172.21.15.38:6815/1286810294","heartbeat_front_addr":"172.21.15.38:6813/1286810294","state":["exists","up"]},{"osd":4,"uuid":"7c9f9314-99d7-4d73-8392-37809a9f54b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6803","nonce":3401837571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6805","nonce":3401837571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6809","nonce":3401837571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6807","nonce":3401837571}]},"public_addr":"172.21.15.195:6803/3401837571","cluster_addr":"172.21.15.195:6805/3401837571","heartbeat_back_addr":"172.21.15.195:6809/3401837571","heartbeat_front_addr":"172.21.15.195:6807/3401837571","state":["exists","up"]},{"osd":5,"uuid":"bde4fe9f-8414-4bba-bf4a-43973d1976bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6811","nonce":2080960180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6813","nonce":2080960180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6817","nonce":2080960180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6815","nonce":2080960180}]},"public_addr":"172.21.15.195:6811/2080960180","cluster_addr":"172.21.15.195:6813/2080960180","heartbeat_back_addr":"172.21.15.195:6817/2080960180","heartbeat_front_addr":"172.21.15.195:6815/2080960180","state":["exists","up"]},{"osd":6,"uuid":"6ee6d7f4-1a5f-4a4c-ac16-48fe05f6a002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6827","nonce":4055825675}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6829","nonce":4055825675}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6833","nonce":4055825675}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6831","nonce":4055825675}]},"public_addr":"172.21.15.195:6827/4055825675","cluster_addr":"172.21.15.195:6829/4055825675","heartbeat_back_addr":"172.21.15.195:6833/4055825675","heartbeat_front_addr":"172.21.15.195:6831/4055825675","state":["exists","up"]},{"osd":7,"uuid":"efeb5f51-329a-4484-8290-92e7d3503d05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6819","nonce":1606106342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6821","nonce":1606106342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6825","nonce":1606106342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6823","nonce":1606106342}]},"public_addr":"172.21.15.195:6819/1606106342","cluster_addr":"172.21.15.195:6821/1606106342","heartbeat_back_addr":"172.21.15.195:6825/1606106342","heartbeat_front_addr":"172.21.15.195:6823/1606106342","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.901343+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:21.115003+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.901623+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.835238+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.132199+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:19.817560+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.829690+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:21.087300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T03:39:05.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:05.769+0000 7feb02bad640 1 -- 172.21.15.38:0/1020563789 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7feadc078b90 msgr2=0x7feadc07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:05.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:05.769+0000 7feb02bad640 1 --2- 172.21.15.38:0/1020563789 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7feadc078b90 0x7feadc07b0a0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7feaec009bc0 tx=0x7feaec005f90 comp rx=0 tx=0).stop 2024-09-21T03:39:05.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:05.769+0000 7feb02bad640 1 -- 172.21.15.38:0/1020563789 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feafc1f8db0 msgr2=0x7feafc1f91c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:05.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:05.769+0000 7feb02bad640 1 --2- 172.21.15.38:0/1020563789 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feafc1f8db0 0x7feafc1f91c0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7feaf402b870 tx=0x7feaf4033bf0 comp rx=0 tx=0).stop 2024-09-21T03:39:05.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:05.769+0000 7feb02bad640 1 -- 172.21.15.38:0/1020563789 shutdown_connections 2024-09-21T03:39:05.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:05.769+0000 7feb02bad640 1 --2- 172.21.15.38:0/1020563789 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feafc1f8db0 0x7feafc1f91c0 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:05.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:05.769+0000 7feb02bad640 1 --2- 172.21.15.38:0/1020563789 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feafc1f58c0 0x7feafc1f5cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:05.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:05.769+0000 7feb02bad640 1 --2- 172.21.15.38:0/1020563789 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7feadc078b90 0x7feadc07b0a0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:05.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:05.769+0000 7feb02bad640 1 --2- 172.21.15.38:0/1020563789 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7feae8004d40 0x7feafc15c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:05.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:05.769+0000 7feb02bad640 1 -- 172.21.15.38:0/1020563789 >> 172.21.15.38:0/1020563789 conn(0x7feafc0d4440 msgr2=0x7feafc06d020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:05.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:05.769+0000 7feb02bad640 1 -- 172.21.15.38:0/1020563789 shutdown_connections 2024-09-21T03:39:05.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:05.769+0000 7feb02bad640 1 -- 172.21.15.38:0/1020563789 wait complete. 2024-09-21T03:39:05.782 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T03:39:05.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:05.898+0000 7f53b732b640 1 -- 172.21.15.38:0/2796176922 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5398003a00 msgr2=0x7f5398003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:05.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:05.898+0000 7f53b732b640 1 --2- 172.21.15.38:0/2796176922 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5398003a00 0x7f5398003e70 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f53a8009b50 tx=0x7f5398005830 comp rx=0 tx=0).stop 2024-09-21T03:39:05.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:05.898+0000 7f53b58c8640 1 -- 172.21.15.38:0/2796176922 reap_dead start 2024-09-21T03:39:05.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:05.898+0000 7f53b732b640 1 -- 172.21.15.38:0/2796176922 shutdown_connections 2024-09-21T03:39:05.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:05.898+0000 7f53b732b640 1 -- 172.21.15.38:0/2796176922 >> 172.21.15.38:0/2796176922 conn(0x7f53b00d4440 msgr2=0x7f53b00d4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:05.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:05.898+0000 7f53b732b640 1 -- 172.21.15.38:0/2796176922 shutdown_connections 2024-09-21T03:39:05.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:05.898+0000 7f53b732b640 1 -- 172.21.15.38:0/2796176922 wait complete. 2024-09-21T03:39:05.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:05.900+0000 7f53b732b640 1 Processor -- start 2024-09-21T03:39:05.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:05.900+0000 7f53b732b640 1 -- start start 2024-09-21T03:39:05.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:05.901+0000 7f53b732b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5398003a00 0x7f53b00cdc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:05.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:05.901+0000 7f53b732b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f53b00ab840 0x7f53b01f8980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:05.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:05.901+0000 7f53b732b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f53b01f8ec0 0x7f53b01f92d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:05.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:05.901+0000 7f53b732b640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f53b00d2a20 con 0x7f5398003a00 2024-09-21T03:39:05.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:05.901+0000 7f53b732b640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f53b00b3b90 con 0x7f53b01f8ec0 2024-09-21T03:39:05.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:05.901+0000 7f53b732b640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f53b00d2ba0 con 0x7f53b00ab840 2024-09-21T03:39:05.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:05.901+0000 7f53b50c7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5398003a00 0x7f53b00cdc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:05.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:05.901+0000 7f53b50c7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5398003a00 0x7f53b00cdc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:52576/0 (socket says 172.21.15.38:52576) 2024-09-21T03:39:05.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:05.901+0000 7f53b50c7640 1 -- 172.21.15.38:0/3759100868 learned_addr learned my addr 172.21.15.38:0/3759100868 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:39:05.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:05.902+0000 7f53b58c8640 1 --2- 172.21.15.38:0/3759100868 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f53b01f8ec0 0x7f53b01f92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:05.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:05.902+0000 7f53b48c6640 1 --2- 172.21.15.38:0/3759100868 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f53b00ab840 0x7f53b01f8980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:05.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:05.902+0000 7f53b50c7640 1 -- 172.21.15.38:0/3759100868 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f53b00ab840 msgr2=0x7f53b01f8980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:05.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:05.902+0000 7f53b50c7640 1 --2- 172.21.15.38:0/3759100868 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f53b00ab840 0x7f53b01f8980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:05.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:05.902+0000 7f53b50c7640 1 -- 172.21.15.38:0/3759100868 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f53b01f8ec0 msgr2=0x7f53b01f92d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:05.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:05.902+0000 7f53b50c7640 1 --2- 172.21.15.38:0/3759100868 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f53b01f8ec0 0x7f53b01f92d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:05.906 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:05.902+0000 7f53b50c7640 1 -- 172.21.15.38:0/3759100868 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53b00ca490 con 0x7f5398003a00 2024-09-21T03:39:05.906 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:05.902+0000 7f53b50c7640 1 --2- 172.21.15.38:0/3759100868 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5398003a00 0x7f53b00cdc20 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f53a8036040 tx=0x7f53a8033670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:05.906 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:05.902+0000 7f53adffb640 1 -- 172.21.15.38:0/3759100868 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53a8038050 con 0x7f5398003a00 2024-09-21T03:39:05.906 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:05.902+0000 7f53adffb640 1 -- 172.21.15.38:0/3759100868 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f53a8039040 con 0x7f5398003a00 2024-09-21T03:39:05.906 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:05.902+0000 7f53adffb640 1 -- 172.21.15.38:0/3759100868 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53a8033c50 con 0x7f5398003a00 2024-09-21T03:39:05.906 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:05.902+0000 7f53b732b640 1 -- 172.21.15.38:0/3759100868 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53b00ca770 con 0x7f5398003a00 2024-09-21T03:39:05.906 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:05.902+0000 7f53b732b640 1 -- 172.21.15.38:0/3759100868 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53b00cad30 con 0x7f5398003a00 2024-09-21T03:39:05.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:05.903+0000 7f53b732b640 1 -- 172.21.15.38:0/3759100868 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53b00d30a0 con 0x7f5398003a00 2024-09-21T03:39:05.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:05.904+0000 7f53adffb640 1 -- 172.21.15.38:0/3759100868 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f53a8046020 con 0x7f5398003a00 2024-09-21T03:39:05.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:05.905+0000 7f53adffb640 1 --2- 172.21.15.38:0/3759100868 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f539c078bc0 0x7f539c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:05.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:05.905+0000 7f53adffb640 1 -- 172.21.15.38:0/3759100868 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 6101+0+0 (secure 0 0 0) 0x7f53a80bb8c0 con 0x7f5398003a00 2024-09-21T03:39:05.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:05.905+0000 7f53b48c6640 1 --2- 172.21.15.38:0/3759100868 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f539c078bc0 0x7f539c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:05.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:05.906+0000 7f53b48c6640 1 --2- 172.21.15.38:0/3759100868 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f539c078bc0 0x7f539c07b0d0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f53a4005eb0 tx=0x7f53a4005f90 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:05.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:05.909+0000 7f53adffb640 1 -- 172.21.15.38:0/3759100868 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f53a804d020 con 0x7f5398003a00 2024-09-21T03:39:06.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:06.096+0000 7f53b732b640 1 -- 172.21.15.38:0/3759100868 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f53b00b33a0 con 0x7f5398003a00 2024-09-21T03:39:06.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:06.097+0000 7f53adffb640 1 -- 172.21.15.38:0/3759100868 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+16982 (secure 0 0 0) 0x7f53a8083c90 con 0x7f5398003a00 2024-09-21T03:39:06.099 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:39:06.099 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":81,"fsid":"9d50d66f-656b-4fe2-8e3b-fe53b79c4d43","created":"2024-09-21T03:33:10.677865+0000","modified":"2024-09-21T03:39:04.779987+0000","last_up_change":"2024-09-21T03:33:24.023421+0000","last_in_change":"2024-09-21T03:33:14.296193+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T03:33:22.992573+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-09-21T03:39:02.446050+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-09-21T03:39:03.739922+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dbab183f-7e63-48b8-960a-ae55fd5bf565","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6801","nonce":2381639433}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6803","nonce":2381639433}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6807","nonce":2381639433}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6805","nonce":2381639433}]},"public_addr":"172.21.15.38:6801/2381639433","cluster_addr":"172.21.15.38:6803/2381639433","heartbeat_back_addr":"172.21.15.38:6807/2381639433","heartbeat_front_addr":"172.21.15.38:6805/2381639433","state":["exists","up"]},{"osd":1,"uuid":"fa1c0e8a-ebc1-4359-8146-bd5342abbc16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6817","nonce":1491818732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6819","nonce":1491818732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6823","nonce":1491818732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6821","nonce":1491818732}]},"public_addr":"172.21.15.38:6817/1491818732","cluster_addr":"172.21.15.38:6819/1491818732","heartbeat_back_addr":"172.21.15.38:6823/1491818732","heartbeat_front_addr":"172.21.15.38:6821/1491818732","state":["exists","up"]},{"osd":2,"uuid":"2ed3cd2d-385d-4fc6-9869-214552a84803","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6825","nonce":2577858902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6827","nonce":2577858902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6831","nonce":2577858902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6829","nonce":2577858902}]},"public_addr":"172.21.15.38:6825/2577858902","cluster_addr":"172.21.15.38:6827/2577858902","heartbeat_back_addr":"172.21.15.38:6831/2577858902","heartbeat_front_addr":"172.21.15.38:6829/2577858902","state":["exists","up"]},{"osd":3,"uuid":"62d1d95e-919d-4250-be71-1d34cfa70dbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6809","nonce":1286810294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6811","nonce":1286810294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6815","nonce":1286810294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6813","nonce":1286810294}]},"public_addr":"172.21.15.38:6809/1286810294","cluster_addr":"172.21.15.38:6811/1286810294","heartbeat_back_addr":"172.21.15.38:6815/1286810294","heartbeat_front_addr":"172.21.15.38:6813/1286810294","state":["exists","up"]},{"osd":4,"uuid":"7c9f9314-99d7-4d73-8392-37809a9f54b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6803","nonce":3401837571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6805","nonce":3401837571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6809","nonce":3401837571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6807","nonce":3401837571}]},"public_addr":"172.21.15.195:6803/3401837571","cluster_addr":"172.21.15.195:6805/3401837571","heartbeat_back_addr":"172.21.15.195:6809/3401837571","heartbeat_front_addr":"172.21.15.195:6807/3401837571","state":["exists","up"]},{"osd":5,"uuid":"bde4fe9f-8414-4bba-bf4a-43973d1976bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6811","nonce":2080960180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6813","nonce":2080960180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6817","nonce":2080960180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6815","nonce":2080960180}]},"public_addr":"172.21.15.195:6811/2080960180","cluster_addr":"172.21.15.195:6813/2080960180","heartbeat_back_addr":"172.21.15.195:6817/2080960180","heartbeat_front_addr":"172.21.15.195:6815/2080960180","state":["exists","up"]},{"osd":6,"uuid":"6ee6d7f4-1a5f-4a4c-ac16-48fe05f6a002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6827","nonce":4055825675}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6829","nonce":4055825675}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6833","nonce":4055825675}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6831","nonce":4055825675}]},"public_addr":"172.21.15.195:6827/4055825675","cluster_addr":"172.21.15.195:6829/4055825675","heartbeat_back_addr":"172.21.15.195:6833/4055825675","heartbeat_front_addr":"172.21.15.195:6831/4055825675","state":["exists","up"]},{"osd":7,"uuid":"efeb5f51-329a-4484-8290-92e7d3503d05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6819","nonce":1606106342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6821","nonce":1606106342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6825","nonce":1606106342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6823","nonce":1606106342}]},"public_addr":"172.21.15.195:6819/1606106342","cluster_addr":"172.21.15.195:6821/1606106342","heartbeat_back_addr":"172.21.15.195:6825/1606106342","heartbeat_front_addr":"172.21.15.195:6823/1606106342","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.901343+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:21.115003+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.901623+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.835238+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.132199+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:19.817560+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.829690+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:21.087300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T03:39:06.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:06.099+0000 7f53b732b640 1 -- 172.21.15.38:0/3759100868 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f539c078bc0 msgr2=0x7f539c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:06.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:06.099+0000 7f53b732b640 1 --2- 172.21.15.38:0/3759100868 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f539c078bc0 0x7f539c07b0d0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f53a4005eb0 tx=0x7f53a4005f90 comp rx=0 tx=0).stop 2024-09-21T03:39:06.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:06.099+0000 7f53b732b640 1 -- 172.21.15.38:0/3759100868 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5398003a00 msgr2=0x7f53b00cdc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:06.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:06.099+0000 7f53b732b640 1 --2- 172.21.15.38:0/3759100868 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5398003a00 0x7f53b00cdc20 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f53a8036040 tx=0x7f53a8033670 comp rx=0 tx=0).stop 2024-09-21T03:39:06.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:06.099+0000 7f53b732b640 1 -- 172.21.15.38:0/3759100868 shutdown_connections 2024-09-21T03:39:06.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:06.099+0000 7f53b732b640 1 --2- 172.21.15.38:0/3759100868 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f53b01f8ec0 0x7f53b01f92d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:06.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:06.099+0000 7f53b732b640 1 --2- 172.21.15.38:0/3759100868 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f539c078bc0 0x7f539c07b0d0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:06.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:06.099+0000 7f53b732b640 1 --2- 172.21.15.38:0/3759100868 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f53b00ab840 0x7f53b01f8980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:06.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:06.099+0000 7f53b732b640 1 --2- 172.21.15.38:0/3759100868 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5398003a00 0x7f53b00cdc20 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:06.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:06.099+0000 7f53b732b640 1 -- 172.21.15.38:0/3759100868 >> 172.21.15.38:0/3759100868 conn(0x7f53b00d4440 msgr2=0x7f53b00d15f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:06.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:06.099+0000 7f53b732b640 1 -- 172.21.15.38:0/3759100868 shutdown_connections 2024-09-21T03:39:06.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:06.099+0000 7f53b732b640 1 -- 172.21.15.38:0/3759100868 wait complete. 2024-09-21T03:39:06.113 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-21T03:39:06.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:06.231+0000 7f31797ba640 1 -- 172.21.15.38:0/3693868985 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f317415ee80 msgr2=0x7f31740d0170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:06.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:06.231+0000 7f31797ba640 1 --2- 172.21.15.38:0/3693868985 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f317415ee80 0x7f31740d0170 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f31740d06b0 tx=0x7f316402a8f0 comp rx=0 tx=0).stop 2024-09-21T03:39:06.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:06.231+0000 7f31797ba640 1 -- 172.21.15.38:0/3693868985 shutdown_connections 2024-09-21T03:39:06.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:06.231+0000 7f31797ba640 1 --2- 172.21.15.38:0/3693868985 >> v2:172.21.15.195:3300/0 conn(0x7f31740b9ad0 0x7f31740d0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:06.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:06.231+0000 7f31797ba640 1 --2- 172.21.15.38:0/3693868985 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f317415ee80 0x7f31740d0170 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:06.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:06.231+0000 7f31797ba640 1 -- 172.21.15.38:0/3693868985 >> 172.21.15.38:0/3693868985 conn(0x7f317409d830 msgr2=0x7f31740b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:06.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:06.231+0000 7f31797ba640 1 -- 172.21.15.38:0/3693868985 shutdown_connections 2024-09-21T03:39:06.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:06.231+0000 7f31797ba640 1 -- 172.21.15.38:0/3693868985 wait complete. 2024-09-21T03:39:06.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:06.233+0000 7f31797ba640 1 Processor -- start 2024-09-21T03:39:06.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:06.233+0000 7f31797ba640 1 -- start start 2024-09-21T03:39:06.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:06.234+0000 7f31797ba640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f31740b9ad0 0x7f317406f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:06.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:06.234+0000 7f31797ba640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f31740d1120 0x7f317406d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:06.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:06.234+0000 7f31797ba640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f317415ee80 0x7f31741585f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:06.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:06.234+0000 7f31797ba640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f31740b6700 con 0x7f31740b9ad0 2024-09-21T03:39:06.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:06.234+0000 7f31797ba640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f31740b6580 con 0x7f31740d1120 2024-09-21T03:39:06.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:06.234+0000 7f31797ba640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f31740b6880 con 0x7f317415ee80 2024-09-21T03:39:06.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:06.234+0000 7f3172ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f31740b9ad0 0x7f317406f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:06.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:06.234+0000 7f3172ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f31740b9ad0 0x7f317406f580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:52596/0 (socket says 172.21.15.38:52596) 2024-09-21T03:39:06.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:06.234+0000 7f3172ffd640 1 -- 172.21.15.38:0/1938408578 learned_addr learned my addr 172.21.15.38:0/1938408578 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:39:06.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:06.235+0000 7f31727fc640 1 --2- 172.21.15.38:0/1938408578 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f31740d1120 0x7f317406d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:06.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:06.235+0000 7f31737fe640 1 --2- 172.21.15.38:0/1938408578 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f317415ee80 0x7f31741585f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:06.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:06.235+0000 7f3172ffd640 1 -- 172.21.15.38:0/1938408578 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f317415ee80 msgr2=0x7f31741585f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:06.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:06.235+0000 7f3172ffd640 1 --2- 172.21.15.38:0/1938408578 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f317415ee80 0x7f31741585f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:06.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:06.235+0000 7f3172ffd640 1 -- 172.21.15.38:0/1938408578 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f31740d1120 msgr2=0x7f317406d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:06.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:06.235+0000 7f3172ffd640 1 --2- 172.21.15.38:0/1938408578 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f31740d1120 0x7f317406d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:06.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:06.235+0000 7f3172ffd640 1 -- 172.21.15.38:0/1938408578 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31740b2d10 con 0x7f31740b9ad0 2024-09-21T03:39:06.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:06.235+0000 7f3172ffd640 1 --2- 172.21.15.38:0/1938408578 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f31740b9ad0 0x7f317406f580 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f3164033040 tx=0x7f3164003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:06.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:06.235+0000 7f316b7fe640 1 -- 172.21.15.38:0/1938408578 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3164004240 con 0x7f31740b9ad0 2024-09-21T03:39:06.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:06.235+0000 7f316b7fe640 1 -- 172.21.15.38:0/1938408578 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f316402d070 con 0x7f31740b9ad0 2024-09-21T03:39:06.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:06.235+0000 7f31797ba640 1 -- 172.21.15.38:0/1938408578 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31740b2ff0 con 0x7f31740b9ad0 2024-09-21T03:39:06.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:06.235+0000 7f316b7fe640 1 -- 172.21.15.38:0/1938408578 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3164004680 con 0x7f31740b9ad0 2024-09-21T03:39:06.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:06.235+0000 7f31797ba640 1 -- 172.21.15.38:0/1938408578 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31741e8470 con 0x7f31740b9ad0 2024-09-21T03:39:06.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:06.236+0000 7f31797ba640 1 -- 172.21.15.38:0/1938408578 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31740b3510 con 0x7f31740b9ad0 2024-09-21T03:39:06.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:06.237+0000 7f316b7fe640 1 -- 172.21.15.38:0/1938408578 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f3164031480 con 0x7f31740b9ad0 2024-09-21T03:39:06.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:06.238+0000 7f316b7fe640 1 --2- 172.21.15.38:0/1938408578 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3154078bc0 0x7f315407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:06.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:06.238+0000 7f316b7fe640 1 -- 172.21.15.38:0/1938408578 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 6101+0+0 (secure 0 0 0) 0x7f31640bc740 con 0x7f31740b9ad0 2024-09-21T03:39:06.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:06.238+0000 7f31727fc640 1 --2- 172.21.15.38:0/1938408578 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3154078bc0 0x7f315407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:06.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:06.238+0000 7f31727fc640 1 --2- 172.21.15.38:0/1938408578 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3154078bc0 0x7f315407b0d0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f31740b1880 tx=0x7f315c00b2e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:06.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:06.242+0000 7f316b7fe640 1 -- 172.21.15.38:0/1938408578 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f31740b3510 con 0x7f31740b9ad0 2024-09-21T03:39:06.379 INFO:tasks.ceph.mds.c.smithi038.stderr:2024-09-21T03:39:06.377+0000 7f6ea6c2e640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T03:39:06.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:06.468+0000 7f31797ba640 1 -- 172.21.15.38:0/1938408578 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f31740b6d70 con 0x7f31740b9ad0 2024-09-21T03:39:07.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:07.423+0000 7f316b7fe640 1 -- 172.21.15.38:0/1938408578 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v66) ==== 118+0+0 (secure 0 0 0) 0x7f31741592d0 con 0x7f31740b9ad0 2024-09-21T03:39:07.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:07.426+0000 7f31797ba640 1 -- 172.21.15.38:0/1938408578 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3154078bc0 msgr2=0x7f315407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:07.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:07.426+0000 7f31797ba640 1 --2- 172.21.15.38:0/1938408578 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3154078bc0 0x7f315407b0d0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f31740b1880 tx=0x7f315c00b2e0 comp rx=0 tx=0).stop 2024-09-21T03:39:07.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:07.426+0000 7f31797ba640 1 -- 172.21.15.38:0/1938408578 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f31740b9ad0 msgr2=0x7f317406f580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:07.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:07.426+0000 7f31797ba640 1 --2- 172.21.15.38:0/1938408578 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f31740b9ad0 0x7f317406f580 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f3164033040 tx=0x7f3164003c20 comp rx=0 tx=0).stop 2024-09-21T03:39:07.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:07.426+0000 7f31797ba640 1 -- 172.21.15.38:0/1938408578 shutdown_connections 2024-09-21T03:39:07.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:07.426+0000 7f31797ba640 1 --2- 172.21.15.38:0/1938408578 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3154078bc0 0x7f315407b0d0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:07.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:07.426+0000 7f31797ba640 1 --2- 172.21.15.38:0/1938408578 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f317415ee80 0x7f31741585f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:07.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:07.426+0000 7f31797ba640 1 --2- 172.21.15.38:0/1938408578 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f31740d1120 0x7f317406d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:07.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:07.426+0000 7f31797ba640 1 --2- 172.21.15.38:0/1938408578 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f31740b9ad0 0x7f317406f580 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:07.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:07.426+0000 7f31797ba640 1 -- 172.21.15.38:0/1938408578 >> 172.21.15.38:0/1938408578 conn(0x7f317409d830 msgr2=0x7f31740b7a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:07.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:07.427+0000 7f31797ba640 1 -- 172.21.15.38:0/1938408578 shutdown_connections 2024-09-21T03:39:07.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:07.427+0000 7f31797ba640 1 -- 172.21.15.38:0/1938408578 wait complete. 2024-09-21T03:39:07.443 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-21T03:39:07.443 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-21T03:39:07.551 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a has been restored 2024-09-21T03:39:07.551 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c has been restored 2024-09-21T03:39:07.552 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b has been restored 2024-09-21T03:39:07.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:07.569+0000 7f9c6db52640 1 -- 172.21.15.38:0/2447192841 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9c680ceb90 msgr2=0x7f9c680cefa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:07.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:07.569+0000 7f9c6db52640 1 --2- 172.21.15.38:0/2447192841 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9c680ceb90 0x7f9c680cefa0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f9c680cfba0 tx=0x7f9c6002a8f0 comp rx=0 tx=0).stop 2024-09-21T03:39:07.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:07.569+0000 7f9c6db52640 1 -- 172.21.15.38:0/2447192841 shutdown_connections 2024-09-21T03:39:07.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:07.569+0000 7f9c6db52640 1 --2- 172.21.15.38:0/2447192841 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9c680ceb90 0x7f9c680cefa0 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:07.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:07.569+0000 7f9c6db52640 1 --2- 172.21.15.38:0/2447192841 >> v2:172.21.15.195:3300/0 conn(0x7f9c6815ee80 0x7f9c680d0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:07.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:07.569+0000 7f9c6db52640 1 -- 172.21.15.38:0/2447192841 >> 172.21.15.38:0/2447192841 conn(0x7f9c6809d830 msgr2=0x7f9c680b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:07.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:07.569+0000 7f9c6db52640 1 -- 172.21.15.38:0/2447192841 shutdown_connections 2024-09-21T03:39:07.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:07.569+0000 7f9c6db52640 1 -- 172.21.15.38:0/2447192841 wait complete. 2024-09-21T03:39:07.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:07.570+0000 7f9c6db52640 1 Processor -- start 2024-09-21T03:39:07.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:07.571+0000 7f9c6db52640 1 -- start start 2024-09-21T03:39:07.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:07.572+0000 7f9c6db52640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9c680b9ad0 0x7f9c68157d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:07.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:07.572+0000 7f9c6db52640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9c680ceb90 0x7f9c680cb610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:07.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:07.572+0000 7f9c6db52640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9c6815ee80 0x7f9c6814c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:07.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:07.572+0000 7f9c6db52640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f9c680b6700 con 0x7f9c6815ee80 2024-09-21T03:39:07.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:07.572+0000 7f9c6db52640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f9c680b6580 con 0x7f9c680ceb90 2024-09-21T03:39:07.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:07.572+0000 7f9c6db52640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f9c680b6880 con 0x7f9c680b9ad0 2024-09-21T03:39:07.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:07.572+0000 7f9c67fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9c6815ee80 0x7f9c6814c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:07.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:07.572+0000 7f9c67fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9c6815ee80 0x7f9c6814c7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:52622/0 (socket says 172.21.15.38:52622) 2024-09-21T03:39:07.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:07.572+0000 7f9c67fff640 1 -- 172.21.15.38:0/1113919548 learned_addr learned my addr 172.21.15.38:0/1113919548 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:39:07.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:07.572+0000 7f9c66ffd640 1 --2- 172.21.15.38:0/1113919548 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9c680ceb90 0x7f9c680cb610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:07.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:07.572+0000 7f9c677fe640 1 --2- 172.21.15.38:0/1113919548 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9c680b9ad0 0x7f9c68157d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:07.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:07.573+0000 7f9c67fff640 1 -- 172.21.15.38:0/1113919548 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9c680b9ad0 msgr2=0x7f9c68157d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:07.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:07.573+0000 7f9c67fff640 1 --2- 172.21.15.38:0/1113919548 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9c680b9ad0 0x7f9c68157d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:07.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:07.573+0000 7f9c67fff640 1 -- 172.21.15.38:0/1113919548 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9c680ceb90 msgr2=0x7f9c680cb610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:07.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:07.573+0000 7f9c67fff640 1 --2- 172.21.15.38:0/1113919548 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9c680ceb90 0x7f9c680cb610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:07.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:07.573+0000 7f9c67fff640 1 -- 172.21.15.38:0/1113919548 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c680cbc60 con 0x7f9c6815ee80 2024-09-21T03:39:07.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:07.573+0000 7f9c67fff640 1 --2- 172.21.15.38:0/1113919548 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9c6815ee80 0x7f9c6814c7a0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f9c60009bf0 tx=0x7f9c60003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:07.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:07.573+0000 7f9c4ffff640 1 -- 172.21.15.38:0/1113919548 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c60004040 con 0x7f9c6815ee80 2024-09-21T03:39:07.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:07.573+0000 7f9c4ffff640 1 -- 172.21.15.38:0/1113919548 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9c6002d070 con 0x7f9c6815ee80 2024-09-21T03:39:07.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:07.573+0000 7f9c6db52640 1 -- 172.21.15.38:0/1113919548 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c6806e4a0 con 0x7f9c6815ee80 2024-09-21T03:39:07.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:07.573+0000 7f9c4ffff640 1 -- 172.21.15.38:0/1113919548 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c60004420 con 0x7f9c6815ee80 2024-09-21T03:39:07.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:07.573+0000 7f9c6db52640 1 -- 172.21.15.38:0/1113919548 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c6806ea60 con 0x7f9c6815ee80 2024-09-21T03:39:07.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:07.574+0000 7f9c6db52640 1 -- 172.21.15.38:0/1113919548 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c680b6df0 con 0x7f9c6815ee80 2024-09-21T03:39:07.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:07.575+0000 7f9c4ffff640 1 -- 172.21.15.38:0/1113919548 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f9c600045c0 con 0x7f9c6815ee80 2024-09-21T03:39:07.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:07.576+0000 7f9c4ffff640 1 --2- 172.21.15.38:0/1113919548 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9c54078b90 0x7f9c5407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:07.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:07.576+0000 7f9c4ffff640 1 -- 172.21.15.38:0/1113919548 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 6101+0+0 (secure 0 0 0) 0x7f9c600bc6d0 con 0x7f9c6815ee80 2024-09-21T03:39:07.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:07.576+0000 7f9c677fe640 1 --2- 172.21.15.38:0/1113919548 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9c54078b90 0x7f9c5407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:07.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:07.576+0000 7f9c677fe640 1 --2- 172.21.15.38:0/1113919548 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9c54078b90 0x7f9c5407b0a0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f9c5800bb50 tx=0x7f9c5800b2e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:07.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:07.580+0000 7f9c4ffff640 1 -- 172.21.15.38:0/1113919548 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f9c681592d0 con 0x7f9c6815ee80 2024-09-21T03:39:07.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:07.808+0000 7f9c6db52640 1 -- 172.21.15.38:0/1113919548 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f9c680d0470 con 0x7f9c6815ee80 2024-09-21T03:39:08.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:08.426+0000 7f9c4ffff640 1 -- 172.21.15.38:0/1113919548 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v67) ==== 122+0+0 (secure 0 0 0) 0x7f9c600843c0 con 0x7f9c6815ee80 2024-09-21T03:39:08.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:08.429+0000 7f9c6db52640 1 -- 172.21.15.38:0/1113919548 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9c54078b90 msgr2=0x7f9c5407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:08.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:08.429+0000 7f9c6db52640 1 --2- 172.21.15.38:0/1113919548 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9c54078b90 0x7f9c5407b0a0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f9c5800bb50 tx=0x7f9c5800b2e0 comp rx=0 tx=0).stop 2024-09-21T03:39:08.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:08.429+0000 7f9c6db52640 1 -- 172.21.15.38:0/1113919548 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9c6815ee80 msgr2=0x7f9c6814c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:08.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:08.429+0000 7f9c6db52640 1 --2- 172.21.15.38:0/1113919548 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9c6815ee80 0x7f9c6814c7a0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f9c60009bf0 tx=0x7f9c60003ba0 comp rx=0 tx=0).stop 2024-09-21T03:39:08.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:08.429+0000 7f9c6db52640 1 -- 172.21.15.38:0/1113919548 shutdown_connections 2024-09-21T03:39:08.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:08.429+0000 7f9c6db52640 1 --2- 172.21.15.38:0/1113919548 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9c6815ee80 0x7f9c6814c7a0 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:08.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:08.429+0000 7f9c6db52640 1 --2- 172.21.15.38:0/1113919548 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9c680ceb90 0x7f9c680cb610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:08.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:08.429+0000 7f9c6db52640 1 --2- 172.21.15.38:0/1113919548 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9c54078b90 0x7f9c5407b0a0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:08.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:08.429+0000 7f9c6db52640 1 --2- 172.21.15.38:0/1113919548 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9c680b9ad0 0x7f9c68157d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:08.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:08.429+0000 7f9c6db52640 1 -- 172.21.15.38:0/1113919548 >> 172.21.15.38:0/1113919548 conn(0x7f9c6809d830 msgr2=0x7f9c680d06e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:08.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:08.429+0000 7f9c6db52640 1 -- 172.21.15.38:0/1113919548 shutdown_connections 2024-09-21T03:39:08.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:08.430+0000 7f9c6db52640 1 -- 172.21.15.38:0/1113919548 wait complete. 2024-09-21T03:39:08.446 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-21T03:39:08.446 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:39:08.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:08.568+0000 7f18b0c7e640 1 -- 172.21.15.38:0/835951139 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1898003a50 msgr2=0x7f1898003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:08.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:08.568+0000 7f18b0c7e640 1 --2- 172.21.15.38:0/835951139 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1898003a50 0x7f1898003ec0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f189c00c750 tx=0x7f1898005830 comp rx=0 tx=0).stop 2024-09-21T03:39:08.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:08.568+0000 7f18aad9d640 1 -- 172.21.15.38:0/835951139 reap_dead start 2024-09-21T03:39:08.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:08.568+0000 7f18b0c7e640 1 -- 172.21.15.38:0/835951139 shutdown_connections 2024-09-21T03:39:08.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:08.568+0000 7f18b0c7e640 1 -- 172.21.15.38:0/835951139 >> 172.21.15.38:0/835951139 conn(0x7f18ac0d4440 msgr2=0x7f18ac0d4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:08.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:08.568+0000 7f18b0c7e640 1 -- 172.21.15.38:0/835951139 shutdown_connections 2024-09-21T03:39:08.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:08.568+0000 7f18b0c7e640 1 -- 172.21.15.38:0/835951139 wait complete. 2024-09-21T03:39:08.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:08.570+0000 7f18b0c7e640 1 Processor -- start 2024-09-21T03:39:08.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:08.570+0000 7f18b0c7e640 1 -- start start 2024-09-21T03:39:08.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:08.571+0000 7f18b0c7e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1898003a50 0x7f18ac1f8610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:08.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:08.571+0000 7f18b0c7e640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f18ac0aade0 0x7f18ac1f8d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:08.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:08.571+0000 7f18b0c7e640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f18ac1f9270 0x7f18ac1f6a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:08.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:08.571+0000 7f18b0c7e640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f18ac162b20 con 0x7f1898003a50 2024-09-21T03:39:08.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:08.571+0000 7f18b0c7e640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f18ac1629a0 con 0x7f18ac0aade0 2024-09-21T03:39:08.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:08.571+0000 7f18b0c7e640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f18ac162ca0 con 0x7f18ac1f9270 2024-09-21T03:39:08.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:08.572+0000 7f18aa59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1898003a50 0x7f18ac1f8610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:08.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:08.572+0000 7f18aa59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1898003a50 0x7f18ac1f8610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:52630/0 (socket says 172.21.15.38:52630) 2024-09-21T03:39:08.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:08.572+0000 7f18aa59c640 1 -- 172.21.15.38:0/2555007496 learned_addr learned my addr 172.21.15.38:0/2555007496 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:39:08.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:08.572+0000 7f18a9d9b640 1 --2- 172.21.15.38:0/2555007496 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f18ac0aade0 0x7f18ac1f8d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:08.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:08.572+0000 7f18aad9d640 1 --2- 172.21.15.38:0/2555007496 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f18ac1f9270 0x7f18ac1f6a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:08.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:08.572+0000 7f18aa59c640 1 -- 172.21.15.38:0/2555007496 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f18ac1f9270 msgr2=0x7f18ac1f6a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:08.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:08.572+0000 7f18aa59c640 1 --2- 172.21.15.38:0/2555007496 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f18ac1f9270 0x7f18ac1f6a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:08.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:08.572+0000 7f18aa59c640 1 -- 172.21.15.38:0/2555007496 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f18ac0aade0 msgr2=0x7f18ac1f8d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:08.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:08.572+0000 7f18aa59c640 1 --2- 172.21.15.38:0/2555007496 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f18ac0aade0 0x7f18ac1f8d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:08.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:08.572+0000 7f18aa59c640 1 -- 172.21.15.38:0/2555007496 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18ac1f9680 con 0x7f1898003a50 2024-09-21T03:39:08.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:08.572+0000 7f18aa59c640 1 --2- 172.21.15.38:0/2555007496 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1898003a50 0x7f18ac1f8610 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f189c00c750 tx=0x7f189c00c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:08.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:08.572+0000 7f18a2ffd640 1 -- 172.21.15.38:0/2555007496 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f189c02b990 con 0x7f1898003a50 2024-09-21T03:39:08.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:08.572+0000 7f18a2ffd640 1 -- 172.21.15.38:0/2555007496 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f189c02bb30 con 0x7f1898003a50 2024-09-21T03:39:08.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:08.572+0000 7f18b0c7e640 1 -- 172.21.15.38:0/2555007496 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18ac0c9b70 con 0x7f1898003a50 2024-09-21T03:39:08.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:08.572+0000 7f18a2ffd640 1 -- 172.21.15.38:0/2555007496 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f189c035430 con 0x7f1898003a50 2024-09-21T03:39:08.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:08.572+0000 7f18b0c7e640 1 -- 172.21.15.38:0/2555007496 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18ac0ca130 con 0x7f1898003a50 2024-09-21T03:39:08.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:08.573+0000 7f18b0c7e640 1 -- 172.21.15.38:0/2555007496 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18ac0b3a70 con 0x7f1898003a50 2024-09-21T03:39:08.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:08.574+0000 7f18a2ffd640 1 -- 172.21.15.38:0/2555007496 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f189c0355d0 con 0x7f1898003a50 2024-09-21T03:39:08.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:08.575+0000 7f18a2ffd640 1 --2- 172.21.15.38:0/2555007496 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f188c078b90 0x7f188c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:08.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:08.575+0000 7f18a2ffd640 1 -- 172.21.15.38:0/2555007496 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 6101+0+0 (secure 0 0 0) 0x7f189c03b020 con 0x7f1898003a50 2024-09-21T03:39:08.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:08.575+0000 7f18a9d9b640 1 --2- 172.21.15.38:0/2555007496 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f188c078b90 0x7f188c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:08.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:08.576+0000 7f18a9d9b640 1 --2- 172.21.15.38:0/2555007496 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f188c078b90 0x7f188c07b0a0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f18ac15dcd0 tx=0x7f1894005f90 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:08.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:08.580+0000 7f18a2ffd640 1 -- 172.21.15.38:0/2555007496 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f18ac1f9680 con 0x7f1898003a50 2024-09-21T03:39:08.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:08.805+0000 7f18b0c7e640 1 -- 172.21.15.38:0/2555007496 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f18ac1606a0 con 0x7f1898003a50 2024-09-21T03:39:08.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:08.806+0000 7f18a2ffd640 1 -- 172.21.15.38:0/2555007496 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) ==== 94+0+5231 (secure 0 0 0) 0x7f18ac162ee0 con 0x7f1898003a50 2024-09-21T03:39:08.808 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:39:08.808 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":67,"btime":"2024-09-21T03:39:08:424230+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5783,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/1358588311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1358588311},{"type":"v1","addr":"172.21.15.38:6835","nonce":1358588311}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63},{"gid":5789,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/1676483003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":1676483003},{"type":"v1","addr":"172.21.15.195:6837","nonce":1676483003}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63},{"gid":6193,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/2771781100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":2771781100},{"type":"v1","addr":"172.21.15.195:6836","nonce":2771781100}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:39:04.779948+0000","modified":"2024-09-21T03:39:07.810041+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6546},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6546":{"gid":6546,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":3,"addr":"172.21.15.38:6833/1144075315","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":1144075315},{"type":"v1","addr":"172.21.15.38:6833","nonce":1144075315}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6546,"qdb_cluster":[6546]},"id":5}]} 2024-09-21T03:39:08.808 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 67 2024-09-21T03:39:08.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:08.808+0000 7f18b0c7e640 1 -- 172.21.15.38:0/2555007496 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f188c078b90 msgr2=0x7f188c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:08.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:08.808+0000 7f18b0c7e640 1 --2- 172.21.15.38:0/2555007496 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f188c078b90 0x7f188c07b0a0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f18ac15dcd0 tx=0x7f1894005f90 comp rx=0 tx=0).stop 2024-09-21T03:39:08.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:08.808+0000 7f18b0c7e640 1 -- 172.21.15.38:0/2555007496 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1898003a50 msgr2=0x7f18ac1f8610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:08.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:08.808+0000 7f18b0c7e640 1 --2- 172.21.15.38:0/2555007496 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1898003a50 0x7f18ac1f8610 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f189c00c750 tx=0x7f189c00c4b0 comp rx=0 tx=0).stop 2024-09-21T03:39:08.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:08.808+0000 7f18b0c7e640 1 -- 172.21.15.38:0/2555007496 shutdown_connections 2024-09-21T03:39:08.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:08.808+0000 7f18b0c7e640 1 --2- 172.21.15.38:0/2555007496 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f188c078b90 0x7f188c07b0a0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:08.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:08.808+0000 7f18b0c7e640 1 --2- 172.21.15.38:0/2555007496 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f18ac1f9270 0x7f18ac1f6a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:08.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:08.808+0000 7f18b0c7e640 1 --2- 172.21.15.38:0/2555007496 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f18ac0aade0 0x7f18ac1f8d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:08.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:08.808+0000 7f18b0c7e640 1 --2- 172.21.15.38:0/2555007496 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1898003a50 0x7f18ac1f8610 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:08.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:08.808+0000 7f18b0c7e640 1 -- 172.21.15.38:0/2555007496 >> 172.21.15.38:0/2555007496 conn(0x7f18ac0d4440 msgr2=0x7f18ac0d4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:08.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:08.808+0000 7f18b0c7e640 1 -- 172.21.15.38:0/2555007496 shutdown_connections 2024-09-21T03:39:08.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:08.808+0000 7f18b0c7e640 1 -- 172.21.15.38:0/2555007496 wait complete. 2024-09-21T03:39:08.821 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T03:39:08.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:08.937+0000 7fada87b0640 1 -- 172.21.15.38:0/911986258 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fad88004d40 msgr2=0x7fad88005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:08.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:08.937+0000 7fada87b0640 1 --2- 172.21.15.38:0/911986258 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fad88004d40 0x7fad88005150 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fad9800c470 tx=0x7fad88005830 comp rx=0 tx=0).stop 2024-09-21T03:39:08.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:08.937+0000 7fada6d4d640 1 -- 172.21.15.38:0/911986258 reap_dead start 2024-09-21T03:39:08.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:08.937+0000 7fada87b0640 1 -- 172.21.15.38:0/911986258 shutdown_connections 2024-09-21T03:39:08.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:08.937+0000 7fada87b0640 1 -- 172.21.15.38:0/911986258 >> 172.21.15.38:0/911986258 conn(0x7fada00d4620 msgr2=0x7fada00d4a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:08.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:08.937+0000 7fada87b0640 1 -- 172.21.15.38:0/911986258 shutdown_connections 2024-09-21T03:39:08.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:08.937+0000 7fada87b0640 1 -- 172.21.15.38:0/911986258 wait complete. 2024-09-21T03:39:08.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:08.939+0000 7fada87b0640 1 Processor -- start 2024-09-21T03:39:08.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:08.939+0000 7fada87b0640 1 -- start start 2024-09-21T03:39:08.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:08.940+0000 7fada87b0640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fad88004d40 0x7fada014da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:08.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:08.940+0000 7fada87b0640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fada015e750 0x7fada00d0c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:08.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:08.940+0000 7fada87b0640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fada00d1170 0x7fada01f94c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:08.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:08.940+0000 7fada87b0640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fada00b3c70 con 0x7fada015e750 2024-09-21T03:39:08.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:08.940+0000 7fada87b0640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fada00b3af0 con 0x7fada00d1170 2024-09-21T03:39:08.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:08.940+0000 7fada87b0640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fada00b3df0 con 0x7fad88004d40 2024-09-21T03:39:08.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:08.940+0000 7fada5d4b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fada015e750 0x7fada00d0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:08.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:08.940+0000 7fada5d4b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fada015e750 0x7fada00d0c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:52652/0 (socket says 172.21.15.38:52652) 2024-09-21T03:39:08.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:08.940+0000 7fada5d4b640 1 -- 172.21.15.38:0/1101841230 learned_addr learned my addr 172.21.15.38:0/1101841230 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:39:08.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:08.940+0000 7fada654c640 1 --2- 172.21.15.38:0/1101841230 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fad88004d40 0x7fada014da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:08.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:08.940+0000 7fada6d4d640 1 --2- 172.21.15.38:0/1101841230 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fada00d1170 0x7fada01f94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:08.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:08.940+0000 7fada5d4b640 1 -- 172.21.15.38:0/1101841230 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fad88004d40 msgr2=0x7fada014da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:08.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:08.940+0000 7fada5d4b640 1 --2- 172.21.15.38:0/1101841230 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fad88004d40 0x7fada014da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:08.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:08.940+0000 7fada5d4b640 1 -- 172.21.15.38:0/1101841230 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fada00d1170 msgr2=0x7fada01f94c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:08.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:08.940+0000 7fada5d4b640 1 --2- 172.21.15.38:0/1101841230 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fada00d1170 0x7fada01f94c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:08.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:08.940+0000 7fada5d4b640 1 -- 172.21.15.38:0/1101841230 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fada00d1580 con 0x7fada015e750 2024-09-21T03:39:08.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:08.940+0000 7fada654c640 1 --2- 172.21.15.38:0/1101841230 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fad88004d40 0x7fada014da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:39:08.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:08.940+0000 7fada6d4d640 1 --2- 172.21.15.38:0/1101841230 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fada00d1170 0x7fada01f94c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:39:08.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:08.941+0000 7fada5d4b640 1 --2- 172.21.15.38:0/1101841230 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fada015e750 0x7fada00d0c30 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fad94011ba0 tx=0x7fad9400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:08.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:08.941+0000 7fad9effd640 1 -- 172.21.15.38:0/1101841230 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad9401e070 con 0x7fada015e750 2024-09-21T03:39:08.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:08.941+0000 7fad9effd640 1 -- 172.21.15.38:0/1101841230 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fad9400e7c0 con 0x7fada015e750 2024-09-21T03:39:08.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:08.941+0000 7fad9effd640 1 -- 172.21.15.38:0/1101841230 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad9400ebd0 con 0x7fada015e750 2024-09-21T03:39:08.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:08.941+0000 7fada87b0640 1 -- 172.21.15.38:0/1101841230 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fada01515c0 con 0x7fada015e750 2024-09-21T03:39:08.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:08.941+0000 7fada87b0640 1 -- 172.21.15.38:0/1101841230 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fada0152c20 con 0x7fada015e750 2024-09-21T03:39:08.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:08.942+0000 7fada87b0640 1 -- 172.21.15.38:0/1101841230 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fada0162ba0 con 0x7fada015e750 2024-09-21T03:39:08.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:08.943+0000 7fad9effd640 1 -- 172.21.15.38:0/1101841230 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fad94021070 con 0x7fada015e750 2024-09-21T03:39:08.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:08.943+0000 7fad9effd640 1 --2- 172.21.15.38:0/1101841230 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fad8c078bc0 0x7fad8c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:08.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:08.943+0000 7fad9effd640 1 -- 172.21.15.38:0/1101841230 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 6101+0+0 (secure 0 0 0) 0x7fad9409bf10 con 0x7fada015e750 2024-09-21T03:39:08.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:08.943+0000 7fada654c640 1 --2- 172.21.15.38:0/1101841230 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fad8c078bc0 0x7fad8c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:08.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:08.944+0000 7fada654c640 1 --2- 172.21.15.38:0/1101841230 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fad8c078bc0 0x7fad8c07b0d0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fad90005fd0 tx=0x7fad900094a0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:08.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:08.948+0000 7fad9effd640 1 -- 172.21.15.38:0/1101841230 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fada00d1580 con 0x7fada015e750 2024-09-21T03:39:09.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:09.138+0000 7fada87b0640 1 -- 172.21.15.38:0/1101841230 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fada0151320 con 0x7fada015e750 2024-09-21T03:39:09.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:09.139+0000 7fad9effd640 1 -- 172.21.15.38:0/1101841230 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+16969 (secure 0 0 0) 0x7fad94063c00 con 0x7fada015e750 2024-09-21T03:39:09.141 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:39:09.142 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":82,"fsid":"9d50d66f-656b-4fe2-8e3b-fe53b79c4d43","created":"2024-09-21T03:33:10.677865+0000","modified":"2024-09-21T03:39:06.408093+0000","last_up_change":"2024-09-21T03:33:24.023421+0000","last_in_change":"2024-09-21T03:33:14.296193+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T03:33:22.992573+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-09-21T03:39:02.446050+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-09-21T03:39:03.739922+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dbab183f-7e63-48b8-960a-ae55fd5bf565","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6801","nonce":2381639433}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6803","nonce":2381639433}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6807","nonce":2381639433}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6805","nonce":2381639433}]},"public_addr":"172.21.15.38:6801/2381639433","cluster_addr":"172.21.15.38:6803/2381639433","heartbeat_back_addr":"172.21.15.38:6807/2381639433","heartbeat_front_addr":"172.21.15.38:6805/2381639433","state":["exists","up"]},{"osd":1,"uuid":"fa1c0e8a-ebc1-4359-8146-bd5342abbc16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6817","nonce":1491818732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6819","nonce":1491818732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6823","nonce":1491818732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6821","nonce":1491818732}]},"public_addr":"172.21.15.38:6817/1491818732","cluster_addr":"172.21.15.38:6819/1491818732","heartbeat_back_addr":"172.21.15.38:6823/1491818732","heartbeat_front_addr":"172.21.15.38:6821/1491818732","state":["exists","up"]},{"osd":2,"uuid":"2ed3cd2d-385d-4fc6-9869-214552a84803","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6825","nonce":2577858902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6827","nonce":2577858902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6831","nonce":2577858902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6829","nonce":2577858902}]},"public_addr":"172.21.15.38:6825/2577858902","cluster_addr":"172.21.15.38:6827/2577858902","heartbeat_back_addr":"172.21.15.38:6831/2577858902","heartbeat_front_addr":"172.21.15.38:6829/2577858902","state":["exists","up"]},{"osd":3,"uuid":"62d1d95e-919d-4250-be71-1d34cfa70dbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6809","nonce":1286810294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6811","nonce":1286810294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6815","nonce":1286810294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6813","nonce":1286810294}]},"public_addr":"172.21.15.38:6809/1286810294","cluster_addr":"172.21.15.38:6811/1286810294","heartbeat_back_addr":"172.21.15.38:6815/1286810294","heartbeat_front_addr":"172.21.15.38:6813/1286810294","state":["exists","up"]},{"osd":4,"uuid":"7c9f9314-99d7-4d73-8392-37809a9f54b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6803","nonce":3401837571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6805","nonce":3401837571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6809","nonce":3401837571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6807","nonce":3401837571}]},"public_addr":"172.21.15.195:6803/3401837571","cluster_addr":"172.21.15.195:6805/3401837571","heartbeat_back_addr":"172.21.15.195:6809/3401837571","heartbeat_front_addr":"172.21.15.195:6807/3401837571","state":["exists","up"]},{"osd":5,"uuid":"bde4fe9f-8414-4bba-bf4a-43973d1976bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6811","nonce":2080960180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6813","nonce":2080960180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6817","nonce":2080960180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6815","nonce":2080960180}]},"public_addr":"172.21.15.195:6811/2080960180","cluster_addr":"172.21.15.195:6813/2080960180","heartbeat_back_addr":"172.21.15.195:6817/2080960180","heartbeat_front_addr":"172.21.15.195:6815/2080960180","state":["exists","up"]},{"osd":6,"uuid":"6ee6d7f4-1a5f-4a4c-ac16-48fe05f6a002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6827","nonce":4055825675}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6829","nonce":4055825675}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6833","nonce":4055825675}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6831","nonce":4055825675}]},"public_addr":"172.21.15.195:6827/4055825675","cluster_addr":"172.21.15.195:6829/4055825675","heartbeat_back_addr":"172.21.15.195:6833/4055825675","heartbeat_front_addr":"172.21.15.195:6831/4055825675","state":["exists","up"]},{"osd":7,"uuid":"efeb5f51-329a-4484-8290-92e7d3503d05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6819","nonce":1606106342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6821","nonce":1606106342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6825","nonce":1606106342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6823","nonce":1606106342}]},"public_addr":"172.21.15.195:6819/1606106342","cluster_addr":"172.21.15.195:6821/1606106342","heartbeat_back_addr":"172.21.15.195:6825/1606106342","heartbeat_front_addr":"172.21.15.195:6823/1606106342","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.901343+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:21.115003+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.901623+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.835238+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.132199+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:19.817560+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.829690+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:21.087300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T03:39:09.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:09.142+0000 7fada87b0640 1 -- 172.21.15.38:0/1101841230 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fad8c078bc0 msgr2=0x7fad8c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:09.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:09.142+0000 7fada87b0640 1 --2- 172.21.15.38:0/1101841230 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fad8c078bc0 0x7fad8c07b0d0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fad90005fd0 tx=0x7fad900094a0 comp rx=0 tx=0).stop 2024-09-21T03:39:09.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:09.142+0000 7fada87b0640 1 -- 172.21.15.38:0/1101841230 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fada015e750 msgr2=0x7fada00d0c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:09.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:09.142+0000 7fada87b0640 1 --2- 172.21.15.38:0/1101841230 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fada015e750 0x7fada00d0c30 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fad94011ba0 tx=0x7fad9400f790 comp rx=0 tx=0).stop 2024-09-21T03:39:09.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:09.142+0000 7fada87b0640 1 -- 172.21.15.38:0/1101841230 shutdown_connections 2024-09-21T03:39:09.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:09.142+0000 7fada87b0640 1 --2- 172.21.15.38:0/1101841230 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fada00d1170 0x7fada01f94c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:09.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:09.142+0000 7fada87b0640 1 --2- 172.21.15.38:0/1101841230 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fada015e750 0x7fada00d0c30 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:09.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:09.142+0000 7fada87b0640 1 --2- 172.21.15.38:0/1101841230 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fad8c078bc0 0x7fad8c07b0d0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:09.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:09.142+0000 7fada87b0640 1 --2- 172.21.15.38:0/1101841230 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fad88004d40 0x7fada014da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:09.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:09.142+0000 7fada87b0640 1 -- 172.21.15.38:0/1101841230 >> 172.21.15.38:0/1101841230 conn(0x7fada00d4620 msgr2=0x7fada0155910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:09.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:09.142+0000 7fada87b0640 1 -- 172.21.15.38:0/1101841230 shutdown_connections 2024-09-21T03:39:09.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:09.142+0000 7fada87b0640 1 -- 172.21.15.38:0/1101841230 wait complete. 2024-09-21T03:39:09.155 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-21T03:39:09.156 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T03:39:09.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:09.270+0000 7fa486c84640 1 -- 172.21.15.38:0/2947161863 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa480162590 msgr2=0x7fa4801629a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:09.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:09.270+0000 7fa486c84640 1 --2- 172.21.15.38:0/2947161863 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa480162590 0x7fa4801629a0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fa4800b5a40 tx=0x7fa47002aac0 comp rx=0 tx=0).stop 2024-09-21T03:39:09.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:09.270+0000 7fa486c84640 1 -- 172.21.15.38:0/2947161863 shutdown_connections 2024-09-21T03:39:09.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:09.270+0000 7fa486c84640 1 --2- 172.21.15.38:0/2947161863 >> v2:172.21.15.195:3300/0 conn(0x7fa4800b2e60 0x7fa4800b3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:09.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:09.270+0000 7fa486c84640 1 --2- 172.21.15.38:0/2947161863 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa480162590 0x7fa4801629a0 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:09.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:09.270+0000 7fa486c84640 1 -- 172.21.15.38:0/2947161863 >> 172.21.15.38:0/2947161863 conn(0x7fa4800d4440 msgr2=0x7fa4800d4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:09.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:09.270+0000 7fa486c84640 1 -- 172.21.15.38:0/2947161863 shutdown_connections 2024-09-21T03:39:09.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:09.270+0000 7fa486c84640 1 -- 172.21.15.38:0/2947161863 wait complete. 2024-09-21T03:39:09.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:09.272+0000 7fa486c84640 1 Processor -- start 2024-09-21T03:39:09.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:09.272+0000 7fa486c84640 1 -- start start 2024-09-21T03:39:09.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:09.274+0000 7fa486c84640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa4800b2e60 0x7fa48015e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:09.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:09.274+0000 7fa486c84640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa4800cfe00 0x7fa4801478c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:09.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:09.274+0000 7fa486c84640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa480162590 0x7fa480147e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:09.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:09.274+0000 7fa486c84640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa4800b3930 con 0x7fa4800b2e60 2024-09-21T03:39:09.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:09.274+0000 7fa486c84640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fa4800b37b0 con 0x7fa4800cfe00 2024-09-21T03:39:09.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:09.274+0000 7fa486c84640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fa4800b3ab0 con 0x7fa480162590 2024-09-21T03:39:09.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:09.274+0000 7fa484a20640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa4800b2e60 0x7fa48015e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:09.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:09.274+0000 7fa484a20640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa4800b2e60 0x7fa48015e0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:52680/0 (socket says 172.21.15.38:52680) 2024-09-21T03:39:09.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:09.274+0000 7fa484a20640 1 -- 172.21.15.38:0/131287912 learned_addr learned my addr 172.21.15.38:0/131287912 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:39:09.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:09.274+0000 7fa485221640 1 --2- 172.21.15.38:0/131287912 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa480162590 0x7fa480147e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:09.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:09.274+0000 7fa47ffff640 1 --2- 172.21.15.38:0/131287912 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa4800cfe00 0x7fa4801478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:09.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:09.274+0000 7fa484a20640 1 -- 172.21.15.38:0/131287912 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa480162590 msgr2=0x7fa480147e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:09.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:09.274+0000 7fa484a20640 1 --2- 172.21.15.38:0/131287912 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa480162590 0x7fa480147e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:09.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:09.274+0000 7fa484a20640 1 -- 172.21.15.38:0/131287912 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa4800cfe00 msgr2=0x7fa4801478c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:09.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:09.274+0000 7fa484a20640 1 --2- 172.21.15.38:0/131287912 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa4800cfe00 0x7fa4801478c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:09.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:09.274+0000 7fa484a20640 1 -- 172.21.15.38:0/131287912 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4800c87c0 con 0x7fa4800b2e60 2024-09-21T03:39:09.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:09.274+0000 7fa484a20640 1 --2- 172.21.15.38:0/131287912 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa4800b2e60 0x7fa48015e0e0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fa46800c3a0 tx=0x7fa46800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:09.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:09.275+0000 7fa47d7fa640 1 -- 172.21.15.38:0/131287912 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa468017070 con 0x7fa4800b2e60 2024-09-21T03:39:09.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:09.275+0000 7fa47d7fa640 1 -- 172.21.15.38:0/131287912 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa4680053c0 con 0x7fa4800b2e60 2024-09-21T03:39:09.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:09.275+0000 7fa47d7fa640 1 -- 172.21.15.38:0/131287912 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa468003850 con 0x7fa4800b2e60 2024-09-21T03:39:09.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:09.275+0000 7fa486c84640 1 -- 172.21.15.38:0/131287912 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4800c84e0 con 0x7fa4800b2e60 2024-09-21T03:39:09.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:09.275+0000 7fa486c84640 1 -- 172.21.15.38:0/131287912 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa48015a460 con 0x7fa4800b2e60 2024-09-21T03:39:09.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:09.276+0000 7fa486c84640 1 -- 172.21.15.38:0/131287912 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4800b3270 con 0x7fa4800b2e60 2024-09-21T03:39:09.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:09.276+0000 7fa47d7fa640 1 -- 172.21.15.38:0/131287912 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa46801a070 con 0x7fa4800b2e60 2024-09-21T03:39:09.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:09.277+0000 7fa47d7fa640 1 --2- 172.21.15.38:0/131287912 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa460078bc0 0x7fa46007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:09.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:09.277+0000 7fa47d7fa640 1 -- 172.21.15.38:0/131287912 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 6101+0+0 (secure 0 0 0) 0x7fa46809a030 con 0x7fa4800b2e60 2024-09-21T03:39:09.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:09.277+0000 7fa47ffff640 1 --2- 172.21.15.38:0/131287912 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa460078bc0 0x7fa46007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:09.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:09.278+0000 7fa47ffff640 1 --2- 172.21.15.38:0/131287912 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa460078bc0 0x7fa46007b0d0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fa470035000 tx=0x7fa470038040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:09.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:09.282+0000 7fa47d7fa640 1 -- 172.21.15.38:0/131287912 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fa480162ee0 con 0x7fa4800b2e60 2024-09-21T03:39:09.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:09.467+0000 7fa486c84640 1 -- 172.21.15.38:0/131287912 --> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa480162a40 con 0x7fa460078bc0 2024-09-21T03:39:09.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:09.472+0000 7fa47d7fa640 1 -- 172.21.15.38:0/131287912 <== mgr.4117 v2:172.21.15.195:6800/1631901344 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251482 (secure 0 0 0) 0x7fa480162a40 con 0x7fa460078bc0 2024-09-21T03:39:09.476 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:39:09.478 INFO:teuthology.orchestra.run.smithi038.stderr:dumped all 2024-09-21T03:39:09.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:09.475+0000 7fa486c84640 1 -- 172.21.15.38:0/131287912 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa460078bc0 msgr2=0x7fa46007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:09.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:09.475+0000 7fa486c84640 1 --2- 172.21.15.38:0/131287912 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa460078bc0 0x7fa46007b0d0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fa470035000 tx=0x7fa470038040 comp rx=0 tx=0).stop 2024-09-21T03:39:09.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:09.475+0000 7fa486c84640 1 -- 172.21.15.38:0/131287912 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa4800b2e60 msgr2=0x7fa48015e0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:09.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:09.475+0000 7fa486c84640 1 --2- 172.21.15.38:0/131287912 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa4800b2e60 0x7fa48015e0e0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fa46800c3a0 tx=0x7fa46800c860 comp rx=0 tx=0).stop 2024-09-21T03:39:09.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:09.476+0000 7fa486c84640 1 -- 172.21.15.38:0/131287912 shutdown_connections 2024-09-21T03:39:09.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:09.476+0000 7fa486c84640 1 --2- 172.21.15.38:0/131287912 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa460078bc0 0x7fa46007b0d0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:09.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:09.476+0000 7fa486c84640 1 --2- 172.21.15.38:0/131287912 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa480162590 0x7fa480147e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:09.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:09.476+0000 7fa486c84640 1 --2- 172.21.15.38:0/131287912 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa4800cfe00 0x7fa4801478c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:09.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:09.476+0000 7fa486c84640 1 --2- 172.21.15.38:0/131287912 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa4800b2e60 0x7fa48015e0e0 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:09.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:09.476+0000 7fa486c84640 1 -- 172.21.15.38:0/131287912 >> 172.21.15.38:0/131287912 conn(0x7fa4800d4440 msgr2=0x7fa4800d4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:09.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:09.476+0000 7fa486c84640 1 -- 172.21.15.38:0/131287912 shutdown_connections 2024-09-21T03:39:09.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:09.476+0000 7fa486c84640 1 -- 172.21.15.38:0/131287912 wait complete. 2024-09-21T03:39:09.489 INFO:teuthology.orchestra.run.smithi038.stdout:{"pg_ready":true,"pg_map":{"version":248,"stamp":"2024-09-21T03:39:09.071943+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592744,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":137,"num_write_kb":1635,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":105,"ondisk_log_size":105,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":154,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":275136,"kb_used_data":8960,"kb_used_omap":12,"kb_used_meta":265971,"kb_avail":754699584,"statfs":{"total":773094113280,"available":772812374016,"internally_reserved":0,"allocated":9175040,"data_stored":4848499,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12748,"internal_metadata":272354868},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2376,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":36,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.687763"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-09-21T03:39:04.404050+0000","last_change":"2024-09-21T03:39:04.404050+0000","last_active":"2024-09-21T03:39:04.383211+0000","last_peered":"2024-09-21T03:39:04.383211+0000","last_clean":"2024-09-21T03:39:04.383211+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:39:04.404050+0000","last_undegraded":"2024-09-21T03:39:04.404050+0000","last_fullsized":"2024-09-21T03:39:04.404050+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.419248+0000","last_change":"2024-09-21T03:39:05.431615+0000","last_active":"2024-09-21T03:39:06.419248+0000","last_peered":"2024-09-21T03:39:06.419248+0000","last_clean":"2024-09-21T03:39:06.419248+0000","last_became_active":"2024-09-21T03:39:05.429700+0000","last_became_peered":"2024-09-21T03:39:05.429700+0000","last_unstale":"2024-09-21T03:39:06.419248+0000","last_undegraded":"2024-09-21T03:39:06.419248+0000","last_fullsized":"2024-09-21T03:39:06.419248+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:22:26.967805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.419361+0000","last_change":"2024-09-21T03:39:05.434451+0000","last_active":"2024-09-21T03:39:06.419361+0000","last_peered":"2024-09-21T03:39:06.419361+0000","last_clean":"2024-09-21T03:39:06.419361+0000","last_became_active":"2024-09-21T03:39:05.432952+0000","last_became_peered":"2024-09-21T03:39:05.432952+0000","last_unstale":"2024-09-21T03:39:06.419361+0000","last_undegraded":"2024-09-21T03:39:06.419361+0000","last_fullsized":"2024-09-21T03:39:06.419361+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:30:48.915097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.569594+0000","last_change":"2024-09-21T03:39:05.426032+0000","last_active":"2024-09-21T03:39:06.569594+0000","last_peered":"2024-09-21T03:39:06.569594+0000","last_clean":"2024-09-21T03:39:06.569594+0000","last_became_active":"2024-09-21T03:39:05.425731+0000","last_became_peered":"2024-09-21T03:39:05.425731+0000","last_unstale":"2024-09-21T03:39:06.569594+0000","last_undegraded":"2024-09-21T03:39:06.569594+0000","last_fullsized":"2024-09-21T03:39:06.569594+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:01:01.380435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.570162+0000","last_change":"2024-09-21T03:39:05.428805+0000","last_active":"2024-09-21T03:39:06.570162+0000","last_peered":"2024-09-21T03:39:06.570162+0000","last_clean":"2024-09-21T03:39:06.570162+0000","last_became_active":"2024-09-21T03:39:05.428394+0000","last_became_peered":"2024-09-21T03:39:05.428394+0000","last_unstale":"2024-09-21T03:39:06.570162+0000","last_undegraded":"2024-09-21T03:39:06.570162+0000","last_fullsized":"2024-09-21T03:39:06.570162+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:57:28.504333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.415502+0000","last_change":"2024-09-21T03:39:05.418164+0000","last_active":"2024-09-21T03:39:06.415502+0000","last_peered":"2024-09-21T03:39:06.415502+0000","last_clean":"2024-09-21T03:39:06.415502+0000","last_became_active":"2024-09-21T03:39:05.416861+0000","last_became_peered":"2024-09-21T03:39:05.416861+0000","last_unstale":"2024-09-21T03:39:06.415502+0000","last_undegraded":"2024-09-21T03:39:06.415502+0000","last_fullsized":"2024-09-21T03:39:06.415502+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:06:43.522600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.569829+0000","last_change":"2024-09-21T03:39:05.429537+0000","last_active":"2024-09-21T03:39:06.569829+0000","last_peered":"2024-09-21T03:39:06.569829+0000","last_clean":"2024-09-21T03:39:06.569829+0000","last_became_active":"2024-09-21T03:39:05.428077+0000","last_became_peered":"2024-09-21T03:39:05.428077+0000","last_unstale":"2024-09-21T03:39:06.569829+0000","last_undegraded":"2024-09-21T03:39:06.569829+0000","last_fullsized":"2024-09-21T03:39:06.569829+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:38:28.792089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.414574+0000","last_change":"2024-09-21T03:39:05.420868+0000","last_active":"2024-09-21T03:39:06.414574+0000","last_peered":"2024-09-21T03:39:06.414574+0000","last_clean":"2024-09-21T03:39:06.414574+0000","last_became_active":"2024-09-21T03:39:05.420546+0000","last_became_peered":"2024-09-21T03:39:05.420546+0000","last_unstale":"2024-09-21T03:39:06.414574+0000","last_undegraded":"2024-09-21T03:39:06.414574+0000","last_fullsized":"2024-09-21T03:39:06.414574+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:00:25.925784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.570276+0000","last_change":"2024-09-21T03:39:05.424217+0000","last_active":"2024-09-21T03:39:06.570276+0000","last_peered":"2024-09-21T03:39:06.570276+0000","last_clean":"2024-09-21T03:39:06.570276+0000","last_became_active":"2024-09-21T03:39:05.423922+0000","last_became_peered":"2024-09-21T03:39:05.423922+0000","last_unstale":"2024-09-21T03:39:06.570276+0000","last_undegraded":"2024-09-21T03:39:06.570276+0000","last_fullsized":"2024-09-21T03:39:06.570276+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:30:48.915097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-09-21T03:39:04.403441+0000","last_change":"2024-09-21T03:39:04.403441+0000","last_active":"2024-09-21T03:39:04.383211+0000","last_peered":"2024-09-21T03:39:04.383211+0000","last_clean":"2024-09-21T03:39:04.383211+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:39:04.403441+0000","last_undegraded":"2024-09-21T03:39:04.403441+0000","last_fullsized":"2024-09-21T03:39:04.403441+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.419460+0000","last_change":"2024-09-21T03:39:05.427718+0000","last_active":"2024-09-21T03:39:06.419460+0000","last_peered":"2024-09-21T03:39:06.419460+0000","last_clean":"2024-09-21T03:39:06.419460+0000","last_became_active":"2024-09-21T03:39:05.427387+0000","last_became_peered":"2024-09-21T03:39:05.427387+0000","last_unstale":"2024-09-21T03:39:06.419460+0000","last_undegraded":"2024-09-21T03:39:06.419460+0000","last_fullsized":"2024-09-21T03:39:06.419460+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:58:09.229781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.570387+0000","last_change":"2024-09-21T03:39:05.421453+0000","last_active":"2024-09-21T03:39:06.570387+0000","last_peered":"2024-09-21T03:39:06.570387+0000","last_clean":"2024-09-21T03:39:06.570387+0000","last_became_active":"2024-09-21T03:39:05.421133+0000","last_became_peered":"2024-09-21T03:39:05.421133+0000","last_unstale":"2024-09-21T03:39:06.570387+0000","last_undegraded":"2024-09-21T03:39:06.570387+0000","last_fullsized":"2024-09-21T03:39:06.570387+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:30:07.399497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.418721+0000","last_change":"2024-09-21T03:39:05.426022+0000","last_active":"2024-09-21T03:39:06.418721+0000","last_peered":"2024-09-21T03:39:06.418721+0000","last_clean":"2024-09-21T03:39:06.418721+0000","last_became_active":"2024-09-21T03:39:05.425625+0000","last_became_peered":"2024-09-21T03:39:05.425625+0000","last_unstale":"2024-09-21T03:39:06.418721+0000","last_undegraded":"2024-09-21T03:39:06.418721+0000","last_fullsized":"2024-09-21T03:39:06.418721+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:03:54.230793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.414472+0000","last_change":"2024-09-21T03:39:05.421310+0000","last_active":"2024-09-21T03:39:06.414472+0000","last_peered":"2024-09-21T03:39:06.414472+0000","last_clean":"2024-09-21T03:39:06.414472+0000","last_became_active":"2024-09-21T03:39:05.420314+0000","last_became_peered":"2024-09-21T03:39:05.420314+0000","last_unstale":"2024-09-21T03:39:06.414472+0000","last_undegraded":"2024-09-21T03:39:06.414472+0000","last_fullsized":"2024-09-21T03:39:06.414472+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:00:07.867021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.570061+0000","last_change":"2024-09-21T03:39:05.427360+0000","last_active":"2024-09-21T03:39:06.570061+0000","last_peered":"2024-09-21T03:39:06.570061+0000","last_clean":"2024-09-21T03:39:06.570061+0000","last_became_active":"2024-09-21T03:39:05.427009+0000","last_became_peered":"2024-09-21T03:39:05.427009+0000","last_unstale":"2024-09-21T03:39:06.570061+0000","last_undegraded":"2024-09-21T03:39:06.570061+0000","last_fullsized":"2024-09-21T03:39:06.570061+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:54:27.965337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.569046+0000","last_change":"2024-09-21T03:39:05.430747+0000","last_active":"2024-09-21T03:39:06.569046+0000","last_peered":"2024-09-21T03:39:06.569046+0000","last_clean":"2024-09-21T03:39:06.569046+0000","last_became_active":"2024-09-21T03:39:05.429872+0000","last_became_peered":"2024-09-21T03:39:05.429872+0000","last_unstale":"2024-09-21T03:39:06.569046+0000","last_undegraded":"2024-09-21T03:39:06.569046+0000","last_fullsized":"2024-09-21T03:39:06.569046+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:01:18.107849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.415027+0000","last_change":"2024-09-21T03:39:05.418812+0000","last_active":"2024-09-21T03:39:06.415027+0000","last_peered":"2024-09-21T03:39:06.415027+0000","last_clean":"2024-09-21T03:39:06.415027+0000","last_became_active":"2024-09-21T03:39:05.418487+0000","last_became_peered":"2024-09-21T03:39:05.418487+0000","last_unstale":"2024-09-21T03:39:06.415027+0000","last_undegraded":"2024-09-21T03:39:06.415027+0000","last_fullsized":"2024-09-21T03:39:06.415027+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:37:29.087437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.419123+0000","last_change":"2024-09-21T03:39:05.428238+0000","last_active":"2024-09-21T03:39:06.419123+0000","last_peered":"2024-09-21T03:39:06.419123+0000","last_clean":"2024-09-21T03:39:06.419123+0000","last_became_active":"2024-09-21T03:39:05.427915+0000","last_became_peered":"2024-09-21T03:39:05.427915+0000","last_unstale":"2024-09-21T03:39:06.419123+0000","last_undegraded":"2024-09-21T03:39:06.419123+0000","last_fullsized":"2024-09-21T03:39:06.419123+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:22:48.194041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.414304+0000","last_change":"2024-09-21T03:39:05.412642+0000","last_active":"2024-09-21T03:39:06.414304+0000","last_peered":"2024-09-21T03:39:06.414304+0000","last_clean":"2024-09-21T03:39:06.414304+0000","last_became_active":"2024-09-21T03:39:05.412274+0000","last_became_peered":"2024-09-21T03:39:05.412274+0000","last_unstale":"2024-09-21T03:39:06.414304+0000","last_undegraded":"2024-09-21T03:39:06.414304+0000","last_fullsized":"2024-09-21T03:39:06.414304+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:24:30.153279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.415023+0000","last_change":"2024-09-21T03:39:05.428953+0000","last_active":"2024-09-21T03:39:06.415023+0000","last_peered":"2024-09-21T03:39:06.415023+0000","last_clean":"2024-09-21T03:39:06.415023+0000","last_became_active":"2024-09-21T03:39:05.428223+0000","last_became_peered":"2024-09-21T03:39:05.428223+0000","last_unstale":"2024-09-21T03:39:06.415023+0000","last_undegraded":"2024-09-21T03:39:06.415023+0000","last_fullsized":"2024-09-21T03:39:06.415023+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:16:38.764525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.570578+0000","last_change":"2024-09-21T03:39:05.409645+0000","last_active":"2024-09-21T03:39:06.570578+0000","last_peered":"2024-09-21T03:39:06.570578+0000","last_clean":"2024-09-21T03:39:06.570578+0000","last_became_active":"2024-09-21T03:39:05.409315+0000","last_became_peered":"2024-09-21T03:39:05.409315+0000","last_unstale":"2024-09-21T03:39:06.570578+0000","last_undegraded":"2024-09-21T03:39:06.570578+0000","last_fullsized":"2024-09-21T03:39:06.570578+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:49:12.163503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.418882+0000","last_change":"2024-09-21T03:39:05.427679+0000","last_active":"2024-09-21T03:39:06.418882+0000","last_peered":"2024-09-21T03:39:06.418882+0000","last_clean":"2024-09-21T03:39:06.418882+0000","last_became_active":"2024-09-21T03:39:05.426044+0000","last_became_peered":"2024-09-21T03:39:05.426044+0000","last_unstale":"2024-09-21T03:39:06.418882+0000","last_undegraded":"2024-09-21T03:39:06.418882+0000","last_fullsized":"2024-09-21T03:39:06.418882+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:20:35.883679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-09-21T03:39:04.402899+0000","last_change":"2024-09-21T03:39:04.402899+0000","last_active":"2024-09-21T03:39:04.383211+0000","last_peered":"2024-09-21T03:39:04.383211+0000","last_clean":"2024-09-21T03:39:04.383211+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:39:04.402899+0000","last_undegraded":"2024-09-21T03:39:04.402899+0000","last_fullsized":"2024-09-21T03:39:04.402899+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.418484+0000","last_change":"2024-09-21T03:39:05.427956+0000","last_active":"2024-09-21T03:39:06.418484+0000","last_peered":"2024-09-21T03:39:06.418484+0000","last_clean":"2024-09-21T03:39:06.418484+0000","last_became_active":"2024-09-21T03:39:05.425708+0000","last_became_peered":"2024-09-21T03:39:05.425708+0000","last_unstale":"2024-09-21T03:39:06.418484+0000","last_undegraded":"2024-09-21T03:39:06.418484+0000","last_fullsized":"2024-09-21T03:39:06.418484+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:14:22.373008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.415419+0000","last_change":"2024-09-21T03:39:05.420911+0000","last_active":"2024-09-21T03:39:06.415419+0000","last_peered":"2024-09-21T03:39:06.415419+0000","last_clean":"2024-09-21T03:39:06.415419+0000","last_became_active":"2024-09-21T03:39:05.420587+0000","last_became_peered":"2024-09-21T03:39:05.420587+0000","last_unstale":"2024-09-21T03:39:06.415419+0000","last_undegraded":"2024-09-21T03:39:06.415419+0000","last_fullsized":"2024-09-21T03:39:06.415419+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:37:32.783221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.569302+0000","last_change":"2024-09-21T03:39:05.426555+0000","last_active":"2024-09-21T03:39:06.569302+0000","last_peered":"2024-09-21T03:39:06.569302+0000","last_clean":"2024-09-21T03:39:06.569302+0000","last_became_active":"2024-09-21T03:39:05.426265+0000","last_became_peered":"2024-09-21T03:39:05.426265+0000","last_unstale":"2024-09-21T03:39:06.569302+0000","last_undegraded":"2024-09-21T03:39:06.569302+0000","last_fullsized":"2024-09-21T03:39:06.569302+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:03:54.230793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.569945+0000","last_change":"2024-09-21T03:39:05.427771+0000","last_active":"2024-09-21T03:39:06.569945+0000","last_peered":"2024-09-21T03:39:06.569945+0000","last_clean":"2024-09-21T03:39:06.569945+0000","last_became_active":"2024-09-21T03:39:05.427310+0000","last_became_peered":"2024-09-21T03:39:05.427310+0000","last_unstale":"2024-09-21T03:39:06.569945+0000","last_undegraded":"2024-09-21T03:39:06.569945+0000","last_fullsized":"2024-09-21T03:39:06.569945+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:22:48.194041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.570546+0000","last_change":"2024-09-21T03:39:05.431170+0000","last_active":"2024-09-21T03:39:06.570546+0000","last_peered":"2024-09-21T03:39:06.570546+0000","last_clean":"2024-09-21T03:39:06.570546+0000","last_became_active":"2024-09-21T03:39:05.430855+0000","last_became_peered":"2024-09-21T03:39:05.430855+0000","last_unstale":"2024-09-21T03:39:06.570546+0000","last_undegraded":"2024-09-21T03:39:06.570546+0000","last_fullsized":"2024-09-21T03:39:06.570546+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:13:27.854189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.414268+0000","last_change":"2024-09-21T03:39:05.429739+0000","last_active":"2024-09-21T03:39:06.414268+0000","last_peered":"2024-09-21T03:39:06.414268+0000","last_clean":"2024-09-21T03:39:06.414268+0000","last_became_active":"2024-09-21T03:39:05.429401+0000","last_became_peered":"2024-09-21T03:39:05.429401+0000","last_unstale":"2024-09-21T03:39:06.414268+0000","last_undegraded":"2024-09-21T03:39:06.414268+0000","last_fullsized":"2024-09-21T03:39:06.414268+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:29:58.646678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-09-21T03:39:04.402307+0000","last_change":"2024-09-21T03:39:04.402307+0000","last_active":"2024-09-21T03:39:04.383211+0000","last_peered":"2024-09-21T03:39:04.383211+0000","last_clean":"2024-09-21T03:39:04.383211+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:39:04.402307+0000","last_undegraded":"2024-09-21T03:39:04.402307+0000","last_fullsized":"2024-09-21T03:39:04.402307+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.418906+0000","last_change":"2024-09-21T03:39:05.430619+0000","last_active":"2024-09-21T03:39:06.418906+0000","last_peered":"2024-09-21T03:39:06.418906+0000","last_clean":"2024-09-21T03:39:06.418906+0000","last_became_active":"2024-09-21T03:39:05.429959+0000","last_became_peered":"2024-09-21T03:39:05.429959+0000","last_unstale":"2024-09-21T03:39:06.418906+0000","last_undegraded":"2024-09-21T03:39:06.418906+0000","last_fullsized":"2024-09-21T03:39:06.418906+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:26:04.045027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.569279+0000","last_change":"2024-09-21T03:39:05.422391+0000","last_active":"2024-09-21T03:39:06.569279+0000","last_peered":"2024-09-21T03:39:06.569279+0000","last_clean":"2024-09-21T03:39:06.569279+0000","last_became_active":"2024-09-21T03:39:05.422073+0000","last_became_peered":"2024-09-21T03:39:05.422073+0000","last_unstale":"2024-09-21T03:39:06.569279+0000","last_undegraded":"2024-09-21T03:39:06.569279+0000","last_fullsized":"2024-09-21T03:39:06.569279+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:14:22.373008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.570338+0000","last_change":"2024-09-21T03:39:05.430675+0000","last_active":"2024-09-21T03:39:06.570338+0000","last_peered":"2024-09-21T03:39:06.570338+0000","last_clean":"2024-09-21T03:39:06.570338+0000","last_became_active":"2024-09-21T03:39:05.430203+0000","last_became_peered":"2024-09-21T03:39:05.430203+0000","last_unstale":"2024-09-21T03:39:06.570338+0000","last_undegraded":"2024-09-21T03:39:06.570338+0000","last_fullsized":"2024-09-21T03:39:06.570338+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:27:11.780682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"81'4","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.570214+0000","last_change":"2024-09-21T03:39:04.406626+0000","last_active":"2024-09-21T03:39:06.570214+0000","last_peered":"2024-09-21T03:39:06.570214+0000","last_clean":"2024-09-21T03:39:06.570214+0000","last_became_active":"2024-09-21T03:39:04.406318+0000","last_became_peered":"2024-09-21T03:39:04.406318+0000","last_unstale":"2024-09-21T03:39:06.570214+0000","last_undegraded":"2024-09-21T03:39:06.570214+0000","last_fullsized":"2024-09-21T03:39:06.570214+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_clean_scrub_stamp":"2024-09-21T03:39:03.377315+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:00:13.859613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.569891+0000","last_change":"2024-09-21T03:39:05.428305+0000","last_active":"2024-09-21T03:39:06.569891+0000","last_peered":"2024-09-21T03:39:06.569891+0000","last_clean":"2024-09-21T03:39:06.569891+0000","last_became_active":"2024-09-21T03:39:05.427992+0000","last_became_peered":"2024-09-21T03:39:05.427992+0000","last_unstale":"2024-09-21T03:39:06.569891+0000","last_undegraded":"2024-09-21T03:39:06.569891+0000","last_fullsized":"2024-09-21T03:39:06.569891+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:15:23.391245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1e","version":"81'1","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.570008+0000","last_change":"2024-09-21T03:39:04.411255+0000","last_active":"2024-09-21T03:39:06.570008+0000","last_peered":"2024-09-21T03:39:06.570008+0000","last_clean":"2024-09-21T03:39:06.570008+0000","last_became_active":"2024-09-21T03:39:04.410962+0000","last_became_peered":"2024-09-21T03:39:04.410962+0000","last_unstale":"2024-09-21T03:39:06.570008+0000","last_undegraded":"2024-09-21T03:39:06.570008+0000","last_fullsized":"2024-09-21T03:39:06.570008+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_clean_scrub_stamp":"2024-09-21T03:39:03.377315+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:27:36.848833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.569439+0000","last_change":"2024-09-21T03:39:05.415000+0000","last_active":"2024-09-21T03:39:06.569439+0000","last_peered":"2024-09-21T03:39:06.569439+0000","last_clean":"2024-09-21T03:39:06.569439+0000","last_became_active":"2024-09-21T03:39:05.414695+0000","last_became_peered":"2024-09-21T03:39:05.414695+0000","last_unstale":"2024-09-21T03:39:06.569439+0000","last_undegraded":"2024-09-21T03:39:06.569439+0000","last_fullsized":"2024-09-21T03:39:06.569439+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:58:09.229781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T03:39:04.413513+0000","last_change":"2024-09-21T03:39:04.404636+0000","last_active":"2024-09-21T03:39:04.413513+0000","last_peered":"2024-09-21T03:39:04.413513+0000","last_clean":"2024-09-21T03:39:04.413513+0000","last_became_active":"2024-09-21T03:39:04.404270+0000","last_became_peered":"2024-09-21T03:39:04.404270+0000","last_unstale":"2024-09-21T03:39:04.413513+0000","last_undegraded":"2024-09-21T03:39:04.413513+0000","last_fullsized":"2024-09-21T03:39:04.413513+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_clean_scrub_stamp":"2024-09-21T03:39:03.377315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:09:17.529140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.569256+0000","last_change":"2024-09-21T03:39:05.425119+0000","last_active":"2024-09-21T03:39:06.569256+0000","last_peered":"2024-09-21T03:39:06.569256+0000","last_clean":"2024-09-21T03:39:06.569256+0000","last_became_active":"2024-09-21T03:39:05.424804+0000","last_became_peered":"2024-09-21T03:39:05.424804+0000","last_unstale":"2024-09-21T03:39:06.569256+0000","last_undegraded":"2024-09-21T03:39:06.569256+0000","last_fullsized":"2024-09-21T03:39:06.569256+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:13:42.059728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"81'2","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.570641+0000","last_change":"2024-09-21T03:39:04.407381+0000","last_active":"2024-09-21T03:39:06.570641+0000","last_peered":"2024-09-21T03:39:06.570641+0000","last_clean":"2024-09-21T03:39:06.570641+0000","last_became_active":"2024-09-21T03:39:04.407044+0000","last_became_peered":"2024-09-21T03:39:04.407044+0000","last_unstale":"2024-09-21T03:39:06.570641+0000","last_undegraded":"2024-09-21T03:39:06.570641+0000","last_fullsized":"2024-09-21T03:39:06.570641+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_clean_scrub_stamp":"2024-09-21T03:39:03.377315+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:14:21.367112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.415145+0000","last_change":"2024-09-21T03:39:05.425026+0000","last_active":"2024-09-21T03:39:06.415145+0000","last_peered":"2024-09-21T03:39:06.415145+0000","last_clean":"2024-09-21T03:39:06.415145+0000","last_became_active":"2024-09-21T03:39:05.424667+0000","last_became_peered":"2024-09-21T03:39:05.424667+0000","last_unstale":"2024-09-21T03:39:06.415145+0000","last_undegraded":"2024-09-21T03:39:06.415145+0000","last_fullsized":"2024-09-21T03:39:06.415145+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:46:01.041621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.1b","version":"81'1","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.570542+0000","last_change":"2024-09-21T03:39:04.414608+0000","last_active":"2024-09-21T03:39:06.570542+0000","last_peered":"2024-09-21T03:39:06.570542+0000","last_clean":"2024-09-21T03:39:06.570542+0000","last_became_active":"2024-09-21T03:39:04.414274+0000","last_became_peered":"2024-09-21T03:39:04.414274+0000","last_unstale":"2024-09-21T03:39:06.570542+0000","last_undegraded":"2024-09-21T03:39:06.570542+0000","last_fullsized":"2024-09-21T03:39:06.570542+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_clean_scrub_stamp":"2024-09-21T03:39:03.377315+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:13:41.053832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.419607+0000","last_change":"2024-09-21T03:39:05.434557+0000","last_active":"2024-09-21T03:39:06.419607+0000","last_peered":"2024-09-21T03:39:06.419607+0000","last_clean":"2024-09-21T03:39:06.419607+0000","last_became_active":"2024-09-21T03:39:05.430126+0000","last_became_peered":"2024-09-21T03:39:05.430126+0000","last_unstale":"2024-09-21T03:39:06.419607+0000","last_undegraded":"2024-09-21T03:39:06.419607+0000","last_fullsized":"2024-09-21T03:39:06.419607+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:27:34.908887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"81'1","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.570010+0000","last_change":"2024-09-21T03:39:04.397543+0000","last_active":"2024-09-21T03:39:06.570010+0000","last_peered":"2024-09-21T03:39:06.570010+0000","last_clean":"2024-09-21T03:39:06.570010+0000","last_became_active":"2024-09-21T03:39:04.397220+0000","last_became_peered":"2024-09-21T03:39:04.397220+0000","last_unstale":"2024-09-21T03:39:06.570010+0000","last_undegraded":"2024-09-21T03:39:06.570010+0000","last_fullsized":"2024-09-21T03:39:06.570010+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_clean_scrub_stamp":"2024-09-21T03:39:03.377315+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:21:47.012786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.570317+0000","last_change":"2024-09-21T03:39:05.413837+0000","last_active":"2024-09-21T03:39:06.570317+0000","last_peered":"2024-09-21T03:39:06.570317+0000","last_clean":"2024-09-21T03:39:06.570317+0000","last_became_active":"2024-09-21T03:39:05.413498+0000","last_became_peered":"2024-09-21T03:39:05.413498+0000","last_unstale":"2024-09-21T03:39:06.570317+0000","last_undegraded":"2024-09-21T03:39:06.570317+0000","last_fullsized":"2024-09-21T03:39:06.570317+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:33:39.426980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.413804+0000","last_change":"2024-09-21T03:39:04.408211+0000","last_active":"2024-09-21T03:39:06.413804+0000","last_peered":"2024-09-21T03:39:06.413804+0000","last_clean":"2024-09-21T03:39:06.413804+0000","last_became_active":"2024-09-21T03:39:04.404927+0000","last_became_peered":"2024-09-21T03:39:04.404927+0000","last_unstale":"2024-09-21T03:39:06.413804+0000","last_undegraded":"2024-09-21T03:39:06.413804+0000","last_fullsized":"2024-09-21T03:39:06.413804+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_clean_scrub_stamp":"2024-09-21T03:39:03.377315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:53:29.829387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.569462+0000","last_change":"2024-09-21T03:39:05.419988+0000","last_active":"2024-09-21T03:39:06.569462+0000","last_peered":"2024-09-21T03:39:06.569462+0000","last_clean":"2024-09-21T03:39:06.569462+0000","last_became_active":"2024-09-21T03:39:05.419693+0000","last_became_peered":"2024-09-21T03:39:05.419693+0000","last_unstale":"2024-09-21T03:39:06.569462+0000","last_undegraded":"2024-09-21T03:39:06.569462+0000","last_fullsized":"2024-09-21T03:39:06.569462+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:08:48.101749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.418856+0000","last_change":"2024-09-21T03:39:04.410302+0000","last_active":"2024-09-21T03:39:06.418856+0000","last_peered":"2024-09-21T03:39:06.418856+0000","last_clean":"2024-09-21T03:39:06.418856+0000","last_became_active":"2024-09-21T03:39:04.409959+0000","last_became_peered":"2024-09-21T03:39:04.409959+0000","last_unstale":"2024-09-21T03:39:06.418856+0000","last_undegraded":"2024-09-21T03:39:06.418856+0000","last_fullsized":"2024-09-21T03:39:06.418856+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_clean_scrub_stamp":"2024-09-21T03:39:03.377315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:18:35.050119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.419019+0000","last_change":"2024-09-21T03:39:05.429795+0000","last_active":"2024-09-21T03:39:06.419019+0000","last_peered":"2024-09-21T03:39:06.419019+0000","last_clean":"2024-09-21T03:39:06.419019+0000","last_became_active":"2024-09-21T03:39:05.426181+0000","last_became_peered":"2024-09-21T03:39:05.426181+0000","last_unstale":"2024-09-21T03:39:06.419019+0000","last_undegraded":"2024-09-21T03:39:06.419019+0000","last_fullsized":"2024-09-21T03:39:06.419019+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:38:28.792089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.570075+0000","last_change":"2024-09-21T03:39:04.410453+0000","last_active":"2024-09-21T03:39:06.570075+0000","last_peered":"2024-09-21T03:39:06.570075+0000","last_clean":"2024-09-21T03:39:06.570075+0000","last_became_active":"2024-09-21T03:39:04.410082+0000","last_became_peered":"2024-09-21T03:39:04.410082+0000","last_unstale":"2024-09-21T03:39:06.570075+0000","last_undegraded":"2024-09-21T03:39:06.570075+0000","last_fullsized":"2024-09-21T03:39:06.570075+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_clean_scrub_stamp":"2024-09-21T03:39:03.377315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:56:14.520557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.569202+0000","last_change":"2024-09-21T03:39:05.422217+0000","last_active":"2024-09-21T03:39:06.569202+0000","last_peered":"2024-09-21T03:39:06.569202+0000","last_clean":"2024-09-21T03:39:06.569202+0000","last_became_active":"2024-09-21T03:39:05.421890+0000","last_became_peered":"2024-09-21T03:39:05.421890+0000","last_unstale":"2024-09-21T03:39:06.569202+0000","last_undegraded":"2024-09-21T03:39:06.569202+0000","last_fullsized":"2024-09-21T03:39:06.569202+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:40:45.946118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T03:39:04.414955+0000","last_change":"2024-09-21T03:39:04.411385+0000","last_active":"2024-09-21T03:39:04.414955+0000","last_peered":"2024-09-21T03:39:04.414955+0000","last_clean":"2024-09-21T03:39:04.414955+0000","last_became_active":"2024-09-21T03:39:04.410963+0000","last_became_peered":"2024-09-21T03:39:04.410963+0000","last_unstale":"2024-09-21T03:39:04.414955+0000","last_undegraded":"2024-09-21T03:39:04.414955+0000","last_fullsized":"2024-09-21T03:39:04.414955+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_clean_scrub_stamp":"2024-09-21T03:39:03.377315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:53:58.324899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.414821+0000","last_change":"2024-09-21T03:39:05.419517+0000","last_active":"2024-09-21T03:39:06.414821+0000","last_peered":"2024-09-21T03:39:06.414821+0000","last_clean":"2024-09-21T03:39:06.414821+0000","last_became_active":"2024-09-21T03:39:05.418599+0000","last_became_peered":"2024-09-21T03:39:05.418599+0000","last_unstale":"2024-09-21T03:39:06.414821+0000","last_undegraded":"2024-09-21T03:39:06.414821+0000","last_fullsized":"2024-09-21T03:39:06.414821+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:59:45.194584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.15","version":"81'3","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.413935+0000","last_change":"2024-09-21T03:39:04.407595+0000","last_active":"2024-09-21T03:39:06.413935+0000","last_peered":"2024-09-21T03:39:06.413935+0000","last_clean":"2024-09-21T03:39:06.413935+0000","last_became_active":"2024-09-21T03:39:04.407278+0000","last_became_peered":"2024-09-21T03:39:04.407278+0000","last_unstale":"2024-09-21T03:39:06.413935+0000","last_undegraded":"2024-09-21T03:39:06.413935+0000","last_fullsized":"2024-09-21T03:39:06.413935+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_clean_scrub_stamp":"2024-09-21T03:39:03.377315+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:41:01.617219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.414516+0000","last_change":"2024-09-21T03:39:05.419678+0000","last_active":"2024-09-21T03:39:06.414516+0000","last_peered":"2024-09-21T03:39:06.414516+0000","last_clean":"2024-09-21T03:39:06.414516+0000","last_became_active":"2024-09-21T03:39:05.419119+0000","last_became_peered":"2024-09-21T03:39:05.419119+0000","last_unstale":"2024-09-21T03:39:06.414516+0000","last_undegraded":"2024-09-21T03:39:06.414516+0000","last_fullsized":"2024-09-21T03:39:06.414516+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:03:30.086684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.14","version":"81'6","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.569939+0000","last_change":"2024-09-21T03:39:04.396680+0000","last_active":"2024-09-21T03:39:06.569939+0000","last_peered":"2024-09-21T03:39:06.569939+0000","last_clean":"2024-09-21T03:39:06.569939+0000","last_became_active":"2024-09-21T03:39:04.396344+0000","last_became_peered":"2024-09-21T03:39:04.396344+0000","last_unstale":"2024-09-21T03:39:06.569939+0000","last_undegraded":"2024-09-21T03:39:06.569939+0000","last_fullsized":"2024-09-21T03:39:06.569939+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_clean_scrub_stamp":"2024-09-21T03:39:03.377315+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:12:41.701014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.569672+0000","last_change":"2024-09-21T03:39:05.423246+0000","last_active":"2024-09-21T03:39:06.569672+0000","last_peered":"2024-09-21T03:39:06.569672+0000","last_clean":"2024-09-21T03:39:06.569672+0000","last_became_active":"2024-09-21T03:39:05.422924+0000","last_became_peered":"2024-09-21T03:39:05.422924+0000","last_unstale":"2024-09-21T03:39:06.569672+0000","last_undegraded":"2024-09-21T03:39:06.569672+0000","last_fullsized":"2024-09-21T03:39:06.569672+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:51:42.048958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T03:39:04.407961+0000","last_change":"2024-09-21T03:39:04.402594+0000","last_active":"2024-09-21T03:39:04.407961+0000","last_peered":"2024-09-21T03:39:04.407961+0000","last_clean":"2024-09-21T03:39:04.407961+0000","last_became_active":"2024-09-21T03:39:04.402242+0000","last_became_peered":"2024-09-21T03:39:04.402242+0000","last_unstale":"2024-09-21T03:39:04.407961+0000","last_undegraded":"2024-09-21T03:39:04.407961+0000","last_fullsized":"2024-09-21T03:39:04.407961+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_clean_scrub_stamp":"2024-09-21T03:39:03.377315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:23:27.346296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-09-21T03:39:04.401407+0000","last_change":"2024-09-21T03:39:04.401407+0000","last_active":"2024-09-21T03:39:04.383211+0000","last_peered":"2024-09-21T03:39:04.383211+0000","last_clean":"2024-09-21T03:39:04.383211+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:39:04.401407+0000","last_undegraded":"2024-09-21T03:39:04.401407+0000","last_fullsized":"2024-09-21T03:39:04.401407+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.569717+0000","last_change":"2024-09-21T03:39:04.409288+0000","last_active":"2024-09-21T03:39:06.569717+0000","last_peered":"2024-09-21T03:39:06.569717+0000","last_clean":"2024-09-21T03:39:06.569717+0000","last_became_active":"2024-09-21T03:39:04.408965+0000","last_became_peered":"2024-09-21T03:39:04.408965+0000","last_unstale":"2024-09-21T03:39:06.569717+0000","last_undegraded":"2024-09-21T03:39:06.569717+0000","last_fullsized":"2024-09-21T03:39:06.569717+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_clean_scrub_stamp":"2024-09-21T03:39:03.377315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:01:17.101953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.570597+0000","last_change":"2024-09-21T03:39:05.423231+0000","last_active":"2024-09-21T03:39:06.570597+0000","last_peered":"2024-09-21T03:39:06.570597+0000","last_clean":"2024-09-21T03:39:06.570597+0000","last_became_active":"2024-09-21T03:39:05.422931+0000","last_became_peered":"2024-09-21T03:39:05.422931+0000","last_unstale":"2024-09-21T03:39:06.570597+0000","last_undegraded":"2024-09-21T03:39:06.570597+0000","last_fullsized":"2024-09-21T03:39:06.570597+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:35:18.019438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.570103+0000","last_change":"2024-09-21T03:39:04.405877+0000","last_active":"2024-09-21T03:39:06.570103+0000","last_peered":"2024-09-21T03:39:06.570103+0000","last_clean":"2024-09-21T03:39:06.570103+0000","last_became_active":"2024-09-21T03:39:04.405583+0000","last_became_peered":"2024-09-21T03:39:04.405583+0000","last_unstale":"2024-09-21T03:39:06.570103+0000","last_undegraded":"2024-09-21T03:39:06.570103+0000","last_fullsized":"2024-09-21T03:39:06.570103+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_clean_scrub_stamp":"2024-09-21T03:39:03.377315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:03:56.352603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-09-21T03:39:04.402886+0000","last_change":"2024-09-21T03:39:04.402886+0000","last_active":"2024-09-21T03:39:04.383211+0000","last_peered":"2024-09-21T03:39:04.383211+0000","last_clean":"2024-09-21T03:39:04.383211+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:39:04.402886+0000","last_undegraded":"2024-09-21T03:39:04.402886+0000","last_fullsized":"2024-09-21T03:39:04.402886+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.10","version":"81'8","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.413938+0000","last_change":"2024-09-21T03:39:04.411109+0000","last_active":"2024-09-21T03:39:06.413938+0000","last_peered":"2024-09-21T03:39:06.413938+0000","last_clean":"2024-09-21T03:39:06.413938+0000","last_became_active":"2024-09-21T03:39:04.410744+0000","last_became_peered":"2024-09-21T03:39:04.410744+0000","last_unstale":"2024-09-21T03:39:06.413938+0000","last_undegraded":"2024-09-21T03:39:06.413938+0000","last_fullsized":"2024-09-21T03:39:06.413938+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_clean_scrub_stamp":"2024-09-21T03:39:03.377315+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:13:48.637151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.570028+0000","last_change":"2024-09-21T03:39:05.432432+0000","last_active":"2024-09-21T03:39:06.570028+0000","last_peered":"2024-09-21T03:39:06.570028+0000","last_clean":"2024-09-21T03:39:06.570028+0000","last_became_active":"2024-09-21T03:39:05.432136+0000","last_became_peered":"2024-09-21T03:39:05.432136+0000","last_unstale":"2024-09-21T03:39:06.570028+0000","last_undegraded":"2024-09-21T03:39:06.570028+0000","last_fullsized":"2024-09-21T03:39:06.570028+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:10:20.019199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.f","version":"81'1","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.570192+0000","last_change":"2024-09-21T03:39:04.403666+0000","last_active":"2024-09-21T03:39:06.570192+0000","last_peered":"2024-09-21T03:39:06.570192+0000","last_clean":"2024-09-21T03:39:06.570192+0000","last_became_active":"2024-09-21T03:39:04.403310+0000","last_became_peered":"2024-09-21T03:39:04.403310+0000","last_unstale":"2024-09-21T03:39:06.570192+0000","last_undegraded":"2024-09-21T03:39:06.570192+0000","last_fullsized":"2024-09-21T03:39:06.570192+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_clean_scrub_stamp":"2024-09-21T03:39:03.377315+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:18:35.050119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.418952+0000","last_change":"2024-09-21T03:39:05.432320+0000","last_active":"2024-09-21T03:39:06.418952+0000","last_peered":"2024-09-21T03:39:06.418952+0000","last_clean":"2024-09-21T03:39:06.418952+0000","last_became_active":"2024-09-21T03:39:05.431965+0000","last_became_peered":"2024-09-21T03:39:05.431965+0000","last_unstale":"2024-09-21T03:39:06.418952+0000","last_undegraded":"2024-09-21T03:39:06.418952+0000","last_fullsized":"2024-09-21T03:39:06.418952+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:51:42.048958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.e","version":"81'2","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.569527+0000","last_change":"2024-09-21T03:39:04.403354+0000","last_active":"2024-09-21T03:39:06.569527+0000","last_peered":"2024-09-21T03:39:06.569527+0000","last_clean":"2024-09-21T03:39:06.569527+0000","last_became_active":"2024-09-21T03:39:04.401729+0000","last_became_peered":"2024-09-21T03:39:04.401729+0000","last_unstale":"2024-09-21T03:39:06.569527+0000","last_undegraded":"2024-09-21T03:39:06.569527+0000","last_fullsized":"2024-09-21T03:39:06.569527+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_clean_scrub_stamp":"2024-09-21T03:39:03.377315+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:05:16.494460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.414672+0000","last_change":"2024-09-21T03:39:05.421453+0000","last_active":"2024-09-21T03:39:06.414672+0000","last_peered":"2024-09-21T03:39:06.414672+0000","last_clean":"2024-09-21T03:39:06.414672+0000","last_became_active":"2024-09-21T03:39:05.420867+0000","last_became_peered":"2024-09-21T03:39:05.420867+0000","last_unstale":"2024-09-21T03:39:06.414672+0000","last_undegraded":"2024-09-21T03:39:06.414672+0000","last_fullsized":"2024-09-21T03:39:06.414672+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:47:58.257466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.d","version":"81'1","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.570103+0000","last_change":"2024-09-21T03:39:04.408221+0000","last_active":"2024-09-21T03:39:06.570103+0000","last_peered":"2024-09-21T03:39:06.570103+0000","last_clean":"2024-09-21T03:39:06.570103+0000","last_became_active":"2024-09-21T03:39:04.407900+0000","last_became_peered":"2024-09-21T03:39:04.407900+0000","last_unstale":"2024-09-21T03:39:06.570103+0000","last_undegraded":"2024-09-21T03:39:06.570103+0000","last_fullsized":"2024-09-21T03:39:06.570103+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_clean_scrub_stamp":"2024-09-21T03:39:03.377315+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:26:11.862449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.570468+0000","last_change":"2024-09-21T03:39:05.423205+0000","last_active":"2024-09-21T03:39:06.570468+0000","last_peered":"2024-09-21T03:39:06.570468+0000","last_clean":"2024-09-21T03:39:06.570468+0000","last_became_active":"2024-09-21T03:39:05.422867+0000","last_became_peered":"2024-09-21T03:39:05.422867+0000","last_unstale":"2024-09-21T03:39:06.570468+0000","last_undegraded":"2024-09-21T03:39:06.570468+0000","last_fullsized":"2024-09-21T03:39:06.570468+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:33:29.949291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.570257+0000","last_change":"2024-09-21T03:39:04.401682+0000","last_active":"2024-09-21T03:39:06.570257+0000","last_peered":"2024-09-21T03:39:06.570257+0000","last_clean":"2024-09-21T03:39:06.570257+0000","last_became_active":"2024-09-21T03:39:04.401354+0000","last_became_peered":"2024-09-21T03:39:04.401354+0000","last_unstale":"2024-09-21T03:39:06.570257+0000","last_undegraded":"2024-09-21T03:39:06.570257+0000","last_fullsized":"2024-09-21T03:39:06.570257+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_clean_scrub_stamp":"2024-09-21T03:39:03.377315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:02:56.975258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.570149+0000","last_change":"2024-09-21T03:39:05.424271+0000","last_active":"2024-09-21T03:39:06.570149+0000","last_peered":"2024-09-21T03:39:06.570149+0000","last_clean":"2024-09-21T03:39:06.570149+0000","last_became_active":"2024-09-21T03:39:05.423967+0000","last_became_peered":"2024-09-21T03:39:05.423967+0000","last_unstale":"2024-09-21T03:39:06.570149+0000","last_undegraded":"2024-09-21T03:39:06.570149+0000","last_fullsized":"2024-09-21T03:39:06.570149+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:29:22.009760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"81'2","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.418688+0000","last_change":"2024-09-21T03:39:04.406956+0000","last_active":"2024-09-21T03:39:06.418688+0000","last_peered":"2024-09-21T03:39:06.418688+0000","last_clean":"2024-09-21T03:39:06.418688+0000","last_became_active":"2024-09-21T03:39:04.406604+0000","last_became_peered":"2024-09-21T03:39:04.406604+0000","last_unstale":"2024-09-21T03:39:06.418688+0000","last_undegraded":"2024-09-21T03:39:06.418688+0000","last_fullsized":"2024-09-21T03:39:06.418688+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_clean_scrub_stamp":"2024-09-21T03:39:03.377315+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:56:14.520557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.570451+0000","last_change":"2024-09-21T03:39:05.424012+0000","last_active":"2024-09-21T03:39:06.570451+0000","last_peered":"2024-09-21T03:39:06.570451+0000","last_clean":"2024-09-21T03:39:06.570451+0000","last_became_active":"2024-09-21T03:39:05.423713+0000","last_became_peered":"2024-09-21T03:39:05.423713+0000","last_unstale":"2024-09-21T03:39:06.570451+0000","last_undegraded":"2024-09-21T03:39:06.570451+0000","last_fullsized":"2024-09-21T03:39:06.570451+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:30:06.424731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.413940+0000","last_change":"2024-09-21T03:39:04.395490+0000","last_active":"2024-09-21T03:39:06.413940+0000","last_peered":"2024-09-21T03:39:06.413940+0000","last_clean":"2024-09-21T03:39:06.413940+0000","last_became_active":"2024-09-21T03:39:04.394934+0000","last_became_peered":"2024-09-21T03:39:04.394934+0000","last_unstale":"2024-09-21T03:39:06.413940+0000","last_undegraded":"2024-09-21T03:39:06.413940+0000","last_fullsized":"2024-09-21T03:39:06.413940+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_clean_scrub_stamp":"2024-09-21T03:39:03.377315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:30:01.896910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.569691+0000","last_change":"2024-09-21T03:39:05.418742+0000","last_active":"2024-09-21T03:39:06.569691+0000","last_peered":"2024-09-21T03:39:06.569691+0000","last_clean":"2024-09-21T03:39:06.569691+0000","last_became_active":"2024-09-21T03:39:05.418366+0000","last_became_peered":"2024-09-21T03:39:05.418366+0000","last_unstale":"2024-09-21T03:39:06.569691+0000","last_undegraded":"2024-09-21T03:39:06.569691+0000","last_fullsized":"2024-09-21T03:39:06.569691+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:09:46.888780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.418986+0000","last_change":"2024-09-21T03:39:04.408050+0000","last_active":"2024-09-21T03:39:06.418986+0000","last_peered":"2024-09-21T03:39:06.418986+0000","last_clean":"2024-09-21T03:39:06.418986+0000","last_became_active":"2024-09-21T03:39:04.407753+0000","last_became_peered":"2024-09-21T03:39:04.407753+0000","last_unstale":"2024-09-21T03:39:06.418986+0000","last_undegraded":"2024-09-21T03:39:06.418986+0000","last_fullsized":"2024-09-21T03:39:06.418986+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_clean_scrub_stamp":"2024-09-21T03:39:03.377315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:05:16.494460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.569516+0000","last_change":"2024-09-21T03:39:05.431029+0000","last_active":"2024-09-21T03:39:06.569516+0000","last_peered":"2024-09-21T03:39:06.569516+0000","last_clean":"2024-09-21T03:39:06.569516+0000","last_became_active":"2024-09-21T03:39:05.427562+0000","last_became_peered":"2024-09-21T03:39:05.427562+0000","last_unstale":"2024-09-21T03:39:06.569516+0000","last_undegraded":"2024-09-21T03:39:06.569516+0000","last_fullsized":"2024-09-21T03:39:06.569516+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:20:35.883679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T03:39:04.414902+0000","last_change":"2024-09-21T03:39:04.407689+0000","last_active":"2024-09-21T03:39:04.414902+0000","last_peered":"2024-09-21T03:39:04.414902+0000","last_clean":"2024-09-21T03:39:04.414902+0000","last_became_active":"2024-09-21T03:39:04.407356+0000","last_became_peered":"2024-09-21T03:39:04.407356+0000","last_unstale":"2024-09-21T03:39:04.414902+0000","last_undegraded":"2024-09-21T03:39:04.414902+0000","last_fullsized":"2024-09-21T03:39:04.414902+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_clean_scrub_stamp":"2024-09-21T03:39:03.377315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:25:15.786223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-09-21T03:39:04.400780+0000","last_change":"2024-09-21T03:39:04.400780+0000","last_active":"2024-09-21T03:39:04.383211+0000","last_peered":"2024-09-21T03:39:04.383211+0000","last_clean":"2024-09-21T03:39:04.383211+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:39:04.400780+0000","last_undegraded":"2024-09-21T03:39:04.400780+0000","last_fullsized":"2024-09-21T03:39:04.400780+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.6","version":"82'1","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.417795+0000","last_change":"2024-09-21T03:39:04.411532+0000","last_active":"2024-09-21T03:39:06.417795+0000","last_peered":"2024-09-21T03:39:06.417795+0000","last_clean":"2024-09-21T03:39:06.417795+0000","last_became_active":"2024-09-21T03:39:04.411189+0000","last_became_peered":"2024-09-21T03:39:04.411189+0000","last_unstale":"2024-09-21T03:39:06.417795+0000","last_undegraded":"2024-09-21T03:39:06.417795+0000","last_fullsized":"2024-09-21T03:39:06.417795+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_clean_scrub_stamp":"2024-09-21T03:39:03.377315+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:31:32.273309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.569653+0000","last_change":"2024-09-21T03:39:05.414196+0000","last_active":"2024-09-21T03:39:06.569653+0000","last_peered":"2024-09-21T03:39:06.569653+0000","last_clean":"2024-09-21T03:39:06.569653+0000","last_became_active":"2024-09-21T03:39:05.413871+0000","last_became_peered":"2024-09-21T03:39:05.413871+0000","last_unstale":"2024-09-21T03:39:06.569653+0000","last_undegraded":"2024-09-21T03:39:06.569653+0000","last_fullsized":"2024-09-21T03:39:06.569653+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:27:34.908887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T03:39:04.413543+0000","last_change":"2024-09-21T03:39:04.405352+0000","last_active":"2024-09-21T03:39:04.413543+0000","last_peered":"2024-09-21T03:39:04.413543+0000","last_clean":"2024-09-21T03:39:04.413543+0000","last_became_active":"2024-09-21T03:39:04.405020+0000","last_became_peered":"2024-09-21T03:39:04.405020+0000","last_unstale":"2024-09-21T03:39:04.413543+0000","last_undegraded":"2024-09-21T03:39:04.413543+0000","last_fullsized":"2024-09-21T03:39:04.413543+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_clean_scrub_stamp":"2024-09-21T03:39:03.377315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:37:33.420266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.415196+0000","last_change":"2024-09-21T03:39:05.415347+0000","last_active":"2024-09-21T03:39:06.415196+0000","last_peered":"2024-09-21T03:39:06.415196+0000","last_clean":"2024-09-21T03:39:06.415196+0000","last_became_active":"2024-09-21T03:39:05.414476+0000","last_became_peered":"2024-09-21T03:39:05.414476+0000","last_unstale":"2024-09-21T03:39:06.415196+0000","last_undegraded":"2024-09-21T03:39:06.415196+0000","last_fullsized":"2024-09-21T03:39:06.415196+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:49:45.640520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.569666+0000","last_change":"2024-09-21T03:39:04.410318+0000","last_active":"2024-09-21T03:39:06.569666+0000","last_peered":"2024-09-21T03:39:06.569666+0000","last_clean":"2024-09-21T03:39:06.569666+0000","last_became_active":"2024-09-21T03:39:04.409989+0000","last_became_peered":"2024-09-21T03:39:04.409989+0000","last_unstale":"2024-09-21T03:39:06.569666+0000","last_undegraded":"2024-09-21T03:39:06.569666+0000","last_fullsized":"2024-09-21T03:39:06.569666+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_clean_scrub_stamp":"2024-09-21T03:39:03.377315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:49:27.896288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.570622+0000","last_change":"2024-09-21T03:39:05.421291+0000","last_active":"2024-09-21T03:39:06.570622+0000","last_peered":"2024-09-21T03:39:06.570622+0000","last_clean":"2024-09-21T03:39:06.570622+0000","last_became_active":"2024-09-21T03:39:05.420933+0000","last_became_peered":"2024-09-21T03:39:05.420933+0000","last_unstale":"2024-09-21T03:39:06.570622+0000","last_undegraded":"2024-09-21T03:39:06.570622+0000","last_fullsized":"2024-09-21T03:39:06.570622+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:18:54.425316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.569375+0000","last_change":"2024-09-21T03:39:04.405531+0000","last_active":"2024-09-21T03:39:06.569375+0000","last_peered":"2024-09-21T03:39:06.569375+0000","last_clean":"2024-09-21T03:39:06.569375+0000","last_became_active":"2024-09-21T03:39:04.405193+0000","last_became_peered":"2024-09-21T03:39:04.405193+0000","last_unstale":"2024-09-21T03:39:06.569375+0000","last_undegraded":"2024-09-21T03:39:06.569375+0000","last_fullsized":"2024-09-21T03:39:06.569375+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_clean_scrub_stamp":"2024-09-21T03:39:03.377315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:41:35.755494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.569966+0000","last_change":"2024-09-21T03:39:05.431541+0000","last_active":"2024-09-21T03:39:06.569966+0000","last_peered":"2024-09-21T03:39:06.569966+0000","last_clean":"2024-09-21T03:39:06.569966+0000","last_became_active":"2024-09-21T03:39:05.428130+0000","last_became_peered":"2024-09-21T03:39:05.428130+0000","last_unstale":"2024-09-21T03:39:06.569966+0000","last_undegraded":"2024-09-21T03:39:06.569966+0000","last_fullsized":"2024-09-21T03:39:06.569966+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:22:26.967805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.415539+0000","last_change":"2024-09-21T03:39:04.403462+0000","last_active":"2024-09-21T03:39:06.415539+0000","last_peered":"2024-09-21T03:39:06.415539+0000","last_clean":"2024-09-21T03:39:06.415539+0000","last_became_active":"2024-09-21T03:39:04.403114+0000","last_became_peered":"2024-09-21T03:39:04.403114+0000","last_unstale":"2024-09-21T03:39:06.415539+0000","last_undegraded":"2024-09-21T03:39:06.415539+0000","last_fullsized":"2024-09-21T03:39:06.415539+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_clean_scrub_stamp":"2024-09-21T03:39:03.377315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:38:07.041511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.419363+0000","last_change":"2024-09-21T03:39:05.429296+0000","last_active":"2024-09-21T03:39:06.419363+0000","last_peered":"2024-09-21T03:39:06.419363+0000","last_clean":"2024-09-21T03:39:06.419363+0000","last_became_active":"2024-09-21T03:39:05.426008+0000","last_became_peered":"2024-09-21T03:39:05.426008+0000","last_unstale":"2024-09-21T03:39:06.419363+0000","last_undegraded":"2024-09-21T03:39:06.419363+0000","last_fullsized":"2024-09-21T03:39:06.419363+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:16:42.786975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.568997+0000","last_change":"2024-09-21T03:39:04.407078+0000","last_active":"2024-09-21T03:39:06.568997+0000","last_peered":"2024-09-21T03:39:06.568997+0000","last_clean":"2024-09-21T03:39:06.568997+0000","last_became_active":"2024-09-21T03:39:04.406757+0000","last_became_peered":"2024-09-21T03:39:04.406757+0000","last_unstale":"2024-09-21T03:39:06.568997+0000","last_undegraded":"2024-09-21T03:39:06.568997+0000","last_fullsized":"2024-09-21T03:39:06.568997+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_clean_scrub_stamp":"2024-09-21T03:39:03.377315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:08:19.857227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.570315+0000","last_change":"2024-09-21T03:39:05.431328+0000","last_active":"2024-09-21T03:39:06.570315+0000","last_peered":"2024-09-21T03:39:06.570315+0000","last_clean":"2024-09-21T03:39:06.570315+0000","last_became_active":"2024-09-21T03:39:05.431007+0000","last_became_peered":"2024-09-21T03:39:05.431007+0000","last_unstale":"2024-09-21T03:39:06.570315+0000","last_undegraded":"2024-09-21T03:39:06.570315+0000","last_fullsized":"2024-09-21T03:39:06.570315+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:09:46.155413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.419086+0000","last_change":"2024-09-21T03:39:04.412555+0000","last_active":"2024-09-21T03:39:06.419086+0000","last_peered":"2024-09-21T03:39:06.419086+0000","last_clean":"2024-09-21T03:39:06.419086+0000","last_became_active":"2024-09-21T03:39:04.412258+0000","last_became_peered":"2024-09-21T03:39:04.412258+0000","last_unstale":"2024-09-21T03:39:06.419086+0000","last_undegraded":"2024-09-21T03:39:06.419086+0000","last_fullsized":"2024-09-21T03:39:06.419086+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_clean_scrub_stamp":"2024-09-21T03:39:03.377315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:01:13.191529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.570331+0000","last_change":"2024-09-21T03:39:05.418570+0000","last_active":"2024-09-21T03:39:06.570331+0000","last_peered":"2024-09-21T03:39:06.570331+0000","last_clean":"2024-09-21T03:39:06.570331+0000","last_became_active":"2024-09-21T03:39:05.418257+0000","last_became_peered":"2024-09-21T03:39:05.418257+0000","last_unstale":"2024-09-21T03:39:06.570331+0000","last_undegraded":"2024-09-21T03:39:06.570331+0000","last_fullsized":"2024-09-21T03:39:06.570331+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:38:35.279761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.415650+0000","last_change":"2024-09-21T03:39:04.414590+0000","last_active":"2024-09-21T03:39:06.415650+0000","last_peered":"2024-09-21T03:39:06.415650+0000","last_clean":"2024-09-21T03:39:06.415650+0000","last_became_active":"2024-09-21T03:39:04.414355+0000","last_became_peered":"2024-09-21T03:39:04.414355+0000","last_unstale":"2024-09-21T03:39:06.415650+0000","last_undegraded":"2024-09-21T03:39:06.415650+0000","last_fullsized":"2024-09-21T03:39:06.415650+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_clean_scrub_stamp":"2024-09-21T03:39:03.377315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:03:09.175309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":223,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T03:39:04.391409+0000","last_change":"2024-09-21T03:33:25.319776+0000","last_active":"2024-09-21T03:39:04.391409+0000","last_peered":"2024-09-21T03:39:04.391409+0000","last_clean":"2024-09-21T03:39:04.391409+0000","last_became_active":"2024-09-21T03:33:25.318399+0000","last_became_peered":"2024-09-21T03:33:25.318399+0000","last_unstale":"2024-09-21T03:39:04.391409+0000","last_undegraded":"2024-09-21T03:39:04.391409+0000","last_fullsized":"2024-09-21T03:39:04.391409+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:23.016672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:23.016672+0000","last_clean_scrub_stamp":"2024-09-21T03:33:23.016672+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:32:35.906114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":2376,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":36,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":33,"ondisk_log_size":33,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":1,"up_from":16,"seq":68719476808,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32720,"kb_used_data":1528,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339120,"statfs":{"total":96636764160,"available":96603258880,"internally_reserved":0,"allocated":1564672,"data_stored":1036315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509513,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36328,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335512,"statfs":{"total":96636764160,"available":96599564288,"internally_reserved":0,"allocated":1003520,"data_stored":460458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":36174283},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509512,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32184,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339656,"statfs":{"total":96636764160,"available":96603807744,"internally_reserved":0,"allocated":1015808,"data_stored":461522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509513,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32164,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339676,"statfs":{"total":96636764160,"available":96603828224,"internally_reserved":0,"allocated":995328,"data_stored":459806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509512,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32168,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339672,"statfs":{"total":96636764160,"available":96603824128,"internally_reserved":0,"allocated":999424,"data_stored":459828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542216,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36332,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335508,"statfs":{"total":96636764160,"available":96599560192,"internally_reserved":0,"allocated":1003520,"data_stored":460478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574920,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36912,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334928,"statfs":{"total":96636764160,"available":96598966272,"internally_reserved":0,"allocated":1593344,"data_stored":1050264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542217,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36328,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335512,"statfs":{"total":96636764160,"available":96599564288,"internally_reserved":0,"allocated":999424,"data_stored":459828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T03:39:09.493 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T03:39:09.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:09.643+0000 7fbcc1c9b640 1 -- 172.21.15.38:0/851356841 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbcbc15efb0 msgr2=0x7fbcbc073ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:09.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:09.643+0000 7fbcc1c9b640 1 --2- 172.21.15.38:0/851356841 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbcbc15efb0 0x7fbcbc073ea0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc073080 tx=0x7fbcb002a4d0 comp rx=0 tx=0).stop 2024-09-21T03:39:09.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:09.643+0000 7fbcc1c9b640 1 -- 172.21.15.38:0/851356841 shutdown_connections 2024-09-21T03:39:09.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:09.643+0000 7fbcc1c9b640 1 --2- 172.21.15.38:0/851356841 >> v2:172.21.15.195:3300/0 conn(0x7fbcbc0cf130 0x7fbcbc0743e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:09.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:09.643+0000 7fbcc1c9b640 1 --2- 172.21.15.38:0/851356841 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbcbc15efb0 0x7fbcbc073ea0 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:09.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:09.643+0000 7fbcc1c9b640 1 --2- 172.21.15.38:0/851356841 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbcbc15b5f0 0x7fbcbc06ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:09.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:09.643+0000 7fbcc1c9b640 1 -- 172.21.15.38:0/851356841 >> 172.21.15.38:0/851356841 conn(0x7fbcbc0d1140 msgr2=0x7fbcbc0d1550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:09.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:09.643+0000 7fbcc1c9b640 1 -- 172.21.15.38:0/851356841 shutdown_connections 2024-09-21T03:39:09.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:09.643+0000 7fbcc1c9b640 1 -- 172.21.15.38:0/851356841 wait complete. 2024-09-21T03:39:09.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:09.645+0000 7fbcc1c9b640 1 Processor -- start 2024-09-21T03:39:09.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:09.645+0000 7fbcc1c9b640 1 -- start start 2024-09-21T03:39:09.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:09.646+0000 7fbcc1c9b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbcbc0cf130 0x7fbcbc1f6a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:09.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:09.646+0000 7fbcc1c9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbcbc15b5f0 0x7fbcbc159310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:09.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:09.646+0000 7fbcc1c9b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbcbc15efb0 0x7fbcbc15ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:09.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:09.646+0000 7fbcc1c9b640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fbcbc0ae460 con 0x7fbcbc15b5f0 2024-09-21T03:39:09.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:09.646+0000 7fbcc1c9b640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fbcbc0ae2e0 con 0x7fbcbc15efb0 2024-09-21T03:39:09.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:09.646+0000 7fbcc1c9b640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fbcbc0ae5e0 con 0x7fbcbc0cf130 2024-09-21T03:39:09.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:09.646+0000 7fbcbaffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbcbc15b5f0 0x7fbcbc159310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:09.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:09.647+0000 7fbcbaffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbcbc15b5f0 0x7fbcbc159310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:52706/0 (socket says 172.21.15.38:52706) 2024-09-21T03:39:09.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:09.647+0000 7fbcbaffd640 1 -- 172.21.15.38:0/791828903 learned_addr learned my addr 172.21.15.38:0/791828903 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:39:09.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:09.647+0000 7fbcbbfff640 1 --2- 172.21.15.38:0/791828903 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbcbc15efb0 0x7fbcbc15ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:09.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:09.647+0000 7fbcbb7fe640 1 --2- 172.21.15.38:0/791828903 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbcbc0cf130 0x7fbcbc1f6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:09.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:09.647+0000 7fbcbaffd640 1 -- 172.21.15.38:0/791828903 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbcbc0cf130 msgr2=0x7fbcbc1f6a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:09.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:09.647+0000 7fbcbaffd640 1 --2- 172.21.15.38:0/791828903 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbcbc0cf130 0x7fbcbc1f6a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:09.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:09.647+0000 7fbcbaffd640 1 -- 172.21.15.38:0/791828903 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbcbc15efb0 msgr2=0x7fbcbc15ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:09.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:09.647+0000 7fbcbaffd640 1 --2- 172.21.15.38:0/791828903 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbcbc15efb0 0x7fbcbc15ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:09.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:09.647+0000 7fbcbaffd640 1 -- 172.21.15.38:0/791828903 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcbc09e150 con 0x7fbcbc15b5f0 2024-09-21T03:39:09.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:09.647+0000 7fbcbb7fe640 1 --2- 172.21.15.38:0/791828903 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbcbc0cf130 0x7fbcbc1f6a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:39:09.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:09.647+0000 7fbcbaffd640 1 --2- 172.21.15.38:0/791828903 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbcbc15b5f0 0x7fbcbc159310 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fbcb0034040 tx=0x7fbcb00039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:09.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:09.647+0000 7fbca3fff640 1 -- 172.21.15.38:0/791828903 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcb003e030 con 0x7fbcbc15b5f0 2024-09-21T03:39:09.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:09.647+0000 7fbca3fff640 1 -- 172.21.15.38:0/791828903 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbcb00047a0 con 0x7fbcbc15b5f0 2024-09-21T03:39:09.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:09.647+0000 7fbcc1c9b640 1 -- 172.21.15.38:0/791828903 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcbc09e430 con 0x7fbcbc15b5f0 2024-09-21T03:39:09.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:09.647+0000 7fbca3fff640 1 -- 172.21.15.38:0/791828903 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcb0035040 con 0x7fbcbc15b5f0 2024-09-21T03:39:09.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:09.647+0000 7fbcc1c9b640 1 -- 172.21.15.38:0/791828903 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcbc09b950 con 0x7fbcbc15b5f0 2024-09-21T03:39:09.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:09.649+0000 7fbcc1c9b640 1 -- 172.21.15.38:0/791828903 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbcbc0aeb60 con 0x7fbcbc15b5f0 2024-09-21T03:39:09.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:09.649+0000 7fbca3fff640 1 -- 172.21.15.38:0/791828903 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fbcb0041030 con 0x7fbcbc15b5f0 2024-09-21T03:39:09.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:09.650+0000 7fbca3fff640 1 --2- 172.21.15.38:0/791828903 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fbca8078b90 0x7fbca807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:09.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:09.650+0000 7fbca3fff640 1 -- 172.21.15.38:0/791828903 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 6101+0+0 (secure 0 0 0) 0x7fbcb00bb700 con 0x7fbcbc15b5f0 2024-09-21T03:39:09.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:09.650+0000 7fbcbb7fe640 1 --2- 172.21.15.38:0/791828903 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fbca8078b90 0x7fbca807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:09.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:09.650+0000 7fbcbb7fe640 1 --2- 172.21.15.38:0/791828903 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fbca8078b90 0x7fbca807b0a0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fbcac009890 tx=0x7fbcac008040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:09.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:09.654+0000 7fbca3fff640 1 -- 172.21.15.38:0/791828903 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fbcb0083350 con 0x7fbcbc15b5f0 2024-09-21T03:39:09.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:09.837+0000 7fbcc1c9b640 1 -- 172.21.15.38:0/791828903 --> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fbcbc09df00 con 0x7fbca8078b90 2024-09-21T03:39:09.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:09.841+0000 7fbca3fff640 1 -- 172.21.15.38:0/791828903 <== mgr.4117 v2:172.21.15.195:6800/1631901344 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251482 (secure 0 0 0) 0x7fbcbc09df00 con 0x7fbca8078b90 2024-09-21T03:39:09.846 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:39:09.847 INFO:teuthology.orchestra.run.smithi038.stderr:dumped all 2024-09-21T03:39:09.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:09.845+0000 7fbcc1c9b640 1 -- 172.21.15.38:0/791828903 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fbca8078b90 msgr2=0x7fbca807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:09.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:09.845+0000 7fbcc1c9b640 1 --2- 172.21.15.38:0/791828903 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fbca8078b90 0x7fbca807b0a0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fbcac009890 tx=0x7fbcac008040 comp rx=0 tx=0).stop 2024-09-21T03:39:09.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:09.845+0000 7fbcc1c9b640 1 -- 172.21.15.38:0/791828903 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbcbc15b5f0 msgr2=0x7fbcbc159310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:09.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:09.845+0000 7fbcc1c9b640 1 --2- 172.21.15.38:0/791828903 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbcbc15b5f0 0x7fbcbc159310 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fbcb0034040 tx=0x7fbcb00039e0 comp rx=0 tx=0).stop 2024-09-21T03:39:09.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:09.845+0000 7fbcc1c9b640 1 -- 172.21.15.38:0/791828903 shutdown_connections 2024-09-21T03:39:09.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:09.845+0000 7fbcc1c9b640 1 --2- 172.21.15.38:0/791828903 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbcbc15efb0 0x7fbcbc15ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:09.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:09.845+0000 7fbcc1c9b640 1 --2- 172.21.15.38:0/791828903 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbcbc15b5f0 0x7fbcbc159310 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:09.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:09.845+0000 7fbcc1c9b640 1 --2- 172.21.15.38:0/791828903 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fbca8078b90 0x7fbca807b0a0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:09.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:09.845+0000 7fbcc1c9b640 1 --2- 172.21.15.38:0/791828903 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbcbc0cf130 0x7fbcbc1f6a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:09.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:09.845+0000 7fbcc1c9b640 1 -- 172.21.15.38:0/791828903 >> 172.21.15.38:0/791828903 conn(0x7fbcbc0d1140 msgr2=0x7fbcbc0b1d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:09.850 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:09.845+0000 7fbcc1c9b640 1 -- 172.21.15.38:0/791828903 shutdown_connections 2024-09-21T03:39:09.850 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:09.845+0000 7fbcc1c9b640 1 -- 172.21.15.38:0/791828903 wait complete. 2024-09-21T03:39:09.858 INFO:teuthology.orchestra.run.smithi038.stdout:{"pg_ready":true,"pg_map":{"version":248,"stamp":"2024-09-21T03:39:09.071943+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592744,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":137,"num_write_kb":1635,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":105,"ondisk_log_size":105,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":154,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":275136,"kb_used_data":8960,"kb_used_omap":12,"kb_used_meta":265971,"kb_avail":754699584,"statfs":{"total":773094113280,"available":772812374016,"internally_reserved":0,"allocated":9175040,"data_stored":4848499,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12748,"internal_metadata":272354868},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2376,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":36,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.687763"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-09-21T03:39:04.404050+0000","last_change":"2024-09-21T03:39:04.404050+0000","last_active":"2024-09-21T03:39:04.383211+0000","last_peered":"2024-09-21T03:39:04.383211+0000","last_clean":"2024-09-21T03:39:04.383211+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:39:04.404050+0000","last_undegraded":"2024-09-21T03:39:04.404050+0000","last_fullsized":"2024-09-21T03:39:04.404050+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.419248+0000","last_change":"2024-09-21T03:39:05.431615+0000","last_active":"2024-09-21T03:39:06.419248+0000","last_peered":"2024-09-21T03:39:06.419248+0000","last_clean":"2024-09-21T03:39:06.419248+0000","last_became_active":"2024-09-21T03:39:05.429700+0000","last_became_peered":"2024-09-21T03:39:05.429700+0000","last_unstale":"2024-09-21T03:39:06.419248+0000","last_undegraded":"2024-09-21T03:39:06.419248+0000","last_fullsized":"2024-09-21T03:39:06.419248+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:22:26.967805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.419361+0000","last_change":"2024-09-21T03:39:05.434451+0000","last_active":"2024-09-21T03:39:06.419361+0000","last_peered":"2024-09-21T03:39:06.419361+0000","last_clean":"2024-09-21T03:39:06.419361+0000","last_became_active":"2024-09-21T03:39:05.432952+0000","last_became_peered":"2024-09-21T03:39:05.432952+0000","last_unstale":"2024-09-21T03:39:06.419361+0000","last_undegraded":"2024-09-21T03:39:06.419361+0000","last_fullsized":"2024-09-21T03:39:06.419361+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:30:48.915097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.569594+0000","last_change":"2024-09-21T03:39:05.426032+0000","last_active":"2024-09-21T03:39:06.569594+0000","last_peered":"2024-09-21T03:39:06.569594+0000","last_clean":"2024-09-21T03:39:06.569594+0000","last_became_active":"2024-09-21T03:39:05.425731+0000","last_became_peered":"2024-09-21T03:39:05.425731+0000","last_unstale":"2024-09-21T03:39:06.569594+0000","last_undegraded":"2024-09-21T03:39:06.569594+0000","last_fullsized":"2024-09-21T03:39:06.569594+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:01:01.380435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.570162+0000","last_change":"2024-09-21T03:39:05.428805+0000","last_active":"2024-09-21T03:39:06.570162+0000","last_peered":"2024-09-21T03:39:06.570162+0000","last_clean":"2024-09-21T03:39:06.570162+0000","last_became_active":"2024-09-21T03:39:05.428394+0000","last_became_peered":"2024-09-21T03:39:05.428394+0000","last_unstale":"2024-09-21T03:39:06.570162+0000","last_undegraded":"2024-09-21T03:39:06.570162+0000","last_fullsized":"2024-09-21T03:39:06.570162+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:57:28.504333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.415502+0000","last_change":"2024-09-21T03:39:05.418164+0000","last_active":"2024-09-21T03:39:06.415502+0000","last_peered":"2024-09-21T03:39:06.415502+0000","last_clean":"2024-09-21T03:39:06.415502+0000","last_became_active":"2024-09-21T03:39:05.416861+0000","last_became_peered":"2024-09-21T03:39:05.416861+0000","last_unstale":"2024-09-21T03:39:06.415502+0000","last_undegraded":"2024-09-21T03:39:06.415502+0000","last_fullsized":"2024-09-21T03:39:06.415502+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:06:43.522600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.569829+0000","last_change":"2024-09-21T03:39:05.429537+0000","last_active":"2024-09-21T03:39:06.569829+0000","last_peered":"2024-09-21T03:39:06.569829+0000","last_clean":"2024-09-21T03:39:06.569829+0000","last_became_active":"2024-09-21T03:39:05.428077+0000","last_became_peered":"2024-09-21T03:39:05.428077+0000","last_unstale":"2024-09-21T03:39:06.569829+0000","last_undegraded":"2024-09-21T03:39:06.569829+0000","last_fullsized":"2024-09-21T03:39:06.569829+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:38:28.792089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.414574+0000","last_change":"2024-09-21T03:39:05.420868+0000","last_active":"2024-09-21T03:39:06.414574+0000","last_peered":"2024-09-21T03:39:06.414574+0000","last_clean":"2024-09-21T03:39:06.414574+0000","last_became_active":"2024-09-21T03:39:05.420546+0000","last_became_peered":"2024-09-21T03:39:05.420546+0000","last_unstale":"2024-09-21T03:39:06.414574+0000","last_undegraded":"2024-09-21T03:39:06.414574+0000","last_fullsized":"2024-09-21T03:39:06.414574+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:00:25.925784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.570276+0000","last_change":"2024-09-21T03:39:05.424217+0000","last_active":"2024-09-21T03:39:06.570276+0000","last_peered":"2024-09-21T03:39:06.570276+0000","last_clean":"2024-09-21T03:39:06.570276+0000","last_became_active":"2024-09-21T03:39:05.423922+0000","last_became_peered":"2024-09-21T03:39:05.423922+0000","last_unstale":"2024-09-21T03:39:06.570276+0000","last_undegraded":"2024-09-21T03:39:06.570276+0000","last_fullsized":"2024-09-21T03:39:06.570276+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:30:48.915097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-09-21T03:39:04.403441+0000","last_change":"2024-09-21T03:39:04.403441+0000","last_active":"2024-09-21T03:39:04.383211+0000","last_peered":"2024-09-21T03:39:04.383211+0000","last_clean":"2024-09-21T03:39:04.383211+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:39:04.403441+0000","last_undegraded":"2024-09-21T03:39:04.403441+0000","last_fullsized":"2024-09-21T03:39:04.403441+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.419460+0000","last_change":"2024-09-21T03:39:05.427718+0000","last_active":"2024-09-21T03:39:06.419460+0000","last_peered":"2024-09-21T03:39:06.419460+0000","last_clean":"2024-09-21T03:39:06.419460+0000","last_became_active":"2024-09-21T03:39:05.427387+0000","last_became_peered":"2024-09-21T03:39:05.427387+0000","last_unstale":"2024-09-21T03:39:06.419460+0000","last_undegraded":"2024-09-21T03:39:06.419460+0000","last_fullsized":"2024-09-21T03:39:06.419460+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:58:09.229781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.570387+0000","last_change":"2024-09-21T03:39:05.421453+0000","last_active":"2024-09-21T03:39:06.570387+0000","last_peered":"2024-09-21T03:39:06.570387+0000","last_clean":"2024-09-21T03:39:06.570387+0000","last_became_active":"2024-09-21T03:39:05.421133+0000","last_became_peered":"2024-09-21T03:39:05.421133+0000","last_unstale":"2024-09-21T03:39:06.570387+0000","last_undegraded":"2024-09-21T03:39:06.570387+0000","last_fullsized":"2024-09-21T03:39:06.570387+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:30:07.399497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.418721+0000","last_change":"2024-09-21T03:39:05.426022+0000","last_active":"2024-09-21T03:39:06.418721+0000","last_peered":"2024-09-21T03:39:06.418721+0000","last_clean":"2024-09-21T03:39:06.418721+0000","last_became_active":"2024-09-21T03:39:05.425625+0000","last_became_peered":"2024-09-21T03:39:05.425625+0000","last_unstale":"2024-09-21T03:39:06.418721+0000","last_undegraded":"2024-09-21T03:39:06.418721+0000","last_fullsized":"2024-09-21T03:39:06.418721+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:03:54.230793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.414472+0000","last_change":"2024-09-21T03:39:05.421310+0000","last_active":"2024-09-21T03:39:06.414472+0000","last_peered":"2024-09-21T03:39:06.414472+0000","last_clean":"2024-09-21T03:39:06.414472+0000","last_became_active":"2024-09-21T03:39:05.420314+0000","last_became_peered":"2024-09-21T03:39:05.420314+0000","last_unstale":"2024-09-21T03:39:06.414472+0000","last_undegraded":"2024-09-21T03:39:06.414472+0000","last_fullsized":"2024-09-21T03:39:06.414472+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:00:07.867021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.570061+0000","last_change":"2024-09-21T03:39:05.427360+0000","last_active":"2024-09-21T03:39:06.570061+0000","last_peered":"2024-09-21T03:39:06.570061+0000","last_clean":"2024-09-21T03:39:06.570061+0000","last_became_active":"2024-09-21T03:39:05.427009+0000","last_became_peered":"2024-09-21T03:39:05.427009+0000","last_unstale":"2024-09-21T03:39:06.570061+0000","last_undegraded":"2024-09-21T03:39:06.570061+0000","last_fullsized":"2024-09-21T03:39:06.570061+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:54:27.965337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.569046+0000","last_change":"2024-09-21T03:39:05.430747+0000","last_active":"2024-09-21T03:39:06.569046+0000","last_peered":"2024-09-21T03:39:06.569046+0000","last_clean":"2024-09-21T03:39:06.569046+0000","last_became_active":"2024-09-21T03:39:05.429872+0000","last_became_peered":"2024-09-21T03:39:05.429872+0000","last_unstale":"2024-09-21T03:39:06.569046+0000","last_undegraded":"2024-09-21T03:39:06.569046+0000","last_fullsized":"2024-09-21T03:39:06.569046+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:01:18.107849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.415027+0000","last_change":"2024-09-21T03:39:05.418812+0000","last_active":"2024-09-21T03:39:06.415027+0000","last_peered":"2024-09-21T03:39:06.415027+0000","last_clean":"2024-09-21T03:39:06.415027+0000","last_became_active":"2024-09-21T03:39:05.418487+0000","last_became_peered":"2024-09-21T03:39:05.418487+0000","last_unstale":"2024-09-21T03:39:06.415027+0000","last_undegraded":"2024-09-21T03:39:06.415027+0000","last_fullsized":"2024-09-21T03:39:06.415027+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:37:29.087437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.419123+0000","last_change":"2024-09-21T03:39:05.428238+0000","last_active":"2024-09-21T03:39:06.419123+0000","last_peered":"2024-09-21T03:39:06.419123+0000","last_clean":"2024-09-21T03:39:06.419123+0000","last_became_active":"2024-09-21T03:39:05.427915+0000","last_became_peered":"2024-09-21T03:39:05.427915+0000","last_unstale":"2024-09-21T03:39:06.419123+0000","last_undegraded":"2024-09-21T03:39:06.419123+0000","last_fullsized":"2024-09-21T03:39:06.419123+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:22:48.194041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.414304+0000","last_change":"2024-09-21T03:39:05.412642+0000","last_active":"2024-09-21T03:39:06.414304+0000","last_peered":"2024-09-21T03:39:06.414304+0000","last_clean":"2024-09-21T03:39:06.414304+0000","last_became_active":"2024-09-21T03:39:05.412274+0000","last_became_peered":"2024-09-21T03:39:05.412274+0000","last_unstale":"2024-09-21T03:39:06.414304+0000","last_undegraded":"2024-09-21T03:39:06.414304+0000","last_fullsized":"2024-09-21T03:39:06.414304+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:24:30.153279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.415023+0000","last_change":"2024-09-21T03:39:05.428953+0000","last_active":"2024-09-21T03:39:06.415023+0000","last_peered":"2024-09-21T03:39:06.415023+0000","last_clean":"2024-09-21T03:39:06.415023+0000","last_became_active":"2024-09-21T03:39:05.428223+0000","last_became_peered":"2024-09-21T03:39:05.428223+0000","last_unstale":"2024-09-21T03:39:06.415023+0000","last_undegraded":"2024-09-21T03:39:06.415023+0000","last_fullsized":"2024-09-21T03:39:06.415023+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:16:38.764525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.570578+0000","last_change":"2024-09-21T03:39:05.409645+0000","last_active":"2024-09-21T03:39:06.570578+0000","last_peered":"2024-09-21T03:39:06.570578+0000","last_clean":"2024-09-21T03:39:06.570578+0000","last_became_active":"2024-09-21T03:39:05.409315+0000","last_became_peered":"2024-09-21T03:39:05.409315+0000","last_unstale":"2024-09-21T03:39:06.570578+0000","last_undegraded":"2024-09-21T03:39:06.570578+0000","last_fullsized":"2024-09-21T03:39:06.570578+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:49:12.163503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.418882+0000","last_change":"2024-09-21T03:39:05.427679+0000","last_active":"2024-09-21T03:39:06.418882+0000","last_peered":"2024-09-21T03:39:06.418882+0000","last_clean":"2024-09-21T03:39:06.418882+0000","last_became_active":"2024-09-21T03:39:05.426044+0000","last_became_peered":"2024-09-21T03:39:05.426044+0000","last_unstale":"2024-09-21T03:39:06.418882+0000","last_undegraded":"2024-09-21T03:39:06.418882+0000","last_fullsized":"2024-09-21T03:39:06.418882+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:20:35.883679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-09-21T03:39:04.402899+0000","last_change":"2024-09-21T03:39:04.402899+0000","last_active":"2024-09-21T03:39:04.383211+0000","last_peered":"2024-09-21T03:39:04.383211+0000","last_clean":"2024-09-21T03:39:04.383211+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:39:04.402899+0000","last_undegraded":"2024-09-21T03:39:04.402899+0000","last_fullsized":"2024-09-21T03:39:04.402899+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.418484+0000","last_change":"2024-09-21T03:39:05.427956+0000","last_active":"2024-09-21T03:39:06.418484+0000","last_peered":"2024-09-21T03:39:06.418484+0000","last_clean":"2024-09-21T03:39:06.418484+0000","last_became_active":"2024-09-21T03:39:05.425708+0000","last_became_peered":"2024-09-21T03:39:05.425708+0000","last_unstale":"2024-09-21T03:39:06.418484+0000","last_undegraded":"2024-09-21T03:39:06.418484+0000","last_fullsized":"2024-09-21T03:39:06.418484+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:14:22.373008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.415419+0000","last_change":"2024-09-21T03:39:05.420911+0000","last_active":"2024-09-21T03:39:06.415419+0000","last_peered":"2024-09-21T03:39:06.415419+0000","last_clean":"2024-09-21T03:39:06.415419+0000","last_became_active":"2024-09-21T03:39:05.420587+0000","last_became_peered":"2024-09-21T03:39:05.420587+0000","last_unstale":"2024-09-21T03:39:06.415419+0000","last_undegraded":"2024-09-21T03:39:06.415419+0000","last_fullsized":"2024-09-21T03:39:06.415419+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:37:32.783221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.569302+0000","last_change":"2024-09-21T03:39:05.426555+0000","last_active":"2024-09-21T03:39:06.569302+0000","last_peered":"2024-09-21T03:39:06.569302+0000","last_clean":"2024-09-21T03:39:06.569302+0000","last_became_active":"2024-09-21T03:39:05.426265+0000","last_became_peered":"2024-09-21T03:39:05.426265+0000","last_unstale":"2024-09-21T03:39:06.569302+0000","last_undegraded":"2024-09-21T03:39:06.569302+0000","last_fullsized":"2024-09-21T03:39:06.569302+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:03:54.230793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.569945+0000","last_change":"2024-09-21T03:39:05.427771+0000","last_active":"2024-09-21T03:39:06.569945+0000","last_peered":"2024-09-21T03:39:06.569945+0000","last_clean":"2024-09-21T03:39:06.569945+0000","last_became_active":"2024-09-21T03:39:05.427310+0000","last_became_peered":"2024-09-21T03:39:05.427310+0000","last_unstale":"2024-09-21T03:39:06.569945+0000","last_undegraded":"2024-09-21T03:39:06.569945+0000","last_fullsized":"2024-09-21T03:39:06.569945+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:22:48.194041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.570546+0000","last_change":"2024-09-21T03:39:05.431170+0000","last_active":"2024-09-21T03:39:06.570546+0000","last_peered":"2024-09-21T03:39:06.570546+0000","last_clean":"2024-09-21T03:39:06.570546+0000","last_became_active":"2024-09-21T03:39:05.430855+0000","last_became_peered":"2024-09-21T03:39:05.430855+0000","last_unstale":"2024-09-21T03:39:06.570546+0000","last_undegraded":"2024-09-21T03:39:06.570546+0000","last_fullsized":"2024-09-21T03:39:06.570546+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:13:27.854189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.414268+0000","last_change":"2024-09-21T03:39:05.429739+0000","last_active":"2024-09-21T03:39:06.414268+0000","last_peered":"2024-09-21T03:39:06.414268+0000","last_clean":"2024-09-21T03:39:06.414268+0000","last_became_active":"2024-09-21T03:39:05.429401+0000","last_became_peered":"2024-09-21T03:39:05.429401+0000","last_unstale":"2024-09-21T03:39:06.414268+0000","last_undegraded":"2024-09-21T03:39:06.414268+0000","last_fullsized":"2024-09-21T03:39:06.414268+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:29:58.646678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-09-21T03:39:04.402307+0000","last_change":"2024-09-21T03:39:04.402307+0000","last_active":"2024-09-21T03:39:04.383211+0000","last_peered":"2024-09-21T03:39:04.383211+0000","last_clean":"2024-09-21T03:39:04.383211+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:39:04.402307+0000","last_undegraded":"2024-09-21T03:39:04.402307+0000","last_fullsized":"2024-09-21T03:39:04.402307+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.418906+0000","last_change":"2024-09-21T03:39:05.430619+0000","last_active":"2024-09-21T03:39:06.418906+0000","last_peered":"2024-09-21T03:39:06.418906+0000","last_clean":"2024-09-21T03:39:06.418906+0000","last_became_active":"2024-09-21T03:39:05.429959+0000","last_became_peered":"2024-09-21T03:39:05.429959+0000","last_unstale":"2024-09-21T03:39:06.418906+0000","last_undegraded":"2024-09-21T03:39:06.418906+0000","last_fullsized":"2024-09-21T03:39:06.418906+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:26:04.045027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.569279+0000","last_change":"2024-09-21T03:39:05.422391+0000","last_active":"2024-09-21T03:39:06.569279+0000","last_peered":"2024-09-21T03:39:06.569279+0000","last_clean":"2024-09-21T03:39:06.569279+0000","last_became_active":"2024-09-21T03:39:05.422073+0000","last_became_peered":"2024-09-21T03:39:05.422073+0000","last_unstale":"2024-09-21T03:39:06.569279+0000","last_undegraded":"2024-09-21T03:39:06.569279+0000","last_fullsized":"2024-09-21T03:39:06.569279+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:14:22.373008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.570338+0000","last_change":"2024-09-21T03:39:05.430675+0000","last_active":"2024-09-21T03:39:06.570338+0000","last_peered":"2024-09-21T03:39:06.570338+0000","last_clean":"2024-09-21T03:39:06.570338+0000","last_became_active":"2024-09-21T03:39:05.430203+0000","last_became_peered":"2024-09-21T03:39:05.430203+0000","last_unstale":"2024-09-21T03:39:06.570338+0000","last_undegraded":"2024-09-21T03:39:06.570338+0000","last_fullsized":"2024-09-21T03:39:06.570338+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:27:11.780682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"81'4","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.570214+0000","last_change":"2024-09-21T03:39:04.406626+0000","last_active":"2024-09-21T03:39:06.570214+0000","last_peered":"2024-09-21T03:39:06.570214+0000","last_clean":"2024-09-21T03:39:06.570214+0000","last_became_active":"2024-09-21T03:39:04.406318+0000","last_became_peered":"2024-09-21T03:39:04.406318+0000","last_unstale":"2024-09-21T03:39:06.570214+0000","last_undegraded":"2024-09-21T03:39:06.570214+0000","last_fullsized":"2024-09-21T03:39:06.570214+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_clean_scrub_stamp":"2024-09-21T03:39:03.377315+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:00:13.859613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.569891+0000","last_change":"2024-09-21T03:39:05.428305+0000","last_active":"2024-09-21T03:39:06.569891+0000","last_peered":"2024-09-21T03:39:06.569891+0000","last_clean":"2024-09-21T03:39:06.569891+0000","last_became_active":"2024-09-21T03:39:05.427992+0000","last_became_peered":"2024-09-21T03:39:05.427992+0000","last_unstale":"2024-09-21T03:39:06.569891+0000","last_undegraded":"2024-09-21T03:39:06.569891+0000","last_fullsized":"2024-09-21T03:39:06.569891+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:15:23.391245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1e","version":"81'1","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.570008+0000","last_change":"2024-09-21T03:39:04.411255+0000","last_active":"2024-09-21T03:39:06.570008+0000","last_peered":"2024-09-21T03:39:06.570008+0000","last_clean":"2024-09-21T03:39:06.570008+0000","last_became_active":"2024-09-21T03:39:04.410962+0000","last_became_peered":"2024-09-21T03:39:04.410962+0000","last_unstale":"2024-09-21T03:39:06.570008+0000","last_undegraded":"2024-09-21T03:39:06.570008+0000","last_fullsized":"2024-09-21T03:39:06.570008+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_clean_scrub_stamp":"2024-09-21T03:39:03.377315+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:27:36.848833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.569439+0000","last_change":"2024-09-21T03:39:05.415000+0000","last_active":"2024-09-21T03:39:06.569439+0000","last_peered":"2024-09-21T03:39:06.569439+0000","last_clean":"2024-09-21T03:39:06.569439+0000","last_became_active":"2024-09-21T03:39:05.414695+0000","last_became_peered":"2024-09-21T03:39:05.414695+0000","last_unstale":"2024-09-21T03:39:06.569439+0000","last_undegraded":"2024-09-21T03:39:06.569439+0000","last_fullsized":"2024-09-21T03:39:06.569439+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:58:09.229781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T03:39:04.413513+0000","last_change":"2024-09-21T03:39:04.404636+0000","last_active":"2024-09-21T03:39:04.413513+0000","last_peered":"2024-09-21T03:39:04.413513+0000","last_clean":"2024-09-21T03:39:04.413513+0000","last_became_active":"2024-09-21T03:39:04.404270+0000","last_became_peered":"2024-09-21T03:39:04.404270+0000","last_unstale":"2024-09-21T03:39:04.413513+0000","last_undegraded":"2024-09-21T03:39:04.413513+0000","last_fullsized":"2024-09-21T03:39:04.413513+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_clean_scrub_stamp":"2024-09-21T03:39:03.377315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:09:17.529140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.569256+0000","last_change":"2024-09-21T03:39:05.425119+0000","last_active":"2024-09-21T03:39:06.569256+0000","last_peered":"2024-09-21T03:39:06.569256+0000","last_clean":"2024-09-21T03:39:06.569256+0000","last_became_active":"2024-09-21T03:39:05.424804+0000","last_became_peered":"2024-09-21T03:39:05.424804+0000","last_unstale":"2024-09-21T03:39:06.569256+0000","last_undegraded":"2024-09-21T03:39:06.569256+0000","last_fullsized":"2024-09-21T03:39:06.569256+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:13:42.059728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"81'2","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.570641+0000","last_change":"2024-09-21T03:39:04.407381+0000","last_active":"2024-09-21T03:39:06.570641+0000","last_peered":"2024-09-21T03:39:06.570641+0000","last_clean":"2024-09-21T03:39:06.570641+0000","last_became_active":"2024-09-21T03:39:04.407044+0000","last_became_peered":"2024-09-21T03:39:04.407044+0000","last_unstale":"2024-09-21T03:39:06.570641+0000","last_undegraded":"2024-09-21T03:39:06.570641+0000","last_fullsized":"2024-09-21T03:39:06.570641+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_clean_scrub_stamp":"2024-09-21T03:39:03.377315+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:14:21.367112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.415145+0000","last_change":"2024-09-21T03:39:05.425026+0000","last_active":"2024-09-21T03:39:06.415145+0000","last_peered":"2024-09-21T03:39:06.415145+0000","last_clean":"2024-09-21T03:39:06.415145+0000","last_became_active":"2024-09-21T03:39:05.424667+0000","last_became_peered":"2024-09-21T03:39:05.424667+0000","last_unstale":"2024-09-21T03:39:06.415145+0000","last_undegraded":"2024-09-21T03:39:06.415145+0000","last_fullsized":"2024-09-21T03:39:06.415145+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:46:01.041621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.1b","version":"81'1","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.570542+0000","last_change":"2024-09-21T03:39:04.414608+0000","last_active":"2024-09-21T03:39:06.570542+0000","last_peered":"2024-09-21T03:39:06.570542+0000","last_clean":"2024-09-21T03:39:06.570542+0000","last_became_active":"2024-09-21T03:39:04.414274+0000","last_became_peered":"2024-09-21T03:39:04.414274+0000","last_unstale":"2024-09-21T03:39:06.570542+0000","last_undegraded":"2024-09-21T03:39:06.570542+0000","last_fullsized":"2024-09-21T03:39:06.570542+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_clean_scrub_stamp":"2024-09-21T03:39:03.377315+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:13:41.053832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.419607+0000","last_change":"2024-09-21T03:39:05.434557+0000","last_active":"2024-09-21T03:39:06.419607+0000","last_peered":"2024-09-21T03:39:06.419607+0000","last_clean":"2024-09-21T03:39:06.419607+0000","last_became_active":"2024-09-21T03:39:05.430126+0000","last_became_peered":"2024-09-21T03:39:05.430126+0000","last_unstale":"2024-09-21T03:39:06.419607+0000","last_undegraded":"2024-09-21T03:39:06.419607+0000","last_fullsized":"2024-09-21T03:39:06.419607+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:27:34.908887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"81'1","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.570010+0000","last_change":"2024-09-21T03:39:04.397543+0000","last_active":"2024-09-21T03:39:06.570010+0000","last_peered":"2024-09-21T03:39:06.570010+0000","last_clean":"2024-09-21T03:39:06.570010+0000","last_became_active":"2024-09-21T03:39:04.397220+0000","last_became_peered":"2024-09-21T03:39:04.397220+0000","last_unstale":"2024-09-21T03:39:06.570010+0000","last_undegraded":"2024-09-21T03:39:06.570010+0000","last_fullsized":"2024-09-21T03:39:06.570010+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_clean_scrub_stamp":"2024-09-21T03:39:03.377315+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:21:47.012786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.570317+0000","last_change":"2024-09-21T03:39:05.413837+0000","last_active":"2024-09-21T03:39:06.570317+0000","last_peered":"2024-09-21T03:39:06.570317+0000","last_clean":"2024-09-21T03:39:06.570317+0000","last_became_active":"2024-09-21T03:39:05.413498+0000","last_became_peered":"2024-09-21T03:39:05.413498+0000","last_unstale":"2024-09-21T03:39:06.570317+0000","last_undegraded":"2024-09-21T03:39:06.570317+0000","last_fullsized":"2024-09-21T03:39:06.570317+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:33:39.426980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.413804+0000","last_change":"2024-09-21T03:39:04.408211+0000","last_active":"2024-09-21T03:39:06.413804+0000","last_peered":"2024-09-21T03:39:06.413804+0000","last_clean":"2024-09-21T03:39:06.413804+0000","last_became_active":"2024-09-21T03:39:04.404927+0000","last_became_peered":"2024-09-21T03:39:04.404927+0000","last_unstale":"2024-09-21T03:39:06.413804+0000","last_undegraded":"2024-09-21T03:39:06.413804+0000","last_fullsized":"2024-09-21T03:39:06.413804+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_clean_scrub_stamp":"2024-09-21T03:39:03.377315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:53:29.829387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.569462+0000","last_change":"2024-09-21T03:39:05.419988+0000","last_active":"2024-09-21T03:39:06.569462+0000","last_peered":"2024-09-21T03:39:06.569462+0000","last_clean":"2024-09-21T03:39:06.569462+0000","last_became_active":"2024-09-21T03:39:05.419693+0000","last_became_peered":"2024-09-21T03:39:05.419693+0000","last_unstale":"2024-09-21T03:39:06.569462+0000","last_undegraded":"2024-09-21T03:39:06.569462+0000","last_fullsized":"2024-09-21T03:39:06.569462+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:08:48.101749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.418856+0000","last_change":"2024-09-21T03:39:04.410302+0000","last_active":"2024-09-21T03:39:06.418856+0000","last_peered":"2024-09-21T03:39:06.418856+0000","last_clean":"2024-09-21T03:39:06.418856+0000","last_became_active":"2024-09-21T03:39:04.409959+0000","last_became_peered":"2024-09-21T03:39:04.409959+0000","last_unstale":"2024-09-21T03:39:06.418856+0000","last_undegraded":"2024-09-21T03:39:06.418856+0000","last_fullsized":"2024-09-21T03:39:06.418856+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_clean_scrub_stamp":"2024-09-21T03:39:03.377315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:18:35.050119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.419019+0000","last_change":"2024-09-21T03:39:05.429795+0000","last_active":"2024-09-21T03:39:06.419019+0000","last_peered":"2024-09-21T03:39:06.419019+0000","last_clean":"2024-09-21T03:39:06.419019+0000","last_became_active":"2024-09-21T03:39:05.426181+0000","last_became_peered":"2024-09-21T03:39:05.426181+0000","last_unstale":"2024-09-21T03:39:06.419019+0000","last_undegraded":"2024-09-21T03:39:06.419019+0000","last_fullsized":"2024-09-21T03:39:06.419019+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:38:28.792089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.570075+0000","last_change":"2024-09-21T03:39:04.410453+0000","last_active":"2024-09-21T03:39:06.570075+0000","last_peered":"2024-09-21T03:39:06.570075+0000","last_clean":"2024-09-21T03:39:06.570075+0000","last_became_active":"2024-09-21T03:39:04.410082+0000","last_became_peered":"2024-09-21T03:39:04.410082+0000","last_unstale":"2024-09-21T03:39:06.570075+0000","last_undegraded":"2024-09-21T03:39:06.570075+0000","last_fullsized":"2024-09-21T03:39:06.570075+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_clean_scrub_stamp":"2024-09-21T03:39:03.377315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:56:14.520557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.569202+0000","last_change":"2024-09-21T03:39:05.422217+0000","last_active":"2024-09-21T03:39:06.569202+0000","last_peered":"2024-09-21T03:39:06.569202+0000","last_clean":"2024-09-21T03:39:06.569202+0000","last_became_active":"2024-09-21T03:39:05.421890+0000","last_became_peered":"2024-09-21T03:39:05.421890+0000","last_unstale":"2024-09-21T03:39:06.569202+0000","last_undegraded":"2024-09-21T03:39:06.569202+0000","last_fullsized":"2024-09-21T03:39:06.569202+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:40:45.946118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T03:39:04.414955+0000","last_change":"2024-09-21T03:39:04.411385+0000","last_active":"2024-09-21T03:39:04.414955+0000","last_peered":"2024-09-21T03:39:04.414955+0000","last_clean":"2024-09-21T03:39:04.414955+0000","last_became_active":"2024-09-21T03:39:04.410963+0000","last_became_peered":"2024-09-21T03:39:04.410963+0000","last_unstale":"2024-09-21T03:39:04.414955+0000","last_undegraded":"2024-09-21T03:39:04.414955+0000","last_fullsized":"2024-09-21T03:39:04.414955+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_clean_scrub_stamp":"2024-09-21T03:39:03.377315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:53:58.324899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.414821+0000","last_change":"2024-09-21T03:39:05.419517+0000","last_active":"2024-09-21T03:39:06.414821+0000","last_peered":"2024-09-21T03:39:06.414821+0000","last_clean":"2024-09-21T03:39:06.414821+0000","last_became_active":"2024-09-21T03:39:05.418599+0000","last_became_peered":"2024-09-21T03:39:05.418599+0000","last_unstale":"2024-09-21T03:39:06.414821+0000","last_undegraded":"2024-09-21T03:39:06.414821+0000","last_fullsized":"2024-09-21T03:39:06.414821+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:59:45.194584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.15","version":"81'3","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.413935+0000","last_change":"2024-09-21T03:39:04.407595+0000","last_active":"2024-09-21T03:39:06.413935+0000","last_peered":"2024-09-21T03:39:06.413935+0000","last_clean":"2024-09-21T03:39:06.413935+0000","last_became_active":"2024-09-21T03:39:04.407278+0000","last_became_peered":"2024-09-21T03:39:04.407278+0000","last_unstale":"2024-09-21T03:39:06.413935+0000","last_undegraded":"2024-09-21T03:39:06.413935+0000","last_fullsized":"2024-09-21T03:39:06.413935+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_clean_scrub_stamp":"2024-09-21T03:39:03.377315+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:41:01.617219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.414516+0000","last_change":"2024-09-21T03:39:05.419678+0000","last_active":"2024-09-21T03:39:06.414516+0000","last_peered":"2024-09-21T03:39:06.414516+0000","last_clean":"2024-09-21T03:39:06.414516+0000","last_became_active":"2024-09-21T03:39:05.419119+0000","last_became_peered":"2024-09-21T03:39:05.419119+0000","last_unstale":"2024-09-21T03:39:06.414516+0000","last_undegraded":"2024-09-21T03:39:06.414516+0000","last_fullsized":"2024-09-21T03:39:06.414516+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:03:30.086684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.14","version":"81'6","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.569939+0000","last_change":"2024-09-21T03:39:04.396680+0000","last_active":"2024-09-21T03:39:06.569939+0000","last_peered":"2024-09-21T03:39:06.569939+0000","last_clean":"2024-09-21T03:39:06.569939+0000","last_became_active":"2024-09-21T03:39:04.396344+0000","last_became_peered":"2024-09-21T03:39:04.396344+0000","last_unstale":"2024-09-21T03:39:06.569939+0000","last_undegraded":"2024-09-21T03:39:06.569939+0000","last_fullsized":"2024-09-21T03:39:06.569939+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_clean_scrub_stamp":"2024-09-21T03:39:03.377315+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:12:41.701014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.569672+0000","last_change":"2024-09-21T03:39:05.423246+0000","last_active":"2024-09-21T03:39:06.569672+0000","last_peered":"2024-09-21T03:39:06.569672+0000","last_clean":"2024-09-21T03:39:06.569672+0000","last_became_active":"2024-09-21T03:39:05.422924+0000","last_became_peered":"2024-09-21T03:39:05.422924+0000","last_unstale":"2024-09-21T03:39:06.569672+0000","last_undegraded":"2024-09-21T03:39:06.569672+0000","last_fullsized":"2024-09-21T03:39:06.569672+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:51:42.048958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T03:39:04.407961+0000","last_change":"2024-09-21T03:39:04.402594+0000","last_active":"2024-09-21T03:39:04.407961+0000","last_peered":"2024-09-21T03:39:04.407961+0000","last_clean":"2024-09-21T03:39:04.407961+0000","last_became_active":"2024-09-21T03:39:04.402242+0000","last_became_peered":"2024-09-21T03:39:04.402242+0000","last_unstale":"2024-09-21T03:39:04.407961+0000","last_undegraded":"2024-09-21T03:39:04.407961+0000","last_fullsized":"2024-09-21T03:39:04.407961+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_clean_scrub_stamp":"2024-09-21T03:39:03.377315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:23:27.346296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-09-21T03:39:04.401407+0000","last_change":"2024-09-21T03:39:04.401407+0000","last_active":"2024-09-21T03:39:04.383211+0000","last_peered":"2024-09-21T03:39:04.383211+0000","last_clean":"2024-09-21T03:39:04.383211+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:39:04.401407+0000","last_undegraded":"2024-09-21T03:39:04.401407+0000","last_fullsized":"2024-09-21T03:39:04.401407+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.569717+0000","last_change":"2024-09-21T03:39:04.409288+0000","last_active":"2024-09-21T03:39:06.569717+0000","last_peered":"2024-09-21T03:39:06.569717+0000","last_clean":"2024-09-21T03:39:06.569717+0000","last_became_active":"2024-09-21T03:39:04.408965+0000","last_became_peered":"2024-09-21T03:39:04.408965+0000","last_unstale":"2024-09-21T03:39:06.569717+0000","last_undegraded":"2024-09-21T03:39:06.569717+0000","last_fullsized":"2024-09-21T03:39:06.569717+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_clean_scrub_stamp":"2024-09-21T03:39:03.377315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:01:17.101953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.570597+0000","last_change":"2024-09-21T03:39:05.423231+0000","last_active":"2024-09-21T03:39:06.570597+0000","last_peered":"2024-09-21T03:39:06.570597+0000","last_clean":"2024-09-21T03:39:06.570597+0000","last_became_active":"2024-09-21T03:39:05.422931+0000","last_became_peered":"2024-09-21T03:39:05.422931+0000","last_unstale":"2024-09-21T03:39:06.570597+0000","last_undegraded":"2024-09-21T03:39:06.570597+0000","last_fullsized":"2024-09-21T03:39:06.570597+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:35:18.019438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.570103+0000","last_change":"2024-09-21T03:39:04.405877+0000","last_active":"2024-09-21T03:39:06.570103+0000","last_peered":"2024-09-21T03:39:06.570103+0000","last_clean":"2024-09-21T03:39:06.570103+0000","last_became_active":"2024-09-21T03:39:04.405583+0000","last_became_peered":"2024-09-21T03:39:04.405583+0000","last_unstale":"2024-09-21T03:39:06.570103+0000","last_undegraded":"2024-09-21T03:39:06.570103+0000","last_fullsized":"2024-09-21T03:39:06.570103+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_clean_scrub_stamp":"2024-09-21T03:39:03.377315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:03:56.352603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-09-21T03:39:04.402886+0000","last_change":"2024-09-21T03:39:04.402886+0000","last_active":"2024-09-21T03:39:04.383211+0000","last_peered":"2024-09-21T03:39:04.383211+0000","last_clean":"2024-09-21T03:39:04.383211+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:39:04.402886+0000","last_undegraded":"2024-09-21T03:39:04.402886+0000","last_fullsized":"2024-09-21T03:39:04.402886+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.10","version":"81'8","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.413938+0000","last_change":"2024-09-21T03:39:04.411109+0000","last_active":"2024-09-21T03:39:06.413938+0000","last_peered":"2024-09-21T03:39:06.413938+0000","last_clean":"2024-09-21T03:39:06.413938+0000","last_became_active":"2024-09-21T03:39:04.410744+0000","last_became_peered":"2024-09-21T03:39:04.410744+0000","last_unstale":"2024-09-21T03:39:06.413938+0000","last_undegraded":"2024-09-21T03:39:06.413938+0000","last_fullsized":"2024-09-21T03:39:06.413938+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_clean_scrub_stamp":"2024-09-21T03:39:03.377315+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:13:48.637151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.570028+0000","last_change":"2024-09-21T03:39:05.432432+0000","last_active":"2024-09-21T03:39:06.570028+0000","last_peered":"2024-09-21T03:39:06.570028+0000","last_clean":"2024-09-21T03:39:06.570028+0000","last_became_active":"2024-09-21T03:39:05.432136+0000","last_became_peered":"2024-09-21T03:39:05.432136+0000","last_unstale":"2024-09-21T03:39:06.570028+0000","last_undegraded":"2024-09-21T03:39:06.570028+0000","last_fullsized":"2024-09-21T03:39:06.570028+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:10:20.019199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.f","version":"81'1","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.570192+0000","last_change":"2024-09-21T03:39:04.403666+0000","last_active":"2024-09-21T03:39:06.570192+0000","last_peered":"2024-09-21T03:39:06.570192+0000","last_clean":"2024-09-21T03:39:06.570192+0000","last_became_active":"2024-09-21T03:39:04.403310+0000","last_became_peered":"2024-09-21T03:39:04.403310+0000","last_unstale":"2024-09-21T03:39:06.570192+0000","last_undegraded":"2024-09-21T03:39:06.570192+0000","last_fullsized":"2024-09-21T03:39:06.570192+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_clean_scrub_stamp":"2024-09-21T03:39:03.377315+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:18:35.050119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.418952+0000","last_change":"2024-09-21T03:39:05.432320+0000","last_active":"2024-09-21T03:39:06.418952+0000","last_peered":"2024-09-21T03:39:06.418952+0000","last_clean":"2024-09-21T03:39:06.418952+0000","last_became_active":"2024-09-21T03:39:05.431965+0000","last_became_peered":"2024-09-21T03:39:05.431965+0000","last_unstale":"2024-09-21T03:39:06.418952+0000","last_undegraded":"2024-09-21T03:39:06.418952+0000","last_fullsized":"2024-09-21T03:39:06.418952+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:51:42.048958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.e","version":"81'2","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.569527+0000","last_change":"2024-09-21T03:39:04.403354+0000","last_active":"2024-09-21T03:39:06.569527+0000","last_peered":"2024-09-21T03:39:06.569527+0000","last_clean":"2024-09-21T03:39:06.569527+0000","last_became_active":"2024-09-21T03:39:04.401729+0000","last_became_peered":"2024-09-21T03:39:04.401729+0000","last_unstale":"2024-09-21T03:39:06.569527+0000","last_undegraded":"2024-09-21T03:39:06.569527+0000","last_fullsized":"2024-09-21T03:39:06.569527+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_clean_scrub_stamp":"2024-09-21T03:39:03.377315+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:05:16.494460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.414672+0000","last_change":"2024-09-21T03:39:05.421453+0000","last_active":"2024-09-21T03:39:06.414672+0000","last_peered":"2024-09-21T03:39:06.414672+0000","last_clean":"2024-09-21T03:39:06.414672+0000","last_became_active":"2024-09-21T03:39:05.420867+0000","last_became_peered":"2024-09-21T03:39:05.420867+0000","last_unstale":"2024-09-21T03:39:06.414672+0000","last_undegraded":"2024-09-21T03:39:06.414672+0000","last_fullsized":"2024-09-21T03:39:06.414672+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:47:58.257466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.d","version":"81'1","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.570103+0000","last_change":"2024-09-21T03:39:04.408221+0000","last_active":"2024-09-21T03:39:06.570103+0000","last_peered":"2024-09-21T03:39:06.570103+0000","last_clean":"2024-09-21T03:39:06.570103+0000","last_became_active":"2024-09-21T03:39:04.407900+0000","last_became_peered":"2024-09-21T03:39:04.407900+0000","last_unstale":"2024-09-21T03:39:06.570103+0000","last_undegraded":"2024-09-21T03:39:06.570103+0000","last_fullsized":"2024-09-21T03:39:06.570103+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_clean_scrub_stamp":"2024-09-21T03:39:03.377315+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:26:11.862449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.570468+0000","last_change":"2024-09-21T03:39:05.423205+0000","last_active":"2024-09-21T03:39:06.570468+0000","last_peered":"2024-09-21T03:39:06.570468+0000","last_clean":"2024-09-21T03:39:06.570468+0000","last_became_active":"2024-09-21T03:39:05.422867+0000","last_became_peered":"2024-09-21T03:39:05.422867+0000","last_unstale":"2024-09-21T03:39:06.570468+0000","last_undegraded":"2024-09-21T03:39:06.570468+0000","last_fullsized":"2024-09-21T03:39:06.570468+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:33:29.949291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.570257+0000","last_change":"2024-09-21T03:39:04.401682+0000","last_active":"2024-09-21T03:39:06.570257+0000","last_peered":"2024-09-21T03:39:06.570257+0000","last_clean":"2024-09-21T03:39:06.570257+0000","last_became_active":"2024-09-21T03:39:04.401354+0000","last_became_peered":"2024-09-21T03:39:04.401354+0000","last_unstale":"2024-09-21T03:39:06.570257+0000","last_undegraded":"2024-09-21T03:39:06.570257+0000","last_fullsized":"2024-09-21T03:39:06.570257+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_clean_scrub_stamp":"2024-09-21T03:39:03.377315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:02:56.975258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.570149+0000","last_change":"2024-09-21T03:39:05.424271+0000","last_active":"2024-09-21T03:39:06.570149+0000","last_peered":"2024-09-21T03:39:06.570149+0000","last_clean":"2024-09-21T03:39:06.570149+0000","last_became_active":"2024-09-21T03:39:05.423967+0000","last_became_peered":"2024-09-21T03:39:05.423967+0000","last_unstale":"2024-09-21T03:39:06.570149+0000","last_undegraded":"2024-09-21T03:39:06.570149+0000","last_fullsized":"2024-09-21T03:39:06.570149+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:29:22.009760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"81'2","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.418688+0000","last_change":"2024-09-21T03:39:04.406956+0000","last_active":"2024-09-21T03:39:06.418688+0000","last_peered":"2024-09-21T03:39:06.418688+0000","last_clean":"2024-09-21T03:39:06.418688+0000","last_became_active":"2024-09-21T03:39:04.406604+0000","last_became_peered":"2024-09-21T03:39:04.406604+0000","last_unstale":"2024-09-21T03:39:06.418688+0000","last_undegraded":"2024-09-21T03:39:06.418688+0000","last_fullsized":"2024-09-21T03:39:06.418688+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_clean_scrub_stamp":"2024-09-21T03:39:03.377315+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:56:14.520557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.570451+0000","last_change":"2024-09-21T03:39:05.424012+0000","last_active":"2024-09-21T03:39:06.570451+0000","last_peered":"2024-09-21T03:39:06.570451+0000","last_clean":"2024-09-21T03:39:06.570451+0000","last_became_active":"2024-09-21T03:39:05.423713+0000","last_became_peered":"2024-09-21T03:39:05.423713+0000","last_unstale":"2024-09-21T03:39:06.570451+0000","last_undegraded":"2024-09-21T03:39:06.570451+0000","last_fullsized":"2024-09-21T03:39:06.570451+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:30:06.424731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.413940+0000","last_change":"2024-09-21T03:39:04.395490+0000","last_active":"2024-09-21T03:39:06.413940+0000","last_peered":"2024-09-21T03:39:06.413940+0000","last_clean":"2024-09-21T03:39:06.413940+0000","last_became_active":"2024-09-21T03:39:04.394934+0000","last_became_peered":"2024-09-21T03:39:04.394934+0000","last_unstale":"2024-09-21T03:39:06.413940+0000","last_undegraded":"2024-09-21T03:39:06.413940+0000","last_fullsized":"2024-09-21T03:39:06.413940+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_clean_scrub_stamp":"2024-09-21T03:39:03.377315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:30:01.896910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.569691+0000","last_change":"2024-09-21T03:39:05.418742+0000","last_active":"2024-09-21T03:39:06.569691+0000","last_peered":"2024-09-21T03:39:06.569691+0000","last_clean":"2024-09-21T03:39:06.569691+0000","last_became_active":"2024-09-21T03:39:05.418366+0000","last_became_peered":"2024-09-21T03:39:05.418366+0000","last_unstale":"2024-09-21T03:39:06.569691+0000","last_undegraded":"2024-09-21T03:39:06.569691+0000","last_fullsized":"2024-09-21T03:39:06.569691+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:09:46.888780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.418986+0000","last_change":"2024-09-21T03:39:04.408050+0000","last_active":"2024-09-21T03:39:06.418986+0000","last_peered":"2024-09-21T03:39:06.418986+0000","last_clean":"2024-09-21T03:39:06.418986+0000","last_became_active":"2024-09-21T03:39:04.407753+0000","last_became_peered":"2024-09-21T03:39:04.407753+0000","last_unstale":"2024-09-21T03:39:06.418986+0000","last_undegraded":"2024-09-21T03:39:06.418986+0000","last_fullsized":"2024-09-21T03:39:06.418986+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_clean_scrub_stamp":"2024-09-21T03:39:03.377315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:05:16.494460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.569516+0000","last_change":"2024-09-21T03:39:05.431029+0000","last_active":"2024-09-21T03:39:06.569516+0000","last_peered":"2024-09-21T03:39:06.569516+0000","last_clean":"2024-09-21T03:39:06.569516+0000","last_became_active":"2024-09-21T03:39:05.427562+0000","last_became_peered":"2024-09-21T03:39:05.427562+0000","last_unstale":"2024-09-21T03:39:06.569516+0000","last_undegraded":"2024-09-21T03:39:06.569516+0000","last_fullsized":"2024-09-21T03:39:06.569516+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:20:35.883679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T03:39:04.414902+0000","last_change":"2024-09-21T03:39:04.407689+0000","last_active":"2024-09-21T03:39:04.414902+0000","last_peered":"2024-09-21T03:39:04.414902+0000","last_clean":"2024-09-21T03:39:04.414902+0000","last_became_active":"2024-09-21T03:39:04.407356+0000","last_became_peered":"2024-09-21T03:39:04.407356+0000","last_unstale":"2024-09-21T03:39:04.414902+0000","last_undegraded":"2024-09-21T03:39:04.414902+0000","last_fullsized":"2024-09-21T03:39:04.414902+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_clean_scrub_stamp":"2024-09-21T03:39:03.377315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:25:15.786223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-09-21T03:39:04.400780+0000","last_change":"2024-09-21T03:39:04.400780+0000","last_active":"2024-09-21T03:39:04.383211+0000","last_peered":"2024-09-21T03:39:04.383211+0000","last_clean":"2024-09-21T03:39:04.383211+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:39:04.400780+0000","last_undegraded":"2024-09-21T03:39:04.400780+0000","last_fullsized":"2024-09-21T03:39:04.400780+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.6","version":"82'1","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.417795+0000","last_change":"2024-09-21T03:39:04.411532+0000","last_active":"2024-09-21T03:39:06.417795+0000","last_peered":"2024-09-21T03:39:06.417795+0000","last_clean":"2024-09-21T03:39:06.417795+0000","last_became_active":"2024-09-21T03:39:04.411189+0000","last_became_peered":"2024-09-21T03:39:04.411189+0000","last_unstale":"2024-09-21T03:39:06.417795+0000","last_undegraded":"2024-09-21T03:39:06.417795+0000","last_fullsized":"2024-09-21T03:39:06.417795+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_clean_scrub_stamp":"2024-09-21T03:39:03.377315+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:31:32.273309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.569653+0000","last_change":"2024-09-21T03:39:05.414196+0000","last_active":"2024-09-21T03:39:06.569653+0000","last_peered":"2024-09-21T03:39:06.569653+0000","last_clean":"2024-09-21T03:39:06.569653+0000","last_became_active":"2024-09-21T03:39:05.413871+0000","last_became_peered":"2024-09-21T03:39:05.413871+0000","last_unstale":"2024-09-21T03:39:06.569653+0000","last_undegraded":"2024-09-21T03:39:06.569653+0000","last_fullsized":"2024-09-21T03:39:06.569653+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:27:34.908887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T03:39:04.413543+0000","last_change":"2024-09-21T03:39:04.405352+0000","last_active":"2024-09-21T03:39:04.413543+0000","last_peered":"2024-09-21T03:39:04.413543+0000","last_clean":"2024-09-21T03:39:04.413543+0000","last_became_active":"2024-09-21T03:39:04.405020+0000","last_became_peered":"2024-09-21T03:39:04.405020+0000","last_unstale":"2024-09-21T03:39:04.413543+0000","last_undegraded":"2024-09-21T03:39:04.413543+0000","last_fullsized":"2024-09-21T03:39:04.413543+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_clean_scrub_stamp":"2024-09-21T03:39:03.377315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:37:33.420266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.415196+0000","last_change":"2024-09-21T03:39:05.415347+0000","last_active":"2024-09-21T03:39:06.415196+0000","last_peered":"2024-09-21T03:39:06.415196+0000","last_clean":"2024-09-21T03:39:06.415196+0000","last_became_active":"2024-09-21T03:39:05.414476+0000","last_became_peered":"2024-09-21T03:39:05.414476+0000","last_unstale":"2024-09-21T03:39:06.415196+0000","last_undegraded":"2024-09-21T03:39:06.415196+0000","last_fullsized":"2024-09-21T03:39:06.415196+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:49:45.640520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.569666+0000","last_change":"2024-09-21T03:39:04.410318+0000","last_active":"2024-09-21T03:39:06.569666+0000","last_peered":"2024-09-21T03:39:06.569666+0000","last_clean":"2024-09-21T03:39:06.569666+0000","last_became_active":"2024-09-21T03:39:04.409989+0000","last_became_peered":"2024-09-21T03:39:04.409989+0000","last_unstale":"2024-09-21T03:39:06.569666+0000","last_undegraded":"2024-09-21T03:39:06.569666+0000","last_fullsized":"2024-09-21T03:39:06.569666+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_clean_scrub_stamp":"2024-09-21T03:39:03.377315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:49:27.896288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.570622+0000","last_change":"2024-09-21T03:39:05.421291+0000","last_active":"2024-09-21T03:39:06.570622+0000","last_peered":"2024-09-21T03:39:06.570622+0000","last_clean":"2024-09-21T03:39:06.570622+0000","last_became_active":"2024-09-21T03:39:05.420933+0000","last_became_peered":"2024-09-21T03:39:05.420933+0000","last_unstale":"2024-09-21T03:39:06.570622+0000","last_undegraded":"2024-09-21T03:39:06.570622+0000","last_fullsized":"2024-09-21T03:39:06.570622+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:18:54.425316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.569375+0000","last_change":"2024-09-21T03:39:04.405531+0000","last_active":"2024-09-21T03:39:06.569375+0000","last_peered":"2024-09-21T03:39:06.569375+0000","last_clean":"2024-09-21T03:39:06.569375+0000","last_became_active":"2024-09-21T03:39:04.405193+0000","last_became_peered":"2024-09-21T03:39:04.405193+0000","last_unstale":"2024-09-21T03:39:06.569375+0000","last_undegraded":"2024-09-21T03:39:06.569375+0000","last_fullsized":"2024-09-21T03:39:06.569375+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_clean_scrub_stamp":"2024-09-21T03:39:03.377315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:41:35.755494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.569966+0000","last_change":"2024-09-21T03:39:05.431541+0000","last_active":"2024-09-21T03:39:06.569966+0000","last_peered":"2024-09-21T03:39:06.569966+0000","last_clean":"2024-09-21T03:39:06.569966+0000","last_became_active":"2024-09-21T03:39:05.428130+0000","last_became_peered":"2024-09-21T03:39:05.428130+0000","last_unstale":"2024-09-21T03:39:06.569966+0000","last_undegraded":"2024-09-21T03:39:06.569966+0000","last_fullsized":"2024-09-21T03:39:06.569966+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:22:26.967805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.415539+0000","last_change":"2024-09-21T03:39:04.403462+0000","last_active":"2024-09-21T03:39:06.415539+0000","last_peered":"2024-09-21T03:39:06.415539+0000","last_clean":"2024-09-21T03:39:06.415539+0000","last_became_active":"2024-09-21T03:39:04.403114+0000","last_became_peered":"2024-09-21T03:39:04.403114+0000","last_unstale":"2024-09-21T03:39:06.415539+0000","last_undegraded":"2024-09-21T03:39:06.415539+0000","last_fullsized":"2024-09-21T03:39:06.415539+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_clean_scrub_stamp":"2024-09-21T03:39:03.377315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:38:07.041511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.419363+0000","last_change":"2024-09-21T03:39:05.429296+0000","last_active":"2024-09-21T03:39:06.419363+0000","last_peered":"2024-09-21T03:39:06.419363+0000","last_clean":"2024-09-21T03:39:06.419363+0000","last_became_active":"2024-09-21T03:39:05.426008+0000","last_became_peered":"2024-09-21T03:39:05.426008+0000","last_unstale":"2024-09-21T03:39:06.419363+0000","last_undegraded":"2024-09-21T03:39:06.419363+0000","last_fullsized":"2024-09-21T03:39:06.419363+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:16:42.786975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.568997+0000","last_change":"2024-09-21T03:39:04.407078+0000","last_active":"2024-09-21T03:39:06.568997+0000","last_peered":"2024-09-21T03:39:06.568997+0000","last_clean":"2024-09-21T03:39:06.568997+0000","last_became_active":"2024-09-21T03:39:04.406757+0000","last_became_peered":"2024-09-21T03:39:04.406757+0000","last_unstale":"2024-09-21T03:39:06.568997+0000","last_undegraded":"2024-09-21T03:39:06.568997+0000","last_fullsized":"2024-09-21T03:39:06.568997+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_clean_scrub_stamp":"2024-09-21T03:39:03.377315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:08:19.857227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.570315+0000","last_change":"2024-09-21T03:39:05.431328+0000","last_active":"2024-09-21T03:39:06.570315+0000","last_peered":"2024-09-21T03:39:06.570315+0000","last_clean":"2024-09-21T03:39:06.570315+0000","last_became_active":"2024-09-21T03:39:05.431007+0000","last_became_peered":"2024-09-21T03:39:05.431007+0000","last_unstale":"2024-09-21T03:39:06.570315+0000","last_undegraded":"2024-09-21T03:39:06.570315+0000","last_fullsized":"2024-09-21T03:39:06.570315+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:09:46.155413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.419086+0000","last_change":"2024-09-21T03:39:04.412555+0000","last_active":"2024-09-21T03:39:06.419086+0000","last_peered":"2024-09-21T03:39:06.419086+0000","last_clean":"2024-09-21T03:39:06.419086+0000","last_became_active":"2024-09-21T03:39:04.412258+0000","last_became_peered":"2024-09-21T03:39:04.412258+0000","last_unstale":"2024-09-21T03:39:06.419086+0000","last_undegraded":"2024-09-21T03:39:06.419086+0000","last_fullsized":"2024-09-21T03:39:06.419086+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_clean_scrub_stamp":"2024-09-21T03:39:03.377315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:01:13.191529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.570331+0000","last_change":"2024-09-21T03:39:05.418570+0000","last_active":"2024-09-21T03:39:06.570331+0000","last_peered":"2024-09-21T03:39:06.570331+0000","last_clean":"2024-09-21T03:39:06.570331+0000","last_became_active":"2024-09-21T03:39:05.418257+0000","last_became_peered":"2024-09-21T03:39:05.418257+0000","last_unstale":"2024-09-21T03:39:06.570331+0000","last_undegraded":"2024-09-21T03:39:06.570331+0000","last_fullsized":"2024-09-21T03:39:06.570331+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:38:35.279761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.415650+0000","last_change":"2024-09-21T03:39:04.414590+0000","last_active":"2024-09-21T03:39:06.415650+0000","last_peered":"2024-09-21T03:39:06.415650+0000","last_clean":"2024-09-21T03:39:06.415650+0000","last_became_active":"2024-09-21T03:39:04.414355+0000","last_became_peered":"2024-09-21T03:39:04.414355+0000","last_unstale":"2024-09-21T03:39:06.415650+0000","last_undegraded":"2024-09-21T03:39:06.415650+0000","last_fullsized":"2024-09-21T03:39:06.415650+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_clean_scrub_stamp":"2024-09-21T03:39:03.377315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:03:09.175309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":223,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T03:39:04.391409+0000","last_change":"2024-09-21T03:33:25.319776+0000","last_active":"2024-09-21T03:39:04.391409+0000","last_peered":"2024-09-21T03:39:04.391409+0000","last_clean":"2024-09-21T03:39:04.391409+0000","last_became_active":"2024-09-21T03:33:25.318399+0000","last_became_peered":"2024-09-21T03:33:25.318399+0000","last_unstale":"2024-09-21T03:39:04.391409+0000","last_undegraded":"2024-09-21T03:39:04.391409+0000","last_fullsized":"2024-09-21T03:39:04.391409+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:23.016672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:23.016672+0000","last_clean_scrub_stamp":"2024-09-21T03:33:23.016672+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:32:35.906114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":2376,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":36,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":33,"ondisk_log_size":33,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":1,"up_from":16,"seq":68719476808,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32720,"kb_used_data":1528,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339120,"statfs":{"total":96636764160,"available":96603258880,"internally_reserved":0,"allocated":1564672,"data_stored":1036315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509513,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36328,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335512,"statfs":{"total":96636764160,"available":96599564288,"internally_reserved":0,"allocated":1003520,"data_stored":460458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":36174283},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509512,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32184,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339656,"statfs":{"total":96636764160,"available":96603807744,"internally_reserved":0,"allocated":1015808,"data_stored":461522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509513,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32164,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339676,"statfs":{"total":96636764160,"available":96603828224,"internally_reserved":0,"allocated":995328,"data_stored":459806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509512,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32168,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339672,"statfs":{"total":96636764160,"available":96603824128,"internally_reserved":0,"allocated":999424,"data_stored":459828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542216,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36332,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335508,"statfs":{"total":96636764160,"available":96599560192,"internally_reserved":0,"allocated":1003520,"data_stored":460478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574920,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36912,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334928,"statfs":{"total":96636764160,"available":96598966272,"internally_reserved":0,"allocated":1593344,"data_stored":1050264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542217,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36328,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335512,"statfs":{"total":96636764160,"available":96599564288,"internally_reserved":0,"allocated":999424,"data_stored":459828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T03:39:09.862 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T03:39:10.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:10.012+0000 7f72b3230640 1 -- 172.21.15.38:0/933625265 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f72ac0b2e40 msgr2=0x7f72ac0b3250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:10.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:10.012+0000 7f72b3230640 1 --2- 172.21.15.38:0/933625265 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f72ac0b2e40 0x7f72ac0b3250 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f72ac0d5040 tx=0x7f72a402aa30 comp rx=0 tx=0).stop 2024-09-21T03:39:10.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:10.012+0000 7f72b3230640 1 -- 172.21.15.38:0/933625265 shutdown_connections 2024-09-21T03:39:10.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:10.012+0000 7f72b3230640 1 --2- 172.21.15.38:0/933625265 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f72ac0b2e40 0x7f72ac0b3250 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:10.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:10.012+0000 7f72b3230640 1 --2- 172.21.15.38:0/933625265 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f72ac162280 0x7f72ac162690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:10.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:10.012+0000 7f72b3230640 1 --2- 172.21.15.38:0/933625265 >> v2:172.21.15.195:3300/0 conn(0x7f72ac162f00 0x7f72ac0d24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:10.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:10.012+0000 7f72b3230640 1 -- 172.21.15.38:0/933625265 >> 172.21.15.38:0/933625265 conn(0x7f72ac0b6b20 msgr2=0x7f72ac0d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:10.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:10.012+0000 7f72b3230640 1 -- 172.21.15.38:0/933625265 shutdown_connections 2024-09-21T03:39:10.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:10.012+0000 7f72b3230640 1 -- 172.21.15.38:0/933625265 wait complete. 2024-09-21T03:39:10.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:10.014+0000 7f72b3230640 1 Processor -- start 2024-09-21T03:39:10.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:10.014+0000 7f72b3230640 1 -- start start 2024-09-21T03:39:10.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:10.015+0000 7f72b3230640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f72ac0b2e40 0x7f72ac1f89b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:10.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:10.015+0000 7f72b3230640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f72ac162280 0x7f72ac1f5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:10.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:10.015+0000 7f72b3230640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f72ac159550 0x7f72ac1eb360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:10.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:10.015+0000 7f72b3230640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f72ac0b3dd0 con 0x7f72ac0b2e40 2024-09-21T03:39:10.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:10.015+0000 7f72b3230640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f72ac0b39a0 con 0x7f72ac162280 2024-09-21T03:39:10.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:10.015+0000 7f72b3230640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f72ac0b3f50 con 0x7f72ac159550 2024-09-21T03:39:10.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:10.015+0000 7f72b0fcc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f72ac0b2e40 0x7f72ac1f89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:10.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:10.015+0000 7f72b0fcc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f72ac0b2e40 0x7f72ac1f89b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:52724/0 (socket says 172.21.15.38:52724) 2024-09-21T03:39:10.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:10.015+0000 7f72b0fcc640 1 -- 172.21.15.38:0/3990217326 learned_addr learned my addr 172.21.15.38:0/3990217326 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:39:10.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:10.016+0000 7f72b17cd640 1 --2- 172.21.15.38:0/3990217326 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f72ac159550 0x7f72ac1eb360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:10.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:10.016+0000 7f72abfff640 1 --2- 172.21.15.38:0/3990217326 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f72ac162280 0x7f72ac1f5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:10.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:10.016+0000 7f72b0fcc640 1 -- 172.21.15.38:0/3990217326 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f72ac159550 msgr2=0x7f72ac1eb360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:10.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:10.016+0000 7f72b0fcc640 1 --2- 172.21.15.38:0/3990217326 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f72ac159550 0x7f72ac1eb360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:10.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:10.016+0000 7f72b0fcc640 1 -- 172.21.15.38:0/3990217326 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f72ac162280 msgr2=0x7f72ac1f5cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:10.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:10.016+0000 7f72b0fcc640 1 --2- 172.21.15.38:0/3990217326 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f72ac162280 0x7f72ac1f5cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:10.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:10.016+0000 7f72b0fcc640 1 -- 172.21.15.38:0/3990217326 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72ac1f9200 con 0x7f72ac0b2e40 2024-09-21T03:39:10.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:10.016+0000 7f72b0fcc640 1 --2- 172.21.15.38:0/3990217326 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f72ac0b2e40 0x7f72ac1f89b0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f729c00d740 tx=0x7f729c00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:10.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:10.016+0000 7f72a97fa640 1 -- 172.21.15.38:0/3990217326 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f729c017070 con 0x7f72ac0b2e40 2024-09-21T03:39:10.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:10.016+0000 7f72a97fa640 1 -- 172.21.15.38:0/3990217326 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f729c009d20 con 0x7f72ac0b2e40 2024-09-21T03:39:10.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:10.016+0000 7f72a97fa640 1 -- 172.21.15.38:0/3990217326 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f729c0106c0 con 0x7f72ac0b2e40 2024-09-21T03:39:10.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:10.016+0000 7f72b3230640 1 -- 172.21.15.38:0/3990217326 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72ac1f94e0 con 0x7f72ac0b2e40 2024-09-21T03:39:10.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:10.016+0000 7f72b3230640 1 -- 172.21.15.38:0/3990217326 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72ac1f54d0 con 0x7f72ac0b2e40 2024-09-21T03:39:10.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:10.017+0000 7f72b3230640 1 -- 172.21.15.38:0/3990217326 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72ac162f00 con 0x7f72ac0b2e40 2024-09-21T03:39:10.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:10.018+0000 7f72a97fa640 1 -- 172.21.15.38:0/3990217326 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f729c01a070 con 0x7f72ac0b2e40 2024-09-21T03:39:10.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:10.018+0000 7f72a97fa640 1 --2- 172.21.15.38:0/3990217326 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7298078b90 0x7f729807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:10.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:10.018+0000 7f72a97fa640 1 -- 172.21.15.38:0/3990217326 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 6101+0+0 (secure 0 0 0) 0x7f729c09b0a0 con 0x7f72ac0b2e40 2024-09-21T03:39:10.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:10.019+0000 7f72abfff640 1 --2- 172.21.15.38:0/3990217326 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7298078b90 0x7f729807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:10.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:10.019+0000 7f72abfff640 1 --2- 172.21.15.38:0/3990217326 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7298078b90 0x7f729807b0a0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f72ac158e10 tx=0x7f72a0005d70 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:10.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:10.023+0000 7f72a97fa640 1 -- 172.21.15.38:0/3990217326 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f729c063350 con 0x7f72ac0b2e40 2024-09-21T03:39:10.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:10.205+0000 7f72b3230640 1 -- 172.21.15.38:0/3990217326 --> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f72ac1629b0 con 0x7f7298078b90 2024-09-21T03:39:10.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:10.210+0000 7f72a97fa640 1 -- 172.21.15.38:0/3990217326 <== mgr.4117 v2:172.21.15.195:6800/1631901344 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251482 (secure 0 0 0) 0x7f72ac1629b0 con 0x7f7298078b90 2024-09-21T03:39:10.214 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:39:10.215 INFO:teuthology.orchestra.run.smithi038.stderr:dumped all 2024-09-21T03:39:10.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:10.213+0000 7f72b3230640 1 -- 172.21.15.38:0/3990217326 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7298078b90 msgr2=0x7f729807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:10.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:10.213+0000 7f72b3230640 1 --2- 172.21.15.38:0/3990217326 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7298078b90 0x7f729807b0a0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f72ac158e10 tx=0x7f72a0005d70 comp rx=0 tx=0).stop 2024-09-21T03:39:10.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:10.213+0000 7f72b3230640 1 -- 172.21.15.38:0/3990217326 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f72ac0b2e40 msgr2=0x7f72ac1f89b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:10.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:10.213+0000 7f72b3230640 1 --2- 172.21.15.38:0/3990217326 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f72ac0b2e40 0x7f72ac1f89b0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f729c00d740 tx=0x7f729c00dc00 comp rx=0 tx=0).stop 2024-09-21T03:39:10.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:10.214+0000 7f72b3230640 1 -- 172.21.15.38:0/3990217326 shutdown_connections 2024-09-21T03:39:10.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:10.214+0000 7f72b3230640 1 --2- 172.21.15.38:0/3990217326 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7298078b90 0x7f729807b0a0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:10.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:10.214+0000 7f72b3230640 1 --2- 172.21.15.38:0/3990217326 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f72ac159550 0x7f72ac1eb360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:10.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:10.214+0000 7f72b3230640 1 --2- 172.21.15.38:0/3990217326 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f72ac162280 0x7f72ac1f5cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:10.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:10.214+0000 7f72b3230640 1 --2- 172.21.15.38:0/3990217326 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f72ac0b2e40 0x7f72ac1f89b0 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:10.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:10.214+0000 7f72b3230640 1 -- 172.21.15.38:0/3990217326 >> 172.21.15.38:0/3990217326 conn(0x7f72ac0b6b20 msgr2=0x7f72ac1eaaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:10.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:10.214+0000 7f72b3230640 1 -- 172.21.15.38:0/3990217326 shutdown_connections 2024-09-21T03:39:10.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:10.214+0000 7f72b3230640 1 -- 172.21.15.38:0/3990217326 wait complete. 2024-09-21T03:39:10.226 INFO:teuthology.orchestra.run.smithi038.stdout:{"pg_ready":true,"pg_map":{"version":248,"stamp":"2024-09-21T03:39:09.071943+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592744,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":137,"num_write_kb":1635,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":105,"ondisk_log_size":105,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":154,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":275136,"kb_used_data":8960,"kb_used_omap":12,"kb_used_meta":265971,"kb_avail":754699584,"statfs":{"total":773094113280,"available":772812374016,"internally_reserved":0,"allocated":9175040,"data_stored":4848499,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12748,"internal_metadata":272354868},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2376,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":36,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.687763"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-09-21T03:39:04.404050+0000","last_change":"2024-09-21T03:39:04.404050+0000","last_active":"2024-09-21T03:39:04.383211+0000","last_peered":"2024-09-21T03:39:04.383211+0000","last_clean":"2024-09-21T03:39:04.383211+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:39:04.404050+0000","last_undegraded":"2024-09-21T03:39:04.404050+0000","last_fullsized":"2024-09-21T03:39:04.404050+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.419248+0000","last_change":"2024-09-21T03:39:05.431615+0000","last_active":"2024-09-21T03:39:06.419248+0000","last_peered":"2024-09-21T03:39:06.419248+0000","last_clean":"2024-09-21T03:39:06.419248+0000","last_became_active":"2024-09-21T03:39:05.429700+0000","last_became_peered":"2024-09-21T03:39:05.429700+0000","last_unstale":"2024-09-21T03:39:06.419248+0000","last_undegraded":"2024-09-21T03:39:06.419248+0000","last_fullsized":"2024-09-21T03:39:06.419248+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:22:26.967805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.419361+0000","last_change":"2024-09-21T03:39:05.434451+0000","last_active":"2024-09-21T03:39:06.419361+0000","last_peered":"2024-09-21T03:39:06.419361+0000","last_clean":"2024-09-21T03:39:06.419361+0000","last_became_active":"2024-09-21T03:39:05.432952+0000","last_became_peered":"2024-09-21T03:39:05.432952+0000","last_unstale":"2024-09-21T03:39:06.419361+0000","last_undegraded":"2024-09-21T03:39:06.419361+0000","last_fullsized":"2024-09-21T03:39:06.419361+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:30:48.915097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.569594+0000","last_change":"2024-09-21T03:39:05.426032+0000","last_active":"2024-09-21T03:39:06.569594+0000","last_peered":"2024-09-21T03:39:06.569594+0000","last_clean":"2024-09-21T03:39:06.569594+0000","last_became_active":"2024-09-21T03:39:05.425731+0000","last_became_peered":"2024-09-21T03:39:05.425731+0000","last_unstale":"2024-09-21T03:39:06.569594+0000","last_undegraded":"2024-09-21T03:39:06.569594+0000","last_fullsized":"2024-09-21T03:39:06.569594+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:01:01.380435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.570162+0000","last_change":"2024-09-21T03:39:05.428805+0000","last_active":"2024-09-21T03:39:06.570162+0000","last_peered":"2024-09-21T03:39:06.570162+0000","last_clean":"2024-09-21T03:39:06.570162+0000","last_became_active":"2024-09-21T03:39:05.428394+0000","last_became_peered":"2024-09-21T03:39:05.428394+0000","last_unstale":"2024-09-21T03:39:06.570162+0000","last_undegraded":"2024-09-21T03:39:06.570162+0000","last_fullsized":"2024-09-21T03:39:06.570162+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:57:28.504333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.415502+0000","last_change":"2024-09-21T03:39:05.418164+0000","last_active":"2024-09-21T03:39:06.415502+0000","last_peered":"2024-09-21T03:39:06.415502+0000","last_clean":"2024-09-21T03:39:06.415502+0000","last_became_active":"2024-09-21T03:39:05.416861+0000","last_became_peered":"2024-09-21T03:39:05.416861+0000","last_unstale":"2024-09-21T03:39:06.415502+0000","last_undegraded":"2024-09-21T03:39:06.415502+0000","last_fullsized":"2024-09-21T03:39:06.415502+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:06:43.522600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.569829+0000","last_change":"2024-09-21T03:39:05.429537+0000","last_active":"2024-09-21T03:39:06.569829+0000","last_peered":"2024-09-21T03:39:06.569829+0000","last_clean":"2024-09-21T03:39:06.569829+0000","last_became_active":"2024-09-21T03:39:05.428077+0000","last_became_peered":"2024-09-21T03:39:05.428077+0000","last_unstale":"2024-09-21T03:39:06.569829+0000","last_undegraded":"2024-09-21T03:39:06.569829+0000","last_fullsized":"2024-09-21T03:39:06.569829+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:38:28.792089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.414574+0000","last_change":"2024-09-21T03:39:05.420868+0000","last_active":"2024-09-21T03:39:06.414574+0000","last_peered":"2024-09-21T03:39:06.414574+0000","last_clean":"2024-09-21T03:39:06.414574+0000","last_became_active":"2024-09-21T03:39:05.420546+0000","last_became_peered":"2024-09-21T03:39:05.420546+0000","last_unstale":"2024-09-21T03:39:06.414574+0000","last_undegraded":"2024-09-21T03:39:06.414574+0000","last_fullsized":"2024-09-21T03:39:06.414574+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:00:25.925784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.570276+0000","last_change":"2024-09-21T03:39:05.424217+0000","last_active":"2024-09-21T03:39:06.570276+0000","last_peered":"2024-09-21T03:39:06.570276+0000","last_clean":"2024-09-21T03:39:06.570276+0000","last_became_active":"2024-09-21T03:39:05.423922+0000","last_became_peered":"2024-09-21T03:39:05.423922+0000","last_unstale":"2024-09-21T03:39:06.570276+0000","last_undegraded":"2024-09-21T03:39:06.570276+0000","last_fullsized":"2024-09-21T03:39:06.570276+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:30:48.915097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-09-21T03:39:04.403441+0000","last_change":"2024-09-21T03:39:04.403441+0000","last_active":"2024-09-21T03:39:04.383211+0000","last_peered":"2024-09-21T03:39:04.383211+0000","last_clean":"2024-09-21T03:39:04.383211+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:39:04.403441+0000","last_undegraded":"2024-09-21T03:39:04.403441+0000","last_fullsized":"2024-09-21T03:39:04.403441+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.419460+0000","last_change":"2024-09-21T03:39:05.427718+0000","last_active":"2024-09-21T03:39:06.419460+0000","last_peered":"2024-09-21T03:39:06.419460+0000","last_clean":"2024-09-21T03:39:06.419460+0000","last_became_active":"2024-09-21T03:39:05.427387+0000","last_became_peered":"2024-09-21T03:39:05.427387+0000","last_unstale":"2024-09-21T03:39:06.419460+0000","last_undegraded":"2024-09-21T03:39:06.419460+0000","last_fullsized":"2024-09-21T03:39:06.419460+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:58:09.229781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.570387+0000","last_change":"2024-09-21T03:39:05.421453+0000","last_active":"2024-09-21T03:39:06.570387+0000","last_peered":"2024-09-21T03:39:06.570387+0000","last_clean":"2024-09-21T03:39:06.570387+0000","last_became_active":"2024-09-21T03:39:05.421133+0000","last_became_peered":"2024-09-21T03:39:05.421133+0000","last_unstale":"2024-09-21T03:39:06.570387+0000","last_undegraded":"2024-09-21T03:39:06.570387+0000","last_fullsized":"2024-09-21T03:39:06.570387+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:30:07.399497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.418721+0000","last_change":"2024-09-21T03:39:05.426022+0000","last_active":"2024-09-21T03:39:06.418721+0000","last_peered":"2024-09-21T03:39:06.418721+0000","last_clean":"2024-09-21T03:39:06.418721+0000","last_became_active":"2024-09-21T03:39:05.425625+0000","last_became_peered":"2024-09-21T03:39:05.425625+0000","last_unstale":"2024-09-21T03:39:06.418721+0000","last_undegraded":"2024-09-21T03:39:06.418721+0000","last_fullsized":"2024-09-21T03:39:06.418721+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:03:54.230793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.414472+0000","last_change":"2024-09-21T03:39:05.421310+0000","last_active":"2024-09-21T03:39:06.414472+0000","last_peered":"2024-09-21T03:39:06.414472+0000","last_clean":"2024-09-21T03:39:06.414472+0000","last_became_active":"2024-09-21T03:39:05.420314+0000","last_became_peered":"2024-09-21T03:39:05.420314+0000","last_unstale":"2024-09-21T03:39:06.414472+0000","last_undegraded":"2024-09-21T03:39:06.414472+0000","last_fullsized":"2024-09-21T03:39:06.414472+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:00:07.867021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.570061+0000","last_change":"2024-09-21T03:39:05.427360+0000","last_active":"2024-09-21T03:39:06.570061+0000","last_peered":"2024-09-21T03:39:06.570061+0000","last_clean":"2024-09-21T03:39:06.570061+0000","last_became_active":"2024-09-21T03:39:05.427009+0000","last_became_peered":"2024-09-21T03:39:05.427009+0000","last_unstale":"2024-09-21T03:39:06.570061+0000","last_undegraded":"2024-09-21T03:39:06.570061+0000","last_fullsized":"2024-09-21T03:39:06.570061+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:54:27.965337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.569046+0000","last_change":"2024-09-21T03:39:05.430747+0000","last_active":"2024-09-21T03:39:06.569046+0000","last_peered":"2024-09-21T03:39:06.569046+0000","last_clean":"2024-09-21T03:39:06.569046+0000","last_became_active":"2024-09-21T03:39:05.429872+0000","last_became_peered":"2024-09-21T03:39:05.429872+0000","last_unstale":"2024-09-21T03:39:06.569046+0000","last_undegraded":"2024-09-21T03:39:06.569046+0000","last_fullsized":"2024-09-21T03:39:06.569046+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:01:18.107849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.415027+0000","last_change":"2024-09-21T03:39:05.418812+0000","last_active":"2024-09-21T03:39:06.415027+0000","last_peered":"2024-09-21T03:39:06.415027+0000","last_clean":"2024-09-21T03:39:06.415027+0000","last_became_active":"2024-09-21T03:39:05.418487+0000","last_became_peered":"2024-09-21T03:39:05.418487+0000","last_unstale":"2024-09-21T03:39:06.415027+0000","last_undegraded":"2024-09-21T03:39:06.415027+0000","last_fullsized":"2024-09-21T03:39:06.415027+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:37:29.087437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.419123+0000","last_change":"2024-09-21T03:39:05.428238+0000","last_active":"2024-09-21T03:39:06.419123+0000","last_peered":"2024-09-21T03:39:06.419123+0000","last_clean":"2024-09-21T03:39:06.419123+0000","last_became_active":"2024-09-21T03:39:05.427915+0000","last_became_peered":"2024-09-21T03:39:05.427915+0000","last_unstale":"2024-09-21T03:39:06.419123+0000","last_undegraded":"2024-09-21T03:39:06.419123+0000","last_fullsized":"2024-09-21T03:39:06.419123+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:22:48.194041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.414304+0000","last_change":"2024-09-21T03:39:05.412642+0000","last_active":"2024-09-21T03:39:06.414304+0000","last_peered":"2024-09-21T03:39:06.414304+0000","last_clean":"2024-09-21T03:39:06.414304+0000","last_became_active":"2024-09-21T03:39:05.412274+0000","last_became_peered":"2024-09-21T03:39:05.412274+0000","last_unstale":"2024-09-21T03:39:06.414304+0000","last_undegraded":"2024-09-21T03:39:06.414304+0000","last_fullsized":"2024-09-21T03:39:06.414304+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:24:30.153279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.415023+0000","last_change":"2024-09-21T03:39:05.428953+0000","last_active":"2024-09-21T03:39:06.415023+0000","last_peered":"2024-09-21T03:39:06.415023+0000","last_clean":"2024-09-21T03:39:06.415023+0000","last_became_active":"2024-09-21T03:39:05.428223+0000","last_became_peered":"2024-09-21T03:39:05.428223+0000","last_unstale":"2024-09-21T03:39:06.415023+0000","last_undegraded":"2024-09-21T03:39:06.415023+0000","last_fullsized":"2024-09-21T03:39:06.415023+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:16:38.764525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.570578+0000","last_change":"2024-09-21T03:39:05.409645+0000","last_active":"2024-09-21T03:39:06.570578+0000","last_peered":"2024-09-21T03:39:06.570578+0000","last_clean":"2024-09-21T03:39:06.570578+0000","last_became_active":"2024-09-21T03:39:05.409315+0000","last_became_peered":"2024-09-21T03:39:05.409315+0000","last_unstale":"2024-09-21T03:39:06.570578+0000","last_undegraded":"2024-09-21T03:39:06.570578+0000","last_fullsized":"2024-09-21T03:39:06.570578+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:49:12.163503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.418882+0000","last_change":"2024-09-21T03:39:05.427679+0000","last_active":"2024-09-21T03:39:06.418882+0000","last_peered":"2024-09-21T03:39:06.418882+0000","last_clean":"2024-09-21T03:39:06.418882+0000","last_became_active":"2024-09-21T03:39:05.426044+0000","last_became_peered":"2024-09-21T03:39:05.426044+0000","last_unstale":"2024-09-21T03:39:06.418882+0000","last_undegraded":"2024-09-21T03:39:06.418882+0000","last_fullsized":"2024-09-21T03:39:06.418882+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:20:35.883679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-09-21T03:39:04.402899+0000","last_change":"2024-09-21T03:39:04.402899+0000","last_active":"2024-09-21T03:39:04.383211+0000","last_peered":"2024-09-21T03:39:04.383211+0000","last_clean":"2024-09-21T03:39:04.383211+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:39:04.402899+0000","last_undegraded":"2024-09-21T03:39:04.402899+0000","last_fullsized":"2024-09-21T03:39:04.402899+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.418484+0000","last_change":"2024-09-21T03:39:05.427956+0000","last_active":"2024-09-21T03:39:06.418484+0000","last_peered":"2024-09-21T03:39:06.418484+0000","last_clean":"2024-09-21T03:39:06.418484+0000","last_became_active":"2024-09-21T03:39:05.425708+0000","last_became_peered":"2024-09-21T03:39:05.425708+0000","last_unstale":"2024-09-21T03:39:06.418484+0000","last_undegraded":"2024-09-21T03:39:06.418484+0000","last_fullsized":"2024-09-21T03:39:06.418484+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:14:22.373008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.415419+0000","last_change":"2024-09-21T03:39:05.420911+0000","last_active":"2024-09-21T03:39:06.415419+0000","last_peered":"2024-09-21T03:39:06.415419+0000","last_clean":"2024-09-21T03:39:06.415419+0000","last_became_active":"2024-09-21T03:39:05.420587+0000","last_became_peered":"2024-09-21T03:39:05.420587+0000","last_unstale":"2024-09-21T03:39:06.415419+0000","last_undegraded":"2024-09-21T03:39:06.415419+0000","last_fullsized":"2024-09-21T03:39:06.415419+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:37:32.783221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.569302+0000","last_change":"2024-09-21T03:39:05.426555+0000","last_active":"2024-09-21T03:39:06.569302+0000","last_peered":"2024-09-21T03:39:06.569302+0000","last_clean":"2024-09-21T03:39:06.569302+0000","last_became_active":"2024-09-21T03:39:05.426265+0000","last_became_peered":"2024-09-21T03:39:05.426265+0000","last_unstale":"2024-09-21T03:39:06.569302+0000","last_undegraded":"2024-09-21T03:39:06.569302+0000","last_fullsized":"2024-09-21T03:39:06.569302+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:03:54.230793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.569945+0000","last_change":"2024-09-21T03:39:05.427771+0000","last_active":"2024-09-21T03:39:06.569945+0000","last_peered":"2024-09-21T03:39:06.569945+0000","last_clean":"2024-09-21T03:39:06.569945+0000","last_became_active":"2024-09-21T03:39:05.427310+0000","last_became_peered":"2024-09-21T03:39:05.427310+0000","last_unstale":"2024-09-21T03:39:06.569945+0000","last_undegraded":"2024-09-21T03:39:06.569945+0000","last_fullsized":"2024-09-21T03:39:06.569945+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:22:48.194041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.570546+0000","last_change":"2024-09-21T03:39:05.431170+0000","last_active":"2024-09-21T03:39:06.570546+0000","last_peered":"2024-09-21T03:39:06.570546+0000","last_clean":"2024-09-21T03:39:06.570546+0000","last_became_active":"2024-09-21T03:39:05.430855+0000","last_became_peered":"2024-09-21T03:39:05.430855+0000","last_unstale":"2024-09-21T03:39:06.570546+0000","last_undegraded":"2024-09-21T03:39:06.570546+0000","last_fullsized":"2024-09-21T03:39:06.570546+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:13:27.854189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.414268+0000","last_change":"2024-09-21T03:39:05.429739+0000","last_active":"2024-09-21T03:39:06.414268+0000","last_peered":"2024-09-21T03:39:06.414268+0000","last_clean":"2024-09-21T03:39:06.414268+0000","last_became_active":"2024-09-21T03:39:05.429401+0000","last_became_peered":"2024-09-21T03:39:05.429401+0000","last_unstale":"2024-09-21T03:39:06.414268+0000","last_undegraded":"2024-09-21T03:39:06.414268+0000","last_fullsized":"2024-09-21T03:39:06.414268+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:29:58.646678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-09-21T03:39:04.402307+0000","last_change":"2024-09-21T03:39:04.402307+0000","last_active":"2024-09-21T03:39:04.383211+0000","last_peered":"2024-09-21T03:39:04.383211+0000","last_clean":"2024-09-21T03:39:04.383211+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:39:04.402307+0000","last_undegraded":"2024-09-21T03:39:04.402307+0000","last_fullsized":"2024-09-21T03:39:04.402307+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.418906+0000","last_change":"2024-09-21T03:39:05.430619+0000","last_active":"2024-09-21T03:39:06.418906+0000","last_peered":"2024-09-21T03:39:06.418906+0000","last_clean":"2024-09-21T03:39:06.418906+0000","last_became_active":"2024-09-21T03:39:05.429959+0000","last_became_peered":"2024-09-21T03:39:05.429959+0000","last_unstale":"2024-09-21T03:39:06.418906+0000","last_undegraded":"2024-09-21T03:39:06.418906+0000","last_fullsized":"2024-09-21T03:39:06.418906+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:26:04.045027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.569279+0000","last_change":"2024-09-21T03:39:05.422391+0000","last_active":"2024-09-21T03:39:06.569279+0000","last_peered":"2024-09-21T03:39:06.569279+0000","last_clean":"2024-09-21T03:39:06.569279+0000","last_became_active":"2024-09-21T03:39:05.422073+0000","last_became_peered":"2024-09-21T03:39:05.422073+0000","last_unstale":"2024-09-21T03:39:06.569279+0000","last_undegraded":"2024-09-21T03:39:06.569279+0000","last_fullsized":"2024-09-21T03:39:06.569279+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:14:22.373008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.570338+0000","last_change":"2024-09-21T03:39:05.430675+0000","last_active":"2024-09-21T03:39:06.570338+0000","last_peered":"2024-09-21T03:39:06.570338+0000","last_clean":"2024-09-21T03:39:06.570338+0000","last_became_active":"2024-09-21T03:39:05.430203+0000","last_became_peered":"2024-09-21T03:39:05.430203+0000","last_unstale":"2024-09-21T03:39:06.570338+0000","last_undegraded":"2024-09-21T03:39:06.570338+0000","last_fullsized":"2024-09-21T03:39:06.570338+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:27:11.780682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"81'4","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.570214+0000","last_change":"2024-09-21T03:39:04.406626+0000","last_active":"2024-09-21T03:39:06.570214+0000","last_peered":"2024-09-21T03:39:06.570214+0000","last_clean":"2024-09-21T03:39:06.570214+0000","last_became_active":"2024-09-21T03:39:04.406318+0000","last_became_peered":"2024-09-21T03:39:04.406318+0000","last_unstale":"2024-09-21T03:39:06.570214+0000","last_undegraded":"2024-09-21T03:39:06.570214+0000","last_fullsized":"2024-09-21T03:39:06.570214+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_clean_scrub_stamp":"2024-09-21T03:39:03.377315+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:00:13.859613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.569891+0000","last_change":"2024-09-21T03:39:05.428305+0000","last_active":"2024-09-21T03:39:06.569891+0000","last_peered":"2024-09-21T03:39:06.569891+0000","last_clean":"2024-09-21T03:39:06.569891+0000","last_became_active":"2024-09-21T03:39:05.427992+0000","last_became_peered":"2024-09-21T03:39:05.427992+0000","last_unstale":"2024-09-21T03:39:06.569891+0000","last_undegraded":"2024-09-21T03:39:06.569891+0000","last_fullsized":"2024-09-21T03:39:06.569891+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:15:23.391245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1e","version":"81'1","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.570008+0000","last_change":"2024-09-21T03:39:04.411255+0000","last_active":"2024-09-21T03:39:06.570008+0000","last_peered":"2024-09-21T03:39:06.570008+0000","last_clean":"2024-09-21T03:39:06.570008+0000","last_became_active":"2024-09-21T03:39:04.410962+0000","last_became_peered":"2024-09-21T03:39:04.410962+0000","last_unstale":"2024-09-21T03:39:06.570008+0000","last_undegraded":"2024-09-21T03:39:06.570008+0000","last_fullsized":"2024-09-21T03:39:06.570008+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_clean_scrub_stamp":"2024-09-21T03:39:03.377315+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:27:36.848833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.569439+0000","last_change":"2024-09-21T03:39:05.415000+0000","last_active":"2024-09-21T03:39:06.569439+0000","last_peered":"2024-09-21T03:39:06.569439+0000","last_clean":"2024-09-21T03:39:06.569439+0000","last_became_active":"2024-09-21T03:39:05.414695+0000","last_became_peered":"2024-09-21T03:39:05.414695+0000","last_unstale":"2024-09-21T03:39:06.569439+0000","last_undegraded":"2024-09-21T03:39:06.569439+0000","last_fullsized":"2024-09-21T03:39:06.569439+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:58:09.229781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T03:39:04.413513+0000","last_change":"2024-09-21T03:39:04.404636+0000","last_active":"2024-09-21T03:39:04.413513+0000","last_peered":"2024-09-21T03:39:04.413513+0000","last_clean":"2024-09-21T03:39:04.413513+0000","last_became_active":"2024-09-21T03:39:04.404270+0000","last_became_peered":"2024-09-21T03:39:04.404270+0000","last_unstale":"2024-09-21T03:39:04.413513+0000","last_undegraded":"2024-09-21T03:39:04.413513+0000","last_fullsized":"2024-09-21T03:39:04.413513+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_clean_scrub_stamp":"2024-09-21T03:39:03.377315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:09:17.529140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.569256+0000","last_change":"2024-09-21T03:39:05.425119+0000","last_active":"2024-09-21T03:39:06.569256+0000","last_peered":"2024-09-21T03:39:06.569256+0000","last_clean":"2024-09-21T03:39:06.569256+0000","last_became_active":"2024-09-21T03:39:05.424804+0000","last_became_peered":"2024-09-21T03:39:05.424804+0000","last_unstale":"2024-09-21T03:39:06.569256+0000","last_undegraded":"2024-09-21T03:39:06.569256+0000","last_fullsized":"2024-09-21T03:39:06.569256+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:13:42.059728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"81'2","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.570641+0000","last_change":"2024-09-21T03:39:04.407381+0000","last_active":"2024-09-21T03:39:06.570641+0000","last_peered":"2024-09-21T03:39:06.570641+0000","last_clean":"2024-09-21T03:39:06.570641+0000","last_became_active":"2024-09-21T03:39:04.407044+0000","last_became_peered":"2024-09-21T03:39:04.407044+0000","last_unstale":"2024-09-21T03:39:06.570641+0000","last_undegraded":"2024-09-21T03:39:06.570641+0000","last_fullsized":"2024-09-21T03:39:06.570641+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_clean_scrub_stamp":"2024-09-21T03:39:03.377315+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:14:21.367112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.415145+0000","last_change":"2024-09-21T03:39:05.425026+0000","last_active":"2024-09-21T03:39:06.415145+0000","last_peered":"2024-09-21T03:39:06.415145+0000","last_clean":"2024-09-21T03:39:06.415145+0000","last_became_active":"2024-09-21T03:39:05.424667+0000","last_became_peered":"2024-09-21T03:39:05.424667+0000","last_unstale":"2024-09-21T03:39:06.415145+0000","last_undegraded":"2024-09-21T03:39:06.415145+0000","last_fullsized":"2024-09-21T03:39:06.415145+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:46:01.041621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.1b","version":"81'1","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.570542+0000","last_change":"2024-09-21T03:39:04.414608+0000","last_active":"2024-09-21T03:39:06.570542+0000","last_peered":"2024-09-21T03:39:06.570542+0000","last_clean":"2024-09-21T03:39:06.570542+0000","last_became_active":"2024-09-21T03:39:04.414274+0000","last_became_peered":"2024-09-21T03:39:04.414274+0000","last_unstale":"2024-09-21T03:39:06.570542+0000","last_undegraded":"2024-09-21T03:39:06.570542+0000","last_fullsized":"2024-09-21T03:39:06.570542+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_clean_scrub_stamp":"2024-09-21T03:39:03.377315+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:13:41.053832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.419607+0000","last_change":"2024-09-21T03:39:05.434557+0000","last_active":"2024-09-21T03:39:06.419607+0000","last_peered":"2024-09-21T03:39:06.419607+0000","last_clean":"2024-09-21T03:39:06.419607+0000","last_became_active":"2024-09-21T03:39:05.430126+0000","last_became_peered":"2024-09-21T03:39:05.430126+0000","last_unstale":"2024-09-21T03:39:06.419607+0000","last_undegraded":"2024-09-21T03:39:06.419607+0000","last_fullsized":"2024-09-21T03:39:06.419607+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:27:34.908887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"81'1","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.570010+0000","last_change":"2024-09-21T03:39:04.397543+0000","last_active":"2024-09-21T03:39:06.570010+0000","last_peered":"2024-09-21T03:39:06.570010+0000","last_clean":"2024-09-21T03:39:06.570010+0000","last_became_active":"2024-09-21T03:39:04.397220+0000","last_became_peered":"2024-09-21T03:39:04.397220+0000","last_unstale":"2024-09-21T03:39:06.570010+0000","last_undegraded":"2024-09-21T03:39:06.570010+0000","last_fullsized":"2024-09-21T03:39:06.570010+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_clean_scrub_stamp":"2024-09-21T03:39:03.377315+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:21:47.012786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.570317+0000","last_change":"2024-09-21T03:39:05.413837+0000","last_active":"2024-09-21T03:39:06.570317+0000","last_peered":"2024-09-21T03:39:06.570317+0000","last_clean":"2024-09-21T03:39:06.570317+0000","last_became_active":"2024-09-21T03:39:05.413498+0000","last_became_peered":"2024-09-21T03:39:05.413498+0000","last_unstale":"2024-09-21T03:39:06.570317+0000","last_undegraded":"2024-09-21T03:39:06.570317+0000","last_fullsized":"2024-09-21T03:39:06.570317+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:33:39.426980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.413804+0000","last_change":"2024-09-21T03:39:04.408211+0000","last_active":"2024-09-21T03:39:06.413804+0000","last_peered":"2024-09-21T03:39:06.413804+0000","last_clean":"2024-09-21T03:39:06.413804+0000","last_became_active":"2024-09-21T03:39:04.404927+0000","last_became_peered":"2024-09-21T03:39:04.404927+0000","last_unstale":"2024-09-21T03:39:06.413804+0000","last_undegraded":"2024-09-21T03:39:06.413804+0000","last_fullsized":"2024-09-21T03:39:06.413804+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_clean_scrub_stamp":"2024-09-21T03:39:03.377315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:53:29.829387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.569462+0000","last_change":"2024-09-21T03:39:05.419988+0000","last_active":"2024-09-21T03:39:06.569462+0000","last_peered":"2024-09-21T03:39:06.569462+0000","last_clean":"2024-09-21T03:39:06.569462+0000","last_became_active":"2024-09-21T03:39:05.419693+0000","last_became_peered":"2024-09-21T03:39:05.419693+0000","last_unstale":"2024-09-21T03:39:06.569462+0000","last_undegraded":"2024-09-21T03:39:06.569462+0000","last_fullsized":"2024-09-21T03:39:06.569462+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:08:48.101749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.418856+0000","last_change":"2024-09-21T03:39:04.410302+0000","last_active":"2024-09-21T03:39:06.418856+0000","last_peered":"2024-09-21T03:39:06.418856+0000","last_clean":"2024-09-21T03:39:06.418856+0000","last_became_active":"2024-09-21T03:39:04.409959+0000","last_became_peered":"2024-09-21T03:39:04.409959+0000","last_unstale":"2024-09-21T03:39:06.418856+0000","last_undegraded":"2024-09-21T03:39:06.418856+0000","last_fullsized":"2024-09-21T03:39:06.418856+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_clean_scrub_stamp":"2024-09-21T03:39:03.377315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:18:35.050119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.419019+0000","last_change":"2024-09-21T03:39:05.429795+0000","last_active":"2024-09-21T03:39:06.419019+0000","last_peered":"2024-09-21T03:39:06.419019+0000","last_clean":"2024-09-21T03:39:06.419019+0000","last_became_active":"2024-09-21T03:39:05.426181+0000","last_became_peered":"2024-09-21T03:39:05.426181+0000","last_unstale":"2024-09-21T03:39:06.419019+0000","last_undegraded":"2024-09-21T03:39:06.419019+0000","last_fullsized":"2024-09-21T03:39:06.419019+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:38:28.792089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.570075+0000","last_change":"2024-09-21T03:39:04.410453+0000","last_active":"2024-09-21T03:39:06.570075+0000","last_peered":"2024-09-21T03:39:06.570075+0000","last_clean":"2024-09-21T03:39:06.570075+0000","last_became_active":"2024-09-21T03:39:04.410082+0000","last_became_peered":"2024-09-21T03:39:04.410082+0000","last_unstale":"2024-09-21T03:39:06.570075+0000","last_undegraded":"2024-09-21T03:39:06.570075+0000","last_fullsized":"2024-09-21T03:39:06.570075+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_clean_scrub_stamp":"2024-09-21T03:39:03.377315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:56:14.520557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.569202+0000","last_change":"2024-09-21T03:39:05.422217+0000","last_active":"2024-09-21T03:39:06.569202+0000","last_peered":"2024-09-21T03:39:06.569202+0000","last_clean":"2024-09-21T03:39:06.569202+0000","last_became_active":"2024-09-21T03:39:05.421890+0000","last_became_peered":"2024-09-21T03:39:05.421890+0000","last_unstale":"2024-09-21T03:39:06.569202+0000","last_undegraded":"2024-09-21T03:39:06.569202+0000","last_fullsized":"2024-09-21T03:39:06.569202+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:40:45.946118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T03:39:04.414955+0000","last_change":"2024-09-21T03:39:04.411385+0000","last_active":"2024-09-21T03:39:04.414955+0000","last_peered":"2024-09-21T03:39:04.414955+0000","last_clean":"2024-09-21T03:39:04.414955+0000","last_became_active":"2024-09-21T03:39:04.410963+0000","last_became_peered":"2024-09-21T03:39:04.410963+0000","last_unstale":"2024-09-21T03:39:04.414955+0000","last_undegraded":"2024-09-21T03:39:04.414955+0000","last_fullsized":"2024-09-21T03:39:04.414955+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_clean_scrub_stamp":"2024-09-21T03:39:03.377315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:53:58.324899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.414821+0000","last_change":"2024-09-21T03:39:05.419517+0000","last_active":"2024-09-21T03:39:06.414821+0000","last_peered":"2024-09-21T03:39:06.414821+0000","last_clean":"2024-09-21T03:39:06.414821+0000","last_became_active":"2024-09-21T03:39:05.418599+0000","last_became_peered":"2024-09-21T03:39:05.418599+0000","last_unstale":"2024-09-21T03:39:06.414821+0000","last_undegraded":"2024-09-21T03:39:06.414821+0000","last_fullsized":"2024-09-21T03:39:06.414821+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:59:45.194584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.15","version":"81'3","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.413935+0000","last_change":"2024-09-21T03:39:04.407595+0000","last_active":"2024-09-21T03:39:06.413935+0000","last_peered":"2024-09-21T03:39:06.413935+0000","last_clean":"2024-09-21T03:39:06.413935+0000","last_became_active":"2024-09-21T03:39:04.407278+0000","last_became_peered":"2024-09-21T03:39:04.407278+0000","last_unstale":"2024-09-21T03:39:06.413935+0000","last_undegraded":"2024-09-21T03:39:06.413935+0000","last_fullsized":"2024-09-21T03:39:06.413935+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_clean_scrub_stamp":"2024-09-21T03:39:03.377315+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:41:01.617219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.414516+0000","last_change":"2024-09-21T03:39:05.419678+0000","last_active":"2024-09-21T03:39:06.414516+0000","last_peered":"2024-09-21T03:39:06.414516+0000","last_clean":"2024-09-21T03:39:06.414516+0000","last_became_active":"2024-09-21T03:39:05.419119+0000","last_became_peered":"2024-09-21T03:39:05.419119+0000","last_unstale":"2024-09-21T03:39:06.414516+0000","last_undegraded":"2024-09-21T03:39:06.414516+0000","last_fullsized":"2024-09-21T03:39:06.414516+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:03:30.086684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.14","version":"81'6","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.569939+0000","last_change":"2024-09-21T03:39:04.396680+0000","last_active":"2024-09-21T03:39:06.569939+0000","last_peered":"2024-09-21T03:39:06.569939+0000","last_clean":"2024-09-21T03:39:06.569939+0000","last_became_active":"2024-09-21T03:39:04.396344+0000","last_became_peered":"2024-09-21T03:39:04.396344+0000","last_unstale":"2024-09-21T03:39:06.569939+0000","last_undegraded":"2024-09-21T03:39:06.569939+0000","last_fullsized":"2024-09-21T03:39:06.569939+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_clean_scrub_stamp":"2024-09-21T03:39:03.377315+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:12:41.701014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.569672+0000","last_change":"2024-09-21T03:39:05.423246+0000","last_active":"2024-09-21T03:39:06.569672+0000","last_peered":"2024-09-21T03:39:06.569672+0000","last_clean":"2024-09-21T03:39:06.569672+0000","last_became_active":"2024-09-21T03:39:05.422924+0000","last_became_peered":"2024-09-21T03:39:05.422924+0000","last_unstale":"2024-09-21T03:39:06.569672+0000","last_undegraded":"2024-09-21T03:39:06.569672+0000","last_fullsized":"2024-09-21T03:39:06.569672+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:51:42.048958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T03:39:04.407961+0000","last_change":"2024-09-21T03:39:04.402594+0000","last_active":"2024-09-21T03:39:04.407961+0000","last_peered":"2024-09-21T03:39:04.407961+0000","last_clean":"2024-09-21T03:39:04.407961+0000","last_became_active":"2024-09-21T03:39:04.402242+0000","last_became_peered":"2024-09-21T03:39:04.402242+0000","last_unstale":"2024-09-21T03:39:04.407961+0000","last_undegraded":"2024-09-21T03:39:04.407961+0000","last_fullsized":"2024-09-21T03:39:04.407961+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_clean_scrub_stamp":"2024-09-21T03:39:03.377315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:23:27.346296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-09-21T03:39:04.401407+0000","last_change":"2024-09-21T03:39:04.401407+0000","last_active":"2024-09-21T03:39:04.383211+0000","last_peered":"2024-09-21T03:39:04.383211+0000","last_clean":"2024-09-21T03:39:04.383211+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:39:04.401407+0000","last_undegraded":"2024-09-21T03:39:04.401407+0000","last_fullsized":"2024-09-21T03:39:04.401407+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.569717+0000","last_change":"2024-09-21T03:39:04.409288+0000","last_active":"2024-09-21T03:39:06.569717+0000","last_peered":"2024-09-21T03:39:06.569717+0000","last_clean":"2024-09-21T03:39:06.569717+0000","last_became_active":"2024-09-21T03:39:04.408965+0000","last_became_peered":"2024-09-21T03:39:04.408965+0000","last_unstale":"2024-09-21T03:39:06.569717+0000","last_undegraded":"2024-09-21T03:39:06.569717+0000","last_fullsized":"2024-09-21T03:39:06.569717+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_clean_scrub_stamp":"2024-09-21T03:39:03.377315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:01:17.101953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.570597+0000","last_change":"2024-09-21T03:39:05.423231+0000","last_active":"2024-09-21T03:39:06.570597+0000","last_peered":"2024-09-21T03:39:06.570597+0000","last_clean":"2024-09-21T03:39:06.570597+0000","last_became_active":"2024-09-21T03:39:05.422931+0000","last_became_peered":"2024-09-21T03:39:05.422931+0000","last_unstale":"2024-09-21T03:39:06.570597+0000","last_undegraded":"2024-09-21T03:39:06.570597+0000","last_fullsized":"2024-09-21T03:39:06.570597+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:35:18.019438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.570103+0000","last_change":"2024-09-21T03:39:04.405877+0000","last_active":"2024-09-21T03:39:06.570103+0000","last_peered":"2024-09-21T03:39:06.570103+0000","last_clean":"2024-09-21T03:39:06.570103+0000","last_became_active":"2024-09-21T03:39:04.405583+0000","last_became_peered":"2024-09-21T03:39:04.405583+0000","last_unstale":"2024-09-21T03:39:06.570103+0000","last_undegraded":"2024-09-21T03:39:06.570103+0000","last_fullsized":"2024-09-21T03:39:06.570103+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_clean_scrub_stamp":"2024-09-21T03:39:03.377315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:03:56.352603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-09-21T03:39:04.402886+0000","last_change":"2024-09-21T03:39:04.402886+0000","last_active":"2024-09-21T03:39:04.383211+0000","last_peered":"2024-09-21T03:39:04.383211+0000","last_clean":"2024-09-21T03:39:04.383211+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:39:04.402886+0000","last_undegraded":"2024-09-21T03:39:04.402886+0000","last_fullsized":"2024-09-21T03:39:04.402886+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.10","version":"81'8","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.413938+0000","last_change":"2024-09-21T03:39:04.411109+0000","last_active":"2024-09-21T03:39:06.413938+0000","last_peered":"2024-09-21T03:39:06.413938+0000","last_clean":"2024-09-21T03:39:06.413938+0000","last_became_active":"2024-09-21T03:39:04.410744+0000","last_became_peered":"2024-09-21T03:39:04.410744+0000","last_unstale":"2024-09-21T03:39:06.413938+0000","last_undegraded":"2024-09-21T03:39:06.413938+0000","last_fullsized":"2024-09-21T03:39:06.413938+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_clean_scrub_stamp":"2024-09-21T03:39:03.377315+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:13:48.637151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.570028+0000","last_change":"2024-09-21T03:39:05.432432+0000","last_active":"2024-09-21T03:39:06.570028+0000","last_peered":"2024-09-21T03:39:06.570028+0000","last_clean":"2024-09-21T03:39:06.570028+0000","last_became_active":"2024-09-21T03:39:05.432136+0000","last_became_peered":"2024-09-21T03:39:05.432136+0000","last_unstale":"2024-09-21T03:39:06.570028+0000","last_undegraded":"2024-09-21T03:39:06.570028+0000","last_fullsized":"2024-09-21T03:39:06.570028+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:10:20.019199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.f","version":"81'1","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.570192+0000","last_change":"2024-09-21T03:39:04.403666+0000","last_active":"2024-09-21T03:39:06.570192+0000","last_peered":"2024-09-21T03:39:06.570192+0000","last_clean":"2024-09-21T03:39:06.570192+0000","last_became_active":"2024-09-21T03:39:04.403310+0000","last_became_peered":"2024-09-21T03:39:04.403310+0000","last_unstale":"2024-09-21T03:39:06.570192+0000","last_undegraded":"2024-09-21T03:39:06.570192+0000","last_fullsized":"2024-09-21T03:39:06.570192+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_clean_scrub_stamp":"2024-09-21T03:39:03.377315+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:18:35.050119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.418952+0000","last_change":"2024-09-21T03:39:05.432320+0000","last_active":"2024-09-21T03:39:06.418952+0000","last_peered":"2024-09-21T03:39:06.418952+0000","last_clean":"2024-09-21T03:39:06.418952+0000","last_became_active":"2024-09-21T03:39:05.431965+0000","last_became_peered":"2024-09-21T03:39:05.431965+0000","last_unstale":"2024-09-21T03:39:06.418952+0000","last_undegraded":"2024-09-21T03:39:06.418952+0000","last_fullsized":"2024-09-21T03:39:06.418952+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:51:42.048958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.e","version":"81'2","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.569527+0000","last_change":"2024-09-21T03:39:04.403354+0000","last_active":"2024-09-21T03:39:06.569527+0000","last_peered":"2024-09-21T03:39:06.569527+0000","last_clean":"2024-09-21T03:39:06.569527+0000","last_became_active":"2024-09-21T03:39:04.401729+0000","last_became_peered":"2024-09-21T03:39:04.401729+0000","last_unstale":"2024-09-21T03:39:06.569527+0000","last_undegraded":"2024-09-21T03:39:06.569527+0000","last_fullsized":"2024-09-21T03:39:06.569527+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_clean_scrub_stamp":"2024-09-21T03:39:03.377315+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:05:16.494460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.414672+0000","last_change":"2024-09-21T03:39:05.421453+0000","last_active":"2024-09-21T03:39:06.414672+0000","last_peered":"2024-09-21T03:39:06.414672+0000","last_clean":"2024-09-21T03:39:06.414672+0000","last_became_active":"2024-09-21T03:39:05.420867+0000","last_became_peered":"2024-09-21T03:39:05.420867+0000","last_unstale":"2024-09-21T03:39:06.414672+0000","last_undegraded":"2024-09-21T03:39:06.414672+0000","last_fullsized":"2024-09-21T03:39:06.414672+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:47:58.257466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.d","version":"81'1","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.570103+0000","last_change":"2024-09-21T03:39:04.408221+0000","last_active":"2024-09-21T03:39:06.570103+0000","last_peered":"2024-09-21T03:39:06.570103+0000","last_clean":"2024-09-21T03:39:06.570103+0000","last_became_active":"2024-09-21T03:39:04.407900+0000","last_became_peered":"2024-09-21T03:39:04.407900+0000","last_unstale":"2024-09-21T03:39:06.570103+0000","last_undegraded":"2024-09-21T03:39:06.570103+0000","last_fullsized":"2024-09-21T03:39:06.570103+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_clean_scrub_stamp":"2024-09-21T03:39:03.377315+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:26:11.862449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.570468+0000","last_change":"2024-09-21T03:39:05.423205+0000","last_active":"2024-09-21T03:39:06.570468+0000","last_peered":"2024-09-21T03:39:06.570468+0000","last_clean":"2024-09-21T03:39:06.570468+0000","last_became_active":"2024-09-21T03:39:05.422867+0000","last_became_peered":"2024-09-21T03:39:05.422867+0000","last_unstale":"2024-09-21T03:39:06.570468+0000","last_undegraded":"2024-09-21T03:39:06.570468+0000","last_fullsized":"2024-09-21T03:39:06.570468+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:33:29.949291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.570257+0000","last_change":"2024-09-21T03:39:04.401682+0000","last_active":"2024-09-21T03:39:06.570257+0000","last_peered":"2024-09-21T03:39:06.570257+0000","last_clean":"2024-09-21T03:39:06.570257+0000","last_became_active":"2024-09-21T03:39:04.401354+0000","last_became_peered":"2024-09-21T03:39:04.401354+0000","last_unstale":"2024-09-21T03:39:06.570257+0000","last_undegraded":"2024-09-21T03:39:06.570257+0000","last_fullsized":"2024-09-21T03:39:06.570257+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_clean_scrub_stamp":"2024-09-21T03:39:03.377315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:02:56.975258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.570149+0000","last_change":"2024-09-21T03:39:05.424271+0000","last_active":"2024-09-21T03:39:06.570149+0000","last_peered":"2024-09-21T03:39:06.570149+0000","last_clean":"2024-09-21T03:39:06.570149+0000","last_became_active":"2024-09-21T03:39:05.423967+0000","last_became_peered":"2024-09-21T03:39:05.423967+0000","last_unstale":"2024-09-21T03:39:06.570149+0000","last_undegraded":"2024-09-21T03:39:06.570149+0000","last_fullsized":"2024-09-21T03:39:06.570149+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:29:22.009760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"81'2","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.418688+0000","last_change":"2024-09-21T03:39:04.406956+0000","last_active":"2024-09-21T03:39:06.418688+0000","last_peered":"2024-09-21T03:39:06.418688+0000","last_clean":"2024-09-21T03:39:06.418688+0000","last_became_active":"2024-09-21T03:39:04.406604+0000","last_became_peered":"2024-09-21T03:39:04.406604+0000","last_unstale":"2024-09-21T03:39:06.418688+0000","last_undegraded":"2024-09-21T03:39:06.418688+0000","last_fullsized":"2024-09-21T03:39:06.418688+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_clean_scrub_stamp":"2024-09-21T03:39:03.377315+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:56:14.520557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.570451+0000","last_change":"2024-09-21T03:39:05.424012+0000","last_active":"2024-09-21T03:39:06.570451+0000","last_peered":"2024-09-21T03:39:06.570451+0000","last_clean":"2024-09-21T03:39:06.570451+0000","last_became_active":"2024-09-21T03:39:05.423713+0000","last_became_peered":"2024-09-21T03:39:05.423713+0000","last_unstale":"2024-09-21T03:39:06.570451+0000","last_undegraded":"2024-09-21T03:39:06.570451+0000","last_fullsized":"2024-09-21T03:39:06.570451+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:30:06.424731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.413940+0000","last_change":"2024-09-21T03:39:04.395490+0000","last_active":"2024-09-21T03:39:06.413940+0000","last_peered":"2024-09-21T03:39:06.413940+0000","last_clean":"2024-09-21T03:39:06.413940+0000","last_became_active":"2024-09-21T03:39:04.394934+0000","last_became_peered":"2024-09-21T03:39:04.394934+0000","last_unstale":"2024-09-21T03:39:06.413940+0000","last_undegraded":"2024-09-21T03:39:06.413940+0000","last_fullsized":"2024-09-21T03:39:06.413940+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_clean_scrub_stamp":"2024-09-21T03:39:03.377315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:30:01.896910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.569691+0000","last_change":"2024-09-21T03:39:05.418742+0000","last_active":"2024-09-21T03:39:06.569691+0000","last_peered":"2024-09-21T03:39:06.569691+0000","last_clean":"2024-09-21T03:39:06.569691+0000","last_became_active":"2024-09-21T03:39:05.418366+0000","last_became_peered":"2024-09-21T03:39:05.418366+0000","last_unstale":"2024-09-21T03:39:06.569691+0000","last_undegraded":"2024-09-21T03:39:06.569691+0000","last_fullsized":"2024-09-21T03:39:06.569691+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:09:46.888780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.418986+0000","last_change":"2024-09-21T03:39:04.408050+0000","last_active":"2024-09-21T03:39:06.418986+0000","last_peered":"2024-09-21T03:39:06.418986+0000","last_clean":"2024-09-21T03:39:06.418986+0000","last_became_active":"2024-09-21T03:39:04.407753+0000","last_became_peered":"2024-09-21T03:39:04.407753+0000","last_unstale":"2024-09-21T03:39:06.418986+0000","last_undegraded":"2024-09-21T03:39:06.418986+0000","last_fullsized":"2024-09-21T03:39:06.418986+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_clean_scrub_stamp":"2024-09-21T03:39:03.377315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:05:16.494460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.569516+0000","last_change":"2024-09-21T03:39:05.431029+0000","last_active":"2024-09-21T03:39:06.569516+0000","last_peered":"2024-09-21T03:39:06.569516+0000","last_clean":"2024-09-21T03:39:06.569516+0000","last_became_active":"2024-09-21T03:39:05.427562+0000","last_became_peered":"2024-09-21T03:39:05.427562+0000","last_unstale":"2024-09-21T03:39:06.569516+0000","last_undegraded":"2024-09-21T03:39:06.569516+0000","last_fullsized":"2024-09-21T03:39:06.569516+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:20:35.883679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T03:39:04.414902+0000","last_change":"2024-09-21T03:39:04.407689+0000","last_active":"2024-09-21T03:39:04.414902+0000","last_peered":"2024-09-21T03:39:04.414902+0000","last_clean":"2024-09-21T03:39:04.414902+0000","last_became_active":"2024-09-21T03:39:04.407356+0000","last_became_peered":"2024-09-21T03:39:04.407356+0000","last_unstale":"2024-09-21T03:39:04.414902+0000","last_undegraded":"2024-09-21T03:39:04.414902+0000","last_fullsized":"2024-09-21T03:39:04.414902+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_clean_scrub_stamp":"2024-09-21T03:39:03.377315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:25:15.786223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-09-21T03:39:04.400780+0000","last_change":"2024-09-21T03:39:04.400780+0000","last_active":"2024-09-21T03:39:04.383211+0000","last_peered":"2024-09-21T03:39:04.383211+0000","last_clean":"2024-09-21T03:39:04.383211+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:39:04.400780+0000","last_undegraded":"2024-09-21T03:39:04.400780+0000","last_fullsized":"2024-09-21T03:39:04.400780+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.6","version":"82'1","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.417795+0000","last_change":"2024-09-21T03:39:04.411532+0000","last_active":"2024-09-21T03:39:06.417795+0000","last_peered":"2024-09-21T03:39:06.417795+0000","last_clean":"2024-09-21T03:39:06.417795+0000","last_became_active":"2024-09-21T03:39:04.411189+0000","last_became_peered":"2024-09-21T03:39:04.411189+0000","last_unstale":"2024-09-21T03:39:06.417795+0000","last_undegraded":"2024-09-21T03:39:06.417795+0000","last_fullsized":"2024-09-21T03:39:06.417795+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_clean_scrub_stamp":"2024-09-21T03:39:03.377315+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:31:32.273309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.569653+0000","last_change":"2024-09-21T03:39:05.414196+0000","last_active":"2024-09-21T03:39:06.569653+0000","last_peered":"2024-09-21T03:39:06.569653+0000","last_clean":"2024-09-21T03:39:06.569653+0000","last_became_active":"2024-09-21T03:39:05.413871+0000","last_became_peered":"2024-09-21T03:39:05.413871+0000","last_unstale":"2024-09-21T03:39:06.569653+0000","last_undegraded":"2024-09-21T03:39:06.569653+0000","last_fullsized":"2024-09-21T03:39:06.569653+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:27:34.908887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T03:39:04.413543+0000","last_change":"2024-09-21T03:39:04.405352+0000","last_active":"2024-09-21T03:39:04.413543+0000","last_peered":"2024-09-21T03:39:04.413543+0000","last_clean":"2024-09-21T03:39:04.413543+0000","last_became_active":"2024-09-21T03:39:04.405020+0000","last_became_peered":"2024-09-21T03:39:04.405020+0000","last_unstale":"2024-09-21T03:39:04.413543+0000","last_undegraded":"2024-09-21T03:39:04.413543+0000","last_fullsized":"2024-09-21T03:39:04.413543+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_clean_scrub_stamp":"2024-09-21T03:39:03.377315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:37:33.420266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.415196+0000","last_change":"2024-09-21T03:39:05.415347+0000","last_active":"2024-09-21T03:39:06.415196+0000","last_peered":"2024-09-21T03:39:06.415196+0000","last_clean":"2024-09-21T03:39:06.415196+0000","last_became_active":"2024-09-21T03:39:05.414476+0000","last_became_peered":"2024-09-21T03:39:05.414476+0000","last_unstale":"2024-09-21T03:39:06.415196+0000","last_undegraded":"2024-09-21T03:39:06.415196+0000","last_fullsized":"2024-09-21T03:39:06.415196+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:49:45.640520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.569666+0000","last_change":"2024-09-21T03:39:04.410318+0000","last_active":"2024-09-21T03:39:06.569666+0000","last_peered":"2024-09-21T03:39:06.569666+0000","last_clean":"2024-09-21T03:39:06.569666+0000","last_became_active":"2024-09-21T03:39:04.409989+0000","last_became_peered":"2024-09-21T03:39:04.409989+0000","last_unstale":"2024-09-21T03:39:06.569666+0000","last_undegraded":"2024-09-21T03:39:06.569666+0000","last_fullsized":"2024-09-21T03:39:06.569666+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_clean_scrub_stamp":"2024-09-21T03:39:03.377315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:49:27.896288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.570622+0000","last_change":"2024-09-21T03:39:05.421291+0000","last_active":"2024-09-21T03:39:06.570622+0000","last_peered":"2024-09-21T03:39:06.570622+0000","last_clean":"2024-09-21T03:39:06.570622+0000","last_became_active":"2024-09-21T03:39:05.420933+0000","last_became_peered":"2024-09-21T03:39:05.420933+0000","last_unstale":"2024-09-21T03:39:06.570622+0000","last_undegraded":"2024-09-21T03:39:06.570622+0000","last_fullsized":"2024-09-21T03:39:06.570622+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:18:54.425316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.569375+0000","last_change":"2024-09-21T03:39:04.405531+0000","last_active":"2024-09-21T03:39:06.569375+0000","last_peered":"2024-09-21T03:39:06.569375+0000","last_clean":"2024-09-21T03:39:06.569375+0000","last_became_active":"2024-09-21T03:39:04.405193+0000","last_became_peered":"2024-09-21T03:39:04.405193+0000","last_unstale":"2024-09-21T03:39:06.569375+0000","last_undegraded":"2024-09-21T03:39:06.569375+0000","last_fullsized":"2024-09-21T03:39:06.569375+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_clean_scrub_stamp":"2024-09-21T03:39:03.377315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:41:35.755494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.569966+0000","last_change":"2024-09-21T03:39:05.431541+0000","last_active":"2024-09-21T03:39:06.569966+0000","last_peered":"2024-09-21T03:39:06.569966+0000","last_clean":"2024-09-21T03:39:06.569966+0000","last_became_active":"2024-09-21T03:39:05.428130+0000","last_became_peered":"2024-09-21T03:39:05.428130+0000","last_unstale":"2024-09-21T03:39:06.569966+0000","last_undegraded":"2024-09-21T03:39:06.569966+0000","last_fullsized":"2024-09-21T03:39:06.569966+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:22:26.967805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.415539+0000","last_change":"2024-09-21T03:39:04.403462+0000","last_active":"2024-09-21T03:39:06.415539+0000","last_peered":"2024-09-21T03:39:06.415539+0000","last_clean":"2024-09-21T03:39:06.415539+0000","last_became_active":"2024-09-21T03:39:04.403114+0000","last_became_peered":"2024-09-21T03:39:04.403114+0000","last_unstale":"2024-09-21T03:39:06.415539+0000","last_undegraded":"2024-09-21T03:39:06.415539+0000","last_fullsized":"2024-09-21T03:39:06.415539+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_clean_scrub_stamp":"2024-09-21T03:39:03.377315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:38:07.041511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.419363+0000","last_change":"2024-09-21T03:39:05.429296+0000","last_active":"2024-09-21T03:39:06.419363+0000","last_peered":"2024-09-21T03:39:06.419363+0000","last_clean":"2024-09-21T03:39:06.419363+0000","last_became_active":"2024-09-21T03:39:05.426008+0000","last_became_peered":"2024-09-21T03:39:05.426008+0000","last_unstale":"2024-09-21T03:39:06.419363+0000","last_undegraded":"2024-09-21T03:39:06.419363+0000","last_fullsized":"2024-09-21T03:39:06.419363+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:16:42.786975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.568997+0000","last_change":"2024-09-21T03:39:04.407078+0000","last_active":"2024-09-21T03:39:06.568997+0000","last_peered":"2024-09-21T03:39:06.568997+0000","last_clean":"2024-09-21T03:39:06.568997+0000","last_became_active":"2024-09-21T03:39:04.406757+0000","last_became_peered":"2024-09-21T03:39:04.406757+0000","last_unstale":"2024-09-21T03:39:06.568997+0000","last_undegraded":"2024-09-21T03:39:06.568997+0000","last_fullsized":"2024-09-21T03:39:06.568997+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_clean_scrub_stamp":"2024-09-21T03:39:03.377315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:08:19.857227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.570315+0000","last_change":"2024-09-21T03:39:05.431328+0000","last_active":"2024-09-21T03:39:06.570315+0000","last_peered":"2024-09-21T03:39:06.570315+0000","last_clean":"2024-09-21T03:39:06.570315+0000","last_became_active":"2024-09-21T03:39:05.431007+0000","last_became_peered":"2024-09-21T03:39:05.431007+0000","last_unstale":"2024-09-21T03:39:06.570315+0000","last_undegraded":"2024-09-21T03:39:06.570315+0000","last_fullsized":"2024-09-21T03:39:06.570315+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:09:46.155413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.419086+0000","last_change":"2024-09-21T03:39:04.412555+0000","last_active":"2024-09-21T03:39:06.419086+0000","last_peered":"2024-09-21T03:39:06.419086+0000","last_clean":"2024-09-21T03:39:06.419086+0000","last_became_active":"2024-09-21T03:39:04.412258+0000","last_became_peered":"2024-09-21T03:39:04.412258+0000","last_unstale":"2024-09-21T03:39:06.419086+0000","last_undegraded":"2024-09-21T03:39:06.419086+0000","last_fullsized":"2024-09-21T03:39:06.419086+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_clean_scrub_stamp":"2024-09-21T03:39:03.377315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:01:13.191529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.570331+0000","last_change":"2024-09-21T03:39:05.418570+0000","last_active":"2024-09-21T03:39:06.570331+0000","last_peered":"2024-09-21T03:39:06.570331+0000","last_clean":"2024-09-21T03:39:06.570331+0000","last_became_active":"2024-09-21T03:39:05.418257+0000","last_became_peered":"2024-09-21T03:39:05.418257+0000","last_unstale":"2024-09-21T03:39:06.570331+0000","last_undegraded":"2024-09-21T03:39:06.570331+0000","last_fullsized":"2024-09-21T03:39:06.570331+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:38:35.279761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.415650+0000","last_change":"2024-09-21T03:39:04.414590+0000","last_active":"2024-09-21T03:39:06.415650+0000","last_peered":"2024-09-21T03:39:06.415650+0000","last_clean":"2024-09-21T03:39:06.415650+0000","last_became_active":"2024-09-21T03:39:04.414355+0000","last_became_peered":"2024-09-21T03:39:04.414355+0000","last_unstale":"2024-09-21T03:39:06.415650+0000","last_undegraded":"2024-09-21T03:39:06.415650+0000","last_fullsized":"2024-09-21T03:39:06.415650+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_clean_scrub_stamp":"2024-09-21T03:39:03.377315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:03:09.175309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":223,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T03:39:04.391409+0000","last_change":"2024-09-21T03:33:25.319776+0000","last_active":"2024-09-21T03:39:04.391409+0000","last_peered":"2024-09-21T03:39:04.391409+0000","last_clean":"2024-09-21T03:39:04.391409+0000","last_became_active":"2024-09-21T03:33:25.318399+0000","last_became_peered":"2024-09-21T03:33:25.318399+0000","last_unstale":"2024-09-21T03:39:04.391409+0000","last_undegraded":"2024-09-21T03:39:04.391409+0000","last_fullsized":"2024-09-21T03:39:04.391409+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:23.016672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:23.016672+0000","last_clean_scrub_stamp":"2024-09-21T03:33:23.016672+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:32:35.906114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":2376,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":36,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":33,"ondisk_log_size":33,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":1,"up_from":16,"seq":68719476808,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32720,"kb_used_data":1528,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339120,"statfs":{"total":96636764160,"available":96603258880,"internally_reserved":0,"allocated":1564672,"data_stored":1036315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509513,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36328,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335512,"statfs":{"total":96636764160,"available":96599564288,"internally_reserved":0,"allocated":1003520,"data_stored":460458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":36174283},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509512,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32184,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339656,"statfs":{"total":96636764160,"available":96603807744,"internally_reserved":0,"allocated":1015808,"data_stored":461522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509513,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32164,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339676,"statfs":{"total":96636764160,"available":96603828224,"internally_reserved":0,"allocated":995328,"data_stored":459806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509512,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32168,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339672,"statfs":{"total":96636764160,"available":96603824128,"internally_reserved":0,"allocated":999424,"data_stored":459828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542216,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36332,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335508,"statfs":{"total":96636764160,"available":96599560192,"internally_reserved":0,"allocated":1003520,"data_stored":460478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574920,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36912,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334928,"statfs":{"total":96636764160,"available":96598966272,"internally_reserved":0,"allocated":1593344,"data_stored":1050264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542217,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36328,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335512,"statfs":{"total":96636764160,"available":96599564288,"internally_reserved":0,"allocated":999424,"data_stored":459828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T03:39:10.263 INFO:tasks.ceph.ceph_manager.ceph:PG 12.3f is not active+clean 2024-09-21T03:39:10.263 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '12.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-09-21T03:39:04.404050+0000', 'last_change': '2024-09-21T03:39:04.404050+0000', 'last_active': '2024-09-21T03:39:04.383211+0000', 'last_peered': '2024-09-21T03:39:04.383211+0000', 'last_clean': '2024-09-21T03:39:04.383211+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T03:39:04.404050+0000', 'last_undegraded': '2024-09-21T03:39:04.404050+0000', 'last_fullsized': '2024-09-21T03:39:04.404050+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T03:39:04.383211+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T03:39:04.383211+0000', 'last_clean_scrub_stamp': '2024-09-21T03:39:04.383211+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-21T03:39:10.263 INFO:tasks.ceph.ceph_manager.ceph:PG 12.36 is not active+clean 2024-09-21T03:39:10.263 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '12.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-09-21T03:39:04.403441+0000', 'last_change': '2024-09-21T03:39:04.403441+0000', 'last_active': '2024-09-21T03:39:04.383211+0000', 'last_peered': '2024-09-21T03:39:04.383211+0000', 'last_clean': '2024-09-21T03:39:04.383211+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T03:39:04.403441+0000', 'last_undegraded': '2024-09-21T03:39:04.403441+0000', 'last_fullsized': '2024-09-21T03:39:04.403441+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T03:39:04.383211+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T03:39:04.383211+0000', 'last_clean_scrub_stamp': '2024-09-21T03:39:04.383211+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-21T03:39:10.264 INFO:tasks.ceph.ceph_manager.ceph:PG 12.29 is not active+clean 2024-09-21T03:39:10.264 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '12.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-09-21T03:39:04.402899+0000', 'last_change': '2024-09-21T03:39:04.402899+0000', 'last_active': '2024-09-21T03:39:04.383211+0000', 'last_peered': '2024-09-21T03:39:04.383211+0000', 'last_clean': '2024-09-21T03:39:04.383211+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T03:39:04.402899+0000', 'last_undegraded': '2024-09-21T03:39:04.402899+0000', 'last_fullsized': '2024-09-21T03:39:04.402899+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T03:39:04.383211+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T03:39:04.383211+0000', 'last_clean_scrub_stamp': '2024-09-21T03:39:04.383211+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-21T03:39:10.264 INFO:tasks.ceph.ceph_manager.ceph:PG 12.22 is not active+clean 2024-09-21T03:39:10.264 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '12.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-09-21T03:39:04.402307+0000', 'last_change': '2024-09-21T03:39:04.402307+0000', 'last_active': '2024-09-21T03:39:04.383211+0000', 'last_peered': '2024-09-21T03:39:04.383211+0000', 'last_clean': '2024-09-21T03:39:04.383211+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T03:39:04.402307+0000', 'last_undegraded': '2024-09-21T03:39:04.402307+0000', 'last_fullsized': '2024-09-21T03:39:04.402307+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T03:39:04.383211+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T03:39:04.383211+0000', 'last_clean_scrub_stamp': '2024-09-21T03:39:04.383211+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-21T03:39:10.264 INFO:tasks.ceph.ceph_manager.ceph:PG 12.15 is not active+clean 2024-09-21T03:39:10.264 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '12.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-09-21T03:39:04.401407+0000', 'last_change': '2024-09-21T03:39:04.401407+0000', 'last_active': '2024-09-21T03:39:04.383211+0000', 'last_peered': '2024-09-21T03:39:04.383211+0000', 'last_clean': '2024-09-21T03:39:04.383211+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T03:39:04.401407+0000', 'last_undegraded': '2024-09-21T03:39:04.401407+0000', 'last_fullsized': '2024-09-21T03:39:04.401407+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T03:39:04.383211+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T03:39:04.383211+0000', 'last_clean_scrub_stamp': '2024-09-21T03:39:04.383211+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-21T03:39:10.264 INFO:tasks.ceph.ceph_manager.ceph:PG 12.17 is not active+clean 2024-09-21T03:39:10.264 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '12.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-09-21T03:39:04.402886+0000', 'last_change': '2024-09-21T03:39:04.402886+0000', 'last_active': '2024-09-21T03:39:04.383211+0000', 'last_peered': '2024-09-21T03:39:04.383211+0000', 'last_clean': '2024-09-21T03:39:04.383211+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T03:39:04.402886+0000', 'last_undegraded': '2024-09-21T03:39:04.402886+0000', 'last_fullsized': '2024-09-21T03:39:04.402886+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T03:39:04.383211+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T03:39:04.383211+0000', 'last_clean_scrub_stamp': '2024-09-21T03:39:04.383211+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-21T03:39:10.264 INFO:tasks.ceph.ceph_manager.ceph:PG 12.1 is not active+clean 2024-09-21T03:39:10.264 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '12.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-09-21T03:39:04.400780+0000', 'last_change': '2024-09-21T03:39:04.400780+0000', 'last_active': '2024-09-21T03:39:04.383211+0000', 'last_peered': '2024-09-21T03:39:04.383211+0000', 'last_clean': '2024-09-21T03:39:04.383211+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T03:39:04.400780+0000', 'last_undegraded': '2024-09-21T03:39:04.400780+0000', 'last_fullsized': '2024-09-21T03:39:04.400780+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T03:39:04.383211+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T03:39:04.383211+0000', 'last_clean_scrub_stamp': '2024-09-21T03:39:04.383211+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-21T03:39:10.265 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-21T03:39:10.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:10.381+0000 7fba41741640 1 -- 172.21.15.38:0/3213162919 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fba3c162280 msgr2=0x7fba3c162690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:10.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:10.381+0000 7fba41741640 1 --2- 172.21.15.38:0/3213162919 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fba3c162280 0x7fba3c162690 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fba3c0d5040 tx=0x7fba2402a560 comp rx=0 tx=0).stop 2024-09-21T03:39:10.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:10.381+0000 7fba41741640 1 -- 172.21.15.38:0/3213162919 shutdown_connections 2024-09-21T03:39:10.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:10.381+0000 7fba41741640 1 --2- 172.21.15.38:0/3213162919 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fba3c162280 0x7fba3c162690 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:10.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:10.381+0000 7fba41741640 1 --2- 172.21.15.38:0/3213162919 >> v2:172.21.15.195:3300/0 conn(0x7fba3c162f00 0x7fba3c0d24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:10.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:10.381+0000 7fba41741640 1 -- 172.21.15.38:0/3213162919 >> 172.21.15.38:0/3213162919 conn(0x7fba3c0b6b20 msgr2=0x7fba3c0d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:10.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:10.381+0000 7fba41741640 1 -- 172.21.15.38:0/3213162919 shutdown_connections 2024-09-21T03:39:10.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:10.381+0000 7fba41741640 1 -- 172.21.15.38:0/3213162919 wait complete. 2024-09-21T03:39:10.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:10.383+0000 7fba41741640 1 Processor -- start 2024-09-21T03:39:10.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:10.383+0000 7fba41741640 1 -- start start 2024-09-21T03:39:10.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:10.384+0000 7fba41741640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fba3c0b2e40 0x7fba3c0b2750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:10.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:10.384+0000 7fba41741640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fba3c162280 0x7fba3c1f2530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:10.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:10.384+0000 7fba41741640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fba3c0aadc0 0x7fba3c0ab1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:10.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:10.384+0000 7fba41741640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fba3c15e630 con 0x7fba3c0b2e40 2024-09-21T03:39:10.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:10.384+0000 7fba41741640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fba3c15e4b0 con 0x7fba3c162280 2024-09-21T03:39:10.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:10.384+0000 7fba41741640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fba3c15e7b0 con 0x7fba3c0aadc0 2024-09-21T03:39:10.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:10.384+0000 7fba3affd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fba3c0b2e40 0x7fba3c0b2750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:10.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:10.384+0000 7fba3affd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fba3c0b2e40 0x7fba3c0b2750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:52730/0 (socket says 172.21.15.38:52730) 2024-09-21T03:39:10.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:10.384+0000 7fba3affd640 1 -- 172.21.15.38:0/3449415181 learned_addr learned my addr 172.21.15.38:0/3449415181 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:39:10.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:10.384+0000 7fba3b7fe640 1 --2- 172.21.15.38:0/3449415181 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fba3c0aadc0 0x7fba3c0ab1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:10.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:10.384+0000 7fba33fff640 1 --2- 172.21.15.38:0/3449415181 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fba3c162280 0x7fba3c1f2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:10.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:10.385+0000 7fba3affd640 1 -- 172.21.15.38:0/3449415181 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fba3c0aadc0 msgr2=0x7fba3c0ab1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:10.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:10.385+0000 7fba3affd640 1 --2- 172.21.15.38:0/3449415181 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fba3c0aadc0 0x7fba3c0ab1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:10.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:10.385+0000 7fba3affd640 1 -- 172.21.15.38:0/3449415181 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fba3c162280 msgr2=0x7fba3c1f2530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:10.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:10.385+0000 7fba3affd640 1 --2- 172.21.15.38:0/3449415181 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fba3c162280 0x7fba3c1f2530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:10.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:10.385+0000 7fba3affd640 1 -- 172.21.15.38:0/3449415181 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba3c1f2a70 con 0x7fba3c0b2e40 2024-09-21T03:39:10.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:10.385+0000 7fba3b7fe640 1 --2- 172.21.15.38:0/3449415181 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fba3c0aadc0 0x7fba3c0ab1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:39:10.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:10.385+0000 7fba3affd640 1 --2- 172.21.15.38:0/3449415181 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fba3c0b2e40 0x7fba3c0b2750 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fba2c00e790 tx=0x7fba2c00ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:10.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:10.385+0000 7fba337fe640 1 -- 172.21.15.38:0/3449415181 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba2c00eea0 con 0x7fba3c0b2e40 2024-09-21T03:39:10.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:10.385+0000 7fba337fe640 1 -- 172.21.15.38:0/3449415181 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fba2c00cd50 con 0x7fba3c0b2e40 2024-09-21T03:39:10.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:10.385+0000 7fba41741640 1 -- 172.21.15.38:0/3449415181 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba3c0b1990 con 0x7fba3c0b2e40 2024-09-21T03:39:10.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:10.385+0000 7fba337fe640 1 -- 172.21.15.38:0/3449415181 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba2c010690 con 0x7fba3c0b2e40 2024-09-21T03:39:10.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:10.385+0000 7fba41741640 1 -- 172.21.15.38:0/3449415181 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba3c0b1f50 con 0x7fba3c0b2e40 2024-09-21T03:39:10.390 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:10.386+0000 7fba41741640 1 -- 172.21.15.38:0/3449415181 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba3c162f00 con 0x7fba3c0b2e40 2024-09-21T03:39:10.390 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:10.387+0000 7fba337fe640 1 -- 172.21.15.38:0/3449415181 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fba2c010830 con 0x7fba3c0b2e40 2024-09-21T03:39:10.390 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:10.388+0000 7fba337fe640 1 --2- 172.21.15.38:0/3449415181 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fba1c078b90 0x7fba1c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:10.390 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:10.388+0000 7fba337fe640 1 -- 172.21.15.38:0/3449415181 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 6101+0+0 (secure 0 0 0) 0x7fba2c09c000 con 0x7fba3c0b2e40 2024-09-21T03:39:10.390 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:10.388+0000 7fba33fff640 1 --2- 172.21.15.38:0/3449415181 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fba1c078b90 0x7fba1c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:10.390 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:10.389+0000 7fba33fff640 1 --2- 172.21.15.38:0/3449415181 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fba1c078b90 0x7fba1c07b0a0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fba24033000 tx=0x7fba24004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:10.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:10.391+0000 7fba337fe640 1 -- 172.21.15.38:0/3449415181 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fba3c1f2a70 con 0x7fba3c0b2e40 2024-09-21T03:39:10.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:10.656+0000 7fba41741640 1 -- 172.21.15.38:0/3449415181 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7fba3c0cfc40 con 0x7fba3c0b2e40 2024-09-21T03:39:10.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:10.657+0000 7fba337fe640 1 -- 172.21.15.38:0/3449415181 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1142 (secure 0 0 0) 0x7fba2c063cd0 con 0x7fba3c0b2e40 2024-09-21T03:39:10.659 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:39:10.659 INFO:teuthology.orchestra.run.smithi038.stdout:{"fsid":"9d50d66f-656b-4fe2-8e3b-fe53b79c4d43","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":359,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":82,"num_osds":8,"num_up_osds":8,"osd_up_since":1726889604,"num_in_osds":8,"osd_in_since":1726889594,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":90},{"state_name":"creating+peering","count":7}],"num_pgs":97,"num_pools":3,"num_objects":20,"data_bytes":592744,"bytes_used":281739264,"bytes_avail":772812374016,"bytes_total":773094113280,"inactive_pgs_ratio":0.072164945304393768,"write_bytes_sec":1260,"read_op_per_sec":0,"write_op_per_sec":6},"fsmap":{"epoch":67,"btime":"2024-09-21T03:39:08:424230+0000","id":5,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":5,"rank":0,"name":"c","status":"up:active","gid":6546}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":15,"modified":"2024-09-21T03:38:19.058208+0000","services":{}},"progress_events":{}} 2024-09-21T03:39:10.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:10.659+0000 7fba41741640 1 -- 172.21.15.38:0/3449415181 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fba1c078b90 msgr2=0x7fba1c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:10.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:10.659+0000 7fba41741640 1 --2- 172.21.15.38:0/3449415181 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fba1c078b90 0x7fba1c07b0a0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fba24033000 tx=0x7fba24004730 comp rx=0 tx=0).stop 2024-09-21T03:39:10.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:10.659+0000 7fba41741640 1 -- 172.21.15.38:0/3449415181 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fba3c0b2e40 msgr2=0x7fba3c0b2750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:10.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:10.659+0000 7fba41741640 1 --2- 172.21.15.38:0/3449415181 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fba3c0b2e40 0x7fba3c0b2750 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fba2c00e790 tx=0x7fba2c00ec50 comp rx=0 tx=0).stop 2024-09-21T03:39:10.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:10.659+0000 7fba41741640 1 -- 172.21.15.38:0/3449415181 shutdown_connections 2024-09-21T03:39:10.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:10.659+0000 7fba41741640 1 --2- 172.21.15.38:0/3449415181 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fba1c078b90 0x7fba1c07b0a0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:10.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:10.659+0000 7fba41741640 1 --2- 172.21.15.38:0/3449415181 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fba3c0aadc0 0x7fba3c0ab1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:10.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:10.659+0000 7fba41741640 1 --2- 172.21.15.38:0/3449415181 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fba3c162280 0x7fba3c1f2530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:10.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:10.659+0000 7fba41741640 1 --2- 172.21.15.38:0/3449415181 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fba3c0b2e40 0x7fba3c0b2750 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:10.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:10.659+0000 7fba41741640 1 -- 172.21.15.38:0/3449415181 >> 172.21.15.38:0/3449415181 conn(0x7fba3c0b6b20 msgr2=0x7fba3c06dc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:10.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:10.659+0000 7fba41741640 1 -- 172.21.15.38:0/3449415181 shutdown_connections 2024-09-21T03:39:10.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:10.659+0000 7fba41741640 1 -- 172.21.15.38:0/3449415181 wait complete. 2024-09-21T03:39:10.672 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-21T03:39:10.672 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T03:39:10.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:10.786+0000 7fb360577640 1 -- 172.21.15.38:0/1414564064 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb358162590 msgr2=0x7fb3581629a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:10.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:10.786+0000 7fb360577640 1 --2- 172.21.15.38:0/1414564064 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb358162590 0x7fb3581629a0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fb3580b5a40 tx=0x7fb34802a5f0 comp rx=0 tx=0).stop 2024-09-21T03:39:10.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:10.787+0000 7fb360577640 1 -- 172.21.15.38:0/1414564064 shutdown_connections 2024-09-21T03:39:10.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:10.787+0000 7fb360577640 1 --2- 172.21.15.38:0/1414564064 >> v2:172.21.15.195:3300/0 conn(0x7fb3580b2e60 0x7fb3580b3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:10.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:10.787+0000 7fb360577640 1 --2- 172.21.15.38:0/1414564064 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb358162590 0x7fb3581629a0 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:10.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:10.787+0000 7fb360577640 1 -- 172.21.15.38:0/1414564064 >> 172.21.15.38:0/1414564064 conn(0x7fb3580d4440 msgr2=0x7fb3580d4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:10.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:10.787+0000 7fb360577640 1 -- 172.21.15.38:0/1414564064 shutdown_connections 2024-09-21T03:39:10.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:10.787+0000 7fb360577640 1 -- 172.21.15.38:0/1414564064 wait complete. 2024-09-21T03:39:10.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:10.789+0000 7fb360577640 1 Processor -- start 2024-09-21T03:39:10.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:10.789+0000 7fb360577640 1 -- start start 2024-09-21T03:39:10.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:10.790+0000 7fb360577640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb3580b2e60 0x7fb35815dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:10.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:10.790+0000 7fb360577640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb3580cfe00 0x7fb3580c94e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:10.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:10.790+0000 7fb360577640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb358162590 0x7fb3581f8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:10.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:10.790+0000 7fb360577640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fb3580b3930 con 0x7fb3580b2e60 2024-09-21T03:39:10.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:10.790+0000 7fb360577640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fb3580b37b0 con 0x7fb358162590 2024-09-21T03:39:10.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:10.790+0000 7fb360577640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fb3580b3ab0 con 0x7fb3580cfe00 2024-09-21T03:39:10.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:10.790+0000 7fb35e313640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb3580b2e60 0x7fb35815dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:10.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:10.790+0000 7fb35e313640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb3580b2e60 0x7fb35815dbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:52760/0 (socket says 172.21.15.38:52760) 2024-09-21T03:39:10.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:10.790+0000 7fb35e313640 1 -- 172.21.15.38:0/118108053 learned_addr learned my addr 172.21.15.38:0/118108053 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:39:10.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:10.790+0000 7fb35eb14640 1 --2- 172.21.15.38:0/118108053 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb358162590 0x7fb3581f8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:10.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:10.790+0000 7fb35db12640 1 --2- 172.21.15.38:0/118108053 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb3580cfe00 0x7fb3580c94e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:10.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:10.790+0000 7fb35e313640 1 -- 172.21.15.38:0/118108053 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb3580cfe00 msgr2=0x7fb3580c94e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:10.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:10.790+0000 7fb35e313640 1 --2- 172.21.15.38:0/118108053 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb3580cfe00 0x7fb3580c94e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:10.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:10.790+0000 7fb35e313640 1 -- 172.21.15.38:0/118108053 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb358162590 msgr2=0x7fb3581f8af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:10.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:10.790+0000 7fb35e313640 1 --2- 172.21.15.38:0/118108053 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb358162590 0x7fb3581f8af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:10.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:10.790+0000 7fb35e313640 1 -- 172.21.15.38:0/118108053 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb35815abe0 con 0x7fb3580b2e60 2024-09-21T03:39:10.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:10.791+0000 7fb35e313640 1 --2- 172.21.15.38:0/118108053 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb3580b2e60 0x7fb35815dbb0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fb35400c3d0 tx=0x7fb35400c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:10.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:10.791+0000 7fb34effd640 1 -- 172.21.15.38:0/118108053 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb354017070 con 0x7fb3580b2e60 2024-09-21T03:39:10.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:10.791+0000 7fb34effd640 1 -- 172.21.15.38:0/118108053 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb3540053c0 con 0x7fb3580b2e60 2024-09-21T03:39:10.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:10.791+0000 7fb360577640 1 -- 172.21.15.38:0/118108053 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb35815aec0 con 0x7fb3580b2e60 2024-09-21T03:39:10.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:10.791+0000 7fb34effd640 1 -- 172.21.15.38:0/118108053 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb354003850 con 0x7fb3580b2e60 2024-09-21T03:39:10.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:10.791+0000 7fb360577640 1 -- 172.21.15.38:0/118108053 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb35815b480 con 0x7fb3580b2e60 2024-09-21T03:39:10.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:10.792+0000 7fb360577640 1 -- 172.21.15.38:0/118108053 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3580b3270 con 0x7fb3580b2e60 2024-09-21T03:39:10.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:10.793+0000 7fb34effd640 1 -- 172.21.15.38:0/118108053 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb35401a070 con 0x7fb3580b2e60 2024-09-21T03:39:10.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:10.793+0000 7fb34effd640 1 --2- 172.21.15.38:0/118108053 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb33c078b90 0x7fb33c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:10.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:10.793+0000 7fb34effd640 1 -- 172.21.15.38:0/118108053 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 6101+0+0 (secure 0 0 0) 0x7fb35409a0a0 con 0x7fb3580b2e60 2024-09-21T03:39:10.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:10.794+0000 7fb35db12640 1 --2- 172.21.15.38:0/118108053 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb33c078b90 0x7fb33c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:10.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:10.794+0000 7fb35db12640 1 --2- 172.21.15.38:0/118108053 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb33c078b90 0x7fb33c07b0a0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fb348033000 tx=0x7fb348004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:10.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:10.798+0000 7fb34effd640 1 -- 172.21.15.38:0/118108053 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fb358162ee0 con 0x7fb3580b2e60 2024-09-21T03:39:10.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:10.984+0000 7fb360577640 1 -- 172.21.15.38:0/118108053 --> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb3580ceca0 con 0x7fb33c078b90 2024-09-21T03:39:10.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:10.989+0000 7fb34effd640 1 -- 172.21.15.38:0/118108053 <== mgr.4117 v2:172.21.15.195:6800/1631901344 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251482 (secure 0 0 0) 0x7fb3580ceca0 con 0x7fb33c078b90 2024-09-21T03:39:10.993 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:39:10.994 INFO:teuthology.orchestra.run.smithi038.stderr:dumped all 2024-09-21T03:39:10.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:10.991+0000 7fb360577640 1 -- 172.21.15.38:0/118108053 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb33c078b90 msgr2=0x7fb33c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:10.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:10.991+0000 7fb360577640 1 --2- 172.21.15.38:0/118108053 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb33c078b90 0x7fb33c07b0a0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fb348033000 tx=0x7fb348004730 comp rx=0 tx=0).stop 2024-09-21T03:39:10.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:10.991+0000 7fb360577640 1 -- 172.21.15.38:0/118108053 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb3580b2e60 msgr2=0x7fb35815dbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:10.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:10.991+0000 7fb360577640 1 --2- 172.21.15.38:0/118108053 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb3580b2e60 0x7fb35815dbb0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fb35400c3d0 tx=0x7fb35400c890 comp rx=0 tx=0).stop 2024-09-21T03:39:10.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:10.991+0000 7fb360577640 1 -- 172.21.15.38:0/118108053 shutdown_connections 2024-09-21T03:39:10.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:10.991+0000 7fb360577640 1 --2- 172.21.15.38:0/118108053 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb358162590 0x7fb3581f8af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:10.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:10.991+0000 7fb360577640 1 --2- 172.21.15.38:0/118108053 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb33c078b90 0x7fb33c07b0a0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:10.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:10.991+0000 7fb360577640 1 --2- 172.21.15.38:0/118108053 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb3580cfe00 0x7fb3580c94e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:10.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:10.991+0000 7fb360577640 1 --2- 172.21.15.38:0/118108053 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb3580b2e60 0x7fb35815dbb0 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:10.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:10.991+0000 7fb360577640 1 -- 172.21.15.38:0/118108053 >> 172.21.15.38:0/118108053 conn(0x7fb3580d4440 msgr2=0x7fb3580d15a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:10.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:10.992+0000 7fb360577640 1 -- 172.21.15.38:0/118108053 shutdown_connections 2024-09-21T03:39:10.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:10.992+0000 7fb360577640 1 -- 172.21.15.38:0/118108053 wait complete. 2024-09-21T03:39:11.005 INFO:teuthology.orchestra.run.smithi038.stdout:{"pg_ready":true,"pg_map":{"version":248,"stamp":"2024-09-21T03:39:09.071943+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592744,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":137,"num_write_kb":1635,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":105,"ondisk_log_size":105,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":154,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":275136,"kb_used_data":8960,"kb_used_omap":12,"kb_used_meta":265971,"kb_avail":754699584,"statfs":{"total":773094113280,"available":772812374016,"internally_reserved":0,"allocated":9175040,"data_stored":4848499,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12748,"internal_metadata":272354868},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2376,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":36,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.687763"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-09-21T03:39:04.404050+0000","last_change":"2024-09-21T03:39:04.404050+0000","last_active":"2024-09-21T03:39:04.383211+0000","last_peered":"2024-09-21T03:39:04.383211+0000","last_clean":"2024-09-21T03:39:04.383211+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:39:04.404050+0000","last_undegraded":"2024-09-21T03:39:04.404050+0000","last_fullsized":"2024-09-21T03:39:04.404050+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.419248+0000","last_change":"2024-09-21T03:39:05.431615+0000","last_active":"2024-09-21T03:39:06.419248+0000","last_peered":"2024-09-21T03:39:06.419248+0000","last_clean":"2024-09-21T03:39:06.419248+0000","last_became_active":"2024-09-21T03:39:05.429700+0000","last_became_peered":"2024-09-21T03:39:05.429700+0000","last_unstale":"2024-09-21T03:39:06.419248+0000","last_undegraded":"2024-09-21T03:39:06.419248+0000","last_fullsized":"2024-09-21T03:39:06.419248+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:22:26.967805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.419361+0000","last_change":"2024-09-21T03:39:05.434451+0000","last_active":"2024-09-21T03:39:06.419361+0000","last_peered":"2024-09-21T03:39:06.419361+0000","last_clean":"2024-09-21T03:39:06.419361+0000","last_became_active":"2024-09-21T03:39:05.432952+0000","last_became_peered":"2024-09-21T03:39:05.432952+0000","last_unstale":"2024-09-21T03:39:06.419361+0000","last_undegraded":"2024-09-21T03:39:06.419361+0000","last_fullsized":"2024-09-21T03:39:06.419361+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:30:48.915097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.569594+0000","last_change":"2024-09-21T03:39:05.426032+0000","last_active":"2024-09-21T03:39:06.569594+0000","last_peered":"2024-09-21T03:39:06.569594+0000","last_clean":"2024-09-21T03:39:06.569594+0000","last_became_active":"2024-09-21T03:39:05.425731+0000","last_became_peered":"2024-09-21T03:39:05.425731+0000","last_unstale":"2024-09-21T03:39:06.569594+0000","last_undegraded":"2024-09-21T03:39:06.569594+0000","last_fullsized":"2024-09-21T03:39:06.569594+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:01:01.380435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.570162+0000","last_change":"2024-09-21T03:39:05.428805+0000","last_active":"2024-09-21T03:39:06.570162+0000","last_peered":"2024-09-21T03:39:06.570162+0000","last_clean":"2024-09-21T03:39:06.570162+0000","last_became_active":"2024-09-21T03:39:05.428394+0000","last_became_peered":"2024-09-21T03:39:05.428394+0000","last_unstale":"2024-09-21T03:39:06.570162+0000","last_undegraded":"2024-09-21T03:39:06.570162+0000","last_fullsized":"2024-09-21T03:39:06.570162+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:57:28.504333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.415502+0000","last_change":"2024-09-21T03:39:05.418164+0000","last_active":"2024-09-21T03:39:06.415502+0000","last_peered":"2024-09-21T03:39:06.415502+0000","last_clean":"2024-09-21T03:39:06.415502+0000","last_became_active":"2024-09-21T03:39:05.416861+0000","last_became_peered":"2024-09-21T03:39:05.416861+0000","last_unstale":"2024-09-21T03:39:06.415502+0000","last_undegraded":"2024-09-21T03:39:06.415502+0000","last_fullsized":"2024-09-21T03:39:06.415502+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:06:43.522600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.569829+0000","last_change":"2024-09-21T03:39:05.429537+0000","last_active":"2024-09-21T03:39:06.569829+0000","last_peered":"2024-09-21T03:39:06.569829+0000","last_clean":"2024-09-21T03:39:06.569829+0000","last_became_active":"2024-09-21T03:39:05.428077+0000","last_became_peered":"2024-09-21T03:39:05.428077+0000","last_unstale":"2024-09-21T03:39:06.569829+0000","last_undegraded":"2024-09-21T03:39:06.569829+0000","last_fullsized":"2024-09-21T03:39:06.569829+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:38:28.792089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.414574+0000","last_change":"2024-09-21T03:39:05.420868+0000","last_active":"2024-09-21T03:39:06.414574+0000","last_peered":"2024-09-21T03:39:06.414574+0000","last_clean":"2024-09-21T03:39:06.414574+0000","last_became_active":"2024-09-21T03:39:05.420546+0000","last_became_peered":"2024-09-21T03:39:05.420546+0000","last_unstale":"2024-09-21T03:39:06.414574+0000","last_undegraded":"2024-09-21T03:39:06.414574+0000","last_fullsized":"2024-09-21T03:39:06.414574+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:00:25.925784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.570276+0000","last_change":"2024-09-21T03:39:05.424217+0000","last_active":"2024-09-21T03:39:06.570276+0000","last_peered":"2024-09-21T03:39:06.570276+0000","last_clean":"2024-09-21T03:39:06.570276+0000","last_became_active":"2024-09-21T03:39:05.423922+0000","last_became_peered":"2024-09-21T03:39:05.423922+0000","last_unstale":"2024-09-21T03:39:06.570276+0000","last_undegraded":"2024-09-21T03:39:06.570276+0000","last_fullsized":"2024-09-21T03:39:06.570276+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:30:48.915097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-09-21T03:39:04.403441+0000","last_change":"2024-09-21T03:39:04.403441+0000","last_active":"2024-09-21T03:39:04.383211+0000","last_peered":"2024-09-21T03:39:04.383211+0000","last_clean":"2024-09-21T03:39:04.383211+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:39:04.403441+0000","last_undegraded":"2024-09-21T03:39:04.403441+0000","last_fullsized":"2024-09-21T03:39:04.403441+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.419460+0000","last_change":"2024-09-21T03:39:05.427718+0000","last_active":"2024-09-21T03:39:06.419460+0000","last_peered":"2024-09-21T03:39:06.419460+0000","last_clean":"2024-09-21T03:39:06.419460+0000","last_became_active":"2024-09-21T03:39:05.427387+0000","last_became_peered":"2024-09-21T03:39:05.427387+0000","last_unstale":"2024-09-21T03:39:06.419460+0000","last_undegraded":"2024-09-21T03:39:06.419460+0000","last_fullsized":"2024-09-21T03:39:06.419460+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:58:09.229781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.570387+0000","last_change":"2024-09-21T03:39:05.421453+0000","last_active":"2024-09-21T03:39:06.570387+0000","last_peered":"2024-09-21T03:39:06.570387+0000","last_clean":"2024-09-21T03:39:06.570387+0000","last_became_active":"2024-09-21T03:39:05.421133+0000","last_became_peered":"2024-09-21T03:39:05.421133+0000","last_unstale":"2024-09-21T03:39:06.570387+0000","last_undegraded":"2024-09-21T03:39:06.570387+0000","last_fullsized":"2024-09-21T03:39:06.570387+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:30:07.399497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.418721+0000","last_change":"2024-09-21T03:39:05.426022+0000","last_active":"2024-09-21T03:39:06.418721+0000","last_peered":"2024-09-21T03:39:06.418721+0000","last_clean":"2024-09-21T03:39:06.418721+0000","last_became_active":"2024-09-21T03:39:05.425625+0000","last_became_peered":"2024-09-21T03:39:05.425625+0000","last_unstale":"2024-09-21T03:39:06.418721+0000","last_undegraded":"2024-09-21T03:39:06.418721+0000","last_fullsized":"2024-09-21T03:39:06.418721+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:03:54.230793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.414472+0000","last_change":"2024-09-21T03:39:05.421310+0000","last_active":"2024-09-21T03:39:06.414472+0000","last_peered":"2024-09-21T03:39:06.414472+0000","last_clean":"2024-09-21T03:39:06.414472+0000","last_became_active":"2024-09-21T03:39:05.420314+0000","last_became_peered":"2024-09-21T03:39:05.420314+0000","last_unstale":"2024-09-21T03:39:06.414472+0000","last_undegraded":"2024-09-21T03:39:06.414472+0000","last_fullsized":"2024-09-21T03:39:06.414472+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:00:07.867021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.570061+0000","last_change":"2024-09-21T03:39:05.427360+0000","last_active":"2024-09-21T03:39:06.570061+0000","last_peered":"2024-09-21T03:39:06.570061+0000","last_clean":"2024-09-21T03:39:06.570061+0000","last_became_active":"2024-09-21T03:39:05.427009+0000","last_became_peered":"2024-09-21T03:39:05.427009+0000","last_unstale":"2024-09-21T03:39:06.570061+0000","last_undegraded":"2024-09-21T03:39:06.570061+0000","last_fullsized":"2024-09-21T03:39:06.570061+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:54:27.965337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.569046+0000","last_change":"2024-09-21T03:39:05.430747+0000","last_active":"2024-09-21T03:39:06.569046+0000","last_peered":"2024-09-21T03:39:06.569046+0000","last_clean":"2024-09-21T03:39:06.569046+0000","last_became_active":"2024-09-21T03:39:05.429872+0000","last_became_peered":"2024-09-21T03:39:05.429872+0000","last_unstale":"2024-09-21T03:39:06.569046+0000","last_undegraded":"2024-09-21T03:39:06.569046+0000","last_fullsized":"2024-09-21T03:39:06.569046+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:01:18.107849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.415027+0000","last_change":"2024-09-21T03:39:05.418812+0000","last_active":"2024-09-21T03:39:06.415027+0000","last_peered":"2024-09-21T03:39:06.415027+0000","last_clean":"2024-09-21T03:39:06.415027+0000","last_became_active":"2024-09-21T03:39:05.418487+0000","last_became_peered":"2024-09-21T03:39:05.418487+0000","last_unstale":"2024-09-21T03:39:06.415027+0000","last_undegraded":"2024-09-21T03:39:06.415027+0000","last_fullsized":"2024-09-21T03:39:06.415027+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:37:29.087437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.419123+0000","last_change":"2024-09-21T03:39:05.428238+0000","last_active":"2024-09-21T03:39:06.419123+0000","last_peered":"2024-09-21T03:39:06.419123+0000","last_clean":"2024-09-21T03:39:06.419123+0000","last_became_active":"2024-09-21T03:39:05.427915+0000","last_became_peered":"2024-09-21T03:39:05.427915+0000","last_unstale":"2024-09-21T03:39:06.419123+0000","last_undegraded":"2024-09-21T03:39:06.419123+0000","last_fullsized":"2024-09-21T03:39:06.419123+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:22:48.194041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.414304+0000","last_change":"2024-09-21T03:39:05.412642+0000","last_active":"2024-09-21T03:39:06.414304+0000","last_peered":"2024-09-21T03:39:06.414304+0000","last_clean":"2024-09-21T03:39:06.414304+0000","last_became_active":"2024-09-21T03:39:05.412274+0000","last_became_peered":"2024-09-21T03:39:05.412274+0000","last_unstale":"2024-09-21T03:39:06.414304+0000","last_undegraded":"2024-09-21T03:39:06.414304+0000","last_fullsized":"2024-09-21T03:39:06.414304+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:24:30.153279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.415023+0000","last_change":"2024-09-21T03:39:05.428953+0000","last_active":"2024-09-21T03:39:06.415023+0000","last_peered":"2024-09-21T03:39:06.415023+0000","last_clean":"2024-09-21T03:39:06.415023+0000","last_became_active":"2024-09-21T03:39:05.428223+0000","last_became_peered":"2024-09-21T03:39:05.428223+0000","last_unstale":"2024-09-21T03:39:06.415023+0000","last_undegraded":"2024-09-21T03:39:06.415023+0000","last_fullsized":"2024-09-21T03:39:06.415023+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:16:38.764525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.570578+0000","last_change":"2024-09-21T03:39:05.409645+0000","last_active":"2024-09-21T03:39:06.570578+0000","last_peered":"2024-09-21T03:39:06.570578+0000","last_clean":"2024-09-21T03:39:06.570578+0000","last_became_active":"2024-09-21T03:39:05.409315+0000","last_became_peered":"2024-09-21T03:39:05.409315+0000","last_unstale":"2024-09-21T03:39:06.570578+0000","last_undegraded":"2024-09-21T03:39:06.570578+0000","last_fullsized":"2024-09-21T03:39:06.570578+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:49:12.163503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.418882+0000","last_change":"2024-09-21T03:39:05.427679+0000","last_active":"2024-09-21T03:39:06.418882+0000","last_peered":"2024-09-21T03:39:06.418882+0000","last_clean":"2024-09-21T03:39:06.418882+0000","last_became_active":"2024-09-21T03:39:05.426044+0000","last_became_peered":"2024-09-21T03:39:05.426044+0000","last_unstale":"2024-09-21T03:39:06.418882+0000","last_undegraded":"2024-09-21T03:39:06.418882+0000","last_fullsized":"2024-09-21T03:39:06.418882+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:20:35.883679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-09-21T03:39:04.402899+0000","last_change":"2024-09-21T03:39:04.402899+0000","last_active":"2024-09-21T03:39:04.383211+0000","last_peered":"2024-09-21T03:39:04.383211+0000","last_clean":"2024-09-21T03:39:04.383211+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:39:04.402899+0000","last_undegraded":"2024-09-21T03:39:04.402899+0000","last_fullsized":"2024-09-21T03:39:04.402899+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.418484+0000","last_change":"2024-09-21T03:39:05.427956+0000","last_active":"2024-09-21T03:39:06.418484+0000","last_peered":"2024-09-21T03:39:06.418484+0000","last_clean":"2024-09-21T03:39:06.418484+0000","last_became_active":"2024-09-21T03:39:05.425708+0000","last_became_peered":"2024-09-21T03:39:05.425708+0000","last_unstale":"2024-09-21T03:39:06.418484+0000","last_undegraded":"2024-09-21T03:39:06.418484+0000","last_fullsized":"2024-09-21T03:39:06.418484+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:14:22.373008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.415419+0000","last_change":"2024-09-21T03:39:05.420911+0000","last_active":"2024-09-21T03:39:06.415419+0000","last_peered":"2024-09-21T03:39:06.415419+0000","last_clean":"2024-09-21T03:39:06.415419+0000","last_became_active":"2024-09-21T03:39:05.420587+0000","last_became_peered":"2024-09-21T03:39:05.420587+0000","last_unstale":"2024-09-21T03:39:06.415419+0000","last_undegraded":"2024-09-21T03:39:06.415419+0000","last_fullsized":"2024-09-21T03:39:06.415419+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:37:32.783221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.569302+0000","last_change":"2024-09-21T03:39:05.426555+0000","last_active":"2024-09-21T03:39:06.569302+0000","last_peered":"2024-09-21T03:39:06.569302+0000","last_clean":"2024-09-21T03:39:06.569302+0000","last_became_active":"2024-09-21T03:39:05.426265+0000","last_became_peered":"2024-09-21T03:39:05.426265+0000","last_unstale":"2024-09-21T03:39:06.569302+0000","last_undegraded":"2024-09-21T03:39:06.569302+0000","last_fullsized":"2024-09-21T03:39:06.569302+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:03:54.230793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.569945+0000","last_change":"2024-09-21T03:39:05.427771+0000","last_active":"2024-09-21T03:39:06.569945+0000","last_peered":"2024-09-21T03:39:06.569945+0000","last_clean":"2024-09-21T03:39:06.569945+0000","last_became_active":"2024-09-21T03:39:05.427310+0000","last_became_peered":"2024-09-21T03:39:05.427310+0000","last_unstale":"2024-09-21T03:39:06.569945+0000","last_undegraded":"2024-09-21T03:39:06.569945+0000","last_fullsized":"2024-09-21T03:39:06.569945+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:22:48.194041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.570546+0000","last_change":"2024-09-21T03:39:05.431170+0000","last_active":"2024-09-21T03:39:06.570546+0000","last_peered":"2024-09-21T03:39:06.570546+0000","last_clean":"2024-09-21T03:39:06.570546+0000","last_became_active":"2024-09-21T03:39:05.430855+0000","last_became_peered":"2024-09-21T03:39:05.430855+0000","last_unstale":"2024-09-21T03:39:06.570546+0000","last_undegraded":"2024-09-21T03:39:06.570546+0000","last_fullsized":"2024-09-21T03:39:06.570546+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:13:27.854189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.414268+0000","last_change":"2024-09-21T03:39:05.429739+0000","last_active":"2024-09-21T03:39:06.414268+0000","last_peered":"2024-09-21T03:39:06.414268+0000","last_clean":"2024-09-21T03:39:06.414268+0000","last_became_active":"2024-09-21T03:39:05.429401+0000","last_became_peered":"2024-09-21T03:39:05.429401+0000","last_unstale":"2024-09-21T03:39:06.414268+0000","last_undegraded":"2024-09-21T03:39:06.414268+0000","last_fullsized":"2024-09-21T03:39:06.414268+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:29:58.646678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-09-21T03:39:04.402307+0000","last_change":"2024-09-21T03:39:04.402307+0000","last_active":"2024-09-21T03:39:04.383211+0000","last_peered":"2024-09-21T03:39:04.383211+0000","last_clean":"2024-09-21T03:39:04.383211+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:39:04.402307+0000","last_undegraded":"2024-09-21T03:39:04.402307+0000","last_fullsized":"2024-09-21T03:39:04.402307+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.418906+0000","last_change":"2024-09-21T03:39:05.430619+0000","last_active":"2024-09-21T03:39:06.418906+0000","last_peered":"2024-09-21T03:39:06.418906+0000","last_clean":"2024-09-21T03:39:06.418906+0000","last_became_active":"2024-09-21T03:39:05.429959+0000","last_became_peered":"2024-09-21T03:39:05.429959+0000","last_unstale":"2024-09-21T03:39:06.418906+0000","last_undegraded":"2024-09-21T03:39:06.418906+0000","last_fullsized":"2024-09-21T03:39:06.418906+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:26:04.045027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.569279+0000","last_change":"2024-09-21T03:39:05.422391+0000","last_active":"2024-09-21T03:39:06.569279+0000","last_peered":"2024-09-21T03:39:06.569279+0000","last_clean":"2024-09-21T03:39:06.569279+0000","last_became_active":"2024-09-21T03:39:05.422073+0000","last_became_peered":"2024-09-21T03:39:05.422073+0000","last_unstale":"2024-09-21T03:39:06.569279+0000","last_undegraded":"2024-09-21T03:39:06.569279+0000","last_fullsized":"2024-09-21T03:39:06.569279+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:14:22.373008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.570338+0000","last_change":"2024-09-21T03:39:05.430675+0000","last_active":"2024-09-21T03:39:06.570338+0000","last_peered":"2024-09-21T03:39:06.570338+0000","last_clean":"2024-09-21T03:39:06.570338+0000","last_became_active":"2024-09-21T03:39:05.430203+0000","last_became_peered":"2024-09-21T03:39:05.430203+0000","last_unstale":"2024-09-21T03:39:06.570338+0000","last_undegraded":"2024-09-21T03:39:06.570338+0000","last_fullsized":"2024-09-21T03:39:06.570338+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:27:11.780682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"81'4","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.570214+0000","last_change":"2024-09-21T03:39:04.406626+0000","last_active":"2024-09-21T03:39:06.570214+0000","last_peered":"2024-09-21T03:39:06.570214+0000","last_clean":"2024-09-21T03:39:06.570214+0000","last_became_active":"2024-09-21T03:39:04.406318+0000","last_became_peered":"2024-09-21T03:39:04.406318+0000","last_unstale":"2024-09-21T03:39:06.570214+0000","last_undegraded":"2024-09-21T03:39:06.570214+0000","last_fullsized":"2024-09-21T03:39:06.570214+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_clean_scrub_stamp":"2024-09-21T03:39:03.377315+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:00:13.859613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.569891+0000","last_change":"2024-09-21T03:39:05.428305+0000","last_active":"2024-09-21T03:39:06.569891+0000","last_peered":"2024-09-21T03:39:06.569891+0000","last_clean":"2024-09-21T03:39:06.569891+0000","last_became_active":"2024-09-21T03:39:05.427992+0000","last_became_peered":"2024-09-21T03:39:05.427992+0000","last_unstale":"2024-09-21T03:39:06.569891+0000","last_undegraded":"2024-09-21T03:39:06.569891+0000","last_fullsized":"2024-09-21T03:39:06.569891+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:15:23.391245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1e","version":"81'1","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.570008+0000","last_change":"2024-09-21T03:39:04.411255+0000","last_active":"2024-09-21T03:39:06.570008+0000","last_peered":"2024-09-21T03:39:06.570008+0000","last_clean":"2024-09-21T03:39:06.570008+0000","last_became_active":"2024-09-21T03:39:04.410962+0000","last_became_peered":"2024-09-21T03:39:04.410962+0000","last_unstale":"2024-09-21T03:39:06.570008+0000","last_undegraded":"2024-09-21T03:39:06.570008+0000","last_fullsized":"2024-09-21T03:39:06.570008+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_clean_scrub_stamp":"2024-09-21T03:39:03.377315+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:27:36.848833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.569439+0000","last_change":"2024-09-21T03:39:05.415000+0000","last_active":"2024-09-21T03:39:06.569439+0000","last_peered":"2024-09-21T03:39:06.569439+0000","last_clean":"2024-09-21T03:39:06.569439+0000","last_became_active":"2024-09-21T03:39:05.414695+0000","last_became_peered":"2024-09-21T03:39:05.414695+0000","last_unstale":"2024-09-21T03:39:06.569439+0000","last_undegraded":"2024-09-21T03:39:06.569439+0000","last_fullsized":"2024-09-21T03:39:06.569439+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:58:09.229781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T03:39:04.413513+0000","last_change":"2024-09-21T03:39:04.404636+0000","last_active":"2024-09-21T03:39:04.413513+0000","last_peered":"2024-09-21T03:39:04.413513+0000","last_clean":"2024-09-21T03:39:04.413513+0000","last_became_active":"2024-09-21T03:39:04.404270+0000","last_became_peered":"2024-09-21T03:39:04.404270+0000","last_unstale":"2024-09-21T03:39:04.413513+0000","last_undegraded":"2024-09-21T03:39:04.413513+0000","last_fullsized":"2024-09-21T03:39:04.413513+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_clean_scrub_stamp":"2024-09-21T03:39:03.377315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:09:17.529140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.569256+0000","last_change":"2024-09-21T03:39:05.425119+0000","last_active":"2024-09-21T03:39:06.569256+0000","last_peered":"2024-09-21T03:39:06.569256+0000","last_clean":"2024-09-21T03:39:06.569256+0000","last_became_active":"2024-09-21T03:39:05.424804+0000","last_became_peered":"2024-09-21T03:39:05.424804+0000","last_unstale":"2024-09-21T03:39:06.569256+0000","last_undegraded":"2024-09-21T03:39:06.569256+0000","last_fullsized":"2024-09-21T03:39:06.569256+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:13:42.059728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"81'2","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.570641+0000","last_change":"2024-09-21T03:39:04.407381+0000","last_active":"2024-09-21T03:39:06.570641+0000","last_peered":"2024-09-21T03:39:06.570641+0000","last_clean":"2024-09-21T03:39:06.570641+0000","last_became_active":"2024-09-21T03:39:04.407044+0000","last_became_peered":"2024-09-21T03:39:04.407044+0000","last_unstale":"2024-09-21T03:39:06.570641+0000","last_undegraded":"2024-09-21T03:39:06.570641+0000","last_fullsized":"2024-09-21T03:39:06.570641+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_clean_scrub_stamp":"2024-09-21T03:39:03.377315+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:14:21.367112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.415145+0000","last_change":"2024-09-21T03:39:05.425026+0000","last_active":"2024-09-21T03:39:06.415145+0000","last_peered":"2024-09-21T03:39:06.415145+0000","last_clean":"2024-09-21T03:39:06.415145+0000","last_became_active":"2024-09-21T03:39:05.424667+0000","last_became_peered":"2024-09-21T03:39:05.424667+0000","last_unstale":"2024-09-21T03:39:06.415145+0000","last_undegraded":"2024-09-21T03:39:06.415145+0000","last_fullsized":"2024-09-21T03:39:06.415145+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:46:01.041621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.1b","version":"81'1","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.570542+0000","last_change":"2024-09-21T03:39:04.414608+0000","last_active":"2024-09-21T03:39:06.570542+0000","last_peered":"2024-09-21T03:39:06.570542+0000","last_clean":"2024-09-21T03:39:06.570542+0000","last_became_active":"2024-09-21T03:39:04.414274+0000","last_became_peered":"2024-09-21T03:39:04.414274+0000","last_unstale":"2024-09-21T03:39:06.570542+0000","last_undegraded":"2024-09-21T03:39:06.570542+0000","last_fullsized":"2024-09-21T03:39:06.570542+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_clean_scrub_stamp":"2024-09-21T03:39:03.377315+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:13:41.053832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.419607+0000","last_change":"2024-09-21T03:39:05.434557+0000","last_active":"2024-09-21T03:39:06.419607+0000","last_peered":"2024-09-21T03:39:06.419607+0000","last_clean":"2024-09-21T03:39:06.419607+0000","last_became_active":"2024-09-21T03:39:05.430126+0000","last_became_peered":"2024-09-21T03:39:05.430126+0000","last_unstale":"2024-09-21T03:39:06.419607+0000","last_undegraded":"2024-09-21T03:39:06.419607+0000","last_fullsized":"2024-09-21T03:39:06.419607+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:27:34.908887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"81'1","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.570010+0000","last_change":"2024-09-21T03:39:04.397543+0000","last_active":"2024-09-21T03:39:06.570010+0000","last_peered":"2024-09-21T03:39:06.570010+0000","last_clean":"2024-09-21T03:39:06.570010+0000","last_became_active":"2024-09-21T03:39:04.397220+0000","last_became_peered":"2024-09-21T03:39:04.397220+0000","last_unstale":"2024-09-21T03:39:06.570010+0000","last_undegraded":"2024-09-21T03:39:06.570010+0000","last_fullsized":"2024-09-21T03:39:06.570010+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_clean_scrub_stamp":"2024-09-21T03:39:03.377315+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:21:47.012786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.570317+0000","last_change":"2024-09-21T03:39:05.413837+0000","last_active":"2024-09-21T03:39:06.570317+0000","last_peered":"2024-09-21T03:39:06.570317+0000","last_clean":"2024-09-21T03:39:06.570317+0000","last_became_active":"2024-09-21T03:39:05.413498+0000","last_became_peered":"2024-09-21T03:39:05.413498+0000","last_unstale":"2024-09-21T03:39:06.570317+0000","last_undegraded":"2024-09-21T03:39:06.570317+0000","last_fullsized":"2024-09-21T03:39:06.570317+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:33:39.426980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.413804+0000","last_change":"2024-09-21T03:39:04.408211+0000","last_active":"2024-09-21T03:39:06.413804+0000","last_peered":"2024-09-21T03:39:06.413804+0000","last_clean":"2024-09-21T03:39:06.413804+0000","last_became_active":"2024-09-21T03:39:04.404927+0000","last_became_peered":"2024-09-21T03:39:04.404927+0000","last_unstale":"2024-09-21T03:39:06.413804+0000","last_undegraded":"2024-09-21T03:39:06.413804+0000","last_fullsized":"2024-09-21T03:39:06.413804+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_clean_scrub_stamp":"2024-09-21T03:39:03.377315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:53:29.829387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.569462+0000","last_change":"2024-09-21T03:39:05.419988+0000","last_active":"2024-09-21T03:39:06.569462+0000","last_peered":"2024-09-21T03:39:06.569462+0000","last_clean":"2024-09-21T03:39:06.569462+0000","last_became_active":"2024-09-21T03:39:05.419693+0000","last_became_peered":"2024-09-21T03:39:05.419693+0000","last_unstale":"2024-09-21T03:39:06.569462+0000","last_undegraded":"2024-09-21T03:39:06.569462+0000","last_fullsized":"2024-09-21T03:39:06.569462+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:08:48.101749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.418856+0000","last_change":"2024-09-21T03:39:04.410302+0000","last_active":"2024-09-21T03:39:06.418856+0000","last_peered":"2024-09-21T03:39:06.418856+0000","last_clean":"2024-09-21T03:39:06.418856+0000","last_became_active":"2024-09-21T03:39:04.409959+0000","last_became_peered":"2024-09-21T03:39:04.409959+0000","last_unstale":"2024-09-21T03:39:06.418856+0000","last_undegraded":"2024-09-21T03:39:06.418856+0000","last_fullsized":"2024-09-21T03:39:06.418856+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_clean_scrub_stamp":"2024-09-21T03:39:03.377315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:18:35.050119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.419019+0000","last_change":"2024-09-21T03:39:05.429795+0000","last_active":"2024-09-21T03:39:06.419019+0000","last_peered":"2024-09-21T03:39:06.419019+0000","last_clean":"2024-09-21T03:39:06.419019+0000","last_became_active":"2024-09-21T03:39:05.426181+0000","last_became_peered":"2024-09-21T03:39:05.426181+0000","last_unstale":"2024-09-21T03:39:06.419019+0000","last_undegraded":"2024-09-21T03:39:06.419019+0000","last_fullsized":"2024-09-21T03:39:06.419019+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:38:28.792089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.570075+0000","last_change":"2024-09-21T03:39:04.410453+0000","last_active":"2024-09-21T03:39:06.570075+0000","last_peered":"2024-09-21T03:39:06.570075+0000","last_clean":"2024-09-21T03:39:06.570075+0000","last_became_active":"2024-09-21T03:39:04.410082+0000","last_became_peered":"2024-09-21T03:39:04.410082+0000","last_unstale":"2024-09-21T03:39:06.570075+0000","last_undegraded":"2024-09-21T03:39:06.570075+0000","last_fullsized":"2024-09-21T03:39:06.570075+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_clean_scrub_stamp":"2024-09-21T03:39:03.377315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:56:14.520557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.569202+0000","last_change":"2024-09-21T03:39:05.422217+0000","last_active":"2024-09-21T03:39:06.569202+0000","last_peered":"2024-09-21T03:39:06.569202+0000","last_clean":"2024-09-21T03:39:06.569202+0000","last_became_active":"2024-09-21T03:39:05.421890+0000","last_became_peered":"2024-09-21T03:39:05.421890+0000","last_unstale":"2024-09-21T03:39:06.569202+0000","last_undegraded":"2024-09-21T03:39:06.569202+0000","last_fullsized":"2024-09-21T03:39:06.569202+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:40:45.946118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T03:39:04.414955+0000","last_change":"2024-09-21T03:39:04.411385+0000","last_active":"2024-09-21T03:39:04.414955+0000","last_peered":"2024-09-21T03:39:04.414955+0000","last_clean":"2024-09-21T03:39:04.414955+0000","last_became_active":"2024-09-21T03:39:04.410963+0000","last_became_peered":"2024-09-21T03:39:04.410963+0000","last_unstale":"2024-09-21T03:39:04.414955+0000","last_undegraded":"2024-09-21T03:39:04.414955+0000","last_fullsized":"2024-09-21T03:39:04.414955+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_clean_scrub_stamp":"2024-09-21T03:39:03.377315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:53:58.324899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.414821+0000","last_change":"2024-09-21T03:39:05.419517+0000","last_active":"2024-09-21T03:39:06.414821+0000","last_peered":"2024-09-21T03:39:06.414821+0000","last_clean":"2024-09-21T03:39:06.414821+0000","last_became_active":"2024-09-21T03:39:05.418599+0000","last_became_peered":"2024-09-21T03:39:05.418599+0000","last_unstale":"2024-09-21T03:39:06.414821+0000","last_undegraded":"2024-09-21T03:39:06.414821+0000","last_fullsized":"2024-09-21T03:39:06.414821+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:59:45.194584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.15","version":"81'3","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.413935+0000","last_change":"2024-09-21T03:39:04.407595+0000","last_active":"2024-09-21T03:39:06.413935+0000","last_peered":"2024-09-21T03:39:06.413935+0000","last_clean":"2024-09-21T03:39:06.413935+0000","last_became_active":"2024-09-21T03:39:04.407278+0000","last_became_peered":"2024-09-21T03:39:04.407278+0000","last_unstale":"2024-09-21T03:39:06.413935+0000","last_undegraded":"2024-09-21T03:39:06.413935+0000","last_fullsized":"2024-09-21T03:39:06.413935+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_clean_scrub_stamp":"2024-09-21T03:39:03.377315+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:41:01.617219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.414516+0000","last_change":"2024-09-21T03:39:05.419678+0000","last_active":"2024-09-21T03:39:06.414516+0000","last_peered":"2024-09-21T03:39:06.414516+0000","last_clean":"2024-09-21T03:39:06.414516+0000","last_became_active":"2024-09-21T03:39:05.419119+0000","last_became_peered":"2024-09-21T03:39:05.419119+0000","last_unstale":"2024-09-21T03:39:06.414516+0000","last_undegraded":"2024-09-21T03:39:06.414516+0000","last_fullsized":"2024-09-21T03:39:06.414516+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:03:30.086684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.14","version":"81'6","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.569939+0000","last_change":"2024-09-21T03:39:04.396680+0000","last_active":"2024-09-21T03:39:06.569939+0000","last_peered":"2024-09-21T03:39:06.569939+0000","last_clean":"2024-09-21T03:39:06.569939+0000","last_became_active":"2024-09-21T03:39:04.396344+0000","last_became_peered":"2024-09-21T03:39:04.396344+0000","last_unstale":"2024-09-21T03:39:06.569939+0000","last_undegraded":"2024-09-21T03:39:06.569939+0000","last_fullsized":"2024-09-21T03:39:06.569939+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_clean_scrub_stamp":"2024-09-21T03:39:03.377315+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:12:41.701014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.569672+0000","last_change":"2024-09-21T03:39:05.423246+0000","last_active":"2024-09-21T03:39:06.569672+0000","last_peered":"2024-09-21T03:39:06.569672+0000","last_clean":"2024-09-21T03:39:06.569672+0000","last_became_active":"2024-09-21T03:39:05.422924+0000","last_became_peered":"2024-09-21T03:39:05.422924+0000","last_unstale":"2024-09-21T03:39:06.569672+0000","last_undegraded":"2024-09-21T03:39:06.569672+0000","last_fullsized":"2024-09-21T03:39:06.569672+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:51:42.048958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T03:39:04.407961+0000","last_change":"2024-09-21T03:39:04.402594+0000","last_active":"2024-09-21T03:39:04.407961+0000","last_peered":"2024-09-21T03:39:04.407961+0000","last_clean":"2024-09-21T03:39:04.407961+0000","last_became_active":"2024-09-21T03:39:04.402242+0000","last_became_peered":"2024-09-21T03:39:04.402242+0000","last_unstale":"2024-09-21T03:39:04.407961+0000","last_undegraded":"2024-09-21T03:39:04.407961+0000","last_fullsized":"2024-09-21T03:39:04.407961+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_clean_scrub_stamp":"2024-09-21T03:39:03.377315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:23:27.346296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-09-21T03:39:04.401407+0000","last_change":"2024-09-21T03:39:04.401407+0000","last_active":"2024-09-21T03:39:04.383211+0000","last_peered":"2024-09-21T03:39:04.383211+0000","last_clean":"2024-09-21T03:39:04.383211+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:39:04.401407+0000","last_undegraded":"2024-09-21T03:39:04.401407+0000","last_fullsized":"2024-09-21T03:39:04.401407+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.569717+0000","last_change":"2024-09-21T03:39:04.409288+0000","last_active":"2024-09-21T03:39:06.569717+0000","last_peered":"2024-09-21T03:39:06.569717+0000","last_clean":"2024-09-21T03:39:06.569717+0000","last_became_active":"2024-09-21T03:39:04.408965+0000","last_became_peered":"2024-09-21T03:39:04.408965+0000","last_unstale":"2024-09-21T03:39:06.569717+0000","last_undegraded":"2024-09-21T03:39:06.569717+0000","last_fullsized":"2024-09-21T03:39:06.569717+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_clean_scrub_stamp":"2024-09-21T03:39:03.377315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:01:17.101953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.570597+0000","last_change":"2024-09-21T03:39:05.423231+0000","last_active":"2024-09-21T03:39:06.570597+0000","last_peered":"2024-09-21T03:39:06.570597+0000","last_clean":"2024-09-21T03:39:06.570597+0000","last_became_active":"2024-09-21T03:39:05.422931+0000","last_became_peered":"2024-09-21T03:39:05.422931+0000","last_unstale":"2024-09-21T03:39:06.570597+0000","last_undegraded":"2024-09-21T03:39:06.570597+0000","last_fullsized":"2024-09-21T03:39:06.570597+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:35:18.019438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.570103+0000","last_change":"2024-09-21T03:39:04.405877+0000","last_active":"2024-09-21T03:39:06.570103+0000","last_peered":"2024-09-21T03:39:06.570103+0000","last_clean":"2024-09-21T03:39:06.570103+0000","last_became_active":"2024-09-21T03:39:04.405583+0000","last_became_peered":"2024-09-21T03:39:04.405583+0000","last_unstale":"2024-09-21T03:39:06.570103+0000","last_undegraded":"2024-09-21T03:39:06.570103+0000","last_fullsized":"2024-09-21T03:39:06.570103+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_clean_scrub_stamp":"2024-09-21T03:39:03.377315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:03:56.352603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-09-21T03:39:04.402886+0000","last_change":"2024-09-21T03:39:04.402886+0000","last_active":"2024-09-21T03:39:04.383211+0000","last_peered":"2024-09-21T03:39:04.383211+0000","last_clean":"2024-09-21T03:39:04.383211+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:39:04.402886+0000","last_undegraded":"2024-09-21T03:39:04.402886+0000","last_fullsized":"2024-09-21T03:39:04.402886+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.10","version":"81'8","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.413938+0000","last_change":"2024-09-21T03:39:04.411109+0000","last_active":"2024-09-21T03:39:06.413938+0000","last_peered":"2024-09-21T03:39:06.413938+0000","last_clean":"2024-09-21T03:39:06.413938+0000","last_became_active":"2024-09-21T03:39:04.410744+0000","last_became_peered":"2024-09-21T03:39:04.410744+0000","last_unstale":"2024-09-21T03:39:06.413938+0000","last_undegraded":"2024-09-21T03:39:06.413938+0000","last_fullsized":"2024-09-21T03:39:06.413938+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_clean_scrub_stamp":"2024-09-21T03:39:03.377315+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:13:48.637151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.570028+0000","last_change":"2024-09-21T03:39:05.432432+0000","last_active":"2024-09-21T03:39:06.570028+0000","last_peered":"2024-09-21T03:39:06.570028+0000","last_clean":"2024-09-21T03:39:06.570028+0000","last_became_active":"2024-09-21T03:39:05.432136+0000","last_became_peered":"2024-09-21T03:39:05.432136+0000","last_unstale":"2024-09-21T03:39:06.570028+0000","last_undegraded":"2024-09-21T03:39:06.570028+0000","last_fullsized":"2024-09-21T03:39:06.570028+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:10:20.019199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.f","version":"81'1","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.570192+0000","last_change":"2024-09-21T03:39:04.403666+0000","last_active":"2024-09-21T03:39:06.570192+0000","last_peered":"2024-09-21T03:39:06.570192+0000","last_clean":"2024-09-21T03:39:06.570192+0000","last_became_active":"2024-09-21T03:39:04.403310+0000","last_became_peered":"2024-09-21T03:39:04.403310+0000","last_unstale":"2024-09-21T03:39:06.570192+0000","last_undegraded":"2024-09-21T03:39:06.570192+0000","last_fullsized":"2024-09-21T03:39:06.570192+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_clean_scrub_stamp":"2024-09-21T03:39:03.377315+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:18:35.050119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.418952+0000","last_change":"2024-09-21T03:39:05.432320+0000","last_active":"2024-09-21T03:39:06.418952+0000","last_peered":"2024-09-21T03:39:06.418952+0000","last_clean":"2024-09-21T03:39:06.418952+0000","last_became_active":"2024-09-21T03:39:05.431965+0000","last_became_peered":"2024-09-21T03:39:05.431965+0000","last_unstale":"2024-09-21T03:39:06.418952+0000","last_undegraded":"2024-09-21T03:39:06.418952+0000","last_fullsized":"2024-09-21T03:39:06.418952+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:51:42.048958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.e","version":"81'2","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.569527+0000","last_change":"2024-09-21T03:39:04.403354+0000","last_active":"2024-09-21T03:39:06.569527+0000","last_peered":"2024-09-21T03:39:06.569527+0000","last_clean":"2024-09-21T03:39:06.569527+0000","last_became_active":"2024-09-21T03:39:04.401729+0000","last_became_peered":"2024-09-21T03:39:04.401729+0000","last_unstale":"2024-09-21T03:39:06.569527+0000","last_undegraded":"2024-09-21T03:39:06.569527+0000","last_fullsized":"2024-09-21T03:39:06.569527+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_clean_scrub_stamp":"2024-09-21T03:39:03.377315+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:05:16.494460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.414672+0000","last_change":"2024-09-21T03:39:05.421453+0000","last_active":"2024-09-21T03:39:06.414672+0000","last_peered":"2024-09-21T03:39:06.414672+0000","last_clean":"2024-09-21T03:39:06.414672+0000","last_became_active":"2024-09-21T03:39:05.420867+0000","last_became_peered":"2024-09-21T03:39:05.420867+0000","last_unstale":"2024-09-21T03:39:06.414672+0000","last_undegraded":"2024-09-21T03:39:06.414672+0000","last_fullsized":"2024-09-21T03:39:06.414672+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:47:58.257466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.d","version":"81'1","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.570103+0000","last_change":"2024-09-21T03:39:04.408221+0000","last_active":"2024-09-21T03:39:06.570103+0000","last_peered":"2024-09-21T03:39:06.570103+0000","last_clean":"2024-09-21T03:39:06.570103+0000","last_became_active":"2024-09-21T03:39:04.407900+0000","last_became_peered":"2024-09-21T03:39:04.407900+0000","last_unstale":"2024-09-21T03:39:06.570103+0000","last_undegraded":"2024-09-21T03:39:06.570103+0000","last_fullsized":"2024-09-21T03:39:06.570103+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_clean_scrub_stamp":"2024-09-21T03:39:03.377315+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:26:11.862449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.570468+0000","last_change":"2024-09-21T03:39:05.423205+0000","last_active":"2024-09-21T03:39:06.570468+0000","last_peered":"2024-09-21T03:39:06.570468+0000","last_clean":"2024-09-21T03:39:06.570468+0000","last_became_active":"2024-09-21T03:39:05.422867+0000","last_became_peered":"2024-09-21T03:39:05.422867+0000","last_unstale":"2024-09-21T03:39:06.570468+0000","last_undegraded":"2024-09-21T03:39:06.570468+0000","last_fullsized":"2024-09-21T03:39:06.570468+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:33:29.949291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.570257+0000","last_change":"2024-09-21T03:39:04.401682+0000","last_active":"2024-09-21T03:39:06.570257+0000","last_peered":"2024-09-21T03:39:06.570257+0000","last_clean":"2024-09-21T03:39:06.570257+0000","last_became_active":"2024-09-21T03:39:04.401354+0000","last_became_peered":"2024-09-21T03:39:04.401354+0000","last_unstale":"2024-09-21T03:39:06.570257+0000","last_undegraded":"2024-09-21T03:39:06.570257+0000","last_fullsized":"2024-09-21T03:39:06.570257+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_clean_scrub_stamp":"2024-09-21T03:39:03.377315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:02:56.975258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.570149+0000","last_change":"2024-09-21T03:39:05.424271+0000","last_active":"2024-09-21T03:39:06.570149+0000","last_peered":"2024-09-21T03:39:06.570149+0000","last_clean":"2024-09-21T03:39:06.570149+0000","last_became_active":"2024-09-21T03:39:05.423967+0000","last_became_peered":"2024-09-21T03:39:05.423967+0000","last_unstale":"2024-09-21T03:39:06.570149+0000","last_undegraded":"2024-09-21T03:39:06.570149+0000","last_fullsized":"2024-09-21T03:39:06.570149+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:29:22.009760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"81'2","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.418688+0000","last_change":"2024-09-21T03:39:04.406956+0000","last_active":"2024-09-21T03:39:06.418688+0000","last_peered":"2024-09-21T03:39:06.418688+0000","last_clean":"2024-09-21T03:39:06.418688+0000","last_became_active":"2024-09-21T03:39:04.406604+0000","last_became_peered":"2024-09-21T03:39:04.406604+0000","last_unstale":"2024-09-21T03:39:06.418688+0000","last_undegraded":"2024-09-21T03:39:06.418688+0000","last_fullsized":"2024-09-21T03:39:06.418688+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_clean_scrub_stamp":"2024-09-21T03:39:03.377315+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:56:14.520557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.570451+0000","last_change":"2024-09-21T03:39:05.424012+0000","last_active":"2024-09-21T03:39:06.570451+0000","last_peered":"2024-09-21T03:39:06.570451+0000","last_clean":"2024-09-21T03:39:06.570451+0000","last_became_active":"2024-09-21T03:39:05.423713+0000","last_became_peered":"2024-09-21T03:39:05.423713+0000","last_unstale":"2024-09-21T03:39:06.570451+0000","last_undegraded":"2024-09-21T03:39:06.570451+0000","last_fullsized":"2024-09-21T03:39:06.570451+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:30:06.424731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.413940+0000","last_change":"2024-09-21T03:39:04.395490+0000","last_active":"2024-09-21T03:39:06.413940+0000","last_peered":"2024-09-21T03:39:06.413940+0000","last_clean":"2024-09-21T03:39:06.413940+0000","last_became_active":"2024-09-21T03:39:04.394934+0000","last_became_peered":"2024-09-21T03:39:04.394934+0000","last_unstale":"2024-09-21T03:39:06.413940+0000","last_undegraded":"2024-09-21T03:39:06.413940+0000","last_fullsized":"2024-09-21T03:39:06.413940+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_clean_scrub_stamp":"2024-09-21T03:39:03.377315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:30:01.896910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.569691+0000","last_change":"2024-09-21T03:39:05.418742+0000","last_active":"2024-09-21T03:39:06.569691+0000","last_peered":"2024-09-21T03:39:06.569691+0000","last_clean":"2024-09-21T03:39:06.569691+0000","last_became_active":"2024-09-21T03:39:05.418366+0000","last_became_peered":"2024-09-21T03:39:05.418366+0000","last_unstale":"2024-09-21T03:39:06.569691+0000","last_undegraded":"2024-09-21T03:39:06.569691+0000","last_fullsized":"2024-09-21T03:39:06.569691+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:09:46.888780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.418986+0000","last_change":"2024-09-21T03:39:04.408050+0000","last_active":"2024-09-21T03:39:06.418986+0000","last_peered":"2024-09-21T03:39:06.418986+0000","last_clean":"2024-09-21T03:39:06.418986+0000","last_became_active":"2024-09-21T03:39:04.407753+0000","last_became_peered":"2024-09-21T03:39:04.407753+0000","last_unstale":"2024-09-21T03:39:06.418986+0000","last_undegraded":"2024-09-21T03:39:06.418986+0000","last_fullsized":"2024-09-21T03:39:06.418986+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_clean_scrub_stamp":"2024-09-21T03:39:03.377315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:05:16.494460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.569516+0000","last_change":"2024-09-21T03:39:05.431029+0000","last_active":"2024-09-21T03:39:06.569516+0000","last_peered":"2024-09-21T03:39:06.569516+0000","last_clean":"2024-09-21T03:39:06.569516+0000","last_became_active":"2024-09-21T03:39:05.427562+0000","last_became_peered":"2024-09-21T03:39:05.427562+0000","last_unstale":"2024-09-21T03:39:06.569516+0000","last_undegraded":"2024-09-21T03:39:06.569516+0000","last_fullsized":"2024-09-21T03:39:06.569516+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:20:35.883679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T03:39:04.414902+0000","last_change":"2024-09-21T03:39:04.407689+0000","last_active":"2024-09-21T03:39:04.414902+0000","last_peered":"2024-09-21T03:39:04.414902+0000","last_clean":"2024-09-21T03:39:04.414902+0000","last_became_active":"2024-09-21T03:39:04.407356+0000","last_became_peered":"2024-09-21T03:39:04.407356+0000","last_unstale":"2024-09-21T03:39:04.414902+0000","last_undegraded":"2024-09-21T03:39:04.414902+0000","last_fullsized":"2024-09-21T03:39:04.414902+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_clean_scrub_stamp":"2024-09-21T03:39:03.377315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:25:15.786223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-09-21T03:39:04.400780+0000","last_change":"2024-09-21T03:39:04.400780+0000","last_active":"2024-09-21T03:39:04.383211+0000","last_peered":"2024-09-21T03:39:04.383211+0000","last_clean":"2024-09-21T03:39:04.383211+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:39:04.400780+0000","last_undegraded":"2024-09-21T03:39:04.400780+0000","last_fullsized":"2024-09-21T03:39:04.400780+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.6","version":"82'1","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.417795+0000","last_change":"2024-09-21T03:39:04.411532+0000","last_active":"2024-09-21T03:39:06.417795+0000","last_peered":"2024-09-21T03:39:06.417795+0000","last_clean":"2024-09-21T03:39:06.417795+0000","last_became_active":"2024-09-21T03:39:04.411189+0000","last_became_peered":"2024-09-21T03:39:04.411189+0000","last_unstale":"2024-09-21T03:39:06.417795+0000","last_undegraded":"2024-09-21T03:39:06.417795+0000","last_fullsized":"2024-09-21T03:39:06.417795+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_clean_scrub_stamp":"2024-09-21T03:39:03.377315+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:31:32.273309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.569653+0000","last_change":"2024-09-21T03:39:05.414196+0000","last_active":"2024-09-21T03:39:06.569653+0000","last_peered":"2024-09-21T03:39:06.569653+0000","last_clean":"2024-09-21T03:39:06.569653+0000","last_became_active":"2024-09-21T03:39:05.413871+0000","last_became_peered":"2024-09-21T03:39:05.413871+0000","last_unstale":"2024-09-21T03:39:06.569653+0000","last_undegraded":"2024-09-21T03:39:06.569653+0000","last_fullsized":"2024-09-21T03:39:06.569653+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:27:34.908887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T03:39:04.413543+0000","last_change":"2024-09-21T03:39:04.405352+0000","last_active":"2024-09-21T03:39:04.413543+0000","last_peered":"2024-09-21T03:39:04.413543+0000","last_clean":"2024-09-21T03:39:04.413543+0000","last_became_active":"2024-09-21T03:39:04.405020+0000","last_became_peered":"2024-09-21T03:39:04.405020+0000","last_unstale":"2024-09-21T03:39:04.413543+0000","last_undegraded":"2024-09-21T03:39:04.413543+0000","last_fullsized":"2024-09-21T03:39:04.413543+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_clean_scrub_stamp":"2024-09-21T03:39:03.377315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:37:33.420266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.415196+0000","last_change":"2024-09-21T03:39:05.415347+0000","last_active":"2024-09-21T03:39:06.415196+0000","last_peered":"2024-09-21T03:39:06.415196+0000","last_clean":"2024-09-21T03:39:06.415196+0000","last_became_active":"2024-09-21T03:39:05.414476+0000","last_became_peered":"2024-09-21T03:39:05.414476+0000","last_unstale":"2024-09-21T03:39:06.415196+0000","last_undegraded":"2024-09-21T03:39:06.415196+0000","last_fullsized":"2024-09-21T03:39:06.415196+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:49:45.640520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.569666+0000","last_change":"2024-09-21T03:39:04.410318+0000","last_active":"2024-09-21T03:39:06.569666+0000","last_peered":"2024-09-21T03:39:06.569666+0000","last_clean":"2024-09-21T03:39:06.569666+0000","last_became_active":"2024-09-21T03:39:04.409989+0000","last_became_peered":"2024-09-21T03:39:04.409989+0000","last_unstale":"2024-09-21T03:39:06.569666+0000","last_undegraded":"2024-09-21T03:39:06.569666+0000","last_fullsized":"2024-09-21T03:39:06.569666+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_clean_scrub_stamp":"2024-09-21T03:39:03.377315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:49:27.896288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.570622+0000","last_change":"2024-09-21T03:39:05.421291+0000","last_active":"2024-09-21T03:39:06.570622+0000","last_peered":"2024-09-21T03:39:06.570622+0000","last_clean":"2024-09-21T03:39:06.570622+0000","last_became_active":"2024-09-21T03:39:05.420933+0000","last_became_peered":"2024-09-21T03:39:05.420933+0000","last_unstale":"2024-09-21T03:39:06.570622+0000","last_undegraded":"2024-09-21T03:39:06.570622+0000","last_fullsized":"2024-09-21T03:39:06.570622+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:18:54.425316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.569375+0000","last_change":"2024-09-21T03:39:04.405531+0000","last_active":"2024-09-21T03:39:06.569375+0000","last_peered":"2024-09-21T03:39:06.569375+0000","last_clean":"2024-09-21T03:39:06.569375+0000","last_became_active":"2024-09-21T03:39:04.405193+0000","last_became_peered":"2024-09-21T03:39:04.405193+0000","last_unstale":"2024-09-21T03:39:06.569375+0000","last_undegraded":"2024-09-21T03:39:06.569375+0000","last_fullsized":"2024-09-21T03:39:06.569375+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_clean_scrub_stamp":"2024-09-21T03:39:03.377315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:41:35.755494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.569966+0000","last_change":"2024-09-21T03:39:05.431541+0000","last_active":"2024-09-21T03:39:06.569966+0000","last_peered":"2024-09-21T03:39:06.569966+0000","last_clean":"2024-09-21T03:39:06.569966+0000","last_became_active":"2024-09-21T03:39:05.428130+0000","last_became_peered":"2024-09-21T03:39:05.428130+0000","last_unstale":"2024-09-21T03:39:06.569966+0000","last_undegraded":"2024-09-21T03:39:06.569966+0000","last_fullsized":"2024-09-21T03:39:06.569966+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:22:26.967805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.415539+0000","last_change":"2024-09-21T03:39:04.403462+0000","last_active":"2024-09-21T03:39:06.415539+0000","last_peered":"2024-09-21T03:39:06.415539+0000","last_clean":"2024-09-21T03:39:06.415539+0000","last_became_active":"2024-09-21T03:39:04.403114+0000","last_became_peered":"2024-09-21T03:39:04.403114+0000","last_unstale":"2024-09-21T03:39:06.415539+0000","last_undegraded":"2024-09-21T03:39:06.415539+0000","last_fullsized":"2024-09-21T03:39:06.415539+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_clean_scrub_stamp":"2024-09-21T03:39:03.377315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:38:07.041511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.419363+0000","last_change":"2024-09-21T03:39:05.429296+0000","last_active":"2024-09-21T03:39:06.419363+0000","last_peered":"2024-09-21T03:39:06.419363+0000","last_clean":"2024-09-21T03:39:06.419363+0000","last_became_active":"2024-09-21T03:39:05.426008+0000","last_became_peered":"2024-09-21T03:39:05.426008+0000","last_unstale":"2024-09-21T03:39:06.419363+0000","last_undegraded":"2024-09-21T03:39:06.419363+0000","last_fullsized":"2024-09-21T03:39:06.419363+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:16:42.786975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.568997+0000","last_change":"2024-09-21T03:39:04.407078+0000","last_active":"2024-09-21T03:39:06.568997+0000","last_peered":"2024-09-21T03:39:06.568997+0000","last_clean":"2024-09-21T03:39:06.568997+0000","last_became_active":"2024-09-21T03:39:04.406757+0000","last_became_peered":"2024-09-21T03:39:04.406757+0000","last_unstale":"2024-09-21T03:39:06.568997+0000","last_undegraded":"2024-09-21T03:39:06.568997+0000","last_fullsized":"2024-09-21T03:39:06.568997+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_clean_scrub_stamp":"2024-09-21T03:39:03.377315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:08:19.857227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.570315+0000","last_change":"2024-09-21T03:39:05.431328+0000","last_active":"2024-09-21T03:39:06.570315+0000","last_peered":"2024-09-21T03:39:06.570315+0000","last_clean":"2024-09-21T03:39:06.570315+0000","last_became_active":"2024-09-21T03:39:05.431007+0000","last_became_peered":"2024-09-21T03:39:05.431007+0000","last_unstale":"2024-09-21T03:39:06.570315+0000","last_undegraded":"2024-09-21T03:39:06.570315+0000","last_fullsized":"2024-09-21T03:39:06.570315+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:09:46.155413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.419086+0000","last_change":"2024-09-21T03:39:04.412555+0000","last_active":"2024-09-21T03:39:06.419086+0000","last_peered":"2024-09-21T03:39:06.419086+0000","last_clean":"2024-09-21T03:39:06.419086+0000","last_became_active":"2024-09-21T03:39:04.412258+0000","last_became_peered":"2024-09-21T03:39:04.412258+0000","last_unstale":"2024-09-21T03:39:06.419086+0000","last_undegraded":"2024-09-21T03:39:06.419086+0000","last_fullsized":"2024-09-21T03:39:06.419086+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_clean_scrub_stamp":"2024-09-21T03:39:03.377315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:01:13.191529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.570331+0000","last_change":"2024-09-21T03:39:05.418570+0000","last_active":"2024-09-21T03:39:06.570331+0000","last_peered":"2024-09-21T03:39:06.570331+0000","last_clean":"2024-09-21T03:39:06.570331+0000","last_became_active":"2024-09-21T03:39:05.418257+0000","last_became_peered":"2024-09-21T03:39:05.418257+0000","last_unstale":"2024-09-21T03:39:06.570331+0000","last_undegraded":"2024-09-21T03:39:06.570331+0000","last_fullsized":"2024-09-21T03:39:06.570331+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:38:35.279761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.415650+0000","last_change":"2024-09-21T03:39:04.414590+0000","last_active":"2024-09-21T03:39:06.415650+0000","last_peered":"2024-09-21T03:39:06.415650+0000","last_clean":"2024-09-21T03:39:06.415650+0000","last_became_active":"2024-09-21T03:39:04.414355+0000","last_became_peered":"2024-09-21T03:39:04.414355+0000","last_unstale":"2024-09-21T03:39:06.415650+0000","last_undegraded":"2024-09-21T03:39:06.415650+0000","last_fullsized":"2024-09-21T03:39:06.415650+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_clean_scrub_stamp":"2024-09-21T03:39:03.377315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:03:09.175309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":223,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-21T03:39:04.391409+0000","last_change":"2024-09-21T03:33:25.319776+0000","last_active":"2024-09-21T03:39:04.391409+0000","last_peered":"2024-09-21T03:39:04.391409+0000","last_clean":"2024-09-21T03:39:04.391409+0000","last_became_active":"2024-09-21T03:33:25.318399+0000","last_became_peered":"2024-09-21T03:33:25.318399+0000","last_unstale":"2024-09-21T03:39:04.391409+0000","last_undegraded":"2024-09-21T03:39:04.391409+0000","last_fullsized":"2024-09-21T03:39:04.391409+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:23.016672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:23.016672+0000","last_clean_scrub_stamp":"2024-09-21T03:33:23.016672+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:32:35.906114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":2376,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":36,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":33,"ondisk_log_size":33,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":1,"up_from":16,"seq":68719476808,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32720,"kb_used_data":1528,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339120,"statfs":{"total":96636764160,"available":96603258880,"internally_reserved":0,"allocated":1564672,"data_stored":1036315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509513,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36328,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335512,"statfs":{"total":96636764160,"available":96599564288,"internally_reserved":0,"allocated":1003520,"data_stored":460458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":36174283},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509512,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32184,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339656,"statfs":{"total":96636764160,"available":96603807744,"internally_reserved":0,"allocated":1015808,"data_stored":461522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509513,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32164,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339676,"statfs":{"total":96636764160,"available":96603828224,"internally_reserved":0,"allocated":995328,"data_stored":459806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509512,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32168,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339672,"statfs":{"total":96636764160,"available":96603824128,"internally_reserved":0,"allocated":999424,"data_stored":459828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542216,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36332,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335508,"statfs":{"total":96636764160,"available":96599560192,"internally_reserved":0,"allocated":1003520,"data_stored":460478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574920,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36912,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334928,"statfs":{"total":96636764160,"available":96598966272,"internally_reserved":0,"allocated":1593344,"data_stored":1050264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542217,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36328,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335512,"statfs":{"total":96636764160,"available":96599564288,"internally_reserved":0,"allocated":999424,"data_stored":459828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T03:39:14.009 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T03:39:14.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:14.139+0000 7f20986ec640 1 -- 172.21.15.38:0/3215661353 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f207c004800 msgr2=0x7f207c004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:14.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:14.139+0000 7f20986ec640 1 --2- 172.21.15.38:0/3215661353 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f207c004800 0x7f207c004c10 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f208000c310 tx=0x7f207c005830 comp rx=0 tx=0).stop 2024-09-21T03:39:14.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:14.139+0000 7f20986ec640 1 -- 172.21.15.38:0/3215661353 shutdown_connections 2024-09-21T03:39:14.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:14.139+0000 7f20986ec640 1 --2- 172.21.15.38:0/3215661353 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f207c004800 0x7f207c004c10 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:14.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:14.139+0000 7f20986ec640 1 -- 172.21.15.38:0/3215661353 >> 172.21.15.38:0/3215661353 conn(0x7f20900d4440 msgr2=0x7f20900d4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:14.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:14.139+0000 7f20986ec640 1 -- 172.21.15.38:0/3215661353 shutdown_connections 2024-09-21T03:39:14.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:14.139+0000 7f20986ec640 1 -- 172.21.15.38:0/3215661353 wait complete. 2024-09-21T03:39:14.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:14.141+0000 7f20986ec640 1 Processor -- start 2024-09-21T03:39:14.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:14.141+0000 7f20986ec640 1 -- start start 2024-09-21T03:39:14.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:14.143+0000 7f20986ec640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f207c004800 0x7f20900b2510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:14.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:14.143+0000 7f20986ec640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f20900b2a50 0x7f20900c7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:14.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:14.143+0000 7f20986ec640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f20900c8520 0x7f209014e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:14.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:14.143+0000 7f20986ec640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f2090162e90 con 0x7f20900b2a50 2024-09-21T03:39:14.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:14.143+0000 7f20986ec640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f2090162d10 con 0x7f20900c8520 2024-09-21T03:39:14.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:14.143+0000 7f20986ec640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f20900d2a20 con 0x7f207c004800 2024-09-21T03:39:14.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:14.143+0000 7f2095c87640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f20900b2a50 0x7f20900c7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:14.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:14.143+0000 7f2095c87640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f20900b2a50 0x7f20900c7fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43602/0 (socket says 172.21.15.38:43602) 2024-09-21T03:39:14.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:14.143+0000 7f2095c87640 1 -- 172.21.15.38:0/2512662264 learned_addr learned my addr 172.21.15.38:0/2512662264 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:39:14.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:14.143+0000 7f2096488640 1 --2- 172.21.15.38:0/2512662264 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f207c004800 0x7f20900b2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:14.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:14.143+0000 7f2096c89640 1 --2- 172.21.15.38:0/2512662264 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f20900c8520 0x7f209014e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:14.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:14.143+0000 7f2095c87640 1 -- 172.21.15.38:0/2512662264 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f207c004800 msgr2=0x7f20900b2510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:14.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:14.143+0000 7f2095c87640 1 --2- 172.21.15.38:0/2512662264 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f207c004800 0x7f20900b2510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:14.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:14.143+0000 7f2095c87640 1 -- 172.21.15.38:0/2512662264 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f20900c8520 msgr2=0x7f209014e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:14.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:14.143+0000 7f2095c87640 1 --2- 172.21.15.38:0/2512662264 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f20900c8520 0x7f209014e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:14.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:14.143+0000 7f2095c87640 1 -- 172.21.15.38:0/2512662264 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20900c8d50 con 0x7f20900b2a50 2024-09-21T03:39:14.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:14.143+0000 7f2095c87640 1 --2- 172.21.15.38:0/2512662264 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f20900b2a50 0x7f20900c7fe0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f208002b160 tx=0x7f2080004bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:14.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:14.144+0000 7f2086ffd640 1 -- 172.21.15.38:0/2512662264 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2080031040 con 0x7f20900b2a50 2024-09-21T03:39:14.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:14.144+0000 7f2086ffd640 1 -- 172.21.15.38:0/2512662264 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2080006420 con 0x7f20900b2a50 2024-09-21T03:39:14.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:14.144+0000 7f20986ec640 1 -- 172.21.15.38:0/2512662264 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20900c9030 con 0x7f20900b2a50 2024-09-21T03:39:14.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:14.144+0000 7f2086ffd640 1 -- 172.21.15.38:0/2512662264 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2080006c70 con 0x7f20900b2a50 2024-09-21T03:39:14.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:14.144+0000 7f20986ec640 1 -- 172.21.15.38:0/2512662264 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20900c95f0 con 0x7f20900b2a50 2024-09-21T03:39:14.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:14.145+0000 7f20986ec640 1 -- 172.21.15.38:0/2512662264 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20900c8ac0 con 0x7f20900b2a50 2024-09-21T03:39:14.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:14.145+0000 7f2086ffd640 1 -- 172.21.15.38:0/2512662264 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f2080034070 con 0x7f20900b2a50 2024-09-21T03:39:14.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:14.146+0000 7f2086ffd640 1 --2- 172.21.15.38:0/2512662264 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2074078b90 0x7f207407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:14.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:14.146+0000 7f2086ffd640 1 -- 172.21.15.38:0/2512662264 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 6101+0+0 (secure 0 0 0) 0x7f20800bb0c0 con 0x7f20900b2a50 2024-09-21T03:39:14.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:14.146+0000 7f2096488640 1 --2- 172.21.15.38:0/2512662264 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2074078b90 0x7f207407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:14.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:14.147+0000 7f2096488640 1 --2- 172.21.15.38:0/2512662264 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2074078b90 0x7f207407b0a0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f208c009e50 tx=0x7f208c005f90 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:14.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:14.150+0000 7f2086ffd640 1 -- 172.21.15.38:0/2512662264 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f2080082d20 con 0x7f20900b2a50 2024-09-21T03:39:14.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:14.333+0000 7f20986ec640 1 -- 172.21.15.38:0/2512662264 --> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f20900d2f80 con 0x7f2074078b90 2024-09-21T03:39:14.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:14.338+0000 7f2086ffd640 1 -- 172.21.15.38:0/2512662264 <== mgr.4117 v2:172.21.15.195:6800/1631901344 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252027 (secure 0 0 0) 0x7f20900d2f80 con 0x7f2074078b90 2024-09-21T03:39:14.343 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:39:14.345 INFO:teuthology.orchestra.run.smithi038.stderr:dumped all 2024-09-21T03:39:14.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:14.342+0000 7f20986ec640 1 -- 172.21.15.38:0/2512662264 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2074078b90 msgr2=0x7f207407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:14.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:14.342+0000 7f20986ec640 1 --2- 172.21.15.38:0/2512662264 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2074078b90 0x7f207407b0a0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f208c009e50 tx=0x7f208c005f90 comp rx=0 tx=0).stop 2024-09-21T03:39:14.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:14.342+0000 7f20986ec640 1 -- 172.21.15.38:0/2512662264 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f20900b2a50 msgr2=0x7f20900c7fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:14.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:14.342+0000 7f20986ec640 1 --2- 172.21.15.38:0/2512662264 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f20900b2a50 0x7f20900c7fe0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f208002b160 tx=0x7f2080004bc0 comp rx=0 tx=0).stop 2024-09-21T03:39:14.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:14.342+0000 7f20986ec640 1 -- 172.21.15.38:0/2512662264 shutdown_connections 2024-09-21T03:39:14.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:14.342+0000 7f20986ec640 1 --2- 172.21.15.38:0/2512662264 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f20900c8520 0x7f209014e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:14.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:14.342+0000 7f20986ec640 1 --2- 172.21.15.38:0/2512662264 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f20900b2a50 0x7f20900c7fe0 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:14.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:14.342+0000 7f20986ec640 1 --2- 172.21.15.38:0/2512662264 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2074078b90 0x7f207407b0a0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:14.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:14.342+0000 7f20986ec640 1 --2- 172.21.15.38:0/2512662264 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f207c004800 0x7f20900b2510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:14.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:14.342+0000 7f20986ec640 1 -- 172.21.15.38:0/2512662264 >> 172.21.15.38:0/2512662264 conn(0x7f20900d4440 msgr2=0x7f209015e4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:14.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:14.342+0000 7f20986ec640 1 -- 172.21.15.38:0/2512662264 shutdown_connections 2024-09-21T03:39:14.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:14.342+0000 7f20986ec640 1 -- 172.21.15.38:0/2512662264 wait complete. 2024-09-21T03:39:14.356 INFO:teuthology.orchestra.run.smithi038.stdout:{"pg_ready":true,"pg_map":{"version":250,"stamp":"2024-09-21T03:39:13.072996+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":275172,"kb_used_data":9000,"kb_used_omap":12,"kb_used_meta":265971,"kb_avail":754699548,"statfs":{"total":773094113280,"available":772812337152,"internally_reserved":0,"allocated":9216000,"data_stored":4864004,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12748,"internal_metadata":272354868},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002131"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.415455+0000","last_change":"2024-09-21T03:39:05.423256+0000","last_active":"2024-09-21T03:39:06.415455+0000","last_peered":"2024-09-21T03:39:06.415455+0000","last_clean":"2024-09-21T03:39:06.415455+0000","last_became_active":"2024-09-21T03:39:05.422248+0000","last_became_peered":"2024-09-21T03:39:05.422248+0000","last_unstale":"2024-09-21T03:39:06.415455+0000","last_undegraded":"2024-09-21T03:39:06.415455+0000","last_fullsized":"2024-09-21T03:39:06.415455+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:28:09.468861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.419248+0000","last_change":"2024-09-21T03:39:05.431615+0000","last_active":"2024-09-21T03:39:06.419248+0000","last_peered":"2024-09-21T03:39:06.419248+0000","last_clean":"2024-09-21T03:39:06.419248+0000","last_became_active":"2024-09-21T03:39:05.429700+0000","last_became_peered":"2024-09-21T03:39:05.429700+0000","last_unstale":"2024-09-21T03:39:06.419248+0000","last_undegraded":"2024-09-21T03:39:06.419248+0000","last_fullsized":"2024-09-21T03:39:06.419248+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:22:26.967805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.419361+0000","last_change":"2024-09-21T03:39:05.434451+0000","last_active":"2024-09-21T03:39:06.419361+0000","last_peered":"2024-09-21T03:39:06.419361+0000","last_clean":"2024-09-21T03:39:06.419361+0000","last_became_active":"2024-09-21T03:39:05.432952+0000","last_became_peered":"2024-09-21T03:39:05.432952+0000","last_unstale":"2024-09-21T03:39:06.419361+0000","last_undegraded":"2024-09-21T03:39:06.419361+0000","last_fullsized":"2024-09-21T03:39:06.419361+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:30:48.915097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.569594+0000","last_change":"2024-09-21T03:39:05.426032+0000","last_active":"2024-09-21T03:39:06.569594+0000","last_peered":"2024-09-21T03:39:06.569594+0000","last_clean":"2024-09-21T03:39:06.569594+0000","last_became_active":"2024-09-21T03:39:05.425731+0000","last_became_peered":"2024-09-21T03:39:05.425731+0000","last_unstale":"2024-09-21T03:39:06.569594+0000","last_undegraded":"2024-09-21T03:39:06.569594+0000","last_fullsized":"2024-09-21T03:39:06.569594+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:01:01.380435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.570162+0000","last_change":"2024-09-21T03:39:05.428805+0000","last_active":"2024-09-21T03:39:06.570162+0000","last_peered":"2024-09-21T03:39:06.570162+0000","last_clean":"2024-09-21T03:39:06.570162+0000","last_became_active":"2024-09-21T03:39:05.428394+0000","last_became_peered":"2024-09-21T03:39:05.428394+0000","last_unstale":"2024-09-21T03:39:06.570162+0000","last_undegraded":"2024-09-21T03:39:06.570162+0000","last_fullsized":"2024-09-21T03:39:06.570162+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:57:28.504333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.415502+0000","last_change":"2024-09-21T03:39:05.418164+0000","last_active":"2024-09-21T03:39:06.415502+0000","last_peered":"2024-09-21T03:39:06.415502+0000","last_clean":"2024-09-21T03:39:06.415502+0000","last_became_active":"2024-09-21T03:39:05.416861+0000","last_became_peered":"2024-09-21T03:39:05.416861+0000","last_unstale":"2024-09-21T03:39:06.415502+0000","last_undegraded":"2024-09-21T03:39:06.415502+0000","last_fullsized":"2024-09-21T03:39:06.415502+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:06:43.522600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.569829+0000","last_change":"2024-09-21T03:39:05.429537+0000","last_active":"2024-09-21T03:39:06.569829+0000","last_peered":"2024-09-21T03:39:06.569829+0000","last_clean":"2024-09-21T03:39:06.569829+0000","last_became_active":"2024-09-21T03:39:05.428077+0000","last_became_peered":"2024-09-21T03:39:05.428077+0000","last_unstale":"2024-09-21T03:39:06.569829+0000","last_undegraded":"2024-09-21T03:39:06.569829+0000","last_fullsized":"2024-09-21T03:39:06.569829+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:38:28.792089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.414574+0000","last_change":"2024-09-21T03:39:05.420868+0000","last_active":"2024-09-21T03:39:06.414574+0000","last_peered":"2024-09-21T03:39:06.414574+0000","last_clean":"2024-09-21T03:39:06.414574+0000","last_became_active":"2024-09-21T03:39:05.420546+0000","last_became_peered":"2024-09-21T03:39:05.420546+0000","last_unstale":"2024-09-21T03:39:06.414574+0000","last_undegraded":"2024-09-21T03:39:06.414574+0000","last_fullsized":"2024-09-21T03:39:06.414574+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:00:25.925784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.570276+0000","last_change":"2024-09-21T03:39:05.424217+0000","last_active":"2024-09-21T03:39:06.570276+0000","last_peered":"2024-09-21T03:39:06.570276+0000","last_clean":"2024-09-21T03:39:06.570276+0000","last_became_active":"2024-09-21T03:39:05.423922+0000","last_became_peered":"2024-09-21T03:39:05.423922+0000","last_unstale":"2024-09-21T03:39:06.570276+0000","last_undegraded":"2024-09-21T03:39:06.570276+0000","last_fullsized":"2024-09-21T03:39:06.570276+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:30:48.915097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.415328+0000","last_change":"2024-09-21T03:39:05.423068+0000","last_active":"2024-09-21T03:39:06.415328+0000","last_peered":"2024-09-21T03:39:06.415328+0000","last_clean":"2024-09-21T03:39:06.415328+0000","last_became_active":"2024-09-21T03:39:05.422727+0000","last_became_peered":"2024-09-21T03:39:05.422727+0000","last_unstale":"2024-09-21T03:39:06.415328+0000","last_undegraded":"2024-09-21T03:39:06.415328+0000","last_fullsized":"2024-09-21T03:39:06.415328+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:23:30.576192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.419460+0000","last_change":"2024-09-21T03:39:05.427718+0000","last_active":"2024-09-21T03:39:06.419460+0000","last_peered":"2024-09-21T03:39:06.419460+0000","last_clean":"2024-09-21T03:39:06.419460+0000","last_became_active":"2024-09-21T03:39:05.427387+0000","last_became_peered":"2024-09-21T03:39:05.427387+0000","last_unstale":"2024-09-21T03:39:06.419460+0000","last_undegraded":"2024-09-21T03:39:06.419460+0000","last_fullsized":"2024-09-21T03:39:06.419460+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:58:09.229781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.570387+0000","last_change":"2024-09-21T03:39:05.421453+0000","last_active":"2024-09-21T03:39:06.570387+0000","last_peered":"2024-09-21T03:39:06.570387+0000","last_clean":"2024-09-21T03:39:06.570387+0000","last_became_active":"2024-09-21T03:39:05.421133+0000","last_became_peered":"2024-09-21T03:39:05.421133+0000","last_unstale":"2024-09-21T03:39:06.570387+0000","last_undegraded":"2024-09-21T03:39:06.570387+0000","last_fullsized":"2024-09-21T03:39:06.570387+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:30:07.399497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.418721+0000","last_change":"2024-09-21T03:39:05.426022+0000","last_active":"2024-09-21T03:39:06.418721+0000","last_peered":"2024-09-21T03:39:06.418721+0000","last_clean":"2024-09-21T03:39:06.418721+0000","last_became_active":"2024-09-21T03:39:05.425625+0000","last_became_peered":"2024-09-21T03:39:05.425625+0000","last_unstale":"2024-09-21T03:39:06.418721+0000","last_undegraded":"2024-09-21T03:39:06.418721+0000","last_fullsized":"2024-09-21T03:39:06.418721+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:03:54.230793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.414472+0000","last_change":"2024-09-21T03:39:05.421310+0000","last_active":"2024-09-21T03:39:06.414472+0000","last_peered":"2024-09-21T03:39:06.414472+0000","last_clean":"2024-09-21T03:39:06.414472+0000","last_became_active":"2024-09-21T03:39:05.420314+0000","last_became_peered":"2024-09-21T03:39:05.420314+0000","last_unstale":"2024-09-21T03:39:06.414472+0000","last_undegraded":"2024-09-21T03:39:06.414472+0000","last_fullsized":"2024-09-21T03:39:06.414472+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:00:07.867021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.570061+0000","last_change":"2024-09-21T03:39:05.427360+0000","last_active":"2024-09-21T03:39:06.570061+0000","last_peered":"2024-09-21T03:39:06.570061+0000","last_clean":"2024-09-21T03:39:06.570061+0000","last_became_active":"2024-09-21T03:39:05.427009+0000","last_became_peered":"2024-09-21T03:39:05.427009+0000","last_unstale":"2024-09-21T03:39:06.570061+0000","last_undegraded":"2024-09-21T03:39:06.570061+0000","last_fullsized":"2024-09-21T03:39:06.570061+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:54:27.965337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.569046+0000","last_change":"2024-09-21T03:39:05.430747+0000","last_active":"2024-09-21T03:39:06.569046+0000","last_peered":"2024-09-21T03:39:06.569046+0000","last_clean":"2024-09-21T03:39:06.569046+0000","last_became_active":"2024-09-21T03:39:05.429872+0000","last_became_peered":"2024-09-21T03:39:05.429872+0000","last_unstale":"2024-09-21T03:39:06.569046+0000","last_undegraded":"2024-09-21T03:39:06.569046+0000","last_fullsized":"2024-09-21T03:39:06.569046+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:01:18.107849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.415027+0000","last_change":"2024-09-21T03:39:05.418812+0000","last_active":"2024-09-21T03:39:06.415027+0000","last_peered":"2024-09-21T03:39:06.415027+0000","last_clean":"2024-09-21T03:39:06.415027+0000","last_became_active":"2024-09-21T03:39:05.418487+0000","last_became_peered":"2024-09-21T03:39:05.418487+0000","last_unstale":"2024-09-21T03:39:06.415027+0000","last_undegraded":"2024-09-21T03:39:06.415027+0000","last_fullsized":"2024-09-21T03:39:06.415027+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:37:29.087437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.419123+0000","last_change":"2024-09-21T03:39:05.428238+0000","last_active":"2024-09-21T03:39:06.419123+0000","last_peered":"2024-09-21T03:39:06.419123+0000","last_clean":"2024-09-21T03:39:06.419123+0000","last_became_active":"2024-09-21T03:39:05.427915+0000","last_became_peered":"2024-09-21T03:39:05.427915+0000","last_unstale":"2024-09-21T03:39:06.419123+0000","last_undegraded":"2024-09-21T03:39:06.419123+0000","last_fullsized":"2024-09-21T03:39:06.419123+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:22:48.194041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.414304+0000","last_change":"2024-09-21T03:39:05.412642+0000","last_active":"2024-09-21T03:39:06.414304+0000","last_peered":"2024-09-21T03:39:06.414304+0000","last_clean":"2024-09-21T03:39:06.414304+0000","last_became_active":"2024-09-21T03:39:05.412274+0000","last_became_peered":"2024-09-21T03:39:05.412274+0000","last_unstale":"2024-09-21T03:39:06.414304+0000","last_undegraded":"2024-09-21T03:39:06.414304+0000","last_fullsized":"2024-09-21T03:39:06.414304+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:24:30.153279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.415023+0000","last_change":"2024-09-21T03:39:05.428953+0000","last_active":"2024-09-21T03:39:06.415023+0000","last_peered":"2024-09-21T03:39:06.415023+0000","last_clean":"2024-09-21T03:39:06.415023+0000","last_became_active":"2024-09-21T03:39:05.428223+0000","last_became_peered":"2024-09-21T03:39:05.428223+0000","last_unstale":"2024-09-21T03:39:06.415023+0000","last_undegraded":"2024-09-21T03:39:06.415023+0000","last_fullsized":"2024-09-21T03:39:06.415023+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:16:38.764525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.570578+0000","last_change":"2024-09-21T03:39:05.409645+0000","last_active":"2024-09-21T03:39:06.570578+0000","last_peered":"2024-09-21T03:39:06.570578+0000","last_clean":"2024-09-21T03:39:06.570578+0000","last_became_active":"2024-09-21T03:39:05.409315+0000","last_became_peered":"2024-09-21T03:39:05.409315+0000","last_unstale":"2024-09-21T03:39:06.570578+0000","last_undegraded":"2024-09-21T03:39:06.570578+0000","last_fullsized":"2024-09-21T03:39:06.570578+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:49:12.163503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.418882+0000","last_change":"2024-09-21T03:39:05.427679+0000","last_active":"2024-09-21T03:39:06.418882+0000","last_peered":"2024-09-21T03:39:06.418882+0000","last_clean":"2024-09-21T03:39:06.418882+0000","last_became_active":"2024-09-21T03:39:05.426044+0000","last_became_peered":"2024-09-21T03:39:05.426044+0000","last_unstale":"2024-09-21T03:39:06.418882+0000","last_undegraded":"2024-09-21T03:39:06.418882+0000","last_fullsized":"2024-09-21T03:39:06.418882+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:20:35.883679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.414865+0000","last_change":"2024-09-21T03:39:05.426047+0000","last_active":"2024-09-21T03:39:06.414865+0000","last_peered":"2024-09-21T03:39:06.414865+0000","last_clean":"2024-09-21T03:39:06.414865+0000","last_became_active":"2024-09-21T03:39:05.425717+0000","last_became_peered":"2024-09-21T03:39:05.425717+0000","last_unstale":"2024-09-21T03:39:06.414865+0000","last_undegraded":"2024-09-21T03:39:06.414865+0000","last_fullsized":"2024-09-21T03:39:06.414865+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:36:02.976589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.418484+0000","last_change":"2024-09-21T03:39:05.427956+0000","last_active":"2024-09-21T03:39:06.418484+0000","last_peered":"2024-09-21T03:39:06.418484+0000","last_clean":"2024-09-21T03:39:06.418484+0000","last_became_active":"2024-09-21T03:39:05.425708+0000","last_became_peered":"2024-09-21T03:39:05.425708+0000","last_unstale":"2024-09-21T03:39:06.418484+0000","last_undegraded":"2024-09-21T03:39:06.418484+0000","last_fullsized":"2024-09-21T03:39:06.418484+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:14:22.373008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.415419+0000","last_change":"2024-09-21T03:39:05.420911+0000","last_active":"2024-09-21T03:39:06.415419+0000","last_peered":"2024-09-21T03:39:06.415419+0000","last_clean":"2024-09-21T03:39:06.415419+0000","last_became_active":"2024-09-21T03:39:05.420587+0000","last_became_peered":"2024-09-21T03:39:05.420587+0000","last_unstale":"2024-09-21T03:39:06.415419+0000","last_undegraded":"2024-09-21T03:39:06.415419+0000","last_fullsized":"2024-09-21T03:39:06.415419+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:37:32.783221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.569302+0000","last_change":"2024-09-21T03:39:05.426555+0000","last_active":"2024-09-21T03:39:06.569302+0000","last_peered":"2024-09-21T03:39:06.569302+0000","last_clean":"2024-09-21T03:39:06.569302+0000","last_became_active":"2024-09-21T03:39:05.426265+0000","last_became_peered":"2024-09-21T03:39:05.426265+0000","last_unstale":"2024-09-21T03:39:06.569302+0000","last_undegraded":"2024-09-21T03:39:06.569302+0000","last_fullsized":"2024-09-21T03:39:06.569302+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:03:54.230793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.569945+0000","last_change":"2024-09-21T03:39:05.427771+0000","last_active":"2024-09-21T03:39:06.569945+0000","last_peered":"2024-09-21T03:39:06.569945+0000","last_clean":"2024-09-21T03:39:06.569945+0000","last_became_active":"2024-09-21T03:39:05.427310+0000","last_became_peered":"2024-09-21T03:39:05.427310+0000","last_unstale":"2024-09-21T03:39:06.569945+0000","last_undegraded":"2024-09-21T03:39:06.569945+0000","last_fullsized":"2024-09-21T03:39:06.569945+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:22:48.194041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.570546+0000","last_change":"2024-09-21T03:39:05.431170+0000","last_active":"2024-09-21T03:39:06.570546+0000","last_peered":"2024-09-21T03:39:06.570546+0000","last_clean":"2024-09-21T03:39:06.570546+0000","last_became_active":"2024-09-21T03:39:05.430855+0000","last_became_peered":"2024-09-21T03:39:05.430855+0000","last_unstale":"2024-09-21T03:39:06.570546+0000","last_undegraded":"2024-09-21T03:39:06.570546+0000","last_fullsized":"2024-09-21T03:39:06.570546+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:13:27.854189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.414268+0000","last_change":"2024-09-21T03:39:05.429739+0000","last_active":"2024-09-21T03:39:06.414268+0000","last_peered":"2024-09-21T03:39:06.414268+0000","last_clean":"2024-09-21T03:39:06.414268+0000","last_became_active":"2024-09-21T03:39:05.429401+0000","last_became_peered":"2024-09-21T03:39:05.429401+0000","last_unstale":"2024-09-21T03:39:06.414268+0000","last_undegraded":"2024-09-21T03:39:06.414268+0000","last_fullsized":"2024-09-21T03:39:06.414268+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:29:58.646678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.414926+0000","last_change":"2024-09-21T03:39:05.420835+0000","last_active":"2024-09-21T03:39:06.414926+0000","last_peered":"2024-09-21T03:39:06.414926+0000","last_clean":"2024-09-21T03:39:06.414926+0000","last_became_active":"2024-09-21T03:39:05.420498+0000","last_became_peered":"2024-09-21T03:39:05.420498+0000","last_unstale":"2024-09-21T03:39:06.414926+0000","last_undegraded":"2024-09-21T03:39:06.414926+0000","last_fullsized":"2024-09-21T03:39:06.414926+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:01:05.591656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.418906+0000","last_change":"2024-09-21T03:39:05.430619+0000","last_active":"2024-09-21T03:39:06.418906+0000","last_peered":"2024-09-21T03:39:06.418906+0000","last_clean":"2024-09-21T03:39:06.418906+0000","last_became_active":"2024-09-21T03:39:05.429959+0000","last_became_peered":"2024-09-21T03:39:05.429959+0000","last_unstale":"2024-09-21T03:39:06.418906+0000","last_undegraded":"2024-09-21T03:39:06.418906+0000","last_fullsized":"2024-09-21T03:39:06.418906+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:26:04.045027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.569279+0000","last_change":"2024-09-21T03:39:05.422391+0000","last_active":"2024-09-21T03:39:06.569279+0000","last_peered":"2024-09-21T03:39:06.569279+0000","last_clean":"2024-09-21T03:39:06.569279+0000","last_became_active":"2024-09-21T03:39:05.422073+0000","last_became_peered":"2024-09-21T03:39:05.422073+0000","last_unstale":"2024-09-21T03:39:06.569279+0000","last_undegraded":"2024-09-21T03:39:06.569279+0000","last_fullsized":"2024-09-21T03:39:06.569279+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:14:22.373008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.570338+0000","last_change":"2024-09-21T03:39:05.430675+0000","last_active":"2024-09-21T03:39:06.570338+0000","last_peered":"2024-09-21T03:39:06.570338+0000","last_clean":"2024-09-21T03:39:06.570338+0000","last_became_active":"2024-09-21T03:39:05.430203+0000","last_became_peered":"2024-09-21T03:39:05.430203+0000","last_unstale":"2024-09-21T03:39:06.570338+0000","last_undegraded":"2024-09-21T03:39:06.570338+0000","last_fullsized":"2024-09-21T03:39:06.570338+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:27:11.780682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"81'4","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.570214+0000","last_change":"2024-09-21T03:39:04.406626+0000","last_active":"2024-09-21T03:39:06.570214+0000","last_peered":"2024-09-21T03:39:06.570214+0000","last_clean":"2024-09-21T03:39:06.570214+0000","last_became_active":"2024-09-21T03:39:04.406318+0000","last_became_peered":"2024-09-21T03:39:04.406318+0000","last_unstale":"2024-09-21T03:39:06.570214+0000","last_undegraded":"2024-09-21T03:39:06.570214+0000","last_fullsized":"2024-09-21T03:39:06.570214+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_clean_scrub_stamp":"2024-09-21T03:39:03.377315+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:00:13.859613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.569891+0000","last_change":"2024-09-21T03:39:05.428305+0000","last_active":"2024-09-21T03:39:06.569891+0000","last_peered":"2024-09-21T03:39:06.569891+0000","last_clean":"2024-09-21T03:39:06.569891+0000","last_became_active":"2024-09-21T03:39:05.427992+0000","last_became_peered":"2024-09-21T03:39:05.427992+0000","last_unstale":"2024-09-21T03:39:06.569891+0000","last_undegraded":"2024-09-21T03:39:06.569891+0000","last_fullsized":"2024-09-21T03:39:06.569891+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:15:23.391245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1e","version":"81'1","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.570008+0000","last_change":"2024-09-21T03:39:04.411255+0000","last_active":"2024-09-21T03:39:06.570008+0000","last_peered":"2024-09-21T03:39:06.570008+0000","last_clean":"2024-09-21T03:39:06.570008+0000","last_became_active":"2024-09-21T03:39:04.410962+0000","last_became_peered":"2024-09-21T03:39:04.410962+0000","last_unstale":"2024-09-21T03:39:06.570008+0000","last_undegraded":"2024-09-21T03:39:06.570008+0000","last_fullsized":"2024-09-21T03:39:06.570008+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_clean_scrub_stamp":"2024-09-21T03:39:03.377315+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:27:36.848833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.569439+0000","last_change":"2024-09-21T03:39:05.415000+0000","last_active":"2024-09-21T03:39:06.569439+0000","last_peered":"2024-09-21T03:39:06.569439+0000","last_clean":"2024-09-21T03:39:06.569439+0000","last_became_active":"2024-09-21T03:39:05.414695+0000","last_became_peered":"2024-09-21T03:39:05.414695+0000","last_unstale":"2024-09-21T03:39:06.569439+0000","last_undegraded":"2024-09-21T03:39:06.569439+0000","last_fullsized":"2024-09-21T03:39:06.569439+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:58:09.229781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1d","version":"81'2","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.415216+0000","last_change":"2024-09-21T03:39:04.404636+0000","last_active":"2024-09-21T03:39:06.415216+0000","last_peered":"2024-09-21T03:39:06.415216+0000","last_clean":"2024-09-21T03:39:06.415216+0000","last_became_active":"2024-09-21T03:39:04.404270+0000","last_became_peered":"2024-09-21T03:39:04.404270+0000","last_unstale":"2024-09-21T03:39:06.415216+0000","last_undegraded":"2024-09-21T03:39:06.415216+0000","last_fullsized":"2024-09-21T03:39:06.415216+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_clean_scrub_stamp":"2024-09-21T03:39:03.377315+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:54:26.959441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.569256+0000","last_change":"2024-09-21T03:39:05.425119+0000","last_active":"2024-09-21T03:39:06.569256+0000","last_peered":"2024-09-21T03:39:06.569256+0000","last_clean":"2024-09-21T03:39:06.569256+0000","last_became_active":"2024-09-21T03:39:05.424804+0000","last_became_peered":"2024-09-21T03:39:05.424804+0000","last_unstale":"2024-09-21T03:39:06.569256+0000","last_undegraded":"2024-09-21T03:39:06.569256+0000","last_fullsized":"2024-09-21T03:39:06.569256+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:13:42.059728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"81'2","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.570641+0000","last_change":"2024-09-21T03:39:04.407381+0000","last_active":"2024-09-21T03:39:06.570641+0000","last_peered":"2024-09-21T03:39:06.570641+0000","last_clean":"2024-09-21T03:39:06.570641+0000","last_became_active":"2024-09-21T03:39:04.407044+0000","last_became_peered":"2024-09-21T03:39:04.407044+0000","last_unstale":"2024-09-21T03:39:06.570641+0000","last_undegraded":"2024-09-21T03:39:06.570641+0000","last_fullsized":"2024-09-21T03:39:06.570641+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_clean_scrub_stamp":"2024-09-21T03:39:03.377315+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:14:21.367112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.415145+0000","last_change":"2024-09-21T03:39:05.425026+0000","last_active":"2024-09-21T03:39:06.415145+0000","last_peered":"2024-09-21T03:39:06.415145+0000","last_clean":"2024-09-21T03:39:06.415145+0000","last_became_active":"2024-09-21T03:39:05.424667+0000","last_became_peered":"2024-09-21T03:39:05.424667+0000","last_unstale":"2024-09-21T03:39:06.415145+0000","last_undegraded":"2024-09-21T03:39:06.415145+0000","last_fullsized":"2024-09-21T03:39:06.415145+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:46:01.041621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.1b","version":"81'1","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.570542+0000","last_change":"2024-09-21T03:39:04.414608+0000","last_active":"2024-09-21T03:39:06.570542+0000","last_peered":"2024-09-21T03:39:06.570542+0000","last_clean":"2024-09-21T03:39:06.570542+0000","last_became_active":"2024-09-21T03:39:04.414274+0000","last_became_peered":"2024-09-21T03:39:04.414274+0000","last_unstale":"2024-09-21T03:39:06.570542+0000","last_undegraded":"2024-09-21T03:39:06.570542+0000","last_fullsized":"2024-09-21T03:39:06.570542+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_clean_scrub_stamp":"2024-09-21T03:39:03.377315+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:13:41.053832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.419607+0000","last_change":"2024-09-21T03:39:05.434557+0000","last_active":"2024-09-21T03:39:06.419607+0000","last_peered":"2024-09-21T03:39:06.419607+0000","last_clean":"2024-09-21T03:39:06.419607+0000","last_became_active":"2024-09-21T03:39:05.430126+0000","last_became_peered":"2024-09-21T03:39:05.430126+0000","last_unstale":"2024-09-21T03:39:06.419607+0000","last_undegraded":"2024-09-21T03:39:06.419607+0000","last_fullsized":"2024-09-21T03:39:06.419607+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:27:34.908887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"81'1","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.570010+0000","last_change":"2024-09-21T03:39:04.397543+0000","last_active":"2024-09-21T03:39:06.570010+0000","last_peered":"2024-09-21T03:39:06.570010+0000","last_clean":"2024-09-21T03:39:06.570010+0000","last_became_active":"2024-09-21T03:39:04.397220+0000","last_became_peered":"2024-09-21T03:39:04.397220+0000","last_unstale":"2024-09-21T03:39:06.570010+0000","last_undegraded":"2024-09-21T03:39:06.570010+0000","last_fullsized":"2024-09-21T03:39:06.570010+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_clean_scrub_stamp":"2024-09-21T03:39:03.377315+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:21:47.012786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.570317+0000","last_change":"2024-09-21T03:39:05.413837+0000","last_active":"2024-09-21T03:39:06.570317+0000","last_peered":"2024-09-21T03:39:06.570317+0000","last_clean":"2024-09-21T03:39:06.570317+0000","last_became_active":"2024-09-21T03:39:05.413498+0000","last_became_peered":"2024-09-21T03:39:05.413498+0000","last_unstale":"2024-09-21T03:39:06.570317+0000","last_undegraded":"2024-09-21T03:39:06.570317+0000","last_fullsized":"2024-09-21T03:39:06.570317+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:33:39.426980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.413804+0000","last_change":"2024-09-21T03:39:04.408211+0000","last_active":"2024-09-21T03:39:06.413804+0000","last_peered":"2024-09-21T03:39:06.413804+0000","last_clean":"2024-09-21T03:39:06.413804+0000","last_became_active":"2024-09-21T03:39:04.404927+0000","last_became_peered":"2024-09-21T03:39:04.404927+0000","last_unstale":"2024-09-21T03:39:06.413804+0000","last_undegraded":"2024-09-21T03:39:06.413804+0000","last_fullsized":"2024-09-21T03:39:06.413804+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_clean_scrub_stamp":"2024-09-21T03:39:03.377315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:53:29.829387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.569462+0000","last_change":"2024-09-21T03:39:05.419988+0000","last_active":"2024-09-21T03:39:06.569462+0000","last_peered":"2024-09-21T03:39:06.569462+0000","last_clean":"2024-09-21T03:39:06.569462+0000","last_became_active":"2024-09-21T03:39:05.419693+0000","last_became_peered":"2024-09-21T03:39:05.419693+0000","last_unstale":"2024-09-21T03:39:06.569462+0000","last_undegraded":"2024-09-21T03:39:06.569462+0000","last_fullsized":"2024-09-21T03:39:06.569462+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:08:48.101749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.418856+0000","last_change":"2024-09-21T03:39:04.410302+0000","last_active":"2024-09-21T03:39:06.418856+0000","last_peered":"2024-09-21T03:39:06.418856+0000","last_clean":"2024-09-21T03:39:06.418856+0000","last_became_active":"2024-09-21T03:39:04.409959+0000","last_became_peered":"2024-09-21T03:39:04.409959+0000","last_unstale":"2024-09-21T03:39:06.418856+0000","last_undegraded":"2024-09-21T03:39:06.418856+0000","last_fullsized":"2024-09-21T03:39:06.418856+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_clean_scrub_stamp":"2024-09-21T03:39:03.377315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:18:35.050119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.419019+0000","last_change":"2024-09-21T03:39:05.429795+0000","last_active":"2024-09-21T03:39:06.419019+0000","last_peered":"2024-09-21T03:39:06.419019+0000","last_clean":"2024-09-21T03:39:06.419019+0000","last_became_active":"2024-09-21T03:39:05.426181+0000","last_became_peered":"2024-09-21T03:39:05.426181+0000","last_unstale":"2024-09-21T03:39:06.419019+0000","last_undegraded":"2024-09-21T03:39:06.419019+0000","last_fullsized":"2024-09-21T03:39:06.419019+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:38:28.792089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.570075+0000","last_change":"2024-09-21T03:39:04.410453+0000","last_active":"2024-09-21T03:39:06.570075+0000","last_peered":"2024-09-21T03:39:06.570075+0000","last_clean":"2024-09-21T03:39:06.570075+0000","last_became_active":"2024-09-21T03:39:04.410082+0000","last_became_peered":"2024-09-21T03:39:04.410082+0000","last_unstale":"2024-09-21T03:39:06.570075+0000","last_undegraded":"2024-09-21T03:39:06.570075+0000","last_fullsized":"2024-09-21T03:39:06.570075+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_clean_scrub_stamp":"2024-09-21T03:39:03.377315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:56:14.520557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.569202+0000","last_change":"2024-09-21T03:39:05.422217+0000","last_active":"2024-09-21T03:39:06.569202+0000","last_peered":"2024-09-21T03:39:06.569202+0000","last_clean":"2024-09-21T03:39:06.569202+0000","last_became_active":"2024-09-21T03:39:05.421890+0000","last_became_peered":"2024-09-21T03:39:05.421890+0000","last_unstale":"2024-09-21T03:39:06.569202+0000","last_undegraded":"2024-09-21T03:39:06.569202+0000","last_fullsized":"2024-09-21T03:39:06.569202+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:40:45.946118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.415360+0000","last_change":"2024-09-21T03:39:04.411385+0000","last_active":"2024-09-21T03:39:06.415360+0000","last_peered":"2024-09-21T03:39:06.415360+0000","last_clean":"2024-09-21T03:39:06.415360+0000","last_became_active":"2024-09-21T03:39:04.410963+0000","last_became_peered":"2024-09-21T03:39:04.410963+0000","last_unstale":"2024-09-21T03:39:06.415360+0000","last_undegraded":"2024-09-21T03:39:06.415360+0000","last_fullsized":"2024-09-21T03:39:06.415360+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_clean_scrub_stamp":"2024-09-21T03:39:03.377315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:13:50.076197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.414821+0000","last_change":"2024-09-21T03:39:05.419517+0000","last_active":"2024-09-21T03:39:06.414821+0000","last_peered":"2024-09-21T03:39:06.414821+0000","last_clean":"2024-09-21T03:39:06.414821+0000","last_became_active":"2024-09-21T03:39:05.418599+0000","last_became_peered":"2024-09-21T03:39:05.418599+0000","last_unstale":"2024-09-21T03:39:06.414821+0000","last_undegraded":"2024-09-21T03:39:06.414821+0000","last_fullsized":"2024-09-21T03:39:06.414821+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:59:45.194584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.15","version":"81'3","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.413935+0000","last_change":"2024-09-21T03:39:04.407595+0000","last_active":"2024-09-21T03:39:06.413935+0000","last_peered":"2024-09-21T03:39:06.413935+0000","last_clean":"2024-09-21T03:39:06.413935+0000","last_became_active":"2024-09-21T03:39:04.407278+0000","last_became_peered":"2024-09-21T03:39:04.407278+0000","last_unstale":"2024-09-21T03:39:06.413935+0000","last_undegraded":"2024-09-21T03:39:06.413935+0000","last_fullsized":"2024-09-21T03:39:06.413935+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_clean_scrub_stamp":"2024-09-21T03:39:03.377315+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:41:01.617219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.414516+0000","last_change":"2024-09-21T03:39:05.419678+0000","last_active":"2024-09-21T03:39:06.414516+0000","last_peered":"2024-09-21T03:39:06.414516+0000","last_clean":"2024-09-21T03:39:06.414516+0000","last_became_active":"2024-09-21T03:39:05.419119+0000","last_became_peered":"2024-09-21T03:39:05.419119+0000","last_unstale":"2024-09-21T03:39:06.414516+0000","last_undegraded":"2024-09-21T03:39:06.414516+0000","last_fullsized":"2024-09-21T03:39:06.414516+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:03:30.086684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.14","version":"81'6","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.569939+0000","last_change":"2024-09-21T03:39:04.396680+0000","last_active":"2024-09-21T03:39:06.569939+0000","last_peered":"2024-09-21T03:39:06.569939+0000","last_clean":"2024-09-21T03:39:06.569939+0000","last_became_active":"2024-09-21T03:39:04.396344+0000","last_became_peered":"2024-09-21T03:39:04.396344+0000","last_unstale":"2024-09-21T03:39:06.569939+0000","last_undegraded":"2024-09-21T03:39:06.569939+0000","last_fullsized":"2024-09-21T03:39:06.569939+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_clean_scrub_stamp":"2024-09-21T03:39:03.377315+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:12:41.701014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.569672+0000","last_change":"2024-09-21T03:39:05.423246+0000","last_active":"2024-09-21T03:39:06.569672+0000","last_peered":"2024-09-21T03:39:06.569672+0000","last_clean":"2024-09-21T03:39:06.569672+0000","last_became_active":"2024-09-21T03:39:05.422924+0000","last_became_peered":"2024-09-21T03:39:05.422924+0000","last_unstale":"2024-09-21T03:39:06.569672+0000","last_undegraded":"2024-09-21T03:39:06.569672+0000","last_fullsized":"2024-09-21T03:39:06.569672+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:51:42.048958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.13","version":"81'2","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.415005+0000","last_change":"2024-09-21T03:39:04.402594+0000","last_active":"2024-09-21T03:39:06.415005+0000","last_peered":"2024-09-21T03:39:06.415005+0000","last_clean":"2024-09-21T03:39:06.415005+0000","last_became_active":"2024-09-21T03:39:04.402242+0000","last_became_peered":"2024-09-21T03:39:04.402242+0000","last_unstale":"2024-09-21T03:39:06.415005+0000","last_undegraded":"2024-09-21T03:39:06.415005+0000","last_fullsized":"2024-09-21T03:39:06.415005+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_clean_scrub_stamp":"2024-09-21T03:39:03.377315+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:04:37.656163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.415215+0000","last_change":"2024-09-21T03:39:05.423609+0000","last_active":"2024-09-21T03:39:06.415215+0000","last_peered":"2024-09-21T03:39:06.415215+0000","last_clean":"2024-09-21T03:39:06.415215+0000","last_became_active":"2024-09-21T03:39:05.423278+0000","last_became_peered":"2024-09-21T03:39:05.423278+0000","last_unstale":"2024-09-21T03:39:06.415215+0000","last_undegraded":"2024-09-21T03:39:06.415215+0000","last_fullsized":"2024-09-21T03:39:06.415215+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:50:38.477525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.569717+0000","last_change":"2024-09-21T03:39:04.409288+0000","last_active":"2024-09-21T03:39:06.569717+0000","last_peered":"2024-09-21T03:39:06.569717+0000","last_clean":"2024-09-21T03:39:06.569717+0000","last_became_active":"2024-09-21T03:39:04.408965+0000","last_became_peered":"2024-09-21T03:39:04.408965+0000","last_unstale":"2024-09-21T03:39:06.569717+0000","last_undegraded":"2024-09-21T03:39:06.569717+0000","last_fullsized":"2024-09-21T03:39:06.569717+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_clean_scrub_stamp":"2024-09-21T03:39:03.377315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:01:17.101953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.570597+0000","last_change":"2024-09-21T03:39:05.423231+0000","last_active":"2024-09-21T03:39:06.570597+0000","last_peered":"2024-09-21T03:39:06.570597+0000","last_clean":"2024-09-21T03:39:06.570597+0000","last_became_active":"2024-09-21T03:39:05.422931+0000","last_became_peered":"2024-09-21T03:39:05.422931+0000","last_unstale":"2024-09-21T03:39:06.570597+0000","last_undegraded":"2024-09-21T03:39:06.570597+0000","last_fullsized":"2024-09-21T03:39:06.570597+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:35:18.019438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.570103+0000","last_change":"2024-09-21T03:39:04.405877+0000","last_active":"2024-09-21T03:39:06.570103+0000","last_peered":"2024-09-21T03:39:06.570103+0000","last_clean":"2024-09-21T03:39:06.570103+0000","last_became_active":"2024-09-21T03:39:04.405583+0000","last_became_peered":"2024-09-21T03:39:04.405583+0000","last_unstale":"2024-09-21T03:39:06.570103+0000","last_undegraded":"2024-09-21T03:39:06.570103+0000","last_fullsized":"2024-09-21T03:39:06.570103+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_clean_scrub_stamp":"2024-09-21T03:39:03.377315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:03:56.352603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.415315+0000","last_change":"2024-09-21T03:39:05.419775+0000","last_active":"2024-09-21T03:39:06.415315+0000","last_peered":"2024-09-21T03:39:06.415315+0000","last_clean":"2024-09-21T03:39:06.415315+0000","last_became_active":"2024-09-21T03:39:05.419450+0000","last_became_peered":"2024-09-21T03:39:05.419450+0000","last_unstale":"2024-09-21T03:39:06.415315+0000","last_undegraded":"2024-09-21T03:39:06.415315+0000","last_fullsized":"2024-09-21T03:39:06.415315+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:50:53.609486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.10","version":"81'8","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.413938+0000","last_change":"2024-09-21T03:39:04.411109+0000","last_active":"2024-09-21T03:39:06.413938+0000","last_peered":"2024-09-21T03:39:06.413938+0000","last_clean":"2024-09-21T03:39:06.413938+0000","last_became_active":"2024-09-21T03:39:04.410744+0000","last_became_peered":"2024-09-21T03:39:04.410744+0000","last_unstale":"2024-09-21T03:39:06.413938+0000","last_undegraded":"2024-09-21T03:39:06.413938+0000","last_fullsized":"2024-09-21T03:39:06.413938+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_clean_scrub_stamp":"2024-09-21T03:39:03.377315+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:13:48.637151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.570028+0000","last_change":"2024-09-21T03:39:05.432432+0000","last_active":"2024-09-21T03:39:06.570028+0000","last_peered":"2024-09-21T03:39:06.570028+0000","last_clean":"2024-09-21T03:39:06.570028+0000","last_became_active":"2024-09-21T03:39:05.432136+0000","last_became_peered":"2024-09-21T03:39:05.432136+0000","last_unstale":"2024-09-21T03:39:06.570028+0000","last_undegraded":"2024-09-21T03:39:06.570028+0000","last_fullsized":"2024-09-21T03:39:06.570028+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:10:20.019199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.f","version":"81'1","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.570192+0000","last_change":"2024-09-21T03:39:04.403666+0000","last_active":"2024-09-21T03:39:06.570192+0000","last_peered":"2024-09-21T03:39:06.570192+0000","last_clean":"2024-09-21T03:39:06.570192+0000","last_became_active":"2024-09-21T03:39:04.403310+0000","last_became_peered":"2024-09-21T03:39:04.403310+0000","last_unstale":"2024-09-21T03:39:06.570192+0000","last_undegraded":"2024-09-21T03:39:06.570192+0000","last_fullsized":"2024-09-21T03:39:06.570192+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_clean_scrub_stamp":"2024-09-21T03:39:03.377315+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:18:35.050119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.418952+0000","last_change":"2024-09-21T03:39:05.432320+0000","last_active":"2024-09-21T03:39:06.418952+0000","last_peered":"2024-09-21T03:39:06.418952+0000","last_clean":"2024-09-21T03:39:06.418952+0000","last_became_active":"2024-09-21T03:39:05.431965+0000","last_became_peered":"2024-09-21T03:39:05.431965+0000","last_unstale":"2024-09-21T03:39:06.418952+0000","last_undegraded":"2024-09-21T03:39:06.418952+0000","last_fullsized":"2024-09-21T03:39:06.418952+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:51:42.048958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.e","version":"81'2","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.569527+0000","last_change":"2024-09-21T03:39:04.403354+0000","last_active":"2024-09-21T03:39:06.569527+0000","last_peered":"2024-09-21T03:39:06.569527+0000","last_clean":"2024-09-21T03:39:06.569527+0000","last_became_active":"2024-09-21T03:39:04.401729+0000","last_became_peered":"2024-09-21T03:39:04.401729+0000","last_unstale":"2024-09-21T03:39:06.569527+0000","last_undegraded":"2024-09-21T03:39:06.569527+0000","last_fullsized":"2024-09-21T03:39:06.569527+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_clean_scrub_stamp":"2024-09-21T03:39:03.377315+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:05:16.494460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.414672+0000","last_change":"2024-09-21T03:39:05.421453+0000","last_active":"2024-09-21T03:39:06.414672+0000","last_peered":"2024-09-21T03:39:06.414672+0000","last_clean":"2024-09-21T03:39:06.414672+0000","last_became_active":"2024-09-21T03:39:05.420867+0000","last_became_peered":"2024-09-21T03:39:05.420867+0000","last_unstale":"2024-09-21T03:39:06.414672+0000","last_undegraded":"2024-09-21T03:39:06.414672+0000","last_fullsized":"2024-09-21T03:39:06.414672+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:47:58.257466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.d","version":"81'1","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.570103+0000","last_change":"2024-09-21T03:39:04.408221+0000","last_active":"2024-09-21T03:39:06.570103+0000","last_peered":"2024-09-21T03:39:06.570103+0000","last_clean":"2024-09-21T03:39:06.570103+0000","last_became_active":"2024-09-21T03:39:04.407900+0000","last_became_peered":"2024-09-21T03:39:04.407900+0000","last_unstale":"2024-09-21T03:39:06.570103+0000","last_undegraded":"2024-09-21T03:39:06.570103+0000","last_fullsized":"2024-09-21T03:39:06.570103+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_clean_scrub_stamp":"2024-09-21T03:39:03.377315+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:26:11.862449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.570468+0000","last_change":"2024-09-21T03:39:05.423205+0000","last_active":"2024-09-21T03:39:06.570468+0000","last_peered":"2024-09-21T03:39:06.570468+0000","last_clean":"2024-09-21T03:39:06.570468+0000","last_became_active":"2024-09-21T03:39:05.422867+0000","last_became_peered":"2024-09-21T03:39:05.422867+0000","last_unstale":"2024-09-21T03:39:06.570468+0000","last_undegraded":"2024-09-21T03:39:06.570468+0000","last_fullsized":"2024-09-21T03:39:06.570468+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:33:29.949291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.570257+0000","last_change":"2024-09-21T03:39:04.401682+0000","last_active":"2024-09-21T03:39:06.570257+0000","last_peered":"2024-09-21T03:39:06.570257+0000","last_clean":"2024-09-21T03:39:06.570257+0000","last_became_active":"2024-09-21T03:39:04.401354+0000","last_became_peered":"2024-09-21T03:39:04.401354+0000","last_unstale":"2024-09-21T03:39:06.570257+0000","last_undegraded":"2024-09-21T03:39:06.570257+0000","last_fullsized":"2024-09-21T03:39:06.570257+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_clean_scrub_stamp":"2024-09-21T03:39:03.377315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:02:56.975258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.570149+0000","last_change":"2024-09-21T03:39:05.424271+0000","last_active":"2024-09-21T03:39:06.570149+0000","last_peered":"2024-09-21T03:39:06.570149+0000","last_clean":"2024-09-21T03:39:06.570149+0000","last_became_active":"2024-09-21T03:39:05.423967+0000","last_became_peered":"2024-09-21T03:39:05.423967+0000","last_unstale":"2024-09-21T03:39:06.570149+0000","last_undegraded":"2024-09-21T03:39:06.570149+0000","last_fullsized":"2024-09-21T03:39:06.570149+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:29:22.009760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"81'2","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.418688+0000","last_change":"2024-09-21T03:39:04.406956+0000","last_active":"2024-09-21T03:39:06.418688+0000","last_peered":"2024-09-21T03:39:06.418688+0000","last_clean":"2024-09-21T03:39:06.418688+0000","last_became_active":"2024-09-21T03:39:04.406604+0000","last_became_peered":"2024-09-21T03:39:04.406604+0000","last_unstale":"2024-09-21T03:39:06.418688+0000","last_undegraded":"2024-09-21T03:39:06.418688+0000","last_fullsized":"2024-09-21T03:39:06.418688+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_clean_scrub_stamp":"2024-09-21T03:39:03.377315+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:56:14.520557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.570451+0000","last_change":"2024-09-21T03:39:05.424012+0000","last_active":"2024-09-21T03:39:06.570451+0000","last_peered":"2024-09-21T03:39:06.570451+0000","last_clean":"2024-09-21T03:39:06.570451+0000","last_became_active":"2024-09-21T03:39:05.423713+0000","last_became_peered":"2024-09-21T03:39:05.423713+0000","last_unstale":"2024-09-21T03:39:06.570451+0000","last_undegraded":"2024-09-21T03:39:06.570451+0000","last_fullsized":"2024-09-21T03:39:06.570451+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:30:06.424731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.413940+0000","last_change":"2024-09-21T03:39:04.395490+0000","last_active":"2024-09-21T03:39:06.413940+0000","last_peered":"2024-09-21T03:39:06.413940+0000","last_clean":"2024-09-21T03:39:06.413940+0000","last_became_active":"2024-09-21T03:39:04.394934+0000","last_became_peered":"2024-09-21T03:39:04.394934+0000","last_unstale":"2024-09-21T03:39:06.413940+0000","last_undegraded":"2024-09-21T03:39:06.413940+0000","last_fullsized":"2024-09-21T03:39:06.413940+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_clean_scrub_stamp":"2024-09-21T03:39:03.377315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:30:01.896910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.569691+0000","last_change":"2024-09-21T03:39:05.418742+0000","last_active":"2024-09-21T03:39:06.569691+0000","last_peered":"2024-09-21T03:39:06.569691+0000","last_clean":"2024-09-21T03:39:06.569691+0000","last_became_active":"2024-09-21T03:39:05.418366+0000","last_became_peered":"2024-09-21T03:39:05.418366+0000","last_unstale":"2024-09-21T03:39:06.569691+0000","last_undegraded":"2024-09-21T03:39:06.569691+0000","last_fullsized":"2024-09-21T03:39:06.569691+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:09:46.888780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.418986+0000","last_change":"2024-09-21T03:39:04.408050+0000","last_active":"2024-09-21T03:39:06.418986+0000","last_peered":"2024-09-21T03:39:06.418986+0000","last_clean":"2024-09-21T03:39:06.418986+0000","last_became_active":"2024-09-21T03:39:04.407753+0000","last_became_peered":"2024-09-21T03:39:04.407753+0000","last_unstale":"2024-09-21T03:39:06.418986+0000","last_undegraded":"2024-09-21T03:39:06.418986+0000","last_fullsized":"2024-09-21T03:39:06.418986+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_clean_scrub_stamp":"2024-09-21T03:39:03.377315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:05:16.494460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.569516+0000","last_change":"2024-09-21T03:39:05.431029+0000","last_active":"2024-09-21T03:39:06.569516+0000","last_peered":"2024-09-21T03:39:06.569516+0000","last_clean":"2024-09-21T03:39:06.569516+0000","last_became_active":"2024-09-21T03:39:05.427562+0000","last_became_peered":"2024-09-21T03:39:05.427562+0000","last_unstale":"2024-09-21T03:39:06.569516+0000","last_undegraded":"2024-09-21T03:39:06.569516+0000","last_fullsized":"2024-09-21T03:39:06.569516+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:20:35.883679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.7","version":"81'2","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.415504+0000","last_change":"2024-09-21T03:39:04.407689+0000","last_active":"2024-09-21T03:39:06.415504+0000","last_peered":"2024-09-21T03:39:06.415504+0000","last_clean":"2024-09-21T03:39:06.415504+0000","last_became_active":"2024-09-21T03:39:04.407356+0000","last_became_peered":"2024-09-21T03:39:04.407356+0000","last_unstale":"2024-09-21T03:39:06.415504+0000","last_undegraded":"2024-09-21T03:39:06.415504+0000","last_fullsized":"2024-09-21T03:39:06.415504+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_clean_scrub_stamp":"2024-09-21T03:39:03.377315+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:29:47.796675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.415122+0000","last_change":"2024-09-21T03:39:05.426825+0000","last_active":"2024-09-21T03:39:06.415122+0000","last_peered":"2024-09-21T03:39:06.415122+0000","last_clean":"2024-09-21T03:39:06.415122+0000","last_became_active":"2024-09-21T03:39:05.426472+0000","last_became_peered":"2024-09-21T03:39:05.426472+0000","last_unstale":"2024-09-21T03:39:06.415122+0000","last_undegraded":"2024-09-21T03:39:06.415122+0000","last_fullsized":"2024-09-21T03:39:06.415122+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:53:42.451088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.6","version":"82'1","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.417795+0000","last_change":"2024-09-21T03:39:04.411532+0000","last_active":"2024-09-21T03:39:06.417795+0000","last_peered":"2024-09-21T03:39:06.417795+0000","last_clean":"2024-09-21T03:39:06.417795+0000","last_became_active":"2024-09-21T03:39:04.411189+0000","last_became_peered":"2024-09-21T03:39:04.411189+0000","last_unstale":"2024-09-21T03:39:06.417795+0000","last_undegraded":"2024-09-21T03:39:06.417795+0000","last_fullsized":"2024-09-21T03:39:06.417795+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_clean_scrub_stamp":"2024-09-21T03:39:03.377315+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:31:32.273309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.569653+0000","last_change":"2024-09-21T03:39:05.414196+0000","last_active":"2024-09-21T03:39:06.569653+0000","last_peered":"2024-09-21T03:39:06.569653+0000","last_clean":"2024-09-21T03:39:06.569653+0000","last_became_active":"2024-09-21T03:39:05.413871+0000","last_became_peered":"2024-09-21T03:39:05.413871+0000","last_unstale":"2024-09-21T03:39:06.569653+0000","last_undegraded":"2024-09-21T03:39:06.569653+0000","last_fullsized":"2024-09-21T03:39:06.569653+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:27:34.908887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.415341+0000","last_change":"2024-09-21T03:39:04.405352+0000","last_active":"2024-09-21T03:39:06.415341+0000","last_peered":"2024-09-21T03:39:06.415341+0000","last_clean":"2024-09-21T03:39:06.415341+0000","last_became_active":"2024-09-21T03:39:04.405020+0000","last_became_peered":"2024-09-21T03:39:04.405020+0000","last_unstale":"2024-09-21T03:39:06.415341+0000","last_undegraded":"2024-09-21T03:39:06.415341+0000","last_fullsized":"2024-09-21T03:39:06.415341+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_clean_scrub_stamp":"2024-09-21T03:39:03.377315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:40:15.360637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.415196+0000","last_change":"2024-09-21T03:39:05.415347+0000","last_active":"2024-09-21T03:39:06.415196+0000","last_peered":"2024-09-21T03:39:06.415196+0000","last_clean":"2024-09-21T03:39:06.415196+0000","last_became_active":"2024-09-21T03:39:05.414476+0000","last_became_peered":"2024-09-21T03:39:05.414476+0000","last_unstale":"2024-09-21T03:39:06.415196+0000","last_undegraded":"2024-09-21T03:39:06.415196+0000","last_fullsized":"2024-09-21T03:39:06.415196+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:49:45.640520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.569666+0000","last_change":"2024-09-21T03:39:04.410318+0000","last_active":"2024-09-21T03:39:06.569666+0000","last_peered":"2024-09-21T03:39:06.569666+0000","last_clean":"2024-09-21T03:39:06.569666+0000","last_became_active":"2024-09-21T03:39:04.409989+0000","last_became_peered":"2024-09-21T03:39:04.409989+0000","last_unstale":"2024-09-21T03:39:06.569666+0000","last_undegraded":"2024-09-21T03:39:06.569666+0000","last_fullsized":"2024-09-21T03:39:06.569666+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_clean_scrub_stamp":"2024-09-21T03:39:03.377315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:49:27.896288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.570622+0000","last_change":"2024-09-21T03:39:05.421291+0000","last_active":"2024-09-21T03:39:06.570622+0000","last_peered":"2024-09-21T03:39:06.570622+0000","last_clean":"2024-09-21T03:39:06.570622+0000","last_became_active":"2024-09-21T03:39:05.420933+0000","last_became_peered":"2024-09-21T03:39:05.420933+0000","last_unstale":"2024-09-21T03:39:06.570622+0000","last_undegraded":"2024-09-21T03:39:06.570622+0000","last_fullsized":"2024-09-21T03:39:06.570622+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:18:54.425316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.569375+0000","last_change":"2024-09-21T03:39:04.405531+0000","last_active":"2024-09-21T03:39:06.569375+0000","last_peered":"2024-09-21T03:39:06.569375+0000","last_clean":"2024-09-21T03:39:06.569375+0000","last_became_active":"2024-09-21T03:39:04.405193+0000","last_became_peered":"2024-09-21T03:39:04.405193+0000","last_unstale":"2024-09-21T03:39:06.569375+0000","last_undegraded":"2024-09-21T03:39:06.569375+0000","last_fullsized":"2024-09-21T03:39:06.569375+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_clean_scrub_stamp":"2024-09-21T03:39:03.377315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:41:35.755494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.569966+0000","last_change":"2024-09-21T03:39:05.431541+0000","last_active":"2024-09-21T03:39:06.569966+0000","last_peered":"2024-09-21T03:39:06.569966+0000","last_clean":"2024-09-21T03:39:06.569966+0000","last_became_active":"2024-09-21T03:39:05.428130+0000","last_became_peered":"2024-09-21T03:39:05.428130+0000","last_unstale":"2024-09-21T03:39:06.569966+0000","last_undegraded":"2024-09-21T03:39:06.569966+0000","last_fullsized":"2024-09-21T03:39:06.569966+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:22:26.967805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.415539+0000","last_change":"2024-09-21T03:39:04.403462+0000","last_active":"2024-09-21T03:39:06.415539+0000","last_peered":"2024-09-21T03:39:06.415539+0000","last_clean":"2024-09-21T03:39:06.415539+0000","last_became_active":"2024-09-21T03:39:04.403114+0000","last_became_peered":"2024-09-21T03:39:04.403114+0000","last_unstale":"2024-09-21T03:39:06.415539+0000","last_undegraded":"2024-09-21T03:39:06.415539+0000","last_fullsized":"2024-09-21T03:39:06.415539+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_clean_scrub_stamp":"2024-09-21T03:39:03.377315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:38:07.041511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.419363+0000","last_change":"2024-09-21T03:39:05.429296+0000","last_active":"2024-09-21T03:39:06.419363+0000","last_peered":"2024-09-21T03:39:06.419363+0000","last_clean":"2024-09-21T03:39:06.419363+0000","last_became_active":"2024-09-21T03:39:05.426008+0000","last_became_peered":"2024-09-21T03:39:05.426008+0000","last_unstale":"2024-09-21T03:39:06.419363+0000","last_undegraded":"2024-09-21T03:39:06.419363+0000","last_fullsized":"2024-09-21T03:39:06.419363+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:16:42.786975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.568997+0000","last_change":"2024-09-21T03:39:04.407078+0000","last_active":"2024-09-21T03:39:06.568997+0000","last_peered":"2024-09-21T03:39:06.568997+0000","last_clean":"2024-09-21T03:39:06.568997+0000","last_became_active":"2024-09-21T03:39:04.406757+0000","last_became_peered":"2024-09-21T03:39:04.406757+0000","last_unstale":"2024-09-21T03:39:06.568997+0000","last_undegraded":"2024-09-21T03:39:06.568997+0000","last_fullsized":"2024-09-21T03:39:06.568997+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_clean_scrub_stamp":"2024-09-21T03:39:03.377315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:08:19.857227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.570315+0000","last_change":"2024-09-21T03:39:05.431328+0000","last_active":"2024-09-21T03:39:06.570315+0000","last_peered":"2024-09-21T03:39:06.570315+0000","last_clean":"2024-09-21T03:39:06.570315+0000","last_became_active":"2024-09-21T03:39:05.431007+0000","last_became_peered":"2024-09-21T03:39:05.431007+0000","last_unstale":"2024-09-21T03:39:06.570315+0000","last_undegraded":"2024-09-21T03:39:06.570315+0000","last_fullsized":"2024-09-21T03:39:06.570315+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:09:46.155413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.419086+0000","last_change":"2024-09-21T03:39:04.412555+0000","last_active":"2024-09-21T03:39:06.419086+0000","last_peered":"2024-09-21T03:39:06.419086+0000","last_clean":"2024-09-21T03:39:06.419086+0000","last_became_active":"2024-09-21T03:39:04.412258+0000","last_became_peered":"2024-09-21T03:39:04.412258+0000","last_unstale":"2024-09-21T03:39:06.419086+0000","last_undegraded":"2024-09-21T03:39:06.419086+0000","last_fullsized":"2024-09-21T03:39:06.419086+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_clean_scrub_stamp":"2024-09-21T03:39:03.377315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:01:13.191529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.570331+0000","last_change":"2024-09-21T03:39:05.418570+0000","last_active":"2024-09-21T03:39:06.570331+0000","last_peered":"2024-09-21T03:39:06.570331+0000","last_clean":"2024-09-21T03:39:06.570331+0000","last_became_active":"2024-09-21T03:39:05.418257+0000","last_became_peered":"2024-09-21T03:39:05.418257+0000","last_unstale":"2024-09-21T03:39:06.570331+0000","last_undegraded":"2024-09-21T03:39:06.570331+0000","last_fullsized":"2024-09-21T03:39:06.570331+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:04.383211+0000","last_clean_scrub_stamp":"2024-09-21T03:39:04.383211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:38:35.279761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.415650+0000","last_change":"2024-09-21T03:39:04.414590+0000","last_active":"2024-09-21T03:39:06.415650+0000","last_peered":"2024-09-21T03:39:06.415650+0000","last_clean":"2024-09-21T03:39:06.415650+0000","last_became_active":"2024-09-21T03:39:04.414355+0000","last_became_peered":"2024-09-21T03:39:04.414355+0000","last_unstale":"2024-09-21T03:39:06.415650+0000","last_undegraded":"2024-09-21T03:39:06.415650+0000","last_fullsized":"2024-09-21T03:39:06.415650+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:03.377315+0000","last_clean_scrub_stamp":"2024-09-21T03:39:03.377315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:03:09.175309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":227,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:39:06.414755+0000","last_change":"2024-09-21T03:33:25.319776+0000","last_active":"2024-09-21T03:39:06.414755+0000","last_peered":"2024-09-21T03:39:06.414755+0000","last_clean":"2024-09-21T03:39:06.414755+0000","last_became_active":"2024-09-21T03:33:25.318399+0000","last_became_peered":"2024-09-21T03:33:25.318399+0000","last_unstale":"2024-09-21T03:39:06.414755+0000","last_undegraded":"2024-09-21T03:39:06.414755+0000","last_fullsized":"2024-09-21T03:39:06.414755+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:23.016672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:23.016672+0000","last_clean_scrub_stamp":"2024-09-21T03:33:23.016672+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:32:35.906114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":1,"up_from":16,"seq":68719476809,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32756,"kb_used_data":1568,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339084,"statfs":{"total":96636764160,"available":96603222016,"internally_reserved":0,"allocated":1605632,"data_stored":1051820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509513,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36328,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335512,"statfs":{"total":96636764160,"available":96599564288,"internally_reserved":0,"allocated":1003520,"data_stored":460458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":36174283},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509512,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32184,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339656,"statfs":{"total":96636764160,"available":96603807744,"internally_reserved":0,"allocated":1015808,"data_stored":461522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509513,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32164,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339676,"statfs":{"total":96636764160,"available":96603828224,"internally_reserved":0,"allocated":995328,"data_stored":459806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509512,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32168,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339672,"statfs":{"total":96636764160,"available":96603824128,"internally_reserved":0,"allocated":999424,"data_stored":459828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542217,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36332,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335508,"statfs":{"total":96636764160,"available":96599560192,"internally_reserved":0,"allocated":1003520,"data_stored":460478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574921,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36912,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334928,"statfs":{"total":96636764160,"available":96598966272,"internally_reserved":0,"allocated":1593344,"data_stored":1050264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542218,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36328,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335512,"statfs":{"total":96636764160,"available":96599564288,"internally_reserved":0,"allocated":999424,"data_stored":459828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T03:39:14.360 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-21T03:39:14.360 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-21T03:39:14.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:14.518+0000 7f7405856640 1 -- 172.21.15.38:0/1035410998 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7400162590 msgr2=0x7f74001629a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:14.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:14.518+0000 7f7405856640 1 --2- 172.21.15.38:0/1035410998 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7400162590 0x7f74001629a0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f740008d890 tx=0x7f73f402a560 comp rx=0 tx=0).stop 2024-09-21T03:39:14.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:14.518+0000 7f7405856640 1 -- 172.21.15.38:0/1035410998 shutdown_connections 2024-09-21T03:39:14.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:14.518+0000 7f7405856640 1 --2- 172.21.15.38:0/1035410998 >> v2:172.21.15.195:3300/0 conn(0x7f74000b2e60 0x7f74000b3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:14.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:14.518+0000 7f7405856640 1 --2- 172.21.15.38:0/1035410998 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7400162590 0x7f74001629a0 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:14.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:14.518+0000 7f7405856640 1 --2- 172.21.15.38:0/1035410998 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f74000cfe00 0x7f74000d2a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:14.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:14.518+0000 7f7405856640 1 -- 172.21.15.38:0/1035410998 >> 172.21.15.38:0/1035410998 conn(0x7f74000d4440 msgr2=0x7f74000d4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:14.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:14.518+0000 7f7405856640 1 -- 172.21.15.38:0/1035410998 shutdown_connections 2024-09-21T03:39:14.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:14.518+0000 7f7405856640 1 -- 172.21.15.38:0/1035410998 wait complete. 2024-09-21T03:39:14.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:14.520+0000 7f7405856640 1 Processor -- start 2024-09-21T03:39:14.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:14.520+0000 7f7405856640 1 -- start start 2024-09-21T03:39:14.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:14.522+0000 7f7405856640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f74000b2e60 0x7f740015c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:14.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:14.522+0000 7f7405856640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f74000cfe00 0x7f74000c7f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:14.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:14.522+0000 7f7405856640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7400162590 0x7f74001576e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:14.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:14.522+0000 7f7405856640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f74000b37b0 con 0x7f7400162590 2024-09-21T03:39:14.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:14.522+0000 7f7405856640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f74000cf5a0 con 0x7f74000cfe00 2024-09-21T03:39:14.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:14.522+0000 7f7405856640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f74000b3930 con 0x7f74000b2e60 2024-09-21T03:39:14.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:14.522+0000 7f73ff7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7400162590 0x7f74001576e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:14.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:14.522+0000 7f73ff7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7400162590 0x7f74001576e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43618/0 (socket says 172.21.15.38:43618) 2024-09-21T03:39:14.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:14.522+0000 7f73ff7fe640 1 -- 172.21.15.38:0/508653540 learned_addr learned my addr 172.21.15.38:0/508653540 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:39:14.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:14.522+0000 7f73fe7fc640 1 --2- 172.21.15.38:0/508653540 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f74000cfe00 0x7f74000c7f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:14.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:14.522+0000 7f73feffd640 1 --2- 172.21.15.38:0/508653540 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f74000b2e60 0x7f740015c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:14.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:14.522+0000 7f73ff7fe640 1 -- 172.21.15.38:0/508653540 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f74000b2e60 msgr2=0x7f740015c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:14.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:14.522+0000 7f73ff7fe640 1 --2- 172.21.15.38:0/508653540 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f74000b2e60 0x7f740015c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:14.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:14.522+0000 7f73ff7fe640 1 -- 172.21.15.38:0/508653540 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f74000cfe00 msgr2=0x7f74000c7f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:14.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:14.522+0000 7f73ff7fe640 1 --2- 172.21.15.38:0/508653540 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f74000cfe00 0x7f74000c7f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:14.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:14.522+0000 7f73ff7fe640 1 -- 172.21.15.38:0/508653540 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74000c8bd0 con 0x7f7400162590 2024-09-21T03:39:14.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:14.522+0000 7f73ff7fe640 1 --2- 172.21.15.38:0/508653540 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7400162590 0x7f74001576e0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f73f800ea00 tx=0x7f73f800eec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:14.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:14.522+0000 7f73e7fff640 1 -- 172.21.15.38:0/508653540 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73f800cde0 con 0x7f7400162590 2024-09-21T03:39:14.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:14.522+0000 7f73e7fff640 1 -- 172.21.15.38:0/508653540 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f73f8004b20 con 0x7f7400162590 2024-09-21T03:39:14.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:14.522+0000 7f7405856640 1 -- 172.21.15.38:0/508653540 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74000c8eb0 con 0x7f7400162590 2024-09-21T03:39:14.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:14.523+0000 7f73e7fff640 1 -- 172.21.15.38:0/508653540 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73f8005020 con 0x7f7400162590 2024-09-21T03:39:14.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:14.523+0000 7f7405856640 1 -- 172.21.15.38:0/508653540 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f740015b950 con 0x7f7400162590 2024-09-21T03:39:14.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:14.523+0000 7f7405856640 1 -- 172.21.15.38:0/508653540 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74000d2a20 con 0x7f7400162590 2024-09-21T03:39:14.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:14.524+0000 7f73e7fff640 1 -- 172.21.15.38:0/508653540 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f73f80051c0 con 0x7f7400162590 2024-09-21T03:39:14.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:14.525+0000 7f73e7fff640 1 --2- 172.21.15.38:0/508653540 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f73ec078b90 0x7f73ec07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:14.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:14.525+0000 7f73e7fff640 1 -- 172.21.15.38:0/508653540 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 6101+0+0 (secure 0 0 0) 0x7f73f809b9e0 con 0x7f7400162590 2024-09-21T03:39:14.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:14.525+0000 7f73feffd640 1 --2- 172.21.15.38:0/508653540 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f73ec078b90 0x7f73ec07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:14.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:14.526+0000 7f73feffd640 1 --2- 172.21.15.38:0/508653540 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f73ec078b90 0x7f73ec07b0a0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f73f0005fd0 tx=0x7f73f000b2e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:14.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:14.529+0000 7f73e7fff640 1 -- 172.21.15.38:0/508653540 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f73f8063640 con 0x7f7400162590 2024-09-21T03:39:14.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:14.790+0000 7f7405856640 1 -- 172.21.15.38:0/508653540 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f7400162bc0 con 0x7f7400162590 2024-09-21T03:39:14.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:14.793+0000 7f73e7fff640 1 -- 172.21.15.38:0/508653540 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v21) ==== 190+0+147 (secure 0 0 0) 0x7f73f8068070 con 0x7f7400162590 2024-09-21T03:39:14.796 INFO:teuthology.orchestra.run.smithi038.stdout:[client.0] 2024-09-21T03:39:14.796 INFO:teuthology.orchestra.run.smithi038.stdout: key = AQBQPu5m6ydBDxAAlWZ2XzQTPBcYPriPGRJVeA== 2024-09-21T03:39:14.796 INFO:teuthology.orchestra.run.smithi038.stdout: caps mds = "allow" 2024-09-21T03:39:14.796 INFO:teuthology.orchestra.run.smithi038.stdout: caps mon = "allow r" 2024-09-21T03:39:14.796 INFO:teuthology.orchestra.run.smithi038.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-21T03:39:14.796 INFO:teuthology.orchestra.run.smithi038.stderr:updated caps for client.0 2024-09-21T03:39:14.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:14.795+0000 7f7405856640 1 -- 172.21.15.38:0/508653540 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f73ec078b90 msgr2=0x7f73ec07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:14.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:14.795+0000 7f7405856640 1 --2- 172.21.15.38:0/508653540 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f73ec078b90 0x7f73ec07b0a0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f73f0005fd0 tx=0x7f73f000b2e0 comp rx=0 tx=0).stop 2024-09-21T03:39:14.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:14.795+0000 7f7405856640 1 -- 172.21.15.38:0/508653540 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7400162590 msgr2=0x7f74001576e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:14.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:14.795+0000 7f7405856640 1 --2- 172.21.15.38:0/508653540 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7400162590 0x7f74001576e0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f73f800ea00 tx=0x7f73f800eec0 comp rx=0 tx=0).stop 2024-09-21T03:39:14.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:14.795+0000 7f7405856640 1 -- 172.21.15.38:0/508653540 shutdown_connections 2024-09-21T03:39:14.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:14.795+0000 7f7405856640 1 --2- 172.21.15.38:0/508653540 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7400162590 0x7f74001576e0 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:14.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:14.795+0000 7f7405856640 1 --2- 172.21.15.38:0/508653540 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f74000cfe00 0x7f74000c7f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:14.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:14.795+0000 7f7405856640 1 --2- 172.21.15.38:0/508653540 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f73ec078b90 0x7f73ec07b0a0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:14.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:14.795+0000 7f7405856640 1 --2- 172.21.15.38:0/508653540 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f74000b2e60 0x7f740015c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:14.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:14.795+0000 7f7405856640 1 -- 172.21.15.38:0/508653540 >> 172.21.15.38:0/508653540 conn(0x7f74000d4440 msgr2=0x7f7400158d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:14.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:14.795+0000 7f7405856640 1 -- 172.21.15.38:0/508653540 shutdown_connections 2024-09-21T03:39:14.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:14.795+0000 7f7405856640 1 -- 172.21.15.38:0/508653540 wait complete. 2024-09-21T03:39:14.809 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:39:14.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:14.922+0000 7f35e7030640 1 -- 172.21.15.38:0/1052141099 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f35e015f4c0 msgr2=0x7f35e00b2e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:14.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:14.922+0000 7f35e7030640 1 --2- 172.21.15.38:0/1052141099 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f35e015f4c0 0x7f35e00b2e40 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f35d8005fd0 tx=0x7f35d802aa10 comp rx=0 tx=0).stop 2024-09-21T03:39:14.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:14.922+0000 7f35e7030640 1 -- 172.21.15.38:0/1052141099 shutdown_connections 2024-09-21T03:39:14.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:14.922+0000 7f35e7030640 1 --2- 172.21.15.38:0/1052141099 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f35e015f4c0 0x7f35e00b2e40 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:14.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:14.922+0000 7f35e7030640 1 -- 172.21.15.38:0/1052141099 >> 172.21.15.38:0/1052141099 conn(0x7f35e00b6b20 msgr2=0x7f35e00d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:14.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:14.922+0000 7f35e7030640 1 -- 172.21.15.38:0/1052141099 shutdown_connections 2024-09-21T03:39:14.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:14.922+0000 7f35e7030640 1 -- 172.21.15.38:0/1052141099 wait complete. 2024-09-21T03:39:14.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:14.924+0000 7f35e7030640 1 Processor -- start 2024-09-21T03:39:14.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:14.924+0000 7f35e7030640 1 -- start start 2024-09-21T03:39:14.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:14.925+0000 7f35e7030640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f35e015f4c0 0x7f35e00ce320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:14.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:14.925+0000 7f35e7030640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f35e0162280 0x7f35e01f86e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:14.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:14.926+0000 7f35e7030640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f35e01f8c20 0x7f35e01e9e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:14.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:14.926+0000 7f35e7030640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f35e00b3710 con 0x7f35e015f4c0 2024-09-21T03:39:14.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:14.926+0000 7f35e7030640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f35e00b3590 con 0x7f35e0162280 2024-09-21T03:39:14.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:14.926+0000 7f35e7030640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f35e00b3890 con 0x7f35e01f8c20 2024-09-21T03:39:14.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:14.926+0000 7f35e4dcc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f35e015f4c0 0x7f35e00ce320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:14.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:14.926+0000 7f35e4dcc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f35e015f4c0 0x7f35e00ce320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43640/0 (socket says 172.21.15.38:43640) 2024-09-21T03:39:14.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:14.926+0000 7f35e4dcc640 1 -- 172.21.15.38:0/2908468673 learned_addr learned my addr 172.21.15.38:0/2908468673 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:39:14.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:14.926+0000 7f35dffff640 1 --2- 172.21.15.38:0/2908468673 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f35e0162280 0x7f35e01f86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:14.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:14.926+0000 7f35e55cd640 1 --2- 172.21.15.38:0/2908468673 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f35e01f8c20 0x7f35e01e9e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:14.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:14.926+0000 7f35e4dcc640 1 -- 172.21.15.38:0/2908468673 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f35e01f8c20 msgr2=0x7f35e01e9e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:14.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:14.926+0000 7f35e4dcc640 1 --2- 172.21.15.38:0/2908468673 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f35e01f8c20 0x7f35e01e9e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:14.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:14.926+0000 7f35e4dcc640 1 -- 172.21.15.38:0/2908468673 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f35e0162280 msgr2=0x7f35e01f86e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:14.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:14.926+0000 7f35e4dcc640 1 --2- 172.21.15.38:0/2908468673 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f35e0162280 0x7f35e01f86e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:14.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:14.926+0000 7f35e4dcc640 1 -- 172.21.15.38:0/2908468673 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35e01f50f0 con 0x7f35e015f4c0 2024-09-21T03:39:14.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:14.926+0000 7f35e4dcc640 1 --2- 172.21.15.38:0/2908468673 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f35e015f4c0 0x7f35e00ce320 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f35d400c3c0 tx=0x7f35d400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:14.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:14.926+0000 7f35dd7fa640 1 -- 172.21.15.38:0/2908468673 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35d4017070 con 0x7f35e015f4c0 2024-09-21T03:39:14.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:14.926+0000 7f35dd7fa640 1 -- 172.21.15.38:0/2908468673 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f35d40035f0 con 0x7f35e015f4c0 2024-09-21T03:39:14.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:14.926+0000 7f35e7030640 1 -- 172.21.15.38:0/2908468673 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35e01f9410 con 0x7f35e015f4c0 2024-09-21T03:39:14.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:14.926+0000 7f35dd7fa640 1 -- 172.21.15.38:0/2908468673 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35d4003a80 con 0x7f35e015f4c0 2024-09-21T03:39:14.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:14.926+0000 7f35e7030640 1 -- 172.21.15.38:0/2908468673 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35e01f5690 con 0x7f35e015f4c0 2024-09-21T03:39:14.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:14.928+0000 7f35e7030640 1 -- 172.21.15.38:0/2908468673 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35e0162f00 con 0x7f35e015f4c0 2024-09-21T03:39:14.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:14.928+0000 7f35dd7fa640 1 -- 172.21.15.38:0/2908468673 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f35d401a070 con 0x7f35e015f4c0 2024-09-21T03:39:14.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:14.929+0000 7f35dd7fa640 1 --2- 172.21.15.38:0/2908468673 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f35c4078b90 0x7f35c407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:14.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:14.929+0000 7f35dd7fa640 1 -- 172.21.15.38:0/2908468673 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 6101+0+0 (secure 0 0 0) 0x7f35d409a0a0 con 0x7f35e015f4c0 2024-09-21T03:39:14.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:14.929+0000 7f35dffff640 1 --2- 172.21.15.38:0/2908468673 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f35c4078b90 0x7f35c407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:14.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:14.929+0000 7f35dffff640 1 --2- 172.21.15.38:0/2908468673 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f35c4078b90 0x7f35c407b0a0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f35cc006fd0 tx=0x7f35cc009060 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:14.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:14.933+0000 7f35dd7fa640 1 -- 172.21.15.38:0/2908468673 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f35d4061d90 con 0x7f35e015f4c0 2024-09-21T03:39:15.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.163+0000 7f35e7030640 1 -- 172.21.15.38:0/2908468673 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f35e00ba270 con 0x7f35e015f4c0 2024-09-21T03:39:15.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.164+0000 7f35dd7fa640 1 -- 172.21.15.38:0/2908468673 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) ==== 94+0+5231 (secure 0 0 0) 0x7f35d40667c0 con 0x7f35e015f4c0 2024-09-21T03:39:15.167 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:39:15.167 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":67,"btime":"2024-09-21T03:39:08:424230+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5783,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/1358588311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1358588311},{"type":"v1","addr":"172.21.15.38:6835","nonce":1358588311}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63},{"gid":5789,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/1676483003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":1676483003},{"type":"v1","addr":"172.21.15.195:6837","nonce":1676483003}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63},{"gid":6193,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/2771781100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":2771781100},{"type":"v1","addr":"172.21.15.195:6836","nonce":2771781100}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:39:04.779948+0000","modified":"2024-09-21T03:39:07.810041+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6546},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6546":{"gid":6546,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":3,"addr":"172.21.15.38:6833/1144075315","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":1144075315},{"type":"v1","addr":"172.21.15.38:6833","nonce":1144075315}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6546,"qdb_cluster":[6546]},"id":5}]} 2024-09-21T03:39:15.167 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 67 2024-09-21T03:39:15.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.166+0000 7f35e7030640 1 -- 172.21.15.38:0/2908468673 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f35c4078b90 msgr2=0x7f35c407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:15.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.166+0000 7f35e7030640 1 --2- 172.21.15.38:0/2908468673 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f35c4078b90 0x7f35c407b0a0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f35cc006fd0 tx=0x7f35cc009060 comp rx=0 tx=0).stop 2024-09-21T03:39:15.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.166+0000 7f35e7030640 1 -- 172.21.15.38:0/2908468673 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f35e015f4c0 msgr2=0x7f35e00ce320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:15.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.166+0000 7f35e7030640 1 --2- 172.21.15.38:0/2908468673 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f35e015f4c0 0x7f35e00ce320 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f35d400c3c0 tx=0x7f35d400c880 comp rx=0 tx=0).stop 2024-09-21T03:39:15.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.166+0000 7f35e7030640 1 -- 172.21.15.38:0/2908468673 shutdown_connections 2024-09-21T03:39:15.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.166+0000 7f35e7030640 1 --2- 172.21.15.38:0/2908468673 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f35c4078b90 0x7f35c407b0a0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:15.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.166+0000 7f35e7030640 1 --2- 172.21.15.38:0/2908468673 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f35e01f8c20 0x7f35e01e9e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:15.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.166+0000 7f35e7030640 1 --2- 172.21.15.38:0/2908468673 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f35e0162280 0x7f35e01f86e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:15.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.166+0000 7f35e7030640 1 --2- 172.21.15.38:0/2908468673 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f35e015f4c0 0x7f35e00ce320 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:15.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.166+0000 7f35e7030640 1 -- 172.21.15.38:0/2908468673 >> 172.21.15.38:0/2908468673 conn(0x7f35e00b6b20 msgr2=0x7f35e006e610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:15.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.167+0000 7f35e7030640 1 -- 172.21.15.38:0/2908468673 shutdown_connections 2024-09-21T03:39:15.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.167+0000 7f35e7030640 1 -- 172.21.15.38:0/2908468673 wait complete. 2024-09-21T03:39:15.583 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 67, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T03:39:04.779948+0000', 'modified': '2024-09-21T03:39:07.810041+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6546}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6546': {'gid': 6546, 'name': 'c', 'rank': 0, 'incarnation': 64, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.38:6833/1144075315', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.38:6832', 'nonce': 1144075315}, {'type': 'v1', 'addr': '172.21.15.38:6833', 'nonce': 1144075315}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6546, 'qdb_cluster': [6546]} 2024-09-21T03:39:15.583 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-21T03:39:15.583 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-21T03:39:15.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.702+0000 7f826ec39640 1 -- 172.21.15.38:0/2626567570 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8258003a00 msgr2=0x7f8258003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:15.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.702+0000 7f826ec39640 1 --2- 172.21.15.38:0/2626567570 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8258003a00 0x7f8258003e70 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f825c009b50 tx=0x7f8258005830 comp rx=0 tx=0).stop 2024-09-21T03:39:15.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.702+0000 7f826d1d6640 1 -- 172.21.15.38:0/2626567570 reap_dead start 2024-09-21T03:39:15.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.702+0000 7f826ec39640 1 -- 172.21.15.38:0/2626567570 shutdown_connections 2024-09-21T03:39:15.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.702+0000 7f826ec39640 1 -- 172.21.15.38:0/2626567570 >> 172.21.15.38:0/2626567570 conn(0x7f82680b46e0 msgr2=0x7f82680d23e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:15.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.702+0000 7f826ec39640 1 -- 172.21.15.38:0/2626567570 shutdown_connections 2024-09-21T03:39:15.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.702+0000 7f826ec39640 1 -- 172.21.15.38:0/2626567570 wait complete. 2024-09-21T03:39:15.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.704+0000 7f826ec39640 1 Processor -- start 2024-09-21T03:39:15.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.704+0000 7f826ec39640 1 -- start start 2024-09-21T03:39:15.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.705+0000 7f826ec39640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8258003a00 0x7f8268155a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:15.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.705+0000 7f826ec39640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f82680cc080 0x7f8268155f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:15.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.705+0000 7f826ec39640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f82681f8f70 0x7f82681f9380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:15.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.705+0000 7f826ec39640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f82680d0670 con 0x7f8258003a00 2024-09-21T03:39:15.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.705+0000 7f826ec39640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f82680b6e00 con 0x7f82680cc080 2024-09-21T03:39:15.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.705+0000 7f826ec39640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f82680d07f0 con 0x7f82681f8f70 2024-09-21T03:39:15.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.705+0000 7f826c9d5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8258003a00 0x7f8268155a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:15.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.706+0000 7f826c9d5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8258003a00 0x7f8268155a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43656/0 (socket says 172.21.15.38:43656) 2024-09-21T03:39:15.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.706+0000 7f826c9d5640 1 -- 172.21.15.38:0/3190440968 learned_addr learned my addr 172.21.15.38:0/3190440968 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:39:15.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.706+0000 7f8267fff640 1 --2- 172.21.15.38:0/3190440968 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f82680cc080 0x7f8268155f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:15.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.706+0000 7f826d1d6640 1 --2- 172.21.15.38:0/3190440968 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f82681f8f70 0x7f82681f9380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:15.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.706+0000 7f826c9d5640 1 -- 172.21.15.38:0/3190440968 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f82681f8f70 msgr2=0x7f82681f9380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:15.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.706+0000 7f826c9d5640 1 --2- 172.21.15.38:0/3190440968 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f82681f8f70 0x7f82681f9380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:15.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.706+0000 7f826c9d5640 1 -- 172.21.15.38:0/3190440968 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f82680cc080 msgr2=0x7f8268155f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:15.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.706+0000 7f826c9d5640 1 --2- 172.21.15.38:0/3190440968 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f82680cc080 0x7f8268155f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:15.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.706+0000 7f826c9d5640 1 -- 172.21.15.38:0/3190440968 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82681f58d0 con 0x7f8258003a00 2024-09-21T03:39:15.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.706+0000 7f826d1d6640 1 --2- 172.21.15.38:0/3190440968 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f82681f8f70 0x7f82681f9380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:39:15.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.706+0000 7f8267fff640 1 --2- 172.21.15.38:0/3190440968 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f82680cc080 0x7f8268155f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:39:15.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.706+0000 7f826c9d5640 1 --2- 172.21.15.38:0/3190440968 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8258003a00 0x7f8268155a00 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f825c009a70 tx=0x7f825c009850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:15.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.706+0000 7f82657fa640 1 -- 172.21.15.38:0/3190440968 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f825c038050 con 0x7f8258003a00 2024-09-21T03:39:15.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.706+0000 7f82657fa640 1 -- 172.21.15.38:0/3190440968 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f825c039040 con 0x7f8258003a00 2024-09-21T03:39:15.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.706+0000 7f826ec39640 1 -- 172.21.15.38:0/3190440968 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82681f5bb0 con 0x7f8258003a00 2024-09-21T03:39:15.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.706+0000 7f82657fa640 1 -- 172.21.15.38:0/3190440968 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f825c033e40 con 0x7f8258003a00 2024-09-21T03:39:15.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.706+0000 7f826ec39640 1 -- 172.21.15.38:0/3190440968 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82681f6170 con 0x7f8258003a00 2024-09-21T03:39:15.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.708+0000 7f82657fa640 1 -- 172.21.15.38:0/3190440968 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f825c02b8c0 con 0x7f8258003a00 2024-09-21T03:39:15.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.709+0000 7f82657fa640 1 --2- 172.21.15.38:0/3190440968 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f824c078b90 0x7f824c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:15.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.709+0000 7f82657fa640 1 -- 172.21.15.38:0/3190440968 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 6101+0+0 (secure 0 0 0) 0x7f825c0bc0a0 con 0x7f8258003a00 2024-09-21T03:39:15.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.709+0000 7f8267fff640 1 --2- 172.21.15.38:0/3190440968 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f824c078b90 0x7f824c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:15.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.710+0000 7f8267fff640 1 --2- 172.21.15.38:0/3190440968 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f824c078b90 0x7f824c07b0a0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f82680c9c30 tx=0x7f8254005f90 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:15.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.710+0000 7f8275c82740 1 Processor -- start 2024-09-21T03:39:15.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.710+0000 7f8275c82740 1 -- start start 2024-09-21T03:39:15.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.711+0000 7f8275c82740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56055cfb2270 0x56055d383fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:15.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.711+0000 7f8275c82740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56055cf893b0 0x56055d3844e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:15.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.711+0000 7f8275c82740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56055cf91590 0x56055d3a13e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:15.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.711+0000 7f8275c82740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x56055d2cc820 con 0x56055cfb2270 2024-09-21T03:39:15.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.711+0000 7f8275c82740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x56055d2bbc50 con 0x56055cf91590 2024-09-21T03:39:15.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.711+0000 7f8275c82740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x56055d2bfa60 con 0x56055cf893b0 2024-09-21T03:39:15.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.711+0000 7f826c9d5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56055cfb2270 0x56055d383fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:15.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.711+0000 7f826c9d5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56055cfb2270 0x56055d383fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43670/0 (socket says 172.21.15.38:43670) 2024-09-21T03:39:15.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.711+0000 7f826c9d5640 1 -- 172.21.15.38:0/396001613 learned_addr learned my addr 172.21.15.38:0/396001613 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:39:15.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.711+0000 7f8267fff640 1 --2- 172.21.15.38:0/396001613 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56055cf893b0 0x56055d3844e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:15.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.711+0000 7f826d1d6640 1 --2- 172.21.15.38:0/396001613 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56055cf91590 0x56055d3a13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:15.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.712+0000 7f826c9d5640 1 -- 172.21.15.38:0/396001613 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56055cf893b0 msgr2=0x56055d3844e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:15.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.712+0000 7f826c9d5640 1 --2- 172.21.15.38:0/396001613 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56055cf893b0 0x56055d3844e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:15.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.712+0000 7f826c9d5640 1 -- 172.21.15.38:0/396001613 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56055cf91590 msgr2=0x56055d3a13e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:15.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.712+0000 7f826c9d5640 1 --2- 172.21.15.38:0/396001613 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56055cf91590 0x56055d3a13e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:15.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.712+0000 7f826c9d5640 1 -- 172.21.15.38:0/396001613 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56055d01d850 con 0x56055cfb2270 2024-09-21T03:39:15.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.712+0000 7f826c9d5640 1 --2- 172.21.15.38:0/396001613 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56055cfb2270 0x56055d383fa0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f825c0ba440 tx=0x7f825c0833c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37d37219568596a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:15.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.712+0000 7f82527fc640 1 -- 172.21.15.38:0/396001613 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f825c083c50 con 0x56055cfb2270 2024-09-21T03:39:15.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.712+0000 7f82527fc640 1 -- 172.21.15.38:0/396001613 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f825c088730 con 0x56055cfb2270 2024-09-21T03:39:15.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.712+0000 7f82527fc640 1 -- 172.21.15.38:0/396001613 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f825c08d670 con 0x56055cfb2270 2024-09-21T03:39:15.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.713+0000 7f8275c82740 1 -- 172.21.15.38:0/396001613 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56055cfb2270 msgr2=0x56055d383fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:15.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.713+0000 7f8275c82740 1 --2- 172.21.15.38:0/396001613 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56055cfb2270 0x56055d383fa0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f825c0ba440 tx=0x7f825c0833c0 comp rx=0 tx=0).stop 2024-09-21T03:39:15.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.713+0000 7f8275c82740 1 -- 172.21.15.38:0/396001613 shutdown_connections 2024-09-21T03:39:15.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.713+0000 7f8275c82740 1 --2- 172.21.15.38:0/396001613 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56055cf91590 0x56055d3a13e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:15.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.713+0000 7f8275c82740 1 --2- 172.21.15.38:0/396001613 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56055cf893b0 0x56055d3844e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:15.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.713+0000 7f8275c82740 1 --2- 172.21.15.38:0/396001613 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56055cfb2270 0x56055d383fa0 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:15.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.713+0000 7f8275c82740 1 -- 172.21.15.38:0/396001613 >> 172.21.15.38:0/396001613 conn(0x56055cfa63c0 msgr2=0x56055d35a260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:15.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.713+0000 7f8275c82740 1 -- 172.21.15.38:0/396001613 shutdown_connections 2024-09-21T03:39:15.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.713+0000 7f8275c82740 1 -- 172.21.15.38:0/396001613 wait complete. 2024-09-21T03:39:15.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.714+0000 7f8275c82740 1 -- start start 2024-09-21T03:39:15.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.714+0000 7f8275c82740 1 Processor -- start 2024-09-21T03:39:15.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.715+0000 7f8275c82740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56055cf893b0 0x56055d398c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:15.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.715+0000 7f8275c82740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56055cf91590 0x56055d371ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:15.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.715+0000 7f8275c82740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56055cfb2270 0x56055d396e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:15.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.715+0000 7f8275c82740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x56055d02c620 con 0x56055cfb2270 2024-09-21T03:39:15.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.715+0000 7f8275c82740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x56055d2615f0 con 0x56055cf91590 2024-09-21T03:39:15.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.715+0000 7f8275c82740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x56055d2bbc50 con 0x56055cf893b0 2024-09-21T03:39:15.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.715+0000 7f8275c82740 10 client.0 _ll_register_callbacks cb 0x56055d2650a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:39:15.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.715+0000 7f826d1d6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56055cfb2270 0x56055d396e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:15.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.715+0000 7f826d1d6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56055cfb2270 0x56055d396e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43672/0 (socket says 172.21.15.38:43672) 2024-09-21T03:39:15.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.715+0000 7f826d1d6640 1 -- 172.21.15.38:0/3870073385 learned_addr learned my addr 172.21.15.38:0/3870073385 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:39:15.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.716+0000 7f826c9d5640 1 --2- 172.21.15.38:0/3870073385 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56055cf893b0 0x56055d398c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:15.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.716+0000 7f8267fff640 1 --2- 172.21.15.38:0/3870073385 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56055cf91590 0x56055d371ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:15.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.716+0000 7f826d1d6640 1 -- 172.21.15.38:0/3870073385 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56055cf893b0 msgr2=0x56055d398c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:15.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.716+0000 7f826d1d6640 1 --2- 172.21.15.38:0/3870073385 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56055cf893b0 0x56055d398c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:15.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.716+0000 7f826d1d6640 1 -- 172.21.15.38:0/3870073385 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56055cf91590 msgr2=0x56055d371ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:15.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.716+0000 7f826d1d6640 1 --2- 172.21.15.38:0/3870073385 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56055cf91590 0x56055d371ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:15.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.716+0000 7f826d1d6640 1 -- 172.21.15.38:0/3870073385 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56055d0946b0 con 0x56055cfb2270 2024-09-21T03:39:15.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.716+0000 7f826d1d6640 1 --2- 172.21.15.38:0/3870073385 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56055cfb2270 0x56055d396e70 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f82680cc9e0 tx=0x7f8260011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9031d2632d43da9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:15.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.716+0000 7f82477fe640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-21T03:39:15.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.716+0000 7f82477fe640 1 -- 172.21.15.38:0/3870073385 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8260017020 con 0x56055cfb2270 2024-09-21T03:39:15.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.716+0000 7f8275c82740 1 -- 172.21.15.38:0/3870073385 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f825c088730 con 0x56055cfb2270 2024-09-21T03:39:15.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.716+0000 7f82477fe640 1 -- 172.21.15.38:0/3870073385 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 6101+0+0 (secure 0 0 0) 0x7f8260003b80 con 0x56055cfb2270 2024-09-21T03:39:15.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.716+0000 7f82477fe640 1 -- 172.21.15.38:0/3870073385 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7f8260021050 con 0x56055cfb2270 2024-09-21T03:39:15.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.716+0000 7f8275c82740 10 client.6699 fetch_fsmap learned FSMap version 67 2024-09-21T03:39:15.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.716+0000 7f8275c82740 1 -- 172.21.15.38:0/3870073385 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=2+}) -- 0x56055d372280 con 0x56055cfb2270 2024-09-21T03:39:15.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.716+0000 7f82477fe640 1 -- 172.21.15.38:0/3870073385 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f825c088730 con 0x56055cfb2270 2024-09-21T03:39:15.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.717+0000 7f82477fe640 1 -- 172.21.15.38:0/3870073385 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 67) ==== 3502+0+0 (secure 0 0 0) 0x7f8260024020 con 0x56055cfb2270 2024-09-21T03:39:15.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.717+0000 7f8275c82740 10 client.6699 fetch_fsmap finished waiting for FSMap version 67 2024-09-21T03:39:15.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.717+0000 7f8275c82740 10 client.6699 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T03:39:15.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.717+0000 7f8275c82740 20 client.6699 populate_metadata read hostname 'smithi038' 2024-09-21T03:39:15.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.717+0000 7f8275c82740 1 --2- 172.21.15.38:0/3870073385 >> [v2:172.21.15.38:6832/1144075315,v1:172.21.15.38:6833/1144075315] conn(0x56055cf70e40 0x56055d395b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:15.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.717+0000 7f8275c82740 4 client.6699 mds_command: new command op to 6546 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:39:15.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.717+0000 7f8275c82740 1 -- 172.21.15.38:0/3870073385 --> [v2:172.21.15.38:6832/1144075315,v1:172.21.15.38:6833/1144075315] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56055d2ce770 con 0x56055cf70e40 2024-09-21T03:39:15.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.717+0000 7f826c9d5640 1 --2- 172.21.15.38:0/3870073385 >> [v2:172.21.15.38:6832/1144075315,v1:172.21.15.38:6833/1144075315] conn(0x56055cf70e40 0x56055d395b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:15.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.717+0000 7f82477fe640 10 client.6699 ms_handle_connect on v2:172.21.15.38:6832/1144075315 2024-09-21T03:39:15.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.717+0000 7f826c9d5640 1 --2- 172.21.15.38:0/3870073385 >> [v2:172.21.15.38:6832/1144075315,v1:172.21.15.38:6833/1144075315] conn(0x56055cf70e40 0x56055d395b90 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=821f7ee23b0aa458 server_cookie=b2bd35bdfd348774 in_seq=0 out_seq=0 2024-09-21T03:39:15.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.718+0000 7f82477fe640 1 -- 172.21.15.38:0/3870073385 <== mds.0 v2:172.21.15.38:6832/1144075315 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56055d2ce770 con 0x56055cf70e40 2024-09-21T03:39:15.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.718+0000 7f82477fe640 10 client.6699 handle_command_reply: tid=0 2024-09-21T03:39:15.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.718+0000 7f8275c82740 1 client.6699 shutdown 2024-09-21T03:39:15.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.718+0000 7f8275c82740 1 -- 172.21.15.38:0/3870073385 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56055cfb2270 msgr2=0x56055d396e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:15.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.718+0000 7f8275c82740 1 --2- 172.21.15.38:0/3870073385 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56055cfb2270 0x56055d396e70 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f82680cc9e0 tx=0x7f8260011a30 comp rx=0 tx=0).stop 2024-09-21T03:39:15.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.718+0000 7f8275c82740 1 -- 172.21.15.38:0/3870073385 shutdown_connections 2024-09-21T03:39:15.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.718+0000 7f8275c82740 1 --2- 172.21.15.38:0/3870073385 >> [v2:172.21.15.38:6832/1144075315,v1:172.21.15.38:6833/1144075315] conn(0x56055cf70e40 0x56055d395b90 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:15.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.718+0000 7f8275c82740 1 --2- 172.21.15.38:0/3870073385 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56055cfb2270 0x56055d396e70 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:15.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.718+0000 7f8275c82740 1 --2- 172.21.15.38:0/3870073385 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56055cf91590 0x56055d371ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:15.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.718+0000 7f82477fe640 0 client.6699 ms_handle_reset on v2:172.21.15.38:6832/1144075315 2024-09-21T03:39:15.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.718+0000 7f8275c82740 1 --2- 172.21.15.38:0/3870073385 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56055cf893b0 0x56055d398c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:15.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.718+0000 7f8275c82740 1 -- 172.21.15.38:0/3870073385 >> 172.21.15.38:0/3870073385 conn(0x56055cfa63c0 msgr2=0x56055d35b260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:15.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.718+0000 7f8275c82740 1 -- 172.21.15.38:0/3870073385 shutdown_connections 2024-09-21T03:39:15.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.718+0000 7f8275c82740 1 -- 172.21.15.38:0/3870073385 wait complete. 2024-09-21T03:39:15.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.718+0000 7f8275c82740 20 client.6699 trim_cache size 0 max 16384 2024-09-21T03:39:15.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.737+0000 7f8275c82740 1 Processor -- start 2024-09-21T03:39:15.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.737+0000 7f8275c82740 1 -- start start 2024-09-21T03:39:15.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.738+0000 7f8275c82740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56055cf893b0 0x56055d35a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:15.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.738+0000 7f8275c82740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56055cf91590 0x56055d3c1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:15.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.738+0000 7f8275c82740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56055cfb2270 0x56055d383ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:15.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.738+0000 7f8275c82740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f8268201c50 con 0x56055cfb2270 2024-09-21T03:39:15.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.738+0000 7f8275c82740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f8268201040 con 0x56055cf893b0 2024-09-21T03:39:15.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.738+0000 7f8275c82740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f8268201600 con 0x56055cf91590 2024-09-21T03:39:15.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.738+0000 7f826d1d6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56055cfb2270 0x56055d383ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:15.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.738+0000 7f826d1d6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56055cfb2270 0x56055d383ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43686/0 (socket says 172.21.15.38:43686) 2024-09-21T03:39:15.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.738+0000 7f826d1d6640 1 -- 172.21.15.38:0/1936305945 learned_addr learned my addr 172.21.15.38:0/1936305945 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:39:15.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.738+0000 7f826c9d5640 1 --2- 172.21.15.38:0/1936305945 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56055cf893b0 0x56055d35a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:15.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.738+0000 7f8267fff640 1 --2- 172.21.15.38:0/1936305945 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56055cf91590 0x56055d3c1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:15.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.739+0000 7f826d1d6640 1 -- 172.21.15.38:0/1936305945 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56055cf91590 msgr2=0x56055d3c1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:15.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.739+0000 7f826d1d6640 1 --2- 172.21.15.38:0/1936305945 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56055cf91590 0x56055d3c1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:15.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.739+0000 7f826d1d6640 1 -- 172.21.15.38:0/1936305945 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56055cf893b0 msgr2=0x56055d35a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:15.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.739+0000 7f826d1d6640 1 --2- 172.21.15.38:0/1936305945 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56055cf893b0 0x56055d35a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:15.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.739+0000 7f826d1d6640 1 -- 172.21.15.38:0/1936305945 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56055d3733b0 con 0x56055cfb2270 2024-09-21T03:39:15.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.739+0000 7f826d1d6640 1 --2- 172.21.15.38:0/1936305945 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56055cfb2270 0x56055d383ce0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f82600052d0 tx=0x7f8260005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df0a3a8034e81feb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:15.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.739+0000 7f8251ffb640 1 -- 172.21.15.38:0/1936305945 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8260017050 con 0x56055cfb2270 2024-09-21T03:39:15.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.739+0000 7f8251ffb640 1 -- 172.21.15.38:0/1936305945 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8260003cc0 con 0x56055cfb2270 2024-09-21T03:39:15.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.739+0000 7f8251ffb640 1 -- 172.21.15.38:0/1936305945 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8260005430 con 0x56055cfb2270 2024-09-21T03:39:15.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.740+0000 7f8275c82740 1 -- 172.21.15.38:0/1936305945 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56055cfb2270 msgr2=0x56055d383ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:15.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.740+0000 7f8275c82740 1 --2- 172.21.15.38:0/1936305945 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56055cfb2270 0x56055d383ce0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f82600052d0 tx=0x7f8260005e50 comp rx=0 tx=0).stop 2024-09-21T03:39:15.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.740+0000 7f8275c82740 1 -- 172.21.15.38:0/1936305945 shutdown_connections 2024-09-21T03:39:15.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.740+0000 7f8275c82740 1 --2- 172.21.15.38:0/1936305945 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56055cfb2270 0x56055d383ce0 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:15.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.740+0000 7f8275c82740 1 --2- 172.21.15.38:0/1936305945 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56055cf91590 0x56055d3c1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:15.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.740+0000 7f8275c82740 1 --2- 172.21.15.38:0/1936305945 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56055cf893b0 0x56055d35a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:15.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.740+0000 7f8275c82740 1 -- 172.21.15.38:0/1936305945 >> 172.21.15.38:0/1936305945 conn(0x56055cfa63c0 msgr2=0x56055d3a0890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:15.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.740+0000 7f8275c82740 1 -- 172.21.15.38:0/1936305945 shutdown_connections 2024-09-21T03:39:15.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.740+0000 7f8275c82740 1 -- 172.21.15.38:0/1936305945 wait complete. 2024-09-21T03:39:15.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.741+0000 7f8275c82740 1 -- start start 2024-09-21T03:39:15.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.741+0000 7f8275c82740 1 Processor -- start 2024-09-21T03:39:15.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.742+0000 7f8275c82740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56055cf893b0 0x56055d35a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:15.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.742+0000 7f8275c82740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56055cf91590 0x56055d35a630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:15.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.742+0000 7f8275c82740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56055cfb2270 0x56055d370690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:15.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.742+0000 7f8275c82740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x56055d37e8e0 con 0x56055cfb2270 2024-09-21T03:39:15.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.742+0000 7f8275c82740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x56055d2cc820 con 0x56055cf893b0 2024-09-21T03:39:15.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.742+0000 7f8275c82740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x56055d37ef00 con 0x56055cf91590 2024-09-21T03:39:15.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.742+0000 7f8275c82740 10 client.0 _ll_register_callbacks cb 0x7f8260001ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:39:15.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.742+0000 7f826d1d6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56055cfb2270 0x56055d370690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:15.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.742+0000 7f826d1d6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56055cfb2270 0x56055d370690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43694/0 (socket says 172.21.15.38:43694) 2024-09-21T03:39:15.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.742+0000 7f826d1d6640 1 -- 172.21.15.38:0/4092687770 learned_addr learned my addr 172.21.15.38:0/4092687770 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:39:15.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.742+0000 7f826c9d5640 1 --2- 172.21.15.38:0/4092687770 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56055cf893b0 0x56055d35a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:15.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.742+0000 7f8267fff640 1 --2- 172.21.15.38:0/4092687770 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56055cf91590 0x56055d35a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:15.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.743+0000 7f826d1d6640 1 -- 172.21.15.38:0/4092687770 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56055cf91590 msgr2=0x56055d35a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:15.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.743+0000 7f826d1d6640 1 --2- 172.21.15.38:0/4092687770 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56055cf91590 0x56055d35a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:15.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.743+0000 7f826d1d6640 1 -- 172.21.15.38:0/4092687770 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56055cf893b0 msgr2=0x56055d35a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:15.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.743+0000 7f826d1d6640 1 --2- 172.21.15.38:0/4092687770 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56055cf893b0 0x56055d35a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:15.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.743+0000 7f826d1d6640 1 -- 172.21.15.38:0/4092687770 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56055d396db0 con 0x56055cfb2270 2024-09-21T03:39:15.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.743+0000 7f8267fff640 1 --2- 172.21.15.38:0/4092687770 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56055cf91590 0x56055d35a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:39:15.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.743+0000 7f826d1d6640 1 --2- 172.21.15.38:0/4092687770 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56055cfb2270 0x56055d370690 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f8260027000 tx=0x7f8260012230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f20a3af6b324f929 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:15.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.743+0000 7f82477fe640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-21T03:39:15.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.743+0000 7f82477fe640 1 -- 172.21.15.38:0/4092687770 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8260034060 con 0x56055cfb2270 2024-09-21T03:39:15.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.743+0000 7f82477fe640 1 -- 172.21.15.38:0/4092687770 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 6101+0+0 (secure 0 0 0) 0x7f8260005640 con 0x56055cfb2270 2024-09-21T03:39:15.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.743+0000 7f8275c82740 1 -- 172.21.15.38:0/4092687770 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8260003cc0 con 0x56055cfb2270 2024-09-21T03:39:15.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.743+0000 7f82477fe640 1 -- 172.21.15.38:0/4092687770 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7f826003a050 con 0x56055cfb2270 2024-09-21T03:39:15.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.743+0000 7f8275c82740 10 client.6705 fetch_fsmap learned FSMap version 67 2024-09-21T03:39:15.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.743+0000 7f8275c82740 1 -- 172.21.15.38:0/4092687770 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=2+}) -- 0x56055d371600 con 0x56055cfb2270 2024-09-21T03:39:15.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.743+0000 7f82477fe640 1 -- 172.21.15.38:0/4092687770 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8260003cc0 con 0x56055cfb2270 2024-09-21T03:39:15.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.743+0000 7f82477fe640 1 -- 172.21.15.38:0/4092687770 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 67) ==== 3502+0+0 (secure 0 0 0) 0x7f826003d020 con 0x56055cfb2270 2024-09-21T03:39:15.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.743+0000 7f8275c82740 10 client.6705 fetch_fsmap finished waiting for FSMap version 67 2024-09-21T03:39:15.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.743+0000 7f8275c82740 10 client.6705 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T03:39:15.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.743+0000 7f8275c82740 20 client.6705 populate_metadata read hostname 'smithi038' 2024-09-21T03:39:15.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.743+0000 7f8275c82740 1 --2- 172.21.15.38:0/4092687770 >> [v2:172.21.15.38:6832/1144075315,v1:172.21.15.38:6833/1144075315] conn(0x56055d371020 0x56055d3c0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:15.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.744+0000 7f8275c82740 4 client.6705 mds_command: new command op to 6546 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T03:39:15.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.744+0000 7f8275c82740 1 -- 172.21.15.38:0/4092687770 --> [v2:172.21.15.38:6832/1144075315,v1:172.21.15.38:6833/1144075315] -- command(tid 0: {"prefix": "status"}) -- 0x56055d2ca220 con 0x56055d371020 2024-09-21T03:39:15.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.744+0000 7f826c9d5640 1 --2- 172.21.15.38:0/4092687770 >> [v2:172.21.15.38:6832/1144075315,v1:172.21.15.38:6833/1144075315] conn(0x56055d371020 0x56055d3c0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:15.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.744+0000 7f82477fe640 10 client.6705 ms_handle_connect on v2:172.21.15.38:6832/1144075315 2024-09-21T03:39:15.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.744+0000 7f826c9d5640 1 --2- 172.21.15.38:0/4092687770 >> [v2:172.21.15.38:6832/1144075315,v1:172.21.15.38:6833/1144075315] conn(0x56055d371020 0x56055d3c0430 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa896dec1efc4656 server_cookie=173d4da41673b6e5 in_seq=0 out_seq=0 2024-09-21T03:39:15.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.744+0000 7f82477fe640 1 -- 172.21.15.38:0/4092687770 <== mds.0 v2:172.21.15.38:6832/1144075315 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x56055d2ca220 con 0x56055d371020 2024-09-21T03:39:15.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.744+0000 7f82477fe640 10 client.6705 handle_command_reply: tid=0 2024-09-21T03:39:15.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.744+0000 7f8275c82740 1 client.6705 shutdown 2024-09-21T03:39:15.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.744+0000 7f8275c82740 1 -- 172.21.15.38:0/4092687770 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56055cfb2270 msgr2=0x56055d370690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:15.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.744+0000 7f8275c82740 1 --2- 172.21.15.38:0/4092687770 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56055cfb2270 0x56055d370690 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f8260027000 tx=0x7f8260012230 comp rx=0 tx=0).stop 2024-09-21T03:39:15.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.745+0000 7f8275c82740 1 -- 172.21.15.38:0/4092687770 shutdown_connections 2024-09-21T03:39:15.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.745+0000 7f8275c82740 1 --2- 172.21.15.38:0/4092687770 >> [v2:172.21.15.38:6832/1144075315,v1:172.21.15.38:6833/1144075315] conn(0x56055d371020 0x56055d3c0430 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:15.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.745+0000 7f8275c82740 1 --2- 172.21.15.38:0/4092687770 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56055cfb2270 0x56055d370690 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:15.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.745+0000 7f8275c82740 1 --2- 172.21.15.38:0/4092687770 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56055cf91590 0x56055d35a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:15.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.745+0000 7f82477fe640 0 client.6705 ms_handle_reset on v2:172.21.15.38:6832/1144075315 2024-09-21T03:39:15.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.745+0000 7f8275c82740 1 --2- 172.21.15.38:0/4092687770 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56055cf893b0 0x56055d35a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:15.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.745+0000 7f8275c82740 1 -- 172.21.15.38:0/4092687770 >> 172.21.15.38:0/4092687770 conn(0x56055cfa63c0 msgr2=0x56055d3a27e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:15.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.745+0000 7f8275c82740 1 -- 172.21.15.38:0/4092687770 shutdown_connections 2024-09-21T03:39:15.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.745+0000 7f8275c82740 1 -- 172.21.15.38:0/4092687770 wait complete. 2024-09-21T03:39:15.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.745+0000 7f8275c82740 20 client.6705 trim_cache size 0 max 16384 2024-09-21T03:39:15.753 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-09-21T03:39:15.753 INFO:teuthology.orchestra.run.smithi038.stdout: "cluster_fsid": "9d50d66f-656b-4fe2-8e3b-fe53b79c4d43", 2024-09-21T03:39:15.753 INFO:teuthology.orchestra.run.smithi038.stdout: "whoami": 0, 2024-09-21T03:39:15.753 INFO:teuthology.orchestra.run.smithi038.stdout: "id": 6546, 2024-09-21T03:39:15.753 INFO:teuthology.orchestra.run.smithi038.stdout: "want_state": "up:active", 2024-09-21T03:39:15.753 INFO:teuthology.orchestra.run.smithi038.stdout: "state": "up:active", 2024-09-21T03:39:15.753 INFO:teuthology.orchestra.run.smithi038.stdout: "fs_name": "cephfs", 2024-09-21T03:39:15.753 INFO:teuthology.orchestra.run.smithi038.stdout: "rank_uptime": 10.337535474999999, 2024-09-21T03:39:15.753 INFO:teuthology.orchestra.run.smithi038.stdout: "mdsmap_epoch": 67, 2024-09-21T03:39:15.753 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch": 82, 2024-09-21T03:39:15.753 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch_barrier": 82, 2024-09-21T03:39:15.753 INFO:teuthology.orchestra.run.smithi038.stdout: "uptime": 14.983897437 2024-09-21T03:39:15.753 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-09-21T03:39:15.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.745+0000 7f826ec39640 1 -- 172.21.15.38:0/3190440968 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f824c078b90 msgr2=0x7f824c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:15.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.745+0000 7f826ec39640 1 --2- 172.21.15.38:0/3190440968 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f824c078b90 0x7f824c07b0a0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f82680c9c30 tx=0x7f8254005f90 comp rx=0 tx=0).stop 2024-09-21T03:39:15.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.745+0000 7f826ec39640 1 -- 172.21.15.38:0/3190440968 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8258003a00 msgr2=0x7f8268155a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:15.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.745+0000 7f826ec39640 1 --2- 172.21.15.38:0/3190440968 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8258003a00 0x7f8268155a00 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f825c009a70 tx=0x7f825c009850 comp rx=0 tx=0).stop 2024-09-21T03:39:15.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.745+0000 7f826ec39640 1 -- 172.21.15.38:0/3190440968 shutdown_connections 2024-09-21T03:39:15.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.745+0000 7f826ec39640 1 --2- 172.21.15.38:0/3190440968 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f824c078b90 0x7f824c07b0a0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:15.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.745+0000 7f826ec39640 1 --2- 172.21.15.38:0/3190440968 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f82681f8f70 0x7f82681f9380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:15.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.745+0000 7f826ec39640 1 --2- 172.21.15.38:0/3190440968 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f82680cc080 0x7f8268155f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:15.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.745+0000 7f826ec39640 1 --2- 172.21.15.38:0/3190440968 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8258003a00 0x7f8268155a00 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:15.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.745+0000 7f826ec39640 1 -- 172.21.15.38:0/3190440968 >> 172.21.15.38:0/3190440968 conn(0x7f82680b46e0 msgr2=0x7f82680b32a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:15.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.745+0000 7f826ec39640 1 -- 172.21.15.38:0/3190440968 shutdown_connections 2024-09-21T03:39:15.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.745+0000 7f826ec39640 1 -- 172.21.15.38:0/3190440968 wait complete. 2024-09-21T03:39:15.759 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T03:39:15.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.873+0000 7fcaab989640 1 -- 172.21.15.38:0/2568610914 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcaa40d1090 msgr2=0x7fcaa415a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:15.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.873+0000 7fcaab989640 1 --2- 172.21.15.38:0/2568610914 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcaa40d1090 0x7fcaa415a6f0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fca9c005fd0 tx=0x7fca9c02a940 comp rx=0 tx=0).stop 2024-09-21T03:39:15.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.873+0000 7fcaab989640 1 -- 172.21.15.38:0/2568610914 shutdown_connections 2024-09-21T03:39:15.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.873+0000 7fcaab989640 1 --2- 172.21.15.38:0/2568610914 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcaa40d1090 0x7fcaa415a6f0 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:15.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.873+0000 7fcaab989640 1 -- 172.21.15.38:0/2568610914 >> 172.21.15.38:0/2568610914 conn(0x7fcaa4072990 msgr2=0x7fcaa4072da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:15.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.873+0000 7fcaab989640 1 -- 172.21.15.38:0/2568610914 shutdown_connections 2024-09-21T03:39:15.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.873+0000 7fcaab989640 1 -- 172.21.15.38:0/2568610914 wait complete. 2024-09-21T03:39:15.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.875+0000 7fcaab989640 1 Processor -- start 2024-09-21T03:39:15.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.875+0000 7fcaab989640 1 -- start start 2024-09-21T03:39:15.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.877+0000 7fcaab989640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcaa40d01f0 0x7fcaa414d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:15.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.877+0000 7fcaab989640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcaa415daf0 0x7fcaa4149f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:15.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.877+0000 7fcaab989640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcaa41f9370 0x7fcaa41f8760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:15.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.877+0000 7fcaab989640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fcaa40cd3d0 con 0x7fcaa41f9370 2024-09-21T03:39:15.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.877+0000 7fcaab989640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fcaa40cd250 con 0x7fcaa40d01f0 2024-09-21T03:39:15.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.877+0000 7fcaab989640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fcaa40cd550 con 0x7fcaa415daf0 2024-09-21T03:39:15.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.877+0000 7fcaa9f26640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcaa41f9370 0x7fcaa41f8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:15.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.877+0000 7fcaa9f26640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcaa41f9370 0x7fcaa41f8760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43720/0 (socket says 172.21.15.38:43720) 2024-09-21T03:39:15.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.877+0000 7fcaa9f26640 1 -- 172.21.15.38:0/3721625363 learned_addr learned my addr 172.21.15.38:0/3721625363 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:39:15.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.877+0000 7fcaa9725640 1 --2- 172.21.15.38:0/3721625363 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcaa40d01f0 0x7fcaa414d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:15.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.877+0000 7fcaa8f24640 1 --2- 172.21.15.38:0/3721625363 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcaa415daf0 0x7fcaa4149f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:15.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.877+0000 7fcaa9f26640 1 -- 172.21.15.38:0/3721625363 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcaa415daf0 msgr2=0x7fcaa4149f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:15.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.877+0000 7fcaa9f26640 1 --2- 172.21.15.38:0/3721625363 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcaa415daf0 0x7fcaa4149f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:15.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.877+0000 7fcaa9f26640 1 -- 172.21.15.38:0/3721625363 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcaa40d01f0 msgr2=0x7fcaa414d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:15.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.877+0000 7fcaa9f26640 1 --2- 172.21.15.38:0/3721625363 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcaa40d01f0 0x7fcaa414d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:15.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.877+0000 7fcaa9f26640 1 -- 172.21.15.38:0/3721625363 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcaa40cc0e0 con 0x7fcaa41f9370 2024-09-21T03:39:15.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.877+0000 7fcaa9f26640 1 --2- 172.21.15.38:0/3721625363 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcaa41f9370 0x7fcaa41f8760 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fca9c009cd0 tx=0x7fca9c031ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:15.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.877+0000 7fca99ffb640 1 -- 172.21.15.38:0/3721625363 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca9c03e070 con 0x7fcaa41f9370 2024-09-21T03:39:15.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.878+0000 7fca99ffb640 1 -- 172.21.15.38:0/3721625363 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fca9c004600 con 0x7fcaa41f9370 2024-09-21T03:39:15.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.878+0000 7fcaab989640 1 -- 172.21.15.38:0/3721625363 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcaa40cc3c0 con 0x7fcaa41f9370 2024-09-21T03:39:15.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.878+0000 7fca99ffb640 1 -- 172.21.15.38:0/3721625363 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca9c043460 con 0x7fcaa41f9370 2024-09-21T03:39:15.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.878+0000 7fcaab989640 1 -- 172.21.15.38:0/3721625363 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcaa406e370 con 0x7fcaa41f9370 2024-09-21T03:39:15.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.879+0000 7fcaab989640 1 -- 172.21.15.38:0/3721625363 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcaa40cc600 con 0x7fcaa41f9370 2024-09-21T03:39:15.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.879+0000 7fca99ffb640 1 -- 172.21.15.38:0/3721625363 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fca9c043600 con 0x7fcaa41f9370 2024-09-21T03:39:15.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.880+0000 7fca99ffb640 1 --2- 172.21.15.38:0/3721625363 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fca88078b90 0x7fca8807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:15.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.880+0000 7fca99ffb640 1 -- 172.21.15.38:0/3721625363 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 6101+0+0 (secure 0 0 0) 0x7fca9c0bc9e0 con 0x7fcaa41f9370 2024-09-21T03:39:15.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.880+0000 7fcaa9725640 1 --2- 172.21.15.38:0/3721625363 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fca88078b90 0x7fca8807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:15.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.881+0000 7fcaa9725640 1 --2- 172.21.15.38:0/3721625363 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fca88078b90 0x7fca8807b0a0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fcaa0000f60 tx=0x7fcaa0009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:15.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:15.885+0000 7fca99ffb640 1 -- 172.21.15.38:0/3721625363 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fcaa40cc600 con 0x7fcaa41f9370 2024-09-21T03:39:16.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:16.116+0000 7fcaab989640 1 -- 172.21.15.38:0/3721625363 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fcaa415a6f0 con 0x7fcaa41f9370 2024-09-21T03:39:16.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:16.117+0000 7fca99ffb640 1 -- 172.21.15.38:0/3721625363 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v67) ==== 53+0+73 (secure 0 0 0) 0x7fca9c0845a0 con 0x7fcaa41f9370 2024-09-21T03:39:16.119 INFO:teuthology.orchestra.run.smithi038.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T03:39:16.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:16.119+0000 7fcaab989640 1 -- 172.21.15.38:0/3721625363 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fca88078b90 msgr2=0x7fca8807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:16.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:16.119+0000 7fcaab989640 1 --2- 172.21.15.38:0/3721625363 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fca88078b90 0x7fca8807b0a0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fcaa0000f60 tx=0x7fcaa0009040 comp rx=0 tx=0).stop 2024-09-21T03:39:16.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:16.119+0000 7fcaab989640 1 -- 172.21.15.38:0/3721625363 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcaa41f9370 msgr2=0x7fcaa41f8760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:16.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:16.119+0000 7fcaab989640 1 --2- 172.21.15.38:0/3721625363 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcaa41f9370 0x7fcaa41f8760 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fca9c009cd0 tx=0x7fca9c031ea0 comp rx=0 tx=0).stop 2024-09-21T03:39:16.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:16.119+0000 7fcaab989640 1 -- 172.21.15.38:0/3721625363 shutdown_connections 2024-09-21T03:39:16.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:16.119+0000 7fcaab989640 1 --2- 172.21.15.38:0/3721625363 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcaa41f9370 0x7fcaa41f8760 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:16.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:16.119+0000 7fcaab989640 1 --2- 172.21.15.38:0/3721625363 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fca88078b90 0x7fca8807b0a0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:16.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:16.119+0000 7fcaab989640 1 --2- 172.21.15.38:0/3721625363 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcaa415daf0 0x7fcaa4149f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:16.123 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:16.119+0000 7fcaab989640 1 --2- 172.21.15.38:0/3721625363 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcaa40d01f0 0x7fcaa414d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:16.123 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:16.119+0000 7fcaab989640 1 -- 172.21.15.38:0/3721625363 >> 172.21.15.38:0/3721625363 conn(0x7fcaa4072990 msgr2=0x7fcaa4068d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:16.123 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:16.119+0000 7fcaab989640 1 -- 172.21.15.38:0/3721625363 shutdown_connections 2024-09-21T03:39:16.123 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:16.119+0000 7fcaab989640 1 -- 172.21.15.38:0/3721625363 wait complete. 2024-09-21T03:39:16.133 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-21T03:39:16.133 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T03:39:16.133 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi038.front.sepia.ceph.com 2024-09-21T03:39:16.133 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-21T03:39:16.133 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T03:39:16.133 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-21T03:39:16.133 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-21T03:39:16.133 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T03:39:16.133 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T03:39:16.133 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:39:16.134 DEBUG:teuthology.orchestra.run.smithi038:> ip addr 2024-09-21T03:39:16.149 INFO:teuthology.orchestra.run.smithi038.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T03:39:16.150 INFO:teuthology.orchestra.run.smithi038.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T03:39:16.150 INFO:teuthology.orchestra.run.smithi038.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T03:39:16.150 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:39:16.150 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 ::1/128 scope host 2024-09-21T03:39:16.150 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:39:16.150 INFO:teuthology.orchestra.run.smithi038.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T03:39:16.150 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:6b:fd:a2 brd ff:ff:ff:ff:ff:ff 2024-09-21T03:39:16.150 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp7s0f0 2024-09-21T03:39:16.150 INFO:teuthology.orchestra.run.smithi038.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T03:39:16.150 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:6b:fd:a3 brd ff:ff:ff:ff:ff:ff 2024-09-21T03:39:16.150 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp7s0f1 2024-09-21T03:39:16.150 INFO:teuthology.orchestra.run.smithi038.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T03:39:16.150 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:bd:15:fc brd ff:ff:ff:ff:ff:ff 2024-09-21T03:39:16.151 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp1s0f0 2024-09-21T03:39:16.151 INFO:teuthology.orchestra.run.smithi038.stdout: inet 172.21.15.38/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-21T03:39:16.151 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft 42453sec preferred_lft 42453sec 2024-09-21T03:39:16.151 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 fe80::ec4:7aff:febd:15fc/64 scope link 2024-09-21T03:39:16.151 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:39:16.151 INFO:teuthology.orchestra.run.smithi038.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T03:39:16.151 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:bd:15:fd brd ff:ff:ff:ff:ff:ff 2024-09-21T03:39:16.151 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp1s0f1 2024-09-21T03:39:16.151 INFO:teuthology.orchestra.run.smithi038.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T03:39:16.151 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 92:55:c7:97:6f:b5 brd ff:ff:ff:ff:ff:ff 2024-09-21T03:39:16.151 INFO:teuthology.orchestra.run.smithi038.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-21T03:39:16.151 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:39:16.151 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 fe80::c49f:c1ff:fe21:b12a/64 scope link 2024-09-21T03:39:16.152 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:39:16.152 INFO:teuthology.orchestra.run.smithi038.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T03:39:16.152 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether a6:3a:83:ad:db:d5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T03:39:16.152 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 fe80::a43a:83ff:fead:dbd5/64 scope link 2024-09-21T03:39:16.152 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:39:16.152 INFO:teuthology.orchestra.run.smithi038.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T03:39:16.152 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 92:55:c7:97:6f:b5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T03:39:16.152 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 fe80::9055:c7ff:fe97:6fb5/64 scope link 2024-09-21T03:39:16.152 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:39:16.152 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:39:16.152 DEBUG:teuthology.orchestra.run.smithi038:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T03:39:16.213 INFO:teuthology.orchestra.run.smithi038.stdout:1 2024-09-21T03:39:16.215 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:39:16.215 DEBUG:teuthology.orchestra.run.smithi038:> route 2024-09-21T03:39:16.272 INFO:teuthology.orchestra.run.smithi038.stdout:Kernel IP routing table 2024-09-21T03:39:16.272 INFO:teuthology.orchestra.run.smithi038.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T03:39:16.272 INFO:teuthology.orchestra.run.smithi038.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-21T03:39:16.272 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-21T03:39:16.272 INFO:teuthology.orchestra.run.smithi038.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T03:39:16.273 DEBUG:teuthology.orchestra.run.smithi038:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T03:39:16.273 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-09-21T03:39:16.273 DEBUG:teuthology.orchestra.run.smithi038:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-21T03:39:16.273 DEBUG:teuthology.orchestra.run.smithi038:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-21T03:39:16.273 DEBUG:teuthology.orchestra.run.smithi038:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-21T03:39:16.273 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-09-21T03:39:16.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:16 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T03:39:16.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:39:16.402 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:39:16.402 DEBUG:teuthology.orchestra.run.smithi038:> ip netns list 2024-09-21T03:39:16.418 INFO:teuthology.orchestra.run.smithi038.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-21T03:39:16.418 INFO:teuthology.orchestra.run.smithi038.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-21T03:39:16.418 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T03:39:16.418 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:39:16.418 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip link set brx.0 up 2024-09-21T03:39:16.483 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T03:39:16.483 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-21T03:39:16.539 INFO:teuthology.orchestra.run.smithi038.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-21T03:39:16.539 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T03:39:16.539 DEBUG:teuthology.orchestra.run.smithi038:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-21T03:39:16.596 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-21T03:39:16.596 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:39:16.596 DEBUG:teuthology.orchestra.run.smithi038:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T03:39:16.692 INFO:teuthology.orchestra.run.smithi038.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T03:39:16.692 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T03:39:16.692 INFO:teuthology.orchestra.run.smithi038.stdout:invalid new device string format 2024-09-21T03:39:16.703 INFO:teuthology.orchestra.run.smithi038.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T03:39:16.703 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T03:39:16.703 INFO:teuthology.orchestra.run.smithi038.stdout:invalid new device string format 2024-09-21T03:39:16.703 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: resolved to: "172.21.15.38:3300,172.21.15.195:3300,172.21.15.195:3301" 2024-09-21T03:39:16.704 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T03:39:16.709 INFO:teuthology.orchestra.run.smithi038.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T03:39:16.709 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T03:39:16.709 INFO:teuthology.orchestra.run.smithi038.stdout:invalid new device string format 2024-09-21T03:39:16.709 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: resolved to: "172.21.15.38:3300,172.21.15.195:3300,172.21.15.195:3301" 2024-09-21T03:39:16.709 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T03:39:16.710 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: resolved to: "172.21.15.38:6789,172.21.15.195:6789,172.21.15.195:6790" 2024-09-21T03:39:16.710 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.38:6789,172.21.15.195:6789,172.21.15.195:6790:/ 2024-09-21T03:39:16.710 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=9d50d66f-656b-4fe2-8e3b-fe53b79c4d43" will pass to kernel 2024-09-21T03:39:16.710 INFO:teuthology.orchestra.run.smithi038.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-21T03:39:16.710 INFO:teuthology.orchestra.run.smithi038.stdout: You just mounted a file system that supports labels which does not 2024-09-21T03:39:16.710 INFO:teuthology.orchestra.run.smithi038.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-21T03:39:16.710 INFO:teuthology.orchestra.run.smithi038.stdout: applications will generate AVC messages and not be allowed access to 2024-09-21T03:39:16.710 INFO:teuthology.orchestra.run.smithi038.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-21T03:39:16.711 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T03:39:16.712 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:39:16.712 DEBUG:teuthology.orchestra.run.smithi038:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-21T03:39:16.737 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T03:39:16.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:16 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T03:39:16.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:39:16.839 INFO:teuthology.orchestra.run.smithi038.stdout:client6717 2024-09-21T03:39:16.840 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-21T03:39:16.840 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:39:16.856 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:39:17.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:17.008+0000 7efe04769640 1 -- 172.21.15.38:0/2609474606 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efdfc15cb10 msgr2=0x7efdfc15cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:17.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:17.008+0000 7efe04769640 1 --2- 172.21.15.38:0/2609474606 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efdfc15cb10 0x7efdfc15cf20 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7efdf8009830 tx=0x7efdf802a510 comp rx=0 tx=0).stop 2024-09-21T03:39:17.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:17.008+0000 7efe04769640 1 -- 172.21.15.38:0/2609474606 shutdown_connections 2024-09-21T03:39:17.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:17.008+0000 7efe04769640 1 --2- 172.21.15.38:0/2609474606 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efdfc0d12e0 0x7efdfc0d16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:17.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:17.008+0000 7efe04769640 1 --2- 172.21.15.38:0/2609474606 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efdfc15cb10 0x7efdfc15cf20 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:17.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:17.009+0000 7efe04769640 1 -- 172.21.15.38:0/2609474606 >> 172.21.15.38:0/2609474606 conn(0x7efdfc0d2350 msgr2=0x7efdfc0d2760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:17.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:17.009+0000 7efe04769640 1 -- 172.21.15.38:0/2609474606 shutdown_connections 2024-09-21T03:39:17.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:17.009+0000 7efe04769640 1 -- 172.21.15.38:0/2609474606 wait complete. 2024-09-21T03:39:17.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:17.010+0000 7efe04769640 1 Processor -- start 2024-09-21T03:39:17.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:17.010+0000 7efe04769640 1 -- start start 2024-09-21T03:39:17.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:17.012+0000 7efe04769640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efdfc0d12e0 0x7efdfc1f6440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:17.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:17.012+0000 7efe04769640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efdfc15cb10 0x7efdfc06eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:17.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:17.012+0000 7efe04769640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efdfc160100 0x7efdfc1e9390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:17.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:17.012+0000 7efe04769640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7efdfc0b73e0 con 0x7efdfc0d12e0 2024-09-21T03:39:17.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:17.012+0000 7efe04769640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7efdfc0b7260 con 0x7efdfc160100 2024-09-21T03:39:17.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:17.012+0000 7efe04769640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7efdfc160c20 con 0x7efdfc15cb10 2024-09-21T03:39:17.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:17.012+0000 7efe02505640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efdfc0d12e0 0x7efdfc1f6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:17.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:17.012+0000 7efe02505640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efdfc0d12e0 0x7efdfc1f6440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43746/0 (socket says 172.21.15.38:43746) 2024-09-21T03:39:17.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:17.012+0000 7efe02505640 1 -- 172.21.15.38:0/246378321 learned_addr learned my addr 172.21.15.38:0/246378321 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:39:17.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:17.012+0000 7efe01d04640 1 --2- 172.21.15.38:0/246378321 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efdfc15cb10 0x7efdfc06eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:17.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:17.012+0000 7efe02d06640 1 --2- 172.21.15.38:0/246378321 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efdfc160100 0x7efdfc1e9390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:17.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:17.012+0000 7efe02505640 1 -- 172.21.15.38:0/246378321 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efdfc15cb10 msgr2=0x7efdfc06eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:17.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:17.012+0000 7efe02505640 1 --2- 172.21.15.38:0/246378321 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efdfc15cb10 0x7efdfc06eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:17.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:17.012+0000 7efe02505640 1 -- 172.21.15.38:0/246378321 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efdfc160100 msgr2=0x7efdfc1e9390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:17.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:17.012+0000 7efe02505640 1 --2- 172.21.15.38:0/246378321 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efdfc160100 0x7efdfc1e9390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:17.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:17.012+0000 7efe02505640 1 -- 172.21.15.38:0/246378321 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdfc06f4d0 con 0x7efdfc0d12e0 2024-09-21T03:39:17.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:17.012+0000 7efe02505640 1 --2- 172.21.15.38:0/246378321 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efdfc0d12e0 0x7efdfc1f6440 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7efdf8009950 tx=0x7efdf8033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:17.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:17.012+0000 7efdf2ffd640 1 -- 172.21.15.38:0/246378321 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdf8031d60 con 0x7efdfc0d12e0 2024-09-21T03:39:17.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:17.013+0000 7efe04769640 1 -- 172.21.15.38:0/246378321 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdfc06f7b0 con 0x7efdfc0d12e0 2024-09-21T03:39:17.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:17.013+0000 7efdf2ffd640 1 -- 172.21.15.38:0/246378321 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efdf8033e10 con 0x7efdfc0d12e0 2024-09-21T03:39:17.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:17.013+0000 7efdf2ffd640 1 -- 172.21.15.38:0/246378321 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdf8005260 con 0x7efdfc0d12e0 2024-09-21T03:39:17.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:17.013+0000 7efe04769640 1 -- 172.21.15.38:0/246378321 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdfc06fd70 con 0x7efdfc0d12e0 2024-09-21T03:39:17.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:17.013+0000 7efe04769640 1 -- 172.21.15.38:0/246378321 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efdfc0d16f0 con 0x7efdfc0d12e0 2024-09-21T03:39:17.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:17.014+0000 7efdf2ffd640 1 -- 172.21.15.38:0/246378321 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7efdf8005400 con 0x7efdfc0d12e0 2024-09-21T03:39:17.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:17.015+0000 7efdf2ffd640 1 --2- 172.21.15.38:0/246378321 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7efde0078bc0 0x7efde007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:17.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:17.015+0000 7efdf2ffd640 1 -- 172.21.15.38:0/246378321 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 6101+0+0 (secure 0 0 0) 0x7efdf80bc620 con 0x7efdfc0d12e0 2024-09-21T03:39:17.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:17.015+0000 7efe01d04640 1 --2- 172.21.15.38:0/246378321 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7efde0078bc0 0x7efde007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:17.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:17.016+0000 7efe01d04640 1 --2- 172.21.15.38:0/246378321 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7efde0078bc0 0x7efde007b0d0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7efdec0058c0 tx=0x7efdec005850 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:17.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:17.019+0000 7efdf2ffd640 1 -- 172.21.15.38:0/246378321 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7efdf803f030 con 0x7efdfc0d12e0 2024-09-21T03:39:17.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:17.244+0000 7efe04769640 1 -- 172.21.15.38:0/246378321 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efdfc0b56b0 con 0x7efdfc0d12e0 2024-09-21T03:39:17.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:17.244+0000 7efdf2ffd640 1 -- 172.21.15.38:0/246378321 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) ==== 94+0+5231 (secure 0 0 0) 0x7efdf80848d0 con 0x7efdfc0d12e0 2024-09-21T03:39:17.247 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:39:17.247 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":67,"btime":"2024-09-21T03:39:08:424230+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5783,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/1358588311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1358588311},{"type":"v1","addr":"172.21.15.38:6835","nonce":1358588311}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63},{"gid":5789,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/1676483003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":1676483003},{"type":"v1","addr":"172.21.15.195:6837","nonce":1676483003}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63},{"gid":6193,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/2771781100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":2771781100},{"type":"v1","addr":"172.21.15.195:6836","nonce":2771781100}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:39:04.779948+0000","modified":"2024-09-21T03:39:07.810041+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6546},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6546":{"gid":6546,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":3,"addr":"172.21.15.38:6833/1144075315","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":1144075315},{"type":"v1","addr":"172.21.15.38:6833","nonce":1144075315}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6546,"qdb_cluster":[6546]},"id":5}]} 2024-09-21T03:39:17.247 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 67 2024-09-21T03:39:17.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:17.246+0000 7efe04769640 1 -- 172.21.15.38:0/246378321 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7efde0078bc0 msgr2=0x7efde007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:17.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:17.246+0000 7efe04769640 1 --2- 172.21.15.38:0/246378321 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7efde0078bc0 0x7efde007b0d0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7efdec0058c0 tx=0x7efdec005850 comp rx=0 tx=0).stop 2024-09-21T03:39:17.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:17.246+0000 7efe04769640 1 -- 172.21.15.38:0/246378321 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efdfc0d12e0 msgr2=0x7efdfc1f6440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:17.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:17.246+0000 7efe04769640 1 --2- 172.21.15.38:0/246378321 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efdfc0d12e0 0x7efdfc1f6440 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7efdf8009950 tx=0x7efdf8033c20 comp rx=0 tx=0).stop 2024-09-21T03:39:17.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:17.246+0000 7efe04769640 1 -- 172.21.15.38:0/246378321 shutdown_connections 2024-09-21T03:39:17.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:17.246+0000 7efe04769640 1 --2- 172.21.15.38:0/246378321 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efdfc160100 0x7efdfc1e9390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:17.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:17.247+0000 7efe04769640 1 --2- 172.21.15.38:0/246378321 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7efde0078bc0 0x7efde007b0d0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:17.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:17.247+0000 7efe04769640 1 --2- 172.21.15.38:0/246378321 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efdfc15cb10 0x7efdfc06eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:17.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:17.247+0000 7efe04769640 1 --2- 172.21.15.38:0/246378321 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efdfc0d12e0 0x7efdfc1f6440 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:17.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:17.247+0000 7efe04769640 1 -- 172.21.15.38:0/246378321 >> 172.21.15.38:0/246378321 conn(0x7efdfc0d2350 msgr2=0x7efdfc0d2760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:17.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:17.247+0000 7efe04769640 1 -- 172.21.15.38:0/246378321 shutdown_connections 2024-09-21T03:39:17.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:17.247+0000 7efe04769640 1 -- 172.21.15.38:0/246378321 wait complete. 2024-09-21T03:39:17.261 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds freeze 5:0 true 2024-09-21T03:39:17.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:17.376+0000 7f7d41113640 1 -- 172.21.15.38:0/4275977650 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7d3c0ceb90 msgr2=0x7f7d3c0cefa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:17.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:17.376+0000 7f7d41113640 1 --2- 172.21.15.38:0/4275977650 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7d3c0ceb90 0x7f7d3c0cefa0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f7d34009830 tx=0x7f7d3402a4a0 comp rx=0 tx=0).stop 2024-09-21T03:39:17.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:17.376+0000 7f7d41113640 1 -- 172.21.15.38:0/4275977650 shutdown_connections 2024-09-21T03:39:17.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:17.376+0000 7f7d41113640 1 --2- 172.21.15.38:0/4275977650 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7d3c0ceb90 0x7f7d3c0cefa0 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:17.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:17.376+0000 7f7d41113640 1 -- 172.21.15.38:0/4275977650 >> 172.21.15.38:0/4275977650 conn(0x7f7d3c09d830 msgr2=0x7f7d3c0b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:17.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:17.376+0000 7f7d41113640 1 -- 172.21.15.38:0/4275977650 shutdown_connections 2024-09-21T03:39:17.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:17.376+0000 7f7d41113640 1 -- 172.21.15.38:0/4275977650 wait complete. 2024-09-21T03:39:17.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:17.378+0000 7f7d41113640 1 Processor -- start 2024-09-21T03:39:17.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:17.378+0000 7f7d41113640 1 -- start start 2024-09-21T03:39:17.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:17.380+0000 7f7d41113640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7d3c0ceb90 0x7f7d3c15b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:17.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:17.380+0000 7f7d41113640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7d3c0d0170 0x7f7d3c15bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:17.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:17.380+0000 7f7d41113640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7d3c15ee80 0x7f7d3c06d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:17.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:17.380+0000 7f7d41113640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f7d3c0b6910 con 0x7f7d3c0ceb90 2024-09-21T03:39:17.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:17.380+0000 7f7d41113640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f7d3c0b6790 con 0x7f7d3c15ee80 2024-09-21T03:39:17.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:17.380+0000 7f7d41113640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f7d3c0b6a90 con 0x7f7d3c0d0170 2024-09-21T03:39:17.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:17.380+0000 7f7d3ad9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7d3c0ceb90 0x7f7d3c15b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:17.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:17.380+0000 7f7d3ad9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7d3c0ceb90 0x7f7d3c15b890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43770/0 (socket says 172.21.15.38:43770) 2024-09-21T03:39:17.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:17.380+0000 7f7d3ad9d640 1 -- 172.21.15.38:0/3078358931 learned_addr learned my addr 172.21.15.38:0/3078358931 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:39:17.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:17.380+0000 7f7d3a59c640 1 --2- 172.21.15.38:0/3078358931 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7d3c0d0170 0x7f7d3c15bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:17.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:17.380+0000 7f7d3b59e640 1 --2- 172.21.15.38:0/3078358931 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7d3c15ee80 0x7f7d3c06d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:17.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:17.380+0000 7f7d3ad9d640 1 -- 172.21.15.38:0/3078358931 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7d3c0d0170 msgr2=0x7f7d3c15bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:17.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:17.380+0000 7f7d3ad9d640 1 --2- 172.21.15.38:0/3078358931 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7d3c0d0170 0x7f7d3c15bdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:17.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:17.380+0000 7f7d3ad9d640 1 -- 172.21.15.38:0/3078358931 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7d3c15ee80 msgr2=0x7f7d3c06d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:17.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:17.380+0000 7f7d3ad9d640 1 --2- 172.21.15.38:0/3078358931 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7d3c15ee80 0x7f7d3c06d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:17.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:17.380+0000 7f7d3ad9d640 1 -- 172.21.15.38:0/3078358931 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d3c0d32e0 con 0x7f7d3c0ceb90 2024-09-21T03:39:17.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:17.380+0000 7f7d3a59c640 1 --2- 172.21.15.38:0/3078358931 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7d3c0d0170 0x7f7d3c15bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:39:17.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:17.380+0000 7f7d3ad9d640 1 --2- 172.21.15.38:0/3078358931 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7d3c0ceb90 0x7f7d3c15b890 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f7d2c00c780 tx=0x7f7d2c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:17.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:17.380+0000 7f7d237fe640 1 -- 172.21.15.38:0/3078358931 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d2c003b70 con 0x7f7d3c0ceb90 2024-09-21T03:39:17.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:17.381+0000 7f7d237fe640 1 -- 172.21.15.38:0/3078358931 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7d2c003d10 con 0x7f7d3c0ceb90 2024-09-21T03:39:17.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:17.381+0000 7f7d41113640 1 -- 172.21.15.38:0/3078358931 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d3c0d35c0 con 0x7f7d3c0ceb90 2024-09-21T03:39:17.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:17.381+0000 7f7d237fe640 1 -- 172.21.15.38:0/3078358931 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d2c00aab0 con 0x7f7d3c0ceb90 2024-09-21T03:39:17.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:17.381+0000 7f7d41113640 1 -- 172.21.15.38:0/3078358931 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d3c0d3b80 con 0x7f7d3c0ceb90 2024-09-21T03:39:17.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:17.382+0000 7f7d41113640 1 -- 172.21.15.38:0/3078358931 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d3c0b5e80 con 0x7f7d3c0ceb90 2024-09-21T03:39:17.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:17.383+0000 7f7d237fe640 1 -- 172.21.15.38:0/3078358931 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f7d2c00ac50 con 0x7f7d3c0ceb90 2024-09-21T03:39:17.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:17.383+0000 7f7d237fe640 1 --2- 172.21.15.38:0/3078358931 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7d28078bc0 0x7f7d2807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:17.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:17.383+0000 7f7d237fe640 1 -- 172.21.15.38:0/3078358931 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 6101+0+0 (secure 0 0 0) 0x7f7d2c09ad60 con 0x7f7d3c0ceb90 2024-09-21T03:39:17.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:17.384+0000 7f7d3a59c640 1 --2- 172.21.15.38:0/3078358931 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7d28078bc0 0x7f7d2807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:17.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:17.384+0000 7f7d3a59c640 1 --2- 172.21.15.38:0/3078358931 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7d28078bc0 0x7f7d2807b0d0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f7d3c06fde0 tx=0x7f7d30009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:17.391 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:17.388+0000 7f7d237fe640 1 -- 172.21.15.38:0/3078358931 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f7d3c0b5e80 con 0x7f7d3c0ceb90 2024-09-21T03:39:17.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:17.655+0000 7f7d41113640 1 -- 172.21.15.38:0/3078358931 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "mds freeze", "role_or_gid": "5:0", "val": "true"} v 0) -- 0x7f7d3c0cefa0 con 0x7f7d3c0ceb90 2024-09-21T03:39:18.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.472+0000 7f7d237fe640 1 -- 172.21.15.38:0/3078358931 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "mds freeze", "role_or_gid": "5:0", "val": "true"}]=0 freezing mds.6546 v68) ==== 112+0+0 (secure 0 0 0) 0x7f7d3c0cf4e0 con 0x7f7d3c0ceb90 2024-09-21T03:39:18.479 INFO:teuthology.orchestra.run.smithi038.stderr:freezing mds.6546 2024-09-21T03:39:18.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.475+0000 7f7d41113640 1 -- 172.21.15.38:0/3078358931 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7d28078bc0 msgr2=0x7f7d2807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:18.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.475+0000 7f7d41113640 1 --2- 172.21.15.38:0/3078358931 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7d28078bc0 0x7f7d2807b0d0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f7d3c06fde0 tx=0x7f7d30009040 comp rx=0 tx=0).stop 2024-09-21T03:39:18.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.475+0000 7f7d41113640 1 -- 172.21.15.38:0/3078358931 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7d3c0ceb90 msgr2=0x7f7d3c15b890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:18.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.475+0000 7f7d41113640 1 --2- 172.21.15.38:0/3078358931 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7d3c0ceb90 0x7f7d3c15b890 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f7d2c00c780 tx=0x7f7d2c00cc40 comp rx=0 tx=0).stop 2024-09-21T03:39:18.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.475+0000 7f7d41113640 1 -- 172.21.15.38:0/3078358931 shutdown_connections 2024-09-21T03:39:18.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.475+0000 7f7d41113640 1 --2- 172.21.15.38:0/3078358931 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7d3c15ee80 0x7f7d3c06d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:18.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.475+0000 7f7d41113640 1 --2- 172.21.15.38:0/3078358931 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7d28078bc0 0x7f7d2807b0d0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:18.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.475+0000 7f7d41113640 1 --2- 172.21.15.38:0/3078358931 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7d3c0d0170 0x7f7d3c15bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:18.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.475+0000 7f7d41113640 1 --2- 172.21.15.38:0/3078358931 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7d3c0ceb90 0x7f7d3c15b890 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:18.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.475+0000 7f7d41113640 1 -- 172.21.15.38:0/3078358931 >> 172.21.15.38:0/3078358931 conn(0x7f7d3c09d830 msgr2=0x7f7d3c0b3240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:18.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.476+0000 7f7d41113640 1 -- 172.21.15.38:0/3078358931 shutdown_connections 2024-09-21T03:39:18.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.476+0000 7f7d41113640 1 -- 172.21.15.38:0/3078358931 wait complete. 2024-09-21T03:39:18.493 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'mkdir -p a/{b,c}/d') 2024-09-21T03:39:18.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T03:39:18.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:39:18.576 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.5:0 flush journal 2024-09-21T03:39:18.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.696+0000 7fba004c4640 1 -- 172.21.15.38:0/831365049 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb9e4003a00 msgr2=0x7fb9e4003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:18.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.696+0000 7fba004c4640 1 --2- 172.21.15.38:0/831365049 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb9e4003a00 0x7fb9e4003e50 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7fb9f400c7a0 tx=0x7fb9e4005830 comp rx=0 tx=0).stop 2024-09-21T03:39:18.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.696+0000 7fba004c4640 1 -- 172.21.15.38:0/831365049 shutdown_connections 2024-09-21T03:39:18.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.696+0000 7fba004c4640 1 --2- 172.21.15.38:0/831365049 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb9e4003a00 0x7fb9e4003e50 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:18.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.696+0000 7fba004c4640 1 -- 172.21.15.38:0/831365049 >> 172.21.15.38:0/831365049 conn(0x7fb9f809d830 msgr2=0x7fb9f80b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:18.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.696+0000 7fba004c4640 1 -- 172.21.15.38:0/831365049 shutdown_connections 2024-09-21T03:39:18.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.696+0000 7fba004c4640 1 -- 172.21.15.38:0/831365049 wait complete. 2024-09-21T03:39:18.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.698+0000 7fba004c4640 1 Processor -- start 2024-09-21T03:39:18.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.698+0000 7fba004c4640 1 -- start start 2024-09-21T03:39:18.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.699+0000 7fba004c4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb9e4003a00 0x7fb9f8157a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:18.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.699+0000 7fba004c4640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb9f80ce230 0x7fb9f80b2810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:18.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.699+0000 7fba004c4640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb9f80b2d50 0x7fb9f81f8840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:18.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.699+0000 7fba004c4640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fb9f80b9c50 con 0x7fb9e4003a00 2024-09-21T03:39:18.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.699+0000 7fba004c4640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fb9f80b9ad0 con 0x7fb9f80b2d50 2024-09-21T03:39:18.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.699+0000 7fba004c4640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fb9f80b9dd0 con 0x7fb9f80ce230 2024-09-21T03:39:18.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.699+0000 7fb9fe260640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb9e4003a00 0x7fb9f8157a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:18.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.699+0000 7fb9fe260640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb9e4003a00 0x7fb9f8157a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43786/0 (socket says 172.21.15.38:43786) 2024-09-21T03:39:18.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.699+0000 7fb9fe260640 1 -- 172.21.15.38:0/2432798494 learned_addr learned my addr 172.21.15.38:0/2432798494 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:39:18.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.699+0000 7fb9fda5f640 1 --2- 172.21.15.38:0/2432798494 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb9f80ce230 0x7fb9f80b2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:18.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.699+0000 7fb9fea61640 1 --2- 172.21.15.38:0/2432798494 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb9f80b2d50 0x7fb9f81f8840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:18.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.699+0000 7fb9fe260640 1 -- 172.21.15.38:0/2432798494 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb9f80ce230 msgr2=0x7fb9f80b2810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:18.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.699+0000 7fb9fe260640 1 --2- 172.21.15.38:0/2432798494 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb9f80ce230 0x7fb9f80b2810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:18.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.699+0000 7fb9fe260640 1 -- 172.21.15.38:0/2432798494 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb9f80b2d50 msgr2=0x7fb9f81f8840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:18.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.699+0000 7fb9fe260640 1 --2- 172.21.15.38:0/2432798494 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb9f80b2d50 0x7fb9f81f8840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:18.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.699+0000 7fb9fe260640 1 -- 172.21.15.38:0/2432798494 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9f80b3160 con 0x7fb9e4003a00 2024-09-21T03:39:18.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.699+0000 7fb9fea61640 1 --2- 172.21.15.38:0/2432798494 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb9f80b2d50 0x7fb9f81f8840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:39:18.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.700+0000 7fb9fe260640 1 --2- 172.21.15.38:0/2432798494 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb9e4003a00 0x7fb9f8157a60 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fb9f402b680 tx=0x7fb9f4001350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:18.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.700+0000 7fb9eeffd640 1 -- 172.21.15.38:0/2432798494 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9f4003970 con 0x7fb9e4003a00 2024-09-21T03:39:18.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.700+0000 7fb9eeffd640 1 -- 172.21.15.38:0/2432798494 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb9f4003b10 con 0x7fb9e4003a00 2024-09-21T03:39:18.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.700+0000 7fba004c4640 1 -- 172.21.15.38:0/2432798494 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9f8157fa0 con 0x7fb9e4003a00 2024-09-21T03:39:18.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.700+0000 7fb9eeffd640 1 -- 172.21.15.38:0/2432798494 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9f40060c0 con 0x7fb9e4003a00 2024-09-21T03:39:18.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.700+0000 7fba004c4640 1 -- 172.21.15.38:0/2432798494 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9f806d310 con 0x7fb9e4003a00 2024-09-21T03:39:18.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.702+0000 7fb9eeffd640 1 -- 172.21.15.38:0/2432798494 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb9f4006260 con 0x7fb9e4003a00 2024-09-21T03:39:18.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.702+0000 7fb9eeffd640 1 --2- 172.21.15.38:0/2432798494 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb9dc078bc0 0x7fb9dc07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:18.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.702+0000 7fb9eeffd640 1 -- 172.21.15.38:0/2432798494 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 6101+0+0 (secure 0 0 0) 0x7fb9f40bb5b0 con 0x7fb9e4003a00 2024-09-21T03:39:18.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.703+0000 7fb9fda5f640 1 --2- 172.21.15.38:0/2432798494 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb9dc078bc0 0x7fb9dc07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:18.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.703+0000 7fba07430740 1 Processor -- start 2024-09-21T03:39:18.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.703+0000 7fb9fda5f640 1 --2- 172.21.15.38:0/2432798494 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb9dc078bc0 0x7fb9dc07b0d0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fb9f815c990 tx=0x7fb9e800b260 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:18.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.703+0000 7fba07430740 1 -- start start 2024-09-21T03:39:18.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.705+0000 7fba07430740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c188cde390 0x55c1890caa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:18.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.705+0000 7fba07430740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55c188cb54d0 0x55c18909f860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:18.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.705+0000 7fba07430740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c188cbd6b0 0x55c18909fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:18.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.705+0000 7fba07430740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55c188ff8a60 con 0x55c188cbd6b0 2024-09-21T03:39:18.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.705+0000 7fba07430740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55c188fe7e90 con 0x55c188cde390 2024-09-21T03:39:18.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.705+0000 7fba07430740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55c188febca0 con 0x55c188cb54d0 2024-09-21T03:39:18.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.705+0000 7fb9fea61640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c188cbd6b0 0x55c18909fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:18.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.705+0000 7fb9fea61640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c188cbd6b0 0x55c18909fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43794/0 (socket says 172.21.15.38:43794) 2024-09-21T03:39:18.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.705+0000 7fb9fea61640 1 -- 172.21.15.38:0/1359911484 learned_addr learned my addr 172.21.15.38:0/1359911484 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:39:18.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.705+0000 7fb9fda5f640 1 --2- 172.21.15.38:0/1359911484 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55c188cb54d0 0x55c18909f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:18.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.705+0000 7fb9fe260640 1 --2- 172.21.15.38:0/1359911484 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c188cde390 0x55c1890caa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:18.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.705+0000 7fb9fea61640 1 -- 172.21.15.38:0/1359911484 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55c188cb54d0 msgr2=0x55c18909f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:18.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.705+0000 7fb9fea61640 1 --2- 172.21.15.38:0/1359911484 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55c188cb54d0 0x55c18909f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:18.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.705+0000 7fb9fea61640 1 -- 172.21.15.38:0/1359911484 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c188cde390 msgr2=0x55c1890caa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:18.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.705+0000 7fb9fea61640 1 --2- 172.21.15.38:0/1359911484 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c188cde390 0x55c1890caa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:18.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.705+0000 7fb9fea61640 1 -- 172.21.15.38:0/1359911484 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c188dc07d0 con 0x55c188cbd6b0 2024-09-21T03:39:18.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.705+0000 7fb9fe260640 1 --2- 172.21.15.38:0/1359911484 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c188cde390 0x55c1890caa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:39:18.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.705+0000 7fb9fea61640 1 --2- 172.21.15.38:0/1359911484 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c188cbd6b0 0x55c18909fda0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fb9f815c9c0 tx=0x7fb9f0003c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f31c3963378a4cc5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:18.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.706+0000 7fb9dbfff640 1 -- 172.21.15.38:0/1359911484 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9f000f8b0 con 0x55c188cbd6b0 2024-09-21T03:39:18.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.706+0000 7fb9dbfff640 1 -- 172.21.15.38:0/1359911484 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb9f0018070 con 0x55c188cbd6b0 2024-09-21T03:39:18.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.706+0000 7fb9dbfff640 1 -- 172.21.15.38:0/1359911484 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9f000fcd0 con 0x55c188cbd6b0 2024-09-21T03:39:18.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.706+0000 7fba07430740 1 -- 172.21.15.38:0/1359911484 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c188cbd6b0 msgr2=0x55c18909fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:18.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.706+0000 7fba07430740 1 --2- 172.21.15.38:0/1359911484 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c188cbd6b0 0x55c18909fda0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fb9f815c9c0 tx=0x7fb9f0003c00 comp rx=0 tx=0).stop 2024-09-21T03:39:18.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.706+0000 7fba07430740 1 -- 172.21.15.38:0/1359911484 shutdown_connections 2024-09-21T03:39:18.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.706+0000 7fba07430740 1 --2- 172.21.15.38:0/1359911484 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c188cbd6b0 0x55c18909fda0 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:18.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.706+0000 7fba07430740 1 --2- 172.21.15.38:0/1359911484 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55c188cb54d0 0x55c18909f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:18.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.706+0000 7fba07430740 1 --2- 172.21.15.38:0/1359911484 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c188cde390 0x55c1890caa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:18.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.706+0000 7fba07430740 1 -- 172.21.15.38:0/1359911484 >> 172.21.15.38:0/1359911484 conn(0x55c188cd24e0 msgr2=0x55c189085d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:18.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.707+0000 7fba07430740 1 -- 172.21.15.38:0/1359911484 shutdown_connections 2024-09-21T03:39:18.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.707+0000 7fba07430740 1 -- 172.21.15.38:0/1359911484 wait complete. 2024-09-21T03:39:18.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.707+0000 7fba07430740 1 -- start start 2024-09-21T03:39:18.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.708+0000 7fba07430740 1 Processor -- start 2024-09-21T03:39:18.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.709+0000 7fba07430740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55c188cb54d0 0x55c189087a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:18.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.709+0000 7fba07430740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c188cbd6b0 0x55c1890840f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:18.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.709+0000 7fba07430740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c188cde390 0x55c1890d3590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:18.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.709+0000 7fba07430740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55c188d58740 con 0x55c188cde390 2024-09-21T03:39:18.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.709+0000 7fba07430740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55c188f8d970 con 0x55c188cbd6b0 2024-09-21T03:39:18.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.709+0000 7fba07430740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55c1890882d0 con 0x55c188cb54d0 2024-09-21T03:39:18.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.709+0000 7fba07430740 10 client.0 _ll_register_callbacks cb 0x55c188f9c780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:39:18.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.709+0000 7fb9fea61640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c188cde390 0x55c1890d3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:18.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.709+0000 7fb9fea61640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c188cde390 0x55c1890d3590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43796/0 (socket says 172.21.15.38:43796) 2024-09-21T03:39:18.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.709+0000 7fb9fea61640 1 -- 172.21.15.38:0/1530913669 learned_addr learned my addr 172.21.15.38:0/1530913669 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:39:18.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.709+0000 7fb9fe260640 1 --2- 172.21.15.38:0/1530913669 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55c188cb54d0 0x55c189087a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:18.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.709+0000 7fb9fda5f640 1 --2- 172.21.15.38:0/1530913669 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c188cbd6b0 0x55c1890840f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:18.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.709+0000 7fb9fea61640 1 -- 172.21.15.38:0/1530913669 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55c188cb54d0 msgr2=0x55c189087a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:18.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.709+0000 7fb9fea61640 1 --2- 172.21.15.38:0/1530913669 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55c188cb54d0 0x55c189087a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:18.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.709+0000 7fb9fea61640 1 -- 172.21.15.38:0/1530913669 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c188cbd6b0 msgr2=0x55c1890840f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:18.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.709+0000 7fb9fea61640 1 --2- 172.21.15.38:0/1530913669 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c188cbd6b0 0x55c1890840f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:18.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.710+0000 7fb9fea61640 1 -- 172.21.15.38:0/1530913669 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c188d49970 con 0x55c188cde390 2024-09-21T03:39:18.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.710+0000 7fb9fea61640 1 --2- 172.21.15.38:0/1530913669 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c188cde390 0x55c1890d3590 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fb9f000f720 tx=0x7fb9f00063a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a17571cdc5499b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:18.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.710+0000 7fb9d97fa640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-21T03:39:18.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.710+0000 7fb9d97fa640 1 -- 172.21.15.38:0/1530913669 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9f0012a10 con 0x55c188cde390 2024-09-21T03:39:18.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.710+0000 7fb9d97fa640 1 -- 172.21.15.38:0/1530913669 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 6101+0+0 (secure 0 0 0) 0x7fb9f0012bb0 con 0x55c188cde390 2024-09-21T03:39:18.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.710+0000 7fba07430740 1 -- 172.21.15.38:0/1530913669 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb9f0018070 con 0x55c188cde390 2024-09-21T03:39:18.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.710+0000 7fb9d97fa640 1 -- 172.21.15.38:0/1530913669 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) ==== 24+0+0 (secure 0 0 0) 0x7fb9f002f050 con 0x55c188cde390 2024-09-21T03:39:18.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.710+0000 7fba07430740 10 client.6747 fetch_fsmap learned FSMap version 68 2024-09-21T03:39:18.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.710+0000 7fba07430740 1 -- 172.21.15.38:0/1530913669 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=2+}) -- 0x55c188d276a0 con 0x55c188cde390 2024-09-21T03:39:18.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.710+0000 7fb9d97fa640 1 -- 172.21.15.38:0/1530913669 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb9f0018070 con 0x55c188cde390 2024-09-21T03:39:18.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.710+0000 7fb9d97fa640 1 -- 172.21.15.38:0/1530913669 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 68) ==== 3502+0+0 (secure 0 0 0) 0x7fb9f0032020 con 0x55c188cde390 2024-09-21T03:39:18.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.710+0000 7fba07430740 10 client.6747 fetch_fsmap finished waiting for FSMap version 68 2024-09-21T03:39:18.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.710+0000 7fba07430740 10 client.6747 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-09-21T03:39:18.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.710+0000 7fba07430740 20 client.6747 populate_metadata read hostname 'smithi038' 2024-09-21T03:39:18.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.710+0000 7fba07430740 1 --2- 172.21.15.38:0/1530913669 >> [v2:172.21.15.38:6832/1144075315,v1:172.21.15.38:6833/1144075315] conn(0x55c188c9cf60 0x55c1890ad620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:18.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.710+0000 7fba07430740 4 client.6747 mds_command: new command op to 6546 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:39:18.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.710+0000 7fba07430740 1 -- 172.21.15.38:0/1530913669 --> [v2:172.21.15.38:6832/1144075315,v1:172.21.15.38:6833/1144075315] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c188ffa9b0 con 0x55c188c9cf60 2024-09-21T03:39:18.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.711+0000 7fb9fe260640 1 --2- 172.21.15.38:0/1530913669 >> [v2:172.21.15.38:6832/1144075315,v1:172.21.15.38:6833/1144075315] conn(0x55c188c9cf60 0x55c1890ad620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:18.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.711+0000 7fb9d97fa640 10 client.6747 ms_handle_connect on v2:172.21.15.38:6832/1144075315 2024-09-21T03:39:18.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.711+0000 7fb9fe260640 1 --2- 172.21.15.38:0/1530913669 >> [v2:172.21.15.38:6832/1144075315,v1:172.21.15.38:6833/1144075315] conn(0x55c188c9cf60 0x55c1890ad620 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=49982688a1b44f0b server_cookie=d0e3622f3cfc16fd in_seq=0 out_seq=0 2024-09-21T03:39:18.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.712+0000 7fb9d97fa640 1 -- 172.21.15.38:0/1530913669 <== mds.0 v2:172.21.15.38:6832/1144075315 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c188ffa9b0 con 0x55c188c9cf60 2024-09-21T03:39:18.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.712+0000 7fb9d97fa640 10 client.6747 handle_command_reply: tid=0 2024-09-21T03:39:18.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.712+0000 7fba07430740 1 client.6747 shutdown 2024-09-21T03:39:18.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.712+0000 7fba07430740 1 -- 172.21.15.38:0/1530913669 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c188cde390 msgr2=0x55c1890d3590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:18.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.712+0000 7fba07430740 1 --2- 172.21.15.38:0/1530913669 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c188cde390 0x55c1890d3590 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fb9f000f720 tx=0x7fb9f00063a0 comp rx=0 tx=0).stop 2024-09-21T03:39:18.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.712+0000 7fba07430740 1 -- 172.21.15.38:0/1530913669 shutdown_connections 2024-09-21T03:39:18.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.712+0000 7fba07430740 1 --2- 172.21.15.38:0/1530913669 >> [v2:172.21.15.38:6832/1144075315,v1:172.21.15.38:6833/1144075315] conn(0x55c188c9cf60 0x55c1890ad620 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:18.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.712+0000 7fba07430740 1 --2- 172.21.15.38:0/1530913669 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c188cde390 0x55c1890d3590 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:18.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.712+0000 7fba07430740 1 --2- 172.21.15.38:0/1530913669 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c188cbd6b0 0x55c1890840f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:18.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.712+0000 7fb9d97fa640 0 client.6747 ms_handle_reset on v2:172.21.15.38:6832/1144075315 2024-09-21T03:39:18.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.712+0000 7fba07430740 1 --2- 172.21.15.38:0/1530913669 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55c188cb54d0 0x55c189087a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:18.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.712+0000 7fba07430740 1 -- 172.21.15.38:0/1530913669 >> 172.21.15.38:0/1530913669 conn(0x55c188cd24e0 msgr2=0x55c18909f860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:18.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.712+0000 7fba07430740 1 -- 172.21.15.38:0/1530913669 shutdown_connections 2024-09-21T03:39:18.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.712+0000 7fba07430740 1 -- 172.21.15.38:0/1530913669 wait complete. 2024-09-21T03:39:18.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.712+0000 7fba07430740 20 client.6747 trim_cache size 0 max 16384 2024-09-21T03:39:18.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.727+0000 7fba07430740 1 Processor -- start 2024-09-21T03:39:18.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.727+0000 7fba07430740 1 -- start start 2024-09-21T03:39:18.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.728+0000 7fba07430740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55c188cb54d0 0x55c1890c6f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:18.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.728+0000 7fba07430740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c188cbd6b0 0x55c1890c74a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:18.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.729+0000 7fba07430740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c188cde390 0x55c18907edd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:18.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.729+0000 7fba07430740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fb9f8202bb0 con 0x55c188cbd6b0 2024-09-21T03:39:18.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.729+0000 7fba07430740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fb9f8201bb0 con 0x55c188cde390 2024-09-21T03:39:18.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.729+0000 7fba07430740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fb9f8200f40 con 0x55c188cb54d0 2024-09-21T03:39:18.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.729+0000 7fb9fda5f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c188cbd6b0 0x55c1890c74a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:18.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.729+0000 7fb9fda5f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c188cbd6b0 0x55c1890c74a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43808/0 (socket says 172.21.15.38:43808) 2024-09-21T03:39:18.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.729+0000 7fb9fda5f640 1 -- 172.21.15.38:0/180653996 learned_addr learned my addr 172.21.15.38:0/180653996 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:39:18.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.729+0000 7fb9fea61640 1 --2- 172.21.15.38:0/180653996 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c188cde390 0x55c18907edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:18.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.729+0000 7fb9fe260640 1 --2- 172.21.15.38:0/180653996 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55c188cb54d0 0x55c1890c6f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:18.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.729+0000 7fb9fda5f640 1 -- 172.21.15.38:0/180653996 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55c188cb54d0 msgr2=0x55c1890c6f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:18.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.729+0000 7fb9fda5f640 1 --2- 172.21.15.38:0/180653996 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55c188cb54d0 0x55c1890c6f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:18.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.729+0000 7fb9fda5f640 1 -- 172.21.15.38:0/180653996 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c188cde390 msgr2=0x55c18907edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:18.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.729+0000 7fb9fda5f640 1 --2- 172.21.15.38:0/180653996 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c188cde390 0x55c18907edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:18.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.729+0000 7fb9fda5f640 1 -- 172.21.15.38:0/180653996 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c189084460 con 0x55c188cbd6b0 2024-09-21T03:39:18.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.729+0000 7fb9fea61640 1 --2- 172.21.15.38:0/180653996 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c188cde390 0x55c18907edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:39:18.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.729+0000 7fb9fda5f640 1 --2- 172.21.15.38:0/180653996 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c188cbd6b0 0x55c1890c74a0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fb9f808d980 tx=0x7fb9e8014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e373d2580434494 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:18.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.729+0000 7fb9db7fe640 1 -- 172.21.15.38:0/180653996 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9e8012aa0 con 0x55c188cbd6b0 2024-09-21T03:39:18.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.729+0000 7fb9db7fe640 1 -- 172.21.15.38:0/180653996 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb9e8012c40 con 0x55c188cbd6b0 2024-09-21T03:39:18.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.729+0000 7fb9db7fe640 1 -- 172.21.15.38:0/180653996 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9e8020430 con 0x55c188cbd6b0 2024-09-21T03:39:18.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.730+0000 7fba07430740 1 -- 172.21.15.38:0/180653996 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c188cbd6b0 msgr2=0x55c1890c74a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:18.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.730+0000 7fba07430740 1 --2- 172.21.15.38:0/180653996 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c188cbd6b0 0x55c1890c74a0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fb9f808d980 tx=0x7fb9e8014000 comp rx=0 tx=0).stop 2024-09-21T03:39:18.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.730+0000 7fba07430740 1 -- 172.21.15.38:0/180653996 shutdown_connections 2024-09-21T03:39:18.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.730+0000 7fba07430740 1 --2- 172.21.15.38:0/180653996 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c188cde390 0x55c18907edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:18.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.730+0000 7fba07430740 1 --2- 172.21.15.38:0/180653996 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c188cbd6b0 0x55c1890c74a0 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:18.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.730+0000 7fba07430740 1 --2- 172.21.15.38:0/180653996 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55c188cb54d0 0x55c1890c6f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:18.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.730+0000 7fba07430740 1 -- 172.21.15.38:0/180653996 >> 172.21.15.38:0/180653996 conn(0x55c188cd24e0 msgr2=0x55c1890878c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:18.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.730+0000 7fba07430740 1 -- 172.21.15.38:0/180653996 shutdown_connections 2024-09-21T03:39:18.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.730+0000 7fba07430740 1 -- 172.21.15.38:0/180653996 wait complete. 2024-09-21T03:39:18.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.731+0000 7fba07430740 1 -- start start 2024-09-21T03:39:18.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.731+0000 7fba07430740 1 Processor -- start 2024-09-21T03:39:18.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.732+0000 7fba07430740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55c188cb54d0 0x55c1891433a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:18.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.733+0000 7fba07430740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c188cbd6b0 0x55c1890c6f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:18.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.733+0000 7fba07430740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c188cde390 0x55c1890c74a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:18.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.733+0000 7fba07430740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55c1890d9340 con 0x55c188cbd6b0 2024-09-21T03:39:18.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.733+0000 7fba07430740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fb9f8201560 con 0x55c188cde390 2024-09-21T03:39:18.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.733+0000 7fba07430740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55c189149ba0 con 0x55c188cb54d0 2024-09-21T03:39:18.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.733+0000 7fba07430740 10 client.0 _ll_register_callbacks cb 0x7fb9f0006560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:39:18.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.733+0000 7fb9fda5f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c188cbd6b0 0x55c1890c6f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:18.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.733+0000 7fb9fda5f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c188cbd6b0 0x55c1890c6f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43820/0 (socket says 172.21.15.38:43820) 2024-09-21T03:39:18.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.733+0000 7fb9fda5f640 1 -- 172.21.15.38:0/2357496657 learned_addr learned my addr 172.21.15.38:0/2357496657 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:39:18.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.733+0000 7fb9fe260640 1 --2- 172.21.15.38:0/2357496657 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55c188cb54d0 0x55c1891433a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:18.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.733+0000 7fb9fea61640 1 --2- 172.21.15.38:0/2357496657 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c188cde390 0x55c1890c74a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:18.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.733+0000 7fb9fda5f640 1 -- 172.21.15.38:0/2357496657 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55c188cb54d0 msgr2=0x55c1891433a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:18.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.733+0000 7fb9fda5f640 1 --2- 172.21.15.38:0/2357496657 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55c188cb54d0 0x55c1891433a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:18.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.733+0000 7fb9fda5f640 1 -- 172.21.15.38:0/2357496657 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c188cde390 msgr2=0x55c1890c74a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:18.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.733+0000 7fb9fda5f640 1 --2- 172.21.15.38:0/2357496657 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c188cde390 0x55c1890c74a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:18.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.733+0000 7fb9fda5f640 1 -- 172.21.15.38:0/2357496657 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c1891438e0 con 0x55c188cbd6b0 2024-09-21T03:39:18.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.733+0000 7fb9fda5f640 1 --2- 172.21.15.38:0/2357496657 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c188cbd6b0 0x55c1890c6f60 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fb9f80d28f0 tx=0x7fb9e802b6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=254043f5275aad54 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:18.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.733+0000 7fb9d97fa640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-21T03:39:18.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.733+0000 7fb9d97fa640 1 -- 172.21.15.38:0/2357496657 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9e8036020 con 0x55c188cbd6b0 2024-09-21T03:39:18.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.733+0000 7fb9d97fa640 1 -- 172.21.15.38:0/2357496657 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 6101+0+0 (secure 0 0 0) 0x7fb9e802f070 con 0x55c188cbd6b0 2024-09-21T03:39:18.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.733+0000 7fba07430740 1 -- 172.21.15.38:0/2357496657 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb9e8012c40 con 0x55c188cbd6b0 2024-09-21T03:39:18.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.734+0000 7fb9d97fa640 1 -- 172.21.15.38:0/2357496657 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) ==== 24+0+0 (secure 0 0 0) 0x7fb9e803d050 con 0x55c188cbd6b0 2024-09-21T03:39:18.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.734+0000 7fba07430740 10 client.6753 fetch_fsmap learned FSMap version 68 2024-09-21T03:39:18.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.734+0000 7fba07430740 1 -- 172.21.15.38:0/2357496657 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=2+}) -- 0x55c1890d3630 con 0x55c188cbd6b0 2024-09-21T03:39:18.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.734+0000 7fb9d97fa640 1 -- 172.21.15.38:0/2357496657 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb9e8012c40 con 0x55c188cbd6b0 2024-09-21T03:39:18.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.734+0000 7fb9d97fa640 1 -- 172.21.15.38:0/2357496657 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 68) ==== 3502+0+0 (secure 0 0 0) 0x7fb9e8040020 con 0x55c188cbd6b0 2024-09-21T03:39:18.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.734+0000 7fba07430740 10 client.6753 fetch_fsmap finished waiting for FSMap version 68 2024-09-21T03:39:18.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.734+0000 7fba07430740 10 client.6753 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-09-21T03:39:18.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.734+0000 7fba07430740 20 client.6753 populate_metadata read hostname 'smithi038' 2024-09-21T03:39:18.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.734+0000 7fba07430740 1 --2- 172.21.15.38:0/2357496657 >> [v2:172.21.15.38:6832/1144075315,v1:172.21.15.38:6833/1144075315] conn(0x55c18907b790 0x55c18914ce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:18.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.734+0000 7fba07430740 4 client.6753 mds_command: new command op to 6546 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T03:39:18.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.734+0000 7fba07430740 1 -- 172.21.15.38:0/2357496657 --> [v2:172.21.15.38:6832/1144075315,v1:172.21.15.38:6833/1144075315] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c188ff6460 con 0x55c18907b790 2024-09-21T03:39:18.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.734+0000 7fb9fe260640 1 --2- 172.21.15.38:0/2357496657 >> [v2:172.21.15.38:6832/1144075315,v1:172.21.15.38:6833/1144075315] conn(0x55c18907b790 0x55c18914ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:18.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.735+0000 7fb9d97fa640 10 client.6753 ms_handle_connect on v2:172.21.15.38:6832/1144075315 2024-09-21T03:39:18.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.735+0000 7fb9fe260640 1 --2- 172.21.15.38:0/2357496657 >> [v2:172.21.15.38:6832/1144075315,v1:172.21.15.38:6833/1144075315] conn(0x55c18907b790 0x55c18914ce30 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=112b85c33c080847 server_cookie=98174d4e634debbe in_seq=0 out_seq=0 2024-09-21T03:39:18.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.757+0000 7fb9d97fa640 1 -- 172.21.15.38:0/2357496657 <== mds.0 v2:172.21.15.38:6832/1144075315 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c188ff6460 con 0x55c18907b790 2024-09-21T03:39:18.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.757+0000 7fb9d97fa640 10 client.6753 handle_command_reply: tid=0 2024-09-21T03:39:18.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.757+0000 7fba07430740 1 client.6753 shutdown 2024-09-21T03:39:18.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.757+0000 7fba07430740 1 -- 172.21.15.38:0/2357496657 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c188cbd6b0 msgr2=0x55c1890c6f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:18.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.757+0000 7fba07430740 1 --2- 172.21.15.38:0/2357496657 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c188cbd6b0 0x55c1890c6f60 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fb9f80d28f0 tx=0x7fb9e802b6e0 comp rx=0 tx=0).stop 2024-09-21T03:39:18.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.757+0000 7fba07430740 1 -- 172.21.15.38:0/2357496657 shutdown_connections 2024-09-21T03:39:18.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.757+0000 7fba07430740 1 --2- 172.21.15.38:0/2357496657 >> [v2:172.21.15.38:6832/1144075315,v1:172.21.15.38:6833/1144075315] conn(0x55c18907b790 0x55c18914ce30 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:18.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.757+0000 7fba07430740 1 --2- 172.21.15.38:0/2357496657 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c188cde390 0x55c1890c74a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:18.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.757+0000 7fba07430740 1 --2- 172.21.15.38:0/2357496657 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c188cbd6b0 0x55c1890c6f60 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:18.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.757+0000 7fb9d97fa640 0 client.6753 ms_handle_reset on v2:172.21.15.38:6832/1144075315 2024-09-21T03:39:18.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.757+0000 7fba07430740 1 --2- 172.21.15.38:0/2357496657 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55c188cb54d0 0x55c1891433a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:18.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.757+0000 7fba07430740 1 -- 172.21.15.38:0/2357496657 >> 172.21.15.38:0/2357496657 conn(0x55c188cd24e0 msgr2=0x55c1890878c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:18.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.757+0000 7fba07430740 1 -- 172.21.15.38:0/2357496657 shutdown_connections 2024-09-21T03:39:18.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.757+0000 7fba07430740 1 -- 172.21.15.38:0/2357496657 wait complete. 2024-09-21T03:39:18.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.757+0000 7fba07430740 20 client.6753 trim_cache size 0 max 16384 2024-09-21T03:39:18.762 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-09-21T03:39:18.762 INFO:teuthology.orchestra.run.smithi038.stdout: "message": "", 2024-09-21T03:39:18.762 INFO:teuthology.orchestra.run.smithi038.stdout: "return_code": 0 2024-09-21T03:39:18.762 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-09-21T03:39:18.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.758+0000 7fba004c4640 1 -- 172.21.15.38:0/2432798494 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb9dc078bc0 msgr2=0x7fb9dc07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:18.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.758+0000 7fba004c4640 1 --2- 172.21.15.38:0/2432798494 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb9dc078bc0 0x7fb9dc07b0d0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fb9f815c990 tx=0x7fb9e800b260 comp rx=0 tx=0).stop 2024-09-21T03:39:18.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.758+0000 7fba004c4640 1 -- 172.21.15.38:0/2432798494 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb9e4003a00 msgr2=0x7fb9f8157a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:18.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.758+0000 7fba004c4640 1 --2- 172.21.15.38:0/2432798494 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb9e4003a00 0x7fb9f8157a60 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fb9f402b680 tx=0x7fb9f4001350 comp rx=0 tx=0).stop 2024-09-21T03:39:18.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.758+0000 7fba004c4640 1 -- 172.21.15.38:0/2432798494 shutdown_connections 2024-09-21T03:39:18.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.758+0000 7fba004c4640 1 --2- 172.21.15.38:0/2432798494 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb9f80b2d50 0x7fb9f81f8840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:18.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.758+0000 7fba004c4640 1 --2- 172.21.15.38:0/2432798494 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb9dc078bc0 0x7fb9dc07b0d0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:18.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.758+0000 7fba004c4640 1 --2- 172.21.15.38:0/2432798494 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb9f80ce230 0x7fb9f80b2810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:18.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.758+0000 7fba004c4640 1 --2- 172.21.15.38:0/2432798494 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb9e4003a00 0x7fb9f8157a60 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:18.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.758+0000 7fba004c4640 1 -- 172.21.15.38:0/2432798494 >> 172.21.15.38:0/2432798494 conn(0x7fb9f809d830 msgr2=0x7fb9f81589a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:18.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.758+0000 7fba004c4640 1 -- 172.21.15.38:0/2432798494 shutdown_connections 2024-09-21T03:39:18.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.758+0000 7fba004c4640 1 -- 172.21.15.38:0/2432798494 wait complete. 2024-09-21T03:39:18.773 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_inject_rename_corrupt_dentry_first 1.0 2024-09-21T03:39:18.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.890+0000 7efc477a8640 1 -- 172.21.15.38:0/1674984936 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efc2c003a00 msgr2=0x7efc2c003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:18.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.890+0000 7efc477a8640 1 --2- 172.21.15.38:0/1674984936 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efc2c003a00 0x7efc2c003e70 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7efc3c00c4a0 tx=0x7efc2c005830 comp rx=0 tx=0).stop 2024-09-21T03:39:18.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.890+0000 7efc45d45640 1 -- 172.21.15.38:0/1674984936 reap_dead start 2024-09-21T03:39:18.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.890+0000 7efc477a8640 1 -- 172.21.15.38:0/1674984936 shutdown_connections 2024-09-21T03:39:18.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.890+0000 7efc477a8640 1 -- 172.21.15.38:0/1674984936 >> 172.21.15.38:0/1674984936 conn(0x7efc400cd460 msgr2=0x7efc400cd870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:18.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.890+0000 7efc477a8640 1 -- 172.21.15.38:0/1674984936 shutdown_connections 2024-09-21T03:39:18.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.890+0000 7efc477a8640 1 -- 172.21.15.38:0/1674984936 wait complete. 2024-09-21T03:39:18.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.892+0000 7efc477a8640 1 Processor -- start 2024-09-21T03:39:18.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.892+0000 7efc477a8640 1 -- start start 2024-09-21T03:39:18.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.893+0000 7efc477a8640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efc2c003a00 0x7efc40146b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:18.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.893+0000 7efc477a8640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efc4015a960 0x7efc401f6080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:18.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.893+0000 7efc477a8640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efc401f65c0 0x7efc401543b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:18.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.893+0000 7efc477a8640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7efc40073c70 con 0x7efc4015a960 2024-09-21T03:39:18.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.893+0000 7efc477a8640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7efc40073af0 con 0x7efc401f65c0 2024-09-21T03:39:18.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.893+0000 7efc477a8640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7efc40073df0 con 0x7efc2c003a00 2024-09-21T03:39:18.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.893+0000 7efc44d43640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efc4015a960 0x7efc401f6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:18.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.894+0000 7efc44d43640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efc4015a960 0x7efc401f6080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43828/0 (socket says 172.21.15.38:43828) 2024-09-21T03:39:18.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.894+0000 7efc44d43640 1 -- 172.21.15.38:0/1512848935 learned_addr learned my addr 172.21.15.38:0/1512848935 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:39:18.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.894+0000 7efc45d45640 1 --2- 172.21.15.38:0/1512848935 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efc401f65c0 0x7efc401543b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:18.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.894+0000 7efc45544640 1 --2- 172.21.15.38:0/1512848935 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efc2c003a00 0x7efc40146b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:18.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.894+0000 7efc44d43640 1 -- 172.21.15.38:0/1512848935 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efc2c003a00 msgr2=0x7efc40146b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:18.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.894+0000 7efc44d43640 1 --2- 172.21.15.38:0/1512848935 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efc2c003a00 0x7efc40146b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:18.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.894+0000 7efc44d43640 1 -- 172.21.15.38:0/1512848935 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efc401f65c0 msgr2=0x7efc401543b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:18.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.894+0000 7efc44d43640 1 --2- 172.21.15.38:0/1512848935 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efc401f65c0 0x7efc401543b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:18.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.894+0000 7efc44d43640 1 -- 172.21.15.38:0/1512848935 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc401f69d0 con 0x7efc4015a960 2024-09-21T03:39:18.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.894+0000 7efc44d43640 1 --2- 172.21.15.38:0/1512848935 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efc4015a960 0x7efc401f6080 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7efc30011880 tx=0x7efc30011d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:18.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.894+0000 7efc35ffb640 1 -- 172.21.15.38:0/1512848935 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc30004460 con 0x7efc4015a960 2024-09-21T03:39:18.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.894+0000 7efc35ffb640 1 -- 172.21.15.38:0/1512848935 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efc30016070 con 0x7efc4015a960 2024-09-21T03:39:18.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.894+0000 7efc477a8640 1 -- 172.21.15.38:0/1512848935 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc401495e0 con 0x7efc4015a960 2024-09-21T03:39:18.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.894+0000 7efc35ffb640 1 -- 172.21.15.38:0/1512848935 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc3000fd30 con 0x7efc4015a960 2024-09-21T03:39:18.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.895+0000 7efc477a8640 1 -- 172.21.15.38:0/1512848935 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc40149ba0 con 0x7efc4015a960 2024-09-21T03:39:18.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.895+0000 7efc477a8640 1 -- 172.21.15.38:0/1512848935 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc400b4580 con 0x7efc4015a960 2024-09-21T03:39:18.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.896+0000 7efc35ffb640 1 -- 172.21.15.38:0/1512848935 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7efc30018030 con 0x7efc4015a960 2024-09-21T03:39:18.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.897+0000 7efc35ffb640 1 --2- 172.21.15.38:0/1512848935 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7efc24078b90 0x7efc2407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:18.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.897+0000 7efc35ffb640 1 -- 172.21.15.38:0/1512848935 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 6101+0+0 (secure 0 0 0) 0x7efc3009bec0 con 0x7efc4015a960 2024-09-21T03:39:18.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.897+0000 7efc45544640 1 --2- 172.21.15.38:0/1512848935 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7efc24078b90 0x7efc2407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:18.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.898+0000 7efc45544640 1 --2- 172.21.15.38:0/1512848935 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7efc24078b90 0x7efc2407b0a0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7efc3c033000 tx=0x7efc3c038040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:18.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:18.901+0000 7efc35ffb640 1 -- 172.21.15.38:0/1512848935 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7efc401f69d0 con 0x7efc4015a960 2024-09-21T03:39:19.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:19.090+0000 7efc477a8640 1 -- 172.21.15.38:0/1512848935 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command([{prefix=config set, name=mds_inject_rename_corrupt_dentry_first}] v 0) -- 0x7efc4006e460 con 0x7efc4015a960 2024-09-21T03:39:19.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:19.094+0000 7efc35ffb640 1 -- 172.21.15.38:0/1512848935 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_inject_rename_corrupt_dentry_first}]=0 v44)=0 v44) ==== 138+0+0 (secure 0 0 0) 0x7efc30063b90 con 0x7efc4015a960 2024-09-21T03:39:19.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:19.096+0000 7efc477a8640 1 -- 172.21.15.38:0/1512848935 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7efc24078b90 msgr2=0x7efc2407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:19.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:19.096+0000 7efc477a8640 1 --2- 172.21.15.38:0/1512848935 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7efc24078b90 0x7efc2407b0a0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7efc3c033000 tx=0x7efc3c038040 comp rx=0 tx=0).stop 2024-09-21T03:39:19.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:19.096+0000 7efc477a8640 1 -- 172.21.15.38:0/1512848935 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efc4015a960 msgr2=0x7efc401f6080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:19.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:19.096+0000 7efc477a8640 1 --2- 172.21.15.38:0/1512848935 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efc4015a960 0x7efc401f6080 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7efc30011880 tx=0x7efc30011d40 comp rx=0 tx=0).stop 2024-09-21T03:39:19.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:19.096+0000 7efc477a8640 1 -- 172.21.15.38:0/1512848935 shutdown_connections 2024-09-21T03:39:19.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:19.096+0000 7efc477a8640 1 --2- 172.21.15.38:0/1512848935 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efc401f65c0 0x7efc401543b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:19.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:19.096+0000 7efc477a8640 1 --2- 172.21.15.38:0/1512848935 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efc4015a960 0x7efc401f6080 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:19.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:19.096+0000 7efc477a8640 1 --2- 172.21.15.38:0/1512848935 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7efc24078b90 0x7efc2407b0a0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:19.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:19.096+0000 7efc477a8640 1 --2- 172.21.15.38:0/1512848935 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efc2c003a00 0x7efc40146b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:19.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:19.096+0000 7efc477a8640 1 -- 172.21.15.38:0/1512848935 >> 172.21.15.38:0/1512848935 conn(0x7efc400cd460 msgr2=0x7efc40151840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:19.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:19.096+0000 7efc477a8640 1 -- 172.21.15.38:0/1512848935 shutdown_connections 2024-09-21T03:39:19.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:19.096+0000 7efc477a8640 1 -- 172.21.15.38:0/1512848935 wait complete. 2024-09-21T03:39:24.112 DEBUG:teuthology.orchestra.run.smithi038:> sudo daemon-helper kill ceph --cluster ceph -w 2024-09-21T03:39:24.115 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'timeout 60 mv a/b a/z') 2024-09-21T03:39:24.118 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok --format=json config get mds_beacon_grace 2024-09-21T03:39:24.180 INFO:tasks.ceph.mds.c.smithi038.stderr:2024-09-21T03:39:24.178+0000 7f6ea9433640 -1 mds.0.cache.den(0x10000000000 z) newly corrupt dentry to be committed: [dentry #0x1/a/z [fffffffffffffff6,head] auth NULL (dn xlock x=1 by request(client.6717:47 nref=7 cr=0x556587681500)) (dversion lock w=1 last_client=6717) pv=11 v=10 ap=2 ino=(nil) state=1073741824 | request=1 lock=2 authpin=1 0x5565874e1680] with next_snap: 2 2024-09-21T03:39:24.180 INFO:tasks.ceph.mds.c.smithi038.stderr:2024-09-21T03:39:24.178+0000 7f6ea9433640 -1 log_channel(cluster) log [ERR] : MDS abort because newly corrupt dentry to be committed: [dentry #0x1/a/z [fffffffffffffff6,head] auth NULL (dn xlock x=1 by request(client.6717:47 nref=7 cr=0x556587681500)) (dversion lock w=1 last_client=6717) pv=11 v=10 ap=2 ino=(nil) state=1073741824 | request=1 lock=2 authpin=1 0x5565874e1680] 2024-09-21T03:39:24.181 INFO:tasks.ceph.mds.c.smithi038.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5120-g18432218/rpm/el9/BUILD/ceph-19.3.0-5120-g18432218/src/mds/MDSRank.cc: In function 'void MDSRank::abort(std::string_view)' thread 7f6ea9433640 time 2024-09-21T03:39:24.179706+0000 2024-09-21T03:39:24.181 INFO:tasks.ceph.mds.c.smithi038.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5120-g18432218/rpm/el9/BUILD/ceph-19.3.0-5120-g18432218/src/mds/MDSRank.cc: 913: ceph_abort_msg("abort() called") 2024-09-21T03:39:24.181 INFO:tasks.ceph.mds.c.smithi038.stderr: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T03:39:24.181 INFO:tasks.ceph.mds.c.smithi038.stderr: 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xcf) [0x7f6eaf257214] 2024-09-21T03:39:24.181 INFO:tasks.ceph.mds.c.smithi038.stderr: 2: (MDSRank::abort(std::basic_string_view >)+0x5f) [0x55657fa22159] 2024-09-21T03:39:24.182 INFO:tasks.ceph.mds.c.smithi038.stderr: 3: (CDentry::check_corruption(bool)+0x54e) [0x55657fd123ca] 2024-09-21T03:39:24.182 INFO:tasks.ceph.mds.c.smithi038.stderr: 4: ceph-mds(+0x175d76) [0x55657fa8ad76] 2024-09-21T03:39:24.182 INFO:tasks.ceph.mds.c.smithi038.stderr: 5: ceph-mds(+0x176176) [0x55657fa8b176] 2024-09-21T03:39:24.182 INFO:tasks.ceph.mds.c.smithi038.stderr: 6: (Server::_rename_prepare(boost::intrusive_ptr const&, EMetaBlob*, ceph::buffer::v15_2_0::list*, CDentry*, CDentry*, std::basic_string_view >, CDentry*)+0x2571) [0x55657fb30afd] 2024-09-21T03:39:24.182 INFO:tasks.ceph.mds.c.smithi038.stderr: 7: (Server::handle_client_rename(boost::intrusive_ptr const&)+0x2ec1) [0x55657fb342f1] 2024-09-21T03:39:24.182 INFO:tasks.ceph.mds.c.smithi038.stderr: 8: (Server::dispatch_client_request(boost::intrusive_ptr const&)+0xd27) [0x55657faf8be5] 2024-09-21T03:39:24.182 INFO:tasks.ceph.mds.c.smithi038.stderr: 9: (Server::handle_client_request(boost::intrusive_ptr const&)+0xa2a) [0x55657faf9892] 2024-09-21T03:39:24.182 INFO:tasks.ceph.mds.c.smithi038.stderr: 10: (Server::dispatch(boost::intrusive_ptr const&)+0xc0) [0x55657fae03cc] 2024-09-21T03:39:24.182 INFO:tasks.ceph.mds.c.smithi038.stderr: 11: (MDSRank::handle_message(boost::intrusive_ptr const&)+0x5ac) [0x55657fa2c6ea] 2024-09-21T03:39:24.182 INFO:tasks.ceph.mds.c.smithi038.stderr: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x28f) [0x55657fa30523] 2024-09-21T03:39:24.182 INFO:tasks.ceph.mds.c.smithi038.stderr: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x94) [0x55657fa30e4a] 2024-09-21T03:39:24.182 INFO:tasks.ceph.mds.c.smithi038.stderr: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x3f0) [0x55657fa13fa4] 2024-09-21T03:39:24.182 INFO:tasks.ceph.mds.c.smithi038.stderr: 15: /usr/lib64/ceph/libceph-common.so.2(+0x3a57df) [0x7f6eaf3a57df] 2024-09-21T03:39:24.183 INFO:tasks.ceph.mds.c.smithi038.stderr: 16: (DispatchQueue::entry()+0x60f) [0x7f6eaf3a60d1] 2024-09-21T03:39:24.183 INFO:tasks.ceph.mds.c.smithi038.stderr: 17: /usr/lib64/ceph/libceph-common.so.2(+0x493f5b) [0x7f6eaf493f5b] 2024-09-21T03:39:24.183 INFO:tasks.ceph.mds.c.smithi038.stderr: 18: (Thread::entry_wrapper()+0x43) [0x7f6eaf22fd57] 2024-09-21T03:39:24.183 INFO:tasks.ceph.mds.c.smithi038.stderr: 19: (Thread::_entry_func(void*)+0xd) [0x7f6eaf22fd73] 2024-09-21T03:39:24.183 INFO:tasks.ceph.mds.c.smithi038.stderr: 20: /lib64/libc.so.6(+0x89c52) [0x7f6eae889c52] 2024-09-21T03:39:24.183 INFO:tasks.ceph.mds.c.smithi038.stderr: 21: /lib64/libc.so.6(+0x10ec80) [0x7f6eae90ec80] 2024-09-21T03:39:24.183 INFO:tasks.ceph.mds.c.smithi038.stderr:*** Caught signal (Aborted) ** 2024-09-21T03:39:24.183 INFO:tasks.ceph.mds.c.smithi038.stderr: in thread 7f6ea9433640 thread_name:ms_dispatch 2024-09-21T03:39:24.183 INFO:tasks.ceph.mds.c.smithi038.stderr:2024-09-21T03:39:24.179+0000 7f6ea9433640 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5120-g18432218/rpm/el9/BUILD/ceph-19.3.0-5120-g18432218/src/mds/MDSRank.cc: In function 'void MDSRank::abort(std::string_view)' thread 7f6ea9433640 time 2024-09-21T03:39:24.179706+0000 2024-09-21T03:39:24.183 INFO:tasks.ceph.mds.c.smithi038.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5120-g18432218/rpm/el9/BUILD/ceph-19.3.0-5120-g18432218/src/mds/MDSRank.cc: 913: ceph_abort_msg("abort() called") 2024-09-21T03:39:24.183 INFO:tasks.ceph.mds.c.smithi038.stderr: 2024-09-21T03:39:24.183 INFO:tasks.ceph.mds.c.smithi038.stderr: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T03:39:24.183 INFO:tasks.ceph.mds.c.smithi038.stderr: 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xcf) [0x7f6eaf257214] 2024-09-21T03:39:24.184 INFO:tasks.ceph.mds.c.smithi038.stderr: 2: (MDSRank::abort(std::basic_string_view >)+0x5f) [0x55657fa22159] 2024-09-21T03:39:24.184 INFO:tasks.ceph.mds.c.smithi038.stderr: 3: (CDentry::check_corruption(bool)+0x54e) [0x55657fd123ca] 2024-09-21T03:39:24.184 INFO:tasks.ceph.mds.c.smithi038.stderr: 4: ceph-mds(+0x175d76) [0x55657fa8ad76] 2024-09-21T03:39:24.184 INFO:tasks.ceph.mds.c.smithi038.stderr: 5: ceph-mds(+0x176176) [0x55657fa8b176] 2024-09-21T03:39:24.184 INFO:tasks.ceph.mds.c.smithi038.stderr: 6: (Server::_rename_prepare(boost::intrusive_ptr const&, EMetaBlob*, ceph::buffer::v15_2_0::list*, CDentry*, CDentry*, std::basic_string_view >, CDentry*)+0x2571) [0x55657fb30afd] 2024-09-21T03:39:24.184 INFO:tasks.ceph.mds.c.smithi038.stderr: 7: (Server::handle_client_rename(boost::intrusive_ptr const&)+0x2ec1) [0x55657fb342f1] 2024-09-21T03:39:24.184 INFO:tasks.ceph.mds.c.smithi038.stderr: 8: (Server::dispatch_client_request(boost::intrusive_ptr const&)+0xd27) [0x55657faf8be5] 2024-09-21T03:39:24.184 INFO:tasks.ceph.mds.c.smithi038.stderr: 9: (Server::handle_client_request(boost::intrusive_ptr const&)+0xa2a) [0x55657faf9892] 2024-09-21T03:39:24.184 INFO:tasks.ceph.mds.c.smithi038.stderr: 10: (Server::dispatch(boost::intrusive_ptr const&)+0xc0) [0x55657fae03cc] 2024-09-21T03:39:24.184 INFO:tasks.ceph.mds.c.smithi038.stderr: 11: (MDSRank::handle_message(boost::intrusive_ptr const&)+0x5ac) [0x55657fa2c6ea] 2024-09-21T03:39:24.184 INFO:tasks.ceph.mds.c.smithi038.stderr: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x28f) [0x55657fa30523] 2024-09-21T03:39:24.184 INFO:tasks.ceph.mds.c.smithi038.stderr: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x94) [0x55657fa30e4a] 2024-09-21T03:39:24.185 INFO:tasks.ceph.mds.c.smithi038.stderr: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x3f0) [0x55657fa13fa4] 2024-09-21T03:39:24.185 INFO:tasks.ceph.mds.c.smithi038.stderr: 15: /usr/lib64/ceph/libceph-common.so.2(+0x3a57df) [0x7f6eaf3a57df] 2024-09-21T03:39:24.185 INFO:tasks.ceph.mds.c.smithi038.stderr: 16: (DispatchQueue::entry()+0x60f) [0x7f6eaf3a60d1] 2024-09-21T03:39:24.185 INFO:tasks.ceph.mds.c.smithi038.stderr: 17: /usr/lib64/ceph/libceph-common.so.2(+0x493f5b) [0x7f6eaf493f5b] 2024-09-21T03:39:24.185 INFO:tasks.ceph.mds.c.smithi038.stderr: 18: (Thread::entry_wrapper()+0x43) [0x7f6eaf22fd57] 2024-09-21T03:39:24.185 INFO:tasks.ceph.mds.c.smithi038.stderr: 19: (Thread::_entry_func(void*)+0xd) [0x7f6eaf22fd73] 2024-09-21T03:39:24.185 INFO:tasks.ceph.mds.c.smithi038.stderr: 20: /lib64/libc.so.6(+0x89c52) [0x7f6eae889c52] 2024-09-21T03:39:24.185 INFO:tasks.ceph.mds.c.smithi038.stderr: 21: /lib64/libc.so.6(+0x10ec80) [0x7f6eae90ec80] 2024-09-21T03:39:24.185 INFO:tasks.ceph.mds.c.smithi038.stderr: 2024-09-21T03:39:24.186 INFO:tasks.ceph.mds.c.smithi038.stderr: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T03:39:24.186 INFO:tasks.ceph.mds.c.smithi038.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7f6eae83e6f0] 2024-09-21T03:39:24.186 INFO:tasks.ceph.mds.c.smithi038.stderr: 2: /lib64/libc.so.6(+0x8b99c) [0x7f6eae88b99c] 2024-09-21T03:39:24.186 INFO:tasks.ceph.mds.c.smithi038.stderr: 3: raise() 2024-09-21T03:39:24.186 INFO:tasks.ceph.mds.c.smithi038.stderr: 4: abort() 2024-09-21T03:39:24.186 INFO:tasks.ceph.mds.c.smithi038.stderr: 5: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0x241) [0x7f6eaf257386] 2024-09-21T03:39:24.186 INFO:tasks.ceph.mds.c.smithi038.stderr: 6: (MDSRank::abort(std::basic_string_view >)+0x5f) [0x55657fa22159] 2024-09-21T03:39:24.186 INFO:tasks.ceph.mds.c.smithi038.stderr: 7: (CDentry::check_corruption(bool)+0x54e) [0x55657fd123ca] 2024-09-21T03:39:24.186 INFO:tasks.ceph.mds.c.smithi038.stderr: 8: ceph-mds(+0x175d76) [0x55657fa8ad76] 2024-09-21T03:39:24.186 INFO:tasks.ceph.mds.c.smithi038.stderr: 9: ceph-mds(+0x176176) [0x55657fa8b176] 2024-09-21T03:39:24.186 INFO:tasks.ceph.mds.c.smithi038.stderr: 10: (Server::_rename_prepare(boost::intrusive_ptr const&, EMetaBlob*, ceph::buffer::v15_2_0::list*, CDentry*, CDentry*, std::basic_string_view >, CDentry*)+0x2571) [0x55657fb30afd] 2024-09-21T03:39:24.186 INFO:tasks.ceph.mds.c.smithi038.stderr: 11: (Server::handle_client_rename(boost::intrusive_ptr const&)+0x2ec1) [0x55657fb342f1] 2024-09-21T03:39:24.187 INFO:tasks.ceph.mds.c.smithi038.stderr: 12: (Server::dispatch_client_request(boost::intrusive_ptr const&)+0xd27) [0x55657faf8be5] 2024-09-21T03:39:24.187 INFO:tasks.ceph.mds.c.smithi038.stderr: 13: (Server::handle_client_request(boost::intrusive_ptr const&)+0xa2a) [0x55657faf9892] 2024-09-21T03:39:24.187 INFO:tasks.ceph.mds.c.smithi038.stderr: 14: (Server::dispatch(boost::intrusive_ptr const&)+0xc0) [0x55657fae03cc] 2024-09-21T03:39:24.187 INFO:tasks.ceph.mds.c.smithi038.stderr: 15: (MDSRank::handle_message(boost::intrusive_ptr const&)+0x5ac) [0x55657fa2c6ea] 2024-09-21T03:39:24.187 INFO:tasks.ceph.mds.c.smithi038.stderr: 16: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x28f) [0x55657fa30523] 2024-09-21T03:39:24.187 INFO:tasks.ceph.mds.c.smithi038.stderr: 17: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x94) [0x55657fa30e4a] 2024-09-21T03:39:24.187 INFO:tasks.ceph.mds.c.smithi038.stderr: 18: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x3f0) [0x55657fa13fa4] 2024-09-21T03:39:24.187 INFO:tasks.ceph.mds.c.smithi038.stderr: 19: /usr/lib64/ceph/libceph-common.so.2(+0x3a57df) [0x7f6eaf3a57df] 2024-09-21T03:39:24.187 INFO:tasks.ceph.mds.c.smithi038.stderr: 20: (DispatchQueue::entry()+0x60f) [0x7f6eaf3a60d1] 2024-09-21T03:39:24.187 INFO:tasks.ceph.mds.c.smithi038.stderr: 21: /usr/lib64/ceph/libceph-common.so.2(+0x493f5b) [0x7f6eaf493f5b] 2024-09-21T03:39:24.187 INFO:tasks.ceph.mds.c.smithi038.stderr: 22: (Thread::entry_wrapper()+0x43) [0x7f6eaf22fd57] 2024-09-21T03:39:24.187 INFO:tasks.ceph.mds.c.smithi038.stderr: 23: (Thread::_entry_func(void*)+0xd) [0x7f6eaf22fd73] 2024-09-21T03:39:24.187 INFO:tasks.ceph.mds.c.smithi038.stderr: 24: /lib64/libc.so.6(+0x89c52) [0x7f6eae889c52] 2024-09-21T03:39:24.188 INFO:tasks.ceph.mds.c.smithi038.stderr: 25: /lib64/libc.so.6(+0x10ec80) [0x7f6eae90ec80] 2024-09-21T03:39:24.188 INFO:tasks.ceph.mds.c.smithi038.stderr:2024-09-21T03:39:24.181+0000 7f6ea9433640 -1 *** Caught signal (Aborted) ** 2024-09-21T03:39:24.188 INFO:tasks.ceph.mds.c.smithi038.stderr: in thread 7f6ea9433640 thread_name:ms_dispatch 2024-09-21T03:39:24.188 INFO:tasks.ceph.mds.c.smithi038.stderr: 2024-09-21T03:39:24.188 INFO:tasks.ceph.mds.c.smithi038.stderr: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T03:39:24.188 INFO:tasks.ceph.mds.c.smithi038.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7f6eae83e6f0] 2024-09-21T03:39:24.188 INFO:tasks.ceph.mds.c.smithi038.stderr: 2: /lib64/libc.so.6(+0x8b99c) [0x7f6eae88b99c] 2024-09-21T03:39:24.188 INFO:tasks.ceph.mds.c.smithi038.stderr: 3: raise() 2024-09-21T03:39:24.188 INFO:tasks.ceph.mds.c.smithi038.stderr: 4: abort() 2024-09-21T03:39:24.188 INFO:tasks.ceph.mds.c.smithi038.stderr: 5: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0x241) [0x7f6eaf257386] 2024-09-21T03:39:24.188 INFO:tasks.ceph.mds.c.smithi038.stderr: 6: (MDSRank::abort(std::basic_string_view >)+0x5f) [0x55657fa22159] 2024-09-21T03:39:24.188 INFO:tasks.ceph.mds.c.smithi038.stderr: 7: (CDentry::check_corruption(bool)+0x54e) [0x55657fd123ca] 2024-09-21T03:39:24.189 INFO:tasks.ceph.mds.c.smithi038.stderr: 8: ceph-mds(+0x175d76) [0x55657fa8ad76] 2024-09-21T03:39:24.189 INFO:tasks.ceph.mds.c.smithi038.stderr: 9: ceph-mds(+0x176176) [0x55657fa8b176] 2024-09-21T03:39:24.189 INFO:tasks.ceph.mds.c.smithi038.stderr: 10: (Server::_rename_prepare(boost::intrusive_ptr const&, EMetaBlob*, ceph::buffer::v15_2_0::list*, CDentry*, CDentry*, std::basic_string_view >, CDentry*)+0x2571) [0x55657fb30afd] 2024-09-21T03:39:24.189 INFO:tasks.ceph.mds.c.smithi038.stderr: 11: (Server::handle_client_rename(boost::intrusive_ptr const&)+0x2ec1) [0x55657fb342f1] 2024-09-21T03:39:24.189 INFO:tasks.ceph.mds.c.smithi038.stderr: 12: (Server::dispatch_client_request(boost::intrusive_ptr const&)+0xd27) [0x55657faf8be5] 2024-09-21T03:39:24.189 INFO:tasks.ceph.mds.c.smithi038.stderr: 13: (Server::handle_client_request(boost::intrusive_ptr const&)+0xa2a) [0x55657faf9892] 2024-09-21T03:39:24.189 INFO:tasks.ceph.mds.c.smithi038.stderr: 14: (Server::dispatch(boost::intrusive_ptr const&)+0xc0) [0x55657fae03cc] 2024-09-21T03:39:24.189 INFO:tasks.ceph.mds.c.smithi038.stderr: 15: (MDSRank::handle_message(boost::intrusive_ptr const&)+0x5ac) [0x55657fa2c6ea] 2024-09-21T03:39:24.189 INFO:tasks.ceph.mds.c.smithi038.stderr: 16: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x28f) [0x55657fa30523] 2024-09-21T03:39:24.189 INFO:tasks.ceph.mds.c.smithi038.stderr: 17: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x94) [0x55657fa30e4a] 2024-09-21T03:39:24.189 INFO:tasks.ceph.mds.c.smithi038.stderr: 18: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x3f0) [0x55657fa13fa4] 2024-09-21T03:39:24.189 INFO:tasks.ceph.mds.c.smithi038.stderr: 19: /usr/lib64/ceph/libceph-common.so.2(+0x3a57df) [0x7f6eaf3a57df] 2024-09-21T03:39:24.189 INFO:tasks.ceph.mds.c.smithi038.stderr: 20: (DispatchQueue::entry()+0x60f) [0x7f6eaf3a60d1] 2024-09-21T03:39:24.189 INFO:tasks.ceph.mds.c.smithi038.stderr: 21: /usr/lib64/ceph/libceph-common.so.2(+0x493f5b) [0x7f6eaf493f5b] 2024-09-21T03:39:24.190 INFO:tasks.ceph.mds.c.smithi038.stderr: 22: (Thread::entry_wrapper()+0x43) [0x7f6eaf22fd57] 2024-09-21T03:39:24.190 INFO:tasks.ceph.mds.c.smithi038.stderr: 23: (Thread::_entry_func(void*)+0xd) [0x7f6eaf22fd73] 2024-09-21T03:39:24.190 INFO:tasks.ceph.mds.c.smithi038.stderr: 24: /lib64/libc.so.6(+0x89c52) [0x7f6eae889c52] 2024-09-21T03:39:24.190 INFO:tasks.ceph.mds.c.smithi038.stderr: 25: /lib64/libc.so.6(+0x10ec80) [0x7f6eae90ec80] 2024-09-21T03:39:24.190 INFO:tasks.ceph.mds.c.smithi038.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-09-21T03:39:24.190 INFO:tasks.ceph.mds.c.smithi038.stderr: 2024-09-21T03:39:24.190 INFO:tasks.ceph.mds.c.smithi038.stderr: -5258> 2024-09-21T03:39:06.377+0000 7f6ea6c2e640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T03:39:24.194 INFO:tasks.ceph.mds.c.smithi038.stderr: -9> 2024-09-21T03:39:24.178+0000 7f6ea9433640 -1 mds.0.cache.den(0x10000000000 z) newly corrupt dentry to be committed: [dentry #0x1/a/z [fffffffffffffff6,head] auth NULL (dn xlock x=1 by request(client.6717:47 nref=7 cr=0x556587681500)) (dversion lock w=1 last_client=6717) pv=11 v=10 ap=2 ino=(nil) state=1073741824 | request=1 lock=2 authpin=1 0x5565874e1680] with next_snap: 2 2024-09-21T03:39:24.194 INFO:tasks.ceph.mds.c.smithi038.stderr: -7> 2024-09-21T03:39:24.178+0000 7f6ea9433640 -1 log_channel(cluster) log [ERR] : MDS abort because newly corrupt dentry to be committed: [dentry #0x1/a/z [fffffffffffffff6,head] auth NULL (dn xlock x=1 by request(client.6717:47 nref=7 cr=0x556587681500)) (dversion lock w=1 last_client=6717) pv=11 v=10 ap=2 ino=(nil) state=1073741824 | request=1 lock=2 authpin=1 0x5565874e1680] 2024-09-21T03:39:24.194 INFO:tasks.ceph.mds.c.smithi038.stderr: -1> 2024-09-21T03:39:24.179+0000 7f6ea9433640 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5120-g18432218/rpm/el9/BUILD/ceph-19.3.0-5120-g18432218/src/mds/MDSRank.cc: In function 'void MDSRank::abort(std::string_view)' thread 7f6ea9433640 time 2024-09-21T03:39:24.179706+0000 2024-09-21T03:39:24.194 INFO:tasks.ceph.mds.c.smithi038.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5120-g18432218/rpm/el9/BUILD/ceph-19.3.0-5120-g18432218/src/mds/MDSRank.cc: 913: ceph_abort_msg("abort() called") 2024-09-21T03:39:24.194 INFO:tasks.ceph.mds.c.smithi038.stderr: 2024-09-21T03:39:24.195 INFO:tasks.ceph.mds.c.smithi038.stderr: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T03:39:24.195 INFO:tasks.ceph.mds.c.smithi038.stderr: 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xcf) [0x7f6eaf257214] 2024-09-21T03:39:24.195 INFO:tasks.ceph.mds.c.smithi038.stderr: 2: (MDSRank::abort(std::basic_string_view >)+0x5f) [0x55657fa22159] 2024-09-21T03:39:24.195 INFO:tasks.ceph.mds.c.smithi038.stderr: 3: (CDentry::check_corruption(bool)+0x54e) [0x55657fd123ca] 2024-09-21T03:39:24.195 INFO:tasks.ceph.mds.c.smithi038.stderr: 4: ceph-mds(+0x175d76) [0x55657fa8ad76] 2024-09-21T03:39:24.195 INFO:tasks.ceph.mds.c.smithi038.stderr: 5: ceph-mds(+0x176176) [0x55657fa8b176] 2024-09-21T03:39:24.195 INFO:tasks.ceph.mds.c.smithi038.stderr: 6: (Server::_rename_prepare(boost::intrusive_ptr const&, EMetaBlob*, ceph::buffer::v15_2_0::list*, CDentry*, CDentry*, std::basic_string_view >, CDentry*)+0x2571) [0x55657fb30afd] 2024-09-21T03:39:24.195 INFO:tasks.ceph.mds.c.smithi038.stderr: 7: (Server::handle_client_rename(boost::intrusive_ptr const&)+0x2ec1) [0x55657fb342f1] 2024-09-21T03:39:24.195 INFO:tasks.ceph.mds.c.smithi038.stderr: 8: (Server::dispatch_client_request(boost::intrusive_ptr const&)+0xd27) [0x55657faf8be5] 2024-09-21T03:39:24.195 INFO:tasks.ceph.mds.c.smithi038.stderr: 9: (Server::handle_client_request(boost::intrusive_ptr const&)+0xa2a) [0x55657faf9892] 2024-09-21T03:39:24.195 INFO:tasks.ceph.mds.c.smithi038.stderr: 10: (Server::dispatch(boost::intrusive_ptr const&)+0xc0) [0x55657fae03cc] 2024-09-21T03:39:24.195 INFO:tasks.ceph.mds.c.smithi038.stderr: 11: (MDSRank::handle_message(boost::intrusive_ptr const&)+0x5ac) [0x55657fa2c6ea] 2024-09-21T03:39:24.196 INFO:tasks.ceph.mds.c.smithi038.stderr: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x28f) [0x55657fa30523] 2024-09-21T03:39:24.196 INFO:tasks.ceph.mds.c.smithi038.stderr: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x94) [0x55657fa30e4a] 2024-09-21T03:39:24.196 INFO:tasks.ceph.mds.c.smithi038.stderr: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x3f0) [0x55657fa13fa4] 2024-09-21T03:39:24.196 INFO:tasks.ceph.mds.c.smithi038.stderr: 15: /usr/lib64/ceph/libceph-common.so.2(+0x3a57df) [0x7f6eaf3a57df] 2024-09-21T03:39:24.196 INFO:tasks.ceph.mds.c.smithi038.stderr: 16: (DispatchQueue::entry()+0x60f) [0x7f6eaf3a60d1] 2024-09-21T03:39:24.196 INFO:tasks.ceph.mds.c.smithi038.stderr: 17: /usr/lib64/ceph/libceph-common.so.2(+0x493f5b) [0x7f6eaf493f5b] 2024-09-21T03:39:24.196 INFO:tasks.ceph.mds.c.smithi038.stderr: 18: (Thread::entry_wrapper()+0x43) [0x7f6eaf22fd57] 2024-09-21T03:39:24.196 INFO:tasks.ceph.mds.c.smithi038.stderr: 19: (Thread::_entry_func(void*)+0xd) [0x7f6eaf22fd73] 2024-09-21T03:39:24.196 INFO:tasks.ceph.mds.c.smithi038.stderr: 20: /lib64/libc.so.6(+0x89c52) [0x7f6eae889c52] 2024-09-21T03:39:24.196 INFO:tasks.ceph.mds.c.smithi038.stderr: 21: /lib64/libc.so.6(+0x10ec80) [0x7f6eae90ec80] 2024-09-21T03:39:24.196 INFO:tasks.ceph.mds.c.smithi038.stderr: 2024-09-21T03:39:24.196 INFO:tasks.ceph.mds.c.smithi038.stderr: 0> 2024-09-21T03:39:24.181+0000 7f6ea9433640 -1 *** Caught signal (Aborted) ** 2024-09-21T03:39:24.197 INFO:tasks.ceph.mds.c.smithi038.stderr: in thread 7f6ea9433640 thread_name:ms_dispatch 2024-09-21T03:39:24.197 INFO:tasks.ceph.mds.c.smithi038.stderr: 2024-09-21T03:39:24.197 INFO:tasks.ceph.mds.c.smithi038.stderr: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T03:39:24.197 INFO:tasks.ceph.mds.c.smithi038.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7f6eae83e6f0] 2024-09-21T03:39:24.197 INFO:tasks.ceph.mds.c.smithi038.stderr: 2: /lib64/libc.so.6(+0x8b99c) [0x7f6eae88b99c] 2024-09-21T03:39:24.197 INFO:tasks.ceph.mds.c.smithi038.stderr: 3: raise() 2024-09-21T03:39:24.197 INFO:tasks.ceph.mds.c.smithi038.stderr: 4: abort() 2024-09-21T03:39:24.197 INFO:tasks.ceph.mds.c.smithi038.stderr: 5: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0x241) [0x7f6eaf257386] 2024-09-21T03:39:24.197 INFO:tasks.ceph.mds.c.smithi038.stderr: 6: (MDSRank::abort(std::basic_string_view >)+0x5f) [0x55657fa22159] 2024-09-21T03:39:24.197 INFO:tasks.ceph.mds.c.smithi038.stderr: 7: (CDentry::check_corruption(bool)+0x54e) [0x55657fd123ca] 2024-09-21T03:39:24.197 INFO:tasks.ceph.mds.c.smithi038.stderr: 8: ceph-mds(+0x175d76) [0x55657fa8ad76] 2024-09-21T03:39:24.197 INFO:tasks.ceph.mds.c.smithi038.stderr: 9: ceph-mds(+0x176176) [0x55657fa8b176] 2024-09-21T03:39:24.197 INFO:tasks.ceph.mds.c.smithi038.stderr: 10: (Server::_rename_prepare(boost::intrusive_ptr const&, EMetaBlob*, ceph::buffer::v15_2_0::list*, CDentry*, CDentry*, std::basic_string_view >, CDentry*)+0x2571) [0x55657fb30afd] 2024-09-21T03:39:24.197 INFO:tasks.ceph.mds.c.smithi038.stderr: 11: (Server::handle_client_rename(boost::intrusive_ptr const&)+0x2ec1) [0x55657fb342f1] 2024-09-21T03:39:24.197 INFO:tasks.ceph.mds.c.smithi038.stderr: 12: (Server::dispatch_client_request(boost::intrusive_ptr const&)+0xd27) [0x55657faf8be5] 2024-09-21T03:39:24.198 INFO:tasks.ceph.mds.c.smithi038.stderr: 13: (Server::handle_client_request(boost::intrusive_ptr const&)+0xa2a) [0x55657faf9892] 2024-09-21T03:39:24.198 INFO:tasks.ceph.mds.c.smithi038.stderr: 14: (Server::dispatch(boost::intrusive_ptr const&)+0xc0) [0x55657fae03cc] 2024-09-21T03:39:24.198 INFO:tasks.ceph.mds.c.smithi038.stderr: 15: (MDSRank::handle_message(boost::intrusive_ptr const&)+0x5ac) [0x55657fa2c6ea] 2024-09-21T03:39:24.198 INFO:tasks.ceph.mds.c.smithi038.stderr: 16: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x28f) [0x55657fa30523] 2024-09-21T03:39:24.198 INFO:tasks.ceph.mds.c.smithi038.stderr: 17: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x94) [0x55657fa30e4a] 2024-09-21T03:39:24.198 INFO:tasks.ceph.mds.c.smithi038.stderr: 18: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x3f0) [0x55657fa13fa4] 2024-09-21T03:39:24.198 INFO:tasks.ceph.mds.c.smithi038.stderr: 19: /usr/lib64/ceph/libceph-common.so.2(+0x3a57df) [0x7f6eaf3a57df] 2024-09-21T03:39:24.198 INFO:tasks.ceph.mds.c.smithi038.stderr: 20: (DispatchQueue::entry()+0x60f) [0x7f6eaf3a60d1] 2024-09-21T03:39:24.198 INFO:tasks.ceph.mds.c.smithi038.stderr: 21: /usr/lib64/ceph/libceph-common.so.2(+0x493f5b) [0x7f6eaf493f5b] 2024-09-21T03:39:24.198 INFO:tasks.ceph.mds.c.smithi038.stderr: 22: (Thread::entry_wrapper()+0x43) [0x7f6eaf22fd57] 2024-09-21T03:39:24.198 INFO:tasks.ceph.mds.c.smithi038.stderr: 23: (Thread::_entry_func(void*)+0xd) [0x7f6eaf22fd73] 2024-09-21T03:39:24.198 INFO:tasks.ceph.mds.c.smithi038.stderr: 24: /lib64/libc.so.6(+0x89c52) [0x7f6eae889c52] 2024-09-21T03:39:24.198 INFO:tasks.ceph.mds.c.smithi038.stderr: 25: /lib64/libc.so.6(+0x10ec80) [0x7f6eae90ec80] 2024-09-21T03:39:24.198 INFO:tasks.ceph.mds.c.smithi038.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-09-21T03:39:24.198 INFO:tasks.ceph.mds.c.smithi038.stderr: 2024-09-21T03:39:24.199 INFO:tasks.ceph.mds.c.smithi038.stderr: -5258> 2024-09-21T03:39:06.377+0000 7f6ea6c2e640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T03:39:24.202 INFO:tasks.ceph.mds.c.smithi038.stderr: -9> 2024-09-21T03:39:24.178+0000 7f6ea9433640 -1 mds.0.cache.den(0x10000000000 z) newly corrupt dentry to be committed: [dentry #0x1/a/z [fffffffffffffff6,head] auth NULL (dn xlock x=1 by request(client.6717:47 nref=7 cr=0x556587681500)) (dversion lock w=1 last_client=6717) pv=11 v=10 ap=2 ino=(nil) state=1073741824 | request=1 lock=2 authpin=1 0x5565874e1680] with next_snap: 2 2024-09-21T03:39:24.202 INFO:tasks.ceph.mds.c.smithi038.stderr: -7> 2024-09-21T03:39:24.178+0000 7f6ea9433640 -1 log_channel(cluster) log [ERR] : MDS abort because newly corrupt dentry to be committed: [dentry #0x1/a/z [fffffffffffffff6,head] auth NULL (dn xlock x=1 by request(client.6717:47 nref=7 cr=0x556587681500)) (dversion lock w=1 last_client=6717) pv=11 v=10 ap=2 ino=(nil) state=1073741824 | request=1 lock=2 authpin=1 0x5565874e1680] 2024-09-21T03:39:24.202 INFO:tasks.ceph.mds.c.smithi038.stderr: -1> 2024-09-21T03:39:24.179+0000 7f6ea9433640 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5120-g18432218/rpm/el9/BUILD/ceph-19.3.0-5120-g18432218/src/mds/MDSRank.cc: In function 'void MDSRank::abort(std::string_view)' thread 7f6ea9433640 time 2024-09-21T03:39:24.179706+0000 2024-09-21T03:39:24.202 INFO:tasks.ceph.mds.c.smithi038.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5120-g18432218/rpm/el9/BUILD/ceph-19.3.0-5120-g18432218/src/mds/MDSRank.cc: 913: ceph_abort_msg("abort() called") 2024-09-21T03:39:24.202 INFO:tasks.ceph.mds.c.smithi038.stderr: 2024-09-21T03:39:24.202 INFO:tasks.ceph.mds.c.smithi038.stderr: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T03:39:24.202 INFO:tasks.ceph.mds.c.smithi038.stderr: 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xcf) [0x7f6eaf257214] 2024-09-21T03:39:24.202 INFO:tasks.ceph.mds.c.smithi038.stderr: 2: (MDSRank::abort(std::basic_string_view >)+0x5f) [0x55657fa22159] 2024-09-21T03:39:24.202 INFO:tasks.ceph.mds.c.smithi038.stderr: 3: (CDentry::check_corruption(bool)+0x54e) [0x55657fd123ca] 2024-09-21T03:39:24.202 INFO:tasks.ceph.mds.c.smithi038.stderr: 4: ceph-mds(+0x175d76) [0x55657fa8ad76] 2024-09-21T03:39:24.202 INFO:tasks.ceph.mds.c.smithi038.stderr: 5: ceph-mds(+0x176176) [0x55657fa8b176] 2024-09-21T03:39:24.203 INFO:tasks.ceph.mds.c.smithi038.stderr: 6: (Server::_rename_prepare(boost::intrusive_ptr const&, EMetaBlob*, ceph::buffer::v15_2_0::list*, CDentry*, CDentry*, std::basic_string_view >, CDentry*)+0x2571) [0x55657fb30afd] 2024-09-21T03:39:24.203 INFO:tasks.ceph.mds.c.smithi038.stderr: 7: (Server::handle_client_rename(boost::intrusive_ptr const&)+0x2ec1) [0x55657fb342f1] 2024-09-21T03:39:24.203 INFO:tasks.ceph.mds.c.smithi038.stderr: 8: (Server::dispatch_client_request(boost::intrusive_ptr const&)+0xd27) [0x55657faf8be5] 2024-09-21T03:39:24.203 INFO:tasks.ceph.mds.c.smithi038.stderr: 9: (Server::handle_client_request(boost::intrusive_ptr const&)+0xa2a) [0x55657faf9892] 2024-09-21T03:39:24.203 INFO:tasks.ceph.mds.c.smithi038.stderr: 10: (Server::dispatch(boost::intrusive_ptr const&)+0xc0) [0x55657fae03cc] 2024-09-21T03:39:24.203 INFO:tasks.ceph.mds.c.smithi038.stderr: 11: (MDSRank::handle_message(boost::intrusive_ptr const&)+0x5ac) [0x55657fa2c6ea] 2024-09-21T03:39:24.203 INFO:tasks.ceph.mds.c.smithi038.stderr: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x28f) [0x55657fa30523] 2024-09-21T03:39:24.203 INFO:tasks.ceph.mds.c.smithi038.stderr: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x94) [0x55657fa30e4a] 2024-09-21T03:39:24.203 INFO:tasks.ceph.mds.c.smithi038.stderr: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x3f0) [0x55657fa13fa4] 2024-09-21T03:39:24.203 INFO:tasks.ceph.mds.c.smithi038.stderr: 15: /usr/lib64/ceph/libceph-common.so.2(+0x3a57df) [0x7f6eaf3a57df] 2024-09-21T03:39:24.203 INFO:tasks.ceph.mds.c.smithi038.stderr: 16: (DispatchQueue::entry()+0x60f) [0x7f6eaf3a60d1] 2024-09-21T03:39:24.203 INFO:tasks.ceph.mds.c.smithi038.stderr: 17: /usr/lib64/ceph/libceph-common.so.2(+0x493f5b) [0x7f6eaf493f5b] 2024-09-21T03:39:24.203 INFO:tasks.ceph.mds.c.smithi038.stderr: 18: (Thread::entry_wrapper()+0x43) [0x7f6eaf22fd57] 2024-09-21T03:39:24.204 INFO:tasks.ceph.mds.c.smithi038.stderr: 19: (Thread::_entry_func(void*)+0xd) [0x7f6eaf22fd73] 2024-09-21T03:39:24.204 INFO:tasks.ceph.mds.c.smithi038.stderr: 20: /lib64/libc.so.6(+0x89c52) [0x7f6eae889c52] 2024-09-21T03:39:24.204 INFO:tasks.ceph.mds.c.smithi038.stderr: 21: /lib64/libc.so.6(+0x10ec80) [0x7f6eae90ec80] 2024-09-21T03:39:24.204 INFO:tasks.ceph.mds.c.smithi038.stderr: 2024-09-21T03:39:24.204 INFO:tasks.ceph.mds.c.smithi038.stderr: 0> 2024-09-21T03:39:24.181+0000 7f6ea9433640 -1 *** Caught signal (Aborted) ** 2024-09-21T03:39:24.204 INFO:tasks.ceph.mds.c.smithi038.stderr: in thread 7f6ea9433640 thread_name:ms_dispatch 2024-09-21T03:39:24.204 INFO:tasks.ceph.mds.c.smithi038.stderr: 2024-09-21T03:39:24.204 INFO:tasks.ceph.mds.c.smithi038.stderr: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T03:39:24.204 INFO:tasks.ceph.mds.c.smithi038.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7f6eae83e6f0] 2024-09-21T03:39:24.204 INFO:tasks.ceph.mds.c.smithi038.stderr: 2: /lib64/libc.so.6(+0x8b99c) [0x7f6eae88b99c] 2024-09-21T03:39:24.204 INFO:tasks.ceph.mds.c.smithi038.stderr: 3: raise() 2024-09-21T03:39:24.204 INFO:tasks.ceph.mds.c.smithi038.stderr: 4: abort() 2024-09-21T03:39:24.204 INFO:tasks.ceph.mds.c.smithi038.stderr: 5: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0x241) [0x7f6eaf257386] 2024-09-21T03:39:24.204 INFO:tasks.ceph.mds.c.smithi038.stderr: 6: (MDSRank::abort(std::basic_string_view >)+0x5f) [0x55657fa22159] 2024-09-21T03:39:24.204 INFO:tasks.ceph.mds.c.smithi038.stderr: 7: (CDentry::check_corruption(bool)+0x54e) [0x55657fd123ca] 2024-09-21T03:39:24.205 INFO:tasks.ceph.mds.c.smithi038.stderr: 8: ceph-mds(+0x175d76) [0x55657fa8ad76] 2024-09-21T03:39:24.205 INFO:tasks.ceph.mds.c.smithi038.stderr: 9: ceph-mds(+0x176176) [0x55657fa8b176] 2024-09-21T03:39:24.205 INFO:tasks.ceph.mds.c.smithi038.stderr: 10: (Server::_rename_prepare(boost::intrusive_ptr const&, EMetaBlob*, ceph::buffer::v15_2_0::list*, CDentry*, CDentry*, std::basic_string_view >, CDentry*)+0x2571) [0x55657fb30afd] 2024-09-21T03:39:24.205 INFO:tasks.ceph.mds.c.smithi038.stderr: 11: (Server::handle_client_rename(boost::intrusive_ptr const&)+0x2ec1) [0x55657fb342f1] 2024-09-21T03:39:24.205 INFO:tasks.ceph.mds.c.smithi038.stderr: 12: (Server::dispatch_client_request(boost::intrusive_ptr const&)+0xd27) [0x55657faf8be5] 2024-09-21T03:39:24.205 INFO:tasks.ceph.mds.c.smithi038.stderr: 13: (Server::handle_client_request(boost::intrusive_ptr const&)+0xa2a) [0x55657faf9892] 2024-09-21T03:39:24.205 INFO:tasks.ceph.mds.c.smithi038.stderr: 14: (Server::dispatch(boost::intrusive_ptr const&)+0xc0) [0x55657fae03cc] 2024-09-21T03:39:24.205 INFO:tasks.ceph.mds.c.smithi038.stderr: 15: (MDSRank::handle_message(boost::intrusive_ptr const&)+0x5ac) [0x55657fa2c6ea] 2024-09-21T03:39:24.205 INFO:tasks.ceph.mds.c.smithi038.stderr: 16: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x28f) [0x55657fa30523] 2024-09-21T03:39:24.205 INFO:tasks.ceph.mds.c.smithi038.stderr: 17: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x94) [0x55657fa30e4a] 2024-09-21T03:39:24.205 INFO:tasks.ceph.mds.c.smithi038.stderr: 18: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x3f0) [0x55657fa13fa4] 2024-09-21T03:39:24.205 INFO:tasks.ceph.mds.c.smithi038.stderr: 19: /usr/lib64/ceph/libceph-common.so.2(+0x3a57df) [0x7f6eaf3a57df] 2024-09-21T03:39:24.205 INFO:tasks.ceph.mds.c.smithi038.stderr: 20: (DispatchQueue::entry()+0x60f) [0x7f6eaf3a60d1] 2024-09-21T03:39:24.206 INFO:tasks.ceph.mds.c.smithi038.stderr: 21: /usr/lib64/ceph/libceph-common.so.2(+0x493f5b) [0x7f6eaf493f5b] 2024-09-21T03:39:24.206 INFO:tasks.ceph.mds.c.smithi038.stderr: 22: (Thread::entry_wrapper()+0x43) [0x7f6eaf22fd57] 2024-09-21T03:39:24.206 INFO:tasks.ceph.mds.c.smithi038.stderr: 23: (Thread::_entry_func(void*)+0xd) [0x7f6eaf22fd73] 2024-09-21T03:39:24.206 INFO:tasks.ceph.mds.c.smithi038.stderr: 24: /lib64/libc.so.6(+0x89c52) [0x7f6eae889c52] 2024-09-21T03:39:24.206 INFO:tasks.ceph.mds.c.smithi038.stderr: 25: /lib64/libc.so.6(+0x10ec80) [0x7f6eae90ec80] 2024-09-21T03:39:24.206 INFO:tasks.ceph.mds.c.smithi038.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-09-21T03:39:24.206 INFO:tasks.ceph.mds.c.smithi038.stderr: 2024-09-21T03:39:24.257 INFO:teuthology.orchestra.run.smithi038.stdout:{"mds_beacon_grace":"15.000000"} 2024-09-21T03:39:24.257 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mds_beacon_grace": "15.000000" } 2024-09-21T03:39:24.257 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:39:24.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:24.274+0000 7f07f969b640 1 -- 172.21.15.38:0/3655252629 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f07f4074520 msgr2=0x7f07f40d0b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:24.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:24.274+0000 7f07f969b640 1 --2- 172.21.15.38:0/3655252629 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f07f4074520 0x7f07f40d0b40 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f07e8009830 tx=0x7f07e802a530 comp rx=0 tx=0).stop 2024-09-21T03:39:24.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:24.275+0000 7f07f969b640 1 -- 172.21.15.38:0/3655252629 shutdown_connections 2024-09-21T03:39:24.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:24.275+0000 7f07f969b640 1 --2- 172.21.15.38:0/3655252629 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f07f4074520 0x7f07f40d0b40 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:24.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:24.275+0000 7f07f969b640 1 -- 172.21.15.38:0/3655252629 >> 172.21.15.38:0/3655252629 conn(0x7f07f40d1110 msgr2=0x7f07f40d1520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:24.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:24.275+0000 7f07f969b640 1 -- 172.21.15.38:0/3655252629 shutdown_connections 2024-09-21T03:39:24.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:24.275+0000 7f07f969b640 1 -- 172.21.15.38:0/3655252629 wait complete. 2024-09-21T03:39:24.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:24.277+0000 7f07f969b640 1 Processor -- start 2024-09-21T03:39:24.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:24.277+0000 7f07f969b640 1 -- start start 2024-09-21T03:39:24.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:24.278+0000 7f07f969b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f07f406cde0 0x7f07f4141680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:24.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:24.278+0000 7f07f969b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f07f406dfb0 0x7f07f415ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:24.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:24.278+0000 7f07f969b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f07f41442c0 0x7f07f413b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:24.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:24.278+0000 7f07f969b640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f07f4145470 con 0x7f07f406cde0 2024-09-21T03:39:24.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:24.278+0000 7f07f969b640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f07f41452f0 con 0x7f07f41442c0 2024-09-21T03:39:24.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:24.278+0000 7f07f969b640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f07f41455f0 con 0x7f07f406dfb0 2024-09-21T03:39:24.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:24.278+0000 7f07f2ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f07f406cde0 0x7f07f4141680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:24.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:24.278+0000 7f07f2ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f07f406cde0 0x7f07f4141680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60582/0 (socket says 172.21.15.38:60582) 2024-09-21T03:39:24.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:24.278+0000 7f07f2ffd640 1 -- 172.21.15.38:0/2145970265 learned_addr learned my addr 172.21.15.38:0/2145970265 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:39:24.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:24.278+0000 7f07f27fc640 1 --2- 172.21.15.38:0/2145970265 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f07f406dfb0 0x7f07f415ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:24.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:24.278+0000 7f07f37fe640 1 --2- 172.21.15.38:0/2145970265 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f07f41442c0 0x7f07f413b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:24.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:24.279+0000 7f07f2ffd640 1 -- 172.21.15.38:0/2145970265 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f07f406dfb0 msgr2=0x7f07f415ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:24.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:24.279+0000 7f07f2ffd640 1 --2- 172.21.15.38:0/2145970265 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f07f406dfb0 0x7f07f415ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:24.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:24.279+0000 7f07f2ffd640 1 -- 172.21.15.38:0/2145970265 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f07f41442c0 msgr2=0x7f07f413b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:24.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:24.279+0000 7f07f2ffd640 1 --2- 172.21.15.38:0/2145970265 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f07f41442c0 0x7f07f413b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:24.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:24.279+0000 7f07f2ffd640 1 -- 172.21.15.38:0/2145970265 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07f40cce80 con 0x7f07f406cde0 2024-09-21T03:39:24.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:24.279+0000 7f07f2ffd640 1 --2- 172.21.15.38:0/2145970265 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f07f406cde0 0x7f07f4141680 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f07e400c430 tx=0x7f07e400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:24.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:24.279+0000 7f07db7fe640 1 -- 172.21.15.38:0/2145970265 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07e4017070 con 0x7f07f406cde0 2024-09-21T03:39:24.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:24.279+0000 7f07db7fe640 1 -- 172.21.15.38:0/2145970265 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f07e40053c0 con 0x7f07f406cde0 2024-09-21T03:39:24.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:24.279+0000 7f07db7fe640 1 -- 172.21.15.38:0/2145970265 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07e4003850 con 0x7f07f406cde0 2024-09-21T03:39:24.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:24.279+0000 7f07f969b640 1 -- 172.21.15.38:0/2145970265 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07f4144860 con 0x7f07f406cde0 2024-09-21T03:39:24.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:24.279+0000 7f07f969b640 1 -- 172.21.15.38:0/2145970265 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07f415e690 con 0x7f07f406cde0 2024-09-21T03:39:24.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:24.280+0000 7f07f969b640 1 -- 172.21.15.38:0/2145970265 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f07f40ce8d0 con 0x7f07f406cde0 2024-09-21T03:39:24.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:24.281+0000 7f07db7fe640 1 -- 172.21.15.38:0/2145970265 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f07e401a070 con 0x7f07f406cde0 2024-09-21T03:39:24.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:24.282+0000 7f07db7fe640 1 --2- 172.21.15.38:0/2145970265 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f07e0078bc0 0x7f07e007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:24.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:24.282+0000 7f07db7fe640 1 -- 172.21.15.38:0/2145970265 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 6101+0+0 (secure 0 0 0) 0x7f07e409a140 con 0x7f07f406cde0 2024-09-21T03:39:24.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:24.282+0000 7f07db7fe640 1 -- 172.21.15.38:0/2145970265 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+511 (secure 0 0 0) 0x7f07e409c7c0 con 0x7f07f406cde0 2024-09-21T03:39:24.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:24.282+0000 7f07f27fc640 1 --2- 172.21.15.38:0/2145970265 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f07e0078bc0 0x7f07e007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:24.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:24.282+0000 7f07f969b640 1 -- 172.21.15.38:0/2145970265 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({log-info=0+}) -- 0x7f07f4140bb0 con 0x7f07f406cde0 2024-09-21T03:39:24.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:24.283+0000 7f07f27fc640 1 --2- 172.21.15.38:0/2145970265 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f07e0078bc0 0x7f07e007b0d0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f07e8034000 tx=0x7f07e8004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:24.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:24.374+0000 7f37ed12b640 1 -- 172.21.15.38:0/2429225342 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f37e8074460 msgr2=0x7f37e80b43d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:24.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:24.374+0000 7f37ed12b640 1 --2- 172.21.15.38:0/2429225342 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f37e8074460 0x7f37e80b43d0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f37e806e3e0 tx=0x7f37e002a890 comp rx=0 tx=0).stop 2024-09-21T03:39:24.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:24.374+0000 7f37ed12b640 1 -- 172.21.15.38:0/2429225342 shutdown_connections 2024-09-21T03:39:24.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:24.374+0000 7f37ed12b640 1 --2- 172.21.15.38:0/2429225342 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f37e8074460 0x7f37e80b43d0 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:24.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:24.374+0000 7f37ed12b640 1 -- 172.21.15.38:0/2429225342 >> 172.21.15.38:0/2429225342 conn(0x7f37e80b2cc0 msgr2=0x7f37e80b30d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:24.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:24.374+0000 7f37ed12b640 1 -- 172.21.15.38:0/2429225342 shutdown_connections 2024-09-21T03:39:24.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:24.374+0000 7f37ed12b640 1 -- 172.21.15.38:0/2429225342 wait complete. 2024-09-21T03:39:24.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:24.376+0000 7f37ed12b640 1 Processor -- start 2024-09-21T03:39:24.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:24.377+0000 7f37ed12b640 1 -- start start 2024-09-21T03:39:24.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:24.378+0000 7f37ed12b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f37e80b3fc0 0x7f37e81f46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:24.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:24.378+0000 7f37ed12b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f37e80d0b40 0x7f37e80c5f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:24.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:24.378+0000 7f37ed12b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f37e80c6470 0x7f37e81e8340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:24.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:24.378+0000 7f37ed12b640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f37e815b960 con 0x7f37e80b3fc0 2024-09-21T03:39:24.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:24.378+0000 7f37ed12b640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f37e815b7e0 con 0x7f37e80d0b40 2024-09-21T03:39:24.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:24.378+0000 7f37ed12b640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f37e815bae0 con 0x7f37e80c6470 2024-09-21T03:39:24.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:24.378+0000 7f37e6d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f37e80b3fc0 0x7f37e81f46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:24.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:24.378+0000 7f37e6d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f37e80b3fc0 0x7f37e81f46a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60602/0 (socket says 172.21.15.38:60602) 2024-09-21T03:39:24.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:24.378+0000 7f37e6d9d640 1 -- 172.21.15.38:0/3586719807 learned_addr learned my addr 172.21.15.38:0/3586719807 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:39:24.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:24.378+0000 7f37e759e640 1 --2- 172.21.15.38:0/3586719807 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f37e80c6470 0x7f37e81e8340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:24.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:24.378+0000 7f37de59c640 1 --2- 172.21.15.38:0/3586719807 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f37e80d0b40 0x7f37e80c5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:24.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:24.378+0000 7f37e6d9d640 1 -- 172.21.15.38:0/3586719807 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f37e80c6470 msgr2=0x7f37e81e8340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:24.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:24.378+0000 7f37e6d9d640 1 --2- 172.21.15.38:0/3586719807 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f37e80c6470 0x7f37e81e8340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:24.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:24.378+0000 7f37e6d9d640 1 -- 172.21.15.38:0/3586719807 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f37e80d0b40 msgr2=0x7f37e80c5f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:24.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:24.378+0000 7f37e6d9d640 1 --2- 172.21.15.38:0/3586719807 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f37e80d0b40 0x7f37e80c5f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:24.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:24.378+0000 7f37e6d9d640 1 -- 172.21.15.38:0/3586719807 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37e80c6c40 con 0x7f37e80b3fc0 2024-09-21T03:39:24.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:24.379+0000 7f37e6d9d640 1 --2- 172.21.15.38:0/3586719807 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f37e80b3fc0 0x7f37e81f46a0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f37d800c3c0 tx=0x7f37d800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:24.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:24.379+0000 7f37dffff640 1 -- 172.21.15.38:0/3586719807 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37d8017070 con 0x7f37e80b3fc0 2024-09-21T03:39:24.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:24.379+0000 7f37dffff640 1 -- 172.21.15.38:0/3586719807 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f37d80035f0 con 0x7f37e80b3fc0 2024-09-21T03:39:24.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:24.379+0000 7f37ed12b640 1 -- 172.21.15.38:0/3586719807 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37e80c6f20 con 0x7f37e80b3fc0 2024-09-21T03:39:24.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:24.379+0000 7f37dffff640 1 -- 172.21.15.38:0/3586719807 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37d8003a80 con 0x7f37e80b3fc0 2024-09-21T03:39:24.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:24.379+0000 7f37ed12b640 1 -- 172.21.15.38:0/3586719807 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37e80c74e0 con 0x7f37e80b3fc0 2024-09-21T03:39:24.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:24.380+0000 7f37ed12b640 1 -- 172.21.15.38:0/3586719807 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37e815bfd0 con 0x7f37e80b3fc0 2024-09-21T03:39:24.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:24.381+0000 7f37dffff640 1 -- 172.21.15.38:0/3586719807 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f37d801a070 con 0x7f37e80b3fc0 2024-09-21T03:39:24.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:24.381+0000 7f37dffff640 1 --2- 172.21.15.38:0/3586719807 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f37c8078b90 0x7f37c807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:24.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:24.381+0000 7f37dffff640 1 -- 172.21.15.38:0/3586719807 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 6101+0+0 (secure 0 0 0) 0x7f37d809a120 con 0x7f37e80b3fc0 2024-09-21T03:39:24.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:24.381+0000 7f37de59c640 1 --2- 172.21.15.38:0/3586719807 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f37c8078b90 0x7f37c807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:24.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:24.382+0000 7f37de59c640 1 --2- 172.21.15.38:0/3586719807 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f37c8078b90 0x7f37c807b0a0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f37e80c8110 tx=0x7f37d0009060 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:24.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:24.386+0000 7f37dffff640 1 -- 172.21.15.38:0/3586719807 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f37d8061d90 con 0x7f37e80b3fc0 2024-09-21T03:39:24.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:24.491+0000 7f07db7fe640 1 -- 172.21.15.38:0/2145970265 <== mon.0 v2:172.21.15.38:3300/0 7 ==== log(1 entries from seq 1 at 2024-09-21T03:39:24.179673+0000) ==== 468+0+0 (secure 0 0 0) 0x7f07e40985f0 con 0x7f07f406cde0 2024-09-21T03:39:24.493 INFO:teuthology.orchestra.run.smithi038.stdout: cluster: 2024-09-21T03:39:24.494 INFO:teuthology.orchestra.run.smithi038.stdout: id: 9d50d66f-656b-4fe2-8e3b-fe53b79c4d43 2024-09-21T03:39:24.494 INFO:teuthology.orchestra.run.smithi038.stdout: health: HEALTH_OK 2024-09-21T03:39:24.494 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:39:24.494 INFO:teuthology.orchestra.run.smithi038.stdout: services: 2024-09-21T03:39:24.494 INFO:teuthology.orchestra.run.smithi038.stdout: mon: 3 daemons, quorum a,b,c (age 6m) 2024-09-21T03:39:24.494 INFO:teuthology.orchestra.run.smithi038.stdout: mgr: x(active, since 6m), standbys: y 2024-09-21T03:39:24.494 INFO:teuthology.orchestra.run.smithi038.stdout: mds: 1/1 daemons up, 3 standby 2024-09-21T03:39:24.494 INFO:teuthology.orchestra.run.smithi038.stdout: osd: 8 osds: 8 up (since 6m), 8 in (since 6m) 2024-09-21T03:39:24.494 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:39:24.494 INFO:teuthology.orchestra.run.smithi038.stdout: data: 2024-09-21T03:39:24.494 INFO:teuthology.orchestra.run.smithi038.stdout: volumes: 1/1 healthy 2024-09-21T03:39:24.494 INFO:teuthology.orchestra.run.smithi038.stdout: pools: 3 pools, 97 pgs 2024-09-21T03:39:24.495 INFO:teuthology.orchestra.run.smithi038.stdout: objects: 29 objects, 596 KiB 2024-09-21T03:39:24.495 INFO:teuthology.orchestra.run.smithi038.stdout: usage: 269 MiB used, 720 GiB / 720 GiB avail 2024-09-21T03:39:24.495 INFO:teuthology.orchestra.run.smithi038.stdout: pgs: 97 active+clean 2024-09-21T03:39:24.495 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:39:24.495 INFO:teuthology.orchestra.run.smithi038.stdout: io: 2024-09-21T03:39:24.495 INFO:teuthology.orchestra.run.smithi038.stdout: client: 0 B/s rd, 2.7 KiB/s wr, 0 op/s rd, 2 op/s wr 2024-09-21T03:39:24.495 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:39:24.495 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:39:24.495 INFO:teuthology.orchestra.run.smithi038.stdout:2024-09-21T03:39:24.179673+0000 mds.c [ERR] MDS abort because newly corrupt dentry to be committed: [dentry #0x1/a/z [fffffffffffffff6,head] auth NULL (dn xlock x=1 by request(client.6717:47 nref=7 cr=0x556587681500)) (dversion lock w=1 last_client=6717) pv=11 v=10 ap=2 ino=(nil) state=1073741824 | request=1 lock=2 authpin=1 0x5565874e1680] 2024-09-21T03:39:24.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:24.611+0000 7f37ed12b640 1 -- 172.21.15.38:0/3586719807 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f37e80cff90 con 0x7f37e80b3fc0 2024-09-21T03:39:24.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:24.611+0000 7f37dffff640 1 -- 172.21.15.38:0/3586719807 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) ==== 94+0+5231 (secure 0 0 0) 0x7f37d80667c0 con 0x7f37e80b3fc0 2024-09-21T03:39:24.613 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:39:24.614 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":68,"btime":"2024-09-21T03:39:18:466470+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5783,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/1358588311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1358588311},{"type":"v1","addr":"172.21.15.38:6835","nonce":1358588311}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63},{"gid":5789,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/1676483003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":1676483003},{"type":"v1","addr":"172.21.15.195:6837","nonce":1676483003}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63},{"gid":6193,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/2771781100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":2771781100},{"type":"v1","addr":"172.21.15.195:6836","nonce":2771781100}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:39:04.779948+0000","modified":"2024-09-21T03:39:17.658058+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6546},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6546":{"gid":6546,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":3,"addr":"172.21.15.38:6833/1144075315","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":1144075315},{"type":"v1","addr":"172.21.15.38:6833","nonce":1144075315}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6546,"qdb_cluster":[6546]},"id":5}]} 2024-09-21T03:39:24.614 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 68 2024-09-21T03:39:24.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:24.613+0000 7f37ed12b640 1 -- 172.21.15.38:0/3586719807 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f37c8078b90 msgr2=0x7f37c807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:24.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:24.613+0000 7f37ed12b640 1 --2- 172.21.15.38:0/3586719807 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f37c8078b90 0x7f37c807b0a0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f37e80c8110 tx=0x7f37d0009060 comp rx=0 tx=0).stop 2024-09-21T03:39:24.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:24.613+0000 7f37ed12b640 1 -- 172.21.15.38:0/3586719807 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f37e80b3fc0 msgr2=0x7f37e81f46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:24.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:24.613+0000 7f37ed12b640 1 --2- 172.21.15.38:0/3586719807 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f37e80b3fc0 0x7f37e81f46a0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f37d800c3c0 tx=0x7f37d800c880 comp rx=0 tx=0).stop 2024-09-21T03:39:24.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:24.614+0000 7f37ed12b640 1 -- 172.21.15.38:0/3586719807 shutdown_connections 2024-09-21T03:39:24.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:24.614+0000 7f37ed12b640 1 --2- 172.21.15.38:0/3586719807 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f37c8078b90 0x7f37c807b0a0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:24.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:24.614+0000 7f37ed12b640 1 --2- 172.21.15.38:0/3586719807 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f37e80c6470 0x7f37e81e8340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:24.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:24.614+0000 7f37ed12b640 1 --2- 172.21.15.38:0/3586719807 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f37e80d0b40 0x7f37e80c5f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:24.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:24.614+0000 7f37ed12b640 1 --2- 172.21.15.38:0/3586719807 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f37e80b3fc0 0x7f37e81f46a0 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:24.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:24.614+0000 7f37ed12b640 1 -- 172.21.15.38:0/3586719807 >> 172.21.15.38:0/3586719807 conn(0x7f37e80b2cc0 msgr2=0x7f37e80cf040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:24.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:24.614+0000 7f37ed12b640 1 -- 172.21.15.38:0/3586719807 shutdown_connections 2024-09-21T03:39:24.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:24.614+0000 7f37ed12b640 1 -- 172.21.15.38:0/3586719807 wait complete. 2024-09-21T03:39:24.627 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2024-09-21T03:39:24.960 INFO:tasks.ceph.mds.c.smithi038.stderr:daemon-helper: command crashed with signal 6 2024-09-21T03:39:27.586 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~0s 2024-09-21T03:39:29.628 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:39:29.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:29.758+0000 7f93bcf4a640 1 -- 172.21.15.38:0/3530605736 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f93b80d0980 msgr2=0x7f93b80d0d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:29.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:29.758+0000 7f93bcf4a640 1 --2- 172.21.15.38:0/3530605736 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f93b80d0980 0x7f93b80d0d90 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f93b0005fd0 tx=0x7f93b002aa90 comp rx=0 tx=0).stop 2024-09-21T03:39:29.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:29.759+0000 7f93bcf4a640 1 -- 172.21.15.38:0/3530605736 shutdown_connections 2024-09-21T03:39:29.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:29.759+0000 7f93bcf4a640 1 --2- 172.21.15.38:0/3530605736 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f93b80d0980 0x7f93b80d0d90 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:29.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:29.759+0000 7f93bcf4a640 1 --2- 172.21.15.38:0/3530605736 >> v2:172.21.15.195:3300/0 conn(0x7f93b80d01f0 0x7f93b815daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:29.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:29.759+0000 7f93bcf4a640 1 -- 172.21.15.38:0/3530605736 >> 172.21.15.38:0/3530605736 conn(0x7f93b8072990 msgr2=0x7f93b8072da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:29.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:29.759+0000 7f93bcf4a640 1 -- 172.21.15.38:0/3530605736 shutdown_connections 2024-09-21T03:39:29.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:29.759+0000 7f93bcf4a640 1 -- 172.21.15.38:0/3530605736 wait complete. 2024-09-21T03:39:29.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:29.761+0000 7f93bcf4a640 1 Processor -- start 2024-09-21T03:39:29.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:29.761+0000 7f93bcf4a640 1 -- start start 2024-09-21T03:39:29.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:29.762+0000 7f93bcf4a640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f93b80d01f0 0x7f93b8145ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:29.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:29.762+0000 7f93bcf4a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f93b80d0980 0x7f93b8146220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:29.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:29.762+0000 7f93bcf4a640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f93b815a6f0 0x7f93b8146c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:29.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:29.762+0000 7f93bcf4a640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f93b80cd340 con 0x7f93b80d0980 2024-09-21T03:39:29.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:29.762+0000 7f93bcf4a640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f93b80cd1c0 con 0x7f93b80d01f0 2024-09-21T03:39:29.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:29.762+0000 7f93bcf4a640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f93b80cd4c0 con 0x7f93b815a6f0 2024-09-21T03:39:29.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:29.762+0000 7f93b5d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f93b80d0980 0x7f93b8146220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:29.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:29.762+0000 7f93b5d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f93b80d0980 0x7f93b8146220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60614/0 (socket says 172.21.15.38:60614) 2024-09-21T03:39:29.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:29.762+0000 7f93b5d9b640 1 -- 172.21.15.38:0/772734604 learned_addr learned my addr 172.21.15.38:0/772734604 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:39:29.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:29.762+0000 7f93b659c640 1 --2- 172.21.15.38:0/772734604 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f93b80d01f0 0x7f93b8145ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:29.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:29.762+0000 7f93b6d9d640 1 --2- 172.21.15.38:0/772734604 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f93b815a6f0 0x7f93b8146c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:29.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:29.762+0000 7f93b5d9b640 1 -- 172.21.15.38:0/772734604 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f93b815a6f0 msgr2=0x7f93b8146c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:29.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:29.762+0000 7f93b5d9b640 1 --2- 172.21.15.38:0/772734604 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f93b815a6f0 0x7f93b8146c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:29.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:29.762+0000 7f93b5d9b640 1 -- 172.21.15.38:0/772734604 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f93b80d01f0 msgr2=0x7f93b8145ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:29.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:29.762+0000 7f93b5d9b640 1 --2- 172.21.15.38:0/772734604 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f93b80d01f0 0x7f93b8145ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:29.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:29.762+0000 7f93b5d9b640 1 -- 172.21.15.38:0/772734604 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93b80c86a0 con 0x7f93b80d0980 2024-09-21T03:39:29.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:29.763+0000 7f93b659c640 1 --2- 172.21.15.38:0/772734604 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f93b80d01f0 0x7f93b8145ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:39:29.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:29.763+0000 7f93b5d9b640 1 --2- 172.21.15.38:0/772734604 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f93b80d0980 0x7f93b8146220 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f93ac00c730 tx=0x7f93ac00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:29.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:29.763+0000 7f93a6ffd640 1 -- 172.21.15.38:0/772734604 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93ac0052d0 con 0x7f93b80d0980 2024-09-21T03:39:29.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:29.763+0000 7f93a6ffd640 1 -- 172.21.15.38:0/772734604 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f93ac005490 con 0x7f93b80d0980 2024-09-21T03:39:29.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:29.763+0000 7f93bcf4a640 1 -- 172.21.15.38:0/772734604 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93b80c78e0 con 0x7f93b80d0980 2024-09-21T03:39:29.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:29.763+0000 7f93a6ffd640 1 -- 172.21.15.38:0/772734604 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93ac00a8a0 con 0x7f93b80d0980 2024-09-21T03:39:29.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:29.763+0000 7f93bcf4a640 1 -- 172.21.15.38:0/772734604 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93b80c7ea0 con 0x7f93b80d0980 2024-09-21T03:39:29.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:29.764+0000 7f93bcf4a640 1 -- 172.21.15.38:0/772734604 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93b80cc470 con 0x7f93b80d0980 2024-09-21T03:39:29.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:29.765+0000 7f93a6ffd640 1 -- 172.21.15.38:0/772734604 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f93ac00aa40 con 0x7f93b80d0980 2024-09-21T03:39:29.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:29.766+0000 7f93a6ffd640 1 --2- 172.21.15.38:0/772734604 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9398078bc0 0x7f939807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:29.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:29.766+0000 7f93a6ffd640 1 -- 172.21.15.38:0/772734604 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 6101+0+0 (secure 0 0 0) 0x7f93ac09a7d0 con 0x7f93b80d0980 2024-09-21T03:39:29.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:29.766+0000 7f93b659c640 1 --2- 172.21.15.38:0/772734604 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9398078bc0 0x7f939807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:29.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:29.767+0000 7f93b659c640 1 --2- 172.21.15.38:0/772734604 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9398078bc0 0x7f939807b0d0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f93a80095f0 tx=0x7f93a8005f90 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:29.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:29.770+0000 7f93a6ffd640 1 -- 172.21.15.38:0/772734604 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f93b80d12d0 con 0x7f93b80d0980 2024-09-21T03:39:29.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:29.997+0000 7f93bcf4a640 1 -- 172.21.15.38:0/772734604 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f93b815daf0 con 0x7f93b80d0980 2024-09-21T03:39:30.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:29.998+0000 7f93a6ffd640 1 -- 172.21.15.38:0/772734604 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) ==== 94+0+5231 (secure 0 0 0) 0x7f93ac0624c0 con 0x7f93b80d0980 2024-09-21T03:39:30.000 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:39:30.000 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":68,"btime":"2024-09-21T03:39:18:466470+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5783,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/1358588311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1358588311},{"type":"v1","addr":"172.21.15.38:6835","nonce":1358588311}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63},{"gid":5789,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/1676483003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":1676483003},{"type":"v1","addr":"172.21.15.195:6837","nonce":1676483003}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63},{"gid":6193,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/2771781100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":2771781100},{"type":"v1","addr":"172.21.15.195:6836","nonce":2771781100}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:39:04.779948+0000","modified":"2024-09-21T03:39:17.658058+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6546},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6546":{"gid":6546,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":3,"addr":"172.21.15.38:6833/1144075315","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":1144075315},{"type":"v1","addr":"172.21.15.38:6833","nonce":1144075315}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6546,"qdb_cluster":[6546]},"id":5}]} 2024-09-21T03:39:30.000 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 68 2024-09-21T03:39:30.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:30.000+0000 7f93bcf4a640 1 -- 172.21.15.38:0/772734604 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9398078bc0 msgr2=0x7f939807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:30.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:30.000+0000 7f93bcf4a640 1 --2- 172.21.15.38:0/772734604 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9398078bc0 0x7f939807b0d0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f93a80095f0 tx=0x7f93a8005f90 comp rx=0 tx=0).stop 2024-09-21T03:39:30.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:30.000+0000 7f93bcf4a640 1 -- 172.21.15.38:0/772734604 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f93b80d0980 msgr2=0x7f93b8146220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:30.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:30.000+0000 7f93bcf4a640 1 --2- 172.21.15.38:0/772734604 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f93b80d0980 0x7f93b8146220 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f93ac00c730 tx=0x7f93ac00cbf0 comp rx=0 tx=0).stop 2024-09-21T03:39:30.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:30.000+0000 7f93bcf4a640 1 -- 172.21.15.38:0/772734604 shutdown_connections 2024-09-21T03:39:30.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:30.000+0000 7f93bcf4a640 1 --2- 172.21.15.38:0/772734604 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9398078bc0 0x7f939807b0d0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:30.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:30.000+0000 7f93bcf4a640 1 --2- 172.21.15.38:0/772734604 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f93b815a6f0 0x7f93b8146c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:30.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:30.000+0000 7f93bcf4a640 1 --2- 172.21.15.38:0/772734604 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f93b80d0980 0x7f93b8146220 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:30.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:30.000+0000 7f93bcf4a640 1 --2- 172.21.15.38:0/772734604 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f93b80d01f0 0x7f93b8145ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:30.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:30.000+0000 7f93bcf4a640 1 -- 172.21.15.38:0/772734604 >> 172.21.15.38:0/772734604 conn(0x7f93b8072990 msgr2=0x7f93b815eb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:30.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:30.000+0000 7f93bcf4a640 1 -- 172.21.15.38:0/772734604 shutdown_connections 2024-09-21T03:39:30.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:30.000+0000 7f93bcf4a640 1 -- 172.21.15.38:0/772734604 wait complete. 2024-09-21T03:39:30.013 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2024-09-21T03:39:30.910 DEBUG:teuthology.orchestra.run.smithi038:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T03:39:30.914 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T03:39:34.198 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~7s 2024-09-21T03:39:35.014 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:39:35.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:35.144+0000 7f91486cc640 1 -- 172.21.15.38:0/1975827279 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f91400cfe00 msgr2=0x7f91400d2a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:35.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:35.144+0000 7f91486cc640 1 --2- 172.21.15.38:0/1975827279 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f91400cfe00 0x7f91400d2a20 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f913c005fd0 tx=0x7f913c02a5d0 comp rx=0 tx=0).stop 2024-09-21T03:39:35.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:35.144+0000 7f91486cc640 1 -- 172.21.15.38:0/1975827279 shutdown_connections 2024-09-21T03:39:35.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:35.144+0000 7f91486cc640 1 --2- 172.21.15.38:0/1975827279 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f91400b2e60 0x7f91400b3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:35.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:35.144+0000 7f91486cc640 1 --2- 172.21.15.38:0/1975827279 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f91400cfe00 0x7f91400d2a20 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:35.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:35.144+0000 7f91486cc640 1 -- 172.21.15.38:0/1975827279 >> 172.21.15.38:0/1975827279 conn(0x7f91400d4440 msgr2=0x7f91400d4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:35.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:35.144+0000 7f91486cc640 1 -- 172.21.15.38:0/1975827279 shutdown_connections 2024-09-21T03:39:35.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:35.144+0000 7f91486cc640 1 -- 172.21.15.38:0/1975827279 wait complete. 2024-09-21T03:39:35.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:35.146+0000 7f91486cc640 1 Processor -- start 2024-09-21T03:39:35.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:35.146+0000 7f91486cc640 1 -- start start 2024-09-21T03:39:35.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:35.147+0000 7f91486cc640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f91400b2e60 0x7f914015e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:35.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:35.147+0000 7f91486cc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f91400cfe00 0x7f9140149dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:35.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:35.147+0000 7f91486cc640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9140162590 0x7f9140160910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:35.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:35.147+0000 7f91486cc640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f91400b3930 con 0x7f91400cfe00 2024-09-21T03:39:35.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:35.147+0000 7f91486cc640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f91400b37b0 con 0x7f91400b2e60 2024-09-21T03:39:35.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:35.147+0000 7f91486cc640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f91400b3ab0 con 0x7f9140162590 2024-09-21T03:39:35.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:35.147+0000 7f9145c67640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f91400cfe00 0x7f9140149dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:35.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:35.147+0000 7f9145c67640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f91400cfe00 0x7f9140149dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:36356/0 (socket says 172.21.15.38:36356) 2024-09-21T03:39:35.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:35.147+0000 7f9145c67640 1 -- 172.21.15.38:0/2661419797 learned_addr learned my addr 172.21.15.38:0/2661419797 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:39:35.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:35.148+0000 7f9146468640 1 --2- 172.21.15.38:0/2661419797 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f91400b2e60 0x7f914015e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:35.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:35.148+0000 7f9146c69640 1 --2- 172.21.15.38:0/2661419797 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9140162590 0x7f9140160910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:35.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:35.148+0000 7f9145c67640 1 -- 172.21.15.38:0/2661419797 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9140162590 msgr2=0x7f9140160910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:35.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:35.148+0000 7f9145c67640 1 --2- 172.21.15.38:0/2661419797 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9140162590 0x7f9140160910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:35.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:35.148+0000 7f9145c67640 1 -- 172.21.15.38:0/2661419797 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f91400b2e60 msgr2=0x7f914015e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:35.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:35.148+0000 7f9145c67640 1 --2- 172.21.15.38:0/2661419797 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f91400b2e60 0x7f914015e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:35.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:35.148+0000 7f9145c67640 1 -- 172.21.15.38:0/2661419797 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91401478f0 con 0x7f91400cfe00 2024-09-21T03:39:35.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:35.148+0000 7f9146c69640 1 --2- 172.21.15.38:0/2661419797 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9140162590 0x7f9140160910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:39:35.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:35.148+0000 7f9145c67640 1 --2- 172.21.15.38:0/2661419797 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f91400cfe00 0x7f9140149dd0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f913000c430 tx=0x7f913000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:35.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:35.148+0000 7f9136ffd640 1 -- 172.21.15.38:0/2661419797 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9130017070 con 0x7f91400cfe00 2024-09-21T03:39:35.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:35.148+0000 7f9136ffd640 1 -- 172.21.15.38:0/2661419797 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f91300053c0 con 0x7f91400cfe00 2024-09-21T03:39:35.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:35.148+0000 7f9136ffd640 1 -- 172.21.15.38:0/2661419797 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9130003850 con 0x7f91400cfe00 2024-09-21T03:39:35.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:35.148+0000 7f91486cc640 1 -- 172.21.15.38:0/2661419797 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9140147b90 con 0x7f91400cfe00 2024-09-21T03:39:35.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:35.148+0000 7f91486cc640 1 -- 172.21.15.38:0/2661419797 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f914014a7b0 con 0x7f91400cfe00 2024-09-21T03:39:35.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:35.149+0000 7f91486cc640 1 -- 172.21.15.38:0/2661419797 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91400b3270 con 0x7f91400cfe00 2024-09-21T03:39:35.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:35.150+0000 7f9136ffd640 1 -- 172.21.15.38:0/2661419797 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f913001a070 con 0x7f91400cfe00 2024-09-21T03:39:35.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:35.151+0000 7f9136ffd640 1 --2- 172.21.15.38:0/2661419797 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9124078b90 0x7f912407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:35.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:35.151+0000 7f9136ffd640 1 -- 172.21.15.38:0/2661419797 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 6101+0+0 (secure 0 0 0) 0x7f913009a190 con 0x7f91400cfe00 2024-09-21T03:39:35.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:35.151+0000 7f9146468640 1 --2- 172.21.15.38:0/2661419797 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9124078b90 0x7f912407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:35.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:35.152+0000 7f9146468640 1 --2- 172.21.15.38:0/2661419797 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9124078b90 0x7f912407b0a0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f913c033000 tx=0x7f913c004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:35.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:35.155+0000 7f9136ffd640 1 -- 172.21.15.38:0/2661419797 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f9130061e80 con 0x7f91400cfe00 2024-09-21T03:39:35.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:35.385+0000 7f91486cc640 1 -- 172.21.15.38:0/2661419797 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f914014a540 con 0x7f91400cfe00 2024-09-21T03:39:35.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:35.386+0000 7f9136ffd640 1 -- 172.21.15.38:0/2661419797 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) ==== 94+0+5231 (secure 0 0 0) 0x7f91300668b0 con 0x7f91400cfe00 2024-09-21T03:39:35.388 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:39:35.388 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":68,"btime":"2024-09-21T03:39:18:466470+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5783,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/1358588311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1358588311},{"type":"v1","addr":"172.21.15.38:6835","nonce":1358588311}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63},{"gid":5789,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/1676483003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":1676483003},{"type":"v1","addr":"172.21.15.195:6837","nonce":1676483003}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63},{"gid":6193,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/2771781100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":2771781100},{"type":"v1","addr":"172.21.15.195:6836","nonce":2771781100}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:39:04.779948+0000","modified":"2024-09-21T03:39:17.658058+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6546},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6546":{"gid":6546,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":3,"addr":"172.21.15.38:6833/1144075315","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":1144075315},{"type":"v1","addr":"172.21.15.38:6833","nonce":1144075315}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6546,"qdb_cluster":[6546]},"id":5}]} 2024-09-21T03:39:35.388 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 68 2024-09-21T03:39:35.390 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:35.388+0000 7f91486cc640 1 -- 172.21.15.38:0/2661419797 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9124078b90 msgr2=0x7f912407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:35.391 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:35.388+0000 7f91486cc640 1 --2- 172.21.15.38:0/2661419797 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9124078b90 0x7f912407b0a0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f913c033000 tx=0x7f913c004730 comp rx=0 tx=0).stop 2024-09-21T03:39:35.391 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:35.388+0000 7f91486cc640 1 -- 172.21.15.38:0/2661419797 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f91400cfe00 msgr2=0x7f9140149dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:35.391 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:35.388+0000 7f91486cc640 1 --2- 172.21.15.38:0/2661419797 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f91400cfe00 0x7f9140149dd0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f913000c430 tx=0x7f913000c8f0 comp rx=0 tx=0).stop 2024-09-21T03:39:35.391 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:35.388+0000 7f91486cc640 1 -- 172.21.15.38:0/2661419797 shutdown_connections 2024-09-21T03:39:35.391 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:35.388+0000 7f91486cc640 1 --2- 172.21.15.38:0/2661419797 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9124078b90 0x7f912407b0a0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:35.391 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:35.388+0000 7f91486cc640 1 --2- 172.21.15.38:0/2661419797 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9140162590 0x7f9140160910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:35.391 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:35.388+0000 7f91486cc640 1 --2- 172.21.15.38:0/2661419797 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f91400cfe00 0x7f9140149dd0 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:35.391 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:35.388+0000 7f91486cc640 1 --2- 172.21.15.38:0/2661419797 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f91400b2e60 0x7f914015e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:35.391 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:35.388+0000 7f91486cc640 1 -- 172.21.15.38:0/2661419797 >> 172.21.15.38:0/2661419797 conn(0x7f91400d4440 msgr2=0x7f914006ddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:35.391 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:35.388+0000 7f91486cc640 1 -- 172.21.15.38:0/2661419797 shutdown_connections 2024-09-21T03:39:35.391 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:35.388+0000 7f91486cc640 1 -- 172.21.15.38:0/2661419797 wait complete. 2024-09-21T03:39:35.402 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2024-09-21T03:39:40.402 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:39:40.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:40.533+0000 7fe8145a1640 1 -- 172.21.15.38:0/1383585064 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe80c0b2e60 msgr2=0x7fe80c0b3270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:40.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:40.533+0000 7fe8145a1640 1 --2- 172.21.15.38:0/1383585064 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe80c0b2e60 0x7fe80c0b3270 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fe80c0cef90 tx=0x7fe80402a8b0 comp rx=0 tx=0).stop 2024-09-21T03:39:40.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:40.533+0000 7fe8145a1640 1 -- 172.21.15.38:0/1383585064 shutdown_connections 2024-09-21T03:39:40.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:40.533+0000 7fe8145a1640 1 --2- 172.21.15.38:0/1383585064 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe80c0b2e60 0x7fe80c0b3270 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:40.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:40.533+0000 7fe8145a1640 1 --2- 172.21.15.38:0/1383585064 >> v2:172.21.15.195:3300/0 conn(0x7fe80c162590 0x7fe80c1629a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:40.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:40.533+0000 7fe8145a1640 1 -- 172.21.15.38:0/1383585064 >> 172.21.15.38:0/1383585064 conn(0x7fe80c0d4440 msgr2=0x7fe80c0d4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:40.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:40.533+0000 7fe8145a1640 1 -- 172.21.15.38:0/1383585064 shutdown_connections 2024-09-21T03:39:40.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:40.533+0000 7fe8145a1640 1 -- 172.21.15.38:0/1383585064 wait complete. 2024-09-21T03:39:40.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:40.535+0000 7fe8145a1640 1 Processor -- start 2024-09-21T03:39:40.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:40.535+0000 7fe8145a1640 1 -- start start 2024-09-21T03:39:40.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:40.537+0000 7fe8145a1640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe80c0b2e60 0x7fe80c1e8ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:40.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:40.537+0000 7fe8145a1640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe80c0cfe00 0x7fe80c15b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:40.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:40.537+0000 7fe8145a1640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe80c162590 0x7fe80c14b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:40.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:40.537+0000 7fe8145a1640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fe80c0b3930 con 0x7fe80c0cfe00 2024-09-21T03:39:40.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:40.537+0000 7fe8145a1640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fe80c0b37b0 con 0x7fe80c162590 2024-09-21T03:39:40.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:40.537+0000 7fe8145a1640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fe80c0b3ab0 con 0x7fe80c0b2e60 2024-09-21T03:39:40.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:40.537+0000 7fe811b3c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe80c0cfe00 0x7fe80c15b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:40.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:40.537+0000 7fe811b3c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe80c0cfe00 0x7fe80c15b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:36382/0 (socket says 172.21.15.38:36382) 2024-09-21T03:39:40.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:40.537+0000 7fe811b3c640 1 -- 172.21.15.38:0/2721952212 learned_addr learned my addr 172.21.15.38:0/2721952212 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:39:40.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:40.537+0000 7fe812b3e640 1 --2- 172.21.15.38:0/2721952212 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe80c162590 0x7fe80c14b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:40.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:40.537+0000 7fe81233d640 1 --2- 172.21.15.38:0/2721952212 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe80c0b2e60 0x7fe80c1e8ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:40.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:40.537+0000 7fe811b3c640 1 -- 172.21.15.38:0/2721952212 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe80c0b2e60 msgr2=0x7fe80c1e8ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:40.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:40.537+0000 7fe811b3c640 1 --2- 172.21.15.38:0/2721952212 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe80c0b2e60 0x7fe80c1e8ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:40.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:40.537+0000 7fe811b3c640 1 -- 172.21.15.38:0/2721952212 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe80c162590 msgr2=0x7fe80c14b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:40.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:40.537+0000 7fe811b3c640 1 --2- 172.21.15.38:0/2721952212 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe80c162590 0x7fe80c14b620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:40.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:40.537+0000 7fe811b3c640 1 -- 172.21.15.38:0/2721952212 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe80c15beb0 con 0x7fe80c0cfe00 2024-09-21T03:39:40.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:40.537+0000 7fe81233d640 1 --2- 172.21.15.38:0/2721952212 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe80c0b2e60 0x7fe80c1e8ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:39:40.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:40.537+0000 7fe811b3c640 1 --2- 172.21.15.38:0/2721952212 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe80c0cfe00 0x7fe80c15b970 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fe7fc00e790 tx=0x7fe7fc00ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:40.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:40.540+0000 7fe802ffd640 1 -- 172.21.15.38:0/2721952212 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7fc00eea0 con 0x7fe80c0cfe00 2024-09-21T03:39:40.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:40.540+0000 7fe802ffd640 1 -- 172.21.15.38:0/2721952212 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe7fc00cdb0 con 0x7fe80c0cfe00 2024-09-21T03:39:40.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:40.540+0000 7fe8145a1640 1 -- 172.21.15.38:0/2721952212 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe80c15c3c0 con 0x7fe80c0cfe00 2024-09-21T03:39:40.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:40.540+0000 7fe802ffd640 1 -- 172.21.15.38:0/2721952212 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7fc010690 con 0x7fe80c0cfe00 2024-09-21T03:39:40.543 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:40.540+0000 7fe8145a1640 1 -- 172.21.15.38:0/2721952212 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe80c1e86e0 con 0x7fe80c0cfe00 2024-09-21T03:39:40.543 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:40.541+0000 7fe8145a1640 1 -- 172.21.15.38:0/2721952212 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe80c0b3270 con 0x7fe80c0cfe00 2024-09-21T03:39:40.544 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:40.542+0000 7fe802ffd640 1 -- 172.21.15.38:0/2721952212 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fe7fc010830 con 0x7fe80c0cfe00 2024-09-21T03:39:40.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:40.543+0000 7fe802ffd640 1 --2- 172.21.15.38:0/2721952212 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fe7f0078bc0 0x7fe7f007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:40.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:40.543+0000 7fe802ffd640 1 -- 172.21.15.38:0/2721952212 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 6101+0+0 (secure 0 0 0) 0x7fe7fc09b7d0 con 0x7fe80c0cfe00 2024-09-21T03:39:40.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:40.543+0000 7fe81233d640 1 --2- 172.21.15.38:0/2721952212 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fe7f0078bc0 0x7fe7f007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:40.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:40.544+0000 7fe81233d640 1 --2- 172.21.15.38:0/2721952212 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fe7f0078bc0 0x7fe7f007b0d0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fe8080013f0 tx=0x7fe808009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:40.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:40.548+0000 7fe802ffd640 1 -- 172.21.15.38:0/2721952212 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fe7fc063440 con 0x7fe80c0cfe00 2024-09-21T03:39:40.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:40.773+0000 7fe8145a1640 1 -- 172.21.15.38:0/2721952212 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe80c15c150 con 0x7fe80c0cfe00 2024-09-21T03:39:40.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:40.774+0000 7fe802ffd640 1 -- 172.21.15.38:0/2721952212 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) ==== 94+0+5279 (secure 0 0 0) 0x7fe7fc067e70 con 0x7fe80c0cfe00 2024-09-21T03:39:40.776 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:39:40.776 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":69,"btime":"2024-09-21T03:39:40:522063+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5783,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/1358588311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1358588311},{"type":"v1","addr":"172.21.15.38:6835","nonce":1358588311}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63},{"gid":5789,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/1676483003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":1676483003},{"type":"v1","addr":"172.21.15.195:6837","nonce":1676483003}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63},{"gid":6193,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/2771781100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":2771781100},{"type":"v1","addr":"172.21.15.195:6836","nonce":2771781100}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":69,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:39:04.779948+0000","modified":"2024-09-21T03:39:40.522031+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6546},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6546":{"gid":6546,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":3,"addr":"172.21.15.38:6833/1144075315","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":1144075315},{"type":"v1","addr":"172.21.15.38:6833","nonce":1144075315}]},"join_fscid":-1,"laggy_since":"2024-09-21T03:39:40.522030+0000","export_targets":[],"features":4540719139924082687,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6546,"qdb_cluster":[6546]},"id":5}]} 2024-09-21T03:39:40.776 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 69 2024-09-21T03:39:40.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:40.776+0000 7fe8145a1640 1 -- 172.21.15.38:0/2721952212 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fe7f0078bc0 msgr2=0x7fe7f007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:40.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:40.776+0000 7fe8145a1640 1 --2- 172.21.15.38:0/2721952212 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fe7f0078bc0 0x7fe7f007b0d0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fe8080013f0 tx=0x7fe808009040 comp rx=0 tx=0).stop 2024-09-21T03:39:40.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:40.776+0000 7fe8145a1640 1 -- 172.21.15.38:0/2721952212 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe80c0cfe00 msgr2=0x7fe80c15b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:40.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:40.776+0000 7fe8145a1640 1 --2- 172.21.15.38:0/2721952212 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe80c0cfe00 0x7fe80c15b970 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fe7fc00e790 tx=0x7fe7fc00ec50 comp rx=0 tx=0).stop 2024-09-21T03:39:40.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:40.776+0000 7fe8145a1640 1 -- 172.21.15.38:0/2721952212 shutdown_connections 2024-09-21T03:39:40.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:40.776+0000 7fe8145a1640 1 --2- 172.21.15.38:0/2721952212 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe80c162590 0x7fe80c14b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:40.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:40.776+0000 7fe8145a1640 1 --2- 172.21.15.38:0/2721952212 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe80c0cfe00 0x7fe80c15b970 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:40.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:40.776+0000 7fe8145a1640 1 --2- 172.21.15.38:0/2721952212 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fe7f0078bc0 0x7fe7f007b0d0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:40.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:40.776+0000 7fe8145a1640 1 --2- 172.21.15.38:0/2721952212 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe80c0b2e60 0x7fe80c1e8ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:40.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:40.776+0000 7fe8145a1640 1 -- 172.21.15.38:0/2721952212 >> 172.21.15.38:0/2721952212 conn(0x7fe80c0d4440 msgr2=0x7fe80c15e470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:40.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:40.776+0000 7fe8145a1640 1 -- 172.21.15.38:0/2721952212 shutdown_connections 2024-09-21T03:39:40.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:40.776+0000 7fe8145a1640 1 -- 172.21.15.38:0/2721952212 wait complete. 2024-09-21T03:39:40.790 DEBUG:tasks.ceph_test_case:wait_until_true: success in 15s and 0 retries 2024-09-21T03:39:40.809 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~13s 2024-09-21T03:39:40.891 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_inject_rename_corrupt_dentry_first 2024-09-21T03:39:41.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:41.007+0000 7f9a04e64640 1 -- 172.21.15.38:0/2085725655 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9a0015ca90 msgr2=0x7f9a000d0670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:41.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:41.007+0000 7f9a04e64640 1 --2- 172.21.15.38:0/2085725655 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9a0015ca90 0x7f9a000d0670 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f99f4005fd0 tx=0x7f99f402a5f0 comp rx=0 tx=0).stop 2024-09-21T03:39:41.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:41.008+0000 7f9a04e64640 1 -- 172.21.15.38:0/2085725655 shutdown_connections 2024-09-21T03:39:41.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:41.008+0000 7f9a04e64640 1 --2- 172.21.15.38:0/2085725655 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9a0015ca90 0x7f9a000d0670 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:41.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:41.008+0000 7f9a04e64640 1 -- 172.21.15.38:0/2085725655 >> 172.21.15.38:0/2085725655 conn(0x7f9a000b46e0 msgr2=0x7f9a000d23e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:41.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:41.008+0000 7f9a04e64640 1 -- 172.21.15.38:0/2085725655 shutdown_connections 2024-09-21T03:39:41.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:41.008+0000 7f9a04e64640 1 -- 172.21.15.38:0/2085725655 wait complete. 2024-09-21T03:39:41.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:41.010+0000 7f9a04e64640 1 Processor -- start 2024-09-21T03:39:41.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:41.010+0000 7f9a04e64640 1 -- start start 2024-09-21T03:39:41.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:41.011+0000 7f9a04e64640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9a000ccc40 0x7f9a00156df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:41.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:41.011+0000 7f9a04e64640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9a000d0bb0 0x7f9a000c7570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:41.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:41.011+0000 7f9a04e64640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9a0015ca90 0x7f9a000c7ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:41.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:41.011+0000 7f9a04e64640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f9a000b6840 con 0x7f9a000ccc40 2024-09-21T03:39:41.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:41.011+0000 7f9a04e64640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f9a000b66c0 con 0x7f9a000d0bb0 2024-09-21T03:39:41.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:41.011+0000 7f9a04e64640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f9a000b69c0 con 0x7f9a0015ca90 2024-09-21T03:39:41.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:41.011+0000 7f99fe59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9a000ccc40 0x7f9a00156df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:41.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:41.011+0000 7f99fe59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9a000ccc40 0x7f9a00156df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:36404/0 (socket says 172.21.15.38:36404) 2024-09-21T03:39:41.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:41.011+0000 7f99fe59c640 1 -- 172.21.15.38:0/49584940 learned_addr learned my addr 172.21.15.38:0/49584940 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:39:41.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:41.011+0000 7f99fdd9b640 1 --2- 172.21.15.38:0/49584940 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9a000d0bb0 0x7f9a000c7570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:41.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:41.011+0000 7f99fed9d640 1 --2- 172.21.15.38:0/49584940 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9a0015ca90 0x7f9a000c7ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:41.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:41.011+0000 7f99fe59c640 1 -- 172.21.15.38:0/49584940 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9a0015ca90 msgr2=0x7f9a000c7ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:41.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:41.011+0000 7f99fe59c640 1 --2- 172.21.15.38:0/49584940 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9a0015ca90 0x7f9a000c7ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:41.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:41.011+0000 7f99fe59c640 1 -- 172.21.15.38:0/49584940 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9a000d0bb0 msgr2=0x7f9a000c7570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:41.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:41.011+0000 7f99fe59c640 1 --2- 172.21.15.38:0/49584940 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9a000d0bb0 0x7f9a000c7570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:41.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:41.011+0000 7f99fe59c640 1 -- 172.21.15.38:0/49584940 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a0006feb0 con 0x7f9a000ccc40 2024-09-21T03:39:41.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:41.012+0000 7f99fe59c640 1 --2- 172.21.15.38:0/49584940 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9a000ccc40 0x7f9a00156df0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f99f000c430 tx=0x7f99f000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:41.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:41.012+0000 7f99eeffd640 1 -- 172.21.15.38:0/49584940 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99f0017070 con 0x7f9a000ccc40 2024-09-21T03:39:41.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:41.012+0000 7f99eeffd640 1 -- 172.21.15.38:0/49584940 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f99f00053c0 con 0x7f9a000ccc40 2024-09-21T03:39:41.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:41.012+0000 7f9a04e64640 1 -- 172.21.15.38:0/49584940 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a0006fbd0 con 0x7f9a000ccc40 2024-09-21T03:39:41.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:41.012+0000 7f99eeffd640 1 -- 172.21.15.38:0/49584940 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99f0003850 con 0x7f9a000ccc40 2024-09-21T03:39:41.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:41.012+0000 7f9a04e64640 1 -- 172.21.15.38:0/49584940 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a001f8f80 con 0x7f9a000ccc40 2024-09-21T03:39:41.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:41.013+0000 7f9a04e64640 1 -- 172.21.15.38:0/49584940 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a000d0670 con 0x7f9a000ccc40 2024-09-21T03:39:41.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:41.013+0000 7f99eeffd640 1 -- 172.21.15.38:0/49584940 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f99f001a070 con 0x7f9a000ccc40 2024-09-21T03:39:41.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:41.014+0000 7f99eeffd640 1 --2- 172.21.15.38:0/49584940 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f99e0078bc0 0x7f99e007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:41.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:41.014+0000 7f99eeffd640 1 -- 172.21.15.38:0/49584940 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 6101+0+0 (secure 0 0 0) 0x7f99f009a240 con 0x7f9a000ccc40 2024-09-21T03:39:41.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:41.014+0000 7f99fdd9b640 1 --2- 172.21.15.38:0/49584940 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f99e0078bc0 0x7f99e007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:41.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:41.015+0000 7f99fdd9b640 1 --2- 172.21.15.38:0/49584940 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f99e0078bc0 0x7f99e007b0d0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f99f4033000 tx=0x7f99f4004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:41.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:41.019+0000 7f99eeffd640 1 -- 172.21.15.38:0/49584940 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f9a000d0fc0 con 0x7f9a000ccc40 2024-09-21T03:39:41.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:41.207+0000 7f9a04e64640 1 -- 172.21.15.38:0/49584940 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_inject_rename_corrupt_dentry_first"} v 0) -- 0x7f9a00157330 con 0x7f9a000ccc40 2024-09-21T03:39:41.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:41.210+0000 7f99eeffd640 1 -- 172.21.15.38:0/49584940 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_inject_rename_corrupt_dentry_first"}]=0 v45) ==== 121+0+0 (secure 0 0 0) 0x7f99f0062580 con 0x7f9a000ccc40 2024-09-21T03:39:41.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:41.213+0000 7f9a04e64640 1 -- 172.21.15.38:0/49584940 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f99e0078bc0 msgr2=0x7f99e007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:41.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:41.213+0000 7f9a04e64640 1 --2- 172.21.15.38:0/49584940 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f99e0078bc0 0x7f99e007b0d0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f99f4033000 tx=0x7f99f4004730 comp rx=0 tx=0).stop 2024-09-21T03:39:41.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:41.213+0000 7f9a04e64640 1 -- 172.21.15.38:0/49584940 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9a000ccc40 msgr2=0x7f9a00156df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:41.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:41.213+0000 7f9a04e64640 1 --2- 172.21.15.38:0/49584940 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9a000ccc40 0x7f9a00156df0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f99f000c430 tx=0x7f99f000c8f0 comp rx=0 tx=0).stop 2024-09-21T03:39:41.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:41.213+0000 7f9a04e64640 1 -- 172.21.15.38:0/49584940 shutdown_connections 2024-09-21T03:39:41.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:41.213+0000 7f9a04e64640 1 --2- 172.21.15.38:0/49584940 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f99e0078bc0 0x7f99e007b0d0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:41.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:41.213+0000 7f9a04e64640 1 --2- 172.21.15.38:0/49584940 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9a0015ca90 0x7f9a000c7ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:41.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:41.213+0000 7f9a04e64640 1 --2- 172.21.15.38:0/49584940 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9a000d0bb0 0x7f9a000c7570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:41.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:41.213+0000 7f9a04e64640 1 --2- 172.21.15.38:0/49584940 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9a000ccc40 0x7f9a00156df0 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:41.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:41.213+0000 7f9a04e64640 1 -- 172.21.15.38:0/49584940 >> 172.21.15.38:0/49584940 conn(0x7f9a000b46e0 msgr2=0x7f9a000cd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:41.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:41.213+0000 7f9a04e64640 1 -- 172.21.15.38:0/49584940 shutdown_connections 2024-09-21T03:39:41.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:41.213+0000 7f9a04e64640 1 -- 172.21.15.38:0/49584940 wait complete. 2024-09-21T03:39:41.226 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds freeze 5:0 false 2024-09-21T03:39:41.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:41.344+0000 7f484179d640 1 -- 172.21.15.38:0/394866370 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f483c1606b0 msgr2=0x7f483c0705c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:41.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:41.344+0000 7f484179d640 1 --2- 172.21.15.38:0/394866370 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f483c1606b0 0x7f483c0705c0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f4834005fd0 tx=0x7f483402aa90 comp rx=0 tx=0).stop 2024-09-21T03:39:41.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:41.344+0000 7f484179d640 1 -- 172.21.15.38:0/394866370 shutdown_connections 2024-09-21T03:39:41.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:41.344+0000 7f484179d640 1 --2- 172.21.15.38:0/394866370 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f483c1606b0 0x7f483c0705c0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:41.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:41.344+0000 7f484179d640 1 --2- 172.21.15.38:0/394866370 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f483c0d0950 0x7f483c06f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:41.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:41.344+0000 7f484179d640 1 -- 172.21.15.38:0/394866370 >> 172.21.15.38:0/394866370 conn(0x7f483c0b2fd0 msgr2=0x7f483c0b33e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:41.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:41.344+0000 7f484179d640 1 -- 172.21.15.38:0/394866370 shutdown_connections 2024-09-21T03:39:41.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:41.344+0000 7f484179d640 1 -- 172.21.15.38:0/394866370 wait complete. 2024-09-21T03:39:41.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:41.346+0000 7f484179d640 1 Processor -- start 2024-09-21T03:39:41.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:41.346+0000 7f484179d640 1 -- start start 2024-09-21T03:39:41.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:41.347+0000 7f484179d640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f483c0cd860 0x7f483c0cc9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:41.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:41.347+0000 7f484179d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f483c0d0950 0x7f483c0c8380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:41.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:41.347+0000 7f484179d640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f483c1f6040 0x7f483c1f6450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:41.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:41.347+0000 7f484179d640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f483c071000 con 0x7f483c0d0950 2024-09-21T03:39:41.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:41.347+0000 7f484179d640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f483c070e80 con 0x7f483c1f6040 2024-09-21T03:39:41.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:41.347+0000 7f484179d640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f483c071180 con 0x7f483c0cd860 2024-09-21T03:39:41.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:41.347+0000 7f483b7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f483c0d0950 0x7f483c0c8380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:41.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:41.347+0000 7f483b7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f483c0d0950 0x7f483c0c8380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:36408/0 (socket says 172.21.15.38:36408) 2024-09-21T03:39:41.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:41.347+0000 7f483b7fe640 1 -- 172.21.15.38:0/2087843476 learned_addr learned my addr 172.21.15.38:0/2087843476 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:39:41.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:41.347+0000 7f483bfff640 1 --2- 172.21.15.38:0/2087843476 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f483c0cd860 0x7f483c0cc9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:41.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:41.348+0000 7f4840f9c640 1 --2- 172.21.15.38:0/2087843476 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f483c1f6040 0x7f483c1f6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:41.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:41.348+0000 7f483b7fe640 1 -- 172.21.15.38:0/2087843476 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f483c0cd860 msgr2=0x7f483c0cc9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:41.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:41.348+0000 7f483b7fe640 1 --2- 172.21.15.38:0/2087843476 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f483c0cd860 0x7f483c0cc9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:41.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:41.348+0000 7f483b7fe640 1 -- 172.21.15.38:0/2087843476 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f483c1f6040 msgr2=0x7f483c1f6450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:41.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:41.348+0000 7f483b7fe640 1 --2- 172.21.15.38:0/2087843476 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f483c1f6040 0x7f483c1f6450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:41.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:41.348+0000 7f483b7fe640 1 -- 172.21.15.38:0/2087843476 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f483c1f6990 con 0x7f483c0d0950 2024-09-21T03:39:41.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:41.348+0000 7f483bfff640 1 --2- 172.21.15.38:0/2087843476 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f483c0cd860 0x7f483c0cc9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:39:41.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:41.348+0000 7f483b7fe640 1 --2- 172.21.15.38:0/2087843476 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f483c0d0950 0x7f483c0c8380 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f483000c3a0 tx=0x7f483000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:41.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:41.348+0000 7f4838ff9640 1 -- 172.21.15.38:0/2087843476 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4830017070 con 0x7f483c0d0950 2024-09-21T03:39:41.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:41.348+0000 7f484179d640 1 -- 172.21.15.38:0/2087843476 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f483c1f3b80 con 0x7f483c0d0950 2024-09-21T03:39:41.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:41.348+0000 7f4838ff9640 1 -- 172.21.15.38:0/2087843476 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f48300053c0 con 0x7f483c0d0950 2024-09-21T03:39:41.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:41.348+0000 7f4838ff9640 1 -- 172.21.15.38:0/2087843476 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4830003850 con 0x7f483c0d0950 2024-09-21T03:39:41.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:41.348+0000 7f484179d640 1 -- 172.21.15.38:0/2087843476 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f483c1f4140 con 0x7f483c0d0950 2024-09-21T03:39:41.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:41.349+0000 7f484179d640 1 -- 172.21.15.38:0/2087843476 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f483c071630 con 0x7f483c0d0950 2024-09-21T03:39:41.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:41.350+0000 7f4838ff9640 1 -- 172.21.15.38:0/2087843476 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f483001a070 con 0x7f483c0d0950 2024-09-21T03:39:41.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:41.351+0000 7f4838ff9640 1 --2- 172.21.15.38:0/2087843476 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4828078b90 0x7f482807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:41.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:41.351+0000 7f4838ff9640 1 -- 172.21.15.38:0/2087843476 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 6101+0+0 (secure 0 0 0) 0x7f483009a210 con 0x7f483c0d0950 2024-09-21T03:39:41.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:41.351+0000 7f483bfff640 1 --2- 172.21.15.38:0/2087843476 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4828078b90 0x7f482807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:41.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:41.351+0000 7f483bfff640 1 --2- 172.21.15.38:0/2087843476 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4828078b90 0x7f482807b0a0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f482c009b80 tx=0x7f482c005f90 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:41.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:41.355+0000 7f4838ff9640 1 -- 172.21.15.38:0/2087843476 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f483c1f6990 con 0x7f483c0d0950 2024-09-21T03:39:41.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:41.619+0000 7f484179d640 1 -- 172.21.15.38:0/2087843476 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "mds freeze", "role_or_gid": "5:0", "val": "false"} v 0) -- 0x7f483c15bf60 con 0x7f483c0d0950 2024-09-21T03:39:42.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:42.553+0000 7f4838ff9640 1 -- 172.21.15.38:0/2087843476 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "mds freeze", "role_or_gid": "5:0", "val": "false"}]=0 unfreezing mds.6546 v70) ==== 115+0+0 (secure 0 0 0) 0x7f483c0ce8e0 con 0x7f483c0d0950 2024-09-21T03:39:42.555 INFO:teuthology.orchestra.run.smithi038.stderr:unfreezing mds.6546 2024-09-21T03:39:42.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:42.556+0000 7f484179d640 1 -- 172.21.15.38:0/2087843476 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4828078b90 msgr2=0x7f482807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:42.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:42.556+0000 7f484179d640 1 --2- 172.21.15.38:0/2087843476 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4828078b90 0x7f482807b0a0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f482c009b80 tx=0x7f482c005f90 comp rx=0 tx=0).stop 2024-09-21T03:39:42.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:42.556+0000 7f484179d640 1 -- 172.21.15.38:0/2087843476 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f483c0d0950 msgr2=0x7f483c0c8380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:42.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:42.556+0000 7f484179d640 1 --2- 172.21.15.38:0/2087843476 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f483c0d0950 0x7f483c0c8380 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f483000c3a0 tx=0x7f483000c860 comp rx=0 tx=0).stop 2024-09-21T03:39:42.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:42.556+0000 7f484179d640 1 -- 172.21.15.38:0/2087843476 shutdown_connections 2024-09-21T03:39:42.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:42.556+0000 7f484179d640 1 --2- 172.21.15.38:0/2087843476 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f483c1f6040 0x7f483c1f6450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:42.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:42.556+0000 7f484179d640 1 --2- 172.21.15.38:0/2087843476 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f483c0d0950 0x7f483c0c8380 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:42.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:42.556+0000 7f484179d640 1 --2- 172.21.15.38:0/2087843476 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4828078b90 0x7f482807b0a0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:42.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:42.556+0000 7f484179d640 1 --2- 172.21.15.38:0/2087843476 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f483c0cd860 0x7f483c0cc9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:42.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:42.556+0000 7f484179d640 1 -- 172.21.15.38:0/2087843476 >> 172.21.15.38:0/2087843476 conn(0x7f483c0b2fd0 msgr2=0x7f483c15d9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:42.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:42.556+0000 7f484179d640 1 -- 172.21.15.38:0/2087843476 shutdown_connections 2024-09-21T03:39:42.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:42.556+0000 7f484179d640 1 -- 172.21.15.38:0/2087843476 wait complete. 2024-09-21T03:39:42.577 DEBUG:teuthology.orchestra.run.smithi038:> sudo sysctl -n kernel.core_pattern 2024-09-21T03:39:42.603 INFO:teuthology.orchestra.run.smithi038.stdout:/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T03:39:42.605 INFO:tasks.cephfs.cephfs_test_case:Clearing core from directory: /home/ubuntu/cephtest/archive/coredump 2024-09-21T03:39:42.605 DEBUG:teuthology.orchestra.run.smithi038:> cd /home/ubuntu/cephtest/archive/coredump && sudo ls | sudo xargs file 2024-09-21T03:39:42.676 INFO:teuthology.orchestra.run.smithi038.stdout:1726889964.60258.core: ELF 64-bit LSB core file, x86-64, version 1 (SYSV), SVR4-style, from 'ceph-mds -f --cluster ceph -i c', real uid: 0, effective uid: 0, real gid: 0, effective gid: 0, execfn: '/bin/ceph-mds', platform: 'x86_64' 2024-09-21T03:39:42.678 INFO:tasks.cephfs.cephfs_test_case:Enumerated cores: ['1726889964.60258.core'] 2024-09-21T03:39:42.679 INFO:tasks.cephfs.cephfs_test_case:Found core file 1726889964.60258.core, deleting it 2024-09-21T03:39:42.679 DEBUG:teuthology.orchestra.run.smithi038:> cd /home/ubuntu/cephtest/archive/coredump && sudo rm -f 1726889964.60258.core 2024-09-21T03:39:42.756 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-21T03:39:42.756 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-21T03:39:42.756 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-21T03:39:42.756 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T03:39:42.757 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T03:39:42.757 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/daemon/state.py", line 146, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi038 with status 1: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c' 2024-09-21T03:39:42.757 INFO:tasks.ceph.mds.c:Stopped 2024-09-21T03:39:42.757 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-21T03:39:42.799 INFO:tasks.ceph.mds.c:Started 2024-09-21T03:39:42.799 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:39:42.887 INFO:tasks.ceph.mds.c.smithi038.stdout:starting mds.c at 2024-09-21T03:39:42.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:42.915+0000 7fe4a2141640 1 -- 172.21.15.38:0/1256570521 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe49c1536e0 msgr2=0x7fe49c0d17c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:42.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:42.915+0000 7fe4a2141640 1 --2- 172.21.15.38:0/1256570521 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe49c1536e0 0x7fe49c0d17c0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7fe49c0b2db0 tx=0x7fe48c02a580 comp rx=0 tx=0).stop 2024-09-21T03:39:42.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:42.915+0000 7fe4a2141640 1 -- 172.21.15.38:0/1256570521 shutdown_connections 2024-09-21T03:39:42.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:42.915+0000 7fe4a2141640 1 --2- 172.21.15.38:0/1256570521 >> v2:172.21.15.195:3300/0 conn(0x7fe49c0cd040 0x7fe49c0d04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:42.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:42.915+0000 7fe4a2141640 1 --2- 172.21.15.38:0/1256570521 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe49c1536e0 0x7fe49c0d17c0 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:42.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:42.915+0000 7fe4a2141640 1 -- 172.21.15.38:0/1256570521 >> 172.21.15.38:0/1256570521 conn(0x7fe49c0cef40 msgr2=0x7fe49c0cf350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:42.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:42.916+0000 7fe4a2141640 1 -- 172.21.15.38:0/1256570521 shutdown_connections 2024-09-21T03:39:42.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:42.916+0000 7fe4a2141640 1 -- 172.21.15.38:0/1256570521 wait complete. 2024-09-21T03:39:42.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:42.917+0000 7fe4a2141640 1 Processor -- start 2024-09-21T03:39:42.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:42.917+0000 7fe4a2141640 1 -- start start 2024-09-21T03:39:42.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:42.919+0000 7fe4a2141640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe49c0d1d00 0x7fe49c152670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:42.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:42.919+0000 7fe4a2141640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe49c1536e0 0x7fe49c1e7300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:42.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:42.919+0000 7fe4a2141640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe49c1e7840 0x7fe49c0cb7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:42.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:42.919+0000 7fe4a2141640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fe49c0d0d90 con 0x7fe49c1536e0 2024-09-21T03:39:42.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:42.919+0000 7fe4a2141640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fe49c0d0c10 con 0x7fe49c1e7840 2024-09-21T03:39:42.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:42.919+0000 7fe4a2141640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fe49c0d0f10 con 0x7fe49c0d1d00 2024-09-21T03:39:42.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:42.919+0000 7fe492ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe49c1536e0 0x7fe49c1e7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:42.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:42.919+0000 7fe492ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe49c1536e0 0x7fe49c1e7300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42230/0 (socket says 172.21.15.38:42230) 2024-09-21T03:39:42.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:42.919+0000 7fe492ffd640 1 -- 172.21.15.38:0/1723379799 learned_addr learned my addr 172.21.15.38:0/1723379799 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:39:42.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:42.919+0000 7fe49b7fe640 1 --2- 172.21.15.38:0/1723379799 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe49c0d1d00 0x7fe49c152670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:42.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:42.919+0000 7fe49bfff640 1 --2- 172.21.15.38:0/1723379799 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe49c1e7840 0x7fe49c0cb7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:42.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:42.919+0000 7fe492ffd640 1 -- 172.21.15.38:0/1723379799 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe49c0d1d00 msgr2=0x7fe49c152670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:42.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:42.919+0000 7fe492ffd640 1 --2- 172.21.15.38:0/1723379799 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe49c0d1d00 0x7fe49c152670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:42.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:42.919+0000 7fe492ffd640 1 -- 172.21.15.38:0/1723379799 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe49c1e7840 msgr2=0x7fe49c0cb7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:42.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:42.919+0000 7fe492ffd640 1 --2- 172.21.15.38:0/1723379799 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe49c1e7840 0x7fe49c0cb7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:42.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:42.919+0000 7fe492ffd640 1 -- 172.21.15.38:0/1723379799 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe49c1f7610 con 0x7fe49c1536e0 2024-09-21T03:39:42.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:42.920+0000 7fe492ffd640 1 --2- 172.21.15.38:0/1723379799 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe49c1536e0 0x7fe49c1e7300 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7fe48400b3e0 tx=0x7fe48400b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:42.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:42.920+0000 7fe498ff9640 1 -- 172.21.15.38:0/1723379799 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe484015020 con 0x7fe49c1536e0 2024-09-21T03:39:42.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:42.920+0000 7fe498ff9640 1 -- 172.21.15.38:0/1723379799 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe484011940 con 0x7fe49c1536e0 2024-09-21T03:39:42.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:42.920+0000 7fe498ff9640 1 -- 172.21.15.38:0/1723379799 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe484011d90 con 0x7fe49c1536e0 2024-09-21T03:39:42.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:42.920+0000 7fe4a2141640 1 -- 172.21.15.38:0/1723379799 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe49c1f78f0 con 0x7fe49c1536e0 2024-09-21T03:39:42.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:42.920+0000 7fe4a2141640 1 -- 172.21.15.38:0/1723379799 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe49c1f7eb0 con 0x7fe49c1536e0 2024-09-21T03:39:42.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:42.921+0000 7fe4a2141640 1 -- 172.21.15.38:0/1723379799 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe49c0d04c0 con 0x7fe49c1536e0 2024-09-21T03:39:42.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:42.922+0000 7fe498ff9640 1 -- 172.21.15.38:0/1723379799 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fe484018020 con 0x7fe49c1536e0 2024-09-21T03:39:42.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:42.922+0000 7fe498ff9640 1 --2- 172.21.15.38:0/1723379799 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fe47c078b90 0x7fe47c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:42.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:42.922+0000 7fe498ff9640 1 -- 172.21.15.38:0/1723379799 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 6187+0+0 (secure 0 0 0) 0x7fe48409ae20 con 0x7fe49c1536e0 2024-09-21T03:39:42.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:42.922+0000 7fe49b7fe640 1 --2- 172.21.15.38:0/1723379799 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fe47c078b90 0x7fe47c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:42.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:42.923+0000 7fe49b7fe640 1 --2- 172.21.15.38:0/1723379799 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fe47c078b90 0x7fe47c07b0a0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fe48c033000 tx=0x7fe48c004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:42.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:42.927+0000 7fe498ff9640 1 -- 172.21.15.38:0/1723379799 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fe48401f020 con 0x7fe49c1536e0 2024-09-21T03:39:43.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:43.154+0000 7fe4a2141640 1 -- 172.21.15.38:0/1723379799 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe49c152f20 con 0x7fe49c1536e0 2024-09-21T03:39:43.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:43.154+0000 7fe498ff9640 1 -- 172.21.15.38:0/1723379799 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) ==== 94+0+4400 (secure 0 0 0) 0x7fe4840629b0 con 0x7fe49c1536e0 2024-09-21T03:39:43.156 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:39:43.156 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":71,"btime":"2024-09-21T03:39:42:557064+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5783,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/1358588311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1358588311},{"type":"v1","addr":"172.21.15.38:6835","nonce":1358588311}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63},{"gid":5789,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/1676483003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":1676483003},{"type":"v1","addr":"172.21.15.195:6837","nonce":1676483003}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:39:04.779948+0000","modified":"2024-09-21T03:39:42.557060+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6193},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6193":{"gid":6193,"name":"d","rank":0,"incarnation":71,"state":"up:replay","state_seq":1,"addr":"172.21.15.195:6836/2771781100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":2771781100},{"type":"v1","addr":"172.21.15.195:6836","nonce":2771781100}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":5}]} 2024-09-21T03:39:43.157 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 71 2024-09-21T03:39:43.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:43.157+0000 7fe4a2141640 1 -- 172.21.15.38:0/1723379799 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fe47c078b90 msgr2=0x7fe47c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:43.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:43.157+0000 7fe4a2141640 1 --2- 172.21.15.38:0/1723379799 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fe47c078b90 0x7fe47c07b0a0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fe48c033000 tx=0x7fe48c004730 comp rx=0 tx=0).stop 2024-09-21T03:39:43.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:43.157+0000 7fe4a2141640 1 -- 172.21.15.38:0/1723379799 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe49c1536e0 msgr2=0x7fe49c1e7300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:43.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:43.157+0000 7fe4a2141640 1 --2- 172.21.15.38:0/1723379799 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe49c1536e0 0x7fe49c1e7300 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7fe48400b3e0 tx=0x7fe48400b8a0 comp rx=0 tx=0).stop 2024-09-21T03:39:43.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:43.157+0000 7fe4a2141640 1 -- 172.21.15.38:0/1723379799 shutdown_connections 2024-09-21T03:39:43.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:43.157+0000 7fe4a2141640 1 --2- 172.21.15.38:0/1723379799 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe49c1e7840 0x7fe49c0cb7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:43.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:43.157+0000 7fe4a2141640 1 --2- 172.21.15.38:0/1723379799 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe49c1536e0 0x7fe49c1e7300 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:43.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:43.157+0000 7fe4a2141640 1 --2- 172.21.15.38:0/1723379799 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fe47c078b90 0x7fe47c07b0a0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:43.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:43.157+0000 7fe4a2141640 1 --2- 172.21.15.38:0/1723379799 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe49c0d1d00 0x7fe49c152670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:43.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:43.157+0000 7fe4a2141640 1 -- 172.21.15.38:0/1723379799 >> 172.21.15.38:0/1723379799 conn(0x7fe49c0cef40 msgr2=0x7fe49c0cf350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:43.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:43.157+0000 7fe4a2141640 1 -- 172.21.15.38:0/1723379799 shutdown_connections 2024-09-21T03:39:43.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:43.157+0000 7fe4a2141640 1 -- 172.21.15.38:0/1723379799 wait complete. 2024-09-21T03:39:43.572 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 71, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T03:39:04.779948+0000', 'modified': '2024-09-21T03:39:42.557060+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 83, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6193}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6193': {'gid': 6193, 'name': 'd', 'rank': 0, 'incarnation': 71, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.195:6836/2771781100', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6834', 'nonce': 2771781100}, {'type': 'v1', 'addr': '172.21.15.195:6836', 'nonce': 2771781100}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-09-21T03:39:43.573 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6193:up:replay 2024-09-21T03:39:44.574 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:39:44.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:44.703+0000 7fc434182640 1 -- 172.21.15.38:0/610770072 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc42c162f00 msgr2=0x7fc42c0d24d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:44.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:44.703+0000 7fc434182640 1 --2- 172.21.15.38:0/610770072 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc42c162f00 0x7fc42c0d24d0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7fc428009830 tx=0x7fc42802a530 comp rx=0 tx=0).stop 2024-09-21T03:39:44.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:44.703+0000 7fc434182640 1 -- 172.21.15.38:0/610770072 shutdown_connections 2024-09-21T03:39:44.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:44.703+0000 7fc434182640 1 --2- 172.21.15.38:0/610770072 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc42c162f00 0x7fc42c0d24d0 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:44.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:44.703+0000 7fc434182640 1 -- 172.21.15.38:0/610770072 >> 172.21.15.38:0/610770072 conn(0x7fc42c0b6b20 msgr2=0x7fc42c0d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:44.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:44.703+0000 7fc434182640 1 -- 172.21.15.38:0/610770072 shutdown_connections 2024-09-21T03:39:44.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:44.703+0000 7fc434182640 1 -- 172.21.15.38:0/610770072 wait complete. 2024-09-21T03:39:44.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:44.705+0000 7fc434182640 1 Processor -- start 2024-09-21T03:39:44.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:44.705+0000 7fc434182640 1 -- start start 2024-09-21T03:39:44.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:44.706+0000 7fc434182640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc42c0b2e40 0x7fc42c157800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:44.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:44.706+0000 7fc434182640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc42c162280 0x7fc42c1f8760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:44.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:44.706+0000 7fc434182640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc42c1f8ca0 0x7fc42c1f90b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:44.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:44.706+0000 7fc434182640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fc42c15f6d0 con 0x7fc42c0b2e40 2024-09-21T03:39:44.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:44.706+0000 7fc434182640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fc42c0d2c00 con 0x7fc42c1f8ca0 2024-09-21T03:39:44.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:44.706+0000 7fc434182640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fc42c15f850 con 0x7fc42c162280 2024-09-21T03:39:44.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:44.707+0000 7fc431f1e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc42c0b2e40 0x7fc42c157800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:44.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:44.707+0000 7fc431f1e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc42c0b2e40 0x7fc42c157800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42250/0 (socket says 172.21.15.38:42250) 2024-09-21T03:39:44.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:44.707+0000 7fc431f1e640 1 -- 172.21.15.38:0/1625362230 learned_addr learned my addr 172.21.15.38:0/1625362230 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:39:44.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:44.707+0000 7fc43271f640 1 --2- 172.21.15.38:0/1625362230 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc42c1f8ca0 0x7fc42c1f90b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:44.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:44.707+0000 7fc43171d640 1 --2- 172.21.15.38:0/1625362230 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc42c162280 0x7fc42c1f8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:44.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:44.707+0000 7fc431f1e640 1 -- 172.21.15.38:0/1625362230 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc42c162280 msgr2=0x7fc42c1f8760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:44.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:44.707+0000 7fc431f1e640 1 --2- 172.21.15.38:0/1625362230 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc42c162280 0x7fc42c1f8760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:44.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:44.707+0000 7fc431f1e640 1 -- 172.21.15.38:0/1625362230 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc42c1f8ca0 msgr2=0x7fc42c1f90b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:44.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:44.707+0000 7fc431f1e640 1 --2- 172.21.15.38:0/1625362230 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc42c1f8ca0 0x7fc42c1f90b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:44.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:44.707+0000 7fc431f1e640 1 -- 172.21.15.38:0/1625362230 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc42c0aadc0 con 0x7fc42c0b2e40 2024-09-21T03:39:44.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:44.707+0000 7fc431f1e640 1 --2- 172.21.15.38:0/1625362230 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc42c0b2e40 0x7fc42c157800 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7fc428009950 tx=0x7fc428033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:44.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:44.707+0000 7fc4227fc640 1 -- 172.21.15.38:0/1625362230 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc428031d60 con 0x7fc42c0b2e40 2024-09-21T03:39:44.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:44.707+0000 7fc4227fc640 1 -- 172.21.15.38:0/1625362230 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc428033e10 con 0x7fc42c0b2e40 2024-09-21T03:39:44.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:44.707+0000 7fc434182640 1 -- 172.21.15.38:0/1625362230 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc42c0ab0a0 con 0x7fc42c0b2e40 2024-09-21T03:39:44.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:44.707+0000 7fc4227fc640 1 -- 172.21.15.38:0/1625362230 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc428033310 con 0x7fc42c0b2e40 2024-09-21T03:39:44.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:44.708+0000 7fc434182640 1 -- 172.21.15.38:0/1625362230 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc42c0ab660 con 0x7fc42c0b2e40 2024-09-21T03:39:44.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:44.708+0000 7fc434182640 1 -- 172.21.15.38:0/1625362230 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc42c162f00 con 0x7fc42c0b2e40 2024-09-21T03:39:44.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:44.709+0000 7fc4227fc640 1 -- 172.21.15.38:0/1625362230 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc4280334b0 con 0x7fc42c0b2e40 2024-09-21T03:39:44.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:44.710+0000 7fc4227fc640 1 --2- 172.21.15.38:0/1625362230 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc410078bc0 0x7fc41007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:44.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:44.710+0000 7fc4227fc640 1 -- 172.21.15.38:0/1625362230 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 6187+0+0 (secure 0 0 0) 0x7fc4280bc1e0 con 0x7fc42c0b2e40 2024-09-21T03:39:44.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:44.710+0000 7fc43171d640 1 --2- 172.21.15.38:0/1625362230 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc410078bc0 0x7fc41007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:44.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:44.711+0000 7fc43171d640 1 --2- 172.21.15.38:0/1625362230 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc410078bc0 0x7fc41007b0d0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fc42c0ba4f0 tx=0x7fc41c009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:44.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:44.714+0000 7fc4227fc640 1 -- 172.21.15.38:0/1625362230 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fc42803f030 con 0x7fc42c0b2e40 2024-09-21T03:39:44.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:44.943+0000 7fc434182640 1 -- 172.21.15.38:0/1625362230 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc42c161dd0 con 0x7fc42c0b2e40 2024-09-21T03:39:44.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:44.944+0000 7fc4227fc640 1 -- 172.21.15.38:0/1625362230 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) ==== 94+0+5226 (secure 0 0 0) 0x7fc428083df0 con 0x7fc42c0b2e40 2024-09-21T03:39:44.946 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:39:44.946 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":73,"btime":"2024-09-21T03:39:44:564346+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5783,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/1358588311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1358588311},{"type":"v1","addr":"172.21.15.38:6835","nonce":1358588311}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63},{"gid":5789,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/1676483003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":1676483003},{"type":"v1","addr":"172.21.15.195:6837","nonce":1676483003}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63},{"gid":6813,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/3644074938","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":3644074938},{"type":"v1","addr":"172.21.15.38:6833","nonce":3644074938}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":72}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:39:04.779948+0000","modified":"2024-09-21T03:39:43.567214+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6193},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6193":{"gid":6193,"name":"d","rank":0,"incarnation":71,"state":"up:rejoin","state_seq":13,"addr":"172.21.15.195:6836/2771781100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":2771781100},{"type":"v1","addr":"172.21.15.195:6836","nonce":2771781100}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":5}]} 2024-09-21T03:39:44.946 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 73 2024-09-21T03:39:44.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:44.946+0000 7fc434182640 1 -- 172.21.15.38:0/1625362230 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc410078bc0 msgr2=0x7fc41007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:44.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:44.946+0000 7fc434182640 1 --2- 172.21.15.38:0/1625362230 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc410078bc0 0x7fc41007b0d0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fc42c0ba4f0 tx=0x7fc41c009040 comp rx=0 tx=0).stop 2024-09-21T03:39:44.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:44.946+0000 7fc434182640 1 -- 172.21.15.38:0/1625362230 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc42c0b2e40 msgr2=0x7fc42c157800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:44.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:44.946+0000 7fc434182640 1 --2- 172.21.15.38:0/1625362230 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc42c0b2e40 0x7fc42c157800 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7fc428009950 tx=0x7fc428033c20 comp rx=0 tx=0).stop 2024-09-21T03:39:44.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:44.946+0000 7fc434182640 1 -- 172.21.15.38:0/1625362230 shutdown_connections 2024-09-21T03:39:44.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:44.946+0000 7fc434182640 1 --2- 172.21.15.38:0/1625362230 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc42c1f8ca0 0x7fc42c1f90b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:44.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:44.946+0000 7fc434182640 1 --2- 172.21.15.38:0/1625362230 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc410078bc0 0x7fc41007b0d0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:44.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:44.946+0000 7fc434182640 1 --2- 172.21.15.38:0/1625362230 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc42c162280 0x7fc42c1f8760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:44.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:44.946+0000 7fc434182640 1 --2- 172.21.15.38:0/1625362230 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc42c0b2e40 0x7fc42c157800 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:44.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:44.946+0000 7fc434182640 1 -- 172.21.15.38:0/1625362230 >> 172.21.15.38:0/1625362230 conn(0x7fc42c0b6b20 msgr2=0x7fc42c15e4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:44.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:44.946+0000 7fc434182640 1 -- 172.21.15.38:0/1625362230 shutdown_connections 2024-09-21T03:39:44.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:44.946+0000 7fc434182640 1 -- 172.21.15.38:0/1625362230 wait complete. 2024-09-21T03:39:45.362 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 73, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T03:39:04.779948+0000', 'modified': '2024-09-21T03:39:43.567214+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 83, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6193}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6193': {'gid': 6193, 'name': 'd', 'rank': 0, 'incarnation': 71, 'state': 'up:rejoin', 'state_seq': 13, 'addr': '172.21.15.195:6836/2771781100', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6834', 'nonce': 2771781100}, {'type': 'v1', 'addr': '172.21.15.195:6836', 'nonce': 2771781100}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-09-21T03:39:45.362 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6193:up:rejoin 2024-09-21T03:39:45.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:39:46.363 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:39:46.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:46.493+0000 7f3a8d6d3640 1 -- 172.21.15.38:0/2297099576 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3a880d0b40 msgr2=0x7f3a8806e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:46.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:46.493+0000 7f3a8d6d3640 1 --2- 172.21.15.38:0/2297099576 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3a880d0b40 0x7f3a8806e6f0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f3a88074d30 tx=0x7f3a7802a5f0 comp rx=0 tx=0).stop 2024-09-21T03:39:46.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:46.493+0000 7f3a8d6d3640 1 -- 172.21.15.38:0/2297099576 shutdown_connections 2024-09-21T03:39:46.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:46.493+0000 7f3a8d6d3640 1 --2- 172.21.15.38:0/2297099576 >> v2:172.21.15.195:3300/0 conn(0x7f3a88073d50 0x7f3a88074160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:46.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:46.493+0000 7f3a8d6d3640 1 --2- 172.21.15.38:0/2297099576 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3a880d0b40 0x7f3a8806e6f0 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:46.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:46.493+0000 7f3a8d6d3640 1 -- 172.21.15.38:0/2297099576 >> 172.21.15.38:0/2297099576 conn(0x7f3a880b2cc0 msgr2=0x7f3a880b30d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:46.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:46.493+0000 7f3a8d6d3640 1 -- 172.21.15.38:0/2297099576 shutdown_connections 2024-09-21T03:39:46.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:46.493+0000 7f3a8d6d3640 1 -- 172.21.15.38:0/2297099576 wait complete. 2024-09-21T03:39:46.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:46.495+0000 7f3a8d6d3640 1 Processor -- start 2024-09-21T03:39:46.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:46.495+0000 7f3a8d6d3640 1 -- start start 2024-09-21T03:39:46.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:46.496+0000 7f3a8d6d3640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3a8806ec30 0x7f3a8815e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:46.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:46.496+0000 7f3a8d6d3640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3a88073d50 0x7f3a881505a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:46.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:46.496+0000 7f3a8d6d3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3a880d0b40 0x7f3a880c7ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:46.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:46.496+0000 7f3a8d6d3640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f3a8815b750 con 0x7f3a880d0b40 2024-09-21T03:39:46.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:46.496+0000 7f3a8d6d3640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f3a8815b5d0 con 0x7f3a88073d50 2024-09-21T03:39:46.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:46.496+0000 7f3a8d6d3640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f3a8815b8d0 con 0x7f3a8806ec30 2024-09-21T03:39:46.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:46.496+0000 7f3a877fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3a880d0b40 0x7f3a880c7ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:46.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:46.496+0000 7f3a877fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3a880d0b40 0x7f3a880c7ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42258/0 (socket says 172.21.15.38:42258) 2024-09-21T03:39:46.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:46.496+0000 7f3a877fe640 1 -- 172.21.15.38:0/2082161134 learned_addr learned my addr 172.21.15.38:0/2082161134 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:39:46.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:46.496+0000 7f3a7ffff640 1 --2- 172.21.15.38:0/2082161134 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3a88073d50 0x7f3a881505a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:46.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:46.496+0000 7f3a86ffd640 1 --2- 172.21.15.38:0/2082161134 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3a8806ec30 0x7f3a8815e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:46.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:46.496+0000 7f3a877fe640 1 -- 172.21.15.38:0/2082161134 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3a8806ec30 msgr2=0x7f3a8815e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:46.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:46.496+0000 7f3a877fe640 1 --2- 172.21.15.38:0/2082161134 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3a8806ec30 0x7f3a8815e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:46.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:46.496+0000 7f3a877fe640 1 -- 172.21.15.38:0/2082161134 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3a88073d50 msgr2=0x7f3a881505a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:46.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:46.496+0000 7f3a877fe640 1 --2- 172.21.15.38:0/2082161134 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3a88073d50 0x7f3a881505a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:46.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:46.497+0000 7f3a877fe640 1 -- 172.21.15.38:0/2082161134 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a8815ea80 con 0x7f3a880d0b40 2024-09-21T03:39:46.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:46.497+0000 7f3a877fe640 1 --2- 172.21.15.38:0/2082161134 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3a880d0b40 0x7f3a880c7ba0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f3a8000e990 tx=0x7f3a8000ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:46.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:46.497+0000 7f3a7f7fe640 1 -- 172.21.15.38:0/2082161134 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a80004d20 con 0x7f3a880d0b40 2024-09-21T03:39:46.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:46.497+0000 7f3a7f7fe640 1 -- 172.21.15.38:0/2082161134 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3a8000cb00 con 0x7f3a880d0b40 2024-09-21T03:39:46.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:46.497+0000 7f3a7f7fe640 1 -- 172.21.15.38:0/2082161134 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a80005020 con 0x7f3a880d0b40 2024-09-21T03:39:46.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:46.497+0000 7f3a8d6d3640 1 -- 172.21.15.38:0/2082161134 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a8806ce80 con 0x7f3a880d0b40 2024-09-21T03:39:46.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:46.497+0000 7f3a8d6d3640 1 -- 172.21.15.38:0/2082161134 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a8806d400 con 0x7f3a880d0b40 2024-09-21T03:39:46.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:46.498+0000 7f3a8d6d3640 1 -- 172.21.15.38:0/2082161134 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a8815bdf0 con 0x7f3a880d0b40 2024-09-21T03:39:46.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:46.499+0000 7f3a7f7fe640 1 -- 172.21.15.38:0/2082161134 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f3a800051c0 con 0x7f3a880d0b40 2024-09-21T03:39:46.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:46.499+0000 7f3a7f7fe640 1 --2- 172.21.15.38:0/2082161134 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3a68078b90 0x7f3a6807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:46.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:46.499+0000 7f3a7f7fe640 1 -- 172.21.15.38:0/2082161134 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 6187+0+0 (secure 0 0 0) 0x7f3a8009b830 con 0x7f3a880d0b40 2024-09-21T03:39:46.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:46.500+0000 7f3a86ffd640 1 --2- 172.21.15.38:0/2082161134 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3a68078b90 0x7f3a6807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:46.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:46.500+0000 7f3a86ffd640 1 --2- 172.21.15.38:0/2082161134 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3a68078b90 0x7f3a6807b0a0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f3a78033000 tx=0x7f3a78004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:46.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:46.504+0000 7f3a7f7fe640 1 -- 172.21.15.38:0/2082161134 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f3a8806ce80 con 0x7f3a880d0b40 2024-09-21T03:39:46.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:46.729+0000 7f3a8d6d3640 1 -- 172.21.15.38:0/2082161134 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3a8815e790 con 0x7f3a880d0b40 2024-09-21T03:39:46.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:46.729+0000 7f3a7f7fe640 1 -- 172.21.15.38:0/2082161134 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) ==== 94+0+5233 (secure 0 0 0) 0x7f3a80016030 con 0x7f3a880d0b40 2024-09-21T03:39:46.732 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:39:46.732 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":74,"btime":"2024-09-21T03:39:45:574648+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5783,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/1358588311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1358588311},{"type":"v1","addr":"172.21.15.38:6835","nonce":1358588311}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63},{"gid":5789,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/1676483003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":1676483003},{"type":"v1","addr":"172.21.15.195:6837","nonce":1676483003}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63},{"gid":6813,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/3644074938","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":3644074938},{"type":"v1","addr":"172.21.15.38:6833","nonce":3644074938}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":72}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:39:04.779948+0000","modified":"2024-09-21T03:39:45.574643+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6193},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6193":{"gid":6193,"name":"d","rank":0,"incarnation":71,"state":"up:active","state_seq":14,"addr":"172.21.15.195:6836/2771781100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":2771781100},{"type":"v1","addr":"172.21.15.195:6836","nonce":2771781100}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6193,"qdb_cluster":[6193]},"id":5}]} 2024-09-21T03:39:46.732 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 74 2024-09-21T03:39:46.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:46.732+0000 7f3a8d6d3640 1 -- 172.21.15.38:0/2082161134 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3a68078b90 msgr2=0x7f3a6807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:46.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:46.732+0000 7f3a8d6d3640 1 --2- 172.21.15.38:0/2082161134 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3a68078b90 0x7f3a6807b0a0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f3a78033000 tx=0x7f3a78004730 comp rx=0 tx=0).stop 2024-09-21T03:39:46.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:46.732+0000 7f3a8d6d3640 1 -- 172.21.15.38:0/2082161134 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3a880d0b40 msgr2=0x7f3a880c7ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:46.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:46.732+0000 7f3a8d6d3640 1 --2- 172.21.15.38:0/2082161134 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3a880d0b40 0x7f3a880c7ba0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f3a8000e990 tx=0x7f3a8000ee50 comp rx=0 tx=0).stop 2024-09-21T03:39:46.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:46.732+0000 7f3a8d6d3640 1 -- 172.21.15.38:0/2082161134 shutdown_connections 2024-09-21T03:39:46.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:46.732+0000 7f3a8d6d3640 1 --2- 172.21.15.38:0/2082161134 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3a880d0b40 0x7f3a880c7ba0 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:46.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:46.732+0000 7f3a8d6d3640 1 --2- 172.21.15.38:0/2082161134 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3a88073d50 0x7f3a881505a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:46.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:46.732+0000 7f3a8d6d3640 1 --2- 172.21.15.38:0/2082161134 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3a68078b90 0x7f3a6807b0a0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:46.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:46.732+0000 7f3a8d6d3640 1 --2- 172.21.15.38:0/2082161134 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3a8806ec30 0x7f3a8815e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:46.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:46.732+0000 7f3a8d6d3640 1 -- 172.21.15.38:0/2082161134 >> 172.21.15.38:0/2082161134 conn(0x7f3a880b2cc0 msgr2=0x7f3a8815eeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:46.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:46.732+0000 7f3a8d6d3640 1 -- 172.21.15.38:0/2082161134 shutdown_connections 2024-09-21T03:39:46.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:46.732+0000 7f3a8d6d3640 1 -- 172.21.15.38:0/2082161134 wait complete. 2024-09-21T03:39:47.148 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 74, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T03:39:04.779948+0000', 'modified': '2024-09-21T03:39:45.574643+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 83, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6193}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6193': {'gid': 6193, 'name': 'd', 'rank': 0, 'incarnation': 71, 'state': 'up:active', 'state_seq': 14, 'addr': '172.21.15.195:6836/2771781100', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6834', 'nonce': 2771781100}, {'type': 'v1', 'addr': '172.21.15.195:6836', 'nonce': 2771781100}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6193, 'qdb_cluster': [6193]} 2024-09-21T03:39:47.149 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-21T03:39:47.149 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-09-21T03:39:47.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.269+0000 7f28fd5a8640 1 -- 172.21.15.38:0/1416865520 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f28f815ee80 msgr2=0x7f28f80d0170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:47.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.269+0000 7f28fd5a8640 1 --2- 172.21.15.38:0/1416865520 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f28f815ee80 0x7f28f80d0170 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f28e8005fd0 tx=0x7f28e802a820 comp rx=0 tx=0).stop 2024-09-21T03:39:47.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.269+0000 7f28fd5a8640 1 -- 172.21.15.38:0/1416865520 shutdown_connections 2024-09-21T03:39:47.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.269+0000 7f28fd5a8640 1 --2- 172.21.15.38:0/1416865520 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f28f80ceb90 0x7f28f80cefa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:47.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.269+0000 7f28fd5a8640 1 --2- 172.21.15.38:0/1416865520 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f28f815ee80 0x7f28f80d0170 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:47.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.269+0000 7f28fd5a8640 1 -- 172.21.15.38:0/1416865520 >> 172.21.15.38:0/1416865520 conn(0x7f28f809d830 msgr2=0x7f28f80b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:47.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.269+0000 7f28fd5a8640 1 -- 172.21.15.38:0/1416865520 shutdown_connections 2024-09-21T03:39:47.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.269+0000 7f28fd5a8640 1 -- 172.21.15.38:0/1416865520 wait complete. 2024-09-21T03:39:47.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.271+0000 7f28fd5a8640 1 Processor -- start 2024-09-21T03:39:47.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.271+0000 7f28fd5a8640 1 -- start start 2024-09-21T03:39:47.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.272+0000 7f28fd5a8640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f28f80b9ad0 0x7f28f81f5f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:47.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.272+0000 7f28fd5a8640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f28f80ceb90 0x7f28f80ccbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:47.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.272+0000 7f28fd5a8640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f28f815ee80 0x7f28f80d3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:47.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.272+0000 7f28fd5a8640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f28f80b6700 con 0x7f28f80ceb90 2024-09-21T03:39:47.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.272+0000 7f28fd5a8640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f28f80b6580 con 0x7f28f80b9ad0 2024-09-21T03:39:47.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.272+0000 7f28fd5a8640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f28f80b6880 con 0x7f28f815ee80 2024-09-21T03:39:47.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.272+0000 7f28f67fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f28f80ceb90 0x7f28f80ccbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:47.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.273+0000 7f28f67fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f28f80ceb90 0x7f28f80ccbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42266/0 (socket says 172.21.15.38:42266) 2024-09-21T03:39:47.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.273+0000 7f28f67fc640 1 -- 172.21.15.38:0/2080067025 learned_addr learned my addr 172.21.15.38:0/2080067025 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:39:47.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.273+0000 7f28f6ffd640 1 --2- 172.21.15.38:0/2080067025 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f28f80b9ad0 0x7f28f81f5f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:47.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.273+0000 7f28f77fe640 1 --2- 172.21.15.38:0/2080067025 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f28f815ee80 0x7f28f80d3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:47.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.273+0000 7f28f67fc640 1 -- 172.21.15.38:0/2080067025 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f28f815ee80 msgr2=0x7f28f80d3bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:47.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.273+0000 7f28f67fc640 1 --2- 172.21.15.38:0/2080067025 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f28f815ee80 0x7f28f80d3bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:47.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.273+0000 7f28f67fc640 1 -- 172.21.15.38:0/2080067025 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f28f80b9ad0 msgr2=0x7f28f81f5f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:47.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.273+0000 7f28f67fc640 1 --2- 172.21.15.38:0/2080067025 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f28f80b9ad0 0x7f28f81f5f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:47.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.273+0000 7f28f67fc640 1 -- 172.21.15.38:0/2080067025 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28f806f0a0 con 0x7f28f80ceb90 2024-09-21T03:39:47.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.273+0000 7f28f67fc640 1 --2- 172.21.15.38:0/2080067025 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f28f80ceb90 0x7f28f80ccbe0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f28e000c3f0 tx=0x7f28e000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:47.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.273+0000 7f28ef7fe640 1 -- 172.21.15.38:0/2080067025 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28e0017070 con 0x7f28f80ceb90 2024-09-21T03:39:47.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.273+0000 7f28ef7fe640 1 -- 172.21.15.38:0/2080067025 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f28e00035f0 con 0x7f28f80ceb90 2024-09-21T03:39:47.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.273+0000 7f28ef7fe640 1 -- 172.21.15.38:0/2080067025 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28e0003a40 con 0x7f28f80ceb90 2024-09-21T03:39:47.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.273+0000 7f28fd5a8640 1 -- 172.21.15.38:0/2080067025 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28f806ce80 con 0x7f28f80ceb90 2024-09-21T03:39:47.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.273+0000 7f28fd5a8640 1 -- 172.21.15.38:0/2080067025 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28f806d440 con 0x7f28f80ceb90 2024-09-21T03:39:47.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.275+0000 7f28ef7fe640 1 -- 172.21.15.38:0/2080067025 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f28e001a070 con 0x7f28f80ceb90 2024-09-21T03:39:47.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.276+0000 7f28ef7fe640 1 --2- 172.21.15.38:0/2080067025 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f28d8078b90 0x7f28d807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:47.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.276+0000 7f28ef7fe640 1 -- 172.21.15.38:0/2080067025 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 6187+0+0 (secure 0 0 0) 0x7f28e009a130 con 0x7f28f80ceb90 2024-09-21T03:39:47.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.276+0000 7f28f6ffd640 1 --2- 172.21.15.38:0/2080067025 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f28d8078b90 0x7f28d807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:47.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.276+0000 7f29045f0740 1 Processor -- start 2024-09-21T03:39:47.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.277+0000 7f28f6ffd640 1 --2- 172.21.15.38:0/2080067025 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f28d8078b90 0x7f28d807b0a0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f28e8034000 tx=0x7f28e8004ab0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:47.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.277+0000 7f29045f0740 1 -- start start 2024-09-21T03:39:47.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.278+0000 7f29045f0740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d94e112270 0x55d94e4e3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:47.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.278+0000 7f29045f0740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55d94e0e93b0 0x55d94e4e44e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:47.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.278+0000 7f29045f0740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d94e0f1590 0x55d94e5013e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:47.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.278+0000 7f29045f0740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55d94e42c820 con 0x55d94e0f1590 2024-09-21T03:39:47.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.278+0000 7f29045f0740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55d94e41bc50 con 0x55d94e112270 2024-09-21T03:39:47.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.278+0000 7f29045f0740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55d94e41fa60 con 0x55d94e0e93b0 2024-09-21T03:39:47.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.278+0000 7f28f77fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d94e0f1590 0x55d94e5013e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:47.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.278+0000 7f28f77fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d94e0f1590 0x55d94e5013e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42274/0 (socket says 172.21.15.38:42274) 2024-09-21T03:39:47.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.278+0000 7f28f77fe640 1 -- 172.21.15.38:0/3465483082 learned_addr learned my addr 172.21.15.38:0/3465483082 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:39:47.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.278+0000 7f28f67fc640 1 --2- 172.21.15.38:0/3465483082 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55d94e0e93b0 0x55d94e4e44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:47.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.278+0000 7f28f6ffd640 1 --2- 172.21.15.38:0/3465483082 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d94e112270 0x55d94e4e3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:47.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.278+0000 7f28f77fe640 1 -- 172.21.15.38:0/3465483082 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55d94e0e93b0 msgr2=0x55d94e4e44e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:47.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.278+0000 7f28f77fe640 1 --2- 172.21.15.38:0/3465483082 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55d94e0e93b0 0x55d94e4e44e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:47.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.278+0000 7f28f77fe640 1 -- 172.21.15.38:0/3465483082 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d94e112270 msgr2=0x55d94e4e3fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:47.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.278+0000 7f28f77fe640 1 --2- 172.21.15.38:0/3465483082 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d94e112270 0x55d94e4e3fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:47.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.278+0000 7f28f77fe640 1 -- 172.21.15.38:0/3465483082 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d94e17d850 con 0x55d94e0f1590 2024-09-21T03:39:47.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.278+0000 7f28f67fc640 1 --2- 172.21.15.38:0/3465483082 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55d94e0e93b0 0x55d94e4e44e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:39:47.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.278+0000 7f28f6ffd640 1 --2- 172.21.15.38:0/3465483082 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d94e112270 0x55d94e4e3fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:39:47.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.279+0000 7f28f77fe640 1 --2- 172.21.15.38:0/3465483082 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d94e0f1590 0x55d94e5013e0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f28f000aae0 tx=0x7f28f000f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64b2683f66ec0ebe server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:47.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.279+0000 7f28ecff9640 1 -- 172.21.15.38:0/3465483082 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28f001d070 con 0x55d94e0f1590 2024-09-21T03:39:47.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.279+0000 7f28ecff9640 1 -- 172.21.15.38:0/3465483082 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f28f0018440 con 0x55d94e0f1590 2024-09-21T03:39:47.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.279+0000 7f28ecff9640 1 -- 172.21.15.38:0/3465483082 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28f0018850 con 0x55d94e0f1590 2024-09-21T03:39:47.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.280+0000 7f29045f0740 1 -- 172.21.15.38:0/3465483082 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d94e0f1590 msgr2=0x55d94e5013e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:47.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.280+0000 7f29045f0740 1 --2- 172.21.15.38:0/3465483082 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d94e0f1590 0x55d94e5013e0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f28f000aae0 tx=0x7f28f000f3c0 comp rx=0 tx=0).stop 2024-09-21T03:39:47.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.280+0000 7f29045f0740 1 -- 172.21.15.38:0/3465483082 shutdown_connections 2024-09-21T03:39:47.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.280+0000 7f29045f0740 1 --2- 172.21.15.38:0/3465483082 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d94e0f1590 0x55d94e5013e0 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:47.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.280+0000 7f29045f0740 1 --2- 172.21.15.38:0/3465483082 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55d94e0e93b0 0x55d94e4e44e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:47.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.280+0000 7f29045f0740 1 --2- 172.21.15.38:0/3465483082 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d94e112270 0x55d94e4e3fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:47.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.280+0000 7f29045f0740 1 -- 172.21.15.38:0/3465483082 >> 172.21.15.38:0/3465483082 conn(0x55d94e1063c0 msgr2=0x55d94e4ba260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:47.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.280+0000 7f29045f0740 1 -- 172.21.15.38:0/3465483082 shutdown_connections 2024-09-21T03:39:47.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.280+0000 7f29045f0740 1 -- 172.21.15.38:0/3465483082 wait complete. 2024-09-21T03:39:47.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.281+0000 7f29045f0740 1 -- start start 2024-09-21T03:39:47.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.281+0000 7f29045f0740 1 Processor -- start 2024-09-21T03:39:47.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.282+0000 7f29045f0740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d94e0e93b0 0x55d94e512040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:47.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.282+0000 7f29045f0740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55d94e0f1590 0x55d94e50d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:47.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.282+0000 7f29045f0740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d94e112270 0x55d94e4f5d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:47.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.282+0000 7f29045f0740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55d94e18c620 con 0x55d94e0e93b0 2024-09-21T03:39:47.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.282+0000 7f29045f0740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55d94e3c15f0 con 0x55d94e112270 2024-09-21T03:39:47.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.282+0000 7f29045f0740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55d94e4e4a20 con 0x55d94e0f1590 2024-09-21T03:39:47.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.282+0000 7f29045f0740 10 client.0 _ll_register_callbacks cb 0x55d94e3c50a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:39:47.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.282+0000 7f28f6ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d94e0e93b0 0x55d94e512040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:47.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.282+0000 7f28f6ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d94e0e93b0 0x55d94e512040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42288/0 (socket says 172.21.15.38:42288) 2024-09-21T03:39:47.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.282+0000 7f28f6ffd640 1 -- 172.21.15.38:0/3583683968 learned_addr learned my addr 172.21.15.38:0/3583683968 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:39:47.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.282+0000 7f28f67fc640 1 --2- 172.21.15.38:0/3583683968 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55d94e0f1590 0x55d94e50d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:47.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.282+0000 7f28f77fe640 1 --2- 172.21.15.38:0/3583683968 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d94e112270 0x55d94e4f5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:47.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.283+0000 7f28f6ffd640 1 -- 172.21.15.38:0/3583683968 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55d94e0f1590 msgr2=0x55d94e50d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:47.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.283+0000 7f28f6ffd640 1 --2- 172.21.15.38:0/3583683968 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55d94e0f1590 0x55d94e50d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:47.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.283+0000 7f28f6ffd640 1 -- 172.21.15.38:0/3583683968 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d94e112270 msgr2=0x55d94e4f5d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:47.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.283+0000 7f28f6ffd640 1 --2- 172.21.15.38:0/3583683968 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d94e112270 0x55d94e4f5d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:47.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.283+0000 7f28f6ffd640 1 -- 172.21.15.38:0/3583683968 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d94e1f46b0 con 0x55d94e0e93b0 2024-09-21T03:39:47.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.283+0000 7f28f6ffd640 1 --2- 172.21.15.38:0/3583683968 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d94e0e93b0 0x55d94e512040 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f28e80099a0 tx=0x7f28e8009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f41416a0d57e101 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:47.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.283+0000 7f28d5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-21T03:39:47.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.283+0000 7f28d5ffb640 1 -- 172.21.15.38:0/3583683968 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28e804d070 con 0x55d94e0e93b0 2024-09-21T03:39:47.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.283+0000 7f28d5ffb640 1 -- 172.21.15.38:0/3583683968 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 6187+0+0 (secure 0 0 0) 0x7f28e803d440 con 0x55d94e0e93b0 2024-09-21T03:39:47.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.283+0000 7f29045f0740 1 -- 172.21.15.38:0/3583683968 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f28f0018440 con 0x55d94e0e93b0 2024-09-21T03:39:47.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.283+0000 7f28d5ffb640 1 -- 172.21.15.38:0/3583683968 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7f28e8037040 con 0x55d94e0e93b0 2024-09-21T03:39:47.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.283+0000 7f29045f0740 10 client.6849 fetch_fsmap learned FSMap version 74 2024-09-21T03:39:47.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.283+0000 7f29045f0740 1 -- 172.21.15.38:0/3583683968 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) -- 0x55d94e512580 con 0x55d94e0e93b0 2024-09-21T03:39:47.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.283+0000 7f28d5ffb640 1 -- 172.21.15.38:0/3583683968 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f28f0018440 con 0x55d94e0e93b0 2024-09-21T03:39:47.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.284+0000 7f28d5ffb640 1 -- 172.21.15.38:0/3583683968 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 74) ==== 3502+0+0 (secure 0 0 0) 0x7f28e8048040 con 0x55d94e0e93b0 2024-09-21T03:39:47.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.284+0000 7f29045f0740 10 client.6849 fetch_fsmap finished waiting for FSMap version 74 2024-09-21T03:39:47.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.284+0000 7f29045f0740 10 client.6849 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-21T03:39:47.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.284+0000 7f29045f0740 20 client.6849 populate_metadata read hostname 'smithi038' 2024-09-21T03:39:47.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.284+0000 7f29045f0740 1 --2- 172.21.15.38:0/3583683968 >> [v2:172.21.15.195:6834/2771781100,v1:172.21.15.195:6836/2771781100] conn(0x55d94e0d0e40 0x55d94e4cdf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:47.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.284+0000 7f29045f0740 4 client.6849 mds_command: new command op to 6193 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:39:47.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.284+0000 7f29045f0740 1 -- 172.21.15.38:0/3583683968 --> [v2:172.21.15.195:6834/2771781100,v1:172.21.15.195:6836/2771781100] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d94e42e770 con 0x55d94e0d0e40 2024-09-21T03:39:47.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.284+0000 7f28f67fc640 1 --2- 172.21.15.38:0/3583683968 >> [v2:172.21.15.195:6834/2771781100,v1:172.21.15.195:6836/2771781100] conn(0x55d94e0d0e40 0x55d94e4cdf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:47.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.284+0000 7f28d5ffb640 10 client.6849 ms_handle_connect on v2:172.21.15.195:6834/2771781100 2024-09-21T03:39:47.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.284+0000 7f28f67fc640 1 --2- 172.21.15.38:0/3583683968 >> [v2:172.21.15.195:6834/2771781100,v1:172.21.15.195:6836/2771781100] conn(0x55d94e0d0e40 0x55d94e4cdf60 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7af0909aa38ae428 server_cookie=c89382dbf69dc5d3 in_seq=0 out_seq=0 2024-09-21T03:39:47.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.285+0000 7f28d5ffb640 1 -- 172.21.15.38:0/3583683968 <== mds.0 v2:172.21.15.195:6834/2771781100 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d94e42e770 con 0x55d94e0d0e40 2024-09-21T03:39:47.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.285+0000 7f28d5ffb640 10 client.6849 handle_command_reply: tid=0 2024-09-21T03:39:47.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.285+0000 7f29045f0740 1 client.6849 shutdown 2024-09-21T03:39:47.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.285+0000 7f29045f0740 1 -- 172.21.15.38:0/3583683968 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d94e0e93b0 msgr2=0x55d94e512040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:47.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.285+0000 7f29045f0740 1 --2- 172.21.15.38:0/3583683968 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d94e0e93b0 0x55d94e512040 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f28e80099a0 tx=0x7f28e8009500 comp rx=0 tx=0).stop 2024-09-21T03:39:47.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.285+0000 7f29045f0740 1 -- 172.21.15.38:0/3583683968 shutdown_connections 2024-09-21T03:39:47.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.285+0000 7f29045f0740 1 --2- 172.21.15.38:0/3583683968 >> [v2:172.21.15.195:6834/2771781100,v1:172.21.15.195:6836/2771781100] conn(0x55d94e0d0e40 0x55d94e4cdf60 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:47.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.285+0000 7f29045f0740 1 --2- 172.21.15.38:0/3583683968 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d94e112270 0x55d94e4f5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:47.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.286+0000 7f29045f0740 1 --2- 172.21.15.38:0/3583683968 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55d94e0f1590 0x55d94e50d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:47.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.286+0000 7f29045f0740 1 --2- 172.21.15.38:0/3583683968 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d94e0e93b0 0x55d94e512040 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:47.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.286+0000 7f28d5ffb640 0 client.6849 ms_handle_reset on v2:172.21.15.195:6834/2771781100 2024-09-21T03:39:47.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.286+0000 7f29045f0740 1 -- 172.21.15.38:0/3583683968 >> 172.21.15.38:0/3583683968 conn(0x55d94e1063c0 msgr2=0x55d94e4b4c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:47.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.286+0000 7f29045f0740 1 -- 172.21.15.38:0/3583683968 shutdown_connections 2024-09-21T03:39:47.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.286+0000 7f29045f0740 1 -- 172.21.15.38:0/3583683968 wait complete. 2024-09-21T03:39:47.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.286+0000 7f29045f0740 20 client.6849 trim_cache size 0 max 16384 2024-09-21T03:39:47.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.304+0000 7f29045f0740 1 Processor -- start 2024-09-21T03:39:47.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.305+0000 7f29045f0740 1 -- start start 2024-09-21T03:39:47.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.306+0000 7f29045f0740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d94e0e93b0 0x55d94e4c9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:47.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.306+0000 7f29045f0740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d94e0f1590 0x55d94e4ca410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:47.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.306+0000 7f29045f0740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55d94e112270 0x55d94e4ba0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:47.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.306+0000 7f29045f0740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f28f8201740 con 0x55d94e0f1590 2024-09-21T03:39:47.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.306+0000 7f29045f0740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f28f81f9720 con 0x55d94e0e93b0 2024-09-21T03:39:47.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.306+0000 7f29045f0740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f28f82010f0 con 0x55d94e112270 2024-09-21T03:39:47.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.306+0000 7f28f67fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d94e0f1590 0x55d94e4ca410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:47.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.306+0000 7f28f67fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d94e0f1590 0x55d94e4ca410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42294/0 (socket says 172.21.15.38:42294) 2024-09-21T03:39:47.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.306+0000 7f28f67fc640 1 -- 172.21.15.38:0/2664968302 learned_addr learned my addr 172.21.15.38:0/2664968302 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:39:47.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.306+0000 7f28f6ffd640 1 --2- 172.21.15.38:0/2664968302 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d94e0e93b0 0x55d94e4c9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:47.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.306+0000 7f28f77fe640 1 --2- 172.21.15.38:0/2664968302 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55d94e112270 0x55d94e4ba0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:47.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.306+0000 7f28f67fc640 1 -- 172.21.15.38:0/2664968302 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55d94e112270 msgr2=0x55d94e4ba0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:47.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.306+0000 7f28f67fc640 1 --2- 172.21.15.38:0/2664968302 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55d94e112270 0x55d94e4ba0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:47.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.306+0000 7f28f67fc640 1 -- 172.21.15.38:0/2664968302 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d94e0e93b0 msgr2=0x55d94e4c9ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:47.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.306+0000 7f28f67fc640 1 --2- 172.21.15.38:0/2664968302 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d94e0e93b0 0x55d94e4c9ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:47.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.306+0000 7f28f67fc640 1 -- 172.21.15.38:0/2664968302 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d94e4f7950 con 0x55d94e0f1590 2024-09-21T03:39:47.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.306+0000 7f28f6ffd640 1 --2- 172.21.15.38:0/2664968302 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d94e0e93b0 0x55d94e4c9ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:39:47.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.306+0000 7f28f67fc640 1 --2- 172.21.15.38:0/2664968302 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d94e0f1590 0x55d94e4ca410 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f28e0061d80 tx=0x7f28e009a050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfb34f106b4dc238 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:47.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.307+0000 7f28d7fff640 1 -- 172.21.15.38:0/2664968302 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28e006b6d0 con 0x55d94e0f1590 2024-09-21T03:39:47.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.307+0000 7f28d7fff640 1 -- 172.21.15.38:0/2664968302 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f28e0088030 con 0x55d94e0f1590 2024-09-21T03:39:47.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.307+0000 7f28d7fff640 1 -- 172.21.15.38:0/2664968302 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28e006b240 con 0x55d94e0f1590 2024-09-21T03:39:47.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.307+0000 7f29045f0740 1 -- 172.21.15.38:0/2664968302 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d94e0f1590 msgr2=0x55d94e4ca410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:47.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.307+0000 7f29045f0740 1 --2- 172.21.15.38:0/2664968302 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d94e0f1590 0x55d94e4ca410 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f28e0061d80 tx=0x7f28e009a050 comp rx=0 tx=0).stop 2024-09-21T03:39:47.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.307+0000 7f29045f0740 1 -- 172.21.15.38:0/2664968302 shutdown_connections 2024-09-21T03:39:47.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.307+0000 7f29045f0740 1 --2- 172.21.15.38:0/2664968302 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55d94e112270 0x55d94e4ba0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:47.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.307+0000 7f29045f0740 1 --2- 172.21.15.38:0/2664968302 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d94e0f1590 0x55d94e4ca410 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:47.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.307+0000 7f29045f0740 1 --2- 172.21.15.38:0/2664968302 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d94e0e93b0 0x55d94e4c9ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:47.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.307+0000 7f29045f0740 1 -- 172.21.15.38:0/2664968302 >> 172.21.15.38:0/2664968302 conn(0x55d94e1063c0 msgr2=0x55d94e4f7fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:47.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.307+0000 7f29045f0740 1 -- 172.21.15.38:0/2664968302 shutdown_connections 2024-09-21T03:39:47.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.307+0000 7f29045f0740 1 -- 172.21.15.38:0/2664968302 wait complete. 2024-09-21T03:39:47.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.308+0000 7f29045f0740 1 -- start start 2024-09-21T03:39:47.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.308+0000 7f29045f0740 1 Processor -- start 2024-09-21T03:39:47.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.310+0000 7f29045f0740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d94e0e93b0 0x55d94e4d0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:47.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.310+0000 7f29045f0740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d94e0f1590 0x55d94e57db30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:47.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.310+0000 7f29045f0740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55d94e112270 0x55d94e581900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:47.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.310+0000 7f29045f0740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55d94e4cf6f0 con 0x55d94e0e93b0 2024-09-21T03:39:47.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.310+0000 7f29045f0740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55d94e50df70 con 0x55d94e0f1590 2024-09-21T03:39:47.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.310+0000 7f29045f0740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55d94e4d1840 con 0x55d94e112270 2024-09-21T03:39:47.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.310+0000 7f29045f0740 10 client.0 _ll_register_callbacks cb 0x7f28e8009a60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:39:47.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.310+0000 7f28f6ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d94e0e93b0 0x55d94e4d0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:47.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.310+0000 7f28f6ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d94e0e93b0 0x55d94e4d0ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42310/0 (socket says 172.21.15.38:42310) 2024-09-21T03:39:47.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.310+0000 7f28f6ffd640 1 -- 172.21.15.38:0/2015689753 learned_addr learned my addr 172.21.15.38:0/2015689753 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:39:47.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.310+0000 7f28f67fc640 1 --2- 172.21.15.38:0/2015689753 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d94e0f1590 0x55d94e57db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:47.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.310+0000 7f28f77fe640 1 --2- 172.21.15.38:0/2015689753 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55d94e112270 0x55d94e581900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:47.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.310+0000 7f28f6ffd640 1 -- 172.21.15.38:0/2015689753 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55d94e112270 msgr2=0x55d94e581900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:47.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.310+0000 7f28f6ffd640 1 --2- 172.21.15.38:0/2015689753 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55d94e112270 0x55d94e581900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:47.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.310+0000 7f28f6ffd640 1 -- 172.21.15.38:0/2015689753 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d94e0f1590 msgr2=0x55d94e57db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:47.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.310+0000 7f28f6ffd640 1 --2- 172.21.15.38:0/2015689753 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d94e0f1590 0x55d94e57db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:47.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.310+0000 7f28f6ffd640 1 -- 172.21.15.38:0/2015689753 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d94e5119c0 con 0x55d94e0e93b0 2024-09-21T03:39:47.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.310+0000 7f28f6ffd640 1 --2- 172.21.15.38:0/2015689753 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d94e0e93b0 0x55d94e4d0ad0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f28e80099a0 tx=0x7f28e803ede0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2a1258398caed86 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:47.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.310+0000 7f28d5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-21T03:39:47.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.310+0000 7f28d5ffb640 1 -- 172.21.15.38:0/2015689753 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28e804d040 con 0x55d94e0e93b0 2024-09-21T03:39:47.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.310+0000 7f28d5ffb640 1 -- 172.21.15.38:0/2015689753 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 6187+0+0 (secure 0 0 0) 0x7f28e805dcd0 con 0x55d94e0e93b0 2024-09-21T03:39:47.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.310+0000 7f29045f0740 1 -- 172.21.15.38:0/2015689753 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f28e0088030 con 0x55d94e0e93b0 2024-09-21T03:39:47.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.310+0000 7f28d5ffb640 1 -- 172.21.15.38:0/2015689753 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7f28e80043e0 con 0x55d94e0e93b0 2024-09-21T03:39:47.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.310+0000 7f29045f0740 10 client.6855 fetch_fsmap learned FSMap version 74 2024-09-21T03:39:47.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.310+0000 7f29045f0740 1 -- 172.21.15.38:0/2015689753 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) -- 0x55d94e4d1010 con 0x55d94e0e93b0 2024-09-21T03:39:47.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.311+0000 7f28d5ffb640 1 -- 172.21.15.38:0/2015689753 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f28e0088030 con 0x55d94e0e93b0 2024-09-21T03:39:47.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.311+0000 7f28d5ffb640 1 -- 172.21.15.38:0/2015689753 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 74) ==== 3502+0+0 (secure 0 0 0) 0x7f28e8064050 con 0x55d94e0e93b0 2024-09-21T03:39:47.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.311+0000 7f29045f0740 10 client.6855 fetch_fsmap finished waiting for FSMap version 74 2024-09-21T03:39:47.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.311+0000 7f29045f0740 10 client.6855 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-21T03:39:47.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.311+0000 7f29045f0740 20 client.6855 populate_metadata read hostname 'smithi038' 2024-09-21T03:39:47.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.311+0000 7f29045f0740 1 --2- 172.21.15.38:0/2015689753 >> [v2:172.21.15.195:6834/2771781100,v1:172.21.15.195:6836/2771781100] conn(0x55d94e4ba740 0x55d94e50cc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:47.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.311+0000 7f29045f0740 4 client.6855 mds_command: new command op to 6193 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T03:39:47.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.311+0000 7f29045f0740 1 -- 172.21.15.38:0/2015689753 --> [v2:172.21.15.195:6834/2771781100,v1:172.21.15.195:6836/2771781100] -- command(tid 0: {"prefix": "status"}) -- 0x55d94e42a220 con 0x55d94e4ba740 2024-09-21T03:39:47.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.311+0000 7f28f67fc640 1 --2- 172.21.15.38:0/2015689753 >> [v2:172.21.15.195:6834/2771781100,v1:172.21.15.195:6836/2771781100] conn(0x55d94e4ba740 0x55d94e50cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:47.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.311+0000 7f28f67fc640 1 --2- 172.21.15.38:0/2015689753 >> [v2:172.21.15.195:6834/2771781100,v1:172.21.15.195:6836/2771781100] conn(0x55d94e4ba740 0x55d94e50cc00 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5aea4bd8ac8b5b95 server_cookie=675e909f4558dd3c in_seq=0 out_seq=0 2024-09-21T03:39:47.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.311+0000 7f28d5ffb640 10 client.6855 ms_handle_connect on v2:172.21.15.195:6834/2771781100 2024-09-21T03:39:47.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.312+0000 7f28d5ffb640 1 -- 172.21.15.38:0/2015689753 <== mds.0 v2:172.21.15.195:6834/2771781100 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x55d94e42a220 con 0x55d94e4ba740 2024-09-21T03:39:47.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.312+0000 7f28d5ffb640 10 client.6855 handle_command_reply: tid=0 2024-09-21T03:39:47.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.312+0000 7f29045f0740 1 client.6855 shutdown 2024-09-21T03:39:47.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.312+0000 7f29045f0740 1 -- 172.21.15.38:0/2015689753 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d94e0e93b0 msgr2=0x55d94e4d0ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:47.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.312+0000 7f29045f0740 1 --2- 172.21.15.38:0/2015689753 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d94e0e93b0 0x55d94e4d0ad0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f28e80099a0 tx=0x7f28e803ede0 comp rx=0 tx=0).stop 2024-09-21T03:39:47.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.312+0000 7f29045f0740 1 -- 172.21.15.38:0/2015689753 shutdown_connections 2024-09-21T03:39:47.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.312+0000 7f29045f0740 1 --2- 172.21.15.38:0/2015689753 >> [v2:172.21.15.195:6834/2771781100,v1:172.21.15.195:6836/2771781100] conn(0x55d94e4ba740 0x55d94e50cc00 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:47.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.312+0000 7f29045f0740 1 --2- 172.21.15.38:0/2015689753 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55d94e112270 0x55d94e581900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:47.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.312+0000 7f28d5ffb640 0 client.6855 ms_handle_reset on v2:172.21.15.195:6834/2771781100 2024-09-21T03:39:47.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.312+0000 7f29045f0740 1 --2- 172.21.15.38:0/2015689753 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d94e0f1590 0x55d94e57db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:47.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.312+0000 7f29045f0740 1 --2- 172.21.15.38:0/2015689753 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d94e0e93b0 0x55d94e4d0ad0 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:47.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.312+0000 7f29045f0740 1 -- 172.21.15.38:0/2015689753 >> 172.21.15.38:0/2015689753 conn(0x55d94e1063c0 msgr2=0x55d94e4e2a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:47.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.312+0000 7f29045f0740 1 -- 172.21.15.38:0/2015689753 shutdown_connections 2024-09-21T03:39:47.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.312+0000 7f29045f0740 1 -- 172.21.15.38:0/2015689753 wait complete. 2024-09-21T03:39:47.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.312+0000 7f29045f0740 20 client.6855 trim_cache size 0 max 16384 2024-09-21T03:39:47.319 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-09-21T03:39:47.319 INFO:teuthology.orchestra.run.smithi038.stdout: "cluster_fsid": "9d50d66f-656b-4fe2-8e3b-fe53b79c4d43", 2024-09-21T03:39:47.319 INFO:teuthology.orchestra.run.smithi038.stdout: "whoami": 0, 2024-09-21T03:39:47.319 INFO:teuthology.orchestra.run.smithi038.stdout: "id": 6193, 2024-09-21T03:39:47.319 INFO:teuthology.orchestra.run.smithi038.stdout: "want_state": "up:active", 2024-09-21T03:39:47.319 INFO:teuthology.orchestra.run.smithi038.stdout: "state": "up:active", 2024-09-21T03:39:47.319 INFO:teuthology.orchestra.run.smithi038.stdout: "fs_name": "cephfs", 2024-09-21T03:39:47.319 INFO:teuthology.orchestra.run.smithi038.stdout: "rank_uptime": 4.7533957620000002, 2024-09-21T03:39:47.319 INFO:teuthology.orchestra.run.smithi038.stdout: "mdsmap_epoch": 74, 2024-09-21T03:39:47.319 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch": 83, 2024-09-21T03:39:47.320 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch_barrier": 83, 2024-09-21T03:39:47.320 INFO:teuthology.orchestra.run.smithi038.stdout: "uptime": 46.537178963000002 2024-09-21T03:39:47.320 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-09-21T03:39:47.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.313+0000 7f28fd5a8640 1 -- 172.21.15.38:0/2080067025 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f28d8078b90 msgr2=0x7f28d807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:47.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.313+0000 7f28fd5a8640 1 --2- 172.21.15.38:0/2080067025 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f28d8078b90 0x7f28d807b0a0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f28e8034000 tx=0x7f28e8004ab0 comp rx=0 tx=0).stop 2024-09-21T03:39:47.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.313+0000 7f28fd5a8640 1 -- 172.21.15.38:0/2080067025 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f28f80ceb90 msgr2=0x7f28f80ccbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:47.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.313+0000 7f28fd5a8640 1 --2- 172.21.15.38:0/2080067025 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f28f80ceb90 0x7f28f80ccbe0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f28e000c3f0 tx=0x7f28e000c8b0 comp rx=0 tx=0).stop 2024-09-21T03:39:47.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.313+0000 7f28fd5a8640 1 -- 172.21.15.38:0/2080067025 shutdown_connections 2024-09-21T03:39:47.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.313+0000 7f28fd5a8640 1 --2- 172.21.15.38:0/2080067025 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f28d8078b90 0x7f28d807b0a0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:47.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.313+0000 7f28fd5a8640 1 --2- 172.21.15.38:0/2080067025 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f28f815ee80 0x7f28f80d3bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:47.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.313+0000 7f28fd5a8640 1 --2- 172.21.15.38:0/2080067025 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f28f80ceb90 0x7f28f80ccbe0 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:47.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.313+0000 7f28fd5a8640 1 --2- 172.21.15.38:0/2080067025 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f28f80b9ad0 0x7f28f81f5f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:47.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.313+0000 7f28fd5a8640 1 -- 172.21.15.38:0/2080067025 >> 172.21.15.38:0/2080067025 conn(0x7f28f809d830 msgr2=0x7f28f815d1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:47.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.313+0000 7f28fd5a8640 1 -- 172.21.15.38:0/2080067025 shutdown_connections 2024-09-21T03:39:47.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.313+0000 7f28fd5a8640 1 -- 172.21.15.38:0/2080067025 wait complete. 2024-09-21T03:39:47.326 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat a/ && find a/') 2024-09-21T03:39:47.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T03:39:47.378 INFO:teuthology.orchestra.run.smithi038.stdout: File: a/ 2024-09-21T03:39:47.378 INFO:teuthology.orchestra.run.smithi038.stdout: Size: 2 Blocks: 0 IO Block: 65536 directory 2024-09-21T03:39:47.378 INFO:teuthology.orchestra.run.smithi038.stdout:Device: 29h/41d Inode: 1099511627776 Links: 4 2024-09-21T03:39:47.378 INFO:teuthology.orchestra.run.smithi038.stdout:Access: (0755/drwxr-xr-x) Uid: ( 1000/ ubuntu) Gid: ( 1333/ ubuntu) 2024-09-21T03:39:47.378 INFO:teuthology.orchestra.run.smithi038.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-09-21T03:39:47.378 INFO:teuthology.orchestra.run.smithi038.stdout:Access: 2024-09-21 03:39:18.552867392 +0000 2024-09-21T03:39:47.379 INFO:teuthology.orchestra.run.smithi038.stdout:Modify: 2024-09-21 03:39:24.177760832 +0000 2024-09-21T03:39:47.379 INFO:teuthology.orchestra.run.smithi038.stdout:Change: 2024-09-21 03:39:24.177760832 +0000 2024-09-21T03:39:47.379 INFO:teuthology.orchestra.run.smithi038.stdout: Birth: 2024-09-21 03:39:18.552867392 +0000 2024-09-21T03:39:47.382 INFO:tasks.ceph.mds.d.smithi195.stderr:2024-09-21T03:39:47.381+0000 7f462f303640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T03:39:47.383 INFO:teuthology.orchestra.run.smithi038.stdout:a/ 2024-09-21T03:39:47.383 INFO:teuthology.orchestra.run.smithi038.stdout:a/z 2024-09-21T03:39:47.383 INFO:teuthology.orchestra.run.smithi038.stdout:a/z/d 2024-09-21T03:39:47.383 INFO:teuthology.orchestra.run.smithi038.stdout:a/c 2024-09-21T03:39:47.383 INFO:teuthology.orchestra.run.smithi038.stdout:a/c/d 2024-09-21T03:39:47.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:39:47.389 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.5:0 flush journal 2024-09-21T03:39:47.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.506+0000 7f33d0271640 1 -- 172.21.15.38:0/3452058242 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f33c806def0 msgr2=0x7f33c806f410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:47.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.506+0000 7f33d0271640 1 --2- 172.21.15.38:0/3452058242 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f33c806def0 0x7f33c806f410 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f33b8005fd0 tx=0x7f33b802a5d0 comp rx=0 tx=0).stop 2024-09-21T03:39:47.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.506+0000 7f33d0271640 1 -- 172.21.15.38:0/3452058242 shutdown_connections 2024-09-21T03:39:47.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.506+0000 7f33d0271640 1 --2- 172.21.15.38:0/3452058242 >> v2:172.21.15.195:3300/0 conn(0x7f33c80b5670 0x7f33c815f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:47.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.506+0000 7f33d0271640 1 --2- 172.21.15.38:0/3452058242 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f33c806def0 0x7f33c806f410 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:47.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.506+0000 7f33d0271640 1 -- 172.21.15.38:0/3452058242 >> 172.21.15.38:0/3452058242 conn(0x7f33c80cdca0 msgr2=0x7f33c80ce0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:47.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.506+0000 7f33d0271640 1 -- 172.21.15.38:0/3452058242 shutdown_connections 2024-09-21T03:39:47.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.506+0000 7f33d0271640 1 -- 172.21.15.38:0/3452058242 wait complete. 2024-09-21T03:39:47.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.507+0000 7f33d0271640 1 Processor -- start 2024-09-21T03:39:47.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.508+0000 7f33d0271640 1 -- start start 2024-09-21T03:39:47.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.509+0000 7f33d0271640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f33c806def0 0x7f33c80cae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:47.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.509+0000 7f33d0271640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f33c80b5260 0x7f33c806d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:47.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.509+0000 7f33d0271640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f33c80b5670 0x7f33c806d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:47.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.509+0000 7f33d0271640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f33c815ffb0 con 0x7f33c806def0 2024-09-21T03:39:47.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.509+0000 7f33d0271640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f33c815fe30 con 0x7f33c80b5260 2024-09-21T03:39:47.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.509+0000 7f33d0271640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f33c8071300 con 0x7f33c80b5670 2024-09-21T03:39:47.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.509+0000 7f33ce00d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f33c806def0 0x7f33c80cae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:47.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.509+0000 7f33ce00d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f33c806def0 0x7f33c80cae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42318/0 (socket says 172.21.15.38:42318) 2024-09-21T03:39:47.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.509+0000 7f33ce00d640 1 -- 172.21.15.38:0/247317362 learned_addr learned my addr 172.21.15.38:0/247317362 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:39:47.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.509+0000 7f33cd80c640 1 --2- 172.21.15.38:0/247317362 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f33c80b5260 0x7f33c806d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:47.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.509+0000 7f33ce80e640 1 --2- 172.21.15.38:0/247317362 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f33c80b5670 0x7f33c806d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:47.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.509+0000 7f33ce00d640 1 -- 172.21.15.38:0/247317362 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f33c80b5670 msgr2=0x7f33c806d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:47.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.509+0000 7f33ce00d640 1 --2- 172.21.15.38:0/247317362 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f33c80b5670 0x7f33c806d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:47.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.509+0000 7f33ce00d640 1 -- 172.21.15.38:0/247317362 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f33c80b5260 msgr2=0x7f33c806d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:47.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.509+0000 7f33ce00d640 1 --2- 172.21.15.38:0/247317362 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f33c80b5260 0x7f33c806d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:47.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.509+0000 7f33ce00d640 1 -- 172.21.15.38:0/247317362 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33c814c690 con 0x7f33c806def0 2024-09-21T03:39:47.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.509+0000 7f33cd80c640 1 --2- 172.21.15.38:0/247317362 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f33c80b5260 0x7f33c806d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:39:47.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.510+0000 7f33ce00d640 1 --2- 172.21.15.38:0/247317362 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f33c806def0 0x7f33c80cae20 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f33b8034040 tx=0x7f33b80035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:47.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.510+0000 7f33c6ffd640 1 -- 172.21.15.38:0/247317362 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33b803e030 con 0x7f33c806def0 2024-09-21T03:39:47.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.510+0000 7f33d0271640 1 -- 172.21.15.38:0/247317362 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33c814bdf0 con 0x7f33c806def0 2024-09-21T03:39:47.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.510+0000 7f33c6ffd640 1 -- 172.21.15.38:0/247317362 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f33b8003b00 con 0x7f33c806def0 2024-09-21T03:39:47.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.510+0000 7f33c6ffd640 1 -- 172.21.15.38:0/247317362 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33b8035040 con 0x7f33c806def0 2024-09-21T03:39:47.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.510+0000 7f33d0271640 1 -- 172.21.15.38:0/247317362 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33c814c3b0 con 0x7f33c806def0 2024-09-21T03:39:47.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.512+0000 7f33c6ffd640 1 -- 172.21.15.38:0/247317362 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f33b8041030 con 0x7f33c806def0 2024-09-21T03:39:47.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.512+0000 7f33c6ffd640 1 --2- 172.21.15.38:0/247317362 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f33b4078b90 0x7f33b407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:47.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.513+0000 7f33c6ffd640 1 -- 172.21.15.38:0/247317362 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 6187+0+0 (secure 0 0 0) 0x7f33b80bb2e0 con 0x7f33c806def0 2024-09-21T03:39:47.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.513+0000 7f33cd80c640 1 --2- 172.21.15.38:0/247317362 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f33b4078b90 0x7f33b407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:47.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.513+0000 7f33d71d9740 1 Processor -- start 2024-09-21T03:39:47.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.513+0000 7f33cd80c640 1 --2- 172.21.15.38:0/247317362 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f33b4078b90 0x7f33b407b0a0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f33c809edf0 tx=0x7f33bc009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:47.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.513+0000 7f33d71d9740 1 -- start start 2024-09-21T03:39:47.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.515+0000 7f33d71d9740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f8c157b390 0x55f8c194c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:47.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.515+0000 7f33d71d9740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f8c15524d0 0x55f8c194cc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:47.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.515+0000 7f33d71d9740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f8c155a6b0 0x55f8c1969b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:47.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.515+0000 7f33d71d9740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55f8c1895a60 con 0x55f8c15524d0 2024-09-21T03:39:47.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.515+0000 7f33d71d9740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55f8c1884e90 con 0x55f8c157b390 2024-09-21T03:39:47.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.515+0000 7f33d71d9740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55f8c1888ca0 con 0x55f8c155a6b0 2024-09-21T03:39:47.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.515+0000 7f33cd80c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f8c15524d0 0x55f8c194cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:47.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.515+0000 7f33cd80c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f8c15524d0 0x55f8c194cc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42324/0 (socket says 172.21.15.38:42324) 2024-09-21T03:39:47.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.515+0000 7f33cd80c640 1 -- 172.21.15.38:0/156003966 learned_addr learned my addr 172.21.15.38:0/156003966 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:39:47.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.515+0000 7f33ce80e640 1 --2- 172.21.15.38:0/156003966 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f8c155a6b0 0x55f8c1969b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:47.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.515+0000 7f33ce00d640 1 --2- 172.21.15.38:0/156003966 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f8c157b390 0x55f8c194c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:47.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.515+0000 7f33cd80c640 1 -- 172.21.15.38:0/156003966 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f8c155a6b0 msgr2=0x55f8c1969b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:47.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.515+0000 7f33cd80c640 1 --2- 172.21.15.38:0/156003966 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f8c155a6b0 0x55f8c1969b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:47.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.515+0000 7f33cd80c640 1 -- 172.21.15.38:0/156003966 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f8c157b390 msgr2=0x55f8c194c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:47.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.515+0000 7f33cd80c640 1 --2- 172.21.15.38:0/156003966 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f8c157b390 0x55f8c194c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:47.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.515+0000 7f33cd80c640 1 -- 172.21.15.38:0/156003966 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f8c165d7d0 con 0x55f8c15524d0 2024-09-21T03:39:47.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.515+0000 7f33cd80c640 1 --2- 172.21.15.38:0/156003966 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f8c15524d0 0x55f8c194cc80 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f33bc0144a0 tx=0x7f33bc012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bec17c092669881b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:47.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.516+0000 7f33affff640 1 -- 172.21.15.38:0/156003966 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33bc025070 con 0x55f8c15524d0 2024-09-21T03:39:47.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.516+0000 7f33affff640 1 -- 172.21.15.38:0/156003966 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f33bc01b050 con 0x55f8c15524d0 2024-09-21T03:39:47.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.516+0000 7f33affff640 1 -- 172.21.15.38:0/156003966 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33bc013800 con 0x55f8c15524d0 2024-09-21T03:39:47.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.516+0000 7f33d71d9740 1 -- 172.21.15.38:0/156003966 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f8c15524d0 msgr2=0x55f8c194cc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:47.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.516+0000 7f33d71d9740 1 --2- 172.21.15.38:0/156003966 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f8c15524d0 0x55f8c194cc80 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f33bc0144a0 tx=0x7f33bc012000 comp rx=0 tx=0).stop 2024-09-21T03:39:47.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.517+0000 7f33d71d9740 1 -- 172.21.15.38:0/156003966 shutdown_connections 2024-09-21T03:39:47.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.517+0000 7f33d71d9740 1 --2- 172.21.15.38:0/156003966 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f8c155a6b0 0x55f8c1969b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:47.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.517+0000 7f33d71d9740 1 --2- 172.21.15.38:0/156003966 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f8c15524d0 0x55f8c194cc80 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:47.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.517+0000 7f33d71d9740 1 --2- 172.21.15.38:0/156003966 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f8c157b390 0x55f8c194c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:47.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.517+0000 7f33d71d9740 1 -- 172.21.15.38:0/156003966 >> 172.21.15.38:0/156003966 conn(0x55f8c156f4e0 msgr2=0x55f8c1922a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:47.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.517+0000 7f33d71d9740 1 -- 172.21.15.38:0/156003966 shutdown_connections 2024-09-21T03:39:47.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.517+0000 7f33d71d9740 1 -- 172.21.15.38:0/156003966 wait complete. 2024-09-21T03:39:47.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.518+0000 7f33d71d9740 1 -- start start 2024-09-21T03:39:47.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.518+0000 7f33d71d9740 1 Processor -- start 2024-09-21T03:39:47.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.519+0000 7f33d71d9740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f8c15524d0 0x55f8c1936b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:47.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.519+0000 7f33d71d9740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f8c155a6b0 0x55f8c195c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:47.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.519+0000 7f33d71d9740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f8c157b390 0x55f8c196aad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:47.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.519+0000 7f33d71d9740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55f8c15f5740 con 0x55f8c155a6b0 2024-09-21T03:39:47.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.519+0000 7f33d71d9740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55f8c182a970 con 0x55f8c157b390 2024-09-21T03:39:47.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.519+0000 7f33d71d9740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55f8c194d1c0 con 0x55f8c15524d0 2024-09-21T03:39:47.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.519+0000 7f33d71d9740 10 client.0 _ll_register_callbacks cb 0x55f8c182dbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:39:47.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.519+0000 7f33cd80c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f8c155a6b0 0x55f8c195c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:47.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.519+0000 7f33cd80c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f8c155a6b0 0x55f8c195c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42330/0 (socket says 172.21.15.38:42330) 2024-09-21T03:39:47.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.519+0000 7f33cd80c640 1 -- 172.21.15.38:0/365638776 learned_addr learned my addr 172.21.15.38:0/365638776 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:39:47.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.519+0000 7f33ce80e640 1 --2- 172.21.15.38:0/365638776 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f8c157b390 0x55f8c196aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:47.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.519+0000 7f33ce00d640 1 --2- 172.21.15.38:0/365638776 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f8c15524d0 0x55f8c1936b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:47.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.520+0000 7f33cd80c640 1 -- 172.21.15.38:0/365638776 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f8c15524d0 msgr2=0x55f8c1936b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:47.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.520+0000 7f33cd80c640 1 --2- 172.21.15.38:0/365638776 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f8c15524d0 0x55f8c1936b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:47.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.520+0000 7f33cd80c640 1 -- 172.21.15.38:0/365638776 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f8c157b390 msgr2=0x55f8c196aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:47.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.520+0000 7f33cd80c640 1 --2- 172.21.15.38:0/365638776 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f8c157b390 0x55f8c196aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:47.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.520+0000 7f33cd80c640 1 -- 172.21.15.38:0/365638776 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f8c15e6970 con 0x55f8c155a6b0 2024-09-21T03:39:47.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.520+0000 7f33cd80c640 1 --2- 172.21.15.38:0/365638776 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f8c155a6b0 0x55f8c195c9c0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f33bc023040 tx=0x7f33bc013c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6e1f3c5bd123624 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:47.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.520+0000 7f33ad7fa640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-21T03:39:47.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.520+0000 7f33ad7fa640 1 -- 172.21.15.38:0/365638776 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33bc02e020 con 0x55f8c155a6b0 2024-09-21T03:39:47.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.520+0000 7f33ad7fa640 1 -- 172.21.15.38:0/365638776 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 6187+0+0 (secure 0 0 0) 0x7f33bc02a410 con 0x55f8c155a6b0 2024-09-21T03:39:47.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.520+0000 7f33d71d9740 1 -- 172.21.15.38:0/365638776 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f33bc01b050 con 0x55f8c155a6b0 2024-09-21T03:39:47.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.520+0000 7f33ad7fa640 1 -- 172.21.15.38:0/365638776 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7f33bc038050 con 0x55f8c155a6b0 2024-09-21T03:39:47.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.520+0000 7f33d71d9740 10 client.6864 fetch_fsmap learned FSMap version 74 2024-09-21T03:39:47.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.520+0000 7f33d71d9740 1 -- 172.21.15.38:0/365638776 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) -- 0x55f8c15c46a0 con 0x55f8c155a6b0 2024-09-21T03:39:47.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.520+0000 7f33ad7fa640 1 -- 172.21.15.38:0/365638776 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f33bc01b050 con 0x55f8c155a6b0 2024-09-21T03:39:47.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.521+0000 7f33ad7fa640 1 -- 172.21.15.38:0/365638776 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 74) ==== 3502+0+0 (secure 0 0 0) 0x7f33bc03b020 con 0x55f8c155a6b0 2024-09-21T03:39:47.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.521+0000 7f33d71d9740 10 client.6864 fetch_fsmap finished waiting for FSMap version 74 2024-09-21T03:39:47.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.521+0000 7f33d71d9740 10 client.6864 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.d 2024-09-21T03:39:47.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.521+0000 7f33d71d9740 20 client.6864 populate_metadata read hostname 'smithi038' 2024-09-21T03:39:47.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.521+0000 7f33d71d9740 1 --2- 172.21.15.38:0/365638776 >> [v2:172.21.15.195:6834/2771781100,v1:172.21.15.195:6836/2771781100] conn(0x55f8c1539f60 0x55f8c195f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:47.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.521+0000 7f33d71d9740 4 client.6864 mds_command: new command op to 6193 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:39:47.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.521+0000 7f33d71d9740 1 -- 172.21.15.38:0/365638776 --> [v2:172.21.15.195:6834/2771781100,v1:172.21.15.195:6836/2771781100] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f8c18979b0 con 0x55f8c1539f60 2024-09-21T03:39:47.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.521+0000 7f33ce00d640 1 --2- 172.21.15.38:0/365638776 >> [v2:172.21.15.195:6834/2771781100,v1:172.21.15.195:6836/2771781100] conn(0x55f8c1539f60 0x55f8c195f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:47.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.521+0000 7f33ce00d640 1 --2- 172.21.15.38:0/365638776 >> [v2:172.21.15.195:6834/2771781100,v1:172.21.15.195:6836/2771781100] conn(0x55f8c1539f60 0x55f8c195f0c0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=16be1793d903e5c3 server_cookie=29e2dc27323c4689 in_seq=0 out_seq=0 2024-09-21T03:39:47.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.521+0000 7f33ad7fa640 10 client.6864 ms_handle_connect on v2:172.21.15.195:6834/2771781100 2024-09-21T03:39:47.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.522+0000 7f33ad7fa640 1 -- 172.21.15.38:0/365638776 <== mds.0 v2:172.21.15.195:6834/2771781100 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f8c18979b0 con 0x55f8c1539f60 2024-09-21T03:39:47.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.522+0000 7f33ad7fa640 10 client.6864 handle_command_reply: tid=0 2024-09-21T03:39:47.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.522+0000 7f33d71d9740 1 client.6864 shutdown 2024-09-21T03:39:47.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.522+0000 7f33d71d9740 1 -- 172.21.15.38:0/365638776 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f8c155a6b0 msgr2=0x55f8c195c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:47.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.522+0000 7f33d71d9740 1 --2- 172.21.15.38:0/365638776 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f8c155a6b0 0x55f8c195c9c0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f33bc023040 tx=0x7f33bc013c10 comp rx=0 tx=0).stop 2024-09-21T03:39:47.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.522+0000 7f33d71d9740 1 -- 172.21.15.38:0/365638776 shutdown_connections 2024-09-21T03:39:47.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.522+0000 7f33d71d9740 1 --2- 172.21.15.38:0/365638776 >> [v2:172.21.15.195:6834/2771781100,v1:172.21.15.195:6836/2771781100] conn(0x55f8c1539f60 0x55f8c195f0c0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:47.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.522+0000 7f33d71d9740 1 --2- 172.21.15.38:0/365638776 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f8c157b390 0x55f8c196aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:47.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.522+0000 7f33d71d9740 1 --2- 172.21.15.38:0/365638776 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f8c155a6b0 0x55f8c195c9c0 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:47.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.522+0000 7f33d71d9740 1 --2- 172.21.15.38:0/365638776 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f8c15524d0 0x55f8c1936b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:47.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.522+0000 7f33ad7fa640 0 client.6864 ms_handle_reset on v2:172.21.15.195:6834/2771781100 2024-09-21T03:39:47.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.522+0000 7f33d71d9740 1 -- 172.21.15.38:0/365638776 >> 172.21.15.38:0/365638776 conn(0x55f8c156f4e0 msgr2=0x55f8c19228b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:47.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.522+0000 7f33d71d9740 1 -- 172.21.15.38:0/365638776 shutdown_connections 2024-09-21T03:39:47.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.522+0000 7f33d71d9740 1 -- 172.21.15.38:0/365638776 wait complete. 2024-09-21T03:39:47.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.523+0000 7f33d71d9740 20 client.6864 trim_cache size 0 max 16384 2024-09-21T03:39:47.529 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c has been restored 2024-09-21T03:39:47.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.538+0000 7f33d71d9740 1 Processor -- start 2024-09-21T03:39:47.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.538+0000 7f33d71d9740 1 -- start start 2024-09-21T03:39:47.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.539+0000 7f33d71d9740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f8c15524d0 0x55f8c19385d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:47.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.539+0000 7f33d71d9740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f8c155a6b0 0x55f8c1938b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:47.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.539+0000 7f33d71d9740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f8c157b390 0x55f8c195b640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:47.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.539+0000 7f33d71d9740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f33c82027d0 con 0x55f8c15524d0 2024-09-21T03:39:47.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.539+0000 7f33d71d9740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f33c8201870 con 0x55f8c157b390 2024-09-21T03:39:47.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.539+0000 7f33d71d9740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f33c8200bd0 con 0x55f8c155a6b0 2024-09-21T03:39:47.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.539+0000 7f33ce00d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f8c15524d0 0x55f8c19385d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:47.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.539+0000 7f33ce00d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f8c15524d0 0x55f8c19385d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42332/0 (socket says 172.21.15.38:42332) 2024-09-21T03:39:47.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.539+0000 7f33ce00d640 1 -- 172.21.15.38:0/3463559660 learned_addr learned my addr 172.21.15.38:0/3463559660 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:39:47.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.539+0000 7f33ce80e640 1 --2- 172.21.15.38:0/3463559660 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f8c157b390 0x55f8c195b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:47.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.539+0000 7f33ce00d640 1 -- 172.21.15.38:0/3463559660 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f8c155a6b0 msgr2=0x55f8c1938b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:47.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.539+0000 7f33cd80c640 1 --2- 172.21.15.38:0/3463559660 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f8c155a6b0 0x55f8c1938b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:47.543 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.539+0000 7f33ce00d640 1 --2- 172.21.15.38:0/3463559660 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f8c155a6b0 0x55f8c1938b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:47.543 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.539+0000 7f33ce00d640 1 -- 172.21.15.38:0/3463559660 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f8c157b390 msgr2=0x55f8c195b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:47.543 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.539+0000 7f33ce00d640 1 --2- 172.21.15.38:0/3463559660 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f8c157b390 0x55f8c195b640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:47.543 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.539+0000 7f33ce00d640 1 -- 172.21.15.38:0/3463559660 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f8c196ae40 con 0x55f8c15524d0 2024-09-21T03:39:47.543 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.539+0000 7f33ce80e640 1 --2- 172.21.15.38:0/3463559660 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f8c157b390 0x55f8c195b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:39:47.543 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.539+0000 7f33cd80c640 1 --2- 172.21.15.38:0/3463559660 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f8c155a6b0 0x55f8c1938b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:39:47.543 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.540+0000 7f33ce00d640 1 --2- 172.21.15.38:0/3463559660 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f8c15524d0 0x55f8c19385d0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f33b80b9620 tx=0x7f33b80b94f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b62b9516f60b0fd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:47.543 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.540+0000 7f33af7fe640 1 -- 172.21.15.38:0/3463559660 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33b8048070 con 0x55f8c15524d0 2024-09-21T03:39:47.543 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.540+0000 7f33af7fe640 1 -- 172.21.15.38:0/3463559660 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f33b8086040 con 0x55f8c15524d0 2024-09-21T03:39:47.543 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.540+0000 7f33af7fe640 1 -- 172.21.15.38:0/3463559660 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33b8085040 con 0x55f8c15524d0 2024-09-21T03:39:47.544 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.540+0000 7f33d71d9740 1 -- 172.21.15.38:0/3463559660 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f8c15524d0 msgr2=0x55f8c19385d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:47.544 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.540+0000 7f33d71d9740 1 --2- 172.21.15.38:0/3463559660 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f8c15524d0 0x55f8c19385d0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f33b80b9620 tx=0x7f33b80b94f0 comp rx=0 tx=0).stop 2024-09-21T03:39:47.544 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.541+0000 7f33d71d9740 1 -- 172.21.15.38:0/3463559660 shutdown_connections 2024-09-21T03:39:47.544 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.541+0000 7f33d71d9740 1 --2- 172.21.15.38:0/3463559660 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f8c157b390 0x55f8c195b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:47.544 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.541+0000 7f33d71d9740 1 --2- 172.21.15.38:0/3463559660 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f8c155a6b0 0x55f8c1938b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:47.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.541+0000 7f33d71d9740 1 --2- 172.21.15.38:0/3463559660 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f8c15524d0 0x55f8c19385d0 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:47.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.541+0000 7f33d71d9740 1 -- 172.21.15.38:0/3463559660 >> 172.21.15.38:0/3463559660 conn(0x55f8c156f4e0 msgr2=0x55f8c19369c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:47.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.541+0000 7f33d71d9740 1 -- 172.21.15.38:0/3463559660 shutdown_connections 2024-09-21T03:39:47.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.541+0000 7f33d71d9740 1 -- 172.21.15.38:0/3463559660 wait complete. 2024-09-21T03:39:47.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.541+0000 7f33d71d9740 1 -- start start 2024-09-21T03:39:47.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.542+0000 7f33d71d9740 1 Processor -- start 2024-09-21T03:39:47.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.543+0000 7f33d71d9740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f8c15524d0 0x55f8c1977540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:47.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.543+0000 7f33d71d9740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f8c155a6b0 0x55f8c1938570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:47.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.543+0000 7f33d71d9740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f8c157b390 0x55f8c1938ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:47.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.543+0000 7f33d71d9740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f33c8201220 con 0x55f8c157b390 2024-09-21T03:39:47.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.543+0000 7f33d71d9740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55f8c194bcf0 con 0x55f8c155a6b0 2024-09-21T03:39:47.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.543+0000 7f33d71d9740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55f8c19e73a0 con 0x55f8c15524d0 2024-09-21T03:39:47.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.543+0000 7f33d71d9740 10 client.0 _ll_register_callbacks cb 0x7f33bc023070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:39:47.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.543+0000 7f33ce80e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f8c157b390 0x55f8c1938ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:47.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.543+0000 7f33ce80e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f8c157b390 0x55f8c1938ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42338/0 (socket says 172.21.15.38:42338) 2024-09-21T03:39:47.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.543+0000 7f33ce80e640 1 -- 172.21.15.38:0/1222355801 learned_addr learned my addr 172.21.15.38:0/1222355801 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:39:47.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.543+0000 7f33ce00d640 1 --2- 172.21.15.38:0/1222355801 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f8c15524d0 0x55f8c1977540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:47.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.543+0000 7f33cd80c640 1 --2- 172.21.15.38:0/1222355801 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f8c155a6b0 0x55f8c1938570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:47.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.543+0000 7f33ce80e640 1 -- 172.21.15.38:0/1222355801 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f8c15524d0 msgr2=0x55f8c1977540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:47.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.543+0000 7f33ce80e640 1 --2- 172.21.15.38:0/1222355801 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f8c15524d0 0x55f8c1977540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:47.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.544+0000 7f33ce80e640 1 -- 172.21.15.38:0/1222355801 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f8c155a6b0 msgr2=0x55f8c1938570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:47.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.544+0000 7f33ce80e640 1 --2- 172.21.15.38:0/1222355801 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f8c155a6b0 0x55f8c1938570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:47.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.544+0000 7f33ce80e640 1 -- 172.21.15.38:0/1222355801 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f8c194d150 con 0x55f8c157b390 2024-09-21T03:39:47.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.544+0000 7f33ce80e640 1 --2- 172.21.15.38:0/1222355801 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f8c157b390 0x55f8c1938ab0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x55f8c194d5a0 tx=0x7f33c0010b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71f8151fe4873866 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:47.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.544+0000 7f33ad7fa640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-21T03:39:47.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.544+0000 7f33ad7fa640 1 -- 172.21.15.38:0/1222355801 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33c0023060 con 0x55f8c157b390 2024-09-21T03:39:47.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.544+0000 7f33ad7fa640 1 -- 172.21.15.38:0/1222355801 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 6187+0+0 (secure 0 0 0) 0x7f33c0004460 con 0x55f8c157b390 2024-09-21T03:39:47.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.544+0000 7f33d71d9740 1 -- 172.21.15.38:0/1222355801 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f33b8086040 con 0x55f8c157b390 2024-09-21T03:39:47.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.544+0000 7f33ad7fa640 1 -- 172.21.15.38:0/1222355801 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7f33c0029050 con 0x55f8c157b390 2024-09-21T03:39:47.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.544+0000 7f33d71d9740 10 client.6870 fetch_fsmap learned FSMap version 74 2024-09-21T03:39:47.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.544+0000 7f33d71d9740 1 -- 172.21.15.38:0/1222355801 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) -- 0x55f8c1977a80 con 0x55f8c157b390 2024-09-21T03:39:47.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.544+0000 7f33ad7fa640 1 -- 172.21.15.38:0/1222355801 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f33b8086040 con 0x55f8c157b390 2024-09-21T03:39:47.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.544+0000 7f33ad7fa640 1 -- 172.21.15.38:0/1222355801 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 74) ==== 3502+0+0 (secure 0 0 0) 0x7f33c002c020 con 0x55f8c157b390 2024-09-21T03:39:47.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.544+0000 7f33d71d9740 10 client.6870 fetch_fsmap finished waiting for FSMap version 74 2024-09-21T03:39:47.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.544+0000 7f33d71d9740 10 client.6870 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.d 2024-09-21T03:39:47.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.544+0000 7f33d71d9740 20 client.6870 populate_metadata read hostname 'smithi038' 2024-09-21T03:39:47.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.544+0000 7f33d71d9740 1 --2- 172.21.15.38:0/1222355801 >> [v2:172.21.15.195:6834/2771781100,v1:172.21.15.195:6836/2771781100] conn(0x55f8c1923de0 0x55f8c19e6410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:47.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.544+0000 7f33d71d9740 4 client.6870 mds_command: new command op to 6193 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T03:39:47.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.544+0000 7f33d71d9740 1 -- 172.21.15.38:0/1222355801 --> [v2:172.21.15.195:6834/2771781100,v1:172.21.15.195:6836/2771781100] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f8c1893460 con 0x55f8c1923de0 2024-09-21T03:39:47.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.545+0000 7f33ce00d640 1 --2- 172.21.15.38:0/1222355801 >> [v2:172.21.15.195:6834/2771781100,v1:172.21.15.195:6836/2771781100] conn(0x55f8c1923de0 0x55f8c19e6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:47.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.545+0000 7f33ce00d640 1 --2- 172.21.15.38:0/1222355801 >> [v2:172.21.15.195:6834/2771781100,v1:172.21.15.195:6836/2771781100] conn(0x55f8c1923de0 0x55f8c19e6410 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=24fd3e9bc83e54b0 server_cookie=ad75bde764391a89 in_seq=0 out_seq=0 2024-09-21T03:39:47.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.545+0000 7f33ad7fa640 10 client.6870 ms_handle_connect on v2:172.21.15.195:6834/2771781100 2024-09-21T03:39:47.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.566+0000 7f33ad7fa640 1 -- 172.21.15.38:0/1222355801 <== mds.0 v2:172.21.15.195:6834/2771781100 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f8c1893460 con 0x55f8c1923de0 2024-09-21T03:39:47.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.566+0000 7f33ad7fa640 10 client.6870 handle_command_reply: tid=0 2024-09-21T03:39:47.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.566+0000 7f33d71d9740 1 client.6870 shutdown 2024-09-21T03:39:47.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.566+0000 7f33d71d9740 1 -- 172.21.15.38:0/1222355801 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f8c157b390 msgr2=0x55f8c1938ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:47.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.566+0000 7f33d71d9740 1 --2- 172.21.15.38:0/1222355801 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f8c157b390 0x55f8c1938ab0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x55f8c194d5a0 tx=0x7f33c0010b40 comp rx=0 tx=0).stop 2024-09-21T03:39:47.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.566+0000 7f33d71d9740 1 -- 172.21.15.38:0/1222355801 shutdown_connections 2024-09-21T03:39:47.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.566+0000 7f33d71d9740 1 --2- 172.21.15.38:0/1222355801 >> [v2:172.21.15.195:6834/2771781100,v1:172.21.15.195:6836/2771781100] conn(0x55f8c1923de0 0x55f8c19e6410 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:47.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.566+0000 7f33d71d9740 1 --2- 172.21.15.38:0/1222355801 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f8c157b390 0x55f8c1938ab0 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:47.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.566+0000 7f33ad7fa640 0 client.6870 ms_handle_reset on v2:172.21.15.195:6834/2771781100 2024-09-21T03:39:47.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.566+0000 7f33d71d9740 1 --2- 172.21.15.38:0/1222355801 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f8c155a6b0 0x55f8c1938570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:47.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.566+0000 7f33d71d9740 1 --2- 172.21.15.38:0/1222355801 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f8c15524d0 0x55f8c1977540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:47.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.566+0000 7f33d71d9740 1 -- 172.21.15.38:0/1222355801 >> 172.21.15.38:0/1222355801 conn(0x55f8c156f4e0 msgr2=0x55f8c19369a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:47.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.567+0000 7f33d71d9740 1 -- 172.21.15.38:0/1222355801 shutdown_connections 2024-09-21T03:39:47.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.567+0000 7f33d71d9740 1 -- 172.21.15.38:0/1222355801 wait complete. 2024-09-21T03:39:47.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.567+0000 7f33d71d9740 20 client.6870 trim_cache size 0 max 16384 2024-09-21T03:39:47.570 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-09-21T03:39:47.570 INFO:teuthology.orchestra.run.smithi038.stdout: "message": "", 2024-09-21T03:39:47.570 INFO:teuthology.orchestra.run.smithi038.stdout: "return_code": 0 2024-09-21T03:39:47.570 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-09-21T03:39:47.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.567+0000 7f33d0271640 1 -- 172.21.15.38:0/247317362 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f33b4078b90 msgr2=0x7f33b407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:47.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.567+0000 7f33d0271640 1 --2- 172.21.15.38:0/247317362 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f33b4078b90 0x7f33b407b0a0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f33c809edf0 tx=0x7f33bc009040 comp rx=0 tx=0).stop 2024-09-21T03:39:47.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.567+0000 7f33d0271640 1 -- 172.21.15.38:0/247317362 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f33c806def0 msgr2=0x7f33c80cae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:47.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.567+0000 7f33d0271640 1 --2- 172.21.15.38:0/247317362 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f33c806def0 0x7f33c80cae20 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f33b8034040 tx=0x7f33b80035f0 comp rx=0 tx=0).stop 2024-09-21T03:39:47.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.567+0000 7f33d0271640 1 -- 172.21.15.38:0/247317362 shutdown_connections 2024-09-21T03:39:47.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.567+0000 7f33d0271640 1 --2- 172.21.15.38:0/247317362 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f33b4078b90 0x7f33b407b0a0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:47.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.567+0000 7f33d0271640 1 --2- 172.21.15.38:0/247317362 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f33c80b5670 0x7f33c806d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:47.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.567+0000 7f33d0271640 1 --2- 172.21.15.38:0/247317362 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f33c80b5260 0x7f33c806d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:47.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.567+0000 7f33d0271640 1 --2- 172.21.15.38:0/247317362 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f33c806def0 0x7f33c80cae20 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:47.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.567+0000 7f33d0271640 1 -- 172.21.15.38:0/247317362 >> 172.21.15.38:0/247317362 conn(0x7f33c80cdca0 msgr2=0x7f33c80ae3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:47.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.567+0000 7f33d0271640 1 -- 172.21.15.38:0/247317362 shutdown_connections 2024-09-21T03:39:47.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.567+0000 7f33d0271640 1 -- 172.21.15.38:0/247317362 wait complete. 2024-09-21T03:39:47.581 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-21T03:39:47.581 DEBUG:teuthology.orchestra.run.smithi038:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T03:39:47.624 DEBUG:teuthology.orchestra.run.smithi195:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T03:39:47.674 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-21T03:39:47.674 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T03:39:47.675 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-21T03:39:47.675 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:39:47.691 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-21T03:39:47.691 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:39:47.748 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T03:39:47.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.903+0000 7f93b887e640 1 -- 172.21.15.38:0/1006678979 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f939c003fd0 msgr2=0x7f939c004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:47.906 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.903+0000 7f93b887e640 1 --2- 172.21.15.38:0/1006678979 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f939c003fd0 0x7f939c004800 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f93b40700d0 tx=0x7f939c005830 comp rx=0 tx=0).stop 2024-09-21T03:39:47.906 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.903+0000 7f93b2d9d640 1 -- 172.21.15.38:0/1006678979 reap_dead start 2024-09-21T03:39:47.906 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.903+0000 7f93b887e640 1 -- 172.21.15.38:0/1006678979 shutdown_connections 2024-09-21T03:39:47.906 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.903+0000 7f93b887e640 1 -- 172.21.15.38:0/1006678979 >> 172.21.15.38:0/1006678979 conn(0x7f93b409d830 msgr2=0x7f93b40b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:47.906 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.903+0000 7f93b887e640 1 -- 172.21.15.38:0/1006678979 shutdown_connections 2024-09-21T03:39:47.906 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.903+0000 7f93b887e640 1 -- 172.21.15.38:0/1006678979 wait complete. 2024-09-21T03:39:47.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.905+0000 7f93b887e640 1 Processor -- start 2024-09-21T03:39:47.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.905+0000 7f93b887e640 1 -- start start 2024-09-21T03:39:47.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.907+0000 7f93b887e640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f939c003fd0 0x7f93b409e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:47.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.907+0000 7f93b887e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f93b4157c10 0x7f93b41574a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:47.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.907+0000 7f93b887e640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f93b40a0560 0x7f93b40a0970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:47.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.907+0000 7f93b887e640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f93b40d0be0 con 0x7f93b4157c10 2024-09-21T03:39:47.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.907+0000 7f93b887e640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f93b40d0d60 con 0x7f939c003fd0 2024-09-21T03:39:47.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.907+0000 7f93b887e640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f93b40d0ee0 con 0x7f93b40a0560 2024-09-21T03:39:47.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.907+0000 7f93b1d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f93b4157c10 0x7f93b41574a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:47.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.907+0000 7f93b1d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f93b4157c10 0x7f93b41574a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42354/0 (socket says 172.21.15.38:42354) 2024-09-21T03:39:47.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.907+0000 7f93b1d9b640 1 -- 172.21.15.38:0/3261290585 learned_addr learned my addr 172.21.15.38:0/3261290585 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:39:47.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.907+0000 7f93b2d9d640 1 --2- 172.21.15.38:0/3261290585 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f93b40a0560 0x7f93b40a0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:47.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.907+0000 7f93b259c640 1 --2- 172.21.15.38:0/3261290585 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f939c003fd0 0x7f93b409e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:47.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.907+0000 7f93b1d9b640 1 -- 172.21.15.38:0/3261290585 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f93b40a0560 msgr2=0x7f93b40a0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:47.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.907+0000 7f93b1d9b640 1 --2- 172.21.15.38:0/3261290585 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f93b40a0560 0x7f93b40a0970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:47.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.907+0000 7f93b1d9b640 1 -- 172.21.15.38:0/3261290585 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f939c003fd0 msgr2=0x7f93b409e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:47.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.907+0000 7f93b1d9b640 1 --2- 172.21.15.38:0/3261290585 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f939c003fd0 0x7f93b409e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:47.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.907+0000 7f93b1d9b640 1 -- 172.21.15.38:0/3261290585 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93b41579e0 con 0x7f93b4157c10 2024-09-21T03:39:47.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.907+0000 7f93b1d9b640 1 --2- 172.21.15.38:0/3261290585 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f93b4157c10 0x7f93b41574a0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f93a802b460 tx=0x7f93a80336f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:47.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.907+0000 7f93a2ffd640 1 -- 172.21.15.38:0/3261290585 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93a8038040 con 0x7f93b4157c10 2024-09-21T03:39:47.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.908+0000 7f93a2ffd640 1 -- 172.21.15.38:0/3261290585 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f93a8041070 con 0x7f93b4157c10 2024-09-21T03:39:47.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.908+0000 7f93a2ffd640 1 -- 172.21.15.38:0/3261290585 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93a8033c70 con 0x7f93b4157c10 2024-09-21T03:39:47.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.908+0000 7f93b887e640 1 -- 172.21.15.38:0/3261290585 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93b41f6330 con 0x7f93b4157c10 2024-09-21T03:39:47.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.908+0000 7f93b887e640 1 -- 172.21.15.38:0/3261290585 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93b40ce150 con 0x7f93b4157c10 2024-09-21T03:39:47.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.909+0000 7f93b887e640 1 -- 172.21.15.38:0/3261290585 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93b40d0170 con 0x7f93b4157c10 2024-09-21T03:39:47.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.910+0000 7f93a2ffd640 1 -- 172.21.15.38:0/3261290585 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f93a8046460 con 0x7f93b4157c10 2024-09-21T03:39:47.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.910+0000 7f93a2ffd640 1 --2- 172.21.15.38:0/3261290585 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9394078b90 0x7f939407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:47.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.910+0000 7f93a2ffd640 1 -- 172.21.15.38:0/3261290585 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 6187+0+0 (secure 0 0 0) 0x7f93a80bb770 con 0x7f93b4157c10 2024-09-21T03:39:47.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.910+0000 7f93b259c640 1 --2- 172.21.15.38:0/3261290585 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9394078b90 0x7f939407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:47.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.911+0000 7f93b259c640 1 --2- 172.21.15.38:0/3261290585 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9394078b90 0x7f939407b0a0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f93a4009b10 tx=0x7f93a4005f90 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:47.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:47.915+0000 7f93a2ffd640 1 -- 172.21.15.38:0/3261290585 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f93a8083ad0 con 0x7f93b4157c10 2024-09-21T03:39:48.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:48.114+0000 7f93b887e640 1 -- 172.21.15.38:0/3261290585 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f93b40b4a90 con 0x7f93b4157c10 2024-09-21T03:39:48.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:48.114+0000 7f93a2ffd640 1 -- 172.21.15.38:0/3261290585 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v83) ==== 80+0+122 (secure 0 0 0) 0x7f93a80833d0 con 0x7f93b4157c10 2024-09-21T03:39:48.116 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6833/1144075315 2024-09-22T03:39:42.556819+0000 2024-09-21T03:39:48.116 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6832/1144075315 2024-09-22T03:39:42.556819+0000 2024-09-21T03:39:48.116 INFO:teuthology.orchestra.run.smithi038.stderr:listed 2 entries 2024-09-21T03:39:48.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:48.116+0000 7f93b887e640 1 -- 172.21.15.38:0/3261290585 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9394078b90 msgr2=0x7f939407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:48.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:48.116+0000 7f93b887e640 1 --2- 172.21.15.38:0/3261290585 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9394078b90 0x7f939407b0a0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f93a4009b10 tx=0x7f93a4005f90 comp rx=0 tx=0).stop 2024-09-21T03:39:48.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:48.116+0000 7f93b887e640 1 -- 172.21.15.38:0/3261290585 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f93b4157c10 msgr2=0x7f93b41574a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:48.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:48.116+0000 7f93b887e640 1 --2- 172.21.15.38:0/3261290585 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f93b4157c10 0x7f93b41574a0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f93a802b460 tx=0x7f93a80336f0 comp rx=0 tx=0).stop 2024-09-21T03:39:48.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:48.116+0000 7f93b887e640 1 -- 172.21.15.38:0/3261290585 shutdown_connections 2024-09-21T03:39:48.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:48.116+0000 7f93b887e640 1 --2- 172.21.15.38:0/3261290585 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9394078b90 0x7f939407b0a0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:48.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:48.116+0000 7f93b887e640 1 --2- 172.21.15.38:0/3261290585 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f93b40a0560 0x7f93b40a0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:48.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:48.116+0000 7f93b887e640 1 --2- 172.21.15.38:0/3261290585 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f93b4157c10 0x7f93b41574a0 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:48.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:48.116+0000 7f93b887e640 1 --2- 172.21.15.38:0/3261290585 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f939c003fd0 0x7f93b409e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:48.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:48.116+0000 7f93b887e640 1 -- 172.21.15.38:0/3261290585 >> 172.21.15.38:0/3261290585 conn(0x7f93b409d830 msgr2=0x7f93b40b31a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:48.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:48.116+0000 7f93b887e640 1 -- 172.21.15.38:0/3261290585 shutdown_connections 2024-09-21T03:39:48.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:48.116+0000 7f93b887e640 1 -- 172.21.15.38:0/3261290585 wait complete. 2024-09-21T03:39:48.129 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-21T03:39:48.129 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:39:48.129 DEBUG:teuthology.orchestra.run.smithi038:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-21T03:39:48.158 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi038.front.sepia.ceph.com 2024-09-21T03:39:48.159 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:39:48.159 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T03:39:48.215 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T03:39:48.215 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-09-21T03:39:48.215 DEBUG:teuthology.orchestra.run.smithi195:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:39:48.235 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056014,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497444k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=8978 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248720k,nr_inodes=812180,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497444k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T03:39:48.235 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T03:39:48.235 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-21T03:39:48.235 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:39:48.272 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056795,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2354 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T03:39:48.272 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T03:39:48.272 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-09-21T03:39:48.272 DEBUG:teuthology.orchestra.run.smithi195:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:39:48.292 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056014,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497444k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=8978 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248720k,nr_inodes=812180,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497444k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T03:39:48.293 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:39:48.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:48.427+0000 7f683048c640 1 -- 172.21.15.38:0/3187782085 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6814004800 msgr2=0x7f6814004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:48.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:48.427+0000 7f683048c640 1 --2- 172.21.15.38:0/3187782085 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6814004800 0x7f6814004c10 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f68180016c0 tx=0x7f6814005830 comp rx=0 tx=0).stop 2024-09-21T03:39:48.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:48.427+0000 7f683048c640 1 -- 172.21.15.38:0/3187782085 shutdown_connections 2024-09-21T03:39:48.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:48.427+0000 7f683048c640 1 --2- 172.21.15.38:0/3187782085 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6814004800 0x7f6814004c10 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:48.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:48.427+0000 7f683048c640 1 -- 172.21.15.38:0/3187782085 >> 172.21.15.38:0/3187782085 conn(0x7f68280cef40 msgr2=0x7f68280cf350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:48.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:48.427+0000 7f683048c640 1 -- 172.21.15.38:0/3187782085 shutdown_connections 2024-09-21T03:39:48.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:48.427+0000 7f683048c640 1 -- 172.21.15.38:0/3187782085 wait complete. 2024-09-21T03:39:48.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:48.429+0000 7f683048c640 1 Processor -- start 2024-09-21T03:39:48.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:48.429+0000 7f683048c640 1 -- start start 2024-09-21T03:39:48.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:48.430+0000 7f683048c640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6814004800 0x7f682815b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:48.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:48.430+0000 7f683048c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f682815bd00 0x7f68280cba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:48.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:48.430+0000 7f683048c640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f682814f650 0x7f68280cbfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:48.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:48.430+0000 7f683048c640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f68280d1b50 con 0x7f682815bd00 2024-09-21T03:39:48.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:48.430+0000 7f683048c640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f68280d19d0 con 0x7f6814004800 2024-09-21T03:39:48.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:48.430+0000 7f683048c640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f68280d1cd0 con 0x7f682814f650 2024-09-21T03:39:48.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:48.430+0000 7f682da27640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f682815bd00 0x7f68280cba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:48.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:48.430+0000 7f682da27640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f682815bd00 0x7f68280cba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42376/0 (socket says 172.21.15.38:42376) 2024-09-21T03:39:48.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:48.430+0000 7f682da27640 1 -- 172.21.15.38:0/2865375941 learned_addr learned my addr 172.21.15.38:0/2865375941 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:39:48.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:48.431+0000 7f682ea29640 1 --2- 172.21.15.38:0/2865375941 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f682814f650 0x7f68280cbfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:48.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:48.431+0000 7f682e228640 1 --2- 172.21.15.38:0/2865375941 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6814004800 0x7f682815b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:48.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:48.431+0000 7f682da27640 1 -- 172.21.15.38:0/2865375941 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f682814f650 msgr2=0x7f68280cbfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:48.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:48.431+0000 7f682da27640 1 --2- 172.21.15.38:0/2865375941 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f682814f650 0x7f68280cbfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:48.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:48.431+0000 7f682da27640 1 -- 172.21.15.38:0/2865375941 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6814004800 msgr2=0x7f682815b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:48.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:48.431+0000 7f682da27640 1 --2- 172.21.15.38:0/2865375941 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6814004800 0x7f682815b7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:48.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:48.431+0000 7f682da27640 1 -- 172.21.15.38:0/2865375941 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68281f4fc0 con 0x7f682815bd00 2024-09-21T03:39:48.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:48.431+0000 7f682da27640 1 --2- 172.21.15.38:0/2865375941 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f682815bd00 0x7f68280cba80 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f68180017e0 tx=0x7f68180063a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:48.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:48.431+0000 7f681effd640 1 -- 172.21.15.38:0/2865375941 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6818042070 con 0x7f682815bd00 2024-09-21T03:39:48.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:48.431+0000 7f681effd640 1 -- 172.21.15.38:0/2865375941 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6818039070 con 0x7f682815bd00 2024-09-21T03:39:48.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:48.431+0000 7f683048c640 1 -- 172.21.15.38:0/2865375941 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68281f52a0 con 0x7f682815bd00 2024-09-21T03:39:48.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:48.431+0000 7f681effd640 1 -- 172.21.15.38:0/2865375941 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6818003d70 con 0x7f682815bd00 2024-09-21T03:39:48.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:48.431+0000 7f683048c640 1 -- 172.21.15.38:0/2865375941 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68281f5860 con 0x7f682815bd00 2024-09-21T03:39:48.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:48.432+0000 7f683048c640 1 -- 172.21.15.38:0/2865375941 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f682814f230 con 0x7f682815bd00 2024-09-21T03:39:48.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:48.433+0000 7f681effd640 1 -- 172.21.15.38:0/2865375941 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f681803b070 con 0x7f682815bd00 2024-09-21T03:39:48.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:48.434+0000 7f681effd640 1 --2- 172.21.15.38:0/2865375941 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f680c078bf0 0x7f680c07b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:48.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:48.434+0000 7f681effd640 1 -- 172.21.15.38:0/2865375941 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 6187+0+0 (secure 0 0 0) 0x7f68180ba5b0 con 0x7f682815bd00 2024-09-21T03:39:48.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:48.434+0000 7f682e228640 1 --2- 172.21.15.38:0/2865375941 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f680c078bf0 0x7f680c07b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:48.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:48.435+0000 7f682e228640 1 --2- 172.21.15.38:0/2865375941 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f680c078bf0 0x7f680c07b100 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f6824001eb0 tx=0x7f6824000ed0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:48.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:48.438+0000 7f681effd640 1 -- 172.21.15.38:0/2865375941 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f6818082800 con 0x7f682815bd00 2024-09-21T03:39:48.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:48.662+0000 7f683048c640 1 -- 172.21.15.38:0/2865375941 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f68280cccb0 con 0x7f682815bd00 2024-09-21T03:39:48.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:48.663+0000 7f681effd640 1 -- 172.21.15.38:0/2865375941 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) ==== 94+0+5233 (secure 0 0 0) 0x7f6818082100 con 0x7f682815bd00 2024-09-21T03:39:48.665 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:39:48.665 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":74,"btime":"2024-09-21T03:39:45:574648+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5783,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/1358588311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1358588311},{"type":"v1","addr":"172.21.15.38:6835","nonce":1358588311}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63},{"gid":5789,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/1676483003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":1676483003},{"type":"v1","addr":"172.21.15.195:6837","nonce":1676483003}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63},{"gid":6813,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/3644074938","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":3644074938},{"type":"v1","addr":"172.21.15.38:6833","nonce":3644074938}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":72}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:39:04.779948+0000","modified":"2024-09-21T03:39:45.574643+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6193},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6193":{"gid":6193,"name":"d","rank":0,"incarnation":71,"state":"up:active","state_seq":14,"addr":"172.21.15.195:6836/2771781100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":2771781100},{"type":"v1","addr":"172.21.15.195:6836","nonce":2771781100}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6193,"qdb_cluster":[6193]},"id":5}]} 2024-09-21T03:39:48.665 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 74 2024-09-21T03:39:48.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:48.665+0000 7f683048c640 1 -- 172.21.15.38:0/2865375941 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f680c078bf0 msgr2=0x7f680c07b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:48.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:48.665+0000 7f683048c640 1 --2- 172.21.15.38:0/2865375941 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f680c078bf0 0x7f680c07b100 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f6824001eb0 tx=0x7f6824000ed0 comp rx=0 tx=0).stop 2024-09-21T03:39:48.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:48.665+0000 7f683048c640 1 -- 172.21.15.38:0/2865375941 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f682815bd00 msgr2=0x7f68280cba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:48.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:48.665+0000 7f683048c640 1 --2- 172.21.15.38:0/2865375941 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f682815bd00 0x7f68280cba80 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f68180017e0 tx=0x7f68180063a0 comp rx=0 tx=0).stop 2024-09-21T03:39:48.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:48.665+0000 7f683048c640 1 -- 172.21.15.38:0/2865375941 shutdown_connections 2024-09-21T03:39:48.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:48.665+0000 7f683048c640 1 --2- 172.21.15.38:0/2865375941 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f680c078bf0 0x7f680c07b100 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:48.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:48.665+0000 7f683048c640 1 --2- 172.21.15.38:0/2865375941 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f682814f650 0x7f68280cbfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:48.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:48.665+0000 7f683048c640 1 --2- 172.21.15.38:0/2865375941 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f682815bd00 0x7f68280cba80 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:48.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:48.665+0000 7f683048c640 1 --2- 172.21.15.38:0/2865375941 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6814004800 0x7f682815b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:48.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:48.665+0000 7f683048c640 1 -- 172.21.15.38:0/2865375941 >> 172.21.15.38:0/2865375941 conn(0x7f68280cef40 msgr2=0x7f68281522a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:48.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:48.665+0000 7f683048c640 1 -- 172.21.15.38:0/2865375941 shutdown_connections 2024-09-21T03:39:48.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:48.665+0000 7f683048c640 1 -- 172.21.15.38:0/2865375941 wait complete. 2024-09-21T03:39:48.678 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:39:48.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:48.793+0000 7f5f5f5e6640 1 -- 172.21.15.38:0/2536272740 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5f44004800 msgr2=0x7f5f44004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:48.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:48.793+0000 7f5f5f5e6640 1 --2- 172.21.15.38:0/2536272740 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5f44004800 0x7f5f44004c10 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f5f5808d890 tx=0x7f5f44005830 comp rx=0 tx=0).stop 2024-09-21T03:39:48.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:48.793+0000 7f5f5db83640 1 -- 172.21.15.38:0/2536272740 reap_dead start 2024-09-21T03:39:48.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:48.793+0000 7f5f5f5e6640 1 -- 172.21.15.38:0/2536272740 shutdown_connections 2024-09-21T03:39:48.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:48.793+0000 7f5f5f5e6640 1 -- 172.21.15.38:0/2536272740 >> 172.21.15.38:0/2536272740 conn(0x7f5f580b6b20 msgr2=0x7f5f580d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:48.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:48.793+0000 7f5f5f5e6640 1 -- 172.21.15.38:0/2536272740 shutdown_connections 2024-09-21T03:39:48.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:48.794+0000 7f5f5f5e6640 1 -- 172.21.15.38:0/2536272740 wait complete. 2024-09-21T03:39:48.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:48.795+0000 7f5f5f5e6640 1 Processor -- start 2024-09-21T03:39:48.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:48.795+0000 7f5f5f5e6640 1 -- start start 2024-09-21T03:39:48.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:48.797+0000 7f5f5f5e6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5f44004800 0x7f5f581f59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:48.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:48.797+0000 7f5f5f5e6640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5f58151cf0 0x7f5f580ce340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:48.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:48.797+0000 7f5f5f5e6640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5f580ce880 0x7f5f581f8560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:48.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:48.797+0000 7f5f5f5e6640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f5f58162690 con 0x7f5f44004800 2024-09-21T03:39:48.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:48.797+0000 7f5f5f5e6640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f5f580b39a0 con 0x7f5f580ce880 2024-09-21T03:39:48.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:48.797+0000 7f5f5f5e6640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f5f58162810 con 0x7f5f58151cf0 2024-09-21T03:39:48.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:48.797+0000 7f5f5d382640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5f44004800 0x7f5f581f59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:48.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:48.797+0000 7f5f5d382640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5f44004800 0x7f5f581f59a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42400/0 (socket says 172.21.15.38:42400) 2024-09-21T03:39:48.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:48.797+0000 7f5f5d382640 1 -- 172.21.15.38:0/3324519245 learned_addr learned my addr 172.21.15.38:0/3324519245 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:39:48.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:48.797+0000 7f5f5cb81640 1 --2- 172.21.15.38:0/3324519245 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5f58151cf0 0x7f5f580ce340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:48.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:48.797+0000 7f5f5db83640 1 --2- 172.21.15.38:0/3324519245 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5f580ce880 0x7f5f581f8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:48.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:48.797+0000 7f5f5d382640 1 -- 172.21.15.38:0/3324519245 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5f58151cf0 msgr2=0x7f5f580ce340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:48.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:48.797+0000 7f5f5d382640 1 --2- 172.21.15.38:0/3324519245 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5f58151cf0 0x7f5f580ce340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:48.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:48.797+0000 7f5f5d382640 1 -- 172.21.15.38:0/3324519245 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5f580ce880 msgr2=0x7f5f581f8560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:48.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:48.797+0000 7f5f5d382640 1 --2- 172.21.15.38:0/3324519245 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5f580ce880 0x7f5f581f8560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:48.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:48.797+0000 7f5f5d382640 1 -- 172.21.15.38:0/3324519245 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f580cec90 con 0x7f5f44004800 2024-09-21T03:39:48.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:48.797+0000 7f5f5cb81640 1 --2- 172.21.15.38:0/3324519245 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5f58151cf0 0x7f5f580ce340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:39:48.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:48.797+0000 7f5f5db83640 1 --2- 172.21.15.38:0/3324519245 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5f580ce880 0x7f5f581f8560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:39:48.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:48.797+0000 7f5f5d382640 1 --2- 172.21.15.38:0/3324519245 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5f44004800 0x7f5f581f59a0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f5f4c006600 tx=0x7f5f4c0039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:48.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:48.797+0000 7f5f49ffb640 1 -- 172.21.15.38:0/3324519245 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f4c015070 con 0x7f5f44004800 2024-09-21T03:39:48.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:48.797+0000 7f5f49ffb640 1 -- 172.21.15.38:0/3324519245 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5f4c011920 con 0x7f5f44004800 2024-09-21T03:39:48.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:48.797+0000 7f5f5f5e6640 1 -- 172.21.15.38:0/3324519245 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f581f5ee0 con 0x7f5f44004800 2024-09-21T03:39:48.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:48.797+0000 7f5f49ffb640 1 -- 172.21.15.38:0/3324519245 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f4c011d70 con 0x7f5f44004800 2024-09-21T03:39:48.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:48.798+0000 7f5f5f5e6640 1 -- 172.21.15.38:0/3324519245 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f5815a7b0 con 0x7f5f44004800 2024-09-21T03:39:48.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:48.799+0000 7f5f5f5e6640 1 -- 172.21.15.38:0/3324519245 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f58162f00 con 0x7f5f44004800 2024-09-21T03:39:48.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:48.799+0000 7f5f49ffb640 1 -- 172.21.15.38:0/3324519245 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f5f4c018070 con 0x7f5f44004800 2024-09-21T03:39:48.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:48.800+0000 7f5f49ffb640 1 --2- 172.21.15.38:0/3324519245 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5f3c078b90 0x7f5f3c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:48.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:48.800+0000 7f5f49ffb640 1 -- 172.21.15.38:0/3324519245 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 6187+0+0 (secure 0 0 0) 0x7f5f4c09bc90 con 0x7f5f44004800 2024-09-21T03:39:48.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:48.800+0000 7f5f5cb81640 1 --2- 172.21.15.38:0/3324519245 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5f3c078b90 0x7f5f3c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:48.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:48.801+0000 7f5f5cb81640 1 --2- 172.21.15.38:0/3324519245 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5f3c078b90 0x7f5f3c07b0a0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f5f5808d890 tx=0x7f5f540046c0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:48.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:48.805+0000 7f5f49ffb640 1 -- 172.21.15.38:0/3324519245 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f5f580cec90 con 0x7f5f44004800 2024-09-21T03:39:49.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:49.029+0000 7f5f5f5e6640 1 -- 172.21.15.38:0/3324519245 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5f58068fa0 con 0x7f5f44004800 2024-09-21T03:39:49.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:49.029+0000 7f5f49ffb640 1 -- 172.21.15.38:0/3324519245 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) ==== 94+0+5233 (secure 0 0 0) 0x7f5f58162bd0 con 0x7f5f44004800 2024-09-21T03:39:49.031 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:39:49.031 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":74,"btime":"2024-09-21T03:39:45:574648+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5783,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/1358588311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1358588311},{"type":"v1","addr":"172.21.15.38:6835","nonce":1358588311}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63},{"gid":5789,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/1676483003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":1676483003},{"type":"v1","addr":"172.21.15.195:6837","nonce":1676483003}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63},{"gid":6813,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/3644074938","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":3644074938},{"type":"v1","addr":"172.21.15.38:6833","nonce":3644074938}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":72}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:39:04.779948+0000","modified":"2024-09-21T03:39:45.574643+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6193},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6193":{"gid":6193,"name":"d","rank":0,"incarnation":71,"state":"up:active","state_seq":14,"addr":"172.21.15.195:6836/2771781100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":2771781100},{"type":"v1","addr":"172.21.15.195:6836","nonce":2771781100}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6193,"qdb_cluster":[6193]},"id":5}]} 2024-09-21T03:39:49.032 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 74 2024-09-21T03:39:49.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:49.031+0000 7f5f5f5e6640 1 -- 172.21.15.38:0/3324519245 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5f3c078b90 msgr2=0x7f5f3c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:49.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:49.031+0000 7f5f5f5e6640 1 --2- 172.21.15.38:0/3324519245 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5f3c078b90 0x7f5f3c07b0a0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f5f5808d890 tx=0x7f5f540046c0 comp rx=0 tx=0).stop 2024-09-21T03:39:49.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:49.031+0000 7f5f5f5e6640 1 -- 172.21.15.38:0/3324519245 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5f44004800 msgr2=0x7f5f581f59a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:49.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:49.031+0000 7f5f5f5e6640 1 --2- 172.21.15.38:0/3324519245 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5f44004800 0x7f5f581f59a0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f5f4c006600 tx=0x7f5f4c0039c0 comp rx=0 tx=0).stop 2024-09-21T03:39:49.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:49.031+0000 7f5f5f5e6640 1 -- 172.21.15.38:0/3324519245 shutdown_connections 2024-09-21T03:39:49.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:49.031+0000 7f5f5f5e6640 1 --2- 172.21.15.38:0/3324519245 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5f580ce880 0x7f5f581f8560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:49.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:49.031+0000 7f5f5f5e6640 1 --2- 172.21.15.38:0/3324519245 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5f3c078b90 0x7f5f3c07b0a0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:49.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:49.031+0000 7f5f5f5e6640 1 --2- 172.21.15.38:0/3324519245 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5f58151cf0 0x7f5f580ce340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:49.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:49.031+0000 7f5f5f5e6640 1 --2- 172.21.15.38:0/3324519245 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5f44004800 0x7f5f581f59a0 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:49.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:49.031+0000 7f5f5f5e6640 1 -- 172.21.15.38:0/3324519245 >> 172.21.15.38:0/3324519245 conn(0x7f5f580b6b20 msgr2=0x7f5f580d1630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:49.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:49.032+0000 7f5f5f5e6640 1 -- 172.21.15.38:0/3324519245 shutdown_connections 2024-09-21T03:39:49.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:49.032+0000 7f5f5f5e6640 1 -- 172.21.15.38:0/3324519245 wait complete. 2024-09-21T03:39:49.045 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T03:39:49.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:49.160+0000 7fa1e1320640 1 -- 172.21.15.38:0/2422407482 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa1dc162f00 msgr2=0x7fa1dc0d24d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:49.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:49.160+0000 7fa1e1320640 1 --2- 172.21.15.38:0/2422407482 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa1dc162f00 0x7fa1dc0d24d0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fa1c4009860 tx=0x7fa1c402a530 comp rx=0 tx=0).stop 2024-09-21T03:39:49.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:49.160+0000 7fa1e1320640 1 -- 172.21.15.38:0/2422407482 shutdown_connections 2024-09-21T03:39:49.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:49.160+0000 7fa1e1320640 1 --2- 172.21.15.38:0/2422407482 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa1dc162f00 0x7fa1dc0d24d0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:49.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:49.160+0000 7fa1e1320640 1 -- 172.21.15.38:0/2422407482 >> 172.21.15.38:0/2422407482 conn(0x7fa1dc0b6b20 msgr2=0x7fa1dc0d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:49.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:49.160+0000 7fa1e1320640 1 -- 172.21.15.38:0/2422407482 shutdown_connections 2024-09-21T03:39:49.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:49.160+0000 7fa1e1320640 1 -- 172.21.15.38:0/2422407482 wait complete. 2024-09-21T03:39:49.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:49.162+0000 7fa1e1320640 1 Processor -- start 2024-09-21T03:39:49.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:49.162+0000 7fa1e1320640 1 -- start start 2024-09-21T03:39:49.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:49.163+0000 7fa1e1320640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa1dc0b2e40 0x7fa1dc15c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:49.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:49.163+0000 7fa1e1320640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa1dc162280 0x7fa1dc1f9590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:49.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:49.163+0000 7fa1e1320640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa1dc15aa20 0x7fa1dc158950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:49.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:49.163+0000 7fa1e1320640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa1dc15f850 con 0x7fa1dc15aa20 2024-09-21T03:39:49.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:49.163+0000 7fa1e1320640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fa1dc15f6d0 con 0x7fa1dc0b2e40 2024-09-21T03:39:49.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:49.163+0000 7fa1e1320640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fa1dc15f9d0 con 0x7fa1dc162280 2024-09-21T03:39:49.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:49.163+0000 7fa1db7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa1dc15aa20 0x7fa1dc158950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:49.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:49.163+0000 7fa1db7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa1dc15aa20 0x7fa1dc158950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42410/0 (socket says 172.21.15.38:42410) 2024-09-21T03:39:49.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:49.163+0000 7fa1db7fe640 1 -- 172.21.15.38:0/1597806579 learned_addr learned my addr 172.21.15.38:0/1597806579 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:39:49.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:49.163+0000 7fa1daffd640 1 --2- 172.21.15.38:0/1597806579 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa1dc0b2e40 0x7fa1dc15c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:49.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:49.163+0000 7fa1da7fc640 1 --2- 172.21.15.38:0/1597806579 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa1dc162280 0x7fa1dc1f9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:49.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:49.163+0000 7fa1db7fe640 1 -- 172.21.15.38:0/1597806579 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa1dc162280 msgr2=0x7fa1dc1f9590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:49.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:49.163+0000 7fa1db7fe640 1 --2- 172.21.15.38:0/1597806579 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa1dc162280 0x7fa1dc1f9590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:49.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:49.163+0000 7fa1db7fe640 1 -- 172.21.15.38:0/1597806579 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa1dc0b2e40 msgr2=0x7fa1dc15c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:49.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:49.163+0000 7fa1db7fe640 1 --2- 172.21.15.38:0/1597806579 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa1dc0b2e40 0x7fa1dc15c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:49.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:49.163+0000 7fa1db7fe640 1 -- 172.21.15.38:0/1597806579 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1dc158e90 con 0x7fa1dc15aa20 2024-09-21T03:39:49.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:49.164+0000 7fa1db7fe640 1 --2- 172.21.15.38:0/1597806579 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa1dc15aa20 0x7fa1dc158950 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7fa1d400c700 tx=0x7fa1d400cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:49.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:49.164+0000 7fa1d37fe640 1 -- 172.21.15.38:0/1597806579 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1d4003b70 con 0x7fa1dc15aa20 2024-09-21T03:39:49.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:49.164+0000 7fa1d37fe640 1 -- 172.21.15.38:0/1597806579 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa1d4003d10 con 0x7fa1dc15aa20 2024-09-21T03:39:49.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:49.164+0000 7fa1d37fe640 1 -- 172.21.15.38:0/1597806579 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1d400aab0 con 0x7fa1dc15aa20 2024-09-21T03:39:49.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:49.164+0000 7fa1e1320640 1 -- 172.21.15.38:0/1597806579 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1dc15b100 con 0x7fa1dc15aa20 2024-09-21T03:39:49.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:49.164+0000 7fa1e1320640 1 -- 172.21.15.38:0/1597806579 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1dc15b740 con 0x7fa1dc15aa20 2024-09-21T03:39:49.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:49.165+0000 7fa1e1320640 1 -- 172.21.15.38:0/1597806579 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1dc15ae70 con 0x7fa1dc15aa20 2024-09-21T03:39:49.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:49.166+0000 7fa1d37fe640 1 -- 172.21.15.38:0/1597806579 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa1d400ac50 con 0x7fa1dc15aa20 2024-09-21T03:39:49.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:49.166+0000 7fa1d37fe640 1 --2- 172.21.15.38:0/1597806579 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa1bc078bc0 0x7fa1bc07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:49.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:49.166+0000 7fa1d37fe640 1 -- 172.21.15.38:0/1597806579 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 6187+0+0 (secure 0 0 0) 0x7fa1d409acb0 con 0x7fa1dc15aa20 2024-09-21T03:39:49.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:49.167+0000 7fa1daffd640 1 --2- 172.21.15.38:0/1597806579 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa1bc078bc0 0x7fa1bc07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:49.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:49.167+0000 7fa1daffd640 1 --2- 172.21.15.38:0/1597806579 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa1bc078bc0 0x7fa1bc07b0d0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fa1c4034000 tx=0x7fa1c4004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:49.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:49.171+0000 7fa1d37fe640 1 -- 172.21.15.38:0/1597806579 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fa1d4062e70 con 0x7fa1dc15aa20 2024-09-21T03:39:49.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:49.358+0000 7fa1e1320640 1 -- 172.21.15.38:0/1597806579 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa1dc162f00 con 0x7fa1dc15aa20 2024-09-21T03:39:49.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:49.359+0000 7fa1d37fe640 1 -- 172.21.15.38:0/1597806579 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+17098 (secure 0 0 0) 0x7fa1d4062770 con 0x7fa1dc15aa20 2024-09-21T03:39:49.361 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:39:49.361 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":83,"fsid":"9d50d66f-656b-4fe2-8e3b-fe53b79c4d43","created":"2024-09-21T03:33:10.677865+0000","modified":"2024-09-21T03:39:42.556893+0000","last_up_change":"2024-09-21T03:33:24.023421+0000","last_in_change":"2024-09-21T03:33:14.296193+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T03:33:22.992573+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-09-21T03:39:02.446050+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-09-21T03:39:03.739922+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dbab183f-7e63-48b8-960a-ae55fd5bf565","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6801","nonce":2381639433}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6803","nonce":2381639433}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6807","nonce":2381639433}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6805","nonce":2381639433}]},"public_addr":"172.21.15.38:6801/2381639433","cluster_addr":"172.21.15.38:6803/2381639433","heartbeat_back_addr":"172.21.15.38:6807/2381639433","heartbeat_front_addr":"172.21.15.38:6805/2381639433","state":["exists","up"]},{"osd":1,"uuid":"fa1c0e8a-ebc1-4359-8146-bd5342abbc16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6817","nonce":1491818732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6819","nonce":1491818732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6823","nonce":1491818732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6821","nonce":1491818732}]},"public_addr":"172.21.15.38:6817/1491818732","cluster_addr":"172.21.15.38:6819/1491818732","heartbeat_back_addr":"172.21.15.38:6823/1491818732","heartbeat_front_addr":"172.21.15.38:6821/1491818732","state":["exists","up"]},{"osd":2,"uuid":"2ed3cd2d-385d-4fc6-9869-214552a84803","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6825","nonce":2577858902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6827","nonce":2577858902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6831","nonce":2577858902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6829","nonce":2577858902}]},"public_addr":"172.21.15.38:6825/2577858902","cluster_addr":"172.21.15.38:6827/2577858902","heartbeat_back_addr":"172.21.15.38:6831/2577858902","heartbeat_front_addr":"172.21.15.38:6829/2577858902","state":["exists","up"]},{"osd":3,"uuid":"62d1d95e-919d-4250-be71-1d34cfa70dbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6809","nonce":1286810294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6811","nonce":1286810294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6815","nonce":1286810294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6813","nonce":1286810294}]},"public_addr":"172.21.15.38:6809/1286810294","cluster_addr":"172.21.15.38:6811/1286810294","heartbeat_back_addr":"172.21.15.38:6815/1286810294","heartbeat_front_addr":"172.21.15.38:6813/1286810294","state":["exists","up"]},{"osd":4,"uuid":"7c9f9314-99d7-4d73-8392-37809a9f54b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6803","nonce":3401837571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6805","nonce":3401837571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6809","nonce":3401837571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6807","nonce":3401837571}]},"public_addr":"172.21.15.195:6803/3401837571","cluster_addr":"172.21.15.195:6805/3401837571","heartbeat_back_addr":"172.21.15.195:6809/3401837571","heartbeat_front_addr":"172.21.15.195:6807/3401837571","state":["exists","up"]},{"osd":5,"uuid":"bde4fe9f-8414-4bba-bf4a-43973d1976bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6811","nonce":2080960180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6813","nonce":2080960180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6817","nonce":2080960180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6815","nonce":2080960180}]},"public_addr":"172.21.15.195:6811/2080960180","cluster_addr":"172.21.15.195:6813/2080960180","heartbeat_back_addr":"172.21.15.195:6817/2080960180","heartbeat_front_addr":"172.21.15.195:6815/2080960180","state":["exists","up"]},{"osd":6,"uuid":"6ee6d7f4-1a5f-4a4c-ac16-48fe05f6a002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6827","nonce":4055825675}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6829","nonce":4055825675}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6833","nonce":4055825675}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6831","nonce":4055825675}]},"public_addr":"172.21.15.195:6827/4055825675","cluster_addr":"172.21.15.195:6829/4055825675","heartbeat_back_addr":"172.21.15.195:6833/4055825675","heartbeat_front_addr":"172.21.15.195:6831/4055825675","state":["exists","up"]},{"osd":7,"uuid":"efeb5f51-329a-4484-8290-92e7d3503d05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6819","nonce":1606106342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6821","nonce":1606106342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6825","nonce":1606106342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6823","nonce":1606106342}]},"public_addr":"172.21.15.195:6819/1606106342","cluster_addr":"172.21.15.195:6821/1606106342","heartbeat_back_addr":"172.21.15.195:6825/1606106342","heartbeat_front_addr":"172.21.15.195:6823/1606106342","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.901343+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:21.115003+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.901623+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.835238+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.132199+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:19.817560+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.829690+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:21.087300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.38:6833/1144075315":"2024-09-22T03:39:42.556819+0000","172.21.15.38:6832/1144075315":"2024-09-22T03:39:42.556819+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T03:39:49.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:49.361+0000 7fa1e1320640 1 -- 172.21.15.38:0/1597806579 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa1bc078bc0 msgr2=0x7fa1bc07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:49.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:49.361+0000 7fa1e1320640 1 --2- 172.21.15.38:0/1597806579 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa1bc078bc0 0x7fa1bc07b0d0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fa1c4034000 tx=0x7fa1c4004730 comp rx=0 tx=0).stop 2024-09-21T03:39:49.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:49.361+0000 7fa1e1320640 1 -- 172.21.15.38:0/1597806579 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa1dc15aa20 msgr2=0x7fa1dc158950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:49.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:49.361+0000 7fa1e1320640 1 --2- 172.21.15.38:0/1597806579 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa1dc15aa20 0x7fa1dc158950 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7fa1d400c700 tx=0x7fa1d400cbc0 comp rx=0 tx=0).stop 2024-09-21T03:39:49.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:49.361+0000 7fa1e1320640 1 -- 172.21.15.38:0/1597806579 shutdown_connections 2024-09-21T03:39:49.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:49.361+0000 7fa1e1320640 1 --2- 172.21.15.38:0/1597806579 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa1dc15aa20 0x7fa1dc158950 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:49.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:49.361+0000 7fa1e1320640 1 --2- 172.21.15.38:0/1597806579 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa1bc078bc0 0x7fa1bc07b0d0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:49.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:49.361+0000 7fa1e1320640 1 --2- 172.21.15.38:0/1597806579 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa1dc162280 0x7fa1dc1f9590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:49.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:49.361+0000 7fa1e1320640 1 --2- 172.21.15.38:0/1597806579 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa1dc0b2e40 0x7fa1dc15c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:49.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:49.361+0000 7fa1e1320640 1 -- 172.21.15.38:0/1597806579 >> 172.21.15.38:0/1597806579 conn(0x7fa1dc0b6b20 msgr2=0x7fa1dc0b3250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:49.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:49.361+0000 7fa1e1320640 1 -- 172.21.15.38:0/1597806579 shutdown_connections 2024-09-21T03:39:49.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:49.361+0000 7fa1e1320640 1 -- 172.21.15.38:0/1597806579 wait complete. 2024-09-21T03:39:49.374 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-21T03:39:49.374 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:39:49.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:49.489+0000 7f965f640640 1 -- 172.21.15.38:0/220189274 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9644004d40 msgr2=0x7f9644005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:49.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:49.489+0000 7f965f640640 1 --2- 172.21.15.38:0/220189274 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9644004d40 0x7f9644005150 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f9650005fd0 tx=0x7f9644005830 comp rx=0 tx=0).stop 2024-09-21T03:39:49.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:49.489+0000 7f965dbdd640 1 -- 172.21.15.38:0/220189274 reap_dead start 2024-09-21T03:39:49.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:49.489+0000 7f965f640640 1 -- 172.21.15.38:0/220189274 shutdown_connections 2024-09-21T03:39:49.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:49.489+0000 7f965f640640 1 -- 172.21.15.38:0/220189274 >> 172.21.15.38:0/220189274 conn(0x7f96580b6b20 msgr2=0x7f96580d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:49.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:49.490+0000 7f965f640640 1 -- 172.21.15.38:0/220189274 shutdown_connections 2024-09-21T03:39:49.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:49.490+0000 7f965f640640 1 -- 172.21.15.38:0/220189274 wait complete. 2024-09-21T03:39:49.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:49.491+0000 7f965f640640 1 Processor -- start 2024-09-21T03:39:49.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:49.491+0000 7f965f640640 1 -- start start 2024-09-21T03:39:49.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:49.493+0000 7f965f640640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9644004d40 0x7f96580ca680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:49.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:49.493+0000 7f965f640640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f965815ac10 0x7f96580c8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:49.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:49.493+0000 7f965f640640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f96580c94f0 0x7f96580c9900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:49.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:49.493+0000 7f965f640640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f96580b35e0 con 0x7f965815ac10 2024-09-21T03:39:49.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:49.493+0000 7f965f640640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f96580b3460 con 0x7f9644004d40 2024-09-21T03:39:49.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:49.493+0000 7f965f640640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f96580b3760 con 0x7f96580c94f0 2024-09-21T03:39:49.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:49.493+0000 7f965cbdb640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f965815ac10 0x7f96580c8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:49.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:49.493+0000 7f965cbdb640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f965815ac10 0x7f96580c8fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42426/0 (socket says 172.21.15.38:42426) 2024-09-21T03:39:49.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:49.493+0000 7f965cbdb640 1 -- 172.21.15.38:0/3894353820 learned_addr learned my addr 172.21.15.38:0/3894353820 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:39:49.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:49.493+0000 7f965d3dc640 1 --2- 172.21.15.38:0/3894353820 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9644004d40 0x7f96580ca680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:49.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:49.493+0000 7f965dbdd640 1 --2- 172.21.15.38:0/3894353820 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f96580c94f0 0x7f96580c9900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:49.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:49.493+0000 7f965cbdb640 1 -- 172.21.15.38:0/3894353820 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f96580c94f0 msgr2=0x7f96580c9900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:49.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:49.493+0000 7f965cbdb640 1 --2- 172.21.15.38:0/3894353820 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f96580c94f0 0x7f96580c9900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:49.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:49.493+0000 7f965cbdb640 1 -- 172.21.15.38:0/3894353820 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9644004d40 msgr2=0x7f96580ca680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:49.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:49.493+0000 7f965cbdb640 1 --2- 172.21.15.38:0/3894353820 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9644004d40 0x7f96580ca680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:49.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:49.493+0000 7f965cbdb640 1 -- 172.21.15.38:0/3894353820 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f965814f0b0 con 0x7f965815ac10 2024-09-21T03:39:49.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:49.493+0000 7f965cbdb640 1 --2- 172.21.15.38:0/3894353820 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f965815ac10 0x7f96580c8fb0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f9648011ae0 tx=0x7f9648009f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:49.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:49.493+0000 7f964dffb640 1 -- 172.21.15.38:0/3894353820 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f964800fc50 con 0x7f965815ac10 2024-09-21T03:39:49.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:49.494+0000 7f964dffb640 1 -- 172.21.15.38:0/3894353820 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f964800fdf0 con 0x7f965815ac10 2024-09-21T03:39:49.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:49.494+0000 7f965f640640 1 -- 172.21.15.38:0/3894353820 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f965814f370 con 0x7f965815ac10 2024-09-21T03:39:49.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:49.494+0000 7f964dffb640 1 -- 172.21.15.38:0/3894353820 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9648004280 con 0x7f965815ac10 2024-09-21T03:39:49.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:49.494+0000 7f965f640640 1 -- 172.21.15.38:0/3894353820 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f965814f930 con 0x7f965815ac10 2024-09-21T03:39:49.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:49.495+0000 7f965f640640 1 -- 172.21.15.38:0/3894353820 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9658162f00 con 0x7f965815ac10 2024-09-21T03:39:49.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:49.496+0000 7f964dffb640 1 -- 172.21.15.38:0/3894353820 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f96480035f0 con 0x7f965815ac10 2024-09-21T03:39:49.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:49.496+0000 7f964dffb640 1 --2- 172.21.15.38:0/3894353820 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f963c078bc0 0x7f963c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:49.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:49.496+0000 7f964dffb640 1 -- 172.21.15.38:0/3894353820 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 6187+0+0 (secure 0 0 0) 0x7f964809b870 con 0x7f965815ac10 2024-09-21T03:39:49.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:49.496+0000 7f965d3dc640 1 --2- 172.21.15.38:0/3894353820 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f963c078bc0 0x7f963c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:49.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:49.497+0000 7f965d3dc640 1 --2- 172.21.15.38:0/3894353820 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f963c078bc0 0x7f963c07b0d0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f9654000fb0 tx=0x7f9654006950 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:49.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:49.501+0000 7f964dffb640 1 -- 172.21.15.38:0/3894353820 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f9648063510 con 0x7f965815ac10 2024-09-21T03:39:49.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:49.728+0000 7f965f640640 1 -- 172.21.15.38:0/3894353820 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f965815dfd0 con 0x7f965815ac10 2024-09-21T03:39:49.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:49.728+0000 7f964dffb640 1 -- 172.21.15.38:0/3894353820 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) ==== 94+0+5233 (secure 0 0 0) 0x7f9648067f40 con 0x7f965815ac10 2024-09-21T03:39:49.730 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:39:49.731 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":74,"btime":"2024-09-21T03:39:45:574648+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5783,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/1358588311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1358588311},{"type":"v1","addr":"172.21.15.38:6835","nonce":1358588311}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63},{"gid":5789,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/1676483003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":1676483003},{"type":"v1","addr":"172.21.15.195:6837","nonce":1676483003}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63},{"gid":6813,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/3644074938","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":3644074938},{"type":"v1","addr":"172.21.15.38:6833","nonce":3644074938}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":72}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:39:04.779948+0000","modified":"2024-09-21T03:39:45.574643+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6193},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6193":{"gid":6193,"name":"d","rank":0,"incarnation":71,"state":"up:active","state_seq":14,"addr":"172.21.15.195:6836/2771781100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":2771781100},{"type":"v1","addr":"172.21.15.195:6836","nonce":2771781100}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6193,"qdb_cluster":[6193]},"id":5}]} 2024-09-21T03:39:49.731 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 74 2024-09-21T03:39:49.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:49.731+0000 7f965f640640 1 -- 172.21.15.38:0/3894353820 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f963c078bc0 msgr2=0x7f963c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:49.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:49.731+0000 7f965f640640 1 --2- 172.21.15.38:0/3894353820 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f963c078bc0 0x7f963c07b0d0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f9654000fb0 tx=0x7f9654006950 comp rx=0 tx=0).stop 2024-09-21T03:39:49.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:49.731+0000 7f965f640640 1 -- 172.21.15.38:0/3894353820 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f965815ac10 msgr2=0x7f96580c8fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:49.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:49.731+0000 7f965f640640 1 --2- 172.21.15.38:0/3894353820 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f965815ac10 0x7f96580c8fb0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f9648011ae0 tx=0x7f9648009f10 comp rx=0 tx=0).stop 2024-09-21T03:39:49.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:49.731+0000 7f965f640640 1 -- 172.21.15.38:0/3894353820 shutdown_connections 2024-09-21T03:39:49.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:49.731+0000 7f965f640640 1 --2- 172.21.15.38:0/3894353820 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f963c078bc0 0x7f963c07b0d0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:49.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:49.731+0000 7f965f640640 1 --2- 172.21.15.38:0/3894353820 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f96580c94f0 0x7f96580c9900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:49.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:49.731+0000 7f965f640640 1 --2- 172.21.15.38:0/3894353820 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f965815ac10 0x7f96580c8fb0 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:49.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:49.731+0000 7f965f640640 1 --2- 172.21.15.38:0/3894353820 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9644004d40 0x7f96580ca680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:49.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:49.731+0000 7f965f640640 1 -- 172.21.15.38:0/3894353820 >> 172.21.15.38:0/3894353820 conn(0x7f96580b6b20 msgr2=0x7f96580d0c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:49.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:49.731+0000 7f965f640640 1 -- 172.21.15.38:0/3894353820 shutdown_connections 2024-09-21T03:39:49.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:49.731+0000 7f965f640640 1 -- 172.21.15.38:0/3894353820 wait complete. 2024-09-21T03:39:49.744 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:39:49.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:49.859+0000 7f4ee99ad640 1 -- 172.21.15.38:0/3387047795 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4ee4162f00 msgr2=0x7f4ee40d24d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:49.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:49.859+0000 7f4ee99ad640 1 --2- 172.21.15.38:0/3387047795 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4ee4162f00 0x7f4ee40d24d0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f4edc009830 tx=0x7f4edc02a510 comp rx=0 tx=0).stop 2024-09-21T03:39:49.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:49.859+0000 7f4ee99ad640 1 -- 172.21.15.38:0/3387047795 shutdown_connections 2024-09-21T03:39:49.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:49.859+0000 7f4ee99ad640 1 --2- 172.21.15.38:0/3387047795 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4ee4162280 0x7f4ee4162690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:49.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:49.859+0000 7f4ee99ad640 1 --2- 172.21.15.38:0/3387047795 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4ee4162f00 0x7f4ee40d24d0 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:49.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:49.859+0000 7f4ee99ad640 1 -- 172.21.15.38:0/3387047795 >> 172.21.15.38:0/3387047795 conn(0x7f4ee40b6b20 msgr2=0x7f4ee40d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:49.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:49.859+0000 7f4ee99ad640 1 -- 172.21.15.38:0/3387047795 shutdown_connections 2024-09-21T03:39:49.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:49.859+0000 7f4ee99ad640 1 -- 172.21.15.38:0/3387047795 wait complete. 2024-09-21T03:39:49.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:49.860+0000 7f4ee99ad640 1 Processor -- start 2024-09-21T03:39:49.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:49.861+0000 7f4ee99ad640 1 -- start start 2024-09-21T03:39:49.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:49.862+0000 7f4ee99ad640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4ee40b2e40 0x7f4ee41f9390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:49.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:49.862+0000 7f4ee99ad640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4ee4162280 0x7f4ee40ad410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:49.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:49.862+0000 7f4ee99ad640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4ee415b480 0x7f4ee41f63c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:49.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:49.862+0000 7f4ee99ad640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f4ee415e630 con 0x7f4ee40b2e40 2024-09-21T03:39:49.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:49.862+0000 7f4ee99ad640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f4ee415e4b0 con 0x7f4ee415b480 2024-09-21T03:39:49.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:49.862+0000 7f4ee99ad640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f4ee415e7b0 con 0x7f4ee4162280 2024-09-21T03:39:49.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:49.862+0000 7f4ee2ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4ee40b2e40 0x7f4ee41f9390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:49.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:49.862+0000 7f4ee2ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4ee40b2e40 0x7f4ee41f9390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42444/0 (socket says 172.21.15.38:42444) 2024-09-21T03:39:49.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:49.862+0000 7f4ee2ffd640 1 -- 172.21.15.38:0/2499962791 learned_addr learned my addr 172.21.15.38:0/2499962791 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:39:49.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:49.862+0000 7f4ee27fc640 1 --2- 172.21.15.38:0/2499962791 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4ee4162280 0x7f4ee40ad410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:49.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:49.862+0000 7f4ee37fe640 1 --2- 172.21.15.38:0/2499962791 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4ee415b480 0x7f4ee41f63c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:49.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:49.862+0000 7f4ee2ffd640 1 -- 172.21.15.38:0/2499962791 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4ee4162280 msgr2=0x7f4ee40ad410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:49.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:49.862+0000 7f4ee2ffd640 1 --2- 172.21.15.38:0/2499962791 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4ee4162280 0x7f4ee40ad410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:49.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:49.862+0000 7f4ee2ffd640 1 -- 172.21.15.38:0/2499962791 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4ee415b480 msgr2=0x7f4ee41f63c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:49.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:49.862+0000 7f4ee2ffd640 1 --2- 172.21.15.38:0/2499962791 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4ee415b480 0x7f4ee41f63c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:49.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:49.862+0000 7f4ee2ffd640 1 -- 172.21.15.38:0/2499962791 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ee4159350 con 0x7f4ee40b2e40 2024-09-21T03:39:49.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:49.862+0000 7f4ee27fc640 1 --2- 172.21.15.38:0/2499962791 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4ee4162280 0x7f4ee40ad410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:39:49.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:49.862+0000 7f4ee37fe640 1 --2- 172.21.15.38:0/2499962791 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4ee415b480 0x7f4ee41f63c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:39:49.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:49.863+0000 7f4ee2ffd640 1 --2- 172.21.15.38:0/2499962791 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4ee40b2e40 0x7f4ee41f9390 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f4edc009950 tx=0x7f4edc033c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:49.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:49.863+0000 7f4edbfff640 1 -- 172.21.15.38:0/2499962791 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4edc005c90 con 0x7f4ee40b2e40 2024-09-21T03:39:49.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:49.863+0000 7f4edbfff640 1 -- 172.21.15.38:0/2499962791 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4edc031e50 con 0x7f4ee40b2e40 2024-09-21T03:39:49.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:49.863+0000 7f4ee99ad640 1 -- 172.21.15.38:0/2499962791 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ee4159630 con 0x7f4ee40b2e40 2024-09-21T03:39:49.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:49.863+0000 7f4edbfff640 1 -- 172.21.15.38:0/2499962791 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4edc0332f0 con 0x7f4ee40b2e40 2024-09-21T03:39:49.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:49.863+0000 7f4ee99ad640 1 -- 172.21.15.38:0/2499962791 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ee4159bf0 con 0x7f4ee40b2e40 2024-09-21T03:39:49.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:49.864+0000 7f4ee99ad640 1 -- 172.21.15.38:0/2499962791 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ee4162f00 con 0x7f4ee40b2e40 2024-09-21T03:39:49.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:49.865+0000 7f4edbfff640 1 -- 172.21.15.38:0/2499962791 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f4edc033490 con 0x7f4ee40b2e40 2024-09-21T03:39:49.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:49.865+0000 7f4edbfff640 1 --2- 172.21.15.38:0/2499962791 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4ec4078b90 0x7f4ec407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:49.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:49.865+0000 7f4edbfff640 1 -- 172.21.15.38:0/2499962791 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 6187+0+0 (secure 0 0 0) 0x7f4edc0bc8e0 con 0x7f4ee40b2e40 2024-09-21T03:39:49.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:49.866+0000 7f4ee27fc640 1 --2- 172.21.15.38:0/2499962791 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4ec4078b90 0x7f4ec407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:49.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:49.866+0000 7f4ee27fc640 1 --2- 172.21.15.38:0/2499962791 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4ec4078b90 0x7f4ec407b0a0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f4ee40b5230 tx=0x7f4ecc005f10 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:49.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:49.869+0000 7f4edbfff640 1 -- 172.21.15.38:0/2499962791 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f4edc051020 con 0x7f4ee40b2e40 2024-09-21T03:39:50.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:50.098+0000 7f4ee99ad640 1 -- 172.21.15.38:0/2499962791 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4ee415de10 con 0x7f4ee40b2e40 2024-09-21T03:39:50.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:50.099+0000 7f4edbfff640 1 -- 172.21.15.38:0/2499962791 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) ==== 94+0+5233 (secure 0 0 0) 0x7f4edc03f030 con 0x7f4ee40b2e40 2024-09-21T03:39:50.101 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:39:50.101 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":74,"btime":"2024-09-21T03:39:45:574648+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5783,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/1358588311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1358588311},{"type":"v1","addr":"172.21.15.38:6835","nonce":1358588311}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63},{"gid":5789,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/1676483003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":1676483003},{"type":"v1","addr":"172.21.15.195:6837","nonce":1676483003}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63},{"gid":6813,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/3644074938","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":3644074938},{"type":"v1","addr":"172.21.15.38:6833","nonce":3644074938}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":72}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:39:04.779948+0000","modified":"2024-09-21T03:39:45.574643+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6193},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6193":{"gid":6193,"name":"d","rank":0,"incarnation":71,"state":"up:active","state_seq":14,"addr":"172.21.15.195:6836/2771781100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":2771781100},{"type":"v1","addr":"172.21.15.195:6836","nonce":2771781100}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6193,"qdb_cluster":[6193]},"id":5}]} 2024-09-21T03:39:50.101 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 74 2024-09-21T03:39:50.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:50.101+0000 7f4ee99ad640 1 -- 172.21.15.38:0/2499962791 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4ec4078b90 msgr2=0x7f4ec407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:50.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:50.101+0000 7f4ee99ad640 1 --2- 172.21.15.38:0/2499962791 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4ec4078b90 0x7f4ec407b0a0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f4ee40b5230 tx=0x7f4ecc005f10 comp rx=0 tx=0).stop 2024-09-21T03:39:50.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:50.101+0000 7f4ee99ad640 1 -- 172.21.15.38:0/2499962791 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4ee40b2e40 msgr2=0x7f4ee41f9390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:50.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:50.101+0000 7f4ee99ad640 1 --2- 172.21.15.38:0/2499962791 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4ee40b2e40 0x7f4ee41f9390 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f4edc009950 tx=0x7f4edc033c90 comp rx=0 tx=0).stop 2024-09-21T03:39:50.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:50.101+0000 7f4ee99ad640 1 -- 172.21.15.38:0/2499962791 shutdown_connections 2024-09-21T03:39:50.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:50.101+0000 7f4ee99ad640 1 --2- 172.21.15.38:0/2499962791 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4ee415b480 0x7f4ee41f63c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:50.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:50.101+0000 7f4ee99ad640 1 --2- 172.21.15.38:0/2499962791 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4ec4078b90 0x7f4ec407b0a0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:50.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:50.101+0000 7f4ee99ad640 1 --2- 172.21.15.38:0/2499962791 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4ee4162280 0x7f4ee40ad410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:50.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:50.101+0000 7f4ee99ad640 1 --2- 172.21.15.38:0/2499962791 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4ee40b2e40 0x7f4ee41f9390 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:50.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:50.101+0000 7f4ee99ad640 1 -- 172.21.15.38:0/2499962791 >> 172.21.15.38:0/2499962791 conn(0x7f4ee40b6b20 msgr2=0x7f4ee40b7a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:50.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:50.101+0000 7f4ee99ad640 1 -- 172.21.15.38:0/2499962791 shutdown_connections 2024-09-21T03:39:50.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:50.101+0000 7f4ee99ad640 1 -- 172.21.15.38:0/2499962791 wait complete. 2024-09-21T03:39:50.115 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T03:39:50.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:50.230+0000 7fb369407640 1 -- 172.21.15.38:0/1788863310 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb34c004d40 msgr2=0x7fb34c005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:50.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:50.230+0000 7fb369407640 1 --2- 172.21.15.38:0/1788863310 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb34c004d40 0x7fb34c005150 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fb35c009ed0 tx=0x7fb34c005830 comp rx=0 tx=0).stop 2024-09-21T03:39:50.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:50.230+0000 7fb3637fe640 1 -- 172.21.15.38:0/1788863310 reap_dead start 2024-09-21T03:39:50.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:50.230+0000 7fb369407640 1 -- 172.21.15.38:0/1788863310 shutdown_connections 2024-09-21T03:39:50.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:50.230+0000 7fb369407640 1 -- 172.21.15.38:0/1788863310 >> 172.21.15.38:0/1788863310 conn(0x7fb3640d4440 msgr2=0x7fb3640d4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:50.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:50.230+0000 7fb369407640 1 -- 172.21.15.38:0/1788863310 shutdown_connections 2024-09-21T03:39:50.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:50.230+0000 7fb369407640 1 -- 172.21.15.38:0/1788863310 wait complete. 2024-09-21T03:39:50.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:50.232+0000 7fb369407640 1 Processor -- start 2024-09-21T03:39:50.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:50.232+0000 7fb369407640 1 -- start start 2024-09-21T03:39:50.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:50.233+0000 7fb369407640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb34c004d40 0x7fb36415c230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:50.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:50.234+0000 7fb369407640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb3641f58c0 0x7fb3641f5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:50.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:50.234+0000 7fb369407640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb3641f8db0 0x7fb3641f91c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:50.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:50.234+0000 7fb369407640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fb3640b33f0 con 0x7fb3641f8db0 2024-09-21T03:39:50.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:50.234+0000 7fb369407640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fb3640b3270 con 0x7fb34c004d40 2024-09-21T03:39:50.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:50.234+0000 7fb369407640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fb3640b3570 con 0x7fb3641f58c0 2024-09-21T03:39:50.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:50.234+0000 7fb3637fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb3641f8db0 0x7fb3641f91c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:50.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:50.234+0000 7fb3637fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb3641f8db0 0x7fb3641f91c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42456/0 (socket says 172.21.15.38:42456) 2024-09-21T03:39:50.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:50.234+0000 7fb3637fe640 1 -- 172.21.15.38:0/342245296 learned_addr learned my addr 172.21.15.38:0/342245296 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:39:50.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:50.234+0000 7fb3627fc640 1 --2- 172.21.15.38:0/342245296 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb3641f58c0 0x7fb3641f5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:50.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:50.234+0000 7fb362ffd640 1 --2- 172.21.15.38:0/342245296 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb34c004d40 0x7fb36415c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:50.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:50.234+0000 7fb3637fe640 1 -- 172.21.15.38:0/342245296 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb3641f58c0 msgr2=0x7fb3641f5cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:50.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:50.234+0000 7fb3637fe640 1 --2- 172.21.15.38:0/342245296 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb3641f58c0 0x7fb3641f5cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:50.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:50.234+0000 7fb3637fe640 1 -- 172.21.15.38:0/342245296 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb34c004d40 msgr2=0x7fb36415c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:50.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:50.234+0000 7fb3637fe640 1 --2- 172.21.15.38:0/342245296 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb34c004d40 0x7fb36415c230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:50.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:50.234+0000 7fb3637fe640 1 -- 172.21.15.38:0/342245296 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3641f9700 con 0x7fb3641f8db0 2024-09-21T03:39:50.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:50.234+0000 7fb3637fe640 1 --2- 172.21.15.38:0/342245296 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb3641f8db0 0x7fb3641f91c0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fb35c02b870 tx=0x7fb35c033bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:50.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:50.234+0000 7fb34b7fe640 1 -- 172.21.15.38:0/342245296 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb35c042070 con 0x7fb3641f8db0 2024-09-21T03:39:50.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:50.234+0000 7fb34b7fe640 1 -- 172.21.15.38:0/342245296 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb35c003970 con 0x7fb3641f8db0 2024-09-21T03:39:50.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:50.235+0000 7fb369407640 1 -- 172.21.15.38:0/342245296 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3641f6680 con 0x7fb3641f8db0 2024-09-21T03:39:50.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:50.235+0000 7fb34b7fe640 1 -- 172.21.15.38:0/342245296 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb35c038040 con 0x7fb3641f8db0 2024-09-21T03:39:50.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:50.235+0000 7fb369407640 1 -- 172.21.15.38:0/342245296 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb36415dde0 con 0x7fb3641f8db0 2024-09-21T03:39:50.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:50.236+0000 7fb369407640 1 -- 172.21.15.38:0/342245296 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3641f63f0 con 0x7fb3641f8db0 2024-09-21T03:39:50.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:50.236+0000 7fb34b7fe640 1 -- 172.21.15.38:0/342245296 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb35c004030 con 0x7fb3641f8db0 2024-09-21T03:39:50.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:50.237+0000 7fb34b7fe640 1 --2- 172.21.15.38:0/342245296 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb350078b90 0x7fb35007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:50.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:50.237+0000 7fb34b7fe640 1 -- 172.21.15.38:0/342245296 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 6187+0+0 (secure 0 0 0) 0x7fb35c0bb350 con 0x7fb3641f8db0 2024-09-21T03:39:50.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:50.237+0000 7fb362ffd640 1 --2- 172.21.15.38:0/342245296 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb350078b90 0x7fb35007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:50.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:50.238+0000 7fb362ffd640 1 --2- 172.21.15.38:0/342245296 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb350078b90 0x7fb35007b0a0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fb354009cd0 tx=0x7fb354005f90 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:50.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:50.242+0000 7fb34b7fe640 1 -- 172.21.15.38:0/342245296 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fb3641f9700 con 0x7fb3641f8db0 2024-09-21T03:39:50.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:50.433+0000 7fb369407640 1 -- 172.21.15.38:0/342245296 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb364162380 con 0x7fb3641f8db0 2024-09-21T03:39:50.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:50.434+0000 7fb34b7fe640 1 -- 172.21.15.38:0/342245296 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+17098 (secure 0 0 0) 0x7fb364162380 con 0x7fb3641f8db0 2024-09-21T03:39:50.436 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:39:50.437 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":83,"fsid":"9d50d66f-656b-4fe2-8e3b-fe53b79c4d43","created":"2024-09-21T03:33:10.677865+0000","modified":"2024-09-21T03:39:42.556893+0000","last_up_change":"2024-09-21T03:33:24.023421+0000","last_in_change":"2024-09-21T03:33:14.296193+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T03:33:22.992573+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-09-21T03:39:02.446050+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-09-21T03:39:03.739922+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dbab183f-7e63-48b8-960a-ae55fd5bf565","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6801","nonce":2381639433}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6803","nonce":2381639433}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6807","nonce":2381639433}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6805","nonce":2381639433}]},"public_addr":"172.21.15.38:6801/2381639433","cluster_addr":"172.21.15.38:6803/2381639433","heartbeat_back_addr":"172.21.15.38:6807/2381639433","heartbeat_front_addr":"172.21.15.38:6805/2381639433","state":["exists","up"]},{"osd":1,"uuid":"fa1c0e8a-ebc1-4359-8146-bd5342abbc16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6817","nonce":1491818732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6819","nonce":1491818732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6823","nonce":1491818732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6821","nonce":1491818732}]},"public_addr":"172.21.15.38:6817/1491818732","cluster_addr":"172.21.15.38:6819/1491818732","heartbeat_back_addr":"172.21.15.38:6823/1491818732","heartbeat_front_addr":"172.21.15.38:6821/1491818732","state":["exists","up"]},{"osd":2,"uuid":"2ed3cd2d-385d-4fc6-9869-214552a84803","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6825","nonce":2577858902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6827","nonce":2577858902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6831","nonce":2577858902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6829","nonce":2577858902}]},"public_addr":"172.21.15.38:6825/2577858902","cluster_addr":"172.21.15.38:6827/2577858902","heartbeat_back_addr":"172.21.15.38:6831/2577858902","heartbeat_front_addr":"172.21.15.38:6829/2577858902","state":["exists","up"]},{"osd":3,"uuid":"62d1d95e-919d-4250-be71-1d34cfa70dbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6809","nonce":1286810294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6811","nonce":1286810294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6815","nonce":1286810294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6813","nonce":1286810294}]},"public_addr":"172.21.15.38:6809/1286810294","cluster_addr":"172.21.15.38:6811/1286810294","heartbeat_back_addr":"172.21.15.38:6815/1286810294","heartbeat_front_addr":"172.21.15.38:6813/1286810294","state":["exists","up"]},{"osd":4,"uuid":"7c9f9314-99d7-4d73-8392-37809a9f54b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6803","nonce":3401837571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6805","nonce":3401837571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6809","nonce":3401837571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6807","nonce":3401837571}]},"public_addr":"172.21.15.195:6803/3401837571","cluster_addr":"172.21.15.195:6805/3401837571","heartbeat_back_addr":"172.21.15.195:6809/3401837571","heartbeat_front_addr":"172.21.15.195:6807/3401837571","state":["exists","up"]},{"osd":5,"uuid":"bde4fe9f-8414-4bba-bf4a-43973d1976bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6811","nonce":2080960180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6813","nonce":2080960180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6817","nonce":2080960180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6815","nonce":2080960180}]},"public_addr":"172.21.15.195:6811/2080960180","cluster_addr":"172.21.15.195:6813/2080960180","heartbeat_back_addr":"172.21.15.195:6817/2080960180","heartbeat_front_addr":"172.21.15.195:6815/2080960180","state":["exists","up"]},{"osd":6,"uuid":"6ee6d7f4-1a5f-4a4c-ac16-48fe05f6a002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6827","nonce":4055825675}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6829","nonce":4055825675}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6833","nonce":4055825675}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6831","nonce":4055825675}]},"public_addr":"172.21.15.195:6827/4055825675","cluster_addr":"172.21.15.195:6829/4055825675","heartbeat_back_addr":"172.21.15.195:6833/4055825675","heartbeat_front_addr":"172.21.15.195:6831/4055825675","state":["exists","up"]},{"osd":7,"uuid":"efeb5f51-329a-4484-8290-92e7d3503d05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6819","nonce":1606106342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6821","nonce":1606106342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6825","nonce":1606106342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6823","nonce":1606106342}]},"public_addr":"172.21.15.195:6819/1606106342","cluster_addr":"172.21.15.195:6821/1606106342","heartbeat_back_addr":"172.21.15.195:6825/1606106342","heartbeat_front_addr":"172.21.15.195:6823/1606106342","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.901343+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:21.115003+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.901623+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.835238+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.132199+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:19.817560+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.829690+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:21.087300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.38:6833/1144075315":"2024-09-22T03:39:42.556819+0000","172.21.15.38:6832/1144075315":"2024-09-22T03:39:42.556819+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T03:39:50.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:50.437+0000 7fb369407640 1 -- 172.21.15.38:0/342245296 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb350078b90 msgr2=0x7fb35007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:50.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:50.437+0000 7fb369407640 1 --2- 172.21.15.38:0/342245296 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb350078b90 0x7fb35007b0a0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fb354009cd0 tx=0x7fb354005f90 comp rx=0 tx=0).stop 2024-09-21T03:39:50.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:50.437+0000 7fb369407640 1 -- 172.21.15.38:0/342245296 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb3641f8db0 msgr2=0x7fb3641f91c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:50.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:50.437+0000 7fb369407640 1 --2- 172.21.15.38:0/342245296 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb3641f8db0 0x7fb3641f91c0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fb35c02b870 tx=0x7fb35c033bf0 comp rx=0 tx=0).stop 2024-09-21T03:39:50.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:50.437+0000 7fb369407640 1 -- 172.21.15.38:0/342245296 shutdown_connections 2024-09-21T03:39:50.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:50.437+0000 7fb369407640 1 --2- 172.21.15.38:0/342245296 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb3641f8db0 0x7fb3641f91c0 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:50.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:50.437+0000 7fb369407640 1 --2- 172.21.15.38:0/342245296 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb350078b90 0x7fb35007b0a0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:50.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:50.437+0000 7fb369407640 1 --2- 172.21.15.38:0/342245296 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb3641f58c0 0x7fb3641f5cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:50.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:50.437+0000 7fb369407640 1 --2- 172.21.15.38:0/342245296 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb34c004d40 0x7fb36415c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:50.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:50.437+0000 7fb369407640 1 -- 172.21.15.38:0/342245296 >> 172.21.15.38:0/342245296 conn(0x7fb3640d4440 msgr2=0x7fb36406d020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:50.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:50.437+0000 7fb369407640 1 -- 172.21.15.38:0/342245296 shutdown_connections 2024-09-21T03:39:50.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:50.437+0000 7fb369407640 1 -- 172.21.15.38:0/342245296 wait complete. 2024-09-21T03:39:50.451 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-21T03:39:50.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:50.567+0000 7f8f09215640 1 -- 172.21.15.38:0/835100297 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8ef0004800 msgr2=0x7f8ef0004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:50.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:50.567+0000 7f8f09215640 1 --2- 172.21.15.38:0/835100297 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8ef0004800 0x7f8ef0004c10 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f8f040b5300 tx=0x7f8ef0005830 comp rx=0 tx=0).stop 2024-09-21T03:39:50.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:50.567+0000 7f8f0359e640 1 -- 172.21.15.38:0/835100297 reap_dead start 2024-09-21T03:39:50.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:50.567+0000 7f8f09215640 1 -- 172.21.15.38:0/835100297 shutdown_connections 2024-09-21T03:39:50.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:50.567+0000 7f8f09215640 1 -- 172.21.15.38:0/835100297 >> 172.21.15.38:0/835100297 conn(0x7f8f040b46e0 msgr2=0x7f8f040d23e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:50.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:50.567+0000 7f8f09215640 1 -- 172.21.15.38:0/835100297 shutdown_connections 2024-09-21T03:39:50.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:50.567+0000 7f8f09215640 1 -- 172.21.15.38:0/835100297 wait complete. 2024-09-21T03:39:50.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:50.569+0000 7f8f09215640 1 Processor -- start 2024-09-21T03:39:50.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:50.569+0000 7f8f09215640 1 -- start start 2024-09-21T03:39:50.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:50.570+0000 7f8f09215640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8ef0004800 0x7f8f04153910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:50.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:50.570+0000 7f8f09215640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8f04151940 0x7f8f041f63a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:50.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:50.570+0000 7f8f09215640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8f04150970 0x7f8f041f68e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:50.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:50.570+0000 7f8f09215640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f8f04165540 con 0x7f8f04151940 2024-09-21T03:39:50.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:50.570+0000 7f8f09215640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f8f040b6e00 con 0x7f8ef0004800 2024-09-21T03:39:50.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:50.570+0000 7f8f09215640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f8f041656c0 con 0x7f8f04150970 2024-09-21T03:39:50.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:50.570+0000 7f8f0259c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8f04151940 0x7f8f041f63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:50.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:50.570+0000 7f8f0259c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8f04151940 0x7f8f041f63a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42476/0 (socket says 172.21.15.38:42476) 2024-09-21T03:39:50.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:50.570+0000 7f8f0259c640 1 -- 172.21.15.38:0/3057586613 learned_addr learned my addr 172.21.15.38:0/3057586613 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:39:50.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:50.570+0000 7f8f02d9d640 1 --2- 172.21.15.38:0/3057586613 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8ef0004800 0x7f8f04153910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:50.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:50.570+0000 7f8f0359e640 1 --2- 172.21.15.38:0/3057586613 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8f04150970 0x7f8f041f68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:50.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:50.571+0000 7f8f0259c640 1 -- 172.21.15.38:0/3057586613 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8f04150970 msgr2=0x7f8f041f68e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:50.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:50.571+0000 7f8f0259c640 1 --2- 172.21.15.38:0/3057586613 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8f04150970 0x7f8f041f68e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:50.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:50.571+0000 7f8f0259c640 1 -- 172.21.15.38:0/3057586613 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8ef0004800 msgr2=0x7f8f04153910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:50.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:50.571+0000 7f8f0259c640 1 --2- 172.21.15.38:0/3057586613 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8ef0004800 0x7f8f04153910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:50.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:50.571+0000 7f8f0259c640 1 -- 172.21.15.38:0/3057586613 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f0415eb10 con 0x7f8f04151940 2024-09-21T03:39:50.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:50.571+0000 7f8f02d9d640 1 --2- 172.21.15.38:0/3057586613 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8ef0004800 0x7f8f04153910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:39:50.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:50.571+0000 7f8f0359e640 1 --2- 172.21.15.38:0/3057586613 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8f04150970 0x7f8f041f68e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:39:50.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:50.571+0000 7f8f0259c640 1 --2- 172.21.15.38:0/3057586613 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8f04151940 0x7f8f041f63a0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f8eec00c320 tx=0x7f8eec001220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:50.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:50.571+0000 7f8efb7fe640 1 -- 172.21.15.38:0/3057586613 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eec003810 con 0x7f8f04151940 2024-09-21T03:39:50.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:50.571+0000 7f8efb7fe640 1 -- 172.21.15.38:0/3057586613 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8eec003aa0 con 0x7f8f04151940 2024-09-21T03:39:50.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:50.571+0000 7f8f09215640 1 -- 172.21.15.38:0/3057586613 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f04154490 con 0x7f8f04151940 2024-09-21T03:39:50.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:50.571+0000 7f8efb7fe640 1 -- 172.21.15.38:0/3057586613 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eec02e480 con 0x7f8f04151940 2024-09-21T03:39:50.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:50.571+0000 7f8f09215640 1 -- 172.21.15.38:0/3057586613 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f041f7250 con 0x7f8f04151940 2024-09-21T03:39:50.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:50.572+0000 7f8f09215640 1 -- 172.21.15.38:0/3057586613 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f04156710 con 0x7f8f04151940 2024-09-21T03:39:50.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:50.573+0000 7f8efb7fe640 1 -- 172.21.15.38:0/3057586613 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f8eec035020 con 0x7f8f04151940 2024-09-21T03:39:50.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:50.573+0000 7f8efb7fe640 1 --2- 172.21.15.38:0/3057586613 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8ee4078b90 0x7f8ee407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:50.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:50.573+0000 7f8efb7fe640 1 -- 172.21.15.38:0/3057586613 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 6187+0+0 (secure 0 0 0) 0x7f8eec0bb5d0 con 0x7f8f04151940 2024-09-21T03:39:50.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:50.574+0000 7f8f02d9d640 1 --2- 172.21.15.38:0/3057586613 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8ee4078b90 0x7f8ee407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:50.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:50.574+0000 7f8f02d9d640 1 --2- 172.21.15.38:0/3057586613 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8ee4078b90 0x7f8ee407b0a0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f8ef4000f80 tx=0x7f8ef4006ef0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:50.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:50.578+0000 7f8efb7fe640 1 -- 172.21.15.38:0/3057586613 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f8eec083830 con 0x7f8f04151940 2024-09-21T03:39:50.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:50.805+0000 7f8f09215640 1 -- 172.21.15.38:0/3057586613 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f8f04154220 con 0x7f8f04151940 2024-09-21T03:39:50.808 INFO:tasks.ceph.mon.a.smithi038.stderr:2024-09-21T03:39:50.806+0000 7f3f45713640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T03:39:50.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:50.807+0000 7f8efb7fe640 1 -- 172.21.15.38:0/3057586613 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v75) ==== 161+0+0 (secure 0 0 0) 0x7f8eec083130 con 0x7f8f04151940 2024-09-21T03:39:50.809 INFO:teuthology.orchestra.run.smithi038.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-21T03:39:50.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:50.811+0000 7f8f09215640 1 -- 172.21.15.38:0/3057586613 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8ee4078b90 msgr2=0x7f8ee407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:50.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:50.811+0000 7f8f09215640 1 --2- 172.21.15.38:0/3057586613 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8ee4078b90 0x7f8ee407b0a0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f8ef4000f80 tx=0x7f8ef4006ef0 comp rx=0 tx=0).stop 2024-09-21T03:39:50.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:50.811+0000 7f8f09215640 1 -- 172.21.15.38:0/3057586613 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8f04151940 msgr2=0x7f8f041f63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:50.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:50.811+0000 7f8f09215640 1 --2- 172.21.15.38:0/3057586613 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8f04151940 0x7f8f041f63a0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f8eec00c320 tx=0x7f8eec001220 comp rx=0 tx=0).stop 2024-09-21T03:39:50.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:50.811+0000 7f8f09215640 1 -- 172.21.15.38:0/3057586613 shutdown_connections 2024-09-21T03:39:50.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:50.811+0000 7f8f09215640 1 --2- 172.21.15.38:0/3057586613 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8ee4078b90 0x7f8ee407b0a0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:50.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:50.811+0000 7f8f09215640 1 --2- 172.21.15.38:0/3057586613 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8f04150970 0x7f8f041f68e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:50.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:50.811+0000 7f8f09215640 1 --2- 172.21.15.38:0/3057586613 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8f04151940 0x7f8f041f63a0 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:50.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:50.811+0000 7f8f09215640 1 --2- 172.21.15.38:0/3057586613 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8ef0004800 0x7f8f04153910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:50.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:50.811+0000 7f8f09215640 1 -- 172.21.15.38:0/3057586613 >> 172.21.15.38:0/3057586613 conn(0x7f8f040b46e0 msgr2=0x7f8f040b28f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:50.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:50.811+0000 7f8f09215640 1 -- 172.21.15.38:0/3057586613 shutdown_connections 2024-09-21T03:39:50.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:50.811+0000 7f8f09215640 1 -- 172.21.15.38:0/3057586613 wait complete. 2024-09-21T03:39:50.817 INFO:tasks.ceph.mds.d.smithi195.stderr: -873> 2024-09-21T03:39:47.381+0000 7f462f303640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T03:39:50.824 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-21T03:39:50.869 INFO:tasks.ceph.mds.d.smithi195.stdout:starting mds.d at 2024-09-21T03:39:50.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:50.941+0000 7f5fff769640 1 -- 172.21.15.38:0/2998886337 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5ff80d0bb0 msgr2=0x7f5ff80b5be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:50.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:50.941+0000 7f5fff769640 1 --2- 172.21.15.38:0/2998886337 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5ff80d0bb0 0x7f5ff80b5be0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f5fe8005fd0 tx=0x7f5fe802a820 comp rx=0 tx=0).stop 2024-09-21T03:39:50.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:50.941+0000 7f5fff769640 1 -- 172.21.15.38:0/2998886337 shutdown_connections 2024-09-21T03:39:50.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:50.941+0000 7f5fff769640 1 --2- 172.21.15.38:0/2998886337 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5ff80d0bb0 0x7f5ff80b5be0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:50.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:50.941+0000 7f5fff769640 1 --2- 172.21.15.38:0/2998886337 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5ff815ca90 0x7f5ff80d0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:50.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:50.942+0000 7f5fff769640 1 -- 172.21.15.38:0/2998886337 >> 172.21.15.38:0/2998886337 conn(0x7f5ff80b46e0 msgr2=0x7f5ff80d23e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:50.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:50.942+0000 7f5fff769640 1 -- 172.21.15.38:0/2998886337 shutdown_connections 2024-09-21T03:39:50.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:50.942+0000 7f5fff769640 1 -- 172.21.15.38:0/2998886337 wait complete. 2024-09-21T03:39:50.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:50.943+0000 7f5fff769640 1 Processor -- start 2024-09-21T03:39:50.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:50.943+0000 7f5fff769640 1 -- start start 2024-09-21T03:39:50.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:50.945+0000 7f5fff769640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5ff80ccc40 0x7f5ff814f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:50.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:50.945+0000 7f5fff769640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5ff80d0bb0 0x7f5ff81f6810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:50.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:50.945+0000 7f5fff769640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5ff815ca90 0x7f5ff81f6d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:50.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:50.945+0000 7f5fff769640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f5ff80b65c0 con 0x7f5ff80d0bb0 2024-09-21T03:39:50.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:50.945+0000 7f5fff769640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f5ff80b6440 con 0x7f5ff815ca90 2024-09-21T03:39:50.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:50.945+0000 7f5fff769640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f5ff80b6740 con 0x7f5ff80ccc40 2024-09-21T03:39:50.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:50.945+0000 7f5ffcd04640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5ff80d0bb0 0x7f5ff81f6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:50.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:50.945+0000 7f5ffcd04640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5ff80d0bb0 0x7f5ff81f6810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42482/0 (socket says 172.21.15.38:42482) 2024-09-21T03:39:50.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:50.945+0000 7f5ffcd04640 1 -- 172.21.15.38:0/1777910586 learned_addr learned my addr 172.21.15.38:0/1777910586 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:39:50.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:50.945+0000 7f5ffd505640 1 --2- 172.21.15.38:0/1777910586 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5ff80ccc40 0x7f5ff814f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:50.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:50.945+0000 7f5ffdd06640 1 --2- 172.21.15.38:0/1777910586 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5ff815ca90 0x7f5ff81f6d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:50.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:50.945+0000 7f5ffcd04640 1 -- 172.21.15.38:0/1777910586 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5ff80ccc40 msgr2=0x7f5ff814f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:50.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:50.945+0000 7f5ffcd04640 1 --2- 172.21.15.38:0/1777910586 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5ff80ccc40 0x7f5ff814f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:50.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:50.945+0000 7f5ffcd04640 1 -- 172.21.15.38:0/1777910586 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5ff815ca90 msgr2=0x7f5ff81f6d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:50.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:50.945+0000 7f5ffcd04640 1 --2- 172.21.15.38:0/1777910586 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5ff815ca90 0x7f5ff81f6d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:50.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:50.945+0000 7f5ffcd04640 1 -- 172.21.15.38:0/1777910586 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ff80aa5b0 con 0x7f5ff80d0bb0 2024-09-21T03:39:50.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:50.945+0000 7f5ffcd04640 1 --2- 172.21.15.38:0/1777910586 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5ff80d0bb0 0x7f5ff81f6810 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f5fec009fd0 tx=0x7f5fec00ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:50.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:50.946+0000 7f5ff5ffb640 1 -- 172.21.15.38:0/1777910586 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fec00ccc0 con 0x7f5ff80d0bb0 2024-09-21T03:39:50.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:50.946+0000 7f5ff5ffb640 1 -- 172.21.15.38:0/1777910586 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5fec00ee70 con 0x7f5ff80d0bb0 2024-09-21T03:39:50.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:50.946+0000 7f5fff769640 1 -- 172.21.15.38:0/1777910586 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ff81f3870 con 0x7f5ff80d0bb0 2024-09-21T03:39:50.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:50.946+0000 7f5ff5ffb640 1 -- 172.21.15.38:0/1777910586 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fec00ce60 con 0x7f5ff80d0bb0 2024-09-21T03:39:50.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:50.946+0000 7f5fff769640 1 -- 172.21.15.38:0/1777910586 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ff81f3e30 con 0x7f5ff80d0bb0 2024-09-21T03:39:50.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:50.947+0000 7f5fff769640 1 -- 172.21.15.38:0/1777910586 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ff81f35e0 con 0x7f5ff80d0bb0 2024-09-21T03:39:50.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:50.948+0000 7f5ff5ffb640 1 -- 172.21.15.38:0/1777910586 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f5fec004280 con 0x7f5ff80d0bb0 2024-09-21T03:39:50.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:50.948+0000 7f5ff5ffb640 1 --2- 172.21.15.38:0/1777910586 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5fe4078bc0 0x7f5fe407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:50.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:50.948+0000 7f5ff5ffb640 1 -- 172.21.15.38:0/1777910586 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 6273+0+0 (secure 0 0 0) 0x7f5fec09b690 con 0x7f5ff80d0bb0 2024-09-21T03:39:50.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:50.948+0000 7f5ffd505640 1 --2- 172.21.15.38:0/1777910586 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5fe4078bc0 0x7f5fe407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:50.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:50.949+0000 7f5ffd505640 1 --2- 172.21.15.38:0/1777910586 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5fe4078bc0 0x7f5fe407b0d0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f5ff0000f60 tx=0x7f5ff0009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:50.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:50.953+0000 7f5ff5ffb640 1 -- 172.21.15.38:0/1777910586 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f5fec0632d0 con 0x7f5ff80d0bb0 2024-09-21T03:39:51.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:51.186+0000 7f5fff769640 1 -- 172.21.15.38:0/1777910586 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f5ff80b6cd0 con 0x7f5ff80d0bb0 2024-09-21T03:39:51.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:51.818+0000 7f5ff5ffb640 1 -- 172.21.15.38:0/1777910586 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v76) ==== 104+0+0 (secure 0 0 0) 0x7f5fec067d00 con 0x7f5ff80d0bb0 2024-09-21T03:39:51.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:51.822+0000 7f5fff769640 1 -- 172.21.15.38:0/1777910586 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5fe4078bc0 msgr2=0x7f5fe407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:51.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:51.822+0000 7f5fff769640 1 --2- 172.21.15.38:0/1777910586 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5fe4078bc0 0x7f5fe407b0d0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f5ff0000f60 tx=0x7f5ff0009040 comp rx=0 tx=0).stop 2024-09-21T03:39:51.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:51.822+0000 7f5fff769640 1 -- 172.21.15.38:0/1777910586 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5ff80d0bb0 msgr2=0x7f5ff81f6810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:51.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:51.822+0000 7f5fff769640 1 --2- 172.21.15.38:0/1777910586 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5ff80d0bb0 0x7f5ff81f6810 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f5fec009fd0 tx=0x7f5fec00ec20 comp rx=0 tx=0).stop 2024-09-21T03:39:51.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:51.822+0000 7f5fff769640 1 -- 172.21.15.38:0/1777910586 shutdown_connections 2024-09-21T03:39:51.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:51.822+0000 7f5fff769640 1 --2- 172.21.15.38:0/1777910586 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5ff815ca90 0x7f5ff81f6d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:51.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:51.822+0000 7f5fff769640 1 --2- 172.21.15.38:0/1777910586 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5ff80d0bb0 0x7f5ff81f6810 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:51.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:51.822+0000 7f5fff769640 1 --2- 172.21.15.38:0/1777910586 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5fe4078bc0 0x7f5fe407b0d0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:51.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:51.822+0000 7f5fff769640 1 --2- 172.21.15.38:0/1777910586 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5ff80ccc40 0x7f5ff814f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:51.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:51.822+0000 7f5fff769640 1 -- 172.21.15.38:0/1777910586 >> 172.21.15.38:0/1777910586 conn(0x7f5ff80b46e0 msgr2=0x7f5ff8070100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:51.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:51.822+0000 7f5fff769640 1 -- 172.21.15.38:0/1777910586 shutdown_connections 2024-09-21T03:39:51.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:51.822+0000 7f5fff769640 1 -- 172.21.15.38:0/1777910586 wait complete. 2024-09-21T03:39:51.838 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-21T03:39:51.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:51.965+0000 7f46e5739640 1 -- 172.21.15.38:0/31512422 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f46cc003c10 msgr2=0x7f46cc004060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:51.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:51.965+0000 7f46e5739640 1 --2- 172.21.15.38:0/31512422 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f46cc003c10 0x7f46cc004060 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f46d0009b80 tx=0x7f46cc005830 comp rx=0 tx=0).stop 2024-09-21T03:39:51.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:51.965+0000 7f46df7fe640 1 -- 172.21.15.38:0/31512422 reap_dead start 2024-09-21T03:39:51.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:51.965+0000 7f46e5739640 1 -- 172.21.15.38:0/31512422 shutdown_connections 2024-09-21T03:39:51.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:51.965+0000 7f46e5739640 1 -- 172.21.15.38:0/31512422 >> 172.21.15.38:0/31512422 conn(0x7f46e006cde0 msgr2=0x7f46e006d1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:51.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:51.965+0000 7f46e5739640 1 -- 172.21.15.38:0/31512422 shutdown_connections 2024-09-21T03:39:51.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:51.965+0000 7f46e5739640 1 -- 172.21.15.38:0/31512422 wait complete. 2024-09-21T03:39:51.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:51.967+0000 7f46e5739640 1 Processor -- start 2024-09-21T03:39:51.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:51.967+0000 7f46e5739640 1 -- start start 2024-09-21T03:39:51.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:51.968+0000 7f46e5739640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f46cc003c10 0x7f46e00b6920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:51.972 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:51.968+0000 7f46e5739640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f46e0159d80 0x7f46e015a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:51.972 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:51.968+0000 7f46e5739640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f46e00c7f40 0x7f46e00c8350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:51.972 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:51.968+0000 7f46e5739640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f46e015c7d0 con 0x7f46cc003c10 2024-09-21T03:39:51.972 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:51.968+0000 7f46e5739640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f46e015c650 con 0x7f46e00c7f40 2024-09-21T03:39:51.972 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:51.968+0000 7f46e5739640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f46e015c950 con 0x7f46e0159d80 2024-09-21T03:39:51.972 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:51.969+0000 7f46deffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f46cc003c10 0x7f46e00b6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:51.972 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:51.969+0000 7f46deffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f46cc003c10 0x7f46e00b6920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42496/0 (socket says 172.21.15.38:42496) 2024-09-21T03:39:51.972 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:51.969+0000 7f46deffd640 1 -- 172.21.15.38:0/525755372 learned_addr learned my addr 172.21.15.38:0/525755372 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:39:51.972 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:51.969+0000 7f46d7fff640 1 --2- 172.21.15.38:0/525755372 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f46e0159d80 0x7f46e015a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:51.972 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:51.969+0000 7f46df7fe640 1 --2- 172.21.15.38:0/525755372 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f46e00c7f40 0x7f46e00c8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:51.972 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:51.969+0000 7f46deffd640 1 -- 172.21.15.38:0/525755372 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f46e0159d80 msgr2=0x7f46e015a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:51.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:51.969+0000 7f46deffd640 1 --2- 172.21.15.38:0/525755372 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f46e0159d80 0x7f46e015a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:51.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:51.969+0000 7f46deffd640 1 -- 172.21.15.38:0/525755372 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f46e00c7f40 msgr2=0x7f46e00c8350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:51.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:51.969+0000 7f46deffd640 1 --2- 172.21.15.38:0/525755372 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f46e00c7f40 0x7f46e00c8350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:51.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:51.969+0000 7f46deffd640 1 -- 172.21.15.38:0/525755372 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46e015a6d0 con 0x7f46cc003c10 2024-09-21T03:39:51.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:51.969+0000 7f46deffd640 1 --2- 172.21.15.38:0/525755372 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f46cc003c10 0x7f46e00b6920 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f46d0009ca0 tx=0x7f46d0033670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:51.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:51.969+0000 7f46d77fe640 1 -- 172.21.15.38:0/525755372 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46d0038040 con 0x7f46cc003c10 2024-09-21T03:39:51.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:51.969+0000 7f46d77fe640 1 -- 172.21.15.38:0/525755372 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f46d0004020 con 0x7f46cc003c10 2024-09-21T03:39:51.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:51.969+0000 7f46e5739640 1 -- 172.21.15.38:0/525755372 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46e00c8de0 con 0x7f46cc003c10 2024-09-21T03:39:51.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:51.969+0000 7f46d77fe640 1 -- 172.21.15.38:0/525755372 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46d0004470 con 0x7f46cc003c10 2024-09-21T03:39:51.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:51.969+0000 7f46e5739640 1 -- 172.21.15.38:0/525755372 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46e00c93a0 con 0x7f46cc003c10 2024-09-21T03:39:51.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:51.970+0000 7f46e5739640 1 -- 172.21.15.38:0/525755372 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46e00b38d0 con 0x7f46cc003c10 2024-09-21T03:39:51.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:51.971+0000 7f46d77fe640 1 -- 172.21.15.38:0/525755372 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f46d0045020 con 0x7f46cc003c10 2024-09-21T03:39:51.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:51.972+0000 7f46d77fe640 1 --2- 172.21.15.38:0/525755372 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f46c0078b90 0x7f46c007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:51.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:51.972+0000 7f46d77fe640 1 -- 172.21.15.38:0/525755372 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 6273+0+0 (secure 0 0 0) 0x7f46d00bb8a0 con 0x7f46cc003c10 2024-09-21T03:39:51.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:51.972+0000 7f46d7fff640 1 --2- 172.21.15.38:0/525755372 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f46c0078b90 0x7f46c007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:51.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:51.972+0000 7f46d7fff640 1 --2- 172.21.15.38:0/525755372 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f46c0078b90 0x7f46c007b0a0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f46e00ba5d0 tx=0x7f46c8001090 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:51.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:51.976+0000 7f46d77fe640 1 -- 172.21.15.38:0/525755372 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f46e015a6d0 con 0x7f46cc003c10 2024-09-21T03:39:52.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:52.166+0000 7f46e5739640 1 -- 172.21.15.38:0/525755372 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f46e01eb6d0 con 0x7f46cc003c10 2024-09-21T03:39:52.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:52.824+0000 7f46d77fe640 1 -- 172.21.15.38:0/525755372 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v85) ==== 181+0+0 (secure 0 0 0) 0x7f46d00833b0 con 0x7f46cc003c10 2024-09-21T03:39:52.826 INFO:teuthology.orchestra.run.smithi038.stderr:pool 'cephfs_metadata' removed 2024-09-21T03:39:52.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:52.829+0000 7f46e5739640 1 -- 172.21.15.38:0/525755372 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f46c0078b90 msgr2=0x7f46c007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:52.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:52.829+0000 7f46e5739640 1 --2- 172.21.15.38:0/525755372 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f46c0078b90 0x7f46c007b0a0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f46e00ba5d0 tx=0x7f46c8001090 comp rx=0 tx=0).stop 2024-09-21T03:39:52.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:52.829+0000 7f46e5739640 1 -- 172.21.15.38:0/525755372 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f46cc003c10 msgr2=0x7f46e00b6920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:52.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:52.829+0000 7f46e5739640 1 --2- 172.21.15.38:0/525755372 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f46cc003c10 0x7f46e00b6920 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f46d0009ca0 tx=0x7f46d0033670 comp rx=0 tx=0).stop 2024-09-21T03:39:52.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:52.829+0000 7f46e5739640 1 -- 172.21.15.38:0/525755372 shutdown_connections 2024-09-21T03:39:52.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:52.829+0000 7f46e5739640 1 --2- 172.21.15.38:0/525755372 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f46e00c7f40 0x7f46e00c8350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:52.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:52.829+0000 7f46e5739640 1 --2- 172.21.15.38:0/525755372 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f46c0078b90 0x7f46c007b0a0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:52.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:52.829+0000 7f46e5739640 1 --2- 172.21.15.38:0/525755372 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f46e0159d80 0x7f46e015a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:52.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:52.829+0000 7f46e5739640 1 --2- 172.21.15.38:0/525755372 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f46cc003c10 0x7f46e00b6920 secure :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f46d0009ca0 tx=0x7f46d0033670 comp rx=0 tx=0).stop 2024-09-21T03:39:52.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:52.829+0000 7f46e5739640 1 -- 172.21.15.38:0/525755372 >> 172.21.15.38:0/525755372 conn(0x7f46e006cde0 msgr2=0x7f46e006d1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:52.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:52.830+0000 7f46e5739640 1 -- 172.21.15.38:0/525755372 shutdown_connections 2024-09-21T03:39:52.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:52.830+0000 7f46e5739640 1 -- 172.21.15.38:0/525755372 wait complete. 2024-09-21T03:39:52.849 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-21T03:39:52.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:52.974+0000 7f5c91007640 1 -- 172.21.15.38:0/2360050946 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5c8c162cf0 msgr2=0x7f5c8c0b3000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:52.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:52.974+0000 7f5c91007640 1 --2- 172.21.15.38:0/2360050946 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5c8c162cf0 0x7f5c8c0b3000 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f5c8c0d2120 tx=0x7f5c8402a930 comp rx=0 tx=0).stop 2024-09-21T03:39:52.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:52.975+0000 7f5c91007640 1 -- 172.21.15.38:0/2360050946 shutdown_connections 2024-09-21T03:39:52.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:52.975+0000 7f5c91007640 1 --2- 172.21.15.38:0/2360050946 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5c8c162cf0 0x7f5c8c0b3000 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:52.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:52.975+0000 7f5c91007640 1 --2- 172.21.15.38:0/2360050946 >> v2:172.21.15.195:3300/0 conn(0x7f5c8c0d0e40 0x7f5c8c1627b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:52.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:52.975+0000 7f5c91007640 1 --2- 172.21.15.38:0/2360050946 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5c8c163350 0x7f5c8c15c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:52.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:52.975+0000 7f5c91007640 1 -- 172.21.15.38:0/2360050946 >> 172.21.15.38:0/2360050946 conn(0x7f5c8c06cde0 msgr2=0x7f5c8c06d1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:52.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:52.975+0000 7f5c91007640 1 -- 172.21.15.38:0/2360050946 shutdown_connections 2024-09-21T03:39:52.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:52.975+0000 7f5c91007640 1 -- 172.21.15.38:0/2360050946 wait complete. 2024-09-21T03:39:52.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:52.977+0000 7f5c91007640 1 Processor -- start 2024-09-21T03:39:52.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:52.977+0000 7f5c91007640 1 -- start start 2024-09-21T03:39:52.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:52.978+0000 7f5c91007640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5c8c0d0e40 0x7f5c8c0caa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:52.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:52.978+0000 7f5c91007640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5c8c162cf0 0x7f5c8c0b5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:52.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:52.978+0000 7f5c91007640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5c8c163350 0x7f5c8c0b5d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:52.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:52.978+0000 7f5c91007640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f5c8c0b38d0 con 0x7f5c8c0d0e40 2024-09-21T03:39:52.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:52.978+0000 7f5c91007640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f5c8c0b3750 con 0x7f5c8c163350 2024-09-21T03:39:52.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:52.978+0000 7f5c91007640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f5c8c0b3a50 con 0x7f5c8c162cf0 2024-09-21T03:39:52.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:52.978+0000 7f5c8ad9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5c8c0d0e40 0x7f5c8c0caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:52.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:52.978+0000 7f5c8ad9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5c8c0d0e40 0x7f5c8c0caa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:45156/0 (socket says 172.21.15.38:45156) 2024-09-21T03:39:52.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:52.978+0000 7f5c8ad9d640 1 -- 172.21.15.38:0/4016972231 learned_addr learned my addr 172.21.15.38:0/4016972231 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:39:52.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:52.978+0000 7f5c8a59c640 1 --2- 172.21.15.38:0/4016972231 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5c8c162cf0 0x7f5c8c0b5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:52.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:52.978+0000 7f5c8b59e640 1 --2- 172.21.15.38:0/4016972231 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5c8c163350 0x7f5c8c0b5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:52.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:52.979+0000 7f5c8ad9d640 1 -- 172.21.15.38:0/4016972231 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5c8c162cf0 msgr2=0x7f5c8c0b5460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:52.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:52.979+0000 7f5c8ad9d640 1 --2- 172.21.15.38:0/4016972231 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5c8c162cf0 0x7f5c8c0b5460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:52.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:52.979+0000 7f5c8ad9d640 1 -- 172.21.15.38:0/4016972231 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5c8c163350 msgr2=0x7f5c8c0b5d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:52.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:52.979+0000 7f5c8ad9d640 1 --2- 172.21.15.38:0/4016972231 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5c8c163350 0x7f5c8c0b5d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:52.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:52.979+0000 7f5c8ad9d640 1 -- 172.21.15.38:0/4016972231 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c8c0b43c0 con 0x7f5c8c0d0e40 2024-09-21T03:39:52.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:52.979+0000 7f5c8a59c640 1 --2- 172.21.15.38:0/4016972231 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5c8c162cf0 0x7f5c8c0b5460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:39:52.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:52.979+0000 7f5c8ad9d640 1 --2- 172.21.15.38:0/4016972231 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5c8c0d0e40 0x7f5c8c0caa90 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f5c7c00e7c0 tx=0x7f5c7c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:52.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:52.979+0000 7f5c737fe640 1 -- 172.21.15.38:0/4016972231 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c7c00ccc0 con 0x7f5c8c0d0e40 2024-09-21T03:39:52.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:52.979+0000 7f5c737fe640 1 -- 172.21.15.38:0/4016972231 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5c7c00ce80 con 0x7f5c8c0d0e40 2024-09-21T03:39:52.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:52.979+0000 7f5c91007640 1 -- 172.21.15.38:0/4016972231 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c8c0b46a0 con 0x7f5c8c0d0e40 2024-09-21T03:39:52.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:52.979+0000 7f5c737fe640 1 -- 172.21.15.38:0/4016972231 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c7c005280 con 0x7f5c8c0d0e40 2024-09-21T03:39:52.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:52.979+0000 7f5c91007640 1 -- 172.21.15.38:0/4016972231 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c8c0b4c60 con 0x7f5c8c0d0e40 2024-09-21T03:39:52.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:52.980+0000 7f5c91007640 1 -- 172.21.15.38:0/4016972231 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c8c0b3000 con 0x7f5c8c0d0e40 2024-09-21T03:39:52.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:52.981+0000 7f5c737fe640 1 -- 172.21.15.38:0/4016972231 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f5c7c005420 con 0x7f5c8c0d0e40 2024-09-21T03:39:52.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:52.981+0000 7f5c737fe640 1 --2- 172.21.15.38:0/4016972231 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5c78078bc0 0x7f5c7807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:52.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:52.981+0000 7f5c737fe640 1 -- 172.21.15.38:0/4016972231 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5835+0+0 (secure 0 0 0) 0x7f5c7c09b860 con 0x7f5c8c0d0e40 2024-09-21T03:39:52.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:52.982+0000 7f5c8a59c640 1 --2- 172.21.15.38:0/4016972231 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5c78078bc0 0x7f5c7807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:52.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:52.982+0000 7f5c8a59c640 1 --2- 172.21.15.38:0/4016972231 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5c78078bc0 0x7f5c7807b0d0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f5c8c0ca0e0 tx=0x7f5c80005f90 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:52.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:52.986+0000 7f5c737fe640 1 -- 172.21.15.38:0/4016972231 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f5c7c063ba0 con 0x7f5c8c0d0e40 2024-09-21T03:39:53.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:53.174+0000 7f5c91007640 1 -- 172.21.15.38:0/4016972231 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f5c8c0b8470 con 0x7f5c8c0d0e40 2024-09-21T03:39:53.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:53.829+0000 7f5c737fe640 1 -- 172.21.15.38:0/4016972231 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v86) ==== 169+0+0 (secure 0 0 0) 0x7f5c7c0634a0 con 0x7f5c8c0d0e40 2024-09-21T03:39:53.831 INFO:teuthology.orchestra.run.smithi038.stderr:pool 'cephfs_data' removed 2024-09-21T03:39:53.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:53.833+0000 7f5c91007640 1 -- 172.21.15.38:0/4016972231 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5c78078bc0 msgr2=0x7f5c7807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:53.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:53.833+0000 7f5c91007640 1 --2- 172.21.15.38:0/4016972231 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5c78078bc0 0x7f5c7807b0d0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f5c8c0ca0e0 tx=0x7f5c80005f90 comp rx=0 tx=0).stop 2024-09-21T03:39:53.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:53.833+0000 7f5c91007640 1 -- 172.21.15.38:0/4016972231 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5c8c0d0e40 msgr2=0x7f5c8c0caa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:53.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:53.833+0000 7f5c91007640 1 --2- 172.21.15.38:0/4016972231 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5c8c0d0e40 0x7f5c8c0caa90 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f5c7c00e7c0 tx=0x7f5c7c00ec80 comp rx=0 tx=0).stop 2024-09-21T03:39:53.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:53.836+0000 7f5c91007640 1 -- 172.21.15.38:0/4016972231 shutdown_connections 2024-09-21T03:39:53.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:53.836+0000 7f5c91007640 1 --2- 172.21.15.38:0/4016972231 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5c8c163350 0x7f5c8c0b5d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:53.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:53.836+0000 7f5c91007640 1 --2- 172.21.15.38:0/4016972231 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5c78078bc0 0x7f5c7807b0d0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:53.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:53.836+0000 7f5c91007640 1 --2- 172.21.15.38:0/4016972231 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5c8c162cf0 0x7f5c8c0b5460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:53.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:53.836+0000 7f5c91007640 1 --2- 172.21.15.38:0/4016972231 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5c8c0d0e40 0x7f5c8c0caa90 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:53.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:53.836+0000 7f5c91007640 1 -- 172.21.15.38:0/4016972231 >> 172.21.15.38:0/4016972231 conn(0x7f5c8c06cde0 msgr2=0x7f5c8c0d2b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:53.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:53.837+0000 7f5c91007640 1 -- 172.21.15.38:0/4016972231 shutdown_connections 2024-09-21T03:39:53.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:53.838+0000 7f5c91007640 1 -- 172.21.15.38:0/4016972231 wait complete. 2024-09-21T03:39:53.856 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 31 2024-09-21T03:39:53.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:53.978+0000 7f15e4752640 1 -- 172.21.15.38:0/1032545210 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f15c8003a50 msgr2=0x7f15c8003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:53.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:53.978+0000 7f15e4752640 1 --2- 172.21.15.38:0/1032545210 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f15c8003a50 0x7f15c8003ec0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f15d800c3f0 tx=0x7f15c8005830 comp rx=0 tx=0).stop 2024-09-21T03:39:53.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:53.979+0000 7f15e4752640 1 -- 172.21.15.38:0/1032545210 shutdown_connections 2024-09-21T03:39:53.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:53.979+0000 7f15e4752640 1 --2- 172.21.15.38:0/1032545210 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f15c8003a50 0x7f15c8003ec0 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:53.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:53.979+0000 7f15e4752640 1 -- 172.21.15.38:0/1032545210 >> 172.21.15.38:0/1032545210 conn(0x7f15dc09d830 msgr2=0x7f15dc0b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:53.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:53.979+0000 7f15e4752640 1 -- 172.21.15.38:0/1032545210 shutdown_connections 2024-09-21T03:39:53.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:53.979+0000 7f15e4752640 1 -- 172.21.15.38:0/1032545210 wait complete. 2024-09-21T03:39:53.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:53.980+0000 7f15e4752640 1 Processor -- start 2024-09-21T03:39:53.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:53.980+0000 7f15e4752640 1 -- start start 2024-09-21T03:39:53.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:53.982+0000 7f15e4752640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f15c8003a50 0x7f15dc157b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:53.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:53.982+0000 7f15e4752640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f15dc15c240 0x7f15dc0d40d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:53.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:53.982+0000 7f15e4752640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f15dc0d4610 0x7f15dc06cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:53.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:53.982+0000 7f15e4752640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f15dc0b9c50 con 0x7f15dc15c240 2024-09-21T03:39:53.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:53.982+0000 7f15e4752640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f15dc0b9ad0 con 0x7f15dc0d4610 2024-09-21T03:39:53.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:53.982+0000 7f15e4752640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f15dc0b9dd0 con 0x7f15c8003a50 2024-09-21T03:39:53.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:53.982+0000 7f15e1ced640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f15dc15c240 0x7f15dc0d40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:53.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:53.982+0000 7f15e1ced640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f15dc15c240 0x7f15dc0d40d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:45184/0 (socket says 172.21.15.38:45184) 2024-09-21T03:39:53.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:53.982+0000 7f15e1ced640 1 -- 172.21.15.38:0/654900135 learned_addr learned my addr 172.21.15.38:0/654900135 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:39:53.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:53.982+0000 7f15e24ee640 1 --2- 172.21.15.38:0/654900135 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f15c8003a50 0x7f15dc157b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:53.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:53.982+0000 7f15e2cef640 1 --2- 172.21.15.38:0/654900135 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f15dc0d4610 0x7f15dc06cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:53.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:53.982+0000 7f15e1ced640 1 -- 172.21.15.38:0/654900135 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f15c8003a50 msgr2=0x7f15dc157b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:53.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:53.982+0000 7f15e1ced640 1 --2- 172.21.15.38:0/654900135 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f15c8003a50 0x7f15dc157b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:53.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:53.982+0000 7f15e1ced640 1 -- 172.21.15.38:0/654900135 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f15dc0d4610 msgr2=0x7f15dc06cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:53.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:53.982+0000 7f15e1ced640 1 --2- 172.21.15.38:0/654900135 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f15dc0d4610 0x7f15dc06cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:53.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:53.982+0000 7f15e1ced640 1 -- 172.21.15.38:0/654900135 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15dc0d4a20 con 0x7f15dc15c240 2024-09-21T03:39:53.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:53.982+0000 7f15e1ced640 1 --2- 172.21.15.38:0/654900135 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f15dc15c240 0x7f15dc0d40d0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f15cc011800 tx=0x7f15cc011cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:53.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:53.983+0000 7f15d2ffd640 1 -- 172.21.15.38:0/654900135 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15cc004460 con 0x7f15dc15c240 2024-09-21T03:39:53.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:53.983+0000 7f15d2ffd640 1 -- 172.21.15.38:0/654900135 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f15cc016070 con 0x7f15dc15c240 2024-09-21T03:39:53.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:53.983+0000 7f15e4752640 1 -- 172.21.15.38:0/654900135 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15dc1f96b0 con 0x7f15dc15c240 2024-09-21T03:39:53.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:53.983+0000 7f15d2ffd640 1 -- 172.21.15.38:0/654900135 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15cc00fb40 con 0x7f15dc15c240 2024-09-21T03:39:53.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:53.983+0000 7f15e4752640 1 -- 172.21.15.38:0/654900135 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15dc159700 con 0x7f15dc15c240 2024-09-21T03:39:53.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:53.984+0000 7f15e4752640 1 -- 172.21.15.38:0/654900135 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15dc0d4e70 con 0x7f15dc15c240 2024-09-21T03:39:53.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:53.984+0000 7f15d2ffd640 1 -- 172.21.15.38:0/654900135 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f15cc018030 con 0x7f15dc15c240 2024-09-21T03:39:53.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:53.985+0000 7f15d2ffd640 1 --2- 172.21.15.38:0/654900135 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f15c0078b90 0x7f15c007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:53.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:53.985+0000 7f15d2ffd640 1 -- 172.21.15.38:0/654900135 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5421+0+0 (secure 0 0 0) 0x7f15cc09b310 con 0x7f15dc15c240 2024-09-21T03:39:53.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:53.985+0000 7f15e24ee640 1 --2- 172.21.15.38:0/654900135 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f15c0078b90 0x7f15c007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:53.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:53.986+0000 7f15e24ee640 1 --2- 172.21.15.38:0/654900135 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f15c0078b90 0x7f15c007b0a0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f15d8033000 tx=0x7f15d8000fe0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:53.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:53.990+0000 7f15d2ffd640 1 -- 172.21.15.38:0/654900135 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f15dc0d4e70 con 0x7f15dc15c240 2024-09-21T03:39:54.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:54.177+0000 7f15e4752640 1 -- 172.21.15.38:0/654900135 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "config reset", "num": 31} v 0) -- 0x7f15dc0cec70 con 0x7f15dc15c240 2024-09-21T03:39:54.183 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:54.181+0000 7f15d2ffd640 1 -- 172.21.15.38:0/654900135 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 31}]=0 v46) ==== 71+0+0 (secure 0 0 0) 0x7f15dc0d4a20 con 0x7f15dc15c240 2024-09-21T03:39:54.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:54.184+0000 7f15e4752640 1 -- 172.21.15.38:0/654900135 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f15c0078b90 msgr2=0x7f15c007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:54.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:54.184+0000 7f15e4752640 1 --2- 172.21.15.38:0/654900135 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f15c0078b90 0x7f15c007b0a0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f15d8033000 tx=0x7f15d8000fe0 comp rx=0 tx=0).stop 2024-09-21T03:39:54.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:54.184+0000 7f15e4752640 1 -- 172.21.15.38:0/654900135 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f15dc15c240 msgr2=0x7f15dc0d40d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:54.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:54.184+0000 7f15e4752640 1 --2- 172.21.15.38:0/654900135 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f15dc15c240 0x7f15dc0d40d0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f15cc011800 tx=0x7f15cc011cc0 comp rx=0 tx=0).stop 2024-09-21T03:39:54.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:54.184+0000 7f15e4752640 1 -- 172.21.15.38:0/654900135 shutdown_connections 2024-09-21T03:39:54.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:54.184+0000 7f15e4752640 1 --2- 172.21.15.38:0/654900135 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f15dc0d4610 0x7f15dc06cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:54.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:54.184+0000 7f15e4752640 1 --2- 172.21.15.38:0/654900135 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f15dc15c240 0x7f15dc0d40d0 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:54.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:54.184+0000 7f15e4752640 1 --2- 172.21.15.38:0/654900135 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f15c0078b90 0x7f15c007b0a0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:54.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:54.184+0000 7f15e4752640 1 --2- 172.21.15.38:0/654900135 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f15c8003a50 0x7f15dc157b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:54.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:54.184+0000 7f15e4752640 1 -- 172.21.15.38:0/654900135 >> 172.21.15.38:0/654900135 conn(0x7f15dc09d830 msgr2=0x7f15dc15d5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:54.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:54.184+0000 7f15e4752640 1 -- 172.21.15.38:0/654900135 shutdown_connections 2024-09-21T03:39:54.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:54.184+0000 7f15e4752640 1 -- 172.21.15.38:0/654900135 wait complete. 2024-09-21T03:39:54.201 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_damage.TestDamage.test_dentry_first_preflush' 2024-09-21T03:39:54.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:54.315+0000 7fecbeabd640 1 -- 172.21.15.38:0/312132416 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fecb81536e0 msgr2=0x7fecb80d17c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:54.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:54.315+0000 7fecbeabd640 1 --2- 172.21.15.38:0/312132416 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fecb81536e0 0x7fecb80d17c0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7fecb80b2db0 tx=0x7fecac02a540 comp rx=0 tx=0).stop 2024-09-21T03:39:54.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:54.315+0000 7fecbeabd640 1 -- 172.21.15.38:0/312132416 shutdown_connections 2024-09-21T03:39:54.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:54.315+0000 7fecbeabd640 1 --2- 172.21.15.38:0/312132416 >> v2:172.21.15.195:3300/0 conn(0x7fecb80cd040 0x7fecb80d04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:54.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:54.315+0000 7fecbeabd640 1 --2- 172.21.15.38:0/312132416 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fecb81536e0 0x7fecb80d17c0 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:54.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:54.315+0000 7fecbeabd640 1 -- 172.21.15.38:0/312132416 >> 172.21.15.38:0/312132416 conn(0x7fecb80cef40 msgr2=0x7fecb80cf350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:54.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:54.315+0000 7fecbeabd640 1 -- 172.21.15.38:0/312132416 shutdown_connections 2024-09-21T03:39:54.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:54.315+0000 7fecbeabd640 1 -- 172.21.15.38:0/312132416 wait complete. 2024-09-21T03:39:54.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:54.317+0000 7fecbeabd640 1 Processor -- start 2024-09-21T03:39:54.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:54.317+0000 7fecbeabd640 1 -- start start 2024-09-21T03:39:54.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:54.319+0000 7fecbeabd640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fecb80d1d00 0x7fecb814b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:54.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:54.319+0000 7fecbeabd640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fecb81536e0 0x7fecb814bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:54.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:54.319+0000 7fecbeabd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fecb80c0300 0x7fecb81f66d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:54.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:54.319+0000 7fecbeabd640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fecb80d0d90 con 0x7fecb80c0300 2024-09-21T03:39:54.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:54.319+0000 7fecbeabd640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fecb80d0c10 con 0x7fecb80d1d00 2024-09-21T03:39:54.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:54.319+0000 7fecbeabd640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fecb80d0f10 con 0x7fecb81536e0 2024-09-21T03:39:54.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:54.319+0000 7fecbd05a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fecb80c0300 0x7fecb81f66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:54.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:54.319+0000 7fecbd05a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fecb80c0300 0x7fecb81f66d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:45208/0 (socket says 172.21.15.38:45208) 2024-09-21T03:39:54.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:54.319+0000 7fecbd05a640 1 -- 172.21.15.38:0/2631874821 learned_addr learned my addr 172.21.15.38:0/2631874821 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:39:54.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:54.319+0000 7fecbc859640 1 --2- 172.21.15.38:0/2631874821 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fecb80d1d00 0x7fecb814b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:54.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:54.319+0000 7fecb7fff640 1 --2- 172.21.15.38:0/2631874821 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fecb81536e0 0x7fecb814bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:54.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:54.319+0000 7fecbd05a640 1 -- 172.21.15.38:0/2631874821 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fecb81536e0 msgr2=0x7fecb814bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:54.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:54.319+0000 7fecbd05a640 1 --2- 172.21.15.38:0/2631874821 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fecb81536e0 0x7fecb814bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:54.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:54.319+0000 7fecbd05a640 1 -- 172.21.15.38:0/2631874821 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fecb80d1d00 msgr2=0x7fecb814b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:54.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:54.319+0000 7fecbd05a640 1 --2- 172.21.15.38:0/2631874821 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fecb80d1d00 0x7fecb814b690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:54.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:54.319+0000 7fecbd05a640 1 -- 172.21.15.38:0/2631874821 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecb81f31c0 con 0x7fecb80c0300 2024-09-21T03:39:54.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:54.319+0000 7fecbc859640 1 --2- 172.21.15.38:0/2631874821 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fecb80d1d00 0x7fecb814b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:39:54.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:54.319+0000 7fecbd05a640 1 --2- 172.21.15.38:0/2631874821 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fecb80c0300 0x7fecb81f66d0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fecb000ca90 tx=0x7fecb000cf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:54.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:54.320+0000 7fecb57fa640 1 -- 172.21.15.38:0/2631874821 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb0007df0 con 0x7fecb80c0300 2024-09-21T03:39:54.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:54.320+0000 7fecb57fa640 1 -- 172.21.15.38:0/2631874821 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fecb000f450 con 0x7fecb80c0300 2024-09-21T03:39:54.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:54.320+0000 7fecb57fa640 1 -- 172.21.15.38:0/2631874821 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb000f8c0 con 0x7fecb80c0300 2024-09-21T03:39:54.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:54.320+0000 7fecbeabd640 1 -- 172.21.15.38:0/2631874821 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fecb81f34a0 con 0x7fecb80c0300 2024-09-21T03:39:54.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:54.320+0000 7fecbeabd640 1 -- 172.21.15.38:0/2631874821 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fecb81f3a60 con 0x7fecb80c0300 2024-09-21T03:39:54.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:54.321+0000 7fecbeabd640 1 -- 172.21.15.38:0/2631874821 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fecb81f3e40 con 0x7fecb80c0300 2024-09-21T03:39:54.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:54.321+0000 7fecb57fa640 1 -- 172.21.15.38:0/2631874821 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fecb000fa60 con 0x7fecb80c0300 2024-09-21T03:39:54.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:54.322+0000 7fecb57fa640 1 --2- 172.21.15.38:0/2631874821 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fec9c078b90 0x7fec9c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:54.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:54.322+0000 7fecb57fa640 1 -- 172.21.15.38:0/2631874821 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5421+0+0 (secure 0 0 0) 0x7fecb009b780 con 0x7fecb80c0300 2024-09-21T03:39:54.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:54.322+0000 7fecbc859640 1 --2- 172.21.15.38:0/2631874821 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fec9c078b90 0x7fec9c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:54.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:54.323+0000 7fecbc859640 1 --2- 172.21.15.38:0/2631874821 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fec9c078b90 0x7fec9c07b0a0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fecac034000 tx=0x7fecac004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:54.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:54.326+0000 7fecb57fa640 1 -- 172.21.15.38:0/2631874821 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fecb81f3e40 con 0x7fecb80c0300 2024-09-21T03:39:54.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:54.588+0000 7fecbeabd640 1 -- 172.21.15.38:0/2631874821 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_damage.TestDamage.test_dentry_first_preflush"]} v 0) -- 0x7fecb8070f50 con 0x7fecb80c0300 2024-09-21T03:39:54.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:54.831+0000 7fecb57fa640 1 -- 172.21.15.38:0/2631874821 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_damage.TestDamage.test_dentry_first_preflush"]}]=0 v308) ==== 141+0+0 (secure 0 0 0) 0x7fecb0014030 con 0x7fecb80c0300 2024-09-21T03:39:54.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:54.833+0000 7fecbeabd640 1 -- 172.21.15.38:0/2631874821 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fec9c078b90 msgr2=0x7fec9c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:54.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:54.833+0000 7fecbeabd640 1 --2- 172.21.15.38:0/2631874821 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fec9c078b90 0x7fec9c07b0a0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fecac034000 tx=0x7fecac004730 comp rx=0 tx=0).stop 2024-09-21T03:39:54.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:54.833+0000 7fecbeabd640 1 -- 172.21.15.38:0/2631874821 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fecb80c0300 msgr2=0x7fecb81f66d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:54.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:54.833+0000 7fecbeabd640 1 --2- 172.21.15.38:0/2631874821 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fecb80c0300 0x7fecb81f66d0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fecb000ca90 tx=0x7fecb000cf50 comp rx=0 tx=0).stop 2024-09-21T03:39:54.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:54.834+0000 7fecbeabd640 1 -- 172.21.15.38:0/2631874821 shutdown_connections 2024-09-21T03:39:54.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:54.834+0000 7fecbeabd640 1 --2- 172.21.15.38:0/2631874821 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fecb80c0300 0x7fecb81f66d0 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:54.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:54.834+0000 7fecbeabd640 1 --2- 172.21.15.38:0/2631874821 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fec9c078b90 0x7fec9c07b0a0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:54.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:54.834+0000 7fecbeabd640 1 --2- 172.21.15.38:0/2631874821 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fecb81536e0 0x7fecb814bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:54.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:54.834+0000 7fecbeabd640 1 --2- 172.21.15.38:0/2631874821 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fecb80d1d00 0x7fecb814b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:54.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:54.834+0000 7fecbeabd640 1 -- 172.21.15.38:0/2631874821 >> 172.21.15.38:0/2631874821 conn(0x7fecb80cef40 msgr2=0x7fecb80cf350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:54.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:54.834+0000 7fecbeabd640 1 -- 172.21.15.38:0/2631874821 shutdown_connections 2024-09-21T03:39:54.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:54.834+0000 7fecbeabd640 1 -- 172.21.15.38:0/2631874821 wait complete. 2024-09-21T03:39:54.848 INFO:tasks.cephfs_test_runner:That the MDS won't write a dentry with new damage to CDentry::first ... ok 2024-09-21T03:39:54.848 INFO:tasks.cephfs_test_runner:Starting test: test_object_deletion (tasks.cephfs.test_damage.TestDamage) That the MDS has a clean 'damaged' response to loss of any single metadata object 2024-09-21T03:39:54.848 INFO:tasks.cephfs_test_runner:test_object_deletion (tasks.cephfs.test_damage.TestDamage) 2024-09-21T03:39:54.848 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T03:39:54.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:54.963+0000 7f3033fa1640 1 -- 172.21.15.38:0/2654332262 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f302c0d12e0 msgr2=0x7f302c0d16f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:54.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:54.963+0000 7f3033fa1640 1 --2- 172.21.15.38:0/2654332262 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f302c0d12e0 0x7f302c0d16f0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f3020009830 tx=0x7f302002a510 comp rx=0 tx=0).stop 2024-09-21T03:39:54.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:54.963+0000 7f3033fa1640 1 -- 172.21.15.38:0/2654332262 shutdown_connections 2024-09-21T03:39:54.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:54.963+0000 7f3033fa1640 1 --2- 172.21.15.38:0/2654332262 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f302c0d12e0 0x7f302c0d16f0 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:54.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:54.963+0000 7f3033fa1640 1 --2- 172.21.15.38:0/2654332262 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f302c15cb10 0x7f302c15cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:54.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:54.963+0000 7f3033fa1640 1 -- 172.21.15.38:0/2654332262 >> 172.21.15.38:0/2654332262 conn(0x7f302c0d2350 msgr2=0x7f302c0d2760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:54.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:54.963+0000 7f3033fa1640 1 -- 172.21.15.38:0/2654332262 shutdown_connections 2024-09-21T03:39:54.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:54.963+0000 7f3033fa1640 1 -- 172.21.15.38:0/2654332262 wait complete. 2024-09-21T03:39:54.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:54.965+0000 7f3033fa1640 1 Processor -- start 2024-09-21T03:39:54.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:54.965+0000 7f3033fa1640 1 -- start start 2024-09-21T03:39:54.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:54.966+0000 7f3033fa1640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f302c0d12e0 0x7f302c151dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:54.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:54.966+0000 7f3033fa1640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f302c15cb10 0x7f302c158c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:54.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:54.966+0000 7f3033fa1640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f302c160100 0x7f302c159180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:54.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:54.966+0000 7f3033fa1640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f302c0b73e0 con 0x7f302c160100 2024-09-21T03:39:54.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:54.966+0000 7f3033fa1640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f302c0b7260 con 0x7f302c15cb10 2024-09-21T03:39:54.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:54.966+0000 7f3033fa1640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f302c160c20 con 0x7f302c0d12e0 2024-09-21T03:39:54.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:54.967+0000 7f303253e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f302c160100 0x7f302c159180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:54.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:54.967+0000 7f303253e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f302c160100 0x7f302c159180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:45232/0 (socket says 172.21.15.38:45232) 2024-09-21T03:39:54.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:54.967+0000 7f303253e640 1 -- 172.21.15.38:0/1518488950 learned_addr learned my addr 172.21.15.38:0/1518488950 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:39:54.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:54.967+0000 7f3031d3d640 1 --2- 172.21.15.38:0/1518488950 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f302c0d12e0 0x7f302c151dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:54.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:54.967+0000 7f303153c640 1 --2- 172.21.15.38:0/1518488950 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f302c15cb10 0x7f302c158c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:54.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:54.967+0000 7f303253e640 1 -- 172.21.15.38:0/1518488950 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f302c0d12e0 msgr2=0x7f302c151dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:54.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:54.967+0000 7f303253e640 1 --2- 172.21.15.38:0/1518488950 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f302c0d12e0 0x7f302c151dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:54.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:54.967+0000 7f303253e640 1 -- 172.21.15.38:0/1518488950 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f302c15cb10 msgr2=0x7f302c158c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:54.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:54.967+0000 7f303253e640 1 --2- 172.21.15.38:0/1518488950 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f302c15cb10 0x7f302c158c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:54.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:54.967+0000 7f303253e640 1 -- 172.21.15.38:0/1518488950 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f302c0ca750 con 0x7f302c160100 2024-09-21T03:39:54.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:54.967+0000 7f303153c640 1 --2- 172.21.15.38:0/1518488950 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f302c15cb10 0x7f302c158c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:39:54.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:54.967+0000 7f303253e640 1 --2- 172.21.15.38:0/1518488950 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f302c160100 0x7f302c159180 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f301c00c760 tx=0x7f301c00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:54.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:54.967+0000 7f302a7fc640 1 -- 172.21.15.38:0/1518488950 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f301c017070 con 0x7f302c160100 2024-09-21T03:39:54.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:54.967+0000 7f302a7fc640 1 -- 172.21.15.38:0/1518488950 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f301c00ce10 con 0x7f302c160100 2024-09-21T03:39:54.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:54.967+0000 7f3033fa1640 1 -- 172.21.15.38:0/1518488950 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f302c0c7170 con 0x7f302c160100 2024-09-21T03:39:54.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:54.967+0000 7f302a7fc640 1 -- 172.21.15.38:0/1518488950 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f301c0052b0 con 0x7f302c160100 2024-09-21T03:39:54.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:54.967+0000 7f3033fa1640 1 -- 172.21.15.38:0/1518488950 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f302c0c7710 con 0x7f302c160100 2024-09-21T03:39:54.972 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:54.968+0000 7f3033fa1640 1 -- 172.21.15.38:0/1518488950 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f302c0ca920 con 0x7f302c160100 2024-09-21T03:39:54.972 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:54.969+0000 7f302a7fc640 1 -- 172.21.15.38:0/1518488950 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f301c01a070 con 0x7f302c160100 2024-09-21T03:39:54.972 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:54.970+0000 7f302a7fc640 1 --2- 172.21.15.38:0/1518488950 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3018078b90 0x7f301807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:54.972 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:54.970+0000 7f302a7fc640 1 -- 172.21.15.38:0/1518488950 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5421+0+0 (secure 0 0 0) 0x7f301c09a140 con 0x7f302c160100 2024-09-21T03:39:54.972 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:54.970+0000 7f3031d3d640 1 --2- 172.21.15.38:0/1518488950 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3018078b90 0x7f301807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:54.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:54.971+0000 7f3031d3d640 1 --2- 172.21.15.38:0/1518488950 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3018078b90 0x7f301807b0a0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f302400bb50 tx=0x7f302400b2e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:54.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:54.975+0000 7f302a7fc640 1 -- 172.21.15.38:0/1518488950 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f301c0620e0 con 0x7f302c160100 2024-09-21T03:39:55.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:55.163+0000 7f3033fa1640 1 -- 172.21.15.38:0/1518488950 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f302c0c7e90 con 0x7f302c160100 2024-09-21T03:39:55.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:55.164+0000 7f302a7fc640 1 -- 172.21.15.38:0/1518488950 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+13462 (secure 0 0 0) 0x7f301c066b10 con 0x7f302c160100 2024-09-21T03:39:55.167 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:39:55.167 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":86,"fsid":"9d50d66f-656b-4fe2-8e3b-fe53b79c4d43","created":"2024-09-21T03:33:10.677865+0000","modified":"2024-09-21T03:39:53.825030+0000","last_up_change":"2024-09-21T03:33:24.023421+0000","last_in_change":"2024-09-21T03:33:14.296193+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T03:33:22.992573+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dbab183f-7e63-48b8-960a-ae55fd5bf565","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6801","nonce":2381639433}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6803","nonce":2381639433}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6807","nonce":2381639433}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6805","nonce":2381639433}]},"public_addr":"172.21.15.38:6801/2381639433","cluster_addr":"172.21.15.38:6803/2381639433","heartbeat_back_addr":"172.21.15.38:6807/2381639433","heartbeat_front_addr":"172.21.15.38:6805/2381639433","state":["exists","up"]},{"osd":1,"uuid":"fa1c0e8a-ebc1-4359-8146-bd5342abbc16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6817","nonce":1491818732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6819","nonce":1491818732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6823","nonce":1491818732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6821","nonce":1491818732}]},"public_addr":"172.21.15.38:6817/1491818732","cluster_addr":"172.21.15.38:6819/1491818732","heartbeat_back_addr":"172.21.15.38:6823/1491818732","heartbeat_front_addr":"172.21.15.38:6821/1491818732","state":["exists","up"]},{"osd":2,"uuid":"2ed3cd2d-385d-4fc6-9869-214552a84803","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6825","nonce":2577858902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6827","nonce":2577858902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6831","nonce":2577858902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6829","nonce":2577858902}]},"public_addr":"172.21.15.38:6825/2577858902","cluster_addr":"172.21.15.38:6827/2577858902","heartbeat_back_addr":"172.21.15.38:6831/2577858902","heartbeat_front_addr":"172.21.15.38:6829/2577858902","state":["exists","up"]},{"osd":3,"uuid":"62d1d95e-919d-4250-be71-1d34cfa70dbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6809","nonce":1286810294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6811","nonce":1286810294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6815","nonce":1286810294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6813","nonce":1286810294}]},"public_addr":"172.21.15.38:6809/1286810294","cluster_addr":"172.21.15.38:6811/1286810294","heartbeat_back_addr":"172.21.15.38:6815/1286810294","heartbeat_front_addr":"172.21.15.38:6813/1286810294","state":["exists","up"]},{"osd":4,"uuid":"7c9f9314-99d7-4d73-8392-37809a9f54b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6803","nonce":3401837571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6805","nonce":3401837571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6809","nonce":3401837571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6807","nonce":3401837571}]},"public_addr":"172.21.15.195:6803/3401837571","cluster_addr":"172.21.15.195:6805/3401837571","heartbeat_back_addr":"172.21.15.195:6809/3401837571","heartbeat_front_addr":"172.21.15.195:6807/3401837571","state":["exists","up"]},{"osd":5,"uuid":"bde4fe9f-8414-4bba-bf4a-43973d1976bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6811","nonce":2080960180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6813","nonce":2080960180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6817","nonce":2080960180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6815","nonce":2080960180}]},"public_addr":"172.21.15.195:6811/2080960180","cluster_addr":"172.21.15.195:6813/2080960180","heartbeat_back_addr":"172.21.15.195:6817/2080960180","heartbeat_front_addr":"172.21.15.195:6815/2080960180","state":["exists","up"]},{"osd":6,"uuid":"6ee6d7f4-1a5f-4a4c-ac16-48fe05f6a002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6827","nonce":4055825675}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6829","nonce":4055825675}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6833","nonce":4055825675}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6831","nonce":4055825675}]},"public_addr":"172.21.15.195:6827/4055825675","cluster_addr":"172.21.15.195:6829/4055825675","heartbeat_back_addr":"172.21.15.195:6833/4055825675","heartbeat_front_addr":"172.21.15.195:6831/4055825675","state":["exists","up"]},{"osd":7,"uuid":"efeb5f51-329a-4484-8290-92e7d3503d05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6819","nonce":1606106342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6821","nonce":1606106342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6825","nonce":1606106342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6823","nonce":1606106342}]},"public_addr":"172.21.15.195:6819/1606106342","cluster_addr":"172.21.15.195:6821/1606106342","heartbeat_back_addr":"172.21.15.195:6825/1606106342","heartbeat_front_addr":"172.21.15.195:6823/1606106342","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.901343+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:21.115003+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.901623+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.835238+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.132199+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:19.817560+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.829690+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:21.087300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.195:6834/2771781100":"2024-09-22T03:39:50.807095+0000","172.21.15.195:6836/2771781100":"2024-09-22T03:39:50.807095+0000","172.21.15.38:6833/1144075315":"2024-09-22T03:39:42.556819+0000","172.21.15.38:6832/1144075315":"2024-09-22T03:39:42.556819+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T03:39:55.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:55.167+0000 7f3033fa1640 1 -- 172.21.15.38:0/1518488950 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3018078b90 msgr2=0x7f301807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:55.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:55.167+0000 7f3033fa1640 1 --2- 172.21.15.38:0/1518488950 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3018078b90 0x7f301807b0a0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f302400bb50 tx=0x7f302400b2e0 comp rx=0 tx=0).stop 2024-09-21T03:39:55.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:55.167+0000 7f3033fa1640 1 -- 172.21.15.38:0/1518488950 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f302c160100 msgr2=0x7f302c159180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:55.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:55.167+0000 7f3033fa1640 1 --2- 172.21.15.38:0/1518488950 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f302c160100 0x7f302c159180 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f301c00c760 tx=0x7f301c00cc20 comp rx=0 tx=0).stop 2024-09-21T03:39:55.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:55.167+0000 7f3033fa1640 1 -- 172.21.15.38:0/1518488950 shutdown_connections 2024-09-21T03:39:55.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:55.167+0000 7f3033fa1640 1 --2- 172.21.15.38:0/1518488950 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f302c160100 0x7f302c159180 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:55.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:55.167+0000 7f3033fa1640 1 --2- 172.21.15.38:0/1518488950 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f302c15cb10 0x7f302c158c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:55.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:55.167+0000 7f3033fa1640 1 --2- 172.21.15.38:0/1518488950 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3018078b90 0x7f301807b0a0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:55.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:55.167+0000 7f3033fa1640 1 --2- 172.21.15.38:0/1518488950 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f302c0d12e0 0x7f302c151dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:55.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:55.167+0000 7f3033fa1640 1 -- 172.21.15.38:0/1518488950 >> 172.21.15.38:0/1518488950 conn(0x7f302c0d2350 msgr2=0x7f302c15e190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:55.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:55.167+0000 7f3033fa1640 1 -- 172.21.15.38:0/1518488950 shutdown_connections 2024-09-21T03:39:55.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:55.167+0000 7f3033fa1640 1 -- 172.21.15.38:0/1518488950 wait complete. 2024-09-21T03:39:55.181 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T03:33:22.992573+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-21T03:39:55.181 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-21T03:39:55.300 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:55.298+0000 7f8d01ef9640 1 -- 172.21.15.38:0/2336713679 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8cfc0ceb90 msgr2=0x7f8cfc0cefa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:55.301 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:55.298+0000 7f8d01ef9640 1 --2- 172.21.15.38:0/2336713679 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8cfc0ceb90 0x7f8cfc0cefa0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f8cfc0cfba0 tx=0x7f8cf402a9a0 comp rx=0 tx=0).stop 2024-09-21T03:39:55.301 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:55.298+0000 7f8d01ef9640 1 -- 172.21.15.38:0/2336713679 shutdown_connections 2024-09-21T03:39:55.301 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:55.298+0000 7f8d01ef9640 1 --2- 172.21.15.38:0/2336713679 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8cfc0ceb90 0x7f8cfc0cefa0 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:55.301 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:55.298+0000 7f8d01ef9640 1 --2- 172.21.15.38:0/2336713679 >> v2:172.21.15.195:3300/0 conn(0x7f8cfc15ee80 0x7f8cfc0d0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:55.301 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:55.298+0000 7f8d01ef9640 1 -- 172.21.15.38:0/2336713679 >> 172.21.15.38:0/2336713679 conn(0x7f8cfc09d830 msgr2=0x7f8cfc0b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:55.301 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:55.298+0000 7f8d01ef9640 1 -- 172.21.15.38:0/2336713679 shutdown_connections 2024-09-21T03:39:55.301 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:55.298+0000 7f8d01ef9640 1 -- 172.21.15.38:0/2336713679 wait complete. 2024-09-21T03:39:55.302 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:55.300+0000 7f8d01ef9640 1 Processor -- start 2024-09-21T03:39:55.302 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:55.300+0000 7f8d01ef9640 1 -- start start 2024-09-21T03:39:55.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:55.301+0000 7f8d01ef9640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8cfc0b9ad0 0x7f8cfc0cb610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:55.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:55.301+0000 7f8d01ef9640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8cfc0ceb90 0x7f8cfc1f9a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:55.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:55.301+0000 7f8d01ef9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8cfc15ee80 0x7f8cfc14fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:55.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:55.301+0000 7f8d01ef9640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f8cfc0b6700 con 0x7f8cfc15ee80 2024-09-21T03:39:55.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:55.301+0000 7f8d01ef9640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f8cfc0b6580 con 0x7f8cfc0ceb90 2024-09-21T03:39:55.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:55.301+0000 7f8d01ef9640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f8cfc0b6880 con 0x7f8cfc0b9ad0 2024-09-21T03:39:55.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:55.301+0000 7f8cfbfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8cfc15ee80 0x7f8cfc14fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:55.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:55.302+0000 7f8cfbfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8cfc15ee80 0x7f8cfc14fa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:45250/0 (socket says 172.21.15.38:45250) 2024-09-21T03:39:55.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:55.302+0000 7f8cfbfff640 1 -- 172.21.15.38:0/2167024020 learned_addr learned my addr 172.21.15.38:0/2167024020 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:39:55.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:55.302+0000 7f8cfaffd640 1 --2- 172.21.15.38:0/2167024020 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8cfc0ceb90 0x7f8cfc1f9a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:55.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:55.302+0000 7f8cfb7fe640 1 --2- 172.21.15.38:0/2167024020 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8cfc0b9ad0 0x7f8cfc0cb610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:55.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:55.302+0000 7f8cfbfff640 1 -- 172.21.15.38:0/2167024020 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8cfc0b9ad0 msgr2=0x7f8cfc0cb610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:55.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:55.302+0000 7f8cfbfff640 1 --2- 172.21.15.38:0/2167024020 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8cfc0b9ad0 0x7f8cfc0cb610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:55.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:55.302+0000 7f8cfbfff640 1 -- 172.21.15.38:0/2167024020 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8cfc0ceb90 msgr2=0x7f8cfc1f9a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:55.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:55.302+0000 7f8cfbfff640 1 --2- 172.21.15.38:0/2167024020 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8cfc0ceb90 0x7f8cfc1f9a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:55.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:55.302+0000 7f8cfbfff640 1 -- 172.21.15.38:0/2167024020 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cfc1f6120 con 0x7f8cfc15ee80 2024-09-21T03:39:55.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:55.302+0000 7f8cfbfff640 1 --2- 172.21.15.38:0/2167024020 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8cfc15ee80 0x7f8cfc14fa90 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f8cf4005fd0 tx=0x7f8cf4003f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:55.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:55.302+0000 7f8ce3fff640 1 -- 172.21.15.38:0/2167024020 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cf4033050 con 0x7f8cfc15ee80 2024-09-21T03:39:55.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:55.302+0000 7f8ce3fff640 1 -- 172.21.15.38:0/2167024020 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8cf40044d0 con 0x7f8cfc15ee80 2024-09-21T03:39:55.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:55.302+0000 7f8d01ef9640 1 -- 172.21.15.38:0/2167024020 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cfc1f6400 con 0x7f8cfc15ee80 2024-09-21T03:39:55.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:55.302+0000 7f8ce3fff640 1 -- 172.21.15.38:0/2167024020 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cf4038450 con 0x7f8cfc15ee80 2024-09-21T03:39:55.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:55.302+0000 7f8d01ef9640 1 -- 172.21.15.38:0/2167024020 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cfc14ffd0 con 0x7f8cfc15ee80 2024-09-21T03:39:55.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:55.303+0000 7f8d01ef9640 1 -- 172.21.15.38:0/2167024020 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8cfc1f5e90 con 0x7f8cfc15ee80 2024-09-21T03:39:55.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:55.304+0000 7f8ce3fff640 1 -- 172.21.15.38:0/2167024020 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f8cf4036070 con 0x7f8cfc15ee80 2024-09-21T03:39:55.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:55.305+0000 7f8ce3fff640 1 --2- 172.21.15.38:0/2167024020 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8ce8078b90 0x7f8ce807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:55.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:55.305+0000 7f8ce3fff640 1 -- 172.21.15.38:0/2167024020 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5421+0+0 (secure 0 0 0) 0x7f8cf40bbdc0 con 0x7f8cfc15ee80 2024-09-21T03:39:55.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:55.305+0000 7f8cfb7fe640 1 --2- 172.21.15.38:0/2167024020 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8ce8078b90 0x7f8ce807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:55.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:55.306+0000 7f8cfb7fe640 1 --2- 172.21.15.38:0/2167024020 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8ce8078b90 0x7f8ce807b0a0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f8cec0058c0 tx=0x7f8cec005f60 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:55.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:55.310+0000 7f8ce3fff640 1 -- 172.21.15.38:0/2167024020 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f8cf4083d50 con 0x7f8cfc15ee80 2024-09-21T03:39:55.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:55.497+0000 7f8d01ef9640 1 -- 172.21.15.38:0/2167024020 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f8cfc0d4c90 con 0x7f8cfc15ee80 2024-09-21T03:39:55.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:55.497+0000 7f8ce3fff640 1 -- 172.21.15.38:0/2167024020 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v86) ==== 93+0+10 (secure 0 0 0) 0x7f8cf4088780 con 0x7f8cfc15ee80 2024-09-21T03:39:55.499 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 1 2024-09-21T03:39:55.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:55.500+0000 7f8d01ef9640 1 -- 172.21.15.38:0/2167024020 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8ce8078b90 msgr2=0x7f8ce807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:55.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:55.500+0000 7f8d01ef9640 1 --2- 172.21.15.38:0/2167024020 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8ce8078b90 0x7f8ce807b0a0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f8cec0058c0 tx=0x7f8cec005f60 comp rx=0 tx=0).stop 2024-09-21T03:39:55.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:55.500+0000 7f8d01ef9640 1 -- 172.21.15.38:0/2167024020 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8cfc15ee80 msgr2=0x7f8cfc14fa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:55.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:55.500+0000 7f8d01ef9640 1 --2- 172.21.15.38:0/2167024020 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8cfc15ee80 0x7f8cfc14fa90 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f8cf4005fd0 tx=0x7f8cf4003f70 comp rx=0 tx=0).stop 2024-09-21T03:39:55.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:55.500+0000 7f8d01ef9640 1 -- 172.21.15.38:0/2167024020 shutdown_connections 2024-09-21T03:39:55.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:55.500+0000 7f8d01ef9640 1 --2- 172.21.15.38:0/2167024020 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8cfc15ee80 0x7f8cfc14fa90 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:55.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:55.500+0000 7f8d01ef9640 1 --2- 172.21.15.38:0/2167024020 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8cfc0ceb90 0x7f8cfc1f9a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:55.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:55.500+0000 7f8d01ef9640 1 --2- 172.21.15.38:0/2167024020 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8ce8078b90 0x7f8ce807b0a0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:55.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:55.500+0000 7f8d01ef9640 1 --2- 172.21.15.38:0/2167024020 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8cfc0b9ad0 0x7f8cfc0cb610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:55.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:55.500+0000 7f8d01ef9640 1 -- 172.21.15.38:0/2167024020 >> 172.21.15.38:0/2167024020 conn(0x7f8cfc09d830 msgr2=0x7f8cfc15d400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:55.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:55.500+0000 7f8d01ef9640 1 -- 172.21.15.38:0/2167024020 shutdown_connections 2024-09-21T03:39:55.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:55.500+0000 7f8d01ef9640 1 -- 172.21.15.38:0/2167024020 wait complete. 2024-09-21T03:39:55.513 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_damage.TestDamage.test_object_deletion' 2024-09-21T03:39:55.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:55.628+0000 7f81a6328640 1 -- 172.21.15.38:0/2753071008 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f81a0162f00 msgr2=0x7f81a00d24d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:55.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:55.628+0000 7f81a6328640 1 --2- 172.21.15.38:0/2753071008 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f81a0162f00 0x7f81a00d24d0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f81a00d0c00 tx=0x7f819002a4a0 comp rx=0 tx=0).stop 2024-09-21T03:39:55.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:55.628+0000 7f81a6328640 1 -- 172.21.15.38:0/2753071008 shutdown_connections 2024-09-21T03:39:55.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:55.628+0000 7f81a6328640 1 --2- 172.21.15.38:0/2753071008 >> v2:172.21.15.195:3300/0 conn(0x7f81a00b2e40 0x7f81a00b3250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:55.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:55.628+0000 7f81a6328640 1 --2- 172.21.15.38:0/2753071008 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f81a0162f00 0x7f81a00d24d0 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:55.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:55.628+0000 7f81a6328640 1 -- 172.21.15.38:0/2753071008 >> 172.21.15.38:0/2753071008 conn(0x7f81a00b6b20 msgr2=0x7f81a00d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:55.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:55.628+0000 7f81a6328640 1 -- 172.21.15.38:0/2753071008 shutdown_connections 2024-09-21T03:39:55.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:55.628+0000 7f81a6328640 1 -- 172.21.15.38:0/2753071008 wait complete. 2024-09-21T03:39:55.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:55.630+0000 7f81a6328640 1 Processor -- start 2024-09-21T03:39:55.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:55.630+0000 7f81a6328640 1 -- start start 2024-09-21T03:39:55.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:55.631+0000 7f81a6328640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f81a00b2e40 0x7f81a01607d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:55.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:55.631+0000 7f81a6328640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f81a0162280 0x7f81a00b26c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:55.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:55.631+0000 7f81a6328640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f81a01f5100 0x7f81a015b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:55.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:55.631+0000 7f81a6328640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f81a015e4b0 con 0x7f81a0162280 2024-09-21T03:39:55.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:55.631+0000 7f81a6328640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f81a00b39a0 con 0x7f81a00b2e40 2024-09-21T03:39:55.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:55.631+0000 7f81a6328640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f81a015e630 con 0x7f81a01f5100 2024-09-21T03:39:55.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:55.631+0000 7f819f7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f81a0162280 0x7f81a00b26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:55.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:55.631+0000 7f819f7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f81a0162280 0x7f81a00b26c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:45280/0 (socket says 172.21.15.38:45280) 2024-09-21T03:39:55.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:55.631+0000 7f819f7fe640 1 -- 172.21.15.38:0/1008440454 learned_addr learned my addr 172.21.15.38:0/1008440454 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:39:55.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:55.632+0000 7f81a48c5640 1 --2- 172.21.15.38:0/1008440454 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f81a01f5100 0x7f81a015b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:55.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:55.632+0000 7f819ffff640 1 --2- 172.21.15.38:0/1008440454 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f81a00b2e40 0x7f81a01607d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:55.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:55.632+0000 7f819f7fe640 1 -- 172.21.15.38:0/1008440454 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f81a01f5100 msgr2=0x7f81a015b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:55.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:55.632+0000 7f819f7fe640 1 --2- 172.21.15.38:0/1008440454 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f81a01f5100 0x7f81a015b920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:55.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:55.632+0000 7f819f7fe640 1 -- 172.21.15.38:0/1008440454 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f81a00b2e40 msgr2=0x7f81a01607d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:55.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:55.632+0000 7f819f7fe640 1 --2- 172.21.15.38:0/1008440454 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f81a00b2e40 0x7f81a01607d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:55.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:55.632+0000 7f819f7fe640 1 -- 172.21.15.38:0/1008440454 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81a0160d10 con 0x7f81a0162280 2024-09-21T03:39:55.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:55.632+0000 7f81a48c5640 1 --2- 172.21.15.38:0/1008440454 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f81a01f5100 0x7f81a015b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:39:55.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:55.632+0000 7f819ffff640 1 --2- 172.21.15.38:0/1008440454 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f81a00b2e40 0x7f81a01607d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:39:55.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:55.632+0000 7f819f7fe640 1 --2- 172.21.15.38:0/1008440454 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f81a0162280 0x7f81a00b26c0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f819400c3c0 tx=0x7f819400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:55.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:55.632+0000 7f819cff9640 1 -- 172.21.15.38:0/1008440454 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8194017070 con 0x7f81a0162280 2024-09-21T03:39:55.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:55.632+0000 7f819cff9640 1 -- 172.21.15.38:0/1008440454 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f81940035f0 con 0x7f81a0162280 2024-09-21T03:39:55.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:55.632+0000 7f81a6328640 1 -- 172.21.15.38:0/1008440454 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81a01f5bf0 con 0x7f81a0162280 2024-09-21T03:39:55.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:55.632+0000 7f819cff9640 1 -- 172.21.15.38:0/1008440454 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8194003a40 con 0x7f81a0162280 2024-09-21T03:39:55.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:55.632+0000 7f81a6328640 1 -- 172.21.15.38:0/1008440454 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81a015ffd0 con 0x7f81a0162280 2024-09-21T03:39:55.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:55.633+0000 7f81a6328640 1 -- 172.21.15.38:0/1008440454 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81a015fdc0 con 0x7f81a0162280 2024-09-21T03:39:55.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:55.634+0000 7f819cff9640 1 -- 172.21.15.38:0/1008440454 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f819401a070 con 0x7f81a0162280 2024-09-21T03:39:55.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:55.634+0000 7f819cff9640 1 --2- 172.21.15.38:0/1008440454 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f818c078b90 0x7f818c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:55.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:55.635+0000 7f819cff9640 1 -- 172.21.15.38:0/1008440454 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5421+0+0 (secure 0 0 0) 0x7f8194099e80 con 0x7f81a0162280 2024-09-21T03:39:55.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:55.635+0000 7f819ffff640 1 --2- 172.21.15.38:0/1008440454 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f818c078b90 0x7f818c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:55.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:55.635+0000 7f819ffff640 1 --2- 172.21.15.38:0/1008440454 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f818c078b90 0x7f818c07b0a0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f8190034000 tx=0x7f8190004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:55.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:55.639+0000 7f819cff9640 1 -- 172.21.15.38:0/1008440454 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f8194062470 con 0x7f81a0162280 2024-09-21T03:39:55.906 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:55.904+0000 7f81a6328640 1 -- 172.21.15.38:0/1008440454 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_damage.TestDamage.test_object_deletion"]} v 0) -- 0x7f81a0162f00 con 0x7f81a0162280 2024-09-21T03:39:56.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:56.839+0000 7f819cff9640 1 -- 172.21.15.38:0/1008440454 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_damage.TestDamage.test_object_deletion"]}]=0 v310) ==== 138+0+0 (secure 0 0 0) 0x7f8194061d70 con 0x7f81a0162280 2024-09-21T03:39:56.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:56.842+0000 7f81a6328640 1 -- 172.21.15.38:0/1008440454 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f818c078b90 msgr2=0x7f818c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:56.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:56.842+0000 7f81a6328640 1 --2- 172.21.15.38:0/1008440454 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f818c078b90 0x7f818c07b0a0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f8190034000 tx=0x7f8190004730 comp rx=0 tx=0).stop 2024-09-21T03:39:56.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:56.842+0000 7f81a6328640 1 -- 172.21.15.38:0/1008440454 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f81a0162280 msgr2=0x7f81a00b26c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:56.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:56.842+0000 7f81a6328640 1 --2- 172.21.15.38:0/1008440454 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f81a0162280 0x7f81a00b26c0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f819400c3c0 tx=0x7f819400c880 comp rx=0 tx=0).stop 2024-09-21T03:39:56.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:56.842+0000 7f81a6328640 1 -- 172.21.15.38:0/1008440454 shutdown_connections 2024-09-21T03:39:56.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:56.842+0000 7f81a6328640 1 --2- 172.21.15.38:0/1008440454 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f818c078b90 0x7f818c07b0a0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:56.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:56.842+0000 7f81a6328640 1 --2- 172.21.15.38:0/1008440454 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f81a01f5100 0x7f81a015b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:56.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:56.842+0000 7f81a6328640 1 --2- 172.21.15.38:0/1008440454 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f81a0162280 0x7f81a00b26c0 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:56.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:56.842+0000 7f81a6328640 1 --2- 172.21.15.38:0/1008440454 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f81a00b2e40 0x7f81a01607d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:56.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:56.842+0000 7f81a6328640 1 -- 172.21.15.38:0/1008440454 >> 172.21.15.38:0/1008440454 conn(0x7f81a00b6b20 msgr2=0x7f81a006e0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:56.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:56.842+0000 7f81a6328640 1 -- 172.21.15.38:0/1008440454 shutdown_connections 2024-09-21T03:39:56.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:56.842+0000 7f81a6328640 1 -- 172.21.15.38:0/1008440454 wait complete. 2024-09-21T03:39:56.858 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-21T03:39:56.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:56.980+0000 7f8e7d2e5640 1 -- 172.21.15.38:0/3766018011 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8e7806e940 msgr2=0x7f8e7806ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:56.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:56.980+0000 7f8e7d2e5640 1 --2- 172.21.15.38:0/3766018011 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8e7806e940 0x7f8e7806ed50 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f8e780ce420 tx=0x7f8e6002a4a0 comp rx=0 tx=0).stop 2024-09-21T03:39:56.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:56.980+0000 7f8e7d2e5640 1 -- 172.21.15.38:0/3766018011 shutdown_connections 2024-09-21T03:39:56.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:56.980+0000 7f8e7d2e5640 1 --2- 172.21.15.38:0/3766018011 >> v2:172.21.15.195:3300/0 conn(0x7f8e7806f290 0x7f8e780b40c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:56.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:56.980+0000 7f8e7d2e5640 1 --2- 172.21.15.38:0/3766018011 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8e7806e940 0x7f8e7806ed50 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:56.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:56.980+0000 7f8e7d2e5640 1 -- 172.21.15.38:0/3766018011 >> 172.21.15.38:0/3766018011 conn(0x7f8e78073e30 msgr2=0x7f8e78074240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:56.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:56.980+0000 7f8e7d2e5640 1 -- 172.21.15.38:0/3766018011 shutdown_connections 2024-09-21T03:39:56.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:56.980+0000 7f8e7d2e5640 1 -- 172.21.15.38:0/3766018011 wait complete. 2024-09-21T03:39:56.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:56.982+0000 7f8e7d2e5640 1 Processor -- start 2024-09-21T03:39:56.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:56.982+0000 7f8e7d2e5640 1 -- start start 2024-09-21T03:39:56.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:56.984+0000 7f8e7d2e5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8e7806e940 0x7f8e780aff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:56.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:56.984+0000 7f8e7d2e5640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8e7806f290 0x7f8e78149400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:56.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:56.984+0000 7f8e7d2e5640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8e780cee70 0x7f8e78147370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:56.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:56.984+0000 7f8e7d2e5640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f8e780b4990 con 0x7f8e7806e940 2024-09-21T03:39:56.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:56.984+0000 7f8e7d2e5640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f8e780b4810 con 0x7f8e7806f290 2024-09-21T03:39:56.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:56.984+0000 7f8e7d2e5640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f8e780b4b10 con 0x7f8e780cee70 2024-09-21T03:39:56.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:56.984+0000 7f8e76ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8e7806e940 0x7f8e780aff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:56.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:56.984+0000 7f8e76ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8e7806e940 0x7f8e780aff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:45298/0 (socket says 172.21.15.38:45298) 2024-09-21T03:39:56.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:56.984+0000 7f8e76ffd640 1 -- 172.21.15.38:0/702521835 learned_addr learned my addr 172.21.15.38:0/702521835 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:39:56.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:56.984+0000 7f8e767fc640 1 --2- 172.21.15.38:0/702521835 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8e7806f290 0x7f8e78149400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:56.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:56.984+0000 7f8e777fe640 1 --2- 172.21.15.38:0/702521835 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8e780cee70 0x7f8e78147370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:56.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:56.984+0000 7f8e76ffd640 1 -- 172.21.15.38:0/702521835 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8e780cee70 msgr2=0x7f8e78147370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:56.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:56.984+0000 7f8e76ffd640 1 --2- 172.21.15.38:0/702521835 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8e780cee70 0x7f8e78147370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:56.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:56.984+0000 7f8e76ffd640 1 -- 172.21.15.38:0/702521835 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8e7806f290 msgr2=0x7f8e78149400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:56.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:56.984+0000 7f8e76ffd640 1 --2- 172.21.15.38:0/702521835 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8e7806f290 0x7f8e78149400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:56.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:56.984+0000 7f8e76ffd640 1 -- 172.21.15.38:0/702521835 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e7814a050 con 0x7f8e7806e940 2024-09-21T03:39:56.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:56.984+0000 7f8e76ffd640 1 --2- 172.21.15.38:0/702521835 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8e7806e940 0x7f8e780aff30 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f8e7000cb70 tx=0x7f8e7000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:56.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:56.985+0000 7f8e6f7fe640 1 -- 172.21.15.38:0/702521835 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e70016020 con 0x7f8e7806e940 2024-09-21T03:39:56.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:56.985+0000 7f8e6f7fe640 1 -- 172.21.15.38:0/702521835 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8e7000ae50 con 0x7f8e7806e940 2024-09-21T03:39:56.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:56.985+0000 7f8e6f7fe640 1 -- 172.21.15.38:0/702521835 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e70012680 con 0x7f8e7806e940 2024-09-21T03:39:56.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:56.985+0000 7f8e7d2e5640 1 -- 172.21.15.38:0/702521835 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e780af450 con 0x7f8e7806e940 2024-09-21T03:39:56.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:56.985+0000 7f8e7d2e5640 1 -- 172.21.15.38:0/702521835 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e780afa10 con 0x7f8e7806e940 2024-09-21T03:39:56.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:56.986+0000 7f8e7d2e5640 1 -- 172.21.15.38:0/702521835 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e780b4fc0 con 0x7f8e7806e940 2024-09-21T03:39:56.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:56.987+0000 7f8e6f7fe640 1 -- 172.21.15.38:0/702521835 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f8e70019020 con 0x7f8e7806e940 2024-09-21T03:39:56.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:56.987+0000 7f8e6f7fe640 1 --2- 172.21.15.38:0/702521835 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8e58078b90 0x7f8e5807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:56.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:56.987+0000 7f8e6f7fe640 1 -- 172.21.15.38:0/702521835 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5421+0+0 (secure 0 0 0) 0x7f8e7009af80 con 0x7f8e7806e940 2024-09-21T03:39:56.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:56.987+0000 7f8e767fc640 1 --2- 172.21.15.38:0/702521835 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8e58078b90 0x7f8e5807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:56.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:56.988+0000 7f8e767fc640 1 --2- 172.21.15.38:0/702521835 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8e58078b90 0x7f8e5807b0a0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f8e60034000 tx=0x7f8e60004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:56.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:56.992+0000 7f8e6f7fe640 1 -- 172.21.15.38:0/702521835 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f8e70020030 con 0x7f8e7806e940 2024-09-21T03:39:57.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:57.181+0000 7f8e7d2e5640 1 -- 172.21.15.38:0/702521835 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f8e780b5850 con 0x7f8e7806e940 2024-09-21T03:39:57.183 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:57.181+0000 7f8e6f7fe640 1 -- 172.21.15.38:0/702521835 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v46) ==== 59+0+2294 (secure 0 0 0) 0x7f8e70063570 con 0x7f8e7806e940 2024-09-21T03:39:57.183 INFO:teuthology.orchestra.run.smithi038.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-21T03:39:57.183 INFO:teuthology.orchestra.run.smithi038.stdout:mon advanced mon_op_complaint_time 120 2024-09-21T03:39:57.183 INFO:teuthology.orchestra.run.smithi038.stdout:mgr advanced client_mount_timeout 30 2024-09-21T03:39:57.184 INFO:teuthology.orchestra.run.smithi038.stdout:mgr advanced debug_client 20/20 2024-09-21T03:39:57.184 INFO:teuthology.orchestra.run.smithi038.stdout:mgr advanced debug_mgr 20/20 2024-09-21T03:39:57.184 INFO:teuthology.orchestra.run.smithi038.stdout:mgr advanced debug_ms 1/1 2024-09-21T03:39:57.184 INFO:teuthology.orchestra.run.smithi038.stdout:mgr dev mon_warn_on_pool_no_app false 2024-09-21T03:39:57.184 INFO:teuthology.orchestra.run.smithi038.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-21T03:39:57.184 INFO:teuthology.orchestra.run.smithi038.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 3844.962174 2024-09-21T03:39:57.184 INFO:teuthology.orchestra.run.smithi038.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 4916.015278 2024-09-21T03:39:57.184 INFO:teuthology.orchestra.run.smithi038.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 4165.407418 2024-09-21T03:39:57.184 INFO:teuthology.orchestra.run.smithi038.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 3760.481604 2024-09-21T03:39:57.184 INFO:teuthology.orchestra.run.smithi038.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 6052.882314 2024-09-21T03:39:57.184 INFO:teuthology.orchestra.run.smithi038.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 8967.576248 2024-09-21T03:39:57.184 INFO:teuthology.orchestra.run.smithi038.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 6129.316416 2024-09-21T03:39:57.184 INFO:teuthology.orchestra.run.smithi038.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 8751.950197 2024-09-21T03:39:57.185 INFO:teuthology.orchestra.run.smithi038.stdout:mds advanced debug_mds 20/20 2024-09-21T03:39:57.185 INFO:teuthology.orchestra.run.smithi038.stdout:mds advanced debug_mds_balancer 20/20 2024-09-21T03:39:57.185 INFO:teuthology.orchestra.run.smithi038.stdout:mds advanced debug_ms 2/2 2024-09-21T03:39:57.185 INFO:teuthology.orchestra.run.smithi038.stdout:mds dev mds_debug_frag true 2024-09-21T03:39:57.185 INFO:teuthology.orchestra.run.smithi038.stdout:mds dev mds_debug_scatterstat true 2024-09-21T03:39:57.185 INFO:teuthology.orchestra.run.smithi038.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-21T03:39:57.185 INFO:teuthology.orchestra.run.smithi038.stdout:mds dev mds_verify_scatter true 2024-09-21T03:39:57.185 INFO:teuthology.orchestra.run.smithi038.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-21T03:39:57.185 INFO:teuthology.orchestra.run.smithi038.stdout:mds advanced rados_mon_op_timeout 900 2024-09-21T03:39:57.185 INFO:teuthology.orchestra.run.smithi038.stdout:mds advanced rados_osd_op_timeout 900 2024-09-21T03:39:57.185 INFO:teuthology.orchestra.run.smithi038.stdout:client advanced client_mount_timeout 600 2024-09-21T03:39:57.185 INFO:teuthology.orchestra.run.smithi038.stdout:client advanced debug_client 20/20 2024-09-21T03:39:57.185 INFO:teuthology.orchestra.run.smithi038.stdout:client advanced debug_ms 1/1 2024-09-21T03:39:57.185 INFO:teuthology.orchestra.run.smithi038.stdout:client advanced rados_mon_op_timeout 900 2024-09-21T03:39:57.186 INFO:teuthology.orchestra.run.smithi038.stdout:client advanced rados_osd_op_timeout 900 2024-09-21T03:39:57.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:57.183+0000 7f8e7d2e5640 1 -- 172.21.15.38:0/702521835 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8e58078b90 msgr2=0x7f8e5807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:57.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:57.183+0000 7f8e7d2e5640 1 --2- 172.21.15.38:0/702521835 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8e58078b90 0x7f8e5807b0a0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f8e60034000 tx=0x7f8e60004730 comp rx=0 tx=0).stop 2024-09-21T03:39:57.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:57.184+0000 7f8e7d2e5640 1 -- 172.21.15.38:0/702521835 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8e7806e940 msgr2=0x7f8e780aff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:57.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:57.184+0000 7f8e7d2e5640 1 --2- 172.21.15.38:0/702521835 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8e7806e940 0x7f8e780aff30 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f8e7000cb70 tx=0x7f8e7000a430 comp rx=0 tx=0).stop 2024-09-21T03:39:57.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:57.184+0000 7f8e7d2e5640 1 -- 172.21.15.38:0/702521835 shutdown_connections 2024-09-21T03:39:57.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:57.184+0000 7f8e7d2e5640 1 --2- 172.21.15.38:0/702521835 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8e58078b90 0x7f8e5807b0a0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:57.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:57.184+0000 7f8e7d2e5640 1 --2- 172.21.15.38:0/702521835 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8e780cee70 0x7f8e78147370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:57.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:57.184+0000 7f8e7d2e5640 1 --2- 172.21.15.38:0/702521835 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8e7806f290 0x7f8e78149400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:57.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:57.184+0000 7f8e7d2e5640 1 --2- 172.21.15.38:0/702521835 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8e7806e940 0x7f8e780aff30 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:57.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:57.184+0000 7f8e7d2e5640 1 -- 172.21.15.38:0/702521835 >> 172.21.15.38:0/702521835 conn(0x7f8e78073e30 msgr2=0x7f8e780d0530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:57.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:57.184+0000 7f8e7d2e5640 1 -- 172.21.15.38:0/702521835 shutdown_connections 2024-09-21T03:39:57.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:57.184+0000 7f8e7d2e5640 1 -- 172.21.15.38:0/702521835 wait complete. 2024-09-21T03:39:57.197 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-21T03:39:57.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:57.315+0000 7f2d2b7a1640 1 -- 172.21.15.38:0/2514564106 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2d10004b60 msgr2=0x7f2d10004fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:57.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:57.315+0000 7f2d2b7a1640 1 --2- 172.21.15.38:0/2514564106 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2d10004b60 0x7f2d10004fd0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f2d200016c0 tx=0x7f2d10006940 comp rx=0 tx=0).stop 2024-09-21T03:39:57.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:57.315+0000 7f2d2b7a1640 1 -- 172.21.15.38:0/2514564106 shutdown_connections 2024-09-21T03:39:57.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:57.315+0000 7f2d2b7a1640 1 --2- 172.21.15.38:0/2514564106 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2d10004b60 0x7f2d10004fd0 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:57.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:57.315+0000 7f2d2b7a1640 1 -- 172.21.15.38:0/2514564106 >> 172.21.15.38:0/2514564106 conn(0x7f2d240b1b90 msgr2=0x7f2d240b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:57.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:57.315+0000 7f2d2b7a1640 1 -- 172.21.15.38:0/2514564106 shutdown_connections 2024-09-21T03:39:57.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:57.316+0000 7f2d2b7a1640 1 -- 172.21.15.38:0/2514564106 wait complete. 2024-09-21T03:39:57.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:57.317+0000 7f2d2b7a1640 1 Processor -- start 2024-09-21T03:39:57.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:57.317+0000 7f2d2b7a1640 1 -- start start 2024-09-21T03:39:57.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:57.319+0000 7f2d2b7a1640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2d10004b60 0x7f2d241503a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:57.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:57.319+0000 7f2d2b7a1640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2d24152660 0x7f2d2414e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:57.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:57.319+0000 7f2d2b7a1640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2d24152a70 0x7f2d241f6600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:57.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:57.319+0000 7f2d2b7a1640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f2d2406df70 con 0x7f2d24152660 2024-09-21T03:39:57.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:57.319+0000 7f2d2b7a1640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f2d2406ddf0 con 0x7f2d10004b60 2024-09-21T03:39:57.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:57.319+0000 7f2d2b7a1640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f2d2406e0f0 con 0x7f2d24152a70 2024-09-21T03:39:57.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:57.319+0000 7f2d28d3c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2d24152660 0x7f2d2414e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:57.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:57.319+0000 7f2d28d3c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2d24152660 0x7f2d2414e530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:45308/0 (socket says 172.21.15.38:45308) 2024-09-21T03:39:57.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:57.319+0000 7f2d28d3c640 1 -- 172.21.15.38:0/2491329826 learned_addr learned my addr 172.21.15.38:0/2491329826 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:39:57.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:57.319+0000 7f2d2953d640 1 --2- 172.21.15.38:0/2491329826 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2d10004b60 0x7f2d241503a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:57.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:57.319+0000 7f2d29d3e640 1 --2- 172.21.15.38:0/2491329826 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2d24152a70 0x7f2d241f6600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:57.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:57.319+0000 7f2d28d3c640 1 -- 172.21.15.38:0/2491329826 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2d24152a70 msgr2=0x7f2d241f6600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:57.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:57.319+0000 7f2d28d3c640 1 --2- 172.21.15.38:0/2491329826 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2d24152a70 0x7f2d241f6600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:57.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:57.319+0000 7f2d28d3c640 1 -- 172.21.15.38:0/2491329826 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2d10004b60 msgr2=0x7f2d241503a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:57.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:57.319+0000 7f2d28d3c640 1 --2- 172.21.15.38:0/2491329826 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2d10004b60 0x7f2d241503a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:57.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:57.319+0000 7f2d28d3c640 1 -- 172.21.15.38:0/2491329826 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d24153210 con 0x7f2d24152660 2024-09-21T03:39:57.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:57.319+0000 7f2d29d3e640 1 --2- 172.21.15.38:0/2491329826 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2d24152a70 0x7f2d241f6600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:39:57.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:57.319+0000 7f2d28d3c640 1 --2- 172.21.15.38:0/2491329826 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2d24152660 0x7f2d2414e530 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f2d14011ab0 tx=0x7f2d14011f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:57.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:57.319+0000 7f2d19ffb640 1 -- 172.21.15.38:0/2491329826 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d1401e070 con 0x7f2d24152660 2024-09-21T03:39:57.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:57.319+0000 7f2d19ffb640 1 -- 172.21.15.38:0/2491329826 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2d1400fc70 con 0x7f2d24152660 2024-09-21T03:39:57.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:57.319+0000 7f2d2b7a1640 1 -- 172.21.15.38:0/2491329826 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d2414d490 con 0x7f2d24152660 2024-09-21T03:39:57.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:57.319+0000 7f2d19ffb640 1 -- 172.21.15.38:0/2491329826 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d14019670 con 0x7f2d24152660 2024-09-21T03:39:57.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:57.320+0000 7f2d2b7a1640 1 -- 172.21.15.38:0/2491329826 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d2414da50 con 0x7f2d24152660 2024-09-21T03:39:57.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:57.321+0000 7f2d2b7a1640 1 -- 172.21.15.38:0/2491329826 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d2414d200 con 0x7f2d24152660 2024-09-21T03:39:57.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:57.322+0000 7f2d19ffb640 1 -- 172.21.15.38:0/2491329826 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f2d14021070 con 0x7f2d24152660 2024-09-21T03:39:57.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:57.322+0000 7f2d19ffb640 1 --2- 172.21.15.38:0/2491329826 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2d08078b90 0x7f2d0807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:57.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:57.322+0000 7f2d19ffb640 1 -- 172.21.15.38:0/2491329826 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5421+0+0 (secure 0 0 0) 0x7f2d1409b2e0 con 0x7f2d24152660 2024-09-21T03:39:57.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:57.322+0000 7f2d2953d640 1 --2- 172.21.15.38:0/2491329826 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2d08078b90 0x7f2d0807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:57.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:57.323+0000 7f2d2953d640 1 --2- 172.21.15.38:0/2491329826 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2d08078b90 0x7f2d0807b0a0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f2d20032000 tx=0x7f2d2002c040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:57.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:57.327+0000 7f2d19ffb640 1 -- 172.21.15.38:0/2491329826 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f2d24153210 con 0x7f2d24152660 2024-09-21T03:39:57.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:57.513+0000 7f2d2b7a1640 1 -- 172.21.15.38:0/2491329826 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f2d2415aa90 con 0x7f2d24152660 2024-09-21T03:39:57.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:57.518+0000 7f2d19ffb640 1 -- 172.21.15.38:0/2491329826 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v47)=0 v47) ==== 122+0+0 (secure 0 0 0) 0x7f2d1400fe60 con 0x7f2d24152660 2024-09-21T03:39:57.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:57.520+0000 7f2d2b7a1640 1 -- 172.21.15.38:0/2491329826 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2d08078b90 msgr2=0x7f2d0807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:57.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:57.520+0000 7f2d2b7a1640 1 --2- 172.21.15.38:0/2491329826 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2d08078b90 0x7f2d0807b0a0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f2d20032000 tx=0x7f2d2002c040 comp rx=0 tx=0).stop 2024-09-21T03:39:57.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:57.520+0000 7f2d2b7a1640 1 -- 172.21.15.38:0/2491329826 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2d24152660 msgr2=0x7f2d2414e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:57.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:57.520+0000 7f2d2b7a1640 1 --2- 172.21.15.38:0/2491329826 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2d24152660 0x7f2d2414e530 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f2d14011ab0 tx=0x7f2d14011f70 comp rx=0 tx=0).stop 2024-09-21T03:39:57.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:57.520+0000 7f2d2b7a1640 1 -- 172.21.15.38:0/2491329826 shutdown_connections 2024-09-21T03:39:57.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:57.520+0000 7f2d2b7a1640 1 --2- 172.21.15.38:0/2491329826 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2d08078b90 0x7f2d0807b0a0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:57.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:57.520+0000 7f2d2b7a1640 1 --2- 172.21.15.38:0/2491329826 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2d24152a70 0x7f2d241f6600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:57.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:57.520+0000 7f2d2b7a1640 1 --2- 172.21.15.38:0/2491329826 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2d24152660 0x7f2d2414e530 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:57.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:57.520+0000 7f2d2b7a1640 1 --2- 172.21.15.38:0/2491329826 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2d10004b60 0x7f2d241503a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:57.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:57.520+0000 7f2d2b7a1640 1 -- 172.21.15.38:0/2491329826 >> 172.21.15.38:0/2491329826 conn(0x7f2d240b1b90 msgr2=0x7f2d240cdc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:57.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:57.520+0000 7f2d2b7a1640 1 -- 172.21.15.38:0/2491329826 shutdown_connections 2024-09-21T03:39:57.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:57.520+0000 7f2d2b7a1640 1 -- 172.21.15.38:0/2491329826 wait complete. 2024-09-21T03:39:57.533 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-21T03:39:57.533 DEBUG:teuthology.orchestra.run.smithi038:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T03:39:57.577 DEBUG:teuthology.orchestra.run.smithi195:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T03:39:57.618 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-21T03:39:57.618 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-21T03:39:57.618 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:39:57.634 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056795,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2354 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T03:39:57.635 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-09-21T03:39:57.635 DEBUG:teuthology.orchestra.run.smithi195:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:39:57.676 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056014,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497444k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=8978 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248720k,nr_inodes=812180,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497444k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T03:39:57.677 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-21T03:39:57.677 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:39:57.693 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056795,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2354 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T03:39:57.693 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-09-21T03:39:57.693 DEBUG:teuthology.orchestra.run.smithi195:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:39:57.733 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056014,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497444k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=8978 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248720k,nr_inodes=812180,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497444k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T03:39:57.733 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-21T03:39:57.734 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:39:57.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:57.849+0000 7f912ac07640 1 -- 172.21.15.38:0/977894787 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f91240c5700 msgr2=0x7f91240afa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:57.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:57.849+0000 7f912ac07640 1 --2- 172.21.15.38:0/977894787 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f91240c5700 0x7f91240afa50 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f9110005fd0 tx=0x7f911002b880 comp rx=0 tx=0).stop 2024-09-21T03:39:57.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:57.850+0000 7f912ac07640 1 -- 172.21.15.38:0/977894787 shutdown_connections 2024-09-21T03:39:57.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:57.850+0000 7f912ac07640 1 --2- 172.21.15.38:0/977894787 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f91240c5700 0x7f91240afa50 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:57.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:57.850+0000 7f912ac07640 1 --2- 172.21.15.38:0/977894787 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f912415ad20 0x7f91240cc1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:57.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:57.850+0000 7f912ac07640 1 -- 172.21.15.38:0/977894787 >> 172.21.15.38:0/977894787 conn(0x7f9124073560 msgr2=0x7f9124073970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:57.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:57.850+0000 7f912ac07640 1 -- 172.21.15.38:0/977894787 shutdown_connections 2024-09-21T03:39:57.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:57.850+0000 7f912ac07640 1 -- 172.21.15.38:0/977894787 wait complete. 2024-09-21T03:39:57.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:57.851+0000 7f912ac07640 1 Processor -- start 2024-09-21T03:39:57.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:57.851+0000 7f912ac07640 1 -- start start 2024-09-21T03:39:57.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:57.853+0000 7f912ac07640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f912406ddf0 0x7f912414e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:57.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:57.853+0000 7f912ac07640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f91240c5700 0x7f91240c95e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:57.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:57.853+0000 7f912ac07640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f912415ad20 0x7f91240c9b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:57.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:57.853+0000 7f912ac07640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f9124069a50 con 0x7f912406ddf0 2024-09-21T03:39:57.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:57.853+0000 7f912ac07640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f91240698d0 con 0x7f912415ad20 2024-09-21T03:39:57.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:57.853+0000 7f912ac07640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f9124069bd0 con 0x7f91240c5700 2024-09-21T03:39:57.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:57.853+0000 7f91289a3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f912406ddf0 0x7f912414e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:57.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:57.853+0000 7f91289a3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f912406ddf0 0x7f912414e340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:45330/0 (socket says 172.21.15.38:45330) 2024-09-21T03:39:57.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:57.853+0000 7f91289a3640 1 -- 172.21.15.38:0/2298551519 learned_addr learned my addr 172.21.15.38:0/2298551519 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:39:57.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:57.853+0000 7f91291a4640 1 --2- 172.21.15.38:0/2298551519 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f912415ad20 0x7f91240c9b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:57.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:57.853+0000 7f91289a3640 1 -- 172.21.15.38:0/2298551519 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f91240c5700 msgr2=0x7f91240c95e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:57.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:57.853+0000 7f9123fff640 1 --2- 172.21.15.38:0/2298551519 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f91240c5700 0x7f91240c95e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:57.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:57.853+0000 7f91289a3640 1 --2- 172.21.15.38:0/2298551519 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f91240c5700 0x7f91240c95e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:57.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:57.853+0000 7f91289a3640 1 -- 172.21.15.38:0/2298551519 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f912415ad20 msgr2=0x7f91240c9b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:57.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:57.853+0000 7f91289a3640 1 --2- 172.21.15.38:0/2298551519 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f912415ad20 0x7f91240c9b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:57.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:57.853+0000 7f91289a3640 1 -- 172.21.15.38:0/2298551519 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9124156950 con 0x7f912406ddf0 2024-09-21T03:39:57.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:57.853+0000 7f91291a4640 1 --2- 172.21.15.38:0/2298551519 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f912415ad20 0x7f91240c9b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:39:57.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:57.854+0000 7f91289a3640 1 --2- 172.21.15.38:0/2298551519 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f912406ddf0 0x7f912414e340 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f911800d740 tx=0x7f911800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:57.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:57.854+0000 7f91217fa640 1 -- 172.21.15.38:0/2298551519 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9118017070 con 0x7f912406ddf0 2024-09-21T03:39:57.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:57.854+0000 7f91217fa640 1 -- 172.21.15.38:0/2298551519 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9118009d20 con 0x7f912406ddf0 2024-09-21T03:39:57.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:57.854+0000 7f91217fa640 1 -- 172.21.15.38:0/2298551519 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91180106c0 con 0x7f912406ddf0 2024-09-21T03:39:57.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:57.854+0000 7f912ac07640 1 -- 172.21.15.38:0/2298551519 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91240caa70 con 0x7f912406ddf0 2024-09-21T03:39:57.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:57.854+0000 7f912ac07640 1 -- 172.21.15.38:0/2298551519 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91240ae110 con 0x7f912406ddf0 2024-09-21T03:39:57.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:57.855+0000 7f912ac07640 1 -- 172.21.15.38:0/2298551519 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91240afa50 con 0x7f912406ddf0 2024-09-21T03:39:57.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:57.856+0000 7f91217fa640 1 -- 172.21.15.38:0/2298551519 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f911801a070 con 0x7f912406ddf0 2024-09-21T03:39:57.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:57.856+0000 7f91217fa640 1 --2- 172.21.15.38:0/2298551519 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9108078bc0 0x7f910807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:57.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:57.856+0000 7f91217fa640 1 -- 172.21.15.38:0/2298551519 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5421+0+0 (secure 0 0 0) 0x7f911809ad40 con 0x7f912406ddf0 2024-09-21T03:39:57.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:57.857+0000 7f9123fff640 1 --2- 172.21.15.38:0/2298551519 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9108078bc0 0x7f910807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:57.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:57.857+0000 7f9123fff640 1 --2- 172.21.15.38:0/2298551519 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9108078bc0 0x7f910807b0d0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f9110035000 tx=0x7f9110004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:57.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:57.861+0000 7f91217fa640 1 -- 172.21.15.38:0/2298551519 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f9118063330 con 0x7f912406ddf0 2024-09-21T03:39:58.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:58.087+0000 7f912ac07640 1 -- 172.21.15.38:0/2298551519 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f91240c6c20 con 0x7f912406ddf0 2024-09-21T03:39:58.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:58.087+0000 7f91217fa640 1 -- 172.21.15.38:0/2298551519 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) ==== 94+0+3917 (secure 0 0 0) 0x7f9118062c30 con 0x7f912406ddf0 2024-09-21T03:39:58.089 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:39:58.089 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":76,"btime":"2024-09-21T03:39:51:809181+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5783,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/1358588311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1358588311},{"type":"v1","addr":"172.21.15.38:6835","nonce":1358588311}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63},{"gid":5789,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/1676483003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":1676483003},{"type":"v1","addr":"172.21.15.195:6837","nonce":1676483003}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63},{"gid":6029,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/4112188347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":4112188347},{"type":"v1","addr":"172.21.15.195:6836","nonce":4112188347}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76},{"gid":6813,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/3644074938","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":3644074938},{"type":"v1","addr":"172.21.15.38:6833","nonce":3644074938}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":72}],"filesystems":[]} 2024-09-21T03:39:58.090 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 76 2024-09-21T03:39:58.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:58.090+0000 7f912ac07640 1 -- 172.21.15.38:0/2298551519 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9108078bc0 msgr2=0x7f910807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:58.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:58.090+0000 7f912ac07640 1 --2- 172.21.15.38:0/2298551519 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9108078bc0 0x7f910807b0d0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f9110035000 tx=0x7f9110004730 comp rx=0 tx=0).stop 2024-09-21T03:39:58.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:58.090+0000 7f912ac07640 1 -- 172.21.15.38:0/2298551519 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f912406ddf0 msgr2=0x7f912414e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:58.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:58.090+0000 7f912ac07640 1 --2- 172.21.15.38:0/2298551519 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f912406ddf0 0x7f912414e340 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f911800d740 tx=0x7f911800dc00 comp rx=0 tx=0).stop 2024-09-21T03:39:58.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:58.090+0000 7f912ac07640 1 -- 172.21.15.38:0/2298551519 shutdown_connections 2024-09-21T03:39:58.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:58.090+0000 7f912ac07640 1 --2- 172.21.15.38:0/2298551519 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f912415ad20 0x7f91240c9b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:58.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:58.090+0000 7f912ac07640 1 --2- 172.21.15.38:0/2298551519 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9108078bc0 0x7f910807b0d0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:58.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:58.090+0000 7f912ac07640 1 --2- 172.21.15.38:0/2298551519 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f91240c5700 0x7f91240c95e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:58.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:58.090+0000 7f912ac07640 1 --2- 172.21.15.38:0/2298551519 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f912406ddf0 0x7f912414e340 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:58.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:58.090+0000 7f912ac07640 1 -- 172.21.15.38:0/2298551519 >> 172.21.15.38:0/2298551519 conn(0x7f9124073560 msgr2=0x7f91240b6140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:58.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:58.090+0000 7f912ac07640 1 -- 172.21.15.38:0/2298551519 shutdown_connections 2024-09-21T03:39:58.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:58.090+0000 7f912ac07640 1 -- 172.21.15.38:0/2298551519 wait complete. 2024-09-21T03:39:58.103 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-21T03:39:58.104 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-21T03:39:58.104 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-21T03:39:58.104 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T03:39:58.104 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-21T03:39:58.104 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-21T03:39:58.104 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-21T03:39:58.104 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T03:39:58.105 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-21T03:39:58.105 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-21T03:39:58.105 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-21T03:39:58.105 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T03:39:58.105 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-21T03:39:58.105 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-21T03:39:58.105 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-21T03:39:58.106 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T03:39:58.114 INFO:tasks.ceph.mds.a:Stopped 2024-09-21T03:39:58.114 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-21T03:39:58.114 INFO:tasks.ceph.mds.c:Stopped 2024-09-21T03:39:58.115 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-21T03:39:58.118 INFO:tasks.ceph.mds.b:Stopped 2024-09-21T03:39:58.118 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-21T03:39:58.121 INFO:tasks.ceph.mds.d:Stopped 2024-09-21T03:39:58.121 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-21T03:39:58.156 INFO:tasks.ceph.mds.a:Started 2024-09-21T03:39:58.156 DEBUG:teuthology.parallel:result is None 2024-09-21T03:39:58.157 INFO:tasks.ceph.mds.c:Started 2024-09-21T03:39:58.157 DEBUG:teuthology.parallel:result is None 2024-09-21T03:39:58.162 INFO:tasks.ceph.mds.b:Started 2024-09-21T03:39:58.163 DEBUG:teuthology.parallel:result is None 2024-09-21T03:39:58.164 INFO:tasks.ceph.mds.d:Started 2024-09-21T03:39:58.164 DEBUG:teuthology.parallel:result is None 2024-09-21T03:39:58.164 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-21T03:39:58.244 INFO:tasks.ceph.mds.c.smithi038.stdout:starting mds.c at 2024-09-21T03:39:58.246 INFO:tasks.ceph.mds.a.smithi038.stdout:starting mds.a at 2024-09-21T03:39:58.261 INFO:tasks.ceph.mds.b.smithi195.stdout:starting mds.b at 2024-09-21T03:39:58.261 INFO:tasks.ceph.mds.d.smithi195.stdout:starting mds.d at 2024-09-21T03:39:58.262 INFO:tasks.ceph.mds.b.smithi195.stderr:2024-09-21T03:39:58.260+0000 7f4ab5d6f640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2024-09-21T03:39:58.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:58.287+0000 7fcf37d82640 1 -- 172.21.15.38:0/1354946424 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcf300d0170 msgr2=0x7fcf300d0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:58.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:58.287+0000 7fcf37d82640 1 --2- 172.21.15.38:0/1354946424 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcf300d0170 0x7fcf300d0be0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fcf2c009850 tx=0x7fcf2c02a4f0 comp rx=0 tx=0).stop 2024-09-21T03:39:58.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:58.287+0000 7fcf37d82640 1 -- 172.21.15.38:0/1354946424 shutdown_connections 2024-09-21T03:39:58.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:58.287+0000 7fcf37d82640 1 --2- 172.21.15.38:0/1354946424 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcf300d0170 0x7fcf300d0be0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:58.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:58.287+0000 7fcf37d82640 1 -- 172.21.15.38:0/1354946424 >> 172.21.15.38:0/1354946424 conn(0x7fcf3009d830 msgr2=0x7fcf300b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:58.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:58.287+0000 7fcf37d82640 1 -- 172.21.15.38:0/1354946424 shutdown_connections 2024-09-21T03:39:58.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:58.287+0000 7fcf37d82640 1 -- 172.21.15.38:0/1354946424 wait complete. 2024-09-21T03:39:58.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:58.289+0000 7fcf37d82640 1 Processor -- start 2024-09-21T03:39:58.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:58.289+0000 7fcf37d82640 1 -- start start 2024-09-21T03:39:58.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:58.290+0000 7fcf37d82640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcf300d0170 0x7fcf300b26a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:58.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:58.290+0000 7fcf37d82640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcf300d1120 0x7fcf30154a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:58.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:58.290+0000 7fcf37d82640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcf3015ee80 0x7fcf300ccbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:58.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:58.290+0000 7fcf37d82640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fcf300b6910 con 0x7fcf300d0170 2024-09-21T03:39:58.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:58.290+0000 7fcf37d82640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fcf300b6790 con 0x7fcf3015ee80 2024-09-21T03:39:58.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:58.290+0000 7fcf37d82640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fcf300b6a90 con 0x7fcf300d1120 2024-09-21T03:39:58.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:58.290+0000 7fcf35b1e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcf300d0170 0x7fcf300b26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:58.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:58.291+0000 7fcf35b1e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcf300d0170 0x7fcf300b26a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:45366/0 (socket says 172.21.15.38:45366) 2024-09-21T03:39:58.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:58.291+0000 7fcf35b1e640 1 -- 172.21.15.38:0/1824600399 learned_addr learned my addr 172.21.15.38:0/1824600399 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:39:58.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:58.291+0000 7fcf3631f640 1 --2- 172.21.15.38:0/1824600399 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcf3015ee80 0x7fcf300ccbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:58.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:58.291+0000 7fcf3531d640 1 --2- 172.21.15.38:0/1824600399 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcf300d1120 0x7fcf30154a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:58.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:58.291+0000 7fcf35b1e640 1 -- 172.21.15.38:0/1824600399 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcf300d1120 msgr2=0x7fcf30154a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:58.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:58.291+0000 7fcf35b1e640 1 --2- 172.21.15.38:0/1824600399 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcf300d1120 0x7fcf30154a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:58.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:58.291+0000 7fcf35b1e640 1 -- 172.21.15.38:0/1824600399 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcf3015ee80 msgr2=0x7fcf300ccbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:58.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:58.291+0000 7fcf35b1e640 1 --2- 172.21.15.38:0/1824600399 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcf3015ee80 0x7fcf300ccbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:58.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:58.291+0000 7fcf35b1e640 1 -- 172.21.15.38:0/1824600399 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf301f97a0 con 0x7fcf300d0170 2024-09-21T03:39:58.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:58.291+0000 7fcf3531d640 1 --2- 172.21.15.38:0/1824600399 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcf300d1120 0x7fcf30154a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:39:58.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:58.291+0000 7fcf3631f640 1 --2- 172.21.15.38:0/1824600399 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcf3015ee80 0x7fcf300ccbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:39:58.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:58.291+0000 7fcf35b1e640 1 --2- 172.21.15.38:0/1824600399 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcf300d0170 0x7fcf300b26a0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7fcf2400c460 tx=0x7fcf2400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:58.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:58.291+0000 7fcf227fc640 1 -- 172.21.15.38:0/1824600399 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf240052d0 con 0x7fcf300d0170 2024-09-21T03:39:58.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:58.291+0000 7fcf227fc640 1 -- 172.21.15.38:0/1824600399 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcf24005490 con 0x7fcf300d0170 2024-09-21T03:39:58.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:58.291+0000 7fcf37d82640 1 -- 172.21.15.38:0/1824600399 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf3006ed30 con 0x7fcf300d0170 2024-09-21T03:39:58.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:58.291+0000 7fcf227fc640 1 -- 172.21.15.38:0/1824600399 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf24012870 con 0x7fcf300d0170 2024-09-21T03:39:58.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:58.291+0000 7fcf37d82640 1 -- 172.21.15.38:0/1824600399 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf3006f2f0 con 0x7fcf300d0170 2024-09-21T03:39:58.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:58.292+0000 7fcf37d82640 1 -- 172.21.15.38:0/1824600399 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf300d0be0 con 0x7fcf300d0170 2024-09-21T03:39:58.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:58.293+0000 7fcf227fc640 1 -- 172.21.15.38:0/1824600399 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fcf24012a10 con 0x7fcf300d0170 2024-09-21T03:39:58.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:58.294+0000 7fcf227fc640 1 --2- 172.21.15.38:0/1824600399 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fcf14078b90 0x7fcf1407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:58.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:58.294+0000 7fcf227fc640 1 -- 172.21.15.38:0/1824600399 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5421+0+0 (secure 0 0 0) 0x7fcf2409a760 con 0x7fcf300d0170 2024-09-21T03:39:58.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:58.294+0000 7fcf3531d640 1 --2- 172.21.15.38:0/1824600399 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fcf14078b90 0x7fcf1407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:58.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:58.295+0000 7fcf3531d640 1 --2- 172.21.15.38:0/1824600399 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fcf14078b90 0x7fcf1407b0a0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fcf2c034000 tx=0x7fcf2c004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:58.300 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:58.299+0000 7fcf227fc640 1 -- 172.21.15.38:0/1824600399 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fcf24062cc0 con 0x7fcf300d0170 2024-09-21T03:39:58.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:58.497+0000 7fcf37d82640 1 -- 172.21.15.38:0/1824600399 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fcf300b5850 con 0x7fcf300d0170 2024-09-21T03:39:58.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:58.861+0000 7fcf227fc640 1 -- 172.21.15.38:0/1824600399 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v88) ==== 97+0+0 (secure 0 0 0) 0x7fcf240625c0 con 0x7fcf300d0170 2024-09-21T03:39:58.864 INFO:teuthology.orchestra.run.smithi038.stderr: removed all blocklist entries 2024-09-21T03:39:58.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:58.864+0000 7fcf37d82640 1 -- 172.21.15.38:0/1824600399 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fcf14078b90 msgr2=0x7fcf1407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:58.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:58.864+0000 7fcf37d82640 1 --2- 172.21.15.38:0/1824600399 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fcf14078b90 0x7fcf1407b0a0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fcf2c034000 tx=0x7fcf2c004730 comp rx=0 tx=0).stop 2024-09-21T03:39:58.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:58.864+0000 7fcf37d82640 1 -- 172.21.15.38:0/1824600399 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcf300d0170 msgr2=0x7fcf300b26a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:58.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:58.864+0000 7fcf37d82640 1 --2- 172.21.15.38:0/1824600399 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcf300d0170 0x7fcf300b26a0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7fcf2400c460 tx=0x7fcf2400c920 comp rx=0 tx=0).stop 2024-09-21T03:39:58.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:58.864+0000 7fcf37d82640 1 -- 172.21.15.38:0/1824600399 shutdown_connections 2024-09-21T03:39:58.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:58.864+0000 7fcf37d82640 1 --2- 172.21.15.38:0/1824600399 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcf3015ee80 0x7fcf300ccbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:58.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:58.864+0000 7fcf37d82640 1 --2- 172.21.15.38:0/1824600399 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fcf14078b90 0x7fcf1407b0a0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:58.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:58.864+0000 7fcf37d82640 1 --2- 172.21.15.38:0/1824600399 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcf300d1120 0x7fcf30154a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:58.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:58.864+0000 7fcf37d82640 1 --2- 172.21.15.38:0/1824600399 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcf300d0170 0x7fcf300b26a0 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:58.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:58.864+0000 7fcf37d82640 1 -- 172.21.15.38:0/1824600399 >> 172.21.15.38:0/1824600399 conn(0x7fcf3009d830 msgr2=0x7fcf3015de90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:58.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:58.864+0000 7fcf37d82640 1 -- 172.21.15.38:0/1824600399 shutdown_connections 2024-09-21T03:39:58.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:58.864+0000 7fcf37d82640 1 -- 172.21.15.38:0/1824600399 wait complete. 2024-09-21T03:39:58.880 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-21T03:39:58.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:58.996+0000 7fb5f068c640 1 -- 172.21.15.38:0/627729389 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb5d4004090 msgr2=0x7fb5d4004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:58.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:58.996+0000 7fb5f068c640 1 --2- 172.21.15.38:0/627729389 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb5d4004090 0x7fb5d4004800 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7fb5d800c710 tx=0x7fb5d4005830 comp rx=0 tx=0).stop 2024-09-21T03:39:58.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:58.996+0000 7fb5eec29640 1 -- 172.21.15.38:0/627729389 reap_dead start 2024-09-21T03:39:58.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:58.996+0000 7fb5f068c640 1 -- 172.21.15.38:0/627729389 shutdown_connections 2024-09-21T03:39:58.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:58.996+0000 7fb5f068c640 1 -- 172.21.15.38:0/627729389 >> 172.21.15.38:0/627729389 conn(0x7fb5e80b6b20 msgr2=0x7fb5e80d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:58.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:58.996+0000 7fb5f068c640 1 -- 172.21.15.38:0/627729389 shutdown_connections 2024-09-21T03:39:58.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:58.996+0000 7fb5f068c640 1 -- 172.21.15.38:0/627729389 wait complete. 2024-09-21T03:39:58.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:58.997+0000 7fb5f068c640 1 Processor -- start 2024-09-21T03:39:58.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:58.998+0000 7fb5f068c640 1 -- start start 2024-09-21T03:39:59.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:58.999+0000 7fb5f068c640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb5d4004090 0x7fb5e80b26e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:59.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:58.999+0000 7fb5f068c640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb5e80b3e60 0x7fb5e815b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:59.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:58.999+0000 7fb5f068c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb5e815c5d0 0x7fb5e815c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:59.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:58.999+0000 7fb5f068c640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fb5e80b2fc0 con 0x7fb5e815c5d0 2024-09-21T03:39:59.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:58.999+0000 7fb5f068c640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fb5e80b2e40 con 0x7fb5e80b3e60 2024-09-21T03:39:59.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:58.999+0000 7fb5f068c640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fb5e80b3140 con 0x7fb5d4004090 2024-09-21T03:39:59.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:58.999+0000 7fb5eec29640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb5e815c5d0 0x7fb5e815c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:59.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:58.999+0000 7fb5eec29640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb5e815c5d0 0x7fb5e815c9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:45388/0 (socket says 172.21.15.38:45388) 2024-09-21T03:39:59.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:58.999+0000 7fb5eec29640 1 -- 172.21.15.38:0/915232722 learned_addr learned my addr 172.21.15.38:0/915232722 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:39:59.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:58.999+0000 7fb5edc27640 1 --2- 172.21.15.38:0/915232722 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb5e80b3e60 0x7fb5e815b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:59.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:58.999+0000 7fb5ee428640 1 --2- 172.21.15.38:0/915232722 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb5d4004090 0x7fb5e80b26e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:59.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:58.999+0000 7fb5eec29640 1 -- 172.21.15.38:0/915232722 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb5d4004090 msgr2=0x7fb5e80b26e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:59.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:58.999+0000 7fb5eec29640 1 --2- 172.21.15.38:0/915232722 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb5d4004090 0x7fb5e80b26e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:59.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:58.999+0000 7fb5eec29640 1 -- 172.21.15.38:0/915232722 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb5e80b3e60 msgr2=0x7fb5e815b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:59.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:58.999+0000 7fb5eec29640 1 --2- 172.21.15.38:0/915232722 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb5e80b3e60 0x7fb5e815b730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:59.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:58.999+0000 7fb5eec29640 1 -- 172.21.15.38:0/915232722 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5e815cf20 con 0x7fb5e815c5d0 2024-09-21T03:39:59.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:59.000+0000 7fb5eec29640 1 --2- 172.21.15.38:0/915232722 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb5e815c5d0 0x7fb5e815c9e0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7fb5e0005fd0 tx=0x7fb5e000eda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:59.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:59.000+0000 7fb5deffd640 1 -- 172.21.15.38:0/915232722 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5e0004b20 con 0x7fb5e815c5d0 2024-09-21T03:39:59.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:59.000+0000 7fb5deffd640 1 -- 172.21.15.38:0/915232722 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb5e0004db0 con 0x7fb5e815c5d0 2024-09-21T03:39:59.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:59.000+0000 7fb5f068c640 1 -- 172.21.15.38:0/915232722 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5e81f5eb0 con 0x7fb5e815c5d0 2024-09-21T03:39:59.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:59.000+0000 7fb5deffd640 1 -- 172.21.15.38:0/915232722 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5e000ccc0 con 0x7fb5e815c5d0 2024-09-21T03:39:59.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:59.000+0000 7fb5f068c640 1 -- 172.21.15.38:0/915232722 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5e81f6470 con 0x7fb5e815c5d0 2024-09-21T03:39:59.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:59.001+0000 7fb5f068c640 1 -- 172.21.15.38:0/915232722 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5e8162f00 con 0x7fb5e815c5d0 2024-09-21T03:39:59.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:59.002+0000 7fb5deffd640 1 -- 172.21.15.38:0/915232722 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb5e001b020 con 0x7fb5e815c5d0 2024-09-21T03:39:59.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:59.002+0000 7fb5deffd640 1 --2- 172.21.15.38:0/915232722 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb5cc078b90 0x7fb5cc07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:59.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:59.002+0000 7fb5deffd640 1 -- 172.21.15.38:0/915232722 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5249+0+0 (secure 0 0 0) 0x7fb5e009a970 con 0x7fb5e815c5d0 2024-09-21T03:39:59.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:59.003+0000 7fb5ee428640 1 --2- 172.21.15.38:0/915232722 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb5cc078b90 0x7fb5cc07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:59.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:59.003+0000 7fb5ee428640 1 --2- 172.21.15.38:0/915232722 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb5cc078b90 0x7fb5cc07b0a0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fb5e4001220 tx=0x7fb5e4009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:59.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:59.007+0000 7fb5deffd640 1 -- 172.21.15.38:0/915232722 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fb5e815cf20 con 0x7fb5e815c5d0 2024-09-21T03:39:59.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:59.269+0000 7fb5f068c640 1 -- 172.21.15.38:0/915232722 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fb5e80cfb10 con 0x7fb5e815c5d0 2024-09-21T03:39:59.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:59.269+0000 7fb5deffd640 1 -- 172.21.15.38:0/915232722 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v21) ==== 80+0+5304 (secure 0 0 0) 0x7fb5e000ce60 con 0x7fb5e815c5d0 2024-09-21T03:39:59.271 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:39:59.271 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-09-21T03:39:59.271 INFO:teuthology.orchestra.run.smithi038.stdout: "auth_dump": [ 2024-09-21T03:39:59.271 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-21T03:39:59.272 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "mds.a", 2024-09-21T03:39:59.272 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQBPPu5mfzGIMRAAmq0TaC3J1VU3GxzhTgJ/Zw==", 2024-09-21T03:39:59.272 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-09-21T03:39:59.272 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow", 2024-09-21T03:39:59.272 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow *", 2024-09-21T03:39:59.272 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow *", 2024-09-21T03:39:59.272 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-09-21T03:39:59.272 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-21T03:39:59.272 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-21T03:39:59.272 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-21T03:39:59.272 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "mds.b", 2024-09-21T03:39:59.272 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQBQPu5m/fzuAhAApyigRlQ8OrV/sf0ap6Fn6A==", 2024-09-21T03:39:59.272 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-09-21T03:39:59.273 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow", 2024-09-21T03:39:59.273 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow *", 2024-09-21T03:39:59.273 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow *", 2024-09-21T03:39:59.273 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-09-21T03:39:59.273 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-21T03:39:59.273 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-21T03:39:59.273 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-21T03:39:59.273 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "mds.c", 2024-09-21T03:39:59.273 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQBPPu5mQipIORAArRn8k8SvvpG8x03F9I2/gQ==", 2024-09-21T03:39:59.273 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-09-21T03:39:59.273 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow", 2024-09-21T03:39:59.273 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow *", 2024-09-21T03:39:59.273 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow *", 2024-09-21T03:39:59.273 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-09-21T03:39:59.274 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-21T03:39:59.274 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-21T03:39:59.274 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-21T03:39:59.274 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "mds.d", 2024-09-21T03:39:59.274 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQBQPu5mzlyDChAAqbPKXS3EtnOqqeHhA1ehIw==", 2024-09-21T03:39:59.274 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-09-21T03:39:59.274 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow", 2024-09-21T03:39:59.274 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow *", 2024-09-21T03:39:59.274 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow *", 2024-09-21T03:39:59.274 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-09-21T03:39:59.274 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-21T03:39:59.274 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-21T03:39:59.274 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-21T03:39:59.275 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "osd.0", 2024-09-21T03:39:59.275 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQBXPu5mMZggAhAAtMPdJipheizZhpby9APj4A==", 2024-09-21T03:39:59.275 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-09-21T03:39:59.275 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow profile osd", 2024-09-21T03:39:59.275 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile osd", 2024-09-21T03:39:59.275 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-09-21T03:39:59.275 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-21T03:39:59.275 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-21T03:39:59.275 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-21T03:39:59.275 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "osd.1", 2024-09-21T03:39:59.275 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQBaPu5mm9TFIhAAWPJCvBYMgMbKr7Z7Diq58A==", 2024-09-21T03:39:59.275 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-09-21T03:39:59.275 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow profile osd", 2024-09-21T03:39:59.275 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile osd", 2024-09-21T03:39:59.276 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-09-21T03:39:59.276 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-21T03:39:59.276 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-21T03:39:59.276 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-21T03:39:59.276 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "osd.2", 2024-09-21T03:39:59.276 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQBePu5mqjljCBAAcdEpzJq6D8e1L+q1s0GjWQ==", 2024-09-21T03:39:59.276 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-09-21T03:39:59.276 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow profile osd", 2024-09-21T03:39:59.276 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile osd", 2024-09-21T03:39:59.276 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-09-21T03:39:59.276 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-21T03:39:59.276 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-21T03:39:59.276 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-21T03:39:59.277 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "osd.3", 2024-09-21T03:39:59.277 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQBhPu5mRjfdKBAAJ6+PlEiRp/xvxny6SjHV8Q==", 2024-09-21T03:39:59.277 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-09-21T03:39:59.277 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow profile osd", 2024-09-21T03:39:59.277 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile osd", 2024-09-21T03:39:59.277 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-09-21T03:39:59.277 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-21T03:39:59.277 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-21T03:39:59.277 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-21T03:39:59.277 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "osd.4", 2024-09-21T03:39:59.277 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQBlPu5m8bNyNxAA2rERruGojEDKmcXN2K6uog==", 2024-09-21T03:39:59.277 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-09-21T03:39:59.277 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow profile osd", 2024-09-21T03:39:59.278 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile osd", 2024-09-21T03:39:59.278 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-09-21T03:39:59.278 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-21T03:39:59.278 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-21T03:39:59.278 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-21T03:39:59.278 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "osd.5", 2024-09-21T03:39:59.278 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQBpPu5mBB2iDBAAvsvhAPMvNWII2Zr9Je45YQ==", 2024-09-21T03:39:59.278 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-09-21T03:39:59.278 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow profile osd", 2024-09-21T03:39:59.278 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile osd", 2024-09-21T03:39:59.278 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-09-21T03:39:59.278 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-21T03:39:59.278 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-21T03:39:59.279 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-21T03:39:59.279 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "osd.6", 2024-09-21T03:39:59.279 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQBsPu5mMQMaLRAAznzYdoNGHWOI2p2mNN/rQw==", 2024-09-21T03:39:59.279 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-09-21T03:39:59.279 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow profile osd", 2024-09-21T03:39:59.279 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile osd", 2024-09-21T03:39:59.279 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-09-21T03:39:59.279 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-21T03:39:59.279 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-21T03:39:59.279 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-21T03:39:59.279 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "osd.7", 2024-09-21T03:39:59.279 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQBwPu5mHu0WERAAir8bmtuNOOj/2D4hLaoYOA==", 2024-09-21T03:39:59.279 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-09-21T03:39:59.279 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow profile osd", 2024-09-21T03:39:59.280 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile osd", 2024-09-21T03:39:59.280 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-09-21T03:39:59.280 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-21T03:39:59.280 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-21T03:39:59.280 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-21T03:39:59.280 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "client.0", 2024-09-21T03:39:59.280 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQBQPu5m6ydBDxAAlWZ2XzQTPBcYPriPGRJVeA==", 2024-09-21T03:39:59.280 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-09-21T03:39:59.280 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow", 2024-09-21T03:39:59.280 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow r", 2024-09-21T03:39:59.280 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-21T03:39:59.280 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-21T03:39:59.280 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-21T03:39:59.280 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-21T03:39:59.281 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "client.1", 2024-09-21T03:39:59.281 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQBQPu5mlWP0FhAAkNwfaJ6rPr3r4ZRVbPxX8g==", 2024-09-21T03:39:59.281 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-09-21T03:39:59.281 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow", 2024-09-21T03:39:59.281 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow r", 2024-09-21T03:39:59.281 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow rw", 2024-09-21T03:39:59.281 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow rwx" 2024-09-21T03:39:59.281 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-21T03:39:59.281 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-21T03:39:59.281 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-21T03:39:59.281 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "client.2", 2024-09-21T03:39:59.281 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQBQPu5m48oaExAAOzYrR/SsykpzH2TkWHs39Q==", 2024-09-21T03:39:59.281 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-09-21T03:39:59.281 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow", 2024-09-21T03:39:59.281 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow r", 2024-09-21T03:39:59.282 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow rw", 2024-09-21T03:39:59.282 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow rwx" 2024-09-21T03:39:59.282 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-21T03:39:59.282 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-21T03:39:59.282 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-21T03:39:59.282 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "client.3", 2024-09-21T03:39:59.282 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQBQPu5mVQSzGhAAAo6vizLc7GOnenQRsgtn1A==", 2024-09-21T03:39:59.282 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-09-21T03:39:59.282 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow", 2024-09-21T03:39:59.282 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow r", 2024-09-21T03:39:59.282 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow rw", 2024-09-21T03:39:59.282 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow rwx" 2024-09-21T03:39:59.282 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-21T03:39:59.283 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-21T03:39:59.283 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-21T03:39:59.283 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "client.admin", 2024-09-21T03:39:59.283 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQBPPu5mPHphExAA7D+/V7qhoNyoc6h0GnNMlQ==", 2024-09-21T03:39:59.283 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-09-21T03:39:59.283 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow *", 2024-09-21T03:39:59.283 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow *", 2024-09-21T03:39:59.283 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow *", 2024-09-21T03:39:59.283 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-09-21T03:39:59.283 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-21T03:39:59.283 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-21T03:39:59.283 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-21T03:39:59.283 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "mgr.x", 2024-09-21T03:39:59.284 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQBPPu5m6K2oLRAAzldoBXAMSnWhxVFwPZkHOw==", 2024-09-21T03:39:59.284 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-09-21T03:39:59.284 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow *", 2024-09-21T03:39:59.284 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile mgr", 2024-09-21T03:39:59.284 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-09-21T03:39:59.284 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-21T03:39:59.284 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-21T03:39:59.284 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-21T03:39:59.284 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "mgr.y", 2024-09-21T03:39:59.284 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQBPPu5mWEz+KBAASx2KzDril3lD9cEawy56jA==", 2024-09-21T03:39:59.284 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-09-21T03:39:59.284 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow *", 2024-09-21T03:39:59.284 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile mgr", 2024-09-21T03:39:59.285 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-09-21T03:39:59.285 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-21T03:39:59.285 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-21T03:39:59.285 INFO:teuthology.orchestra.run.smithi038.stdout: ] 2024-09-21T03:39:59.285 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-09-21T03:39:59.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:59.271+0000 7fb5f068c640 1 -- 172.21.15.38:0/915232722 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb5cc078b90 msgr2=0x7fb5cc07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:59.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:59.271+0000 7fb5f068c640 1 --2- 172.21.15.38:0/915232722 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb5cc078b90 0x7fb5cc07b0a0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fb5e4001220 tx=0x7fb5e4009040 comp rx=0 tx=0).stop 2024-09-21T03:39:59.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:59.271+0000 7fb5f068c640 1 -- 172.21.15.38:0/915232722 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb5e815c5d0 msgr2=0x7fb5e815c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:59.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:59.271+0000 7fb5f068c640 1 --2- 172.21.15.38:0/915232722 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb5e815c5d0 0x7fb5e815c9e0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7fb5e0005fd0 tx=0x7fb5e000eda0 comp rx=0 tx=0).stop 2024-09-21T03:39:59.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:59.271+0000 7fb5f068c640 1 -- 172.21.15.38:0/915232722 shutdown_connections 2024-09-21T03:39:59.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:59.271+0000 7fb5f068c640 1 --2- 172.21.15.38:0/915232722 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb5e815c5d0 0x7fb5e815c9e0 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:59.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:59.271+0000 7fb5f068c640 1 --2- 172.21.15.38:0/915232722 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb5e80b3e60 0x7fb5e815b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:59.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:59.271+0000 7fb5f068c640 1 --2- 172.21.15.38:0/915232722 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb5cc078b90 0x7fb5cc07b0a0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:59.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:59.271+0000 7fb5f068c640 1 --2- 172.21.15.38:0/915232722 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb5d4004090 0x7fb5e80b26e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:59.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:59.271+0000 7fb5f068c640 1 -- 172.21.15.38:0/915232722 >> 172.21.15.38:0/915232722 conn(0x7fb5e80b6b20 msgr2=0x7fb5e8162690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:59.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:59.271+0000 7fb5f068c640 1 -- 172.21.15.38:0/915232722 shutdown_connections 2024-09-21T03:39:59.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:59.272+0000 7fb5f068c640 1 -- 172.21.15.38:0/915232722 wait complete. 2024-09-21T03:39:59.287 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-21T03:39:59.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:59.401+0000 7fd7c2bed640 1 -- 172.21.15.38:0/3679631185 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7bc0cd040 msgr2=0x7fd7bc167310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:59.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:59.401+0000 7fd7c2bed640 1 --2- 172.21.15.38:0/3679631185 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7bc0cd040 0x7fd7bc167310 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7fd7b4005fd0 tx=0x7fd7b402a940 comp rx=0 tx=0).stop 2024-09-21T03:39:59.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:59.401+0000 7fd7c2bed640 1 -- 172.21.15.38:0/3679631185 shutdown_connections 2024-09-21T03:39:59.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:59.401+0000 7fd7c2bed640 1 --2- 172.21.15.38:0/3679631185 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7bc0cd040 0x7fd7bc167310 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:59.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:59.401+0000 7fd7c2bed640 1 -- 172.21.15.38:0/3679631185 >> 172.21.15.38:0/3679631185 conn(0x7fd7bc0cef40 msgr2=0x7fd7bc0cf350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:59.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:59.401+0000 7fd7c2bed640 1 -- 172.21.15.38:0/3679631185 shutdown_connections 2024-09-21T03:39:59.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:59.402+0000 7fd7c2bed640 1 -- 172.21.15.38:0/3679631185 wait complete. 2024-09-21T03:39:59.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:59.403+0000 7fd7c2bed640 1 Processor -- start 2024-09-21T03:39:59.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:59.403+0000 7fd7c2bed640 1 -- start start 2024-09-21T03:39:59.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:59.405+0000 7fd7c2bed640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd7bc0d1ed0 0x7fd7bc0a9450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:59.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:59.405+0000 7fd7c2bed640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd7bc1536e0 0x7fd7bc0cc090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:59.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:59.405+0000 7fd7c2bed640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7bc0ab420 0x7fd7bc0b75b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:59.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:59.405+0000 7fd7c2bed640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fd7bc167be0 con 0x7fd7bc0ab420 2024-09-21T03:39:59.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:59.405+0000 7fd7c2bed640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd7bc167a60 con 0x7fd7bc0d1ed0 2024-09-21T03:39:59.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:59.405+0000 7fd7c2bed640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fd7bc167d60 con 0x7fd7bc1536e0 2024-09-21T03:39:59.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:59.405+0000 7fd7c118a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7bc0ab420 0x7fd7bc0b75b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:59.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:59.405+0000 7fd7c118a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7bc0ab420 0x7fd7bc0b75b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:45414/0 (socket says 172.21.15.38:45414) 2024-09-21T03:39:59.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:59.405+0000 7fd7c118a640 1 -- 172.21.15.38:0/1646555359 learned_addr learned my addr 172.21.15.38:0/1646555359 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:39:59.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:59.405+0000 7fd7c0989640 1 --2- 172.21.15.38:0/1646555359 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd7bc0d1ed0 0x7fd7bc0a9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:59.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:59.405+0000 7fd7bbfff640 1 --2- 172.21.15.38:0/1646555359 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd7bc1536e0 0x7fd7bc0cc090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:59.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:59.405+0000 7fd7c118a640 1 -- 172.21.15.38:0/1646555359 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd7bc1536e0 msgr2=0x7fd7bc0cc090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:59.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:59.405+0000 7fd7c118a640 1 --2- 172.21.15.38:0/1646555359 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd7bc1536e0 0x7fd7bc0cc090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:59.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:59.405+0000 7fd7c118a640 1 -- 172.21.15.38:0/1646555359 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd7bc0d1ed0 msgr2=0x7fd7bc0a9450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:59.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:59.405+0000 7fd7c118a640 1 --2- 172.21.15.38:0/1646555359 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd7bc0d1ed0 0x7fd7bc0a9450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:59.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:59.405+0000 7fd7c118a640 1 -- 172.21.15.38:0/1646555359 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7bc0a9dc0 con 0x7fd7bc0ab420 2024-09-21T03:39:59.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:59.405+0000 7fd7c0989640 1 --2- 172.21.15.38:0/1646555359 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd7bc0d1ed0 0x7fd7bc0a9450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:39:59.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:59.405+0000 7fd7c118a640 1 --2- 172.21.15.38:0/1646555359 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7bc0ab420 0x7fd7bc0b75b0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7fd7b4006010 tx=0x7fd7b4031e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:59.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:59.406+0000 7fd7b97fa640 1 -- 172.21.15.38:0/1646555359 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7b403e070 con 0x7fd7bc0ab420 2024-09-21T03:39:59.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:59.406+0000 7fd7b97fa640 1 -- 172.21.15.38:0/1646555359 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd7b4004540 con 0x7fd7bc0ab420 2024-09-21T03:39:59.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:59.406+0000 7fd7c2bed640 1 -- 172.21.15.38:0/1646555359 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7bc0aa0a0 con 0x7fd7bc0ab420 2024-09-21T03:39:59.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:59.406+0000 7fd7b97fa640 1 -- 172.21.15.38:0/1646555359 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7b4043460 con 0x7fd7bc0ab420 2024-09-21T03:39:59.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:59.406+0000 7fd7c2bed640 1 -- 172.21.15.38:0/1646555359 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7bc0aa660 con 0x7fd7bc0ab420 2024-09-21T03:39:59.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:59.407+0000 7fd7c2bed640 1 -- 172.21.15.38:0/1646555359 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7bc0a9b30 con 0x7fd7bc0ab420 2024-09-21T03:39:59.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:59.407+0000 7fd7b97fa640 1 -- 172.21.15.38:0/1646555359 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fd7b4043600 con 0x7fd7bc0ab420 2024-09-21T03:39:59.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:59.408+0000 7fd7b97fa640 1 --2- 172.21.15.38:0/1646555359 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd7a0078b90 0x7fd7a007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:59.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:59.408+0000 7fd7b97fa640 1 -- 172.21.15.38:0/1646555359 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5249+0+0 (secure 0 0 0) 0x7fd7b40bc5d0 con 0x7fd7bc0ab420 2024-09-21T03:39:59.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:59.408+0000 7fd7c0989640 1 --2- 172.21.15.38:0/1646555359 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd7a0078b90 0x7fd7a007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:59.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:59.409+0000 7fd7c0989640 1 --2- 172.21.15.38:0/1646555359 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd7a0078b90 0x7fd7a007b0a0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fd7b00016a0 tx=0x7fd7b0009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:59.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:59.413+0000 7fd7b97fa640 1 -- 172.21.15.38:0/1646555359 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fd7b404a030 con 0x7fd7bc0ab420 2024-09-21T03:39:59.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:59.604+0000 7fd7c2bed640 1 -- 172.21.15.38:0/1646555359 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fd7bc0ad5d0 con 0x7fd7bc0ab420 2024-09-21T03:39:59.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:59.605+0000 7fd7b97fa640 1 -- 172.21.15.38:0/1646555359 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v90) ==== 84+0+65 (secure 0 0 0) 0x7fd7b4084b50 con 0x7fd7bc0ab420 2024-09-21T03:39:59.607 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:39:59.607 INFO:teuthology.orchestra.run.smithi038.stdout:[ 2024-09-21T03:39:59.607 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-21T03:39:59.607 INFO:teuthology.orchestra.run.smithi038.stdout: "poolnum": 1, 2024-09-21T03:39:59.607 INFO:teuthology.orchestra.run.smithi038.stdout: "poolname": ".mgr" 2024-09-21T03:39:59.607 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-21T03:39:59.607 INFO:teuthology.orchestra.run.smithi038.stdout:] 2024-09-21T03:39:59.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:59.607+0000 7fd7c2bed640 1 -- 172.21.15.38:0/1646555359 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd7a0078b90 msgr2=0x7fd7a007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:59.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:59.607+0000 7fd7c2bed640 1 --2- 172.21.15.38:0/1646555359 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd7a0078b90 0x7fd7a007b0a0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fd7b00016a0 tx=0x7fd7b0009040 comp rx=0 tx=0).stop 2024-09-21T03:39:59.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:59.607+0000 7fd7c2bed640 1 -- 172.21.15.38:0/1646555359 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7bc0ab420 msgr2=0x7fd7bc0b75b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:59.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:59.607+0000 7fd7c2bed640 1 --2- 172.21.15.38:0/1646555359 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7bc0ab420 0x7fd7bc0b75b0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7fd7b4006010 tx=0x7fd7b4031e20 comp rx=0 tx=0).stop 2024-09-21T03:39:59.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:59.607+0000 7fd7c2bed640 1 -- 172.21.15.38:0/1646555359 shutdown_connections 2024-09-21T03:39:59.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:59.607+0000 7fd7c2bed640 1 --2- 172.21.15.38:0/1646555359 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7bc0ab420 0x7fd7bc0b75b0 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:59.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:59.607+0000 7fd7c2bed640 1 --2- 172.21.15.38:0/1646555359 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd7a0078b90 0x7fd7a007b0a0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:59.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:59.607+0000 7fd7c2bed640 1 --2- 172.21.15.38:0/1646555359 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd7bc1536e0 0x7fd7bc0cc090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:59.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:59.607+0000 7fd7c2bed640 1 --2- 172.21.15.38:0/1646555359 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd7bc0d1ed0 0x7fd7bc0a9450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:59.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:59.607+0000 7fd7c2bed640 1 -- 172.21.15.38:0/1646555359 >> 172.21.15.38:0/1646555359 conn(0x7fd7bc0cef40 msgr2=0x7fd7bc0d22e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:59.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:59.607+0000 7fd7c2bed640 1 -- 172.21.15.38:0/1646555359 shutdown_connections 2024-09-21T03:39:59.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:59.607+0000 7fd7c2bed640 1 -- 172.21.15.38:0/1646555359 wait complete. 2024-09-21T03:39:59.620 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-21T03:39:59.621 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-21T03:39:59.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:59.740+0000 7f42e53a1640 1 -- 172.21.15.38:0/4267572597 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f42cc003a50 msgr2=0x7f42cc003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:59.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:59.740+0000 7f42e53a1640 1 --2- 172.21.15.38:0/4267572597 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f42cc003a50 0x7f42cc003ec0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f42d0009ed0 tx=0x7f42cc005830 comp rx=0 tx=0).stop 2024-09-21T03:39:59.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:59.740+0000 7f42e53a1640 1 -- 172.21.15.38:0/4267572597 shutdown_connections 2024-09-21T03:39:59.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:59.740+0000 7f42e53a1640 1 --2- 172.21.15.38:0/4267572597 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f42cc003a50 0x7f42cc003ec0 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:59.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:59.740+0000 7f42e53a1640 1 -- 172.21.15.38:0/4267572597 >> 172.21.15.38:0/4267572597 conn(0x7f42e00d4510 msgr2=0x7f42e00d4920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:59.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:59.740+0000 7f42e53a1640 1 -- 172.21.15.38:0/4267572597 shutdown_connections 2024-09-21T03:39:59.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:59.740+0000 7f42e53a1640 1 -- 172.21.15.38:0/4267572597 wait complete. 2024-09-21T03:39:59.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:59.742+0000 7f42e53a1640 1 Processor -- start 2024-09-21T03:39:59.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:59.742+0000 7f42e53a1640 1 -- start start 2024-09-21T03:39:59.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:59.743+0000 7f42e53a1640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f42cc003a50 0x7f42e00ca6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:59.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:59.743+0000 7f42e53a1640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f42e0152740 0x7f42e01f8c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:59.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:59.743+0000 7f42e53a1640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f42e0152b50 0x7f42e01f9140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:59.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:59.743+0000 7f42e53a1640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f42e00b7f70 con 0x7f42e0152b50 2024-09-21T03:39:59.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:59.743+0000 7f42e53a1640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f42e01621f0 con 0x7f42cc003a50 2024-09-21T03:39:59.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:59.743+0000 7f42e53a1640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f42e00b80f0 con 0x7f42e0152740 2024-09-21T03:39:59.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:59.743+0000 7f42df7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f42e0152b50 0x7f42e01f9140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:59.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:59.743+0000 7f42df7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f42e0152b50 0x7f42e01f9140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:45430/0 (socket says 172.21.15.38:45430) 2024-09-21T03:39:59.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:59.743+0000 7f42df7fe640 1 -- 172.21.15.38:0/3989256766 learned_addr learned my addr 172.21.15.38:0/3989256766 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:39:59.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:59.743+0000 7f42deffd640 1 --2- 172.21.15.38:0/3989256766 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f42cc003a50 0x7f42e00ca6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:59.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:59.743+0000 7f42de7fc640 1 --2- 172.21.15.38:0/3989256766 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f42e0152740 0x7f42e01f8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:59.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:59.743+0000 7f42df7fe640 1 -- 172.21.15.38:0/3989256766 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f42e0152740 msgr2=0x7f42e01f8c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:59.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:59.744+0000 7f42df7fe640 1 --2- 172.21.15.38:0/3989256766 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f42e0152740 0x7f42e01f8c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:59.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:59.744+0000 7f42df7fe640 1 -- 172.21.15.38:0/3989256766 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f42cc003a50 msgr2=0x7f42e00ca6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:59.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:59.744+0000 7f42df7fe640 1 --2- 172.21.15.38:0/3989256766 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f42cc003a50 0x7f42e00ca6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:59.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:59.744+0000 7f42df7fe640 1 -- 172.21.15.38:0/3989256766 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42e015f7b0 con 0x7f42e0152b50 2024-09-21T03:39:59.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:59.744+0000 7f42df7fe640 1 --2- 172.21.15.38:0/3989256766 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f42e0152b50 0x7f42e01f9140 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f42e00d43b0 tx=0x7f42d800e6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:59.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:59.744+0000 7f42d77fe640 1 -- 172.21.15.38:0/3989256766 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42d8011660 con 0x7f42e0152b50 2024-09-21T03:39:59.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:59.744+0000 7f42d77fe640 1 -- 172.21.15.38:0/3989256766 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f42d8017070 con 0x7f42e0152b50 2024-09-21T03:39:59.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:59.744+0000 7f42e53a1640 1 -- 172.21.15.38:0/3989256766 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42e00c7ef0 con 0x7f42e0152b50 2024-09-21T03:39:59.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:59.744+0000 7f42d77fe640 1 -- 172.21.15.38:0/3989256766 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42d8011b90 con 0x7f42e0152b50 2024-09-21T03:39:59.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:59.744+0000 7f42e53a1640 1 -- 172.21.15.38:0/3989256766 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42e00c84b0 con 0x7f42e0152b50 2024-09-21T03:39:59.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:59.745+0000 7f42e53a1640 1 -- 172.21.15.38:0/3989256766 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42e00cf370 con 0x7f42e0152b50 2024-09-21T03:39:59.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:59.746+0000 7f42d77fe640 1 -- 172.21.15.38:0/3989256766 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f42d801c460 con 0x7f42e0152b50 2024-09-21T03:39:59.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:59.746+0000 7f42d77fe640 1 --2- 172.21.15.38:0/3989256766 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f42c0078b90 0x7f42c007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:59.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:59.746+0000 7f42d77fe640 1 -- 172.21.15.38:0/3989256766 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5249+0+0 (secure 0 0 0) 0x7f42d809aaf0 con 0x7f42e0152b50 2024-09-21T03:39:59.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:59.746+0000 7f42deffd640 1 --2- 172.21.15.38:0/3989256766 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f42c0078b90 0x7f42c007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:59.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:59.747+0000 7f42deffd640 1 --2- 172.21.15.38:0/3989256766 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f42c0078b90 0x7f42c007b0a0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f42d0035000 tx=0x7f42d0004a40 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:59.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:59.751+0000 7f42d77fe640 1 -- 172.21.15.38:0/3989256766 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f42d8062bc0 con 0x7f42e0152b50 2024-09-21T03:39:59.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:39:59.940+0000 7f42e53a1640 1 -- 172.21.15.38:0/3989256766 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f42e00b9eb0 con 0x7f42e0152b50 2024-09-21T03:40:00.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:00.877+0000 7f42d77fe640 1 -- 172.21.15.38:0/3989256766 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v91) ==== 138+0+0 (secure 0 0 0) 0x7f42d80675f0 con 0x7f42e0152b50 2024-09-21T03:40:00.879 INFO:teuthology.orchestra.run.smithi038.stderr:pool 'cephfs_metadata' created 2024-09-21T03:40:00.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:00.882+0000 7f42e53a1640 1 -- 172.21.15.38:0/3989256766 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f42c0078b90 msgr2=0x7f42c007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:00.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:00.882+0000 7f42e53a1640 1 --2- 172.21.15.38:0/3989256766 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f42c0078b90 0x7f42c007b0a0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f42d0035000 tx=0x7f42d0004a40 comp rx=0 tx=0).stop 2024-09-21T03:40:00.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:00.882+0000 7f42e53a1640 1 -- 172.21.15.38:0/3989256766 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f42e0152b50 msgr2=0x7f42e01f9140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:00.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:00.882+0000 7f42e53a1640 1 --2- 172.21.15.38:0/3989256766 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f42e0152b50 0x7f42e01f9140 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f42e00d43b0 tx=0x7f42d800e6b0 comp rx=0 tx=0).stop 2024-09-21T03:40:00.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:00.883+0000 7f42e53a1640 1 -- 172.21.15.38:0/3989256766 shutdown_connections 2024-09-21T03:40:00.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:00.883+0000 7f42e53a1640 1 --2- 172.21.15.38:0/3989256766 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f42e0152b50 0x7f42e01f9140 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:00.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:00.883+0000 7f42e53a1640 1 --2- 172.21.15.38:0/3989256766 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f42c0078b90 0x7f42c007b0a0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:00.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:00.883+0000 7f42e53a1640 1 --2- 172.21.15.38:0/3989256766 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f42e0152740 0x7f42e01f8c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:00.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:00.883+0000 7f42e53a1640 1 --2- 172.21.15.38:0/3989256766 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f42cc003a50 0x7f42e00ca6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:00.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:00.883+0000 7f42e53a1640 1 -- 172.21.15.38:0/3989256766 >> 172.21.15.38:0/3989256766 conn(0x7f42e00d4510 msgr2=0x7f42e00b6720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:00.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:00.883+0000 7f42e53a1640 1 -- 172.21.15.38:0/3989256766 shutdown_connections 2024-09-21T03:40:00.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:00.883+0000 7f42e53a1640 1 -- 172.21.15.38:0/3989256766 wait complete. 2024-09-21T03:40:00.902 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-21T03:40:00.980 DEBUG:teuthology.orchestra.run.smithi038:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T03:40:00.983 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T03:40:01.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:01.032+0000 7fc6c52f2640 1 -- 172.21.15.38:0/252718208 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc6c0155230 msgr2=0x7fc6c00b3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:01.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:01.032+0000 7fc6c52f2640 1 --2- 172.21.15.38:0/252718208 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc6c0155230 0x7fc6c00b3800 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7fc6c00b5c60 tx=0x7fc6b802aa30 comp rx=0 tx=0).stop 2024-09-21T03:40:01.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:01.032+0000 7fc6c52f2640 1 -- 172.21.15.38:0/252718208 shutdown_connections 2024-09-21T03:40:01.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:01.032+0000 7fc6c52f2640 1 --2- 172.21.15.38:0/252718208 >> v2:172.21.15.195:3300/0 conn(0x7fc6c006dd40 0x7fc6c006e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:01.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:01.032+0000 7fc6c52f2640 1 --2- 172.21.15.38:0/252718208 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc6c0155230 0x7fc6c00b3800 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:01.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:01.032+0000 7fc6c52f2640 1 -- 172.21.15.38:0/252718208 >> 172.21.15.38:0/252718208 conn(0x7fc6c00d0a30 msgr2=0x7fc6c00d0e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:01.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:01.032+0000 7fc6c52f2640 1 -- 172.21.15.38:0/252718208 shutdown_connections 2024-09-21T03:40:01.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:01.032+0000 7fc6c52f2640 1 -- 172.21.15.38:0/252718208 wait complete. 2024-09-21T03:40:01.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:01.034+0000 7fc6c52f2640 1 Processor -- start 2024-09-21T03:40:01.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:01.034+0000 7fc6c52f2640 1 -- start start 2024-09-21T03:40:01.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:01.036+0000 7fc6c52f2640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc6c006dd40 0x7fc6c01f4560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:01.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:01.036+0000 7fc6c52f2640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc6c00701d0 0x7fc6c00abbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:01.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:01.036+0000 7fc6c52f2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc6c0155230 0x7fc6c0166880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:01.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:01.036+0000 7fc6c52f2640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fc6c0069a50 con 0x7fc6c0155230 2024-09-21T03:40:01.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:01.036+0000 7fc6c52f2640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fc6c00698d0 con 0x7fc6c006dd40 2024-09-21T03:40:01.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:01.036+0000 7fc6c52f2640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fc6c0069bd0 con 0x7fc6c00701d0 2024-09-21T03:40:01.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:01.036+0000 7fc6bf7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc6c0155230 0x7fc6c0166880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:01.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:01.036+0000 7fc6bf7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc6c0155230 0x7fc6c0166880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:45448/0 (socket says 172.21.15.38:45448) 2024-09-21T03:40:01.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:01.036+0000 7fc6bf7fe640 1 -- 172.21.15.38:0/2485760074 learned_addr learned my addr 172.21.15.38:0/2485760074 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:01.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:01.036+0000 7fc6beffd640 1 --2- 172.21.15.38:0/2485760074 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc6c006dd40 0x7fc6c01f4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:01.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:01.036+0000 7fc6be7fc640 1 --2- 172.21.15.38:0/2485760074 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc6c00701d0 0x7fc6c00abbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:01.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:01.036+0000 7fc6bf7fe640 1 -- 172.21.15.38:0/2485760074 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc6c00701d0 msgr2=0x7fc6c00abbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:01.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:01.036+0000 7fc6bf7fe640 1 --2- 172.21.15.38:0/2485760074 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc6c00701d0 0x7fc6c00abbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:01.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:01.036+0000 7fc6bf7fe640 1 -- 172.21.15.38:0/2485760074 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc6c006dd40 msgr2=0x7fc6c01f4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:01.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:01.036+0000 7fc6bf7fe640 1 --2- 172.21.15.38:0/2485760074 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc6c006dd40 0x7fc6c01f4560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:01.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:01.036+0000 7fc6bf7fe640 1 -- 172.21.15.38:0/2485760074 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6c01f4aa0 con 0x7fc6c0155230 2024-09-21T03:40:01.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:01.036+0000 7fc6bf7fe640 1 --2- 172.21.15.38:0/2485760074 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc6c0155230 0x7fc6c0166880 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7fc6b000d660 tx=0x7fc6b000db20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:01.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:01.036+0000 7fc6b77fe640 1 -- 172.21.15.38:0/2485760074 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6b0017070 con 0x7fc6c0155230 2024-09-21T03:40:01.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:01.037+0000 7fc6b77fe640 1 -- 172.21.15.38:0/2485760074 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc6b0005310 con 0x7fc6c0155230 2024-09-21T03:40:01.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:01.037+0000 7fc6c52f2640 1 -- 172.21.15.38:0/2485760074 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6c01592e0 con 0x7fc6c0155230 2024-09-21T03:40:01.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:01.037+0000 7fc6b77fe640 1 -- 172.21.15.38:0/2485760074 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6b0009820 con 0x7fc6c0155230 2024-09-21T03:40:01.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:01.037+0000 7fc6c52f2640 1 -- 172.21.15.38:0/2485760074 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6c01598a0 con 0x7fc6c0155230 2024-09-21T03:40:01.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:01.038+0000 7fc6c52f2640 1 -- 172.21.15.38:0/2485760074 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6c00b1b60 con 0x7fc6c0155230 2024-09-21T03:40:01.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:01.039+0000 7fc6b77fe640 1 -- 172.21.15.38:0/2485760074 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc6b00099c0 con 0x7fc6c0155230 2024-09-21T03:40:01.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:01.039+0000 7fc6b77fe640 1 --2- 172.21.15.38:0/2485760074 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc6a0078b90 0x7fc6a007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:01.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:01.039+0000 7fc6b77fe640 1 -- 172.21.15.38:0/2485760074 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5619+0+0 (secure 0 0 0) 0x7fc6b009b010 con 0x7fc6c0155230 2024-09-21T03:40:01.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:01.039+0000 7fc6beffd640 1 --2- 172.21.15.38:0/2485760074 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc6a0078b90 0x7fc6a007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:01.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:01.040+0000 7fc6beffd640 1 --2- 172.21.15.38:0/2485760074 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc6a0078b90 0x7fc6a007b0a0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fc6b8035000 tx=0x7fc6b803b040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:01.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:01.043+0000 7fc6b77fe640 1 -- 172.21.15.38:0/2485760074 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fc6c00b1b60 con 0x7fc6c0155230 2024-09-21T03:40:01.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:01.235+0000 7fc6c52f2640 1 -- 172.21.15.38:0/2485760074 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fc6c006e150 con 0x7fc6c0155230 2024-09-21T03:40:01.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:01.885+0000 7fc6b77fe640 1 -- 172.21.15.38:0/2485760074 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v92) ==== 170+0+0 (secure 0 0 0) 0x7fc6c01f4aa0 con 0x7fc6c0155230 2024-09-21T03:40:01.887 INFO:teuthology.orchestra.run.smithi038.stderr:pool 'cephfs_data' created 2024-09-21T03:40:01.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:01.889+0000 7fc6c52f2640 1 -- 172.21.15.38:0/2485760074 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc6a0078b90 msgr2=0x7fc6a007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:01.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:01.889+0000 7fc6c52f2640 1 --2- 172.21.15.38:0/2485760074 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc6a0078b90 0x7fc6a007b0a0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fc6b8035000 tx=0x7fc6b803b040 comp rx=0 tx=0).stop 2024-09-21T03:40:01.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:01.889+0000 7fc6c52f2640 1 -- 172.21.15.38:0/2485760074 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc6c0155230 msgr2=0x7fc6c0166880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:01.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:01.889+0000 7fc6c52f2640 1 --2- 172.21.15.38:0/2485760074 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc6c0155230 0x7fc6c0166880 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7fc6b000d660 tx=0x7fc6b000db20 comp rx=0 tx=0).stop 2024-09-21T03:40:01.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:01.891+0000 7fc6c52f2640 1 -- 172.21.15.38:0/2485760074 shutdown_connections 2024-09-21T03:40:01.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:01.891+0000 7fc6c52f2640 1 --2- 172.21.15.38:0/2485760074 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc6c0155230 0x7fc6c0166880 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:01.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:01.891+0000 7fc6c52f2640 1 --2- 172.21.15.38:0/2485760074 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc6a0078b90 0x7fc6a007b0a0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:01.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:01.891+0000 7fc6c52f2640 1 --2- 172.21.15.38:0/2485760074 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc6c00701d0 0x7fc6c00abbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:01.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:01.891+0000 7fc6c52f2640 1 --2- 172.21.15.38:0/2485760074 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc6c006dd40 0x7fc6c01f4560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:01.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:01.891+0000 7fc6c52f2640 1 -- 172.21.15.38:0/2485760074 >> 172.21.15.38:0/2485760074 conn(0x7fc6c00d0a30 msgr2=0x7fc6c015e6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:01.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:01.891+0000 7fc6c52f2640 1 -- 172.21.15.38:0/2485760074 shutdown_connections 2024-09-21T03:40:01.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:01.891+0000 7fc6c52f2640 1 -- 172.21.15.38:0/2485760074 wait complete. 2024-09-21T03:40:01.918 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-21T03:40:02.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:02.079+0000 7fd650a9b640 1 -- 172.21.15.38:0/4233259440 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd638003c10 msgr2=0x7fd638004060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:02.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:02.079+0000 7fd650a9b640 1 --2- 172.21.15.38:0/4233259440 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd638003c10 0x7fd638004060 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fd63c001800 tx=0x7fd638005830 comp rx=0 tx=0).stop 2024-09-21T03:40:02.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:02.079+0000 7fd64ad9d640 1 -- 172.21.15.38:0/4233259440 reap_dead start 2024-09-21T03:40:02.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:02.079+0000 7fd650a9b640 1 -- 172.21.15.38:0/4233259440 shutdown_connections 2024-09-21T03:40:02.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:02.079+0000 7fd650a9b640 1 -- 172.21.15.38:0/4233259440 >> 172.21.15.38:0/4233259440 conn(0x7fd64c0cdca0 msgr2=0x7fd64c0ce0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:02.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:02.079+0000 7fd650a9b640 1 -- 172.21.15.38:0/4233259440 shutdown_connections 2024-09-21T03:40:02.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:02.079+0000 7fd650a9b640 1 -- 172.21.15.38:0/4233259440 wait complete. 2024-09-21T03:40:02.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:02.081+0000 7fd650a9b640 1 Processor -- start 2024-09-21T03:40:02.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:02.081+0000 7fd650a9b640 1 -- start start 2024-09-21T03:40:02.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:02.082+0000 7fd650a9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd638003c10 0x7fd64c1f4f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:02.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:02.082+0000 7fd650a9b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd64c09e3e0 0x7fd64c0cfe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:02.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:02.082+0000 7fd650a9b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd64c09e7f0 0x7fd64c0d0380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:02.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:02.082+0000 7fd650a9b640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fd64c15fb80 con 0x7fd638003c10 2024-09-21T03:40:02.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:02.082+0000 7fd650a9b640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd64c15fa00 con 0x7fd64c09e7f0 2024-09-21T03:40:02.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:02.082+0000 7fd650a9b640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fd64c15fd00 con 0x7fd64c09e3e0 2024-09-21T03:40:02.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:02.082+0000 7fd64a59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd638003c10 0x7fd64c1f4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:02.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:02.082+0000 7fd64a59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd638003c10 0x7fd64c1f4f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:45468/0 (socket says 172.21.15.38:45468) 2024-09-21T03:40:02.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:02.082+0000 7fd64a59c640 1 -- 172.21.15.38:0/3832629929 learned_addr learned my addr 172.21.15.38:0/3832629929 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:02.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:02.083+0000 7fd649d9b640 1 --2- 172.21.15.38:0/3832629929 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd64c09e3e0 0x7fd64c0cfe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:02.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:02.083+0000 7fd64ad9d640 1 --2- 172.21.15.38:0/3832629929 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd64c09e7f0 0x7fd64c0d0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:02.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:02.083+0000 7fd64a59c640 1 -- 172.21.15.38:0/3832629929 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd64c09e3e0 msgr2=0x7fd64c0cfe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:02.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:02.083+0000 7fd64a59c640 1 --2- 172.21.15.38:0/3832629929 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd64c09e3e0 0x7fd64c0cfe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:02.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:02.083+0000 7fd64a59c640 1 -- 172.21.15.38:0/3832629929 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd64c09e7f0 msgr2=0x7fd64c0d0380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:02.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:02.083+0000 7fd64a59c640 1 --2- 172.21.15.38:0/3832629929 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd64c09e7f0 0x7fd64c0d0380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:02.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:02.083+0000 7fd64a59c640 1 -- 172.21.15.38:0/3832629929 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd64c1f54d0 con 0x7fd638003c10 2024-09-21T03:40:02.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:02.083+0000 7fd64a59c640 1 --2- 172.21.15.38:0/3832629929 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd638003c10 0x7fd64c1f4f90 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fd63c001920 tx=0x7fd63c0060e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:02.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:02.083+0000 7fd642ffd640 1 -- 172.21.15.38:0/3832629929 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd63c003a00 con 0x7fd638003c10 2024-09-21T03:40:02.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:02.083+0000 7fd642ffd640 1 -- 172.21.15.38:0/3832629929 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd63c003c90 con 0x7fd638003c10 2024-09-21T03:40:02.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:02.083+0000 7fd650a9b640 1 -- 172.21.15.38:0/3832629929 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd64c1447c0 con 0x7fd638003c10 2024-09-21T03:40:02.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:02.083+0000 7fd642ffd640 1 -- 172.21.15.38:0/3832629929 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd63c0336e0 con 0x7fd638003c10 2024-09-21T03:40:02.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:02.083+0000 7fd650a9b640 1 -- 172.21.15.38:0/3832629929 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd64c1470a0 con 0x7fd638003c10 2024-09-21T03:40:02.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:02.085+0000 7fd650a9b640 1 -- 172.21.15.38:0/3832629929 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd64c06f410 con 0x7fd638003c10 2024-09-21T03:40:02.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:02.085+0000 7fd642ffd640 1 -- 172.21.15.38:0/3832629929 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fd63c045020 con 0x7fd638003c10 2024-09-21T03:40:02.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:02.086+0000 7fd642ffd640 1 --2- 172.21.15.38:0/3832629929 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd62c078bc0 0x7fd62c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:02.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:02.086+0000 7fd642ffd640 1 -- 172.21.15.38:0/3832629929 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 6001+0+0 (secure 0 0 0) 0x7fd63c0bad20 con 0x7fd638003c10 2024-09-21T03:40:02.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:02.086+0000 7fd649d9b640 1 --2- 172.21.15.38:0/3832629929 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd62c078bc0 0x7fd62c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:02.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:02.087+0000 7fd649d9b640 1 --2- 172.21.15.38:0/3832629929 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd62c078bc0 0x7fd62c07b0d0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fd64c09c170 tx=0x7fd6340011c0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:02.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:02.091+0000 7fd642ffd640 1 -- 172.21.15.38:0/3832629929 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fd64c1f54d0 con 0x7fd638003c10 2024-09-21T03:40:02.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:02.317+0000 7fd650a9b640 1 -- 172.21.15.38:0/3832629929 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fd64c074120 con 0x7fd638003c10 2024-09-21T03:40:02.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:02.886+0000 7fd642ffd640 1 -- 172.21.15.38:0/3832629929 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 13 and data pool 14 v79) ==== 174+0+0 (secure 0 0 0) 0x7fd63c0829d0 con 0x7fd638003c10 2024-09-21T03:40:02.888 INFO:teuthology.orchestra.run.smithi038.stderr:new fs with metadata pool 13 and data pool 14 2024-09-21T03:40:02.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:02.889+0000 7fd650a9b640 1 -- 172.21.15.38:0/3832629929 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd62c078bc0 msgr2=0x7fd62c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:02.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:02.889+0000 7fd650a9b640 1 --2- 172.21.15.38:0/3832629929 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd62c078bc0 0x7fd62c07b0d0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fd64c09c170 tx=0x7fd6340011c0 comp rx=0 tx=0).stop 2024-09-21T03:40:02.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:02.889+0000 7fd650a9b640 1 -- 172.21.15.38:0/3832629929 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd638003c10 msgr2=0x7fd64c1f4f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:02.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:02.889+0000 7fd650a9b640 1 --2- 172.21.15.38:0/3832629929 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd638003c10 0x7fd64c1f4f90 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fd63c001920 tx=0x7fd63c0060e0 comp rx=0 tx=0).stop 2024-09-21T03:40:02.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:02.892+0000 7fd650a9b640 1 -- 172.21.15.38:0/3832629929 shutdown_connections 2024-09-21T03:40:02.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:02.892+0000 7fd650a9b640 1 --2- 172.21.15.38:0/3832629929 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd64c09e7f0 0x7fd64c0d0380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:02.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:02.892+0000 7fd650a9b640 1 --2- 172.21.15.38:0/3832629929 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd62c078bc0 0x7fd62c07b0d0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:02.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:02.892+0000 7fd650a9b640 1 --2- 172.21.15.38:0/3832629929 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd64c09e3e0 0x7fd64c0cfe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:02.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:02.893+0000 7fd650a9b640 1 --2- 172.21.15.38:0/3832629929 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd638003c10 0x7fd64c1f4f90 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:02.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:02.893+0000 7fd650a9b640 1 -- 172.21.15.38:0/3832629929 >> 172.21.15.38:0/3832629929 conn(0x7fd64c0cdca0 msgr2=0x7fd64c071420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:02.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:02.896+0000 7fd650a9b640 1 -- 172.21.15.38:0/3832629929 shutdown_connections 2024-09-21T03:40:02.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:02.896+0000 7fd650a9b640 1 -- 172.21.15.38:0/3832629929 wait complete. 2024-09-21T03:40:02.923 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T03:40:03.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:03.040+0000 7f73493da640 1 -- 172.21.15.38:0/333146988 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7344162590 msgr2=0x7f73441629a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:03.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:03.040+0000 7f73493da640 1 --2- 172.21.15.38:0/333146988 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7344162590 0x7f73441629a0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f732c009900 tx=0x7f732c02a610 comp rx=0 tx=0).stop 2024-09-21T03:40:03.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:03.040+0000 7f73493da640 1 -- 172.21.15.38:0/333146988 shutdown_connections 2024-09-21T03:40:03.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:03.040+0000 7f73493da640 1 --2- 172.21.15.38:0/333146988 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f73440b2e60 0x7f73440b3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:03.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:03.040+0000 7f73493da640 1 --2- 172.21.15.38:0/333146988 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7344162590 0x7f73441629a0 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:03.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:03.040+0000 7f73493da640 1 -- 172.21.15.38:0/333146988 >> 172.21.15.38:0/333146988 conn(0x7f73440d4440 msgr2=0x7f73440d4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:03.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:03.040+0000 7f73493da640 1 -- 172.21.15.38:0/333146988 shutdown_connections 2024-09-21T03:40:03.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:03.040+0000 7f73493da640 1 -- 172.21.15.38:0/333146988 wait complete. 2024-09-21T03:40:03.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:03.042+0000 7f73493da640 1 Processor -- start 2024-09-21T03:40:03.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:03.042+0000 7f73493da640 1 -- start start 2024-09-21T03:40:03.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:03.043+0000 7f73493da640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f73440b2e60 0x7f7344157230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:03.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:03.043+0000 7f73493da640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f73440cfe00 0x7f73441f9110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:03.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:03.043+0000 7f73493da640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7344162590 0x7f73441e8360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:03.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:03.043+0000 7f73493da640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f73440b3b40 con 0x7f73440b2e60 2024-09-21T03:40:03.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:03.043+0000 7f73493da640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f73440b39c0 con 0x7f73440cfe00 2024-09-21T03:40:03.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:03.043+0000 7f73493da640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f73440cf590 con 0x7f7344162590 2024-09-21T03:40:03.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:03.043+0000 7f7342ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f73440b2e60 0x7f7344157230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:03.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:03.043+0000 7f7342ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f73440b2e60 0x7f7344157230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:55498/0 (socket says 172.21.15.38:55498) 2024-09-21T03:40:03.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:03.043+0000 7f7342ffd640 1 -- 172.21.15.38:0/822863520 learned_addr learned my addr 172.21.15.38:0/822863520 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:03.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:03.044+0000 7f733bfff640 1 --2- 172.21.15.38:0/822863520 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f73440cfe00 0x7f73441f9110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:03.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:03.044+0000 7f73437fe640 1 --2- 172.21.15.38:0/822863520 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7344162590 0x7f73441e8360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:03.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:03.044+0000 7f7342ffd640 1 -- 172.21.15.38:0/822863520 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7344162590 msgr2=0x7f73441e8360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:03.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:03.044+0000 7f7342ffd640 1 --2- 172.21.15.38:0/822863520 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7344162590 0x7f73441e8360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:03.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:03.044+0000 7f7342ffd640 1 -- 172.21.15.38:0/822863520 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f73440cfe00 msgr2=0x7f73441f9110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:03.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:03.044+0000 7f7342ffd640 1 --2- 172.21.15.38:0/822863520 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f73440cfe00 0x7f73441f9110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:03.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:03.044+0000 7f7342ffd640 1 -- 172.21.15.38:0/822863520 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73441f5b20 con 0x7f73440b2e60 2024-09-21T03:40:03.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:03.044+0000 7f7342ffd640 1 --2- 172.21.15.38:0/822863520 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f73440b2e60 0x7f7344157230 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f733400c3a0 tx=0x7f733400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:03.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:03.044+0000 7f733b7fe640 1 -- 172.21.15.38:0/822863520 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7334017070 con 0x7f73440b2e60 2024-09-21T03:40:03.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:03.044+0000 7f733b7fe640 1 -- 172.21.15.38:0/822863520 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f73340053c0 con 0x7f73440b2e60 2024-09-21T03:40:03.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:03.044+0000 7f73493da640 1 -- 172.21.15.38:0/822863520 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73441f98d0 con 0x7f73440b2e60 2024-09-21T03:40:03.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:03.044+0000 7f733b7fe640 1 -- 172.21.15.38:0/822863520 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7334003850 con 0x7f73440b2e60 2024-09-21T03:40:03.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:03.044+0000 7f73493da640 1 -- 172.21.15.38:0/822863520 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73441f9e90 con 0x7f73440b2e60 2024-09-21T03:40:03.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:03.045+0000 7f73493da640 1 -- 172.21.15.38:0/822863520 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73440b3270 con 0x7f73440b2e60 2024-09-21T03:40:03.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:03.046+0000 7f733b7fe640 1 -- 172.21.15.38:0/822863520 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f733401a070 con 0x7f73440b2e60 2024-09-21T03:40:03.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:03.047+0000 7f733b7fe640 1 --2- 172.21.15.38:0/822863520 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7324078b90 0x7f732407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:03.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:03.047+0000 7f733b7fe640 1 -- 172.21.15.38:0/822863520 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6101+0+0 (secure 0 0 0) 0x7f733409a1b0 con 0x7f73440b2e60 2024-09-21T03:40:03.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:03.047+0000 7f733bfff640 1 --2- 172.21.15.38:0/822863520 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7324078b90 0x7f732407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:03.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:03.047+0000 7f733bfff640 1 --2- 172.21.15.38:0/822863520 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7324078b90 0x7f732407b0a0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f732c033000 tx=0x7f732c004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:03.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:03.055+0000 7f733b7fe640 1 -- 172.21.15.38:0/822863520 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f73441f5b20 con 0x7f73440b2e60 2024-09-21T03:40:03.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:03.246+0000 7f73493da640 1 -- 172.21.15.38:0/822863520 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f73440d48f0 con 0x7f73440b2e60 2024-09-21T03:40:03.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:03.247+0000 7f733b7fe640 1 -- 172.21.15.38:0/822863520 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+17030 (secure 0 0 0) 0x7f7344162ee0 con 0x7f73440b2e60 2024-09-21T03:40:03.249 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:40:03.249 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":93,"fsid":"9d50d66f-656b-4fe2-8e3b-fe53b79c4d43","created":"2024-09-21T03:33:10.677865+0000","modified":"2024-09-21T03:40:02.319351+0000","last_up_change":"2024-09-21T03:33:24.023421+0000","last_in_change":"2024-09-21T03:33:14.296193+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T03:33:22.992573+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-09-21T03:39:59.945436+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-09-21T03:40:01.239789+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dbab183f-7e63-48b8-960a-ae55fd5bf565","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6801","nonce":2381639433}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6803","nonce":2381639433}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6807","nonce":2381639433}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6805","nonce":2381639433}]},"public_addr":"172.21.15.38:6801/2381639433","cluster_addr":"172.21.15.38:6803/2381639433","heartbeat_back_addr":"172.21.15.38:6807/2381639433","heartbeat_front_addr":"172.21.15.38:6805/2381639433","state":["exists","up"]},{"osd":1,"uuid":"fa1c0e8a-ebc1-4359-8146-bd5342abbc16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6817","nonce":1491818732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6819","nonce":1491818732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6823","nonce":1491818732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6821","nonce":1491818732}]},"public_addr":"172.21.15.38:6817/1491818732","cluster_addr":"172.21.15.38:6819/1491818732","heartbeat_back_addr":"172.21.15.38:6823/1491818732","heartbeat_front_addr":"172.21.15.38:6821/1491818732","state":["exists","up"]},{"osd":2,"uuid":"2ed3cd2d-385d-4fc6-9869-214552a84803","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6825","nonce":2577858902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6827","nonce":2577858902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6831","nonce":2577858902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6829","nonce":2577858902}]},"public_addr":"172.21.15.38:6825/2577858902","cluster_addr":"172.21.15.38:6827/2577858902","heartbeat_back_addr":"172.21.15.38:6831/2577858902","heartbeat_front_addr":"172.21.15.38:6829/2577858902","state":["exists","up"]},{"osd":3,"uuid":"62d1d95e-919d-4250-be71-1d34cfa70dbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6809","nonce":1286810294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6811","nonce":1286810294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6815","nonce":1286810294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6813","nonce":1286810294}]},"public_addr":"172.21.15.38:6809/1286810294","cluster_addr":"172.21.15.38:6811/1286810294","heartbeat_back_addr":"172.21.15.38:6815/1286810294","heartbeat_front_addr":"172.21.15.38:6813/1286810294","state":["exists","up"]},{"osd":4,"uuid":"7c9f9314-99d7-4d73-8392-37809a9f54b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6803","nonce":3401837571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6805","nonce":3401837571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6809","nonce":3401837571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6807","nonce":3401837571}]},"public_addr":"172.21.15.195:6803/3401837571","cluster_addr":"172.21.15.195:6805/3401837571","heartbeat_back_addr":"172.21.15.195:6809/3401837571","heartbeat_front_addr":"172.21.15.195:6807/3401837571","state":["exists","up"]},{"osd":5,"uuid":"bde4fe9f-8414-4bba-bf4a-43973d1976bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6811","nonce":2080960180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6813","nonce":2080960180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6817","nonce":2080960180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6815","nonce":2080960180}]},"public_addr":"172.21.15.195:6811/2080960180","cluster_addr":"172.21.15.195:6813/2080960180","heartbeat_back_addr":"172.21.15.195:6817/2080960180","heartbeat_front_addr":"172.21.15.195:6815/2080960180","state":["exists","up"]},{"osd":6,"uuid":"6ee6d7f4-1a5f-4a4c-ac16-48fe05f6a002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6827","nonce":4055825675}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6829","nonce":4055825675}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6833","nonce":4055825675}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6831","nonce":4055825675}]},"public_addr":"172.21.15.195:6827/4055825675","cluster_addr":"172.21.15.195:6829/4055825675","heartbeat_back_addr":"172.21.15.195:6833/4055825675","heartbeat_front_addr":"172.21.15.195:6831/4055825675","state":["exists","up"]},{"osd":7,"uuid":"efeb5f51-329a-4484-8290-92e7d3503d05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6819","nonce":1606106342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6821","nonce":1606106342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6825","nonce":1606106342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6823","nonce":1606106342}]},"public_addr":"172.21.15.195:6819/1606106342","cluster_addr":"172.21.15.195:6821/1606106342","heartbeat_back_addr":"172.21.15.195:6825/1606106342","heartbeat_front_addr":"172.21.15.195:6823/1606106342","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.901343+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:21.115003+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.901623+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.835238+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.132199+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:19.817560+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.829690+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:21.087300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T03:40:03.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:03.249+0000 7f73493da640 1 -- 172.21.15.38:0/822863520 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7324078b90 msgr2=0x7f732407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:03.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:03.249+0000 7f73493da640 1 --2- 172.21.15.38:0/822863520 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7324078b90 0x7f732407b0a0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f732c033000 tx=0x7f732c004730 comp rx=0 tx=0).stop 2024-09-21T03:40:03.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:03.249+0000 7f73493da640 1 -- 172.21.15.38:0/822863520 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f73440b2e60 msgr2=0x7f7344157230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:03.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:03.249+0000 7f73493da640 1 --2- 172.21.15.38:0/822863520 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f73440b2e60 0x7f7344157230 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f733400c3a0 tx=0x7f733400c860 comp rx=0 tx=0).stop 2024-09-21T03:40:03.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:03.249+0000 7f73493da640 1 -- 172.21.15.38:0/822863520 shutdown_connections 2024-09-21T03:40:03.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:03.249+0000 7f73493da640 1 --2- 172.21.15.38:0/822863520 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7324078b90 0x7f732407b0a0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:03.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:03.249+0000 7f73493da640 1 --2- 172.21.15.38:0/822863520 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7344162590 0x7f73441e8360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:03.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:03.249+0000 7f73493da640 1 --2- 172.21.15.38:0/822863520 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f73440cfe00 0x7f73441f9110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:03.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:03.249+0000 7f73493da640 1 --2- 172.21.15.38:0/822863520 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f73440b2e60 0x7f7344157230 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:03.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:03.249+0000 7f73493da640 1 -- 172.21.15.38:0/822863520 >> 172.21.15.38:0/822863520 conn(0x7f73440d4440 msgr2=0x7f734406e340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:03.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:03.249+0000 7f73493da640 1 -- 172.21.15.38:0/822863520 shutdown_connections 2024-09-21T03:40:03.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:03.249+0000 7f73493da640 1 -- 172.21.15.38:0/822863520 wait complete. 2024-09-21T03:40:03.263 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T03:40:03.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:03.378+0000 7f4869bba640 1 -- 172.21.15.38:0/150410601 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f484c005350 msgr2=0x7f484c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:03.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:03.378+0000 7f4869bba640 1 --2- 172.21.15.38:0/150410601 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f484c005350 0x7f484c00b9c0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f485c005fd0 tx=0x7f484c005830 comp rx=0 tx=0).stop 2024-09-21T03:40:03.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:03.378+0000 7f4869bba640 1 -- 172.21.15.38:0/150410601 shutdown_connections 2024-09-21T03:40:03.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:03.378+0000 7f4869bba640 1 --2- 172.21.15.38:0/150410601 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f484c005350 0x7f484c00b9c0 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:03.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:03.378+0000 7f4869bba640 1 -- 172.21.15.38:0/150410601 >> 172.21.15.38:0/150410601 conn(0x7f48640cca40 msgr2=0x7f48640cce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:03.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:03.378+0000 7f4869bba640 1 -- 172.21.15.38:0/150410601 shutdown_connections 2024-09-21T03:40:03.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:03.378+0000 7f4869bba640 1 -- 172.21.15.38:0/150410601 wait complete. 2024-09-21T03:40:03.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:03.380+0000 7f4869bba640 1 Processor -- start 2024-09-21T03:40:03.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:03.380+0000 7f4869bba640 1 -- start start 2024-09-21T03:40:03.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:03.382+0000 7f4869bba640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f484c005350 0x7f48640c7a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:03.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:03.382+0000 7f4869bba640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f486409e380 0x7f486406ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:03.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:03.382+0000 7f4869bba640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f486406f170 0x7f486415d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:03.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:03.382+0000 7f4869bba640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f486415fe50 con 0x7f486409e380 2024-09-21T03:40:03.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:03.382+0000 7f4869bba640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f486415fcd0 con 0x7f484c005350 2024-09-21T03:40:03.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:03.382+0000 7f4869bba640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f48640cefa0 con 0x7f486406f170 2024-09-21T03:40:03.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:03.382+0000 7f4862ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f486409e380 0x7f486406ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:03.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:03.382+0000 7f4862ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f486409e380 0x7f486406ec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:55528/0 (socket says 172.21.15.38:55528) 2024-09-21T03:40:03.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:03.382+0000 7f4862ffd640 1 -- 172.21.15.38:0/1239890328 learned_addr learned my addr 172.21.15.38:0/1239890328 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:03.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:03.382+0000 7f48637fe640 1 --2- 172.21.15.38:0/1239890328 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f484c005350 0x7f48640c7a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:03.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:03.382+0000 7f4863fff640 1 --2- 172.21.15.38:0/1239890328 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f486406f170 0x7f486415d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:03.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:03.382+0000 7f4862ffd640 1 -- 172.21.15.38:0/1239890328 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f486406f170 msgr2=0x7f486415d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:03.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:03.382+0000 7f4862ffd640 1 --2- 172.21.15.38:0/1239890328 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f486406f170 0x7f486415d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:03.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:03.382+0000 7f4862ffd640 1 -- 172.21.15.38:0/1239890328 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f484c005350 msgr2=0x7f48640c7a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:03.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:03.382+0000 7f4862ffd640 1 --2- 172.21.15.38:0/1239890328 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f484c005350 0x7f48640c7a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:03.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:03.382+0000 7f4862ffd640 1 -- 172.21.15.38:0/1239890328 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f486406f580 con 0x7f486409e380 2024-09-21T03:40:03.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:03.382+0000 7f48637fe640 1 --2- 172.21.15.38:0/1239890328 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f484c005350 0x7f48640c7a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:40:03.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:03.382+0000 7f4862ffd640 1 --2- 172.21.15.38:0/1239890328 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f486409e380 0x7f486406ec30 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f485800e590 tx=0x7f485800ea50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:03.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:03.383+0000 7f484bfff640 1 -- 172.21.15.38:0/1239890328 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4858015070 con 0x7f486409e380 2024-09-21T03:40:03.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:03.383+0000 7f484bfff640 1 -- 172.21.15.38:0/1239890328 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f485800c9e0 con 0x7f486409e380 2024-09-21T03:40:03.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:03.383+0000 7f484bfff640 1 -- 172.21.15.38:0/1239890328 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f485800ce30 con 0x7f486409e380 2024-09-21T03:40:03.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:03.383+0000 7f4869bba640 1 -- 172.21.15.38:0/1239890328 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f486415ce80 con 0x7f486409e380 2024-09-21T03:40:03.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:03.383+0000 7f4869bba640 1 -- 172.21.15.38:0/1239890328 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f486415d120 con 0x7f486409e380 2024-09-21T03:40:03.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:03.384+0000 7f4869bba640 1 -- 172.21.15.38:0/1239890328 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f486406acf0 con 0x7f486409e380 2024-09-21T03:40:03.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:03.384+0000 7f484bfff640 1 -- 172.21.15.38:0/1239890328 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f4858018070 con 0x7f486409e380 2024-09-21T03:40:03.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:03.385+0000 7f484bfff640 1 --2- 172.21.15.38:0/1239890328 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4850078b90 0x7f485007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:03.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:03.385+0000 7f484bfff640 1 -- 172.21.15.38:0/1239890328 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6101+0+0 (secure 0 0 0) 0x7f485809b430 con 0x7f486409e380 2024-09-21T03:40:03.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:03.385+0000 7f48637fe640 1 --2- 172.21.15.38:0/1239890328 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4850078b90 0x7f485007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:03.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:03.386+0000 7f48637fe640 1 --2- 172.21.15.38:0/1239890328 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4850078b90 0x7f485007b0a0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f4854003830 tx=0x7f4854006c20 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:03.391 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:03.389+0000 7f484bfff640 1 -- 172.21.15.38:0/1239890328 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f486406f580 con 0x7f486409e380 2024-09-21T03:40:03.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:03.576+0000 7f4869bba640 1 -- 172.21.15.38:0/1239890328 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f48640caab0 con 0x7f486409e380 2024-09-21T03:40:03.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:03.577+0000 7f484bfff640 1 -- 172.21.15.38:0/1239890328 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+17030 (secure 0 0 0) 0x7f4858063640 con 0x7f486409e380 2024-09-21T03:40:03.579 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:40:03.579 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":93,"fsid":"9d50d66f-656b-4fe2-8e3b-fe53b79c4d43","created":"2024-09-21T03:33:10.677865+0000","modified":"2024-09-21T03:40:02.319351+0000","last_up_change":"2024-09-21T03:33:24.023421+0000","last_in_change":"2024-09-21T03:33:14.296193+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T03:33:22.992573+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-09-21T03:39:59.945436+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-09-21T03:40:01.239789+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dbab183f-7e63-48b8-960a-ae55fd5bf565","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6801","nonce":2381639433}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6803","nonce":2381639433}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6807","nonce":2381639433}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6805","nonce":2381639433}]},"public_addr":"172.21.15.38:6801/2381639433","cluster_addr":"172.21.15.38:6803/2381639433","heartbeat_back_addr":"172.21.15.38:6807/2381639433","heartbeat_front_addr":"172.21.15.38:6805/2381639433","state":["exists","up"]},{"osd":1,"uuid":"fa1c0e8a-ebc1-4359-8146-bd5342abbc16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6817","nonce":1491818732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6819","nonce":1491818732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6823","nonce":1491818732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6821","nonce":1491818732}]},"public_addr":"172.21.15.38:6817/1491818732","cluster_addr":"172.21.15.38:6819/1491818732","heartbeat_back_addr":"172.21.15.38:6823/1491818732","heartbeat_front_addr":"172.21.15.38:6821/1491818732","state":["exists","up"]},{"osd":2,"uuid":"2ed3cd2d-385d-4fc6-9869-214552a84803","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6825","nonce":2577858902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6827","nonce":2577858902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6831","nonce":2577858902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6829","nonce":2577858902}]},"public_addr":"172.21.15.38:6825/2577858902","cluster_addr":"172.21.15.38:6827/2577858902","heartbeat_back_addr":"172.21.15.38:6831/2577858902","heartbeat_front_addr":"172.21.15.38:6829/2577858902","state":["exists","up"]},{"osd":3,"uuid":"62d1d95e-919d-4250-be71-1d34cfa70dbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6809","nonce":1286810294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6811","nonce":1286810294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6815","nonce":1286810294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6813","nonce":1286810294}]},"public_addr":"172.21.15.38:6809/1286810294","cluster_addr":"172.21.15.38:6811/1286810294","heartbeat_back_addr":"172.21.15.38:6815/1286810294","heartbeat_front_addr":"172.21.15.38:6813/1286810294","state":["exists","up"]},{"osd":4,"uuid":"7c9f9314-99d7-4d73-8392-37809a9f54b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6803","nonce":3401837571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6805","nonce":3401837571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6809","nonce":3401837571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6807","nonce":3401837571}]},"public_addr":"172.21.15.195:6803/3401837571","cluster_addr":"172.21.15.195:6805/3401837571","heartbeat_back_addr":"172.21.15.195:6809/3401837571","heartbeat_front_addr":"172.21.15.195:6807/3401837571","state":["exists","up"]},{"osd":5,"uuid":"bde4fe9f-8414-4bba-bf4a-43973d1976bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6811","nonce":2080960180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6813","nonce":2080960180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6817","nonce":2080960180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6815","nonce":2080960180}]},"public_addr":"172.21.15.195:6811/2080960180","cluster_addr":"172.21.15.195:6813/2080960180","heartbeat_back_addr":"172.21.15.195:6817/2080960180","heartbeat_front_addr":"172.21.15.195:6815/2080960180","state":["exists","up"]},{"osd":6,"uuid":"6ee6d7f4-1a5f-4a4c-ac16-48fe05f6a002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6827","nonce":4055825675}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6829","nonce":4055825675}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6833","nonce":4055825675}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6831","nonce":4055825675}]},"public_addr":"172.21.15.195:6827/4055825675","cluster_addr":"172.21.15.195:6829/4055825675","heartbeat_back_addr":"172.21.15.195:6833/4055825675","heartbeat_front_addr":"172.21.15.195:6831/4055825675","state":["exists","up"]},{"osd":7,"uuid":"efeb5f51-329a-4484-8290-92e7d3503d05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6819","nonce":1606106342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6821","nonce":1606106342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6825","nonce":1606106342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6823","nonce":1606106342}]},"public_addr":"172.21.15.195:6819/1606106342","cluster_addr":"172.21.15.195:6821/1606106342","heartbeat_back_addr":"172.21.15.195:6825/1606106342","heartbeat_front_addr":"172.21.15.195:6823/1606106342","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.901343+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:21.115003+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.901623+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.835238+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.132199+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:19.817560+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.829690+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:21.087300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T03:40:03.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:03.579+0000 7f4869bba640 1 -- 172.21.15.38:0/1239890328 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4850078b90 msgr2=0x7f485007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:03.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:03.579+0000 7f4869bba640 1 --2- 172.21.15.38:0/1239890328 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4850078b90 0x7f485007b0a0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f4854003830 tx=0x7f4854006c20 comp rx=0 tx=0).stop 2024-09-21T03:40:03.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:03.579+0000 7f4869bba640 1 -- 172.21.15.38:0/1239890328 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f486409e380 msgr2=0x7f486406ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:03.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:03.579+0000 7f4869bba640 1 --2- 172.21.15.38:0/1239890328 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f486409e380 0x7f486406ec30 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f485800e590 tx=0x7f485800ea50 comp rx=0 tx=0).stop 2024-09-21T03:40:03.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:03.579+0000 7f4869bba640 1 -- 172.21.15.38:0/1239890328 shutdown_connections 2024-09-21T03:40:03.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:03.579+0000 7f4869bba640 1 --2- 172.21.15.38:0/1239890328 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4850078b90 0x7f485007b0a0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:03.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:03.579+0000 7f4869bba640 1 --2- 172.21.15.38:0/1239890328 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f486406f170 0x7f486415d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:03.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:03.579+0000 7f4869bba640 1 --2- 172.21.15.38:0/1239890328 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f486409e380 0x7f486406ec30 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:03.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:03.579+0000 7f4869bba640 1 --2- 172.21.15.38:0/1239890328 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f484c005350 0x7f48640c7a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:03.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:03.579+0000 7f4869bba640 1 -- 172.21.15.38:0/1239890328 >> 172.21.15.38:0/1239890328 conn(0x7f48640cca40 msgr2=0x7f48640cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:03.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:03.579+0000 7f4869bba640 1 -- 172.21.15.38:0/1239890328 shutdown_connections 2024-09-21T03:40:03.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:03.579+0000 7f4869bba640 1 -- 172.21.15.38:0/1239890328 wait complete. 2024-09-21T03:40:03.593 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-21T03:40:03.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:03.711+0000 7f4d7ffc3640 1 -- 172.21.15.38:0/4163447473 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4d780d0d80 msgr2=0x7f4d78167550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:03.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:03.711+0000 7f4d7ffc3640 1 --2- 172.21.15.38:0/4163447473 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4d780d0d80 0x7f4d78167550 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f4d70005fd0 tx=0x7f4d7002aaa0 comp rx=0 tx=0).stop 2024-09-21T03:40:03.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:03.711+0000 7f4d7ffc3640 1 -- 172.21.15.38:0/4163447473 shutdown_connections 2024-09-21T03:40:03.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:03.711+0000 7f4d7ffc3640 1 --2- 172.21.15.38:0/4163447473 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4d780d0d80 0x7f4d78167550 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:03.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:03.712+0000 7f4d7ffc3640 1 -- 172.21.15.38:0/4163447473 >> 172.21.15.38:0/4163447473 conn(0x7f4d780b46e0 msgr2=0x7f4d780d23e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:03.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:03.712+0000 7f4d7ffc3640 1 -- 172.21.15.38:0/4163447473 shutdown_connections 2024-09-21T03:40:03.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:03.712+0000 7f4d7ffc3640 1 -- 172.21.15.38:0/4163447473 wait complete. 2024-09-21T03:40:03.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:03.713+0000 7f4d7ffc3640 1 Processor -- start 2024-09-21T03:40:03.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:03.713+0000 7f4d7ffc3640 1 -- start start 2024-09-21T03:40:03.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:03.715+0000 7f4d7ffc3640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4d780ccc40 0x7f4d78070730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:03.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:03.715+0000 7f4d7ffc3640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4d7815ca90 0x7f4d7815d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:03.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:03.715+0000 7f4d7ffc3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4d7815a650 0x7f4d7815d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:03.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:03.715+0000 7f4d7ffc3640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f4d78167e20 con 0x7f4d7815a650 2024-09-21T03:40:03.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:03.715+0000 7f4d7ffc3640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f4d78167ca0 con 0x7f4d780ccc40 2024-09-21T03:40:03.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:03.715+0000 7f4d7ffc3640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f4d78167fa0 con 0x7f4d7815ca90 2024-09-21T03:40:03.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:03.715+0000 7f4d7e560640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4d7815a650 0x7f4d7815d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:03.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:03.715+0000 7f4d7e560640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4d7815a650 0x7f4d7815d550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:55544/0 (socket says 172.21.15.38:55544) 2024-09-21T03:40:03.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:03.715+0000 7f4d7e560640 1 -- 172.21.15.38:0/2075764908 learned_addr learned my addr 172.21.15.38:0/2075764908 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:03.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:03.715+0000 7f4d7dd5f640 1 --2- 172.21.15.38:0/2075764908 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4d780ccc40 0x7f4d78070730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:03.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:03.715+0000 7f4d7d55e640 1 --2- 172.21.15.38:0/2075764908 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4d7815ca90 0x7f4d7815d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:03.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:03.715+0000 7f4d7e560640 1 -- 172.21.15.38:0/2075764908 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4d7815ca90 msgr2=0x7f4d7815d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:03.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:03.715+0000 7f4d7e560640 1 --2- 172.21.15.38:0/2075764908 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4d7815ca90 0x7f4d7815d010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:03.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:03.715+0000 7f4d7e560640 1 -- 172.21.15.38:0/2075764908 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4d780ccc40 msgr2=0x7f4d78070730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:03.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:03.715+0000 7f4d7e560640 1 --2- 172.21.15.38:0/2075764908 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4d780ccc40 0x7f4d78070730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:03.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:03.715+0000 7f4d7e560640 1 -- 172.21.15.38:0/2075764908 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d7815f040 con 0x7f4d7815a650 2024-09-21T03:40:03.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:03.715+0000 7f4d7dd5f640 1 --2- 172.21.15.38:0/2075764908 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4d780ccc40 0x7f4d78070730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:40:03.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:03.715+0000 7f4d7e560640 1 --2- 172.21.15.38:0/2075764908 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4d7815a650 0x7f4d7815d550 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f4d70006010 tx=0x7f4d70003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:03.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:03.715+0000 7f4d6e7fc640 1 -- 172.21.15.38:0/2075764908 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d70044070 con 0x7f4d7815a650 2024-09-21T03:40:03.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:03.716+0000 7f4d6e7fc640 1 -- 172.21.15.38:0/2075764908 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4d70004250 con 0x7f4d7815a650 2024-09-21T03:40:03.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:03.716+0000 7f4d7ffc3640 1 -- 172.21.15.38:0/2075764908 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d7815f2e0 con 0x7f4d7815a650 2024-09-21T03:40:03.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:03.716+0000 7f4d6e7fc640 1 -- 172.21.15.38:0/2075764908 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d70004680 con 0x7f4d7815a650 2024-09-21T03:40:03.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:03.716+0000 7f4d7ffc3640 1 -- 172.21.15.38:0/2075764908 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d7815f8a0 con 0x7f4d7815a650 2024-09-21T03:40:03.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:03.716+0000 7f4d7ffc3640 1 -- 172.21.15.38:0/2075764908 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d780b7ce0 con 0x7f4d7815a650 2024-09-21T03:40:03.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:03.717+0000 7f4d6e7fc640 1 -- 172.21.15.38:0/2075764908 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f4d70047070 con 0x7f4d7815a650 2024-09-21T03:40:03.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:03.718+0000 7f4d6e7fc640 1 --2- 172.21.15.38:0/2075764908 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4d5c078b90 0x7f4d5c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:03.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:03.718+0000 7f4d6e7fc640 1 -- 172.21.15.38:0/2075764908 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6101+0+0 (secure 0 0 0) 0x7f4d700bc140 con 0x7f4d7815a650 2024-09-21T03:40:03.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:03.718+0000 7f4d7dd5f640 1 --2- 172.21.15.38:0/2075764908 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4d5c078b90 0x7f4d5c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:03.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:03.719+0000 7f4d7dd5f640 1 --2- 172.21.15.38:0/2075764908 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4d5c078b90 0x7f4d5c07b0a0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f4d740016a0 tx=0x7f4d74009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:03.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:03.722+0000 7f4d6e7fc640 1 -- 172.21.15.38:0/2075764908 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f4d7004e080 con 0x7f4d7815a650 2024-09-21T03:40:03.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:03.952+0000 7f4d7ffc3640 1 -- 172.21.15.38:0/2075764908 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f4d78070c70 con 0x7f4d7815a650 2024-09-21T03:40:04.868 INFO:tasks.ceph.mds.b.smithi195.stderr:2024-09-21T03:40:04.865+0000 7f4aafd63640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T03:40:04.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:04.914+0000 7f4d6e7fc640 1 -- 172.21.15.38:0/2075764908 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v82) ==== 118+0+0 (secure 0 0 0) 0x7f4d7815f040 con 0x7f4d7815a650 2024-09-21T03:40:04.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:04.917+0000 7f4d7ffc3640 1 -- 172.21.15.38:0/2075764908 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4d5c078b90 msgr2=0x7f4d5c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:04.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:04.917+0000 7f4d7ffc3640 1 --2- 172.21.15.38:0/2075764908 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4d5c078b90 0x7f4d5c07b0a0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f4d740016a0 tx=0x7f4d74009040 comp rx=0 tx=0).stop 2024-09-21T03:40:04.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:04.918+0000 7f4d7ffc3640 1 -- 172.21.15.38:0/2075764908 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4d7815a650 msgr2=0x7f4d7815d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:04.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:04.918+0000 7f4d7ffc3640 1 --2- 172.21.15.38:0/2075764908 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4d7815a650 0x7f4d7815d550 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f4d70006010 tx=0x7f4d70003f10 comp rx=0 tx=0).stop 2024-09-21T03:40:04.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:04.918+0000 7f4d7ffc3640 1 -- 172.21.15.38:0/2075764908 shutdown_connections 2024-09-21T03:40:04.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:04.918+0000 7f4d7ffc3640 1 --2- 172.21.15.38:0/2075764908 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4d7815a650 0x7f4d7815d550 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:04.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:04.918+0000 7f4d7ffc3640 1 --2- 172.21.15.38:0/2075764908 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4d5c078b90 0x7f4d5c07b0a0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:04.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:04.918+0000 7f4d7ffc3640 1 --2- 172.21.15.38:0/2075764908 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4d7815ca90 0x7f4d7815d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:04.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:04.918+0000 7f4d7ffc3640 1 --2- 172.21.15.38:0/2075764908 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4d780ccc40 0x7f4d78070730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:04.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:04.918+0000 7f4d7ffc3640 1 -- 172.21.15.38:0/2075764908 >> 172.21.15.38:0/2075764908 conn(0x7f4d780b46e0 msgr2=0x7f4d780d23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:04.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:04.918+0000 7f4d7ffc3640 1 -- 172.21.15.38:0/2075764908 shutdown_connections 2024-09-21T03:40:04.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:04.918+0000 7f4d7ffc3640 1 -- 172.21.15.38:0/2075764908 wait complete. 2024-09-21T03:40:04.935 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-21T03:40:04.935 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-21T03:40:05.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:05.060+0000 7ff4450de640 1 -- 172.21.15.38:0/2014945153 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff428003a50 msgr2=0x7ff428003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:05.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:05.060+0000 7ff4450de640 1 --2- 172.21.15.38:0/2014945153 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff428003a50 0x7ff428003ec0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7ff43000bb20 tx=0x7ff428005830 comp rx=0 tx=0).stop 2024-09-21T03:40:05.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:05.060+0000 7ff4450de640 1 -- 172.21.15.38:0/2014945153 shutdown_connections 2024-09-21T03:40:05.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:05.060+0000 7ff4450de640 1 --2- 172.21.15.38:0/2014945153 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff428003a50 0x7ff428003ec0 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:05.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:05.060+0000 7ff4450de640 1 -- 172.21.15.38:0/2014945153 >> 172.21.15.38:0/2014945153 conn(0x7ff44009d830 msgr2=0x7ff4400b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:05.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:05.060+0000 7ff4450de640 1 -- 172.21.15.38:0/2014945153 shutdown_connections 2024-09-21T03:40:05.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:05.060+0000 7ff4450de640 1 -- 172.21.15.38:0/2014945153 wait complete. 2024-09-21T03:40:05.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:05.062+0000 7ff4450de640 1 Processor -- start 2024-09-21T03:40:05.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:05.062+0000 7ff4450de640 1 -- start start 2024-09-21T03:40:05.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:05.063+0000 7ff4450de640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff428003a50 0x7ff4401f60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:05.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:05.063+0000 7ff4450de640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff4400d3e60 0x7ff44006d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:05.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:05.063+0000 7ff4450de640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff440157860 0x7ff4401594b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:05.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:05.063+0000 7ff4450de640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ff4400cf120 con 0x7ff440157860 2024-09-21T03:40:05.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:05.064+0000 7ff4450de640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ff4400cefa0 con 0x7ff428003a50 2024-09-21T03:40:05.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:05.064+0000 7ff4450de640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7ff4400cf2a0 con 0x7ff4400d3e60 2024-09-21T03:40:05.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:05.064+0000 7ff43f59e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff440157860 0x7ff4401594b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:05.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:05.064+0000 7ff43f59e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff440157860 0x7ff4401594b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:55562/0 (socket says 172.21.15.38:55562) 2024-09-21T03:40:05.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:05.064+0000 7ff43ed9d640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff428003a50 0x7ff4401f60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:05.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:05.064+0000 7ff43f59e640 1 -- 172.21.15.38:0/2585153042 learned_addr learned my addr 172.21.15.38:0/2585153042 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:05.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:05.064+0000 7ff43e59c640 1 --2- 172.21.15.38:0/2585153042 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff4400d3e60 0x7ff44006d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:05.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:05.064+0000 7ff43ed9d640 1 -- 172.21.15.38:0/2585153042 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff4400d3e60 msgr2=0x7ff44006d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:05.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:05.064+0000 7ff43ed9d640 1 --2- 172.21.15.38:0/2585153042 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff4400d3e60 0x7ff44006d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:05.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:05.064+0000 7ff43ed9d640 1 -- 172.21.15.38:0/2585153042 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff440157860 msgr2=0x7ff4401594b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:05.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:05.064+0000 7ff43ed9d640 1 --2- 172.21.15.38:0/2585153042 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff440157860 0x7ff4401594b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:05.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:05.064+0000 7ff43ed9d640 1 -- 172.21.15.38:0/2585153042 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff44006ce80 con 0x7ff428003a50 2024-09-21T03:40:05.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:05.064+0000 7ff43f59e640 1 --2- 172.21.15.38:0/2585153042 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff440157860 0x7ff4401594b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:40:05.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:05.064+0000 7ff43ed9d640 1 --2- 172.21.15.38:0/2585153042 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff428003a50 0x7ff4401f60e0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7ff430033000 tx=0x7ff43000b6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:05.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:05.065+0000 7ff4277fe640 1 -- 172.21.15.38:0/2585153042 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff430040070 con 0x7ff428003a50 2024-09-21T03:40:05.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:05.065+0000 7ff4277fe640 1 -- 172.21.15.38:0/2585153042 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff430004350 con 0x7ff428003a50 2024-09-21T03:40:05.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:05.065+0000 7ff4450de640 1 -- 172.21.15.38:0/2585153042 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4400d4570 con 0x7ff428003a50 2024-09-21T03:40:05.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:05.065+0000 7ff4277fe640 1 -- 172.21.15.38:0/2585153042 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff430035040 con 0x7ff428003a50 2024-09-21T03:40:05.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:05.065+0000 7ff4450de640 1 -- 172.21.15.38:0/2585153042 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4400d4850 con 0x7ff428003a50 2024-09-21T03:40:05.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:05.066+0000 7ff4450de640 1 -- 172.21.15.38:0/2585153042 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4400d0be0 con 0x7ff428003a50 2024-09-21T03:40:05.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:05.067+0000 7ff4277fe640 1 -- 172.21.15.38:0/2585153042 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ff430038020 con 0x7ff428003a50 2024-09-21T03:40:05.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:05.067+0000 7ff4277fe640 1 --2- 172.21.15.38:0/2585153042 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff42c078bc0 0x7ff42c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:05.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:05.067+0000 7ff4277fe640 1 -- 172.21.15.38:0/2585153042 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6101+0+0 (secure 0 0 0) 0x7ff4300bbed0 con 0x7ff428003a50 2024-09-21T03:40:05.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:05.067+0000 7ff43e59c640 1 --2- 172.21.15.38:0/2585153042 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff42c078bc0 0x7ff42c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:05.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:05.068+0000 7ff43e59c640 1 --2- 172.21.15.38:0/2585153042 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff42c078bc0 0x7ff42c07b0d0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7ff440158840 tx=0x7ff434006d60 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:05.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:05.071+0000 7ff4277fe640 1 -- 172.21.15.38:0/2585153042 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7ff430053080 con 0x7ff428003a50 2024-09-21T03:40:05.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:05.301+0000 7ff4450de640 1 -- 172.21.15.38:0/2585153042 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7ff4400b12c0 con 0x7ff428003a50 2024-09-21T03:40:05.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:05.927+0000 7ff4277fe640 1 -- 172.21.15.38:0/2585153042 <== mon.1 v2:172.21.15.195:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v83) ==== 122+0+0 (secure 0 0 0) 0x7ff430083bc0 con 0x7ff428003a50 2024-09-21T03:40:05.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:05.930+0000 7ff4450de640 1 -- 172.21.15.38:0/2585153042 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff42c078bc0 msgr2=0x7ff42c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:05.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:05.930+0000 7ff4450de640 1 --2- 172.21.15.38:0/2585153042 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff42c078bc0 0x7ff42c07b0d0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7ff440158840 tx=0x7ff434006d60 comp rx=0 tx=0).stop 2024-09-21T03:40:05.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:05.930+0000 7ff4450de640 1 -- 172.21.15.38:0/2585153042 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff428003a50 msgr2=0x7ff4401f60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:05.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:05.930+0000 7ff4450de640 1 --2- 172.21.15.38:0/2585153042 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff428003a50 0x7ff4401f60e0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7ff430033000 tx=0x7ff43000b6f0 comp rx=0 tx=0).stop 2024-09-21T03:40:05.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:05.930+0000 7ff4450de640 1 -- 172.21.15.38:0/2585153042 shutdown_connections 2024-09-21T03:40:05.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:05.930+0000 7ff4450de640 1 --2- 172.21.15.38:0/2585153042 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff440157860 0x7ff4401594b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:05.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:05.930+0000 7ff4450de640 1 --2- 172.21.15.38:0/2585153042 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff42c078bc0 0x7ff42c07b0d0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:05.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:05.930+0000 7ff4450de640 1 --2- 172.21.15.38:0/2585153042 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff4400d3e60 0x7ff44006d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:05.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:05.930+0000 7ff4450de640 1 --2- 172.21.15.38:0/2585153042 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff428003a50 0x7ff4401f60e0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:05.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:05.930+0000 7ff4450de640 1 -- 172.21.15.38:0/2585153042 >> 172.21.15.38:0/2585153042 conn(0x7ff44009d830 msgr2=0x7ff440158a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:05.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:05.930+0000 7ff4450de640 1 -- 172.21.15.38:0/2585153042 shutdown_connections 2024-09-21T03:40:05.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:05.930+0000 7ff4450de640 1 -- 172.21.15.38:0/2585153042 wait complete. 2024-09-21T03:40:05.947 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-21T03:40:05.947 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:40:06.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:06.067+0000 7febbdb2b640 1 -- 172.21.15.38:0/2614865700 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7febb815cb10 msgr2=0x7febb815cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:06.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:06.067+0000 7febbdb2b640 1 --2- 172.21.15.38:0/2614865700 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7febb815cb10 0x7febb815cf20 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7feba8009830 tx=0x7feba802a510 comp rx=0 tx=0).stop 2024-09-21T03:40:06.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:06.067+0000 7febbdb2b640 1 -- 172.21.15.38:0/2614865700 shutdown_connections 2024-09-21T03:40:06.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:06.067+0000 7febbdb2b640 1 --2- 172.21.15.38:0/2614865700 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7febb80b6250 0x7febb80b6660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:06.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:06.067+0000 7febbdb2b640 1 --2- 172.21.15.38:0/2614865700 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7febb815cb10 0x7febb815cf20 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:06.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:06.067+0000 7febbdb2b640 1 -- 172.21.15.38:0/2614865700 >> 172.21.15.38:0/2614865700 conn(0x7febb80d2350 msgr2=0x7febb80d2760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:06.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:06.068+0000 7febbdb2b640 1 -- 172.21.15.38:0/2614865700 shutdown_connections 2024-09-21T03:40:06.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:06.068+0000 7febbdb2b640 1 -- 172.21.15.38:0/2614865700 wait complete. 2024-09-21T03:40:06.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:06.069+0000 7febbdb2b640 1 Processor -- start 2024-09-21T03:40:06.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:06.070+0000 7febbdb2b640 1 -- start start 2024-09-21T03:40:06.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:06.071+0000 7febbdb2b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7febb80b6250 0x7febb8070ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:06.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:06.071+0000 7febbdb2b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7febb80d12e0 0x7febb81e8d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:06.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:06.071+0000 7febbdb2b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7febb815cb10 0x7febb813d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:06.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:06.071+0000 7febbdb2b640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7febb80b6f30 con 0x7febb815cb10 2024-09-21T03:40:06.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:06.071+0000 7febbdb2b640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7febb80b6db0 con 0x7febb80b6250 2024-09-21T03:40:06.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:06.071+0000 7febbdb2b640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7febb80b70b0 con 0x7febb80d12e0 2024-09-21T03:40:06.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:06.071+0000 7febb7fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7febb815cb10 0x7febb813d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:06.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:06.071+0000 7febb7fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7febb815cb10 0x7febb813d920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:55570/0 (socket says 172.21.15.38:55570) 2024-09-21T03:40:06.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:06.071+0000 7febb7fff640 1 -- 172.21.15.38:0/3651578015 learned_addr learned my addr 172.21.15.38:0/3651578015 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:06.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:06.071+0000 7febb77fe640 1 --2- 172.21.15.38:0/3651578015 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7febb80b6250 0x7febb8070ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:06.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:06.071+0000 7febb6ffd640 1 --2- 172.21.15.38:0/3651578015 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7febb80d12e0 0x7febb81e8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:06.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:06.071+0000 7febb7fff640 1 -- 172.21.15.38:0/3651578015 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7febb80d12e0 msgr2=0x7febb81e8d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:06.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:06.071+0000 7febb7fff640 1 --2- 172.21.15.38:0/3651578015 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7febb80d12e0 0x7febb81e8d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:06.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:06.071+0000 7febb7fff640 1 -- 172.21.15.38:0/3651578015 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7febb80b6250 msgr2=0x7febb8070ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:06.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:06.071+0000 7febb7fff640 1 --2- 172.21.15.38:0/3651578015 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7febb80b6250 0x7febb8070ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:06.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:06.071+0000 7febb7fff640 1 -- 172.21.15.38:0/3651578015 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febb806f510 con 0x7febb815cb10 2024-09-21T03:40:06.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:06.072+0000 7febb7fff640 1 --2- 172.21.15.38:0/3651578015 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7febb815cb10 0x7febb813d920 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7febb000ea10 tx=0x7febb000eed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:06.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:06.072+0000 7feb9ffff640 1 -- 172.21.15.38:0/3651578015 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febb000cde0 con 0x7febb815cb10 2024-09-21T03:40:06.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:06.072+0000 7febbdb2b640 1 -- 172.21.15.38:0/3651578015 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febb806fa10 con 0x7febb815cb10 2024-09-21T03:40:06.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:06.072+0000 7feb9ffff640 1 -- 172.21.15.38:0/3651578015 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7febb0004c10 con 0x7febb815cb10 2024-09-21T03:40:06.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:06.072+0000 7feb9ffff640 1 -- 172.21.15.38:0/3651578015 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febb0005020 con 0x7febb815cb10 2024-09-21T03:40:06.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:06.072+0000 7febbdb2b640 1 -- 172.21.15.38:0/3651578015 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febb806ffd0 con 0x7febb815cb10 2024-09-21T03:40:06.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:06.073+0000 7febbdb2b640 1 -- 172.21.15.38:0/3651578015 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7febb80d1c20 con 0x7febb815cb10 2024-09-21T03:40:06.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:06.074+0000 7feb9ffff640 1 -- 172.21.15.38:0/3651578015 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7febb00051c0 con 0x7febb815cb10 2024-09-21T03:40:06.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:06.074+0000 7feb9ffff640 1 --2- 172.21.15.38:0/3651578015 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7feba4078b90 0x7feba407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:06.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:06.074+0000 7feb9ffff640 1 -- 172.21.15.38:0/3651578015 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6101+0+0 (secure 0 0 0) 0x7febb009bb00 con 0x7febb815cb10 2024-09-21T03:40:06.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:06.074+0000 7febb77fe640 1 --2- 172.21.15.38:0/3651578015 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7feba4078b90 0x7feba407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:06.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:06.075+0000 7febb77fe640 1 --2- 172.21.15.38:0/3651578015 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7feba4078b90 0x7feba407b0a0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7feba8034000 tx=0x7feba8004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:06.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:06.079+0000 7feb9ffff640 1 -- 172.21.15.38:0/3651578015 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7febb80d1c20 con 0x7febb815cb10 2024-09-21T03:40:06.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:06.303+0000 7febbdb2b640 1 -- 172.21.15.38:0/3651578015 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7febb8051080 con 0x7febb815cb10 2024-09-21T03:40:06.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:06.304+0000 7feb9ffff640 1 -- 172.21.15.38:0/3651578015 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) ==== 94+0+5222 (secure 0 0 0) 0x7febb8051080 con 0x7febb815cb10 2024-09-21T03:40:06.307 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:40:06.307 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":83,"btime":"2024-09-21T03:40:05:917318+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6086,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/965972663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":965972663},{"type":"v1","addr":"172.21.15.195:6836","nonce":965972663}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":7020,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/751618334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":751618334},{"type":"v1","addr":"172.21.15.38:6833","nonce":751618334}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":7023,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/2934569248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":2934569248},{"type":"v1","addr":"172.21.15.38:6835","nonce":2934569248}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":83,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:40:02.319282+0000","modified":"2024-09-21T03:40:05.304556+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7035},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7035":{"gid":7035,"name":"b","rank":0,"incarnation":79,"state":"up:active","state_seq":3,"addr":"172.21.15.195:6837/712370966","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":712370966},{"type":"v1","addr":"172.21.15.195:6837","nonce":712370966}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7035,"qdb_cluster":[7035]},"id":6}]} 2024-09-21T03:40:06.307 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 83 2024-09-21T03:40:06.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:06.306+0000 7febbdb2b640 1 -- 172.21.15.38:0/3651578015 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7feba4078b90 msgr2=0x7feba407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:06.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:06.306+0000 7febbdb2b640 1 --2- 172.21.15.38:0/3651578015 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7feba4078b90 0x7feba407b0a0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7feba8034000 tx=0x7feba8004730 comp rx=0 tx=0).stop 2024-09-21T03:40:06.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:06.306+0000 7febbdb2b640 1 -- 172.21.15.38:0/3651578015 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7febb815cb10 msgr2=0x7febb813d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:06.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:06.306+0000 7febbdb2b640 1 --2- 172.21.15.38:0/3651578015 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7febb815cb10 0x7febb813d920 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7febb000ea10 tx=0x7febb000eed0 comp rx=0 tx=0).stop 2024-09-21T03:40:06.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:06.306+0000 7febbdb2b640 1 -- 172.21.15.38:0/3651578015 shutdown_connections 2024-09-21T03:40:06.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:06.306+0000 7febbdb2b640 1 --2- 172.21.15.38:0/3651578015 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7febb815cb10 0x7febb813d920 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:06.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:06.306+0000 7febbdb2b640 1 --2- 172.21.15.38:0/3651578015 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7feba4078b90 0x7feba407b0a0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:06.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:06.306+0000 7febbdb2b640 1 --2- 172.21.15.38:0/3651578015 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7febb80d12e0 0x7febb81e8d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:06.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:06.306+0000 7febbdb2b640 1 --2- 172.21.15.38:0/3651578015 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7febb80b6250 0x7febb8070ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:06.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:06.306+0000 7febbdb2b640 1 -- 172.21.15.38:0/3651578015 >> 172.21.15.38:0/3651578015 conn(0x7febb80d2350 msgr2=0x7febb815b5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:06.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:06.306+0000 7febbdb2b640 1 -- 172.21.15.38:0/3651578015 shutdown_connections 2024-09-21T03:40:06.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:06.306+0000 7febbdb2b640 1 -- 172.21.15.38:0/3651578015 wait complete. 2024-09-21T03:40:06.320 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T03:40:06.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:06.433+0000 7fc54d0cc640 1 -- 172.21.15.38:0/3032675927 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5480cd040 msgr2=0x7fc5480d04c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:06.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:06.433+0000 7fc54d0cc640 1 --2- 172.21.15.38:0/3032675927 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5480cd040 0x7fc5480d04c0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7fc5480b4ab0 tx=0x7fc54002a8b0 comp rx=0 tx=0).stop 2024-09-21T03:40:06.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:06.433+0000 7fc54d0cc640 1 -- 172.21.15.38:0/3032675927 shutdown_connections 2024-09-21T03:40:06.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:06.433+0000 7fc54d0cc640 1 --2- 172.21.15.38:0/3032675927 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5480cd040 0x7fc5480d04c0 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:06.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:06.433+0000 7fc54d0cc640 1 --2- 172.21.15.38:0/3032675927 >> v2:172.21.15.195:3300/0 conn(0x7fc5481536e0 0x7fc5480d17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:06.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:06.433+0000 7fc54d0cc640 1 -- 172.21.15.38:0/3032675927 >> 172.21.15.38:0/3032675927 conn(0x7fc5480cef40 msgr2=0x7fc5480cf350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:06.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:06.433+0000 7fc54d0cc640 1 -- 172.21.15.38:0/3032675927 shutdown_connections 2024-09-21T03:40:06.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:06.433+0000 7fc54d0cc640 1 -- 172.21.15.38:0/3032675927 wait complete. 2024-09-21T03:40:06.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:06.435+0000 7fc54d0cc640 1 Processor -- start 2024-09-21T03:40:06.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:06.435+0000 7fc54d0cc640 1 -- start start 2024-09-21T03:40:06.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:06.437+0000 7fc54d0cc640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc5480d1d00 0x7fc5480cc9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:06.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:06.437+0000 7fc54d0cc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5481536e0 0x7fc5481e3780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:06.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:06.437+0000 7fc54d0cc640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc548070f50 0x7fc548071360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:06.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:06.437+0000 7fc54d0cc640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fc5480d0d90 con 0x7fc5481536e0 2024-09-21T03:40:06.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:06.437+0000 7fc54d0cc640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fc5480d0c10 con 0x7fc5480d1d00 2024-09-21T03:40:06.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:06.437+0000 7fc54d0cc640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fc5480d0f10 con 0x7fc548070f50 2024-09-21T03:40:06.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:06.437+0000 7fc54659c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5481536e0 0x7fc5481e3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:06.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:06.437+0000 7fc54659c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5481536e0 0x7fc5481e3780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:55578/0 (socket says 172.21.15.38:55578) 2024-09-21T03:40:06.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:06.437+0000 7fc54659c640 1 -- 172.21.15.38:0/1957506032 learned_addr learned my addr 172.21.15.38:0/1957506032 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:06.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:06.437+0000 7fc546d9d640 1 --2- 172.21.15.38:0/1957506032 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc5480d1d00 0x7fc5480cc9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:06.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:06.437+0000 7fc54759e640 1 --2- 172.21.15.38:0/1957506032 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc548070f50 0x7fc548071360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:06.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:06.437+0000 7fc54659c640 1 -- 172.21.15.38:0/1957506032 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc548070f50 msgr2=0x7fc548071360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:06.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:06.437+0000 7fc54659c640 1 --2- 172.21.15.38:0/1957506032 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc548070f50 0x7fc548071360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:06.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:06.437+0000 7fc54659c640 1 -- 172.21.15.38:0/1957506032 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc5480d1d00 msgr2=0x7fc5480cc9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:06.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:06.437+0000 7fc54659c640 1 --2- 172.21.15.38:0/1957506032 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc5480d1d00 0x7fc5480cc9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:06.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:06.437+0000 7fc54659c640 1 -- 172.21.15.38:0/1957506032 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5480718a0 con 0x7fc5481536e0 2024-09-21T03:40:06.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:06.437+0000 7fc54659c640 1 --2- 172.21.15.38:0/1957506032 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5481536e0 0x7fc5481e3780 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7fc53c00c430 tx=0x7fc53c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:06.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:06.437+0000 7fc52f7fe640 1 -- 172.21.15.38:0/1957506032 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc53c011790 con 0x7fc5481536e0 2024-09-21T03:40:06.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:06.438+0000 7fc52f7fe640 1 -- 172.21.15.38:0/1957506032 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc53c017070 con 0x7fc5481536e0 2024-09-21T03:40:06.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:06.438+0000 7fc54d0cc640 1 -- 172.21.15.38:0/1957506032 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5481e9be0 con 0x7fc5481536e0 2024-09-21T03:40:06.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:06.438+0000 7fc52f7fe640 1 -- 172.21.15.38:0/1957506032 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc53c011ce0 con 0x7fc5481536e0 2024-09-21T03:40:06.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:06.438+0000 7fc54d0cc640 1 -- 172.21.15.38:0/1957506032 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5481e8e20 con 0x7fc5481536e0 2024-09-21T03:40:06.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:06.439+0000 7fc54d0cc640 1 -- 172.21.15.38:0/1957506032 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5481e8890 con 0x7fc5481536e0 2024-09-21T03:40:06.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:06.439+0000 7fc52f7fe640 1 -- 172.21.15.38:0/1957506032 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc53c01c460 con 0x7fc5481536e0 2024-09-21T03:40:06.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:06.440+0000 7fc52f7fe640 1 --2- 172.21.15.38:0/1957506032 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc534078b90 0x7fc53407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:06.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:06.440+0000 7fc52f7fe640 1 -- 172.21.15.38:0/1957506032 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6101+0+0 (secure 0 0 0) 0x7fc53c09ad10 con 0x7fc5481536e0 2024-09-21T03:40:06.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:06.440+0000 7fc546d9d640 1 --2- 172.21.15.38:0/1957506032 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc534078b90 0x7fc53407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:06.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:06.441+0000 7fc546d9d640 1 --2- 172.21.15.38:0/1957506032 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc534078b90 0x7fc53407b0a0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fc53800bb50 tx=0x7fc53800b2e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:06.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:06.444+0000 7fc52f7fe640 1 -- 172.21.15.38:0/1957506032 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fc5480718a0 con 0x7fc5481536e0 2024-09-21T03:40:06.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:06.632+0000 7fc54d0cc640 1 -- 172.21.15.38:0/1957506032 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc5480d04c0 con 0x7fc5481536e0 2024-09-21T03:40:06.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:06.633+0000 7fc52f7fe640 1 -- 172.21.15.38:0/1957506032 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+17017 (secure 0 0 0) 0x7fc53c062fc0 con 0x7fc5481536e0 2024-09-21T03:40:06.636 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:40:06.636 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":94,"fsid":"9d50d66f-656b-4fe2-8e3b-fe53b79c4d43","created":"2024-09-21T03:33:10.677865+0000","modified":"2024-09-21T03:40:03.898883+0000","last_up_change":"2024-09-21T03:33:24.023421+0000","last_in_change":"2024-09-21T03:33:14.296193+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T03:33:22.992573+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-09-21T03:39:59.945436+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-09-21T03:40:01.239789+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dbab183f-7e63-48b8-960a-ae55fd5bf565","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6801","nonce":2381639433}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6803","nonce":2381639433}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6807","nonce":2381639433}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6805","nonce":2381639433}]},"public_addr":"172.21.15.38:6801/2381639433","cluster_addr":"172.21.15.38:6803/2381639433","heartbeat_back_addr":"172.21.15.38:6807/2381639433","heartbeat_front_addr":"172.21.15.38:6805/2381639433","state":["exists","up"]},{"osd":1,"uuid":"fa1c0e8a-ebc1-4359-8146-bd5342abbc16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6817","nonce":1491818732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6819","nonce":1491818732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6823","nonce":1491818732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6821","nonce":1491818732}]},"public_addr":"172.21.15.38:6817/1491818732","cluster_addr":"172.21.15.38:6819/1491818732","heartbeat_back_addr":"172.21.15.38:6823/1491818732","heartbeat_front_addr":"172.21.15.38:6821/1491818732","state":["exists","up"]},{"osd":2,"uuid":"2ed3cd2d-385d-4fc6-9869-214552a84803","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6825","nonce":2577858902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6827","nonce":2577858902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6831","nonce":2577858902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6829","nonce":2577858902}]},"public_addr":"172.21.15.38:6825/2577858902","cluster_addr":"172.21.15.38:6827/2577858902","heartbeat_back_addr":"172.21.15.38:6831/2577858902","heartbeat_front_addr":"172.21.15.38:6829/2577858902","state":["exists","up"]},{"osd":3,"uuid":"62d1d95e-919d-4250-be71-1d34cfa70dbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6809","nonce":1286810294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6811","nonce":1286810294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6815","nonce":1286810294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6813","nonce":1286810294}]},"public_addr":"172.21.15.38:6809/1286810294","cluster_addr":"172.21.15.38:6811/1286810294","heartbeat_back_addr":"172.21.15.38:6815/1286810294","heartbeat_front_addr":"172.21.15.38:6813/1286810294","state":["exists","up"]},{"osd":4,"uuid":"7c9f9314-99d7-4d73-8392-37809a9f54b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6803","nonce":3401837571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6805","nonce":3401837571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6809","nonce":3401837571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6807","nonce":3401837571}]},"public_addr":"172.21.15.195:6803/3401837571","cluster_addr":"172.21.15.195:6805/3401837571","heartbeat_back_addr":"172.21.15.195:6809/3401837571","heartbeat_front_addr":"172.21.15.195:6807/3401837571","state":["exists","up"]},{"osd":5,"uuid":"bde4fe9f-8414-4bba-bf4a-43973d1976bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6811","nonce":2080960180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6813","nonce":2080960180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6817","nonce":2080960180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6815","nonce":2080960180}]},"public_addr":"172.21.15.195:6811/2080960180","cluster_addr":"172.21.15.195:6813/2080960180","heartbeat_back_addr":"172.21.15.195:6817/2080960180","heartbeat_front_addr":"172.21.15.195:6815/2080960180","state":["exists","up"]},{"osd":6,"uuid":"6ee6d7f4-1a5f-4a4c-ac16-48fe05f6a002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6827","nonce":4055825675}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6829","nonce":4055825675}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6833","nonce":4055825675}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6831","nonce":4055825675}]},"public_addr":"172.21.15.195:6827/4055825675","cluster_addr":"172.21.15.195:6829/4055825675","heartbeat_back_addr":"172.21.15.195:6833/4055825675","heartbeat_front_addr":"172.21.15.195:6831/4055825675","state":["exists","up"]},{"osd":7,"uuid":"efeb5f51-329a-4484-8290-92e7d3503d05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6819","nonce":1606106342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6821","nonce":1606106342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6825","nonce":1606106342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6823","nonce":1606106342}]},"public_addr":"172.21.15.195:6819/1606106342","cluster_addr":"172.21.15.195:6821/1606106342","heartbeat_back_addr":"172.21.15.195:6825/1606106342","heartbeat_front_addr":"172.21.15.195:6823/1606106342","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.901343+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:21.115003+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.901623+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.835238+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.132199+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:19.817560+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.829690+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:21.087300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T03:40:06.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:06.635+0000 7fc54d0cc640 1 -- 172.21.15.38:0/1957506032 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc534078b90 msgr2=0x7fc53407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:06.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:06.635+0000 7fc54d0cc640 1 --2- 172.21.15.38:0/1957506032 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc534078b90 0x7fc53407b0a0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fc53800bb50 tx=0x7fc53800b2e0 comp rx=0 tx=0).stop 2024-09-21T03:40:06.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:06.635+0000 7fc54d0cc640 1 -- 172.21.15.38:0/1957506032 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5481536e0 msgr2=0x7fc5481e3780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:06.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:06.635+0000 7fc54d0cc640 1 --2- 172.21.15.38:0/1957506032 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5481536e0 0x7fc5481e3780 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7fc53c00c430 tx=0x7fc53c00c8f0 comp rx=0 tx=0).stop 2024-09-21T03:40:06.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:06.635+0000 7fc54d0cc640 1 -- 172.21.15.38:0/1957506032 shutdown_connections 2024-09-21T03:40:06.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:06.635+0000 7fc54d0cc640 1 --2- 172.21.15.38:0/1957506032 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc534078b90 0x7fc53407b0a0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:06.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:06.635+0000 7fc54d0cc640 1 --2- 172.21.15.38:0/1957506032 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc548070f50 0x7fc548071360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:06.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:06.635+0000 7fc54d0cc640 1 --2- 172.21.15.38:0/1957506032 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5481536e0 0x7fc5481e3780 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:06.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:06.635+0000 7fc54d0cc640 1 --2- 172.21.15.38:0/1957506032 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc5480d1d00 0x7fc5480cc9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:06.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:06.635+0000 7fc54d0cc640 1 -- 172.21.15.38:0/1957506032 >> 172.21.15.38:0/1957506032 conn(0x7fc5480cef40 msgr2=0x7fc5480d2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:06.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:06.635+0000 7fc54d0cc640 1 -- 172.21.15.38:0/1957506032 shutdown_connections 2024-09-21T03:40:06.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:06.635+0000 7fc54d0cc640 1 -- 172.21.15.38:0/1957506032 wait complete. 2024-09-21T03:40:06.649 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-21T03:40:06.650 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T03:40:06.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:06.763+0000 7f53f5dc6640 1 -- 172.21.15.38:0/339852533 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f53f0162280 msgr2=0x7f53f0162690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:06.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:06.763+0000 7f53f5dc6640 1 --2- 172.21.15.38:0/339852533 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f53f0162280 0x7f53f0162690 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f53e4005fd0 tx=0x7f53e402a5d0 comp rx=0 tx=0).stop 2024-09-21T03:40:06.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:06.763+0000 7f53f5dc6640 1 -- 172.21.15.38:0/339852533 shutdown_connections 2024-09-21T03:40:06.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:06.763+0000 7f53f5dc6640 1 --2- 172.21.15.38:0/339852533 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f53f00b2e40 0x7f53f00b3250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:06.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:06.763+0000 7f53f5dc6640 1 --2- 172.21.15.38:0/339852533 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f53f0162280 0x7f53f0162690 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:06.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:06.763+0000 7f53f5dc6640 1 -- 172.21.15.38:0/339852533 >> 172.21.15.38:0/339852533 conn(0x7f53f00b6b20 msgr2=0x7f53f00d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:06.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:06.763+0000 7f53f5dc6640 1 -- 172.21.15.38:0/339852533 shutdown_connections 2024-09-21T03:40:06.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:06.763+0000 7f53f5dc6640 1 -- 172.21.15.38:0/339852533 wait complete. 2024-09-21T03:40:06.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:06.765+0000 7f53f5dc6640 1 Processor -- start 2024-09-21T03:40:06.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:06.765+0000 7f53f5dc6640 1 -- start start 2024-09-21T03:40:06.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:06.767+0000 7f53f5dc6640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f53f00b2e40 0x7f53f01580d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:06.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:06.767+0000 7f53f5dc6640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f53f0162280 0x7f53f01f2620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:06.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:06.767+0000 7f53f5dc6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f53f01f87d0 0x7f53f01f8be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:06.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:06.767+0000 7f53f5dc6640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f53f015e4b0 con 0x7f53f01f87d0 2024-09-21T03:40:06.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:06.767+0000 7f53f5dc6640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f53f00b39d0 con 0x7f53f0162280 2024-09-21T03:40:06.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:06.767+0000 7f53f5dc6640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f53f015e630 con 0x7f53f00b2e40 2024-09-21T03:40:06.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:06.767+0000 7f53effff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f53f01f87d0 0x7f53f01f8be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:06.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:06.767+0000 7f53effff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f53f01f87d0 0x7f53f01f8be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:55594/0 (socket says 172.21.15.38:55594) 2024-09-21T03:40:06.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:06.767+0000 7f53effff640 1 -- 172.21.15.38:0/4119381637 learned_addr learned my addr 172.21.15.38:0/4119381637 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:06.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:06.767+0000 7f53ef7fe640 1 --2- 172.21.15.38:0/4119381637 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f53f00b2e40 0x7f53f01580d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:06.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:06.767+0000 7f53eeffd640 1 --2- 172.21.15.38:0/4119381637 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f53f0162280 0x7f53f01f2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:06.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:06.767+0000 7f53effff640 1 -- 172.21.15.38:0/4119381637 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f53f00b2e40 msgr2=0x7f53f01580d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:06.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:06.767+0000 7f53effff640 1 --2- 172.21.15.38:0/4119381637 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f53f00b2e40 0x7f53f01580d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:06.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:06.767+0000 7f53effff640 1 -- 172.21.15.38:0/4119381637 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f53f0162280 msgr2=0x7f53f01f2620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:06.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:06.767+0000 7f53effff640 1 --2- 172.21.15.38:0/4119381637 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f53f0162280 0x7f53f01f2620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:06.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:06.767+0000 7f53effff640 1 -- 172.21.15.38:0/4119381637 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53f01f9120 con 0x7f53f01f87d0 2024-09-21T03:40:06.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:06.767+0000 7f53effff640 1 --2- 172.21.15.38:0/4119381637 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f53f01f87d0 0x7f53f01f8be0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f53e800e990 tx=0x7f53e800ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:06.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:06.767+0000 7f53d7fff640 1 -- 172.21.15.38:0/4119381637 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e8004d20 con 0x7f53f01f87d0 2024-09-21T03:40:06.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:06.767+0000 7f53d7fff640 1 -- 172.21.15.38:0/4119381637 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f53e800caa0 con 0x7f53f01f87d0 2024-09-21T03:40:06.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:06.768+0000 7f53d7fff640 1 -- 172.21.15.38:0/4119381637 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e8005020 con 0x7f53f01f87d0 2024-09-21T03:40:06.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:06.768+0000 7f53f5dc6640 1 -- 172.21.15.38:0/4119381637 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53f0158ab0 con 0x7f53f01f87d0 2024-09-21T03:40:06.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:06.768+0000 7f53f5dc6640 1 -- 172.21.15.38:0/4119381637 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53f0159070 con 0x7f53f01f87d0 2024-09-21T03:40:06.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:06.769+0000 7f53f5dc6640 1 -- 172.21.15.38:0/4119381637 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53f0162f00 con 0x7f53f01f87d0 2024-09-21T03:40:06.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:06.769+0000 7f53d7fff640 1 -- 172.21.15.38:0/4119381637 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f53e80051c0 con 0x7f53f01f87d0 2024-09-21T03:40:06.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:06.770+0000 7f53d7fff640 1 --2- 172.21.15.38:0/4119381637 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f53dc078b90 0x7f53dc07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:06.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:06.770+0000 7f53d7fff640 1 -- 172.21.15.38:0/4119381637 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6101+0+0 (secure 0 0 0) 0x7f53e809b790 con 0x7f53f01f87d0 2024-09-21T03:40:06.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:06.770+0000 7f53ef7fe640 1 --2- 172.21.15.38:0/4119381637 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f53dc078b90 0x7f53dc07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:06.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:06.771+0000 7f53ef7fe640 1 --2- 172.21.15.38:0/4119381637 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f53dc078b90 0x7f53dc07b0a0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f53e0001720 tx=0x7f53e0009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:06.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:06.775+0000 7f53d7fff640 1 -- 172.21.15.38:0/4119381637 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f53f01f9120 con 0x7f53f01f87d0 2024-09-21T03:40:06.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:06.957+0000 7f53f5dc6640 1 -- 172.21.15.38:0/4119381637 --> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f53f015dd00 con 0x7f53dc078b90 2024-09-21T03:40:06.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:06.962+0000 7f53d7fff640 1 -- 172.21.15.38:0/4119381637 <== mgr.4117 v2:172.21.15.195:6800/1631901344 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251493 (secure 0 0 0) 0x7f53f015dd00 con 0x7f53dc078b90 2024-09-21T03:40:06.967 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:40:06.969 INFO:teuthology.orchestra.run.smithi038.stderr:dumped all 2024-09-21T03:40:06.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:06.965+0000 7f53f5dc6640 1 -- 172.21.15.38:0/4119381637 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f53dc078b90 msgr2=0x7f53dc07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:06.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:06.965+0000 7f53f5dc6640 1 --2- 172.21.15.38:0/4119381637 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f53dc078b90 0x7f53dc07b0a0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f53e0001720 tx=0x7f53e0009040 comp rx=0 tx=0).stop 2024-09-21T03:40:06.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:06.965+0000 7f53f5dc6640 1 -- 172.21.15.38:0/4119381637 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f53f01f87d0 msgr2=0x7f53f01f8be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:06.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:06.965+0000 7f53f5dc6640 1 --2- 172.21.15.38:0/4119381637 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f53f01f87d0 0x7f53f01f8be0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f53e800e990 tx=0x7f53e800ee50 comp rx=0 tx=0).stop 2024-09-21T03:40:06.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:06.965+0000 7f53f5dc6640 1 -- 172.21.15.38:0/4119381637 shutdown_connections 2024-09-21T03:40:06.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:06.965+0000 7f53f5dc6640 1 --2- 172.21.15.38:0/4119381637 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f53f01f87d0 0x7f53f01f8be0 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:06.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:06.965+0000 7f53f5dc6640 1 --2- 172.21.15.38:0/4119381637 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f53f0162280 0x7f53f01f2620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:06.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:06.965+0000 7f53f5dc6640 1 --2- 172.21.15.38:0/4119381637 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f53dc078b90 0x7f53dc07b0a0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:06.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:06.965+0000 7f53f5dc6640 1 --2- 172.21.15.38:0/4119381637 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f53f00b2e40 0x7f53f01580d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:06.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:06.965+0000 7f53f5dc6640 1 -- 172.21.15.38:0/4119381637 >> 172.21.15.38:0/4119381637 conn(0x7f53f00b6b20 msgr2=0x7f53f006cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:06.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:06.965+0000 7f53f5dc6640 1 -- 172.21.15.38:0/4119381637 shutdown_connections 2024-09-21T03:40:06.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:06.965+0000 7f53f5dc6640 1 -- 172.21.15.38:0/4119381637 wait complete. 2024-09-21T03:40:06.979 INFO:teuthology.orchestra.run.smithi038.stdout:{"pg_ready":true,"pg_map":{"version":288,"stamp":"2024-09-21T03:40:05.087782+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":145,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":109,"ondisk_log_size":109,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":99,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":276236,"kb_used_data":10064,"kb_used_omap":12,"kb_used_meta":265971,"kb_avail":754698484,"statfs":{"total":773094113280,"available":772811247616,"internally_reserved":0,"allocated":10305536,"data_stored":5382448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12748,"internal_metadata":272354868},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.208473"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-21T03:40:02.927515+0000","last_change":"2024-09-21T03:40:02.922416+0000","last_active":"2024-09-21T03:40:02.927515+0000","last_peered":"2024-09-21T03:40:02.927515+0000","last_clean":"2024-09-21T03:40:02.927515+0000","last_became_active":"2024-09-21T03:40:02.922118+0000","last_became_peered":"2024-09-21T03:40:02.922118+0000","last_unstale":"2024-09-21T03:40:02.927515+0000","last_undegraded":"2024-09-21T03:40:02.927515+0000","last_fullsized":"2024-09-21T03:40:02.927515+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_clean_scrub_stamp":"2024-09-21T03:40:01.878704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:02:41.238627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T03:40:03.911346+0000","last_change":"2024-09-21T03:40:02.918908+0000","last_active":"2024-09-21T03:40:03.911346+0000","last_peered":"2024-09-21T03:40:03.911346+0000","last_clean":"2024-09-21T03:40:03.911346+0000","last_became_active":"2024-09-21T03:40:02.918227+0000","last_became_peered":"2024-09-21T03:40:02.918227+0000","last_unstale":"2024-09-21T03:40:03.911346+0000","last_undegraded":"2024-09-21T03:40:03.911346+0000","last_fullsized":"2024-09-21T03:40:03.911346+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_clean_scrub_stamp":"2024-09-21T03:40:01.878704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:46:27.097203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-21T03:40:02.920916+0000","last_change":"2024-09-21T03:40:02.920038+0000","last_active":"2024-09-21T03:40:02.920916+0000","last_peered":"2024-09-21T03:40:02.920916+0000","last_clean":"2024-09-21T03:40:02.920916+0000","last_became_active":"2024-09-21T03:40:02.919683+0000","last_became_peered":"2024-09-21T03:40:02.919683+0000","last_unstale":"2024-09-21T03:40:02.920916+0000","last_undegraded":"2024-09-21T03:40:02.920916+0000","last_fullsized":"2024-09-21T03:40:02.920916+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_clean_scrub_stamp":"2024-09-21T03:40:01.878704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:56:14.770281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-21T03:40:02.910220+0000","last_change":"2024-09-21T03:40:02.903330+0000","last_active":"2024-09-21T03:40:02.910220+0000","last_peered":"2024-09-21T03:40:02.910220+0000","last_clean":"2024-09-21T03:40:02.910220+0000","last_became_active":"2024-09-21T03:40:02.903023+0000","last_became_peered":"2024-09-21T03:40:02.903023+0000","last_unstale":"2024-09-21T03:40:02.910220+0000","last_undegraded":"2024-09-21T03:40:02.910220+0000","last_fullsized":"2024-09-21T03:40:02.910220+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_clean_scrub_stamp":"2024-09-21T03:40:01.878704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:55:50.721982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T03:40:03.910733+0000","last_change":"2024-09-21T03:40:02.898458+0000","last_active":"2024-09-21T03:40:03.910733+0000","last_peered":"2024-09-21T03:40:03.910733+0000","last_clean":"2024-09-21T03:40:03.910733+0000","last_became_active":"2024-09-21T03:40:02.898121+0000","last_became_peered":"2024-09-21T03:40:02.898121+0000","last_unstale":"2024-09-21T03:40:03.910733+0000","last_undegraded":"2024-09-21T03:40:03.910733+0000","last_fullsized":"2024-09-21T03:40:03.910733+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_clean_scrub_stamp":"2024-09-21T03:40:01.878704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:40:31.254797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-21T03:40:02.925096+0000","last_change":"2024-09-21T03:40:02.924187+0000","last_active":"2024-09-21T03:40:02.925096+0000","last_peered":"2024-09-21T03:40:02.925096+0000","last_clean":"2024-09-21T03:40:02.925096+0000","last_became_active":"2024-09-21T03:40:02.923857+0000","last_became_peered":"2024-09-21T03:40:02.923857+0000","last_unstale":"2024-09-21T03:40:02.925096+0000","last_undegraded":"2024-09-21T03:40:02.925096+0000","last_fullsized":"2024-09-21T03:40:02.925096+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_clean_scrub_stamp":"2024-09-21T03:40:01.878704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:09:01.113089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-21T03:40:02.915449+0000","last_change":"2024-09-21T03:40:02.914279+0000","last_active":"2024-09-21T03:40:02.915449+0000","last_peered":"2024-09-21T03:40:02.915449+0000","last_clean":"2024-09-21T03:40:02.915449+0000","last_became_active":"2024-09-21T03:40:02.914000+0000","last_became_peered":"2024-09-21T03:40:02.914000+0000","last_unstale":"2024-09-21T03:40:02.915449+0000","last_undegraded":"2024-09-21T03:40:02.915449+0000","last_fullsized":"2024-09-21T03:40:02.915449+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_clean_scrub_stamp":"2024-09-21T03:40:01.878704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:01:18.125708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-21T03:40:02.908057+0000","last_change":"2024-09-21T03:40:02.907470+0000","last_active":"2024-09-21T03:40:02.908057+0000","last_peered":"2024-09-21T03:40:02.908057+0000","last_clean":"2024-09-21T03:40:02.908057+0000","last_became_active":"2024-09-21T03:40:02.907125+0000","last_became_peered":"2024-09-21T03:40:02.907125+0000","last_unstale":"2024-09-21T03:40:02.908057+0000","last_undegraded":"2024-09-21T03:40:02.908057+0000","last_fullsized":"2024-09-21T03:40:02.908057+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_clean_scrub_stamp":"2024-09-21T03:40:01.878704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:39:32.775254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-21T03:40:02.906008+0000","last_change":"2024-09-21T03:40:02.900085+0000","last_active":"2024-09-21T03:40:02.906008+0000","last_peered":"2024-09-21T03:40:02.906008+0000","last_clean":"2024-09-21T03:40:02.906008+0000","last_became_active":"2024-09-21T03:40:02.899781+0000","last_became_peered":"2024-09-21T03:40:02.899781+0000","last_unstale":"2024-09-21T03:40:02.906008+0000","last_undegraded":"2024-09-21T03:40:02.906008+0000","last_fullsized":"2024-09-21T03:40:02.906008+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_clean_scrub_stamp":"2024-09-21T03:40:01.878704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:00:13.266042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-21T03:40:02.916329+0000","last_change":"2024-09-21T03:40:02.912456+0000","last_active":"2024-09-21T03:40:02.916329+0000","last_peered":"2024-09-21T03:40:02.916329+0000","last_clean":"2024-09-21T03:40:02.916329+0000","last_became_active":"2024-09-21T03:40:02.912153+0000","last_became_peered":"2024-09-21T03:40:02.912153+0000","last_unstale":"2024-09-21T03:40:02.916329+0000","last_undegraded":"2024-09-21T03:40:02.916329+0000","last_fullsized":"2024-09-21T03:40:02.916329+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_clean_scrub_stamp":"2024-09-21T03:40:01.878704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:16:06.081301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-21T03:40:02.909748+0000","last_change":"2024-09-21T03:40:02.905686+0000","last_active":"2024-09-21T03:40:02.909748+0000","last_peered":"2024-09-21T03:40:02.909748+0000","last_clean":"2024-09-21T03:40:02.909748+0000","last_became_active":"2024-09-21T03:40:02.905353+0000","last_became_peered":"2024-09-21T03:40:02.905353+0000","last_unstale":"2024-09-21T03:40:02.909748+0000","last_undegraded":"2024-09-21T03:40:02.909748+0000","last_fullsized":"2024-09-21T03:40:02.909748+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_clean_scrub_stamp":"2024-09-21T03:40:01.878704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:17:18.448103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-21T03:40:02.916817+0000","last_change":"2024-09-21T03:40:02.914584+0000","last_active":"2024-09-21T03:40:02.916817+0000","last_peered":"2024-09-21T03:40:02.916817+0000","last_clean":"2024-09-21T03:40:02.916817+0000","last_became_active":"2024-09-21T03:40:02.913212+0000","last_became_peered":"2024-09-21T03:40:02.913212+0000","last_unstale":"2024-09-21T03:40:02.916817+0000","last_undegraded":"2024-09-21T03:40:02.916817+0000","last_fullsized":"2024-09-21T03:40:02.916817+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_clean_scrub_stamp":"2024-09-21T03:40:01.878704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:37:43.695022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-21T03:40:01.897358+0000","last_change":"2024-09-21T03:40:01.897358+0000","last_active":"2024-09-21T03:40:01.878704+0000","last_peered":"2024-09-21T03:40:01.878704+0000","last_clean":"2024-09-21T03:40:01.878704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:40:01.897358+0000","last_undegraded":"2024-09-21T03:40:01.897358+0000","last_fullsized":"2024-09-21T03:40:01.897358+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_clean_scrub_stamp":"2024-09-21T03:40:01.878704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-21T03:40:02.915738+0000","last_change":"2024-09-21T03:40:02.911800+0000","last_active":"2024-09-21T03:40:02.915738+0000","last_peered":"2024-09-21T03:40:02.915738+0000","last_clean":"2024-09-21T03:40:02.915738+0000","last_became_active":"2024-09-21T03:40:02.911488+0000","last_became_peered":"2024-09-21T03:40:02.911488+0000","last_unstale":"2024-09-21T03:40:02.915738+0000","last_undegraded":"2024-09-21T03:40:02.915738+0000","last_fullsized":"2024-09-21T03:40:02.915738+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_clean_scrub_stamp":"2024-09-21T03:40:01.878704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:22:37.186117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-21T03:40:02.917537+0000","last_change":"2024-09-21T03:40:02.916645+0000","last_active":"2024-09-21T03:40:02.917537+0000","last_peered":"2024-09-21T03:40:02.917537+0000","last_clean":"2024-09-21T03:40:02.917537+0000","last_became_active":"2024-09-21T03:40:02.916319+0000","last_became_peered":"2024-09-21T03:40:02.916319+0000","last_unstale":"2024-09-21T03:40:02.917537+0000","last_undegraded":"2024-09-21T03:40:02.917537+0000","last_fullsized":"2024-09-21T03:40:02.917537+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_clean_scrub_stamp":"2024-09-21T03:40:01.878704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:27:09.793327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-21T03:40:02.920537+0000","last_change":"2024-09-21T03:40:02.919274+0000","last_active":"2024-09-21T03:40:02.920537+0000","last_peered":"2024-09-21T03:40:02.920537+0000","last_clean":"2024-09-21T03:40:02.920537+0000","last_became_active":"2024-09-21T03:40:02.918722+0000","last_became_peered":"2024-09-21T03:40:02.918722+0000","last_unstale":"2024-09-21T03:40:02.920537+0000","last_undegraded":"2024-09-21T03:40:02.920537+0000","last_fullsized":"2024-09-21T03:40:02.920537+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_clean_scrub_stamp":"2024-09-21T03:40:01.878704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:42:13.584645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-21T03:40:02.911825+0000","last_change":"2024-09-21T03:40:02.910611+0000","last_active":"2024-09-21T03:40:02.911825+0000","last_peered":"2024-09-21T03:40:02.911825+0000","last_clean":"2024-09-21T03:40:02.911825+0000","last_became_active":"2024-09-21T03:40:02.910292+0000","last_became_peered":"2024-09-21T03:40:02.910292+0000","last_unstale":"2024-09-21T03:40:02.911825+0000","last_undegraded":"2024-09-21T03:40:02.911825+0000","last_fullsized":"2024-09-21T03:40:02.911825+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_clean_scrub_stamp":"2024-09-21T03:40:01.878704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:45:36.917957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-21T03:40:02.910636+0000","last_change":"2024-09-21T03:40:02.904596+0000","last_active":"2024-09-21T03:40:02.910636+0000","last_peered":"2024-09-21T03:40:02.910636+0000","last_clean":"2024-09-21T03:40:02.910636+0000","last_became_active":"2024-09-21T03:40:02.904299+0000","last_became_peered":"2024-09-21T03:40:02.904299+0000","last_unstale":"2024-09-21T03:40:02.910636+0000","last_undegraded":"2024-09-21T03:40:02.910636+0000","last_fullsized":"2024-09-21T03:40:02.910636+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_clean_scrub_stamp":"2024-09-21T03:40:01.878704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:17:45.648298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-21T03:40:02.916206+0000","last_change":"2024-09-21T03:40:02.911816+0000","last_active":"2024-09-21T03:40:02.916206+0000","last_peered":"2024-09-21T03:40:02.916206+0000","last_clean":"2024-09-21T03:40:02.916206+0000","last_became_active":"2024-09-21T03:40:02.911504+0000","last_became_peered":"2024-09-21T03:40:02.911504+0000","last_unstale":"2024-09-21T03:40:02.916206+0000","last_undegraded":"2024-09-21T03:40:02.916206+0000","last_fullsized":"2024-09-21T03:40:02.916206+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_clean_scrub_stamp":"2024-09-21T03:40:01.878704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:22:48.720231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-21T03:40:02.928221+0000","last_change":"2024-09-21T03:40:02.924813+0000","last_active":"2024-09-21T03:40:02.928221+0000","last_peered":"2024-09-21T03:40:02.928221+0000","last_clean":"2024-09-21T03:40:02.928221+0000","last_became_active":"2024-09-21T03:40:02.924537+0000","last_became_peered":"2024-09-21T03:40:02.924537+0000","last_unstale":"2024-09-21T03:40:02.928221+0000","last_undegraded":"2024-09-21T03:40:02.928221+0000","last_fullsized":"2024-09-21T03:40:02.928221+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_clean_scrub_stamp":"2024-09-21T03:40:01.878704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:12:15.980838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T03:40:03.911087+0000","last_change":"2024-09-21T03:40:02.909856+0000","last_active":"2024-09-21T03:40:03.911087+0000","last_peered":"2024-09-21T03:40:03.911087+0000","last_clean":"2024-09-21T03:40:03.911087+0000","last_became_active":"2024-09-21T03:40:02.908152+0000","last_became_peered":"2024-09-21T03:40:02.908152+0000","last_unstale":"2024-09-21T03:40:03.911087+0000","last_undegraded":"2024-09-21T03:40:03.911087+0000","last_fullsized":"2024-09-21T03:40:03.911087+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_clean_scrub_stamp":"2024-09-21T03:40:01.878704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:36:26.820027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-21T03:40:02.921522+0000","last_change":"2024-09-21T03:40:02.916741+0000","last_active":"2024-09-21T03:40:02.921522+0000","last_peered":"2024-09-21T03:40:02.921522+0000","last_clean":"2024-09-21T03:40:02.921522+0000","last_became_active":"2024-09-21T03:40:02.916442+0000","last_became_peered":"2024-09-21T03:40:02.916442+0000","last_unstale":"2024-09-21T03:40:02.921522+0000","last_undegraded":"2024-09-21T03:40:02.921522+0000","last_fullsized":"2024-09-21T03:40:02.921522+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_clean_scrub_stamp":"2024-09-21T03:40:01.878704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:40:32.479886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T03:40:03.910597+0000","last_change":"2024-09-21T03:40:02.920348+0000","last_active":"2024-09-21T03:40:03.910597+0000","last_peered":"2024-09-21T03:40:03.910597+0000","last_clean":"2024-09-21T03:40:03.910597+0000","last_became_active":"2024-09-21T03:40:02.919968+0000","last_became_peered":"2024-09-21T03:40:02.919968+0000","last_unstale":"2024-09-21T03:40:03.910597+0000","last_undegraded":"2024-09-21T03:40:03.910597+0000","last_fullsized":"2024-09-21T03:40:03.910597+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_clean_scrub_stamp":"2024-09-21T03:40:01.878704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:39:21.091609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-21T03:40:02.927463+0000","last_change":"2024-09-21T03:40:02.923791+0000","last_active":"2024-09-21T03:40:02.927463+0000","last_peered":"2024-09-21T03:40:02.927463+0000","last_clean":"2024-09-21T03:40:02.927463+0000","last_became_active":"2024-09-21T03:40:02.923469+0000","last_became_peered":"2024-09-21T03:40:02.923469+0000","last_unstale":"2024-09-21T03:40:02.927463+0000","last_undegraded":"2024-09-21T03:40:02.927463+0000","last_fullsized":"2024-09-21T03:40:02.927463+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_clean_scrub_stamp":"2024-09-21T03:40:01.878704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:18:39.471497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T03:40:03.911072+0000","last_change":"2024-09-21T03:40:02.907650+0000","last_active":"2024-09-21T03:40:03.911072+0000","last_peered":"2024-09-21T03:40:03.911072+0000","last_clean":"2024-09-21T03:40:03.911072+0000","last_became_active":"2024-09-21T03:40:02.906437+0000","last_became_peered":"2024-09-21T03:40:02.906437+0000","last_unstale":"2024-09-21T03:40:03.911072+0000","last_undegraded":"2024-09-21T03:40:03.911072+0000","last_fullsized":"2024-09-21T03:40:03.911072+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_clean_scrub_stamp":"2024-09-21T03:40:01.878704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:21:26.719708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T03:40:03.911437+0000","last_change":"2024-09-21T03:40:02.919213+0000","last_active":"2024-09-21T03:40:03.911437+0000","last_peered":"2024-09-21T03:40:03.911437+0000","last_clean":"2024-09-21T03:40:03.911437+0000","last_became_active":"2024-09-21T03:40:02.918596+0000","last_became_peered":"2024-09-21T03:40:02.918596+0000","last_unstale":"2024-09-21T03:40:03.911437+0000","last_undegraded":"2024-09-21T03:40:03.911437+0000","last_fullsized":"2024-09-21T03:40:03.911437+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_clean_scrub_stamp":"2024-09-21T03:40:01.878704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:19:29.651722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-21T03:40:02.910299+0000","last_change":"2024-09-21T03:40:02.904727+0000","last_active":"2024-09-21T03:40:02.910299+0000","last_peered":"2024-09-21T03:40:02.910299+0000","last_clean":"2024-09-21T03:40:02.910299+0000","last_became_active":"2024-09-21T03:40:02.904362+0000","last_became_peered":"2024-09-21T03:40:02.904362+0000","last_unstale":"2024-09-21T03:40:02.910299+0000","last_undegraded":"2024-09-21T03:40:02.910299+0000","last_fullsized":"2024-09-21T03:40:02.910299+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_clean_scrub_stamp":"2024-09-21T03:40:01.878704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:13:49.164794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-21T03:40:01.897736+0000","last_change":"2024-09-21T03:40:01.897736+0000","last_active":"2024-09-21T03:40:01.878704+0000","last_peered":"2024-09-21T03:40:01.878704+0000","last_clean":"2024-09-21T03:40:01.878704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:40:01.897736+0000","last_undegraded":"2024-09-21T03:40:01.897736+0000","last_fullsized":"2024-09-21T03:40:01.897736+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_clean_scrub_stamp":"2024-09-21T03:40:01.878704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T03:40:03.911138+0000","last_change":"2024-09-21T03:40:02.926297+0000","last_active":"2024-09-21T03:40:03.911138+0000","last_peered":"2024-09-21T03:40:03.911138+0000","last_clean":"2024-09-21T03:40:03.911138+0000","last_became_active":"2024-09-21T03:40:02.925950+0000","last_became_peered":"2024-09-21T03:40:02.925950+0000","last_unstale":"2024-09-21T03:40:03.911138+0000","last_undegraded":"2024-09-21T03:40:03.911138+0000","last_fullsized":"2024-09-21T03:40:03.911138+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_clean_scrub_stamp":"2024-09-21T03:40:01.878704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:06:56.133248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T03:40:03.910624+0000","last_change":"2024-09-21T03:40:02.907384+0000","last_active":"2024-09-21T03:40:03.910624+0000","last_peered":"2024-09-21T03:40:03.910624+0000","last_clean":"2024-09-21T03:40:03.910624+0000","last_became_active":"2024-09-21T03:40:02.907112+0000","last_became_peered":"2024-09-21T03:40:02.907112+0000","last_unstale":"2024-09-21T03:40:03.910624+0000","last_undegraded":"2024-09-21T03:40:03.910624+0000","last_fullsized":"2024-09-21T03:40:03.910624+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_clean_scrub_stamp":"2024-09-21T03:40:01.878704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:47:17.983636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-21T03:40:02.926742+0000","last_change":"2024-09-21T03:40:02.923715+0000","last_active":"2024-09-21T03:40:02.926742+0000","last_peered":"2024-09-21T03:40:02.926742+0000","last_clean":"2024-09-21T03:40:02.926742+0000","last_became_active":"2024-09-21T03:40:02.923375+0000","last_became_peered":"2024-09-21T03:40:02.923375+0000","last_unstale":"2024-09-21T03:40:02.926742+0000","last_undegraded":"2024-09-21T03:40:02.926742+0000","last_fullsized":"2024-09-21T03:40:02.926742+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_clean_scrub_stamp":"2024-09-21T03:40:01.878704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:31:25.195627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-21T03:40:02.906514+0000","last_change":"2024-09-21T03:40:02.904512+0000","last_active":"2024-09-21T03:40:02.906514+0000","last_peered":"2024-09-21T03:40:02.906514+0000","last_clean":"2024-09-21T03:40:02.906514+0000","last_became_active":"2024-09-21T03:40:02.904158+0000","last_became_peered":"2024-09-21T03:40:02.904158+0000","last_unstale":"2024-09-21T03:40:02.906514+0000","last_undegraded":"2024-09-21T03:40:02.906514+0000","last_fullsized":"2024-09-21T03:40:02.906514+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_clean_scrub_stamp":"2024-09-21T03:40:01.878704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:10:44.384273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-21T03:40:02.928015+0000","last_change":"2024-09-21T03:40:02.923176+0000","last_active":"2024-09-21T03:40:02.928015+0000","last_peered":"2024-09-21T03:40:02.928015+0000","last_clean":"2024-09-21T03:40:02.928015+0000","last_became_active":"2024-09-21T03:40:02.922863+0000","last_became_peered":"2024-09-21T03:40:02.922863+0000","last_unstale":"2024-09-21T03:40:02.928015+0000","last_undegraded":"2024-09-21T03:40:02.928015+0000","last_fullsized":"2024-09-21T03:40:02.928015+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_clean_scrub_stamp":"2024-09-21T03:40:01.878704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:40:32.479886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"93'4","reported_seq":19,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T03:40:03.911493+0000","last_change":"2024-09-21T03:40:01.912737+0000","last_active":"2024-09-21T03:40:03.911493+0000","last_peered":"2024-09-21T03:40:03.911493+0000","last_clean":"2024-09-21T03:40:03.911493+0000","last_became_active":"2024-09-21T03:40:01.912428+0000","last_became_peered":"2024-09-21T03:40:01.912428+0000","last_unstale":"2024-09-21T03:40:03.911493+0000","last_undegraded":"2024-09-21T03:40:03.911493+0000","last_fullsized":"2024-09-21T03:40:03.911493+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:00.876417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:00.876417+0000","last_clean_scrub_stamp":"2024-09-21T03:40:00.876417+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:01:26.321596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-21T03:40:02.920474+0000","last_change":"2024-09-21T03:40:02.917342+0000","last_active":"2024-09-21T03:40:02.920474+0000","last_peered":"2024-09-21T03:40:02.920474+0000","last_clean":"2024-09-21T03:40:02.920474+0000","last_became_active":"2024-09-21T03:40:02.916990+0000","last_became_peered":"2024-09-21T03:40:02.916990+0000","last_unstale":"2024-09-21T03:40:02.920474+0000","last_undegraded":"2024-09-21T03:40:02.920474+0000","last_fullsized":"2024-09-21T03:40:02.920474+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_clean_scrub_stamp":"2024-09-21T03:40:01.878704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:22:30.311493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1e","version":"93'1","reported_seq":14,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-21T03:40:02.916928+0000","last_change":"2024-09-21T03:40:01.912954+0000","last_active":"2024-09-21T03:40:02.916928+0000","last_peered":"2024-09-21T03:40:02.916928+0000","last_clean":"2024-09-21T03:40:02.916928+0000","last_became_active":"2024-09-21T03:40:01.912245+0000","last_became_peered":"2024-09-21T03:40:01.912245+0000","last_unstale":"2024-09-21T03:40:02.916928+0000","last_undegraded":"2024-09-21T03:40:02.916928+0000","last_fullsized":"2024-09-21T03:40:02.916928+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:00.876417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:00.876417+0000","last_clean_scrub_stamp":"2024-09-21T03:40:00.876417+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:42:38.676959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-21T03:40:02.915646+0000","last_change":"2024-09-21T03:40:02.914079+0000","last_active":"2024-09-21T03:40:02.915646+0000","last_peered":"2024-09-21T03:40:02.915646+0000","last_clean":"2024-09-21T03:40:02.915646+0000","last_became_active":"2024-09-21T03:40:02.913725+0000","last_became_peered":"2024-09-21T03:40:02.913725+0000","last_unstale":"2024-09-21T03:40:02.915646+0000","last_undegraded":"2024-09-21T03:40:02.915646+0000","last_fullsized":"2024-09-21T03:40:02.915646+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_clean_scrub_stamp":"2024-09-21T03:40:01.878704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:34:33.361204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1d","version":"93'2","reported_seq":17,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T03:40:03.911178+0000","last_change":"2024-09-21T03:40:01.904239+0000","last_active":"2024-09-21T03:40:03.911178+0000","last_peered":"2024-09-21T03:40:03.911178+0000","last_clean":"2024-09-21T03:40:03.911178+0000","last_became_active":"2024-09-21T03:40:01.903799+0000","last_became_peered":"2024-09-21T03:40:01.903799+0000","last_unstale":"2024-09-21T03:40:03.911178+0000","last_undegraded":"2024-09-21T03:40:03.911178+0000","last_fullsized":"2024-09-21T03:40:03.911178+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:00.876417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:00.876417+0000","last_clean_scrub_stamp":"2024-09-21T03:40:00.876417+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:33:53.314541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-21T03:40:02.922069+0000","last_change":"2024-09-21T03:40:02.912850+0000","last_active":"2024-09-21T03:40:02.922069+0000","last_peered":"2024-09-21T03:40:02.922069+0000","last_clean":"2024-09-21T03:40:02.922069+0000","last_became_active":"2024-09-21T03:40:02.912563+0000","last_became_peered":"2024-09-21T03:40:02.912563+0000","last_unstale":"2024-09-21T03:40:02.922069+0000","last_undegraded":"2024-09-21T03:40:02.922069+0000","last_fullsized":"2024-09-21T03:40:02.922069+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_clean_scrub_stamp":"2024-09-21T03:40:01.878704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:56:47.573639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1c","version":"93'2","reported_seq":15,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-21T03:40:02.915090+0000","last_change":"2024-09-21T03:40:01.910064+0000","last_active":"2024-09-21T03:40:02.915090+0000","last_peered":"2024-09-21T03:40:02.915090+0000","last_clean":"2024-09-21T03:40:02.915090+0000","last_became_active":"2024-09-21T03:40:01.909721+0000","last_became_peered":"2024-09-21T03:40:01.909721+0000","last_unstale":"2024-09-21T03:40:02.915090+0000","last_undegraded":"2024-09-21T03:40:02.915090+0000","last_fullsized":"2024-09-21T03:40:02.915090+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:00.876417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:00.876417+0000","last_clean_scrub_stamp":"2024-09-21T03:40:00.876417+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:29:24.100238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-21T03:40:01.896546+0000","last_change":"2024-09-21T03:40:01.896546+0000","last_active":"2024-09-21T03:40:01.878704+0000","last_peered":"2024-09-21T03:40:01.878704+0000","last_clean":"2024-09-21T03:40:01.878704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:40:01.896546+0000","last_undegraded":"2024-09-21T03:40:01.896546+0000","last_fullsized":"2024-09-21T03:40:01.896546+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_clean_scrub_stamp":"2024-09-21T03:40:01.878704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.1b","version":"93'1","reported_seq":14,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-21T03:40:02.920027+0000","last_change":"2024-09-21T03:40:01.911561+0000","last_active":"2024-09-21T03:40:02.920027+0000","last_peered":"2024-09-21T03:40:02.920027+0000","last_clean":"2024-09-21T03:40:02.920027+0000","last_became_active":"2024-09-21T03:40:01.911068+0000","last_became_peered":"2024-09-21T03:40:01.911068+0000","last_unstale":"2024-09-21T03:40:02.920027+0000","last_undegraded":"2024-09-21T03:40:02.920027+0000","last_fullsized":"2024-09-21T03:40:02.920027+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:00.876417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:00.876417+0000","last_clean_scrub_stamp":"2024-09-21T03:40:00.876417+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:22:36.183830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-21T03:40:02.909921+0000","last_change":"2024-09-21T03:40:02.908192+0000","last_active":"2024-09-21T03:40:02.909921+0000","last_peered":"2024-09-21T03:40:02.909921+0000","last_clean":"2024-09-21T03:40:02.909921+0000","last_became_active":"2024-09-21T03:40:02.907881+0000","last_became_peered":"2024-09-21T03:40:02.907881+0000","last_unstale":"2024-09-21T03:40:02.909921+0000","last_undegraded":"2024-09-21T03:40:02.909921+0000","last_fullsized":"2024-09-21T03:40:02.909921+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_clean_scrub_stamp":"2024-09-21T03:40:01.878704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:15:36.427378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-21T03:40:01.904698+0000","last_change":"2024-09-21T03:40:01.903860+0000","last_active":"2024-09-21T03:40:01.904698+0000","last_peered":"2024-09-21T03:40:01.904698+0000","last_clean":"2024-09-21T03:40:01.904698+0000","last_became_active":"2024-09-21T03:40:01.903486+0000","last_became_peered":"2024-09-21T03:40:01.903486+0000","last_unstale":"2024-09-21T03:40:01.904698+0000","last_undegraded":"2024-09-21T03:40:01.904698+0000","last_fullsized":"2024-09-21T03:40:01.904698+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:00.876417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:00.876417+0000","last_clean_scrub_stamp":"2024-09-21T03:40:00.876417+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:19:46.518928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-21T03:40:02.914054+0000","last_change":"2024-09-21T03:40:02.912360+0000","last_active":"2024-09-21T03:40:02.914054+0000","last_peered":"2024-09-21T03:40:02.914054+0000","last_clean":"2024-09-21T03:40:02.914054+0000","last_became_active":"2024-09-21T03:40:02.912052+0000","last_became_peered":"2024-09-21T03:40:02.912052+0000","last_unstale":"2024-09-21T03:40:02.914054+0000","last_undegraded":"2024-09-21T03:40:02.914054+0000","last_fullsized":"2024-09-21T03:40:02.914054+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_clean_scrub_stamp":"2024-09-21T03:40:01.878704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:21:25.884628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":13,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-21T03:40:02.906096+0000","last_change":"2024-09-21T03:40:01.907992+0000","last_active":"2024-09-21T03:40:02.906096+0000","last_peered":"2024-09-21T03:40:02.906096+0000","last_clean":"2024-09-21T03:40:02.906096+0000","last_became_active":"2024-09-21T03:40:01.907676+0000","last_became_peered":"2024-09-21T03:40:01.907676+0000","last_unstale":"2024-09-21T03:40:02.906096+0000","last_undegraded":"2024-09-21T03:40:02.906096+0000","last_fullsized":"2024-09-21T03:40:02.906096+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:00.876417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:00.876417+0000","last_clean_scrub_stamp":"2024-09-21T03:40:00.876417+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:06:01.990231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-21T03:40:02.929905+0000","last_change":"2024-09-21T03:40:02.928175+0000","last_active":"2024-09-21T03:40:02.929905+0000","last_peered":"2024-09-21T03:40:02.929905+0000","last_clean":"2024-09-21T03:40:02.929905+0000","last_became_active":"2024-09-21T03:40:02.927778+0000","last_became_peered":"2024-09-21T03:40:02.927778+0000","last_unstale":"2024-09-21T03:40:02.929905+0000","last_undegraded":"2024-09-21T03:40:02.929905+0000","last_fullsized":"2024-09-21T03:40:02.929905+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_clean_scrub_stamp":"2024-09-21T03:40:01.878704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:37:57.479481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T03:40:03.910522+0000","last_change":"2024-09-21T03:40:01.896514+0000","last_active":"2024-09-21T03:40:03.910522+0000","last_peered":"2024-09-21T03:40:03.910522+0000","last_clean":"2024-09-21T03:40:03.910522+0000","last_became_active":"2024-09-21T03:40:01.896081+0000","last_became_peered":"2024-09-21T03:40:01.896081+0000","last_unstale":"2024-09-21T03:40:03.910522+0000","last_undegraded":"2024-09-21T03:40:03.910522+0000","last_fullsized":"2024-09-21T03:40:03.910522+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:00.876417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:00.876417+0000","last_clean_scrub_stamp":"2024-09-21T03:40:00.876417+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:41:21.728049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T03:40:03.911532+0000","last_change":"2024-09-21T03:40:02.915883+0000","last_active":"2024-09-21T03:40:03.911532+0000","last_peered":"2024-09-21T03:40:03.911532+0000","last_clean":"2024-09-21T03:40:03.911532+0000","last_became_active":"2024-09-21T03:40:02.915585+0000","last_became_peered":"2024-09-21T03:40:02.915585+0000","last_unstale":"2024-09-21T03:40:03.911532+0000","last_undegraded":"2024-09-21T03:40:03.911532+0000","last_fullsized":"2024-09-21T03:40:03.911532+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_clean_scrub_stamp":"2024-09-21T03:40:01.878704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:16:25.774858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T03:40:03.911546+0000","last_change":"2024-09-21T03:40:01.904264+0000","last_active":"2024-09-21T03:40:03.911546+0000","last_peered":"2024-09-21T03:40:03.911546+0000","last_clean":"2024-09-21T03:40:03.911546+0000","last_became_active":"2024-09-21T03:40:01.903744+0000","last_became_peered":"2024-09-21T03:40:01.903744+0000","last_unstale":"2024-09-21T03:40:03.911546+0000","last_undegraded":"2024-09-21T03:40:03.911546+0000","last_fullsized":"2024-09-21T03:40:03.911546+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:00.876417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:00.876417+0000","last_clean_scrub_stamp":"2024-09-21T03:40:00.876417+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:31:34.629665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-21T03:40:02.906339+0000","last_change":"2024-09-21T03:40:02.899323+0000","last_active":"2024-09-21T03:40:02.906339+0000","last_peered":"2024-09-21T03:40:02.906339+0000","last_clean":"2024-09-21T03:40:02.906339+0000","last_became_active":"2024-09-21T03:40:02.899022+0000","last_became_peered":"2024-09-21T03:40:02.899022+0000","last_unstale":"2024-09-21T03:40:02.906339+0000","last_undegraded":"2024-09-21T03:40:02.906339+0000","last_fullsized":"2024-09-21T03:40:02.906339+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_clean_scrub_stamp":"2024-09-21T03:40:01.878704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:04:00.546812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":13,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-21T03:40:02.897933+0000","last_change":"2024-09-21T03:40:01.905670+0000","last_active":"2024-09-21T03:40:02.897933+0000","last_peered":"2024-09-21T03:40:02.897933+0000","last_clean":"2024-09-21T03:40:02.897933+0000","last_became_active":"2024-09-21T03:40:01.905260+0000","last_became_peered":"2024-09-21T03:40:01.905260+0000","last_unstale":"2024-09-21T03:40:02.897933+0000","last_undegraded":"2024-09-21T03:40:02.897933+0000","last_fullsized":"2024-09-21T03:40:02.897933+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:00.876417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:00.876417+0000","last_clean_scrub_stamp":"2024-09-21T03:40:00.876417+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:20:54.046019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-21T03:40:02.926121+0000","last_change":"2024-09-21T03:40:02.922314+0000","last_active":"2024-09-21T03:40:02.926121+0000","last_peered":"2024-09-21T03:40:02.926121+0000","last_clean":"2024-09-21T03:40:02.926121+0000","last_became_active":"2024-09-21T03:40:02.922018+0000","last_became_peered":"2024-09-21T03:40:02.922018+0000","last_unstale":"2024-09-21T03:40:02.926121+0000","last_undegraded":"2024-09-21T03:40:02.926121+0000","last_fullsized":"2024-09-21T03:40:02.926121+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_clean_scrub_stamp":"2024-09-21T03:40:01.878704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:39:20.895207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"93'3","reported_seq":16,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-21T03:40:02.923589+0000","last_change":"2024-09-21T03:40:01.902013+0000","last_active":"2024-09-21T03:40:02.923589+0000","last_peered":"2024-09-21T03:40:02.923589+0000","last_clean":"2024-09-21T03:40:02.923589+0000","last_became_active":"2024-09-21T03:40:01.901656+0000","last_became_peered":"2024-09-21T03:40:01.901656+0000","last_unstale":"2024-09-21T03:40:02.923589+0000","last_undegraded":"2024-09-21T03:40:02.923589+0000","last_fullsized":"2024-09-21T03:40:02.923589+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:00.876417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:00.876417+0000","last_clean_scrub_stamp":"2024-09-21T03:40:00.876417+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:06:22.432740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-21T03:40:02.920492+0000","last_change":"2024-09-21T03:40:02.914504+0000","last_active":"2024-09-21T03:40:02.920492+0000","last_peered":"2024-09-21T03:40:02.920492+0000","last_clean":"2024-09-21T03:40:02.920492+0000","last_became_active":"2024-09-21T03:40:02.914193+0000","last_became_peered":"2024-09-21T03:40:02.914193+0000","last_unstale":"2024-09-21T03:40:02.920492+0000","last_undegraded":"2024-09-21T03:40:02.920492+0000","last_fullsized":"2024-09-21T03:40:02.920492+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_clean_scrub_stamp":"2024-09-21T03:40:01.878704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:43:42.170185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.14","version":"93'6","reported_seq":19,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-21T03:40:02.930286+0000","last_change":"2024-09-21T03:40:01.907985+0000","last_active":"2024-09-21T03:40:02.930286+0000","last_peered":"2024-09-21T03:40:02.930286+0000","last_clean":"2024-09-21T03:40:02.930286+0000","last_became_active":"2024-09-21T03:40:01.907666+0000","last_became_peered":"2024-09-21T03:40:01.907666+0000","last_unstale":"2024-09-21T03:40:02.930286+0000","last_undegraded":"2024-09-21T03:40:02.930286+0000","last_fullsized":"2024-09-21T03:40:02.930286+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:00.876417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:00.876417+0000","last_clean_scrub_stamp":"2024-09-21T03:40:00.876417+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:33:59.304186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-21T03:40:02.926070+0000","last_change":"2024-09-21T03:40:02.920112+0000","last_active":"2024-09-21T03:40:02.926070+0000","last_peered":"2024-09-21T03:40:02.926070+0000","last_clean":"2024-09-21T03:40:02.926070+0000","last_became_active":"2024-09-21T03:40:02.919797+0000","last_became_peered":"2024-09-21T03:40:02.919797+0000","last_unstale":"2024-09-21T03:40:02.926070+0000","last_undegraded":"2024-09-21T03:40:02.926070+0000","last_fullsized":"2024-09-21T03:40:02.926070+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_clean_scrub_stamp":"2024-09-21T03:40:01.878704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:45:24.118610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.13","version":"93'2","reported_seq":17,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T03:40:03.911214+0000","last_change":"2024-09-21T03:40:01.904443+0000","last_active":"2024-09-21T03:40:03.911214+0000","last_peered":"2024-09-21T03:40:03.911214+0000","last_clean":"2024-09-21T03:40:03.911214+0000","last_became_active":"2024-09-21T03:40:01.903631+0000","last_became_peered":"2024-09-21T03:40:01.903631+0000","last_unstale":"2024-09-21T03:40:03.911214+0000","last_undegraded":"2024-09-21T03:40:03.911214+0000","last_fullsized":"2024-09-21T03:40:03.911214+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:00.876417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:00.876417+0000","last_clean_scrub_stamp":"2024-09-21T03:40:00.876417+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:04:48.156614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T03:40:03.910855+0000","last_change":"2024-09-21T03:40:02.912394+0000","last_active":"2024-09-21T03:40:03.910855+0000","last_peered":"2024-09-21T03:40:03.910855+0000","last_clean":"2024-09-21T03:40:03.910855+0000","last_became_active":"2024-09-21T03:40:02.912063+0000","last_became_peered":"2024-09-21T03:40:02.912063+0000","last_unstale":"2024-09-21T03:40:03.910855+0000","last_undegraded":"2024-09-21T03:40:03.910855+0000","last_fullsized":"2024-09-21T03:40:03.910855+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_clean_scrub_stamp":"2024-09-21T03:40:01.878704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:09:36.512689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":13,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-21T03:40:02.913044+0000","last_change":"2024-09-21T03:40:01.910583+0000","last_active":"2024-09-21T03:40:02.913044+0000","last_peered":"2024-09-21T03:40:02.913044+0000","last_clean":"2024-09-21T03:40:02.913044+0000","last_became_active":"2024-09-21T03:40:01.910276+0000","last_became_peered":"2024-09-21T03:40:01.910276+0000","last_unstale":"2024-09-21T03:40:02.913044+0000","last_undegraded":"2024-09-21T03:40:02.913044+0000","last_fullsized":"2024-09-21T03:40:02.913044+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:00.876417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:00.876417+0000","last_clean_scrub_stamp":"2024-09-21T03:40:00.876417+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:03:39.665337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-21T03:40:02.927760+0000","last_change":"2024-09-21T03:40:02.925807+0000","last_active":"2024-09-21T03:40:02.927760+0000","last_peered":"2024-09-21T03:40:02.927760+0000","last_clean":"2024-09-21T03:40:02.927760+0000","last_became_active":"2024-09-21T03:40:02.925472+0000","last_became_peered":"2024-09-21T03:40:02.925472+0000","last_unstale":"2024-09-21T03:40:02.927760+0000","last_undegraded":"2024-09-21T03:40:02.927760+0000","last_fullsized":"2024-09-21T03:40:02.927760+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_clean_scrub_stamp":"2024-09-21T03:40:01.878704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:31:25.195627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":13,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-21T03:40:02.913473+0000","last_change":"2024-09-21T03:40:01.904974+0000","last_active":"2024-09-21T03:40:02.913473+0000","last_peered":"2024-09-21T03:40:02.913473+0000","last_clean":"2024-09-21T03:40:02.913473+0000","last_became_active":"2024-09-21T03:40:01.904403+0000","last_became_peered":"2024-09-21T03:40:01.904403+0000","last_unstale":"2024-09-21T03:40:02.913473+0000","last_undegraded":"2024-09-21T03:40:02.913473+0000","last_fullsized":"2024-09-21T03:40:02.913473+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:00.876417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:00.876417+0000","last_clean_scrub_stamp":"2024-09-21T03:40:00.876417+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:54:02.728898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-21T03:40:01.896863+0000","last_change":"2024-09-21T03:40:01.896863+0000","last_active":"2024-09-21T03:40:01.878704+0000","last_peered":"2024-09-21T03:40:01.878704+0000","last_clean":"2024-09-21T03:40:01.878704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:40:01.896863+0000","last_undegraded":"2024-09-21T03:40:01.896863+0000","last_fullsized":"2024-09-21T03:40:01.896863+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_clean_scrub_stamp":"2024-09-21T03:40:01.878704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.10","version":"93'8","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-21T03:40:02.927016+0000","last_change":"2024-09-21T03:40:01.900585+0000","last_active":"2024-09-21T03:40:02.927016+0000","last_peered":"2024-09-21T03:40:02.927016+0000","last_clean":"2024-09-21T03:40:02.927016+0000","last_became_active":"2024-09-21T03:40:01.900220+0000","last_became_peered":"2024-09-21T03:40:01.900220+0000","last_unstale":"2024-09-21T03:40:02.927016+0000","last_undegraded":"2024-09-21T03:40:02.927016+0000","last_fullsized":"2024-09-21T03:40:02.927016+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:00.876417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:00.876417+0000","last_clean_scrub_stamp":"2024-09-21T03:40:00.876417+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:58:19.443646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-21T03:40:01.898325+0000","last_change":"2024-09-21T03:40:01.898325+0000","last_active":"2024-09-21T03:40:01.878704+0000","last_peered":"2024-09-21T03:40:01.878704+0000","last_clean":"2024-09-21T03:40:01.878704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:40:01.898325+0000","last_undegraded":"2024-09-21T03:40:01.898325+0000","last_fullsized":"2024-09-21T03:40:01.898325+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_clean_scrub_stamp":"2024-09-21T03:40:01.878704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.f","version":"93'1","reported_seq":14,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-21T03:40:02.913110+0000","last_change":"2024-09-21T03:40:01.912993+0000","last_active":"2024-09-21T03:40:02.913110+0000","last_peered":"2024-09-21T03:40:02.913110+0000","last_clean":"2024-09-21T03:40:02.913110+0000","last_became_active":"2024-09-21T03:40:01.912652+0000","last_became_peered":"2024-09-21T03:40:01.912652+0000","last_unstale":"2024-09-21T03:40:02.913110+0000","last_undegraded":"2024-09-21T03:40:02.913110+0000","last_fullsized":"2024-09-21T03:40:02.913110+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:00.876417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:00.876417+0000","last_clean_scrub_stamp":"2024-09-21T03:40:00.876417+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:27:14.167603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-21T03:40:02.909960+0000","last_change":"2024-09-21T03:40:02.907423+0000","last_active":"2024-09-21T03:40:02.909960+0000","last_peered":"2024-09-21T03:40:02.909960+0000","last_clean":"2024-09-21T03:40:02.909960+0000","last_became_active":"2024-09-21T03:40:02.907141+0000","last_became_peered":"2024-09-21T03:40:02.907141+0000","last_unstale":"2024-09-21T03:40:02.909960+0000","last_undegraded":"2024-09-21T03:40:02.909960+0000","last_fullsized":"2024-09-21T03:40:02.909960+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_clean_scrub_stamp":"2024-09-21T03:40:01.878704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:14:25.530924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.e","version":"93'2","reported_seq":15,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-21T03:40:02.920840+0000","last_change":"2024-09-21T03:40:01.913277+0000","last_active":"2024-09-21T03:40:02.920840+0000","last_peered":"2024-09-21T03:40:02.920840+0000","last_clean":"2024-09-21T03:40:02.920840+0000","last_became_active":"2024-09-21T03:40:01.912647+0000","last_became_peered":"2024-09-21T03:40:01.912647+0000","last_unstale":"2024-09-21T03:40:02.920840+0000","last_undegraded":"2024-09-21T03:40:02.920840+0000","last_fullsized":"2024-09-21T03:40:02.920840+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:00.876417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:00.876417+0000","last_clean_scrub_stamp":"2024-09-21T03:40:00.876417+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:54:02.728898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T03:40:03.911235+0000","last_change":"2024-09-21T03:40:02.919564+0000","last_active":"2024-09-21T03:40:03.911235+0000","last_peered":"2024-09-21T03:40:03.911235+0000","last_clean":"2024-09-21T03:40:03.911235+0000","last_became_active":"2024-09-21T03:40:02.917147+0000","last_became_peered":"2024-09-21T03:40:02.917147+0000","last_unstale":"2024-09-21T03:40:03.911235+0000","last_undegraded":"2024-09-21T03:40:03.911235+0000","last_fullsized":"2024-09-21T03:40:03.911235+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_clean_scrub_stamp":"2024-09-21T03:40:01.878704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:07:04.991136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.d","version":"93'1","reported_seq":14,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-21T03:40:02.922147+0000","last_change":"2024-09-21T03:40:01.906570+0000","last_active":"2024-09-21T03:40:02.922147+0000","last_peered":"2024-09-21T03:40:02.922147+0000","last_clean":"2024-09-21T03:40:02.922147+0000","last_became_active":"2024-09-21T03:40:01.906254+0000","last_became_peered":"2024-09-21T03:40:01.906254+0000","last_unstale":"2024-09-21T03:40:02.922147+0000","last_undegraded":"2024-09-21T03:40:02.922147+0000","last_fullsized":"2024-09-21T03:40:02.922147+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:00.876417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:00.876417+0000","last_clean_scrub_stamp":"2024-09-21T03:40:00.876417+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:03:39.665337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-21T03:40:02.918625+0000","last_change":"2024-09-21T03:40:02.912020+0000","last_active":"2024-09-21T03:40:02.918625+0000","last_peered":"2024-09-21T03:40:02.918625+0000","last_clean":"2024-09-21T03:40:02.918625+0000","last_became_active":"2024-09-21T03:40:02.911699+0000","last_became_peered":"2024-09-21T03:40:02.911699+0000","last_unstale":"2024-09-21T03:40:02.918625+0000","last_undegraded":"2024-09-21T03:40:02.918625+0000","last_fullsized":"2024-09-21T03:40:02.918625+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_clean_scrub_stamp":"2024-09-21T03:40:01.878704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:27:20.703323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.b","version":"93'2","reported_seq":15,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-21T03:40:02.926632+0000","last_change":"2024-09-21T03:40:01.899140+0000","last_active":"2024-09-21T03:40:02.926632+0000","last_peered":"2024-09-21T03:40:02.926632+0000","last_clean":"2024-09-21T03:40:02.926632+0000","last_became_active":"2024-09-21T03:40:01.898802+0000","last_became_peered":"2024-09-21T03:40:01.898802+0000","last_unstale":"2024-09-21T03:40:02.926632+0000","last_undegraded":"2024-09-21T03:40:02.926632+0000","last_fullsized":"2024-09-21T03:40:02.926632+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:00.876417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:00.876417+0000","last_clean_scrub_stamp":"2024-09-21T03:40:00.876417+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:31:06.035346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-21T03:40:02.916495+0000","last_change":"2024-09-21T03:40:02.910834+0000","last_active":"2024-09-21T03:40:02.916495+0000","last_peered":"2024-09-21T03:40:02.916495+0000","last_clean":"2024-09-21T03:40:02.916495+0000","last_became_active":"2024-09-21T03:40:02.910535+0000","last_became_peered":"2024-09-21T03:40:02.910535+0000","last_unstale":"2024-09-21T03:40:02.916495+0000","last_undegraded":"2024-09-21T03:40:02.916495+0000","last_fullsized":"2024-09-21T03:40:02.916495+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_clean_scrub_stamp":"2024-09-21T03:40:01.878704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:34:00.306473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":13,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-21T03:40:02.897708+0000","last_change":"2024-09-21T03:40:01.907352+0000","last_active":"2024-09-21T03:40:02.897708+0000","last_peered":"2024-09-21T03:40:02.897708+0000","last_clean":"2024-09-21T03:40:02.897708+0000","last_became_active":"2024-09-21T03:40:01.906943+0000","last_became_peered":"2024-09-21T03:40:01.906943+0000","last_unstale":"2024-09-21T03:40:02.897708+0000","last_undegraded":"2024-09-21T03:40:02.897708+0000","last_fullsized":"2024-09-21T03:40:02.897708+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:00.876417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:00.876417+0000","last_clean_scrub_stamp":"2024-09-21T03:40:00.876417+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:09:44.594955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-21T03:40:02.916083+0000","last_change":"2024-09-21T03:40:02.911571+0000","last_active":"2024-09-21T03:40:02.916083+0000","last_peered":"2024-09-21T03:40:02.916083+0000","last_clean":"2024-09-21T03:40:02.916083+0000","last_became_active":"2024-09-21T03:40:02.911066+0000","last_became_peered":"2024-09-21T03:40:02.911066+0000","last_unstale":"2024-09-21T03:40:02.916083+0000","last_undegraded":"2024-09-21T03:40:02.916083+0000","last_fullsized":"2024-09-21T03:40:02.916083+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_clean_scrub_stamp":"2024-09-21T03:40:01.878704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:39:20.895207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":13,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-21T03:40:02.891635+0000","last_change":"2024-09-21T03:40:01.905588+0000","last_active":"2024-09-21T03:40:02.891635+0000","last_peered":"2024-09-21T03:40:02.891635+0000","last_clean":"2024-09-21T03:40:02.891635+0000","last_became_active":"2024-09-21T03:40:01.905258+0000","last_became_peered":"2024-09-21T03:40:01.905258+0000","last_unstale":"2024-09-21T03:40:02.891635+0000","last_undegraded":"2024-09-21T03:40:02.891635+0000","last_fullsized":"2024-09-21T03:40:02.891635+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:00.876417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:00.876417+0000","last_clean_scrub_stamp":"2024-09-21T03:40:00.876417+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:05:08.585778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-21T03:40:02.926253+0000","last_change":"2024-09-21T03:40:02.922401+0000","last_active":"2024-09-21T03:40:02.926253+0000","last_peered":"2024-09-21T03:40:02.926253+0000","last_clean":"2024-09-21T03:40:02.926253+0000","last_became_active":"2024-09-21T03:40:02.922131+0000","last_became_peered":"2024-09-21T03:40:02.922131+0000","last_unstale":"2024-09-21T03:40:02.926253+0000","last_undegraded":"2024-09-21T03:40:02.926253+0000","last_fullsized":"2024-09-21T03:40:02.926253+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_clean_scrub_stamp":"2024-09-21T03:40:01.878704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:22:48.720231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-21T03:40:01.909500+0000","last_change":"2024-09-21T03:40:01.908592+0000","last_active":"2024-09-21T03:40:01.909500+0000","last_peered":"2024-09-21T03:40:01.909500+0000","last_clean":"2024-09-21T03:40:01.909500+0000","last_became_active":"2024-09-21T03:40:01.908171+0000","last_became_peered":"2024-09-21T03:40:01.908171+0000","last_unstale":"2024-09-21T03:40:01.909500+0000","last_undegraded":"2024-09-21T03:40:01.909500+0000","last_fullsized":"2024-09-21T03:40:01.909500+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:00.876417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:00.876417+0000","last_clean_scrub_stamp":"2024-09-21T03:40:00.876417+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:56:30.916353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-21T03:40:02.921431+0000","last_change":"2024-09-21T03:40:02.917952+0000","last_active":"2024-09-21T03:40:02.921431+0000","last_peered":"2024-09-21T03:40:02.921431+0000","last_clean":"2024-09-21T03:40:02.921431+0000","last_became_active":"2024-09-21T03:40:02.917563+0000","last_became_peered":"2024-09-21T03:40:02.917563+0000","last_unstale":"2024-09-21T03:40:02.921431+0000","last_undegraded":"2024-09-21T03:40:02.921431+0000","last_fullsized":"2024-09-21T03:40:02.921431+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_clean_scrub_stamp":"2024-09-21T03:40:01.878704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:18:39.471497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.7","version":"93'2","reported_seq":15,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-21T03:40:02.906206+0000","last_change":"2024-09-21T03:40:01.910590+0000","last_active":"2024-09-21T03:40:02.906206+0000","last_peered":"2024-09-21T03:40:02.906206+0000","last_clean":"2024-09-21T03:40:02.906206+0000","last_became_active":"2024-09-21T03:40:01.910244+0000","last_became_peered":"2024-09-21T03:40:01.910244+0000","last_unstale":"2024-09-21T03:40:02.906206+0000","last_undegraded":"2024-09-21T03:40:02.906206+0000","last_fullsized":"2024-09-21T03:40:02.906206+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:00.876417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:00.876417+0000","last_clean_scrub_stamp":"2024-09-21T03:40:00.876417+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:06:19.242099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-21T03:40:01.897889+0000","last_change":"2024-09-21T03:40:01.897889+0000","last_active":"2024-09-21T03:40:01.878704+0000","last_peered":"2024-09-21T03:40:01.878704+0000","last_clean":"2024-09-21T03:40:01.878704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:40:01.897889+0000","last_undegraded":"2024-09-21T03:40:01.897889+0000","last_fullsized":"2024-09-21T03:40:01.897889+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_clean_scrub_stamp":"2024-09-21T03:40:01.878704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":13,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-21T03:40:02.892098+0000","last_change":"2024-09-21T03:40:01.901781+0000","last_active":"2024-09-21T03:40:02.892098+0000","last_peered":"2024-09-21T03:40:02.892098+0000","last_clean":"2024-09-21T03:40:02.892098+0000","last_became_active":"2024-09-21T03:40:01.901440+0000","last_became_peered":"2024-09-21T03:40:01.901440+0000","last_unstale":"2024-09-21T03:40:02.892098+0000","last_undegraded":"2024-09-21T03:40:02.892098+0000","last_fullsized":"2024-09-21T03:40:02.892098+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:00.876417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:00.876417+0000","last_clean_scrub_stamp":"2024-09-21T03:40:00.876417+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:21:08.007365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-21T03:40:01.898442+0000","last_change":"2024-09-21T03:40:01.898442+0000","last_active":"2024-09-21T03:40:01.878704+0000","last_peered":"2024-09-21T03:40:01.878704+0000","last_clean":"2024-09-21T03:40:01.878704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:40:01.898442+0000","last_undegraded":"2024-09-21T03:40:01.898442+0000","last_fullsized":"2024-09-21T03:40:01.898442+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_clean_scrub_stamp":"2024-09-21T03:40:01.878704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":13,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-21T03:40:02.901203+0000","last_change":"2024-09-21T03:40:01.910362+0000","last_active":"2024-09-21T03:40:02.901203+0000","last_peered":"2024-09-21T03:40:02.901203+0000","last_clean":"2024-09-21T03:40:02.901203+0000","last_became_active":"2024-09-21T03:40:01.910042+0000","last_became_peered":"2024-09-21T03:40:01.910042+0000","last_unstale":"2024-09-21T03:40:02.901203+0000","last_undegraded":"2024-09-21T03:40:02.901203+0000","last_fullsized":"2024-09-21T03:40:02.901203+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:00.876417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:00.876417+0000","last_clean_scrub_stamp":"2024-09-21T03:40:00.876417+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:45:23.116322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T03:40:03.911380+0000","last_change":"2024-09-21T03:40:02.912299+0000","last_active":"2024-09-21T03:40:03.911380+0000","last_peered":"2024-09-21T03:40:03.911380+0000","last_clean":"2024-09-21T03:40:03.911380+0000","last_became_active":"2024-09-21T03:40:02.911984+0000","last_became_peered":"2024-09-21T03:40:02.911984+0000","last_unstale":"2024-09-21T03:40:03.911380+0000","last_undegraded":"2024-09-21T03:40:03.911380+0000","last_fullsized":"2024-09-21T03:40:03.911380+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_clean_scrub_stamp":"2024-09-21T03:40:01.878704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:49:47.824284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":13,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-21T03:40:02.892586+0000","last_change":"2024-09-21T03:40:01.907327+0000","last_active":"2024-09-21T03:40:02.892586+0000","last_peered":"2024-09-21T03:40:02.892586+0000","last_clean":"2024-09-21T03:40:02.892586+0000","last_became_active":"2024-09-21T03:40:01.906998+0000","last_became_peered":"2024-09-21T03:40:01.906998+0000","last_unstale":"2024-09-21T03:40:02.892586+0000","last_undegraded":"2024-09-21T03:40:02.892586+0000","last_fullsized":"2024-09-21T03:40:02.892586+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:00.876417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:00.876417+0000","last_clean_scrub_stamp":"2024-09-21T03:40:00.876417+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:16:30.469693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-21T03:40:02.918576+0000","last_change":"2024-09-21T03:40:02.911748+0000","last_active":"2024-09-21T03:40:02.918576+0000","last_peered":"2024-09-21T03:40:02.918576+0000","last_clean":"2024-09-21T03:40:02.918576+0000","last_became_active":"2024-09-21T03:40:02.911352+0000","last_became_peered":"2024-09-21T03:40:02.911352+0000","last_unstale":"2024-09-21T03:40:02.918576+0000","last_undegraded":"2024-09-21T03:40:02.918576+0000","last_fullsized":"2024-09-21T03:40:02.918576+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_clean_scrub_stamp":"2024-09-21T03:40:01.878704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:34:02.996149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":13,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-21T03:40:02.917104+0000","last_change":"2024-09-21T03:40:01.912608+0000","last_active":"2024-09-21T03:40:02.917104+0000","last_peered":"2024-09-21T03:40:02.917104+0000","last_clean":"2024-09-21T03:40:02.917104+0000","last_became_active":"2024-09-21T03:40:01.912288+0000","last_became_peered":"2024-09-21T03:40:01.912288+0000","last_unstale":"2024-09-21T03:40:02.917104+0000","last_undegraded":"2024-09-21T03:40:02.917104+0000","last_fullsized":"2024-09-21T03:40:02.917104+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:00.876417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:00.876417+0000","last_clean_scrub_stamp":"2024-09-21T03:40:00.876417+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:27:14.167603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-21T03:40:02.926341+0000","last_change":"2024-09-21T03:40:02.924595+0000","last_active":"2024-09-21T03:40:02.926341+0000","last_peered":"2024-09-21T03:40:02.926341+0000","last_clean":"2024-09-21T03:40:02.926341+0000","last_became_active":"2024-09-21T03:40:02.924257+0000","last_became_peered":"2024-09-21T03:40:02.924257+0000","last_unstale":"2024-09-21T03:40:02.926341+0000","last_undegraded":"2024-09-21T03:40:02.926341+0000","last_fullsized":"2024-09-21T03:40:02.926341+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_clean_scrub_stamp":"2024-09-21T03:40:01.878704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:33:26.902361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T03:40:03.910623+0000","last_change":"2024-09-21T03:40:01.912018+0000","last_active":"2024-09-21T03:40:03.910623+0000","last_peered":"2024-09-21T03:40:03.910623+0000","last_clean":"2024-09-21T03:40:03.910623+0000","last_became_active":"2024-09-21T03:40:01.911663+0000","last_became_peered":"2024-09-21T03:40:01.911663+0000","last_unstale":"2024-09-21T03:40:03.910623+0000","last_undegraded":"2024-09-21T03:40:03.910623+0000","last_fullsized":"2024-09-21T03:40:03.910623+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:00.876417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:00.876417+0000","last_clean_scrub_stamp":"2024-09-21T03:40:00.876417+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:47:22.885563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-21T03:40:02.921481+0000","last_change":"2024-09-21T03:40:02.915873+0000","last_active":"2024-09-21T03:40:02.921481+0000","last_peered":"2024-09-21T03:40:02.921481+0000","last_clean":"2024-09-21T03:40:02.921481+0000","last_became_active":"2024-09-21T03:40:02.915180+0000","last_became_peered":"2024-09-21T03:40:02.915180+0000","last_unstale":"2024-09-21T03:40:02.921481+0000","last_undegraded":"2024-09-21T03:40:02.921481+0000","last_fullsized":"2024-09-21T03:40:02.921481+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_clean_scrub_stamp":"2024-09-21T03:40:01.878704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:02:41.238627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T03:40:03.910722+0000","last_change":"2024-09-21T03:40:01.904427+0000","last_active":"2024-09-21T03:40:03.910722+0000","last_peered":"2024-09-21T03:40:03.910722+0000","last_clean":"2024-09-21T03:40:03.910722+0000","last_became_active":"2024-09-21T03:40:01.904062+0000","last_became_peered":"2024-09-21T03:40:01.904062+0000","last_unstale":"2024-09-21T03:40:03.910722+0000","last_undegraded":"2024-09-21T03:40:03.910722+0000","last_fullsized":"2024-09-21T03:40:03.910722+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:00.876417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:00.876417+0000","last_clean_scrub_stamp":"2024-09-21T03:40:00.876417+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:29:20.149004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-21T03:40:02.929753+0000","last_change":"2024-09-21T03:40:02.928726+0000","last_active":"2024-09-21T03:40:02.929753+0000","last_peered":"2024-09-21T03:40:02.929753+0000","last_clean":"2024-09-21T03:40:02.929753+0000","last_became_active":"2024-09-21T03:40:02.928441+0000","last_became_peered":"2024-09-21T03:40:02.928441+0000","last_unstale":"2024-09-21T03:40:02.929753+0000","last_undegraded":"2024-09-21T03:40:02.929753+0000","last_fullsized":"2024-09-21T03:40:02.929753+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_clean_scrub_stamp":"2024-09-21T03:40:01.878704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:36:09.296049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T03:40:03.910625+0000","last_change":"2024-09-21T03:40:01.914749+0000","last_active":"2024-09-21T03:40:03.910625+0000","last_peered":"2024-09-21T03:40:03.910625+0000","last_clean":"2024-09-21T03:40:03.910625+0000","last_became_active":"2024-09-21T03:40:01.913928+0000","last_became_peered":"2024-09-21T03:40:01.913928+0000","last_unstale":"2024-09-21T03:40:03.910625+0000","last_undegraded":"2024-09-21T03:40:03.910625+0000","last_fullsized":"2024-09-21T03:40:03.910625+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:00.876417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:00.876417+0000","last_clean_scrub_stamp":"2024-09-21T03:40:00.876417+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:50:43.304717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T03:40:03.911255+0000","last_change":"2024-09-21T03:40:02.917652+0000","last_active":"2024-09-21T03:40:03.911255+0000","last_peered":"2024-09-21T03:40:03.911255+0000","last_clean":"2024-09-21T03:40:03.911255+0000","last_became_active":"2024-09-21T03:40:02.917354+0000","last_became_peered":"2024-09-21T03:40:02.917354+0000","last_unstale":"2024-09-21T03:40:03.911255+0000","last_undegraded":"2024-09-21T03:40:03.911255+0000","last_fullsized":"2024-09-21T03:40:03.911255+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_clean_scrub_stamp":"2024-09-21T03:40:01.878704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:59:43.575963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":13,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-21T03:40:02.919154+0000","last_change":"2024-09-21T03:40:01.907963+0000","last_active":"2024-09-21T03:40:02.919154+0000","last_peered":"2024-09-21T03:40:02.919154+0000","last_clean":"2024-09-21T03:40:02.919154+0000","last_became_active":"2024-09-21T03:40:01.907626+0000","last_became_peered":"2024-09-21T03:40:01.907626+0000","last_unstale":"2024-09-21T03:40:02.919154+0000","last_undegraded":"2024-09-21T03:40:02.919154+0000","last_fullsized":"2024-09-21T03:40:02.919154+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:00.876417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:00.876417+0000","last_clean_scrub_stamp":"2024-09-21T03:40:00.876417+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:42:33.254595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":250,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T03:40:03.910691+0000","last_change":"2024-09-21T03:33:25.319776+0000","last_active":"2024-09-21T03:40:03.910691+0000","last_peered":"2024-09-21T03:40:03.910691+0000","last_clean":"2024-09-21T03:40:03.910691+0000","last_became_active":"2024-09-21T03:33:25.318399+0000","last_became_peered":"2024-09-21T03:33:25.318399+0000","last_unstale":"2024-09-21T03:40:03.910691+0000","last_undegraded":"2024-09-21T03:40:03.910691+0000","last_fullsized":"2024-09-21T03:40:03.910691+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:23.016672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:23.016672+0000","last_clean_scrub_stamp":"2024-09-21T03:33:23.016672+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:32:35.906114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":37,"ondisk_log_size":37,"up":64,"acting":64,"num_store_stats":8},{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":1,"up_from":16,"seq":68719476820,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32904,"kb_used_data":1716,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338936,"statfs":{"total":96636764160,"available":96603070464,"internally_reserved":0,"allocated":1757184,"data_stored":1125259,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509524,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36468,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335372,"statfs":{"total":96636764160,"available":96599420928,"internally_reserved":0,"allocated":1138688,"data_stored":527513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":36174283},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509523,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32320,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339520,"statfs":{"total":96636764160,"available":96603668480,"internally_reserved":0,"allocated":1159168,"data_stored":529237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509524,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32308,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339532,"statfs":{"total":96636764160,"available":96603680768,"internally_reserved":0,"allocated":1146880,"data_stored":528117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509523,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32304,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339536,"statfs":{"total":96636764160,"available":96603684864,"internally_reserved":0,"allocated":1142784,"data_stored":527583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542227,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36456,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335384,"statfs":{"total":96636764160,"available":96599433216,"internally_reserved":0,"allocated":1126400,"data_stored":520346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574931,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37024,"kb_used_data":1668,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334816,"statfs":{"total":96636764160,"available":96598851584,"internally_reserved":0,"allocated":1708032,"data_stored":1104047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542228,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36452,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335388,"statfs":{"total":96636764160,"available":96599437312,"internally_reserved":0,"allocated":1126400,"data_stored":520346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T03:40:06.984 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T03:40:07.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:07.132+0000 7f2f45c94640 1 -- 172.21.15.38:0/2893808692 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2f40162590 msgr2=0x7f2f401629a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:07.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:07.132+0000 7f2f45c94640 1 --2- 172.21.15.38:0/2893808692 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2f40162590 0x7f2f401629a0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f2f4008d3b0 tx=0x7f2f3402a560 comp rx=0 tx=0).stop 2024-09-21T03:40:07.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:07.132+0000 7f2f45c94640 1 -- 172.21.15.38:0/2893808692 shutdown_connections 2024-09-21T03:40:07.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:07.133+0000 7f2f45c94640 1 --2- 172.21.15.38:0/2893808692 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2f400b2e60 0x7f2f400b3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:07.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:07.133+0000 7f2f45c94640 1 --2- 172.21.15.38:0/2893808692 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2f40162590 0x7f2f401629a0 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:07.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:07.133+0000 7f2f45c94640 1 --2- 172.21.15.38:0/2893808692 >> v2:172.21.15.195:3300/0 conn(0x7f2f400cfe00 0x7f2f400d2a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:07.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:07.133+0000 7f2f45c94640 1 -- 172.21.15.38:0/2893808692 >> 172.21.15.38:0/2893808692 conn(0x7f2f400d4440 msgr2=0x7f2f400d4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:07.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:07.133+0000 7f2f45c94640 1 -- 172.21.15.38:0/2893808692 shutdown_connections 2024-09-21T03:40:07.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:07.133+0000 7f2f45c94640 1 -- 172.21.15.38:0/2893808692 wait complete. 2024-09-21T03:40:07.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:07.134+0000 7f2f45c94640 1 Processor -- start 2024-09-21T03:40:07.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:07.135+0000 7f2f45c94640 1 -- start start 2024-09-21T03:40:07.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:07.136+0000 7f2f45c94640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2f400b2e60 0x7f2f40158030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:07.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:07.136+0000 7f2f45c94640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2f400cfe00 0x7f2f400c9720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:07.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:07.136+0000 7f2f45c94640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2f40162590 0x7f2f400c9c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:07.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:07.136+0000 7f2f45c94640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f2f400b37b0 con 0x7f2f400b2e60 2024-09-21T03:40:07.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:07.136+0000 7f2f45c94640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f2f400cf5a0 con 0x7f2f400cfe00 2024-09-21T03:40:07.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:07.136+0000 7f2f45c94640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f2f400b3930 con 0x7f2f40162590 2024-09-21T03:40:07.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:07.136+0000 7f2f3f7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2f400b2e60 0x7f2f40158030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:07.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:07.136+0000 7f2f3f7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2f400b2e60 0x7f2f40158030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:55614/0 (socket says 172.21.15.38:55614) 2024-09-21T03:40:07.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:07.136+0000 7f2f3f7fe640 1 -- 172.21.15.38:0/232981594 learned_addr learned my addr 172.21.15.38:0/232981594 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:07.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:07.136+0000 7f2f3ffff640 1 --2- 172.21.15.38:0/232981594 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2f40162590 0x7f2f400c9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:07.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:07.136+0000 7f2f3effd640 1 --2- 172.21.15.38:0/232981594 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2f400cfe00 0x7f2f400c9720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:07.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:07.136+0000 7f2f3f7fe640 1 -- 172.21.15.38:0/232981594 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2f40162590 msgr2=0x7f2f400c9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:07.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:07.136+0000 7f2f3f7fe640 1 --2- 172.21.15.38:0/232981594 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2f40162590 0x7f2f400c9c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:07.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:07.136+0000 7f2f3f7fe640 1 -- 172.21.15.38:0/232981594 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2f400cfe00 msgr2=0x7f2f400c9720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:07.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:07.136+0000 7f2f3f7fe640 1 --2- 172.21.15.38:0/232981594 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2f400cfe00 0x7f2f400c9720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:07.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:07.136+0000 7f2f3f7fe640 1 -- 172.21.15.38:0/232981594 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f401f6000 con 0x7f2f400b2e60 2024-09-21T03:40:07.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:07.136+0000 7f2f3ffff640 1 --2- 172.21.15.38:0/232981594 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2f40162590 0x7f2f400c9c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:40:07.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:07.137+0000 7f2f3f7fe640 1 --2- 172.21.15.38:0/232981594 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2f400b2e60 0x7f2f40158030 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f2f3000d740 tx=0x7f2f3000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:07.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:07.137+0000 7f2f27fff640 1 -- 172.21.15.38:0/232981594 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f30017070 con 0x7f2f400b2e60 2024-09-21T03:40:07.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:07.137+0000 7f2f27fff640 1 -- 172.21.15.38:0/232981594 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2f30009d20 con 0x7f2f400b2e60 2024-09-21T03:40:07.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:07.137+0000 7f2f45c94640 1 -- 172.21.15.38:0/232981594 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f401f62e0 con 0x7f2f400b2e60 2024-09-21T03:40:07.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:07.137+0000 7f2f27fff640 1 -- 172.21.15.38:0/232981594 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f300106c0 con 0x7f2f400b2e60 2024-09-21T03:40:07.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:07.137+0000 7f2f45c94640 1 -- 172.21.15.38:0/232981594 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f4015b830 con 0x7f2f400b2e60 2024-09-21T03:40:07.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:07.138+0000 7f2f45c94640 1 -- 172.21.15.38:0/232981594 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f400c81f0 con 0x7f2f400b2e60 2024-09-21T03:40:07.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:07.139+0000 7f2f27fff640 1 -- 172.21.15.38:0/232981594 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f2f3001a070 con 0x7f2f400b2e60 2024-09-21T03:40:07.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:07.139+0000 7f2f27fff640 1 --2- 172.21.15.38:0/232981594 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2f2c078b90 0x7f2f2c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:07.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:07.139+0000 7f2f27fff640 1 -- 172.21.15.38:0/232981594 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6101+0+0 (secure 0 0 0) 0x7f2f3009afd0 con 0x7f2f400b2e60 2024-09-21T03:40:07.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:07.139+0000 7f2f3effd640 1 --2- 172.21.15.38:0/232981594 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2f2c078b90 0x7f2f2c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:07.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:07.140+0000 7f2f3effd640 1 --2- 172.21.15.38:0/232981594 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2f2c078b90 0x7f2f2c07b0a0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f2f34033000 tx=0x7f2f34004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:07.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:07.144+0000 7f2f27fff640 1 -- 172.21.15.38:0/232981594 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f2f300633a0 con 0x7f2f400b2e60 2024-09-21T03:40:07.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:07.326+0000 7f2f45c94640 1 -- 172.21.15.38:0/232981594 --> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2f4015bad0 con 0x7f2f2c078b90 2024-09-21T03:40:07.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:07.331+0000 7f2f27fff640 1 -- 172.21.15.38:0/232981594 <== mgr.4117 v2:172.21.15.195:6800/1631901344 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252063 (secure 0 0 0) 0x7f2f4015bad0 con 0x7f2f2c078b90 2024-09-21T03:40:07.337 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:40:07.338 INFO:teuthology.orchestra.run.smithi038.stderr:dumped all 2024-09-21T03:40:07.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:07.335+0000 7f2f45c94640 1 -- 172.21.15.38:0/232981594 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2f2c078b90 msgr2=0x7f2f2c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:07.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:07.335+0000 7f2f45c94640 1 --2- 172.21.15.38:0/232981594 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2f2c078b90 0x7f2f2c07b0a0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f2f34033000 tx=0x7f2f34004730 comp rx=0 tx=0).stop 2024-09-21T03:40:07.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:07.335+0000 7f2f45c94640 1 -- 172.21.15.38:0/232981594 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2f400b2e60 msgr2=0x7f2f40158030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:07.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:07.335+0000 7f2f45c94640 1 --2- 172.21.15.38:0/232981594 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2f400b2e60 0x7f2f40158030 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f2f3000d740 tx=0x7f2f3000dc00 comp rx=0 tx=0).stop 2024-09-21T03:40:07.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:07.335+0000 7f2f45c94640 1 -- 172.21.15.38:0/232981594 shutdown_connections 2024-09-21T03:40:07.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:07.335+0000 7f2f45c94640 1 --2- 172.21.15.38:0/232981594 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2f2c078b90 0x7f2f2c07b0a0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:07.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:07.335+0000 7f2f45c94640 1 --2- 172.21.15.38:0/232981594 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2f40162590 0x7f2f400c9c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:07.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:07.335+0000 7f2f45c94640 1 --2- 172.21.15.38:0/232981594 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2f400cfe00 0x7f2f400c9720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:07.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:07.335+0000 7f2f45c94640 1 --2- 172.21.15.38:0/232981594 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2f400b2e60 0x7f2f40158030 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:07.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:07.335+0000 7f2f45c94640 1 -- 172.21.15.38:0/232981594 >> 172.21.15.38:0/232981594 conn(0x7f2f400d4440 msgr2=0x7f2f400d4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:07.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:07.335+0000 7f2f45c94640 1 -- 172.21.15.38:0/232981594 shutdown_connections 2024-09-21T03:40:07.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:07.335+0000 7f2f45c94640 1 -- 172.21.15.38:0/232981594 wait complete. 2024-09-21T03:40:07.349 INFO:teuthology.orchestra.run.smithi038.stdout:{"pg_ready":true,"pg_map":{"version":289,"stamp":"2024-09-21T03:40:07.088456+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":145,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110,"ondisk_log_size":110,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":118,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":276272,"kb_used_data":10100,"kb_used_omap":12,"kb_used_meta":265971,"kb_avail":754698448,"statfs":{"total":773094113280,"available":772811210752,"internally_reserved":0,"allocated":10342400,"data_stored":5402974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12748,"internal_metadata":272354868},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001757"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-21T03:40:02.927515+0000","last_change":"2024-09-21T03:40:02.922416+0000","last_active":"2024-09-21T03:40:02.927515+0000","last_peered":"2024-09-21T03:40:02.927515+0000","last_clean":"2024-09-21T03:40:02.927515+0000","last_became_active":"2024-09-21T03:40:02.922118+0000","last_became_peered":"2024-09-21T03:40:02.922118+0000","last_unstale":"2024-09-21T03:40:02.927515+0000","last_undegraded":"2024-09-21T03:40:02.927515+0000","last_fullsized":"2024-09-21T03:40:02.927515+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_clean_scrub_stamp":"2024-09-21T03:40:01.878704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:02:41.238627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T03:40:03.911346+0000","last_change":"2024-09-21T03:40:02.918908+0000","last_active":"2024-09-21T03:40:03.911346+0000","last_peered":"2024-09-21T03:40:03.911346+0000","last_clean":"2024-09-21T03:40:03.911346+0000","last_became_active":"2024-09-21T03:40:02.918227+0000","last_became_peered":"2024-09-21T03:40:02.918227+0000","last_unstale":"2024-09-21T03:40:03.911346+0000","last_undegraded":"2024-09-21T03:40:03.911346+0000","last_fullsized":"2024-09-21T03:40:03.911346+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_clean_scrub_stamp":"2024-09-21T03:40:01.878704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:46:27.097203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-21T03:40:02.920916+0000","last_change":"2024-09-21T03:40:02.920038+0000","last_active":"2024-09-21T03:40:02.920916+0000","last_peered":"2024-09-21T03:40:02.920916+0000","last_clean":"2024-09-21T03:40:02.920916+0000","last_became_active":"2024-09-21T03:40:02.919683+0000","last_became_peered":"2024-09-21T03:40:02.919683+0000","last_unstale":"2024-09-21T03:40:02.920916+0000","last_undegraded":"2024-09-21T03:40:02.920916+0000","last_fullsized":"2024-09-21T03:40:02.920916+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_clean_scrub_stamp":"2024-09-21T03:40:01.878704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:56:14.770281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-21T03:40:02.910220+0000","last_change":"2024-09-21T03:40:02.903330+0000","last_active":"2024-09-21T03:40:02.910220+0000","last_peered":"2024-09-21T03:40:02.910220+0000","last_clean":"2024-09-21T03:40:02.910220+0000","last_became_active":"2024-09-21T03:40:02.903023+0000","last_became_peered":"2024-09-21T03:40:02.903023+0000","last_unstale":"2024-09-21T03:40:02.910220+0000","last_undegraded":"2024-09-21T03:40:02.910220+0000","last_fullsized":"2024-09-21T03:40:02.910220+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_clean_scrub_stamp":"2024-09-21T03:40:01.878704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:55:50.721982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T03:40:03.910733+0000","last_change":"2024-09-21T03:40:02.898458+0000","last_active":"2024-09-21T03:40:03.910733+0000","last_peered":"2024-09-21T03:40:03.910733+0000","last_clean":"2024-09-21T03:40:03.910733+0000","last_became_active":"2024-09-21T03:40:02.898121+0000","last_became_peered":"2024-09-21T03:40:02.898121+0000","last_unstale":"2024-09-21T03:40:03.910733+0000","last_undegraded":"2024-09-21T03:40:03.910733+0000","last_fullsized":"2024-09-21T03:40:03.910733+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_clean_scrub_stamp":"2024-09-21T03:40:01.878704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:40:31.254797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-21T03:40:02.925096+0000","last_change":"2024-09-21T03:40:02.924187+0000","last_active":"2024-09-21T03:40:02.925096+0000","last_peered":"2024-09-21T03:40:02.925096+0000","last_clean":"2024-09-21T03:40:02.925096+0000","last_became_active":"2024-09-21T03:40:02.923857+0000","last_became_peered":"2024-09-21T03:40:02.923857+0000","last_unstale":"2024-09-21T03:40:02.925096+0000","last_undegraded":"2024-09-21T03:40:02.925096+0000","last_fullsized":"2024-09-21T03:40:02.925096+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_clean_scrub_stamp":"2024-09-21T03:40:01.878704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:09:01.113089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-21T03:40:02.915449+0000","last_change":"2024-09-21T03:40:02.914279+0000","last_active":"2024-09-21T03:40:02.915449+0000","last_peered":"2024-09-21T03:40:02.915449+0000","last_clean":"2024-09-21T03:40:02.915449+0000","last_became_active":"2024-09-21T03:40:02.914000+0000","last_became_peered":"2024-09-21T03:40:02.914000+0000","last_unstale":"2024-09-21T03:40:02.915449+0000","last_undegraded":"2024-09-21T03:40:02.915449+0000","last_fullsized":"2024-09-21T03:40:02.915449+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_clean_scrub_stamp":"2024-09-21T03:40:01.878704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:01:18.125708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-21T03:40:02.908057+0000","last_change":"2024-09-21T03:40:02.907470+0000","last_active":"2024-09-21T03:40:02.908057+0000","last_peered":"2024-09-21T03:40:02.908057+0000","last_clean":"2024-09-21T03:40:02.908057+0000","last_became_active":"2024-09-21T03:40:02.907125+0000","last_became_peered":"2024-09-21T03:40:02.907125+0000","last_unstale":"2024-09-21T03:40:02.908057+0000","last_undegraded":"2024-09-21T03:40:02.908057+0000","last_fullsized":"2024-09-21T03:40:02.908057+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_clean_scrub_stamp":"2024-09-21T03:40:01.878704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:39:32.775254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-21T03:40:02.906008+0000","last_change":"2024-09-21T03:40:02.900085+0000","last_active":"2024-09-21T03:40:02.906008+0000","last_peered":"2024-09-21T03:40:02.906008+0000","last_clean":"2024-09-21T03:40:02.906008+0000","last_became_active":"2024-09-21T03:40:02.899781+0000","last_became_peered":"2024-09-21T03:40:02.899781+0000","last_unstale":"2024-09-21T03:40:02.906008+0000","last_undegraded":"2024-09-21T03:40:02.906008+0000","last_fullsized":"2024-09-21T03:40:02.906008+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_clean_scrub_stamp":"2024-09-21T03:40:01.878704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:00:13.266042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-21T03:40:02.916329+0000","last_change":"2024-09-21T03:40:02.912456+0000","last_active":"2024-09-21T03:40:02.916329+0000","last_peered":"2024-09-21T03:40:02.916329+0000","last_clean":"2024-09-21T03:40:02.916329+0000","last_became_active":"2024-09-21T03:40:02.912153+0000","last_became_peered":"2024-09-21T03:40:02.912153+0000","last_unstale":"2024-09-21T03:40:02.916329+0000","last_undegraded":"2024-09-21T03:40:02.916329+0000","last_fullsized":"2024-09-21T03:40:02.916329+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_clean_scrub_stamp":"2024-09-21T03:40:01.878704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:16:06.081301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-21T03:40:02.909748+0000","last_change":"2024-09-21T03:40:02.905686+0000","last_active":"2024-09-21T03:40:02.909748+0000","last_peered":"2024-09-21T03:40:02.909748+0000","last_clean":"2024-09-21T03:40:02.909748+0000","last_became_active":"2024-09-21T03:40:02.905353+0000","last_became_peered":"2024-09-21T03:40:02.905353+0000","last_unstale":"2024-09-21T03:40:02.909748+0000","last_undegraded":"2024-09-21T03:40:02.909748+0000","last_fullsized":"2024-09-21T03:40:02.909748+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_clean_scrub_stamp":"2024-09-21T03:40:01.878704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:17:18.448103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-21T03:40:02.916817+0000","last_change":"2024-09-21T03:40:02.914584+0000","last_active":"2024-09-21T03:40:02.916817+0000","last_peered":"2024-09-21T03:40:02.916817+0000","last_clean":"2024-09-21T03:40:02.916817+0000","last_became_active":"2024-09-21T03:40:02.913212+0000","last_became_peered":"2024-09-21T03:40:02.913212+0000","last_unstale":"2024-09-21T03:40:02.916817+0000","last_undegraded":"2024-09-21T03:40:02.916817+0000","last_fullsized":"2024-09-21T03:40:02.916817+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_clean_scrub_stamp":"2024-09-21T03:40:01.878704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:37:43.695022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T03:40:03.909270+0000","last_change":"2024-09-21T03:40:02.922991+0000","last_active":"2024-09-21T03:40:03.909270+0000","last_peered":"2024-09-21T03:40:03.909270+0000","last_clean":"2024-09-21T03:40:03.909270+0000","last_became_active":"2024-09-21T03:40:02.922717+0000","last_became_peered":"2024-09-21T03:40:02.922717+0000","last_unstale":"2024-09-21T03:40:03.909270+0000","last_undegraded":"2024-09-21T03:40:03.909270+0000","last_fullsized":"2024-09-21T03:40:03.909270+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_clean_scrub_stamp":"2024-09-21T03:40:01.878704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:09:51.120222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-21T03:40:02.915738+0000","last_change":"2024-09-21T03:40:02.911800+0000","last_active":"2024-09-21T03:40:02.915738+0000","last_peered":"2024-09-21T03:40:02.915738+0000","last_clean":"2024-09-21T03:40:02.915738+0000","last_became_active":"2024-09-21T03:40:02.911488+0000","last_became_peered":"2024-09-21T03:40:02.911488+0000","last_unstale":"2024-09-21T03:40:02.915738+0000","last_undegraded":"2024-09-21T03:40:02.915738+0000","last_fullsized":"2024-09-21T03:40:02.915738+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_clean_scrub_stamp":"2024-09-21T03:40:01.878704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:22:37.186117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-21T03:40:02.917537+0000","last_change":"2024-09-21T03:40:02.916645+0000","last_active":"2024-09-21T03:40:02.917537+0000","last_peered":"2024-09-21T03:40:02.917537+0000","last_clean":"2024-09-21T03:40:02.917537+0000","last_became_active":"2024-09-21T03:40:02.916319+0000","last_became_peered":"2024-09-21T03:40:02.916319+0000","last_unstale":"2024-09-21T03:40:02.917537+0000","last_undegraded":"2024-09-21T03:40:02.917537+0000","last_fullsized":"2024-09-21T03:40:02.917537+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_clean_scrub_stamp":"2024-09-21T03:40:01.878704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:27:09.793327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-21T03:40:02.920537+0000","last_change":"2024-09-21T03:40:02.919274+0000","last_active":"2024-09-21T03:40:02.920537+0000","last_peered":"2024-09-21T03:40:02.920537+0000","last_clean":"2024-09-21T03:40:02.920537+0000","last_became_active":"2024-09-21T03:40:02.918722+0000","last_became_peered":"2024-09-21T03:40:02.918722+0000","last_unstale":"2024-09-21T03:40:02.920537+0000","last_undegraded":"2024-09-21T03:40:02.920537+0000","last_fullsized":"2024-09-21T03:40:02.920537+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_clean_scrub_stamp":"2024-09-21T03:40:01.878704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:42:13.584645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-21T03:40:02.911825+0000","last_change":"2024-09-21T03:40:02.910611+0000","last_active":"2024-09-21T03:40:02.911825+0000","last_peered":"2024-09-21T03:40:02.911825+0000","last_clean":"2024-09-21T03:40:02.911825+0000","last_became_active":"2024-09-21T03:40:02.910292+0000","last_became_peered":"2024-09-21T03:40:02.910292+0000","last_unstale":"2024-09-21T03:40:02.911825+0000","last_undegraded":"2024-09-21T03:40:02.911825+0000","last_fullsized":"2024-09-21T03:40:02.911825+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_clean_scrub_stamp":"2024-09-21T03:40:01.878704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:45:36.917957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-21T03:40:02.910636+0000","last_change":"2024-09-21T03:40:02.904596+0000","last_active":"2024-09-21T03:40:02.910636+0000","last_peered":"2024-09-21T03:40:02.910636+0000","last_clean":"2024-09-21T03:40:02.910636+0000","last_became_active":"2024-09-21T03:40:02.904299+0000","last_became_peered":"2024-09-21T03:40:02.904299+0000","last_unstale":"2024-09-21T03:40:02.910636+0000","last_undegraded":"2024-09-21T03:40:02.910636+0000","last_fullsized":"2024-09-21T03:40:02.910636+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_clean_scrub_stamp":"2024-09-21T03:40:01.878704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:17:45.648298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-21T03:40:02.916206+0000","last_change":"2024-09-21T03:40:02.911816+0000","last_active":"2024-09-21T03:40:02.916206+0000","last_peered":"2024-09-21T03:40:02.916206+0000","last_clean":"2024-09-21T03:40:02.916206+0000","last_became_active":"2024-09-21T03:40:02.911504+0000","last_became_peered":"2024-09-21T03:40:02.911504+0000","last_unstale":"2024-09-21T03:40:02.916206+0000","last_undegraded":"2024-09-21T03:40:02.916206+0000","last_fullsized":"2024-09-21T03:40:02.916206+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_clean_scrub_stamp":"2024-09-21T03:40:01.878704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:22:48.720231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-21T03:40:02.928221+0000","last_change":"2024-09-21T03:40:02.924813+0000","last_active":"2024-09-21T03:40:02.928221+0000","last_peered":"2024-09-21T03:40:02.928221+0000","last_clean":"2024-09-21T03:40:02.928221+0000","last_became_active":"2024-09-21T03:40:02.924537+0000","last_became_peered":"2024-09-21T03:40:02.924537+0000","last_unstale":"2024-09-21T03:40:02.928221+0000","last_undegraded":"2024-09-21T03:40:02.928221+0000","last_fullsized":"2024-09-21T03:40:02.928221+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_clean_scrub_stamp":"2024-09-21T03:40:01.878704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:12:15.980838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T03:40:03.911087+0000","last_change":"2024-09-21T03:40:02.909856+0000","last_active":"2024-09-21T03:40:03.911087+0000","last_peered":"2024-09-21T03:40:03.911087+0000","last_clean":"2024-09-21T03:40:03.911087+0000","last_became_active":"2024-09-21T03:40:02.908152+0000","last_became_peered":"2024-09-21T03:40:02.908152+0000","last_unstale":"2024-09-21T03:40:03.911087+0000","last_undegraded":"2024-09-21T03:40:03.911087+0000","last_fullsized":"2024-09-21T03:40:03.911087+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_clean_scrub_stamp":"2024-09-21T03:40:01.878704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:36:26.820027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-21T03:40:02.921522+0000","last_change":"2024-09-21T03:40:02.916741+0000","last_active":"2024-09-21T03:40:02.921522+0000","last_peered":"2024-09-21T03:40:02.921522+0000","last_clean":"2024-09-21T03:40:02.921522+0000","last_became_active":"2024-09-21T03:40:02.916442+0000","last_became_peered":"2024-09-21T03:40:02.916442+0000","last_unstale":"2024-09-21T03:40:02.921522+0000","last_undegraded":"2024-09-21T03:40:02.921522+0000","last_fullsized":"2024-09-21T03:40:02.921522+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_clean_scrub_stamp":"2024-09-21T03:40:01.878704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:40:32.479886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T03:40:03.910597+0000","last_change":"2024-09-21T03:40:02.920348+0000","last_active":"2024-09-21T03:40:03.910597+0000","last_peered":"2024-09-21T03:40:03.910597+0000","last_clean":"2024-09-21T03:40:03.910597+0000","last_became_active":"2024-09-21T03:40:02.919968+0000","last_became_peered":"2024-09-21T03:40:02.919968+0000","last_unstale":"2024-09-21T03:40:03.910597+0000","last_undegraded":"2024-09-21T03:40:03.910597+0000","last_fullsized":"2024-09-21T03:40:03.910597+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_clean_scrub_stamp":"2024-09-21T03:40:01.878704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:39:21.091609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-21T03:40:02.927463+0000","last_change":"2024-09-21T03:40:02.923791+0000","last_active":"2024-09-21T03:40:02.927463+0000","last_peered":"2024-09-21T03:40:02.927463+0000","last_clean":"2024-09-21T03:40:02.927463+0000","last_became_active":"2024-09-21T03:40:02.923469+0000","last_became_peered":"2024-09-21T03:40:02.923469+0000","last_unstale":"2024-09-21T03:40:02.927463+0000","last_undegraded":"2024-09-21T03:40:02.927463+0000","last_fullsized":"2024-09-21T03:40:02.927463+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_clean_scrub_stamp":"2024-09-21T03:40:01.878704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:18:39.471497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T03:40:03.911072+0000","last_change":"2024-09-21T03:40:02.907650+0000","last_active":"2024-09-21T03:40:03.911072+0000","last_peered":"2024-09-21T03:40:03.911072+0000","last_clean":"2024-09-21T03:40:03.911072+0000","last_became_active":"2024-09-21T03:40:02.906437+0000","last_became_peered":"2024-09-21T03:40:02.906437+0000","last_unstale":"2024-09-21T03:40:03.911072+0000","last_undegraded":"2024-09-21T03:40:03.911072+0000","last_fullsized":"2024-09-21T03:40:03.911072+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_clean_scrub_stamp":"2024-09-21T03:40:01.878704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:21:26.719708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T03:40:03.911437+0000","last_change":"2024-09-21T03:40:02.919213+0000","last_active":"2024-09-21T03:40:03.911437+0000","last_peered":"2024-09-21T03:40:03.911437+0000","last_clean":"2024-09-21T03:40:03.911437+0000","last_became_active":"2024-09-21T03:40:02.918596+0000","last_became_peered":"2024-09-21T03:40:02.918596+0000","last_unstale":"2024-09-21T03:40:03.911437+0000","last_undegraded":"2024-09-21T03:40:03.911437+0000","last_fullsized":"2024-09-21T03:40:03.911437+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_clean_scrub_stamp":"2024-09-21T03:40:01.878704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:19:29.651722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-21T03:40:02.910299+0000","last_change":"2024-09-21T03:40:02.904727+0000","last_active":"2024-09-21T03:40:02.910299+0000","last_peered":"2024-09-21T03:40:02.910299+0000","last_clean":"2024-09-21T03:40:02.910299+0000","last_became_active":"2024-09-21T03:40:02.904362+0000","last_became_peered":"2024-09-21T03:40:02.904362+0000","last_unstale":"2024-09-21T03:40:02.910299+0000","last_undegraded":"2024-09-21T03:40:02.910299+0000","last_fullsized":"2024-09-21T03:40:02.910299+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_clean_scrub_stamp":"2024-09-21T03:40:01.878704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:13:49.164794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T03:40:03.908497+0000","last_change":"2024-09-21T03:40:02.916725+0000","last_active":"2024-09-21T03:40:03.908497+0000","last_peered":"2024-09-21T03:40:03.908497+0000","last_clean":"2024-09-21T03:40:03.908497+0000","last_became_active":"2024-09-21T03:40:02.916414+0000","last_became_peered":"2024-09-21T03:40:02.916414+0000","last_unstale":"2024-09-21T03:40:03.908497+0000","last_undegraded":"2024-09-21T03:40:03.908497+0000","last_fullsized":"2024-09-21T03:40:03.908497+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_clean_scrub_stamp":"2024-09-21T03:40:01.878704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:41:51.447303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T03:40:03.911138+0000","last_change":"2024-09-21T03:40:02.926297+0000","last_active":"2024-09-21T03:40:03.911138+0000","last_peered":"2024-09-21T03:40:03.911138+0000","last_clean":"2024-09-21T03:40:03.911138+0000","last_became_active":"2024-09-21T03:40:02.925950+0000","last_became_peered":"2024-09-21T03:40:02.925950+0000","last_unstale":"2024-09-21T03:40:03.911138+0000","last_undegraded":"2024-09-21T03:40:03.911138+0000","last_fullsized":"2024-09-21T03:40:03.911138+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_clean_scrub_stamp":"2024-09-21T03:40:01.878704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:06:56.133248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T03:40:03.910624+0000","last_change":"2024-09-21T03:40:02.907384+0000","last_active":"2024-09-21T03:40:03.910624+0000","last_peered":"2024-09-21T03:40:03.910624+0000","last_clean":"2024-09-21T03:40:03.910624+0000","last_became_active":"2024-09-21T03:40:02.907112+0000","last_became_peered":"2024-09-21T03:40:02.907112+0000","last_unstale":"2024-09-21T03:40:03.910624+0000","last_undegraded":"2024-09-21T03:40:03.910624+0000","last_fullsized":"2024-09-21T03:40:03.910624+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_clean_scrub_stamp":"2024-09-21T03:40:01.878704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:47:17.983636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-21T03:40:02.926742+0000","last_change":"2024-09-21T03:40:02.923715+0000","last_active":"2024-09-21T03:40:02.926742+0000","last_peered":"2024-09-21T03:40:02.926742+0000","last_clean":"2024-09-21T03:40:02.926742+0000","last_became_active":"2024-09-21T03:40:02.923375+0000","last_became_peered":"2024-09-21T03:40:02.923375+0000","last_unstale":"2024-09-21T03:40:02.926742+0000","last_undegraded":"2024-09-21T03:40:02.926742+0000","last_fullsized":"2024-09-21T03:40:02.926742+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_clean_scrub_stamp":"2024-09-21T03:40:01.878704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:31:25.195627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-21T03:40:02.906514+0000","last_change":"2024-09-21T03:40:02.904512+0000","last_active":"2024-09-21T03:40:02.906514+0000","last_peered":"2024-09-21T03:40:02.906514+0000","last_clean":"2024-09-21T03:40:02.906514+0000","last_became_active":"2024-09-21T03:40:02.904158+0000","last_became_peered":"2024-09-21T03:40:02.904158+0000","last_unstale":"2024-09-21T03:40:02.906514+0000","last_undegraded":"2024-09-21T03:40:02.906514+0000","last_fullsized":"2024-09-21T03:40:02.906514+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_clean_scrub_stamp":"2024-09-21T03:40:01.878704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:10:44.384273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-21T03:40:02.928015+0000","last_change":"2024-09-21T03:40:02.923176+0000","last_active":"2024-09-21T03:40:02.928015+0000","last_peered":"2024-09-21T03:40:02.928015+0000","last_clean":"2024-09-21T03:40:02.928015+0000","last_became_active":"2024-09-21T03:40:02.922863+0000","last_became_peered":"2024-09-21T03:40:02.922863+0000","last_unstale":"2024-09-21T03:40:02.928015+0000","last_undegraded":"2024-09-21T03:40:02.928015+0000","last_fullsized":"2024-09-21T03:40:02.928015+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_clean_scrub_stamp":"2024-09-21T03:40:01.878704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:40:32.479886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"93'4","reported_seq":19,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T03:40:03.911493+0000","last_change":"2024-09-21T03:40:01.912737+0000","last_active":"2024-09-21T03:40:03.911493+0000","last_peered":"2024-09-21T03:40:03.911493+0000","last_clean":"2024-09-21T03:40:03.911493+0000","last_became_active":"2024-09-21T03:40:01.912428+0000","last_became_peered":"2024-09-21T03:40:01.912428+0000","last_unstale":"2024-09-21T03:40:03.911493+0000","last_undegraded":"2024-09-21T03:40:03.911493+0000","last_fullsized":"2024-09-21T03:40:03.911493+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:00.876417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:00.876417+0000","last_clean_scrub_stamp":"2024-09-21T03:40:00.876417+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:01:26.321596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-21T03:40:02.920474+0000","last_change":"2024-09-21T03:40:02.917342+0000","last_active":"2024-09-21T03:40:02.920474+0000","last_peered":"2024-09-21T03:40:02.920474+0000","last_clean":"2024-09-21T03:40:02.920474+0000","last_became_active":"2024-09-21T03:40:02.916990+0000","last_became_peered":"2024-09-21T03:40:02.916990+0000","last_unstale":"2024-09-21T03:40:02.920474+0000","last_undegraded":"2024-09-21T03:40:02.920474+0000","last_fullsized":"2024-09-21T03:40:02.920474+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_clean_scrub_stamp":"2024-09-21T03:40:01.878704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:22:30.311493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1e","version":"93'1","reported_seq":14,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-21T03:40:02.916928+0000","last_change":"2024-09-21T03:40:01.912954+0000","last_active":"2024-09-21T03:40:02.916928+0000","last_peered":"2024-09-21T03:40:02.916928+0000","last_clean":"2024-09-21T03:40:02.916928+0000","last_became_active":"2024-09-21T03:40:01.912245+0000","last_became_peered":"2024-09-21T03:40:01.912245+0000","last_unstale":"2024-09-21T03:40:02.916928+0000","last_undegraded":"2024-09-21T03:40:02.916928+0000","last_fullsized":"2024-09-21T03:40:02.916928+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:00.876417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:00.876417+0000","last_clean_scrub_stamp":"2024-09-21T03:40:00.876417+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:42:38.676959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-21T03:40:02.915646+0000","last_change":"2024-09-21T03:40:02.914079+0000","last_active":"2024-09-21T03:40:02.915646+0000","last_peered":"2024-09-21T03:40:02.915646+0000","last_clean":"2024-09-21T03:40:02.915646+0000","last_became_active":"2024-09-21T03:40:02.913725+0000","last_became_peered":"2024-09-21T03:40:02.913725+0000","last_unstale":"2024-09-21T03:40:02.915646+0000","last_undegraded":"2024-09-21T03:40:02.915646+0000","last_fullsized":"2024-09-21T03:40:02.915646+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_clean_scrub_stamp":"2024-09-21T03:40:01.878704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:34:33.361204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1d","version":"93'2","reported_seq":17,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T03:40:03.911178+0000","last_change":"2024-09-21T03:40:01.904239+0000","last_active":"2024-09-21T03:40:03.911178+0000","last_peered":"2024-09-21T03:40:03.911178+0000","last_clean":"2024-09-21T03:40:03.911178+0000","last_became_active":"2024-09-21T03:40:01.903799+0000","last_became_peered":"2024-09-21T03:40:01.903799+0000","last_unstale":"2024-09-21T03:40:03.911178+0000","last_undegraded":"2024-09-21T03:40:03.911178+0000","last_fullsized":"2024-09-21T03:40:03.911178+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:00.876417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:00.876417+0000","last_clean_scrub_stamp":"2024-09-21T03:40:00.876417+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:33:53.314541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-21T03:40:02.922069+0000","last_change":"2024-09-21T03:40:02.912850+0000","last_active":"2024-09-21T03:40:02.922069+0000","last_peered":"2024-09-21T03:40:02.922069+0000","last_clean":"2024-09-21T03:40:02.922069+0000","last_became_active":"2024-09-21T03:40:02.912563+0000","last_became_peered":"2024-09-21T03:40:02.912563+0000","last_unstale":"2024-09-21T03:40:02.922069+0000","last_undegraded":"2024-09-21T03:40:02.922069+0000","last_fullsized":"2024-09-21T03:40:02.922069+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_clean_scrub_stamp":"2024-09-21T03:40:01.878704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:56:47.573639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1c","version":"93'2","reported_seq":15,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-21T03:40:02.915090+0000","last_change":"2024-09-21T03:40:01.910064+0000","last_active":"2024-09-21T03:40:02.915090+0000","last_peered":"2024-09-21T03:40:02.915090+0000","last_clean":"2024-09-21T03:40:02.915090+0000","last_became_active":"2024-09-21T03:40:01.909721+0000","last_became_peered":"2024-09-21T03:40:01.909721+0000","last_unstale":"2024-09-21T03:40:02.915090+0000","last_undegraded":"2024-09-21T03:40:02.915090+0000","last_fullsized":"2024-09-21T03:40:02.915090+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:00.876417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:00.876417+0000","last_clean_scrub_stamp":"2024-09-21T03:40:00.876417+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:29:24.100238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T03:40:03.908741+0000","last_change":"2024-09-21T03:40:02.927283+0000","last_active":"2024-09-21T03:40:03.908741+0000","last_peered":"2024-09-21T03:40:03.908741+0000","last_clean":"2024-09-21T03:40:03.908741+0000","last_became_active":"2024-09-21T03:40:02.926982+0000","last_became_peered":"2024-09-21T03:40:02.926982+0000","last_unstale":"2024-09-21T03:40:03.908741+0000","last_undegraded":"2024-09-21T03:40:03.908741+0000","last_fullsized":"2024-09-21T03:40:03.908741+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_clean_scrub_stamp":"2024-09-21T03:40:01.878704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:49:03.507936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.1b","version":"93'1","reported_seq":14,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-21T03:40:02.920027+0000","last_change":"2024-09-21T03:40:01.911561+0000","last_active":"2024-09-21T03:40:02.920027+0000","last_peered":"2024-09-21T03:40:02.920027+0000","last_clean":"2024-09-21T03:40:02.920027+0000","last_became_active":"2024-09-21T03:40:01.911068+0000","last_became_peered":"2024-09-21T03:40:01.911068+0000","last_unstale":"2024-09-21T03:40:02.920027+0000","last_undegraded":"2024-09-21T03:40:02.920027+0000","last_fullsized":"2024-09-21T03:40:02.920027+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:00.876417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:00.876417+0000","last_clean_scrub_stamp":"2024-09-21T03:40:00.876417+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:22:36.183830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-21T03:40:02.909921+0000","last_change":"2024-09-21T03:40:02.908192+0000","last_active":"2024-09-21T03:40:02.909921+0000","last_peered":"2024-09-21T03:40:02.909921+0000","last_clean":"2024-09-21T03:40:02.909921+0000","last_became_active":"2024-09-21T03:40:02.907881+0000","last_became_peered":"2024-09-21T03:40:02.907881+0000","last_unstale":"2024-09-21T03:40:02.909921+0000","last_undegraded":"2024-09-21T03:40:02.909921+0000","last_fullsized":"2024-09-21T03:40:02.909921+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_clean_scrub_stamp":"2024-09-21T03:40:01.878704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:15:36.427378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1a","version":"93'1","reported_seq":16,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T03:40:03.907992+0000","last_change":"2024-09-21T03:40:01.903860+0000","last_active":"2024-09-21T03:40:03.907992+0000","last_peered":"2024-09-21T03:40:03.907992+0000","last_clean":"2024-09-21T03:40:03.907992+0000","last_became_active":"2024-09-21T03:40:01.903486+0000","last_became_peered":"2024-09-21T03:40:01.903486+0000","last_unstale":"2024-09-21T03:40:03.907992+0000","last_undegraded":"2024-09-21T03:40:03.907992+0000","last_fullsized":"2024-09-21T03:40:03.907992+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:00.876417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:00.876417+0000","last_clean_scrub_stamp":"2024-09-21T03:40:00.876417+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:14:16.421060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-21T03:40:02.914054+0000","last_change":"2024-09-21T03:40:02.912360+0000","last_active":"2024-09-21T03:40:02.914054+0000","last_peered":"2024-09-21T03:40:02.914054+0000","last_clean":"2024-09-21T03:40:02.914054+0000","last_became_active":"2024-09-21T03:40:02.912052+0000","last_became_peered":"2024-09-21T03:40:02.912052+0000","last_unstale":"2024-09-21T03:40:02.914054+0000","last_undegraded":"2024-09-21T03:40:02.914054+0000","last_fullsized":"2024-09-21T03:40:02.914054+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_clean_scrub_stamp":"2024-09-21T03:40:01.878704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:21:25.884628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":13,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-21T03:40:02.906096+0000","last_change":"2024-09-21T03:40:01.907992+0000","last_active":"2024-09-21T03:40:02.906096+0000","last_peered":"2024-09-21T03:40:02.906096+0000","last_clean":"2024-09-21T03:40:02.906096+0000","last_became_active":"2024-09-21T03:40:01.907676+0000","last_became_peered":"2024-09-21T03:40:01.907676+0000","last_unstale":"2024-09-21T03:40:02.906096+0000","last_undegraded":"2024-09-21T03:40:02.906096+0000","last_fullsized":"2024-09-21T03:40:02.906096+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:00.876417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:00.876417+0000","last_clean_scrub_stamp":"2024-09-21T03:40:00.876417+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:06:01.990231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-21T03:40:02.929905+0000","last_change":"2024-09-21T03:40:02.928175+0000","last_active":"2024-09-21T03:40:02.929905+0000","last_peered":"2024-09-21T03:40:02.929905+0000","last_clean":"2024-09-21T03:40:02.929905+0000","last_became_active":"2024-09-21T03:40:02.927778+0000","last_became_peered":"2024-09-21T03:40:02.927778+0000","last_unstale":"2024-09-21T03:40:02.929905+0000","last_undegraded":"2024-09-21T03:40:02.929905+0000","last_fullsized":"2024-09-21T03:40:02.929905+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_clean_scrub_stamp":"2024-09-21T03:40:01.878704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:37:57.479481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T03:40:03.910522+0000","last_change":"2024-09-21T03:40:01.896514+0000","last_active":"2024-09-21T03:40:03.910522+0000","last_peered":"2024-09-21T03:40:03.910522+0000","last_clean":"2024-09-21T03:40:03.910522+0000","last_became_active":"2024-09-21T03:40:01.896081+0000","last_became_peered":"2024-09-21T03:40:01.896081+0000","last_unstale":"2024-09-21T03:40:03.910522+0000","last_undegraded":"2024-09-21T03:40:03.910522+0000","last_fullsized":"2024-09-21T03:40:03.910522+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:00.876417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:00.876417+0000","last_clean_scrub_stamp":"2024-09-21T03:40:00.876417+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:41:21.728049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T03:40:03.911532+0000","last_change":"2024-09-21T03:40:02.915883+0000","last_active":"2024-09-21T03:40:03.911532+0000","last_peered":"2024-09-21T03:40:03.911532+0000","last_clean":"2024-09-21T03:40:03.911532+0000","last_became_active":"2024-09-21T03:40:02.915585+0000","last_became_peered":"2024-09-21T03:40:02.915585+0000","last_unstale":"2024-09-21T03:40:03.911532+0000","last_undegraded":"2024-09-21T03:40:03.911532+0000","last_fullsized":"2024-09-21T03:40:03.911532+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_clean_scrub_stamp":"2024-09-21T03:40:01.878704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:16:25.774858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T03:40:03.911546+0000","last_change":"2024-09-21T03:40:01.904264+0000","last_active":"2024-09-21T03:40:03.911546+0000","last_peered":"2024-09-21T03:40:03.911546+0000","last_clean":"2024-09-21T03:40:03.911546+0000","last_became_active":"2024-09-21T03:40:01.903744+0000","last_became_peered":"2024-09-21T03:40:01.903744+0000","last_unstale":"2024-09-21T03:40:03.911546+0000","last_undegraded":"2024-09-21T03:40:03.911546+0000","last_fullsized":"2024-09-21T03:40:03.911546+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:00.876417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:00.876417+0000","last_clean_scrub_stamp":"2024-09-21T03:40:00.876417+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:31:34.629665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-21T03:40:02.906339+0000","last_change":"2024-09-21T03:40:02.899323+0000","last_active":"2024-09-21T03:40:02.906339+0000","last_peered":"2024-09-21T03:40:02.906339+0000","last_clean":"2024-09-21T03:40:02.906339+0000","last_became_active":"2024-09-21T03:40:02.899022+0000","last_became_peered":"2024-09-21T03:40:02.899022+0000","last_unstale":"2024-09-21T03:40:02.906339+0000","last_undegraded":"2024-09-21T03:40:02.906339+0000","last_fullsized":"2024-09-21T03:40:02.906339+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_clean_scrub_stamp":"2024-09-21T03:40:01.878704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:04:00.546812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":13,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-21T03:40:02.897933+0000","last_change":"2024-09-21T03:40:01.905670+0000","last_active":"2024-09-21T03:40:02.897933+0000","last_peered":"2024-09-21T03:40:02.897933+0000","last_clean":"2024-09-21T03:40:02.897933+0000","last_became_active":"2024-09-21T03:40:01.905260+0000","last_became_peered":"2024-09-21T03:40:01.905260+0000","last_unstale":"2024-09-21T03:40:02.897933+0000","last_undegraded":"2024-09-21T03:40:02.897933+0000","last_fullsized":"2024-09-21T03:40:02.897933+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:00.876417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:00.876417+0000","last_clean_scrub_stamp":"2024-09-21T03:40:00.876417+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:20:54.046019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-21T03:40:02.926121+0000","last_change":"2024-09-21T03:40:02.922314+0000","last_active":"2024-09-21T03:40:02.926121+0000","last_peered":"2024-09-21T03:40:02.926121+0000","last_clean":"2024-09-21T03:40:02.926121+0000","last_became_active":"2024-09-21T03:40:02.922018+0000","last_became_peered":"2024-09-21T03:40:02.922018+0000","last_unstale":"2024-09-21T03:40:02.926121+0000","last_undegraded":"2024-09-21T03:40:02.926121+0000","last_fullsized":"2024-09-21T03:40:02.926121+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_clean_scrub_stamp":"2024-09-21T03:40:01.878704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:39:20.895207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"93'3","reported_seq":16,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-21T03:40:02.923589+0000","last_change":"2024-09-21T03:40:01.902013+0000","last_active":"2024-09-21T03:40:02.923589+0000","last_peered":"2024-09-21T03:40:02.923589+0000","last_clean":"2024-09-21T03:40:02.923589+0000","last_became_active":"2024-09-21T03:40:01.901656+0000","last_became_peered":"2024-09-21T03:40:01.901656+0000","last_unstale":"2024-09-21T03:40:02.923589+0000","last_undegraded":"2024-09-21T03:40:02.923589+0000","last_fullsized":"2024-09-21T03:40:02.923589+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:00.876417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:00.876417+0000","last_clean_scrub_stamp":"2024-09-21T03:40:00.876417+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:06:22.432740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-21T03:40:02.920492+0000","last_change":"2024-09-21T03:40:02.914504+0000","last_active":"2024-09-21T03:40:02.920492+0000","last_peered":"2024-09-21T03:40:02.920492+0000","last_clean":"2024-09-21T03:40:02.920492+0000","last_became_active":"2024-09-21T03:40:02.914193+0000","last_became_peered":"2024-09-21T03:40:02.914193+0000","last_unstale":"2024-09-21T03:40:02.920492+0000","last_undegraded":"2024-09-21T03:40:02.920492+0000","last_fullsized":"2024-09-21T03:40:02.920492+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_clean_scrub_stamp":"2024-09-21T03:40:01.878704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:43:42.170185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.14","version":"93'6","reported_seq":19,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-21T03:40:02.930286+0000","last_change":"2024-09-21T03:40:01.907985+0000","last_active":"2024-09-21T03:40:02.930286+0000","last_peered":"2024-09-21T03:40:02.930286+0000","last_clean":"2024-09-21T03:40:02.930286+0000","last_became_active":"2024-09-21T03:40:01.907666+0000","last_became_peered":"2024-09-21T03:40:01.907666+0000","last_unstale":"2024-09-21T03:40:02.930286+0000","last_undegraded":"2024-09-21T03:40:02.930286+0000","last_fullsized":"2024-09-21T03:40:02.930286+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:00.876417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:00.876417+0000","last_clean_scrub_stamp":"2024-09-21T03:40:00.876417+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:33:59.304186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-21T03:40:02.926070+0000","last_change":"2024-09-21T03:40:02.920112+0000","last_active":"2024-09-21T03:40:02.926070+0000","last_peered":"2024-09-21T03:40:02.926070+0000","last_clean":"2024-09-21T03:40:02.926070+0000","last_became_active":"2024-09-21T03:40:02.919797+0000","last_became_peered":"2024-09-21T03:40:02.919797+0000","last_unstale":"2024-09-21T03:40:02.926070+0000","last_undegraded":"2024-09-21T03:40:02.926070+0000","last_fullsized":"2024-09-21T03:40:02.926070+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_clean_scrub_stamp":"2024-09-21T03:40:01.878704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:45:24.118610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.13","version":"93'2","reported_seq":17,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T03:40:03.911214+0000","last_change":"2024-09-21T03:40:01.904443+0000","last_active":"2024-09-21T03:40:03.911214+0000","last_peered":"2024-09-21T03:40:03.911214+0000","last_clean":"2024-09-21T03:40:03.911214+0000","last_became_active":"2024-09-21T03:40:01.903631+0000","last_became_peered":"2024-09-21T03:40:01.903631+0000","last_unstale":"2024-09-21T03:40:03.911214+0000","last_undegraded":"2024-09-21T03:40:03.911214+0000","last_fullsized":"2024-09-21T03:40:03.911214+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:00.876417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:00.876417+0000","last_clean_scrub_stamp":"2024-09-21T03:40:00.876417+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:04:48.156614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T03:40:03.910855+0000","last_change":"2024-09-21T03:40:02.912394+0000","last_active":"2024-09-21T03:40:03.910855+0000","last_peered":"2024-09-21T03:40:03.910855+0000","last_clean":"2024-09-21T03:40:03.910855+0000","last_became_active":"2024-09-21T03:40:02.912063+0000","last_became_peered":"2024-09-21T03:40:02.912063+0000","last_unstale":"2024-09-21T03:40:03.910855+0000","last_undegraded":"2024-09-21T03:40:03.910855+0000","last_fullsized":"2024-09-21T03:40:03.910855+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_clean_scrub_stamp":"2024-09-21T03:40:01.878704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:09:36.512689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":13,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-21T03:40:02.913044+0000","last_change":"2024-09-21T03:40:01.910583+0000","last_active":"2024-09-21T03:40:02.913044+0000","last_peered":"2024-09-21T03:40:02.913044+0000","last_clean":"2024-09-21T03:40:02.913044+0000","last_became_active":"2024-09-21T03:40:01.910276+0000","last_became_peered":"2024-09-21T03:40:01.910276+0000","last_unstale":"2024-09-21T03:40:02.913044+0000","last_undegraded":"2024-09-21T03:40:02.913044+0000","last_fullsized":"2024-09-21T03:40:02.913044+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:00.876417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:00.876417+0000","last_clean_scrub_stamp":"2024-09-21T03:40:00.876417+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:03:39.665337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-21T03:40:02.927760+0000","last_change":"2024-09-21T03:40:02.925807+0000","last_active":"2024-09-21T03:40:02.927760+0000","last_peered":"2024-09-21T03:40:02.927760+0000","last_clean":"2024-09-21T03:40:02.927760+0000","last_became_active":"2024-09-21T03:40:02.925472+0000","last_became_peered":"2024-09-21T03:40:02.925472+0000","last_unstale":"2024-09-21T03:40:02.927760+0000","last_undegraded":"2024-09-21T03:40:02.927760+0000","last_fullsized":"2024-09-21T03:40:02.927760+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_clean_scrub_stamp":"2024-09-21T03:40:01.878704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:31:25.195627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":13,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-21T03:40:02.913473+0000","last_change":"2024-09-21T03:40:01.904974+0000","last_active":"2024-09-21T03:40:02.913473+0000","last_peered":"2024-09-21T03:40:02.913473+0000","last_clean":"2024-09-21T03:40:02.913473+0000","last_became_active":"2024-09-21T03:40:01.904403+0000","last_became_peered":"2024-09-21T03:40:01.904403+0000","last_unstale":"2024-09-21T03:40:02.913473+0000","last_undegraded":"2024-09-21T03:40:02.913473+0000","last_fullsized":"2024-09-21T03:40:02.913473+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:00.876417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:00.876417+0000","last_clean_scrub_stamp":"2024-09-21T03:40:00.876417+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:54:02.728898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T03:40:03.908568+0000","last_change":"2024-09-21T03:40:02.917902+0000","last_active":"2024-09-21T03:40:03.908568+0000","last_peered":"2024-09-21T03:40:03.908568+0000","last_clean":"2024-09-21T03:40:03.908568+0000","last_became_active":"2024-09-21T03:40:02.917549+0000","last_became_peered":"2024-09-21T03:40:02.917549+0000","last_unstale":"2024-09-21T03:40:03.908568+0000","last_undegraded":"2024-09-21T03:40:03.908568+0000","last_fullsized":"2024-09-21T03:40:03.908568+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_clean_scrub_stamp":"2024-09-21T03:40:01.878704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:50:42.336771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.10","version":"93'8","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-21T03:40:02.927016+0000","last_change":"2024-09-21T03:40:01.900585+0000","last_active":"2024-09-21T03:40:02.927016+0000","last_peered":"2024-09-21T03:40:02.927016+0000","last_clean":"2024-09-21T03:40:02.927016+0000","last_became_active":"2024-09-21T03:40:01.900220+0000","last_became_peered":"2024-09-21T03:40:01.900220+0000","last_unstale":"2024-09-21T03:40:02.927016+0000","last_undegraded":"2024-09-21T03:40:02.927016+0000","last_fullsized":"2024-09-21T03:40:02.927016+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:00.876417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:00.876417+0000","last_clean_scrub_stamp":"2024-09-21T03:40:00.876417+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:58:19.443646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T03:40:03.908264+0000","last_change":"2024-09-21T03:40:02.917597+0000","last_active":"2024-09-21T03:40:03.908264+0000","last_peered":"2024-09-21T03:40:03.908264+0000","last_clean":"2024-09-21T03:40:03.908264+0000","last_became_active":"2024-09-21T03:40:02.917282+0000","last_became_peered":"2024-09-21T03:40:02.917282+0000","last_unstale":"2024-09-21T03:40:03.908264+0000","last_undegraded":"2024-09-21T03:40:03.908264+0000","last_fullsized":"2024-09-21T03:40:03.908264+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_clean_scrub_stamp":"2024-09-21T03:40:01.878704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:39:10.597097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.f","version":"93'1","reported_seq":14,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-21T03:40:02.913110+0000","last_change":"2024-09-21T03:40:01.912993+0000","last_active":"2024-09-21T03:40:02.913110+0000","last_peered":"2024-09-21T03:40:02.913110+0000","last_clean":"2024-09-21T03:40:02.913110+0000","last_became_active":"2024-09-21T03:40:01.912652+0000","last_became_peered":"2024-09-21T03:40:01.912652+0000","last_unstale":"2024-09-21T03:40:02.913110+0000","last_undegraded":"2024-09-21T03:40:02.913110+0000","last_fullsized":"2024-09-21T03:40:02.913110+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:00.876417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:00.876417+0000","last_clean_scrub_stamp":"2024-09-21T03:40:00.876417+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:27:14.167603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-21T03:40:02.909960+0000","last_change":"2024-09-21T03:40:02.907423+0000","last_active":"2024-09-21T03:40:02.909960+0000","last_peered":"2024-09-21T03:40:02.909960+0000","last_clean":"2024-09-21T03:40:02.909960+0000","last_became_active":"2024-09-21T03:40:02.907141+0000","last_became_peered":"2024-09-21T03:40:02.907141+0000","last_unstale":"2024-09-21T03:40:02.909960+0000","last_undegraded":"2024-09-21T03:40:02.909960+0000","last_fullsized":"2024-09-21T03:40:02.909960+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_clean_scrub_stamp":"2024-09-21T03:40:01.878704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:14:25.530924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.e","version":"93'2","reported_seq":15,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-21T03:40:02.920840+0000","last_change":"2024-09-21T03:40:01.913277+0000","last_active":"2024-09-21T03:40:02.920840+0000","last_peered":"2024-09-21T03:40:02.920840+0000","last_clean":"2024-09-21T03:40:02.920840+0000","last_became_active":"2024-09-21T03:40:01.912647+0000","last_became_peered":"2024-09-21T03:40:01.912647+0000","last_unstale":"2024-09-21T03:40:02.920840+0000","last_undegraded":"2024-09-21T03:40:02.920840+0000","last_fullsized":"2024-09-21T03:40:02.920840+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:00.876417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:00.876417+0000","last_clean_scrub_stamp":"2024-09-21T03:40:00.876417+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:54:02.728898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T03:40:03.911235+0000","last_change":"2024-09-21T03:40:02.919564+0000","last_active":"2024-09-21T03:40:03.911235+0000","last_peered":"2024-09-21T03:40:03.911235+0000","last_clean":"2024-09-21T03:40:03.911235+0000","last_became_active":"2024-09-21T03:40:02.917147+0000","last_became_peered":"2024-09-21T03:40:02.917147+0000","last_unstale":"2024-09-21T03:40:03.911235+0000","last_undegraded":"2024-09-21T03:40:03.911235+0000","last_fullsized":"2024-09-21T03:40:03.911235+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_clean_scrub_stamp":"2024-09-21T03:40:01.878704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:07:04.991136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.d","version":"93'1","reported_seq":14,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-21T03:40:02.922147+0000","last_change":"2024-09-21T03:40:01.906570+0000","last_active":"2024-09-21T03:40:02.922147+0000","last_peered":"2024-09-21T03:40:02.922147+0000","last_clean":"2024-09-21T03:40:02.922147+0000","last_became_active":"2024-09-21T03:40:01.906254+0000","last_became_peered":"2024-09-21T03:40:01.906254+0000","last_unstale":"2024-09-21T03:40:02.922147+0000","last_undegraded":"2024-09-21T03:40:02.922147+0000","last_fullsized":"2024-09-21T03:40:02.922147+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:00.876417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:00.876417+0000","last_clean_scrub_stamp":"2024-09-21T03:40:00.876417+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:03:39.665337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-21T03:40:02.918625+0000","last_change":"2024-09-21T03:40:02.912020+0000","last_active":"2024-09-21T03:40:02.918625+0000","last_peered":"2024-09-21T03:40:02.918625+0000","last_clean":"2024-09-21T03:40:02.918625+0000","last_became_active":"2024-09-21T03:40:02.911699+0000","last_became_peered":"2024-09-21T03:40:02.911699+0000","last_unstale":"2024-09-21T03:40:02.918625+0000","last_undegraded":"2024-09-21T03:40:02.918625+0000","last_fullsized":"2024-09-21T03:40:02.918625+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_clean_scrub_stamp":"2024-09-21T03:40:01.878704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:27:20.703323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.b","version":"93'2","reported_seq":15,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-21T03:40:02.926632+0000","last_change":"2024-09-21T03:40:01.899140+0000","last_active":"2024-09-21T03:40:02.926632+0000","last_peered":"2024-09-21T03:40:02.926632+0000","last_clean":"2024-09-21T03:40:02.926632+0000","last_became_active":"2024-09-21T03:40:01.898802+0000","last_became_peered":"2024-09-21T03:40:01.898802+0000","last_unstale":"2024-09-21T03:40:02.926632+0000","last_undegraded":"2024-09-21T03:40:02.926632+0000","last_fullsized":"2024-09-21T03:40:02.926632+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:00.876417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:00.876417+0000","last_clean_scrub_stamp":"2024-09-21T03:40:00.876417+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:31:06.035346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-21T03:40:02.916495+0000","last_change":"2024-09-21T03:40:02.910834+0000","last_active":"2024-09-21T03:40:02.916495+0000","last_peered":"2024-09-21T03:40:02.916495+0000","last_clean":"2024-09-21T03:40:02.916495+0000","last_became_active":"2024-09-21T03:40:02.910535+0000","last_became_peered":"2024-09-21T03:40:02.910535+0000","last_unstale":"2024-09-21T03:40:02.916495+0000","last_undegraded":"2024-09-21T03:40:02.916495+0000","last_fullsized":"2024-09-21T03:40:02.916495+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_clean_scrub_stamp":"2024-09-21T03:40:01.878704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:34:00.306473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":13,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-21T03:40:02.897708+0000","last_change":"2024-09-21T03:40:01.907352+0000","last_active":"2024-09-21T03:40:02.897708+0000","last_peered":"2024-09-21T03:40:02.897708+0000","last_clean":"2024-09-21T03:40:02.897708+0000","last_became_active":"2024-09-21T03:40:01.906943+0000","last_became_peered":"2024-09-21T03:40:01.906943+0000","last_unstale":"2024-09-21T03:40:02.897708+0000","last_undegraded":"2024-09-21T03:40:02.897708+0000","last_fullsized":"2024-09-21T03:40:02.897708+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:00.876417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:00.876417+0000","last_clean_scrub_stamp":"2024-09-21T03:40:00.876417+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:09:44.594955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-21T03:40:02.916083+0000","last_change":"2024-09-21T03:40:02.911571+0000","last_active":"2024-09-21T03:40:02.916083+0000","last_peered":"2024-09-21T03:40:02.916083+0000","last_clean":"2024-09-21T03:40:02.916083+0000","last_became_active":"2024-09-21T03:40:02.911066+0000","last_became_peered":"2024-09-21T03:40:02.911066+0000","last_unstale":"2024-09-21T03:40:02.916083+0000","last_undegraded":"2024-09-21T03:40:02.916083+0000","last_fullsized":"2024-09-21T03:40:02.916083+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_clean_scrub_stamp":"2024-09-21T03:40:01.878704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:39:20.895207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":13,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-21T03:40:02.891635+0000","last_change":"2024-09-21T03:40:01.905588+0000","last_active":"2024-09-21T03:40:02.891635+0000","last_peered":"2024-09-21T03:40:02.891635+0000","last_clean":"2024-09-21T03:40:02.891635+0000","last_became_active":"2024-09-21T03:40:01.905258+0000","last_became_peered":"2024-09-21T03:40:01.905258+0000","last_unstale":"2024-09-21T03:40:02.891635+0000","last_undegraded":"2024-09-21T03:40:02.891635+0000","last_fullsized":"2024-09-21T03:40:02.891635+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:00.876417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:00.876417+0000","last_clean_scrub_stamp":"2024-09-21T03:40:00.876417+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:05:08.585778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-21T03:40:02.926253+0000","last_change":"2024-09-21T03:40:02.922401+0000","last_active":"2024-09-21T03:40:02.926253+0000","last_peered":"2024-09-21T03:40:02.926253+0000","last_clean":"2024-09-21T03:40:02.926253+0000","last_became_active":"2024-09-21T03:40:02.922131+0000","last_became_peered":"2024-09-21T03:40:02.922131+0000","last_unstale":"2024-09-21T03:40:02.926253+0000","last_undegraded":"2024-09-21T03:40:02.926253+0000","last_fullsized":"2024-09-21T03:40:02.926253+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_clean_scrub_stamp":"2024-09-21T03:40:01.878704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:22:48.720231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T03:40:03.908922+0000","last_change":"2024-09-21T03:40:01.908592+0000","last_active":"2024-09-21T03:40:03.908922+0000","last_peered":"2024-09-21T03:40:03.908922+0000","last_clean":"2024-09-21T03:40:03.908922+0000","last_became_active":"2024-09-21T03:40:01.908171+0000","last_became_peered":"2024-09-21T03:40:01.908171+0000","last_unstale":"2024-09-21T03:40:03.908922+0000","last_undegraded":"2024-09-21T03:40:03.908922+0000","last_fullsized":"2024-09-21T03:40:03.908922+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:00.876417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:00.876417+0000","last_clean_scrub_stamp":"2024-09-21T03:40:00.876417+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:35:38.075633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-21T03:40:02.921431+0000","last_change":"2024-09-21T03:40:02.917952+0000","last_active":"2024-09-21T03:40:02.921431+0000","last_peered":"2024-09-21T03:40:02.921431+0000","last_clean":"2024-09-21T03:40:02.921431+0000","last_became_active":"2024-09-21T03:40:02.917563+0000","last_became_peered":"2024-09-21T03:40:02.917563+0000","last_unstale":"2024-09-21T03:40:02.921431+0000","last_undegraded":"2024-09-21T03:40:02.921431+0000","last_fullsized":"2024-09-21T03:40:02.921431+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_clean_scrub_stamp":"2024-09-21T03:40:01.878704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:18:39.471497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.7","version":"93'2","reported_seq":15,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-21T03:40:02.906206+0000","last_change":"2024-09-21T03:40:01.910590+0000","last_active":"2024-09-21T03:40:02.906206+0000","last_peered":"2024-09-21T03:40:02.906206+0000","last_clean":"2024-09-21T03:40:02.906206+0000","last_became_active":"2024-09-21T03:40:01.910244+0000","last_became_peered":"2024-09-21T03:40:01.910244+0000","last_unstale":"2024-09-21T03:40:02.906206+0000","last_undegraded":"2024-09-21T03:40:02.906206+0000","last_fullsized":"2024-09-21T03:40:02.906206+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:00.876417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:00.876417+0000","last_clean_scrub_stamp":"2024-09-21T03:40:00.876417+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:06:19.242099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T03:40:03.907934+0000","last_change":"2024-09-21T03:40:02.914565+0000","last_active":"2024-09-21T03:40:03.907934+0000","last_peered":"2024-09-21T03:40:03.907934+0000","last_clean":"2024-09-21T03:40:03.907934+0000","last_became_active":"2024-09-21T03:40:02.914231+0000","last_became_peered":"2024-09-21T03:40:02.914231+0000","last_unstale":"2024-09-21T03:40:03.907934+0000","last_undegraded":"2024-09-21T03:40:03.907934+0000","last_fullsized":"2024-09-21T03:40:03.907934+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_clean_scrub_stamp":"2024-09-21T03:40:01.878704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:57:58.819881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":13,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-21T03:40:02.892098+0000","last_change":"2024-09-21T03:40:01.901781+0000","last_active":"2024-09-21T03:40:02.892098+0000","last_peered":"2024-09-21T03:40:02.892098+0000","last_clean":"2024-09-21T03:40:02.892098+0000","last_became_active":"2024-09-21T03:40:01.901440+0000","last_became_peered":"2024-09-21T03:40:01.901440+0000","last_unstale":"2024-09-21T03:40:02.892098+0000","last_undegraded":"2024-09-21T03:40:02.892098+0000","last_fullsized":"2024-09-21T03:40:02.892098+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:00.876417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:00.876417+0000","last_clean_scrub_stamp":"2024-09-21T03:40:00.876417+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:21:08.007365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T03:40:03.908429+0000","last_change":"2024-09-21T03:40:02.914459+0000","last_active":"2024-09-21T03:40:03.908429+0000","last_peered":"2024-09-21T03:40:03.908429+0000","last_clean":"2024-09-21T03:40:03.908429+0000","last_became_active":"2024-09-21T03:40:02.914130+0000","last_became_peered":"2024-09-21T03:40:02.914130+0000","last_unstale":"2024-09-21T03:40:03.908429+0000","last_undegraded":"2024-09-21T03:40:03.908429+0000","last_fullsized":"2024-09-21T03:40:03.908429+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_clean_scrub_stamp":"2024-09-21T03:40:01.878704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:22:02.746181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":13,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-21T03:40:02.901203+0000","last_change":"2024-09-21T03:40:01.910362+0000","last_active":"2024-09-21T03:40:02.901203+0000","last_peered":"2024-09-21T03:40:02.901203+0000","last_clean":"2024-09-21T03:40:02.901203+0000","last_became_active":"2024-09-21T03:40:01.910042+0000","last_became_peered":"2024-09-21T03:40:01.910042+0000","last_unstale":"2024-09-21T03:40:02.901203+0000","last_undegraded":"2024-09-21T03:40:02.901203+0000","last_fullsized":"2024-09-21T03:40:02.901203+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:00.876417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:00.876417+0000","last_clean_scrub_stamp":"2024-09-21T03:40:00.876417+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:45:23.116322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T03:40:03.911380+0000","last_change":"2024-09-21T03:40:02.912299+0000","last_active":"2024-09-21T03:40:03.911380+0000","last_peered":"2024-09-21T03:40:03.911380+0000","last_clean":"2024-09-21T03:40:03.911380+0000","last_became_active":"2024-09-21T03:40:02.911984+0000","last_became_peered":"2024-09-21T03:40:02.911984+0000","last_unstale":"2024-09-21T03:40:03.911380+0000","last_undegraded":"2024-09-21T03:40:03.911380+0000","last_fullsized":"2024-09-21T03:40:03.911380+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_clean_scrub_stamp":"2024-09-21T03:40:01.878704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:49:47.824284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":13,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-21T03:40:02.892586+0000","last_change":"2024-09-21T03:40:01.907327+0000","last_active":"2024-09-21T03:40:02.892586+0000","last_peered":"2024-09-21T03:40:02.892586+0000","last_clean":"2024-09-21T03:40:02.892586+0000","last_became_active":"2024-09-21T03:40:01.906998+0000","last_became_peered":"2024-09-21T03:40:01.906998+0000","last_unstale":"2024-09-21T03:40:02.892586+0000","last_undegraded":"2024-09-21T03:40:02.892586+0000","last_fullsized":"2024-09-21T03:40:02.892586+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:00.876417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:00.876417+0000","last_clean_scrub_stamp":"2024-09-21T03:40:00.876417+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:16:30.469693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-21T03:40:02.918576+0000","last_change":"2024-09-21T03:40:02.911748+0000","last_active":"2024-09-21T03:40:02.918576+0000","last_peered":"2024-09-21T03:40:02.918576+0000","last_clean":"2024-09-21T03:40:02.918576+0000","last_became_active":"2024-09-21T03:40:02.911352+0000","last_became_peered":"2024-09-21T03:40:02.911352+0000","last_unstale":"2024-09-21T03:40:02.918576+0000","last_undegraded":"2024-09-21T03:40:02.918576+0000","last_fullsized":"2024-09-21T03:40:02.918576+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_clean_scrub_stamp":"2024-09-21T03:40:01.878704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:34:02.996149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":13,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-21T03:40:02.917104+0000","last_change":"2024-09-21T03:40:01.912608+0000","last_active":"2024-09-21T03:40:02.917104+0000","last_peered":"2024-09-21T03:40:02.917104+0000","last_clean":"2024-09-21T03:40:02.917104+0000","last_became_active":"2024-09-21T03:40:01.912288+0000","last_became_peered":"2024-09-21T03:40:01.912288+0000","last_unstale":"2024-09-21T03:40:02.917104+0000","last_undegraded":"2024-09-21T03:40:02.917104+0000","last_fullsized":"2024-09-21T03:40:02.917104+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:00.876417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:00.876417+0000","last_clean_scrub_stamp":"2024-09-21T03:40:00.876417+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:27:14.167603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-21T03:40:02.926341+0000","last_change":"2024-09-21T03:40:02.924595+0000","last_active":"2024-09-21T03:40:02.926341+0000","last_peered":"2024-09-21T03:40:02.926341+0000","last_clean":"2024-09-21T03:40:02.926341+0000","last_became_active":"2024-09-21T03:40:02.924257+0000","last_became_peered":"2024-09-21T03:40:02.924257+0000","last_unstale":"2024-09-21T03:40:02.926341+0000","last_undegraded":"2024-09-21T03:40:02.926341+0000","last_fullsized":"2024-09-21T03:40:02.926341+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_clean_scrub_stamp":"2024-09-21T03:40:01.878704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:33:26.902361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T03:40:03.910623+0000","last_change":"2024-09-21T03:40:01.912018+0000","last_active":"2024-09-21T03:40:03.910623+0000","last_peered":"2024-09-21T03:40:03.910623+0000","last_clean":"2024-09-21T03:40:03.910623+0000","last_became_active":"2024-09-21T03:40:01.911663+0000","last_became_peered":"2024-09-21T03:40:01.911663+0000","last_unstale":"2024-09-21T03:40:03.910623+0000","last_undegraded":"2024-09-21T03:40:03.910623+0000","last_fullsized":"2024-09-21T03:40:03.910623+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:00.876417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:00.876417+0000","last_clean_scrub_stamp":"2024-09-21T03:40:00.876417+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:47:22.885563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-21T03:40:02.921481+0000","last_change":"2024-09-21T03:40:02.915873+0000","last_active":"2024-09-21T03:40:02.921481+0000","last_peered":"2024-09-21T03:40:02.921481+0000","last_clean":"2024-09-21T03:40:02.921481+0000","last_became_active":"2024-09-21T03:40:02.915180+0000","last_became_peered":"2024-09-21T03:40:02.915180+0000","last_unstale":"2024-09-21T03:40:02.921481+0000","last_undegraded":"2024-09-21T03:40:02.921481+0000","last_fullsized":"2024-09-21T03:40:02.921481+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_clean_scrub_stamp":"2024-09-21T03:40:01.878704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:02:41.238627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T03:40:03.910722+0000","last_change":"2024-09-21T03:40:01.904427+0000","last_active":"2024-09-21T03:40:03.910722+0000","last_peered":"2024-09-21T03:40:03.910722+0000","last_clean":"2024-09-21T03:40:03.910722+0000","last_became_active":"2024-09-21T03:40:01.904062+0000","last_became_peered":"2024-09-21T03:40:01.904062+0000","last_unstale":"2024-09-21T03:40:03.910722+0000","last_undegraded":"2024-09-21T03:40:03.910722+0000","last_fullsized":"2024-09-21T03:40:03.910722+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:00.876417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:00.876417+0000","last_clean_scrub_stamp":"2024-09-21T03:40:00.876417+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:29:20.149004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-21T03:40:02.929753+0000","last_change":"2024-09-21T03:40:02.928726+0000","last_active":"2024-09-21T03:40:02.929753+0000","last_peered":"2024-09-21T03:40:02.929753+0000","last_clean":"2024-09-21T03:40:02.929753+0000","last_became_active":"2024-09-21T03:40:02.928441+0000","last_became_peered":"2024-09-21T03:40:02.928441+0000","last_unstale":"2024-09-21T03:40:02.929753+0000","last_undegraded":"2024-09-21T03:40:02.929753+0000","last_fullsized":"2024-09-21T03:40:02.929753+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_clean_scrub_stamp":"2024-09-21T03:40:01.878704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:36:09.296049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T03:40:03.910625+0000","last_change":"2024-09-21T03:40:01.914749+0000","last_active":"2024-09-21T03:40:03.910625+0000","last_peered":"2024-09-21T03:40:03.910625+0000","last_clean":"2024-09-21T03:40:03.910625+0000","last_became_active":"2024-09-21T03:40:01.913928+0000","last_became_peered":"2024-09-21T03:40:01.913928+0000","last_unstale":"2024-09-21T03:40:03.910625+0000","last_undegraded":"2024-09-21T03:40:03.910625+0000","last_fullsized":"2024-09-21T03:40:03.910625+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:00.876417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:00.876417+0000","last_clean_scrub_stamp":"2024-09-21T03:40:00.876417+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:50:43.304717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T03:40:03.911255+0000","last_change":"2024-09-21T03:40:02.917652+0000","last_active":"2024-09-21T03:40:03.911255+0000","last_peered":"2024-09-21T03:40:03.911255+0000","last_clean":"2024-09-21T03:40:03.911255+0000","last_became_active":"2024-09-21T03:40:02.917354+0000","last_became_peered":"2024-09-21T03:40:02.917354+0000","last_unstale":"2024-09-21T03:40:03.911255+0000","last_undegraded":"2024-09-21T03:40:03.911255+0000","last_fullsized":"2024-09-21T03:40:03.911255+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:01.878704+0000","last_clean_scrub_stamp":"2024-09-21T03:40:01.878704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:59:43.575963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":13,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-21T03:40:02.919154+0000","last_change":"2024-09-21T03:40:01.907963+0000","last_active":"2024-09-21T03:40:02.919154+0000","last_peered":"2024-09-21T03:40:02.919154+0000","last_clean":"2024-09-21T03:40:02.919154+0000","last_became_active":"2024-09-21T03:40:01.907626+0000","last_became_peered":"2024-09-21T03:40:01.907626+0000","last_unstale":"2024-09-21T03:40:02.919154+0000","last_undegraded":"2024-09-21T03:40:02.919154+0000","last_fullsized":"2024-09-21T03:40:02.919154+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:40:00.876417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:40:00.876417+0000","last_clean_scrub_stamp":"2024-09-21T03:40:00.876417+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:42:33.254595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":250,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T03:40:03.910691+0000","last_change":"2024-09-21T03:33:25.319776+0000","last_active":"2024-09-21T03:40:03.910691+0000","last_peered":"2024-09-21T03:40:03.910691+0000","last_clean":"2024-09-21T03:40:03.910691+0000","last_became_active":"2024-09-21T03:33:25.318399+0000","last_became_peered":"2024-09-21T03:33:25.318399+0000","last_unstale":"2024-09-21T03:40:03.910691+0000","last_undegraded":"2024-09-21T03:40:03.910691+0000","last_fullsized":"2024-09-21T03:40:03.910691+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:23.016672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:23.016672+0000","last_clean_scrub_stamp":"2024-09-21T03:33:23.016672+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:32:35.906114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":1,"up_from":16,"seq":68719476820,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32904,"kb_used_data":1716,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338936,"statfs":{"total":96636764160,"available":96603070464,"internally_reserved":0,"allocated":1757184,"data_stored":1125259,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509524,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36468,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335372,"statfs":{"total":96636764160,"available":96599420928,"internally_reserved":0,"allocated":1138688,"data_stored":527513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":36174283},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509523,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32320,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339520,"statfs":{"total":96636764160,"available":96603668480,"internally_reserved":0,"allocated":1159168,"data_stored":529237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509524,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32308,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339532,"statfs":{"total":96636764160,"available":96603680768,"internally_reserved":0,"allocated":1146880,"data_stored":528117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509523,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32304,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339536,"statfs":{"total":96636764160,"available":96603684864,"internally_reserved":0,"allocated":1142784,"data_stored":527583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542227,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36456,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335384,"statfs":{"total":96636764160,"available":96599433216,"internally_reserved":0,"allocated":1126400,"data_stored":520346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574932,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37060,"kb_used_data":1704,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334780,"statfs":{"total":96636764160,"available":96598814720,"internally_reserved":0,"allocated":1744896,"data_stored":1124573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542228,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36452,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335388,"statfs":{"total":96636764160,"available":96599437312,"internally_reserved":0,"allocated":1126400,"data_stored":520346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T03:40:07.353 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-21T03:40:07.353 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-21T03:40:07.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:07.511+0000 7f17fe382640 1 -- 172.21.15.38:0/269406844 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f17f80b3fc0 msgr2=0x7f17f80b43d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:07.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:07.511+0000 7f17fe382640 1 --2- 172.21.15.38:0/269406844 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f17f80b3fc0 0x7f17f80b43d0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f17ec009830 tx=0x7f17ec02a500 comp rx=0 tx=0).stop 2024-09-21T03:40:07.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:07.511+0000 7f17fe382640 1 -- 172.21.15.38:0/269406844 shutdown_connections 2024-09-21T03:40:07.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:07.511+0000 7f17fe382640 1 --2- 172.21.15.38:0/269406844 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f17f8073d50 0x7f17f8074160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:07.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:07.511+0000 7f17fe382640 1 --2- 172.21.15.38:0/269406844 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f17f80b3fc0 0x7f17f80b43d0 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:07.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:07.511+0000 7f17fe382640 1 -- 172.21.15.38:0/269406844 >> 172.21.15.38:0/269406844 conn(0x7f17f80b2cc0 msgr2=0x7f17f80b30d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:07.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:07.512+0000 7f17fe382640 1 -- 172.21.15.38:0/269406844 shutdown_connections 2024-09-21T03:40:07.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:07.512+0000 7f17fe382640 1 -- 172.21.15.38:0/269406844 wait complete. 2024-09-21T03:40:07.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:07.513+0000 7f17fe382640 1 Processor -- start 2024-09-21T03:40:07.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:07.514+0000 7f17fe382640 1 -- start start 2024-09-21T03:40:07.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:07.515+0000 7f17fe382640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f17f8073d50 0x7f17f81576f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:07.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:07.515+0000 7f17fe382640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f17f80b3fc0 0x7f17f80cc0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:07.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:07.515+0000 7f17fe382640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f17f80d0b40 0x7f17f806cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:07.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:07.515+0000 7f17fe382640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f17f815b750 con 0x7f17f8073d50 2024-09-21T03:40:07.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:07.515+0000 7f17fe382640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f17f815b5d0 con 0x7f17f80b3fc0 2024-09-21T03:40:07.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:07.515+0000 7f17fe382640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f17f815b8d0 con 0x7f17f80d0b40 2024-09-21T03:40:07.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:07.515+0000 7f17f7fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f17f8073d50 0x7f17f81576f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:07.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:07.515+0000 7f17f7fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f17f8073d50 0x7f17f81576f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:55628/0 (socket says 172.21.15.38:55628) 2024-09-21T03:40:07.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:07.515+0000 7f17f7fff640 1 -- 172.21.15.38:0/3400953846 learned_addr learned my addr 172.21.15.38:0/3400953846 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:07.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:07.515+0000 7f17f77fe640 1 --2- 172.21.15.38:0/3400953846 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f17f80b3fc0 0x7f17f80cc0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:07.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:07.515+0000 7f17fc91f640 1 --2- 172.21.15.38:0/3400953846 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f17f80d0b40 0x7f17f806cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:07.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:07.515+0000 7f17f7fff640 1 -- 172.21.15.38:0/3400953846 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f17f80d0b40 msgr2=0x7f17f806cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:07.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:07.515+0000 7f17f7fff640 1 --2- 172.21.15.38:0/3400953846 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f17f80d0b40 0x7f17f806cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:07.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:07.515+0000 7f17f7fff640 1 -- 172.21.15.38:0/3400953846 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f17f80b3fc0 msgr2=0x7f17f80cc0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:07.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:07.515+0000 7f17f7fff640 1 --2- 172.21.15.38:0/3400953846 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f17f80b3fc0 0x7f17f80cc0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:07.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:07.515+0000 7f17f7fff640 1 -- 172.21.15.38:0/3400953846 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17f8157c30 con 0x7f17f8073d50 2024-09-21T03:40:07.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:07.515+0000 7f17f7fff640 1 --2- 172.21.15.38:0/3400953846 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f17f8073d50 0x7f17f81576f0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f17e800c3a0 tx=0x7f17e800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:07.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:07.516+0000 7f17f4ff9640 1 -- 172.21.15.38:0/3400953846 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17e8011790 con 0x7f17f8073d50 2024-09-21T03:40:07.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:07.516+0000 7f17f4ff9640 1 -- 172.21.15.38:0/3400953846 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f17e8017070 con 0x7f17f8073d50 2024-09-21T03:40:07.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:07.516+0000 7f17fe382640 1 -- 172.21.15.38:0/3400953846 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17f81f8b70 con 0x7f17f8073d50 2024-09-21T03:40:07.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:07.516+0000 7f17f4ff9640 1 -- 172.21.15.38:0/3400953846 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17e8011d20 con 0x7f17f8073d50 2024-09-21T03:40:07.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:07.516+0000 7f17fe382640 1 -- 172.21.15.38:0/3400953846 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17f81f8e10 con 0x7f17f8073d50 2024-09-21T03:40:07.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:07.517+0000 7f17fe382640 1 -- 172.21.15.38:0/3400953846 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17f815ea40 con 0x7f17f8073d50 2024-09-21T03:40:07.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:07.518+0000 7f17f4ff9640 1 -- 172.21.15.38:0/3400953846 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f17e801c460 con 0x7f17f8073d50 2024-09-21T03:40:07.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:07.518+0000 7f17f4ff9640 1 --2- 172.21.15.38:0/3400953846 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f17e4078bc0 0x7f17e407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:07.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:07.518+0000 7f17f4ff9640 1 -- 172.21.15.38:0/3400953846 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6101+0+0 (secure 0 0 0) 0x7f17e809ac90 con 0x7f17f8073d50 2024-09-21T03:40:07.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:07.518+0000 7f17f77fe640 1 --2- 172.21.15.38:0/3400953846 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f17e4078bc0 0x7f17e407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:07.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:07.519+0000 7f17f77fe640 1 --2- 172.21.15.38:0/3400953846 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f17e4078bc0 0x7f17e407b0d0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f17ec034000 tx=0x7f17ec004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:07.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:07.523+0000 7f17f4ff9640 1 -- 172.21.15.38:0/3400953846 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f17f815ea40 con 0x7f17f8073d50 2024-09-21T03:40:07.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:07.782+0000 7f17fe382640 1 -- 172.21.15.38:0/3400953846 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f17f815bd80 con 0x7f17f8073d50 2024-09-21T03:40:07.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:07.786+0000 7f17f4ff9640 1 -- 172.21.15.38:0/3400953846 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v22) ==== 190+0+147 (secure 0 0 0) 0x7f17f8157c30 con 0x7f17f8073d50 2024-09-21T03:40:07.789 INFO:teuthology.orchestra.run.smithi038.stdout:[client.0] 2024-09-21T03:40:07.789 INFO:teuthology.orchestra.run.smithi038.stdout: key = AQBQPu5m6ydBDxAAlWZ2XzQTPBcYPriPGRJVeA== 2024-09-21T03:40:07.789 INFO:teuthology.orchestra.run.smithi038.stdout: caps mds = "allow" 2024-09-21T03:40:07.789 INFO:teuthology.orchestra.run.smithi038.stdout: caps mon = "allow r" 2024-09-21T03:40:07.790 INFO:teuthology.orchestra.run.smithi038.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-21T03:40:07.790 INFO:teuthology.orchestra.run.smithi038.stderr:updated caps for client.0 2024-09-21T03:40:07.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:07.788+0000 7f17fe382640 1 -- 172.21.15.38:0/3400953846 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f17e4078bc0 msgr2=0x7f17e407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:07.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:07.788+0000 7f17fe382640 1 --2- 172.21.15.38:0/3400953846 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f17e4078bc0 0x7f17e407b0d0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f17ec034000 tx=0x7f17ec004730 comp rx=0 tx=0).stop 2024-09-21T03:40:07.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:07.788+0000 7f17fe382640 1 -- 172.21.15.38:0/3400953846 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f17f8073d50 msgr2=0x7f17f81576f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:07.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:07.788+0000 7f17fe382640 1 --2- 172.21.15.38:0/3400953846 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f17f8073d50 0x7f17f81576f0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f17e800c3a0 tx=0x7f17e800c860 comp rx=0 tx=0).stop 2024-09-21T03:40:07.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:07.788+0000 7f17fe382640 1 -- 172.21.15.38:0/3400953846 shutdown_connections 2024-09-21T03:40:07.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:07.788+0000 7f17fe382640 1 --2- 172.21.15.38:0/3400953846 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f17e4078bc0 0x7f17e407b0d0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:07.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:07.788+0000 7f17fe382640 1 --2- 172.21.15.38:0/3400953846 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f17f80d0b40 0x7f17f806cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:07.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:07.788+0000 7f17fe382640 1 --2- 172.21.15.38:0/3400953846 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f17f80b3fc0 0x7f17f80cc0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:07.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:07.788+0000 7f17fe382640 1 --2- 172.21.15.38:0/3400953846 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f17f8073d50 0x7f17f81576f0 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:07.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:07.788+0000 7f17fe382640 1 -- 172.21.15.38:0/3400953846 >> 172.21.15.38:0/3400953846 conn(0x7f17f80b2cc0 msgr2=0x7f17f81540c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:07.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:07.789+0000 7f17fe382640 1 -- 172.21.15.38:0/3400953846 shutdown_connections 2024-09-21T03:40:07.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:07.789+0000 7f17fe382640 1 -- 172.21.15.38:0/3400953846 wait complete. 2024-09-21T03:40:07.803 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:40:07.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:07.917+0000 7fac4818a640 1 -- 172.21.15.38:0/3469242090 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fac2c004d40 msgr2=0x7fac2c0051f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:07.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:07.917+0000 7fac4818a640 1 --2- 172.21.15.38:0/3469242090 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fac2c004d40 0x7fac2c0051f0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7fac38001d90 tx=0x7fac2c005830 comp rx=0 tx=0).stop 2024-09-21T03:40:07.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:07.917+0000 7fac46727640 1 -- 172.21.15.38:0/3469242090 reap_dead start 2024-09-21T03:40:07.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:07.917+0000 7fac4818a640 1 -- 172.21.15.38:0/3469242090 shutdown_connections 2024-09-21T03:40:07.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:07.917+0000 7fac4818a640 1 -- 172.21.15.38:0/3469242090 >> 172.21.15.38:0/3469242090 conn(0x7fac400d4440 msgr2=0x7fac400d4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:07.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:07.918+0000 7fac4818a640 1 -- 172.21.15.38:0/3469242090 shutdown_connections 2024-09-21T03:40:07.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:07.918+0000 7fac4818a640 1 -- 172.21.15.38:0/3469242090 wait complete. 2024-09-21T03:40:07.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:07.919+0000 7fac4818a640 1 Processor -- start 2024-09-21T03:40:07.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:07.919+0000 7fac4818a640 1 -- start start 2024-09-21T03:40:07.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:07.921+0000 7fac4818a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fac2c004d40 0x7fac4015e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:07.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:07.921+0000 7fac4818a640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fac401f58d0 0x7fac401f5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:07.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:07.921+0000 7fac4818a640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fac400d0c30 0x7fac400d1040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:07.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:07.921+0000 7fac4818a640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fac400b50f0 con 0x7fac2c004d40 2024-09-21T03:40:07.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:07.921+0000 7fac4818a640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fac400b4f70 con 0x7fac401f58d0 2024-09-21T03:40:07.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:07.921+0000 7fac4818a640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fac400b5270 con 0x7fac400d0c30 2024-09-21T03:40:07.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:07.921+0000 7fac45f26640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fac2c004d40 0x7fac4015e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:07.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:07.921+0000 7fac45f26640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fac2c004d40 0x7fac4015e530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:55642/0 (socket says 172.21.15.38:55642) 2024-09-21T03:40:07.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:07.921+0000 7fac45f26640 1 -- 172.21.15.38:0/2404658985 learned_addr learned my addr 172.21.15.38:0/2404658985 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:07.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:07.921+0000 7fac45725640 1 --2- 172.21.15.38:0/2404658985 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fac401f58d0 0x7fac401f5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:07.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:07.921+0000 7fac46727640 1 --2- 172.21.15.38:0/2404658985 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fac400d0c30 0x7fac400d1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:07.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:07.921+0000 7fac45f26640 1 -- 172.21.15.38:0/2404658985 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fac400d0c30 msgr2=0x7fac400d1040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:07.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:07.921+0000 7fac45f26640 1 --2- 172.21.15.38:0/2404658985 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fac400d0c30 0x7fac400d1040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:07.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:07.921+0000 7fac45f26640 1 -- 172.21.15.38:0/2404658985 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fac401f58d0 msgr2=0x7fac401f5ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:07.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:07.921+0000 7fac45f26640 1 --2- 172.21.15.38:0/2404658985 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fac401f58d0 0x7fac401f5ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:07.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:07.921+0000 7fac45f26640 1 -- 172.21.15.38:0/2404658985 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac400d1580 con 0x7fac2c004d40 2024-09-21T03:40:07.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:07.921+0000 7fac45725640 1 --2- 172.21.15.38:0/2404658985 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fac401f58d0 0x7fac401f5ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:40:07.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:07.921+0000 7fac46727640 1 --2- 172.21.15.38:0/2404658985 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fac400d0c30 0x7fac400d1040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:40:07.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:07.921+0000 7fac45f26640 1 --2- 172.21.15.38:0/2404658985 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fac2c004d40 0x7fac4015e530 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7fac3c011ae0 tx=0x7fac3c009e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:07.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:07.922+0000 7fac367fc640 1 -- 172.21.15.38:0/2404658985 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac3c01e070 con 0x7fac2c004d40 2024-09-21T03:40:07.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:07.922+0000 7fac367fc640 1 -- 172.21.15.38:0/2404658985 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fac3c00fe70 con 0x7fac2c004d40 2024-09-21T03:40:07.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:07.922+0000 7fac367fc640 1 -- 172.21.15.38:0/2404658985 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac3c004290 con 0x7fac2c004d40 2024-09-21T03:40:07.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:07.922+0000 7fac4818a640 1 -- 172.21.15.38:0/2404658985 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac4015ea70 con 0x7fac2c004d40 2024-09-21T03:40:07.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:07.922+0000 7fac4818a640 1 -- 172.21.15.38:0/2404658985 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac4015b9f0 con 0x7fac2c004d40 2024-09-21T03:40:07.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:07.923+0000 7fac4818a640 1 -- 172.21.15.38:0/2404658985 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac400d2d10 con 0x7fac2c004d40 2024-09-21T03:40:07.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:07.923+0000 7fac367fc640 1 -- 172.21.15.38:0/2404658985 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fac3c021070 con 0x7fac2c004d40 2024-09-21T03:40:07.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:07.924+0000 7fac367fc640 1 --2- 172.21.15.38:0/2404658985 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fac24078b90 0x7fac2407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:07.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:07.924+0000 7fac367fc640 1 -- 172.21.15.38:0/2404658985 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6101+0+0 (secure 0 0 0) 0x7fac3c09b950 con 0x7fac2c004d40 2024-09-21T03:40:07.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:07.924+0000 7fac45725640 1 --2- 172.21.15.38:0/2404658985 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fac24078b90 0x7fac2407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:07.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:07.925+0000 7fac45725640 1 --2- 172.21.15.38:0/2404658985 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fac24078b90 0x7fac2407b0a0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fac30000fb0 tx=0x7fac300067a0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:07.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:07.929+0000 7fac367fc640 1 -- 172.21.15.38:0/2404658985 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fac400d1580 con 0x7fac2c004d40 2024-09-21T03:40:08.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.154+0000 7fac4818a640 1 -- 172.21.15.38:0/2404658985 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fac400d1800 con 0x7fac2c004d40 2024-09-21T03:40:08.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.155+0000 7fac367fc640 1 -- 172.21.15.38:0/2404658985 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) ==== 94+0+5222 (secure 0 0 0) 0x7fac3c0636f0 con 0x7fac2c004d40 2024-09-21T03:40:08.157 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:40:08.158 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":83,"btime":"2024-09-21T03:40:05:917318+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6086,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/965972663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":965972663},{"type":"v1","addr":"172.21.15.195:6836","nonce":965972663}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":7020,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/751618334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":751618334},{"type":"v1","addr":"172.21.15.38:6833","nonce":751618334}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":7023,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/2934569248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":2934569248},{"type":"v1","addr":"172.21.15.38:6835","nonce":2934569248}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":83,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:40:02.319282+0000","modified":"2024-09-21T03:40:05.304556+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7035},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7035":{"gid":7035,"name":"b","rank":0,"incarnation":79,"state":"up:active","state_seq":3,"addr":"172.21.15.195:6837/712370966","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":712370966},{"type":"v1","addr":"172.21.15.195:6837","nonce":712370966}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7035,"qdb_cluster":[7035]},"id":6}]} 2024-09-21T03:40:08.158 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 83 2024-09-21T03:40:08.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.157+0000 7fac4818a640 1 -- 172.21.15.38:0/2404658985 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fac24078b90 msgr2=0x7fac2407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:08.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.157+0000 7fac4818a640 1 --2- 172.21.15.38:0/2404658985 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fac24078b90 0x7fac2407b0a0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fac30000fb0 tx=0x7fac300067a0 comp rx=0 tx=0).stop 2024-09-21T03:40:08.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.157+0000 7fac4818a640 1 -- 172.21.15.38:0/2404658985 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fac2c004d40 msgr2=0x7fac4015e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:08.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.157+0000 7fac4818a640 1 --2- 172.21.15.38:0/2404658985 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fac2c004d40 0x7fac4015e530 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7fac3c011ae0 tx=0x7fac3c009e90 comp rx=0 tx=0).stop 2024-09-21T03:40:08.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.157+0000 7fac4818a640 1 -- 172.21.15.38:0/2404658985 shutdown_connections 2024-09-21T03:40:08.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.157+0000 7fac4818a640 1 --2- 172.21.15.38:0/2404658985 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fac24078b90 0x7fac2407b0a0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:08.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.157+0000 7fac4818a640 1 --2- 172.21.15.38:0/2404658985 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fac400d0c30 0x7fac400d1040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:08.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.157+0000 7fac4818a640 1 --2- 172.21.15.38:0/2404658985 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fac401f58d0 0x7fac401f5ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:08.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.157+0000 7fac4818a640 1 --2- 172.21.15.38:0/2404658985 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fac2c004d40 0x7fac4015e530 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:08.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.157+0000 7fac4818a640 1 -- 172.21.15.38:0/2404658985 >> 172.21.15.38:0/2404658985 conn(0x7fac400d4440 msgr2=0x7fac400d4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:08.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.157+0000 7fac4818a640 1 -- 172.21.15.38:0/2404658985 shutdown_connections 2024-09-21T03:40:08.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.157+0000 7fac4818a640 1 -- 172.21.15.38:0/2404658985 wait complete. 2024-09-21T03:40:08.573 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 83, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T03:40:02.319282+0000', 'modified': '2024-09-21T03:40:05.304556+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7035}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7035': {'gid': 7035, 'name': 'b', 'rank': 0, 'incarnation': 79, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.195:6837/712370966', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6835', 'nonce': 712370966}, {'type': 'v1', 'addr': '172.21.15.195:6837', 'nonce': 712370966}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7035, 'qdb_cluster': [7035]} 2024-09-21T03:40:08.574 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-21T03:40:08.574 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-09-21T03:40:08.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.694+0000 7faf010be640 1 -- 172.21.15.38:0/3465412763 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faee4005300 msgr2=0x7faee400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:08.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.694+0000 7faf010be640 1 --2- 172.21.15.38:0/3465412763 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faee4005300 0x7faee400b9c0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7faef400c3f0 tx=0x7faee4005830 comp rx=0 tx=0).stop 2024-09-21T03:40:08.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.694+0000 7faf010be640 1 -- 172.21.15.38:0/3465412763 shutdown_connections 2024-09-21T03:40:08.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.694+0000 7faf010be640 1 --2- 172.21.15.38:0/3465412763 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faee4005300 0x7faee400b9c0 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:08.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.694+0000 7faf010be640 1 -- 172.21.15.38:0/3465412763 >> 172.21.15.38:0/3465412763 conn(0x7faefc09d830 msgr2=0x7faefc0b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:08.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.694+0000 7faf010be640 1 -- 172.21.15.38:0/3465412763 shutdown_connections 2024-09-21T03:40:08.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.695+0000 7faf010be640 1 -- 172.21.15.38:0/3465412763 wait complete. 2024-09-21T03:40:08.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.697+0000 7faf010be640 1 Processor -- start 2024-09-21T03:40:08.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.697+0000 7faf010be640 1 -- start start 2024-09-21T03:40:08.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.698+0000 7faf010be640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faee4005300 0x7faefc15c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:08.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.698+0000 7faf010be640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faefc1f5370 0x7faefc06f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:08.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.698+0000 7faf010be640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faefc06e380 0x7faefc06d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:08.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.698+0000 7faf010be640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7faefc0cf120 con 0x7faefc06e380 2024-09-21T03:40:08.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.698+0000 7faf010be640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7faefc0cefa0 con 0x7faee4005300 2024-09-21T03:40:08.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.698+0000 7faf010be640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7faefc0cf2a0 con 0x7faefc1f5370 2024-09-21T03:40:08.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.698+0000 7faefb59e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faefc06e380 0x7faefc06d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:08.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.698+0000 7faefb59e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faefc06e380 0x7faefc06d550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:55656/0 (socket says 172.21.15.38:55656) 2024-09-21T03:40:08.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.698+0000 7faefb59e640 1 -- 172.21.15.38:0/650631781 learned_addr learned my addr 172.21.15.38:0/650631781 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:08.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.698+0000 7faefa59c640 1 --2- 172.21.15.38:0/650631781 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faefc1f5370 0x7faefc06f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:08.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.698+0000 7faefad9d640 1 --2- 172.21.15.38:0/650631781 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faee4005300 0x7faefc15c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:08.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.698+0000 7faefb59e640 1 -- 172.21.15.38:0/650631781 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faefc1f5370 msgr2=0x7faefc06f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:08.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.698+0000 7faefb59e640 1 --2- 172.21.15.38:0/650631781 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faefc1f5370 0x7faefc06f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:08.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.698+0000 7faefb59e640 1 -- 172.21.15.38:0/650631781 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faee4005300 msgr2=0x7faefc15c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:08.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.698+0000 7faefb59e640 1 --2- 172.21.15.38:0/650631781 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faee4005300 0x7faefc15c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:08.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.699+0000 7faefb59e640 1 -- 172.21.15.38:0/650631781 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faefc1f5b80 con 0x7faefc06e380 2024-09-21T03:40:08.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.699+0000 7faefad9d640 1 --2- 172.21.15.38:0/650631781 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faee4005300 0x7faefc15c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:40:08.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.699+0000 7faefb59e640 1 --2- 172.21.15.38:0/650631781 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faefc06e380 0x7faefc06d550 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7faef400c510 tx=0x7faef4001a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:08.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.699+0000 7faee37fe640 1 -- 172.21.15.38:0/650631781 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faef40065f0 con 0x7faefc06e380 2024-09-21T03:40:08.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.699+0000 7faee37fe640 1 -- 172.21.15.38:0/650631781 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faef403a070 con 0x7faefc06e380 2024-09-21T03:40:08.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.699+0000 7faee37fe640 1 -- 172.21.15.38:0/650631781 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faef4003980 con 0x7faefc06e380 2024-09-21T03:40:08.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.699+0000 7faf010be640 1 -- 172.21.15.38:0/650631781 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faefc1f5e60 con 0x7faefc06e380 2024-09-21T03:40:08.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.699+0000 7faf010be640 1 -- 172.21.15.38:0/650631781 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faefc06fae0 con 0x7faefc06e380 2024-09-21T03:40:08.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.701+0000 7faee37fe640 1 -- 172.21.15.38:0/650631781 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7faef4003b20 con 0x7faefc06e380 2024-09-21T03:40:08.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.701+0000 7faee37fe640 1 --2- 172.21.15.38:0/650631781 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7faee8078b90 0x7faee807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:08.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.702+0000 7faee37fe640 1 -- 172.21.15.38:0/650631781 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6101+0+0 (secure 0 0 0) 0x7faef40bac60 con 0x7faefc06e380 2024-09-21T03:40:08.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.702+0000 7faefad9d640 1 --2- 172.21.15.38:0/650631781 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7faee8078b90 0x7faee807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:08.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.702+0000 7faf08023740 1 Processor -- start 2024-09-21T03:40:08.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.702+0000 7faefad9d640 1 --2- 172.21.15.38:0/650631781 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7faee8078b90 0x7faee807b0a0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7faeec005fd0 tx=0x7faeec009500 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:08.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.702+0000 7faf08023740 1 -- start start 2024-09-21T03:40:08.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.704+0000 7faf08023740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562f664b6270 0x562f66887fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:08.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.704+0000 7faf08023740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562f6648d3b0 0x562f668884e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:08.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.704+0000 7faf08023740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562f66495590 0x562f668a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:08.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.704+0000 7faf08023740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x562f667d0820 con 0x562f6648d3b0 2024-09-21T03:40:08.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.704+0000 7faf08023740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x562f667bfc50 con 0x562f664b6270 2024-09-21T03:40:08.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.704+0000 7faf08023740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x562f667c3a60 con 0x562f66495590 2024-09-21T03:40:08.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.704+0000 7faefa59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562f6648d3b0 0x562f668884e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:08.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.704+0000 7faefa59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562f6648d3b0 0x562f668884e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:55668/0 (socket says 172.21.15.38:55668) 2024-09-21T03:40:08.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.704+0000 7faefa59c640 1 -- 172.21.15.38:0/3106947942 learned_addr learned my addr 172.21.15.38:0/3106947942 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:08.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.704+0000 7faefad9d640 1 --2- 172.21.15.38:0/3106947942 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562f664b6270 0x562f66887fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:08.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.704+0000 7faefb59e640 1 --2- 172.21.15.38:0/3106947942 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562f66495590 0x562f668a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:08.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.704+0000 7faefa59c640 1 -- 172.21.15.38:0/3106947942 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562f66495590 msgr2=0x562f668a53e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:08.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.704+0000 7faefa59c640 1 --2- 172.21.15.38:0/3106947942 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562f66495590 0x562f668a53e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:08.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.704+0000 7faefa59c640 1 -- 172.21.15.38:0/3106947942 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562f664b6270 msgr2=0x562f66887fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:08.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.704+0000 7faefa59c640 1 --2- 172.21.15.38:0/3106947942 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562f664b6270 0x562f66887fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:08.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.704+0000 7faefa59c640 1 -- 172.21.15.38:0/3106947942 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f66521850 con 0x562f6648d3b0 2024-09-21T03:40:08.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.704+0000 7faefa59c640 1 --2- 172.21.15.38:0/3106947942 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562f6648d3b0 0x562f668884e0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7faefc08d980 tx=0x7faef0003a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14045440005d70af server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:08.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.704+0000 7faee0ff9640 1 -- 172.21.15.38:0/3106947942 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faef0012920 con 0x562f6648d3b0 2024-09-21T03:40:08.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.704+0000 7faee0ff9640 1 -- 172.21.15.38:0/3106947942 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faef0012ac0 con 0x562f6648d3b0 2024-09-21T03:40:08.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.704+0000 7faee0ff9640 1 -- 172.21.15.38:0/3106947942 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faef0001330 con 0x562f6648d3b0 2024-09-21T03:40:08.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.705+0000 7faf08023740 1 -- 172.21.15.38:0/3106947942 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562f6648d3b0 msgr2=0x562f668884e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:08.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.705+0000 7faf08023740 1 --2- 172.21.15.38:0/3106947942 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562f6648d3b0 0x562f668884e0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7faefc08d980 tx=0x7faef0003a10 comp rx=0 tx=0).stop 2024-09-21T03:40:08.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.705+0000 7faf08023740 1 -- 172.21.15.38:0/3106947942 shutdown_connections 2024-09-21T03:40:08.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.705+0000 7faf08023740 1 --2- 172.21.15.38:0/3106947942 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562f66495590 0x562f668a53e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:08.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.705+0000 7faf08023740 1 --2- 172.21.15.38:0/3106947942 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562f6648d3b0 0x562f668884e0 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:08.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.705+0000 7faf08023740 1 --2- 172.21.15.38:0/3106947942 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562f664b6270 0x562f66887fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:08.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.705+0000 7faf08023740 1 -- 172.21.15.38:0/3106947942 >> 172.21.15.38:0/3106947942 conn(0x562f664aa3c0 msgr2=0x562f6685e260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:08.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.705+0000 7faf08023740 1 -- 172.21.15.38:0/3106947942 shutdown_connections 2024-09-21T03:40:08.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.705+0000 7faf08023740 1 -- 172.21.15.38:0/3106947942 wait complete. 2024-09-21T03:40:08.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.706+0000 7faf08023740 1 -- start start 2024-09-21T03:40:08.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.706+0000 7faf08023740 1 Processor -- start 2024-09-21T03:40:08.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.708+0000 7faf08023740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562f6648d3b0 0x562f6689c930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:08.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.708+0000 7faf08023740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562f66495590 0x562f668b52c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:08.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.708+0000 7faf08023740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562f664b6270 0x562f668b5800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:08.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.708+0000 7faf08023740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x562f66530620 con 0x562f6648d3b0 2024-09-21T03:40:08.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.708+0000 7faf08023740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x562f667655f0 con 0x562f664b6270 2024-09-21T03:40:08.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.708+0000 7faf08023740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x562f66888a20 con 0x562f66495590 2024-09-21T03:40:08.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.708+0000 7faf08023740 10 client.0 _ll_register_callbacks cb 0x562f667690a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:40:08.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.708+0000 7faefad9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562f6648d3b0 0x562f6689c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:08.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.708+0000 7faefad9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562f6648d3b0 0x562f6689c930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:55678/0 (socket says 172.21.15.38:55678) 2024-09-21T03:40:08.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.708+0000 7faefad9d640 1 -- 172.21.15.38:0/1483570928 learned_addr learned my addr 172.21.15.38:0/1483570928 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:08.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.708+0000 7faefa59c640 1 --2- 172.21.15.38:0/1483570928 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562f66495590 0x562f668b52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:08.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.708+0000 7faefb59e640 1 --2- 172.21.15.38:0/1483570928 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562f664b6270 0x562f668b5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:08.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.708+0000 7faefad9d640 1 -- 172.21.15.38:0/1483570928 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562f66495590 msgr2=0x562f668b52c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:08.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.708+0000 7faefad9d640 1 --2- 172.21.15.38:0/1483570928 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562f66495590 0x562f668b52c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:08.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.708+0000 7faefad9d640 1 -- 172.21.15.38:0/1483570928 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562f664b6270 msgr2=0x562f668b5800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:08.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.708+0000 7faefad9d640 1 --2- 172.21.15.38:0/1483570928 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562f664b6270 0x562f668b5800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:08.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.708+0000 7faefad9d640 1 -- 172.21.15.38:0/1483570928 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f665986b0 con 0x562f6648d3b0 2024-09-21T03:40:08.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.708+0000 7faefad9d640 1 --2- 172.21.15.38:0/1483570928 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562f6648d3b0 0x562f6689c930 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7faeec004280 tx=0x7faeec004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f769723a79f2c46 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:08.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.708+0000 7faed9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-21T03:40:08.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.708+0000 7faed9ffb640 1 -- 172.21.15.38:0/1483570928 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faeec02e060 con 0x562f6648d3b0 2024-09-21T03:40:08.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.708+0000 7faed9ffb640 1 -- 172.21.15.38:0/1483570928 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 6101+0+0 (secure 0 0 0) 0x7faeec004800 con 0x562f6648d3b0 2024-09-21T03:40:08.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.708+0000 7faf08023740 1 -- 172.21.15.38:0/1483570928 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faef0012ac0 con 0x562f6648d3b0 2024-09-21T03:40:08.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.709+0000 7faed9ffb640 1 -- 172.21.15.38:0/1483570928 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7faeec034050 con 0x562f6648d3b0 2024-09-21T03:40:08.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.709+0000 7faf08023740 10 client.7155 fetch_fsmap learned FSMap version 83 2024-09-21T03:40:08.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.709+0000 7faf08023740 1 -- 172.21.15.38:0/1483570928 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=83,monmap=2+}) -- 0x562f6685d3d0 con 0x562f6648d3b0 2024-09-21T03:40:08.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.709+0000 7faed9ffb640 1 -- 172.21.15.38:0/1483570928 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faef0012ac0 con 0x562f6648d3b0 2024-09-21T03:40:08.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.709+0000 7faed9ffb640 1 -- 172.21.15.38:0/1483570928 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 83) ==== 3502+0+0 (secure 0 0 0) 0x7faeec037020 con 0x562f6648d3b0 2024-09-21T03:40:08.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.709+0000 7faf08023740 10 client.7155 fetch_fsmap finished waiting for FSMap version 83 2024-09-21T03:40:08.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.709+0000 7faf08023740 10 client.7155 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-21T03:40:08.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.709+0000 7faf08023740 20 client.7155 populate_metadata read hostname 'smithi038' 2024-09-21T03:40:08.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.709+0000 7faf08023740 1 --2- 172.21.15.38:0/1483570928 >> [v2:172.21.15.195:6835/712370966,v1:172.21.15.195:6837/712370966] conn(0x562f66474e40 0x562f668bf6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:08.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.709+0000 7faf08023740 4 client.7155 mds_command: new command op to 7035 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:40:08.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.709+0000 7faf08023740 1 -- 172.21.15.38:0/1483570928 --> [v2:172.21.15.195:6835/712370966,v1:172.21.15.195:6837/712370966] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562f667d2770 con 0x562f66474e40 2024-09-21T03:40:08.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.709+0000 7faefa59c640 1 --2- 172.21.15.38:0/1483570928 >> [v2:172.21.15.195:6835/712370966,v1:172.21.15.195:6837/712370966] conn(0x562f66474e40 0x562f668bf6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:08.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.710+0000 7faed9ffb640 10 client.7155 ms_handle_connect on v2:172.21.15.195:6835/712370966 2024-09-21T03:40:08.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.710+0000 7faefa59c640 1 --2- 172.21.15.38:0/1483570928 >> [v2:172.21.15.195:6835/712370966,v1:172.21.15.195:6837/712370966] conn(0x562f66474e40 0x562f668bf6d0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f05c439eb15ced5 server_cookie=a76f165ebe918bc0 in_seq=0 out_seq=0 2024-09-21T03:40:08.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.711+0000 7faed9ffb640 1 -- 172.21.15.38:0/1483570928 <== mds.0 v2:172.21.15.195:6835/712370966 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562f667d2770 con 0x562f66474e40 2024-09-21T03:40:08.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.711+0000 7faed9ffb640 10 client.7155 handle_command_reply: tid=0 2024-09-21T03:40:08.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.711+0000 7faf08023740 1 client.7155 shutdown 2024-09-21T03:40:08.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.711+0000 7faf08023740 1 -- 172.21.15.38:0/1483570928 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562f6648d3b0 msgr2=0x562f6689c930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:08.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.711+0000 7faf08023740 1 --2- 172.21.15.38:0/1483570928 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562f6648d3b0 0x562f6689c930 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7faeec004280 tx=0x7faeec004a40 comp rx=0 tx=0).stop 2024-09-21T03:40:08.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.711+0000 7faf08023740 1 -- 172.21.15.38:0/1483570928 shutdown_connections 2024-09-21T03:40:08.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.711+0000 7faf08023740 1 --2- 172.21.15.38:0/1483570928 >> [v2:172.21.15.195:6835/712370966,v1:172.21.15.195:6837/712370966] conn(0x562f66474e40 0x562f668bf6d0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:08.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.711+0000 7faf08023740 1 --2- 172.21.15.38:0/1483570928 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562f664b6270 0x562f668b5800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:08.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.711+0000 7faf08023740 1 --2- 172.21.15.38:0/1483570928 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562f66495590 0x562f668b52c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:08.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.711+0000 7faed9ffb640 0 client.7155 ms_handle_reset on v2:172.21.15.195:6835/712370966 2024-09-21T03:40:08.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.711+0000 7faf08023740 1 --2- 172.21.15.38:0/1483570928 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562f6648d3b0 0x562f6689c930 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:08.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.711+0000 7faf08023740 1 -- 172.21.15.38:0/1483570928 >> 172.21.15.38:0/1483570928 conn(0x562f664aa3c0 msgr2=0x562f6685e110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:08.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.711+0000 7faf08023740 1 -- 172.21.15.38:0/1483570928 shutdown_connections 2024-09-21T03:40:08.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.711+0000 7faf08023740 1 -- 172.21.15.38:0/1483570928 wait complete. 2024-09-21T03:40:08.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.711+0000 7faf08023740 20 client.7155 trim_cache size 0 max 16384 2024-09-21T03:40:08.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.729+0000 7faf08023740 1 Processor -- start 2024-09-21T03:40:08.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.729+0000 7faf08023740 1 -- start start 2024-09-21T03:40:08.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.731+0000 7faf08023740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562f6648d3b0 0x562f668b0a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:08.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.731+0000 7faf08023740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562f66495590 0x562f66918fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:08.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.731+0000 7faf08023740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562f664b6270 0x562f66919520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:08.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.731+0000 7faf08023740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7faefc201c50 con 0x562f664b6270 2024-09-21T03:40:08.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.731+0000 7faf08023740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7faefc200fb0 con 0x562f6648d3b0 2024-09-21T03:40:08.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.731+0000 7faf08023740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7faefc201600 con 0x562f66495590 2024-09-21T03:40:08.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.731+0000 7faefb59e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562f664b6270 0x562f66919520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:08.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.731+0000 7faefb59e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562f664b6270 0x562f66919520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:55680/0 (socket says 172.21.15.38:55680) 2024-09-21T03:40:08.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.731+0000 7faefb59e640 1 -- 172.21.15.38:0/1769654619 learned_addr learned my addr 172.21.15.38:0/1769654619 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:08.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.731+0000 7faefa59c640 1 --2- 172.21.15.38:0/1769654619 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562f66495590 0x562f66918fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:08.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.731+0000 7faefad9d640 1 --2- 172.21.15.38:0/1769654619 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562f6648d3b0 0x562f668b0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:08.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.731+0000 7faefb59e640 1 -- 172.21.15.38:0/1769654619 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562f66495590 msgr2=0x562f66918fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:08.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.731+0000 7faefb59e640 1 --2- 172.21.15.38:0/1769654619 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562f66495590 0x562f66918fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:08.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.731+0000 7faefb59e640 1 -- 172.21.15.38:0/1769654619 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562f6648d3b0 msgr2=0x562f668b0a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:08.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.731+0000 7faefb59e640 1 --2- 172.21.15.38:0/1769654619 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562f6648d3b0 0x562f668b0a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:08.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.731+0000 7faefb59e640 1 -- 172.21.15.38:0/1769654619 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f66863a40 con 0x562f664b6270 2024-09-21T03:40:08.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.731+0000 7faefad9d640 1 --2- 172.21.15.38:0/1769654619 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562f6648d3b0 0x562f668b0a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:40:08.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.731+0000 7faefb59e640 1 --2- 172.21.15.38:0/1769654619 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562f664b6270 0x562f66919520 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7faef400c3f0 tx=0x7faef4086040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3d2d7525f254196 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:08.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.732+0000 7faedbfff640 1 -- 172.21.15.38:0/1769654619 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faef40a1070 con 0x562f664b6270 2024-09-21T03:40:08.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.732+0000 7faedbfff640 1 -- 172.21.15.38:0/1769654619 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faef409c030 con 0x562f664b6270 2024-09-21T03:40:08.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.732+0000 7faedbfff640 1 -- 172.21.15.38:0/1769654619 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faef4083df0 con 0x562f664b6270 2024-09-21T03:40:08.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.732+0000 7faf08023740 1 -- 172.21.15.38:0/1769654619 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562f664b6270 msgr2=0x562f66919520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:08.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.732+0000 7faf08023740 1 --2- 172.21.15.38:0/1769654619 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562f664b6270 0x562f66919520 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7faef400c3f0 tx=0x7faef4086040 comp rx=0 tx=0).stop 2024-09-21T03:40:08.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.732+0000 7faf08023740 1 -- 172.21.15.38:0/1769654619 shutdown_connections 2024-09-21T03:40:08.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.732+0000 7faf08023740 1 --2- 172.21.15.38:0/1769654619 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562f664b6270 0x562f66919520 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:08.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.732+0000 7faf08023740 1 --2- 172.21.15.38:0/1769654619 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562f66495590 0x562f66918fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:08.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.732+0000 7faf08023740 1 --2- 172.21.15.38:0/1769654619 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562f6648d3b0 0x562f668b0a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:08.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.732+0000 7faf08023740 1 -- 172.21.15.38:0/1769654619 >> 172.21.15.38:0/1769654619 conn(0x562f664aa3c0 msgr2=0x562f6685f180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:08.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.732+0000 7faf08023740 1 -- 172.21.15.38:0/1769654619 shutdown_connections 2024-09-21T03:40:08.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.732+0000 7faf08023740 1 -- 172.21.15.38:0/1769654619 wait complete. 2024-09-21T03:40:08.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.733+0000 7faf08023740 1 -- start start 2024-09-21T03:40:08.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.733+0000 7faf08023740 1 Processor -- start 2024-09-21T03:40:08.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.735+0000 7faf08023740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562f6648d3b0 0x562f6690a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:08.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.735+0000 7faf08023740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562f66495590 0x562f668b3a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:08.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.735+0000 7faf08023740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562f664b6270 0x562f668b3f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:08.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.735+0000 7faf08023740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x562f66863230 con 0x562f664b6270 2024-09-21T03:40:08.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.735+0000 7faf08023740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x562f668642b0 con 0x562f66495590 2024-09-21T03:40:08.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.735+0000 7faf08023740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x562f668637f0 con 0x562f6648d3b0 2024-09-21T03:40:08.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.735+0000 7faf08023740 10 client.0 _ll_register_callbacks cb 0x7faeec003a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:40:08.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.735+0000 7faefb59e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562f664b6270 0x562f668b3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:08.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.735+0000 7faefb59e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562f664b6270 0x562f668b3f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:55688/0 (socket says 172.21.15.38:55688) 2024-09-21T03:40:08.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.735+0000 7faefb59e640 1 -- 172.21.15.38:0/1734776920 learned_addr learned my addr 172.21.15.38:0/1734776920 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:08.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.735+0000 7faefad9d640 1 --2- 172.21.15.38:0/1734776920 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562f6648d3b0 0x562f6690a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:08.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.735+0000 7faefa59c640 1 --2- 172.21.15.38:0/1734776920 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562f66495590 0x562f668b3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:08.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.735+0000 7faefb59e640 1 -- 172.21.15.38:0/1734776920 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562f6648d3b0 msgr2=0x562f6690a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:08.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.735+0000 7faefb59e640 1 --2- 172.21.15.38:0/1734776920 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562f6648d3b0 0x562f6690a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:08.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.735+0000 7faefb59e640 1 -- 172.21.15.38:0/1734776920 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562f66495590 msgr2=0x562f668b3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:08.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.735+0000 7faefb59e640 1 --2- 172.21.15.38:0/1734776920 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562f66495590 0x562f668b3a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:08.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.735+0000 7faefb59e640 1 -- 172.21.15.38:0/1734776920 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f6690acb0 con 0x562f664b6270 2024-09-21T03:40:08.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.735+0000 7faefa59c640 1 --2- 172.21.15.38:0/1734776920 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562f66495590 0x562f668b3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:40:08.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.735+0000 7faefad9d640 1 --2- 172.21.15.38:0/1734776920 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562f6648d3b0 0x562f6690a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:40:08.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.735+0000 7faefb59e640 1 --2- 172.21.15.38:0/1734776920 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562f664b6270 0x562f668b3f50 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7faef4001410 tx=0x7faef4083de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a98c5a78d4e08b6f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:08.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.735+0000 7faed9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-21T03:40:08.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.735+0000 7faed9ffb640 1 -- 172.21.15.38:0/1734776920 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faef4091060 con 0x562f664b6270 2024-09-21T03:40:08.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.735+0000 7faed9ffb640 1 -- 172.21.15.38:0/1734776920 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 6101+0+0 (secure 0 0 0) 0x7faef407e1d0 con 0x562f664b6270 2024-09-21T03:40:08.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.735+0000 7faf08023740 1 -- 172.21.15.38:0/1734776920 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faef409c030 con 0x562f664b6270 2024-09-21T03:40:08.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.736+0000 7faed9ffb640 1 -- 172.21.15.38:0/1734776920 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7faef40a1050 con 0x562f664b6270 2024-09-21T03:40:08.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.736+0000 7faf08023740 10 client.7161 fetch_fsmap learned FSMap version 83 2024-09-21T03:40:08.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.736+0000 7faf08023740 1 -- 172.21.15.38:0/1734776920 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=83,monmap=2+}) -- 0x562f66862350 con 0x562f664b6270 2024-09-21T03:40:08.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.736+0000 7faed9ffb640 1 -- 172.21.15.38:0/1734776920 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faef409c030 con 0x562f664b6270 2024-09-21T03:40:08.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.736+0000 7faed9ffb640 1 -- 172.21.15.38:0/1734776920 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 83) ==== 3502+0+0 (secure 0 0 0) 0x7faef4047030 con 0x562f664b6270 2024-09-21T03:40:08.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.736+0000 7faf08023740 10 client.7161 fetch_fsmap finished waiting for FSMap version 83 2024-09-21T03:40:08.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.736+0000 7faf08023740 10 client.7161 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-21T03:40:08.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.736+0000 7faf08023740 20 client.7161 populate_metadata read hostname 'smithi038' 2024-09-21T03:40:08.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.736+0000 7faf08023740 1 --2- 172.21.15.38:0/1734776920 >> [v2:172.21.15.195:6835/712370966,v1:172.21.15.195:6837/712370966] conn(0x562f669269d0 0x562f66926de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:08.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.736+0000 7faf08023740 4 client.7161 mds_command: new command op to 7035 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T03:40:08.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.736+0000 7faf08023740 1 -- 172.21.15.38:0/1734776920 --> [v2:172.21.15.195:6835/712370966,v1:172.21.15.195:6837/712370966] -- command(tid 0: {"prefix": "status"}) -- 0x562f667ce220 con 0x562f669269d0 2024-09-21T03:40:08.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.736+0000 7faefad9d640 1 --2- 172.21.15.38:0/1734776920 >> [v2:172.21.15.195:6835/712370966,v1:172.21.15.195:6837/712370966] conn(0x562f669269d0 0x562f66926de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:08.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.737+0000 7faed9ffb640 10 client.7161 ms_handle_connect on v2:172.21.15.195:6835/712370966 2024-09-21T03:40:08.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.737+0000 7faefad9d640 1 --2- 172.21.15.38:0/1734776920 >> [v2:172.21.15.195:6835/712370966,v1:172.21.15.195:6837/712370966] conn(0x562f669269d0 0x562f66926de0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f7d6f4bca3671ec server_cookie=bb7853f5c3c1f499 in_seq=0 out_seq=0 2024-09-21T03:40:08.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.737+0000 7faed9ffb640 1 -- 172.21.15.38:0/1734776920 <== mds.0 v2:172.21.15.195:6835/712370966 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x562f667ce220 con 0x562f669269d0 2024-09-21T03:40:08.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.737+0000 7faed9ffb640 10 client.7161 handle_command_reply: tid=0 2024-09-21T03:40:08.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.737+0000 7faf08023740 1 client.7161 shutdown 2024-09-21T03:40:08.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.737+0000 7faf08023740 1 -- 172.21.15.38:0/1734776920 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562f664b6270 msgr2=0x562f668b3f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:08.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.737+0000 7faf08023740 1 --2- 172.21.15.38:0/1734776920 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562f664b6270 0x562f668b3f50 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7faef4001410 tx=0x7faef4083de0 comp rx=0 tx=0).stop 2024-09-21T03:40:08.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.737+0000 7faf08023740 1 -- 172.21.15.38:0/1734776920 shutdown_connections 2024-09-21T03:40:08.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.737+0000 7faf08023740 1 --2- 172.21.15.38:0/1734776920 >> [v2:172.21.15.195:6835/712370966,v1:172.21.15.195:6837/712370966] conn(0x562f669269d0 0x562f66926de0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:08.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.737+0000 7faf08023740 1 --2- 172.21.15.38:0/1734776920 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562f664b6270 0x562f668b3f50 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:08.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.737+0000 7faf08023740 1 --2- 172.21.15.38:0/1734776920 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562f66495590 0x562f668b3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:08.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.737+0000 7faed9ffb640 0 client.7161 ms_handle_reset on v2:172.21.15.195:6835/712370966 2024-09-21T03:40:08.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.737+0000 7faf08023740 1 --2- 172.21.15.38:0/1734776920 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562f6648d3b0 0x562f6690a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:08.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.737+0000 7faf08023740 1 -- 172.21.15.38:0/1734776920 >> 172.21.15.38:0/1734776920 conn(0x562f664aa3c0 msgr2=0x562f6685f180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:08.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.737+0000 7faf08023740 1 -- 172.21.15.38:0/1734776920 shutdown_connections 2024-09-21T03:40:08.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.737+0000 7faf08023740 1 -- 172.21.15.38:0/1734776920 wait complete. 2024-09-21T03:40:08.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.737+0000 7faf08023740 20 client.7161 trim_cache size 0 max 16384 2024-09-21T03:40:08.746 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-09-21T03:40:08.746 INFO:teuthology.orchestra.run.smithi038.stdout: "cluster_fsid": "9d50d66f-656b-4fe2-8e3b-fe53b79c4d43", 2024-09-21T03:40:08.746 INFO:teuthology.orchestra.run.smithi038.stdout: "whoami": 0, 2024-09-21T03:40:08.746 INFO:teuthology.orchestra.run.smithi038.stdout: "id": 7035, 2024-09-21T03:40:08.746 INFO:teuthology.orchestra.run.smithi038.stdout: "want_state": "up:active", 2024-09-21T03:40:08.746 INFO:teuthology.orchestra.run.smithi038.stdout: "state": "up:active", 2024-09-21T03:40:08.746 INFO:teuthology.orchestra.run.smithi038.stdout: "fs_name": "cephfs", 2024-09-21T03:40:08.746 INFO:teuthology.orchestra.run.smithi038.stdout: "rank_uptime": 5.8505695360000001, 2024-09-21T03:40:08.746 INFO:teuthology.orchestra.run.smithi038.stdout: "mdsmap_epoch": 83, 2024-09-21T03:40:08.746 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch": 94, 2024-09-21T03:40:08.746 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch_barrier": 94, 2024-09-21T03:40:08.747 INFO:teuthology.orchestra.run.smithi038.stdout: "uptime": 10.477098316999999 2024-09-21T03:40:08.747 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-09-21T03:40:08.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.738+0000 7faf010be640 1 -- 172.21.15.38:0/650631781 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7faee8078b90 msgr2=0x7faee807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:08.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.738+0000 7faf010be640 1 --2- 172.21.15.38:0/650631781 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7faee8078b90 0x7faee807b0a0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7faeec005fd0 tx=0x7faeec009500 comp rx=0 tx=0).stop 2024-09-21T03:40:08.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.738+0000 7faf010be640 1 -- 172.21.15.38:0/650631781 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faefc06e380 msgr2=0x7faefc06d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:08.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.738+0000 7faf010be640 1 --2- 172.21.15.38:0/650631781 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faefc06e380 0x7faefc06d550 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7faef400c510 tx=0x7faef4001a90 comp rx=0 tx=0).stop 2024-09-21T03:40:08.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.738+0000 7faf010be640 1 -- 172.21.15.38:0/650631781 shutdown_connections 2024-09-21T03:40:08.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.738+0000 7faf010be640 1 --2- 172.21.15.38:0/650631781 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faefc06e380 0x7faefc06d550 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:08.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.738+0000 7faf010be640 1 --2- 172.21.15.38:0/650631781 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7faee8078b90 0x7faee807b0a0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:08.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.738+0000 7faf010be640 1 --2- 172.21.15.38:0/650631781 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faefc1f5370 0x7faefc06f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:08.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.738+0000 7faf010be640 1 --2- 172.21.15.38:0/650631781 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faee4005300 0x7faefc15c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:08.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.738+0000 7faf010be640 1 -- 172.21.15.38:0/650631781 >> 172.21.15.38:0/650631781 conn(0x7faefc09d830 msgr2=0x7faefc161190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:08.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.738+0000 7faf010be640 1 -- 172.21.15.38:0/650631781 shutdown_connections 2024-09-21T03:40:08.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.738+0000 7faf010be640 1 -- 172.21.15.38:0/650631781 wait complete. 2024-09-21T03:40:08.752 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T03:40:08.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.865+0000 7f3c4c2a9640 1 -- 172.21.15.38:0/901426971 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3c440cce70 msgr2=0x7f3c440d04c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:08.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.865+0000 7f3c4c2a9640 1 --2- 172.21.15.38:0/901426971 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3c440cce70 0x7f3c440d04c0 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f3c3c009830 tx=0x7f3c3c02a510 comp rx=0 tx=0).stop 2024-09-21T03:40:08.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.865+0000 7f3c4c2a9640 1 -- 172.21.15.38:0/901426971 shutdown_connections 2024-09-21T03:40:08.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.865+0000 7f3c4c2a9640 1 --2- 172.21.15.38:0/901426971 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3c440cce70 0x7f3c440d04c0 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:08.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.865+0000 7f3c4c2a9640 1 --2- 172.21.15.38:0/901426971 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3c440d1ed0 0x7f3c440cc930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:08.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.865+0000 7f3c4c2a9640 1 -- 172.21.15.38:0/901426971 >> 172.21.15.38:0/901426971 conn(0x7f3c440cef40 msgr2=0x7f3c440cf350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:08.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.865+0000 7f3c4c2a9640 1 -- 172.21.15.38:0/901426971 shutdown_connections 2024-09-21T03:40:08.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.865+0000 7f3c4c2a9640 1 -- 172.21.15.38:0/901426971 wait complete. 2024-09-21T03:40:08.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.867+0000 7f3c4c2a9640 1 Processor -- start 2024-09-21T03:40:08.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.867+0000 7f3c4c2a9640 1 -- start start 2024-09-21T03:40:08.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.868+0000 7f3c4c2a9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3c440cce70 0x7f3c4414fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:08.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.868+0000 7f3c4c2a9640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3c440d1ed0 0x7f3c441e7310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:08.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.868+0000 7f3c4c2a9640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3c441536e0 0x7f3c44155480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:08.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.868+0000 7f3c4c2a9640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f3c440d0d90 con 0x7f3c440cce70 2024-09-21T03:40:08.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.868+0000 7f3c4c2a9640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f3c440d0c10 con 0x7f3c441536e0 2024-09-21T03:40:08.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.868+0000 7f3c4c2a9640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f3c440d0f10 con 0x7f3c440d1ed0 2024-09-21T03:40:08.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.868+0000 7f3c4a045640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3c440cce70 0x7f3c4414fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:08.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.868+0000 7f3c4a045640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3c440cce70 0x7f3c4414fbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:55704/0 (socket says 172.21.15.38:55704) 2024-09-21T03:40:08.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.868+0000 7f3c4a045640 1 -- 172.21.15.38:0/647884340 learned_addr learned my addr 172.21.15.38:0/647884340 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:08.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.869+0000 7f3c49844640 1 --2- 172.21.15.38:0/647884340 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3c440d1ed0 0x7f3c441e7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:08.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.869+0000 7f3c4a846640 1 --2- 172.21.15.38:0/647884340 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3c441536e0 0x7f3c44155480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:08.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.869+0000 7f3c4a045640 1 -- 172.21.15.38:0/647884340 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3c440d1ed0 msgr2=0x7f3c441e7310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:08.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.869+0000 7f3c4a045640 1 --2- 172.21.15.38:0/647884340 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3c440d1ed0 0x7f3c441e7310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:08.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.869+0000 7f3c4a045640 1 -- 172.21.15.38:0/647884340 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3c441536e0 msgr2=0x7f3c44155480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:08.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.869+0000 7f3c4a045640 1 --2- 172.21.15.38:0/647884340 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3c441536e0 0x7f3c44155480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:08.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.869+0000 7f3c4a045640 1 -- 172.21.15.38:0/647884340 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c441546e0 con 0x7f3c440cce70 2024-09-21T03:40:08.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.869+0000 7f3c4a045640 1 --2- 172.21.15.38:0/647884340 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3c440cce70 0x7f3c4414fbc0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f3c3400cbf0 tx=0x7f3c3400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:08.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.869+0000 7f3c42ffd640 1 -- 172.21.15.38:0/647884340 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c34016020 con 0x7f3c440cce70 2024-09-21T03:40:08.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.869+0000 7f3c42ffd640 1 -- 172.21.15.38:0/647884340 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3c3400ad90 con 0x7f3c440cce70 2024-09-21T03:40:08.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.869+0000 7f3c4c2a9640 1 -- 172.21.15.38:0/647884340 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c441549c0 con 0x7f3c440cce70 2024-09-21T03:40:08.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.869+0000 7f3c42ffd640 1 -- 172.21.15.38:0/647884340 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c34012680 con 0x7f3c440cce70 2024-09-21T03:40:08.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.869+0000 7f3c4c2a9640 1 -- 172.21.15.38:0/647884340 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c44154f80 con 0x7f3c440cce70 2024-09-21T03:40:08.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.870+0000 7f3c4c2a9640 1 -- 172.21.15.38:0/647884340 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c440d04c0 con 0x7f3c440cce70 2024-09-21T03:40:08.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.871+0000 7f3c42ffd640 1 -- 172.21.15.38:0/647884340 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f3c34019020 con 0x7f3c440cce70 2024-09-21T03:40:08.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.872+0000 7f3c42ffd640 1 --2- 172.21.15.38:0/647884340 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3c30078b90 0x7f3c3007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:08.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.872+0000 7f3c42ffd640 1 -- 172.21.15.38:0/647884340 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6101+0+0 (secure 0 0 0) 0x7f3c3409b670 con 0x7f3c440cce70 2024-09-21T03:40:08.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.872+0000 7f3c49844640 1 --2- 172.21.15.38:0/647884340 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3c30078b90 0x7f3c3007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:08.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.872+0000 7f3c49844640 1 --2- 172.21.15.38:0/647884340 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3c30078b90 0x7f3c3007b0a0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f3c38005fd0 tx=0x7f3c38005850 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:08.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:08.876+0000 7f3c42ffd640 1 -- 172.21.15.38:0/647884340 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f3c34020020 con 0x7f3c440cce70 2024-09-21T03:40:09.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:09.109+0000 7f3c4c2a9640 1 -- 172.21.15.38:0/647884340 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f3c44152f90 con 0x7f3c440cce70 2024-09-21T03:40:09.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:09.109+0000 7f3c42ffd640 1 -- 172.21.15.38:0/647884340 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v83) ==== 53+0+73 (secure 0 0 0) 0x7f3c440cd280 con 0x7f3c440cce70 2024-09-21T03:40:09.113 INFO:teuthology.orchestra.run.smithi038.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T03:40:09.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:09.112+0000 7f3c4c2a9640 1 -- 172.21.15.38:0/647884340 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3c30078b90 msgr2=0x7f3c3007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:09.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:09.112+0000 7f3c4c2a9640 1 --2- 172.21.15.38:0/647884340 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3c30078b90 0x7f3c3007b0a0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f3c38005fd0 tx=0x7f3c38005850 comp rx=0 tx=0).stop 2024-09-21T03:40:09.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:09.112+0000 7f3c4c2a9640 1 -- 172.21.15.38:0/647884340 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3c440cce70 msgr2=0x7f3c4414fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:09.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:09.112+0000 7f3c4c2a9640 1 --2- 172.21.15.38:0/647884340 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3c440cce70 0x7f3c4414fbc0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f3c3400cbf0 tx=0x7f3c3400a060 comp rx=0 tx=0).stop 2024-09-21T03:40:09.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:09.112+0000 7f3c4c2a9640 1 -- 172.21.15.38:0/647884340 shutdown_connections 2024-09-21T03:40:09.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:09.112+0000 7f3c4c2a9640 1 --2- 172.21.15.38:0/647884340 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3c441536e0 0x7f3c44155480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:09.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:09.112+0000 7f3c4c2a9640 1 --2- 172.21.15.38:0/647884340 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3c30078b90 0x7f3c3007b0a0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:09.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:09.112+0000 7f3c4c2a9640 1 --2- 172.21.15.38:0/647884340 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3c440d1ed0 0x7f3c441e7310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:09.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:09.112+0000 7f3c4c2a9640 1 --2- 172.21.15.38:0/647884340 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3c440cce70 0x7f3c4414fbc0 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:09.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:09.112+0000 7f3c4c2a9640 1 -- 172.21.15.38:0/647884340 >> 172.21.15.38:0/647884340 conn(0x7f3c440cef40 msgr2=0x7f3c44071560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:09.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:09.112+0000 7f3c4c2a9640 1 -- 172.21.15.38:0/647884340 shutdown_connections 2024-09-21T03:40:09.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:09.112+0000 7f3c4c2a9640 1 -- 172.21.15.38:0/647884340 wait complete. 2024-09-21T03:40:09.126 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-21T03:40:09.126 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T03:40:09.126 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi038.front.sepia.ceph.com 2024-09-21T03:40:09.126 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-21T03:40:09.126 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T03:40:09.126 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-21T03:40:09.126 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-21T03:40:09.126 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T03:40:09.126 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T03:40:09.127 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:40:09.127 DEBUG:teuthology.orchestra.run.smithi038:> ip addr 2024-09-21T03:40:09.143 INFO:teuthology.orchestra.run.smithi038.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T03:40:09.143 INFO:teuthology.orchestra.run.smithi038.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T03:40:09.143 INFO:teuthology.orchestra.run.smithi038.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T03:40:09.143 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:40:09.143 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 ::1/128 scope host 2024-09-21T03:40:09.143 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:40:09.144 INFO:teuthology.orchestra.run.smithi038.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T03:40:09.144 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:6b:fd:a2 brd ff:ff:ff:ff:ff:ff 2024-09-21T03:40:09.144 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp7s0f0 2024-09-21T03:40:09.144 INFO:teuthology.orchestra.run.smithi038.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T03:40:09.144 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:6b:fd:a3 brd ff:ff:ff:ff:ff:ff 2024-09-21T03:40:09.144 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp7s0f1 2024-09-21T03:40:09.144 INFO:teuthology.orchestra.run.smithi038.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T03:40:09.144 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:bd:15:fc brd ff:ff:ff:ff:ff:ff 2024-09-21T03:40:09.144 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp1s0f0 2024-09-21T03:40:09.144 INFO:teuthology.orchestra.run.smithi038.stdout: inet 172.21.15.38/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-21T03:40:09.144 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft 42400sec preferred_lft 42400sec 2024-09-21T03:40:09.144 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 fe80::ec4:7aff:febd:15fc/64 scope link 2024-09-21T03:40:09.144 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:40:09.145 INFO:teuthology.orchestra.run.smithi038.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T03:40:09.145 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:bd:15:fd brd ff:ff:ff:ff:ff:ff 2024-09-21T03:40:09.145 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp1s0f1 2024-09-21T03:40:09.145 INFO:teuthology.orchestra.run.smithi038.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T03:40:09.145 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 92:55:c7:97:6f:b5 brd ff:ff:ff:ff:ff:ff 2024-09-21T03:40:09.145 INFO:teuthology.orchestra.run.smithi038.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-21T03:40:09.145 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:40:09.145 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 fe80::c49f:c1ff:fe21:b12a/64 scope link 2024-09-21T03:40:09.145 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:40:09.145 INFO:teuthology.orchestra.run.smithi038.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T03:40:09.145 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether a6:3a:83:ad:db:d5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T03:40:09.145 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 fe80::a43a:83ff:fead:dbd5/64 scope link 2024-09-21T03:40:09.145 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:40:09.145 INFO:teuthology.orchestra.run.smithi038.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T03:40:09.146 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 92:55:c7:97:6f:b5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T03:40:09.146 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 fe80::9055:c7ff:fe97:6fb5/64 scope link 2024-09-21T03:40:09.146 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:40:09.146 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:40:09.146 DEBUG:teuthology.orchestra.run.smithi038:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T03:40:09.207 INFO:teuthology.orchestra.run.smithi038.stdout:1 2024-09-21T03:40:09.209 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:40:09.209 DEBUG:teuthology.orchestra.run.smithi038:> route 2024-09-21T03:40:09.265 INFO:teuthology.orchestra.run.smithi038.stdout:Kernel IP routing table 2024-09-21T03:40:09.266 INFO:teuthology.orchestra.run.smithi038.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T03:40:09.266 INFO:teuthology.orchestra.run.smithi038.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-21T03:40:09.266 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-21T03:40:09.266 INFO:teuthology.orchestra.run.smithi038.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T03:40:09.266 DEBUG:teuthology.orchestra.run.smithi038:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T03:40:09.266 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-09-21T03:40:09.266 DEBUG:teuthology.orchestra.run.smithi038:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-21T03:40:09.266 DEBUG:teuthology.orchestra.run.smithi038:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-21T03:40:09.266 DEBUG:teuthology.orchestra.run.smithi038:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-21T03:40:09.267 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-09-21T03:40:09.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:09 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T03:40:09.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:40:09.395 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:40:09.395 DEBUG:teuthology.orchestra.run.smithi038:> ip netns list 2024-09-21T03:40:09.410 INFO:teuthology.orchestra.run.smithi038.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-21T03:40:09.410 INFO:teuthology.orchestra.run.smithi038.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-21T03:40:09.411 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T03:40:09.411 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:40:09.411 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip link set brx.0 up 2024-09-21T03:40:09.476 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T03:40:09.476 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-21T03:40:09.532 INFO:teuthology.orchestra.run.smithi038.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-21T03:40:09.533 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T03:40:09.533 DEBUG:teuthology.orchestra.run.smithi038:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-21T03:40:09.588 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-21T03:40:09.588 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:40:09.588 DEBUG:teuthology.orchestra.run.smithi038:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T03:40:09.685 INFO:teuthology.orchestra.run.smithi038.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T03:40:09.703 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T03:40:09.704 INFO:teuthology.orchestra.run.smithi038.stdout:invalid new device string format 2024-09-21T03:40:09.704 INFO:teuthology.orchestra.run.smithi038.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T03:40:09.704 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T03:40:09.704 INFO:teuthology.orchestra.run.smithi038.stdout:invalid new device string format 2024-09-21T03:40:09.704 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: resolved to: "172.21.15.38:3300,172.21.15.195:3300,172.21.15.195:3301" 2024-09-21T03:40:09.705 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T03:40:09.705 INFO:teuthology.orchestra.run.smithi038.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T03:40:09.705 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T03:40:09.705 INFO:teuthology.orchestra.run.smithi038.stdout:invalid new device string format 2024-09-21T03:40:09.705 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: resolved to: "172.21.15.38:3300,172.21.15.195:3300,172.21.15.195:3301" 2024-09-21T03:40:09.705 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T03:40:09.705 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: resolved to: "172.21.15.38:6789,172.21.15.195:6789,172.21.15.195:6790" 2024-09-21T03:40:09.705 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.38:6789,172.21.15.195:6789,172.21.15.195:6790:/ 2024-09-21T03:40:09.705 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=9d50d66f-656b-4fe2-8e3b-fe53b79c4d43" will pass to kernel 2024-09-21T03:40:09.706 INFO:teuthology.orchestra.run.smithi038.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-21T03:40:09.706 INFO:teuthology.orchestra.run.smithi038.stdout: You just mounted a file system that supports labels which does not 2024-09-21T03:40:09.706 INFO:teuthology.orchestra.run.smithi038.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-21T03:40:09.706 INFO:teuthology.orchestra.run.smithi038.stdout: applications will generate AVC messages and not be allowed access to 2024-09-21T03:40:09.706 INFO:teuthology.orchestra.run.smithi038.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-21T03:40:09.707 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T03:40:09.707 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:40:09.707 DEBUG:teuthology.orchestra.run.smithi038:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-21T03:40:09.735 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T03:40:09.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:09 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T03:40:09.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:40:09.844 INFO:teuthology.orchestra.run.smithi038.stdout:client6673 2024-09-21T03:40:09.844 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-21T03:40:09.844 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:40:09.865 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:40:09.865 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir subdir) 2024-09-21T03:40:09.929 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-21T03:40:09.929 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:40:09.988 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:40:09.989 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=subdir/sixmegs bs=1M conv=fdatasync count=6 seek=0) 2024-09-21T03:40:10.078 INFO:teuthology.orchestra.run.smithi038.stderr:6+0 records in 2024-09-21T03:40:10.078 INFO:teuthology.orchestra.run.smithi038.stderr:6+0 records out 2024-09-21T03:40:10.078 INFO:teuthology.orchestra.run.smithi038.stderr:6291456 bytes (6.3 MB, 6.0 MiB) copied, 0.0299922 s, 210 MB/s 2024-09-21T03:40:10.079 DEBUG:teuthology.orchestra.run.smithi038:> stdin-killer -- python3 -c ' 2024-09-21T03:40:10.079 DEBUG:teuthology.orchestra.run.smithi038:> import os 2024-09-21T03:40:10.079 DEBUG:teuthology.orchestra.run.smithi038:> import stat 2024-09-21T03:40:10.079 DEBUG:teuthology.orchestra.run.smithi038:> import json 2024-09-21T03:40:10.079 DEBUG:teuthology.orchestra.run.smithi038:> import sys 2024-09-21T03:40:10.079 DEBUG:teuthology.orchestra.run.smithi038:> 2024-09-21T03:40:10.079 DEBUG:teuthology.orchestra.run.smithi038:> try: 2024-09-21T03:40:10.079 DEBUG:teuthology.orchestra.run.smithi038:> s = os.stat('"'"'/home/ubuntu/cephtest/mnt.0/subdir/sixmegs'"'"') 2024-09-21T03:40:10.080 DEBUG:teuthology.orchestra.run.smithi038:> except OSError as e: 2024-09-21T03:40:10.080 DEBUG:teuthology.orchestra.run.smithi038:> sys.exit(e.errno) 2024-09-21T03:40:10.080 DEBUG:teuthology.orchestra.run.smithi038:> 2024-09-21T03:40:10.080 DEBUG:teuthology.orchestra.run.smithi038:> attrs = ["st_mode", "st_ino", "st_dev", "st_nlink", "st_uid", "st_gid", "st_size", "st_atime", "st_mtime", "st_ctime"] 2024-09-21T03:40:10.080 DEBUG:teuthology.orchestra.run.smithi038:> print(json.dumps( 2024-09-21T03:40:10.080 DEBUG:teuthology.orchestra.run.smithi038:> dict([(a, getattr(s, a)) for a in attrs]), 2024-09-21T03:40:10.080 DEBUG:teuthology.orchestra.run.smithi038:> indent=2)) 2024-09-21T03:40:10.080 DEBUG:teuthology.orchestra.run.smithi038:> ' 2024-09-21T03:40:10.149 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-09-21T03:40:10.149 INFO:teuthology.orchestra.run.smithi038.stdout: "st_mode": 33188, 2024-09-21T03:40:10.149 INFO:teuthology.orchestra.run.smithi038.stdout: "st_ino": 1099511627777, 2024-09-21T03:40:10.149 INFO:teuthology.orchestra.run.smithi038.stdout: "st_dev": 41, 2024-09-21T03:40:10.149 INFO:teuthology.orchestra.run.smithi038.stdout: "st_nlink": 1, 2024-09-21T03:40:10.149 INFO:teuthology.orchestra.run.smithi038.stdout: "st_uid": 1000, 2024-09-21T03:40:10.149 INFO:teuthology.orchestra.run.smithi038.stdout: "st_gid": 1333, 2024-09-21T03:40:10.149 INFO:teuthology.orchestra.run.smithi038.stdout: "st_size": 6291456, 2024-09-21T03:40:10.150 INFO:teuthology.orchestra.run.smithi038.stdout: "st_atime": 1726890010.0448925, 2024-09-21T03:40:10.150 INFO:teuthology.orchestra.run.smithi038.stdout: "st_mtime": 1726890010.0628922, 2024-09-21T03:40:10.150 INFO:teuthology.orchestra.run.smithi038.stdout: "st_ctime": 1726890010.0628922 2024-09-21T03:40:10.150 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-09-21T03:40:10.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:40:10.157 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-21T03:40:10.157 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:40:10.214 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-21T03:40:10.214 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:40:10.271 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T03:40:10.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:10.426+0000 7f0fbd038640 1 -- 172.21.15.38:0/712826368 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0fa4005250 msgr2=0x7f0fa400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:10.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:10.426+0000 7f0fbd038640 1 --2- 172.21.15.38:0/712826368 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0fa4005250 0x7f0fa400b9c0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f0fb0001690 tx=0x7f0fa4005830 comp rx=0 tx=0).stop 2024-09-21T03:40:10.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:10.427+0000 7f0fbd038640 1 -- 172.21.15.38:0/712826368 shutdown_connections 2024-09-21T03:40:10.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:10.427+0000 7f0fbd038640 1 --2- 172.21.15.38:0/712826368 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0fa4005250 0x7f0fa400b9c0 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:10.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:10.427+0000 7f0fbd038640 1 -- 172.21.15.38:0/712826368 >> 172.21.15.38:0/712826368 conn(0x7f0fb80b1b90 msgr2=0x7f0fb80b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:10.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:10.427+0000 7f0fbd038640 1 -- 172.21.15.38:0/712826368 shutdown_connections 2024-09-21T03:40:10.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:10.427+0000 7f0fbd038640 1 -- 172.21.15.38:0/712826368 wait complete. 2024-09-21T03:40:10.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:10.429+0000 7f0fbd038640 1 Processor -- start 2024-09-21T03:40:10.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:10.429+0000 7f0fbd038640 1 -- start start 2024-09-21T03:40:10.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:10.430+0000 7f0fbd038640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0fa4005250 0x7f0fb8156090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:10.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:10.430+0000 7f0fbd038640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0fb80b2ed0 0x7f0fb80b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:10.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:10.430+0000 7f0fbd038640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0fb80cbba0 0x7f0fb80cbfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:10.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:10.430+0000 7f0fbd038640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f0fb80749b0 con 0x7f0fa4005250 2024-09-21T03:40:10.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:10.430+0000 7f0fbd038640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f0fb8074830 con 0x7f0fb80b2ed0 2024-09-21T03:40:10.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:10.430+0000 7f0fbd038640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f0fb806ddf0 con 0x7f0fb80cbba0 2024-09-21T03:40:10.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:10.430+0000 7f0fb6d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0fa4005250 0x7f0fb8156090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:10.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:10.430+0000 7f0fb6d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0fa4005250 0x7f0fb8156090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:55726/0 (socket says 172.21.15.38:55726) 2024-09-21T03:40:10.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:10.430+0000 7f0fb6d9d640 1 -- 172.21.15.38:0/2517963382 learned_addr learned my addr 172.21.15.38:0/2517963382 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:10.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:10.430+0000 7f0fb659c640 1 --2- 172.21.15.38:0/2517963382 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0fb80b2ed0 0x7f0fb80b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:10.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:10.430+0000 7f0fb759e640 1 --2- 172.21.15.38:0/2517963382 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0fb80cbba0 0x7f0fb80cbfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:10.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:10.431+0000 7f0fb6d9d640 1 -- 172.21.15.38:0/2517963382 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0fb80cbba0 msgr2=0x7f0fb80cbfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:10.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:10.431+0000 7f0fb6d9d640 1 --2- 172.21.15.38:0/2517963382 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0fb80cbba0 0x7f0fb80cbfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:10.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:10.431+0000 7f0fb6d9d640 1 -- 172.21.15.38:0/2517963382 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0fb80b2ed0 msgr2=0x7f0fb80b4d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:10.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:10.431+0000 7f0fb6d9d640 1 --2- 172.21.15.38:0/2517963382 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0fb80b2ed0 0x7f0fb80b4d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:10.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:10.431+0000 7f0fb6d9d640 1 -- 172.21.15.38:0/2517963382 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fb80cc4f0 con 0x7f0fa4005250 2024-09-21T03:40:10.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:10.431+0000 7f0fb6d9d640 1 --2- 172.21.15.38:0/2517963382 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0fa4005250 0x7f0fb8156090 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f0fa8011c70 tx=0x7f0fa800f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:10.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:10.431+0000 7f0faf7fe640 1 -- 172.21.15.38:0/2517963382 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fa8003e40 con 0x7f0fa4005250 2024-09-21T03:40:10.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:10.431+0000 7f0faf7fe640 1 -- 172.21.15.38:0/2517963382 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0fa8004b20 con 0x7f0fa4005250 2024-09-21T03:40:10.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:10.431+0000 7f0fbd038640 1 -- 172.21.15.38:0/2517963382 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fb81f8560 con 0x7f0fa4005250 2024-09-21T03:40:10.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:10.431+0000 7f0faf7fe640 1 -- 172.21.15.38:0/2517963382 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fa8004050 con 0x7f0fa4005250 2024-09-21T03:40:10.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:10.431+0000 7f0fbd038640 1 -- 172.21.15.38:0/2517963382 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fb81f8d40 con 0x7f0fa4005250 2024-09-21T03:40:10.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:10.432+0000 7f0fbd038640 1 -- 172.21.15.38:0/2517963382 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0fb806e350 con 0x7f0fa4005250 2024-09-21T03:40:10.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:10.433+0000 7f0faf7fe640 1 -- 172.21.15.38:0/2517963382 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f0fa80041f0 con 0x7f0fa4005250 2024-09-21T03:40:10.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:10.433+0000 7f0faf7fe640 1 --2- 172.21.15.38:0/2517963382 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0f98078bc0 0x7f0f9807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:10.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:10.433+0000 7f0faf7fe640 1 -- 172.21.15.38:0/2517963382 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6101+0+0 (secure 0 0 0) 0x7f0fa809c0d0 con 0x7f0fa4005250 2024-09-21T03:40:10.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:10.434+0000 7f0fb659c640 1 --2- 172.21.15.38:0/2517963382 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0f98078bc0 0x7f0f9807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:10.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:10.434+0000 7f0fb659c640 1 --2- 172.21.15.38:0/2517963382 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0f98078bc0 0x7f0f9807b0d0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f0fa0001010 tx=0x7f0fa00067a0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:10.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:10.438+0000 7f0faf7fe640 1 -- 172.21.15.38:0/2517963382 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f0fb80cc4f0 con 0x7f0fa4005250 2024-09-21T03:40:10.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:10.637+0000 7f0fbd038640 1 -- 172.21.15.38:0/2517963382 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f0fb80716d0 con 0x7f0fa4005250 2024-09-21T03:40:10.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:10.637+0000 7f0faf7fe640 1 -- 172.21.15.38:0/2517963382 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v94) ==== 80+0+0 (secure 0 0 0) 0x7f0fa8063cb0 con 0x7f0fa4005250 2024-09-21T03:40:10.640 INFO:teuthology.orchestra.run.smithi038.stderr:listed 0 entries 2024-09-21T03:40:10.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:10.639+0000 7f0fbd038640 1 -- 172.21.15.38:0/2517963382 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0f98078bc0 msgr2=0x7f0f9807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:10.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:10.639+0000 7f0fbd038640 1 --2- 172.21.15.38:0/2517963382 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0f98078bc0 0x7f0f9807b0d0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f0fa0001010 tx=0x7f0fa00067a0 comp rx=0 tx=0).stop 2024-09-21T03:40:10.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:10.639+0000 7f0fbd038640 1 -- 172.21.15.38:0/2517963382 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0fa4005250 msgr2=0x7f0fb8156090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:10.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:10.639+0000 7f0fbd038640 1 --2- 172.21.15.38:0/2517963382 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0fa4005250 0x7f0fb8156090 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f0fa8011c70 tx=0x7f0fa800f790 comp rx=0 tx=0).stop 2024-09-21T03:40:10.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:10.639+0000 7f0fbd038640 1 -- 172.21.15.38:0/2517963382 shutdown_connections 2024-09-21T03:40:10.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:10.639+0000 7f0fbd038640 1 --2- 172.21.15.38:0/2517963382 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0f98078bc0 0x7f0f9807b0d0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:10.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:10.639+0000 7f0fbd038640 1 --2- 172.21.15.38:0/2517963382 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0fb80cbba0 0x7f0fb80cbfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:10.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:10.639+0000 7f0fbd038640 1 --2- 172.21.15.38:0/2517963382 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0fb80b2ed0 0x7f0fb80b4d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:10.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:10.639+0000 7f0fbd038640 1 --2- 172.21.15.38:0/2517963382 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0fa4005250 0x7f0fb8156090 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:10.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:10.639+0000 7f0fbd038640 1 -- 172.21.15.38:0/2517963382 >> 172.21.15.38:0/2517963382 conn(0x7f0fb80b1b90 msgr2=0x7f0fb80b1400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:10.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:10.639+0000 7f0fbd038640 1 -- 172.21.15.38:0/2517963382 shutdown_connections 2024-09-21T03:40:10.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:10.639+0000 7f0fbd038640 1 -- 172.21.15.38:0/2517963382 wait complete. 2024-09-21T03:40:10.653 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-21T03:40:10.654 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:40:10.654 DEBUG:teuthology.orchestra.run.smithi038:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-21T03:40:10.709 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi038.front.sepia.ceph.com 2024-09-21T03:40:10.709 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:40:10.709 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T03:40:10.729 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:40:10.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:10.884+0000 7fa77792b640 1 -- 172.21.15.38:0/1799705685 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa7700d19f0 msgr2=0x7fa77015cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:10.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:10.884+0000 7fa77792b640 1 --2- 172.21.15.38:0/1799705685 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa7700d19f0 0x7fa77015cf20 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7fa7700b3cf0 tx=0x7fa76402a5f0 comp rx=0 tx=0).stop 2024-09-21T03:40:10.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:10.884+0000 7fa77792b640 1 -- 172.21.15.38:0/1799705685 shutdown_connections 2024-09-21T03:40:10.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:10.884+0000 7fa77792b640 1 --2- 172.21.15.38:0/1799705685 >> v2:172.21.15.195:3300/0 conn(0x7fa7700b6250 0x7fa7700b6660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:10.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:10.884+0000 7fa77792b640 1 --2- 172.21.15.38:0/1799705685 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa7700d19f0 0x7fa77015cf20 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:10.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:10.884+0000 7fa77792b640 1 -- 172.21.15.38:0/1799705685 >> 172.21.15.38:0/1799705685 conn(0x7fa7700d2350 msgr2=0x7fa7700d2760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:10.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:10.884+0000 7fa77792b640 1 -- 172.21.15.38:0/1799705685 shutdown_connections 2024-09-21T03:40:10.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:10.884+0000 7fa77792b640 1 -- 172.21.15.38:0/1799705685 wait complete. 2024-09-21T03:40:10.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:10.886+0000 7fa77792b640 1 Processor -- start 2024-09-21T03:40:10.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:10.886+0000 7fa77792b640 1 -- start start 2024-09-21T03:40:10.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:10.887+0000 7fa77792b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa7700b6250 0x7fa77014c180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:10.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:10.887+0000 7fa77792b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa7700d19f0 0x7fa77013fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:10.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:10.887+0000 7fa77792b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa77015cb10 0x7fa7701f6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:10.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:10.887+0000 7fa77792b640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa7700b6d20 con 0x7fa77015cb10 2024-09-21T03:40:10.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:10.887+0000 7fa77792b640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fa7700b6ba0 con 0x7fa7700b6250 2024-09-21T03:40:10.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:10.887+0000 7fa77792b640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fa7700b6ea0 con 0x7fa7700d19f0 2024-09-21T03:40:10.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:10.887+0000 7fa775ec8640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa77015cb10 0x7fa7701f6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:10.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:10.887+0000 7fa775ec8640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa77015cb10 0x7fa7701f6ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:55746/0 (socket says 172.21.15.38:55746) 2024-09-21T03:40:10.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:10.887+0000 7fa775ec8640 1 -- 172.21.15.38:0/1463339860 learned_addr learned my addr 172.21.15.38:0/1463339860 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:10.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:10.887+0000 7fa774ec6640 1 --2- 172.21.15.38:0/1463339860 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa7700d19f0 0x7fa77013fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:10.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:10.887+0000 7fa7756c7640 1 --2- 172.21.15.38:0/1463339860 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa7700b6250 0x7fa77014c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:10.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:10.888+0000 7fa775ec8640 1 -- 172.21.15.38:0/1463339860 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa7700d19f0 msgr2=0x7fa77013fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:10.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:10.888+0000 7fa775ec8640 1 --2- 172.21.15.38:0/1463339860 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa7700d19f0 0x7fa77013fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:10.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:10.888+0000 7fa775ec8640 1 -- 172.21.15.38:0/1463339860 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa7700b6250 msgr2=0x7fa77014c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:10.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:10.888+0000 7fa775ec8640 1 --2- 172.21.15.38:0/1463339860 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa7700b6250 0x7fa77014c180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:10.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:10.888+0000 7fa775ec8640 1 -- 172.21.15.38:0/1463339860 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa77014c6c0 con 0x7fa77015cb10 2024-09-21T03:40:10.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:10.888+0000 7fa775ec8640 1 --2- 172.21.15.38:0/1463339860 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa77015cb10 0x7fa7701f6ca0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7fa768005d00 tx=0x7fa76800dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:10.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:10.888+0000 7fa76dffb640 1 -- 172.21.15.38:0/1463339860 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa768017070 con 0x7fa77015cb10 2024-09-21T03:40:10.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:10.888+0000 7fa76dffb640 1 -- 172.21.15.38:0/1463339860 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa768005330 con 0x7fa77015cb10 2024-09-21T03:40:10.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:10.888+0000 7fa77792b640 1 -- 172.21.15.38:0/1463339860 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa77014a820 con 0x7fa77015cb10 2024-09-21T03:40:10.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:10.888+0000 7fa76dffb640 1 -- 172.21.15.38:0/1463339860 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7680047a0 con 0x7fa77015cb10 2024-09-21T03:40:10.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:10.888+0000 7fa77792b640 1 -- 172.21.15.38:0/1463339860 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa77014ade0 con 0x7fa77015cb10 2024-09-21T03:40:10.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:10.889+0000 7fa77792b640 1 -- 172.21.15.38:0/1463339860 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7700b7350 con 0x7fa77015cb10 2024-09-21T03:40:10.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:10.890+0000 7fa76dffb640 1 -- 172.21.15.38:0/1463339860 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa768004940 con 0x7fa77015cb10 2024-09-21T03:40:10.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:10.891+0000 7fa76dffb640 1 --2- 172.21.15.38:0/1463339860 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa75c078b90 0x7fa75c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:10.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:10.891+0000 7fa76dffb640 1 -- 172.21.15.38:0/1463339860 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6101+0+0 (secure 0 0 0) 0x7fa76809af70 con 0x7fa77015cb10 2024-09-21T03:40:10.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:10.891+0000 7fa7756c7640 1 --2- 172.21.15.38:0/1463339860 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa75c078b90 0x7fa75c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:10.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:10.892+0000 7fa7756c7640 1 --2- 172.21.15.38:0/1463339860 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa75c078b90 0x7fa75c07b0a0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fa760001e20 tx=0x7fa760009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:10.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:10.896+0000 7fa76dffb640 1 -- 172.21.15.38:0/1463339860 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fa768062b30 con 0x7fa77015cb10 2024-09-21T03:40:11.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.125+0000 7fa77792b640 1 -- 172.21.15.38:0/1463339860 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa7700b56b0 con 0x7fa77015cb10 2024-09-21T03:40:11.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.125+0000 7fa76dffb640 1 -- 172.21.15.38:0/1463339860 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) ==== 94+0+5222 (secure 0 0 0) 0x7fa768067560 con 0x7fa77015cb10 2024-09-21T03:40:11.128 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:40:11.128 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":83,"btime":"2024-09-21T03:40:05:917318+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6086,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/965972663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":965972663},{"type":"v1","addr":"172.21.15.195:6836","nonce":965972663}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":7020,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/751618334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":751618334},{"type":"v1","addr":"172.21.15.38:6833","nonce":751618334}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":7023,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/2934569248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":2934569248},{"type":"v1","addr":"172.21.15.38:6835","nonce":2934569248}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":83,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:40:02.319282+0000","modified":"2024-09-21T03:40:05.304556+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7035},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7035":{"gid":7035,"name":"b","rank":0,"incarnation":79,"state":"up:active","state_seq":3,"addr":"172.21.15.195:6837/712370966","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":712370966},{"type":"v1","addr":"172.21.15.195:6837","nonce":712370966}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7035,"qdb_cluster":[7035]},"id":6}]} 2024-09-21T03:40:11.129 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 83 2024-09-21T03:40:11.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.128+0000 7fa77792b640 1 -- 172.21.15.38:0/1463339860 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa75c078b90 msgr2=0x7fa75c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:11.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.128+0000 7fa77792b640 1 --2- 172.21.15.38:0/1463339860 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa75c078b90 0x7fa75c07b0a0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fa760001e20 tx=0x7fa760009040 comp rx=0 tx=0).stop 2024-09-21T03:40:11.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.128+0000 7fa77792b640 1 -- 172.21.15.38:0/1463339860 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa77015cb10 msgr2=0x7fa7701f6ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:11.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.128+0000 7fa77792b640 1 --2- 172.21.15.38:0/1463339860 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa77015cb10 0x7fa7701f6ca0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7fa768005d00 tx=0x7fa76800dae0 comp rx=0 tx=0).stop 2024-09-21T03:40:11.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.128+0000 7fa77792b640 1 -- 172.21.15.38:0/1463339860 shutdown_connections 2024-09-21T03:40:11.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.128+0000 7fa77792b640 1 --2- 172.21.15.38:0/1463339860 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa77015cb10 0x7fa7701f6ca0 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:11.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.128+0000 7fa77792b640 1 --2- 172.21.15.38:0/1463339860 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa75c078b90 0x7fa75c07b0a0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:11.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.128+0000 7fa77792b640 1 --2- 172.21.15.38:0/1463339860 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa7700d19f0 0x7fa77013fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:11.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.128+0000 7fa77792b640 1 --2- 172.21.15.38:0/1463339860 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa7700b6250 0x7fa77014c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:11.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.128+0000 7fa77792b640 1 -- 172.21.15.38:0/1463339860 >> 172.21.15.38:0/1463339860 conn(0x7fa7700d2350 msgr2=0x7fa77015e190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:11.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.128+0000 7fa77792b640 1 -- 172.21.15.38:0/1463339860 shutdown_connections 2024-09-21T03:40:11.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.128+0000 7fa77792b640 1 -- 172.21.15.38:0/1463339860 wait complete. 2024-09-21T03:40:11.142 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json flush journal 2024-09-21T03:40:11.281 INFO:teuthology.orchestra.run.smithi195.stdout:{"message":"","return_code":0} 2024-09-21T03:40:11.281 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-09-21T03:40:11.282 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-21T03:40:11.399 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.396+0000 7fc273ed5640 1 -- 172.21.15.38:0/642492400 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc26c15bfc0 msgr2=0x7fc26c15f630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:11.399 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.396+0000 7fc273ed5640 1 --2- 172.21.15.38:0/642492400 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc26c15bfc0 0x7fc26c15f630 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7fc26c075150 tx=0x7fc26402a8b0 comp rx=0 tx=0).stop 2024-09-21T03:40:11.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.396+0000 7fc273ed5640 1 -- 172.21.15.38:0/642492400 shutdown_connections 2024-09-21T03:40:11.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.396+0000 7fc273ed5640 1 --2- 172.21.15.38:0/642492400 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc26c15bfc0 0x7fc26c15f630 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:11.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.396+0000 7fc273ed5640 1 --2- 172.21.15.38:0/642492400 >> v2:172.21.15.195:3300/0 conn(0x7fc26c0b5260 0x7fc26c15ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:11.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.396+0000 7fc273ed5640 1 -- 172.21.15.38:0/642492400 >> 172.21.15.38:0/642492400 conn(0x7fc26c0cdca0 msgr2=0x7fc26c0ce0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:11.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.396+0000 7fc273ed5640 1 -- 172.21.15.38:0/642492400 shutdown_connections 2024-09-21T03:40:11.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.396+0000 7fc273ed5640 1 -- 172.21.15.38:0/642492400 wait complete. 2024-09-21T03:40:11.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.398+0000 7fc273ed5640 1 Processor -- start 2024-09-21T03:40:11.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.398+0000 7fc273ed5640 1 -- start start 2024-09-21T03:40:11.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.399+0000 7fc273ed5640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc26c06def0 0x7fc26c1f8a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:11.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.399+0000 7fc273ed5640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc26c0b5260 0x7fc26c0ca8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:11.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.399+0000 7fc273ed5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc26c15bfc0 0x7fc26c1e91e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:11.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.399+0000 7fc273ed5640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fc26c15ff40 con 0x7fc26c15bfc0 2024-09-21T03:40:11.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.399+0000 7fc273ed5640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fc26c15fdc0 con 0x7fc26c06def0 2024-09-21T03:40:11.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.399+0000 7fc273ed5640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fc26c071300 con 0x7fc26c0b5260 2024-09-21T03:40:11.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.400+0000 7fc272472640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc26c15bfc0 0x7fc26c1e91e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:11.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.400+0000 7fc272472640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc26c15bfc0 0x7fc26c1e91e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:55768/0 (socket says 172.21.15.38:55768) 2024-09-21T03:40:11.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.400+0000 7fc272472640 1 -- 172.21.15.38:0/856093330 learned_addr learned my addr 172.21.15.38:0/856093330 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:11.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.400+0000 7fc271470640 1 --2- 172.21.15.38:0/856093330 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc26c0b5260 0x7fc26c0ca8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:11.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.400+0000 7fc271c71640 1 --2- 172.21.15.38:0/856093330 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc26c06def0 0x7fc26c1f8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:11.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.400+0000 7fc272472640 1 -- 172.21.15.38:0/856093330 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc26c0b5260 msgr2=0x7fc26c0ca8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:11.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.400+0000 7fc272472640 1 --2- 172.21.15.38:0/856093330 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc26c0b5260 0x7fc26c0ca8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:11.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.400+0000 7fc272472640 1 -- 172.21.15.38:0/856093330 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc26c06def0 msgr2=0x7fc26c1f8a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:11.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.400+0000 7fc272472640 1 --2- 172.21.15.38:0/856093330 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc26c06def0 0x7fc26c1f8a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:11.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.400+0000 7fc272472640 1 -- 172.21.15.38:0/856093330 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc26c0cc390 con 0x7fc26c15bfc0 2024-09-21T03:40:11.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.400+0000 7fc272472640 1 --2- 172.21.15.38:0/856093330 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc26c15bfc0 0x7fc26c1e91e0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7fc26402aea0 tx=0x7fc2640039f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:11.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.400+0000 7fc2627fc640 1 -- 172.21.15.38:0/856093330 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc264031990 con 0x7fc26c15bfc0 2024-09-21T03:40:11.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.400+0000 7fc2627fc640 1 -- 172.21.15.38:0/856093330 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc264031c20 con 0x7fc26c15bfc0 2024-09-21T03:40:11.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.400+0000 7fc273ed5640 1 -- 172.21.15.38:0/856093330 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc26c1f92b0 con 0x7fc26c15bfc0 2024-09-21T03:40:11.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.400+0000 7fc2627fc640 1 -- 172.21.15.38:0/856093330 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc264033330 con 0x7fc26c15bfc0 2024-09-21T03:40:11.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.400+0000 7fc273ed5640 1 -- 172.21.15.38:0/856093330 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc26c09ad80 con 0x7fc26c15bfc0 2024-09-21T03:40:11.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.401+0000 7fc273ed5640 1 -- 172.21.15.38:0/856093330 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc26c09afe0 con 0x7fc26c15bfc0 2024-09-21T03:40:11.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.402+0000 7fc2627fc640 1 -- 172.21.15.38:0/856093330 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc2640334d0 con 0x7fc26c15bfc0 2024-09-21T03:40:11.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.403+0000 7fc2627fc640 1 --2- 172.21.15.38:0/856093330 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc250078b90 0x7fc25007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:11.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.403+0000 7fc2627fc640 1 -- 172.21.15.38:0/856093330 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6101+0+0 (secure 0 0 0) 0x7fc2640bc6f0 con 0x7fc26c15bfc0 2024-09-21T03:40:11.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.403+0000 7fc271c71640 1 --2- 172.21.15.38:0/856093330 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc250078b90 0x7fc25007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:11.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.404+0000 7fc271c71640 1 --2- 172.21.15.38:0/856093330 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc250078b90 0x7fc25007b0a0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fc26c15efd0 tx=0x7fc268007e90 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:11.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.407+0000 7fc2627fc640 1 -- 172.21.15.38:0/856093330 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fc264052080 con 0x7fc26c15bfc0 2024-09-21T03:40:11.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.631+0000 7fc273ed5640 1 -- 172.21.15.38:0/856093330 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fc26c15ba80 con 0x7fc26c15bfc0 2024-09-21T03:40:11.634 INFO:tasks.ceph.mon.a.smithi038.stderr:2024-09-21T03:40:11.632+0000 7f3f45713640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T03:40:11.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.634+0000 7fc2627fc640 1 -- 172.21.15.38:0/856093330 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v84) ==== 161+0+0 (secure 0 0 0) 0x7fc26403f030 con 0x7fc26c15bfc0 2024-09-21T03:40:11.636 INFO:teuthology.orchestra.run.smithi038.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-21T03:40:11.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.635+0000 7fc273ed5640 1 -- 172.21.15.38:0/856093330 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc250078b90 msgr2=0x7fc25007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:11.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.635+0000 7fc273ed5640 1 --2- 172.21.15.38:0/856093330 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc250078b90 0x7fc25007b0a0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fc26c15efd0 tx=0x7fc268007e90 comp rx=0 tx=0).stop 2024-09-21T03:40:11.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.635+0000 7fc273ed5640 1 -- 172.21.15.38:0/856093330 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc26c15bfc0 msgr2=0x7fc26c1e91e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:11.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.635+0000 7fc273ed5640 1 --2- 172.21.15.38:0/856093330 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc26c15bfc0 0x7fc26c1e91e0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7fc26402aea0 tx=0x7fc2640039f0 comp rx=0 tx=0).stop 2024-09-21T03:40:11.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.635+0000 7fc273ed5640 1 -- 172.21.15.38:0/856093330 shutdown_connections 2024-09-21T03:40:11.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.635+0000 7fc273ed5640 1 --2- 172.21.15.38:0/856093330 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc26c15bfc0 0x7fc26c1e91e0 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:11.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.635+0000 7fc273ed5640 1 --2- 172.21.15.38:0/856093330 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc250078b90 0x7fc25007b0a0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:11.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.635+0000 7fc273ed5640 1 --2- 172.21.15.38:0/856093330 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc26c0b5260 0x7fc26c0ca8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:11.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.635+0000 7fc273ed5640 1 --2- 172.21.15.38:0/856093330 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc26c06def0 0x7fc26c1f8a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:11.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.635+0000 7fc273ed5640 1 -- 172.21.15.38:0/856093330 >> 172.21.15.38:0/856093330 conn(0x7fc26c0cdca0 msgr2=0x7fc26c068d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:11.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.635+0000 7fc273ed5640 1 -- 172.21.15.38:0/856093330 shutdown_connections 2024-09-21T03:40:11.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.635+0000 7fc273ed5640 1 -- 172.21.15.38:0/856093330 wait complete. 2024-09-21T03:40:11.640 INFO:tasks.ceph.mds.b.smithi195.stderr: -6254> 2024-09-21T03:39:58.260+0000 7f4ab5d6f640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2024-09-21T03:40:11.641 INFO:tasks.ceph.mds.b.smithi195.stderr: -4995> 2024-09-21T03:40:04.865+0000 7f4aafd63640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T03:40:11.651 DEBUG:teuthology.orchestra.run.smithi038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata export - 2024-09-21T03:40:11.695 INFO:tasks.ceph.mds.b.smithi195.stdout:starting mds.b at 2024-09-21T03:40:11.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.703+0000 7f1909119f40 1 Processor -- start 2024-09-21T03:40:11.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.703+0000 7f1909119f40 1 -- start start 2024-09-21T03:40:11.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.704+0000 7f1909119f40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556215b08240 0x556215bc2910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:11.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.704+0000 7f1909119f40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x556215b4f090 0x556215b03660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:11.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.704+0000 7f1909119f40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556215bc4520 0x556215b03ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:11.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.704+0000 7f1909119f40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x556215bdbaa0 con 0x556215b08240 2024-09-21T03:40:11.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.704+0000 7f1909119f40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x556215a55d60 con 0x556215bc4520 2024-09-21T03:40:11.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.704+0000 7f1909119f40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x556215b369b0 con 0x556215b4f090 2024-09-21T03:40:11.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.704+0000 7f1902bd3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556215b08240 0x556215bc2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:11.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.704+0000 7f1902bd3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556215b08240 0x556215bc2910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:55784/0 (socket says 172.21.15.38:55784) 2024-09-21T03:40:11.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.704+0000 7f1902bd3640 1 -- 172.21.15.38:0/2737255754 learned_addr learned my addr 172.21.15.38:0/2737255754 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:11.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.704+0000 7f19033d4640 1 --2- 172.21.15.38:0/2737255754 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556215bc4520 0x556215b03ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:11.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.704+0000 7f19023d2640 1 --2- 172.21.15.38:0/2737255754 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x556215b4f090 0x556215b03660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:11.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.704+0000 7f1902bd3640 1 -- 172.21.15.38:0/2737255754 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x556215b4f090 msgr2=0x556215b03660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:11.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.704+0000 7f1902bd3640 1 --2- 172.21.15.38:0/2737255754 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x556215b4f090 0x556215b03660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:11.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.704+0000 7f1902bd3640 1 -- 172.21.15.38:0/2737255754 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556215bc4520 msgr2=0x556215b03ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:11.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.704+0000 7f1902bd3640 1 --2- 172.21.15.38:0/2737255754 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556215bc4520 0x556215b03ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:11.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.704+0000 7f1902bd3640 1 -- 172.21.15.38:0/2737255754 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556215bd66e0 con 0x556215b08240 2024-09-21T03:40:11.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.704+0000 7f1902bd3640 1 --2- 172.21.15.38:0/2737255754 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556215b08240 0x556215bc2910 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f18f400b630 tx=0x7f18f400baf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9500ca13f9a50a2a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:11.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.705+0000 7f19013d0640 1 -- 172.21.15.38:0/2737255754 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18f400a940 con 0x556215b08240 2024-09-21T03:40:11.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.705+0000 7f19013d0640 1 -- 172.21.15.38:0/2737255754 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f18f400aae0 con 0x556215b08240 2024-09-21T03:40:11.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.705+0000 7f19013d0640 1 -- 172.21.15.38:0/2737255754 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18f4008450 con 0x556215b08240 2024-09-21T03:40:11.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.705+0000 7f1909119f40 1 -- 172.21.15.38:0/2737255754 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556215b08240 msgr2=0x556215bc2910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:11.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.705+0000 7f1909119f40 1 --2- 172.21.15.38:0/2737255754 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556215b08240 0x556215bc2910 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f18f400b630 tx=0x7f18f400baf0 comp rx=0 tx=0).stop 2024-09-21T03:40:11.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.705+0000 7f1909119f40 1 -- 172.21.15.38:0/2737255754 shutdown_connections 2024-09-21T03:40:11.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.705+0000 7f1909119f40 1 --2- 172.21.15.38:0/2737255754 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556215bc4520 0x556215b03ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:11.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.705+0000 7f1909119f40 1 --2- 172.21.15.38:0/2737255754 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x556215b4f090 0x556215b03660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:11.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.705+0000 7f1909119f40 1 --2- 172.21.15.38:0/2737255754 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556215b08240 0x556215bc2910 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:11.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.705+0000 7f1909119f40 1 -- 172.21.15.38:0/2737255754 >> 172.21.15.38:0/2737255754 conn(0x556215a26ac0 msgr2=0x556215bda070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:11.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.705+0000 7f1909119f40 1 -- 172.21.15.38:0/2737255754 shutdown_connections 2024-09-21T03:40:11.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.705+0000 7f1909119f40 1 -- 172.21.15.38:0/2737255754 wait complete. 2024-09-21T03:40:11.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.706+0000 7f1909119f40 1 Processor -- start 2024-09-21T03:40:11.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.706+0000 7f1909119f40 1 -- start start 2024-09-21T03:40:11.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.707+0000 7f1909119f40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556215b08240 0x556215b2a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:11.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.708+0000 7f1909119f40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x556215b4f090 0x556215b34c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:11.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.708+0000 7f1909119f40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556215bc4520 0x556215b351c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:11.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.708+0000 7f1909119f40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x556215bd6150 con 0x556215b08240 2024-09-21T03:40:11.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.708+0000 7f1909119f40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x556215b285d0 con 0x556215bc4520 2024-09-21T03:40:11.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.708+0000 7f1909119f40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x556215c77920 con 0x556215b4f090 2024-09-21T03:40:11.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.708+0000 7f1902bd3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556215b08240 0x556215b2a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:11.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.708+0000 7f1902bd3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556215b08240 0x556215b2a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:55794/0 (socket says 172.21.15.38:55794) 2024-09-21T03:40:11.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.708+0000 7f1902bd3640 1 -- 172.21.15.38:0/4193049548 learned_addr learned my addr 172.21.15.38:0/4193049548 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:11.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.708+0000 7f19023d2640 1 --2- 172.21.15.38:0/4193049548 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x556215b4f090 0x556215b34c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:11.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.708+0000 7f19033d4640 1 --2- 172.21.15.38:0/4193049548 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556215bc4520 0x556215b351c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:11.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.708+0000 7f1902bd3640 1 -- 172.21.15.38:0/4193049548 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x556215b4f090 msgr2=0x556215b34c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:11.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.708+0000 7f1902bd3640 1 --2- 172.21.15.38:0/4193049548 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x556215b4f090 0x556215b34c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:11.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.708+0000 7f1902bd3640 1 -- 172.21.15.38:0/4193049548 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556215bc4520 msgr2=0x556215b351c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:11.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.708+0000 7f1902bd3640 1 --2- 172.21.15.38:0/4193049548 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556215bc4520 0x556215b351c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:11.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.708+0000 7f1902bd3640 1 -- 172.21.15.38:0/4193049548 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556215c77b70 con 0x556215b08240 2024-09-21T03:40:11.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.708+0000 7f1902bd3640 1 --2- 172.21.15.38:0/4193049548 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556215b08240 0x556215b2a1b0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f18f4016650 tx=0x7f18f4016680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:11.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.708+0000 7f18ebfff640 1 -- 172.21.15.38:0/4193049548 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18f4020470 con 0x556215b08240 2024-09-21T03:40:11.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.708+0000 7f18ebfff640 1 -- 172.21.15.38:0/4193049548 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f18f4020610 con 0x556215b08240 2024-09-21T03:40:11.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.708+0000 7f18ebfff640 1 -- 172.21.15.38:0/4193049548 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18f4020860 con 0x556215b08240 2024-09-21T03:40:11.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.708+0000 7f1909119f40 1 -- 172.21.15.38:0/4193049548 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x556215c77e50 con 0x556215b08240 2024-09-21T03:40:11.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.708+0000 7f1909119f40 1 -- 172.21.15.38:0/4193049548 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556215c78410 con 0x556215b08240 2024-09-21T03:40:11.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.710+0000 7f18ebfff640 1 -- 172.21.15.38:0/4193049548 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f18f4020a00 con 0x556215b08240 2024-09-21T03:40:11.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.711+0000 7f18ebfff640 1 --2- 172.21.15.38:0/4193049548 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f18f0077570 0x7f18f0079a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:11.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.711+0000 7f18ebfff640 1 -- 172.21.15.38:0/4193049548 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6187+0+0 (secure 0 0 0) 0x7f18f409bc30 con 0x556215b08240 2024-09-21T03:40:11.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.711+0000 7f19023d2640 1 --2- 172.21.15.38:0/4193049548 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f18f0077570 0x7f18f0079a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:11.716 INFO:teuthology.orchestra.run.smithi038.stdout:���� 2024-09-21T03:40:11.717 INFO:teuthology.orchestra.run.smithi038.stdout: �� 2024-09-21T03:40:11.717 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:40:11.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.711+0000 7f1909119f40 1 --2- 172.21.15.38:0/4193049548 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x556215b27c30 0x556215c74560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:11.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.711+0000 7f1909119f40 1 -- 172.21.15.38:0/4193049548 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:1 13.0 13:00000000:%01:::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x556215b2a6f0 con 0x556215b27c30 2024-09-21T03:40:11.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.711+0000 7f19033d4640 1 --2- 172.21.15.38:0/4193049548 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x556215b27c30 0x556215c74560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:11.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.711+0000 7f19023d2640 1 --2- 172.21.15.38:0/4193049548 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f18f0077570 0x7f18f0079a80 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x556215b35c10 tx=0x7f18f8004880 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:11.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.711+0000 7f19033d4640 1 --2- 172.21.15.38:0/4193049548 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x556215b27c30 0x556215c74560 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:11.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.714+0000 7f19033d4640 1 -- 172.21.15.38:0/4193049548 <== osd.1 v2:172.21.15.38:6816/1491818732 1 ==== osd_op_reply(1 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f18fc00b040 con 0x556215b27c30 2024-09-21T03:40:11.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.714+0000 7f19033d4640 1 --2- 172.21.15.38:0/4193049548 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x7f18fc022d70 0x7f18fc061930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:11.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.714+0000 7f19033d4640 1 -- 172.21.15.38:0/4193049548 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:2 13.10 13:08000000:%01:::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x556215b2a6f0 con 0x7f18fc022d70 2024-09-21T03:40:11.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.714+0000 7f1902bd3640 1 --2- 172.21.15.38:0/4193049548 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x7f18fc022d70 0x7f18fc061930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:11.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.714+0000 7f1902bd3640 1 --2- 172.21.15.38:0/4193049548 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x7f18fc022d70 0x7f18fc061930 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:11.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.717+0000 7f1902bd3640 1 -- 172.21.15.38:0/4193049548 <== osd.7 v2:172.21.15.195:6818/1606106342 1 ==== osd_op_reply(2 [pgnls start_epoch 95 out=177b] v94'10 uv10 ondisk = 1) ==== 144+0+177 (crc 0 0 0) 0x7f18f4020cf0 con 0x7f18fc022d70 2024-09-21T03:40:11.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.717+0000 7f1902bd3640 1 --2- 172.21.15.38:0/4193049548 >> [v2:172.21.15.195:6810/2080960180,v1:172.21.15.195:6811/2080960180] conn(0x7f18f4064be0 0x7f18f4062630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:11.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.717+0000 7f1902bd3640 1 -- 172.21.15.38:0/4193049548 --> [v2:172.21.15.195:6810/2080960180,v1:172.21.15.195:6811/2080960180] -- osd_op(unknown.0.0:3 13.8 13:10000000:%01:::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x556215b2a6f0 con 0x7f18f4064be0 2024-09-21T03:40:11.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.717+0000 7f19023d2640 1 --2- 172.21.15.38:0/4193049548 >> [v2:172.21.15.195:6810/2080960180,v1:172.21.15.195:6811/2080960180] conn(0x7f18f4064be0 0x7f18f4062630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:11.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.717+0000 7f19023d2640 1 --2- 172.21.15.38:0/4193049548 >> [v2:172.21.15.195:6810/2080960180,v1:172.21.15.195:6811/2080960180] conn(0x7f18f4064be0 0x7f18f4062630 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:11.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.719+0000 7f19023d2640 1 -- 172.21.15.38:0/4193049548 <== osd.5 v2:172.21.15.195:6810/2080960180 1 ==== osd_op_reply(3 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f18f800c050 con 0x7f18f4064be0 2024-09-21T03:40:11.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.719+0000 7f19023d2640 1 -- 172.21.15.38:0/4193049548 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:4 13.18 13:18000000:%01:::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x556215b2a6f0 con 0x556215b27c30 2024-09-21T03:40:11.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.719+0000 7f19033d4640 1 -- 172.21.15.38:0/4193049548 <== osd.1 v2:172.21.15.38:6816/1491818732 2 ==== osd_op_reply(4 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f18fc00b040 con 0x556215b27c30 2024-09-21T03:40:11.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.719+0000 7f19033d4640 1 --2- 172.21.15.38:0/4193049548 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x7f18fc01f6f0 0x7f18fc055920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:11.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.719+0000 7f19033d4640 1 -- 172.21.15.38:0/4193049548 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:5 13.4 13:20000000:%01:::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x556215b2a6f0 con 0x7f18fc01f6f0 2024-09-21T03:40:11.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.719+0000 7f19033d4640 1 --2- 172.21.15.38:0/4193049548 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x7f18fc01f6f0 0x7f18fc055920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:11.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.720+0000 7f19033d4640 1 --2- 172.21.15.38:0/4193049548 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x7f18fc01f6f0 0x7f18fc055920 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:11.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.720+0000 7f19033d4640 1 -- 172.21.15.38:0/4193049548 <== osd.3 v2:172.21.15.38:6808/1286810294 1 ==== osd_op_reply(5 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f18fc00b040 con 0x7f18fc01f6f0 2024-09-21T03:40:11.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.720+0000 7f19033d4640 1 --2- 172.21.15.38:0/4193049548 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x7f18fc00c2b0 0x7f18fc071410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:11.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.720+0000 7f19033d4640 1 -- 172.21.15.38:0/4193049548 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:6 13.14 13:28000000:%01:::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x556215b2a6f0 con 0x7f18fc00c2b0 2024-09-21T03:40:11.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.720+0000 7f1902bd3640 1 --2- 172.21.15.38:0/4193049548 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x7f18fc00c2b0 0x7f18fc071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:11.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.721+0000 7f1902bd3640 1 --2- 172.21.15.38:0/4193049548 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x7f18fc00c2b0 0x7f18fc071410 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:11.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.723+0000 7f1902bd3640 1 -- 172.21.15.38:0/4193049548 <== osd.6 v2:172.21.15.195:6826/4055825675 1 ==== osd_op_reply(6 [pgnls start_epoch 95 out=149b] v94'18 uv18 ondisk = 1) ==== 144+0+149 (crc 0 0 0) 0x7f18f407dd70 con 0x7f18fc00c2b0 2024-09-21T03:40:11.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.723+0000 7f1902bd3640 1 -- 172.21.15.38:0/4193049548 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:7 13.c 13:30000000:%01:::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x556215b2a6f0 con 0x7f18fc00c2b0 2024-09-21T03:40:11.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.723+0000 7f1902bd3640 1 -- 172.21.15.38:0/4193049548 <== osd.6 v2:172.21.15.195:6826/4055825675 2 ==== osd_op_reply(7 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f18f4083040 con 0x7f18fc00c2b0 2024-09-21T03:40:11.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.723+0000 7f1902bd3640 1 --2- 172.21.15.38:0/4193049548 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x7f18f4063bc0 0x7f18f407a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:11.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.723+0000 7f1902bd3640 1 -- 172.21.15.38:0/4193049548 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:8 13.1c 13:38000000:%01:::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x556215b2a6f0 con 0x7f18f4063bc0 2024-09-21T03:40:11.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.723+0000 7f19023d2640 1 --2- 172.21.15.38:0/4193049548 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x7f18f4063bc0 0x7f18f407a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:11.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.724+0000 7f19023d2640 1 --2- 172.21.15.38:0/4193049548 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x7f18f4063bc0 0x7f18f407a9e0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:11.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.726+0000 7f19023d2640 1 -- 172.21.15.38:0/4193049548 <== osd.0 v2:172.21.15.38:6800/2381639433 1 ==== osd_op_reply(8 [pgnls start_epoch 95 out=73b] v93'2 uv2 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x7f18f800c050 con 0x7f18f4063bc0 2024-09-21T03:40:11.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.726+0000 7f19023d2640 1 -- 172.21.15.38:0/4193049548 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:9 13.2 13:40000000:%01:::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x556215b2a6f0 con 0x556215b27c30 2024-09-21T03:40:11.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.726+0000 7f19033d4640 1 -- 172.21.15.38:0/4193049548 <== osd.1 v2:172.21.15.38:6816/1491818732 3 ==== osd_op_reply(9 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f18fc00b040 con 0x556215b27c30 2024-09-21T03:40:11.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.726+0000 7f19033d4640 1 -- 172.21.15.38:0/4193049548 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:10 13.12 13:48000000:%01:::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x556215b2a6f0 con 0x7f18fc00c2b0 2024-09-21T03:40:11.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.727+0000 7f1902bd3640 1 -- 172.21.15.38:0/4193049548 <== osd.6 v2:172.21.15.195:6826/4055825675 3 ==== osd_op_reply(10 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f18f4083040 con 0x7f18fc00c2b0 2024-09-21T03:40:11.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.727+0000 7f1902bd3640 1 --2- 172.21.15.38:0/4193049548 >> [v2:172.21.15.195:6802/3401837571,v1:172.21.15.195:6803/3401837571] conn(0x7f18f406cdb0 0x7f18f4061d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:11.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.727+0000 7f1902bd3640 1 -- 172.21.15.38:0/4193049548 --> [v2:172.21.15.195:6802/3401837571,v1:172.21.15.195:6803/3401837571] -- osd_op(unknown.0.0:11 13.a 13:50000000:%01:::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x556215b2a6f0 con 0x7f18f406cdb0 2024-09-21T03:40:11.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.727+0000 7f19033d4640 1 --2- 172.21.15.38:0/4193049548 >> [v2:172.21.15.195:6802/3401837571,v1:172.21.15.195:6803/3401837571] conn(0x7f18f406cdb0 0x7f18f4061d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:11.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.727+0000 7f19033d4640 1 --2- 172.21.15.38:0/4193049548 >> [v2:172.21.15.195:6802/3401837571,v1:172.21.15.195:6803/3401837571] conn(0x7f18f406cdb0 0x7f18f4061d30 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:11.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.728+0000 7f19033d4640 1 -- 172.21.15.38:0/4193049548 <== osd.4 v2:172.21.15.195:6802/3401837571 1 ==== osd_op_reply(11 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f18fc00b040 con 0x7f18f406cdb0 2024-09-21T03:40:11.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.728+0000 7f19033d4640 1 -- 172.21.15.38:0/4193049548 --> [v2:172.21.15.195:6810/2080960180,v1:172.21.15.195:6811/2080960180] -- osd_op(unknown.0.0:12 13.1a 13:58000000:%01:::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x556215b2a6f0 con 0x7f18f4064be0 2024-09-21T03:40:11.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.728+0000 7f19023d2640 1 -- 172.21.15.38:0/4193049548 <== osd.5 v2:172.21.15.195:6810/2080960180 2 ==== osd_op_reply(12 [pgnls start_epoch 95 out=49b] v93'1 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f18f800c050 con 0x7f18f4064be0 2024-09-21T03:40:11.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.728+0000 7f19023d2640 1 -- 172.21.15.38:0/4193049548 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:13 13.6 13:60000000:%01:::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x556215b2a6f0 con 0x7f18fc01f6f0 2024-09-21T03:40:11.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.728+0000 7f19033d4640 1 -- 172.21.15.38:0/4193049548 <== osd.3 v2:172.21.15.38:6808/1286810294 2 ==== osd_op_reply(13 [pgnls start_epoch 95 out=77b] v94'2 uv2 ondisk = 1) ==== 144+0+77 (crc 0 0 0) 0x7f18fc00b040 con 0x7f18fc01f6f0 2024-09-21T03:40:11.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.728+0000 7f19033d4640 1 -- 172.21.15.38:0/4193049548 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:14 13.16 13:68000000:%01:::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x556215b2a6f0 con 0x7f18fc022d70 2024-09-21T03:40:11.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.729+0000 7f1902bd3640 1 -- 172.21.15.38:0/4193049548 <== osd.7 v2:172.21.15.195:6818/1606106342 2 ==== osd_op_reply(14 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f18f4083040 con 0x7f18fc022d70 2024-09-21T03:40:11.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.729+0000 7f1902bd3640 1 -- 172.21.15.38:0/4193049548 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:15 13.e 13:70000000:%01:::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x556215b2a6f0 con 0x7f18f4063bc0 2024-09-21T03:40:11.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.729+0000 7f19023d2640 1 -- 172.21.15.38:0/4193049548 <== osd.0 v2:172.21.15.38:6800/2381639433 2 ==== osd_op_reply(15 [pgnls start_epoch 95 out=73b] v93'2 uv2 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x7f18f800c050 con 0x7f18f4063bc0 2024-09-21T03:40:11.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.729+0000 7f19023d2640 1 -- 172.21.15.38:0/4193049548 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:16 13.1e 13:78000000:%01:::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x556215b2a6f0 con 0x7f18f4063bc0 2024-09-21T03:40:11.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.729+0000 7f19023d2640 1 -- 172.21.15.38:0/4193049548 <== osd.0 v2:172.21.15.38:6800/2381639433 3 ==== osd_op_reply(16 [pgnls start_epoch 95 out=49b] v93'1 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f18f800c050 con 0x7f18f4063bc0 2024-09-21T03:40:11.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.729+0000 7f19023d2640 1 -- 172.21.15.38:0/4193049548 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:17 13.1 13:80000000:%01:::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x556215b2a6f0 con 0x556215b27c30 2024-09-21T03:40:11.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.729+0000 7f19033d4640 1 -- 172.21.15.38:0/4193049548 <== osd.1 v2:172.21.15.38:6816/1491818732 4 ==== osd_op_reply(17 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f18fc00b040 con 0x556215b27c30 2024-09-21T03:40:11.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.729+0000 7f19033d4640 1 -- 172.21.15.38:0/4193049548 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:18 13.11 13:88000000:%01:::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x556215b2a6f0 con 0x7f18fc00c2b0 2024-09-21T03:40:11.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.730+0000 7f1902bd3640 1 -- 172.21.15.38:0/4193049548 <== osd.6 v2:172.21.15.195:6826/4055825675 4 ==== osd_op_reply(18 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f18f4083040 con 0x7f18fc00c2b0 2024-09-21T03:40:11.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.730+0000 7f1902bd3640 1 -- 172.21.15.38:0/4193049548 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:19 13.9 13:90000000:%01:::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x556215b2a6f0 con 0x7f18f4063bc0 2024-09-21T03:40:11.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.730+0000 7f19023d2640 1 -- 172.21.15.38:0/4193049548 <== osd.0 v2:172.21.15.38:6800/2381639433 4 ==== osd_op_reply(19 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f18f800c050 con 0x7f18f4063bc0 2024-09-21T03:40:11.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.730+0000 7f19023d2640 1 -- 172.21.15.38:0/4193049548 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:20 13.19 13:98000000:%01:::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x556215b2a6f0 con 0x7f18fc022d70 2024-09-21T03:40:11.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.730+0000 7f1902bd3640 1 -- 172.21.15.38:0/4193049548 <== osd.7 v2:172.21.15.195:6818/1606106342 3 ==== osd_op_reply(20 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f18f4083040 con 0x7f18fc022d70 2024-09-21T03:40:11.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.730+0000 7f1902bd3640 1 --2- 172.21.15.38:0/4193049548 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x7f18f406a550 0x7f18f4079370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:11.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.730+0000 7f1902bd3640 1 -- 172.21.15.38:0/4193049548 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:21 13.5 13:a0000000:%01:::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x556215b2a6f0 con 0x7f18f406a550 2024-09-21T03:40:11.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.731+0000 7f1902bd3640 1 --2- 172.21.15.38:0/4193049548 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x7f18f406a550 0x7f18f4079370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:11.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.731+0000 7f1902bd3640 1 --2- 172.21.15.38:0/4193049548 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x7f18f406a550 0x7f18f4079370 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:11.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.731+0000 7f1902bd3640 1 -- 172.21.15.38:0/4193049548 <== osd.2 v2:172.21.15.38:6824/2577858902 1 ==== osd_op_reply(21 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f18f4083040 con 0x7f18f406a550 2024-09-21T03:40:11.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.731+0000 7f1902bd3640 1 -- 172.21.15.38:0/4193049548 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:22 13.15 13:a8000000:%01:::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x556215b2a6f0 con 0x7f18fc01f6f0 2024-09-21T03:40:11.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.732+0000 7f19033d4640 1 -- 172.21.15.38:0/4193049548 <== osd.3 v2:172.21.15.38:6808/1286810294 3 ==== osd_op_reply(22 [pgnls start_epoch 95 out=97b] v93'3 uv3 ondisk = 1) ==== 144+0+97 (crc 0 0 0) 0x7f18fc00b040 con 0x7f18fc01f6f0 2024-09-21T03:40:11.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.732+0000 7f19033d4640 1 -- 172.21.15.38:0/4193049548 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:23 13.d 13:b0000000:%01:::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x556215b2a6f0 con 0x7f18f4063bc0 2024-09-21T03:40:11.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.732+0000 7f19023d2640 1 -- 172.21.15.38:0/4193049548 <== osd.0 v2:172.21.15.38:6800/2381639433 5 ==== osd_op_reply(23 [pgnls start_epoch 95 out=74b] v93'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f18f800c050 con 0x7f18f4063bc0 2024-09-21T03:40:11.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.732+0000 7f19023d2640 1 -- 172.21.15.38:0/4193049548 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:24 13.1d 13:b8000000:%01:::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x556215b2a6f0 con 0x556215b27c30 2024-09-21T03:40:11.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.732+0000 7f19033d4640 1 -- 172.21.15.38:0/4193049548 <== osd.1 v2:172.21.15.38:6816/1491818732 5 ==== osd_op_reply(24 [pgnls start_epoch 95 out=73b] v93'2 uv2 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x7f18fc00b040 con 0x556215b27c30 2024-09-21T03:40:11.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.732+0000 7f19033d4640 1 -- 172.21.15.38:0/4193049548 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:25 13.3 13:c0000000:%01:::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x556215b2a6f0 con 0x7f18fc00c2b0 2024-09-21T03:40:11.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.733+0000 7f1902bd3640 1 -- 172.21.15.38:0/4193049548 <== osd.6 v2:172.21.15.195:6826/4055825675 5 ==== osd_op_reply(25 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f18f4083040 con 0x7f18fc00c2b0 2024-09-21T03:40:11.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.733+0000 7f1902bd3640 1 -- 172.21.15.38:0/4193049548 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:26 13.13 13:c8000000:%01:::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x556215b2a6f0 con 0x556215b27c30 2024-09-21T03:40:11.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.733+0000 7f19033d4640 1 -- 172.21.15.38:0/4193049548 <== osd.1 v2:172.21.15.38:6816/1491818732 6 ==== osd_op_reply(26 [pgnls start_epoch 95 out=98b] v94'3 uv3 ondisk = 1) ==== 144+0+98 (crc 0 0 0) 0x7f18fc00b040 con 0x556215b27c30 2024-09-21T03:40:11.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.733+0000 7f19033d4640 1 -- 172.21.15.38:0/4193049548 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:27 13.b 13:d0000000:%01:::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x556215b2a6f0 con 0x7f18fc01f6f0 2024-09-21T03:40:11.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.733+0000 7f19033d4640 1 -- 172.21.15.38:0/4193049548 <== osd.3 v2:172.21.15.38:6808/1286810294 4 ==== osd_op_reply(27 [pgnls start_epoch 95 out=76b] v94'3 uv3 ondisk = 1) ==== 144+0+76 (crc 0 0 0) 0x7f18fc00b040 con 0x7f18fc01f6f0 2024-09-21T03:40:11.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.733+0000 7f19033d4640 1 -- 172.21.15.38:0/4193049548 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:28 13.1b 13:d8000000:%01:::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x556215b2a6f0 con 0x7f18f406a550 2024-09-21T03:40:11.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.733+0000 7f1902bd3640 1 -- 172.21.15.38:0/4193049548 <== osd.2 v2:172.21.15.38:6824/2577858902 2 ==== osd_op_reply(28 [pgnls start_epoch 95 out=49b] v93'1 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f18f4083040 con 0x7f18f406a550 2024-09-21T03:40:11.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.734+0000 7f1902bd3640 1 -- 172.21.15.38:0/4193049548 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:29 13.7 13:e0000000:%01:::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x556215b2a6f0 con 0x7f18fc01f6f0 2024-09-21T03:40:11.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.734+0000 7f19033d4640 1 -- 172.21.15.38:0/4193049548 <== osd.3 v2:172.21.15.38:6808/1286810294 5 ==== osd_op_reply(29 [pgnls start_epoch 95 out=73b] v93'2 uv2 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x7f18fc00b040 con 0x7f18fc01f6f0 2024-09-21T03:40:11.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.734+0000 7f19033d4640 1 -- 172.21.15.38:0/4193049548 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:30 13.17 13:e8000000:%01:::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x556215b2a6f0 con 0x556215b27c30 2024-09-21T03:40:11.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.734+0000 7f19033d4640 1 -- 172.21.15.38:0/4193049548 <== osd.1 v2:172.21.15.38:6816/1491818732 7 ==== osd_op_reply(30 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f18fc00b040 con 0x556215b27c30 2024-09-21T03:40:11.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.734+0000 7f19033d4640 1 -- 172.21.15.38:0/4193049548 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:31 13.f 13:f0000000:%01:::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x556215b2a6f0 con 0x7f18f406a550 2024-09-21T03:40:11.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.734+0000 7f1902bd3640 1 -- 172.21.15.38:0/4193049548 <== osd.2 v2:172.21.15.38:6824/2577858902 3 ==== osd_op_reply(31 [pgnls start_epoch 95 out=73b] v93'1 uv1 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x7f18f4083040 con 0x7f18f406a550 2024-09-21T03:40:11.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.735+0000 7f1902bd3640 1 -- 172.21.15.38:0/4193049548 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:32 13.1f 13:f8000000:%01:::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x556215b2a6f0 con 0x556215b27c30 2024-09-21T03:40:11.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.735+0000 7f19033d4640 1 -- 172.21.15.38:0/4193049548 <== osd.1 v2:172.21.15.38:6816/1491818732 8 ==== osd_op_reply(32 [pgnls start_epoch 95 out=101b] v94'6 uv6 ondisk = 1) ==== 144+0+101 (crc 0 0 0) 0x7f18fc00b040 con 0x556215b27c30 2024-09-21T03:40:11.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.735+0000 7f1909119f40 1 -- 172.21.15.38:0/4193049548 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:33 13.10 13:09401a81:::601.00000000:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x556215c74aa0 con 0x7f18fc022d70 2024-09-21T03:40:11.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.735+0000 7f1902bd3640 1 -- 172.21.15.38:0/4193049548 <== osd.7 v2:172.21.15.195:6818/1606106342 4 ==== osd_op_reply(33 601.00000000 [read 0~0] v0'0 uv4 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f18f4083040 con 0x7f18fc022d70 2024-09-21T03:40:11.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.736+0000 7f1909119f40 1 -- 172.21.15.38:0/4193049548 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:34 13.10 13:09401a81:::601.00000000:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x556215b4c050 con 0x7f18fc022d70 2024-09-21T03:40:11.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.736+0000 7f1902bd3640 1 -- 172.21.15.38:0/4193049548 <== osd.7 v2:172.21.15.195:6818/1606106342 5 ==== osd_op_reply(34 601.00000000 [getxattrs out=124b] v0'0 uv4 ondisk = 0) ==== 156+0+124 (crc 0 0 0) 0x7f18f4083040 con 0x7f18fc022d70 2024-09-21T03:40:11.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.736+0000 7f1909119f40 1 -- 172.21.15.38:0/4193049548 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:35 13.10 13:09401a81:::601.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x556215b4aeb0 con 0x7f18fc022d70 2024-09-21T03:40:11.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.736+0000 7f1902bd3640 1 -- 172.21.15.38:0/4193049548 <== osd.7 v2:172.21.15.195:6818/1606106342 6 ==== osd_op_reply(35 601.00000000 [omap-get-header out=274b] v0'0 uv4 ondisk = 0) ==== 156+0+274 (crc 0 0 0) 0x7f18f4083040 con 0x7f18fc022d70 2024-09-21T03:40:11.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.736+0000 7f1909119f40 1 -- 172.21.15.38:0/4193049548 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:36 13.10 13:09401a81:::601.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x556215b4b2d0 con 0x7f18fc022d70 2024-09-21T03:40:11.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.737+0000 7f1902bd3640 1 -- 172.21.15.38:0/4193049548 <== osd.7 v2:172.21.15.195:6818/1606106342 7 ==== osd_op_reply(36 601.00000000 [omap-get-vals out=5b] v0'0 uv4 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x7f18f4083040 con 0x7f18fc022d70 2024-09-21T03:40:11.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.737+0000 7f1909119f40 1 -- 172.21.15.38:0/4193049548 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:37 13.10 13:0bd6d154:::602.00000000:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x556215b4b6f0 con 0x7f18fc022d70 2024-09-21T03:40:11.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.737+0000 7f1902bd3640 1 -- 172.21.15.38:0/4193049548 <== osd.7 v2:172.21.15.195:6818/1606106342 8 ==== osd_op_reply(37 602.00000000 [read 0~0] v0'0 uv6 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f18f4083040 con 0x7f18fc022d70 2024-09-21T03:40:11.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.737+0000 7f1909119f40 1 -- 172.21.15.38:0/4193049548 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:38 13.10 13:0bd6d154:::602.00000000:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x556215b4bae0 con 0x7f18fc022d70 2024-09-21T03:40:11.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.737+0000 7f1902bd3640 1 -- 172.21.15.38:0/4193049548 <== osd.7 v2:172.21.15.195:6818/1606106342 9 ==== osd_op_reply(38 602.00000000 [getxattrs out=124b] v0'0 uv6 ondisk = 0) ==== 156+0+124 (crc 0 0 0) 0x7f18f4083040 con 0x7f18fc022d70 2024-09-21T03:40:11.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.737+0000 7f1909119f40 1 -- 172.21.15.38:0/4193049548 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:39 13.10 13:0bd6d154:::602.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x556215c80940 con 0x7f18fc022d70 2024-09-21T03:40:11.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.737+0000 7f1902bd3640 1 -- 172.21.15.38:0/4193049548 <== osd.7 v2:172.21.15.195:6818/1606106342 10 ==== osd_op_reply(39 602.00000000 [omap-get-header out=274b] v0'0 uv6 ondisk = 0) ==== 156+0+274 (crc 0 0 0) 0x7f18f4083040 con 0x7f18fc022d70 2024-09-21T03:40:11.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.737+0000 7f1909119f40 1 -- 172.21.15.38:0/4193049548 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:40 13.10 13:0bd6d154:::602.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x556215c81950 con 0x7f18fc022d70 2024-09-21T03:40:11.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.738+0000 7f1902bd3640 1 -- 172.21.15.38:0/4193049548 <== osd.7 v2:172.21.15.195:6818/1606106342 11 ==== osd_op_reply(40 602.00000000 [omap-get-vals out=5b] v0'0 uv6 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x7f18f4083040 con 0x7f18fc022d70 2024-09-21T03:40:11.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.738+0000 7f1909119f40 1 -- 172.21.15.38:0/4193049548 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:41 13.10 13:0cf6ad0f:::10000000000.00000000:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x556215c80940 con 0x7f18fc022d70 2024-09-21T03:40:11.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.738+0000 7f1902bd3640 1 -- 172.21.15.38:0/4193049548 <== osd.7 v2:172.21.15.195:6818/1606106342 12 ==== osd_op_reply(41 10000000000.00000000 [read 0~0] v0'0 uv10 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f18f4083040 con 0x7f18fc022d70 2024-09-21T03:40:11.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.738+0000 7f1909119f40 1 -- 172.21.15.38:0/4193049548 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:42 13.10 13:0cf6ad0f:::10000000000.00000000:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x556215c80940 con 0x7f18fc022d70 2024-09-21T03:40:11.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.738+0000 7f1902bd3640 1 -- 172.21.15.38:0/4193049548 <== osd.7 v2:172.21.15.195:6818/1606106342 13 ==== osd_op_reply(42 10000000000.00000000 [getxattrs out=124b] v0'0 uv10 ondisk = 0) ==== 164+0+124 (crc 0 0 0) 0x7f18f4083040 con 0x7f18fc022d70 2024-09-21T03:40:11.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.738+0000 7f1909119f40 1 -- 172.21.15.38:0/4193049548 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:43 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x556215c80940 con 0x7f18fc022d70 2024-09-21T03:40:11.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.739+0000 7f1902bd3640 1 -- 172.21.15.38:0/4193049548 <== osd.7 v2:172.21.15.195:6818/1606106342 14 ==== osd_op_reply(43 10000000000.00000000 [omap-get-header out=274b] v0'0 uv10 ondisk = 0) ==== 164+0+274 (crc 0 0 0) 0x7f18f4083040 con 0x7f18fc022d70 2024-09-21T03:40:11.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.739+0000 7f1909119f40 1 -- 172.21.15.38:0/4193049548 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:44 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x556215c81950 con 0x7f18fc022d70 2024-09-21T03:40:11.748 INFO:teuthology.orchestra.run.smithi038.stdout: ��*$- 601.00000000�������������������!�������������������������� ���~_layout��������_parent>8stray1 ���� �� ((88���� �� ��*$- 602.00000000�������������������!�������������������������� ���~_layout��������_parent>8stray2 ���� �� ((88���� �� ��2,510000000000.00000000�������������������!�������������������������� ���~_layout��������_parent>8subdir ���� ��  (@�f%�(@�f%�8`@�fP��8`@�fP������ ��OI sixmegs_head1i"�@�fP�����5@@`��������@�fP��@�f%�(8`@�fP��8`@�fP�� 2024-09-21T03:40:11.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.739+0000 7f1902bd3640 1 -- 172.21.15.38:0/4193049548 <== osd.7 v2:172.21.15.195:6818/1606106342 15 ==== osd_op_reply(44 10000000000.00000000 [omap-get-vals out=586b] v0'0 uv10 ondisk = 0) ==== 164+0+586 (crc 0 0 0) 0x7f18f4083040 con 0x7f18fc022d70 2024-09-21T03:40:11.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.739+0000 7f1909119f40 1 -- 172.21.15.38:0/4193049548 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:45 13.10 13:0d82a743:::600.00000000:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x556215c80940 con 0x7f18fc022d70 2024-09-21T03:40:11.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.739+0000 7f1902bd3640 1 -- 172.21.15.38:0/4193049548 <== osd.7 v2:172.21.15.195:6818/1606106342 16 ==== osd_op_reply(45 600.00000000 [read 0~0] v0'0 uv2 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f18f4083040 con 0x7f18fc022d70 2024-09-21T03:40:11.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.739+0000 7f1909119f40 1 -- 172.21.15.38:0/4193049548 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:46 13.10 13:0d82a743:::600.00000000:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x556215c80940 con 0x7f18fc022d70 2024-09-21T03:40:11.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.740+0000 7f1902bd3640 1 -- 172.21.15.38:0/4193049548 <== osd.7 v2:172.21.15.195:6818/1606106342 17 ==== osd_op_reply(46 600.00000000 [getxattrs out=124b] v0'0 uv2 ondisk = 0) ==== 156+0+124 (crc 0 0 0) 0x7f18f4083040 con 0x7f18fc022d70 2024-09-21T03:40:11.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.740+0000 7f1909119f40 1 -- 172.21.15.38:0/4193049548 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:47 13.10 13:0d82a743:::600.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x556215c80940 con 0x7f18fc022d70 2024-09-21T03:40:11.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.740+0000 7f1902bd3640 1 -- 172.21.15.38:0/4193049548 <== osd.7 v2:172.21.15.195:6818/1606106342 18 ==== osd_op_reply(47 600.00000000 [omap-get-header out=274b] v0'0 uv2 ondisk = 0) ==== 156+0+274 (crc 0 0 0) 0x7f18f4083040 con 0x7f18fc022d70 2024-09-21T03:40:11.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.740+0000 7f1909119f40 1 -- 172.21.15.38:0/4193049548 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:48 13.10 13:0d82a743:::600.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x556215c81950 con 0x7f18fc022d70 2024-09-21T03:40:11.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.740+0000 7f1902bd3640 1 -- 172.21.15.38:0/4193049548 <== osd.7 v2:172.21.15.195:6818/1606106342 19 ==== osd_op_reply(48 600.00000000 [omap-get-vals out=5b] v0'0 uv2 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x7f18f4083040 con 0x7f18fc022d70 2024-09-21T03:40:11.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.740+0000 7f1909119f40 1 -- 172.21.15.38:0/4193049548 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:49 13.10 13:0d89b25e:::603.00000000:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x556215c80940 con 0x7f18fc022d70 2024-09-21T03:40:11.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.740+0000 7f1902bd3640 1 -- 172.21.15.38:0/4193049548 <== osd.7 v2:172.21.15.195:6818/1606106342 20 ==== osd_op_reply(49 603.00000000 [read 0~0] v0'0 uv8 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f18f4083040 con 0x7f18fc022d70 2024-09-21T03:40:11.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.740+0000 7f1909119f40 1 -- 172.21.15.38:0/4193049548 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:50 13.10 13:0d89b25e:::603.00000000:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x556215c80940 con 0x7f18fc022d70 2024-09-21T03:40:11.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.741+0000 7f1902bd3640 1 -- 172.21.15.38:0/4193049548 <== osd.7 v2:172.21.15.195:6818/1606106342 21 ==== osd_op_reply(50 603.00000000 [getxattrs out=124b] v0'0 uv8 ondisk = 0) ==== 156+0+124 (crc 0 0 0) 0x7f18f4083040 con 0x7f18fc022d70 2024-09-21T03:40:11.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.741+0000 7f1909119f40 1 -- 172.21.15.38:0/4193049548 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:51 13.10 13:0d89b25e:::603.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x556215c80940 con 0x7f18fc022d70 2024-09-21T03:40:11.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.741+0000 7f1902bd3640 1 -- 172.21.15.38:0/4193049548 <== osd.7 v2:172.21.15.195:6818/1606106342 22 ==== osd_op_reply(51 603.00000000 [omap-get-header out=274b] v0'0 uv8 ondisk = 0) ==== 156+0+274 (crc 0 0 0) 0x7f18f4083040 con 0x7f18fc022d70 2024-09-21T03:40:11.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.741+0000 7f1909119f40 1 -- 172.21.15.38:0/4193049548 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:52 13.10 13:0d89b25e:::603.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x556215c81950 con 0x7f18fc022d70 2024-09-21T03:40:11.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.741+0000 7f1902bd3640 1 -- 172.21.15.38:0/4193049548 <== osd.7 v2:172.21.15.195:6818/1606106342 23 ==== osd_op_reply(52 603.00000000 [omap-get-vals out=5b] v0'0 uv8 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x7f18f4083040 con 0x7f18fc022d70 2024-09-21T03:40:11.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.741+0000 7f1909119f40 1 -- 172.21.15.38:0/4193049548 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:53 13.14 13:287034c4:::1.00000000.inode:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x556215c80940 con 0x7f18fc00c2b0 2024-09-21T03:40:11.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.742+0000 7f1902bd3640 1 -- 172.21.15.38:0/4193049548 <== osd.6 v2:172.21.15.195:6826/4055825675 6 ==== osd_op_reply(53 1.00000000.inode [read 0~582 out=582b] v0'0 uv15 ondisk = 0) ==== 160+0+582 (crc 0 0 0) 0x7f18f4083040 con 0x7f18fc00c2b0 2024-09-21T03:40:11.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.742+0000 7f1909119f40 1 -- 172.21.15.38:0/4193049548 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:54 13.14 13:287034c4:::1.00000000.inode:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x556215c80940 con 0x7f18fc00c2b0 2024-09-21T03:40:11.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.742+0000 7f1902bd3640 1 -- 172.21.15.38:0/4193049548 <== osd.6 v2:172.21.15.195:6826/4055825675 7 ==== osd_op_reply(54 1.00000000.inode [getxattrs out=4b] v0'0 uv15 ondisk = 0) ==== 160+0+4 (crc 0 0 0) 0x7f18f4083040 con 0x7f18fc00c2b0 2024-09-21T03:40:11.756 INFO:teuthology.orchestra.run.smithi038.stdout:��������@�f%�����security.selinux%unconfined_u:object_r:unlabeled_t:s0������������ �� ��*$- 600.00000000�������������������!�������������������������� ���~_layout��������_parent>8stray0 ���� �� ((88���� �� ��*$- 603.00000000�������������������!�������������������������� ���~_layout��������_parent>8stray3 ���� �� ((88���� �� ��.(11.00000000.inode�������������������!�������������������������� ��`ZFFceph fs volume v011)�@�f�A7�C@@��������@�f�A7(@�f�A78@�f�A78@�f��5@�f��5����NH@�f��5������������ �� 2024-09-21T03:40:11.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.742+0000 7f1909119f40 1 -- 172.21.15.38:0/4193049548 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:55 13.14 13:287034c4:::1.00000000.inode:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x556215c80940 con 0x7f18fc00c2b0 2024-09-21T03:40:11.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.742+0000 7f1902bd3640 1 -- 172.21.15.38:0/4193049548 <== osd.6 v2:172.21.15.195:6826/4055825675 8 ==== osd_op_reply(55 1.00000000.inode [omap-get-header] v0'0 uv15 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f18f4083040 con 0x7f18fc00c2b0 2024-09-21T03:40:11.757 INFO:teuthology.orchestra.run.smithi038.stdout:���� �� 2024-09-21T03:40:11.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.742+0000 7f1909119f40 1 -- 172.21.15.38:0/4193049548 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:56 13.14 13:287034c4:::1.00000000.inode:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x556215c81950 con 0x7f18fc00c2b0 2024-09-21T03:40:11.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.743+0000 7f1902bd3640 1 -- 172.21.15.38:0/4193049548 <== osd.6 v2:172.21.15.195:6826/4055825675 9 ==== osd_op_reply(56 1.00000000.inode [omap-get-vals out=5b] v0'0 uv15 ondisk = 0) ==== 160+0+5 (crc 0 0 0) 0x7f18f4083040 con 0x7f18fc00c2b0 2024-09-21T03:40:11.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.743+0000 7f1909119f40 1 -- 172.21.15.38:0/4193049548 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:57 13.14 13:292cf221:::200.00000000:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x556215c80940 con 0x7f18fc00c2b0 2024-09-21T03:40:11.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.743+0000 7f1902bd3640 1 -- 172.21.15.38:0/4193049548 <== osd.6 v2:172.21.15.195:6826/4055825675 10 ==== osd_op_reply(57 200.00000000 [read 0~90 out=90b] v0'0 uv18 ondisk = 0) ==== 156+0+90 (crc 0 0 0) 0x7f18f4083040 con 0x7f18fc00c2b0 2024-09-21T03:40:11.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.743+0000 7f1909119f40 1 -- 172.21.15.38:0/4193049548 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:58 13.14 13:292cf221:::200.00000000:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x556215c80940 con 0x7f18fc00c2b0 2024-09-21T03:40:11.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.743+0000 7f1902bd3640 1 -- 172.21.15.38:0/4193049548 <== osd.6 v2:172.21.15.195:6826/4055825675 11 ==== osd_op_reply(58 200.00000000 [getxattrs out=4b] v0'0 uv18 ondisk = 0) ==== 156+0+4 (crc 0 0 0) 0x7f18f4083040 con 0x7f18fc00c2b0 2024-09-21T03:40:11.759 INFO:teuthology.orchestra.run.smithi038.stdout:���� �� ��*$- 200.00000000�������������������!�������������������������� ��tnZZTceph fs volume v011@d<@d<@RD@@@ ���� �� 2024-09-21T03:40:11.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.743+0000 7f1909119f40 1 -- 172.21.15.38:0/4193049548 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:59 13.14 13:292cf221:::200.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x556215c80940 con 0x7f18fc00c2b0 2024-09-21T03:40:11.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.743+0000 7f1902bd3640 1 -- 172.21.15.38:0/4193049548 <== osd.6 v2:172.21.15.195:6826/4055825675 12 ==== osd_op_reply(59 200.00000000 [omap-get-header] v0'0 uv18 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f18f4083040 con 0x7f18fc00c2b0 2024-09-21T03:40:11.760 INFO:teuthology.orchestra.run.smithi038.stdout:���� �� 2024-09-21T03:40:11.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.743+0000 7f1909119f40 1 -- 172.21.15.38:0/4193049548 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:60 13.14 13:292cf221:::200.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x556215c81950 con 0x7f18fc00c2b0 2024-09-21T03:40:11.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.744+0000 7f1902bd3640 1 -- 172.21.15.38:0/4193049548 <== osd.6 v2:172.21.15.195:6826/4055825675 13 ==== osd_op_reply(60 200.00000000 [omap-get-vals out=5b] v0'0 uv18 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x7f18f4083040 con 0x7f18fc00c2b0 2024-09-21T03:40:11.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.744+0000 7f1909119f40 1 -- 172.21.15.38:0/4193049548 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:61 13.14 13:2e2fa760:::200.00000001:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x556215c80940 con 0x7f18fc00c2b0 2024-09-21T03:40:11.762 INFO:teuthology.orchestra.run.smithi038.stdout:���� �� ��*$- 200.00000001�������������������!�������������������������� ��lDfDRDRD��XS&YA1e@��XS&YA1h^T@�f�*52 ((88 2 ( 2024-09-21T03:40:11.762 INFO:teuthology.orchestra.run.smithi038.stdout:( 2024-09-21T03:40:11.762 INFO:teuthology.orchestra.run.smithi038.stdout:8 2024-09-21T03:40:11.762 INFO:teuthology.orchestra.run.smithi038.stdout:@�f-� 58 2024-09-21T03:40:11.762 INFO:teuthology.orchestra.run.smithi038.stdout:@�f-� 5 ��������@0@��XS&YA1�� 2024-09-21T03:40:11.763 INFO:teuthology.orchestra.run.smithi038.stdout:�@�f?S�)(�Lk�&� entity_id0hostname smithi038kernel_version5.14.0-508.el9.x86_64root/�� ����@��XS&YA1���@�f[��+n���������@��XS&YA1@�f���+setattr� C���������@�fݟ�+�C@@��������@�f��5(8@�fݟ�+8@�f��5@�f��5����NH@�f��5����������������X@��XS&YA1��@�fq�"7mkdir�p (@�f�A7(@�f�A78@�f�A78@�f�A7J 8subdir���������@�f�A7�A�5����������������@�f�A7@�f�A7(88��������@�f�A7����security.selinux%unconfined_u:object_r:unlabeled_t:s0��������2 ((88  C���������@�f�A7�C@@��������@�f�A7(@�f�A78@�f�A78@�f��5@�f��5����NH@�f��5������������������@��XS&YA1�@�fV>�openc�p (@�f�A7(@�f�A78@�f%�8@�f�A7J 8subdir���������@�f%��A�5����������������@�f%�@�f�A7(@�f%�8@�f%�8@�f%���������@�f�A7����security.selinux%unconfined_u:object_r:unlabeled_t:s0��������� (@�f%�(@�f%�8@�f%�8@�f%�m [sixmegs���������@�f%����5@@��������@�f%�@�f%�@(88��������@�f%�����security.selinux%unconfined_u:object_r:unlabeled_t:s0��������!!���������@��XS&YA1 2024-09-21T03:40:11.763 INFO:teuthology.orchestra.run.smithi038.stdout:@�f�`F 2024-09-21T03:40:11.763 INFO:teuthology.orchestra.run.smithi038.stdout:cap update�p (@�f�A7(@�f�A78 @�f�'8@�f�A7J 8subdir���������@�f%��A�5����������������@�f%�@�f�A7(@�f%�8 @�f�'8 @�f�'��������@�f�A7����security.selinux%unconfined_u:object_r:unlabeled_t:s0��������� (@�f%�(@�f%�8 @�f�'8 @�f�'m [sixmegs���������@�f�'���5@@ ��������@�f�'@�f%��(8 @�f�'8 @�f�'��������@�f%�����security.selinux%unconfined_u:object_r:unlabeled_t:s0�����������������@��XS&YA1 2024-09-21T03:40:11.763 INFO:teuthology.orchestra.run.smithi038.stdout:@�f�� 2024-09-21T03:40:11.763 INFO:teuthology.orchestra.run.smithi038.stdout:cap update�p  (@�f�A7(@�f�A78P@�f��8@�f�A7J 8subdir���������@�f%��A�5����������������@�f%�@�f�A7(@�f%�8P@�f��8P@�f����������@�f�A7����security.selinux%unconfined_u:object_r:unlabeled_t:s0��������� (@�f%�(@�f%�8P@�f��8P@�f��m [sixmegs���������@�f�����5@@P��������@�f��@�f%��(8P@�f��8P@�f����������@�f%�����security.selinux%unconfined_u:object_r:unlabeled_t:s0����������������� @��XS&YA1 2024-09-21T03:40:11.763 INFO:teuthology.orchestra.run.smithi038.stdout:@�f�%v( 2024-09-21T03:40:11.763 INFO:teuthology.orchestra.run.smithi038.stdout:cap update�p  (@�f�A7(@�f�A78`@�fP��8@�f�A7J 8subdir�������� 2024-09-21T03:40:11.763 INFO:teuthology.orchestra.run.smithi038.stdout:�@�f%��A�5����������������@�f%�@�f�A7(@�f%�8`@�fP��8`@�fP�� 2024-09-21T03:40:11.764 INFO:teuthology.orchestra.run.smithi038.stdout:��������@�f�A7����security.selinux%unconfined_u:object_r:unlabeled_t:s0���������  (@�f%�(@�f%�8`@�fP��8`@�fP��m [sixmegs���������@�fP�����5@@`��������@�fP��@�f%�(8`@�fP��8`@�fP����������@�f%�����security.selinux%unconfined_u:object_r:unlabeled_t:s0����������������)@��XS&YA1�� 2024-09-21T03:40:11.764 INFO:teuthology.orchestra.run.smithi038.stdout:�@�fX �((�Lk�&���=1@��XS&YA1 2024-09-21T03:40:11.764 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:40:11.764 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:40:11.764 INFO:teuthology.orchestra.run.smithi038.stdout:@�f���)check_inode_max_size� p  (@�f�A7(@�f�A78`@�fP��8@�f�A7J 8subdir�������� �@�f%��A�5����������������@�f%�@�f�A7(@�f%�8`@�fP��8`@�fP�� ��������@�f�A7����security.selinux%unconfined_u:object_r:unlabeled_t:s0���������  (@�f%�(@�f%�8`@�fP��8`@�fP��� 5sixmegs�������� 2024-09-21T03:40:11.764 INFO:teuthology.orchestra.run.smithi038.stdout:�@�fP�����5@@`��������@�fP��@�f%�(8`@�fP��8`@�fP�� 2024-09-21T03:40:11.764 INFO:teuthology.orchestra.run.smithi038.stdout:��������@�f%�����security.selinux%unconfined_u:object_r:unlabeled_t:s0�������� 5sixmegs�������� 2024-09-21T03:40:11.764 INFO:teuthology.orchestra.run.smithi038.stdout:�@�fP�����5@@`��������@�fP��@�f%�(8`@�fP��8`@�fP�� 2024-09-21T03:40:11.764 INFO:teuthology.orchestra.run.smithi038.stdout:��������@�f%�����security.selinux%unconfined_u:object_r:unlabeled_t:s0�����������������1@��XS&YA18.$�<@��XS&YA1h^T@�fd,�2  (@�f�A7(@�f�A78`@�fP��8@�f�A7 2 ( 2024-09-21T03:40:11.765 INFO:teuthology.orchestra.run.smithi038.stdout:( 2024-09-21T03:40:11.765 INFO:teuthology.orchestra.run.smithi038.stdout:8 2024-09-21T03:40:11.765 INFO:teuthology.orchestra.run.smithi038.stdout:@�f-� 58 2024-09-21T03:40:11.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.744+0000 7f1902bd3640 1 -- 172.21.15.38:0/4193049548 <== osd.6 v2:172.21.15.195:6826/4055825675 14 ==== osd_op_reply(61 200.00000001 [read 0~17490 out=17490b] v0'0 uv16 ondisk = 0) ==== 156+0+17490 (crc 0 0 0) 0x7f18f4083040 con 0x7f18fc00c2b0 2024-09-21T03:40:11.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.744+0000 7f1909119f40 1 -- 172.21.15.38:0/4193049548 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:62 13.14 13:2e2fa760:::200.00000001:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x556215c80940 con 0x7f18fc00c2b0 2024-09-21T03:40:11.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.744+0000 7f1902bd3640 1 -- 172.21.15.38:0/4193049548 <== osd.6 v2:172.21.15.195:6826/4055825675 15 ==== osd_op_reply(62 200.00000001 [getxattrs out=4b] v0'0 uv16 ondisk = 0) ==== 156+0+4 (crc 0 0 0) 0x7f18f4083040 con 0x7f18fc00c2b0 2024-09-21T03:40:11.766 INFO:teuthology.orchestra.run.smithi038.stdout:@�f-� 5 ��������@d<@��XS&YA1^TJ@�fe��scatter_writebehind2 (@�f�A7(@�f�A78`@�fP��8`@�fP��  C���������@�f�A7�C@@��������@�f�A7(@�f�A78`@�fP��8@�f��5@�f��5����NH@�f��5�����������������?@���� �� 2024-09-21T03:40:11.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.744+0000 7f1909119f40 1 -- 172.21.15.38:0/4193049548 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:63 13.14 13:2e2fa760:::200.00000001:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x556215c80940 con 0x7f18fc00c2b0 2024-09-21T03:40:11.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.744+0000 7f1902bd3640 1 -- 172.21.15.38:0/4193049548 <== osd.6 v2:172.21.15.195:6826/4055825675 16 ==== osd_op_reply(63 200.00000001 [omap-get-header] v0'0 uv16 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f18f4083040 con 0x7f18fc00c2b0 2024-09-21T03:40:11.767 INFO:teuthology.orchestra.run.smithi038.stdout:���� �� 2024-09-21T03:40:11.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.744+0000 7f1909119f40 1 -- 172.21.15.38:0/4193049548 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:64 13.14 13:2e2fa760:::200.00000001:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x556215c81950 con 0x7f18fc00c2b0 2024-09-21T03:40:11.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.745+0000 7f1902bd3640 1 -- 172.21.15.38:0/4193049548 <== osd.6 v2:172.21.15.195:6826/4055825675 17 ==== osd_op_reply(64 200.00000001 [omap-get-vals out=5b] v0'0 uv16 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x7f18f4083040 con 0x7f18fc00c2b0 2024-09-21T03:40:11.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.745+0000 7f1909119f40 1 -- 172.21.15.38:0/4193049548 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:65 13.14 13:2f55791f:::606.00000000:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x556215c80940 con 0x7f18fc00c2b0 2024-09-21T03:40:11.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.745+0000 7f1902bd3640 1 -- 172.21.15.38:0/4193049548 <== osd.6 v2:172.21.15.195:6826/4055825675 18 ==== osd_op_reply(65 606.00000000 [read 0~0] v0'0 uv2 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f18f4083040 con 0x7f18fc00c2b0 2024-09-21T03:40:11.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.745+0000 7f1909119f40 1 -- 172.21.15.38:0/4193049548 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:66 13.14 13:2f55791f:::606.00000000:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x556215c80940 con 0x7f18fc00c2b0 2024-09-21T03:40:11.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.745+0000 7f1902bd3640 1 -- 172.21.15.38:0/4193049548 <== osd.6 v2:172.21.15.195:6826/4055825675 19 ==== osd_op_reply(66 606.00000000 [getxattrs out=124b] v0'0 uv2 ondisk = 0) ==== 156+0+124 (crc 0 0 0) 0x7f18f4083040 con 0x7f18fc00c2b0 2024-09-21T03:40:11.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.745+0000 7f1909119f40 1 -- 172.21.15.38:0/4193049548 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:67 13.14 13:2f55791f:::606.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x556215c80940 con 0x7f18fc00c2b0 2024-09-21T03:40:11.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.745+0000 7f1902bd3640 1 -- 172.21.15.38:0/4193049548 <== osd.6 v2:172.21.15.195:6826/4055825675 20 ==== osd_op_reply(67 606.00000000 [omap-get-header out=274b] v0'0 uv2 ondisk = 0) ==== 156+0+274 (crc 0 0 0) 0x7f18f4083040 con 0x7f18fc00c2b0 2024-09-21T03:40:11.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.745+0000 7f1909119f40 1 -- 172.21.15.38:0/4193049548 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:68 13.14 13:2f55791f:::606.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x556215c81950 con 0x7f18fc00c2b0 2024-09-21T03:40:11.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.746+0000 7f1902bd3640 1 -- 172.21.15.38:0/4193049548 <== osd.6 v2:172.21.15.195:6826/4055825675 21 ==== osd_op_reply(68 606.00000000 [omap-get-vals out=5b] v0'0 uv2 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x7f18f4083040 con 0x7f18fc00c2b0 2024-09-21T03:40:11.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.746+0000 7f1909119f40 1 -- 172.21.15.38:0/4193049548 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:69 13.1c 13:3ed09add:::607.00000000:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x556215c80940 con 0x7f18f4063bc0 2024-09-21T03:40:11.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.746+0000 7f19023d2640 1 -- 172.21.15.38:0/4193049548 <== osd.0 v2:172.21.15.38:6800/2381639433 6 ==== osd_op_reply(69 607.00000000 [read 0~0] v0'0 uv2 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f18f800c050 con 0x7f18f4063bc0 2024-09-21T03:40:11.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.746+0000 7f1909119f40 1 -- 172.21.15.38:0/4193049548 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:70 13.1c 13:3ed09add:::607.00000000:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x556215c80ce0 con 0x7f18f4063bc0 2024-09-21T03:40:11.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.746+0000 7f19023d2640 1 -- 172.21.15.38:0/4193049548 <== osd.0 v2:172.21.15.38:6800/2381639433 7 ==== osd_op_reply(70 607.00000000 [getxattrs out=124b] v0'0 uv2 ondisk = 0) ==== 156+0+124 (crc 0 0 0) 0x7f18f800c050 con 0x7f18f4063bc0 2024-09-21T03:40:11.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.747+0000 7f1909119f40 1 -- 172.21.15.38:0/4193049548 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:71 13.1c 13:3ed09add:::607.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x556215c81080 con 0x7f18f4063bc0 2024-09-21T03:40:11.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.747+0000 7f19023d2640 1 -- 172.21.15.38:0/4193049548 <== osd.0 v2:172.21.15.38:6800/2381639433 8 ==== osd_op_reply(71 607.00000000 [omap-get-header out=274b] v0'0 uv2 ondisk = 0) ==== 156+0+274 (crc 0 0 0) 0x7f18f800c050 con 0x7f18f4063bc0 2024-09-21T03:40:11.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.747+0000 7f1909119f40 1 -- 172.21.15.38:0/4193049548 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:72 13.1c 13:3ed09add:::607.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x556215c82430 con 0x7f18f4063bc0 2024-09-21T03:40:11.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.747+0000 7f19023d2640 1 -- 172.21.15.38:0/4193049548 <== osd.0 v2:172.21.15.38:6800/2381639433 9 ==== osd_op_reply(72 607.00000000 [omap-get-vals out=5b] v0'0 uv2 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x7f18f800c050 con 0x7f18f4063bc0 2024-09-21T03:40:11.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.747+0000 7f1909119f40 1 -- 172.21.15.38:0/4193049548 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:73 13.6 13:654134d2:::mds0_openfiles.0:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x556215c81420 con 0x7f18fc01f6f0 2024-09-21T03:40:11.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.748+0000 7f19033d4640 1 -- 172.21.15.38:0/4193049548 <== osd.3 v2:172.21.15.38:6808/1286810294 6 ==== osd_op_reply(73 mds0_openfiles.0 [read 0~0] v0'0 uv2 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f18fc00b040 con 0x7f18fc01f6f0 2024-09-21T03:40:11.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.748+0000 7f1909119f40 1 -- 172.21.15.38:0/4193049548 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:74 13.6 13:654134d2:::mds0_openfiles.0:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x556215c817c0 con 0x7f18fc01f6f0 2024-09-21T03:40:11.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.748+0000 7f19033d4640 1 -- 172.21.15.38:0/4193049548 <== osd.3 v2:172.21.15.38:6808/1286810294 7 ==== osd_op_reply(74 mds0_openfiles.0 [getxattrs out=4b] v0'0 uv2 ondisk = 0) ==== 160+0+4 (crc 0 0 0) 0x7f18fc00b040 con 0x7f18fc01f6f0 2024-09-21T03:40:11.774 INFO:teuthology.orchestra.run.smithi038.stdout:���� �� ��*$- 606.00000000�������������������!�������������������������� ���~_layout��������_parent>8stray6 ���� �� ((88���� �� ��*$- 607.00000000�������������������!�������������������������� ���~_layout��������_parent>8stray7 ���� �� ((88���� �� ��.(1mds0_openfiles.0�������������������!�������������������������� �� 2024-09-21T03:40:11.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.748+0000 7f1909119f40 1 -- 172.21.15.38:0/4193049548 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:75 13.6 13:654134d2:::mds0_openfiles.0:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x556215c81c80 con 0x7f18fc01f6f0 2024-09-21T03:40:11.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.748+0000 7f19033d4640 1 -- 172.21.15.38:0/4193049548 <== osd.3 v2:172.21.15.38:6808/1286810294 8 ==== osd_op_reply(75 mds0_openfiles.0 [omap-get-header out=42b] v0'0 uv2 ondisk = 0) ==== 160+0+42 (crc 0 0 0) 0x7f18fc00b040 con 0x7f18fc01f6f0 2024-09-21T03:40:11.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.748+0000 7f1909119f40 1 -- 172.21.15.38:0/4193049548 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:76 13.6 13:654134d2:::mds0_openfiles.0:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x556215c837e0 con 0x7f18fc01f6f0 2024-09-21T03:40:11.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.749+0000 7f19033d4640 1 -- 172.21.15.38:0/4193049548 <== osd.3 v2:172.21.15.38:6808/1286810294 9 ==== osd_op_reply(76 mds0_openfiles.0 [omap-get-vals out=5b] v0'0 uv2 ondisk = 0) ==== 160+0+5 (crc 0 0 0) 0x7f18fc00b040 con 0x7f18fc01f6f0 2024-09-21T03:40:11.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.749+0000 7f1909119f40 1 -- 172.21.15.38:0/4193049548 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:77 13.e 13:717a0223:::608.00000000:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x556215c827d0 con 0x7f18f4063bc0 2024-09-21T03:40:11.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.749+0000 7f19023d2640 1 -- 172.21.15.38:0/4193049548 <== osd.0 v2:172.21.15.38:6800/2381639433 10 ==== osd_op_reply(77 608.00000000 [read 0~0] v0'0 uv2 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f18f800c050 con 0x7f18f4063bc0 2024-09-21T03:40:11.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.749+0000 7f1909119f40 1 -- 172.21.15.38:0/4193049548 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:78 13.e 13:717a0223:::608.00000000:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x556215c82bc0 con 0x7f18f4063bc0 2024-09-21T03:40:11.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.749+0000 7f19023d2640 1 -- 172.21.15.38:0/4193049548 <== osd.0 v2:172.21.15.38:6800/2381639433 11 ==== osd_op_reply(78 608.00000000 [getxattrs out=124b] v0'0 uv2 ondisk = 0) ==== 156+0+124 (crc 0 0 0) 0x7f18f800c050 con 0x7f18f4063bc0 2024-09-21T03:40:11.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.749+0000 7f1909119f40 1 -- 172.21.15.38:0/4193049548 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:79 13.e 13:717a0223:::608.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x556215c82fb0 con 0x7f18f4063bc0 2024-09-21T03:40:11.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.750+0000 7f19023d2640 1 -- 172.21.15.38:0/4193049548 <== osd.0 v2:172.21.15.38:6800/2381639433 12 ==== osd_op_reply(79 608.00000000 [omap-get-header out=274b] v0'0 uv2 ondisk = 0) ==== 156+0+274 (crc 0 0 0) 0x7f18f800c050 con 0x7f18f4063bc0 2024-09-21T03:40:11.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.750+0000 7f1909119f40 1 -- 172.21.15.38:0/4193049548 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:80 13.e 13:717a0223:::608.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x556215c82fb0 con 0x7f18f4063bc0 2024-09-21T03:40:11.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.750+0000 7f19023d2640 1 -- 172.21.15.38:0/4193049548 <== osd.0 v2:172.21.15.38:6800/2381639433 13 ==== osd_op_reply(80 608.00000000 [omap-get-vals out=5b] v0'0 uv2 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x7f18f800c050 con 0x7f18f4063bc0 2024-09-21T03:40:11.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.750+0000 7f1909119f40 1 -- 172.21.15.38:0/4193049548 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:81 13.15 13:a93a17c2:::604.00000000:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x556215c82fb0 con 0x7f18fc01f6f0 2024-09-21T03:40:11.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.750+0000 7f19033d4640 1 -- 172.21.15.38:0/4193049548 <== osd.3 v2:172.21.15.38:6808/1286810294 10 ==== osd_op_reply(81 604.00000000 [read 0~0] v0'0 uv2 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f18fc00b040 con 0x7f18fc01f6f0 2024-09-21T03:40:11.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.750+0000 7f1909119f40 1 -- 172.21.15.38:0/4193049548 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:82 13.15 13:a93a17c2:::604.00000000:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x556215c833a0 con 0x7f18fc01f6f0 2024-09-21T03:40:11.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.751+0000 7f19033d4640 1 -- 172.21.15.38:0/4193049548 <== osd.3 v2:172.21.15.38:6808/1286810294 11 ==== osd_op_reply(82 604.00000000 [getxattrs out=124b] v0'0 uv2 ondisk = 0) ==== 156+0+124 (crc 0 0 0) 0x7f18fc00b040 con 0x7f18fc01f6f0 2024-09-21T03:40:11.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.751+0000 7f1909119f40 1 -- 172.21.15.38:0/4193049548 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:83 13.15 13:a93a17c2:::604.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x556215c83b80 con 0x7f18fc01f6f0 2024-09-21T03:40:11.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.751+0000 7f19033d4640 1 -- 172.21.15.38:0/4193049548 <== osd.3 v2:172.21.15.38:6808/1286810294 12 ==== osd_op_reply(83 604.00000000 [omap-get-header out=274b] v0'0 uv2 ondisk = 0) ==== 156+0+274 (crc 0 0 0) 0x7f18fc00b040 con 0x7f18fc01f6f0 2024-09-21T03:40:11.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.751+0000 7f1909119f40 1 -- 172.21.15.38:0/4193049548 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:84 13.15 13:a93a17c2:::604.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x556215c84b90 con 0x7f18fc01f6f0 2024-09-21T03:40:11.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.751+0000 7f19033d4640 1 -- 172.21.15.38:0/4193049548 <== osd.3 v2:172.21.15.38:6808/1286810294 13 ==== osd_op_reply(84 604.00000000 [omap-get-vals out=5b] v0'0 uv2 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x7f18fc00b040 con 0x7f18fc01f6f0 2024-09-21T03:40:11.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.751+0000 7f1909119f40 1 -- 172.21.15.38:0/4193049548 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:85 13.15 13:aa448500:::500.00000000:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x556215c83b80 con 0x7f18fc01f6f0 2024-09-21T03:40:11.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.752+0000 7f19033d4640 1 -- 172.21.15.38:0/4193049548 <== osd.3 v2:172.21.15.38:6808/1286810294 14 ==== osd_op_reply(85 500.00000000 [read 0~90 out=90b] v0'0 uv3 ondisk = 0) ==== 156+0+90 (crc 0 0 0) 0x7f18fc00b040 con 0x7f18fc01f6f0 2024-09-21T03:40:11.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.752+0000 7f1909119f40 1 -- 172.21.15.38:0/4193049548 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:86 13.15 13:aa448500:::500.00000000:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x556215c83b80 con 0x7f18fc01f6f0 2024-09-21T03:40:11.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.752+0000 7f19033d4640 1 -- 172.21.15.38:0/4193049548 <== osd.3 v2:172.21.15.38:6808/1286810294 15 ==== osd_op_reply(86 500.00000000 [getxattrs out=4b] v0'0 uv3 ondisk = 0) ==== 156+0+4 (crc 0 0 0) 0x7f18fc00b040 con 0x7f18fc01f6f0 2024-09-21T03:40:11.781 INFO:teuthology.orchestra.run.smithi038.stdout:���� ��4.*ceph fs volume v011 ���� �� ��*$- 608.00000000�������������������!�������������������������� ���~_layout��������_parent>8stray8 ���� �� ((88���� �� ��*$- 604.00000000�������������������!�������������������������� ���~_layout��������_parent>8stray4 ���� �� ((88���� �� ��*$- 500.00000000�������������������!�������������������������� ��tnZZTceph fs volume v011@@@@@@ ���� �� 2024-09-21T03:40:11.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.752+0000 7f1909119f40 1 -- 172.21.15.38:0/4193049548 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:87 13.15 13:aa448500:::500.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x556215c83b80 con 0x7f18fc01f6f0 2024-09-21T03:40:11.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.752+0000 7f19033d4640 1 -- 172.21.15.38:0/4193049548 <== osd.3 v2:172.21.15.38:6808/1286810294 16 ==== osd_op_reply(87 500.00000000 [omap-get-header] v0'0 uv3 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f18fc00b040 con 0x7f18fc01f6f0 2024-09-21T03:40:11.782 INFO:teuthology.orchestra.run.smithi038.stdout:���� �� 2024-09-21T03:40:11.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.752+0000 7f1909119f40 1 -- 172.21.15.38:0/4193049548 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:88 13.15 13:aa448500:::500.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x556215c84b90 con 0x7f18fc01f6f0 2024-09-21T03:40:11.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.752+0000 7f19033d4640 1 -- 172.21.15.38:0/4193049548 <== osd.3 v2:172.21.15.38:6808/1286810294 17 ==== osd_op_reply(88 500.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x7f18fc00b040 con 0x7f18fc01f6f0 2024-09-21T03:40:11.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.753+0000 7f1909119f40 1 -- 172.21.15.38:0/4193049548 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:89 13.d 13:b50e409b:::mds_snaptable:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x556215c83b80 con 0x7f18f4063bc0 2024-09-21T03:40:11.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.753+0000 7f19023d2640 1 -- 172.21.15.38:0/4193049548 <== osd.0 v2:172.21.15.38:6800/2381639433 14 ==== osd_op_reply(89 mds_snaptable [read 0~70 out=70b] v0'0 uv1 ondisk = 0) ==== 157+0+70 (crc 0 0 0) 0x7f18f800c050 con 0x7f18f4063bc0 2024-09-21T03:40:11.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.753+0000 7f1909119f40 1 -- 172.21.15.38:0/4193049548 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:90 13.d 13:b50e409b:::mds_snaptable:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x556215c83b80 con 0x7f18f4063bc0 2024-09-21T03:40:11.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.753+0000 7f19023d2640 1 -- 172.21.15.38:0/4193049548 <== osd.0 v2:172.21.15.38:6800/2381639433 15 ==== osd_op_reply(90 mds_snaptable [getxattrs out=4b] v0'0 uv1 ondisk = 0) ==== 157+0+4 (crc 0 0 0) 0x7f18f800c050 con 0x7f18f4063bc0 2024-09-21T03:40:11.784 INFO:teuthology.orchestra.run.smithi038.stdout:���� �� ��+%. mds_snaptable�������������������!�������������������������� ��`ZFF4���� �� 2024-09-21T03:40:11.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.753+0000 7f1909119f40 1 -- 172.21.15.38:0/4193049548 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:91 13.d 13:b50e409b:::mds_snaptable:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x556215c83b80 con 0x7f18f4063bc0 2024-09-21T03:40:11.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.754+0000 7f19023d2640 1 -- 172.21.15.38:0/4193049548 <== osd.0 v2:172.21.15.38:6800/2381639433 16 ==== osd_op_reply(91 mds_snaptable [omap-get-header] v0'0 uv1 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f18f800c050 con 0x7f18f4063bc0 2024-09-21T03:40:11.785 INFO:teuthology.orchestra.run.smithi038.stdout:���� �� 2024-09-21T03:40:11.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.754+0000 7f1909119f40 1 -- 172.21.15.38:0/4193049548 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:92 13.d 13:b50e409b:::mds_snaptable:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x556215c84b90 con 0x7f18f4063bc0 2024-09-21T03:40:11.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.754+0000 7f19023d2640 1 -- 172.21.15.38:0/4193049548 <== osd.0 v2:172.21.15.38:6800/2381639433 17 ==== osd_op_reply(92 mds_snaptable [omap-get-vals out=5b] v0'0 uv1 ondisk = 0) ==== 157+0+5 (crc 0 0 0) 0x7f18f800c050 con 0x7f18f4063bc0 2024-09-21T03:40:11.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.754+0000 7f1909119f40 1 -- 172.21.15.38:0/4193049548 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:93 13.1d 13:b871830b:::605.00000000:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x556215c83b80 con 0x556215b27c30 2024-09-21T03:40:11.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.754+0000 7f19033d4640 1 -- 172.21.15.38:0/4193049548 <== osd.1 v2:172.21.15.38:6816/1491818732 9 ==== osd_op_reply(93 605.00000000 [read 0~0] v0'0 uv2 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f18fc00b040 con 0x556215b27c30 2024-09-21T03:40:11.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.754+0000 7f1909119f40 1 -- 172.21.15.38:0/4193049548 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:94 13.1d 13:b871830b:::605.00000000:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x556215c83b80 con 0x556215b27c30 2024-09-21T03:40:11.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.754+0000 7f19033d4640 1 -- 172.21.15.38:0/4193049548 <== osd.1 v2:172.21.15.38:6816/1491818732 10 ==== osd_op_reply(94 605.00000000 [getxattrs out=124b] v0'0 uv2 ondisk = 0) ==== 156+0+124 (crc 0 0 0) 0x7f18fc00b040 con 0x556215b27c30 2024-09-21T03:40:11.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.754+0000 7f1909119f40 1 -- 172.21.15.38:0/4193049548 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:95 13.1d 13:b871830b:::605.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x556215c83c40 con 0x556215b27c30 2024-09-21T03:40:11.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.755+0000 7f19033d4640 1 -- 172.21.15.38:0/4193049548 <== osd.1 v2:172.21.15.38:6816/1491818732 11 ==== osd_op_reply(95 605.00000000 [omap-get-header out=274b] v0'0 uv2 ondisk = 0) ==== 156+0+274 (crc 0 0 0) 0x7f18fc00b040 con 0x556215b27c30 2024-09-21T03:40:11.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.755+0000 7f1909119f40 1 -- 172.21.15.38:0/4193049548 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:96 13.1d 13:b871830b:::605.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x556215c83c00 con 0x556215b27c30 2024-09-21T03:40:11.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.755+0000 7f19033d4640 1 -- 172.21.15.38:0/4193049548 <== osd.1 v2:172.21.15.38:6816/1491818732 12 ==== osd_op_reply(96 605.00000000 [omap-get-vals out=5b] v0'0 uv2 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x7f18fc00b040 con 0x556215b27c30 2024-09-21T03:40:11.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.755+0000 7f1909119f40 1 -- 172.21.15.38:0/4193049548 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:97 13.13 13:c91d4a1d:::mds0_inotable:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x556215c83b80 con 0x556215b27c30 2024-09-21T03:40:11.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.755+0000 7f19033d4640 1 -- 172.21.15.38:0/4193049548 <== osd.1 v2:172.21.15.38:6816/1491818732 13 ==== osd_op_reply(97 mds0_inotable [read 0~34 out=34b] v0'0 uv3 ondisk = 0) ==== 157+0+34 (crc 0 0 0) 0x7f18fc00b040 con 0x556215b27c30 2024-09-21T03:40:11.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.755+0000 7f1909119f40 1 -- 172.21.15.38:0/4193049548 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:98 13.13 13:c91d4a1d:::mds0_inotable:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x556215c83b80 con 0x556215b27c30 2024-09-21T03:40:11.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.756+0000 7f19033d4640 1 -- 172.21.15.38:0/4193049548 <== osd.1 v2:172.21.15.38:6816/1491818732 14 ==== osd_op_reply(98 mds0_inotable [getxattrs out=4b] v0'0 uv3 ondisk = 0) ==== 157+0+4 (crc 0 0 0) 0x7f18fc00b040 con 0x556215b27c30 2024-09-21T03:40:11.789 INFO:teuthology.orchestra.run.smithi038.stdout:���� �� ��*$- 605.00000000�������������������!�������������������������� ���~_layout��������_parent>8stray5 ���� �� ((88���� �� ��+%. mds0_inotable�������������������!�������������������������� ��<6""��������� �� 2024-09-21T03:40:11.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.756+0000 7f1909119f40 1 -- 172.21.15.38:0/4193049548 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:99 13.13 13:c91d4a1d:::mds0_inotable:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x556215c83b80 con 0x556215b27c30 2024-09-21T03:40:11.790 INFO:teuthology.orchestra.run.smithi038.stdout:���� �� 2024-09-21T03:40:11.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.756+0000 7f19033d4640 1 -- 172.21.15.38:0/4193049548 <== osd.1 v2:172.21.15.38:6816/1491818732 15 ==== osd_op_reply(99 mds0_inotable [omap-get-header] v0'0 uv3 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f18fc00b040 con 0x556215b27c30 2024-09-21T03:40:11.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.756+0000 7f1909119f40 1 -- 172.21.15.38:0/4193049548 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:100 13.13 13:c91d4a1d:::mds0_inotable:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x556215c83b80 con 0x556215b27c30 2024-09-21T03:40:11.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.756+0000 7f19033d4640 1 -- 172.21.15.38:0/4193049548 <== osd.1 v2:172.21.15.38:6816/1491818732 16 ==== osd_op_reply(100 mds0_inotable [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 157+0+5 (crc 0 0 0) 0x7f18fc00b040 con 0x556215b27c30 2024-09-21T03:40:11.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.756+0000 7f1909119f40 1 -- 172.21.15.38:0/4193049548 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:101 13.13 13:cd5a64a3:::100.00000000:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x556215c83b80 con 0x556215b27c30 2024-09-21T03:40:11.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.756+0000 7f19033d4640 1 -- 172.21.15.38:0/4193049548 <== osd.1 v2:172.21.15.38:6816/1491818732 17 ==== osd_op_reply(101 100.00000000 [read 0~0] v0'0 uv1 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f18fc00b040 con 0x556215b27c30 2024-09-21T03:40:11.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.756+0000 7f1909119f40 1 -- 172.21.15.38:0/4193049548 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:102 13.13 13:cd5a64a3:::100.00000000:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x556215c83b80 con 0x556215b27c30 2024-09-21T03:40:11.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.757+0000 7f19033d4640 1 -- 172.21.15.38:0/4193049548 <== osd.1 v2:172.21.15.38:6816/1491818732 18 ==== osd_op_reply(102 100.00000000 [getxattrs out=4b] v0'0 uv1 ondisk = 0) ==== 156+0+4 (crc 0 0 0) 0x7f18fc00b040 con 0x556215b27c30 2024-09-21T03:40:11.792 INFO:teuthology.orchestra.run.smithi038.stdout:���� �� ��*$- 100.00000000�������������������!�������������������������� �� 2024-09-21T03:40:11.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.757+0000 7f1909119f40 1 -- 172.21.15.38:0/4193049548 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:103 13.13 13:cd5a64a3:::100.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x556215c83b80 con 0x556215b27c30 2024-09-21T03:40:11.792 INFO:teuthology.orchestra.run.smithi038.stdout:���� �� ( 2024-09-21T03:40:11.792 INFO:teuthology.orchestra.run.smithi038.stdout:( 2024-09-21T03:40:11.792 INFO:teuthology.orchestra.run.smithi038.stdout:8 2024-09-21T03:40:11.792 INFO:teuthology.orchestra.run.smithi038.stdout:@�f-� 58 2024-09-21T03:40:11.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.757+0000 7f19033d4640 1 -- 172.21.15.38:0/4193049548 <== osd.1 v2:172.21.15.38:6816/1491818732 19 ==== osd_op_reply(103 100.00000000 [omap-get-header out=274b] v0'0 uv1 ondisk = 0) ==== 156+0+274 (crc 0 0 0) 0x7f18fc00b040 con 0x556215b27c30 2024-09-21T03:40:11.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.757+0000 7f1909119f40 1 -- 172.21.15.38:0/4193049548 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:104 13.13 13:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x556215c83b80 con 0x556215b27c30 2024-09-21T03:40:11.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.757+0000 7f19033d4640 1 -- 172.21.15.38:0/4193049548 <== osd.1 v2:172.21.15.38:6816/1491818732 20 ==== osd_op_reply(104 100.00000000 [omap-get-vals out=5195b] v0'0 uv1 ondisk = 0) ==== 156+0+5195 (crc 0 0 0) 0x7f18fc00b040 con 0x556215b27c30 2024-09-21T03:40:11.794 INFO:teuthology.orchestra.run.smithi038.stdout:@�f-� 5���� ��PJ 2024-09-21T03:40:11.794 INFO:teuthology.orchestra.run.smithi038.stdout: stray0_head�i���@�fLh5@A����������������@�fLh5(8@�fLh58@�fLh5@�fLh5������������ stray1_head�i���@�f+y5@A����������������@�f+y5(8@�f+y58@�f+y5@�f+y5������������ stray2_head�i���@�f5�5@A����������������@�f5�5(8@�f5�58@�f5�5@�f5�5������������ stray3_head�i���@�f�5@A����������������@�f�5(8@�f�58@�f�5@�f�5������������ stray4_head�i���@�f�5@A����������������@�f�5(8@�f�58@�f�5@�f�5������������ stray5_head�i���@�f.X5@A����������������@�f.X5(8@�f.X58@�f.X5@�f.X5������������ stray6_head�i���@�f�r5@A����������������@�f�r5(8@�f�r58@�f�r5@�f�r5������������ stray7_head�i���@�f 2024-09-21T03:40:11.794 INFO:teuthology.orchestra.run.smithi038.stdout:5@A����������������@�f 2024-09-21T03:40:11.794 INFO:teuthology.orchestra.run.smithi038.stdout:5(8@�f 2024-09-21T03:40:11.794 INFO:teuthology.orchestra.run.smithi038.stdout:58@�f 2024-09-21T03:40:11.794 INFO:teuthology.orchestra.run.smithi038.stdout:5@�f 2024-09-21T03:40:11.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.757+0000 7f1909119f40 1 -- 172.21.15.38:0/4193049548 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:105 13.b 13:d0630e4c:::mds0_sessionmap:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x556215c83fe0 con 0x7f18fc01f6f0 2024-09-21T03:40:11.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.758+0000 7f19033d4640 1 -- 172.21.15.38:0/4193049548 <== osd.3 v2:172.21.15.38:6808/1286810294 18 ==== osd_op_reply(105 mds0_sessionmap [read 0~0] v0'0 uv3 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f18fc00b040 con 0x7f18fc01f6f0 2024-09-21T03:40:11.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.758+0000 7f1909119f40 1 -- 172.21.15.38:0/4193049548 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:106 13.b 13:d0630e4c:::mds0_sessionmap:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x556215c83fe0 con 0x7f18fc01f6f0 2024-09-21T03:40:11.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.758+0000 7f19033d4640 1 -- 172.21.15.38:0/4193049548 <== osd.3 v2:172.21.15.38:6808/1286810294 19 ==== osd_op_reply(106 mds0_sessionmap [getxattrs out=4b] v0'0 uv3 ondisk = 0) ==== 159+0+4 (crc 0 0 0) 0x7f18fc00b040 con 0x7f18fc01f6f0 2024-09-21T03:40:11.795 INFO:teuthology.orchestra.run.smithi038.stdout:5������������ stray8_head�i���@�f"5@A����������������@�f"5(8@�f"58@�f"5@�f"5������������ stray9_head�i��� @�f-� 5@A����������������@�f-� 5(8@�f-� 58@�f-� 5@�f-� 5���������������� �� ��-'0mds0_sessionmap�������������������!�������������������������� �� 2024-09-21T03:40:11.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.758+0000 7f1909119f40 1 -- 172.21.15.38:0/4193049548 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:107 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x556215c83b80 con 0x7f18fc01f6f0 2024-09-21T03:40:11.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.758+0000 7f19033d4640 1 -- 172.21.15.38:0/4193049548 <== osd.3 v2:172.21.15.38:6808/1286810294 20 ==== osd_op_reply(107 mds0_sessionmap [omap-get-header out=14b] v0'0 uv3 ondisk = 0) ==== 159+0+14 (crc 0 0 0) 0x7f18fc00b040 con 0x7f18fc01f6f0 2024-09-21T03:40:11.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.758+0000 7f1909119f40 1 -- 172.21.15.38:0/4193049548 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:108 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x556215c83b80 con 0x7f18fc01f6f0 2024-09-21T03:40:11.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.758+0000 7f19033d4640 1 -- 172.21.15.38:0/4193049548 <== osd.3 v2:172.21.15.38:6808/1286810294 21 ==== osd_op_reply(108 mds0_sessionmap [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 159+0+5 (crc 0 0 0) 0x7f18fc00b040 con 0x7f18fc01f6f0 2024-09-21T03:40:11.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.758+0000 7f1909119f40 1 -- 172.21.15.38:0/4193049548 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:109 13.7 13:e0b41d06:::609.00000000:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x556215c83b80 con 0x7f18fc01f6f0 2024-09-21T03:40:11.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.759+0000 7f19033d4640 1 -- 172.21.15.38:0/4193049548 <== osd.3 v2:172.21.15.38:6808/1286810294 22 ==== osd_op_reply(109 609.00000000 [read 0~0] v0'0 uv2 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f18fc00b040 con 0x7f18fc01f6f0 2024-09-21T03:40:11.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.759+0000 7f1909119f40 1 -- 172.21.15.38:0/4193049548 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:110 13.7 13:e0b41d06:::609.00000000:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x556215c83b80 con 0x7f18fc01f6f0 2024-09-21T03:40:11.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.759+0000 7f19033d4640 1 -- 172.21.15.38:0/4193049548 <== osd.3 v2:172.21.15.38:6808/1286810294 23 ==== osd_op_reply(110 609.00000000 [getxattrs out=124b] v0'0 uv2 ondisk = 0) ==== 156+0+124 (crc 0 0 0) 0x7f18fc00b040 con 0x7f18fc01f6f0 2024-09-21T03:40:11.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.759+0000 7f1909119f40 1 -- 172.21.15.38:0/4193049548 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:111 13.7 13:e0b41d06:::609.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x556215c83ff0 con 0x7f18fc01f6f0 2024-09-21T03:40:11.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.759+0000 7f19033d4640 1 -- 172.21.15.38:0/4193049548 <== osd.3 v2:172.21.15.38:6808/1286810294 24 ==== osd_op_reply(111 609.00000000 [omap-get-header out=274b] v0'0 uv2 ondisk = 0) ==== 156+0+274 (crc 0 0 0) 0x7f18fc00b040 con 0x7f18fc01f6f0 2024-09-21T03:40:11.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.759+0000 7f1909119f40 1 -- 172.21.15.38:0/4193049548 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:112 13.7 13:e0b41d06:::609.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x556215c83ff0 con 0x7f18fc01f6f0 2024-09-21T03:40:11.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.760+0000 7f19033d4640 1 -- 172.21.15.38:0/4193049548 <== osd.3 v2:172.21.15.38:6808/1286810294 25 ==== osd_op_reply(112 609.00000000 [omap-get-vals out=5b] v0'0 uv2 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x7f18fc00b040 con 0x7f18fc01f6f0 2024-09-21T03:40:11.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.760+0000 7f1909119f40 1 -- 172.21.15.38:0/4193049548 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x556215c83c10 con 0x7f18f406a550 2024-09-21T03:40:11.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.760+0000 7f1902bd3640 1 -- 172.21.15.38:0/4193049548 <== osd.2 v2:172.21.15.38:6824/2577858902 4 ==== osd_op_reply(113 400.00000000 [read 0~22 out=22b] v0'0 uv1 ondisk = 0) ==== 156+0+22 (crc 0 0 0) 0x7f18f4083040 con 0x7f18f406a550 2024-09-21T03:40:11.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.760+0000 7f1909119f40 1 -- 172.21.15.38:0/4193049548 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:114 13.f 13:f1f69726:::400.00000000:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x556215c83c10 con 0x7f18f406a550 2024-09-21T03:40:11.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.761+0000 7f1902bd3640 1 -- 172.21.15.38:0/4193049548 <== osd.2 v2:172.21.15.38:6824/2577858902 5 ==== osd_op_reply(114 400.00000000 [getxattrs out=4b] v0'0 uv1 ondisk = 0) ==== 156+0+4 (crc 0 0 0) 0x7f18f4083040 con 0x7f18f406a550 2024-09-21T03:40:11.801 INFO:teuthology.orchestra.run.smithi038.stdout:���� ������ �� ��*$- 609.00000000�������������������!�������������������������� ���~_layout��������_parent>8 stray9 ���� �� ((88���� �� ��*$- 400.00000000�������������������!�������������������������� ��0*���� �� 2024-09-21T03:40:11.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.761+0000 7f1909119f40 1 -- 172.21.15.38:0/4193049548 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:115 13.f 13:f1f69726:::400.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x556215c83c10 con 0x7f18f406a550 2024-09-21T03:40:11.801 INFO:teuthology.orchestra.run.smithi038.stdout:���� �� 2024-09-21T03:40:11.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.761+0000 7f1902bd3640 1 -- 172.21.15.38:0/4193049548 <== osd.2 v2:172.21.15.38:6824/2577858902 6 ==== osd_op_reply(115 400.00000000 [omap-get-header] v0'0 uv1 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f18f4083040 con 0x7f18f406a550 2024-09-21T03:40:11.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.761+0000 7f1909119f40 1 -- 172.21.15.38:0/4193049548 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:116 13.f 13:f1f69726:::400.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x556215c83c10 con 0x7f18f406a550 2024-09-21T03:40:11.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.761+0000 7f1902bd3640 1 -- 172.21.15.38:0/4193049548 <== osd.2 v2:172.21.15.38:6824/2577858902 7 ==== osd_op_reply(116 400.00000000 [omap-get-vals out=5b] v0'0 uv1 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x7f18f4083040 con 0x7f18f406a550 2024-09-21T03:40:11.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.761+0000 7f1909119f40 1 -- 172.21.15.38:0/4193049548 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:117 13.1f 13:fe07bba1:::100.00000000.inode:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x556215c83c10 con 0x556215b27c30 2024-09-21T03:40:11.803 INFO:teuthology.orchestra.run.smithi038.stdout:���� �� ��0*3100.00000000.inode�������������������!�������������������������� ��`ZFFceph fs volume v011)�@�fF� 5@A����������������@�fF� 5( 2024-09-21T03:40:11.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.762+0000 7f19033d4640 1 -- 172.21.15.38:0/4193049548 <== osd.1 v2:172.21.15.38:6816/1491818732 21 ==== osd_op_reply(117 100.00000000.inode [read 0~582 out=582b] v0'0 uv3 ondisk = 0) ==== 162+0+582 (crc 0 0 0) 0x7f18fc00b040 con 0x556215b27c30 2024-09-21T03:40:11.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.762+0000 7f1909119f40 1 -- 172.21.15.38:0/4193049548 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:118 13.1f 13:fe07bba1:::100.00000000.inode:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x556215c83c10 con 0x556215b27c30 2024-09-21T03:40:11.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.762+0000 7f19033d4640 1 -- 172.21.15.38:0/4193049548 <== osd.1 v2:172.21.15.38:6816/1491818732 22 ==== osd_op_reply(118 100.00000000.inode [getxattrs out=4b] v0'0 uv3 ondisk = 0) ==== 162+0+4 (crc 0 0 0) 0x7f18fc00b040 con 0x556215b27c30 2024-09-21T03:40:11.804 INFO:teuthology.orchestra.run.smithi038.stdout:8 @�f-� 58 @�f-� 5@�fF� 5����NH������������ �� 2024-09-21T03:40:11.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.762+0000 7f1909119f40 1 -- 172.21.15.38:0/4193049548 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:119 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x556215c84050 con 0x556215b27c30 2024-09-21T03:40:11.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.762+0000 7f19033d4640 1 -- 172.21.15.38:0/4193049548 <== osd.1 v2:172.21.15.38:6816/1491818732 23 ==== osd_op_reply(119 100.00000000.inode [omap-get-header] v0'0 uv3 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f18fc00b040 con 0x556215b27c30 2024-09-21T03:40:11.804 INFO:teuthology.orchestra.run.smithi038.stdout:���� �� 2024-09-21T03:40:11.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.762+0000 7f1909119f40 1 -- 172.21.15.38:0/4193049548 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:120 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x556215c84050 con 0x556215b27c30 2024-09-21T03:40:11.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.763+0000 7f19033d4640 1 -- 172.21.15.38:0/4193049548 <== osd.1 v2:172.21.15.38:6816/1491818732 24 ==== osd_op_reply(120 100.00000000.inode [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 162+0+5 (crc 0 0 0) 0x7f18fc00b040 con 0x556215b27c30 2024-09-21T03:40:11.805 INFO:teuthology.orchestra.run.smithi038.stdout:���� �� ��("+ 2024-09-21T03:40:11.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.763+0000 7f1909119f40 1 -- 172.21.15.38:0/4193049548 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:121 13.1f 13:ff5b34d6:::1.00000000:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x556215c84000 con 0x556215b27c30 2024-09-21T03:40:11.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.763+0000 7f19033d4640 1 -- 172.21.15.38:0/4193049548 <== osd.1 v2:172.21.15.38:6816/1491818732 25 ==== osd_op_reply(121 1.00000000 [read 0~0] v0'0 uv6 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f18fc00b040 con 0x556215b27c30 2024-09-21T03:40:11.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.763+0000 7f1909119f40 1 -- 172.21.15.38:0/4193049548 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:122 13.1f 13:ff5b34d6:::1.00000000:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x556215c84000 con 0x556215b27c30 2024-09-21T03:40:11.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.763+0000 7f19033d4640 1 -- 172.21.15.38:0/4193049548 <== osd.1 v2:172.21.15.38:6816/1491818732 26 ==== osd_op_reply(122 1.00000000 [getxattrs out=92b] v0'0 uv6 ondisk = 0) ==== 154+0+92 (crc 0 0 0) 0x7f18fc00b040 con 0x556215b27c30 2024-09-21T03:40:11.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.763+0000 7f1909119f40 1 -- 172.21.15.38:0/4193049548 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:123 13.1f 13:ff5b34d6:::1.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x556215c84000 con 0x556215b27c30 2024-09-21T03:40:11.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.764+0000 7f19033d4640 1 -- 172.21.15.38:0/4193049548 <== osd.1 v2:172.21.15.38:6816/1491818732 27 ==== osd_op_reply(123 1.00000000 [omap-get-header out=274b] v0'0 uv6 ondisk = 0) ==== 154+0+274 (crc 0 0 0) 0x7f18fc00b040 con 0x556215b27c30 2024-09-21T03:40:11.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.764+0000 7f1909119f40 1 -- 172.21.15.38:0/4193049548 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:124 13.1f 13:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x556215c84000 con 0x556215b27c30 2024-09-21T03:40:11.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.764+0000 7f19033d4640 1 -- 172.21.15.38:0/4193049548 <== osd.1 v2:172.21.15.38:6816/1491818732 28 ==== osd_op_reply(124 1.00000000 [omap-get-vals out=585b] v0'0 uv6 ondisk = 0) ==== 154+0+585 (crc 0 0 0) 0x7f18fc00b040 con 0x556215b27c30 2024-09-21T03:40:11.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.764+0000 7f1909119f40 1 -- 172.21.15.38:0/4193049548 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x7f18f4063bc0 msgr2=0x7f18f407a9e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:11.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.764+0000 7f1909119f40 1 --2- 172.21.15.38:0/4193049548 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x7f18f4063bc0 0x7f18f407a9e0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:11.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.764+0000 7f1909119f40 1 -- 172.21.15.38:0/4193049548 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x556215b27c30 msgr2=0x556215c74560 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:11.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.764+0000 7f1909119f40 1 --2- 172.21.15.38:0/4193049548 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x556215b27c30 0x556215c74560 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:11.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.764+0000 7f1909119f40 1 -- 172.21.15.38:0/4193049548 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x7f18f406a550 msgr2=0x7f18f4079370 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:11.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.764+0000 7f1909119f40 1 --2- 172.21.15.38:0/4193049548 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x7f18f406a550 0x7f18f4079370 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:11.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.764+0000 7f1909119f40 1 -- 172.21.15.38:0/4193049548 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x7f18fc01f6f0 msgr2=0x7f18fc055920 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:11.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.764+0000 7f1909119f40 1 --2- 172.21.15.38:0/4193049548 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x7f18fc01f6f0 0x7f18fc055920 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:11.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.764+0000 7f1909119f40 1 -- 172.21.15.38:0/4193049548 >> [v2:172.21.15.195:6802/3401837571,v1:172.21.15.195:6803/3401837571] conn(0x7f18f406cdb0 msgr2=0x7f18f4061d30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:11.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.764+0000 7f1909119f40 1 --2- 172.21.15.38:0/4193049548 >> [v2:172.21.15.195:6802/3401837571,v1:172.21.15.195:6803/3401837571] conn(0x7f18f406cdb0 0x7f18f4061d30 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:11.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.764+0000 7f19033d4640 1 -- 172.21.15.38:0/4193049548 reap_dead start 2024-09-21T03:40:11.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.764+0000 7f19033d4640 1 -- 172.21.15.38:0/4193049548 reap_dead start 2024-09-21T03:40:11.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.764+0000 7f1909119f40 1 -- 172.21.15.38:0/4193049548 >> [v2:172.21.15.195:6810/2080960180,v1:172.21.15.195:6811/2080960180] conn(0x7f18f4064be0 msgr2=0x7f18f4062630 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:11.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.764+0000 7f1909119f40 1 --2- 172.21.15.38:0/4193049548 >> [v2:172.21.15.195:6810/2080960180,v1:172.21.15.195:6811/2080960180] conn(0x7f18f4064be0 0x7f18f4062630 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:11.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.764+0000 7f1909119f40 1 -- 172.21.15.38:0/4193049548 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x7f18fc00c2b0 msgr2=0x7f18fc071410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:11.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.764+0000 7f1909119f40 1 --2- 172.21.15.38:0/4193049548 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x7f18fc00c2b0 0x7f18fc071410 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:11.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.764+0000 7f1909119f40 1 -- 172.21.15.38:0/4193049548 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x7f18fc022d70 msgr2=0x7f18fc061930 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:11.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.764+0000 7f1909119f40 1 --2- 172.21.15.38:0/4193049548 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x7f18fc022d70 0x7f18fc061930 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:11.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.764+0000 7f1909119f40 1 -- 172.21.15.38:0/4193049548 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f18f0077570 msgr2=0x7f18f0079a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:11.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.764+0000 7f1909119f40 1 --2- 172.21.15.38:0/4193049548 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f18f0077570 0x7f18f0079a80 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x556215b35c10 tx=0x7f18f8004880 comp rx=0 tx=0).stop 2024-09-21T03:40:11.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.764+0000 7f1909119f40 1 -- 172.21.15.38:0/4193049548 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556215b08240 msgr2=0x556215b2a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:11.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.764+0000 7f1909119f40 1 --2- 172.21.15.38:0/4193049548 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556215b08240 0x556215b2a1b0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f18f4016650 tx=0x7f18f4016680 comp rx=0 tx=0).stop 2024-09-21T03:40:11.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.764+0000 7f19033d4640 1 -- 172.21.15.38:0/4193049548 reap_dead start 2024-09-21T03:40:11.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.764+0000 7f19033d4640 1 -- 172.21.15.38:0/4193049548 reap_dead start 2024-09-21T03:40:11.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.764+0000 7f1909119f40 1 -- 172.21.15.38:0/4193049548 shutdown_connections 2024-09-21T03:40:11.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.764+0000 7f1909119f40 1 -- 172.21.15.38:0/4193049548 >> 172.21.15.38:0/4193049548 conn(0x556215a26ac0 msgr2=0x556215bda070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:11.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.764+0000 7f1909119f40 1 -- 172.21.15.38:0/4193049548 shutdown_connections 2024-09-21T03:40:11.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.764+0000 7f1909119f40 1 -- 172.21.15.38:0/4193049548 wait complete. 2024-09-21T03:40:11.812 INFO:teuthology.orchestra.run.smithi038.stdout:1.00000000�������������������!�������������������������� ��d^_layout@@_parent ���� ��  (@�f�A7(@�f�A78`@�fP��8@�f�A7���� ��NH subdir_head1i"�@�f%��A�5����������������@�f%�@�f�A7(@�f%�8`@�fP��8`@�fP�� ��������@�f�A7����security.selinux%unconfined_u:object_r:unlabeled_t:s0������������ �� �� 2024-09-21T03:40:11.812 DEBUG:teuthology.orchestra.run.smithi038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata ls 2024-09-21T03:40:11.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.863+0000 7f0fbc95ff40 1 Processor -- start 2024-09-21T03:40:11.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.863+0000 7f0fbc95ff40 1 -- start start 2024-09-21T03:40:11.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.864+0000 7f0fbc95ff40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bed91f9240 0x55bed92ca640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:11.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.864+0000 7f0fbc95ff40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55bed92b5610 0x55bed92cb970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:11.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.864+0000 7f0fbc95ff40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bed9224d80 0x55bed92ce8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:11.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.864+0000 7f0fbc95ff40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55bed9146d60 con 0x55bed91f9240 2024-09-21T03:40:11.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.864+0000 7f0fbc95ff40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55bed91a5430 con 0x55bed9224d80 2024-09-21T03:40:11.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.864+0000 7f0fbc95ff40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55bed92ccd20 con 0x55bed92b5610 2024-09-21T03:40:11.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.865+0000 7f0fb6419640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bed91f9240 0x55bed92ca640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:11.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.865+0000 7f0fb6419640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bed91f9240 0x55bed92ca640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:55814/0 (socket says 172.21.15.38:55814) 2024-09-21T03:40:11.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.865+0000 7f0fb6419640 1 -- 172.21.15.38:0/3063457308 learned_addr learned my addr 172.21.15.38:0/3063457308 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:11.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.865+0000 7f0fb6c1a640 1 --2- 172.21.15.38:0/3063457308 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bed9224d80 0x55bed92ce8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:11.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.865+0000 7f0fb6419640 1 -- 172.21.15.38:0/3063457308 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55bed92b5610 msgr2=0x55bed92cb970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:11.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.865+0000 7f0fb5c18640 1 --2- 172.21.15.38:0/3063457308 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55bed92b5610 0x55bed92cb970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:11.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.865+0000 7f0fb6419640 1 --2- 172.21.15.38:0/3063457308 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55bed92b5610 0x55bed92cb970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:11.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.865+0000 7f0fb6419640 1 -- 172.21.15.38:0/3063457308 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bed9224d80 msgr2=0x55bed92ce8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:11.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.865+0000 7f0fb6419640 1 --2- 172.21.15.38:0/3063457308 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bed9224d80 0x55bed92ce8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:11.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.865+0000 7f0fb6419640 1 -- 172.21.15.38:0/3063457308 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bed9221f20 con 0x55bed91f9240 2024-09-21T03:40:11.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.865+0000 7f0fb5c18640 1 --2- 172.21.15.38:0/3063457308 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55bed92b5610 0x55bed92cb970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:40:11.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.865+0000 7f0fb6c1a640 1 --2- 172.21.15.38:0/3063457308 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bed9224d80 0x55bed92ce8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:40:11.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.865+0000 7f0fb6419640 1 --2- 172.21.15.38:0/3063457308 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bed91f9240 0x55bed92ca640 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f0fa8025f40 tx=0x7f0fa8002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb0675085dc3fafd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:11.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.865+0000 7f0fb4c16640 1 -- 172.21.15.38:0/3063457308 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fa804d9a0 con 0x55bed91f9240 2024-09-21T03:40:11.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.865+0000 7f0fb4c16640 1 -- 172.21.15.38:0/3063457308 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0fa8056670 con 0x55bed91f9240 2024-09-21T03:40:11.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.865+0000 7f0fb4c16640 1 -- 172.21.15.38:0/3063457308 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fa8056ac0 con 0x55bed91f9240 2024-09-21T03:40:11.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.866+0000 7f0fbc95ff40 1 -- 172.21.15.38:0/3063457308 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bed91f9240 msgr2=0x55bed92ca640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:11.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.866+0000 7f0fbc95ff40 1 --2- 172.21.15.38:0/3063457308 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bed91f9240 0x55bed92ca640 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f0fa8025f40 tx=0x7f0fa8002fb0 comp rx=0 tx=0).stop 2024-09-21T03:40:11.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.866+0000 7f0fbc95ff40 1 -- 172.21.15.38:0/3063457308 shutdown_connections 2024-09-21T03:40:11.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.866+0000 7f0fbc95ff40 1 --2- 172.21.15.38:0/3063457308 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bed9224d80 0x55bed92ce8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:11.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.866+0000 7f0fbc95ff40 1 --2- 172.21.15.38:0/3063457308 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55bed92b5610 0x55bed92cb970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:11.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.866+0000 7f0fbc95ff40 1 --2- 172.21.15.38:0/3063457308 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bed91f9240 0x55bed92ca640 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:11.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.866+0000 7f0fbc95ff40 1 -- 172.21.15.38:0/3063457308 >> 172.21.15.38:0/3063457308 conn(0x55bed9117ac0 msgr2=0x55bed91f4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:11.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.866+0000 7f0fbc95ff40 1 -- 172.21.15.38:0/3063457308 shutdown_connections 2024-09-21T03:40:11.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.866+0000 7f0fbc95ff40 1 -- 172.21.15.38:0/3063457308 wait complete. 2024-09-21T03:40:11.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.867+0000 7f0fbc95ff40 1 Processor -- start 2024-09-21T03:40:11.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.867+0000 7f0fbc95ff40 1 -- start start 2024-09-21T03:40:11.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.868+0000 7f0fbc95ff40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bed91f9240 0x55bed92b2f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:11.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.868+0000 7f0fbc95ff40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55bed9224d80 0x55bed92c8a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:11.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.868+0000 7f0fbc95ff40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bed92b5610 0x55bed92af700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:11.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.868+0000 7f0fbc95ff40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55bed92c5c10 con 0x55bed91f9240 2024-09-21T03:40:11.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.868+0000 7f0fbc95ff40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55bed9221240 con 0x55bed92b5610 2024-09-21T03:40:11.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.868+0000 7f0fbc95ff40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55bed92b4c40 con 0x55bed9224d80 2024-09-21T03:40:11.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.868+0000 7f0fb6419640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bed91f9240 0x55bed92b2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:11.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.868+0000 7f0fb6419640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bed91f9240 0x55bed92b2f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:55826/0 (socket says 172.21.15.38:55826) 2024-09-21T03:40:11.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.868+0000 7f0fb6419640 1 -- 172.21.15.38:0/4145525204 learned_addr learned my addr 172.21.15.38:0/4145525204 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:11.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.869+0000 7f0fb5c18640 1 --2- 172.21.15.38:0/4145525204 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55bed9224d80 0x55bed92c8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:11.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.869+0000 7f0fb6c1a640 1 --2- 172.21.15.38:0/4145525204 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bed92b5610 0x55bed92af700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:11.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.869+0000 7f0fb6419640 1 -- 172.21.15.38:0/4145525204 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55bed9224d80 msgr2=0x55bed92c8a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:11.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.869+0000 7f0fb6419640 1 --2- 172.21.15.38:0/4145525204 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55bed9224d80 0x55bed92c8a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:11.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.869+0000 7f0fb6419640 1 -- 172.21.15.38:0/4145525204 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bed92b5610 msgr2=0x55bed92af700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:11.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.869+0000 7f0fb6419640 1 --2- 172.21.15.38:0/4145525204 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bed92b5610 0x55bed92af700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:11.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.869+0000 7f0fb6419640 1 -- 172.21.15.38:0/4145525204 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bed92b2790 con 0x55bed91f9240 2024-09-21T03:40:11.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.869+0000 7f0fb6419640 1 --2- 172.21.15.38:0/4145525204 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bed91f9240 0x55bed92b2f50 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f0fa8046640 tx=0x7f0fa8057bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:11.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.869+0000 7f0fa77fe640 1 -- 172.21.15.38:0/4145525204 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fa806f030 con 0x55bed91f9240 2024-09-21T03:40:11.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.869+0000 7f0fa77fe640 1 -- 172.21.15.38:0/4145525204 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0fa8057d20 con 0x55bed91f9240 2024-09-21T03:40:11.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.869+0000 7f0fbc95ff40 1 -- 172.21.15.38:0/4145525204 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55bed9240380 con 0x55bed91f9240 2024-09-21T03:40:11.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.869+0000 7f0fa77fe640 1 -- 172.21.15.38:0/4145525204 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fa8067040 con 0x55bed91f9240 2024-09-21T03:40:11.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.869+0000 7f0fbc95ff40 1 -- 172.21.15.38:0/4145525204 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bed92b3490 con 0x55bed91f9240 2024-09-21T03:40:11.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.871+0000 7f0fa77fe640 1 -- 172.21.15.38:0/4145525204 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f0fa8072070 con 0x55bed91f9240 2024-09-21T03:40:11.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.871+0000 7f0fa77fe640 1 --2- 172.21.15.38:0/4145525204 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0f98077570 0x7f0f98079a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:11.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.871+0000 7f0fa77fe640 1 -- 172.21.15.38:0/4145525204 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6187+0+0 (secure 0 0 0) 0x7f0fa80ded80 con 0x55bed91f9240 2024-09-21T03:40:11.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.872+0000 7f0fb5c18640 1 --2- 172.21.15.38:0/4145525204 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0f98077570 0x7f0f98079a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:11.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.872+0000 7f0fbc95ff40 1 --2- 172.21.15.38:0/4145525204 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x55bed92cd680 0x55bed92cda90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:11.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.872+0000 7f0fbc95ff40 1 -- 172.21.15.38:0/4145525204 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:1 13.0 13:00000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x55bed92c8fc0 con 0x55bed92cd680 2024-09-21T03:40:11.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.872+0000 7f0fb6c1a640 1 --2- 172.21.15.38:0/4145525204 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x55bed92cd680 0x55bed92cda90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:11.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.872+0000 7f0fb5c18640 1 --2- 172.21.15.38:0/4145525204 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0f98077570 0x7f0f98079a80 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x55bed92254c0 tx=0x7f0fac00d040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:11.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.872+0000 7f0fb6c1a640 1 --2- 172.21.15.38:0/4145525204 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x55bed92cd680 0x55bed92cda90 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:11.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.873+0000 7f0fb6c1a640 1 -- 172.21.15.38:0/4145525204 <== osd.1 v2:172.21.15.38:6816/1491818732 1 ==== osd_op_reply(1 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0fb0010550 con 0x55bed92cd680 2024-09-21T03:40:11.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.873+0000 7f0fb6c1a640 1 --2- 172.21.15.38:0/4145525204 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x7f0fb0002980 0x7f0fb00112c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:11.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.873+0000 7f0fb6c1a640 1 -- 172.21.15.38:0/4145525204 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:2 13.10 13:08000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x55bed92c8fc0 con 0x7f0fb0002980 2024-09-21T03:40:11.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.873+0000 7f0fb6419640 1 --2- 172.21.15.38:0/4145525204 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x7f0fb0002980 0x7f0fb00112c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:11.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.873+0000 7f0fb6419640 1 --2- 172.21.15.38:0/4145525204 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x7f0fb0002980 0x7f0fb00112c0 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:11.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.873+0000 7f0fb6419640 1 -- 172.21.15.38:0/4145525204 <== osd.7 v2:172.21.15.195:6818/1606106342 1 ==== osd_op_reply(2 [pgnls start_epoch 95 out=177b] v94'10 uv10 ondisk = 1) ==== 144+0+177 (crc 0 0 0) 0x7f0fa805f030 con 0x7f0fb0002980 2024-09-21T03:40:11.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.873+0000 7f0fb6419640 1 --2- 172.21.15.38:0/4145525204 >> [v2:172.21.15.195:6810/2080960180,v1:172.21.15.195:6811/2080960180] conn(0x7f0fa8022f00 0x7f0fa80aeac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:11.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.873+0000 7f0fb6419640 1 -- 172.21.15.38:0/4145525204 --> [v2:172.21.15.195:6810/2080960180,v1:172.21.15.195:6811/2080960180] -- osd_op(unknown.0.0:3 13.8 13:10000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x55bed92c8fc0 con 0x7f0fa8022f00 2024-09-21T03:40:11.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.874+0000 7f0fb5c18640 1 --2- 172.21.15.38:0/4145525204 >> [v2:172.21.15.195:6810/2080960180,v1:172.21.15.195:6811/2080960180] conn(0x7f0fa8022f00 0x7f0fa80aeac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:11.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.874+0000 7f0fb5c18640 1 --2- 172.21.15.38:0/4145525204 >> [v2:172.21.15.195:6810/2080960180,v1:172.21.15.195:6811/2080960180] conn(0x7f0fa8022f00 0x7f0fa80aeac0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:11.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.874+0000 7f0fb5c18640 1 -- 172.21.15.38:0/4145525204 <== osd.5 v2:172.21.15.195:6810/2080960180 1 ==== osd_op_reply(3 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0fac009620 con 0x7f0fa8022f00 2024-09-21T03:40:11.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.874+0000 7f0fb5c18640 1 -- 172.21.15.38:0/4145525204 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:4 13.18 13:18000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x55bed92c8fc0 con 0x55bed92cd680 2024-09-21T03:40:11.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.875+0000 7f0fb6c1a640 1 -- 172.21.15.38:0/4145525204 <== osd.1 v2:172.21.15.38:6816/1491818732 2 ==== osd_op_reply(4 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0fb0010550 con 0x55bed92cd680 2024-09-21T03:40:11.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.875+0000 7f0fb6c1a640 1 --2- 172.21.15.38:0/4145525204 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x7f0fb0014340 0x7f0fb0014750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:11.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.875+0000 7f0fb6c1a640 1 -- 172.21.15.38:0/4145525204 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:5 13.4 13:20000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x55bed92c8fc0 con 0x7f0fb0014340 2024-09-21T03:40:11.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.875+0000 7f0fb6c1a640 1 --2- 172.21.15.38:0/4145525204 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x7f0fb0014340 0x7f0fb0014750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:11.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.875+0000 7f0fb6c1a640 1 --2- 172.21.15.38:0/4145525204 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x7f0fb0014340 0x7f0fb0014750 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:11.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.875+0000 7f0fb6c1a640 1 -- 172.21.15.38:0/4145525204 <== osd.3 v2:172.21.15.38:6808/1286810294 1 ==== osd_op_reply(5 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0fb0010550 con 0x7f0fb0014340 2024-09-21T03:40:11.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.875+0000 7f0fb6c1a640 1 --2- 172.21.15.38:0/4145525204 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x7f0fb001a950 0x7f0fb0018430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:11.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.875+0000 7f0fb6c1a640 1 -- 172.21.15.38:0/4145525204 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:6 13.14 13:28000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x55bed92c8fc0 con 0x7f0fb001a950 2024-09-21T03:40:11.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.876+0000 7f0fb6419640 1 --2- 172.21.15.38:0/4145525204 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x7f0fb001a950 0x7f0fb0018430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:11.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.876+0000 7f0fb6419640 1 --2- 172.21.15.38:0/4145525204 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x7f0fb001a950 0x7f0fb0018430 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:11.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.876+0000 7f0fb6419640 1 -- 172.21.15.38:0/4145525204 <== osd.6 v2:172.21.15.195:6826/4055825675 1 ==== osd_op_reply(6 [pgnls start_epoch 95 out=149b] v94'18 uv18 ondisk = 1) ==== 144+0+149 (crc 0 0 0) 0x7f0fa805f030 con 0x7f0fb001a950 2024-09-21T03:40:11.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.876+0000 7f0fb6419640 1 -- 172.21.15.38:0/4145525204 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:7 13.c 13:30000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x55bed92c8fc0 con 0x7f0fb001a950 2024-09-21T03:40:11.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.876+0000 7f0fb6419640 1 -- 172.21.15.38:0/4145525204 <== osd.6 v2:172.21.15.195:6826/4055825675 2 ==== osd_op_reply(7 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0fa805f030 con 0x7f0fb001a950 2024-09-21T03:40:11.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.876+0000 7f0fb6419640 1 --2- 172.21.15.38:0/4145525204 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x7f0fa801f880 0x7f0fa80ad970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:11.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.876+0000 7f0fb6419640 1 -- 172.21.15.38:0/4145525204 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:8 13.1c 13:38000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x55bed92c8fc0 con 0x7f0fa801f880 2024-09-21T03:40:11.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.877+0000 7f0fb5c18640 1 --2- 172.21.15.38:0/4145525204 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x7f0fa801f880 0x7f0fa80ad970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:11.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.877+0000 7f0fb5c18640 1 --2- 172.21.15.38:0/4145525204 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x7f0fa801f880 0x7f0fa80ad970 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:11.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.877+0000 7f0fb5c18640 1 -- 172.21.15.38:0/4145525204 <== osd.0 v2:172.21.15.38:6800/2381639433 1 ==== osd_op_reply(8 [pgnls start_epoch 95 out=73b] v93'2 uv2 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x7f0fac009620 con 0x7f0fa801f880 2024-09-21T03:40:11.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.877+0000 7f0fb5c18640 1 -- 172.21.15.38:0/4145525204 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:9 13.2 13:40000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x55bed92c8fc0 con 0x55bed92cd680 2024-09-21T03:40:11.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.878+0000 7f0fb6c1a640 1 -- 172.21.15.38:0/4145525204 <== osd.1 v2:172.21.15.38:6816/1491818732 3 ==== osd_op_reply(9 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0fb0010550 con 0x55bed92cd680 2024-09-21T03:40:11.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.878+0000 7f0fb6c1a640 1 -- 172.21.15.38:0/4145525204 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:10 13.12 13:48000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x55bed92c8fc0 con 0x7f0fb001a950 2024-09-21T03:40:11.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.878+0000 7f0fb6419640 1 -- 172.21.15.38:0/4145525204 <== osd.6 v2:172.21.15.195:6826/4055825675 3 ==== osd_op_reply(10 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0fa805f030 con 0x7f0fb001a950 2024-09-21T03:40:11.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.878+0000 7f0fb6419640 1 --2- 172.21.15.38:0/4145525204 >> [v2:172.21.15.195:6802/3401837571,v1:172.21.15.195:6803/3401837571] conn(0x7f0fa800c440 0x7f0fa80ab110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:11.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.878+0000 7f0fb6419640 1 -- 172.21.15.38:0/4145525204 --> [v2:172.21.15.195:6802/3401837571,v1:172.21.15.195:6803/3401837571] -- osd_op(unknown.0.0:11 13.a 13:50000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x55bed92c8fc0 con 0x7f0fa800c440 2024-09-21T03:40:11.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.878+0000 7f0fb6c1a640 1 --2- 172.21.15.38:0/4145525204 >> [v2:172.21.15.195:6802/3401837571,v1:172.21.15.195:6803/3401837571] conn(0x7f0fa800c440 0x7f0fa80ab110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:11.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.879+0000 7f0fb6c1a640 1 --2- 172.21.15.38:0/4145525204 >> [v2:172.21.15.195:6802/3401837571,v1:172.21.15.195:6803/3401837571] conn(0x7f0fa800c440 0x7f0fa80ab110 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:11.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.879+0000 7f0fb6c1a640 1 -- 172.21.15.38:0/4145525204 <== osd.4 v2:172.21.15.195:6802/3401837571 1 ==== osd_op_reply(11 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0fb0010550 con 0x7f0fa800c440 2024-09-21T03:40:11.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.879+0000 7f0fb6c1a640 1 -- 172.21.15.38:0/4145525204 --> [v2:172.21.15.195:6810/2080960180,v1:172.21.15.195:6811/2080960180] -- osd_op(unknown.0.0:12 13.1a 13:58000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x55bed92c8fc0 con 0x7f0fa8022f00 2024-09-21T03:40:11.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.879+0000 7f0fb5c18640 1 -- 172.21.15.38:0/4145525204 <== osd.5 v2:172.21.15.195:6810/2080960180 2 ==== osd_op_reply(12 [pgnls start_epoch 95 out=49b] v93'1 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0fac009620 con 0x7f0fa8022f00 2024-09-21T03:40:11.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.879+0000 7f0fb5c18640 1 -- 172.21.15.38:0/4145525204 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:13 13.6 13:60000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x55bed92c8fc0 con 0x7f0fb0014340 2024-09-21T03:40:11.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.880+0000 7f0fb6c1a640 1 -- 172.21.15.38:0/4145525204 <== osd.3 v2:172.21.15.38:6808/1286810294 2 ==== osd_op_reply(13 [pgnls start_epoch 95 out=77b] v94'2 uv2 ondisk = 1) ==== 144+0+77 (crc 0 0 0) 0x7f0fb0010550 con 0x7f0fb0014340 2024-09-21T03:40:11.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.880+0000 7f0fb6c1a640 1 -- 172.21.15.38:0/4145525204 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:14 13.16 13:68000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x55bed92c8fc0 con 0x7f0fb0002980 2024-09-21T03:40:11.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.880+0000 7f0fb6419640 1 -- 172.21.15.38:0/4145525204 <== osd.7 v2:172.21.15.195:6818/1606106342 2 ==== osd_op_reply(14 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0fa805f030 con 0x7f0fb0002980 2024-09-21T03:40:11.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.880+0000 7f0fb6419640 1 -- 172.21.15.38:0/4145525204 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:15 13.e 13:70000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x55bed92c8fc0 con 0x7f0fa801f880 2024-09-21T03:40:11.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.880+0000 7f0fb5c18640 1 -- 172.21.15.38:0/4145525204 <== osd.0 v2:172.21.15.38:6800/2381639433 2 ==== osd_op_reply(15 [pgnls start_epoch 95 out=73b] v93'2 uv2 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x7f0fac009620 con 0x7f0fa801f880 2024-09-21T03:40:11.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.880+0000 7f0fb5c18640 1 -- 172.21.15.38:0/4145525204 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:16 13.1e 13:78000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x55bed92c8fc0 con 0x7f0fa801f880 2024-09-21T03:40:11.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.881+0000 7f0fb5c18640 1 -- 172.21.15.38:0/4145525204 <== osd.0 v2:172.21.15.38:6800/2381639433 3 ==== osd_op_reply(16 [pgnls start_epoch 95 out=49b] v93'1 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0fac009620 con 0x7f0fa801f880 2024-09-21T03:40:11.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.881+0000 7f0fb5c18640 1 -- 172.21.15.38:0/4145525204 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:17 13.1 13:80000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x55bed92c8fc0 con 0x55bed92cd680 2024-09-21T03:40:11.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.881+0000 7f0fb6c1a640 1 -- 172.21.15.38:0/4145525204 <== osd.1 v2:172.21.15.38:6816/1491818732 4 ==== osd_op_reply(17 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0fb0010550 con 0x55bed92cd680 2024-09-21T03:40:11.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.881+0000 7f0fb6c1a640 1 -- 172.21.15.38:0/4145525204 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:18 13.11 13:88000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x55bed92c8fc0 con 0x7f0fb001a950 2024-09-21T03:40:11.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.881+0000 7f0fb6419640 1 -- 172.21.15.38:0/4145525204 <== osd.6 v2:172.21.15.195:6826/4055825675 4 ==== osd_op_reply(18 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0fa805f030 con 0x7f0fb001a950 2024-09-21T03:40:11.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.881+0000 7f0fb6419640 1 -- 172.21.15.38:0/4145525204 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:19 13.9 13:90000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x55bed92c8fc0 con 0x7f0fa801f880 2024-09-21T03:40:11.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.881+0000 7f0fb5c18640 1 -- 172.21.15.38:0/4145525204 <== osd.0 v2:172.21.15.38:6800/2381639433 4 ==== osd_op_reply(19 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0fac009620 con 0x7f0fa801f880 2024-09-21T03:40:11.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.881+0000 7f0fb5c18640 1 -- 172.21.15.38:0/4145525204 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:20 13.19 13:98000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x55bed92c8fc0 con 0x7f0fb0002980 2024-09-21T03:40:11.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.882+0000 7f0fb6419640 1 -- 172.21.15.38:0/4145525204 <== osd.7 v2:172.21.15.195:6818/1606106342 3 ==== osd_op_reply(20 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0fa805f030 con 0x7f0fb0002980 2024-09-21T03:40:11.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.882+0000 7f0fb6419640 1 --2- 172.21.15.38:0/4145525204 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x7f0fa80b01f0 0x7f0fa80a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:11.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.882+0000 7f0fb6419640 1 -- 172.21.15.38:0/4145525204 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:21 13.5 13:a0000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x55bed92c8fc0 con 0x7f0fa80b01f0 2024-09-21T03:40:11.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.882+0000 7f0fb6419640 1 --2- 172.21.15.38:0/4145525204 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x7f0fa80b01f0 0x7f0fa80a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:11.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.882+0000 7f0fb6419640 1 --2- 172.21.15.38:0/4145525204 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x7f0fa80b01f0 0x7f0fa80a5170 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:11.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.883+0000 7f0fb6419640 1 -- 172.21.15.38:0/4145525204 <== osd.2 v2:172.21.15.38:6824/2577858902 1 ==== osd_op_reply(21 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0fa805f030 con 0x7f0fa80b01f0 2024-09-21T03:40:11.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.883+0000 7f0fb6419640 1 -- 172.21.15.38:0/4145525204 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:22 13.15 13:a8000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x55bed92c8fc0 con 0x7f0fb0014340 2024-09-21T03:40:11.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.883+0000 7f0fb6c1a640 1 -- 172.21.15.38:0/4145525204 <== osd.3 v2:172.21.15.38:6808/1286810294 3 ==== osd_op_reply(22 [pgnls start_epoch 95 out=97b] v93'3 uv3 ondisk = 1) ==== 144+0+97 (crc 0 0 0) 0x7f0fb0010550 con 0x7f0fb0014340 2024-09-21T03:40:11.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.883+0000 7f0fb6c1a640 1 -- 172.21.15.38:0/4145525204 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:23 13.d 13:b0000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x55bed92c8fc0 con 0x7f0fa801f880 2024-09-21T03:40:11.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.883+0000 7f0fb5c18640 1 -- 172.21.15.38:0/4145525204 <== osd.0 v2:172.21.15.38:6800/2381639433 5 ==== osd_op_reply(23 [pgnls start_epoch 95 out=74b] v93'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f0fac009620 con 0x7f0fa801f880 2024-09-21T03:40:11.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.883+0000 7f0fb5c18640 1 -- 172.21.15.38:0/4145525204 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:24 13.1d 13:b8000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x55bed92c8fc0 con 0x55bed92cd680 2024-09-21T03:40:11.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.883+0000 7f0fb6c1a640 1 -- 172.21.15.38:0/4145525204 <== osd.1 v2:172.21.15.38:6816/1491818732 5 ==== osd_op_reply(24 [pgnls start_epoch 95 out=73b] v93'2 uv2 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x7f0fb0010550 con 0x55bed92cd680 2024-09-21T03:40:11.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.884+0000 7f0fb6c1a640 1 -- 172.21.15.38:0/4145525204 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:25 13.3 13:c0000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x55bed92c8fc0 con 0x7f0fb001a950 2024-09-21T03:40:11.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.884+0000 7f0fb6419640 1 -- 172.21.15.38:0/4145525204 <== osd.6 v2:172.21.15.195:6826/4055825675 5 ==== osd_op_reply(25 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0fa805f030 con 0x7f0fb001a950 2024-09-21T03:40:11.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.884+0000 7f0fb6419640 1 -- 172.21.15.38:0/4145525204 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:26 13.13 13:c8000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x55bed92c8fc0 con 0x55bed92cd680 2024-09-21T03:40:11.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.884+0000 7f0fb6c1a640 1 -- 172.21.15.38:0/4145525204 <== osd.1 v2:172.21.15.38:6816/1491818732 6 ==== osd_op_reply(26 [pgnls start_epoch 95 out=98b] v94'3 uv3 ondisk = 1) ==== 144+0+98 (crc 0 0 0) 0x7f0fb0010550 con 0x55bed92cd680 2024-09-21T03:40:11.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.884+0000 7f0fb6c1a640 1 -- 172.21.15.38:0/4145525204 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:27 13.b 13:d0000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x55bed92c8fc0 con 0x7f0fb0014340 2024-09-21T03:40:11.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.884+0000 7f0fb6c1a640 1 -- 172.21.15.38:0/4145525204 <== osd.3 v2:172.21.15.38:6808/1286810294 4 ==== osd_op_reply(27 [pgnls start_epoch 95 out=76b] v94'3 uv3 ondisk = 1) ==== 144+0+76 (crc 0 0 0) 0x7f0fb0010550 con 0x7f0fb0014340 2024-09-21T03:40:11.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.884+0000 7f0fb6c1a640 1 -- 172.21.15.38:0/4145525204 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:28 13.1b 13:d8000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x55bed92c8fc0 con 0x7f0fa80b01f0 2024-09-21T03:40:11.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.885+0000 7f0fb6419640 1 -- 172.21.15.38:0/4145525204 <== osd.2 v2:172.21.15.38:6824/2577858902 2 ==== osd_op_reply(28 [pgnls start_epoch 95 out=49b] v93'1 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0fa805f030 con 0x7f0fa80b01f0 2024-09-21T03:40:11.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.885+0000 7f0fb6419640 1 -- 172.21.15.38:0/4145525204 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:29 13.7 13:e0000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x55bed92c8fc0 con 0x7f0fb0014340 2024-09-21T03:40:11.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.885+0000 7f0fb6c1a640 1 -- 172.21.15.38:0/4145525204 <== osd.3 v2:172.21.15.38:6808/1286810294 5 ==== osd_op_reply(29 [pgnls start_epoch 95 out=73b] v93'2 uv2 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x7f0fb0010550 con 0x7f0fb0014340 2024-09-21T03:40:11.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.885+0000 7f0fb6c1a640 1 -- 172.21.15.38:0/4145525204 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:30 13.17 13:e8000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x55bed92c8fc0 con 0x55bed92cd680 2024-09-21T03:40:11.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.885+0000 7f0fb6c1a640 1 -- 172.21.15.38:0/4145525204 <== osd.1 v2:172.21.15.38:6816/1491818732 7 ==== osd_op_reply(30 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0fb0010550 con 0x55bed92cd680 2024-09-21T03:40:11.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.885+0000 7f0fb6c1a640 1 -- 172.21.15.38:0/4145525204 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:31 13.f 13:f0000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x55bed92c8fc0 con 0x7f0fa80b01f0 2024-09-21T03:40:11.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.886+0000 7f0fb6419640 1 -- 172.21.15.38:0/4145525204 <== osd.2 v2:172.21.15.38:6824/2577858902 3 ==== osd_op_reply(31 [pgnls start_epoch 95 out=73b] v93'1 uv1 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x7f0fa805f030 con 0x7f0fa80b01f0 2024-09-21T03:40:11.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.886+0000 7f0fb6419640 1 -- 172.21.15.38:0/4145525204 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:32 13.1f 13:f8000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x55bed92c8fc0 con 0x55bed92cd680 2024-09-21T03:40:11.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.886+0000 7f0fb6c1a640 1 -- 172.21.15.38:0/4145525204 <== osd.1 v2:172.21.15.38:6816/1491818732 8 ==== osd_op_reply(32 [pgnls start_epoch 95 out=101b] v94'6 uv6 ondisk = 1) ==== 144+0+101 (crc 0 0 0) 0x7f0fb0010550 con 0x55bed92cd680 2024-09-21T03:40:11.892 INFO:teuthology.orchestra.run.smithi038.stdout:601.00000000 2024-09-21T03:40:11.892 INFO:teuthology.orchestra.run.smithi038.stdout:602.00000000 2024-09-21T03:40:11.893 INFO:teuthology.orchestra.run.smithi038.stdout:10000000000.00000000 2024-09-21T03:40:11.893 INFO:teuthology.orchestra.run.smithi038.stdout:600.00000000 2024-09-21T03:40:11.893 INFO:teuthology.orchestra.run.smithi038.stdout:603.00000000 2024-09-21T03:40:11.893 INFO:teuthology.orchestra.run.smithi038.stdout:1.00000000.inode 2024-09-21T03:40:11.893 INFO:teuthology.orchestra.run.smithi038.stdout:200.00000000 2024-09-21T03:40:11.893 INFO:teuthology.orchestra.run.smithi038.stdout:200.00000001 2024-09-21T03:40:11.893 INFO:teuthology.orchestra.run.smithi038.stdout:606.00000000 2024-09-21T03:40:11.893 INFO:teuthology.orchestra.run.smithi038.stdout:607.00000000 2024-09-21T03:40:11.893 INFO:teuthology.orchestra.run.smithi038.stdout:mds0_openfiles.0 2024-09-21T03:40:11.893 INFO:teuthology.orchestra.run.smithi038.stdout:608.00000000 2024-09-21T03:40:11.893 INFO:teuthology.orchestra.run.smithi038.stdout:604.00000000 2024-09-21T03:40:11.893 INFO:teuthology.orchestra.run.smithi038.stdout:500.00000000 2024-09-21T03:40:11.893 INFO:teuthology.orchestra.run.smithi038.stdout:mds_snaptable 2024-09-21T03:40:11.894 INFO:teuthology.orchestra.run.smithi038.stdout:605.00000000 2024-09-21T03:40:11.894 INFO:teuthology.orchestra.run.smithi038.stdout:mds0_inotable 2024-09-21T03:40:11.894 INFO:teuthology.orchestra.run.smithi038.stdout:100.00000000 2024-09-21T03:40:11.894 INFO:teuthology.orchestra.run.smithi038.stdout:mds0_sessionmap 2024-09-21T03:40:11.894 INFO:teuthology.orchestra.run.smithi038.stdout:609.00000000 2024-09-21T03:40:11.894 INFO:teuthology.orchestra.run.smithi038.stdout:400.00000000 2024-09-21T03:40:11.894 INFO:teuthology.orchestra.run.smithi038.stdout:100.00000000.inode 2024-09-21T03:40:11.894 INFO:teuthology.orchestra.run.smithi038.stdout:1.00000000 2024-09-21T03:40:11.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.886+0000 7f0fbc95ff40 1 -- 172.21.15.38:0/4145525204 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x7f0fa801f880 msgr2=0x7f0fa80ad970 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:11.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.886+0000 7f0fbc95ff40 1 --2- 172.21.15.38:0/4145525204 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x7f0fa801f880 0x7f0fa80ad970 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:11.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.886+0000 7f0fbc95ff40 1 -- 172.21.15.38:0/4145525204 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x55bed92cd680 msgr2=0x55bed92cda90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:11.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.886+0000 7f0fbc95ff40 1 --2- 172.21.15.38:0/4145525204 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x55bed92cd680 0x55bed92cda90 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:11.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.886+0000 7f0fbc95ff40 1 -- 172.21.15.38:0/4145525204 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x7f0fa80b01f0 msgr2=0x7f0fa80a5170 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:11.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.886+0000 7f0fbc95ff40 1 --2- 172.21.15.38:0/4145525204 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x7f0fa80b01f0 0x7f0fa80a5170 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:11.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.886+0000 7f0fbc95ff40 1 -- 172.21.15.38:0/4145525204 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x7f0fb0014340 msgr2=0x7f0fb0014750 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:11.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.886+0000 7f0fbc95ff40 1 --2- 172.21.15.38:0/4145525204 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x7f0fb0014340 0x7f0fb0014750 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:11.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.886+0000 7f0fbc95ff40 1 -- 172.21.15.38:0/4145525204 >> [v2:172.21.15.195:6802/3401837571,v1:172.21.15.195:6803/3401837571] conn(0x7f0fa800c440 msgr2=0x7f0fa80ab110 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:11.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.886+0000 7f0fbc95ff40 1 --2- 172.21.15.38:0/4145525204 >> [v2:172.21.15.195:6802/3401837571,v1:172.21.15.195:6803/3401837571] conn(0x7f0fa800c440 0x7f0fa80ab110 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:11.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.886+0000 7f0fbc95ff40 1 -- 172.21.15.38:0/4145525204 >> [v2:172.21.15.195:6810/2080960180,v1:172.21.15.195:6811/2080960180] conn(0x7f0fa8022f00 msgr2=0x7f0fa80aeac0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:11.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.886+0000 7f0fbc95ff40 1 --2- 172.21.15.38:0/4145525204 >> [v2:172.21.15.195:6810/2080960180,v1:172.21.15.195:6811/2080960180] conn(0x7f0fa8022f00 0x7f0fa80aeac0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:11.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.886+0000 7f0fbc95ff40 1 -- 172.21.15.38:0/4145525204 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x7f0fb001a950 msgr2=0x7f0fb0018430 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:11.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.886+0000 7f0fbc95ff40 1 --2- 172.21.15.38:0/4145525204 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x7f0fb001a950 0x7f0fb0018430 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:11.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.886+0000 7f0fbc95ff40 1 -- 172.21.15.38:0/4145525204 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x7f0fb0002980 msgr2=0x7f0fb00112c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:11.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.886+0000 7f0fb6c1a640 1 -- 172.21.15.38:0/4145525204 reap_dead start 2024-09-21T03:40:11.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.886+0000 7f0fbc95ff40 1 --2- 172.21.15.38:0/4145525204 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x7f0fb0002980 0x7f0fb00112c0 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:11.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.886+0000 7f0fb6c1a640 1 -- 172.21.15.38:0/4145525204 reap_dead start 2024-09-21T03:40:11.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.886+0000 7f0fb6c1a640 1 -- 172.21.15.38:0/4145525204 reap_dead start 2024-09-21T03:40:11.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.886+0000 7f0fb6c1a640 1 -- 172.21.15.38:0/4145525204 reap_dead start 2024-09-21T03:40:11.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.886+0000 7f0fbc95ff40 1 -- 172.21.15.38:0/4145525204 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0f98077570 msgr2=0x7f0f98079a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:11.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.886+0000 7f0fbc95ff40 1 --2- 172.21.15.38:0/4145525204 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0f98077570 0x7f0f98079a80 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x55bed92254c0 tx=0x7f0fac00d040 comp rx=0 tx=0).stop 2024-09-21T03:40:11.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.886+0000 7f0fbc95ff40 1 -- 172.21.15.38:0/4145525204 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bed91f9240 msgr2=0x55bed92b2f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:11.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.886+0000 7f0fbc95ff40 1 --2- 172.21.15.38:0/4145525204 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bed91f9240 0x55bed92b2f50 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f0fa8046640 tx=0x7f0fa8057bc0 comp rx=0 tx=0).stop 2024-09-21T03:40:11.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.886+0000 7f0fbc95ff40 1 -- 172.21.15.38:0/4145525204 shutdown_connections 2024-09-21T03:40:11.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.886+0000 7f0fbc95ff40 1 --2- 172.21.15.38:0/4145525204 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0f98077570 0x7f0f98079a80 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:11.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.886+0000 7f0fbc95ff40 1 --2- 172.21.15.38:0/4145525204 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bed91f9240 0x55bed92b2f50 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:11.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.886+0000 7f0fbc95ff40 1 -- 172.21.15.38:0/4145525204 >> 172.21.15.38:0/4145525204 conn(0x55bed9117ac0 msgr2=0x55bed92cafa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:11.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.886+0000 7f0fbc95ff40 1 -- 172.21.15.38:0/4145525204 shutdown_connections 2024-09-21T03:40:11.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.886+0000 7f0fbc95ff40 1 -- 172.21.15.38:0/4145525204 wait complete. 2024-09-21T03:40:11.899 DEBUG:teuthology.orchestra.run.smithi038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getomapheader 10000000000.00000000 2024-09-21T03:40:11.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.953+0000 7fe6eb54cf40 1 Processor -- start 2024-09-21T03:40:11.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.953+0000 7fe6eb54cf40 1 -- start start 2024-09-21T03:40:11.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.954+0000 7fe6eb54cf40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560046230d00 0x5600462e7780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:11.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.954+0000 7fe6eb54cf40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56004625ce30 0x5600462e93e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:11.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.954+0000 7fe6eb54cf40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5600462eb950 0x5600462e9fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:11.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.954+0000 7fe6eb54cf40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5600461dd430 con 0x560046230d00 2024-09-21T03:40:11.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.954+0000 7fe6eb54cf40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5600461ea1f0 con 0x56004625ce30 2024-09-21T03:40:11.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.954+0000 7fe6eb54cf40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x56004617ed60 con 0x5600462eb950 2024-09-21T03:40:11.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.955+0000 7fe6e5006640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560046230d00 0x5600462e7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:11.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.955+0000 7fe6e5006640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560046230d00 0x5600462e7780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:55842/0 (socket says 172.21.15.38:55842) 2024-09-21T03:40:11.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.955+0000 7fe6e5006640 1 -- 172.21.15.38:0/3488639088 learned_addr learned my addr 172.21.15.38:0/3488639088 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:11.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.955+0000 7fe6e5807640 1 --2- 172.21.15.38:0/3488639088 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5600462eb950 0x5600462e9fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:11.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.955+0000 7fe6e4805640 1 --2- 172.21.15.38:0/3488639088 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56004625ce30 0x5600462e93e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:11.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.955+0000 7fe6e5006640 1 -- 172.21.15.38:0/3488639088 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5600462eb950 msgr2=0x5600462e9fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:11.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.955+0000 7fe6e5006640 1 --2- 172.21.15.38:0/3488639088 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5600462eb950 0x5600462e9fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:11.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.955+0000 7fe6e5006640 1 -- 172.21.15.38:0/3488639088 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56004625ce30 msgr2=0x5600462e93e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:11.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.955+0000 7fe6e5006640 1 --2- 172.21.15.38:0/3488639088 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56004625ce30 0x5600462e93e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:11.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.955+0000 7fe6e5006640 1 -- 172.21.15.38:0/3488639088 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56004625f0c0 con 0x560046230d00 2024-09-21T03:40:11.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.955+0000 7fe6e5006640 1 --2- 172.21.15.38:0/3488639088 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560046230d00 0x5600462e7780 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7fe6d8025ee0 tx=0x7fe6d8002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3299d84b5a5ae7a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:11.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.956+0000 7fe6d77fe640 1 -- 172.21.15.38:0/3488639088 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6d8064020 con 0x560046230d00 2024-09-21T03:40:11.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.956+0000 7fe6d77fe640 1 -- 172.21.15.38:0/3488639088 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe6d804d900 con 0x560046230d00 2024-09-21T03:40:11.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.956+0000 7fe6d77fe640 1 -- 172.21.15.38:0/3488639088 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6d80606d0 con 0x560046230d00 2024-09-21T03:40:11.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.956+0000 7fe6eb54cf40 1 -- 172.21.15.38:0/3488639088 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560046230d00 msgr2=0x5600462e7780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:11.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.956+0000 7fe6eb54cf40 1 --2- 172.21.15.38:0/3488639088 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560046230d00 0x5600462e7780 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7fe6d8025ee0 tx=0x7fe6d8002fb0 comp rx=0 tx=0).stop 2024-09-21T03:40:11.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.956+0000 7fe6eb54cf40 1 -- 172.21.15.38:0/3488639088 shutdown_connections 2024-09-21T03:40:11.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.956+0000 7fe6eb54cf40 1 --2- 172.21.15.38:0/3488639088 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5600462eb950 0x5600462e9fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:11.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.956+0000 7fe6eb54cf40 1 --2- 172.21.15.38:0/3488639088 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56004625ce30 0x5600462e93e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:11.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.956+0000 7fe6eb54cf40 1 --2- 172.21.15.38:0/3488639088 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560046230d00 0x5600462e7780 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:11.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.956+0000 7fe6eb54cf40 1 -- 172.21.15.38:0/3488639088 >> 172.21.15.38:0/3488639088 conn(0x56004614fac0 msgr2=0x560046306350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:11.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.956+0000 7fe6eb54cf40 1 -- 172.21.15.38:0/3488639088 shutdown_connections 2024-09-21T03:40:11.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.956+0000 7fe6eb54cf40 1 -- 172.21.15.38:0/3488639088 wait complete. 2024-09-21T03:40:11.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.957+0000 7fe6eb54cf40 1 Processor -- start 2024-09-21T03:40:11.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.957+0000 7fe6eb54cf40 1 -- start start 2024-09-21T03:40:11.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.960+0000 7fe6eb54cf40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560046230d00 0x5600462e1e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:11.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.960+0000 7fe6eb54cf40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56004625ce30 0x5600463016b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:11.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.960+0000 7fe6eb54cf40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5600462eb950 0x5600462e3840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:11.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.960+0000 7fe6eb54cf40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5600462e9920 con 0x56004625ce30 2024-09-21T03:40:11.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.960+0000 7fe6eb54cf40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5600462eaa50 con 0x5600462eb950 2024-09-21T03:40:11.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.960+0000 7fe6eb54cf40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x5600462dfef0 con 0x560046230d00 2024-09-21T03:40:11.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.960+0000 7fe6e4805640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56004625ce30 0x5600463016b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:11.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.960+0000 7fe6e4805640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56004625ce30 0x5600463016b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:55856/0 (socket says 172.21.15.38:55856) 2024-09-21T03:40:11.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.960+0000 7fe6e4805640 1 -- 172.21.15.38:0/3910974944 learned_addr learned my addr 172.21.15.38:0/3910974944 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:11.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.960+0000 7fe6e5006640 1 --2- 172.21.15.38:0/3910974944 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560046230d00 0x5600462e1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:11.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.960+0000 7fe6e5807640 1 --2- 172.21.15.38:0/3910974944 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5600462eb950 0x5600462e3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:11.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.960+0000 7fe6e4805640 1 -- 172.21.15.38:0/3910974944 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560046230d00 msgr2=0x5600462e1e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:11.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.960+0000 7fe6e4805640 1 --2- 172.21.15.38:0/3910974944 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560046230d00 0x5600462e1e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:11.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.960+0000 7fe6e4805640 1 -- 172.21.15.38:0/3910974944 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5600462eb950 msgr2=0x5600462e3840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:11.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.960+0000 7fe6e4805640 1 --2- 172.21.15.38:0/3910974944 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5600462eb950 0x5600462e3840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:11.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.960+0000 7fe6e4805640 1 -- 172.21.15.38:0/3910974944 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600462d2240 con 0x56004625ce30 2024-09-21T03:40:11.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.960+0000 7fe6e4805640 1 --2- 172.21.15.38:0/3910974944 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56004625ce30 0x5600463016b0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7fe6dc00efb0 tx=0x7fe6dc00c620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:11.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.960+0000 7fe6d67fc640 1 -- 172.21.15.38:0/3910974944 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6dc002e50 con 0x56004625ce30 2024-09-21T03:40:11.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.960+0000 7fe6d67fc640 1 -- 172.21.15.38:0/3910974944 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe6dc00ccb0 con 0x56004625ce30 2024-09-21T03:40:11.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.960+0000 7fe6eb54cf40 1 -- 172.21.15.38:0/3910974944 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56004625b0a0 con 0x56004625ce30 2024-09-21T03:40:11.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.960+0000 7fe6d67fc640 1 -- 172.21.15.38:0/3910974944 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6dc001c30 con 0x56004625ce30 2024-09-21T03:40:11.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.960+0000 7fe6eb54cf40 1 -- 172.21.15.38:0/3910974944 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560046302710 con 0x56004625ce30 2024-09-21T03:40:11.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.961+0000 7fe6d67fc640 1 -- 172.21.15.38:0/3910974944 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fe6dc002030 con 0x56004625ce30 2024-09-21T03:40:11.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.962+0000 7fe6d67fc640 1 --2- 172.21.15.38:0/3910974944 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fe6c8077710 0x7fe6c8079c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:11.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.962+0000 7fe6d67fc640 1 -- 172.21.15.38:0/3910974944 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6187+0+0 (secure 0 0 0) 0x7fe6dc0997c0 con 0x56004625ce30 2024-09-21T03:40:11.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.962+0000 7fe6e5006640 1 --2- 172.21.15.38:0/3910974944 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fe6c8077710 0x7fe6c8079c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:11.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.962+0000 7fe6eb54cf40 1 --2- 172.21.15.38:0/3910974944 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x5600462ffe30 0x56004639cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:11.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.962+0000 7fe6eb54cf40 1 -- 172.21.15.38:0/3910974944 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:1 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x560046301bf0 con 0x5600462ffe30 2024-09-21T03:40:11.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.962+0000 7fe6e5807640 1 --2- 172.21.15.38:0/3910974944 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x5600462ffe30 0x56004639cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:11.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.962+0000 7fe6e5006640 1 --2- 172.21.15.38:0/3910974944 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fe6c8077710 0x7fe6c8079c20 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fe6d8065000 tx=0x7fe6d8004880 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:11.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.963+0000 7fe6e5807640 1 --2- 172.21.15.38:0/3910974944 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x5600462ffe30 0x56004639cde0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:11.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.963+0000 7fe6e5807640 1 -- 172.21.15.38:0/3910974944 <== osd.7 v2:172.21.15.195:6818/1606106342 1 ==== osd_op_reply(1 10000000000.00000000 [omap-get-header out=274b] v0'0 uv10 ondisk = 0) ==== 164+0+274 (crc 0 0 0) 0x7fe6e0002040 con 0x5600462ffe30 2024-09-21T03:40:11.967 INFO:teuthology.orchestra.run.smithi038.stdout:header (274 bytes) : 2024-09-21T03:40:11.967 INFO:teuthology.orchestra.run.smithi038.stdout:00000000 04 03 0c 01 00 00 0b 00 00 00 00 00 00 00 00 00 |................| 2024-09-21T03:40:11.967 INFO:teuthology.orchestra.run.smithi038.stdout:00000010 00 00 00 00 00 00 03 02 28 00 00 00 00 00 00 00 |........(.......| 2024-09-21T03:40:11.967 INFO:teuthology.orchestra.run.smithi038.stdout:00000020 00 00 00 00 1a 40 ee 66 25 01 ad 02 01 00 00 00 |.....@.f%.......| 2024-09-21T03:40:11.967 INFO:teuthology.orchestra.run.smithi038.stdout:00000030 00 00 00 00 00 00 00 00 00 00 00 00 01 00 00 00 |................| 2024-09-21T03:40:11.967 INFO:teuthology.orchestra.run.smithi038.stdout:00000040 00 00 00 00 03 02 28 00 00 00 00 00 00 00 00 00 |......(.........| 2024-09-21T03:40:11.967 INFO:teuthology.orchestra.run.smithi038.stdout:00000050 00 00 1a 40 ee 66 25 01 ad 02 01 00 00 00 00 00 |...@.f%.........| 2024-09-21T03:40:11.968 INFO:teuthology.orchestra.run.smithi038.stdout:00000060 00 00 00 00 00 00 00 00 00 00 01 00 00 00 00 00 |................| 2024-09-21T03:40:11.968 INFO:teuthology.orchestra.run.smithi038.stdout:00000070 00 00 03 02 38 00 00 00 00 00 00 00 00 00 00 00 |....8...........| 2024-09-21T03:40:11.968 INFO:teuthology.orchestra.run.smithi038.stdout:00000080 00 00 60 00 00 00 00 00 01 00 00 00 00 00 00 00 |..`.............| 2024-09-21T03:40:11.968 INFO:teuthology.orchestra.run.smithi038.stdout:00000090 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 2024-09-21T03:40:11.968 INFO:teuthology.orchestra.run.smithi038.stdout:000000a0 00 00 00 00 00 00 00 00 1a 40 ee 66 50 a8 bf 03 |.........@.fP...| 2024-09-21T03:40:11.968 INFO:teuthology.orchestra.run.smithi038.stdout:000000b0 03 02 38 00 00 00 00 00 00 00 00 00 00 00 00 00 |..8.............| 2024-09-21T03:40:11.968 INFO:teuthology.orchestra.run.smithi038.stdout:000000c0 60 00 00 00 00 00 01 00 00 00 00 00 00 00 00 00 |`...............| 2024-09-21T03:40:11.968 INFO:teuthology.orchestra.run.smithi038.stdout:000000d0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 2024-09-21T03:40:11.968 INFO:teuthology.orchestra.run.smithi038.stdout:000000e0 00 00 00 00 00 00 1a 40 ee 66 50 a8 bf 03 00 00 |.......@.fP.....| 2024-09-21T03:40:11.968 INFO:teuthology.orchestra.run.smithi038.stdout:000000f0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 2024-09-21T03:40:11.968 INFO:teuthology.orchestra.run.smithi038.stdout:* 2024-09-21T03:40:11.968 INFO:teuthology.orchestra.run.smithi038.stdout:00000110 00 00 |..| 2024-09-21T03:40:11.968 INFO:teuthology.orchestra.run.smithi038.stdout:00000112 2024-09-21T03:40:11.969 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:40:11.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.963+0000 7fe6eb54cf40 1 -- 172.21.15.38:0/3910974944 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x5600462ffe30 msgr2=0x56004639cde0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:11.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.963+0000 7fe6eb54cf40 1 --2- 172.21.15.38:0/3910974944 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x5600462ffe30 0x56004639cde0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:11.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.963+0000 7fe6eb54cf40 1 -- 172.21.15.38:0/3910974944 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fe6c8077710 msgr2=0x7fe6c8079c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:11.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.963+0000 7fe6eb54cf40 1 --2- 172.21.15.38:0/3910974944 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fe6c8077710 0x7fe6c8079c20 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fe6d8065000 tx=0x7fe6d8004880 comp rx=0 tx=0).stop 2024-09-21T03:40:11.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.963+0000 7fe6eb54cf40 1 -- 172.21.15.38:0/3910974944 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56004625ce30 msgr2=0x5600463016b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:11.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.963+0000 7fe6eb54cf40 1 --2- 172.21.15.38:0/3910974944 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56004625ce30 0x5600463016b0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7fe6dc00efb0 tx=0x7fe6dc00c620 comp rx=0 tx=0).stop 2024-09-21T03:40:11.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.963+0000 7fe6e5807640 1 -- 172.21.15.38:0/3910974944 reap_dead start 2024-09-21T03:40:11.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.963+0000 7fe6eb54cf40 1 -- 172.21.15.38:0/3910974944 shutdown_connections 2024-09-21T03:40:11.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.963+0000 7fe6eb54cf40 1 -- 172.21.15.38:0/3910974944 >> 172.21.15.38:0/3910974944 conn(0x56004614fac0 msgr2=0x560046306350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:11.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.963+0000 7fe6eb54cf40 1 -- 172.21.15.38:0/3910974944 shutdown_connections 2024-09-21T03:40:11.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:11.963+0000 7fe6eb54cf40 1 -- 172.21.15.38:0/3910974944 wait complete. 2024-09-21T03:40:11.971 DEBUG:teuthology.orchestra.run.smithi038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getomapheader 600.00000000 2024-09-21T03:40:12.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.023+0000 7fd3c63b4f40 1 Processor -- start 2024-09-21T03:40:12.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.023+0000 7fd3c63b4f40 1 -- start start 2024-09-21T03:40:12.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.026+0000 7fd3c63b4f40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560c6420fd00 0x560c64254430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:12.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.026+0000 7fd3c63b4f40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560c642ca950 0x560c642e20b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:12.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.026+0000 7fd3c63b4f40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560c642e5670 0x560c642e25f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:12.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.026+0000 7fd3c63b4f40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x560c642581d0 con 0x560c642ca950 2024-09-21T03:40:12.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.026+0000 7fd3c63b4f40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x560c6415dd60 con 0x560c642e5670 2024-09-21T03:40:12.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.026+0000 7fd3c63b4f40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x560c642577b0 con 0x560c6420fd00 2024-09-21T03:40:12.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.026+0000 7fd3bf66d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560c642ca950 0x560c642e20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:12.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.026+0000 7fd3bf66d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560c642ca950 0x560c642e20b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:55878/0 (socket says 172.21.15.38:55878) 2024-09-21T03:40:12.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.026+0000 7fd3bf66d640 1 -- 172.21.15.38:0/2520232533 learned_addr learned my addr 172.21.15.38:0/2520232533 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:12.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.026+0000 7fd3bfe6e640 1 --2- 172.21.15.38:0/2520232533 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560c6420fd00 0x560c64254430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:12.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.026+0000 7fd3c066f640 1 --2- 172.21.15.38:0/2520232533 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560c642e5670 0x560c642e25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:12.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.026+0000 7fd3bf66d640 1 -- 172.21.15.38:0/2520232533 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560c6420fd00 msgr2=0x560c64254430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:12.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.026+0000 7fd3bf66d640 1 --2- 172.21.15.38:0/2520232533 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560c6420fd00 0x560c64254430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:12.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.026+0000 7fd3bf66d640 1 -- 172.21.15.38:0/2520232533 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560c642e5670 msgr2=0x560c642e25f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:12.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.026+0000 7fd3bf66d640 1 --2- 172.21.15.38:0/2520232533 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560c642e5670 0x560c642e25f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:12.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.026+0000 7fd3bf66d640 1 -- 172.21.15.38:0/2520232533 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560c6420c450 con 0x560c642ca950 2024-09-21T03:40:12.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.026+0000 7fd3bf66d640 1 --2- 172.21.15.38:0/2520232533 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560c642ca950 0x560c642e20b0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7fd3b4025e50 tx=0x7fd3b4002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47050c2a9cb0933b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:12.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.026+0000 7fd3be66b640 1 -- 172.21.15.38:0/2520232533 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3b404dd70 con 0x560c642ca950 2024-09-21T03:40:12.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.026+0000 7fd3be66b640 1 -- 172.21.15.38:0/2520232533 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd3b404d890 con 0x560c642ca950 2024-09-21T03:40:12.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.026+0000 7fd3be66b640 1 -- 172.21.15.38:0/2520232533 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3b40566b0 con 0x560c642ca950 2024-09-21T03:40:12.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.026+0000 7fd3c63b4f40 1 -- 172.21.15.38:0/2520232533 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560c642ca950 msgr2=0x560c642e20b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:12.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.026+0000 7fd3c63b4f40 1 --2- 172.21.15.38:0/2520232533 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560c642ca950 0x560c642e20b0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7fd3b4025e50 tx=0x7fd3b4002fb0 comp rx=0 tx=0).stop 2024-09-21T03:40:12.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.026+0000 7fd3c63b4f40 1 -- 172.21.15.38:0/2520232533 shutdown_connections 2024-09-21T03:40:12.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.026+0000 7fd3c63b4f40 1 --2- 172.21.15.38:0/2520232533 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560c642e5670 0x560c642e25f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:12.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.026+0000 7fd3c63b4f40 1 --2- 172.21.15.38:0/2520232533 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560c642ca950 0x560c642e20b0 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:12.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.026+0000 7fd3c63b4f40 1 --2- 172.21.15.38:0/2520232533 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560c6420fd00 0x560c64254430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:12.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.026+0000 7fd3c63b4f40 1 -- 172.21.15.38:0/2520232533 >> 172.21.15.38:0/2520232533 conn(0x560c6412eac0 msgr2=0x560c642e5a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:12.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.026+0000 7fd3c63b4f40 1 -- 172.21.15.38:0/2520232533 shutdown_connections 2024-09-21T03:40:12.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.026+0000 7fd3c63b4f40 1 -- 172.21.15.38:0/2520232533 wait complete. 2024-09-21T03:40:12.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.027+0000 7fd3c63b4f40 1 Processor -- start 2024-09-21T03:40:12.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.027+0000 7fd3c63b4f40 1 -- start start 2024-09-21T03:40:12.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.028+0000 7fd3c63b4f40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560c6420fd00 0x560c64253410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:12.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.028+0000 7fd3c63b4f40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560c642ca950 0x560c6437d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:12.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.028+0000 7fd3c63b4f40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560c642e5670 0x560c6436e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:12.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.028+0000 7fd3c63b4f40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x560c642e3950 con 0x560c6420fd00 2024-09-21T03:40:12.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.028+0000 7fd3c63b4f40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x560c642e3310 con 0x560c642e5670 2024-09-21T03:40:12.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.028+0000 7fd3c63b4f40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x560c642527f0 con 0x560c642ca950 2024-09-21T03:40:12.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.028+0000 7fd3bfe6e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560c6420fd00 0x560c64253410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:12.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.029+0000 7fd3bfe6e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560c6420fd00 0x560c64253410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:55884/0 (socket says 172.21.15.38:55884) 2024-09-21T03:40:12.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.029+0000 7fd3bfe6e640 1 -- 172.21.15.38:0/1749409884 learned_addr learned my addr 172.21.15.38:0/1749409884 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:12.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.029+0000 7fd3c066f640 1 --2- 172.21.15.38:0/1749409884 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560c642e5670 0x560c6436e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:12.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.029+0000 7fd3bf66d640 1 --2- 172.21.15.38:0/1749409884 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560c642ca950 0x560c6437d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:12.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.029+0000 7fd3bfe6e640 1 -- 172.21.15.38:0/1749409884 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560c642ca950 msgr2=0x560c6437d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:12.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.029+0000 7fd3bfe6e640 1 --2- 172.21.15.38:0/1749409884 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560c642ca950 0x560c6437d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:12.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.029+0000 7fd3bfe6e640 1 -- 172.21.15.38:0/1749409884 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560c642e5670 msgr2=0x560c6436e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:12.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.029+0000 7fd3bfe6e640 1 --2- 172.21.15.38:0/1749409884 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560c642e5670 0x560c6436e070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:12.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.029+0000 7fd3bfe6e640 1 -- 172.21.15.38:0/1749409884 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560c6437d7a0 con 0x560c6420fd00 2024-09-21T03:40:12.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.029+0000 7fd3bfe6e640 1 --2- 172.21.15.38:0/1749409884 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560c6420fd00 0x560c64253410 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7fd3b000a6e0 tx=0x7fd3b000aba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:12.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.029+0000 7fd3bd669640 1 -- 172.21.15.38:0/1749409884 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3b0009b40 con 0x560c6420fd00 2024-09-21T03:40:12.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.029+0000 7fd3bd669640 1 -- 172.21.15.38:0/1749409884 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd3b0009ce0 con 0x560c6420fd00 2024-09-21T03:40:12.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.029+0000 7fd3bd669640 1 -- 172.21.15.38:0/1749409884 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3b0017690 con 0x560c6420fd00 2024-09-21T03:40:12.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.029+0000 7fd3c63b4f40 1 -- 172.21.15.38:0/1749409884 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x560c6436f160 con 0x560c6420fd00 2024-09-21T03:40:12.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.029+0000 7fd3c63b4f40 1 -- 172.21.15.38:0/1749409884 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560c6436d870 con 0x560c6420fd00 2024-09-21T03:40:12.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.031+0000 7fd3bd669640 1 -- 172.21.15.38:0/1749409884 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fd3b0017830 con 0x560c6420fd00 2024-09-21T03:40:12.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.033+0000 7fd3bd669640 1 --2- 172.21.15.38:0/1749409884 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd3ac0775a0 0x7fd3ac079ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:12.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.033+0000 7fd3bd669640 1 -- 172.21.15.38:0/1749409884 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6187+0+0 (secure 0 0 0) 0x7fd3b009a5b0 con 0x560c6420fd00 2024-09-21T03:40:12.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.033+0000 7fd3bf66d640 1 --2- 172.21.15.38:0/1749409884 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd3ac0775a0 0x7fd3ac079ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:12.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.033+0000 7fd3c63b4f40 1 --2- 172.21.15.38:0/1749409884 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x560c6436b4b0 0x560c6436b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:12.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.033+0000 7fd3c63b4f40 1 -- 172.21.15.38:0/1749409884 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:1 13.10 13:0d82a743:::600.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x560c6436c5c0 con 0x560c6436b4b0 2024-09-21T03:40:12.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.033+0000 7fd3c066f640 1 --2- 172.21.15.38:0/1749409884 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x560c6436b4b0 0x560c6436b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:12.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.033+0000 7fd3bf66d640 1 --2- 172.21.15.38:0/1749409884 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd3ac0775a0 0x7fd3ac079ab0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fd3b4025e50 tx=0x7fd3b4004940 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:12.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.033+0000 7fd3c066f640 1 --2- 172.21.15.38:0/1749409884 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x560c6436b4b0 0x560c6436b8c0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:12.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.033+0000 7fd3c066f640 1 -- 172.21.15.38:0/1749409884 <== osd.7 v2:172.21.15.195:6818/1606106342 1 ==== osd_op_reply(1 600.00000000 [omap-get-header out=274b] v0'0 uv2 ondisk = 0) ==== 156+0+274 (crc 0 0 0) 0x7fd3b8003040 con 0x560c6436b4b0 2024-09-21T03:40:12.037 INFO:teuthology.orchestra.run.smithi038.stdout:header (274 bytes) : 2024-09-21T03:40:12.037 INFO:teuthology.orchestra.run.smithi038.stdout:00000000 04 03 0c 01 00 00 01 00 00 00 00 00 00 00 00 00 |................| 2024-09-21T03:40:12.037 INFO:teuthology.orchestra.run.smithi038.stdout:00000010 00 00 00 00 00 00 03 02 28 00 00 00 00 00 00 00 |........(.......| 2024-09-21T03:40:12.037 INFO:teuthology.orchestra.run.smithi038.stdout:00000020 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 2024-09-21T03:40:12.037 INFO:teuthology.orchestra.run.smithi038.stdout:* 2024-09-21T03:40:12.037 INFO:teuthology.orchestra.run.smithi038.stdout:00000040 00 00 00 00 03 02 28 00 00 00 00 00 00 00 00 00 |......(.........| 2024-09-21T03:40:12.037 INFO:teuthology.orchestra.run.smithi038.stdout:00000050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 2024-09-21T03:40:12.037 INFO:teuthology.orchestra.run.smithi038.stdout:* 2024-09-21T03:40:12.037 INFO:teuthology.orchestra.run.smithi038.stdout:00000070 00 00 03 02 38 00 00 00 00 00 00 00 00 00 00 00 |....8...........| 2024-09-21T03:40:12.038 INFO:teuthology.orchestra.run.smithi038.stdout:00000080 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 2024-09-21T03:40:12.038 INFO:teuthology.orchestra.run.smithi038.stdout:* 2024-09-21T03:40:12.038 INFO:teuthology.orchestra.run.smithi038.stdout:000000b0 03 02 38 00 00 00 00 00 00 00 00 00 00 00 00 00 |..8.............| 2024-09-21T03:40:12.038 INFO:teuthology.orchestra.run.smithi038.stdout:000000c0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 2024-09-21T03:40:12.038 INFO:teuthology.orchestra.run.smithi038.stdout:* 2024-09-21T03:40:12.038 INFO:teuthology.orchestra.run.smithi038.stdout:00000110 00 00 |..| 2024-09-21T03:40:12.038 INFO:teuthology.orchestra.run.smithi038.stdout:00000112 2024-09-21T03:40:12.038 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:40:12.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.033+0000 7fd3c63b4f40 1 -- 172.21.15.38:0/1749409884 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x560c6436b4b0 msgr2=0x560c6436b8c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:12.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.033+0000 7fd3c63b4f40 1 --2- 172.21.15.38:0/1749409884 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x560c6436b4b0 0x560c6436b8c0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:12.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.033+0000 7fd3c63b4f40 1 -- 172.21.15.38:0/1749409884 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd3ac0775a0 msgr2=0x7fd3ac079ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:12.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.033+0000 7fd3c63b4f40 1 --2- 172.21.15.38:0/1749409884 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd3ac0775a0 0x7fd3ac079ab0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fd3b4025e50 tx=0x7fd3b4004940 comp rx=0 tx=0).stop 2024-09-21T03:40:12.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.033+0000 7fd3c63b4f40 1 -- 172.21.15.38:0/1749409884 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560c6420fd00 msgr2=0x560c64253410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:12.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.033+0000 7fd3c63b4f40 1 --2- 172.21.15.38:0/1749409884 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560c6420fd00 0x560c64253410 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7fd3b000a6e0 tx=0x7fd3b000aba0 comp rx=0 tx=0).stop 2024-09-21T03:40:12.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.033+0000 7fd3c066f640 1 -- 172.21.15.38:0/1749409884 reap_dead start 2024-09-21T03:40:12.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.033+0000 7fd3c63b4f40 1 -- 172.21.15.38:0/1749409884 shutdown_connections 2024-09-21T03:40:12.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.033+0000 7fd3c63b4f40 1 -- 172.21.15.38:0/1749409884 >> 172.21.15.38:0/1749409884 conn(0x560c6412eac0 msgr2=0x560c6420b660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:12.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.033+0000 7fd3c63b4f40 1 -- 172.21.15.38:0/1749409884 shutdown_connections 2024-09-21T03:40:12.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.033+0000 7fd3c63b4f40 1 -- 172.21.15.38:0/1749409884 wait complete. 2024-09-21T03:40:12.041 DEBUG:teuthology.orchestra.run.smithi038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getomapheader 1.00000000.inode 2024-09-21T03:40:12.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.094+0000 7f2d2dc16f40 1 Processor -- start 2024-09-21T03:40:12.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.094+0000 7f2d2dc16f40 1 -- start start 2024-09-21T03:40:12.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.094+0000 7f2d2dc16f40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561cce1fc240 0x561cce2b6d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:12.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.094+0000 7f2d2dc16f40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561cce243090 0x561cce2d08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:12.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.094+0000 7f2d2dc16f40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x561cce2b8520 0x561cce364270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:12.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.094+0000 7f2d2dc16f40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x561cce1a8430 con 0x561cce243090 2024-09-21T03:40:12.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.094+0000 7f2d2dc16f40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x561cce1b51f0 con 0x561cce1fc240 2024-09-21T03:40:12.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.094+0000 7f2d2dc16f40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x561cce149d60 con 0x561cce2b8520 2024-09-21T03:40:12.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.094+0000 7f2d2625f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561cce243090 0x561cce2d08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:12.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.095+0000 7f2d2625f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561cce243090 0x561cce2d08b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:55894/0 (socket says 172.21.15.38:55894) 2024-09-21T03:40:12.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.095+0000 7f2d2625f640 1 -- 172.21.15.38:0/1761102720 learned_addr learned my addr 172.21.15.38:0/1761102720 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:12.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.095+0000 7f2d26a60640 1 --2- 172.21.15.38:0/1761102720 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561cce1fc240 0x561cce2b6d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:12.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.095+0000 7f2d27261640 1 --2- 172.21.15.38:0/1761102720 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x561cce2b8520 0x561cce364270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:12.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.095+0000 7f2d2625f640 1 -- 172.21.15.38:0/1761102720 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x561cce2b8520 msgr2=0x561cce364270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:12.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.095+0000 7f2d2625f640 1 --2- 172.21.15.38:0/1761102720 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x561cce2b8520 0x561cce364270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:12.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.095+0000 7f2d2625f640 1 -- 172.21.15.38:0/1761102720 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561cce1fc240 msgr2=0x561cce2b6d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:12.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.095+0000 7f2d2625f640 1 --2- 172.21.15.38:0/1761102720 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561cce1fc240 0x561cce2b6d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:12.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.095+0000 7f2d2625f640 1 -- 172.21.15.38:0/1761102720 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561cce2b0a60 con 0x561cce243090 2024-09-21T03:40:12.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.095+0000 7f2d2625f640 1 --2- 172.21.15.38:0/1761102720 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561cce243090 0x561cce2d08b0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c046600 tx=0x7f2d1c002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c763456cef3c3585 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:12.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.095+0000 7f2d2525d640 1 -- 172.21.15.38:0/1761102720 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d1c066070 con 0x561cce243090 2024-09-21T03:40:12.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.095+0000 7f2d2525d640 1 -- 172.21.15.38:0/1761102720 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2d1c056560 con 0x561cce243090 2024-09-21T03:40:12.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.095+0000 7f2d2525d640 1 -- 172.21.15.38:0/1761102720 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d1c056910 con 0x561cce243090 2024-09-21T03:40:12.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.096+0000 7f2d2dc16f40 1 -- 172.21.15.38:0/1761102720 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561cce243090 msgr2=0x561cce2d08b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:12.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.096+0000 7f2d2dc16f40 1 --2- 172.21.15.38:0/1761102720 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561cce243090 0x561cce2d08b0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c046600 tx=0x7f2d1c002fb0 comp rx=0 tx=0).stop 2024-09-21T03:40:12.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.096+0000 7f2d2dc16f40 1 -- 172.21.15.38:0/1761102720 shutdown_connections 2024-09-21T03:40:12.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.096+0000 7f2d2dc16f40 1 --2- 172.21.15.38:0/1761102720 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x561cce2b8520 0x561cce364270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:12.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.096+0000 7f2d2dc16f40 1 --2- 172.21.15.38:0/1761102720 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561cce243090 0x561cce2d08b0 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:12.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.096+0000 7f2d2dc16f40 1 --2- 172.21.15.38:0/1761102720 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561cce1fc240 0x561cce2b6d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:12.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.096+0000 7f2d2dc16f40 1 -- 172.21.15.38:0/1761102720 >> 172.21.15.38:0/1761102720 conn(0x561cce11aac0 msgr2=0x561cce2c8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:12.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.096+0000 7f2d2dc16f40 1 -- 172.21.15.38:0/1761102720 shutdown_connections 2024-09-21T03:40:12.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.096+0000 7f2d2dc16f40 1 -- 172.21.15.38:0/1761102720 wait complete. 2024-09-21T03:40:12.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.098+0000 7f2d2dc16f40 1 Processor -- start 2024-09-21T03:40:12.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.098+0000 7f2d2dc16f40 1 -- start start 2024-09-21T03:40:12.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.099+0000 7f2d2dc16f40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561cce1fc240 0x561cce2ce3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:12.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.099+0000 7f2d2dc16f40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x561cce243090 0x561cce23f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:12.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.099+0000 7f2d2dc16f40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561cce2b8520 0x561cce2add90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:12.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.099+0000 7f2d2dc16f40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x561cce2b1050 con 0x561cce1fc240 2024-09-21T03:40:12.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.099+0000 7f2d2dc16f40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x561cce36ca30 con 0x561cce2b8520 2024-09-21T03:40:12.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.099+0000 7f2d2dc16f40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x561cce2ae930 con 0x561cce243090 2024-09-21T03:40:12.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.099+0000 7f2d26a60640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561cce1fc240 0x561cce2ce3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:12.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.099+0000 7f2d26a60640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561cce1fc240 0x561cce2ce3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:55906/0 (socket says 172.21.15.38:55906) 2024-09-21T03:40:12.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.099+0000 7f2d26a60640 1 -- 172.21.15.38:0/1782560803 learned_addr learned my addr 172.21.15.38:0/1782560803 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:12.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.099+0000 7f2d27261640 1 --2- 172.21.15.38:0/1782560803 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561cce2b8520 0x561cce2add90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:12.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.099+0000 7f2d2625f640 1 --2- 172.21.15.38:0/1782560803 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x561cce243090 0x561cce23f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:12.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.099+0000 7f2d26a60640 1 -- 172.21.15.38:0/1782560803 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x561cce243090 msgr2=0x561cce23f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:12.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.099+0000 7f2d26a60640 1 --2- 172.21.15.38:0/1782560803 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x561cce243090 0x561cce23f1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:12.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.099+0000 7f2d26a60640 1 -- 172.21.15.38:0/1782560803 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561cce2b8520 msgr2=0x561cce2add90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:12.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.099+0000 7f2d26a60640 1 --2- 172.21.15.38:0/1782560803 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561cce2b8520 0x561cce2add90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:12.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.099+0000 7f2d26a60640 1 -- 172.21.15.38:0/1782560803 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561cce23ea10 con 0x561cce1fc240 2024-09-21T03:40:12.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.099+0000 7f2d2625f640 1 --2- 172.21.15.38:0/1782560803 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x561cce243090 0x561cce23f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:40:12.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.099+0000 7f2d26a60640 1 --2- 172.21.15.38:0/1782560803 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561cce1fc240 0x561cce2ce3b0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f2d1800ef30 tx=0x7f2d1800c620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:12.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.099+0000 7f2d0ffff640 1 -- 172.21.15.38:0/1782560803 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d18015020 con 0x561cce1fc240 2024-09-21T03:40:12.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.099+0000 7f2d0ffff640 1 -- 172.21.15.38:0/1782560803 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2d18011450 con 0x561cce1fc240 2024-09-21T03:40:12.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.099+0000 7f2d2dc16f40 1 -- 172.21.15.38:0/1782560803 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x561cce36c770 con 0x561cce1fc240 2024-09-21T03:40:12.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.099+0000 7f2d0ffff640 1 -- 172.21.15.38:0/1782560803 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d18011860 con 0x561cce1fc240 2024-09-21T03:40:12.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.099+0000 7f2d2dc16f40 1 -- 172.21.15.38:0/1782560803 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561cce23f710 con 0x561cce1fc240 2024-09-21T03:40:12.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.101+0000 7f2d0ffff640 1 -- 172.21.15.38:0/1782560803 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f2d18018020 con 0x561cce1fc240 2024-09-21T03:40:12.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.101+0000 7f2d0ffff640 1 --2- 172.21.15.38:0/1782560803 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2d140775a0 0x7f2d14079ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:12.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.101+0000 7f2d0ffff640 1 -- 172.21.15.38:0/1782560803 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6187+0+0 (secure 0 0 0) 0x7f2d18099c80 con 0x561cce1fc240 2024-09-21T03:40:12.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.102+0000 7f2d2dc16f40 1 --2- 172.21.15.38:0/1782560803 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x561cce2cc870 0x561cce36bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:12.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.102+0000 7f2d2625f640 1 --2- 172.21.15.38:0/1782560803 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2d140775a0 0x7f2d14079ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:12.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.102+0000 7f2d2dc16f40 1 -- 172.21.15.38:0/1782560803 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:1 13.14 13:287034c4:::1.00000000.inode:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x561cce2ac520 con 0x561cce2cc870 2024-09-21T03:40:12.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.102+0000 7f2d27261640 1 --2- 172.21.15.38:0/1782560803 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x561cce2cc870 0x561cce36bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:12.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.102+0000 7f2d2625f640 1 --2- 172.21.15.38:0/1782560803 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2d140775a0 0x7f2d14079ab0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c063000 tx=0x7f2d1c004880 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:12.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.102+0000 7f2d27261640 1 --2- 172.21.15.38:0/1782560803 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x561cce2cc870 0x561cce36bfd0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:12.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.102+0000 7f2d27261640 1 -- 172.21.15.38:0/1782560803 <== osd.6 v2:172.21.15.195:6826/4055825675 1 ==== osd_op_reply(1 1.00000000.inode [omap-get-header] v0'0 uv15 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f2d200028a0 con 0x561cce2cc870 2024-09-21T03:40:12.107 INFO:teuthology.orchestra.run.smithi038.stdout:header (0 bytes) : 2024-09-21T03:40:12.107 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:40:12.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.102+0000 7f2d2dc16f40 1 -- 172.21.15.38:0/1782560803 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x561cce2cc870 msgr2=0x561cce36bfd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:12.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.102+0000 7f2d2dc16f40 1 --2- 172.21.15.38:0/1782560803 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x561cce2cc870 0x561cce36bfd0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:12.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.103+0000 7f2d2dc16f40 1 -- 172.21.15.38:0/1782560803 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2d140775a0 msgr2=0x7f2d14079ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:12.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.103+0000 7f2d2dc16f40 1 --2- 172.21.15.38:0/1782560803 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2d140775a0 0x7f2d14079ab0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c063000 tx=0x7f2d1c004880 comp rx=0 tx=0).stop 2024-09-21T03:40:12.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.103+0000 7f2d2dc16f40 1 -- 172.21.15.38:0/1782560803 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561cce1fc240 msgr2=0x561cce2ce3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:12.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.103+0000 7f2d2dc16f40 1 --2- 172.21.15.38:0/1782560803 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561cce1fc240 0x561cce2ce3b0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f2d1800ef30 tx=0x7f2d1800c620 comp rx=0 tx=0).stop 2024-09-21T03:40:12.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.103+0000 7f2d27261640 1 -- 172.21.15.38:0/1782560803 reap_dead start 2024-09-21T03:40:12.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.103+0000 7f2d2dc16f40 1 -- 172.21.15.38:0/1782560803 shutdown_connections 2024-09-21T03:40:12.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.103+0000 7f2d2dc16f40 1 -- 172.21.15.38:0/1782560803 >> 172.21.15.38:0/1782560803 conn(0x561cce11aac0 msgr2=0x561cce2c8c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:12.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.103+0000 7f2d2dc16f40 1 -- 172.21.15.38:0/1782560803 shutdown_connections 2024-09-21T03:40:12.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.103+0000 7f2d2dc16f40 1 -- 172.21.15.38:0/1782560803 wait complete. 2024-09-21T03:40:12.109 DEBUG:teuthology.orchestra.run.smithi038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getomapheader 200.00000000 2024-09-21T03:40:12.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.162+0000 7fa12ba13f40 1 Processor -- start 2024-09-21T03:40:12.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.162+0000 7fa12ba13f40 1 -- start start 2024-09-21T03:40:12.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.163+0000 7fa12ba13f40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa1140022d0 0x562b54c9e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:12.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.163+0000 7fa12ba13f40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562b54d75320 0x562b54d765a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:12.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.163+0000 7fa12ba13f40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562b54d788f0 0x562b54d78d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:12.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.163+0000 7fa12ba13f40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x562b54c4f430 con 0x7fa1140022d0 2024-09-21T03:40:12.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.163+0000 7fa12ba13f40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x562b54c5c1f0 con 0x562b54d788f0 2024-09-21T03:40:12.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.163+0000 7fa12ba13f40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x562b54bf0d60 con 0x562b54d75320 2024-09-21T03:40:12.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.163+0000 7fa124860640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa1140022d0 0x562b54c9e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:12.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.163+0000 7fa124860640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa1140022d0 0x562b54c9e900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:55928/0 (socket says 172.21.15.38:55928) 2024-09-21T03:40:12.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.163+0000 7fa124860640 1 -- 172.21.15.38:0/2998646831 learned_addr learned my addr 172.21.15.38:0/2998646831 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:12.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.164+0000 7fa11ffff640 1 --2- 172.21.15.38:0/2998646831 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562b54d75320 0x562b54d765a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:12.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.164+0000 7fa125061640 1 --2- 172.21.15.38:0/2998646831 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562b54d788f0 0x562b54d78d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:12.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.164+0000 7fa124860640 1 -- 172.21.15.38:0/2998646831 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562b54d75320 msgr2=0x562b54d765a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:12.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.164+0000 7fa124860640 1 --2- 172.21.15.38:0/2998646831 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562b54d75320 0x562b54d765a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:12.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.164+0000 7fa124860640 1 -- 172.21.15.38:0/2998646831 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562b54d788f0 msgr2=0x562b54d78d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:12.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.164+0000 7fa124860640 1 --2- 172.21.15.38:0/2998646831 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562b54d788f0 0x562b54d78d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:12.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.164+0000 7fa124860640 1 -- 172.21.15.38:0/2998646831 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b54d75c50 con 0x7fa1140022d0 2024-09-21T03:40:12.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.164+0000 7fa124860640 1 --2- 172.21.15.38:0/2998646831 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa1140022d0 0x562b54c9e900 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7fa118053040 tx=0x7fa1180012f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a974f81e56c9cdae server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:12.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.164+0000 7fa11effd640 1 -- 172.21.15.38:0/2998646831 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1180507c0 con 0x7fa1140022d0 2024-09-21T03:40:12.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.164+0000 7fa11effd640 1 -- 172.21.15.38:0/2998646831 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa118050960 con 0x7fa1140022d0 2024-09-21T03:40:12.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.164+0000 7fa11effd640 1 -- 172.21.15.38:0/2998646831 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa118050d90 con 0x7fa1140022d0 2024-09-21T03:40:12.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.165+0000 7fa12ba13f40 1 -- 172.21.15.38:0/2998646831 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa1140022d0 msgr2=0x562b54c9e900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:12.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.165+0000 7fa12ba13f40 1 --2- 172.21.15.38:0/2998646831 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa1140022d0 0x562b54c9e900 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7fa118053040 tx=0x7fa1180012f0 comp rx=0 tx=0).stop 2024-09-21T03:40:12.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.165+0000 7fa12ba13f40 1 -- 172.21.15.38:0/2998646831 shutdown_connections 2024-09-21T03:40:12.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.165+0000 7fa12ba13f40 1 --2- 172.21.15.38:0/2998646831 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562b54d788f0 0x562b54d78d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:12.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.165+0000 7fa12ba13f40 1 --2- 172.21.15.38:0/2998646831 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562b54d75320 0x562b54d765a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:12.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.165+0000 7fa12ba13f40 1 --2- 172.21.15.38:0/2998646831 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa1140022d0 0x562b54c9e900 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:12.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.165+0000 7fa12ba13f40 1 -- 172.21.15.38:0/2998646831 >> 172.21.15.38:0/2998646831 conn(0x562b54bc1ac0 msgr2=0x562b54d5c760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:12.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.165+0000 7fa12ba13f40 1 -- 172.21.15.38:0/2998646831 shutdown_connections 2024-09-21T03:40:12.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.165+0000 7fa12ba13f40 1 -- 172.21.15.38:0/2998646831 wait complete. 2024-09-21T03:40:12.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.166+0000 7fa12ba13f40 1 Processor -- start 2024-09-21T03:40:12.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.166+0000 7fa12ba13f40 1 -- start start 2024-09-21T03:40:12.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.167+0000 7fa12ba13f40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562b54d75320 0x562b54d58670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:12.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.167+0000 7fa12ba13f40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562b54d788f0 0x562b54d5c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:12.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.167+0000 7fa12ba13f40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa1140022d0 0x562b54d5ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:12.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.167+0000 7fa12ba13f40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x562b54d761f0 con 0x562b54d788f0 2024-09-21T03:40:12.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.167+0000 7fa12ba13f40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x562b54c9ee40 con 0x562b54d75320 2024-09-21T03:40:12.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.167+0000 7fa12ba13f40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x562b54d70620 con 0x7fa1140022d0 2024-09-21T03:40:12.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.167+0000 7fa11ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562b54d788f0 0x562b54d5c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:12.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.167+0000 7fa11ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562b54d788f0 0x562b54d5c910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:55944/0 (socket says 172.21.15.38:55944) 2024-09-21T03:40:12.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.167+0000 7fa11ffff640 1 -- 172.21.15.38:0/3955724536 learned_addr learned my addr 172.21.15.38:0/3955724536 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:12.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.167+0000 7fa125061640 1 --2- 172.21.15.38:0/3955724536 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa1140022d0 0x562b54d5ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:12.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.167+0000 7fa124860640 1 --2- 172.21.15.38:0/3955724536 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562b54d75320 0x562b54d58670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:12.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.167+0000 7fa11ffff640 1 -- 172.21.15.38:0/3955724536 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa1140022d0 msgr2=0x562b54d5ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:12.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.167+0000 7fa11ffff640 1 --2- 172.21.15.38:0/3955724536 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa1140022d0 0x562b54d5ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:12.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.167+0000 7fa11ffff640 1 -- 172.21.15.38:0/3955724536 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562b54d75320 msgr2=0x562b54d58670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:12.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.167+0000 7fa11ffff640 1 --2- 172.21.15.38:0/3955724536 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562b54d75320 0x562b54d58670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:12.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.167+0000 7fa11ffff640 1 -- 172.21.15.38:0/3955724536 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b54d58bb0 con 0x562b54d788f0 2024-09-21T03:40:12.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.168+0000 7fa11ffff640 1 --2- 172.21.15.38:0/3955724536 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562b54d788f0 0x562b54d5c910 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fa110010410 tx=0x7fa1100108d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:12.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.168+0000 7fa11dffb640 1 -- 172.21.15.38:0/3955724536 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa11000e8c0 con 0x562b54d788f0 2024-09-21T03:40:12.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.168+0000 7fa11dffb640 1 -- 172.21.15.38:0/3955724536 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa11000ea60 con 0x562b54d788f0 2024-09-21T03:40:12.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.168+0000 7fa12ba13f40 1 -- 172.21.15.38:0/3955724536 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x562b54d5be00 con 0x562b54d788f0 2024-09-21T03:40:12.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.168+0000 7fa11dffb640 1 -- 172.21.15.38:0/3955724536 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa11000ee50 con 0x562b54d788f0 2024-09-21T03:40:12.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.168+0000 7fa12ba13f40 1 -- 172.21.15.38:0/3955724536 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b54d5a2a0 con 0x562b54d788f0 2024-09-21T03:40:12.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.170+0000 7fa11dffb640 1 -- 172.21.15.38:0/3955724536 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa110019440 con 0x562b54d788f0 2024-09-21T03:40:12.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.170+0000 7fa11dffb640 1 --2- 172.21.15.38:0/3955724536 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa108077570 0x7fa108079a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:12.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.170+0000 7fa11dffb640 1 -- 172.21.15.38:0/3955724536 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6187+0+0 (secure 0 0 0) 0x7fa11009a490 con 0x562b54d788f0 2024-09-21T03:40:12.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.170+0000 7fa124860640 1 --2- 172.21.15.38:0/3955724536 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa108077570 0x7fa108079a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:12.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.170+0000 7fa12ba13f40 1 --2- 172.21.15.38:0/3955724536 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x562b54d6cc70 0x562b54d6d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:12.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.170+0000 7fa12ba13f40 1 -- 172.21.15.38:0/3955724536 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:1 13.14 13:292cf221:::200.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x562b54d6d5c0 con 0x562b54d6cc70 2024-09-21T03:40:12.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.171+0000 7fa125061640 1 --2- 172.21.15.38:0/3955724536 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x562b54d6cc70 0x562b54d6d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:12.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.171+0000 7fa124860640 1 --2- 172.21.15.38:0/3955724536 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa108077570 0x7fa108079a80 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fa118065000 tx=0x7fa11805c040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:12.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.171+0000 7fa125061640 1 --2- 172.21.15.38:0/3955724536 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x562b54d6cc70 0x562b54d6d080 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:12.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.171+0000 7fa125061640 1 -- 172.21.15.38:0/3955724536 <== osd.6 v2:172.21.15.195:6826/4055825675 1 ==== osd_op_reply(1 200.00000000 [omap-get-header] v0'0 uv18 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa120003040 con 0x562b54d6cc70 2024-09-21T03:40:12.176 INFO:teuthology.orchestra.run.smithi038.stdout:header (0 bytes) : 2024-09-21T03:40:12.176 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:40:12.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.171+0000 7fa12ba13f40 1 -- 172.21.15.38:0/3955724536 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x562b54d6cc70 msgr2=0x562b54d6d080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:12.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.171+0000 7fa12ba13f40 1 --2- 172.21.15.38:0/3955724536 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x562b54d6cc70 0x562b54d6d080 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:12.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.171+0000 7fa12ba13f40 1 -- 172.21.15.38:0/3955724536 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa108077570 msgr2=0x7fa108079a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:12.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.171+0000 7fa12ba13f40 1 --2- 172.21.15.38:0/3955724536 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa108077570 0x7fa108079a80 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fa118065000 tx=0x7fa11805c040 comp rx=0 tx=0).stop 2024-09-21T03:40:12.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.171+0000 7fa12ba13f40 1 -- 172.21.15.38:0/3955724536 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562b54d788f0 msgr2=0x562b54d5c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:12.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.171+0000 7fa12ba13f40 1 --2- 172.21.15.38:0/3955724536 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562b54d788f0 0x562b54d5c910 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fa110010410 tx=0x7fa1100108d0 comp rx=0 tx=0).stop 2024-09-21T03:40:12.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.171+0000 7fa125061640 1 -- 172.21.15.38:0/3955724536 reap_dead start 2024-09-21T03:40:12.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.172+0000 7fa12ba13f40 1 -- 172.21.15.38:0/3955724536 shutdown_connections 2024-09-21T03:40:12.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.172+0000 7fa12ba13f40 1 -- 172.21.15.38:0/3955724536 >> 172.21.15.38:0/3955724536 conn(0x562b54bc1ac0 msgr2=0x562b54d77000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:12.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.172+0000 7fa12ba13f40 1 -- 172.21.15.38:0/3955724536 shutdown_connections 2024-09-21T03:40:12.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.172+0000 7fa12ba13f40 1 -- 172.21.15.38:0/3955724536 wait complete. 2024-09-21T03:40:12.178 DEBUG:teuthology.orchestra.run.smithi038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getomapheader 200.00000001 2024-09-21T03:40:12.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.230+0000 7fa9d4b57f40 1 Processor -- start 2024-09-21T03:40:12.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.230+0000 7fa9d4b57f40 1 -- start start 2024-09-21T03:40:12.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.231+0000 7fa9d4b57f40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55c04831dd00 0x55c0483d4350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:12.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.231+0000 7fa9d4b57f40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c0483d8950 0x55c0483d2c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:12.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.232+0000 7fa9d4b57f40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c0483f3670 0x55c0483d3170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:12.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.232+0000 7fa9d4b57f40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55c04826bd60 con 0x55c0483f3670 2024-09-21T03:40:12.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.232+0000 7fa9d4b57f40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55c0482ca430 con 0x55c0483d8950 2024-09-21T03:40:12.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.232+0000 7fa9d4b57f40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55c0483661d0 con 0x55c04831dd00 2024-09-21T03:40:12.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.232+0000 7fa9ce611640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55c04831dd00 0x55c0483d4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:12.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.232+0000 7fa9cee12640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c0483f3670 0x55c0483d3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:12.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.232+0000 7fa9ce611640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55c04831dd00 0x55c0483d4350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.38:46312/0 (socket says 172.21.15.38:46312) 2024-09-21T03:40:12.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.232+0000 7fa9ce611640 1 -- 172.21.15.38:0/2558595754 learned_addr learned my addr 172.21.15.38:0/2558595754 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:12.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.232+0000 7fa9cde10640 1 --2- 172.21.15.38:0/2558595754 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c0483d8950 0x55c0483d2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:12.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.232+0000 7fa9ce611640 1 -- 172.21.15.38:0/2558595754 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c0483d8950 msgr2=0x55c0483d2c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:12.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.232+0000 7fa9ce611640 1 --2- 172.21.15.38:0/2558595754 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c0483d8950 0x55c0483d2c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:12.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.232+0000 7fa9ce611640 1 -- 172.21.15.38:0/2558595754 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c0483f3670 msgr2=0x55c0483d3170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:12.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.232+0000 7fa9ce611640 1 --2- 172.21.15.38:0/2558595754 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c0483f3670 0x55c0483d3170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:12.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.232+0000 7fa9ce611640 1 -- 172.21.15.38:0/2558595754 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c0483d3af0 con 0x55c04831dd00 2024-09-21T03:40:12.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.232+0000 7fa9cde10640 1 --2- 172.21.15.38:0/2558595754 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c0483d8950 0x55c0483d2c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:40:12.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.232+0000 7fa9ce611640 1 --2- 172.21.15.38:0/2558595754 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55c04831dd00 0x55c0483d4350 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fa9c000b7c0 tx=0x7fa9c000bc80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e66a1a4a662335df server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:12.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.232+0000 7fa9cce0e640 1 -- 172.21.15.38:0/2558595754 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9c000ab40 con 0x55c04831dd00 2024-09-21T03:40:12.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.233+0000 7fa9cce0e640 1 -- 172.21.15.38:0/2558595754 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa9c000ace0 con 0x55c04831dd00 2024-09-21T03:40:12.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.233+0000 7fa9cce0e640 1 -- 172.21.15.38:0/2558595754 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9c0015680 con 0x55c04831dd00 2024-09-21T03:40:12.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.233+0000 7fa9d4b57f40 1 -- 172.21.15.38:0/2558595754 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55c04831dd00 msgr2=0x55c0483d4350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:12.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.233+0000 7fa9d4b57f40 1 --2- 172.21.15.38:0/2558595754 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55c04831dd00 0x55c0483d4350 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fa9c000b7c0 tx=0x7fa9c000bc80 comp rx=0 tx=0).stop 2024-09-21T03:40:12.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.233+0000 7fa9d4b57f40 1 -- 172.21.15.38:0/2558595754 shutdown_connections 2024-09-21T03:40:12.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.233+0000 7fa9d4b57f40 1 --2- 172.21.15.38:0/2558595754 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c0483f3670 0x55c0483d3170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:12.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.233+0000 7fa9d4b57f40 1 --2- 172.21.15.38:0/2558595754 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c0483d8950 0x55c0483d2c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:12.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.233+0000 7fa9d4b57f40 1 --2- 172.21.15.38:0/2558595754 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55c04831dd00 0x55c0483d4350 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:12.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.233+0000 7fa9d4b57f40 1 -- 172.21.15.38:0/2558595754 >> 172.21.15.38:0/2558595754 conn(0x55c04823cac0 msgr2=0x55c0483f3a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:12.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.233+0000 7fa9d4b57f40 1 -- 172.21.15.38:0/2558595754 shutdown_connections 2024-09-21T03:40:12.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.233+0000 7fa9d4b57f40 1 -- 172.21.15.38:0/2558595754 wait complete. 2024-09-21T03:40:12.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.234+0000 7fa9d4b57f40 1 Processor -- start 2024-09-21T03:40:12.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.234+0000 7fa9d4b57f40 1 -- start start 2024-09-21T03:40:12.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.235+0000 7fa9d4b57f40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c04831dd00 0x55c0483d2330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:12.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.235+0000 7fa9d4b57f40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c0483d8950 0x55c04847a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:12.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.235+0000 7fa9d4b57f40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55c0483f3670 0x55c04847ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:12.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.235+0000 7fa9d4b57f40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55c04834a7b0 con 0x55c0483d8950 2024-09-21T03:40:12.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.235+0000 7fa9d4b57f40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55c048346720 con 0x55c04831dd00 2024-09-21T03:40:12.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.235+0000 7fa9d4b57f40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55c0483d66c0 con 0x55c0483f3670 2024-09-21T03:40:12.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.235+0000 7fa9cde10640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c0483d8950 0x55c04847a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:12.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.236+0000 7fa9cde10640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c0483d8950 0x55c04847a810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:55954/0 (socket says 172.21.15.38:55954) 2024-09-21T03:40:12.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.236+0000 7fa9cde10640 1 -- 172.21.15.38:0/3868043271 learned_addr learned my addr 172.21.15.38:0/3868043271 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:12.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.236+0000 7fa9cee12640 1 --2- 172.21.15.38:0/3868043271 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55c0483f3670 0x55c04847ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:12.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.236+0000 7fa9ce611640 1 --2- 172.21.15.38:0/3868043271 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c04831dd00 0x55c0483d2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:12.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.236+0000 7fa9cde10640 1 -- 172.21.15.38:0/3868043271 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55c0483f3670 msgr2=0x55c04847ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:12.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.236+0000 7fa9cde10640 1 --2- 172.21.15.38:0/3868043271 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55c0483f3670 0x55c04847ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:12.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.236+0000 7fa9cde10640 1 -- 172.21.15.38:0/3868043271 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c04831dd00 msgr2=0x55c0483d2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:12.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.236+0000 7fa9cde10640 1 --2- 172.21.15.38:0/3868043271 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c04831dd00 0x55c0483d2330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:12.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.236+0000 7fa9cde10640 1 -- 172.21.15.38:0/3868043271 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c0483d7da0 con 0x55c0483d8950 2024-09-21T03:40:12.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.236+0000 7fa9cee12640 1 --2- 172.21.15.38:0/3868043271 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55c0483f3670 0x55c04847ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:40:12.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.236+0000 7fa9ce611640 1 --2- 172.21.15.38:0/3868043271 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c04831dd00 0x55c0483d2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:40:12.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.236+0000 7fa9cde10640 1 --2- 172.21.15.38:0/3868043271 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c0483d8950 0x55c04847a810 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7fa9c4046600 tx=0x7fa9c404d7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:12.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.236+0000 7fa9bf7fe640 1 -- 172.21.15.38:0/3868043271 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9c4067070 con 0x55c0483d8950 2024-09-21T03:40:12.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.236+0000 7fa9bf7fe640 1 -- 172.21.15.38:0/3868043271 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa9c4062760 con 0x55c0483d8950 2024-09-21T03:40:12.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.236+0000 7fa9d4b57f40 1 -- 172.21.15.38:0/3868043271 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55c0483d5930 con 0x55c0483d8950 2024-09-21T03:40:12.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.236+0000 7fa9bf7fe640 1 -- 172.21.15.38:0/3868043271 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9c4062b70 con 0x55c0483d8950 2024-09-21T03:40:12.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.236+0000 7fa9d4b57f40 1 -- 172.21.15.38:0/3868043271 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c0483ec590 con 0x55c0483d8950 2024-09-21T03:40:12.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.238+0000 7fa9bf7fe640 1 -- 172.21.15.38:0/3868043271 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa9c406a070 con 0x55c0483d8950 2024-09-21T03:40:12.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.238+0000 7fa9bf7fe640 1 --2- 172.21.15.38:0/3868043271 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa9b00775a0 0x7fa9b0079ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:12.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.239+0000 7fa9bf7fe640 1 -- 172.21.15.38:0/3868043271 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6187+0+0 (secure 0 0 0) 0x7fa9c40dd6d0 con 0x55c0483d8950 2024-09-21T03:40:12.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.239+0000 7fa9ce611640 1 --2- 172.21.15.38:0/3868043271 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa9b00775a0 0x7fa9b0079ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:12.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.239+0000 7fa9d4b57f40 1 --2- 172.21.15.38:0/3868043271 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x55c0483d5fc0 0x55c04848dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:12.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.239+0000 7fa9d4b57f40 1 -- 172.21.15.38:0/3868043271 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:1 13.14 13:2e2fa760:::200.00000001:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x55c04834beb0 con 0x55c0483d5fc0 2024-09-21T03:40:12.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.239+0000 7fa9cee12640 1 --2- 172.21.15.38:0/3868043271 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x55c0483d5fc0 0x55c04848dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:12.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.239+0000 7fa9ce611640 1 --2- 172.21.15.38:0/3868043271 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa9b00775a0 0x7fa9b0079ab0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fa9c0012040 tx=0x7fa9c0017040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:12.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.239+0000 7fa9cee12640 1 --2- 172.21.15.38:0/3868043271 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x55c0483d5fc0 0x55c04848dcf0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:12.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.240+0000 7fa9cee12640 1 -- 172.21.15.38:0/3868043271 <== osd.6 v2:172.21.15.195:6826/4055825675 1 ==== osd_op_reply(1 200.00000001 [omap-get-header] v0'0 uv16 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa9c8002640 con 0x55c0483d5fc0 2024-09-21T03:40:12.244 INFO:teuthology.orchestra.run.smithi038.stdout:header (0 bytes) : 2024-09-21T03:40:12.244 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:40:12.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.240+0000 7fa9d4b57f40 1 -- 172.21.15.38:0/3868043271 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x55c0483d5fc0 msgr2=0x55c04848dcf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:12.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.240+0000 7fa9d4b57f40 1 --2- 172.21.15.38:0/3868043271 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x55c0483d5fc0 0x55c04848dcf0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:12.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.240+0000 7fa9d4b57f40 1 -- 172.21.15.38:0/3868043271 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa9b00775a0 msgr2=0x7fa9b0079ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:12.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.240+0000 7fa9d4b57f40 1 --2- 172.21.15.38:0/3868043271 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa9b00775a0 0x7fa9b0079ab0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fa9c0012040 tx=0x7fa9c0017040 comp rx=0 tx=0).stop 2024-09-21T03:40:12.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.240+0000 7fa9d4b57f40 1 -- 172.21.15.38:0/3868043271 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c0483d8950 msgr2=0x55c04847a810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:12.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.240+0000 7fa9d4b57f40 1 --2- 172.21.15.38:0/3868043271 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c0483d8950 0x55c04847a810 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7fa9c4046600 tx=0x7fa9c404d7e0 comp rx=0 tx=0).stop 2024-09-21T03:40:12.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.240+0000 7fa9cee12640 1 -- 172.21.15.38:0/3868043271 reap_dead start 2024-09-21T03:40:12.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.240+0000 7fa9d4b57f40 1 -- 172.21.15.38:0/3868043271 shutdown_connections 2024-09-21T03:40:12.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.240+0000 7fa9d4b57f40 1 -- 172.21.15.38:0/3868043271 >> 172.21.15.38:0/3868043271 conn(0x55c04823cac0 msgr2=0x55c0483f3a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:12.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.240+0000 7fa9d4b57f40 1 -- 172.21.15.38:0/3868043271 shutdown_connections 2024-09-21T03:40:12.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.240+0000 7fa9d4b57f40 1 -- 172.21.15.38:0/3868043271 wait complete. 2024-09-21T03:40:12.247 DEBUG:teuthology.orchestra.run.smithi038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getomapheader mds0_openfiles.0 2024-09-21T03:40:12.302 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.299+0000 7fbee8153f40 1 Processor -- start 2024-09-21T03:40:12.302 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.299+0000 7fbee8153f40 1 -- start start 2024-09-21T03:40:12.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.300+0000 7fbee8153f40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x557d876e5240 0x557d877b2da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:12.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.300+0000 7fbee8153f40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557d8772c090 0x557d8779cd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:12.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.300+0000 7fbee8153f40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557d877a1520 0x557d876e3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:12.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.300+0000 7fbee8153f40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x557d87632d60 con 0x557d877a1520 2024-09-21T03:40:12.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.300+0000 7fbee8153f40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x557d87691430 con 0x557d8772c090 2024-09-21T03:40:12.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.300+0000 7fbee8153f40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x557d877b8aa0 con 0x557d876e5240 2024-09-21T03:40:12.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.300+0000 7fbee240e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557d877a1520 0x557d876e3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:12.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.300+0000 7fbee240e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557d877a1520 0x557d876e3cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54252/0 (socket says 172.21.15.38:54252) 2024-09-21T03:40:12.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.300+0000 7fbee240e640 1 -- 172.21.15.38:0/2996610034 learned_addr learned my addr 172.21.15.38:0/2996610034 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:12.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.300+0000 7fbee140c640 1 --2- 172.21.15.38:0/2996610034 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557d8772c090 0x557d8779cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:12.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.301+0000 7fbee1c0d640 1 --2- 172.21.15.38:0/2996610034 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x557d876e5240 0x557d877b2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:12.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.301+0000 7fbee240e640 1 -- 172.21.15.38:0/2996610034 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x557d876e5240 msgr2=0x557d877b2da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:12.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.301+0000 7fbee240e640 1 --2- 172.21.15.38:0/2996610034 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x557d876e5240 0x557d877b2da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:12.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.301+0000 7fbee240e640 1 -- 172.21.15.38:0/2996610034 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557d8772c090 msgr2=0x557d8779cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:12.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.301+0000 7fbee240e640 1 --2- 172.21.15.38:0/2996610034 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557d8772c090 0x557d8779cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:12.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.301+0000 7fbee240e640 1 -- 172.21.15.38:0/2996610034 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d8770f010 con 0x557d877a1520 2024-09-21T03:40:12.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.301+0000 7fbee1c0d640 1 --2- 172.21.15.38:0/2996610034 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x557d876e5240 0x557d877b2da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:40:12.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.301+0000 7fbee240e640 1 --2- 172.21.15.38:0/2996610034 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557d877a1520 0x557d876e3cd0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7fbedc001600 tx=0x7fbedc001ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f13b9fd43e09ba82 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:12.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.301+0000 7fbed3fff640 1 -- 172.21.15.38:0/2996610034 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbedc016070 con 0x557d877a1520 2024-09-21T03:40:12.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.301+0000 7fbed3fff640 1 -- 172.21.15.38:0/2996610034 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbedc00f510 con 0x557d877a1520 2024-09-21T03:40:12.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.301+0000 7fbed3fff640 1 -- 172.21.15.38:0/2996610034 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbedc00f940 con 0x557d877a1520 2024-09-21T03:40:12.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.302+0000 7fbee8153f40 1 -- 172.21.15.38:0/2996610034 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557d877a1520 msgr2=0x557d876e3cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:12.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.302+0000 7fbee8153f40 1 --2- 172.21.15.38:0/2996610034 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557d877a1520 0x557d876e3cd0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7fbedc001600 tx=0x7fbedc001ac0 comp rx=0 tx=0).stop 2024-09-21T03:40:12.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.302+0000 7fbee8153f40 1 -- 172.21.15.38:0/2996610034 shutdown_connections 2024-09-21T03:40:12.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.302+0000 7fbee8153f40 1 --2- 172.21.15.38:0/2996610034 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557d877a1520 0x557d876e3cd0 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:12.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.302+0000 7fbee8153f40 1 --2- 172.21.15.38:0/2996610034 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557d8772c090 0x557d8779cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:12.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.302+0000 7fbee8153f40 1 --2- 172.21.15.38:0/2996610034 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x557d876e5240 0x557d877b2da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:12.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.302+0000 7fbee8153f40 1 -- 172.21.15.38:0/2996610034 >> 172.21.15.38:0/2996610034 conn(0x557d87603ac0 msgr2=0x557d877b1f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:12.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.302+0000 7fbee8153f40 1 -- 172.21.15.38:0/2996610034 shutdown_connections 2024-09-21T03:40:12.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.302+0000 7fbee8153f40 1 -- 172.21.15.38:0/2996610034 wait complete. 2024-09-21T03:40:12.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.303+0000 7fbee8153f40 1 Processor -- start 2024-09-21T03:40:12.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.303+0000 7fbee8153f40 1 -- start start 2024-09-21T03:40:12.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.304+0000 7fbee8153f40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557d876e5240 0x557d8779cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:12.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.304+0000 7fbee8153f40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x557d8772c090 0x557d877b11d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:12.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.304+0000 7fbee8153f40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557d877a1520 0x557d877b1710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:12.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.304+0000 7fbee8153f40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x557d8770f360 con 0x557d876e5240 2024-09-21T03:40:12.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.304+0000 7fbee8153f40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x557d8779bb50 con 0x557d877a1520 2024-09-21T03:40:12.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.304+0000 7fbee8153f40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x557d87848de0 con 0x557d8772c090 2024-09-21T03:40:12.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.304+0000 7fbee1c0d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557d876e5240 0x557d8779cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:12.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.304+0000 7fbee1c0d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557d876e5240 0x557d8779cf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54254/0 (socket says 172.21.15.38:54254) 2024-09-21T03:40:12.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.304+0000 7fbee1c0d640 1 -- 172.21.15.38:0/2504343061 learned_addr learned my addr 172.21.15.38:0/2504343061 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:12.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.304+0000 7fbee140c640 1 --2- 172.21.15.38:0/2504343061 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x557d8772c090 0x557d877b11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:12.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.304+0000 7fbee1c0d640 1 -- 172.21.15.38:0/2504343061 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x557d8772c090 msgr2=0x557d877b11d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:12.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.304+0000 7fbee1c0d640 1 --2- 172.21.15.38:0/2504343061 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x557d8772c090 0x557d877b11d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:12.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.304+0000 7fbee1c0d640 1 -- 172.21.15.38:0/2504343061 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557d877a1520 msgr2=0x557d877b1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:12.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.304+0000 7fbee1c0d640 1 --2- 172.21.15.38:0/2504343061 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557d877a1520 0x557d877b1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:12.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.304+0000 7fbee1c0d640 1 -- 172.21.15.38:0/2504343061 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d8779a870 con 0x557d876e5240 2024-09-21T03:40:12.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.304+0000 7fbee140c640 1 --2- 172.21.15.38:0/2504343061 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x557d8772c090 0x557d877b11d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:40:12.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.304+0000 7fbee1c0d640 1 --2- 172.21.15.38:0/2504343061 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557d876e5240 0x557d8779cf00 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7fbed4009670 tx=0x7fbed4009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:12.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.305+0000 7fbed2ffd640 1 -- 172.21.15.38:0/2504343061 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbed4008990 con 0x557d876e5240 2024-09-21T03:40:12.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.305+0000 7fbed2ffd640 1 -- 172.21.15.38:0/2504343061 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbed4008b30 con 0x557d876e5240 2024-09-21T03:40:12.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.305+0000 7fbee8153f40 1 -- 172.21.15.38:0/2504343061 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x557d8779ad70 con 0x557d876e5240 2024-09-21T03:40:12.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.305+0000 7fbed2ffd640 1 -- 172.21.15.38:0/2504343061 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbed4014410 con 0x557d876e5240 2024-09-21T03:40:12.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.305+0000 7fbee8153f40 1 -- 172.21.15.38:0/2504343061 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d8770ea50 con 0x557d876e5240 2024-09-21T03:40:12.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.306+0000 7fbed2ffd640 1 -- 172.21.15.38:0/2504343061 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fbed40145b0 con 0x557d876e5240 2024-09-21T03:40:12.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.307+0000 7fbed2ffd640 1 --2- 172.21.15.38:0/2504343061 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fbec40775a0 0x7fbec4079ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:12.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.307+0000 7fbed2ffd640 1 -- 172.21.15.38:0/2504343061 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6187+0+0 (secure 0 0 0) 0x7fbed4099920 con 0x557d876e5240 2024-09-21T03:40:12.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.307+0000 7fbee140c640 1 --2- 172.21.15.38:0/2504343061 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fbec40775a0 0x7fbec4079ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:12.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.307+0000 7fbee8153f40 1 --2- 172.21.15.38:0/2504343061 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x557d877b5070 0x557d87798a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:12.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.307+0000 7fbee8153f40 1 -- 172.21.15.38:0/2504343061 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:1 13.6 13:654134d2:::mds0_openfiles.0:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x557d8779d440 con 0x557d877b5070 2024-09-21T03:40:12.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.307+0000 7fbee240e640 1 --2- 172.21.15.38:0/2504343061 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x557d877b5070 0x557d87798a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:12.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.307+0000 7fbee140c640 1 --2- 172.21.15.38:0/2504343061 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fbec40775a0 0x7fbec4079ab0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fbed8055000 tx=0x7fbed8058040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:12.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.308+0000 7fbee240e640 1 --2- 172.21.15.38:0/2504343061 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x557d877b5070 0x557d87798a10 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:12.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.308+0000 7fbee240e640 1 -- 172.21.15.38:0/2504343061 <== osd.3 v2:172.21.15.38:6808/1286810294 1 ==== osd_op_reply(1 mds0_openfiles.0 [omap-get-header out=42b] v0'0 uv2 ondisk = 0) ==== 160+0+42 (crc 0 0 0) 0x7fbedc00f070 con 0x557d877b5070 2024-09-21T03:40:12.313 INFO:teuthology.orchestra.run.smithi038.stdout:header (42 bytes) : 2024-09-21T03:40:12.313 INFO:teuthology.orchestra.run.smithi038.stdout:00000000 13 00 00 00 63 65 70 68 20 66 73 20 76 6f 6c 75 |....ceph fs volu| 2024-09-21T03:40:12.313 INFO:teuthology.orchestra.run.smithi038.stdout:00000010 6d 65 20 76 30 31 31 01 01 0d 00 00 00 02 00 00 |me v011.........| 2024-09-21T03:40:12.313 INFO:teuthology.orchestra.run.smithi038.stdout:00000020 00 00 00 00 00 01 00 00 00 00 |..........| 2024-09-21T03:40:12.313 INFO:teuthology.orchestra.run.smithi038.stdout:0000002a 2024-09-21T03:40:12.313 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:40:12.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.308+0000 7fbee8153f40 1 -- 172.21.15.38:0/2504343061 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x557d877b5070 msgr2=0x557d87798a10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:12.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.308+0000 7fbee8153f40 1 --2- 172.21.15.38:0/2504343061 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x557d877b5070 0x557d87798a10 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:12.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.308+0000 7fbee8153f40 1 -- 172.21.15.38:0/2504343061 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fbec40775a0 msgr2=0x7fbec4079ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:12.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.308+0000 7fbee8153f40 1 --2- 172.21.15.38:0/2504343061 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fbec40775a0 0x7fbec4079ab0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fbed8055000 tx=0x7fbed8058040 comp rx=0 tx=0).stop 2024-09-21T03:40:12.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.308+0000 7fbee8153f40 1 -- 172.21.15.38:0/2504343061 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557d876e5240 msgr2=0x557d8779cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:12.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.308+0000 7fbee8153f40 1 --2- 172.21.15.38:0/2504343061 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557d876e5240 0x557d8779cf00 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7fbed4009670 tx=0x7fbed4009b30 comp rx=0 tx=0).stop 2024-09-21T03:40:12.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.308+0000 7fbee240e640 1 -- 172.21.15.38:0/2504343061 reap_dead start 2024-09-21T03:40:12.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.308+0000 7fbee8153f40 1 -- 172.21.15.38:0/2504343061 shutdown_connections 2024-09-21T03:40:12.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.308+0000 7fbee8153f40 1 -- 172.21.15.38:0/2504343061 >> 172.21.15.38:0/2504343061 conn(0x557d87603ac0 msgr2=0x557d876e42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:12.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.308+0000 7fbee8153f40 1 -- 172.21.15.38:0/2504343061 shutdown_connections 2024-09-21T03:40:12.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.308+0000 7fbee8153f40 1 -- 172.21.15.38:0/2504343061 wait complete. 2024-09-21T03:40:12.315 DEBUG:teuthology.orchestra.run.smithi038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getomapheader 500.00000000 2024-09-21T03:40:12.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.368+0000 7fb78211cf40 1 Processor -- start 2024-09-21T03:40:12.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.368+0000 7fb78211cf40 1 -- start start 2024-09-21T03:40:12.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.369+0000 7fb78211cf40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55eee85c3d00 0x55eee867c500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:12.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.369+0000 7fb78211cf40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55eee85efe30 0x55eee8699350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:12.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.369+0000 7fb78211cf40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eee867e950 0x55eee85f0b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:12.374 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.369+0000 7fb78211cf40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55eee8570430 con 0x55eee85efe30 2024-09-21T03:40:12.374 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.369+0000 7fb78211cf40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55eee857d1f0 con 0x55eee867e950 2024-09-21T03:40:12.374 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.369+0000 7fb78211cf40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55eee8511d60 con 0x55eee85c3d00 2024-09-21T03:40:12.374 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.369+0000 7fb77aab6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55eee85efe30 0x55eee8699350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:12.374 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.369+0000 7fb77aab6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55eee85efe30 0x55eee8699350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54262/0 (socket says 172.21.15.38:54262) 2024-09-21T03:40:12.374 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.369+0000 7fb77aab6640 1 -- 172.21.15.38:0/387614244 learned_addr learned my addr 172.21.15.38:0/387614244 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:12.374 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.369+0000 7fb77b2b7640 1 --2- 172.21.15.38:0/387614244 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55eee85c3d00 0x55eee867c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:12.374 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.369+0000 7fb77bab8640 1 --2- 172.21.15.38:0/387614244 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eee867e950 0x55eee85f0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:12.374 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.369+0000 7fb77aab6640 1 -- 172.21.15.38:0/387614244 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55eee85c3d00 msgr2=0x55eee867c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:12.374 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.369+0000 7fb77aab6640 1 --2- 172.21.15.38:0/387614244 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55eee85c3d00 0x55eee867c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:12.374 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.369+0000 7fb77aab6640 1 -- 172.21.15.38:0/387614244 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eee867e950 msgr2=0x55eee85f0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:12.374 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.369+0000 7fb77aab6640 1 --2- 172.21.15.38:0/387614244 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eee867e950 0x55eee85f0b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:12.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.369+0000 7fb77aab6640 1 -- 172.21.15.38:0/387614244 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eee867da00 con 0x55eee85efe30 2024-09-21T03:40:12.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.369+0000 7fb77bab8640 1 --2- 172.21.15.38:0/387614244 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eee867e950 0x55eee85f0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:40:12.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.369+0000 7fb77aab6640 1 --2- 172.21.15.38:0/387614244 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55eee85efe30 0x55eee8699350 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7fb764001d60 tx=0x7fb76400b670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34a411adac799df2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:12.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.370+0000 7fb779ab4640 1 -- 172.21.15.38:0/387614244 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb76400e830 con 0x55eee85efe30 2024-09-21T03:40:12.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.370+0000 7fb779ab4640 1 -- 172.21.15.38:0/387614244 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb764006050 con 0x55eee85efe30 2024-09-21T03:40:12.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.370+0000 7fb779ab4640 1 -- 172.21.15.38:0/387614244 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb76400ed80 con 0x55eee85efe30 2024-09-21T03:40:12.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.370+0000 7fb78211cf40 1 -- 172.21.15.38:0/387614244 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55eee85efe30 msgr2=0x55eee8699350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:12.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.370+0000 7fb78211cf40 1 --2- 172.21.15.38:0/387614244 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55eee85efe30 0x55eee8699350 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7fb764001d60 tx=0x7fb76400b670 comp rx=0 tx=0).stop 2024-09-21T03:40:12.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.370+0000 7fb78211cf40 1 -- 172.21.15.38:0/387614244 shutdown_connections 2024-09-21T03:40:12.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.370+0000 7fb78211cf40 1 --2- 172.21.15.38:0/387614244 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eee867e950 0x55eee85f0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:12.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.370+0000 7fb78211cf40 1 --2- 172.21.15.38:0/387614244 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55eee85efe30 0x55eee8699350 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:12.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.370+0000 7fb78211cf40 1 --2- 172.21.15.38:0/387614244 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55eee85c3d00 0x55eee867c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:12.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.370+0000 7fb78211cf40 1 -- 172.21.15.38:0/387614244 >> 172.21.15.38:0/387614244 conn(0x55eee84e2ac0 msgr2=0x55eee867a710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:12.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.370+0000 7fb78211cf40 1 -- 172.21.15.38:0/387614244 shutdown_connections 2024-09-21T03:40:12.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.370+0000 7fb78211cf40 1 -- 172.21.15.38:0/387614244 wait complete. 2024-09-21T03:40:12.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.371+0000 7fb78211cf40 1 Processor -- start 2024-09-21T03:40:12.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.371+0000 7fb78211cf40 1 -- start start 2024-09-21T03:40:12.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.374+0000 7fb78211cf40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55eee85c3d00 0x55eee8678310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:12.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.374+0000 7fb78211cf40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55eee85efe30 0x55eee85bf900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:12.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.374+0000 7fb78211cf40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eee867e950 0x55eee85bfe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:12.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.374+0000 7fb78211cf40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55eee85f22c0 con 0x55eee85c3d00 2024-09-21T03:40:12.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.374+0000 7fb78211cf40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55eee8699890 con 0x55eee867e950 2024-09-21T03:40:12.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.374+0000 7fb78211cf40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55eee8697480 con 0x55eee85efe30 2024-09-21T03:40:12.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.374+0000 7fb77b2b7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55eee85c3d00 0x55eee8678310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:12.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.374+0000 7fb77b2b7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55eee85c3d00 0x55eee8678310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54274/0 (socket says 172.21.15.38:54274) 2024-09-21T03:40:12.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.374+0000 7fb77b2b7640 1 -- 172.21.15.38:0/362819530 learned_addr learned my addr 172.21.15.38:0/362819530 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:12.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.374+0000 7fb77aab6640 1 --2- 172.21.15.38:0/362819530 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55eee85efe30 0x55eee85bf900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:12.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.374+0000 7fb77bab8640 1 --2- 172.21.15.38:0/362819530 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eee867e950 0x55eee85bfe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:12.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.374+0000 7fb77b2b7640 1 -- 172.21.15.38:0/362819530 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55eee85efe30 msgr2=0x55eee85bf900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:12.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.374+0000 7fb77b2b7640 1 --2- 172.21.15.38:0/362819530 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55eee85efe30 0x55eee85bf900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:12.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.374+0000 7fb77b2b7640 1 -- 172.21.15.38:0/362819530 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eee867e950 msgr2=0x55eee85bfe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:12.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.374+0000 7fb77b2b7640 1 --2- 172.21.15.38:0/362819530 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eee867e950 0x55eee85bfe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:12.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.374+0000 7fb77b2b7640 1 -- 172.21.15.38:0/362819530 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eee8678850 con 0x55eee85c3d00 2024-09-21T03:40:12.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.374+0000 7fb77b2b7640 1 --2- 172.21.15.38:0/362819530 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55eee85c3d00 0x55eee8678310 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7fb76c057040 tx=0x7fb76c0048c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:12.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.374+0000 7fb778ab2640 1 -- 172.21.15.38:0/362819530 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb76c066030 con 0x55eee85c3d00 2024-09-21T03:40:12.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.374+0000 7fb778ab2640 1 -- 172.21.15.38:0/362819530 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb76c062460 con 0x55eee85c3d00 2024-09-21T03:40:12.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.374+0000 7fb78211cf40 1 -- 172.21.15.38:0/362819530 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55eee8678f90 con 0x55eee85c3d00 2024-09-21T03:40:12.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.374+0000 7fb778ab2640 1 -- 172.21.15.38:0/362819530 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb76c0626b0 con 0x55eee85c3d00 2024-09-21T03:40:12.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.374+0000 7fb78211cf40 1 -- 172.21.15.38:0/362819530 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eee85f0340 con 0x55eee85c3d00 2024-09-21T03:40:12.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.375+0000 7fb778ab2640 1 -- 172.21.15.38:0/362819530 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb76c069030 con 0x55eee85c3d00 2024-09-21T03:40:12.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.376+0000 7fb778ab2640 1 --2- 172.21.15.38:0/362819530 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb75c077570 0x7fb75c079a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:12.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.376+0000 7fb778ab2640 1 -- 172.21.15.38:0/362819530 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6187+0+0 (secure 0 0 0) 0x7fb76c0de270 con 0x55eee85c3d00 2024-09-21T03:40:12.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.376+0000 7fb77aab6640 1 --2- 172.21.15.38:0/362819530 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb75c077570 0x7fb75c079a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:12.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.376+0000 7fb78211cf40 1 --2- 172.21.15.38:0/362819530 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x55eee86794e0 0x55eee8607280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:12.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.376+0000 7fb78211cf40 1 -- 172.21.15.38:0/362819530 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:1 13.15 13:aa448500:::500.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x55eee86077c0 con 0x55eee86794e0 2024-09-21T03:40:12.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.376+0000 7fb77bab8640 1 --2- 172.21.15.38:0/362819530 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x55eee86794e0 0x55eee8607280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:12.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.376+0000 7fb77aab6640 1 --2- 172.21.15.38:0/362819530 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb75c077570 0x7fb75c079a80 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x55eee85c04c0 tx=0x7fb764010040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:12.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.376+0000 7fb77bab8640 1 --2- 172.21.15.38:0/362819530 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x55eee86794e0 0x55eee8607280 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:12.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.377+0000 7fb77bab8640 1 -- 172.21.15.38:0/362819530 <== osd.3 v2:172.21.15.38:6808/1286810294 1 ==== osd_op_reply(1 500.00000000 [omap-get-header] v0'0 uv3 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb774010350 con 0x55eee86794e0 2024-09-21T03:40:12.381 INFO:teuthology.orchestra.run.smithi038.stdout:header (0 bytes) : 2024-09-21T03:40:12.382 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:40:12.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.377+0000 7fb78211cf40 1 -- 172.21.15.38:0/362819530 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x55eee86794e0 msgr2=0x55eee8607280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:12.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.377+0000 7fb78211cf40 1 --2- 172.21.15.38:0/362819530 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x55eee86794e0 0x55eee8607280 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:12.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.377+0000 7fb78211cf40 1 -- 172.21.15.38:0/362819530 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb75c077570 msgr2=0x7fb75c079a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:12.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.377+0000 7fb78211cf40 1 --2- 172.21.15.38:0/362819530 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb75c077570 0x7fb75c079a80 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x55eee85c04c0 tx=0x7fb764010040 comp rx=0 tx=0).stop 2024-09-21T03:40:12.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.377+0000 7fb78211cf40 1 -- 172.21.15.38:0/362819530 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55eee85c3d00 msgr2=0x55eee8678310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:12.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.377+0000 7fb78211cf40 1 --2- 172.21.15.38:0/362819530 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55eee85c3d00 0x55eee8678310 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7fb76c057040 tx=0x7fb76c0048c0 comp rx=0 tx=0).stop 2024-09-21T03:40:12.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.377+0000 7fb77bab8640 1 -- 172.21.15.38:0/362819530 reap_dead start 2024-09-21T03:40:12.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.377+0000 7fb78211cf40 1 -- 172.21.15.38:0/362819530 shutdown_connections 2024-09-21T03:40:12.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.377+0000 7fb78211cf40 1 -- 172.21.15.38:0/362819530 >> 172.21.15.38:0/362819530 conn(0x55eee84e2ac0 msgr2=0x55eee867a5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:12.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.377+0000 7fb78211cf40 1 -- 172.21.15.38:0/362819530 shutdown_connections 2024-09-21T03:40:12.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.377+0000 7fb78211cf40 1 -- 172.21.15.38:0/362819530 wait complete. 2024-09-21T03:40:12.384 DEBUG:teuthology.orchestra.run.smithi038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getomapheader mds_snaptable 2024-09-21T03:40:12.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.437+0000 7f888507bf40 1 Processor -- start 2024-09-21T03:40:12.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.437+0000 7f888507bf40 1 -- start start 2024-09-21T03:40:12.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.438+0000 7f888507bf40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562a4a7f2090 0x562a4a7f0950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:12.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.438+0000 7f888507bf40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562a4a867520 0x562a4a865c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:12.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.438+0000 7f888507bf40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562a4a866180 0x562a4a866590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:12.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.438+0000 7f888507bf40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x562a4a6f8d60 con 0x562a4a866180 2024-09-21T03:40:12.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.438+0000 7f888507bf40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x562a4a757430 con 0x562a4a867520 2024-09-21T03:40:12.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.438+0000 7f888507bf40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x562a4a87f350 con 0x562a4a7f2090 2024-09-21T03:40:12.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.439+0000 7f887f336640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562a4a866180 0x562a4a866590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:12.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.439+0000 7f887f336640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562a4a866180 0x562a4a866590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54300/0 (socket says 172.21.15.38:54300) 2024-09-21T03:40:12.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.439+0000 7f887f336640 1 -- 172.21.15.38:0/2256794587 learned_addr learned my addr 172.21.15.38:0/2256794587 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:12.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.439+0000 7f887e334640 1 --2- 172.21.15.38:0/2256794587 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562a4a867520 0x562a4a865c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:12.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.439+0000 7f887eb35640 1 --2- 172.21.15.38:0/2256794587 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562a4a7f2090 0x562a4a7f0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:12.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.439+0000 7f887f336640 1 -- 172.21.15.38:0/2256794587 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562a4a7f2090 msgr2=0x562a4a7f0950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:12.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.439+0000 7f887f336640 1 --2- 172.21.15.38:0/2256794587 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562a4a7f2090 0x562a4a7f0950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:12.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.439+0000 7f887f336640 1 -- 172.21.15.38:0/2256794587 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562a4a867520 msgr2=0x562a4a865c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:12.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.439+0000 7f887f336640 1 --2- 172.21.15.38:0/2256794587 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562a4a867520 0x562a4a865c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:12.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.439+0000 7f887f336640 1 -- 172.21.15.38:0/2256794587 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a4a7d35f0 con 0x562a4a866180 2024-09-21T03:40:12.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.439+0000 7f887f336640 1 --2- 172.21.15.38:0/2256794587 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562a4a866180 0x562a4a866590 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f8878046600 tx=0x7f8878002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed4e876d4803b20f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:12.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.439+0000 7f887d332640 1 -- 172.21.15.38:0/2256794587 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f887804d960 con 0x562a4a866180 2024-09-21T03:40:12.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.439+0000 7f887d332640 1 -- 172.21.15.38:0/2256794587 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8878056690 con 0x562a4a866180 2024-09-21T03:40:12.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.439+0000 7f887d332640 1 -- 172.21.15.38:0/2256794587 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8878056ae0 con 0x562a4a866180 2024-09-21T03:40:12.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.440+0000 7f888507bf40 1 -- 172.21.15.38:0/2256794587 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562a4a866180 msgr2=0x562a4a866590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:12.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.440+0000 7f888507bf40 1 --2- 172.21.15.38:0/2256794587 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562a4a866180 0x562a4a866590 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f8878046600 tx=0x7f8878002fb0 comp rx=0 tx=0).stop 2024-09-21T03:40:12.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.440+0000 7f888507bf40 1 -- 172.21.15.38:0/2256794587 shutdown_connections 2024-09-21T03:40:12.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.440+0000 7f888507bf40 1 --2- 172.21.15.38:0/2256794587 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562a4a866180 0x562a4a866590 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:12.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.440+0000 7f888507bf40 1 --2- 172.21.15.38:0/2256794587 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562a4a867520 0x562a4a865c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:12.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.440+0000 7f888507bf40 1 --2- 172.21.15.38:0/2256794587 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562a4a7f2090 0x562a4a7f0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:12.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.440+0000 7f888507bf40 1 -- 172.21.15.38:0/2256794587 >> 172.21.15.38:0/2256794587 conn(0x562a4a6c9ac0 msgr2=0x562a4a7aaf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:12.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.440+0000 7f888507bf40 1 -- 172.21.15.38:0/2256794587 shutdown_connections 2024-09-21T03:40:12.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.440+0000 7f888507bf40 1 -- 172.21.15.38:0/2256794587 wait complete. 2024-09-21T03:40:12.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.441+0000 7f888507bf40 1 Processor -- start 2024-09-21T03:40:12.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.441+0000 7f888507bf40 1 -- start start 2024-09-21T03:40:12.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.442+0000 7f888507bf40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562a4a7f2090 0x562a4a861f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:12.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.442+0000 7f888507bf40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562a4a866180 0x562a4a8611a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:12.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.442+0000 7f888507bf40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562a4a867520 0x562a4a864db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:12.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.442+0000 7f888507bf40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x562a4a8657b0 con 0x562a4a867520 2024-09-21T03:40:12.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.442+0000 7f888507bf40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x562a4a8789f0 con 0x562a4a866180 2024-09-21T03:40:12.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.442+0000 7f888507bf40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x562a4a878cd0 con 0x562a4a7f2090 2024-09-21T03:40:12.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.442+0000 7f887f336640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562a4a867520 0x562a4a864db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:12.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.442+0000 7f887f336640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562a4a867520 0x562a4a864db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54304/0 (socket says 172.21.15.38:54304) 2024-09-21T03:40:12.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.442+0000 7f887f336640 1 -- 172.21.15.38:0/1035277656 learned_addr learned my addr 172.21.15.38:0/1035277656 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:12.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.442+0000 7f887e334640 1 --2- 172.21.15.38:0/1035277656 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562a4a866180 0x562a4a8611a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:12.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.442+0000 7f887eb35640 1 --2- 172.21.15.38:0/1035277656 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562a4a7f2090 0x562a4a861f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:12.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.443+0000 7f887f336640 1 -- 172.21.15.38:0/1035277656 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562a4a7f2090 msgr2=0x562a4a861f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:12.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.443+0000 7f887f336640 1 --2- 172.21.15.38:0/1035277656 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562a4a7f2090 0x562a4a861f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:12.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.443+0000 7f887f336640 1 -- 172.21.15.38:0/1035277656 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562a4a866180 msgr2=0x562a4a8611a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:12.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.443+0000 7f887f336640 1 --2- 172.21.15.38:0/1035277656 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562a4a866180 0x562a4a8611a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:12.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.443+0000 7f887f336640 1 -- 172.21.15.38:0/1035277656 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a4a865d50 con 0x562a4a867520 2024-09-21T03:40:12.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.443+0000 7f887f336640 1 --2- 172.21.15.38:0/1035277656 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562a4a867520 0x562a4a864db0 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f8878004f50 tx=0x7f8878056a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:12.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.443+0000 7f8867fff640 1 -- 172.21.15.38:0/1035277656 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f887806f070 con 0x562a4a867520 2024-09-21T03:40:12.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.443+0000 7f8867fff640 1 -- 172.21.15.38:0/1035277656 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f887806a420 con 0x562a4a867520 2024-09-21T03:40:12.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.443+0000 7f888507bf40 1 -- 172.21.15.38:0/1035277656 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x562a4a90b350 con 0x562a4a867520 2024-09-21T03:40:12.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.443+0000 7f8867fff640 1 -- 172.21.15.38:0/1035277656 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f887806a870 con 0x562a4a867520 2024-09-21T03:40:12.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.443+0000 7f888507bf40 1 -- 172.21.15.38:0/1035277656 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a4a862460 con 0x562a4a867520 2024-09-21T03:40:12.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.445+0000 7f8867fff640 1 -- 172.21.15.38:0/1035277656 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f8878060070 con 0x562a4a867520 2024-09-21T03:40:12.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.445+0000 7f8867fff640 1 --2- 172.21.15.38:0/1035277656 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f886c077570 0x7f886c079a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:12.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.445+0000 7f8867fff640 1 -- 172.21.15.38:0/1035277656 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6187+0+0 (secure 0 0 0) 0x7f88780dea30 con 0x562a4a867520 2024-09-21T03:40:12.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.445+0000 7f887eb35640 1 --2- 172.21.15.38:0/1035277656 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f886c077570 0x7f886c079a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:12.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.446+0000 7f888507bf40 1 --2- 172.21.15.38:0/1035277656 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x562a4a866670 0x562a4a87d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:12.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.446+0000 7f888507bf40 1 -- 172.21.15.38:0/1035277656 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:1 13.d 13:b50e409b:::mds_snaptable:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x562a4a7edeb0 con 0x562a4a866670 2024-09-21T03:40:12.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.446+0000 7f887e334640 1 --2- 172.21.15.38:0/1035277656 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x562a4a866670 0x562a4a87d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:12.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.446+0000 7f887eb35640 1 --2- 172.21.15.38:0/1035277656 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f886c077570 0x7f886c079a80 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f8870010990 tx=0x7f8870004ce0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:12.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.446+0000 7f887e334640 1 --2- 172.21.15.38:0/1035277656 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x562a4a866670 0x562a4a87d440 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:12.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.446+0000 7f887e334640 1 -- 172.21.15.38:0/1035277656 <== osd.0 v2:172.21.15.38:6800/2381639433 1 ==== osd_op_reply(1 mds_snaptable [omap-get-header] v0'0 uv1 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f8874011640 con 0x562a4a866670 2024-09-21T03:40:12.451 INFO:teuthology.orchestra.run.smithi038.stdout:header (0 bytes) : 2024-09-21T03:40:12.451 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:40:12.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.446+0000 7f888507bf40 1 -- 172.21.15.38:0/1035277656 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x562a4a866670 msgr2=0x562a4a87d440 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:12.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.446+0000 7f888507bf40 1 --2- 172.21.15.38:0/1035277656 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x562a4a866670 0x562a4a87d440 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:12.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.446+0000 7f888507bf40 1 -- 172.21.15.38:0/1035277656 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f886c077570 msgr2=0x7f886c079a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:12.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.446+0000 7f888507bf40 1 --2- 172.21.15.38:0/1035277656 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f886c077570 0x7f886c079a80 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f8870010990 tx=0x7f8870004ce0 comp rx=0 tx=0).stop 2024-09-21T03:40:12.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.446+0000 7f888507bf40 1 -- 172.21.15.38:0/1035277656 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562a4a867520 msgr2=0x562a4a864db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:12.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.446+0000 7f888507bf40 1 --2- 172.21.15.38:0/1035277656 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562a4a867520 0x562a4a864db0 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f8878004f50 tx=0x7f8878056a50 comp rx=0 tx=0).stop 2024-09-21T03:40:12.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.446+0000 7f887f336640 1 -- 172.21.15.38:0/1035277656 reap_dead start 2024-09-21T03:40:12.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.447+0000 7f888507bf40 1 -- 172.21.15.38:0/1035277656 shutdown_connections 2024-09-21T03:40:12.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.447+0000 7f888507bf40 1 -- 172.21.15.38:0/1035277656 >> 172.21.15.38:0/1035277656 conn(0x562a4a6c9ac0 msgr2=0x562a4a7d6aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:12.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.447+0000 7f888507bf40 1 -- 172.21.15.38:0/1035277656 shutdown_connections 2024-09-21T03:40:12.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.447+0000 7f888507bf40 1 -- 172.21.15.38:0/1035277656 wait complete. 2024-09-21T03:40:12.454 DEBUG:teuthology.orchestra.run.smithi038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getomapheader mds0_inotable 2024-09-21T03:40:12.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.506+0000 7f3493979f40 1 Processor -- start 2024-09-21T03:40:12.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.507+0000 7f3493979f40 1 -- start start 2024-09-21T03:40:12.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.508+0000 7f3493979f40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e48d921d00 0x55e48d9ebca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:12.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.508+0000 7f3493979f40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e48d94de30 0x55e48d9d6430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:12.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.508+0000 7f3493979f40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e48d9dc950 0x55e48d94ee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:12.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.508+0000 7f3493979f40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55e48d8ce430 con 0x55e48d94de30 2024-09-21T03:40:12.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.508+0000 7f3493979f40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55e48d8db1f0 con 0x55e48d921d00 2024-09-21T03:40:12.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.508+0000 7f3493979f40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55e48d86fd60 con 0x55e48d9dc950 2024-09-21T03:40:12.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.508+0000 7f348cc32640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e48d94de30 0x55e48d9d6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:12.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.508+0000 7f348cc32640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e48d94de30 0x55e48d9d6430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54320/0 (socket says 172.21.15.38:54320) 2024-09-21T03:40:12.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.508+0000 7f348cc32640 1 -- 172.21.15.38:0/237353467 learned_addr learned my addr 172.21.15.38:0/237353467 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:12.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.508+0000 7f348dc34640 1 --2- 172.21.15.38:0/237353467 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e48d9dc950 0x55e48d94ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:12.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.508+0000 7f348d433640 1 --2- 172.21.15.38:0/237353467 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e48d921d00 0x55e48d9ebca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:12.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.508+0000 7f348cc32640 1 -- 172.21.15.38:0/237353467 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e48d9dc950 msgr2=0x55e48d94ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:12.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.508+0000 7f348cc32640 1 --2- 172.21.15.38:0/237353467 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e48d9dc950 0x55e48d94ee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:12.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.508+0000 7f348cc32640 1 -- 172.21.15.38:0/237353467 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e48d921d00 msgr2=0x55e48d9ebca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:12.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.508+0000 7f348cc32640 1 --2- 172.21.15.38:0/237353467 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e48d921d00 0x55e48d9ebca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:12.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.508+0000 7f348cc32640 1 -- 172.21.15.38:0/237353467 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e48d94f8e0 con 0x55e48d94de30 2024-09-21T03:40:12.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.508+0000 7f348d433640 1 --2- 172.21.15.38:0/237353467 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e48d921d00 0x55e48d9ebca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:40:12.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.508+0000 7f348cc32640 1 --2- 172.21.15.38:0/237353467 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e48d94de30 0x55e48d9d6430 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f3478004140 tx=0x7f3478002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=484cc199ad42d3a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:12.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.509+0000 7f34877fe640 1 -- 172.21.15.38:0/237353467 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3478066070 con 0x55e48d94de30 2024-09-21T03:40:12.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.509+0000 7f34877fe640 1 -- 172.21.15.38:0/237353467 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f347804d8e0 con 0x55e48d94de30 2024-09-21T03:40:12.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.509+0000 7f34877fe640 1 -- 172.21.15.38:0/237353467 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34780566b0 con 0x55e48d94de30 2024-09-21T03:40:12.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.509+0000 7f3493979f40 1 -- 172.21.15.38:0/237353467 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e48d94de30 msgr2=0x55e48d9d6430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:12.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.509+0000 7f3493979f40 1 --2- 172.21.15.38:0/237353467 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e48d94de30 0x55e48d9d6430 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f3478004140 tx=0x7f3478002fb0 comp rx=0 tx=0).stop 2024-09-21T03:40:12.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.509+0000 7f3493979f40 1 -- 172.21.15.38:0/237353467 shutdown_connections 2024-09-21T03:40:12.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.509+0000 7f3493979f40 1 --2- 172.21.15.38:0/237353467 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e48d9dc950 0x55e48d94ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:12.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.509+0000 7f3493979f40 1 --2- 172.21.15.38:0/237353467 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e48d94de30 0x55e48d9d6430 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:12.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.509+0000 7f3493979f40 1 --2- 172.21.15.38:0/237353467 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e48d921d00 0x55e48d9ebca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:12.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.509+0000 7f3493979f40 1 -- 172.21.15.38:0/237353467 >> 172.21.15.38:0/237353467 conn(0x55e48d840ac0 msgr2=0x55e48d9f37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:12.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.509+0000 7f3493979f40 1 -- 172.21.15.38:0/237353467 shutdown_connections 2024-09-21T03:40:12.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.509+0000 7f3493979f40 1 -- 172.21.15.38:0/237353467 wait complete. 2024-09-21T03:40:12.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.510+0000 7f3493979f40 1 Processor -- start 2024-09-21T03:40:12.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.510+0000 7f3493979f40 1 -- start start 2024-09-21T03:40:12.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.511+0000 7f3493979f40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e48d921d00 0x55e48d9ee320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:12.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.511+0000 7f3493979f40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e48d94de30 0x55e48d94a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:12.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.511+0000 7f3493979f40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e48d9dc950 0x55e48d94ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:12.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.511+0000 7f3493979f40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55e48d94f680 con 0x55e48d9dc950 2024-09-21T03:40:12.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.511+0000 7f3493979f40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55e48d9dbe10 con 0x55e48d94de30 2024-09-21T03:40:12.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.511+0000 7f3493979f40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55e48d9eea50 con 0x55e48d921d00 2024-09-21T03:40:12.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.512+0000 7f348dc34640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e48d9dc950 0x55e48d94ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:12.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.512+0000 7f348d433640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e48d921d00 0x55e48d9ee320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:12.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.512+0000 7f348dc34640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e48d9dc950 0x55e48d94ab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54326/0 (socket says 172.21.15.38:54326) 2024-09-21T03:40:12.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.512+0000 7f348d433640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e48d921d00 0x55e48d9ee320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.38:44604/0 (socket says 172.21.15.38:44604) 2024-09-21T03:40:12.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.512+0000 7f348dc34640 1 -- 172.21.15.38:0/2996668572 learned_addr learned my addr 172.21.15.38:0/2996668572 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:12.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.512+0000 7f348cc32640 1 --2- 172.21.15.38:0/2996668572 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e48d94de30 0x55e48d94a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:12.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.512+0000 7f348dc34640 1 -- 172.21.15.38:0/2996668572 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e48d921d00 msgr2=0x55e48d9ee320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:12.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.512+0000 7f348dc34640 1 --2- 172.21.15.38:0/2996668572 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e48d921d00 0x55e48d9ee320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:12.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.512+0000 7f348dc34640 1 -- 172.21.15.38:0/2996668572 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e48d94de30 msgr2=0x55e48d94a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:12.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.512+0000 7f348dc34640 1 --2- 172.21.15.38:0/2996668572 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e48d94de30 0x55e48d94a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:12.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.512+0000 7f348dc34640 1 -- 172.21.15.38:0/2996668572 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e48d9ee860 con 0x55e48d9dc950 2024-09-21T03:40:12.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.512+0000 7f348d433640 1 --2- 172.21.15.38:0/2996668572 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e48d921d00 0x55e48d9ee320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:40:12.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.512+0000 7f348dc34640 1 --2- 172.21.15.38:0/2996668572 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e48d9dc950 0x55e48d94ab50 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f348800ccb0 tx=0x7f348800a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:12.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.512+0000 7f34867fc640 1 -- 172.21.15.38:0/2996668572 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3488016600 con 0x55e48d9dc950 2024-09-21T03:40:12.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.512+0000 7f34867fc640 1 -- 172.21.15.38:0/2996668572 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3488006050 con 0x55e48d9dc950 2024-09-21T03:40:12.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.512+0000 7f3493979f40 1 -- 172.21.15.38:0/2996668572 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55e48d94e9c0 con 0x55e48d9dc950 2024-09-21T03:40:12.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.512+0000 7f34867fc640 1 -- 172.21.15.38:0/2996668572 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3488016a60 con 0x55e48d9dc950 2024-09-21T03:40:12.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.512+0000 7f3493979f40 1 -- 172.21.15.38:0/2996668572 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e48d9f28d0 con 0x55e48d9dc950 2024-09-21T03:40:12.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.514+0000 7f34867fc640 1 -- 172.21.15.38:0/2996668572 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f348801f440 con 0x55e48d9dc950 2024-09-21T03:40:12.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.515+0000 7f34867fc640 1 --2- 172.21.15.38:0/2996668572 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f34700775a0 0x7f3470079ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:12.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.515+0000 7f34867fc640 1 -- 172.21.15.38:0/2996668572 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6187+0+0 (secure 0 0 0) 0x7f348809a0a0 con 0x55e48d9dc950 2024-09-21T03:40:12.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.515+0000 7f348d433640 1 --2- 172.21.15.38:0/2996668572 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f34700775a0 0x7f3470079ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:12.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.515+0000 7f3493979f40 1 --2- 172.21.15.38:0/2996668572 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x55e48d9d4340 0x55e48d9f4b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:12.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.515+0000 7f3493979f40 1 -- 172.21.15.38:0/2996668572 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:1 13.13 13:c91d4a1d:::mds0_inotable:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x55e48d9eefa0 con 0x55e48d9d4340 2024-09-21T03:40:12.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.515+0000 7f348cc32640 1 --2- 172.21.15.38:0/2996668572 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x55e48d9d4340 0x55e48d9f4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:12.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.515+0000 7f348d433640 1 --2- 172.21.15.38:0/2996668572 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f34700775a0 0x7f3470079ab0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f34800069b0 tx=0x7f3480008040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:12.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.516+0000 7f348cc32640 1 --2- 172.21.15.38:0/2996668572 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x55e48d9d4340 0x55e48d9f4b00 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:12.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.516+0000 7f348cc32640 1 -- 172.21.15.38:0/2996668572 <== osd.1 v2:172.21.15.38:6816/1491818732 1 ==== osd_op_reply(1 mds0_inotable [omap-get-header] v0'0 uv3 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f347805f070 con 0x55e48d9d4340 2024-09-21T03:40:12.521 INFO:teuthology.orchestra.run.smithi038.stdout:header (0 bytes) : 2024-09-21T03:40:12.522 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:40:12.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.516+0000 7f3493979f40 1 -- 172.21.15.38:0/2996668572 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x55e48d9d4340 msgr2=0x55e48d9f4b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:12.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.516+0000 7f3493979f40 1 --2- 172.21.15.38:0/2996668572 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x55e48d9d4340 0x55e48d9f4b00 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:12.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.516+0000 7f3493979f40 1 -- 172.21.15.38:0/2996668572 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f34700775a0 msgr2=0x7f3470079ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:12.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.516+0000 7f3493979f40 1 --2- 172.21.15.38:0/2996668572 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f34700775a0 0x7f3470079ab0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f34800069b0 tx=0x7f3480008040 comp rx=0 tx=0).stop 2024-09-21T03:40:12.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.516+0000 7f3493979f40 1 -- 172.21.15.38:0/2996668572 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e48d9dc950 msgr2=0x55e48d94ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:12.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.516+0000 7f3493979f40 1 --2- 172.21.15.38:0/2996668572 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e48d9dc950 0x55e48d94ab50 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f348800ccb0 tx=0x7f348800a660 comp rx=0 tx=0).stop 2024-09-21T03:40:12.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.516+0000 7f348dc34640 1 -- 172.21.15.38:0/2996668572 reap_dead start 2024-09-21T03:40:12.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.516+0000 7f3493979f40 1 -- 172.21.15.38:0/2996668572 shutdown_connections 2024-09-21T03:40:12.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.516+0000 7f3493979f40 1 -- 172.21.15.38:0/2996668572 >> 172.21.15.38:0/2996668572 conn(0x55e48d840ac0 msgr2=0x55e48d967bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:12.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.516+0000 7f3493979f40 1 -- 172.21.15.38:0/2996668572 shutdown_connections 2024-09-21T03:40:12.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.516+0000 7f3493979f40 1 -- 172.21.15.38:0/2996668572 wait complete. 2024-09-21T03:40:12.524 DEBUG:teuthology.orchestra.run.smithi038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getomapheader 100.00000000 2024-09-21T03:40:12.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.576+0000 7f3d0f722f40 1 Processor -- start 2024-09-21T03:40:12.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.576+0000 7f3d0f722f40 1 -- start start 2024-09-21T03:40:12.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.577+0000 7f3d0f722f40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555dfd9f1d00 0x555dfdac3970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:12.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.577+0000 7f3d0f722f40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555dfda1de30 0x555dfdaabb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:12.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.577+0000 7f3d0f722f40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555dfdaac950 0x555dfda1f500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:12.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.578+0000 7f3d0f722f40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x555dfd99e430 con 0x555dfdaac950 2024-09-21T03:40:12.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.578+0000 7f3d0f722f40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x555dfd9ab1f0 con 0x555dfd9f1d00 2024-09-21T03:40:12.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.578+0000 7f3d0f722f40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x555dfd93fd60 con 0x555dfda1de30 2024-09-21T03:40:12.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.578+0000 7f3d099df640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555dfdaac950 0x555dfda1f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:12.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.578+0000 7f3d099df640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555dfdaac950 0x555dfda1f500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54338/0 (socket says 172.21.15.38:54338) 2024-09-21T03:40:12.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.578+0000 7f3d099df640 1 -- 172.21.15.38:0/219466822 learned_addr learned my addr 172.21.15.38:0/219466822 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:12.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.578+0000 7f3d089dd640 1 --2- 172.21.15.38:0/219466822 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555dfda1de30 0x555dfdaabb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:12.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.578+0000 7f3d091de640 1 --2- 172.21.15.38:0/219466822 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555dfd9f1d00 0x555dfdac3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:12.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.578+0000 7f3d099df640 1 -- 172.21.15.38:0/219466822 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555dfda1de30 msgr2=0x555dfdaabb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:12.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.578+0000 7f3d099df640 1 --2- 172.21.15.38:0/219466822 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555dfda1de30 0x555dfdaabb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:12.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.578+0000 7f3d099df640 1 -- 172.21.15.38:0/219466822 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555dfd9f1d00 msgr2=0x555dfdac3970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:12.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.578+0000 7f3d099df640 1 --2- 172.21.15.38:0/219466822 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555dfd9f1d00 0x555dfdac3970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:12.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.578+0000 7f3d099df640 1 -- 172.21.15.38:0/219466822 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555dfdaa7d30 con 0x555dfdaac950 2024-09-21T03:40:12.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.578+0000 7f3d089dd640 1 --2- 172.21.15.38:0/219466822 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555dfda1de30 0x555dfdaabb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:40:12.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.578+0000 7f3d099df640 1 --2- 172.21.15.38:0/219466822 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555dfdaac950 0x555dfda1f500 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f3d04001560 tx=0x7f3d04001a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85c55047a3201fed server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:12.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.578+0000 7f3cfb7fe640 1 -- 172.21.15.38:0/219466822 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d04017070 con 0x555dfdaac950 2024-09-21T03:40:12.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.578+0000 7f3cfb7fe640 1 -- 172.21.15.38:0/219466822 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3d040027a0 con 0x555dfdaac950 2024-09-21T03:40:12.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.578+0000 7f3cfb7fe640 1 -- 172.21.15.38:0/219466822 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d04002bf0 con 0x555dfdaac950 2024-09-21T03:40:12.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.579+0000 7f3d0f722f40 1 -- 172.21.15.38:0/219466822 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555dfdaac950 msgr2=0x555dfda1f500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:12.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.579+0000 7f3d0f722f40 1 --2- 172.21.15.38:0/219466822 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555dfdaac950 0x555dfda1f500 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f3d04001560 tx=0x7f3d04001a20 comp rx=0 tx=0).stop 2024-09-21T03:40:12.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.579+0000 7f3d0f722f40 1 -- 172.21.15.38:0/219466822 shutdown_connections 2024-09-21T03:40:12.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.579+0000 7f3d0f722f40 1 --2- 172.21.15.38:0/219466822 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555dfdaac950 0x555dfda1f500 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:12.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.579+0000 7f3d0f722f40 1 --2- 172.21.15.38:0/219466822 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555dfda1de30 0x555dfdaabb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:12.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.579+0000 7f3d0f722f40 1 --2- 172.21.15.38:0/219466822 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555dfd9f1d00 0x555dfdac3970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:12.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.579+0000 7f3d0f722f40 1 -- 172.21.15.38:0/219466822 >> 172.21.15.38:0/219466822 conn(0x555dfd910ac0 msgr2=0x555dfdaa8680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:12.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.579+0000 7f3d0f722f40 1 -- 172.21.15.38:0/219466822 shutdown_connections 2024-09-21T03:40:12.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.579+0000 7f3d0f722f40 1 -- 172.21.15.38:0/219466822 wait complete. 2024-09-21T03:40:12.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.580+0000 7f3d0f722f40 1 Processor -- start 2024-09-21T03:40:12.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.580+0000 7f3d0f722f40 1 -- start start 2024-09-21T03:40:12.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.581+0000 7f3d0f722f40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555dfd9f1d00 0x555dfdabc5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:12.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.581+0000 7f3d0f722f40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555dfda1de30 0x555dfdac40f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:12.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.581+0000 7f3d0f722f40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555dfdaac950 0x555dfdac4630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:12.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.581+0000 7f3d0f722f40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x555dfdac75b0 con 0x555dfd9f1d00 2024-09-21T03:40:12.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.581+0000 7f3d0f722f40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x555dfdac7c10 con 0x555dfda1de30 2024-09-21T03:40:12.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.581+0000 7f3d0f722f40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x555dfdac7850 con 0x555dfdaac950 2024-09-21T03:40:12.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.581+0000 7f3d091de640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555dfd9f1d00 0x555dfdabc5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:12.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.581+0000 7f3d091de640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555dfd9f1d00 0x555dfdabc5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54354/0 (socket says 172.21.15.38:54354) 2024-09-21T03:40:12.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.581+0000 7f3d091de640 1 -- 172.21.15.38:0/413126013 learned_addr learned my addr 172.21.15.38:0/413126013 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:12.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.582+0000 7f3d089dd640 1 --2- 172.21.15.38:0/413126013 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555dfda1de30 0x555dfdac40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:12.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.582+0000 7f3d099df640 1 --2- 172.21.15.38:0/413126013 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555dfdaac950 0x555dfdac4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:12.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.582+0000 7f3d091de640 1 -- 172.21.15.38:0/413126013 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555dfdaac950 msgr2=0x555dfdac4630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:12.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.582+0000 7f3d091de640 1 --2- 172.21.15.38:0/413126013 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555dfdaac950 0x555dfdac4630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:12.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.582+0000 7f3d091de640 1 -- 172.21.15.38:0/413126013 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555dfda1de30 msgr2=0x555dfdac40f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:12.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.582+0000 7f3d091de640 1 --2- 172.21.15.38:0/413126013 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555dfda1de30 0x555dfdac40f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:12.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.582+0000 7f3d091de640 1 -- 172.21.15.38:0/413126013 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555dfdb625d0 con 0x555dfd9f1d00 2024-09-21T03:40:12.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.582+0000 7f3d091de640 1 --2- 172.21.15.38:0/413126013 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555dfd9f1d00 0x555dfdabc5e0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f3cfc00ef60 tx=0x7f3cfc00ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:12.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.582+0000 7f3cfa7fc640 1 -- 172.21.15.38:0/413126013 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cfc015020 con 0x555dfd9f1d00 2024-09-21T03:40:12.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.582+0000 7f3cfa7fc640 1 -- 172.21.15.38:0/413126013 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3cfc011450 con 0x555dfd9f1d00 2024-09-21T03:40:12.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.582+0000 7f3d0f722f40 1 -- 172.21.15.38:0/413126013 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x555dfdac0480 con 0x555dfd9f1d00 2024-09-21T03:40:12.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.582+0000 7f3cfa7fc640 1 -- 172.21.15.38:0/413126013 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cfc011880 con 0x555dfd9f1d00 2024-09-21T03:40:12.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.582+0000 7f3d0f722f40 1 -- 172.21.15.38:0/413126013 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555dfdabcb20 con 0x555dfd9f1d00 2024-09-21T03:40:12.589 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.584+0000 7f3cfa7fc640 1 -- 172.21.15.38:0/413126013 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f3cfc018020 con 0x555dfd9f1d00 2024-09-21T03:40:12.589 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.584+0000 7f3cfa7fc640 1 --2- 172.21.15.38:0/413126013 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3cec077570 0x7f3cec079a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:12.589 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.584+0000 7f3cfa7fc640 1 -- 172.21.15.38:0/413126013 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6187+0+0 (secure 0 0 0) 0x7f3cfc099bb0 con 0x555dfd9f1d00 2024-09-21T03:40:12.589 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.585+0000 7f3d089dd640 1 --2- 172.21.15.38:0/413126013 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3cec077570 0x7f3cec079a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:12.589 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.585+0000 7f3d0f722f40 1 --2- 172.21.15.38:0/413126013 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x555dfdac0850 0x555dfdac6a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:12.589 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.585+0000 7f3d0f722f40 1 -- 172.21.15.38:0/413126013 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:1 13.13 13:cd5a64a3:::100.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x555dfda33d20 con 0x555dfdac0850 2024-09-21T03:40:12.589 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.585+0000 7f3d099df640 1 --2- 172.21.15.38:0/413126013 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x555dfdac0850 0x555dfdac6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:12.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.585+0000 7f3d089dd640 1 --2- 172.21.15.38:0/413126013 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3cec077570 0x7f3cec079a80 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f3d00055000 tx=0x7f3d00003020 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:12.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.585+0000 7f3d099df640 1 --2- 172.21.15.38:0/413126013 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x555dfdac0850 0x555dfdac6a40 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:12.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.585+0000 7f3d099df640 1 -- 172.21.15.38:0/413126013 <== osd.1 v2:172.21.15.38:6816/1491818732 1 ==== osd_op_reply(1 100.00000000 [omap-get-header out=274b] v0'0 uv1 ondisk = 0) ==== 156+0+274 (crc 0 0 0) 0x7f3d0400f070 con 0x555dfdac0850 2024-09-21T03:40:12.590 INFO:teuthology.orchestra.run.smithi038.stdout:header (274 bytes) : 2024-09-21T03:40:12.590 INFO:teuthology.orchestra.run.smithi038.stdout:00000000 04 03 0c 01 00 00 01 00 00 00 00 00 00 00 00 00 |................| 2024-09-21T03:40:12.590 INFO:teuthology.orchestra.run.smithi038.stdout:00000010 00 00 00 00 00 00 03 02 28 00 00 00 00 00 00 00 |........(.......| 2024-09-21T03:40:12.590 INFO:teuthology.orchestra.run.smithi038.stdout:00000020 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 2024-09-21T03:40:12.590 INFO:teuthology.orchestra.run.smithi038.stdout:00000030 00 00 00 00 0a 00 00 00 00 00 00 00 00 00 00 00 |................| 2024-09-21T03:40:12.590 INFO:teuthology.orchestra.run.smithi038.stdout:00000040 00 00 00 00 03 02 28 00 00 00 00 00 00 00 00 00 |......(.........| 2024-09-21T03:40:12.590 INFO:teuthology.orchestra.run.smithi038.stdout:00000050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 2024-09-21T03:40:12.591 INFO:teuthology.orchestra.run.smithi038.stdout:00000060 00 00 0a 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 2024-09-21T03:40:12.591 INFO:teuthology.orchestra.run.smithi038.stdout:00000070 00 00 03 02 38 00 00 00 00 00 00 00 00 00 00 00 |....8...........| 2024-09-21T03:40:12.591 INFO:teuthology.orchestra.run.smithi038.stdout:00000080 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 2024-09-21T03:40:12.591 INFO:teuthology.orchestra.run.smithi038.stdout:00000090 0a 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 2024-09-21T03:40:12.591 INFO:teuthology.orchestra.run.smithi038.stdout:000000a0 00 00 00 00 00 00 00 00 12 40 ee 66 2d d5 20 35 |.........@.f-. 5| 2024-09-21T03:40:12.591 INFO:teuthology.orchestra.run.smithi038.stdout:000000b0 03 02 38 00 00 00 00 00 00 00 00 00 00 00 00 00 |..8.............| 2024-09-21T03:40:12.591 INFO:teuthology.orchestra.run.smithi038.stdout:000000c0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0a 00 |................| 2024-09-21T03:40:12.591 INFO:teuthology.orchestra.run.smithi038.stdout:000000d0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 2024-09-21T03:40:12.591 INFO:teuthology.orchestra.run.smithi038.stdout:000000e0 00 00 00 00 00 00 12 40 ee 66 2d d5 20 35 00 00 |.......@.f-. 5..| 2024-09-21T03:40:12.591 INFO:teuthology.orchestra.run.smithi038.stdout:000000f0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 2024-09-21T03:40:12.591 INFO:teuthology.orchestra.run.smithi038.stdout:* 2024-09-21T03:40:12.591 INFO:teuthology.orchestra.run.smithi038.stdout:00000110 00 00 |..| 2024-09-21T03:40:12.591 INFO:teuthology.orchestra.run.smithi038.stdout:00000112 2024-09-21T03:40:12.591 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:40:12.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.586+0000 7f3d0f722f40 1 -- 172.21.15.38:0/413126013 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x555dfdac0850 msgr2=0x555dfdac6a40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:12.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.586+0000 7f3d0f722f40 1 --2- 172.21.15.38:0/413126013 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x555dfdac0850 0x555dfdac6a40 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:12.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.586+0000 7f3d0f722f40 1 -- 172.21.15.38:0/413126013 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3cec077570 msgr2=0x7f3cec079a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:12.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.586+0000 7f3d0f722f40 1 --2- 172.21.15.38:0/413126013 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3cec077570 0x7f3cec079a80 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f3d00055000 tx=0x7f3d00003020 comp rx=0 tx=0).stop 2024-09-21T03:40:12.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.586+0000 7f3d0f722f40 1 -- 172.21.15.38:0/413126013 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555dfd9f1d00 msgr2=0x555dfdabc5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:12.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.586+0000 7f3d0f722f40 1 --2- 172.21.15.38:0/413126013 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555dfd9f1d00 0x555dfdabc5e0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f3cfc00ef60 tx=0x7f3cfc00ef90 comp rx=0 tx=0).stop 2024-09-21T03:40:12.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.586+0000 7f3d099df640 1 -- 172.21.15.38:0/413126013 reap_dead start 2024-09-21T03:40:12.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.586+0000 7f3d0f722f40 1 -- 172.21.15.38:0/413126013 shutdown_connections 2024-09-21T03:40:12.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.586+0000 7f3d0f722f40 1 -- 172.21.15.38:0/413126013 >> 172.21.15.38:0/413126013 conn(0x555dfd910ac0 msgr2=0x555dfdaa8560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:12.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.586+0000 7f3d0f722f40 1 -- 172.21.15.38:0/413126013 shutdown_connections 2024-09-21T03:40:12.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.586+0000 7f3d0f722f40 1 -- 172.21.15.38:0/413126013 wait complete. 2024-09-21T03:40:12.594 DEBUG:teuthology.orchestra.run.smithi038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getomapheader mds0_sessionmap 2024-09-21T03:40:12.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.647+0000 7fe94c979f40 1 Processor -- start 2024-09-21T03:40:12.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.647+0000 7fe94c979f40 1 -- start start 2024-09-21T03:40:12.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.648+0000 7fe94c979f40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562e122e2d00 0x562e12397110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:12.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.648+0000 7fe94c979f40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562e1230ee30 0x562e12310620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:12.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.648+0000 7fe94c979f40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562e1239d950 0x562e12310b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:12.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.648+0000 7fe94c979f40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x562e12230d60 con 0x562e1239d950 2024-09-21T03:40:12.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.648+0000 7fe94c979f40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x562e1228f430 con 0x562e122e2d00 2024-09-21T03:40:12.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.648+0000 7fe94c979f40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x562e123291b0 con 0x562e1230ee30 2024-09-21T03:40:12.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.648+0000 7fe946c34640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562e1239d950 0x562e12310b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:12.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.648+0000 7fe946c34640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562e1239d950 0x562e12310b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54372/0 (socket says 172.21.15.38:54372) 2024-09-21T03:40:12.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.648+0000 7fe946c34640 1 -- 172.21.15.38:0/733413211 learned_addr learned my addr 172.21.15.38:0/733413211 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:12.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.648+0000 7fe946433640 1 --2- 172.21.15.38:0/733413211 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562e122e2d00 0x562e12397110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:12.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.648+0000 7fe945c32640 1 --2- 172.21.15.38:0/733413211 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562e1230ee30 0x562e12310620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:12.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.648+0000 7fe946c34640 1 -- 172.21.15.38:0/733413211 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562e1230ee30 msgr2=0x562e12310620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:12.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.648+0000 7fe946c34640 1 --2- 172.21.15.38:0/733413211 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562e1230ee30 0x562e12310620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:12.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.648+0000 7fe946c34640 1 -- 172.21.15.38:0/733413211 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562e122e2d00 msgr2=0x562e12397110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:12.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.648+0000 7fe946c34640 1 --2- 172.21.15.38:0/733413211 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562e122e2d00 0x562e12397110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:12.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.648+0000 7fe946c34640 1 -- 172.21.15.38:0/733413211 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e1239cd10 con 0x562e1239d950 2024-09-21T03:40:12.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.650+0000 7fe946c34640 1 --2- 172.21.15.38:0/733413211 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562e1239d950 0x562e12310b60 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7fe940025e50 tx=0x7fe940003330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d18e78c37afe5a74 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:12.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.650+0000 7fe944c30640 1 -- 172.21.15.38:0/733413211 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe940064030 con 0x562e1239d950 2024-09-21T03:40:12.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.650+0000 7fe944c30640 1 -- 172.21.15.38:0/733413211 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe940004020 con 0x562e1239d950 2024-09-21T03:40:12.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.650+0000 7fe944c30640 1 -- 172.21.15.38:0/733413211 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe940004270 con 0x562e1239d950 2024-09-21T03:40:12.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.651+0000 7fe94c979f40 1 -- 172.21.15.38:0/733413211 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562e1239d950 msgr2=0x562e12310b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:12.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.651+0000 7fe94c979f40 1 --2- 172.21.15.38:0/733413211 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562e1239d950 0x562e12310b60 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7fe940025e50 tx=0x7fe940003330 comp rx=0 tx=0).stop 2024-09-21T03:40:12.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.651+0000 7fe94c979f40 1 -- 172.21.15.38:0/733413211 shutdown_connections 2024-09-21T03:40:12.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.651+0000 7fe94c979f40 1 --2- 172.21.15.38:0/733413211 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562e1239d950 0x562e12310b60 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:12.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.651+0000 7fe94c979f40 1 --2- 172.21.15.38:0/733413211 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562e1230ee30 0x562e12310620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:12.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.651+0000 7fe94c979f40 1 --2- 172.21.15.38:0/733413211 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562e122e2d00 0x562e12397110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:12.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.651+0000 7fe94c979f40 1 -- 172.21.15.38:0/733413211 >> 172.21.15.38:0/733413211 conn(0x562e12201ac0 msgr2=0x562e123b47c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:12.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.651+0000 7fe94c979f40 1 -- 172.21.15.38:0/733413211 shutdown_connections 2024-09-21T03:40:12.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.651+0000 7fe94c979f40 1 -- 172.21.15.38:0/733413211 wait complete. 2024-09-21T03:40:12.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.652+0000 7fe94c979f40 1 Processor -- start 2024-09-21T03:40:12.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.652+0000 7fe94c979f40 1 -- start start 2024-09-21T03:40:12.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.653+0000 7fe94c979f40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562e122e2d00 0x562e123b50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:12.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.653+0000 7fe94c979f40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562e1230ee30 0x562e123af5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:12.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.653+0000 7fe94c979f40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562e1239d950 0x562e123b4390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:12.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.653+0000 7fe94c979f40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x562e1230f510 con 0x562e1239d950 2024-09-21T03:40:12.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.653+0000 7fe94c979f40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x562e1230f7b0 con 0x562e122e2d00 2024-09-21T03:40:12.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.653+0000 7fe94c979f40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x562e122e0cf0 con 0x562e1230ee30 2024-09-21T03:40:12.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.653+0000 7fe946c34640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562e1239d950 0x562e123b4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:12.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.653+0000 7fe946c34640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562e1239d950 0x562e123b4390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54374/0 (socket says 172.21.15.38:54374) 2024-09-21T03:40:12.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.653+0000 7fe946c34640 1 -- 172.21.15.38:0/3808113690 learned_addr learned my addr 172.21.15.38:0/3808113690 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:12.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.653+0000 7fe946433640 1 --2- 172.21.15.38:0/3808113690 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562e122e2d00 0x562e123b50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:12.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.653+0000 7fe945c32640 1 --2- 172.21.15.38:0/3808113690 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562e1230ee30 0x562e123af5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:12.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.654+0000 7fe946433640 1 -- 172.21.15.38:0/3808113690 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562e1230ee30 msgr2=0x562e123af5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:12.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.654+0000 7fe946433640 1 --2- 172.21.15.38:0/3808113690 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562e1230ee30 0x562e123af5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:12.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.654+0000 7fe946433640 1 -- 172.21.15.38:0/3808113690 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562e1239d950 msgr2=0x562e123b4390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:12.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.654+0000 7fe946433640 1 --2- 172.21.15.38:0/3808113690 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562e1239d950 0x562e123b4390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:12.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.654+0000 7fe946433640 1 -- 172.21.15.38:0/3808113690 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e123b8f70 con 0x562e122e2d00 2024-09-21T03:40:12.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.654+0000 7fe946c34640 1 --2- 172.21.15.38:0/3808113690 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562e1239d950 0x562e123b4390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:40:12.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.654+0000 7fe946433640 1 --2- 172.21.15.38:0/3808113690 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562e122e2d00 0x562e123b50f0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fe93800a710 tx=0x7fe93800abd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:12.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.654+0000 7fe9377fe640 1 -- 172.21.15.38:0/3808113690 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe938009b40 con 0x562e122e2d00 2024-09-21T03:40:12.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.654+0000 7fe9377fe640 1 -- 172.21.15.38:0/3808113690 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe938009ce0 con 0x562e122e2d00 2024-09-21T03:40:12.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.654+0000 7fe94c979f40 1 -- 172.21.15.38:0/3808113690 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x562e123959b0 con 0x562e122e2d00 2024-09-21T03:40:12.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.654+0000 7fe9377fe640 1 -- 172.21.15.38:0/3808113690 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe938017670 con 0x562e122e2d00 2024-09-21T03:40:12.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.654+0000 7fe94c979f40 1 -- 172.21.15.38:0/3808113690 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e123afe40 con 0x562e122e2d00 2024-09-21T03:40:12.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.656+0000 7fe9377fe640 1 -- 172.21.15.38:0/3808113690 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fe938017810 con 0x562e122e2d00 2024-09-21T03:40:12.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.656+0000 7fe9377fe640 1 --2- 172.21.15.38:0/3808113690 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fe928077570 0x7fe928079a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:12.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.656+0000 7fe9377fe640 1 -- 172.21.15.38:0/3808113690 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6187+0+0 (secure 0 0 0) 0x7fe93809a700 con 0x562e122e2d00 2024-09-21T03:40:12.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.656+0000 7fe945c32640 1 --2- 172.21.15.38:0/3808113690 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fe928077570 0x7fe928079a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:12.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.656+0000 7fe94c979f40 1 --2- 172.21.15.38:0/3808113690 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x562e123aed40 0x562e12452f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:12.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.656+0000 7fe94c979f40 1 -- 172.21.15.38:0/3808113690 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:1 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x562e12396450 con 0x562e123aed40 2024-09-21T03:40:12.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.656+0000 7fe946c34640 1 --2- 172.21.15.38:0/3808113690 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x562e123aed40 0x562e12452f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:12.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.656+0000 7fe945c32640 1 --2- 172.21.15.38:0/3808113690 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fe928077570 0x7fe928079a80 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x562e12328120 tx=0x7fe93c001730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:12.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.656+0000 7fe946c34640 1 --2- 172.21.15.38:0/3808113690 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x562e123aed40 0x562e12452f90 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:12.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.658+0000 7fe946c34640 1 -- 172.21.15.38:0/3808113690 <== osd.3 v2:172.21.15.38:6808/1286810294 1 ==== osd_op_reply(1 mds0_sessionmap [omap-get-header out=14b] v0'0 uv3 ondisk = 0) ==== 159+0+14 (crc 0 0 0) 0x7fe94000b040 con 0x562e123aed40 2024-09-21T03:40:12.660 INFO:teuthology.orchestra.run.smithi038.stdout:header (14 bytes) : 2024-09-21T03:40:12.661 INFO:teuthology.orchestra.run.smithi038.stdout:00000000 01 01 08 00 00 00 04 00 00 00 00 00 00 00 |..............| 2024-09-21T03:40:12.661 INFO:teuthology.orchestra.run.smithi038.stdout:0000000e 2024-09-21T03:40:12.661 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:40:12.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.658+0000 7fe94c979f40 1 -- 172.21.15.38:0/3808113690 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x562e123aed40 msgr2=0x562e12452f90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:12.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.658+0000 7fe94c979f40 1 --2- 172.21.15.38:0/3808113690 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x562e123aed40 0x562e12452f90 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:12.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.658+0000 7fe94c979f40 1 -- 172.21.15.38:0/3808113690 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fe928077570 msgr2=0x7fe928079a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:12.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.658+0000 7fe94c979f40 1 --2- 172.21.15.38:0/3808113690 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fe928077570 0x7fe928079a80 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x562e12328120 tx=0x7fe93c001730 comp rx=0 tx=0).stop 2024-09-21T03:40:12.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.658+0000 7fe94c979f40 1 -- 172.21.15.38:0/3808113690 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562e122e2d00 msgr2=0x562e123b50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:12.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.658+0000 7fe94c979f40 1 --2- 172.21.15.38:0/3808113690 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562e122e2d00 0x562e123b50f0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fe93800a710 tx=0x7fe93800abd0 comp rx=0 tx=0).stop 2024-09-21T03:40:12.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.658+0000 7fe946c34640 1 -- 172.21.15.38:0/3808113690 reap_dead start 2024-09-21T03:40:12.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.658+0000 7fe94c979f40 1 -- 172.21.15.38:0/3808113690 shutdown_connections 2024-09-21T03:40:12.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.658+0000 7fe94c979f40 1 -- 172.21.15.38:0/3808113690 >> 172.21.15.38:0/3808113690 conn(0x562e12201ac0 msgr2=0x562e1239afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:12.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.658+0000 7fe94c979f40 1 -- 172.21.15.38:0/3808113690 shutdown_connections 2024-09-21T03:40:12.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.658+0000 7fe94c979f40 1 -- 172.21.15.38:0/3808113690 wait complete. 2024-09-21T03:40:12.663 DEBUG:teuthology.orchestra.run.smithi038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getomapheader 400.00000000 2024-09-21T03:40:12.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.716+0000 7fcb9a68cf40 1 Processor -- start 2024-09-21T03:40:12.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.717+0000 7fcb9a68cf40 1 -- start start 2024-09-21T03:40:12.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.718+0000 7fcb9a68cf40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559359e7bd00 0x559359f4da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:12.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.718+0000 7fcb9a68cf40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x559359f36950 0x559359f4d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:12.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.718+0000 7fcb9a68cf40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559359f51670 0x559359e77900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:12.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.718+0000 7fcb9a68cf40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x559359dc9d60 con 0x559359e7bd00 2024-09-21T03:40:12.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.718+0000 7fcb9a68cf40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x559359e28430 con 0x559359f51670 2024-09-21T03:40:12.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.718+0000 7fcb9a68cf40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x559359ec37b0 con 0x559359f36950 2024-09-21T03:40:12.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.718+0000 7fcb94146640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559359e7bd00 0x559359f4da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:12.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.718+0000 7fcb94146640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559359e7bd00 0x559359f4da60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54390/0 (socket says 172.21.15.38:54390) 2024-09-21T03:40:12.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.718+0000 7fcb94146640 1 -- 172.21.15.38:0/2430359147 learned_addr learned my addr 172.21.15.38:0/2430359147 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:12.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.718+0000 7fcb94947640 1 --2- 172.21.15.38:0/2430359147 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559359f51670 0x559359e77900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:12.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.718+0000 7fcb94146640 1 -- 172.21.15.38:0/2430359147 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x559359f36950 msgr2=0x559359f4d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:12.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.718+0000 7fcb8bfff640 1 --2- 172.21.15.38:0/2430359147 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x559359f36950 0x559359f4d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:12.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.718+0000 7fcb94146640 1 --2- 172.21.15.38:0/2430359147 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x559359f36950 0x559359f4d3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:12.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.718+0000 7fcb94146640 1 -- 172.21.15.38:0/2430359147 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559359f51670 msgr2=0x559359e77900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:12.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.718+0000 7fcb94146640 1 --2- 172.21.15.38:0/2430359147 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559359f51670 0x559359e77900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:12.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.718+0000 7fcb94146640 1 -- 172.21.15.38:0/2430359147 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559359ea9ca0 con 0x559359e7bd00 2024-09-21T03:40:12.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.718+0000 7fcb94947640 1 --2- 172.21.15.38:0/2430359147 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559359f51670 0x559359e77900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:40:12.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.718+0000 7fcb94146640 1 --2- 172.21.15.38:0/2430359147 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559359e7bd00 0x559359f4da60 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7fcb84004e90 tx=0x7fcb84002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1daec667840cb084 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:12.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.719+0000 7fcb93144640 1 -- 172.21.15.38:0/2430359147 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb84066070 con 0x559359e7bd00 2024-09-21T03:40:12.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.719+0000 7fcb93144640 1 -- 172.21.15.38:0/2430359147 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcb8404de40 con 0x559359e7bd00 2024-09-21T03:40:12.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.719+0000 7fcb93144640 1 -- 172.21.15.38:0/2430359147 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb840566b0 con 0x559359e7bd00 2024-09-21T03:40:12.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.719+0000 7fcb9a68cf40 1 -- 172.21.15.38:0/2430359147 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559359e7bd00 msgr2=0x559359f4da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:12.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.719+0000 7fcb9a68cf40 1 --2- 172.21.15.38:0/2430359147 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559359e7bd00 0x559359f4da60 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7fcb84004e90 tx=0x7fcb84002fb0 comp rx=0 tx=0).stop 2024-09-21T03:40:12.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.719+0000 7fcb9a68cf40 1 -- 172.21.15.38:0/2430359147 shutdown_connections 2024-09-21T03:40:12.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.719+0000 7fcb9a68cf40 1 --2- 172.21.15.38:0/2430359147 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559359f51670 0x559359e77900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:12.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.719+0000 7fcb9a68cf40 1 --2- 172.21.15.38:0/2430359147 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x559359f36950 0x559359f4d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:12.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.719+0000 7fcb9a68cf40 1 --2- 172.21.15.38:0/2430359147 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559359e7bd00 0x559359f4da60 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:12.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.719+0000 7fcb9a68cf40 1 -- 172.21.15.38:0/2430359147 >> 172.21.15.38:0/2430359147 conn(0x559359d9aac0 msgr2=0x559359f32710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:12.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.719+0000 7fcb9a68cf40 1 -- 172.21.15.38:0/2430359147 shutdown_connections 2024-09-21T03:40:12.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.719+0000 7fcb9a68cf40 1 -- 172.21.15.38:0/2430359147 wait complete. 2024-09-21T03:40:12.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.720+0000 7fcb9a68cf40 1 Processor -- start 2024-09-21T03:40:12.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.720+0000 7fcb9a68cf40 1 -- start start 2024-09-21T03:40:12.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.722+0000 7fcb9a68cf40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559359e7bd00 0x559359f51c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:12.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.722+0000 7fcb9a68cf40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559359f36950 0x559359f50630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:12.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.722+0000 7fcb9a68cf40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x559359f51670 0x559359f50b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:12.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.722+0000 7fcb9a68cf40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x559359ea9f40 con 0x559359e7bd00 2024-09-21T03:40:12.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.722+0000 7fcb9a68cf40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x559359eaa1e0 con 0x559359f36950 2024-09-21T03:40:12.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.722+0000 7fcb9a68cf40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x559359ea8000 con 0x559359f51670 2024-09-21T03:40:12.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.722+0000 7fcb94146640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559359e7bd00 0x559359f51c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:12.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.722+0000 7fcb94146640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559359e7bd00 0x559359f51c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54396/0 (socket says 172.21.15.38:54396) 2024-09-21T03:40:12.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.722+0000 7fcb94146640 1 -- 172.21.15.38:0/2210499123 learned_addr learned my addr 172.21.15.38:0/2210499123 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:12.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.722+0000 7fcb8bfff640 1 --2- 172.21.15.38:0/2210499123 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559359f36950 0x559359f50630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:12.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.722+0000 7fcb94947640 1 --2- 172.21.15.38:0/2210499123 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x559359f51670 0x559359f50b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:12.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.722+0000 7fcb94146640 1 -- 172.21.15.38:0/2210499123 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x559359f51670 msgr2=0x559359f50b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:12.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.722+0000 7fcb94146640 1 --2- 172.21.15.38:0/2210499123 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x559359f51670 0x559359f50b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:12.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.722+0000 7fcb94146640 1 -- 172.21.15.38:0/2210499123 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559359f36950 msgr2=0x559359f50630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:12.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.722+0000 7fcb94146640 1 --2- 172.21.15.38:0/2210499123 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559359f36950 0x559359f50630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:12.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.722+0000 7fcb94146640 1 -- 172.21.15.38:0/2210499123 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559359f32790 con 0x559359e7bd00 2024-09-21T03:40:12.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.722+0000 7fcb94146640 1 --2- 172.21.15.38:0/2210499123 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559359e7bd00 0x559359f51c70 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7fcb84025e60 tx=0x7fcb84056b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:12.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.722+0000 7fcb92142640 1 -- 172.21.15.38:0/2210499123 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb8406f030 con 0x559359e7bd00 2024-09-21T03:40:12.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.722+0000 7fcb92142640 1 -- 172.21.15.38:0/2210499123 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcb8406b480 con 0x559359e7bd00 2024-09-21T03:40:12.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.722+0000 7fcb9a68cf40 1 -- 172.21.15.38:0/2210499123 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x559359f34770 con 0x559359e7bd00 2024-09-21T03:40:12.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.722+0000 7fcb92142640 1 -- 172.21.15.38:0/2210499123 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb84066040 con 0x559359e7bd00 2024-09-21T03:40:12.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.723+0000 7fcb9a68cf40 1 -- 172.21.15.38:0/2210499123 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559359f4ae80 con 0x559359e7bd00 2024-09-21T03:40:12.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.724+0000 7fcb92142640 1 -- 172.21.15.38:0/2210499123 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fcb84072070 con 0x559359e7bd00 2024-09-21T03:40:12.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.724+0000 7fcb92142640 1 --2- 172.21.15.38:0/2210499123 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fcb740775a0 0x7fcb74079ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:12.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.725+0000 7fcb92142640 1 -- 172.21.15.38:0/2210499123 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6187+0+0 (secure 0 0 0) 0x7fcb840dedb0 con 0x559359e7bd00 2024-09-21T03:40:12.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.725+0000 7fcb8bfff640 1 --2- 172.21.15.38:0/2210499123 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fcb740775a0 0x7fcb74079ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:12.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.725+0000 7fcb9a68cf40 1 --2- 172.21.15.38:0/2210499123 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x559359f4cd10 0x559359fec700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:12.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.725+0000 7fcb9a68cf40 1 -- 172.21.15.38:0/2210499123 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:1 13.f 13:f1f69726:::400.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x559359f4d3a0 con 0x559359f4cd10 2024-09-21T03:40:12.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.725+0000 7fcb94947640 1 --2- 172.21.15.38:0/2210499123 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x559359f4cd10 0x559359fec700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:12.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.725+0000 7fcb8bfff640 1 --2- 172.21.15.38:0/2210499123 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fcb740775a0 0x7fcb74079ab0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x559359f4dcd0 tx=0x7fcb7c00f040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:12.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.725+0000 7fcb94947640 1 --2- 172.21.15.38:0/2210499123 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x559359f4cd10 0x559359fec700 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:12.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.726+0000 7fcb94947640 1 -- 172.21.15.38:0/2210499123 <== osd.2 v2:172.21.15.38:6824/2577858902 1 ==== osd_op_reply(1 400.00000000 [omap-get-header] v0'0 uv1 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcb8c0026a0 con 0x559359f4cd10 2024-09-21T03:40:12.729 INFO:teuthology.orchestra.run.smithi038.stdout:header (0 bytes) : 2024-09-21T03:40:12.730 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:40:12.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.726+0000 7fcb9a68cf40 1 -- 172.21.15.38:0/2210499123 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x559359f4cd10 msgr2=0x559359fec700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:12.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.726+0000 7fcb9a68cf40 1 --2- 172.21.15.38:0/2210499123 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x559359f4cd10 0x559359fec700 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:12.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.726+0000 7fcb9a68cf40 1 -- 172.21.15.38:0/2210499123 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fcb740775a0 msgr2=0x7fcb74079ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:12.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.726+0000 7fcb9a68cf40 1 --2- 172.21.15.38:0/2210499123 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fcb740775a0 0x7fcb74079ab0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x559359f4dcd0 tx=0x7fcb7c00f040 comp rx=0 tx=0).stop 2024-09-21T03:40:12.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.726+0000 7fcb9a68cf40 1 -- 172.21.15.38:0/2210499123 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559359e7bd00 msgr2=0x559359f51c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:12.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.726+0000 7fcb9a68cf40 1 --2- 172.21.15.38:0/2210499123 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559359e7bd00 0x559359f51c70 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7fcb84025e60 tx=0x7fcb84056b80 comp rx=0 tx=0).stop 2024-09-21T03:40:12.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.726+0000 7fcb94947640 1 -- 172.21.15.38:0/2210499123 reap_dead start 2024-09-21T03:40:12.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.726+0000 7fcb9a68cf40 1 -- 172.21.15.38:0/2210499123 shutdown_connections 2024-09-21T03:40:12.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.726+0000 7fcb9a68cf40 1 -- 172.21.15.38:0/2210499123 >> 172.21.15.38:0/2210499123 conn(0x559359d9aac0 msgr2=0x559359f33fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:12.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.726+0000 7fcb9a68cf40 1 -- 172.21.15.38:0/2210499123 shutdown_connections 2024-09-21T03:40:12.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.726+0000 7fcb9a68cf40 1 -- 172.21.15.38:0/2210499123 wait complete. 2024-09-21T03:40:12.732 DEBUG:teuthology.orchestra.run.smithi038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getomapheader 100.00000000.inode 2024-09-21T03:40:12.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.785+0000 7f5e4ed65f40 1 Processor -- start 2024-09-21T03:40:12.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.785+0000 7f5e4ed65f40 1 -- start start 2024-09-21T03:40:12.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.787+0000 7f5e4ed65f40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5567d6d60240 0x5567d6e2da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:12.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.787+0000 7f5e4ed65f40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5567d6da7090 0x5567d6d5b850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:12.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.787+0000 7f5e4ed65f40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5567d6e1c520 0x5567d6d88450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:12.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.787+0000 7f5e4ed65f40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5567d6cadd60 con 0x5567d6da7090 2024-09-21T03:40:12.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.787+0000 7f5e4ed65f40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5567d6d0c430 con 0x5567d6d60240 2024-09-21T03:40:12.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.787+0000 7f5e4ed65f40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x5567d6e33aa0 con 0x5567d6e1c520 2024-09-21T03:40:12.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.787+0000 7f5e4801e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5567d6da7090 0x5567d6d5b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:12.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.787+0000 7f5e4801e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5567d6da7090 0x5567d6d5b850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54402/0 (socket says 172.21.15.38:54402) 2024-09-21T03:40:12.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.787+0000 7f5e4801e640 1 -- 172.21.15.38:0/3012965294 learned_addr learned my addr 172.21.15.38:0/3012965294 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:12.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.787+0000 7f5e49020640 1 --2- 172.21.15.38:0/3012965294 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5567d6e1c520 0x5567d6d88450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:12.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.787+0000 7f5e4881f640 1 --2- 172.21.15.38:0/3012965294 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5567d6d60240 0x5567d6e2da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:12.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.787+0000 7f5e4801e640 1 -- 172.21.15.38:0/3012965294 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5567d6e1c520 msgr2=0x5567d6d88450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:12.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.787+0000 7f5e4801e640 1 --2- 172.21.15.38:0/3012965294 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5567d6e1c520 0x5567d6d88450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:12.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.787+0000 7f5e4801e640 1 -- 172.21.15.38:0/3012965294 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5567d6d60240 msgr2=0x5567d6e2da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:12.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.787+0000 7f5e4801e640 1 --2- 172.21.15.38:0/3012965294 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5567d6d60240 0x5567d6e2da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:12.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.787+0000 7f5e4801e640 1 -- 172.21.15.38:0/3012965294 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567d6e2dfc0 con 0x5567d6da7090 2024-09-21T03:40:12.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.787+0000 7f5e4801e640 1 --2- 172.21.15.38:0/3012965294 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5567d6da7090 0x5567d6d5b850 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c046680 tx=0x7f5e3c002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c26e76c0bad94ef server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:12.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.787+0000 7f5e4701c640 1 -- 172.21.15.38:0/3012965294 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e3c066070 con 0x5567d6da7090 2024-09-21T03:40:12.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.787+0000 7f5e4701c640 1 -- 172.21.15.38:0/3012965294 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5e3c056470 con 0x5567d6da7090 2024-09-21T03:40:12.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.787+0000 7f5e4701c640 1 -- 172.21.15.38:0/3012965294 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e3c0568a0 con 0x5567d6da7090 2024-09-21T03:40:12.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.788+0000 7f5e4ed65f40 1 -- 172.21.15.38:0/3012965294 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5567d6da7090 msgr2=0x5567d6d5b850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:12.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.788+0000 7f5e4ed65f40 1 --2- 172.21.15.38:0/3012965294 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5567d6da7090 0x5567d6d5b850 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c046680 tx=0x7f5e3c002fb0 comp rx=0 tx=0).stop 2024-09-21T03:40:12.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.788+0000 7f5e4ed65f40 1 -- 172.21.15.38:0/3012965294 shutdown_connections 2024-09-21T03:40:12.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.788+0000 7f5e4ed65f40 1 --2- 172.21.15.38:0/3012965294 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5567d6e1c520 0x5567d6d88450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:12.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.788+0000 7f5e4ed65f40 1 --2- 172.21.15.38:0/3012965294 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5567d6da7090 0x5567d6d5b850 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:12.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.788+0000 7f5e4ed65f40 1 --2- 172.21.15.38:0/3012965294 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5567d6d60240 0x5567d6e2da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:12.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.788+0000 7f5e4ed65f40 1 -- 172.21.15.38:0/3012965294 >> 172.21.15.38:0/3012965294 conn(0x5567d6c7eac0 msgr2=0x5567d6d89b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:12.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.788+0000 7f5e4ed65f40 1 -- 172.21.15.38:0/3012965294 shutdown_connections 2024-09-21T03:40:12.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.788+0000 7f5e4ed65f40 1 -- 172.21.15.38:0/3012965294 wait complete. 2024-09-21T03:40:12.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.789+0000 7f5e4ed65f40 1 Processor -- start 2024-09-21T03:40:12.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.789+0000 7f5e4ed65f40 1 -- start start 2024-09-21T03:40:12.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.790+0000 7f5e4ed65f40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5567d6d60240 0x5567d6d89640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:12.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.790+0000 7f5e4ed65f40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5567d6da7090 0x5567d6e2bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:12.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.790+0000 7f5e4ed65f40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5567d6e1c520 0x5567d6d889f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:12.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.790+0000 7f5e4ed65f40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5567d6e2e710 con 0x5567d6d60240 2024-09-21T03:40:12.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.790+0000 7f5e4ed65f40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5567d6e09470 con 0x5567d6e1c520 2024-09-21T03:40:12.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.790+0000 7f5e4ed65f40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x5567d6e30a80 con 0x5567d6da7090 2024-09-21T03:40:12.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.790+0000 7f5e4881f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5567d6d60240 0x5567d6d89640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:12.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.790+0000 7f5e4881f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5567d6d60240 0x5567d6d89640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54418/0 (socket says 172.21.15.38:54418) 2024-09-21T03:40:12.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.791+0000 7f5e4881f640 1 -- 172.21.15.38:0/3057296167 learned_addr learned my addr 172.21.15.38:0/3057296167 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:12.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.791+0000 7f5e4801e640 1 --2- 172.21.15.38:0/3057296167 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5567d6da7090 0x5567d6e2bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:12.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.791+0000 7f5e49020640 1 --2- 172.21.15.38:0/3057296167 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5567d6e1c520 0x5567d6d889f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:12.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.791+0000 7f5e4881f640 1 -- 172.21.15.38:0/3057296167 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5567d6da7090 msgr2=0x5567d6e2bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:12.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.791+0000 7f5e4881f640 1 --2- 172.21.15.38:0/3057296167 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5567d6da7090 0x5567d6e2bf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:12.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.791+0000 7f5e4881f640 1 -- 172.21.15.38:0/3057296167 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5567d6e1c520 msgr2=0x5567d6d889f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:12.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.791+0000 7f5e4881f640 1 --2- 172.21.15.38:0/3057296167 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5567d6e1c520 0x5567d6d889f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:12.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.791+0000 7f5e4881f640 1 -- 172.21.15.38:0/3057296167 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567d6d89f90 con 0x5567d6d60240 2024-09-21T03:40:12.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.791+0000 7f5e4881f640 1 --2- 172.21.15.38:0/3057296167 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5567d6d60240 0x5567d6d89640 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f5e3800bd60 tx=0x7f5e3800a750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:12.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.791+0000 7f5e4601a640 1 -- 172.21.15.38:0/3057296167 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e3800be50 con 0x5567d6d60240 2024-09-21T03:40:12.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.791+0000 7f5e4601a640 1 -- 172.21.15.38:0/3057296167 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5e380085e0 con 0x5567d6d60240 2024-09-21T03:40:12.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.791+0000 7f5e4ed65f40 1 -- 172.21.15.38:0/3057296167 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5567d6e176d0 con 0x5567d6d60240 2024-09-21T03:40:12.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.791+0000 7f5e4601a640 1 -- 172.21.15.38:0/3057296167 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e38008a10 con 0x5567d6d60240 2024-09-21T03:40:12.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.791+0000 7f5e4ed65f40 1 -- 172.21.15.38:0/3057296167 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5567d6e17c90 con 0x5567d6d60240 2024-09-21T03:40:12.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.793+0000 7f5e4601a640 1 -- 172.21.15.38:0/3057296167 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f5e38008bb0 con 0x5567d6d60240 2024-09-21T03:40:12.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.793+0000 7f5e4601a640 1 --2- 172.21.15.38:0/3057296167 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5e34077740 0x7f5e34079c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:12.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.793+0000 7f5e4601a640 1 -- 172.21.15.38:0/3057296167 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6187+0+0 (secure 0 0 0) 0x7f5e38099c00 con 0x5567d6d60240 2024-09-21T03:40:12.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.794+0000 7f5e4801e640 1 --2- 172.21.15.38:0/3057296167 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5e34077740 0x7f5e34079c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:12.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.794+0000 7f5e4ed65f40 1 --2- 172.21.15.38:0/3057296167 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x5567d6ece3d0 0x5567d6ece7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:12.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.794+0000 7f5e4ed65f40 1 -- 172.21.15.38:0/3057296167 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:1 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x5567d6e2c460 con 0x5567d6ece3d0 2024-09-21T03:40:12.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.794+0000 7f5e49020640 1 --2- 172.21.15.38:0/3057296167 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x5567d6ece3d0 0x5567d6ece7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:12.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.794+0000 7f5e4801e640 1 --2- 172.21.15.38:0/3057296167 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5e34077740 0x7f5e34079c50 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c046680 tx=0x7f5e3c004940 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:12.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.794+0000 7f5e49020640 1 --2- 172.21.15.38:0/3057296167 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x5567d6ece3d0 0x5567d6ece7e0 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:12.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.794+0000 7f5e49020640 1 -- 172.21.15.38:0/3057296167 <== osd.1 v2:172.21.15.38:6816/1491818732 1 ==== osd_op_reply(1 100.00000000.inode [omap-get-header] v0'0 uv3 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f5e40003040 con 0x5567d6ece3d0 2024-09-21T03:40:12.798 INFO:teuthology.orchestra.run.smithi038.stdout:header (0 bytes) : 2024-09-21T03:40:12.798 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:40:12.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.795+0000 7f5e4ed65f40 1 -- 172.21.15.38:0/3057296167 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x5567d6ece3d0 msgr2=0x5567d6ece7e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:12.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.795+0000 7f5e4ed65f40 1 --2- 172.21.15.38:0/3057296167 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x5567d6ece3d0 0x5567d6ece7e0 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:12.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.795+0000 7f5e4ed65f40 1 -- 172.21.15.38:0/3057296167 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5e34077740 msgr2=0x7f5e34079c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:12.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.795+0000 7f5e4ed65f40 1 --2- 172.21.15.38:0/3057296167 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5e34077740 0x7f5e34079c50 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c046680 tx=0x7f5e3c004940 comp rx=0 tx=0).stop 2024-09-21T03:40:12.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.795+0000 7f5e4ed65f40 1 -- 172.21.15.38:0/3057296167 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5567d6d60240 msgr2=0x5567d6d89640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:12.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.795+0000 7f5e4ed65f40 1 --2- 172.21.15.38:0/3057296167 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5567d6d60240 0x5567d6d89640 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f5e3800bd60 tx=0x7f5e3800a750 comp rx=0 tx=0).stop 2024-09-21T03:40:12.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.795+0000 7f5e49020640 1 -- 172.21.15.38:0/3057296167 reap_dead start 2024-09-21T03:40:12.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.795+0000 7f5e4ed65f40 1 -- 172.21.15.38:0/3057296167 shutdown_connections 2024-09-21T03:40:12.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.795+0000 7f5e4ed65f40 1 -- 172.21.15.38:0/3057296167 >> 172.21.15.38:0/3057296167 conn(0x5567d6c7eac0 msgr2=0x5567d6da6610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:12.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.795+0000 7f5e4ed65f40 1 -- 172.21.15.38:0/3057296167 shutdown_connections 2024-09-21T03:40:12.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.795+0000 7f5e4ed65f40 1 -- 172.21.15.38:0/3057296167 wait complete. 2024-09-21T03:40:12.801 DEBUG:teuthology.orchestra.run.smithi038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getomapheader 1.00000000 2024-09-21T03:40:12.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.855+0000 7f788afd5f40 1 Processor -- start 2024-09-21T03:40:12.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.855+0000 7f788afd5f40 1 -- start start 2024-09-21T03:40:12.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.856+0000 7f788afd5f40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a603375d00 0x55a60342a420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:12.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.856+0000 7f788afd5f40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a6033a1e30 0x55a6034dab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:12.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.856+0000 7f788afd5f40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a603430950 0x55a6034db040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:12.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.856+0000 7f788afd5f40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55a6032c3d60 con 0x55a603430950 2024-09-21T03:40:12.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.856+0000 7f788afd5f40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55a603322430 con 0x55a6033a1e30 2024-09-21T03:40:12.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.856+0000 7f788afd5f40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55a6033bd7b0 con 0x55a603375d00 2024-09-21T03:40:12.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.856+0000 7f7885290640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a603430950 0x55a6034db040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:12.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.856+0000 7f7885290640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a603430950 0x55a6034db040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54442/0 (socket says 172.21.15.38:54442) 2024-09-21T03:40:12.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.856+0000 7f7885290640 1 -- 172.21.15.38:0/4150140301 learned_addr learned my addr 172.21.15.38:0/4150140301 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:12.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.856+0000 7f7885290640 1 -- 172.21.15.38:0/4150140301 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a603375d00 msgr2=0x55a60342a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:12.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.856+0000 7f7885290640 1 --2- 172.21.15.38:0/4150140301 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a603375d00 0x55a60342a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:12.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.856+0000 7f7885290640 1 -- 172.21.15.38:0/4150140301 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a6033a1e30 msgr2=0x55a6034dab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:12.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.856+0000 7f788428e640 1 --2- 172.21.15.38:0/4150140301 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a6033a1e30 0x55a6034dab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:12.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.856+0000 7f7885290640 1 --2- 172.21.15.38:0/4150140301 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a6033a1e30 0x55a6034dab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:12.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.856+0000 7f7885290640 1 -- 172.21.15.38:0/4150140301 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a60342cbb0 con 0x55a603430950 2024-09-21T03:40:12.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.856+0000 7f788428e640 1 --2- 172.21.15.38:0/4150140301 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a6033a1e30 0x55a6034dab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:40:12.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.857+0000 7f7885290640 1 --2- 172.21.15.38:0/4150140301 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a603430950 0x55a6034db040 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f787c0464b0 tx=0x7f787c003330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8310599b40d2ce server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:12.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.857+0000 7f788328c640 1 -- 172.21.15.38:0/4150140301 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f787c004220 con 0x55a603430950 2024-09-21T03:40:12.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.857+0000 7f788328c640 1 -- 172.21.15.38:0/4150140301 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f787c04daa0 con 0x55a603430950 2024-09-21T03:40:12.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.857+0000 7f788328c640 1 -- 172.21.15.38:0/4150140301 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f787c05c470 con 0x55a603430950 2024-09-21T03:40:12.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.857+0000 7f788afd5f40 1 -- 172.21.15.38:0/4150140301 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a603430950 msgr2=0x55a6034db040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:12.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.857+0000 7f788afd5f40 1 --2- 172.21.15.38:0/4150140301 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a603430950 0x55a6034db040 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f787c0464b0 tx=0x7f787c003330 comp rx=0 tx=0).stop 2024-09-21T03:40:12.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.857+0000 7f788afd5f40 1 -- 172.21.15.38:0/4150140301 shutdown_connections 2024-09-21T03:40:12.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.857+0000 7f788afd5f40 1 --2- 172.21.15.38:0/4150140301 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a603430950 0x55a6034db040 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:12.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.857+0000 7f788afd5f40 1 --2- 172.21.15.38:0/4150140301 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a6033a1e30 0x55a6034dab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:12.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.857+0000 7f788afd5f40 1 --2- 172.21.15.38:0/4150140301 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a603375d00 0x55a60342a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:12.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.857+0000 7f788afd5f40 1 -- 172.21.15.38:0/4150140301 >> 172.21.15.38:0/4150140301 conn(0x55a603294ac0 msgr2=0x55a60344b5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:12.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.858+0000 7f788afd5f40 1 -- 172.21.15.38:0/4150140301 shutdown_connections 2024-09-21T03:40:12.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.858+0000 7f788afd5f40 1 -- 172.21.15.38:0/4150140301 wait complete. 2024-09-21T03:40:12.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.858+0000 7f788afd5f40 1 Processor -- start 2024-09-21T03:40:12.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.858+0000 7f788afd5f40 1 -- start start 2024-09-21T03:40:12.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.860+0000 7f788afd5f40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a603375d00 0x55a60342af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:12.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.860+0000 7f788afd5f40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a6033a1e30 0x55a60342b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:12.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.860+0000 7f788afd5f40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a603430950 0x55a603449130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:12.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.860+0000 7f788afd5f40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55a6033a0250 con 0x55a603375d00 2024-09-21T03:40:12.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.860+0000 7f788afd5f40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55a6033a04b0 con 0x55a603430950 2024-09-21T03:40:12.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.860+0000 7f788afd5f40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55a6034d2950 con 0x55a6033a1e30 2024-09-21T03:40:12.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.860+0000 7f7884a8f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a603375d00 0x55a60342af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:12.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.860+0000 7f7884a8f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a603375d00 0x55a60342af60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54448/0 (socket says 172.21.15.38:54448) 2024-09-21T03:40:12.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.860+0000 7f7884a8f640 1 -- 172.21.15.38:0/4092265200 learned_addr learned my addr 172.21.15.38:0/4092265200 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:12.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.860+0000 7f7885290640 1 --2- 172.21.15.38:0/4092265200 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a603430950 0x55a603449130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:12.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.860+0000 7f788428e640 1 --2- 172.21.15.38:0/4092265200 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a6033a1e30 0x55a60342b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:12.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.860+0000 7f7884a8f640 1 -- 172.21.15.38:0/4092265200 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a6033a1e30 msgr2=0x55a60342b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:12.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.860+0000 7f7884a8f640 1 --2- 172.21.15.38:0/4092265200 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a6033a1e30 0x55a60342b4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:12.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.860+0000 7f7884a8f640 1 -- 172.21.15.38:0/4092265200 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a603430950 msgr2=0x55a603449130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:12.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.860+0000 7f7884a8f640 1 --2- 172.21.15.38:0/4092265200 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a603430950 0x55a603449130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:12.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.860+0000 7f7884a8f640 1 -- 172.21.15.38:0/4092265200 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a603449f70 con 0x55a603375d00 2024-09-21T03:40:12.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.860+0000 7f7884a8f640 1 --2- 172.21.15.38:0/4092265200 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a603375d00 0x55a60342af60 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f787400a480 tx=0x7f787400a940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:12.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.861+0000 7f788228a640 1 -- 172.21.15.38:0/4092265200 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7874019070 con 0x55a603375d00 2024-09-21T03:40:12.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.861+0000 7f788228a640 1 -- 172.21.15.38:0/4092265200 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f78740087c0 con 0x55a603375d00 2024-09-21T03:40:12.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.861+0000 7f788afd5f40 1 -- 172.21.15.38:0/4092265200 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55a60344a250 con 0x55a603375d00 2024-09-21T03:40:12.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.861+0000 7f788228a640 1 -- 172.21.15.38:0/4092265200 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7874008c30 con 0x55a603375d00 2024-09-21T03:40:12.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.861+0000 7f788afd5f40 1 -- 172.21.15.38:0/4092265200 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a60342f810 con 0x55a603375d00 2024-09-21T03:40:12.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.862+0000 7f788228a640 1 -- 172.21.15.38:0/4092265200 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f787401c070 con 0x55a603375d00 2024-09-21T03:40:12.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.863+0000 7f788228a640 1 --2- 172.21.15.38:0/4092265200 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7870077570 0x7f7870079a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:12.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.863+0000 7f788228a640 1 -- 172.21.15.38:0/4092265200 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6187+0+0 (secure 0 0 0) 0x7f78740997d0 con 0x55a603375d00 2024-09-21T03:40:12.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.863+0000 7f788428e640 1 --2- 172.21.15.38:0/4092265200 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7870077570 0x7f7870079a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:12.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.863+0000 7f788afd5f40 1 --2- 172.21.15.38:0/4092265200 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x55a6034d21c0 0x55a6034e1330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:12.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.863+0000 7f788afd5f40 1 -- 172.21.15.38:0/4092265200 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:1 13.1f 13:ff5b34d6:::1.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x55a6034e1870 con 0x55a6034d21c0 2024-09-21T03:40:12.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.863+0000 7f7885290640 1 --2- 172.21.15.38:0/4092265200 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x55a6034d21c0 0x55a6034e1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:12.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.864+0000 7f788428e640 1 --2- 172.21.15.38:0/4092265200 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7870077570 0x7f7870079a80 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x55a60342aaa0 tx=0x7f7878001720 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:12.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.864+0000 7f7885290640 1 --2- 172.21.15.38:0/4092265200 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x55a6034d21c0 0x55a6034e1330 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:12.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.864+0000 7f7885290640 1 -- 172.21.15.38:0/4092265200 <== osd.1 v2:172.21.15.38:6816/1491818732 1 ==== osd_op_reply(1 1.00000000 [omap-get-header out=274b] v0'0 uv6 ondisk = 0) ==== 154+0+274 (crc 0 0 0) 0x7f787c00b040 con 0x55a6034d21c0 2024-09-21T03:40:12.868 INFO:teuthology.orchestra.run.smithi038.stdout:header (274 bytes) : 2024-09-21T03:40:12.868 INFO:teuthology.orchestra.run.smithi038.stdout:00000000 04 03 0c 01 00 00 0d 00 00 00 00 00 00 00 00 00 |................| 2024-09-21T03:40:12.868 INFO:teuthology.orchestra.run.smithi038.stdout:00000010 00 00 00 00 00 00 03 02 28 00 00 00 00 00 00 00 |........(.......| 2024-09-21T03:40:12.868 INFO:teuthology.orchestra.run.smithi038.stdout:00000020 00 00 00 00 19 40 ee 66 aa 41 02 37 00 00 00 00 |.....@.f.A.7....| 2024-09-21T03:40:12.868 INFO:teuthology.orchestra.run.smithi038.stdout:00000030 00 00 00 00 01 00 00 00 00 00 00 00 01 00 00 00 |................| 2024-09-21T03:40:12.868 INFO:teuthology.orchestra.run.smithi038.stdout:00000040 00 00 00 00 03 02 28 00 00 00 00 00 00 00 00 00 |......(.........| 2024-09-21T03:40:12.868 INFO:teuthology.orchestra.run.smithi038.stdout:00000050 00 00 19 40 ee 66 aa 41 02 37 00 00 00 00 00 00 |...@.f.A.7......| 2024-09-21T03:40:12.868 INFO:teuthology.orchestra.run.smithi038.stdout:00000060 00 00 01 00 00 00 00 00 00 00 01 00 00 00 00 00 |................| 2024-09-21T03:40:12.868 INFO:teuthology.orchestra.run.smithi038.stdout:00000070 00 00 03 02 38 00 00 00 00 00 00 00 00 00 00 00 |....8...........| 2024-09-21T03:40:12.868 INFO:teuthology.orchestra.run.smithi038.stdout:00000080 00 00 60 00 00 00 00 00 01 00 00 00 00 00 00 00 |..`.............| 2024-09-21T03:40:12.868 INFO:teuthology.orchestra.run.smithi038.stdout:00000090 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 2024-09-21T03:40:12.869 INFO:teuthology.orchestra.run.smithi038.stdout:000000a0 00 00 00 00 00 00 00 00 1a 40 ee 66 50 a8 bf 03 |.........@.fP...| 2024-09-21T03:40:12.869 INFO:teuthology.orchestra.run.smithi038.stdout:000000b0 03 02 38 00 00 00 00 00 00 00 00 00 00 00 00 00 |..8.............| 2024-09-21T03:40:12.869 INFO:teuthology.orchestra.run.smithi038.stdout:000000c0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 00 |................| 2024-09-21T03:40:12.869 INFO:teuthology.orchestra.run.smithi038.stdout:000000d0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 2024-09-21T03:40:12.869 INFO:teuthology.orchestra.run.smithi038.stdout:000000e0 00 00 00 00 00 00 19 40 ee 66 aa 41 02 37 00 00 |.......@.f.A.7..| 2024-09-21T03:40:12.869 INFO:teuthology.orchestra.run.smithi038.stdout:000000f0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 2024-09-21T03:40:12.869 INFO:teuthology.orchestra.run.smithi038.stdout:* 2024-09-21T03:40:12.869 INFO:teuthology.orchestra.run.smithi038.stdout:00000110 00 00 |..| 2024-09-21T03:40:12.869 INFO:teuthology.orchestra.run.smithi038.stdout:00000112 2024-09-21T03:40:12.869 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:40:12.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.864+0000 7f788afd5f40 1 -- 172.21.15.38:0/4092265200 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x55a6034d21c0 msgr2=0x55a6034e1330 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:12.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.864+0000 7f788afd5f40 1 --2- 172.21.15.38:0/4092265200 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x55a6034d21c0 0x55a6034e1330 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:12.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.864+0000 7f788afd5f40 1 -- 172.21.15.38:0/4092265200 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7870077570 msgr2=0x7f7870079a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:12.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.864+0000 7f788afd5f40 1 --2- 172.21.15.38:0/4092265200 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7870077570 0x7f7870079a80 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x55a60342aaa0 tx=0x7f7878001720 comp rx=0 tx=0).stop 2024-09-21T03:40:12.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.864+0000 7f788afd5f40 1 -- 172.21.15.38:0/4092265200 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a603375d00 msgr2=0x55a60342af60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:12.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.864+0000 7f788afd5f40 1 --2- 172.21.15.38:0/4092265200 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a603375d00 0x55a60342af60 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f787400a480 tx=0x7f787400a940 comp rx=0 tx=0).stop 2024-09-21T03:40:12.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.864+0000 7f7885290640 1 -- 172.21.15.38:0/4092265200 reap_dead start 2024-09-21T03:40:12.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.864+0000 7f788afd5f40 1 -- 172.21.15.38:0/4092265200 shutdown_connections 2024-09-21T03:40:12.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.864+0000 7f788afd5f40 1 -- 172.21.15.38:0/4092265200 >> 172.21.15.38:0/4092265200 conn(0x55a603294ac0 msgr2=0x55a60344b5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:12.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.864+0000 7f788afd5f40 1 -- 172.21.15.38:0/4092265200 shutdown_connections 2024-09-21T03:40:12.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.864+0000 7f788afd5f40 1 -- 172.21.15.38:0/4092265200 wait complete. 2024-09-21T03:40:12.872 DEBUG:teuthology.orchestra.run.smithi038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata listomapkeys 10000000000.00000000 2024-09-21T03:40:12.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.925+0000 7f6544d29f40 1 Processor -- start 2024-09-21T03:40:12.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.925+0000 7f6544d29f40 1 -- start start 2024-09-21T03:40:12.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.926+0000 7f6544d29f40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55904b668d00 0x55904b71d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:12.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.926+0000 7f6544d29f40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55904b694e30 0x55904b73e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:12.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.926+0000 7f6544d29f40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55904b723950 0x55904b73e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:12.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.926+0000 7f6544d29f40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55904b5b6d60 con 0x55904b723950 2024-09-21T03:40:12.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.926+0000 7f6544d29f40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55904b615430 con 0x55904b694e30 2024-09-21T03:40:12.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.926+0000 7f6544d29f40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55904b6af1b0 con 0x55904b668d00 2024-09-21T03:40:12.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.926+0000 7f653efe4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55904b723950 0x55904b73e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:12.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.926+0000 7f653efe4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55904b723950 0x55904b73e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54474/0 (socket says 172.21.15.38:54474) 2024-09-21T03:40:12.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.926+0000 7f653efe4640 1 -- 172.21.15.38:0/3254747091 learned_addr learned my addr 172.21.15.38:0/3254747091 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:12.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.926+0000 7f653e7e3640 1 --2- 172.21.15.38:0/3254747091 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55904b668d00 0x55904b71d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:12.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.926+0000 7f653dfe2640 1 --2- 172.21.15.38:0/3254747091 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55904b694e30 0x55904b73e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:12.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.926+0000 7f653efe4640 1 -- 172.21.15.38:0/3254747091 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55904b668d00 msgr2=0x55904b71d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:12.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.926+0000 7f653efe4640 1 --2- 172.21.15.38:0/3254747091 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55904b668d00 0x55904b71d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:12.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.926+0000 7f653efe4640 1 -- 172.21.15.38:0/3254747091 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55904b694e30 msgr2=0x55904b73e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:12.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.926+0000 7f653efe4640 1 --2- 172.21.15.38:0/3254747091 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55904b694e30 0x55904b73e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:12.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.926+0000 7f653efe4640 1 -- 172.21.15.38:0/3254747091 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55904b696d50 con 0x55904b723950 2024-09-21T03:40:12.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.927+0000 7f653efe4640 1 --2- 172.21.15.38:0/3254747091 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55904b723950 0x55904b73e890 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f653800a550 tx=0x7f6538003330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16f53fae227acb7f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:12.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.927+0000 7f653cfe0640 1 -- 172.21.15.38:0/3254747091 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6538064030 con 0x55904b723950 2024-09-21T03:40:12.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.927+0000 7f653cfe0640 1 -- 172.21.15.38:0/3254747091 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6538004020 con 0x55904b723950 2024-09-21T03:40:12.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.927+0000 7f653cfe0640 1 -- 172.21.15.38:0/3254747091 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6538004270 con 0x55904b723950 2024-09-21T03:40:12.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.927+0000 7f6544d29f40 1 -- 172.21.15.38:0/3254747091 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55904b723950 msgr2=0x55904b73e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:12.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.927+0000 7f6544d29f40 1 --2- 172.21.15.38:0/3254747091 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55904b723950 0x55904b73e890 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f653800a550 tx=0x7f6538003330 comp rx=0 tx=0).stop 2024-09-21T03:40:12.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.928+0000 7f6544d29f40 1 -- 172.21.15.38:0/3254747091 shutdown_connections 2024-09-21T03:40:12.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.928+0000 7f6544d29f40 1 --2- 172.21.15.38:0/3254747091 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55904b723950 0x55904b73e890 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:12.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.928+0000 7f6544d29f40 1 --2- 172.21.15.38:0/3254747091 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55904b694e30 0x55904b73e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:12.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.928+0000 7f6544d29f40 1 --2- 172.21.15.38:0/3254747091 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55904b668d00 0x55904b71d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:12.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.928+0000 7f6544d29f40 1 -- 172.21.15.38:0/3254747091 >> 172.21.15.38:0/3254747091 conn(0x55904b587ac0 msgr2=0x55904b692940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:12.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.928+0000 7f6544d29f40 1 -- 172.21.15.38:0/3254747091 shutdown_connections 2024-09-21T03:40:12.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.928+0000 7f6544d29f40 1 -- 172.21.15.38:0/3254747091 wait complete. 2024-09-21T03:40:12.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.928+0000 7f6544d29f40 1 Processor -- start 2024-09-21T03:40:12.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.928+0000 7f6544d29f40 1 -- start start 2024-09-21T03:40:12.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.930+0000 7f6544d29f40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55904b668d00 0x55904b7d4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:12.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.930+0000 7f6544d29f40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55904b694e30 0x55904b664870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:12.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.930+0000 7f6544d29f40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55904b723950 0x55904b664db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:12.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.930+0000 7f6544d29f40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55904b696ff0 con 0x55904b668d00 2024-09-21T03:40:12.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.930+0000 7f6544d29f40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55904b690dd0 con 0x55904b694e30 2024-09-21T03:40:12.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.930+0000 7f6544d29f40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55904b73c5e0 con 0x55904b723950 2024-09-21T03:40:12.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.930+0000 7f653e7e3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55904b668d00 0x55904b7d4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:12.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.930+0000 7f653e7e3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55904b668d00 0x55904b7d4e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54488/0 (socket says 172.21.15.38:54488) 2024-09-21T03:40:12.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.930+0000 7f653e7e3640 1 -- 172.21.15.38:0/148530444 learned_addr learned my addr 172.21.15.38:0/148530444 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:12.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.930+0000 7f653dfe2640 1 --2- 172.21.15.38:0/148530444 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55904b694e30 0x55904b664870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:12.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.930+0000 7f653efe4640 1 --2- 172.21.15.38:0/148530444 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55904b723950 0x55904b664db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:12.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.930+0000 7f653e7e3640 1 -- 172.21.15.38:0/148530444 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55904b723950 msgr2=0x55904b664db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:12.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.930+0000 7f653e7e3640 1 --2- 172.21.15.38:0/148530444 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55904b723950 0x55904b664db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:12.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.930+0000 7f653e7e3640 1 -- 172.21.15.38:0/148530444 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55904b694e30 msgr2=0x55904b664870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:12.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.930+0000 7f653e7e3640 1 --2- 172.21.15.38:0/148530444 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55904b694e30 0x55904b664870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:12.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.930+0000 7f653e7e3640 1 -- 172.21.15.38:0/148530444 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55904b73a720 con 0x55904b668d00 2024-09-21T03:40:12.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.930+0000 7f653e7e3640 1 --2- 172.21.15.38:0/148530444 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55904b668d00 0x55904b7d4e90 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f653000bfd0 tx=0x7f653000ab50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:12.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.930+0000 7f652f7fe640 1 -- 172.21.15.38:0/148530444 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f653001c070 con 0x55904b668d00 2024-09-21T03:40:12.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.931+0000 7f652f7fe640 1 -- 172.21.15.38:0/148530444 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6530008da0 con 0x55904b668d00 2024-09-21T03:40:12.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.931+0000 7f6544d29f40 1 -- 172.21.15.38:0/148530444 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55904b735ca0 con 0x55904b668d00 2024-09-21T03:40:12.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.931+0000 7f652f7fe640 1 -- 172.21.15.38:0/148530444 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f653000ae60 con 0x55904b668d00 2024-09-21T03:40:12.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.931+0000 7f6544d29f40 1 -- 172.21.15.38:0/148530444 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55904b73c320 con 0x55904b668d00 2024-09-21T03:40:12.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.932+0000 7f652f7fe640 1 -- 172.21.15.38:0/148530444 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f653001f070 con 0x55904b668d00 2024-09-21T03:40:12.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.933+0000 7f652f7fe640 1 --2- 172.21.15.38:0/148530444 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6520077570 0x7f6520079a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:12.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.933+0000 7f652f7fe640 1 -- 172.21.15.38:0/148530444 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6187+0+0 (secure 0 0 0) 0x7f6530099fa0 con 0x55904b668d00 2024-09-21T03:40:12.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.933+0000 7f653dfe2640 1 --2- 172.21.15.38:0/148530444 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6520077570 0x7f6520079a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:12.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.933+0000 7f6544d29f40 1 --2- 172.21.15.38:0/148530444 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x55904b722220 0x55904b73b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:12.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.933+0000 7f6544d29f40 1 -- 172.21.15.38:0/148530444 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:1 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-get-keys in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x55904b73bcf0 con 0x55904b722220 2024-09-21T03:40:12.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.933+0000 7f653efe4640 1 --2- 172.21.15.38:0/148530444 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x55904b722220 0x55904b73b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:12.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.933+0000 7f653dfe2640 1 --2- 172.21.15.38:0/148530444 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6520077570 0x7f6520079a80 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x55904b696490 tx=0x7f6534002b50 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:12.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.934+0000 7f653efe4640 1 --2- 172.21.15.38:0/148530444 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x55904b722220 0x55904b73b7b0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:12.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.934+0000 7f653efe4640 1 -- 172.21.15.38:0/148530444 <== osd.7 v2:172.21.15.195:6818/1606106342 1 ==== osd_op_reply(1 10000000000.00000000 [omap-get-keys out=21b] v0'0 uv10 ondisk = 0) ==== 164+0+21 (crc 0 0 0) 0x7f653800b040 con 0x55904b722220 2024-09-21T03:40:12.937 INFO:teuthology.orchestra.run.smithi038.stdout:sixmegs_head 2024-09-21T03:40:12.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.934+0000 7f6544d29f40 1 -- 172.21.15.38:0/148530444 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x55904b722220 msgr2=0x55904b73b7b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:12.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.934+0000 7f6544d29f40 1 --2- 172.21.15.38:0/148530444 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x55904b722220 0x55904b73b7b0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:12.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.934+0000 7f6544d29f40 1 -- 172.21.15.38:0/148530444 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6520077570 msgr2=0x7f6520079a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:12.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.934+0000 7f6544d29f40 1 --2- 172.21.15.38:0/148530444 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6520077570 0x7f6520079a80 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x55904b696490 tx=0x7f6534002b50 comp rx=0 tx=0).stop 2024-09-21T03:40:12.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.934+0000 7f6544d29f40 1 -- 172.21.15.38:0/148530444 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55904b668d00 msgr2=0x55904b7d4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:12.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.934+0000 7f6544d29f40 1 --2- 172.21.15.38:0/148530444 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55904b668d00 0x55904b7d4e90 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f653000bfd0 tx=0x7f653000ab50 comp rx=0 tx=0).stop 2024-09-21T03:40:12.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.934+0000 7f653efe4640 1 -- 172.21.15.38:0/148530444 reap_dead start 2024-09-21T03:40:12.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.934+0000 7f6544d29f40 1 -- 172.21.15.38:0/148530444 shutdown_connections 2024-09-21T03:40:12.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.934+0000 7f6544d29f40 1 -- 172.21.15.38:0/148530444 >> 172.21.15.38:0/148530444 conn(0x55904b587ac0 msgr2=0x55904b73e350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:12.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.934+0000 7f6544d29f40 1 -- 172.21.15.38:0/148530444 shutdown_connections 2024-09-21T03:40:12.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.934+0000 7f6544d29f40 1 -- 172.21.15.38:0/148530444 wait complete. 2024-09-21T03:40:12.940 DEBUG:teuthology.orchestra.run.smithi038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata listomapkeys 600.00000000 2024-09-21T03:40:12.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.993+0000 7fd7075d3f40 1 Processor -- start 2024-09-21T03:40:12.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.993+0000 7fd7075d3f40 1 -- start start 2024-09-21T03:40:12.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.994+0000 7fd7075d3f40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5561ab622240 0x5561ab64a650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:12.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.994+0000 7fd7075d3f40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5561ab669090 0x5561ab6dd150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:12.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.994+0000 7fd7075d3f40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5561ab6de520 0x5561ab6f65a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:12.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.994+0000 7fd7075d3f40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5561ab56fd60 con 0x5561ab6de520 2024-09-21T03:40:12.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.994+0000 7fd7075d3f40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5561ab5ce430 con 0x5561ab622240 2024-09-21T03:40:12.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.994+0000 7fd7075d3f40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x5561ab6f5810 con 0x5561ab669090 2024-09-21T03:40:12.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.994+0000 7fd70188e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5561ab6de520 0x5561ab6f65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:12.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.994+0000 7fd70188e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5561ab6de520 0x5561ab6f65a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54502/0 (socket says 172.21.15.38:54502) 2024-09-21T03:40:12.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.994+0000 7fd70188e640 1 -- 172.21.15.38:0/1448986829 learned_addr learned my addr 172.21.15.38:0/1448986829 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:12.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.994+0000 7fd6fbfff640 1 --2- 172.21.15.38:0/1448986829 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5561ab669090 0x5561ab6dd150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:12.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.994+0000 7fd70108d640 1 --2- 172.21.15.38:0/1448986829 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5561ab622240 0x5561ab64a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:12.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.995+0000 7fd70188e640 1 -- 172.21.15.38:0/1448986829 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5561ab669090 msgr2=0x5561ab6dd150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:12.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.995+0000 7fd70188e640 1 --2- 172.21.15.38:0/1448986829 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5561ab669090 0x5561ab6dd150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:12.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.995+0000 7fd70188e640 1 -- 172.21.15.38:0/1448986829 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5561ab622240 msgr2=0x5561ab64a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:12.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.995+0000 7fd70188e640 1 --2- 172.21.15.38:0/1448986829 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5561ab622240 0x5561ab64a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:12.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.995+0000 7fd70188e640 1 -- 172.21.15.38:0/1448986829 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561ab668c80 con 0x5561ab6de520 2024-09-21T03:40:12.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.995+0000 7fd70188e640 1 --2- 172.21.15.38:0/1448986829 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5561ab6de520 0x5561ab6f65a0 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7fd6fc025e70 tx=0x7fd6fc003330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d9f9574b99d7974 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:12.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.995+0000 7fd6fb7fe640 1 -- 172.21.15.38:0/1448986829 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6fc064030 con 0x5561ab6de520 2024-09-21T03:40:12.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.995+0000 7fd6fb7fe640 1 -- 172.21.15.38:0/1448986829 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd6fc004020 con 0x5561ab6de520 2024-09-21T03:40:12.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.995+0000 7fd6fb7fe640 1 -- 172.21.15.38:0/1448986829 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6fc004270 con 0x5561ab6de520 2024-09-21T03:40:12.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.995+0000 7fd7075d3f40 1 -- 172.21.15.38:0/1448986829 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5561ab6de520 msgr2=0x5561ab6f65a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:12.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.995+0000 7fd7075d3f40 1 --2- 172.21.15.38:0/1448986829 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5561ab6de520 0x5561ab6f65a0 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7fd6fc025e70 tx=0x7fd6fc003330 comp rx=0 tx=0).stop 2024-09-21T03:40:12.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.996+0000 7fd7075d3f40 1 -- 172.21.15.38:0/1448986829 shutdown_connections 2024-09-21T03:40:12.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.996+0000 7fd7075d3f40 1 --2- 172.21.15.38:0/1448986829 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5561ab6de520 0x5561ab6f65a0 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:12.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.996+0000 7fd7075d3f40 1 --2- 172.21.15.38:0/1448986829 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5561ab669090 0x5561ab6dd150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:12.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.996+0000 7fd7075d3f40 1 --2- 172.21.15.38:0/1448986829 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5561ab622240 0x5561ab64a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:12.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.996+0000 7fd7075d3f40 1 -- 172.21.15.38:0/1448986829 >> 172.21.15.38:0/1448986829 conn(0x5561ab540ac0 msgr2=0x5561ab6eedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:13.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.996+0000 7fd7075d3f40 1 -- 172.21.15.38:0/1448986829 shutdown_connections 2024-09-21T03:40:13.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.996+0000 7fd7075d3f40 1 -- 172.21.15.38:0/1448986829 wait complete. 2024-09-21T03:40:13.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.996+0000 7fd7075d3f40 1 Processor -- start 2024-09-21T03:40:13.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.997+0000 7fd7075d3f40 1 -- start start 2024-09-21T03:40:13.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.998+0000 7fd7075d3f40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5561ab622240 0x5561ab6f7780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:13.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.998+0000 7fd7075d3f40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5561ab669090 0x5561ab787020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:13.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.998+0000 7fd7075d3f40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5561ab6de520 0x5561ab61d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:13.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.998+0000 7fd7075d3f40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5561ab64c500 con 0x5561ab622240 2024-09-21T03:40:13.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.998+0000 7fd7075d3f40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5561ab64ba80 con 0x5561ab669090 2024-09-21T03:40:13.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.998+0000 7fd7075d3f40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x5561ab64c060 con 0x5561ab6de520 2024-09-21T03:40:13.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.998+0000 7fd70108d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5561ab622240 0x5561ab6f7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:13.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.998+0000 7fd70108d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5561ab622240 0x5561ab6f7780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54510/0 (socket says 172.21.15.38:54510) 2024-09-21T03:40:13.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.998+0000 7fd70108d640 1 -- 172.21.15.38:0/10475233 learned_addr learned my addr 172.21.15.38:0/10475233 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:13.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.998+0000 7fd6fbfff640 1 --2- 172.21.15.38:0/10475233 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5561ab669090 0x5561ab787020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:13.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.998+0000 7fd70188e640 1 --2- 172.21.15.38:0/10475233 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5561ab6de520 0x5561ab61d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:13.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.998+0000 7fd70108d640 1 -- 172.21.15.38:0/10475233 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5561ab6de520 msgr2=0x5561ab61d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:13.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.998+0000 7fd70108d640 1 --2- 172.21.15.38:0/10475233 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5561ab6de520 0x5561ab61d850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:13.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.998+0000 7fd70108d640 1 -- 172.21.15.38:0/10475233 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5561ab669090 msgr2=0x5561ab787020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:13.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.998+0000 7fd70108d640 1 --2- 172.21.15.38:0/10475233 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5561ab669090 0x5561ab787020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:13.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.998+0000 7fd70108d640 1 -- 172.21.15.38:0/10475233 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561ab6dd190 con 0x5561ab622240 2024-09-21T03:40:13.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.998+0000 7fd70108d640 1 --2- 172.21.15.38:0/10475233 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5561ab622240 0x5561ab6f7780 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7fd6f4002d20 tx=0x7fd6f400c620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:13.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.999+0000 7fd6fa7fc640 1 -- 172.21.15.38:0/10475233 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6f4015020 con 0x5561ab622240 2024-09-21T03:40:13.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.999+0000 7fd6fa7fc640 1 -- 172.21.15.38:0/10475233 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd6f400cd30 con 0x5561ab622240 2024-09-21T03:40:13.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.999+0000 7fd6fa7fc640 1 -- 172.21.15.38:0/10475233 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6f40116b0 con 0x5561ab622240 2024-09-21T03:40:13.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.999+0000 7fd7075d3f40 1 -- 172.21.15.38:0/10475233 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5561ab6dd470 con 0x5561ab622240 2024-09-21T03:40:13.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:12.999+0000 7fd7075d3f40 1 -- 172.21.15.38:0/10475233 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5561ab6dc0c0 con 0x5561ab622240 2024-09-21T03:40:13.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.000+0000 7fd6fa7fc640 1 -- 172.21.15.38:0/10475233 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fd6f4018020 con 0x5561ab622240 2024-09-21T03:40:13.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.001+0000 7fd6fa7fc640 1 --2- 172.21.15.38:0/10475233 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd6e4077570 0x7fd6e4079a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:13.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.001+0000 7fd6fa7fc640 1 -- 172.21.15.38:0/10475233 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6187+0+0 (secure 0 0 0) 0x7fd6f4099be0 con 0x5561ab622240 2024-09-21T03:40:13.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.001+0000 7fd6fbfff640 1 --2- 172.21.15.38:0/10475233 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd6e4077570 0x7fd6e4079a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:13.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.001+0000 7fd7075d3f40 1 --2- 172.21.15.38:0/10475233 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x5561ab61dd90 0x5561ab790720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:13.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.001+0000 7fd7075d3f40 1 -- 172.21.15.38:0/10475233 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:1 13.10 13:0d82a743:::600.00000000:head [omap-get-keys in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x5561ab6dc4c0 con 0x5561ab61dd90 2024-09-21T03:40:13.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.001+0000 7fd70188e640 1 --2- 172.21.15.38:0/10475233 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x5561ab61dd90 0x5561ab790720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:13.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.001+0000 7fd6fbfff640 1 --2- 172.21.15.38:0/10475233 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd6e4077570 0x7fd6e4079a80 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x5561ab784990 tx=0x7fd6ec004880 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:13.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.002+0000 7fd70188e640 1 --2- 172.21.15.38:0/10475233 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x5561ab61dd90 0x5561ab790720 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:13.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.002+0000 7fd70188e640 1 -- 172.21.15.38:0/10475233 <== osd.7 v2:172.21.15.195:6818/1606106342 1 ==== osd_op_reply(1 600.00000000 [omap-get-keys out=5b] v0'0 uv2 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x7fd6fc00b040 con 0x5561ab61dd90 2024-09-21T03:40:13.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.002+0000 7fd7075d3f40 1 -- 172.21.15.38:0/10475233 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x5561ab61dd90 msgr2=0x5561ab790720 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:13.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.002+0000 7fd7075d3f40 1 --2- 172.21.15.38:0/10475233 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x5561ab61dd90 0x5561ab790720 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:13.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.002+0000 7fd7075d3f40 1 -- 172.21.15.38:0/10475233 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd6e4077570 msgr2=0x7fd6e4079a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:13.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.002+0000 7fd7075d3f40 1 --2- 172.21.15.38:0/10475233 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd6e4077570 0x7fd6e4079a80 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x5561ab784990 tx=0x7fd6ec004880 comp rx=0 tx=0).stop 2024-09-21T03:40:13.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.002+0000 7fd7075d3f40 1 -- 172.21.15.38:0/10475233 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5561ab622240 msgr2=0x5561ab6f7780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:13.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.002+0000 7fd7075d3f40 1 --2- 172.21.15.38:0/10475233 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5561ab622240 0x5561ab6f7780 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7fd6f4002d20 tx=0x7fd6f400c620 comp rx=0 tx=0).stop 2024-09-21T03:40:13.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.002+0000 7fd70188e640 1 -- 172.21.15.38:0/10475233 reap_dead start 2024-09-21T03:40:13.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.002+0000 7fd7075d3f40 1 -- 172.21.15.38:0/10475233 shutdown_connections 2024-09-21T03:40:13.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.002+0000 7fd7075d3f40 1 -- 172.21.15.38:0/10475233 >> 172.21.15.38:0/10475233 conn(0x5561ab540ac0 msgr2=0x5561ab6f7e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:13.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.002+0000 7fd7075d3f40 1 -- 172.21.15.38:0/10475233 shutdown_connections 2024-09-21T03:40:13.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.002+0000 7fd7075d3f40 1 -- 172.21.15.38:0/10475233 wait complete. 2024-09-21T03:40:13.008 DEBUG:teuthology.orchestra.run.smithi038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata listomapkeys 1.00000000.inode 2024-09-21T03:40:13.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.061+0000 7f9908bc3f40 1 Processor -- start 2024-09-21T03:40:13.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.061+0000 7f9908bc3f40 1 -- start start 2024-09-21T03:40:13.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.062+0000 7f9908bc3f40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f5b1570d00 0x55f5b1641cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:13.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.062+0000 7f9908bc3f40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f5b159de40 0x55f5b16467f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:13.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.062+0000 7f9908bc3f40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f5b162b950 0x55f5b1629010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:13.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.062+0000 7f9908bc3f40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55f5b151d430 con 0x55f5b162b950 2024-09-21T03:40:13.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.062+0000 7f9908bc3f40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55f5b152a1f0 con 0x55f5b159de40 2024-09-21T03:40:13.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.062+0000 7f9908bc3f40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55f5b14bed60 con 0x55f5b1570d00 2024-09-21T03:40:13.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.062+0000 7f9902e7e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f5b162b950 0x55f5b1629010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:13.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.062+0000 7f9902e7e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f5b162b950 0x55f5b1629010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54538/0 (socket says 172.21.15.38:54538) 2024-09-21T03:40:13.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.062+0000 7f9902e7e640 1 -- 172.21.15.38:0/15601708 learned_addr learned my addr 172.21.15.38:0/15601708 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:13.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.062+0000 7f9901e7c640 1 --2- 172.21.15.38:0/15601708 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f5b159de40 0x55f5b16467f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:13.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.062+0000 7f990267d640 1 --2- 172.21.15.38:0/15601708 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f5b1570d00 0x55f5b1641cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:13.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.062+0000 7f9902e7e640 1 -- 172.21.15.38:0/15601708 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f5b1570d00 msgr2=0x55f5b1641cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:13.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.062+0000 7f9902e7e640 1 --2- 172.21.15.38:0/15601708 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f5b1570d00 0x55f5b1641cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:13.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.062+0000 7f9902e7e640 1 -- 172.21.15.38:0/15601708 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f5b159de40 msgr2=0x55f5b16467f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:13.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.062+0000 7f9902e7e640 1 --2- 172.21.15.38:0/15601708 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f5b159de40 0x55f5b16467f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:13.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.062+0000 7f9902e7e640 1 -- 172.21.15.38:0/15601708 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5b159a960 con 0x55f5b162b950 2024-09-21T03:40:13.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.062+0000 7f990267d640 1 --2- 172.21.15.38:0/15601708 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f5b1570d00 0x55f5b1641cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:40:13.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.063+0000 7f9902e7e640 1 --2- 172.21.15.38:0/15601708 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f5b162b950 0x55f5b1629010 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f98fc046410 tx=0x7f98fc003330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1085872f1ac6732f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:13.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.063+0000 7f9900e7a640 1 -- 172.21.15.38:0/15601708 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98fc064070 con 0x55f5b162b950 2024-09-21T03:40:13.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.063+0000 7f9900e7a640 1 -- 172.21.15.38:0/15601708 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f98fc004020 con 0x55f5b162b950 2024-09-21T03:40:13.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.063+0000 7f9900e7a640 1 -- 172.21.15.38:0/15601708 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98fc04da00 con 0x55f5b162b950 2024-09-21T03:40:13.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.063+0000 7f9908bc3f40 1 -- 172.21.15.38:0/15601708 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f5b162b950 msgr2=0x55f5b1629010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:13.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.063+0000 7f9908bc3f40 1 --2- 172.21.15.38:0/15601708 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f5b162b950 0x55f5b1629010 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f98fc046410 tx=0x7f98fc003330 comp rx=0 tx=0).stop 2024-09-21T03:40:13.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.063+0000 7f9908bc3f40 1 -- 172.21.15.38:0/15601708 shutdown_connections 2024-09-21T03:40:13.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.063+0000 7f9908bc3f40 1 --2- 172.21.15.38:0/15601708 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f5b162b950 0x55f5b1629010 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:13.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.063+0000 7f9908bc3f40 1 --2- 172.21.15.38:0/15601708 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f5b159de40 0x55f5b16467f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:13.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.063+0000 7f9908bc3f40 1 --2- 172.21.15.38:0/15601708 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f5b1570d00 0x55f5b1641cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:13.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.063+0000 7f9908bc3f40 1 -- 172.21.15.38:0/15601708 >> 172.21.15.38:0/15601708 conn(0x55f5b148fac0 msgr2=0x55f5b159e300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:13.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.063+0000 7f9908bc3f40 1 -- 172.21.15.38:0/15601708 shutdown_connections 2024-09-21T03:40:13.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.063+0000 7f9908bc3f40 1 -- 172.21.15.38:0/15601708 wait complete. 2024-09-21T03:40:13.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.064+0000 7f9908bc3f40 1 Processor -- start 2024-09-21T03:40:13.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.064+0000 7f9908bc3f40 1 -- start start 2024-09-21T03:40:13.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.066+0000 7f9908bc3f40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f5b1570d00 0x55f5b1640f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:13.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.066+0000 7f9908bc3f40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f5b159de40 0x55f5b16408c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:13.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.066+0000 7f9908bc3f40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f5b162b950 0x55f5b159e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:13.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.066+0000 7f9908bc3f40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55f5b16473b0 con 0x55f5b159de40 2024-09-21T03:40:13.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.066+0000 7f9908bc3f40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55f5b159a6c0 con 0x55f5b162b950 2024-09-21T03:40:13.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.066+0000 7f9908bc3f40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55f5b16cd6b0 con 0x55f5b1570d00 2024-09-21T03:40:13.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.066+0000 7f9901e7c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f5b159de40 0x55f5b16408c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:13.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.066+0000 7f9901e7c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f5b159de40 0x55f5b16408c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54548/0 (socket says 172.21.15.38:54548) 2024-09-21T03:40:13.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.066+0000 7f9901e7c640 1 -- 172.21.15.38:0/161418202 learned_addr learned my addr 172.21.15.38:0/161418202 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:13.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.066+0000 7f9902e7e640 1 --2- 172.21.15.38:0/161418202 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f5b162b950 0x55f5b159e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:13.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.066+0000 7f990267d640 1 --2- 172.21.15.38:0/161418202 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f5b1570d00 0x55f5b1640f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:13.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.066+0000 7f9901e7c640 1 -- 172.21.15.38:0/161418202 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f5b1570d00 msgr2=0x55f5b1640f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:13.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.066+0000 7f9901e7c640 1 --2- 172.21.15.38:0/161418202 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f5b1570d00 0x55f5b1640f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:13.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.066+0000 7f9901e7c640 1 -- 172.21.15.38:0/161418202 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f5b162b950 msgr2=0x55f5b159e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:13.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.066+0000 7f9901e7c640 1 --2- 172.21.15.38:0/161418202 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f5b162b950 0x55f5b159e300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:13.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.066+0000 7f9901e7c640 1 -- 172.21.15.38:0/161418202 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5b16e13e0 con 0x55f5b159de40 2024-09-21T03:40:13.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.066+0000 7f9901e7c640 1 --2- 172.21.15.38:0/161418202 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f5b159de40 0x55f5b16408c0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f98f800df50 tx=0x7f98f800b650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:13.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.066+0000 7f98f37fe640 1 -- 172.21.15.38:0/161418202 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98f80032e0 con 0x55f5b159de40 2024-09-21T03:40:13.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.067+0000 7f98f37fe640 1 -- 172.21.15.38:0/161418202 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f98f8016070 con 0x55f5b159de40 2024-09-21T03:40:13.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.067+0000 7f9908bc3f40 1 -- 172.21.15.38:0/161418202 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55f5b16e16c0 con 0x55f5b159de40 2024-09-21T03:40:13.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.067+0000 7f98f37fe640 1 -- 172.21.15.38:0/161418202 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98f8001860 con 0x55f5b159de40 2024-09-21T03:40:13.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.067+0000 7f9908bc3f40 1 -- 172.21.15.38:0/161418202 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f5b15b3250 con 0x55f5b159de40 2024-09-21T03:40:13.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.068+0000 7f98f37fe640 1 -- 172.21.15.38:0/161418202 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f98f8001a00 con 0x55f5b159de40 2024-09-21T03:40:13.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.069+0000 7f98f37fe640 1 --2- 172.21.15.38:0/161418202 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f98e4077570 0x7f98e4079a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:13.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.069+0000 7f98f37fe640 1 -- 172.21.15.38:0/161418202 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6187+0+0 (secure 0 0 0) 0x7f98f8099c60 con 0x55f5b159de40 2024-09-21T03:40:13.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.069+0000 7f990267d640 1 --2- 172.21.15.38:0/161418202 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f98e4077570 0x7f98e4079a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:13.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.069+0000 7f9908bc3f40 1 --2- 172.21.15.38:0/161418202 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x55f5b15b3c80 0x55f5b15b4090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:13.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.069+0000 7f9908bc3f40 1 -- 172.21.15.38:0/161418202 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:1 13.14 13:287034c4:::1.00000000.inode:head [omap-get-keys in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x55f5b16e9a80 con 0x55f5b15b3c80 2024-09-21T03:40:13.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.069+0000 7f9902e7e640 1 --2- 172.21.15.38:0/161418202 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x55f5b15b3c80 0x55f5b15b4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:13.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.069+0000 7f990267d640 1 --2- 172.21.15.38:0/161418202 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f98e4077570 0x7f98e4079a80 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f98f4000f60 tx=0x7f98f4001710 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:13.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.070+0000 7f9902e7e640 1 --2- 172.21.15.38:0/161418202 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x55f5b15b3c80 0x55f5b15b4090 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:13.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.070+0000 7f9902e7e640 1 -- 172.21.15.38:0/161418202 <== osd.6 v2:172.21.15.195:6826/4055825675 1 ==== osd_op_reply(1 1.00000000.inode [omap-get-keys out=5b] v0'0 uv15 ondisk = 0) ==== 160+0+5 (crc 0 0 0) 0x7f98fc00b040 con 0x55f5b15b3c80 2024-09-21T03:40:13.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.070+0000 7f9908bc3f40 1 -- 172.21.15.38:0/161418202 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x55f5b15b3c80 msgr2=0x55f5b15b4090 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:13.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.070+0000 7f9908bc3f40 1 --2- 172.21.15.38:0/161418202 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x55f5b15b3c80 0x55f5b15b4090 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:13.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.070+0000 7f9908bc3f40 1 -- 172.21.15.38:0/161418202 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f98e4077570 msgr2=0x7f98e4079a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:13.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.070+0000 7f9908bc3f40 1 --2- 172.21.15.38:0/161418202 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f98e4077570 0x7f98e4079a80 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f98f4000f60 tx=0x7f98f4001710 comp rx=0 tx=0).stop 2024-09-21T03:40:13.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.070+0000 7f9908bc3f40 1 -- 172.21.15.38:0/161418202 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f5b159de40 msgr2=0x55f5b16408c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:13.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.070+0000 7f9908bc3f40 1 --2- 172.21.15.38:0/161418202 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f5b159de40 0x55f5b16408c0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f98f800df50 tx=0x7f98f800b650 comp rx=0 tx=0).stop 2024-09-21T03:40:13.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.070+0000 7f9902e7e640 1 -- 172.21.15.38:0/161418202 reap_dead start 2024-09-21T03:40:13.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.070+0000 7f9908bc3f40 1 -- 172.21.15.38:0/161418202 shutdown_connections 2024-09-21T03:40:13.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.070+0000 7f9908bc3f40 1 -- 172.21.15.38:0/161418202 >> 172.21.15.38:0/161418202 conn(0x55f5b148fac0 msgr2=0x55f5b16cdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:13.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.070+0000 7f9908bc3f40 1 -- 172.21.15.38:0/161418202 shutdown_connections 2024-09-21T03:40:13.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.070+0000 7f9908bc3f40 1 -- 172.21.15.38:0/161418202 wait complete. 2024-09-21T03:40:13.078 DEBUG:teuthology.orchestra.run.smithi038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata listomapkeys 200.00000000 2024-09-21T03:40:13.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.131+0000 7fe36cd68f40 1 Processor -- start 2024-09-21T03:40:13.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.131+0000 7fe36cd68f40 1 -- start start 2024-09-21T03:40:13.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.132+0000 7fe36cd68f40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e541b57d00 0x55e541cb7120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:13.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.132+0000 7fe36cd68f40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e541c12950 0x55e541c0d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:13.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.132+0000 7fe36cd68f40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e541c2d670 0x55e541cb44f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:13.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.132+0000 7fe36cd68f40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55e541aa5d60 con 0x55e541c2d670 2024-09-21T03:40:13.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.132+0000 7fe36cd68f40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55e541b04430 con 0x55e541c12950 2024-09-21T03:40:13.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.132+0000 7fe36cd68f40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55e541b9f7b0 con 0x55e541b57d00 2024-09-21T03:40:13.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.132+0000 7fe367023640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e541c2d670 0x55e541cb44f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:13.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.132+0000 7fe367023640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e541c2d670 0x55e541cb44f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54570/0 (socket says 172.21.15.38:54570) 2024-09-21T03:40:13.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.132+0000 7fe367023640 1 -- 172.21.15.38:0/1959216393 learned_addr learned my addr 172.21.15.38:0/1959216393 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:13.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.132+0000 7fe366021640 1 --2- 172.21.15.38:0/1959216393 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e541c12950 0x55e541c0d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:13.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.132+0000 7fe366822640 1 --2- 172.21.15.38:0/1959216393 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e541b57d00 0x55e541cb7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:13.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.133+0000 7fe367023640 1 -- 172.21.15.38:0/1959216393 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e541b57d00 msgr2=0x55e541cb7120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:13.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.133+0000 7fe367023640 1 --2- 172.21.15.38:0/1959216393 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e541b57d00 0x55e541cb7120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:13.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.133+0000 7fe367023640 1 -- 172.21.15.38:0/1959216393 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e541c12950 msgr2=0x55e541c0d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:13.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.133+0000 7fe367023640 1 --2- 172.21.15.38:0/1959216393 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e541c12950 0x55e541c0d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:13.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.133+0000 7fe367023640 1 -- 172.21.15.38:0/1959216393 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e541c0d6f0 con 0x55e541c2d670 2024-09-21T03:40:13.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.133+0000 7fe367023640 1 --2- 172.21.15.38:0/1959216393 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e541c2d670 0x55e541cb44f0 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7fe3600014e0 tx=0x7fe3600019a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f50a350a8e1b8205 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:13.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.133+0000 7fe36501f640 1 -- 172.21.15.38:0/1959216393 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe360017070 con 0x55e541c2d670 2024-09-21T03:40:13.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.133+0000 7fe36501f640 1 -- 172.21.15.38:0/1959216393 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe3600027a0 con 0x55e541c2d670 2024-09-21T03:40:13.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.133+0000 7fe36501f640 1 -- 172.21.15.38:0/1959216393 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe360002bf0 con 0x55e541c2d670 2024-09-21T03:40:13.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.134+0000 7fe36cd68f40 1 -- 172.21.15.38:0/1959216393 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e541c2d670 msgr2=0x55e541cb44f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:13.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.134+0000 7fe36cd68f40 1 --2- 172.21.15.38:0/1959216393 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e541c2d670 0x55e541cb44f0 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7fe3600014e0 tx=0x7fe3600019a0 comp rx=0 tx=0).stop 2024-09-21T03:40:13.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.134+0000 7fe36cd68f40 1 -- 172.21.15.38:0/1959216393 shutdown_connections 2024-09-21T03:40:13.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.134+0000 7fe36cd68f40 1 --2- 172.21.15.38:0/1959216393 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e541c2d670 0x55e541cb44f0 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:13.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.134+0000 7fe36cd68f40 1 --2- 172.21.15.38:0/1959216393 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e541c12950 0x55e541c0d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:13.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.134+0000 7fe36cd68f40 1 --2- 172.21.15.38:0/1959216393 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e541b57d00 0x55e541cb7120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:13.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.134+0000 7fe36cd68f40 1 -- 172.21.15.38:0/1959216393 >> 172.21.15.38:0/1959216393 conn(0x55e541a76ac0 msgr2=0x55e541c2da80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:13.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.134+0000 7fe36cd68f40 1 -- 172.21.15.38:0/1959216393 shutdown_connections 2024-09-21T03:40:13.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.134+0000 7fe36cd68f40 1 -- 172.21.15.38:0/1959216393 wait complete. 2024-09-21T03:40:13.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.135+0000 7fe36cd68f40 1 Processor -- start 2024-09-21T03:40:13.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.135+0000 7fe36cd68f40 1 -- start start 2024-09-21T03:40:13.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.136+0000 7fe36cd68f40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e541b57d00 0x55e541cb62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:13.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.136+0000 7fe36cd68f40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e541c12950 0x55e541cb6820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:13.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.136+0000 7fe36cd68f40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e541c2d670 0x55e541c2aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:13.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.136+0000 7fe36cd68f40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55e541c0ca40 con 0x55e541c12950 2024-09-21T03:40:13.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.136+0000 7fe36cd68f40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55e541cb78e0 con 0x55e541c2d670 2024-09-21T03:40:13.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.136+0000 7fe36cd68f40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55e541b82510 con 0x55e541b57d00 2024-09-21T03:40:13.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.136+0000 7fe366021640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e541c12950 0x55e541cb6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:13.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.136+0000 7fe366021640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e541c12950 0x55e541cb6820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54584/0 (socket says 172.21.15.38:54584) 2024-09-21T03:40:13.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.136+0000 7fe366021640 1 -- 172.21.15.38:0/1177843261 learned_addr learned my addr 172.21.15.38:0/1177843261 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:13.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.136+0000 7fe366822640 1 --2- 172.21.15.38:0/1177843261 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e541b57d00 0x55e541cb62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:13.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.136+0000 7fe367023640 1 --2- 172.21.15.38:0/1177843261 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e541c2d670 0x55e541c2aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:13.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.136+0000 7fe366021640 1 -- 172.21.15.38:0/1177843261 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e541b57d00 msgr2=0x55e541cb62e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:13.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.136+0000 7fe366021640 1 --2- 172.21.15.38:0/1177843261 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e541b57d00 0x55e541cb62e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:13.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.136+0000 7fe366021640 1 -- 172.21.15.38:0/1177843261 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e541c2d670 msgr2=0x55e541c2aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:13.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.136+0000 7fe366021640 1 --2- 172.21.15.38:0/1177843261 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e541c2d670 0x55e541c2aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:13.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.136+0000 7fe366021640 1 -- 172.21.15.38:0/1177843261 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e541cc2dd0 con 0x55e541c12950 2024-09-21T03:40:13.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.137+0000 7fe366021640 1 --2- 172.21.15.38:0/1177843261 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e541c12950 0x55e541cb6820 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7fe35c025e30 tx=0x7fe35c04d830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:13.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.137+0000 7fe34ffff640 1 -- 172.21.15.38:0/1177843261 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe35c058190 con 0x55e541c12950 2024-09-21T03:40:13.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.137+0000 7fe34ffff640 1 -- 172.21.15.38:0/1177843261 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe35c058510 con 0x55e541c12950 2024-09-21T03:40:13.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.137+0000 7fe36cd68f40 1 -- 172.21.15.38:0/1177843261 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55e541cc30b0 con 0x55e541c12950 2024-09-21T03:40:13.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.137+0000 7fe34ffff640 1 -- 172.21.15.38:0/1177843261 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe35c058920 con 0x55e541c12950 2024-09-21T03:40:13.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.137+0000 7fe36cd68f40 1 -- 172.21.15.38:0/1177843261 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e541cb4bb0 con 0x55e541c12950 2024-09-21T03:40:13.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.139+0000 7fe34ffff640 1 -- 172.21.15.38:0/1177843261 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fe35c058ac0 con 0x55e541c12950 2024-09-21T03:40:13.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.139+0000 7fe34ffff640 1 --2- 172.21.15.38:0/1177843261 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fe3540775a0 0x7fe354079ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:13.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.139+0000 7fe34ffff640 1 -- 172.21.15.38:0/1177843261 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6187+0+0 (secure 0 0 0) 0x7fe35c0ddab0 con 0x55e541c12950 2024-09-21T03:40:13.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.139+0000 7fe366822640 1 --2- 172.21.15.38:0/1177843261 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fe3540775a0 0x7fe354079ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:13.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.139+0000 7fe36cd68f40 1 --2- 172.21.15.38:0/1177843261 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x55e541c24b40 0x55e541c2b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:13.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.139+0000 7fe36cd68f40 1 -- 172.21.15.38:0/1177843261 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:1 13.14 13:292cf221:::200.00000000:head [omap-get-keys in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x55e541c0baa0 con 0x55e541c24b40 2024-09-21T03:40:13.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.139+0000 7fe367023640 1 --2- 172.21.15.38:0/1177843261 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x55e541c24b40 0x55e541c2b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:13.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.140+0000 7fe366822640 1 --2- 172.21.15.38:0/1177843261 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fe3540775a0 0x7fe354079ab0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fe35800c700 tx=0x7fe35800e040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:13.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.140+0000 7fe367023640 1 --2- 172.21.15.38:0/1177843261 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x55e541c24b40 0x55e541c2b4a0 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:13.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.140+0000 7fe367023640 1 -- 172.21.15.38:0/1177843261 <== osd.6 v2:172.21.15.195:6826/4055825675 1 ==== osd_op_reply(1 200.00000000 [omap-get-keys out=5b] v0'0 uv18 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x7fe36000f070 con 0x55e541c24b40 2024-09-21T03:40:13.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.140+0000 7fe36cd68f40 1 -- 172.21.15.38:0/1177843261 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x55e541c24b40 msgr2=0x55e541c2b4a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:13.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.140+0000 7fe36cd68f40 1 --2- 172.21.15.38:0/1177843261 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x55e541c24b40 0x55e541c2b4a0 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:13.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.140+0000 7fe36cd68f40 1 -- 172.21.15.38:0/1177843261 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fe3540775a0 msgr2=0x7fe354079ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:13.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.140+0000 7fe36cd68f40 1 --2- 172.21.15.38:0/1177843261 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fe3540775a0 0x7fe354079ab0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fe35800c700 tx=0x7fe35800e040 comp rx=0 tx=0).stop 2024-09-21T03:40:13.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.140+0000 7fe36cd68f40 1 -- 172.21.15.38:0/1177843261 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e541c12950 msgr2=0x55e541cb6820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:13.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.140+0000 7fe36cd68f40 1 --2- 172.21.15.38:0/1177843261 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e541c12950 0x55e541cb6820 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7fe35c025e30 tx=0x7fe35c04d830 comp rx=0 tx=0).stop 2024-09-21T03:40:13.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.140+0000 7fe367023640 1 -- 172.21.15.38:0/1177843261 reap_dead start 2024-09-21T03:40:13.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.140+0000 7fe36cd68f40 1 -- 172.21.15.38:0/1177843261 shutdown_connections 2024-09-21T03:40:13.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.140+0000 7fe36cd68f40 1 -- 172.21.15.38:0/1177843261 >> 172.21.15.38:0/1177843261 conn(0x55e541a76ac0 msgr2=0x55e541c29020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:13.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.140+0000 7fe36cd68f40 1 -- 172.21.15.38:0/1177843261 shutdown_connections 2024-09-21T03:40:13.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.140+0000 7fe36cd68f40 1 -- 172.21.15.38:0/1177843261 wait complete. 2024-09-21T03:40:13.146 DEBUG:teuthology.orchestra.run.smithi038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata listomapkeys 200.00000001 2024-09-21T03:40:13.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.199+0000 7fc6a515cf40 1 Processor -- start 2024-09-21T03:40:13.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.199+0000 7fc6a515cf40 1 -- start start 2024-09-21T03:40:13.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.200+0000 7fc6a515cf40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555b21e07090 0x555b21de8840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:13.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.200+0000 7fc6a515cf40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555b21e7c520 0x555b21de9c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:13.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.200+0000 7fc6a515cf40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555b21dea190 0x555b21e8ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:13.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.200+0000 7fc6a515cf40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x555b21d6c430 con 0x555b21e7c520 2024-09-21T03:40:13.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.200+0000 7fc6a515cf40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x555b21d791f0 con 0x555b21e07090 2024-09-21T03:40:13.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.200+0000 7fc6a515cf40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x555b21d0dd60 con 0x555b21dea190 2024-09-21T03:40:13.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.200+0000 7fc69e415640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555b21e7c520 0x555b21de9c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:13.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.200+0000 7fc69e415640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555b21e7c520 0x555b21de9c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54600/0 (socket says 172.21.15.38:54600) 2024-09-21T03:40:13.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.200+0000 7fc69ec16640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555b21e07090 0x555b21de8840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:13.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.200+0000 7fc69e415640 1 -- 172.21.15.38:0/3744604442 learned_addr learned my addr 172.21.15.38:0/3744604442 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:13.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.200+0000 7fc69f417640 1 --2- 172.21.15.38:0/3744604442 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555b21dea190 0x555b21e8ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:13.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.200+0000 7fc69e415640 1 -- 172.21.15.38:0/3744604442 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555b21dea190 msgr2=0x555b21e8ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:13.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.200+0000 7fc69e415640 1 --2- 172.21.15.38:0/3744604442 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555b21dea190 0x555b21e8ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:13.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.200+0000 7fc69e415640 1 -- 172.21.15.38:0/3744604442 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555b21e07090 msgr2=0x555b21de8840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:13.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.200+0000 7fc69e415640 1 --2- 172.21.15.38:0/3744604442 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555b21e07090 0x555b21de8840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:13.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.200+0000 7fc69e415640 1 -- 172.21.15.38:0/3744604442 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b21e7bd80 con 0x555b21e7c520 2024-09-21T03:40:13.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.200+0000 7fc69ec16640 1 --2- 172.21.15.38:0/3744604442 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555b21e07090 0x555b21de8840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:40:13.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.201+0000 7fc69e415640 1 --2- 172.21.15.38:0/3744604442 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555b21e7c520 0x555b21de9c50 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7fc694001d60 tx=0x7fc69400b5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30f00898f9cc909 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:13.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.201+0000 7fc69d413640 1 -- 172.21.15.38:0/3744604442 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc69400bd80 con 0x555b21e7c520 2024-09-21T03:40:13.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.201+0000 7fc69d413640 1 -- 172.21.15.38:0/3744604442 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc69400f400 con 0x555b21e7c520 2024-09-21T03:40:13.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.201+0000 7fc69d413640 1 -- 172.21.15.38:0/3744604442 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc69400f850 con 0x555b21e7c520 2024-09-21T03:40:13.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.201+0000 7fc6a515cf40 1 -- 172.21.15.38:0/3744604442 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555b21e7c520 msgr2=0x555b21de9c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:13.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.201+0000 7fc6a515cf40 1 --2- 172.21.15.38:0/3744604442 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555b21e7c520 0x555b21de9c50 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7fc694001d60 tx=0x7fc69400b5b0 comp rx=0 tx=0).stop 2024-09-21T03:40:13.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.201+0000 7fc6a515cf40 1 -- 172.21.15.38:0/3744604442 shutdown_connections 2024-09-21T03:40:13.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.201+0000 7fc6a515cf40 1 --2- 172.21.15.38:0/3744604442 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555b21dea190 0x555b21e8ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:13.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.201+0000 7fc6a515cf40 1 --2- 172.21.15.38:0/3744604442 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555b21e7c520 0x555b21de9c50 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:13.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.201+0000 7fc6a515cf40 1 --2- 172.21.15.38:0/3744604442 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555b21e07090 0x555b21de8840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:13.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.201+0000 7fc6a515cf40 1 -- 172.21.15.38:0/3744604442 >> 172.21.15.38:0/3744604442 conn(0x555b21cdeac0 msgr2=0x555b21e7af40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:13.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.202+0000 7fc6a515cf40 1 -- 172.21.15.38:0/3744604442 shutdown_connections 2024-09-21T03:40:13.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.202+0000 7fc6a515cf40 1 -- 172.21.15.38:0/3744604442 wait complete. 2024-09-21T03:40:13.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.202+0000 7fc6a515cf40 1 Processor -- start 2024-09-21T03:40:13.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.202+0000 7fc6a515cf40 1 -- start start 2024-09-21T03:40:13.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.204+0000 7fc6a515cf40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555b21dea190 0x555b21e7b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:13.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.204+0000 7fc6a515cf40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555b21e07090 0x555b21de8a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:13.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.204+0000 7fc6a515cf40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555b21e7c520 0x555b21e74aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:13.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.204+0000 7fc6a515cf40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x555b21e76020 con 0x555b21e7c520 2024-09-21T03:40:13.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.204+0000 7fc6a515cf40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x555b21e8d570 con 0x555b21e07090 2024-09-21T03:40:13.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.204+0000 7fc6a515cf40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x555b21dfe0a0 con 0x555b21dea190 2024-09-21T03:40:13.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.204+0000 7fc69f417640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555b21e7c520 0x555b21e74aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:13.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.204+0000 7fc69f417640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555b21e7c520 0x555b21e74aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54606/0 (socket says 172.21.15.38:54606) 2024-09-21T03:40:13.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.204+0000 7fc69f417640 1 -- 172.21.15.38:0/1194608184 learned_addr learned my addr 172.21.15.38:0/1194608184 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:13.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.204+0000 7fc69e415640 1 --2- 172.21.15.38:0/1194608184 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555b21e07090 0x555b21de8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:13.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.204+0000 7fc69ec16640 1 --2- 172.21.15.38:0/1194608184 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555b21dea190 0x555b21e7b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:13.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.204+0000 7fc69f417640 1 -- 172.21.15.38:0/1194608184 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555b21dea190 msgr2=0x555b21e7b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:13.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.204+0000 7fc69f417640 1 --2- 172.21.15.38:0/1194608184 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555b21dea190 0x555b21e7b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:13.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.204+0000 7fc69f417640 1 -- 172.21.15.38:0/1194608184 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555b21e07090 msgr2=0x555b21de8a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:13.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.204+0000 7fc69f417640 1 --2- 172.21.15.38:0/1194608184 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555b21e07090 0x555b21de8a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:13.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.204+0000 7fc69f417640 1 -- 172.21.15.38:0/1194608184 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b21f2a170 con 0x555b21e7c520 2024-09-21T03:40:13.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.204+0000 7fc69f417640 1 --2- 172.21.15.38:0/1194608184 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555b21e7c520 0x555b21e74aa0 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7fc69800a630 tx=0x7fc698004560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:13.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.205+0000 7fc687fff640 1 -- 172.21.15.38:0/1194608184 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc698066030 con 0x555b21e7c520 2024-09-21T03:40:13.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.205+0000 7fc687fff640 1 -- 172.21.15.38:0/1194608184 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc69804d820 con 0x555b21e7c520 2024-09-21T03:40:13.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.205+0000 7fc6a515cf40 1 -- 172.21.15.38:0/1194608184 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x555b21e04730 con 0x555b21e7c520 2024-09-21T03:40:13.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.205+0000 7fc687fff640 1 -- 172.21.15.38:0/1194608184 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc698004020 con 0x555b21e7c520 2024-09-21T03:40:13.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.205+0000 7fc6a515cf40 1 -- 172.21.15.38:0/1194608184 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b21dfe460 con 0x555b21e7c520 2024-09-21T03:40:13.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.206+0000 7fc687fff640 1 -- 172.21.15.38:0/1194608184 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc698053030 con 0x555b21e7c520 2024-09-21T03:40:13.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.207+0000 7fc687fff640 1 --2- 172.21.15.38:0/1194608184 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc68c077570 0x7fc68c079a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:13.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.207+0000 7fc687fff640 1 -- 172.21.15.38:0/1194608184 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6187+0+0 (secure 0 0 0) 0x7fc6980de200 con 0x555b21e7c520 2024-09-21T03:40:13.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.207+0000 7fc69ec16640 1 --2- 172.21.15.38:0/1194608184 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc68c077570 0x7fc68c079a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:13.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.207+0000 7fc6a515cf40 1 --2- 172.21.15.38:0/1194608184 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x555b21f2dbf0 0x555b21f2e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:13.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.207+0000 7fc6a515cf40 1 -- 172.21.15.38:0/1194608184 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:1 13.14 13:2e2fa760:::200.00000001:head [omap-get-keys in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x555b21e76a80 con 0x555b21f2dbf0 2024-09-21T03:40:13.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.207+0000 7fc69e415640 1 --2- 172.21.15.38:0/1194608184 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x555b21f2dbf0 0x555b21f2e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:13.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.207+0000 7fc69ec16640 1 --2- 172.21.15.38:0/1194608184 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc68c077570 0x7fc68c079a80 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fc69000f620 tx=0x7fc690002a90 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:13.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.208+0000 7fc69e415640 1 --2- 172.21.15.38:0/1194608184 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x555b21f2dbf0 0x555b21f2e000 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:13.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.208+0000 7fc69e415640 1 -- 172.21.15.38:0/1194608184 <== osd.6 v2:172.21.15.195:6826/4055825675 1 ==== osd_op_reply(1 200.00000001 [omap-get-keys out=5b] v0'0 uv16 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x7fc69401c5c0 con 0x555b21f2dbf0 2024-09-21T03:40:13.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.208+0000 7fc6a515cf40 1 -- 172.21.15.38:0/1194608184 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x555b21f2dbf0 msgr2=0x555b21f2e000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:13.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.208+0000 7fc6a515cf40 1 --2- 172.21.15.38:0/1194608184 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x555b21f2dbf0 0x555b21f2e000 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:13.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.208+0000 7fc6a515cf40 1 -- 172.21.15.38:0/1194608184 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc68c077570 msgr2=0x7fc68c079a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:13.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.208+0000 7fc6a515cf40 1 --2- 172.21.15.38:0/1194608184 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc68c077570 0x7fc68c079a80 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fc69000f620 tx=0x7fc690002a90 comp rx=0 tx=0).stop 2024-09-21T03:40:13.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.208+0000 7fc6a515cf40 1 -- 172.21.15.38:0/1194608184 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555b21e7c520 msgr2=0x555b21e74aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:13.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.208+0000 7fc6a515cf40 1 --2- 172.21.15.38:0/1194608184 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555b21e7c520 0x555b21e74aa0 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7fc69800a630 tx=0x7fc698004560 comp rx=0 tx=0).stop 2024-09-21T03:40:13.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.208+0000 7fc69f417640 1 -- 172.21.15.38:0/1194608184 reap_dead start 2024-09-21T03:40:13.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.208+0000 7fc6a515cf40 1 -- 172.21.15.38:0/1194608184 shutdown_connections 2024-09-21T03:40:13.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.208+0000 7fc6a515cf40 1 -- 172.21.15.38:0/1194608184 >> 172.21.15.38:0/1194608184 conn(0x555b21cdeac0 msgr2=0x555b21de99a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:13.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.208+0000 7fc6a515cf40 1 -- 172.21.15.38:0/1194608184 shutdown_connections 2024-09-21T03:40:13.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.208+0000 7fc6a515cf40 1 -- 172.21.15.38:0/1194608184 wait complete. 2024-09-21T03:40:13.214 DEBUG:teuthology.orchestra.run.smithi038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata listomapkeys mds0_openfiles.0 2024-09-21T03:40:13.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.269+0000 7f6fffe0ef40 1 Processor -- start 2024-09-21T03:40:13.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.269+0000 7f6fffe0ef40 1 -- start start 2024-09-21T03:40:13.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.270+0000 7f6fffe0ef40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b85f639d00 0x55b85f70aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:13.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.270+0000 7f6fffe0ef40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b85f6f4950 0x55b85f6efee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:13.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.270+0000 7f6fffe0ef40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b85f70f670 0x55b85f70b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:13.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.270+0000 7f6fffe0ef40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55b85f587d60 con 0x55b85f639d00 2024-09-21T03:40:13.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.270+0000 7f6fffe0ef40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55b85f5e6430 con 0x55b85f6f4950 2024-09-21T03:40:13.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.270+0000 7f6fffe0ef40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55b85f6817b0 con 0x55b85f70f670 2024-09-21T03:40:13.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.270+0000 7f6ff8c79640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b85f639d00 0x55b85f70aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:13.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.270+0000 7f6ff8c79640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b85f639d00 0x55b85f70aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54622/0 (socket says 172.21.15.38:54622) 2024-09-21T03:40:13.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.270+0000 7f6ff8c79640 1 -- 172.21.15.38:0/4267447459 learned_addr learned my addr 172.21.15.38:0/4267447459 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:13.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.270+0000 7f6ff3fff640 1 --2- 172.21.15.38:0/4267447459 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b85f6f4950 0x55b85f6efee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:13.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.270+0000 7f6ff947a640 1 --2- 172.21.15.38:0/4267447459 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b85f70f670 0x55b85f70b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:13.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.270+0000 7f6ff8c79640 1 -- 172.21.15.38:0/4267447459 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b85f70f670 msgr2=0x55b85f70b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:13.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.270+0000 7f6ff8c79640 1 --2- 172.21.15.38:0/4267447459 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b85f70f670 0x55b85f70b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:13.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.270+0000 7f6ff8c79640 1 -- 172.21.15.38:0/4267447459 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b85f6f4950 msgr2=0x55b85f6efee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:13.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.270+0000 7f6ff8c79640 1 --2- 172.21.15.38:0/4267447459 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b85f6f4950 0x55b85f6efee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:13.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.270+0000 7f6ff8c79640 1 -- 172.21.15.38:0/4267447459 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b85f666000 con 0x55b85f639d00 2024-09-21T03:40:13.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.271+0000 7f6ff8c79640 1 --2- 172.21.15.38:0/4267447459 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b85f639d00 0x55b85f70aa10 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f6fec00b7c0 tx=0x7f6fec00bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b09d9acfb3e6ac server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:13.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.271+0000 7f6ff2ffd640 1 -- 172.21.15.38:0/4267447459 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fec00ab40 con 0x55b85f639d00 2024-09-21T03:40:13.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.271+0000 7f6ff2ffd640 1 -- 172.21.15.38:0/4267447459 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6fec00ace0 con 0x55b85f639d00 2024-09-21T03:40:13.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.271+0000 7f6ff2ffd640 1 -- 172.21.15.38:0/4267447459 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fec015680 con 0x55b85f639d00 2024-09-21T03:40:13.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.272+0000 7f6fffe0ef40 1 -- 172.21.15.38:0/4267447459 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b85f639d00 msgr2=0x55b85f70aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:13.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.272+0000 7f6fffe0ef40 1 --2- 172.21.15.38:0/4267447459 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b85f639d00 0x55b85f70aa10 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f6fec00b7c0 tx=0x7f6fec00bc80 comp rx=0 tx=0).stop 2024-09-21T03:40:13.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.272+0000 7f6fffe0ef40 1 -- 172.21.15.38:0/4267447459 shutdown_connections 2024-09-21T03:40:13.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.272+0000 7f6fffe0ef40 1 --2- 172.21.15.38:0/4267447459 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b85f70f670 0x55b85f70b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:13.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.272+0000 7f6fffe0ef40 1 --2- 172.21.15.38:0/4267447459 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b85f6f4950 0x55b85f6efee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:13.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.272+0000 7f6fffe0ef40 1 --2- 172.21.15.38:0/4267447459 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b85f639d00 0x55b85f70aa10 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:13.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.272+0000 7f6fffe0ef40 1 -- 172.21.15.38:0/4267447459 >> 172.21.15.38:0/4267447459 conn(0x55b85f558ac0 msgr2=0x55b85f70fa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:13.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.272+0000 7f6fffe0ef40 1 -- 172.21.15.38:0/4267447459 shutdown_connections 2024-09-21T03:40:13.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.272+0000 7f6fffe0ef40 1 -- 172.21.15.38:0/4267447459 wait complete. 2024-09-21T03:40:13.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.273+0000 7f6fffe0ef40 1 Processor -- start 2024-09-21T03:40:13.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.273+0000 7f6fffe0ef40 1 -- start start 2024-09-21T03:40:13.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.274+0000 7f6fffe0ef40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b85f639d00 0x55b85f6ece40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:13.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.274+0000 7f6fffe0ef40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b85f6f4950 0x55b85f635be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:13.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.274+0000 7f6fffe0ef40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b85f70f670 0x55b85f636120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:13.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.274+0000 7f6fffe0ef40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55b85f6662a0 con 0x55b85f639d00 2024-09-21T03:40:13.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.274+0000 7f6fffe0ef40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55b85f6681d0 con 0x55b85f6f4950 2024-09-21T03:40:13.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.274+0000 7f6fffe0ef40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55b85f7064c0 con 0x55b85f70f670 2024-09-21T03:40:13.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.274+0000 7f6ff8c79640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b85f639d00 0x55b85f6ece40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:13.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.274+0000 7f6ff8c79640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b85f639d00 0x55b85f6ece40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54624/0 (socket says 172.21.15.38:54624) 2024-09-21T03:40:13.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.274+0000 7f6ff8c79640 1 -- 172.21.15.38:0/4072514414 learned_addr learned my addr 172.21.15.38:0/4072514414 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:13.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.274+0000 7f6ff3fff640 1 --2- 172.21.15.38:0/4072514414 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b85f6f4950 0x55b85f635be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:13.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.274+0000 7f6ff947a640 1 --2- 172.21.15.38:0/4072514414 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b85f70f670 0x55b85f636120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:13.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.275+0000 7f6ff8c79640 1 -- 172.21.15.38:0/4072514414 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b85f70f670 msgr2=0x55b85f636120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:13.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.275+0000 7f6ff8c79640 1 --2- 172.21.15.38:0/4072514414 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b85f70f670 0x55b85f636120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:13.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.275+0000 7f6ff8c79640 1 -- 172.21.15.38:0/4072514414 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b85f6f4950 msgr2=0x55b85f635be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:13.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.275+0000 7f6ff8c79640 1 --2- 172.21.15.38:0/4072514414 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b85f6f4950 0x55b85f635be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:13.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.275+0000 7f6ff8c79640 1 -- 172.21.15.38:0/4072514414 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b85f6ee160 con 0x55b85f639d00 2024-09-21T03:40:13.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.275+0000 7f6ff3fff640 1 --2- 172.21.15.38:0/4072514414 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b85f6f4950 0x55b85f635be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:40:13.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.275+0000 7f6ff8c79640 1 --2- 172.21.15.38:0/4072514414 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b85f639d00 0x55b85f6ece40 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x55b85f6ef8d0 tx=0x7f6fec015650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:13.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.275+0000 7f6ff1ffb640 1 -- 172.21.15.38:0/4072514414 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fec023070 con 0x55b85f639d00 2024-09-21T03:40:13.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.275+0000 7f6ff1ffb640 1 -- 172.21.15.38:0/4072514414 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6fec015a50 con 0x55b85f639d00 2024-09-21T03:40:13.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.275+0000 7f6fffe0ef40 1 -- 172.21.15.38:0/4072514414 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55b85f70a040 con 0x55b85f639d00 2024-09-21T03:40:13.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.275+0000 7f6ff1ffb640 1 -- 172.21.15.38:0/4072514414 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fec01e410 con 0x55b85f639d00 2024-09-21T03:40:13.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.275+0000 7f6fffe0ef40 1 -- 172.21.15.38:0/4072514414 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b85f6ee6b0 con 0x55b85f639d00 2024-09-21T03:40:13.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.277+0000 7f6ff1ffb640 1 -- 172.21.15.38:0/4072514414 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f6fec026070 con 0x55b85f639d00 2024-09-21T03:40:13.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.278+0000 7f6ff1ffb640 1 --2- 172.21.15.38:0/4072514414 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6fdc0775a0 0x7f6fdc079ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:13.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.278+0000 7f6ff1ffb640 1 -- 172.21.15.38:0/4072514414 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6187+0+0 (secure 0 0 0) 0x7f6fec09ab30 con 0x55b85f639d00 2024-09-21T03:40:13.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.278+0000 7f6ff3fff640 1 --2- 172.21.15.38:0/4072514414 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6fdc0775a0 0x7f6fdc079ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:13.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.278+0000 7f6fffe0ef40 1 --2- 172.21.15.38:0/4072514414 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x55b85f708dd0 0x55b85f7091e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:13.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.278+0000 7f6fffe0ef40 1 -- 172.21.15.38:0/4072514414 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:1 13.6 13:654134d2:::mds0_openfiles.0:head [omap-get-keys in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x55b85f709720 con 0x55b85f708dd0 2024-09-21T03:40:13.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.278+0000 7f6ff947a640 1 --2- 172.21.15.38:0/4072514414 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x55b85f708dd0 0x55b85f7091e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:13.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.279+0000 7f6ff3fff640 1 --2- 172.21.15.38:0/4072514414 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6fdc0775a0 0x7f6fdc079ab0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f6fe4055000 tx=0x7f6fe4003020 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:13.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.279+0000 7f6ff947a640 1 --2- 172.21.15.38:0/4072514414 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x55b85f708dd0 0x55b85f7091e0 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:13.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.279+0000 7f6ff947a640 1 -- 172.21.15.38:0/4072514414 <== osd.3 v2:172.21.15.38:6808/1286810294 1 ==== osd_op_reply(1 mds0_openfiles.0 [omap-get-keys out=5b] v0'0 uv2 ondisk = 0) ==== 160+0+5 (crc 0 0 0) 0x7f6ff4002880 con 0x55b85f708dd0 2024-09-21T03:40:13.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.279+0000 7f6fffe0ef40 1 -- 172.21.15.38:0/4072514414 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x55b85f708dd0 msgr2=0x55b85f7091e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:13.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.279+0000 7f6fffe0ef40 1 --2- 172.21.15.38:0/4072514414 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x55b85f708dd0 0x55b85f7091e0 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:13.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.279+0000 7f6fffe0ef40 1 -- 172.21.15.38:0/4072514414 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6fdc0775a0 msgr2=0x7f6fdc079ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:13.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.279+0000 7f6fffe0ef40 1 --2- 172.21.15.38:0/4072514414 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6fdc0775a0 0x7f6fdc079ab0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f6fe4055000 tx=0x7f6fe4003020 comp rx=0 tx=0).stop 2024-09-21T03:40:13.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.279+0000 7f6fffe0ef40 1 -- 172.21.15.38:0/4072514414 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b85f639d00 msgr2=0x55b85f6ece40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:13.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.279+0000 7f6fffe0ef40 1 --2- 172.21.15.38:0/4072514414 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b85f639d00 0x55b85f6ece40 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x55b85f6ef8d0 tx=0x7f6fec015650 comp rx=0 tx=0).stop 2024-09-21T03:40:13.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.279+0000 7f6ff947a640 1 -- 172.21.15.38:0/4072514414 reap_dead start 2024-09-21T03:40:13.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.279+0000 7f6fffe0ef40 1 -- 172.21.15.38:0/4072514414 shutdown_connections 2024-09-21T03:40:13.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.279+0000 7f6fffe0ef40 1 -- 172.21.15.38:0/4072514414 >> 172.21.15.38:0/4072514414 conn(0x55b85f558ac0 msgr2=0x55b85f70fa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:13.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.279+0000 7f6fffe0ef40 1 -- 172.21.15.38:0/4072514414 shutdown_connections 2024-09-21T03:40:13.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.279+0000 7f6fffe0ef40 1 -- 172.21.15.38:0/4072514414 wait complete. 2024-09-21T03:40:13.285 DEBUG:teuthology.orchestra.run.smithi038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata listomapkeys 500.00000000 2024-09-21T03:40:13.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.337+0000 7f22cb22bf40 1 Processor -- start 2024-09-21T03:40:13.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.337+0000 7f22cb22bf40 1 -- start start 2024-09-21T03:40:13.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.338+0000 7f22cb22bf40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55ac330da090 0x55ac330c01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:13.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.338+0000 7f22cb22bf40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ac3314f520 0x55ac330bbed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:13.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.338+0000 7f22cb22bf40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ac330d7460 0x55ac330bc410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:13.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.338+0000 7f22cb22bf40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55ac32fe0d60 con 0x55ac330d7460 2024-09-21T03:40:13.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.338+0000 7f22cb22bf40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55ac3303f430 con 0x55ac3314f520 2024-09-21T03:40:13.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.338+0000 7f22cb22bf40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55ac33167350 con 0x55ac330da090 2024-09-21T03:40:13.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.339+0000 7f22c54e6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ac330d7460 0x55ac330bc410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:13.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.339+0000 7f22c54e6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ac330d7460 0x55ac330bc410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54636/0 (socket says 172.21.15.38:54636) 2024-09-21T03:40:13.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.339+0000 7f22c54e6640 1 -- 172.21.15.38:0/3075331727 learned_addr learned my addr 172.21.15.38:0/3075331727 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:13.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.339+0000 7f22bffff640 1 --2- 172.21.15.38:0/3075331727 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ac3314f520 0x55ac330bbed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:13.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.339+0000 7f22c4ce5640 1 --2- 172.21.15.38:0/3075331727 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55ac330da090 0x55ac330c01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:13.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.339+0000 7f22c54e6640 1 -- 172.21.15.38:0/3075331727 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55ac330da090 msgr2=0x55ac330c01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:13.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.339+0000 7f22c54e6640 1 --2- 172.21.15.38:0/3075331727 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55ac330da090 0x55ac330c01b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:13.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.339+0000 7f22c54e6640 1 -- 172.21.15.38:0/3075331727 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ac3314f520 msgr2=0x55ac330bbed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:13.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.339+0000 7f22c54e6640 1 --2- 172.21.15.38:0/3075331727 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ac3314f520 0x55ac330bbed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:13.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.339+0000 7f22c54e6640 1 -- 172.21.15.38:0/3075331727 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac33160500 con 0x55ac330d7460 2024-09-21T03:40:13.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.339+0000 7f22c54e6640 1 --2- 172.21.15.38:0/3075331727 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ac330d7460 0x55ac330bc410 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f22c0025e30 tx=0x7f22c0003330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a04b8410fa83ac8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:13.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.339+0000 7f22beffd640 1 -- 172.21.15.38:0/3075331727 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22c0064030 con 0x55ac330d7460 2024-09-21T03:40:13.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.339+0000 7f22beffd640 1 -- 172.21.15.38:0/3075331727 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f22c0004020 con 0x55ac330d7460 2024-09-21T03:40:13.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.339+0000 7f22beffd640 1 -- 172.21.15.38:0/3075331727 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22c0004270 con 0x55ac330d7460 2024-09-21T03:40:13.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.340+0000 7f22cb22bf40 1 -- 172.21.15.38:0/3075331727 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ac330d7460 msgr2=0x55ac330bc410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:13.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.340+0000 7f22cb22bf40 1 --2- 172.21.15.38:0/3075331727 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ac330d7460 0x55ac330bc410 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f22c0025e30 tx=0x7f22c0003330 comp rx=0 tx=0).stop 2024-09-21T03:40:13.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.340+0000 7f22cb22bf40 1 -- 172.21.15.38:0/3075331727 shutdown_connections 2024-09-21T03:40:13.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.340+0000 7f22cb22bf40 1 --2- 172.21.15.38:0/3075331727 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ac330d7460 0x55ac330bc410 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:13.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.340+0000 7f22cb22bf40 1 --2- 172.21.15.38:0/3075331727 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ac3314f520 0x55ac330bbed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:13.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.340+0000 7f22cb22bf40 1 --2- 172.21.15.38:0/3075331727 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55ac330da090 0x55ac330c01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:13.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.340+0000 7f22cb22bf40 1 -- 172.21.15.38:0/3075331727 >> 172.21.15.38:0/3075331727 conn(0x55ac32fb1ac0 msgr2=0x55ac330d87f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:13.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.340+0000 7f22cb22bf40 1 -- 172.21.15.38:0/3075331727 shutdown_connections 2024-09-21T03:40:13.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.340+0000 7f22cb22bf40 1 -- 172.21.15.38:0/3075331727 wait complete. 2024-09-21T03:40:13.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.341+0000 7f22cb22bf40 1 Processor -- start 2024-09-21T03:40:13.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.341+0000 7f22cb22bf40 1 -- start start 2024-09-21T03:40:13.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.342+0000 7f22cb22bf40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55ac330d7460 0x55ac33148d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:13.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.342+0000 7f22cb22bf40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ac330da090 0x55ac330bbca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:13.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.342+0000 7f22cb22bf40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ac3314f520 0x55ac330d87f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:13.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.342+0000 7f22cb22bf40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55ac331609c0 con 0x55ac3314f520 2024-09-21T03:40:13.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.342+0000 7f22cb22bf40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55ac330bc950 con 0x55ac330da090 2024-09-21T03:40:13.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.342+0000 7f22cb22bf40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55ac330d7950 con 0x55ac330d7460 2024-09-21T03:40:13.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.342+0000 7f22c54e6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ac3314f520 0x55ac330d87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:13.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.342+0000 7f22c54e6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ac3314f520 0x55ac330d87f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54642/0 (socket says 172.21.15.38:54642) 2024-09-21T03:40:13.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.342+0000 7f22c54e6640 1 -- 172.21.15.38:0/1986921282 learned_addr learned my addr 172.21.15.38:0/1986921282 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:13.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.342+0000 7f22c4ce5640 1 --2- 172.21.15.38:0/1986921282 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55ac330d7460 0x55ac33148d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:13.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.342+0000 7f22bffff640 1 --2- 172.21.15.38:0/1986921282 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ac330da090 0x55ac330bbca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:13.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.343+0000 7f22c54e6640 1 -- 172.21.15.38:0/1986921282 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55ac330d7460 msgr2=0x55ac33148d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:13.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.343+0000 7f22c54e6640 1 --2- 172.21.15.38:0/1986921282 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55ac330d7460 0x55ac33148d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:13.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.343+0000 7f22c54e6640 1 -- 172.21.15.38:0/1986921282 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ac330da090 msgr2=0x55ac330bbca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:13.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.343+0000 7f22c54e6640 1 --2- 172.21.15.38:0/1986921282 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ac330da090 0x55ac330bbca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:13.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.343+0000 7f22c54e6640 1 -- 172.21.15.38:0/1986921282 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac3314d270 con 0x55ac3314f520 2024-09-21T03:40:13.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.343+0000 7f22c54e6640 1 --2- 172.21.15.38:0/1986921282 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ac3314f520 0x55ac330d87f0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f22c0063040 tx=0x7f22c004db20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:13.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.343+0000 7f22bdffb640 1 -- 172.21.15.38:0/1986921282 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22c006d070 con 0x55ac3314f520 2024-09-21T03:40:13.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.343+0000 7f22bdffb640 1 -- 172.21.15.38:0/1986921282 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f22c0060560 con 0x55ac3314f520 2024-09-21T03:40:13.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.343+0000 7f22cb22bf40 1 -- 172.21.15.38:0/1986921282 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55ac33149be0 con 0x55ac3314f520 2024-09-21T03:40:13.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.343+0000 7f22bdffb640 1 -- 172.21.15.38:0/1986921282 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22c00609b0 con 0x55ac3314f520 2024-09-21T03:40:13.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.343+0000 7f22cb22bf40 1 -- 172.21.15.38:0/1986921282 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac3314a1a0 con 0x55ac3314f520 2024-09-21T03:40:13.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.345+0000 7f22bdffb640 1 -- 172.21.15.38:0/1986921282 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f22c0070070 con 0x55ac3314f520 2024-09-21T03:40:13.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.345+0000 7f22bdffb640 1 --2- 172.21.15.38:0/1986921282 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f22a8077570 0x7f22a8079a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:13.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.345+0000 7f22bdffb640 1 -- 172.21.15.38:0/1986921282 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6187+0+0 (secure 0 0 0) 0x7f22c00def70 con 0x55ac3314f520 2024-09-21T03:40:13.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.345+0000 7f22c4ce5640 1 --2- 172.21.15.38:0/1986921282 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f22a8077570 0x7f22a8079a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:13.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.346+0000 7f22cb22bf40 1 --2- 172.21.15.38:0/1986921282 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x55ac33203070 0x55ac33203480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:13.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.346+0000 7f22cb22bf40 1 -- 172.21.15.38:0/1986921282 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:1 13.15 13:aa448500:::500.00000000:head [omap-get-keys in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x55ac3320bc60 con 0x55ac33203070 2024-09-21T03:40:13.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.346+0000 7f22bffff640 1 --2- 172.21.15.38:0/1986921282 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x55ac33203070 0x55ac33203480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:13.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.346+0000 7f22c4ce5640 1 --2- 172.21.15.38:0/1986921282 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f22a8077570 0x7f22a8079a80 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f22b800c780 tx=0x7f22b800f040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:13.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.346+0000 7f22bffff640 1 --2- 172.21.15.38:0/1986921282 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x55ac33203070 0x55ac33203480 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:13.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.346+0000 7f22bffff640 1 -- 172.21.15.38:0/1986921282 <== osd.3 v2:172.21.15.38:6808/1286810294 1 ==== osd_op_reply(1 500.00000000 [omap-get-keys out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x7f22b0001990 con 0x55ac33203070 2024-09-21T03:40:13.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.346+0000 7f22cb22bf40 1 -- 172.21.15.38:0/1986921282 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x55ac33203070 msgr2=0x55ac33203480 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:13.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.346+0000 7f22cb22bf40 1 --2- 172.21.15.38:0/1986921282 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x55ac33203070 0x55ac33203480 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:13.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.347+0000 7f22cb22bf40 1 -- 172.21.15.38:0/1986921282 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f22a8077570 msgr2=0x7f22a8079a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:13.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.347+0000 7f22cb22bf40 1 --2- 172.21.15.38:0/1986921282 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f22a8077570 0x7f22a8079a80 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f22b800c780 tx=0x7f22b800f040 comp rx=0 tx=0).stop 2024-09-21T03:40:13.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.347+0000 7f22cb22bf40 1 -- 172.21.15.38:0/1986921282 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ac3314f520 msgr2=0x55ac330d87f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:13.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.347+0000 7f22cb22bf40 1 --2- 172.21.15.38:0/1986921282 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ac3314f520 0x55ac330d87f0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f22c0063040 tx=0x7f22c004db20 comp rx=0 tx=0).stop 2024-09-21T03:40:13.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.347+0000 7f22c54e6640 1 -- 172.21.15.38:0/1986921282 reap_dead start 2024-09-21T03:40:13.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.347+0000 7f22cb22bf40 1 -- 172.21.15.38:0/1986921282 shutdown_connections 2024-09-21T03:40:13.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.347+0000 7f22cb22bf40 1 -- 172.21.15.38:0/1986921282 >> 172.21.15.38:0/1986921282 conn(0x55ac32fb1ac0 msgr2=0x55ac330bb460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:13.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.347+0000 7f22cb22bf40 1 -- 172.21.15.38:0/1986921282 shutdown_connections 2024-09-21T03:40:13.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.347+0000 7f22cb22bf40 1 -- 172.21.15.38:0/1986921282 wait complete. 2024-09-21T03:40:13.352 DEBUG:teuthology.orchestra.run.smithi038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata listomapkeys mds_snaptable 2024-09-21T03:40:13.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.406+0000 7fe05d6e8f40 1 Processor -- start 2024-09-21T03:40:13.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.406+0000 7fe05d6e8f40 1 -- start start 2024-09-21T03:40:13.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.407+0000 7fe05d6e8f40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f8453abd00 0x55f8453d9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:13.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.407+0000 7fe05d6e8f40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f845466950 0x55f845486da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:13.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.407+0000 7fe05d6e8f40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f845481670 0x55f8453d7da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:13.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.407+0000 7fe05d6e8f40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55f8453f4460 con 0x55f8453abd00 2024-09-21T03:40:13.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.407+0000 7fe05d6e8f40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55f8452f9d60 con 0x55f845466950 2024-09-21T03:40:13.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.407+0000 7fe05d6e8f40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55f8453f37b0 con 0x55f845481670 2024-09-21T03:40:13.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.407+0000 7fe0571a2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f8453abd00 0x55f8453d9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:13.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.407+0000 7fe0571a2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f8453abd00 0x55f8453d9cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54672/0 (socket says 172.21.15.38:54672) 2024-09-21T03:40:13.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.407+0000 7fe0571a2640 1 -- 172.21.15.38:0/2421186769 learned_addr learned my addr 172.21.15.38:0/2421186769 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:13.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.407+0000 7fe0569a1640 1 --2- 172.21.15.38:0/2421186769 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f845466950 0x55f845486da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:13.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.407+0000 7fe0579a3640 1 --2- 172.21.15.38:0/2421186769 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f845481670 0x55f8453d7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:13.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.407+0000 7fe0571a2640 1 -- 172.21.15.38:0/2421186769 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f845481670 msgr2=0x55f8453d7da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:13.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.407+0000 7fe0571a2640 1 --2- 172.21.15.38:0/2421186769 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f845481670 0x55f8453d7da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:13.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.407+0000 7fe0571a2640 1 -- 172.21.15.38:0/2421186769 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f845466950 msgr2=0x55f845486da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:13.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.407+0000 7fe0571a2640 1 --2- 172.21.15.38:0/2421186769 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f845466950 0x55f845486da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:13.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.407+0000 7fe0571a2640 1 -- 172.21.15.38:0/2421186769 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f8453d4290 con 0x55f8453abd00 2024-09-21T03:40:13.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.407+0000 7fe0569a1640 1 --2- 172.21.15.38:0/2421186769 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f845466950 0x55f845486da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:40:13.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.408+0000 7fe0571a2640 1 --2- 172.21.15.38:0/2421186769 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f8453abd00 0x55f8453d9cf0 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7fe04800b600 tx=0x7fe04800bac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fd91d172ea78cbe server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:13.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.408+0000 7fe05599f640 1 -- 172.21.15.38:0/2421186769 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe04800a8f0 con 0x55f8453abd00 2024-09-21T03:40:13.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.408+0000 7fe05599f640 1 -- 172.21.15.38:0/2421186769 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe04800aa90 con 0x55f8453abd00 2024-09-21T03:40:13.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.408+0000 7fe05599f640 1 -- 172.21.15.38:0/2421186769 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe048008450 con 0x55f8453abd00 2024-09-21T03:40:13.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.408+0000 7fe05d6e8f40 1 -- 172.21.15.38:0/2421186769 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f8453abd00 msgr2=0x55f8453d9cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:13.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.408+0000 7fe05d6e8f40 1 --2- 172.21.15.38:0/2421186769 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f8453abd00 0x55f8453d9cf0 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7fe04800b600 tx=0x7fe04800bac0 comp rx=0 tx=0).stop 2024-09-21T03:40:13.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.408+0000 7fe05d6e8f40 1 -- 172.21.15.38:0/2421186769 shutdown_connections 2024-09-21T03:40:13.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.408+0000 7fe05d6e8f40 1 --2- 172.21.15.38:0/2421186769 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f845481670 0x55f8453d7da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:13.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.408+0000 7fe05d6e8f40 1 --2- 172.21.15.38:0/2421186769 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f845466950 0x55f845486da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:13.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.408+0000 7fe05d6e8f40 1 --2- 172.21.15.38:0/2421186769 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f8453abd00 0x55f8453d9cf0 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:13.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.408+0000 7fe05d6e8f40 1 -- 172.21.15.38:0/2421186769 >> 172.21.15.38:0/2421186769 conn(0x55f8452caac0 msgr2=0x55f845481a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:13.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.408+0000 7fe05d6e8f40 1 -- 172.21.15.38:0/2421186769 shutdown_connections 2024-09-21T03:40:13.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.408+0000 7fe05d6e8f40 1 -- 172.21.15.38:0/2421186769 wait complete. 2024-09-21T03:40:13.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.409+0000 7fe05d6e8f40 1 Processor -- start 2024-09-21T03:40:13.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.409+0000 7fe05d6e8f40 1 -- start start 2024-09-21T03:40:13.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.411+0000 7fe05d6e8f40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f8453abd00 0x55f845517d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:13.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.411+0000 7fe05d6e8f40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f845466950 0x55f8453d7e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:13.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.411+0000 7fe05d6e8f40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f845481670 0x55f845481a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:13.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.411+0000 7fe05d6e8f40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55f845487eb0 con 0x55f845466950 2024-09-21T03:40:13.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.411+0000 7fe05d6e8f40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55f845488100 con 0x55f845481670 2024-09-21T03:40:13.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.411+0000 7fe05d6e8f40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55f845468f60 con 0x55f8453abd00 2024-09-21T03:40:13.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.411+0000 7fe0569a1640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f845466950 0x55f8453d7e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:13.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.411+0000 7fe0569a1640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f845466950 0x55f8453d7e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54674/0 (socket says 172.21.15.38:54674) 2024-09-21T03:40:13.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.411+0000 7fe0569a1640 1 -- 172.21.15.38:0/3113110965 learned_addr learned my addr 172.21.15.38:0/3113110965 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:13.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.411+0000 7fe0579a3640 1 --2- 172.21.15.38:0/3113110965 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f845481670 0x55f845481a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:13.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.411+0000 7fe0571a2640 1 --2- 172.21.15.38:0/3113110965 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f8453abd00 0x55f845517d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:13.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.411+0000 7fe0569a1640 1 -- 172.21.15.38:0/3113110965 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f8453abd00 msgr2=0x55f845517d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:13.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.411+0000 7fe0569a1640 1 --2- 172.21.15.38:0/3113110965 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f8453abd00 0x55f845517d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:13.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.411+0000 7fe0569a1640 1 -- 172.21.15.38:0/3113110965 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f845481670 msgr2=0x55f845481a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:13.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.411+0000 7fe0569a1640 1 --2- 172.21.15.38:0/3113110965 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f845481670 0x55f845481a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:13.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.411+0000 7fe0569a1640 1 -- 172.21.15.38:0/3113110965 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f84545acf0 con 0x55f845466950 2024-09-21T03:40:13.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.411+0000 7fe0569a1640 1 --2- 172.21.15.38:0/3113110965 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f845466950 0x55f8453d7e50 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7fe04c00a680 tx=0x7fe04c00ab40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:13.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.411+0000 7fe05499d640 1 -- 172.21.15.38:0/3113110965 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe04c008990 con 0x55f845466950 2024-09-21T03:40:13.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.411+0000 7fe05499d640 1 -- 172.21.15.38:0/3113110965 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe04c008b30 con 0x55f845466950 2024-09-21T03:40:13.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.411+0000 7fe05499d640 1 -- 172.21.15.38:0/3113110965 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe04c016420 con 0x55f845466950 2024-09-21T03:40:13.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.411+0000 7fe05d6e8f40 1 -- 172.21.15.38:0/3113110965 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55f84545d160 con 0x55f845466950 2024-09-21T03:40:13.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.412+0000 7fe05d6e8f40 1 -- 172.21.15.38:0/3113110965 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f84545d720 con 0x55f845466950 2024-09-21T03:40:13.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.413+0000 7fe05499d640 1 -- 172.21.15.38:0/3113110965 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fe04c01f440 con 0x55f845466950 2024-09-21T03:40:13.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.414+0000 7fe05499d640 1 --2- 172.21.15.38:0/3113110965 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fe044077570 0x7fe044079a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:13.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.414+0000 7fe05499d640 1 -- 172.21.15.38:0/3113110965 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6187+0+0 (secure 0 0 0) 0x7fe04c099c40 con 0x55f845466950 2024-09-21T03:40:13.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.414+0000 7fe0571a2640 1 --2- 172.21.15.38:0/3113110965 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fe044077570 0x7fe044079a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:13.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.414+0000 7fe05d6e8f40 1 --2- 172.21.15.38:0/3113110965 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x55f8453d8390 0x55f845468130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:13.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.414+0000 7fe05d6e8f40 1 -- 172.21.15.38:0/3113110965 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:1 13.d 13:b50e409b:::mds_snaptable:head [omap-get-keys in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x55f845468670 con 0x55f8453d8390 2024-09-21T03:40:13.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.414+0000 7fe0579a3640 1 --2- 172.21.15.38:0/3113110965 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x55f8453d8390 0x55f845468130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:13.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.414+0000 7fe0571a2640 1 --2- 172.21.15.38:0/3113110965 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fe044077570 0x7fe044079a80 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fe048000c00 tx=0x7fe048018040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:13.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.414+0000 7fe0579a3640 1 --2- 172.21.15.38:0/3113110965 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x55f8453d8390 0x55f845468130 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:13.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.415+0000 7fe0579a3640 1 -- 172.21.15.38:0/3113110965 <== osd.0 v2:172.21.15.38:6800/2381639433 1 ==== osd_op_reply(1 mds_snaptable [omap-get-keys out=5b] v0'0 uv1 ondisk = 0) ==== 157+0+5 (crc 0 0 0) 0x7fe05000b040 con 0x55f8453d8390 2024-09-21T03:40:13.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.415+0000 7fe05d6e8f40 1 -- 172.21.15.38:0/3113110965 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x55f8453d8390 msgr2=0x55f845468130 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:13.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.415+0000 7fe05d6e8f40 1 --2- 172.21.15.38:0/3113110965 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x55f8453d8390 0x55f845468130 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:13.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.415+0000 7fe05d6e8f40 1 -- 172.21.15.38:0/3113110965 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fe044077570 msgr2=0x7fe044079a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:13.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.415+0000 7fe05d6e8f40 1 --2- 172.21.15.38:0/3113110965 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fe044077570 0x7fe044079a80 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fe048000c00 tx=0x7fe048018040 comp rx=0 tx=0).stop 2024-09-21T03:40:13.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.415+0000 7fe05d6e8f40 1 -- 172.21.15.38:0/3113110965 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f845466950 msgr2=0x55f8453d7e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:13.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.415+0000 7fe05d6e8f40 1 --2- 172.21.15.38:0/3113110965 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f845466950 0x55f8453d7e50 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7fe04c00a680 tx=0x7fe04c00ab40 comp rx=0 tx=0).stop 2024-09-21T03:40:13.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.415+0000 7fe0579a3640 1 -- 172.21.15.38:0/3113110965 reap_dead start 2024-09-21T03:40:13.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.415+0000 7fe05d6e8f40 1 -- 172.21.15.38:0/3113110965 shutdown_connections 2024-09-21T03:40:13.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.415+0000 7fe05d6e8f40 1 -- 172.21.15.38:0/3113110965 >> 172.21.15.38:0/3113110965 conn(0x55f8452caac0 msgr2=0x55f8454625e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:13.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.415+0000 7fe05d6e8f40 1 -- 172.21.15.38:0/3113110965 shutdown_connections 2024-09-21T03:40:13.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.415+0000 7fe05d6e8f40 1 -- 172.21.15.38:0/3113110965 wait complete. 2024-09-21T03:40:13.421 DEBUG:teuthology.orchestra.run.smithi038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata listomapkeys mds0_inotable 2024-09-21T03:40:13.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.474+0000 7f7e4b281f40 1 Processor -- start 2024-09-21T03:40:13.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.474+0000 7f7e4b281f40 1 -- start start 2024-09-21T03:40:13.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.475+0000 7f7e4b281f40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5638ece4e240 0x5638ecf22a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:13.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.475+0000 7f7e4b281f40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5638ece95090 0x5638ece76a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:13.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.475+0000 7f7e4b281f40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5638ecf0a520 0x5638ecfa9e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:13.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.475+0000 7f7e4b281f40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5638ecd9bd60 con 0x5638ece4e240 2024-09-21T03:40:13.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.475+0000 7f7e4b281f40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5638ecdfa430 con 0x5638ece95090 2024-09-21T03:40:13.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.475+0000 7f7e4b281f40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x5638ecf22350 con 0x5638ecf0a520 2024-09-21T03:40:13.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.475+0000 7f7e44d3d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5638ece4e240 0x5638ecf22a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:13.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.475+0000 7f7e44d3d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5638ece4e240 0x5638ecf22a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54676/0 (socket says 172.21.15.38:54676) 2024-09-21T03:40:13.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.475+0000 7f7e44d3d640 1 -- 172.21.15.38:0/953567194 learned_addr learned my addr 172.21.15.38:0/953567194 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:13.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.476+0000 7f7e4453c640 1 --2- 172.21.15.38:0/953567194 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5638ece95090 0x5638ece76a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:13.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.476+0000 7f7e4553e640 1 --2- 172.21.15.38:0/953567194 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5638ecf0a520 0x5638ecfa9e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:13.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.476+0000 7f7e44d3d640 1 -- 172.21.15.38:0/953567194 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5638ecf0a520 msgr2=0x5638ecfa9e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:13.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.476+0000 7f7e44d3d640 1 --2- 172.21.15.38:0/953567194 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5638ecf0a520 0x5638ecfa9e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:13.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.476+0000 7f7e44d3d640 1 -- 172.21.15.38:0/953567194 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5638ece95090 msgr2=0x5638ece76a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:13.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.476+0000 7f7e44d3d640 1 --2- 172.21.15.38:0/953567194 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5638ece95090 0x5638ece76a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:13.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.476+0000 7f7e44d3d640 1 -- 172.21.15.38:0/953567194 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638ece7c120 con 0x5638ece4e240 2024-09-21T03:40:13.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.476+0000 7f7e4453c640 1 --2- 172.21.15.38:0/953567194 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5638ece95090 0x5638ece76a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:40:13.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.476+0000 7f7e44d3d640 1 --2- 172.21.15.38:0/953567194 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5638ece4e240 0x5638ecf22a10 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f7e34001d60 tx=0x7f7e3400b5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b610af50aa8be85c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:13.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.476+0000 7f7e4353a640 1 -- 172.21.15.38:0/953567194 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e3400bd80 con 0x5638ece4e240 2024-09-21T03:40:13.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.476+0000 7f7e4353a640 1 -- 172.21.15.38:0/953567194 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7e3400f400 con 0x5638ece4e240 2024-09-21T03:40:13.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.476+0000 7f7e4353a640 1 -- 172.21.15.38:0/953567194 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e3400f850 con 0x5638ece4e240 2024-09-21T03:40:13.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.477+0000 7f7e4b281f40 1 -- 172.21.15.38:0/953567194 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5638ece4e240 msgr2=0x5638ecf22a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:13.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.477+0000 7f7e4b281f40 1 --2- 172.21.15.38:0/953567194 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5638ece4e240 0x5638ecf22a10 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f7e34001d60 tx=0x7f7e3400b5b0 comp rx=0 tx=0).stop 2024-09-21T03:40:13.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.477+0000 7f7e4b281f40 1 -- 172.21.15.38:0/953567194 shutdown_connections 2024-09-21T03:40:13.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.477+0000 7f7e4b281f40 1 --2- 172.21.15.38:0/953567194 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5638ecf0a520 0x5638ecfa9e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:13.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.477+0000 7f7e4b281f40 1 --2- 172.21.15.38:0/953567194 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5638ece95090 0x5638ece76a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:13.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.477+0000 7f7e4b281f40 1 --2- 172.21.15.38:0/953567194 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5638ece4e240 0x5638ecf22a10 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:13.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.477+0000 7f7e4b281f40 1 -- 172.21.15.38:0/953567194 >> 172.21.15.38:0/953567194 conn(0x5638ecd6cac0 msgr2=0x5638ece937f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:13.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.477+0000 7f7e4b281f40 1 -- 172.21.15.38:0/953567194 shutdown_connections 2024-09-21T03:40:13.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.477+0000 7f7e4b281f40 1 -- 172.21.15.38:0/953567194 wait complete. 2024-09-21T03:40:13.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.477+0000 7f7e4b281f40 1 Processor -- start 2024-09-21T03:40:13.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.478+0000 7f7e4b281f40 1 -- start start 2024-09-21T03:40:13.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.479+0000 7f7e4b281f40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5638ece4e240 0x5638ece921d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:13.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.479+0000 7f7e4b281f40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5638ece95090 0x5638ecf021f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:13.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.479+0000 7f7e4b281f40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5638ecf0a520 0x5638ece79aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:13.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.479+0000 7f7e4b281f40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5638ecf239a0 con 0x5638ece4e240 2024-09-21T03:40:13.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.479+0000 7f7e4b281f40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5638ecf092b0 con 0x5638ece95090 2024-09-21T03:40:13.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.479+0000 7f7e4b281f40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x5638ecf09590 con 0x5638ecf0a520 2024-09-21T03:40:13.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.479+0000 7f7e44d3d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5638ece4e240 0x5638ece921d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:13.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.479+0000 7f7e44d3d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5638ece4e240 0x5638ece921d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54690/0 (socket says 172.21.15.38:54690) 2024-09-21T03:40:13.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.479+0000 7f7e44d3d640 1 -- 172.21.15.38:0/121409839 learned_addr learned my addr 172.21.15.38:0/121409839 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:13.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.479+0000 7f7e4553e640 1 --2- 172.21.15.38:0/121409839 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5638ecf0a520 0x5638ece79aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:13.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.479+0000 7f7e4453c640 1 --2- 172.21.15.38:0/121409839 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5638ece95090 0x5638ecf021f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:13.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.479+0000 7f7e44d3d640 1 -- 172.21.15.38:0/121409839 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5638ecf0a520 msgr2=0x5638ece79aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:13.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.479+0000 7f7e44d3d640 1 --2- 172.21.15.38:0/121409839 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5638ecf0a520 0x5638ece79aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:13.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.479+0000 7f7e44d3d640 1 -- 172.21.15.38:0/121409839 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5638ece95090 msgr2=0x5638ecf021f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:13.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.479+0000 7f7e44d3d640 1 --2- 172.21.15.38:0/121409839 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5638ece95090 0x5638ecf021f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:13.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.479+0000 7f7e44d3d640 1 -- 172.21.15.38:0/121409839 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638ece93f50 con 0x5638ece4e240 2024-09-21T03:40:13.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.480+0000 7f7e44d3d640 1 --2- 172.21.15.38:0/121409839 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5638ece4e240 0x5638ece921d0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f7e3400bab0 tx=0x7f7e3400ff30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:13.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.480+0000 7f7e42538640 1 -- 172.21.15.38:0/121409839 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e3401b640 con 0x5638ece4e240 2024-09-21T03:40:13.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.480+0000 7f7e42538640 1 -- 172.21.15.38:0/121409839 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7e34020020 con 0x5638ece4e240 2024-09-21T03:40:13.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.480+0000 7f7e42538640 1 -- 172.21.15.38:0/121409839 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e3401b990 con 0x5638ece4e240 2024-09-21T03:40:13.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.480+0000 7f7e4b281f40 1 -- 172.21.15.38:0/121409839 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5638ecf02730 con 0x5638ece4e240 2024-09-21T03:40:13.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.480+0000 7f7e4b281f40 1 -- 172.21.15.38:0/121409839 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5638ece928e0 con 0x5638ece4e240 2024-09-21T03:40:13.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.482+0000 7f7e42538640 1 -- 172.21.15.38:0/121409839 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f7e34017070 con 0x5638ece4e240 2024-09-21T03:40:13.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.482+0000 7f7e42538640 1 --2- 172.21.15.38:0/121409839 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7e30077570 0x7f7e30079a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:13.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.482+0000 7f7e42538640 1 -- 172.21.15.38:0/121409839 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6187+0+0 (secure 0 0 0) 0x7f7e3409adf0 con 0x5638ece4e240 2024-09-21T03:40:13.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.482+0000 7f7e4453c640 1 --2- 172.21.15.38:0/121409839 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7e30077570 0x7f7e30079a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:13.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.482+0000 7f7e4b281f40 1 --2- 172.21.15.38:0/121409839 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x5638ecf06320 0x5638ecf1def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:13.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.482+0000 7f7e4b281f40 1 -- 172.21.15.38:0/121409839 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:1 13.13 13:c91d4a1d:::mds0_inotable:head [omap-get-keys in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x5638ecfc6c80 con 0x5638ecf06320 2024-09-21T03:40:13.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.483+0000 7f7e4553e640 1 --2- 172.21.15.38:0/121409839 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x5638ecf06320 0x5638ecf1def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:13.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.483+0000 7f7e4453c640 1 --2- 172.21.15.38:0/121409839 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7e30077570 0x7f7e30079a80 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f7e38055000 tx=0x7f7e38058040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:13.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.483+0000 7f7e4553e640 1 --2- 172.21.15.38:0/121409839 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x5638ecf06320 0x5638ecf1def0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:13.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.483+0000 7f7e4553e640 1 -- 172.21.15.38:0/121409839 <== osd.1 v2:172.21.15.38:6816/1491818732 1 ==== osd_op_reply(1 mds0_inotable [omap-get-keys out=5b] v0'0 uv3 ondisk = 0) ==== 157+0+5 (crc 0 0 0) 0x7f7e3c00b040 con 0x5638ecf06320 2024-09-21T03:40:13.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.483+0000 7f7e4b281f40 1 -- 172.21.15.38:0/121409839 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x5638ecf06320 msgr2=0x5638ecf1def0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:13.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.483+0000 7f7e4b281f40 1 --2- 172.21.15.38:0/121409839 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x5638ecf06320 0x5638ecf1def0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:13.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.483+0000 7f7e4b281f40 1 -- 172.21.15.38:0/121409839 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7e30077570 msgr2=0x7f7e30079a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:13.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.483+0000 7f7e4b281f40 1 --2- 172.21.15.38:0/121409839 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7e30077570 0x7f7e30079a80 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f7e38055000 tx=0x7f7e38058040 comp rx=0 tx=0).stop 2024-09-21T03:40:13.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.483+0000 7f7e4b281f40 1 -- 172.21.15.38:0/121409839 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5638ece4e240 msgr2=0x5638ece921d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:13.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.483+0000 7f7e4b281f40 1 --2- 172.21.15.38:0/121409839 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5638ece4e240 0x5638ece921d0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f7e3400bab0 tx=0x7f7e3400ff30 comp rx=0 tx=0).stop 2024-09-21T03:40:13.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.483+0000 7f7e4553e640 1 -- 172.21.15.38:0/121409839 reap_dead start 2024-09-21T03:40:13.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.483+0000 7f7e4b281f40 1 -- 172.21.15.38:0/121409839 shutdown_connections 2024-09-21T03:40:13.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.483+0000 7f7e4b281f40 1 -- 172.21.15.38:0/121409839 >> 172.21.15.38:0/121409839 conn(0x5638ecd6cac0 msgr2=0x5638ecfa9e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:13.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.483+0000 7f7e4b281f40 1 -- 172.21.15.38:0/121409839 shutdown_connections 2024-09-21T03:40:13.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.484+0000 7f7e4b281f40 1 -- 172.21.15.38:0/121409839 wait complete. 2024-09-21T03:40:13.489 DEBUG:teuthology.orchestra.run.smithi038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata listomapkeys 100.00000000 2024-09-21T03:40:13.543 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.542+0000 7f80917eef40 1 Processor -- start 2024-09-21T03:40:13.544 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.542+0000 7f80917eef40 1 -- start start 2024-09-21T03:40:13.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.543+0000 7f80917eef40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563dddd61090 0x563dddd43e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:13.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.543+0000 7f80917eef40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x563ddddd6520 0x563dddd42e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:13.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.543+0000 7f80917eef40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563ddddd4ef0 0x563dddd19dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:13.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.543+0000 7f80917eef40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x563dddcc6430 con 0x563dddd61090 2024-09-21T03:40:13.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.543+0000 7f80917eef40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x563dddcd31f0 con 0x563ddddd4ef0 2024-09-21T03:40:13.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.543+0000 7f80917eef40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x563dddc67d60 con 0x563ddddd6520 2024-09-21T03:40:13.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.543+0000 7f808b2aa640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563dddd61090 0x563dddd43e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:13.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.543+0000 7f808b2aa640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563dddd61090 0x563dddd43e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54702/0 (socket says 172.21.15.38:54702) 2024-09-21T03:40:13.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.543+0000 7f808b2aa640 1 -- 172.21.15.38:0/2432431718 learned_addr learned my addr 172.21.15.38:0/2432431718 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:13.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.543+0000 7f808aaa9640 1 --2- 172.21.15.38:0/2432431718 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x563ddddd6520 0x563dddd42e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:13.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.543+0000 7f808baab640 1 --2- 172.21.15.38:0/2432431718 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563ddddd4ef0 0x563dddd19dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:13.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.543+0000 7f808b2aa640 1 -- 172.21.15.38:0/2432431718 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x563ddddd6520 msgr2=0x563dddd42e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:13.547 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.543+0000 7f808b2aa640 1 --2- 172.21.15.38:0/2432431718 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x563ddddd6520 0x563dddd42e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:13.547 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.543+0000 7f808b2aa640 1 -- 172.21.15.38:0/2432431718 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563ddddd4ef0 msgr2=0x563dddd19dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:13.547 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.543+0000 7f808b2aa640 1 --2- 172.21.15.38:0/2432431718 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563ddddd4ef0 0x563dddd19dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:13.547 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.543+0000 7f808b2aa640 1 -- 172.21.15.38:0/2432431718 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563dddde7370 con 0x563dddd61090 2024-09-21T03:40:13.547 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.543+0000 7f808baab640 1 --2- 172.21.15.38:0/2432431718 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563ddddd4ef0 0x563dddd19dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:40:13.547 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.544+0000 7f808b2aa640 1 --2- 172.21.15.38:0/2432431718 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563dddd61090 0x563dddd43e20 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f807c046600 tx=0x7f807c002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15fbf999cb904cb8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:13.547 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.544+0000 7f8089aa7640 1 -- 172.21.15.38:0/2432431718 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f807c066070 con 0x563dddd61090 2024-09-21T03:40:13.547 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.544+0000 7f8089aa7640 1 -- 172.21.15.38:0/2432431718 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f807c056560 con 0x563dddd61090 2024-09-21T03:40:13.547 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.544+0000 7f8089aa7640 1 -- 172.21.15.38:0/2432431718 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f807c056950 con 0x563dddd61090 2024-09-21T03:40:13.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.544+0000 7f80917eef40 1 -- 172.21.15.38:0/2432431718 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563dddd61090 msgr2=0x563dddd43e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:13.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.544+0000 7f80917eef40 1 --2- 172.21.15.38:0/2432431718 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563dddd61090 0x563dddd43e20 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f807c046600 tx=0x7f807c002fb0 comp rx=0 tx=0).stop 2024-09-21T03:40:13.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.544+0000 7f80917eef40 1 -- 172.21.15.38:0/2432431718 shutdown_connections 2024-09-21T03:40:13.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.544+0000 7f80917eef40 1 --2- 172.21.15.38:0/2432431718 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563ddddd4ef0 0x563dddd19dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:13.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.544+0000 7f80917eef40 1 --2- 172.21.15.38:0/2432431718 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x563ddddd6520 0x563dddd42e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:13.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.544+0000 7f80917eef40 1 --2- 172.21.15.38:0/2432431718 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563dddd61090 0x563dddd43e20 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:13.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.544+0000 7f80917eef40 1 -- 172.21.15.38:0/2432431718 >> 172.21.15.38:0/2432431718 conn(0x563dddc38ac0 msgr2=0x563dddd425f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:13.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.544+0000 7f80917eef40 1 -- 172.21.15.38:0/2432431718 shutdown_connections 2024-09-21T03:40:13.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.544+0000 7f80917eef40 1 -- 172.21.15.38:0/2432431718 wait complete. 2024-09-21T03:40:13.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.545+0000 7f80917eef40 1 Processor -- start 2024-09-21T03:40:13.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.545+0000 7f80917eef40 1 -- start start 2024-09-21T03:40:13.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.547+0000 7f80917eef40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563dddd61090 0x563dddd42fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:13.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.547+0000 7f80917eef40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563ddddd4ef0 0x563ddddd47c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:13.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.547+0000 7f80917eef40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x563ddddd6520 0x563ddddebdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:13.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.547+0000 7f80917eef40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x563dddd60240 con 0x563ddddd4ef0 2024-09-21T03:40:13.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.547+0000 7f80917eef40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x563ddde87210 con 0x563dddd61090 2024-09-21T03:40:13.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.547+0000 7f80917eef40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x563dddd436d0 con 0x563ddddd6520 2024-09-21T03:40:13.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.547+0000 7f808aaa9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563ddddd4ef0 0x563ddddd47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:13.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.547+0000 7f808aaa9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563ddddd4ef0 0x563ddddd47c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54712/0 (socket says 172.21.15.38:54712) 2024-09-21T03:40:13.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.547+0000 7f808aaa9640 1 -- 172.21.15.38:0/1042812824 learned_addr learned my addr 172.21.15.38:0/1042812824 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:13.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.547+0000 7f808b2aa640 1 --2- 172.21.15.38:0/1042812824 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563dddd61090 0x563dddd42fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:13.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.547+0000 7f808baab640 1 --2- 172.21.15.38:0/1042812824 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x563ddddd6520 0x563ddddebdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:13.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.547+0000 7f808aaa9640 1 -- 172.21.15.38:0/1042812824 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x563ddddd6520 msgr2=0x563ddddebdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:13.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.547+0000 7f808aaa9640 1 --2- 172.21.15.38:0/1042812824 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x563ddddd6520 0x563ddddebdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:13.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.547+0000 7f808aaa9640 1 -- 172.21.15.38:0/1042812824 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563dddd61090 msgr2=0x563dddd42fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:13.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.547+0000 7f808aaa9640 1 --2- 172.21.15.38:0/1042812824 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563dddd61090 0x563dddd42fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:13.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.547+0000 7f808aaa9640 1 -- 172.21.15.38:0/1042812824 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ddddd4d00 con 0x563ddddd4ef0 2024-09-21T03:40:13.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.547+0000 7f808b2aa640 1 --2- 172.21.15.38:0/1042812824 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563dddd61090 0x563dddd42fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:40:13.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.547+0000 7f808aaa9640 1 --2- 172.21.15.38:0/1042812824 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563ddddd4ef0 0x563ddddd47c0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f8080002f50 tx=0x7f808000c600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:13.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.547+0000 7f8088aa5640 1 -- 172.21.15.38:0/1042812824 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8080015020 con 0x563ddddd4ef0 2024-09-21T03:40:13.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.547+0000 7f8088aa5640 1 -- 172.21.15.38:0/1042812824 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8080009070 con 0x563ddddd4ef0 2024-09-21T03:40:13.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.547+0000 7f80917eef40 1 -- 172.21.15.38:0/1042812824 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x563dddd3ae80 con 0x563ddddd4ef0 2024-09-21T03:40:13.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.547+0000 7f8088aa5640 1 -- 172.21.15.38:0/1042812824 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8080011960 con 0x563ddddd4ef0 2024-09-21T03:40:13.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.547+0000 7f80917eef40 1 -- 172.21.15.38:0/1042812824 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563dddd3b440 con 0x563ddddd4ef0 2024-09-21T03:40:13.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.549+0000 7f8088aa5640 1 -- 172.21.15.38:0/1042812824 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f8080018020 con 0x563ddddd4ef0 2024-09-21T03:40:13.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.549+0000 7f8088aa5640 1 --2- 172.21.15.38:0/1042812824 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8078077570 0x7f8078079a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:13.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.550+0000 7f8088aa5640 1 -- 172.21.15.38:0/1042812824 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6187+0+0 (secure 0 0 0) 0x7f8080099eb0 con 0x563ddddd4ef0 2024-09-21T03:40:13.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.550+0000 7f808b2aa640 1 --2- 172.21.15.38:0/1042812824 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8078077570 0x7f8078079a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:13.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.550+0000 7f80917eef40 1 --2- 172.21.15.38:0/1042812824 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x563ddde86330 0x563ddde86740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:13.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.550+0000 7f80917eef40 1 -- 172.21.15.38:0/1042812824 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:1 13.13 13:cd5a64a3:::100.00000000:head [omap-get-keys in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x563dddd45da0 con 0x563ddde86330 2024-09-21T03:40:13.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.550+0000 7f808baab640 1 --2- 172.21.15.38:0/1042812824 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x563ddde86330 0x563ddde86740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:13.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.550+0000 7f808b2aa640 1 --2- 172.21.15.38:0/1042812824 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8078077570 0x7f8078079a80 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f807c063000 tx=0x7f807c058040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:13.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.550+0000 7f808baab640 1 --2- 172.21.15.38:0/1042812824 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x563ddde86330 0x563ddde86740 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:13.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.551+0000 7f808baab640 1 -- 172.21.15.38:0/1042812824 <== osd.1 v2:172.21.15.38:6816/1491818732 1 ==== osd_op_reply(1 100.00000000 [omap-get-keys out=155b] v0'0 uv1 ondisk = 0) ==== 156+0+155 (crc 0 0 0) 0x7f80840015f0 con 0x563ddde86330 2024-09-21T03:40:13.555 INFO:teuthology.orchestra.run.smithi038.stdout:stray0_head 2024-09-21T03:40:13.555 INFO:teuthology.orchestra.run.smithi038.stdout:stray1_head 2024-09-21T03:40:13.555 INFO:teuthology.orchestra.run.smithi038.stdout:stray2_head 2024-09-21T03:40:13.555 INFO:teuthology.orchestra.run.smithi038.stdout:stray3_head 2024-09-21T03:40:13.555 INFO:teuthology.orchestra.run.smithi038.stdout:stray4_head 2024-09-21T03:40:13.555 INFO:teuthology.orchestra.run.smithi038.stdout:stray5_head 2024-09-21T03:40:13.556 INFO:teuthology.orchestra.run.smithi038.stdout:stray6_head 2024-09-21T03:40:13.556 INFO:teuthology.orchestra.run.smithi038.stdout:stray7_head 2024-09-21T03:40:13.556 INFO:teuthology.orchestra.run.smithi038.stdout:stray8_head 2024-09-21T03:40:13.556 INFO:teuthology.orchestra.run.smithi038.stdout:stray9_head 2024-09-21T03:40:13.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.551+0000 7f80917eef40 1 -- 172.21.15.38:0/1042812824 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x563ddde86330 msgr2=0x563ddde86740 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:13.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.551+0000 7f80917eef40 1 --2- 172.21.15.38:0/1042812824 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x563ddde86330 0x563ddde86740 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:13.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.551+0000 7f80917eef40 1 -- 172.21.15.38:0/1042812824 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8078077570 msgr2=0x7f8078079a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:13.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.551+0000 7f80917eef40 1 --2- 172.21.15.38:0/1042812824 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8078077570 0x7f8078079a80 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f807c063000 tx=0x7f807c058040 comp rx=0 tx=0).stop 2024-09-21T03:40:13.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.551+0000 7f80917eef40 1 -- 172.21.15.38:0/1042812824 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563ddddd4ef0 msgr2=0x563ddddd47c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:13.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.551+0000 7f80917eef40 1 --2- 172.21.15.38:0/1042812824 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563ddddd4ef0 0x563ddddd47c0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f8080002f50 tx=0x7f808000c600 comp rx=0 tx=0).stop 2024-09-21T03:40:13.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.551+0000 7f808baab640 1 -- 172.21.15.38:0/1042812824 reap_dead start 2024-09-21T03:40:13.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.551+0000 7f80917eef40 1 -- 172.21.15.38:0/1042812824 shutdown_connections 2024-09-21T03:40:13.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.551+0000 7f80917eef40 1 -- 172.21.15.38:0/1042812824 >> 172.21.15.38:0/1042812824 conn(0x563dddc38ac0 msgr2=0x563dddd425c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:13.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.551+0000 7f80917eef40 1 -- 172.21.15.38:0/1042812824 shutdown_connections 2024-09-21T03:40:13.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.551+0000 7f80917eef40 1 -- 172.21.15.38:0/1042812824 wait complete. 2024-09-21T03:40:13.559 DEBUG:teuthology.orchestra.run.smithi038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata listomapkeys mds0_sessionmap 2024-09-21T03:40:13.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.613+0000 7fb539042f40 1 Processor -- start 2024-09-21T03:40:13.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.613+0000 7fb539042f40 1 -- start start 2024-09-21T03:40:13.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.615+0000 7fb539042f40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a9391e6090 0x55a939256050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:13.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.615+0000 7fb539042f40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a93925b520 0x55a9391e3450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:13.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.615+0000 7fb539042f40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a9391c7f70 0x55a9391c8380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:13.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.615+0000 7fb539042f40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55a93914b430 con 0x55a9391c7f70 2024-09-21T03:40:13.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.615+0000 7fb539042f40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55a9391581f0 con 0x55a93925b520 2024-09-21T03:40:13.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.615+0000 7fb539042f40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55a9390ecd60 con 0x55a9391e6090 2024-09-21T03:40:13.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.615+0000 7fb5332fd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a9391c7f70 0x55a9391c8380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:13.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.615+0000 7fb5332fd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a9391c7f70 0x55a9391c8380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54724/0 (socket says 172.21.15.38:54724) 2024-09-21T03:40:13.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.615+0000 7fb5332fd640 1 -- 172.21.15.38:0/3021068602 learned_addr learned my addr 172.21.15.38:0/3021068602 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:13.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.615+0000 7fb532afc640 1 --2- 172.21.15.38:0/3021068602 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a9391e6090 0x55a939256050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:13.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.615+0000 7fb5322fb640 1 --2- 172.21.15.38:0/3021068602 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a93925b520 0x55a9391e3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:13.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.615+0000 7fb5332fd640 1 -- 172.21.15.38:0/3021068602 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a9391e6090 msgr2=0x55a939256050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:13.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.615+0000 7fb5332fd640 1 --2- 172.21.15.38:0/3021068602 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a9391e6090 0x55a939256050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:13.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.615+0000 7fb5332fd640 1 -- 172.21.15.38:0/3021068602 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a93925b520 msgr2=0x55a9391e3450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:13.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.615+0000 7fb5332fd640 1 --2- 172.21.15.38:0/3021068602 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a93925b520 0x55a9391e3450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:13.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.615+0000 7fb5332fd640 1 -- 172.21.15.38:0/3021068602 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a9391c88c0 con 0x55a9391c7f70 2024-09-21T03:40:13.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.615+0000 7fb5322fb640 1 --2- 172.21.15.38:0/3021068602 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a93925b520 0x55a9391e3450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:40:13.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.615+0000 7fb532afc640 1 --2- 172.21.15.38:0/3021068602 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a9391e6090 0x55a939256050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:40:13.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.615+0000 7fb5332fd640 1 --2- 172.21.15.38:0/3021068602 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a9391c7f70 0x55a9391c8380 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7fb52c026000 tx=0x7fb52c002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c65d11aaf715cb7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:13.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.615+0000 7fb5312f9640 1 -- 172.21.15.38:0/3021068602 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb52c066070 con 0x55a9391c7f70 2024-09-21T03:40:13.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.615+0000 7fb5312f9640 1 -- 172.21.15.38:0/3021068602 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb52c056470 con 0x55a9391c7f70 2024-09-21T03:40:13.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.615+0000 7fb5312f9640 1 -- 172.21.15.38:0/3021068602 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb52c056860 con 0x55a9391c7f70 2024-09-21T03:40:13.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.616+0000 7fb539042f40 1 -- 172.21.15.38:0/3021068602 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a9391c7f70 msgr2=0x55a9391c8380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:13.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.616+0000 7fb539042f40 1 --2- 172.21.15.38:0/3021068602 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a9391c7f70 0x55a9391c8380 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7fb52c026000 tx=0x7fb52c002fb0 comp rx=0 tx=0).stop 2024-09-21T03:40:13.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.616+0000 7fb539042f40 1 -- 172.21.15.38:0/3021068602 shutdown_connections 2024-09-21T03:40:13.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.616+0000 7fb539042f40 1 --2- 172.21.15.38:0/3021068602 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a9391c7f70 0x55a9391c8380 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:13.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.616+0000 7fb539042f40 1 --2- 172.21.15.38:0/3021068602 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a93925b520 0x55a9391e3450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:13.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.616+0000 7fb539042f40 1 --2- 172.21.15.38:0/3021068602 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a9391e6090 0x55a939256050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:13.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.616+0000 7fb539042f40 1 -- 172.21.15.38:0/3021068602 >> 172.21.15.38:0/3021068602 conn(0x55a9390bdac0 msgr2=0x55a939271480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:13.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.616+0000 7fb539042f40 1 -- 172.21.15.38:0/3021068602 shutdown_connections 2024-09-21T03:40:13.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.616+0000 7fb539042f40 1 -- 172.21.15.38:0/3021068602 wait complete. 2024-09-21T03:40:13.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.617+0000 7fb539042f40 1 Processor -- start 2024-09-21T03:40:13.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.617+0000 7fb539042f40 1 -- start start 2024-09-21T03:40:13.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.618+0000 7fb539042f40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a9391c7f70 0x55a9391cc0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:13.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.618+0000 7fb539042f40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a9391e6090 0x55a939259540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:13.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.618+0000 7fb539042f40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a93925b520 0x55a939251dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:13.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.618+0000 7fb539042f40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55a9391c9420 con 0x55a9391e6090 2024-09-21T03:40:13.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.618+0000 7fb539042f40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55a93926bd90 con 0x55a93925b520 2024-09-21T03:40:13.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.618+0000 7fb539042f40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55a9391c7870 con 0x55a9391c7f70 2024-09-21T03:40:13.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.618+0000 7fb5322fb640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a9391e6090 0x55a939259540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:13.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.618+0000 7fb5322fb640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a9391e6090 0x55a939259540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54740/0 (socket says 172.21.15.38:54740) 2024-09-21T03:40:13.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.618+0000 7fb5322fb640 1 -- 172.21.15.38:0/687720344 learned_addr learned my addr 172.21.15.38:0/687720344 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:13.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.618+0000 7fb5332fd640 1 --2- 172.21.15.38:0/687720344 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a93925b520 0x55a939251dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:13.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.618+0000 7fb532afc640 1 --2- 172.21.15.38:0/687720344 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a9391c7f70 0x55a9391cc0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:13.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.619+0000 7fb5322fb640 1 -- 172.21.15.38:0/687720344 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a9391c7f70 msgr2=0x55a9391cc0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:13.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.619+0000 7fb5322fb640 1 --2- 172.21.15.38:0/687720344 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a9391c7f70 0x55a9391cc0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:13.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.619+0000 7fb5322fb640 1 -- 172.21.15.38:0/687720344 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a93925b520 msgr2=0x55a939251dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:13.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.619+0000 7fb5322fb640 1 --2- 172.21.15.38:0/687720344 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a93925b520 0x55a939251dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:13.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.619+0000 7fb5322fb640 1 -- 172.21.15.38:0/687720344 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a93930f7c0 con 0x55a9391e6090 2024-09-21T03:40:13.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.619+0000 7fb5322fb640 1 --2- 172.21.15.38:0/687720344 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a9391e6090 0x55a939259540 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7fb51c003ea0 tx=0x7fb51c00b590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:13.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.619+0000 7fb52bfff640 1 -- 172.21.15.38:0/687720344 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb51c0032d0 con 0x55a9391e6090 2024-09-21T03:40:13.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.619+0000 7fb52bfff640 1 -- 172.21.15.38:0/687720344 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb51c016070 con 0x55a9391e6090 2024-09-21T03:40:13.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.619+0000 7fb539042f40 1 -- 172.21.15.38:0/687720344 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55a93930faa0 con 0x55a9391e6090 2024-09-21T03:40:13.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.619+0000 7fb52bfff640 1 -- 172.21.15.38:0/687720344 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb51c001770 con 0x55a9391e6090 2024-09-21T03:40:13.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.619+0000 7fb539042f40 1 -- 172.21.15.38:0/687720344 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a939253e70 con 0x55a9391e6090 2024-09-21T03:40:13.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.621+0000 7fb52bfff640 1 -- 172.21.15.38:0/687720344 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb51c001910 con 0x55a9391e6090 2024-09-21T03:40:13.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.621+0000 7fb52bfff640 1 --2- 172.21.15.38:0/687720344 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb514077570 0x7fb514079a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:13.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.621+0000 7fb52bfff640 1 -- 172.21.15.38:0/687720344 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6187+0+0 (secure 0 0 0) 0x7fb51c099b20 con 0x55a9391e6090 2024-09-21T03:40:13.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.621+0000 7fb532afc640 1 --2- 172.21.15.38:0/687720344 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb514077570 0x7fb514079a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:13.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.622+0000 7fb539042f40 1 --2- 172.21.15.38:0/687720344 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x55a93930ccf0 0x55a939271270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:13.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.622+0000 7fb539042f40 1 -- 172.21.15.38:0/687720344 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:1 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-get-keys in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x55a9392717b0 con 0x55a93930ccf0 2024-09-21T03:40:13.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.622+0000 7fb5332fd640 1 --2- 172.21.15.38:0/687720344 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x55a93930ccf0 0x55a939271270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:13.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.622+0000 7fb532afc640 1 --2- 172.21.15.38:0/687720344 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb514077570 0x7fb514079a80 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fb524000f80 tx=0x7fb524008040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:13.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.622+0000 7fb5332fd640 1 --2- 172.21.15.38:0/687720344 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x55a93930ccf0 0x55a939271270 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:13.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.622+0000 7fb5332fd640 1 -- 172.21.15.38:0/687720344 <== osd.3 v2:172.21.15.38:6808/1286810294 1 ==== osd_op_reply(1 mds0_sessionmap [omap-get-keys out=5b] v0'0 uv3 ondisk = 0) ==== 159+0+5 (crc 0 0 0) 0x7fb52c05f070 con 0x55a93930ccf0 2024-09-21T03:40:13.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.622+0000 7fb539042f40 1 -- 172.21.15.38:0/687720344 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x55a93930ccf0 msgr2=0x55a939271270 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:13.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.622+0000 7fb539042f40 1 --2- 172.21.15.38:0/687720344 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x55a93930ccf0 0x55a939271270 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:13.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.623+0000 7fb539042f40 1 -- 172.21.15.38:0/687720344 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb514077570 msgr2=0x7fb514079a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:13.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.623+0000 7fb539042f40 1 --2- 172.21.15.38:0/687720344 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb514077570 0x7fb514079a80 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fb524000f80 tx=0x7fb524008040 comp rx=0 tx=0).stop 2024-09-21T03:40:13.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.623+0000 7fb539042f40 1 -- 172.21.15.38:0/687720344 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a9391e6090 msgr2=0x55a939259540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:13.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.623+0000 7fb539042f40 1 --2- 172.21.15.38:0/687720344 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a9391e6090 0x55a939259540 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7fb51c003ea0 tx=0x7fb51c00b590 comp rx=0 tx=0).stop 2024-09-21T03:40:13.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.623+0000 7fb5332fd640 1 -- 172.21.15.38:0/687720344 reap_dead start 2024-09-21T03:40:13.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.623+0000 7fb539042f40 1 -- 172.21.15.38:0/687720344 shutdown_connections 2024-09-21T03:40:13.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.623+0000 7fb539042f40 1 -- 172.21.15.38:0/687720344 >> 172.21.15.38:0/687720344 conn(0x55a9390bdac0 msgr2=0x55a9391e3450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:13.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.623+0000 7fb539042f40 1 -- 172.21.15.38:0/687720344 shutdown_connections 2024-09-21T03:40:13.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.623+0000 7fb539042f40 1 -- 172.21.15.38:0/687720344 wait complete. 2024-09-21T03:40:13.628 DEBUG:teuthology.orchestra.run.smithi038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata listomapkeys 400.00000000 2024-09-21T03:40:13.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.681+0000 7fdf86d29f40 1 Processor -- start 2024-09-21T03:40:13.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.681+0000 7fdf86d29f40 1 -- start start 2024-09-21T03:40:13.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.682+0000 7fdf86d29f40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a2777c7240 0x55a2777c6440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:13.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.682+0000 7fdf86d29f40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a27780e090 0x55a2777f0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:13.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.682+0000 7fdf86d29f40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a277883520 0x55a277894850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:13.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.682+0000 7fdf86d29f40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55a277714d60 con 0x55a2777c7240 2024-09-21T03:40:13.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.682+0000 7fdf86d29f40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55a277773430 con 0x55a277883520 2024-09-21T03:40:13.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.682+0000 7fdf86d29f40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55a27789aaa0 con 0x55a27780e090 2024-09-21T03:40:13.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.682+0000 7fdf807e3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a2777c7240 0x55a2777c6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:13.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.683+0000 7fdf807e3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a2777c7240 0x55a2777c6440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54754/0 (socket says 172.21.15.38:54754) 2024-09-21T03:40:13.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.683+0000 7fdf807e3640 1 -- 172.21.15.38:0/1016831101 learned_addr learned my addr 172.21.15.38:0/1016831101 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:13.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.683+0000 7fdf7ffe2640 1 --2- 172.21.15.38:0/1016831101 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a27780e090 0x55a2777f0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:13.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.683+0000 7fdf80fe4640 1 --2- 172.21.15.38:0/1016831101 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a277883520 0x55a277894850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:13.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.683+0000 7fdf807e3640 1 -- 172.21.15.38:0/1016831101 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a27780e090 msgr2=0x55a2777f0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:13.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.683+0000 7fdf807e3640 1 --2- 172.21.15.38:0/1016831101 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a27780e090 0x55a2777f0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:13.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.683+0000 7fdf807e3640 1 -- 172.21.15.38:0/1016831101 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a277883520 msgr2=0x55a277894850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:13.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.683+0000 7fdf807e3640 1 --2- 172.21.15.38:0/1016831101 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a277883520 0x55a277894850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:13.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.683+0000 7fdf807e3640 1 -- 172.21.15.38:0/1016831101 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a27789b8e0 con 0x55a2777c7240 2024-09-21T03:40:13.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.683+0000 7fdf807e3640 1 --2- 172.21.15.38:0/1016831101 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a2777c7240 0x55a2777c6440 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7fdf7000b520 tx=0x7fdf7000b9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=149251aad216732f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:13.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.683+0000 7fdf7efe0640 1 -- 172.21.15.38:0/1016831101 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf700099f0 con 0x55a2777c7240 2024-09-21T03:40:13.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.683+0000 7fdf7efe0640 1 -- 172.21.15.38:0/1016831101 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdf70009b90 con 0x55a2777c7240 2024-09-21T03:40:13.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.683+0000 7fdf7efe0640 1 -- 172.21.15.38:0/1016831101 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf70014440 con 0x55a2777c7240 2024-09-21T03:40:13.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.684+0000 7fdf86d29f40 1 -- 172.21.15.38:0/1016831101 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a2777c7240 msgr2=0x55a2777c6440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:13.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.684+0000 7fdf86d29f40 1 --2- 172.21.15.38:0/1016831101 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a2777c7240 0x55a2777c6440 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7fdf7000b520 tx=0x7fdf7000b9e0 comp rx=0 tx=0).stop 2024-09-21T03:40:13.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.684+0000 7fdf86d29f40 1 -- 172.21.15.38:0/1016831101 shutdown_connections 2024-09-21T03:40:13.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.684+0000 7fdf86d29f40 1 --2- 172.21.15.38:0/1016831101 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a277883520 0x55a277894850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:13.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.684+0000 7fdf86d29f40 1 --2- 172.21.15.38:0/1016831101 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a27780e090 0x55a2777f0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:13.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.684+0000 7fdf86d29f40 1 --2- 172.21.15.38:0/1016831101 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a2777c7240 0x55a2777c6440 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:13.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.684+0000 7fdf86d29f40 1 -- 172.21.15.38:0/1016831101 >> 172.21.15.38:0/1016831101 conn(0x55a2776e5ac0 msgr2=0x55a27780d930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:13.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.684+0000 7fdf86d29f40 1 -- 172.21.15.38:0/1016831101 shutdown_connections 2024-09-21T03:40:13.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.684+0000 7fdf86d29f40 1 -- 172.21.15.38:0/1016831101 wait complete. 2024-09-21T03:40:13.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.685+0000 7fdf86d29f40 1 Processor -- start 2024-09-21T03:40:13.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.685+0000 7fdf86d29f40 1 -- start start 2024-09-21T03:40:13.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.686+0000 7fdf86d29f40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a2777c7240 0x55a277877480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:13.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.686+0000 7fdf86d29f40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a27780e090 0x55a277893c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:13.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.686+0000 7fdf86d29f40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a277883520 0x55a2778941c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:13.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.686+0000 7fdf86d29f40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55a277931370 con 0x55a27780e090 2024-09-21T03:40:13.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.686+0000 7fdf86d29f40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55a27787a6a0 con 0x55a2777c7240 2024-09-21T03:40:13.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.686+0000 7fdf86d29f40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55a27787a8e0 con 0x55a277883520 2024-09-21T03:40:13.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.686+0000 7fdf7ffe2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a27780e090 0x55a277893c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:13.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.686+0000 7fdf7ffe2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a27780e090 0x55a277893c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54770/0 (socket says 172.21.15.38:54770) 2024-09-21T03:40:13.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.686+0000 7fdf7ffe2640 1 -- 172.21.15.38:0/237427658 learned_addr learned my addr 172.21.15.38:0/237427658 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:13.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.686+0000 7fdf80fe4640 1 --2- 172.21.15.38:0/237427658 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a277883520 0x55a2778941c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:13.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.686+0000 7fdf807e3640 1 --2- 172.21.15.38:0/237427658 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a2777c7240 0x55a277877480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:13.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.686+0000 7fdf7ffe2640 1 -- 172.21.15.38:0/237427658 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a277883520 msgr2=0x55a2778941c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:13.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.686+0000 7fdf7ffe2640 1 --2- 172.21.15.38:0/237427658 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a277883520 0x55a2778941c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:13.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.686+0000 7fdf7ffe2640 1 -- 172.21.15.38:0/237427658 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a2777c7240 msgr2=0x55a277877480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:13.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.686+0000 7fdf7ffe2640 1 --2- 172.21.15.38:0/237427658 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a2777c7240 0x55a277877480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:13.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.687+0000 7fdf7ffe2640 1 -- 172.21.15.38:0/237427658 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a27787ab30 con 0x55a27780e090 2024-09-21T03:40:13.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.687+0000 7fdf7ffe2640 1 --2- 172.21.15.38:0/237427658 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a27780e090 0x55a277893c80 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7fdf74025f80 tx=0x7fdf7404db00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:13.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.687+0000 7fdf7dfde640 1 -- 172.21.15.38:0/237427658 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf74068070 con 0x55a27780e090 2024-09-21T03:40:13.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.687+0000 7fdf7dfde640 1 -- 172.21.15.38:0/237427658 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdf74058190 con 0x55a27780e090 2024-09-21T03:40:13.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.687+0000 7fdf86d29f40 1 -- 172.21.15.38:0/237427658 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55a2779225c0 con 0x55a27780e090 2024-09-21T03:40:13.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.687+0000 7fdf7dfde640 1 -- 172.21.15.38:0/237427658 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf740587f0 con 0x55a27780e090 2024-09-21T03:40:13.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.687+0000 7fdf86d29f40 1 -- 172.21.15.38:0/237427658 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a277922b80 con 0x55a27780e090 2024-09-21T03:40:13.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.689+0000 7fdf7dfde640 1 -- 172.21.15.38:0/237427658 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fdf7406b070 con 0x55a27780e090 2024-09-21T03:40:13.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.689+0000 7fdf7dfde640 1 --2- 172.21.15.38:0/237427658 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fdf6c077570 0x7fdf6c079a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:13.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.689+0000 7fdf7dfde640 1 -- 172.21.15.38:0/237427658 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6187+0+0 (secure 0 0 0) 0x7fdf740dd9e0 con 0x55a27780e090 2024-09-21T03:40:13.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.689+0000 7fdf807e3640 1 --2- 172.21.15.38:0/237427658 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fdf6c077570 0x7fdf6c079a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:13.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.689+0000 7fdf86d29f40 1 --2- 172.21.15.38:0/237427658 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x55a27793aba0 0x55a27793cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:13.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.689+0000 7fdf86d29f40 1 -- 172.21.15.38:0/237427658 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:1 13.f 13:f1f69726:::400.00000000:head [omap-get-keys in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x55a27789d2b0 con 0x55a27793aba0 2024-09-21T03:40:13.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.690+0000 7fdf80fe4640 1 --2- 172.21.15.38:0/237427658 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x55a27793aba0 0x55a27793cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:13.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.690+0000 7fdf80fe4640 1 --2- 172.21.15.38:0/237427658 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x55a27793aba0 0x55a27793cfc0 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:13.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.690+0000 7fdf807e3640 1 --2- 172.21.15.38:0/237427658 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fdf6c077570 0x7fdf6c079a80 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fdf70004e40 tx=0x7fdf70016040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:13.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.690+0000 7fdf80fe4640 1 -- 172.21.15.38:0/237427658 <== osd.2 v2:172.21.15.38:6824/2577858902 1 ==== osd_op_reply(1 400.00000000 [omap-get-keys out=5b] v0'0 uv1 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x7fdf78003020 con 0x55a27793aba0 2024-09-21T03:40:13.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.690+0000 7fdf86d29f40 1 -- 172.21.15.38:0/237427658 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x55a27793aba0 msgr2=0x55a27793cfc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:13.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.690+0000 7fdf86d29f40 1 --2- 172.21.15.38:0/237427658 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x55a27793aba0 0x55a27793cfc0 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:13.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.690+0000 7fdf86d29f40 1 -- 172.21.15.38:0/237427658 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fdf6c077570 msgr2=0x7fdf6c079a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:13.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.690+0000 7fdf86d29f40 1 --2- 172.21.15.38:0/237427658 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fdf6c077570 0x7fdf6c079a80 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fdf70004e40 tx=0x7fdf70016040 comp rx=0 tx=0).stop 2024-09-21T03:40:13.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.690+0000 7fdf86d29f40 1 -- 172.21.15.38:0/237427658 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a27780e090 msgr2=0x55a277893c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:13.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.690+0000 7fdf86d29f40 1 --2- 172.21.15.38:0/237427658 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a27780e090 0x55a277893c80 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7fdf74025f80 tx=0x7fdf7404db00 comp rx=0 tx=0).stop 2024-09-21T03:40:13.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.690+0000 7fdf80fe4640 1 -- 172.21.15.38:0/237427658 reap_dead start 2024-09-21T03:40:13.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.690+0000 7fdf86d29f40 1 -- 172.21.15.38:0/237427658 shutdown_connections 2024-09-21T03:40:13.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.690+0000 7fdf86d29f40 1 -- 172.21.15.38:0/237427658 >> 172.21.15.38:0/237427658 conn(0x55a2776e5ac0 msgr2=0x55a2777f0bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:13.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.690+0000 7fdf86d29f40 1 -- 172.21.15.38:0/237427658 shutdown_connections 2024-09-21T03:40:13.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.690+0000 7fdf86d29f40 1 -- 172.21.15.38:0/237427658 wait complete. 2024-09-21T03:40:13.695 DEBUG:teuthology.orchestra.run.smithi038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata listomapkeys 100.00000000.inode 2024-09-21T03:40:13.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.748+0000 7fbcd6462f40 1 Processor -- start 2024-09-21T03:40:13.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.748+0000 7fbcd6462f40 1 -- start start 2024-09-21T03:40:13.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.750+0000 7fbcd6462f40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f0903f8d00 0x55f0904b1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:13.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.750+0000 7fbcd6462f40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f090424e30 0x55f0904b2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:13.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.750+0000 7fbcd6462f40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f0904b3950 0x55f09055c110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:13.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.750+0000 7fbcd6462f40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55f0903a5430 con 0x55f0903f8d00 2024-09-21T03:40:13.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.750+0000 7fbcd6462f40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55f0903b21f0 con 0x55f0904b3950 2024-09-21T03:40:13.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.750+0000 7fbcd6462f40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55f090346d60 con 0x55f090424e30 2024-09-21T03:40:13.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.750+0000 7fbccff1c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f0903f8d00 0x55f0904b1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:13.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.750+0000 7fbccff1c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f0903f8d00 0x55f0904b1da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54786/0 (socket says 172.21.15.38:54786) 2024-09-21T03:40:13.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.750+0000 7fbccff1c640 1 -- 172.21.15.38:0/1353968350 learned_addr learned my addr 172.21.15.38:0/1353968350 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:13.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.750+0000 7fbcc771b640 1 --2- 172.21.15.38:0/1353968350 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f090424e30 0x55f0904b2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:13.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.750+0000 7fbcd071d640 1 --2- 172.21.15.38:0/1353968350 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f0904b3950 0x55f09055c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:13.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.750+0000 7fbccff1c640 1 -- 172.21.15.38:0/1353968350 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f090424e30 msgr2=0x55f0904b2ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:13.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.750+0000 7fbccff1c640 1 --2- 172.21.15.38:0/1353968350 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f090424e30 0x55f0904b2ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:13.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.750+0000 7fbccff1c640 1 -- 172.21.15.38:0/1353968350 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f0904b3950 msgr2=0x55f09055c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:13.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.750+0000 7fbccff1c640 1 --2- 172.21.15.38:0/1353968350 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f0904b3950 0x55f09055c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:13.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.750+0000 7fbccff1c640 1 -- 172.21.15.38:0/1353968350 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f0904b19e0 con 0x55f0903f8d00 2024-09-21T03:40:13.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.750+0000 7fbcc771b640 1 --2- 172.21.15.38:0/1353968350 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f090424e30 0x55f0904b2ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:40:13.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.750+0000 7fbccff1c640 1 --2- 172.21.15.38:0/1353968350 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f0903f8d00 0x55f0904b1da0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7fbcc0001e60 tx=0x7fbcc000b730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95a16863d915bdef server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:13.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.750+0000 7fbccef1a640 1 -- 172.21.15.38:0/1353968350 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcc000e830 con 0x55f0903f8d00 2024-09-21T03:40:13.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.750+0000 7fbccef1a640 1 -- 172.21.15.38:0/1353968350 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbcc0006050 con 0x55f0903f8d00 2024-09-21T03:40:13.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.750+0000 7fbccef1a640 1 -- 172.21.15.38:0/1353968350 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcc000ed80 con 0x55f0903f8d00 2024-09-21T03:40:13.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.751+0000 7fbcd6462f40 1 -- 172.21.15.38:0/1353968350 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f0903f8d00 msgr2=0x55f0904b1da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:13.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.751+0000 7fbcd6462f40 1 --2- 172.21.15.38:0/1353968350 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f0903f8d00 0x55f0904b1da0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7fbcc0001e60 tx=0x7fbcc000b730 comp rx=0 tx=0).stop 2024-09-21T03:40:13.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.751+0000 7fbcd6462f40 1 -- 172.21.15.38:0/1353968350 shutdown_connections 2024-09-21T03:40:13.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.751+0000 7fbcd6462f40 1 --2- 172.21.15.38:0/1353968350 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f0904b3950 0x55f09055c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:13.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.751+0000 7fbcd6462f40 1 --2- 172.21.15.38:0/1353968350 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f090424e30 0x55f0904b2ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:13.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.751+0000 7fbcd6462f40 1 --2- 172.21.15.38:0/1353968350 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f0903f8d00 0x55f0904b1da0 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:13.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.751+0000 7fbcd6462f40 1 -- 172.21.15.38:0/1353968350 >> 172.21.15.38:0/1353968350 conn(0x55f090317ac0 msgr2=0x55f0904ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:13.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.751+0000 7fbcd6462f40 1 -- 172.21.15.38:0/1353968350 shutdown_connections 2024-09-21T03:40:13.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.751+0000 7fbcd6462f40 1 -- 172.21.15.38:0/1353968350 wait complete. 2024-09-21T03:40:13.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.752+0000 7fbcd6462f40 1 Processor -- start 2024-09-21T03:40:13.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.752+0000 7fbcd6462f40 1 -- start start 2024-09-21T03:40:13.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.753+0000 7fbcd6462f40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f0903f8d00 0x55f0904c8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:13.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.753+0000 7fbcd6462f40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f090424e30 0x55f0904c7e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:13.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.753+0000 7fbcd6462f40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f0904b3950 0x55f0904cba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:13.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.753+0000 7fbcd6462f40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55f0904b1550 con 0x55f090424e30 2024-09-21T03:40:13.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.753+0000 7fbcd6462f40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55f090425670 con 0x55f0904b3950 2024-09-21T03:40:13.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.753+0000 7fbcd6462f40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55f0904af900 con 0x55f0903f8d00 2024-09-21T03:40:13.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.753+0000 7fbcc771b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f090424e30 0x55f0904c7e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:13.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.754+0000 7fbcc771b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f090424e30 0x55f0904c7e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54798/0 (socket says 172.21.15.38:54798) 2024-09-21T03:40:13.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.754+0000 7fbcc771b640 1 -- 172.21.15.38:0/2526320392 learned_addr learned my addr 172.21.15.38:0/2526320392 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:13.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.754+0000 7fbccff1c640 1 --2- 172.21.15.38:0/2526320392 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f0903f8d00 0x55f0904c8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:13.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.754+0000 7fbcd071d640 1 --2- 172.21.15.38:0/2526320392 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f0904b3950 0x55f0904cba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:13.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.754+0000 7fbcc771b640 1 -- 172.21.15.38:0/2526320392 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f0903f8d00 msgr2=0x55f0904c8b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:13.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.754+0000 7fbcc771b640 1 --2- 172.21.15.38:0/2526320392 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f0903f8d00 0x55f0904c8b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:13.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.754+0000 7fbcc771b640 1 -- 172.21.15.38:0/2526320392 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f0904b3950 msgr2=0x55f0904cba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:13.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.754+0000 7fbcc771b640 1 --2- 172.21.15.38:0/2526320392 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f0904b3950 0x55f0904cba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:13.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.754+0000 7fbcc771b640 1 -- 172.21.15.38:0/2526320392 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f0904aaab0 con 0x55f090424e30 2024-09-21T03:40:13.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.754+0000 7fbcc771b640 1 --2- 172.21.15.38:0/2526320392 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f090424e30 0x55f0904c7e20 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7fbcb8025ed0 tx=0x7fbcb804efd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:13.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.754+0000 7fbccdf18640 1 -- 172.21.15.38:0/2526320392 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcb804d920 con 0x55f090424e30 2024-09-21T03:40:13.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.754+0000 7fbccdf18640 1 -- 172.21.15.38:0/2526320392 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbcb8056560 con 0x55f090424e30 2024-09-21T03:40:13.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.754+0000 7fbcd6462f40 1 -- 172.21.15.38:0/2526320392 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55f0904b2c40 con 0x55f090424e30 2024-09-21T03:40:13.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.754+0000 7fbccdf18640 1 -- 172.21.15.38:0/2526320392 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcb8056750 con 0x55f090424e30 2024-09-21T03:40:13.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.754+0000 7fbcd6462f40 1 -- 172.21.15.38:0/2526320392 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f0904cb220 con 0x55f090424e30 2024-09-21T03:40:13.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.756+0000 7fbccdf18640 1 -- 172.21.15.38:0/2526320392 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fbcb8069020 con 0x55f090424e30 2024-09-21T03:40:13.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.757+0000 7fbccdf18640 1 --2- 172.21.15.38:0/2526320392 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fbcb0077570 0x7fbcb0079a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:13.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.757+0000 7fbccdf18640 1 -- 172.21.15.38:0/2526320392 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6187+0+0 (secure 0 0 0) 0x7fbcb80dd7e0 con 0x55f090424e30 2024-09-21T03:40:13.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.757+0000 7fbcd6462f40 1 --2- 172.21.15.38:0/2526320392 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x55f0904cbf60 0x55f090571f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:13.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.757+0000 7fbcd6462f40 1 -- 172.21.15.38:0/2526320392 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:1 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-get-keys in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x55f090569570 con 0x55f0904cbf60 2024-09-21T03:40:13.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.757+0000 7fbccff1c640 1 --2- 172.21.15.38:0/2526320392 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fbcb0077570 0x7fbcb0079a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:13.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.757+0000 7fbcd071d640 1 --2- 172.21.15.38:0/2526320392 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x55f0904cbf60 0x55f090571f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:13.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.757+0000 7fbcd071d640 1 --2- 172.21.15.38:0/2526320392 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x55f0904cbf60 0x55f090571f00 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:13.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.757+0000 7fbccff1c640 1 --2- 172.21.15.38:0/2526320392 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fbcb0077570 0x7fbcb0079a80 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fbcc0000c00 tx=0x7fbcc0018040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:13.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.758+0000 7fbcd071d640 1 -- 172.21.15.38:0/2526320392 <== osd.1 v2:172.21.15.38:6816/1491818732 1 ==== osd_op_reply(1 100.00000000.inode [omap-get-keys out=5b] v0'0 uv3 ondisk = 0) ==== 162+0+5 (crc 0 0 0) 0x7fbcc8002020 con 0x55f0904cbf60 2024-09-21T03:40:13.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.758+0000 7fbcd6462f40 1 -- 172.21.15.38:0/2526320392 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x55f0904cbf60 msgr2=0x55f090571f00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:13.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.758+0000 7fbcd6462f40 1 --2- 172.21.15.38:0/2526320392 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x55f0904cbf60 0x55f090571f00 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:13.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.758+0000 7fbcd6462f40 1 -- 172.21.15.38:0/2526320392 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fbcb0077570 msgr2=0x7fbcb0079a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:13.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.758+0000 7fbcd6462f40 1 --2- 172.21.15.38:0/2526320392 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fbcb0077570 0x7fbcb0079a80 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fbcc0000c00 tx=0x7fbcc0018040 comp rx=0 tx=0).stop 2024-09-21T03:40:13.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.758+0000 7fbcd6462f40 1 -- 172.21.15.38:0/2526320392 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f090424e30 msgr2=0x55f0904c7e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:13.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.758+0000 7fbcd6462f40 1 --2- 172.21.15.38:0/2526320392 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f090424e30 0x55f0904c7e20 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7fbcb8025ed0 tx=0x7fbcb804efd0 comp rx=0 tx=0).stop 2024-09-21T03:40:13.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.758+0000 7fbcd071d640 1 -- 172.21.15.38:0/2526320392 reap_dead start 2024-09-21T03:40:13.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.758+0000 7fbcd6462f40 1 -- 172.21.15.38:0/2526320392 shutdown_connections 2024-09-21T03:40:13.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.758+0000 7fbcd6462f40 1 -- 172.21.15.38:0/2526320392 >> 172.21.15.38:0/2526320392 conn(0x55f090317ac0 msgr2=0x55f0904ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:13.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.758+0000 7fbcd6462f40 1 -- 172.21.15.38:0/2526320392 shutdown_connections 2024-09-21T03:40:13.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.758+0000 7fbcd6462f40 1 -- 172.21.15.38:0/2526320392 wait complete. 2024-09-21T03:40:13.763 DEBUG:teuthology.orchestra.run.smithi038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata listomapkeys 1.00000000 2024-09-21T03:40:13.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.816+0000 7f8df042bf40 1 Processor -- start 2024-09-21T03:40:13.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.816+0000 7f8df042bf40 1 -- start start 2024-09-21T03:40:13.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.817+0000 7f8df042bf40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55df88eb3d00 0x55df88f682a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:13.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.817+0000 7f8df042bf40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55df88edfe30 0x55df88f89350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:13.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.817+0000 7f8df042bf40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55df88f6e950 0x55df88f89890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:13.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.817+0000 7f8df042bf40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55df88e01d60 con 0x55df88edfe30 2024-09-21T03:40:13.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.817+0000 7f8df042bf40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55df88e60430 con 0x55df88eb3d00 2024-09-21T03:40:13.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.817+0000 7f8df042bf40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55df88efa1b0 con 0x55df88f6e950 2024-09-21T03:40:13.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.817+0000 7f8de96e4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55df88edfe30 0x55df88f89350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:13.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.817+0000 7f8de96e4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55df88edfe30 0x55df88f89350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54816/0 (socket says 172.21.15.38:54816) 2024-09-21T03:40:13.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.817+0000 7f8de96e4640 1 -- 172.21.15.38:0/3410856415 learned_addr learned my addr 172.21.15.38:0/3410856415 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:13.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.817+0000 7f8de9ee5640 1 --2- 172.21.15.38:0/3410856415 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55df88eb3d00 0x55df88f682a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:13.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.818+0000 7f8dea6e6640 1 --2- 172.21.15.38:0/3410856415 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55df88f6e950 0x55df88f89890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:13.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.818+0000 7f8de96e4640 1 -- 172.21.15.38:0/3410856415 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55df88f6e950 msgr2=0x55df88f89890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:13.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.818+0000 7f8de96e4640 1 --2- 172.21.15.38:0/3410856415 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55df88f6e950 0x55df88f89890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:13.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.818+0000 7f8de96e4640 1 -- 172.21.15.38:0/3410856415 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55df88eb3d00 msgr2=0x55df88f682a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:13.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.818+0000 7f8de96e4640 1 --2- 172.21.15.38:0/3410856415 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55df88eb3d00 0x55df88f682a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:13.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.818+0000 7f8de96e4640 1 -- 172.21.15.38:0/3410856415 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df88ee1d50 con 0x55df88edfe30 2024-09-21T03:40:13.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.818+0000 7f8dea6e6640 1 --2- 172.21.15.38:0/3410856415 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55df88f6e950 0x55df88f89890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:40:13.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.818+0000 7f8de96e4640 1 --2- 172.21.15.38:0/3410856415 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55df88edfe30 0x55df88f89350 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f8de0001c70 tx=0x7f8de000b5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d06fe69383648e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:13.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.818+0000 7f8ddbfff640 1 -- 172.21.15.38:0/3410856415 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8de000bcc0 con 0x55df88edfe30 2024-09-21T03:40:13.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.818+0000 7f8ddbfff640 1 -- 172.21.15.38:0/3410856415 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8de000f400 con 0x55df88edfe30 2024-09-21T03:40:13.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.818+0000 7f8ddbfff640 1 -- 172.21.15.38:0/3410856415 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8de000be60 con 0x55df88edfe30 2024-09-21T03:40:13.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.819+0000 7f8df042bf40 1 -- 172.21.15.38:0/3410856415 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55df88edfe30 msgr2=0x55df88f89350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:13.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.819+0000 7f8df042bf40 1 --2- 172.21.15.38:0/3410856415 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55df88edfe30 0x55df88f89350 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f8de0001c70 tx=0x7f8de000b5b0 comp rx=0 tx=0).stop 2024-09-21T03:40:13.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.819+0000 7f8df042bf40 1 -- 172.21.15.38:0/3410856415 shutdown_connections 2024-09-21T03:40:13.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.819+0000 7f8df042bf40 1 --2- 172.21.15.38:0/3410856415 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55df88f6e950 0x55df88f89890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:13.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.819+0000 7f8df042bf40 1 --2- 172.21.15.38:0/3410856415 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55df88edfe30 0x55df88f89350 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:13.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.819+0000 7f8df042bf40 1 --2- 172.21.15.38:0/3410856415 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55df88eb3d00 0x55df88f682a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:13.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.819+0000 7f8df042bf40 1 -- 172.21.15.38:0/3410856415 >> 172.21.15.38:0/3410856415 conn(0x55df88dd2ac0 msgr2=0x55df88edd940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:13.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.819+0000 7f8df042bf40 1 -- 172.21.15.38:0/3410856415 shutdown_connections 2024-09-21T03:40:13.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.819+0000 7f8df042bf40 1 -- 172.21.15.38:0/3410856415 wait complete. 2024-09-21T03:40:13.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.820+0000 7f8df042bf40 1 Processor -- start 2024-09-21T03:40:13.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.820+0000 7f8df042bf40 1 -- start start 2024-09-21T03:40:13.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.821+0000 7f8df042bf40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55df88eb3d00 0x55df8901f970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:13.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.821+0000 7f8df042bf40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55df88edfe30 0x55df88eaf870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:13.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.821+0000 7f8df042bf40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55df88f6e950 0x55df88eafdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:13.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.821+0000 7f8df042bf40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55df88ee1ff0 con 0x55df88eb3d00 2024-09-21T03:40:13.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.821+0000 7f8df042bf40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55df88edbdd0 con 0x55df88edfe30 2024-09-21T03:40:13.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.821+0000 7f8df042bf40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55df88f85700 con 0x55df88f6e950 2024-09-21T03:40:13.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.821+0000 7f8de9ee5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55df88eb3d00 0x55df8901f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:13.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.821+0000 7f8de9ee5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55df88eb3d00 0x55df8901f970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54826/0 (socket says 172.21.15.38:54826) 2024-09-21T03:40:13.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.821+0000 7f8de9ee5640 1 -- 172.21.15.38:0/703055681 learned_addr learned my addr 172.21.15.38:0/703055681 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:13.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.821+0000 7f8de96e4640 1 --2- 172.21.15.38:0/703055681 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55df88edfe30 0x55df88eaf870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:13.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.821+0000 7f8dea6e6640 1 --2- 172.21.15.38:0/703055681 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55df88f6e950 0x55df88eafdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:13.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.821+0000 7f8de9ee5640 1 -- 172.21.15.38:0/703055681 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55df88f6e950 msgr2=0x55df88eafdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:13.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.821+0000 7f8de9ee5640 1 --2- 172.21.15.38:0/703055681 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55df88f6e950 0x55df88eafdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:13.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.821+0000 7f8de9ee5640 1 -- 172.21.15.38:0/703055681 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55df88edfe30 msgr2=0x55df88eaf870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:13.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.821+0000 7f8de9ee5640 1 --2- 172.21.15.38:0/703055681 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55df88edfe30 0x55df88eaf870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:13.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.821+0000 7f8de9ee5640 1 -- 172.21.15.38:0/703055681 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df89023960 con 0x55df88eb3d00 2024-09-21T03:40:13.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.822+0000 7f8de9ee5640 1 --2- 172.21.15.38:0/703055681 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55df88eb3d00 0x55df8901f970 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f8ddc0044b0 tx=0x7f8ddc00ac20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:13.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.822+0000 7f8ddaffd640 1 -- 172.21.15.38:0/703055681 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ddc01c070 con 0x55df88eb3d00 2024-09-21T03:40:13.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.822+0000 7f8ddaffd640 1 -- 172.21.15.38:0/703055681 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8ddc008d40 con 0x55df88eb3d00 2024-09-21T03:40:13.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.822+0000 7f8df042bf40 1 -- 172.21.15.38:0/703055681 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55df89023c40 con 0x55df88eb3d00 2024-09-21T03:40:13.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.822+0000 7f8ddaffd640 1 -- 172.21.15.38:0/703055681 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ddc0176e0 con 0x55df88eb3d00 2024-09-21T03:40:13.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.822+0000 7f8df042bf40 1 -- 172.21.15.38:0/703055681 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df89024200 con 0x55df88eb3d00 2024-09-21T03:40:13.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.824+0000 7f8ddaffd640 1 -- 172.21.15.38:0/703055681 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f8ddc01f070 con 0x55df88eb3d00 2024-09-21T03:40:13.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.824+0000 7f8ddaffd640 1 --2- 172.21.15.38:0/703055681 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8dcc0775a0 0x7f8dcc079ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:13.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.824+0000 7f8ddaffd640 1 -- 172.21.15.38:0/703055681 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6187+0+0 (secure 0 0 0) 0x7f8ddc099f50 con 0x55df88eb3d00 2024-09-21T03:40:13.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.824+0000 7f8de96e4640 1 --2- 172.21.15.38:0/703055681 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8dcc0775a0 0x7f8dcc079ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:13.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.824+0000 7f8df042bf40 1 --2- 172.21.15.38:0/703055681 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x55df88f6dc30 0x55df88f86840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:13.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.824+0000 7f8df042bf40 1 -- 172.21.15.38:0/703055681 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:1 13.1f 13:ff5b34d6:::1.00000000:head [omap-get-keys in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x55df88f86d80 con 0x55df88f6dc30 2024-09-21T03:40:13.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.824+0000 7f8dea6e6640 1 --2- 172.21.15.38:0/703055681 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x55df88f6dc30 0x55df88f86840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:13.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.825+0000 7f8de96e4640 1 --2- 172.21.15.38:0/703055681 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8dcc0775a0 0x7f8dcc079ab0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x55df88f68080 tx=0x7f8de000e000 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:13.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.825+0000 7f8dea6e6640 1 --2- 172.21.15.38:0/703055681 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x55df88f6dc30 0x55df88f86840 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:13.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.825+0000 7f8dea6e6640 1 -- 172.21.15.38:0/703055681 <== osd.1 v2:172.21.15.38:6816/1491818732 1 ==== osd_op_reply(1 1.00000000 [omap-get-keys out=20b] v0'0 uv6 ondisk = 0) ==== 154+0+20 (crc 0 0 0) 0x7f8de400b040 con 0x55df88f6dc30 2024-09-21T03:40:13.829 INFO:teuthology.orchestra.run.smithi038.stdout:subdir_head 2024-09-21T03:40:13.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.825+0000 7f8df042bf40 1 -- 172.21.15.38:0/703055681 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x55df88f6dc30 msgr2=0x55df88f86840 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:13.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.825+0000 7f8df042bf40 1 --2- 172.21.15.38:0/703055681 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x55df88f6dc30 0x55df88f86840 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:13.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.825+0000 7f8df042bf40 1 -- 172.21.15.38:0/703055681 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8dcc0775a0 msgr2=0x7f8dcc079ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:13.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.825+0000 7f8df042bf40 1 --2- 172.21.15.38:0/703055681 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8dcc0775a0 0x7f8dcc079ab0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x55df88f68080 tx=0x7f8de000e000 comp rx=0 tx=0).stop 2024-09-21T03:40:13.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.825+0000 7f8df042bf40 1 -- 172.21.15.38:0/703055681 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55df88eb3d00 msgr2=0x55df8901f970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:13.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.825+0000 7f8df042bf40 1 --2- 172.21.15.38:0/703055681 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55df88eb3d00 0x55df8901f970 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f8ddc0044b0 tx=0x7f8ddc00ac20 comp rx=0 tx=0).stop 2024-09-21T03:40:13.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.825+0000 7f8dea6e6640 1 -- 172.21.15.38:0/703055681 reap_dead start 2024-09-21T03:40:13.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.825+0000 7f8df042bf40 1 -- 172.21.15.38:0/703055681 shutdown_connections 2024-09-21T03:40:13.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.825+0000 7f8df042bf40 1 -- 172.21.15.38:0/703055681 >> 172.21.15.38:0/703055681 conn(0x55df88dd2ac0 msgr2=0x55df88f84930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:13.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.825+0000 7f8df042bf40 1 -- 172.21.15.38:0/703055681 shutdown_connections 2024-09-21T03:40:13.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.826+0000 7f8df042bf40 1 -- 172.21.15.38:0/703055681 wait complete. 2024-09-21T03:40:13.831 DEBUG:teuthology.orchestra.run.smithi038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata stat 10000000000.00000000 2024-09-21T03:40:13.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.884+0000 7f54ae618f40 1 Processor -- start 2024-09-21T03:40:13.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.884+0000 7f54ae618f40 1 -- start start 2024-09-21T03:40:13.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.885+0000 7f54ae618f40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562159c97d00 0x562159d6da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:13.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.885+0000 7f54ae618f40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562159d52950 0x562159d6dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:13.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.885+0000 7f54ae618f40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562159d6d670 0x562159cc3ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:13.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.885+0000 7f54ae618f40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x562159ce0180 con 0x562159c97d00 2024-09-21T03:40:13.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.885+0000 7f54ae618f40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x562159be5d60 con 0x562159d6d670 2024-09-21T03:40:13.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.885+0000 7f54ae618f40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x562159ce0460 con 0x562159d52950 2024-09-21T03:40:13.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.885+0000 7f54a7464640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562159c97d00 0x562159d6da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:13.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.885+0000 7f54a7464640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562159c97d00 0x562159d6da80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54856/0 (socket says 172.21.15.38:54856) 2024-09-21T03:40:13.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.885+0000 7f54a7464640 1 -- 172.21.15.38:0/110414787 learned_addr learned my addr 172.21.15.38:0/110414787 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:13.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.885+0000 7f54a7c65640 1 --2- 172.21.15.38:0/110414787 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562159d6d670 0x562159cc3ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:13.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.885+0000 7f54a6c63640 1 --2- 172.21.15.38:0/110414787 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562159d52950 0x562159d6dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:13.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.885+0000 7f54a7464640 1 -- 172.21.15.38:0/110414787 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562159d52950 msgr2=0x562159d6dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:13.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.885+0000 7f54a7464640 1 --2- 172.21.15.38:0/110414787 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562159d52950 0x562159d6dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:13.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.885+0000 7f54a7464640 1 -- 172.21.15.38:0/110414787 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562159d6d670 msgr2=0x562159cc3ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:13.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.885+0000 7f54a7464640 1 --2- 172.21.15.38:0/110414787 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562159d6d670 0x562159cc3ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:13.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.885+0000 7f54a7464640 1 -- 172.21.15.38:0/110414787 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562159d4ffc0 con 0x562159c97d00 2024-09-21T03:40:13.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.885+0000 7f54a6c63640 1 --2- 172.21.15.38:0/110414787 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562159d52950 0x562159d6dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:40:13.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.886+0000 7f54a7464640 1 --2- 172.21.15.38:0/110414787 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562159c97d00 0x562159d6da80 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f5498025e80 tx=0x7f5498002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc9eafa8d9c5a43b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:13.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.886+0000 7f54a5c61640 1 -- 172.21.15.38:0/110414787 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f549804d8e0 con 0x562159c97d00 2024-09-21T03:40:13.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.886+0000 7f54a5c61640 1 -- 172.21.15.38:0/110414787 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f549804dd70 con 0x562159c97d00 2024-09-21T03:40:13.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.886+0000 7f54a5c61640 1 -- 172.21.15.38:0/110414787 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54980566d0 con 0x562159c97d00 2024-09-21T03:40:13.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.886+0000 7f54ae618f40 1 -- 172.21.15.38:0/110414787 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562159c97d00 msgr2=0x562159d6da80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:13.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.886+0000 7f54ae618f40 1 --2- 172.21.15.38:0/110414787 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562159c97d00 0x562159d6da80 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f5498025e80 tx=0x7f5498002fb0 comp rx=0 tx=0).stop 2024-09-21T03:40:13.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.886+0000 7f54ae618f40 1 -- 172.21.15.38:0/110414787 shutdown_connections 2024-09-21T03:40:13.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.886+0000 7f54ae618f40 1 --2- 172.21.15.38:0/110414787 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562159d6d670 0x562159cc3ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:13.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.886+0000 7f54ae618f40 1 --2- 172.21.15.38:0/110414787 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562159d52950 0x562159d6dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:13.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.886+0000 7f54ae618f40 1 --2- 172.21.15.38:0/110414787 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562159c97d00 0x562159d6da80 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:13.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.887+0000 7f54ae618f40 1 -- 172.21.15.38:0/110414787 >> 172.21.15.38:0/110414787 conn(0x562159bb6ac0 msgr2=0x562159d4e610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:13.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.887+0000 7f54ae618f40 1 -- 172.21.15.38:0/110414787 shutdown_connections 2024-09-21T03:40:13.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.887+0000 7f54ae618f40 1 -- 172.21.15.38:0/110414787 wait complete. 2024-09-21T03:40:13.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.887+0000 7f54ae618f40 1 Processor -- start 2024-09-21T03:40:13.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.888+0000 7f54ae618f40 1 -- start start 2024-09-21T03:40:13.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.889+0000 7f54ae618f40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562159c97d00 0x562159cdb500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:13.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.889+0000 7f54ae618f40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562159d52950 0x562159d6a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:13.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.889+0000 7f54ae618f40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562159d6d670 0x562159d6a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:13.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.889+0000 7f54ae618f40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x562159d69420 con 0x562159c97d00 2024-09-21T03:40:13.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.889+0000 7f54ae618f40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x562159d69690 con 0x562159d52950 2024-09-21T03:40:13.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.889+0000 7f54ae618f40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x562159cdc4a0 con 0x562159d6d670 2024-09-21T03:40:13.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.889+0000 7f54a7464640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562159c97d00 0x562159cdb500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:13.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.889+0000 7f54a7464640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562159c97d00 0x562159cdb500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54870/0 (socket says 172.21.15.38:54870) 2024-09-21T03:40:13.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.889+0000 7f54a7464640 1 -- 172.21.15.38:0/2469987908 learned_addr learned my addr 172.21.15.38:0/2469987908 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:13.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.889+0000 7f54a6c63640 1 --2- 172.21.15.38:0/2469987908 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562159d52950 0x562159d6a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:13.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.889+0000 7f54a7c65640 1 --2- 172.21.15.38:0/2469987908 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562159d6d670 0x562159d6a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:13.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.889+0000 7f54a7464640 1 -- 172.21.15.38:0/2469987908 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562159d6d670 msgr2=0x562159d6a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:13.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.889+0000 7f54a7464640 1 --2- 172.21.15.38:0/2469987908 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562159d6d670 0x562159d6a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:13.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.889+0000 7f54a7464640 1 -- 172.21.15.38:0/2469987908 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562159d52950 msgr2=0x562159d6a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:13.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.889+0000 7f54a7464640 1 --2- 172.21.15.38:0/2469987908 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562159d52950 0x562159d6a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:13.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.889+0000 7f54a7464640 1 -- 172.21.15.38:0/2469987908 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562159d6e300 con 0x562159c97d00 2024-09-21T03:40:13.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.890+0000 7f54a7464640 1 --2- 172.21.15.38:0/2469987908 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562159c97d00 0x562159cdb500 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f5498025e50 tx=0x7f54980575e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:13.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.890+0000 7f54a4c5f640 1 -- 172.21.15.38:0/2469987908 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5498067040 con 0x562159c97d00 2024-09-21T03:40:13.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.890+0000 7f54a4c5f640 1 -- 172.21.15.38:0/2469987908 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5498070070 con 0x562159c97d00 2024-09-21T03:40:13.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.890+0000 7f54ae618f40 1 -- 172.21.15.38:0/2469987908 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x562159d4c570 con 0x562159c97d00 2024-09-21T03:40:13.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.890+0000 7f54a4c5f640 1 -- 172.21.15.38:0/2469987908 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5498057d30 con 0x562159c97d00 2024-09-21T03:40:13.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.890+0000 7f54ae618f40 1 -- 172.21.15.38:0/2469987908 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562159d6ae60 con 0x562159c97d00 2024-09-21T03:40:13.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.892+0000 7f54a4c5f640 1 -- 172.21.15.38:0/2469987908 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f5498060070 con 0x562159c97d00 2024-09-21T03:40:13.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.892+0000 7f54a4c5f640 1 --2- 172.21.15.38:0/2469987908 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5494077570 0x7f5494079a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:13.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.892+0000 7f54a4c5f640 1 -- 172.21.15.38:0/2469987908 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6187+0+0 (secure 0 0 0) 0x7f54980dece0 con 0x562159c97d00 2024-09-21T03:40:13.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.892+0000 7f54a6c63640 1 --2- 172.21.15.38:0/2469987908 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5494077570 0x7f5494079a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:13.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.892+0000 7f54ae618f40 1 --2- 172.21.15.38:0/2469987908 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x562159d4be80 0x562159e07f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:13.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.892+0000 7f54ae618f40 1 -- 172.21.15.38:0/2469987908 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:1 13.10 13:0cf6ad0f:::10000000000.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x562159cdba40 con 0x562159d4be80 2024-09-21T03:40:13.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.893+0000 7f54a7c65640 1 --2- 172.21.15.38:0/2469987908 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x562159d4be80 0x562159e07f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:13.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.893+0000 7f54a6c63640 1 --2- 172.21.15.38:0/2469987908 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5494077570 0x7f5494079a80 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x562159d6d0f0 tx=0x7f549c004940 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:13.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.893+0000 7f54a7c65640 1 --2- 172.21.15.38:0/2469987908 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x562159d4be80 0x562159e07f20 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:13.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.893+0000 7f54a7c65640 1 -- 172.21.15.38:0/2469987908 <== osd.7 v2:172.21.15.195:6818/1606106342 1 ==== osd_op_reply(1 10000000000.00000000 [stat out=16b] v0'0 uv10 ondisk = 0) ==== 164+0+16 (crc 0 0 0) 0x7f54a0003040 con 0x562159d4be80 2024-09-21T03:40:13.898 INFO:teuthology.orchestra.run.smithi038.stdout:cephfs_metadata/10000000000.00000000 mtime 2024-09-21T03:40:11.000000+0000, size 0 2024-09-21T03:40:13.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.893+0000 7f54ae618f40 1 -- 172.21.15.38:0/2469987908 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x562159d4be80 msgr2=0x562159e07f20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:13.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.893+0000 7f54ae618f40 1 --2- 172.21.15.38:0/2469987908 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x562159d4be80 0x562159e07f20 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:13.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.893+0000 7f54ae618f40 1 -- 172.21.15.38:0/2469987908 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5494077570 msgr2=0x7f5494079a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:13.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.893+0000 7f54ae618f40 1 --2- 172.21.15.38:0/2469987908 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5494077570 0x7f5494079a80 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x562159d6d0f0 tx=0x7f549c004940 comp rx=0 tx=0).stop 2024-09-21T03:40:13.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.893+0000 7f54ae618f40 1 -- 172.21.15.38:0/2469987908 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562159c97d00 msgr2=0x562159cdb500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:13.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.893+0000 7f54ae618f40 1 --2- 172.21.15.38:0/2469987908 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562159c97d00 0x562159cdb500 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f5498025e50 tx=0x7f54980575e0 comp rx=0 tx=0).stop 2024-09-21T03:40:13.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.893+0000 7f54a7c65640 1 -- 172.21.15.38:0/2469987908 reap_dead start 2024-09-21T03:40:13.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.893+0000 7f54ae618f40 1 -- 172.21.15.38:0/2469987908 shutdown_connections 2024-09-21T03:40:13.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.893+0000 7f54ae618f40 1 -- 172.21.15.38:0/2469987908 >> 172.21.15.38:0/2469987908 conn(0x562159bb6ac0 msgr2=0x562159cddf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:13.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.894+0000 7f54ae618f40 1 -- 172.21.15.38:0/2469987908 shutdown_connections 2024-09-21T03:40:13.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.894+0000 7f54ae618f40 1 -- 172.21.15.38:0/2469987908 wait complete. 2024-09-21T03:40:13.901 DEBUG:teuthology.orchestra.run.smithi038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata stat 600.00000000 2024-09-21T03:40:13.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.954+0000 7fe18a3d9f40 1 Processor -- start 2024-09-21T03:40:13.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.954+0000 7fe18a3d9f40 1 -- start start 2024-09-21T03:40:13.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.956+0000 7fe18a3d9f40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5588acd5fab0 0x5588acec9910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:13.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.956+0000 7fe18a3d9f40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5588acd5fec0 0x5588acec9e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:13.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.956+0000 7fe18a3d9f40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5588aceca390 0x5588ace34ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:13.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.956+0000 7fe18a3d9f40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5588accb1d60 con 0x5588acd5fab0 2024-09-21T03:40:13.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.956+0000 7fe18a3d9f40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5588acd10430 con 0x5588aceca390 2024-09-21T03:40:13.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.956+0000 7fe18a3d9f40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x5588acdac460 con 0x5588acd5fec0 2024-09-21T03:40:13.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.956+0000 7fe183e93640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5588acd5fab0 0x5588acec9910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:13.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.956+0000 7fe183e93640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5588acd5fab0 0x5588acec9910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54888/0 (socket says 172.21.15.38:54888) 2024-09-21T03:40:13.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.956+0000 7fe183e93640 1 -- 172.21.15.38:0/4040114000 learned_addr learned my addr 172.21.15.38:0/4040114000 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:13.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.956+0000 7fe183692640 1 --2- 172.21.15.38:0/4040114000 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5588acd5fec0 0x5588acec9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:13.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.956+0000 7fe184694640 1 --2- 172.21.15.38:0/4040114000 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5588aceca390 0x5588ace34ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:13.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.956+0000 7fe183e93640 1 -- 172.21.15.38:0/4040114000 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5588acd5fec0 msgr2=0x5588acec9e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:13.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.956+0000 7fe183e93640 1 --2- 172.21.15.38:0/4040114000 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5588acd5fec0 0x5588acec9e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:13.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.956+0000 7fe183e93640 1 -- 172.21.15.38:0/4040114000 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5588aceca390 msgr2=0x5588ace34ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:13.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.956+0000 7fe183e93640 1 --2- 172.21.15.38:0/4040114000 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5588aceca390 0x5588ace34ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:13.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.956+0000 7fe183e93640 1 -- 172.21.15.38:0/4040114000 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588ace049d0 con 0x5588acd5fab0 2024-09-21T03:40:13.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.956+0000 7fe183e93640 1 --2- 172.21.15.38:0/4040114000 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5588acd5fab0 0x5588acec9910 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7fe174001160 tx=0x7fe174009a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ea5f2f5c74c7e33 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:13.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.956+0000 7fe182690640 1 -- 172.21.15.38:0/4040114000 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe174014030 con 0x5588acd5fab0 2024-09-21T03:40:13.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.956+0000 7fe182690640 1 -- 172.21.15.38:0/4040114000 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe17400f6d0 con 0x5588acd5fab0 2024-09-21T03:40:13.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.956+0000 7fe182690640 1 -- 172.21.15.38:0/4040114000 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe17400fb40 con 0x5588acd5fab0 2024-09-21T03:40:13.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.957+0000 7fe18a3d9f40 1 -- 172.21.15.38:0/4040114000 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5588acd5fab0 msgr2=0x5588acec9910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:13.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.957+0000 7fe18a3d9f40 1 --2- 172.21.15.38:0/4040114000 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5588acd5fab0 0x5588acec9910 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7fe174001160 tx=0x7fe174009a00 comp rx=0 tx=0).stop 2024-09-21T03:40:13.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.957+0000 7fe18a3d9f40 1 -- 172.21.15.38:0/4040114000 shutdown_connections 2024-09-21T03:40:13.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.957+0000 7fe18a3d9f40 1 --2- 172.21.15.38:0/4040114000 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5588aceca390 0x5588ace34ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:13.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.957+0000 7fe18a3d9f40 1 --2- 172.21.15.38:0/4040114000 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5588acd5fec0 0x5588acec9e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:13.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.957+0000 7fe18a3d9f40 1 --2- 172.21.15.38:0/4040114000 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5588acd5fab0 0x5588acec9910 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:13.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.957+0000 7fe18a3d9f40 1 -- 172.21.15.38:0/4040114000 >> 172.21.15.38:0/4040114000 conn(0x5588acc82ac0 msgr2=0x5588ace372b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:13.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.957+0000 7fe18a3d9f40 1 -- 172.21.15.38:0/4040114000 shutdown_connections 2024-09-21T03:40:13.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.957+0000 7fe18a3d9f40 1 -- 172.21.15.38:0/4040114000 wait complete. 2024-09-21T03:40:13.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.958+0000 7fe18a3d9f40 1 Processor -- start 2024-09-21T03:40:13.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.958+0000 7fe18a3d9f40 1 -- start start 2024-09-21T03:40:13.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.959+0000 7fe18a3d9f40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5588acd5fab0 0x5588acec6a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:13.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.959+0000 7fe18a3d9f40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5588acd5fec0 0x5588ace1cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:13.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.959+0000 7fe18a3d9f40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5588aceca390 0x5588ace1d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:13.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.959+0000 7fe18a3d9f40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5588ace04770 con 0x5588aceca390 2024-09-21T03:40:13.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.959+0000 7fe18a3d9f40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5588aceca7a0 con 0x5588acd5fec0 2024-09-21T03:40:13.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.959+0000 7fe18a3d9f40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x5588ace05480 con 0x5588acd5fab0 2024-09-21T03:40:13.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.959+0000 7fe184694640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5588aceca390 0x5588ace1d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:13.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.959+0000 7fe184694640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5588aceca390 0x5588ace1d2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54904/0 (socket says 172.21.15.38:54904) 2024-09-21T03:40:13.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.959+0000 7fe184694640 1 -- 172.21.15.38:0/3500015876 learned_addr learned my addr 172.21.15.38:0/3500015876 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:13.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.960+0000 7fe183692640 1 --2- 172.21.15.38:0/3500015876 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5588acd5fec0 0x5588ace1cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:13.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.960+0000 7fe183e93640 1 --2- 172.21.15.38:0/3500015876 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5588acd5fab0 0x5588acec6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:13.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.960+0000 7fe184694640 1 -- 172.21.15.38:0/3500015876 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5588acd5fab0 msgr2=0x5588acec6a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:13.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.960+0000 7fe184694640 1 --2- 172.21.15.38:0/3500015876 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5588acd5fab0 0x5588acec6a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:13.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.960+0000 7fe184694640 1 -- 172.21.15.38:0/3500015876 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5588acd5fec0 msgr2=0x5588ace1cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:13.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.960+0000 7fe184694640 1 --2- 172.21.15.38:0/3500015876 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5588acd5fec0 0x5588ace1cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:13.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.960+0000 7fe184694640 1 -- 172.21.15.38:0/3500015876 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588acec8940 con 0x5588aceca390 2024-09-21T03:40:13.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.960+0000 7fe183692640 1 --2- 172.21.15.38:0/3500015876 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5588acd5fec0 0x5588ace1cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:40:13.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.960+0000 7fe184694640 1 --2- 172.21.15.38:0/3500015876 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5588aceca390 0x5588ace1d2d0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x5588acda9450 tx=0x7fe17c00eb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:13.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.960+0000 7fe18168e640 1 -- 172.21.15.38:0/3500015876 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe17c00ba20 con 0x5588aceca390 2024-09-21T03:40:13.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.960+0000 7fe18168e640 1 -- 172.21.15.38:0/3500015876 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe17c00bbe0 con 0x5588aceca390 2024-09-21T03:40:13.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.960+0000 7fe18a3d9f40 1 -- 172.21.15.38:0/3500015876 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5588ace1c030 con 0x5588aceca390 2024-09-21T03:40:13.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.960+0000 7fe18168e640 1 -- 172.21.15.38:0/3500015876 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe17c01a420 con 0x5588aceca390 2024-09-21T03:40:13.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.960+0000 7fe18a3d9f40 1 -- 172.21.15.38:0/3500015876 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5588ace1a480 con 0x5588aceca390 2024-09-21T03:40:13.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.962+0000 7fe18168e640 1 -- 172.21.15.38:0/3500015876 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fe17c022390 con 0x5588aceca390 2024-09-21T03:40:13.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.962+0000 7fe18168e640 1 --2- 172.21.15.38:0/3500015876 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fe1700775a0 0x7fe170079ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:13.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.962+0000 7fe18168e640 1 -- 172.21.15.38:0/3500015876 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6187+0+0 (secure 0 0 0) 0x7fe17c09aeb0 con 0x5588aceca390 2024-09-21T03:40:13.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.962+0000 7fe183e93640 1 --2- 172.21.15.38:0/3500015876 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fe1700775a0 0x7fe170079ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:13.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.962+0000 7fe18a3d9f40 1 --2- 172.21.15.38:0/3500015876 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x5588ace1d810 0x5588ace1b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:13.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.962+0000 7fe18a3d9f40 1 -- 172.21.15.38:0/3500015876 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:1 13.10 13:0d82a743:::600.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x5588acec6fb0 con 0x5588ace1d810 2024-09-21T03:40:13.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.963+0000 7fe183692640 1 --2- 172.21.15.38:0/3500015876 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x5588ace1d810 0x5588ace1b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:13.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.963+0000 7fe183e93640 1 --2- 172.21.15.38:0/3500015876 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fe1700775a0 0x7fe170079ab0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fe174001160 tx=0x7fe174011040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:13.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.963+0000 7fe183692640 1 --2- 172.21.15.38:0/3500015876 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x5588ace1d810 0x5588ace1b850 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:13.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.963+0000 7fe183692640 1 -- 172.21.15.38:0/3500015876 <== osd.7 v2:172.21.15.195:6818/1606106342 1 ==== osd_op_reply(1 600.00000000 [stat out=16b] v0'0 uv2 ondisk = 0) ==== 156+0+16 (crc 0 0 0) 0x7fe17805d040 con 0x5588ace1d810 2024-09-21T03:40:13.968 INFO:teuthology.orchestra.run.smithi038.stdout:cephfs_metadata/600.00000000 mtime 2024-09-21T03:40:02.000000+0000, size 0 2024-09-21T03:40:13.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.963+0000 7fe18a3d9f40 1 -- 172.21.15.38:0/3500015876 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x5588ace1d810 msgr2=0x5588ace1b850 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:13.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.963+0000 7fe18a3d9f40 1 --2- 172.21.15.38:0/3500015876 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x5588ace1d810 0x5588ace1b850 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:13.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.964+0000 7fe18a3d9f40 1 -- 172.21.15.38:0/3500015876 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fe1700775a0 msgr2=0x7fe170079ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:13.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.964+0000 7fe18a3d9f40 1 --2- 172.21.15.38:0/3500015876 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fe1700775a0 0x7fe170079ab0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fe174001160 tx=0x7fe174011040 comp rx=0 tx=0).stop 2024-09-21T03:40:13.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.964+0000 7fe18a3d9f40 1 -- 172.21.15.38:0/3500015876 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5588aceca390 msgr2=0x5588ace1d2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:13.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.964+0000 7fe18a3d9f40 1 --2- 172.21.15.38:0/3500015876 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5588aceca390 0x5588ace1d2d0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x5588acda9450 tx=0x7fe17c00eb60 comp rx=0 tx=0).stop 2024-09-21T03:40:13.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.964+0000 7fe184694640 1 -- 172.21.15.38:0/3500015876 reap_dead start 2024-09-21T03:40:13.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.964+0000 7fe18a3d9f40 1 -- 172.21.15.38:0/3500015876 shutdown_connections 2024-09-21T03:40:13.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.964+0000 7fe18a3d9f40 1 -- 172.21.15.38:0/3500015876 >> 172.21.15.38:0/3500015876 conn(0x5588acc82ac0 msgr2=0x5588ace37440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:13.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.964+0000 7fe18a3d9f40 1 -- 172.21.15.38:0/3500015876 shutdown_connections 2024-09-21T03:40:13.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:13.964+0000 7fe18a3d9f40 1 -- 172.21.15.38:0/3500015876 wait complete. 2024-09-21T03:40:13.970 DEBUG:teuthology.orchestra.run.smithi038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata stat 1.00000000.inode 2024-09-21T03:40:14.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.024+0000 7fad7a91cf40 1 Processor -- start 2024-09-21T03:40:14.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.024+0000 7fad7a91cf40 1 -- start start 2024-09-21T03:40:14.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.025+0000 7fad7a91cf40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55da097aed00 0x55da0987fc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:14.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.025+0000 7fad7a91cf40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55da09869950 0x55da097f2f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:14.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.025+0000 7fad7a91cf40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55da09884670 0x55da097f3470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:14.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.025+0000 7fad7a91cf40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55da096fcd60 con 0x55da097aed00 2024-09-21T03:40:14.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.025+0000 7fad7a91cf40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55da0975b430 con 0x55da09884670 2024-09-21T03:40:14.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.025+0000 7fad7a91cf40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55da097f7460 con 0x55da09869950 2024-09-21T03:40:14.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.025+0000 7fad743d6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55da097aed00 0x55da0987fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:14.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.025+0000 7fad743d6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55da097aed00 0x55da0987fc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54930/0 (socket says 172.21.15.38:54930) 2024-09-21T03:40:14.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.025+0000 7fad743d6640 1 -- 172.21.15.38:0/2292395688 learned_addr learned my addr 172.21.15.38:0/2292395688 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:14.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.025+0000 7fad743d6640 1 -- 172.21.15.38:0/2292395688 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55da09869950 msgr2=0x55da097f2f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:14.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.025+0000 7fad743d6640 1 --2- 172.21.15.38:0/2292395688 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55da09869950 0x55da097f2f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:14.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.025+0000 7fad74bd7640 1 --2- 172.21.15.38:0/2292395688 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55da09884670 0x55da097f3470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:14.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.025+0000 7fad743d6640 1 -- 172.21.15.38:0/2292395688 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55da09884670 msgr2=0x55da097f3470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:14.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.025+0000 7fad743d6640 1 --2- 172.21.15.38:0/2292395688 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55da09884670 0x55da097f3470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:14.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.025+0000 7fad743d6640 1 -- 172.21.15.38:0/2292395688 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da09883ae0 con 0x55da097aed00 2024-09-21T03:40:14.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.025+0000 7fad743d6640 1 --2- 172.21.15.38:0/2292395688 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55da097aed00 0x55da0987fc70 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7fad64000c00 tx=0x7fad64002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=542f88caa8be86fc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:14.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.026+0000 7fad72bd3640 1 -- 172.21.15.38:0/2292395688 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad6404dd50 con 0x55da097aed00 2024-09-21T03:40:14.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.026+0000 7fad72bd3640 1 -- 172.21.15.38:0/2292395688 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fad64056470 con 0x55da097aed00 2024-09-21T03:40:14.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.026+0000 7fad72bd3640 1 -- 172.21.15.38:0/2292395688 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad640568a0 con 0x55da097aed00 2024-09-21T03:40:14.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.026+0000 7fad7a91cf40 1 -- 172.21.15.38:0/2292395688 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55da097aed00 msgr2=0x55da0987fc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:14.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.026+0000 7fad7a91cf40 1 --2- 172.21.15.38:0/2292395688 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55da097aed00 0x55da0987fc70 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7fad64000c00 tx=0x7fad64002fb0 comp rx=0 tx=0).stop 2024-09-21T03:40:14.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.026+0000 7fad7a91cf40 1 -- 172.21.15.38:0/2292395688 shutdown_connections 2024-09-21T03:40:14.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.026+0000 7fad7a91cf40 1 --2- 172.21.15.38:0/2292395688 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55da09884670 0x55da097f3470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:14.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.026+0000 7fad7a91cf40 1 --2- 172.21.15.38:0/2292395688 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55da09869950 0x55da097f2f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:14.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.026+0000 7fad7a91cf40 1 --2- 172.21.15.38:0/2292395688 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55da097aed00 0x55da0987fc70 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:14.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.026+0000 7fad7a91cf40 1 -- 172.21.15.38:0/2292395688 >> 172.21.15.38:0/2292395688 conn(0x55da096cdac0 msgr2=0x55da098655f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:14.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.026+0000 7fad7a91cf40 1 -- 172.21.15.38:0/2292395688 shutdown_connections 2024-09-21T03:40:14.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.026+0000 7fad7a91cf40 1 -- 172.21.15.38:0/2292395688 wait complete. 2024-09-21T03:40:14.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.027+0000 7fad7a91cf40 1 Processor -- start 2024-09-21T03:40:14.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.027+0000 7fad7a91cf40 1 -- start start 2024-09-21T03:40:14.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.028+0000 7fad7a91cf40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55da097aed00 0x55da0987e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:14.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.028+0000 7fad7a91cf40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55da09869950 0x55da097ec910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:14.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.028+0000 7fad7a91cf40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55da09884670 0x55da097ece50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:14.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.028+0000 7fad7a91cf40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55da09866fc0 con 0x55da097aed00 2024-09-21T03:40:14.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.028+0000 7fad7a91cf40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55da09880230 con 0x55da09884670 2024-09-21T03:40:14.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.029+0000 7fad7a91cf40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55da097ed6f0 con 0x55da09869950 2024-09-21T03:40:14.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.029+0000 7fad743d6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55da097aed00 0x55da0987e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:14.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.029+0000 7fad73bd5640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55da09869950 0x55da097ec910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:14.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.029+0000 7fad743d6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55da097aed00 0x55da0987e1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54942/0 (socket says 172.21.15.38:54942) 2024-09-21T03:40:14.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.029+0000 7fad74bd7640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55da09884670 0x55da097ece50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:14.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.029+0000 7fad743d6640 1 -- 172.21.15.38:0/253588452 learned_addr learned my addr 172.21.15.38:0/253588452 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:14.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.029+0000 7fad743d6640 1 -- 172.21.15.38:0/253588452 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55da09869950 msgr2=0x55da097ec910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:14.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.029+0000 7fad743d6640 1 --2- 172.21.15.38:0/253588452 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55da09869950 0x55da097ec910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:14.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.029+0000 7fad743d6640 1 -- 172.21.15.38:0/253588452 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55da09884670 msgr2=0x55da097ece50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:14.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.029+0000 7fad743d6640 1 --2- 172.21.15.38:0/253588452 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55da09884670 0x55da097ece50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:14.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.029+0000 7fad743d6640 1 -- 172.21.15.38:0/253588452 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da09864090 con 0x55da097aed00 2024-09-21T03:40:14.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.029+0000 7fad74bd7640 1 --2- 172.21.15.38:0/253588452 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55da09884670 0x55da097ece50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:40:14.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.029+0000 7fad743d6640 1 --2- 172.21.15.38:0/253588452 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55da097aed00 0x55da0987e1d0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7fad64046560 tx=0x7fad640575e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:14.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.029+0000 7fad71bd1640 1 -- 172.21.15.38:0/253588452 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad64067040 con 0x55da097aed00 2024-09-21T03:40:14.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.029+0000 7fad71bd1640 1 -- 172.21.15.38:0/253588452 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fad640579a0 con 0x55da097aed00 2024-09-21T03:40:14.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.029+0000 7fad7a91cf40 1 -- 172.21.15.38:0/253588452 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55da097f3690 con 0x55da097aed00 2024-09-21T03:40:14.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.029+0000 7fad71bd1640 1 -- 172.21.15.38:0/253588452 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad64057df0 con 0x55da097aed00 2024-09-21T03:40:14.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.030+0000 7fad7a91cf40 1 -- 172.21.15.38:0/253588452 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da0987be20 con 0x55da097aed00 2024-09-21T03:40:14.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.031+0000 7fad71bd1640 1 -- 172.21.15.38:0/253588452 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fad64072030 con 0x55da097aed00 2024-09-21T03:40:14.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.032+0000 7fad71bd1640 1 --2- 172.21.15.38:0/253588452 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fad60077570 0x7fad60079a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:14.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.032+0000 7fad71bd1640 1 -- 172.21.15.38:0/253588452 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6187+0+0 (secure 0 0 0) 0x7fad640de890 con 0x55da097aed00 2024-09-21T03:40:14.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.032+0000 7fad7a91cf40 1 --2- 172.21.15.38:0/253588452 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x55da098645e0 0x55da0987b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:14.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.032+0000 7fad73bd5640 1 --2- 172.21.15.38:0/253588452 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fad60077570 0x7fad60079a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:14.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.032+0000 7fad7a91cf40 1 -- 172.21.15.38:0/253588452 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:1 13.14 13:287034c4:::1.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x55da0987e710 con 0x55da098645e0 2024-09-21T03:40:14.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.032+0000 7fad74bd7640 1 --2- 172.21.15.38:0/253588452 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x55da098645e0 0x55da0987b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:14.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.033+0000 7fad73bd5640 1 --2- 172.21.15.38:0/253588452 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fad60077570 0x7fad60079a80 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x55da0987f7d0 tx=0x7fad6800d040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:14.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.033+0000 7fad74bd7640 1 --2- 172.21.15.38:0/253588452 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x55da098645e0 0x55da0987b280 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:14.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.033+0000 7fad74bd7640 1 -- 172.21.15.38:0/253588452 <== osd.6 v2:172.21.15.195:6826/4055825675 1 ==== osd_op_reply(1 1.00000000.inode [stat out=16b] v0'0 uv15 ondisk = 0) ==== 160+0+16 (crc 0 0 0) 0x7fad6c003ba0 con 0x55da098645e0 2024-09-21T03:40:14.036 INFO:teuthology.orchestra.run.smithi038.stdout:cephfs_metadata/1.00000000.inode mtime 2024-09-21T03:40:11.000000+0000, size 582 2024-09-21T03:40:14.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.033+0000 7fad7a91cf40 1 -- 172.21.15.38:0/253588452 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x55da098645e0 msgr2=0x55da0987b280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:14.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.033+0000 7fad7a91cf40 1 --2- 172.21.15.38:0/253588452 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x55da098645e0 0x55da0987b280 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:14.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.033+0000 7fad7a91cf40 1 -- 172.21.15.38:0/253588452 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fad60077570 msgr2=0x7fad60079a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:14.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.033+0000 7fad7a91cf40 1 --2- 172.21.15.38:0/253588452 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fad60077570 0x7fad60079a80 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x55da0987f7d0 tx=0x7fad6800d040 comp rx=0 tx=0).stop 2024-09-21T03:40:14.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.033+0000 7fad7a91cf40 1 -- 172.21.15.38:0/253588452 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55da097aed00 msgr2=0x55da0987e1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:14.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.033+0000 7fad7a91cf40 1 --2- 172.21.15.38:0/253588452 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55da097aed00 0x55da0987e1d0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7fad64046560 tx=0x7fad640575e0 comp rx=0 tx=0).stop 2024-09-21T03:40:14.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.033+0000 7fad74bd7640 1 -- 172.21.15.38:0/253588452 reap_dead start 2024-09-21T03:40:14.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.033+0000 7fad7a91cf40 1 -- 172.21.15.38:0/253588452 shutdown_connections 2024-09-21T03:40:14.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.033+0000 7fad7a91cf40 1 -- 172.21.15.38:0/253588452 >> 172.21.15.38:0/253588452 conn(0x55da096cdac0 msgr2=0x55da097f4f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:14.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.033+0000 7fad7a91cf40 1 -- 172.21.15.38:0/253588452 shutdown_connections 2024-09-21T03:40:14.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.033+0000 7fad7a91cf40 1 -- 172.21.15.38:0/253588452 wait complete. 2024-09-21T03:40:14.039 DEBUG:teuthology.orchestra.run.smithi038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata stat 200.00000000 2024-09-21T03:40:14.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.092+0000 7f1c2e51ff40 1 Processor -- start 2024-09-21T03:40:14.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.092+0000 7f1c2e51ff40 1 -- start start 2024-09-21T03:40:14.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.094+0000 7f1c2e51ff40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1c10002550 0x555730841c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:14.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.094+0000 7f1c2e51ff40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5557307a04b0 0x55573081cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:14.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.094+0000 7f1c2e51ff40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55573082f970 0x55573082eba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:14.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.094+0000 7f1c2e51ff40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x555730720430 con 0x7f1c10002550 2024-09-21T03:40:14.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.094+0000 7f1c2e51ff40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55573072d1f0 con 0x55573082f970 2024-09-21T03:40:14.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.094+0000 7f1c2e51ff40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x5557306c1d60 con 0x5557307a04b0 2024-09-21T03:40:14.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.094+0000 7f1c27554640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1c10002550 0x555730841c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:14.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.094+0000 7f1c27554640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1c10002550 0x555730841c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54952/0 (socket says 172.21.15.38:54952) 2024-09-21T03:40:14.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.094+0000 7f1c27554640 1 -- 172.21.15.38:0/3483506942 learned_addr learned my addr 172.21.15.38:0/3483506942 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:14.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.094+0000 7f1c27d55640 1 --2- 172.21.15.38:0/3483506942 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55573082f970 0x55573082eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:14.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.094+0000 7f1c26d53640 1 --2- 172.21.15.38:0/3483506942 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5557307a04b0 0x55573081cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:14.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.094+0000 7f1c27554640 1 -- 172.21.15.38:0/3483506942 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5557307a04b0 msgr2=0x55573081cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:14.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.094+0000 7f1c27554640 1 --2- 172.21.15.38:0/3483506942 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5557307a04b0 0x55573081cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:14.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.094+0000 7f1c27554640 1 -- 172.21.15.38:0/3483506942 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55573082f970 msgr2=0x55573082eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:14.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.094+0000 7f1c27554640 1 --2- 172.21.15.38:0/3483506942 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55573082f970 0x55573082eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:14.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.094+0000 7f1c27554640 1 -- 172.21.15.38:0/3483506942 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557307a1280 con 0x7f1c10002550 2024-09-21T03:40:14.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.094+0000 7f1c27d55640 1 --2- 172.21.15.38:0/3483506942 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55573082f970 0x55573082eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:40:14.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.094+0000 7f1c27554640 1 --2- 172.21.15.38:0/3483506942 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1c10002550 0x555730841c30 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f1c18046990 tx=0x7f1c18003020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30a0c8a7d528adbe server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:14.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.094+0000 7f1c25d51640 1 -- 172.21.15.38:0/3483506942 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c18062030 con 0x7f1c10002550 2024-09-21T03:40:14.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.094+0000 7f1c25d51640 1 -- 172.21.15.38:0/3483506942 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1c1805e400 con 0x7f1c10002550 2024-09-21T03:40:14.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.094+0000 7f1c25d51640 1 -- 172.21.15.38:0/3483506942 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c1805e630 con 0x7f1c10002550 2024-09-21T03:40:14.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.095+0000 7f1c2e51ff40 1 -- 172.21.15.38:0/3483506942 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1c10002550 msgr2=0x555730841c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:14.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.095+0000 7f1c2e51ff40 1 --2- 172.21.15.38:0/3483506942 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1c10002550 0x555730841c30 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f1c18046990 tx=0x7f1c18003020 comp rx=0 tx=0).stop 2024-09-21T03:40:14.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.095+0000 7f1c2e51ff40 1 -- 172.21.15.38:0/3483506942 shutdown_connections 2024-09-21T03:40:14.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.095+0000 7f1c2e51ff40 1 --2- 172.21.15.38:0/3483506942 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55573082f970 0x55573082eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:14.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.095+0000 7f1c2e51ff40 1 --2- 172.21.15.38:0/3483506942 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5557307a04b0 0x55573081cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:14.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.095+0000 7f1c2e51ff40 1 --2- 172.21.15.38:0/3483506942 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1c10002550 0x555730841c30 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:14.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.095+0000 7f1c2e51ff40 1 -- 172.21.15.38:0/3483506942 >> 172.21.15.38:0/3483506942 conn(0x555730692ac0 msgr2=0x55573076f660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:14.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.095+0000 7f1c2e51ff40 1 -- 172.21.15.38:0/3483506942 shutdown_connections 2024-09-21T03:40:14.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.095+0000 7f1c2e51ff40 1 -- 172.21.15.38:0/3483506942 wait complete. 2024-09-21T03:40:14.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.096+0000 7f1c2e51ff40 1 Processor -- start 2024-09-21T03:40:14.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.096+0000 7f1c2e51ff40 1 -- start start 2024-09-21T03:40:14.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.097+0000 7f1c2e51ff40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5557307a04b0 0x55573083fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:14.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.097+0000 7f1c2e51ff40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55573082f970 0x55573079c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:14.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.097+0000 7f1c2e51ff40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1c10002550 0x55573079cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:14.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.097+0000 7f1c2e51ff40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x555730844910 con 0x55573082f970 2024-09-21T03:40:14.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.097+0000 7f1c2e51ff40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5557307a1500 con 0x7f1c10002550 2024-09-21T03:40:14.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.097+0000 7f1c2e51ff40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55573079e160 con 0x5557307a04b0 2024-09-21T03:40:14.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.097+0000 7f1c26d53640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55573082f970 0x55573079c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:14.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.097+0000 7f1c26d53640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55573082f970 0x55573079c610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54968/0 (socket says 172.21.15.38:54968) 2024-09-21T03:40:14.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.097+0000 7f1c26d53640 1 -- 172.21.15.38:0/4175706990 learned_addr learned my addr 172.21.15.38:0/4175706990 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:14.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.098+0000 7f1c27d55640 1 --2- 172.21.15.38:0/4175706990 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1c10002550 0x55573079cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:14.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.098+0000 7f1c27554640 1 --2- 172.21.15.38:0/4175706990 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5557307a04b0 0x55573083fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:14.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.098+0000 7f1c26d53640 1 -- 172.21.15.38:0/4175706990 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5557307a04b0 msgr2=0x55573083fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:14.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.098+0000 7f1c26d53640 1 --2- 172.21.15.38:0/4175706990 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5557307a04b0 0x55573083fa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:14.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.098+0000 7f1c26d53640 1 -- 172.21.15.38:0/4175706990 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1c10002550 msgr2=0x55573079cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:14.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.098+0000 7f1c26d53640 1 --2- 172.21.15.38:0/4175706990 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1c10002550 0x55573079cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:14.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.098+0000 7f1c26d53640 1 -- 172.21.15.38:0/4175706990 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557307ba860 con 0x55573082f970 2024-09-21T03:40:14.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.098+0000 7f1c27d55640 1 --2- 172.21.15.38:0/4175706990 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1c10002550 0x55573079cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:40:14.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.098+0000 7f1c26d53640 1 --2- 172.21.15.38:0/4175706990 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55573082f970 0x55573079c610 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f1c1c00c5b0 tx=0x7f1c1c00ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:14.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.098+0000 7f1c24d4f640 1 -- 172.21.15.38:0/4175706990 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c1c00a940 con 0x55573082f970 2024-09-21T03:40:14.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.098+0000 7f1c24d4f640 1 -- 172.21.15.38:0/4175706990 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1c1c013070 con 0x55573082f970 2024-09-21T03:40:14.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.098+0000 7f1c24d4f640 1 -- 172.21.15.38:0/4175706990 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c1c00ae50 con 0x55573082f970 2024-09-21T03:40:14.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.098+0000 7f1c2e51ff40 1 -- 172.21.15.38:0/4175706990 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x555730841dd0 con 0x55573082f970 2024-09-21T03:40:14.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.098+0000 7f1c2e51ff40 1 -- 172.21.15.38:0/4175706990 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55573083ff40 con 0x55573082f970 2024-09-21T03:40:14.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.100+0000 7f1c24d4f640 1 -- 172.21.15.38:0/4175706990 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f1c1c017420 con 0x55573082f970 2024-09-21T03:40:14.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.101+0000 7f1c24d4f640 1 --2- 172.21.15.38:0/4175706990 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1c140775a0 0x7f1c14079ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:14.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.101+0000 7f1c24d4f640 1 -- 172.21.15.38:0/4175706990 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6187+0+0 (secure 0 0 0) 0x7f1c1c09a140 con 0x55573082f970 2024-09-21T03:40:14.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.101+0000 7f1c27554640 1 --2- 172.21.15.38:0/4175706990 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1c140775a0 0x7f1c14079ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:14.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.101+0000 7f1c2e51ff40 1 --2- 172.21.15.38:0/4175706990 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x55573082d6e0 0x55573082daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:14.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.101+0000 7f1c2e51ff40 1 -- 172.21.15.38:0/4175706990 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:1 13.14 13:292cf221:::200.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x5557308e0cd0 con 0x55573082d6e0 2024-09-21T03:40:14.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.101+0000 7f1c27d55640 1 --2- 172.21.15.38:0/4175706990 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x55573082d6e0 0x55573082daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:14.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.102+0000 7f1c27554640 1 --2- 172.21.15.38:0/4175706990 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1c140775a0 0x7f1c14079ab0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f1c18061040 tx=0x7f1c18004880 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:14.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.102+0000 7f1c27d55640 1 --2- 172.21.15.38:0/4175706990 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x55573082d6e0 0x55573082daf0 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:14.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.102+0000 7f1c27d55640 1 -- 172.21.15.38:0/4175706990 <== osd.6 v2:172.21.15.195:6826/4055825675 1 ==== osd_op_reply(1 200.00000000 [stat out=16b] v0'0 uv18 ondisk = 0) ==== 156+0+16 (crc 0 0 0) 0x7f1c20009bf0 con 0x55573082d6e0 2024-09-21T03:40:14.105 INFO:teuthology.orchestra.run.smithi038.stdout:cephfs_metadata/200.00000000 mtime 2024-09-21T03:40:11.000000+0000, size 90 2024-09-21T03:40:14.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.102+0000 7f1c2e51ff40 1 -- 172.21.15.38:0/4175706990 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x55573082d6e0 msgr2=0x55573082daf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:14.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.102+0000 7f1c2e51ff40 1 --2- 172.21.15.38:0/4175706990 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x55573082d6e0 0x55573082daf0 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:14.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.102+0000 7f1c2e51ff40 1 -- 172.21.15.38:0/4175706990 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1c140775a0 msgr2=0x7f1c14079ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:14.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.102+0000 7f1c2e51ff40 1 --2- 172.21.15.38:0/4175706990 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1c140775a0 0x7f1c14079ab0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f1c18061040 tx=0x7f1c18004880 comp rx=0 tx=0).stop 2024-09-21T03:40:14.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.102+0000 7f1c2e51ff40 1 -- 172.21.15.38:0/4175706990 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55573082f970 msgr2=0x55573079c610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:14.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.102+0000 7f1c2e51ff40 1 --2- 172.21.15.38:0/4175706990 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55573082f970 0x55573079c610 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f1c1c00c5b0 tx=0x7f1c1c00ca70 comp rx=0 tx=0).stop 2024-09-21T03:40:14.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.102+0000 7f1c27d55640 1 -- 172.21.15.38:0/4175706990 reap_dead start 2024-09-21T03:40:14.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.102+0000 7f1c2e51ff40 1 -- 172.21.15.38:0/4175706990 shutdown_connections 2024-09-21T03:40:14.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.102+0000 7f1c2e51ff40 1 -- 172.21.15.38:0/4175706990 >> 172.21.15.38:0/4175706990 conn(0x555730692ac0 msgr2=0x55573076f660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:14.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.103+0000 7f1c2e51ff40 1 -- 172.21.15.38:0/4175706990 shutdown_connections 2024-09-21T03:40:14.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.103+0000 7f1c2e51ff40 1 -- 172.21.15.38:0/4175706990 wait complete. 2024-09-21T03:40:14.108 DEBUG:teuthology.orchestra.run.smithi038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata stat 200.00000001 2024-09-21T03:40:14.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.161+0000 7f69da2ccf40 1 Processor -- start 2024-09-21T03:40:14.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.161+0000 7f69da2ccf40 1 -- start start 2024-09-21T03:40:14.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.162+0000 7f69da2ccf40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x556fe5b29d00 0x556fe5be1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:14.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.162+0000 7f69da2ccf40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556fe5be4950 0x556fe5bfb5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:14.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.162+0000 7f69da2ccf40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556fe5bff670 0x556fe5b25ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:14.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.162+0000 7f69da2ccf40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x556fe5a77d60 con 0x556fe5be4950 2024-09-21T03:40:14.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.162+0000 7f69da2ccf40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x556fe5ad6430 con 0x556fe5bff670 2024-09-21T03:40:14.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.162+0000 7f69da2ccf40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x556fe5b717b0 con 0x556fe5b29d00 2024-09-21T03:40:14.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.162+0000 7f69d3585640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556fe5be4950 0x556fe5bfb5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:14.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.162+0000 7f69d3585640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556fe5be4950 0x556fe5bfb5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54972/0 (socket says 172.21.15.38:54972) 2024-09-21T03:40:14.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.162+0000 7f69d3585640 1 -- 172.21.15.38:0/3359515965 learned_addr learned my addr 172.21.15.38:0/3359515965 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:14.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.162+0000 7f69d3d86640 1 --2- 172.21.15.38:0/3359515965 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x556fe5b29d00 0x556fe5be1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:14.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.162+0000 7f69d4587640 1 --2- 172.21.15.38:0/3359515965 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556fe5bff670 0x556fe5b25ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:14.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.162+0000 7f69d3585640 1 -- 172.21.15.38:0/3359515965 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x556fe5b29d00 msgr2=0x556fe5be1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:14.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.162+0000 7f69d3585640 1 --2- 172.21.15.38:0/3359515965 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x556fe5b29d00 0x556fe5be1fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:14.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.162+0000 7f69d3585640 1 -- 172.21.15.38:0/3359515965 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556fe5bff670 msgr2=0x556fe5b25ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:14.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.162+0000 7f69d3585640 1 --2- 172.21.15.38:0/3359515965 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556fe5bff670 0x556fe5b25ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:14.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.162+0000 7f69d3585640 1 -- 172.21.15.38:0/3359515965 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556fe5be2500 con 0x556fe5be4950 2024-09-21T03:40:14.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.162+0000 7f69d3d86640 1 --2- 172.21.15.38:0/3359515965 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x556fe5b29d00 0x556fe5be1fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:40:14.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.163+0000 7f69d3585640 1 --2- 172.21.15.38:0/3359515965 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556fe5be4950 0x556fe5bfb5b0 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f69bc00b630 tx=0x7f69bc00baf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac46437640d7a002 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:14.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.163+0000 7f69d2583640 1 -- 172.21.15.38:0/3359515965 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69bc00a940 con 0x556fe5be4950 2024-09-21T03:40:14.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.163+0000 7f69d2583640 1 -- 172.21.15.38:0/3359515965 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f69bc00aae0 con 0x556fe5be4950 2024-09-21T03:40:14.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.163+0000 7f69d2583640 1 -- 172.21.15.38:0/3359515965 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69bc008450 con 0x556fe5be4950 2024-09-21T03:40:14.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.163+0000 7f69da2ccf40 1 -- 172.21.15.38:0/3359515965 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556fe5be4950 msgr2=0x556fe5bfb5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:14.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.163+0000 7f69da2ccf40 1 --2- 172.21.15.38:0/3359515965 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556fe5be4950 0x556fe5bfb5b0 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f69bc00b630 tx=0x7f69bc00baf0 comp rx=0 tx=0).stop 2024-09-21T03:40:14.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.163+0000 7f69da2ccf40 1 -- 172.21.15.38:0/3359515965 shutdown_connections 2024-09-21T03:40:14.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.163+0000 7f69da2ccf40 1 --2- 172.21.15.38:0/3359515965 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556fe5bff670 0x556fe5b25ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:14.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.163+0000 7f69da2ccf40 1 --2- 172.21.15.38:0/3359515965 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556fe5be4950 0x556fe5bfb5b0 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:14.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.163+0000 7f69da2ccf40 1 --2- 172.21.15.38:0/3359515965 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x556fe5b29d00 0x556fe5be1fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:14.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.163+0000 7f69da2ccf40 1 -- 172.21.15.38:0/3359515965 >> 172.21.15.38:0/3359515965 conn(0x556fe5a48ac0 msgr2=0x556fe5bffa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:14.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.164+0000 7f69da2ccf40 1 -- 172.21.15.38:0/3359515965 shutdown_connections 2024-09-21T03:40:14.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.164+0000 7f69da2ccf40 1 -- 172.21.15.38:0/3359515965 wait complete. 2024-09-21T03:40:14.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.164+0000 7f69da2ccf40 1 Processor -- start 2024-09-21T03:40:14.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.164+0000 7f69da2ccf40 1 -- start start 2024-09-21T03:40:14.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.166+0000 7f69da2ccf40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x556fe5b29d00 0x556fe5bf8140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:14.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.166+0000 7f69da2ccf40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556fe5be4950 0x556fe5b25e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:14.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.166+0000 7f69da2ccf40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556fe5bff670 0x556fe5b26370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:14.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.166+0000 7f69da2ccf40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x556fe5b27b80 con 0x556fe5bff670 2024-09-21T03:40:14.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.166+0000 7f69da2ccf40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x556fe5b6c6e0 con 0x556fe5be4950 2024-09-21T03:40:14.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.166+0000 7f69da2ccf40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x556fe5bfc1d0 con 0x556fe5b29d00 2024-09-21T03:40:14.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.166+0000 7f69d4587640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556fe5bff670 0x556fe5b26370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:14.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.166+0000 7f69d4587640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556fe5bff670 0x556fe5b26370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54988/0 (socket says 172.21.15.38:54988) 2024-09-21T03:40:14.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.166+0000 7f69d4587640 1 -- 172.21.15.38:0/1759010603 learned_addr learned my addr 172.21.15.38:0/1759010603 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:14.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.166+0000 7f69d3d86640 1 --2- 172.21.15.38:0/1759010603 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x556fe5b29d00 0x556fe5bf8140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:14.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.166+0000 7f69d3585640 1 --2- 172.21.15.38:0/1759010603 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556fe5be4950 0x556fe5b25e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:14.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.166+0000 7f69d4587640 1 -- 172.21.15.38:0/1759010603 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x556fe5b29d00 msgr2=0x556fe5bf8140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:14.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.166+0000 7f69d4587640 1 --2- 172.21.15.38:0/1759010603 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x556fe5b29d00 0x556fe5bf8140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:14.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.166+0000 7f69d4587640 1 -- 172.21.15.38:0/1759010603 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556fe5be4950 msgr2=0x556fe5b25e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:14.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.166+0000 7f69d4587640 1 --2- 172.21.15.38:0/1759010603 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556fe5be4950 0x556fe5b25e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:14.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.166+0000 7f69d4587640 1 -- 172.21.15.38:0/1759010603 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556fe5bfad60 con 0x556fe5bff670 2024-09-21T03:40:14.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.166+0000 7f69d4587640 1 --2- 172.21.15.38:0/1759010603 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556fe5bff670 0x556fe5b26370 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f69c400e430 tx=0x7f69c400e8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:14.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.166+0000 7f69d1581640 1 -- 172.21.15.38:0/1759010603 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69c4017020 con 0x556fe5bff670 2024-09-21T03:40:14.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.166+0000 7f69d1581640 1 -- 172.21.15.38:0/1759010603 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f69c4013770 con 0x556fe5bff670 2024-09-21T03:40:14.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.167+0000 7f69da2ccf40 1 -- 172.21.15.38:0/1759010603 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x556fe5bfc420 con 0x556fe5bff670 2024-09-21T03:40:14.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.167+0000 7f69d1581640 1 -- 172.21.15.38:0/1759010603 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69c4013ba0 con 0x556fe5bff670 2024-09-21T03:40:14.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.167+0000 7f69da2ccf40 1 -- 172.21.15.38:0/1759010603 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556fe5bfc9e0 con 0x556fe5bff670 2024-09-21T03:40:14.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.168+0000 7f69d1581640 1 -- 172.21.15.38:0/1759010603 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f69c401a020 con 0x556fe5bff670 2024-09-21T03:40:14.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.169+0000 7f69d1581640 1 --2- 172.21.15.38:0/1759010603 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f69b4077570 0x7f69b4079a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:14.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.169+0000 7f69d1581640 1 -- 172.21.15.38:0/1759010603 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6187+0+0 (secure 0 0 0) 0x7f69c4099ff0 con 0x556fe5bff670 2024-09-21T03:40:14.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.169+0000 7f69d3d86640 1 --2- 172.21.15.38:0/1759010603 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f69b4077570 0x7f69b4079a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:14.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.169+0000 7f69da2ccf40 1 --2- 172.21.15.38:0/1759010603 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x556fe5c99780 0x556fe5bfda90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:14.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.169+0000 7f69da2ccf40 1 -- 172.21.15.38:0/1759010603 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:1 13.14 13:2e2fa760:::200.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x556fe5bfdfd0 con 0x556fe5c99780 2024-09-21T03:40:14.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.169+0000 7f69d3585640 1 --2- 172.21.15.38:0/1759010603 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x556fe5c99780 0x556fe5bfda90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:14.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.169+0000 7f69d3d86640 1 --2- 172.21.15.38:0/1759010603 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f69b4077570 0x7f69b4079a80 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f69cc055000 tx=0x7f69cc058040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:14.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.170+0000 7f69d3585640 1 --2- 172.21.15.38:0/1759010603 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x556fe5c99780 0x556fe5bfda90 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:14.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.170+0000 7f69d3585640 1 -- 172.21.15.38:0/1759010603 <== osd.6 v2:172.21.15.195:6826/4055825675 1 ==== osd_op_reply(1 200.00000001 [stat out=16b] v0'0 uv16 ondisk = 0) ==== 156+0+16 (crc 0 0 0) 0x7f69bc00aa60 con 0x556fe5c99780 2024-09-21T03:40:14.173 INFO:teuthology.orchestra.run.smithi038.stdout:cephfs_metadata/200.00000001 mtime 2024-09-21T03:40:11.000000+0000, size 17490 2024-09-21T03:40:14.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.170+0000 7f69da2ccf40 1 -- 172.21.15.38:0/1759010603 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x556fe5c99780 msgr2=0x556fe5bfda90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:14.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.170+0000 7f69da2ccf40 1 --2- 172.21.15.38:0/1759010603 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x556fe5c99780 0x556fe5bfda90 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:14.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.170+0000 7f69da2ccf40 1 -- 172.21.15.38:0/1759010603 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f69b4077570 msgr2=0x7f69b4079a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:14.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.170+0000 7f69da2ccf40 1 --2- 172.21.15.38:0/1759010603 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f69b4077570 0x7f69b4079a80 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f69cc055000 tx=0x7f69cc058040 comp rx=0 tx=0).stop 2024-09-21T03:40:14.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.170+0000 7f69da2ccf40 1 -- 172.21.15.38:0/1759010603 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556fe5bff670 msgr2=0x556fe5b26370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:14.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.170+0000 7f69da2ccf40 1 --2- 172.21.15.38:0/1759010603 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556fe5bff670 0x556fe5b26370 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f69c400e430 tx=0x7f69c400e8f0 comp rx=0 tx=0).stop 2024-09-21T03:40:14.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.170+0000 7f69d4587640 1 -- 172.21.15.38:0/1759010603 reap_dead start 2024-09-21T03:40:14.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.170+0000 7f69da2ccf40 1 -- 172.21.15.38:0/1759010603 shutdown_connections 2024-09-21T03:40:14.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.170+0000 7f69da2ccf40 1 -- 172.21.15.38:0/1759010603 >> 172.21.15.38:0/1759010603 conn(0x556fe5a48ac0 msgr2=0x556fe5b575f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:14.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.170+0000 7f69da2ccf40 1 -- 172.21.15.38:0/1759010603 shutdown_connections 2024-09-21T03:40:14.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.170+0000 7f69da2ccf40 1 -- 172.21.15.38:0/1759010603 wait complete. 2024-09-21T03:40:14.176 DEBUG:teuthology.orchestra.run.smithi038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata stat mds0_openfiles.0 2024-09-21T03:40:14.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.229+0000 7ff57c01af40 1 Processor -- start 2024-09-21T03:40:14.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.230+0000 7ff57c01af40 1 -- start start 2024-09-21T03:40:14.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.231+0000 7ff57c01af40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560d67de7090 0x560d67e6d060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:14.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.231+0000 7ff57c01af40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560d67e5c520 0x560d67e6fa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:14.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.231+0000 7ff57c01af40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560d67e745a0 0x560d67f00690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:14.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.231+0000 7ff57c01af40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x560d67cedd60 con 0x560d67e5c520 2024-09-21T03:40:14.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.231+0000 7ff57c01af40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x560d67d4c430 con 0x560d67e745a0 2024-09-21T03:40:14.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.231+0000 7ff57c01af40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x560d67dce9b0 con 0x560d67de7090 2024-09-21T03:40:14.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.231+0000 7ff56ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560d67e5c520 0x560d67e6fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:14.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.231+0000 7ff56ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560d67e5c520 0x560d67e6fa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:55008/0 (socket says 172.21.15.38:55008) 2024-09-21T03:40:14.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.231+0000 7ff56ffff640 1 -- 172.21.15.38:0/1660612524 learned_addr learned my addr 172.21.15.38:0/1660612524 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:14.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.231+0000 7ff575668640 1 --2- 172.21.15.38:0/1660612524 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560d67e745a0 0x560d67f00690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:14.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.231+0000 7ff574e67640 1 --2- 172.21.15.38:0/1660612524 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560d67de7090 0x560d67e6d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:14.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.231+0000 7ff56ffff640 1 -- 172.21.15.38:0/1660612524 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560d67de7090 msgr2=0x560d67e6d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:14.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.231+0000 7ff56ffff640 1 --2- 172.21.15.38:0/1660612524 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560d67de7090 0x560d67e6d060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:14.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.231+0000 7ff56ffff640 1 -- 172.21.15.38:0/1660612524 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560d67e745a0 msgr2=0x560d67f00690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:14.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.231+0000 7ff56ffff640 1 --2- 172.21.15.38:0/1660612524 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560d67e745a0 0x560d67f00690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:14.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.231+0000 7ff56ffff640 1 -- 172.21.15.38:0/1660612524 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d67eff9b0 con 0x560d67e5c520 2024-09-21T03:40:14.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.231+0000 7ff575668640 1 --2- 172.21.15.38:0/1660612524 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560d67e745a0 0x560d67f00690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:40:14.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.231+0000 7ff56ffff640 1 --2- 172.21.15.38:0/1660612524 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560d67e5c520 0x560d67e6fa50 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7ff560001dc0 tx=0x7ff56000b590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=513ca63104947717 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:14.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.232+0000 7ff56effd640 1 -- 172.21.15.38:0/1660612524 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff560016070 con 0x560d67e5c520 2024-09-21T03:40:14.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.232+0000 7ff56effd640 1 -- 172.21.15.38:0/1660612524 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff56000f690 con 0x560d67e5c520 2024-09-21T03:40:14.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.232+0000 7ff56effd640 1 -- 172.21.15.38:0/1660612524 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff56000fae0 con 0x560d67e5c520 2024-09-21T03:40:14.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.232+0000 7ff57c01af40 1 -- 172.21.15.38:0/1660612524 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560d67e5c520 msgr2=0x560d67e6fa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:14.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.232+0000 7ff57c01af40 1 --2- 172.21.15.38:0/1660612524 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560d67e5c520 0x560d67e6fa50 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7ff560001dc0 tx=0x7ff56000b590 comp rx=0 tx=0).stop 2024-09-21T03:40:14.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.232+0000 7ff57c01af40 1 -- 172.21.15.38:0/1660612524 shutdown_connections 2024-09-21T03:40:14.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.232+0000 7ff57c01af40 1 --2- 172.21.15.38:0/1660612524 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560d67e745a0 0x560d67f00690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:14.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.232+0000 7ff57c01af40 1 --2- 172.21.15.38:0/1660612524 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560d67e5c520 0x560d67e6fa50 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:14.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.232+0000 7ff57c01af40 1 --2- 172.21.15.38:0/1660612524 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560d67de7090 0x560d67e6d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:14.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.232+0000 7ff57c01af40 1 -- 172.21.15.38:0/1660612524 >> 172.21.15.38:0/1660612524 conn(0x560d67cbeac0 msgr2=0x560d67e710d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:14.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.232+0000 7ff57c01af40 1 -- 172.21.15.38:0/1660612524 shutdown_connections 2024-09-21T03:40:14.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.232+0000 7ff57c01af40 1 -- 172.21.15.38:0/1660612524 wait complete. 2024-09-21T03:40:14.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.233+0000 7ff57c01af40 1 Processor -- start 2024-09-21T03:40:14.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.233+0000 7ff57c01af40 1 -- start start 2024-09-21T03:40:14.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.234+0000 7ff57c01af40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560d67de7090 0x560d67e579b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:14.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.234+0000 7ff57c01af40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560d67e5c520 0x560d67f0d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:14.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.234+0000 7ff57c01af40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560d67e745a0 0x560d67f0cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:14.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.234+0000 7ff57c01af40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x560d67dca130 con 0x560d67e5c520 2024-09-21T03:40:14.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.234+0000 7ff57c01af40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x560d67effc70 con 0x560d67de7090 2024-09-21T03:40:14.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.234+0000 7ff57c01af40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x560d67e6d410 con 0x560d67e745a0 2024-09-21T03:40:14.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.235+0000 7ff56ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560d67e5c520 0x560d67f0d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:14.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.235+0000 7ff56ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560d67e5c520 0x560d67f0d460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:55014/0 (socket says 172.21.15.38:55014) 2024-09-21T03:40:14.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.235+0000 7ff56ffff640 1 -- 172.21.15.38:0/1512892388 learned_addr learned my addr 172.21.15.38:0/1512892388 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:14.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.235+0000 7ff575668640 1 --2- 172.21.15.38:0/1512892388 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560d67e745a0 0x560d67f0cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:14.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.235+0000 7ff574e67640 1 --2- 172.21.15.38:0/1512892388 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560d67de7090 0x560d67e579b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:14.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.235+0000 7ff56ffff640 1 -- 172.21.15.38:0/1512892388 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560d67e745a0 msgr2=0x560d67f0cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:14.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.235+0000 7ff56ffff640 1 --2- 172.21.15.38:0/1512892388 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560d67e745a0 0x560d67f0cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:14.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.235+0000 7ff56ffff640 1 -- 172.21.15.38:0/1512892388 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560d67de7090 msgr2=0x560d67e579b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:14.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.235+0000 7ff56ffff640 1 --2- 172.21.15.38:0/1512892388 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560d67de7090 0x560d67e579b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:14.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.235+0000 7ff56ffff640 1 -- 172.21.15.38:0/1512892388 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d67f0d9a0 con 0x560d67e5c520 2024-09-21T03:40:14.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.235+0000 7ff574e67640 1 --2- 172.21.15.38:0/1512892388 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560d67de7090 0x560d67e579b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:40:14.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.235+0000 7ff575668640 1 --2- 172.21.15.38:0/1512892388 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560d67e745a0 0x560d67f0cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:40:14.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.235+0000 7ff56ffff640 1 --2- 172.21.15.38:0/1512892388 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560d67e5c520 0x560d67f0d460 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7ff560001eb0 tx=0x7ff5600012b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:14.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.235+0000 7ff56dffb640 1 -- 172.21.15.38:0/1512892388 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff560016030 con 0x560d67e5c520 2024-09-21T03:40:14.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.235+0000 7ff56dffb640 1 -- 172.21.15.38:0/1512892388 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff560021070 con 0x560d67e5c520 2024-09-21T03:40:14.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.235+0000 7ff57c01af40 1 -- 172.21.15.38:0/1512892388 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x560d67e6faf0 con 0x560d67e5c520 2024-09-21T03:40:14.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.235+0000 7ff56dffb640 1 -- 172.21.15.38:0/1512892388 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff560010c20 con 0x560d67e5c520 2024-09-21T03:40:14.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.235+0000 7ff57c01af40 1 -- 172.21.15.38:0/1512892388 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d67e700b0 con 0x560d67e5c520 2024-09-21T03:40:14.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.237+0000 7ff56dffb640 1 -- 172.21.15.38:0/1512892388 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ff560026460 con 0x560d67e5c520 2024-09-21T03:40:14.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.237+0000 7ff56dffb640 1 --2- 172.21.15.38:0/1512892388 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff5580775a0 0x7ff558079ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:14.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.238+0000 7ff56dffb640 1 -- 172.21.15.38:0/1512892388 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6187+0+0 (secure 0 0 0) 0x7ff56009b110 con 0x560d67e5c520 2024-09-21T03:40:14.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.238+0000 7ff574e67640 1 --2- 172.21.15.38:0/1512892388 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff5580775a0 0x7ff558079ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:14.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.238+0000 7ff57c01af40 1 --2- 172.21.15.38:0/1512892388 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x560d67f0d050 0x560d67f008f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:14.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.238+0000 7ff57c01af40 1 -- 172.21.15.38:0/1512892388 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:1 13.6 13:654134d2:::mds0_openfiles.0:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x560d67e75ae0 con 0x560d67f0d050 2024-09-21T03:40:14.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.238+0000 7ff575668640 1 --2- 172.21.15.38:0/1512892388 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x560d67f0d050 0x560d67f008f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:14.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.238+0000 7ff574e67640 1 --2- 172.21.15.38:0/1512892388 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff5580775a0 0x7ff558079ab0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7ff568055000 tx=0x7ff568058040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:14.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.238+0000 7ff575668640 1 --2- 172.21.15.38:0/1512892388 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x560d67f0d050 0x560d67f008f0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:14.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.239+0000 7ff575668640 1 -- 172.21.15.38:0/1512892388 <== osd.3 v2:172.21.15.38:6808/1286810294 1 ==== osd_op_reply(1 mds0_openfiles.0 [stat out=16b] v0'0 uv2 ondisk = 0) ==== 160+0+16 (crc 0 0 0) 0x7ff570003040 con 0x560d67f0d050 2024-09-21T03:40:14.243 INFO:teuthology.orchestra.run.smithi038.stdout:cephfs_metadata/mds0_openfiles.0 mtime 2024-09-21T03:40:11.000000+0000, size 0 2024-09-21T03:40:14.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.239+0000 7ff57c01af40 1 -- 172.21.15.38:0/1512892388 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x560d67f0d050 msgr2=0x560d67f008f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:14.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.239+0000 7ff57c01af40 1 --2- 172.21.15.38:0/1512892388 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x560d67f0d050 0x560d67f008f0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:14.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.239+0000 7ff57c01af40 1 -- 172.21.15.38:0/1512892388 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff5580775a0 msgr2=0x7ff558079ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:14.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.239+0000 7ff57c01af40 1 --2- 172.21.15.38:0/1512892388 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff5580775a0 0x7ff558079ab0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7ff568055000 tx=0x7ff568058040 comp rx=0 tx=0).stop 2024-09-21T03:40:14.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.239+0000 7ff57c01af40 1 -- 172.21.15.38:0/1512892388 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560d67e5c520 msgr2=0x560d67f0d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:14.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.239+0000 7ff57c01af40 1 --2- 172.21.15.38:0/1512892388 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560d67e5c520 0x560d67f0d460 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7ff560001eb0 tx=0x7ff5600012b0 comp rx=0 tx=0).stop 2024-09-21T03:40:14.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.239+0000 7ff575668640 1 -- 172.21.15.38:0/1512892388 reap_dead start 2024-09-21T03:40:14.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.239+0000 7ff57c01af40 1 -- 172.21.15.38:0/1512892388 shutdown_connections 2024-09-21T03:40:14.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.239+0000 7ff57c01af40 1 -- 172.21.15.38:0/1512892388 >> 172.21.15.38:0/1512892388 conn(0x560d67cbeac0 msgr2=0x560d67e719c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:14.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.239+0000 7ff57c01af40 1 -- 172.21.15.38:0/1512892388 shutdown_connections 2024-09-21T03:40:14.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.239+0000 7ff57c01af40 1 -- 172.21.15.38:0/1512892388 wait complete. 2024-09-21T03:40:14.245 DEBUG:teuthology.orchestra.run.smithi038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata stat 500.00000000 2024-09-21T03:40:14.300 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.298+0000 7f767fbfdf40 1 Processor -- start 2024-09-21T03:40:14.300 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.298+0000 7f767fbfdf40 1 -- start start 2024-09-21T03:40:14.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.299+0000 7f767fbfdf40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5612c2a01d00 0x5612c2aba0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:14.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.299+0000 7f767fbfdf40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5612c2a2ee40 0x5612c2ad19c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:14.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.299+0000 7f767fbfdf40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5612c2abc950 0x5612c2ab8e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:14.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.299+0000 7f767fbfdf40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5612c29ae430 con 0x5612c2a01d00 2024-09-21T03:40:14.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.299+0000 7f767fbfdf40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5612c29bb1f0 con 0x5612c2a2ee40 2024-09-21T03:40:14.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.299+0000 7f767fbfdf40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x5612c294fd60 con 0x5612c2abc950 2024-09-21T03:40:14.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.299+0000 7f76796b7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5612c2a01d00 0x5612c2aba0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:14.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.299+0000 7f76796b7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5612c2a01d00 0x5612c2aba0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:55018/0 (socket says 172.21.15.38:55018) 2024-09-21T03:40:14.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.299+0000 7f76796b7640 1 -- 172.21.15.38:0/1346757648 learned_addr learned my addr 172.21.15.38:0/1346757648 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:14.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.300+0000 7f7678eb6640 1 --2- 172.21.15.38:0/1346757648 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5612c2a2ee40 0x5612c2ad19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:14.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.300+0000 7f7679eb8640 1 --2- 172.21.15.38:0/1346757648 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5612c2abc950 0x5612c2ab8e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:14.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.300+0000 7f76796b7640 1 -- 172.21.15.38:0/1346757648 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5612c2abc950 msgr2=0x5612c2ab8e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:14.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.300+0000 7f76796b7640 1 --2- 172.21.15.38:0/1346757648 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5612c2abc950 0x5612c2ab8e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:14.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.300+0000 7f76796b7640 1 -- 172.21.15.38:0/1346757648 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5612c2a2ee40 msgr2=0x5612c2ad19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:14.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.300+0000 7f76796b7640 1 --2- 172.21.15.38:0/1346757648 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5612c2a2ee40 0x5612c2ad19c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:14.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.300+0000 7f76796b7640 1 -- 172.21.15.38:0/1346757648 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5612c2ad3090 con 0x5612c2a01d00 2024-09-21T03:40:14.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.300+0000 7f7678eb6640 1 --2- 172.21.15.38:0/1346757648 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5612c2a2ee40 0x5612c2ad19c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:40:14.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.300+0000 7f7679eb8640 1 --2- 172.21.15.38:0/1346757648 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5612c2abc950 0x5612c2ab8e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:40:14.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.300+0000 7f76796b7640 1 --2- 172.21.15.38:0/1346757648 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5612c2a01d00 0x5612c2aba0a0 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f766c001690 tx=0x7f766c001b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbbe4e43a7134005 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:14.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.300+0000 7f766b7fe640 1 -- 172.21.15.38:0/1346757648 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f766c017070 con 0x5612c2a01d00 2024-09-21T03:40:14.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.300+0000 7f766b7fe640 1 -- 172.21.15.38:0/1346757648 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f766c00f800 con 0x5612c2a01d00 2024-09-21T03:40:14.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.300+0000 7f766b7fe640 1 -- 172.21.15.38:0/1346757648 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f766c00fc50 con 0x5612c2a01d00 2024-09-21T03:40:14.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.301+0000 7f767fbfdf40 1 -- 172.21.15.38:0/1346757648 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5612c2a01d00 msgr2=0x5612c2aba0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:14.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.301+0000 7f767fbfdf40 1 --2- 172.21.15.38:0/1346757648 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5612c2a01d00 0x5612c2aba0a0 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f766c001690 tx=0x7f766c001b50 comp rx=0 tx=0).stop 2024-09-21T03:40:14.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.301+0000 7f767fbfdf40 1 -- 172.21.15.38:0/1346757648 shutdown_connections 2024-09-21T03:40:14.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.301+0000 7f767fbfdf40 1 --2- 172.21.15.38:0/1346757648 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5612c2abc950 0x5612c2ab8e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:14.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.301+0000 7f767fbfdf40 1 --2- 172.21.15.38:0/1346757648 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5612c2a2ee40 0x5612c2ad19c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:14.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.301+0000 7f767fbfdf40 1 --2- 172.21.15.38:0/1346757648 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5612c2a01d00 0x5612c2aba0a0 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:14.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.301+0000 7f767fbfdf40 1 -- 172.21.15.38:0/1346757648 >> 172.21.15.38:0/1346757648 conn(0x5612c2920ac0 msgr2=0x5612c2ad35e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:14.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.301+0000 7f767fbfdf40 1 -- 172.21.15.38:0/1346757648 shutdown_connections 2024-09-21T03:40:14.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.301+0000 7f767fbfdf40 1 -- 172.21.15.38:0/1346757648 wait complete. 2024-09-21T03:40:14.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.302+0000 7f767fbfdf40 1 Processor -- start 2024-09-21T03:40:14.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.302+0000 7f767fbfdf40 1 -- start start 2024-09-21T03:40:14.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.303+0000 7f767fbfdf40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5612c2a01d00 0x5612c2ab9080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:14.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.303+0000 7f767fbfdf40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5612c2a2ee40 0x5612c2acef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:14.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.303+0000 7f767fbfdf40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5612c2abc950 0x5612c2acf450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:14.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.303+0000 7f767fbfdf40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5612c2abbfc0 con 0x5612c2a2ee40 2024-09-21T03:40:14.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.303+0000 7f767fbfdf40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5612c2ad2ce0 con 0x5612c2a01d00 2024-09-21T03:40:14.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.303+0000 7f767fbfdf40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x5612c2a2c2f0 con 0x5612c2abc950 2024-09-21T03:40:14.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.303+0000 7f7678eb6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5612c2a2ee40 0x5612c2acef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:14.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.303+0000 7f7678eb6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5612c2a2ee40 0x5612c2acef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:55024/0 (socket says 172.21.15.38:55024) 2024-09-21T03:40:14.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.303+0000 7f7678eb6640 1 -- 172.21.15.38:0/2724832871 learned_addr learned my addr 172.21.15.38:0/2724832871 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:14.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.303+0000 7f76796b7640 1 --2- 172.21.15.38:0/2724832871 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5612c2a01d00 0x5612c2ab9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:14.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.303+0000 7f7679eb8640 1 --2- 172.21.15.38:0/2724832871 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5612c2abc950 0x5612c2acf450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:14.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.303+0000 7f7678eb6640 1 -- 172.21.15.38:0/2724832871 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5612c2abc950 msgr2=0x5612c2acf450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:14.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.303+0000 7f7678eb6640 1 --2- 172.21.15.38:0/2724832871 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5612c2abc950 0x5612c2acf450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:14.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.303+0000 7f7678eb6640 1 -- 172.21.15.38:0/2724832871 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5612c2a01d00 msgr2=0x5612c2ab9080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:14.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.303+0000 7f7678eb6640 1 --2- 172.21.15.38:0/2724832871 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5612c2a01d00 0x5612c2ab9080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:14.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.303+0000 7f7678eb6640 1 -- 172.21.15.38:0/2724832871 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5612c2b65ae0 con 0x5612c2a2ee40 2024-09-21T03:40:14.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.304+0000 7f7678eb6640 1 --2- 172.21.15.38:0/2724832871 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5612c2a2ee40 0x5612c2acef10 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f767000dea0 tx=0x7f767000df90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:14.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.304+0000 7f766a7fc640 1 -- 172.21.15.38:0/2724832871 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7670013070 con 0x5612c2a2ee40 2024-09-21T03:40:14.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.304+0000 7f766a7fc640 1 -- 172.21.15.38:0/2724832871 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7670006070 con 0x5612c2a2ee40 2024-09-21T03:40:14.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.304+0000 7f767fbfdf40 1 -- 172.21.15.38:0/2724832871 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5612c2abade0 con 0x5612c2a2ee40 2024-09-21T03:40:14.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.304+0000 7f766a7fc640 1 -- 172.21.15.38:0/2724832871 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f767000fb60 con 0x5612c2a2ee40 2024-09-21T03:40:14.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.304+0000 7f767fbfdf40 1 -- 172.21.15.38:0/2724832871 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5612c2abb3a0 con 0x5612c2a2ee40 2024-09-21T03:40:14.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.306+0000 7f766a7fc640 1 -- 172.21.15.38:0/2724832871 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f767000fd00 con 0x5612c2a2ee40 2024-09-21T03:40:14.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.306+0000 7f766a7fc640 1 --2- 172.21.15.38:0/2724832871 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f765c077570 0x7f765c079a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:14.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.306+0000 7f766a7fc640 1 -- 172.21.15.38:0/2724832871 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6187+0+0 (secure 0 0 0) 0x7f7670099be0 con 0x5612c2a2ee40 2024-09-21T03:40:14.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.306+0000 7f767fbfdf40 1 --2- 172.21.15.38:0/2724832871 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x5612c2ab5ef0 0x5612c2b6fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:14.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.306+0000 7f767fbfdf40 1 -- 172.21.15.38:0/2724832871 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:1 13.15 13:aa448500:::500.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x5612c2ab95c0 con 0x5612c2ab5ef0 2024-09-21T03:40:14.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.306+0000 7f76796b7640 1 --2- 172.21.15.38:0/2724832871 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f765c077570 0x7f765c079a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:14.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.307+0000 7f7679eb8640 1 --2- 172.21.15.38:0/2724832871 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x5612c2ab5ef0 0x5612c2b6fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:14.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.307+0000 7f76796b7640 1 --2- 172.21.15.38:0/2724832871 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f765c077570 0x7f765c079a80 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f766c001690 tx=0x7f766c008040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:14.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.307+0000 7f7679eb8640 1 --2- 172.21.15.38:0/2724832871 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x5612c2ab5ef0 0x5612c2b6fbf0 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:14.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.307+0000 7f7679eb8640 1 -- 172.21.15.38:0/2724832871 <== osd.3 v2:172.21.15.38:6808/1286810294 1 ==== osd_op_reply(1 500.00000000 [stat out=16b] v0'0 uv3 ondisk = 0) ==== 156+0+16 (crc 0 0 0) 0x7f7674056470 con 0x5612c2ab5ef0 2024-09-21T03:40:14.311 INFO:teuthology.orchestra.run.smithi038.stdout:cephfs_metadata/500.00000000 mtime 2024-09-21T03:40:02.000000+0000, size 90 2024-09-21T03:40:14.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.307+0000 7f767fbfdf40 1 -- 172.21.15.38:0/2724832871 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x5612c2ab5ef0 msgr2=0x5612c2b6fbf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:14.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.307+0000 7f767fbfdf40 1 --2- 172.21.15.38:0/2724832871 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x5612c2ab5ef0 0x5612c2b6fbf0 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:14.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.307+0000 7f767fbfdf40 1 -- 172.21.15.38:0/2724832871 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f765c077570 msgr2=0x7f765c079a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:14.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.307+0000 7f767fbfdf40 1 --2- 172.21.15.38:0/2724832871 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f765c077570 0x7f765c079a80 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f766c001690 tx=0x7f766c008040 comp rx=0 tx=0).stop 2024-09-21T03:40:14.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.307+0000 7f767fbfdf40 1 -- 172.21.15.38:0/2724832871 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5612c2a2ee40 msgr2=0x5612c2acef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:14.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.307+0000 7f767fbfdf40 1 --2- 172.21.15.38:0/2724832871 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5612c2a2ee40 0x5612c2acef10 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f767000dea0 tx=0x7f767000df90 comp rx=0 tx=0).stop 2024-09-21T03:40:14.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.308+0000 7f7679eb8640 1 -- 172.21.15.38:0/2724832871 reap_dead start 2024-09-21T03:40:14.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.308+0000 7f767fbfdf40 1 -- 172.21.15.38:0/2724832871 shutdown_connections 2024-09-21T03:40:14.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.308+0000 7f767fbfdf40 1 -- 172.21.15.38:0/2724832871 >> 172.21.15.38:0/2724832871 conn(0x5612c2920ac0 msgr2=0x5612c2ad35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:14.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.308+0000 7f767fbfdf40 1 -- 172.21.15.38:0/2724832871 shutdown_connections 2024-09-21T03:40:14.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.308+0000 7f767fbfdf40 1 -- 172.21.15.38:0/2724832871 wait complete. 2024-09-21T03:40:14.314 DEBUG:teuthology.orchestra.run.smithi038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata stat mds_snaptable 2024-09-21T03:40:14.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.366+0000 7f25edb1bf40 1 Processor -- start 2024-09-21T03:40:14.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.366+0000 7f25edb1bf40 1 -- start start 2024-09-21T03:40:14.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.368+0000 7f25edb1bf40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f810451d00 0x55f81047a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:14.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.368+0000 7f25edb1bf40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f81047de30 0x55f8105264b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:14.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.368+0000 7f25edb1bf40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f81050c950 0x55f810509fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:14.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.368+0000 7f25edb1bf40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55f81039fd60 con 0x55f810451d00 2024-09-21T03:40:14.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.368+0000 7f25edb1bf40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55f8103fe430 con 0x55f81050c950 2024-09-21T03:40:14.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.368+0000 7f25edb1bf40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55f81049a460 con 0x55f81047de30 2024-09-21T03:40:14.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.368+0000 7f25e6b27640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f810451d00 0x55f81047a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:14.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.368+0000 7f25e6b27640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f810451d00 0x55f81047a7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:55036/0 (socket says 172.21.15.38:55036) 2024-09-21T03:40:14.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.368+0000 7f25e6b27640 1 -- 172.21.15.38:0/1481755587 learned_addr learned my addr 172.21.15.38:0/1481755587 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:14.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.368+0000 7f25e7328640 1 --2- 172.21.15.38:0/1481755587 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f81050c950 0x55f810509fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:14.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.368+0000 7f25de326640 1 --2- 172.21.15.38:0/1481755587 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f81047de30 0x55f8105264b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:14.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.368+0000 7f25e6b27640 1 -- 172.21.15.38:0/1481755587 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f81047de30 msgr2=0x55f8105264b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:14.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.368+0000 7f25e6b27640 1 --2- 172.21.15.38:0/1481755587 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f81047de30 0x55f8105264b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:14.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.368+0000 7f25e6b27640 1 -- 172.21.15.38:0/1481755587 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f81050c950 msgr2=0x55f810509fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:14.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.368+0000 7f25e6b27640 1 --2- 172.21.15.38:0/1481755587 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f81050c950 0x55f810509fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:14.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.368+0000 7f25e6b27640 1 -- 172.21.15.38:0/1481755587 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f81047f920 con 0x55f810451d00 2024-09-21T03:40:14.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.368+0000 7f25de326640 1 --2- 172.21.15.38:0/1481755587 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f81047de30 0x55f8105264b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:40:14.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.368+0000 7f25e6b27640 1 --2- 172.21.15.38:0/1481755587 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f810451d00 0x55f81047a7c0 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f25d800b350 tx=0x7f25d8001820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e978cf744f38234 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:14.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.369+0000 7f25e5b25640 1 -- 172.21.15.38:0/1481755587 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25d8012070 con 0x55f810451d00 2024-09-21T03:40:14.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.369+0000 7f25e5b25640 1 -- 172.21.15.38:0/1481755587 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f25d8006050 con 0x55f810451d00 2024-09-21T03:40:14.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.369+0000 7f25e5b25640 1 -- 172.21.15.38:0/1481755587 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25d800ebe0 con 0x55f810451d00 2024-09-21T03:40:14.374 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.369+0000 7f25edb1bf40 1 -- 172.21.15.38:0/1481755587 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f810451d00 msgr2=0x55f81047a7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:14.374 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.369+0000 7f25edb1bf40 1 --2- 172.21.15.38:0/1481755587 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f810451d00 0x55f81047a7c0 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f25d800b350 tx=0x7f25d8001820 comp rx=0 tx=0).stop 2024-09-21T03:40:14.374 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.369+0000 7f25edb1bf40 1 -- 172.21.15.38:0/1481755587 shutdown_connections 2024-09-21T03:40:14.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.369+0000 7f25edb1bf40 1 --2- 172.21.15.38:0/1481755587 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f81050c950 0x55f810509fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:14.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.369+0000 7f25edb1bf40 1 --2- 172.21.15.38:0/1481755587 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f81047de30 0x55f8105264b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:14.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.369+0000 7f25edb1bf40 1 --2- 172.21.15.38:0/1481755587 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f810451d00 0x55f81047a7c0 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:14.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.369+0000 7f25edb1bf40 1 -- 172.21.15.38:0/1481755587 >> 172.21.15.38:0/1481755587 conn(0x55f810370ac0 msgr2=0x55f81047f210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:14.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.369+0000 7f25edb1bf40 1 -- 172.21.15.38:0/1481755587 shutdown_connections 2024-09-21T03:40:14.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.369+0000 7f25edb1bf40 1 -- 172.21.15.38:0/1481755587 wait complete. 2024-09-21T03:40:14.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.370+0000 7f25edb1bf40 1 Processor -- start 2024-09-21T03:40:14.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.370+0000 7f25edb1bf40 1 -- start start 2024-09-21T03:40:14.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.371+0000 7f25edb1bf40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f810451d00 0x55f8105becc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:14.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.371+0000 7f25edb1bf40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f81047de30 0x55f810524ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:14.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.371+0000 7f25edb1bf40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f81050c950 0x55f8105264b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:14.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.371+0000 7f25edb1bf40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55f8105b19c0 con 0x55f81047de30 2024-09-21T03:40:14.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.371+0000 7f25edb1bf40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55f81047fc50 con 0x55f810451d00 2024-09-21T03:40:14.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.371+0000 7f25edb1bf40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55f8105be230 con 0x55f81050c950 2024-09-21T03:40:14.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.372+0000 7f25de326640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f81047de30 0x55f810524ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:14.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.372+0000 7f25de326640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f81047de30 0x55f810524ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:55050/0 (socket says 172.21.15.38:55050) 2024-09-21T03:40:14.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.372+0000 7f25de326640 1 -- 172.21.15.38:0/468613024 learned_addr learned my addr 172.21.15.38:0/468613024 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:14.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.372+0000 7f25e6b27640 1 --2- 172.21.15.38:0/468613024 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f810451d00 0x55f8105becc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:14.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.372+0000 7f25de326640 1 -- 172.21.15.38:0/468613024 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f81050c950 msgr2=0x55f8105264b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:14.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.372+0000 7f25de326640 1 --2- 172.21.15.38:0/468613024 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f81050c950 0x55f8105264b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:14.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.372+0000 7f25de326640 1 -- 172.21.15.38:0/468613024 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f810451d00 msgr2=0x55f8105becc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:14.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.372+0000 7f25de326640 1 --2- 172.21.15.38:0/468613024 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f810451d00 0x55f8105becc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:14.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.372+0000 7f25de326640 1 -- 172.21.15.38:0/468613024 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f8105c2620 con 0x55f81047de30 2024-09-21T03:40:14.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.372+0000 7f25de326640 1 --2- 172.21.15.38:0/468613024 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f81047de30 0x55f810524ef0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f25d000a760 tx=0x7f25d000ac20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:14.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.372+0000 7f25e4b23640 1 -- 172.21.15.38:0/468613024 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25d0009ba0 con 0x55f81047de30 2024-09-21T03:40:14.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.372+0000 7f25e4b23640 1 -- 172.21.15.38:0/468613024 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f25d0009d40 con 0x55f81047de30 2024-09-21T03:40:14.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.372+0000 7f25edb1bf40 1 -- 172.21.15.38:0/468613024 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55f810524710 con 0x55f81047de30 2024-09-21T03:40:14.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.372+0000 7f25e4b23640 1 -- 172.21.15.38:0/468613024 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25d0008660 con 0x55f81047de30 2024-09-21T03:40:14.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.372+0000 7f25edb1bf40 1 -- 172.21.15.38:0/468613024 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f81047e880 con 0x55f81047de30 2024-09-21T03:40:14.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.374+0000 7f25e4b23640 1 -- 172.21.15.38:0/468613024 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f25d0008800 con 0x55f81047de30 2024-09-21T03:40:14.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.375+0000 7f25e4b23640 1 --2- 172.21.15.38:0/468613024 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f25c8077570 0x7f25c8079a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:14.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.375+0000 7f25e4b23640 1 -- 172.21.15.38:0/468613024 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6187+0+0 (secure 0 0 0) 0x7f25d009ad40 con 0x55f81047de30 2024-09-21T03:40:14.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.375+0000 7f25e6b27640 1 --2- 172.21.15.38:0/468613024 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f25c8077570 0x7f25c8079a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:14.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.375+0000 7f25edb1bf40 1 --2- 172.21.15.38:0/468613024 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x55f8105c16f0 0x55f8105c1b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:14.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.375+0000 7f25edb1bf40 1 -- 172.21.15.38:0/468613024 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:1 13.d 13:b50e409b:::mds_snaptable:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x55f8105c2040 con 0x55f8105c16f0 2024-09-21T03:40:14.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.375+0000 7f25e7328640 1 --2- 172.21.15.38:0/468613024 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x55f8105c16f0 0x55f8105c1b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:14.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.375+0000 7f25e6b27640 1 --2- 172.21.15.38:0/468613024 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f25c8077570 0x7f25c8079a80 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f25d80017f0 tx=0x7f25d8019040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:14.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.375+0000 7f25e7328640 1 --2- 172.21.15.38:0/468613024 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x55f8105c16f0 0x55f8105c1b00 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:14.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.376+0000 7f25e7328640 1 -- 172.21.15.38:0/468613024 <== osd.0 v2:172.21.15.38:6800/2381639433 1 ==== osd_op_reply(1 mds_snaptable [stat out=16b] v0'0 uv1 ondisk = 0) ==== 157+0+16 (crc 0 0 0) 0x7f25e000b040 con 0x55f8105c16f0 2024-09-21T03:40:14.379 INFO:teuthology.orchestra.run.smithi038.stdout:cephfs_metadata/mds_snaptable mtime 2024-09-21T03:40:02.000000+0000, size 70 2024-09-21T03:40:14.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.376+0000 7f25edb1bf40 1 -- 172.21.15.38:0/468613024 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x55f8105c16f0 msgr2=0x55f8105c1b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:14.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.376+0000 7f25edb1bf40 1 --2- 172.21.15.38:0/468613024 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x55f8105c16f0 0x55f8105c1b00 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:14.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.376+0000 7f25edb1bf40 1 -- 172.21.15.38:0/468613024 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f25c8077570 msgr2=0x7f25c8079a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:14.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.376+0000 7f25edb1bf40 1 --2- 172.21.15.38:0/468613024 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f25c8077570 0x7f25c8079a80 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f25d80017f0 tx=0x7f25d8019040 comp rx=0 tx=0).stop 2024-09-21T03:40:14.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.376+0000 7f25edb1bf40 1 -- 172.21.15.38:0/468613024 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f81047de30 msgr2=0x55f810524ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:14.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.376+0000 7f25edb1bf40 1 --2- 172.21.15.38:0/468613024 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f81047de30 0x55f810524ef0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f25d000a760 tx=0x7f25d000ac20 comp rx=0 tx=0).stop 2024-09-21T03:40:14.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.376+0000 7f25e7328640 1 -- 172.21.15.38:0/468613024 reap_dead start 2024-09-21T03:40:14.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.376+0000 7f25edb1bf40 1 -- 172.21.15.38:0/468613024 shutdown_connections 2024-09-21T03:40:14.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.376+0000 7f25edb1bf40 1 -- 172.21.15.38:0/468613024 >> 172.21.15.38:0/468613024 conn(0x55f810370ac0 msgr2=0x55f81047f1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:14.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.376+0000 7f25edb1bf40 1 -- 172.21.15.38:0/468613024 shutdown_connections 2024-09-21T03:40:14.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.376+0000 7f25edb1bf40 1 -- 172.21.15.38:0/468613024 wait complete. 2024-09-21T03:40:14.382 DEBUG:teuthology.orchestra.run.smithi038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata stat mds0_inotable 2024-09-21T03:40:14.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.435+0000 7f0c3ad1ef40 1 Processor -- start 2024-09-21T03:40:14.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.435+0000 7f0c3ad1ef40 1 -- start start 2024-09-21T03:40:14.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.436+0000 7f0c3ad1ef40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56003b3ced00 0x56003b483210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:14.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.436+0000 7f0c3ad1ef40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56003b3fae30 0x56003b4a1e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:14.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.436+0000 7f0c3ad1ef40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56003b489950 0x56003b4a2340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:14.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.436+0000 7f0c3ad1ef40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x56003b31cd60 con 0x56003b489950 2024-09-21T03:40:14.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.436+0000 7f0c3ad1ef40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x56003b37b430 con 0x56003b3ced00 2024-09-21T03:40:14.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.436+0000 7f0c3ad1ef40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x56003b417460 con 0x56003b3fae30 2024-09-21T03:40:14.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.436+0000 7f0c3450c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56003b489950 0x56003b4a2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:14.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.436+0000 7f0c3450c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56003b489950 0x56003b4a2340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:55068/0 (socket says 172.21.15.38:55068) 2024-09-21T03:40:14.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.436+0000 7f0c3450c640 1 -- 172.21.15.38:0/3364506031 learned_addr learned my addr 172.21.15.38:0/3364506031 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:14.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.436+0000 7f0c3350a640 1 --2- 172.21.15.38:0/3364506031 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56003b3fae30 0x56003b4a1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:14.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.436+0000 7f0c33d0b640 1 --2- 172.21.15.38:0/3364506031 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56003b3ced00 0x56003b483210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:14.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.436+0000 7f0c3450c640 1 -- 172.21.15.38:0/3364506031 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56003b3fae30 msgr2=0x56003b4a1e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:14.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.436+0000 7f0c3450c640 1 --2- 172.21.15.38:0/3364506031 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56003b3fae30 0x56003b4a1e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:14.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.436+0000 7f0c3450c640 1 -- 172.21.15.38:0/3364506031 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56003b3ced00 msgr2=0x56003b483210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:14.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.436+0000 7f0c3450c640 1 --2- 172.21.15.38:0/3364506031 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56003b3ced00 0x56003b483210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:14.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.436+0000 7f0c3450c640 1 -- 172.21.15.38:0/3364506031 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56003b4a3d20 con 0x56003b489950 2024-09-21T03:40:14.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.437+0000 7f0c3450c640 1 --2- 172.21.15.38:0/3364506031 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56003b489950 0x56003b4a2340 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f0c2c057040 tx=0x7f0c2c003330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b3520250cd0a471 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:14.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.437+0000 7f0c32508640 1 -- 172.21.15.38:0/3364506031 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c2c064030 con 0x56003b489950 2024-09-21T03:40:14.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.437+0000 7f0c32508640 1 -- 172.21.15.38:0/3364506031 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0c2c004020 con 0x56003b489950 2024-09-21T03:40:14.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.437+0000 7f0c32508640 1 -- 172.21.15.38:0/3364506031 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c2c004270 con 0x56003b489950 2024-09-21T03:40:14.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.437+0000 7f0c3ad1ef40 1 -- 172.21.15.38:0/3364506031 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56003b489950 msgr2=0x56003b4a2340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:14.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.437+0000 7f0c3ad1ef40 1 --2- 172.21.15.38:0/3364506031 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56003b489950 0x56003b4a2340 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f0c2c057040 tx=0x7f0c2c003330 comp rx=0 tx=0).stop 2024-09-21T03:40:14.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.437+0000 7f0c3ad1ef40 1 -- 172.21.15.38:0/3364506031 shutdown_connections 2024-09-21T03:40:14.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.437+0000 7f0c3ad1ef40 1 --2- 172.21.15.38:0/3364506031 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56003b489950 0x56003b4a2340 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:14.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.437+0000 7f0c3ad1ef40 1 --2- 172.21.15.38:0/3364506031 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56003b3fae30 0x56003b4a1e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:14.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.437+0000 7f0c3ad1ef40 1 --2- 172.21.15.38:0/3364506031 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56003b3ced00 0x56003b483210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:14.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.438+0000 7f0c3ad1ef40 1 -- 172.21.15.38:0/3364506031 >> 172.21.15.38:0/3364506031 conn(0x56003b2edac0 msgr2=0x56003b4a0630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:14.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.438+0000 7f0c3ad1ef40 1 -- 172.21.15.38:0/3364506031 shutdown_connections 2024-09-21T03:40:14.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.438+0000 7f0c3ad1ef40 1 -- 172.21.15.38:0/3364506031 wait complete. 2024-09-21T03:40:14.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.438+0000 7f0c3ad1ef40 1 Processor -- start 2024-09-21T03:40:14.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.439+0000 7f0c3ad1ef40 1 -- start start 2024-09-21T03:40:14.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.440+0000 7f0c3ad1ef40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56003b3ced00 0x56003b539fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:14.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.440+0000 7f0c3ad1ef40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56003b3fae30 0x56003b4a2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:14.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.440+0000 7f0c3ad1ef40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56003b489950 0x56003b4a3000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:14.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.440+0000 7f0c3ad1ef40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x56003b414ec0 con 0x56003b489950 2024-09-21T03:40:14.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.440+0000 7f0c3ad1ef40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x56003b486010 con 0x56003b3fae30 2024-09-21T03:40:14.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.440+0000 7f0c3ad1ef40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x56003b3fc280 con 0x56003b3ced00 2024-09-21T03:40:14.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.440+0000 7f0c3450c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56003b489950 0x56003b4a3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:14.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.440+0000 7f0c3450c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56003b489950 0x56003b4a3000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:55074/0 (socket says 172.21.15.38:55074) 2024-09-21T03:40:14.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.440+0000 7f0c3450c640 1 -- 172.21.15.38:0/3346780613 learned_addr learned my addr 172.21.15.38:0/3346780613 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:14.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.440+0000 7f0c3350a640 1 --2- 172.21.15.38:0/3346780613 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56003b3fae30 0x56003b4a2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:14.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.440+0000 7f0c33d0b640 1 --2- 172.21.15.38:0/3346780613 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56003b3ced00 0x56003b539fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:14.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.440+0000 7f0c3450c640 1 -- 172.21.15.38:0/3346780613 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56003b3ced00 msgr2=0x56003b539fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:14.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.440+0000 7f0c3450c640 1 --2- 172.21.15.38:0/3346780613 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56003b3ced00 0x56003b539fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:14.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.440+0000 7f0c3450c640 1 -- 172.21.15.38:0/3346780613 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56003b3fae30 msgr2=0x56003b4a2ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:14.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.440+0000 7f0c3450c640 1 --2- 172.21.15.38:0/3346780613 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56003b3fae30 0x56003b4a2ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:14.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.440+0000 7f0c3450c640 1 -- 172.21.15.38:0/3346780613 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56003b499050 con 0x56003b489950 2024-09-21T03:40:14.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.440+0000 7f0c3450c640 1 --2- 172.21.15.38:0/3346780613 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56003b489950 0x56003b4a3000 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f0c2c025e70 tx=0x7f0c2c04dbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:14.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.441+0000 7f0c31506640 1 -- 172.21.15.38:0/3346780613 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c2c06d070 con 0x56003b489950 2024-09-21T03:40:14.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.441+0000 7f0c31506640 1 -- 172.21.15.38:0/3346780613 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0c2c060760 con 0x56003b489950 2024-09-21T03:40:14.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.441+0000 7f0c31506640 1 -- 172.21.15.38:0/3346780613 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c2c060b90 con 0x56003b489950 2024-09-21T03:40:14.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.441+0000 7f0c3ad1ef40 1 -- 172.21.15.38:0/3346780613 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56003b499330 con 0x56003b489950 2024-09-21T03:40:14.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.441+0000 7f0c3ad1ef40 1 -- 172.21.15.38:0/3346780613 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56003b4998f0 con 0x56003b489950 2024-09-21T03:40:14.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.442+0000 7f0c31506640 1 -- 172.21.15.38:0/3346780613 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f0c2c070070 con 0x56003b489950 2024-09-21T03:40:14.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.443+0000 7f0c31506640 1 --2- 172.21.15.38:0/3346780613 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0c14077570 0x7f0c14079a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:14.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.443+0000 7f0c31506640 1 -- 172.21.15.38:0/3346780613 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6187+0+0 (secure 0 0 0) 0x7f0c2c0df200 con 0x56003b489950 2024-09-21T03:40:14.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.443+0000 7f0c33d0b640 1 --2- 172.21.15.38:0/3346780613 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0c14077570 0x7f0c14079a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:14.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.443+0000 7f0c3ad1ef40 1 --2- 172.21.15.38:0/3346780613 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x56003b49a600 0x56003b49aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:14.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.443+0000 7f0c3ad1ef40 1 -- 172.21.15.38:0/3346780613 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:1 13.13 13:c91d4a1d:::mds0_inotable:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x56003b47f670 con 0x56003b49a600 2024-09-21T03:40:14.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.443+0000 7f0c3350a640 1 --2- 172.21.15.38:0/3346780613 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x56003b49a600 0x56003b49aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:14.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.444+0000 7f0c33d0b640 1 --2- 172.21.15.38:0/3346780613 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0c14077570 0x7f0c14079a80 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f0c24011480 tx=0x7f0c24001730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:14.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.444+0000 7f0c3350a640 1 --2- 172.21.15.38:0/3346780613 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x56003b49a600 0x56003b49aa10 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:14.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.444+0000 7f0c3350a640 1 -- 172.21.15.38:0/3346780613 <== osd.1 v2:172.21.15.38:6816/1491818732 1 ==== osd_op_reply(1 mds0_inotable [stat out=16b] v0'0 uv3 ondisk = 0) ==== 157+0+16 (crc 0 0 0) 0x7f0c1c00b820 con 0x56003b49a600 2024-09-21T03:40:14.447 INFO:teuthology.orchestra.run.smithi038.stdout:cephfs_metadata/mds0_inotable mtime 2024-09-21T03:40:11.000000+0000, size 34 2024-09-21T03:40:14.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.444+0000 7f0c3ad1ef40 1 -- 172.21.15.38:0/3346780613 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x56003b49a600 msgr2=0x56003b49aa10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:14.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.444+0000 7f0c3ad1ef40 1 --2- 172.21.15.38:0/3346780613 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x56003b49a600 0x56003b49aa10 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:14.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.444+0000 7f0c3ad1ef40 1 -- 172.21.15.38:0/3346780613 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0c14077570 msgr2=0x7f0c14079a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:14.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.444+0000 7f0c3ad1ef40 1 --2- 172.21.15.38:0/3346780613 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0c14077570 0x7f0c14079a80 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f0c24011480 tx=0x7f0c24001730 comp rx=0 tx=0).stop 2024-09-21T03:40:14.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.444+0000 7f0c3ad1ef40 1 -- 172.21.15.38:0/3346780613 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56003b489950 msgr2=0x56003b4a3000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:14.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.444+0000 7f0c3ad1ef40 1 --2- 172.21.15.38:0/3346780613 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56003b489950 0x56003b4a3000 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f0c2c025e70 tx=0x7f0c2c04dbf0 comp rx=0 tx=0).stop 2024-09-21T03:40:14.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.444+0000 7f0c3450c640 1 -- 172.21.15.38:0/3346780613 reap_dead start 2024-09-21T03:40:14.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.444+0000 7f0c3ad1ef40 1 -- 172.21.15.38:0/3346780613 shutdown_connections 2024-09-21T03:40:14.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.444+0000 7f0c3ad1ef40 1 -- 172.21.15.38:0/3346780613 >> 172.21.15.38:0/3346780613 conn(0x56003b2edac0 msgr2=0x56003b483210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:14.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.444+0000 7f0c3ad1ef40 1 -- 172.21.15.38:0/3346780613 shutdown_connections 2024-09-21T03:40:14.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.444+0000 7f0c3ad1ef40 1 -- 172.21.15.38:0/3346780613 wait complete. 2024-09-21T03:40:14.450 DEBUG:teuthology.orchestra.run.smithi038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata stat 100.00000000 2024-09-21T03:40:14.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.503+0000 7f361591cf40 1 Processor -- start 2024-09-21T03:40:14.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.503+0000 7f361591cf40 1 -- start start 2024-09-21T03:40:14.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.504+0000 7f361591cf40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c9553f4d00 0x55c9554ae970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:14.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.504+0000 7f361591cf40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c955420e30 0x55c95541f070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:14.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.504+0000 7f361591cf40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55c9554af950 0x55c955554c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:14.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.504+0000 7f361591cf40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55c955342d60 con 0x55c9553f4d00 2024-09-21T03:40:14.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.504+0000 7f361591cf40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55c9553a1430 con 0x55c955420e30 2024-09-21T03:40:14.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.504+0000 7f361591cf40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55c95543d460 con 0x55c9554af950 2024-09-21T03:40:14.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.504+0000 7f360eab7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c9553f4d00 0x55c9554ae970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:14.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.504+0000 7f360eab7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c9553f4d00 0x55c9554ae970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:55090/0 (socket says 172.21.15.38:55090) 2024-09-21T03:40:14.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.504+0000 7f360eab7640 1 -- 172.21.15.38:0/163354148 learned_addr learned my addr 172.21.15.38:0/163354148 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:14.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.504+0000 7f360eab7640 1 -- 172.21.15.38:0/163354148 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55c9554af950 msgr2=0x55c955554c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:14.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.504+0000 7f360eab7640 1 --2- 172.21.15.38:0/163354148 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55c9554af950 0x55c955554c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:14.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.504+0000 7f360e2b6640 1 --2- 172.21.15.38:0/163354148 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c955420e30 0x55c95541f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:14.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.504+0000 7f360eab7640 1 -- 172.21.15.38:0/163354148 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c955420e30 msgr2=0x55c95541f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:14.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.504+0000 7f360eab7640 1 --2- 172.21.15.38:0/163354148 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c955420e30 0x55c95541f070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:14.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.504+0000 7f360eab7640 1 -- 172.21.15.38:0/163354148 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9554c6930 con 0x55c9553f4d00 2024-09-21T03:40:14.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.505+0000 7f360eab7640 1 --2- 172.21.15.38:0/163354148 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c9553f4d00 0x55c9554ae970 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f3608001e20 tx=0x7f360800b730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74ef77f6579d90f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:14.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.505+0000 7f360d2b4640 1 -- 172.21.15.38:0/163354148 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f360800e830 con 0x55c9553f4d00 2024-09-21T03:40:14.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.505+0000 7f360d2b4640 1 -- 172.21.15.38:0/163354148 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3608006050 con 0x55c9553f4d00 2024-09-21T03:40:14.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.505+0000 7f360d2b4640 1 -- 172.21.15.38:0/163354148 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f360800eda0 con 0x55c9553f4d00 2024-09-21T03:40:14.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.505+0000 7f361591cf40 1 -- 172.21.15.38:0/163354148 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c9553f4d00 msgr2=0x55c9554ae970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:14.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.505+0000 7f361591cf40 1 --2- 172.21.15.38:0/163354148 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c9553f4d00 0x55c9554ae970 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f3608001e20 tx=0x7f360800b730 comp rx=0 tx=0).stop 2024-09-21T03:40:14.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.505+0000 7f361591cf40 1 -- 172.21.15.38:0/163354148 shutdown_connections 2024-09-21T03:40:14.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.505+0000 7f361591cf40 1 --2- 172.21.15.38:0/163354148 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55c9554af950 0x55c955554c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:14.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.505+0000 7f361591cf40 1 --2- 172.21.15.38:0/163354148 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c955420e30 0x55c95541f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:14.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.505+0000 7f361591cf40 1 --2- 172.21.15.38:0/163354148 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c9553f4d00 0x55c9554ae970 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:14.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.505+0000 7f361591cf40 1 -- 172.21.15.38:0/163354148 >> 172.21.15.38:0/163354148 conn(0x55c955313ac0 msgr2=0x55c9554ca350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:14.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.505+0000 7f361591cf40 1 -- 172.21.15.38:0/163354148 shutdown_connections 2024-09-21T03:40:14.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.505+0000 7f361591cf40 1 -- 172.21.15.38:0/163354148 wait complete. 2024-09-21T03:40:14.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.506+0000 7f361591cf40 1 Processor -- start 2024-09-21T03:40:14.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.506+0000 7f361591cf40 1 -- start start 2024-09-21T03:40:14.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.508+0000 7f361591cf40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55c9553f4d00 0x55c955563340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:14.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.508+0000 7f361591cf40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c955420e30 0x55c9554a7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:14.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.508+0000 7f361591cf40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c9554af950 0x55c9554ca350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:14.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.508+0000 7f361591cf40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55c9554c6bd0 con 0x55c955420e30 2024-09-21T03:40:14.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.508+0000 7f361591cf40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55c9555557f0 con 0x55c9554af950 2024-09-21T03:40:14.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.508+0000 7f361591cf40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55c9554caea0 con 0x55c9553f4d00 2024-09-21T03:40:14.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.508+0000 7f360e2b6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c955420e30 0x55c9554a7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:14.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.508+0000 7f360e2b6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c955420e30 0x55c9554a7cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:55096/0 (socket says 172.21.15.38:55096) 2024-09-21T03:40:14.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.508+0000 7f360e2b6640 1 -- 172.21.15.38:0/3932015178 learned_addr learned my addr 172.21.15.38:0/3932015178 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:14.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.508+0000 7f360eab7640 1 --2- 172.21.15.38:0/3932015178 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55c9553f4d00 0x55c955563340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:14.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.508+0000 7f360f2b8640 1 --2- 172.21.15.38:0/3932015178 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c9554af950 0x55c9554ca350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:14.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.508+0000 7f360e2b6640 1 -- 172.21.15.38:0/3932015178 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55c9553f4d00 msgr2=0x55c955563340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:14.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.508+0000 7f360e2b6640 1 --2- 172.21.15.38:0/3932015178 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55c9553f4d00 0x55c955563340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:14.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.508+0000 7f360e2b6640 1 -- 172.21.15.38:0/3932015178 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c9554af950 msgr2=0x55c9554ca350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:14.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.508+0000 7f360e2b6640 1 --2- 172.21.15.38:0/3932015178 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c9554af950 0x55c9554ca350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:14.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.508+0000 7f360e2b6640 1 -- 172.21.15.38:0/3932015178 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9554a81f0 con 0x55c955420e30 2024-09-21T03:40:14.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.508+0000 7f360eab7640 1 --2- 172.21.15.38:0/3932015178 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55c9553f4d00 0x55c955563340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:40:14.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.508+0000 7f360f2b8640 1 --2- 172.21.15.38:0/3932015178 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c9554af950 0x55c9554ca350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:40:14.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.508+0000 7f360e2b6640 1 --2- 172.21.15.38:0/3932015178 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c955420e30 0x55c9554a7cb0 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f35f800a710 tx=0x7f35f800abd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:14.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.508+0000 7f3607fff640 1 -- 172.21.15.38:0/3932015178 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35f80099a0 con 0x55c955420e30 2024-09-21T03:40:14.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.508+0000 7f3607fff640 1 -- 172.21.15.38:0/3932015178 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f35f8009b40 con 0x55c955420e30 2024-09-21T03:40:14.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.508+0000 7f361591cf40 1 -- 172.21.15.38:0/3932015178 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55c9554c47a0 con 0x55c955420e30 2024-09-21T03:40:14.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.508+0000 7f3607fff640 1 -- 172.21.15.38:0/3932015178 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35f8016420 con 0x55c955420e30 2024-09-21T03:40:14.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.508+0000 7f361591cf40 1 -- 172.21.15.38:0/3932015178 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c9554c4d60 con 0x55c955420e30 2024-09-21T03:40:14.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.510+0000 7f3607fff640 1 -- 172.21.15.38:0/3932015178 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f35f80165c0 con 0x55c955420e30 2024-09-21T03:40:14.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.511+0000 7f3607fff640 1 --2- 172.21.15.38:0/3932015178 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f35f00775a0 0x7f35f0079ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:14.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.511+0000 7f3607fff640 1 -- 172.21.15.38:0/3932015178 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6187+0+0 (secure 0 0 0) 0x7f35f809a340 con 0x55c955420e30 2024-09-21T03:40:14.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.511+0000 7f361591cf40 1 --2- 172.21.15.38:0/3932015178 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x55c9554a8510 0x55c9554c8810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:14.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.511+0000 7f360eab7640 1 --2- 172.21.15.38:0/3932015178 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f35f00775a0 0x7f35f0079ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:14.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.511+0000 7f361591cf40 1 -- 172.21.15.38:0/3932015178 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:1 13.13 13:cd5a64a3:::100.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x55c9554c8d50 con 0x55c9554a8510 2024-09-21T03:40:14.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.511+0000 7f360f2b8640 1 --2- 172.21.15.38:0/3932015178 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x55c9554a8510 0x55c9554c8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:14.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.511+0000 7f360f2b8640 1 --2- 172.21.15.38:0/3932015178 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x55c9554a8510 0x55c9554c8810 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:14.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.511+0000 7f360eab7640 1 --2- 172.21.15.38:0/3932015178 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f35f00775a0 0x7f35f0079ab0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f3608001dc0 tx=0x7f3608000ed0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:14.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.512+0000 7f360f2b8640 1 -- 172.21.15.38:0/3932015178 <== osd.1 v2:172.21.15.38:6816/1491818732 1 ==== osd_op_reply(1 100.00000000 [stat out=16b] v0'0 uv1 ondisk = 0) ==== 156+0+16 (crc 0 0 0) 0x7f360000b040 con 0x55c9554a8510 2024-09-21T03:40:14.515 INFO:teuthology.orchestra.run.smithi038.stdout:cephfs_metadata/100.00000000 mtime 2024-09-21T03:40:02.000000+0000, size 0 2024-09-21T03:40:14.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.512+0000 7f361591cf40 1 -- 172.21.15.38:0/3932015178 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x55c9554a8510 msgr2=0x55c9554c8810 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:14.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.512+0000 7f361591cf40 1 --2- 172.21.15.38:0/3932015178 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x55c9554a8510 0x55c9554c8810 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:14.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.512+0000 7f361591cf40 1 -- 172.21.15.38:0/3932015178 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f35f00775a0 msgr2=0x7f35f0079ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:14.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.512+0000 7f361591cf40 1 --2- 172.21.15.38:0/3932015178 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f35f00775a0 0x7f35f0079ab0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f3608001dc0 tx=0x7f3608000ed0 comp rx=0 tx=0).stop 2024-09-21T03:40:14.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.512+0000 7f361591cf40 1 -- 172.21.15.38:0/3932015178 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c955420e30 msgr2=0x55c9554a7cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:14.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.512+0000 7f361591cf40 1 --2- 172.21.15.38:0/3932015178 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c955420e30 0x55c9554a7cb0 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f35f800a710 tx=0x7f35f800abd0 comp rx=0 tx=0).stop 2024-09-21T03:40:14.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.512+0000 7f360f2b8640 1 -- 172.21.15.38:0/3932015178 reap_dead start 2024-09-21T03:40:14.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.512+0000 7f361591cf40 1 -- 172.21.15.38:0/3932015178 shutdown_connections 2024-09-21T03:40:14.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.512+0000 7f361591cf40 1 -- 172.21.15.38:0/3932015178 >> 172.21.15.38:0/3932015178 conn(0x55c955313ac0 msgr2=0x55c955554c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:14.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.512+0000 7f361591cf40 1 -- 172.21.15.38:0/3932015178 shutdown_connections 2024-09-21T03:40:14.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.512+0000 7f361591cf40 1 -- 172.21.15.38:0/3932015178 wait complete. 2024-09-21T03:40:14.518 DEBUG:teuthology.orchestra.run.smithi038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata stat mds0_sessionmap 2024-09-21T03:40:14.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.570+0000 7ffa88d36f40 1 Processor -- start 2024-09-21T03:40:14.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.570+0000 7ffa88d36f40 1 -- start start 2024-09-21T03:40:14.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.572+0000 7ffa88d36f40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558d0a379d00 0x558d0a44db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:14.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.572+0000 7ffa88d36f40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558d0a3a75a0 0x558d0a44e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:14.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.572+0000 7ffa88d36f40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x558d0a457920 0x558d0a44fae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:14.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.572+0000 7ffa88d36f40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x558d0a326430 con 0x558d0a379d00 2024-09-21T03:40:14.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.572+0000 7ffa88d36f40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x558d0a3331f0 con 0x558d0a3a75a0 2024-09-21T03:40:14.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.572+0000 7ffa88d36f40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x558d0a2c7d60 con 0x558d0a457920 2024-09-21T03:40:14.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.572+0000 7ffa827f0640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558d0a379d00 0x558d0a44db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:14.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.572+0000 7ffa827f0640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558d0a379d00 0x558d0a44db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:55112/0 (socket says 172.21.15.38:55112) 2024-09-21T03:40:14.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.572+0000 7ffa827f0640 1 -- 172.21.15.38:0/411425002 learned_addr learned my addr 172.21.15.38:0/411425002 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:14.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.572+0000 7ffa81fef640 1 --2- 172.21.15.38:0/411425002 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558d0a3a75a0 0x558d0a44e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:14.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.572+0000 7ffa82ff1640 1 --2- 172.21.15.38:0/411425002 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x558d0a457920 0x558d0a44fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:14.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.572+0000 7ffa827f0640 1 -- 172.21.15.38:0/411425002 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x558d0a457920 msgr2=0x558d0a44fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:14.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.572+0000 7ffa827f0640 1 --2- 172.21.15.38:0/411425002 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x558d0a457920 0x558d0a44fae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:14.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.572+0000 7ffa827f0640 1 -- 172.21.15.38:0/411425002 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558d0a3a75a0 msgr2=0x558d0a44e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:14.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.572+0000 7ffa827f0640 1 --2- 172.21.15.38:0/411425002 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558d0a3a75a0 0x558d0a44e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:14.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.572+0000 7ffa827f0640 1 -- 172.21.15.38:0/411425002 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d0a451ca0 con 0x558d0a379d00 2024-09-21T03:40:14.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.572+0000 7ffa82ff1640 1 --2- 172.21.15.38:0/411425002 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x558d0a457920 0x558d0a44fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:40:14.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.572+0000 7ffa827f0640 1 --2- 172.21.15.38:0/411425002 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558d0a379d00 0x558d0a44db10 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7ffa7400b270 tx=0x7ffa7400b670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38df1fb51db9c89a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:14.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.572+0000 7ffa80fed640 1 -- 172.21.15.38:0/411425002 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa74016070 con 0x558d0a379d00 2024-09-21T03:40:14.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.573+0000 7ffa80fed640 1 -- 172.21.15.38:0/411425002 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffa7400d7a0 con 0x558d0a379d00 2024-09-21T03:40:14.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.573+0000 7ffa80fed640 1 -- 172.21.15.38:0/411425002 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa7400dbf0 con 0x558d0a379d00 2024-09-21T03:40:14.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.573+0000 7ffa88d36f40 1 -- 172.21.15.38:0/411425002 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558d0a379d00 msgr2=0x558d0a44db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:14.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.573+0000 7ffa88d36f40 1 --2- 172.21.15.38:0/411425002 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558d0a379d00 0x558d0a44db10 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7ffa7400b270 tx=0x7ffa7400b670 comp rx=0 tx=0).stop 2024-09-21T03:40:14.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.573+0000 7ffa88d36f40 1 -- 172.21.15.38:0/411425002 shutdown_connections 2024-09-21T03:40:14.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.573+0000 7ffa88d36f40 1 --2- 172.21.15.38:0/411425002 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x558d0a457920 0x558d0a44fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:14.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.573+0000 7ffa88d36f40 1 --2- 172.21.15.38:0/411425002 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558d0a3a75a0 0x558d0a44e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:14.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.573+0000 7ffa88d36f40 1 --2- 172.21.15.38:0/411425002 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558d0a379d00 0x558d0a44db10 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:14.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.573+0000 7ffa88d36f40 1 -- 172.21.15.38:0/411425002 >> 172.21.15.38:0/411425002 conn(0x558d0a298ac0 msgr2=0x558d0a4475d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:14.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.573+0000 7ffa88d36f40 1 -- 172.21.15.38:0/411425002 shutdown_connections 2024-09-21T03:40:14.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.573+0000 7ffa88d36f40 1 -- 172.21.15.38:0/411425002 wait complete. 2024-09-21T03:40:14.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.574+0000 7ffa88d36f40 1 Processor -- start 2024-09-21T03:40:14.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.574+0000 7ffa88d36f40 1 -- start start 2024-09-21T03:40:14.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.575+0000 7ffa88d36f40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x558d0a379d00 0x558d0a426250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:14.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.575+0000 7ffa88d36f40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558d0a3a75a0 0x558d0a445ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:14.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.575+0000 7ffa88d36f40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558d0a457920 0x558d0a446530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:14.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.575+0000 7ffa88d36f40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x558d0a44f370 con 0x558d0a457920 2024-09-21T03:40:14.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.575+0000 7ffa88d36f40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x558d0a451f40 con 0x558d0a3a75a0 2024-09-21T03:40:14.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.575+0000 7ffa88d36f40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x558d0a4258c0 con 0x558d0a379d00 2024-09-21T03:40:14.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.575+0000 7ffa82ff1640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558d0a457920 0x558d0a446530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:14.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.576+0000 7ffa82ff1640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558d0a457920 0x558d0a446530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:55124/0 (socket says 172.21.15.38:55124) 2024-09-21T03:40:14.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.576+0000 7ffa82ff1640 1 -- 172.21.15.38:0/1849460931 learned_addr learned my addr 172.21.15.38:0/1849460931 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:14.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.576+0000 7ffa827f0640 1 --2- 172.21.15.38:0/1849460931 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x558d0a379d00 0x558d0a426250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:14.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.576+0000 7ffa81fef640 1 --2- 172.21.15.38:0/1849460931 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558d0a3a75a0 0x558d0a445ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:14.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.576+0000 7ffa82ff1640 1 -- 172.21.15.38:0/1849460931 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x558d0a379d00 msgr2=0x558d0a426250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:14.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.576+0000 7ffa82ff1640 1 --2- 172.21.15.38:0/1849460931 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x558d0a379d00 0x558d0a426250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:14.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.576+0000 7ffa82ff1640 1 -- 172.21.15.38:0/1849460931 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558d0a3a75a0 msgr2=0x558d0a445ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:14.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.576+0000 7ffa82ff1640 1 --2- 172.21.15.38:0/1849460931 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558d0a3a75a0 0x558d0a445ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:14.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.576+0000 7ffa82ff1640 1 -- 172.21.15.38:0/1849460931 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d0a451650 con 0x558d0a457920 2024-09-21T03:40:14.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.576+0000 7ffa82ff1640 1 --2- 172.21.15.38:0/1849460931 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558d0a457920 0x558d0a446530 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7ffa7c025b60 tx=0x7ffa7c04ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:14.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.576+0000 7ffa737fe640 1 -- 172.21.15.38:0/1849460931 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa7c065070 con 0x558d0a457920 2024-09-21T03:40:14.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.576+0000 7ffa737fe640 1 -- 172.21.15.38:0/1849460931 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffa7c056400 con 0x558d0a457920 2024-09-21T03:40:14.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.576+0000 7ffa88d36f40 1 -- 172.21.15.38:0/1849460931 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x558d0a423aa0 con 0x558d0a457920 2024-09-21T03:40:14.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.576+0000 7ffa737fe640 1 -- 172.21.15.38:0/1849460931 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa7c0567f0 con 0x558d0a457920 2024-09-21T03:40:14.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.576+0000 7ffa88d36f40 1 -- 172.21.15.38:0/1849460931 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d0a424060 con 0x558d0a457920 2024-09-21T03:40:14.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.578+0000 7ffa737fe640 1 -- 172.21.15.38:0/1849460931 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ffa7c068070 con 0x558d0a457920 2024-09-21T03:40:14.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.578+0000 7ffa737fe640 1 --2- 172.21.15.38:0/1849460931 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ffa640775a0 0x7ffa64079ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:14.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.579+0000 7ffa737fe640 1 -- 172.21.15.38:0/1849460931 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6187+0+0 (secure 0 0 0) 0x7ffa7c0de180 con 0x558d0a457920 2024-09-21T03:40:14.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.579+0000 7ffa827f0640 1 --2- 172.21.15.38:0/1849460931 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ffa640775a0 0x7ffa64079ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:14.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.579+0000 7ffa88d36f40 1 --2- 172.21.15.38:0/1849460931 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x558d0a43b450 0x558d0a44be50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:14.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.579+0000 7ffa88d36f40 1 -- 172.21.15.38:0/1849460931 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:1 13.b 13:d0630e4c:::mds0_sessionmap:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x558d0a44c390 con 0x558d0a43b450 2024-09-21T03:40:14.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.579+0000 7ffa81fef640 1 --2- 172.21.15.38:0/1849460931 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x558d0a43b450 0x558d0a44be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:14.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.579+0000 7ffa827f0640 1 --2- 172.21.15.38:0/1849460931 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ffa640775a0 0x7ffa64079ab0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7ffa7400b270 tx=0x7ffa740011d0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:14.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.579+0000 7ffa81fef640 1 --2- 172.21.15.38:0/1849460931 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x558d0a43b450 0x558d0a44be50 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:14.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.580+0000 7ffa81fef640 1 -- 172.21.15.38:0/1849460931 <== osd.3 v2:172.21.15.38:6808/1286810294 1 ==== osd_op_reply(1 mds0_sessionmap [stat out=16b] v0'0 uv3 ondisk = 0) ==== 159+0+16 (crc 0 0 0) 0x7ffa7800e750 con 0x558d0a43b450 2024-09-21T03:40:14.583 INFO:teuthology.orchestra.run.smithi038.stdout:cephfs_metadata/mds0_sessionmap mtime 2024-09-21T03:40:11.000000+0000, size 0 2024-09-21T03:40:14.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.580+0000 7ffa88d36f40 1 -- 172.21.15.38:0/1849460931 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x558d0a43b450 msgr2=0x558d0a44be50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:14.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.580+0000 7ffa88d36f40 1 --2- 172.21.15.38:0/1849460931 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x558d0a43b450 0x558d0a44be50 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:14.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.580+0000 7ffa88d36f40 1 -- 172.21.15.38:0/1849460931 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ffa640775a0 msgr2=0x7ffa64079ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:14.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.580+0000 7ffa88d36f40 1 --2- 172.21.15.38:0/1849460931 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ffa640775a0 0x7ffa64079ab0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7ffa7400b270 tx=0x7ffa740011d0 comp rx=0 tx=0).stop 2024-09-21T03:40:14.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.580+0000 7ffa88d36f40 1 -- 172.21.15.38:0/1849460931 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558d0a457920 msgr2=0x558d0a446530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:14.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.580+0000 7ffa88d36f40 1 --2- 172.21.15.38:0/1849460931 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558d0a457920 0x558d0a446530 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7ffa7c025b60 tx=0x7ffa7c04ed30 comp rx=0 tx=0).stop 2024-09-21T03:40:14.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.580+0000 7ffa82ff1640 1 -- 172.21.15.38:0/1849460931 reap_dead start 2024-09-21T03:40:14.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.580+0000 7ffa88d36f40 1 -- 172.21.15.38:0/1849460931 shutdown_connections 2024-09-21T03:40:14.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.580+0000 7ffa88d36f40 1 -- 172.21.15.38:0/1849460931 >> 172.21.15.38:0/1849460931 conn(0x558d0a298ac0 msgr2=0x558d0a44fae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:14.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.580+0000 7ffa88d36f40 1 -- 172.21.15.38:0/1849460931 shutdown_connections 2024-09-21T03:40:14.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.580+0000 7ffa88d36f40 1 -- 172.21.15.38:0/1849460931 wait complete. 2024-09-21T03:40:14.585 DEBUG:teuthology.orchestra.run.smithi038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata stat 400.00000000 2024-09-21T03:40:14.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.639+0000 7f5b85526f40 1 Processor -- start 2024-09-21T03:40:14.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.639+0000 7f5b85526f40 1 -- start start 2024-09-21T03:40:14.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.640+0000 7f5b85526f40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562530c33d00 0x562530c61300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:14.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.640+0000 7f5b85526f40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562530c60e40 0x562530c61840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:14.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.640+0000 7f5b85526f40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562530cee950 0x562530d03d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:14.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.640+0000 7f5b85526f40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x562530be0430 con 0x562530c33d00 2024-09-21T03:40:14.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.640+0000 7f5b85526f40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x562530bed1f0 con 0x562530c60e40 2024-09-21T03:40:14.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.640+0000 7f5b85526f40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x562530b81d60 con 0x562530cee950 2024-09-21T03:40:14.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.640+0000 7f5b7e57a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562530c33d00 0x562530c61300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:14.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.640+0000 7f5b7e57a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562530c33d00 0x562530c61300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:55130/0 (socket says 172.21.15.38:55130) 2024-09-21T03:40:14.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.640+0000 7f5b7e57a640 1 -- 172.21.15.38:0/1983429557 learned_addr learned my addr 172.21.15.38:0/1983429557 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:14.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.640+0000 7f5b7e57a640 1 -- 172.21.15.38:0/1983429557 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562530cee950 msgr2=0x562530d03d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:14.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.640+0000 7f5b7e57a640 1 --2- 172.21.15.38:0/1983429557 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562530cee950 0x562530d03d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:14.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.640+0000 7f5b7e57a640 1 -- 172.21.15.38:0/1983429557 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562530c60e40 msgr2=0x562530c61840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:14.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.640+0000 7f5b7e57a640 1 --2- 172.21.15.38:0/1983429557 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562530c60e40 0x562530c61840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:14.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.640+0000 7f5b7e57a640 1 -- 172.21.15.38:0/1983429557 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562530c5e170 con 0x562530c33d00 2024-09-21T03:40:14.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.640+0000 7f5b7e57a640 1 --2- 172.21.15.38:0/1983429557 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562530c33d00 0x562530c61300 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f5b70001e10 tx=0x7f5b7000b650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebec23fe88b16733 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:14.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.641+0000 7f5b7cd77640 1 -- 172.21.15.38:0/1983429557 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b70016070 con 0x562530c33d00 2024-09-21T03:40:14.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.641+0000 7f5b7cd77640 1 -- 172.21.15.38:0/1983429557 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5b7000d7c0 con 0x562530c33d00 2024-09-21T03:40:14.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.641+0000 7f5b7cd77640 1 -- 172.21.15.38:0/1983429557 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b7000dc10 con 0x562530c33d00 2024-09-21T03:40:14.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.641+0000 7f5b85526f40 1 -- 172.21.15.38:0/1983429557 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562530c33d00 msgr2=0x562530c61300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:14.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.641+0000 7f5b85526f40 1 --2- 172.21.15.38:0/1983429557 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562530c33d00 0x562530c61300 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f5b70001e10 tx=0x7f5b7000b650 comp rx=0 tx=0).stop 2024-09-21T03:40:14.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.641+0000 7f5b85526f40 1 -- 172.21.15.38:0/1983429557 shutdown_connections 2024-09-21T03:40:14.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.641+0000 7f5b85526f40 1 --2- 172.21.15.38:0/1983429557 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562530cee950 0x562530d03d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:14.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.641+0000 7f5b85526f40 1 --2- 172.21.15.38:0/1983429557 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562530c60e40 0x562530c61840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:14.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.641+0000 7f5b85526f40 1 --2- 172.21.15.38:0/1983429557 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562530c33d00 0x562530c61300 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:14.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.641+0000 7f5b85526f40 1 -- 172.21.15.38:0/1983429557 >> 172.21.15.38:0/1983429557 conn(0x562530b52ac0 msgr2=0x562530d055c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:14.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.641+0000 7f5b85526f40 1 -- 172.21.15.38:0/1983429557 shutdown_connections 2024-09-21T03:40:14.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.641+0000 7f5b85526f40 1 -- 172.21.15.38:0/1983429557 wait complete. 2024-09-21T03:40:14.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.642+0000 7f5b85526f40 1 Processor -- start 2024-09-21T03:40:14.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.642+0000 7f5b85526f40 1 -- start start 2024-09-21T03:40:14.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.643+0000 7f5b85526f40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562530c33d00 0x562530da2230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:14.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.643+0000 7f5b85526f40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562530c60e40 0x562530d00000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:14.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.643+0000 7f5b85526f40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562530cee950 0x562530d050e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:14.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.643+0000 7f5b85526f40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x562530d04980 con 0x562530c60e40 2024-09-21T03:40:14.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.643+0000 7f5b85526f40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x562530cec260 con 0x562530cee950 2024-09-21T03:40:14.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.643+0000 7f5b85526f40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x562530d00ba0 con 0x562530c33d00 2024-09-21T03:40:14.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.644+0000 7f5b7dd79640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562530c60e40 0x562530d00000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:14.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.644+0000 7f5b7dd79640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562530c60e40 0x562530d00000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:55134/0 (socket says 172.21.15.38:55134) 2024-09-21T03:40:14.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.644+0000 7f5b7dd79640 1 -- 172.21.15.38:0/4154318045 learned_addr learned my addr 172.21.15.38:0/4154318045 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:14.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.644+0000 7f5b7e57a640 1 --2- 172.21.15.38:0/4154318045 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562530c33d00 0x562530da2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:14.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.644+0000 7f5b7ed7b640 1 --2- 172.21.15.38:0/4154318045 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562530cee950 0x562530d050e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:14.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.644+0000 7f5b7dd79640 1 -- 172.21.15.38:0/4154318045 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562530c33d00 msgr2=0x562530da2230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:14.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.644+0000 7f5b7dd79640 1 --2- 172.21.15.38:0/4154318045 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562530c33d00 0x562530da2230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:14.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.644+0000 7f5b7dd79640 1 -- 172.21.15.38:0/4154318045 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562530cee950 msgr2=0x562530d050e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:14.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.644+0000 7f5b7dd79640 1 --2- 172.21.15.38:0/4154318045 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562530cee950 0x562530d050e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:14.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.644+0000 7f5b7dd79640 1 -- 172.21.15.38:0/4154318045 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562530d08850 con 0x562530c60e40 2024-09-21T03:40:14.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.644+0000 7f5b7dd79640 1 --2- 172.21.15.38:0/4154318045 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562530c60e40 0x562530d00000 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f5b7400af70 tx=0x7f5b740084f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:14.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.648+0000 7f5b6f7fe640 1 -- 172.21.15.38:0/4154318045 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b74015070 con 0x562530c60e40 2024-09-21T03:40:14.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.648+0000 7f5b6f7fe640 1 -- 172.21.15.38:0/4154318045 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5b74001b50 con 0x562530c60e40 2024-09-21T03:40:14.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.648+0000 7f5b85526f40 1 -- 172.21.15.38:0/4154318045 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x562530d08b30 con 0x562530c60e40 2024-09-21T03:40:14.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.648+0000 7f5b6f7fe640 1 -- 172.21.15.38:0/4154318045 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b74003070 con 0x562530c60e40 2024-09-21T03:40:14.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.648+0000 7f5b85526f40 1 -- 172.21.15.38:0/4154318045 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562530da1d10 con 0x562530c60e40 2024-09-21T03:40:14.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.650+0000 7f5b6f7fe640 1 -- 172.21.15.38:0/4154318045 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f5b74018070 con 0x562530c60e40 2024-09-21T03:40:14.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.650+0000 7f5b6f7fe640 1 --2- 172.21.15.38:0/4154318045 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5b60077570 0x7f5b60079a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:14.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.650+0000 7f5b6f7fe640 1 -- 172.21.15.38:0/4154318045 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6187+0+0 (secure 0 0 0) 0x7f5b74098be0 con 0x562530c60e40 2024-09-21T03:40:14.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.650+0000 7f5b85526f40 1 --2- 172.21.15.38:0/4154318045 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x562530c2f820 0x562530c2fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:14.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.650+0000 7f5b85526f40 1 -- 172.21.15.38:0/4154318045 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:1 13.f 13:f1f69726:::400.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x562530c30170 con 0x562530c2f820 2024-09-21T03:40:14.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.650+0000 7f5b7e57a640 1 --2- 172.21.15.38:0/4154318045 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5b60077570 0x7f5b60079a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:14.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.650+0000 7f5b7ed7b640 1 --2- 172.21.15.38:0/4154318045 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x562530c2f820 0x562530c2fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:14.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.651+0000 7f5b7ed7b640 1 --2- 172.21.15.38:0/4154318045 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x562530c2f820 0x562530c2fc30 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:14.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.651+0000 7f5b7e57a640 1 --2- 172.21.15.38:0/4154318045 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5b60077570 0x7f5b60079a80 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f5b70000c00 tx=0x7f5b70001080 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:14.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.651+0000 7f5b7ed7b640 1 -- 172.21.15.38:0/4154318045 <== osd.2 v2:172.21.15.38:6824/2577858902 1 ==== osd_op_reply(1 400.00000000 [stat out=16b] v0'0 uv1 ondisk = 0) ==== 156+0+16 (crc 0 0 0) 0x7f5b7800b040 con 0x562530c2f820 2024-09-21T03:40:14.654 INFO:teuthology.orchestra.run.smithi038.stdout:cephfs_metadata/400.00000000 mtime 2024-09-21T03:40:02.000000+0000, size 22 2024-09-21T03:40:14.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.651+0000 7f5b85526f40 1 -- 172.21.15.38:0/4154318045 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x562530c2f820 msgr2=0x562530c2fc30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:14.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.651+0000 7f5b85526f40 1 --2- 172.21.15.38:0/4154318045 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x562530c2f820 0x562530c2fc30 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:14.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.651+0000 7f5b85526f40 1 -- 172.21.15.38:0/4154318045 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5b60077570 msgr2=0x7f5b60079a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:14.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.651+0000 7f5b85526f40 1 --2- 172.21.15.38:0/4154318045 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5b60077570 0x7f5b60079a80 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f5b70000c00 tx=0x7f5b70001080 comp rx=0 tx=0).stop 2024-09-21T03:40:14.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.651+0000 7f5b85526f40 1 -- 172.21.15.38:0/4154318045 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562530c60e40 msgr2=0x562530d00000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:14.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.651+0000 7f5b85526f40 1 --2- 172.21.15.38:0/4154318045 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562530c60e40 0x562530d00000 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f5b7400af70 tx=0x7f5b740084f0 comp rx=0 tx=0).stop 2024-09-21T03:40:14.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.651+0000 7f5b7ed7b640 1 -- 172.21.15.38:0/4154318045 reap_dead start 2024-09-21T03:40:14.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.651+0000 7f5b85526f40 1 -- 172.21.15.38:0/4154318045 shutdown_connections 2024-09-21T03:40:14.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.651+0000 7f5b85526f40 1 -- 172.21.15.38:0/4154318045 >> 172.21.15.38:0/4154318045 conn(0x562530b52ac0 msgr2=0x562530c61300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:14.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.651+0000 7f5b85526f40 1 -- 172.21.15.38:0/4154318045 shutdown_connections 2024-09-21T03:40:14.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.651+0000 7f5b85526f40 1 -- 172.21.15.38:0/4154318045 wait complete. 2024-09-21T03:40:14.656 DEBUG:teuthology.orchestra.run.smithi038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata stat 100.00000000.inode 2024-09-21T03:40:14.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.709+0000 7f5a15c29f40 1 Processor -- start 2024-09-21T03:40:14.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.709+0000 7f5a15c29f40 1 -- start start 2024-09-21T03:40:14.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.710+0000 7f5a15c29f40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e1cd549d00 0x55e1cd61f350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:14.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.710+0000 7f5a15c29f40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e1cd575e30 0x55e1cd600590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:14.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.710+0000 7f5a15c29f40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e1cd604950 0x55e1cd5765f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:14.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.710+0000 7f5a15c29f40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55e1cd497d60 con 0x55e1cd604950 2024-09-21T03:40:14.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.710+0000 7f5a15c29f40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55e1cd4f6430 con 0x55e1cd575e30 2024-09-21T03:40:14.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.710+0000 7f5a15c29f40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55e1cd592460 con 0x55e1cd549d00 2024-09-21T03:40:14.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.711+0000 7f5a0fee4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e1cd604950 0x55e1cd5765f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:14.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.711+0000 7f5a0fee4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e1cd604950 0x55e1cd5765f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:55156/0 (socket says 172.21.15.38:55156) 2024-09-21T03:40:14.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.711+0000 7f5a0fee4640 1 -- 172.21.15.38:0/125431858 learned_addr learned my addr 172.21.15.38:0/125431858 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:14.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.711+0000 7f5a0eee2640 1 --2- 172.21.15.38:0/125431858 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e1cd575e30 0x55e1cd600590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:14.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.711+0000 7f5a0f6e3640 1 --2- 172.21.15.38:0/125431858 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e1cd549d00 0x55e1cd61f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:14.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.711+0000 7f5a0fee4640 1 -- 172.21.15.38:0/125431858 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e1cd549d00 msgr2=0x55e1cd61f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:14.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.711+0000 7f5a0fee4640 1 --2- 172.21.15.38:0/125431858 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e1cd549d00 0x55e1cd61f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:14.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.711+0000 7f5a0fee4640 1 -- 172.21.15.38:0/125431858 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e1cd575e30 msgr2=0x55e1cd600590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:14.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.711+0000 7f5a0fee4640 1 --2- 172.21.15.38:0/125431858 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e1cd575e30 0x55e1cd600590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:14.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.711+0000 7f5a0fee4640 1 -- 172.21.15.38:0/125431858 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e1cd577f80 con 0x55e1cd604950 2024-09-21T03:40:14.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.711+0000 7f5a0eee2640 1 --2- 172.21.15.38:0/125431858 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e1cd575e30 0x55e1cd600590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:40:14.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.711+0000 7f5a0fee4640 1 --2- 172.21.15.38:0/125431858 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e1cd604950 0x55e1cd5765f0 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f5a0800c6a0 tx=0x7f5a0800cb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfd2c093cb7a38bc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:14.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.711+0000 7f5a0dee0640 1 -- 172.21.15.38:0/125431858 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a08004020 con 0x55e1cd604950 2024-09-21T03:40:14.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.711+0000 7f5a0dee0640 1 -- 172.21.15.38:0/125431858 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5a080042b0 con 0x55e1cd604950 2024-09-21T03:40:14.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.711+0000 7f5a0dee0640 1 -- 172.21.15.38:0/125431858 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a0800aa80 con 0x55e1cd604950 2024-09-21T03:40:14.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.712+0000 7f5a15c29f40 1 -- 172.21.15.38:0/125431858 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e1cd604950 msgr2=0x55e1cd5765f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:14.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.712+0000 7f5a15c29f40 1 --2- 172.21.15.38:0/125431858 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e1cd604950 0x55e1cd5765f0 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f5a0800c6a0 tx=0x7f5a0800cb60 comp rx=0 tx=0).stop 2024-09-21T03:40:14.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.712+0000 7f5a15c29f40 1 -- 172.21.15.38:0/125431858 shutdown_connections 2024-09-21T03:40:14.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.712+0000 7f5a15c29f40 1 --2- 172.21.15.38:0/125431858 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e1cd604950 0x55e1cd5765f0 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:14.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.712+0000 7f5a15c29f40 1 --2- 172.21.15.38:0/125431858 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e1cd575e30 0x55e1cd600590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:14.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.712+0000 7f5a15c29f40 1 --2- 172.21.15.38:0/125431858 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e1cd549d00 0x55e1cd61f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:14.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.712+0000 7f5a15c29f40 1 -- 172.21.15.38:0/125431858 >> 172.21.15.38:0/125431858 conn(0x55e1cd468ac0 msgr2=0x55e1cd61b530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:14.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.712+0000 7f5a15c29f40 1 -- 172.21.15.38:0/125431858 shutdown_connections 2024-09-21T03:40:14.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.712+0000 7f5a15c29f40 1 -- 172.21.15.38:0/125431858 wait complete. 2024-09-21T03:40:14.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.713+0000 7f5a15c29f40 1 Processor -- start 2024-09-21T03:40:14.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.713+0000 7f5a15c29f40 1 -- start start 2024-09-21T03:40:14.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.714+0000 7f5a15c29f40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e1cd549d00 0x55e1cd615790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:14.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.714+0000 7f5a15c29f40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e1cd575e30 0x55e1cd612b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:14.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.714+0000 7f5a15c29f40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e1cd604950 0x55e1cd613620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:14.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.714+0000 7f5a15c29f40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55e1cd578220 con 0x55e1cd604950 2024-09-21T03:40:14.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.714+0000 7f5a15c29f40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55e1cd61bdf0 con 0x55e1cd575e30 2024-09-21T03:40:14.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.714+0000 7f5a15c29f40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55e1cd5feb50 con 0x55e1cd549d00 2024-09-21T03:40:14.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.714+0000 7f5a0fee4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e1cd604950 0x55e1cd613620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:14.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.714+0000 7f5a0fee4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e1cd604950 0x55e1cd613620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:55160/0 (socket says 172.21.15.38:55160) 2024-09-21T03:40:14.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.714+0000 7f5a0fee4640 1 -- 172.21.15.38:0/2171952454 learned_addr learned my addr 172.21.15.38:0/2171952454 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:14.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.714+0000 7f5a0f6e3640 1 --2- 172.21.15.38:0/2171952454 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e1cd549d00 0x55e1cd615790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:14.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.714+0000 7f5a0eee2640 1 --2- 172.21.15.38:0/2171952454 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e1cd575e30 0x55e1cd612b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:14.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.714+0000 7f5a0fee4640 1 -- 172.21.15.38:0/2171952454 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e1cd549d00 msgr2=0x55e1cd615790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:14.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.714+0000 7f5a0fee4640 1 --2- 172.21.15.38:0/2171952454 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e1cd549d00 0x55e1cd615790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:14.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.714+0000 7f5a0fee4640 1 -- 172.21.15.38:0/2171952454 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e1cd575e30 msgr2=0x55e1cd612b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:14.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.714+0000 7f5a0fee4640 1 --2- 172.21.15.38:0/2171952454 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e1cd575e30 0x55e1cd612b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:14.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.714+0000 7f5a0fee4640 1 -- 172.21.15.38:0/2171952454 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e1cd615cd0 con 0x55e1cd604950 2024-09-21T03:40:14.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.714+0000 7f5a0f6e3640 1 --2- 172.21.15.38:0/2171952454 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e1cd549d00 0x55e1cd615790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:40:14.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.714+0000 7f5a0eee2640 1 --2- 172.21.15.38:0/2171952454 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e1cd575e30 0x55e1cd612b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:40:14.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.715+0000 7f5a0fee4640 1 --2- 172.21.15.38:0/2171952454 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e1cd604950 0x55e1cd613620 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x55e1cd5ffc10 tx=0x7f5a0800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:14.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.715+0000 7f5a0cede640 1 -- 172.21.15.38:0/2171952454 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a08025070 con 0x55e1cd604950 2024-09-21T03:40:14.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.715+0000 7f5a0cede640 1 -- 172.21.15.38:0/2171952454 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5a080042b0 con 0x55e1cd604950 2024-09-21T03:40:14.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.715+0000 7f5a15c29f40 1 -- 172.21.15.38:0/2171952454 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55e1cd61bb80 con 0x55e1cd604950 2024-09-21T03:40:14.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.715+0000 7f5a0cede640 1 -- 172.21.15.38:0/2171952454 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a0800fa10 con 0x55e1cd604950 2024-09-21T03:40:14.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.715+0000 7f5a15c29f40 1 -- 172.21.15.38:0/2171952454 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e1cd572260 con 0x55e1cd604950 2024-09-21T03:40:14.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.717+0000 7f5a0cede640 1 -- 172.21.15.38:0/2171952454 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f5a08028070 con 0x55e1cd604950 2024-09-21T03:40:14.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.717+0000 7f5a0cede640 1 --2- 172.21.15.38:0/2171952454 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f59fc077570 0x7f59fc079a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:14.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.717+0000 7f5a0cede640 1 -- 172.21.15.38:0/2171952454 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6187+0+0 (secure 0 0 0) 0x7f5a0809b450 con 0x55e1cd604950 2024-09-21T03:40:14.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.717+0000 7f5a0f6e3640 1 --2- 172.21.15.38:0/2171952454 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f59fc077570 0x7f59fc079a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:14.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.717+0000 7f5a15c29f40 1 --2- 172.21.15.38:0/2171952454 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x55e1cd61d650 0x55e1cd61da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:14.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.717+0000 7f5a15c29f40 1 -- 172.21.15.38:0/2171952454 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:1 13.1f 13:fe07bba1:::100.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x55e1cd5fe3d0 con 0x55e1cd61d650 2024-09-21T03:40:14.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.717+0000 7f5a0eee2640 1 --2- 172.21.15.38:0/2171952454 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x55e1cd61d650 0x55e1cd61da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:14.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.718+0000 7f5a0f6e3640 1 --2- 172.21.15.38:0/2171952454 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f59fc077570 0x7f59fc079a80 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f5a00002e70 tx=0x7f5a00002470 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:14.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.718+0000 7f5a0eee2640 1 --2- 172.21.15.38:0/2171952454 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x55e1cd61d650 0x55e1cd61da60 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:14.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.718+0000 7f5a0eee2640 1 -- 172.21.15.38:0/2171952454 <== osd.1 v2:172.21.15.38:6816/1491818732 1 ==== osd_op_reply(1 100.00000000.inode [stat out=16b] v0'0 uv3 ondisk = 0) ==== 162+0+16 (crc 0 0 0) 0x7f5a0404db50 con 0x55e1cd61d650 2024-09-21T03:40:14.722 INFO:teuthology.orchestra.run.smithi038.stdout:cephfs_metadata/100.00000000.inode mtime 2024-09-21T03:40:02.000000+0000, size 582 2024-09-21T03:40:14.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.718+0000 7f5a15c29f40 1 -- 172.21.15.38:0/2171952454 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x55e1cd61d650 msgr2=0x55e1cd61da60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:14.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.718+0000 7f5a15c29f40 1 --2- 172.21.15.38:0/2171952454 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x55e1cd61d650 0x55e1cd61da60 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:14.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.718+0000 7f5a15c29f40 1 -- 172.21.15.38:0/2171952454 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f59fc077570 msgr2=0x7f59fc079a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:14.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.718+0000 7f5a15c29f40 1 --2- 172.21.15.38:0/2171952454 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f59fc077570 0x7f59fc079a80 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f5a00002e70 tx=0x7f5a00002470 comp rx=0 tx=0).stop 2024-09-21T03:40:14.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.718+0000 7f5a15c29f40 1 -- 172.21.15.38:0/2171952454 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e1cd604950 msgr2=0x55e1cd613620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:14.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.718+0000 7f5a15c29f40 1 --2- 172.21.15.38:0/2171952454 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e1cd604950 0x55e1cd613620 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x55e1cd5ffc10 tx=0x7f5a0800f690 comp rx=0 tx=0).stop 2024-09-21T03:40:14.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.718+0000 7f5a0fee4640 1 -- 172.21.15.38:0/2171952454 reap_dead start 2024-09-21T03:40:14.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.718+0000 7f5a15c29f40 1 -- 172.21.15.38:0/2171952454 shutdown_connections 2024-09-21T03:40:14.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.718+0000 7f5a15c29f40 1 -- 172.21.15.38:0/2171952454 >> 172.21.15.38:0/2171952454 conn(0x55e1cd468ac0 msgr2=0x55e1cd5fee50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:14.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.718+0000 7f5a15c29f40 1 -- 172.21.15.38:0/2171952454 shutdown_connections 2024-09-21T03:40:14.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.718+0000 7f5a15c29f40 1 -- 172.21.15.38:0/2171952454 wait complete. 2024-09-21T03:40:14.725 DEBUG:teuthology.orchestra.run.smithi038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata stat 1.00000000 2024-09-21T03:40:14.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.778+0000 7f9762f20f40 1 Processor -- start 2024-09-21T03:40:14.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.778+0000 7f9762f20f40 1 -- start start 2024-09-21T03:40:14.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.779+0000 7f9762f20f40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561f353f2d00 0x561f35420530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:14.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.779+0000 7f9762f20f40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x561f354ad950 0x561f3541fde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:14.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.779+0000 7f9762f20f40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561f354c8670 0x561f354c52e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:14.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.779+0000 7f9762f20f40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x561f35340d60 con 0x561f353f2d00 2024-09-21T03:40:14.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.779+0000 7f9762f20f40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x561f3539f430 con 0x561f354c8670 2024-09-21T03:40:14.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.779+0000 7f9762f20f40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x561f3543a7b0 con 0x561f354ad950 2024-09-21T03:40:14.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.779+0000 7f975c0c0640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561f353f2d00 0x561f35420530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:14.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.779+0000 7f975c0c0640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561f353f2d00 0x561f35420530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:55172/0 (socket says 172.21.15.38:55172) 2024-09-21T03:40:14.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.779+0000 7f975c0c0640 1 -- 172.21.15.38:0/4162440160 learned_addr learned my addr 172.21.15.38:0/4162440160 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:14.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.779+0000 7f975b8bf640 1 --2- 172.21.15.38:0/4162440160 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x561f354ad950 0x561f3541fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:14.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.779+0000 7f975c8c1640 1 --2- 172.21.15.38:0/4162440160 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561f354c8670 0x561f354c52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:14.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.779+0000 7f975c0c0640 1 -- 172.21.15.38:0/4162440160 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x561f354ad950 msgr2=0x561f3541fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:14.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.779+0000 7f975c0c0640 1 --2- 172.21.15.38:0/4162440160 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x561f354ad950 0x561f3541fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:14.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.779+0000 7f975c0c0640 1 -- 172.21.15.38:0/4162440160 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561f354c8670 msgr2=0x561f354c52e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:14.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.779+0000 7f975c0c0640 1 --2- 172.21.15.38:0/4162440160 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561f354c8670 0x561f354c52e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:14.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.779+0000 7f975c0c0640 1 -- 172.21.15.38:0/4162440160 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f354c7910 con 0x561f353f2d00 2024-09-21T03:40:14.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.780+0000 7f975c0c0640 1 --2- 172.21.15.38:0/4162440160 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561f353f2d00 0x561f35420530 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f974c025e80 tx=0x7f974c002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb6c5ceb40e27e14 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:14.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.780+0000 7f975a8bd640 1 -- 172.21.15.38:0/4162440160 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f974c04dd70 con 0x561f353f2d00 2024-09-21T03:40:14.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.780+0000 7f975a8bd640 1 -- 172.21.15.38:0/4162440160 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f974c056470 con 0x561f353f2d00 2024-09-21T03:40:14.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.780+0000 7f975a8bd640 1 -- 172.21.15.38:0/4162440160 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f974c056880 con 0x561f353f2d00 2024-09-21T03:40:14.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.780+0000 7f9762f20f40 1 -- 172.21.15.38:0/4162440160 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561f353f2d00 msgr2=0x561f35420530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:14.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.780+0000 7f9762f20f40 1 --2- 172.21.15.38:0/4162440160 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561f353f2d00 0x561f35420530 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f974c025e80 tx=0x7f974c002fb0 comp rx=0 tx=0).stop 2024-09-21T03:40:14.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.780+0000 7f9762f20f40 1 -- 172.21.15.38:0/4162440160 shutdown_connections 2024-09-21T03:40:14.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.780+0000 7f9762f20f40 1 --2- 172.21.15.38:0/4162440160 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561f354c8670 0x561f354c52e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:14.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.780+0000 7f9762f20f40 1 --2- 172.21.15.38:0/4162440160 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x561f354ad950 0x561f3541fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:14.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.780+0000 7f9762f20f40 1 --2- 172.21.15.38:0/4162440160 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561f353f2d00 0x561f35420530 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:14.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.780+0000 7f9762f20f40 1 -- 172.21.15.38:0/4162440160 >> 172.21.15.38:0/4162440160 conn(0x561f35311ac0 msgr2=0x561f354c8a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:14.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.780+0000 7f9762f20f40 1 -- 172.21.15.38:0/4162440160 shutdown_connections 2024-09-21T03:40:14.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.781+0000 7f9762f20f40 1 -- 172.21.15.38:0/4162440160 wait complete. 2024-09-21T03:40:14.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.781+0000 7f9762f20f40 1 Processor -- start 2024-09-21T03:40:14.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.781+0000 7f9762f20f40 1 -- start start 2024-09-21T03:40:14.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.783+0000 7f9762f20f40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x561f353f2d00 0x561f354c3db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:14.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.783+0000 7f9762f20f40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561f354ad950 0x561f354a5530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:14.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.783+0000 7f9762f20f40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561f354c8670 0x561f355528c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:14.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.783+0000 7f9762f20f40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x561f35553390 con 0x561f354c8670 2024-09-21T03:40:14.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.783+0000 7f9762f20f40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x561f354a4220 con 0x561f354ad950 2024-09-21T03:40:14.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.783+0000 7f9762f20f40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x561f354a50a0 con 0x561f353f2d00 2024-09-21T03:40:14.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.783+0000 7f975c8c1640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561f354c8670 0x561f355528c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:14.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.783+0000 7f975c8c1640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561f354c8670 0x561f355528c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:55178/0 (socket says 172.21.15.38:55178) 2024-09-21T03:40:14.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.783+0000 7f975c8c1640 1 -- 172.21.15.38:0/2592650513 learned_addr learned my addr 172.21.15.38:0/2592650513 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:14.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.783+0000 7f975b8bf640 1 --2- 172.21.15.38:0/2592650513 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561f354ad950 0x561f354a5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:14.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.783+0000 7f975c0c0640 1 --2- 172.21.15.38:0/2592650513 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x561f353f2d00 0x561f354c3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:14.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.783+0000 7f975c8c1640 1 -- 172.21.15.38:0/2592650513 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x561f353f2d00 msgr2=0x561f354c3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:14.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.783+0000 7f975c8c1640 1 --2- 172.21.15.38:0/2592650513 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x561f353f2d00 0x561f354c3db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:14.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.783+0000 7f975c8c1640 1 -- 172.21.15.38:0/2592650513 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561f354ad950 msgr2=0x561f354a5530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:14.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.783+0000 7f975c8c1640 1 --2- 172.21.15.38:0/2592650513 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561f354ad950 0x561f354a5530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:14.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.783+0000 7f975c8c1640 1 -- 172.21.15.38:0/2592650513 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f3541eed0 con 0x561f354c8670 2024-09-21T03:40:14.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.783+0000 7f975c8c1640 1 --2- 172.21.15.38:0/2592650513 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561f354c8670 0x561f355528c0 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f975400a0e0 tx=0x7f975400a5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:14.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.784+0000 7f97598bb640 1 -- 172.21.15.38:0/2592650513 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9754015020 con 0x561f354c8670 2024-09-21T03:40:14.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.784+0000 7f97598bb640 1 -- 172.21.15.38:0/2592650513 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9754011420 con 0x561f354c8670 2024-09-21T03:40:14.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.784+0000 7f9762f20f40 1 -- 172.21.15.38:0/2592650513 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x561f354a4a50 con 0x561f354c8670 2024-09-21T03:40:14.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.784+0000 7f97598bb640 1 -- 172.21.15.38:0/2592650513 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9754011870 con 0x561f354c8670 2024-09-21T03:40:14.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.784+0000 7f9762f20f40 1 -- 172.21.15.38:0/2592650513 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f354c6620 con 0x561f354c8670 2024-09-21T03:40:14.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.785+0000 7f97598bb640 1 -- 172.21.15.38:0/2592650513 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f9754018020 con 0x561f354c8670 2024-09-21T03:40:14.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.786+0000 7f97598bb640 1 --2- 172.21.15.38:0/2592650513 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f97480775a0 0x7f9748079ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:14.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.786+0000 7f97598bb640 1 -- 172.21.15.38:0/2592650513 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6187+0+0 (secure 0 0 0) 0x7f97540999a0 con 0x561f354c8670 2024-09-21T03:40:14.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.786+0000 7f9762f20f40 1 --2- 172.21.15.38:0/2592650513 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x561f354aafc0 0x561f354ab3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:14.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.786+0000 7f9762f20f40 1 -- 172.21.15.38:0/2592650513 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:1 13.1f 13:ff5b34d6:::1.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x561f354c4bb0 con 0x561f354aafc0 2024-09-21T03:40:14.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.786+0000 7f975c0c0640 1 --2- 172.21.15.38:0/2592650513 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f97480775a0 0x7f9748079ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:14.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.786+0000 7f975b8bf640 1 --2- 172.21.15.38:0/2592650513 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x561f354aafc0 0x561f354ab3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:14.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.787+0000 7f975b8bf640 1 --2- 172.21.15.38:0/2592650513 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x561f354aafc0 0x561f354ab3d0 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:14.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.787+0000 7f975c0c0640 1 --2- 172.21.15.38:0/2592650513 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f97480775a0 0x7f9748079ab0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f974c062000 tx=0x7f974c004940 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:14.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.787+0000 7f975b8bf640 1 -- 172.21.15.38:0/2592650513 <== osd.1 v2:172.21.15.38:6816/1491818732 1 ==== osd_op_reply(1 1.00000000 [stat out=16b] v0'0 uv6 ondisk = 0) ==== 154+0+16 (crc 0 0 0) 0x7f975000c020 con 0x561f354aafc0 2024-09-21T03:40:14.790 INFO:teuthology.orchestra.run.smithi038.stdout:cephfs_metadata/1.00000000 mtime 2024-09-21T03:40:11.000000+0000, size 0 2024-09-21T03:40:14.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.787+0000 7f9762f20f40 1 -- 172.21.15.38:0/2592650513 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x561f354aafc0 msgr2=0x561f354ab3d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:14.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.787+0000 7f9762f20f40 1 --2- 172.21.15.38:0/2592650513 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x561f354aafc0 0x561f354ab3d0 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:14.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.787+0000 7f9762f20f40 1 -- 172.21.15.38:0/2592650513 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f97480775a0 msgr2=0x7f9748079ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:14.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.787+0000 7f9762f20f40 1 --2- 172.21.15.38:0/2592650513 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f97480775a0 0x7f9748079ab0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f974c062000 tx=0x7f974c004940 comp rx=0 tx=0).stop 2024-09-21T03:40:14.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.787+0000 7f9762f20f40 1 -- 172.21.15.38:0/2592650513 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561f354c8670 msgr2=0x561f355528c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:14.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.787+0000 7f9762f20f40 1 --2- 172.21.15.38:0/2592650513 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561f354c8670 0x561f355528c0 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f975400a0e0 tx=0x7f975400a5a0 comp rx=0 tx=0).stop 2024-09-21T03:40:14.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.787+0000 7f975c8c1640 1 -- 172.21.15.38:0/2592650513 reap_dead start 2024-09-21T03:40:14.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.787+0000 7f9762f20f40 1 -- 172.21.15.38:0/2592650513 shutdown_connections 2024-09-21T03:40:14.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.787+0000 7f9762f20f40 1 -- 172.21.15.38:0/2592650513 >> 172.21.15.38:0/2592650513 conn(0x561f35311ac0 msgr2=0x561f354c52e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:14.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.787+0000 7f9762f20f40 1 -- 172.21.15.38:0/2592650513 shutdown_connections 2024-09-21T03:40:14.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:14.787+0000 7f9762f20f40 1 -- 172.21.15.38:0/2592650513 wait complete. 2024-09-21T03:40:14.793 INFO:tasks.cephfs.test_damage:Expectation on rm '10000000000.00000000' will be 'no damage' 2024-09-21T03:40:14.793 INFO:tasks.cephfs.test_damage:Expectation on rm '600.00000000' will be 'damaged_on_start' 2024-09-21T03:40:14.793 INFO:tasks.cephfs.test_damage:Expectation on rm '1.00000000.inode' will be 'damaged_on_start' 2024-09-21T03:40:14.793 INFO:tasks.cephfs.test_damage:Expectation on rm '200.00000000' will be 'damaged_on_start' 2024-09-21T03:40:14.793 INFO:tasks.cephfs.test_damage:Expectation on rm '200.00000001' will be 'damaged_on_start' 2024-09-21T03:40:14.793 INFO:tasks.cephfs.test_damage:Expectation on rm 'mds0_openfiles.0' will be 'no damage' 2024-09-21T03:40:14.793 INFO:tasks.cephfs.test_damage:Expectation on rm '500.00000000' will be 'no damage' 2024-09-21T03:40:14.793 INFO:tasks.cephfs.test_damage:Expectation on rm 'mds_snaptable' will be 'damaged_on_start' 2024-09-21T03:40:14.793 INFO:tasks.cephfs.test_damage:Expectation on rm 'mds0_inotable' will be 'damaged_on_start' 2024-09-21T03:40:14.793 INFO:tasks.cephfs.test_damage:Expectation on rm '100.00000000' will be 'damaged_on_start' 2024-09-21T03:40:14.794 INFO:tasks.cephfs.test_damage:Expectation on rm 'mds0_sessionmap' will be 'damaged_on_start' 2024-09-21T03:40:14.794 INFO:tasks.cephfs.test_damage:Expectation on rm '400.00000000' will be 'no damage' 2024-09-21T03:40:14.794 INFO:tasks.cephfs.test_damage:Expectation on rm '100.00000000.inode' will be 'damaged_on_start' 2024-09-21T03:40:14.794 INFO:tasks.cephfs.test_damage:Expectation on rm '1.00000000' will be 'damaged_on_start' 2024-09-21T03:40:14.794 INFO:tasks.cephfs.test_damage:Expectation on corrupt header '10000000000.00000000' will be 'no damage' 2024-09-21T03:40:14.794 INFO:tasks.cephfs.test_damage:Expectation on corrupt header '600.00000000' will be 'damaged_on_start' 2024-09-21T03:40:14.794 INFO:tasks.cephfs.test_damage:Expectation on corrupt header 'mds0_openfiles.0' will be 'no damage' 2024-09-21T03:40:14.794 INFO:tasks.cephfs.test_damage:Expectation on corrupt header '100.00000000' will be 'damaged_on_start' 2024-09-21T03:40:14.794 INFO:tasks.cephfs.test_damage:Expectation on corrupt header 'mds0_sessionmap' will be 'damaged_on_start' 2024-09-21T03:40:14.794 INFO:tasks.cephfs.test_damage:Expectation on corrupt header '1.00000000' will be 'damaged_on_start' 2024-09-21T03:40:14.794 INFO:tasks.cephfs.test_damage:Applying mutation 'Delete 10000000000.00000000' 2024-09-21T03:40:14.794 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-21T03:40:14.795 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:40:14.810 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056795,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2354 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T03:40:14.810 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi038.front.sepia.ceph.com 2024-09-21T03:40:14.811 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:40:14.811 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T03:40:14.866 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T03:40:14.866 INFO:teuthology.orchestra.run.smithi038.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-09-21T03:40:14.867 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-21T03:40:15.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:15.022+0000 7f6ab29b0640 1 -- 172.21.15.38:0/356267092 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6aac0d1120 msgr2=0x7f6aac0ceb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:15.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:15.022+0000 7f6ab29b0640 1 --2- 172.21.15.38:0/356267092 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6aac0d1120 0x7f6aac0ceb90 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f6aa4005fd0 tx=0x7f6aa402aa10 comp rx=0 tx=0).stop 2024-09-21T03:40:15.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:15.022+0000 7f6ab29b0640 1 -- 172.21.15.38:0/356267092 shutdown_connections 2024-09-21T03:40:15.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:15.022+0000 7f6ab29b0640 1 --2- 172.21.15.38:0/356267092 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6aac0d1120 0x7f6aac0ceb90 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:15.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:15.022+0000 7f6ab29b0640 1 --2- 172.21.15.38:0/356267092 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6aac0d0170 0x7f6aac0d0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:15.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:15.022+0000 7f6ab29b0640 1 -- 172.21.15.38:0/356267092 >> 172.21.15.38:0/356267092 conn(0x7f6aac09d830 msgr2=0x7f6aac0b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:15.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:15.022+0000 7f6ab29b0640 1 -- 172.21.15.38:0/356267092 shutdown_connections 2024-09-21T03:40:15.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:15.022+0000 7f6ab29b0640 1 -- 172.21.15.38:0/356267092 wait complete. 2024-09-21T03:40:15.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:15.024+0000 7f6ab29b0640 1 Processor -- start 2024-09-21T03:40:15.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:15.024+0000 7f6ab29b0640 1 -- start start 2024-09-21T03:40:15.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:15.025+0000 7f6ab29b0640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6aac0d0170 0x7f6aac0a1e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:15.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:15.025+0000 7f6ab29b0640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6aac0d1120 0x7f6aac0ce440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:15.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:15.025+0000 7f6ab29b0640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6aac15ee80 0x7f6aac1f87f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:15.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:15.025+0000 7f6ab29b0640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f6aac0b6700 con 0x7f6aac0d0170 2024-09-21T03:40:15.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:15.025+0000 7f6ab29b0640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f6aac0b6580 con 0x7f6aac0d1120 2024-09-21T03:40:15.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:15.025+0000 7f6ab29b0640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f6aac0b6880 con 0x7f6aac15ee80 2024-09-21T03:40:15.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:15.025+0000 7f6aabfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6aac0d0170 0x7f6aac0a1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:15.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:15.025+0000 7f6aabfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6aac0d0170 0x7f6aac0a1e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:55202/0 (socket says 172.21.15.38:55202) 2024-09-21T03:40:15.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:15.025+0000 7f6aabfff640 1 -- 172.21.15.38:0/2180293681 learned_addr learned my addr 172.21.15.38:0/2180293681 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:15.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:15.026+0000 7f6aab7fe640 1 --2- 172.21.15.38:0/2180293681 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6aac0d1120 0x7f6aac0ce440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:15.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:15.026+0000 7f6ab0f4d640 1 --2- 172.21.15.38:0/2180293681 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6aac15ee80 0x7f6aac1f87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:15.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:15.026+0000 7f6aabfff640 1 -- 172.21.15.38:0/2180293681 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6aac15ee80 msgr2=0x7f6aac1f87f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:15.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:15.026+0000 7f6aabfff640 1 --2- 172.21.15.38:0/2180293681 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6aac15ee80 0x7f6aac1f87f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:15.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:15.026+0000 7f6aabfff640 1 -- 172.21.15.38:0/2180293681 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6aac0d1120 msgr2=0x7f6aac0ce440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:15.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:15.026+0000 7f6aabfff640 1 --2- 172.21.15.38:0/2180293681 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6aac0d1120 0x7f6aac0ce440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:15.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:15.026+0000 7f6aabfff640 1 -- 172.21.15.38:0/2180293681 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6aac0ce980 con 0x7f6aac0d0170 2024-09-21T03:40:15.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:15.026+0000 7f6aabfff640 1 --2- 172.21.15.38:0/2180293681 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6aac0d0170 0x7f6aac0a1e10 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f6a9c00c3a0 tx=0x7f6a9c00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:15.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:15.026+0000 7f6aa8ff9640 1 -- 172.21.15.38:0/2180293681 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a9c017070 con 0x7f6aac0d0170 2024-09-21T03:40:15.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:15.026+0000 7f6aa8ff9640 1 -- 172.21.15.38:0/2180293681 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6a9c0053c0 con 0x7f6aac0d0170 2024-09-21T03:40:15.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:15.026+0000 7f6ab29b0640 1 -- 172.21.15.38:0/2180293681 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6aac0cbbe0 con 0x7f6aac0d0170 2024-09-21T03:40:15.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:15.026+0000 7f6aa8ff9640 1 -- 172.21.15.38:0/2180293681 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a9c003850 con 0x7f6aac0d0170 2024-09-21T03:40:15.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:15.026+0000 7f6ab29b0640 1 -- 172.21.15.38:0/2180293681 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6aac0cc1a0 con 0x7f6aac0d0170 2024-09-21T03:40:15.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:15.027+0000 7f6ab29b0640 1 -- 172.21.15.38:0/2180293681 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6aac0b6da0 con 0x7f6aac0d0170 2024-09-21T03:40:15.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:15.028+0000 7f6aa8ff9640 1 -- 172.21.15.38:0/2180293681 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f6a9c01a070 con 0x7f6aac0d0170 2024-09-21T03:40:15.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:15.028+0000 7f6aa8ff9640 1 --2- 172.21.15.38:0/2180293681 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6a98078b90 0x7f6a9807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:15.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:15.028+0000 7f6aa8ff9640 1 -- 172.21.15.38:0/2180293681 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6187+0+0 (secure 0 0 0) 0x7f6a9c09a300 con 0x7f6aac0d0170 2024-09-21T03:40:15.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:15.029+0000 7f6aab7fe640 1 --2- 172.21.15.38:0/2180293681 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6a98078b90 0x7f6a9807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:15.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:15.029+0000 7f6aab7fe640 1 --2- 172.21.15.38:0/2180293681 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6a98078b90 0x7f6a9807b0a0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f6aa0005fd0 tx=0x7f6aa0005de0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:15.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:15.033+0000 7f6aa8ff9640 1 -- 172.21.15.38:0/2180293681 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f6aac1592d0 con 0x7f6aac0d0170 2024-09-21T03:40:15.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:15.258+0000 7f6ab29b0640 1 -- 172.21.15.38:0/2180293681 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f6aac0caf50 con 0x7f6aac0d0170 2024-09-21T03:40:15.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:15.652+0000 7f6aa8ff9640 1 -- 172.21.15.38:0/2180293681 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v86) ==== 161+0+0 (secure 0 0 0) 0x7f6a9c062560 con 0x7f6aac0d0170 2024-09-21T03:40:15.654 INFO:teuthology.orchestra.run.smithi038.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-21T03:40:15.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:15.654+0000 7f6ab29b0640 1 -- 172.21.15.38:0/2180293681 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6a98078b90 msgr2=0x7f6a9807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:15.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:15.654+0000 7f6ab29b0640 1 --2- 172.21.15.38:0/2180293681 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6a98078b90 0x7f6a9807b0a0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f6aa0005fd0 tx=0x7f6aa0005de0 comp rx=0 tx=0).stop 2024-09-21T03:40:15.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:15.654+0000 7f6ab29b0640 1 -- 172.21.15.38:0/2180293681 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6aac0d0170 msgr2=0x7f6aac0a1e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:15.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:15.654+0000 7f6ab29b0640 1 --2- 172.21.15.38:0/2180293681 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6aac0d0170 0x7f6aac0a1e10 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f6a9c00c3a0 tx=0x7f6a9c00c860 comp rx=0 tx=0).stop 2024-09-21T03:40:15.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:15.654+0000 7f6ab29b0640 1 -- 172.21.15.38:0/2180293681 shutdown_connections 2024-09-21T03:40:15.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:15.654+0000 7f6ab29b0640 1 --2- 172.21.15.38:0/2180293681 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6a98078b90 0x7f6a9807b0a0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:15.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:15.654+0000 7f6ab29b0640 1 --2- 172.21.15.38:0/2180293681 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6aac15ee80 0x7f6aac1f87f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:15.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:15.654+0000 7f6ab29b0640 1 --2- 172.21.15.38:0/2180293681 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6aac0d1120 0x7f6aac0ce440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:15.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:15.654+0000 7f6ab29b0640 1 --2- 172.21.15.38:0/2180293681 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6aac0d0170 0x7f6aac0a1e10 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:15.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:15.654+0000 7f6ab29b0640 1 -- 172.21.15.38:0/2180293681 >> 172.21.15.38:0/2180293681 conn(0x7f6aac09d830 msgr2=0x7f6aac0d09e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:15.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:15.655+0000 7f6ab29b0640 1 -- 172.21.15.38:0/2180293681 shutdown_connections 2024-09-21T03:40:15.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:15.655+0000 7f6ab29b0640 1 -- 172.21.15.38:0/2180293681 wait complete. 2024-09-21T03:40:15.669 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-09-21T03:40:15.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:15.787+0000 7f0de7107640 1 -- 172.21.15.38:0/1966333971 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0de00d0170 msgr2=0x7f0de00d0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:15.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:15.787+0000 7f0de7107640 1 --2- 172.21.15.38:0/1966333971 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0de00d0170 0x7f0de00d0be0 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f0dd0009820 tx=0x7f0dd002a3f0 comp rx=0 tx=0).stop 2024-09-21T03:40:15.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:15.788+0000 7f0de7107640 1 -- 172.21.15.38:0/1966333971 shutdown_connections 2024-09-21T03:40:15.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:15.788+0000 7f0de7107640 1 --2- 172.21.15.38:0/1966333971 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0de00d0170 0x7f0de00d0be0 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:15.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:15.788+0000 7f0de7107640 1 -- 172.21.15.38:0/1966333971 >> 172.21.15.38:0/1966333971 conn(0x7f0de009d830 msgr2=0x7f0de00b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:15.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:15.788+0000 7f0de7107640 1 -- 172.21.15.38:0/1966333971 shutdown_connections 2024-09-21T03:40:15.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:15.788+0000 7f0de7107640 1 -- 172.21.15.38:0/1966333971 wait complete. 2024-09-21T03:40:15.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:15.789+0000 7f0de7107640 1 Processor -- start 2024-09-21T03:40:15.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:15.789+0000 7f0de7107640 1 -- start start 2024-09-21T03:40:15.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:15.791+0000 7f0de7107640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0de00d0170 0x7f0de014f900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:15.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:15.791+0000 7f0de7107640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0de00d1120 0x7f0de00cb610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:15.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:15.791+0000 7f0de7107640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0de015ee80 0x7f0de006f300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:15.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:15.791+0000 7f0de7107640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f0de00b6910 con 0x7f0de00d1120 2024-09-21T03:40:15.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:15.791+0000 7f0de7107640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f0de00b6790 con 0x7f0de00d0170 2024-09-21T03:40:15.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:15.791+0000 7f0de7107640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f0de00b6a90 con 0x7f0de015ee80 2024-09-21T03:40:15.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:15.791+0000 7f0dd7fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0de00d1120 0x7f0de00cb610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:15.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:15.791+0000 7f0dd7fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0de00d1120 0x7f0de00cb610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:55220/0 (socket says 172.21.15.38:55220) 2024-09-21T03:40:15.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:15.791+0000 7f0dd7fff640 1 -- 172.21.15.38:0/4211363094 learned_addr learned my addr 172.21.15.38:0/4211363094 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:15.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:15.791+0000 7f0de4ea3640 1 --2- 172.21.15.38:0/4211363094 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0de00d0170 0x7f0de014f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:15.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:15.791+0000 7f0de56a4640 1 --2- 172.21.15.38:0/4211363094 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0de015ee80 0x7f0de006f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:15.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:15.791+0000 7f0dd7fff640 1 -- 172.21.15.38:0/4211363094 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0de015ee80 msgr2=0x7f0de006f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:15.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:15.791+0000 7f0dd7fff640 1 --2- 172.21.15.38:0/4211363094 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0de015ee80 0x7f0de006f300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:15.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:15.791+0000 7f0dd7fff640 1 -- 172.21.15.38:0/4211363094 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0de00d0170 msgr2=0x7f0de014f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:15.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:15.791+0000 7f0dd7fff640 1 --2- 172.21.15.38:0/4211363094 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0de00d0170 0x7f0de014f900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:15.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:15.791+0000 7f0dd7fff640 1 -- 172.21.15.38:0/4211363094 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0de006e2a0 con 0x7f0de00d1120 2024-09-21T03:40:15.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:15.791+0000 7f0de4ea3640 1 --2- 172.21.15.38:0/4211363094 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0de00d0170 0x7f0de014f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:40:15.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:15.791+0000 7f0de56a4640 1 --2- 172.21.15.38:0/4211363094 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0de015ee80 0x7f0de006f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:40:15.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:15.792+0000 7f0dd7fff640 1 --2- 172.21.15.38:0/4211363094 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0de00d1120 0x7f0de00cb610 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f0dd0009940 tx=0x7f0dd0004b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:15.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:15.792+0000 7f0dd57fa640 1 -- 172.21.15.38:0/4211363094 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dd0031b50 con 0x7f0de00d1120 2024-09-21T03:40:15.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:15.792+0000 7f0dd57fa640 1 -- 172.21.15.38:0/4211363094 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0dd0031de0 con 0x7f0de00d1120 2024-09-21T03:40:15.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:15.792+0000 7f0de7107640 1 -- 172.21.15.38:0/4211363094 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0de006e540 con 0x7f0de00d1120 2024-09-21T03:40:15.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:15.792+0000 7f0dd57fa640 1 -- 172.21.15.38:0/4211363094 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dd0033330 con 0x7f0de00d1120 2024-09-21T03:40:15.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:15.792+0000 7f0de7107640 1 -- 172.21.15.38:0/4211363094 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0de006eb00 con 0x7f0de00d1120 2024-09-21T03:40:15.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:15.793+0000 7f0de7107640 1 -- 172.21.15.38:0/4211363094 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0de00d0be0 con 0x7f0de00d1120 2024-09-21T03:40:15.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:15.794+0000 7f0dd57fa640 1 -- 172.21.15.38:0/4211363094 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f0dd00334d0 con 0x7f0de00d1120 2024-09-21T03:40:15.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:15.794+0000 7f0dd57fa640 1 --2- 172.21.15.38:0/4211363094 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0dcc078bc0 0x7f0dcc07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:15.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:15.794+0000 7f0dd57fa640 1 -- 172.21.15.38:0/4211363094 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6187+0+0 (secure 0 0 0) 0x7f0dd00bc310 con 0x7f0de00d1120 2024-09-21T03:40:15.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:15.795+0000 7f0de4ea3640 1 --2- 172.21.15.38:0/4211363094 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0dcc078bc0 0x7f0dcc07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:15.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:15.796+0000 7f0de4ea3640 1 --2- 172.21.15.38:0/4211363094 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0dcc078bc0 0x7f0dcc07b0d0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f0ddc00a830 tx=0x7f0ddc008040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:15.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:15.799+0000 7f0dd57fa640 1 -- 172.21.15.38:0/4211363094 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f0dd003f030 con 0x7f0de00d1120 2024-09-21T03:40:16.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.061+0000 7f0de7107640 1 -- 172.21.15.38:0/4211363094 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7f0de00ceda0 con 0x7f0de00d1120 2024-09-21T03:40:16.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.659+0000 7f0dd57fa640 1 -- 172.21.15.38:0/4211363094 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 nothing to do: rank is not damaged v87) ==== 107+0+0 (secure 0 0 0) 0x7f0dd0084600 con 0x7f0de00d1120 2024-09-21T03:40:16.661 INFO:teuthology.orchestra.run.smithi038.stderr:nothing to do: rank is not damaged 2024-09-21T03:40:16.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.662+0000 7f0de7107640 1 -- 172.21.15.38:0/4211363094 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0dcc078bc0 msgr2=0x7f0dcc07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:16.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.662+0000 7f0de7107640 1 --2- 172.21.15.38:0/4211363094 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0dcc078bc0 0x7f0dcc07b0d0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f0ddc00a830 tx=0x7f0ddc008040 comp rx=0 tx=0).stop 2024-09-21T03:40:16.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.663+0000 7f0de7107640 1 -- 172.21.15.38:0/4211363094 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0de00d1120 msgr2=0x7f0de00cb610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:16.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.663+0000 7f0de7107640 1 --2- 172.21.15.38:0/4211363094 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0de00d1120 0x7f0de00cb610 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f0dd0009940 tx=0x7f0dd0004b90 comp rx=0 tx=0).stop 2024-09-21T03:40:16.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.663+0000 7f0de7107640 1 -- 172.21.15.38:0/4211363094 shutdown_connections 2024-09-21T03:40:16.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.663+0000 7f0de7107640 1 --2- 172.21.15.38:0/4211363094 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0dcc078bc0 0x7f0dcc07b0d0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:16.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.663+0000 7f0de7107640 1 --2- 172.21.15.38:0/4211363094 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0de015ee80 0x7f0de006f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:16.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.663+0000 7f0de7107640 1 --2- 172.21.15.38:0/4211363094 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0de00d1120 0x7f0de00cb610 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:16.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.663+0000 7f0de7107640 1 --2- 172.21.15.38:0/4211363094 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0de00d0170 0x7f0de014f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:16.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.663+0000 7f0de7107640 1 -- 172.21.15.38:0/4211363094 >> 172.21.15.38:0/4211363094 conn(0x7f0de009d830 msgr2=0x7f0de01616a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:16.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.663+0000 7f0de7107640 1 -- 172.21.15.38:0/4211363094 shutdown_connections 2024-09-21T03:40:16.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.663+0000 7f0de7107640 1 -- 172.21.15.38:0/4211363094 wait complete. 2024-09-21T03:40:16.682 DEBUG:teuthology.orchestra.run.smithi038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-09-21T03:40:16.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.738+0000 7fcde6e7ff40 1 Processor -- start 2024-09-21T03:40:16.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.738+0000 7fcde6e7ff40 1 -- start start 2024-09-21T03:40:16.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.740+0000 7fcde6e7ff40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55840292b240 0x5584029ffba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:16.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.740+0000 7fcde6e7ff40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558402972090 0x55840296ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:16.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.740+0000 7fcde6e7ff40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5584029e7520 0x5584029570c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:16.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.740+0000 7fcde6e7ff40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x558402878d60 con 0x55840292b240 2024-09-21T03:40:16.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.740+0000 7fcde6e7ff40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5584028d7430 con 0x558402972090 2024-09-21T03:40:16.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.740+0000 7fcde6e7ff40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x5584029fe810 con 0x5584029e7520 2024-09-21T03:40:16.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.740+0000 7fcde0939640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55840292b240 0x5584029ffba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:16.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.740+0000 7fcde0939640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55840292b240 0x5584029ffba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:55232/0 (socket says 172.21.15.38:55232) 2024-09-21T03:40:16.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.740+0000 7fcde0939640 1 -- 172.21.15.38:0/3203869699 learned_addr learned my addr 172.21.15.38:0/3203869699 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:16.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.740+0000 7fcde0138640 1 --2- 172.21.15.38:0/3203869699 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558402972090 0x55840296ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:16.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.740+0000 7fcde113a640 1 --2- 172.21.15.38:0/3203869699 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5584029e7520 0x5584029570c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:16.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.740+0000 7fcde0939640 1 -- 172.21.15.38:0/3203869699 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5584029e7520 msgr2=0x5584029570c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:16.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.740+0000 7fcde0939640 1 --2- 172.21.15.38:0/3203869699 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5584029e7520 0x5584029570c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:16.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.740+0000 7fcde0939640 1 -- 172.21.15.38:0/3203869699 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558402972090 msgr2=0x55840296ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:16.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.740+0000 7fcde0939640 1 --2- 172.21.15.38:0/3203869699 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558402972090 0x55840296ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:16.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.740+0000 7fcde0939640 1 -- 172.21.15.38:0/3203869699 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558402a000e0 con 0x55840292b240 2024-09-21T03:40:16.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.740+0000 7fcde0939640 1 --2- 172.21.15.38:0/3203869699 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55840292b240 0x5584029ffba0 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7fcdd0046600 tx=0x7fcdd0002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a878aa4c72248a0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:16.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.740+0000 7fcddf136640 1 -- 172.21.15.38:0/3203869699 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdd004d960 con 0x55840292b240 2024-09-21T03:40:16.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.741+0000 7fcddf136640 1 -- 172.21.15.38:0/3203869699 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcdd0056690 con 0x55840292b240 2024-09-21T03:40:16.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.741+0000 7fcddf136640 1 -- 172.21.15.38:0/3203869699 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdd0056ae0 con 0x55840292b240 2024-09-21T03:40:16.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.741+0000 7fcde6e7ff40 1 -- 172.21.15.38:0/3203869699 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55840292b240 msgr2=0x5584029ffba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:16.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.741+0000 7fcde6e7ff40 1 --2- 172.21.15.38:0/3203869699 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55840292b240 0x5584029ffba0 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7fcdd0046600 tx=0x7fcdd0002fb0 comp rx=0 tx=0).stop 2024-09-21T03:40:16.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.741+0000 7fcde6e7ff40 1 -- 172.21.15.38:0/3203869699 shutdown_connections 2024-09-21T03:40:16.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.741+0000 7fcde6e7ff40 1 --2- 172.21.15.38:0/3203869699 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5584029e7520 0x5584029570c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:16.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.741+0000 7fcde6e7ff40 1 --2- 172.21.15.38:0/3203869699 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558402972090 0x55840296ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:16.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.741+0000 7fcde6e7ff40 1 --2- 172.21.15.38:0/3203869699 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55840292b240 0x5584029ffba0 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:16.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.741+0000 7fcde6e7ff40 1 -- 172.21.15.38:0/3203869699 >> 172.21.15.38:0/3203869699 conn(0x558402849ac0 msgr2=0x558402929cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:16.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.741+0000 7fcde6e7ff40 1 -- 172.21.15.38:0/3203869699 shutdown_connections 2024-09-21T03:40:16.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.741+0000 7fcde6e7ff40 1 -- 172.21.15.38:0/3203869699 wait complete. 2024-09-21T03:40:16.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.742+0000 7fcde6e7ff40 1 Processor -- start 2024-09-21T03:40:16.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.742+0000 7fcde6e7ff40 1 -- start start 2024-09-21T03:40:16.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.743+0000 7fcde6e7ff40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55840292b240 0x5584029d24a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:16.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.743+0000 7fcde6e7ff40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558402972090 0x558402a97020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:16.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.743+0000 7fcde6e7ff40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5584029e7520 0x5584029cf9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:16.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.743+0000 7fcde6e7ff40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55840296f200 con 0x5584029e7520 2024-09-21T03:40:16.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.743+0000 7fcde6e7ff40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5584029e5680 con 0x558402972090 2024-09-21T03:40:16.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.743+0000 7fcde6e7ff40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x558402954c20 con 0x55840292b240 2024-09-21T03:40:16.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.744+0000 7fcde113a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5584029e7520 0x5584029cf9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:16.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.744+0000 7fcde113a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5584029e7520 0x5584029cf9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:55236/0 (socket says 172.21.15.38:55236) 2024-09-21T03:40:16.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.744+0000 7fcde113a640 1 -- 172.21.15.38:0/3447408364 learned_addr learned my addr 172.21.15.38:0/3447408364 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:16.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.744+0000 7fcde0138640 1 --2- 172.21.15.38:0/3447408364 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558402972090 0x558402a97020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:16.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.744+0000 7fcde0939640 1 --2- 172.21.15.38:0/3447408364 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55840292b240 0x5584029d24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:16.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.744+0000 7fcde113a640 1 -- 172.21.15.38:0/3447408364 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55840292b240 msgr2=0x5584029d24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:16.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.744+0000 7fcde113a640 1 --2- 172.21.15.38:0/3447408364 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55840292b240 0x5584029d24a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:16.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.744+0000 7fcde113a640 1 -- 172.21.15.38:0/3447408364 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558402972090 msgr2=0x558402a97020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:16.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.744+0000 7fcde113a640 1 --2- 172.21.15.38:0/3447408364 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558402972090 0x558402a97020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:16.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.744+0000 7fcde113a640 1 -- 172.21.15.38:0/3447408364 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55840296ede0 con 0x5584029e7520 2024-09-21T03:40:16.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.744+0000 7fcde113a640 1 --2- 172.21.15.38:0/3447408364 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5584029e7520 0x5584029cf9c0 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7fcdd8002fa0 tx=0x7fcdd800d5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:16.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.744+0000 7fcdde134640 1 -- 172.21.15.38:0/3447408364 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdd8013480 con 0x5584029e7520 2024-09-21T03:40:16.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.744+0000 7fcdde134640 1 -- 172.21.15.38:0/3447408364 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcdd8018070 con 0x5584029e7520 2024-09-21T03:40:16.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.744+0000 7fcde6e7ff40 1 -- 172.21.15.38:0/3447408364 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5584029fb700 con 0x5584029e7520 2024-09-21T03:40:16.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.744+0000 7fcdde134640 1 -- 172.21.15.38:0/3447408364 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdd80139d0 con 0x5584029e7520 2024-09-21T03:40:16.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.744+0000 7fcde6e7ff40 1 -- 172.21.15.38:0/3447408364 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558402a97560 con 0x5584029e7520 2024-09-21T03:40:16.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.746+0000 7fcdde134640 1 -- 172.21.15.38:0/3447408364 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fcdd8013b70 con 0x5584029e7520 2024-09-21T03:40:16.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.747+0000 7fcdde134640 1 --2- 172.21.15.38:0/3447408364 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fcdcc0775a0 0x7fcdcc079ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:16.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.747+0000 7fcdde134640 1 -- 172.21.15.38:0/3447408364 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 6390+0+0 (secure 0 0 0) 0x7fcdd80999b0 con 0x5584029e7520 2024-09-21T03:40:16.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.747+0000 7fcde0939640 1 --2- 172.21.15.38:0/3447408364 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fcdcc0775a0 0x7fcdcc079ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:16.751 INFO:teuthology.orchestra.run.smithi038.stdout:Importing pool 2024-09-21T03:40:16.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.747+0000 7fcde6e7ff40 1 --2- 172.21.15.38:0/3447408364 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x558402aa1cf0 0x558402aa4110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:16.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.747+0000 7fcde6e7ff40 1 -- 172.21.15.38:0/3447408364 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x558402aa4650 con 0x558402aa1cf0 2024-09-21T03:40:16.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.747+0000 7fcde0138640 1 --2- 172.21.15.38:0/3447408364 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x558402aa1cf0 0x558402aa4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:16.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.748+0000 7fcde0939640 1 --2- 172.21.15.38:0/3447408364 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fcdcc0775a0 0x7fcdcc079ab0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fcdd0046600 tx=0x7fcdd0064040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:16.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.748+0000 7fcde0138640 1 --2- 172.21.15.38:0/3447408364 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x558402aa1cf0 0x558402aa4110 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:16.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.749+0000 7fcde0138640 1 -- 172.21.15.38:0/3447408364 <== osd.7 v2:172.21.15.195:6818/1606106342 1 ==== osd_op_reply(1 601.00000000 [create] v96'11 uv4 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fcdd400c9b0 con 0x558402aa1cf0 2024-09-21T03:40:16.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.749+0000 7fcde6e7ff40 1 -- 172.21.15.38:0/3447408364 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e96) -- 0x558402aa49f0 con 0x558402aa1cf0 2024-09-21T03:40:16.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.750+0000 7fcde0138640 1 -- 172.21.15.38:0/3447408364 <== osd.7 v2:172.21.15.195:6818/1606106342 2 ==== osd_op_reply(2 601.00000000 [delete] v96'12 uv12 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcdd400c9b0 con 0x558402aa1cf0 2024-09-21T03:40:16.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.750+0000 7fcde6e7ff40 1 -- 172.21.15.38:0/3447408364 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5584029cff00 con 0x558402aa1cf0 2024-09-21T03:40:16.753 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-09-21T03:40:16.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.751+0000 7fcde0138640 1 -- 172.21.15.38:0/3447408364 <== osd.7 v2:172.21.15.195:6818/1606106342 3 ==== osd_op_reply(3 601.00000000 [create] v96'13 uv13 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcdd400c9b0 con 0x558402aa1cf0 2024-09-21T03:40:16.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.752+0000 7fcde6e7ff40 1 -- 172.21.15.38:0/3447408364 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x558402aa5da0 con 0x558402aa1cf0 2024-09-21T03:40:16.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.753+0000 7fcde0138640 1 -- 172.21.15.38:0/3447408364 <== osd.7 v2:172.21.15.195:6818/1606106342 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v96'14 uv14 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcdd400c9b0 con 0x558402aa1cf0 2024-09-21T03:40:16.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.753+0000 7fcde6e7ff40 1 -- 172.21.15.38:0/3447408364 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x558402aa6140 con 0x558402aa1cf0 2024-09-21T03:40:16.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.754+0000 7fcde0138640 1 -- 172.21.15.38:0/3447408364 <== osd.7 v2:172.21.15.195:6818/1606106342 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v96'15 uv15 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcdd400c9b0 con 0x558402aa1cf0 2024-09-21T03:40:16.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.754+0000 7fcde6e7ff40 1 -- 172.21.15.38:0/3447408364 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x558402aa4d90 con 0x558402aa1cf0 2024-09-21T03:40:16.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.755+0000 7fcde0138640 1 -- 172.21.15.38:0/3447408364 <== osd.7 v2:172.21.15.195:6818/1606106342 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v96'16 uv16 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcdd400c9b0 con 0x558402aa1cf0 2024-09-21T03:40:16.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.755+0000 7fcde6e7ff40 1 -- 172.21.15.38:0/3447408364 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x558402aa51a0 con 0x558402aa1cf0 2024-09-21T03:40:16.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.756+0000 7fcde0138640 1 -- 172.21.15.38:0/3447408364 <== osd.7 v2:172.21.15.195:6818/1606106342 7 ==== osd_op_reply(7 602.00000000 [create] v96'17 uv6 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fcdd400c9b0 con 0x558402aa1cf0 2024-09-21T03:40:16.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.756+0000 7fcde6e7ff40 1 -- 172.21.15.38:0/3447408364 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e96) -- 0x558402aa51a0 con 0x558402aa1cf0 2024-09-21T03:40:16.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.758+0000 7fcde0138640 1 -- 172.21.15.38:0/3447408364 <== osd.7 v2:172.21.15.195:6818/1606106342 8 ==== osd_op_reply(8 602.00000000 [delete] v96'18 uv18 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcdd400c9b0 con 0x558402aa1cf0 2024-09-21T03:40:16.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.758+0000 7fcde6e7ff40 1 -- 172.21.15.38:0/3447408364 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x558402aa51a0 con 0x558402aa1cf0 2024-09-21T03:40:16.761 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-09-21T03:40:16.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.759+0000 7fcde0138640 1 -- 172.21.15.38:0/3447408364 <== osd.7 v2:172.21.15.195:6818/1606106342 9 ==== osd_op_reply(9 602.00000000 [create] v96'19 uv19 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcdd400c9b0 con 0x558402aa1cf0 2024-09-21T03:40:16.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.759+0000 7fcde6e7ff40 1 -- 172.21.15.38:0/3447408364 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x558402aa51a0 con 0x558402aa1cf0 2024-09-21T03:40:16.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.760+0000 7fcde0138640 1 -- 172.21.15.38:0/3447408364 <== osd.7 v2:172.21.15.195:6818/1606106342 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v96'20 uv20 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcdd400c9b0 con 0x558402aa1cf0 2024-09-21T03:40:16.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.760+0000 7fcde6e7ff40 1 -- 172.21.15.38:0/3447408364 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x558402aa51a0 con 0x558402aa1cf0 2024-09-21T03:40:16.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.762+0000 7fcde0138640 1 -- 172.21.15.38:0/3447408364 <== osd.7 v2:172.21.15.195:6818/1606106342 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v96'21 uv21 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcdd400c9b0 con 0x558402aa1cf0 2024-09-21T03:40:16.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.762+0000 7fcde6e7ff40 1 -- 172.21.15.38:0/3447408364 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x558402aa51a0 con 0x558402aa1cf0 2024-09-21T03:40:16.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.763+0000 7fcde0138640 1 -- 172.21.15.38:0/3447408364 <== osd.7 v2:172.21.15.195:6818/1606106342 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v96'22 uv22 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcdd400c9b0 con 0x558402aa1cf0 2024-09-21T03:40:16.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.763+0000 7fcde6e7ff40 1 -- 172.21.15.38:0/3447408364 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:13 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x558402aa51a0 con 0x558402aa1cf0 2024-09-21T03:40:16.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.764+0000 7fcde0138640 1 -- 172.21.15.38:0/3447408364 <== osd.7 v2:172.21.15.195:6818/1606106342 13 ==== osd_op_reply(13 10000000000.00000000 [create] v96'23 uv10 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7fcdd400c9b0 con 0x558402aa1cf0 2024-09-21T03:40:16.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.764+0000 7fcde6e7ff40 1 -- 172.21.15.38:0/3447408364 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:14 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e96) -- 0x558402aa51a0 con 0x558402aa1cf0 2024-09-21T03:40:16.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.766+0000 7fcde0138640 1 -- 172.21.15.38:0/3447408364 <== osd.7 v2:172.21.15.195:6818/1606106342 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v96'24 uv24 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fcdd400c9b0 con 0x558402aa1cf0 2024-09-21T03:40:16.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.766+0000 7fcde6e7ff40 1 -- 172.21.15.38:0/3447408364 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:15 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x558402aa51a0 con 0x558402aa1cf0 2024-09-21T03:40:16.769 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-09-21T03:40:16.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.767+0000 7fcde0138640 1 -- 172.21.15.38:0/3447408364 <== osd.7 v2:172.21.15.195:6818/1606106342 15 ==== osd_op_reply(15 10000000000.00000000 [create] v96'25 uv25 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fcdd400c9b0 con 0x558402aa1cf0 2024-09-21T03:40:16.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.767+0000 7fcde6e7ff40 1 -- 172.21.15.38:0/3447408364 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:16 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x558402aa51a0 con 0x558402aa1cf0 2024-09-21T03:40:16.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.768+0000 7fcde0138640 1 -- 172.21.15.38:0/3447408364 <== osd.7 v2:172.21.15.195:6818/1606106342 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v96'26 uv26 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fcdd400c9b0 con 0x558402aa1cf0 2024-09-21T03:40:16.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.768+0000 7fcde6e7ff40 1 -- 172.21.15.38:0/3447408364 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:17 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x558402aa51a0 con 0x558402aa1cf0 2024-09-21T03:40:16.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.770+0000 7fcde0138640 1 -- 172.21.15.38:0/3447408364 <== osd.7 v2:172.21.15.195:6818/1606106342 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v96'27 uv27 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fcdd400c9b0 con 0x558402aa1cf0 2024-09-21T03:40:16.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.770+0000 7fcde6e7ff40 1 -- 172.21.15.38:0/3447408364 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:18 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x558402aa51a0 con 0x558402aa1cf0 2024-09-21T03:40:16.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.771+0000 7fcde0138640 1 -- 172.21.15.38:0/3447408364 <== osd.7 v2:172.21.15.195:6818/1606106342 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v96'28 uv28 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fcdd400c9b0 con 0x558402aa1cf0 2024-09-21T03:40:16.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.771+0000 7fcde6e7ff40 1 -- 172.21.15.38:0/3447408364 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:19 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=585b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x558402aa51a0 con 0x558402aa1cf0 2024-09-21T03:40:16.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.772+0000 7fcde0138640 1 -- 172.21.15.38:0/3447408364 <== osd.7 v2:172.21.15.195:6818/1606106342 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v96'29 uv29 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fcdd400c9b0 con 0x558402aa1cf0 2024-09-21T03:40:16.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.772+0000 7fcde6e7ff40 1 -- 172.21.15.38:0/3447408364 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:20 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x558402aa51a0 con 0x558402aa1cf0 2024-09-21T03:40:16.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.774+0000 7fcde0138640 1 -- 172.21.15.38:0/3447408364 <== osd.7 v2:172.21.15.195:6818/1606106342 20 ==== osd_op_reply(20 600.00000000 [create] v96'30 uv2 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fcdd400c9b0 con 0x558402aa1cf0 2024-09-21T03:40:16.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.774+0000 7fcde6e7ff40 1 -- 172.21.15.38:0/3447408364 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:21 13.10 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e96) -- 0x558402aa51a0 con 0x558402aa1cf0 2024-09-21T03:40:16.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.775+0000 7fcde0138640 1 -- 172.21.15.38:0/3447408364 <== osd.7 v2:172.21.15.195:6818/1606106342 21 ==== osd_op_reply(21 600.00000000 [delete] v96'31 uv31 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcdd400c9b0 con 0x558402aa1cf0 2024-09-21T03:40:16.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.775+0000 7fcde6e7ff40 1 -- 172.21.15.38:0/3447408364 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:22 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x558402aa51a0 con 0x558402aa1cf0 2024-09-21T03:40:16.778 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-09-21T03:40:16.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.776+0000 7fcde0138640 1 -- 172.21.15.38:0/3447408364 <== osd.7 v2:172.21.15.195:6818/1606106342 22 ==== osd_op_reply(22 600.00000000 [create] v96'32 uv32 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcdd400c9b0 con 0x558402aa1cf0 2024-09-21T03:40:16.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.776+0000 7fcde6e7ff40 1 -- 172.21.15.38:0/3447408364 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:23 13.10 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x558402aa51a0 con 0x558402aa1cf0 2024-09-21T03:40:16.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.777+0000 7fcde0138640 1 -- 172.21.15.38:0/3447408364 <== osd.7 v2:172.21.15.195:6818/1606106342 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v96'33 uv33 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcdd400c9b0 con 0x558402aa1cf0 2024-09-21T03:40:16.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.777+0000 7fcde6e7ff40 1 -- 172.21.15.38:0/3447408364 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:24 13.10 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x558402aa51a0 con 0x558402aa1cf0 2024-09-21T03:40:16.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.778+0000 7fcde0138640 1 -- 172.21.15.38:0/3447408364 <== osd.7 v2:172.21.15.195:6818/1606106342 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v96'34 uv34 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcdd400c9b0 con 0x558402aa1cf0 2024-09-21T03:40:16.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.778+0000 7fcde6e7ff40 1 -- 172.21.15.38:0/3447408364 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:25 13.10 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x558402aa51a0 con 0x558402aa1cf0 2024-09-21T03:40:16.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.780+0000 7fcde0138640 1 -- 172.21.15.38:0/3447408364 <== osd.7 v2:172.21.15.195:6818/1606106342 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v96'35 uv35 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcdd400c9b0 con 0x558402aa1cf0 2024-09-21T03:40:16.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.780+0000 7fcde6e7ff40 1 -- 172.21.15.38:0/3447408364 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:26 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x558402aa51a0 con 0x558402aa1cf0 2024-09-21T03:40:16.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.781+0000 7fcde0138640 1 -- 172.21.15.38:0/3447408364 <== osd.7 v2:172.21.15.195:6818/1606106342 26 ==== osd_op_reply(26 603.00000000 [create] v96'36 uv8 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fcdd400c9b0 con 0x558402aa1cf0 2024-09-21T03:40:16.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.781+0000 7fcde6e7ff40 1 -- 172.21.15.38:0/3447408364 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:27 13.10 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e96) -- 0x558402aa51a0 con 0x558402aa1cf0 2024-09-21T03:40:16.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.782+0000 7fcde0138640 1 -- 172.21.15.38:0/3447408364 <== osd.7 v2:172.21.15.195:6818/1606106342 27 ==== osd_op_reply(27 603.00000000 [delete] v96'37 uv37 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcdd400c9b0 con 0x558402aa1cf0 2024-09-21T03:40:16.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.782+0000 7fcde6e7ff40 1 -- 172.21.15.38:0/3447408364 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:28 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x558402aa51a0 con 0x558402aa1cf0 2024-09-21T03:40:16.785 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-09-21T03:40:16.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.784+0000 7fcde0138640 1 -- 172.21.15.38:0/3447408364 <== osd.7 v2:172.21.15.195:6818/1606106342 28 ==== osd_op_reply(28 603.00000000 [create] v96'38 uv38 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcdd400c9b0 con 0x558402aa1cf0 2024-09-21T03:40:16.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.784+0000 7fcde6e7ff40 1 -- 172.21.15.38:0/3447408364 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:29 13.10 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x558402aa51a0 con 0x558402aa1cf0 2024-09-21T03:40:16.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.785+0000 7fcde0138640 1 -- 172.21.15.38:0/3447408364 <== osd.7 v2:172.21.15.195:6818/1606106342 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v96'39 uv39 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcdd400c9b0 con 0x558402aa1cf0 2024-09-21T03:40:16.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.785+0000 7fcde6e7ff40 1 -- 172.21.15.38:0/3447408364 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:30 13.10 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x558402aa51a0 con 0x558402aa1cf0 2024-09-21T03:40:16.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.786+0000 7fcde0138640 1 -- 172.21.15.38:0/3447408364 <== osd.7 v2:172.21.15.195:6818/1606106342 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v96'40 uv40 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcdd400c9b0 con 0x558402aa1cf0 2024-09-21T03:40:16.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.786+0000 7fcde6e7ff40 1 -- 172.21.15.38:0/3447408364 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:31 13.10 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x558402aa51a0 con 0x558402aa1cf0 2024-09-21T03:40:16.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.787+0000 7fcde0138640 1 -- 172.21.15.38:0/3447408364 <== osd.7 v2:172.21.15.195:6818/1606106342 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v96'41 uv41 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcdd400c9b0 con 0x558402aa1cf0 2024-09-21T03:40:16.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.787+0000 7fcde6e7ff40 1 --2- 172.21.15.38:0/3447408364 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x558402aa64e0 0x558402aa8900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:16.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.787+0000 7fcde6e7ff40 1 -- 172.21.15.38:0/3447408364 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x558402aa51a0 con 0x558402aa64e0 2024-09-21T03:40:16.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.787+0000 7fcde0939640 1 --2- 172.21.15.38:0/3447408364 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x558402aa64e0 0x558402aa8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:16.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.788+0000 7fcde0939640 1 --2- 172.21.15.38:0/3447408364 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x558402aa64e0 0x558402aa8900 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:16.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.789+0000 7fcde0939640 1 -- 172.21.15.38:0/3447408364 <== osd.6 v2:172.21.15.195:6826/4055825675 1 ==== osd_op_reply(32 1.00000000.inode [create] v96'19 uv15 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7fcdd006a040 con 0x558402aa64e0 2024-09-21T03:40:16.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.789+0000 7fcde6e7ff40 1 -- 172.21.15.38:0/3447408364 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e96) -- 0x558402aa8e40 con 0x558402aa64e0 2024-09-21T03:40:16.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.791+0000 7fcde0939640 1 -- 172.21.15.38:0/3447408364 <== osd.6 v2:172.21.15.195:6826/4055825675 2 ==== osd_op_reply(33 1.00000000.inode [delete] v96'20 uv20 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fcdd006a040 con 0x558402aa64e0 2024-09-21T03:40:16.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.791+0000 7fcde6e7ff40 1 -- 172.21.15.38:0/3447408364 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x558402aa91e0 con 0x558402aa64e0 2024-09-21T03:40:16.794 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-09-21T03:40:16.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.792+0000 7fcde0939640 1 -- 172.21.15.38:0/3447408364 <== osd.6 v2:172.21.15.195:6826/4055825675 3 ==== osd_op_reply(34 1.00000000.inode [create] v96'21 uv21 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fcdd006a040 con 0x558402aa64e0 2024-09-21T03:40:16.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.792+0000 7fcde6e7ff40 1 -- 172.21.15.38:0/3447408364 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x558402aa9580 con 0x558402aa64e0 2024-09-21T03:40:16.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.794+0000 7fcde0939640 1 -- 172.21.15.38:0/3447408364 <== osd.6 v2:172.21.15.195:6826/4055825675 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v96'22 uv22 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fcdd006a040 con 0x558402aa64e0 2024-09-21T03:40:16.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.794+0000 7fcde6e7ff40 1 -- 172.21.15.38:0/3447408364 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x558402aa9920 con 0x558402aa64e0 2024-09-21T03:40:16.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.795+0000 7fcde0939640 1 -- 172.21.15.38:0/3447408364 <== osd.6 v2:172.21.15.195:6826/4055825675 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v96'23 uv23 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fcdd006a040 con 0x558402aa64e0 2024-09-21T03:40:16.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.795+0000 7fcde6e7ff40 1 -- 172.21.15.38:0/3447408364 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x558402aa9cc0 con 0x558402aa64e0 2024-09-21T03:40:16.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.796+0000 7fcde0939640 1 -- 172.21.15.38:0/3447408364 <== osd.6 v2:172.21.15.195:6826/4055825675 6 ==== osd_op_reply(37 200.00000000 [create] v96'24 uv18 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fcdd006a040 con 0x558402aa64e0 2024-09-21T03:40:16.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.796+0000 7fcde6e7ff40 1 -- 172.21.15.38:0/3447408364 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e96) -- 0x558402aa9cc0 con 0x558402aa64e0 2024-09-21T03:40:16.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.798+0000 7fcde0939640 1 -- 172.21.15.38:0/3447408364 <== osd.6 v2:172.21.15.195:6826/4055825675 7 ==== osd_op_reply(38 200.00000000 [delete] v96'25 uv25 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcdd006a040 con 0x558402aa64e0 2024-09-21T03:40:16.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.798+0000 7fcde6e7ff40 1 -- 172.21.15.38:0/3447408364 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x558402aa9cc0 con 0x558402aa64e0 2024-09-21T03:40:16.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.799+0000 7fcde0939640 1 -- 172.21.15.38:0/3447408364 <== osd.6 v2:172.21.15.195:6826/4055825675 8 ==== osd_op_reply(39 200.00000000 [create] v96'26 uv26 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcdd006a040 con 0x558402aa64e0 2024-09-21T03:40:16.801 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-09-21T03:40:16.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.799+0000 7fcde6e7ff40 1 -- 172.21.15.38:0/3447408364 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x558402aa9cc0 con 0x558402aa64e0 2024-09-21T03:40:16.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.800+0000 7fcde0939640 1 -- 172.21.15.38:0/3447408364 <== osd.6 v2:172.21.15.195:6826/4055825675 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v96'27 uv27 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcdd006a040 con 0x558402aa64e0 2024-09-21T03:40:16.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.800+0000 7fcde6e7ff40 1 -- 172.21.15.38:0/3447408364 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x558402aa9cc0 con 0x558402aa64e0 2024-09-21T03:40:16.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.802+0000 7fcde0939640 1 -- 172.21.15.38:0/3447408364 <== osd.6 v2:172.21.15.195:6826/4055825675 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v96'28 uv28 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcdd006a040 con 0x558402aa64e0 2024-09-21T03:40:16.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.802+0000 7fcde6e7ff40 1 -- 172.21.15.38:0/3447408364 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:42 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x558402aa9cc0 con 0x558402aa64e0 2024-09-21T03:40:16.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.803+0000 7fcde0939640 1 -- 172.21.15.38:0/3447408364 <== osd.6 v2:172.21.15.195:6826/4055825675 11 ==== osd_op_reply(42 200.00000001 [create] v96'29 uv16 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fcdd006a040 con 0x558402aa64e0 2024-09-21T03:40:16.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.803+0000 7fcde6e7ff40 1 -- 172.21.15.38:0/3447408364 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:43 13.14 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e96) -- 0x558402aa9cc0 con 0x558402aa64e0 2024-09-21T03:40:16.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.804+0000 7fcde0939640 1 -- 172.21.15.38:0/3447408364 <== osd.6 v2:172.21.15.195:6826/4055825675 12 ==== osd_op_reply(43 200.00000001 [delete] v96'30 uv30 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcdd006a040 con 0x558402aa64e0 2024-09-21T03:40:16.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.804+0000 7fcde6e7ff40 1 -- 172.21.15.38:0/3447408364 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:44 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x558402aa9cc0 con 0x558402aa64e0 2024-09-21T03:40:16.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.805+0000 7fcde0939640 1 -- 172.21.15.38:0/3447408364 <== osd.6 v2:172.21.15.195:6826/4055825675 13 ==== osd_op_reply(44 200.00000001 [create] v96'31 uv31 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcdd006a040 con 0x558402aa64e0 2024-09-21T03:40:16.807 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-09-21T03:40:16.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.805+0000 7fcde6e7ff40 1 -- 172.21.15.38:0/3447408364 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:45 13.14 13:2e2fa760:::200.00000001:head [write 0~17490 in=17490b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x558402aae140 con 0x558402aa64e0 2024-09-21T03:40:16.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.807+0000 7fcde0939640 1 -- 172.21.15.38:0/3447408364 <== osd.6 v2:172.21.15.195:6826/4055825675 14 ==== osd_op_reply(45 200.00000001 [write 0~17490] v96'32 uv32 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcdd006a040 con 0x558402aa64e0 2024-09-21T03:40:16.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.807+0000 7fcde6e7ff40 1 -- 172.21.15.38:0/3447408364 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:46 13.14 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x558402aae140 con 0x558402aa64e0 2024-09-21T03:40:16.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.808+0000 7fcde0939640 1 -- 172.21.15.38:0/3447408364 <== osd.6 v2:172.21.15.195:6826/4055825675 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v96'33 uv33 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcdd006a040 con 0x558402aa64e0 2024-09-21T03:40:16.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.808+0000 7fcde6e7ff40 1 -- 172.21.15.38:0/3447408364 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:47 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x558402aa9cc0 con 0x558402aa64e0 2024-09-21T03:40:16.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.809+0000 7fcde0939640 1 -- 172.21.15.38:0/3447408364 <== osd.6 v2:172.21.15.195:6826/4055825675 16 ==== osd_op_reply(47 606.00000000 [create] v96'34 uv2 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fcdd006a040 con 0x558402aa64e0 2024-09-21T03:40:16.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.809+0000 7fcde6e7ff40 1 -- 172.21.15.38:0/3447408364 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:48 13.14 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e96) -- 0x558402aa9cc0 con 0x558402aa64e0 2024-09-21T03:40:16.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.810+0000 7fcde0939640 1 -- 172.21.15.38:0/3447408364 <== osd.6 v2:172.21.15.195:6826/4055825675 17 ==== osd_op_reply(48 606.00000000 [delete] v96'35 uv35 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcdd006a040 con 0x558402aa64e0 2024-09-21T03:40:16.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.811+0000 7fcde6e7ff40 1 -- 172.21.15.38:0/3447408364 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:49 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x558402aa9cc0 con 0x558402aa64e0 2024-09-21T03:40:16.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.812+0000 7fcde0939640 1 -- 172.21.15.38:0/3447408364 <== osd.6 v2:172.21.15.195:6826/4055825675 18 ==== osd_op_reply(49 606.00000000 [create] v96'36 uv36 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcdd006a040 con 0x558402aa64e0 2024-09-21T03:40:16.813 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-09-21T03:40:16.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.812+0000 7fcde6e7ff40 1 -- 172.21.15.38:0/3447408364 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:50 13.14 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x558402aaacd0 con 0x558402aa64e0 2024-09-21T03:40:16.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.813+0000 7fcde0939640 1 -- 172.21.15.38:0/3447408364 <== osd.6 v2:172.21.15.195:6826/4055825675 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v96'37 uv37 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcdd006a040 con 0x558402aa64e0 2024-09-21T03:40:16.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.813+0000 7fcde6e7ff40 1 -- 172.21.15.38:0/3447408364 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:51 13.14 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x558402aaacd0 con 0x558402aa64e0 2024-09-21T03:40:16.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.814+0000 7fcde0939640 1 -- 172.21.15.38:0/3447408364 <== osd.6 v2:172.21.15.195:6826/4055825675 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v96'38 uv38 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcdd006a040 con 0x558402aa64e0 2024-09-21T03:40:16.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.814+0000 7fcde6e7ff40 1 -- 172.21.15.38:0/3447408364 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:52 13.14 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x558402aa9cc0 con 0x558402aa64e0 2024-09-21T03:40:16.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.815+0000 7fcde0939640 1 -- 172.21.15.38:0/3447408364 <== osd.6 v2:172.21.15.195:6826/4055825675 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v96'39 uv39 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcdd006a040 con 0x558402aa64e0 2024-09-21T03:40:16.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.816+0000 7fcde6e7ff40 1 --2- 172.21.15.38:0/3447408364 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x558402aaa1d0 0x558402aac5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:16.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.816+0000 7fcde6e7ff40 1 -- 172.21.15.38:0/3447408364 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:53 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x558402aacb30 con 0x558402aaa1d0 2024-09-21T03:40:16.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.816+0000 7fcde0138640 1 --2- 172.21.15.38:0/3447408364 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x558402aaa1d0 0x558402aac5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:16.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.816+0000 7fcde0138640 1 --2- 172.21.15.38:0/3447408364 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x558402aaa1d0 0x558402aac5f0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:16.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.817+0000 7fcde0138640 1 -- 172.21.15.38:0/3447408364 <== osd.0 v2:172.21.15.38:6800/2381639433 1 ==== osd_op_reply(53 607.00000000 [create] v96'3 uv2 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fcdd400c9b0 con 0x558402aaa1d0 2024-09-21T03:40:16.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.817+0000 7fcde6e7ff40 1 -- 172.21.15.38:0/3447408364 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:54 13.1c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e96) -- 0x558402aacb30 con 0x558402aaa1d0 2024-09-21T03:40:16.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.818+0000 7fcde0138640 1 -- 172.21.15.38:0/3447408364 <== osd.0 v2:172.21.15.38:6800/2381639433 2 ==== osd_op_reply(54 607.00000000 [delete] v96'4 uv4 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcdd400c9b0 con 0x558402aaa1d0 2024-09-21T03:40:16.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.818+0000 7fcde6e7ff40 1 -- 172.21.15.38:0/3447408364 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:55 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x558402aacb30 con 0x558402aaa1d0 2024-09-21T03:40:16.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.819+0000 7fcde0138640 1 -- 172.21.15.38:0/3447408364 <== osd.0 v2:172.21.15.38:6800/2381639433 3 ==== osd_op_reply(55 607.00000000 [create] v96'5 uv5 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcdd400c9b0 con 0x558402aaa1d0 2024-09-21T03:40:16.821 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-09-21T03:40:16.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.819+0000 7fcde6e7ff40 1 -- 172.21.15.38:0/3447408364 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:56 13.1c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x558402aadba0 con 0x558402aaa1d0 2024-09-21T03:40:16.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.820+0000 7fcde0138640 1 -- 172.21.15.38:0/3447408364 <== osd.0 v2:172.21.15.38:6800/2381639433 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v96'6 uv6 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcdd400c9b0 con 0x558402aaa1d0 2024-09-21T03:40:16.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.820+0000 7fcde6e7ff40 1 -- 172.21.15.38:0/3447408364 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:57 13.1c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x558402aadb60 con 0x558402aaa1d0 2024-09-21T03:40:16.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.821+0000 7fcde0138640 1 -- 172.21.15.38:0/3447408364 <== osd.0 v2:172.21.15.38:6800/2381639433 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v96'7 uv7 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcdd400c9b0 con 0x558402aaa1d0 2024-09-21T03:40:16.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.821+0000 7fcde6e7ff40 1 -- 172.21.15.38:0/3447408364 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:58 13.1c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x558402aadb60 con 0x558402aaa1d0 2024-09-21T03:40:16.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.822+0000 7fcde0138640 1 -- 172.21.15.38:0/3447408364 <== osd.0 v2:172.21.15.38:6800/2381639433 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v96'8 uv8 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcdd400c9b0 con 0x558402aaa1d0 2024-09-21T03:40:16.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.822+0000 7fcde6e7ff40 1 --2- 172.21.15.38:0/3447408364 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x558402aad040 0x558402aad450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:16.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.822+0000 7fcde6e7ff40 1 -- 172.21.15.38:0/3447408364 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:59 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x558402ab0080 con 0x558402aad040 2024-09-21T03:40:16.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.822+0000 7fcde113a640 1 --2- 172.21.15.38:0/3447408364 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x558402aad040 0x558402aad450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:16.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.822+0000 7fcde113a640 1 --2- 172.21.15.38:0/3447408364 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x558402aad040 0x558402aad450 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:16.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.824+0000 7fcde113a640 1 -- 172.21.15.38:0/3447408364 <== osd.3 v2:172.21.15.38:6808/1286810294 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v96'3 uv2 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7fcdd805d940 con 0x558402aad040 2024-09-21T03:40:16.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.824+0000 7fcde6e7ff40 1 -- 172.21.15.38:0/3447408364 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:60 13.6 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e96) -- 0x558402ab0420 con 0x558402aad040 2024-09-21T03:40:16.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.825+0000 7fcde113a640 1 -- 172.21.15.38:0/3447408364 <== osd.3 v2:172.21.15.38:6808/1286810294 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v96'4 uv4 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fcdd805d940 con 0x558402aad040 2024-09-21T03:40:16.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.825+0000 7fcde6e7ff40 1 -- 172.21.15.38:0/3447408364 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:61 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x558402ab0880 con 0x558402aad040 2024-09-21T03:40:16.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.826+0000 7fcde113a640 1 -- 172.21.15.38:0/3447408364 <== osd.3 v2:172.21.15.38:6808/1286810294 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v96'5 uv5 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fcdd805d940 con 0x558402aad040 2024-09-21T03:40:16.828 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-09-21T03:40:16.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.826+0000 7fcde6e7ff40 1 -- 172.21.15.38:0/3447408364 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:62 13.6 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x558402ab1140 con 0x558402aad040 2024-09-21T03:40:16.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.827+0000 7fcde113a640 1 -- 172.21.15.38:0/3447408364 <== osd.3 v2:172.21.15.38:6808/1286810294 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v96'6 uv6 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fcdd805d940 con 0x558402aad040 2024-09-21T03:40:16.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.827+0000 7fcde6e7ff40 1 -- 172.21.15.38:0/3447408364 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x558402ab14e0 con 0x558402aaa1d0 2024-09-21T03:40:16.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.828+0000 7fcde0138640 1 -- 172.21.15.38:0/3447408364 <== osd.0 v2:172.21.15.38:6800/2381639433 7 ==== osd_op_reply(63 608.00000000 [create] v96'3 uv2 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fcdd400c9b0 con 0x558402aaa1d0 2024-09-21T03:40:16.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.828+0000 7fcde6e7ff40 1 -- 172.21.15.38:0/3447408364 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e96) -- 0x558402ab14e0 con 0x558402aaa1d0 2024-09-21T03:40:16.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.829+0000 7fcde0138640 1 -- 172.21.15.38:0/3447408364 <== osd.0 v2:172.21.15.38:6800/2381639433 8 ==== osd_op_reply(64 608.00000000 [delete] v96'4 uv4 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcdd400c9b0 con 0x558402aaa1d0 2024-09-21T03:40:16.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.829+0000 7fcde6e7ff40 1 -- 172.21.15.38:0/3447408364 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x558402ab14e0 con 0x558402aaa1d0 2024-09-21T03:40:16.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.831+0000 7fcde0138640 1 -- 172.21.15.38:0/3447408364 <== osd.0 v2:172.21.15.38:6800/2381639433 9 ==== osd_op_reply(65 608.00000000 [create] v96'5 uv5 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcdd400c9b0 con 0x558402aaa1d0 2024-09-21T03:40:16.832 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-09-21T03:40:16.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.831+0000 7fcde6e7ff40 1 -- 172.21.15.38:0/3447408364 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x558402ab24f0 con 0x558402aaa1d0 2024-09-21T03:40:16.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.832+0000 7fcde0138640 1 -- 172.21.15.38:0/3447408364 <== osd.0 v2:172.21.15.38:6800/2381639433 10 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v96'6 uv6 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcdd400c9b0 con 0x558402aaa1d0 2024-09-21T03:40:16.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.832+0000 7fcde6e7ff40 1 -- 172.21.15.38:0/3447408364 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x558402ab24f0 con 0x558402aaa1d0 2024-09-21T03:40:16.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.833+0000 7fcde0138640 1 -- 172.21.15.38:0/3447408364 <== osd.0 v2:172.21.15.38:6800/2381639433 11 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v96'7 uv7 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcdd400c9b0 con 0x558402aaa1d0 2024-09-21T03:40:16.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.833+0000 7fcde6e7ff40 1 -- 172.21.15.38:0/3447408364 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x558402ab14e0 con 0x558402aaa1d0 2024-09-21T03:40:16.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.834+0000 7fcde0138640 1 -- 172.21.15.38:0/3447408364 <== osd.0 v2:172.21.15.38:6800/2381639433 12 ==== osd_op_reply(68 608.00000000 [omap-set-header] v96'8 uv8 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcdd400c9b0 con 0x558402aaa1d0 2024-09-21T03:40:16.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.834+0000 7fcde6e7ff40 1 -- 172.21.15.38:0/3447408364 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x558402ab14e0 con 0x558402aad040 2024-09-21T03:40:16.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.835+0000 7fcde113a640 1 -- 172.21.15.38:0/3447408364 <== osd.3 v2:172.21.15.38:6808/1286810294 5 ==== osd_op_reply(69 604.00000000 [create] v96'4 uv2 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fcdd805d940 con 0x558402aad040 2024-09-21T03:40:16.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.835+0000 7fcde6e7ff40 1 -- 172.21.15.38:0/3447408364 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e96) -- 0x558402ab1880 con 0x558402aad040 2024-09-21T03:40:16.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.836+0000 7fcde113a640 1 -- 172.21.15.38:0/3447408364 <== osd.3 v2:172.21.15.38:6808/1286810294 6 ==== osd_op_reply(70 604.00000000 [delete] v96'5 uv5 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcdd805d940 con 0x558402aad040 2024-09-21T03:40:16.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.836+0000 7fcde6e7ff40 1 -- 172.21.15.38:0/3447408364 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x558402ab1c20 con 0x558402aad040 2024-09-21T03:40:16.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.837+0000 7fcde113a640 1 -- 172.21.15.38:0/3447408364 <== osd.3 v2:172.21.15.38:6808/1286810294 7 ==== osd_op_reply(71 604.00000000 [create] v96'6 uv6 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcdd805d940 con 0x558402aad040 2024-09-21T03:40:16.839 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-09-21T03:40:16.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.837+0000 7fcde6e7ff40 1 -- 172.21.15.38:0/3447408364 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x558402ab2fd0 con 0x558402aad040 2024-09-21T03:40:16.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.838+0000 7fcde113a640 1 -- 172.21.15.38:0/3447408364 <== osd.3 v2:172.21.15.38:6808/1286810294 8 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v96'7 uv7 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcdd805d940 con 0x558402aad040 2024-09-21T03:40:16.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.838+0000 7fcde6e7ff40 1 -- 172.21.15.38:0/3447408364 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x558402ab2fd0 con 0x558402aad040 2024-09-21T03:40:16.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.839+0000 7fcde113a640 1 -- 172.21.15.38:0/3447408364 <== osd.3 v2:172.21.15.38:6808/1286810294 9 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v96'8 uv8 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcdd805d940 con 0x558402aad040 2024-09-21T03:40:16.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.839+0000 7fcde6e7ff40 1 -- 172.21.15.38:0/3447408364 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x558402ab1fc0 con 0x558402aad040 2024-09-21T03:40:16.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.840+0000 7fcde113a640 1 -- 172.21.15.38:0/3447408364 <== osd.3 v2:172.21.15.38:6808/1286810294 10 ==== osd_op_reply(74 604.00000000 [omap-set-header] v96'9 uv9 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcdd805d940 con 0x558402aad040 2024-09-21T03:40:16.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.840+0000 7fcde6e7ff40 1 -- 172.21.15.38:0/3447408364 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x558402ab1fc0 con 0x558402aad040 2024-09-21T03:40:16.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.841+0000 7fcde113a640 1 -- 172.21.15.38:0/3447408364 <== osd.3 v2:172.21.15.38:6808/1286810294 11 ==== osd_op_reply(75 500.00000000 [create] v96'10 uv3 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fcdd805d940 con 0x558402aad040 2024-09-21T03:40:16.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.841+0000 7fcde6e7ff40 1 -- 172.21.15.38:0/3447408364 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e96) -- 0x558402ab1fc0 con 0x558402aad040 2024-09-21T03:40:16.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.842+0000 7fcde113a640 1 -- 172.21.15.38:0/3447408364 <== osd.3 v2:172.21.15.38:6808/1286810294 12 ==== osd_op_reply(76 500.00000000 [delete] v96'11 uv11 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcdd805d940 con 0x558402aad040 2024-09-21T03:40:16.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.842+0000 7fcde6e7ff40 1 -- 172.21.15.38:0/3447408364 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x558402ab1fc0 con 0x558402aad040 2024-09-21T03:40:16.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.843+0000 7fcde113a640 1 -- 172.21.15.38:0/3447408364 <== osd.3 v2:172.21.15.38:6808/1286810294 13 ==== osd_op_reply(77 500.00000000 [create] v96'12 uv12 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcdd805d940 con 0x558402aad040 2024-09-21T03:40:16.844 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-09-21T03:40:16.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.843+0000 7fcde6e7ff40 1 -- 172.21.15.38:0/3447408364 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x558402ab1fc0 con 0x558402aad040 2024-09-21T03:40:16.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.844+0000 7fcde113a640 1 -- 172.21.15.38:0/3447408364 <== osd.3 v2:172.21.15.38:6808/1286810294 14 ==== osd_op_reply(78 500.00000000 [write 0~90] v96'13 uv13 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcdd805d940 con 0x558402aad040 2024-09-21T03:40:16.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.844+0000 7fcde6e7ff40 1 -- 172.21.15.38:0/3447408364 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x558402ab1fc0 con 0x558402aad040 2024-09-21T03:40:16.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.845+0000 7fcde113a640 1 -- 172.21.15.38:0/3447408364 <== osd.3 v2:172.21.15.38:6808/1286810294 15 ==== osd_op_reply(79 500.00000000 [omap-set-header] v96'14 uv14 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcdd805d940 con 0x558402aad040 2024-09-21T03:40:16.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.845+0000 7fcde6e7ff40 1 -- 172.21.15.38:0/3447408364 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:80 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x558402ab1fc0 con 0x558402aaa1d0 2024-09-21T03:40:16.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.846+0000 7fcde0138640 1 -- 172.21.15.38:0/3447408364 <== osd.0 v2:172.21.15.38:6800/2381639433 13 ==== osd_op_reply(80 mds_snaptable [create] v96'2 uv1 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7fcdd400c9b0 con 0x558402aaa1d0 2024-09-21T03:40:16.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.846+0000 7fcde6e7ff40 1 -- 172.21.15.38:0/3447408364 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:81 13.d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e96) -- 0x558402ab1fc0 con 0x558402aaa1d0 2024-09-21T03:40:16.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.847+0000 7fcde0138640 1 -- 172.21.15.38:0/3447408364 <== osd.0 v2:172.21.15.38:6800/2381639433 14 ==== osd_op_reply(81 mds_snaptable [delete] v96'3 uv3 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fcdd400c9b0 con 0x558402aaa1d0 2024-09-21T03:40:16.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.847+0000 7fcde6e7ff40 1 -- 172.21.15.38:0/3447408364 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:82 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x558402ab1fc0 con 0x558402aaa1d0 2024-09-21T03:40:16.849 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-09-21T03:40:16.850 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.848+0000 7fcde0138640 1 -- 172.21.15.38:0/3447408364 <== osd.0 v2:172.21.15.38:6800/2381639433 15 ==== osd_op_reply(82 mds_snaptable [create] v96'4 uv4 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fcdd400c9b0 con 0x558402aaa1d0 2024-09-21T03:40:16.850 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.848+0000 7fcde6e7ff40 1 -- 172.21.15.38:0/3447408364 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:83 13.d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x558402ab1fc0 con 0x558402aaa1d0 2024-09-21T03:40:16.851 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.849+0000 7fcde0138640 1 -- 172.21.15.38:0/3447408364 <== osd.0 v2:172.21.15.38:6800/2381639433 16 ==== osd_op_reply(83 mds_snaptable [write 0~70] v96'5 uv5 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fcdd400c9b0 con 0x558402aaa1d0 2024-09-21T03:40:16.851 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.849+0000 7fcde6e7ff40 1 -- 172.21.15.38:0/3447408364 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:84 13.d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x558402ab1fc0 con 0x558402aaa1d0 2024-09-21T03:40:16.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.850+0000 7fcde0138640 1 -- 172.21.15.38:0/3447408364 <== osd.0 v2:172.21.15.38:6800/2381639433 17 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v96'6 uv6 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fcdd400c9b0 con 0x558402aaa1d0 2024-09-21T03:40:16.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.850+0000 7fcde6e7ff40 1 --2- 172.21.15.38:0/3447408364 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x558402ab2640 0x558402ab4a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:16.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.850+0000 7fcde6e7ff40 1 -- 172.21.15.38:0/3447408364 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x558402ab5100 con 0x558402ab2640 2024-09-21T03:40:16.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.850+0000 7fcde0939640 1 --2- 172.21.15.38:0/3447408364 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x558402ab2640 0x558402ab4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:16.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.851+0000 7fcde0939640 1 --2- 172.21.15.38:0/3447408364 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x558402ab2640 0x558402ab4a80 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:16.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.852+0000 7fcde0939640 1 -- 172.21.15.38:0/3447408364 <== osd.1 v2:172.21.15.38:6816/1491818732 1 ==== osd_op_reply(85 605.00000000 [create] v96'3 uv2 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fcdd006a040 con 0x558402ab2640 2024-09-21T03:40:16.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.852+0000 7fcde6e7ff40 1 -- 172.21.15.38:0/3447408364 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e96) -- 0x558402ab5200 con 0x558402ab2640 2024-09-21T03:40:16.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.853+0000 7fcde0939640 1 -- 172.21.15.38:0/3447408364 <== osd.1 v2:172.21.15.38:6816/1491818732 2 ==== osd_op_reply(86 605.00000000 [delete] v96'4 uv4 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcdd006a040 con 0x558402ab2640 2024-09-21T03:40:16.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.853+0000 7fcde6e7ff40 1 -- 172.21.15.38:0/3447408364 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x558402ab5620 con 0x558402ab2640 2024-09-21T03:40:16.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.854+0000 7fcde0939640 1 -- 172.21.15.38:0/3447408364 <== osd.1 v2:172.21.15.38:6816/1491818732 3 ==== osd_op_reply(87 605.00000000 [create] v96'5 uv5 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcdd006a040 con 0x558402ab2640 2024-09-21T03:40:16.856 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-09-21T03:40:16.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.854+0000 7fcde6e7ff40 1 -- 172.21.15.38:0/3447408364 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x558402ab6630 con 0x558402ab2640 2024-09-21T03:40:16.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.855+0000 7fcde0939640 1 -- 172.21.15.38:0/3447408364 <== osd.1 v2:172.21.15.38:6816/1491818732 4 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v96'6 uv6 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcdd006a040 con 0x558402ab2640 2024-09-21T03:40:16.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.855+0000 7fcde6e7ff40 1 -- 172.21.15.38:0/3447408364 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x558402ab6630 con 0x558402ab2640 2024-09-21T03:40:16.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.856+0000 7fcde0939640 1 -- 172.21.15.38:0/3447408364 <== osd.1 v2:172.21.15.38:6816/1491818732 5 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v96'7 uv7 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcdd006a040 con 0x558402ab2640 2024-09-21T03:40:16.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.856+0000 7fcde6e7ff40 1 -- 172.21.15.38:0/3447408364 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x558402ab5620 con 0x558402ab2640 2024-09-21T03:40:16.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.857+0000 7fcde0939640 1 -- 172.21.15.38:0/3447408364 <== osd.1 v2:172.21.15.38:6816/1491818732 6 ==== osd_op_reply(90 605.00000000 [omap-set-header] v96'8 uv8 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcdd006a040 con 0x558402ab2640 2024-09-21T03:40:16.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.857+0000 7fcde6e7ff40 1 -- 172.21.15.38:0/3447408364 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x558402ab5620 con 0x558402ab2640 2024-09-21T03:40:16.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.858+0000 7fcde0939640 1 -- 172.21.15.38:0/3447408364 <== osd.1 v2:172.21.15.38:6816/1491818732 7 ==== osd_op_reply(91 mds0_inotable [create] v96'4 uv3 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7fcdd006a040 con 0x558402ab2640 2024-09-21T03:40:16.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.858+0000 7fcde6e7ff40 1 -- 172.21.15.38:0/3447408364 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e96) -- 0x558402ab5620 con 0x558402ab2640 2024-09-21T03:40:16.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.859+0000 7fcde0939640 1 -- 172.21.15.38:0/3447408364 <== osd.1 v2:172.21.15.38:6816/1491818732 8 ==== osd_op_reply(92 mds0_inotable [delete] v96'5 uv5 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fcdd006a040 con 0x558402ab2640 2024-09-21T03:40:16.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.859+0000 7fcde6e7ff40 1 -- 172.21.15.38:0/3447408364 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x558402ab5620 con 0x558402ab2640 2024-09-21T03:40:16.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.860+0000 7fcde0939640 1 -- 172.21.15.38:0/3447408364 <== osd.1 v2:172.21.15.38:6816/1491818732 9 ==== osd_op_reply(93 mds0_inotable [create] v96'6 uv6 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fcdd006a040 con 0x558402ab2640 2024-09-21T03:40:16.862 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-09-21T03:40:16.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.860+0000 7fcde6e7ff40 1 -- 172.21.15.38:0/3447408364 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:94 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x558402ab0d30 con 0x558402ab2640 2024-09-21T03:40:16.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.861+0000 7fcde0939640 1 -- 172.21.15.38:0/3447408364 <== osd.1 v2:172.21.15.38:6816/1491818732 10 ==== osd_op_reply(94 mds0_inotable [write 0~34] v96'7 uv7 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fcdd006a040 con 0x558402ab2640 2024-09-21T03:40:16.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.861+0000 7fcde6e7ff40 1 -- 172.21.15.38:0/3447408364 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:95 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x558402ab0d30 con 0x558402ab2640 2024-09-21T03:40:16.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.862+0000 7fcde0939640 1 -- 172.21.15.38:0/3447408364 <== osd.1 v2:172.21.15.38:6816/1491818732 11 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v96'8 uv8 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fcdd006a040 con 0x558402ab2640 2024-09-21T03:40:16.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.862+0000 7fcde6e7ff40 1 -- 172.21.15.38:0/3447408364 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:96 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x558402ab0d30 con 0x558402ab2640 2024-09-21T03:40:16.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.863+0000 7fcde0939640 1 -- 172.21.15.38:0/3447408364 <== osd.1 v2:172.21.15.38:6816/1491818732 12 ==== osd_op_reply(96 100.00000000 [create] v96'9 uv1 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fcdd006a040 con 0x558402ab2640 2024-09-21T03:40:16.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.863+0000 7fcde6e7ff40 1 -- 172.21.15.38:0/3447408364 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:97 13.13 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e96) -- 0x558402ab0d30 con 0x558402ab2640 2024-09-21T03:40:16.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.864+0000 7fcde0939640 1 -- 172.21.15.38:0/3447408364 <== osd.1 v2:172.21.15.38:6816/1491818732 13 ==== osd_op_reply(97 100.00000000 [delete] v96'10 uv10 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcdd006a040 con 0x558402ab2640 2024-09-21T03:40:16.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.864+0000 7fcde6e7ff40 1 -- 172.21.15.38:0/3447408364 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:98 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x558402ab0d30 con 0x558402ab2640 2024-09-21T03:40:16.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.865+0000 7fcde0939640 1 -- 172.21.15.38:0/3447408364 <== osd.1 v2:172.21.15.38:6816/1491818732 14 ==== osd_op_reply(98 100.00000000 [create] v96'11 uv11 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcdd006a040 con 0x558402ab2640 2024-09-21T03:40:16.867 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-09-21T03:40:16.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.865+0000 7fcde6e7ff40 1 -- 172.21.15.38:0/3447408364 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:99 13.13 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x558402ab0d30 con 0x558402ab2640 2024-09-21T03:40:16.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.867+0000 7fcde0939640 1 -- 172.21.15.38:0/3447408364 <== osd.1 v2:172.21.15.38:6816/1491818732 15 ==== osd_op_reply(99 100.00000000 [omap-set-header] v96'12 uv12 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcdd006a040 con 0x558402ab2640 2024-09-21T03:40:16.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.867+0000 7fcde6e7ff40 1 -- 172.21.15.38:0/3447408364 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:100 13.13 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x558402ab8430 con 0x558402ab2640 2024-09-21T03:40:16.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.868+0000 7fcde0939640 1 -- 172.21.15.38:0/3447408364 <== osd.1 v2:172.21.15.38:6816/1491818732 16 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v96'13 uv13 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcdd006a040 con 0x558402ab2640 2024-09-21T03:40:16.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.868+0000 7fcde6e7ff40 1 -- 172.21.15.38:0/3447408364 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x558402ab5ad0 con 0x558402aad040 2024-09-21T03:40:16.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.869+0000 7fcde113a640 1 -- 172.21.15.38:0/3447408364 <== osd.3 v2:172.21.15.38:6808/1286810294 16 ==== osd_op_reply(101 mds0_sessionmap [create] v96'4 uv3 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7fcdd805d940 con 0x558402aad040 2024-09-21T03:40:16.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.869+0000 7fcde6e7ff40 1 -- 172.21.15.38:0/3447408364 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e96) -- 0x558402ab5ad0 con 0x558402aad040 2024-09-21T03:40:16.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.870+0000 7fcde113a640 1 -- 172.21.15.38:0/3447408364 <== osd.3 v2:172.21.15.38:6808/1286810294 17 ==== osd_op_reply(102 mds0_sessionmap [delete] v96'5 uv5 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fcdd805d940 con 0x558402aad040 2024-09-21T03:40:16.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.870+0000 7fcde6e7ff40 1 -- 172.21.15.38:0/3447408364 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:103 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x558402ab5ad0 con 0x558402aad040 2024-09-21T03:40:16.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.871+0000 7fcde113a640 1 -- 172.21.15.38:0/3447408364 <== osd.3 v2:172.21.15.38:6808/1286810294 18 ==== osd_op_reply(103 mds0_sessionmap [create] v96'6 uv6 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fcdd805d940 con 0x558402aad040 2024-09-21T03:40:16.873 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-09-21T03:40:16.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.871+0000 7fcde6e7ff40 1 -- 172.21.15.38:0/3447408364 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:104 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x558402ab5ad0 con 0x558402aad040 2024-09-21T03:40:16.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.872+0000 7fcde113a640 1 -- 172.21.15.38:0/3447408364 <== osd.3 v2:172.21.15.38:6808/1286810294 19 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v96'7 uv7 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fcdd805d940 con 0x558402aad040 2024-09-21T03:40:16.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.872+0000 7fcde6e7ff40 1 -- 172.21.15.38:0/3447408364 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x558402ab5ad0 con 0x558402aad040 2024-09-21T03:40:16.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.873+0000 7fcde113a640 1 -- 172.21.15.38:0/3447408364 <== osd.3 v2:172.21.15.38:6808/1286810294 20 ==== osd_op_reply(105 609.00000000 [create] v96'3 uv2 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fcdd805d940 con 0x558402aad040 2024-09-21T03:40:16.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.873+0000 7fcde6e7ff40 1 -- 172.21.15.38:0/3447408364 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e96) -- 0x558402ab5ad0 con 0x558402aad040 2024-09-21T03:40:16.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.874+0000 7fcde113a640 1 -- 172.21.15.38:0/3447408364 <== osd.3 v2:172.21.15.38:6808/1286810294 21 ==== osd_op_reply(106 609.00000000 [delete] v96'4 uv4 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcdd805d940 con 0x558402aad040 2024-09-21T03:40:16.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.874+0000 7fcde6e7ff40 1 -- 172.21.15.38:0/3447408364 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x558402ab5ad0 con 0x558402aad040 2024-09-21T03:40:16.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.875+0000 7fcde113a640 1 -- 172.21.15.38:0/3447408364 <== osd.3 v2:172.21.15.38:6808/1286810294 22 ==== osd_op_reply(107 609.00000000 [create] v96'5 uv5 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcdd805d940 con 0x558402aad040 2024-09-21T03:40:16.877 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-09-21T03:40:16.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.875+0000 7fcde6e7ff40 1 -- 172.21.15.38:0/3447408364 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x558402ab6ae0 con 0x558402aad040 2024-09-21T03:40:16.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.876+0000 7fcde113a640 1 -- 172.21.15.38:0/3447408364 <== osd.3 v2:172.21.15.38:6808/1286810294 23 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v96'6 uv6 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcdd805d940 con 0x558402aad040 2024-09-21T03:40:16.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.876+0000 7fcde6e7ff40 1 -- 172.21.15.38:0/3447408364 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:109 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x558402ab6ae0 con 0x558402aad040 2024-09-21T03:40:16.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.877+0000 7fcde113a640 1 -- 172.21.15.38:0/3447408364 <== osd.3 v2:172.21.15.38:6808/1286810294 24 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v96'7 uv7 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcdd805d940 con 0x558402aad040 2024-09-21T03:40:16.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.877+0000 7fcde6e7ff40 1 -- 172.21.15.38:0/3447408364 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:110 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x558402ab5ad0 con 0x558402aad040 2024-09-21T03:40:16.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.878+0000 7fcde113a640 1 -- 172.21.15.38:0/3447408364 <== osd.3 v2:172.21.15.38:6808/1286810294 25 ==== osd_op_reply(110 609.00000000 [omap-set-header] v96'8 uv8 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcdd805d940 con 0x558402aad040 2024-09-21T03:40:16.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.878+0000 7fcde6e7ff40 1 --2- 172.21.15.38:0/3447408364 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x558402ab5fe0 0x558402ab63f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:16.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.878+0000 7fcde6e7ff40 1 -- 172.21.15.38:0/3447408364 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x558402ab6930 con 0x558402ab5fe0 2024-09-21T03:40:16.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.878+0000 7fcde0138640 1 --2- 172.21.15.38:0/3447408364 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x558402ab5fe0 0x558402ab63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:16.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.879+0000 7fcde0138640 1 --2- 172.21.15.38:0/3447408364 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x558402ab5fe0 0x558402ab63f0 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:16.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.880+0000 7fcde0138640 1 -- 172.21.15.38:0/3447408364 <== osd.2 v2:172.21.15.38:6824/2577858902 1 ==== osd_op_reply(111 400.00000000 [create] v96'2 uv1 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fcdd4025040 con 0x558402ab5fe0 2024-09-21T03:40:16.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.880+0000 7fcde6e7ff40 1 -- 172.21.15.38:0/3447408364 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e96) -- 0x558402ab6930 con 0x558402ab5fe0 2024-09-21T03:40:16.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.881+0000 7fcde0138640 1 -- 172.21.15.38:0/3447408364 <== osd.2 v2:172.21.15.38:6824/2577858902 2 ==== osd_op_reply(112 400.00000000 [delete] v96'3 uv3 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcdd4025040 con 0x558402ab5fe0 2024-09-21T03:40:16.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.881+0000 7fcde6e7ff40 1 -- 172.21.15.38:0/3447408364 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x558402ab6930 con 0x558402ab5fe0 2024-09-21T03:40:16.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.882+0000 7fcde0138640 1 -- 172.21.15.38:0/3447408364 <== osd.2 v2:172.21.15.38:6824/2577858902 3 ==== osd_op_reply(113 400.00000000 [create] v96'4 uv4 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcdd4025040 con 0x558402ab5fe0 2024-09-21T03:40:16.884 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-09-21T03:40:16.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.882+0000 7fcde6e7ff40 1 -- 172.21.15.38:0/3447408364 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:114 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x558402ab6ec0 con 0x558402ab5fe0 2024-09-21T03:40:16.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.883+0000 7fcde0138640 1 -- 172.21.15.38:0/3447408364 <== osd.2 v2:172.21.15.38:6824/2577858902 4 ==== osd_op_reply(114 400.00000000 [write 0~22] v96'5 uv5 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcdd4025040 con 0x558402ab5fe0 2024-09-21T03:40:16.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.883+0000 7fcde6e7ff40 1 -- 172.21.15.38:0/3447408364 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:115 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x558402ab6ec0 con 0x558402ab5fe0 2024-09-21T03:40:16.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.884+0000 7fcde0138640 1 -- 172.21.15.38:0/3447408364 <== osd.2 v2:172.21.15.38:6824/2577858902 5 ==== osd_op_reply(115 400.00000000 [omap-set-header] v96'6 uv6 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcdd4025040 con 0x558402ab5fe0 2024-09-21T03:40:16.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.884+0000 7fcde6e7ff40 1 -- 172.21.15.38:0/3447408364 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:116 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x558402ab6ec0 con 0x558402ab2640 2024-09-21T03:40:16.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.885+0000 7fcde0939640 1 -- 172.21.15.38:0/3447408364 <== osd.1 v2:172.21.15.38:6816/1491818732 17 ==== osd_op_reply(116 100.00000000.inode [create] v96'7 uv3 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7fcdd006a040 con 0x558402ab2640 2024-09-21T03:40:16.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.885+0000 7fcde6e7ff40 1 -- 172.21.15.38:0/3447408364 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:117 13.1f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e96) -- 0x558402ab6ec0 con 0x558402ab2640 2024-09-21T03:40:16.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.886+0000 7fcde0939640 1 -- 172.21.15.38:0/3447408364 <== osd.1 v2:172.21.15.38:6816/1491818732 18 ==== osd_op_reply(117 100.00000000.inode [delete] v96'8 uv8 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fcdd006a040 con 0x558402ab2640 2024-09-21T03:40:16.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.887+0000 7fcde6e7ff40 1 -- 172.21.15.38:0/3447408364 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:118 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x558402ab6ec0 con 0x558402ab2640 2024-09-21T03:40:16.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.888+0000 7fcde0939640 1 -- 172.21.15.38:0/3447408364 <== osd.1 v2:172.21.15.38:6816/1491818732 19 ==== osd_op_reply(118 100.00000000.inode [create] v96'9 uv9 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fcdd006a040 con 0x558402ab2640 2024-09-21T03:40:16.889 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-09-21T03:40:16.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.888+0000 7fcde6e7ff40 1 -- 172.21.15.38:0/3447408364 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:119 13.1f 13:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x558402ab6ec0 con 0x558402ab2640 2024-09-21T03:40:16.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.889+0000 7fcde0939640 1 -- 172.21.15.38:0/3447408364 <== osd.1 v2:172.21.15.38:6816/1491818732 20 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v96'10 uv10 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fcdd006a040 con 0x558402ab2640 2024-09-21T03:40:16.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.889+0000 7fcde6e7ff40 1 -- 172.21.15.38:0/3447408364 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:120 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x558402ab6ec0 con 0x558402ab2640 2024-09-21T03:40:16.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.890+0000 7fcde0939640 1 -- 172.21.15.38:0/3447408364 <== osd.1 v2:172.21.15.38:6816/1491818732 21 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v96'11 uv11 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fcdd006a040 con 0x558402ab2640 2024-09-21T03:40:16.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.890+0000 7fcde6e7ff40 1 -- 172.21.15.38:0/3447408364 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:121 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x558402ab6ec0 con 0x558402ab2640 2024-09-21T03:40:16.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.891+0000 7fcde0939640 1 -- 172.21.15.38:0/3447408364 <== osd.1 v2:172.21.15.38:6816/1491818732 22 ==== osd_op_reply(121 1.00000000 [create] v96'12 uv6 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7fcdd006a040 con 0x558402ab2640 2024-09-21T03:40:16.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.891+0000 7fcde6e7ff40 1 -- 172.21.15.38:0/3447408364 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:122 13.1f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e96) -- 0x558402ab6ec0 con 0x558402ab2640 2024-09-21T03:40:16.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.892+0000 7fcde0939640 1 -- 172.21.15.38:0/3447408364 <== osd.1 v2:172.21.15.38:6816/1491818732 23 ==== osd_op_reply(122 1.00000000 [delete] v96'13 uv13 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fcdd006a040 con 0x558402ab2640 2024-09-21T03:40:16.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.892+0000 7fcde6e7ff40 1 -- 172.21.15.38:0/3447408364 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:123 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x558402ab6ec0 con 0x558402ab2640 2024-09-21T03:40:16.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.893+0000 7fcde0939640 1 -- 172.21.15.38:0/3447408364 <== osd.1 v2:172.21.15.38:6816/1491818732 24 ==== osd_op_reply(123 1.00000000 [create] v96'14 uv14 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fcdd006a040 con 0x558402ab2640 2024-09-21T03:40:16.895 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-09-21T03:40:16.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.893+0000 7fcde6e7ff40 1 -- 172.21.15.38:0/3447408364 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:124 13.1f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x558402ab6ec0 con 0x558402ab2640 2024-09-21T03:40:16.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.894+0000 7fcde0939640 1 -- 172.21.15.38:0/3447408364 <== osd.1 v2:172.21.15.38:6816/1491818732 25 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v96'15 uv15 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fcdd006a040 con 0x558402ab2640 2024-09-21T03:40:16.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.894+0000 7fcde6e7ff40 1 -- 172.21.15.38:0/3447408364 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:125 13.1f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x558402ab6ec0 con 0x558402ab2640 2024-09-21T03:40:16.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.895+0000 7fcde0939640 1 -- 172.21.15.38:0/3447408364 <== osd.1 v2:172.21.15.38:6816/1491818732 26 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v96'16 uv16 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fcdd006a040 con 0x558402ab2640 2024-09-21T03:40:16.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.895+0000 7fcde6e7ff40 1 -- 172.21.15.38:0/3447408364 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:126 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x558402ab6ec0 con 0x558402ab2640 2024-09-21T03:40:16.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.896+0000 7fcde0939640 1 -- 172.21.15.38:0/3447408364 <== osd.1 v2:172.21.15.38:6816/1491818732 27 ==== osd_op_reply(126 1.00000000 [omap-set-header] v96'17 uv17 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fcdd006a040 con 0x558402ab2640 2024-09-21T03:40:16.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.896+0000 7fcde6e7ff40 1 -- 172.21.15.38:0/3447408364 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:127 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=584b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x558402ab72d0 con 0x558402ab2640 2024-09-21T03:40:16.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.897+0000 7fcde0939640 1 -- 172.21.15.38:0/3447408364 <== osd.1 v2:172.21.15.38:6816/1491818732 28 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v96'18 uv18 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fcdd006a040 con 0x558402ab2640 2024-09-21T03:40:16.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.897+0000 7fcde6e7ff40 1 -- 172.21.15.38:0/3447408364 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x558402aaa1d0 msgr2=0x558402aac5f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:16.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.897+0000 7fcde6e7ff40 1 --2- 172.21.15.38:0/3447408364 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x558402aaa1d0 0x558402aac5f0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:16.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.897+0000 7fcde6e7ff40 1 -- 172.21.15.38:0/3447408364 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x558402ab2640 msgr2=0x558402ab4a80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:16.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.897+0000 7fcde6e7ff40 1 --2- 172.21.15.38:0/3447408364 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x558402ab2640 0x558402ab4a80 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:16.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.897+0000 7fcde6e7ff40 1 -- 172.21.15.38:0/3447408364 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x558402ab5fe0 msgr2=0x558402ab63f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:16.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.897+0000 7fcde6e7ff40 1 --2- 172.21.15.38:0/3447408364 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x558402ab5fe0 0x558402ab63f0 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:16.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.897+0000 7fcde6e7ff40 1 -- 172.21.15.38:0/3447408364 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x558402aad040 msgr2=0x558402aad450 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:16.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.897+0000 7fcde6e7ff40 1 --2- 172.21.15.38:0/3447408364 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x558402aad040 0x558402aad450 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:16.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.897+0000 7fcde113a640 1 -- 172.21.15.38:0/3447408364 reap_dead start 2024-09-21T03:40:16.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.897+0000 7fcde6e7ff40 1 -- 172.21.15.38:0/3447408364 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x558402aa64e0 msgr2=0x558402aa8900 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:16.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.897+0000 7fcde6e7ff40 1 --2- 172.21.15.38:0/3447408364 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x558402aa64e0 0x558402aa8900 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:16.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.897+0000 7fcde113a640 1 -- 172.21.15.38:0/3447408364 reap_dead start 2024-09-21T03:40:16.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.897+0000 7fcde6e7ff40 1 -- 172.21.15.38:0/3447408364 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x558402aa1cf0 msgr2=0x558402aa4110 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:16.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.897+0000 7fcde6e7ff40 1 --2- 172.21.15.38:0/3447408364 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x558402aa1cf0 0x558402aa4110 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:16.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.897+0000 7fcde6e7ff40 1 -- 172.21.15.38:0/3447408364 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fcdcc0775a0 msgr2=0x7fcdcc079ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:16.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.897+0000 7fcde6e7ff40 1 --2- 172.21.15.38:0/3447408364 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fcdcc0775a0 0x7fcdcc079ab0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fcdd0046600 tx=0x7fcdd0064040 comp rx=0 tx=0).stop 2024-09-21T03:40:16.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.898+0000 7fcde6e7ff40 1 -- 172.21.15.38:0/3447408364 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5584029e7520 msgr2=0x5584029cf9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:16.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.898+0000 7fcde6e7ff40 1 --2- 172.21.15.38:0/3447408364 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5584029e7520 0x5584029cf9c0 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7fcdd8002fa0 tx=0x7fcdd800d5f0 comp rx=0 tx=0).stop 2024-09-21T03:40:16.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.898+0000 7fcde6e7ff40 1 -- 172.21.15.38:0/3447408364 shutdown_connections 2024-09-21T03:40:16.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.898+0000 7fcde6e7ff40 1 --2- 172.21.15.38:0/3447408364 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x558402aa1cf0 0x558402aa4110 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:16.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.898+0000 7fcde6e7ff40 1 --2- 172.21.15.38:0/3447408364 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5584029e7520 0x5584029cf9c0 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:16.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.898+0000 7fcde6e7ff40 1 --2- 172.21.15.38:0/3447408364 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fcdcc0775a0 0x7fcdcc079ab0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:16.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.898+0000 7fcde6e7ff40 1 -- 172.21.15.38:0/3447408364 >> 172.21.15.38:0/3447408364 conn(0x558402849ac0 msgr2=0x558402971930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:16.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.898+0000 7fcde6e7ff40 1 -- 172.21.15.38:0/3447408364 shutdown_connections 2024-09-21T03:40:16.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.898+0000 7fcde6e7ff40 1 -- 172.21.15.38:0/3447408364 wait complete. 2024-09-21T03:40:16.903 DEBUG:teuthology.orchestra.run.smithi038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rm 10000000000.00000000 2024-09-21T03:40:16.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.996+0000 7f6d72d1df40 1 Processor -- start 2024-09-21T03:40:16.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.996+0000 7f6d72d1df40 1 -- start start 2024-09-21T03:40:17.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.997+0000 7f6d72d1df40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56469721d240 0x5646972eaad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:17.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.997+0000 7f6d72d1df40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564697264090 0x564697245b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:17.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.997+0000 7f6d72d1df40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5646972d9520 0x564697246040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:17.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.997+0000 7f6d72d1df40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x56469716ad60 con 0x56469721d240 2024-09-21T03:40:17.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.997+0000 7f6d72d1df40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5646971c9430 con 0x564697264090 2024-09-21T03:40:17.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.997+0000 7f6d72d1df40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x5646972f0aa0 con 0x5646972d9520 2024-09-21T03:40:17.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.998+0000 7f6d6bd62640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56469721d240 0x5646972eaad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:17.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.998+0000 7f6d6bd62640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56469721d240 0x5646972eaad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:55248/0 (socket says 172.21.15.38:55248) 2024-09-21T03:40:17.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.998+0000 7f6d6bd62640 1 -- 172.21.15.38:0/3832510927 learned_addr learned my addr 172.21.15.38:0/3832510927 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:17.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.998+0000 7f6d6b561640 1 --2- 172.21.15.38:0/3832510927 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564697264090 0x564697245b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:17.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.998+0000 7f6d6c563640 1 --2- 172.21.15.38:0/3832510927 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5646972d9520 0x564697246040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:17.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.998+0000 7f6d6bd62640 1 -- 172.21.15.38:0/3832510927 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5646972d9520 msgr2=0x564697246040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:17.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.998+0000 7f6d6bd62640 1 --2- 172.21.15.38:0/3832510927 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5646972d9520 0x564697246040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:17.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.998+0000 7f6d6bd62640 1 -- 172.21.15.38:0/3832510927 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564697264090 msgr2=0x564697245b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:17.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.998+0000 7f6d6bd62640 1 --2- 172.21.15.38:0/3832510927 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564697264090 0x564697245b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:17.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.998+0000 7f6d6bd62640 1 -- 172.21.15.38:0/3832510927 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646972c1b80 con 0x56469721d240 2024-09-21T03:40:17.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.998+0000 7f6d6c563640 1 --2- 172.21.15.38:0/3832510927 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5646972d9520 0x564697246040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:40:17.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.998+0000 7f6d6bd62640 1 --2- 172.21.15.38:0/3832510927 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56469721d240 0x5646972eaad0 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f6d5c046640 tx=0x7f6d5c002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1840d2862eb5114 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:17.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.998+0000 7f6d6a55f640 1 -- 172.21.15.38:0/3832510927 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d5c04d9a0 con 0x56469721d240 2024-09-21T03:40:17.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.998+0000 7f6d6a55f640 1 -- 172.21.15.38:0/3832510927 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6d5c056670 con 0x56469721d240 2024-09-21T03:40:17.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.998+0000 7f6d6a55f640 1 -- 172.21.15.38:0/3832510927 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d5c056ac0 con 0x56469721d240 2024-09-21T03:40:17.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.999+0000 7f6d72d1df40 1 -- 172.21.15.38:0/3832510927 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56469721d240 msgr2=0x5646972eaad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:17.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.999+0000 7f6d72d1df40 1 --2- 172.21.15.38:0/3832510927 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56469721d240 0x5646972eaad0 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f6d5c046640 tx=0x7f6d5c002fb0 comp rx=0 tx=0).stop 2024-09-21T03:40:17.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.999+0000 7f6d72d1df40 1 -- 172.21.15.38:0/3832510927 shutdown_connections 2024-09-21T03:40:17.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.999+0000 7f6d72d1df40 1 --2- 172.21.15.38:0/3832510927 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5646972d9520 0x564697246040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:17.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.999+0000 7f6d72d1df40 1 --2- 172.21.15.38:0/3832510927 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564697264090 0x564697245b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:17.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.999+0000 7f6d72d1df40 1 --2- 172.21.15.38:0/3832510927 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56469721d240 0x5646972eaad0 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:17.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.999+0000 7f6d72d1df40 1 -- 172.21.15.38:0/3832510927 >> 172.21.15.38:0/3832510927 conn(0x56469713bac0 msgr2=0x564697246ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:17.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.999+0000 7f6d72d1df40 1 -- 172.21.15.38:0/3832510927 shutdown_connections 2024-09-21T03:40:17.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:16.999+0000 7f6d72d1df40 1 -- 172.21.15.38:0/3832510927 wait complete. 2024-09-21T03:40:17.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:17.000+0000 7f6d72d1df40 1 Processor -- start 2024-09-21T03:40:17.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:17.000+0000 7f6d72d1df40 1 -- start start 2024-09-21T03:40:17.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:17.001+0000 7f6d72d1df40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56469721d240 0x5646972c5a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:17.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:17.001+0000 7f6d72d1df40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564697264090 0x5646972e7350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:17.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:17.001+0000 7f6d72d1df40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5646972d9520 0x564697249af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:17.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:17.001+0000 7f6d72d1df40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x56469721c2f0 con 0x564697264090 2024-09-21T03:40:17.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:17.001+0000 7f6d72d1df40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x564697389570 con 0x5646972d9520 2024-09-21T03:40:17.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:17.001+0000 7f6d72d1df40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x5646972c5120 con 0x56469721d240 2024-09-21T03:40:17.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:17.001+0000 7f6d6b561640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564697264090 0x5646972e7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:17.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:17.001+0000 7f6d6b561640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564697264090 0x5646972e7350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:55252/0 (socket says 172.21.15.38:55252) 2024-09-21T03:40:17.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:17.001+0000 7f6d6b561640 1 -- 172.21.15.38:0/1197019434 learned_addr learned my addr 172.21.15.38:0/1197019434 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:17.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:17.002+0000 7f6d6c563640 1 --2- 172.21.15.38:0/1197019434 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5646972d9520 0x564697249af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:17.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:17.002+0000 7f6d6bd62640 1 --2- 172.21.15.38:0/1197019434 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56469721d240 0x5646972c5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:17.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:17.002+0000 7f6d6b561640 1 -- 172.21.15.38:0/1197019434 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56469721d240 msgr2=0x5646972c5a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:17.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:17.002+0000 7f6d6b561640 1 --2- 172.21.15.38:0/1197019434 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56469721d240 0x5646972c5a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:17.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:17.002+0000 7f6d6b561640 1 -- 172.21.15.38:0/1197019434 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5646972d9520 msgr2=0x564697249af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:17.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:17.002+0000 7f6d6b561640 1 --2- 172.21.15.38:0/1197019434 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5646972d9520 0x564697249af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:17.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:17.002+0000 7f6d6b561640 1 -- 172.21.15.38:0/1197019434 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56469738d150 con 0x564697264090 2024-09-21T03:40:17.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:17.002+0000 7f6d6b561640 1 --2- 172.21.15.38:0/1197019434 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564697264090 0x5646972e7350 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f6d600086f0 tx=0x7f6d600087e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:17.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:17.002+0000 7f6d6955d640 1 -- 172.21.15.38:0/1197019434 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d6000a850 con 0x564697264090 2024-09-21T03:40:17.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:17.002+0000 7f6d6955d640 1 -- 172.21.15.38:0/1197019434 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6d6000a9f0 con 0x564697264090 2024-09-21T03:40:17.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:17.002+0000 7f6d72d1df40 1 -- 172.21.15.38:0/1197019434 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56469738d430 con 0x564697264090 2024-09-21T03:40:17.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:17.002+0000 7f6d6955d640 1 -- 172.21.15.38:0/1197019434 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d6000ae00 con 0x564697264090 2024-09-21T03:40:17.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:17.002+0000 7f6d72d1df40 1 -- 172.21.15.38:0/1197019434 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646972e7890 con 0x564697264090 2024-09-21T03:40:17.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:17.004+0000 7f6d6955d640 1 -- 172.21.15.38:0/1197019434 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f6d60016430 con 0x564697264090 2024-09-21T03:40:17.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:17.004+0000 7f6d6955d640 1 --2- 172.21.15.38:0/1197019434 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6d58077570 0x7f6d58079a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:17.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:17.004+0000 7f6d6955d640 1 -- 172.21.15.38:0/1197019434 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 6390+0+0 (secure 0 0 0) 0x7f6d6009a4a0 con 0x564697264090 2024-09-21T03:40:17.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:17.005+0000 7f6d6bd62640 1 --2- 172.21.15.38:0/1197019434 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6d58077570 0x7f6d58079a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:17.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:17.005+0000 7f6d72d1df40 1 --2- 172.21.15.38:0/1197019434 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x5646972e7bb0 0x5646972e7fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:17.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:17.005+0000 7f6d72d1df40 1 -- 172.21.15.38:0/1197019434 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:1 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e96) -- 0x5646972e8500 con 0x5646972e7bb0 2024-09-21T03:40:17.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:17.005+0000 7f6d6c563640 1 --2- 172.21.15.38:0/1197019434 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x5646972e7bb0 0x5646972e7fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:17.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:17.005+0000 7f6d6bd62640 1 --2- 172.21.15.38:0/1197019434 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6d58077570 0x7f6d58079a80 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f6d5c046640 tx=0x7f6d5c004940 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:17.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:17.005+0000 7f6d6c563640 1 --2- 172.21.15.38:0/1197019434 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x5646972e7bb0 0x5646972e7fc0 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:17.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:17.006+0000 7f6d6c563640 1 -- 172.21.15.38:0/1197019434 <== osd.7 v2:172.21.15.195:6818/1606106342 1 ==== osd_op_reply(1 10000000000.00000000 [delete] v96'42 uv42 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f6d64001540 con 0x5646972e7bb0 2024-09-21T03:40:17.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:17.006+0000 7f6d72d1df40 1 -- 172.21.15.38:0/1197019434 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x5646972e7bb0 msgr2=0x5646972e7fc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:17.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:17.006+0000 7f6d72d1df40 1 --2- 172.21.15.38:0/1197019434 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x5646972e7bb0 0x5646972e7fc0 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:17.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:17.007+0000 7f6d72d1df40 1 -- 172.21.15.38:0/1197019434 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6d58077570 msgr2=0x7f6d58079a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:17.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:17.007+0000 7f6d72d1df40 1 --2- 172.21.15.38:0/1197019434 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6d58077570 0x7f6d58079a80 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f6d5c046640 tx=0x7f6d5c004940 comp rx=0 tx=0).stop 2024-09-21T03:40:17.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:17.007+0000 7f6d72d1df40 1 -- 172.21.15.38:0/1197019434 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564697264090 msgr2=0x5646972e7350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:17.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:17.007+0000 7f6d72d1df40 1 --2- 172.21.15.38:0/1197019434 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564697264090 0x5646972e7350 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f6d600086f0 tx=0x7f6d600087e0 comp rx=0 tx=0).stop 2024-09-21T03:40:17.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:17.007+0000 7f6d6c563640 1 -- 172.21.15.38:0/1197019434 reap_dead start 2024-09-21T03:40:17.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:17.007+0000 7f6d72d1df40 1 -- 172.21.15.38:0/1197019434 shutdown_connections 2024-09-21T03:40:17.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:17.007+0000 7f6d72d1df40 1 -- 172.21.15.38:0/1197019434 >> 172.21.15.38:0/1197019434 conn(0x56469713bac0 msgr2=0x5646972469f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:17.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:17.007+0000 7f6d72d1df40 1 -- 172.21.15.38:0/1197019434 shutdown_connections 2024-09-21T03:40:17.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:17.007+0000 7f6d72d1df40 1 -- 172.21.15.38:0/1197019434 wait complete. 2024-09-21T03:40:17.012 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-21T03:40:17.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:17.129+0000 7f389d5e6640 1 -- 172.21.15.38:0/318833812 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f38980d0170 msgr2=0x7f38980d0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:17.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:17.129+0000 7f389d5e6640 1 --2- 172.21.15.38:0/318833812 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f38980d0170 0x7f38980d0be0 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f388c0098e0 tx=0x7f388c02a4a0 comp rx=0 tx=0).stop 2024-09-21T03:40:17.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:17.129+0000 7f389d5e6640 1 -- 172.21.15.38:0/318833812 shutdown_connections 2024-09-21T03:40:17.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:17.129+0000 7f389d5e6640 1 --2- 172.21.15.38:0/318833812 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f38980d1120 0x7f38980ceb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:17.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:17.129+0000 7f389d5e6640 1 --2- 172.21.15.38:0/318833812 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f38980d0170 0x7f38980d0be0 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:17.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:17.129+0000 7f389d5e6640 1 -- 172.21.15.38:0/318833812 >> 172.21.15.38:0/318833812 conn(0x7f389809d830 msgr2=0x7f38980b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:17.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:17.129+0000 7f389d5e6640 1 -- 172.21.15.38:0/318833812 shutdown_connections 2024-09-21T03:40:17.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:17.129+0000 7f389d5e6640 1 -- 172.21.15.38:0/318833812 wait complete. 2024-09-21T03:40:17.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:17.131+0000 7f389d5e6640 1 Processor -- start 2024-09-21T03:40:17.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:17.131+0000 7f389d5e6640 1 -- start start 2024-09-21T03:40:17.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:17.132+0000 7f389d5e6640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f38980d0170 0x7f38981f9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:17.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:17.132+0000 7f389d5e6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f38980d1120 0x7f38981505c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:17.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:17.132+0000 7f389d5e6640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f389815ee80 0x7f38981f9130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:17.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:17.132+0000 7f389d5e6640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f38980b6700 con 0x7f38980d1120 2024-09-21T03:40:17.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:17.132+0000 7f389d5e6640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f38980b6580 con 0x7f389815ee80 2024-09-21T03:40:17.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:17.132+0000 7f389d5e6640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f38980b6880 con 0x7f38980d0170 2024-09-21T03:40:17.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:17.133+0000 7f38967fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f38980d1120 0x7f38981505c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:17.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:17.133+0000 7f3896ffd640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f38980d0170 0x7f38981f9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:17.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:17.133+0000 7f38967fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f38980d1120 0x7f38981505c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:55280/0 (socket says 172.21.15.38:55280) 2024-09-21T03:40:17.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:17.133+0000 7f38967fc640 1 -- 172.21.15.38:0/3670691061 learned_addr learned my addr 172.21.15.38:0/3670691061 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:17.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:17.133+0000 7f38977fe640 1 --2- 172.21.15.38:0/3670691061 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f389815ee80 0x7f38981f9130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:17.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:17.133+0000 7f38967fc640 1 -- 172.21.15.38:0/3670691061 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f38980d0170 msgr2=0x7f38981f9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:17.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:17.133+0000 7f38967fc640 1 --2- 172.21.15.38:0/3670691061 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f38980d0170 0x7f38981f9cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:17.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:17.133+0000 7f38967fc640 1 -- 172.21.15.38:0/3670691061 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f389815ee80 msgr2=0x7f38981f9130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:17.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:17.133+0000 7f38967fc640 1 --2- 172.21.15.38:0/3670691061 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f389815ee80 0x7f38981f9130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:17.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:17.133+0000 7f38967fc640 1 -- 172.21.15.38:0/3670691061 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f389815add0 con 0x7f38980d1120 2024-09-21T03:40:17.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:17.133+0000 7f3896ffd640 1 --2- 172.21.15.38:0/3670691061 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f38980d0170 0x7f38981f9cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:40:17.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:17.133+0000 7f38967fc640 1 --2- 172.21.15.38:0/3670691061 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f38980d1120 0x7f38981505c0 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f388c009a00 tx=0x7f388c0052f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:17.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:17.133+0000 7f387f7fe640 1 -- 172.21.15.38:0/3670691061 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f388c031e60 con 0x7f38980d1120 2024-09-21T03:40:17.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:17.133+0000 7f387f7fe640 1 -- 172.21.15.38:0/3670691061 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f388c004a90 con 0x7f38980d1120 2024-09-21T03:40:17.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:17.133+0000 7f389d5e6640 1 -- 172.21.15.38:0/3670691061 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3898150b00 con 0x7f38980d1120 2024-09-21T03:40:17.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:17.133+0000 7f387f7fe640 1 -- 172.21.15.38:0/3670691061 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f388c004ea0 con 0x7f38980d1120 2024-09-21T03:40:17.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:17.133+0000 7f389d5e6640 1 -- 172.21.15.38:0/3670691061 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f389815a780 con 0x7f38980d1120 2024-09-21T03:40:17.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:17.134+0000 7f389d5e6640 1 -- 172.21.15.38:0/3670691061 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f389815b950 con 0x7f38980d1120 2024-09-21T03:40:17.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:17.135+0000 7f387f7fe640 1 -- 172.21.15.38:0/3670691061 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f388c03b410 con 0x7f38980d1120 2024-09-21T03:40:17.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:17.136+0000 7f387f7fe640 1 --2- 172.21.15.38:0/3670691061 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3884078b90 0x7f388407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:17.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:17.136+0000 7f387f7fe640 1 -- 172.21.15.38:0/3670691061 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 6390+0+0 (secure 0 0 0) 0x7f388c0bc350 con 0x7f38980d1120 2024-09-21T03:40:17.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:17.136+0000 7f3896ffd640 1 --2- 172.21.15.38:0/3670691061 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3884078b90 0x7f388407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:17.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:17.136+0000 7f3896ffd640 1 --2- 172.21.15.38:0/3670691061 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3884078b90 0x7f388407b0a0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f3888006fd0 tx=0x7f3888009060 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:17.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:17.140+0000 7f387f7fe640 1 -- 172.21.15.38:0/3670691061 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f388c03f030 con 0x7f38980d1120 2024-09-21T03:40:17.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:17.368+0000 7f389d5e6640 1 -- 172.21.15.38:0/3670691061 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f38980b6dc0 con 0x7f38980d1120 2024-09-21T03:40:17.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:17.669+0000 7f387f7fe640 1 -- 172.21.15.38:0/3670691061 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v88) ==== 162+0+0 (secure 0 0 0) 0x7f38981592d0 con 0x7f38980d1120 2024-09-21T03:40:17.671 INFO:teuthology.orchestra.run.smithi038.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-21T03:40:17.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:17.671+0000 7f389d5e6640 1 -- 172.21.15.38:0/3670691061 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3884078b90 msgr2=0x7f388407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:17.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:17.671+0000 7f389d5e6640 1 --2- 172.21.15.38:0/3670691061 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3884078b90 0x7f388407b0a0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f3888006fd0 tx=0x7f3888009060 comp rx=0 tx=0).stop 2024-09-21T03:40:17.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:17.671+0000 7f389d5e6640 1 -- 172.21.15.38:0/3670691061 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f38980d1120 msgr2=0x7f38981505c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:17.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:17.671+0000 7f389d5e6640 1 --2- 172.21.15.38:0/3670691061 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f38980d1120 0x7f38981505c0 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f388c009a00 tx=0x7f388c0052f0 comp rx=0 tx=0).stop 2024-09-21T03:40:17.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:17.671+0000 7f389d5e6640 1 -- 172.21.15.38:0/3670691061 shutdown_connections 2024-09-21T03:40:17.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:17.671+0000 7f389d5e6640 1 --2- 172.21.15.38:0/3670691061 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f389815ee80 0x7f38981f9130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:17.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:17.671+0000 7f389d5e6640 1 --2- 172.21.15.38:0/3670691061 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f38980d1120 0x7f38981505c0 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:17.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:17.671+0000 7f389d5e6640 1 --2- 172.21.15.38:0/3670691061 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3884078b90 0x7f388407b0a0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:17.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:17.671+0000 7f389d5e6640 1 --2- 172.21.15.38:0/3670691061 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f38980d0170 0x7f38981f9cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:17.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:17.671+0000 7f389d5e6640 1 -- 172.21.15.38:0/3670691061 >> 172.21.15.38:0/3670691061 conn(0x7f389809d830 msgr2=0x7f38980d09e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:17.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:17.671+0000 7f389d5e6640 1 -- 172.21.15.38:0/3670691061 shutdown_connections 2024-09-21T03:40:17.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:17.671+0000 7f389d5e6640 1 -- 172.21.15.38:0/3670691061 wait complete. 2024-09-21T03:40:18.093 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:40:18.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:18.219+0000 7fec830bd640 1 -- 172.21.15.38:0/339854035 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fec7c0cfe00 msgr2=0x7fec7c0d2a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:18.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:18.219+0000 7fec830bd640 1 --2- 172.21.15.38:0/339854035 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fec7c0cfe00 0x7fec7c0d2a20 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7fec7c0b5a40 tx=0x7fec6c02a560 comp rx=0 tx=0).stop 2024-09-21T03:40:18.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:18.219+0000 7fec830bd640 1 -- 172.21.15.38:0/339854035 shutdown_connections 2024-09-21T03:40:18.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:18.219+0000 7fec830bd640 1 --2- 172.21.15.38:0/339854035 >> v2:172.21.15.195:3300/0 conn(0x7fec7c0b2e60 0x7fec7c0b3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:18.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:18.219+0000 7fec830bd640 1 --2- 172.21.15.38:0/339854035 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fec7c0cfe00 0x7fec7c0d2a20 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:18.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:18.219+0000 7fec830bd640 1 -- 172.21.15.38:0/339854035 >> 172.21.15.38:0/339854035 conn(0x7fec7c0d4440 msgr2=0x7fec7c0d4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:18.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:18.219+0000 7fec830bd640 1 -- 172.21.15.38:0/339854035 shutdown_connections 2024-09-21T03:40:18.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:18.219+0000 7fec830bd640 1 -- 172.21.15.38:0/339854035 wait complete. 2024-09-21T03:40:18.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:18.221+0000 7fec830bd640 1 Processor -- start 2024-09-21T03:40:18.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:18.221+0000 7fec830bd640 1 -- start start 2024-09-21T03:40:18.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:18.222+0000 7fec830bd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fec7c0b2e60 0x7fec7c151860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:18.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:18.222+0000 7fec830bd640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fec7c0cfe00 0x7fec7c1f99b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:18.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:18.222+0000 7fec830bd640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fec7c162590 0x7fec7c155ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:18.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:18.222+0000 7fec830bd640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fec7c0b3930 con 0x7fec7c0b2e60 2024-09-21T03:40:18.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:18.222+0000 7fec830bd640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fec7c0b37b0 con 0x7fec7c0cfe00 2024-09-21T03:40:18.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:18.222+0000 7fec830bd640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fec7c0b3ab0 con 0x7fec7c162590 2024-09-21T03:40:18.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:18.222+0000 7fec80e59640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fec7c0b2e60 0x7fec7c151860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:18.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:18.222+0000 7fec80e59640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fec7c0b2e60 0x7fec7c151860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:55302/0 (socket says 172.21.15.38:55302) 2024-09-21T03:40:18.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:18.222+0000 7fec80e59640 1 -- 172.21.15.38:0/127568868 learned_addr learned my addr 172.21.15.38:0/127568868 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:18.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:18.222+0000 7fec7bfff640 1 --2- 172.21.15.38:0/127568868 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fec7c0cfe00 0x7fec7c1f99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:18.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:18.222+0000 7fec8165a640 1 --2- 172.21.15.38:0/127568868 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fec7c162590 0x7fec7c155ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:18.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:18.223+0000 7fec80e59640 1 -- 172.21.15.38:0/127568868 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fec7c162590 msgr2=0x7fec7c155ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:18.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:18.223+0000 7fec80e59640 1 --2- 172.21.15.38:0/127568868 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fec7c162590 0x7fec7c155ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:18.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:18.223+0000 7fec80e59640 1 -- 172.21.15.38:0/127568868 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fec7c0cfe00 msgr2=0x7fec7c1f99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:18.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:18.223+0000 7fec80e59640 1 --2- 172.21.15.38:0/127568868 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fec7c0cfe00 0x7fec7c1f99b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:18.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:18.223+0000 7fec80e59640 1 -- 172.21.15.38:0/127568868 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec7c1f59e0 con 0x7fec7c0b2e60 2024-09-21T03:40:18.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:18.223+0000 7fec8165a640 1 --2- 172.21.15.38:0/127568868 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fec7c162590 0x7fec7c155ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:40:18.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:18.223+0000 7fec80e59640 1 --2- 172.21.15.38:0/127568868 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fec7c0b2e60 0x7fec7c151860 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7fec6c034040 tx=0x7fec6c003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:18.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:18.223+0000 7fec797fa640 1 -- 172.21.15.38:0/127568868 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec6c03e030 con 0x7fec7c0b2e60 2024-09-21T03:40:18.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:18.223+0000 7fec797fa640 1 -- 172.21.15.38:0/127568868 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fec6c003e00 con 0x7fec7c0b2e60 2024-09-21T03:40:18.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:18.223+0000 7fec830bd640 1 -- 172.21.15.38:0/127568868 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec7c1f5c50 con 0x7fec7c0b2e60 2024-09-21T03:40:18.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:18.223+0000 7fec797fa640 1 -- 172.21.15.38:0/127568868 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec6c035040 con 0x7fec7c0b2e60 2024-09-21T03:40:18.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:18.223+0000 7fec830bd640 1 -- 172.21.15.38:0/127568868 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec7c1f6210 con 0x7fec7c0b2e60 2024-09-21T03:40:18.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:18.224+0000 7fec830bd640 1 -- 172.21.15.38:0/127568868 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec7c0b3270 con 0x7fec7c0b2e60 2024-09-21T03:40:18.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:18.225+0000 7fec797fa640 1 -- 172.21.15.38:0/127568868 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fec6c041030 con 0x7fec7c0b2e60 2024-09-21T03:40:18.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:18.225+0000 7fec797fa640 1 --2- 172.21.15.38:0/127568868 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fec68078bc0 0x7fec6807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:18.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:18.226+0000 7fec797fa640 1 -- 172.21.15.38:0/127568868 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 6390+0+0 (secure 0 0 0) 0x7fec6c0bb870 con 0x7fec7c0b2e60 2024-09-21T03:40:18.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:18.226+0000 7fec7bfff640 1 --2- 172.21.15.38:0/127568868 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fec68078bc0 0x7fec6807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:18.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:18.226+0000 7fec7bfff640 1 --2- 172.21.15.38:0/127568868 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fec68078bc0 0x7fec6807b0d0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fec7c155880 tx=0x7fec70009060 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:18.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:18.230+0000 7fec797fa640 1 -- 172.21.15.38:0/127568868 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fec6c083290 con 0x7fec7c0b2e60 2024-09-21T03:40:18.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:18.458+0000 7fec830bd640 1 -- 172.21.15.38:0/127568868 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fec7c0d1c90 con 0x7fec7c0b2e60 2024-09-21T03:40:18.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:18.459+0000 7fec797fa640 1 -- 172.21.15.38:0/127568868 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 89 v89) ==== 94+0+5219 (secure 0 0 0) 0x7fec6c087cc0 con 0x7fec7c0b2e60 2024-09-21T03:40:18.461 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:40:18.461 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":89,"btime":"2024-09-21T03:40:17:683879+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6086,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/965972663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":965972663},{"type":"v1","addr":"172.21.15.195:6836","nonce":965972663}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6700,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/3176122334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":3176122334},{"type":"v1","addr":"172.21.15.195:6837","nonce":3176122334}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":7020,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/751618334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":751618334},{"type":"v1","addr":"172.21.15.38:6833","nonce":751618334}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":89,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:40:02.319282+0000","modified":"2024-09-21T03:40:17.683859+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":95,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7023},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7023":{"gid":7023,"name":"a","rank":0,"incarnation":89,"state":"up:replay","state_seq":1,"addr":"172.21.15.38:6835/2934569248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":2934569248},{"type":"v1","addr":"172.21.15.38:6835","nonce":2934569248}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-21T03:40:18.461 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 89 2024-09-21T03:40:18.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:18.461+0000 7fec830bd640 1 -- 172.21.15.38:0/127568868 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fec68078bc0 msgr2=0x7fec6807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:18.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:18.461+0000 7fec830bd640 1 --2- 172.21.15.38:0/127568868 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fec68078bc0 0x7fec6807b0d0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fec7c155880 tx=0x7fec70009060 comp rx=0 tx=0).stop 2024-09-21T03:40:18.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:18.461+0000 7fec830bd640 1 -- 172.21.15.38:0/127568868 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fec7c0b2e60 msgr2=0x7fec7c151860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:18.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:18.461+0000 7fec830bd640 1 --2- 172.21.15.38:0/127568868 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fec7c0b2e60 0x7fec7c151860 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7fec6c034040 tx=0x7fec6c003910 comp rx=0 tx=0).stop 2024-09-21T03:40:18.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:18.461+0000 7fec830bd640 1 -- 172.21.15.38:0/127568868 shutdown_connections 2024-09-21T03:40:18.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:18.461+0000 7fec830bd640 1 --2- 172.21.15.38:0/127568868 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fec68078bc0 0x7fec6807b0d0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:18.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:18.461+0000 7fec830bd640 1 --2- 172.21.15.38:0/127568868 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fec7c162590 0x7fec7c155ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:18.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:18.461+0000 7fec830bd640 1 --2- 172.21.15.38:0/127568868 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fec7c0cfe00 0x7fec7c1f99b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:18.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:18.461+0000 7fec830bd640 1 --2- 172.21.15.38:0/127568868 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fec7c0b2e60 0x7fec7c151860 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:18.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:18.461+0000 7fec830bd640 1 -- 172.21.15.38:0/127568868 >> 172.21.15.38:0/127568868 conn(0x7fec7c0d4440 msgr2=0x7fec7c06e340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:18.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:18.461+0000 7fec830bd640 1 -- 172.21.15.38:0/127568868 shutdown_connections 2024-09-21T03:40:18.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:18.461+0000 7fec830bd640 1 -- 172.21.15.38:0/127568868 wait complete. 2024-09-21T03:40:18.474 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 89, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T03:40:02.319282+0000', 'modified': '2024-09-21T03:40:17.683859+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 95, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7023}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7023': {'gid': 7023, 'name': 'a', 'rank': 0, 'incarnation': 89, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.38:6835/2934569248', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.38:6834', 'nonce': 2934569248}, {'type': 'v1', 'addr': '172.21.15.38:6835', 'nonce': 2934569248}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-09-21T03:40:18.474 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7023:up:replay 2024-09-21T03:40:18.475 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2024-09-21T03:40:23.879 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:40:24.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.009+0000 7fdb3f357640 1 -- 172.21.15.38:0/2666226669 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdb38162280 msgr2=0x7fdb38162690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:24.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.009+0000 7fdb3f357640 1 --2- 172.21.15.38:0/2666226669 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdb38162280 0x7fdb38162690 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7fdb28005fd0 tx=0x7fdb2802a5d0 comp rx=0 tx=0).stop 2024-09-21T03:40:24.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.009+0000 7fdb3f357640 1 -- 172.21.15.38:0/2666226669 shutdown_connections 2024-09-21T03:40:24.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.009+0000 7fdb3f357640 1 --2- 172.21.15.38:0/2666226669 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdb380b2e40 0x7fdb380b3250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:24.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.009+0000 7fdb3f357640 1 --2- 172.21.15.38:0/2666226669 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdb38162280 0x7fdb38162690 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:24.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.009+0000 7fdb3f357640 1 -- 172.21.15.38:0/2666226669 >> 172.21.15.38:0/2666226669 conn(0x7fdb380b6b20 msgr2=0x7fdb380d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:24.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.009+0000 7fdb3f357640 1 -- 172.21.15.38:0/2666226669 shutdown_connections 2024-09-21T03:40:24.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.009+0000 7fdb3f357640 1 -- 172.21.15.38:0/2666226669 wait complete. 2024-09-21T03:40:24.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.011+0000 7fdb3f357640 1 Processor -- start 2024-09-21T03:40:24.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.011+0000 7fdb3f357640 1 -- start start 2024-09-21T03:40:24.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.012+0000 7fdb3f357640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdb380b2e40 0x7fdb380b11d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:24.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.013+0000 7fdb3f357640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdb38162280 0x7fdb3814edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:24.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.013+0000 7fdb3f357640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdb3814cbd0 0x7fdb3814f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:24.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.013+0000 7fdb3f357640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fdb3815e4b0 con 0x7fdb3814cbd0 2024-09-21T03:40:24.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.013+0000 7fdb3f357640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fdb380b39d0 con 0x7fdb38162280 2024-09-21T03:40:24.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.013+0000 7fdb3f357640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fdb3815e630 con 0x7fdb380b2e40 2024-09-21T03:40:24.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.013+0000 7fdb3d8f4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdb3814cbd0 0x7fdb3814f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:24.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.013+0000 7fdb3d0f3640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdb380b2e40 0x7fdb380b11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:24.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.013+0000 7fdb3c8f2640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdb38162280 0x7fdb3814edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:24.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.013+0000 7fdb3d8f4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdb3814cbd0 0x7fdb3814f330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48488/0 (socket says 172.21.15.38:48488) 2024-09-21T03:40:24.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.013+0000 7fdb3d0f3640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdb380b2e40 0x7fdb380b11d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.38:55588/0 (socket says 172.21.15.38:55588) 2024-09-21T03:40:24.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.013+0000 7fdb3d8f4640 1 -- 172.21.15.38:0/4109181132 learned_addr learned my addr 172.21.15.38:0/4109181132 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:24.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.013+0000 7fdb3d8f4640 1 -- 172.21.15.38:0/4109181132 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdb380b2e40 msgr2=0x7fdb380b11d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:24.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.013+0000 7fdb3d8f4640 1 --2- 172.21.15.38:0/4109181132 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdb380b2e40 0x7fdb380b11d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:24.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.013+0000 7fdb3d8f4640 1 -- 172.21.15.38:0/4109181132 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdb38162280 msgr2=0x7fdb3814edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:24.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.013+0000 7fdb3d8f4640 1 --2- 172.21.15.38:0/4109181132 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdb38162280 0x7fdb3814edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:24.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.013+0000 7fdb3d8f4640 1 -- 172.21.15.38:0/4109181132 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb380abf40 con 0x7fdb3814cbd0 2024-09-21T03:40:24.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.013+0000 7fdb3d8f4640 1 --2- 172.21.15.38:0/4109181132 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdb3814cbd0 0x7fdb3814f330 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7fdb3000e990 tx=0x7fdb3000ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:24.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.013+0000 7fdb2dffb640 1 -- 172.21.15.38:0/4109181132 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb30004d20 con 0x7fdb3814cbd0 2024-09-21T03:40:24.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.014+0000 7fdb2dffb640 1 -- 172.21.15.38:0/4109181132 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdb3000caa0 con 0x7fdb3814cbd0 2024-09-21T03:40:24.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.014+0000 7fdb3f357640 1 -- 172.21.15.38:0/4109181132 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb380ac220 con 0x7fdb3814cbd0 2024-09-21T03:40:24.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.014+0000 7fdb2dffb640 1 -- 172.21.15.38:0/4109181132 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb30005020 con 0x7fdb3814cbd0 2024-09-21T03:40:24.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.014+0000 7fdb3f357640 1 -- 172.21.15.38:0/4109181132 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb3814d010 con 0x7fdb3814cbd0 2024-09-21T03:40:24.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.015+0000 7fdb3f357640 1 -- 172.21.15.38:0/4109181132 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb3814f900 con 0x7fdb3814cbd0 2024-09-21T03:40:24.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.016+0000 7fdb2dffb640 1 -- 172.21.15.38:0/4109181132 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fdb300051c0 con 0x7fdb3814cbd0 2024-09-21T03:40:24.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.016+0000 7fdb2dffb640 1 --2- 172.21.15.38:0/4109181132 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fdb1c078b90 0x7fdb1c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:24.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.016+0000 7fdb2dffb640 1 -- 172.21.15.38:0/4109181132 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 6390+0+0 (secure 0 0 0) 0x7fdb3009b760 con 0x7fdb3814cbd0 2024-09-21T03:40:24.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.017+0000 7fdb3d0f3640 1 --2- 172.21.15.38:0/4109181132 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fdb1c078b90 0x7fdb1c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:24.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.017+0000 7fdb3d0f3640 1 --2- 172.21.15.38:0/4109181132 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fdb1c078b90 0x7fdb1c07b0a0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fdb34001720 tx=0x7fdb34009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:24.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.021+0000 7fdb2dffb640 1 -- 172.21.15.38:0/4109181132 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fdb30016030 con 0x7fdb3814cbd0 2024-09-21T03:40:24.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.246+0000 7fdb3f357640 1 -- 172.21.15.38:0/4109181132 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdb38162f00 con 0x7fdb3814cbd0 2024-09-21T03:40:24.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.247+0000 7fdb2dffb640 1 -- 172.21.15.38:0/4109181132 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 92 v92) ==== 94+0+5226 (secure 0 0 0) 0x7fdb300633c0 con 0x7fdb3814cbd0 2024-09-21T03:40:24.249 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:40:24.249 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":92,"btime":"2024-09-21T03:40:20:697046+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6086,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/965972663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":965972663},{"type":"v1","addr":"172.21.15.195:6836","nonce":965972663}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6700,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/3176122334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":3176122334},{"type":"v1","addr":"172.21.15.195:6837","nonce":3176122334}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":7020,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/751618334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":751618334},{"type":"v1","addr":"172.21.15.38:6833","nonce":751618334}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":92,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:40:02.319282+0000","modified":"2024-09-21T03:40:20.697040+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":95,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7023},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7023":{"gid":7023,"name":"a","rank":0,"incarnation":89,"state":"up:active","state_seq":8,"addr":"172.21.15.38:6835/2934569248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":2934569248},{"type":"v1","addr":"172.21.15.38:6835","nonce":2934569248}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7023,"qdb_cluster":[7023]},"id":6}]} 2024-09-21T03:40:24.249 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 92 2024-09-21T03:40:24.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.249+0000 7fdb3f357640 1 -- 172.21.15.38:0/4109181132 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fdb1c078b90 msgr2=0x7fdb1c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:24.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.249+0000 7fdb3f357640 1 --2- 172.21.15.38:0/4109181132 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fdb1c078b90 0x7fdb1c07b0a0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fdb34001720 tx=0x7fdb34009040 comp rx=0 tx=0).stop 2024-09-21T03:40:24.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.249+0000 7fdb3f357640 1 -- 172.21.15.38:0/4109181132 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdb3814cbd0 msgr2=0x7fdb3814f330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:24.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.249+0000 7fdb3f357640 1 --2- 172.21.15.38:0/4109181132 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdb3814cbd0 0x7fdb3814f330 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7fdb3000e990 tx=0x7fdb3000ee50 comp rx=0 tx=0).stop 2024-09-21T03:40:24.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.249+0000 7fdb3f357640 1 -- 172.21.15.38:0/4109181132 shutdown_connections 2024-09-21T03:40:24.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.249+0000 7fdb3f357640 1 --2- 172.21.15.38:0/4109181132 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdb3814cbd0 0x7fdb3814f330 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:24.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.249+0000 7fdb3f357640 1 --2- 172.21.15.38:0/4109181132 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdb38162280 0x7fdb3814edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:24.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.249+0000 7fdb3f357640 1 --2- 172.21.15.38:0/4109181132 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fdb1c078b90 0x7fdb1c07b0a0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:24.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.249+0000 7fdb3f357640 1 --2- 172.21.15.38:0/4109181132 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdb380b2e40 0x7fdb380b11d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:24.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.249+0000 7fdb3f357640 1 -- 172.21.15.38:0/4109181132 >> 172.21.15.38:0/4109181132 conn(0x7fdb380b6b20 msgr2=0x7fdb3815eaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:24.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.250+0000 7fdb3f357640 1 -- 172.21.15.38:0/4109181132 shutdown_connections 2024-09-21T03:40:24.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.250+0000 7fdb3f357640 1 -- 172.21.15.38:0/4109181132 wait complete. 2024-09-21T03:40:24.263 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 92, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T03:40:02.319282+0000', 'modified': '2024-09-21T03:40:20.697040+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 95, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7023}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7023': {'gid': 7023, 'name': 'a', 'rank': 0, 'incarnation': 89, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.38:6835/2934569248', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.38:6834', 'nonce': 2934569248}, {'type': 'v1', 'addr': '172.21.15.38:6835', 'nonce': 2934569248}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7023, 'qdb_cluster': [7023]} 2024-09-21T03:40:24.263 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-21T03:40:24.263 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-21T03:40:24.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.380+0000 7f3a17c07640 1 -- 172.21.15.38:0/3540598940 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f39f8004800 msgr2=0x7f39f8004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:24.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.380+0000 7f3a17c07640 1 --2- 172.21.15.38:0/3540598940 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f39f8004800 0x7f39f8004c10 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f3a100ca850 tx=0x7f39f8005830 comp rx=0 tx=0).stop 2024-09-21T03:40:24.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.380+0000 7f3a17c07640 1 -- 172.21.15.38:0/3540598940 shutdown_connections 2024-09-21T03:40:24.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.380+0000 7f3a17c07640 1 --2- 172.21.15.38:0/3540598940 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f39f8004800 0x7f39f8004c10 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:24.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.380+0000 7f3a17c07640 1 -- 172.21.15.38:0/3540598940 >> 172.21.15.38:0/3540598940 conn(0x7f3a100b1b90 msgr2=0x7f3a100b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:24.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.380+0000 7f3a17c07640 1 -- 172.21.15.38:0/3540598940 shutdown_connections 2024-09-21T03:40:24.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.380+0000 7f3a17c07640 1 -- 172.21.15.38:0/3540598940 wait complete. 2024-09-21T03:40:24.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.382+0000 7f3a17c07640 1 Processor -- start 2024-09-21T03:40:24.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.382+0000 7f3a17c07640 1 -- start start 2024-09-21T03:40:24.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.383+0000 7f3a17c07640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f39f8004800 0x7f3a100a0380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:24.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.383+0000 7f3a17c07640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3a100b5af0 0x7f3a100a08c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:24.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.383+0000 7f3a17c07640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3a1015a940 0x7f3a101f3320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:24.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.383+0000 7f3a17c07640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f3a10070d20 con 0x7f3a100b5af0 2024-09-21T03:40:24.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.383+0000 7f3a17c07640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f3a10070ba0 con 0x7f3a1015a940 2024-09-21T03:40:24.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.383+0000 7f3a17c07640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f3a100cf290 con 0x7f39f8004800 2024-09-21T03:40:24.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.383+0000 7f3a151a2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3a100b5af0 0x7f3a100a08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:24.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.383+0000 7f3a151a2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3a100b5af0 0x7f3a100a08c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48514/0 (socket says 172.21.15.38:48514) 2024-09-21T03:40:24.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.383+0000 7f3a151a2640 1 -- 172.21.15.38:0/2550813462 learned_addr learned my addr 172.21.15.38:0/2550813462 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:24.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.383+0000 7f3a159a3640 1 --2- 172.21.15.38:0/2550813462 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f39f8004800 0x7f3a100a0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:24.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.383+0000 7f3a161a4640 1 --2- 172.21.15.38:0/2550813462 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3a1015a940 0x7f3a101f3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:24.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.383+0000 7f3a151a2640 1 -- 172.21.15.38:0/2550813462 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f39f8004800 msgr2=0x7f3a100a0380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:24.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.383+0000 7f3a151a2640 1 --2- 172.21.15.38:0/2550813462 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f39f8004800 0x7f3a100a0380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:24.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.383+0000 7f3a151a2640 1 -- 172.21.15.38:0/2550813462 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3a1015a940 msgr2=0x7f3a101f3320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:24.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.383+0000 7f3a151a2640 1 --2- 172.21.15.38:0/2550813462 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3a1015a940 0x7f3a101f3320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:24.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.383+0000 7f3a151a2640 1 -- 172.21.15.38:0/2550813462 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a1015ed50 con 0x7f3a100b5af0 2024-09-21T03:40:24.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.384+0000 7f3a151a2640 1 --2- 172.21.15.38:0/2550813462 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3a100b5af0 0x7f3a100a08c0 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f3a04034040 tx=0x7f3a0400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:24.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.384+0000 7f3a0e7fc640 1 -- 172.21.15.38:0/2550813462 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a04035040 con 0x7f3a100b5af0 2024-09-21T03:40:24.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.384+0000 7f3a0e7fc640 1 -- 172.21.15.38:0/2550813462 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3a0403a9e0 con 0x7f3a100b5af0 2024-09-21T03:40:24.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.384+0000 7f3a17c07640 1 -- 172.21.15.38:0/2550813462 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a1015be20 con 0x7f3a100b5af0 2024-09-21T03:40:24.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.384+0000 7f3a0e7fc640 1 -- 172.21.15.38:0/2550813462 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a0403add0 con 0x7f3a100b5af0 2024-09-21T03:40:24.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.384+0000 7f3a17c07640 1 -- 172.21.15.38:0/2550813462 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a100cb3e0 con 0x7f3a100b5af0 2024-09-21T03:40:24.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.386+0000 7f3a0e7fc640 1 -- 172.21.15.38:0/2550813462 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f3a04038020 con 0x7f3a100b5af0 2024-09-21T03:40:24.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.387+0000 7f3a0e7fc640 1 --2- 172.21.15.38:0/2550813462 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f39fc078bc0 0x7f39fc07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:24.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.387+0000 7f3a0e7fc640 1 -- 172.21.15.38:0/2550813462 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 6390+0+0 (secure 0 0 0) 0x7f3a040bc250 con 0x7f3a100b5af0 2024-09-21T03:40:24.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.387+0000 7f3a159a3640 1 --2- 172.21.15.38:0/2550813462 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f39fc078bc0 0x7f39fc07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:24.390 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.388+0000 7f3a1ec4c740 1 Processor -- start 2024-09-21T03:40:24.390 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.388+0000 7f3a1ec4c740 1 -- start start 2024-09-21T03:40:24.390 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.388+0000 7f3a159a3640 1 --2- 172.21.15.38:0/2550813462 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f39fc078bc0 0x7f39fc07b0d0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f3a000013d0 tx=0x7f3a00006b20 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:24.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.389+0000 7f3a1ec4c740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e653460270 0x55e6538363d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:24.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.389+0000 7f3a1ec4c740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e6534373b0 0x55e653836910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:24.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.389+0000 7f3a1ec4c740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e65343f590 0x55e653853810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:24.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.389+0000 7f3a1ec4c740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55e65377a820 con 0x55e6534373b0 2024-09-21T03:40:24.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.389+0000 7f3a1ec4c740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55e653769c50 con 0x55e653460270 2024-09-21T03:40:24.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.389+0000 7f3a1ec4c740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55e65376da60 con 0x55e65343f590 2024-09-21T03:40:24.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.389+0000 7f3a151a2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e6534373b0 0x55e653836910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:24.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.389+0000 7f3a151a2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e6534373b0 0x55e653836910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48518/0 (socket says 172.21.15.38:48518) 2024-09-21T03:40:24.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.389+0000 7f3a151a2640 1 -- 172.21.15.38:0/693304060 learned_addr learned my addr 172.21.15.38:0/693304060 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:24.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.389+0000 7f3a159a3640 1 --2- 172.21.15.38:0/693304060 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e653460270 0x55e6538363d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:24.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.390+0000 7f3a161a4640 1 --2- 172.21.15.38:0/693304060 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e65343f590 0x55e653853810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:24.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.390+0000 7f3a151a2640 1 -- 172.21.15.38:0/693304060 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e65343f590 msgr2=0x55e653853810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:24.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.390+0000 7f3a151a2640 1 --2- 172.21.15.38:0/693304060 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e65343f590 0x55e653853810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:24.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.390+0000 7f3a151a2640 1 -- 172.21.15.38:0/693304060 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e653460270 msgr2=0x55e6538363d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:24.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.390+0000 7f3a151a2640 1 --2- 172.21.15.38:0/693304060 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e653460270 0x55e6538363d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:24.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.390+0000 7f3a151a2640 1 -- 172.21.15.38:0/693304060 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6534cb850 con 0x55e6534373b0 2024-09-21T03:40:24.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.390+0000 7f3a159a3640 1 --2- 172.21.15.38:0/693304060 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e653460270 0x55e6538363d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:40:24.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.390+0000 7f3a151a2640 1 --2- 172.21.15.38:0/693304060 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e6534373b0 0x55e653836910 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f3a040ba3d0 tx=0x7f3a04083890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6118309f71e116cc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:24.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.390+0000 7f39f77fe640 1 -- 172.21.15.38:0/693304060 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a040887b0 con 0x55e6534373b0 2024-09-21T03:40:24.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.390+0000 7f39f77fe640 1 -- 172.21.15.38:0/693304060 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3a0408d660 con 0x55e6534373b0 2024-09-21T03:40:24.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.390+0000 7f39f77fe640 1 -- 172.21.15.38:0/693304060 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a0408d1d0 con 0x55e6534373b0 2024-09-21T03:40:24.394 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.391+0000 7f3a1ec4c740 1 -- 172.21.15.38:0/693304060 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e6534373b0 msgr2=0x55e653836910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:24.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.391+0000 7f3a1ec4c740 1 --2- 172.21.15.38:0/693304060 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e6534373b0 0x55e653836910 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f3a040ba3d0 tx=0x7f3a04083890 comp rx=0 tx=0).stop 2024-09-21T03:40:24.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.391+0000 7f3a1ec4c740 1 -- 172.21.15.38:0/693304060 shutdown_connections 2024-09-21T03:40:24.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.391+0000 7f3a1ec4c740 1 --2- 172.21.15.38:0/693304060 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e65343f590 0x55e653853810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:24.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.391+0000 7f3a1ec4c740 1 --2- 172.21.15.38:0/693304060 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e6534373b0 0x55e653836910 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:24.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.391+0000 7f3a1ec4c740 1 --2- 172.21.15.38:0/693304060 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e653460270 0x55e6538363d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:24.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.391+0000 7f3a1ec4c740 1 -- 172.21.15.38:0/693304060 >> 172.21.15.38:0/693304060 conn(0x55e6534543c0 msgr2=0x55e65380c690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:24.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.391+0000 7f3a1ec4c740 1 -- 172.21.15.38:0/693304060 shutdown_connections 2024-09-21T03:40:24.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.391+0000 7f3a1ec4c740 1 -- 172.21.15.38:0/693304060 wait complete. 2024-09-21T03:40:24.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.392+0000 7f3a1ec4c740 1 -- start start 2024-09-21T03:40:24.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.392+0000 7f3a1ec4c740 1 Processor -- start 2024-09-21T03:40:24.399 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.393+0000 7f3a1ec4c740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e6534373b0 0x55e653847210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:24.399 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.393+0000 7f3a1ec4c740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e65343f590 0x55e653849e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:24.399 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.393+0000 7f3a1ec4c740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e653460270 0x55e653801550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:24.399 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.394+0000 7f3a1ec4c740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55e6534da620 con 0x55e653460270 2024-09-21T03:40:24.399 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.394+0000 7f3a1ec4c740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55e65370f5f0 con 0x55e65343f590 2024-09-21T03:40:24.399 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.394+0000 7f3a1ec4c740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55e65377a820 con 0x55e6534373b0 2024-09-21T03:40:24.399 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.394+0000 7f3a1ec4c740 10 client.0 _ll_register_callbacks cb 0x55e6537130a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:40:24.399 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.394+0000 7f3a161a4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e653460270 0x55e653801550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:24.399 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.394+0000 7f3a161a4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e653460270 0x55e653801550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48532/0 (socket says 172.21.15.38:48532) 2024-09-21T03:40:24.399 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.394+0000 7f3a161a4640 1 -- 172.21.15.38:0/218598204 learned_addr learned my addr 172.21.15.38:0/218598204 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:24.399 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.394+0000 7f3a151a2640 1 --2- 172.21.15.38:0/218598204 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e65343f590 0x55e653849e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:24.399 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.394+0000 7f3a159a3640 1 --2- 172.21.15.38:0/218598204 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e6534373b0 0x55e653847210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:24.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.394+0000 7f3a161a4640 1 -- 172.21.15.38:0/218598204 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e6534373b0 msgr2=0x55e653847210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:24.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.394+0000 7f3a161a4640 1 --2- 172.21.15.38:0/218598204 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e6534373b0 0x55e653847210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:24.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.394+0000 7f3a161a4640 1 -- 172.21.15.38:0/218598204 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e65343f590 msgr2=0x55e653849e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:24.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.394+0000 7f3a161a4640 1 --2- 172.21.15.38:0/218598204 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e65343f590 0x55e653849e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:24.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.394+0000 7f3a161a4640 1 -- 172.21.15.38:0/218598204 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e6535426b0 con 0x55e653460270 2024-09-21T03:40:24.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.394+0000 7f3a161a4640 1 --2- 172.21.15.38:0/218598204 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e653460270 0x55e653801550 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f39f80057e0 tx=0x7f3a08011430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85ca6d3a426a958b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:24.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.394+0000 7f39f4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-21T03:40:24.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.394+0000 7f39f4ff9640 1 -- 172.21.15.38:0/218598204 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a08011e00 con 0x55e653460270 2024-09-21T03:40:24.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.394+0000 7f39f4ff9640 1 -- 172.21.15.38:0/218598204 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 6390+0+0 (secure 0 0 0) 0x7f3a0801b410 con 0x55e653460270 2024-09-21T03:40:24.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.394+0000 7f3a1ec4c740 1 -- 172.21.15.38:0/218598204 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3a0408d660 con 0x55e653460270 2024-09-21T03:40:24.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.395+0000 7f39f4ff9640 1 -- 172.21.15.38:0/218598204 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f3a08022050 con 0x55e653460270 2024-09-21T03:40:24.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.395+0000 7f3a1ec4c740 10 client.7695 fetch_fsmap learned FSMap version 92 2024-09-21T03:40:24.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.395+0000 7f3a1ec4c740 1 -- 172.21.15.38:0/218598204 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=92,monmap=2+}) -- 0x55e653848aa0 con 0x55e653460270 2024-09-21T03:40:24.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.395+0000 7f39f4ff9640 1 -- 172.21.15.38:0/218598204 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3a0408d660 con 0x55e653460270 2024-09-21T03:40:24.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.395+0000 7f39f4ff9640 1 -- 172.21.15.38:0/218598204 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 92) ==== 3502+0+0 (secure 0 0 0) 0x7f3a08025020 con 0x55e653460270 2024-09-21T03:40:24.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.395+0000 7f3a1ec4c740 10 client.7695 fetch_fsmap finished waiting for FSMap version 92 2024-09-21T03:40:24.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.395+0000 7f3a1ec4c740 10 client.7695 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-21T03:40:24.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.395+0000 7f3a1ec4c740 20 client.7695 populate_metadata read hostname 'smithi038' 2024-09-21T03:40:24.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.395+0000 7f3a1ec4c740 1 --2- 172.21.15.38:0/218598204 >> [v2:172.21.15.38:6834/2934569248,v1:172.21.15.38:6835/2934569248] conn(0x55e65341ee40 0x55e653863190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:24.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.395+0000 7f3a1ec4c740 4 client.7695 mds_command: new command op to 7023 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:40:24.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.395+0000 7f3a1ec4c740 1 -- 172.21.15.38:0/218598204 --> [v2:172.21.15.38:6834/2934569248,v1:172.21.15.38:6835/2934569248] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e65377c770 con 0x55e65341ee40 2024-09-21T03:40:24.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.395+0000 7f3a159a3640 1 --2- 172.21.15.38:0/218598204 >> [v2:172.21.15.38:6834/2934569248,v1:172.21.15.38:6835/2934569248] conn(0x55e65341ee40 0x55e653863190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:24.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.395+0000 7f3a159a3640 1 --2- 172.21.15.38:0/218598204 >> [v2:172.21.15.38:6834/2934569248,v1:172.21.15.38:6835/2934569248] conn(0x55e65341ee40 0x55e653863190 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1ebefb109de960a server_cookie=511a54d5eb0761a6 in_seq=0 out_seq=0 2024-09-21T03:40:24.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.395+0000 7f39f4ff9640 10 client.7695 ms_handle_connect on v2:172.21.15.38:6834/2934569248 2024-09-21T03:40:24.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.396+0000 7f39f4ff9640 1 -- 172.21.15.38:0/218598204 <== mds.0 v2:172.21.15.38:6834/2934569248 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e65377c770 con 0x55e65341ee40 2024-09-21T03:40:24.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.396+0000 7f39f4ff9640 10 client.7695 handle_command_reply: tid=0 2024-09-21T03:40:24.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.396+0000 7f3a1ec4c740 1 client.7695 shutdown 2024-09-21T03:40:24.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.396+0000 7f3a1ec4c740 1 -- 172.21.15.38:0/218598204 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e653460270 msgr2=0x55e653801550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:24.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.396+0000 7f3a1ec4c740 1 --2- 172.21.15.38:0/218598204 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e653460270 0x55e653801550 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f39f80057e0 tx=0x7f3a08011430 comp rx=0 tx=0).stop 2024-09-21T03:40:24.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.396+0000 7f3a1ec4c740 1 -- 172.21.15.38:0/218598204 shutdown_connections 2024-09-21T03:40:24.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.396+0000 7f3a1ec4c740 1 --2- 172.21.15.38:0/218598204 >> [v2:172.21.15.38:6834/2934569248,v1:172.21.15.38:6835/2934569248] conn(0x55e65341ee40 0x55e653863190 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:24.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.396+0000 7f3a1ec4c740 1 --2- 172.21.15.38:0/218598204 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e653460270 0x55e653801550 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:24.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.396+0000 7f39f4ff9640 0 client.7695 ms_handle_reset on v2:172.21.15.38:6834/2934569248 2024-09-21T03:40:24.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.396+0000 7f3a1ec4c740 1 --2- 172.21.15.38:0/218598204 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e65343f590 0x55e653849e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:24.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.396+0000 7f3a1ec4c740 1 --2- 172.21.15.38:0/218598204 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e6534373b0 0x55e653847210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:24.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.396+0000 7f3a1ec4c740 1 -- 172.21.15.38:0/218598204 >> 172.21.15.38:0/218598204 conn(0x55e6534543c0 msgr2=0x55e653806f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:24.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.397+0000 7f3a1ec4c740 1 -- 172.21.15.38:0/218598204 shutdown_connections 2024-09-21T03:40:24.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.397+0000 7f3a1ec4c740 1 -- 172.21.15.38:0/218598204 wait complete. 2024-09-21T03:40:24.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.397+0000 7f3a1ec4c740 20 client.7695 trim_cache size 0 max 16384 2024-09-21T03:40:24.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.415+0000 7f3a1ec4c740 1 Processor -- start 2024-09-21T03:40:24.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.415+0000 7f3a1ec4c740 1 -- start start 2024-09-21T03:40:24.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.417+0000 7f3a1ec4c740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e6534373b0 0x55e6537fd960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:24.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.417+0000 7f3a1ec4c740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e65343f590 0x55e65384ab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:24.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.417+0000 7f3a1ec4c740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e653460270 0x55e65384b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:24.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.417+0000 7f3a1ec4c740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f3a101fd960 con 0x55e653460270 2024-09-21T03:40:24.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.417+0000 7f3a1ec4c740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f3a100cea20 con 0x55e6534373b0 2024-09-21T03:40:24.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.417+0000 7f3a1ec4c740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f3a100a0010 con 0x55e65343f590 2024-09-21T03:40:24.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.417+0000 7f3a161a4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e653460270 0x55e65384b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:24.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.417+0000 7f3a161a4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e653460270 0x55e65384b070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48546/0 (socket says 172.21.15.38:48546) 2024-09-21T03:40:24.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.417+0000 7f3a161a4640 1 -- 172.21.15.38:0/1163520421 learned_addr learned my addr 172.21.15.38:0/1163520421 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:24.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.417+0000 7f3a159a3640 1 --2- 172.21.15.38:0/1163520421 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e6534373b0 0x55e6537fd960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:24.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.417+0000 7f3a151a2640 1 --2- 172.21.15.38:0/1163520421 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e65343f590 0x55e65384ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:24.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.417+0000 7f3a161a4640 1 -- 172.21.15.38:0/1163520421 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e65343f590 msgr2=0x55e65384ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:24.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.417+0000 7f3a161a4640 1 --2- 172.21.15.38:0/1163520421 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e65343f590 0x55e65384ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:24.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.417+0000 7f3a161a4640 1 -- 172.21.15.38:0/1163520421 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e6534373b0 msgr2=0x55e6537fd960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:24.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.417+0000 7f3a161a4640 1 --2- 172.21.15.38:0/1163520421 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e6534373b0 0x55e6537fd960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:24.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.417+0000 7f3a161a4640 1 -- 172.21.15.38:0/1163520421 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6538221b0 con 0x55e653460270 2024-09-21T03:40:24.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.417+0000 7f3a161a4640 1 --2- 172.21.15.38:0/1163520421 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e653460270 0x55e65384b070 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f3a08006010 tx=0x7f3a08006c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cde82915d225860f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:24.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.417+0000 7f39f6ffd640 1 -- 172.21.15.38:0/1163520421 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a0801b9a0 con 0x55e653460270 2024-09-21T03:40:24.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.417+0000 7f39f6ffd640 1 -- 172.21.15.38:0/1163520421 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3a0801bb60 con 0x55e653460270 2024-09-21T03:40:24.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.417+0000 7f39f6ffd640 1 -- 172.21.15.38:0/1163520421 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a0801bdb0 con 0x55e653460270 2024-09-21T03:40:24.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.418+0000 7f3a1ec4c740 1 -- 172.21.15.38:0/1163520421 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e653460270 msgr2=0x55e65384b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:24.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.418+0000 7f3a1ec4c740 1 --2- 172.21.15.38:0/1163520421 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e653460270 0x55e65384b070 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f3a08006010 tx=0x7f3a08006c20 comp rx=0 tx=0).stop 2024-09-21T03:40:24.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.418+0000 7f3a1ec4c740 1 -- 172.21.15.38:0/1163520421 shutdown_connections 2024-09-21T03:40:24.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.418+0000 7f3a1ec4c740 1 --2- 172.21.15.38:0/1163520421 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e653460270 0x55e65384b070 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:24.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.418+0000 7f3a1ec4c740 1 --2- 172.21.15.38:0/1163520421 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e65343f590 0x55e65384ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:24.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.418+0000 7f3a1ec4c740 1 --2- 172.21.15.38:0/1163520421 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e6534373b0 0x55e6537fd960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:24.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.418+0000 7f3a1ec4c740 1 -- 172.21.15.38:0/1163520421 >> 172.21.15.38:0/1163520421 conn(0x55e6534543c0 msgr2=0x55e65380d5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:24.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.418+0000 7f3a1ec4c740 1 -- 172.21.15.38:0/1163520421 shutdown_connections 2024-09-21T03:40:24.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.418+0000 7f3a1ec4c740 1 -- 172.21.15.38:0/1163520421 wait complete. 2024-09-21T03:40:24.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.419+0000 7f3a1ec4c740 1 -- start start 2024-09-21T03:40:24.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.419+0000 7f3a1ec4c740 1 Processor -- start 2024-09-21T03:40:24.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.421+0000 7f3a1ec4c740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e6534373b0 0x55e6538b9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:24.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.421+0000 7f3a1ec4c740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e65343f590 0x55e6538cff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:24.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.421+0000 7f3a1ec4c740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e653460270 0x55e653869880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:24.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.421+0000 7f3a1ec4c740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55e65384a370 con 0x55e653460270 2024-09-21T03:40:24.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.421+0000 7f3a1ec4c740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55e653848d80 con 0x55e65343f590 2024-09-21T03:40:24.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.421+0000 7f3a1ec4c740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55e653822610 con 0x55e6534373b0 2024-09-21T03:40:24.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.421+0000 7f3a1ec4c740 10 client.0 _ll_register_callbacks cb 0x7f3a080047d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:40:24.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.421+0000 7f3a161a4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e653460270 0x55e653869880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:24.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.421+0000 7f3a161a4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e653460270 0x55e653869880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48550/0 (socket says 172.21.15.38:48550) 2024-09-21T03:40:24.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.421+0000 7f3a161a4640 1 -- 172.21.15.38:0/2220825547 learned_addr learned my addr 172.21.15.38:0/2220825547 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:24.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.421+0000 7f3a151a2640 1 --2- 172.21.15.38:0/2220825547 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e65343f590 0x55e6538cff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:24.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.421+0000 7f3a159a3640 1 --2- 172.21.15.38:0/2220825547 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e6534373b0 0x55e6538b9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:24.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.421+0000 7f3a161a4640 1 -- 172.21.15.38:0/2220825547 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e6534373b0 msgr2=0x55e6538b9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:24.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.421+0000 7f3a161a4640 1 --2- 172.21.15.38:0/2220825547 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e6534373b0 0x55e6538b9230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:24.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.421+0000 7f3a161a4640 1 -- 172.21.15.38:0/2220825547 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e65343f590 msgr2=0x55e6538cff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:24.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.421+0000 7f3a161a4640 1 --2- 172.21.15.38:0/2220825547 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e65343f590 0x55e6538cff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:24.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.421+0000 7f3a161a4640 1 -- 172.21.15.38:0/2220825547 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e653826400 con 0x55e653460270 2024-09-21T03:40:24.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.421+0000 7f3a159a3640 1 --2- 172.21.15.38:0/2220825547 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e6534373b0 0x55e6538b9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:40:24.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.421+0000 7f3a161a4640 1 --2- 172.21.15.38:0/2220825547 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e653460270 0x55e653869880 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f3a08013040 tx=0x7f3a08026680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b23a77b1f0297376 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:24.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.421+0000 7f39f4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-21T03:40:24.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.421+0000 7f39f4ff9640 1 -- 172.21.15.38:0/2220825547 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a0801b9a0 con 0x55e653460270 2024-09-21T03:40:24.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.422+0000 7f39f4ff9640 1 -- 172.21.15.38:0/2220825547 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 6390+0+0 (secure 0 0 0) 0x7f3a08026a80 con 0x55e653460270 2024-09-21T03:40:24.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.422+0000 7f3a1ec4c740 1 -- 172.21.15.38:0/2220825547 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3a0801bb60 con 0x55e653460270 2024-09-21T03:40:24.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.422+0000 7f39f4ff9640 1 -- 172.21.15.38:0/2220825547 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f3a0803f050 con 0x55e653460270 2024-09-21T03:40:24.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.422+0000 7f3a1ec4c740 10 client.7701 fetch_fsmap learned FSMap version 92 2024-09-21T03:40:24.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.422+0000 7f3a1ec4c740 1 -- 172.21.15.38:0/2220825547 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=92,monmap=2+}) -- 0x55e6538d06c0 con 0x55e653460270 2024-09-21T03:40:24.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.422+0000 7f39f4ff9640 1 -- 172.21.15.38:0/2220825547 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3a0801bb60 con 0x55e653460270 2024-09-21T03:40:24.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.422+0000 7f39f4ff9640 1 -- 172.21.15.38:0/2220825547 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 92) ==== 3502+0+0 (secure 0 0 0) 0x7f3a08042020 con 0x55e653460270 2024-09-21T03:40:24.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.422+0000 7f3a1ec4c740 10 client.7701 fetch_fsmap finished waiting for FSMap version 92 2024-09-21T03:40:24.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.422+0000 7f3a1ec4c740 10 client.7701 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-21T03:40:24.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.422+0000 7f3a1ec4c740 20 client.7701 populate_metadata read hostname 'smithi038' 2024-09-21T03:40:24.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.422+0000 7f3a1ec4c740 1 --2- 172.21.15.38:0/2220825547 >> [v2:172.21.15.38:6834/2934569248,v1:172.21.15.38:6835/2934569248] conn(0x55e65384afc0 0x55e6538205a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:24.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.422+0000 7f3a1ec4c740 4 client.7701 mds_command: new command op to 7023 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T03:40:24.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.422+0000 7f3a1ec4c740 1 -- 172.21.15.38:0/2220825547 --> [v2:172.21.15.38:6834/2934569248,v1:172.21.15.38:6835/2934569248] -- command(tid 0: {"prefix": "status"}) -- 0x55e653852990 con 0x55e65384afc0 2024-09-21T03:40:24.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.422+0000 7f3a159a3640 1 --2- 172.21.15.38:0/2220825547 >> [v2:172.21.15.38:6834/2934569248,v1:172.21.15.38:6835/2934569248] conn(0x55e65384afc0 0x55e6538205a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:24.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.423+0000 7f39f4ff9640 10 client.7701 ms_handle_connect on v2:172.21.15.38:6834/2934569248 2024-09-21T03:40:24.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.423+0000 7f3a159a3640 1 --2- 172.21.15.38:0/2220825547 >> [v2:172.21.15.38:6834/2934569248,v1:172.21.15.38:6835/2934569248] conn(0x55e65384afc0 0x55e6538205a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d4aca9f3d509d7e server_cookie=14032be3e860b216 in_seq=0 out_seq=0 2024-09-21T03:40:24.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.423+0000 7f39f4ff9640 1 -- 172.21.15.38:0/2220825547 <== mds.0 v2:172.21.15.38:6834/2934569248 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x55e653852990 con 0x55e65384afc0 2024-09-21T03:40:24.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.423+0000 7f39f4ff9640 10 client.7701 handle_command_reply: tid=0 2024-09-21T03:40:24.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.423+0000 7f3a1ec4c740 1 client.7701 shutdown 2024-09-21T03:40:24.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.423+0000 7f3a1ec4c740 1 -- 172.21.15.38:0/2220825547 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e653460270 msgr2=0x55e653869880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:24.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.423+0000 7f3a1ec4c740 1 --2- 172.21.15.38:0/2220825547 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e653460270 0x55e653869880 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f3a08013040 tx=0x7f3a08026680 comp rx=0 tx=0).stop 2024-09-21T03:40:24.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.423+0000 7f3a1ec4c740 1 -- 172.21.15.38:0/2220825547 shutdown_connections 2024-09-21T03:40:24.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.423+0000 7f3a1ec4c740 1 --2- 172.21.15.38:0/2220825547 >> [v2:172.21.15.38:6834/2934569248,v1:172.21.15.38:6835/2934569248] conn(0x55e65384afc0 0x55e6538205a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:24.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.423+0000 7f3a1ec4c740 1 --2- 172.21.15.38:0/2220825547 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e653460270 0x55e653869880 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:24.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.423+0000 7f3a1ec4c740 1 --2- 172.21.15.38:0/2220825547 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e65343f590 0x55e6538cff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:24.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.423+0000 7f3a1ec4c740 1 --2- 172.21.15.38:0/2220825547 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e6534373b0 0x55e6538b9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:24.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.423+0000 7f39f4ff9640 0 client.7701 ms_handle_reset on v2:172.21.15.38:6834/2934569248 2024-09-21T03:40:24.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.423+0000 7f3a1ec4c740 1 -- 172.21.15.38:0/2220825547 >> 172.21.15.38:0/2220825547 conn(0x55e6534543c0 msgr2=0x55e65380c520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:24.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.423+0000 7f3a1ec4c740 1 -- 172.21.15.38:0/2220825547 shutdown_connections 2024-09-21T03:40:24.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.423+0000 7f3a1ec4c740 1 -- 172.21.15.38:0/2220825547 wait complete. 2024-09-21T03:40:24.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.423+0000 7f3a1ec4c740 20 client.7701 trim_cache size 0 max 16384 2024-09-21T03:40:24.430 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-09-21T03:40:24.430 INFO:teuthology.orchestra.run.smithi038.stdout: "cluster_fsid": "9d50d66f-656b-4fe2-8e3b-fe53b79c4d43", 2024-09-21T03:40:24.430 INFO:teuthology.orchestra.run.smithi038.stdout: "whoami": 0, 2024-09-21T03:40:24.430 INFO:teuthology.orchestra.run.smithi038.stdout: "id": 7023, 2024-09-21T03:40:24.431 INFO:teuthology.orchestra.run.smithi038.stdout: "want_state": "up:active", 2024-09-21T03:40:24.431 INFO:teuthology.orchestra.run.smithi038.stdout: "state": "up:active", 2024-09-21T03:40:24.431 INFO:teuthology.orchestra.run.smithi038.stdout: "fs_name": "cephfs", 2024-09-21T03:40:24.431 INFO:teuthology.orchestra.run.smithi038.stdout: "rank_uptime": 6.7381515619999997, 2024-09-21T03:40:24.431 INFO:teuthology.orchestra.run.smithi038.stdout: "mdsmap_epoch": 92, 2024-09-21T03:40:24.431 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch": 97, 2024-09-21T03:40:24.431 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch_barrier": 97, 2024-09-21T03:40:24.431 INFO:teuthology.orchestra.run.smithi038.stdout: "uptime": 26.176763958999999 2024-09-21T03:40:24.431 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-09-21T03:40:24.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.424+0000 7f3a17c07640 1 -- 172.21.15.38:0/2550813462 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f39fc078bc0 msgr2=0x7f39fc07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:24.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.424+0000 7f3a17c07640 1 --2- 172.21.15.38:0/2550813462 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f39fc078bc0 0x7f39fc07b0d0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f3a000013d0 tx=0x7f3a00006b20 comp rx=0 tx=0).stop 2024-09-21T03:40:24.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.424+0000 7f3a17c07640 1 -- 172.21.15.38:0/2550813462 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3a100b5af0 msgr2=0x7f3a100a08c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:24.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.424+0000 7f3a17c07640 1 --2- 172.21.15.38:0/2550813462 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3a100b5af0 0x7f3a100a08c0 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f3a04034040 tx=0x7f3a0400b8b0 comp rx=0 tx=0).stop 2024-09-21T03:40:24.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.424+0000 7f3a17c07640 1 -- 172.21.15.38:0/2550813462 shutdown_connections 2024-09-21T03:40:24.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.424+0000 7f3a17c07640 1 --2- 172.21.15.38:0/2550813462 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3a1015a940 0x7f3a101f3320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:24.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.424+0000 7f3a17c07640 1 --2- 172.21.15.38:0/2550813462 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3a100b5af0 0x7f3a100a08c0 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:24.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.424+0000 7f3a17c07640 1 --2- 172.21.15.38:0/2550813462 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f39fc078bc0 0x7f39fc07b0d0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:24.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.424+0000 7f3a17c07640 1 --2- 172.21.15.38:0/2550813462 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f39f8004800 0x7f3a100a0380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:24.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.424+0000 7f3a17c07640 1 -- 172.21.15.38:0/2550813462 >> 172.21.15.38:0/2550813462 conn(0x7f3a100b1b90 msgr2=0x7f3a10154040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:24.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.424+0000 7f3a17c07640 1 -- 172.21.15.38:0/2550813462 shutdown_connections 2024-09-21T03:40:24.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.424+0000 7f3a17c07640 1 -- 172.21.15.38:0/2550813462 wait complete. 2024-09-21T03:40:24.437 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-09-21T03:40:24.437 INFO:tasks.cephfs.test_damage:Daemons came up after mutation 'Delete 10000000000.00000000', proceeding to ls 2024-09-21T03:40:24.437 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T03:40:24.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.552+0000 7f8ab763f640 1 -- 172.21.15.38:0/457582530 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8ab0162f00 msgr2=0x7f8ab00d24d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:24.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.552+0000 7f8ab763f640 1 --2- 172.21.15.38:0/457582530 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8ab0162f00 0x7f8ab00d24d0 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f8aa8009830 tx=0x7f8aa802a530 comp rx=0 tx=0).stop 2024-09-21T03:40:24.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.552+0000 7f8ab763f640 1 -- 172.21.15.38:0/457582530 shutdown_connections 2024-09-21T03:40:24.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.552+0000 7f8ab763f640 1 --2- 172.21.15.38:0/457582530 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8ab0162f00 0x7f8ab00d24d0 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:24.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.552+0000 7f8ab763f640 1 -- 172.21.15.38:0/457582530 >> 172.21.15.38:0/457582530 conn(0x7f8ab00b6b20 msgr2=0x7f8ab00d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:24.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.552+0000 7f8ab763f640 1 -- 172.21.15.38:0/457582530 shutdown_connections 2024-09-21T03:40:24.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.552+0000 7f8ab763f640 1 -- 172.21.15.38:0/457582530 wait complete. 2024-09-21T03:40:24.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.554+0000 7f8ab763f640 1 Processor -- start 2024-09-21T03:40:24.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.554+0000 7f8ab763f640 1 -- start start 2024-09-21T03:40:24.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.555+0000 7f8ab763f640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8ab00b2e40 0x7f8ab00aadc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:24.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.555+0000 7f8ab763f640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8ab0162280 0x7f8ab00afa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:24.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.555+0000 7f8ab763f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8ab00b2570 0x7f8ab014d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:24.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.555+0000 7f8ab763f640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f8ab015f6d0 con 0x7f8ab00b2570 2024-09-21T03:40:24.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.555+0000 7f8ab763f640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f8ab00d2c00 con 0x7f8ab0162280 2024-09-21T03:40:24.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.555+0000 7f8ab763f640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f8ab015f850 con 0x7f8ab00b2e40 2024-09-21T03:40:24.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.555+0000 7f8ab5bdc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8ab00b2570 0x7f8ab014d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:24.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.555+0000 7f8ab5bdc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8ab00b2570 0x7f8ab014d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48570/0 (socket says 172.21.15.38:48570) 2024-09-21T03:40:24.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.556+0000 7f8ab5bdc640 1 -- 172.21.15.38:0/2696945560 learned_addr learned my addr 172.21.15.38:0/2696945560 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:24.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.556+0000 7f8ab4bda640 1 --2- 172.21.15.38:0/2696945560 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8ab0162280 0x7f8ab00afa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:24.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.556+0000 7f8ab53db640 1 --2- 172.21.15.38:0/2696945560 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8ab00b2e40 0x7f8ab00aadc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:24.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.556+0000 7f8ab5bdc640 1 -- 172.21.15.38:0/2696945560 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8ab00b2e40 msgr2=0x7f8ab00aadc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:24.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.556+0000 7f8ab5bdc640 1 --2- 172.21.15.38:0/2696945560 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8ab00b2e40 0x7f8ab00aadc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:24.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.556+0000 7f8ab5bdc640 1 -- 172.21.15.38:0/2696945560 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8ab0162280 msgr2=0x7f8ab00afa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:24.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.556+0000 7f8ab5bdc640 1 --2- 172.21.15.38:0/2696945560 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8ab0162280 0x7f8ab00afa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:24.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.556+0000 7f8ab5bdc640 1 -- 172.21.15.38:0/2696945560 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ab00b1530 con 0x7f8ab00b2570 2024-09-21T03:40:24.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.556+0000 7f8ab53db640 1 --2- 172.21.15.38:0/2696945560 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8ab00b2e40 0x7f8ab00aadc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:40:24.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.556+0000 7f8ab5bdc640 1 --2- 172.21.15.38:0/2696945560 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8ab00b2570 0x7f8ab014d930 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f8aa000cae0 tx=0x7f8aa000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:24.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.556+0000 7f8aadffb640 1 -- 172.21.15.38:0/2696945560 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aa0016020 con 0x7f8ab00b2570 2024-09-21T03:40:24.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.556+0000 7f8aadffb640 1 -- 172.21.15.38:0/2696945560 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8aa000ad30 con 0x7f8ab00b2570 2024-09-21T03:40:24.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.556+0000 7f8ab763f640 1 -- 172.21.15.38:0/2696945560 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ab00b17b0 con 0x7f8ab00b2570 2024-09-21T03:40:24.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.556+0000 7f8aadffb640 1 -- 172.21.15.38:0/2696945560 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aa0012680 con 0x7f8ab00b2570 2024-09-21T03:40:24.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.556+0000 7f8ab763f640 1 -- 172.21.15.38:0/2696945560 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ab00b1d70 con 0x7f8ab00b2570 2024-09-21T03:40:24.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.557+0000 7f8ab763f640 1 -- 172.21.15.38:0/2696945560 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ab0162f00 con 0x7f8ab00b2570 2024-09-21T03:40:24.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.558+0000 7f8aadffb640 1 -- 172.21.15.38:0/2696945560 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f8aa0019020 con 0x7f8ab00b2570 2024-09-21T03:40:24.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.559+0000 7f8aadffb640 1 --2- 172.21.15.38:0/2696945560 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8a9c078bc0 0x7f8a9c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:24.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.559+0000 7f8aadffb640 1 -- 172.21.15.38:0/2696945560 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 6390+0+0 (secure 0 0 0) 0x7f8aa009b600 con 0x7f8ab00b2570 2024-09-21T03:40:24.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.559+0000 7f8ab53db640 1 --2- 172.21.15.38:0/2696945560 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8a9c078bc0 0x7f8a9c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:24.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.560+0000 7f8ab53db640 1 --2- 172.21.15.38:0/2696945560 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8a9c078bc0 0x7f8a9c07b0d0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f8aa8034000 tx=0x7f8aa8004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:24.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.563+0000 7f8aadffb640 1 -- 172.21.15.38:0/2696945560 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f8aa0020030 con 0x7f8ab00b2570 2024-09-21T03:40:24.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.788+0000 7f8ab763f640 1 -- 172.21.15.38:0/2696945560 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f8ab00b4650 con 0x7f8ab00b2570 2024-09-21T03:40:24.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.789+0000 7f8aadffb640 1 -- 172.21.15.38:0/2696945560 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v92) ==== 53+0+73 (secure 0 0 0) 0x7f8ab00b1530 con 0x7f8ab00b2570 2024-09-21T03:40:24.791 INFO:teuthology.orchestra.run.smithi038.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T03:40:24.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.791+0000 7f8ab763f640 1 -- 172.21.15.38:0/2696945560 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8a9c078bc0 msgr2=0x7f8a9c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:24.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.791+0000 7f8ab763f640 1 --2- 172.21.15.38:0/2696945560 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8a9c078bc0 0x7f8a9c07b0d0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f8aa8034000 tx=0x7f8aa8004730 comp rx=0 tx=0).stop 2024-09-21T03:40:24.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.791+0000 7f8ab763f640 1 -- 172.21.15.38:0/2696945560 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8ab00b2570 msgr2=0x7f8ab014d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:24.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.791+0000 7f8ab763f640 1 --2- 172.21.15.38:0/2696945560 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8ab00b2570 0x7f8ab014d930 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f8aa000cae0 tx=0x7f8aa000a060 comp rx=0 tx=0).stop 2024-09-21T03:40:24.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.791+0000 7f8ab763f640 1 -- 172.21.15.38:0/2696945560 shutdown_connections 2024-09-21T03:40:24.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.792+0000 7f8ab763f640 1 --2- 172.21.15.38:0/2696945560 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8ab00b2570 0x7f8ab014d930 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:24.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.792+0000 7f8ab763f640 1 --2- 172.21.15.38:0/2696945560 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8ab0162280 0x7f8ab00afa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:24.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.792+0000 7f8ab763f640 1 --2- 172.21.15.38:0/2696945560 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8a9c078bc0 0x7f8a9c07b0d0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:24.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.792+0000 7f8ab763f640 1 --2- 172.21.15.38:0/2696945560 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8ab00b2e40 0x7f8ab00aadc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:24.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.792+0000 7f8ab763f640 1 -- 172.21.15.38:0/2696945560 >> 172.21.15.38:0/2696945560 conn(0x7f8ab00b6b20 msgr2=0x7f8ab00b3ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:24.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.792+0000 7f8ab763f640 1 -- 172.21.15.38:0/2696945560 shutdown_connections 2024-09-21T03:40:24.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:24.792+0000 7f8ab763f640 1 -- 172.21.15.38:0/2696945560 wait complete. 2024-09-21T03:40:24.805 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-21T03:40:24.805 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T03:40:24.805 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi038.front.sepia.ceph.com 2024-09-21T03:40:24.805 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-21T03:40:24.805 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T03:40:24.805 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-21T03:40:24.805 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-21T03:40:24.805 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T03:40:24.805 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T03:40:24.805 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:40:24.806 DEBUG:teuthology.orchestra.run.smithi038:> ip addr 2024-09-21T03:40:24.822 INFO:teuthology.orchestra.run.smithi038.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T03:40:24.822 INFO:teuthology.orchestra.run.smithi038.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T03:40:24.822 INFO:teuthology.orchestra.run.smithi038.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T03:40:24.822 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:40:24.822 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 ::1/128 scope host 2024-09-21T03:40:24.822 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:40:24.822 INFO:teuthology.orchestra.run.smithi038.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T03:40:24.822 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:6b:fd:a2 brd ff:ff:ff:ff:ff:ff 2024-09-21T03:40:24.823 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp7s0f0 2024-09-21T03:40:24.823 INFO:teuthology.orchestra.run.smithi038.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T03:40:24.823 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:6b:fd:a3 brd ff:ff:ff:ff:ff:ff 2024-09-21T03:40:24.823 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp7s0f1 2024-09-21T03:40:24.823 INFO:teuthology.orchestra.run.smithi038.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T03:40:24.823 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:bd:15:fc brd ff:ff:ff:ff:ff:ff 2024-09-21T03:40:24.823 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp1s0f0 2024-09-21T03:40:24.823 INFO:teuthology.orchestra.run.smithi038.stdout: inet 172.21.15.38/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-21T03:40:24.823 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft 42384sec preferred_lft 42384sec 2024-09-21T03:40:24.823 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 fe80::ec4:7aff:febd:15fc/64 scope link 2024-09-21T03:40:24.823 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:40:24.823 INFO:teuthology.orchestra.run.smithi038.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T03:40:24.823 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:bd:15:fd brd ff:ff:ff:ff:ff:ff 2024-09-21T03:40:24.824 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp1s0f1 2024-09-21T03:40:24.824 INFO:teuthology.orchestra.run.smithi038.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T03:40:24.824 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 92:55:c7:97:6f:b5 brd ff:ff:ff:ff:ff:ff 2024-09-21T03:40:24.824 INFO:teuthology.orchestra.run.smithi038.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-21T03:40:24.824 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:40:24.824 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 fe80::c49f:c1ff:fe21:b12a/64 scope link 2024-09-21T03:40:24.824 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:40:24.824 INFO:teuthology.orchestra.run.smithi038.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T03:40:24.824 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether a6:3a:83:ad:db:d5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T03:40:24.824 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 fe80::a43a:83ff:fead:dbd5/64 scope link 2024-09-21T03:40:24.824 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:40:24.824 INFO:teuthology.orchestra.run.smithi038.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T03:40:24.824 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 92:55:c7:97:6f:b5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T03:40:24.824 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 fe80::9055:c7ff:fe97:6fb5/64 scope link 2024-09-21T03:40:24.825 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:40:24.825 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:40:24.825 DEBUG:teuthology.orchestra.run.smithi038:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T03:40:24.868 INFO:tasks.ceph.mds.a.smithi038.stderr:2024-09-21T03:40:24.866+0000 7fa039119640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T03:40:24.886 INFO:teuthology.orchestra.run.smithi038.stdout:1 2024-09-21T03:40:24.888 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:40:24.888 DEBUG:teuthology.orchestra.run.smithi038:> route 2024-09-21T03:40:24.944 INFO:teuthology.orchestra.run.smithi038.stdout:Kernel IP routing table 2024-09-21T03:40:24.945 INFO:teuthology.orchestra.run.smithi038.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T03:40:24.945 INFO:teuthology.orchestra.run.smithi038.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-21T03:40:24.945 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-21T03:40:24.945 INFO:teuthology.orchestra.run.smithi038.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T03:40:24.945 DEBUG:teuthology.orchestra.run.smithi038:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T03:40:24.945 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-09-21T03:40:24.945 DEBUG:teuthology.orchestra.run.smithi038:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-21T03:40:24.945 DEBUG:teuthology.orchestra.run.smithi038:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-21T03:40:24.945 DEBUG:teuthology.orchestra.run.smithi038:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-21T03:40:24.945 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-09-21T03:40:25.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:25 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T03:40:25.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:40:25.073 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:40:25.073 DEBUG:teuthology.orchestra.run.smithi038:> ip netns list 2024-09-21T03:40:25.090 INFO:teuthology.orchestra.run.smithi038.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-21T03:40:25.090 INFO:teuthology.orchestra.run.smithi038.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-21T03:40:25.090 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T03:40:25.090 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:40:25.090 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip link set brx.0 up 2024-09-21T03:40:25.155 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T03:40:25.156 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-21T03:40:25.211 INFO:teuthology.orchestra.run.smithi038.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-21T03:40:25.212 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T03:40:25.212 DEBUG:teuthology.orchestra.run.smithi038:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-21T03:40:25.271 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-21T03:40:25.651 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:40:25.651 DEBUG:teuthology.orchestra.run.smithi038:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T03:40:25.710 INFO:teuthology.orchestra.run.smithi038.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T03:40:25.711 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T03:40:25.711 INFO:teuthology.orchestra.run.smithi038.stdout:invalid new device string format 2024-09-21T03:40:25.722 INFO:teuthology.orchestra.run.smithi038.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T03:40:25.722 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T03:40:25.722 INFO:teuthology.orchestra.run.smithi038.stdout:invalid new device string format 2024-09-21T03:40:25.722 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: resolved to: "172.21.15.38:3300,172.21.15.195:3300,172.21.15.195:3301" 2024-09-21T03:40:25.722 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T03:40:25.731 INFO:teuthology.orchestra.run.smithi038.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T03:40:25.732 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T03:40:25.732 INFO:teuthology.orchestra.run.smithi038.stdout:invalid new device string format 2024-09-21T03:40:25.732 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: resolved to: "172.21.15.38:3300,172.21.15.195:3300,172.21.15.195:3301" 2024-09-21T03:40:25.732 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T03:40:25.732 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: resolved to: "172.21.15.38:6789,172.21.15.195:6789,172.21.15.195:6790" 2024-09-21T03:40:25.732 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.38:6789,172.21.15.195:6789,172.21.15.195:6790:/ 2024-09-21T03:40:25.732 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=9d50d66f-656b-4fe2-8e3b-fe53b79c4d43" will pass to kernel 2024-09-21T03:40:25.733 INFO:teuthology.orchestra.run.smithi038.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-21T03:40:25.733 INFO:teuthology.orchestra.run.smithi038.stdout: You just mounted a file system that supports labels which does not 2024-09-21T03:40:25.733 INFO:teuthology.orchestra.run.smithi038.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-21T03:40:25.733 INFO:teuthology.orchestra.run.smithi038.stdout: applications will generate AVC messages and not be allowed access to 2024-09-21T03:40:25.733 INFO:teuthology.orchestra.run.smithi038.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-21T03:40:25.734 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T03:40:25.734 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:40:25.735 DEBUG:teuthology.orchestra.run.smithi038:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-21T03:40:25.761 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T03:40:25.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:25 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T03:40:25.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:40:25.862 INFO:teuthology.orchestra.run.smithi038.stdout:client7713 2024-09-21T03:40:25.862 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-21T03:40:25.862 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:40:25.883 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:40:25.883 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls -R .) 2024-09-21T03:40:25.929 INFO:teuthology.orchestra.run:waiting for 20 2024-09-21T03:40:25.943 INFO:tasks.ceph.mds.a.smithi038.stderr:2024-09-21T03:40:25.941+0000 7fa035111640 -1 log_channel(cluster) log [ERR] : dir 0x10000000000 object missing on disk; some files may be lost (/subdir) 2024-09-21T03:40:25.944 INFO:teuthology.orchestra.run.smithi038.stdout:.: 2024-09-21T03:40:25.944 INFO:teuthology.orchestra.run.smithi038.stdout:subdir 2024-09-21T03:40:25.944 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:40:25.944 INFO:teuthology.orchestra.run.smithi038.stdout:./subdir: 2024-09-21T03:40:25.944 INFO:tasks.cephfs.test_damage:Result: Mutation 'Delete 10000000000.00000000' did not caused DAMAGED state 2024-09-21T03:40:25.945 INFO:tasks.cephfs.test_damage:Applying mutation 'Delete 600.00000000' 2024-09-21T03:40:25.945 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-21T03:40:25.945 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:40:26.004 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-21T03:40:26.004 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:40:26.063 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T03:40:26.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:26.219+0000 7f21264b6640 1 -- 172.21.15.38:0/871086682 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2108003a50 msgr2=0x7f2108003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:26.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:26.219+0000 7f21264b6640 1 --2- 172.21.15.38:0/871086682 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2108003a50 0x7f2108003ec0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f211000c3f0 tx=0x7f2108005830 comp rx=0 tx=0).stop 2024-09-21T03:40:26.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:26.219+0000 7f21264b6640 1 -- 172.21.15.38:0/871086682 shutdown_connections 2024-09-21T03:40:26.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:26.219+0000 7f21264b6640 1 --2- 172.21.15.38:0/871086682 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2108003a50 0x7f2108003ec0 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:26.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:26.219+0000 7f21264b6640 1 -- 172.21.15.38:0/871086682 >> 172.21.15.38:0/871086682 conn(0x7f212009d830 msgr2=0x7f21200b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:26.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:26.219+0000 7f21264b6640 1 -- 172.21.15.38:0/871086682 shutdown_connections 2024-09-21T03:40:26.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:26.219+0000 7f21264b6640 1 -- 172.21.15.38:0/871086682 wait complete. 2024-09-21T03:40:26.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:26.221+0000 7f21264b6640 1 Processor -- start 2024-09-21T03:40:26.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:26.221+0000 7f21264b6640 1 -- start start 2024-09-21T03:40:26.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:26.222+0000 7f21264b6640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2108003a50 0x7f212015c320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:26.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:26.222+0000 7f21264b6640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f21201596e0 0x7f2120159af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:26.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:26.222+0000 7f21264b6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f21200b28b0 0x7f21200b2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:26.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:26.222+0000 7f21264b6640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f21200b9c50 con 0x7f21200b28b0 2024-09-21T03:40:26.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:26.222+0000 7f21264b6640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f21200b9ad0 con 0x7f2108003a50 2024-09-21T03:40:26.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:26.222+0000 7f21264b6640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f21200b9dd0 con 0x7f21201596e0 2024-09-21T03:40:26.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:26.222+0000 7f2124a53640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f21200b28b0 0x7f21200b2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:26.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:26.222+0000 7f2124a53640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f21200b28b0 0x7f21200b2cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48596/0 (socket says 172.21.15.38:48596) 2024-09-21T03:40:26.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:26.222+0000 7f2124a53640 1 -- 172.21.15.38:0/2554858434 learned_addr learned my addr 172.21.15.38:0/2554858434 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:26.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:26.222+0000 7f211ffff640 1 --2- 172.21.15.38:0/2554858434 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2108003a50 0x7f212015c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:26.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:26.222+0000 7f211f7fe640 1 --2- 172.21.15.38:0/2554858434 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f21201596e0 0x7f2120159af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:26.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:26.223+0000 7f2124a53640 1 -- 172.21.15.38:0/2554858434 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f21201596e0 msgr2=0x7f2120159af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:26.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:26.223+0000 7f2124a53640 1 --2- 172.21.15.38:0/2554858434 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f21201596e0 0x7f2120159af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:26.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:26.223+0000 7f2124a53640 1 -- 172.21.15.38:0/2554858434 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2108003a50 msgr2=0x7f212015c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:26.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:26.223+0000 7f2124a53640 1 --2- 172.21.15.38:0/2554858434 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2108003a50 0x7f212015c320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:26.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:26.223+0000 7f2124a53640 1 -- 172.21.15.38:0/2554858434 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21200b3200 con 0x7f21200b28b0 2024-09-21T03:40:26.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:26.223+0000 7f211ffff640 1 --2- 172.21.15.38:0/2554858434 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2108003a50 0x7f212015c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:40:26.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:26.223+0000 7f211f7fe640 1 --2- 172.21.15.38:0/2554858434 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f21201596e0 0x7f2120159af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:40:26.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:26.223+0000 7f2124a53640 1 --2- 172.21.15.38:0/2554858434 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f21200b28b0 0x7f21200b2cc0 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f21200b3c20 tx=0x7f21180019b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:26.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:26.223+0000 7f211cff9640 1 -- 172.21.15.38:0/2554858434 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2118011870 con 0x7f21200b28b0 2024-09-21T03:40:26.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:26.223+0000 7f211cff9640 1 -- 172.21.15.38:0/2554858434 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2118017070 con 0x7f21200b28b0 2024-09-21T03:40:26.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:26.223+0000 7f21264b6640 1 -- 172.21.15.38:0/2554858434 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f212015b490 con 0x7f21200b28b0 2024-09-21T03:40:26.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:26.223+0000 7f211cff9640 1 -- 172.21.15.38:0/2554858434 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2118011d80 con 0x7f21200b28b0 2024-09-21T03:40:26.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:26.223+0000 7f21264b6640 1 -- 172.21.15.38:0/2554858434 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21201f5900 con 0x7f21200b28b0 2024-09-21T03:40:26.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:26.224+0000 7f21264b6640 1 -- 172.21.15.38:0/2554858434 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21200cec70 con 0x7f21200b28b0 2024-09-21T03:40:26.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:26.225+0000 7f211cff9640 1 -- 172.21.15.38:0/2554858434 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f211801c460 con 0x7f21200b28b0 2024-09-21T03:40:26.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:26.225+0000 7f211cff9640 1 --2- 172.21.15.38:0/2554858434 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f210c078b90 0x7f210c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:26.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:26.225+0000 7f211cff9640 1 -- 172.21.15.38:0/2554858434 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 6390+0+0 (secure 0 0 0) 0x7f211809b970 con 0x7f21200b28b0 2024-09-21T03:40:26.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:26.226+0000 7f211ffff640 1 --2- 172.21.15.38:0/2554858434 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f210c078b90 0x7f210c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:26.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:26.226+0000 7f211ffff640 1 --2- 172.21.15.38:0/2554858434 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f210c078b90 0x7f210c07b0a0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f2110033000 tx=0x7f2110000fe0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:26.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:26.230+0000 7f211cff9640 1 -- 172.21.15.38:0/2554858434 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f21200b3200 con 0x7f21200b28b0 2024-09-21T03:40:26.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:26.429+0000 7f21264b6640 1 -- 172.21.15.38:0/2554858434 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f21200b3860 con 0x7f21200b28b0 2024-09-21T03:40:26.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:26.430+0000 7f211cff9640 1 -- 172.21.15.38:0/2554858434 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v97) ==== 80+0+122 (secure 0 0 0) 0x7f2118063aa0 con 0x7f21200b28b0 2024-09-21T03:40:26.432 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.195:6837/712370966 2024-09-22T03:40:11.633787+0000 2024-09-21T03:40:26.432 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.195:6835/712370966 2024-09-22T03:40:11.633787+0000 2024-09-21T03:40:26.432 INFO:teuthology.orchestra.run.smithi038.stderr:listed 2 entries 2024-09-21T03:40:26.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:26.432+0000 7f21264b6640 1 -- 172.21.15.38:0/2554858434 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f210c078b90 msgr2=0x7f210c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:26.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:26.432+0000 7f21264b6640 1 --2- 172.21.15.38:0/2554858434 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f210c078b90 0x7f210c07b0a0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f2110033000 tx=0x7f2110000fe0 comp rx=0 tx=0).stop 2024-09-21T03:40:26.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:26.432+0000 7f21264b6640 1 -- 172.21.15.38:0/2554858434 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f21200b28b0 msgr2=0x7f21200b2cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:26.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:26.432+0000 7f21264b6640 1 --2- 172.21.15.38:0/2554858434 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f21200b28b0 0x7f21200b2cc0 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f21200b3c20 tx=0x7f21180019b0 comp rx=0 tx=0).stop 2024-09-21T03:40:26.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:26.432+0000 7f21264b6640 1 -- 172.21.15.38:0/2554858434 shutdown_connections 2024-09-21T03:40:26.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:26.432+0000 7f21264b6640 1 --2- 172.21.15.38:0/2554858434 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f21200b28b0 0x7f21200b2cc0 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:26.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:26.432+0000 7f21264b6640 1 --2- 172.21.15.38:0/2554858434 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f210c078b90 0x7f210c07b0a0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:26.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:26.432+0000 7f21264b6640 1 --2- 172.21.15.38:0/2554858434 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f21201596e0 0x7f2120159af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:26.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:26.432+0000 7f21264b6640 1 --2- 172.21.15.38:0/2554858434 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2108003a50 0x7f212015c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:26.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:26.432+0000 7f21264b6640 1 -- 172.21.15.38:0/2554858434 >> 172.21.15.38:0/2554858434 conn(0x7f212009d830 msgr2=0x7f212015e060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:26.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:26.432+0000 7f21264b6640 1 -- 172.21.15.38:0/2554858434 shutdown_connections 2024-09-21T03:40:26.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:26.432+0000 7f21264b6640 1 -- 172.21.15.38:0/2554858434 wait complete. 2024-09-21T03:40:26.445 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-21T03:40:26.445 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:40:26.445 DEBUG:teuthology.orchestra.run.smithi038:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-09-21T03:40:26.479 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi038.front.sepia.ceph.com 2024-09-21T03:40:26.479 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:40:26.479 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T03:40:26.535 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-21T03:40:26.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:26.691+0000 7fcb97f6e640 1 -- 172.21.15.38:0/1242074959 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcb900d0170 msgr2=0x7fcb900d0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:26.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:26.691+0000 7fcb97f6e640 1 --2- 172.21.15.38:0/1242074959 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcb900d0170 0x7fcb900d0be0 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7fcb900d2080 tx=0x7fcb8002a410 comp rx=0 tx=0).stop 2024-09-21T03:40:26.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:26.691+0000 7fcb97f6e640 1 -- 172.21.15.38:0/1242074959 shutdown_connections 2024-09-21T03:40:26.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:26.691+0000 7fcb97f6e640 1 --2- 172.21.15.38:0/1242074959 >> v2:172.21.15.195:3300/0 conn(0x7fcb900d1120 0x7fcb900ceb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:26.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:26.691+0000 7fcb97f6e640 1 --2- 172.21.15.38:0/1242074959 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcb900d0170 0x7fcb900d0be0 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:26.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:26.691+0000 7fcb97f6e640 1 -- 172.21.15.38:0/1242074959 >> 172.21.15.38:0/1242074959 conn(0x7fcb9009d830 msgr2=0x7fcb900b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:26.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:26.691+0000 7fcb97f6e640 1 -- 172.21.15.38:0/1242074959 shutdown_connections 2024-09-21T03:40:26.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:26.691+0000 7fcb97f6e640 1 -- 172.21.15.38:0/1242074959 wait complete. 2024-09-21T03:40:26.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:26.693+0000 7fcb97f6e640 1 Processor -- start 2024-09-21T03:40:26.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:26.693+0000 7fcb97f6e640 1 -- start start 2024-09-21T03:40:26.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:26.694+0000 7fcb97f6e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcb900d0170 0x7fcb901fa190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:26.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:26.694+0000 7fcb97f6e640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcb900d1120 0x7fcb90157ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:26.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:26.694+0000 7fcb97f6e640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcb9015ee80 0x7fcb9006f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:26.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:26.694+0000 7fcb97f6e640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fcb900b6700 con 0x7fcb900d0170 2024-09-21T03:40:26.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:26.694+0000 7fcb97f6e640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fcb900b6580 con 0x7fcb900d1120 2024-09-21T03:40:26.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:26.694+0000 7fcb97f6e640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fcb900b6880 con 0x7fcb9015ee80 2024-09-21T03:40:26.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:26.694+0000 7fcb95d0a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcb900d0170 0x7fcb901fa190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:26.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:26.695+0000 7fcb95d0a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcb900d0170 0x7fcb901fa190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48608/0 (socket says 172.21.15.38:48608) 2024-09-21T03:40:26.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:26.695+0000 7fcb95d0a640 1 -- 172.21.15.38:0/1734202729 learned_addr learned my addr 172.21.15.38:0/1734202729 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:26.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:26.695+0000 7fcb95509640 1 --2- 172.21.15.38:0/1734202729 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcb900d1120 0x7fcb90157ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:26.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:26.695+0000 7fcb9650b640 1 --2- 172.21.15.38:0/1734202729 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcb9015ee80 0x7fcb9006f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:26.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:26.695+0000 7fcb95d0a640 1 -- 172.21.15.38:0/1734202729 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcb9015ee80 msgr2=0x7fcb9006f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:26.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:26.695+0000 7fcb95d0a640 1 --2- 172.21.15.38:0/1734202729 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcb9015ee80 0x7fcb9006f120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:26.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:26.695+0000 7fcb95d0a640 1 -- 172.21.15.38:0/1734202729 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcb900d1120 msgr2=0x7fcb90157ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:26.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:26.695+0000 7fcb95d0a640 1 --2- 172.21.15.38:0/1734202729 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcb900d1120 0x7fcb90157ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:26.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:26.695+0000 7fcb95d0a640 1 -- 172.21.15.38:0/1734202729 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb901580e0 con 0x7fcb900d0170 2024-09-21T03:40:26.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:26.695+0000 7fcb95d0a640 1 --2- 172.21.15.38:0/1734202729 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcb900d0170 0x7fcb901fa190 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7fcb8c00c3a0 tx=0x7fcb8c00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:26.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:26.695+0000 7fcb867fc640 1 -- 172.21.15.38:0/1734202729 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb8c011790 con 0x7fcb900d0170 2024-09-21T03:40:26.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:26.695+0000 7fcb867fc640 1 -- 172.21.15.38:0/1734202729 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcb8c017070 con 0x7fcb900d0170 2024-09-21T03:40:26.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:26.695+0000 7fcb97f6e640 1 -- 172.21.15.38:0/1734202729 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb9006cde0 con 0x7fcb900d0170 2024-09-21T03:40:26.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:26.695+0000 7fcb867fc640 1 -- 172.21.15.38:0/1734202729 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb8c011cc0 con 0x7fcb900d0170 2024-09-21T03:40:26.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:26.695+0000 7fcb97f6e640 1 -- 172.21.15.38:0/1734202729 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb9006d080 con 0x7fcb900d0170 2024-09-21T03:40:26.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:26.696+0000 7fcb97f6e640 1 -- 172.21.15.38:0/1734202729 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb900b6da0 con 0x7fcb900d0170 2024-09-21T03:40:26.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:26.697+0000 7fcb867fc640 1 -- 172.21.15.38:0/1734202729 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fcb8c01c460 con 0x7fcb900d0170 2024-09-21T03:40:26.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:26.697+0000 7fcb867fc640 1 --2- 172.21.15.38:0/1734202729 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fcb74078b90 0x7fcb7407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:26.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:26.698+0000 7fcb867fc640 1 -- 172.21.15.38:0/1734202729 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 6390+0+0 (secure 0 0 0) 0x7fcb8c09adc0 con 0x7fcb900d0170 2024-09-21T03:40:26.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:26.698+0000 7fcb95509640 1 --2- 172.21.15.38:0/1734202729 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fcb74078b90 0x7fcb7407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:26.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:26.698+0000 7fcb95509640 1 --2- 172.21.15.38:0/1734202729 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fcb74078b90 0x7fcb7407b0a0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fcb80034000 tx=0x7fcb80004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:26.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:26.702+0000 7fcb867fc640 1 -- 172.21.15.38:0/1734202729 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fcb901580e0 con 0x7fcb900d0170 2024-09-21T03:40:26.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:26.933+0000 7fcb97f6e640 1 -- 172.21.15.38:0/1734202729 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fcb900d4f20 con 0x7fcb900d0170 2024-09-21T03:40:26.935 INFO:tasks.ceph.mon.a.smithi038.stderr:2024-09-21T03:40:26.934+0000 7f3f45713640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T03:40:26.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:26.938+0000 7fcb867fc640 1 -- 172.21.15.38:0/1734202729 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v93) ==== 161+0+0 (secure 0 0 0) 0x7fcb901592d0 con 0x7fcb900d0170 2024-09-21T03:40:26.939 INFO:teuthology.orchestra.run.smithi038.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-21T03:40:26.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:26.940+0000 7fcb97f6e640 1 -- 172.21.15.38:0/1734202729 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fcb74078b90 msgr2=0x7fcb7407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:26.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:26.940+0000 7fcb97f6e640 1 --2- 172.21.15.38:0/1734202729 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fcb74078b90 0x7fcb7407b0a0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fcb80034000 tx=0x7fcb80004730 comp rx=0 tx=0).stop 2024-09-21T03:40:26.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:26.940+0000 7fcb97f6e640 1 -- 172.21.15.38:0/1734202729 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcb900d0170 msgr2=0x7fcb901fa190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:26.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:26.940+0000 7fcb97f6e640 1 --2- 172.21.15.38:0/1734202729 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcb900d0170 0x7fcb901fa190 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7fcb8c00c3a0 tx=0x7fcb8c00c860 comp rx=0 tx=0).stop 2024-09-21T03:40:26.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:26.940+0000 7fcb97f6e640 1 -- 172.21.15.38:0/1734202729 shutdown_connections 2024-09-21T03:40:26.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:26.940+0000 7fcb97f6e640 1 --2- 172.21.15.38:0/1734202729 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fcb74078b90 0x7fcb7407b0a0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:26.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:26.940+0000 7fcb97f6e640 1 --2- 172.21.15.38:0/1734202729 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcb9015ee80 0x7fcb9006f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:26.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:26.940+0000 7fcb97f6e640 1 --2- 172.21.15.38:0/1734202729 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcb900d1120 0x7fcb90157ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:26.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:26.940+0000 7fcb97f6e640 1 --2- 172.21.15.38:0/1734202729 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcb900d0170 0x7fcb901fa190 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:26.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:26.940+0000 7fcb97f6e640 1 -- 172.21.15.38:0/1734202729 >> 172.21.15.38:0/1734202729 conn(0x7fcb9009d830 msgr2=0x7fcb9015e030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:26.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:26.940+0000 7fcb97f6e640 1 -- 172.21.15.38:0/1734202729 shutdown_connections 2024-09-21T03:40:26.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:26.940+0000 7fcb97f6e640 1 -- 172.21.15.38:0/1734202729 wait complete. 2024-09-21T03:40:26.945 INFO:tasks.ceph.mds.a.smithi038.stderr: -530> 2024-09-21T03:40:24.866+0000 7fa039119640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T03:40:26.945 INFO:tasks.ceph.mds.a.smithi038.stderr: -134> 2024-09-21T03:40:25.941+0000 7fa035111640 -1 log_channel(cluster) log [ERR] : dir 0x10000000000 object missing on disk; some files may be lost (/subdir) 2024-09-21T03:40:26.955 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-09-21T03:40:26.999 INFO:tasks.ceph.mds.a.smithi038.stdout:starting mds.a at 2024-09-21T03:40:27.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:27.074+0000 7ff1baf32640 1 -- 172.21.15.38:0/1373930209 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff1b40d0bb0 msgr2=0x7ff1b40b5be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:27.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:27.074+0000 7ff1baf32640 1 --2- 172.21.15.38:0/1373930209 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff1b40d0bb0 0x7ff1b40b5be0 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7ff1a4009830 tx=0x7ff1a402a480 comp rx=0 tx=0).stop 2024-09-21T03:40:27.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:27.074+0000 7ff1baf32640 1 -- 172.21.15.38:0/1373930209 shutdown_connections 2024-09-21T03:40:27.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:27.074+0000 7ff1baf32640 1 --2- 172.21.15.38:0/1373930209 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff1b40d0bb0 0x7ff1b40b5be0 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:27.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:27.074+0000 7ff1baf32640 1 --2- 172.21.15.38:0/1373930209 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff1b415ca90 0x7ff1b40d0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:27.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:27.074+0000 7ff1baf32640 1 -- 172.21.15.38:0/1373930209 >> 172.21.15.38:0/1373930209 conn(0x7ff1b40b46e0 msgr2=0x7ff1b40d23e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:27.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:27.074+0000 7ff1baf32640 1 -- 172.21.15.38:0/1373930209 shutdown_connections 2024-09-21T03:40:27.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:27.074+0000 7ff1baf32640 1 -- 172.21.15.38:0/1373930209 wait complete. 2024-09-21T03:40:27.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:27.076+0000 7ff1baf32640 1 Processor -- start 2024-09-21T03:40:27.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:27.076+0000 7ff1baf32640 1 -- start start 2024-09-21T03:40:27.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:27.077+0000 7ff1baf32640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff1b40ccc40 0x7ff1b41e4c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:27.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:27.077+0000 7ff1baf32640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff1b40d0bb0 0x7ff1b40ca7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:27.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:27.077+0000 7ff1baf32640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff1b415ca90 0x7ff1b40cad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:27.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:27.077+0000 7ff1baf32640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ff1b40b6620 con 0x7ff1b40d0bb0 2024-09-21T03:40:27.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:27.077+0000 7ff1baf32640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ff1b40b64a0 con 0x7ff1b40ccc40 2024-09-21T03:40:27.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:27.077+0000 7ff1baf32640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7ff1b40b67a0 con 0x7ff1b415ca90 2024-09-21T03:40:27.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:27.077+0000 7ff1b3fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff1b40d0bb0 0x7ff1b40ca7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:27.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:27.077+0000 7ff1b3fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff1b40d0bb0 0x7ff1b40ca7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48642/0 (socket says 172.21.15.38:48642) 2024-09-21T03:40:27.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:27.077+0000 7ff1b3fff640 1 -- 172.21.15.38:0/2116591386 learned_addr learned my addr 172.21.15.38:0/2116591386 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:27.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:27.078+0000 7ff1b94cf640 1 --2- 172.21.15.38:0/2116591386 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff1b415ca90 0x7ff1b40cad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:27.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:27.078+0000 7ff1b8cce640 1 --2- 172.21.15.38:0/2116591386 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff1b40ccc40 0x7ff1b41e4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:27.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:27.078+0000 7ff1b3fff640 1 -- 172.21.15.38:0/2116591386 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff1b415ca90 msgr2=0x7ff1b40cad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:27.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:27.078+0000 7ff1b3fff640 1 --2- 172.21.15.38:0/2116591386 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff1b415ca90 0x7ff1b40cad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:27.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:27.078+0000 7ff1b3fff640 1 -- 172.21.15.38:0/2116591386 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff1b40ccc40 msgr2=0x7ff1b41e4c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:27.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:27.078+0000 7ff1b3fff640 1 --2- 172.21.15.38:0/2116591386 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff1b40ccc40 0x7ff1b41e4c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:27.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:27.078+0000 7ff1b3fff640 1 -- 172.21.15.38:0/2116591386 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1b41e7b50 con 0x7ff1b40d0bb0 2024-09-21T03:40:27.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:27.078+0000 7ff1b3fff640 1 --2- 172.21.15.38:0/2116591386 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff1b40d0bb0 0x7ff1b40ca7c0 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7ff1a8009fd0 tx=0x7ff1a800ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:27.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:27.078+0000 7ff1b17fa640 1 -- 172.21.15.38:0/2116591386 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1a800cc60 con 0x7ff1b40d0bb0 2024-09-21T03:40:27.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:27.078+0000 7ff1b17fa640 1 -- 172.21.15.38:0/2116591386 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff1a800ce00 con 0x7ff1b40d0bb0 2024-09-21T03:40:27.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:27.078+0000 7ff1baf32640 1 -- 172.21.15.38:0/2116591386 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1b41e6fd0 con 0x7ff1b40d0bb0 2024-09-21T03:40:27.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:27.078+0000 7ff1b17fa640 1 -- 172.21.15.38:0/2116591386 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1a8004280 con 0x7ff1b40d0bb0 2024-09-21T03:40:27.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:27.078+0000 7ff1baf32640 1 -- 172.21.15.38:0/2116591386 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1b41e7590 con 0x7ff1b40d0bb0 2024-09-21T03:40:27.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:27.079+0000 7ff1baf32640 1 -- 172.21.15.38:0/2116591386 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1b41e6d40 con 0x7ff1b40d0bb0 2024-09-21T03:40:27.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:27.080+0000 7ff1b17fa640 1 -- 172.21.15.38:0/2116591386 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ff1a80035f0 con 0x7ff1b40d0bb0 2024-09-21T03:40:27.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:27.081+0000 7ff1b17fa640 1 --2- 172.21.15.38:0/2116591386 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff1a0078b90 0x7ff1a007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:27.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:27.081+0000 7ff1b17fa640 1 -- 172.21.15.38:0/2116591386 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 6476+0+0 (secure 0 0 0) 0x7ff1a809b750 con 0x7ff1b40d0bb0 2024-09-21T03:40:27.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:27.081+0000 7ff1b8cce640 1 --2- 172.21.15.38:0/2116591386 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff1a0078b90 0x7ff1a007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:27.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:27.082+0000 7ff1b8cce640 1 --2- 172.21.15.38:0/2116591386 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff1a0078b90 0x7ff1a007b0a0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7ff1ac002200 tx=0x7ff1ac0019f0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:27.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:27.086+0000 7ff1b17fa640 1 -- 172.21.15.38:0/2116591386 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7ff1a80632d0 con 0x7ff1b40d0bb0 2024-09-21T03:40:27.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:27.351+0000 7ff1baf32640 1 -- 172.21.15.38:0/2116591386 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7ff1b40b6d30 con 0x7ff1b40d0bb0 2024-09-21T03:40:27.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:27.945+0000 7ff1b17fa640 1 -- 172.21.15.38:0/2116591386 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 nothing to do: rank is not damaged v94) ==== 107+0+0 (secure 0 0 0) 0x7ff1a8067d00 con 0x7ff1b40d0bb0 2024-09-21T03:40:27.948 INFO:teuthology.orchestra.run.smithi038.stderr:nothing to do: rank is not damaged 2024-09-21T03:40:27.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:27.949+0000 7ff1baf32640 1 -- 172.21.15.38:0/2116591386 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff1a0078b90 msgr2=0x7ff1a007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:27.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:27.949+0000 7ff1baf32640 1 --2- 172.21.15.38:0/2116591386 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff1a0078b90 0x7ff1a007b0a0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7ff1ac002200 tx=0x7ff1ac0019f0 comp rx=0 tx=0).stop 2024-09-21T03:40:27.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:27.949+0000 7ff1baf32640 1 -- 172.21.15.38:0/2116591386 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff1b40d0bb0 msgr2=0x7ff1b40ca7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:27.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:27.949+0000 7ff1baf32640 1 --2- 172.21.15.38:0/2116591386 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff1b40d0bb0 0x7ff1b40ca7c0 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7ff1a8009fd0 tx=0x7ff1a800ec20 comp rx=0 tx=0).stop 2024-09-21T03:40:27.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:27.949+0000 7ff1baf32640 1 -- 172.21.15.38:0/2116591386 shutdown_connections 2024-09-21T03:40:27.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:27.949+0000 7ff1baf32640 1 --2- 172.21.15.38:0/2116591386 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff1a0078b90 0x7ff1a007b0a0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:27.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:27.949+0000 7ff1baf32640 1 --2- 172.21.15.38:0/2116591386 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff1b415ca90 0x7ff1b40cad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:27.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:27.949+0000 7ff1baf32640 1 --2- 172.21.15.38:0/2116591386 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff1b40d0bb0 0x7ff1b40ca7c0 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:27.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:27.949+0000 7ff1baf32640 1 --2- 172.21.15.38:0/2116591386 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff1b40ccc40 0x7ff1b41e4c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:27.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:27.949+0000 7ff1baf32640 1 -- 172.21.15.38:0/2116591386 >> 172.21.15.38:0/2116591386 conn(0x7ff1b40b46e0 msgr2=0x7ff1b40cd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:27.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:27.949+0000 7ff1baf32640 1 -- 172.21.15.38:0/2116591386 shutdown_connections 2024-09-21T03:40:27.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:27.949+0000 7ff1baf32640 1 -- 172.21.15.38:0/2116591386 wait complete. 2024-09-21T03:40:27.966 DEBUG:teuthology.orchestra.run.smithi038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-09-21T03:40:28.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.024+0000 7f93c340ef40 1 Processor -- start 2024-09-21T03:40:28.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.024+0000 7f93c340ef40 1 -- start start 2024-09-21T03:40:28.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.026+0000 7f93c340ef40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555db5f17090 0x555db5fa0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:28.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.026+0000 7f93c340ef40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555db5f8c520 0x555db5ecfdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:28.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.026+0000 7f93c340ef40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555db5ef9150 0x555db5ef9560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:28.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.026+0000 7f93c340ef40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x555db5e7c430 con 0x555db5ef9150 2024-09-21T03:40:28.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.026+0000 7f93c340ef40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x555db5e891f0 con 0x555db5f8c520 2024-09-21T03:40:28.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.026+0000 7f93c340ef40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x555db5e1dd60 con 0x555db5f17090 2024-09-21T03:40:28.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.026+0000 7f93bc95c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555db5ef9150 0x555db5ef9560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:28.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.026+0000 7f93bc95c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555db5ef9150 0x555db5ef9560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48662/0 (socket says 172.21.15.38:48662) 2024-09-21T03:40:28.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.026+0000 7f93bc95c640 1 -- 172.21.15.38:0/732108489 learned_addr learned my addr 172.21.15.38:0/732108489 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:28.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.026+0000 7f93bc15b640 1 --2- 172.21.15.38:0/732108489 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555db5f17090 0x555db5fa0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:28.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.026+0000 7f93bb95a640 1 --2- 172.21.15.38:0/732108489 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555db5f8c520 0x555db5ecfdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:28.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.026+0000 7f93bc95c640 1 -- 172.21.15.38:0/732108489 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555db5f17090 msgr2=0x555db5fa0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:28.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.026+0000 7f93bc95c640 1 --2- 172.21.15.38:0/732108489 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555db5f17090 0x555db5fa0430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:28.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.026+0000 7f93bc95c640 1 -- 172.21.15.38:0/732108489 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555db5f8c520 msgr2=0x555db5ecfdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:28.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.026+0000 7f93bc95c640 1 --2- 172.21.15.38:0/732108489 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555db5f8c520 0x555db5ecfdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:28.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.026+0000 7f93bc95c640 1 -- 172.21.15.38:0/732108489 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555db5f9dcf0 con 0x555db5ef9150 2024-09-21T03:40:28.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.026+0000 7f93bc95c640 1 --2- 172.21.15.38:0/732108489 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555db5ef9150 0x555db5ef9560 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f93b4001540 tx=0x7f93b4001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce6eaeee4f8600ae server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:28.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.026+0000 7f93ba958640 1 -- 172.21.15.38:0/732108489 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93b4017070 con 0x555db5ef9150 2024-09-21T03:40:28.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.026+0000 7f93ba958640 1 -- 172.21.15.38:0/732108489 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f93b40027a0 con 0x555db5ef9150 2024-09-21T03:40:28.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.026+0000 7f93ba958640 1 -- 172.21.15.38:0/732108489 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93b4002c30 con 0x555db5ef9150 2024-09-21T03:40:28.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.027+0000 7f93c340ef40 1 -- 172.21.15.38:0/732108489 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555db5ef9150 msgr2=0x555db5ef9560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:28.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.027+0000 7f93c340ef40 1 --2- 172.21.15.38:0/732108489 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555db5ef9150 0x555db5ef9560 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f93b4001540 tx=0x7f93b4001a00 comp rx=0 tx=0).stop 2024-09-21T03:40:28.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.027+0000 7f93c340ef40 1 -- 172.21.15.38:0/732108489 shutdown_connections 2024-09-21T03:40:28.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.027+0000 7f93c340ef40 1 --2- 172.21.15.38:0/732108489 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555db5ef9150 0x555db5ef9560 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:28.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.027+0000 7f93c340ef40 1 --2- 172.21.15.38:0/732108489 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555db5f8c520 0x555db5ecfdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:28.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.027+0000 7f93c340ef40 1 --2- 172.21.15.38:0/732108489 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555db5f17090 0x555db5fa0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:28.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.027+0000 7f93c340ef40 1 -- 172.21.15.38:0/732108489 >> 172.21.15.38:0/732108489 conn(0x555db5deeac0 msgr2=0x555db5f15c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:28.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.027+0000 7f93c340ef40 1 -- 172.21.15.38:0/732108489 shutdown_connections 2024-09-21T03:40:28.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.027+0000 7f93c340ef40 1 -- 172.21.15.38:0/732108489 wait complete. 2024-09-21T03:40:28.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.028+0000 7f93c340ef40 1 Processor -- start 2024-09-21T03:40:28.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.028+0000 7f93c340ef40 1 -- start start 2024-09-21T03:40:28.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.029+0000 7f93c340ef40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555db5ef9150 0x555db5f0f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:28.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.029+0000 7f93c340ef40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555db5f17090 0x555db5f88d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:28.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.029+0000 7f93c340ef40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555db5f8c520 0x555db5f892a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:28.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.029+0000 7f93c340ef40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x555db5f9d160 con 0x555db5f8c520 2024-09-21T03:40:28.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.029+0000 7f93c340ef40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x555db5f9cd40 con 0x555db5f17090 2024-09-21T03:40:28.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.029+0000 7f93c340ef40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x555db5f9d380 con 0x555db5ef9150 2024-09-21T03:40:28.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.029+0000 7f93bc95c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555db5f8c520 0x555db5f892a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:28.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.030+0000 7f93bc95c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555db5f8c520 0x555db5f892a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48676/0 (socket says 172.21.15.38:48676) 2024-09-21T03:40:28.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.030+0000 7f93bc95c640 1 -- 172.21.15.38:0/2579886557 learned_addr learned my addr 172.21.15.38:0/2579886557 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:28.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.030+0000 7f93bc15b640 1 --2- 172.21.15.38:0/2579886557 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555db5ef9150 0x555db5f0f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:28.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.030+0000 7f93bb95a640 1 --2- 172.21.15.38:0/2579886557 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555db5f17090 0x555db5f88d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:28.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.030+0000 7f93bc95c640 1 -- 172.21.15.38:0/2579886557 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555db5ef9150 msgr2=0x555db5f0f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:28.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.030+0000 7f93bc95c640 1 --2- 172.21.15.38:0/2579886557 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555db5ef9150 0x555db5f0f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:28.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.030+0000 7f93bc95c640 1 -- 172.21.15.38:0/2579886557 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555db5f17090 msgr2=0x555db5f88d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:28.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.030+0000 7f93bc95c640 1 --2- 172.21.15.38:0/2579886557 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555db5f17090 0x555db5f88d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:28.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.030+0000 7f93bc95c640 1 -- 172.21.15.38:0/2579886557 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555db5f15d50 con 0x555db5f8c520 2024-09-21T03:40:28.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.030+0000 7f93bb95a640 1 --2- 172.21.15.38:0/2579886557 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555db5f17090 0x555db5f88d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:40:28.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.030+0000 7f93bc95c640 1 --2- 172.21.15.38:0/2579886557 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555db5f8c520 0x555db5f892a0 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f93b4001f00 tx=0x7f93b4003000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:28.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.030+0000 7f93b9956640 1 -- 172.21.15.38:0/2579886557 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93b4022030 con 0x555db5f8c520 2024-09-21T03:40:28.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.030+0000 7f93b9956640 1 -- 172.21.15.38:0/2579886557 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f93b401e7f0 con 0x555db5f8c520 2024-09-21T03:40:28.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.030+0000 7f93b9956640 1 -- 172.21.15.38:0/2579886557 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93b4017030 con 0x555db5f8c520 2024-09-21T03:40:28.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.030+0000 7f93c340ef40 1 -- 172.21.15.38:0/2579886557 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x555db5f16030 con 0x555db5f8c520 2024-09-21T03:40:28.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.030+0000 7f93c340ef40 1 -- 172.21.15.38:0/2579886557 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555db5f0f990 con 0x555db5f8c520 2024-09-21T03:40:28.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.032+0000 7f93b9956640 1 -- 172.21.15.38:0/2579886557 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f93b4025070 con 0x555db5f8c520 2024-09-21T03:40:28.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.033+0000 7f93b9956640 1 --2- 172.21.15.38:0/2579886557 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f93a80775a0 0x7f93a8079ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:28.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.033+0000 7f93b9956640 1 -- 172.21.15.38:0/2579886557 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 6476+0+0 (secure 0 0 0) 0x7f93b409a590 con 0x555db5f8c520 2024-09-21T03:40:28.037 INFO:teuthology.orchestra.run.smithi038.stdout:Importing pool 2024-09-21T03:40:28.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.033+0000 7f93bc15b640 1 --2- 172.21.15.38:0/2579886557 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f93a80775a0 0x7f93a8079ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:28.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.033+0000 7f93c340ef40 1 --2- 172.21.15.38:0/2579886557 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x555db603a280 0x555db603a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:28.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.033+0000 7f93c340ef40 1 -- 172.21.15.38:0/2579886557 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x555db603abd0 con 0x555db603a280 2024-09-21T03:40:28.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.033+0000 7f93bb95a640 1 --2- 172.21.15.38:0/2579886557 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x555db603a280 0x555db603a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:28.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.034+0000 7f93bc15b640 1 --2- 172.21.15.38:0/2579886557 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f93a80775a0 0x7f93a8079ab0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f93ac055000 tx=0x7f93ac058040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:28.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.034+0000 7f93bb95a640 1 --2- 172.21.15.38:0/2579886557 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x555db603a280 0x555db603a690 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:28.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.035+0000 7f93bb95a640 1 -- 172.21.15.38:0/2579886557 <== osd.7 v2:172.21.15.195:6818/1606106342 1 ==== osd_op_reply(1 601.00000000 [create] v98'43 uv16 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f93b000e550 con 0x555db603a280 2024-09-21T03:40:28.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.035+0000 7f93c340ef40 1 -- 172.21.15.38:0/2579886557 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e98) -- 0x555db603afa0 con 0x555db603a280 2024-09-21T03:40:28.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.037+0000 7f93bb95a640 1 -- 172.21.15.38:0/2579886557 <== osd.7 v2:172.21.15.195:6818/1606106342 2 ==== osd_op_reply(2 601.00000000 [delete] v98'44 uv44 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f93b000e550 con 0x555db603a280 2024-09-21T03:40:28.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.037+0000 7f93c340ef40 1 -- 172.21.15.38:0/2579886557 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x555db6046be0 con 0x555db603a280 2024-09-21T03:40:28.040 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-09-21T03:40:28.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.038+0000 7f93bb95a640 1 -- 172.21.15.38:0/2579886557 <== osd.7 v2:172.21.15.195:6818/1606106342 3 ==== osd_op_reply(3 601.00000000 [create] v98'45 uv45 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f93b000e550 con 0x555db603a280 2024-09-21T03:40:28.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.038+0000 7f93c340ef40 1 -- 172.21.15.38:0/2579886557 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x555db6047f90 con 0x555db603a280 2024-09-21T03:40:28.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.039+0000 7f93bb95a640 1 -- 172.21.15.38:0/2579886557 <== osd.7 v2:172.21.15.195:6818/1606106342 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v98'46 uv46 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f93b000e550 con 0x555db603a280 2024-09-21T03:40:28.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.039+0000 7f93c340ef40 1 -- 172.21.15.38:0/2579886557 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x555db6048330 con 0x555db603a280 2024-09-21T03:40:28.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.041+0000 7f93bb95a640 1 -- 172.21.15.38:0/2579886557 <== osd.7 v2:172.21.15.195:6818/1606106342 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v98'47 uv47 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f93b000e550 con 0x555db603a280 2024-09-21T03:40:28.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.041+0000 7f93c340ef40 1 -- 172.21.15.38:0/2579886557 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x555db6046f80 con 0x555db603a280 2024-09-21T03:40:28.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.042+0000 7f93bb95a640 1 -- 172.21.15.38:0/2579886557 <== osd.7 v2:172.21.15.195:6818/1606106342 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v98'48 uv48 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f93b000e550 con 0x555db603a280 2024-09-21T03:40:28.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.042+0000 7f93c340ef40 1 -- 172.21.15.38:0/2579886557 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x555db6047370 con 0x555db603a280 2024-09-21T03:40:28.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.044+0000 7f93bb95a640 1 -- 172.21.15.38:0/2579886557 <== osd.7 v2:172.21.15.195:6818/1606106342 7 ==== osd_op_reply(7 602.00000000 [create] v98'49 uv22 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f93b000e550 con 0x555db603a280 2024-09-21T03:40:28.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.044+0000 7f93c340ef40 1 -- 172.21.15.38:0/2579886557 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e98) -- 0x555db6047760 con 0x555db603a280 2024-09-21T03:40:28.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.045+0000 7f93bb95a640 1 -- 172.21.15.38:0/2579886557 <== osd.7 v2:172.21.15.195:6818/1606106342 8 ==== osd_op_reply(8 602.00000000 [delete] v98'50 uv50 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f93b000e550 con 0x555db603a280 2024-09-21T03:40:28.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.045+0000 7f93c340ef40 1 -- 172.21.15.38:0/2579886557 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x555db6047760 con 0x555db603a280 2024-09-21T03:40:28.048 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-09-21T03:40:28.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.046+0000 7f93bb95a640 1 -- 172.21.15.38:0/2579886557 <== osd.7 v2:172.21.15.195:6818/1606106342 9 ==== osd_op_reply(9 602.00000000 [create] v98'51 uv51 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f93b000e550 con 0x555db603a280 2024-09-21T03:40:28.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.047+0000 7f93c340ef40 1 -- 172.21.15.38:0/2579886557 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x555db6047710 con 0x555db603a280 2024-09-21T03:40:28.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.048+0000 7f93bb95a640 1 -- 172.21.15.38:0/2579886557 <== osd.7 v2:172.21.15.195:6818/1606106342 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v98'52 uv52 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f93b000e550 con 0x555db603a280 2024-09-21T03:40:28.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.048+0000 7f93c340ef40 1 -- 172.21.15.38:0/2579886557 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x555db6047710 con 0x555db603a280 2024-09-21T03:40:28.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.049+0000 7f93bb95a640 1 -- 172.21.15.38:0/2579886557 <== osd.7 v2:172.21.15.195:6818/1606106342 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v98'53 uv53 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f93b000e550 con 0x555db603a280 2024-09-21T03:40:28.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.049+0000 7f93c340ef40 1 -- 172.21.15.38:0/2579886557 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x555db6047710 con 0x555db603a280 2024-09-21T03:40:28.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.051+0000 7f93bb95a640 1 -- 172.21.15.38:0/2579886557 <== osd.7 v2:172.21.15.195:6818/1606106342 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v98'54 uv54 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f93b000e550 con 0x555db603a280 2024-09-21T03:40:28.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.051+0000 7f93c340ef40 1 -- 172.21.15.38:0/2579886557 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:13 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x555db6047710 con 0x555db603a280 2024-09-21T03:40:28.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.052+0000 7f93bb95a640 1 -- 172.21.15.38:0/2579886557 <== osd.7 v2:172.21.15.195:6818/1606106342 13 ==== osd_op_reply(13 10000000000.00000000 [create] v98'55 uv55 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f93b000e550 con 0x555db603a280 2024-09-21T03:40:28.054 INFO:teuthology.orchestra.run.smithi038.stdout:Write #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-09-21T03:40:28.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.052+0000 7f93c340ef40 1 -- 172.21.15.38:0/2579886557 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:14 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x555db6047710 con 0x555db603a280 2024-09-21T03:40:28.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.053+0000 7f93bb95a640 1 -- 172.21.15.38:0/2579886557 <== osd.7 v2:172.21.15.195:6818/1606106342 14 ==== osd_op_reply(14 10000000000.00000000 [setxattr (30)] v98'56 uv56 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f93b000e550 con 0x555db603a280 2024-09-21T03:40:28.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.053+0000 7f93c340ef40 1 -- 172.21.15.38:0/2579886557 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:15 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x555db6047710 con 0x555db603a280 2024-09-21T03:40:28.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.055+0000 7f93bb95a640 1 -- 172.21.15.38:0/2579886557 <== osd.7 v2:172.21.15.195:6818/1606106342 15 ==== osd_op_reply(15 10000000000.00000000 [setxattr (62)] v98'57 uv57 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f93b000e550 con 0x555db603a280 2024-09-21T03:40:28.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.055+0000 7f93c340ef40 1 -- 172.21.15.38:0/2579886557 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:16 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x555db6047710 con 0x555db603a280 2024-09-21T03:40:28.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.056+0000 7f93bb95a640 1 -- 172.21.15.38:0/2579886557 <== osd.7 v2:172.21.15.195:6818/1606106342 16 ==== osd_op_reply(16 10000000000.00000000 [omap-set-header] v98'58 uv58 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f93b000e550 con 0x555db603a280 2024-09-21T03:40:28.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.056+0000 7f93c340ef40 1 -- 172.21.15.38:0/2579886557 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:17 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=585b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x555db60486d0 con 0x555db603a280 2024-09-21T03:40:28.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.057+0000 7f93bb95a640 1 -- 172.21.15.38:0/2579886557 <== osd.7 v2:172.21.15.195:6818/1606106342 17 ==== osd_op_reply(17 10000000000.00000000 [omap-set-vals] v98'59 uv59 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f93b000e550 con 0x555db603a280 2024-09-21T03:40:28.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.057+0000 7f93c340ef40 1 -- 172.21.15.38:0/2579886557 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:18 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x555db60486d0 con 0x555db603a280 2024-09-21T03:40:28.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.058+0000 7f93bb95a640 1 -- 172.21.15.38:0/2579886557 <== osd.7 v2:172.21.15.195:6818/1606106342 18 ==== osd_op_reply(18 600.00000000 [create] v98'60 uv35 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f93b000e550 con 0x555db603a280 2024-09-21T03:40:28.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.058+0000 7f93c340ef40 1 -- 172.21.15.38:0/2579886557 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:19 13.10 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e98) -- 0x555db60486d0 con 0x555db603a280 2024-09-21T03:40:28.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.060+0000 7f93bb95a640 1 -- 172.21.15.38:0/2579886557 <== osd.7 v2:172.21.15.195:6818/1606106342 19 ==== osd_op_reply(19 600.00000000 [delete] v98'61 uv61 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f93b000e550 con 0x555db603a280 2024-09-21T03:40:28.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.060+0000 7f93c340ef40 1 -- 172.21.15.38:0/2579886557 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:20 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x555db60486d0 con 0x555db603a280 2024-09-21T03:40:28.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.061+0000 7f93bb95a640 1 -- 172.21.15.38:0/2579886557 <== osd.7 v2:172.21.15.195:6818/1606106342 20 ==== osd_op_reply(20 600.00000000 [create] v98'62 uv62 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f93b000e550 con 0x555db603a280 2024-09-21T03:40:28.063 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-09-21T03:40:28.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.061+0000 7f93c340ef40 1 -- 172.21.15.38:0/2579886557 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:21 13.10 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x555db60496e0 con 0x555db603a280 2024-09-21T03:40:28.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.062+0000 7f93bb95a640 1 -- 172.21.15.38:0/2579886557 <== osd.7 v2:172.21.15.195:6818/1606106342 21 ==== osd_op_reply(21 600.00000000 [setxattr (30)] v98'63 uv63 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f93b000e550 con 0x555db603a280 2024-09-21T03:40:28.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.062+0000 7f93c340ef40 1 -- 172.21.15.38:0/2579886557 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:22 13.10 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x555db60496e0 con 0x555db603a280 2024-09-21T03:40:28.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.063+0000 7f93bb95a640 1 -- 172.21.15.38:0/2579886557 <== osd.7 v2:172.21.15.195:6818/1606106342 22 ==== osd_op_reply(22 600.00000000 [setxattr (62)] v98'64 uv64 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f93b000e550 con 0x555db603a280 2024-09-21T03:40:28.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.063+0000 7f93c340ef40 1 -- 172.21.15.38:0/2579886557 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:23 13.10 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x555db60486d0 con 0x555db603a280 2024-09-21T03:40:28.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.064+0000 7f93bb95a640 1 -- 172.21.15.38:0/2579886557 <== osd.7 v2:172.21.15.195:6818/1606106342 23 ==== osd_op_reply(23 600.00000000 [omap-set-header] v98'65 uv65 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f93b000e550 con 0x555db603a280 2024-09-21T03:40:28.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.065+0000 7f93c340ef40 1 -- 172.21.15.38:0/2579886557 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:24 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x555db60486d0 con 0x555db603a280 2024-09-21T03:40:28.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.066+0000 7f93bb95a640 1 -- 172.21.15.38:0/2579886557 <== osd.7 v2:172.21.15.195:6818/1606106342 24 ==== osd_op_reply(24 603.00000000 [create] v98'66 uv41 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f93b000e550 con 0x555db603a280 2024-09-21T03:40:28.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.066+0000 7f93c340ef40 1 -- 172.21.15.38:0/2579886557 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:25 13.10 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e98) -- 0x555db60486d0 con 0x555db603a280 2024-09-21T03:40:28.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.067+0000 7f93bb95a640 1 -- 172.21.15.38:0/2579886557 <== osd.7 v2:172.21.15.195:6818/1606106342 25 ==== osd_op_reply(25 603.00000000 [delete] v98'67 uv67 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f93b000e550 con 0x555db603a280 2024-09-21T03:40:28.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.067+0000 7f93c340ef40 1 -- 172.21.15.38:0/2579886557 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:26 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x555db60486d0 con 0x555db603a280 2024-09-21T03:40:28.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.068+0000 7f93bb95a640 1 -- 172.21.15.38:0/2579886557 <== osd.7 v2:172.21.15.195:6818/1606106342 26 ==== osd_op_reply(26 603.00000000 [create] v98'68 uv68 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f93b000e550 con 0x555db603a280 2024-09-21T03:40:28.070 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-09-21T03:40:28.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.068+0000 7f93c340ef40 1 -- 172.21.15.38:0/2579886557 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:27 13.10 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x555db60496e0 con 0x555db603a280 2024-09-21T03:40:28.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.070+0000 7f93bb95a640 1 -- 172.21.15.38:0/2579886557 <== osd.7 v2:172.21.15.195:6818/1606106342 27 ==== osd_op_reply(27 603.00000000 [setxattr (30)] v98'69 uv69 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f93b000e550 con 0x555db603a280 2024-09-21T03:40:28.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.070+0000 7f93c340ef40 1 -- 172.21.15.38:0/2579886557 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:28 13.10 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x555db60496e0 con 0x555db603a280 2024-09-21T03:40:28.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.071+0000 7f93bb95a640 1 -- 172.21.15.38:0/2579886557 <== osd.7 v2:172.21.15.195:6818/1606106342 28 ==== osd_op_reply(28 603.00000000 [setxattr (62)] v98'70 uv70 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f93b000e550 con 0x555db603a280 2024-09-21T03:40:28.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.071+0000 7f93c340ef40 1 -- 172.21.15.38:0/2579886557 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:29 13.10 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x555db60486d0 con 0x555db603a280 2024-09-21T03:40:28.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.072+0000 7f93bb95a640 1 -- 172.21.15.38:0/2579886557 <== osd.7 v2:172.21.15.195:6818/1606106342 29 ==== osd_op_reply(29 603.00000000 [omap-set-header] v98'71 uv71 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f93b000e550 con 0x555db603a280 2024-09-21T03:40:28.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.072+0000 7f93c340ef40 1 --2- 172.21.15.38:0/2579886557 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x555db6048be0 0x555db604b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:28.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.072+0000 7f93c340ef40 1 -- 172.21.15.38:0/2579886557 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:30 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x555db604b540 con 0x555db6048be0 2024-09-21T03:40:28.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.072+0000 7f93bc15b640 1 --2- 172.21.15.38:0/2579886557 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x555db6048be0 0x555db604b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:28.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.073+0000 7f93bc15b640 1 --2- 172.21.15.38:0/2579886557 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x555db6048be0 0x555db604b000 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:28.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.074+0000 7f93bc15b640 1 -- 172.21.15.38:0/2579886557 <== osd.6 v2:172.21.15.195:6826/4055825675 1 ==== osd_op_reply(30 1.00000000.inode [create] v98'44 uv23 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f93ac0654f0 con 0x555db6048be0 2024-09-21T03:40:28.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.074+0000 7f93c340ef40 1 -- 172.21.15.38:0/2579886557 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:31 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e98) -- 0x555db604b8e0 con 0x555db6048be0 2024-09-21T03:40:28.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.076+0000 7f93bc15b640 1 -- 172.21.15.38:0/2579886557 <== osd.6 v2:172.21.15.195:6826/4055825675 2 ==== osd_op_reply(31 1.00000000.inode [delete] v98'45 uv45 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f93ac0654f0 con 0x555db6048be0 2024-09-21T03:40:28.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.076+0000 7f93c340ef40 1 -- 172.21.15.38:0/2579886557 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x555db604bc80 con 0x555db6048be0 2024-09-21T03:40:28.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.077+0000 7f93bc15b640 1 -- 172.21.15.38:0/2579886557 <== osd.6 v2:172.21.15.195:6826/4055825675 3 ==== osd_op_reply(32 1.00000000.inode [create] v98'46 uv46 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f93ac0654f0 con 0x555db6048be0 2024-09-21T03:40:28.079 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-09-21T03:40:28.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.077+0000 7f93c340ef40 1 -- 172.21.15.38:0/2579886557 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x555db604c4d0 con 0x555db6048be0 2024-09-21T03:40:28.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.078+0000 7f93bc15b640 1 -- 172.21.15.38:0/2579886557 <== osd.6 v2:172.21.15.195:6826/4055825675 4 ==== osd_op_reply(33 1.00000000.inode [write 0~582] v98'47 uv47 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f93ac0654f0 con 0x555db6048be0 2024-09-21T03:40:28.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.078+0000 7f93c340ef40 1 -- 172.21.15.38:0/2579886557 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x555db604c870 con 0x555db6048be0 2024-09-21T03:40:28.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.080+0000 7f93bc15b640 1 -- 172.21.15.38:0/2579886557 <== osd.6 v2:172.21.15.195:6826/4055825675 5 ==== osd_op_reply(34 1.00000000.inode [omap-set-header] v98'48 uv48 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f93ac0654f0 con 0x555db6048be0 2024-09-21T03:40:28.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.080+0000 7f93c340ef40 1 -- 172.21.15.38:0/2579886557 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:35 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x555db604cc10 con 0x555db6048be0 2024-09-21T03:40:28.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.081+0000 7f93bc15b640 1 -- 172.21.15.38:0/2579886557 <== osd.6 v2:172.21.15.195:6826/4055825675 6 ==== osd_op_reply(35 200.00000000 [create] v98'49 uv42 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f93ac0654f0 con 0x555db6048be0 2024-09-21T03:40:28.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.081+0000 7f93c340ef40 1 -- 172.21.15.38:0/2579886557 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:36 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e98) -- 0x555db604cfb0 con 0x555db6048be0 2024-09-21T03:40:28.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.083+0000 7f93bc15b640 1 -- 172.21.15.38:0/2579886557 <== osd.6 v2:172.21.15.195:6826/4055825675 7 ==== osd_op_reply(36 200.00000000 [delete] v98'50 uv50 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f93ac0654f0 con 0x555db6048be0 2024-09-21T03:40:28.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.083+0000 7f93c340ef40 1 -- 172.21.15.38:0/2579886557 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x555db604cfb0 con 0x555db6048be0 2024-09-21T03:40:28.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.084+0000 7f93bc15b640 1 -- 172.21.15.38:0/2579886557 <== osd.6 v2:172.21.15.195:6826/4055825675 8 ==== osd_op_reply(37 200.00000000 [create] v98'51 uv51 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f93ac0654f0 con 0x555db6048be0 2024-09-21T03:40:28.086 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-09-21T03:40:28.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.084+0000 7f93c340ef40 1 -- 172.21.15.38:0/2579886557 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x555db604cfb0 con 0x555db6048be0 2024-09-21T03:40:28.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.086+0000 7f93bc15b640 1 -- 172.21.15.38:0/2579886557 <== osd.6 v2:172.21.15.195:6826/4055825675 9 ==== osd_op_reply(38 200.00000000 [write 0~90] v98'52 uv52 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f93ac0654f0 con 0x555db6048be0 2024-09-21T03:40:28.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.086+0000 7f93c340ef40 1 -- 172.21.15.38:0/2579886557 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x555db604cfb0 con 0x555db6048be0 2024-09-21T03:40:28.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.087+0000 7f93bc15b640 1 -- 172.21.15.38:0/2579886557 <== osd.6 v2:172.21.15.195:6826/4055825675 10 ==== osd_op_reply(39 200.00000000 [omap-set-header] v98'53 uv53 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f93ac0654f0 con 0x555db6048be0 2024-09-21T03:40:28.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.087+0000 7f93c340ef40 1 -- 172.21.15.38:0/2579886557 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:40 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x555db604cfb0 con 0x555db6048be0 2024-09-21T03:40:28.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.088+0000 7f93bc15b640 1 -- 172.21.15.38:0/2579886557 <== osd.6 v2:172.21.15.195:6826/4055825675 11 ==== osd_op_reply(40 200.00000001 [create] v98'54 uv43 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f93ac0654f0 con 0x555db6048be0 2024-09-21T03:40:28.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.088+0000 7f93c340ef40 1 -- 172.21.15.38:0/2579886557 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:41 13.14 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e98) -- 0x555db604cfb0 con 0x555db6048be0 2024-09-21T03:40:28.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.090+0000 7f93bc15b640 1 -- 172.21.15.38:0/2579886557 <== osd.6 v2:172.21.15.195:6826/4055825675 12 ==== osd_op_reply(41 200.00000001 [delete] v98'55 uv55 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f93ac0654f0 con 0x555db6048be0 2024-09-21T03:40:28.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.090+0000 7f93c340ef40 1 -- 172.21.15.38:0/2579886557 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:42 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x555db604cfb0 con 0x555db6048be0 2024-09-21T03:40:28.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.091+0000 7f93bc15b640 1 -- 172.21.15.38:0/2579886557 <== osd.6 v2:172.21.15.195:6826/4055825675 13 ==== osd_op_reply(42 200.00000001 [create] v98'56 uv56 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f93ac0654f0 con 0x555db6048be0 2024-09-21T03:40:28.093 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-09-21T03:40:28.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.091+0000 7f93c340ef40 1 -- 172.21.15.38:0/2579886557 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:43 13.14 13:2e2fa760:::200.00000001:head [write 0~17490 in=17490b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x555db6051430 con 0x555db6048be0 2024-09-21T03:40:28.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.093+0000 7f93bc15b640 1 -- 172.21.15.38:0/2579886557 <== osd.6 v2:172.21.15.195:6826/4055825675 14 ==== osd_op_reply(43 200.00000001 [write 0~17490] v98'57 uv57 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f93ac0654f0 con 0x555db6048be0 2024-09-21T03:40:28.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.093+0000 7f93c340ef40 1 -- 172.21.15.38:0/2579886557 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:44 13.14 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x555db6051430 con 0x555db6048be0 2024-09-21T03:40:28.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.094+0000 7f93bc15b640 1 -- 172.21.15.38:0/2579886557 <== osd.6 v2:172.21.15.195:6826/4055825675 15 ==== osd_op_reply(44 200.00000001 [omap-set-header] v98'58 uv58 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f93ac0654f0 con 0x555db6048be0 2024-09-21T03:40:28.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.094+0000 7f93c340ef40 1 -- 172.21.15.38:0/2579886557 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:45 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x555db604cfb0 con 0x555db6048be0 2024-09-21T03:40:28.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.095+0000 7f93bc15b640 1 -- 172.21.15.38:0/2579886557 <== osd.6 v2:172.21.15.195:6826/4055825675 16 ==== osd_op_reply(45 606.00000000 [create] v98'59 uv39 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f93ac0654f0 con 0x555db6048be0 2024-09-21T03:40:28.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.095+0000 7f93c340ef40 1 -- 172.21.15.38:0/2579886557 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:46 13.14 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e98) -- 0x555db604cfb0 con 0x555db6048be0 2024-09-21T03:40:28.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.097+0000 7f93bc15b640 1 -- 172.21.15.38:0/2579886557 <== osd.6 v2:172.21.15.195:6826/4055825675 17 ==== osd_op_reply(46 606.00000000 [delete] v98'60 uv60 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f93ac0654f0 con 0x555db6048be0 2024-09-21T03:40:28.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.097+0000 7f93c340ef40 1 -- 172.21.15.38:0/2579886557 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:47 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x555db604cfb0 con 0x555db6048be0 2024-09-21T03:40:28.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.098+0000 7f93bc15b640 1 -- 172.21.15.38:0/2579886557 <== osd.6 v2:172.21.15.195:6826/4055825675 18 ==== osd_op_reply(47 606.00000000 [create] v98'61 uv61 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f93ac0654f0 con 0x555db6048be0 2024-09-21T03:40:28.100 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-09-21T03:40:28.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.098+0000 7f93c340ef40 1 -- 172.21.15.38:0/2579886557 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:48 13.14 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x555db604dfc0 con 0x555db6048be0 2024-09-21T03:40:28.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.099+0000 7f93bc15b640 1 -- 172.21.15.38:0/2579886557 <== osd.6 v2:172.21.15.195:6826/4055825675 19 ==== osd_op_reply(48 606.00000000 [setxattr (30)] v98'62 uv62 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f93ac0654f0 con 0x555db6048be0 2024-09-21T03:40:28.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.099+0000 7f93c340ef40 1 -- 172.21.15.38:0/2579886557 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:49 13.14 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x555db604dfc0 con 0x555db6048be0 2024-09-21T03:40:28.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.100+0000 7f93bc15b640 1 -- 172.21.15.38:0/2579886557 <== osd.6 v2:172.21.15.195:6826/4055825675 20 ==== osd_op_reply(49 606.00000000 [setxattr (62)] v98'63 uv63 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f93ac0654f0 con 0x555db6048be0 2024-09-21T03:40:28.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.100+0000 7f93c340ef40 1 -- 172.21.15.38:0/2579886557 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:50 13.14 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x555db604cfb0 con 0x555db6048be0 2024-09-21T03:40:28.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.102+0000 7f93bc15b640 1 -- 172.21.15.38:0/2579886557 <== osd.6 v2:172.21.15.195:6826/4055825675 21 ==== osd_op_reply(50 606.00000000 [omap-set-header] v98'64 uv64 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f93ac0654f0 con 0x555db6048be0 2024-09-21T03:40:28.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.102+0000 7f93c340ef40 1 --2- 172.21.15.38:0/2579886557 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x555db604d4c0 0x555db604f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:28.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.102+0000 7f93c340ef40 1 -- 172.21.15.38:0/2579886557 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:51 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x555db604fe20 con 0x555db604d4c0 2024-09-21T03:40:28.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.102+0000 7f93bb95a640 1 --2- 172.21.15.38:0/2579886557 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x555db604d4c0 0x555db604f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:28.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.102+0000 7f93bb95a640 1 --2- 172.21.15.38:0/2579886557 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x555db604d4c0 0x555db604f8e0 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:28.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.104+0000 7f93bb95a640 1 -- 172.21.15.38:0/2579886557 <== osd.0 v2:172.21.15.38:6800/2381639433 1 ==== osd_op_reply(51 607.00000000 [create] v98'9 uv8 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f93b000e550 con 0x555db604d4c0 2024-09-21T03:40:28.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.104+0000 7f93c340ef40 1 -- 172.21.15.38:0/2579886557 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:52 13.1c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e98) -- 0x555db604fe60 con 0x555db604d4c0 2024-09-21T03:40:28.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.105+0000 7f93bb95a640 1 -- 172.21.15.38:0/2579886557 <== osd.0 v2:172.21.15.38:6800/2381639433 2 ==== osd_op_reply(52 607.00000000 [delete] v98'10 uv10 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f93b000e550 con 0x555db604d4c0 2024-09-21T03:40:28.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.105+0000 7f93c340ef40 1 -- 172.21.15.38:0/2579886557 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:53 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x555db6050250 con 0x555db604d4c0 2024-09-21T03:40:28.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.106+0000 7f93bb95a640 1 -- 172.21.15.38:0/2579886557 <== osd.0 v2:172.21.15.38:6800/2381639433 3 ==== osd_op_reply(53 607.00000000 [create] v98'11 uv11 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f93b000e550 con 0x555db604d4c0 2024-09-21T03:40:28.108 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-09-21T03:40:28.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.106+0000 7f93c340ef40 1 -- 172.21.15.38:0/2579886557 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:54 13.1c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x555db6051260 con 0x555db604d4c0 2024-09-21T03:40:28.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.107+0000 7f93bb95a640 1 -- 172.21.15.38:0/2579886557 <== osd.0 v2:172.21.15.38:6800/2381639433 4 ==== osd_op_reply(54 607.00000000 [setxattr (30)] v98'12 uv12 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f93b000e550 con 0x555db604d4c0 2024-09-21T03:40:28.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.107+0000 7f93c340ef40 1 -- 172.21.15.38:0/2579886557 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:55 13.1c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x555db6051260 con 0x555db604d4c0 2024-09-21T03:40:28.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.108+0000 7f93bb95a640 1 -- 172.21.15.38:0/2579886557 <== osd.0 v2:172.21.15.38:6800/2381639433 5 ==== osd_op_reply(55 607.00000000 [setxattr (62)] v98'13 uv13 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f93b000e550 con 0x555db604d4c0 2024-09-21T03:40:28.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.108+0000 7f93c340ef40 1 -- 172.21.15.38:0/2579886557 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:56 13.1c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x555db6050250 con 0x555db604d4c0 2024-09-21T03:40:28.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.109+0000 7f93bb95a640 1 -- 172.21.15.38:0/2579886557 <== osd.0 v2:172.21.15.38:6800/2381639433 6 ==== osd_op_reply(56 607.00000000 [omap-set-header] v98'14 uv14 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f93b000e550 con 0x555db604d4c0 2024-09-21T03:40:28.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.109+0000 7f93c340ef40 1 --2- 172.21.15.38:0/2579886557 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x555db60508d0 0x555db6052cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:28.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.109+0000 7f93c340ef40 1 -- 172.21.15.38:0/2579886557 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:57 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x555db6053300 con 0x555db60508d0 2024-09-21T03:40:28.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.109+0000 7f93bc95c640 1 --2- 172.21.15.38:0/2579886557 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x555db60508d0 0x555db6052cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:28.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.109+0000 7f93bc95c640 1 --2- 172.21.15.38:0/2579886557 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x555db60508d0 0x555db6052cf0 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:28.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.110+0000 7f93bc95c640 1 -- 172.21.15.38:0/2579886557 <== osd.3 v2:172.21.15.38:6808/1286810294 1 ==== osd_op_reply(57 mds0_openfiles.0 [create] v98'8 uv7 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f93b400f070 con 0x555db60508d0 2024-09-21T03:40:28.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.110+0000 7f93c340ef40 1 -- 172.21.15.38:0/2579886557 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:58 13.6 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e98) -- 0x555db6053810 con 0x555db60508d0 2024-09-21T03:40:28.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.112+0000 7f93bc95c640 1 -- 172.21.15.38:0/2579886557 <== osd.3 v2:172.21.15.38:6808/1286810294 2 ==== osd_op_reply(58 mds0_openfiles.0 [delete] v98'9 uv9 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f93b400f070 con 0x555db60508d0 2024-09-21T03:40:28.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.112+0000 7f93c340ef40 1 -- 172.21.15.38:0/2579886557 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:59 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x555db6053d10 con 0x555db60508d0 2024-09-21T03:40:28.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.113+0000 7f93bc95c640 1 -- 172.21.15.38:0/2579886557 <== osd.3 v2:172.21.15.38:6808/1286810294 3 ==== osd_op_reply(59 mds0_openfiles.0 [create] v98'10 uv10 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f93b400f070 con 0x555db60508d0 2024-09-21T03:40:28.115 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-09-21T03:40:28.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.113+0000 7f93c340ef40 1 -- 172.21.15.38:0/2579886557 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:60 13.6 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x555db60545d0 con 0x555db60508d0 2024-09-21T03:40:28.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.114+0000 7f93bc95c640 1 -- 172.21.15.38:0/2579886557 <== osd.3 v2:172.21.15.38:6808/1286810294 4 ==== osd_op_reply(60 mds0_openfiles.0 [omap-set-header] v98'11 uv11 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f93b400f070 con 0x555db60508d0 2024-09-21T03:40:28.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.114+0000 7f93c340ef40 1 -- 172.21.15.38:0/2579886557 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:61 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x555db6054970 con 0x555db604d4c0 2024-09-21T03:40:28.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.115+0000 7f93bb95a640 1 -- 172.21.15.38:0/2579886557 <== osd.0 v2:172.21.15.38:6800/2381639433 7 ==== osd_op_reply(61 608.00000000 [create] v98'9 uv8 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f93b000e550 con 0x555db604d4c0 2024-09-21T03:40:28.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.115+0000 7f93c340ef40 1 -- 172.21.15.38:0/2579886557 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:62 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e98) -- 0x555db6054970 con 0x555db604d4c0 2024-09-21T03:40:28.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.116+0000 7f93bb95a640 1 -- 172.21.15.38:0/2579886557 <== osd.0 v2:172.21.15.38:6800/2381639433 8 ==== osd_op_reply(62 608.00000000 [delete] v98'10 uv10 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f93b000e550 con 0x555db604d4c0 2024-09-21T03:40:28.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.116+0000 7f93c340ef40 1 -- 172.21.15.38:0/2579886557 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x555db6054970 con 0x555db604d4c0 2024-09-21T03:40:28.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.118+0000 7f93bb95a640 1 -- 172.21.15.38:0/2579886557 <== osd.0 v2:172.21.15.38:6800/2381639433 9 ==== osd_op_reply(63 608.00000000 [create] v98'11 uv11 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f93b000e550 con 0x555db604d4c0 2024-09-21T03:40:28.119 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-09-21T03:40:28.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.118+0000 7f93c340ef40 1 -- 172.21.15.38:0/2579886557 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x555db6055980 con 0x555db604d4c0 2024-09-21T03:40:28.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.119+0000 7f93bb95a640 1 -- 172.21.15.38:0/2579886557 <== osd.0 v2:172.21.15.38:6800/2381639433 10 ==== osd_op_reply(64 608.00000000 [setxattr (30)] v98'12 uv12 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f93b000e550 con 0x555db604d4c0 2024-09-21T03:40:28.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.119+0000 7f93c340ef40 1 -- 172.21.15.38:0/2579886557 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x555db6055980 con 0x555db604d4c0 2024-09-21T03:40:28.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.120+0000 7f93bb95a640 1 -- 172.21.15.38:0/2579886557 <== osd.0 v2:172.21.15.38:6800/2381639433 11 ==== osd_op_reply(65 608.00000000 [setxattr (62)] v98'13 uv13 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f93b000e550 con 0x555db604d4c0 2024-09-21T03:40:28.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.120+0000 7f93c340ef40 1 -- 172.21.15.38:0/2579886557 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x555db6054970 con 0x555db604d4c0 2024-09-21T03:40:28.123 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.121+0000 7f93bb95a640 1 -- 172.21.15.38:0/2579886557 <== osd.0 v2:172.21.15.38:6800/2381639433 12 ==== osd_op_reply(66 608.00000000 [omap-set-header] v98'14 uv14 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f93b000e550 con 0x555db604d4c0 2024-09-21T03:40:28.123 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.121+0000 7f93c340ef40 1 -- 172.21.15.38:0/2579886557 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:67 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x555db6054970 con 0x555db60508d0 2024-09-21T03:40:28.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.122+0000 7f93bc95c640 1 -- 172.21.15.38:0/2579886557 <== osd.3 v2:172.21.15.38:6808/1286810294 5 ==== osd_op_reply(67 604.00000000 [create] v98'15 uv9 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f93b400f070 con 0x555db60508d0 2024-09-21T03:40:28.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.122+0000 7f93c340ef40 1 -- 172.21.15.38:0/2579886557 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:68 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e98) -- 0x555db6054d10 con 0x555db60508d0 2024-09-21T03:40:28.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.123+0000 7f93bc95c640 1 -- 172.21.15.38:0/2579886557 <== osd.3 v2:172.21.15.38:6808/1286810294 6 ==== osd_op_reply(68 604.00000000 [delete] v98'16 uv16 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f93b400f070 con 0x555db60508d0 2024-09-21T03:40:28.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.123+0000 7f93c340ef40 1 -- 172.21.15.38:0/2579886557 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x555db60550b0 con 0x555db60508d0 2024-09-21T03:40:28.126 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.124+0000 7f93bc95c640 1 -- 172.21.15.38:0/2579886557 <== osd.3 v2:172.21.15.38:6808/1286810294 7 ==== osd_op_reply(69 604.00000000 [create] v98'17 uv17 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f93b400f070 con 0x555db60508d0 2024-09-21T03:40:28.126 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-09-21T03:40:28.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.124+0000 7f93c340ef40 1 -- 172.21.15.38:0/2579886557 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x555db6056460 con 0x555db60508d0 2024-09-21T03:40:28.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.125+0000 7f93bc95c640 1 -- 172.21.15.38:0/2579886557 <== osd.3 v2:172.21.15.38:6808/1286810294 8 ==== osd_op_reply(70 604.00000000 [setxattr (30)] v98'18 uv18 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f93b400f070 con 0x555db60508d0 2024-09-21T03:40:28.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.125+0000 7f93c340ef40 1 -- 172.21.15.38:0/2579886557 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x555db6056460 con 0x555db60508d0 2024-09-21T03:40:28.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.126+0000 7f93bc95c640 1 -- 172.21.15.38:0/2579886557 <== osd.3 v2:172.21.15.38:6808/1286810294 9 ==== osd_op_reply(71 604.00000000 [setxattr (62)] v98'19 uv19 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f93b400f070 con 0x555db60508d0 2024-09-21T03:40:28.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.126+0000 7f93c340ef40 1 -- 172.21.15.38:0/2579886557 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x555db6055450 con 0x555db60508d0 2024-09-21T03:40:28.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.127+0000 7f93bc95c640 1 -- 172.21.15.38:0/2579886557 <== osd.3 v2:172.21.15.38:6808/1286810294 10 ==== osd_op_reply(72 604.00000000 [omap-set-header] v98'20 uv20 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f93b400f070 con 0x555db60508d0 2024-09-21T03:40:28.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.128+0000 7f93c340ef40 1 -- 172.21.15.38:0/2579886557 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:73 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x555db6055450 con 0x555db60508d0 2024-09-21T03:40:28.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.128+0000 7f93bc95c640 1 -- 172.21.15.38:0/2579886557 <== osd.3 v2:172.21.15.38:6808/1286810294 11 ==== osd_op_reply(73 500.00000000 [create] v98'21 uv14 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f93b400f070 con 0x555db60508d0 2024-09-21T03:40:28.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.128+0000 7f93c340ef40 1 -- 172.21.15.38:0/2579886557 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:74 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e98) -- 0x555db6055450 con 0x555db60508d0 2024-09-21T03:40:28.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.130+0000 7f93bc95c640 1 -- 172.21.15.38:0/2579886557 <== osd.3 v2:172.21.15.38:6808/1286810294 12 ==== osd_op_reply(74 500.00000000 [delete] v98'22 uv22 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f93b400f070 con 0x555db60508d0 2024-09-21T03:40:28.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.130+0000 7f93c340ef40 1 -- 172.21.15.38:0/2579886557 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x555db6055450 con 0x555db60508d0 2024-09-21T03:40:28.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.131+0000 7f93bc95c640 1 -- 172.21.15.38:0/2579886557 <== osd.3 v2:172.21.15.38:6808/1286810294 13 ==== osd_op_reply(75 500.00000000 [create] v98'23 uv23 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f93b400f070 con 0x555db60508d0 2024-09-21T03:40:28.133 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-09-21T03:40:28.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.131+0000 7f93c340ef40 1 -- 172.21.15.38:0/2579886557 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x555db6055450 con 0x555db60508d0 2024-09-21T03:40:28.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.132+0000 7f93bc95c640 1 -- 172.21.15.38:0/2579886557 <== osd.3 v2:172.21.15.38:6808/1286810294 14 ==== osd_op_reply(76 500.00000000 [write 0~90] v98'24 uv24 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f93b400f070 con 0x555db60508d0 2024-09-21T03:40:28.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.132+0000 7f93c340ef40 1 -- 172.21.15.38:0/2579886557 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x555db6055450 con 0x555db60508d0 2024-09-21T03:40:28.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.133+0000 7f93bc95c640 1 -- 172.21.15.38:0/2579886557 <== osd.3 v2:172.21.15.38:6808/1286810294 15 ==== osd_op_reply(77 500.00000000 [omap-set-header] v98'25 uv25 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f93b400f070 con 0x555db60508d0 2024-09-21T03:40:28.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.133+0000 7f93c340ef40 1 -- 172.21.15.38:0/2579886557 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:78 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x555db6055450 con 0x555db604d4c0 2024-09-21T03:40:28.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.134+0000 7f93bb95a640 1 -- 172.21.15.38:0/2579886557 <== osd.0 v2:172.21.15.38:6800/2381639433 13 ==== osd_op_reply(78 mds_snaptable [create] v98'7 uv6 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f93b000e550 con 0x555db604d4c0 2024-09-21T03:40:28.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.134+0000 7f93c340ef40 1 -- 172.21.15.38:0/2579886557 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:79 13.d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e98) -- 0x555db6055450 con 0x555db604d4c0 2024-09-21T03:40:28.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.135+0000 7f93bb95a640 1 -- 172.21.15.38:0/2579886557 <== osd.0 v2:172.21.15.38:6800/2381639433 14 ==== osd_op_reply(79 mds_snaptable [delete] v98'8 uv8 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f93b000e550 con 0x555db604d4c0 2024-09-21T03:40:28.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.135+0000 7f93c340ef40 1 -- 172.21.15.38:0/2579886557 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:80 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x555db6055450 con 0x555db604d4c0 2024-09-21T03:40:28.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.136+0000 7f93bb95a640 1 -- 172.21.15.38:0/2579886557 <== osd.0 v2:172.21.15.38:6800/2381639433 15 ==== osd_op_reply(80 mds_snaptable [create] v98'9 uv9 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f93b000e550 con 0x555db604d4c0 2024-09-21T03:40:28.138 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-09-21T03:40:28.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.136+0000 7f93c340ef40 1 -- 172.21.15.38:0/2579886557 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:81 13.d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x555db6055450 con 0x555db604d4c0 2024-09-21T03:40:28.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.137+0000 7f93bb95a640 1 -- 172.21.15.38:0/2579886557 <== osd.0 v2:172.21.15.38:6800/2381639433 16 ==== osd_op_reply(81 mds_snaptable [write 0~70] v98'10 uv10 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f93b000e550 con 0x555db604d4c0 2024-09-21T03:40:28.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.137+0000 7f93c340ef40 1 -- 172.21.15.38:0/2579886557 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:82 13.d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x555db6055450 con 0x555db604d4c0 2024-09-21T03:40:28.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.138+0000 7f93bb95a640 1 -- 172.21.15.38:0/2579886557 <== osd.0 v2:172.21.15.38:6800/2381639433 17 ==== osd_op_reply(82 mds_snaptable [omap-set-header] v98'11 uv11 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f93b000e550 con 0x555db604d4c0 2024-09-21T03:40:28.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.138+0000 7f93c340ef40 1 --2- 172.21.15.38:0/2579886557 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x555db6055ad0 0x555db6057ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:28.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.138+0000 7f93c340ef40 1 -- 172.21.15.38:0/2579886557 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:83 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x555db6058500 con 0x555db6055ad0 2024-09-21T03:40:28.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.138+0000 7f93bc15b640 1 --2- 172.21.15.38:0/2579886557 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x555db6055ad0 0x555db6057ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:28.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.139+0000 7f93bc15b640 1 --2- 172.21.15.38:0/2579886557 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x555db6055ad0 0x555db6057ef0 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:28.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.140+0000 7f93bc15b640 1 -- 172.21.15.38:0/2579886557 <== osd.1 v2:172.21.15.38:6816/1491818732 1 ==== osd_op_reply(83 605.00000000 [create] v98'9 uv8 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f93ac0654f0 con 0x555db6055ad0 2024-09-21T03:40:28.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.140+0000 7f93c340ef40 1 -- 172.21.15.38:0/2579886557 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:84 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e98) -- 0x555db60585c0 con 0x555db6055ad0 2024-09-21T03:40:28.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.141+0000 7f93bc15b640 1 -- 172.21.15.38:0/2579886557 <== osd.1 v2:172.21.15.38:6816/1491818732 2 ==== osd_op_reply(84 605.00000000 [delete] v98'10 uv10 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f93ac0654f0 con 0x555db6055ad0 2024-09-21T03:40:28.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.141+0000 7f93c340ef40 1 -- 172.21.15.38:0/2579886557 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x555db60589e0 con 0x555db6055ad0 2024-09-21T03:40:28.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.142+0000 7f93bc15b640 1 -- 172.21.15.38:0/2579886557 <== osd.1 v2:172.21.15.38:6816/1491818732 3 ==== osd_op_reply(85 605.00000000 [create] v98'11 uv11 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f93ac0654f0 con 0x555db6055ad0 2024-09-21T03:40:28.144 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-09-21T03:40:28.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.142+0000 7f93c340ef40 1 -- 172.21.15.38:0/2579886557 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x555db60599f0 con 0x555db6055ad0 2024-09-21T03:40:28.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.143+0000 7f93bc15b640 1 -- 172.21.15.38:0/2579886557 <== osd.1 v2:172.21.15.38:6816/1491818732 4 ==== osd_op_reply(86 605.00000000 [setxattr (30)] v98'12 uv12 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f93ac0654f0 con 0x555db6055ad0 2024-09-21T03:40:28.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.143+0000 7f93c340ef40 1 -- 172.21.15.38:0/2579886557 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x555db60599f0 con 0x555db6055ad0 2024-09-21T03:40:28.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.144+0000 7f93bc15b640 1 -- 172.21.15.38:0/2579886557 <== osd.1 v2:172.21.15.38:6816/1491818732 5 ==== osd_op_reply(87 605.00000000 [setxattr (62)] v98'13 uv13 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f93ac0654f0 con 0x555db6055ad0 2024-09-21T03:40:28.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.144+0000 7f93c340ef40 1 -- 172.21.15.38:0/2579886557 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x555db60589e0 con 0x555db6055ad0 2024-09-21T03:40:28.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.145+0000 7f93bc15b640 1 -- 172.21.15.38:0/2579886557 <== osd.1 v2:172.21.15.38:6816/1491818732 6 ==== osd_op_reply(88 605.00000000 [omap-set-header] v98'14 uv14 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f93ac0654f0 con 0x555db6055ad0 2024-09-21T03:40:28.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.145+0000 7f93c340ef40 1 -- 172.21.15.38:0/2579886557 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:89 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x555db60589e0 con 0x555db6055ad0 2024-09-21T03:40:28.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.146+0000 7f93bc15b640 1 -- 172.21.15.38:0/2579886557 <== osd.1 v2:172.21.15.38:6816/1491818732 7 ==== osd_op_reply(89 mds0_inotable [create] v98'14 uv8 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f93ac0654f0 con 0x555db6055ad0 2024-09-21T03:40:28.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.146+0000 7f93c340ef40 1 -- 172.21.15.38:0/2579886557 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:90 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e98) -- 0x555db60589e0 con 0x555db6055ad0 2024-09-21T03:40:28.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.147+0000 7f93bc15b640 1 -- 172.21.15.38:0/2579886557 <== osd.1 v2:172.21.15.38:6816/1491818732 8 ==== osd_op_reply(90 mds0_inotable [delete] v98'15 uv15 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f93ac0654f0 con 0x555db6055ad0 2024-09-21T03:40:28.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.147+0000 7f93c340ef40 1 -- 172.21.15.38:0/2579886557 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x555db60589e0 con 0x555db6055ad0 2024-09-21T03:40:28.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.148+0000 7f93bc15b640 1 -- 172.21.15.38:0/2579886557 <== osd.1 v2:172.21.15.38:6816/1491818732 9 ==== osd_op_reply(91 mds0_inotable [create] v98'16 uv16 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f93ac0654f0 con 0x555db6055ad0 2024-09-21T03:40:28.150 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-09-21T03:40:28.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.148+0000 7f93c340ef40 1 -- 172.21.15.38:0/2579886557 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x555db60589e0 con 0x555db6055ad0 2024-09-21T03:40:28.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.149+0000 7f93bc15b640 1 -- 172.21.15.38:0/2579886557 <== osd.1 v2:172.21.15.38:6816/1491818732 10 ==== osd_op_reply(92 mds0_inotable [write 0~34] v98'17 uv17 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f93ac0654f0 con 0x555db6055ad0 2024-09-21T03:40:28.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.149+0000 7f93c340ef40 1 -- 172.21.15.38:0/2579886557 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x555db60589e0 con 0x555db6055ad0 2024-09-21T03:40:28.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.150+0000 7f93bc15b640 1 -- 172.21.15.38:0/2579886557 <== osd.1 v2:172.21.15.38:6816/1491818732 11 ==== osd_op_reply(93 mds0_inotable [omap-set-header] v98'18 uv18 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f93ac0654f0 con 0x555db6055ad0 2024-09-21T03:40:28.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.150+0000 7f93c340ef40 1 -- 172.21.15.38:0/2579886557 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:94 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x555db60589e0 con 0x555db6055ad0 2024-09-21T03:40:28.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.151+0000 7f93bc15b640 1 -- 172.21.15.38:0/2579886557 <== osd.1 v2:172.21.15.38:6816/1491818732 12 ==== osd_op_reply(94 100.00000000 [create] v98'19 uv13 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f93ac0654f0 con 0x555db6055ad0 2024-09-21T03:40:28.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.151+0000 7f93c340ef40 1 -- 172.21.15.38:0/2579886557 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:95 13.13 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e98) -- 0x555db60589e0 con 0x555db6055ad0 2024-09-21T03:40:28.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.153+0000 7f93bc15b640 1 -- 172.21.15.38:0/2579886557 <== osd.1 v2:172.21.15.38:6816/1491818732 13 ==== osd_op_reply(95 100.00000000 [delete] v98'20 uv20 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f93ac0654f0 con 0x555db6055ad0 2024-09-21T03:40:28.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.153+0000 7f93c340ef40 1 -- 172.21.15.38:0/2579886557 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:96 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x555db60589e0 con 0x555db6055ad0 2024-09-21T03:40:28.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.154+0000 7f93bc15b640 1 -- 172.21.15.38:0/2579886557 <== osd.1 v2:172.21.15.38:6816/1491818732 14 ==== osd_op_reply(96 100.00000000 [create] v98'21 uv21 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f93ac0654f0 con 0x555db6055ad0 2024-09-21T03:40:28.156 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-09-21T03:40:28.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.154+0000 7f93c340ef40 1 -- 172.21.15.38:0/2579886557 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:97 13.13 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x555db60589e0 con 0x555db6055ad0 2024-09-21T03:40:28.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.155+0000 7f93bc15b640 1 -- 172.21.15.38:0/2579886557 <== osd.1 v2:172.21.15.38:6816/1491818732 15 ==== osd_op_reply(97 100.00000000 [omap-set-header] v98'22 uv22 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f93ac0654f0 con 0x555db6055ad0 2024-09-21T03:40:28.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.155+0000 7f93c340ef40 1 -- 172.21.15.38:0/2579886557 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:98 13.13 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x555db605b7d0 con 0x555db6055ad0 2024-09-21T03:40:28.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.156+0000 7f93bc15b640 1 -- 172.21.15.38:0/2579886557 <== osd.1 v2:172.21.15.38:6816/1491818732 16 ==== osd_op_reply(98 100.00000000 [omap-set-vals] v98'23 uv23 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f93ac0654f0 con 0x555db6055ad0 2024-09-21T03:40:28.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.156+0000 7f93c340ef40 1 -- 172.21.15.38:0/2579886557 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:99 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x555db605b790 con 0x555db60508d0 2024-09-21T03:40:28.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.157+0000 7f93bc95c640 1 -- 172.21.15.38:0/2579886557 <== osd.3 v2:172.21.15.38:6808/1286810294 16 ==== osd_op_reply(99 mds0_sessionmap [create] v98'9 uv7 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7f93b400f070 con 0x555db60508d0 2024-09-21T03:40:28.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.157+0000 7f93c340ef40 1 -- 172.21.15.38:0/2579886557 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:100 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e98) -- 0x555db605b790 con 0x555db60508d0 2024-09-21T03:40:28.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.158+0000 7f93bc95c640 1 -- 172.21.15.38:0/2579886557 <== osd.3 v2:172.21.15.38:6808/1286810294 17 ==== osd_op_reply(100 mds0_sessionmap [delete] v98'10 uv10 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f93b400f070 con 0x555db60508d0 2024-09-21T03:40:28.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.158+0000 7f93c340ef40 1 -- 172.21.15.38:0/2579886557 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x555db605b790 con 0x555db60508d0 2024-09-21T03:40:28.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.159+0000 7f93bc95c640 1 -- 172.21.15.38:0/2579886557 <== osd.3 v2:172.21.15.38:6808/1286810294 18 ==== osd_op_reply(101 mds0_sessionmap [create] v98'11 uv11 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f93b400f070 con 0x555db60508d0 2024-09-21T03:40:28.160 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-09-21T03:40:28.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.159+0000 7f93c340ef40 1 -- 172.21.15.38:0/2579886557 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x555db605b790 con 0x555db60508d0 2024-09-21T03:40:28.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.159+0000 7f93bc95c640 1 -- 172.21.15.38:0/2579886557 <== osd.3 v2:172.21.15.38:6808/1286810294 19 ==== osd_op_reply(102 mds0_sessionmap [omap-set-header] v98'12 uv12 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f93b400f070 con 0x555db60508d0 2024-09-21T03:40:28.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.159+0000 7f93c340ef40 1 -- 172.21.15.38:0/2579886557 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:103 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x555db605b770 con 0x555db60508d0 2024-09-21T03:40:28.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.160+0000 7f93bc95c640 1 -- 172.21.15.38:0/2579886557 <== osd.3 v2:172.21.15.38:6808/1286810294 20 ==== osd_op_reply(103 609.00000000 [create] v98'9 uv8 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f93b400f070 con 0x555db60508d0 2024-09-21T03:40:28.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.160+0000 7f93c340ef40 1 -- 172.21.15.38:0/2579886557 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:104 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e98) -- 0x555db605b770 con 0x555db60508d0 2024-09-21T03:40:28.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.161+0000 7f93bc95c640 1 -- 172.21.15.38:0/2579886557 <== osd.3 v2:172.21.15.38:6808/1286810294 21 ==== osd_op_reply(104 609.00000000 [delete] v98'10 uv10 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f93b400f070 con 0x555db60508d0 2024-09-21T03:40:28.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.162+0000 7f93c340ef40 1 -- 172.21.15.38:0/2579886557 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x555db605b770 con 0x555db60508d0 2024-09-21T03:40:28.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.162+0000 7f93bc95c640 1 -- 172.21.15.38:0/2579886557 <== osd.3 v2:172.21.15.38:6808/1286810294 22 ==== osd_op_reply(105 609.00000000 [create] v98'11 uv11 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f93b400f070 con 0x555db60508d0 2024-09-21T03:40:28.164 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-09-21T03:40:28.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.162+0000 7f93c340ef40 1 -- 172.21.15.38:0/2579886557 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x555db605b770 con 0x555db60508d0 2024-09-21T03:40:28.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.163+0000 7f93bc95c640 1 -- 172.21.15.38:0/2579886557 <== osd.3 v2:172.21.15.38:6808/1286810294 23 ==== osd_op_reply(106 609.00000000 [setxattr (30)] v98'12 uv12 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f93b400f070 con 0x555db60508d0 2024-09-21T03:40:28.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.163+0000 7f93c340ef40 1 -- 172.21.15.38:0/2579886557 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x555db605b750 con 0x555db60508d0 2024-09-21T03:40:28.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.164+0000 7f93bc95c640 1 -- 172.21.15.38:0/2579886557 <== osd.3 v2:172.21.15.38:6808/1286810294 24 ==== osd_op_reply(107 609.00000000 [setxattr (62)] v98'13 uv13 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f93b400f070 con 0x555db60508d0 2024-09-21T03:40:28.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.164+0000 7f93c340ef40 1 -- 172.21.15.38:0/2579886557 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x555db605b750 con 0x555db60508d0 2024-09-21T03:40:28.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.165+0000 7f93bc95c640 1 -- 172.21.15.38:0/2579886557 <== osd.3 v2:172.21.15.38:6808/1286810294 25 ==== osd_op_reply(108 609.00000000 [omap-set-header] v98'14 uv14 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f93b400f070 con 0x555db60508d0 2024-09-21T03:40:28.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.165+0000 7f93c340ef40 1 --2- 172.21.15.38:0/2579886557 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x555db60592e0 0x555db605be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:28.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.165+0000 7f93c340ef40 1 -- 172.21.15.38:0/2579886557 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:109 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x555db605b750 con 0x555db60592e0 2024-09-21T03:40:28.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.166+0000 7f93bb95a640 1 --2- 172.21.15.38:0/2579886557 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x555db60592e0 0x555db605be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:28.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.166+0000 7f93bb95a640 1 --2- 172.21.15.38:0/2579886557 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x555db60592e0 0x555db605be90 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:28.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.167+0000 7f93bb95a640 1 -- 172.21.15.38:0/2579886557 <== osd.2 v2:172.21.15.38:6824/2577858902 1 ==== osd_op_reply(109 400.00000000 [create] v98'7 uv6 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f93b000e550 con 0x555db60592e0 2024-09-21T03:40:28.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.167+0000 7f93c340ef40 1 -- 172.21.15.38:0/2579886557 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:110 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e98) -- 0x555db605b750 con 0x555db60592e0 2024-09-21T03:40:28.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.168+0000 7f93bb95a640 1 -- 172.21.15.38:0/2579886557 <== osd.2 v2:172.21.15.38:6824/2577858902 2 ==== osd_op_reply(110 400.00000000 [delete] v98'8 uv8 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f93b000e550 con 0x555db60592e0 2024-09-21T03:40:28.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.168+0000 7f93c340ef40 1 -- 172.21.15.38:0/2579886557 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x555db605b750 con 0x555db60592e0 2024-09-21T03:40:28.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.169+0000 7f93bb95a640 1 -- 172.21.15.38:0/2579886557 <== osd.2 v2:172.21.15.38:6824/2577858902 3 ==== osd_op_reply(111 400.00000000 [create] v98'9 uv9 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f93b000e550 con 0x555db60592e0 2024-09-21T03:40:28.171 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-09-21T03:40:28.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.169+0000 7f93c340ef40 1 -- 172.21.15.38:0/2579886557 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x555db605b750 con 0x555db60592e0 2024-09-21T03:40:28.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.170+0000 7f93bb95a640 1 -- 172.21.15.38:0/2579886557 <== osd.2 v2:172.21.15.38:6824/2577858902 4 ==== osd_op_reply(112 400.00000000 [write 0~22] v98'10 uv10 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f93b000e550 con 0x555db60592e0 2024-09-21T03:40:28.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.170+0000 7f93c340ef40 1 -- 172.21.15.38:0/2579886557 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x555db605b750 con 0x555db60592e0 2024-09-21T03:40:28.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.171+0000 7f93bb95a640 1 -- 172.21.15.38:0/2579886557 <== osd.2 v2:172.21.15.38:6824/2577858902 5 ==== osd_op_reply(113 400.00000000 [omap-set-header] v98'11 uv11 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f93b000e550 con 0x555db60592e0 2024-09-21T03:40:28.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.171+0000 7f93c340ef40 1 -- 172.21.15.38:0/2579886557 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:114 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x555db605b750 con 0x555db6055ad0 2024-09-21T03:40:28.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.173+0000 7f93bc15b640 1 -- 172.21.15.38:0/2579886557 <== osd.1 v2:172.21.15.38:6816/1491818732 17 ==== osd_op_reply(114 100.00000000.inode [create] v98'20 uv11 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7f93ac0654f0 con 0x555db6055ad0 2024-09-21T03:40:28.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.173+0000 7f93c340ef40 1 -- 172.21.15.38:0/2579886557 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:115 13.1f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e98) -- 0x555db605b750 con 0x555db6055ad0 2024-09-21T03:40:28.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.174+0000 7f93bc15b640 1 -- 172.21.15.38:0/2579886557 <== osd.1 v2:172.21.15.38:6816/1491818732 18 ==== osd_op_reply(115 100.00000000.inode [delete] v98'21 uv21 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f93ac0654f0 con 0x555db6055ad0 2024-09-21T03:40:28.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.174+0000 7f93c340ef40 1 -- 172.21.15.38:0/2579886557 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:116 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x555db605b750 con 0x555db6055ad0 2024-09-21T03:40:28.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.175+0000 7f93bc15b640 1 -- 172.21.15.38:0/2579886557 <== osd.1 v2:172.21.15.38:6816/1491818732 19 ==== osd_op_reply(116 100.00000000.inode [create] v98'22 uv22 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f93ac0654f0 con 0x555db6055ad0 2024-09-21T03:40:28.177 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-09-21T03:40:28.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.175+0000 7f93c340ef40 1 -- 172.21.15.38:0/2579886557 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:117 13.1f 13:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x555db605b750 con 0x555db6055ad0 2024-09-21T03:40:28.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.176+0000 7f93bc15b640 1 -- 172.21.15.38:0/2579886557 <== osd.1 v2:172.21.15.38:6816/1491818732 20 ==== osd_op_reply(117 100.00000000.inode [write 0~582] v98'23 uv23 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f93ac0654f0 con 0x555db6055ad0 2024-09-21T03:40:28.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.176+0000 7f93c340ef40 1 -- 172.21.15.38:0/2579886557 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:118 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x555db605b730 con 0x555db6055ad0 2024-09-21T03:40:28.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.177+0000 7f93bc15b640 1 -- 172.21.15.38:0/2579886557 <== osd.1 v2:172.21.15.38:6816/1491818732 21 ==== osd_op_reply(118 100.00000000.inode [omap-set-header] v98'24 uv24 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f93ac0654f0 con 0x555db6055ad0 2024-09-21T03:40:28.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.177+0000 7f93c340ef40 1 -- 172.21.15.38:0/2579886557 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:119 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x555db605b730 con 0x555db6055ad0 2024-09-21T03:40:28.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.178+0000 7f93bc15b640 1 -- 172.21.15.38:0/2579886557 <== osd.1 v2:172.21.15.38:6816/1491818732 22 ==== osd_op_reply(119 1.00000000 [create] v98'25 uv18 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7f93ac0654f0 con 0x555db6055ad0 2024-09-21T03:40:28.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.178+0000 7f93c340ef40 1 -- 172.21.15.38:0/2579886557 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:120 13.1f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e98) -- 0x555db605b730 con 0x555db6055ad0 2024-09-21T03:40:28.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.179+0000 7f93bc15b640 1 -- 172.21.15.38:0/2579886557 <== osd.1 v2:172.21.15.38:6816/1491818732 23 ==== osd_op_reply(120 1.00000000 [delete] v98'26 uv26 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f93ac0654f0 con 0x555db6055ad0 2024-09-21T03:40:28.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.179+0000 7f93c340ef40 1 -- 172.21.15.38:0/2579886557 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:121 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x555db605b730 con 0x555db6055ad0 2024-09-21T03:40:28.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.180+0000 7f93bc15b640 1 -- 172.21.15.38:0/2579886557 <== osd.1 v2:172.21.15.38:6816/1491818732 24 ==== osd_op_reply(121 1.00000000 [create] v98'27 uv27 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f93ac0654f0 con 0x555db6055ad0 2024-09-21T03:40:28.182 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-09-21T03:40:28.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.180+0000 7f93c340ef40 1 -- 172.21.15.38:0/2579886557 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:122 13.1f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x555db605b730 con 0x555db6055ad0 2024-09-21T03:40:28.183 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.181+0000 7f93bc15b640 1 -- 172.21.15.38:0/2579886557 <== osd.1 v2:172.21.15.38:6816/1491818732 25 ==== osd_op_reply(122 1.00000000 [setxattr (30)] v98'28 uv28 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f93ac0654f0 con 0x555db6055ad0 2024-09-21T03:40:28.183 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.181+0000 7f93c340ef40 1 -- 172.21.15.38:0/2579886557 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:123 13.1f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x555db605b730 con 0x555db6055ad0 2024-09-21T03:40:28.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.182+0000 7f93bc15b640 1 -- 172.21.15.38:0/2579886557 <== osd.1 v2:172.21.15.38:6816/1491818732 26 ==== osd_op_reply(123 1.00000000 [setxattr (30)] v98'29 uv29 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f93ac0654f0 con 0x555db6055ad0 2024-09-21T03:40:28.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.182+0000 7f93c340ef40 1 -- 172.21.15.38:0/2579886557 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:124 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x555db605b730 con 0x555db6055ad0 2024-09-21T03:40:28.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.183+0000 7f93bc15b640 1 -- 172.21.15.38:0/2579886557 <== osd.1 v2:172.21.15.38:6816/1491818732 27 ==== osd_op_reply(124 1.00000000 [omap-set-header] v98'30 uv30 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f93ac0654f0 con 0x555db6055ad0 2024-09-21T03:40:28.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.183+0000 7f93c340ef40 1 -- 172.21.15.38:0/2579886557 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:125 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=584b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x555db605c420 con 0x555db6055ad0 2024-09-21T03:40:28.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.184+0000 7f93bc15b640 1 -- 172.21.15.38:0/2579886557 <== osd.1 v2:172.21.15.38:6816/1491818732 28 ==== osd_op_reply(125 1.00000000 [omap-set-vals] v98'31 uv31 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f93ac0654f0 con 0x555db6055ad0 2024-09-21T03:40:28.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.184+0000 7f93c340ef40 1 -- 172.21.15.38:0/2579886557 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x555db604d4c0 msgr2=0x555db604f8e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:28.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.184+0000 7f93c340ef40 1 --2- 172.21.15.38:0/2579886557 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x555db604d4c0 0x555db604f8e0 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:28.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.184+0000 7f93c340ef40 1 -- 172.21.15.38:0/2579886557 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x555db6055ad0 msgr2=0x555db6057ef0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:28.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.184+0000 7f93c340ef40 1 --2- 172.21.15.38:0/2579886557 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x555db6055ad0 0x555db6057ef0 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:28.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.184+0000 7f93c340ef40 1 -- 172.21.15.38:0/2579886557 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x555db60592e0 msgr2=0x555db605be90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:28.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.184+0000 7f93c340ef40 1 --2- 172.21.15.38:0/2579886557 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x555db60592e0 0x555db605be90 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:28.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.184+0000 7f93c340ef40 1 -- 172.21.15.38:0/2579886557 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x555db60508d0 msgr2=0x555db6052cf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:28.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.184+0000 7f93c340ef40 1 --2- 172.21.15.38:0/2579886557 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x555db60508d0 0x555db6052cf0 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:28.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.184+0000 7f93bc95c640 1 -- 172.21.15.38:0/2579886557 reap_dead start 2024-09-21T03:40:28.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.184+0000 7f93c340ef40 1 -- 172.21.15.38:0/2579886557 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x555db6048be0 msgr2=0x555db604b000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:28.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.184+0000 7f93bc95c640 1 -- 172.21.15.38:0/2579886557 reap_dead start 2024-09-21T03:40:28.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.184+0000 7f93c340ef40 1 --2- 172.21.15.38:0/2579886557 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x555db6048be0 0x555db604b000 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:28.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.184+0000 7f93c340ef40 1 -- 172.21.15.38:0/2579886557 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x555db603a280 msgr2=0x555db603a690 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:28.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.184+0000 7f93c340ef40 1 --2- 172.21.15.38:0/2579886557 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x555db603a280 0x555db603a690 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:28.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.184+0000 7f93c340ef40 1 -- 172.21.15.38:0/2579886557 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f93a80775a0 msgr2=0x7f93a8079ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:28.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.185+0000 7f93c340ef40 1 --2- 172.21.15.38:0/2579886557 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f93a80775a0 0x7f93a8079ab0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f93ac055000 tx=0x7f93ac058040 comp rx=0 tx=0).stop 2024-09-21T03:40:28.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.185+0000 7f93c340ef40 1 -- 172.21.15.38:0/2579886557 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555db5f8c520 msgr2=0x555db5f892a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:28.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.185+0000 7f93c340ef40 1 --2- 172.21.15.38:0/2579886557 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555db5f8c520 0x555db5f892a0 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f93b4001f00 tx=0x7f93b4003000 comp rx=0 tx=0).stop 2024-09-21T03:40:28.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.185+0000 7f93c340ef40 1 -- 172.21.15.38:0/2579886557 shutdown_connections 2024-09-21T03:40:28.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.185+0000 7f93c340ef40 1 --2- 172.21.15.38:0/2579886557 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x555db6048be0 0x555db604b000 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:28.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.185+0000 7f93c340ef40 1 --2- 172.21.15.38:0/2579886557 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x555db603a280 0x555db603a690 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:28.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.185+0000 7f93c340ef40 1 --2- 172.21.15.38:0/2579886557 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555db5f8c520 0x555db5f892a0 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:28.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.185+0000 7f93c340ef40 1 --2- 172.21.15.38:0/2579886557 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f93a80775a0 0x7f93a8079ab0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:28.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.185+0000 7f93c340ef40 1 -- 172.21.15.38:0/2579886557 >> 172.21.15.38:0/2579886557 conn(0x555db5deeac0 msgr2=0x555db5fa0430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:28.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.185+0000 7f93c340ef40 1 -- 172.21.15.38:0/2579886557 shutdown_connections 2024-09-21T03:40:28.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.185+0000 7f93c340ef40 1 -- 172.21.15.38:0/2579886557 wait complete. 2024-09-21T03:40:28.190 DEBUG:teuthology.orchestra.run.smithi038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rm 600.00000000 2024-09-21T03:40:28.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.284+0000 7f9960d1ef40 1 Processor -- start 2024-09-21T03:40:28.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.284+0000 7f9960d1ef40 1 -- start start 2024-09-21T03:40:28.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.285+0000 7f9960d1ef40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5598c4b4f240 0x5598c4b790a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:28.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.285+0000 7f9960d1ef40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5598c4b96090 0x5598c4b4dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:28.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.285+0000 7f9960d1ef40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5598c4c0b520 0x5598c4b7aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:28.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.285+0000 7f9960d1ef40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5598c4a9cd60 con 0x5598c4c0b520 2024-09-21T03:40:28.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.285+0000 7f9960d1ef40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5598c4afb430 con 0x5598c4b4f240 2024-09-21T03:40:28.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.285+0000 7f9960d1ef40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x5598c4c22aa0 con 0x5598c4b96090 2024-09-21T03:40:28.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.285+0000 7f995a50c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5598c4c0b520 0x5598c4b7aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:28.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.285+0000 7f995a50c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5598c4c0b520 0x5598c4b7aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48694/0 (socket says 172.21.15.38:48694) 2024-09-21T03:40:28.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.285+0000 7f995a50c640 1 -- 172.21.15.38:0/429024943 learned_addr learned my addr 172.21.15.38:0/429024943 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:28.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.285+0000 7f9959d0b640 1 --2- 172.21.15.38:0/429024943 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5598c4b4f240 0x5598c4b790a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:28.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.286+0000 7f995950a640 1 --2- 172.21.15.38:0/429024943 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5598c4b96090 0x5598c4b4dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:28.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.286+0000 7f995a50c640 1 -- 172.21.15.38:0/429024943 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5598c4b96090 msgr2=0x5598c4b4dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:28.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.286+0000 7f995a50c640 1 --2- 172.21.15.38:0/429024943 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5598c4b96090 0x5598c4b4dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:28.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.286+0000 7f995a50c640 1 -- 172.21.15.38:0/429024943 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5598c4b4f240 msgr2=0x5598c4b790a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:28.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.286+0000 7f995a50c640 1 --2- 172.21.15.38:0/429024943 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5598c4b4f240 0x5598c4b790a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:28.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.286+0000 7f995a50c640 1 -- 172.21.15.38:0/429024943 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5598c4b4ede0 con 0x5598c4c0b520 2024-09-21T03:40:28.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.286+0000 7f995a50c640 1 --2- 172.21.15.38:0/429024943 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5598c4c0b520 0x5598c4b7aaa0 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f995400a3e0 tx=0x7f995400a8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b3f90e3c785a00b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:28.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.286+0000 7f9953fff640 1 -- 172.21.15.38:0/429024943 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f995400b920 con 0x5598c4c0b520 2024-09-21T03:40:28.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.286+0000 7f9953fff640 1 -- 172.21.15.38:0/429024943 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f995400f030 con 0x5598c4c0b520 2024-09-21T03:40:28.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.286+0000 7f9953fff640 1 -- 172.21.15.38:0/429024943 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9954014420 con 0x5598c4c0b520 2024-09-21T03:40:28.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.287+0000 7f9960d1ef40 1 -- 172.21.15.38:0/429024943 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5598c4c0b520 msgr2=0x5598c4b7aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:28.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.287+0000 7f9960d1ef40 1 --2- 172.21.15.38:0/429024943 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5598c4c0b520 0x5598c4b7aaa0 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f995400a3e0 tx=0x7f995400a8a0 comp rx=0 tx=0).stop 2024-09-21T03:40:28.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.287+0000 7f9960d1ef40 1 -- 172.21.15.38:0/429024943 shutdown_connections 2024-09-21T03:40:28.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.287+0000 7f9960d1ef40 1 --2- 172.21.15.38:0/429024943 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5598c4c0b520 0x5598c4b7aaa0 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:28.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.287+0000 7f9960d1ef40 1 --2- 172.21.15.38:0/429024943 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5598c4b96090 0x5598c4b4dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:28.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.287+0000 7f9960d1ef40 1 --2- 172.21.15.38:0/429024943 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5598c4b4f240 0x5598c4b790a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:28.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.287+0000 7f9960d1ef40 1 -- 172.21.15.38:0/429024943 >> 172.21.15.38:0/429024943 conn(0x5598c4a6dac0 msgr2=0x5598c4c20f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:28.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.287+0000 7f9960d1ef40 1 -- 172.21.15.38:0/429024943 shutdown_connections 2024-09-21T03:40:28.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.287+0000 7f9960d1ef40 1 -- 172.21.15.38:0/429024943 wait complete. 2024-09-21T03:40:28.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.288+0000 7f9960d1ef40 1 Processor -- start 2024-09-21T03:40:28.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.288+0000 7f9960d1ef40 1 -- start start 2024-09-21T03:40:28.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.289+0000 7f9960d1ef40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5598c4b4f240 0x5598c4c06a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:28.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.289+0000 7f9960d1ef40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5598c4b96090 0x5598c4c03260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:28.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.289+0000 7f9960d1ef40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5598c4c0b520 0x5598c4b7aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:28.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.289+0000 7f9960d1ef40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5598c4b78c10 con 0x5598c4c0b520 2024-09-21T03:40:28.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.289+0000 7f9960d1ef40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5598c4b785c0 con 0x5598c4b96090 2024-09-21T03:40:28.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.289+0000 7f9960d1ef40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x5598c4b788a0 con 0x5598c4b4f240 2024-09-21T03:40:28.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.289+0000 7f995a50c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5598c4c0b520 0x5598c4b7aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:28.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.289+0000 7f995a50c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5598c4c0b520 0x5598c4b7aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48698/0 (socket says 172.21.15.38:48698) 2024-09-21T03:40:28.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.289+0000 7f995a50c640 1 -- 172.21.15.38:0/1137340509 learned_addr learned my addr 172.21.15.38:0/1137340509 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:28.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.289+0000 7f9959d0b640 1 --2- 172.21.15.38:0/1137340509 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5598c4b4f240 0x5598c4c06a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:28.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.289+0000 7f995950a640 1 --2- 172.21.15.38:0/1137340509 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5598c4b96090 0x5598c4c03260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:28.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.290+0000 7f995a50c640 1 -- 172.21.15.38:0/1137340509 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5598c4b4f240 msgr2=0x5598c4c06a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:28.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.290+0000 7f995a50c640 1 --2- 172.21.15.38:0/1137340509 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5598c4b4f240 0x5598c4c06a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:28.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.290+0000 7f995a50c640 1 -- 172.21.15.38:0/1137340509 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5598c4b96090 msgr2=0x5598c4c03260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:28.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.290+0000 7f995a50c640 1 --2- 172.21.15.38:0/1137340509 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5598c4b96090 0x5598c4c03260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:28.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.290+0000 7f995a50c640 1 -- 172.21.15.38:0/1137340509 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5598c4c059a0 con 0x5598c4c0b520 2024-09-21T03:40:28.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.290+0000 7f995a50c640 1 --2- 172.21.15.38:0/1137340509 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5598c4c0b520 0x5598c4b7aaa0 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f9954010040 tx=0x7f995400b950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:28.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.290+0000 7f9952ffd640 1 -- 172.21.15.38:0/1137340509 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f995400bdc0 con 0x5598c4c0b520 2024-09-21T03:40:28.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.290+0000 7f9960d1ef40 1 -- 172.21.15.38:0/1137340509 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5598c4c05c80 con 0x5598c4c0b520 2024-09-21T03:40:28.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.290+0000 7f9952ffd640 1 -- 172.21.15.38:0/1137340509 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f995401c420 con 0x5598c4c0b520 2024-09-21T03:40:28.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.290+0000 7f9952ffd640 1 -- 172.21.15.38:0/1137340509 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f995401c8f0 con 0x5598c4c0b520 2024-09-21T03:40:28.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.290+0000 7f9960d1ef40 1 -- 172.21.15.38:0/1137340509 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5598c4c06240 con 0x5598c4c0b520 2024-09-21T03:40:28.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.292+0000 7f9952ffd640 1 -- 172.21.15.38:0/1137340509 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f9954023030 con 0x5598c4c0b520 2024-09-21T03:40:28.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.292+0000 7f9952ffd640 1 --2- 172.21.15.38:0/1137340509 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f993c0775a0 0x7f993c079ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:28.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.292+0000 7f9952ffd640 1 -- 172.21.15.38:0/1137340509 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 6476+0+0 (secure 0 0 0) 0x7f995409a840 con 0x5598c4c0b520 2024-09-21T03:40:28.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.292+0000 7f9959d0b640 1 --2- 172.21.15.38:0/1137340509 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f993c0775a0 0x7f993c079ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:28.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.292+0000 7f9960d1ef40 1 --2- 172.21.15.38:0/1137340509 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x5598c4cbf200 0x5598c4cbf610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:28.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.292+0000 7f9960d1ef40 1 -- 172.21.15.38:0/1137340509 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:1 13.10 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e98) -- 0x5598c4c037a0 con 0x5598c4cbf200 2024-09-21T03:40:28.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.293+0000 7f995950a640 1 --2- 172.21.15.38:0/1137340509 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x5598c4cbf200 0x5598c4cbf610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:28.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.293+0000 7f9959d0b640 1 --2- 172.21.15.38:0/1137340509 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f993c0775a0 0x7f993c079ab0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f994c056000 tx=0x7f994c0033a0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:28.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.293+0000 7f995950a640 1 --2- 172.21.15.38:0/1137340509 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x5598c4cbf200 0x5598c4cbf610 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:28.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.294+0000 7f995950a640 1 -- 172.21.15.38:0/1137340509 <== osd.7 v2:172.21.15.195:6818/1606106342 1 ==== osd_op_reply(1 600.00000000 [delete] v98'72 uv72 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f994400b820 con 0x5598c4cbf200 2024-09-21T03:40:28.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.294+0000 7f9960d1ef40 1 -- 172.21.15.38:0/1137340509 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x5598c4cbf200 msgr2=0x5598c4cbf610 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:28.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.294+0000 7f9960d1ef40 1 --2- 172.21.15.38:0/1137340509 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x5598c4cbf200 0x5598c4cbf610 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:28.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.295+0000 7f9960d1ef40 1 -- 172.21.15.38:0/1137340509 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f993c0775a0 msgr2=0x7f993c079ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:28.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.295+0000 7f9960d1ef40 1 --2- 172.21.15.38:0/1137340509 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f993c0775a0 0x7f993c079ab0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f994c056000 tx=0x7f994c0033a0 comp rx=0 tx=0).stop 2024-09-21T03:40:28.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.295+0000 7f9960d1ef40 1 -- 172.21.15.38:0/1137340509 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5598c4c0b520 msgr2=0x5598c4b7aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:28.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.295+0000 7f9960d1ef40 1 --2- 172.21.15.38:0/1137340509 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5598c4c0b520 0x5598c4b7aaa0 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f9954010040 tx=0x7f995400b950 comp rx=0 tx=0).stop 2024-09-21T03:40:28.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.295+0000 7f995a50c640 1 -- 172.21.15.38:0/1137340509 reap_dead start 2024-09-21T03:40:28.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.295+0000 7f9960d1ef40 1 -- 172.21.15.38:0/1137340509 shutdown_connections 2024-09-21T03:40:28.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.295+0000 7f9960d1ef40 1 -- 172.21.15.38:0/1137340509 >> 172.21.15.38:0/1137340509 conn(0x5598c4a6dac0 msgr2=0x5598c4c20f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:28.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.295+0000 7f9960d1ef40 1 -- 172.21.15.38:0/1137340509 shutdown_connections 2024-09-21T03:40:28.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.295+0000 7f9960d1ef40 1 -- 172.21.15.38:0/1137340509 wait complete. 2024-09-21T03:40:28.299 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-21T03:40:28.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.417+0000 7f158d60e640 1 -- 172.21.15.38:0/3989766136 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f158806def0 msgr2=0x7f158806f410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:28.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.417+0000 7f158d60e640 1 --2- 172.21.15.38:0/3989766136 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f158806def0 0x7f158806f410 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f1578005fd0 tx=0x7f157802a5d0 comp rx=0 tx=0).stop 2024-09-21T03:40:28.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.417+0000 7f158d60e640 1 -- 172.21.15.38:0/3989766136 shutdown_connections 2024-09-21T03:40:28.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.417+0000 7f158d60e640 1 --2- 172.21.15.38:0/3989766136 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f15880b5670 0x7f158815f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:28.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.417+0000 7f158d60e640 1 --2- 172.21.15.38:0/3989766136 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f158806def0 0x7f158806f410 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:28.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.417+0000 7f158d60e640 1 -- 172.21.15.38:0/3989766136 >> 172.21.15.38:0/3989766136 conn(0x7f15880cdca0 msgr2=0x7f15880ce0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:28.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.417+0000 7f158d60e640 1 -- 172.21.15.38:0/3989766136 shutdown_connections 2024-09-21T03:40:28.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.417+0000 7f158d60e640 1 -- 172.21.15.38:0/3989766136 wait complete. 2024-09-21T03:40:28.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.419+0000 7f158d60e640 1 Processor -- start 2024-09-21T03:40:28.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.419+0000 7f158d60e640 1 -- start start 2024-09-21T03:40:28.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.421+0000 7f158d60e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f158806def0 0x7f158814e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:28.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.421+0000 7f158d60e640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f15880b5260 0x7f15880ca170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:28.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.421+0000 7f158d60e640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f15880b5670 0x7f15880cff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:28.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.421+0000 7f158d60e640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f158815ffb0 con 0x7f158806def0 2024-09-21T03:40:28.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.421+0000 7f158d60e640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f158815fe30 con 0x7f15880b5260 2024-09-21T03:40:28.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.421+0000 7f158d60e640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f1588071300 con 0x7f15880b5670 2024-09-21T03:40:28.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.421+0000 7f1586ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f158806def0 0x7f158814e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:28.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.421+0000 7f1586ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f158806def0 0x7f158814e460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48716/0 (socket says 172.21.15.38:48716) 2024-09-21T03:40:28.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.421+0000 7f1586ffd640 1 -- 172.21.15.38:0/3404679853 learned_addr learned my addr 172.21.15.38:0/3404679853 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:28.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.421+0000 7f157e7fc640 1 --2- 172.21.15.38:0/3404679853 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f15880b5260 0x7f15880ca170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:28.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.421+0000 7f15877fe640 1 --2- 172.21.15.38:0/3404679853 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f15880b5670 0x7f15880cff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:28.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.421+0000 7f1586ffd640 1 -- 172.21.15.38:0/3404679853 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f15880b5670 msgr2=0x7f15880cff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:28.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.421+0000 7f1586ffd640 1 --2- 172.21.15.38:0/3404679853 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f15880b5670 0x7f15880cff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:28.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.421+0000 7f1586ffd640 1 -- 172.21.15.38:0/3404679853 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f15880b5260 msgr2=0x7f15880ca170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:28.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.421+0000 7f1586ffd640 1 --2- 172.21.15.38:0/3404679853 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f15880b5260 0x7f15880ca170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:28.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.421+0000 7f1586ffd640 1 -- 172.21.15.38:0/3404679853 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1588150420 con 0x7f158806def0 2024-09-21T03:40:28.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.421+0000 7f1586ffd640 1 --2- 172.21.15.38:0/3404679853 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f158806def0 0x7f158814e460 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f1578034040 tx=0x7f15780035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:28.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.422+0000 7f157ffff640 1 -- 172.21.15.38:0/3404679853 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f157803e030 con 0x7f158806def0 2024-09-21T03:40:28.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.422+0000 7f158d60e640 1 -- 172.21.15.38:0/3404679853 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15881f7430 con 0x7f158806def0 2024-09-21T03:40:28.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.422+0000 7f157ffff640 1 -- 172.21.15.38:0/3404679853 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1578003ae0 con 0x7f158806def0 2024-09-21T03:40:28.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.422+0000 7f157ffff640 1 -- 172.21.15.38:0/3404679853 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1578035040 con 0x7f158806def0 2024-09-21T03:40:28.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.422+0000 7f158d60e640 1 -- 172.21.15.38:0/3404679853 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15881509a0 con 0x7f158806def0 2024-09-21T03:40:28.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.423+0000 7f158d60e640 1 -- 172.21.15.38:0/3404679853 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f158815f630 con 0x7f158806def0 2024-09-21T03:40:28.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.423+0000 7f157ffff640 1 -- 172.21.15.38:0/3404679853 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f1578041030 con 0x7f158806def0 2024-09-21T03:40:28.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.424+0000 7f157ffff640 1 --2- 172.21.15.38:0/3404679853 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1568078b90 0x7f156807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:28.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.424+0000 7f157ffff640 1 -- 172.21.15.38:0/3404679853 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 6476+0+0 (secure 0 0 0) 0x7f15780bb5a0 con 0x7f158806def0 2024-09-21T03:40:28.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.424+0000 7f157e7fc640 1 --2- 172.21.15.38:0/3404679853 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1568078b90 0x7f156807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:28.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.425+0000 7f157e7fc640 1 --2- 172.21.15.38:0/3404679853 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1568078b90 0x7f156807b0a0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f1588074b20 tx=0x7f1570009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:28.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.428+0000 7f157ffff640 1 -- 172.21.15.38:0/3404679853 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f1578082f70 con 0x7f158806def0 2024-09-21T03:40:28.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.656+0000 7f158d60e640 1 -- 172.21.15.38:0/3404679853 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f158806d940 con 0x7f158806def0 2024-09-21T03:40:28.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.954+0000 7f157ffff640 1 -- 172.21.15.38:0/3404679853 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v95) ==== 162+0+0 (secure 0 0 0) 0x7f15780879a0 con 0x7f158806def0 2024-09-21T03:40:28.957 INFO:teuthology.orchestra.run.smithi038.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-21T03:40:28.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.957+0000 7f158d60e640 1 -- 172.21.15.38:0/3404679853 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1568078b90 msgr2=0x7f156807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:28.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.957+0000 7f158d60e640 1 --2- 172.21.15.38:0/3404679853 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1568078b90 0x7f156807b0a0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f1588074b20 tx=0x7f1570009040 comp rx=0 tx=0).stop 2024-09-21T03:40:28.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.957+0000 7f158d60e640 1 -- 172.21.15.38:0/3404679853 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f158806def0 msgr2=0x7f158814e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:28.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.957+0000 7f158d60e640 1 --2- 172.21.15.38:0/3404679853 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f158806def0 0x7f158814e460 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f1578034040 tx=0x7f15780035f0 comp rx=0 tx=0).stop 2024-09-21T03:40:28.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.957+0000 7f158d60e640 1 -- 172.21.15.38:0/3404679853 shutdown_connections 2024-09-21T03:40:28.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.957+0000 7f158d60e640 1 --2- 172.21.15.38:0/3404679853 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1568078b90 0x7f156807b0a0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:28.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.957+0000 7f158d60e640 1 --2- 172.21.15.38:0/3404679853 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f15880b5670 0x7f15880cff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:28.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.957+0000 7f158d60e640 1 --2- 172.21.15.38:0/3404679853 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f15880b5260 0x7f15880ca170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:28.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.957+0000 7f158d60e640 1 --2- 172.21.15.38:0/3404679853 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f158806def0 0x7f158814e460 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:28.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.957+0000 7f158d60e640 1 -- 172.21.15.38:0/3404679853 >> 172.21.15.38:0/3404679853 conn(0x7f15880cdca0 msgr2=0x7f15880ae1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:28.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.957+0000 7f158d60e640 1 -- 172.21.15.38:0/3404679853 shutdown_connections 2024-09-21T03:40:28.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:28.957+0000 7f158d60e640 1 -- 172.21.15.38:0/3404679853 wait complete. 2024-09-21T03:40:28.973 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:40:29.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:29.088+0000 7f78c5094640 1 -- 172.21.15.38:0/23648916 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f78a80051a0 msgr2=0x7f78a800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:29.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:29.088+0000 7f78c5094640 1 --2- 172.21.15.38:0/23648916 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f78a80051a0 0x7f78a800b9c0 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f78b800c610 tx=0x7f78a8005830 comp rx=0 tx=0).stop 2024-09-21T03:40:29.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:29.088+0000 7f78bf59e640 1 -- 172.21.15.38:0/23648916 reap_dead start 2024-09-21T03:40:29.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:29.088+0000 7f78c5094640 1 -- 172.21.15.38:0/23648916 shutdown_connections 2024-09-21T03:40:29.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:29.088+0000 7f78c5094640 1 -- 172.21.15.38:0/23648916 >> 172.21.15.38:0/23648916 conn(0x7f78c00b6b20 msgr2=0x7f78c00d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:29.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:29.088+0000 7f78c5094640 1 -- 172.21.15.38:0/23648916 shutdown_connections 2024-09-21T03:40:29.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:29.088+0000 7f78c5094640 1 -- 172.21.15.38:0/23648916 wait complete. 2024-09-21T03:40:29.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:29.090+0000 7f78c5094640 1 Processor -- start 2024-09-21T03:40:29.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:29.090+0000 7f78c5094640 1 -- start start 2024-09-21T03:40:29.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:29.091+0000 7f78c5094640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f78c015a7b0 0x7f78c014aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:29.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:29.091+0000 7f78c5094640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f78c00c9ec0 0x7f78c014b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:29.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:29.091+0000 7f78c5094640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f78c00ce650 0x7f78c00cea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:29.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:29.091+0000 7f78c5094640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f78c015f640 con 0x7f78c00ce650 2024-09-21T03:40:29.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:29.091+0000 7f78c5094640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f78c015f4c0 con 0x7f78c015a7b0 2024-09-21T03:40:29.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:29.091+0000 7f78c5094640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f78c015f7c0 con 0x7f78c00c9ec0 2024-09-21T03:40:29.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:29.091+0000 7f78bf59e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f78c00ce650 0x7f78c00cea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:29.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:29.092+0000 7f78bf59e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f78c00ce650 0x7f78c00cea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48746/0 (socket says 172.21.15.38:48746) 2024-09-21T03:40:29.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:29.092+0000 7f78bf59e640 1 -- 172.21.15.38:0/2657919166 learned_addr learned my addr 172.21.15.38:0/2657919166 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:29.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:29.092+0000 7f78bed9d640 1 --2- 172.21.15.38:0/2657919166 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f78c015a7b0 0x7f78c014aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:29.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:29.092+0000 7f78be59c640 1 --2- 172.21.15.38:0/2657919166 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f78c00c9ec0 0x7f78c014b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:29.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:29.092+0000 7f78bf59e640 1 -- 172.21.15.38:0/2657919166 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f78c00c9ec0 msgr2=0x7f78c014b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:29.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:29.092+0000 7f78bf59e640 1 --2- 172.21.15.38:0/2657919166 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f78c00c9ec0 0x7f78c014b6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:29.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:29.092+0000 7f78bf59e640 1 -- 172.21.15.38:0/2657919166 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f78c015a7b0 msgr2=0x7f78c014aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:29.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:29.092+0000 7f78bf59e640 1 --2- 172.21.15.38:0/2657919166 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f78c015a7b0 0x7f78c014aea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:29.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:29.092+0000 7f78bf59e640 1 -- 172.21.15.38:0/2657919166 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78c014b3e0 con 0x7f78c00ce650 2024-09-21T03:40:29.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:29.092+0000 7f78bed9d640 1 --2- 172.21.15.38:0/2657919166 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f78c015a7b0 0x7f78c014aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:40:29.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:29.092+0000 7f78bf59e640 1 --2- 172.21.15.38:0/2657919166 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f78c00ce650 0x7f78c00cea60 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f78b800c730 tx=0x7f78b8004c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:29.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:29.092+0000 7f78a77fe640 1 -- 172.21.15.38:0/2657919166 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78b802b910 con 0x7f78c00ce650 2024-09-21T03:40:29.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:29.092+0000 7f78a77fe640 1 -- 172.21.15.38:0/2657919166 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f78b802bba0 con 0x7f78c00ce650 2024-09-21T03:40:29.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:29.092+0000 7f78c5094640 1 -- 172.21.15.38:0/2657919166 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78c00c7f30 con 0x7f78c00ce650 2024-09-21T03:40:29.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:29.092+0000 7f78a77fe640 1 -- 172.21.15.38:0/2657919166 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78b80060a0 con 0x7f78c00ce650 2024-09-21T03:40:29.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:29.092+0000 7f78c5094640 1 -- 172.21.15.38:0/2657919166 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78c00c84f0 con 0x7f78c00ce650 2024-09-21T03:40:29.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:29.094+0000 7f78c5094640 1 -- 172.21.15.38:0/2657919166 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78c00b3250 con 0x7f78c00ce650 2024-09-21T03:40:29.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:29.094+0000 7f78a77fe640 1 -- 172.21.15.38:0/2657919166 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f78b8006240 con 0x7f78c00ce650 2024-09-21T03:40:29.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:29.095+0000 7f78a77fe640 1 --2- 172.21.15.38:0/2657919166 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f78ac078b90 0x7f78ac07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:29.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:29.095+0000 7f78a77fe640 1 -- 172.21.15.38:0/2657919166 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 6476+0+0 (secure 0 0 0) 0x7f78b80bad80 con 0x7f78c00ce650 2024-09-21T03:40:29.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:29.095+0000 7f78bed9d640 1 --2- 172.21.15.38:0/2657919166 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f78ac078b90 0x7f78ac07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:29.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:29.095+0000 7f78bed9d640 1 --2- 172.21.15.38:0/2657919166 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f78ac078b90 0x7f78ac07b0a0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f78b0009c50 tx=0x7f78b0005f90 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:29.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:29.099+0000 7f78a77fe640 1 -- 172.21.15.38:0/2657919166 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f78b8082fe0 con 0x7f78c00ce650 2024-09-21T03:40:29.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:29.323+0000 7f78c5094640 1 -- 172.21.15.38:0/2657919166 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f78c015ddd0 con 0x7f78c00ce650 2024-09-21T03:40:29.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:29.324+0000 7f78a77fe640 1 -- 172.21.15.38:0/2657919166 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 96 v96) ==== 94+0+5219 (secure 0 0 0) 0x7f78b80828e0 con 0x7f78c00ce650 2024-09-21T03:40:29.326 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:40:29.326 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":96,"btime":"2024-09-21T03:40:28:958279+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6086,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/965972663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":965972663},{"type":"v1","addr":"172.21.15.195:6836","nonce":965972663}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6700,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/3176122334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":3176122334},{"type":"v1","addr":"172.21.15.195:6837","nonce":3176122334}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":7020,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/751618334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":751618334},{"type":"v1","addr":"172.21.15.38:6833","nonce":751618334}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":96,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:40:02.319282+0000","modified":"2024-09-21T03:40:28.958261+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":98,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7737},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7737":{"gid":7737,"name":"a","rank":0,"incarnation":96,"state":"up:replay","state_seq":1,"addr":"172.21.15.38:6835/3864487171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":3864487171},{"type":"v1","addr":"172.21.15.38:6835","nonce":3864487171}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-21T03:40:29.326 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 96 2024-09-21T03:40:29.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:29.325+0000 7f78c5094640 1 -- 172.21.15.38:0/2657919166 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f78ac078b90 msgr2=0x7f78ac07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:29.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:29.325+0000 7f78c5094640 1 --2- 172.21.15.38:0/2657919166 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f78ac078b90 0x7f78ac07b0a0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f78b0009c50 tx=0x7f78b0005f90 comp rx=0 tx=0).stop 2024-09-21T03:40:29.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:29.326+0000 7f78c5094640 1 -- 172.21.15.38:0/2657919166 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f78c00ce650 msgr2=0x7f78c00cea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:29.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:29.326+0000 7f78c5094640 1 --2- 172.21.15.38:0/2657919166 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f78c00ce650 0x7f78c00cea60 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f78b800c730 tx=0x7f78b8004c90 comp rx=0 tx=0).stop 2024-09-21T03:40:29.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:29.326+0000 7f78c5094640 1 -- 172.21.15.38:0/2657919166 shutdown_connections 2024-09-21T03:40:29.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:29.326+0000 7f78c5094640 1 --2- 172.21.15.38:0/2657919166 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f78c00ce650 0x7f78c00cea60 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:29.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:29.326+0000 7f78c5094640 1 --2- 172.21.15.38:0/2657919166 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f78ac078b90 0x7f78ac07b0a0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:29.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:29.326+0000 7f78c5094640 1 --2- 172.21.15.38:0/2657919166 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f78c00c9ec0 0x7f78c014b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:29.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:29.326+0000 7f78c5094640 1 --2- 172.21.15.38:0/2657919166 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f78c015a7b0 0x7f78c014aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:29.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:29.326+0000 7f78c5094640 1 -- 172.21.15.38:0/2657919166 >> 172.21.15.38:0/2657919166 conn(0x7f78c00b6b20 msgr2=0x7f78c006cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:29.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:29.326+0000 7f78c5094640 1 -- 172.21.15.38:0/2657919166 shutdown_connections 2024-09-21T03:40:29.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:29.326+0000 7f78c5094640 1 -- 172.21.15.38:0/2657919166 wait complete. 2024-09-21T03:40:29.339 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2024-09-21T03:40:31.046 DEBUG:teuthology.orchestra.run.smithi038:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T03:40:31.049 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T03:40:31.988 INFO:tasks.ceph.mds.a.smithi038.stderr:2024-09-21T03:40:31.987+0000 7f4e8b8d6640 -1 log_channel(cluster) log [ERR] : dir 0x600 object missing on disk; some files may be lost (~mds0/stray0) 2024-09-21T03:40:31.989 INFO:tasks.ceph.mds.a.smithi038.stderr:2024-09-21T03:40:31.987+0000 7f4e8b8d6640 -1 mds.0.damage notify_dirfrag Damage to fragment * of ino 0x600 is fatal because it is a system directory for this rank 2024-09-21T03:40:31.991 INFO:tasks.ceph.mon.a.smithi038.stderr:2024-09-21T03:40:31.989+0000 7f3f45713640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T03:40:31.991 INFO:tasks.ceph.mon.a.smithi038.stderr:2024-09-21T03:40:31.989+0000 7f3f45713640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-09-21T03:40:33.956 INFO:tasks.ceph.mds.a.smithi038.stderr:2024-09-21T03:40:33.954+0000 7f4e8f8de640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T03:40:34.340 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:40:34.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:34.470+0000 7f8f51ec5640 1 -- 172.21.15.38:0/3872720884 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8f4c162f00 msgr2=0x7f8f4c0d24d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:34.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:34.470+0000 7f8f51ec5640 1 --2- 172.21.15.38:0/3872720884 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8f4c162f00 0x7f8f4c0d24d0 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c005fd0 tx=0x7f8f3c02a5d0 comp rx=0 tx=0).stop 2024-09-21T03:40:34.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:34.470+0000 7f8f51ec5640 1 -- 172.21.15.38:0/3872720884 shutdown_connections 2024-09-21T03:40:34.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:34.470+0000 7f8f51ec5640 1 --2- 172.21.15.38:0/3872720884 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8f4c162280 0x7f8f4c162690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:34.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:34.470+0000 7f8f51ec5640 1 --2- 172.21.15.38:0/3872720884 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8f4c162f00 0x7f8f4c0d24d0 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:34.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:34.470+0000 7f8f51ec5640 1 -- 172.21.15.38:0/3872720884 >> 172.21.15.38:0/3872720884 conn(0x7f8f4c0b6b20 msgr2=0x7f8f4c0d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:34.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:34.470+0000 7f8f51ec5640 1 -- 172.21.15.38:0/3872720884 shutdown_connections 2024-09-21T03:40:34.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:34.471+0000 7f8f51ec5640 1 -- 172.21.15.38:0/3872720884 wait complete. 2024-09-21T03:40:34.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:34.472+0000 7f8f51ec5640 1 Processor -- start 2024-09-21T03:40:34.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:34.472+0000 7f8f51ec5640 1 -- start start 2024-09-21T03:40:34.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:34.474+0000 7f8f51ec5640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8f4c0b2e40 0x7f8f4c0b2870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:34.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:34.474+0000 7f8f51ec5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8f4c162280 0x7f8f4c1f4c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:34.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:34.474+0000 7f8f51ec5640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8f4c0b0a70 0x7f8f4c14c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:34.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:34.474+0000 7f8f51ec5640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f8f4c15e630 con 0x7f8f4c162280 2024-09-21T03:40:34.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:34.474+0000 7f8f51ec5640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f8f4c15e4b0 con 0x7f8f4c0b0a70 2024-09-21T03:40:34.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:34.474+0000 7f8f51ec5640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f8f4c15e7b0 con 0x7f8f4c0b2e40 2024-09-21T03:40:34.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:34.474+0000 7f8f4affd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8f4c162280 0x7f8f4c1f4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:34.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:34.474+0000 7f8f4affd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8f4c162280 0x7f8f4c1f4c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:45112/0 (socket says 172.21.15.38:45112) 2024-09-21T03:40:34.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:34.474+0000 7f8f4affd640 1 -- 172.21.15.38:0/556176872 learned_addr learned my addr 172.21.15.38:0/556176872 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:34.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:34.474+0000 7f8f4b7fe640 1 --2- 172.21.15.38:0/556176872 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8f4c0b2e40 0x7f8f4c0b2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:34.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:34.474+0000 7f8f4bfff640 1 --2- 172.21.15.38:0/556176872 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8f4c0b0a70 0x7f8f4c14c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:34.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:34.474+0000 7f8f4affd640 1 -- 172.21.15.38:0/556176872 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8f4c0b2e40 msgr2=0x7f8f4c0b2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:34.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:34.474+0000 7f8f4affd640 1 --2- 172.21.15.38:0/556176872 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8f4c0b2e40 0x7f8f4c0b2870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:34.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:34.474+0000 7f8f4affd640 1 -- 172.21.15.38:0/556176872 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8f4c0b0a70 msgr2=0x7f8f4c14c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:34.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:34.474+0000 7f8f4affd640 1 --2- 172.21.15.38:0/556176872 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8f4c0b0a70 0x7f8f4c14c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:34.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:34.474+0000 7f8f4affd640 1 -- 172.21.15.38:0/556176872 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f4c1f5520 con 0x7f8f4c162280 2024-09-21T03:40:34.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:34.474+0000 7f8f4affd640 1 --2- 172.21.15.38:0/556176872 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8f4c162280 0x7f8f4c1f4c80 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f8f4000d780 tx=0x7f8f4000dc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:34.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:34.475+0000 7f8f33fff640 1 -- 172.21.15.38:0/556176872 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f40017070 con 0x7f8f4c162280 2024-09-21T03:40:34.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:34.475+0000 7f8f33fff640 1 -- 172.21.15.38:0/556176872 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8f40009b10 con 0x7f8f4c162280 2024-09-21T03:40:34.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:34.475+0000 7f8f51ec5640 1 -- 172.21.15.38:0/556176872 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f4c1f5800 con 0x7f8f4c162280 2024-09-21T03:40:34.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:34.475+0000 7f8f33fff640 1 -- 172.21.15.38:0/556176872 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f40010460 con 0x7f8f4c162280 2024-09-21T03:40:34.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:34.475+0000 7f8f51ec5640 1 -- 172.21.15.38:0/556176872 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f4c14e9a0 con 0x7f8f4c162280 2024-09-21T03:40:34.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:34.476+0000 7f8f51ec5640 1 -- 172.21.15.38:0/556176872 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f4c162f00 con 0x7f8f4c162280 2024-09-21T03:40:34.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:34.477+0000 7f8f33fff640 1 -- 172.21.15.38:0/556176872 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f8f4001a070 con 0x7f8f4c162280 2024-09-21T03:40:34.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:34.477+0000 7f8f33fff640 1 --2- 172.21.15.38:0/556176872 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8f38078bc0 0x7f8f3807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:34.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:34.477+0000 7f8f33fff640 1 -- 172.21.15.38:0/556176872 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 6562+0+0 (secure 0 0 0) 0x7f8f4009b120 con 0x7f8f4c162280 2024-09-21T03:40:34.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:34.477+0000 7f8f4b7fe640 1 --2- 172.21.15.38:0/556176872 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8f38078bc0 0x7f8f3807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:34.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:34.478+0000 7f8f4b7fe640 1 --2- 172.21.15.38:0/556176872 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8f38078bc0 0x7f8f3807b0d0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c033000 tx=0x7f8f3c004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:34.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:34.482+0000 7f8f33fff640 1 -- 172.21.15.38:0/556176872 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f8f40062c40 con 0x7f8f4c162280 2024-09-21T03:40:34.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:34.710+0000 7f8f51ec5640 1 -- 172.21.15.38:0/556176872 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8f4c1f52b0 con 0x7f8f4c162280 2024-09-21T03:40:34.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:34.711+0000 7f8f33fff640 1 -- 172.21.15.38:0/556176872 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 100 v100) ==== 95+0+4387 (secure 0 0 0) 0x7f8f40067670 con 0x7f8f4c162280 2024-09-21T03:40:34.713 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:40:34.713 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":100,"btime":"2024-09-21T03:40:31:990176+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6086,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/965972663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":965972663},{"type":"v1","addr":"172.21.15.195:6836","nonce":965972663}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6700,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/3176122334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":3176122334},{"type":"v1","addr":"172.21.15.195:6837","nonce":3176122334}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":7020,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/751618334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":751618334},{"type":"v1","addr":"172.21.15.38:6833","nonce":751618334}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":100,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:40:02.319282+0000","modified":"2024-09-21T03:40:31.990169+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":99,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-21T03:40:34.713 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 100 2024-09-21T03:40:34.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:34.713+0000 7f8f51ec5640 1 -- 172.21.15.38:0/556176872 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8f38078bc0 msgr2=0x7f8f3807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:34.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:34.713+0000 7f8f51ec5640 1 --2- 172.21.15.38:0/556176872 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8f38078bc0 0x7f8f3807b0d0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c033000 tx=0x7f8f3c004730 comp rx=0 tx=0).stop 2024-09-21T03:40:34.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:34.713+0000 7f8f51ec5640 1 -- 172.21.15.38:0/556176872 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8f4c162280 msgr2=0x7f8f4c1f4c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:34.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:34.713+0000 7f8f51ec5640 1 --2- 172.21.15.38:0/556176872 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8f4c162280 0x7f8f4c1f4c80 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f8f4000d780 tx=0x7f8f4000dc40 comp rx=0 tx=0).stop 2024-09-21T03:40:34.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:34.713+0000 7f8f51ec5640 1 -- 172.21.15.38:0/556176872 shutdown_connections 2024-09-21T03:40:34.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:34.713+0000 7f8f51ec5640 1 --2- 172.21.15.38:0/556176872 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8f4c0b0a70 0x7f8f4c14c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:34.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:34.713+0000 7f8f51ec5640 1 --2- 172.21.15.38:0/556176872 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8f4c162280 0x7f8f4c1f4c80 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:34.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:34.713+0000 7f8f51ec5640 1 --2- 172.21.15.38:0/556176872 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8f38078bc0 0x7f8f3807b0d0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:34.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:34.713+0000 7f8f51ec5640 1 --2- 172.21.15.38:0/556176872 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8f4c0b2e40 0x7f8f4c0b2870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:34.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:34.713+0000 7f8f51ec5640 1 -- 172.21.15.38:0/556176872 >> 172.21.15.38:0/556176872 conn(0x7f8f4c0b6b20 msgr2=0x7f8f4c15daa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:34.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:34.713+0000 7f8f51ec5640 1 -- 172.21.15.38:0/556176872 shutdown_connections 2024-09-21T03:40:34.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:34.713+0000 7f8f51ec5640 1 -- 172.21.15.38:0/556176872 wait complete. 2024-09-21T03:40:34.726 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-09-21T03:40:34.727 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:40:34.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:34.841+0000 7fa2c8752640 1 -- 172.21.15.38:0/3526882677 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa2c015cb10 msgr2=0x7fa2c015cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:34.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:34.841+0000 7fa2c8752640 1 --2- 172.21.15.38:0/3526882677 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa2c015cb10 0x7fa2c015cf20 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7fa2b0005fd0 tx=0x7fa2b002a940 comp rx=0 tx=0).stop 2024-09-21T03:40:34.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:34.841+0000 7fa2c8752640 1 -- 172.21.15.38:0/3526882677 shutdown_connections 2024-09-21T03:40:34.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:34.841+0000 7fa2c8752640 1 --2- 172.21.15.38:0/3526882677 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa2c015cb10 0x7fa2c015cf20 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:34.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:34.841+0000 7fa2c8752640 1 -- 172.21.15.38:0/3526882677 >> 172.21.15.38:0/3526882677 conn(0x7fa2c00d2350 msgr2=0x7fa2c00d2760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:34.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:34.841+0000 7fa2c8752640 1 -- 172.21.15.38:0/3526882677 shutdown_connections 2024-09-21T03:40:34.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:34.841+0000 7fa2c8752640 1 -- 172.21.15.38:0/3526882677 wait complete. 2024-09-21T03:40:34.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:34.843+0000 7fa2c8752640 1 Processor -- start 2024-09-21T03:40:34.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:34.843+0000 7fa2c8752640 1 -- start start 2024-09-21T03:40:34.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:34.844+0000 7fa2c8752640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa2c00b6250 0x7fa2c0159fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:34.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:34.844+0000 7fa2c8752640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa2c00d12e0 0x7fa2c0140470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:34.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:34.844+0000 7fa2c8752640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa2c015cb10 0x7fa2c0071210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:34.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:34.844+0000 7fa2c8752640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa2c00b7100 con 0x7fa2c015cb10 2024-09-21T03:40:34.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:34.844+0000 7fa2c8752640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fa2c00b6f80 con 0x7fa2c00d12e0 2024-09-21T03:40:34.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:34.844+0000 7fa2c8752640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fa2c00b7280 con 0x7fa2c00b6250 2024-09-21T03:40:34.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:34.844+0000 7fa2c6cef640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa2c015cb10 0x7fa2c0071210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:34.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:34.844+0000 7fa2c6cef640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa2c015cb10 0x7fa2c0071210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:45122/0 (socket says 172.21.15.38:45122) 2024-09-21T03:40:34.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:34.844+0000 7fa2c6cef640 1 -- 172.21.15.38:0/2500930418 learned_addr learned my addr 172.21.15.38:0/2500930418 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:34.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:34.844+0000 7fa2c64ee640 1 --2- 172.21.15.38:0/2500930418 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa2c00b6250 0x7fa2c0159fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:34.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:34.844+0000 7fa2c5ced640 1 --2- 172.21.15.38:0/2500930418 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa2c00d12e0 0x7fa2c0140470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:34.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:34.844+0000 7fa2c6cef640 1 -- 172.21.15.38:0/2500930418 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa2c00b6250 msgr2=0x7fa2c0159fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:34.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:34.844+0000 7fa2c6cef640 1 --2- 172.21.15.38:0/2500930418 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa2c00b6250 0x7fa2c0159fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:34.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:34.844+0000 7fa2c6cef640 1 -- 172.21.15.38:0/2500930418 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa2c00d12e0 msgr2=0x7fa2c0140470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:34.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:34.844+0000 7fa2c6cef640 1 --2- 172.21.15.38:0/2500930418 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa2c00d12e0 0x7fa2c0140470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:34.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:34.844+0000 7fa2c6cef640 1 -- 172.21.15.38:0/2500930418 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2c01588b0 con 0x7fa2c015cb10 2024-09-21T03:40:34.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:34.844+0000 7fa2c5ced640 1 --2- 172.21.15.38:0/2500930418 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa2c00d12e0 0x7fa2c0140470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:40:34.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:34.845+0000 7fa2c6cef640 1 --2- 172.21.15.38:0/2500930418 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa2c015cb10 0x7fa2c0071210 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7fa2b800c3f0 tx=0x7fa2b800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:34.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:34.845+0000 7fa2beffd640 1 -- 172.21.15.38:0/2500930418 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2b8017070 con 0x7fa2c015cb10 2024-09-21T03:40:34.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:34.845+0000 7fa2beffd640 1 -- 172.21.15.38:0/2500930418 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa2b80053c0 con 0x7fa2c015cb10 2024-09-21T03:40:34.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:34.845+0000 7fa2c8752640 1 -- 172.21.15.38:0/2500930418 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2c0158b90 con 0x7fa2c015cb10 2024-09-21T03:40:34.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:34.845+0000 7fa2beffd640 1 -- 172.21.15.38:0/2500930418 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2b8003850 con 0x7fa2c015cb10 2024-09-21T03:40:34.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:34.845+0000 7fa2c8752640 1 -- 172.21.15.38:0/2500930418 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2c0159150 con 0x7fa2c015cb10 2024-09-21T03:40:34.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:34.846+0000 7fa2c8752640 1 -- 172.21.15.38:0/2500930418 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2c015cf20 con 0x7fa2c015cb10 2024-09-21T03:40:34.850 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:34.847+0000 7fa2beffd640 1 -- 172.21.15.38:0/2500930418 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa2b801a070 con 0x7fa2c015cb10 2024-09-21T03:40:34.850 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:34.847+0000 7fa2beffd640 1 --2- 172.21.15.38:0/2500930418 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa2ac078b90 0x7fa2ac07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:34.850 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:34.847+0000 7fa2beffd640 1 -- 172.21.15.38:0/2500930418 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 6562+0+0 (secure 0 0 0) 0x7fa2b809a2d0 con 0x7fa2c015cb10 2024-09-21T03:40:34.850 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:34.847+0000 7fa2c64ee640 1 --2- 172.21.15.38:0/2500930418 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa2ac078b90 0x7fa2ac07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:34.850 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:34.848+0000 7fa2c64ee640 1 --2- 172.21.15.38:0/2500930418 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa2ac078b90 0x7fa2ac07b0a0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fa2b0034000 tx=0x7fa2b0004ab0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:34.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:34.852+0000 7fa2beffd640 1 -- 172.21.15.38:0/2500930418 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fa2b8061e20 con 0x7fa2c015cb10 2024-09-21T03:40:35.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:35.079+0000 7fa2c8752640 1 -- 172.21.15.38:0/2500930418 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa2c0070480 con 0x7fa2c015cb10 2024-09-21T03:40:35.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:35.079+0000 7fa2beffd640 1 -- 172.21.15.38:0/2500930418 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 100 v100) ==== 95+0+4387 (secure 0 0 0) 0x7fa2b8066850 con 0x7fa2c015cb10 2024-09-21T03:40:35.081 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:40:35.081 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":100,"btime":"2024-09-21T03:40:31:990176+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6086,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/965972663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":965972663},{"type":"v1","addr":"172.21.15.195:6836","nonce":965972663}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6700,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/3176122334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":3176122334},{"type":"v1","addr":"172.21.15.195:6837","nonce":3176122334}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":7020,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/751618334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":751618334},{"type":"v1","addr":"172.21.15.38:6833","nonce":751618334}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":100,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:40:02.319282+0000","modified":"2024-09-21T03:40:31.990169+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":99,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-21T03:40:35.081 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 100 2024-09-21T03:40:35.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:35.081+0000 7fa2c8752640 1 -- 172.21.15.38:0/2500930418 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa2ac078b90 msgr2=0x7fa2ac07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:35.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:35.081+0000 7fa2c8752640 1 --2- 172.21.15.38:0/2500930418 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa2ac078b90 0x7fa2ac07b0a0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fa2b0034000 tx=0x7fa2b0004ab0 comp rx=0 tx=0).stop 2024-09-21T03:40:35.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:35.081+0000 7fa2c8752640 1 -- 172.21.15.38:0/2500930418 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa2c015cb10 msgr2=0x7fa2c0071210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:35.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:35.081+0000 7fa2c8752640 1 --2- 172.21.15.38:0/2500930418 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa2c015cb10 0x7fa2c0071210 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7fa2b800c3f0 tx=0x7fa2b800c8b0 comp rx=0 tx=0).stop 2024-09-21T03:40:35.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:35.081+0000 7fa2c8752640 1 -- 172.21.15.38:0/2500930418 shutdown_connections 2024-09-21T03:40:35.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:35.081+0000 7fa2c8752640 1 --2- 172.21.15.38:0/2500930418 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa2c015cb10 0x7fa2c0071210 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:35.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:35.081+0000 7fa2c8752640 1 --2- 172.21.15.38:0/2500930418 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa2c00d12e0 0x7fa2c0140470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:35.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:35.081+0000 7fa2c8752640 1 --2- 172.21.15.38:0/2500930418 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa2ac078b90 0x7fa2ac07b0a0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:35.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:35.081+0000 7fa2c8752640 1 --2- 172.21.15.38:0/2500930418 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa2c00b6250 0x7fa2c0159fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:35.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:35.081+0000 7fa2c8752640 1 -- 172.21.15.38:0/2500930418 >> 172.21.15.38:0/2500930418 conn(0x7fa2c00d2350 msgr2=0x7fa2c00d2760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:35.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:35.081+0000 7fa2c8752640 1 -- 172.21.15.38:0/2500930418 shutdown_connections 2024-09-21T03:40:35.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:35.081+0000 7fa2c8752640 1 -- 172.21.15.38:0/2500930418 wait complete. 2024-09-21T03:40:35.094 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-21T03:40:35.095 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:40:35.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:35.209+0000 7fd10adba640 1 -- 172.21.15.38:0/4203780153 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd1040b2e40 msgr2=0x7fd1040b3250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:35.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:35.209+0000 7fd10adba640 1 --2- 172.21.15.38:0/4203780153 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd1040b2e40 0x7fd1040b3250 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7fd1040d5250 tx=0x7fd0fc02aa30 comp rx=0 tx=0).stop 2024-09-21T03:40:35.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:35.209+0000 7fd10adba640 1 -- 172.21.15.38:0/4203780153 shutdown_connections 2024-09-21T03:40:35.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:35.209+0000 7fd10adba640 1 --2- 172.21.15.38:0/4203780153 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd1040b2e40 0x7fd1040b3250 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:35.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:35.209+0000 7fd10adba640 1 --2- 172.21.15.38:0/4203780153 >> v2:172.21.15.195:3300/0 conn(0x7fd104162280 0x7fd104162690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:35.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:35.209+0000 7fd10adba640 1 --2- 172.21.15.38:0/4203780153 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd104162f00 0x7fd1040d24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:35.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:35.209+0000 7fd10adba640 1 -- 172.21.15.38:0/4203780153 >> 172.21.15.38:0/4203780153 conn(0x7fd1040b6b20 msgr2=0x7fd1040d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:35.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:35.209+0000 7fd10adba640 1 -- 172.21.15.38:0/4203780153 shutdown_connections 2024-09-21T03:40:35.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:35.209+0000 7fd10adba640 1 -- 172.21.15.38:0/4203780153 wait complete. 2024-09-21T03:40:35.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:35.211+0000 7fd10adba640 1 Processor -- start 2024-09-21T03:40:35.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:35.211+0000 7fd10adba640 1 -- start start 2024-09-21T03:40:35.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:35.213+0000 7fd10adba640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd1040b2e40 0x7fd1041f5e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:35.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:35.213+0000 7fd10adba640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd104162280 0x7fd1040ce370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:35.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:35.213+0000 7fd10adba640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd104159840 0x7fd10415e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:35.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:35.213+0000 7fd10adba640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fd1040b3dd0 con 0x7fd1040b2e40 2024-09-21T03:40:35.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:35.213+0000 7fd10adba640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd1040b39a0 con 0x7fd104162280 2024-09-21T03:40:35.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:35.213+0000 7fd10adba640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fd1040b3f50 con 0x7fd104159840 2024-09-21T03:40:35.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:35.213+0000 7fd108b56640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd1040b2e40 0x7fd1041f5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:35.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:35.213+0000 7fd108b56640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd1040b2e40 0x7fd1041f5e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:45138/0 (socket says 172.21.15.38:45138) 2024-09-21T03:40:35.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:35.213+0000 7fd108b56640 1 -- 172.21.15.38:0/1455639823 learned_addr learned my addr 172.21.15.38:0/1455639823 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:35.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:35.213+0000 7fd103fff640 1 --2- 172.21.15.38:0/1455639823 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd104162280 0x7fd1040ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:35.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:35.213+0000 7fd109357640 1 --2- 172.21.15.38:0/1455639823 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd104159840 0x7fd10415e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:35.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:35.213+0000 7fd108b56640 1 -- 172.21.15.38:0/1455639823 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd104159840 msgr2=0x7fd10415e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:35.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:35.213+0000 7fd108b56640 1 --2- 172.21.15.38:0/1455639823 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd104159840 0x7fd10415e4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:35.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:35.213+0000 7fd108b56640 1 -- 172.21.15.38:0/1455639823 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd104162280 msgr2=0x7fd1040ce370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:35.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:35.213+0000 7fd108b56640 1 --2- 172.21.15.38:0/1455639823 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd104162280 0x7fd1040ce370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:35.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:35.213+0000 7fd108b56640 1 -- 172.21.15.38:0/1455639823 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1040cb150 con 0x7fd1040b2e40 2024-09-21T03:40:35.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:35.213+0000 7fd108b56640 1 --2- 172.21.15.38:0/1455639823 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd1040b2e40 0x7fd1041f5e10 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7fd0ec00e7c0 tx=0x7fd0ec00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:35.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:35.213+0000 7fd1017fa640 1 -- 172.21.15.38:0/1455639823 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0ec00ccc0 con 0x7fd1040b2e40 2024-09-21T03:40:35.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:35.213+0000 7fd1017fa640 1 -- 172.21.15.38:0/1455639823 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd0ec00ce80 con 0x7fd1040b2e40 2024-09-21T03:40:35.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:35.213+0000 7fd1017fa640 1 -- 172.21.15.38:0/1455639823 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0ec005280 con 0x7fd1040b2e40 2024-09-21T03:40:35.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:35.214+0000 7fd10adba640 1 -- 172.21.15.38:0/1455639823 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1040cd5b0 con 0x7fd1040b2e40 2024-09-21T03:40:35.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:35.214+0000 7fd10adba640 1 -- 172.21.15.38:0/1455639823 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1040cdb70 con 0x7fd1040b2e40 2024-09-21T03:40:35.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:35.215+0000 7fd10adba640 1 -- 172.21.15.38:0/1455639823 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd104162f00 con 0x7fd1040b2e40 2024-09-21T03:40:35.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:35.215+0000 7fd1017fa640 1 -- 172.21.15.38:0/1455639823 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fd0ec005420 con 0x7fd1040b2e40 2024-09-21T03:40:35.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:35.216+0000 7fd1017fa640 1 --2- 172.21.15.38:0/1455639823 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd0e4078b90 0x7fd0e407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:35.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:35.216+0000 7fd1017fa640 1 -- 172.21.15.38:0/1455639823 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 6562+0+0 (secure 0 0 0) 0x7fd0ec09bb60 con 0x7fd1040b2e40 2024-09-21T03:40:35.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:35.216+0000 7fd103fff640 1 --2- 172.21.15.38:0/1455639823 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd0e4078b90 0x7fd0e407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:35.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:35.217+0000 7fd103fff640 1 --2- 172.21.15.38:0/1455639823 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd0e4078b90 0x7fd0e407b0a0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fd1040b5670 tx=0x7fd0f4005c70 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:35.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:35.221+0000 7fd1017fa640 1 -- 172.21.15.38:0/1455639823 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fd0ec063bc0 con 0x7fd1040b2e40 2024-09-21T03:40:35.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:35.444+0000 7fd10adba640 1 -- 172.21.15.38:0/1455639823 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd1040b3570 con 0x7fd1040b2e40 2024-09-21T03:40:35.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:35.445+0000 7fd1017fa640 1 -- 172.21.15.38:0/1455639823 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 100 v100) ==== 95+0+4387 (secure 0 0 0) 0x7fd0ec0634c0 con 0x7fd1040b2e40 2024-09-21T03:40:35.447 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:40:35.447 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":100,"btime":"2024-09-21T03:40:31:990176+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6086,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/965972663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":965972663},{"type":"v1","addr":"172.21.15.195:6836","nonce":965972663}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6700,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/3176122334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":3176122334},{"type":"v1","addr":"172.21.15.195:6837","nonce":3176122334}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":7020,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/751618334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":751618334},{"type":"v1","addr":"172.21.15.38:6833","nonce":751618334}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":100,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:40:02.319282+0000","modified":"2024-09-21T03:40:31.990169+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":99,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-21T03:40:35.447 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 100 2024-09-21T03:40:35.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:35.447+0000 7fd10adba640 1 -- 172.21.15.38:0/1455639823 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd0e4078b90 msgr2=0x7fd0e407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:35.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:35.447+0000 7fd10adba640 1 --2- 172.21.15.38:0/1455639823 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd0e4078b90 0x7fd0e407b0a0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fd1040b5670 tx=0x7fd0f4005c70 comp rx=0 tx=0).stop 2024-09-21T03:40:35.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:35.447+0000 7fd10adba640 1 -- 172.21.15.38:0/1455639823 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd1040b2e40 msgr2=0x7fd1041f5e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:35.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:35.447+0000 7fd10adba640 1 --2- 172.21.15.38:0/1455639823 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd1040b2e40 0x7fd1041f5e10 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7fd0ec00e7c0 tx=0x7fd0ec00ec80 comp rx=0 tx=0).stop 2024-09-21T03:40:35.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:35.447+0000 7fd10adba640 1 -- 172.21.15.38:0/1455639823 shutdown_connections 2024-09-21T03:40:35.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:35.447+0000 7fd10adba640 1 --2- 172.21.15.38:0/1455639823 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd0e4078b90 0x7fd0e407b0a0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:35.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:35.447+0000 7fd10adba640 1 --2- 172.21.15.38:0/1455639823 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd104159840 0x7fd10415e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:35.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:35.447+0000 7fd10adba640 1 --2- 172.21.15.38:0/1455639823 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd104162280 0x7fd1040ce370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:35.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:35.447+0000 7fd10adba640 1 --2- 172.21.15.38:0/1455639823 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd1040b2e40 0x7fd1041f5e10 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:35.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:35.447+0000 7fd10adba640 1 -- 172.21.15.38:0/1455639823 >> 172.21.15.38:0/1455639823 conn(0x7fd1040b6b20 msgr2=0x7fd10415c880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:35.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:35.447+0000 7fd10adba640 1 -- 172.21.15.38:0/1455639823 shutdown_connections 2024-09-21T03:40:35.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:35.447+0000 7fd10adba640 1 -- 172.21.15.38:0/1455639823 wait complete. 2024-09-21T03:40:35.460 INFO:tasks.cephfs.test_damage:Result: Mutation 'Delete 600.00000000' led to DAMAGED state 2024-09-21T03:40:35.460 INFO:tasks.cephfs.test_damage:Applying mutation 'Delete 1.00000000.inode' 2024-09-21T03:40:35.460 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-21T03:40:35.460 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:40:35.476 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056795,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2354 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T03:40:35.477 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi038.front.sepia.ceph.com 2024-09-21T03:40:35.477 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:40:35.477 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T03:40:35.532 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T03:40:35.532 INFO:teuthology.orchestra.run.smithi038.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-09-21T03:40:35.533 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-21T03:40:35.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:35.689+0000 7f7abde07640 1 -- 172.21.15.38:0/1371063234 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7aa0003a50 msgr2=0x7f7aa0003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:35.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:35.689+0000 7f7abde07640 1 --2- 172.21.15.38:0/1371063234 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7aa0003a50 0x7f7aa0003ec0 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f7aa8009b50 tx=0x7f7aa0005830 comp rx=0 tx=0).stop 2024-09-21T03:40:35.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:35.689+0000 7f7abde07640 1 -- 172.21.15.38:0/1371063234 shutdown_connections 2024-09-21T03:40:35.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:35.689+0000 7f7abde07640 1 --2- 172.21.15.38:0/1371063234 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7aa0003a50 0x7f7aa0003ec0 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:35.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:35.689+0000 7f7abde07640 1 -- 172.21.15.38:0/1371063234 >> 172.21.15.38:0/1371063234 conn(0x7f7ab80b46e0 msgr2=0x7f7ab80d23e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:35.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:35.689+0000 7f7abde07640 1 -- 172.21.15.38:0/1371063234 shutdown_connections 2024-09-21T03:40:35.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:35.689+0000 7f7abde07640 1 -- 172.21.15.38:0/1371063234 wait complete. 2024-09-21T03:40:35.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:35.691+0000 7f7abde07640 1 Processor -- start 2024-09-21T03:40:35.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:35.691+0000 7f7abde07640 1 -- start start 2024-09-21T03:40:35.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:35.692+0000 7f7abde07640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7aa0003a50 0x7f7ab81f74c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:35.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:35.692+0000 7f7abde07640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7ab81f4190 0x7f7ab81f07e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:35.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:35.692+0000 7f7abde07640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7ab815e590 0x7f7ab815e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:35.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:35.692+0000 7f7abde07640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f7ab80d0670 con 0x7f7ab815e590 2024-09-21T03:40:35.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:35.692+0000 7f7abde07640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f7ab815ce10 con 0x7f7aa0003a50 2024-09-21T03:40:35.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:35.692+0000 7f7abde07640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f7ab80d07f0 con 0x7f7ab81f4190 2024-09-21T03:40:35.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:35.692+0000 7f7ab7fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7ab815e590 0x7f7ab815e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:35.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:35.692+0000 7f7ab7fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7ab815e590 0x7f7ab815e9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:45150/0 (socket says 172.21.15.38:45150) 2024-09-21T03:40:35.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:35.692+0000 7f7ab7fff640 1 -- 172.21.15.38:0/1832802233 learned_addr learned my addr 172.21.15.38:0/1832802233 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:35.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:35.692+0000 7f7ab6ffd640 1 --2- 172.21.15.38:0/1832802233 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7ab81f4190 0x7f7ab81f07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:35.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:35.692+0000 7f7ab77fe640 1 --2- 172.21.15.38:0/1832802233 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7aa0003a50 0x7f7ab81f74c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:35.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:35.693+0000 7f7ab7fff640 1 -- 172.21.15.38:0/1832802233 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7ab81f4190 msgr2=0x7f7ab81f07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:35.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:35.693+0000 7f7ab7fff640 1 --2- 172.21.15.38:0/1832802233 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7ab81f4190 0x7f7ab81f07e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:35.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:35.693+0000 7f7ab7fff640 1 -- 172.21.15.38:0/1832802233 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7aa0003a50 msgr2=0x7f7ab81f74c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:35.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:35.693+0000 7f7ab7fff640 1 --2- 172.21.15.38:0/1832802233 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7aa0003a50 0x7f7ab81f74c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:35.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:35.693+0000 7f7ab7fff640 1 -- 172.21.15.38:0/1832802233 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ab81f7a00 con 0x7f7ab815e590 2024-09-21T03:40:35.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:35.693+0000 7f7ab7fff640 1 --2- 172.21.15.38:0/1832802233 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7ab815e590 0x7f7ab815e9a0 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f7ab80d34b0 tx=0x7f7ab00019b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:35.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:35.693+0000 7f7a9ffff640 1 -- 172.21.15.38:0/1832802233 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ab0017070 con 0x7f7ab815e590 2024-09-21T03:40:35.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:35.693+0000 7f7a9ffff640 1 -- 172.21.15.38:0/1832802233 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7ab00108c0 con 0x7f7ab815e590 2024-09-21T03:40:35.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:35.693+0000 7f7a9ffff640 1 -- 172.21.15.38:0/1832802233 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ab0010cd0 con 0x7f7ab815e590 2024-09-21T03:40:35.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:35.693+0000 7f7abde07640 1 -- 172.21.15.38:0/1832802233 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ab815eee0 con 0x7f7ab815e590 2024-09-21T03:40:35.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:35.693+0000 7f7abde07640 1 -- 172.21.15.38:0/1832802233 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ab81f36b0 con 0x7f7ab815e590 2024-09-21T03:40:35.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:35.694+0000 7f7abde07640 1 -- 172.21.15.38:0/1832802233 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ab81f3470 con 0x7f7ab815e590 2024-09-21T03:40:35.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:35.695+0000 7f7a9ffff640 1 -- 172.21.15.38:0/1832802233 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f7ab001a070 con 0x7f7ab815e590 2024-09-21T03:40:35.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:35.696+0000 7f7a9ffff640 1 --2- 172.21.15.38:0/1832802233 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7aa4078bc0 0x7f7aa407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:35.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:35.696+0000 7f7a9ffff640 1 -- 172.21.15.38:0/1832802233 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 6562+0+0 (secure 0 0 0) 0x7f7ab009aeb0 con 0x7f7ab815e590 2024-09-21T03:40:35.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:35.696+0000 7f7ab77fe640 1 --2- 172.21.15.38:0/1832802233 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7aa4078bc0 0x7f7aa407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:35.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:35.697+0000 7f7ab77fe640 1 --2- 172.21.15.38:0/1832802233 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7aa4078bc0 0x7f7aa407b0d0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f7aa8035000 tx=0x7f7aa80046c0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:35.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:35.701+0000 7f7a9ffff640 1 -- 172.21.15.38:0/1832802233 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f7ab815eee0 con 0x7f7ab815e590 2024-09-21T03:40:35.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:35.930+0000 7f7abde07640 1 -- 172.21.15.38:0/1832802233 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f7ab80b8000 con 0x7f7ab815e590 2024-09-21T03:40:36.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:36.001+0000 7f7a9ffff640 1 -- 172.21.15.38:0/1832802233 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v101) ==== 161+0+0 (secure 0 0 0) 0x7f7ab81f7a00 con 0x7f7ab815e590 2024-09-21T03:40:36.004 INFO:teuthology.orchestra.run.smithi038.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-21T03:40:36.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:36.003+0000 7f7abde07640 1 -- 172.21.15.38:0/1832802233 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7aa4078bc0 msgr2=0x7f7aa407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:36.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:36.003+0000 7f7abde07640 1 --2- 172.21.15.38:0/1832802233 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7aa4078bc0 0x7f7aa407b0d0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f7aa8035000 tx=0x7f7aa80046c0 comp rx=0 tx=0).stop 2024-09-21T03:40:36.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:36.003+0000 7f7abde07640 1 -- 172.21.15.38:0/1832802233 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7ab815e590 msgr2=0x7f7ab815e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:36.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:36.003+0000 7f7abde07640 1 --2- 172.21.15.38:0/1832802233 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7ab815e590 0x7f7ab815e9a0 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f7ab80d34b0 tx=0x7f7ab00019b0 comp rx=0 tx=0).stop 2024-09-21T03:40:36.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:36.003+0000 7f7abde07640 1 -- 172.21.15.38:0/1832802233 shutdown_connections 2024-09-21T03:40:36.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:36.003+0000 7f7abde07640 1 --2- 172.21.15.38:0/1832802233 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7ab815e590 0x7f7ab815e9a0 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:36.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:36.003+0000 7f7abde07640 1 --2- 172.21.15.38:0/1832802233 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7aa4078bc0 0x7f7aa407b0d0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:36.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:36.003+0000 7f7abde07640 1 --2- 172.21.15.38:0/1832802233 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7ab81f4190 0x7f7ab81f07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:36.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:36.003+0000 7f7abde07640 1 --2- 172.21.15.38:0/1832802233 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7aa0003a50 0x7f7ab81f74c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:36.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:36.003+0000 7f7abde07640 1 -- 172.21.15.38:0/1832802233 >> 172.21.15.38:0/1832802233 conn(0x7f7ab80b46e0 msgr2=0x7f7ab80b78f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:36.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:36.003+0000 7f7abde07640 1 -- 172.21.15.38:0/1832802233 shutdown_connections 2024-09-21T03:40:36.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:36.003+0000 7f7abde07640 1 -- 172.21.15.38:0/1832802233 wait complete. 2024-09-21T03:40:36.017 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-09-21T03:40:36.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:36.133+0000 7f8fc0827640 1 -- 172.21.15.38:0/760919148 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8fbc0d0170 msgr2=0x7f8fbc0d0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:36.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:36.133+0000 7f8fc0827640 1 --2- 172.21.15.38:0/760919148 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8fbc0d0170 0x7f8fbc0d0be0 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f8fb0009820 tx=0x7f8fb002a3f0 comp rx=0 tx=0).stop 2024-09-21T03:40:36.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:36.133+0000 7f8fc0827640 1 -- 172.21.15.38:0/760919148 shutdown_connections 2024-09-21T03:40:36.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:36.133+0000 7f8fc0827640 1 --2- 172.21.15.38:0/760919148 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8fbc0d0170 0x7f8fbc0d0be0 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:36.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:36.133+0000 7f8fc0827640 1 -- 172.21.15.38:0/760919148 >> 172.21.15.38:0/760919148 conn(0x7f8fbc09d830 msgr2=0x7f8fbc0b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:36.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:36.133+0000 7f8fc0827640 1 -- 172.21.15.38:0/760919148 shutdown_connections 2024-09-21T03:40:36.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:36.133+0000 7f8fc0827640 1 -- 172.21.15.38:0/760919148 wait complete. 2024-09-21T03:40:36.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:36.135+0000 7f8fc0827640 1 Processor -- start 2024-09-21T03:40:36.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:36.135+0000 7f8fc0827640 1 -- start start 2024-09-21T03:40:36.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:36.136+0000 7f8fc0827640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8fbc0d0170 0x7f8fbc14f900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:36.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:36.136+0000 7f8fc0827640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8fbc0d1120 0x7f8fbc0cb610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:36.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:36.136+0000 7f8fc0827640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8fbc15ee80 0x7f8fbc06f300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:36.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:36.136+0000 7f8fc0827640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f8fbc0b6910 con 0x7f8fbc0d0170 2024-09-21T03:40:36.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:36.136+0000 7f8fc0827640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f8fbc0b6790 con 0x7f8fbc0d1120 2024-09-21T03:40:36.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:36.136+0000 7f8fc0827640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f8fbc0b6a90 con 0x7f8fbc15ee80 2024-09-21T03:40:36.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:36.136+0000 7f8fba59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8fbc0d0170 0x7f8fbc14f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:36.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:36.136+0000 7f8fba59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8fbc0d0170 0x7f8fbc14f900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:45166/0 (socket says 172.21.15.38:45166) 2024-09-21T03:40:36.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:36.136+0000 7f8fba59c640 1 -- 172.21.15.38:0/3077036886 learned_addr learned my addr 172.21.15.38:0/3077036886 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:36.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:36.137+0000 7f8fb9d9b640 1 --2- 172.21.15.38:0/3077036886 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8fbc0d1120 0x7f8fbc0cb610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:36.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:36.137+0000 7f8fba59c640 1 -- 172.21.15.38:0/3077036886 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8fbc15ee80 msgr2=0x7f8fbc06f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:36.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:36.137+0000 7f8fbad9d640 1 --2- 172.21.15.38:0/3077036886 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8fbc15ee80 0x7f8fbc06f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:36.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:36.137+0000 7f8fba59c640 1 --2- 172.21.15.38:0/3077036886 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8fbc15ee80 0x7f8fbc06f300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:36.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:36.137+0000 7f8fba59c640 1 -- 172.21.15.38:0/3077036886 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8fbc0d1120 msgr2=0x7f8fbc0cb610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:36.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:36.137+0000 7f8fba59c640 1 --2- 172.21.15.38:0/3077036886 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8fbc0d1120 0x7f8fbc0cb610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:36.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:36.137+0000 7f8fba59c640 1 -- 172.21.15.38:0/3077036886 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fbc06e2a0 con 0x7f8fbc0d0170 2024-09-21T03:40:36.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:36.137+0000 7f8fb9d9b640 1 --2- 172.21.15.38:0/3077036886 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8fbc0d1120 0x7f8fbc0cb610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:40:36.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:36.137+0000 7f8fbad9d640 1 --2- 172.21.15.38:0/3077036886 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8fbc15ee80 0x7f8fbc06f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:40:36.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:36.137+0000 7f8fba59c640 1 --2- 172.21.15.38:0/3077036886 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8fbc0d0170 0x7f8fbc14f900 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f8fac00ba50 tx=0x7f8fac00bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:36.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:36.137+0000 7f8faaffd640 1 -- 172.21.15.38:0/3077036886 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fac019070 con 0x7f8fbc0d0170 2024-09-21T03:40:36.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:36.137+0000 7f8faaffd640 1 -- 172.21.15.38:0/3077036886 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8fac00cd10 con 0x7f8fbc0d0170 2024-09-21T03:40:36.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:36.137+0000 7f8fc0827640 1 -- 172.21.15.38:0/3077036886 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fbc06e540 con 0x7f8fbc0d0170 2024-09-21T03:40:36.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:36.137+0000 7f8faaffd640 1 -- 172.21.15.38:0/3077036886 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fac0146a0 con 0x7f8fbc0d0170 2024-09-21T03:40:36.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:36.137+0000 7f8fc0827640 1 -- 172.21.15.38:0/3077036886 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fbc06eb00 con 0x7f8fbc0d0170 2024-09-21T03:40:36.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:36.138+0000 7f8fc0827640 1 -- 172.21.15.38:0/3077036886 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8fbc0d0be0 con 0x7f8fbc0d0170 2024-09-21T03:40:36.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:36.139+0000 7f8faaffd640 1 -- 172.21.15.38:0/3077036886 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f8fac01c070 con 0x7f8fbc0d0170 2024-09-21T03:40:36.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:36.140+0000 7f8faaffd640 1 --2- 172.21.15.38:0/3077036886 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8f9c078bc0 0x7f8f9c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:36.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:36.140+0000 7f8faaffd640 1 -- 172.21.15.38:0/3077036886 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 6562+0+0 (secure 0 0 0) 0x7f8fac09ad10 con 0x7f8fbc0d0170 2024-09-21T03:40:36.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:36.140+0000 7f8fb9d9b640 1 --2- 172.21.15.38:0/3077036886 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8f9c078bc0 0x7f8f9c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:36.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:36.141+0000 7f8fb9d9b640 1 --2- 172.21.15.38:0/3077036886 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8f9c078bc0 0x7f8f9c07b0d0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f8fb0034000 tx=0x7f8fb0004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:36.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:36.144+0000 7f8faaffd640 1 -- 172.21.15.38:0/3077036886 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f8fac0a0020 con 0x7f8fbc0d0170 2024-09-21T03:40:36.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:36.412+0000 7f8fc0827640 1 -- 172.21.15.38:0/3077036886 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7f8fbc0ceda0 con 0x7f8fbc0d0170 2024-09-21T03:40:36.741 INFO:tasks.ceph.mds.a.smithi038.stderr: -55> 2024-09-21T03:40:31.987+0000 7f4e8b8d6640 -1 log_channel(cluster) log [ERR] : dir 0x600 object missing on disk; some files may be lost (~mds0/stray0) 2024-09-21T03:40:36.742 INFO:tasks.ceph.mds.a.smithi038.stderr: -53> 2024-09-21T03:40:31.987+0000 7f4e8b8d6640 -1 mds.0.damage notify_dirfrag Damage to fragment * of ino 0x600 is fatal because it is a system directory for this rank 2024-09-21T03:40:36.742 INFO:tasks.ceph.mds.a.smithi038.stderr: -30> 2024-09-21T03:40:33.954+0000 7f4e8f8de640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T03:40:36.796 INFO:tasks.ceph.mds.a.smithi038.stdout:starting mds.a at 2024-09-21T03:40:37.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.007+0000 7f8faaffd640 1 -- 172.21.15.38:0/3077036886 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v102) ==== 101+0+0 (secure 0 0 0) 0x7f8fac062790 con 0x7f8fbc0d0170 2024-09-21T03:40:37.009 INFO:teuthology.orchestra.run.smithi038.stderr:repaired: restoring rank 6:0 2024-09-21T03:40:37.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.011+0000 7f8fc0827640 1 -- 172.21.15.38:0/3077036886 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8f9c078bc0 msgr2=0x7f8f9c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:37.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.011+0000 7f8fc0827640 1 --2- 172.21.15.38:0/3077036886 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8f9c078bc0 0x7f8f9c07b0d0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f8fb0034000 tx=0x7f8fb0004730 comp rx=0 tx=0).stop 2024-09-21T03:40:37.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.011+0000 7f8fc0827640 1 -- 172.21.15.38:0/3077036886 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8fbc0d0170 msgr2=0x7f8fbc14f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:37.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.011+0000 7f8fc0827640 1 --2- 172.21.15.38:0/3077036886 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8fbc0d0170 0x7f8fbc14f900 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f8fac00ba50 tx=0x7f8fac00bf10 comp rx=0 tx=0).stop 2024-09-21T03:40:37.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.011+0000 7f8fc0827640 1 -- 172.21.15.38:0/3077036886 shutdown_connections 2024-09-21T03:40:37.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.011+0000 7f8fc0827640 1 --2- 172.21.15.38:0/3077036886 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8f9c078bc0 0x7f8f9c07b0d0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:37.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.011+0000 7f8fc0827640 1 --2- 172.21.15.38:0/3077036886 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8fbc15ee80 0x7f8fbc06f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:37.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.011+0000 7f8fc0827640 1 --2- 172.21.15.38:0/3077036886 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8fbc0d1120 0x7f8fbc0cb610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:37.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.011+0000 7f8fc0827640 1 --2- 172.21.15.38:0/3077036886 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8fbc0d0170 0x7f8fbc14f900 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:37.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.011+0000 7f8fc0827640 1 -- 172.21.15.38:0/3077036886 >> 172.21.15.38:0/3077036886 conn(0x7f8fbc09d830 msgr2=0x7f8fbc1616a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:37.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.011+0000 7f8fc0827640 1 -- 172.21.15.38:0/3077036886 shutdown_connections 2024-09-21T03:40:37.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.011+0000 7f8fc0827640 1 -- 172.21.15.38:0/3077036886 wait complete. 2024-09-21T03:40:37.025 DEBUG:teuthology.orchestra.run.smithi038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-09-21T03:40:37.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.079+0000 7f1f76139f40 1 Processor -- start 2024-09-21T03:40:37.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.079+0000 7f1f76139f40 1 -- start start 2024-09-21T03:40:37.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.080+0000 7f1f76139f40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b28c8bbd00 0x55b28c977df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:37.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.080+0000 7f1f76139f40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b28c976950 0x55b28c8b78c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:37.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.080+0000 7f1f76139f40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b28c991670 0x55b28c8b7e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:37.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.080+0000 7f1f76139f40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55b28c809d60 con 0x55b28c8bbd00 2024-09-21T03:40:37.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.080+0000 7f1f76139f40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55b28c868430 con 0x55b28c976950 2024-09-21T03:40:37.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.080+0000 7f1f76139f40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55b28c9037b0 con 0x55b28c991670 2024-09-21T03:40:37.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.080+0000 7f1f6fbf3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b28c8bbd00 0x55b28c977df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:37.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.080+0000 7f1f6fbf3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b28c8bbd00 0x55b28c977df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:45210/0 (socket says 172.21.15.38:45210) 2024-09-21T03:40:37.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.080+0000 7f1f6fbf3640 1 -- 172.21.15.38:0/3720042779 learned_addr learned my addr 172.21.15.38:0/3720042779 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:37.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.080+0000 7f1f6fbf3640 1 -- 172.21.15.38:0/3720042779 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b28c991670 msgr2=0x55b28c8b7e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:37.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.080+0000 7f1f6fbf3640 1 --2- 172.21.15.38:0/3720042779 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b28c991670 0x55b28c8b7e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:37.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.080+0000 7f1f6fbf3640 1 -- 172.21.15.38:0/3720042779 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b28c976950 msgr2=0x55b28c8b78c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:37.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.080+0000 7f1f6fbf3640 1 --2- 172.21.15.38:0/3720042779 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b28c976950 0x55b28c8b78c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:37.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.080+0000 7f1f6fbf3640 1 -- 172.21.15.38:0/3720042779 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b28c971cd0 con 0x55b28c8bbd00 2024-09-21T03:40:37.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.081+0000 7f1f6fbf3640 1 --2- 172.21.15.38:0/3720042779 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b28c8bbd00 0x55b28c977df0 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f1f6000b860 tx=0x7f1f6000bd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86506d25fb1c544d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:37.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.081+0000 7f1f6e3f0640 1 -- 172.21.15.38:0/3720042779 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f6000db30 con 0x55b28c8bbd00 2024-09-21T03:40:37.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.081+0000 7f1f6e3f0640 1 -- 172.21.15.38:0/3720042779 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1f6000ddc0 con 0x55b28c8bbd00 2024-09-21T03:40:37.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.081+0000 7f1f6e3f0640 1 -- 172.21.15.38:0/3720042779 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f6000f690 con 0x55b28c8bbd00 2024-09-21T03:40:37.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.081+0000 7f1f76139f40 1 -- 172.21.15.38:0/3720042779 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b28c8bbd00 msgr2=0x55b28c977df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:37.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.081+0000 7f1f76139f40 1 --2- 172.21.15.38:0/3720042779 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b28c8bbd00 0x55b28c977df0 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f1f6000b860 tx=0x7f1f6000bd20 comp rx=0 tx=0).stop 2024-09-21T03:40:37.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.081+0000 7f1f76139f40 1 -- 172.21.15.38:0/3720042779 shutdown_connections 2024-09-21T03:40:37.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.081+0000 7f1f76139f40 1 --2- 172.21.15.38:0/3720042779 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b28c991670 0x55b28c8b7e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:37.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.081+0000 7f1f76139f40 1 --2- 172.21.15.38:0/3720042779 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b28c976950 0x55b28c8b78c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:37.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.081+0000 7f1f76139f40 1 --2- 172.21.15.38:0/3720042779 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b28c8bbd00 0x55b28c977df0 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:37.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.081+0000 7f1f76139f40 1 -- 172.21.15.38:0/3720042779 >> 172.21.15.38:0/3720042779 conn(0x55b28c7daac0 msgr2=0x55b28c991a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:37.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.081+0000 7f1f76139f40 1 -- 172.21.15.38:0/3720042779 shutdown_connections 2024-09-21T03:40:37.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.081+0000 7f1f76139f40 1 -- 172.21.15.38:0/3720042779 wait complete. 2024-09-21T03:40:37.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.082+0000 7f1f76139f40 1 Processor -- start 2024-09-21T03:40:37.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.082+0000 7f1f76139f40 1 -- start start 2024-09-21T03:40:37.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.084+0000 7f1f76139f40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b28c8bbd00 0x55b28c970280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:37.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.084+0000 7f1f76139f40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b28c976950 0x55b28c988340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:37.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.084+0000 7f1f76139f40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b28c991670 0x55b28c988880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:37.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.084+0000 7f1f76139f40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55b28ca19530 con 0x55b28c8bbd00 2024-09-21T03:40:37.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.084+0000 7f1f76139f40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55b28ca17d60 con 0x55b28c976950 2024-09-21T03:40:37.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.084+0000 7f1f76139f40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55b28ca174b0 con 0x55b28c991670 2024-09-21T03:40:37.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.084+0000 7f1f6fbf3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b28c8bbd00 0x55b28c970280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:37.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.084+0000 7f1f6fbf3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b28c8bbd00 0x55b28c970280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:45224/0 (socket says 172.21.15.38:45224) 2024-09-21T03:40:37.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.084+0000 7f1f6fbf3640 1 -- 172.21.15.38:0/3399854232 learned_addr learned my addr 172.21.15.38:0/3399854232 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:37.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.084+0000 7f1f6f3f2640 1 --2- 172.21.15.38:0/3399854232 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b28c976950 0x55b28c988340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:37.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.084+0000 7f1f703f4640 1 --2- 172.21.15.38:0/3399854232 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b28c991670 0x55b28c988880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:37.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.084+0000 7f1f6fbf3640 1 -- 172.21.15.38:0/3399854232 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b28c991670 msgr2=0x55b28c988880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:37.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.084+0000 7f1f6fbf3640 1 --2- 172.21.15.38:0/3399854232 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b28c991670 0x55b28c988880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:37.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.084+0000 7f1f6fbf3640 1 -- 172.21.15.38:0/3399854232 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b28c976950 msgr2=0x55b28c988340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:37.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.084+0000 7f1f6fbf3640 1 --2- 172.21.15.38:0/3399854232 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b28c976950 0x55b28c988340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:37.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.084+0000 7f1f6fbf3640 1 -- 172.21.15.38:0/3399854232 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b28c8ea210 con 0x55b28c8bbd00 2024-09-21T03:40:37.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.084+0000 7f1f6f3f2640 1 --2- 172.21.15.38:0/3399854232 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b28c976950 0x55b28c988340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:40:37.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.084+0000 7f1f6fbf3640 1 --2- 172.21.15.38:0/3399854232 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b28c8bbd00 0x55b28c970280 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f1f6000dca0 tx=0x7f1f60004fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:37.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.084+0000 7f1f6d3ee640 1 -- 172.21.15.38:0/3399854232 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f60026070 con 0x55b28c8bbd00 2024-09-21T03:40:37.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.084+0000 7f1f6d3ee640 1 -- 172.21.15.38:0/3399854232 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1f60010890 con 0x55b28c8bbd00 2024-09-21T03:40:37.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.084+0000 7f1f76139f40 1 -- 172.21.15.38:0/3399854232 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55b28c9779d0 con 0x55b28c8bbd00 2024-09-21T03:40:37.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.084+0000 7f1f6d3ee640 1 -- 172.21.15.38:0/3399854232 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f60010cc0 con 0x55b28c8bbd00 2024-09-21T03:40:37.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.085+0000 7f1f76139f40 1 -- 172.21.15.38:0/3399854232 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b28c989f00 con 0x55b28c8bbd00 2024-09-21T03:40:37.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.086+0000 7f1f6d3ee640 1 -- 172.21.15.38:0/3399854232 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f1f60003040 con 0x55b28c8bbd00 2024-09-21T03:40:37.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.087+0000 7f1f6d3ee640 1 --2- 172.21.15.38:0/3399854232 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1f5c077570 0x7f1f5c079a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:37.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.087+0000 7f1f6d3ee640 1 -- 172.21.15.38:0/3399854232 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 6562+0+0 (secure 0 0 0) 0x7f1f6009b2b0 con 0x55b28c8bbd00 2024-09-21T03:40:37.090 INFO:teuthology.orchestra.run.smithi038.stdout:Importing pool 2024-09-21T03:40:37.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.087+0000 7f1f76139f40 1 --2- 172.21.15.38:0/3399854232 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x55b28c975fd0 0x55b28c8e93e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:37.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.087+0000 7f1f76139f40 1 -- 172.21.15.38:0/3399854232 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b28c8e9920 con 0x55b28c975fd0 2024-09-21T03:40:37.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.087+0000 7f1f6f3f2640 1 --2- 172.21.15.38:0/3399854232 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1f5c077570 0x7f1f5c079a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:37.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.087+0000 7f1f703f4640 1 --2- 172.21.15.38:0/3399854232 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x55b28c975fd0 0x55b28c8e93e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:37.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.088+0000 7f1f6f3f2640 1 --2- 172.21.15.38:0/3399854232 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1f5c077570 0x7f1f5c079a80 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f1f64055000 tx=0x7f1f64058040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:37.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.088+0000 7f1f703f4640 1 --2- 172.21.15.38:0/3399854232 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x55b28c975fd0 0x55b28c8e93e0 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:37.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.089+0000 7f1f703f4640 1 -- 172.21.15.38:0/3399854232 <== osd.7 v2:172.21.15.195:6818/1606106342 1 ==== osd_op_reply(1 601.00000000 [create] v99'73 uv48 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f1f68010390 con 0x55b28c975fd0 2024-09-21T03:40:37.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.089+0000 7f1f76139f40 1 -- 172.21.15.38:0/3399854232 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e99) -- 0x55b28ca350f0 con 0x55b28c975fd0 2024-09-21T03:40:37.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.091+0000 7f1f703f4640 1 -- 172.21.15.38:0/3399854232 <== osd.7 v2:172.21.15.195:6818/1606106342 2 ==== osd_op_reply(2 601.00000000 [delete] v99'74 uv74 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1f68010390 con 0x55b28c975fd0 2024-09-21T03:40:37.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.091+0000 7f1f76139f40 1 -- 172.21.15.38:0/3399854232 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b28ca35490 con 0x55b28c975fd0 2024-09-21T03:40:37.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.092+0000 7f1f703f4640 1 -- 172.21.15.38:0/3399854232 <== osd.7 v2:172.21.15.195:6818/1606106342 3 ==== osd_op_reply(3 601.00000000 [create] v99'75 uv75 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1f68010390 con 0x55b28c975fd0 2024-09-21T03:40:37.094 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-09-21T03:40:37.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.092+0000 7f1f76139f40 1 -- 172.21.15.38:0/3399854232 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b28ca36840 con 0x55b28c975fd0 2024-09-21T03:40:37.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.093+0000 7f1f703f4640 1 -- 172.21.15.38:0/3399854232 <== osd.7 v2:172.21.15.195:6818/1606106342 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v99'76 uv76 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1f68010390 con 0x55b28c975fd0 2024-09-21T03:40:37.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.093+0000 7f1f76139f40 1 -- 172.21.15.38:0/3399854232 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b28c8e9cf0 con 0x55b28c975fd0 2024-09-21T03:40:37.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.095+0000 7f1f703f4640 1 -- 172.21.15.38:0/3399854232 <== osd.7 v2:172.21.15.195:6818/1606106342 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v99'77 uv77 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1f68010390 con 0x55b28c975fd0 2024-09-21T03:40:37.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.095+0000 7f1f76139f40 1 -- 172.21.15.38:0/3399854232 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b28ca35830 con 0x55b28c975fd0 2024-09-21T03:40:37.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.096+0000 7f1f703f4640 1 -- 172.21.15.38:0/3399854232 <== osd.7 v2:172.21.15.195:6818/1606106342 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v99'78 uv78 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1f68010390 con 0x55b28c975fd0 2024-09-21T03:40:37.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.096+0000 7f1f76139f40 1 -- 172.21.15.38:0/3399854232 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b28ca35bd0 con 0x55b28c975fd0 2024-09-21T03:40:37.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.098+0000 7f1f703f4640 1 -- 172.21.15.38:0/3399854232 <== osd.7 v2:172.21.15.195:6818/1606106342 7 ==== osd_op_reply(7 602.00000000 [create] v99'79 uv54 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f1f68010390 con 0x55b28c975fd0 2024-09-21T03:40:37.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.098+0000 7f1f76139f40 1 -- 172.21.15.38:0/3399854232 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e99) -- 0x55b28ca35f70 con 0x55b28c975fd0 2024-09-21T03:40:37.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.099+0000 7f1f703f4640 1 -- 172.21.15.38:0/3399854232 <== osd.7 v2:172.21.15.195:6818/1606106342 8 ==== osd_op_reply(8 602.00000000 [delete] v99'80 uv80 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1f68010390 con 0x55b28c975fd0 2024-09-21T03:40:37.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.099+0000 7f1f76139f40 1 -- 172.21.15.38:0/3399854232 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b28ca35f70 con 0x55b28c975fd0 2024-09-21T03:40:37.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.101+0000 7f1f703f4640 1 -- 172.21.15.38:0/3399854232 <== osd.7 v2:172.21.15.195:6818/1606106342 9 ==== osd_op_reply(9 602.00000000 [create] v99'81 uv81 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1f68010390 con 0x55b28c975fd0 2024-09-21T03:40:37.103 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-09-21T03:40:37.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.101+0000 7f1f76139f40 1 -- 172.21.15.38:0/3399854232 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b28ca35f70 con 0x55b28c975fd0 2024-09-21T03:40:37.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.102+0000 7f1f703f4640 1 -- 172.21.15.38:0/3399854232 <== osd.7 v2:172.21.15.195:6818/1606106342 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v99'82 uv82 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1f68010390 con 0x55b28c975fd0 2024-09-21T03:40:37.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.102+0000 7f1f76139f40 1 -- 172.21.15.38:0/3399854232 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b28ca35f70 con 0x55b28c975fd0 2024-09-21T03:40:37.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.103+0000 7f1f703f4640 1 -- 172.21.15.38:0/3399854232 <== osd.7 v2:172.21.15.195:6818/1606106342 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v99'83 uv83 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1f68010390 con 0x55b28c975fd0 2024-09-21T03:40:37.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.103+0000 7f1f76139f40 1 -- 172.21.15.38:0/3399854232 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b28ca35f70 con 0x55b28c975fd0 2024-09-21T03:40:37.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.105+0000 7f1f703f4640 1 -- 172.21.15.38:0/3399854232 <== osd.7 v2:172.21.15.195:6818/1606106342 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v99'84 uv84 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1f68010390 con 0x55b28c975fd0 2024-09-21T03:40:37.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.105+0000 7f1f76139f40 1 -- 172.21.15.38:0/3399854232 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:13 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b28ca35f70 con 0x55b28c975fd0 2024-09-21T03:40:37.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.106+0000 7f1f703f4640 1 -- 172.21.15.38:0/3399854232 <== osd.7 v2:172.21.15.195:6818/1606106342 13 ==== osd_op_reply(13 10000000000.00000000 [create] v99'85 uv59 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7f1f68010390 con 0x55b28c975fd0 2024-09-21T03:40:37.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.106+0000 7f1f76139f40 1 -- 172.21.15.38:0/3399854232 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:14 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e99) -- 0x55b28ca35f70 con 0x55b28c975fd0 2024-09-21T03:40:37.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.107+0000 7f1f703f4640 1 -- 172.21.15.38:0/3399854232 <== osd.7 v2:172.21.15.195:6818/1606106342 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v99'86 uv86 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f1f68010390 con 0x55b28c975fd0 2024-09-21T03:40:37.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.108+0000 7f1f76139f40 1 -- 172.21.15.38:0/3399854232 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:15 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b28ca35f70 con 0x55b28c975fd0 2024-09-21T03:40:37.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.109+0000 7f1f703f4640 1 -- 172.21.15.38:0/3399854232 <== osd.7 v2:172.21.15.195:6818/1606106342 15 ==== osd_op_reply(15 10000000000.00000000 [create] v99'87 uv87 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f1f68010390 con 0x55b28c975fd0 2024-09-21T03:40:37.111 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-09-21T03:40:37.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.109+0000 7f1f76139f40 1 -- 172.21.15.38:0/3399854232 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:16 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b28ca35f70 con 0x55b28c975fd0 2024-09-21T03:40:37.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.111+0000 7f1f703f4640 1 -- 172.21.15.38:0/3399854232 <== osd.7 v2:172.21.15.195:6818/1606106342 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v99'88 uv88 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f1f68010390 con 0x55b28c975fd0 2024-09-21T03:40:37.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.111+0000 7f1f76139f40 1 -- 172.21.15.38:0/3399854232 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:17 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b28ca35f70 con 0x55b28c975fd0 2024-09-21T03:40:37.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.111+0000 7f1f703f4640 1 -- 172.21.15.38:0/3399854232 <== osd.7 v2:172.21.15.195:6818/1606106342 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v99'89 uv89 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f1f68010390 con 0x55b28c975fd0 2024-09-21T03:40:37.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.112+0000 7f1f76139f40 1 -- 172.21.15.38:0/3399854232 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:18 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b28ca35f70 con 0x55b28c975fd0 2024-09-21T03:40:37.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.113+0000 7f1f703f4640 1 -- 172.21.15.38:0/3399854232 <== osd.7 v2:172.21.15.195:6818/1606106342 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v99'90 uv90 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f1f68010390 con 0x55b28c975fd0 2024-09-21T03:40:37.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.113+0000 7f1f76139f40 1 -- 172.21.15.38:0/3399854232 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:19 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=585b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b28ca36be0 con 0x55b28c975fd0 2024-09-21T03:40:37.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.115+0000 7f1f703f4640 1 -- 172.21.15.38:0/3399854232 <== osd.7 v2:172.21.15.195:6818/1606106342 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v99'91 uv91 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f1f68010390 con 0x55b28c975fd0 2024-09-21T03:40:37.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.115+0000 7f1f76139f40 1 -- 172.21.15.38:0/3399854232 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:20 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b28ca36be0 con 0x55b28c975fd0 2024-09-21T03:40:37.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.116+0000 7f1f703f4640 1 -- 172.21.15.38:0/3399854232 <== osd.7 v2:172.21.15.195:6818/1606106342 20 ==== osd_op_reply(20 600.00000000 [create] v99'92 uv92 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1f68010390 con 0x55b28c975fd0 2024-09-21T03:40:37.118 INFO:teuthology.orchestra.run.smithi038.stdout:Write #-9223372036854775808:00000000:::600.00000000:head# 2024-09-21T03:40:37.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.116+0000 7f1f76139f40 1 -- 172.21.15.38:0/3399854232 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:21 13.10 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b28ca37bf0 con 0x55b28c975fd0 2024-09-21T03:40:37.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.118+0000 7f1f703f4640 1 -- 172.21.15.38:0/3399854232 <== osd.7 v2:172.21.15.195:6818/1606106342 21 ==== osd_op_reply(21 600.00000000 [setxattr (30)] v99'93 uv93 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1f68010390 con 0x55b28c975fd0 2024-09-21T03:40:37.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.118+0000 7f1f76139f40 1 -- 172.21.15.38:0/3399854232 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:22 13.10 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b28ca37bf0 con 0x55b28c975fd0 2024-09-21T03:40:37.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.119+0000 7f1f703f4640 1 -- 172.21.15.38:0/3399854232 <== osd.7 v2:172.21.15.195:6818/1606106342 22 ==== osd_op_reply(22 600.00000000 [setxattr (62)] v99'94 uv94 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1f68010390 con 0x55b28c975fd0 2024-09-21T03:40:37.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.119+0000 7f1f76139f40 1 -- 172.21.15.38:0/3399854232 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:23 13.10 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b28ca36be0 con 0x55b28c975fd0 2024-09-21T03:40:37.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.120+0000 7f1f703f4640 1 -- 172.21.15.38:0/3399854232 <== osd.7 v2:172.21.15.195:6818/1606106342 23 ==== osd_op_reply(23 600.00000000 [omap-set-header] v99'95 uv95 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1f68010390 con 0x55b28c975fd0 2024-09-21T03:40:37.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.120+0000 7f1f76139f40 1 -- 172.21.15.38:0/3399854232 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:24 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b28ca36be0 con 0x55b28c975fd0 2024-09-21T03:40:37.123 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.121+0000 7f1f703f4640 1 -- 172.21.15.38:0/3399854232 <== osd.7 v2:172.21.15.195:6818/1606106342 24 ==== osd_op_reply(24 603.00000000 [create] v99'96 uv71 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f1f68010390 con 0x55b28c975fd0 2024-09-21T03:40:37.123 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.122+0000 7f1f76139f40 1 -- 172.21.15.38:0/3399854232 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:25 13.10 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e99) -- 0x55b28ca36be0 con 0x55b28c975fd0 2024-09-21T03:40:37.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.123+0000 7f1f703f4640 1 -- 172.21.15.38:0/3399854232 <== osd.7 v2:172.21.15.195:6818/1606106342 25 ==== osd_op_reply(25 603.00000000 [delete] v99'97 uv97 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1f68010390 con 0x55b28c975fd0 2024-09-21T03:40:37.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.123+0000 7f1f76139f40 1 -- 172.21.15.38:0/3399854232 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:26 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b28ca36be0 con 0x55b28c975fd0 2024-09-21T03:40:37.126 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.124+0000 7f1f703f4640 1 -- 172.21.15.38:0/3399854232 <== osd.7 v2:172.21.15.195:6818/1606106342 26 ==== osd_op_reply(26 603.00000000 [create] v99'98 uv98 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1f68010390 con 0x55b28c975fd0 2024-09-21T03:40:37.126 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-09-21T03:40:37.126 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.124+0000 7f1f76139f40 1 -- 172.21.15.38:0/3399854232 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:27 13.10 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b28ca37bf0 con 0x55b28c975fd0 2024-09-21T03:40:37.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.126+0000 7f1f703f4640 1 -- 172.21.15.38:0/3399854232 <== osd.7 v2:172.21.15.195:6818/1606106342 27 ==== osd_op_reply(27 603.00000000 [setxattr (30)] v99'99 uv99 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1f68010390 con 0x55b28c975fd0 2024-09-21T03:40:37.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.126+0000 7f1f76139f40 1 -- 172.21.15.38:0/3399854232 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:28 13.10 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b28ca37bf0 con 0x55b28c975fd0 2024-09-21T03:40:37.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.127+0000 7f1f703f4640 1 -- 172.21.15.38:0/3399854232 <== osd.7 v2:172.21.15.195:6818/1606106342 28 ==== osd_op_reply(28 603.00000000 [setxattr (62)] v99'100 uv100 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1f68010390 con 0x55b28c975fd0 2024-09-21T03:40:37.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.127+0000 7f1f76139f40 1 -- 172.21.15.38:0/3399854232 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:29 13.10 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b28ca36be0 con 0x55b28c975fd0 2024-09-21T03:40:37.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.128+0000 7f1f703f4640 1 -- 172.21.15.38:0/3399854232 <== osd.7 v2:172.21.15.195:6818/1606106342 29 ==== osd_op_reply(29 603.00000000 [omap-set-header] v99'101 uv101 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1f68010390 con 0x55b28c975fd0 2024-09-21T03:40:37.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.129+0000 7f1f76139f40 1 --2- 172.21.15.38:0/3399854232 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x55b28ca370f0 0x55b28ca39510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:37.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.129+0000 7f1f76139f40 1 -- 172.21.15.38:0/3399854232 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:30 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b28ca39a50 con 0x55b28ca370f0 2024-09-21T03:40:37.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.129+0000 7f1f6fbf3640 1 --2- 172.21.15.38:0/3399854232 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x55b28ca370f0 0x55b28ca39510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:37.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.129+0000 7f1f6fbf3640 1 --2- 172.21.15.38:0/3399854232 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x55b28ca370f0 0x55b28ca39510 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:37.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.130+0000 7f1f6fbf3640 1 -- 172.21.15.38:0/3399854232 <== osd.6 v2:172.21.15.195:6826/4055825675 1 ==== osd_op_reply(30 1.00000000.inode [create] v99'66 uv48 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f1f6000b040 con 0x55b28ca370f0 2024-09-21T03:40:37.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.130+0000 7f1f76139f40 1 -- 172.21.15.38:0/3399854232 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:31 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e99) -- 0x55b28ca39df0 con 0x55b28ca370f0 2024-09-21T03:40:37.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.132+0000 7f1f6fbf3640 1 -- 172.21.15.38:0/3399854232 <== osd.6 v2:172.21.15.195:6826/4055825675 2 ==== osd_op_reply(31 1.00000000.inode [delete] v99'67 uv67 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f1f6000b040 con 0x55b28ca370f0 2024-09-21T03:40:37.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.132+0000 7f1f76139f40 1 -- 172.21.15.38:0/3399854232 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b28ca3a190 con 0x55b28ca370f0 2024-09-21T03:40:37.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.133+0000 7f1f6fbf3640 1 -- 172.21.15.38:0/3399854232 <== osd.6 v2:172.21.15.195:6826/4055825675 3 ==== osd_op_reply(32 1.00000000.inode [create] v99'68 uv68 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f1f6000b040 con 0x55b28ca370f0 2024-09-21T03:40:37.135 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-09-21T03:40:37.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.133+0000 7f1f76139f40 1 -- 172.21.15.38:0/3399854232 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b28ca3a9e0 con 0x55b28ca370f0 2024-09-21T03:40:37.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.134+0000 7f1f6fbf3640 1 -- 172.21.15.38:0/3399854232 <== osd.6 v2:172.21.15.195:6826/4055825675 4 ==== osd_op_reply(33 1.00000000.inode [write 0~582] v99'69 uv69 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f1f6000b040 con 0x55b28ca370f0 2024-09-21T03:40:37.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.134+0000 7f1f76139f40 1 -- 172.21.15.38:0/3399854232 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b28ca3ad80 con 0x55b28ca370f0 2024-09-21T03:40:37.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.135+0000 7f1f6fbf3640 1 -- 172.21.15.38:0/3399854232 <== osd.6 v2:172.21.15.195:6826/4055825675 5 ==== osd_op_reply(34 1.00000000.inode [omap-set-header] v99'70 uv70 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f1f6000b040 con 0x55b28ca370f0 2024-09-21T03:40:37.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.135+0000 7f1f76139f40 1 -- 172.21.15.38:0/3399854232 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:35 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b28ca3b120 con 0x55b28ca370f0 2024-09-21T03:40:37.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.137+0000 7f1f6fbf3640 1 -- 172.21.15.38:0/3399854232 <== osd.6 v2:172.21.15.195:6826/4055825675 6 ==== osd_op_reply(35 200.00000000 [create] v99'71 uv53 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f1f6000b040 con 0x55b28ca370f0 2024-09-21T03:40:37.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.137+0000 7f1f76139f40 1 -- 172.21.15.38:0/3399854232 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:36 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e99) -- 0x55b28ca3b4c0 con 0x55b28ca370f0 2024-09-21T03:40:37.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.138+0000 7f1f6fbf3640 1 -- 172.21.15.38:0/3399854232 <== osd.6 v2:172.21.15.195:6826/4055825675 7 ==== osd_op_reply(36 200.00000000 [delete] v99'72 uv72 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1f6000b040 con 0x55b28ca370f0 2024-09-21T03:40:37.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.138+0000 7f1f76139f40 1 -- 172.21.15.38:0/3399854232 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b28ca3b4c0 con 0x55b28ca370f0 2024-09-21T03:40:37.141 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-09-21T03:40:37.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.139+0000 7f1f6fbf3640 1 -- 172.21.15.38:0/3399854232 <== osd.6 v2:172.21.15.195:6826/4055825675 8 ==== osd_op_reply(37 200.00000000 [create] v99'73 uv73 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1f6000b040 con 0x55b28ca370f0 2024-09-21T03:40:37.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.139+0000 7f1f76139f40 1 -- 172.21.15.38:0/3399854232 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b28ca3b4c0 con 0x55b28ca370f0 2024-09-21T03:40:37.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.140+0000 7f1f6fbf3640 1 -- 172.21.15.38:0/3399854232 <== osd.6 v2:172.21.15.195:6826/4055825675 9 ==== osd_op_reply(38 200.00000000 [write 0~90] v99'74 uv74 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1f6000b040 con 0x55b28ca370f0 2024-09-21T03:40:37.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.140+0000 7f1f76139f40 1 -- 172.21.15.38:0/3399854232 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b28ca3b4c0 con 0x55b28ca370f0 2024-09-21T03:40:37.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.142+0000 7f1f6fbf3640 1 -- 172.21.15.38:0/3399854232 <== osd.6 v2:172.21.15.195:6826/4055825675 10 ==== osd_op_reply(39 200.00000000 [omap-set-header] v99'75 uv75 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1f6000b040 con 0x55b28ca370f0 2024-09-21T03:40:37.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.142+0000 7f1f76139f40 1 -- 172.21.15.38:0/3399854232 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:40 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b28ca3b4c0 con 0x55b28ca370f0 2024-09-21T03:40:37.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.143+0000 7f1f6fbf3640 1 -- 172.21.15.38:0/3399854232 <== osd.6 v2:172.21.15.195:6826/4055825675 11 ==== osd_op_reply(40 200.00000001 [create] v99'76 uv58 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f1f6000b040 con 0x55b28ca370f0 2024-09-21T03:40:37.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.143+0000 7f1f76139f40 1 -- 172.21.15.38:0/3399854232 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:41 13.14 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e99) -- 0x55b28ca3b4c0 con 0x55b28ca370f0 2024-09-21T03:40:37.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.144+0000 7f1f6fbf3640 1 -- 172.21.15.38:0/3399854232 <== osd.6 v2:172.21.15.195:6826/4055825675 12 ==== osd_op_reply(41 200.00000001 [delete] v99'77 uv77 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1f6000b040 con 0x55b28ca370f0 2024-09-21T03:40:37.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.144+0000 7f1f76139f40 1 -- 172.21.15.38:0/3399854232 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:42 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b28ca3b4c0 con 0x55b28ca370f0 2024-09-21T03:40:37.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.145+0000 7f1f6fbf3640 1 -- 172.21.15.38:0/3399854232 <== osd.6 v2:172.21.15.195:6826/4055825675 13 ==== osd_op_reply(42 200.00000001 [create] v99'78 uv78 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1f6000b040 con 0x55b28ca370f0 2024-09-21T03:40:37.147 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-09-21T03:40:37.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.145+0000 7f1f76139f40 1 -- 172.21.15.38:0/3399854232 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:43 13.14 13:2e2fa760:::200.00000001:head [write 0~17490 in=17490b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b28ca3f940 con 0x55b28ca370f0 2024-09-21T03:40:37.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.146+0000 7f1f6fbf3640 1 -- 172.21.15.38:0/3399854232 <== osd.6 v2:172.21.15.195:6826/4055825675 14 ==== osd_op_reply(43 200.00000001 [write 0~17490] v99'79 uv79 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1f6000b040 con 0x55b28ca370f0 2024-09-21T03:40:37.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.147+0000 7f1f76139f40 1 -- 172.21.15.38:0/3399854232 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:44 13.14 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b28ca3f940 con 0x55b28ca370f0 2024-09-21T03:40:37.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.148+0000 7f1f6fbf3640 1 -- 172.21.15.38:0/3399854232 <== osd.6 v2:172.21.15.195:6826/4055825675 15 ==== osd_op_reply(44 200.00000001 [omap-set-header] v99'80 uv80 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1f6000b040 con 0x55b28ca370f0 2024-09-21T03:40:37.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.148+0000 7f1f76139f40 1 -- 172.21.15.38:0/3399854232 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:45 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b28ca3b4c0 con 0x55b28ca370f0 2024-09-21T03:40:37.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.149+0000 7f1f6fbf3640 1 -- 172.21.15.38:0/3399854232 <== osd.6 v2:172.21.15.195:6826/4055825675 16 ==== osd_op_reply(45 606.00000000 [create] v99'81 uv64 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f1f6000b040 con 0x55b28ca370f0 2024-09-21T03:40:37.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.149+0000 7f1f76139f40 1 -- 172.21.15.38:0/3399854232 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:46 13.14 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e99) -- 0x55b28ca3b4c0 con 0x55b28ca370f0 2024-09-21T03:40:37.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.150+0000 7f1f6fbf3640 1 -- 172.21.15.38:0/3399854232 <== osd.6 v2:172.21.15.195:6826/4055825675 17 ==== osd_op_reply(46 606.00000000 [delete] v99'82 uv82 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1f6000b040 con 0x55b28ca370f0 2024-09-21T03:40:37.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.150+0000 7f1f76139f40 1 -- 172.21.15.38:0/3399854232 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:47 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b28ca3b4c0 con 0x55b28ca370f0 2024-09-21T03:40:37.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.151+0000 7f1f6fbf3640 1 -- 172.21.15.38:0/3399854232 <== osd.6 v2:172.21.15.195:6826/4055825675 18 ==== osd_op_reply(47 606.00000000 [create] v99'83 uv83 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1f6000b040 con 0x55b28ca370f0 2024-09-21T03:40:37.153 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-09-21T03:40:37.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.151+0000 7f1f76139f40 1 -- 172.21.15.38:0/3399854232 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:48 13.14 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b28ca3c4d0 con 0x55b28ca370f0 2024-09-21T03:40:37.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.153+0000 7f1f6fbf3640 1 -- 172.21.15.38:0/3399854232 <== osd.6 v2:172.21.15.195:6826/4055825675 19 ==== osd_op_reply(48 606.00000000 [setxattr (30)] v99'84 uv84 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1f6000b040 con 0x55b28ca370f0 2024-09-21T03:40:37.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.153+0000 7f1f76139f40 1 -- 172.21.15.38:0/3399854232 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:49 13.14 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b28ca3c4d0 con 0x55b28ca370f0 2024-09-21T03:40:37.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.154+0000 7f1f6fbf3640 1 -- 172.21.15.38:0/3399854232 <== osd.6 v2:172.21.15.195:6826/4055825675 20 ==== osd_op_reply(49 606.00000000 [setxattr (62)] v99'85 uv85 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1f6000b040 con 0x55b28ca370f0 2024-09-21T03:40:37.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.154+0000 7f1f76139f40 1 -- 172.21.15.38:0/3399854232 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:50 13.14 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b28ca3b4c0 con 0x55b28ca370f0 2024-09-21T03:40:37.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.155+0000 7f1f6fbf3640 1 -- 172.21.15.38:0/3399854232 <== osd.6 v2:172.21.15.195:6826/4055825675 21 ==== osd_op_reply(50 606.00000000 [omap-set-header] v99'86 uv86 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1f6000b040 con 0x55b28ca370f0 2024-09-21T03:40:37.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.155+0000 7f1f76139f40 1 --2- 172.21.15.38:0/3399854232 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x55b28ca3b9d0 0x55b28ca3ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:37.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.155+0000 7f1f76139f40 1 -- 172.21.15.38:0/3399854232 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:51 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b28ca3e330 con 0x55b28ca3b9d0 2024-09-21T03:40:37.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.155+0000 7f1f6f3f2640 1 --2- 172.21.15.38:0/3399854232 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x55b28ca3b9d0 0x55b28ca3ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:37.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.156+0000 7f1f6f3f2640 1 --2- 172.21.15.38:0/3399854232 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x55b28ca3b9d0 0x55b28ca3ddf0 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:37.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.157+0000 7f1f6f3f2640 1 -- 172.21.15.38:0/3399854232 <== osd.0 v2:172.21.15.38:6800/2381639433 1 ==== osd_op_reply(51 607.00000000 [create] v99'15 uv14 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f1f64063480 con 0x55b28ca3b9d0 2024-09-21T03:40:37.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.157+0000 7f1f76139f40 1 -- 172.21.15.38:0/3399854232 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:52 13.1c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e99) -- 0x55b28ca3e6d0 con 0x55b28ca3b9d0 2024-09-21T03:40:37.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.158+0000 7f1f6f3f2640 1 -- 172.21.15.38:0/3399854232 <== osd.0 v2:172.21.15.38:6800/2381639433 2 ==== osd_op_reply(52 607.00000000 [delete] v99'16 uv16 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1f64063480 con 0x55b28ca3b9d0 2024-09-21T03:40:37.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.158+0000 7f1f76139f40 1 -- 172.21.15.38:0/3399854232 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:53 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b28ca3ea70 con 0x55b28ca3b9d0 2024-09-21T03:40:37.161 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-09-21T03:40:37.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.159+0000 7f1f6f3f2640 1 -- 172.21.15.38:0/3399854232 <== osd.0 v2:172.21.15.38:6800/2381639433 3 ==== osd_op_reply(53 607.00000000 [create] v99'17 uv17 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1f64063480 con 0x55b28ca3b9d0 2024-09-21T03:40:37.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.159+0000 7f1f76139f40 1 -- 172.21.15.38:0/3399854232 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:54 13.1c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b28ca402d0 con 0x55b28ca3b9d0 2024-09-21T03:40:37.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.160+0000 7f1f6f3f2640 1 -- 172.21.15.38:0/3399854232 <== osd.0 v2:172.21.15.38:6800/2381639433 4 ==== osd_op_reply(54 607.00000000 [setxattr (30)] v99'18 uv18 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1f64063480 con 0x55b28ca3b9d0 2024-09-21T03:40:37.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.160+0000 7f1f76139f40 1 -- 172.21.15.38:0/3399854232 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:55 13.1c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b28ca40670 con 0x55b28ca3b9d0 2024-09-21T03:40:37.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.161+0000 7f1f6f3f2640 1 -- 172.21.15.38:0/3399854232 <== osd.0 v2:172.21.15.38:6800/2381639433 5 ==== osd_op_reply(55 607.00000000 [setxattr (62)] v99'19 uv19 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1f64063480 con 0x55b28ca3b9d0 2024-09-21T03:40:37.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.161+0000 7f1f76139f40 1 -- 172.21.15.38:0/3399854232 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:56 13.1c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b28ca3f2c0 con 0x55b28ca3b9d0 2024-09-21T03:40:37.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.162+0000 7f1f6f3f2640 1 -- 172.21.15.38:0/3399854232 <== osd.0 v2:172.21.15.38:6800/2381639433 6 ==== osd_op_reply(56 607.00000000 [omap-set-header] v99'20 uv20 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1f64063480 con 0x55b28ca3b9d0 2024-09-21T03:40:37.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.162+0000 7f1f76139f40 1 --2- 172.21.15.38:0/3399854232 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x55b28ca3fbc0 0x55b28ca42a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:37.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.162+0000 7f1f76139f40 1 -- 172.21.15.38:0/3399854232 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:57 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b28ca42f60 con 0x55b28ca3fbc0 2024-09-21T03:40:37.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.163+0000 7f1f703f4640 1 --2- 172.21.15.38:0/3399854232 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x55b28ca3fbc0 0x55b28ca42a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:37.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.163+0000 7f1f703f4640 1 --2- 172.21.15.38:0/3399854232 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x55b28ca3fbc0 0x55b28ca42a20 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:37.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.164+0000 7f1f703f4640 1 -- 172.21.15.38:0/3399854232 <== osd.3 v2:172.21.15.38:6808/1286810294 1 ==== osd_op_reply(57 mds0_openfiles.0 [create] v99'12 uv11 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f1f68010390 con 0x55b28ca3fbc0 2024-09-21T03:40:37.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.164+0000 7f1f76139f40 1 -- 172.21.15.38:0/3399854232 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:58 13.6 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e99) -- 0x55b28ca42f60 con 0x55b28ca3fbc0 2024-09-21T03:40:37.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.165+0000 7f1f703f4640 1 -- 172.21.15.38:0/3399854232 <== osd.3 v2:172.21.15.38:6808/1286810294 2 ==== osd_op_reply(58 mds0_openfiles.0 [delete] v99'13 uv13 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f1f68010390 con 0x55b28ca3fbc0 2024-09-21T03:40:37.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.165+0000 7f1f76139f40 1 -- 172.21.15.38:0/3399854232 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:59 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b28ca43350 con 0x55b28ca3fbc0 2024-09-21T03:40:37.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.166+0000 7f1f703f4640 1 -- 172.21.15.38:0/3399854232 <== osd.3 v2:172.21.15.38:6808/1286810294 3 ==== osd_op_reply(59 mds0_openfiles.0 [create] v99'14 uv14 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f1f68010390 con 0x55b28ca3fbc0 2024-09-21T03:40:37.168 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-09-21T03:40:37.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.167+0000 7f1f76139f40 1 -- 172.21.15.38:0/3399854232 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:60 13.6 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b28ca43350 con 0x55b28ca3fbc0 2024-09-21T03:40:37.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.168+0000 7f1f703f4640 1 -- 172.21.15.38:0/3399854232 <== osd.3 v2:172.21.15.38:6808/1286810294 4 ==== osd_op_reply(60 mds0_openfiles.0 [omap-set-header] v99'15 uv15 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f1f68010390 con 0x55b28ca3fbc0 2024-09-21T03:40:37.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.168+0000 7f1f76139f40 1 -- 172.21.15.38:0/3399854232 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:61 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b28ca43350 con 0x55b28ca3b9d0 2024-09-21T03:40:37.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.169+0000 7f1f6f3f2640 1 -- 172.21.15.38:0/3399854232 <== osd.0 v2:172.21.15.38:6800/2381639433 7 ==== osd_op_reply(61 608.00000000 [create] v99'15 uv14 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f1f64063480 con 0x55b28ca3b9d0 2024-09-21T03:40:37.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.169+0000 7f1f76139f40 1 -- 172.21.15.38:0/3399854232 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:62 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e99) -- 0x55b28ca43350 con 0x55b28ca3b9d0 2024-09-21T03:40:37.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.170+0000 7f1f6f3f2640 1 -- 172.21.15.38:0/3399854232 <== osd.0 v2:172.21.15.38:6800/2381639433 8 ==== osd_op_reply(62 608.00000000 [delete] v99'16 uv16 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1f64063480 con 0x55b28ca3b9d0 2024-09-21T03:40:37.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.170+0000 7f1f76139f40 1 -- 172.21.15.38:0/3399854232 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b28ca43350 con 0x55b28ca3b9d0 2024-09-21T03:40:37.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.171+0000 7f1f6f3f2640 1 -- 172.21.15.38:0/3399854232 <== osd.0 v2:172.21.15.38:6800/2381639433 9 ==== osd_op_reply(63 608.00000000 [create] v99'17 uv17 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1f64063480 con 0x55b28ca3b9d0 2024-09-21T03:40:37.173 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-09-21T03:40:37.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.171+0000 7f1f76139f40 1 -- 172.21.15.38:0/3399854232 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b28ca44360 con 0x55b28ca3b9d0 2024-09-21T03:40:37.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.172+0000 7f1f6f3f2640 1 -- 172.21.15.38:0/3399854232 <== osd.0 v2:172.21.15.38:6800/2381639433 10 ==== osd_op_reply(64 608.00000000 [setxattr (30)] v99'18 uv18 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1f64063480 con 0x55b28ca3b9d0 2024-09-21T03:40:37.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.172+0000 7f1f76139f40 1 -- 172.21.15.38:0/3399854232 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b28ca44360 con 0x55b28ca3b9d0 2024-09-21T03:40:37.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.174+0000 7f1f6f3f2640 1 -- 172.21.15.38:0/3399854232 <== osd.0 v2:172.21.15.38:6800/2381639433 11 ==== osd_op_reply(65 608.00000000 [setxattr (62)] v99'19 uv19 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1f64063480 con 0x55b28ca3b9d0 2024-09-21T03:40:37.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.174+0000 7f1f76139f40 1 -- 172.21.15.38:0/3399854232 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b28ca43350 con 0x55b28ca3b9d0 2024-09-21T03:40:37.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.175+0000 7f1f6f3f2640 1 -- 172.21.15.38:0/3399854232 <== osd.0 v2:172.21.15.38:6800/2381639433 12 ==== osd_op_reply(66 608.00000000 [omap-set-header] v99'20 uv20 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1f64063480 con 0x55b28ca3b9d0 2024-09-21T03:40:37.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.175+0000 7f1f76139f40 1 -- 172.21.15.38:0/3399854232 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:67 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b28ca43350 con 0x55b28ca3fbc0 2024-09-21T03:40:37.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.176+0000 7f1f703f4640 1 -- 172.21.15.38:0/3399854232 <== osd.3 v2:172.21.15.38:6808/1286810294 5 ==== osd_op_reply(67 604.00000000 [create] v99'26 uv20 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f1f68010390 con 0x55b28ca3fbc0 2024-09-21T03:40:37.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.176+0000 7f1f76139f40 1 -- 172.21.15.38:0/3399854232 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:68 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e99) -- 0x55b28ca43350 con 0x55b28ca3fbc0 2024-09-21T03:40:37.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.177+0000 7f1f703f4640 1 -- 172.21.15.38:0/3399854232 <== osd.3 v2:172.21.15.38:6808/1286810294 6 ==== osd_op_reply(68 604.00000000 [delete] v99'27 uv27 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1f68010390 con 0x55b28ca3fbc0 2024-09-21T03:40:37.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.177+0000 7f1f76139f40 1 -- 172.21.15.38:0/3399854232 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b28ca43350 con 0x55b28ca3fbc0 2024-09-21T03:40:37.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.178+0000 7f1f703f4640 1 -- 172.21.15.38:0/3399854232 <== osd.3 v2:172.21.15.38:6808/1286810294 7 ==== osd_op_reply(69 604.00000000 [create] v99'28 uv28 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1f68010390 con 0x55b28ca3fbc0 2024-09-21T03:40:37.180 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-09-21T03:40:37.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.178+0000 7f1f76139f40 1 -- 172.21.15.38:0/3399854232 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b28ca44360 con 0x55b28ca3fbc0 2024-09-21T03:40:37.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.179+0000 7f1f703f4640 1 -- 172.21.15.38:0/3399854232 <== osd.3 v2:172.21.15.38:6808/1286810294 8 ==== osd_op_reply(70 604.00000000 [setxattr (30)] v99'29 uv29 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1f68010390 con 0x55b28ca3fbc0 2024-09-21T03:40:37.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.179+0000 7f1f76139f40 1 -- 172.21.15.38:0/3399854232 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b28ca44360 con 0x55b28ca3fbc0 2024-09-21T03:40:37.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.180+0000 7f1f703f4640 1 -- 172.21.15.38:0/3399854232 <== osd.3 v2:172.21.15.38:6808/1286810294 9 ==== osd_op_reply(71 604.00000000 [setxattr (62)] v99'30 uv30 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1f68010390 con 0x55b28ca3fbc0 2024-09-21T03:40:37.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.181+0000 7f1f76139f40 1 -- 172.21.15.38:0/3399854232 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b28ca43350 con 0x55b28ca3fbc0 2024-09-21T03:40:37.183 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.181+0000 7f1f703f4640 1 -- 172.21.15.38:0/3399854232 <== osd.3 v2:172.21.15.38:6808/1286810294 10 ==== osd_op_reply(72 604.00000000 [omap-set-header] v99'31 uv31 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1f68010390 con 0x55b28ca3fbc0 2024-09-21T03:40:37.183 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.182+0000 7f1f76139f40 1 -- 172.21.15.38:0/3399854232 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:73 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b28ca43350 con 0x55b28ca3fbc0 2024-09-21T03:40:37.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.182+0000 7f1f703f4640 1 -- 172.21.15.38:0/3399854232 <== osd.3 v2:172.21.15.38:6808/1286810294 11 ==== osd_op_reply(73 500.00000000 [create] v99'32 uv25 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f1f68010390 con 0x55b28ca3fbc0 2024-09-21T03:40:37.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.182+0000 7f1f76139f40 1 -- 172.21.15.38:0/3399854232 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:74 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e99) -- 0x55b28ca43350 con 0x55b28ca3fbc0 2024-09-21T03:40:37.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.183+0000 7f1f703f4640 1 -- 172.21.15.38:0/3399854232 <== osd.3 v2:172.21.15.38:6808/1286810294 12 ==== osd_op_reply(74 500.00000000 [delete] v99'33 uv33 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1f68010390 con 0x55b28ca3fbc0 2024-09-21T03:40:37.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.183+0000 7f1f76139f40 1 -- 172.21.15.38:0/3399854232 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b28ca43350 con 0x55b28ca3fbc0 2024-09-21T03:40:37.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.184+0000 7f1f703f4640 1 -- 172.21.15.38:0/3399854232 <== osd.3 v2:172.21.15.38:6808/1286810294 13 ==== osd_op_reply(75 500.00000000 [create] v99'34 uv34 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1f68010390 con 0x55b28ca3fbc0 2024-09-21T03:40:37.186 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-09-21T03:40:37.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.184+0000 7f1f76139f40 1 -- 172.21.15.38:0/3399854232 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b28ca43350 con 0x55b28ca3fbc0 2024-09-21T03:40:37.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.185+0000 7f1f703f4640 1 -- 172.21.15.38:0/3399854232 <== osd.3 v2:172.21.15.38:6808/1286810294 14 ==== osd_op_reply(76 500.00000000 [write 0~90] v99'35 uv35 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1f68010390 con 0x55b28ca3fbc0 2024-09-21T03:40:37.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.185+0000 7f1f76139f40 1 -- 172.21.15.38:0/3399854232 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b28ca43350 con 0x55b28ca3fbc0 2024-09-21T03:40:37.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.186+0000 7f1f703f4640 1 -- 172.21.15.38:0/3399854232 <== osd.3 v2:172.21.15.38:6808/1286810294 15 ==== osd_op_reply(77 500.00000000 [omap-set-header] v99'36 uv36 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1f68010390 con 0x55b28ca3fbc0 2024-09-21T03:40:37.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.186+0000 7f1f76139f40 1 -- 172.21.15.38:0/3399854232 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:78 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b28ca43350 con 0x55b28ca3b9d0 2024-09-21T03:40:37.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.187+0000 7f1f6f3f2640 1 -- 172.21.15.38:0/3399854232 <== osd.0 v2:172.21.15.38:6800/2381639433 13 ==== osd_op_reply(78 mds_snaptable [create] v99'12 uv11 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f1f64063480 con 0x55b28ca3b9d0 2024-09-21T03:40:37.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.187+0000 7f1f76139f40 1 -- 172.21.15.38:0/3399854232 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:79 13.d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e99) -- 0x55b28ca43350 con 0x55b28ca3b9d0 2024-09-21T03:40:37.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.189+0000 7f1f6f3f2640 1 -- 172.21.15.38:0/3399854232 <== osd.0 v2:172.21.15.38:6800/2381639433 14 ==== osd_op_reply(79 mds_snaptable [delete] v99'13 uv13 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f1f64063480 con 0x55b28ca3b9d0 2024-09-21T03:40:37.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.189+0000 7f1f76139f40 1 -- 172.21.15.38:0/3399854232 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:80 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b28ca43350 con 0x55b28ca3b9d0 2024-09-21T03:40:37.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.190+0000 7f1f6f3f2640 1 -- 172.21.15.38:0/3399854232 <== osd.0 v2:172.21.15.38:6800/2381639433 15 ==== osd_op_reply(80 mds_snaptable [create] v99'14 uv14 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f1f64063480 con 0x55b28ca3b9d0 2024-09-21T03:40:37.192 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-09-21T03:40:37.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.190+0000 7f1f76139f40 1 -- 172.21.15.38:0/3399854232 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:81 13.d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b28ca43350 con 0x55b28ca3b9d0 2024-09-21T03:40:37.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.191+0000 7f1f6f3f2640 1 -- 172.21.15.38:0/3399854232 <== osd.0 v2:172.21.15.38:6800/2381639433 16 ==== osd_op_reply(81 mds_snaptable [write 0~70] v99'15 uv15 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f1f64063480 con 0x55b28ca3b9d0 2024-09-21T03:40:37.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.191+0000 7f1f76139f40 1 -- 172.21.15.38:0/3399854232 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:82 13.d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b28ca43350 con 0x55b28ca3b9d0 2024-09-21T03:40:37.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.192+0000 7f1f6f3f2640 1 -- 172.21.15.38:0/3399854232 <== osd.0 v2:172.21.15.38:6800/2381639433 17 ==== osd_op_reply(82 mds_snaptable [omap-set-header] v99'16 uv16 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f1f64063480 con 0x55b28ca3b9d0 2024-09-21T03:40:37.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.192+0000 7f1f76139f40 1 --2- 172.21.15.38:0/3399854232 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x55b28ca43c50 0x55b28ca46070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:37.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.192+0000 7f1f76139f40 1 -- 172.21.15.38:0/3399854232 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:83 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b28ca465b0 con 0x55b28ca43c50 2024-09-21T03:40:37.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.192+0000 7f1f6fbf3640 1 --2- 172.21.15.38:0/3399854232 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x55b28ca43c50 0x55b28ca46070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:37.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.192+0000 7f1f6fbf3640 1 --2- 172.21.15.38:0/3399854232 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x55b28ca43c50 0x55b28ca46070 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:37.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.194+0000 7f1f6fbf3640 1 -- 172.21.15.38:0/3399854232 <== osd.1 v2:172.21.15.38:6816/1491818732 1 ==== osd_op_reply(83 605.00000000 [create] v99'15 uv14 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f1f6000b040 con 0x55b28ca43c50 2024-09-21T03:40:37.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.194+0000 7f1f76139f40 1 -- 172.21.15.38:0/3399854232 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:84 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e99) -- 0x55b28ca465f0 con 0x55b28ca43c50 2024-09-21T03:40:37.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.195+0000 7f1f6fbf3640 1 -- 172.21.15.38:0/3399854232 <== osd.1 v2:172.21.15.38:6816/1491818732 2 ==== osd_op_reply(84 605.00000000 [delete] v99'16 uv16 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1f6000b040 con 0x55b28ca43c50 2024-09-21T03:40:37.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.195+0000 7f1f76139f40 1 -- 172.21.15.38:0/3399854232 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b28ca469e0 con 0x55b28ca43c50 2024-09-21T03:40:37.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.196+0000 7f1f6fbf3640 1 -- 172.21.15.38:0/3399854232 <== osd.1 v2:172.21.15.38:6816/1491818732 3 ==== osd_op_reply(85 605.00000000 [create] v99'17 uv17 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1f6000b040 con 0x55b28ca43c50 2024-09-21T03:40:37.198 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-09-21T03:40:37.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.196+0000 7f1f76139f40 1 -- 172.21.15.38:0/3399854232 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b28ca479f0 con 0x55b28ca43c50 2024-09-21T03:40:37.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.197+0000 7f1f6fbf3640 1 -- 172.21.15.38:0/3399854232 <== osd.1 v2:172.21.15.38:6816/1491818732 4 ==== osd_op_reply(86 605.00000000 [setxattr (30)] v99'18 uv18 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1f6000b040 con 0x55b28ca43c50 2024-09-21T03:40:37.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.197+0000 7f1f76139f40 1 -- 172.21.15.38:0/3399854232 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b28ca479f0 con 0x55b28ca43c50 2024-09-21T03:40:37.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.198+0000 7f1f6fbf3640 1 -- 172.21.15.38:0/3399854232 <== osd.1 v2:172.21.15.38:6816/1491818732 5 ==== osd_op_reply(87 605.00000000 [setxattr (62)] v99'19 uv19 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1f6000b040 con 0x55b28ca43c50 2024-09-21T03:40:37.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.198+0000 7f1f76139f40 1 -- 172.21.15.38:0/3399854232 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b28ca469e0 con 0x55b28ca43c50 2024-09-21T03:40:37.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.199+0000 7f1f6fbf3640 1 -- 172.21.15.38:0/3399854232 <== osd.1 v2:172.21.15.38:6816/1491818732 6 ==== osd_op_reply(88 605.00000000 [omap-set-header] v99'20 uv20 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1f6000b040 con 0x55b28ca43c50 2024-09-21T03:40:37.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.199+0000 7f1f76139f40 1 -- 172.21.15.38:0/3399854232 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:89 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b28ca469e0 con 0x55b28ca43c50 2024-09-21T03:40:37.202 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.200+0000 7f1f6fbf3640 1 -- 172.21.15.38:0/3399854232 <== osd.1 v2:172.21.15.38:6816/1491818732 7 ==== osd_op_reply(89 mds0_inotable [create] v99'24 uv18 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f1f6000b040 con 0x55b28ca43c50 2024-09-21T03:40:37.202 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.200+0000 7f1f76139f40 1 -- 172.21.15.38:0/3399854232 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:90 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e99) -- 0x55b28ca469e0 con 0x55b28ca43c50 2024-09-21T03:40:37.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.201+0000 7f1f6fbf3640 1 -- 172.21.15.38:0/3399854232 <== osd.1 v2:172.21.15.38:6816/1491818732 8 ==== osd_op_reply(90 mds0_inotable [delete] v99'25 uv25 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f1f6000b040 con 0x55b28ca43c50 2024-09-21T03:40:37.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.201+0000 7f1f76139f40 1 -- 172.21.15.38:0/3399854232 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b28ca469e0 con 0x55b28ca43c50 2024-09-21T03:40:37.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.202+0000 7f1f6fbf3640 1 -- 172.21.15.38:0/3399854232 <== osd.1 v2:172.21.15.38:6816/1491818732 9 ==== osd_op_reply(91 mds0_inotable [create] v99'26 uv26 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f1f6000b040 con 0x55b28ca43c50 2024-09-21T03:40:37.204 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-09-21T03:40:37.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.202+0000 7f1f76139f40 1 -- 172.21.15.38:0/3399854232 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b28ca3eeb0 con 0x55b28ca43c50 2024-09-21T03:40:37.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.203+0000 7f1f6fbf3640 1 -- 172.21.15.38:0/3399854232 <== osd.1 v2:172.21.15.38:6816/1491818732 10 ==== osd_op_reply(92 mds0_inotable [write 0~34] v99'27 uv27 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f1f6000b040 con 0x55b28ca43c50 2024-09-21T03:40:37.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.203+0000 7f1f76139f40 1 -- 172.21.15.38:0/3399854232 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b28ca3eeb0 con 0x55b28ca43c50 2024-09-21T03:40:37.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.204+0000 7f1f6fbf3640 1 -- 172.21.15.38:0/3399854232 <== osd.1 v2:172.21.15.38:6816/1491818732 11 ==== osd_op_reply(93 mds0_inotable [omap-set-header] v99'28 uv28 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f1f6000b040 con 0x55b28ca43c50 2024-09-21T03:40:37.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.204+0000 7f1f76139f40 1 -- 172.21.15.38:0/3399854232 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:94 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b28ca3eeb0 con 0x55b28ca43c50 2024-09-21T03:40:37.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.205+0000 7f1f6fbf3640 1 -- 172.21.15.38:0/3399854232 <== osd.1 v2:172.21.15.38:6816/1491818732 12 ==== osd_op_reply(94 100.00000000 [create] v99'29 uv23 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f1f6000b040 con 0x55b28ca43c50 2024-09-21T03:40:37.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.205+0000 7f1f76139f40 1 -- 172.21.15.38:0/3399854232 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:95 13.13 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e99) -- 0x55b28ca3eeb0 con 0x55b28ca43c50 2024-09-21T03:40:37.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.206+0000 7f1f6fbf3640 1 -- 172.21.15.38:0/3399854232 <== osd.1 v2:172.21.15.38:6816/1491818732 13 ==== osd_op_reply(95 100.00000000 [delete] v99'30 uv30 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1f6000b040 con 0x55b28ca43c50 2024-09-21T03:40:37.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.206+0000 7f1f76139f40 1 -- 172.21.15.38:0/3399854232 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:96 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b28ca3eeb0 con 0x55b28ca43c50 2024-09-21T03:40:37.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.207+0000 7f1f6fbf3640 1 -- 172.21.15.38:0/3399854232 <== osd.1 v2:172.21.15.38:6816/1491818732 14 ==== osd_op_reply(96 100.00000000 [create] v99'31 uv31 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1f6000b040 con 0x55b28ca43c50 2024-09-21T03:40:37.208 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-09-21T03:40:37.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.207+0000 7f1f76139f40 1 -- 172.21.15.38:0/3399854232 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:97 13.13 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b28ca3eeb0 con 0x55b28ca43c50 2024-09-21T03:40:37.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.208+0000 7f1f6fbf3640 1 -- 172.21.15.38:0/3399854232 <== osd.1 v2:172.21.15.38:6816/1491818732 15 ==== osd_op_reply(97 100.00000000 [omap-set-header] v99'32 uv32 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1f6000b040 con 0x55b28ca43c50 2024-09-21T03:40:37.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.208+0000 7f1f76139f40 1 -- 172.21.15.38:0/3399854232 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:98 13.13 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b28ca49830 con 0x55b28ca43c50 2024-09-21T03:40:37.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.209+0000 7f1f6fbf3640 1 -- 172.21.15.38:0/3399854232 <== osd.1 v2:172.21.15.38:6816/1491818732 16 ==== osd_op_reply(98 100.00000000 [omap-set-vals] v99'33 uv33 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1f6000b040 con 0x55b28ca43c50 2024-09-21T03:40:37.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.209+0000 7f1f76139f40 1 -- 172.21.15.38:0/3399854232 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:99 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b28ca46e90 con 0x55b28ca3fbc0 2024-09-21T03:40:37.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.210+0000 7f1f703f4640 1 -- 172.21.15.38:0/3399854232 <== osd.3 v2:172.21.15.38:6808/1286810294 16 ==== osd_op_reply(99 mds0_sessionmap [create] v99'14 uv12 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7f1f68010390 con 0x55b28ca3fbc0 2024-09-21T03:40:37.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.210+0000 7f1f76139f40 1 -- 172.21.15.38:0/3399854232 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:100 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e99) -- 0x55b28ca46e90 con 0x55b28ca3fbc0 2024-09-21T03:40:37.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.211+0000 7f1f703f4640 1 -- 172.21.15.38:0/3399854232 <== osd.3 v2:172.21.15.38:6808/1286810294 17 ==== osd_op_reply(100 mds0_sessionmap [delete] v99'15 uv15 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f1f68010390 con 0x55b28ca3fbc0 2024-09-21T03:40:37.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.211+0000 7f1f76139f40 1 -- 172.21.15.38:0/3399854232 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b28ca46e90 con 0x55b28ca3fbc0 2024-09-21T03:40:37.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.211+0000 7f1f703f4640 1 -- 172.21.15.38:0/3399854232 <== osd.3 v2:172.21.15.38:6808/1286810294 18 ==== osd_op_reply(101 mds0_sessionmap [create] v99'16 uv16 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f1f68010390 con 0x55b28ca3fbc0 2024-09-21T03:40:37.213 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-09-21T03:40:37.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.211+0000 7f1f76139f40 1 -- 172.21.15.38:0/3399854232 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b28ca46e90 con 0x55b28ca3fbc0 2024-09-21T03:40:37.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.212+0000 7f1f703f4640 1 -- 172.21.15.38:0/3399854232 <== osd.3 v2:172.21.15.38:6808/1286810294 19 ==== osd_op_reply(102 mds0_sessionmap [omap-set-header] v99'17 uv17 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f1f68010390 con 0x55b28ca3fbc0 2024-09-21T03:40:37.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.212+0000 7f1f76139f40 1 -- 172.21.15.38:0/3399854232 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:103 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b28ca46e90 con 0x55b28ca3fbc0 2024-09-21T03:40:37.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.213+0000 7f1f703f4640 1 -- 172.21.15.38:0/3399854232 <== osd.3 v2:172.21.15.38:6808/1286810294 20 ==== osd_op_reply(103 609.00000000 [create] v99'15 uv14 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f1f68010390 con 0x55b28ca3fbc0 2024-09-21T03:40:37.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.213+0000 7f1f76139f40 1 -- 172.21.15.38:0/3399854232 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:104 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e99) -- 0x55b28ca46e90 con 0x55b28ca3fbc0 2024-09-21T03:40:37.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.215+0000 7f1f703f4640 1 -- 172.21.15.38:0/3399854232 <== osd.3 v2:172.21.15.38:6808/1286810294 21 ==== osd_op_reply(104 609.00000000 [delete] v99'16 uv16 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1f68010390 con 0x55b28ca3fbc0 2024-09-21T03:40:37.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.215+0000 7f1f76139f40 1 -- 172.21.15.38:0/3399854232 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b28ca46e90 con 0x55b28ca3fbc0 2024-09-21T03:40:37.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.216+0000 7f1f703f4640 1 -- 172.21.15.38:0/3399854232 <== osd.3 v2:172.21.15.38:6808/1286810294 22 ==== osd_op_reply(105 609.00000000 [create] v99'17 uv17 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1f68010390 con 0x55b28ca3fbc0 2024-09-21T03:40:37.218 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-09-21T03:40:37.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.216+0000 7f1f76139f40 1 -- 172.21.15.38:0/3399854232 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b28ca47ea0 con 0x55b28ca3fbc0 2024-09-21T03:40:37.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.217+0000 7f1f703f4640 1 -- 172.21.15.38:0/3399854232 <== osd.3 v2:172.21.15.38:6808/1286810294 23 ==== osd_op_reply(106 609.00000000 [setxattr (30)] v99'18 uv18 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1f68010390 con 0x55b28ca3fbc0 2024-09-21T03:40:37.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.217+0000 7f1f76139f40 1 -- 172.21.15.38:0/3399854232 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b28ca47ea0 con 0x55b28ca3fbc0 2024-09-21T03:40:37.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.218+0000 7f1f703f4640 1 -- 172.21.15.38:0/3399854232 <== osd.3 v2:172.21.15.38:6808/1286810294 24 ==== osd_op_reply(107 609.00000000 [setxattr (62)] v99'19 uv19 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1f68010390 con 0x55b28ca3fbc0 2024-09-21T03:40:37.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.218+0000 7f1f76139f40 1 -- 172.21.15.38:0/3399854232 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b28ca46e90 con 0x55b28ca3fbc0 2024-09-21T03:40:37.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.219+0000 7f1f703f4640 1 -- 172.21.15.38:0/3399854232 <== osd.3 v2:172.21.15.38:6808/1286810294 25 ==== osd_op_reply(108 609.00000000 [omap-set-header] v99'20 uv20 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1f68010390 con 0x55b28ca3fbc0 2024-09-21T03:40:37.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.219+0000 7f1f76139f40 1 --2- 172.21.15.38:0/3399854232 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x55b28ca47510 0x55b28ca47920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:37.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.219+0000 7f1f76139f40 1 -- 172.21.15.38:0/3399854232 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:109 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b28ca47e60 con 0x55b28ca47510 2024-09-21T03:40:37.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.219+0000 7f1f6f3f2640 1 --2- 172.21.15.38:0/3399854232 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x55b28ca47510 0x55b28ca47920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:37.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.220+0000 7f1f6f3f2640 1 --2- 172.21.15.38:0/3399854232 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x55b28ca47510 0x55b28ca47920 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:37.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.221+0000 7f1f6f3f2640 1 -- 172.21.15.38:0/3399854232 <== osd.2 v2:172.21.15.38:6824/2577858902 1 ==== osd_op_reply(109 400.00000000 [create] v99'12 uv11 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f1f64063480 con 0x55b28ca47510 2024-09-21T03:40:37.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.221+0000 7f1f76139f40 1 -- 172.21.15.38:0/3399854232 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:110 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e99) -- 0x55b28ca47e60 con 0x55b28ca47510 2024-09-21T03:40:37.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.222+0000 7f1f6f3f2640 1 -- 172.21.15.38:0/3399854232 <== osd.2 v2:172.21.15.38:6824/2577858902 2 ==== osd_op_reply(110 400.00000000 [delete] v99'13 uv13 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1f64063480 con 0x55b28ca47510 2024-09-21T03:40:37.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.222+0000 7f1f76139f40 1 -- 172.21.15.38:0/3399854232 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b28ca47e60 con 0x55b28ca47510 2024-09-21T03:40:37.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.223+0000 7f1f6f3f2640 1 -- 172.21.15.38:0/3399854232 <== osd.2 v2:172.21.15.38:6824/2577858902 3 ==== osd_op_reply(111 400.00000000 [create] v99'14 uv14 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1f64063480 con 0x55b28ca47510 2024-09-21T03:40:37.225 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-09-21T03:40:37.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.223+0000 7f1f76139f40 1 -- 172.21.15.38:0/3399854232 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b28ca482d0 con 0x55b28ca47510 2024-09-21T03:40:37.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.224+0000 7f1f6f3f2640 1 -- 172.21.15.38:0/3399854232 <== osd.2 v2:172.21.15.38:6824/2577858902 4 ==== osd_op_reply(112 400.00000000 [write 0~22] v99'15 uv15 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1f64063480 con 0x55b28ca47510 2024-09-21T03:40:37.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.224+0000 7f1f76139f40 1 -- 172.21.15.38:0/3399854232 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b28ca482d0 con 0x55b28ca47510 2024-09-21T03:40:37.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.225+0000 7f1f6f3f2640 1 -- 172.21.15.38:0/3399854232 <== osd.2 v2:172.21.15.38:6824/2577858902 5 ==== osd_op_reply(113 400.00000000 [omap-set-header] v99'16 uv16 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1f64063480 con 0x55b28ca47510 2024-09-21T03:40:37.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.225+0000 7f1f76139f40 1 -- 172.21.15.38:0/3399854232 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:114 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b28ca482d0 con 0x55b28ca43c50 2024-09-21T03:40:37.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.226+0000 7f1f6fbf3640 1 -- 172.21.15.38:0/3399854232 <== osd.1 v2:172.21.15.38:6816/1491818732 17 ==== osd_op_reply(114 100.00000000.inode [create] v99'33 uv24 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7f1f6000b040 con 0x55b28ca43c50 2024-09-21T03:40:37.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.226+0000 7f1f76139f40 1 -- 172.21.15.38:0/3399854232 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:115 13.1f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e99) -- 0x55b28ca482d0 con 0x55b28ca43c50 2024-09-21T03:40:37.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.227+0000 7f1f6fbf3640 1 -- 172.21.15.38:0/3399854232 <== osd.1 v2:172.21.15.38:6816/1491818732 18 ==== osd_op_reply(115 100.00000000.inode [delete] v99'34 uv34 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f1f6000b040 con 0x55b28ca43c50 2024-09-21T03:40:37.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.227+0000 7f1f76139f40 1 -- 172.21.15.38:0/3399854232 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:116 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b28ca482d0 con 0x55b28ca43c50 2024-09-21T03:40:37.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.229+0000 7f1f6fbf3640 1 -- 172.21.15.38:0/3399854232 <== osd.1 v2:172.21.15.38:6816/1491818732 19 ==== osd_op_reply(116 100.00000000.inode [create] v99'35 uv35 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f1f6000b040 con 0x55b28ca43c50 2024-09-21T03:40:37.230 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-09-21T03:40:37.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.229+0000 7f1f76139f40 1 -- 172.21.15.38:0/3399854232 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:117 13.1f 13:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b28ca482d0 con 0x55b28ca43c50 2024-09-21T03:40:37.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.230+0000 7f1f6fbf3640 1 -- 172.21.15.38:0/3399854232 <== osd.1 v2:172.21.15.38:6816/1491818732 20 ==== osd_op_reply(117 100.00000000.inode [write 0~582] v99'36 uv36 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f1f6000b040 con 0x55b28ca43c50 2024-09-21T03:40:37.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.230+0000 7f1f76139f40 1 -- 172.21.15.38:0/3399854232 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:118 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b28ca482d0 con 0x55b28ca43c50 2024-09-21T03:40:37.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.231+0000 7f1f6fbf3640 1 -- 172.21.15.38:0/3399854232 <== osd.1 v2:172.21.15.38:6816/1491818732 21 ==== osd_op_reply(118 100.00000000.inode [omap-set-header] v99'37 uv37 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f1f6000b040 con 0x55b28ca43c50 2024-09-21T03:40:37.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.231+0000 7f1f76139f40 1 -- 172.21.15.38:0/3399854232 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:119 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b28ca482d0 con 0x55b28ca43c50 2024-09-21T03:40:37.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.232+0000 7f1f6fbf3640 1 -- 172.21.15.38:0/3399854232 <== osd.1 v2:172.21.15.38:6816/1491818732 22 ==== osd_op_reply(119 1.00000000 [create] v99'38 uv31 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7f1f6000b040 con 0x55b28ca43c50 2024-09-21T03:40:37.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.232+0000 7f1f76139f40 1 -- 172.21.15.38:0/3399854232 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:120 13.1f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e99) -- 0x55b28ca482d0 con 0x55b28ca43c50 2024-09-21T03:40:37.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.233+0000 7f1f6fbf3640 1 -- 172.21.15.38:0/3399854232 <== osd.1 v2:172.21.15.38:6816/1491818732 23 ==== osd_op_reply(120 1.00000000 [delete] v99'39 uv39 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f1f6000b040 con 0x55b28ca43c50 2024-09-21T03:40:37.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.233+0000 7f1f76139f40 1 -- 172.21.15.38:0/3399854232 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:121 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b28ca482d0 con 0x55b28ca43c50 2024-09-21T03:40:37.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.234+0000 7f1f6fbf3640 1 -- 172.21.15.38:0/3399854232 <== osd.1 v2:172.21.15.38:6816/1491818732 24 ==== osd_op_reply(121 1.00000000 [create] v99'40 uv40 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f1f6000b040 con 0x55b28ca43c50 2024-09-21T03:40:37.236 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-09-21T03:40:37.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.234+0000 7f1f76139f40 1 -- 172.21.15.38:0/3399854232 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:122 13.1f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b28ca482d0 con 0x55b28ca43c50 2024-09-21T03:40:37.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.235+0000 7f1f6fbf3640 1 -- 172.21.15.38:0/3399854232 <== osd.1 v2:172.21.15.38:6816/1491818732 25 ==== osd_op_reply(122 1.00000000 [setxattr (30)] v99'41 uv41 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f1f6000b040 con 0x55b28ca43c50 2024-09-21T03:40:37.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.235+0000 7f1f76139f40 1 -- 172.21.15.38:0/3399854232 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:123 13.1f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b28ca482d0 con 0x55b28ca43c50 2024-09-21T03:40:37.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.236+0000 7f1f6fbf3640 1 -- 172.21.15.38:0/3399854232 <== osd.1 v2:172.21.15.38:6816/1491818732 26 ==== osd_op_reply(123 1.00000000 [setxattr (30)] v99'42 uv42 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f1f6000b040 con 0x55b28ca43c50 2024-09-21T03:40:37.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.236+0000 7f1f76139f40 1 -- 172.21.15.38:0/3399854232 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:124 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b28ca482d0 con 0x55b28ca43c50 2024-09-21T03:40:37.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.237+0000 7f1f6fbf3640 1 -- 172.21.15.38:0/3399854232 <== osd.1 v2:172.21.15.38:6816/1491818732 27 ==== osd_op_reply(124 1.00000000 [omap-set-header] v99'43 uv43 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f1f6000b040 con 0x55b28ca43c50 2024-09-21T03:40:37.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.237+0000 7f1f76139f40 1 -- 172.21.15.38:0/3399854232 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:125 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=584b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b28ca486e0 con 0x55b28ca43c50 2024-09-21T03:40:37.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.238+0000 7f1f6fbf3640 1 -- 172.21.15.38:0/3399854232 <== osd.1 v2:172.21.15.38:6816/1491818732 28 ==== osd_op_reply(125 1.00000000 [omap-set-vals] v99'44 uv44 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f1f6000b040 con 0x55b28ca43c50 2024-09-21T03:40:37.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.238+0000 7f1f76139f40 1 -- 172.21.15.38:0/3399854232 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x55b28ca3b9d0 msgr2=0x55b28ca3ddf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:37.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.238+0000 7f1f76139f40 1 --2- 172.21.15.38:0/3399854232 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x55b28ca3b9d0 0x55b28ca3ddf0 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:37.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.238+0000 7f1f76139f40 1 -- 172.21.15.38:0/3399854232 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x55b28ca43c50 msgr2=0x55b28ca46070 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:37.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.238+0000 7f1f76139f40 1 --2- 172.21.15.38:0/3399854232 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x55b28ca43c50 0x55b28ca46070 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:37.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.238+0000 7f1f76139f40 1 -- 172.21.15.38:0/3399854232 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x55b28ca47510 msgr2=0x55b28ca47920 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:37.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.238+0000 7f1f76139f40 1 --2- 172.21.15.38:0/3399854232 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x55b28ca47510 0x55b28ca47920 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:37.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.238+0000 7f1f76139f40 1 -- 172.21.15.38:0/3399854232 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x55b28ca3fbc0 msgr2=0x55b28ca42a20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:37.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.238+0000 7f1f76139f40 1 --2- 172.21.15.38:0/3399854232 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x55b28ca3fbc0 0x55b28ca42a20 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:37.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.238+0000 7f1f703f4640 1 -- 172.21.15.38:0/3399854232 reap_dead start 2024-09-21T03:40:37.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.238+0000 7f1f76139f40 1 -- 172.21.15.38:0/3399854232 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x55b28ca370f0 msgr2=0x55b28ca39510 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:37.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.238+0000 7f1f76139f40 1 --2- 172.21.15.38:0/3399854232 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x55b28ca370f0 0x55b28ca39510 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:37.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.238+0000 7f1f76139f40 1 -- 172.21.15.38:0/3399854232 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x55b28c975fd0 msgr2=0x55b28c8e93e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:37.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.238+0000 7f1f703f4640 1 -- 172.21.15.38:0/3399854232 reap_dead start 2024-09-21T03:40:37.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.238+0000 7f1f76139f40 1 --2- 172.21.15.38:0/3399854232 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x55b28c975fd0 0x55b28c8e93e0 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:37.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.238+0000 7f1f76139f40 1 -- 172.21.15.38:0/3399854232 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1f5c077570 msgr2=0x7f1f5c079a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:37.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.238+0000 7f1f76139f40 1 --2- 172.21.15.38:0/3399854232 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1f5c077570 0x7f1f5c079a80 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f1f64055000 tx=0x7f1f64058040 comp rx=0 tx=0).stop 2024-09-21T03:40:37.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.239+0000 7f1f76139f40 1 -- 172.21.15.38:0/3399854232 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b28c8bbd00 msgr2=0x55b28c970280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:37.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.239+0000 7f1f76139f40 1 --2- 172.21.15.38:0/3399854232 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b28c8bbd00 0x55b28c970280 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f1f6000dca0 tx=0x7f1f60004fa0 comp rx=0 tx=0).stop 2024-09-21T03:40:37.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.239+0000 7f1f76139f40 1 -- 172.21.15.38:0/3399854232 shutdown_connections 2024-09-21T03:40:37.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.239+0000 7f1f76139f40 1 --2- 172.21.15.38:0/3399854232 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x55b28c975fd0 0x55b28c8e93e0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:37.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.239+0000 7f1f76139f40 1 --2- 172.21.15.38:0/3399854232 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1f5c077570 0x7f1f5c079a80 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:37.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.239+0000 7f1f76139f40 1 --2- 172.21.15.38:0/3399854232 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b28c8bbd00 0x55b28c970280 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:37.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.239+0000 7f1f76139f40 1 -- 172.21.15.38:0/3399854232 >> 172.21.15.38:0/3399854232 conn(0x55b28c7daac0 msgr2=0x55b28c8b7660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:37.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.239+0000 7f1f76139f40 1 -- 172.21.15.38:0/3399854232 shutdown_connections 2024-09-21T03:40:37.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.239+0000 7f1f76139f40 1 -- 172.21.15.38:0/3399854232 wait complete. 2024-09-21T03:40:37.244 DEBUG:teuthology.orchestra.run.smithi038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rm 1.00000000.inode 2024-09-21T03:40:37.299 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.297+0000 7fae8983af40 1 Processor -- start 2024-09-21T03:40:37.299 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.298+0000 7fae8983af40 1 -- start start 2024-09-21T03:40:37.302 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.299+0000 7fae8983af40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5627ce867d00 0x5627ce920b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:37.302 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.299+0000 7fae8983af40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5627ce893e30 0x5627ce920180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:37.302 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.299+0000 7fae8983af40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5627ce922950 0x5627ce921790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:37.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.299+0000 7fae8983af40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5627ce7b5d60 con 0x5627ce867d00 2024-09-21T03:40:37.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.299+0000 7fae8983af40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5627ce814430 con 0x5627ce893e30 2024-09-21T03:40:37.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.299+0000 7fae8983af40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x5627ce8afef0 con 0x5627ce922950 2024-09-21T03:40:37.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.299+0000 7fae832f4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5627ce867d00 0x5627ce920b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:37.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.299+0000 7fae832f4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5627ce867d00 0x5627ce920b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:45234/0 (socket says 172.21.15.38:45234) 2024-09-21T03:40:37.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.299+0000 7fae832f4640 1 -- 172.21.15.38:0/2876049958 learned_addr learned my addr 172.21.15.38:0/2876049958 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:37.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.299+0000 7fae82af3640 1 --2- 172.21.15.38:0/2876049958 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5627ce893e30 0x5627ce920180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:37.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.299+0000 7fae83af5640 1 --2- 172.21.15.38:0/2876049958 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5627ce922950 0x5627ce921790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:37.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.299+0000 7fae832f4640 1 -- 172.21.15.38:0/2876049958 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5627ce922950 msgr2=0x5627ce921790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:37.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.299+0000 7fae832f4640 1 --2- 172.21.15.38:0/2876049958 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5627ce922950 0x5627ce921790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:37.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.299+0000 7fae832f4640 1 -- 172.21.15.38:0/2876049958 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5627ce893e30 msgr2=0x5627ce920180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:37.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.299+0000 7fae832f4640 1 --2- 172.21.15.38:0/2876049958 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5627ce893e30 0x5627ce920180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:37.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.299+0000 7fae832f4640 1 -- 172.21.15.38:0/2876049958 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5627ce866fd0 con 0x5627ce867d00 2024-09-21T03:40:37.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.299+0000 7fae83af5640 1 --2- 172.21.15.38:0/2876049958 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5627ce922950 0x5627ce921790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:40:37.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.300+0000 7fae832f4640 1 --2- 172.21.15.38:0/2876049958 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5627ce867d00 0x5627ce920b40 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7fae74001660 tx=0x7fae74001b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cde7a9be90d6f043 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:37.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.300+0000 7fae81af1640 1 -- 172.21.15.38:0/2876049958 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae74017070 con 0x5627ce867d00 2024-09-21T03:40:37.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.300+0000 7fae81af1640 1 -- 172.21.15.38:0/2876049958 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fae740027a0 con 0x5627ce867d00 2024-09-21T03:40:37.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.300+0000 7fae81af1640 1 -- 172.21.15.38:0/2876049958 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae74002c30 con 0x5627ce867d00 2024-09-21T03:40:37.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.300+0000 7fae8983af40 1 -- 172.21.15.38:0/2876049958 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5627ce867d00 msgr2=0x5627ce920b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:37.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.300+0000 7fae8983af40 1 --2- 172.21.15.38:0/2876049958 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5627ce867d00 0x5627ce920b40 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7fae74001660 tx=0x7fae74001b20 comp rx=0 tx=0).stop 2024-09-21T03:40:37.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.300+0000 7fae8983af40 1 -- 172.21.15.38:0/2876049958 shutdown_connections 2024-09-21T03:40:37.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.300+0000 7fae8983af40 1 --2- 172.21.15.38:0/2876049958 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5627ce922950 0x5627ce921790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:37.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.300+0000 7fae8983af40 1 --2- 172.21.15.38:0/2876049958 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5627ce893e30 0x5627ce920180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:37.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.300+0000 7fae8983af40 1 --2- 172.21.15.38:0/2876049958 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5627ce867d00 0x5627ce920b40 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:37.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.300+0000 7fae8983af40 1 -- 172.21.15.38:0/2876049958 >> 172.21.15.38:0/2876049958 conn(0x5627ce786ac0 msgr2=0x5627ce895280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:37.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.300+0000 7fae8983af40 1 -- 172.21.15.38:0/2876049958 shutdown_connections 2024-09-21T03:40:37.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.300+0000 7fae8983af40 1 -- 172.21.15.38:0/2876049958 wait complete. 2024-09-21T03:40:37.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.301+0000 7fae8983af40 1 Processor -- start 2024-09-21T03:40:37.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.301+0000 7fae8983af40 1 -- start start 2024-09-21T03:40:37.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.303+0000 7fae8983af40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5627ce867d00 0x5627ce9d3d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:37.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.303+0000 7fae8983af40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5627ce893e30 0x5627ce91c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:37.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.303+0000 7fae8983af40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5627ce922950 0x5627ce939210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:37.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.303+0000 7fae8983af40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5627ce8acc10 con 0x5627ce893e30 2024-09-21T03:40:37.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.303+0000 7fae8983af40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5627ce892450 con 0x5627ce867d00 2024-09-21T03:40:37.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.303+0000 7fae8983af40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x5627ce8919c0 con 0x5627ce922950 2024-09-21T03:40:37.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.303+0000 7fae82af3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5627ce893e30 0x5627ce91c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:37.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.303+0000 7fae82af3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5627ce893e30 0x5627ce91c0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:45240/0 (socket says 172.21.15.38:45240) 2024-09-21T03:40:37.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.303+0000 7fae832f4640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5627ce867d00 0x5627ce9d3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:37.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.303+0000 7fae82af3640 1 -- 172.21.15.38:0/1400999300 learned_addr learned my addr 172.21.15.38:0/1400999300 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:37.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.303+0000 7fae83af5640 1 --2- 172.21.15.38:0/1400999300 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5627ce922950 0x5627ce939210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:37.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.303+0000 7fae82af3640 1 -- 172.21.15.38:0/1400999300 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5627ce922950 msgr2=0x5627ce939210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:37.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.303+0000 7fae82af3640 1 --2- 172.21.15.38:0/1400999300 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5627ce922950 0x5627ce939210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:37.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.303+0000 7fae82af3640 1 -- 172.21.15.38:0/1400999300 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5627ce867d00 msgr2=0x5627ce9d3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:37.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.303+0000 7fae82af3640 1 --2- 172.21.15.38:0/1400999300 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5627ce867d00 0x5627ce9d3d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:37.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.303+0000 7fae82af3640 1 -- 172.21.15.38:0/1400999300 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5627ce93a990 con 0x5627ce893e30 2024-09-21T03:40:37.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.303+0000 7fae832f4640 1 --2- 172.21.15.38:0/1400999300 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5627ce867d00 0x5627ce9d3d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:40:37.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.303+0000 7fae82af3640 1 --2- 172.21.15.38:0/1400999300 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5627ce893e30 0x5627ce91c0f0 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7fae6c0044b0 tx=0x7fae6c00ac20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:37.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.303+0000 7fae80aef640 1 -- 172.21.15.38:0/1400999300 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae6c01c070 con 0x5627ce893e30 2024-09-21T03:40:37.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.303+0000 7fae80aef640 1 -- 172.21.15.38:0/1400999300 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fae6c008d40 con 0x5627ce893e30 2024-09-21T03:40:37.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.303+0000 7fae8983af40 1 -- 172.21.15.38:0/1400999300 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5627ce8921a0 con 0x5627ce893e30 2024-09-21T03:40:37.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.304+0000 7fae80aef640 1 -- 172.21.15.38:0/1400999300 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae6c0176e0 con 0x5627ce893e30 2024-09-21T03:40:37.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.304+0000 7fae8983af40 1 -- 172.21.15.38:0/1400999300 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5627ce93a0f0 con 0x5627ce893e30 2024-09-21T03:40:37.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.305+0000 7fae80aef640 1 -- 172.21.15.38:0/1400999300 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fae6c01f070 con 0x5627ce893e30 2024-09-21T03:40:37.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.306+0000 7fae80aef640 1 --2- 172.21.15.38:0/1400999300 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fae64077570 0x7fae64079a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:37.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.306+0000 7fae80aef640 1 -- 172.21.15.38:0/1400999300 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 6562+0+0 (secure 0 0 0) 0x7fae6c09a0d0 con 0x5627ce893e30 2024-09-21T03:40:37.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.306+0000 7fae8983af40 1 --2- 172.21.15.38:0/1400999300 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x5627ce8abc90 0x5627ce93bef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:37.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.306+0000 7fae8983af40 1 -- 172.21.15.38:0/1400999300 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:1 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e99) -- 0x5627ce8ac4e0 con 0x5627ce8abc90 2024-09-21T03:40:37.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.306+0000 7fae832f4640 1 --2- 172.21.15.38:0/1400999300 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fae64077570 0x7fae64079a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:37.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.306+0000 7fae83af5640 1 --2- 172.21.15.38:0/1400999300 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x5627ce8abc90 0x5627ce93bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:37.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.307+0000 7fae832f4640 1 --2- 172.21.15.38:0/1400999300 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fae64077570 0x7fae64079a80 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fae74001660 tx=0x7fae74001400 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:37.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.307+0000 7fae83af5640 1 --2- 172.21.15.38:0/1400999300 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x5627ce8abc90 0x5627ce93bef0 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:37.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.308+0000 7fae83af5640 1 -- 172.21.15.38:0/1400999300 <== osd.6 v2:172.21.15.195:6826/4055825675 1 ==== osd_op_reply(1 1.00000000.inode [delete] v99'87 uv87 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fae7c056470 con 0x5627ce8abc90 2024-09-21T03:40:37.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.308+0000 7fae8983af40 1 -- 172.21.15.38:0/1400999300 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x5627ce8abc90 msgr2=0x5627ce93bef0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:37.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.308+0000 7fae8983af40 1 --2- 172.21.15.38:0/1400999300 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x5627ce8abc90 0x5627ce93bef0 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:37.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.308+0000 7fae8983af40 1 -- 172.21.15.38:0/1400999300 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fae64077570 msgr2=0x7fae64079a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:37.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.308+0000 7fae8983af40 1 --2- 172.21.15.38:0/1400999300 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fae64077570 0x7fae64079a80 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fae74001660 tx=0x7fae74001400 comp rx=0 tx=0).stop 2024-09-21T03:40:37.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.308+0000 7fae8983af40 1 -- 172.21.15.38:0/1400999300 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5627ce893e30 msgr2=0x5627ce91c0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:37.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.308+0000 7fae8983af40 1 --2- 172.21.15.38:0/1400999300 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5627ce893e30 0x5627ce91c0f0 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7fae6c0044b0 tx=0x7fae6c00ac20 comp rx=0 tx=0).stop 2024-09-21T03:40:37.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.308+0000 7fae83af5640 1 -- 172.21.15.38:0/1400999300 reap_dead start 2024-09-21T03:40:37.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.308+0000 7fae8983af40 1 -- 172.21.15.38:0/1400999300 shutdown_connections 2024-09-21T03:40:37.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.308+0000 7fae8983af40 1 -- 172.21.15.38:0/1400999300 >> 172.21.15.38:0/1400999300 conn(0x5627ce786ac0 msgr2=0x5627ce920140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:37.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.308+0000 7fae8983af40 1 -- 172.21.15.38:0/1400999300 shutdown_connections 2024-09-21T03:40:37.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.308+0000 7fae8983af40 1 -- 172.21.15.38:0/1400999300 wait complete. 2024-09-21T03:40:37.313 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-21T03:40:37.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.431+0000 7f832a6f9640 1 -- 172.21.15.38:0/1397166609 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f83240b5260 msgr2=0x7f832415ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:37.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.431+0000 7f832a6f9640 1 --2- 172.21.15.38:0/1397166609 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f83240b5260 0x7f832415ba80 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f8318009830 tx=0x7f831802a5c0 comp rx=0 tx=0).stop 2024-09-21T03:40:37.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.431+0000 7f832a6f9640 1 -- 172.21.15.38:0/1397166609 shutdown_connections 2024-09-21T03:40:37.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.431+0000 7f832a6f9640 1 --2- 172.21.15.38:0/1397166609 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f83240b5260 0x7f832415ba80 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:37.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.431+0000 7f832a6f9640 1 -- 172.21.15.38:0/1397166609 >> 172.21.15.38:0/1397166609 conn(0x7f83240cdca0 msgr2=0x7f83240ce0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:37.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.431+0000 7f832a6f9640 1 -- 172.21.15.38:0/1397166609 shutdown_connections 2024-09-21T03:40:37.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.431+0000 7f832a6f9640 1 -- 172.21.15.38:0/1397166609 wait complete. 2024-09-21T03:40:37.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.432+0000 7f832a6f9640 1 Processor -- start 2024-09-21T03:40:37.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.433+0000 7f832a6f9640 1 -- start start 2024-09-21T03:40:37.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.434+0000 7f832a6f9640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f832406def0 0x7f8324158a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:37.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.434+0000 7f832a6f9640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f83240b5260 0x7f832414c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:37.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.434+0000 7f832a6f9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f832415bfc0 0x7f83240cff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:37.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.434+0000 7f832a6f9640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f8324071300 con 0x7f832415bfc0 2024-09-21T03:40:37.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.434+0000 7f832a6f9640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f8324071480 con 0x7f83240b5260 2024-09-21T03:40:37.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.434+0000 7f832a6f9640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f8324071600 con 0x7f832406def0 2024-09-21T03:40:37.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.434+0000 7f8328c96640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f832415bfc0 0x7f83240cff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:37.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.434+0000 7f8328c96640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f832415bfc0 0x7f83240cff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:45262/0 (socket says 172.21.15.38:45262) 2024-09-21T03:40:37.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.434+0000 7f8328c96640 1 -- 172.21.15.38:0/949034161 learned_addr learned my addr 172.21.15.38:0/949034161 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:37.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.434+0000 7f83237fe640 1 --2- 172.21.15.38:0/949034161 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f83240b5260 0x7f832414c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:37.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.434+0000 7f8323fff640 1 --2- 172.21.15.38:0/949034161 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f832406def0 0x7f8324158a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:37.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.434+0000 7f8328c96640 1 -- 172.21.15.38:0/949034161 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f832406def0 msgr2=0x7f8324158a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:37.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.434+0000 7f8328c96640 1 --2- 172.21.15.38:0/949034161 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f832406def0 0x7f8324158a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:37.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.434+0000 7f8328c96640 1 -- 172.21.15.38:0/949034161 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f83240b5260 msgr2=0x7f832414c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:37.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.434+0000 7f8328c96640 1 --2- 172.21.15.38:0/949034161 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f83240b5260 0x7f832414c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:37.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.434+0000 7f8328c96640 1 -- 172.21.15.38:0/949034161 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83241f3760 con 0x7f832415bfc0 2024-09-21T03:40:37.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.435+0000 7f8328c96640 1 --2- 172.21.15.38:0/949034161 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f832415bfc0 0x7f83240cff70 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f831c00e9b0 tx=0x7f831c00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:37.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.435+0000 7f8320ff9640 1 -- 172.21.15.38:0/949034161 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f831c00ccc0 con 0x7f832415bfc0 2024-09-21T03:40:37.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.435+0000 7f8320ff9640 1 -- 172.21.15.38:0/949034161 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f831c003970 con 0x7f832415bfc0 2024-09-21T03:40:37.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.435+0000 7f832a6f9640 1 -- 172.21.15.38:0/949034161 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83241f2b50 con 0x7f832415bfc0 2024-09-21T03:40:37.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.435+0000 7f8320ff9640 1 -- 172.21.15.38:0/949034161 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f831c003da0 con 0x7f832415bfc0 2024-09-21T03:40:37.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.435+0000 7f832a6f9640 1 -- 172.21.15.38:0/949034161 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83241f3110 con 0x7f832415bfc0 2024-09-21T03:40:37.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.436+0000 7f832a6f9640 1 -- 172.21.15.38:0/949034161 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8324071b80 con 0x7f832415bfc0 2024-09-21T03:40:37.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.437+0000 7f8320ff9640 1 -- 172.21.15.38:0/949034161 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f831c0050d0 con 0x7f832415bfc0 2024-09-21T03:40:37.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.437+0000 7f8320ff9640 1 --2- 172.21.15.38:0/949034161 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8310078bc0 0x7f831007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:37.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.437+0000 7f8320ff9640 1 -- 172.21.15.38:0/949034161 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 6562+0+0 (secure 0 0 0) 0x7f831c09d8a0 con 0x7f832415bfc0 2024-09-21T03:40:37.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.437+0000 7f8323fff640 1 --2- 172.21.15.38:0/949034161 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8310078bc0 0x7f831007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:37.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.438+0000 7f8323fff640 1 --2- 172.21.15.38:0/949034161 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8310078bc0 0x7f831007b0d0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f8314000f60 tx=0x7f8314009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:37.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.442+0000 7f8320ff9640 1 -- 172.21.15.38:0/949034161 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f831c0653c0 con 0x7f832415bfc0 2024-09-21T03:40:37.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:37.666+0000 7f832a6f9640 1 -- 172.21.15.38:0/949034161 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f83241f28e0 con 0x7f832415bfc0 2024-09-21T03:40:38.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:38.015+0000 7f8320ff9640 1 -- 172.21.15.38:0/949034161 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v103) ==== 162+0+0 (secure 0 0 0) 0x7f831c069df0 con 0x7f832415bfc0 2024-09-21T03:40:38.017 INFO:teuthology.orchestra.run.smithi038.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-21T03:40:38.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:38.017+0000 7f832a6f9640 1 -- 172.21.15.38:0/949034161 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8310078bc0 msgr2=0x7f831007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:38.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:38.017+0000 7f832a6f9640 1 --2- 172.21.15.38:0/949034161 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8310078bc0 0x7f831007b0d0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f8314000f60 tx=0x7f8314009040 comp rx=0 tx=0).stop 2024-09-21T03:40:38.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:38.017+0000 7f832a6f9640 1 -- 172.21.15.38:0/949034161 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f832415bfc0 msgr2=0x7f83240cff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:38.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:38.017+0000 7f832a6f9640 1 --2- 172.21.15.38:0/949034161 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f832415bfc0 0x7f83240cff70 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f831c00e9b0 tx=0x7f831c00ee70 comp rx=0 tx=0).stop 2024-09-21T03:40:38.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:38.017+0000 7f832a6f9640 1 -- 172.21.15.38:0/949034161 shutdown_connections 2024-09-21T03:40:38.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:38.017+0000 7f832a6f9640 1 --2- 172.21.15.38:0/949034161 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f832415bfc0 0x7f83240cff70 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:38.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:38.017+0000 7f832a6f9640 1 --2- 172.21.15.38:0/949034161 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f83240b5260 0x7f832414c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:38.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:38.017+0000 7f832a6f9640 1 --2- 172.21.15.38:0/949034161 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8310078bc0 0x7f831007b0d0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:38.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:38.017+0000 7f832a6f9640 1 --2- 172.21.15.38:0/949034161 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f832406def0 0x7f8324158a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:38.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:38.017+0000 7f832a6f9640 1 -- 172.21.15.38:0/949034161 >> 172.21.15.38:0/949034161 conn(0x7f83240cdca0 msgr2=0x7f83240ce0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:38.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:38.018+0000 7f832a6f9640 1 -- 172.21.15.38:0/949034161 shutdown_connections 2024-09-21T03:40:38.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:38.018+0000 7f832a6f9640 1 -- 172.21.15.38:0/949034161 wait complete. 2024-09-21T03:40:38.030 INFO:tasks.ceph.mds.a.smithi038.stderr:2024-09-21T03:40:38.028+0000 7f9460c6c640 -1 mds.0.cache.ino(0x1) No data while reading inode 0x1 2024-09-21T03:40:38.030 INFO:tasks.ceph.mds.a.smithi038.stderr:2024-09-21T03:40:38.028+0000 7f9460c6c640 -1 log_channel(cluster) log [ERR] : Error loading MDS rank 0: (2) No such file or directory 2024-09-21T03:40:38.031 INFO:tasks.ceph.mon.a.smithi038.stderr:2024-09-21T03:40:38.029+0000 7f3f45713640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T03:40:38.031 INFO:tasks.ceph.mon.a.smithi038.stderr:2024-09-21T03:40:38.029+0000 7f3f45713640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-09-21T03:40:38.034 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:40:38.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:38.149+0000 7f96d0863640 1 -- 172.21.15.38:0/3347200661 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f96cc0d12e0 msgr2=0x7f96cc0d16f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:38.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:38.149+0000 7f96d0863640 1 --2- 172.21.15.38:0/3347200661 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f96cc0d12e0 0x7f96cc0d16f0 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f96cc15d7e0 tx=0x7f96b402a540 comp rx=0 tx=0).stop 2024-09-21T03:40:38.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:38.149+0000 7f96d0863640 1 -- 172.21.15.38:0/3347200661 shutdown_connections 2024-09-21T03:40:38.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:38.149+0000 7f96d0863640 1 --2- 172.21.15.38:0/3347200661 >> v2:172.21.15.195:3300/0 conn(0x7f96cc160100 0x7f96cc160510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:38.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:38.149+0000 7f96d0863640 1 --2- 172.21.15.38:0/3347200661 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f96cc0d12e0 0x7f96cc0d16f0 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:38.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:38.149+0000 7f96d0863640 1 --2- 172.21.15.38:0/3347200661 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f96cc15cb10 0x7f96cc15cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:38.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:38.149+0000 7f96d0863640 1 -- 172.21.15.38:0/3347200661 >> 172.21.15.38:0/3347200661 conn(0x7f96cc0d2350 msgr2=0x7f96cc0d2760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:38.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:38.149+0000 7f96d0863640 1 -- 172.21.15.38:0/3347200661 shutdown_connections 2024-09-21T03:40:38.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:38.149+0000 7f96d0863640 1 -- 172.21.15.38:0/3347200661 wait complete. 2024-09-21T03:40:38.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:38.151+0000 7f96d0863640 1 Processor -- start 2024-09-21T03:40:38.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:38.151+0000 7f96d0863640 1 -- start start 2024-09-21T03:40:38.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:38.153+0000 7f96d0863640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f96cc0d12e0 0x7f96cc1f6ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:38.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:38.153+0000 7f96d0863640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f96cc15cb10 0x7f96cc1f7430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:38.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:38.153+0000 7f96d0863640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f96cc160100 0x7f96cc15b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:38.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:38.153+0000 7f96d0863640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f96cc160bd0 con 0x7f96cc0d12e0 2024-09-21T03:40:38.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:38.153+0000 7f96d0863640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f96cc160a50 con 0x7f96cc160100 2024-09-21T03:40:38.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:38.153+0000 7f96d0863640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f96cc160d50 con 0x7f96cc15cb10 2024-09-21T03:40:38.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:38.153+0000 7f96ca59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f96cc0d12e0 0x7f96cc1f6ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:38.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:38.153+0000 7f96ca59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f96cc0d12e0 0x7f96cc1f6ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:45290/0 (socket says 172.21.15.38:45290) 2024-09-21T03:40:38.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:38.153+0000 7f96ca59c640 1 -- 172.21.15.38:0/1325203431 learned_addr learned my addr 172.21.15.38:0/1325203431 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:38.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:38.153+0000 7f96c9d9b640 1 --2- 172.21.15.38:0/1325203431 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f96cc15cb10 0x7f96cc1f7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:38.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:38.153+0000 7f96cad9d640 1 --2- 172.21.15.38:0/1325203431 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f96cc160100 0x7f96cc15b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:38.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:38.153+0000 7f96ca59c640 1 -- 172.21.15.38:0/1325203431 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f96cc15cb10 msgr2=0x7f96cc1f7430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:38.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:38.153+0000 7f96ca59c640 1 --2- 172.21.15.38:0/1325203431 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f96cc15cb10 0x7f96cc1f7430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:38.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:38.153+0000 7f96ca59c640 1 -- 172.21.15.38:0/1325203431 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f96cc160100 msgr2=0x7f96cc15b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:38.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:38.153+0000 7f96ca59c640 1 --2- 172.21.15.38:0/1325203431 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f96cc160100 0x7f96cc15b560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:38.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:38.153+0000 7f96ca59c640 1 -- 172.21.15.38:0/1325203431 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96cc1f7d20 con 0x7f96cc0d12e0 2024-09-21T03:40:38.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:38.153+0000 7f96ca59c640 1 --2- 172.21.15.38:0/1325203431 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f96cc0d12e0 0x7f96cc1f6ef0 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f96c400e7c0 tx=0x7f96c400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:38.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:38.153+0000 7f96c2ffd640 1 -- 172.21.15.38:0/1325203431 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96c400ccc0 con 0x7f96cc0d12e0 2024-09-21T03:40:38.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:38.153+0000 7f96c2ffd640 1 -- 172.21.15.38:0/1325203431 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f96c400ce60 con 0x7f96cc0d12e0 2024-09-21T03:40:38.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:38.153+0000 7f96d0863640 1 -- 172.21.15.38:0/1325203431 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96cc1e6cc0 con 0x7f96cc0d12e0 2024-09-21T03:40:38.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:38.154+0000 7f96c2ffd640 1 -- 172.21.15.38:0/1325203431 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96c4005280 con 0x7f96cc0d12e0 2024-09-21T03:40:38.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:38.154+0000 7f96d0863640 1 -- 172.21.15.38:0/1325203431 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96cc1e7280 con 0x7f96cc0d12e0 2024-09-21T03:40:38.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:38.154+0000 7f96d0863640 1 -- 172.21.15.38:0/1325203431 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96cc15cf20 con 0x7f96cc0d12e0 2024-09-21T03:40:38.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:38.155+0000 7f96c2ffd640 1 -- 172.21.15.38:0/1325203431 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f96c4005420 con 0x7f96cc0d12e0 2024-09-21T03:40:38.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:38.156+0000 7f96c2ffd640 1 --2- 172.21.15.38:0/1325203431 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f96ac078b90 0x7f96ac07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:38.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:38.156+0000 7f96c2ffd640 1 -- 172.21.15.38:0/1325203431 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 6648+0+0 (secure 0 0 0) 0x7f96c409b950 con 0x7f96cc0d12e0 2024-09-21T03:40:38.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:38.156+0000 7f96c9d9b640 1 --2- 172.21.15.38:0/1325203431 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f96ac078b90 0x7f96ac07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:38.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:38.157+0000 7f96c9d9b640 1 --2- 172.21.15.38:0/1325203431 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f96ac078b90 0x7f96ac07b0a0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f96b4033000 tx=0x7f96b4004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:38.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:38.160+0000 7f96c2ffd640 1 -- 172.21.15.38:0/1325203431 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f96c4063380 con 0x7f96cc0d12e0 2024-09-21T03:40:38.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:38.384+0000 7f96d0863640 1 -- 172.21.15.38:0/1325203431 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f96cc0b56b0 con 0x7f96cc0d12e0 2024-09-21T03:40:38.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:38.385+0000 7f96c2ffd640 1 -- 172.21.15.38:0/1325203431 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 105 v105) ==== 95+0+4388 (secure 0 0 0) 0x7f96c4067db0 con 0x7f96cc0d12e0 2024-09-21T03:40:38.387 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:40:38.387 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":105,"btime":"2024-09-21T03:40:38:029932+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6086,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/965972663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":965972663},{"type":"v1","addr":"172.21.15.195:6836","nonce":965972663}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6700,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/3176122334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":3176122334},{"type":"v1","addr":"172.21.15.195:6837","nonce":3176122334}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":7020,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/751618334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":751618334},{"type":"v1","addr":"172.21.15.38:6833","nonce":751618334}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":105,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:40:02.319282+0000","modified":"2024-09-21T03:40:38.029836+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":100,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-21T03:40:38.387 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 105 2024-09-21T03:40:38.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:38.387+0000 7f96d0863640 1 -- 172.21.15.38:0/1325203431 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f96ac078b90 msgr2=0x7f96ac07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:38.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:38.387+0000 7f96d0863640 1 --2- 172.21.15.38:0/1325203431 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f96ac078b90 0x7f96ac07b0a0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f96b4033000 tx=0x7f96b4004730 comp rx=0 tx=0).stop 2024-09-21T03:40:38.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:38.387+0000 7f96d0863640 1 -- 172.21.15.38:0/1325203431 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f96cc0d12e0 msgr2=0x7f96cc1f6ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:38.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:38.387+0000 7f96d0863640 1 --2- 172.21.15.38:0/1325203431 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f96cc0d12e0 0x7f96cc1f6ef0 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f96c400e7c0 tx=0x7f96c400ec80 comp rx=0 tx=0).stop 2024-09-21T03:40:38.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:38.387+0000 7f96d0863640 1 -- 172.21.15.38:0/1325203431 shutdown_connections 2024-09-21T03:40:38.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:38.387+0000 7f96d0863640 1 --2- 172.21.15.38:0/1325203431 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f96cc160100 0x7f96cc15b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:38.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:38.387+0000 7f96d0863640 1 --2- 172.21.15.38:0/1325203431 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f96ac078b90 0x7f96ac07b0a0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:38.390 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:38.387+0000 7f96d0863640 1 --2- 172.21.15.38:0/1325203431 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f96cc15cb10 0x7f96cc1f7430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:38.390 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:38.387+0000 7f96d0863640 1 --2- 172.21.15.38:0/1325203431 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f96cc0d12e0 0x7f96cc1f6ef0 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:38.390 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:38.387+0000 7f96d0863640 1 -- 172.21.15.38:0/1325203431 >> 172.21.15.38:0/1325203431 conn(0x7f96cc0d2350 msgr2=0x7f96cc071430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:38.390 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:38.387+0000 7f96d0863640 1 -- 172.21.15.38:0/1325203431 shutdown_connections 2024-09-21T03:40:38.390 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:38.387+0000 7f96d0863640 1 -- 172.21.15.38:0/1325203431 wait complete. 2024-09-21T03:40:38.400 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-21T03:40:38.401 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:40:38.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:38.516+0000 7f87c1e69640 1 -- 172.21.15.38:0/444809205 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f87bc162f00 msgr2=0x7f87bc0d24d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:38.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:38.516+0000 7f87c1e69640 1 --2- 172.21.15.38:0/444809205 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f87bc162f00 0x7f87bc0d24d0 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f87bc0b7460 tx=0x7f87ac02a560 comp rx=0 tx=0).stop 2024-09-21T03:40:38.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:38.516+0000 7f87c1e69640 1 -- 172.21.15.38:0/444809205 shutdown_connections 2024-09-21T03:40:38.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:38.516+0000 7f87c1e69640 1 --2- 172.21.15.38:0/444809205 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f87bc0b2e40 0x7f87bc0b3250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:38.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:38.516+0000 7f87c1e69640 1 --2- 172.21.15.38:0/444809205 >> v2:172.21.15.195:3300/0 conn(0x7f87bc162280 0x7f87bc162690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:38.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:38.516+0000 7f87c1e69640 1 --2- 172.21.15.38:0/444809205 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f87bc162f00 0x7f87bc0d24d0 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:38.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:38.516+0000 7f87c1e69640 1 -- 172.21.15.38:0/444809205 >> 172.21.15.38:0/444809205 conn(0x7f87bc0b6b20 msgr2=0x7f87bc0d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:38.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:38.516+0000 7f87c1e69640 1 -- 172.21.15.38:0/444809205 shutdown_connections 2024-09-21T03:40:38.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:38.517+0000 7f87c1e69640 1 -- 172.21.15.38:0/444809205 wait complete. 2024-09-21T03:40:38.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:38.518+0000 7f87c1e69640 1 Processor -- start 2024-09-21T03:40:38.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:38.518+0000 7f87c1e69640 1 -- start start 2024-09-21T03:40:38.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:38.520+0000 7f87c1e69640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f87bc0b2e40 0x7f87bc0acde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:38.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:38.520+0000 7f87c1e69640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f87bc162280 0x7f87bc15aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:38.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:38.520+0000 7f87c1e69640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f87bc15afa0 0x7f87bc0b2220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:38.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:38.520+0000 7f87c1e69640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f87bc0b3dd0 con 0x7f87bc0b2e40 2024-09-21T03:40:38.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:38.520+0000 7f87c1e69640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f87bc0b3a20 con 0x7f87bc162280 2024-09-21T03:40:38.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:38.520+0000 7f87c1e69640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f87bc0b3f50 con 0x7f87bc15afa0 2024-09-21T03:40:38.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:38.520+0000 7f87bb7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f87bc0b2e40 0x7f87bc0acde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:38.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:38.520+0000 7f87bb7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f87bc0b2e40 0x7f87bc0acde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:45306/0 (socket says 172.21.15.38:45306) 2024-09-21T03:40:38.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:38.520+0000 7f87bb7fe640 1 -- 172.21.15.38:0/173660732 learned_addr learned my addr 172.21.15.38:0/173660732 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:38.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:38.520+0000 7f87baffd640 1 --2- 172.21.15.38:0/173660732 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f87bc162280 0x7f87bc15aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:38.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:38.520+0000 7f87bbfff640 1 --2- 172.21.15.38:0/173660732 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f87bc15afa0 0x7f87bc0b2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:38.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:38.520+0000 7f87bb7fe640 1 -- 172.21.15.38:0/173660732 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f87bc15afa0 msgr2=0x7f87bc0b2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:38.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:38.520+0000 7f87bb7fe640 1 --2- 172.21.15.38:0/173660732 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f87bc15afa0 0x7f87bc0b2220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:38.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:38.520+0000 7f87bb7fe640 1 -- 172.21.15.38:0/173660732 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f87bc162280 msgr2=0x7f87bc15aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:38.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:38.520+0000 7f87bb7fe640 1 --2- 172.21.15.38:0/173660732 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f87bc162280 0x7f87bc15aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:38.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:38.520+0000 7f87bb7fe640 1 -- 172.21.15.38:0/173660732 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87bc15b3b0 con 0x7f87bc0b2e40 2024-09-21T03:40:38.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:38.520+0000 7f87bb7fe640 1 --2- 172.21.15.38:0/173660732 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f87bc0b2e40 0x7f87bc0acde0 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f87ac000c00 tx=0x7f87ac003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:38.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:38.521+0000 7f87a3fff640 1 -- 172.21.15.38:0/173660732 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87ac03e030 con 0x7f87bc0b2e40 2024-09-21T03:40:38.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:38.521+0000 7f87a3fff640 1 -- 172.21.15.38:0/173660732 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f87ac003e00 con 0x7f87bc0b2e40 2024-09-21T03:40:38.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:38.521+0000 7f87a3fff640 1 -- 172.21.15.38:0/173660732 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87ac035040 con 0x7f87bc0b2e40 2024-09-21T03:40:38.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:38.521+0000 7f87c1e69640 1 -- 172.21.15.38:0/173660732 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87bc1f54c0 con 0x7f87bc0b2e40 2024-09-21T03:40:38.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:38.521+0000 7f87c1e69640 1 -- 172.21.15.38:0/173660732 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87bc1f5a80 con 0x7f87bc0b2e40 2024-09-21T03:40:38.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:38.522+0000 7f87c1e69640 1 -- 172.21.15.38:0/173660732 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87bc162f00 con 0x7f87bc0b2e40 2024-09-21T03:40:38.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:38.523+0000 7f87a3fff640 1 -- 172.21.15.38:0/173660732 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f87ac041030 con 0x7f87bc0b2e40 2024-09-21T03:40:38.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:38.523+0000 7f87a3fff640 1 --2- 172.21.15.38:0/173660732 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f87a8078bc0 0x7f87a807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:38.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:38.523+0000 7f87a3fff640 1 -- 172.21.15.38:0/173660732 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 6648+0+0 (secure 0 0 0) 0x7f87ac0bb920 con 0x7f87bc0b2e40 2024-09-21T03:40:38.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:38.523+0000 7f87baffd640 1 --2- 172.21.15.38:0/173660732 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f87a8078bc0 0x7f87a807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:38.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:38.524+0000 7f87baffd640 1 --2- 172.21.15.38:0/173660732 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f87a8078bc0 0x7f87a807b0d0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f87bc15fa30 tx=0x7f87b000b2e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:38.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:38.528+0000 7f87a3fff640 1 -- 172.21.15.38:0/173660732 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f87bc15b3b0 con 0x7f87bc0b2e40 2024-09-21T03:40:38.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:38.753+0000 7f87c1e69640 1 -- 172.21.15.38:0/173660732 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f87bc0b9820 con 0x7f87bc0b2e40 2024-09-21T03:40:38.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:38.753+0000 7f87a3fff640 1 -- 172.21.15.38:0/173660732 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 105 v105) ==== 95+0+4388 (secure 0 0 0) 0x7f87ac0832e0 con 0x7f87bc0b2e40 2024-09-21T03:40:38.755 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:40:38.755 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":105,"btime":"2024-09-21T03:40:38:029932+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6086,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/965972663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":965972663},{"type":"v1","addr":"172.21.15.195:6836","nonce":965972663}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6700,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/3176122334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":3176122334},{"type":"v1","addr":"172.21.15.195:6837","nonce":3176122334}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":7020,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/751618334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":751618334},{"type":"v1","addr":"172.21.15.38:6833","nonce":751618334}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":105,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:40:02.319282+0000","modified":"2024-09-21T03:40:38.029836+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":100,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-21T03:40:38.755 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 105 2024-09-21T03:40:38.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:38.756+0000 7f87c1e69640 1 -- 172.21.15.38:0/173660732 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f87a8078bc0 msgr2=0x7f87a807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:38.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:38.756+0000 7f87c1e69640 1 --2- 172.21.15.38:0/173660732 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f87a8078bc0 0x7f87a807b0d0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f87bc15fa30 tx=0x7f87b000b2e0 comp rx=0 tx=0).stop 2024-09-21T03:40:38.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:38.756+0000 7f87c1e69640 1 -- 172.21.15.38:0/173660732 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f87bc0b2e40 msgr2=0x7f87bc0acde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:38.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:38.756+0000 7f87c1e69640 1 --2- 172.21.15.38:0/173660732 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f87bc0b2e40 0x7f87bc0acde0 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f87ac000c00 tx=0x7f87ac003910 comp rx=0 tx=0).stop 2024-09-21T03:40:38.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:38.756+0000 7f87c1e69640 1 -- 172.21.15.38:0/173660732 shutdown_connections 2024-09-21T03:40:38.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:38.756+0000 7f87c1e69640 1 --2- 172.21.15.38:0/173660732 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f87a8078bc0 0x7f87a807b0d0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:38.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:38.756+0000 7f87c1e69640 1 --2- 172.21.15.38:0/173660732 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f87bc15afa0 0x7f87bc0b2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:38.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:38.756+0000 7f87c1e69640 1 --2- 172.21.15.38:0/173660732 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f87bc162280 0x7f87bc15aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:38.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:38.756+0000 7f87c1e69640 1 --2- 172.21.15.38:0/173660732 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f87bc0b2e40 0x7f87bc0acde0 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:38.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:38.756+0000 7f87c1e69640 1 -- 172.21.15.38:0/173660732 >> 172.21.15.38:0/173660732 conn(0x7f87bc0b6b20 msgr2=0x7f87bc15d990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:38.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:38.756+0000 7f87c1e69640 1 -- 172.21.15.38:0/173660732 shutdown_connections 2024-09-21T03:40:38.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:38.756+0000 7f87c1e69640 1 -- 172.21.15.38:0/173660732 wait complete. 2024-09-21T03:40:38.769 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-21T03:40:38.769 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:40:38.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:38.884+0000 7f3c8d8e6640 1 -- 172.21.15.38:0/3721085442 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3c8806f530 msgr2=0x7f3c880ce730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:38.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:38.884+0000 7f3c8d8e6640 1 --2- 172.21.15.38:0/3721085442 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3c8806f530 0x7f3c880ce730 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f3c78009830 tx=0x7f3c7802a480 comp rx=0 tx=0).stop 2024-09-21T03:40:38.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:38.884+0000 7f3c8d8e6640 1 -- 172.21.15.38:0/3721085442 shutdown_connections 2024-09-21T03:40:38.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:38.884+0000 7f3c8d8e6640 1 --2- 172.21.15.38:0/3721085442 >> v2:172.21.15.195:3300/0 conn(0x7f3c88073d50 0x7f3c88074160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:38.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:38.884+0000 7f3c8d8e6640 1 --2- 172.21.15.38:0/3721085442 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3c8806f530 0x7f3c880ce730 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:38.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:38.884+0000 7f3c8d8e6640 1 -- 172.21.15.38:0/3721085442 >> 172.21.15.38:0/3721085442 conn(0x7f3c880b2cc0 msgr2=0x7f3c880b30d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:38.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:38.884+0000 7f3c8d8e6640 1 -- 172.21.15.38:0/3721085442 shutdown_connections 2024-09-21T03:40:38.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:38.884+0000 7f3c8d8e6640 1 -- 172.21.15.38:0/3721085442 wait complete. 2024-09-21T03:40:38.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:38.885+0000 7f3c8d8e6640 1 Processor -- start 2024-09-21T03:40:38.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:38.886+0000 7f3c8d8e6640 1 -- start start 2024-09-21T03:40:38.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:38.887+0000 7f3c8d8e6640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3c8806f530 0x7f3c8815c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:38.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:38.887+0000 7f3c8d8e6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3c88073d50 0x7f3c88159750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:38.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:38.887+0000 7f3c8d8e6640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3c8815cc20 0x7f3c881e8360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:38.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:38.887+0000 7f3c8d8e6640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f3c8815e0a0 con 0x7f3c88073d50 2024-09-21T03:40:38.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:38.887+0000 7f3c8d8e6640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f3c8815df20 con 0x7f3c8806f530 2024-09-21T03:40:38.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:38.887+0000 7f3c8d8e6640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f3c8815e220 con 0x7f3c8815cc20 2024-09-21T03:40:38.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:38.887+0000 7f3c867fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3c88073d50 0x7f3c88159750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:38.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:38.887+0000 7f3c867fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3c88073d50 0x7f3c88159750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:45312/0 (socket says 172.21.15.38:45312) 2024-09-21T03:40:38.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:38.887+0000 7f3c867fc640 1 -- 172.21.15.38:0/61032284 learned_addr learned my addr 172.21.15.38:0/61032284 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:38.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:38.887+0000 7f3c877fe640 1 --2- 172.21.15.38:0/61032284 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3c8815cc20 0x7f3c881e8360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:38.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:38.887+0000 7f3c86ffd640 1 --2- 172.21.15.38:0/61032284 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3c8806f530 0x7f3c8815c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:38.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:38.887+0000 7f3c867fc640 1 -- 172.21.15.38:0/61032284 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3c8815cc20 msgr2=0x7f3c881e8360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:38.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:38.887+0000 7f3c867fc640 1 --2- 172.21.15.38:0/61032284 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3c8815cc20 0x7f3c881e8360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:38.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:38.887+0000 7f3c867fc640 1 -- 172.21.15.38:0/61032284 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3c8806f530 msgr2=0x7f3c8815c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:38.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:38.887+0000 7f3c867fc640 1 --2- 172.21.15.38:0/61032284 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3c8806f530 0x7f3c8815c0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:38.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:38.887+0000 7f3c867fc640 1 -- 172.21.15.38:0/61032284 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c88142b00 con 0x7f3c88073d50 2024-09-21T03:40:38.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:38.888+0000 7f3c867fc640 1 --2- 172.21.15.38:0/61032284 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3c88073d50 0x7f3c88159750 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f3c7c00e790 tx=0x7f3c7c00ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:38.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:38.888+0000 7f3c6ffff640 1 -- 172.21.15.38:0/61032284 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c7c00eea0 con 0x7f3c88073d50 2024-09-21T03:40:38.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:38.888+0000 7f3c6ffff640 1 -- 172.21.15.38:0/61032284 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3c7c00cdb0 con 0x7f3c88073d50 2024-09-21T03:40:38.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:38.888+0000 7f3c6ffff640 1 -- 172.21.15.38:0/61032284 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c7c010690 con 0x7f3c88073d50 2024-09-21T03:40:38.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:38.888+0000 7f3c8d8e6640 1 -- 172.21.15.38:0/61032284 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c88142de0 con 0x7f3c88073d50 2024-09-21T03:40:38.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:38.888+0000 7f3c8d8e6640 1 -- 172.21.15.38:0/61032284 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c881433a0 con 0x7f3c88073d50 2024-09-21T03:40:38.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:38.889+0000 7f3c8d8e6640 1 -- 172.21.15.38:0/61032284 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c8815e6d0 con 0x7f3c88073d50 2024-09-21T03:40:38.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:38.890+0000 7f3c6ffff640 1 -- 172.21.15.38:0/61032284 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f3c7c010830 con 0x7f3c88073d50 2024-09-21T03:40:38.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:38.890+0000 7f3c6ffff640 1 --2- 172.21.15.38:0/61032284 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3c74078b90 0x7f3c7407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:38.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:38.891+0000 7f3c6ffff640 1 -- 172.21.15.38:0/61032284 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 6648+0+0 (secure 0 0 0) 0x7f3c7c09c320 con 0x7f3c88073d50 2024-09-21T03:40:38.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:38.891+0000 7f3c86ffd640 1 --2- 172.21.15.38:0/61032284 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3c74078b90 0x7f3c7407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:38.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:38.891+0000 7f3c86ffd640 1 --2- 172.21.15.38:0/61032284 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3c74078b90 0x7f3c7407b0a0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f3c78034000 tx=0x7f3c78004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:38.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:38.894+0000 7f3c6ffff640 1 -- 172.21.15.38:0/61032284 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f3c7c064490 con 0x7f3c88073d50 2024-09-21T03:40:39.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:39.119+0000 7f3c8d8e6640 1 -- 172.21.15.38:0/61032284 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3c8815d440 con 0x7f3c88073d50 2024-09-21T03:40:39.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:39.120+0000 7f3c6ffff640 1 -- 172.21.15.38:0/61032284 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 105 v105) ==== 95+0+4388 (secure 0 0 0) 0x7f3c7c063d90 con 0x7f3c88073d50 2024-09-21T03:40:39.121 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:40:39.122 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":105,"btime":"2024-09-21T03:40:38:029932+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6086,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/965972663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":965972663},{"type":"v1","addr":"172.21.15.195:6836","nonce":965972663}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6700,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/3176122334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":3176122334},{"type":"v1","addr":"172.21.15.195:6837","nonce":3176122334}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":7020,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/751618334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":751618334},{"type":"v1","addr":"172.21.15.38:6833","nonce":751618334}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":105,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:40:02.319282+0000","modified":"2024-09-21T03:40:38.029836+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":100,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-21T03:40:39.122 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 105 2024-09-21T03:40:39.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:39.122+0000 7f3c8d8e6640 1 -- 172.21.15.38:0/61032284 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3c74078b90 msgr2=0x7f3c7407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:39.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:39.122+0000 7f3c8d8e6640 1 --2- 172.21.15.38:0/61032284 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3c74078b90 0x7f3c7407b0a0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f3c78034000 tx=0x7f3c78004730 comp rx=0 tx=0).stop 2024-09-21T03:40:39.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:39.122+0000 7f3c8d8e6640 1 -- 172.21.15.38:0/61032284 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3c88073d50 msgr2=0x7f3c88159750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:39.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:39.122+0000 7f3c8d8e6640 1 --2- 172.21.15.38:0/61032284 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3c88073d50 0x7f3c88159750 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f3c7c00e790 tx=0x7f3c7c00ec50 comp rx=0 tx=0).stop 2024-09-21T03:40:39.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:39.122+0000 7f3c8d8e6640 1 -- 172.21.15.38:0/61032284 shutdown_connections 2024-09-21T03:40:39.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:39.122+0000 7f3c8d8e6640 1 --2- 172.21.15.38:0/61032284 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3c74078b90 0x7f3c7407b0a0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:39.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:39.122+0000 7f3c8d8e6640 1 --2- 172.21.15.38:0/61032284 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3c8815cc20 0x7f3c881e8360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:39.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:39.122+0000 7f3c8d8e6640 1 --2- 172.21.15.38:0/61032284 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3c88073d50 0x7f3c88159750 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:39.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:39.122+0000 7f3c8d8e6640 1 --2- 172.21.15.38:0/61032284 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3c8806f530 0x7f3c8815c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:39.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:39.122+0000 7f3c8d8e6640 1 -- 172.21.15.38:0/61032284 >> 172.21.15.38:0/61032284 conn(0x7f3c880b2cc0 msgr2=0x7f3c881532a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:39.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:39.122+0000 7f3c8d8e6640 1 -- 172.21.15.38:0/61032284 shutdown_connections 2024-09-21T03:40:39.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:39.122+0000 7f3c8d8e6640 1 -- 172.21.15.38:0/61032284 wait complete. 2024-09-21T03:40:39.135 INFO:tasks.cephfs.test_damage:Result: Mutation 'Delete 1.00000000.inode' led to DAMAGED state 2024-09-21T03:40:39.136 INFO:tasks.cephfs.test_damage:Applying mutation 'Delete 200.00000000' 2024-09-21T03:40:39.136 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-21T03:40:39.136 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:40:39.152 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056795,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2354 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T03:40:39.152 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi038.front.sepia.ceph.com 2024-09-21T03:40:39.152 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:40:39.152 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T03:40:39.207 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T03:40:39.207 INFO:teuthology.orchestra.run.smithi038.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-09-21T03:40:39.207 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-21T03:40:39.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:39.363+0000 7f623d0de640 1 -- 172.21.15.38:0/3647748686 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f623815ee80 msgr2=0x7f62380d0170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:39.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:39.363+0000 7f623d0de640 1 --2- 172.21.15.38:0/3647748686 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f623815ee80 0x7f62380d0170 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f6228005fd0 tx=0x7f622802a820 comp rx=0 tx=0).stop 2024-09-21T03:40:39.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:39.363+0000 7f623d0de640 1 -- 172.21.15.38:0/3647748686 shutdown_connections 2024-09-21T03:40:39.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:39.363+0000 7f623d0de640 1 --2- 172.21.15.38:0/3647748686 >> v2:172.21.15.195:3300/0 conn(0x7f62380b9ad0 0x7f62380d0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:39.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:39.363+0000 7f623d0de640 1 --2- 172.21.15.38:0/3647748686 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f623815ee80 0x7f62380d0170 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:39.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:39.364+0000 7f623d0de640 1 -- 172.21.15.38:0/3647748686 >> 172.21.15.38:0/3647748686 conn(0x7f623809d830 msgr2=0x7f62380b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:39.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:39.364+0000 7f623d0de640 1 -- 172.21.15.38:0/3647748686 shutdown_connections 2024-09-21T03:40:39.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:39.364+0000 7f623d0de640 1 -- 172.21.15.38:0/3647748686 wait complete. 2024-09-21T03:40:39.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:39.366+0000 7f623d0de640 1 Processor -- start 2024-09-21T03:40:39.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:39.366+0000 7f623d0de640 1 -- start start 2024-09-21T03:40:39.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:39.367+0000 7f623d0de640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f62380b9ad0 0x7f6238158310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:39.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:39.367+0000 7f623d0de640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f62380d1120 0x7f62380cc3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:39.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:39.367+0000 7f623d0de640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f623815ee80 0x7f62380ce160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:39.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:39.367+0000 7f623d0de640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f62380b6700 con 0x7f623815ee80 2024-09-21T03:40:39.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:39.367+0000 7f623d0de640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f62380b6580 con 0x7f62380d1120 2024-09-21T03:40:39.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:39.367+0000 7f623d0de640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f62380b6880 con 0x7f62380b9ad0 2024-09-21T03:40:39.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:39.367+0000 7f623759e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f623815ee80 0x7f62380ce160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:39.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:39.367+0000 7f623759e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f623815ee80 0x7f62380ce160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:45326/0 (socket says 172.21.15.38:45326) 2024-09-21T03:40:39.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:39.367+0000 7f623759e640 1 -- 172.21.15.38:0/4042212421 learned_addr learned my addr 172.21.15.38:0/4042212421 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:39.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:39.367+0000 7f6236d9d640 1 --2- 172.21.15.38:0/4042212421 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f62380b9ad0 0x7f6238158310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:39.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:39.367+0000 7f623659c640 1 --2- 172.21.15.38:0/4042212421 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f62380d1120 0x7f62380cc3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:39.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:39.367+0000 7f623759e640 1 -- 172.21.15.38:0/4042212421 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f62380b9ad0 msgr2=0x7f6238158310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:39.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:39.367+0000 7f623759e640 1 --2- 172.21.15.38:0/4042212421 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f62380b9ad0 0x7f6238158310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:39.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:39.367+0000 7f623759e640 1 -- 172.21.15.38:0/4042212421 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f62380d1120 msgr2=0x7f62380cc3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:39.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:39.367+0000 7f623759e640 1 --2- 172.21.15.38:0/4042212421 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f62380d1120 0x7f62380cc3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:39.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:39.367+0000 7f623759e640 1 -- 172.21.15.38:0/4042212421 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6238158850 con 0x7f623815ee80 2024-09-21T03:40:39.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:39.368+0000 7f623759e640 1 --2- 172.21.15.38:0/4042212421 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f623815ee80 0x7f62380ce160 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f623000c3c0 tx=0x7f623000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:39.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:39.368+0000 7f622f7fe640 1 -- 172.21.15.38:0/4042212421 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6230006890 con 0x7f623815ee80 2024-09-21T03:40:39.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:39.368+0000 7f622f7fe640 1 -- 172.21.15.38:0/4042212421 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6230006b40 con 0x7f623815ee80 2024-09-21T03:40:39.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:39.368+0000 7f623d0de640 1 -- 172.21.15.38:0/4042212421 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62380a04e0 con 0x7f623815ee80 2024-09-21T03:40:39.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:39.368+0000 7f622f7fe640 1 -- 172.21.15.38:0/4042212421 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6230007020 con 0x7f623815ee80 2024-09-21T03:40:39.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:39.368+0000 7f623d0de640 1 -- 172.21.15.38:0/4042212421 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62380a0aa0 con 0x7f623815ee80 2024-09-21T03:40:39.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:39.369+0000 7f623d0de640 1 -- 172.21.15.38:0/4042212421 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62380b6d30 con 0x7f623815ee80 2024-09-21T03:40:39.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:39.370+0000 7f622f7fe640 1 -- 172.21.15.38:0/4042212421 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f6230019020 con 0x7f623815ee80 2024-09-21T03:40:39.374 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:39.370+0000 7f622f7fe640 1 --2- 172.21.15.38:0/4042212421 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6218078b90 0x7f621807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:39.374 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:39.370+0000 7f622f7fe640 1 -- 172.21.15.38:0/4042212421 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 6648+0+0 (secure 0 0 0) 0x7f623009a750 con 0x7f623815ee80 2024-09-21T03:40:39.374 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:39.371+0000 7f6236d9d640 1 --2- 172.21.15.38:0/4042212421 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6218078b90 0x7f621807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:39.374 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:39.371+0000 7f6236d9d640 1 --2- 172.21.15.38:0/4042212421 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6218078b90 0x7f621807b0a0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f6228034000 tx=0x7f6228004ab0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:39.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:39.375+0000 7f622f7fe640 1 -- 172.21.15.38:0/4042212421 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f62381592d0 con 0x7f623815ee80 2024-09-21T03:40:39.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:39.602+0000 7f623d0de640 1 -- 172.21.15.38:0/4042212421 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f62380d0b10 con 0x7f623815ee80 2024-09-21T03:40:40.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:40.029+0000 7f622f7fe640 1 -- 172.21.15.38:0/4042212421 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v106) ==== 161+0+0 (secure 0 0 0) 0x7f6230062190 con 0x7f623815ee80 2024-09-21T03:40:40.031 INFO:teuthology.orchestra.run.smithi038.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-21T03:40:40.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:40.031+0000 7f623d0de640 1 -- 172.21.15.38:0/4042212421 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6218078b90 msgr2=0x7f621807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:40.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:40.031+0000 7f623d0de640 1 --2- 172.21.15.38:0/4042212421 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6218078b90 0x7f621807b0a0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f6228034000 tx=0x7f6228004ab0 comp rx=0 tx=0).stop 2024-09-21T03:40:40.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:40.031+0000 7f623d0de640 1 -- 172.21.15.38:0/4042212421 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f623815ee80 msgr2=0x7f62380ce160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:40.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:40.031+0000 7f623d0de640 1 --2- 172.21.15.38:0/4042212421 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f623815ee80 0x7f62380ce160 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f623000c3c0 tx=0x7f623000c880 comp rx=0 tx=0).stop 2024-09-21T03:40:40.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:40.031+0000 7f623d0de640 1 -- 172.21.15.38:0/4042212421 shutdown_connections 2024-09-21T03:40:40.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:40.031+0000 7f623d0de640 1 --2- 172.21.15.38:0/4042212421 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f623815ee80 0x7f62380ce160 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:40.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:40.031+0000 7f623d0de640 1 --2- 172.21.15.38:0/4042212421 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f62380d1120 0x7f62380cc3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:40.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:40.031+0000 7f623d0de640 1 --2- 172.21.15.38:0/4042212421 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6218078b90 0x7f621807b0a0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:40.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:40.031+0000 7f623d0de640 1 --2- 172.21.15.38:0/4042212421 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f62380b9ad0 0x7f6238158310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:40.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:40.031+0000 7f623d0de640 1 -- 172.21.15.38:0/4042212421 >> 172.21.15.38:0/4042212421 conn(0x7f623809d830 msgr2=0x7f62380ceb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:40.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:40.032+0000 7f623d0de640 1 -- 172.21.15.38:0/4042212421 shutdown_connections 2024-09-21T03:40:40.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:40.032+0000 7f623d0de640 1 -- 172.21.15.38:0/4042212421 wait complete. 2024-09-21T03:40:40.046 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-09-21T03:40:40.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:40.164+0000 7fb6a4f62640 1 -- 172.21.15.38:0/1813557341 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb6a00b9ad0 msgr2=0x7fb6a00d0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:40.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:40.164+0000 7fb6a4f62640 1 --2- 172.21.15.38:0/1813557341 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb6a00b9ad0 0x7fb6a00d0be0 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7fb6880098e0 tx=0x7fb68802a490 comp rx=0 tx=0).stop 2024-09-21T03:40:40.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:40.164+0000 7fb6a4f62640 1 -- 172.21.15.38:0/1813557341 shutdown_connections 2024-09-21T03:40:40.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:40.164+0000 7fb6a4f62640 1 --2- 172.21.15.38:0/1813557341 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb6a00b9ad0 0x7fb6a00d0be0 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:40.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:40.164+0000 7fb6a4f62640 1 --2- 172.21.15.38:0/1813557341 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb6a015ee80 0x7fb6a00d0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:40.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:40.164+0000 7fb6a4f62640 1 -- 172.21.15.38:0/1813557341 >> 172.21.15.38:0/1813557341 conn(0x7fb6a009d830 msgr2=0x7fb6a00b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:40.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:40.164+0000 7fb6a4f62640 1 -- 172.21.15.38:0/1813557341 shutdown_connections 2024-09-21T03:40:40.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:40.164+0000 7fb6a4f62640 1 -- 172.21.15.38:0/1813557341 wait complete. 2024-09-21T03:40:40.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:40.166+0000 7fb6a4f62640 1 Processor -- start 2024-09-21T03:40:40.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:40.166+0000 7fb6a4f62640 1 -- start start 2024-09-21T03:40:40.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:40.167+0000 7fb6a4f62640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb6a00b9ad0 0x7fb6a014fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:40.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:40.168+0000 7fb6a4f62640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb6a00d1120 0x7fb6a00b2310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:40.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:40.168+0000 7fb6a4f62640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb6a015ee80 0x7fb6a00b2850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:40.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:40.168+0000 7fb6a4f62640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fb6a00b6700 con 0x7fb6a00b9ad0 2024-09-21T03:40:40.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:40.168+0000 7fb6a4f62640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fb6a00b6580 con 0x7fb6a015ee80 2024-09-21T03:40:40.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:40.168+0000 7fb6a4f62640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fb6a00b6880 con 0x7fb6a00d1120 2024-09-21T03:40:40.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:40.168+0000 7fb69e59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb6a00b9ad0 0x7fb6a014fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:40.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:40.168+0000 7fb69e59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb6a00b9ad0 0x7fb6a014fa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:45358/0 (socket says 172.21.15.38:45358) 2024-09-21T03:40:40.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:40.168+0000 7fb69e59c640 1 -- 172.21.15.38:0/354388158 learned_addr learned my addr 172.21.15.38:0/354388158 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:40.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:40.168+0000 7fb69ed9d640 1 --2- 172.21.15.38:0/354388158 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb6a015ee80 0x7fb6a00b2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:40.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:40.168+0000 7fb69dd9b640 1 --2- 172.21.15.38:0/354388158 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb6a00d1120 0x7fb6a00b2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:40.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:40.168+0000 7fb69e59c640 1 -- 172.21.15.38:0/354388158 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb6a00d1120 msgr2=0x7fb6a00b2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:40.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:40.168+0000 7fb69e59c640 1 --2- 172.21.15.38:0/354388158 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb6a00d1120 0x7fb6a00b2310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:40.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:40.168+0000 7fb69e59c640 1 -- 172.21.15.38:0/354388158 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb6a015ee80 msgr2=0x7fb6a00b2850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:40.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:40.168+0000 7fb69e59c640 1 --2- 172.21.15.38:0/354388158 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb6a015ee80 0x7fb6a00b2850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:40.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:40.168+0000 7fb69e59c640 1 -- 172.21.15.38:0/354388158 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6a006e8e0 con 0x7fb6a00b9ad0 2024-09-21T03:40:40.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:40.168+0000 7fb69e59c640 1 --2- 172.21.15.38:0/354388158 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb6a00b9ad0 0x7fb6a014fa90 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7fb69000e9f0 tx=0x7fb69000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:40.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:40.168+0000 7fb696ffd640 1 -- 172.21.15.38:0/354388158 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb690004d20 con 0x7fb6a00b9ad0 2024-09-21T03:40:40.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:40.168+0000 7fb696ffd640 1 -- 172.21.15.38:0/354388158 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb69000cb20 con 0x7fb6a00b9ad0 2024-09-21T03:40:40.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:40.168+0000 7fb696ffd640 1 -- 172.21.15.38:0/354388158 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb690005020 con 0x7fb6a00b9ad0 2024-09-21T03:40:40.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:40.168+0000 7fb6a4f62640 1 -- 172.21.15.38:0/354388158 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6a006ebc0 con 0x7fb6a00b9ad0 2024-09-21T03:40:40.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:40.168+0000 7fb6a4f62640 1 -- 172.21.15.38:0/354388158 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6a006f180 con 0x7fb6a00b9ad0 2024-09-21T03:40:40.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:40.169+0000 7fb6a4f62640 1 -- 172.21.15.38:0/354388158 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6a00b6d90 con 0x7fb6a00b9ad0 2024-09-21T03:40:40.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:40.170+0000 7fb696ffd640 1 -- 172.21.15.38:0/354388158 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb6900051c0 con 0x7fb6a00b9ad0 2024-09-21T03:40:40.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:40.171+0000 7fb696ffd640 1 --2- 172.21.15.38:0/354388158 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb680078b90 0x7fb68007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:40.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:40.171+0000 7fb696ffd640 1 -- 172.21.15.38:0/354388158 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 6648+0+0 (secure 0 0 0) 0x7fb69009be30 con 0x7fb6a00b9ad0 2024-09-21T03:40:40.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:40.171+0000 7fb69dd9b640 1 --2- 172.21.15.38:0/354388158 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb680078b90 0x7fb68007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:40.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:40.172+0000 7fb69dd9b640 1 --2- 172.21.15.38:0/354388158 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb680078b90 0x7fb68007b0a0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fb688034000 tx=0x7fb688004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:40.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:40.175+0000 7fb696ffd640 1 -- 172.21.15.38:0/354388158 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fb690016030 con 0x7fb6a00b9ad0 2024-09-21T03:40:40.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:40.441+0000 7fb6a4f62640 1 -- 172.21.15.38:0/354388158 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7fb6a00cf710 con 0x7fb6a00b9ad0 2024-09-21T03:40:41.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.035+0000 7fb696ffd640 1 -- 172.21.15.38:0/354388158 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v107) ==== 101+0+0 (secure 0 0 0) 0x7fb690063900 con 0x7fb6a00b9ad0 2024-09-21T03:40:41.037 INFO:teuthology.orchestra.run.smithi038.stderr:repaired: restoring rank 6:0 2024-09-21T03:40:41.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.038+0000 7fb6a4f62640 1 -- 172.21.15.38:0/354388158 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb680078b90 msgr2=0x7fb68007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:41.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.038+0000 7fb6a4f62640 1 --2- 172.21.15.38:0/354388158 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb680078b90 0x7fb68007b0a0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fb688034000 tx=0x7fb688004730 comp rx=0 tx=0).stop 2024-09-21T03:40:41.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.038+0000 7fb6a4f62640 1 -- 172.21.15.38:0/354388158 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb6a00b9ad0 msgr2=0x7fb6a014fa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:41.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.038+0000 7fb6a4f62640 1 --2- 172.21.15.38:0/354388158 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb6a00b9ad0 0x7fb6a014fa90 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7fb69000e9f0 tx=0x7fb69000eeb0 comp rx=0 tx=0).stop 2024-09-21T03:40:41.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.038+0000 7fb6a4f62640 1 -- 172.21.15.38:0/354388158 shutdown_connections 2024-09-21T03:40:41.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.038+0000 7fb6a4f62640 1 --2- 172.21.15.38:0/354388158 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb6a015ee80 0x7fb6a00b2850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:41.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.038+0000 7fb6a4f62640 1 --2- 172.21.15.38:0/354388158 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb680078b90 0x7fb68007b0a0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:41.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.038+0000 7fb6a4f62640 1 --2- 172.21.15.38:0/354388158 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb6a00d1120 0x7fb6a00b2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:41.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.038+0000 7fb6a4f62640 1 --2- 172.21.15.38:0/354388158 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb6a00b9ad0 0x7fb6a014fa90 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:41.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.038+0000 7fb6a4f62640 1 -- 172.21.15.38:0/354388158 >> 172.21.15.38:0/354388158 conn(0x7fb6a009d830 msgr2=0x7fb6a00d09e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:41.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.038+0000 7fb6a4f62640 1 -- 172.21.15.38:0/354388158 shutdown_connections 2024-09-21T03:40:41.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.038+0000 7fb6a4f62640 1 -- 172.21.15.38:0/354388158 wait complete. 2024-09-21T03:40:41.054 DEBUG:teuthology.orchestra.run.smithi038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-09-21T03:40:41.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.113+0000 7f804a2e3f40 1 Processor -- start 2024-09-21T03:40:41.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.113+0000 7f804a2e3f40 1 -- start start 2024-09-21T03:40:41.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.114+0000 7f804a2e3f40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558a954c0130 0x558a954bc040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:41.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.114+0000 7f804a2e3f40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558a954d73b0 0x558a9548f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:41.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.114+0000 7f804a2e3f40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x558a954d7dc0 0x558a9548da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:41.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.114+0000 7f804a2e3f40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x558a953ded60 con 0x558a954d73b0 2024-09-21T03:40:41.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.114+0000 7f804a2e3f40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x558a9543d430 con 0x558a954c0130 2024-09-21T03:40:41.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.114+0000 7f804a2e3f40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x558a95563ab0 con 0x558a954d7dc0 2024-09-21T03:40:41.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.114+0000 7f804359c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558a954d73b0 0x558a9548f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:41.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.114+0000 7f804359c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558a954d73b0 0x558a9548f280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:45376/0 (socket says 172.21.15.38:45376) 2024-09-21T03:40:41.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.114+0000 7f804359c640 1 -- 172.21.15.38:0/3439819290 learned_addr learned my addr 172.21.15.38:0/3439819290 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:41.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.114+0000 7f8043d9d640 1 --2- 172.21.15.38:0/3439819290 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558a954c0130 0x558a954bc040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:41.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.114+0000 7f804459e640 1 --2- 172.21.15.38:0/3439819290 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x558a954d7dc0 0x558a9548da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:41.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.115+0000 7f804359c640 1 -- 172.21.15.38:0/3439819290 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x558a954d7dc0 msgr2=0x558a9548da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:41.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.115+0000 7f804359c640 1 --2- 172.21.15.38:0/3439819290 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x558a954d7dc0 0x558a9548da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:41.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.115+0000 7f804359c640 1 -- 172.21.15.38:0/3439819290 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558a954c0130 msgr2=0x558a954bc040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:41.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.115+0000 7f804359c640 1 --2- 172.21.15.38:0/3439819290 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558a954c0130 0x558a954bc040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:41.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.115+0000 7f804359c640 1 -- 172.21.15.38:0/3439819290 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a954d0110 con 0x558a954d73b0 2024-09-21T03:40:41.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.115+0000 7f804459e640 1 --2- 172.21.15.38:0/3439819290 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x558a954d7dc0 0x558a9548da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:40:41.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.115+0000 7f804359c640 1 --2- 172.21.15.38:0/3439819290 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558a954d73b0 0x558a9548f280 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f8038001e10 tx=0x7f803800b650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acbec7c78f65e37 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:41.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.115+0000 7f804259a640 1 -- 172.21.15.38:0/3439819290 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8038016070 con 0x558a954d73b0 2024-09-21T03:40:41.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.115+0000 7f804259a640 1 -- 172.21.15.38:0/3439819290 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f803800d7c0 con 0x558a954d73b0 2024-09-21T03:40:41.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.115+0000 7f804259a640 1 -- 172.21.15.38:0/3439819290 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f803800dc10 con 0x558a954d73b0 2024-09-21T03:40:41.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.116+0000 7f804a2e3f40 1 -- 172.21.15.38:0/3439819290 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558a954d73b0 msgr2=0x558a9548f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:41.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.116+0000 7f804a2e3f40 1 --2- 172.21.15.38:0/3439819290 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558a954d73b0 0x558a9548f280 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f8038001e10 tx=0x7f803800b650 comp rx=0 tx=0).stop 2024-09-21T03:40:41.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.116+0000 7f804a2e3f40 1 -- 172.21.15.38:0/3439819290 shutdown_connections 2024-09-21T03:40:41.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.116+0000 7f804a2e3f40 1 --2- 172.21.15.38:0/3439819290 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x558a954d7dc0 0x558a9548da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:41.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.116+0000 7f804a2e3f40 1 --2- 172.21.15.38:0/3439819290 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558a954d73b0 0x558a9548f280 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:41.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.116+0000 7f804a2e3f40 1 --2- 172.21.15.38:0/3439819290 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558a954c0130 0x558a954bc040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:41.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.116+0000 7f804a2e3f40 1 -- 172.21.15.38:0/3439819290 >> 172.21.15.38:0/3439819290 conn(0x558a953afac0 msgr2=0x558a95566700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:41.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.116+0000 7f804a2e3f40 1 -- 172.21.15.38:0/3439819290 shutdown_connections 2024-09-21T03:40:41.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.116+0000 7f804a2e3f40 1 -- 172.21.15.38:0/3439819290 wait complete. 2024-09-21T03:40:41.123 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.117+0000 7f804a2e3f40 1 Processor -- start 2024-09-21T03:40:41.123 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.117+0000 7f804a2e3f40 1 -- start start 2024-09-21T03:40:41.123 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.118+0000 7f804a2e3f40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558a954c0130 0x558a95547350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:41.123 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.118+0000 7f804a2e3f40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558a954d73b0 0x558a955f0800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:41.123 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.118+0000 7f804a2e3f40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x558a954d7dc0 0x558a955f0d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:41.123 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.118+0000 7f804a2e3f40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x558a95566f40 con 0x558a954d73b0 2024-09-21T03:40:41.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.118+0000 7f804a2e3f40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x558a954cfb30 con 0x558a954c0130 2024-09-21T03:40:41.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.118+0000 7f804a2e3f40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x558a954d54b0 con 0x558a954d7dc0 2024-09-21T03:40:41.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.118+0000 7f804359c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558a954d73b0 0x558a955f0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:41.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.118+0000 7f804359c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558a954d73b0 0x558a955f0800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:45380/0 (socket says 172.21.15.38:45380) 2024-09-21T03:40:41.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.118+0000 7f804359c640 1 -- 172.21.15.38:0/1243523184 learned_addr learned my addr 172.21.15.38:0/1243523184 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:41.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.118+0000 7f804459e640 1 --2- 172.21.15.38:0/1243523184 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x558a954d7dc0 0x558a955f0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:41.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.118+0000 7f8043d9d640 1 --2- 172.21.15.38:0/1243523184 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558a954c0130 0x558a95547350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:41.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.118+0000 7f804359c640 1 -- 172.21.15.38:0/1243523184 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x558a954d7dc0 msgr2=0x558a955f0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:41.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.118+0000 7f804359c640 1 --2- 172.21.15.38:0/1243523184 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x558a954d7dc0 0x558a955f0d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:41.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.118+0000 7f804359c640 1 -- 172.21.15.38:0/1243523184 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558a954c0130 msgr2=0x558a95547350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:41.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.118+0000 7f804359c640 1 --2- 172.21.15.38:0/1243523184 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558a954c0130 0x558a95547350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:41.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.118+0000 7f804359c640 1 -- 172.21.15.38:0/1243523184 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a9554b8d0 con 0x558a954d73b0 2024-09-21T03:40:41.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.118+0000 7f8043d9d640 1 --2- 172.21.15.38:0/1243523184 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558a954c0130 0x558a95547350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:40:41.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.119+0000 7f804359c640 1 --2- 172.21.15.38:0/1243523184 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558a954d73b0 0x558a955f0800 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f8038000c00 tx=0x7f8038001330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:41.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.119+0000 7f8041598640 1 -- 172.21.15.38:0/1243523184 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8038022070 con 0x558a954d73b0 2024-09-21T03:40:41.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.119+0000 7f8041598640 1 -- 172.21.15.38:0/1243523184 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8038017040 con 0x558a954d73b0 2024-09-21T03:40:41.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.119+0000 7f8041598640 1 -- 172.21.15.38:0/1243523184 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8038016070 con 0x558a954d73b0 2024-09-21T03:40:41.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.119+0000 7f804a2e3f40 1 -- 172.21.15.38:0/1243523184 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x558a95547c20 con 0x558a954d73b0 2024-09-21T03:40:41.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.119+0000 7f804a2e3f40 1 -- 172.21.15.38:0/1243523184 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a955fdc80 con 0x558a954d73b0 2024-09-21T03:40:41.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.121+0000 7f8041598640 1 -- 172.21.15.38:0/1243523184 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f8038025070 con 0x558a954d73b0 2024-09-21T03:40:41.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.121+0000 7f8041598640 1 --2- 172.21.15.38:0/1243523184 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8030077570 0x7f8030079a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:41.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.121+0000 7f8041598640 1 -- 172.21.15.38:0/1243523184 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 6648+0+0 (secure 0 0 0) 0x7f803809a960 con 0x558a954d73b0 2024-09-21T03:40:41.126 INFO:teuthology.orchestra.run.smithi038.stdout:Importing pool 2024-09-21T03:40:41.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.121+0000 7f8043d9d640 1 --2- 172.21.15.38:0/1243523184 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8030077570 0x7f8030079a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:41.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.121+0000 7f804a2e3f40 1 --2- 172.21.15.38:0/1243523184 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x558a95607ae0 0x558a95609f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:41.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.121+0000 7f804a2e3f40 1 -- 172.21.15.38:0/1243523184 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x558a9548dcc0 con 0x558a95607ae0 2024-09-21T03:40:41.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.122+0000 7f804459e640 1 --2- 172.21.15.38:0/1243523184 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x558a95607ae0 0x558a95609f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:41.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.122+0000 7f8043d9d640 1 --2- 172.21.15.38:0/1243523184 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8030077570 0x7f8030079a80 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f8034004230 tx=0x7f8034002040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:41.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.122+0000 7f804459e640 1 --2- 172.21.15.38:0/1243523184 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x558a95607ae0 0x558a95609f00 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:41.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.124+0000 7f804459e640 1 -- 172.21.15.38:0/1243523184 <== osd.7 v2:172.21.15.195:6818/1606106342 1 ==== osd_op_reply(1 601.00000000 [create] v100'102 uv78 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f803c00b040 con 0x558a95607ae0 2024-09-21T03:40:41.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.124+0000 7f804a2e3f40 1 -- 172.21.15.38:0/1243523184 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e100) -- 0x558a95566a00 con 0x558a95607ae0 2024-09-21T03:40:41.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.125+0000 7f804459e640 1 -- 172.21.15.38:0/1243523184 <== osd.7 v2:172.21.15.195:6818/1606106342 2 ==== osd_op_reply(2 601.00000000 [delete] v100'103 uv103 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f803c00b040 con 0x558a95607ae0 2024-09-21T03:40:41.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.125+0000 7f804a2e3f40 1 -- 172.21.15.38:0/1243523184 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x558a954d4dd0 con 0x558a95607ae0 2024-09-21T03:40:41.128 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-09-21T03:40:41.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.127+0000 7f804459e640 1 -- 172.21.15.38:0/1243523184 <== osd.7 v2:172.21.15.195:6818/1606106342 3 ==== osd_op_reply(3 601.00000000 [create] v100'104 uv104 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f803c00b040 con 0x558a95607ae0 2024-09-21T03:40:41.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.127+0000 7f804a2e3f40 1 -- 172.21.15.38:0/1243523184 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x558a9560b450 con 0x558a95607ae0 2024-09-21T03:40:41.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.128+0000 7f804459e640 1 -- 172.21.15.38:0/1243523184 <== osd.7 v2:172.21.15.195:6818/1606106342 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v100'105 uv105 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f803c00b040 con 0x558a95607ae0 2024-09-21T03:40:41.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.128+0000 7f804a2e3f40 1 -- 172.21.15.38:0/1243523184 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x558a9560b7f0 con 0x558a95607ae0 2024-09-21T03:40:41.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.129+0000 7f804459e640 1 -- 172.21.15.38:0/1243523184 <== osd.7 v2:172.21.15.195:6818/1606106342 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v100'106 uv106 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f803c00b040 con 0x558a95607ae0 2024-09-21T03:40:41.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.130+0000 7f804a2e3f40 1 -- 172.21.15.38:0/1243523184 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x558a9560a440 con 0x558a95607ae0 2024-09-21T03:40:41.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.131+0000 7f804459e640 1 -- 172.21.15.38:0/1243523184 <== osd.7 v2:172.21.15.195:6818/1606106342 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v100'107 uv107 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f803c00b040 con 0x558a95607ae0 2024-09-21T03:40:41.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.131+0000 7f804a2e3f40 1 -- 172.21.15.38:0/1243523184 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x558a9560a7e0 con 0x558a95607ae0 2024-09-21T03:40:41.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.132+0000 7f804459e640 1 -- 172.21.15.38:0/1243523184 <== osd.7 v2:172.21.15.195:6818/1606106342 7 ==== osd_op_reply(7 602.00000000 [create] v100'108 uv84 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f803c00b040 con 0x558a95607ae0 2024-09-21T03:40:41.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.132+0000 7f804a2e3f40 1 -- 172.21.15.38:0/1243523184 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e100) -- 0x558a9560abd0 con 0x558a95607ae0 2024-09-21T03:40:41.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.134+0000 7f804459e640 1 -- 172.21.15.38:0/1243523184 <== osd.7 v2:172.21.15.195:6818/1606106342 8 ==== osd_op_reply(8 602.00000000 [delete] v100'109 uv109 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f803c00b040 con 0x558a95607ae0 2024-09-21T03:40:41.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.134+0000 7f804a2e3f40 1 -- 172.21.15.38:0/1243523184 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x558a9560afc0 con 0x558a95607ae0 2024-09-21T03:40:41.137 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-09-21T03:40:41.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.136+0000 7f804459e640 1 -- 172.21.15.38:0/1243523184 <== osd.7 v2:172.21.15.195:6818/1606106342 9 ==== osd_op_reply(9 602.00000000 [create] v100'110 uv110 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f803c00b040 con 0x558a95607ae0 2024-09-21T03:40:41.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.136+0000 7f804a2e3f40 1 -- 172.21.15.38:0/1243523184 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x558a9560afc0 con 0x558a95607ae0 2024-09-21T03:40:41.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.137+0000 7f804459e640 1 -- 172.21.15.38:0/1243523184 <== osd.7 v2:172.21.15.195:6818/1606106342 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v100'111 uv111 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f803c00b040 con 0x558a95607ae0 2024-09-21T03:40:41.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.137+0000 7f804a2e3f40 1 -- 172.21.15.38:0/1243523184 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x558a9560af70 con 0x558a95607ae0 2024-09-21T03:40:41.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.138+0000 7f804459e640 1 -- 172.21.15.38:0/1243523184 <== osd.7 v2:172.21.15.195:6818/1606106342 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v100'112 uv112 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f803c00b040 con 0x558a95607ae0 2024-09-21T03:40:41.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.139+0000 7f804a2e3f40 1 -- 172.21.15.38:0/1243523184 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x558a9560af70 con 0x558a95607ae0 2024-09-21T03:40:41.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.140+0000 7f804459e640 1 -- 172.21.15.38:0/1243523184 <== osd.7 v2:172.21.15.195:6818/1606106342 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v100'113 uv113 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f803c00b040 con 0x558a95607ae0 2024-09-21T03:40:41.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.140+0000 7f804a2e3f40 1 -- 172.21.15.38:0/1243523184 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:13 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x558a9560af70 con 0x558a95607ae0 2024-09-21T03:40:41.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.141+0000 7f804459e640 1 -- 172.21.15.38:0/1243523184 <== osd.7 v2:172.21.15.195:6818/1606106342 13 ==== osd_op_reply(13 10000000000.00000000 [create] v100'114 uv91 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7f803c00b040 con 0x558a95607ae0 2024-09-21T03:40:41.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.141+0000 7f804a2e3f40 1 -- 172.21.15.38:0/1243523184 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:14 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e100) -- 0x558a9560af70 con 0x558a95607ae0 2024-09-21T03:40:41.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.143+0000 7f804459e640 1 -- 172.21.15.38:0/1243523184 <== osd.7 v2:172.21.15.195:6818/1606106342 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v100'115 uv115 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f803c00b040 con 0x558a95607ae0 2024-09-21T03:40:41.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.143+0000 7f804a2e3f40 1 -- 172.21.15.38:0/1243523184 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:15 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x558a9560af70 con 0x558a95607ae0 2024-09-21T03:40:41.146 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-09-21T03:40:41.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.144+0000 7f804459e640 1 -- 172.21.15.38:0/1243523184 <== osd.7 v2:172.21.15.195:6818/1606106342 15 ==== osd_op_reply(15 10000000000.00000000 [create] v100'116 uv116 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f803c00b040 con 0x558a95607ae0 2024-09-21T03:40:41.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.144+0000 7f804a2e3f40 1 -- 172.21.15.38:0/1243523184 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:16 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x558a9560af70 con 0x558a95607ae0 2024-09-21T03:40:41.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.146+0000 7f804459e640 1 -- 172.21.15.38:0/1243523184 <== osd.7 v2:172.21.15.195:6818/1606106342 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v100'117 uv117 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f803c00b040 con 0x558a95607ae0 2024-09-21T03:40:41.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.146+0000 7f804a2e3f40 1 -- 172.21.15.38:0/1243523184 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:17 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x558a9560af70 con 0x558a95607ae0 2024-09-21T03:40:41.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.148+0000 7f804459e640 1 -- 172.21.15.38:0/1243523184 <== osd.7 v2:172.21.15.195:6818/1606106342 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v100'118 uv118 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f803c00b040 con 0x558a95607ae0 2024-09-21T03:40:41.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.148+0000 7f804a2e3f40 1 -- 172.21.15.38:0/1243523184 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:18 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x558a9560af70 con 0x558a95607ae0 2024-09-21T03:40:41.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.149+0000 7f804459e640 1 -- 172.21.15.38:0/1243523184 <== osd.7 v2:172.21.15.195:6818/1606106342 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v100'119 uv119 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f803c00b040 con 0x558a95607ae0 2024-09-21T03:40:41.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.149+0000 7f804a2e3f40 1 -- 172.21.15.38:0/1243523184 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:19 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=585b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x558a9560c040 con 0x558a95607ae0 2024-09-21T03:40:41.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.150+0000 7f804459e640 1 -- 172.21.15.38:0/1243523184 <== osd.7 v2:172.21.15.195:6818/1606106342 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v100'120 uv120 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f803c00b040 con 0x558a95607ae0 2024-09-21T03:40:41.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.150+0000 7f804a2e3f40 1 -- 172.21.15.38:0/1243523184 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:20 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x558a9560c040 con 0x558a95607ae0 2024-09-21T03:40:41.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.152+0000 7f804459e640 1 -- 172.21.15.38:0/1243523184 <== osd.7 v2:172.21.15.195:6818/1606106342 20 ==== osd_op_reply(20 600.00000000 [create] v100'121 uv95 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f803c00b040 con 0x558a95607ae0 2024-09-21T03:40:41.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.152+0000 7f804a2e3f40 1 -- 172.21.15.38:0/1243523184 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:21 13.10 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e100) -- 0x558a9560c040 con 0x558a95607ae0 2024-09-21T03:40:41.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.153+0000 7f804459e640 1 -- 172.21.15.38:0/1243523184 <== osd.7 v2:172.21.15.195:6818/1606106342 21 ==== osd_op_reply(21 600.00000000 [delete] v100'122 uv122 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f803c00b040 con 0x558a95607ae0 2024-09-21T03:40:41.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.153+0000 7f804a2e3f40 1 -- 172.21.15.38:0/1243523184 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:22 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x558a9560c040 con 0x558a95607ae0 2024-09-21T03:40:41.157 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-09-21T03:40:41.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.155+0000 7f804459e640 1 -- 172.21.15.38:0/1243523184 <== osd.7 v2:172.21.15.195:6818/1606106342 22 ==== osd_op_reply(22 600.00000000 [create] v100'123 uv123 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f803c00b040 con 0x558a95607ae0 2024-09-21T03:40:41.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.155+0000 7f804a2e3f40 1 -- 172.21.15.38:0/1243523184 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:23 13.10 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x558a9560d050 con 0x558a95607ae0 2024-09-21T03:40:41.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.156+0000 7f804459e640 1 -- 172.21.15.38:0/1243523184 <== osd.7 v2:172.21.15.195:6818/1606106342 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v100'124 uv124 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f803c00b040 con 0x558a95607ae0 2024-09-21T03:40:41.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.156+0000 7f804a2e3f40 1 -- 172.21.15.38:0/1243523184 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:24 13.10 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x558a9560d050 con 0x558a95607ae0 2024-09-21T03:40:41.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.157+0000 7f804459e640 1 -- 172.21.15.38:0/1243523184 <== osd.7 v2:172.21.15.195:6818/1606106342 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v100'125 uv125 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f803c00b040 con 0x558a95607ae0 2024-09-21T03:40:41.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.157+0000 7f804a2e3f40 1 -- 172.21.15.38:0/1243523184 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:25 13.10 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x558a9560c040 con 0x558a95607ae0 2024-09-21T03:40:41.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.158+0000 7f804459e640 1 -- 172.21.15.38:0/1243523184 <== osd.7 v2:172.21.15.195:6818/1606106342 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v100'126 uv126 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f803c00b040 con 0x558a95607ae0 2024-09-21T03:40:41.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.158+0000 7f804a2e3f40 1 -- 172.21.15.38:0/1243523184 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:26 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x558a9560c040 con 0x558a95607ae0 2024-09-21T03:40:41.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.159+0000 7f804459e640 1 -- 172.21.15.38:0/1243523184 <== osd.7 v2:172.21.15.195:6818/1606106342 26 ==== osd_op_reply(26 603.00000000 [create] v100'127 uv101 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f803c00b040 con 0x558a95607ae0 2024-09-21T03:40:41.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.160+0000 7f804a2e3f40 1 -- 172.21.15.38:0/1243523184 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:27 13.10 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e100) -- 0x558a9560c040 con 0x558a95607ae0 2024-09-21T03:40:41.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.161+0000 7f804459e640 1 -- 172.21.15.38:0/1243523184 <== osd.7 v2:172.21.15.195:6818/1606106342 27 ==== osd_op_reply(27 603.00000000 [delete] v100'128 uv128 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f803c00b040 con 0x558a95607ae0 2024-09-21T03:40:41.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.161+0000 7f804a2e3f40 1 -- 172.21.15.38:0/1243523184 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:28 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x558a9560c040 con 0x558a95607ae0 2024-09-21T03:40:41.165 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-09-21T03:40:41.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.162+0000 7f804459e640 1 -- 172.21.15.38:0/1243523184 <== osd.7 v2:172.21.15.195:6818/1606106342 28 ==== osd_op_reply(28 603.00000000 [create] v100'129 uv129 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f803c00b040 con 0x558a95607ae0 2024-09-21T03:40:41.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.162+0000 7f804a2e3f40 1 -- 172.21.15.38:0/1243523184 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:29 13.10 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x558a9560d050 con 0x558a95607ae0 2024-09-21T03:40:41.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.163+0000 7f804459e640 1 -- 172.21.15.38:0/1243523184 <== osd.7 v2:172.21.15.195:6818/1606106342 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v100'130 uv130 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f803c00b040 con 0x558a95607ae0 2024-09-21T03:40:41.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.164+0000 7f804a2e3f40 1 -- 172.21.15.38:0/1243523184 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:30 13.10 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x558a9560d050 con 0x558a95607ae0 2024-09-21T03:40:41.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.165+0000 7f804459e640 1 -- 172.21.15.38:0/1243523184 <== osd.7 v2:172.21.15.195:6818/1606106342 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v100'131 uv131 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f803c00b040 con 0x558a95607ae0 2024-09-21T03:40:41.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.165+0000 7f804a2e3f40 1 -- 172.21.15.38:0/1243523184 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:31 13.10 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x558a9560c040 con 0x558a95607ae0 2024-09-21T03:40:41.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.166+0000 7f804459e640 1 -- 172.21.15.38:0/1243523184 <== osd.7 v2:172.21.15.195:6818/1606106342 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v100'132 uv132 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f803c00b040 con 0x558a95607ae0 2024-09-21T03:40:41.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.166+0000 7f804a2e3f40 1 --2- 172.21.15.38:0/1243523184 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x558a9560c550 0x558a9560e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:41.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.166+0000 7f804a2e3f40 1 -- 172.21.15.38:0/1243523184 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x558a9560eeb0 con 0x558a9560c550 2024-09-21T03:40:41.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.166+0000 7f8043d9d640 1 --2- 172.21.15.38:0/1243523184 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x558a9560c550 0x558a9560e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:41.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.167+0000 7f8043d9d640 1 --2- 172.21.15.38:0/1243523184 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x558a9560c550 0x558a9560e970 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:41.170 INFO:teuthology.orchestra.run.smithi038.stdout:Write #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-09-21T03:40:41.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.168+0000 7f8043d9d640 1 -- 172.21.15.38:0/1243523184 <== osd.6 v2:172.21.15.195:6826/4055825675 1 ==== osd_op_reply(32 1.00000000.inode [create] v100'88 uv88 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f803400a990 con 0x558a9560c550 2024-09-21T03:40:41.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.168+0000 7f804a2e3f40 1 -- 172.21.15.38:0/1243523184 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x558a9560f700 con 0x558a9560c550 2024-09-21T03:40:41.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.170+0000 7f8043d9d640 1 -- 172.21.15.38:0/1243523184 <== osd.6 v2:172.21.15.195:6826/4055825675 2 ==== osd_op_reply(33 1.00000000.inode [write 0~582] v100'89 uv89 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f803400a990 con 0x558a9560c550 2024-09-21T03:40:41.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.170+0000 7f804a2e3f40 1 -- 172.21.15.38:0/1243523184 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x558a9560faa0 con 0x558a9560c550 2024-09-21T03:40:41.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.171+0000 7f8043d9d640 1 -- 172.21.15.38:0/1243523184 <== osd.6 v2:172.21.15.195:6826/4055825675 3 ==== osd_op_reply(34 1.00000000.inode [omap-set-header] v100'90 uv90 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f803400a990 con 0x558a9560c550 2024-09-21T03:40:41.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.171+0000 7f804a2e3f40 1 -- 172.21.15.38:0/1243523184 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:35 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x558a9560fe40 con 0x558a9560c550 2024-09-21T03:40:41.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.172+0000 7f8043d9d640 1 -- 172.21.15.38:0/1243523184 <== osd.6 v2:172.21.15.195:6826/4055825675 4 ==== osd_op_reply(35 200.00000000 [create] v100'91 uv75 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f803400a990 con 0x558a9560c550 2024-09-21T03:40:41.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.172+0000 7f804a2e3f40 1 -- 172.21.15.38:0/1243523184 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:36 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e100) -- 0x558a956101e0 con 0x558a9560c550 2024-09-21T03:40:41.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.174+0000 7f8043d9d640 1 -- 172.21.15.38:0/1243523184 <== osd.6 v2:172.21.15.195:6826/4055825675 5 ==== osd_op_reply(36 200.00000000 [delete] v100'92 uv92 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f803400a990 con 0x558a9560c550 2024-09-21T03:40:41.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.174+0000 7f804a2e3f40 1 -- 172.21.15.38:0/1243523184 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x558a95610580 con 0x558a9560c550 2024-09-21T03:40:41.178 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-09-21T03:40:41.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.175+0000 7f8043d9d640 1 -- 172.21.15.38:0/1243523184 <== osd.6 v2:172.21.15.195:6826/4055825675 6 ==== osd_op_reply(37 200.00000000 [create] v100'93 uv93 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f803400a990 con 0x558a9560c550 2024-09-21T03:40:41.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.176+0000 7f804a2e3f40 1 -- 172.21.15.38:0/1243523184 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x558a95610920 con 0x558a9560c550 2024-09-21T03:40:41.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.177+0000 7f8043d9d640 1 -- 172.21.15.38:0/1243523184 <== osd.6 v2:172.21.15.195:6826/4055825675 7 ==== osd_op_reply(38 200.00000000 [write 0~90] v100'94 uv94 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f803400a990 con 0x558a9560c550 2024-09-21T03:40:41.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.177+0000 7f804a2e3f40 1 -- 172.21.15.38:0/1243523184 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x558a95610cc0 con 0x558a9560c550 2024-09-21T03:40:41.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.178+0000 7f8043d9d640 1 -- 172.21.15.38:0/1243523184 <== osd.6 v2:172.21.15.195:6826/4055825675 8 ==== osd_op_reply(39 200.00000000 [omap-set-header] v100'95 uv95 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f803400a990 con 0x558a9560c550 2024-09-21T03:40:41.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.179+0000 7f804a2e3f40 1 -- 172.21.15.38:0/1243523184 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:40 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x558a95610cc0 con 0x558a9560c550 2024-09-21T03:40:41.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.180+0000 7f8043d9d640 1 -- 172.21.15.38:0/1243523184 <== osd.6 v2:172.21.15.195:6826/4055825675 9 ==== osd_op_reply(40 200.00000001 [create] v100'96 uv80 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f803400a990 con 0x558a9560c550 2024-09-21T03:40:41.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.180+0000 7f804a2e3f40 1 -- 172.21.15.38:0/1243523184 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:41 13.14 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e100) -- 0x558a95610cc0 con 0x558a9560c550 2024-09-21T03:40:41.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.181+0000 7f8043d9d640 1 -- 172.21.15.38:0/1243523184 <== osd.6 v2:172.21.15.195:6826/4055825675 10 ==== osd_op_reply(41 200.00000001 [delete] v100'97 uv97 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f803400a990 con 0x558a9560c550 2024-09-21T03:40:41.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.182+0000 7f804a2e3f40 1 -- 172.21.15.38:0/1243523184 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:42 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x558a95610cc0 con 0x558a9560c550 2024-09-21T03:40:41.189 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-09-21T03:40:41.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.183+0000 7f8043d9d640 1 -- 172.21.15.38:0/1243523184 <== osd.6 v2:172.21.15.195:6826/4055825675 11 ==== osd_op_reply(42 200.00000001 [create] v100'98 uv98 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f803400a990 con 0x558a9560c550 2024-09-21T03:40:41.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.183+0000 7f804a2e3f40 1 -- 172.21.15.38:0/1243523184 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:43 13.14 13:2e2fa760:::200.00000001:head [write 0~17490 in=17490b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x558a95615140 con 0x558a9560c550 2024-09-21T03:40:41.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.185+0000 7f8043d9d640 1 -- 172.21.15.38:0/1243523184 <== osd.6 v2:172.21.15.195:6826/4055825675 12 ==== osd_op_reply(43 200.00000001 [write 0~17490] v100'99 uv99 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f803400a990 con 0x558a9560c550 2024-09-21T03:40:41.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.185+0000 7f804a2e3f40 1 -- 172.21.15.38:0/1243523184 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:44 13.14 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x558a95615140 con 0x558a9560c550 2024-09-21T03:40:41.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.186+0000 7f8043d9d640 1 -- 172.21.15.38:0/1243523184 <== osd.6 v2:172.21.15.195:6826/4055825675 13 ==== osd_op_reply(44 200.00000001 [omap-set-header] v100'100 uv100 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f803400a990 con 0x558a9560c550 2024-09-21T03:40:41.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.186+0000 7f804a2e3f40 1 -- 172.21.15.38:0/1243523184 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:45 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x558a95610cc0 con 0x558a9560c550 2024-09-21T03:40:41.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.188+0000 7f8043d9d640 1 -- 172.21.15.38:0/1243523184 <== osd.6 v2:172.21.15.195:6826/4055825675 14 ==== osd_op_reply(45 606.00000000 [create] v100'101 uv86 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f803400a990 con 0x558a9560c550 2024-09-21T03:40:41.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.188+0000 7f804a2e3f40 1 -- 172.21.15.38:0/1243523184 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:46 13.14 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e100) -- 0x558a95610cc0 con 0x558a9560c550 2024-09-21T03:40:41.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.189+0000 7f8043d9d640 1 -- 172.21.15.38:0/1243523184 <== osd.6 v2:172.21.15.195:6826/4055825675 15 ==== osd_op_reply(46 606.00000000 [delete] v100'102 uv102 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f803400a990 con 0x558a9560c550 2024-09-21T03:40:41.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.189+0000 7f804a2e3f40 1 -- 172.21.15.38:0/1243523184 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:47 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x558a95610cc0 con 0x558a9560c550 2024-09-21T03:40:41.200 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-09-21T03:40:41.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.191+0000 7f8043d9d640 1 -- 172.21.15.38:0/1243523184 <== osd.6 v2:172.21.15.195:6826/4055825675 16 ==== osd_op_reply(47 606.00000000 [create] v100'103 uv103 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f803400a990 con 0x558a9560c550 2024-09-21T03:40:41.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.191+0000 7f804a2e3f40 1 -- 172.21.15.38:0/1243523184 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:48 13.14 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x558a95611cd0 con 0x558a9560c550 2024-09-21T03:40:41.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.192+0000 7f8043d9d640 1 -- 172.21.15.38:0/1243523184 <== osd.6 v2:172.21.15.195:6826/4055825675 17 ==== osd_op_reply(48 606.00000000 [setxattr (30)] v100'104 uv104 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f803400a990 con 0x558a9560c550 2024-09-21T03:40:41.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.192+0000 7f804a2e3f40 1 -- 172.21.15.38:0/1243523184 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:49 13.14 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x558a95611cd0 con 0x558a9560c550 2024-09-21T03:40:41.202 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.194+0000 7f8043d9d640 1 -- 172.21.15.38:0/1243523184 <== osd.6 v2:172.21.15.195:6826/4055825675 18 ==== osd_op_reply(49 606.00000000 [setxattr (62)] v100'105 uv105 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f803400a990 con 0x558a9560c550 2024-09-21T03:40:41.202 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.194+0000 7f804a2e3f40 1 -- 172.21.15.38:0/1243523184 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:50 13.14 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x558a95610cc0 con 0x558a9560c550 2024-09-21T03:40:41.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.195+0000 7f8043d9d640 1 -- 172.21.15.38:0/1243523184 <== osd.6 v2:172.21.15.195:6826/4055825675 19 ==== osd_op_reply(50 606.00000000 [omap-set-header] v100'106 uv106 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f803400a990 con 0x558a9560c550 2024-09-21T03:40:41.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.195+0000 7f804a2e3f40 1 --2- 172.21.15.38:0/1243523184 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x558a956111d0 0x558a956135f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:41.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.195+0000 7f804a2e3f40 1 -- 172.21.15.38:0/1243523184 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:51 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x558a95613b30 con 0x558a956111d0 2024-09-21T03:40:41.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.195+0000 7f804359c640 1 --2- 172.21.15.38:0/1243523184 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x558a956111d0 0x558a956135f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:41.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.196+0000 7f804359c640 1 --2- 172.21.15.38:0/1243523184 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x558a956111d0 0x558a956135f0 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:41.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.197+0000 7f804359c640 1 -- 172.21.15.38:0/1243523184 <== osd.0 v2:172.21.15.38:6800/2381639433 1 ==== osd_op_reply(51 607.00000000 [create] v100'21 uv20 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f8038024030 con 0x558a956111d0 2024-09-21T03:40:41.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.197+0000 7f804a2e3f40 1 -- 172.21.15.38:0/1243523184 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:52 13.1c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e100) -- 0x558a95613ed0 con 0x558a956111d0 2024-09-21T03:40:41.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.198+0000 7f804359c640 1 -- 172.21.15.38:0/1243523184 <== osd.0 v2:172.21.15.38:6800/2381639433 2 ==== osd_op_reply(52 607.00000000 [delete] v100'22 uv22 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8038024030 con 0x558a956111d0 2024-09-21T03:40:41.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.198+0000 7f804a2e3f40 1 -- 172.21.15.38:0/1243523184 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:53 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x558a95614270 con 0x558a956111d0 2024-09-21T03:40:41.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.199+0000 7f804359c640 1 -- 172.21.15.38:0/1243523184 <== osd.0 v2:172.21.15.38:6800/2381639433 3 ==== osd_op_reply(53 607.00000000 [create] v100'23 uv23 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8038024030 con 0x558a956111d0 2024-09-21T03:40:41.204 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-09-21T03:40:41.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.199+0000 7f804a2e3f40 1 -- 172.21.15.38:0/1243523184 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:54 13.1c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x558a95615620 con 0x558a956111d0 2024-09-21T03:40:41.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.200+0000 7f804359c640 1 -- 172.21.15.38:0/1243523184 <== osd.0 v2:172.21.15.38:6800/2381639433 4 ==== osd_op_reply(54 607.00000000 [setxattr (30)] v100'24 uv24 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8038024030 con 0x558a956111d0 2024-09-21T03:40:41.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.200+0000 7f804a2e3f40 1 -- 172.21.15.38:0/1243523184 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:55 13.1c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x558a95615a80 con 0x558a956111d0 2024-09-21T03:40:41.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.201+0000 7f804359c640 1 -- 172.21.15.38:0/1243523184 <== osd.0 v2:172.21.15.38:6800/2381639433 5 ==== osd_op_reply(55 607.00000000 [setxattr (62)] v100'25 uv25 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8038024030 con 0x558a956111d0 2024-09-21T03:40:41.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.201+0000 7f804a2e3f40 1 -- 172.21.15.38:0/1243523184 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:56 13.1c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x558a95614610 con 0x558a956111d0 2024-09-21T03:40:41.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.202+0000 7f804359c640 1 -- 172.21.15.38:0/1243523184 <== osd.0 v2:172.21.15.38:6800/2381639433 6 ==== osd_op_reply(56 607.00000000 [omap-set-header] v100'26 uv26 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8038024030 con 0x558a956111d0 2024-09-21T03:40:41.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.202+0000 7f804a2e3f40 1 --2- 172.21.15.38:0/1243523184 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x558a956150d0 0x558a95617e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:41.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.202+0000 7f804a2e3f40 1 -- 172.21.15.38:0/1243523184 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:57 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x558a95618470 con 0x558a956150d0 2024-09-21T03:40:41.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.202+0000 7f804459e640 1 --2- 172.21.15.38:0/1243523184 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x558a956150d0 0x558a95617e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:41.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.202+0000 7f804459e640 1 --2- 172.21.15.38:0/1243523184 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x558a956150d0 0x558a95617e80 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:41.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.204+0000 7f804459e640 1 -- 172.21.15.38:0/1243523184 <== osd.3 v2:172.21.15.38:6808/1286810294 1 ==== osd_op_reply(57 mds0_openfiles.0 [create] v100'16 uv15 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f803c00b040 con 0x558a956150d0 2024-09-21T03:40:41.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.204+0000 7f804a2e3f40 1 -- 172.21.15.38:0/1243523184 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:58 13.6 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e100) -- 0x558a95618860 con 0x558a956150d0 2024-09-21T03:40:41.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.205+0000 7f804459e640 1 -- 172.21.15.38:0/1243523184 <== osd.3 v2:172.21.15.38:6808/1286810294 2 ==== osd_op_reply(58 mds0_openfiles.0 [delete] v100'17 uv17 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f803c00b040 con 0x558a956150d0 2024-09-21T03:40:41.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.205+0000 7f804a2e3f40 1 -- 172.21.15.38:0/1243523184 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:59 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x558a95618860 con 0x558a956150d0 2024-09-21T03:40:41.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.206+0000 7f804459e640 1 -- 172.21.15.38:0/1243523184 <== osd.3 v2:172.21.15.38:6808/1286810294 3 ==== osd_op_reply(59 mds0_openfiles.0 [create] v100'18 uv18 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f803c00b040 con 0x558a956150d0 2024-09-21T03:40:41.209 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-09-21T03:40:41.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.206+0000 7f804a2e3f40 1 -- 172.21.15.38:0/1243523184 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:60 13.6 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x558a95618860 con 0x558a956150d0 2024-09-21T03:40:41.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.207+0000 7f804459e640 1 -- 172.21.15.38:0/1243523184 <== osd.3 v2:172.21.15.38:6808/1286810294 4 ==== osd_op_reply(60 mds0_openfiles.0 [omap-set-header] v100'19 uv19 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f803c00b040 con 0x558a956150d0 2024-09-21T03:40:41.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.207+0000 7f804a2e3f40 1 -- 172.21.15.38:0/1243523184 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:61 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x558a956187a0 con 0x558a956111d0 2024-09-21T03:40:41.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.208+0000 7f804359c640 1 -- 172.21.15.38:0/1243523184 <== osd.0 v2:172.21.15.38:6800/2381639433 7 ==== osd_op_reply(61 608.00000000 [create] v100'21 uv20 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f8038024030 con 0x558a956111d0 2024-09-21T03:40:41.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.208+0000 7f804a2e3f40 1 -- 172.21.15.38:0/1243523184 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:62 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e100) -- 0x558a95618c80 con 0x558a956111d0 2024-09-21T03:40:41.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.210+0000 7f804359c640 1 -- 172.21.15.38:0/1243523184 <== osd.0 v2:172.21.15.38:6800/2381639433 8 ==== osd_op_reply(62 608.00000000 [delete] v100'22 uv22 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8038024030 con 0x558a956111d0 2024-09-21T03:40:41.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.210+0000 7f804a2e3f40 1 -- 172.21.15.38:0/1243523184 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x558a95618c80 con 0x558a956111d0 2024-09-21T03:40:41.213 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-09-21T03:40:41.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.211+0000 7f804359c640 1 -- 172.21.15.38:0/1243523184 <== osd.0 v2:172.21.15.38:6800/2381639433 9 ==== osd_op_reply(63 608.00000000 [create] v100'23 uv23 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8038024030 con 0x558a956111d0 2024-09-21T03:40:41.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.211+0000 7f804a2e3f40 1 -- 172.21.15.38:0/1243523184 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x558a95619c90 con 0x558a956111d0 2024-09-21T03:40:41.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.212+0000 7f804359c640 1 -- 172.21.15.38:0/1243523184 <== osd.0 v2:172.21.15.38:6800/2381639433 10 ==== osd_op_reply(64 608.00000000 [setxattr (30)] v100'24 uv24 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8038024030 con 0x558a956111d0 2024-09-21T03:40:41.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.212+0000 7f804a2e3f40 1 -- 172.21.15.38:0/1243523184 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x558a95619c90 con 0x558a956111d0 2024-09-21T03:40:41.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.214+0000 7f804359c640 1 -- 172.21.15.38:0/1243523184 <== osd.0 v2:172.21.15.38:6800/2381639433 11 ==== osd_op_reply(65 608.00000000 [setxattr (62)] v100'25 uv25 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8038024030 con 0x558a956111d0 2024-09-21T03:40:41.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.214+0000 7f804a2e3f40 1 -- 172.21.15.38:0/1243523184 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x558a95618c80 con 0x558a956111d0 2024-09-21T03:40:41.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.215+0000 7f804359c640 1 -- 172.21.15.38:0/1243523184 <== osd.0 v2:172.21.15.38:6800/2381639433 12 ==== osd_op_reply(66 608.00000000 [omap-set-header] v100'26 uv26 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8038024030 con 0x558a956111d0 2024-09-21T03:40:41.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.215+0000 7f804a2e3f40 1 -- 172.21.15.38:0/1243523184 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:67 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x558a95618c80 con 0x558a956150d0 2024-09-21T03:40:41.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.216+0000 7f804459e640 1 -- 172.21.15.38:0/1243523184 <== osd.3 v2:172.21.15.38:6808/1286810294 5 ==== osd_op_reply(67 604.00000000 [create] v100'37 uv31 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f803c00b040 con 0x558a956150d0 2024-09-21T03:40:41.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.216+0000 7f804a2e3f40 1 -- 172.21.15.38:0/1243523184 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:68 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e100) -- 0x558a95618c80 con 0x558a956150d0 2024-09-21T03:40:41.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.217+0000 7f804459e640 1 -- 172.21.15.38:0/1243523184 <== osd.3 v2:172.21.15.38:6808/1286810294 6 ==== osd_op_reply(68 604.00000000 [delete] v100'38 uv38 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f803c00b040 con 0x558a956150d0 2024-09-21T03:40:41.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.217+0000 7f804a2e3f40 1 -- 172.21.15.38:0/1243523184 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x558a95618c30 con 0x558a956150d0 2024-09-21T03:40:41.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.218+0000 7f804459e640 1 -- 172.21.15.38:0/1243523184 <== osd.3 v2:172.21.15.38:6808/1286810294 7 ==== osd_op_reply(69 604.00000000 [create] v100'39 uv39 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f803c00b040 con 0x558a956150d0 2024-09-21T03:40:41.223 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-09-21T03:40:41.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.218+0000 7f804a2e3f40 1 -- 172.21.15.38:0/1243523184 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x558a95618c30 con 0x558a956150d0 2024-09-21T03:40:41.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.219+0000 7f804459e640 1 -- 172.21.15.38:0/1243523184 <== osd.3 v2:172.21.15.38:6808/1286810294 8 ==== osd_op_reply(70 604.00000000 [setxattr (30)] v100'40 uv40 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f803c00b040 con 0x558a956150d0 2024-09-21T03:40:41.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.219+0000 7f804a2e3f40 1 -- 172.21.15.38:0/1243523184 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x558a95618c30 con 0x558a956150d0 2024-09-21T03:40:41.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.220+0000 7f804459e640 1 -- 172.21.15.38:0/1243523184 <== osd.3 v2:172.21.15.38:6808/1286810294 9 ==== osd_op_reply(71 604.00000000 [setxattr (62)] v100'41 uv41 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f803c00b040 con 0x558a956150d0 2024-09-21T03:40:41.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.220+0000 7f804a2e3f40 1 -- 172.21.15.38:0/1243523184 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x558a95618c30 con 0x558a956150d0 2024-09-21T03:40:41.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.221+0000 7f804459e640 1 -- 172.21.15.38:0/1243523184 <== osd.3 v2:172.21.15.38:6808/1286810294 10 ==== osd_op_reply(72 604.00000000 [omap-set-header] v100'42 uv42 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f803c00b040 con 0x558a956150d0 2024-09-21T03:40:41.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.221+0000 7f804a2e3f40 1 -- 172.21.15.38:0/1243523184 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:73 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x558a95618c30 con 0x558a956150d0 2024-09-21T03:40:41.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.222+0000 7f804459e640 1 -- 172.21.15.38:0/1243523184 <== osd.3 v2:172.21.15.38:6808/1286810294 11 ==== osd_op_reply(73 500.00000000 [create] v100'43 uv36 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f803c00b040 con 0x558a956150d0 2024-09-21T03:40:41.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.222+0000 7f804a2e3f40 1 -- 172.21.15.38:0/1243523184 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:74 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e100) -- 0x558a95618c30 con 0x558a956150d0 2024-09-21T03:40:41.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.223+0000 7f804459e640 1 -- 172.21.15.38:0/1243523184 <== osd.3 v2:172.21.15.38:6808/1286810294 12 ==== osd_op_reply(74 500.00000000 [delete] v100'44 uv44 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f803c00b040 con 0x558a956150d0 2024-09-21T03:40:41.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.223+0000 7f804a2e3f40 1 -- 172.21.15.38:0/1243523184 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x558a95618c30 con 0x558a956150d0 2024-09-21T03:40:41.229 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-09-21T03:40:41.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.224+0000 7f804459e640 1 -- 172.21.15.38:0/1243523184 <== osd.3 v2:172.21.15.38:6808/1286810294 13 ==== osd_op_reply(75 500.00000000 [create] v100'45 uv45 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f803c00b040 con 0x558a956150d0 2024-09-21T03:40:41.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.224+0000 7f804a2e3f40 1 -- 172.21.15.38:0/1243523184 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x558a95618c30 con 0x558a956150d0 2024-09-21T03:40:41.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.225+0000 7f804459e640 1 -- 172.21.15.38:0/1243523184 <== osd.3 v2:172.21.15.38:6808/1286810294 14 ==== osd_op_reply(76 500.00000000 [write 0~90] v100'46 uv46 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f803c00b040 con 0x558a956150d0 2024-09-21T03:40:41.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.225+0000 7f804a2e3f40 1 -- 172.21.15.38:0/1243523184 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x558a95618c30 con 0x558a956150d0 2024-09-21T03:40:41.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.226+0000 7f804459e640 1 -- 172.21.15.38:0/1243523184 <== osd.3 v2:172.21.15.38:6808/1286810294 15 ==== osd_op_reply(77 500.00000000 [omap-set-header] v100'47 uv47 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f803c00b040 con 0x558a956150d0 2024-09-21T03:40:41.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.226+0000 7f804a2e3f40 1 -- 172.21.15.38:0/1243523184 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:78 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x558a95618c30 con 0x558a956111d0 2024-09-21T03:40:41.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.228+0000 7f804359c640 1 -- 172.21.15.38:0/1243523184 <== osd.0 v2:172.21.15.38:6800/2381639433 13 ==== osd_op_reply(78 mds_snaptable [create] v100'17 uv16 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f8038024030 con 0x558a956111d0 2024-09-21T03:40:41.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.228+0000 7f804a2e3f40 1 -- 172.21.15.38:0/1243523184 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:79 13.d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e100) -- 0x558a95619080 con 0x558a956111d0 2024-09-21T03:40:41.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.229+0000 7f804359c640 1 -- 172.21.15.38:0/1243523184 <== osd.0 v2:172.21.15.38:6800/2381639433 14 ==== osd_op_reply(79 mds_snaptable [delete] v100'18 uv18 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f8038024030 con 0x558a956111d0 2024-09-21T03:40:41.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.229+0000 7f804a2e3f40 1 -- 172.21.15.38:0/1243523184 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:80 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x558a95618c30 con 0x558a956111d0 2024-09-21T03:40:41.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.230+0000 7f804359c640 1 -- 172.21.15.38:0/1243523184 <== osd.0 v2:172.21.15.38:6800/2381639433 15 ==== osd_op_reply(80 mds_snaptable [create] v100'19 uv19 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f8038024030 con 0x558a956111d0 2024-09-21T03:40:41.234 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-09-21T03:40:41.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.230+0000 7f804a2e3f40 1 -- 172.21.15.38:0/1243523184 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:81 13.d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x558a95619080 con 0x558a956111d0 2024-09-21T03:40:41.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.232+0000 7f804359c640 1 -- 172.21.15.38:0/1243523184 <== osd.0 v2:172.21.15.38:6800/2381639433 16 ==== osd_op_reply(81 mds_snaptable [write 0~70] v100'20 uv20 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f8038024030 con 0x558a956111d0 2024-09-21T03:40:41.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.232+0000 7f804a2e3f40 1 -- 172.21.15.38:0/1243523184 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:82 13.d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x558a95619080 con 0x558a956111d0 2024-09-21T03:40:41.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.233+0000 7f804359c640 1 -- 172.21.15.38:0/1243523184 <== osd.0 v2:172.21.15.38:6800/2381639433 17 ==== osd_op_reply(82 mds_snaptable [omap-set-header] v100'21 uv21 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f8038024030 con 0x558a956111d0 2024-09-21T03:40:41.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.233+0000 7f804a2e3f40 1 --2- 172.21.15.38:0/1243523184 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x558a95619590 0x558a9561b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:41.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.233+0000 7f804a2e3f40 1 -- 172.21.15.38:0/1243523184 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:83 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x558a9561bef0 con 0x558a95619590 2024-09-21T03:40:41.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.233+0000 7f8043d9d640 1 --2- 172.21.15.38:0/1243523184 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x558a95619590 0x558a9561b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:41.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.234+0000 7f8043d9d640 1 --2- 172.21.15.38:0/1243523184 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x558a95619590 0x558a9561b9b0 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:41.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.235+0000 7f8043d9d640 1 -- 172.21.15.38:0/1243523184 <== osd.1 v2:172.21.15.38:6816/1491818732 1 ==== osd_op_reply(83 605.00000000 [create] v100'21 uv20 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f803400a990 con 0x558a95619590 2024-09-21T03:40:41.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.235+0000 7f804a2e3f40 1 -- 172.21.15.38:0/1243523184 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:84 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e100) -- 0x558a9561bf70 con 0x558a95619590 2024-09-21T03:40:41.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.238+0000 7f8043d9d640 1 -- 172.21.15.38:0/1243523184 <== osd.1 v2:172.21.15.38:6816/1491818732 2 ==== osd_op_reply(84 605.00000000 [delete] v100'22 uv22 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f803400a990 con 0x558a95619590 2024-09-21T03:40:41.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.238+0000 7f804a2e3f40 1 -- 172.21.15.38:0/1243523184 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x558a9561c390 con 0x558a95619590 2024-09-21T03:40:41.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.239+0000 7f8043d9d640 1 -- 172.21.15.38:0/1243523184 <== osd.1 v2:172.21.15.38:6816/1491818732 3 ==== osd_op_reply(85 605.00000000 [create] v100'23 uv23 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f803400a990 con 0x558a95619590 2024-09-21T03:40:41.244 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-09-21T03:40:41.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.239+0000 7f804a2e3f40 1 -- 172.21.15.38:0/1243523184 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x558a9561d3a0 con 0x558a95619590 2024-09-21T03:40:41.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.240+0000 7f8043d9d640 1 -- 172.21.15.38:0/1243523184 <== osd.1 v2:172.21.15.38:6816/1491818732 4 ==== osd_op_reply(86 605.00000000 [setxattr (30)] v100'24 uv24 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f803400a990 con 0x558a95619590 2024-09-21T03:40:41.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.240+0000 7f804a2e3f40 1 -- 172.21.15.38:0/1243523184 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x558a9561d3a0 con 0x558a95619590 2024-09-21T03:40:41.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.241+0000 7f8043d9d640 1 -- 172.21.15.38:0/1243523184 <== osd.1 v2:172.21.15.38:6816/1491818732 5 ==== osd_op_reply(87 605.00000000 [setxattr (62)] v100'25 uv25 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f803400a990 con 0x558a95619590 2024-09-21T03:40:41.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.241+0000 7f804a2e3f40 1 -- 172.21.15.38:0/1243523184 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x558a9561c390 con 0x558a95619590 2024-09-21T03:40:41.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.242+0000 7f8043d9d640 1 -- 172.21.15.38:0/1243523184 <== osd.1 v2:172.21.15.38:6816/1491818732 6 ==== osd_op_reply(88 605.00000000 [omap-set-header] v100'26 uv26 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f803400a990 con 0x558a95619590 2024-09-21T03:40:41.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.242+0000 7f804a2e3f40 1 -- 172.21.15.38:0/1243523184 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:89 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x558a9561c390 con 0x558a95619590 2024-09-21T03:40:41.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.243+0000 7f8043d9d640 1 -- 172.21.15.38:0/1243523184 <== osd.1 v2:172.21.15.38:6816/1491818732 7 ==== osd_op_reply(89 mds0_inotable [create] v100'34 uv28 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f803400a990 con 0x558a95619590 2024-09-21T03:40:41.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.243+0000 7f804a2e3f40 1 -- 172.21.15.38:0/1243523184 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:90 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e100) -- 0x558a9561c390 con 0x558a95619590 2024-09-21T03:40:41.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.244+0000 7f8043d9d640 1 -- 172.21.15.38:0/1243523184 <== osd.1 v2:172.21.15.38:6816/1491818732 8 ==== osd_op_reply(90 mds0_inotable [delete] v100'35 uv35 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f803400a990 con 0x558a95619590 2024-09-21T03:40:41.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.244+0000 7f804a2e3f40 1 -- 172.21.15.38:0/1243523184 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x558a9561c390 con 0x558a95619590 2024-09-21T03:40:41.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.245+0000 7f8043d9d640 1 -- 172.21.15.38:0/1243523184 <== osd.1 v2:172.21.15.38:6816/1491818732 9 ==== osd_op_reply(91 mds0_inotable [create] v100'36 uv36 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f803400a990 con 0x558a95619590 2024-09-21T03:40:41.250 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-09-21T03:40:41.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.245+0000 7f804a2e3f40 1 -- 172.21.15.38:0/1243523184 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x558a9561c390 con 0x558a95619590 2024-09-21T03:40:41.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.246+0000 7f8043d9d640 1 -- 172.21.15.38:0/1243523184 <== osd.1 v2:172.21.15.38:6816/1491818732 10 ==== osd_op_reply(92 mds0_inotable [write 0~34] v100'37 uv37 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f803400a990 con 0x558a95619590 2024-09-21T03:40:41.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.246+0000 7f804a2e3f40 1 -- 172.21.15.38:0/1243523184 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x558a9561c390 con 0x558a95619590 2024-09-21T03:40:41.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.247+0000 7f8043d9d640 1 -- 172.21.15.38:0/1243523184 <== osd.1 v2:172.21.15.38:6816/1491818732 11 ==== osd_op_reply(93 mds0_inotable [omap-set-header] v100'38 uv38 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f803400a990 con 0x558a95619590 2024-09-21T03:40:41.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.247+0000 7f804a2e3f40 1 -- 172.21.15.38:0/1243523184 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:94 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x558a9561c390 con 0x558a95619590 2024-09-21T03:40:41.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.248+0000 7f8043d9d640 1 -- 172.21.15.38:0/1243523184 <== osd.1 v2:172.21.15.38:6816/1491818732 12 ==== osd_op_reply(94 100.00000000 [create] v100'39 uv33 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f803400a990 con 0x558a95619590 2024-09-21T03:40:41.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.248+0000 7f804a2e3f40 1 -- 172.21.15.38:0/1243523184 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:95 13.13 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e100) -- 0x558a9561c390 con 0x558a95619590 2024-09-21T03:40:41.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.249+0000 7f8043d9d640 1 -- 172.21.15.38:0/1243523184 <== osd.1 v2:172.21.15.38:6816/1491818732 13 ==== osd_op_reply(95 100.00000000 [delete] v100'40 uv40 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f803400a990 con 0x558a95619590 2024-09-21T03:40:41.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.249+0000 7f804a2e3f40 1 -- 172.21.15.38:0/1243523184 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:96 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x558a9561c390 con 0x558a95619590 2024-09-21T03:40:41.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.250+0000 7f8043d9d640 1 -- 172.21.15.38:0/1243523184 <== osd.1 v2:172.21.15.38:6816/1491818732 14 ==== osd_op_reply(96 100.00000000 [create] v100'41 uv41 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f803400a990 con 0x558a95619590 2024-09-21T03:40:41.254 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-09-21T03:40:41.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.250+0000 7f804a2e3f40 1 -- 172.21.15.38:0/1243523184 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:97 13.13 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x558a9561c390 con 0x558a95619590 2024-09-21T03:40:41.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.251+0000 7f8043d9d640 1 -- 172.21.15.38:0/1243523184 <== osd.1 v2:172.21.15.38:6816/1491818732 15 ==== osd_op_reply(97 100.00000000 [omap-set-header] v100'42 uv42 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f803400a990 con 0x558a95619590 2024-09-21T03:40:41.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.251+0000 7f804a2e3f40 1 -- 172.21.15.38:0/1243523184 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:98 13.13 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x558a9561f1a0 con 0x558a95619590 2024-09-21T03:40:41.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.253+0000 7f8043d9d640 1 -- 172.21.15.38:0/1243523184 <== osd.1 v2:172.21.15.38:6816/1491818732 16 ==== osd_op_reply(98 100.00000000 [omap-set-vals] v100'43 uv43 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f803400a990 con 0x558a95619590 2024-09-21T03:40:41.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.253+0000 7f804a2e3f40 1 -- 172.21.15.38:0/1243523184 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:99 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x558a9561c840 con 0x558a956150d0 2024-09-21T03:40:41.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.254+0000 7f804459e640 1 -- 172.21.15.38:0/1243523184 <== osd.3 v2:172.21.15.38:6808/1286810294 16 ==== osd_op_reply(99 mds0_sessionmap [create] v100'18 uv17 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7f803c00b040 con 0x558a956150d0 2024-09-21T03:40:41.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.254+0000 7f804a2e3f40 1 -- 172.21.15.38:0/1243523184 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:100 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e100) -- 0x558a9561c840 con 0x558a956150d0 2024-09-21T03:40:41.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.255+0000 7f804459e640 1 -- 172.21.15.38:0/1243523184 <== osd.3 v2:172.21.15.38:6808/1286810294 17 ==== osd_op_reply(100 mds0_sessionmap [delete] v100'19 uv19 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f803c00b040 con 0x558a956150d0 2024-09-21T03:40:41.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.255+0000 7f804a2e3f40 1 -- 172.21.15.38:0/1243523184 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x558a9561c840 con 0x558a956150d0 2024-09-21T03:40:41.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.256+0000 7f804459e640 1 -- 172.21.15.38:0/1243523184 <== osd.3 v2:172.21.15.38:6808/1286810294 18 ==== osd_op_reply(101 mds0_sessionmap [create] v100'20 uv20 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f803c00b040 con 0x558a956150d0 2024-09-21T03:40:41.258 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-09-21T03:40:41.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.256+0000 7f804a2e3f40 1 -- 172.21.15.38:0/1243523184 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x558a9561c840 con 0x558a956150d0 2024-09-21T03:40:41.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.257+0000 7f804459e640 1 -- 172.21.15.38:0/1243523184 <== osd.3 v2:172.21.15.38:6808/1286810294 19 ==== osd_op_reply(102 mds0_sessionmap [omap-set-header] v100'21 uv21 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f803c00b040 con 0x558a956150d0 2024-09-21T03:40:41.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.257+0000 7f804a2e3f40 1 -- 172.21.15.38:0/1243523184 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:103 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x558a9561c840 con 0x558a956150d0 2024-09-21T03:40:41.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.258+0000 7f804459e640 1 -- 172.21.15.38:0/1243523184 <== osd.3 v2:172.21.15.38:6808/1286810294 20 ==== osd_op_reply(103 609.00000000 [create] v100'21 uv20 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f803c00b040 con 0x558a956150d0 2024-09-21T03:40:41.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.258+0000 7f804a2e3f40 1 -- 172.21.15.38:0/1243523184 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:104 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e100) -- 0x558a9561c840 con 0x558a956150d0 2024-09-21T03:40:41.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.260+0000 7f804459e640 1 -- 172.21.15.38:0/1243523184 <== osd.3 v2:172.21.15.38:6808/1286810294 21 ==== osd_op_reply(104 609.00000000 [delete] v100'22 uv22 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f803c00b040 con 0x558a956150d0 2024-09-21T03:40:41.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.260+0000 7f804a2e3f40 1 -- 172.21.15.38:0/1243523184 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x558a9561c840 con 0x558a956150d0 2024-09-21T03:40:41.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.261+0000 7f804459e640 1 -- 172.21.15.38:0/1243523184 <== osd.3 v2:172.21.15.38:6808/1286810294 22 ==== osd_op_reply(105 609.00000000 [create] v100'23 uv23 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f803c00b040 con 0x558a956150d0 2024-09-21T03:40:41.263 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-09-21T03:40:41.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.261+0000 7f804a2e3f40 1 -- 172.21.15.38:0/1243523184 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x558a9561c840 con 0x558a956150d0 2024-09-21T03:40:41.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.262+0000 7f804459e640 1 -- 172.21.15.38:0/1243523184 <== osd.3 v2:172.21.15.38:6808/1286810294 23 ==== osd_op_reply(106 609.00000000 [setxattr (30)] v100'24 uv24 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f803c00b040 con 0x558a956150d0 2024-09-21T03:40:41.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.262+0000 7f804a2e3f40 1 -- 172.21.15.38:0/1243523184 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x558a9561d850 con 0x558a956150d0 2024-09-21T03:40:41.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.263+0000 7f804459e640 1 -- 172.21.15.38:0/1243523184 <== osd.3 v2:172.21.15.38:6808/1286810294 24 ==== osd_op_reply(107 609.00000000 [setxattr (62)] v100'25 uv25 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f803c00b040 con 0x558a956150d0 2024-09-21T03:40:41.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.263+0000 7f804a2e3f40 1 -- 172.21.15.38:0/1243523184 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x558a9561c840 con 0x558a956150d0 2024-09-21T03:40:41.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.264+0000 7f804459e640 1 -- 172.21.15.38:0/1243523184 <== osd.3 v2:172.21.15.38:6808/1286810294 25 ==== osd_op_reply(108 609.00000000 [omap-set-header] v100'26 uv26 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f803c00b040 con 0x558a956150d0 2024-09-21T03:40:41.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.264+0000 7f804a2e3f40 1 --2- 172.21.15.38:0/1243523184 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x558a9561d140 0x558a9561d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:41.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.264+0000 7f804a2e3f40 1 -- 172.21.15.38:0/1243523184 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:109 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x558a9561dc40 con 0x558a9561d140 2024-09-21T03:40:41.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.264+0000 7f804359c640 1 --2- 172.21.15.38:0/1243523184 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x558a9561d140 0x558a9561d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:41.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.265+0000 7f804359c640 1 --2- 172.21.15.38:0/1243523184 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x558a9561d140 0x558a9561d550 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:41.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.266+0000 7f804359c640 1 -- 172.21.15.38:0/1243523184 <== osd.2 v2:172.21.15.38:6824/2577858902 1 ==== osd_op_reply(109 400.00000000 [create] v100'17 uv16 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f8038024030 con 0x558a9561d140 2024-09-21T03:40:41.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.266+0000 7f804a2e3f40 1 -- 172.21.15.38:0/1243523184 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:110 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e100) -- 0x558a9561dc40 con 0x558a9561d140 2024-09-21T03:40:41.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.267+0000 7f804359c640 1 -- 172.21.15.38:0/1243523184 <== osd.2 v2:172.21.15.38:6824/2577858902 2 ==== osd_op_reply(110 400.00000000 [delete] v100'18 uv18 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8038024030 con 0x558a9561d140 2024-09-21T03:40:41.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.267+0000 7f804a2e3f40 1 -- 172.21.15.38:0/1243523184 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x558a9561dc40 con 0x558a9561d140 2024-09-21T03:40:41.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.268+0000 7f804359c640 1 -- 172.21.15.38:0/1243523184 <== osd.2 v2:172.21.15.38:6824/2577858902 3 ==== osd_op_reply(111 400.00000000 [create] v100'19 uv19 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8038024030 con 0x558a9561d140 2024-09-21T03:40:41.272 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-09-21T03:40:41.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.268+0000 7f804a2e3f40 1 -- 172.21.15.38:0/1243523184 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x558a9561dc40 con 0x558a9561d140 2024-09-21T03:40:41.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.269+0000 7f804359c640 1 -- 172.21.15.38:0/1243523184 <== osd.2 v2:172.21.15.38:6824/2577858902 4 ==== osd_op_reply(112 400.00000000 [write 0~22] v100'20 uv20 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8038024030 con 0x558a9561d140 2024-09-21T03:40:41.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.269+0000 7f804a2e3f40 1 -- 172.21.15.38:0/1243523184 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x558a9561dc40 con 0x558a9561d140 2024-09-21T03:40:41.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.270+0000 7f804359c640 1 -- 172.21.15.38:0/1243523184 <== osd.2 v2:172.21.15.38:6824/2577858902 5 ==== osd_op_reply(113 400.00000000 [omap-set-header] v100'21 uv21 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8038024030 con 0x558a9561d140 2024-09-21T03:40:41.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.270+0000 7f804a2e3f40 1 -- 172.21.15.38:0/1243523184 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:114 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x558a9561dc40 con 0x558a95619590 2024-09-21T03:40:41.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.272+0000 7f8043d9d640 1 -- 172.21.15.38:0/1243523184 <== osd.1 v2:172.21.15.38:6816/1491818732 17 ==== osd_op_reply(114 100.00000000.inode [create] v100'45 uv37 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7f803400a990 con 0x558a95619590 2024-09-21T03:40:41.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.272+0000 7f804a2e3f40 1 -- 172.21.15.38:0/1243523184 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:115 13.1f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e100) -- 0x558a9561dc40 con 0x558a95619590 2024-09-21T03:40:41.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.273+0000 7f8043d9d640 1 -- 172.21.15.38:0/1243523184 <== osd.1 v2:172.21.15.38:6816/1491818732 18 ==== osd_op_reply(115 100.00000000.inode [delete] v100'46 uv46 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f803400a990 con 0x558a95619590 2024-09-21T03:40:41.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.273+0000 7f804a2e3f40 1 -- 172.21.15.38:0/1243523184 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:116 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x558a9561dc40 con 0x558a95619590 2024-09-21T03:40:41.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.274+0000 7f8043d9d640 1 -- 172.21.15.38:0/1243523184 <== osd.1 v2:172.21.15.38:6816/1491818732 19 ==== osd_op_reply(116 100.00000000.inode [create] v100'47 uv47 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f803400a990 con 0x558a95619590 2024-09-21T03:40:41.278 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-09-21T03:40:41.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.274+0000 7f804a2e3f40 1 -- 172.21.15.38:0/1243523184 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:117 13.1f 13:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x558a9561dc40 con 0x558a95619590 2024-09-21T03:40:41.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.275+0000 7f8043d9d640 1 -- 172.21.15.38:0/1243523184 <== osd.1 v2:172.21.15.38:6816/1491818732 20 ==== osd_op_reply(117 100.00000000.inode [write 0~582] v100'48 uv48 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f803400a990 con 0x558a95619590 2024-09-21T03:40:41.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.275+0000 7f804a2e3f40 1 -- 172.21.15.38:0/1243523184 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:118 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x558a9561dc40 con 0x558a95619590 2024-09-21T03:40:41.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.276+0000 7f8043d9d640 1 -- 172.21.15.38:0/1243523184 <== osd.1 v2:172.21.15.38:6816/1491818732 21 ==== osd_op_reply(118 100.00000000.inode [omap-set-header] v100'49 uv49 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f803400a990 con 0x558a95619590 2024-09-21T03:40:41.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.276+0000 7f804a2e3f40 1 -- 172.21.15.38:0/1243523184 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:119 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x558a9561dc40 con 0x558a95619590 2024-09-21T03:40:41.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.277+0000 7f8043d9d640 1 -- 172.21.15.38:0/1243523184 <== osd.1 v2:172.21.15.38:6816/1491818732 22 ==== osd_op_reply(119 1.00000000 [create] v100'50 uv44 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7f803400a990 con 0x558a95619590 2024-09-21T03:40:41.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.277+0000 7f804a2e3f40 1 -- 172.21.15.38:0/1243523184 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:120 13.1f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e100) -- 0x558a9561dc40 con 0x558a95619590 2024-09-21T03:40:41.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.278+0000 7f8043d9d640 1 -- 172.21.15.38:0/1243523184 <== osd.1 v2:172.21.15.38:6816/1491818732 23 ==== osd_op_reply(120 1.00000000 [delete] v100'51 uv51 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f803400a990 con 0x558a95619590 2024-09-21T03:40:41.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.278+0000 7f804a2e3f40 1 -- 172.21.15.38:0/1243523184 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:121 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x558a9561dc40 con 0x558a95619590 2024-09-21T03:40:41.282 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-09-21T03:40:41.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.279+0000 7f8043d9d640 1 -- 172.21.15.38:0/1243523184 <== osd.1 v2:172.21.15.38:6816/1491818732 24 ==== osd_op_reply(121 1.00000000 [create] v100'52 uv52 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f803400a990 con 0x558a95619590 2024-09-21T03:40:41.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.280+0000 7f804a2e3f40 1 -- 172.21.15.38:0/1243523184 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:122 13.1f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x558a9561dc40 con 0x558a95619590 2024-09-21T03:40:41.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.281+0000 7f8043d9d640 1 -- 172.21.15.38:0/1243523184 <== osd.1 v2:172.21.15.38:6816/1491818732 25 ==== osd_op_reply(122 1.00000000 [setxattr (30)] v100'53 uv53 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f803400a990 con 0x558a95619590 2024-09-21T03:40:41.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.281+0000 7f804a2e3f40 1 -- 172.21.15.38:0/1243523184 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:123 13.1f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x558a9561dc40 con 0x558a95619590 2024-09-21T03:40:41.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.282+0000 7f8043d9d640 1 -- 172.21.15.38:0/1243523184 <== osd.1 v2:172.21.15.38:6816/1491818732 26 ==== osd_op_reply(123 1.00000000 [setxattr (30)] v100'54 uv54 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f803400a990 con 0x558a95619590 2024-09-21T03:40:41.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.282+0000 7f804a2e3f40 1 -- 172.21.15.38:0/1243523184 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:124 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x558a9561dc40 con 0x558a95619590 2024-09-21T03:40:41.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.283+0000 7f8043d9d640 1 -- 172.21.15.38:0/1243523184 <== osd.1 v2:172.21.15.38:6816/1491818732 27 ==== osd_op_reply(124 1.00000000 [omap-set-header] v100'55 uv55 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f803400a990 con 0x558a95619590 2024-09-21T03:40:41.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.283+0000 7f804a2e3f40 1 -- 172.21.15.38:0/1243523184 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:125 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=584b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x558a9561e030 con 0x558a95619590 2024-09-21T03:40:41.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.284+0000 7f8043d9d640 1 -- 172.21.15.38:0/1243523184 <== osd.1 v2:172.21.15.38:6816/1491818732 28 ==== osd_op_reply(125 1.00000000 [omap-set-vals] v100'56 uv56 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f803400a990 con 0x558a95619590 2024-09-21T03:40:41.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.284+0000 7f804a2e3f40 1 -- 172.21.15.38:0/1243523184 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x558a956111d0 msgr2=0x558a956135f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:41.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.284+0000 7f804a2e3f40 1 --2- 172.21.15.38:0/1243523184 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x558a956111d0 0x558a956135f0 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:41.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.284+0000 7f804a2e3f40 1 -- 172.21.15.38:0/1243523184 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x558a95619590 msgr2=0x558a9561b9b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:41.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.284+0000 7f804a2e3f40 1 --2- 172.21.15.38:0/1243523184 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x558a95619590 0x558a9561b9b0 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:41.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.284+0000 7f804a2e3f40 1 -- 172.21.15.38:0/1243523184 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x558a9561d140 msgr2=0x558a9561d550 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:41.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.284+0000 7f804a2e3f40 1 --2- 172.21.15.38:0/1243523184 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x558a9561d140 0x558a9561d550 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:41.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.284+0000 7f804a2e3f40 1 -- 172.21.15.38:0/1243523184 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x558a956150d0 msgr2=0x558a95617e80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:41.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.284+0000 7f804a2e3f40 1 --2- 172.21.15.38:0/1243523184 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x558a956150d0 0x558a95617e80 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:41.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.284+0000 7f804459e640 1 -- 172.21.15.38:0/1243523184 reap_dead start 2024-09-21T03:40:41.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.284+0000 7f804a2e3f40 1 -- 172.21.15.38:0/1243523184 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x558a9560c550 msgr2=0x558a9560e970 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:41.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.284+0000 7f804a2e3f40 1 --2- 172.21.15.38:0/1243523184 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x558a9560c550 0x558a9560e970 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:41.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.284+0000 7f804a2e3f40 1 -- 172.21.15.38:0/1243523184 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x558a95607ae0 msgr2=0x558a95609f00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:41.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.284+0000 7f804a2e3f40 1 --2- 172.21.15.38:0/1243523184 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x558a95607ae0 0x558a95609f00 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:41.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.284+0000 7f804a2e3f40 1 -- 172.21.15.38:0/1243523184 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8030077570 msgr2=0x7f8030079a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:41.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.284+0000 7f804a2e3f40 1 --2- 172.21.15.38:0/1243523184 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8030077570 0x7f8030079a80 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f8034004230 tx=0x7f8034002040 comp rx=0 tx=0).stop 2024-09-21T03:40:41.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.284+0000 7f804a2e3f40 1 -- 172.21.15.38:0/1243523184 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558a954d73b0 msgr2=0x558a955f0800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:41.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.284+0000 7f804a2e3f40 1 --2- 172.21.15.38:0/1243523184 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558a954d73b0 0x558a955f0800 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f8038000c00 tx=0x7f8038001330 comp rx=0 tx=0).stop 2024-09-21T03:40:41.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.284+0000 7f804459e640 1 -- 172.21.15.38:0/1243523184 reap_dead start 2024-09-21T03:40:41.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.285+0000 7f804a2e3f40 1 -- 172.21.15.38:0/1243523184 shutdown_connections 2024-09-21T03:40:41.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.285+0000 7f804a2e3f40 1 -- 172.21.15.38:0/1243523184 >> 172.21.15.38:0/1243523184 conn(0x558a953afac0 msgr2=0x558a9548f280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:41.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.285+0000 7f804a2e3f40 1 -- 172.21.15.38:0/1243523184 shutdown_connections 2024-09-21T03:40:41.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.285+0000 7f804a2e3f40 1 -- 172.21.15.38:0/1243523184 wait complete. 2024-09-21T03:40:41.291 DEBUG:teuthology.orchestra.run.smithi038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rm 200.00000000 2024-09-21T03:40:41.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.346+0000 7f5c3dd1df40 1 Processor -- start 2024-09-21T03:40:41.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.347+0000 7f5c3dd1df40 1 -- start start 2024-09-21T03:40:41.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.348+0000 7f5c3dd1df40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55aebb940d00 0x55aebb9f52a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:41.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.348+0000 7f5c3dd1df40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55aebb96ce30 0x55aebba15300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:41.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.348+0000 7f5c3dd1df40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55aebb9fb950 0x55aebba15840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:41.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.348+0000 7f5c3dd1df40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55aebb8ed430 con 0x55aebb96ce30 2024-09-21T03:40:41.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.348+0000 7f5c3dd1df40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55aebb8fa1f0 con 0x55aebb940d00 2024-09-21T03:40:41.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.348+0000 7f5c3dd1df40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55aebb88ed60 con 0x55aebb9fb950 2024-09-21T03:40:41.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.348+0000 7f5c36561640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55aebb96ce30 0x55aebba15300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:41.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.348+0000 7f5c36561640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55aebb96ce30 0x55aebba15300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:45406/0 (socket says 172.21.15.38:45406) 2024-09-21T03:40:41.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.348+0000 7f5c36561640 1 -- 172.21.15.38:0/3972071035 learned_addr learned my addr 172.21.15.38:0/3972071035 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:41.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.348+0000 7f5c36d62640 1 --2- 172.21.15.38:0/3972071035 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55aebb940d00 0x55aebb9f52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:41.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.348+0000 7f5c37563640 1 --2- 172.21.15.38:0/3972071035 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55aebb9fb950 0x55aebba15840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:41.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.348+0000 7f5c36561640 1 -- 172.21.15.38:0/3972071035 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55aebb9fb950 msgr2=0x55aebba15840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:41.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.348+0000 7f5c36561640 1 --2- 172.21.15.38:0/3972071035 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55aebb9fb950 0x55aebba15840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:41.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.348+0000 7f5c36561640 1 -- 172.21.15.38:0/3972071035 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55aebb940d00 msgr2=0x55aebb9f52a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:41.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.348+0000 7f5c36561640 1 --2- 172.21.15.38:0/3972071035 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55aebb940d00 0x55aebb9f52a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:41.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.348+0000 7f5c36561640 1 -- 172.21.15.38:0/3972071035 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aebb986f00 con 0x55aebb96ce30 2024-09-21T03:40:41.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.348+0000 7f5c37563640 1 --2- 172.21.15.38:0/3972071035 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55aebb9fb950 0x55aebba15840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:40:41.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.348+0000 7f5c36d62640 1 --2- 172.21.15.38:0/3972071035 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55aebb940d00 0x55aebb9f52a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:40:41.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.349+0000 7f5c36561640 1 --2- 172.21.15.38:0/3972071035 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55aebb96ce30 0x55aebba15300 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f5c2c026000 tx=0x7f5c2c002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cddf8cc6e7aefe0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:41.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.349+0000 7f5c3555f640 1 -- 172.21.15.38:0/3972071035 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c2c04da60 con 0x55aebb96ce30 2024-09-21T03:40:41.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.349+0000 7f5c3555f640 1 -- 172.21.15.38:0/3972071035 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5c2c056560 con 0x55aebb96ce30 2024-09-21T03:40:41.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.349+0000 7f5c3555f640 1 -- 172.21.15.38:0/3972071035 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c2c056950 con 0x55aebb96ce30 2024-09-21T03:40:41.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.349+0000 7f5c3dd1df40 1 -- 172.21.15.38:0/3972071035 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55aebb96ce30 msgr2=0x55aebba15300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:41.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.349+0000 7f5c3dd1df40 1 --2- 172.21.15.38:0/3972071035 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55aebb96ce30 0x55aebba15300 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f5c2c026000 tx=0x7f5c2c002fb0 comp rx=0 tx=0).stop 2024-09-21T03:40:41.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.349+0000 7f5c3dd1df40 1 -- 172.21.15.38:0/3972071035 shutdown_connections 2024-09-21T03:40:41.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.349+0000 7f5c3dd1df40 1 --2- 172.21.15.38:0/3972071035 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55aebb9fb950 0x55aebba15840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:41.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.349+0000 7f5c3dd1df40 1 --2- 172.21.15.38:0/3972071035 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55aebb96ce30 0x55aebba15300 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:41.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.349+0000 7f5c3dd1df40 1 --2- 172.21.15.38:0/3972071035 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55aebb940d00 0x55aebb9f52a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:41.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.349+0000 7f5c3dd1df40 1 -- 172.21.15.38:0/3972071035 >> 172.21.15.38:0/3972071035 conn(0x55aebb85fac0 msgr2=0x55aebba125a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:41.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.349+0000 7f5c3dd1df40 1 -- 172.21.15.38:0/3972071035 shutdown_connections 2024-09-21T03:40:41.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.349+0000 7f5c3dd1df40 1 -- 172.21.15.38:0/3972071035 wait complete. 2024-09-21T03:40:41.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.350+0000 7f5c3dd1df40 1 Processor -- start 2024-09-21T03:40:41.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.350+0000 7f5c3dd1df40 1 -- start start 2024-09-21T03:40:41.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.352+0000 7f5c3dd1df40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55aebb940d00 0x55aebba121c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:41.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.352+0000 7f5c3dd1df40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55aebb96ce30 0x55aebbaa3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:41.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.352+0000 7f5c3dd1df40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55aebb9fb950 0x55aebbaa3a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:41.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.352+0000 7f5c3dd1df40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55aebb9fb070 con 0x55aebb940d00 2024-09-21T03:40:41.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.352+0000 7f5c3dd1df40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55aebbaa4580 con 0x55aebb96ce30 2024-09-21T03:40:41.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.352+0000 7f5c3dd1df40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55aebbaa55d0 con 0x55aebb9fb950 2024-09-21T03:40:41.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.352+0000 7f5c36d62640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55aebb940d00 0x55aebba121c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:41.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.352+0000 7f5c36d62640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55aebb940d00 0x55aebba121c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:45408/0 (socket says 172.21.15.38:45408) 2024-09-21T03:40:41.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.352+0000 7f5c36d62640 1 -- 172.21.15.38:0/1918769875 learned_addr learned my addr 172.21.15.38:0/1918769875 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:41.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.352+0000 7f5c36561640 1 --2- 172.21.15.38:0/1918769875 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55aebb96ce30 0x55aebbaa3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:41.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.352+0000 7f5c37563640 1 --2- 172.21.15.38:0/1918769875 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55aebb9fb950 0x55aebbaa3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:41.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.352+0000 7f5c36d62640 1 -- 172.21.15.38:0/1918769875 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55aebb9fb950 msgr2=0x55aebbaa3a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:41.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.352+0000 7f5c36d62640 1 --2- 172.21.15.38:0/1918769875 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55aebb9fb950 0x55aebbaa3a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:41.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.352+0000 7f5c36d62640 1 -- 172.21.15.38:0/1918769875 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55aebb96ce30 msgr2=0x55aebbaa3550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:41.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.352+0000 7f5c36d62640 1 --2- 172.21.15.38:0/1918769875 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55aebb96ce30 0x55aebbaa3550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:41.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.352+0000 7f5c36d62640 1 -- 172.21.15.38:0/1918769875 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aebba11c10 con 0x55aebb940d00 2024-09-21T03:40:41.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.352+0000 7f5c36d62640 1 --2- 172.21.15.38:0/1918769875 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55aebb940d00 0x55aebba121c0 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f5c2800dd50 tx=0x7f5c2800b590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:41.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.352+0000 7f5c1ffff640 1 -- 172.21.15.38:0/1918769875 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c28015070 con 0x55aebb940d00 2024-09-21T03:40:41.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.352+0000 7f5c1ffff640 1 -- 172.21.15.38:0/1918769875 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5c28011750 con 0x55aebb940d00 2024-09-21T03:40:41.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.352+0000 7f5c3dd1df40 1 -- 172.21.15.38:0/1918769875 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55aebbaa5820 con 0x55aebb940d00 2024-09-21T03:40:41.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.352+0000 7f5c1ffff640 1 -- 172.21.15.38:0/1918769875 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c28011b80 con 0x55aebb940d00 2024-09-21T03:40:41.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.353+0000 7f5c3dd1df40 1 -- 172.21.15.38:0/1918769875 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aebbaa5b00 con 0x55aebb940d00 2024-09-21T03:40:41.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.354+0000 7f5c1ffff640 1 -- 172.21.15.38:0/1918769875 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f5c28018070 con 0x55aebb940d00 2024-09-21T03:40:41.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.355+0000 7f5c1ffff640 1 --2- 172.21.15.38:0/1918769875 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5c24077570 0x7f5c24079a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:41.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.355+0000 7f5c1ffff640 1 -- 172.21.15.38:0/1918769875 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 6648+0+0 (secure 0 0 0) 0x7f5c28099fc0 con 0x55aebb940d00 2024-09-21T03:40:41.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.355+0000 7f5c36561640 1 --2- 172.21.15.38:0/1918769875 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5c24077570 0x7f5c24079a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:41.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.355+0000 7f5c3dd1df40 1 --2- 172.21.15.38:0/1918769875 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x55aebba12700 0x55aebb9f7450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:41.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.355+0000 7f5c3dd1df40 1 -- 172.21.15.38:0/1918769875 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:1 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e100) -- 0x55aebba101b0 con 0x55aebba12700 2024-09-21T03:40:41.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.355+0000 7f5c37563640 1 --2- 172.21.15.38:0/1918769875 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x55aebba12700 0x55aebb9f7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:41.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.356+0000 7f5c36561640 1 --2- 172.21.15.38:0/1918769875 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5c24077570 0x7f5c24079a80 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f5c2c026000 tx=0x7f5c2c004880 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:41.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.356+0000 7f5c37563640 1 --2- 172.21.15.38:0/1918769875 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x55aebba12700 0x55aebb9f7450 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:41.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.357+0000 7f5c37563640 1 -- 172.21.15.38:0/1918769875 <== osd.6 v2:172.21.15.195:6826/4055825675 1 ==== osd_op_reply(1 200.00000000 [delete] v100'107 uv107 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5c300025f0 con 0x55aebba12700 2024-09-21T03:40:41.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.357+0000 7f5c3dd1df40 1 -- 172.21.15.38:0/1918769875 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x55aebba12700 msgr2=0x55aebb9f7450 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:41.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.357+0000 7f5c3dd1df40 1 --2- 172.21.15.38:0/1918769875 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x55aebba12700 0x55aebb9f7450 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:41.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.357+0000 7f5c3dd1df40 1 -- 172.21.15.38:0/1918769875 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5c24077570 msgr2=0x7f5c24079a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:41.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.357+0000 7f5c3dd1df40 1 --2- 172.21.15.38:0/1918769875 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5c24077570 0x7f5c24079a80 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f5c2c026000 tx=0x7f5c2c004880 comp rx=0 tx=0).stop 2024-09-21T03:40:41.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.357+0000 7f5c3dd1df40 1 -- 172.21.15.38:0/1918769875 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55aebb940d00 msgr2=0x55aebba121c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:41.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.357+0000 7f5c3dd1df40 1 --2- 172.21.15.38:0/1918769875 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55aebb940d00 0x55aebba121c0 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f5c2800dd50 tx=0x7f5c2800b590 comp rx=0 tx=0).stop 2024-09-21T03:40:41.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.357+0000 7f5c37563640 1 -- 172.21.15.38:0/1918769875 reap_dead start 2024-09-21T03:40:41.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.357+0000 7f5c3dd1df40 1 -- 172.21.15.38:0/1918769875 shutdown_connections 2024-09-21T03:40:41.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.357+0000 7f5c3dd1df40 1 -- 172.21.15.38:0/1918769875 >> 172.21.15.38:0/1918769875 conn(0x55aebb85fac0 msgr2=0x55aebb9f52a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:41.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.357+0000 7f5c3dd1df40 1 -- 172.21.15.38:0/1918769875 shutdown_connections 2024-09-21T03:40:41.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.357+0000 7f5c3dd1df40 1 -- 172.21.15.38:0/1918769875 wait complete. 2024-09-21T03:40:41.364 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-21T03:40:41.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.484+0000 7f7e2c015640 1 -- 172.21.15.38:0/2827781755 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e240d1120 msgr2=0x7f7e240ceb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:41.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.484+0000 7f7e2c015640 1 --2- 172.21.15.38:0/2827781755 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e240d1120 0x7f7e240ceb90 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f7e1c009830 tx=0x7f7e1c02a480 comp rx=0 tx=0).stop 2024-09-21T03:40:41.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.484+0000 7f7e2c015640 1 -- 172.21.15.38:0/2827781755 shutdown_connections 2024-09-21T03:40:41.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.484+0000 7f7e2c015640 1 --2- 172.21.15.38:0/2827781755 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e240d1120 0x7f7e240ceb90 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:41.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.484+0000 7f7e2c015640 1 --2- 172.21.15.38:0/2827781755 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7e240d0170 0x7f7e240d0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:41.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.484+0000 7f7e2c015640 1 -- 172.21.15.38:0/2827781755 >> 172.21.15.38:0/2827781755 conn(0x7f7e2409d830 msgr2=0x7f7e240b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:41.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.484+0000 7f7e2c015640 1 -- 172.21.15.38:0/2827781755 shutdown_connections 2024-09-21T03:40:41.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.484+0000 7f7e2c015640 1 -- 172.21.15.38:0/2827781755 wait complete. 2024-09-21T03:40:41.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.486+0000 7f7e2c015640 1 Processor -- start 2024-09-21T03:40:41.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.486+0000 7f7e2c015640 1 -- start start 2024-09-21T03:40:41.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.487+0000 7f7e2c015640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7e240d0170 0x7f7e24155930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:41.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.487+0000 7f7e2c015640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7e240d1120 0x7f7e240a18c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:41.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.487+0000 7f7e2c015640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e2415ee80 0x7f7e2406ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:41.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.487+0000 7f7e2c015640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f7e240b6700 con 0x7f7e2415ee80 2024-09-21T03:40:41.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.487+0000 7f7e2c015640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f7e240b6580 con 0x7f7e240d1120 2024-09-21T03:40:41.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.487+0000 7f7e2c015640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f7e240b6880 con 0x7f7e240d0170 2024-09-21T03:40:41.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.488+0000 7f7e2a5b2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e2415ee80 0x7f7e2406ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:41.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.488+0000 7f7e2a5b2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e2415ee80 0x7f7e2406ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:45434/0 (socket says 172.21.15.38:45434) 2024-09-21T03:40:41.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.488+0000 7f7e2a5b2640 1 -- 172.21.15.38:0/1754914076 learned_addr learned my addr 172.21.15.38:0/1754914076 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:41.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.488+0000 7f7e295b0640 1 --2- 172.21.15.38:0/1754914076 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7e240d1120 0x7f7e240a18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:41.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.488+0000 7f7e29db1640 1 --2- 172.21.15.38:0/1754914076 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7e240d0170 0x7f7e24155930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:41.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.488+0000 7f7e2a5b2640 1 -- 172.21.15.38:0/1754914076 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7e240d0170 msgr2=0x7f7e24155930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:41.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.488+0000 7f7e2a5b2640 1 --2- 172.21.15.38:0/1754914076 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7e240d0170 0x7f7e24155930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:41.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.488+0000 7f7e2a5b2640 1 -- 172.21.15.38:0/1754914076 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7e240d1120 msgr2=0x7f7e240a18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:41.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.488+0000 7f7e2a5b2640 1 --2- 172.21.15.38:0/1754914076 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7e240d1120 0x7f7e240a18c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:41.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.488+0000 7f7e2a5b2640 1 -- 172.21.15.38:0/1754914076 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e2406e170 con 0x7f7e2415ee80 2024-09-21T03:40:41.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.488+0000 7f7e2a5b2640 1 --2- 172.21.15.38:0/1754914076 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e2415ee80 0x7f7e2406ef10 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f7e1c02aa70 tx=0x7f7e1c033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:41.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.488+0000 7f7e1a7fc640 1 -- 172.21.15.38:0/1754914076 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e1c005c90 con 0x7f7e2415ee80 2024-09-21T03:40:41.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.488+0000 7f7e1a7fc640 1 -- 172.21.15.38:0/1754914076 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7e1c031e50 con 0x7f7e2415ee80 2024-09-21T03:40:41.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.488+0000 7f7e2c015640 1 -- 172.21.15.38:0/1754914076 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e2406e450 con 0x7f7e2415ee80 2024-09-21T03:40:41.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.488+0000 7f7e1a7fc640 1 -- 172.21.15.38:0/1754914076 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e1c005220 con 0x7f7e2415ee80 2024-09-21T03:40:41.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.488+0000 7f7e2c015640 1 -- 172.21.15.38:0/1754914076 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e2406ea10 con 0x7f7e2415ee80 2024-09-21T03:40:41.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.489+0000 7f7e2c015640 1 -- 172.21.15.38:0/1754914076 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e24155ef0 con 0x7f7e2415ee80 2024-09-21T03:40:41.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.490+0000 7f7e1a7fc640 1 -- 172.21.15.38:0/1754914076 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f7e1c0053c0 con 0x7f7e2415ee80 2024-09-21T03:40:41.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.491+0000 7f7e1a7fc640 1 --2- 172.21.15.38:0/1754914076 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7e08078b90 0x7f7e0807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:41.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.491+0000 7f7e1a7fc640 1 -- 172.21.15.38:0/1754914076 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 6648+0+0 (secure 0 0 0) 0x7f7e1c0bc050 con 0x7f7e2415ee80 2024-09-21T03:40:41.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.491+0000 7f7e29db1640 1 --2- 172.21.15.38:0/1754914076 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7e08078b90 0x7f7e0807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:41.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.492+0000 7f7e29db1640 1 --2- 172.21.15.38:0/1754914076 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7e08078b90 0x7f7e0807b0a0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f7e200012e0 tx=0x7f7e20009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:41.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.496+0000 7f7e1a7fc640 1 -- 172.21.15.38:0/1754914076 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f7e1c03f030 con 0x7f7e2415ee80 2024-09-21T03:40:41.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:41.722+0000 7f7e2c015640 1 -- 172.21.15.38:0/1754914076 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f7e240b6de0 con 0x7f7e2415ee80 2024-09-21T03:40:42.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:42.041+0000 7f7e1a7fc640 1 -- 172.21.15.38:0/1754914076 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v108) ==== 162+0+0 (secure 0 0 0) 0x7f7e1c031770 con 0x7f7e2415ee80 2024-09-21T03:40:42.043 INFO:teuthology.orchestra.run.smithi038.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-21T03:40:42.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:42.043+0000 7f7e2c015640 1 -- 172.21.15.38:0/1754914076 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7e08078b90 msgr2=0x7f7e0807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:42.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:42.043+0000 7f7e2c015640 1 --2- 172.21.15.38:0/1754914076 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7e08078b90 0x7f7e0807b0a0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f7e200012e0 tx=0x7f7e20009040 comp rx=0 tx=0).stop 2024-09-21T03:40:42.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:42.043+0000 7f7e2c015640 1 -- 172.21.15.38:0/1754914076 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e2415ee80 msgr2=0x7f7e2406ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:42.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:42.043+0000 7f7e2c015640 1 --2- 172.21.15.38:0/1754914076 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e2415ee80 0x7f7e2406ef10 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f7e1c02aa70 tx=0x7f7e1c033c20 comp rx=0 tx=0).stop 2024-09-21T03:40:42.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:42.043+0000 7f7e2c015640 1 -- 172.21.15.38:0/1754914076 shutdown_connections 2024-09-21T03:40:42.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:42.043+0000 7f7e2c015640 1 --2- 172.21.15.38:0/1754914076 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e2415ee80 0x7f7e2406ef10 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:42.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:42.043+0000 7f7e2c015640 1 --2- 172.21.15.38:0/1754914076 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7e240d1120 0x7f7e240a18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:42.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:42.043+0000 7f7e2c015640 1 --2- 172.21.15.38:0/1754914076 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7e08078b90 0x7f7e0807b0a0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:42.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:42.043+0000 7f7e2c015640 1 --2- 172.21.15.38:0/1754914076 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7e240d0170 0x7f7e24155930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:42.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:42.043+0000 7f7e2c015640 1 -- 172.21.15.38:0/1754914076 >> 172.21.15.38:0/1754914076 conn(0x7f7e2409d830 msgr2=0x7f7e2415ffb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:42.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:42.043+0000 7f7e2c015640 1 -- 172.21.15.38:0/1754914076 shutdown_connections 2024-09-21T03:40:42.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:42.043+0000 7f7e2c015640 1 -- 172.21.15.38:0/1754914076 wait complete. 2024-09-21T03:40:42.053 INFO:tasks.ceph.mds.c.smithi038.stderr:2024-09-21T03:40:42.052+0000 7f4d30d80640 -1 log_channel(cluster) log [ERR] : Error recovering journal 0x200: (2) No such file or directory 2024-09-21T03:40:42.054 INFO:tasks.ceph.mon.a.smithi038.stderr:2024-09-21T03:40:42.052+0000 7f3f45713640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T03:40:42.054 INFO:tasks.ceph.mon.a.smithi038.stderr:2024-09-21T03:40:42.052+0000 7f3f45713640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-09-21T03:40:42.060 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:40:42.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:42.175+0000 7fd4d5dda640 1 -- 172.21.15.38:0/2194148241 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd4d006e1d0 msgr2=0x7fd4d00afd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:42.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:42.175+0000 7fd4d5dda640 1 --2- 172.21.15.38:0/2194148241 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd4d006e1d0 0x7fd4d00afd70 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7fd4d00cd2f0 tx=0x7fd4c002aa30 comp rx=0 tx=0).stop 2024-09-21T03:40:42.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:42.175+0000 7fd4d5dda640 1 -- 172.21.15.38:0/2194148241 shutdown_connections 2024-09-21T03:40:42.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:42.175+0000 7fd4d5dda640 1 --2- 172.21.15.38:0/2194148241 >> v2:172.21.15.195:3300/0 conn(0x7fd4d00b4b50 0x7fd4d006f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:42.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:42.175+0000 7fd4d5dda640 1 --2- 172.21.15.38:0/2194148241 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd4d00b4740 0x7fd4d006f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:42.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:42.175+0000 7fd4d5dda640 1 --2- 172.21.15.38:0/2194148241 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd4d006e1d0 0x7fd4d00afd70 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:42.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:42.175+0000 7fd4d5dda640 1 -- 172.21.15.38:0/2194148241 >> 172.21.15.38:0/2194148241 conn(0x7fd4d00d0a30 msgr2=0x7fd4d00d0e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:42.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:42.175+0000 7fd4d5dda640 1 -- 172.21.15.38:0/2194148241 shutdown_connections 2024-09-21T03:40:42.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:42.175+0000 7fd4d5dda640 1 -- 172.21.15.38:0/2194148241 wait complete. 2024-09-21T03:40:42.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:42.177+0000 7fd4d5dda640 1 Processor -- start 2024-09-21T03:40:42.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:42.177+0000 7fd4d5dda640 1 -- start start 2024-09-21T03:40:42.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:42.178+0000 7fd4d5dda640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd4d006e1d0 0x7fd4d01e59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:42.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:42.178+0000 7fd4d5dda640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd4d00b4740 0x7fd4d01481d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:42.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:42.178+0000 7fd4d5dda640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd4d00b4b50 0x7fd4d00aadd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:42.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:42.178+0000 7fd4d5dda640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fd4d015b110 con 0x7fd4d006e1d0 2024-09-21T03:40:42.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:42.178+0000 7fd4d5dda640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd4d015af90 con 0x7fd4d00b4740 2024-09-21T03:40:42.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:42.178+0000 7fd4d5dda640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fd4d015b290 con 0x7fd4d00b4b50 2024-09-21T03:40:42.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:42.178+0000 7fd4cf7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd4d006e1d0 0x7fd4d01e59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:42.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:42.178+0000 7fd4cf7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd4d006e1d0 0x7fd4d01e59c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:45452/0 (socket says 172.21.15.38:45452) 2024-09-21T03:40:42.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:42.178+0000 7fd4cf7fe640 1 -- 172.21.15.38:0/682157741 learned_addr learned my addr 172.21.15.38:0/682157741 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:42.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:42.179+0000 7fd4cffff640 1 --2- 172.21.15.38:0/682157741 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd4d00b4b50 0x7fd4d00aadd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:42.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:42.179+0000 7fd4ceffd640 1 --2- 172.21.15.38:0/682157741 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd4d00b4740 0x7fd4d01481d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:42.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:42.179+0000 7fd4cf7fe640 1 -- 172.21.15.38:0/682157741 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd4d00b4b50 msgr2=0x7fd4d00aadd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:42.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:42.179+0000 7fd4cf7fe640 1 --2- 172.21.15.38:0/682157741 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd4d00b4b50 0x7fd4d00aadd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:42.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:42.179+0000 7fd4cf7fe640 1 -- 172.21.15.38:0/682157741 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd4d00b4740 msgr2=0x7fd4d01481d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:42.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:42.179+0000 7fd4cf7fe640 1 --2- 172.21.15.38:0/682157741 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd4d00b4740 0x7fd4d01481d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:42.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:42.179+0000 7fd4cf7fe640 1 -- 172.21.15.38:0/682157741 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4d01e62c0 con 0x7fd4d006e1d0 2024-09-21T03:40:42.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:42.179+0000 7fd4cf7fe640 1 --2- 172.21.15.38:0/682157741 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd4d006e1d0 0x7fd4d01e59c0 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7fd4c0009d60 tx=0x7fd4c0003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:42.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:42.179+0000 7fd4b7fff640 1 -- 172.21.15.38:0/682157741 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4c0033050 con 0x7fd4d006e1d0 2024-09-21T03:40:42.183 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:42.179+0000 7fd4b7fff640 1 -- 172.21.15.38:0/682157741 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd4c00044d0 con 0x7fd4d006e1d0 2024-09-21T03:40:42.183 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:42.179+0000 7fd4d5dda640 1 -- 172.21.15.38:0/682157741 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4d01e65a0 con 0x7fd4d006e1d0 2024-09-21T03:40:42.183 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:42.179+0000 7fd4b7fff640 1 -- 172.21.15.38:0/682157741 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4c0038450 con 0x7fd4d006e1d0 2024-09-21T03:40:42.183 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:42.179+0000 7fd4d5dda640 1 -- 172.21.15.38:0/682157741 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4d01e6b60 con 0x7fd4d006e1d0 2024-09-21T03:40:42.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:42.180+0000 7fd4d5dda640 1 -- 172.21.15.38:0/682157741 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4d015b7b0 con 0x7fd4d006e1d0 2024-09-21T03:40:42.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:42.181+0000 7fd4b7fff640 1 -- 172.21.15.38:0/682157741 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fd4c0036070 con 0x7fd4d006e1d0 2024-09-21T03:40:42.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:42.182+0000 7fd4b7fff640 1 --2- 172.21.15.38:0/682157741 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd4bc078bc0 0x7fd4bc07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:42.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:42.182+0000 7fd4b7fff640 1 -- 172.21.15.38:0/682157741 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 6734+0+0 (secure 0 0 0) 0x7fd4c00bc560 con 0x7fd4d006e1d0 2024-09-21T03:40:42.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:42.182+0000 7fd4ceffd640 1 --2- 172.21.15.38:0/682157741 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd4bc078bc0 0x7fd4bc07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:42.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:42.182+0000 7fd4ceffd640 1 --2- 172.21.15.38:0/682157741 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd4bc078bc0 0x7fd4bc07b0d0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fd4d01f55f0 tx=0x7fd4c4005f60 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:42.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:42.186+0000 7fd4b7fff640 1 -- 172.21.15.38:0/682157741 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fd4c0083f50 con 0x7fd4d006e1d0 2024-09-21T03:40:42.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:42.411+0000 7fd4d5dda640 1 -- 172.21.15.38:0/682157741 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd4d00765f0 con 0x7fd4d006e1d0 2024-09-21T03:40:42.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:42.412+0000 7fd4b7fff640 1 -- 172.21.15.38:0/682157741 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 110 v110) ==== 95+0+3566 (secure 0 0 0) 0x7fd4c0088980 con 0x7fd4d006e1d0 2024-09-21T03:40:42.414 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:40:42.414 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":110,"btime":"2024-09-21T03:40:42:053536+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6086,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/965972663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":965972663},{"type":"v1","addr":"172.21.15.195:6836","nonce":965972663}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6700,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/3176122334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":3176122334},{"type":"v1","addr":"172.21.15.195:6837","nonce":3176122334}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":110,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:40:02.319282+0000","modified":"2024-09-21T03:40:42.053470+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":101,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-21T03:40:42.414 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 110 2024-09-21T03:40:42.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:42.414+0000 7fd4d5dda640 1 -- 172.21.15.38:0/682157741 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd4bc078bc0 msgr2=0x7fd4bc07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:42.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:42.414+0000 7fd4d5dda640 1 --2- 172.21.15.38:0/682157741 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd4bc078bc0 0x7fd4bc07b0d0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fd4d01f55f0 tx=0x7fd4c4005f60 comp rx=0 tx=0).stop 2024-09-21T03:40:42.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:42.414+0000 7fd4d5dda640 1 -- 172.21.15.38:0/682157741 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd4d006e1d0 msgr2=0x7fd4d01e59c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:42.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:42.414+0000 7fd4d5dda640 1 --2- 172.21.15.38:0/682157741 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd4d006e1d0 0x7fd4d01e59c0 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7fd4c0009d60 tx=0x7fd4c0003f10 comp rx=0 tx=0).stop 2024-09-21T03:40:42.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:42.414+0000 7fd4d5dda640 1 -- 172.21.15.38:0/682157741 shutdown_connections 2024-09-21T03:40:42.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:42.414+0000 7fd4d5dda640 1 --2- 172.21.15.38:0/682157741 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd4bc078bc0 0x7fd4bc07b0d0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:42.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:42.414+0000 7fd4d5dda640 1 --2- 172.21.15.38:0/682157741 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd4d00b4b50 0x7fd4d00aadd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:42.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:42.414+0000 7fd4d5dda640 1 --2- 172.21.15.38:0/682157741 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd4d00b4740 0x7fd4d01481d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:42.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:42.414+0000 7fd4d5dda640 1 --2- 172.21.15.38:0/682157741 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd4d006e1d0 0x7fd4d01e59c0 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:42.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:42.414+0000 7fd4d5dda640 1 -- 172.21.15.38:0/682157741 >> 172.21.15.38:0/682157741 conn(0x7fd4d00d0a30 msgr2=0x7fd4d00b15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:42.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:42.414+0000 7fd4d5dda640 1 -- 172.21.15.38:0/682157741 shutdown_connections 2024-09-21T03:40:42.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:42.414+0000 7fd4d5dda640 1 -- 172.21.15.38:0/682157741 wait complete. 2024-09-21T03:40:42.428 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-21T03:40:42.428 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:40:42.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:42.543+0000 7f7776e0a640 1 -- 172.21.15.38:0/1981193706 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7770162f00 msgr2=0x7f77700d24d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:42.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:42.543+0000 7f7776e0a640 1 --2- 172.21.15.38:0/1981193706 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7770162f00 0x7f77700d24d0 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f7760005fd0 tx=0x7f776002a5d0 comp rx=0 tx=0).stop 2024-09-21T03:40:42.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:42.543+0000 7f7776e0a640 1 -- 172.21.15.38:0/1981193706 shutdown_connections 2024-09-21T03:40:42.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:42.543+0000 7f7776e0a640 1 --2- 172.21.15.38:0/1981193706 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f77700b2e40 0x7f77700b3250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:42.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:42.543+0000 7f7776e0a640 1 --2- 172.21.15.38:0/1981193706 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7770162f00 0x7f77700d24d0 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:42.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:42.543+0000 7f7776e0a640 1 -- 172.21.15.38:0/1981193706 >> 172.21.15.38:0/1981193706 conn(0x7f77700b6b20 msgr2=0x7f77700d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:42.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:42.543+0000 7f7776e0a640 1 -- 172.21.15.38:0/1981193706 shutdown_connections 2024-09-21T03:40:42.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:42.543+0000 7f7776e0a640 1 -- 172.21.15.38:0/1981193706 wait complete. 2024-09-21T03:40:42.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:42.545+0000 7f7776e0a640 1 Processor -- start 2024-09-21T03:40:42.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:42.545+0000 7f7776e0a640 1 -- start start 2024-09-21T03:40:42.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:42.546+0000 7f7776e0a640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f77700b2e40 0x7f77700cef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:42.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:42.546+0000 7f7776e0a640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7770162280 0x7f77700abd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:42.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:42.546+0000 7f7776e0a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f77700ac2d0 0x7f77700ac6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:42.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:42.546+0000 7f7776e0a640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f777015e4b0 con 0x7f77700ac2d0 2024-09-21T03:40:42.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:42.546+0000 7f7776e0a640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f77700b3a20 con 0x7f7770162280 2024-09-21T03:40:42.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:42.546+0000 7f7776e0a640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f777015e630 con 0x7f77700b2e40 2024-09-21T03:40:42.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:42.546+0000 7f77753a7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f77700ac2d0 0x7f77700ac6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:42.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:42.546+0000 7f77753a7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f77700ac2d0 0x7f77700ac6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:58746/0 (socket says 172.21.15.38:58746) 2024-09-21T03:40:42.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:42.546+0000 7f77753a7640 1 -- 172.21.15.38:0/518066249 learned_addr learned my addr 172.21.15.38:0/518066249 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:42.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:42.546+0000 7f7774ba6640 1 --2- 172.21.15.38:0/518066249 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f77700b2e40 0x7f77700cef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:42.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:42.546+0000 7f776ffff640 1 --2- 172.21.15.38:0/518066249 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7770162280 0x7f77700abd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:42.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:42.547+0000 7f77753a7640 1 -- 172.21.15.38:0/518066249 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f77700b2e40 msgr2=0x7f77700cef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:42.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:42.547+0000 7f77753a7640 1 --2- 172.21.15.38:0/518066249 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f77700b2e40 0x7f77700cef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:42.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:42.547+0000 7f77753a7640 1 -- 172.21.15.38:0/518066249 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7770162280 msgr2=0x7f77700abd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:42.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:42.547+0000 7f77753a7640 1 --2- 172.21.15.38:0/518066249 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7770162280 0x7f77700abd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:42.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:42.547+0000 7f77753a7640 1 -- 172.21.15.38:0/518066249 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f777014ef20 con 0x7f77700ac2d0 2024-09-21T03:40:42.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:42.547+0000 7f77753a7640 1 --2- 172.21.15.38:0/518066249 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f77700ac2d0 0x7f77700ac6e0 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f776800e9c0 tx=0x7f776800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:42.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:42.547+0000 7f776d7fa640 1 -- 172.21.15.38:0/518066249 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7768004d20 con 0x7f77700ac2d0 2024-09-21T03:40:42.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:42.547+0000 7f776d7fa640 1 -- 172.21.15.38:0/518066249 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f776800cb20 con 0x7f77700ac2d0 2024-09-21T03:40:42.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:42.547+0000 7f7776e0a640 1 -- 172.21.15.38:0/518066249 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f777014f200 con 0x7f77700ac2d0 2024-09-21T03:40:42.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:42.547+0000 7f776d7fa640 1 -- 172.21.15.38:0/518066249 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7768005020 con 0x7f77700ac2d0 2024-09-21T03:40:42.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:42.547+0000 7f7776e0a640 1 -- 172.21.15.38:0/518066249 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f777014f7c0 con 0x7f77700ac2d0 2024-09-21T03:40:42.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:42.548+0000 7f7776e0a640 1 -- 172.21.15.38:0/518066249 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77700aee00 con 0x7f77700ac2d0 2024-09-21T03:40:42.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:42.549+0000 7f776d7fa640 1 -- 172.21.15.38:0/518066249 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f77680051c0 con 0x7f77700ac2d0 2024-09-21T03:40:42.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:42.549+0000 7f776d7fa640 1 --2- 172.21.15.38:0/518066249 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f775c078b90 0x7f775c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:42.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:42.550+0000 7f776d7fa640 1 -- 172.21.15.38:0/518066249 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 6734+0+0 (secure 0 0 0) 0x7f776809b9a0 con 0x7f77700ac2d0 2024-09-21T03:40:42.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:42.550+0000 7f7774ba6640 1 --2- 172.21.15.38:0/518066249 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f775c078b90 0x7f775c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:42.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:42.550+0000 7f7774ba6640 1 --2- 172.21.15.38:0/518066249 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f775c078b90 0x7f775c07b0a0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f7760033000 tx=0x7f7760004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:42.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:42.554+0000 7f776d7fa640 1 -- 172.21.15.38:0/518066249 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f7768016030 con 0x7f77700ac2d0 2024-09-21T03:40:42.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:42.780+0000 7f7776e0a640 1 -- 172.21.15.38:0/518066249 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7770162f00 con 0x7f77700ac2d0 2024-09-21T03:40:42.782 INFO:tasks.ceph.mds.a.smithi038.stderr: -37> 2024-09-21T03:40:38.028+0000 7f9460c6c640 -1 mds.0.cache.ino(0x1) No data while reading inode 0x1 2024-09-21T03:40:42.782 INFO:tasks.ceph.mds.a.smithi038.stderr: -35> 2024-09-21T03:40:38.028+0000 7f9460c6c640 -1 log_channel(cluster) log [ERR] : Error loading MDS rank 0: (2) No such file or directory 2024-09-21T03:40:42.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:42.780+0000 7f776d7fa640 1 -- 172.21.15.38:0/518066249 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 110 v110) ==== 95+0+3566 (secure 0 0 0) 0x7f77680633a0 con 0x7f77700ac2d0 2024-09-21T03:40:42.783 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:40:42.783 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":110,"btime":"2024-09-21T03:40:42:053536+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6086,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/965972663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":965972663},{"type":"v1","addr":"172.21.15.195:6836","nonce":965972663}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6700,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/3176122334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":3176122334},{"type":"v1","addr":"172.21.15.195:6837","nonce":3176122334}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":110,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:40:02.319282+0000","modified":"2024-09-21T03:40:42.053470+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":101,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-21T03:40:42.783 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 110 2024-09-21T03:40:42.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:42.783+0000 7f7776e0a640 1 -- 172.21.15.38:0/518066249 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f775c078b90 msgr2=0x7f775c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:42.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:42.783+0000 7f7776e0a640 1 --2- 172.21.15.38:0/518066249 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f775c078b90 0x7f775c07b0a0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f7760033000 tx=0x7f7760004730 comp rx=0 tx=0).stop 2024-09-21T03:40:42.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:42.783+0000 7f7776e0a640 1 -- 172.21.15.38:0/518066249 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f77700ac2d0 msgr2=0x7f77700ac6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:42.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:42.783+0000 7f7776e0a640 1 --2- 172.21.15.38:0/518066249 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f77700ac2d0 0x7f77700ac6e0 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f776800e9c0 tx=0x7f776800ee80 comp rx=0 tx=0).stop 2024-09-21T03:40:42.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:42.783+0000 7f7776e0a640 1 -- 172.21.15.38:0/518066249 shutdown_connections 2024-09-21T03:40:42.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:42.783+0000 7f7776e0a640 1 --2- 172.21.15.38:0/518066249 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f77700ac2d0 0x7f77700ac6e0 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:42.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:42.783+0000 7f7776e0a640 1 --2- 172.21.15.38:0/518066249 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7770162280 0x7f77700abd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:42.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:42.783+0000 7f7776e0a640 1 --2- 172.21.15.38:0/518066249 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f775c078b90 0x7f775c07b0a0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:42.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:42.783+0000 7f7776e0a640 1 --2- 172.21.15.38:0/518066249 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f77700b2e40 0x7f77700cef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:42.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:42.783+0000 7f7776e0a640 1 -- 172.21.15.38:0/518066249 >> 172.21.15.38:0/518066249 conn(0x7f77700b6b20 msgr2=0x7f777006dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:42.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:42.783+0000 7f7776e0a640 1 -- 172.21.15.38:0/518066249 shutdown_connections 2024-09-21T03:40:42.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:42.783+0000 7f7776e0a640 1 -- 172.21.15.38:0/518066249 wait complete. 2024-09-21T03:40:42.796 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-21T03:40:42.797 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:40:42.836 INFO:tasks.ceph.mds.a.smithi038.stdout:starting mds.a at 2024-09-21T03:40:42.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:42.914+0000 7f572d332640 1 -- 172.21.15.38:0/4156994677 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5728162590 msgr2=0x7f57281629a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:42.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:42.914+0000 7f572d332640 1 --2- 172.21.15.38:0/4156994677 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5728162590 0x7f57281629a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f5710009830 tx=0x7f571002a510 comp rx=0 tx=0).stop 2024-09-21T03:40:42.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:42.914+0000 7f572d332640 1 -- 172.21.15.38:0/4156994677 shutdown_connections 2024-09-21T03:40:42.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:42.914+0000 7f572d332640 1 --2- 172.21.15.38:0/4156994677 >> v2:172.21.15.195:3300/0 conn(0x7f57280b2e60 0x7f57280b3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:42.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:42.914+0000 7f572d332640 1 --2- 172.21.15.38:0/4156994677 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5728162590 0x7f57281629a0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:42.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:42.914+0000 7f572d332640 1 -- 172.21.15.38:0/4156994677 >> 172.21.15.38:0/4156994677 conn(0x7f57280d4440 msgr2=0x7f57280d4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:42.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:42.914+0000 7f572d332640 1 -- 172.21.15.38:0/4156994677 shutdown_connections 2024-09-21T03:40:42.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:42.914+0000 7f572d332640 1 -- 172.21.15.38:0/4156994677 wait complete. 2024-09-21T03:40:42.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:42.916+0000 7f572d332640 1 Processor -- start 2024-09-21T03:40:42.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:42.916+0000 7f572d332640 1 -- start start 2024-09-21T03:40:42.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:42.918+0000 7f572d332640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f57280b2e60 0x7f57280c8d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:42.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:42.918+0000 7f572d332640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f57280cfe00 0x7f572815be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:42.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:42.918+0000 7f572d332640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5728162590 0x7f572815d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:42.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:42.918+0000 7f572d332640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f57280b3930 con 0x7f57280b2e60 2024-09-21T03:40:42.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:42.918+0000 7f572d332640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f57280b37b0 con 0x7f5728162590 2024-09-21T03:40:42.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:42.918+0000 7f572d332640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f57280b3ab0 con 0x7f57280cfe00 2024-09-21T03:40:42.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:42.918+0000 7f5726ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f57280b2e60 0x7f57280c8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:42.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:42.918+0000 7f5726ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f57280b2e60 0x7f57280c8d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:58774/0 (socket says 172.21.15.38:58774) 2024-09-21T03:40:42.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:42.918+0000 7f5726ffd640 1 -- 172.21.15.38:0/2647375979 learned_addr learned my addr 172.21.15.38:0/2647375979 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:42.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:42.918+0000 7f571ffff640 1 --2- 172.21.15.38:0/2647375979 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f57280cfe00 0x7f572815be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:42.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:42.918+0000 7f57277fe640 1 --2- 172.21.15.38:0/2647375979 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5728162590 0x7f572815d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:42.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:42.918+0000 7f5726ffd640 1 -- 172.21.15.38:0/2647375979 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f57280cfe00 msgr2=0x7f572815be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:42.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:42.918+0000 7f5726ffd640 1 --2- 172.21.15.38:0/2647375979 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f57280cfe00 0x7f572815be80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:42.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:42.918+0000 7f5726ffd640 1 -- 172.21.15.38:0/2647375979 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5728162590 msgr2=0x7f572815d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:42.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:42.918+0000 7f5726ffd640 1 --2- 172.21.15.38:0/2647375979 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5728162590 0x7f572815d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:42.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:42.918+0000 7f5726ffd640 1 -- 172.21.15.38:0/2647375979 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57280c8010 con 0x7f57280b2e60 2024-09-21T03:40:42.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:42.919+0000 7f5726ffd640 1 --2- 172.21.15.38:0/2647375979 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f57280b2e60 0x7f57280c8d90 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f571800c730 tx=0x7f571800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:42.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:42.919+0000 7f571f7fe640 1 -- 172.21.15.38:0/2647375979 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5718006890 con 0x7f57280b2e60 2024-09-21T03:40:42.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:42.919+0000 7f571f7fe640 1 -- 172.21.15.38:0/2647375979 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5718006a30 con 0x7f57280b2e60 2024-09-21T03:40:42.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:42.919+0000 7f572d332640 1 -- 172.21.15.38:0/2647375979 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57280c82b0 con 0x7f57280b2e60 2024-09-21T03:40:42.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:42.919+0000 7f571f7fe640 1 -- 172.21.15.38:0/2647375979 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5718007030 con 0x7f57280b2e60 2024-09-21T03:40:42.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:42.919+0000 7f572d332640 1 -- 172.21.15.38:0/2647375979 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f572815e620 con 0x7f57280b2e60 2024-09-21T03:40:42.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:42.920+0000 7f572d332640 1 -- 172.21.15.38:0/2647375979 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57280b3270 con 0x7f57280b2e60 2024-09-21T03:40:42.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:42.921+0000 7f571f7fe640 1 -- 172.21.15.38:0/2647375979 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f57180071d0 con 0x7f57280b2e60 2024-09-21T03:40:42.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:42.921+0000 7f571f7fe640 1 --2- 172.21.15.38:0/2647375979 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5708078bc0 0x7f570807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:42.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:42.921+0000 7f571f7fe640 1 -- 172.21.15.38:0/2647375979 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 6734+0+0 (secure 0 0 0) 0x7f571809aa60 con 0x7f57280b2e60 2024-09-21T03:40:42.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:42.921+0000 7f571ffff640 1 --2- 172.21.15.38:0/2647375979 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5708078bc0 0x7f570807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:42.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:42.922+0000 7f571ffff640 1 --2- 172.21.15.38:0/2647375979 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5708078bc0 0x7f570807b0d0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f5710034000 tx=0x7f5710004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:42.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:42.926+0000 7f571f7fe640 1 -- 172.21.15.38:0/2647375979 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f5728162ee0 con 0x7f57280b2e60 2024-09-21T03:40:43.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:43.151+0000 7f572d332640 1 -- 172.21.15.38:0/2647375979 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f57280d49f0 con 0x7f57280b2e60 2024-09-21T03:40:43.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:43.152+0000 7f571f7fe640 1 -- 172.21.15.38:0/2647375979 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 111 v111) ==== 95+0+4389 (secure 0 0 0) 0x7f5718062450 con 0x7f57280b2e60 2024-09-21T03:40:43.154 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:40:43.154 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":111,"btime":"2024-09-21T03:40:43:054993+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6086,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/965972663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":965972663},{"type":"v1","addr":"172.21.15.195:6836","nonce":965972663}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6700,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/3176122334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":3176122334},{"type":"v1","addr":"172.21.15.195:6837","nonce":3176122334}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":7926,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/425672974","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":425672974},{"type":"v1","addr":"172.21.15.38:6835","nonce":425672974}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111}],"filesystems":[{"mdsmap":{"epoch":110,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:40:02.319282+0000","modified":"2024-09-21T03:40:42.053470+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":101,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-21T03:40:43.154 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 111 2024-09-21T03:40:43.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:43.154+0000 7f572d332640 1 -- 172.21.15.38:0/2647375979 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5708078bc0 msgr2=0x7f570807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:43.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:43.154+0000 7f572d332640 1 --2- 172.21.15.38:0/2647375979 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5708078bc0 0x7f570807b0d0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f5710034000 tx=0x7f5710004730 comp rx=0 tx=0).stop 2024-09-21T03:40:43.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:43.154+0000 7f572d332640 1 -- 172.21.15.38:0/2647375979 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f57280b2e60 msgr2=0x7f57280c8d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:43.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:43.154+0000 7f572d332640 1 --2- 172.21.15.38:0/2647375979 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f57280b2e60 0x7f57280c8d90 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f571800c730 tx=0x7f571800cbf0 comp rx=0 tx=0).stop 2024-09-21T03:40:43.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:43.154+0000 7f572d332640 1 -- 172.21.15.38:0/2647375979 shutdown_connections 2024-09-21T03:40:43.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:43.154+0000 7f572d332640 1 --2- 172.21.15.38:0/2647375979 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5728162590 0x7f572815d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:43.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:43.154+0000 7f572d332640 1 --2- 172.21.15.38:0/2647375979 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5708078bc0 0x7f570807b0d0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:43.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:43.154+0000 7f572d332640 1 --2- 172.21.15.38:0/2647375979 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f57280cfe00 0x7f572815be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:43.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:43.154+0000 7f572d332640 1 --2- 172.21.15.38:0/2647375979 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f57280b2e60 0x7f57280c8d90 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:43.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:43.154+0000 7f572d332640 1 -- 172.21.15.38:0/2647375979 >> 172.21.15.38:0/2647375979 conn(0x7f57280d4440 msgr2=0x7f572806d7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:43.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:43.154+0000 7f572d332640 1 -- 172.21.15.38:0/2647375979 shutdown_connections 2024-09-21T03:40:43.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:43.154+0000 7f572d332640 1 -- 172.21.15.38:0/2647375979 wait complete. 2024-09-21T03:40:43.167 INFO:tasks.cephfs.test_damage:Result: Mutation 'Delete 200.00000000' led to DAMAGED state 2024-09-21T03:40:43.167 INFO:tasks.cephfs.test_damage:Applying mutation 'Delete 200.00000001' 2024-09-21T03:40:43.167 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-21T03:40:43.167 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:40:43.224 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056795,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2354 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T03:40:43.224 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi038.front.sepia.ceph.com 2024-09-21T03:40:43.224 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:40:43.224 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T03:40:43.279 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T03:40:43.279 INFO:teuthology.orchestra.run.smithi038.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-09-21T03:40:43.280 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-21T03:40:43.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:43.435+0000 7f69c11e6640 1 -- 172.21.15.38:0/1148338124 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f69bc15ee80 msgr2=0x7f69bc0d0170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:43.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:43.435+0000 7f69c11e6640 1 --2- 172.21.15.38:0/1148338124 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f69bc15ee80 0x7f69bc0d0170 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f69bc0d06b0 tx=0x7f69ac02a410 comp rx=0 tx=0).stop 2024-09-21T03:40:43.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:43.435+0000 7f69c11e6640 1 -- 172.21.15.38:0/1148338124 shutdown_connections 2024-09-21T03:40:43.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:43.435+0000 7f69c11e6640 1 --2- 172.21.15.38:0/1148338124 >> v2:172.21.15.195:3300/0 conn(0x7f69bc0b9ad0 0x7f69bc0d0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:43.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:43.435+0000 7f69c11e6640 1 --2- 172.21.15.38:0/1148338124 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f69bc15ee80 0x7f69bc0d0170 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:43.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:43.435+0000 7f69c11e6640 1 -- 172.21.15.38:0/1148338124 >> 172.21.15.38:0/1148338124 conn(0x7f69bc09d830 msgr2=0x7f69bc0b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:43.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:43.435+0000 7f69c11e6640 1 -- 172.21.15.38:0/1148338124 shutdown_connections 2024-09-21T03:40:43.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:43.435+0000 7f69c11e6640 1 -- 172.21.15.38:0/1148338124 wait complete. 2024-09-21T03:40:43.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:43.437+0000 7f69c11e6640 1 Processor -- start 2024-09-21T03:40:43.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:43.437+0000 7f69c11e6640 1 -- start start 2024-09-21T03:40:43.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:43.438+0000 7f69c11e6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f69bc0b9ad0 0x7f69bc06cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:43.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:43.438+0000 7f69c11e6640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f69bc0d1120 0x7f69bc06d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:43.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:43.438+0000 7f69c11e6640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f69bc15ee80 0x7f69bc1f55b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:43.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:43.438+0000 7f69c11e6640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f69bc0b6700 con 0x7f69bc0b9ad0 2024-09-21T03:40:43.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:43.438+0000 7f69c11e6640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f69bc0b6580 con 0x7f69bc0d1120 2024-09-21T03:40:43.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:43.438+0000 7f69c11e6640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f69bc0b6880 con 0x7f69bc15ee80 2024-09-21T03:40:43.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:43.438+0000 7f69bad9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f69bc0b9ad0 0x7f69bc06cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:43.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:43.438+0000 7f69bad9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f69bc0b9ad0 0x7f69bc06cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:58794/0 (socket says 172.21.15.38:58794) 2024-09-21T03:40:43.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:43.438+0000 7f69bad9d640 1 -- 172.21.15.38:0/2997529423 learned_addr learned my addr 172.21.15.38:0/2997529423 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:43.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:43.439+0000 7f69ba59c640 1 --2- 172.21.15.38:0/2997529423 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f69bc0d1120 0x7f69bc06d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:43.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:43.439+0000 7f69bb59e640 1 --2- 172.21.15.38:0/2997529423 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f69bc15ee80 0x7f69bc1f55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:43.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:43.439+0000 7f69bad9d640 1 -- 172.21.15.38:0/2997529423 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f69bc15ee80 msgr2=0x7f69bc1f55b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:43.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:43.439+0000 7f69bad9d640 1 --2- 172.21.15.38:0/2997529423 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f69bc15ee80 0x7f69bc1f55b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:43.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:43.439+0000 7f69bad9d640 1 -- 172.21.15.38:0/2997529423 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f69bc0d1120 msgr2=0x7f69bc06d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:43.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:43.439+0000 7f69bad9d640 1 --2- 172.21.15.38:0/2997529423 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f69bc0d1120 0x7f69bc06d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:43.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:43.439+0000 7f69bad9d640 1 -- 172.21.15.38:0/2997529423 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69bc06fc40 con 0x7f69bc0b9ad0 2024-09-21T03:40:43.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:43.439+0000 7f69ba59c640 1 --2- 172.21.15.38:0/2997529423 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f69bc0d1120 0x7f69bc06d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:40:43.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:43.439+0000 7f69bad9d640 1 --2- 172.21.15.38:0/2997529423 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f69bc0b9ad0 0x7f69bc06cde0 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f69ac009920 tx=0x7f69ac033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:43.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:43.439+0000 7f69a37fe640 1 -- 172.21.15.38:0/2997529423 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69ac031d60 con 0x7f69bc0b9ad0 2024-09-21T03:40:43.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:43.439+0000 7f69a37fe640 1 -- 172.21.15.38:0/2997529423 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f69ac033e10 con 0x7f69bc0b9ad0 2024-09-21T03:40:43.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:43.439+0000 7f69c11e6640 1 -- 172.21.15.38:0/2997529423 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69bc06f680 con 0x7f69bc0b9ad0 2024-09-21T03:40:43.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:43.439+0000 7f69a37fe640 1 -- 172.21.15.38:0/2997529423 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69ac005260 con 0x7f69bc0b9ad0 2024-09-21T03:40:43.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:43.439+0000 7f69c11e6640 1 -- 172.21.15.38:0/2997529423 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69bc0b1830 con 0x7f69bc0b9ad0 2024-09-21T03:40:43.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:43.440+0000 7f69c11e6640 1 -- 172.21.15.38:0/2997529423 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69bc0b6d30 con 0x7f69bc0b9ad0 2024-09-21T03:40:43.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:43.441+0000 7f69a37fe640 1 -- 172.21.15.38:0/2997529423 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f69ac005400 con 0x7f69bc0b9ad0 2024-09-21T03:40:43.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:43.442+0000 7f69a37fe640 1 --2- 172.21.15.38:0/2997529423 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f69a8078b90 0x7f69a807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:43.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:43.442+0000 7f69a37fe640 1 -- 172.21.15.38:0/2997529423 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 6734+0+0 (secure 0 0 0) 0x7f69ac0bc470 con 0x7f69bc0b9ad0 2024-09-21T03:40:43.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:43.442+0000 7f69ba59c640 1 --2- 172.21.15.38:0/2997529423 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f69a8078b90 0x7f69a807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:43.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:43.442+0000 7f69ba59c640 1 --2- 172.21.15.38:0/2997529423 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f69a8078b90 0x7f69a807b0a0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f69b0005860 tx=0x7f69b0005de0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:43.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:43.446+0000 7f69a37fe640 1 -- 172.21.15.38:0/2997529423 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f69ac03f030 con 0x7f69bc0b9ad0 2024-09-21T03:40:43.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:43.671+0000 7f69c11e6640 1 -- 172.21.15.38:0/2997529423 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f69bc160b60 con 0x7f69bc0b9ad0 2024-09-21T03:40:44.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:44.071+0000 7f69a37fe640 1 -- 172.21.15.38:0/2997529423 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v112) ==== 161+0+0 (secure 0 0 0) 0x7f69ac084420 con 0x7f69bc0b9ad0 2024-09-21T03:40:44.073 INFO:teuthology.orchestra.run.smithi038.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-21T03:40:44.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:44.073+0000 7f69c11e6640 1 -- 172.21.15.38:0/2997529423 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f69a8078b90 msgr2=0x7f69a807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:44.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:44.073+0000 7f69c11e6640 1 --2- 172.21.15.38:0/2997529423 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f69a8078b90 0x7f69a807b0a0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f69b0005860 tx=0x7f69b0005de0 comp rx=0 tx=0).stop 2024-09-21T03:40:44.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:44.073+0000 7f69c11e6640 1 -- 172.21.15.38:0/2997529423 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f69bc0b9ad0 msgr2=0x7f69bc06cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:44.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:44.073+0000 7f69c11e6640 1 --2- 172.21.15.38:0/2997529423 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f69bc0b9ad0 0x7f69bc06cde0 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f69ac009920 tx=0x7f69ac033c20 comp rx=0 tx=0).stop 2024-09-21T03:40:44.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:44.074+0000 7f69c11e6640 1 -- 172.21.15.38:0/2997529423 shutdown_connections 2024-09-21T03:40:44.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:44.074+0000 7f69c11e6640 1 --2- 172.21.15.38:0/2997529423 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f69a8078b90 0x7f69a807b0a0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:44.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:44.074+0000 7f69c11e6640 1 --2- 172.21.15.38:0/2997529423 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f69bc15ee80 0x7f69bc1f55b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:44.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:44.074+0000 7f69c11e6640 1 --2- 172.21.15.38:0/2997529423 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f69bc0d1120 0x7f69bc06d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:44.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:44.074+0000 7f69c11e6640 1 --2- 172.21.15.38:0/2997529423 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f69bc0b9ad0 0x7f69bc06cde0 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:44.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:44.074+0000 7f69c11e6640 1 -- 172.21.15.38:0/2997529423 >> 172.21.15.38:0/2997529423 conn(0x7f69bc09d830 msgr2=0x7f69bc15e320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:44.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:44.074+0000 7f69c11e6640 1 -- 172.21.15.38:0/2997529423 shutdown_connections 2024-09-21T03:40:44.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:44.074+0000 7f69c11e6640 1 -- 172.21.15.38:0/2997529423 wait complete. 2024-09-21T03:40:44.088 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-09-21T03:40:44.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:44.208+0000 7f7930fda640 1 -- 172.21.15.38:0/3571712616 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f79140053a0 msgr2=0x7f791400ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:44.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:44.208+0000 7f7930fda640 1 --2- 172.21.15.38:0/3571712616 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f79140053a0 0x7f791400ba50 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f7924009ed0 tx=0x7f7914005830 comp rx=0 tx=0).stop 2024-09-21T03:40:44.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:44.208+0000 7f7930fda640 1 -- 172.21.15.38:0/3571712616 shutdown_connections 2024-09-21T03:40:44.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:44.208+0000 7f7930fda640 1 --2- 172.21.15.38:0/3571712616 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f79140053a0 0x7f791400ba50 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:44.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:44.208+0000 7f7930fda640 1 -- 172.21.15.38:0/3571712616 >> 172.21.15.38:0/3571712616 conn(0x7f792c09d830 msgr2=0x7f792c0b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:44.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:44.208+0000 7f7930fda640 1 -- 172.21.15.38:0/3571712616 shutdown_connections 2024-09-21T03:40:44.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:44.208+0000 7f7930fda640 1 -- 172.21.15.38:0/3571712616 wait complete. 2024-09-21T03:40:44.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:44.210+0000 7f7930fda640 1 Processor -- start 2024-09-21T03:40:44.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:44.210+0000 7f7930fda640 1 -- start start 2024-09-21T03:40:44.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:44.211+0000 7f7930fda640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f79140053a0 0x7f792c0b57a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:44.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:44.211+0000 7f7930fda640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f792c157910 0x7f792c0cc000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:44.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:44.211+0000 7f7930fda640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f792c157d20 0x7f792c0cc540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:44.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:44.211+0000 7f7930fda640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f792c0b9ad0 con 0x7f79140053a0 2024-09-21T03:40:44.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:44.211+0000 7f7930fda640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f792c0cf310 con 0x7f792c157910 2024-09-21T03:40:44.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:44.211+0000 7f7930fda640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f792c0b9c50 con 0x7f792c157d20 2024-09-21T03:40:44.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:44.211+0000 7f792a59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f79140053a0 0x7f792c0b57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:44.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:44.211+0000 7f792a59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f79140053a0 0x7f792c0b57a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:58802/0 (socket says 172.21.15.38:58802) 2024-09-21T03:40:44.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:44.211+0000 7f792a59c640 1 -- 172.21.15.38:0/2998042621 learned_addr learned my addr 172.21.15.38:0/2998042621 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:44.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:44.212+0000 7f7929d9b640 1 --2- 172.21.15.38:0/2998042621 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f792c157910 0x7f792c0cc000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:44.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:44.212+0000 7f792ad9d640 1 --2- 172.21.15.38:0/2998042621 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f792c157d20 0x7f792c0cc540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:44.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:44.212+0000 7f792a59c640 1 -- 172.21.15.38:0/2998042621 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f792c157d20 msgr2=0x7f792c0cc540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:44.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:44.212+0000 7f792a59c640 1 --2- 172.21.15.38:0/2998042621 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f792c157d20 0x7f792c0cc540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:44.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:44.212+0000 7f792a59c640 1 -- 172.21.15.38:0/2998042621 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f792c157910 msgr2=0x7f792c0cc000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:44.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:44.212+0000 7f792a59c640 1 --2- 172.21.15.38:0/2998042621 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f792c157910 0x7f792c0cc000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:44.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:44.212+0000 7f792a59c640 1 -- 172.21.15.38:0/2998042621 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f792c0b5ce0 con 0x7f79140053a0 2024-09-21T03:40:44.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:44.212+0000 7f792a59c640 1 --2- 172.21.15.38:0/2998042621 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f79140053a0 0x7f792c0b57a0 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f791c0068e0 tx=0x7f791c0039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:44.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:44.212+0000 7f791affd640 1 -- 172.21.15.38:0/2998042621 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f791c015070 con 0x7f79140053a0 2024-09-21T03:40:44.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:44.212+0000 7f791affd640 1 -- 172.21.15.38:0/2998042621 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f791c011840 con 0x7f79140053a0 2024-09-21T03:40:44.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:44.212+0000 7f791affd640 1 -- 172.21.15.38:0/2998042621 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f791c011c90 con 0x7f79140053a0 2024-09-21T03:40:44.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:44.212+0000 7f7930fda640 1 -- 172.21.15.38:0/2998042621 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f792c0ce230 con 0x7f79140053a0 2024-09-21T03:40:44.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:44.212+0000 7f7930fda640 1 -- 172.21.15.38:0/2998042621 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f792c0ce7f0 con 0x7f79140053a0 2024-09-21T03:40:44.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:44.213+0000 7f7930fda640 1 -- 172.21.15.38:0/2998042621 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f792c0d0be0 con 0x7f79140053a0 2024-09-21T03:40:44.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:44.214+0000 7f791affd640 1 -- 172.21.15.38:0/2998042621 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f791c018070 con 0x7f79140053a0 2024-09-21T03:40:44.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:44.215+0000 7f791affd640 1 --2- 172.21.15.38:0/2998042621 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f790c078b90 0x7f790c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:44.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:44.215+0000 7f791affd640 1 -- 172.21.15.38:0/2998042621 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 6734+0+0 (secure 0 0 0) 0x7f791c09bdd0 con 0x7f79140053a0 2024-09-21T03:40:44.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:44.215+0000 7f7929d9b640 1 --2- 172.21.15.38:0/2998042621 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f790c078b90 0x7f790c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:44.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:44.216+0000 7f7929d9b640 1 --2- 172.21.15.38:0/2998042621 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f790c078b90 0x7f790c07b0a0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f7920000c00 tx=0x7f7920006bc0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:44.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:44.219+0000 7f791affd640 1 -- 172.21.15.38:0/2998042621 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f792c1592d0 con 0x7f79140053a0 2024-09-21T03:40:44.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:44.486+0000 7f7930fda640 1 -- 172.21.15.38:0/2998042621 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7f792c161a60 con 0x7f79140053a0 2024-09-21T03:40:45.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.077+0000 7f791affd640 1 -- 172.21.15.38:0/2998042621 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v113) ==== 101+0+0 (secure 0 0 0) 0x7f791c063720 con 0x7f79140053a0 2024-09-21T03:40:45.079 INFO:teuthology.orchestra.run.smithi038.stderr:repaired: restoring rank 6:0 2024-09-21T03:40:45.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.079+0000 7f7930fda640 1 -- 172.21.15.38:0/2998042621 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f790c078b90 msgr2=0x7f790c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:45.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.079+0000 7f7930fda640 1 --2- 172.21.15.38:0/2998042621 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f790c078b90 0x7f790c07b0a0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f7920000c00 tx=0x7f7920006bc0 comp rx=0 tx=0).stop 2024-09-21T03:40:45.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.079+0000 7f7930fda640 1 -- 172.21.15.38:0/2998042621 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f79140053a0 msgr2=0x7f792c0b57a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:45.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.079+0000 7f7930fda640 1 --2- 172.21.15.38:0/2998042621 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f79140053a0 0x7f792c0b57a0 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f791c0068e0 tx=0x7f791c0039c0 comp rx=0 tx=0).stop 2024-09-21T03:40:45.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.079+0000 7f7930fda640 1 -- 172.21.15.38:0/2998042621 shutdown_connections 2024-09-21T03:40:45.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.079+0000 7f7930fda640 1 --2- 172.21.15.38:0/2998042621 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f790c078b90 0x7f790c07b0a0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:45.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.079+0000 7f7930fda640 1 --2- 172.21.15.38:0/2998042621 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f792c157d20 0x7f792c0cc540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:45.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.079+0000 7f7930fda640 1 --2- 172.21.15.38:0/2998042621 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f792c157910 0x7f792c0cc000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:45.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.079+0000 7f7930fda640 1 --2- 172.21.15.38:0/2998042621 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f79140053a0 0x7f792c0b57a0 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:45.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.079+0000 7f7930fda640 1 -- 172.21.15.38:0/2998042621 >> 172.21.15.38:0/2998042621 conn(0x7f792c09d830 msgr2=0x7f792c15c9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:45.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.080+0000 7f7930fda640 1 -- 172.21.15.38:0/2998042621 shutdown_connections 2024-09-21T03:40:45.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.080+0000 7f7930fda640 1 -- 172.21.15.38:0/2998042621 wait complete. 2024-09-21T03:40:45.096 DEBUG:teuthology.orchestra.run.smithi038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-09-21T03:40:45.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.155+0000 7f207216df40 1 Processor -- start 2024-09-21T03:40:45.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.155+0000 7f207216df40 1 -- start start 2024-09-21T03:40:45.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.156+0000 7f207216df40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f20540030f0 0x561f4c6e70c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:45.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.156+0000 7f207216df40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561f4c7786f0 0x561f4c81cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:45.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.156+0000 7f207216df40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x561f4c774a60 0x561f4c81d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:45.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.156+0000 7f207216df40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x561f4c66a430 con 0x7f20540030f0 2024-09-21T03:40:45.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.156+0000 7f207216df40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x561f4c6771f0 con 0x561f4c7786f0 2024-09-21T03:40:45.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.156+0000 7f207216df40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x561f4c60bd60 con 0x561f4c774a60 2024-09-21T03:40:45.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.156+0000 7f206bc27640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f20540030f0 0x561f4c6e70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:45.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.156+0000 7f206bc27640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f20540030f0 0x561f4c6e70c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:58820/0 (socket says 172.21.15.38:58820) 2024-09-21T03:40:45.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.156+0000 7f206bc27640 1 -- 172.21.15.38:0/249454632 learned_addr learned my addr 172.21.15.38:0/249454632 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:45.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.157+0000 7f206b426640 1 --2- 172.21.15.38:0/249454632 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561f4c7786f0 0x561f4c81cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:45.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.157+0000 7f206c428640 1 --2- 172.21.15.38:0/249454632 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x561f4c774a60 0x561f4c81d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:45.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.157+0000 7f206bc27640 1 -- 172.21.15.38:0/249454632 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x561f4c774a60 msgr2=0x561f4c81d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:45.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.157+0000 7f206bc27640 1 --2- 172.21.15.38:0/249454632 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x561f4c774a60 0x561f4c81d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:45.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.157+0000 7f206bc27640 1 -- 172.21.15.38:0/249454632 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561f4c7786f0 msgr2=0x561f4c81cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:45.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.157+0000 7f206bc27640 1 --2- 172.21.15.38:0/249454632 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561f4c7786f0 0x561f4c81cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:45.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.157+0000 7f206bc27640 1 -- 172.21.15.38:0/249454632 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f4c6e7e70 con 0x7f20540030f0 2024-09-21T03:40:45.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.157+0000 7f206c428640 1 --2- 172.21.15.38:0/249454632 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x561f4c774a60 0x561f4c81d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:40:45.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.157+0000 7f206b426640 1 --2- 172.21.15.38:0/249454632 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561f4c7786f0 0x561f4c81cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:40:45.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.157+0000 7f206bc27640 1 --2- 172.21.15.38:0/249454632 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f20540030f0 0x561f4c6e70c0 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f205c00b790 tx=0x7f205c00bc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ab13401a4d95ea server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:45.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.157+0000 7f206a424640 1 -- 172.21.15.38:0/249454632 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f205c01a070 con 0x7f20540030f0 2024-09-21T03:40:45.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.157+0000 7f206a424640 1 -- 172.21.15.38:0/249454632 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f205c00ae20 con 0x7f20540030f0 2024-09-21T03:40:45.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.157+0000 7f206a424640 1 -- 172.21.15.38:0/249454632 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f205c008690 con 0x7f20540030f0 2024-09-21T03:40:45.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.158+0000 7f207216df40 1 -- 172.21.15.38:0/249454632 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f20540030f0 msgr2=0x561f4c6e70c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:45.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.158+0000 7f207216df40 1 --2- 172.21.15.38:0/249454632 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f20540030f0 0x561f4c6e70c0 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f205c00b790 tx=0x7f205c00bc50 comp rx=0 tx=0).stop 2024-09-21T03:40:45.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.158+0000 7f207216df40 1 -- 172.21.15.38:0/249454632 shutdown_connections 2024-09-21T03:40:45.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.158+0000 7f207216df40 1 --2- 172.21.15.38:0/249454632 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x561f4c774a60 0x561f4c81d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:45.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.158+0000 7f207216df40 1 --2- 172.21.15.38:0/249454632 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561f4c7786f0 0x561f4c81cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:45.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.158+0000 7f207216df40 1 --2- 172.21.15.38:0/249454632 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f20540030f0 0x561f4c6e70c0 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:45.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.158+0000 7f207216df40 1 -- 172.21.15.38:0/249454632 >> 172.21.15.38:0/249454632 conn(0x561f4c5dcac0 msgr2=0x561f4c78f0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:45.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.158+0000 7f207216df40 1 -- 172.21.15.38:0/249454632 shutdown_connections 2024-09-21T03:40:45.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.158+0000 7f207216df40 1 -- 172.21.15.38:0/249454632 wait complete. 2024-09-21T03:40:45.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.159+0000 7f207216df40 1 Processor -- start 2024-09-21T03:40:45.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.159+0000 7f207216df40 1 -- start start 2024-09-21T03:40:45.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.160+0000 7f207216df40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561f4c774a60 0x561f4c774350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:45.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.160+0000 7f207216df40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x561f4c7786f0 0x561f4c78df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:45.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.160+0000 7f207216df40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f20540030f0 0x561f4c78e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:45.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.160+0000 7f207216df40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x561f4c6e8150 con 0x7f20540030f0 2024-09-21T03:40:45.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.160+0000 7f207216df40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x561f4c81c6c0 con 0x561f4c774a60 2024-09-21T03:40:45.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.160+0000 7f207216df40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x561f4c81a750 con 0x561f4c7786f0 2024-09-21T03:40:45.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.160+0000 7f206c428640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f20540030f0 0x561f4c78e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:45.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.160+0000 7f206c428640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f20540030f0 0x561f4c78e460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:58834/0 (socket says 172.21.15.38:58834) 2024-09-21T03:40:45.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.160+0000 7f206c428640 1 -- 172.21.15.38:0/2306811490 learned_addr learned my addr 172.21.15.38:0/2306811490 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:45.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.160+0000 7f206bc27640 1 --2- 172.21.15.38:0/2306811490 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561f4c774a60 0x561f4c774350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:45.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.160+0000 7f206b426640 1 --2- 172.21.15.38:0/2306811490 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x561f4c7786f0 0x561f4c78df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:45.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.161+0000 7f206c428640 1 -- 172.21.15.38:0/2306811490 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x561f4c7786f0 msgr2=0x561f4c78df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:45.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.161+0000 7f206c428640 1 --2- 172.21.15.38:0/2306811490 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x561f4c7786f0 0x561f4c78df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:45.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.161+0000 7f206c428640 1 -- 172.21.15.38:0/2306811490 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561f4c774a60 msgr2=0x561f4c774350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:45.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.161+0000 7f206c428640 1 --2- 172.21.15.38:0/2306811490 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561f4c774a60 0x561f4c774350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:45.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.161+0000 7f206c428640 1 -- 172.21.15.38:0/2306811490 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f4c704af0 con 0x7f20540030f0 2024-09-21T03:40:45.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.161+0000 7f206b426640 1 --2- 172.21.15.38:0/2306811490 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x561f4c7786f0 0x561f4c78df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:40:45.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.161+0000 7f206c428640 1 --2- 172.21.15.38:0/2306811490 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f20540030f0 0x561f4c78e460 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x561f4c6bfa40 tx=0x7f2064005ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:45.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.161+0000 7f2069422640 1 -- 172.21.15.38:0/2306811490 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2064003920 con 0x7f20540030f0 2024-09-21T03:40:45.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.161+0000 7f2069422640 1 -- 172.21.15.38:0/2306811490 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2064003ac0 con 0x7f20540030f0 2024-09-21T03:40:45.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.161+0000 7f207216df40 1 -- 172.21.15.38:0/2306811490 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x561f4c817ed0 con 0x7f20540030f0 2024-09-21T03:40:45.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.161+0000 7f2069422640 1 -- 172.21.15.38:0/2306811490 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2064015410 con 0x7f20540030f0 2024-09-21T03:40:45.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.161+0000 7f207216df40 1 -- 172.21.15.38:0/2306811490 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f4c817940 con 0x7f20540030f0 2024-09-21T03:40:45.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.163+0000 7f2069422640 1 -- 172.21.15.38:0/2306811490 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f2064013070 con 0x7f20540030f0 2024-09-21T03:40:45.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.163+0000 7f2069422640 1 --2- 172.21.15.38:0/2306811490 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2058077570 0x7f2058079a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:45.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.163+0000 7f2069422640 1 -- 172.21.15.38:0/2306811490 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 6734+0+0 (secure 0 0 0) 0x7f2064099690 con 0x7f20540030f0 2024-09-21T03:40:45.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.163+0000 7f206bc27640 1 --2- 172.21.15.38:0/2306811490 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2058077570 0x7f2058079a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:45.167 INFO:teuthology.orchestra.run.smithi038.stdout:Importing pool 2024-09-21T03:40:45.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.164+0000 7f207216df40 1 --2- 172.21.15.38:0/2306811490 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x561f4c819bf0 0x561f4c834ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:45.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.164+0000 7f207216df40 1 -- 172.21.15.38:0/2306811490 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x561f4c778110 con 0x561f4c819bf0 2024-09-21T03:40:45.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.164+0000 7f206b426640 1 --2- 172.21.15.38:0/2306811490 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x561f4c819bf0 0x561f4c834ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:45.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.164+0000 7f206bc27640 1 --2- 172.21.15.38:0/2306811490 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2058077570 0x7f2058079a80 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f205c018040 tx=0x7f205c003020 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:45.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.164+0000 7f206b426640 1 --2- 172.21.15.38:0/2306811490 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x561f4c819bf0 0x561f4c834ba0 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:45.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.166+0000 7f206b426640 1 -- 172.21.15.38:0/2306811490 <== osd.7 v2:172.21.15.195:6818/1606106342 1 ==== osd_op_reply(1 601.00000000 [create] v101'133 uv107 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f206000b040 con 0x561f4c819bf0 2024-09-21T03:40:45.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.166+0000 7f207216df40 1 -- 172.21.15.38:0/2306811490 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e101) -- 0x561f4c81a000 con 0x561f4c819bf0 2024-09-21T03:40:45.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.168+0000 7f206b426640 1 -- 172.21.15.38:0/2306811490 <== osd.7 v2:172.21.15.195:6818/1606106342 2 ==== osd_op_reply(2 601.00000000 [delete] v101'134 uv134 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f206000b040 con 0x561f4c819bf0 2024-09-21T03:40:45.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.168+0000 7f207216df40 1 -- 172.21.15.38:0/2306811490 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x561f4c8350e0 con 0x561f4c819bf0 2024-09-21T03:40:45.171 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-09-21T03:40:45.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.169+0000 7f206b426640 1 -- 172.21.15.38:0/2306811490 <== osd.7 v2:172.21.15.195:6818/1606106342 3 ==== osd_op_reply(3 601.00000000 [create] v101'135 uv135 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f206000b040 con 0x561f4c819bf0 2024-09-21T03:40:45.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.169+0000 7f207216df40 1 -- 172.21.15.38:0/2306811490 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x561f4c836490 con 0x561f4c819bf0 2024-09-21T03:40:45.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.171+0000 7f206b426640 1 -- 172.21.15.38:0/2306811490 <== osd.7 v2:172.21.15.195:6818/1606106342 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v101'136 uv136 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f206000b040 con 0x561f4c819bf0 2024-09-21T03:40:45.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.171+0000 7f207216df40 1 -- 172.21.15.38:0/2306811490 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x561f4c836830 con 0x561f4c819bf0 2024-09-21T03:40:45.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.172+0000 7f206b426640 1 -- 172.21.15.38:0/2306811490 <== osd.7 v2:172.21.15.195:6818/1606106342 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v101'137 uv137 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f206000b040 con 0x561f4c819bf0 2024-09-21T03:40:45.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.172+0000 7f207216df40 1 -- 172.21.15.38:0/2306811490 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x561f4c835480 con 0x561f4c819bf0 2024-09-21T03:40:45.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.173+0000 7f206b426640 1 -- 172.21.15.38:0/2306811490 <== osd.7 v2:172.21.15.195:6818/1606106342 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v101'138 uv138 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f206000b040 con 0x561f4c819bf0 2024-09-21T03:40:45.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.174+0000 7f207216df40 1 -- 172.21.15.38:0/2306811490 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x561f4c835840 con 0x561f4c819bf0 2024-09-21T03:40:45.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.176+0000 7f206b426640 1 -- 172.21.15.38:0/2306811490 <== osd.7 v2:172.21.15.195:6818/1606106342 7 ==== osd_op_reply(7 602.00000000 [create] v101'139 uv113 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f206000b040 con 0x561f4c819bf0 2024-09-21T03:40:45.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.176+0000 7f207216df40 1 -- 172.21.15.38:0/2306811490 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e101) -- 0x561f4c835be0 con 0x561f4c819bf0 2024-09-21T03:40:45.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.177+0000 7f206b426640 1 -- 172.21.15.38:0/2306811490 <== osd.7 v2:172.21.15.195:6818/1606106342 8 ==== osd_op_reply(8 602.00000000 [delete] v101'140 uv140 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f206000b040 con 0x561f4c819bf0 2024-09-21T03:40:45.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.177+0000 7f207216df40 1 -- 172.21.15.38:0/2306811490 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x561f4c835be0 con 0x561f4c819bf0 2024-09-21T03:40:45.180 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-09-21T03:40:45.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.178+0000 7f206b426640 1 -- 172.21.15.38:0/2306811490 <== osd.7 v2:172.21.15.195:6818/1606106342 9 ==== osd_op_reply(9 602.00000000 [create] v101'141 uv141 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f206000b040 con 0x561f4c819bf0 2024-09-21T03:40:45.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.178+0000 7f207216df40 1 -- 172.21.15.38:0/2306811490 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x561f4c835be0 con 0x561f4c819bf0 2024-09-21T03:40:45.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.180+0000 7f206b426640 1 -- 172.21.15.38:0/2306811490 <== osd.7 v2:172.21.15.195:6818/1606106342 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v101'142 uv142 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f206000b040 con 0x561f4c819bf0 2024-09-21T03:40:45.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.180+0000 7f207216df40 1 -- 172.21.15.38:0/2306811490 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x561f4c835be0 con 0x561f4c819bf0 2024-09-21T03:40:45.183 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.181+0000 7f206b426640 1 -- 172.21.15.38:0/2306811490 <== osd.7 v2:172.21.15.195:6818/1606106342 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v101'143 uv143 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f206000b040 con 0x561f4c819bf0 2024-09-21T03:40:45.183 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.181+0000 7f207216df40 1 -- 172.21.15.38:0/2306811490 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x561f4c835be0 con 0x561f4c819bf0 2024-09-21T03:40:45.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.183+0000 7f206b426640 1 -- 172.21.15.38:0/2306811490 <== osd.7 v2:172.21.15.195:6818/1606106342 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v101'144 uv144 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f206000b040 con 0x561f4c819bf0 2024-09-21T03:40:45.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.183+0000 7f207216df40 1 -- 172.21.15.38:0/2306811490 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:13 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x561f4c835be0 con 0x561f4c819bf0 2024-09-21T03:40:45.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.185+0000 7f206b426640 1 -- 172.21.15.38:0/2306811490 <== osd.7 v2:172.21.15.195:6818/1606106342 13 ==== osd_op_reply(13 10000000000.00000000 [create] v101'145 uv120 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7f206000b040 con 0x561f4c819bf0 2024-09-21T03:40:45.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.185+0000 7f207216df40 1 -- 172.21.15.38:0/2306811490 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:14 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e101) -- 0x561f4c835be0 con 0x561f4c819bf0 2024-09-21T03:40:45.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.186+0000 7f206b426640 1 -- 172.21.15.38:0/2306811490 <== osd.7 v2:172.21.15.195:6818/1606106342 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v101'146 uv146 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f206000b040 con 0x561f4c819bf0 2024-09-21T03:40:45.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.186+0000 7f207216df40 1 -- 172.21.15.38:0/2306811490 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:15 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x561f4c835be0 con 0x561f4c819bf0 2024-09-21T03:40:45.189 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-09-21T03:40:45.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.187+0000 7f206b426640 1 -- 172.21.15.38:0/2306811490 <== osd.7 v2:172.21.15.195:6818/1606106342 15 ==== osd_op_reply(15 10000000000.00000000 [create] v101'147 uv147 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f206000b040 con 0x561f4c819bf0 2024-09-21T03:40:45.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.187+0000 7f207216df40 1 -- 172.21.15.38:0/2306811490 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:16 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x561f4c835be0 con 0x561f4c819bf0 2024-09-21T03:40:45.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.189+0000 7f206b426640 1 -- 172.21.15.38:0/2306811490 <== osd.7 v2:172.21.15.195:6818/1606106342 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v101'148 uv148 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f206000b040 con 0x561f4c819bf0 2024-09-21T03:40:45.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.189+0000 7f207216df40 1 -- 172.21.15.38:0/2306811490 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:17 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x561f4c835be0 con 0x561f4c819bf0 2024-09-21T03:40:45.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.190+0000 7f206b426640 1 -- 172.21.15.38:0/2306811490 <== osd.7 v2:172.21.15.195:6818/1606106342 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v101'149 uv149 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f206000b040 con 0x561f4c819bf0 2024-09-21T03:40:45.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.190+0000 7f207216df40 1 -- 172.21.15.38:0/2306811490 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:18 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x561f4c835be0 con 0x561f4c819bf0 2024-09-21T03:40:45.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.192+0000 7f206b426640 1 -- 172.21.15.38:0/2306811490 <== osd.7 v2:172.21.15.195:6818/1606106342 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v101'150 uv150 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f206000b040 con 0x561f4c819bf0 2024-09-21T03:40:45.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.192+0000 7f207216df40 1 -- 172.21.15.38:0/2306811490 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:19 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=585b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x561f4c835be0 con 0x561f4c819bf0 2024-09-21T03:40:45.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.193+0000 7f206b426640 1 -- 172.21.15.38:0/2306811490 <== osd.7 v2:172.21.15.195:6818/1606106342 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v101'151 uv151 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f206000b040 con 0x561f4c819bf0 2024-09-21T03:40:45.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.193+0000 7f207216df40 1 -- 172.21.15.38:0/2306811490 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:20 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x561f4c835be0 con 0x561f4c819bf0 2024-09-21T03:40:45.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.195+0000 7f206b426640 1 -- 172.21.15.38:0/2306811490 <== osd.7 v2:172.21.15.195:6818/1606106342 20 ==== osd_op_reply(20 600.00000000 [create] v101'152 uv126 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f206000b040 con 0x561f4c819bf0 2024-09-21T03:40:45.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.195+0000 7f207216df40 1 -- 172.21.15.38:0/2306811490 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:21 13.10 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e101) -- 0x561f4c835be0 con 0x561f4c819bf0 2024-09-21T03:40:45.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.197+0000 7f206b426640 1 -- 172.21.15.38:0/2306811490 <== osd.7 v2:172.21.15.195:6818/1606106342 21 ==== osd_op_reply(21 600.00000000 [delete] v101'153 uv153 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f206000b040 con 0x561f4c819bf0 2024-09-21T03:40:45.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.197+0000 7f207216df40 1 -- 172.21.15.38:0/2306811490 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:22 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x561f4c835be0 con 0x561f4c819bf0 2024-09-21T03:40:45.199 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-09-21T03:40:45.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.198+0000 7f206b426640 1 -- 172.21.15.38:0/2306811490 <== osd.7 v2:172.21.15.195:6818/1606106342 22 ==== osd_op_reply(22 600.00000000 [create] v101'154 uv154 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f206000b040 con 0x561f4c819bf0 2024-09-21T03:40:45.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.198+0000 7f207216df40 1 -- 172.21.15.38:0/2306811490 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:23 13.10 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x561f4c835be0 con 0x561f4c819bf0 2024-09-21T03:40:45.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.199+0000 7f206b426640 1 -- 172.21.15.38:0/2306811490 <== osd.7 v2:172.21.15.195:6818/1606106342 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v101'155 uv155 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f206000b040 con 0x561f4c819bf0 2024-09-21T03:40:45.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.199+0000 7f207216df40 1 -- 172.21.15.38:0/2306811490 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:24 13.10 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x561f4c835be0 con 0x561f4c819bf0 2024-09-21T03:40:45.202 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.200+0000 7f206b426640 1 -- 172.21.15.38:0/2306811490 <== osd.7 v2:172.21.15.195:6818/1606106342 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v101'156 uv156 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f206000b040 con 0x561f4c819bf0 2024-09-21T03:40:45.202 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.200+0000 7f207216df40 1 -- 172.21.15.38:0/2306811490 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:25 13.10 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x561f4c835be0 con 0x561f4c819bf0 2024-09-21T03:40:45.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.202+0000 7f206b426640 1 -- 172.21.15.38:0/2306811490 <== osd.7 v2:172.21.15.195:6818/1606106342 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v101'157 uv157 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f206000b040 con 0x561f4c819bf0 2024-09-21T03:40:45.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.202+0000 7f207216df40 1 -- 172.21.15.38:0/2306811490 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:26 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x561f4c835be0 con 0x561f4c819bf0 2024-09-21T03:40:45.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.203+0000 7f206b426640 1 -- 172.21.15.38:0/2306811490 <== osd.7 v2:172.21.15.195:6818/1606106342 26 ==== osd_op_reply(26 603.00000000 [create] v101'158 uv132 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f206000b040 con 0x561f4c819bf0 2024-09-21T03:40:45.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.203+0000 7f207216df40 1 -- 172.21.15.38:0/2306811490 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:27 13.10 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e101) -- 0x561f4c835be0 con 0x561f4c819bf0 2024-09-21T03:40:45.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.204+0000 7f206b426640 1 -- 172.21.15.38:0/2306811490 <== osd.7 v2:172.21.15.195:6818/1606106342 27 ==== osd_op_reply(27 603.00000000 [delete] v101'159 uv159 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f206000b040 con 0x561f4c819bf0 2024-09-21T03:40:45.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.204+0000 7f207216df40 1 -- 172.21.15.38:0/2306811490 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:28 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x561f4c835be0 con 0x561f4c819bf0 2024-09-21T03:40:45.207 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-09-21T03:40:45.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.206+0000 7f206b426640 1 -- 172.21.15.38:0/2306811490 <== osd.7 v2:172.21.15.195:6818/1606106342 28 ==== osd_op_reply(28 603.00000000 [create] v101'160 uv160 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f206000b040 con 0x561f4c819bf0 2024-09-21T03:40:45.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.206+0000 7f207216df40 1 -- 172.21.15.38:0/2306811490 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:29 13.10 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x561f4c835be0 con 0x561f4c819bf0 2024-09-21T03:40:45.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.207+0000 7f206b426640 1 -- 172.21.15.38:0/2306811490 <== osd.7 v2:172.21.15.195:6818/1606106342 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v101'161 uv161 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f206000b040 con 0x561f4c819bf0 2024-09-21T03:40:45.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.207+0000 7f207216df40 1 -- 172.21.15.38:0/2306811490 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:30 13.10 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x561f4c835be0 con 0x561f4c819bf0 2024-09-21T03:40:45.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.208+0000 7f206b426640 1 -- 172.21.15.38:0/2306811490 <== osd.7 v2:172.21.15.195:6818/1606106342 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v101'162 uv162 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f206000b040 con 0x561f4c819bf0 2024-09-21T03:40:45.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.209+0000 7f207216df40 1 -- 172.21.15.38:0/2306811490 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:31 13.10 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x561f4c835be0 con 0x561f4c819bf0 2024-09-21T03:40:45.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.210+0000 7f206b426640 1 -- 172.21.15.38:0/2306811490 <== osd.7 v2:172.21.15.195:6818/1606106342 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v101'163 uv163 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f206000b040 con 0x561f4c819bf0 2024-09-21T03:40:45.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.210+0000 7f207216df40 1 --2- 172.21.15.38:0/2306811490 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x561f4c81d180 0x561f4c8390f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:45.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.210+0000 7f207216df40 1 -- 172.21.15.38:0/2306811490 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x561f4c839630 con 0x561f4c81d180 2024-09-21T03:40:45.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.210+0000 7f206bc27640 1 --2- 172.21.15.38:0/2306811490 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x561f4c81d180 0x561f4c8390f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:45.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.211+0000 7f206bc27640 1 --2- 172.21.15.38:0/2306811490 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x561f4c81d180 0x561f4c8390f0 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:45.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.213+0000 7f206bc27640 1 -- 172.21.15.38:0/2306811490 <== osd.6 v2:172.21.15.195:6826/4055825675 1 ==== osd_op_reply(32 1.00000000.inode [create] v101'108 uv90 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f205c021500 con 0x561f4c81d180 2024-09-21T03:40:45.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.213+0000 7f207216df40 1 -- 172.21.15.38:0/2306811490 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e101) -- 0x561f4c8399d0 con 0x561f4c81d180 2024-09-21T03:40:45.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.214+0000 7f206bc27640 1 -- 172.21.15.38:0/2306811490 <== osd.6 v2:172.21.15.195:6826/4055825675 2 ==== osd_op_reply(33 1.00000000.inode [delete] v101'109 uv109 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f205c021500 con 0x561f4c81d180 2024-09-21T03:40:45.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.214+0000 7f207216df40 1 -- 172.21.15.38:0/2306811490 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x561f4c839d70 con 0x561f4c81d180 2024-09-21T03:40:45.217 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-09-21T03:40:45.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.215+0000 7f206bc27640 1 -- 172.21.15.38:0/2306811490 <== osd.6 v2:172.21.15.195:6826/4055825675 3 ==== osd_op_reply(34 1.00000000.inode [create] v101'110 uv110 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f205c021500 con 0x561f4c81d180 2024-09-21T03:40:45.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.215+0000 7f207216df40 1 -- 172.21.15.38:0/2306811490 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x561f4c83a5c0 con 0x561f4c81d180 2024-09-21T03:40:45.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.217+0000 7f206bc27640 1 -- 172.21.15.38:0/2306811490 <== osd.6 v2:172.21.15.195:6826/4055825675 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v101'111 uv111 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f205c021500 con 0x561f4c81d180 2024-09-21T03:40:45.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.217+0000 7f207216df40 1 -- 172.21.15.38:0/2306811490 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x561f4c83a960 con 0x561f4c81d180 2024-09-21T03:40:45.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.218+0000 7f206bc27640 1 -- 172.21.15.38:0/2306811490 <== osd.6 v2:172.21.15.195:6826/4055825675 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v101'112 uv112 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f205c021500 con 0x561f4c81d180 2024-09-21T03:40:45.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.218+0000 7f207216df40 1 -- 172.21.15.38:0/2306811490 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x561f4c83ad00 con 0x561f4c81d180 2024-09-21T03:40:45.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.219+0000 7f206bc27640 1 -- 172.21.15.38:0/2306811490 <== osd.6 v2:172.21.15.195:6826/4055825675 6 ==== osd_op_reply(37 200.00000000 [create] v101'113 uv113 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f205c021500 con 0x561f4c81d180 2024-09-21T03:40:45.221 INFO:teuthology.orchestra.run.smithi038.stdout:Write #-9223372036854775808:00000000:::200.00000000:head# 2024-09-21T03:40:45.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.219+0000 7f207216df40 1 -- 172.21.15.38:0/2306811490 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x561f4c83b0a0 con 0x561f4c81d180 2024-09-21T03:40:45.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.220+0000 7f206bc27640 1 -- 172.21.15.38:0/2306811490 <== osd.6 v2:172.21.15.195:6826/4055825675 7 ==== osd_op_reply(38 200.00000000 [write 0~90] v101'114 uv114 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f205c021500 con 0x561f4c81d180 2024-09-21T03:40:45.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.220+0000 7f207216df40 1 -- 172.21.15.38:0/2306811490 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x561f4c83b0a0 con 0x561f4c81d180 2024-09-21T03:40:45.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.222+0000 7f206bc27640 1 -- 172.21.15.38:0/2306811490 <== osd.6 v2:172.21.15.195:6826/4055825675 8 ==== osd_op_reply(39 200.00000000 [omap-set-header] v101'115 uv115 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f205c021500 con 0x561f4c81d180 2024-09-21T03:40:45.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.222+0000 7f207216df40 1 -- 172.21.15.38:0/2306811490 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:40 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x561f4c83b0a0 con 0x561f4c81d180 2024-09-21T03:40:45.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.223+0000 7f206bc27640 1 -- 172.21.15.38:0/2306811490 <== osd.6 v2:172.21.15.195:6826/4055825675 9 ==== osd_op_reply(40 200.00000001 [create] v101'116 uv100 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f205c021500 con 0x561f4c81d180 2024-09-21T03:40:45.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.223+0000 7f207216df40 1 -- 172.21.15.38:0/2306811490 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:41 13.14 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e101) -- 0x561f4c83b0a0 con 0x561f4c81d180 2024-09-21T03:40:45.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.224+0000 7f206bc27640 1 -- 172.21.15.38:0/2306811490 <== osd.6 v2:172.21.15.195:6826/4055825675 10 ==== osd_op_reply(41 200.00000001 [delete] v101'117 uv117 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f205c021500 con 0x561f4c81d180 2024-09-21T03:40:45.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.224+0000 7f207216df40 1 -- 172.21.15.38:0/2306811490 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:42 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x561f4c83b0a0 con 0x561f4c81d180 2024-09-21T03:40:45.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.225+0000 7f206bc27640 1 -- 172.21.15.38:0/2306811490 <== osd.6 v2:172.21.15.195:6826/4055825675 11 ==== osd_op_reply(42 200.00000001 [create] v101'118 uv118 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f205c021500 con 0x561f4c81d180 2024-09-21T03:40:45.227 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-09-21T03:40:45.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.225+0000 7f207216df40 1 -- 172.21.15.38:0/2306811490 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:43 13.14 13:2e2fa760:::200.00000001:head [write 0~17490 in=17490b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x561f4c83f520 con 0x561f4c81d180 2024-09-21T03:40:45.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.227+0000 7f206bc27640 1 -- 172.21.15.38:0/2306811490 <== osd.6 v2:172.21.15.195:6826/4055825675 12 ==== osd_op_reply(43 200.00000001 [write 0~17490] v101'119 uv119 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f205c021500 con 0x561f4c81d180 2024-09-21T03:40:45.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.227+0000 7f207216df40 1 -- 172.21.15.38:0/2306811490 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:44 13.14 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x561f4c83f520 con 0x561f4c81d180 2024-09-21T03:40:45.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.228+0000 7f206bc27640 1 -- 172.21.15.38:0/2306811490 <== osd.6 v2:172.21.15.195:6826/4055825675 13 ==== osd_op_reply(44 200.00000001 [omap-set-header] v101'120 uv120 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f205c021500 con 0x561f4c81d180 2024-09-21T03:40:45.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.228+0000 7f207216df40 1 -- 172.21.15.38:0/2306811490 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:45 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x561f4c83b0a0 con 0x561f4c81d180 2024-09-21T03:40:45.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.229+0000 7f206bc27640 1 -- 172.21.15.38:0/2306811490 <== osd.6 v2:172.21.15.195:6826/4055825675 14 ==== osd_op_reply(45 606.00000000 [create] v101'121 uv106 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f205c021500 con 0x561f4c81d180 2024-09-21T03:40:45.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.229+0000 7f207216df40 1 -- 172.21.15.38:0/2306811490 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:46 13.14 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e101) -- 0x561f4c83b0a0 con 0x561f4c81d180 2024-09-21T03:40:45.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.231+0000 7f206bc27640 1 -- 172.21.15.38:0/2306811490 <== osd.6 v2:172.21.15.195:6826/4055825675 15 ==== osd_op_reply(46 606.00000000 [delete] v101'122 uv122 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f205c021500 con 0x561f4c81d180 2024-09-21T03:40:45.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.231+0000 7f207216df40 1 -- 172.21.15.38:0/2306811490 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:47 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x561f4c83b0a0 con 0x561f4c81d180 2024-09-21T03:40:45.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.232+0000 7f206bc27640 1 -- 172.21.15.38:0/2306811490 <== osd.6 v2:172.21.15.195:6826/4055825675 16 ==== osd_op_reply(47 606.00000000 [create] v101'123 uv123 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f205c021500 con 0x561f4c81d180 2024-09-21T03:40:45.234 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-09-21T03:40:45.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.232+0000 7f207216df40 1 -- 172.21.15.38:0/2306811490 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:48 13.14 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x561f4c83c0b0 con 0x561f4c81d180 2024-09-21T03:40:45.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.233+0000 7f206bc27640 1 -- 172.21.15.38:0/2306811490 <== osd.6 v2:172.21.15.195:6826/4055825675 17 ==== osd_op_reply(48 606.00000000 [setxattr (30)] v101'124 uv124 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f205c021500 con 0x561f4c81d180 2024-09-21T03:40:45.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.233+0000 7f207216df40 1 -- 172.21.15.38:0/2306811490 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:49 13.14 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x561f4c83c0b0 con 0x561f4c81d180 2024-09-21T03:40:45.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.234+0000 7f206bc27640 1 -- 172.21.15.38:0/2306811490 <== osd.6 v2:172.21.15.195:6826/4055825675 18 ==== osd_op_reply(49 606.00000000 [setxattr (62)] v101'125 uv125 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f205c021500 con 0x561f4c81d180 2024-09-21T03:40:45.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.234+0000 7f207216df40 1 -- 172.21.15.38:0/2306811490 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:50 13.14 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x561f4c83b0a0 con 0x561f4c81d180 2024-09-21T03:40:45.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.236+0000 7f206bc27640 1 -- 172.21.15.38:0/2306811490 <== osd.6 v2:172.21.15.195:6826/4055825675 19 ==== osd_op_reply(50 606.00000000 [omap-set-header] v101'126 uv126 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f205c021500 con 0x561f4c81d180 2024-09-21T03:40:45.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.236+0000 7f207216df40 1 --2- 172.21.15.38:0/2306811490 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x561f4c83b5b0 0x561f4c83d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:45.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.236+0000 7f207216df40 1 -- 172.21.15.38:0/2306811490 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:51 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x561f4c83df10 con 0x561f4c83b5b0 2024-09-21T03:40:45.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.236+0000 7f206b426640 1 --2- 172.21.15.38:0/2306811490 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x561f4c83b5b0 0x561f4c83d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:45.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.236+0000 7f206b426640 1 --2- 172.21.15.38:0/2306811490 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x561f4c83b5b0 0x561f4c83d9d0 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:45.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.237+0000 7f206b426640 1 -- 172.21.15.38:0/2306811490 <== osd.0 v2:172.21.15.38:6800/2381639433 1 ==== osd_op_reply(51 607.00000000 [create] v101'27 uv26 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f206000b040 con 0x561f4c83b5b0 2024-09-21T03:40:45.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.237+0000 7f207216df40 1 -- 172.21.15.38:0/2306811490 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:52 13.1c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e101) -- 0x561f4c83df10 con 0x561f4c83b5b0 2024-09-21T03:40:45.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.239+0000 7f206b426640 1 -- 172.21.15.38:0/2306811490 <== osd.0 v2:172.21.15.38:6800/2381639433 2 ==== osd_op_reply(52 607.00000000 [delete] v101'28 uv28 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f206000b040 con 0x561f4c83b5b0 2024-09-21T03:40:45.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.239+0000 7f207216df40 1 -- 172.21.15.38:0/2306811490 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:53 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x561f4c83e330 con 0x561f4c83b5b0 2024-09-21T03:40:45.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.240+0000 7f206b426640 1 -- 172.21.15.38:0/2306811490 <== osd.0 v2:172.21.15.38:6800/2381639433 3 ==== osd_op_reply(53 607.00000000 [create] v101'29 uv29 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f206000b040 con 0x561f4c83b5b0 2024-09-21T03:40:45.242 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-09-21T03:40:45.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.240+0000 7f207216df40 1 -- 172.21.15.38:0/2306811490 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:54 13.1c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x561f4c83f340 con 0x561f4c83b5b0 2024-09-21T03:40:45.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.242+0000 7f206b426640 1 -- 172.21.15.38:0/2306811490 <== osd.0 v2:172.21.15.38:6800/2381639433 4 ==== osd_op_reply(54 607.00000000 [setxattr (30)] v101'30 uv30 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f206000b040 con 0x561f4c83b5b0 2024-09-21T03:40:45.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.242+0000 7f207216df40 1 -- 172.21.15.38:0/2306811490 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:55 13.1c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x561f4c83f340 con 0x561f4c83b5b0 2024-09-21T03:40:45.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.243+0000 7f206b426640 1 -- 172.21.15.38:0/2306811490 <== osd.0 v2:172.21.15.38:6800/2381639433 5 ==== osd_op_reply(55 607.00000000 [setxattr (62)] v101'31 uv31 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f206000b040 con 0x561f4c83b5b0 2024-09-21T03:40:45.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.243+0000 7f207216df40 1 -- 172.21.15.38:0/2306811490 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:56 13.1c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x561f4c83e330 con 0x561f4c83b5b0 2024-09-21T03:40:45.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.244+0000 7f206b426640 1 -- 172.21.15.38:0/2306811490 <== osd.0 v2:172.21.15.38:6800/2381639433 6 ==== osd_op_reply(56 607.00000000 [omap-set-header] v101'32 uv32 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f206000b040 con 0x561f4c83b5b0 2024-09-21T03:40:45.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.244+0000 7f207216df40 1 --2- 172.21.15.38:0/2306811490 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x561f4c83e9b0 0x561f4c840dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:45.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.244+0000 7f207216df40 1 -- 172.21.15.38:0/2306811490 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:57 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x561f4c841450 con 0x561f4c83e9b0 2024-09-21T03:40:45.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.244+0000 7f206c428640 1 --2- 172.21.15.38:0/2306811490 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x561f4c83e9b0 0x561f4c840dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:45.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.245+0000 7f206c428640 1 --2- 172.21.15.38:0/2306811490 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x561f4c83e9b0 0x561f4c840dd0 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:45.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.246+0000 7f206c428640 1 -- 172.21.15.38:0/2306811490 <== osd.3 v2:172.21.15.38:6808/1286810294 1 ==== osd_op_reply(57 mds0_openfiles.0 [create] v101'20 uv19 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f2064012030 con 0x561f4c83e9b0 2024-09-21T03:40:45.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.246+0000 7f207216df40 1 -- 172.21.15.38:0/2306811490 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:58 13.6 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e101) -- 0x561f4c841960 con 0x561f4c83e9b0 2024-09-21T03:40:45.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.247+0000 7f206c428640 1 -- 172.21.15.38:0/2306811490 <== osd.3 v2:172.21.15.38:6808/1286810294 2 ==== osd_op_reply(58 mds0_openfiles.0 [delete] v101'21 uv21 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f2064012030 con 0x561f4c83e9b0 2024-09-21T03:40:45.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.247+0000 7f207216df40 1 -- 172.21.15.38:0/2306811490 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:59 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x561f4c841e20 con 0x561f4c83e9b0 2024-09-21T03:40:45.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.249+0000 7f206c428640 1 -- 172.21.15.38:0/2306811490 <== osd.3 v2:172.21.15.38:6808/1286810294 3 ==== osd_op_reply(59 mds0_openfiles.0 [create] v101'22 uv22 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f2064012030 con 0x561f4c83e9b0 2024-09-21T03:40:45.250 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-09-21T03:40:45.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.249+0000 7f207216df40 1 -- 172.21.15.38:0/2306811490 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:60 13.6 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x561f4c842710 con 0x561f4c83e9b0 2024-09-21T03:40:45.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.250+0000 7f206c428640 1 -- 172.21.15.38:0/2306811490 <== osd.3 v2:172.21.15.38:6808/1286810294 4 ==== osd_op_reply(60 mds0_openfiles.0 [omap-set-header] v101'23 uv23 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f2064012030 con 0x561f4c83e9b0 2024-09-21T03:40:45.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.250+0000 7f207216df40 1 -- 172.21.15.38:0/2306811490 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:61 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x561f4c842ab0 con 0x561f4c83b5b0 2024-09-21T03:40:45.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.251+0000 7f206b426640 1 -- 172.21.15.38:0/2306811490 <== osd.0 v2:172.21.15.38:6800/2381639433 7 ==== osd_op_reply(61 608.00000000 [create] v101'27 uv26 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f206000b040 con 0x561f4c83b5b0 2024-09-21T03:40:45.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.251+0000 7f207216df40 1 -- 172.21.15.38:0/2306811490 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:62 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e101) -- 0x561f4c842ab0 con 0x561f4c83b5b0 2024-09-21T03:40:45.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.253+0000 7f206b426640 1 -- 172.21.15.38:0/2306811490 <== osd.0 v2:172.21.15.38:6800/2381639433 8 ==== osd_op_reply(62 608.00000000 [delete] v101'28 uv28 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f206000b040 con 0x561f4c83b5b0 2024-09-21T03:40:45.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.253+0000 7f207216df40 1 -- 172.21.15.38:0/2306811490 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x561f4c842ab0 con 0x561f4c83b5b0 2024-09-21T03:40:45.256 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-09-21T03:40:45.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.254+0000 7f206b426640 1 -- 172.21.15.38:0/2306811490 <== osd.0 v2:172.21.15.38:6800/2381639433 9 ==== osd_op_reply(63 608.00000000 [create] v101'29 uv29 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f206000b040 con 0x561f4c83b5b0 2024-09-21T03:40:45.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.254+0000 7f207216df40 1 -- 172.21.15.38:0/2306811490 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x561f4c843ac0 con 0x561f4c83b5b0 2024-09-21T03:40:45.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.255+0000 7f206b426640 1 -- 172.21.15.38:0/2306811490 <== osd.0 v2:172.21.15.38:6800/2381639433 10 ==== osd_op_reply(64 608.00000000 [setxattr (30)] v101'30 uv30 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f206000b040 con 0x561f4c83b5b0 2024-09-21T03:40:45.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.255+0000 7f207216df40 1 -- 172.21.15.38:0/2306811490 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x561f4c843ac0 con 0x561f4c83b5b0 2024-09-21T03:40:45.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.256+0000 7f206b426640 1 -- 172.21.15.38:0/2306811490 <== osd.0 v2:172.21.15.38:6800/2381639433 11 ==== osd_op_reply(65 608.00000000 [setxattr (62)] v101'31 uv31 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f206000b040 con 0x561f4c83b5b0 2024-09-21T03:40:45.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.257+0000 7f207216df40 1 -- 172.21.15.38:0/2306811490 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x561f4c842ab0 con 0x561f4c83b5b0 2024-09-21T03:40:45.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.258+0000 7f206b426640 1 -- 172.21.15.38:0/2306811490 <== osd.0 v2:172.21.15.38:6800/2381639433 12 ==== osd_op_reply(66 608.00000000 [omap-set-header] v101'32 uv32 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f206000b040 con 0x561f4c83b5b0 2024-09-21T03:40:45.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.258+0000 7f207216df40 1 -- 172.21.15.38:0/2306811490 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:67 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x561f4c842ab0 con 0x561f4c83e9b0 2024-09-21T03:40:45.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.259+0000 7f206c428640 1 -- 172.21.15.38:0/2306811490 <== osd.3 v2:172.21.15.38:6808/1286810294 5 ==== osd_op_reply(67 604.00000000 [create] v101'48 uv42 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f2064012030 con 0x561f4c83e9b0 2024-09-21T03:40:45.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.259+0000 7f207216df40 1 -- 172.21.15.38:0/2306811490 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:68 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e101) -- 0x561f4c842e50 con 0x561f4c83e9b0 2024-09-21T03:40:45.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.260+0000 7f206c428640 1 -- 172.21.15.38:0/2306811490 <== osd.3 v2:172.21.15.38:6808/1286810294 6 ==== osd_op_reply(68 604.00000000 [delete] v101'49 uv49 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2064012030 con 0x561f4c83e9b0 2024-09-21T03:40:45.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.260+0000 7f207216df40 1 -- 172.21.15.38:0/2306811490 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x561f4c8431f0 con 0x561f4c83e9b0 2024-09-21T03:40:45.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.261+0000 7f206c428640 1 -- 172.21.15.38:0/2306811490 <== osd.3 v2:172.21.15.38:6808/1286810294 7 ==== osd_op_reply(69 604.00000000 [create] v101'50 uv50 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2064012030 con 0x561f4c83e9b0 2024-09-21T03:40:45.263 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-09-21T03:40:45.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.261+0000 7f207216df40 1 -- 172.21.15.38:0/2306811490 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x561f4c8445a0 con 0x561f4c83e9b0 2024-09-21T03:40:45.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.262+0000 7f206c428640 1 -- 172.21.15.38:0/2306811490 <== osd.3 v2:172.21.15.38:6808/1286810294 8 ==== osd_op_reply(70 604.00000000 [setxattr (30)] v101'51 uv51 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2064012030 con 0x561f4c83e9b0 2024-09-21T03:40:45.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.262+0000 7f207216df40 1 -- 172.21.15.38:0/2306811490 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x561f4c8445a0 con 0x561f4c83e9b0 2024-09-21T03:40:45.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.263+0000 7f206c428640 1 -- 172.21.15.38:0/2306811490 <== osd.3 v2:172.21.15.38:6808/1286810294 9 ==== osd_op_reply(71 604.00000000 [setxattr (62)] v101'52 uv52 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2064012030 con 0x561f4c83e9b0 2024-09-21T03:40:45.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.263+0000 7f207216df40 1 -- 172.21.15.38:0/2306811490 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x561f4c843590 con 0x561f4c83e9b0 2024-09-21T03:40:45.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.264+0000 7f206c428640 1 -- 172.21.15.38:0/2306811490 <== osd.3 v2:172.21.15.38:6808/1286810294 10 ==== osd_op_reply(72 604.00000000 [omap-set-header] v101'53 uv53 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2064012030 con 0x561f4c83e9b0 2024-09-21T03:40:45.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.264+0000 7f207216df40 1 -- 172.21.15.38:0/2306811490 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:73 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x561f4c843590 con 0x561f4c83e9b0 2024-09-21T03:40:45.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.265+0000 7f206c428640 1 -- 172.21.15.38:0/2306811490 <== osd.3 v2:172.21.15.38:6808/1286810294 11 ==== osd_op_reply(73 500.00000000 [create] v101'54 uv47 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f2064012030 con 0x561f4c83e9b0 2024-09-21T03:40:45.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.265+0000 7f207216df40 1 -- 172.21.15.38:0/2306811490 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:74 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e101) -- 0x561f4c843590 con 0x561f4c83e9b0 2024-09-21T03:40:45.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.266+0000 7f206c428640 1 -- 172.21.15.38:0/2306811490 <== osd.3 v2:172.21.15.38:6808/1286810294 12 ==== osd_op_reply(74 500.00000000 [delete] v101'55 uv55 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2064012030 con 0x561f4c83e9b0 2024-09-21T03:40:45.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.266+0000 7f207216df40 1 -- 172.21.15.38:0/2306811490 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x561f4c843590 con 0x561f4c83e9b0 2024-09-21T03:40:45.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.267+0000 7f206c428640 1 -- 172.21.15.38:0/2306811490 <== osd.3 v2:172.21.15.38:6808/1286810294 13 ==== osd_op_reply(75 500.00000000 [create] v101'56 uv56 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2064012030 con 0x561f4c83e9b0 2024-09-21T03:40:45.269 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-09-21T03:40:45.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.267+0000 7f207216df40 1 -- 172.21.15.38:0/2306811490 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x561f4c843590 con 0x561f4c83e9b0 2024-09-21T03:40:45.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.268+0000 7f206c428640 1 -- 172.21.15.38:0/2306811490 <== osd.3 v2:172.21.15.38:6808/1286810294 14 ==== osd_op_reply(76 500.00000000 [write 0~90] v101'57 uv57 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2064012030 con 0x561f4c83e9b0 2024-09-21T03:40:45.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.268+0000 7f207216df40 1 -- 172.21.15.38:0/2306811490 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x561f4c843590 con 0x561f4c83e9b0 2024-09-21T03:40:45.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.269+0000 7f206c428640 1 -- 172.21.15.38:0/2306811490 <== osd.3 v2:172.21.15.38:6808/1286810294 15 ==== osd_op_reply(77 500.00000000 [omap-set-header] v101'58 uv58 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2064012030 con 0x561f4c83e9b0 2024-09-21T03:40:45.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.269+0000 7f207216df40 1 -- 172.21.15.38:0/2306811490 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:78 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x561f4c843590 con 0x561f4c83b5b0 2024-09-21T03:40:45.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.270+0000 7f206b426640 1 -- 172.21.15.38:0/2306811490 <== osd.0 v2:172.21.15.38:6800/2381639433 13 ==== osd_op_reply(78 mds_snaptable [create] v101'22 uv21 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f206000b040 con 0x561f4c83b5b0 2024-09-21T03:40:45.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.270+0000 7f207216df40 1 -- 172.21.15.38:0/2306811490 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:79 13.d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e101) -- 0x561f4c843590 con 0x561f4c83b5b0 2024-09-21T03:40:45.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.271+0000 7f206b426640 1 -- 172.21.15.38:0/2306811490 <== osd.0 v2:172.21.15.38:6800/2381639433 14 ==== osd_op_reply(79 mds_snaptable [delete] v101'23 uv23 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f206000b040 con 0x561f4c83b5b0 2024-09-21T03:40:45.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.271+0000 7f207216df40 1 -- 172.21.15.38:0/2306811490 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:80 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x561f4c843590 con 0x561f4c83b5b0 2024-09-21T03:40:45.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.272+0000 7f206b426640 1 -- 172.21.15.38:0/2306811490 <== osd.0 v2:172.21.15.38:6800/2381639433 15 ==== osd_op_reply(80 mds_snaptable [create] v101'24 uv24 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f206000b040 con 0x561f4c83b5b0 2024-09-21T03:40:45.274 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-09-21T03:40:45.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.272+0000 7f207216df40 1 -- 172.21.15.38:0/2306811490 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:81 13.d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x561f4c843590 con 0x561f4c83b5b0 2024-09-21T03:40:45.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.273+0000 7f206b426640 1 -- 172.21.15.38:0/2306811490 <== osd.0 v2:172.21.15.38:6800/2381639433 16 ==== osd_op_reply(81 mds_snaptable [write 0~70] v101'25 uv25 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f206000b040 con 0x561f4c83b5b0 2024-09-21T03:40:45.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.273+0000 7f207216df40 1 -- 172.21.15.38:0/2306811490 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:82 13.d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x561f4c843590 con 0x561f4c83b5b0 2024-09-21T03:40:45.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.274+0000 7f206b426640 1 -- 172.21.15.38:0/2306811490 <== osd.0 v2:172.21.15.38:6800/2381639433 17 ==== osd_op_reply(82 mds_snaptable [omap-set-header] v101'26 uv26 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f206000b040 con 0x561f4c83b5b0 2024-09-21T03:40:45.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.275+0000 7f207216df40 1 --2- 172.21.15.38:0/2306811490 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x561f4c843c10 0x561f4c846030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:45.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.275+0000 7f207216df40 1 -- 172.21.15.38:0/2306811490 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:83 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x561f4c846690 con 0x561f4c843c10 2024-09-21T03:40:45.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.275+0000 7f206bc27640 1 --2- 172.21.15.38:0/2306811490 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x561f4c843c10 0x561f4c846030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:45.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.275+0000 7f206bc27640 1 --2- 172.21.15.38:0/2306811490 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x561f4c843c10 0x561f4c846030 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:45.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.276+0000 7f206bc27640 1 -- 172.21.15.38:0/2306811490 <== osd.1 v2:172.21.15.38:6816/1491818732 1 ==== osd_op_reply(83 605.00000000 [create] v101'27 uv26 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f205c021500 con 0x561f4c843c10 2024-09-21T03:40:45.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.276+0000 7f207216df40 1 -- 172.21.15.38:0/2306811490 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:84 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e101) -- 0x561f4c846a60 con 0x561f4c843c10 2024-09-21T03:40:45.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.277+0000 7f206bc27640 1 -- 172.21.15.38:0/2306811490 <== osd.1 v2:172.21.15.38:6816/1491818732 2 ==== osd_op_reply(84 605.00000000 [delete] v101'28 uv28 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f205c021500 con 0x561f4c843c10 2024-09-21T03:40:45.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.277+0000 7f207216df40 1 -- 172.21.15.38:0/2306811490 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x561f4c846a60 con 0x561f4c843c10 2024-09-21T03:40:45.280 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-09-21T03:40:45.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.278+0000 7f206bc27640 1 -- 172.21.15.38:0/2306811490 <== osd.1 v2:172.21.15.38:6816/1491818732 3 ==== osd_op_reply(85 605.00000000 [create] v101'29 uv29 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f205c021500 con 0x561f4c843c10 2024-09-21T03:40:45.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.278+0000 7f207216df40 1 -- 172.21.15.38:0/2306811490 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x561f4c847a70 con 0x561f4c843c10 2024-09-21T03:40:45.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.279+0000 7f206bc27640 1 -- 172.21.15.38:0/2306811490 <== osd.1 v2:172.21.15.38:6816/1491818732 4 ==== osd_op_reply(86 605.00000000 [setxattr (30)] v101'30 uv30 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f205c021500 con 0x561f4c843c10 2024-09-21T03:40:45.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.279+0000 7f207216df40 1 -- 172.21.15.38:0/2306811490 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x561f4c847a70 con 0x561f4c843c10 2024-09-21T03:40:45.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.280+0000 7f206bc27640 1 -- 172.21.15.38:0/2306811490 <== osd.1 v2:172.21.15.38:6816/1491818732 5 ==== osd_op_reply(87 605.00000000 [setxattr (62)] v101'31 uv31 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f205c021500 con 0x561f4c843c10 2024-09-21T03:40:45.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.280+0000 7f207216df40 1 -- 172.21.15.38:0/2306811490 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x561f4c846a60 con 0x561f4c843c10 2024-09-21T03:40:45.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.281+0000 7f206bc27640 1 -- 172.21.15.38:0/2306811490 <== osd.1 v2:172.21.15.38:6816/1491818732 6 ==== osd_op_reply(88 605.00000000 [omap-set-header] v101'32 uv32 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f205c021500 con 0x561f4c843c10 2024-09-21T03:40:45.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.281+0000 7f207216df40 1 -- 172.21.15.38:0/2306811490 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:89 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x561f4c846a60 con 0x561f4c843c10 2024-09-21T03:40:45.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.282+0000 7f206bc27640 1 -- 172.21.15.38:0/2306811490 <== osd.1 v2:172.21.15.38:6816/1491818732 7 ==== osd_op_reply(89 mds0_inotable [create] v101'44 uv38 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f205c021500 con 0x561f4c843c10 2024-09-21T03:40:45.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.282+0000 7f207216df40 1 -- 172.21.15.38:0/2306811490 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:90 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e101) -- 0x561f4c846a60 con 0x561f4c843c10 2024-09-21T03:40:45.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.283+0000 7f206bc27640 1 -- 172.21.15.38:0/2306811490 <== osd.1 v2:172.21.15.38:6816/1491818732 8 ==== osd_op_reply(90 mds0_inotable [delete] v101'45 uv45 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f205c021500 con 0x561f4c843c10 2024-09-21T03:40:45.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.283+0000 7f207216df40 1 -- 172.21.15.38:0/2306811490 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x561f4c846a60 con 0x561f4c843c10 2024-09-21T03:40:45.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.284+0000 7f206bc27640 1 -- 172.21.15.38:0/2306811490 <== osd.1 v2:172.21.15.38:6816/1491818732 9 ==== osd_op_reply(91 mds0_inotable [create] v101'46 uv46 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f205c021500 con 0x561f4c843c10 2024-09-21T03:40:45.286 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-09-21T03:40:45.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.284+0000 7f207216df40 1 -- 172.21.15.38:0/2306811490 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x561f4c842320 con 0x561f4c843c10 2024-09-21T03:40:45.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.285+0000 7f206bc27640 1 -- 172.21.15.38:0/2306811490 <== osd.1 v2:172.21.15.38:6816/1491818732 10 ==== osd_op_reply(92 mds0_inotable [write 0~34] v101'47 uv47 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f205c021500 con 0x561f4c843c10 2024-09-21T03:40:45.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.285+0000 7f207216df40 1 -- 172.21.15.38:0/2306811490 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x561f4c842320 con 0x561f4c843c10 2024-09-21T03:40:45.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.286+0000 7f206bc27640 1 -- 172.21.15.38:0/2306811490 <== osd.1 v2:172.21.15.38:6816/1491818732 11 ==== osd_op_reply(93 mds0_inotable [omap-set-header] v101'48 uv48 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f205c021500 con 0x561f4c843c10 2024-09-21T03:40:45.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.286+0000 7f207216df40 1 -- 172.21.15.38:0/2306811490 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:94 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x561f4c842300 con 0x561f4c843c10 2024-09-21T03:40:45.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.287+0000 7f206bc27640 1 -- 172.21.15.38:0/2306811490 <== osd.1 v2:172.21.15.38:6816/1491818732 12 ==== osd_op_reply(94 100.00000000 [create] v101'49 uv43 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f205c021500 con 0x561f4c843c10 2024-09-21T03:40:45.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.287+0000 7f207216df40 1 -- 172.21.15.38:0/2306811490 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:95 13.13 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e101) -- 0x561f4c842300 con 0x561f4c843c10 2024-09-21T03:40:45.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.289+0000 7f206bc27640 1 -- 172.21.15.38:0/2306811490 <== osd.1 v2:172.21.15.38:6816/1491818732 13 ==== osd_op_reply(95 100.00000000 [delete] v101'50 uv50 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f205c021500 con 0x561f4c843c10 2024-09-21T03:40:45.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.289+0000 7f207216df40 1 -- 172.21.15.38:0/2306811490 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:96 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x561f4c842300 con 0x561f4c843c10 2024-09-21T03:40:45.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.290+0000 7f206bc27640 1 -- 172.21.15.38:0/2306811490 <== osd.1 v2:172.21.15.38:6816/1491818732 14 ==== osd_op_reply(96 100.00000000 [create] v101'51 uv51 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f205c021500 con 0x561f4c843c10 2024-09-21T03:40:45.292 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-09-21T03:40:45.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.290+0000 7f207216df40 1 -- 172.21.15.38:0/2306811490 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:97 13.13 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x561f4c842300 con 0x561f4c843c10 2024-09-21T03:40:45.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.291+0000 7f206bc27640 1 -- 172.21.15.38:0/2306811490 <== osd.1 v2:172.21.15.38:6816/1491818732 15 ==== osd_op_reply(97 100.00000000 [omap-set-header] v101'52 uv52 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f205c021500 con 0x561f4c843c10 2024-09-21T03:40:45.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.291+0000 7f207216df40 1 -- 172.21.15.38:0/2306811490 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:98 13.13 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x561f4c8498f0 con 0x561f4c843c10 2024-09-21T03:40:45.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.292+0000 7f206bc27640 1 -- 172.21.15.38:0/2306811490 <== osd.1 v2:172.21.15.38:6816/1491818732 16 ==== osd_op_reply(98 100.00000000 [omap-set-vals] v101'53 uv53 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f205c021500 con 0x561f4c843c10 2024-09-21T03:40:45.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.292+0000 7f207216df40 1 -- 172.21.15.38:0/2306811490 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:99 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x561f4c846a60 con 0x561f4c83e9b0 2024-09-21T03:40:45.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.293+0000 7f206c428640 1 -- 172.21.15.38:0/2306811490 <== osd.3 v2:172.21.15.38:6808/1286810294 16 ==== osd_op_reply(99 mds0_sessionmap [create] v101'22 uv21 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7f2064012030 con 0x561f4c83e9b0 2024-09-21T03:40:45.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.293+0000 7f207216df40 1 -- 172.21.15.38:0/2306811490 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:100 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e101) -- 0x561f4c846a60 con 0x561f4c83e9b0 2024-09-21T03:40:45.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.294+0000 7f206c428640 1 -- 172.21.15.38:0/2306811490 <== osd.3 v2:172.21.15.38:6808/1286810294 17 ==== osd_op_reply(100 mds0_sessionmap [delete] v101'23 uv23 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f2064012030 con 0x561f4c83e9b0 2024-09-21T03:40:45.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.294+0000 7f207216df40 1 -- 172.21.15.38:0/2306811490 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x561f4c846a60 con 0x561f4c83e9b0 2024-09-21T03:40:45.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.295+0000 7f206c428640 1 -- 172.21.15.38:0/2306811490 <== osd.3 v2:172.21.15.38:6808/1286810294 18 ==== osd_op_reply(101 mds0_sessionmap [create] v101'24 uv24 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f2064012030 con 0x561f4c83e9b0 2024-09-21T03:40:45.297 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-09-21T03:40:45.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.295+0000 7f207216df40 1 -- 172.21.15.38:0/2306811490 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x561f4c846a60 con 0x561f4c83e9b0 2024-09-21T03:40:45.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.296+0000 7f206c428640 1 -- 172.21.15.38:0/2306811490 <== osd.3 v2:172.21.15.38:6808/1286810294 19 ==== osd_op_reply(102 mds0_sessionmap [omap-set-header] v101'25 uv25 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f2064012030 con 0x561f4c83e9b0 2024-09-21T03:40:45.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.296+0000 7f207216df40 1 -- 172.21.15.38:0/2306811490 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:103 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x561f4c846f10 con 0x561f4c83e9b0 2024-09-21T03:40:45.299 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.297+0000 7f206c428640 1 -- 172.21.15.38:0/2306811490 <== osd.3 v2:172.21.15.38:6808/1286810294 20 ==== osd_op_reply(103 609.00000000 [create] v101'27 uv26 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f2064012030 con 0x561f4c83e9b0 2024-09-21T03:40:45.299 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.297+0000 7f207216df40 1 -- 172.21.15.38:0/2306811490 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:104 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e101) -- 0x561f4c846f10 con 0x561f4c83e9b0 2024-09-21T03:40:45.300 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.298+0000 7f206c428640 1 -- 172.21.15.38:0/2306811490 <== osd.3 v2:172.21.15.38:6808/1286810294 21 ==== osd_op_reply(104 609.00000000 [delete] v101'28 uv28 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2064012030 con 0x561f4c83e9b0 2024-09-21T03:40:45.300 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.298+0000 7f207216df40 1 -- 172.21.15.38:0/2306811490 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x561f4c846f10 con 0x561f4c83e9b0 2024-09-21T03:40:45.301 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.299+0000 7f206c428640 1 -- 172.21.15.38:0/2306811490 <== osd.3 v2:172.21.15.38:6808/1286810294 22 ==== osd_op_reply(105 609.00000000 [create] v101'29 uv29 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2064012030 con 0x561f4c83e9b0 2024-09-21T03:40:45.301 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-09-21T03:40:45.301 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.299+0000 7f207216df40 1 -- 172.21.15.38:0/2306811490 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x561f4c847f20 con 0x561f4c83e9b0 2024-09-21T03:40:45.302 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.300+0000 7f206c428640 1 -- 172.21.15.38:0/2306811490 <== osd.3 v2:172.21.15.38:6808/1286810294 23 ==== osd_op_reply(106 609.00000000 [setxattr (30)] v101'30 uv30 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2064012030 con 0x561f4c83e9b0 2024-09-21T03:40:45.302 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.300+0000 7f207216df40 1 -- 172.21.15.38:0/2306811490 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x561f4c847f20 con 0x561f4c83e9b0 2024-09-21T03:40:45.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.301+0000 7f206c428640 1 -- 172.21.15.38:0/2306811490 <== osd.3 v2:172.21.15.38:6808/1286810294 24 ==== osd_op_reply(107 609.00000000 [setxattr (62)] v101'31 uv31 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2064012030 con 0x561f4c83e9b0 2024-09-21T03:40:45.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.301+0000 7f207216df40 1 -- 172.21.15.38:0/2306811490 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x561f4c846f10 con 0x561f4c83e9b0 2024-09-21T03:40:45.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.302+0000 7f206c428640 1 -- 172.21.15.38:0/2306811490 <== osd.3 v2:172.21.15.38:6808/1286810294 25 ==== osd_op_reply(108 609.00000000 [omap-set-header] v101'32 uv32 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2064012030 con 0x561f4c83e9b0 2024-09-21T03:40:45.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.302+0000 7f207216df40 1 --2- 172.21.15.38:0/2306811490 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x561f4c847590 0x561f4c8499b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:45.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.302+0000 7f207216df40 1 -- 172.21.15.38:0/2306811490 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:109 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x561f4c849fb0 con 0x561f4c847590 2024-09-21T03:40:45.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.302+0000 7f206b426640 1 --2- 172.21.15.38:0/2306811490 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x561f4c847590 0x561f4c8499b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:45.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.303+0000 7f206b426640 1 --2- 172.21.15.38:0/2306811490 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x561f4c847590 0x561f4c8499b0 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:45.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.304+0000 7f206b426640 1 -- 172.21.15.38:0/2306811490 <== osd.2 v2:172.21.15.38:6824/2577858902 1 ==== osd_op_reply(109 400.00000000 [create] v101'22 uv21 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f206000b040 con 0x561f4c847590 2024-09-21T03:40:45.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.304+0000 7f207216df40 1 -- 172.21.15.38:0/2306811490 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:110 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e101) -- 0x561f4c849fb0 con 0x561f4c847590 2024-09-21T03:40:45.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.305+0000 7f206b426640 1 -- 172.21.15.38:0/2306811490 <== osd.2 v2:172.21.15.38:6824/2577858902 2 ==== osd_op_reply(110 400.00000000 [delete] v101'23 uv23 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f206000b040 con 0x561f4c847590 2024-09-21T03:40:45.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.305+0000 7f207216df40 1 -- 172.21.15.38:0/2306811490 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x561f4c849fb0 con 0x561f4c847590 2024-09-21T03:40:45.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.306+0000 7f206b426640 1 -- 172.21.15.38:0/2306811490 <== osd.2 v2:172.21.15.38:6824/2577858902 3 ==== osd_op_reply(111 400.00000000 [create] v101'24 uv24 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f206000b040 con 0x561f4c847590 2024-09-21T03:40:45.308 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-09-21T03:40:45.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.306+0000 7f207216df40 1 -- 172.21.15.38:0/2306811490 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x561f4c84a3f0 con 0x561f4c847590 2024-09-21T03:40:45.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.307+0000 7f206b426640 1 -- 172.21.15.38:0/2306811490 <== osd.2 v2:172.21.15.38:6824/2577858902 4 ==== osd_op_reply(112 400.00000000 [write 0~22] v101'25 uv25 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f206000b040 con 0x561f4c847590 2024-09-21T03:40:45.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.307+0000 7f207216df40 1 -- 172.21.15.38:0/2306811490 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x561f4c84a3f0 con 0x561f4c847590 2024-09-21T03:40:45.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.308+0000 7f206b426640 1 -- 172.21.15.38:0/2306811490 <== osd.2 v2:172.21.15.38:6824/2577858902 5 ==== osd_op_reply(113 400.00000000 [omap-set-header] v101'26 uv26 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f206000b040 con 0x561f4c847590 2024-09-21T03:40:45.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.308+0000 7f207216df40 1 -- 172.21.15.38:0/2306811490 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:114 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x561f4c84a3f0 con 0x561f4c843c10 2024-09-21T03:40:45.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.309+0000 7f206bc27640 1 -- 172.21.15.38:0/2306811490 <== osd.1 v2:172.21.15.38:6816/1491818732 17 ==== osd_op_reply(114 100.00000000.inode [create] v101'57 uv49 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7f205c021500 con 0x561f4c843c10 2024-09-21T03:40:45.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.309+0000 7f207216df40 1 -- 172.21.15.38:0/2306811490 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:115 13.1f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e101) -- 0x561f4c84a3f0 con 0x561f4c843c10 2024-09-21T03:40:45.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.310+0000 7f206bc27640 1 -- 172.21.15.38:0/2306811490 <== osd.1 v2:172.21.15.38:6816/1491818732 18 ==== osd_op_reply(115 100.00000000.inode [delete] v101'58 uv58 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f205c021500 con 0x561f4c843c10 2024-09-21T03:40:45.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.310+0000 7f207216df40 1 -- 172.21.15.38:0/2306811490 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:116 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x561f4c84a3f0 con 0x561f4c843c10 2024-09-21T03:40:45.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.311+0000 7f206bc27640 1 -- 172.21.15.38:0/2306811490 <== osd.1 v2:172.21.15.38:6816/1491818732 19 ==== osd_op_reply(116 100.00000000.inode [create] v101'59 uv59 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f205c021500 con 0x561f4c843c10 2024-09-21T03:40:45.313 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-09-21T03:40:45.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.311+0000 7f207216df40 1 -- 172.21.15.38:0/2306811490 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:117 13.1f 13:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x561f4c84a3f0 con 0x561f4c843c10 2024-09-21T03:40:45.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.313+0000 7f206bc27640 1 -- 172.21.15.38:0/2306811490 <== osd.1 v2:172.21.15.38:6816/1491818732 20 ==== osd_op_reply(117 100.00000000.inode [write 0~582] v101'60 uv60 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f205c021500 con 0x561f4c843c10 2024-09-21T03:40:45.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.313+0000 7f207216df40 1 -- 172.21.15.38:0/2306811490 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:118 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x561f4c84a3f0 con 0x561f4c843c10 2024-09-21T03:40:45.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.314+0000 7f206bc27640 1 -- 172.21.15.38:0/2306811490 <== osd.1 v2:172.21.15.38:6816/1491818732 21 ==== osd_op_reply(118 100.00000000.inode [omap-set-header] v101'61 uv61 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f205c021500 con 0x561f4c843c10 2024-09-21T03:40:45.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.314+0000 7f207216df40 1 -- 172.21.15.38:0/2306811490 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:119 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x561f4c84a3f0 con 0x561f4c843c10 2024-09-21T03:40:45.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.315+0000 7f206bc27640 1 -- 172.21.15.38:0/2306811490 <== osd.1 v2:172.21.15.38:6816/1491818732 22 ==== osd_op_reply(119 1.00000000 [create] v101'62 uv56 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7f205c021500 con 0x561f4c843c10 2024-09-21T03:40:45.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.315+0000 7f207216df40 1 -- 172.21.15.38:0/2306811490 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:120 13.1f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e101) -- 0x561f4c84a3f0 con 0x561f4c843c10 2024-09-21T03:40:45.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.316+0000 7f206bc27640 1 -- 172.21.15.38:0/2306811490 <== osd.1 v2:172.21.15.38:6816/1491818732 23 ==== osd_op_reply(120 1.00000000 [delete] v101'63 uv63 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f205c021500 con 0x561f4c843c10 2024-09-21T03:40:45.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.316+0000 7f207216df40 1 -- 172.21.15.38:0/2306811490 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:121 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x561f4c84a3f0 con 0x561f4c843c10 2024-09-21T03:40:45.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.317+0000 7f206bc27640 1 -- 172.21.15.38:0/2306811490 <== osd.1 v2:172.21.15.38:6816/1491818732 24 ==== osd_op_reply(121 1.00000000 [create] v101'64 uv64 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f205c021500 con 0x561f4c843c10 2024-09-21T03:40:45.319 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-09-21T03:40:45.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.317+0000 7f207216df40 1 -- 172.21.15.38:0/2306811490 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:122 13.1f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x561f4c84b400 con 0x561f4c843c10 2024-09-21T03:40:45.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.318+0000 7f206bc27640 1 -- 172.21.15.38:0/2306811490 <== osd.1 v2:172.21.15.38:6816/1491818732 25 ==== osd_op_reply(122 1.00000000 [setxattr (30)] v101'65 uv65 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f205c021500 con 0x561f4c843c10 2024-09-21T03:40:45.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.318+0000 7f207216df40 1 -- 172.21.15.38:0/2306811490 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:123 13.1f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x561f4c84b400 con 0x561f4c843c10 2024-09-21T03:40:45.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.319+0000 7f206bc27640 1 -- 172.21.15.38:0/2306811490 <== osd.1 v2:172.21.15.38:6816/1491818732 26 ==== osd_op_reply(123 1.00000000 [setxattr (30)] v101'66 uv66 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f205c021500 con 0x561f4c843c10 2024-09-21T03:40:45.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.319+0000 7f207216df40 1 -- 172.21.15.38:0/2306811490 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:124 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x561f4c84a3f0 con 0x561f4c843c10 2024-09-21T03:40:45.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.321+0000 7f206bc27640 1 -- 172.21.15.38:0/2306811490 <== osd.1 v2:172.21.15.38:6816/1491818732 27 ==== osd_op_reply(124 1.00000000 [omap-set-header] v101'67 uv67 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f205c021500 con 0x561f4c843c10 2024-09-21T03:40:45.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.321+0000 7f207216df40 1 -- 172.21.15.38:0/2306811490 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:125 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=584b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x561f4c84a8a0 con 0x561f4c843c10 2024-09-21T03:40:45.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.322+0000 7f206bc27640 1 -- 172.21.15.38:0/2306811490 <== osd.1 v2:172.21.15.38:6816/1491818732 28 ==== osd_op_reply(125 1.00000000 [omap-set-vals] v101'68 uv68 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f205c021500 con 0x561f4c843c10 2024-09-21T03:40:45.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.322+0000 7f207216df40 1 -- 172.21.15.38:0/2306811490 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x561f4c83b5b0 msgr2=0x561f4c83d9d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:45.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.322+0000 7f207216df40 1 --2- 172.21.15.38:0/2306811490 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x561f4c83b5b0 0x561f4c83d9d0 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:45.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.322+0000 7f207216df40 1 -- 172.21.15.38:0/2306811490 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x561f4c843c10 msgr2=0x561f4c846030 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:45.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.322+0000 7f207216df40 1 --2- 172.21.15.38:0/2306811490 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x561f4c843c10 0x561f4c846030 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:45.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.322+0000 7f207216df40 1 -- 172.21.15.38:0/2306811490 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x561f4c847590 msgr2=0x561f4c8499b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:45.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.322+0000 7f207216df40 1 --2- 172.21.15.38:0/2306811490 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x561f4c847590 0x561f4c8499b0 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:45.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.322+0000 7f207216df40 1 -- 172.21.15.38:0/2306811490 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x561f4c83e9b0 msgr2=0x561f4c840dd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:45.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.322+0000 7f206c428640 1 -- 172.21.15.38:0/2306811490 reap_dead start 2024-09-21T03:40:45.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.322+0000 7f207216df40 1 --2- 172.21.15.38:0/2306811490 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x561f4c83e9b0 0x561f4c840dd0 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:45.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.322+0000 7f207216df40 1 -- 172.21.15.38:0/2306811490 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x561f4c81d180 msgr2=0x561f4c8390f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:45.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.322+0000 7f207216df40 1 --2- 172.21.15.38:0/2306811490 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x561f4c81d180 0x561f4c8390f0 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:45.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.322+0000 7f207216df40 1 -- 172.21.15.38:0/2306811490 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x561f4c819bf0 msgr2=0x561f4c834ba0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:45.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.322+0000 7f207216df40 1 --2- 172.21.15.38:0/2306811490 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x561f4c819bf0 0x561f4c834ba0 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:45.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.322+0000 7f207216df40 1 -- 172.21.15.38:0/2306811490 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2058077570 msgr2=0x7f2058079a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:45.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.322+0000 7f207216df40 1 --2- 172.21.15.38:0/2306811490 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2058077570 0x7f2058079a80 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f205c018040 tx=0x7f205c003020 comp rx=0 tx=0).stop 2024-09-21T03:40:45.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.322+0000 7f207216df40 1 -- 172.21.15.38:0/2306811490 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f20540030f0 msgr2=0x561f4c78e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:45.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.322+0000 7f207216df40 1 --2- 172.21.15.38:0/2306811490 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f20540030f0 0x561f4c78e460 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x561f4c6bfa40 tx=0x7f2064005ca0 comp rx=0 tx=0).stop 2024-09-21T03:40:45.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.322+0000 7f206c428640 1 -- 172.21.15.38:0/2306811490 reap_dead start 2024-09-21T03:40:45.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.322+0000 7f207216df40 1 -- 172.21.15.38:0/2306811490 shutdown_connections 2024-09-21T03:40:45.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.322+0000 7f207216df40 1 -- 172.21.15.38:0/2306811490 >> 172.21.15.38:0/2306811490 conn(0x561f4c5dcac0 msgr2=0x561f4c78f2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:45.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.322+0000 7f207216df40 1 -- 172.21.15.38:0/2306811490 shutdown_connections 2024-09-21T03:40:45.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.322+0000 7f207216df40 1 -- 172.21.15.38:0/2306811490 wait complete. 2024-09-21T03:40:45.328 DEBUG:teuthology.orchestra.run.smithi038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rm 200.00000001 2024-09-21T03:40:45.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.421+0000 7fba7f729f40 1 Processor -- start 2024-09-21T03:40:45.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.421+0000 7fba7f729f40 1 -- start start 2024-09-21T03:40:45.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.422+0000 7fba7f729f40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x564e65644d00 0x564e65672e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:45.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.422+0000 7fba7f729f40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564e65670e30 0x564e6571a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:45.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.422+0000 7fba7f729f40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564e656ff950 0x564e65671490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:45.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.422+0000 7fba7f729f40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x564e65592d60 con 0x564e656ff950 2024-09-21T03:40:45.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.422+0000 7fba7f729f40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x564e655f1430 con 0x564e65670e30 2024-09-21T03:40:45.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.422+0000 7fba7f729f40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x564e6568b1b0 con 0x564e65644d00 2024-09-21T03:40:45.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.422+0000 7fba799e4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564e656ff950 0x564e65671490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:45.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.422+0000 7fba799e4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564e656ff950 0x564e65671490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:58846/0 (socket says 172.21.15.38:58846) 2024-09-21T03:40:45.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.422+0000 7fba799e4640 1 -- 172.21.15.38:0/3048617143 learned_addr learned my addr 172.21.15.38:0/3048617143 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:45.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.423+0000 7fba791e3640 1 --2- 172.21.15.38:0/3048617143 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x564e65644d00 0x564e65672e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:45.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.423+0000 7fba73fff640 1 --2- 172.21.15.38:0/3048617143 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564e65670e30 0x564e6571a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:45.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.423+0000 7fba799e4640 1 -- 172.21.15.38:0/3048617143 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x564e65644d00 msgr2=0x564e65672e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:45.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.423+0000 7fba799e4640 1 --2- 172.21.15.38:0/3048617143 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x564e65644d00 0x564e65672e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:45.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.423+0000 7fba799e4640 1 -- 172.21.15.38:0/3048617143 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564e65670e30 msgr2=0x564e6571a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:45.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.423+0000 7fba799e4640 1 --2- 172.21.15.38:0/3048617143 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564e65670e30 0x564e6571a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:45.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.423+0000 7fba799e4640 1 -- 172.21.15.38:0/3048617143 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e656feba0 con 0x564e656ff950 2024-09-21T03:40:45.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.423+0000 7fba791e3640 1 --2- 172.21.15.38:0/3048617143 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x564e65644d00 0x564e65672e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:40:45.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.423+0000 7fba73fff640 1 --2- 172.21.15.38:0/3048617143 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564e65670e30 0x564e6571a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:40:45.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.423+0000 7fba799e4640 1 --2- 172.21.15.38:0/3048617143 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564e656ff950 0x564e65671490 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7fba74057040 tx=0x7fba74003330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ac947e03df70586 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:45.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.423+0000 7fba737fe640 1 -- 172.21.15.38:0/3048617143 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba74064030 con 0x564e656ff950 2024-09-21T03:40:45.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.423+0000 7fba737fe640 1 -- 172.21.15.38:0/3048617143 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fba74004020 con 0x564e656ff950 2024-09-21T03:40:45.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.423+0000 7fba737fe640 1 -- 172.21.15.38:0/3048617143 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba74004270 con 0x564e656ff950 2024-09-21T03:40:45.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.424+0000 7fba7f729f40 1 -- 172.21.15.38:0/3048617143 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564e656ff950 msgr2=0x564e65671490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:45.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.424+0000 7fba7f729f40 1 --2- 172.21.15.38:0/3048617143 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564e656ff950 0x564e65671490 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7fba74057040 tx=0x7fba74003330 comp rx=0 tx=0).stop 2024-09-21T03:40:45.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.424+0000 7fba7f729f40 1 -- 172.21.15.38:0/3048617143 shutdown_connections 2024-09-21T03:40:45.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.424+0000 7fba7f729f40 1 --2- 172.21.15.38:0/3048617143 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564e656ff950 0x564e65671490 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:45.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.424+0000 7fba7f729f40 1 --2- 172.21.15.38:0/3048617143 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564e65670e30 0x564e6571a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:45.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.424+0000 7fba7f729f40 1 --2- 172.21.15.38:0/3048617143 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x564e65644d00 0x564e65672e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:45.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.424+0000 7fba7f729f40 1 -- 172.21.15.38:0/3048617143 >> 172.21.15.38:0/3048617143 conn(0x564e65563ac0 msgr2=0x564e657167c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:45.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.424+0000 7fba7f729f40 1 -- 172.21.15.38:0/3048617143 shutdown_connections 2024-09-21T03:40:45.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.424+0000 7fba7f729f40 1 -- 172.21.15.38:0/3048617143 wait complete. 2024-09-21T03:40:45.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.425+0000 7fba7f729f40 1 Processor -- start 2024-09-21T03:40:45.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.425+0000 7fba7f729f40 1 -- start start 2024-09-21T03:40:45.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.426+0000 7fba7f729f40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564e65644d00 0x564e656f7b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:45.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.426+0000 7fba7f729f40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x564e65670e30 0x564e65719780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:45.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.426+0000 7fba7f729f40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564e656ff950 0x564e6570fa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:45.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.426+0000 7fba7f729f40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x564e6571a890 con 0x564e65644d00 2024-09-21T03:40:45.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.426+0000 7fba7f729f40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x564e656fee40 con 0x564e656ff950 2024-09-21T03:40:45.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.426+0000 7fba7f729f40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x564e656726c0 con 0x564e65670e30 2024-09-21T03:40:45.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.426+0000 7fba791e3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564e65644d00 0x564e656f7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:45.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.426+0000 7fba791e3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564e65644d00 0x564e656f7b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:58856/0 (socket says 172.21.15.38:58856) 2024-09-21T03:40:45.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.426+0000 7fba791e3640 1 -- 172.21.15.38:0/112065912 learned_addr learned my addr 172.21.15.38:0/112065912 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:45.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.426+0000 7fba73fff640 1 --2- 172.21.15.38:0/112065912 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x564e65670e30 0x564e65719780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:45.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.426+0000 7fba799e4640 1 --2- 172.21.15.38:0/112065912 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564e656ff950 0x564e6570fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:45.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.426+0000 7fba791e3640 1 -- 172.21.15.38:0/112065912 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x564e65670e30 msgr2=0x564e65719780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:45.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.426+0000 7fba791e3640 1 --2- 172.21.15.38:0/112065912 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x564e65670e30 0x564e65719780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:45.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.426+0000 7fba791e3640 1 -- 172.21.15.38:0/112065912 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564e656ff950 msgr2=0x564e6570fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:45.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.426+0000 7fba791e3640 1 --2- 172.21.15.38:0/112065912 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564e656ff950 0x564e6570fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:45.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.426+0000 7fba791e3640 1 -- 172.21.15.38:0/112065912 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e65671a50 con 0x564e65644d00 2024-09-21T03:40:45.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.427+0000 7fba791e3640 1 --2- 172.21.15.38:0/112065912 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564e65644d00 0x564e656f7b90 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7fba6c00a710 tx=0x7fba6c00abd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:45.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.427+0000 7fba727fc640 1 -- 172.21.15.38:0/112065912 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba6c009940 con 0x564e65644d00 2024-09-21T03:40:45.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.427+0000 7fba727fc640 1 -- 172.21.15.38:0/112065912 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fba6c009ae0 con 0x564e65644d00 2024-09-21T03:40:45.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.427+0000 7fba7f729f40 1 -- 172.21.15.38:0/112065912 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x564e656f8240 con 0x564e65644d00 2024-09-21T03:40:45.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.427+0000 7fba727fc640 1 -- 172.21.15.38:0/112065912 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba6c016420 con 0x564e65644d00 2024-09-21T03:40:45.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.427+0000 7fba7f729f40 1 -- 172.21.15.38:0/112065912 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e65719260 con 0x564e65644d00 2024-09-21T03:40:45.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.429+0000 7fba727fc640 1 -- 172.21.15.38:0/112065912 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fba6c0165c0 con 0x564e65644d00 2024-09-21T03:40:45.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.429+0000 7fba727fc640 1 --2- 172.21.15.38:0/112065912 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fba5c0775a0 0x7fba5c079ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:45.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.429+0000 7fba727fc640 1 -- 172.21.15.38:0/112065912 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 6734+0+0 (secure 0 0 0) 0x7fba6c09a4e0 con 0x564e65644d00 2024-09-21T03:40:45.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.429+0000 7fba73fff640 1 --2- 172.21.15.38:0/112065912 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fba5c0775a0 0x7fba5c079ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:45.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.429+0000 7fba7f729f40 1 --2- 172.21.15.38:0/112065912 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x564e65672f20 0x564e65717600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:45.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.429+0000 7fba7f729f40 1 -- 172.21.15.38:0/112065912 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:1 13.14 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e101) -- 0x564e65719cc0 con 0x564e65672f20 2024-09-21T03:40:45.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.430+0000 7fba799e4640 1 --2- 172.21.15.38:0/112065912 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x564e65672f20 0x564e65717600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:45.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.430+0000 7fba73fff640 1 --2- 172.21.15.38:0/112065912 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fba5c0775a0 0x7fba5c079ab0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x564e6571a740 tx=0x7fba64002920 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:45.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.430+0000 7fba799e4640 1 --2- 172.21.15.38:0/112065912 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x564e65672f20 0x564e65717600 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:45.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.431+0000 7fba799e4640 1 -- 172.21.15.38:0/112065912 <== osd.6 v2:172.21.15.195:6826/4055825675 1 ==== osd_op_reply(1 200.00000001 [delete] v101'127 uv127 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fba7400b040 con 0x564e65672f20 2024-09-21T03:40:45.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.431+0000 7fba7f729f40 1 -- 172.21.15.38:0/112065912 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x564e65672f20 msgr2=0x564e65717600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:45.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.431+0000 7fba7f729f40 1 --2- 172.21.15.38:0/112065912 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x564e65672f20 0x564e65717600 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:45.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.432+0000 7fba7f729f40 1 -- 172.21.15.38:0/112065912 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fba5c0775a0 msgr2=0x7fba5c079ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:45.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.432+0000 7fba7f729f40 1 --2- 172.21.15.38:0/112065912 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fba5c0775a0 0x7fba5c079ab0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x564e6571a740 tx=0x7fba64002920 comp rx=0 tx=0).stop 2024-09-21T03:40:45.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.432+0000 7fba7f729f40 1 -- 172.21.15.38:0/112065912 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564e65644d00 msgr2=0x564e656f7b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:45.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.432+0000 7fba7f729f40 1 --2- 172.21.15.38:0/112065912 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564e65644d00 0x564e656f7b90 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7fba6c00a710 tx=0x7fba6c00abd0 comp rx=0 tx=0).stop 2024-09-21T03:40:45.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.432+0000 7fba799e4640 1 -- 172.21.15.38:0/112065912 reap_dead start 2024-09-21T03:40:45.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.432+0000 7fba7f729f40 1 -- 172.21.15.38:0/112065912 shutdown_connections 2024-09-21T03:40:45.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.432+0000 7fba7f729f40 1 -- 172.21.15.38:0/112065912 >> 172.21.15.38:0/112065912 conn(0x564e65563ac0 msgr2=0x564e65716730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:45.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.432+0000 7fba7f729f40 1 -- 172.21.15.38:0/112065912 shutdown_connections 2024-09-21T03:40:45.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.432+0000 7fba7f729f40 1 -- 172.21.15.38:0/112065912 wait complete. 2024-09-21T03:40:45.436 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-21T03:40:45.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.553+0000 7fe74cd2b640 1 -- 172.21.15.38:0/3736412889 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe7480ceb90 msgr2=0x7fe7480cefa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:45.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.553+0000 7fe74cd2b640 1 --2- 172.21.15.38:0/3736412889 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe7480ceb90 0x7fe7480cefa0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe740005fd0 tx=0x7fe74002a540 comp rx=0 tx=0).stop 2024-09-21T03:40:45.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.553+0000 7fe74cd2b640 1 -- 172.21.15.38:0/3736412889 shutdown_connections 2024-09-21T03:40:45.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.553+0000 7fe74cd2b640 1 --2- 172.21.15.38:0/3736412889 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe7480ceb90 0x7fe7480cefa0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:45.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.553+0000 7fe74cd2b640 1 --2- 172.21.15.38:0/3736412889 >> v2:172.21.15.195:3300/0 conn(0x7fe74815ee80 0x7fe7480d0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:45.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.553+0000 7fe74cd2b640 1 -- 172.21.15.38:0/3736412889 >> 172.21.15.38:0/3736412889 conn(0x7fe74809d830 msgr2=0x7fe7480b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:45.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.553+0000 7fe74cd2b640 1 -- 172.21.15.38:0/3736412889 shutdown_connections 2024-09-21T03:40:45.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.553+0000 7fe74cd2b640 1 -- 172.21.15.38:0/3736412889 wait complete. 2024-09-21T03:40:45.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.555+0000 7fe74cd2b640 1 Processor -- start 2024-09-21T03:40:45.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.555+0000 7fe74cd2b640 1 -- start start 2024-09-21T03:40:45.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.557+0000 7fe74cd2b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe7480b9ad0 0x7fe748150980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:45.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.557+0000 7fe74cd2b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe7480ceb90 0x7fe7480cd150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:45.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.557+0000 7fe74cd2b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe74815ee80 0x7fe74815da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:45.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.557+0000 7fe74cd2b640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fe7480b6700 con 0x7fe74815ee80 2024-09-21T03:40:45.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.557+0000 7fe74cd2b640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fe7480b6580 con 0x7fe7480b9ad0 2024-09-21T03:40:45.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.557+0000 7fe74cd2b640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fe7480b6880 con 0x7fe7480ceb90 2024-09-21T03:40:45.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.557+0000 7fe746d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe74815ee80 0x7fe74815da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:45.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.557+0000 7fe746d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe74815ee80 0x7fe74815da90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:58866/0 (socket says 172.21.15.38:58866) 2024-09-21T03:40:45.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.557+0000 7fe746d9d640 1 -- 172.21.15.38:0/3296089712 learned_addr learned my addr 172.21.15.38:0/3296089712 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:45.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.557+0000 7fe74659c640 1 --2- 172.21.15.38:0/3296089712 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe7480b9ad0 0x7fe748150980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:45.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.557+0000 7fe745d9b640 1 --2- 172.21.15.38:0/3296089712 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe7480ceb90 0x7fe7480cd150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:45.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.557+0000 7fe746d9d640 1 -- 172.21.15.38:0/3296089712 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe7480ceb90 msgr2=0x7fe7480cd150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:45.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.557+0000 7fe746d9d640 1 --2- 172.21.15.38:0/3296089712 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe7480ceb90 0x7fe7480cd150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:45.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.557+0000 7fe746d9d640 1 -- 172.21.15.38:0/3296089712 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe7480b9ad0 msgr2=0x7fe748150980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:45.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.557+0000 7fe746d9d640 1 --2- 172.21.15.38:0/3296089712 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe7480b9ad0 0x7fe748150980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:45.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.557+0000 7fe746d9d640 1 -- 172.21.15.38:0/3296089712 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7481f56c0 con 0x7fe74815ee80 2024-09-21T03:40:45.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.557+0000 7fe746d9d640 1 --2- 172.21.15.38:0/3296089712 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe74815ee80 0x7fe74815da90 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7fe740009230 tx=0x7fe740003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:45.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.558+0000 7fe736ffd640 1 -- 172.21.15.38:0/3296089712 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe74003e030 con 0x7fe74815ee80 2024-09-21T03:40:45.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.558+0000 7fe736ffd640 1 -- 172.21.15.38:0/3296089712 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe740003d10 con 0x7fe74815ee80 2024-09-21T03:40:45.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.558+0000 7fe736ffd640 1 -- 172.21.15.38:0/3296089712 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe740035040 con 0x7fe74815ee80 2024-09-21T03:40:45.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.558+0000 7fe74cd2b640 1 -- 172.21.15.38:0/3296089712 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe74806eda0 con 0x7fe74815ee80 2024-09-21T03:40:45.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.558+0000 7fe74cd2b640 1 -- 172.21.15.38:0/3296089712 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7481f5c60 con 0x7fe74815ee80 2024-09-21T03:40:45.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.559+0000 7fe74cd2b640 1 -- 172.21.15.38:0/3296089712 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7480cb660 con 0x7fe74815ee80 2024-09-21T03:40:45.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.559+0000 7fe736ffd640 1 -- 172.21.15.38:0/3296089712 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fe740041030 con 0x7fe74815ee80 2024-09-21T03:40:45.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.560+0000 7fe736ffd640 1 --2- 172.21.15.38:0/3296089712 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fe728078b90 0x7fe72807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:45.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.560+0000 7fe736ffd640 1 -- 172.21.15.38:0/3296089712 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 6734+0+0 (secure 0 0 0) 0x7fe7400bb960 con 0x7fe74815ee80 2024-09-21T03:40:45.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.560+0000 7fe74659c640 1 --2- 172.21.15.38:0/3296089712 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fe728078b90 0x7fe72807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:45.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.561+0000 7fe74659c640 1 --2- 172.21.15.38:0/3296089712 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fe728078b90 0x7fe72807b0a0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fe738005fd0 tx=0x7fe738009500 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:45.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.564+0000 7fe736ffd640 1 -- 172.21.15.38:0/3296089712 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fe740083900 con 0x7fe74815ee80 2024-09-21T03:40:45.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:45.791+0000 7fe74cd2b640 1 -- 172.21.15.38:0/3296089712 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7fe7480b6d30 con 0x7fe74815ee80 2024-09-21T03:40:46.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:46.088+0000 7fe736ffd640 1 -- 172.21.15.38:0/3296089712 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v114) ==== 162+0+0 (secure 0 0 0) 0x7fe740083200 con 0x7fe74815ee80 2024-09-21T03:40:46.090 INFO:teuthology.orchestra.run.smithi038.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-21T03:40:46.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:46.090+0000 7fe74cd2b640 1 -- 172.21.15.38:0/3296089712 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fe728078b90 msgr2=0x7fe72807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:46.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:46.090+0000 7fe74cd2b640 1 --2- 172.21.15.38:0/3296089712 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fe728078b90 0x7fe72807b0a0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fe738005fd0 tx=0x7fe738009500 comp rx=0 tx=0).stop 2024-09-21T03:40:46.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:46.090+0000 7fe74cd2b640 1 -- 172.21.15.38:0/3296089712 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe74815ee80 msgr2=0x7fe74815da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:46.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:46.090+0000 7fe74cd2b640 1 --2- 172.21.15.38:0/3296089712 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe74815ee80 0x7fe74815da90 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7fe740009230 tx=0x7fe740003820 comp rx=0 tx=0).stop 2024-09-21T03:40:46.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:46.090+0000 7fe74cd2b640 1 -- 172.21.15.38:0/3296089712 shutdown_connections 2024-09-21T03:40:46.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:46.090+0000 7fe74cd2b640 1 --2- 172.21.15.38:0/3296089712 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe74815ee80 0x7fe74815da90 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:46.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:46.090+0000 7fe74cd2b640 1 --2- 172.21.15.38:0/3296089712 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fe728078b90 0x7fe72807b0a0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:46.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:46.090+0000 7fe74cd2b640 1 --2- 172.21.15.38:0/3296089712 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe7480ceb90 0x7fe7480cd150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:46.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:46.090+0000 7fe74cd2b640 1 --2- 172.21.15.38:0/3296089712 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe7480b9ad0 0x7fe748150980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:46.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:46.090+0000 7fe74cd2b640 1 -- 172.21.15.38:0/3296089712 >> 172.21.15.38:0/3296089712 conn(0x7fe74809d830 msgr2=0x7fe7480b5840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:46.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:46.090+0000 7fe74cd2b640 1 -- 172.21.15.38:0/3296089712 shutdown_connections 2024-09-21T03:40:46.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:46.090+0000 7fe74cd2b640 1 -- 172.21.15.38:0/3296089712 wait complete. 2024-09-21T03:40:46.106 INFO:tasks.ceph.mds.a.smithi038.stderr:2024-09-21T03:40:46.104+0000 7f9668d00640 -1 log_channel(cluster) log [ERR] : missing journal object 2024-09-21T03:40:46.107 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:40:46.107 INFO:tasks.ceph.mon.a.smithi038.stderr:2024-09-21T03:40:46.105+0000 7f3f45713640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T03:40:46.107 INFO:tasks.ceph.mon.a.smithi038.stderr:2024-09-21T03:40:46.105+0000 7f3f45713640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-09-21T03:40:46.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:46.223+0000 7f7a26e63640 1 -- 172.21.15.38:0/4095018187 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7a200d1ed0 msgr2=0x7f7a200cc930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:46.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:46.223+0000 7f7a26e63640 1 --2- 172.21.15.38:0/4095018187 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7a200d1ed0 0x7f7a200cc930 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f7a0c009830 tx=0x7f7a0c02a5c0 comp rx=0 tx=0).stop 2024-09-21T03:40:46.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:46.223+0000 7f7a26e63640 1 -- 172.21.15.38:0/4095018187 shutdown_connections 2024-09-21T03:40:46.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:46.223+0000 7f7a26e63640 1 --2- 172.21.15.38:0/4095018187 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7a200d1ed0 0x7f7a200cc930 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:46.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:46.223+0000 7f7a26e63640 1 -- 172.21.15.38:0/4095018187 >> 172.21.15.38:0/4095018187 conn(0x7f7a200cef40 msgr2=0x7f7a200cf350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:46.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:46.223+0000 7f7a26e63640 1 -- 172.21.15.38:0/4095018187 shutdown_connections 2024-09-21T03:40:46.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:46.223+0000 7f7a26e63640 1 -- 172.21.15.38:0/4095018187 wait complete. 2024-09-21T03:40:46.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:46.225+0000 7f7a26e63640 1 Processor -- start 2024-09-21T03:40:46.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:46.225+0000 7f7a26e63640 1 -- start start 2024-09-21T03:40:46.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:46.226+0000 7f7a26e63640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7a200cce70 0x7f7a201f48f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:46.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:46.226+0000 7f7a26e63640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7a200d1ed0 0x7f7a200cbc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:46.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:46.226+0000 7f7a26e63640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7a201536e0 0x7f7a201e2da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:46.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:46.226+0000 7f7a26e63640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f7a200d0f60 con 0x7f7a200d1ed0 2024-09-21T03:40:46.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:46.226+0000 7f7a26e63640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f7a200d0de0 con 0x7f7a201536e0 2024-09-21T03:40:46.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:46.226+0000 7f7a26e63640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f7a200d10e0 con 0x7f7a200cce70 2024-09-21T03:40:46.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:46.226+0000 7f7a1ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7a200d1ed0 0x7f7a200cbc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:46.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:46.226+0000 7f7a1ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7a200d1ed0 0x7f7a200cbc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:58884/0 (socket says 172.21.15.38:58884) 2024-09-21T03:40:46.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:46.226+0000 7f7a1ffff640 1 -- 172.21.15.38:0/1106795815 learned_addr learned my addr 172.21.15.38:0/1106795815 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:46.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:46.227+0000 7f7a24bff640 1 --2- 172.21.15.38:0/1106795815 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7a200cce70 0x7f7a201f48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:46.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:46.227+0000 7f7a25400640 1 --2- 172.21.15.38:0/1106795815 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7a201536e0 0x7f7a201e2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:46.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:46.227+0000 7f7a1ffff640 1 -- 172.21.15.38:0/1106795815 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7a200cce70 msgr2=0x7f7a201f48f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:46.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:46.227+0000 7f7a1ffff640 1 --2- 172.21.15.38:0/1106795815 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7a200cce70 0x7f7a201f48f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:46.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:46.227+0000 7f7a1ffff640 1 -- 172.21.15.38:0/1106795815 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7a201536e0 msgr2=0x7f7a201e2da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:46.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:46.227+0000 7f7a1ffff640 1 --2- 172.21.15.38:0/1106795815 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7a201536e0 0x7f7a201e2da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:46.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:46.227+0000 7f7a1ffff640 1 -- 172.21.15.38:0/1106795815 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a201f4fb0 con 0x7f7a200d1ed0 2024-09-21T03:40:46.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:46.227+0000 7f7a24bff640 1 --2- 172.21.15.38:0/1106795815 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7a200cce70 0x7f7a201f48f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:40:46.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:46.227+0000 7f7a25400640 1 --2- 172.21.15.38:0/1106795815 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7a201536e0 0x7f7a201e2da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:40:46.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:46.227+0000 7f7a1ffff640 1 --2- 172.21.15.38:0/1106795815 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7a200d1ed0 0x7f7a200cbc60 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f7a0c02abb0 tx=0x7f7a0c033c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:46.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:46.227+0000 7f7a1d7fa640 1 -- 172.21.15.38:0/1106795815 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a0c031b60 con 0x7f7a200d1ed0 2024-09-21T03:40:46.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:46.227+0000 7f7a1d7fa640 1 -- 172.21.15.38:0/1106795815 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7a0c031df0 con 0x7f7a200d1ed0 2024-09-21T03:40:46.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:46.227+0000 7f7a26e63640 1 -- 172.21.15.38:0/1106795815 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a201f5290 con 0x7f7a200d1ed0 2024-09-21T03:40:46.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:46.227+0000 7f7a1d7fa640 1 -- 172.21.15.38:0/1106795815 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a0c005280 con 0x7f7a200d1ed0 2024-09-21T03:40:46.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:46.227+0000 7f7a26e63640 1 -- 172.21.15.38:0/1106795815 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a201f5850 con 0x7f7a200d1ed0 2024-09-21T03:40:46.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:46.228+0000 7f7a26e63640 1 -- 172.21.15.38:0/1106795815 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a201576e0 con 0x7f7a200d1ed0 2024-09-21T03:40:46.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:46.229+0000 7f7a1d7fa640 1 -- 172.21.15.38:0/1106795815 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f7a0c005420 con 0x7f7a200d1ed0 2024-09-21T03:40:46.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:46.230+0000 7f7a1d7fa640 1 --2- 172.21.15.38:0/1106795815 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7a04078bc0 0x7f7a0407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:46.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:46.230+0000 7f7a1d7fa640 1 -- 172.21.15.38:0/1106795815 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 6820+0+0 (secure 0 0 0) 0x7f7a0c0bc450 con 0x7f7a200d1ed0 2024-09-21T03:40:46.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:46.230+0000 7f7a24bff640 1 --2- 172.21.15.38:0/1106795815 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7a04078bc0 0x7f7a0407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:46.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:46.230+0000 7f7a24bff640 1 --2- 172.21.15.38:0/1106795815 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7a04078bc0 0x7f7a0407b0d0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f7a14007fd0 tx=0x7f7a14009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:46.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:46.234+0000 7f7a1d7fa640 1 -- 172.21.15.38:0/1106795815 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f7a0c03f030 con 0x7f7a200d1ed0 2024-09-21T03:40:46.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:46.458+0000 7f7a26e63640 1 -- 172.21.15.38:0/1106795815 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7a200cc930 con 0x7f7a200d1ed0 2024-09-21T03:40:46.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:46.458+0000 7f7a1d7fa640 1 -- 172.21.15.38:0/1106795815 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 116 v116) ==== 95+0+3566 (secure 0 0 0) 0x7f7a0c031770 con 0x7f7a200d1ed0 2024-09-21T03:40:46.464 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:40:46.464 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":116,"btime":"2024-09-21T03:40:46:105792+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6086,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/965972663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":965972663},{"type":"v1","addr":"172.21.15.195:6836","nonce":965972663}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6700,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/3176122334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":3176122334},{"type":"v1","addr":"172.21.15.195:6837","nonce":3176122334}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":116,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:40:02.319282+0000","modified":"2024-09-21T03:40:46.105727+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":102,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-21T03:40:46.465 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 116 2024-09-21T03:40:46.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:46.460+0000 7f7a26e63640 1 -- 172.21.15.38:0/1106795815 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7a04078bc0 msgr2=0x7f7a0407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:46.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:46.460+0000 7f7a26e63640 1 --2- 172.21.15.38:0/1106795815 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7a04078bc0 0x7f7a0407b0d0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f7a14007fd0 tx=0x7f7a14009040 comp rx=0 tx=0).stop 2024-09-21T03:40:46.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:46.460+0000 7f7a26e63640 1 -- 172.21.15.38:0/1106795815 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7a200d1ed0 msgr2=0x7f7a200cbc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:46.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:46.460+0000 7f7a26e63640 1 --2- 172.21.15.38:0/1106795815 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7a200d1ed0 0x7f7a200cbc60 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f7a0c02abb0 tx=0x7f7a0c033c10 comp rx=0 tx=0).stop 2024-09-21T03:40:46.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:46.460+0000 7f7a26e63640 1 -- 172.21.15.38:0/1106795815 shutdown_connections 2024-09-21T03:40:46.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:46.460+0000 7f7a26e63640 1 --2- 172.21.15.38:0/1106795815 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7a201536e0 0x7f7a201e2da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:46.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:46.460+0000 7f7a26e63640 1 --2- 172.21.15.38:0/1106795815 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7a200d1ed0 0x7f7a200cbc60 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:46.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:46.460+0000 7f7a26e63640 1 --2- 172.21.15.38:0/1106795815 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7a04078bc0 0x7f7a0407b0d0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:46.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:46.460+0000 7f7a26e63640 1 --2- 172.21.15.38:0/1106795815 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7a200cce70 0x7f7a201f48f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:46.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:46.460+0000 7f7a26e63640 1 -- 172.21.15.38:0/1106795815 >> 172.21.15.38:0/1106795815 conn(0x7f7a200cef40 msgr2=0x7f7a200b3ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:46.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:46.460+0000 7f7a26e63640 1 -- 172.21.15.38:0/1106795815 shutdown_connections 2024-09-21T03:40:46.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:46.460+0000 7f7a26e63640 1 -- 172.21.15.38:0/1106795815 wait complete. 2024-09-21T03:40:46.478 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-21T03:40:46.479 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:40:46.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:46.601+0000 7f42ba271640 1 -- 172.21.15.38:0/2908158508 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f42b4074cb0 msgr2=0x7f42b4073dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:46.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:46.601+0000 7f42ba271640 1 --2- 172.21.15.38:0/2908158508 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f42b4074cb0 0x7f42b4073dd0 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f42b415d980 tx=0x7f42a402a5f0 comp rx=0 tx=0).stop 2024-09-21T03:40:46.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:46.601+0000 7f42ba271640 1 -- 172.21.15.38:0/2908158508 shutdown_connections 2024-09-21T03:40:46.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:46.601+0000 7f42ba271640 1 --2- 172.21.15.38:0/2908158508 >> v2:172.21.15.195:3300/0 conn(0x7f42b406e500 0x7f42b40cc020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:46.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:46.601+0000 7f42ba271640 1 --2- 172.21.15.38:0/2908158508 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f42b4074cb0 0x7f42b4073dd0 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:46.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:46.601+0000 7f42ba271640 1 -- 172.21.15.38:0/2908158508 >> 172.21.15.38:0/2908158508 conn(0x7f42b40cca40 msgr2=0x7f42b40cce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:46.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:46.601+0000 7f42ba271640 1 -- 172.21.15.38:0/2908158508 shutdown_connections 2024-09-21T03:40:46.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:46.601+0000 7f42ba271640 1 -- 172.21.15.38:0/2908158508 wait complete. 2024-09-21T03:40:46.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:46.603+0000 7f42ba271640 1 Processor -- start 2024-09-21T03:40:46.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:46.603+0000 7f42ba271640 1 -- start start 2024-09-21T03:40:46.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:46.604+0000 7f42ba271640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f42b4074cb0 0x7f42b40b12e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:46.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:46.604+0000 7f42ba271640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f42b40cbc10 0x7f42b409c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:46.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:46.604+0000 7f42ba271640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f42b40cb680 0x7f42b409ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:46.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:46.604+0000 7f42ba271640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f42b406f360 con 0x7f42b4074cb0 2024-09-21T03:40:46.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:46.604+0000 7f42ba271640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f42b406f1e0 con 0x7f42b40cbc10 2024-09-21T03:40:46.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:46.604+0000 7f42ba271640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f42b406f4e0 con 0x7f42b40cb680 2024-09-21T03:40:46.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:46.605+0000 7f42b3fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f42b4074cb0 0x7f42b40b12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:46.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:46.605+0000 7f42b3fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f42b4074cb0 0x7f42b40b12e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:58896/0 (socket says 172.21.15.38:58896) 2024-09-21T03:40:46.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:46.605+0000 7f42b3fff640 1 -- 172.21.15.38:0/734185928 learned_addr learned my addr 172.21.15.38:0/734185928 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:46.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:46.605+0000 7f42b37fe640 1 --2- 172.21.15.38:0/734185928 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f42b40cbc10 0x7f42b409c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:46.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:46.605+0000 7f42b880e640 1 --2- 172.21.15.38:0/734185928 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f42b40cb680 0x7f42b409ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:46.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:46.605+0000 7f42b3fff640 1 -- 172.21.15.38:0/734185928 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f42b40cb680 msgr2=0x7f42b409ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:46.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:46.605+0000 7f42b3fff640 1 --2- 172.21.15.38:0/734185928 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f42b40cb680 0x7f42b409ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:46.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:46.605+0000 7f42b3fff640 1 -- 172.21.15.38:0/734185928 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f42b40cbc10 msgr2=0x7f42b409c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:46.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:46.605+0000 7f42b3fff640 1 --2- 172.21.15.38:0/734185928 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f42b40cbc10 0x7f42b409c990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:46.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:46.605+0000 7f42b3fff640 1 -- 172.21.15.38:0/734185928 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42b409c4e0 con 0x7f42b4074cb0 2024-09-21T03:40:46.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:46.605+0000 7f42b3fff640 1 --2- 172.21.15.38:0/734185928 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f42b4074cb0 0x7f42b40b12e0 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f42a4034040 tx=0x7f42a4003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:46.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:46.605+0000 7f42b0ff9640 1 -- 172.21.15.38:0/734185928 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42a403e030 con 0x7f42b4074cb0 2024-09-21T03:40:46.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:46.605+0000 7f42b0ff9640 1 -- 172.21.15.38:0/734185928 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f42a4003e00 con 0x7f42b4074cb0 2024-09-21T03:40:46.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:46.605+0000 7f42b0ff9640 1 -- 172.21.15.38:0/734185928 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42a4035040 con 0x7f42b4074cb0 2024-09-21T03:40:46.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:46.605+0000 7f42ba271640 1 -- 172.21.15.38:0/734185928 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42b41571f0 con 0x7f42b4074cb0 2024-09-21T03:40:46.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:46.605+0000 7f42ba271640 1 -- 172.21.15.38:0/734185928 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42b4156ca0 con 0x7f42b4074cb0 2024-09-21T03:40:46.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:46.606+0000 7f42ba271640 1 -- 172.21.15.38:0/734185928 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42b406f990 con 0x7f42b4074cb0 2024-09-21T03:40:46.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:46.607+0000 7f42b0ff9640 1 -- 172.21.15.38:0/734185928 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f42a4041030 con 0x7f42b4074cb0 2024-09-21T03:40:46.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:46.608+0000 7f42b0ff9640 1 --2- 172.21.15.38:0/734185928 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f42a0078bc0 0x7f42a007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:46.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:46.608+0000 7f42b0ff9640 1 -- 172.21.15.38:0/734185928 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 6820+0+0 (secure 0 0 0) 0x7f42a40bb830 con 0x7f42b4074cb0 2024-09-21T03:40:46.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:46.608+0000 7f42b37fe640 1 --2- 172.21.15.38:0/734185928 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f42a0078bc0 0x7f42a007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:46.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:46.608+0000 7f42b37fe640 1 --2- 172.21.15.38:0/734185928 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f42a0078bc0 0x7f42a007b0d0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f42b41e90d0 tx=0x7f42a8009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:46.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:46.612+0000 7f42b0ff9640 1 -- 172.21.15.38:0/734185928 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f42a40832e0 con 0x7f42b4074cb0 2024-09-21T03:40:46.806 INFO:tasks.ceph.mds.c.smithi038.stderr: -41> 2024-09-21T03:40:42.052+0000 7f4d30d80640 -1 log_channel(cluster) log [ERR] : Error recovering journal 0x200: (2) No such file or directory 2024-09-21T03:40:46.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:46.839+0000 7f42ba271640 1 -- 172.21.15.38:0/734185928 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f42b406e700 con 0x7f42b4074cb0 2024-09-21T03:40:46.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:46.839+0000 7f42b0ff9640 1 -- 172.21.15.38:0/734185928 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 116 v116) ==== 95+0+3566 (secure 0 0 0) 0x7f42a4087d10 con 0x7f42b4074cb0 2024-09-21T03:40:46.841 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:40:46.841 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":116,"btime":"2024-09-21T03:40:46:105792+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6086,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/965972663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":965972663},{"type":"v1","addr":"172.21.15.195:6836","nonce":965972663}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6700,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/3176122334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":3176122334},{"type":"v1","addr":"172.21.15.195:6837","nonce":3176122334}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":116,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:40:02.319282+0000","modified":"2024-09-21T03:40:46.105727+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":102,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-21T03:40:46.842 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 116 2024-09-21T03:40:46.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:46.842+0000 7f42ba271640 1 -- 172.21.15.38:0/734185928 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f42a0078bc0 msgr2=0x7f42a007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:46.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:46.842+0000 7f42ba271640 1 --2- 172.21.15.38:0/734185928 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f42a0078bc0 0x7f42a007b0d0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f42b41e90d0 tx=0x7f42a8009040 comp rx=0 tx=0).stop 2024-09-21T03:40:46.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:46.842+0000 7f42ba271640 1 -- 172.21.15.38:0/734185928 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f42b4074cb0 msgr2=0x7f42b40b12e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:46.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:46.842+0000 7f42ba271640 1 --2- 172.21.15.38:0/734185928 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f42b4074cb0 0x7f42b40b12e0 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f42a4034040 tx=0x7f42a4003910 comp rx=0 tx=0).stop 2024-09-21T03:40:46.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:46.842+0000 7f42ba271640 1 -- 172.21.15.38:0/734185928 shutdown_connections 2024-09-21T03:40:46.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:46.842+0000 7f42ba271640 1 --2- 172.21.15.38:0/734185928 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f42a0078bc0 0x7f42a007b0d0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:46.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:46.842+0000 7f42ba271640 1 --2- 172.21.15.38:0/734185928 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f42b40cb680 0x7f42b409ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:46.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:46.842+0000 7f42ba271640 1 --2- 172.21.15.38:0/734185928 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f42b40cbc10 0x7f42b409c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:46.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:46.842+0000 7f42ba271640 1 --2- 172.21.15.38:0/734185928 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f42b4074cb0 0x7f42b40b12e0 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:46.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:46.842+0000 7f42ba271640 1 -- 172.21.15.38:0/734185928 >> 172.21.15.38:0/734185928 conn(0x7f42b40cca40 msgr2=0x7f42b415b250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:46.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:46.842+0000 7f42ba271640 1 -- 172.21.15.38:0/734185928 shutdown_connections 2024-09-21T03:40:46.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:46.842+0000 7f42ba271640 1 -- 172.21.15.38:0/734185928 wait complete. 2024-09-21T03:40:46.855 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-21T03:40:46.855 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:40:46.856 INFO:tasks.ceph.mds.c.smithi038.stdout:starting mds.c at 2024-09-21T03:40:46.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:46.969+0000 7f2622972640 1 -- 172.21.15.38:0/3076328376 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f261c0cfe00 msgr2=0x7f261c0d2a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:46.972 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:46.969+0000 7f2622972640 1 --2- 172.21.15.38:0/3076328376 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f261c0cfe00 0x7f261c0d2a20 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f261c0cef90 tx=0x7f260c02a560 comp rx=0 tx=0).stop 2024-09-21T03:40:46.972 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:46.969+0000 7f2622972640 1 -- 172.21.15.38:0/3076328376 shutdown_connections 2024-09-21T03:40:46.972 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:46.969+0000 7f2622972640 1 --2- 172.21.15.38:0/3076328376 >> v2:172.21.15.195:3300/0 conn(0x7f261c162590 0x7f261c1629a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:46.972 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:46.969+0000 7f2622972640 1 --2- 172.21.15.38:0/3076328376 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f261c0cfe00 0x7f261c0d2a20 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:46.972 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:46.969+0000 7f2622972640 1 -- 172.21.15.38:0/3076328376 >> 172.21.15.38:0/3076328376 conn(0x7f261c0d4440 msgr2=0x7f261c0d4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:46.972 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:46.969+0000 7f2622972640 1 -- 172.21.15.38:0/3076328376 shutdown_connections 2024-09-21T03:40:46.972 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:46.969+0000 7f2622972640 1 -- 172.21.15.38:0/3076328376 wait complete. 2024-09-21T03:40:46.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:46.971+0000 7f2622972640 1 Processor -- start 2024-09-21T03:40:46.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:46.971+0000 7f2622972640 1 -- start start 2024-09-21T03:40:46.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:46.973+0000 7f2622972640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f261c0b2e60 0x7f261c15d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:46.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:46.973+0000 7f2622972640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f261c0cfe00 0x7f261c1480a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:46.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:46.973+0000 7f2622972640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f261c162590 0x7f261c1485e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:46.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:46.973+0000 7f2622972640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f261c0b3980 con 0x7f261c162590 2024-09-21T03:40:46.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:46.973+0000 7f2622972640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f261c0cf5a0 con 0x7f261c0cfe00 2024-09-21T03:40:46.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:46.973+0000 7f2622972640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f261c0b3b00 con 0x7f261c0b2e60 2024-09-21T03:40:46.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:46.973+0000 7f2620f0f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f261c162590 0x7f261c1485e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:46.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:46.973+0000 7f2620f0f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f261c162590 0x7f261c1485e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:58946/0 (socket says 172.21.15.38:58946) 2024-09-21T03:40:46.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:46.973+0000 7f2620f0f640 1 -- 172.21.15.38:0/3608547229 learned_addr learned my addr 172.21.15.38:0/3608547229 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:46.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:46.973+0000 7f261bfff640 1 --2- 172.21.15.38:0/3608547229 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f261c0b2e60 0x7f261c15d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:46.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:46.973+0000 7f261b7fe640 1 --2- 172.21.15.38:0/3608547229 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f261c0cfe00 0x7f261c1480a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:46.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:46.973+0000 7f2620f0f640 1 -- 172.21.15.38:0/3608547229 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f261c0b2e60 msgr2=0x7f261c15d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:46.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:46.973+0000 7f2620f0f640 1 --2- 172.21.15.38:0/3608547229 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f261c0b2e60 0x7f261c15d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:46.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:46.973+0000 7f2620f0f640 1 -- 172.21.15.38:0/3608547229 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f261c0cfe00 msgr2=0x7f261c1480a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:46.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:46.973+0000 7f2620f0f640 1 --2- 172.21.15.38:0/3608547229 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f261c0cfe00 0x7f261c1480a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:46.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:46.973+0000 7f2620f0f640 1 -- 172.21.15.38:0/3608547229 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f261c148be0 con 0x7f261c162590 2024-09-21T03:40:46.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:46.973+0000 7f261b7fe640 1 --2- 172.21.15.38:0/3608547229 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f261c0cfe00 0x7f261c1480a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:40:46.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:46.973+0000 7f2620f0f640 1 --2- 172.21.15.38:0/3608547229 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f261c162590 0x7f261c1485e0 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f261400b9c0 tx=0x7f261400be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:46.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:46.973+0000 7f2618ff9640 1 -- 172.21.15.38:0/3608547229 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2614019070 con 0x7f261c162590 2024-09-21T03:40:46.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:46.973+0000 7f2622972640 1 -- 172.21.15.38:0/3608547229 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f261c148ec0 con 0x7f261c162590 2024-09-21T03:40:46.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:46.974+0000 7f2618ff9640 1 -- 172.21.15.38:0/3608547229 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f261400cc90 con 0x7f261c162590 2024-09-21T03:40:46.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:46.974+0000 7f2618ff9640 1 -- 172.21.15.38:0/3608547229 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26140146a0 con 0x7f261c162590 2024-09-21T03:40:46.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:46.974+0000 7f2622972640 1 -- 172.21.15.38:0/3608547229 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f261c15dca0 con 0x7f261c162590 2024-09-21T03:40:46.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:46.975+0000 7f2622972640 1 -- 172.21.15.38:0/3608547229 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f261c15dfb0 con 0x7f261c162590 2024-09-21T03:40:46.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:46.976+0000 7f2618ff9640 1 -- 172.21.15.38:0/3608547229 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f261401c070 con 0x7f261c162590 2024-09-21T03:40:46.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:46.976+0000 7f2618ff9640 1 --2- 172.21.15.38:0/3608547229 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2608078b90 0x7f260807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:46.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:46.976+0000 7f2618ff9640 1 -- 172.21.15.38:0/3608547229 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 6820+0+0 (secure 0 0 0) 0x7f261409af70 con 0x7f261c162590 2024-09-21T03:40:46.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:46.977+0000 7f261bfff640 1 --2- 172.21.15.38:0/3608547229 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2608078b90 0x7f260807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:46.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:46.977+0000 7f261bfff640 1 --2- 172.21.15.38:0/3608547229 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2608078b90 0x7f260807b0a0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f260c033000 tx=0x7f260c004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:46.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:46.981+0000 7f2618ff9640 1 -- 172.21.15.38:0/3608547229 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f2614062880 con 0x7f261c162590 2024-09-21T03:40:47.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:47.212+0000 7f2622972640 1 -- 172.21.15.38:0/3608547229 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f261c0d2a20 con 0x7f261c162590 2024-09-21T03:40:47.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:47.213+0000 7f2618ff9640 1 -- 172.21.15.38:0/3608547229 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 117 v117) ==== 95+0+4392 (secure 0 0 0) 0x7f26140672b0 con 0x7f261c162590 2024-09-21T03:40:47.215 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:40:47.215 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":117,"btime":"2024-09-21T03:40:47:107013+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6086,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/965972663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":965972663},{"type":"v1","addr":"172.21.15.195:6836","nonce":965972663}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6700,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/3176122334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":3176122334},{"type":"v1","addr":"172.21.15.195:6837","nonce":3176122334}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":7995,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/3710736169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":3710736169},{"type":"v1","addr":"172.21.15.38:6833","nonce":3710736169}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":117}],"filesystems":[{"mdsmap":{"epoch":116,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:40:02.319282+0000","modified":"2024-09-21T03:40:46.105727+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":102,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-21T03:40:47.215 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 117 2024-09-21T03:40:47.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:47.215+0000 7f2622972640 1 -- 172.21.15.38:0/3608547229 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2608078b90 msgr2=0x7f260807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:47.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:47.215+0000 7f2622972640 1 --2- 172.21.15.38:0/3608547229 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2608078b90 0x7f260807b0a0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f260c033000 tx=0x7f260c004730 comp rx=0 tx=0).stop 2024-09-21T03:40:47.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:47.215+0000 7f2622972640 1 -- 172.21.15.38:0/3608547229 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f261c162590 msgr2=0x7f261c1485e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:47.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:47.215+0000 7f2622972640 1 --2- 172.21.15.38:0/3608547229 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f261c162590 0x7f261c1485e0 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f261400b9c0 tx=0x7f261400be80 comp rx=0 tx=0).stop 2024-09-21T03:40:47.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:47.215+0000 7f2622972640 1 -- 172.21.15.38:0/3608547229 shutdown_connections 2024-09-21T03:40:47.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:47.215+0000 7f2622972640 1 --2- 172.21.15.38:0/3608547229 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f261c162590 0x7f261c1485e0 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:47.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:47.215+0000 7f2622972640 1 --2- 172.21.15.38:0/3608547229 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f261c0cfe00 0x7f261c1480a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:47.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:47.215+0000 7f2622972640 1 --2- 172.21.15.38:0/3608547229 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2608078b90 0x7f260807b0a0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:47.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:47.215+0000 7f2622972640 1 --2- 172.21.15.38:0/3608547229 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f261c0b2e60 0x7f261c15d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:47.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:47.215+0000 7f2622972640 1 -- 172.21.15.38:0/3608547229 >> 172.21.15.38:0/3608547229 conn(0x7f261c0d4440 msgr2=0x7f261c0d4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:47.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:47.215+0000 7f2622972640 1 -- 172.21.15.38:0/3608547229 shutdown_connections 2024-09-21T03:40:47.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:47.215+0000 7f2622972640 1 -- 172.21.15.38:0/3608547229 wait complete. 2024-09-21T03:40:47.228 INFO:tasks.cephfs.test_damage:Result: Mutation 'Delete 200.00000001' led to DAMAGED state 2024-09-21T03:40:47.228 INFO:tasks.cephfs.test_damage:Applying mutation 'Delete mds0_openfiles.0' 2024-09-21T03:40:47.229 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-21T03:40:47.229 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:40:47.245 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056795,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2354 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T03:40:47.245 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi038.front.sepia.ceph.com 2024-09-21T03:40:47.245 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:40:47.245 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T03:40:47.300 INFO:teuthology.orchestra.run.smithi038.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-09-21T03:40:47.301 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T03:40:47.301 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-21T03:40:47.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:47.458+0000 7f7af1815640 1 -- 172.21.15.38:0/848471633 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7ad8004d40 msgr2=0x7f7ad8005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:47.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:47.458+0000 7f7af1815640 1 --2- 172.21.15.38:0/848471633 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7ad8004d40 0x7f7ad8005150 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f7ae4001050 tx=0x7f7ad8005830 comp rx=0 tx=0).stop 2024-09-21T03:40:47.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:47.458+0000 7f7aeb7fe640 1 -- 172.21.15.38:0/848471633 reap_dead start 2024-09-21T03:40:47.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:47.458+0000 7f7af1815640 1 -- 172.21.15.38:0/848471633 shutdown_connections 2024-09-21T03:40:47.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:47.458+0000 7f7af1815640 1 -- 172.21.15.38:0/848471633 >> 172.21.15.38:0/848471633 conn(0x7f7aec0b46e0 msgr2=0x7f7aec0d23e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:47.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:47.458+0000 7f7af1815640 1 -- 172.21.15.38:0/848471633 shutdown_connections 2024-09-21T03:40:47.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:47.458+0000 7f7af1815640 1 -- 172.21.15.38:0/848471633 wait complete. 2024-09-21T03:40:47.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:47.460+0000 7f7af1815640 1 Processor -- start 2024-09-21T03:40:47.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:47.460+0000 7f7af1815640 1 -- start start 2024-09-21T03:40:47.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:47.461+0000 7f7af1815640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7ad8004d40 0x7f7aec14c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:47.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:47.461+0000 7f7af1815640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7aec06f4a0 0x7f7aec06f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:47.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:47.461+0000 7f7af1815640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7aec0cb660 0x7f7aec0cba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:47.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:47.461+0000 7f7af1815640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f7aec0d0670 con 0x7f7aec0cb660 2024-09-21T03:40:47.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:47.461+0000 7f7af1815640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f7aec0b6e00 con 0x7f7aec06f4a0 2024-09-21T03:40:47.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:47.461+0000 7f7af1815640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f7aec0d07f0 con 0x7f7ad8004d40 2024-09-21T03:40:47.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:47.461+0000 7f7aeb7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7aec0cb660 0x7f7aec0cba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:47.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:47.461+0000 7f7aeb7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7aec0cb660 0x7f7aec0cba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:58966/0 (socket says 172.21.15.38:58966) 2024-09-21T03:40:47.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:47.461+0000 7f7aeb7fe640 1 -- 172.21.15.38:0/675710977 learned_addr learned my addr 172.21.15.38:0/675710977 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:47.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:47.462+0000 7f7aea7fc640 1 --2- 172.21.15.38:0/675710977 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7aec06f4a0 0x7f7aec06f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:47.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:47.462+0000 7f7aeaffd640 1 --2- 172.21.15.38:0/675710977 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7ad8004d40 0x7f7aec14c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:47.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:47.462+0000 7f7aeb7fe640 1 -- 172.21.15.38:0/675710977 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7ad8004d40 msgr2=0x7f7aec14c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:47.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:47.462+0000 7f7aeb7fe640 1 --2- 172.21.15.38:0/675710977 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7ad8004d40 0x7f7aec14c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:47.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:47.462+0000 7f7aeb7fe640 1 -- 172.21.15.38:0/675710977 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7aec06f4a0 msgr2=0x7f7aec06f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:47.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:47.462+0000 7f7aeb7fe640 1 --2- 172.21.15.38:0/675710977 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7aec06f4a0 0x7f7aec06f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:47.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:47.462+0000 7f7aeb7fe640 1 -- 172.21.15.38:0/675710977 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7aec0cbfb0 con 0x7f7aec0cb660 2024-09-21T03:40:47.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:47.462+0000 7f7aeb7fe640 1 --2- 172.21.15.38:0/675710977 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7aec0cb660 0x7f7aec0cba70 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f7aec0cc9e0 tx=0x7f7ae4006060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:47.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:47.462+0000 7f7ae3fff640 1 -- 172.21.15.38:0/675710977 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ae4039070 con 0x7f7aec0cb660 2024-09-21T03:40:47.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:47.462+0000 7f7ae3fff640 1 -- 172.21.15.38:0/675710977 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7ae4006700 con 0x7f7aec0cb660 2024-09-21T03:40:47.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:47.462+0000 7f7ae3fff640 1 -- 172.21.15.38:0/675710977 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ae402b880 con 0x7f7aec0cb660 2024-09-21T03:40:47.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:47.462+0000 7f7af1815640 1 -- 172.21.15.38:0/675710977 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7aec1f8c30 con 0x7f7aec0cb660 2024-09-21T03:40:47.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:47.462+0000 7f7af1815640 1 -- 172.21.15.38:0/675710977 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7aec1f91b0 con 0x7f7aec0cb660 2024-09-21T03:40:47.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:47.463+0000 7f7af1815640 1 -- 172.21.15.38:0/675710977 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7aec15ca90 con 0x7f7aec0cb660 2024-09-21T03:40:47.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:47.464+0000 7f7ae3fff640 1 -- 172.21.15.38:0/675710977 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f7ae402ba20 con 0x7f7aec0cb660 2024-09-21T03:40:47.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:47.465+0000 7f7ae3fff640 1 --2- 172.21.15.38:0/675710977 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7acc078b90 0x7f7acc07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:47.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:47.465+0000 7f7ae3fff640 1 -- 172.21.15.38:0/675710977 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 6820+0+0 (secure 0 0 0) 0x7f7ae40baa20 con 0x7f7aec0cb660 2024-09-21T03:40:47.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:47.465+0000 7f7aeaffd640 1 --2- 172.21.15.38:0/675710977 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7acc078b90 0x7f7acc07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:47.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:47.465+0000 7f7aeaffd640 1 --2- 172.21.15.38:0/675710977 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7acc078b90 0x7f7acc07b0a0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f7adc005fd0 tx=0x7f7adc009500 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:47.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:47.469+0000 7f7ae3fff640 1 -- 172.21.15.38:0/675710977 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f7aec0cbfb0 con 0x7f7aec0cb660 2024-09-21T03:40:47.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:47.696+0000 7f7af1815640 1 -- 172.21.15.38:0/675710977 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f7aec0b6690 con 0x7f7aec0cb660 2024-09-21T03:40:48.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:48.112+0000 7f7ae3fff640 1 -- 172.21.15.38:0/675710977 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v118) ==== 161+0+0 (secure 0 0 0) 0x7f7aec0d0fc0 con 0x7f7aec0cb660 2024-09-21T03:40:48.114 INFO:teuthology.orchestra.run.smithi038.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-21T03:40:48.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:48.114+0000 7f7af1815640 1 -- 172.21.15.38:0/675710977 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7acc078b90 msgr2=0x7f7acc07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:48.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:48.114+0000 7f7af1815640 1 --2- 172.21.15.38:0/675710977 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7acc078b90 0x7f7acc07b0a0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f7adc005fd0 tx=0x7f7adc009500 comp rx=0 tx=0).stop 2024-09-21T03:40:48.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:48.114+0000 7f7af1815640 1 -- 172.21.15.38:0/675710977 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7aec0cb660 msgr2=0x7f7aec0cba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:48.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:48.114+0000 7f7af1815640 1 --2- 172.21.15.38:0/675710977 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7aec0cb660 0x7f7aec0cba70 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f7aec0cc9e0 tx=0x7f7ae4006060 comp rx=0 tx=0).stop 2024-09-21T03:40:48.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:48.114+0000 7f7af1815640 1 -- 172.21.15.38:0/675710977 shutdown_connections 2024-09-21T03:40:48.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:48.115+0000 7f7af1815640 1 --2- 172.21.15.38:0/675710977 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7aec0cb660 0x7f7aec0cba70 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:48.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:48.115+0000 7f7af1815640 1 --2- 172.21.15.38:0/675710977 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7aec06f4a0 0x7f7aec06f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:48.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:48.115+0000 7f7af1815640 1 --2- 172.21.15.38:0/675710977 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7acc078b90 0x7f7acc07b0a0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:48.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:48.115+0000 7f7af1815640 1 --2- 172.21.15.38:0/675710977 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7ad8004d40 0x7f7aec14c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:48.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:48.115+0000 7f7af1815640 1 -- 172.21.15.38:0/675710977 >> 172.21.15.38:0/675710977 conn(0x7f7aec0b46e0 msgr2=0x7f7aec0c7ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:48.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:48.115+0000 7f7af1815640 1 -- 172.21.15.38:0/675710977 shutdown_connections 2024-09-21T03:40:48.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:48.115+0000 7f7af1815640 1 -- 172.21.15.38:0/675710977 wait complete. 2024-09-21T03:40:48.130 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-09-21T03:40:48.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:48.250+0000 7fa40ef0b640 1 -- 172.21.15.38:0/4134018583 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa4080cd440 msgr2=0x7fa4080d0670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:48.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:48.250+0000 7fa40ef0b640 1 --2- 172.21.15.38:0/4134018583 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa4080cd440 0x7fa4080d0670 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7fa3fc009830 tx=0x7fa3fc02a480 comp rx=0 tx=0).stop 2024-09-21T03:40:48.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:48.250+0000 7fa40ef0b640 1 -- 172.21.15.38:0/4134018583 shutdown_connections 2024-09-21T03:40:48.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:48.250+0000 7fa40ef0b640 1 --2- 172.21.15.38:0/4134018583 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa4080cd440 0x7fa4080d0670 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:48.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:48.250+0000 7fa40ef0b640 1 --2- 172.21.15.38:0/4134018583 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa4080ccc40 0x7fa40815ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:48.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:48.250+0000 7fa40ef0b640 1 -- 172.21.15.38:0/4134018583 >> 172.21.15.38:0/4134018583 conn(0x7fa4080b46e0 msgr2=0x7fa4080d23e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:48.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:48.250+0000 7fa40ef0b640 1 -- 172.21.15.38:0/4134018583 shutdown_connections 2024-09-21T03:40:48.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:48.250+0000 7fa40ef0b640 1 -- 172.21.15.38:0/4134018583 wait complete. 2024-09-21T03:40:48.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:48.252+0000 7fa40ef0b640 1 Processor -- start 2024-09-21T03:40:48.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:48.252+0000 7fa40ef0b640 1 -- start start 2024-09-21T03:40:48.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:48.253+0000 7fa40ef0b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa4080ccc40 0x7fa4081f5320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:48.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:48.253+0000 7fa40ef0b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa4080cd440 0x7fa40815f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:48.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:48.253+0000 7fa40ef0b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa4080d0bb0 0x7fa4080b0a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:48.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:48.253+0000 7fa40ef0b640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa408070810 con 0x7fa4080cd440 2024-09-21T03:40:48.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:48.253+0000 7fa40ef0b640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fa4080b6e00 con 0x7fa4080d0bb0 2024-09-21T03:40:48.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:48.253+0000 7fa40ef0b640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fa408070990 con 0x7fa4080ccc40 2024-09-21T03:40:48.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:48.254+0000 7fa407fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa4080cd440 0x7fa40815f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:48.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:48.254+0000 7fa407fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa4080cd440 0x7fa40815f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:58978/0 (socket says 172.21.15.38:58978) 2024-09-21T03:40:48.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:48.254+0000 7fa407fff640 1 -- 172.21.15.38:0/693910238 learned_addr learned my addr 172.21.15.38:0/693910238 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:48.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:48.254+0000 7fa40cca7640 1 --2- 172.21.15.38:0/693910238 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa4080ccc40 0x7fa4081f5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:48.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:48.254+0000 7fa40d4a8640 1 --2- 172.21.15.38:0/693910238 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa4080d0bb0 0x7fa4080b0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:48.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:48.254+0000 7fa407fff640 1 -- 172.21.15.38:0/693910238 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa4080ccc40 msgr2=0x7fa4081f5320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:48.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:48.254+0000 7fa407fff640 1 --2- 172.21.15.38:0/693910238 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa4080ccc40 0x7fa4081f5320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:48.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:48.254+0000 7fa407fff640 1 -- 172.21.15.38:0/693910238 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa4080d0bb0 msgr2=0x7fa4080b0a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:48.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:48.254+0000 7fa407fff640 1 --2- 172.21.15.38:0/693910238 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa4080d0bb0 0x7fa4080b0a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:48.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:48.254+0000 7fa407fff640 1 -- 172.21.15.38:0/693910238 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4081f5e50 con 0x7fa4080cd440 2024-09-21T03:40:48.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:48.254+0000 7fa407fff640 1 --2- 172.21.15.38:0/693910238 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa4080cd440 0x7fa40815f5d0 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7fa3fc009950 tx=0x7fa3fc033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:48.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:48.254+0000 7fa4057fa640 1 -- 172.21.15.38:0/693910238 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3fc005c90 con 0x7fa4080cd440 2024-09-21T03:40:48.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:48.254+0000 7fa4057fa640 1 -- 172.21.15.38:0/693910238 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa3fc031e50 con 0x7fa4080cd440 2024-09-21T03:40:48.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:48.254+0000 7fa40ef0b640 1 -- 172.21.15.38:0/693910238 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4081f6130 con 0x7fa4080cd440 2024-09-21T03:40:48.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:48.254+0000 7fa4057fa640 1 -- 172.21.15.38:0/693910238 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3fc005260 con 0x7fa4080cd440 2024-09-21T03:40:48.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:48.254+0000 7fa40ef0b640 1 -- 172.21.15.38:0/693910238 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4081f69d0 con 0x7fa4080cd440 2024-09-21T03:40:48.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:48.255+0000 7fa40ef0b640 1 -- 172.21.15.38:0/693910238 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4081f5bc0 con 0x7fa4080cd440 2024-09-21T03:40:48.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:48.256+0000 7fa4057fa640 1 -- 172.21.15.38:0/693910238 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa3fc005400 con 0x7fa4080cd440 2024-09-21T03:40:48.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:48.257+0000 7fa4057fa640 1 --2- 172.21.15.38:0/693910238 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa3f4078b90 0x7fa3f407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:48.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:48.257+0000 7fa4057fa640 1 -- 172.21.15.38:0/693910238 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 6820+0+0 (secure 0 0 0) 0x7fa3fc0bc2c0 con 0x7fa4080cd440 2024-09-21T03:40:48.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:48.257+0000 7fa40cca7640 1 --2- 172.21.15.38:0/693910238 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa3f4078b90 0x7fa3f407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:48.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:48.257+0000 7fa40cca7640 1 --2- 172.21.15.38:0/693910238 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa3f4078b90 0x7fa3f407b0a0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fa3f800bb50 tx=0x7fa3f800b2e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:48.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:48.261+0000 7fa4057fa640 1 -- 172.21.15.38:0/693910238 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fa3fc03f030 con 0x7fa4080cd440 2024-09-21T03:40:48.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:48.527+0000 7fa40ef0b640 1 -- 172.21.15.38:0/693910238 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7fa408070ed0 con 0x7fa4080cd440 2024-09-21T03:40:49.126 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.124+0000 7fa4057fa640 1 -- 172.21.15.38:0/693910238 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v119) ==== 101+0+0 (secure 0 0 0) 0x7fa3fc083c40 con 0x7fa4080cd440 2024-09-21T03:40:49.126 INFO:teuthology.orchestra.run.smithi038.stderr:repaired: restoring rank 6:0 2024-09-21T03:40:49.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.127+0000 7fa40ef0b640 1 -- 172.21.15.38:0/693910238 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa3f4078b90 msgr2=0x7fa3f407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:49.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.127+0000 7fa40ef0b640 1 --2- 172.21.15.38:0/693910238 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa3f4078b90 0x7fa3f407b0a0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fa3f800bb50 tx=0x7fa3f800b2e0 comp rx=0 tx=0).stop 2024-09-21T03:40:49.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.127+0000 7fa40ef0b640 1 -- 172.21.15.38:0/693910238 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa4080cd440 msgr2=0x7fa40815f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:49.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.127+0000 7fa40ef0b640 1 --2- 172.21.15.38:0/693910238 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa4080cd440 0x7fa40815f5d0 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7fa3fc009950 tx=0x7fa3fc033c20 comp rx=0 tx=0).stop 2024-09-21T03:40:49.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.127+0000 7fa40ef0b640 1 -- 172.21.15.38:0/693910238 shutdown_connections 2024-09-21T03:40:49.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.127+0000 7fa40ef0b640 1 --2- 172.21.15.38:0/693910238 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa4080d0bb0 0x7fa4080b0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:49.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.127+0000 7fa40ef0b640 1 --2- 172.21.15.38:0/693910238 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa4080cd440 0x7fa40815f5d0 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:49.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.127+0000 7fa40ef0b640 1 --2- 172.21.15.38:0/693910238 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa3f4078b90 0x7fa3f407b0a0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:49.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.127+0000 7fa40ef0b640 1 --2- 172.21.15.38:0/693910238 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa4080ccc40 0x7fa4081f5320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:49.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.127+0000 7fa40ef0b640 1 -- 172.21.15.38:0/693910238 >> 172.21.15.38:0/693910238 conn(0x7fa4080b46e0 msgr2=0x7fa4080b31a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:49.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.127+0000 7fa40ef0b640 1 -- 172.21.15.38:0/693910238 shutdown_connections 2024-09-21T03:40:49.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.127+0000 7fa40ef0b640 1 -- 172.21.15.38:0/693910238 wait complete. 2024-09-21T03:40:49.142 DEBUG:teuthology.orchestra.run.smithi038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-09-21T03:40:49.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.204+0000 7f4f54979f40 1 Processor -- start 2024-09-21T03:40:49.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.204+0000 7f4f54979f40 1 -- start start 2024-09-21T03:40:49.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.205+0000 7f4f54979f40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5590c04d3d00 0x5590c04fd920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:49.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.205+0000 7f4f54979f40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5590c0500e40 0x5590c058c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:49.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.205+0000 7f4f54979f40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5590c058e950 0x5590c04fc610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:49.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.205+0000 7f4f54979f40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5590c0480430 con 0x5590c0500e40 2024-09-21T03:40:49.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.205+0000 7f4f54979f40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5590c048d1f0 con 0x5590c058e950 2024-09-21T03:40:49.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.205+0000 7f4f54979f40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x5590c0421d60 con 0x5590c04d3d00 2024-09-21T03:40:49.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.205+0000 7f4f4dc34640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5590c0500e40 0x5590c058c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:49.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.205+0000 7f4f4dc34640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5590c0500e40 0x5590c058c570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:58990/0 (socket says 172.21.15.38:58990) 2024-09-21T03:40:49.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.205+0000 7f4f4dc34640 1 -- 172.21.15.38:0/3027472802 learned_addr learned my addr 172.21.15.38:0/3027472802 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:49.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.205+0000 7f4f4ec36640 1 --2- 172.21.15.38:0/3027472802 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5590c058e950 0x5590c04fc610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:49.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.205+0000 7f4f4e435640 1 --2- 172.21.15.38:0/3027472802 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5590c04d3d00 0x5590c04fd920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:49.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.205+0000 7f4f4dc34640 1 -- 172.21.15.38:0/3027472802 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5590c04d3d00 msgr2=0x5590c04fd920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:49.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.205+0000 7f4f4dc34640 1 --2- 172.21.15.38:0/3027472802 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5590c04d3d00 0x5590c04fd920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:49.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.205+0000 7f4f4dc34640 1 -- 172.21.15.38:0/3027472802 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5590c058e950 msgr2=0x5590c04fc610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:49.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.205+0000 7f4f4dc34640 1 --2- 172.21.15.38:0/3027472802 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5590c058e950 0x5590c04fc610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:49.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.205+0000 7f4f4dc34640 1 -- 172.21.15.38:0/3027472802 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5590c04cfc50 con 0x5590c0500e40 2024-09-21T03:40:49.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.205+0000 7f4f4e435640 1 --2- 172.21.15.38:0/3027472802 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5590c04d3d00 0x5590c04fd920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:40:49.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.206+0000 7f4f4dc34640 1 --2- 172.21.15.38:0/3027472802 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5590c0500e40 0x5590c058c570 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f4f44001d20 tx=0x7f4f4400b5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=142131cfce162c6d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:49.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.206+0000 7f4f4cc32640 1 -- 172.21.15.38:0/3027472802 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f4400bd80 con 0x5590c0500e40 2024-09-21T03:40:49.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.206+0000 7f4f4cc32640 1 -- 172.21.15.38:0/3027472802 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4f4400f400 con 0x5590c0500e40 2024-09-21T03:40:49.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.206+0000 7f4f4cc32640 1 -- 172.21.15.38:0/3027472802 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f4400f890 con 0x5590c0500e40 2024-09-21T03:40:49.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.206+0000 7f4f54979f40 1 -- 172.21.15.38:0/3027472802 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5590c0500e40 msgr2=0x5590c058c570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:49.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.206+0000 7f4f54979f40 1 --2- 172.21.15.38:0/3027472802 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5590c0500e40 0x5590c058c570 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f4f44001d20 tx=0x7f4f4400b5b0 comp rx=0 tx=0).stop 2024-09-21T03:40:49.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.206+0000 7f4f54979f40 1 -- 172.21.15.38:0/3027472802 shutdown_connections 2024-09-21T03:40:49.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.206+0000 7f4f54979f40 1 --2- 172.21.15.38:0/3027472802 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5590c058e950 0x5590c04fc610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:49.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.206+0000 7f4f54979f40 1 --2- 172.21.15.38:0/3027472802 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5590c0500e40 0x5590c058c570 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:49.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.206+0000 7f4f54979f40 1 --2- 172.21.15.38:0/3027472802 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5590c04d3d00 0x5590c04fd920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:49.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.206+0000 7f4f54979f40 1 -- 172.21.15.38:0/3027472802 >> 172.21.15.38:0/3027472802 conn(0x5590c03f2ac0 msgr2=0x5590c05a57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:49.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.207+0000 7f4f54979f40 1 -- 172.21.15.38:0/3027472802 shutdown_connections 2024-09-21T03:40:49.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.207+0000 7f4f54979f40 1 -- 172.21.15.38:0/3027472802 wait complete. 2024-09-21T03:40:49.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.207+0000 7f4f54979f40 1 Processor -- start 2024-09-21T03:40:49.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.207+0000 7f4f54979f40 1 -- start start 2024-09-21T03:40:49.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.209+0000 7f4f54979f40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5590c04d3d00 0x5590c05a3610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:49.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.209+0000 7f4f54979f40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5590c0500e40 0x5590c0587100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:49.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.209+0000 7f4f54979f40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5590c058e950 0x5590c05018c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:49.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.209+0000 7f4f54979f40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5590c058c0e0 con 0x5590c058e950 2024-09-21T03:40:49.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.209+0000 7f4f54979f40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5590c04fe4a0 con 0x5590c0500e40 2024-09-21T03:40:49.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.209+0000 7f4f54979f40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x5590c04fe1e0 con 0x5590c04d3d00 2024-09-21T03:40:49.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.209+0000 7f4f4ec36640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5590c058e950 0x5590c05018c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:49.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.209+0000 7f4f4ec36640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5590c058e950 0x5590c05018c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:58998/0 (socket says 172.21.15.38:58998) 2024-09-21T03:40:49.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.209+0000 7f4f4ec36640 1 -- 172.21.15.38:0/3704611312 learned_addr learned my addr 172.21.15.38:0/3704611312 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:49.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.209+0000 7f4f4dc34640 1 --2- 172.21.15.38:0/3704611312 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5590c0500e40 0x5590c0587100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:49.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.209+0000 7f4f4e435640 1 --2- 172.21.15.38:0/3704611312 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5590c04d3d00 0x5590c05a3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:49.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.209+0000 7f4f4ec36640 1 -- 172.21.15.38:0/3704611312 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5590c04d3d00 msgr2=0x5590c05a3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:49.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.209+0000 7f4f4ec36640 1 --2- 172.21.15.38:0/3704611312 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5590c04d3d00 0x5590c05a3610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:49.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.209+0000 7f4f4ec36640 1 -- 172.21.15.38:0/3704611312 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5590c0500e40 msgr2=0x5590c0587100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:49.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.209+0000 7f4f4ec36640 1 --2- 172.21.15.38:0/3704611312 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5590c0500e40 0x5590c0587100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:49.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.209+0000 7f4f4ec36640 1 -- 172.21.15.38:0/3704611312 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5590c0587c50 con 0x5590c058e950 2024-09-21T03:40:49.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.209+0000 7f4f4dc34640 1 --2- 172.21.15.38:0/3704611312 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5590c0500e40 0x5590c0587100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:40:49.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.209+0000 7f4f4ec36640 1 --2- 172.21.15.38:0/3704611312 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5590c058e950 0x5590c05018c0 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f4f48025fd0 tx=0x7f4f48055be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:49.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.210+0000 7f4f3f7fe640 1 -- 172.21.15.38:0/3704611312 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f48055dd0 con 0x5590c058e950 2024-09-21T03:40:49.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.210+0000 7f4f3f7fe640 1 -- 172.21.15.38:0/3704611312 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4f4804d920 con 0x5590c058e950 2024-09-21T03:40:49.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.210+0000 7f4f54979f40 1 -- 172.21.15.38:0/3704611312 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5590c05879b0 con 0x5590c058e950 2024-09-21T03:40:49.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.210+0000 7f4f3f7fe640 1 -- 172.21.15.38:0/3704611312 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f48004280 con 0x5590c058e950 2024-09-21T03:40:49.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.210+0000 7f4f54979f40 1 -- 172.21.15.38:0/3704611312 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5590c05013a0 con 0x5590c058e950 2024-09-21T03:40:49.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.211+0000 7f4f3f7fe640 1 -- 172.21.15.38:0/3704611312 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f4f4805e040 con 0x5590c058e950 2024-09-21T03:40:49.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.212+0000 7f4f3f7fe640 1 --2- 172.21.15.38:0/3704611312 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4f300775a0 0x7f4f30079ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:49.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.212+0000 7f4f3f7fe640 1 -- 172.21.15.38:0/3704611312 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 6820+0+0 (secure 0 0 0) 0x7f4f480de770 con 0x5590c058e950 2024-09-21T03:40:49.215 INFO:teuthology.orchestra.run.smithi038.stdout:Importing pool 2024-09-21T03:40:49.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.212+0000 7f4f4e435640 1 --2- 172.21.15.38:0/3704611312 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4f300775a0 0x7f4f30079ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:49.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.212+0000 7f4f54979f40 1 --2- 172.21.15.38:0/3704611312 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x5590c04fcd20 0x5590c04d0330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:49.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.212+0000 7f4f54979f40 1 -- 172.21.15.38:0/3704611312 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x5590c0588420 con 0x5590c04fcd20 2024-09-21T03:40:49.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.213+0000 7f4f4dc34640 1 --2- 172.21.15.38:0/3704611312 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x5590c04fcd20 0x5590c04d0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:49.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.213+0000 7f4f4e435640 1 --2- 172.21.15.38:0/3704611312 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4f300775a0 0x7f4f30079ab0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f4f40000c00 tx=0x7f4f40003070 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:49.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.213+0000 7f4f4dc34640 1 --2- 172.21.15.38:0/3704611312 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x5590c04fcd20 0x5590c04d0330 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:49.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.215+0000 7f4f4dc34640 1 -- 172.21.15.38:0/3704611312 <== osd.7 v2:172.21.15.195:6818/1606106342 1 ==== osd_op_reply(1 601.00000000 [create] v102'164 uv138 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f4f4400f580 con 0x5590c04fcd20 2024-09-21T03:40:49.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.215+0000 7f4f54979f40 1 -- 172.21.15.38:0/3704611312 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e102) -- 0x5590c064cc00 con 0x5590c04fcd20 2024-09-21T03:40:49.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.216+0000 7f4f4dc34640 1 -- 172.21.15.38:0/3704611312 <== osd.7 v2:172.21.15.195:6818/1606106342 2 ==== osd_op_reply(2 601.00000000 [delete] v102'165 uv165 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4f4400f580 con 0x5590c04fcd20 2024-09-21T03:40:49.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.216+0000 7f4f54979f40 1 -- 172.21.15.38:0/3704611312 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x5590c064cfa0 con 0x5590c04fcd20 2024-09-21T03:40:49.219 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-09-21T03:40:49.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.218+0000 7f4f4dc34640 1 -- 172.21.15.38:0/3704611312 <== osd.7 v2:172.21.15.195:6818/1606106342 3 ==== osd_op_reply(3 601.00000000 [create] v102'166 uv166 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4f4400f580 con 0x5590c04fcd20 2024-09-21T03:40:49.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.218+0000 7f4f54979f40 1 -- 172.21.15.38:0/3704611312 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x5590c064e350 con 0x5590c04fcd20 2024-09-21T03:40:49.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.219+0000 7f4f4dc34640 1 -- 172.21.15.38:0/3704611312 <== osd.7 v2:172.21.15.195:6818/1606106342 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v102'167 uv167 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4f4400f580 con 0x5590c04fcd20 2024-09-21T03:40:49.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.219+0000 7f4f54979f40 1 -- 172.21.15.38:0/3704611312 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x5590c064e6f0 con 0x5590c04fcd20 2024-09-21T03:40:49.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.221+0000 7f4f4dc34640 1 -- 172.21.15.38:0/3704611312 <== osd.7 v2:172.21.15.195:6818/1606106342 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v102'168 uv168 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4f4400f580 con 0x5590c04fcd20 2024-09-21T03:40:49.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.221+0000 7f4f54979f40 1 -- 172.21.15.38:0/3704611312 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x5590c064d340 con 0x5590c04fcd20 2024-09-21T03:40:49.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.222+0000 7f4f4dc34640 1 -- 172.21.15.38:0/3704611312 <== osd.7 v2:172.21.15.195:6818/1606106342 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v102'169 uv169 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4f4400f580 con 0x5590c04fcd20 2024-09-21T03:40:49.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.222+0000 7f4f54979f40 1 -- 172.21.15.38:0/3704611312 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x5590c064d730 con 0x5590c04fcd20 2024-09-21T03:40:49.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.224+0000 7f4f4dc34640 1 -- 172.21.15.38:0/3704611312 <== osd.7 v2:172.21.15.195:6818/1606106342 7 ==== osd_op_reply(7 602.00000000 [create] v102'170 uv144 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f4f4400f580 con 0x5590c04fcd20 2024-09-21T03:40:49.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.224+0000 7f4f54979f40 1 -- 172.21.15.38:0/3704611312 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e102) -- 0x5590c064dad0 con 0x5590c04fcd20 2024-09-21T03:40:49.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.225+0000 7f4f4dc34640 1 -- 172.21.15.38:0/3704611312 <== osd.7 v2:172.21.15.195:6818/1606106342 8 ==== osd_op_reply(8 602.00000000 [delete] v102'171 uv171 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4f4400f580 con 0x5590c04fcd20 2024-09-21T03:40:49.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.225+0000 7f4f54979f40 1 -- 172.21.15.38:0/3704611312 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x5590c064dad0 con 0x5590c04fcd20 2024-09-21T03:40:49.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.226+0000 7f4f4dc34640 1 -- 172.21.15.38:0/3704611312 <== osd.7 v2:172.21.15.195:6818/1606106342 9 ==== osd_op_reply(9 602.00000000 [create] v102'172 uv172 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4f4400f580 con 0x5590c04fcd20 2024-09-21T03:40:49.228 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-09-21T03:40:49.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.227+0000 7f4f54979f40 1 -- 172.21.15.38:0/3704611312 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x5590c064dad0 con 0x5590c04fcd20 2024-09-21T03:40:49.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.228+0000 7f4f4dc34640 1 -- 172.21.15.38:0/3704611312 <== osd.7 v2:172.21.15.195:6818/1606106342 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v102'173 uv173 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4f4400f580 con 0x5590c04fcd20 2024-09-21T03:40:49.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.228+0000 7f4f54979f40 1 -- 172.21.15.38:0/3704611312 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x5590c064dad0 con 0x5590c04fcd20 2024-09-21T03:40:49.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.230+0000 7f4f4dc34640 1 -- 172.21.15.38:0/3704611312 <== osd.7 v2:172.21.15.195:6818/1606106342 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v102'174 uv174 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4f4400f580 con 0x5590c04fcd20 2024-09-21T03:40:49.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.230+0000 7f4f54979f40 1 -- 172.21.15.38:0/3704611312 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x5590c064dad0 con 0x5590c04fcd20 2024-09-21T03:40:49.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.231+0000 7f4f4dc34640 1 -- 172.21.15.38:0/3704611312 <== osd.7 v2:172.21.15.195:6818/1606106342 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v102'175 uv175 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4f4400f580 con 0x5590c04fcd20 2024-09-21T03:40:49.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.231+0000 7f4f54979f40 1 -- 172.21.15.38:0/3704611312 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:13 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x5590c064dad0 con 0x5590c04fcd20 2024-09-21T03:40:49.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.232+0000 7f4f4dc34640 1 -- 172.21.15.38:0/3704611312 <== osd.7 v2:172.21.15.195:6818/1606106342 13 ==== osd_op_reply(13 10000000000.00000000 [create] v102'176 uv151 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7f4f4400f580 con 0x5590c04fcd20 2024-09-21T03:40:49.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.232+0000 7f4f54979f40 1 -- 172.21.15.38:0/3704611312 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:14 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e102) -- 0x5590c064dad0 con 0x5590c04fcd20 2024-09-21T03:40:49.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.234+0000 7f4f4dc34640 1 -- 172.21.15.38:0/3704611312 <== osd.7 v2:172.21.15.195:6818/1606106342 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v102'177 uv177 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f4f4400f580 con 0x5590c04fcd20 2024-09-21T03:40:49.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.234+0000 7f4f54979f40 1 -- 172.21.15.38:0/3704611312 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:15 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x5590c064dad0 con 0x5590c04fcd20 2024-09-21T03:40:49.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.235+0000 7f4f4dc34640 1 -- 172.21.15.38:0/3704611312 <== osd.7 v2:172.21.15.195:6818/1606106342 15 ==== osd_op_reply(15 10000000000.00000000 [create] v102'178 uv178 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f4f4400f580 con 0x5590c04fcd20 2024-09-21T03:40:49.237 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-09-21T03:40:49.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.235+0000 7f4f54979f40 1 -- 172.21.15.38:0/3704611312 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:16 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x5590c064dad0 con 0x5590c04fcd20 2024-09-21T03:40:49.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.236+0000 7f4f4dc34640 1 -- 172.21.15.38:0/3704611312 <== osd.7 v2:172.21.15.195:6818/1606106342 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v102'179 uv179 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f4f4400f580 con 0x5590c04fcd20 2024-09-21T03:40:49.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.237+0000 7f4f54979f40 1 -- 172.21.15.38:0/3704611312 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:17 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x5590c064dad0 con 0x5590c04fcd20 2024-09-21T03:40:49.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.238+0000 7f4f4dc34640 1 -- 172.21.15.38:0/3704611312 <== osd.7 v2:172.21.15.195:6818/1606106342 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v102'180 uv180 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f4f4400f580 con 0x5590c04fcd20 2024-09-21T03:40:49.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.238+0000 7f4f54979f40 1 -- 172.21.15.38:0/3704611312 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:18 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x5590c064dad0 con 0x5590c04fcd20 2024-09-21T03:40:49.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.240+0000 7f4f4dc34640 1 -- 172.21.15.38:0/3704611312 <== osd.7 v2:172.21.15.195:6818/1606106342 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v102'181 uv181 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f4f4400f580 con 0x5590c04fcd20 2024-09-21T03:40:49.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.240+0000 7f4f54979f40 1 -- 172.21.15.38:0/3704611312 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:19 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=585b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x5590c064dad0 con 0x5590c04fcd20 2024-09-21T03:40:49.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.241+0000 7f4f4dc34640 1 -- 172.21.15.38:0/3704611312 <== osd.7 v2:172.21.15.195:6818/1606106342 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v102'182 uv182 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f4f4400f580 con 0x5590c04fcd20 2024-09-21T03:40:49.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.241+0000 7f4f54979f40 1 -- 172.21.15.38:0/3704611312 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:20 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x5590c064dad0 con 0x5590c04fcd20 2024-09-21T03:40:49.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.243+0000 7f4f4dc34640 1 -- 172.21.15.38:0/3704611312 <== osd.7 v2:172.21.15.195:6818/1606106342 20 ==== osd_op_reply(20 600.00000000 [create] v102'183 uv157 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f4f4400f580 con 0x5590c04fcd20 2024-09-21T03:40:49.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.243+0000 7f4f54979f40 1 -- 172.21.15.38:0/3704611312 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:21 13.10 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e102) -- 0x5590c064dad0 con 0x5590c04fcd20 2024-09-21T03:40:49.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.244+0000 7f4f4dc34640 1 -- 172.21.15.38:0/3704611312 <== osd.7 v2:172.21.15.195:6818/1606106342 21 ==== osd_op_reply(21 600.00000000 [delete] v102'184 uv184 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4f4400f580 con 0x5590c04fcd20 2024-09-21T03:40:49.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.244+0000 7f4f54979f40 1 -- 172.21.15.38:0/3704611312 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:22 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x5590c064dad0 con 0x5590c04fcd20 2024-09-21T03:40:49.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.245+0000 7f4f4dc34640 1 -- 172.21.15.38:0/3704611312 <== osd.7 v2:172.21.15.195:6818/1606106342 22 ==== osd_op_reply(22 600.00000000 [create] v102'185 uv185 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4f4400f580 con 0x5590c04fcd20 2024-09-21T03:40:49.247 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-09-21T03:40:49.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.245+0000 7f4f54979f40 1 -- 172.21.15.38:0/3704611312 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:23 13.10 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x5590c064dad0 con 0x5590c04fcd20 2024-09-21T03:40:49.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.247+0000 7f4f4dc34640 1 -- 172.21.15.38:0/3704611312 <== osd.7 v2:172.21.15.195:6818/1606106342 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v102'186 uv186 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4f4400f580 con 0x5590c04fcd20 2024-09-21T03:40:49.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.247+0000 7f4f54979f40 1 -- 172.21.15.38:0/3704611312 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:24 13.10 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x5590c064dad0 con 0x5590c04fcd20 2024-09-21T03:40:49.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.248+0000 7f4f4dc34640 1 -- 172.21.15.38:0/3704611312 <== osd.7 v2:172.21.15.195:6818/1606106342 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v102'187 uv187 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4f4400f580 con 0x5590c04fcd20 2024-09-21T03:40:49.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.248+0000 7f4f54979f40 1 -- 172.21.15.38:0/3704611312 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:25 13.10 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x5590c064dad0 con 0x5590c04fcd20 2024-09-21T03:40:49.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.250+0000 7f4f4dc34640 1 -- 172.21.15.38:0/3704611312 <== osd.7 v2:172.21.15.195:6818/1606106342 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v102'188 uv188 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4f4400f580 con 0x5590c04fcd20 2024-09-21T03:40:49.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.250+0000 7f4f54979f40 1 -- 172.21.15.38:0/3704611312 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:26 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x5590c064dad0 con 0x5590c04fcd20 2024-09-21T03:40:49.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.251+0000 7f4f4dc34640 1 -- 172.21.15.38:0/3704611312 <== osd.7 v2:172.21.15.195:6818/1606106342 26 ==== osd_op_reply(26 603.00000000 [create] v102'189 uv163 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f4f4400f580 con 0x5590c04fcd20 2024-09-21T03:40:49.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.251+0000 7f4f54979f40 1 -- 172.21.15.38:0/3704611312 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:27 13.10 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e102) -- 0x5590c064dad0 con 0x5590c04fcd20 2024-09-21T03:40:49.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.252+0000 7f4f4dc34640 1 -- 172.21.15.38:0/3704611312 <== osd.7 v2:172.21.15.195:6818/1606106342 27 ==== osd_op_reply(27 603.00000000 [delete] v102'190 uv190 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4f4400f580 con 0x5590c04fcd20 2024-09-21T03:40:49.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.252+0000 7f4f54979f40 1 -- 172.21.15.38:0/3704611312 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:28 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x5590c064dad0 con 0x5590c04fcd20 2024-09-21T03:40:49.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.254+0000 7f4f4dc34640 1 -- 172.21.15.38:0/3704611312 <== osd.7 v2:172.21.15.195:6818/1606106342 28 ==== osd_op_reply(28 603.00000000 [create] v102'191 uv191 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4f4400f580 con 0x5590c04fcd20 2024-09-21T03:40:49.256 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-09-21T03:40:49.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.254+0000 7f4f54979f40 1 -- 172.21.15.38:0/3704611312 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:29 13.10 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x5590c064dad0 con 0x5590c04fcd20 2024-09-21T03:40:49.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.255+0000 7f4f4dc34640 1 -- 172.21.15.38:0/3704611312 <== osd.7 v2:172.21.15.195:6818/1606106342 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v102'192 uv192 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4f4400f580 con 0x5590c04fcd20 2024-09-21T03:40:49.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.255+0000 7f4f54979f40 1 -- 172.21.15.38:0/3704611312 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:30 13.10 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x5590c064dad0 con 0x5590c04fcd20 2024-09-21T03:40:49.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.256+0000 7f4f4dc34640 1 -- 172.21.15.38:0/3704611312 <== osd.7 v2:172.21.15.195:6818/1606106342 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v102'193 uv193 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4f4400f580 con 0x5590c04fcd20 2024-09-21T03:40:49.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.257+0000 7f4f54979f40 1 -- 172.21.15.38:0/3704611312 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:31 13.10 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x5590c064dad0 con 0x5590c04fcd20 2024-09-21T03:40:49.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.258+0000 7f4f4dc34640 1 -- 172.21.15.38:0/3704611312 <== osd.7 v2:172.21.15.195:6818/1606106342 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v102'194 uv194 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4f4400f580 con 0x5590c04fcd20 2024-09-21T03:40:49.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.258+0000 7f4f54979f40 1 --2- 172.21.15.38:0/3704611312 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x5590c064ded0 0x5590c0650fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:49.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.258+0000 7f4f54979f40 1 -- 172.21.15.38:0/3704611312 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x5590c064dad0 con 0x5590c064ded0 2024-09-21T03:40:49.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.258+0000 7f4f4e435640 1 --2- 172.21.15.38:0/3704611312 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x5590c064ded0 0x5590c0650fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:49.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.259+0000 7f4f4e435640 1 --2- 172.21.15.38:0/3704611312 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x5590c064ded0 0x5590c0650fb0 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:49.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.260+0000 7f4f4e435640 1 -- 172.21.15.38:0/3704611312 <== osd.6 v2:172.21.15.195:6826/4055825675 1 ==== osd_op_reply(32 1.00000000.inode [create] v102'128 uv112 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f4f4000e040 con 0x5590c064ded0 2024-09-21T03:40:49.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.260+0000 7f4f54979f40 1 -- 172.21.15.38:0/3704611312 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e102) -- 0x5590c06514f0 con 0x5590c064ded0 2024-09-21T03:40:49.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.262+0000 7f4f4e435640 1 -- 172.21.15.38:0/3704611312 <== osd.6 v2:172.21.15.195:6826/4055825675 2 ==== osd_op_reply(33 1.00000000.inode [delete] v102'129 uv129 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f4f4000e040 con 0x5590c064ded0 2024-09-21T03:40:49.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.262+0000 7f4f54979f40 1 -- 172.21.15.38:0/3704611312 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x5590c0651890 con 0x5590c064ded0 2024-09-21T03:40:49.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.263+0000 7f4f4e435640 1 -- 172.21.15.38:0/3704611312 <== osd.6 v2:172.21.15.195:6826/4055825675 3 ==== osd_op_reply(34 1.00000000.inode [create] v102'130 uv130 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f4f4000e040 con 0x5590c064ded0 2024-09-21T03:40:49.265 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-09-21T03:40:49.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.263+0000 7f4f54979f40 1 -- 172.21.15.38:0/3704611312 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x5590c06520e0 con 0x5590c064ded0 2024-09-21T03:40:49.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.265+0000 7f4f4e435640 1 -- 172.21.15.38:0/3704611312 <== osd.6 v2:172.21.15.195:6826/4055825675 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v102'131 uv131 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f4f4000e040 con 0x5590c064ded0 2024-09-21T03:40:49.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.265+0000 7f4f54979f40 1 -- 172.21.15.38:0/3704611312 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x5590c0652480 con 0x5590c064ded0 2024-09-21T03:40:49.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.266+0000 7f4f4e435640 1 -- 172.21.15.38:0/3704611312 <== osd.6 v2:172.21.15.195:6826/4055825675 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v102'132 uv132 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f4f4000e040 con 0x5590c064ded0 2024-09-21T03:40:49.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.266+0000 7f4f54979f40 1 -- 172.21.15.38:0/3704611312 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x5590c0652820 con 0x5590c064ded0 2024-09-21T03:40:49.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.268+0000 7f4f4e435640 1 -- 172.21.15.38:0/3704611312 <== osd.6 v2:172.21.15.195:6826/4055825675 6 ==== osd_op_reply(37 200.00000000 [create] v102'133 uv115 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f4f4000e040 con 0x5590c064ded0 2024-09-21T03:40:49.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.268+0000 7f4f54979f40 1 -- 172.21.15.38:0/3704611312 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e102) -- 0x5590c0652c10 con 0x5590c064ded0 2024-09-21T03:40:49.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.269+0000 7f4f4e435640 1 -- 172.21.15.38:0/3704611312 <== osd.6 v2:172.21.15.195:6826/4055825675 7 ==== osd_op_reply(38 200.00000000 [delete] v102'134 uv134 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4f4000e040 con 0x5590c064ded0 2024-09-21T03:40:49.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.269+0000 7f4f54979f40 1 -- 172.21.15.38:0/3704611312 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x5590c0653000 con 0x5590c064ded0 2024-09-21T03:40:49.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.271+0000 7f4f4e435640 1 -- 172.21.15.38:0/3704611312 <== osd.6 v2:172.21.15.195:6826/4055825675 8 ==== osd_op_reply(39 200.00000000 [create] v102'135 uv135 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4f4000e040 con 0x5590c064ded0 2024-09-21T03:40:49.273 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-09-21T03:40:49.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.271+0000 7f4f54979f40 1 -- 172.21.15.38:0/3704611312 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x5590c06533f0 con 0x5590c064ded0 2024-09-21T03:40:49.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.272+0000 7f4f4e435640 1 -- 172.21.15.38:0/3704611312 <== osd.6 v2:172.21.15.195:6826/4055825675 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v102'136 uv136 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4f4000e040 con 0x5590c064ded0 2024-09-21T03:40:49.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.272+0000 7f4f54979f40 1 -- 172.21.15.38:0/3704611312 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x5590c06533f0 con 0x5590c064ded0 2024-09-21T03:40:49.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.274+0000 7f4f4e435640 1 -- 172.21.15.38:0/3704611312 <== osd.6 v2:172.21.15.195:6826/4055825675 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v102'137 uv137 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4f4000e040 con 0x5590c064ded0 2024-09-21T03:40:49.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.274+0000 7f4f54979f40 1 -- 172.21.15.38:0/3704611312 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:42 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x5590c06530e0 con 0x5590c064ded0 2024-09-21T03:40:49.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.275+0000 7f4f4e435640 1 -- 172.21.15.38:0/3704611312 <== osd.6 v2:172.21.15.195:6826/4055825675 11 ==== osd_op_reply(42 200.00000001 [create] v102'138 uv138 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4f4000e040 con 0x5590c064ded0 2024-09-21T03:40:49.277 INFO:teuthology.orchestra.run.smithi038.stdout:Write #-9223372036854775808:00000000:::200.00000001:head# 2024-09-21T03:40:49.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.275+0000 7f4f54979f40 1 -- 172.21.15.38:0/3704611312 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:43 13.14 13:2e2fa760:::200.00000001:head [write 0~17490 in=17490b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x5590c06530e0 con 0x5590c064ded0 2024-09-21T03:40:49.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.277+0000 7f4f4e435640 1 -- 172.21.15.38:0/3704611312 <== osd.6 v2:172.21.15.195:6826/4055825675 12 ==== osd_op_reply(43 200.00000001 [write 0~17490] v102'139 uv139 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4f4000e040 con 0x5590c064ded0 2024-09-21T03:40:49.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.277+0000 7f4f54979f40 1 -- 172.21.15.38:0/3704611312 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:44 13.14 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x5590c06530e0 con 0x5590c064ded0 2024-09-21T03:40:49.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.278+0000 7f4f4e435640 1 -- 172.21.15.38:0/3704611312 <== osd.6 v2:172.21.15.195:6826/4055825675 13 ==== osd_op_reply(44 200.00000001 [omap-set-header] v102'140 uv140 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4f4000e040 con 0x5590c064ded0 2024-09-21T03:40:49.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.279+0000 7f4f54979f40 1 -- 172.21.15.38:0/3704611312 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:45 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x5590c06530e0 con 0x5590c064ded0 2024-09-21T03:40:49.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.280+0000 7f4f4e435640 1 -- 172.21.15.38:0/3704611312 <== osd.6 v2:172.21.15.195:6826/4055825675 14 ==== osd_op_reply(45 606.00000000 [create] v102'141 uv126 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f4f4000e040 con 0x5590c064ded0 2024-09-21T03:40:49.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.280+0000 7f4f54979f40 1 -- 172.21.15.38:0/3704611312 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:46 13.14 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e102) -- 0x5590c06530e0 con 0x5590c064ded0 2024-09-21T03:40:49.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.281+0000 7f4f4e435640 1 -- 172.21.15.38:0/3704611312 <== osd.6 v2:172.21.15.195:6826/4055825675 15 ==== osd_op_reply(46 606.00000000 [delete] v102'142 uv142 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4f4000e040 con 0x5590c064ded0 2024-09-21T03:40:49.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.281+0000 7f4f54979f40 1 -- 172.21.15.38:0/3704611312 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:47 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x5590c06530e0 con 0x5590c064ded0 2024-09-21T03:40:49.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.283+0000 7f4f4e435640 1 -- 172.21.15.38:0/3704611312 <== osd.6 v2:172.21.15.195:6826/4055825675 16 ==== osd_op_reply(47 606.00000000 [create] v102'143 uv143 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4f4000e040 con 0x5590c064ded0 2024-09-21T03:40:49.285 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-09-21T03:40:49.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.283+0000 7f4f54979f40 1 -- 172.21.15.38:0/3704611312 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:48 13.14 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x5590c06540f0 con 0x5590c064ded0 2024-09-21T03:40:49.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.284+0000 7f4f4e435640 1 -- 172.21.15.38:0/3704611312 <== osd.6 v2:172.21.15.195:6826/4055825675 17 ==== osd_op_reply(48 606.00000000 [setxattr (30)] v102'144 uv144 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4f4000e040 con 0x5590c064ded0 2024-09-21T03:40:49.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.284+0000 7f4f54979f40 1 -- 172.21.15.38:0/3704611312 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:49 13.14 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x5590c06540f0 con 0x5590c064ded0 2024-09-21T03:40:49.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.285+0000 7f4f4e435640 1 -- 172.21.15.38:0/3704611312 <== osd.6 v2:172.21.15.195:6826/4055825675 18 ==== osd_op_reply(49 606.00000000 [setxattr (62)] v102'145 uv145 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4f4000e040 con 0x5590c064ded0 2024-09-21T03:40:49.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.285+0000 7f4f54979f40 1 -- 172.21.15.38:0/3704611312 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:50 13.14 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x5590c06530e0 con 0x5590c064ded0 2024-09-21T03:40:49.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.287+0000 7f4f4e435640 1 -- 172.21.15.38:0/3704611312 <== osd.6 v2:172.21.15.195:6826/4055825675 19 ==== osd_op_reply(50 606.00000000 [omap-set-header] v102'146 uv146 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4f4000e040 con 0x5590c064ded0 2024-09-21T03:40:49.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.287+0000 7f4f54979f40 1 --2- 172.21.15.38:0/3704611312 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x5590c0653760 0x5590c0655b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:49.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.287+0000 7f4f54979f40 1 -- 172.21.15.38:0/3704611312 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:51 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x5590c0656190 con 0x5590c0653760 2024-09-21T03:40:49.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.287+0000 7f4f4dc34640 1 --2- 172.21.15.38:0/3704611312 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x5590c0653760 0x5590c0655b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:49.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.288+0000 7f4f4dc34640 1 --2- 172.21.15.38:0/3704611312 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x5590c0653760 0x5590c0655b80 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:49.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.289+0000 7f4f4dc34640 1 -- 172.21.15.38:0/3704611312 <== osd.0 v2:172.21.15.38:6800/2381639433 1 ==== osd_op_reply(51 607.00000000 [create] v102'33 uv32 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f4f4400f580 con 0x5590c0653760 2024-09-21T03:40:49.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.289+0000 7f4f54979f40 1 -- 172.21.15.38:0/3704611312 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:52 13.1c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e102) -- 0x5590c06565b0 con 0x5590c0653760 2024-09-21T03:40:49.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.290+0000 7f4f4dc34640 1 -- 172.21.15.38:0/3704611312 <== osd.0 v2:172.21.15.38:6800/2381639433 2 ==== osd_op_reply(52 607.00000000 [delete] v102'34 uv34 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4f4400f580 con 0x5590c0653760 2024-09-21T03:40:49.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.290+0000 7f4f54979f40 1 -- 172.21.15.38:0/3704611312 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:53 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x5590c06565b0 con 0x5590c0653760 2024-09-21T03:40:49.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.291+0000 7f4f4dc34640 1 -- 172.21.15.38:0/3704611312 <== osd.0 v2:172.21.15.38:6800/2381639433 3 ==== osd_op_reply(53 607.00000000 [create] v102'35 uv35 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4f4400f580 con 0x5590c0653760 2024-09-21T03:40:49.293 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-09-21T03:40:49.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.291+0000 7f4f54979f40 1 -- 172.21.15.38:0/3704611312 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:54 13.1c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x5590c06575c0 con 0x5590c0653760 2024-09-21T03:40:49.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.292+0000 7f4f4dc34640 1 -- 172.21.15.38:0/3704611312 <== osd.0 v2:172.21.15.38:6800/2381639433 4 ==== osd_op_reply(54 607.00000000 [setxattr (30)] v102'36 uv36 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4f4400f580 con 0x5590c0653760 2024-09-21T03:40:49.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.292+0000 7f4f54979f40 1 -- 172.21.15.38:0/3704611312 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:55 13.1c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x5590c06575c0 con 0x5590c0653760 2024-09-21T03:40:49.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.293+0000 7f4f4dc34640 1 -- 172.21.15.38:0/3704611312 <== osd.0 v2:172.21.15.38:6800/2381639433 5 ==== osd_op_reply(55 607.00000000 [setxattr (62)] v102'37 uv37 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4f4400f580 con 0x5590c0653760 2024-09-21T03:40:49.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.293+0000 7f4f54979f40 1 -- 172.21.15.38:0/3704611312 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:56 13.1c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x5590c06565b0 con 0x5590c0653760 2024-09-21T03:40:49.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.294+0000 7f4f4dc34640 1 -- 172.21.15.38:0/3704611312 <== osd.0 v2:172.21.15.38:6800/2381639433 6 ==== osd_op_reply(56 607.00000000 [omap-set-header] v102'38 uv38 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4f4400f580 con 0x5590c0653760 2024-09-21T03:40:49.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.294+0000 7f4f54979f40 1 --2- 172.21.15.38:0/3704611312 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x5590c0656c30 0x5590c0659050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:49.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.294+0000 7f4f54979f40 1 -- 172.21.15.38:0/3704611312 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:57 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x5590c0659700 con 0x5590c0656c30 2024-09-21T03:40:49.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.295+0000 7f4f4ec36640 1 --2- 172.21.15.38:0/3704611312 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x5590c0656c30 0x5590c0659050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:49.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.295+0000 7f4f4ec36640 1 --2- 172.21.15.38:0/3704611312 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x5590c0656c30 0x5590c0659050 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:49.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.296+0000 7f4f4ec36640 1 -- 172.21.15.38:0/3704611312 <== osd.3 v2:172.21.15.38:6808/1286810294 1 ==== osd_op_reply(57 mds0_openfiles.0 [create] v102'24 uv23 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f4f4800b040 con 0x5590c0656c30 2024-09-21T03:40:49.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.296+0000 7f4f54979f40 1 -- 172.21.15.38:0/3704611312 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:58 13.6 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e102) -- 0x5590c0659c10 con 0x5590c0656c30 2024-09-21T03:40:49.299 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.298+0000 7f4f4ec36640 1 -- 172.21.15.38:0/3704611312 <== osd.3 v2:172.21.15.38:6808/1286810294 2 ==== osd_op_reply(58 mds0_openfiles.0 [delete] v102'25 uv25 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f4f4800b040 con 0x5590c0656c30 2024-09-21T03:40:49.299 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.298+0000 7f4f54979f40 1 -- 172.21.15.38:0/3704611312 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:59 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x5590c065a110 con 0x5590c0656c30 2024-09-21T03:40:49.301 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.299+0000 7f4f4ec36640 1 -- 172.21.15.38:0/3704611312 <== osd.3 v2:172.21.15.38:6808/1286810294 3 ==== osd_op_reply(59 mds0_openfiles.0 [create] v102'26 uv26 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f4f4800b040 con 0x5590c0656c30 2024-09-21T03:40:49.301 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-09-21T03:40:49.301 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.299+0000 7f4f54979f40 1 -- 172.21.15.38:0/3704611312 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:60 13.6 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x5590c065a9d0 con 0x5590c0656c30 2024-09-21T03:40:49.302 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.300+0000 7f4f4ec36640 1 -- 172.21.15.38:0/3704611312 <== osd.3 v2:172.21.15.38:6808/1286810294 4 ==== osd_op_reply(60 mds0_openfiles.0 [omap-set-header] v102'27 uv27 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f4f4800b040 con 0x5590c0656c30 2024-09-21T03:40:49.302 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.300+0000 7f4f54979f40 1 -- 172.21.15.38:0/3704611312 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:61 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x5590c065ad70 con 0x5590c0653760 2024-09-21T03:40:49.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.301+0000 7f4f4dc34640 1 -- 172.21.15.38:0/3704611312 <== osd.0 v2:172.21.15.38:6800/2381639433 7 ==== osd_op_reply(61 608.00000000 [create] v102'33 uv32 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f4f4400f580 con 0x5590c0653760 2024-09-21T03:40:49.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.301+0000 7f4f54979f40 1 -- 172.21.15.38:0/3704611312 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:62 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e102) -- 0x5590c065ad70 con 0x5590c0653760 2024-09-21T03:40:49.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.303+0000 7f4f4dc34640 1 -- 172.21.15.38:0/3704611312 <== osd.0 v2:172.21.15.38:6800/2381639433 8 ==== osd_op_reply(62 608.00000000 [delete] v102'34 uv34 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4f4400f580 con 0x5590c0653760 2024-09-21T03:40:49.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.303+0000 7f4f54979f40 1 -- 172.21.15.38:0/3704611312 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x5590c065ad70 con 0x5590c0653760 2024-09-21T03:40:49.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.304+0000 7f4f4dc34640 1 -- 172.21.15.38:0/3704611312 <== osd.0 v2:172.21.15.38:6800/2381639433 9 ==== osd_op_reply(63 608.00000000 [create] v102'35 uv35 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4f4400f580 con 0x5590c0653760 2024-09-21T03:40:49.306 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-09-21T03:40:49.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.304+0000 7f4f54979f40 1 -- 172.21.15.38:0/3704611312 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x5590c065bd80 con 0x5590c0653760 2024-09-21T03:40:49.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.305+0000 7f4f4dc34640 1 -- 172.21.15.38:0/3704611312 <== osd.0 v2:172.21.15.38:6800/2381639433 10 ==== osd_op_reply(64 608.00000000 [setxattr (30)] v102'36 uv36 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4f4400f580 con 0x5590c0653760 2024-09-21T03:40:49.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.305+0000 7f4f54979f40 1 -- 172.21.15.38:0/3704611312 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x5590c065bd80 con 0x5590c0653760 2024-09-21T03:40:49.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.307+0000 7f4f4dc34640 1 -- 172.21.15.38:0/3704611312 <== osd.0 v2:172.21.15.38:6800/2381639433 11 ==== osd_op_reply(65 608.00000000 [setxattr (62)] v102'37 uv37 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4f4400f580 con 0x5590c0653760 2024-09-21T03:40:49.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.307+0000 7f4f54979f40 1 -- 172.21.15.38:0/3704611312 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x5590c065ad70 con 0x5590c0653760 2024-09-21T03:40:49.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.308+0000 7f4f4dc34640 1 -- 172.21.15.38:0/3704611312 <== osd.0 v2:172.21.15.38:6800/2381639433 12 ==== osd_op_reply(66 608.00000000 [omap-set-header] v102'38 uv38 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4f4400f580 con 0x5590c0653760 2024-09-21T03:40:49.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.308+0000 7f4f54979f40 1 -- 172.21.15.38:0/3704611312 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:67 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x5590c065ad70 con 0x5590c0656c30 2024-09-21T03:40:49.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.310+0000 7f4f4ec36640 1 -- 172.21.15.38:0/3704611312 <== osd.3 v2:172.21.15.38:6808/1286810294 5 ==== osd_op_reply(67 604.00000000 [create] v102'59 uv53 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f4f4800b040 con 0x5590c0656c30 2024-09-21T03:40:49.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.310+0000 7f4f54979f40 1 -- 172.21.15.38:0/3704611312 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:68 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e102) -- 0x5590c065b110 con 0x5590c0656c30 2024-09-21T03:40:49.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.311+0000 7f4f4ec36640 1 -- 172.21.15.38:0/3704611312 <== osd.3 v2:172.21.15.38:6808/1286810294 6 ==== osd_op_reply(68 604.00000000 [delete] v102'60 uv60 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4f4800b040 con 0x5590c0656c30 2024-09-21T03:40:49.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.311+0000 7f4f54979f40 1 -- 172.21.15.38:0/3704611312 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x5590c065b4b0 con 0x5590c0656c30 2024-09-21T03:40:49.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.312+0000 7f4f4ec36640 1 -- 172.21.15.38:0/3704611312 <== osd.3 v2:172.21.15.38:6808/1286810294 7 ==== osd_op_reply(69 604.00000000 [create] v102'61 uv61 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4f4800b040 con 0x5590c0656c30 2024-09-21T03:40:49.314 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-09-21T03:40:49.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.312+0000 7f4f54979f40 1 -- 172.21.15.38:0/3704611312 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x5590c065c4c0 con 0x5590c0656c30 2024-09-21T03:40:49.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.313+0000 7f4f4ec36640 1 -- 172.21.15.38:0/3704611312 <== osd.3 v2:172.21.15.38:6808/1286810294 8 ==== osd_op_reply(70 604.00000000 [setxattr (30)] v102'62 uv62 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4f4800b040 con 0x5590c0656c30 2024-09-21T03:40:49.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.313+0000 7f4f54979f40 1 -- 172.21.15.38:0/3704611312 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x5590c065c4c0 con 0x5590c0656c30 2024-09-21T03:40:49.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.314+0000 7f4f4ec36640 1 -- 172.21.15.38:0/3704611312 <== osd.3 v2:172.21.15.38:6808/1286810294 9 ==== osd_op_reply(71 604.00000000 [setxattr (62)] v102'63 uv63 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4f4800b040 con 0x5590c0656c30 2024-09-21T03:40:49.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.314+0000 7f4f54979f40 1 -- 172.21.15.38:0/3704611312 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x5590c065b4b0 con 0x5590c0656c30 2024-09-21T03:40:49.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.315+0000 7f4f4ec36640 1 -- 172.21.15.38:0/3704611312 <== osd.3 v2:172.21.15.38:6808/1286810294 10 ==== osd_op_reply(72 604.00000000 [omap-set-header] v102'64 uv64 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4f4800b040 con 0x5590c0656c30 2024-09-21T03:40:49.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.315+0000 7f4f54979f40 1 -- 172.21.15.38:0/3704611312 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:73 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x5590c065b4b0 con 0x5590c0656c30 2024-09-21T03:40:49.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.316+0000 7f4f4ec36640 1 -- 172.21.15.38:0/3704611312 <== osd.3 v2:172.21.15.38:6808/1286810294 11 ==== osd_op_reply(73 500.00000000 [create] v102'65 uv58 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f4f4800b040 con 0x5590c0656c30 2024-09-21T03:40:49.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.316+0000 7f4f54979f40 1 -- 172.21.15.38:0/3704611312 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:74 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e102) -- 0x5590c065b4b0 con 0x5590c0656c30 2024-09-21T03:40:49.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.317+0000 7f4f4ec36640 1 -- 172.21.15.38:0/3704611312 <== osd.3 v2:172.21.15.38:6808/1286810294 12 ==== osd_op_reply(74 500.00000000 [delete] v102'66 uv66 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4f4800b040 con 0x5590c0656c30 2024-09-21T03:40:49.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.317+0000 7f4f54979f40 1 -- 172.21.15.38:0/3704611312 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x5590c065b4b0 con 0x5590c0656c30 2024-09-21T03:40:49.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.318+0000 7f4f4ec36640 1 -- 172.21.15.38:0/3704611312 <== osd.3 v2:172.21.15.38:6808/1286810294 13 ==== osd_op_reply(75 500.00000000 [create] v102'67 uv67 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4f4800b040 con 0x5590c0656c30 2024-09-21T03:40:49.320 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-09-21T03:40:49.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.318+0000 7f4f54979f40 1 -- 172.21.15.38:0/3704611312 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x5590c065b4b0 con 0x5590c0656c30 2024-09-21T03:40:49.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.320+0000 7f4f4ec36640 1 -- 172.21.15.38:0/3704611312 <== osd.3 v2:172.21.15.38:6808/1286810294 14 ==== osd_op_reply(76 500.00000000 [write 0~90] v102'68 uv68 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4f4800b040 con 0x5590c0656c30 2024-09-21T03:40:49.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.320+0000 7f4f54979f40 1 -- 172.21.15.38:0/3704611312 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x5590c065b4b0 con 0x5590c0656c30 2024-09-21T03:40:49.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.321+0000 7f4f4ec36640 1 -- 172.21.15.38:0/3704611312 <== osd.3 v2:172.21.15.38:6808/1286810294 15 ==== osd_op_reply(77 500.00000000 [omap-set-header] v102'69 uv69 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4f4800b040 con 0x5590c0656c30 2024-09-21T03:40:49.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.321+0000 7f4f54979f40 1 -- 172.21.15.38:0/3704611312 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:78 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x5590c065b4b0 con 0x5590c0653760 2024-09-21T03:40:49.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.322+0000 7f4f4dc34640 1 -- 172.21.15.38:0/3704611312 <== osd.0 v2:172.21.15.38:6800/2381639433 13 ==== osd_op_reply(78 mds_snaptable [create] v102'27 uv26 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f4f4400f580 con 0x5590c0653760 2024-09-21T03:40:49.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.322+0000 7f4f54979f40 1 -- 172.21.15.38:0/3704611312 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:79 13.d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e102) -- 0x5590c065b4b0 con 0x5590c0653760 2024-09-21T03:40:49.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.323+0000 7f4f4dc34640 1 -- 172.21.15.38:0/3704611312 <== osd.0 v2:172.21.15.38:6800/2381639433 14 ==== osd_op_reply(79 mds_snaptable [delete] v102'28 uv28 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f4f4400f580 con 0x5590c0653760 2024-09-21T03:40:49.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.323+0000 7f4f54979f40 1 -- 172.21.15.38:0/3704611312 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:80 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x5590c065b4b0 con 0x5590c0653760 2024-09-21T03:40:49.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.324+0000 7f4f4dc34640 1 -- 172.21.15.38:0/3704611312 <== osd.0 v2:172.21.15.38:6800/2381639433 15 ==== osd_op_reply(80 mds_snaptable [create] v102'29 uv29 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f4f4400f580 con 0x5590c0653760 2024-09-21T03:40:49.326 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-09-21T03:40:49.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.324+0000 7f4f54979f40 1 -- 172.21.15.38:0/3704611312 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:81 13.d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x5590c065b4b0 con 0x5590c0653760 2024-09-21T03:40:49.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.325+0000 7f4f4dc34640 1 -- 172.21.15.38:0/3704611312 <== osd.0 v2:172.21.15.38:6800/2381639433 16 ==== osd_op_reply(81 mds_snaptable [write 0~70] v102'30 uv30 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f4f4400f580 con 0x5590c0653760 2024-09-21T03:40:49.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.325+0000 7f4f54979f40 1 -- 172.21.15.38:0/3704611312 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:82 13.d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x5590c065b4b0 con 0x5590c0653760 2024-09-21T03:40:49.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.326+0000 7f4f4dc34640 1 -- 172.21.15.38:0/3704611312 <== osd.0 v2:172.21.15.38:6800/2381639433 17 ==== osd_op_reply(82 mds_snaptable [omap-set-header] v102'31 uv31 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f4f4400f580 con 0x5590c0653760 2024-09-21T03:40:49.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.326+0000 7f4f54979f40 1 --2- 172.21.15.38:0/3704611312 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x5590c065bb30 0x5590c065df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:49.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.326+0000 7f4f54979f40 1 -- 172.21.15.38:0/3704611312 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:83 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x5590c065e560 con 0x5590c065bb30 2024-09-21T03:40:49.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.326+0000 7f4f4e435640 1 --2- 172.21.15.38:0/3704611312 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x5590c065bb30 0x5590c065df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:49.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.327+0000 7f4f4e435640 1 --2- 172.21.15.38:0/3704611312 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x5590c065bb30 0x5590c065df50 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:49.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.328+0000 7f4f4e435640 1 -- 172.21.15.38:0/3704611312 <== osd.1 v2:172.21.15.38:6816/1491818732 1 ==== osd_op_reply(83 605.00000000 [create] v102'33 uv32 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f4f4000e040 con 0x5590c065bb30 2024-09-21T03:40:49.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.328+0000 7f4f54979f40 1 -- 172.21.15.38:0/3704611312 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:84 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e102) -- 0x5590c065e930 con 0x5590c065bb30 2024-09-21T03:40:49.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.329+0000 7f4f4e435640 1 -- 172.21.15.38:0/3704611312 <== osd.1 v2:172.21.15.38:6816/1491818732 2 ==== osd_op_reply(84 605.00000000 [delete] v102'34 uv34 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4f4000e040 con 0x5590c065bb30 2024-09-21T03:40:49.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.329+0000 7f4f54979f40 1 -- 172.21.15.38:0/3704611312 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x5590c065e930 con 0x5590c065bb30 2024-09-21T03:40:49.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.330+0000 7f4f4e435640 1 -- 172.21.15.38:0/3704611312 <== osd.1 v2:172.21.15.38:6816/1491818732 3 ==== osd_op_reply(85 605.00000000 [create] v102'35 uv35 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4f4000e040 con 0x5590c065bb30 2024-09-21T03:40:49.332 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-09-21T03:40:49.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.330+0000 7f4f54979f40 1 -- 172.21.15.38:0/3704611312 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x5590c065f940 con 0x5590c065bb30 2024-09-21T03:40:49.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.331+0000 7f4f4e435640 1 -- 172.21.15.38:0/3704611312 <== osd.1 v2:172.21.15.38:6816/1491818732 4 ==== osd_op_reply(86 605.00000000 [setxattr (30)] v102'36 uv36 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4f4000e040 con 0x5590c065bb30 2024-09-21T03:40:49.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.331+0000 7f4f54979f40 1 -- 172.21.15.38:0/3704611312 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x5590c065f940 con 0x5590c065bb30 2024-09-21T03:40:49.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.332+0000 7f4f4e435640 1 -- 172.21.15.38:0/3704611312 <== osd.1 v2:172.21.15.38:6816/1491818732 5 ==== osd_op_reply(87 605.00000000 [setxattr (62)] v102'37 uv37 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4f4000e040 con 0x5590c065bb30 2024-09-21T03:40:49.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.332+0000 7f4f54979f40 1 -- 172.21.15.38:0/3704611312 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x5590c065e930 con 0x5590c065bb30 2024-09-21T03:40:49.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.333+0000 7f4f4e435640 1 -- 172.21.15.38:0/3704611312 <== osd.1 v2:172.21.15.38:6816/1491818732 6 ==== osd_op_reply(88 605.00000000 [omap-set-header] v102'38 uv38 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4f4000e040 con 0x5590c065bb30 2024-09-21T03:40:49.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.333+0000 7f4f54979f40 1 -- 172.21.15.38:0/3704611312 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:89 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x5590c065e930 con 0x5590c065bb30 2024-09-21T03:40:49.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.334+0000 7f4f4e435640 1 -- 172.21.15.38:0/3704611312 <== osd.1 v2:172.21.15.38:6816/1491818732 7 ==== osd_op_reply(89 mds0_inotable [create] v102'54 uv48 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f4f4000e040 con 0x5590c065bb30 2024-09-21T03:40:49.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.334+0000 7f4f54979f40 1 -- 172.21.15.38:0/3704611312 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:90 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e102) -- 0x5590c065e930 con 0x5590c065bb30 2024-09-21T03:40:49.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.335+0000 7f4f4e435640 1 -- 172.21.15.38:0/3704611312 <== osd.1 v2:172.21.15.38:6816/1491818732 8 ==== osd_op_reply(90 mds0_inotable [delete] v102'55 uv55 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f4f4000e040 con 0x5590c065bb30 2024-09-21T03:40:49.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.335+0000 7f4f54979f40 1 -- 172.21.15.38:0/3704611312 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x5590c065e930 con 0x5590c065bb30 2024-09-21T03:40:49.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.336+0000 7f4f4e435640 1 -- 172.21.15.38:0/3704611312 <== osd.1 v2:172.21.15.38:6816/1491818732 9 ==== osd_op_reply(91 mds0_inotable [create] v102'56 uv56 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f4f4000e040 con 0x5590c065bb30 2024-09-21T03:40:49.338 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-09-21T03:40:49.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.336+0000 7f4f54979f40 1 -- 172.21.15.38:0/3704611312 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x5590c065a5e0 con 0x5590c065bb30 2024-09-21T03:40:49.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.338+0000 7f4f4e435640 1 -- 172.21.15.38:0/3704611312 <== osd.1 v2:172.21.15.38:6816/1491818732 10 ==== osd_op_reply(92 mds0_inotable [write 0~34] v102'57 uv57 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f4f4000e040 con 0x5590c065bb30 2024-09-21T03:40:49.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.338+0000 7f4f54979f40 1 -- 172.21.15.38:0/3704611312 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x5590c065a5e0 con 0x5590c065bb30 2024-09-21T03:40:49.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.339+0000 7f4f4e435640 1 -- 172.21.15.38:0/3704611312 <== osd.1 v2:172.21.15.38:6816/1491818732 11 ==== osd_op_reply(93 mds0_inotable [omap-set-header] v102'58 uv58 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f4f4000e040 con 0x5590c065bb30 2024-09-21T03:40:49.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.339+0000 7f4f54979f40 1 -- 172.21.15.38:0/3704611312 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:94 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x5590c065a5c0 con 0x5590c065bb30 2024-09-21T03:40:49.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.340+0000 7f4f4e435640 1 -- 172.21.15.38:0/3704611312 <== osd.1 v2:172.21.15.38:6816/1491818732 12 ==== osd_op_reply(94 100.00000000 [create] v102'59 uv53 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f4f4000e040 con 0x5590c065bb30 2024-09-21T03:40:49.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.340+0000 7f4f54979f40 1 -- 172.21.15.38:0/3704611312 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:95 13.13 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e102) -- 0x5590c065a5c0 con 0x5590c065bb30 2024-09-21T03:40:49.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.341+0000 7f4f4e435640 1 -- 172.21.15.38:0/3704611312 <== osd.1 v2:172.21.15.38:6816/1491818732 13 ==== osd_op_reply(95 100.00000000 [delete] v102'60 uv60 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4f4000e040 con 0x5590c065bb30 2024-09-21T03:40:49.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.341+0000 7f4f54979f40 1 -- 172.21.15.38:0/3704611312 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:96 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x5590c065a5c0 con 0x5590c065bb30 2024-09-21T03:40:49.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.342+0000 7f4f4e435640 1 -- 172.21.15.38:0/3704611312 <== osd.1 v2:172.21.15.38:6816/1491818732 14 ==== osd_op_reply(96 100.00000000 [create] v102'61 uv61 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4f4000e040 con 0x5590c065bb30 2024-09-21T03:40:49.344 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-09-21T03:40:49.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.342+0000 7f4f54979f40 1 -- 172.21.15.38:0/3704611312 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:97 13.13 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x5590c065a5c0 con 0x5590c065bb30 2024-09-21T03:40:49.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.343+0000 7f4f4e435640 1 -- 172.21.15.38:0/3704611312 <== osd.1 v2:172.21.15.38:6816/1491818732 15 ==== osd_op_reply(97 100.00000000 [omap-set-header] v102'62 uv62 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4f4000e040 con 0x5590c065bb30 2024-09-21T03:40:49.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.343+0000 7f4f54979f40 1 -- 172.21.15.38:0/3704611312 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:98 13.13 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x5590c06617f0 con 0x5590c065bb30 2024-09-21T03:40:49.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.344+0000 7f4f4e435640 1 -- 172.21.15.38:0/3704611312 <== osd.1 v2:172.21.15.38:6816/1491818732 16 ==== osd_op_reply(98 100.00000000 [omap-set-vals] v102'63 uv63 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4f4000e040 con 0x5590c065bb30 2024-09-21T03:40:49.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.344+0000 7f4f54979f40 1 -- 172.21.15.38:0/3704611312 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:99 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x5590c065e930 con 0x5590c0656c30 2024-09-21T03:40:49.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.345+0000 7f4f4ec36640 1 -- 172.21.15.38:0/3704611312 <== osd.3 v2:172.21.15.38:6808/1286810294 16 ==== osd_op_reply(99 mds0_sessionmap [create] v102'26 uv25 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7f4f4800b040 con 0x5590c0656c30 2024-09-21T03:40:49.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.345+0000 7f4f54979f40 1 -- 172.21.15.38:0/3704611312 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:100 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e102) -- 0x5590c065e930 con 0x5590c0656c30 2024-09-21T03:40:49.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.346+0000 7f4f4ec36640 1 -- 172.21.15.38:0/3704611312 <== osd.3 v2:172.21.15.38:6808/1286810294 17 ==== osd_op_reply(100 mds0_sessionmap [delete] v102'27 uv27 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f4f4800b040 con 0x5590c0656c30 2024-09-21T03:40:49.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.346+0000 7f4f54979f40 1 -- 172.21.15.38:0/3704611312 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x5590c065e930 con 0x5590c0656c30 2024-09-21T03:40:49.350 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-09-21T03:40:49.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.347+0000 7f4f4ec36640 1 -- 172.21.15.38:0/3704611312 <== osd.3 v2:172.21.15.38:6808/1286810294 18 ==== osd_op_reply(101 mds0_sessionmap [create] v102'28 uv28 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f4f4800b040 con 0x5590c0656c30 2024-09-21T03:40:49.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.347+0000 7f4f54979f40 1 -- 172.21.15.38:0/3704611312 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x5590c065e930 con 0x5590c0656c30 2024-09-21T03:40:49.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.348+0000 7f4f4ec36640 1 -- 172.21.15.38:0/3704611312 <== osd.3 v2:172.21.15.38:6808/1286810294 19 ==== osd_op_reply(102 mds0_sessionmap [omap-set-header] v102'29 uv29 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f4f4800b040 con 0x5590c0656c30 2024-09-21T03:40:49.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.348+0000 7f4f54979f40 1 -- 172.21.15.38:0/3704611312 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:103 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x5590c065ede0 con 0x5590c0656c30 2024-09-21T03:40:49.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.349+0000 7f4f4ec36640 1 -- 172.21.15.38:0/3704611312 <== osd.3 v2:172.21.15.38:6808/1286810294 20 ==== osd_op_reply(103 609.00000000 [create] v102'33 uv32 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f4f4800b040 con 0x5590c0656c30 2024-09-21T03:40:49.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.349+0000 7f4f54979f40 1 -- 172.21.15.38:0/3704611312 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:104 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e102) -- 0x5590c065ede0 con 0x5590c0656c30 2024-09-21T03:40:49.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.350+0000 7f4f4ec36640 1 -- 172.21.15.38:0/3704611312 <== osd.3 v2:172.21.15.38:6808/1286810294 21 ==== osd_op_reply(104 609.00000000 [delete] v102'34 uv34 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4f4800b040 con 0x5590c0656c30 2024-09-21T03:40:49.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.350+0000 7f4f54979f40 1 -- 172.21.15.38:0/3704611312 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x5590c065ede0 con 0x5590c0656c30 2024-09-21T03:40:49.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.351+0000 7f4f4ec36640 1 -- 172.21.15.38:0/3704611312 <== osd.3 v2:172.21.15.38:6808/1286810294 22 ==== osd_op_reply(105 609.00000000 [create] v102'35 uv35 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4f4800b040 con 0x5590c0656c30 2024-09-21T03:40:49.353 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-09-21T03:40:49.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.351+0000 7f4f54979f40 1 -- 172.21.15.38:0/3704611312 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x5590c065fdf0 con 0x5590c0656c30 2024-09-21T03:40:49.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.353+0000 7f4f4ec36640 1 -- 172.21.15.38:0/3704611312 <== osd.3 v2:172.21.15.38:6808/1286810294 23 ==== osd_op_reply(106 609.00000000 [setxattr (30)] v102'36 uv36 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4f4800b040 con 0x5590c0656c30 2024-09-21T03:40:49.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.353+0000 7f4f54979f40 1 -- 172.21.15.38:0/3704611312 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x5590c065fdf0 con 0x5590c0656c30 2024-09-21T03:40:49.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.354+0000 7f4f4ec36640 1 -- 172.21.15.38:0/3704611312 <== osd.3 v2:172.21.15.38:6808/1286810294 24 ==== osd_op_reply(107 609.00000000 [setxattr (62)] v102'37 uv37 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4f4800b040 con 0x5590c0656c30 2024-09-21T03:40:49.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.354+0000 7f4f54979f40 1 -- 172.21.15.38:0/3704611312 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x5590c065ede0 con 0x5590c0656c30 2024-09-21T03:40:49.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.355+0000 7f4f4ec36640 1 -- 172.21.15.38:0/3704611312 <== osd.3 v2:172.21.15.38:6808/1286810294 25 ==== osd_op_reply(108 609.00000000 [omap-set-header] v102'38 uv38 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4f4800b040 con 0x5590c0656c30 2024-09-21T03:40:49.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.355+0000 7f4f54979f40 1 --2- 172.21.15.38:0/3704611312 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x5590c065f460 0x5590c0661880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:49.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.355+0000 7f4f54979f40 1 -- 172.21.15.38:0/3704611312 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:109 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x5590c0661dc0 con 0x5590c065f460 2024-09-21T03:40:49.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.355+0000 7f4f4dc34640 1 --2- 172.21.15.38:0/3704611312 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x5590c065f460 0x5590c0661880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:49.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.355+0000 7f4f4dc34640 1 --2- 172.21.15.38:0/3704611312 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x5590c065f460 0x5590c0661880 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:49.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.356+0000 7f4f4dc34640 1 -- 172.21.15.38:0/3704611312 <== osd.2 v2:172.21.15.38:6824/2577858902 1 ==== osd_op_reply(109 400.00000000 [create] v102'27 uv26 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f4f4400f580 con 0x5590c065f460 2024-09-21T03:40:49.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.356+0000 7f4f54979f40 1 -- 172.21.15.38:0/3704611312 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:110 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e102) -- 0x5590c0661dc0 con 0x5590c065f460 2024-09-21T03:40:49.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.358+0000 7f4f4dc34640 1 -- 172.21.15.38:0/3704611312 <== osd.2 v2:172.21.15.38:6824/2577858902 2 ==== osd_op_reply(110 400.00000000 [delete] v102'28 uv28 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4f4400f580 con 0x5590c065f460 2024-09-21T03:40:49.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.358+0000 7f4f54979f40 1 -- 172.21.15.38:0/3704611312 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x5590c0661e40 con 0x5590c065f460 2024-09-21T03:40:49.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.359+0000 7f4f4dc34640 1 -- 172.21.15.38:0/3704611312 <== osd.2 v2:172.21.15.38:6824/2577858902 3 ==== osd_op_reply(111 400.00000000 [create] v102'29 uv29 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4f4400f580 con 0x5590c065f460 2024-09-21T03:40:49.360 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-09-21T03:40:49.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.359+0000 7f4f54979f40 1 -- 172.21.15.38:0/3704611312 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x5590c0662230 con 0x5590c065f460 2024-09-21T03:40:49.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.360+0000 7f4f4dc34640 1 -- 172.21.15.38:0/3704611312 <== osd.2 v2:172.21.15.38:6824/2577858902 4 ==== osd_op_reply(112 400.00000000 [write 0~22] v102'30 uv30 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4f4400f580 con 0x5590c065f460 2024-09-21T03:40:49.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.360+0000 7f4f54979f40 1 -- 172.21.15.38:0/3704611312 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x5590c0662230 con 0x5590c065f460 2024-09-21T03:40:49.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.361+0000 7f4f4dc34640 1 -- 172.21.15.38:0/3704611312 <== osd.2 v2:172.21.15.38:6824/2577858902 5 ==== osd_op_reply(113 400.00000000 [omap-set-header] v102'31 uv31 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4f4400f580 con 0x5590c065f460 2024-09-21T03:40:49.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.361+0000 7f4f54979f40 1 -- 172.21.15.38:0/3704611312 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:114 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x5590c0662230 con 0x5590c065bb30 2024-09-21T03:40:49.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.362+0000 7f4f4e435640 1 -- 172.21.15.38:0/3704611312 <== osd.1 v2:172.21.15.38:6816/1491818732 17 ==== osd_op_reply(114 100.00000000.inode [create] v102'69 uv61 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7f4f4000e040 con 0x5590c065bb30 2024-09-21T03:40:49.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.362+0000 7f4f54979f40 1 -- 172.21.15.38:0/3704611312 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:115 13.1f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e102) -- 0x5590c0662230 con 0x5590c065bb30 2024-09-21T03:40:49.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.363+0000 7f4f4e435640 1 -- 172.21.15.38:0/3704611312 <== osd.1 v2:172.21.15.38:6816/1491818732 18 ==== osd_op_reply(115 100.00000000.inode [delete] v102'70 uv70 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f4f4000e040 con 0x5590c065bb30 2024-09-21T03:40:49.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.363+0000 7f4f54979f40 1 -- 172.21.15.38:0/3704611312 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:116 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x5590c0662230 con 0x5590c065bb30 2024-09-21T03:40:49.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.364+0000 7f4f4e435640 1 -- 172.21.15.38:0/3704611312 <== osd.1 v2:172.21.15.38:6816/1491818732 19 ==== osd_op_reply(116 100.00000000.inode [create] v102'71 uv71 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f4f4000e040 con 0x5590c065bb30 2024-09-21T03:40:49.366 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-09-21T03:40:49.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.364+0000 7f4f54979f40 1 -- 172.21.15.38:0/3704611312 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:117 13.1f 13:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x5590c0662230 con 0x5590c065bb30 2024-09-21T03:40:49.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.365+0000 7f4f4e435640 1 -- 172.21.15.38:0/3704611312 <== osd.1 v2:172.21.15.38:6816/1491818732 20 ==== osd_op_reply(117 100.00000000.inode [write 0~582] v102'72 uv72 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f4f4000e040 con 0x5590c065bb30 2024-09-21T03:40:49.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.365+0000 7f4f54979f40 1 -- 172.21.15.38:0/3704611312 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:118 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x5590c0662230 con 0x5590c065bb30 2024-09-21T03:40:49.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.366+0000 7f4f4e435640 1 -- 172.21.15.38:0/3704611312 <== osd.1 v2:172.21.15.38:6816/1491818732 21 ==== osd_op_reply(118 100.00000000.inode [omap-set-header] v102'73 uv73 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f4f4000e040 con 0x5590c065bb30 2024-09-21T03:40:49.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.366+0000 7f4f54979f40 1 -- 172.21.15.38:0/3704611312 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:119 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x5590c0662230 con 0x5590c065bb30 2024-09-21T03:40:49.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.367+0000 7f4f4e435640 1 -- 172.21.15.38:0/3704611312 <== osd.1 v2:172.21.15.38:6816/1491818732 22 ==== osd_op_reply(119 1.00000000 [create] v102'74 uv68 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7f4f4000e040 con 0x5590c065bb30 2024-09-21T03:40:49.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.367+0000 7f4f54979f40 1 -- 172.21.15.38:0/3704611312 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:120 13.1f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e102) -- 0x5590c0662230 con 0x5590c065bb30 2024-09-21T03:40:49.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.368+0000 7f4f4e435640 1 -- 172.21.15.38:0/3704611312 <== osd.1 v2:172.21.15.38:6816/1491818732 23 ==== osd_op_reply(120 1.00000000 [delete] v102'75 uv75 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f4f4000e040 con 0x5590c065bb30 2024-09-21T03:40:49.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.368+0000 7f4f54979f40 1 -- 172.21.15.38:0/3704611312 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:121 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x5590c0662230 con 0x5590c065bb30 2024-09-21T03:40:49.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.369+0000 7f4f4e435640 1 -- 172.21.15.38:0/3704611312 <== osd.1 v2:172.21.15.38:6816/1491818732 24 ==== osd_op_reply(121 1.00000000 [create] v102'76 uv76 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f4f4000e040 con 0x5590c065bb30 2024-09-21T03:40:49.371 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-09-21T03:40:49.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.369+0000 7f4f54979f40 1 -- 172.21.15.38:0/3704611312 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:122 13.1f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x5590c0663240 con 0x5590c065bb30 2024-09-21T03:40:49.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.370+0000 7f4f4e435640 1 -- 172.21.15.38:0/3704611312 <== osd.1 v2:172.21.15.38:6816/1491818732 25 ==== osd_op_reply(122 1.00000000 [setxattr (30)] v102'77 uv77 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f4f4000e040 con 0x5590c065bb30 2024-09-21T03:40:49.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.370+0000 7f4f54979f40 1 -- 172.21.15.38:0/3704611312 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:123 13.1f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x5590c0663240 con 0x5590c065bb30 2024-09-21T03:40:49.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.371+0000 7f4f4e435640 1 -- 172.21.15.38:0/3704611312 <== osd.1 v2:172.21.15.38:6816/1491818732 26 ==== osd_op_reply(123 1.00000000 [setxattr (30)] v102'78 uv78 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f4f4000e040 con 0x5590c065bb30 2024-09-21T03:40:49.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.371+0000 7f4f54979f40 1 -- 172.21.15.38:0/3704611312 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:124 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x5590c0662230 con 0x5590c065bb30 2024-09-21T03:40:49.374 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.372+0000 7f4f4e435640 1 -- 172.21.15.38:0/3704611312 <== osd.1 v2:172.21.15.38:6816/1491818732 27 ==== osd_op_reply(124 1.00000000 [omap-set-header] v102'79 uv79 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f4f4000e040 con 0x5590c065bb30 2024-09-21T03:40:49.374 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.372+0000 7f4f54979f40 1 -- 172.21.15.38:0/3704611312 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:125 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=584b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x5590c06626e0 con 0x5590c065bb30 2024-09-21T03:40:49.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.373+0000 7f4f4e435640 1 -- 172.21.15.38:0/3704611312 <== osd.1 v2:172.21.15.38:6816/1491818732 28 ==== osd_op_reply(125 1.00000000 [omap-set-vals] v102'80 uv80 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f4f4000e040 con 0x5590c065bb30 2024-09-21T03:40:49.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.373+0000 7f4f54979f40 1 -- 172.21.15.38:0/3704611312 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x5590c0653760 msgr2=0x5590c0655b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:49.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.373+0000 7f4f54979f40 1 --2- 172.21.15.38:0/3704611312 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x5590c0653760 0x5590c0655b80 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:49.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.373+0000 7f4f54979f40 1 -- 172.21.15.38:0/3704611312 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x5590c065bb30 msgr2=0x5590c065df50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:49.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.373+0000 7f4f54979f40 1 --2- 172.21.15.38:0/3704611312 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x5590c065bb30 0x5590c065df50 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:49.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.373+0000 7f4f54979f40 1 -- 172.21.15.38:0/3704611312 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x5590c065f460 msgr2=0x5590c0661880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:49.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.373+0000 7f4f54979f40 1 --2- 172.21.15.38:0/3704611312 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x5590c065f460 0x5590c0661880 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:49.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.373+0000 7f4f54979f40 1 -- 172.21.15.38:0/3704611312 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x5590c0656c30 msgr2=0x5590c0659050 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:49.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.373+0000 7f4f54979f40 1 --2- 172.21.15.38:0/3704611312 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x5590c0656c30 0x5590c0659050 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:49.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.373+0000 7f4f4ec36640 1 -- 172.21.15.38:0/3704611312 reap_dead start 2024-09-21T03:40:49.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.373+0000 7f4f54979f40 1 -- 172.21.15.38:0/3704611312 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x5590c064ded0 msgr2=0x5590c0650fb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:49.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.373+0000 7f4f54979f40 1 --2- 172.21.15.38:0/3704611312 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x5590c064ded0 0x5590c0650fb0 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:49.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.373+0000 7f4f4ec36640 1 -- 172.21.15.38:0/3704611312 reap_dead start 2024-09-21T03:40:49.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.373+0000 7f4f54979f40 1 -- 172.21.15.38:0/3704611312 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x5590c04fcd20 msgr2=0x5590c04d0330 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:49.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.373+0000 7f4f54979f40 1 --2- 172.21.15.38:0/3704611312 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x5590c04fcd20 0x5590c04d0330 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:49.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.373+0000 7f4f54979f40 1 -- 172.21.15.38:0/3704611312 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4f300775a0 msgr2=0x7f4f30079ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:49.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.373+0000 7f4f54979f40 1 --2- 172.21.15.38:0/3704611312 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4f300775a0 0x7f4f30079ab0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f4f40000c00 tx=0x7f4f40003070 comp rx=0 tx=0).stop 2024-09-21T03:40:49.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.373+0000 7f4f54979f40 1 -- 172.21.15.38:0/3704611312 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5590c058e950 msgr2=0x5590c05018c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:49.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.373+0000 7f4f54979f40 1 --2- 172.21.15.38:0/3704611312 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5590c058e950 0x5590c05018c0 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f4f48025fd0 tx=0x7f4f48055be0 comp rx=0 tx=0).stop 2024-09-21T03:40:49.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.373+0000 7f4f54979f40 1 -- 172.21.15.38:0/3704611312 shutdown_connections 2024-09-21T03:40:49.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.373+0000 7f4f54979f40 1 --2- 172.21.15.38:0/3704611312 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x5590c04fcd20 0x5590c04d0330 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:49.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.373+0000 7f4f54979f40 1 --2- 172.21.15.38:0/3704611312 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5590c058e950 0x5590c05018c0 unknown :-1 s=CLOSED pgs=1350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:49.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.373+0000 7f4f54979f40 1 --2- 172.21.15.38:0/3704611312 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4f300775a0 0x7f4f30079ab0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:49.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.373+0000 7f4f54979f40 1 -- 172.21.15.38:0/3704611312 >> 172.21.15.38:0/3704611312 conn(0x5590c03f2ac0 msgr2=0x5590c058ae90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:49.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.373+0000 7f4f54979f40 1 -- 172.21.15.38:0/3704611312 shutdown_connections 2024-09-21T03:40:49.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.373+0000 7f4f54979f40 1 -- 172.21.15.38:0/3704611312 wait complete. 2024-09-21T03:40:49.379 DEBUG:teuthology.orchestra.run.smithi038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rm mds0_openfiles.0 2024-09-21T03:40:49.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.474+0000 7fc35ab1ef40 1 Processor -- start 2024-09-21T03:40:49.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.474+0000 7fc35ab1ef40 1 -- start start 2024-09-21T03:40:49.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.475+0000 7fc35ab1ef40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc33c002500 0x5572b101e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:49.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.475+0000 7fc35ab1ef40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5572b1153640 0x5572b10b0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:49.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.475+0000 7fc35ab1ef40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5572b1153a50 0x5572b10b0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:49.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.475+0000 7fc35ab1ef40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5572b0fa2430 con 0x7fc33c002500 2024-09-21T03:40:49.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.475+0000 7fc35ab1ef40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5572b0faf1f0 con 0x5572b1153640 2024-09-21T03:40:49.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.475+0000 7fc35ab1ef40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x5572b0f43d60 con 0x5572b1153a50 2024-09-21T03:40:49.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.475+0000 7fc353cc0640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc33c002500 0x5572b101e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:49.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.475+0000 7fc353cc0640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc33c002500 0x5572b101e910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:59020/0 (socket says 172.21.15.38:59020) 2024-09-21T03:40:49.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.475+0000 7fc353cc0640 1 -- 172.21.15.38:0/2424191489 learned_addr learned my addr 172.21.15.38:0/2424191489 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:49.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.475+0000 7fc3534bf640 1 --2- 172.21.15.38:0/2424191489 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5572b1153640 0x5572b10b0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:49.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.475+0000 7fc3544c1640 1 --2- 172.21.15.38:0/2424191489 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5572b1153a50 0x5572b10b0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:49.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.476+0000 7fc353cc0640 1 -- 172.21.15.38:0/2424191489 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5572b1153a50 msgr2=0x5572b10b0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:49.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.476+0000 7fc353cc0640 1 --2- 172.21.15.38:0/2424191489 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5572b1153a50 0x5572b10b0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:49.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.476+0000 7fc353cc0640 1 -- 172.21.15.38:0/2424191489 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5572b1153640 msgr2=0x5572b10b0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:49.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.476+0000 7fc353cc0640 1 --2- 172.21.15.38:0/2424191489 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5572b1153640 0x5572b10b0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:49.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.476+0000 7fc353cc0640 1 -- 172.21.15.38:0/2424191489 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5572b101eee0 con 0x7fc33c002500 2024-09-21T03:40:49.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.476+0000 7fc3534bf640 1 --2- 172.21.15.38:0/2424191489 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5572b1153640 0x5572b10b0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:40:49.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.476+0000 7fc353cc0640 1 --2- 172.21.15.38:0/2424191489 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc33c002500 0x5572b101e910 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7fc3440266d0 tx=0x7fc3440033a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c096ee7ae7acf4f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:49.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.476+0000 7fc3524bd640 1 -- 172.21.15.38:0/2424191489 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc344064030 con 0x7fc33c002500 2024-09-21T03:40:49.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.476+0000 7fc3524bd640 1 -- 172.21.15.38:0/2424191489 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc3440557e0 con 0x7fc33c002500 2024-09-21T03:40:49.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.476+0000 7fc3524bd640 1 -- 172.21.15.38:0/2424191489 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc344055c30 con 0x7fc33c002500 2024-09-21T03:40:49.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.477+0000 7fc35ab1ef40 1 -- 172.21.15.38:0/2424191489 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc33c002500 msgr2=0x5572b101e910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:49.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.477+0000 7fc35ab1ef40 1 --2- 172.21.15.38:0/2424191489 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc33c002500 0x5572b101e910 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7fc3440266d0 tx=0x7fc3440033a0 comp rx=0 tx=0).stop 2024-09-21T03:40:49.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.477+0000 7fc35ab1ef40 1 -- 172.21.15.38:0/2424191489 shutdown_connections 2024-09-21T03:40:49.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.477+0000 7fc35ab1ef40 1 --2- 172.21.15.38:0/2424191489 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5572b1153a50 0x5572b10b0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:49.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.477+0000 7fc35ab1ef40 1 --2- 172.21.15.38:0/2424191489 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5572b1153640 0x5572b10b0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:49.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.477+0000 7fc35ab1ef40 1 --2- 172.21.15.38:0/2424191489 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc33c002500 0x5572b101e910 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:49.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.477+0000 7fc35ab1ef40 1 -- 172.21.15.38:0/2424191489 >> 172.21.15.38:0/2424191489 conn(0x5572b0f14ac0 msgr2=0x5572b0ff1660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:49.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.477+0000 7fc35ab1ef40 1 -- 172.21.15.38:0/2424191489 shutdown_connections 2024-09-21T03:40:49.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.477+0000 7fc35ab1ef40 1 -- 172.21.15.38:0/2424191489 wait complete. 2024-09-21T03:40:49.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.477+0000 7fc35ab1ef40 1 Processor -- start 2024-09-21T03:40:49.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.478+0000 7fc35ab1ef40 1 -- start start 2024-09-21T03:40:49.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.479+0000 7fc35ab1ef40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5572b1153640 0x5572b10abce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:49.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.479+0000 7fc35ab1ef40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5572b1153a50 0x5572b1156b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:49.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.479+0000 7fc35ab1ef40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc33c002500 0x5572b1157040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:49.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.479+0000 7fc35ab1ef40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5572b1154390 con 0x5572b1153640 2024-09-21T03:40:49.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.479+0000 7fc35ab1ef40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5572b10ad160 con 0x5572b1153a50 2024-09-21T03:40:49.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.479+0000 7fc35ab1ef40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x5572b10ace80 con 0x7fc33c002500 2024-09-21T03:40:49.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.479+0000 7fc353cc0640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5572b1153640 0x5572b10abce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:49.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.479+0000 7fc353cc0640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5572b1153640 0x5572b10abce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:59034/0 (socket says 172.21.15.38:59034) 2024-09-21T03:40:49.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.479+0000 7fc353cc0640 1 -- 172.21.15.38:0/3954561897 learned_addr learned my addr 172.21.15.38:0/3954561897 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:49.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.479+0000 7fc3544c1640 1 --2- 172.21.15.38:0/3954561897 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc33c002500 0x5572b1157040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:49.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.479+0000 7fc3534bf640 1 --2- 172.21.15.38:0/3954561897 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5572b1153a50 0x5572b1156b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:49.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.479+0000 7fc353cc0640 1 -- 172.21.15.38:0/3954561897 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc33c002500 msgr2=0x5572b1157040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:49.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.479+0000 7fc353cc0640 1 --2- 172.21.15.38:0/3954561897 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc33c002500 0x5572b1157040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:49.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.479+0000 7fc353cc0640 1 -- 172.21.15.38:0/3954561897 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5572b1153a50 msgr2=0x5572b1156b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:49.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.479+0000 7fc353cc0640 1 --2- 172.21.15.38:0/3954561897 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5572b1153a50 0x5572b1156b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:49.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.479+0000 7fc353cc0640 1 -- 172.21.15.38:0/3954561897 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5572b1023ac0 con 0x5572b1153640 2024-09-21T03:40:49.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.479+0000 7fc353cc0640 1 --2- 172.21.15.38:0/3954561897 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5572b1153640 0x5572b10abce0 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7fc34400b8b0 tx=0x7fc344056d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:49.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.480+0000 7fc3514bb640 1 -- 172.21.15.38:0/3954561897 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc344070020 con 0x5572b1153640 2024-09-21T03:40:49.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.480+0000 7fc3514bb640 1 -- 172.21.15.38:0/3954561897 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc344060730 con 0x5572b1153640 2024-09-21T03:40:49.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.480+0000 7fc35ab1ef40 1 -- 172.21.15.38:0/3954561897 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5572b1038140 con 0x5572b1153640 2024-09-21T03:40:49.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.480+0000 7fc3514bb640 1 -- 172.21.15.38:0/3954561897 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc344060b80 con 0x5572b1153640 2024-09-21T03:40:49.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.480+0000 7fc35ab1ef40 1 -- 172.21.15.38:0/3954561897 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5572b10afd30 con 0x5572b1153640 2024-09-21T03:40:49.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.481+0000 7fc3514bb640 1 -- 172.21.15.38:0/3954561897 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc344073020 con 0x5572b1153640 2024-09-21T03:40:49.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.482+0000 7fc3514bb640 1 --2- 172.21.15.38:0/3954561897 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc3400775a0 0x7fc340079ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:49.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.482+0000 7fc3514bb640 1 -- 172.21.15.38:0/3954561897 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 6820+0+0 (secure 0 0 0) 0x7fc3440dfbe0 con 0x5572b1153640 2024-09-21T03:40:49.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.482+0000 7fc3534bf640 1 --2- 172.21.15.38:0/3954561897 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc3400775a0 0x7fc340079ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:49.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.482+0000 7fc35ab1ef40 1 --2- 172.21.15.38:0/3954561897 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x5572b10ac7b0 0x5572b103a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:49.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.482+0000 7fc35ab1ef40 1 -- 172.21.15.38:0/3954561897 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:1 13.6 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e102) -- 0x5572b1021aa0 con 0x5572b10ac7b0 2024-09-21T03:40:49.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.482+0000 7fc3544c1640 1 --2- 172.21.15.38:0/3954561897 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x5572b10ac7b0 0x5572b103a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:49.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.483+0000 7fc3534bf640 1 --2- 172.21.15.38:0/3954561897 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc3400775a0 0x7fc340079ab0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fc348001050 tx=0x7fc348003040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:49.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.483+0000 7fc3544c1640 1 --2- 172.21.15.38:0/3954561897 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x5572b10ac7b0 0x5572b103a130 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:49.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.484+0000 7fc3544c1640 1 -- 172.21.15.38:0/3954561897 <== osd.3 v2:172.21.15.38:6808/1286810294 1 ==== osd_op_reply(1 mds0_openfiles.0 [delete] v102'28 uv28 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fc34c00b040 con 0x5572b10ac7b0 2024-09-21T03:40:49.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.485+0000 7fc35ab1ef40 1 -- 172.21.15.38:0/3954561897 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x5572b10ac7b0 msgr2=0x5572b103a130 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:49.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.485+0000 7fc35ab1ef40 1 --2- 172.21.15.38:0/3954561897 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x5572b10ac7b0 0x5572b103a130 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:49.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.485+0000 7fc35ab1ef40 1 -- 172.21.15.38:0/3954561897 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc3400775a0 msgr2=0x7fc340079ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:49.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.485+0000 7fc35ab1ef40 1 --2- 172.21.15.38:0/3954561897 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc3400775a0 0x7fc340079ab0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fc348001050 tx=0x7fc348003040 comp rx=0 tx=0).stop 2024-09-21T03:40:49.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.485+0000 7fc35ab1ef40 1 -- 172.21.15.38:0/3954561897 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5572b1153640 msgr2=0x5572b10abce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:49.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.485+0000 7fc35ab1ef40 1 --2- 172.21.15.38:0/3954561897 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5572b1153640 0x5572b10abce0 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7fc34400b8b0 tx=0x7fc344056d20 comp rx=0 tx=0).stop 2024-09-21T03:40:49.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.485+0000 7fc3544c1640 1 -- 172.21.15.38:0/3954561897 reap_dead start 2024-09-21T03:40:49.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.485+0000 7fc35ab1ef40 1 -- 172.21.15.38:0/3954561897 shutdown_connections 2024-09-21T03:40:49.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.485+0000 7fc35ab1ef40 1 -- 172.21.15.38:0/3954561897 >> 172.21.15.38:0/3954561897 conn(0x5572b0f14ac0 msgr2=0x5572b0ff2010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:49.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.485+0000 7fc35ab1ef40 1 -- 172.21.15.38:0/3954561897 shutdown_connections 2024-09-21T03:40:49.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.485+0000 7fc35ab1ef40 1 -- 172.21.15.38:0/3954561897 wait complete. 2024-09-21T03:40:49.490 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-21T03:40:49.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.606+0000 7fc2f85f1640 1 -- 172.21.15.38:0/2916742288 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc2f015ee80 msgr2=0x7fc2f00d0170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:49.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.606+0000 7fc2f85f1640 1 --2- 172.21.15.38:0/2916742288 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc2f015ee80 0x7fc2f00d0170 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec005fd0 tx=0x7fc2ec02a820 comp rx=0 tx=0).stop 2024-09-21T03:40:49.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.606+0000 7fc2f85f1640 1 -- 172.21.15.38:0/2916742288 shutdown_connections 2024-09-21T03:40:49.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.606+0000 7fc2f85f1640 1 --2- 172.21.15.38:0/2916742288 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc2f015ee80 0x7fc2f00d0170 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:49.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.606+0000 7fc2f85f1640 1 -- 172.21.15.38:0/2916742288 >> 172.21.15.38:0/2916742288 conn(0x7fc2f009d830 msgr2=0x7fc2f00b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:49.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.606+0000 7fc2f85f1640 1 -- 172.21.15.38:0/2916742288 shutdown_connections 2024-09-21T03:40:49.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.606+0000 7fc2f85f1640 1 -- 172.21.15.38:0/2916742288 wait complete. 2024-09-21T03:40:49.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.608+0000 7fc2f85f1640 1 Processor -- start 2024-09-21T03:40:49.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.608+0000 7fc2f85f1640 1 -- start start 2024-09-21T03:40:49.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.609+0000 7fc2f85f1640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc2f00b9ad0 0x7fc2f0158900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:49.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.609+0000 7fc2f85f1640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc2f00ceb90 0x7fc2f015af10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:49.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.610+0000 7fc2f85f1640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc2f015ee80 0x7fc2f015b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:49.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.610+0000 7fc2f85f1640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fc2f00b6910 con 0x7fc2f00b9ad0 2024-09-21T03:40:49.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.610+0000 7fc2f85f1640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fc2f00b6790 con 0x7fc2f015ee80 2024-09-21T03:40:49.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.610+0000 7fc2f85f1640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fc2f00b6a90 con 0x7fc2f00ceb90 2024-09-21T03:40:49.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.610+0000 7fc2f638d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc2f00b9ad0 0x7fc2f0158900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:49.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.610+0000 7fc2f638d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc2f00b9ad0 0x7fc2f0158900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:59062/0 (socket says 172.21.15.38:59062) 2024-09-21T03:40:49.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.610+0000 7fc2f638d640 1 -- 172.21.15.38:0/2661324572 learned_addr learned my addr 172.21.15.38:0/2661324572 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:49.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.610+0000 7fc2f6b8e640 1 --2- 172.21.15.38:0/2661324572 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc2f015ee80 0x7fc2f015b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:49.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.610+0000 7fc2f5b8c640 1 --2- 172.21.15.38:0/2661324572 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc2f00ceb90 0x7fc2f015af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:49.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.610+0000 7fc2f638d640 1 -- 172.21.15.38:0/2661324572 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc2f00ceb90 msgr2=0x7fc2f015af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:49.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.610+0000 7fc2f638d640 1 --2- 172.21.15.38:0/2661324572 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc2f00ceb90 0x7fc2f015af10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:49.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.610+0000 7fc2f638d640 1 -- 172.21.15.38:0/2661324572 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc2f015ee80 msgr2=0x7fc2f015b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:49.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.610+0000 7fc2f638d640 1 --2- 172.21.15.38:0/2661324572 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc2f015ee80 0x7fc2f015b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:49.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.610+0000 7fc2f638d640 1 -- 172.21.15.38:0/2661324572 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2f0159650 con 0x7fc2f00b9ad0 2024-09-21T03:40:49.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.610+0000 7fc2f6b8e640 1 --2- 172.21.15.38:0/2661324572 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc2f015ee80 0x7fc2f015b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:40:49.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.610+0000 7fc2f638d640 1 --2- 172.21.15.38:0/2661324572 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc2f00b9ad0 0x7fc2f0158900 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec035040 tx=0x7fc2ec032c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:49.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.610+0000 7fc2e6ffd640 1 -- 172.21.15.38:0/2661324572 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2ec037040 con 0x7fc2f00b9ad0 2024-09-21T03:40:49.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.610+0000 7fc2e6ffd640 1 -- 172.21.15.38:0/2661324572 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc2ec042070 con 0x7fc2f00b9ad0 2024-09-21T03:40:49.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.610+0000 7fc2e6ffd640 1 -- 172.21.15.38:0/2661324572 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2ec0042a0 con 0x7fc2f00b9ad0 2024-09-21T03:40:49.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.610+0000 7fc2f85f1640 1 -- 172.21.15.38:0/2661324572 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2f0159930 con 0x7fc2f00b9ad0 2024-09-21T03:40:49.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.610+0000 7fc2f85f1640 1 -- 172.21.15.38:0/2661324572 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2f0159ef0 con 0x7fc2f00b9ad0 2024-09-21T03:40:49.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.611+0000 7fc2f85f1640 1 -- 172.21.15.38:0/2661324572 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2f00d0170 con 0x7fc2f00b9ad0 2024-09-21T03:40:49.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.612+0000 7fc2e6ffd640 1 -- 172.21.15.38:0/2661324572 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc2ec004440 con 0x7fc2f00b9ad0 2024-09-21T03:40:49.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.613+0000 7fc2e6ffd640 1 --2- 172.21.15.38:0/2661324572 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc2d4078b90 0x7fc2d407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:49.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.613+0000 7fc2e6ffd640 1 -- 172.21.15.38:0/2661324572 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 6820+0+0 (secure 0 0 0) 0x7fc2ec0bc030 con 0x7fc2f00b9ad0 2024-09-21T03:40:49.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.613+0000 7fc2f5b8c640 1 --2- 172.21.15.38:0/2661324572 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc2d4078b90 0x7fc2d407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:49.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.613+0000 7fc2f5b8c640 1 --2- 172.21.15.38:0/2661324572 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc2d4078b90 0x7fc2d407b0a0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fc2f00ce560 tx=0x7fc2e0009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:49.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.617+0000 7fc2e6ffd640 1 -- 172.21.15.38:0/2661324572 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fc2ec0840a0 con 0x7fc2f00b9ad0 2024-09-21T03:40:49.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:49.844+0000 7fc2f85f1640 1 -- 172.21.15.38:0/2661324572 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7fc2f00b4110 con 0x7fc2f00b9ad0 2024-09-21T03:40:50.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:50.129+0000 7fc2e6ffd640 1 -- 172.21.15.38:0/2661324572 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v120) ==== 162+0+0 (secure 0 0 0) 0x7fc2ec0839a0 con 0x7fc2f00b9ad0 2024-09-21T03:40:50.132 INFO:teuthology.orchestra.run.smithi038.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-21T03:40:50.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:50.132+0000 7fc2f85f1640 1 -- 172.21.15.38:0/2661324572 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc2d4078b90 msgr2=0x7fc2d407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:50.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:50.132+0000 7fc2f85f1640 1 --2- 172.21.15.38:0/2661324572 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc2d4078b90 0x7fc2d407b0a0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fc2f00ce560 tx=0x7fc2e0009040 comp rx=0 tx=0).stop 2024-09-21T03:40:50.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:50.132+0000 7fc2f85f1640 1 -- 172.21.15.38:0/2661324572 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc2f00b9ad0 msgr2=0x7fc2f0158900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:50.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:50.132+0000 7fc2f85f1640 1 --2- 172.21.15.38:0/2661324572 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc2f00b9ad0 0x7fc2f0158900 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec035040 tx=0x7fc2ec032c90 comp rx=0 tx=0).stop 2024-09-21T03:40:50.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:50.132+0000 7fc2f85f1640 1 -- 172.21.15.38:0/2661324572 shutdown_connections 2024-09-21T03:40:50.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:50.132+0000 7fc2f85f1640 1 --2- 172.21.15.38:0/2661324572 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc2f015ee80 0x7fc2f015b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:50.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:50.132+0000 7fc2f85f1640 1 --2- 172.21.15.38:0/2661324572 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc2d4078b90 0x7fc2d407b0a0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:50.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:50.132+0000 7fc2f85f1640 1 --2- 172.21.15.38:0/2661324572 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc2f00ceb90 0x7fc2f015af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:50.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:50.132+0000 7fc2f85f1640 1 --2- 172.21.15.38:0/2661324572 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc2f00b9ad0 0x7fc2f0158900 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:50.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:50.132+0000 7fc2f85f1640 1 -- 172.21.15.38:0/2661324572 >> 172.21.15.38:0/2661324572 conn(0x7fc2f009d830 msgr2=0x7fc2f0160ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:50.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:50.132+0000 7fc2f85f1640 1 -- 172.21.15.38:0/2661324572 shutdown_connections 2024-09-21T03:40:50.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:50.132+0000 7fc2f85f1640 1 -- 172.21.15.38:0/2661324572 wait complete. 2024-09-21T03:40:50.139 INFO:tasks.ceph.mds.c.smithi038.stderr:2024-09-21T03:40:50.137+0000 7f413f25b640 -1 mds.0.openfiles _load_finish got (2) No such file or directory 2024-09-21T03:40:50.550 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:40:50.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:50.674+0000 7f0f2fe55640 1 -- 172.21.15.38:0/3513447554 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0f280b4f70 msgr2=0x7f0f280b2e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:50.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:50.674+0000 7f0f2fe55640 1 --2- 172.21.15.38:0/3513447554 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0f280b4f70 0x7f0f280b2e60 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f0f20005fd0 tx=0x7f0f2002a940 comp rx=0 tx=0).stop 2024-09-21T03:40:50.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:50.675+0000 7f0f2fe55640 1 -- 172.21.15.38:0/3513447554 shutdown_connections 2024-09-21T03:40:50.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:50.675+0000 7f0f2fe55640 1 --2- 172.21.15.38:0/3513447554 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0f280b4f70 0x7f0f280b2e60 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:50.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:50.675+0000 7f0f2fe55640 1 -- 172.21.15.38:0/3513447554 >> 172.21.15.38:0/3513447554 conn(0x7f0f280d4440 msgr2=0x7f0f280d4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:50.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:50.675+0000 7f0f2fe55640 1 -- 172.21.15.38:0/3513447554 shutdown_connections 2024-09-21T03:40:50.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:50.675+0000 7f0f2fe55640 1 -- 172.21.15.38:0/3513447554 wait complete. 2024-09-21T03:40:50.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:50.676+0000 7f0f2fe55640 1 Processor -- start 2024-09-21T03:40:50.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:50.677+0000 7f0f2fe55640 1 -- start start 2024-09-21T03:40:50.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:50.678+0000 7f0f2fe55640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0f280b4f70 0x7f0f28152f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:50.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:50.678+0000 7f0f2fe55640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0f280cfe00 0x7f0f281534b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:50.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:50.678+0000 7f0f2fe55640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0f28162590 0x7f0f281f9190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:50.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:50.678+0000 7f0f2fe55640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f0f280b3730 con 0x7f0f28162590 2024-09-21T03:40:50.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:50.678+0000 7f0f2fe55640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f0f280b35b0 con 0x7f0f280b4f70 2024-09-21T03:40:50.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:50.678+0000 7f0f2fe55640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f0f280b38b0 con 0x7f0f280cfe00 2024-09-21T03:40:50.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:50.678+0000 7f0f2e3f2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0f28162590 0x7f0f281f9190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:50.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:50.678+0000 7f0f2e3f2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0f28162590 0x7f0f281f9190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:59074/0 (socket says 172.21.15.38:59074) 2024-09-21T03:40:50.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:50.678+0000 7f0f2e3f2640 1 -- 172.21.15.38:0/3745031934 learned_addr learned my addr 172.21.15.38:0/3745031934 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:50.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:50.678+0000 7f0f2dbf1640 1 --2- 172.21.15.38:0/3745031934 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0f280b4f70 0x7f0f28152f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:50.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:50.678+0000 7f0f2d3f0640 1 --2- 172.21.15.38:0/3745031934 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0f280cfe00 0x7f0f281534b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:50.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:50.678+0000 7f0f2e3f2640 1 -- 172.21.15.38:0/3745031934 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0f280cfe00 msgr2=0x7f0f281534b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:50.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:50.678+0000 7f0f2e3f2640 1 --2- 172.21.15.38:0/3745031934 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0f280cfe00 0x7f0f281534b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:50.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:50.678+0000 7f0f2e3f2640 1 -- 172.21.15.38:0/3745031934 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0f280b4f70 msgr2=0x7f0f28152f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:50.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:50.678+0000 7f0f2e3f2640 1 --2- 172.21.15.38:0/3745031934 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0f280b4f70 0x7f0f28152f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:50.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:50.678+0000 7f0f2e3f2640 1 -- 172.21.15.38:0/3745031934 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f28151ae0 con 0x7f0f28162590 2024-09-21T03:40:50.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:50.678+0000 7f0f2dbf1640 1 --2- 172.21.15.38:0/3745031934 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0f280b4f70 0x7f0f28152f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:40:50.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:50.678+0000 7f0f2e3f2640 1 --2- 172.21.15.38:0/3745031934 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0f28162590 0x7f0f281f9190 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f0f20009cd0 tx=0x7f0f20031ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:50.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:50.679+0000 7f0f1e7fc640 1 -- 172.21.15.38:0/3745031934 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f2003e070 con 0x7f0f28162590 2024-09-21T03:40:50.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:50.679+0000 7f0f1e7fc640 1 -- 172.21.15.38:0/3745031934 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0f20004560 con 0x7f0f28162590 2024-09-21T03:40:50.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:50.679+0000 7f0f2fe55640 1 -- 172.21.15.38:0/3745031934 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f28151dc0 con 0x7f0f28162590 2024-09-21T03:40:50.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:50.679+0000 7f0f1e7fc640 1 -- 172.21.15.38:0/3745031934 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f20043460 con 0x7f0f28162590 2024-09-21T03:40:50.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:50.679+0000 7f0f2fe55640 1 -- 172.21.15.38:0/3745031934 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f28152380 con 0x7f0f28162590 2024-09-21T03:40:50.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:50.680+0000 7f0f2fe55640 1 -- 172.21.15.38:0/3745031934 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f28151850 con 0x7f0f28162590 2024-09-21T03:40:50.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:50.681+0000 7f0f1e7fc640 1 -- 172.21.15.38:0/3745031934 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f0f2003a440 con 0x7f0f28162590 2024-09-21T03:40:50.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:50.681+0000 7f0f1e7fc640 1 --2- 172.21.15.38:0/3745031934 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0f0c078b90 0x7f0f0c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:50.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:50.681+0000 7f0f1e7fc640 1 -- 172.21.15.38:0/3745031934 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 6820+0+0 (secure 0 0 0) 0x7f0f200bcd20 con 0x7f0f28162590 2024-09-21T03:40:50.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:50.681+0000 7f0f2dbf1640 1 --2- 172.21.15.38:0/3745031934 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0f0c078b90 0x7f0f0c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:50.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:50.682+0000 7f0f2dbf1640 1 --2- 172.21.15.38:0/3745031934 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0f0c078b90 0x7f0f0c07b0a0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f0f18007a50 tx=0x7f0f18009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:50.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:50.686+0000 7f0f1e7fc640 1 -- 172.21.15.38:0/3745031934 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f0f20084d00 con 0x7f0f28162590 2024-09-21T03:40:50.858 INFO:tasks.ceph.mds.a.smithi038.stderr: -43> 2024-09-21T03:40:46.104+0000 7f9668d00640 -1 log_channel(cluster) log [ERR] : missing journal object 2024-09-21T03:40:50.912 INFO:tasks.ceph.mds.a.smithi038.stdout:starting mds.a at 2024-09-21T03:40:50.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:50.913+0000 7f0f2fe55640 1 -- 172.21.15.38:0/3745031934 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0f280d1cf0 con 0x7f0f28162590 2024-09-21T03:40:50.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:50.914+0000 7f0f1e7fc640 1 -- 172.21.15.38:0/3745031934 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 121 v121) ==== 95+0+4401 (secure 0 0 0) 0x7f0f280d1cf0 con 0x7f0f28162590 2024-09-21T03:40:50.916 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:40:50.916 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":121,"btime":"2024-09-21T03:40:50:133029+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6086,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/965972663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":965972663},{"type":"v1","addr":"172.21.15.195:6836","nonce":965972663}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6700,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/3176122334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":3176122334},{"type":"v1","addr":"172.21.15.195:6837","nonce":3176122334}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":121,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:40:02.319282+0000","modified":"2024-09-21T03:40:50.133016+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":102,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7995},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7995":{"gid":7995,"name":"c","rank":0,"incarnation":121,"state":"up:replay","state_seq":1,"addr":"172.21.15.38:6833/3710736169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":3710736169},{"type":"v1","addr":"172.21.15.38:6833","nonce":3710736169}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-21T03:40:50.916 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 121 2024-09-21T03:40:50.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:50.916+0000 7f0f2fe55640 1 -- 172.21.15.38:0/3745031934 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0f0c078b90 msgr2=0x7f0f0c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:50.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:50.916+0000 7f0f2fe55640 1 --2- 172.21.15.38:0/3745031934 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0f0c078b90 0x7f0f0c07b0a0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f0f18007a50 tx=0x7f0f18009040 comp rx=0 tx=0).stop 2024-09-21T03:40:50.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:50.916+0000 7f0f2fe55640 1 -- 172.21.15.38:0/3745031934 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0f28162590 msgr2=0x7f0f281f9190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:50.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:50.916+0000 7f0f2fe55640 1 --2- 172.21.15.38:0/3745031934 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0f28162590 0x7f0f281f9190 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f0f20009cd0 tx=0x7f0f20031ea0 comp rx=0 tx=0).stop 2024-09-21T03:40:50.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:50.916+0000 7f0f2fe55640 1 -- 172.21.15.38:0/3745031934 shutdown_connections 2024-09-21T03:40:50.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:50.916+0000 7f0f2fe55640 1 --2- 172.21.15.38:0/3745031934 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0f28162590 0x7f0f281f9190 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:50.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:50.916+0000 7f0f2fe55640 1 --2- 172.21.15.38:0/3745031934 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0f0c078b90 0x7f0f0c07b0a0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:50.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:50.916+0000 7f0f2fe55640 1 --2- 172.21.15.38:0/3745031934 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0f280cfe00 0x7f0f281534b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:50.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:50.916+0000 7f0f2fe55640 1 --2- 172.21.15.38:0/3745031934 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0f280b4f70 0x7f0f28152f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:50.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:50.916+0000 7f0f2fe55640 1 -- 172.21.15.38:0/3745031934 >> 172.21.15.38:0/3745031934 conn(0x7f0f280d4440 msgr2=0x7f0f2815eea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:50.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:50.916+0000 7f0f2fe55640 1 -- 172.21.15.38:0/3745031934 shutdown_connections 2024-09-21T03:40:50.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:50.916+0000 7f0f2fe55640 1 -- 172.21.15.38:0/3745031934 wait complete. 2024-09-21T03:40:50.930 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 121, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T03:40:02.319282+0000', 'modified': '2024-09-21T03:40:50.133016+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 102, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7995}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7995': {'gid': 7995, 'name': 'c', 'rank': 0, 'incarnation': 121, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.38:6833/3710736169', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.38:6832', 'nonce': 3710736169}, {'type': 'v1', 'addr': '172.21.15.38:6833', 'nonce': 3710736169}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-09-21T03:40:50.930 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7995:up:replay 2024-09-21T03:40:50.930 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2024-09-21T03:40:56.334 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:40:56.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:56.463+0000 7f83a4ed5640 1 -- 172.21.15.38:0/1278545439 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f83a0162280 msgr2=0x7f83a0162690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:56.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:56.463+0000 7f83a4ed5640 1 --2- 172.21.15.38:0/1278545439 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f83a0162280 0x7f83a0162690 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f83a00d0c00 tx=0x7f839402a4d0 comp rx=0 tx=0).stop 2024-09-21T03:40:56.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:56.463+0000 7f83a4ed5640 1 -- 172.21.15.38:0/1278545439 shutdown_connections 2024-09-21T03:40:56.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:56.463+0000 7f83a4ed5640 1 --2- 172.21.15.38:0/1278545439 >> v2:172.21.15.195:3300/0 conn(0x7f83a00b2e40 0x7f83a00b3250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:56.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:56.463+0000 7f83a4ed5640 1 --2- 172.21.15.38:0/1278545439 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f83a0162280 0x7f83a0162690 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:56.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:56.463+0000 7f83a4ed5640 1 -- 172.21.15.38:0/1278545439 >> 172.21.15.38:0/1278545439 conn(0x7f83a00b6b20 msgr2=0x7f83a00d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:56.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:56.463+0000 7f83a4ed5640 1 -- 172.21.15.38:0/1278545439 shutdown_connections 2024-09-21T03:40:56.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:56.463+0000 7f83a4ed5640 1 -- 172.21.15.38:0/1278545439 wait complete. 2024-09-21T03:40:56.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:56.465+0000 7f83a4ed5640 1 Processor -- start 2024-09-21T03:40:56.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:56.465+0000 7f83a4ed5640 1 -- start start 2024-09-21T03:40:56.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:56.467+0000 7f83a4ed5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f83a00b2e40 0x7f83a00ce940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:56.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:56.467+0000 7f83a4ed5640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f83a0162280 0x7f83a00cee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:56.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:56.467+0000 7f83a4ed5640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f83a00b1b30 0x7f83a015ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:56.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:56.467+0000 7f83a4ed5640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f83a015e4b0 con 0x7f83a00b2e40 2024-09-21T03:40:56.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:56.467+0000 7f83a4ed5640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f83a00b39a0 con 0x7f83a00b1b30 2024-09-21T03:40:56.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:56.467+0000 7f83a4ed5640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f83a015e630 con 0x7f83a0162280 2024-09-21T03:40:56.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:56.467+0000 7f839e59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f83a00b2e40 0x7f83a00ce940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:56.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:56.467+0000 7f839e59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f83a00b2e40 0x7f83a00ce940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:39132/0 (socket says 172.21.15.38:39132) 2024-09-21T03:40:56.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:56.467+0000 7f839e59c640 1 -- 172.21.15.38:0/1389537863 learned_addr learned my addr 172.21.15.38:0/1389537863 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:56.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:56.467+0000 7f839ed9d640 1 --2- 172.21.15.38:0/1389537863 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f83a00b1b30 0x7f83a015ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:56.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:56.467+0000 7f839dd9b640 1 --2- 172.21.15.38:0/1389537863 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f83a0162280 0x7f83a00cee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:56.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:56.467+0000 7f839e59c640 1 -- 172.21.15.38:0/1389537863 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f83a0162280 msgr2=0x7f83a00cee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:56.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:56.467+0000 7f839e59c640 1 --2- 172.21.15.38:0/1389537863 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f83a0162280 0x7f83a00cee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:56.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:56.467+0000 7f839e59c640 1 -- 172.21.15.38:0/1389537863 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f83a00b1b30 msgr2=0x7f83a015ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:56.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:56.467+0000 7f839e59c640 1 --2- 172.21.15.38:0/1389537863 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f83a00b1b30 0x7f83a015ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:56.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:56.467+0000 7f839e59c640 1 -- 172.21.15.38:0/1389537863 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83a00abe10 con 0x7f83a00b2e40 2024-09-21T03:40:56.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:56.468+0000 7f839e59c640 1 --2- 172.21.15.38:0/1389537863 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f83a00b2e40 0x7f83a00ce940 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f839000ba50 tx=0x7f839000bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:56.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:56.468+0000 7f838effd640 1 -- 172.21.15.38:0/1389537863 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8390019070 con 0x7f83a00b2e40 2024-09-21T03:40:56.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:56.468+0000 7f838effd640 1 -- 172.21.15.38:0/1389537863 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f839000ccb0 con 0x7f83a00b2e40 2024-09-21T03:40:56.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:56.468+0000 7f83a4ed5640 1 -- 172.21.15.38:0/1389537863 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83a00ac280 con 0x7f83a00b2e40 2024-09-21T03:40:56.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:56.468+0000 7f838effd640 1 -- 172.21.15.38:0/1389537863 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83900146a0 con 0x7f83a00b2e40 2024-09-21T03:40:56.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:56.468+0000 7f83a4ed5640 1 -- 172.21.15.38:0/1389537863 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83a00ac840 con 0x7f83a00b2e40 2024-09-21T03:40:56.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:56.469+0000 7f83a4ed5640 1 -- 172.21.15.38:0/1389537863 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83a0162f00 con 0x7f83a00b2e40 2024-09-21T03:40:56.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:56.470+0000 7f838effd640 1 -- 172.21.15.38:0/1389537863 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f839001c070 con 0x7f83a00b2e40 2024-09-21T03:40:56.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:56.470+0000 7f838effd640 1 --2- 172.21.15.38:0/1389537863 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8380078b90 0x7f838007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:56.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:56.470+0000 7f838effd640 1 -- 172.21.15.38:0/1389537863 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 6820+0+0 (secure 0 0 0) 0x7f839009ae50 con 0x7f83a00b2e40 2024-09-21T03:40:56.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:56.470+0000 7f839dd9b640 1 --2- 172.21.15.38:0/1389537863 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8380078b90 0x7f838007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:56.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:56.471+0000 7f839dd9b640 1 --2- 172.21.15.38:0/1389537863 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8380078b90 0x7f838007b0a0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f8394033000 tx=0x7f8394004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:56.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:56.475+0000 7f838effd640 1 -- 172.21.15.38:0/1389537863 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f8390062760 con 0x7f83a00b2e40 2024-09-21T03:40:56.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:56.700+0000 7f83a4ed5640 1 -- 172.21.15.38:0/1389537863 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f83a006d760 con 0x7f83a00b2e40 2024-09-21T03:40:56.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:56.700+0000 7f838effd640 1 -- 172.21.15.38:0/1389537863 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 124 v124) ==== 95+0+5234 (secure 0 0 0) 0x7f8390067190 con 0x7f83a00b2e40 2024-09-21T03:40:56.702 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:40:56.703 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":124,"btime":"2024-09-21T03:40:53:147947+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6086,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/965972663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":965972663},{"type":"v1","addr":"172.21.15.195:6836","nonce":965972663}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6700,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/3176122334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":3176122334},{"type":"v1","addr":"172.21.15.195:6837","nonce":3176122334}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":8061,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/2748959970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":2748959970},{"type":"v1","addr":"172.21.15.38:6835","nonce":2748959970}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":122}],"filesystems":[{"mdsmap":{"epoch":124,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:40:02.319282+0000","modified":"2024-09-21T03:40:53.147941+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":102,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7995},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7995":{"gid":7995,"name":"c","rank":0,"incarnation":121,"state":"up:active","state_seq":4,"addr":"172.21.15.38:6833/3710736169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":3710736169},{"type":"v1","addr":"172.21.15.38:6833","nonce":3710736169}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7995,"qdb_cluster":[7995]},"id":6}]} 2024-09-21T03:40:56.703 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 124 2024-09-21T03:40:56.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:56.702+0000 7f83a4ed5640 1 -- 172.21.15.38:0/1389537863 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8380078b90 msgr2=0x7f838007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:57.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:56.702+0000 7f83a4ed5640 1 --2- 172.21.15.38:0/1389537863 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8380078b90 0x7f838007b0a0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f8394033000 tx=0x7f8394004730 comp rx=0 tx=0).stop 2024-09-21T03:40:57.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:56.702+0000 7f83a4ed5640 1 -- 172.21.15.38:0/1389537863 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f83a00b2e40 msgr2=0x7f83a00ce940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:57.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:56.702+0000 7f83a4ed5640 1 --2- 172.21.15.38:0/1389537863 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f83a00b2e40 0x7f83a00ce940 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f839000ba50 tx=0x7f839000bf10 comp rx=0 tx=0).stop 2024-09-21T03:40:57.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:56.702+0000 7f83a4ed5640 1 -- 172.21.15.38:0/1389537863 shutdown_connections 2024-09-21T03:40:57.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:56.702+0000 7f83a4ed5640 1 --2- 172.21.15.38:0/1389537863 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f83a00b1b30 0x7f83a015ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:57.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:56.702+0000 7f83a4ed5640 1 --2- 172.21.15.38:0/1389537863 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8380078b90 0x7f838007b0a0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:57.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:56.702+0000 7f83a4ed5640 1 --2- 172.21.15.38:0/1389537863 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f83a0162280 0x7f83a00cee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:57.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:56.702+0000 7f83a4ed5640 1 --2- 172.21.15.38:0/1389537863 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f83a00b2e40 0x7f83a00ce940 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:57.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:56.702+0000 7f83a4ed5640 1 -- 172.21.15.38:0/1389537863 >> 172.21.15.38:0/1389537863 conn(0x7f83a00b6b20 msgr2=0x7f83a0068f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:57.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:56.702+0000 7f83a4ed5640 1 -- 172.21.15.38:0/1389537863 shutdown_connections 2024-09-21T03:40:57.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:56.702+0000 7f83a4ed5640 1 -- 172.21.15.38:0/1389537863 wait complete. 2024-09-21T03:40:57.148 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 124, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T03:40:02.319282+0000', 'modified': '2024-09-21T03:40:53.147941+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 102, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7995}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7995': {'gid': 7995, 'name': 'c', 'rank': 0, 'incarnation': 121, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.38:6833/3710736169', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.38:6832', 'nonce': 3710736169}, {'type': 'v1', 'addr': '172.21.15.38:6833', 'nonce': 3710736169}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7995, 'qdb_cluster': [7995]} 2024-09-21T03:40:57.148 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-21T03:40:57.148 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-21T03:40:57.149 INFO:tasks.ceph.mds.c.smithi038.stderr:2024-09-21T03:40:57.116+0000 7f4143263640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T03:40:57.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.268+0000 7f41fb207640 1 -- 172.21.15.38:0/1766223337 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f41f40b9ad0 msgr2=0x7f41f40d0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:57.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.268+0000 7f41fb207640 1 --2- 172.21.15.38:0/1766223337 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f41f40b9ad0 0x7f41f40d0be0 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f41f41623b0 tx=0x7f41e802a4b0 comp rx=0 tx=0).stop 2024-09-21T03:40:57.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.268+0000 7f41fb207640 1 -- 172.21.15.38:0/1766223337 shutdown_connections 2024-09-21T03:40:57.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.268+0000 7f41fb207640 1 --2- 172.21.15.38:0/1766223337 >> v2:172.21.15.195:3300/0 conn(0x7f41f40d1120 0x7f41f40ceb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:57.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.268+0000 7f41fb207640 1 --2- 172.21.15.38:0/1766223337 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f41f40b9ad0 0x7f41f40d0be0 unknown :-1 s=CLOSED pgs=1365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:57.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.268+0000 7f41fb207640 1 --2- 172.21.15.38:0/1766223337 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f41f415ee80 0x7f41f40d0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:57.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.268+0000 7f41fb207640 1 -- 172.21.15.38:0/1766223337 >> 172.21.15.38:0/1766223337 conn(0x7f41f409d830 msgr2=0x7f41f40b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:57.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.268+0000 7f41fb207640 1 -- 172.21.15.38:0/1766223337 shutdown_connections 2024-09-21T03:40:57.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.268+0000 7f41fb207640 1 -- 172.21.15.38:0/1766223337 wait complete. 2024-09-21T03:40:57.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.270+0000 7f41fb207640 1 Processor -- start 2024-09-21T03:40:57.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.270+0000 7f41fb207640 1 -- start start 2024-09-21T03:40:57.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.271+0000 7f41fb207640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f41f40b9ad0 0x7f41f40ca820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:57.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.271+0000 7f41fb207640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f41f40d1120 0x7f41f41f6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:57.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.271+0000 7f41fb207640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f41f415ee80 0x7f41f41f71e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:57.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.271+0000 7f41fb207640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f41f40cf460 con 0x7f41f40b9ad0 2024-09-21T03:40:57.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.271+0000 7f41fb207640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f41f40cf2e0 con 0x7f41f415ee80 2024-09-21T03:40:57.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.271+0000 7f41fb207640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f41f40b6580 con 0x7f41f40d1120 2024-09-21T03:40:57.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.271+0000 7f41f8fa3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f41f40b9ad0 0x7f41f40ca820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:57.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.271+0000 7f41f8fa3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f41f40b9ad0 0x7f41f40ca820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:39148/0 (socket says 172.21.15.38:39148) 2024-09-21T03:40:57.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.271+0000 7f41f8fa3640 1 -- 172.21.15.38:0/667036796 learned_addr learned my addr 172.21.15.38:0/667036796 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:57.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.271+0000 7f41f97a4640 1 --2- 172.21.15.38:0/667036796 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f41f415ee80 0x7f41f41f71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:57.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.271+0000 7f41f3fff640 1 --2- 172.21.15.38:0/667036796 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f41f40d1120 0x7f41f41f6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:57.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.272+0000 7f41f8fa3640 1 -- 172.21.15.38:0/667036796 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f41f40d1120 msgr2=0x7f41f41f6ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:57.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.272+0000 7f41f8fa3640 1 --2- 172.21.15.38:0/667036796 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f41f40d1120 0x7f41f41f6ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:57.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.272+0000 7f41f8fa3640 1 -- 172.21.15.38:0/667036796 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f41f415ee80 msgr2=0x7f41f41f71e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:57.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.272+0000 7f41f8fa3640 1 --2- 172.21.15.38:0/667036796 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f41f415ee80 0x7f41f41f71e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:57.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.272+0000 7f41f8fa3640 1 -- 172.21.15.38:0/667036796 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41f406fc00 con 0x7f41f40b9ad0 2024-09-21T03:40:57.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.272+0000 7f41f8fa3640 1 --2- 172.21.15.38:0/667036796 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f41f40b9ad0 0x7f41f40ca820 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f41e400d700 tx=0x7f41e400dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:57.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.272+0000 7f41f17fa640 1 -- 172.21.15.38:0/667036796 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41e4017070 con 0x7f41f40b9ad0 2024-09-21T03:40:57.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.272+0000 7f41f17fa640 1 -- 172.21.15.38:0/667036796 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f41e4005310 con 0x7f41f40b9ad0 2024-09-21T03:40:57.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.272+0000 7f41fb207640 1 -- 172.21.15.38:0/667036796 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41f4158640 con 0x7f41f40b9ad0 2024-09-21T03:40:57.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.272+0000 7f41f17fa640 1 -- 172.21.15.38:0/667036796 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41e4009820 con 0x7f41f40b9ad0 2024-09-21T03:40:57.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.272+0000 7f41fb207640 1 -- 172.21.15.38:0/667036796 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41f4156440 con 0x7f41f40b9ad0 2024-09-21T03:40:57.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.274+0000 7f41f17fa640 1 -- 172.21.15.38:0/667036796 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f41e40099c0 con 0x7f41f40b9ad0 2024-09-21T03:40:57.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.274+0000 7f41f17fa640 1 --2- 172.21.15.38:0/667036796 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f41e0078bc0 0x7f41e007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:57.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.275+0000 7f41f17fa640 1 -- 172.21.15.38:0/667036796 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 6820+0+0 (secure 0 0 0) 0x7f41e409b310 con 0x7f41f40b9ad0 2024-09-21T03:40:57.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.275+0000 7f41f3fff640 1 --2- 172.21.15.38:0/667036796 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f41e0078bc0 0x7f41e007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:57.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.275+0000 7f41f3fff640 1 --2- 172.21.15.38:0/667036796 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f41e0078bc0 0x7f41e007b0d0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f41e8034000 tx=0x7f41e8004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:57.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.275+0000 7f420224e740 1 Processor -- start 2024-09-21T03:40:57.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.275+0000 7f420224e740 1 -- start start 2024-09-21T03:40:57.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.277+0000 7f420224e740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5613b31ad270 0x5613b357efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:57.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.277+0000 7f420224e740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5613b31843b0 0x5613b357f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:57.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.277+0000 7f420224e740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5613b318c590 0x5613b359c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:57.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.277+0000 7f420224e740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5613b34c7820 con 0x5613b318c590 2024-09-21T03:40:57.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.277+0000 7f420224e740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5613b34b6c50 con 0x5613b31843b0 2024-09-21T03:40:57.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.277+0000 7f420224e740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x5613b34baa60 con 0x5613b31ad270 2024-09-21T03:40:57.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.277+0000 7f41f97a4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5613b318c590 0x5613b359c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:57.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.277+0000 7f41f97a4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5613b318c590 0x5613b359c3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:39150/0 (socket says 172.21.15.38:39150) 2024-09-21T03:40:57.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.277+0000 7f41f97a4640 1 -- 172.21.15.38:0/2214642612 learned_addr learned my addr 172.21.15.38:0/2214642612 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:57.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.277+0000 7f41f3fff640 1 --2- 172.21.15.38:0/2214642612 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5613b31843b0 0x5613b357f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:57.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.277+0000 7f41f8fa3640 1 --2- 172.21.15.38:0/2214642612 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5613b31ad270 0x5613b357efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:57.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.277+0000 7f41f97a4640 1 -- 172.21.15.38:0/2214642612 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5613b31ad270 msgr2=0x5613b357efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:57.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.277+0000 7f41f97a4640 1 --2- 172.21.15.38:0/2214642612 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5613b31ad270 0x5613b357efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:57.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.277+0000 7f41f97a4640 1 -- 172.21.15.38:0/2214642612 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5613b31843b0 msgr2=0x5613b357f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:57.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.277+0000 7f41f97a4640 1 --2- 172.21.15.38:0/2214642612 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5613b31843b0 0x5613b357f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:57.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.277+0000 7f41f97a4640 1 -- 172.21.15.38:0/2214642612 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5613b3218850 con 0x5613b318c590 2024-09-21T03:40:57.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.277+0000 7f41f97a4640 1 --2- 172.21.15.38:0/2214642612 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5613b318c590 0x5613b359c3e0 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f41ec009e00 tx=0x7f41ec0117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15c560a74b83fee7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:57.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.278+0000 7f41da7fc640 1 -- 172.21.15.38:0/2214642612 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41ec004d20 con 0x5613b318c590 2024-09-21T03:40:57.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.278+0000 7f41da7fc640 1 -- 172.21.15.38:0/2214642612 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f41ec005020 con 0x5613b318c590 2024-09-21T03:40:57.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.278+0000 7f41da7fc640 1 -- 172.21.15.38:0/2214642612 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41ec005450 con 0x5613b318c590 2024-09-21T03:40:57.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.278+0000 7f420224e740 1 -- 172.21.15.38:0/2214642612 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5613b318c590 msgr2=0x5613b359c3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:57.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.278+0000 7f420224e740 1 --2- 172.21.15.38:0/2214642612 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5613b318c590 0x5613b359c3e0 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f41ec009e00 tx=0x7f41ec0117b0 comp rx=0 tx=0).stop 2024-09-21T03:40:57.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.278+0000 7f420224e740 1 -- 172.21.15.38:0/2214642612 shutdown_connections 2024-09-21T03:40:57.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.278+0000 7f420224e740 1 --2- 172.21.15.38:0/2214642612 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5613b318c590 0x5613b359c3e0 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:57.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.278+0000 7f420224e740 1 --2- 172.21.15.38:0/2214642612 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5613b31843b0 0x5613b357f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:57.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.278+0000 7f420224e740 1 --2- 172.21.15.38:0/2214642612 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5613b31ad270 0x5613b357efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:57.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.278+0000 7f420224e740 1 -- 172.21.15.38:0/2214642612 >> 172.21.15.38:0/2214642612 conn(0x5613b31a13c0 msgr2=0x5613b3555260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:57.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.278+0000 7f420224e740 1 -- 172.21.15.38:0/2214642612 shutdown_connections 2024-09-21T03:40:57.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.278+0000 7f420224e740 1 -- 172.21.15.38:0/2214642612 wait complete. 2024-09-21T03:40:57.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.279+0000 7f420224e740 1 -- start start 2024-09-21T03:40:57.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.279+0000 7f420224e740 1 Processor -- start 2024-09-21T03:40:57.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.281+0000 7f420224e740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5613b31843b0 0x5613b35a91d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:57.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.281+0000 7f420224e740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5613b318c590 0x5613b359a250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:57.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.281+0000 7f420224e740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5613b31ad270 0x5613b355e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:57.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.281+0000 7f420224e740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5613b3227620 con 0x5613b31843b0 2024-09-21T03:40:57.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.281+0000 7f420224e740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5613b345c5f0 con 0x5613b31ad270 2024-09-21T03:40:57.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.281+0000 7f420224e740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x5613b357fa20 con 0x5613b318c590 2024-09-21T03:40:57.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.281+0000 7f420224e740 10 client.0 _ll_register_callbacks cb 0x5613b34600a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:40:57.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.281+0000 7f41f8fa3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5613b31843b0 0x5613b35a91d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:57.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.281+0000 7f41f8fa3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5613b31843b0 0x5613b35a91d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:39156/0 (socket says 172.21.15.38:39156) 2024-09-21T03:40:57.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.281+0000 7f41f8fa3640 1 -- 172.21.15.38:0/4028463114 learned_addr learned my addr 172.21.15.38:0/4028463114 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:57.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.281+0000 7f41f3fff640 1 --2- 172.21.15.38:0/4028463114 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5613b318c590 0x5613b359a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:57.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.281+0000 7f41f97a4640 1 --2- 172.21.15.38:0/4028463114 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5613b31ad270 0x5613b355e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:57.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.281+0000 7f41f8fa3640 1 -- 172.21.15.38:0/4028463114 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5613b318c590 msgr2=0x5613b359a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:57.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.281+0000 7f41f8fa3640 1 --2- 172.21.15.38:0/4028463114 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5613b318c590 0x5613b359a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:57.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.281+0000 7f41f8fa3640 1 -- 172.21.15.38:0/4028463114 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5613b31ad270 msgr2=0x5613b355e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:57.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.281+0000 7f41f8fa3640 1 --2- 172.21.15.38:0/4028463114 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5613b31ad270 0x5613b355e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:57.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.281+0000 7f41f8fa3640 1 -- 172.21.15.38:0/4028463114 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5613b328f6b0 con 0x5613b31843b0 2024-09-21T03:40:57.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.281+0000 7f41f8fa3640 1 --2- 172.21.15.38:0/4028463114 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5613b31843b0 0x5613b35a91d0 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f41e4009610 tx=0x7f41e40625a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51d79964fbd260be server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:57.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.281+0000 7f41d37fe640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-21T03:40:57.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.282+0000 7f41d37fe640 1 -- 172.21.15.38:0/4028463114 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41e4020030 con 0x5613b31843b0 2024-09-21T03:40:57.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.282+0000 7f41d37fe640 1 -- 172.21.15.38:0/4028463114 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 6820+0+0 (secure 0 0 0) 0x7f41e409b0b0 con 0x5613b31843b0 2024-09-21T03:40:57.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.282+0000 7f420224e740 1 -- 172.21.15.38:0/4028463114 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f41ec005020 con 0x5613b31843b0 2024-09-21T03:40:57.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.282+0000 7f41d37fe640 1 -- 172.21.15.38:0/4028463114 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=124) ==== 24+0+0 (secure 0 0 0) 0x7f41e40675b0 con 0x5613b31843b0 2024-09-21T03:40:57.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.282+0000 7f420224e740 10 client.8079 fetch_fsmap learned FSMap version 124 2024-09-21T03:40:57.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.282+0000 7f420224e740 1 -- 172.21.15.38:0/4028463114 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=124,monmap=2+}) -- 0x5613b35ad410 con 0x5613b31843b0 2024-09-21T03:40:57.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.282+0000 7f41d37fe640 1 -- 172.21.15.38:0/4028463114 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f41ec005020 con 0x5613b31843b0 2024-09-21T03:40:57.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.282+0000 7f41d37fe640 1 -- 172.21.15.38:0/4028463114 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 124) ==== 3502+0+0 (secure 0 0 0) 0x7f41e4067100 con 0x5613b31843b0 2024-09-21T03:40:57.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.282+0000 7f420224e740 10 client.8079 fetch_fsmap finished waiting for FSMap version 124 2024-09-21T03:40:57.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.282+0000 7f420224e740 10 client.8079 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T03:40:57.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.282+0000 7f420224e740 20 client.8079 populate_metadata read hostname 'smithi038' 2024-09-21T03:40:57.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.282+0000 7f420224e740 1 --2- 172.21.15.38:0/4028463114 >> [v2:172.21.15.38:6832/3710736169,v1:172.21.15.38:6833/3710736169] conn(0x5613b316be40 0x5613b3590b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:57.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.282+0000 7f420224e740 4 client.8079 mds_command: new command op to 7995 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:40:57.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.282+0000 7f420224e740 1 -- 172.21.15.38:0/4028463114 --> [v2:172.21.15.38:6832/3710736169,v1:172.21.15.38:6833/3710736169] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5613b34c9770 con 0x5613b316be40 2024-09-21T03:40:57.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.282+0000 7f41f3fff640 1 --2- 172.21.15.38:0/4028463114 >> [v2:172.21.15.38:6832/3710736169,v1:172.21.15.38:6833/3710736169] conn(0x5613b316be40 0x5613b3590b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:57.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.283+0000 7f41f3fff640 1 --2- 172.21.15.38:0/4028463114 >> [v2:172.21.15.38:6832/3710736169,v1:172.21.15.38:6833/3710736169] conn(0x5613b316be40 0x5613b3590b90 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=faa3652f90066175 server_cookie=56f2fc14f93cfcdb in_seq=0 out_seq=0 2024-09-21T03:40:57.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.283+0000 7f41d37fe640 10 client.8079 ms_handle_connect on v2:172.21.15.38:6832/3710736169 2024-09-21T03:40:57.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.283+0000 7f41d37fe640 1 -- 172.21.15.38:0/4028463114 <== mds.0 v2:172.21.15.38:6832/3710736169 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5613b34c9770 con 0x5613b316be40 2024-09-21T03:40:57.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.283+0000 7f41d37fe640 10 client.8079 handle_command_reply: tid=0 2024-09-21T03:40:57.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.283+0000 7f420224e740 1 client.8079 shutdown 2024-09-21T03:40:57.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.284+0000 7f420224e740 1 -- 172.21.15.38:0/4028463114 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5613b31843b0 msgr2=0x5613b35a91d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:57.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.284+0000 7f420224e740 1 --2- 172.21.15.38:0/4028463114 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5613b31843b0 0x5613b35a91d0 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f41e4009610 tx=0x7f41e40625a0 comp rx=0 tx=0).stop 2024-09-21T03:40:57.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.284+0000 7f420224e740 1 -- 172.21.15.38:0/4028463114 shutdown_connections 2024-09-21T03:40:57.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.284+0000 7f420224e740 1 --2- 172.21.15.38:0/4028463114 >> [v2:172.21.15.38:6832/3710736169,v1:172.21.15.38:6833/3710736169] conn(0x5613b316be40 0x5613b3590b90 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:57.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.284+0000 7f420224e740 1 --2- 172.21.15.38:0/4028463114 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5613b31ad270 0x5613b355e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:57.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.284+0000 7f420224e740 1 --2- 172.21.15.38:0/4028463114 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5613b318c590 0x5613b359a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:57.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.284+0000 7f41d37fe640 0 client.8079 ms_handle_reset on v2:172.21.15.38:6832/3710736169 2024-09-21T03:40:57.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.284+0000 7f420224e740 1 --2- 172.21.15.38:0/4028463114 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5613b31843b0 0x5613b35a91d0 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:57.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.284+0000 7f420224e740 1 -- 172.21.15.38:0/4028463114 >> 172.21.15.38:0/4028463114 conn(0x5613b31a13c0 msgr2=0x5613b354f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:57.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.284+0000 7f420224e740 1 -- 172.21.15.38:0/4028463114 shutdown_connections 2024-09-21T03:40:57.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.284+0000 7f420224e740 1 -- 172.21.15.38:0/4028463114 wait complete. 2024-09-21T03:40:57.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.284+0000 7f420224e740 20 client.8079 trim_cache size 0 max 16384 2024-09-21T03:40:57.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.302+0000 7f420224e740 1 Processor -- start 2024-09-21T03:40:57.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.302+0000 7f420224e740 1 -- start start 2024-09-21T03:40:57.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.304+0000 7f420224e740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5613b31843b0 0x5613b3594350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:57.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.304+0000 7f420224e740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5613b318c590 0x5613b35948b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:57.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.304+0000 7f420224e740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5613b31ad270 0x5613b35a6290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:57.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.304+0000 7f420224e740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f41f4201d10 con 0x5613b318c590 2024-09-21T03:40:57.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.304+0000 7f420224e740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f41f4201020 con 0x5613b31ad270 2024-09-21T03:40:57.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.304+0000 7f420224e740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f41f4201640 con 0x5613b31843b0 2024-09-21T03:40:57.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.304+0000 7f41f3fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5613b318c590 0x5613b35948b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:57.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.304+0000 7f41f3fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5613b318c590 0x5613b35948b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:39164/0 (socket says 172.21.15.38:39164) 2024-09-21T03:40:57.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.304+0000 7f41f3fff640 1 -- 172.21.15.38:0/1917957006 learned_addr learned my addr 172.21.15.38:0/1917957006 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:57.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.304+0000 7f41f97a4640 1 --2- 172.21.15.38:0/1917957006 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5613b31ad270 0x5613b35a6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:57.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.304+0000 7f41f8fa3640 1 --2- 172.21.15.38:0/1917957006 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5613b31843b0 0x5613b3594350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:57.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.304+0000 7f41f3fff640 1 -- 172.21.15.38:0/1917957006 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5613b31843b0 msgr2=0x5613b3594350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:57.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.304+0000 7f41f3fff640 1 --2- 172.21.15.38:0/1917957006 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5613b31843b0 0x5613b3594350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:57.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.304+0000 7f41f3fff640 1 -- 172.21.15.38:0/1917957006 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5613b31ad270 msgr2=0x5613b35a6290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:57.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.304+0000 7f41f3fff640 1 --2- 172.21.15.38:0/1917957006 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5613b31ad270 0x5613b35a6290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:57.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.304+0000 7f41f3fff640 1 -- 172.21.15.38:0/1917957006 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5613b3591ff0 con 0x5613b318c590 2024-09-21T03:40:57.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.304+0000 7f41f97a4640 1 --2- 172.21.15.38:0/1917957006 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5613b31ad270 0x5613b35a6290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:40:57.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.304+0000 7f41f3fff640 1 --2- 172.21.15.38:0/1917957006 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5613b318c590 0x5613b35948b0 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f41e80099a0 tx=0x7f41e803e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55f55d97e7d334b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:57.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.304+0000 7f41d9ffb640 1 -- 172.21.15.38:0/1917957006 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41e8004230 con 0x5613b318c590 2024-09-21T03:40:57.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.304+0000 7f41d9ffb640 1 -- 172.21.15.38:0/1917957006 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f41e80043d0 con 0x5613b318c590 2024-09-21T03:40:57.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.304+0000 7f41d9ffb640 1 -- 172.21.15.38:0/1917957006 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41e8003850 con 0x5613b318c590 2024-09-21T03:40:57.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.305+0000 7f420224e740 1 -- 172.21.15.38:0/1917957006 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5613b318c590 msgr2=0x5613b35948b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:57.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.305+0000 7f420224e740 1 --2- 172.21.15.38:0/1917957006 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5613b318c590 0x5613b35948b0 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f41e80099a0 tx=0x7f41e803e040 comp rx=0 tx=0).stop 2024-09-21T03:40:57.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.305+0000 7f420224e740 1 -- 172.21.15.38:0/1917957006 shutdown_connections 2024-09-21T03:40:57.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.305+0000 7f420224e740 1 --2- 172.21.15.38:0/1917957006 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5613b31ad270 0x5613b35a6290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:57.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.305+0000 7f420224e740 1 --2- 172.21.15.38:0/1917957006 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5613b318c590 0x5613b35948b0 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:57.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.305+0000 7f420224e740 1 --2- 172.21.15.38:0/1917957006 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5613b31843b0 0x5613b3594350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:57.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.305+0000 7f420224e740 1 -- 172.21.15.38:0/1917957006 >> 172.21.15.38:0/1917957006 conn(0x5613b31a13c0 msgr2=0x5613b3596440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:57.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.305+0000 7f420224e740 1 -- 172.21.15.38:0/1917957006 shutdown_connections 2024-09-21T03:40:57.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.305+0000 7f420224e740 1 -- 172.21.15.38:0/1917957006 wait complete. 2024-09-21T03:40:57.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.306+0000 7f420224e740 1 -- start start 2024-09-21T03:40:57.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.306+0000 7f420224e740 1 Processor -- start 2024-09-21T03:40:57.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.308+0000 7f420224e740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5613b31843b0 0x5613b3616680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:57.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.308+0000 7f420224e740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5613b318c590 0x5613b356d190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:57.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.308+0000 7f420224e740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5613b31ad270 0x5613b356d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:57.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.308+0000 7f420224e740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5613b35ac900 con 0x5613b31843b0 2024-09-21T03:40:57.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.308+0000 7f420224e740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5613b35a9810 con 0x5613b31ad270 2024-09-21T03:40:57.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.308+0000 7f420224e740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x5613b357e670 con 0x5613b318c590 2024-09-21T03:40:57.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.308+0000 7f420224e740 10 client.0 _ll_register_callbacks cb 0x7f41e40629e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:40:57.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.308+0000 7f41f8fa3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5613b31843b0 0x5613b3616680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:57.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.308+0000 7f41f8fa3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5613b31843b0 0x5613b3616680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:39170/0 (socket says 172.21.15.38:39170) 2024-09-21T03:40:57.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.308+0000 7f41f8fa3640 1 -- 172.21.15.38:0/398603081 learned_addr learned my addr 172.21.15.38:0/398603081 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:57.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.308+0000 7f41f3fff640 1 --2- 172.21.15.38:0/398603081 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5613b318c590 0x5613b356d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:57.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.308+0000 7f41f97a4640 1 --2- 172.21.15.38:0/398603081 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5613b31ad270 0x5613b356d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:57.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.308+0000 7f41f8fa3640 1 -- 172.21.15.38:0/398603081 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5613b318c590 msgr2=0x5613b356d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:57.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.308+0000 7f41f8fa3640 1 --2- 172.21.15.38:0/398603081 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5613b318c590 0x5613b356d190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:57.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.308+0000 7f41f8fa3640 1 -- 172.21.15.38:0/398603081 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5613b31ad270 msgr2=0x5613b356d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:57.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.308+0000 7f41f8fa3640 1 --2- 172.21.15.38:0/398603081 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5613b31ad270 0x5613b356d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:57.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.308+0000 7f41f8fa3640 1 -- 172.21.15.38:0/398603081 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5613b36192b0 con 0x5613b31843b0 2024-09-21T03:40:57.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.308+0000 7f41f97a4640 1 --2- 172.21.15.38:0/398603081 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5613b31ad270 0x5613b356d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:40:57.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.308+0000 7f41f3fff640 1 --2- 172.21.15.38:0/398603081 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5613b318c590 0x5613b356d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:40:57.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.308+0000 7f41f8fa3640 1 --2- 172.21.15.38:0/398603081 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5613b31843b0 0x5613b3616680 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f41e4062880 tx=0x7f41e4005bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77671a01461e0ba8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:57.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.308+0000 7f41d37fe640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-21T03:40:57.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.308+0000 7f41d37fe640 1 -- 172.21.15.38:0/398603081 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41e4020070 con 0x5613b31843b0 2024-09-21T03:40:57.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.308+0000 7f41d37fe640 1 -- 172.21.15.38:0/398603081 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 6820+0+0 (secure 0 0 0) 0x7f41e409b0b0 con 0x5613b31843b0 2024-09-21T03:40:57.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.308+0000 7f420224e740 1 -- 172.21.15.38:0/398603081 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f41e80043d0 con 0x5613b31843b0 2024-09-21T03:40:57.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.308+0000 7f41d37fe640 1 -- 172.21.15.38:0/398603081 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=124) ==== 24+0+0 (secure 0 0 0) 0x7f41e406c4f0 con 0x5613b31843b0 2024-09-21T03:40:57.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.308+0000 7f420224e740 10 client.8085 fetch_fsmap learned FSMap version 124 2024-09-21T03:40:57.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.308+0000 7f420224e740 1 -- 172.21.15.38:0/398603081 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=124,monmap=2+}) -- 0x5613b36194c0 con 0x5613b31843b0 2024-09-21T03:40:57.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.309+0000 7f41d37fe640 1 -- 172.21.15.38:0/398603081 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f41e80043d0 con 0x5613b31843b0 2024-09-21T03:40:57.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.309+0000 7f41d37fe640 1 -- 172.21.15.38:0/398603081 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 124) ==== 3502+0+0 (secure 0 0 0) 0x7f41e405e4d0 con 0x5613b31843b0 2024-09-21T03:40:57.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.309+0000 7f420224e740 10 client.8085 fetch_fsmap finished waiting for FSMap version 124 2024-09-21T03:40:57.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.309+0000 7f420224e740 10 client.8085 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T03:40:57.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.309+0000 7f420224e740 20 client.8085 populate_metadata read hostname 'smithi038' 2024-09-21T03:40:57.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.309+0000 7f420224e740 1 --2- 172.21.15.38:0/398603081 >> [v2:172.21.15.38:6832/3710736169,v1:172.21.15.38:6833/3710736169] conn(0x5613b3599fc0 0x5613b361d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:57.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.309+0000 7f420224e740 4 client.8085 mds_command: new command op to 7995 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T03:40:57.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.309+0000 7f420224e740 1 -- 172.21.15.38:0/398603081 --> [v2:172.21.15.38:6832/3710736169,v1:172.21.15.38:6833/3710736169] -- command(tid 0: {"prefix": "status"}) -- 0x5613b34c5220 con 0x5613b3599fc0 2024-09-21T03:40:57.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.309+0000 7f41f3fff640 1 --2- 172.21.15.38:0/398603081 >> [v2:172.21.15.38:6832/3710736169,v1:172.21.15.38:6833/3710736169] conn(0x5613b3599fc0 0x5613b361d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:57.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.309+0000 7f41d37fe640 10 client.8085 ms_handle_connect on v2:172.21.15.38:6832/3710736169 2024-09-21T03:40:57.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.309+0000 7f41f3fff640 1 --2- 172.21.15.38:0/398603081 >> [v2:172.21.15.38:6832/3710736169,v1:172.21.15.38:6833/3710736169] conn(0x5613b3599fc0 0x5613b361d120 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1e92e962041f339 server_cookie=8b7b255a55b87647 in_seq=0 out_seq=0 2024-09-21T03:40:57.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.310+0000 7f41d37fe640 1 -- 172.21.15.38:0/398603081 <== mds.0 v2:172.21.15.38:6832/3710736169 1 ==== command_reply(tid 0: 0 ) ==== 8+0+328 (crc 0 0 0) 0x5613b34c5220 con 0x5613b3599fc0 2024-09-21T03:40:57.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.310+0000 7f41d37fe640 10 client.8085 handle_command_reply: tid=0 2024-09-21T03:40:57.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.310+0000 7f420224e740 1 client.8085 shutdown 2024-09-21T03:40:57.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.310+0000 7f420224e740 1 -- 172.21.15.38:0/398603081 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5613b31843b0 msgr2=0x5613b3616680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:57.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.310+0000 7f420224e740 1 --2- 172.21.15.38:0/398603081 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5613b31843b0 0x5613b3616680 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f41e4062880 tx=0x7f41e4005bb0 comp rx=0 tx=0).stop 2024-09-21T03:40:57.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.310+0000 7f420224e740 1 -- 172.21.15.38:0/398603081 shutdown_connections 2024-09-21T03:40:57.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.310+0000 7f420224e740 1 --2- 172.21.15.38:0/398603081 >> [v2:172.21.15.38:6832/3710736169,v1:172.21.15.38:6833/3710736169] conn(0x5613b3599fc0 0x5613b361d120 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:57.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.310+0000 7f420224e740 1 --2- 172.21.15.38:0/398603081 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5613b31ad270 0x5613b356d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:57.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.310+0000 7f420224e740 1 --2- 172.21.15.38:0/398603081 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5613b318c590 0x5613b356d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:57.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.310+0000 7f41d37fe640 0 client.8085 ms_handle_reset on v2:172.21.15.38:6832/3710736169 2024-09-21T03:40:57.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.310+0000 7f420224e740 1 --2- 172.21.15.38:0/398603081 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5613b31843b0 0x5613b3616680 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:57.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.310+0000 7f420224e740 1 -- 172.21.15.38:0/398603081 >> 172.21.15.38:0/398603081 conn(0x5613b31a13c0 msgr2=0x5613b35a6540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:57.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.310+0000 7f420224e740 1 -- 172.21.15.38:0/398603081 shutdown_connections 2024-09-21T03:40:57.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.310+0000 7f420224e740 1 -- 172.21.15.38:0/398603081 wait complete. 2024-09-21T03:40:57.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.310+0000 7f420224e740 20 client.8085 trim_cache size 0 max 16384 2024-09-21T03:40:57.319 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-09-21T03:40:57.319 INFO:teuthology.orchestra.run.smithi038.stdout: "cluster_fsid": "9d50d66f-656b-4fe2-8e3b-fe53b79c4d43", 2024-09-21T03:40:57.319 INFO:teuthology.orchestra.run.smithi038.stdout: "whoami": 0, 2024-09-21T03:40:57.319 INFO:teuthology.orchestra.run.smithi038.stdout: "id": 7995, 2024-09-21T03:40:57.319 INFO:teuthology.orchestra.run.smithi038.stdout: "want_state": "up:active", 2024-09-21T03:40:57.319 INFO:teuthology.orchestra.run.smithi038.stdout: "state": "up:active", 2024-09-21T03:40:57.319 INFO:teuthology.orchestra.run.smithi038.stdout: "fs_name": "cephfs", 2024-09-21T03:40:57.319 INFO:teuthology.orchestra.run.smithi038.stdout: "rank_uptime": 7.1771458130000001, 2024-09-21T03:40:57.319 INFO:teuthology.orchestra.run.smithi038.stdout: "mdsmap_epoch": 124, 2024-09-21T03:40:57.319 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch": 102, 2024-09-21T03:40:57.319 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch_barrier": 102, 2024-09-21T03:40:57.319 INFO:teuthology.orchestra.run.smithi038.stdout: "uptime": 10.454326149 2024-09-21T03:40:57.320 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-09-21T03:40:57.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.311+0000 7f41fb207640 1 -- 172.21.15.38:0/667036796 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f41e0078bc0 msgr2=0x7f41e007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:57.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.311+0000 7f41fb207640 1 --2- 172.21.15.38:0/667036796 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f41e0078bc0 0x7f41e007b0d0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f41e8034000 tx=0x7f41e8004730 comp rx=0 tx=0).stop 2024-09-21T03:40:57.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.311+0000 7f41fb207640 1 -- 172.21.15.38:0/667036796 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f41f40b9ad0 msgr2=0x7f41f40ca820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:57.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.311+0000 7f41fb207640 1 --2- 172.21.15.38:0/667036796 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f41f40b9ad0 0x7f41f40ca820 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f41e400d700 tx=0x7f41e400dbc0 comp rx=0 tx=0).stop 2024-09-21T03:40:57.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.311+0000 7f41fb207640 1 -- 172.21.15.38:0/667036796 shutdown_connections 2024-09-21T03:40:57.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.311+0000 7f41fb207640 1 --2- 172.21.15.38:0/667036796 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f41f415ee80 0x7f41f41f71e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:57.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.311+0000 7f41fb207640 1 --2- 172.21.15.38:0/667036796 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f41e0078bc0 0x7f41e007b0d0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:57.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.311+0000 7f41fb207640 1 --2- 172.21.15.38:0/667036796 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f41f40d1120 0x7f41f41f6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:57.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.311+0000 7f41fb207640 1 --2- 172.21.15.38:0/667036796 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f41f40b9ad0 0x7f41f40ca820 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:57.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.311+0000 7f41fb207640 1 -- 172.21.15.38:0/667036796 >> 172.21.15.38:0/667036796 conn(0x7f41f409d830 msgr2=0x7f41f40d09e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:57.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.311+0000 7f41fb207640 1 -- 172.21.15.38:0/667036796 shutdown_connections 2024-09-21T03:40:57.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.311+0000 7f41fb207640 1 -- 172.21.15.38:0/667036796 wait complete. 2024-09-21T03:40:57.325 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-09-21T03:40:57.325 INFO:tasks.cephfs.test_damage:Daemons came up after mutation 'Delete mds0_openfiles.0', proceeding to ls 2024-09-21T03:40:57.325 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T03:40:57.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.439+0000 7f622f80a640 1 -- 172.21.15.38:0/2242260527 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6214004800 msgr2=0x7f6214004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:57.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.439+0000 7f622f80a640 1 --2- 172.21.15.38:0/2242260527 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6214004800 0x7f6214004c10 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f6218009b50 tx=0x7f6214005830 comp rx=0 tx=0).stop 2024-09-21T03:40:57.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.439+0000 7f622f80a640 1 -- 172.21.15.38:0/2242260527 shutdown_connections 2024-09-21T03:40:57.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.439+0000 7f622f80a640 1 --2- 172.21.15.38:0/2242260527 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6214004800 0x7f6214004c10 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:57.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.439+0000 7f622f80a640 1 -- 172.21.15.38:0/2242260527 >> 172.21.15.38:0/2242260527 conn(0x7f62280b6b20 msgr2=0x7f62280d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:57.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.439+0000 7f622f80a640 1 -- 172.21.15.38:0/2242260527 shutdown_connections 2024-09-21T03:40:57.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.439+0000 7f622f80a640 1 -- 172.21.15.38:0/2242260527 wait complete. 2024-09-21T03:40:57.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.441+0000 7f622f80a640 1 Processor -- start 2024-09-21T03:40:57.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.441+0000 7f622f80a640 1 -- start start 2024-09-21T03:40:57.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.442+0000 7f622f80a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6214004800 0x7f622815a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:57.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.442+0000 7f622f80a640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6228155f00 0x7f6228153850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:57.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.442+0000 7f622f80a640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6228160680 0x7f6228160a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:57.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.442+0000 7f622f80a640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f6228162810 con 0x7f6214004800 2024-09-21T03:40:57.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.442+0000 7f622f80a640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f6228162690 con 0x7f6228155f00 2024-09-21T03:40:57.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.442+0000 7f622f80a640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f6228162990 con 0x7f6228160680 2024-09-21T03:40:57.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.443+0000 7f622d5a6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6214004800 0x7f622815a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:57.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.443+0000 7f622d5a6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6214004800 0x7f622815a810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:39194/0 (socket says 172.21.15.38:39194) 2024-09-21T03:40:57.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.443+0000 7f622d5a6640 1 -- 172.21.15.38:0/1416915616 learned_addr learned my addr 172.21.15.38:0/1416915616 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:57.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.443+0000 7f622cda5640 1 --2- 172.21.15.38:0/1416915616 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6228155f00 0x7f6228153850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:57.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.443+0000 7f622dda7640 1 --2- 172.21.15.38:0/1416915616 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6228160680 0x7f6228160a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:57.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.443+0000 7f622d5a6640 1 -- 172.21.15.38:0/1416915616 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6228160680 msgr2=0x7f6228160a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:57.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.443+0000 7f622d5a6640 1 --2- 172.21.15.38:0/1416915616 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6228160680 0x7f6228160a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:57.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.443+0000 7f622d5a6640 1 -- 172.21.15.38:0/1416915616 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6228155f00 msgr2=0x7f6228153850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:57.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.443+0000 7f622d5a6640 1 --2- 172.21.15.38:0/1416915616 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6228155f00 0x7f6228153850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:57.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.443+0000 7f622d5a6640 1 -- 172.21.15.38:0/1416915616 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6228160fd0 con 0x7f6214004800 2024-09-21T03:40:57.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.443+0000 7f622dda7640 1 --2- 172.21.15.38:0/1416915616 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6228160680 0x7f6228160a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:40:57.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.443+0000 7f622d5a6640 1 --2- 172.21.15.38:0/1416915616 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6214004800 0x7f622815a810 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f6224006690 tx=0x7f62240016f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:57.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.443+0000 7f621dffb640 1 -- 172.21.15.38:0/1416915616 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6224015070 con 0x7f6214004800 2024-09-21T03:40:57.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.443+0000 7f621dffb640 1 -- 172.21.15.38:0/1416915616 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6224011900 con 0x7f6214004800 2024-09-21T03:40:57.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.443+0000 7f622f80a640 1 -- 172.21.15.38:0/1416915616 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62281f8de0 con 0x7f6214004800 2024-09-21T03:40:57.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.443+0000 7f621dffb640 1 -- 172.21.15.38:0/1416915616 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6224011d30 con 0x7f6214004800 2024-09-21T03:40:57.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.444+0000 7f622f80a640 1 -- 172.21.15.38:0/1416915616 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62281f93a0 con 0x7f6214004800 2024-09-21T03:40:57.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.444+0000 7f622f80a640 1 -- 172.21.15.38:0/1416915616 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62280d2850 con 0x7f6214004800 2024-09-21T03:40:57.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.445+0000 7f621dffb640 1 -- 172.21.15.38:0/1416915616 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f6224018070 con 0x7f6214004800 2024-09-21T03:40:57.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.446+0000 7f621dffb640 1 --2- 172.21.15.38:0/1416915616 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f620c078bc0 0x7f620c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:57.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.446+0000 7f621dffb640 1 -- 172.21.15.38:0/1416915616 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 6820+0+0 (secure 0 0 0) 0x7f622409bd70 con 0x7f6214004800 2024-09-21T03:40:57.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.446+0000 7f622cda5640 1 --2- 172.21.15.38:0/1416915616 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f620c078bc0 0x7f620c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:57.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.447+0000 7f622cda5640 1 --2- 172.21.15.38:0/1416915616 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f620c078bc0 0x7f620c07b0d0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f6218035000 tx=0x7f62180046c0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:57.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.451+0000 7f621dffb640 1 -- 172.21.15.38:0/1416915616 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f6228160fd0 con 0x7f6214004800 2024-09-21T03:40:57.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.675+0000 7f622f80a640 1 -- 172.21.15.38:0/1416915616 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f62280b8d30 con 0x7f6214004800 2024-09-21T03:40:57.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.675+0000 7f621dffb640 1 -- 172.21.15.38:0/1416915616 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v124) ==== 53+0+73 (secure 0 0 0) 0x7f62240636f0 con 0x7f6214004800 2024-09-21T03:40:57.679 INFO:teuthology.orchestra.run.smithi038.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T03:40:57.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.677+0000 7f622f80a640 1 -- 172.21.15.38:0/1416915616 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f620c078bc0 msgr2=0x7f620c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:57.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.677+0000 7f622f80a640 1 --2- 172.21.15.38:0/1416915616 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f620c078bc0 0x7f620c07b0d0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f6218035000 tx=0x7f62180046c0 comp rx=0 tx=0).stop 2024-09-21T03:40:57.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.677+0000 7f622f80a640 1 -- 172.21.15.38:0/1416915616 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6214004800 msgr2=0x7f622815a810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:57.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.677+0000 7f622f80a640 1 --2- 172.21.15.38:0/1416915616 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6214004800 0x7f622815a810 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f6224006690 tx=0x7f62240016f0 comp rx=0 tx=0).stop 2024-09-21T03:40:57.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.677+0000 7f622f80a640 1 -- 172.21.15.38:0/1416915616 shutdown_connections 2024-09-21T03:40:57.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.677+0000 7f622f80a640 1 --2- 172.21.15.38:0/1416915616 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f620c078bc0 0x7f620c07b0d0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:57.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.677+0000 7f622f80a640 1 --2- 172.21.15.38:0/1416915616 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6228160680 0x7f6228160a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:57.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.678+0000 7f622f80a640 1 --2- 172.21.15.38:0/1416915616 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6228155f00 0x7f6228153850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:57.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.678+0000 7f622f80a640 1 --2- 172.21.15.38:0/1416915616 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6214004800 0x7f622815a810 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:57.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.678+0000 7f622f80a640 1 -- 172.21.15.38:0/1416915616 >> 172.21.15.38:0/1416915616 conn(0x7f62280b6b20 msgr2=0x7f62280b4540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:57.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.678+0000 7f622f80a640 1 -- 172.21.15.38:0/1416915616 shutdown_connections 2024-09-21T03:40:57.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57.678+0000 7f622f80a640 1 -- 172.21.15.38:0/1416915616 wait complete. 2024-09-21T03:40:57.692 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-21T03:40:57.692 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T03:40:57.692 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi038.front.sepia.ceph.com 2024-09-21T03:40:57.692 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-21T03:40:57.693 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T03:40:57.693 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-21T03:40:57.693 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-21T03:40:57.693 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T03:40:57.693 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T03:40:57.693 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:40:57.693 DEBUG:teuthology.orchestra.run.smithi038:> ip addr 2024-09-21T03:40:57.709 INFO:teuthology.orchestra.run.smithi038.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T03:40:57.710 INFO:teuthology.orchestra.run.smithi038.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T03:40:57.710 INFO:teuthology.orchestra.run.smithi038.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T03:40:57.710 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:40:57.710 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 ::1/128 scope host 2024-09-21T03:40:57.710 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:40:57.710 INFO:teuthology.orchestra.run.smithi038.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T03:40:57.710 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:6b:fd:a2 brd ff:ff:ff:ff:ff:ff 2024-09-21T03:40:57.710 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp7s0f0 2024-09-21T03:40:57.710 INFO:teuthology.orchestra.run.smithi038.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T03:40:57.711 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:6b:fd:a3 brd ff:ff:ff:ff:ff:ff 2024-09-21T03:40:57.711 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp7s0f1 2024-09-21T03:40:57.711 INFO:teuthology.orchestra.run.smithi038.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T03:40:57.711 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:bd:15:fc brd ff:ff:ff:ff:ff:ff 2024-09-21T03:40:57.711 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp1s0f0 2024-09-21T03:40:57.711 INFO:teuthology.orchestra.run.smithi038.stdout: inet 172.21.15.38/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-21T03:40:57.711 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft 42351sec preferred_lft 42351sec 2024-09-21T03:40:57.711 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 fe80::ec4:7aff:febd:15fc/64 scope link 2024-09-21T03:40:57.711 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:40:57.711 INFO:teuthology.orchestra.run.smithi038.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T03:40:57.711 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:bd:15:fd brd ff:ff:ff:ff:ff:ff 2024-09-21T03:40:57.712 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp1s0f1 2024-09-21T03:40:57.712 INFO:teuthology.orchestra.run.smithi038.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T03:40:57.712 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 92:55:c7:97:6f:b5 brd ff:ff:ff:ff:ff:ff 2024-09-21T03:40:57.712 INFO:teuthology.orchestra.run.smithi038.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-21T03:40:57.712 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:40:57.712 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 fe80::c49f:c1ff:fe21:b12a/64 scope link 2024-09-21T03:40:57.712 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:40:57.712 INFO:teuthology.orchestra.run.smithi038.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T03:40:57.712 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether a6:3a:83:ad:db:d5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T03:40:57.712 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 fe80::a43a:83ff:fead:dbd5/64 scope link 2024-09-21T03:40:57.712 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:40:57.712 INFO:teuthology.orchestra.run.smithi038.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T03:40:57.712 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 92:55:c7:97:6f:b5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T03:40:57.712 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 fe80::9055:c7ff:fe97:6fb5/64 scope link 2024-09-21T03:40:57.713 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:40:57.713 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:40:57.713 DEBUG:teuthology.orchestra.run.smithi038:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T03:40:57.774 INFO:teuthology.orchestra.run.smithi038.stdout:1 2024-09-21T03:40:57.775 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:40:57.775 DEBUG:teuthology.orchestra.run.smithi038:> route 2024-09-21T03:40:57.832 INFO:teuthology.orchestra.run.smithi038.stdout:Kernel IP routing table 2024-09-21T03:40:57.832 INFO:teuthology.orchestra.run.smithi038.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T03:40:57.833 INFO:teuthology.orchestra.run.smithi038.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-21T03:40:57.833 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-21T03:40:57.833 INFO:teuthology.orchestra.run.smithi038.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T03:40:57.833 DEBUG:teuthology.orchestra.run.smithi038:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T03:40:57.833 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-09-21T03:40:57.833 DEBUG:teuthology.orchestra.run.smithi038:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-21T03:40:57.833 DEBUG:teuthology.orchestra.run.smithi038:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-21T03:40:57.833 DEBUG:teuthology.orchestra.run.smithi038:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-21T03:40:57.833 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-09-21T03:40:57.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T03:40:57.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:40:57.962 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:40:57.963 DEBUG:teuthology.orchestra.run.smithi038:> ip netns list 2024-09-21T03:40:57.978 INFO:teuthology.orchestra.run.smithi038.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-21T03:40:57.978 INFO:teuthology.orchestra.run.smithi038.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-21T03:40:57.979 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T03:40:57.979 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:40:57.979 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip link set brx.0 up 2024-09-21T03:40:58.044 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T03:40:58.045 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-21T03:40:58.101 INFO:teuthology.orchestra.run.smithi038.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-21T03:40:58.101 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T03:40:58.101 DEBUG:teuthology.orchestra.run.smithi038:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-21T03:40:58.156 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-21T03:40:58.157 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:40:58.157 DEBUG:teuthology.orchestra.run.smithi038:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T03:40:58.252 INFO:teuthology.orchestra.run.smithi038.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T03:40:58.253 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T03:40:58.253 INFO:teuthology.orchestra.run.smithi038.stdout:invalid new device string format 2024-09-21T03:40:58.263 INFO:teuthology.orchestra.run.smithi038.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T03:40:58.264 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T03:40:58.264 INFO:teuthology.orchestra.run.smithi038.stdout:invalid new device string format 2024-09-21T03:40:58.264 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: resolved to: "172.21.15.38:3300,172.21.15.195:3300,172.21.15.195:3301" 2024-09-21T03:40:58.264 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T03:40:58.270 INFO:teuthology.orchestra.run.smithi038.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T03:40:58.270 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T03:40:58.270 INFO:teuthology.orchestra.run.smithi038.stdout:invalid new device string format 2024-09-21T03:40:58.270 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: resolved to: "172.21.15.38:3300,172.21.15.195:3300,172.21.15.195:3301" 2024-09-21T03:40:58.270 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T03:40:58.270 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: resolved to: "172.21.15.38:6789,172.21.15.195:6789,172.21.15.195:6790" 2024-09-21T03:40:58.270 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.38:6789,172.21.15.195:6789,172.21.15.195:6790:/ 2024-09-21T03:40:58.270 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=9d50d66f-656b-4fe2-8e3b-fe53b79c4d43" will pass to kernel 2024-09-21T03:40:58.271 INFO:teuthology.orchestra.run.smithi038.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-21T03:40:58.271 INFO:teuthology.orchestra.run.smithi038.stdout: You just mounted a file system that supports labels which does not 2024-09-21T03:40:58.271 INFO:teuthology.orchestra.run.smithi038.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-21T03:40:58.271 INFO:teuthology.orchestra.run.smithi038.stdout: applications will generate AVC messages and not be allowed access to 2024-09-21T03:40:58.271 INFO:teuthology.orchestra.run.smithi038.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-21T03:40:58.272 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T03:40:58.272 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:40:58.273 DEBUG:teuthology.orchestra.run.smithi038:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-21T03:40:58.298 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T03:40:58.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:58 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T03:40:58.390 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:40:58.395 INFO:teuthology.orchestra.run.smithi038.stdout:client7417 2024-09-21T03:40:58.396 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-21T03:40:58.396 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:40:58.412 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:40:58.412 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls -R .) 2024-09-21T03:40:58.455 INFO:teuthology.orchestra.run:waiting for 20 2024-09-21T03:40:58.470 INFO:teuthology.orchestra.run.smithi038.stdout:.: 2024-09-21T03:40:58.470 INFO:teuthology.orchestra.run.smithi038.stdout:subdir 2024-09-21T03:40:58.470 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:40:58.471 INFO:teuthology.orchestra.run.smithi038.stdout:./subdir: 2024-09-21T03:40:58.471 INFO:teuthology.orchestra.run.smithi038.stdout:sixmegs 2024-09-21T03:40:58.471 INFO:tasks.cephfs.test_damage:Result: Mutation 'Delete mds0_openfiles.0' did not caused DAMAGED state 2024-09-21T03:40:58.471 INFO:tasks.cephfs.test_damage:Applying mutation 'Delete 500.00000000' 2024-09-21T03:40:58.471 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-21T03:40:58.471 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:40:58.527 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-21T03:40:58.527 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:40:58.585 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T03:40:58.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:58.739+0000 7f24e33ad640 1 -- 172.21.15.38:0/509926742 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f24dc15ee80 msgr2=0x7f24dc0d0170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:58.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:58.739+0000 7f24e33ad640 1 --2- 172.21.15.38:0/509926742 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f24dc15ee80 0x7f24dc0d0170 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7f24dc0d06b0 tx=0x7f24d802a4d0 comp rx=0 tx=0).stop 2024-09-21T03:40:58.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:58.739+0000 7f24e33ad640 1 -- 172.21.15.38:0/509926742 shutdown_connections 2024-09-21T03:40:58.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:58.739+0000 7f24e33ad640 1 --2- 172.21.15.38:0/509926742 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f24dc0d1120 0x7f24dc0ceb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:58.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:58.739+0000 7f24e33ad640 1 --2- 172.21.15.38:0/509926742 >> v2:172.21.15.195:3300/0 conn(0x7f24dc0b9ad0 0x7f24dc0d0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:58.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:58.739+0000 7f24e33ad640 1 --2- 172.21.15.38:0/509926742 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f24dc15ee80 0x7f24dc0d0170 unknown :-1 s=CLOSED pgs=1374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:58.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:58.739+0000 7f24e33ad640 1 -- 172.21.15.38:0/509926742 >> 172.21.15.38:0/509926742 conn(0x7f24dc09d830 msgr2=0x7f24dc0b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:58.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:58.739+0000 7f24e33ad640 1 -- 172.21.15.38:0/509926742 shutdown_connections 2024-09-21T03:40:58.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:58.739+0000 7f24e33ad640 1 -- 172.21.15.38:0/509926742 wait complete. 2024-09-21T03:40:58.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:58.741+0000 7f24e33ad640 1 Processor -- start 2024-09-21T03:40:58.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:58.741+0000 7f24e33ad640 1 -- start start 2024-09-21T03:40:58.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:58.742+0000 7f24e33ad640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f24dc0b9ad0 0x7f24dc0cc340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:58.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:58.742+0000 7f24e33ad640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f24dc0d1120 0x7f24dc156b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:58.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:58.742+0000 7f24e33ad640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f24dc15ee80 0x7f24dc1578a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:58.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:58.742+0000 7f24e33ad640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f24dc0cf520 con 0x7f24dc15ee80 2024-09-21T03:40:58.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:58.742+0000 7f24e33ad640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f24dc0cf3a0 con 0x7f24dc0d1120 2024-09-21T03:40:58.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:58.742+0000 7f24e33ad640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f24dc0b6580 con 0x7f24dc0b9ad0 2024-09-21T03:40:58.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:58.742+0000 7f24e194a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f24dc15ee80 0x7f24dc1578a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:58.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:58.743+0000 7f24e194a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f24dc15ee80 0x7f24dc1578a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:39216/0 (socket says 172.21.15.38:39216) 2024-09-21T03:40:58.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:58.743+0000 7f24e194a640 1 -- 172.21.15.38:0/3449500990 learned_addr learned my addr 172.21.15.38:0/3449500990 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:58.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:58.743+0000 7f24e1149640 1 --2- 172.21.15.38:0/3449500990 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f24dc0b9ad0 0x7f24dc0cc340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:58.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:58.743+0000 7f24e0948640 1 --2- 172.21.15.38:0/3449500990 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f24dc0d1120 0x7f24dc156b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:58.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:58.743+0000 7f24e194a640 1 -- 172.21.15.38:0/3449500990 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f24dc0b9ad0 msgr2=0x7f24dc0cc340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:58.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:58.743+0000 7f24e194a640 1 --2- 172.21.15.38:0/3449500990 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f24dc0b9ad0 0x7f24dc0cc340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:58.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:58.743+0000 7f24e194a640 1 -- 172.21.15.38:0/3449500990 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f24dc0d1120 msgr2=0x7f24dc156b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:58.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:58.743+0000 7f24e194a640 1 --2- 172.21.15.38:0/3449500990 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f24dc0d1120 0x7f24dc156b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:58.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:58.743+0000 7f24e194a640 1 -- 172.21.15.38:0/3449500990 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24dc0cc880 con 0x7f24dc15ee80 2024-09-21T03:40:58.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:58.743+0000 7f24e194a640 1 --2- 172.21.15.38:0/3449500990 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f24dc15ee80 0x7f24dc1578a0 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f24d400e960 tx=0x7f24d400ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:58.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:58.743+0000 7f24d1ffb640 1 -- 172.21.15.38:0/3449500990 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24d4004d20 con 0x7f24dc15ee80 2024-09-21T03:40:58.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:58.743+0000 7f24d1ffb640 1 -- 172.21.15.38:0/3449500990 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f24d400cb20 con 0x7f24dc15ee80 2024-09-21T03:40:58.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:58.743+0000 7f24e33ad640 1 -- 172.21.15.38:0/3449500990 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24dc1f6630 con 0x7f24dc15ee80 2024-09-21T03:40:58.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:58.743+0000 7f24d1ffb640 1 -- 172.21.15.38:0/3449500990 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24d4005020 con 0x7f24dc15ee80 2024-09-21T03:40:58.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:58.744+0000 7f24e33ad640 1 -- 172.21.15.38:0/3449500990 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24dc0cbb40 con 0x7f24dc15ee80 2024-09-21T03:40:58.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:58.745+0000 7f24e33ad640 1 -- 172.21.15.38:0/3449500990 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24dc0ceb90 con 0x7f24dc15ee80 2024-09-21T03:40:58.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:58.745+0000 7f24d1ffb640 1 -- 172.21.15.38:0/3449500990 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f24d40051c0 con 0x7f24dc15ee80 2024-09-21T03:40:58.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:58.746+0000 7f24d1ffb640 1 --2- 172.21.15.38:0/3449500990 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f24c0078b90 0x7f24c007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:58.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:58.746+0000 7f24d1ffb640 1 -- 172.21.15.38:0/3449500990 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 6820+0+0 (secure 0 0 0) 0x7f24d409ba20 con 0x7f24dc15ee80 2024-09-21T03:40:58.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:58.746+0000 7f24e1149640 1 --2- 172.21.15.38:0/3449500990 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f24c0078b90 0x7f24c007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:58.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:58.746+0000 7f24e1149640 1 --2- 172.21.15.38:0/3449500990 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f24c0078b90 0x7f24c007b0a0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f24d8033000 tx=0x7f24d8004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:58.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:58.750+0000 7f24d1ffb640 1 -- 172.21.15.38:0/3449500990 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f24d4016030 con 0x7f24dc15ee80 2024-09-21T03:40:58.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:58.952+0000 7f24e33ad640 1 -- 172.21.15.38:0/3449500990 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f24dc158f90 con 0x7f24dc15ee80 2024-09-21T03:40:58.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:58.953+0000 7f24d1ffb640 1 -- 172.21.15.38:0/3449500990 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 12 entries v102) ==== 81+0+728 (secure 0 0 0) 0x7f24d40633b0 con 0x7f24dc15ee80 2024-09-21T03:40:58.956 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6835/2751962827 2024-09-22T03:40:38.029580+0000 2024-09-21T03:40:58.956 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6835/425672974 2024-09-22T03:40:46.105521+0000 2024-09-21T03:40:58.956 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6834/425672974 2024-09-22T03:40:46.105521+0000 2024-09-21T03:40:58.956 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6834/2751962827 2024-09-22T03:40:38.029580+0000 2024-09-21T03:40:58.956 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6834/3864487171 2024-09-22T03:40:31.989837+0000 2024-09-21T03:40:58.956 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6835/3864487171 2024-09-22T03:40:31.989837+0000 2024-09-21T03:40:58.956 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6834/2934569248 2024-09-22T03:40:26.934591+0000 2024-09-21T03:40:58.956 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6833/751618334 2024-09-22T03:40:42.053279+0000 2024-09-21T03:40:58.957 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6835/2934569248 2024-09-22T03:40:26.934591+0000 2024-09-21T03:40:58.957 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.195:6837/712370966 2024-09-22T03:40:11.633787+0000 2024-09-21T03:40:58.957 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6832/751618334 2024-09-22T03:40:42.053279+0000 2024-09-21T03:40:58.957 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.195:6835/712370966 2024-09-22T03:40:11.633787+0000 2024-09-21T03:40:58.957 INFO:teuthology.orchestra.run.smithi038.stderr:listed 12 entries 2024-09-21T03:40:58.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:58.955+0000 7f24e33ad640 1 -- 172.21.15.38:0/3449500990 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f24c0078b90 msgr2=0x7f24c007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:58.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:58.955+0000 7f24e33ad640 1 --2- 172.21.15.38:0/3449500990 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f24c0078b90 0x7f24c007b0a0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f24d8033000 tx=0x7f24d8004730 comp rx=0 tx=0).stop 2024-09-21T03:40:58.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:58.955+0000 7f24e33ad640 1 -- 172.21.15.38:0/3449500990 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f24dc15ee80 msgr2=0x7f24dc1578a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:58.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:58.955+0000 7f24e33ad640 1 --2- 172.21.15.38:0/3449500990 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f24dc15ee80 0x7f24dc1578a0 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f24d400e960 tx=0x7f24d400ee20 comp rx=0 tx=0).stop 2024-09-21T03:40:58.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:58.955+0000 7f24e33ad640 1 -- 172.21.15.38:0/3449500990 shutdown_connections 2024-09-21T03:40:58.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:58.955+0000 7f24e33ad640 1 --2- 172.21.15.38:0/3449500990 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f24dc15ee80 0x7f24dc1578a0 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:58.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:58.955+0000 7f24e33ad640 1 --2- 172.21.15.38:0/3449500990 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f24dc0d1120 0x7f24dc156b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:58.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:58.955+0000 7f24e33ad640 1 --2- 172.21.15.38:0/3449500990 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f24c0078b90 0x7f24c007b0a0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:58.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:58.955+0000 7f24e33ad640 1 --2- 172.21.15.38:0/3449500990 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f24dc0b9ad0 0x7f24dc0cc340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:58.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:58.955+0000 7f24e33ad640 1 -- 172.21.15.38:0/3449500990 >> 172.21.15.38:0/3449500990 conn(0x7f24dc09d830 msgr2=0x7f24dc160b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:58.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:58.955+0000 7f24e33ad640 1 -- 172.21.15.38:0/3449500990 shutdown_connections 2024-09-21T03:40:58.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:58.955+0000 7f24e33ad640 1 -- 172.21.15.38:0/3449500990 wait complete. 2024-09-21T03:40:58.970 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-21T03:40:58.970 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:40:58.970 DEBUG:teuthology.orchestra.run.smithi038:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-09-21T03:40:59.004 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi038.front.sepia.ceph.com 2024-09-21T03:40:59.004 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:40:59.004 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T03:40:59.059 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-21T03:40:59.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:59.214+0000 7f08fca69640 1 -- 172.21.15.38:0/455507485 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f08f80ceb90 msgr2=0x7f08f80cefa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:59.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:59.214+0000 7f08fca69640 1 --2- 172.21.15.38:0/455507485 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f08f80ceb90 0x7f08f80cefa0 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f08f80cfba0 tx=0x7f08f002a8f0 comp rx=0 tx=0).stop 2024-09-21T03:40:59.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:59.215+0000 7f08fca69640 1 -- 172.21.15.38:0/455507485 shutdown_connections 2024-09-21T03:40:59.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:59.215+0000 7f08fca69640 1 --2- 172.21.15.38:0/455507485 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f08f80ceb90 0x7f08f80cefa0 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:59.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:59.215+0000 7f08fca69640 1 --2- 172.21.15.38:0/455507485 >> v2:172.21.15.195:3300/0 conn(0x7f08f815ee80 0x7f08f80d0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:59.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:59.215+0000 7f08fca69640 1 -- 172.21.15.38:0/455507485 >> 172.21.15.38:0/455507485 conn(0x7f08f809d830 msgr2=0x7f08f80b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:59.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:59.215+0000 7f08fca69640 1 -- 172.21.15.38:0/455507485 shutdown_connections 2024-09-21T03:40:59.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:59.215+0000 7f08fca69640 1 -- 172.21.15.38:0/455507485 wait complete. 2024-09-21T03:40:59.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:59.217+0000 7f08fca69640 1 Processor -- start 2024-09-21T03:40:59.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:59.217+0000 7f08fca69640 1 -- start start 2024-09-21T03:40:59.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:59.218+0000 7f08fca69640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f08f80b9ad0 0x7f08f80ccb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:59.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:59.218+0000 7f08fca69640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f08f80ceb90 0x7f08f814b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:59.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:59.218+0000 7f08fca69640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f08f815ee80 0x7f08f80b2450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:59.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:59.218+0000 7f08fca69640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f08f80b6700 con 0x7f08f80b9ad0 2024-09-21T03:40:59.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:59.218+0000 7f08fca69640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f08f80b6580 con 0x7f08f815ee80 2024-09-21T03:40:59.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:59.218+0000 7f08fca69640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f08f80b6880 con 0x7f08f80ceb90 2024-09-21T03:40:59.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:59.218+0000 7f08f659c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f08f80b9ad0 0x7f08f80ccb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:59.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:59.218+0000 7f08f659c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f08f80b9ad0 0x7f08f80ccb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:39234/0 (socket says 172.21.15.38:39234) 2024-09-21T03:40:59.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:59.218+0000 7f08f659c640 1 -- 172.21.15.38:0/354820923 learned_addr learned my addr 172.21.15.38:0/354820923 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:59.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:59.218+0000 7f08f6d9d640 1 --2- 172.21.15.38:0/354820923 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f08f815ee80 0x7f08f80b2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:59.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:59.218+0000 7f08f5d9b640 1 --2- 172.21.15.38:0/354820923 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f08f80ceb90 0x7f08f814b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:59.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:59.218+0000 7f08f659c640 1 -- 172.21.15.38:0/354820923 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f08f80ceb90 msgr2=0x7f08f814b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:59.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:59.218+0000 7f08f659c640 1 --2- 172.21.15.38:0/354820923 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f08f80ceb90 0x7f08f814b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:59.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:59.218+0000 7f08f659c640 1 -- 172.21.15.38:0/354820923 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f08f815ee80 msgr2=0x7f08f80b2450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:59.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:59.218+0000 7f08f659c640 1 --2- 172.21.15.38:0/354820923 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f08f815ee80 0x7f08f80b2450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:59.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:59.218+0000 7f08f659c640 1 -- 172.21.15.38:0/354820923 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08f814c530 con 0x7f08f80b9ad0 2024-09-21T03:40:59.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:59.218+0000 7f08f5d9b640 1 --2- 172.21.15.38:0/354820923 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f08f80ceb90 0x7f08f814b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:40:59.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:59.219+0000 7f08f659c640 1 --2- 172.21.15.38:0/354820923 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f08f80b9ad0 0x7f08f80ccb60 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f08e800c9a0 tx=0x7f08e800ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:59.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:59.219+0000 7f08e6ffd640 1 -- 172.21.15.38:0/354820923 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08e8007bb0 con 0x7f08f80b9ad0 2024-09-21T03:40:59.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:59.219+0000 7f08e6ffd640 1 -- 172.21.15.38:0/354820923 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f08e8007e40 con 0x7f08f80b9ad0 2024-09-21T03:40:59.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:59.219+0000 7f08fca69640 1 -- 172.21.15.38:0/354820923 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08f81f8d70 con 0x7f08f80b9ad0 2024-09-21T03:40:59.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:59.219+0000 7f08e6ffd640 1 -- 172.21.15.38:0/354820923 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08e800f690 con 0x7f08f80b9ad0 2024-09-21T03:40:59.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:59.219+0000 7f08fca69640 1 -- 172.21.15.38:0/354820923 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08f814be20 con 0x7f08f80b9ad0 2024-09-21T03:40:59.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:59.220+0000 7f08fca69640 1 -- 172.21.15.38:0/354820923 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08f80b6df0 con 0x7f08f80b9ad0 2024-09-21T03:40:59.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:59.221+0000 7f08e6ffd640 1 -- 172.21.15.38:0/354820923 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f08e800f830 con 0x7f08f80b9ad0 2024-09-21T03:40:59.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:59.221+0000 7f08e6ffd640 1 --2- 172.21.15.38:0/354820923 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f08d8078b90 0x7f08d807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:59.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:59.222+0000 7f08e6ffd640 1 -- 172.21.15.38:0/354820923 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 6820+0+0 (secure 0 0 0) 0x7f08e809b880 con 0x7f08f80b9ad0 2024-09-21T03:40:59.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:59.222+0000 7f08f5d9b640 1 --2- 172.21.15.38:0/354820923 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f08d8078b90 0x7f08d807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:59.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:59.222+0000 7f08f5d9b640 1 --2- 172.21.15.38:0/354820923 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f08d8078b90 0x7f08d807b0a0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f08f81f8ba0 tx=0x7f08ec008040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:59.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:59.226+0000 7f08e6ffd640 1 -- 172.21.15.38:0/354820923 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f08e8014030 con 0x7f08f80b9ad0 2024-09-21T03:40:59.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:59.452+0000 7f08fca69640 1 -- 172.21.15.38:0/354820923 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f08f8161520 con 0x7f08f80b9ad0 2024-09-21T03:40:59.455 INFO:tasks.ceph.mon.a.smithi038.stderr:2024-09-21T03:40:59.452+0000 7f3f45713640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T03:40:59.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:59.455+0000 7f08e6ffd640 1 -- 172.21.15.38:0/354820923 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v125) ==== 161+0+0 (secure 0 0 0) 0x7f08e8063980 con 0x7f08f80b9ad0 2024-09-21T03:40:59.458 INFO:teuthology.orchestra.run.smithi038.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-21T03:40:59.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:59.458+0000 7f08fca69640 1 -- 172.21.15.38:0/354820923 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f08d8078b90 msgr2=0x7f08d807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:59.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:59.458+0000 7f08fca69640 1 --2- 172.21.15.38:0/354820923 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f08d8078b90 0x7f08d807b0a0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f08f81f8ba0 tx=0x7f08ec008040 comp rx=0 tx=0).stop 2024-09-21T03:40:59.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:59.458+0000 7f08fca69640 1 -- 172.21.15.38:0/354820923 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f08f80b9ad0 msgr2=0x7f08f80ccb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:59.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:59.458+0000 7f08fca69640 1 --2- 172.21.15.38:0/354820923 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f08f80b9ad0 0x7f08f80ccb60 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f08e800c9a0 tx=0x7f08e800ce60 comp rx=0 tx=0).stop 2024-09-21T03:40:59.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:59.458+0000 7f08fca69640 1 -- 172.21.15.38:0/354820923 shutdown_connections 2024-09-21T03:40:59.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:59.458+0000 7f08fca69640 1 --2- 172.21.15.38:0/354820923 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f08f815ee80 0x7f08f80b2450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:59.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:59.458+0000 7f08fca69640 1 --2- 172.21.15.38:0/354820923 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f08d8078b90 0x7f08d807b0a0 secure :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f08f81f8ba0 tx=0x7f08ec008040 comp rx=0 tx=0).stop 2024-09-21T03:40:59.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:59.458+0000 7f08fca69640 1 --2- 172.21.15.38:0/354820923 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f08f80ceb90 0x7f08f814b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:59.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:59.458+0000 7f08fca69640 1 --2- 172.21.15.38:0/354820923 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f08f80b9ad0 0x7f08f80ccb60 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:59.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:59.458+0000 7f08fca69640 1 -- 172.21.15.38:0/354820923 >> 172.21.15.38:0/354820923 conn(0x7f08f809d830 msgr2=0x7f08f815e050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:59.462 INFO:tasks.ceph.mds.c.smithi038.stderr: -1906> 2024-09-21T03:40:50.137+0000 7f413f25b640 -1 mds.0.openfiles _load_finish got (2) No such file or directory 2024-09-21T03:40:59.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:59.459+0000 7f08fca69640 1 -- 172.21.15.38:0/354820923 shutdown_connections 2024-09-21T03:40:59.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:59.459+0000 7f08fca69640 1 -- 172.21.15.38:0/354820923 wait complete. 2024-09-21T03:40:59.463 INFO:tasks.ceph.mds.c.smithi038.stderr: -604> 2024-09-21T03:40:57.116+0000 7f4143263640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T03:40:59.473 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-09-21T03:40:59.518 INFO:tasks.ceph.mds.c.smithi038.stdout:starting mds.c at 2024-09-21T03:40:59.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:59.591+0000 7f114e69b640 1 -- 172.21.15.38:0/638129058 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1130004020 msgr2=0x7f1130004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:59.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:59.591+0000 7f114e69b640 1 --2- 172.21.15.38:0/638129058 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1130004020 0x7f1130004800 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f113c00c3f0 tx=0x7f1130005830 comp rx=0 tx=0).stop 2024-09-21T03:40:59.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:59.591+0000 7f114cc38640 1 -- 172.21.15.38:0/638129058 reap_dead start 2024-09-21T03:40:59.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:59.591+0000 7f114e69b640 1 -- 172.21.15.38:0/638129058 shutdown_connections 2024-09-21T03:40:59.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:59.591+0000 7f114e69b640 1 -- 172.21.15.38:0/638129058 >> 172.21.15.38:0/638129058 conn(0x7f114809d830 msgr2=0x7f11480b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:59.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:59.591+0000 7f114e69b640 1 -- 172.21.15.38:0/638129058 shutdown_connections 2024-09-21T03:40:59.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:59.591+0000 7f114e69b640 1 -- 172.21.15.38:0/638129058 wait complete. 2024-09-21T03:40:59.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:59.593+0000 7f114e69b640 1 Processor -- start 2024-09-21T03:40:59.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:59.593+0000 7f114e69b640 1 -- start start 2024-09-21T03:40:59.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:59.594+0000 7f114e69b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1130004020 0x7f11481594b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:59.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:59.594+0000 7f114e69b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f11481f92d0 0x7f11481f96e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:59.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:59.595+0000 7f114e69b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f114815bbc0 0x7f114815bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:59.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:59.595+0000 7f114e69b640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f11480d02f0 con 0x7f1130004020 2024-09-21T03:40:59.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:59.595+0000 7f114e69b640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f11480d0170 con 0x7f11481f92d0 2024-09-21T03:40:59.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:59.595+0000 7f114e69b640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f11480d0470 con 0x7f114815bbc0 2024-09-21T03:40:59.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:59.595+0000 7f1147fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1130004020 0x7f11481594b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:59.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:59.595+0000 7f1147fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1130004020 0x7f11481594b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:39274/0 (socket says 172.21.15.38:39274) 2024-09-21T03:40:59.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:59.595+0000 7f1147fff640 1 -- 172.21.15.38:0/2492035605 learned_addr learned my addr 172.21.15.38:0/2492035605 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:40:59.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:59.595+0000 7f11477fe640 1 --2- 172.21.15.38:0/2492035605 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f11481f92d0 0x7f11481f96e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:59.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:59.595+0000 7f114cc38640 1 --2- 172.21.15.38:0/2492035605 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f114815bbc0 0x7f114815bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:59.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:59.595+0000 7f1147fff640 1 -- 172.21.15.38:0/2492035605 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f114815bbc0 msgr2=0x7f114815bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:59.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:59.595+0000 7f1147fff640 1 --2- 172.21.15.38:0/2492035605 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f114815bbc0 0x7f114815bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:59.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:59.595+0000 7f1147fff640 1 -- 172.21.15.38:0/2492035605 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f11481f92d0 msgr2=0x7f11481f96e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:59.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:59.595+0000 7f1147fff640 1 --2- 172.21.15.38:0/2492035605 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f11481f92d0 0x7f11481f96e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:59.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:59.595+0000 7f1147fff640 1 -- 172.21.15.38:0/2492035605 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f114806d8f0 con 0x7f1130004020 2024-09-21T03:40:59.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:59.595+0000 7f1147fff640 1 --2- 172.21.15.38:0/2492035605 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1130004020 0x7f11481594b0 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f1138011a50 tx=0x7f1138011f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:59.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:59.595+0000 7f1144ff9640 1 -- 172.21.15.38:0/2492035605 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f113801f070 con 0x7f1130004020 2024-09-21T03:40:59.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:59.595+0000 7f1144ff9640 1 -- 172.21.15.38:0/2492035605 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f113800fe80 con 0x7f1130004020 2024-09-21T03:40:59.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:59.595+0000 7f114e69b640 1 -- 172.21.15.38:0/2492035605 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f114815c510 con 0x7f1130004020 2024-09-21T03:40:59.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:59.595+0000 7f1144ff9640 1 -- 172.21.15.38:0/2492035605 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1138012690 con 0x7f1130004020 2024-09-21T03:40:59.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:59.595+0000 7f114e69b640 1 -- 172.21.15.38:0/2492035605 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f114806e0e0 con 0x7f1130004020 2024-09-21T03:40:59.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:59.597+0000 7f114e69b640 1 -- 172.21.15.38:0/2492035605 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11480cefa0 con 0x7f1130004020 2024-09-21T03:40:59.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:59.597+0000 7f1144ff9640 1 -- 172.21.15.38:0/2492035605 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f1138022070 con 0x7f1130004020 2024-09-21T03:40:59.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:59.598+0000 7f1144ff9640 1 --2- 172.21.15.38:0/2492035605 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1134078b90 0x7f113407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:59.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:59.598+0000 7f1144ff9640 1 -- 172.21.15.38:0/2492035605 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 6906+0+0 (secure 0 0 0) 0x7f113809bc80 con 0x7f1130004020 2024-09-21T03:40:59.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:59.598+0000 7f11477fe640 1 --2- 172.21.15.38:0/2492035605 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1134078b90 0x7f113407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:59.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:59.599+0000 7f11477fe640 1 --2- 172.21.15.38:0/2492035605 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1134078b90 0x7f113407b0a0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f113c033000 tx=0x7f113c02d070 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:59.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:59.602+0000 7f1144ff9640 1 -- 172.21.15.38:0/2492035605 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f1138063650 con 0x7f1130004020 2024-09-21T03:40:59.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:40:59.867+0000 7f114e69b640 1 -- 172.21.15.38:0/2492035605 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7f11480b31a0 con 0x7f1130004020 2024-09-21T03:41:00.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.457+0000 7f1144ff9640 1 -- 172.21.15.38:0/2492035605 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 nothing to do: rank is not damaged v126) ==== 107+0+0 (secure 0 0 0) 0x7f1138068080 con 0x7f1130004020 2024-09-21T03:41:00.460 INFO:teuthology.orchestra.run.smithi038.stderr:nothing to do: rank is not damaged 2024-09-21T03:41:00.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.460+0000 7f114e69b640 1 -- 172.21.15.38:0/2492035605 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1134078b90 msgr2=0x7f113407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:00.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.460+0000 7f114e69b640 1 --2- 172.21.15.38:0/2492035605 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1134078b90 0x7f113407b0a0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f113c033000 tx=0x7f113c02d070 comp rx=0 tx=0).stop 2024-09-21T03:41:00.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.460+0000 7f114e69b640 1 -- 172.21.15.38:0/2492035605 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1130004020 msgr2=0x7f11481594b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:00.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.460+0000 7f114e69b640 1 --2- 172.21.15.38:0/2492035605 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1130004020 0x7f11481594b0 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f1138011a50 tx=0x7f1138011f10 comp rx=0 tx=0).stop 2024-09-21T03:41:00.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.460+0000 7f114e69b640 1 -- 172.21.15.38:0/2492035605 shutdown_connections 2024-09-21T03:41:00.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.460+0000 7f114e69b640 1 --2- 172.21.15.38:0/2492035605 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1134078b90 0x7f113407b0a0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:00.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.460+0000 7f114e69b640 1 --2- 172.21.15.38:0/2492035605 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f114815bbc0 0x7f114815bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:00.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.460+0000 7f114e69b640 1 --2- 172.21.15.38:0/2492035605 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f11481f92d0 0x7f11481f96e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:00.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.460+0000 7f114e69b640 1 --2- 172.21.15.38:0/2492035605 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1130004020 0x7f11481594b0 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:00.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.460+0000 7f114e69b640 1 -- 172.21.15.38:0/2492035605 >> 172.21.15.38:0/2492035605 conn(0x7f114809d830 msgr2=0x7f114815e290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:00.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.460+0000 7f114e69b640 1 -- 172.21.15.38:0/2492035605 shutdown_connections 2024-09-21T03:41:00.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.460+0000 7f114e69b640 1 -- 172.21.15.38:0/2492035605 wait complete. 2024-09-21T03:41:00.478 DEBUG:teuthology.orchestra.run.smithi038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-09-21T03:41:00.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.536+0000 7f828e54af40 1 Processor -- start 2024-09-21T03:41:00.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.536+0000 7f828e54af40 1 -- start start 2024-09-21T03:41:00.541 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.537+0000 7f828e54af40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f82700030f0 0x558423a970f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:00.541 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.537+0000 7f828e54af40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558423a99450 0x5584239c3d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:00.541 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.537+0000 7f828e54af40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x558423a7aae0 0x558423a7aef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:00.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.537+0000 7f828e54af40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x558423971430 con 0x7f82700030f0 2024-09-21T03:41:00.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.537+0000 7f828e54af40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55842397e1f0 con 0x558423a99450 2024-09-21T03:41:00.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.537+0000 7f828e54af40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x558423912d60 con 0x558423a7aae0 2024-09-21T03:41:00.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.538+0000 7f8288004640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f82700030f0 0x558423a970f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:00.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.538+0000 7f8288004640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f82700030f0 0x558423a970f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:39286/0 (socket says 172.21.15.38:39286) 2024-09-21T03:41:00.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.538+0000 7f8288004640 1 -- 172.21.15.38:0/3927289501 learned_addr learned my addr 172.21.15.38:0/3927289501 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:41:00.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.538+0000 7f8287803640 1 --2- 172.21.15.38:0/3927289501 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558423a99450 0x5584239c3d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:00.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.538+0000 7f8288805640 1 --2- 172.21.15.38:0/3927289501 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x558423a7aae0 0x558423a7aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:00.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.538+0000 7f8288004640 1 -- 172.21.15.38:0/3927289501 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x558423a7aae0 msgr2=0x558423a7aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:00.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.538+0000 7f8288004640 1 --2- 172.21.15.38:0/3927289501 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x558423a7aae0 0x558423a7aef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:00.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.538+0000 7f8288004640 1 -- 172.21.15.38:0/3927289501 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558423a99450 msgr2=0x5584239c3d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:00.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.538+0000 7f8288004640 1 --2- 172.21.15.38:0/3927289501 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558423a99450 0x5584239c3d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:00.543 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.538+0000 7f8288004640 1 -- 172.21.15.38:0/3927289501 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558423a7b8a0 con 0x7f82700030f0 2024-09-21T03:41:00.543 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.538+0000 7f8287803640 1 --2- 172.21.15.38:0/3927289501 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558423a99450 0x5584239c3d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:41:00.543 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.538+0000 7f8288805640 1 --2- 172.21.15.38:0/3927289501 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x558423a7aae0 0x558423a7aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:41:00.543 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.538+0000 7f8288004640 1 --2- 172.21.15.38:0/3927289501 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f82700030f0 0x558423a970f0 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f827800d540 tx=0x7f8278001f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af0112da22f7c1b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:00.543 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.538+0000 7f8286801640 1 -- 172.21.15.38:0/3927289501 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f827800f4f0 con 0x7f82700030f0 2024-09-21T03:41:00.543 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.538+0000 7f8286801640 1 -- 172.21.15.38:0/3927289501 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8278014020 con 0x7f82700030f0 2024-09-21T03:41:00.543 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.538+0000 7f8286801640 1 -- 172.21.15.38:0/3927289501 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f827800f910 con 0x7f82700030f0 2024-09-21T03:41:00.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.539+0000 7f828e54af40 1 -- 172.21.15.38:0/3927289501 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f82700030f0 msgr2=0x558423a970f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:00.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.539+0000 7f828e54af40 1 --2- 172.21.15.38:0/3927289501 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f82700030f0 0x558423a970f0 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f827800d540 tx=0x7f8278001f90 comp rx=0 tx=0).stop 2024-09-21T03:41:00.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.539+0000 7f828e54af40 1 -- 172.21.15.38:0/3927289501 shutdown_connections 2024-09-21T03:41:00.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.539+0000 7f828e54af40 1 --2- 172.21.15.38:0/3927289501 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x558423a7aae0 0x558423a7aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:00.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.539+0000 7f828e54af40 1 --2- 172.21.15.38:0/3927289501 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558423a99450 0x5584239c3d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:00.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.539+0000 7f828e54af40 1 --2- 172.21.15.38:0/3927289501 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f82700030f0 0x558423a970f0 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:00.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.539+0000 7f828e54af40 1 -- 172.21.15.38:0/3927289501 >> 172.21.15.38:0/3927289501 conn(0x5584238e3ac0 msgr2=0x558423a79a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:00.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.539+0000 7f828e54af40 1 -- 172.21.15.38:0/3927289501 shutdown_connections 2024-09-21T03:41:00.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.539+0000 7f828e54af40 1 -- 172.21.15.38:0/3927289501 wait complete. 2024-09-21T03:41:00.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.540+0000 7f828e54af40 1 Processor -- start 2024-09-21T03:41:00.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.540+0000 7f828e54af40 1 -- start start 2024-09-21T03:41:00.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.541+0000 7f828e54af40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x558423a7aae0 0x558423a95320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:00.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.541+0000 7f828e54af40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558423a99450 0x558423a92ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:00.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.541+0000 7f828e54af40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f82700030f0 0x558423b32ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:00.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.541+0000 7f828e54af40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5584239c2b90 con 0x558423a99450 2024-09-21T03:41:00.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.541+0000 7f828e54af40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x558423a9a4d0 con 0x7f82700030f0 2024-09-21T03:41:00.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.541+0000 7f828e54af40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x558423a8eae0 con 0x558423a7aae0 2024-09-21T03:41:00.547 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.542+0000 7f8287803640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558423a99450 0x558423a92ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:00.547 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.542+0000 7f8287803640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558423a99450 0x558423a92ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:39298/0 (socket says 172.21.15.38:39298) 2024-09-21T03:41:00.547 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.542+0000 7f8287803640 1 -- 172.21.15.38:0/1792112991 learned_addr learned my addr 172.21.15.38:0/1792112991 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:41:00.547 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.542+0000 7f8288805640 1 --2- 172.21.15.38:0/1792112991 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f82700030f0 0x558423b32ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:00.547 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.542+0000 7f8288004640 1 --2- 172.21.15.38:0/1792112991 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x558423a7aae0 0x558423a95320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:00.547 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.542+0000 7f8287803640 1 -- 172.21.15.38:0/1792112991 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x558423a7aae0 msgr2=0x558423a95320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:00.547 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.542+0000 7f8287803640 1 --2- 172.21.15.38:0/1792112991 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x558423a7aae0 0x558423a95320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:00.547 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.542+0000 7f8287803640 1 -- 172.21.15.38:0/1792112991 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f82700030f0 msgr2=0x558423b32ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:00.547 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.542+0000 7f8287803640 1 --2- 172.21.15.38:0/1792112991 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f82700030f0 0x558423b32ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:00.547 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.542+0000 7f8287803640 1 -- 172.21.15.38:0/1792112991 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558423a94560 con 0x558423a99450 2024-09-21T03:41:00.547 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.542+0000 7f8287803640 1 --2- 172.21.15.38:0/1792112991 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558423a99450 0x558423a92ac0 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f827c046750 tx=0x7f827c001110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:00.547 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.542+0000 7f82857ff640 1 -- 172.21.15.38:0/1792112991 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f827c064020 con 0x558423a99450 2024-09-21T03:41:00.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.542+0000 7f82857ff640 1 -- 172.21.15.38:0/1792112991 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f827c04da70 con 0x558423a99450 2024-09-21T03:41:00.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.542+0000 7f828e54af40 1 -- 172.21.15.38:0/1792112991 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x558423a94840 con 0x558423a99450 2024-09-21T03:41:00.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.542+0000 7f82857ff640 1 -- 172.21.15.38:0/1792112991 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f827c060470 con 0x558423a99450 2024-09-21T03:41:00.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.542+0000 7f828e54af40 1 -- 172.21.15.38:0/1792112991 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558423a94e00 con 0x558423a99450 2024-09-21T03:41:00.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.544+0000 7f82857ff640 1 -- 172.21.15.38:0/1792112991 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f827c067020 con 0x558423a99450 2024-09-21T03:41:00.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.545+0000 7f82857ff640 1 --2- 172.21.15.38:0/1792112991 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f82740775a0 0x7f8274079ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:00.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.545+0000 7f82857ff640 1 -- 172.21.15.38:0/1792112991 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 6906+0+0 (secure 0 0 0) 0x7f827c0dd8c0 con 0x558423a99450 2024-09-21T03:41:00.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.545+0000 7f8288004640 1 --2- 172.21.15.38:0/1792112991 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f82740775a0 0x7f8274079ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:00.549 INFO:teuthology.orchestra.run.smithi038.stdout:Importing pool 2024-09-21T03:41:00.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.545+0000 7f828e54af40 1 --2- 172.21.15.38:0/1792112991 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x558423b350d0 0x558423b354e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:00.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.545+0000 7f828e54af40 1 -- 172.21.15.38:0/1792112991 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x558423a96870 con 0x558423b350d0 2024-09-21T03:41:00.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.545+0000 7f8288805640 1 --2- 172.21.15.38:0/1792112991 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x558423b350d0 0x558423b354e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:00.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.546+0000 7f8288004640 1 --2- 172.21.15.38:0/1792112991 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f82740775a0 0x7f8274079ab0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f82780011a0 tx=0x7f8278001110 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:00.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.546+0000 7f8288805640 1 --2- 172.21.15.38:0/1792112991 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x558423b350d0 0x558423b354e0 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:00.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.548+0000 7f8288805640 1 -- 172.21.15.38:0/1792112991 <== osd.7 v2:172.21.15.195:6818/1606106342 1 ==== osd_op_reply(1 601.00000000 [create] v103'195 uv169 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f8280003040 con 0x558423b350d0 2024-09-21T03:41:00.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.548+0000 7f828e54af40 1 -- 172.21.15.38:0/1792112991 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e103) -- 0x558423b35a20 con 0x558423b350d0 2024-09-21T03:41:00.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.549+0000 7f8288805640 1 -- 172.21.15.38:0/1792112991 <== osd.7 v2:172.21.15.195:6818/1606106342 2 ==== osd_op_reply(2 601.00000000 [delete] v103'196 uv196 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8280003040 con 0x558423b350d0 2024-09-21T03:41:00.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.549+0000 7f828e54af40 1 -- 172.21.15.38:0/1792112991 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x558423b3e260 con 0x558423b350d0 2024-09-21T03:41:00.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.551+0000 7f8288805640 1 -- 172.21.15.38:0/1792112991 <== osd.7 v2:172.21.15.195:6818/1606106342 3 ==== osd_op_reply(3 601.00000000 [create] v103'197 uv197 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8280003040 con 0x558423b350d0 2024-09-21T03:41:00.554 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-09-21T03:41:00.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.551+0000 7f828e54af40 1 -- 172.21.15.38:0/1792112991 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x558423b3f610 con 0x558423b350d0 2024-09-21T03:41:00.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.552+0000 7f8288805640 1 -- 172.21.15.38:0/1792112991 <== osd.7 v2:172.21.15.195:6818/1606106342 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v103'198 uv198 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8280003040 con 0x558423b350d0 2024-09-21T03:41:00.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.552+0000 7f828e54af40 1 -- 172.21.15.38:0/1792112991 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x558423b3f9b0 con 0x558423b350d0 2024-09-21T03:41:00.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.554+0000 7f8288805640 1 -- 172.21.15.38:0/1792112991 <== osd.7 v2:172.21.15.195:6818/1606106342 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v103'199 uv199 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8280003040 con 0x558423b350d0 2024-09-21T03:41:00.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.554+0000 7f828e54af40 1 -- 172.21.15.38:0/1792112991 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x558423b3e600 con 0x558423b350d0 2024-09-21T03:41:00.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.555+0000 7f8288805640 1 -- 172.21.15.38:0/1792112991 <== osd.7 v2:172.21.15.195:6818/1606106342 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v103'200 uv200 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8280003040 con 0x558423b350d0 2024-09-21T03:41:00.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.555+0000 7f828e54af40 1 -- 172.21.15.38:0/1792112991 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x558423b3e9a0 con 0x558423b350d0 2024-09-21T03:41:00.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.556+0000 7f8288805640 1 -- 172.21.15.38:0/1792112991 <== osd.7 v2:172.21.15.195:6818/1606106342 7 ==== osd_op_reply(7 602.00000000 [create] v103'201 uv175 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f8280003040 con 0x558423b350d0 2024-09-21T03:41:00.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.557+0000 7f828e54af40 1 -- 172.21.15.38:0/1792112991 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e103) -- 0x558423b3ed40 con 0x558423b350d0 2024-09-21T03:41:00.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.558+0000 7f8288805640 1 -- 172.21.15.38:0/1792112991 <== osd.7 v2:172.21.15.195:6818/1606106342 8 ==== osd_op_reply(8 602.00000000 [delete] v103'202 uv202 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8280003040 con 0x558423b350d0 2024-09-21T03:41:00.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.558+0000 7f828e54af40 1 -- 172.21.15.38:0/1792112991 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x558423b3ed40 con 0x558423b350d0 2024-09-21T03:41:00.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.559+0000 7f8288805640 1 -- 172.21.15.38:0/1792112991 <== osd.7 v2:172.21.15.195:6818/1606106342 9 ==== osd_op_reply(9 602.00000000 [create] v103'203 uv203 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8280003040 con 0x558423b350d0 2024-09-21T03:41:00.563 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-09-21T03:41:00.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.560+0000 7f828e54af40 1 -- 172.21.15.38:0/1792112991 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x558423b3ed40 con 0x558423b350d0 2024-09-21T03:41:00.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.562+0000 7f8288805640 1 -- 172.21.15.38:0/1792112991 <== osd.7 v2:172.21.15.195:6818/1606106342 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v103'204 uv204 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8280003040 con 0x558423b350d0 2024-09-21T03:41:00.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.562+0000 7f828e54af40 1 -- 172.21.15.38:0/1792112991 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x558423b3ed40 con 0x558423b350d0 2024-09-21T03:41:00.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.563+0000 7f8288805640 1 -- 172.21.15.38:0/1792112991 <== osd.7 v2:172.21.15.195:6818/1606106342 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v103'205 uv205 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8280003040 con 0x558423b350d0 2024-09-21T03:41:00.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.563+0000 7f828e54af40 1 -- 172.21.15.38:0/1792112991 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x558423b3ed40 con 0x558423b350d0 2024-09-21T03:41:00.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.564+0000 7f8288805640 1 -- 172.21.15.38:0/1792112991 <== osd.7 v2:172.21.15.195:6818/1606106342 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v103'206 uv206 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8280003040 con 0x558423b350d0 2024-09-21T03:41:00.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.564+0000 7f828e54af40 1 -- 172.21.15.38:0/1792112991 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:13 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x558423b3ed40 con 0x558423b350d0 2024-09-21T03:41:00.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.567+0000 7f8288805640 1 -- 172.21.15.38:0/1792112991 <== osd.7 v2:172.21.15.195:6818/1606106342 13 ==== osd_op_reply(13 10000000000.00000000 [create] v103'207 uv182 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7f8280003040 con 0x558423b350d0 2024-09-21T03:41:00.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.567+0000 7f828e54af40 1 -- 172.21.15.38:0/1792112991 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:14 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e103) -- 0x558423b3ed40 con 0x558423b350d0 2024-09-21T03:41:00.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.567+0000 7f8288805640 1 -- 172.21.15.38:0/1792112991 <== osd.7 v2:172.21.15.195:6818/1606106342 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v103'208 uv208 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f8280003040 con 0x558423b350d0 2024-09-21T03:41:00.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.567+0000 7f828e54af40 1 -- 172.21.15.38:0/1792112991 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:15 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x558423b3ed40 con 0x558423b350d0 2024-09-21T03:41:00.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.570+0000 7f8288805640 1 -- 172.21.15.38:0/1792112991 <== osd.7 v2:172.21.15.195:6818/1606106342 15 ==== osd_op_reply(15 10000000000.00000000 [create] v103'209 uv209 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f8280003040 con 0x558423b350d0 2024-09-21T03:41:00.572 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-09-21T03:41:00.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.570+0000 7f828e54af40 1 -- 172.21.15.38:0/1792112991 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:16 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x558423b3ed40 con 0x558423b350d0 2024-09-21T03:41:00.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.570+0000 7f8288805640 1 -- 172.21.15.38:0/1792112991 <== osd.7 v2:172.21.15.195:6818/1606106342 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v103'210 uv210 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f8280003040 con 0x558423b350d0 2024-09-21T03:41:00.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.570+0000 7f828e54af40 1 -- 172.21.15.38:0/1792112991 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:17 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x558423b3ed40 con 0x558423b350d0 2024-09-21T03:41:00.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.573+0000 7f8288805640 1 -- 172.21.15.38:0/1792112991 <== osd.7 v2:172.21.15.195:6818/1606106342 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v103'211 uv211 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f8280003040 con 0x558423b350d0 2024-09-21T03:41:00.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.573+0000 7f828e54af40 1 -- 172.21.15.38:0/1792112991 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:18 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x558423b3ed40 con 0x558423b350d0 2024-09-21T03:41:00.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.573+0000 7f8288805640 1 -- 172.21.15.38:0/1792112991 <== osd.7 v2:172.21.15.195:6818/1606106342 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v103'212 uv212 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f8280003040 con 0x558423b350d0 2024-09-21T03:41:00.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.573+0000 7f828e54af40 1 -- 172.21.15.38:0/1792112991 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:19 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=585b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x558423b3fd50 con 0x558423b350d0 2024-09-21T03:41:00.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.574+0000 7f8288805640 1 -- 172.21.15.38:0/1792112991 <== osd.7 v2:172.21.15.195:6818/1606106342 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v103'213 uv213 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f8280003040 con 0x558423b350d0 2024-09-21T03:41:00.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.575+0000 7f828e54af40 1 -- 172.21.15.38:0/1792112991 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:20 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x558423b3fd50 con 0x558423b350d0 2024-09-21T03:41:00.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.577+0000 7f8288805640 1 -- 172.21.15.38:0/1792112991 <== osd.7 v2:172.21.15.195:6818/1606106342 20 ==== osd_op_reply(20 600.00000000 [create] v103'214 uv188 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f8280003040 con 0x558423b350d0 2024-09-21T03:41:00.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.577+0000 7f828e54af40 1 -- 172.21.15.38:0/1792112991 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:21 13.10 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e103) -- 0x558423b3fd50 con 0x558423b350d0 2024-09-21T03:41:00.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.577+0000 7f8288805640 1 -- 172.21.15.38:0/1792112991 <== osd.7 v2:172.21.15.195:6818/1606106342 21 ==== osd_op_reply(21 600.00000000 [delete] v103'215 uv215 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8280003040 con 0x558423b350d0 2024-09-21T03:41:00.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.578+0000 7f828e54af40 1 -- 172.21.15.38:0/1792112991 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:22 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x558423b3fd50 con 0x558423b350d0 2024-09-21T03:41:00.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.579+0000 7f8288805640 1 -- 172.21.15.38:0/1792112991 <== osd.7 v2:172.21.15.195:6818/1606106342 22 ==== osd_op_reply(22 600.00000000 [create] v103'216 uv216 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8280003040 con 0x558423b350d0 2024-09-21T03:41:00.582 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-09-21T03:41:00.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.579+0000 7f828e54af40 1 -- 172.21.15.38:0/1792112991 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:23 13.10 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x558423b40d60 con 0x558423b350d0 2024-09-21T03:41:00.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.580+0000 7f8288805640 1 -- 172.21.15.38:0/1792112991 <== osd.7 v2:172.21.15.195:6818/1606106342 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v103'217 uv217 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8280003040 con 0x558423b350d0 2024-09-21T03:41:00.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.580+0000 7f828e54af40 1 -- 172.21.15.38:0/1792112991 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:24 13.10 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x558423b40d60 con 0x558423b350d0 2024-09-21T03:41:00.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.582+0000 7f8288805640 1 -- 172.21.15.38:0/1792112991 <== osd.7 v2:172.21.15.195:6818/1606106342 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v103'218 uv218 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8280003040 con 0x558423b350d0 2024-09-21T03:41:00.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.582+0000 7f828e54af40 1 -- 172.21.15.38:0/1792112991 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:25 13.10 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x558423b3fd50 con 0x558423b350d0 2024-09-21T03:41:00.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.583+0000 7f8288805640 1 -- 172.21.15.38:0/1792112991 <== osd.7 v2:172.21.15.195:6818/1606106342 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v103'219 uv219 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8280003040 con 0x558423b350d0 2024-09-21T03:41:00.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.583+0000 7f828e54af40 1 -- 172.21.15.38:0/1792112991 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:26 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x558423b3fd50 con 0x558423b350d0 2024-09-21T03:41:00.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.584+0000 7f8288805640 1 -- 172.21.15.38:0/1792112991 <== osd.7 v2:172.21.15.195:6818/1606106342 26 ==== osd_op_reply(26 603.00000000 [create] v103'220 uv194 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f8280003040 con 0x558423b350d0 2024-09-21T03:41:00.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.584+0000 7f828e54af40 1 -- 172.21.15.38:0/1792112991 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:27 13.10 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e103) -- 0x558423b3fd50 con 0x558423b350d0 2024-09-21T03:41:00.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.585+0000 7f8288805640 1 -- 172.21.15.38:0/1792112991 <== osd.7 v2:172.21.15.195:6818/1606106342 27 ==== osd_op_reply(27 603.00000000 [delete] v103'221 uv221 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8280003040 con 0x558423b350d0 2024-09-21T03:41:00.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.585+0000 7f828e54af40 1 -- 172.21.15.38:0/1792112991 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:28 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x558423b3fd50 con 0x558423b350d0 2024-09-21T03:41:00.589 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.587+0000 7f8288805640 1 -- 172.21.15.38:0/1792112991 <== osd.7 v2:172.21.15.195:6818/1606106342 28 ==== osd_op_reply(28 603.00000000 [create] v103'222 uv222 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8280003040 con 0x558423b350d0 2024-09-21T03:41:00.590 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-09-21T03:41:00.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.587+0000 7f828e54af40 1 -- 172.21.15.38:0/1792112991 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:29 13.10 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x558423b40d60 con 0x558423b350d0 2024-09-21T03:41:00.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.588+0000 7f8288805640 1 -- 172.21.15.38:0/1792112991 <== osd.7 v2:172.21.15.195:6818/1606106342 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v103'223 uv223 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8280003040 con 0x558423b350d0 2024-09-21T03:41:00.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.588+0000 7f828e54af40 1 -- 172.21.15.38:0/1792112991 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:30 13.10 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x558423b40d60 con 0x558423b350d0 2024-09-21T03:41:00.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.589+0000 7f8288805640 1 -- 172.21.15.38:0/1792112991 <== osd.7 v2:172.21.15.195:6818/1606106342 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v103'224 uv224 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8280003040 con 0x558423b350d0 2024-09-21T03:41:00.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.589+0000 7f828e54af40 1 -- 172.21.15.38:0/1792112991 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:31 13.10 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x558423b3fd50 con 0x558423b350d0 2024-09-21T03:41:00.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.591+0000 7f8288805640 1 -- 172.21.15.38:0/1792112991 <== osd.7 v2:172.21.15.195:6818/1606106342 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v103'225 uv225 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8280003040 con 0x558423b350d0 2024-09-21T03:41:00.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.591+0000 7f828e54af40 1 --2- 172.21.15.38:0/1792112991 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x558423b40260 0x558423b42680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:00.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.591+0000 7f828e54af40 1 -- 172.21.15.38:0/1792112991 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x558423b42bc0 con 0x558423b40260 2024-09-21T03:41:00.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.591+0000 7f8288004640 1 --2- 172.21.15.38:0/1792112991 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x558423b40260 0x558423b42680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:00.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.591+0000 7f8288004640 1 --2- 172.21.15.38:0/1792112991 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x558423b40260 0x558423b42680 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:00.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.593+0000 7f8288004640 1 -- 172.21.15.38:0/1792112991 <== osd.6 v2:172.21.15.195:6826/4055825675 1 ==== osd_op_reply(32 1.00000000.inode [create] v103'151 uv132 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f8278008070 con 0x558423b40260 2024-09-21T03:41:00.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.593+0000 7f828e54af40 1 -- 172.21.15.38:0/1792112991 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e103) -- 0x558423b42f60 con 0x558423b40260 2024-09-21T03:41:00.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.594+0000 7f8288004640 1 -- 172.21.15.38:0/1792112991 <== osd.6 v2:172.21.15.195:6826/4055825675 2 ==== osd_op_reply(33 1.00000000.inode [delete] v103'152 uv152 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f8278008070 con 0x558423b40260 2024-09-21T03:41:00.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.594+0000 7f828e54af40 1 -- 172.21.15.38:0/1792112991 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x558423b43300 con 0x558423b40260 2024-09-21T03:41:00.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.596+0000 7f8288004640 1 -- 172.21.15.38:0/1792112991 <== osd.6 v2:172.21.15.195:6826/4055825675 3 ==== osd_op_reply(34 1.00000000.inode [create] v103'153 uv153 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f8278008070 con 0x558423b40260 2024-09-21T03:41:00.599 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-09-21T03:41:00.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.596+0000 7f828e54af40 1 -- 172.21.15.38:0/1792112991 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x558423b43b50 con 0x558423b40260 2024-09-21T03:41:00.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.597+0000 7f8288004640 1 -- 172.21.15.38:0/1792112991 <== osd.6 v2:172.21.15.195:6826/4055825675 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v103'154 uv154 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f8278008070 con 0x558423b40260 2024-09-21T03:41:00.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.597+0000 7f828e54af40 1 -- 172.21.15.38:0/1792112991 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x558423b43ef0 con 0x558423b40260 2024-09-21T03:41:00.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.598+0000 7f8288004640 1 -- 172.21.15.38:0/1792112991 <== osd.6 v2:172.21.15.195:6826/4055825675 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v103'155 uv155 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f8278008070 con 0x558423b40260 2024-09-21T03:41:00.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.598+0000 7f828e54af40 1 -- 172.21.15.38:0/1792112991 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x558423b44290 con 0x558423b40260 2024-09-21T03:41:00.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.600+0000 7f8288004640 1 -- 172.21.15.38:0/1792112991 <== osd.6 v2:172.21.15.195:6826/4055825675 6 ==== osd_op_reply(37 200.00000000 [create] v103'156 uv149 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f8278008070 con 0x558423b40260 2024-09-21T03:41:00.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.600+0000 7f828e54af40 1 -- 172.21.15.38:0/1792112991 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e103) -- 0x558423b44630 con 0x558423b40260 2024-09-21T03:41:00.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.601+0000 7f8288004640 1 -- 172.21.15.38:0/1792112991 <== osd.6 v2:172.21.15.195:6826/4055825675 7 ==== osd_op_reply(38 200.00000000 [delete] v103'157 uv157 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8278008070 con 0x558423b40260 2024-09-21T03:41:00.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.601+0000 7f828e54af40 1 -- 172.21.15.38:0/1792112991 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x558423b449d0 con 0x558423b40260 2024-09-21T03:41:00.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.603+0000 7f8288004640 1 -- 172.21.15.38:0/1792112991 <== osd.6 v2:172.21.15.195:6826/4055825675 8 ==== osd_op_reply(39 200.00000000 [create] v103'158 uv158 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8278008070 con 0x558423b40260 2024-09-21T03:41:00.605 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-09-21T03:41:00.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.603+0000 7f828e54af40 1 -- 172.21.15.38:0/1792112991 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x558423b449d0 con 0x558423b40260 2024-09-21T03:41:00.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.604+0000 7f8288004640 1 -- 172.21.15.38:0/1792112991 <== osd.6 v2:172.21.15.195:6826/4055825675 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v103'159 uv159 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8278008070 con 0x558423b40260 2024-09-21T03:41:00.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.604+0000 7f828e54af40 1 -- 172.21.15.38:0/1792112991 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x558423b449d0 con 0x558423b40260 2024-09-21T03:41:00.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.605+0000 7f8288004640 1 -- 172.21.15.38:0/1792112991 <== osd.6 v2:172.21.15.195:6826/4055825675 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v103'160 uv160 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8278008070 con 0x558423b40260 2024-09-21T03:41:00.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.605+0000 7f828e54af40 1 -- 172.21.15.38:0/1792112991 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:42 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x558423b449d0 con 0x558423b40260 2024-09-21T03:41:00.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.607+0000 7f8288004640 1 -- 172.21.15.38:0/1792112991 <== osd.6 v2:172.21.15.195:6826/4055825675 11 ==== osd_op_reply(42 200.00000001 [create] v103'161 uv150 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f8278008070 con 0x558423b40260 2024-09-21T03:41:00.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.607+0000 7f828e54af40 1 -- 172.21.15.38:0/1792112991 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:43 13.14 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e103) -- 0x558423b449d0 con 0x558423b40260 2024-09-21T03:41:00.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.608+0000 7f8288004640 1 -- 172.21.15.38:0/1792112991 <== osd.6 v2:172.21.15.195:6826/4055825675 12 ==== osd_op_reply(43 200.00000001 [delete] v103'162 uv162 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8278008070 con 0x558423b40260 2024-09-21T03:41:00.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.608+0000 7f828e54af40 1 -- 172.21.15.38:0/1792112991 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:44 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x558423b449d0 con 0x558423b40260 2024-09-21T03:41:00.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.609+0000 7f8288004640 1 -- 172.21.15.38:0/1792112991 <== osd.6 v2:172.21.15.195:6826/4055825675 13 ==== osd_op_reply(44 200.00000001 [create] v103'163 uv163 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8278008070 con 0x558423b40260 2024-09-21T03:41:00.612 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-09-21T03:41:00.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.609+0000 7f828e54af40 1 -- 172.21.15.38:0/1792112991 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:45 13.14 13:2e2fa760:::200.00000001:head [write 0~17490 in=17490b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x558423b48e50 con 0x558423b40260 2024-09-21T03:41:00.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.611+0000 7f8288004640 1 -- 172.21.15.38:0/1792112991 <== osd.6 v2:172.21.15.195:6826/4055825675 14 ==== osd_op_reply(45 200.00000001 [write 0~17490] v103'164 uv164 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8278008070 con 0x558423b40260 2024-09-21T03:41:00.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.611+0000 7f828e54af40 1 -- 172.21.15.38:0/1792112991 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:46 13.14 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x558423b48e50 con 0x558423b40260 2024-09-21T03:41:00.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.612+0000 7f8288004640 1 -- 172.21.15.38:0/1792112991 <== osd.6 v2:172.21.15.195:6826/4055825675 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v103'165 uv165 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8278008070 con 0x558423b40260 2024-09-21T03:41:00.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.612+0000 7f828e54af40 1 -- 172.21.15.38:0/1792112991 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:47 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x558423b449d0 con 0x558423b40260 2024-09-21T03:41:00.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.613+0000 7f8288004640 1 -- 172.21.15.38:0/1792112991 <== osd.6 v2:172.21.15.195:6826/4055825675 16 ==== osd_op_reply(47 606.00000000 [create] v103'166 uv146 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f8278008070 con 0x558423b40260 2024-09-21T03:41:00.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.613+0000 7f828e54af40 1 -- 172.21.15.38:0/1792112991 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:48 13.14 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e103) -- 0x558423b449d0 con 0x558423b40260 2024-09-21T03:41:00.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.615+0000 7f8288004640 1 -- 172.21.15.38:0/1792112991 <== osd.6 v2:172.21.15.195:6826/4055825675 17 ==== osd_op_reply(48 606.00000000 [delete] v103'167 uv167 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8278008070 con 0x558423b40260 2024-09-21T03:41:00.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.615+0000 7f828e54af40 1 -- 172.21.15.38:0/1792112991 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:49 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x558423b449d0 con 0x558423b40260 2024-09-21T03:41:00.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.616+0000 7f8288004640 1 -- 172.21.15.38:0/1792112991 <== osd.6 v2:172.21.15.195:6826/4055825675 18 ==== osd_op_reply(49 606.00000000 [create] v103'168 uv168 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8278008070 con 0x558423b40260 2024-09-21T03:41:00.619 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-09-21T03:41:00.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.616+0000 7f828e54af40 1 -- 172.21.15.38:0/1792112991 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:50 13.14 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x558423b459e0 con 0x558423b40260 2024-09-21T03:41:00.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.618+0000 7f8288004640 1 -- 172.21.15.38:0/1792112991 <== osd.6 v2:172.21.15.195:6826/4055825675 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v103'169 uv169 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8278008070 con 0x558423b40260 2024-09-21T03:41:00.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.618+0000 7f828e54af40 1 -- 172.21.15.38:0/1792112991 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:51 13.14 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x558423b459e0 con 0x558423b40260 2024-09-21T03:41:00.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.619+0000 7f8288004640 1 -- 172.21.15.38:0/1792112991 <== osd.6 v2:172.21.15.195:6826/4055825675 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v103'170 uv170 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8278008070 con 0x558423b40260 2024-09-21T03:41:00.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.619+0000 7f828e54af40 1 -- 172.21.15.38:0/1792112991 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:52 13.14 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x558423b449d0 con 0x558423b40260 2024-09-21T03:41:00.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.620+0000 7f8288004640 1 -- 172.21.15.38:0/1792112991 <== osd.6 v2:172.21.15.195:6826/4055825675 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v103'171 uv171 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8278008070 con 0x558423b40260 2024-09-21T03:41:00.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.620+0000 7f828e54af40 1 --2- 172.21.15.38:0/1792112991 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x558423b44ee0 0x558423b47300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:00.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.620+0000 7f828e54af40 1 -- 172.21.15.38:0/1792112991 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:53 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x558423b478d0 con 0x558423b44ee0 2024-09-21T03:41:00.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.620+0000 7f8287803640 1 --2- 172.21.15.38:0/1792112991 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x558423b44ee0 0x558423b47300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:00.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.621+0000 7f8287803640 1 --2- 172.21.15.38:0/1792112991 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x558423b44ee0 0x558423b47300 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:00.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.622+0000 7f8287803640 1 -- 172.21.15.38:0/1792112991 <== osd.0 v2:172.21.15.38:6800/2381639433 1 ==== osd_op_reply(53 607.00000000 [create] v103'39 uv38 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f827c0a15f0 con 0x558423b44ee0 2024-09-21T03:41:00.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.622+0000 7f828e54af40 1 -- 172.21.15.38:0/1792112991 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:54 13.1c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e103) -- 0x558423b47d50 con 0x558423b44ee0 2024-09-21T03:41:00.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.623+0000 7f8287803640 1 -- 172.21.15.38:0/1792112991 <== osd.0 v2:172.21.15.38:6800/2381639433 2 ==== osd_op_reply(54 607.00000000 [delete] v103'40 uv40 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f827c0a15f0 con 0x558423b44ee0 2024-09-21T03:41:00.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.623+0000 7f828e54af40 1 -- 172.21.15.38:0/1792112991 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:55 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x558423b48200 con 0x558423b44ee0 2024-09-21T03:41:00.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.624+0000 7f8287803640 1 -- 172.21.15.38:0/1792112991 <== osd.0 v2:172.21.15.38:6800/2381639433 3 ==== osd_op_reply(55 607.00000000 [create] v103'41 uv41 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f827c0a15f0 con 0x558423b44ee0 2024-09-21T03:41:00.627 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-09-21T03:41:00.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.624+0000 7f828e54af40 1 -- 172.21.15.38:0/1792112991 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:56 13.1c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x558423b49ab0 con 0x558423b44ee0 2024-09-21T03:41:00.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.625+0000 7f8287803640 1 -- 172.21.15.38:0/1792112991 <== osd.0 v2:172.21.15.38:6800/2381639433 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v103'42 uv42 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f827c0a15f0 con 0x558423b44ee0 2024-09-21T03:41:00.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.625+0000 7f828e54af40 1 -- 172.21.15.38:0/1792112991 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:57 13.1c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x558423b49e50 con 0x558423b44ee0 2024-09-21T03:41:00.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.626+0000 7f8287803640 1 -- 172.21.15.38:0/1792112991 <== osd.0 v2:172.21.15.38:6800/2381639433 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v103'43 uv43 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f827c0a15f0 con 0x558423b44ee0 2024-09-21T03:41:00.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.627+0000 7f828e54af40 1 -- 172.21.15.38:0/1792112991 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:58 13.1c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x558423b48aa0 con 0x558423b44ee0 2024-09-21T03:41:00.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.628+0000 7f8287803640 1 -- 172.21.15.38:0/1792112991 <== osd.0 v2:172.21.15.38:6800/2381639433 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v103'44 uv44 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f827c0a15f0 con 0x558423b44ee0 2024-09-21T03:41:00.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.628+0000 7f828e54af40 1 --2- 172.21.15.38:0/1792112991 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x558423b493a0 0x558423b4c200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:00.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.628+0000 7f828e54af40 1 -- 172.21.15.38:0/1792112991 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:59 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x558423b4c740 con 0x558423b493a0 2024-09-21T03:41:00.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.628+0000 7f8288805640 1 --2- 172.21.15.38:0/1792112991 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x558423b493a0 0x558423b4c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:00.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.628+0000 7f8288805640 1 --2- 172.21.15.38:0/1792112991 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x558423b493a0 0x558423b4c200 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:00.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.629+0000 7f8288805640 1 -- 172.21.15.38:0/1792112991 <== osd.3 v2:172.21.15.38:6808/1286810294 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v103'30 uv29 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f8280003040 con 0x558423b493a0 2024-09-21T03:41:00.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.629+0000 7f828e54af40 1 -- 172.21.15.38:0/1792112991 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:60 13.6 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e103) -- 0x558423b4c740 con 0x558423b493a0 2024-09-21T03:41:00.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.631+0000 7f8288805640 1 -- 172.21.15.38:0/1792112991 <== osd.3 v2:172.21.15.38:6808/1286810294 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v103'31 uv31 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f8280003040 con 0x558423b493a0 2024-09-21T03:41:00.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.631+0000 7f828e54af40 1 -- 172.21.15.38:0/1792112991 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:61 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x558423b4cb30 con 0x558423b493a0 2024-09-21T03:41:00.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.632+0000 7f8288805640 1 -- 172.21.15.38:0/1792112991 <== osd.3 v2:172.21.15.38:6808/1286810294 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v103'32 uv32 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f8280003040 con 0x558423b493a0 2024-09-21T03:41:00.635 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-09-21T03:41:00.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.632+0000 7f828e54af40 1 -- 172.21.15.38:0/1792112991 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:62 13.6 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x558423b4cb30 con 0x558423b493a0 2024-09-21T03:41:00.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.633+0000 7f8288805640 1 -- 172.21.15.38:0/1792112991 <== osd.3 v2:172.21.15.38:6808/1286810294 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v103'33 uv33 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f8280003040 con 0x558423b493a0 2024-09-21T03:41:00.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.633+0000 7f828e54af40 1 -- 172.21.15.38:0/1792112991 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x558423b4cb30 con 0x558423b44ee0 2024-09-21T03:41:00.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.634+0000 7f8287803640 1 -- 172.21.15.38:0/1792112991 <== osd.0 v2:172.21.15.38:6800/2381639433 7 ==== osd_op_reply(63 608.00000000 [create] v103'39 uv38 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f827c0a15f0 con 0x558423b44ee0 2024-09-21T03:41:00.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.634+0000 7f828e54af40 1 -- 172.21.15.38:0/1792112991 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e103) -- 0x558423b4ced0 con 0x558423b44ee0 2024-09-21T03:41:00.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.635+0000 7f8287803640 1 -- 172.21.15.38:0/1792112991 <== osd.0 v2:172.21.15.38:6800/2381639433 8 ==== osd_op_reply(64 608.00000000 [delete] v103'40 uv40 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f827c0a15f0 con 0x558423b44ee0 2024-09-21T03:41:00.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.635+0000 7f828e54af40 1 -- 172.21.15.38:0/1792112991 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x558423b4ced0 con 0x558423b44ee0 2024-09-21T03:41:00.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.636+0000 7f8287803640 1 -- 172.21.15.38:0/1792112991 <== osd.0 v2:172.21.15.38:6800/2381639433 9 ==== osd_op_reply(65 608.00000000 [create] v103'41 uv41 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f827c0a15f0 con 0x558423b44ee0 2024-09-21T03:41:00.639 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-09-21T03:41:00.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.636+0000 7f828e54af40 1 -- 172.21.15.38:0/1792112991 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x558423b4dee0 con 0x558423b44ee0 2024-09-21T03:41:00.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.637+0000 7f8287803640 1 -- 172.21.15.38:0/1792112991 <== osd.0 v2:172.21.15.38:6800/2381639433 10 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v103'42 uv42 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f827c0a15f0 con 0x558423b44ee0 2024-09-21T03:41:00.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.637+0000 7f828e54af40 1 -- 172.21.15.38:0/1792112991 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x558423b4dee0 con 0x558423b44ee0 2024-09-21T03:41:00.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.638+0000 7f8287803640 1 -- 172.21.15.38:0/1792112991 <== osd.0 v2:172.21.15.38:6800/2381639433 11 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v103'43 uv43 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f827c0a15f0 con 0x558423b44ee0 2024-09-21T03:41:00.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.638+0000 7f828e54af40 1 -- 172.21.15.38:0/1792112991 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x558423b4ced0 con 0x558423b44ee0 2024-09-21T03:41:00.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.639+0000 7f8287803640 1 -- 172.21.15.38:0/1792112991 <== osd.0 v2:172.21.15.38:6800/2381639433 12 ==== osd_op_reply(68 608.00000000 [omap-set-header] v103'44 uv44 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f827c0a15f0 con 0x558423b44ee0 2024-09-21T03:41:00.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.639+0000 7f828e54af40 1 -- 172.21.15.38:0/1792112991 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x558423b4ced0 con 0x558423b493a0 2024-09-21T03:41:00.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.641+0000 7f8288805640 1 -- 172.21.15.38:0/1792112991 <== osd.3 v2:172.21.15.38:6808/1286810294 5 ==== osd_op_reply(69 604.00000000 [create] v103'70 uv64 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f8280003040 con 0x558423b493a0 2024-09-21T03:41:00.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.641+0000 7f828e54af40 1 -- 172.21.15.38:0/1792112991 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e103) -- 0x558423b4ced0 con 0x558423b493a0 2024-09-21T03:41:00.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.642+0000 7f8288805640 1 -- 172.21.15.38:0/1792112991 <== osd.3 v2:172.21.15.38:6808/1286810294 6 ==== osd_op_reply(70 604.00000000 [delete] v103'71 uv71 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8280003040 con 0x558423b493a0 2024-09-21T03:41:00.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.642+0000 7f828e54af40 1 -- 172.21.15.38:0/1792112991 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x558423b4ced0 con 0x558423b493a0 2024-09-21T03:41:00.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.643+0000 7f8288805640 1 -- 172.21.15.38:0/1792112991 <== osd.3 v2:172.21.15.38:6808/1286810294 7 ==== osd_op_reply(71 604.00000000 [create] v103'72 uv72 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8280003040 con 0x558423b493a0 2024-09-21T03:41:00.646 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-09-21T03:41:00.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.643+0000 7f828e54af40 1 -- 172.21.15.38:0/1792112991 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x558423b4dee0 con 0x558423b493a0 2024-09-21T03:41:00.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.644+0000 7f8288805640 1 -- 172.21.15.38:0/1792112991 <== osd.3 v2:172.21.15.38:6808/1286810294 8 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v103'73 uv73 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8280003040 con 0x558423b493a0 2024-09-21T03:41:00.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.644+0000 7f828e54af40 1 -- 172.21.15.38:0/1792112991 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x558423b4dee0 con 0x558423b493a0 2024-09-21T03:41:00.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.645+0000 7f8288805640 1 -- 172.21.15.38:0/1792112991 <== osd.3 v2:172.21.15.38:6808/1286810294 9 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v103'74 uv74 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8280003040 con 0x558423b493a0 2024-09-21T03:41:00.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.645+0000 7f828e54af40 1 -- 172.21.15.38:0/1792112991 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x558423b4ced0 con 0x558423b493a0 2024-09-21T03:41:00.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.646+0000 7f8288805640 1 -- 172.21.15.38:0/1792112991 <== osd.3 v2:172.21.15.38:6808/1286810294 10 ==== osd_op_reply(74 604.00000000 [omap-set-header] v103'75 uv75 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8280003040 con 0x558423b493a0 2024-09-21T03:41:00.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.646+0000 7f828e54af40 1 -- 172.21.15.38:0/1792112991 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x558423b4ced0 con 0x558423b493a0 2024-09-21T03:41:00.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.647+0000 7f8288805640 1 -- 172.21.15.38:0/1792112991 <== osd.3 v2:172.21.15.38:6808/1286810294 11 ==== osd_op_reply(75 500.00000000 [create] v103'76 uv69 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f8280003040 con 0x558423b493a0 2024-09-21T03:41:00.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.647+0000 7f828e54af40 1 -- 172.21.15.38:0/1792112991 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e103) -- 0x558423b4ced0 con 0x558423b493a0 2024-09-21T03:41:00.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.648+0000 7f8288805640 1 -- 172.21.15.38:0/1792112991 <== osd.3 v2:172.21.15.38:6808/1286810294 12 ==== osd_op_reply(76 500.00000000 [delete] v103'77 uv77 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8280003040 con 0x558423b493a0 2024-09-21T03:41:00.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.648+0000 7f828e54af40 1 -- 172.21.15.38:0/1792112991 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x558423b4ced0 con 0x558423b493a0 2024-09-21T03:41:00.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.649+0000 7f8288805640 1 -- 172.21.15.38:0/1792112991 <== osd.3 v2:172.21.15.38:6808/1286810294 13 ==== osd_op_reply(77 500.00000000 [create] v103'78 uv78 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8280003040 con 0x558423b493a0 2024-09-21T03:41:00.651 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-09-21T03:41:00.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.649+0000 7f828e54af40 1 -- 172.21.15.38:0/1792112991 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x558423b4ced0 con 0x558423b493a0 2024-09-21T03:41:00.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.650+0000 7f8288805640 1 -- 172.21.15.38:0/1792112991 <== osd.3 v2:172.21.15.38:6808/1286810294 14 ==== osd_op_reply(78 500.00000000 [write 0~90] v103'79 uv79 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8280003040 con 0x558423b493a0 2024-09-21T03:41:00.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.650+0000 7f828e54af40 1 -- 172.21.15.38:0/1792112991 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x558423b4ced0 con 0x558423b493a0 2024-09-21T03:41:00.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.651+0000 7f8288805640 1 -- 172.21.15.38:0/1792112991 <== osd.3 v2:172.21.15.38:6808/1286810294 15 ==== osd_op_reply(79 500.00000000 [omap-set-header] v103'80 uv80 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8280003040 con 0x558423b493a0 2024-09-21T03:41:00.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.651+0000 7f828e54af40 1 -- 172.21.15.38:0/1792112991 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:80 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x558423b4ced0 con 0x558423b44ee0 2024-09-21T03:41:00.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.652+0000 7f8287803640 1 -- 172.21.15.38:0/1792112991 <== osd.0 v2:172.21.15.38:6800/2381639433 13 ==== osd_op_reply(80 mds_snaptable [create] v103'32 uv31 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f827c0a15f0 con 0x558423b44ee0 2024-09-21T03:41:00.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.652+0000 7f828e54af40 1 -- 172.21.15.38:0/1792112991 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:81 13.d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e103) -- 0x558423b4ced0 con 0x558423b44ee0 2024-09-21T03:41:00.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.653+0000 7f8287803640 1 -- 172.21.15.38:0/1792112991 <== osd.0 v2:172.21.15.38:6800/2381639433 14 ==== osd_op_reply(81 mds_snaptable [delete] v103'33 uv33 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f827c0a15f0 con 0x558423b44ee0 2024-09-21T03:41:00.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.653+0000 7f828e54af40 1 -- 172.21.15.38:0/1792112991 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:82 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x558423b4ced0 con 0x558423b44ee0 2024-09-21T03:41:00.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.654+0000 7f8287803640 1 -- 172.21.15.38:0/1792112991 <== osd.0 v2:172.21.15.38:6800/2381639433 15 ==== osd_op_reply(82 mds_snaptable [create] v103'34 uv34 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f827c0a15f0 con 0x558423b44ee0 2024-09-21T03:41:00.657 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-09-21T03:41:00.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.654+0000 7f828e54af40 1 -- 172.21.15.38:0/1792112991 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:83 13.d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x558423b4ced0 con 0x558423b44ee0 2024-09-21T03:41:00.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.655+0000 7f8287803640 1 -- 172.21.15.38:0/1792112991 <== osd.0 v2:172.21.15.38:6800/2381639433 16 ==== osd_op_reply(83 mds_snaptable [write 0~70] v103'35 uv35 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f827c0a15f0 con 0x558423b44ee0 2024-09-21T03:41:00.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.655+0000 7f828e54af40 1 -- 172.21.15.38:0/1792112991 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:84 13.d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x558423b4ced0 con 0x558423b44ee0 2024-09-21T03:41:00.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.656+0000 7f8287803640 1 -- 172.21.15.38:0/1792112991 <== osd.0 v2:172.21.15.38:6800/2381639433 17 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v103'36 uv36 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f827c0a15f0 con 0x558423b44ee0 2024-09-21T03:41:00.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.656+0000 7f828e54af40 1 --2- 172.21.15.38:0/1792112991 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x558423b4d550 0x558423b4f970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:00.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.657+0000 7f828e54af40 1 -- 172.21.15.38:0/1792112991 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x558423b4ff40 con 0x558423b4d550 2024-09-21T03:41:00.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.657+0000 7f8288004640 1 --2- 172.21.15.38:0/1792112991 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x558423b4d550 0x558423b4f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:00.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.657+0000 7f8288004640 1 --2- 172.21.15.38:0/1792112991 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x558423b4d550 0x558423b4f970 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:00.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.658+0000 7f8288004640 1 -- 172.21.15.38:0/1792112991 <== osd.1 v2:172.21.15.38:6816/1491818732 1 ==== osd_op_reply(85 605.00000000 [create] v103'39 uv38 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f8278008070 con 0x558423b4d550 2024-09-21T03:41:00.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.658+0000 7f828e54af40 1 -- 172.21.15.38:0/1792112991 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e103) -- 0x558423b50310 con 0x558423b4d550 2024-09-21T03:41:00.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.659+0000 7f8288004640 1 -- 172.21.15.38:0/1792112991 <== osd.1 v2:172.21.15.38:6816/1491818732 2 ==== osd_op_reply(86 605.00000000 [delete] v103'40 uv40 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8278008070 con 0x558423b4d550 2024-09-21T03:41:00.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.659+0000 7f828e54af40 1 -- 172.21.15.38:0/1792112991 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x558423b50310 con 0x558423b4d550 2024-09-21T03:41:00.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.660+0000 7f8288004640 1 -- 172.21.15.38:0/1792112991 <== osd.1 v2:172.21.15.38:6816/1491818732 3 ==== osd_op_reply(87 605.00000000 [create] v103'41 uv41 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8278008070 con 0x558423b4d550 2024-09-21T03:41:00.663 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-09-21T03:41:00.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.660+0000 7f828e54af40 1 -- 172.21.15.38:0/1792112991 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x558423b51320 con 0x558423b4d550 2024-09-21T03:41:00.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.661+0000 7f8288004640 1 -- 172.21.15.38:0/1792112991 <== osd.1 v2:172.21.15.38:6816/1491818732 4 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v103'42 uv42 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8278008070 con 0x558423b4d550 2024-09-21T03:41:00.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.661+0000 7f828e54af40 1 -- 172.21.15.38:0/1792112991 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x558423b51320 con 0x558423b4d550 2024-09-21T03:41:00.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.662+0000 7f8288004640 1 -- 172.21.15.38:0/1792112991 <== osd.1 v2:172.21.15.38:6816/1491818732 5 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v103'43 uv43 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8278008070 con 0x558423b4d550 2024-09-21T03:41:00.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.662+0000 7f828e54af40 1 -- 172.21.15.38:0/1792112991 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x558423b50310 con 0x558423b4d550 2024-09-21T03:41:00.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.663+0000 7f8288004640 1 -- 172.21.15.38:0/1792112991 <== osd.1 v2:172.21.15.38:6816/1491818732 6 ==== osd_op_reply(90 605.00000000 [omap-set-header] v103'44 uv44 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8278008070 con 0x558423b4d550 2024-09-21T03:41:00.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.663+0000 7f828e54af40 1 -- 172.21.15.38:0/1792112991 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x558423b50310 con 0x558423b4d550 2024-09-21T03:41:00.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.664+0000 7f8288004640 1 -- 172.21.15.38:0/1792112991 <== osd.1 v2:172.21.15.38:6816/1491818732 7 ==== osd_op_reply(91 mds0_inotable [create] v103'64 uv58 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f8278008070 con 0x558423b4d550 2024-09-21T03:41:00.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.664+0000 7f828e54af40 1 -- 172.21.15.38:0/1792112991 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e103) -- 0x558423b50310 con 0x558423b4d550 2024-09-21T03:41:00.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.665+0000 7f8288004640 1 -- 172.21.15.38:0/1792112991 <== osd.1 v2:172.21.15.38:6816/1491818732 8 ==== osd_op_reply(92 mds0_inotable [delete] v103'65 uv65 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f8278008070 con 0x558423b4d550 2024-09-21T03:41:00.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.665+0000 7f828e54af40 1 -- 172.21.15.38:0/1792112991 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x558423b50310 con 0x558423b4d550 2024-09-21T03:41:00.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.666+0000 7f8288004640 1 -- 172.21.15.38:0/1792112991 <== osd.1 v2:172.21.15.38:6816/1491818732 9 ==== osd_op_reply(93 mds0_inotable [create] v103'66 uv66 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f8278008070 con 0x558423b4d550 2024-09-21T03:41:00.669 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-09-21T03:41:00.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.666+0000 7f828e54af40 1 -- 172.21.15.38:0/1792112991 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:94 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x558423b48690 con 0x558423b4d550 2024-09-21T03:41:00.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.667+0000 7f8288004640 1 -- 172.21.15.38:0/1792112991 <== osd.1 v2:172.21.15.38:6816/1491818732 10 ==== osd_op_reply(94 mds0_inotable [write 0~34] v103'67 uv67 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f8278008070 con 0x558423b4d550 2024-09-21T03:41:00.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.667+0000 7f828e54af40 1 -- 172.21.15.38:0/1792112991 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:95 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x558423b48690 con 0x558423b4d550 2024-09-21T03:41:00.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.668+0000 7f8288004640 1 -- 172.21.15.38:0/1792112991 <== osd.1 v2:172.21.15.38:6816/1491818732 11 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v103'68 uv68 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f8278008070 con 0x558423b4d550 2024-09-21T03:41:00.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.668+0000 7f828e54af40 1 -- 172.21.15.38:0/1792112991 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:96 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x558423b48690 con 0x558423b4d550 2024-09-21T03:41:00.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.669+0000 7f8288004640 1 -- 172.21.15.38:0/1792112991 <== osd.1 v2:172.21.15.38:6816/1491818732 12 ==== osd_op_reply(96 100.00000000 [create] v103'69 uv63 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f8278008070 con 0x558423b4d550 2024-09-21T03:41:00.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.669+0000 7f828e54af40 1 -- 172.21.15.38:0/1792112991 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:97 13.13 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e103) -- 0x558423b48690 con 0x558423b4d550 2024-09-21T03:41:00.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.670+0000 7f8288004640 1 -- 172.21.15.38:0/1792112991 <== osd.1 v2:172.21.15.38:6816/1491818732 13 ==== osd_op_reply(97 100.00000000 [delete] v103'70 uv70 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8278008070 con 0x558423b4d550 2024-09-21T03:41:00.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.670+0000 7f828e54af40 1 -- 172.21.15.38:0/1792112991 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:98 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x558423b48690 con 0x558423b4d550 2024-09-21T03:41:00.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.671+0000 7f8288004640 1 -- 172.21.15.38:0/1792112991 <== osd.1 v2:172.21.15.38:6816/1491818732 14 ==== osd_op_reply(98 100.00000000 [create] v103'71 uv71 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8278008070 con 0x558423b4d550 2024-09-21T03:41:00.674 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-09-21T03:41:00.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.671+0000 7f828e54af40 1 -- 172.21.15.38:0/1792112991 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:99 13.13 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x558423b48690 con 0x558423b4d550 2024-09-21T03:41:00.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.672+0000 7f8288004640 1 -- 172.21.15.38:0/1792112991 <== osd.1 v2:172.21.15.38:6816/1491818732 15 ==== osd_op_reply(99 100.00000000 [omap-set-header] v103'72 uv72 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8278008070 con 0x558423b4d550 2024-09-21T03:41:00.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.672+0000 7f828e54af40 1 -- 172.21.15.38:0/1792112991 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:100 13.13 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x558423b53120 con 0x558423b4d550 2024-09-21T03:41:00.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.673+0000 7f8288004640 1 -- 172.21.15.38:0/1792112991 <== osd.1 v2:172.21.15.38:6816/1491818732 16 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v103'73 uv73 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8278008070 con 0x558423b4d550 2024-09-21T03:41:00.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.673+0000 7f828e54af40 1 -- 172.21.15.38:0/1792112991 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x558423b507c0 con 0x558423b493a0 2024-09-21T03:41:00.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.674+0000 7f8288805640 1 -- 172.21.15.38:0/1792112991 <== osd.3 v2:172.21.15.38:6808/1286810294 16 ==== osd_op_reply(101 mds0_sessionmap [create] v103'31 uv29 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7f8280003040 con 0x558423b493a0 2024-09-21T03:41:00.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.674+0000 7f828e54af40 1 -- 172.21.15.38:0/1792112991 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e103) -- 0x558423b507c0 con 0x558423b493a0 2024-09-21T03:41:00.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.676+0000 7f8288805640 1 -- 172.21.15.38:0/1792112991 <== osd.3 v2:172.21.15.38:6808/1286810294 17 ==== osd_op_reply(102 mds0_sessionmap [delete] v103'32 uv32 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f8280003040 con 0x558423b493a0 2024-09-21T03:41:00.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.676+0000 7f828e54af40 1 -- 172.21.15.38:0/1792112991 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:103 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x558423b507c0 con 0x558423b493a0 2024-09-21T03:41:00.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.677+0000 7f8288805640 1 -- 172.21.15.38:0/1792112991 <== osd.3 v2:172.21.15.38:6808/1286810294 18 ==== osd_op_reply(103 mds0_sessionmap [create] v103'33 uv33 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f8280003040 con 0x558423b493a0 2024-09-21T03:41:00.679 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-09-21T03:41:00.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.677+0000 7f828e54af40 1 -- 172.21.15.38:0/1792112991 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:104 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x558423b507c0 con 0x558423b493a0 2024-09-21T03:41:00.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.678+0000 7f8288805640 1 -- 172.21.15.38:0/1792112991 <== osd.3 v2:172.21.15.38:6808/1286810294 19 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v103'34 uv34 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f8280003040 con 0x558423b493a0 2024-09-21T03:41:00.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.678+0000 7f828e54af40 1 -- 172.21.15.38:0/1792112991 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x558423b507c0 con 0x558423b493a0 2024-09-21T03:41:00.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.679+0000 7f8288805640 1 -- 172.21.15.38:0/1792112991 <== osd.3 v2:172.21.15.38:6808/1286810294 20 ==== osd_op_reply(105 609.00000000 [create] v103'39 uv38 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f8280003040 con 0x558423b493a0 2024-09-21T03:41:00.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.679+0000 7f828e54af40 1 -- 172.21.15.38:0/1792112991 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e103) -- 0x558423b507c0 con 0x558423b493a0 2024-09-21T03:41:00.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.680+0000 7f8288805640 1 -- 172.21.15.38:0/1792112991 <== osd.3 v2:172.21.15.38:6808/1286810294 21 ==== osd_op_reply(106 609.00000000 [delete] v103'40 uv40 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8280003040 con 0x558423b493a0 2024-09-21T03:41:00.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.680+0000 7f828e54af40 1 -- 172.21.15.38:0/1792112991 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x558423b507c0 con 0x558423b493a0 2024-09-21T03:41:00.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.681+0000 7f8288805640 1 -- 172.21.15.38:0/1792112991 <== osd.3 v2:172.21.15.38:6808/1286810294 22 ==== osd_op_reply(107 609.00000000 [create] v103'41 uv41 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8280003040 con 0x558423b493a0 2024-09-21T03:41:00.684 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-09-21T03:41:00.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.681+0000 7f828e54af40 1 -- 172.21.15.38:0/1792112991 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x558423b517d0 con 0x558423b493a0 2024-09-21T03:41:00.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.682+0000 7f8288805640 1 -- 172.21.15.38:0/1792112991 <== osd.3 v2:172.21.15.38:6808/1286810294 23 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v103'42 uv42 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8280003040 con 0x558423b493a0 2024-09-21T03:41:00.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.682+0000 7f828e54af40 1 -- 172.21.15.38:0/1792112991 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:109 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x558423b517d0 con 0x558423b493a0 2024-09-21T03:41:00.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.683+0000 7f8288805640 1 -- 172.21.15.38:0/1792112991 <== osd.3 v2:172.21.15.38:6808/1286810294 24 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v103'43 uv43 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8280003040 con 0x558423b493a0 2024-09-21T03:41:00.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.683+0000 7f828e54af40 1 -- 172.21.15.38:0/1792112991 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:110 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x558423b507c0 con 0x558423b493a0 2024-09-21T03:41:00.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.684+0000 7f8288805640 1 -- 172.21.15.38:0/1792112991 <== osd.3 v2:172.21.15.38:6808/1286810294 25 ==== osd_op_reply(110 609.00000000 [omap-set-header] v103'44 uv44 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8280003040 con 0x558423b493a0 2024-09-21T03:41:00.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.684+0000 7f828e54af40 1 --2- 172.21.15.38:0/1792112991 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x558423b50e40 0x558423b51250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:00.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.684+0000 7f828e54af40 1 -- 172.21.15.38:0/1792112991 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x558423b51790 con 0x558423b50e40 2024-09-21T03:41:00.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.684+0000 7f8287803640 1 --2- 172.21.15.38:0/1792112991 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x558423b50e40 0x558423b51250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:00.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.684+0000 7f8287803640 1 --2- 172.21.15.38:0/1792112991 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x558423b50e40 0x558423b51250 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:00.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.685+0000 7f8287803640 1 -- 172.21.15.38:0/1792112991 <== osd.2 v2:172.21.15.38:6824/2577858902 1 ==== osd_op_reply(111 400.00000000 [create] v103'32 uv31 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f827c0a15f0 con 0x558423b50e40 2024-09-21T03:41:00.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.685+0000 7f828e54af40 1 -- 172.21.15.38:0/1792112991 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e103) -- 0x558423b51790 con 0x558423b50e40 2024-09-21T03:41:00.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.686+0000 7f8287803640 1 -- 172.21.15.38:0/1792112991 <== osd.2 v2:172.21.15.38:6824/2577858902 2 ==== osd_op_reply(112 400.00000000 [delete] v103'33 uv33 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f827c0a15f0 con 0x558423b50e40 2024-09-21T03:41:00.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.686+0000 7f828e54af40 1 -- 172.21.15.38:0/1792112991 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x558423b51bb0 con 0x558423b50e40 2024-09-21T03:41:00.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.687+0000 7f8287803640 1 -- 172.21.15.38:0/1792112991 <== osd.2 v2:172.21.15.38:6824/2577858902 3 ==== osd_op_reply(113 400.00000000 [create] v103'34 uv34 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f827c0a15f0 con 0x558423b50e40 2024-09-21T03:41:00.690 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-09-21T03:41:00.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.687+0000 7f828e54af40 1 -- 172.21.15.38:0/1792112991 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:114 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x558423b51bb0 con 0x558423b50e40 2024-09-21T03:41:00.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.689+0000 7f8287803640 1 -- 172.21.15.38:0/1792112991 <== osd.2 v2:172.21.15.38:6824/2577858902 4 ==== osd_op_reply(114 400.00000000 [write 0~22] v103'35 uv35 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f827c0a15f0 con 0x558423b50e40 2024-09-21T03:41:00.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.689+0000 7f828e54af40 1 -- 172.21.15.38:0/1792112991 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:115 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x558423b51bb0 con 0x558423b50e40 2024-09-21T03:41:00.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.690+0000 7f8287803640 1 -- 172.21.15.38:0/1792112991 <== osd.2 v2:172.21.15.38:6824/2577858902 5 ==== osd_op_reply(115 400.00000000 [omap-set-header] v103'36 uv36 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f827c0a15f0 con 0x558423b50e40 2024-09-21T03:41:00.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.690+0000 7f828e54af40 1 -- 172.21.15.38:0/1792112991 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:116 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x558423b51bb0 con 0x558423b4d550 2024-09-21T03:41:00.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.691+0000 7f8288004640 1 -- 172.21.15.38:0/1792112991 <== osd.1 v2:172.21.15.38:6816/1491818732 17 ==== osd_op_reply(116 100.00000000.inode [create] v103'82 uv73 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7f8278008070 con 0x558423b4d550 2024-09-21T03:41:00.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.691+0000 7f828e54af40 1 -- 172.21.15.38:0/1792112991 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:117 13.1f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e103) -- 0x558423b51bb0 con 0x558423b4d550 2024-09-21T03:41:00.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.692+0000 7f8288004640 1 -- 172.21.15.38:0/1792112991 <== osd.1 v2:172.21.15.38:6816/1491818732 18 ==== osd_op_reply(117 100.00000000.inode [delete] v103'83 uv83 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f8278008070 con 0x558423b4d550 2024-09-21T03:41:00.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.692+0000 7f828e54af40 1 -- 172.21.15.38:0/1792112991 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:118 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x558423b51bb0 con 0x558423b4d550 2024-09-21T03:41:00.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.693+0000 7f8288004640 1 -- 172.21.15.38:0/1792112991 <== osd.1 v2:172.21.15.38:6816/1491818732 19 ==== osd_op_reply(118 100.00000000.inode [create] v103'84 uv84 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f8278008070 con 0x558423b4d550 2024-09-21T03:41:00.696 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-09-21T03:41:00.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.693+0000 7f828e54af40 1 -- 172.21.15.38:0/1792112991 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:119 13.1f 13:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x558423b51bb0 con 0x558423b4d550 2024-09-21T03:41:00.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.694+0000 7f8288004640 1 -- 172.21.15.38:0/1792112991 <== osd.1 v2:172.21.15.38:6816/1491818732 20 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v103'85 uv85 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f8278008070 con 0x558423b4d550 2024-09-21T03:41:00.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.694+0000 7f828e54af40 1 -- 172.21.15.38:0/1792112991 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:120 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x558423b51bb0 con 0x558423b4d550 2024-09-21T03:41:00.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.695+0000 7f8288004640 1 -- 172.21.15.38:0/1792112991 <== osd.1 v2:172.21.15.38:6816/1491818732 21 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v103'86 uv86 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f8278008070 con 0x558423b4d550 2024-09-21T03:41:00.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.695+0000 7f828e54af40 1 -- 172.21.15.38:0/1792112991 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:121 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x558423b51bb0 con 0x558423b4d550 2024-09-21T03:41:00.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.697+0000 7f8288004640 1 -- 172.21.15.38:0/1792112991 <== osd.1 v2:172.21.15.38:6816/1491818732 22 ==== osd_op_reply(121 1.00000000 [create] v103'87 uv80 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7f8278008070 con 0x558423b4d550 2024-09-21T03:41:00.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.697+0000 7f828e54af40 1 -- 172.21.15.38:0/1792112991 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:122 13.1f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e103) -- 0x558423b51bb0 con 0x558423b4d550 2024-09-21T03:41:00.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.698+0000 7f8288004640 1 -- 172.21.15.38:0/1792112991 <== osd.1 v2:172.21.15.38:6816/1491818732 23 ==== osd_op_reply(122 1.00000000 [delete] v103'88 uv88 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f8278008070 con 0x558423b4d550 2024-09-21T03:41:00.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.698+0000 7f828e54af40 1 -- 172.21.15.38:0/1792112991 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:123 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x558423b51bb0 con 0x558423b4d550 2024-09-21T03:41:00.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.699+0000 7f8288004640 1 -- 172.21.15.38:0/1792112991 <== osd.1 v2:172.21.15.38:6816/1491818732 24 ==== osd_op_reply(123 1.00000000 [create] v103'89 uv89 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f8278008070 con 0x558423b4d550 2024-09-21T03:41:00.702 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-09-21T03:41:00.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.699+0000 7f828e54af40 1 -- 172.21.15.38:0/1792112991 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:124 13.1f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x558423b51bb0 con 0x558423b4d550 2024-09-21T03:41:00.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.700+0000 7f8288004640 1 -- 172.21.15.38:0/1792112991 <== osd.1 v2:172.21.15.38:6816/1491818732 25 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v103'90 uv90 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f8278008070 con 0x558423b4d550 2024-09-21T03:41:00.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.700+0000 7f828e54af40 1 -- 172.21.15.38:0/1792112991 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:125 13.1f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x558423b51bb0 con 0x558423b4d550 2024-09-21T03:41:00.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.701+0000 7f8288004640 1 -- 172.21.15.38:0/1792112991 <== osd.1 v2:172.21.15.38:6816/1491818732 26 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v103'91 uv91 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f8278008070 con 0x558423b4d550 2024-09-21T03:41:00.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.701+0000 7f828e54af40 1 -- 172.21.15.38:0/1792112991 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:126 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x558423b51bb0 con 0x558423b4d550 2024-09-21T03:41:00.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.702+0000 7f8288004640 1 -- 172.21.15.38:0/1792112991 <== osd.1 v2:172.21.15.38:6816/1491818732 27 ==== osd_op_reply(126 1.00000000 [omap-set-header] v103'92 uv92 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f8278008070 con 0x558423b4d550 2024-09-21T03:41:00.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.702+0000 7f828e54af40 1 -- 172.21.15.38:0/1792112991 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:127 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=584b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x558423b51fc0 con 0x558423b4d550 2024-09-21T03:41:00.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.704+0000 7f8288004640 1 -- 172.21.15.38:0/1792112991 <== osd.1 v2:172.21.15.38:6816/1491818732 28 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v103'93 uv93 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f8278008070 con 0x558423b4d550 2024-09-21T03:41:00.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.704+0000 7f828e54af40 1 -- 172.21.15.38:0/1792112991 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x558423b44ee0 msgr2=0x558423b47300 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:00.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.704+0000 7f828e54af40 1 --2- 172.21.15.38:0/1792112991 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x558423b44ee0 0x558423b47300 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:00.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.704+0000 7f828e54af40 1 -- 172.21.15.38:0/1792112991 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x558423b4d550 msgr2=0x558423b4f970 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:00.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.704+0000 7f828e54af40 1 --2- 172.21.15.38:0/1792112991 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x558423b4d550 0x558423b4f970 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:00.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.704+0000 7f828e54af40 1 -- 172.21.15.38:0/1792112991 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x558423b50e40 msgr2=0x558423b51250 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:00.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.704+0000 7f828e54af40 1 --2- 172.21.15.38:0/1792112991 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x558423b50e40 0x558423b51250 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:00.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.704+0000 7f828e54af40 1 -- 172.21.15.38:0/1792112991 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x558423b493a0 msgr2=0x558423b4c200 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:00.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.704+0000 7f8288805640 1 -- 172.21.15.38:0/1792112991 reap_dead start 2024-09-21T03:41:00.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.704+0000 7f828e54af40 1 --2- 172.21.15.38:0/1792112991 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x558423b493a0 0x558423b4c200 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:00.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.704+0000 7f828e54af40 1 -- 172.21.15.38:0/1792112991 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x558423b40260 msgr2=0x558423b42680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:00.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.704+0000 7f828e54af40 1 --2- 172.21.15.38:0/1792112991 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x558423b40260 0x558423b42680 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:00.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.704+0000 7f828e54af40 1 -- 172.21.15.38:0/1792112991 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x558423b350d0 msgr2=0x558423b354e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:00.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.704+0000 7f828e54af40 1 --2- 172.21.15.38:0/1792112991 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x558423b350d0 0x558423b354e0 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:00.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.704+0000 7f828e54af40 1 -- 172.21.15.38:0/1792112991 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f82740775a0 msgr2=0x7f8274079ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:00.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.704+0000 7f828e54af40 1 --2- 172.21.15.38:0/1792112991 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f82740775a0 0x7f8274079ab0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f82780011a0 tx=0x7f8278001110 comp rx=0 tx=0).stop 2024-09-21T03:41:00.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.704+0000 7f828e54af40 1 -- 172.21.15.38:0/1792112991 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558423a99450 msgr2=0x558423a92ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:00.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.704+0000 7f828e54af40 1 --2- 172.21.15.38:0/1792112991 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558423a99450 0x558423a92ac0 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f827c046750 tx=0x7f827c001110 comp rx=0 tx=0).stop 2024-09-21T03:41:00.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.704+0000 7f8288805640 1 -- 172.21.15.38:0/1792112991 reap_dead start 2024-09-21T03:41:00.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.704+0000 7f828e54af40 1 -- 172.21.15.38:0/1792112991 shutdown_connections 2024-09-21T03:41:00.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.704+0000 7f828e54af40 1 -- 172.21.15.38:0/1792112991 >> 172.21.15.38:0/1792112991 conn(0x5584238e3ac0 msgr2=0x558423a7aef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:00.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.704+0000 7f828e54af40 1 -- 172.21.15.38:0/1792112991 shutdown_connections 2024-09-21T03:41:00.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.704+0000 7f828e54af40 1 -- 172.21.15.38:0/1792112991 wait complete. 2024-09-21T03:41:00.710 DEBUG:teuthology.orchestra.run.smithi038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rm 500.00000000 2024-09-21T03:41:00.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.803+0000 7f5808d1ef40 1 Processor -- start 2024-09-21T03:41:00.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.803+0000 7f5808d1ef40 1 -- start start 2024-09-21T03:41:00.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.804+0000 7f5808d1ef40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558191e6bd00 0x558191f25490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:00.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.804+0000 7f5808d1ef40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x558191e97e30 0x558191f220d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:00.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.804+0000 7f5808d1ef40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558191f26950 0x558191f21430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:00.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.804+0000 7f5808d1ef40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x558191db9d60 con 0x558191f26950 2024-09-21T03:41:00.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.804+0000 7f5808d1ef40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x558191e18430 con 0x558191e6bd00 2024-09-21T03:41:00.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.804+0000 7f5808d1ef40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x558191e684f0 con 0x558191e97e30 2024-09-21T03:41:00.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.804+0000 7f5802574640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558191f26950 0x558191f21430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:00.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.804+0000 7f5802574640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558191f26950 0x558191f21430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:39320/0 (socket says 172.21.15.38:39320) 2024-09-21T03:41:00.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.804+0000 7f5802574640 1 -- 172.21.15.38:0/2016900817 learned_addr learned my addr 172.21.15.38:0/2016900817 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:41:00.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.804+0000 7f5801d73640 1 --2- 172.21.15.38:0/2016900817 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558191e6bd00 0x558191f25490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:00.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.804+0000 7f5801572640 1 --2- 172.21.15.38:0/2016900817 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x558191e97e30 0x558191f220d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:00.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.805+0000 7f5802574640 1 -- 172.21.15.38:0/2016900817 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x558191e97e30 msgr2=0x558191f220d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:00.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.805+0000 7f5802574640 1 --2- 172.21.15.38:0/2016900817 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x558191e97e30 0x558191f220d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:00.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.805+0000 7f5802574640 1 -- 172.21.15.38:0/2016900817 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558191e6bd00 msgr2=0x558191f25490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:00.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.805+0000 7f5802574640 1 --2- 172.21.15.38:0/2016900817 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558191e6bd00 0x558191f25490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:00.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.805+0000 7f5802574640 1 -- 172.21.15.38:0/2016900817 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558191f22610 con 0x558191f26950 2024-09-21T03:41:00.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.805+0000 7f5801d73640 1 --2- 172.21.15.38:0/2016900817 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558191e6bd00 0x558191f25490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:41:00.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.805+0000 7f5801572640 1 --2- 172.21.15.38:0/2016900817 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x558191e97e30 0x558191f220d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:41:00.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.805+0000 7f5802574640 1 --2- 172.21.15.38:0/2016900817 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558191f26950 0x558191f21430 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f57fc00b640 tx=0x7f57fc00bb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e474af8abf925a02 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:00.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.805+0000 7f57f3fff640 1 -- 172.21.15.38:0/2016900817 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57fc009990 con 0x558191f26950 2024-09-21T03:41:00.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.805+0000 7f57f3fff640 1 -- 172.21.15.38:0/2016900817 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f57fc009c20 con 0x558191f26950 2024-09-21T03:41:00.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.805+0000 7f57f3fff640 1 -- 172.21.15.38:0/2016900817 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57fc00e6b0 con 0x558191f26950 2024-09-21T03:41:00.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.806+0000 7f5808d1ef40 1 -- 172.21.15.38:0/2016900817 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558191f26950 msgr2=0x558191f21430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:00.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.806+0000 7f5808d1ef40 1 --2- 172.21.15.38:0/2016900817 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558191f26950 0x558191f21430 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f57fc00b640 tx=0x7f57fc00bb00 comp rx=0 tx=0).stop 2024-09-21T03:41:00.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.806+0000 7f5808d1ef40 1 -- 172.21.15.38:0/2016900817 shutdown_connections 2024-09-21T03:41:00.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.806+0000 7f5808d1ef40 1 --2- 172.21.15.38:0/2016900817 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558191f26950 0x558191f21430 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:00.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.806+0000 7f5808d1ef40 1 --2- 172.21.15.38:0/2016900817 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x558191e97e30 0x558191f220d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:00.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.806+0000 7f5808d1ef40 1 --2- 172.21.15.38:0/2016900817 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558191e6bd00 0x558191f25490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:00.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.806+0000 7f5808d1ef40 1 -- 172.21.15.38:0/2016900817 >> 172.21.15.38:0/2016900817 conn(0x558191d8aac0 msgr2=0x558191f24d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:00.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.806+0000 7f5808d1ef40 1 -- 172.21.15.38:0/2016900817 shutdown_connections 2024-09-21T03:41:00.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.806+0000 7f5808d1ef40 1 -- 172.21.15.38:0/2016900817 wait complete. 2024-09-21T03:41:00.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.807+0000 7f5808d1ef40 1 Processor -- start 2024-09-21T03:41:00.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.807+0000 7f5808d1ef40 1 -- start start 2024-09-21T03:41:00.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.808+0000 7f5808d1ef40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558191e6bd00 0x558191f3c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:00.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.808+0000 7f5808d1ef40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x558191e97e30 0x558191f3c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:00.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.808+0000 7f5808d1ef40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558191f26950 0x558191fc7e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:00.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.808+0000 7f5808d1ef40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x558191fc8ef0 con 0x558191f26950 2024-09-21T03:41:00.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.808+0000 7f5808d1ef40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x558191f20740 con 0x558191e6bd00 2024-09-21T03:41:00.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.808+0000 7f5808d1ef40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x558191f20200 con 0x558191e97e30 2024-09-21T03:41:00.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.808+0000 7f5802574640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558191f26950 0x558191fc7e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:00.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.808+0000 7f5802574640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558191f26950 0x558191fc7e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:39334/0 (socket says 172.21.15.38:39334) 2024-09-21T03:41:00.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.808+0000 7f5802574640 1 -- 172.21.15.38:0/2428973272 learned_addr learned my addr 172.21.15.38:0/2428973272 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:41:00.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.808+0000 7f5801572640 1 --2- 172.21.15.38:0/2428973272 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x558191e97e30 0x558191f3c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:00.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.808+0000 7f5801d73640 1 --2- 172.21.15.38:0/2428973272 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558191e6bd00 0x558191f3c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:00.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.808+0000 7f5802574640 1 -- 172.21.15.38:0/2428973272 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x558191e97e30 msgr2=0x558191f3c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:00.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.808+0000 7f5802574640 1 --2- 172.21.15.38:0/2428973272 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x558191e97e30 0x558191f3c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:00.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.808+0000 7f5802574640 1 -- 172.21.15.38:0/2428973272 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558191e6bd00 msgr2=0x558191f3c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:00.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.808+0000 7f5802574640 1 --2- 172.21.15.38:0/2428973272 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558191e6bd00 0x558191f3c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:00.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.808+0000 7f5802574640 1 -- 172.21.15.38:0/2428973272 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558191f1ff20 con 0x558191f26950 2024-09-21T03:41:00.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.808+0000 7f5801d73640 1 --2- 172.21.15.38:0/2428973272 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558191e6bd00 0x558191f3c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:41:00.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.808+0000 7f5801572640 1 --2- 172.21.15.38:0/2428973272 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x558191e97e30 0x558191f3c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:41:00.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.809+0000 7f5802574640 1 --2- 172.21.15.38:0/2428973272 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558191f26950 0x558191fc7e80 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f57fc003300 tx=0x7f57fc00e6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:00.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.809+0000 7f57f2ffd640 1 -- 172.21.15.38:0/2428973272 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57fc00ea70 con 0x558191f26950 2024-09-21T03:41:00.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.809+0000 7f57f2ffd640 1 -- 172.21.15.38:0/2428973272 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f57fc00ec30 con 0x558191f26950 2024-09-21T03:41:00.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.809+0000 7f5808d1ef40 1 -- 172.21.15.38:0/2428973272 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x558191f3cd70 con 0x558191f26950 2024-09-21T03:41:00.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.809+0000 7f57f2ffd640 1 -- 172.21.15.38:0/2428973272 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57fc020680 con 0x558191f26950 2024-09-21T03:41:00.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.809+0000 7f5808d1ef40 1 -- 172.21.15.38:0/2428973272 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558191f3dc70 con 0x558191f26950 2024-09-21T03:41:00.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.810+0000 7f57f2ffd640 1 -- 172.21.15.38:0/2428973272 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f57fc029440 con 0x558191f26950 2024-09-21T03:41:00.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.811+0000 7f57f2ffd640 1 --2- 172.21.15.38:0/2428973272 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f57e4077570 0x7f57e4079a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:00.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.811+0000 7f57f2ffd640 1 -- 172.21.15.38:0/2428973272 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 6906+0+0 (secure 0 0 0) 0x7f57fc09ba10 con 0x558191f26950 2024-09-21T03:41:00.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.811+0000 7f5808d1ef40 1 --2- 172.21.15.38:0/2428973272 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x558191fcb250 0x558191fcb660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:00.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.811+0000 7f5808d1ef40 1 -- 172.21.15.38:0/2428973272 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:1 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e103) -- 0x558191fcbba0 con 0x558191fcb250 2024-09-21T03:41:00.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.811+0000 7f5801d73640 1 --2- 172.21.15.38:0/2428973272 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f57e4077570 0x7f57e4079a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:00.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.811+0000 7f5801572640 1 --2- 172.21.15.38:0/2428973272 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x558191fcb250 0x558191fcb660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:00.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.812+0000 7f5801572640 1 --2- 172.21.15.38:0/2428973272 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x558191fcb250 0x558191fcb660 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:00.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.812+0000 7f5801d73640 1 --2- 172.21.15.38:0/2428973272 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f57e4077570 0x7f57e4079a80 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f57f4000f60 tx=0x7f57f4003500 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:00.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.813+0000 7f5801572640 1 -- 172.21.15.38:0/2428973272 <== osd.3 v2:172.21.15.38:6808/1286810294 1 ==== osd_op_reply(1 500.00000000 [delete] v103'81 uv81 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f57f8054370 con 0x558191fcb250 2024-09-21T03:41:00.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.813+0000 7f5808d1ef40 1 -- 172.21.15.38:0/2428973272 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x558191fcb250 msgr2=0x558191fcb660 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:00.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.813+0000 7f5808d1ef40 1 --2- 172.21.15.38:0/2428973272 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x558191fcb250 0x558191fcb660 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:00.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.813+0000 7f5808d1ef40 1 -- 172.21.15.38:0/2428973272 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f57e4077570 msgr2=0x7f57e4079a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:00.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.813+0000 7f5808d1ef40 1 --2- 172.21.15.38:0/2428973272 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f57e4077570 0x7f57e4079a80 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f57f4000f60 tx=0x7f57f4003500 comp rx=0 tx=0).stop 2024-09-21T03:41:00.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.813+0000 7f5808d1ef40 1 -- 172.21.15.38:0/2428973272 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558191f26950 msgr2=0x558191fc7e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:00.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.813+0000 7f5808d1ef40 1 --2- 172.21.15.38:0/2428973272 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558191f26950 0x558191fc7e80 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f57fc003300 tx=0x7f57fc00e6f0 comp rx=0 tx=0).stop 2024-09-21T03:41:00.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.813+0000 7f5802574640 1 -- 172.21.15.38:0/2428973272 reap_dead start 2024-09-21T03:41:00.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.813+0000 7f5808d1ef40 1 -- 172.21.15.38:0/2428973272 shutdown_connections 2024-09-21T03:41:00.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.813+0000 7f5808d1ef40 1 -- 172.21.15.38:0/2428973272 >> 172.21.15.38:0/2428973272 conn(0x558191d8aac0 msgr2=0x558191f24d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:00.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.813+0000 7f5808d1ef40 1 -- 172.21.15.38:0/2428973272 shutdown_connections 2024-09-21T03:41:00.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.813+0000 7f5808d1ef40 1 -- 172.21.15.38:0/2428973272 wait complete. 2024-09-21T03:41:00.819 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-21T03:41:00.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.935+0000 7f88a5d71640 1 -- 172.21.15.38:0/4182175866 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f88a00d0d80 msgr2=0x7f88a00b5be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:00.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.935+0000 7f88a5d71640 1 --2- 172.21.15.38:0/4182175866 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f88a00d0d80 0x7f88a00b5be0 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f88a0071cd0 tx=0x7f889802aa30 comp rx=0 tx=0).stop 2024-09-21T03:41:00.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.935+0000 7f88a5d71640 1 -- 172.21.15.38:0/4182175866 shutdown_connections 2024-09-21T03:41:00.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.935+0000 7f88a5d71640 1 --2- 172.21.15.38:0/4182175866 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f88a00d0d80 0x7f88a00b5be0 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:00.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.935+0000 7f88a5d71640 1 --2- 172.21.15.38:0/4182175866 >> v2:172.21.15.195:3300/0 conn(0x7f88a00ccc40 0x7f88a015ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:00.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.935+0000 7f88a5d71640 1 -- 172.21.15.38:0/4182175866 >> 172.21.15.38:0/4182175866 conn(0x7f88a00b46e0 msgr2=0x7f88a00d23e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:00.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.935+0000 7f88a5d71640 1 -- 172.21.15.38:0/4182175866 shutdown_connections 2024-09-21T03:41:00.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.936+0000 7f88a5d71640 1 -- 172.21.15.38:0/4182175866 wait complete. 2024-09-21T03:41:00.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.937+0000 7f88a5d71640 1 Processor -- start 2024-09-21T03:41:00.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.937+0000 7f88a5d71640 1 -- start start 2024-09-21T03:41:00.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.939+0000 7f88a5d71640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f88a00ccc40 0x7f88a00a9450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:00.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.939+0000 7f88a5d71640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f88a00cd440 0x7f88a015de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:00.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.939+0000 7f88a5d71640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f88a015f920 0x7f88a015fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:00.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.939+0000 7f88a5d71640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f88a00b6650 con 0x7f88a00cd440 2024-09-21T03:41:00.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.939+0000 7f88a5d71640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f88a00b64d0 con 0x7f88a00ccc40 2024-09-21T03:41:00.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.939+0000 7f88a5d71640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f88a00b67d0 con 0x7f88a015f920 2024-09-21T03:41:00.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.939+0000 7f889effd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f88a00cd440 0x7f88a015de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:00.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.939+0000 7f889effd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f88a00cd440 0x7f88a015de10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:39348/0 (socket says 172.21.15.38:39348) 2024-09-21T03:41:00.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.939+0000 7f889effd640 1 -- 172.21.15.38:0/3517488544 learned_addr learned my addr 172.21.15.38:0/3517488544 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:41:00.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.939+0000 7f889f7fe640 1 --2- 172.21.15.38:0/3517488544 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f88a00ccc40 0x7f88a00a9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:00.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.939+0000 7f889ffff640 1 --2- 172.21.15.38:0/3517488544 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f88a015f920 0x7f88a015fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:00.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.939+0000 7f889effd640 1 -- 172.21.15.38:0/3517488544 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f88a015f920 msgr2=0x7f88a015fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:00.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.939+0000 7f889effd640 1 --2- 172.21.15.38:0/3517488544 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f88a015f920 0x7f88a015fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:00.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.939+0000 7f889effd640 1 -- 172.21.15.38:0/3517488544 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f88a00ccc40 msgr2=0x7f88a00a9450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:00.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.939+0000 7f889effd640 1 --2- 172.21.15.38:0/3517488544 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f88a00ccc40 0x7f88a00a9450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:00.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.939+0000 7f889effd640 1 -- 172.21.15.38:0/3517488544 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88a00a9990 con 0x7f88a00cd440 2024-09-21T03:41:00.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.939+0000 7f889effd640 1 --2- 172.21.15.38:0/3517488544 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f88a00cd440 0x7f88a015de10 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f889400c3f0 tx=0x7f889400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:00.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.940+0000 7f8887fff640 1 -- 172.21.15.38:0/3517488544 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88940052d0 con 0x7f88a00cd440 2024-09-21T03:41:00.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.940+0000 7f8887fff640 1 -- 172.21.15.38:0/3517488544 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8894007020 con 0x7f88a00cd440 2024-09-21T03:41:00.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.940+0000 7f88a5d71640 1 -- 172.21.15.38:0/3517488544 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88a015cd70 con 0x7f88a00cd440 2024-09-21T03:41:00.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.940+0000 7f8887fff640 1 -- 172.21.15.38:0/3517488544 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8894007470 con 0x7f88a00cd440 2024-09-21T03:41:00.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.940+0000 7f88a5d71640 1 -- 172.21.15.38:0/3517488544 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88a015d330 con 0x7f88a00cd440 2024-09-21T03:41:00.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.941+0000 7f88a5d71640 1 -- 172.21.15.38:0/3517488544 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88a00b6cf0 con 0x7f88a00cd440 2024-09-21T03:41:00.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.942+0000 7f8887fff640 1 -- 172.21.15.38:0/3517488544 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f88940067d0 con 0x7f88a00cd440 2024-09-21T03:41:00.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.942+0000 7f8887fff640 1 --2- 172.21.15.38:0/3517488544 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f888c078b90 0x7f888c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:00.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.942+0000 7f8887fff640 1 -- 172.21.15.38:0/3517488544 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 6906+0+0 (secure 0 0 0) 0x7f889409a4d0 con 0x7f88a00cd440 2024-09-21T03:41:00.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.943+0000 7f889f7fe640 1 --2- 172.21.15.38:0/3517488544 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f888c078b90 0x7f888c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:00.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.943+0000 7f889f7fe640 1 --2- 172.21.15.38:0/3517488544 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f888c078b90 0x7f888c07b0a0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f8890009b30 tx=0x7f8890005f90 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:00.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:00.947+0000 7f8887fff640 1 -- 172.21.15.38:0/3517488544 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f8894061ea0 con 0x7f88a00cd440 2024-09-21T03:41:01.118 DEBUG:teuthology.orchestra.run.smithi038:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T03:41:01.121 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T03:41:01.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:01.175+0000 7f88a5d71640 1 -- 172.21.15.38:0/3517488544 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f88a00d0ee0 con 0x7f88a00cd440 2024-09-21T03:41:01.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:01.463+0000 7f8887fff640 1 -- 172.21.15.38:0/3517488544 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v127) ==== 162+0+0 (secure 0 0 0) 0x7f88940668d0 con 0x7f88a00cd440 2024-09-21T03:41:01.466 INFO:teuthology.orchestra.run.smithi038.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-21T03:41:01.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:01.465+0000 7f88a5d71640 1 -- 172.21.15.38:0/3517488544 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f888c078b90 msgr2=0x7f888c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:01.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:01.465+0000 7f88a5d71640 1 --2- 172.21.15.38:0/3517488544 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f888c078b90 0x7f888c07b0a0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f8890009b30 tx=0x7f8890005f90 comp rx=0 tx=0).stop 2024-09-21T03:41:01.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:01.465+0000 7f88a5d71640 1 -- 172.21.15.38:0/3517488544 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f88a00cd440 msgr2=0x7f88a015de10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:01.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:01.465+0000 7f88a5d71640 1 --2- 172.21.15.38:0/3517488544 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f88a00cd440 0x7f88a015de10 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f889400c3f0 tx=0x7f889400c8b0 comp rx=0 tx=0).stop 2024-09-21T03:41:01.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:01.465+0000 7f88a5d71640 1 -- 172.21.15.38:0/3517488544 shutdown_connections 2024-09-21T03:41:01.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:01.465+0000 7f88a5d71640 1 --2- 172.21.15.38:0/3517488544 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f888c078b90 0x7f888c07b0a0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:01.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:01.465+0000 7f88a5d71640 1 --2- 172.21.15.38:0/3517488544 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f88a015f920 0x7f88a015fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:01.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:01.465+0000 7f88a5d71640 1 --2- 172.21.15.38:0/3517488544 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f88a00cd440 0x7f88a015de10 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:01.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:01.465+0000 7f88a5d71640 1 --2- 172.21.15.38:0/3517488544 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f88a00ccc40 0x7f88a00a9450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:01.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:01.465+0000 7f88a5d71640 1 -- 172.21.15.38:0/3517488544 >> 172.21.15.38:0/3517488544 conn(0x7f88a00b46e0 msgr2=0x7f88a0071350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:01.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:01.466+0000 7f88a5d71640 1 -- 172.21.15.38:0/3517488544 shutdown_connections 2024-09-21T03:41:01.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:01.466+0000 7f88a5d71640 1 -- 172.21.15.38:0/3517488544 wait complete. 2024-09-21T03:41:01.886 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:41:02.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:02.015+0000 7f7fd4b52640 1 -- 172.21.15.38:0/3193984332 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7fd0162590 msgr2=0x7f7fd01629a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:02.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:02.015+0000 7f7fd4b52640 1 --2- 172.21.15.38:0/3193984332 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7fd0162590 0x7f7fd01629a0 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f7fd008d890 tx=0x7f7fc402a560 comp rx=0 tx=0).stop 2024-09-21T03:41:02.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:02.015+0000 7f7fd4b52640 1 -- 172.21.15.38:0/3193984332 shutdown_connections 2024-09-21T03:41:02.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:02.015+0000 7f7fd4b52640 1 --2- 172.21.15.38:0/3193984332 >> v2:172.21.15.195:3300/0 conn(0x7f7fd00b2e60 0x7f7fd00b3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:02.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:02.015+0000 7f7fd4b52640 1 --2- 172.21.15.38:0/3193984332 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7fd0162590 0x7f7fd01629a0 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:02.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:02.015+0000 7f7fd4b52640 1 --2- 172.21.15.38:0/3193984332 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7fd00cfe00 0x7f7fd00d2a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:02.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:02.015+0000 7f7fd4b52640 1 -- 172.21.15.38:0/3193984332 >> 172.21.15.38:0/3193984332 conn(0x7f7fd00d4440 msgr2=0x7f7fd00d4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:02.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:02.015+0000 7f7fd4b52640 1 -- 172.21.15.38:0/3193984332 shutdown_connections 2024-09-21T03:41:02.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:02.015+0000 7f7fd4b52640 1 -- 172.21.15.38:0/3193984332 wait complete. 2024-09-21T03:41:02.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:02.017+0000 7f7fd4b52640 1 Processor -- start 2024-09-21T03:41:02.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:02.017+0000 7f7fd4b52640 1 -- start start 2024-09-21T03:41:02.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:02.018+0000 7f7fd4b52640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7fd00b2e60 0x7f7fd01f29a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:02.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:02.018+0000 7f7fd4b52640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7fd00cfe00 0x7f7fd015b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:02.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:02.018+0000 7f7fd4b52640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7fd0162590 0x7f7fd015d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:02.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:02.018+0000 7f7fd4b52640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f7fd00b37b0 con 0x7f7fd00b2e60 2024-09-21T03:41:02.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:02.018+0000 7f7fd4b52640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f7fd00cf5a0 con 0x7f7fd00cfe00 2024-09-21T03:41:02.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:02.018+0000 7f7fd4b52640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f7fd00b3930 con 0x7f7fd0162590 2024-09-21T03:41:02.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:02.019+0000 7f7fce59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7fd00b2e60 0x7f7fd01f29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:02.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:02.019+0000 7f7fce59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7fd00b2e60 0x7f7fd01f29a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:39368/0 (socket says 172.21.15.38:39368) 2024-09-21T03:41:02.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:02.019+0000 7f7fce59c640 1 -- 172.21.15.38:0/524687571 learned_addr learned my addr 172.21.15.38:0/524687571 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:41:02.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:02.019+0000 7f7fced9d640 1 --2- 172.21.15.38:0/524687571 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7fd0162590 0x7f7fd015d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:02.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:02.019+0000 7f7fcdd9b640 1 --2- 172.21.15.38:0/524687571 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7fd00cfe00 0x7f7fd015b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:02.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:02.019+0000 7f7fce59c640 1 -- 172.21.15.38:0/524687571 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7fd0162590 msgr2=0x7f7fd015d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:02.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:02.019+0000 7f7fce59c640 1 --2- 172.21.15.38:0/524687571 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7fd0162590 0x7f7fd015d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:02.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:02.019+0000 7f7fce59c640 1 -- 172.21.15.38:0/524687571 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7fd00cfe00 msgr2=0x7f7fd015b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:02.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:02.019+0000 7f7fce59c640 1 --2- 172.21.15.38:0/524687571 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7fd00cfe00 0x7f7fd015b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:02.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:02.019+0000 7f7fce59c640 1 -- 172.21.15.38:0/524687571 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fd01f5920 con 0x7f7fd00b2e60 2024-09-21T03:41:02.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:02.019+0000 7f7fce59c640 1 --2- 172.21.15.38:0/524687571 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7fd00b2e60 0x7f7fd01f29a0 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f7fc000d700 tx=0x7f7fc000dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:02.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:02.019+0000 7f7fbeffd640 1 -- 172.21.15.38:0/524687571 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fc0017070 con 0x7f7fd00b2e60 2024-09-21T03:41:02.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:02.019+0000 7f7fbeffd640 1 -- 172.21.15.38:0/524687571 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7fc0005310 con 0x7f7fd00b2e60 2024-09-21T03:41:02.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:02.019+0000 7f7fd4b52640 1 -- 172.21.15.38:0/524687571 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fd015bcd0 con 0x7f7fd00b2e60 2024-09-21T03:41:02.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:02.019+0000 7f7fbeffd640 1 -- 172.21.15.38:0/524687571 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fc00097c0 con 0x7f7fd00b2e60 2024-09-21T03:41:02.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:02.020+0000 7f7fd4b52640 1 -- 172.21.15.38:0/524687571 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fd015c290 con 0x7f7fd00b2e60 2024-09-21T03:41:02.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:02.021+0000 7f7fd4b52640 1 -- 172.21.15.38:0/524687571 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7fd00d2a20 con 0x7f7fd00b2e60 2024-09-21T03:41:02.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:02.021+0000 7f7fbeffd640 1 -- 172.21.15.38:0/524687571 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f7fc0009960 con 0x7f7fd00b2e60 2024-09-21T03:41:02.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:02.022+0000 7f7fbeffd640 1 --2- 172.21.15.38:0/524687571 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7fb0078bc0 0x7f7fb007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:02.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:02.022+0000 7f7fbeffd640 1 -- 172.21.15.38:0/524687571 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 6906+0+0 (secure 0 0 0) 0x7f7fc009b1c0 con 0x7f7fd00b2e60 2024-09-21T03:41:02.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:02.022+0000 7f7fcdd9b640 1 --2- 172.21.15.38:0/524687571 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7fb0078bc0 0x7f7fb007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:02.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:02.023+0000 7f7fcdd9b640 1 --2- 172.21.15.38:0/524687571 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7fb0078bc0 0x7f7fb007b0d0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f7fc4033000 tx=0x7f7fc4004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:02.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:02.027+0000 7f7fbeffd640 1 -- 172.21.15.38:0/524687571 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f7fc0062af0 con 0x7f7fd00b2e60 2024-09-21T03:41:02.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:02.250+0000 7f7fd4b52640 1 -- 172.21.15.38:0/524687571 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7fd00d49f0 con 0x7f7fd00b2e60 2024-09-21T03:41:02.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:02.251+0000 7f7fbeffd640 1 -- 172.21.15.38:0/524687571 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 128 v128) ==== 95+0+5224 (secure 0 0 0) 0x7f7fc0067520 con 0x7f7fd00b2e60 2024-09-21T03:41:02.254 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:41:02.254 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":128,"btime":"2024-09-21T03:41:01:468393+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6086,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/965972663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":965972663},{"type":"v1","addr":"172.21.15.195:6836","nonce":965972663}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6700,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/3176122334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":3176122334},{"type":"v1","addr":"172.21.15.195:6837","nonce":3176122334}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":8061,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/2748959970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":2748959970},{"type":"v1","addr":"172.21.15.38:6835","nonce":2748959970}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":122}],"filesystems":[{"mdsmap":{"epoch":128,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:40:02.319282+0000","modified":"2024-09-21T03:41:01.468362+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":103,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8109},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8109":{"gid":8109,"name":"c","rank":0,"incarnation":128,"state":"up:replay","state_seq":1,"addr":"172.21.15.38:6833/121492560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":121492560},{"type":"v1","addr":"172.21.15.38:6833","nonce":121492560}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-21T03:41:02.255 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 128 2024-09-21T03:41:02.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:02.253+0000 7f7fd4b52640 1 -- 172.21.15.38:0/524687571 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7fb0078bc0 msgr2=0x7f7fb007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:02.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:02.253+0000 7f7fd4b52640 1 --2- 172.21.15.38:0/524687571 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7fb0078bc0 0x7f7fb007b0d0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f7fc4033000 tx=0x7f7fc4004730 comp rx=0 tx=0).stop 2024-09-21T03:41:02.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:02.254+0000 7f7fd4b52640 1 -- 172.21.15.38:0/524687571 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7fd00b2e60 msgr2=0x7f7fd01f29a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:02.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:02.254+0000 7f7fd4b52640 1 --2- 172.21.15.38:0/524687571 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7fd00b2e60 0x7f7fd01f29a0 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f7fc000d700 tx=0x7f7fc000dbc0 comp rx=0 tx=0).stop 2024-09-21T03:41:02.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:02.254+0000 7f7fd4b52640 1 -- 172.21.15.38:0/524687571 shutdown_connections 2024-09-21T03:41:02.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:02.254+0000 7f7fd4b52640 1 --2- 172.21.15.38:0/524687571 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7fb0078bc0 0x7f7fb007b0d0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:02.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:02.254+0000 7f7fd4b52640 1 --2- 172.21.15.38:0/524687571 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7fd0162590 0x7f7fd015d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:02.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:02.254+0000 7f7fd4b52640 1 --2- 172.21.15.38:0/524687571 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7fd00cfe00 0x7f7fd015b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:02.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:02.254+0000 7f7fd4b52640 1 --2- 172.21.15.38:0/524687571 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7fd00b2e60 0x7f7fd01f29a0 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:02.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:02.254+0000 7f7fd4b52640 1 -- 172.21.15.38:0/524687571 >> 172.21.15.38:0/524687571 conn(0x7f7fd00d4440 msgr2=0x7f7fd01617b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:02.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:02.254+0000 7f7fd4b52640 1 -- 172.21.15.38:0/524687571 shutdown_connections 2024-09-21T03:41:02.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:02.254+0000 7f7fd4b52640 1 -- 172.21.15.38:0/524687571 wait complete. 2024-09-21T03:41:02.268 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 128, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T03:40:02.319282+0000', 'modified': '2024-09-21T03:41:01.468362+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 103, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8109}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8109': {'gid': 8109, 'name': 'c', 'rank': 0, 'incarnation': 128, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.38:6833/121492560', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.38:6832', 'nonce': 121492560}, {'type': 'v1', 'addr': '172.21.15.38:6833', 'nonce': 121492560}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-09-21T03:41:02.268 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8109:up:replay 2024-09-21T03:41:02.268 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2024-09-21T03:41:05.469 INFO:tasks.ceph.mds.c.smithi038.stderr:2024-09-21T03:41:05.467+0000 7fdeedc4b640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T03:41:07.672 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:41:07.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:07.802+0000 7f56e5694640 1 -- 172.21.15.38:0/1360283020 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f56c8004800 msgr2=0x7f56c8004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:07.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:07.802+0000 7f56e5694640 1 --2- 172.21.15.38:0/1360283020 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f56c8004800 0x7f56c8004c10 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f56d4005fd0 tx=0x7f56c8005830 comp rx=0 tx=0).stop 2024-09-21T03:41:07.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:07.802+0000 7f56df7fe640 1 -- 172.21.15.38:0/1360283020 reap_dead start 2024-09-21T03:41:07.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:07.802+0000 7f56e5694640 1 -- 172.21.15.38:0/1360283020 shutdown_connections 2024-09-21T03:41:07.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:07.802+0000 7f56e5694640 1 -- 172.21.15.38:0/1360283020 >> 172.21.15.38:0/1360283020 conn(0x7f56e00b6b20 msgr2=0x7f56e00d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:07.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:07.802+0000 7f56e5694640 1 -- 172.21.15.38:0/1360283020 shutdown_connections 2024-09-21T03:41:07.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:07.802+0000 7f56e5694640 1 -- 172.21.15.38:0/1360283020 wait complete. 2024-09-21T03:41:07.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:07.804+0000 7f56e5694640 1 Processor -- start 2024-09-21T03:41:07.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:07.804+0000 7f56e5694640 1 -- start start 2024-09-21T03:41:07.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:07.806+0000 7f56e5694640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f56c8004800 0x7f56e01f46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:07.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:07.806+0000 7f56e5694640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f56e00d1540 0x7f56e015a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:07.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:07.806+0000 7f56e5694640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f56e00ab5b0 0x7f56e00cce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:07.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:07.806+0000 7f56e5694640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f56e0162810 con 0x7f56e00ab5b0 2024-09-21T03:41:07.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:07.806+0000 7f56e5694640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f56e0162690 con 0x7f56e00d1540 2024-09-21T03:41:07.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:07.806+0000 7f56e5694640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f56e0162990 con 0x7f56c8004800 2024-09-21T03:41:07.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:07.806+0000 7f56df7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f56e00ab5b0 0x7f56e00cce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:07.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:07.806+0000 7f56df7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f56e00ab5b0 0x7f56e00cce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50414/0 (socket says 172.21.15.38:50414) 2024-09-21T03:41:07.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:07.806+0000 7f56df7fe640 1 -- 172.21.15.38:0/3732559468 learned_addr learned my addr 172.21.15.38:0/3732559468 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:41:07.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:07.806+0000 7f56de7fc640 1 --2- 172.21.15.38:0/3732559468 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f56e00d1540 0x7f56e015a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:07.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:07.806+0000 7f56deffd640 1 --2- 172.21.15.38:0/3732559468 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f56c8004800 0x7f56e01f46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:07.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:07.806+0000 7f56df7fe640 1 -- 172.21.15.38:0/3732559468 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f56c8004800 msgr2=0x7f56e01f46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:07.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:07.806+0000 7f56df7fe640 1 --2- 172.21.15.38:0/3732559468 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f56c8004800 0x7f56e01f46a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:07.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:07.806+0000 7f56df7fe640 1 -- 172.21.15.38:0/3732559468 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f56e00d1540 msgr2=0x7f56e015a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:07.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:07.806+0000 7f56df7fe640 1 --2- 172.21.15.38:0/3732559468 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f56e00d1540 0x7f56e015a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:07.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:07.806+0000 7f56df7fe640 1 -- 172.21.15.38:0/3732559468 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56e015ad50 con 0x7f56e00ab5b0 2024-09-21T03:41:07.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:07.806+0000 7f56df7fe640 1 --2- 172.21.15.38:0/3732559468 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f56e00ab5b0 0x7f56e00cce00 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f56d8000fb0 tx=0x7f56d80039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:07.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:07.807+0000 7f56c77fe640 1 -- 172.21.15.38:0/3732559468 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56d8011840 con 0x7f56e00ab5b0 2024-09-21T03:41:07.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:07.807+0000 7f56c77fe640 1 -- 172.21.15.38:0/3732559468 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f56d8018070 con 0x7f56e00ab5b0 2024-09-21T03:41:07.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:07.807+0000 7f56e5694640 1 -- 172.21.15.38:0/3732559468 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56e0159770 con 0x7f56e00ab5b0 2024-09-21T03:41:07.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:07.807+0000 7f56c77fe640 1 -- 172.21.15.38:0/3732559468 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56d8011c80 con 0x7f56e00ab5b0 2024-09-21T03:41:07.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:07.807+0000 7f56e5694640 1 -- 172.21.15.38:0/3732559468 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56e0159d30 con 0x7f56e00ab5b0 2024-09-21T03:41:07.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:07.808+0000 7f56e5694640 1 -- 172.21.15.38:0/3732559468 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56e0162f00 con 0x7f56e00ab5b0 2024-09-21T03:41:07.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:07.808+0000 7f56c77fe640 1 -- 172.21.15.38:0/3732559468 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f56d801d460 con 0x7f56e00ab5b0 2024-09-21T03:41:07.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:07.809+0000 7f56c77fe640 1 --2- 172.21.15.38:0/3732559468 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f56cc078bf0 0x7f56cc07b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:07.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:07.809+0000 7f56c77fe640 1 -- 172.21.15.38:0/3732559468 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 6906+0+0 (secure 0 0 0) 0x7f56d809b790 con 0x7f56e00ab5b0 2024-09-21T03:41:07.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:07.809+0000 7f56deffd640 1 --2- 172.21.15.38:0/3732559468 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f56cc078bf0 0x7f56cc07b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:07.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:07.810+0000 7f56deffd640 1 --2- 172.21.15.38:0/3732559468 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f56cc078bf0 0x7f56cc07b100 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f56d0000c00 tx=0x7f56d0006b70 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:07.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:07.814+0000 7f56c77fe640 1 -- 172.21.15.38:0/3732559468 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f56e0162bd0 con 0x7f56e00ab5b0 2024-09-21T03:41:08.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.039+0000 7f56e5694640 1 -- 172.21.15.38:0/3732559468 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f56e006d160 con 0x7f56e00ab5b0 2024-09-21T03:41:08.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.040+0000 7f56c77fe640 1 -- 172.21.15.38:0/3732559468 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 131 v131) ==== 95+0+5231 (secure 0 0 0) 0x7f56d8011e20 con 0x7f56e00ab5b0 2024-09-21T03:41:08.042 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:41:08.042 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":131,"btime":"2024-09-21T03:41:04:479641+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6086,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/965972663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":965972663},{"type":"v1","addr":"172.21.15.195:6836","nonce":965972663}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6700,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/3176122334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":3176122334},{"type":"v1","addr":"172.21.15.195:6837","nonce":3176122334}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":8061,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/2748959970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":2748959970},{"type":"v1","addr":"172.21.15.38:6835","nonce":2748959970}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":122}],"filesystems":[{"mdsmap":{"epoch":131,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:40:02.319282+0000","modified":"2024-09-21T03:41:04.479633+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":103,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8109},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8109":{"gid":8109,"name":"c","rank":0,"incarnation":128,"state":"up:active","state_seq":4,"addr":"172.21.15.38:6833/121492560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":121492560},{"type":"v1","addr":"172.21.15.38:6833","nonce":121492560}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8109,"qdb_cluster":[8109]},"id":6}]} 2024-09-21T03:41:08.042 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 131 2024-09-21T03:41:08.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.042+0000 7f56e5694640 1 -- 172.21.15.38:0/3732559468 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f56cc078bf0 msgr2=0x7f56cc07b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:08.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.042+0000 7f56e5694640 1 --2- 172.21.15.38:0/3732559468 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f56cc078bf0 0x7f56cc07b100 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f56d0000c00 tx=0x7f56d0006b70 comp rx=0 tx=0).stop 2024-09-21T03:41:08.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.042+0000 7f56e5694640 1 -- 172.21.15.38:0/3732559468 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f56e00ab5b0 msgr2=0x7f56e00cce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:08.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.042+0000 7f56e5694640 1 --2- 172.21.15.38:0/3732559468 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f56e00ab5b0 0x7f56e00cce00 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f56d8000fb0 tx=0x7f56d80039c0 comp rx=0 tx=0).stop 2024-09-21T03:41:08.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.042+0000 7f56e5694640 1 -- 172.21.15.38:0/3732559468 shutdown_connections 2024-09-21T03:41:08.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.042+0000 7f56e5694640 1 --2- 172.21.15.38:0/3732559468 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f56e00ab5b0 0x7f56e00cce00 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:08.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.042+0000 7f56e5694640 1 --2- 172.21.15.38:0/3732559468 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f56e00d1540 0x7f56e015a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:08.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.042+0000 7f56e5694640 1 --2- 172.21.15.38:0/3732559468 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f56cc078bf0 0x7f56cc07b100 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:08.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.042+0000 7f56e5694640 1 --2- 172.21.15.38:0/3732559468 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f56c8004800 0x7f56e01f46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:08.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.042+0000 7f56e5694640 1 -- 172.21.15.38:0/3732559468 >> 172.21.15.38:0/3732559468 conn(0x7f56e00b6b20 msgr2=0x7f56e015dac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:08.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.042+0000 7f56e5694640 1 -- 172.21.15.38:0/3732559468 shutdown_connections 2024-09-21T03:41:08.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.042+0000 7f56e5694640 1 -- 172.21.15.38:0/3732559468 wait complete. 2024-09-21T03:41:08.055 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 131, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T03:40:02.319282+0000', 'modified': '2024-09-21T03:41:04.479633+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 103, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8109}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8109': {'gid': 8109, 'name': 'c', 'rank': 0, 'incarnation': 128, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.38:6833/121492560', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.38:6832', 'nonce': 121492560}, {'type': 'v1', 'addr': '172.21.15.38:6833', 'nonce': 121492560}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 8109, 'qdb_cluster': [8109]} 2024-09-21T03:41:08.055 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-21T03:41:08.055 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-21T03:41:08.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.171+0000 7fd367cbd640 1 -- 172.21.15.38:0/2947987046 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd3600b9ad0 msgr2=0x7fd3600d0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:08.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.171+0000 7fd367cbd640 1 --2- 172.21.15.38:0/2947987046 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd3600b9ad0 0x7fd3600d0be0 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7fd3540098e0 tx=0x7fd35402a530 comp rx=0 tx=0).stop 2024-09-21T03:41:08.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.171+0000 7fd367cbd640 1 -- 172.21.15.38:0/2947987046 shutdown_connections 2024-09-21T03:41:08.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.171+0000 7fd367cbd640 1 --2- 172.21.15.38:0/2947987046 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd3600b9ad0 0x7fd3600d0be0 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:08.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.171+0000 7fd367cbd640 1 --2- 172.21.15.38:0/2947987046 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd36015ee80 0x7fd3600d0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:08.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.171+0000 7fd367cbd640 1 -- 172.21.15.38:0/2947987046 >> 172.21.15.38:0/2947987046 conn(0x7fd36009d830 msgr2=0x7fd3600b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:08.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.171+0000 7fd367cbd640 1 -- 172.21.15.38:0/2947987046 shutdown_connections 2024-09-21T03:41:08.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.171+0000 7fd367cbd640 1 -- 172.21.15.38:0/2947987046 wait complete. 2024-09-21T03:41:08.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.173+0000 7fd367cbd640 1 Processor -- start 2024-09-21T03:41:08.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.173+0000 7fd367cbd640 1 -- start start 2024-09-21T03:41:08.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.174+0000 7fd367cbd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd3600b9ad0 0x7fd36006f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:08.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.174+0000 7fd367cbd640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd3600d1120 0x7fd360157500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:08.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.174+0000 7fd367cbd640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd36015ee80 0x7fd360157a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:08.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.174+0000 7fd367cbd640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fd3600b6700 con 0x7fd3600b9ad0 2024-09-21T03:41:08.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.174+0000 7fd367cbd640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd3600b6580 con 0x7fd36015ee80 2024-09-21T03:41:08.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.174+0000 7fd367cbd640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fd3600b6880 con 0x7fd3600d1120 2024-09-21T03:41:08.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.174+0000 7fd365a59640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd3600b9ad0 0x7fd36006f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:08.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.174+0000 7fd365a59640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd3600b9ad0 0x7fd36006f010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50420/0 (socket says 172.21.15.38:50420) 2024-09-21T03:41:08.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.174+0000 7fd365a59640 1 -- 172.21.15.38:0/3023561834 learned_addr learned my addr 172.21.15.38:0/3023561834 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:41:08.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.174+0000 7fd365258640 1 --2- 172.21.15.38:0/3023561834 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd3600d1120 0x7fd360157500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:08.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.174+0000 7fd36625a640 1 --2- 172.21.15.38:0/3023561834 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd36015ee80 0x7fd360157a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:08.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.175+0000 7fd365a59640 1 -- 172.21.15.38:0/3023561834 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd3600d1120 msgr2=0x7fd360157500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:08.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.175+0000 7fd365a59640 1 --2- 172.21.15.38:0/3023561834 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd3600d1120 0x7fd360157500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:08.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.175+0000 7fd365a59640 1 -- 172.21.15.38:0/3023561834 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd36015ee80 msgr2=0x7fd360157a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:08.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.175+0000 7fd365a59640 1 --2- 172.21.15.38:0/3023561834 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd36015ee80 0x7fd360157a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:08.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.175+0000 7fd365a59640 1 -- 172.21.15.38:0/3023561834 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3601595e0 con 0x7fd3600b9ad0 2024-09-21T03:41:08.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.175+0000 7fd36625a640 1 --2- 172.21.15.38:0/3023561834 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd36015ee80 0x7fd360157a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:41:08.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.175+0000 7fd365a59640 1 --2- 172.21.15.38:0/3023561834 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd3600b9ad0 0x7fd36006f010 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7fd35800d700 tx=0x7fd35800dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:08.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.175+0000 7fd35e7fc640 1 -- 172.21.15.38:0/3023561834 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd358017070 con 0x7fd3600b9ad0 2024-09-21T03:41:08.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.175+0000 7fd35e7fc640 1 -- 172.21.15.38:0/3023561834 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd358005310 con 0x7fd3600b9ad0 2024-09-21T03:41:08.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.175+0000 7fd367cbd640 1 -- 172.21.15.38:0/3023561834 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3601598c0 con 0x7fd3600b9ad0 2024-09-21T03:41:08.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.175+0000 7fd35e7fc640 1 -- 172.21.15.38:0/3023561834 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3580097c0 con 0x7fd3600b9ad0 2024-09-21T03:41:08.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.175+0000 7fd367cbd640 1 -- 172.21.15.38:0/3023561834 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd360159e80 con 0x7fd3600b9ad0 2024-09-21T03:41:08.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.177+0000 7fd35e7fc640 1 -- 172.21.15.38:0/3023561834 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fd358009960 con 0x7fd3600b9ad0 2024-09-21T03:41:08.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.178+0000 7fd35e7fc640 1 --2- 172.21.15.38:0/3023561834 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd34c078b90 0x7fd34c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:08.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.178+0000 7fd35e7fc640 1 -- 172.21.15.38:0/3023561834 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 6906+0+0 (secure 0 0 0) 0x7fd35809b2a0 con 0x7fd3600b9ad0 2024-09-21T03:41:08.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.178+0000 7fd365258640 1 --2- 172.21.15.38:0/3023561834 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd34c078b90 0x7fd34c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:08.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.178+0000 7fd36ec2d740 1 Processor -- start 2024-09-21T03:41:08.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.178+0000 7fd365258640 1 --2- 172.21.15.38:0/3023561834 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd34c078b90 0x7fd34c07b0a0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fd354034000 tx=0x7fd354004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:08.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.179+0000 7fd36ec2d740 1 -- start start 2024-09-21T03:41:08.183 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.180+0000 7fd36ec2d740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f2ae9dd270 0x55f2aedaefa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:08.183 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.180+0000 7fd36ec2d740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f2ae9b43b0 0x55f2aedaf4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:08.183 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.180+0000 7fd36ec2d740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f2ae9bc590 0x55f2aedcc3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:08.183 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.180+0000 7fd36ec2d740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55f2aecf7820 con 0x55f2ae9bc590 2024-09-21T03:41:08.183 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.180+0000 7fd36ec2d740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55f2aece6c50 con 0x55f2ae9dd270 2024-09-21T03:41:08.183 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.180+0000 7fd36ec2d740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55f2aeceaa60 con 0x55f2ae9b43b0 2024-09-21T03:41:08.183 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.180+0000 7fd36625a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f2ae9bc590 0x55f2aedcc3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:08.183 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.180+0000 7fd36625a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f2ae9bc590 0x55f2aedcc3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50434/0 (socket says 172.21.15.38:50434) 2024-09-21T03:41:08.183 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.180+0000 7fd36625a640 1 -- 172.21.15.38:0/2559660739 learned_addr learned my addr 172.21.15.38:0/2559660739 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:41:08.183 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.180+0000 7fd365a59640 1 --2- 172.21.15.38:0/2559660739 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f2ae9dd270 0x55f2aedaefa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:08.183 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.180+0000 7fd365258640 1 --2- 172.21.15.38:0/2559660739 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f2ae9b43b0 0x55f2aedaf4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:08.183 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.180+0000 7fd36625a640 1 -- 172.21.15.38:0/2559660739 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f2ae9b43b0 msgr2=0x55f2aedaf4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:08.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.180+0000 7fd36625a640 1 --2- 172.21.15.38:0/2559660739 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f2ae9b43b0 0x55f2aedaf4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:08.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.180+0000 7fd36625a640 1 -- 172.21.15.38:0/2559660739 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f2ae9dd270 msgr2=0x55f2aedaefa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:08.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.180+0000 7fd36625a640 1 --2- 172.21.15.38:0/2559660739 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f2ae9dd270 0x55f2aedaefa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:08.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.180+0000 7fd36625a640 1 -- 172.21.15.38:0/2559660739 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f2aea48850 con 0x55f2ae9bc590 2024-09-21T03:41:08.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.180+0000 7fd36625a640 1 --2- 172.21.15.38:0/2559660739 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f2ae9bc590 0x55f2aedcc3e0 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7fd350001c50 tx=0x7fd350002110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6dd6bb04a1f8d59 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:08.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.181+0000 7fd3477fe640 1 -- 172.21.15.38:0/2559660739 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd350019070 con 0x55f2ae9bc590 2024-09-21T03:41:08.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.181+0000 7fd3477fe640 1 -- 172.21.15.38:0/2559660739 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd350003b80 con 0x55f2ae9bc590 2024-09-21T03:41:08.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.181+0000 7fd3477fe640 1 -- 172.21.15.38:0/2559660739 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd35000e860 con 0x55f2ae9bc590 2024-09-21T03:41:08.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.181+0000 7fd36ec2d740 1 -- 172.21.15.38:0/2559660739 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f2ae9bc590 msgr2=0x55f2aedcc3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:08.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.181+0000 7fd36ec2d740 1 --2- 172.21.15.38:0/2559660739 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f2ae9bc590 0x55f2aedcc3e0 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7fd350001c50 tx=0x7fd350002110 comp rx=0 tx=0).stop 2024-09-21T03:41:08.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.182+0000 7fd36ec2d740 1 -- 172.21.15.38:0/2559660739 shutdown_connections 2024-09-21T03:41:08.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.182+0000 7fd36ec2d740 1 --2- 172.21.15.38:0/2559660739 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f2ae9bc590 0x55f2aedcc3e0 unknown :-1 s=CLOSED pgs=1401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:08.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.182+0000 7fd36ec2d740 1 --2- 172.21.15.38:0/2559660739 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f2ae9b43b0 0x55f2aedaf4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:08.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.182+0000 7fd36ec2d740 1 --2- 172.21.15.38:0/2559660739 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f2ae9dd270 0x55f2aedaefa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:08.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.182+0000 7fd36ec2d740 1 -- 172.21.15.38:0/2559660739 >> 172.21.15.38:0/2559660739 conn(0x55f2ae9d13c0 msgr2=0x55f2aed85260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:08.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.182+0000 7fd36ec2d740 1 -- 172.21.15.38:0/2559660739 shutdown_connections 2024-09-21T03:41:08.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.182+0000 7fd36ec2d740 1 -- 172.21.15.38:0/2559660739 wait complete. 2024-09-21T03:41:08.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.182+0000 7fd36ec2d740 1 -- start start 2024-09-21T03:41:08.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.183+0000 7fd36ec2d740 1 Processor -- start 2024-09-21T03:41:08.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.184+0000 7fd36ec2d740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f2ae9b43b0 0x55f2aedd9110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:08.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.184+0000 7fd36ec2d740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f2ae9bc590 0x55f2aedca2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:08.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.184+0000 7fd36ec2d740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f2ae9dd270 0x55f2aed996d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:08.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.184+0000 7fd36ec2d740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55f2aea57620 con 0x55f2ae9dd270 2024-09-21T03:41:08.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.184+0000 7fd36ec2d740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55f2aec8c5f0 con 0x55f2ae9b43b0 2024-09-21T03:41:08.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.184+0000 7fd36ec2d740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55f2aedafa20 con 0x55f2ae9bc590 2024-09-21T03:41:08.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.184+0000 7fd36ec2d740 10 client.0 _ll_register_callbacks cb 0x55f2aec900a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:41:08.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.184+0000 7fd36625a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f2ae9dd270 0x55f2aed996d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:08.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.184+0000 7fd36625a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f2ae9dd270 0x55f2aed996d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50438/0 (socket says 172.21.15.38:50438) 2024-09-21T03:41:08.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.184+0000 7fd36625a640 1 -- 172.21.15.38:0/2389794109 learned_addr learned my addr 172.21.15.38:0/2389794109 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:41:08.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.184+0000 7fd365a59640 1 --2- 172.21.15.38:0/2389794109 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f2ae9b43b0 0x55f2aedd9110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:08.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.184+0000 7fd365258640 1 --2- 172.21.15.38:0/2389794109 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f2ae9bc590 0x55f2aedca2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:08.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.184+0000 7fd36625a640 1 -- 172.21.15.38:0/2389794109 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f2ae9bc590 msgr2=0x55f2aedca2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:08.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.185+0000 7fd36625a640 1 --2- 172.21.15.38:0/2389794109 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f2ae9bc590 0x55f2aedca2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:08.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.185+0000 7fd36625a640 1 -- 172.21.15.38:0/2389794109 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f2ae9b43b0 msgr2=0x55f2aedd9110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:08.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.185+0000 7fd36625a640 1 --2- 172.21.15.38:0/2389794109 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f2ae9b43b0 0x55f2aedd9110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:08.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.185+0000 7fd36625a640 1 -- 172.21.15.38:0/2389794109 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f2aeabf6b0 con 0x55f2ae9dd270 2024-09-21T03:41:08.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.185+0000 7fd36625a640 1 --2- 172.21.15.38:0/2389794109 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f2ae9dd270 0x55f2aed996d0 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7fd350001a30 tx=0x7fd350004fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a87d5d3bdaf51fb7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:08.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.185+0000 7fd344ff9640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-21T03:41:08.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.185+0000 7fd344ff9640 1 -- 172.21.15.38:0/2389794109 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd350019040 con 0x55f2ae9dd270 2024-09-21T03:41:08.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.185+0000 7fd344ff9640 1 -- 172.21.15.38:0/2389794109 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 6906+0+0 (secure 0 0 0) 0x7fd350006b50 con 0x55f2ae9dd270 2024-09-21T03:41:08.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.185+0000 7fd36ec2d740 1 -- 172.21.15.38:0/2389794109 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd350003b80 con 0x55f2ae9dd270 2024-09-21T03:41:08.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.185+0000 7fd344ff9640 1 -- 172.21.15.38:0/2389794109 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=131) ==== 24+0+0 (secure 0 0 0) 0x7fd350029050 con 0x55f2ae9dd270 2024-09-21T03:41:08.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.185+0000 7fd36ec2d740 10 client.8166 fetch_fsmap learned FSMap version 131 2024-09-21T03:41:08.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.185+0000 7fd36ec2d740 1 -- 172.21.15.38:0/2389794109 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=131,monmap=2+}) -- 0x55f2aed991b0 con 0x55f2ae9dd270 2024-09-21T03:41:08.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.185+0000 7fd344ff9640 1 -- 172.21.15.38:0/2389794109 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd350003b80 con 0x55f2ae9dd270 2024-09-21T03:41:08.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.185+0000 7fd344ff9640 1 -- 172.21.15.38:0/2389794109 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 131) ==== 3502+0+0 (secure 0 0 0) 0x7fd35002c020 con 0x55f2ae9dd270 2024-09-21T03:41:08.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.185+0000 7fd36ec2d740 10 client.8166 fetch_fsmap finished waiting for FSMap version 131 2024-09-21T03:41:08.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.185+0000 7fd36ec2d740 10 client.8166 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T03:41:08.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.185+0000 7fd36ec2d740 20 client.8166 populate_metadata read hostname 'smithi038' 2024-09-21T03:41:08.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.185+0000 7fd36ec2d740 1 --2- 172.21.15.38:0/2389794109 >> [v2:172.21.15.38:6832/121492560,v1:172.21.15.38:6833/121492560] conn(0x55f2ae99be40 0x55f2aedc0d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:08.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.185+0000 7fd36ec2d740 4 client.8166 mds_command: new command op to 8109 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:41:08.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.185+0000 7fd36ec2d740 1 -- 172.21.15.38:0/2389794109 --> [v2:172.21.15.38:6832/121492560,v1:172.21.15.38:6833/121492560] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f2aecf9770 con 0x55f2ae99be40 2024-09-21T03:41:08.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.186+0000 7fd365a59640 1 --2- 172.21.15.38:0/2389794109 >> [v2:172.21.15.38:6832/121492560,v1:172.21.15.38:6833/121492560] conn(0x55f2ae99be40 0x55f2aedc0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:08.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.186+0000 7fd344ff9640 10 client.8166 ms_handle_connect on v2:172.21.15.38:6832/121492560 2024-09-21T03:41:08.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.186+0000 7fd365a59640 1 --2- 172.21.15.38:0/2389794109 >> [v2:172.21.15.38:6832/121492560,v1:172.21.15.38:6833/121492560] conn(0x55f2ae99be40 0x55f2aedc0d40 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7739e1e49554fc17 server_cookie=312df49bbb753ed7 in_seq=0 out_seq=0 2024-09-21T03:41:08.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.187+0000 7fd344ff9640 1 -- 172.21.15.38:0/2389794109 <== mds.0 v2:172.21.15.38:6832/121492560 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f2aecf9770 con 0x55f2ae99be40 2024-09-21T03:41:08.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.187+0000 7fd344ff9640 10 client.8166 handle_command_reply: tid=0 2024-09-21T03:41:08.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.187+0000 7fd36ec2d740 1 client.8166 shutdown 2024-09-21T03:41:08.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.187+0000 7fd36ec2d740 1 -- 172.21.15.38:0/2389794109 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f2ae9dd270 msgr2=0x55f2aed996d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:08.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.187+0000 7fd36ec2d740 1 --2- 172.21.15.38:0/2389794109 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f2ae9dd270 0x55f2aed996d0 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7fd350001a30 tx=0x7fd350004fd0 comp rx=0 tx=0).stop 2024-09-21T03:41:08.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.187+0000 7fd36ec2d740 1 -- 172.21.15.38:0/2389794109 shutdown_connections 2024-09-21T03:41:08.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.187+0000 7fd36ec2d740 1 --2- 172.21.15.38:0/2389794109 >> [v2:172.21.15.38:6832/121492560,v1:172.21.15.38:6833/121492560] conn(0x55f2ae99be40 0x55f2aedc0d40 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:08.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.187+0000 7fd36ec2d740 1 --2- 172.21.15.38:0/2389794109 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f2ae9dd270 0x55f2aed996d0 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:08.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.187+0000 7fd36ec2d740 1 --2- 172.21.15.38:0/2389794109 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f2ae9bc590 0x55f2aedca2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:08.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.187+0000 7fd344ff9640 0 client.8166 ms_handle_reset on v2:172.21.15.38:6832/121492560 2024-09-21T03:41:08.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.187+0000 7fd36ec2d740 1 --2- 172.21.15.38:0/2389794109 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f2ae9b43b0 0x55f2aedd9110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:08.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.187+0000 7fd36ec2d740 1 -- 172.21.15.38:0/2389794109 >> 172.21.15.38:0/2389794109 conn(0x55f2ae9d13c0 msgr2=0x55f2aed85110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:08.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.187+0000 7fd36ec2d740 1 -- 172.21.15.38:0/2389794109 shutdown_connections 2024-09-21T03:41:08.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.187+0000 7fd36ec2d740 1 -- 172.21.15.38:0/2389794109 wait complete. 2024-09-21T03:41:08.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.187+0000 7fd36ec2d740 20 client.8166 trim_cache size 0 max 16384 2024-09-21T03:41:08.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.206+0000 7fd36ec2d740 1 Processor -- start 2024-09-21T03:41:08.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.206+0000 7fd36ec2d740 1 -- start start 2024-09-21T03:41:08.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.207+0000 7fd36ec2d740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f2ae9b43b0 0x55f2aee3fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:08.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.207+0000 7fd36ec2d740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f2ae9bc590 0x55f2aedd8b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:08.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.207+0000 7fd36ec2d740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f2ae9dd270 0x55f2aedd9070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:08.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.207+0000 7fd36ec2d740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fd360201f40 con 0x55f2ae9bc590 2024-09-21T03:41:08.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.207+0000 7fd36ec2d740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd360201270 con 0x55f2ae9dd270 2024-09-21T03:41:08.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.207+0000 7fd36ec2d740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fd360201890 con 0x55f2ae9b43b0 2024-09-21T03:41:08.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.207+0000 7fd365258640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f2ae9bc590 0x55f2aedd8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:08.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.207+0000 7fd365258640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f2ae9bc590 0x55f2aedd8b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50452/0 (socket says 172.21.15.38:50452) 2024-09-21T03:41:08.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.207+0000 7fd365258640 1 -- 172.21.15.38:0/332181648 learned_addr learned my addr 172.21.15.38:0/332181648 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:41:08.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.207+0000 7fd365258640 1 -- 172.21.15.38:0/332181648 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f2ae9b43b0 msgr2=0x55f2aee3fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:08.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.207+0000 7fd365258640 1 --2- 172.21.15.38:0/332181648 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f2ae9b43b0 0x55f2aee3fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:08.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.207+0000 7fd365258640 1 -- 172.21.15.38:0/332181648 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f2ae9dd270 msgr2=0x55f2aedd9070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:08.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.207+0000 7fd365258640 1 --2- 172.21.15.38:0/332181648 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f2ae9dd270 0x55f2aedd9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:08.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.207+0000 7fd365258640 1 -- 172.21.15.38:0/332181648 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f2aedcb230 con 0x55f2ae9bc590 2024-09-21T03:41:08.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.208+0000 7fd365258640 1 --2- 172.21.15.38:0/332181648 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f2ae9bc590 0x55f2aedd8b30 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7fd3540097b0 tx=0x7fd35403e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=797b6b2845847810 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:08.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.208+0000 7fd346ffd640 1 -- 172.21.15.38:0/332181648 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd354056070 con 0x55f2ae9bc590 2024-09-21T03:41:08.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.208+0000 7fd346ffd640 1 -- 172.21.15.38:0/332181648 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd354004ba0 con 0x55f2ae9bc590 2024-09-21T03:41:08.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.208+0000 7fd346ffd640 1 -- 172.21.15.38:0/332181648 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd35403c420 con 0x55f2ae9bc590 2024-09-21T03:41:08.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.208+0000 7fd36ec2d740 1 -- 172.21.15.38:0/332181648 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f2ae9bc590 msgr2=0x55f2aedd8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:08.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.208+0000 7fd36ec2d740 1 --2- 172.21.15.38:0/332181648 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f2ae9bc590 0x55f2aedd8b30 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7fd3540097b0 tx=0x7fd35403e040 comp rx=0 tx=0).stop 2024-09-21T03:41:08.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.209+0000 7fd36ec2d740 1 -- 172.21.15.38:0/332181648 shutdown_connections 2024-09-21T03:41:08.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.209+0000 7fd36ec2d740 1 --2- 172.21.15.38:0/332181648 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f2ae9dd270 0x55f2aedd9070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:08.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.209+0000 7fd36ec2d740 1 --2- 172.21.15.38:0/332181648 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f2ae9bc590 0x55f2aedd8b30 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:08.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.209+0000 7fd36ec2d740 1 --2- 172.21.15.38:0/332181648 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f2ae9b43b0 0x55f2aee3fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:08.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.209+0000 7fd36ec2d740 1 -- 172.21.15.38:0/332181648 >> 172.21.15.38:0/332181648 conn(0x55f2ae9d13c0 msgr2=0x55f2aed86390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:08.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.209+0000 7fd36ec2d740 1 -- 172.21.15.38:0/332181648 shutdown_connections 2024-09-21T03:41:08.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.209+0000 7fd36ec2d740 1 -- 172.21.15.38:0/332181648 wait complete. 2024-09-21T03:41:08.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.209+0000 7fd36ec2d740 1 -- start start 2024-09-21T03:41:08.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.210+0000 7fd36ec2d740 1 Processor -- start 2024-09-21T03:41:08.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.211+0000 7fd36ec2d740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f2ae9b43b0 0x55f2aedc9430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:08.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.211+0000 7fd36ec2d740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f2ae9bc590 0x55f2aee42940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:08.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.211+0000 7fd36ec2d740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f2ae9dd270 0x55f2aee42e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:08.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.211+0000 7fd36ec2d740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55f2aeddc8a0 con 0x55f2ae9dd270 2024-09-21T03:41:08.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.211+0000 7fd36ec2d740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55f2aedd9700 con 0x55f2ae9b43b0 2024-09-21T03:41:08.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.211+0000 7fd36ec2d740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55f2aed99f10 con 0x55f2ae9bc590 2024-09-21T03:41:08.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.211+0000 7fd36ec2d740 10 client.0 _ll_register_callbacks cb 0x7fd350017070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:41:08.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.211+0000 7fd36625a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f2ae9dd270 0x55f2aee42e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:08.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.211+0000 7fd36625a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f2ae9dd270 0x55f2aee42e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50468/0 (socket says 172.21.15.38:50468) 2024-09-21T03:41:08.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.211+0000 7fd36625a640 1 -- 172.21.15.38:0/877701897 learned_addr learned my addr 172.21.15.38:0/877701897 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:41:08.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.211+0000 7fd36625a640 1 -- 172.21.15.38:0/877701897 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f2ae9bc590 msgr2=0x55f2aee42940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:08.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.211+0000 7fd365258640 1 --2- 172.21.15.38:0/877701897 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f2ae9bc590 0x55f2aee42940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:08.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.211+0000 7fd365a59640 1 --2- 172.21.15.38:0/877701897 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f2ae9b43b0 0x55f2aedc9430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:08.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.211+0000 7fd36625a640 1 --2- 172.21.15.38:0/877701897 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f2ae9bc590 0x55f2aee42940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:08.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.211+0000 7fd36625a640 1 -- 172.21.15.38:0/877701897 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f2ae9b43b0 msgr2=0x55f2aedc9430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:08.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.211+0000 7fd36625a640 1 --2- 172.21.15.38:0/877701897 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f2ae9b43b0 0x55f2aedc9430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:08.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.211+0000 7fd36625a640 1 -- 172.21.15.38:0/877701897 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f2aee3c790 con 0x55f2ae9dd270 2024-09-21T03:41:08.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.212+0000 7fd36625a640 1 --2- 172.21.15.38:0/877701897 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f2ae9dd270 0x55f2aee42e80 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7fd350000f80 tx=0x7fd3500055e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d8f89837edeb96 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:08.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.212+0000 7fd344ff9640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-21T03:41:08.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.212+0000 7fd344ff9640 1 -- 172.21.15.38:0/877701897 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd350019070 con 0x55f2ae9dd270 2024-09-21T03:41:08.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.212+0000 7fd344ff9640 1 -- 172.21.15.38:0/877701897 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 6906+0+0 (secure 0 0 0) 0x7fd3500057d0 con 0x55f2ae9dd270 2024-09-21T03:41:08.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.212+0000 7fd36ec2d740 1 -- 172.21.15.38:0/877701897 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd354004ba0 con 0x55f2ae9dd270 2024-09-21T03:41:08.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.212+0000 7fd344ff9640 1 -- 172.21.15.38:0/877701897 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=131) ==== 24+0+0 (secure 0 0 0) 0x7fd350006b50 con 0x55f2ae9dd270 2024-09-21T03:41:08.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.212+0000 7fd36ec2d740 10 client.8172 fetch_fsmap learned FSMap version 131 2024-09-21T03:41:08.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.212+0000 7fd36ec2d740 1 -- 172.21.15.38:0/877701897 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=131,monmap=2+}) -- 0x55f2aeddb8f0 con 0x55f2ae9dd270 2024-09-21T03:41:08.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.212+0000 7fd344ff9640 1 -- 172.21.15.38:0/877701897 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd354004ba0 con 0x55f2ae9dd270 2024-09-21T03:41:08.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.212+0000 7fd344ff9640 1 -- 172.21.15.38:0/877701897 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 131) ==== 3502+0+0 (secure 0 0 0) 0x7fd350035050 con 0x55f2ae9dd270 2024-09-21T03:41:08.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.212+0000 7fd36ec2d740 10 client.8172 fetch_fsmap finished waiting for FSMap version 131 2024-09-21T03:41:08.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.212+0000 7fd36ec2d740 10 client.8172 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T03:41:08.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.212+0000 7fd36ec2d740 20 client.8172 populate_metadata read hostname 'smithi038' 2024-09-21T03:41:08.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.212+0000 7fd36ec2d740 1 --2- 172.21.15.38:0/877701897 >> [v2:172.21.15.38:6832/121492560,v1:172.21.15.38:6833/121492560] conn(0x55f2aee40ee0 0x55f2aee3fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:08.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.212+0000 7fd36ec2d740 4 client.8172 mds_command: new command op to 8109 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T03:41:08.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.212+0000 7fd36ec2d740 1 -- 172.21.15.38:0/877701897 --> [v2:172.21.15.38:6832/121492560,v1:172.21.15.38:6833/121492560] -- command(tid 0: {"prefix": "status"}) -- 0x55f2aed80310 con 0x55f2aee40ee0 2024-09-21T03:41:08.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.212+0000 7fd365a59640 1 --2- 172.21.15.38:0/877701897 >> [v2:172.21.15.38:6832/121492560,v1:172.21.15.38:6833/121492560] conn(0x55f2aee40ee0 0x55f2aee3fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:08.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.213+0000 7fd344ff9640 10 client.8172 ms_handle_connect on v2:172.21.15.38:6832/121492560 2024-09-21T03:41:08.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.213+0000 7fd365a59640 1 --2- 172.21.15.38:0/877701897 >> [v2:172.21.15.38:6832/121492560,v1:172.21.15.38:6833/121492560] conn(0x55f2aee40ee0 0x55f2aee3fc30 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea0f2ff66df2c2fa server_cookie=31b1dea6625a45e7 in_seq=0 out_seq=0 2024-09-21T03:41:08.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.213+0000 7fd344ff9640 1 -- 172.21.15.38:0/877701897 <== mds.0 v2:172.21.15.38:6832/121492560 1 ==== command_reply(tid 0: 0 ) ==== 8+0+334 (crc 0 0 0) 0x55f2aed80310 con 0x55f2aee40ee0 2024-09-21T03:41:08.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.213+0000 7fd344ff9640 10 client.8172 handle_command_reply: tid=0 2024-09-21T03:41:08.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.213+0000 7fd36ec2d740 1 client.8172 shutdown 2024-09-21T03:41:08.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.213+0000 7fd36ec2d740 1 -- 172.21.15.38:0/877701897 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f2ae9dd270 msgr2=0x55f2aee42e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:08.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.213+0000 7fd36ec2d740 1 --2- 172.21.15.38:0/877701897 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f2ae9dd270 0x55f2aee42e80 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7fd350000f80 tx=0x7fd3500055e0 comp rx=0 tx=0).stop 2024-09-21T03:41:08.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.213+0000 7fd36ec2d740 1 -- 172.21.15.38:0/877701897 shutdown_connections 2024-09-21T03:41:08.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.213+0000 7fd36ec2d740 1 --2- 172.21.15.38:0/877701897 >> [v2:172.21.15.38:6832/121492560,v1:172.21.15.38:6833/121492560] conn(0x55f2aee40ee0 0x55f2aee3fc30 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:08.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.213+0000 7fd36ec2d740 1 --2- 172.21.15.38:0/877701897 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f2ae9dd270 0x55f2aee42e80 unknown :-1 s=CLOSED pgs=1404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:08.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.213+0000 7fd344ff9640 0 client.8172 ms_handle_reset on v2:172.21.15.38:6832/121492560 2024-09-21T03:41:08.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.213+0000 7fd36ec2d740 1 --2- 172.21.15.38:0/877701897 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f2ae9bc590 0x55f2aee42940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:08.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.213+0000 7fd36ec2d740 1 --2- 172.21.15.38:0/877701897 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f2ae9b43b0 0x55f2aedc9430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:08.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.213+0000 7fd36ec2d740 1 -- 172.21.15.38:0/877701897 >> 172.21.15.38:0/877701897 conn(0x55f2ae9d13c0 msgr2=0x55f2aed9a360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:08.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.213+0000 7fd36ec2d740 1 -- 172.21.15.38:0/877701897 shutdown_connections 2024-09-21T03:41:08.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.214+0000 7fd36ec2d740 1 -- 172.21.15.38:0/877701897 wait complete. 2024-09-21T03:41:08.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.214+0000 7fd36ec2d740 20 client.8172 trim_cache size 0 max 16384 2024-09-21T03:41:08.221 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-09-21T03:41:08.221 INFO:teuthology.orchestra.run.smithi038.stdout: "cluster_fsid": "9d50d66f-656b-4fe2-8e3b-fe53b79c4d43", 2024-09-21T03:41:08.221 INFO:teuthology.orchestra.run.smithi038.stdout: "whoami": 0, 2024-09-21T03:41:08.221 INFO:teuthology.orchestra.run.smithi038.stdout: "id": 8109, 2024-09-21T03:41:08.221 INFO:teuthology.orchestra.run.smithi038.stdout: "want_state": "up:active", 2024-09-21T03:41:08.221 INFO:teuthology.orchestra.run.smithi038.stdout: "state": "up:active", 2024-09-21T03:41:08.221 INFO:teuthology.orchestra.run.smithi038.stdout: "fs_name": "cephfs", 2024-09-21T03:41:08.221 INFO:teuthology.orchestra.run.smithi038.stdout: "rank_uptime": 6.7443449019999999, 2024-09-21T03:41:08.221 INFO:teuthology.orchestra.run.smithi038.stdout: "mdsmap_epoch": 131, 2024-09-21T03:41:08.221 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch": 103, 2024-09-21T03:41:08.221 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch_barrier": 103, 2024-09-21T03:41:08.221 INFO:teuthology.orchestra.run.smithi038.stdout: "uptime": 8.6958770439999995 2024-09-21T03:41:08.221 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-09-21T03:41:08.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.214+0000 7fd367cbd640 1 -- 172.21.15.38:0/3023561834 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd34c078b90 msgr2=0x7fd34c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:08.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.214+0000 7fd367cbd640 1 --2- 172.21.15.38:0/3023561834 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd34c078b90 0x7fd34c07b0a0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fd354034000 tx=0x7fd354004730 comp rx=0 tx=0).stop 2024-09-21T03:41:08.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.214+0000 7fd367cbd640 1 -- 172.21.15.38:0/3023561834 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd3600b9ad0 msgr2=0x7fd36006f010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:08.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.214+0000 7fd367cbd640 1 --2- 172.21.15.38:0/3023561834 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd3600b9ad0 0x7fd36006f010 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7fd35800d700 tx=0x7fd35800dbc0 comp rx=0 tx=0).stop 2024-09-21T03:41:08.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.214+0000 7fd367cbd640 1 -- 172.21.15.38:0/3023561834 shutdown_connections 2024-09-21T03:41:08.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.214+0000 7fd367cbd640 1 --2- 172.21.15.38:0/3023561834 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd36015ee80 0x7fd360157a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:08.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.214+0000 7fd367cbd640 1 --2- 172.21.15.38:0/3023561834 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd34c078b90 0x7fd34c07b0a0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:08.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.214+0000 7fd367cbd640 1 --2- 172.21.15.38:0/3023561834 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd3600d1120 0x7fd360157500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:08.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.214+0000 7fd367cbd640 1 --2- 172.21.15.38:0/3023561834 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd3600b9ad0 0x7fd36006f010 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:08.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.214+0000 7fd367cbd640 1 -- 172.21.15.38:0/3023561834 >> 172.21.15.38:0/3023561834 conn(0x7fd36009d830 msgr2=0x7fd36015e260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:08.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.214+0000 7fd367cbd640 1 -- 172.21.15.38:0/3023561834 shutdown_connections 2024-09-21T03:41:08.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.214+0000 7fd367cbd640 1 -- 172.21.15.38:0/3023561834 wait complete. 2024-09-21T03:41:08.227 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-09-21T03:41:08.227 INFO:tasks.cephfs.test_damage:Daemons came up after mutation 'Delete 500.00000000', proceeding to ls 2024-09-21T03:41:08.227 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T03:41:08.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.341+0000 7f3721aa4640 1 -- 172.21.15.38:0/222109797 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f371c0cce70 msgr2=0x7f371c160190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:08.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.341+0000 7f3721aa4640 1 --2- 172.21.15.38:0/222109797 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f371c0cce70 0x7f371c160190 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7f371c0b4ab0 tx=0x7f371402aa30 comp rx=0 tx=0).stop 2024-09-21T03:41:08.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.341+0000 7f3721aa4640 1 -- 172.21.15.38:0/222109797 shutdown_connections 2024-09-21T03:41:08.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.341+0000 7f3721aa4640 1 --2- 172.21.15.38:0/222109797 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f371c0cce70 0x7f371c160190 unknown :-1 s=CLOSED pgs=1405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:08.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.341+0000 7f3721aa4640 1 --2- 172.21.15.38:0/222109797 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f371c0d1d00 0x7f371c0cc930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:08.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.341+0000 7f3721aa4640 1 --2- 172.21.15.38:0/222109797 >> v2:172.21.15.195:3300/0 conn(0x7f371c1536e0 0x7f371c0d17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:08.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.341+0000 7f3721aa4640 1 -- 172.21.15.38:0/222109797 >> 172.21.15.38:0/222109797 conn(0x7f371c0cef40 msgr2=0x7f371c0cf350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:08.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.341+0000 7f3721aa4640 1 -- 172.21.15.38:0/222109797 shutdown_connections 2024-09-21T03:41:08.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.341+0000 7f3721aa4640 1 -- 172.21.15.38:0/222109797 wait complete. 2024-09-21T03:41:08.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.343+0000 7f3721aa4640 1 Processor -- start 2024-09-21T03:41:08.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.343+0000 7f3721aa4640 1 -- start start 2024-09-21T03:41:08.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.345+0000 7f3721aa4640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f371c0cce70 0x7f371c1f61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:08.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.345+0000 7f3721aa4640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f371c0d1d00 0x7f371c156aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:08.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.345+0000 7f3721aa4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f371c1536e0 0x7f371c1ef600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:08.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.345+0000 7f3721aa4640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f371c1606d0 con 0x7f371c1536e0 2024-09-21T03:41:08.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.345+0000 7f3721aa4640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f371c160850 con 0x7f371c0d1d00 2024-09-21T03:41:08.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.345+0000 7f3721aa4640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f371c1609d0 con 0x7f371c0cce70 2024-09-21T03:41:08.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.345+0000 7f371bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f371c1536e0 0x7f371c1ef600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:08.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.345+0000 7f371bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f371c1536e0 0x7f371c1ef600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50488/0 (socket says 172.21.15.38:50488) 2024-09-21T03:41:08.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.345+0000 7f371bfff640 1 -- 172.21.15.38:0/2558139426 learned_addr learned my addr 172.21.15.38:0/2558139426 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:41:08.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.345+0000 7f371affd640 1 --2- 172.21.15.38:0/2558139426 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f371c0d1d00 0x7f371c156aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:08.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.345+0000 7f371b7fe640 1 --2- 172.21.15.38:0/2558139426 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f371c0cce70 0x7f371c1f61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:08.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.345+0000 7f371bfff640 1 -- 172.21.15.38:0/2558139426 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f371c0cce70 msgr2=0x7f371c1f61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:08.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.345+0000 7f371bfff640 1 --2- 172.21.15.38:0/2558139426 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f371c0cce70 0x7f371c1f61d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:08.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.345+0000 7f371bfff640 1 -- 172.21.15.38:0/2558139426 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f371c0d1d00 msgr2=0x7f371c156aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:08.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.345+0000 7f371bfff640 1 --2- 172.21.15.38:0/2558139426 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f371c0d1d00 0x7f371c156aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:08.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.345+0000 7f371bfff640 1 -- 172.21.15.38:0/2558139426 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f371c1f5730 con 0x7f371c1536e0 2024-09-21T03:41:08.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.345+0000 7f371bfff640 1 --2- 172.21.15.38:0/2558139426 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f371c1536e0 0x7f371c1ef600 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f3714009d60 tx=0x7f3714003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:08.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.346+0000 7f3713fff640 1 -- 172.21.15.38:0/2558139426 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3714033050 con 0x7f371c1536e0 2024-09-21T03:41:08.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.346+0000 7f3713fff640 1 -- 172.21.15.38:0/2558139426 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3714004450 con 0x7f371c1536e0 2024-09-21T03:41:08.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.346+0000 7f3721aa4640 1 -- 172.21.15.38:0/2558139426 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f371c14d840 con 0x7f371c1536e0 2024-09-21T03:41:08.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.346+0000 7f3713fff640 1 -- 172.21.15.38:0/2558139426 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3714038450 con 0x7f371c1536e0 2024-09-21T03:41:08.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.346+0000 7f3721aa4640 1 -- 172.21.15.38:0/2558139426 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f371c1f5cb0 con 0x7f371c1536e0 2024-09-21T03:41:08.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.347+0000 7f3721aa4640 1 -- 172.21.15.38:0/2558139426 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f371c14d5b0 con 0x7f371c1536e0 2024-09-21T03:41:08.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.347+0000 7f3713fff640 1 -- 172.21.15.38:0/2558139426 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f3714036070 con 0x7f371c1536e0 2024-09-21T03:41:08.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.348+0000 7f3713fff640 1 --2- 172.21.15.38:0/2558139426 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f36fc078b90 0x7f36fc07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:08.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.348+0000 7f3713fff640 1 -- 172.21.15.38:0/2558139426 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 6906+0+0 (secure 0 0 0) 0x7f37140bc590 con 0x7f371c1536e0 2024-09-21T03:41:08.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.348+0000 7f371b7fe640 1 --2- 172.21.15.38:0/2558139426 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f36fc078b90 0x7f36fc07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:08.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.349+0000 7f371b7fe640 1 --2- 172.21.15.38:0/2558139426 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f36fc078b90 0x7f36fc07b0a0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f370c009b30 tx=0x7f370c005f90 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:08.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.352+0000 7f3713fff640 1 -- 172.21.15.38:0/2558139426 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f3714083ec0 con 0x7f371c1536e0 2024-09-21T03:41:08.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.579+0000 7f3721aa4640 1 -- 172.21.15.38:0/2558139426 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f371c160ef0 con 0x7f371c1536e0 2024-09-21T03:41:08.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.580+0000 7f3713fff640 1 -- 172.21.15.38:0/2558139426 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v131) ==== 53+0+73 (secure 0 0 0) 0x7f37140888f0 con 0x7f371c1536e0 2024-09-21T03:41:08.582 INFO:teuthology.orchestra.run.smithi038.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T03:41:08.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.582+0000 7f3721aa4640 1 -- 172.21.15.38:0/2558139426 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f36fc078b90 msgr2=0x7f36fc07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:08.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.582+0000 7f3721aa4640 1 --2- 172.21.15.38:0/2558139426 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f36fc078b90 0x7f36fc07b0a0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f370c009b30 tx=0x7f370c005f90 comp rx=0 tx=0).stop 2024-09-21T03:41:08.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.582+0000 7f3721aa4640 1 -- 172.21.15.38:0/2558139426 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f371c1536e0 msgr2=0x7f371c1ef600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:08.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.582+0000 7f3721aa4640 1 --2- 172.21.15.38:0/2558139426 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f371c1536e0 0x7f371c1ef600 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f3714009d60 tx=0x7f3714003f10 comp rx=0 tx=0).stop 2024-09-21T03:41:08.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.582+0000 7f3721aa4640 1 -- 172.21.15.38:0/2558139426 shutdown_connections 2024-09-21T03:41:08.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.582+0000 7f3721aa4640 1 --2- 172.21.15.38:0/2558139426 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f371c1536e0 0x7f371c1ef600 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:08.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.582+0000 7f3721aa4640 1 --2- 172.21.15.38:0/2558139426 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f371c0d1d00 0x7f371c156aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:08.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.582+0000 7f3721aa4640 1 --2- 172.21.15.38:0/2558139426 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f36fc078b90 0x7f36fc07b0a0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:08.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.582+0000 7f3721aa4640 1 --2- 172.21.15.38:0/2558139426 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f371c0cce70 0x7f371c1f61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:08.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.582+0000 7f3721aa4640 1 -- 172.21.15.38:0/2558139426 >> 172.21.15.38:0/2558139426 conn(0x7f371c0cef40 msgr2=0x7f371c0cf350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:08.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.582+0000 7f3721aa4640 1 -- 172.21.15.38:0/2558139426 shutdown_connections 2024-09-21T03:41:08.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08.582+0000 7f3721aa4640 1 -- 172.21.15.38:0/2558139426 wait complete. 2024-09-21T03:41:08.595 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-21T03:41:08.595 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T03:41:08.595 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi038.front.sepia.ceph.com 2024-09-21T03:41:08.595 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-21T03:41:08.596 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T03:41:08.596 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-21T03:41:08.596 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-21T03:41:08.596 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T03:41:08.596 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T03:41:08.596 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:41:08.596 DEBUG:teuthology.orchestra.run.smithi038:> ip addr 2024-09-21T03:41:08.612 INFO:teuthology.orchestra.run.smithi038.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T03:41:08.613 INFO:teuthology.orchestra.run.smithi038.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T03:41:08.613 INFO:teuthology.orchestra.run.smithi038.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T03:41:08.613 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:41:08.613 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 ::1/128 scope host 2024-09-21T03:41:08.613 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:41:08.613 INFO:teuthology.orchestra.run.smithi038.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T03:41:08.613 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:6b:fd:a2 brd ff:ff:ff:ff:ff:ff 2024-09-21T03:41:08.613 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp7s0f0 2024-09-21T03:41:08.613 INFO:teuthology.orchestra.run.smithi038.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T03:41:08.613 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:6b:fd:a3 brd ff:ff:ff:ff:ff:ff 2024-09-21T03:41:08.613 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp7s0f1 2024-09-21T03:41:08.614 INFO:teuthology.orchestra.run.smithi038.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T03:41:08.614 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:bd:15:fc brd ff:ff:ff:ff:ff:ff 2024-09-21T03:41:08.614 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp1s0f0 2024-09-21T03:41:08.614 INFO:teuthology.orchestra.run.smithi038.stdout: inet 172.21.15.38/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-21T03:41:08.614 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft 42340sec preferred_lft 42340sec 2024-09-21T03:41:08.614 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 fe80::ec4:7aff:febd:15fc/64 scope link 2024-09-21T03:41:08.614 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:41:08.614 INFO:teuthology.orchestra.run.smithi038.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T03:41:08.614 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:bd:15:fd brd ff:ff:ff:ff:ff:ff 2024-09-21T03:41:08.614 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp1s0f1 2024-09-21T03:41:08.614 INFO:teuthology.orchestra.run.smithi038.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T03:41:08.614 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 92:55:c7:97:6f:b5 brd ff:ff:ff:ff:ff:ff 2024-09-21T03:41:08.614 INFO:teuthology.orchestra.run.smithi038.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-21T03:41:08.615 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:41:08.615 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 fe80::c49f:c1ff:fe21:b12a/64 scope link 2024-09-21T03:41:08.615 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:41:08.615 INFO:teuthology.orchestra.run.smithi038.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T03:41:08.615 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether a6:3a:83:ad:db:d5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T03:41:08.615 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 fe80::a43a:83ff:fead:dbd5/64 scope link 2024-09-21T03:41:08.615 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:41:08.615 INFO:teuthology.orchestra.run.smithi038.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T03:41:08.615 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 92:55:c7:97:6f:b5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T03:41:08.615 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 fe80::9055:c7ff:fe97:6fb5/64 scope link 2024-09-21T03:41:08.615 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:41:08.616 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:41:08.616 DEBUG:teuthology.orchestra.run.smithi038:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T03:41:08.677 INFO:teuthology.orchestra.run.smithi038.stdout:1 2024-09-21T03:41:08.678 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:41:08.678 DEBUG:teuthology.orchestra.run.smithi038:> route 2024-09-21T03:41:08.735 INFO:teuthology.orchestra.run.smithi038.stdout:Kernel IP routing table 2024-09-21T03:41:08.735 INFO:teuthology.orchestra.run.smithi038.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T03:41:08.735 INFO:teuthology.orchestra.run.smithi038.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-21T03:41:08.736 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-21T03:41:08.736 INFO:teuthology.orchestra.run.smithi038.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T03:41:08.736 DEBUG:teuthology.orchestra.run.smithi038:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T03:41:08.736 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-09-21T03:41:08.736 DEBUG:teuthology.orchestra.run.smithi038:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-21T03:41:08.736 DEBUG:teuthology.orchestra.run.smithi038:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-21T03:41:08.736 DEBUG:teuthology.orchestra.run.smithi038:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-21T03:41:08.736 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-09-21T03:41:08.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T03:41:08.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:41:08.864 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:41:08.865 DEBUG:teuthology.orchestra.run.smithi038:> ip netns list 2024-09-21T03:41:08.881 INFO:teuthology.orchestra.run.smithi038.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-21T03:41:08.881 INFO:teuthology.orchestra.run.smithi038.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-21T03:41:08.881 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T03:41:08.881 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:41:08.881 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip link set brx.0 up 2024-09-21T03:41:08.946 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T03:41:08.947 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-21T03:41:09.002 INFO:teuthology.orchestra.run.smithi038.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-21T03:41:09.003 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T03:41:09.003 DEBUG:teuthology.orchestra.run.smithi038:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-21T03:41:09.058 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-21T03:41:09.058 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:41:09.058 DEBUG:teuthology.orchestra.run.smithi038:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T03:41:09.154 INFO:teuthology.orchestra.run.smithi038.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T03:41:09.154 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T03:41:09.155 INFO:teuthology.orchestra.run.smithi038.stdout:invalid new device string format 2024-09-21T03:41:09.165 INFO:teuthology.orchestra.run.smithi038.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T03:41:09.165 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T03:41:09.165 INFO:teuthology.orchestra.run.smithi038.stdout:invalid new device string format 2024-09-21T03:41:09.165 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: resolved to: "172.21.15.38:3300,172.21.15.195:3300,172.21.15.195:3301" 2024-09-21T03:41:09.166 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T03:41:09.172 INFO:teuthology.orchestra.run.smithi038.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T03:41:09.173 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T03:41:09.173 INFO:teuthology.orchestra.run.smithi038.stdout:invalid new device string format 2024-09-21T03:41:09.173 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: resolved to: "172.21.15.38:3300,172.21.15.195:3300,172.21.15.195:3301" 2024-09-21T03:41:09.173 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T03:41:09.173 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: resolved to: "172.21.15.38:6789,172.21.15.195:6789,172.21.15.195:6790" 2024-09-21T03:41:09.173 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.38:6789,172.21.15.195:6789,172.21.15.195:6790:/ 2024-09-21T03:41:09.173 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=9d50d66f-656b-4fe2-8e3b-fe53b79c4d43" will pass to kernel 2024-09-21T03:41:09.174 INFO:teuthology.orchestra.run.smithi038.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-21T03:41:09.174 INFO:teuthology.orchestra.run.smithi038.stdout: You just mounted a file system that supports labels which does not 2024-09-21T03:41:09.174 INFO:teuthology.orchestra.run.smithi038.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-21T03:41:09.174 INFO:teuthology.orchestra.run.smithi038.stdout: applications will generate AVC messages and not be allowed access to 2024-09-21T03:41:09.174 INFO:teuthology.orchestra.run.smithi038.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-21T03:41:09.175 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T03:41:09.175 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:41:09.176 DEBUG:teuthology.orchestra.run.smithi038:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-21T03:41:09.206 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T03:41:09.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:09 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T03:41:09.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:41:09.304 INFO:teuthology.orchestra.run.smithi038.stdout:client6794 2024-09-21T03:41:09.304 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-21T03:41:09.304 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:41:09.320 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:41:09.320 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls -R .) 2024-09-21T03:41:09.363 INFO:teuthology.orchestra.run:waiting for 20 2024-09-21T03:41:09.379 INFO:teuthology.orchestra.run.smithi038.stdout:.: 2024-09-21T03:41:09.379 INFO:teuthology.orchestra.run.smithi038.stdout:subdir 2024-09-21T03:41:09.379 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:41:09.379 INFO:teuthology.orchestra.run.smithi038.stdout:./subdir: 2024-09-21T03:41:09.379 INFO:teuthology.orchestra.run.smithi038.stdout:sixmegs 2024-09-21T03:41:09.379 INFO:tasks.cephfs.test_damage:Result: Mutation 'Delete 500.00000000' did not caused DAMAGED state 2024-09-21T03:41:09.379 INFO:tasks.cephfs.test_damage:Applying mutation 'Delete mds_snaptable' 2024-09-21T03:41:09.380 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-21T03:41:09.380 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:41:09.436 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-21T03:41:09.436 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:41:09.493 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T03:41:09.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:09.649+0000 7fe9c599d640 1 -- 172.21.15.38:0/4108491921 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe9ac0053a0 msgr2=0x7fe9ac00ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:09.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:09.649+0000 7fe9c599d640 1 --2- 172.21.15.38:0/4108491921 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe9ac0053a0 0x7fe9ac00ba20 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7fe9b800c610 tx=0x7fe9ac005830 comp rx=0 tx=0).stop 2024-09-21T03:41:09.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:09.649+0000 7fe9c599d640 1 -- 172.21.15.38:0/4108491921 shutdown_connections 2024-09-21T03:41:09.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:09.649+0000 7fe9c599d640 1 --2- 172.21.15.38:0/4108491921 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe9ac0053a0 0x7fe9ac00ba20 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:09.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:09.649+0000 7fe9c599d640 1 -- 172.21.15.38:0/4108491921 >> 172.21.15.38:0/4108491921 conn(0x7fe9c00d41e0 msgr2=0x7fe9c00d45f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:09.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:09.649+0000 7fe9c599d640 1 -- 172.21.15.38:0/4108491921 shutdown_connections 2024-09-21T03:41:09.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:09.649+0000 7fe9c599d640 1 -- 172.21.15.38:0/4108491921 wait complete. 2024-09-21T03:41:09.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:09.651+0000 7fe9c599d640 1 Processor -- start 2024-09-21T03:41:09.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:09.651+0000 7fe9c599d640 1 -- start start 2024-09-21T03:41:09.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:09.652+0000 7fe9c599d640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe9ac0053a0 0x7fe9c0156b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:09.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:09.652+0000 7fe9c599d640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe9c01570a0 0x7fe9c0150d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:09.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:09.652+0000 7fe9c599d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe9c01512a0 0x7fe9c0159b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:09.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:09.652+0000 7fe9c599d640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fe9c00cf6a0 con 0x7fe9c01512a0 2024-09-21T03:41:09.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:09.652+0000 7fe9c599d640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fe9c00cf520 con 0x7fe9ac0053a0 2024-09-21T03:41:09.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:09.652+0000 7fe9c599d640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fe9c00cf820 con 0x7fe9c01570a0 2024-09-21T03:41:09.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:09.653+0000 7fe9c519c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe9c01512a0 0x7fe9c0159b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:09.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:09.653+0000 7fe9c519c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe9c01512a0 0x7fe9c0159b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50510/0 (socket says 172.21.15.38:50510) 2024-09-21T03:41:09.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:09.653+0000 7fe9c519c640 1 -- 172.21.15.38:0/1461308330 learned_addr learned my addr 172.21.15.38:0/1461308330 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:41:09.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:09.653+0000 7fe9c499b640 1 --2- 172.21.15.38:0/1461308330 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe9ac0053a0 0x7fe9c0156b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:09.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:09.653+0000 7fe9bffff640 1 --2- 172.21.15.38:0/1461308330 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe9c01570a0 0x7fe9c0150d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:09.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:09.653+0000 7fe9c519c640 1 -- 172.21.15.38:0/1461308330 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe9c01570a0 msgr2=0x7fe9c0150d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:09.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:09.653+0000 7fe9c519c640 1 --2- 172.21.15.38:0/1461308330 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe9c01570a0 0x7fe9c0150d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:09.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:09.653+0000 7fe9c519c640 1 -- 172.21.15.38:0/1461308330 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe9ac0053a0 msgr2=0x7fe9c0156b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:09.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:09.653+0000 7fe9c519c640 1 --2- 172.21.15.38:0/1461308330 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe9ac0053a0 0x7fe9c0156b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:09.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:09.653+0000 7fe9c519c640 1 -- 172.21.15.38:0/1461308330 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9c00b3e90 con 0x7fe9c01512a0 2024-09-21T03:41:09.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:09.653+0000 7fe9c519c640 1 --2- 172.21.15.38:0/1461308330 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe9c01512a0 0x7fe9c0159b60 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7fe9b800c730 tx=0x7fe9b8003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:09.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:09.653+0000 7fe9bd7fa640 1 -- 172.21.15.38:0/1461308330 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9b802bb70 con 0x7fe9c01512a0 2024-09-21T03:41:09.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:09.653+0000 7fe9bd7fa640 1 -- 172.21.15.38:0/1461308330 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe9b803a070 con 0x7fe9c01512a0 2024-09-21T03:41:09.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:09.653+0000 7fe9c599d640 1 -- 172.21.15.38:0/1461308330 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9c00b4170 con 0x7fe9c01512a0 2024-09-21T03:41:09.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:09.653+0000 7fe9bd7fa640 1 -- 172.21.15.38:0/1461308330 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9b802be60 con 0x7fe9c01512a0 2024-09-21T03:41:09.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:09.653+0000 7fe9c599d640 1 -- 172.21.15.38:0/1461308330 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9c00b4730 con 0x7fe9c01512a0 2024-09-21T03:41:09.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:09.654+0000 7fe9c599d640 1 -- 172.21.15.38:0/1461308330 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9c0157500 con 0x7fe9c01512a0 2024-09-21T03:41:09.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:09.655+0000 7fe9bd7fa640 1 -- 172.21.15.38:0/1461308330 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fe9b8006330 con 0x7fe9c01512a0 2024-09-21T03:41:09.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:09.656+0000 7fe9bd7fa640 1 --2- 172.21.15.38:0/1461308330 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fe9a0078b90 0x7fe9a007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:09.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:09.656+0000 7fe9bd7fa640 1 -- 172.21.15.38:0/1461308330 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 6906+0+0 (secure 0 0 0) 0x7fe9b80bb1f0 con 0x7fe9c01512a0 2024-09-21T03:41:09.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:09.656+0000 7fe9c499b640 1 --2- 172.21.15.38:0/1461308330 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fe9a0078b90 0x7fe9a007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:09.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:09.657+0000 7fe9c499b640 1 --2- 172.21.15.38:0/1461308330 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fe9a0078b90 0x7fe9a007b0a0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fe9a8009bd0 tx=0x7fe9a8005f90 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:09.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:09.661+0000 7fe9bd7fa640 1 -- 172.21.15.38:0/1461308330 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fe9b8082d10 con 0x7fe9c01512a0 2024-09-21T03:41:09.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:09.861+0000 7fe9c599d640 1 -- 172.21.15.38:0/1461308330 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fe9c015e750 con 0x7fe9c01512a0 2024-09-21T03:41:09.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:09.861+0000 7fe9bd7fa640 1 -- 172.21.15.38:0/1461308330 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 14 entries v103) ==== 81+0+850 (secure 0 0 0) 0x7fe9b8087740 con 0x7fe9c01512a0 2024-09-21T03:41:09.864 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6833/3710736169 2024-09-22T03:40:59.454480+0000 2024-09-21T03:41:09.864 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6832/751618334 2024-09-22T03:40:42.053279+0000 2024-09-21T03:41:09.864 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6835/2934569248 2024-09-22T03:40:26.934591+0000 2024-09-21T03:41:09.864 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6833/751618334 2024-09-22T03:40:42.053279+0000 2024-09-21T03:41:09.864 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6835/3864487171 2024-09-22T03:40:31.989837+0000 2024-09-21T03:41:09.864 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.195:6835/712370966 2024-09-22T03:40:11.633787+0000 2024-09-21T03:41:09.864 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6834/2934569248 2024-09-22T03:40:26.934591+0000 2024-09-21T03:41:09.864 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6834/3864487171 2024-09-22T03:40:31.989837+0000 2024-09-21T03:41:09.864 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.195:6837/712370966 2024-09-22T03:40:11.633787+0000 2024-09-21T03:41:09.864 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6834/425672974 2024-09-22T03:40:46.105521+0000 2024-09-21T03:41:09.864 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6835/425672974 2024-09-22T03:40:46.105521+0000 2024-09-21T03:41:09.865 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6834/2751962827 2024-09-22T03:40:38.029580+0000 2024-09-21T03:41:09.865 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6835/2751962827 2024-09-22T03:40:38.029580+0000 2024-09-21T03:41:09.865 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6832/3710736169 2024-09-22T03:40:59.454480+0000 2024-09-21T03:41:09.865 INFO:teuthology.orchestra.run.smithi038.stderr:listed 14 entries 2024-09-21T03:41:09.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:09.864+0000 7fe9c599d640 1 -- 172.21.15.38:0/1461308330 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fe9a0078b90 msgr2=0x7fe9a007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:09.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:09.864+0000 7fe9c599d640 1 --2- 172.21.15.38:0/1461308330 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fe9a0078b90 0x7fe9a007b0a0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fe9a8009bd0 tx=0x7fe9a8005f90 comp rx=0 tx=0).stop 2024-09-21T03:41:09.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:09.864+0000 7fe9c599d640 1 -- 172.21.15.38:0/1461308330 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe9c01512a0 msgr2=0x7fe9c0159b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:09.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:09.864+0000 7fe9c599d640 1 --2- 172.21.15.38:0/1461308330 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe9c01512a0 0x7fe9c0159b60 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7fe9b800c730 tx=0x7fe9b8003c20 comp rx=0 tx=0).stop 2024-09-21T03:41:09.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:09.864+0000 7fe9c599d640 1 -- 172.21.15.38:0/1461308330 shutdown_connections 2024-09-21T03:41:09.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:09.864+0000 7fe9c599d640 1 --2- 172.21.15.38:0/1461308330 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe9c01512a0 0x7fe9c0159b60 unknown :-1 s=CLOSED pgs=1409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:09.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:09.864+0000 7fe9c599d640 1 --2- 172.21.15.38:0/1461308330 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fe9a0078b90 0x7fe9a007b0a0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:09.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:09.864+0000 7fe9c599d640 1 --2- 172.21.15.38:0/1461308330 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe9c01570a0 0x7fe9c0150d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:09.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:09.864+0000 7fe9c599d640 1 --2- 172.21.15.38:0/1461308330 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe9ac0053a0 0x7fe9c0156b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:09.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:09.864+0000 7fe9c599d640 1 -- 172.21.15.38:0/1461308330 >> 172.21.15.38:0/1461308330 conn(0x7fe9c00d41e0 msgr2=0x7fe9c01e8080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:09.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:09.864+0000 7fe9c599d640 1 -- 172.21.15.38:0/1461308330 shutdown_connections 2024-09-21T03:41:09.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:09.864+0000 7fe9c599d640 1 -- 172.21.15.38:0/1461308330 wait complete. 2024-09-21T03:41:09.877 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-21T03:41:09.877 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:41:09.877 DEBUG:teuthology.orchestra.run.smithi038:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-09-21T03:41:09.920 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi038.front.sepia.ceph.com 2024-09-21T03:41:09.920 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:41:09.920 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T03:41:09.976 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-21T03:41:10.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:10.131+0000 7f4ee020a640 1 -- 172.21.15.38:0/3958636655 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4ed815ee80 msgr2=0x7f4ed80d0170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:10.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:10.131+0000 7f4ee020a640 1 --2- 172.21.15.38:0/3958636655 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4ed815ee80 0x7f4ed80d0170 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f4ed80d06b0 tx=0x7f4ed002a440 comp rx=0 tx=0).stop 2024-09-21T03:41:10.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:10.131+0000 7f4ee020a640 1 -- 172.21.15.38:0/3958636655 shutdown_connections 2024-09-21T03:41:10.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:10.131+0000 7f4ee020a640 1 --2- 172.21.15.38:0/3958636655 >> v2:172.21.15.195:3300/0 conn(0x7f4ed80b9ad0 0x7f4ed80d0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:10.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:10.131+0000 7f4ee020a640 1 --2- 172.21.15.38:0/3958636655 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4ed815ee80 0x7f4ed80d0170 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:10.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:10.131+0000 7f4ee020a640 1 -- 172.21.15.38:0/3958636655 >> 172.21.15.38:0/3958636655 conn(0x7f4ed809d830 msgr2=0x7f4ed80b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:10.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:10.131+0000 7f4ee020a640 1 -- 172.21.15.38:0/3958636655 shutdown_connections 2024-09-21T03:41:10.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:10.131+0000 7f4ee020a640 1 -- 172.21.15.38:0/3958636655 wait complete. 2024-09-21T03:41:10.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:10.133+0000 7f4ee020a640 1 Processor -- start 2024-09-21T03:41:10.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:10.133+0000 7f4ee020a640 1 -- start start 2024-09-21T03:41:10.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:10.134+0000 7f4ee020a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4ed80b9ad0 0x7f4ed81f8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:10.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:10.134+0000 7f4ee020a640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4ed80d1120 0x7f4ed81f5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:10.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:10.134+0000 7f4ee020a640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4ed806f920 0x7f4ed815a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:10.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:10.134+0000 7f4ee020a640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f4ed80b6700 con 0x7f4ed80b9ad0 2024-09-21T03:41:10.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:10.134+0000 7f4ee020a640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f4ed80b6580 con 0x7f4ed806f920 2024-09-21T03:41:10.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:10.134+0000 7f4ee020a640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f4ed80b6880 con 0x7f4ed80d1120 2024-09-21T03:41:10.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:10.134+0000 7f4eddfa6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4ed80b9ad0 0x7f4ed81f8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:10.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:10.134+0000 7f4eddfa6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4ed80b9ad0 0x7f4ed81f8b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50530/0 (socket says 172.21.15.38:50530) 2024-09-21T03:41:10.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:10.135+0000 7f4eddfa6640 1 -- 172.21.15.38:0/1761503012 learned_addr learned my addr 172.21.15.38:0/1761503012 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:41:10.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:10.135+0000 7f4edd7a5640 1 --2- 172.21.15.38:0/1761503012 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4ed80d1120 0x7f4ed81f5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:10.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:10.135+0000 7f4ede7a7640 1 --2- 172.21.15.38:0/1761503012 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4ed806f920 0x7f4ed815a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:10.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:10.135+0000 7f4eddfa6640 1 -- 172.21.15.38:0/1761503012 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4ed80d1120 msgr2=0x7f4ed81f5eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:10.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:10.135+0000 7f4eddfa6640 1 --2- 172.21.15.38:0/1761503012 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4ed80d1120 0x7f4ed81f5eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:10.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:10.135+0000 7f4eddfa6640 1 -- 172.21.15.38:0/1761503012 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4ed806f920 msgr2=0x7f4ed815a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:10.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:10.135+0000 7f4eddfa6640 1 --2- 172.21.15.38:0/1761503012 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4ed806f920 0x7f4ed815a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:10.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:10.135+0000 7f4eddfa6640 1 -- 172.21.15.38:0/1761503012 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ed81f93e0 con 0x7f4ed80b9ad0 2024-09-21T03:41:10.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:10.135+0000 7f4eddfa6640 1 --2- 172.21.15.38:0/1761503012 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4ed80b9ad0 0x7f4ed81f8b90 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f4ed0000c00 tx=0x7f4ed00039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:10.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:10.135+0000 7f4ed67fc640 1 -- 172.21.15.38:0/1761503012 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ed003f070 con 0x7f4ed80b9ad0 2024-09-21T03:41:10.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:10.135+0000 7f4ed67fc640 1 -- 172.21.15.38:0/1761503012 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4ed00047a0 con 0x7f4ed80b9ad0 2024-09-21T03:41:10.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:10.135+0000 7f4ee020a640 1 -- 172.21.15.38:0/1761503012 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ed81f96c0 con 0x7f4ed80b9ad0 2024-09-21T03:41:10.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:10.135+0000 7f4ed67fc640 1 -- 172.21.15.38:0/1761503012 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ed0035030 con 0x7f4ed80b9ad0 2024-09-21T03:41:10.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:10.135+0000 7f4ee020a640 1 -- 172.21.15.38:0/1761503012 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ed81f56b0 con 0x7f4ed80b9ad0 2024-09-21T03:41:10.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:10.136+0000 7f4ee020a640 1 -- 172.21.15.38:0/1761503012 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ed80b6d90 con 0x7f4ed80b9ad0 2024-09-21T03:41:10.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:10.137+0000 7f4ed67fc640 1 -- 172.21.15.38:0/1761503012 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f4ed0042070 con 0x7f4ed80b9ad0 2024-09-21T03:41:10.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:10.138+0000 7f4ed67fc640 1 --2- 172.21.15.38:0/1761503012 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4ec4078b90 0x7f4ec407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:10.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:10.138+0000 7f4ed67fc640 1 -- 172.21.15.38:0/1761503012 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 6906+0+0 (secure 0 0 0) 0x7f4ed00bc100 con 0x7f4ed80b9ad0 2024-09-21T03:41:10.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:10.138+0000 7f4edd7a5640 1 --2- 172.21.15.38:0/1761503012 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4ec4078b90 0x7f4ec407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:10.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:10.138+0000 7f4edd7a5640 1 --2- 172.21.15.38:0/1761503012 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4ec4078b90 0x7f4ec407b0a0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f4ed815c2e0 tx=0x7f4ecc005f90 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:10.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:10.142+0000 7f4ed67fc640 1 -- 172.21.15.38:0/1761503012 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f4ed81592d0 con 0x7f4ed80b9ad0 2024-09-21T03:41:10.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:10.370+0000 7f4ee020a640 1 -- 172.21.15.38:0/1761503012 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f4ed80b4140 con 0x7f4ed80b9ad0 2024-09-21T03:41:10.373 INFO:tasks.ceph.mon.a.smithi038.stderr:2024-09-21T03:41:10.371+0000 7f3f45713640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T03:41:10.374 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:10.373+0000 7f4ed67fc640 1 -- 172.21.15.38:0/1761503012 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v132) ==== 161+0+0 (secure 0 0 0) 0x7f4ed0083a30 con 0x7f4ed80b9ad0 2024-09-21T03:41:10.375 INFO:teuthology.orchestra.run.smithi038.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-21T03:41:10.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:10.376+0000 7f4ee020a640 1 -- 172.21.15.38:0/1761503012 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4ec4078b90 msgr2=0x7f4ec407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:10.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:10.376+0000 7f4ee020a640 1 --2- 172.21.15.38:0/1761503012 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4ec4078b90 0x7f4ec407b0a0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f4ed815c2e0 tx=0x7f4ecc005f90 comp rx=0 tx=0).stop 2024-09-21T03:41:10.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:10.376+0000 7f4ee020a640 1 -- 172.21.15.38:0/1761503012 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4ed80b9ad0 msgr2=0x7f4ed81f8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:10.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:10.376+0000 7f4ee020a640 1 --2- 172.21.15.38:0/1761503012 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4ed80b9ad0 0x7f4ed81f8b90 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f4ed0000c00 tx=0x7f4ed00039e0 comp rx=0 tx=0).stop 2024-09-21T03:41:10.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:10.376+0000 7f4ee020a640 1 -- 172.21.15.38:0/1761503012 shutdown_connections 2024-09-21T03:41:10.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:10.376+0000 7f4ee020a640 1 --2- 172.21.15.38:0/1761503012 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4ed806f920 0x7f4ed815a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:10.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:10.376+0000 7f4ee020a640 1 --2- 172.21.15.38:0/1761503012 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4ec4078b90 0x7f4ec407b0a0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:10.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:10.376+0000 7f4ee020a640 1 --2- 172.21.15.38:0/1761503012 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4ed80d1120 0x7f4ed81f5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:10.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:10.376+0000 7f4ee020a640 1 --2- 172.21.15.38:0/1761503012 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4ed80b9ad0 0x7f4ed81f8b90 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:10.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:10.376+0000 7f4ee020a640 1 -- 172.21.15.38:0/1761503012 >> 172.21.15.38:0/1761503012 conn(0x7f4ed809d830 msgr2=0x7f4ed815d990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:10.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:10.377+0000 7f4ee020a640 1 -- 172.21.15.38:0/1761503012 shutdown_connections 2024-09-21T03:41:10.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:10.377+0000 7f4ee020a640 1 -- 172.21.15.38:0/1761503012 wait complete. 2024-09-21T03:41:10.380 INFO:tasks.ceph.mds.c.smithi038.stderr: -692> 2024-09-21T03:41:05.467+0000 7fdeedc4b640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T03:41:10.391 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-09-21T03:41:10.438 INFO:tasks.ceph.mds.c.smithi038.stdout:starting mds.c at 2024-09-21T03:41:10.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:10.509+0000 7fa53d494640 1 -- 172.21.15.38:0/1556175800 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa53806def0 msgr2=0x7fa53806f410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:10.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:10.509+0000 7fa53d494640 1 --2- 172.21.15.38:0/1556175800 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa53806def0 0x7fa53806f410 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7fa53806e6e0 tx=0x7fa52802a540 comp rx=0 tx=0).stop 2024-09-21T03:41:10.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:10.509+0000 7fa53d494640 1 -- 172.21.15.38:0/1556175800 shutdown_connections 2024-09-21T03:41:10.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:10.509+0000 7fa53d494640 1 --2- 172.21.15.38:0/1556175800 >> v2:172.21.15.195:3300/0 conn(0x7fa53815bfc0 0x7fa53815f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:10.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:10.509+0000 7fa53d494640 1 --2- 172.21.15.38:0/1556175800 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa5380b5260 0x7fa53815ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:10.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:10.509+0000 7fa53d494640 1 --2- 172.21.15.38:0/1556175800 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa53806def0 0x7fa53806f410 unknown :-1 s=CLOSED pgs=1415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:10.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:10.509+0000 7fa53d494640 1 -- 172.21.15.38:0/1556175800 >> 172.21.15.38:0/1556175800 conn(0x7fa5380cdca0 msgr2=0x7fa5380ce0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:10.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:10.509+0000 7fa53d494640 1 -- 172.21.15.38:0/1556175800 shutdown_connections 2024-09-21T03:41:10.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:10.509+0000 7fa53d494640 1 -- 172.21.15.38:0/1556175800 wait complete. 2024-09-21T03:41:10.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:10.511+0000 7fa53d494640 1 Processor -- start 2024-09-21T03:41:10.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:10.511+0000 7fa53d494640 1 -- start start 2024-09-21T03:41:10.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:10.513+0000 7fa53d494640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa53806def0 0x7fa538144780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:10.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:10.513+0000 7fa53d494640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa5380b5260 0x7fa53809a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:10.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:10.513+0000 7fa53d494640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa53815bfc0 0x7fa538145a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:10.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:10.513+0000 7fa53d494640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa53815fcf0 con 0x7fa5380b5260 2024-09-21T03:41:10.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:10.513+0000 7fa53d494640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fa53815fb70 con 0x7fa53815bfc0 2024-09-21T03:41:10.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:10.513+0000 7fa53d494640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fa53815fe70 con 0x7fa53806def0 2024-09-21T03:41:10.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:10.513+0000 7fa5367fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa5380b5260 0x7fa53809a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:10.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:10.513+0000 7fa5367fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa5380b5260 0x7fa53809a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50566/0 (socket says 172.21.15.38:50566) 2024-09-21T03:41:10.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:10.513+0000 7fa5367fc640 1 -- 172.21.15.38:0/3187504880 learned_addr learned my addr 172.21.15.38:0/3187504880 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:41:10.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:10.513+0000 7fa5377fe640 1 --2- 172.21.15.38:0/3187504880 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa53815bfc0 0x7fa538145a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:10.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:10.513+0000 7fa536ffd640 1 --2- 172.21.15.38:0/3187504880 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa53806def0 0x7fa538144780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:10.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:10.513+0000 7fa5367fc640 1 -- 172.21.15.38:0/3187504880 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa53806def0 msgr2=0x7fa538144780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:10.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:10.513+0000 7fa5367fc640 1 --2- 172.21.15.38:0/3187504880 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa53806def0 0x7fa538144780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:10.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:10.513+0000 7fa5367fc640 1 -- 172.21.15.38:0/3187504880 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa53815bfc0 msgr2=0x7fa538145a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:10.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:10.513+0000 7fa5367fc640 1 --2- 172.21.15.38:0/3187504880 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa53815bfc0 0x7fa538145a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:10.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:10.513+0000 7fa5367fc640 1 -- 172.21.15.38:0/3187504880 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa538144cc0 con 0x7fa5380b5260 2024-09-21T03:41:10.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:10.513+0000 7fa5367fc640 1 --2- 172.21.15.38:0/3187504880 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa5380b5260 0x7fa53809a9e0 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7fa52c00e7f0 tx=0x7fa52c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:10.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:10.513+0000 7fa51f7fe640 1 -- 172.21.15.38:0/3187504880 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa52c00ccc0 con 0x7fa5380b5260 2024-09-21T03:41:10.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:10.513+0000 7fa51f7fe640 1 -- 172.21.15.38:0/3187504880 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa52c00ce80 con 0x7fa5380b5260 2024-09-21T03:41:10.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:10.513+0000 7fa53d494640 1 -- 172.21.15.38:0/3187504880 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5381e5270 con 0x7fa5380b5260 2024-09-21T03:41:10.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:10.513+0000 7fa51f7fe640 1 -- 172.21.15.38:0/3187504880 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa52c005280 con 0x7fa5380b5260 2024-09-21T03:41:10.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:10.514+0000 7fa53d494640 1 -- 172.21.15.38:0/3187504880 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa538145270 con 0x7fa5380b5260 2024-09-21T03:41:10.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:10.514+0000 7fa53d494640 1 -- 172.21.15.38:0/3187504880 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa53815ba80 con 0x7fa5380b5260 2024-09-21T03:41:10.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:10.515+0000 7fa51f7fe640 1 -- 172.21.15.38:0/3187504880 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa52c005420 con 0x7fa5380b5260 2024-09-21T03:41:10.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:10.516+0000 7fa51f7fe640 1 --2- 172.21.15.38:0/3187504880 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa524078b90 0x7fa52407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:10.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:10.516+0000 7fa51f7fe640 1 -- 172.21.15.38:0/3187504880 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 6992+0+0 (secure 0 0 0) 0x7fa52c09ba90 con 0x7fa5380b5260 2024-09-21T03:41:10.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:10.516+0000 7fa536ffd640 1 --2- 172.21.15.38:0/3187504880 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa524078b90 0x7fa52407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:10.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:10.517+0000 7fa536ffd640 1 --2- 172.21.15.38:0/3187504880 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa524078b90 0x7fa52407b0a0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fa528033000 tx=0x7fa528004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:10.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:10.520+0000 7fa51f7fe640 1 -- 172.21.15.38:0/3187504880 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fa52c063380 con 0x7fa5380b5260 2024-09-21T03:41:10.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:10.787+0000 7fa53d494640 1 -- 172.21.15.38:0/3187504880 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7fa5381e5000 con 0x7fa5380b5260 2024-09-21T03:41:11.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.376+0000 7fa51f7fe640 1 -- 172.21.15.38:0/3187504880 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 nothing to do: rank is not damaged v133) ==== 107+0+0 (secure 0 0 0) 0x7fa52c067db0 con 0x7fa5380b5260 2024-09-21T03:41:11.378 INFO:teuthology.orchestra.run.smithi038.stderr:nothing to do: rank is not damaged 2024-09-21T03:41:11.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.378+0000 7fa53d494640 1 -- 172.21.15.38:0/3187504880 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa524078b90 msgr2=0x7fa52407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:11.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.378+0000 7fa53d494640 1 --2- 172.21.15.38:0/3187504880 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa524078b90 0x7fa52407b0a0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fa528033000 tx=0x7fa528004730 comp rx=0 tx=0).stop 2024-09-21T03:41:11.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.378+0000 7fa53d494640 1 -- 172.21.15.38:0/3187504880 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa5380b5260 msgr2=0x7fa53809a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:11.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.378+0000 7fa53d494640 1 --2- 172.21.15.38:0/3187504880 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa5380b5260 0x7fa53809a9e0 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7fa52c00e7f0 tx=0x7fa52c00ecb0 comp rx=0 tx=0).stop 2024-09-21T03:41:11.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.379+0000 7fa53d494640 1 -- 172.21.15.38:0/3187504880 shutdown_connections 2024-09-21T03:41:11.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.379+0000 7fa53d494640 1 --2- 172.21.15.38:0/3187504880 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa53815bfc0 0x7fa538145a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:11.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.379+0000 7fa53d494640 1 --2- 172.21.15.38:0/3187504880 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa5380b5260 0x7fa53809a9e0 unknown :-1 s=CLOSED pgs=1416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:11.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.379+0000 7fa53d494640 1 --2- 172.21.15.38:0/3187504880 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa524078b90 0x7fa52407b0a0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:11.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.379+0000 7fa53d494640 1 --2- 172.21.15.38:0/3187504880 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa53806def0 0x7fa538144780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:11.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.379+0000 7fa53d494640 1 -- 172.21.15.38:0/3187504880 >> 172.21.15.38:0/3187504880 conn(0x7fa5380cdca0 msgr2=0x7fa5380b3130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:11.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.379+0000 7fa53d494640 1 -- 172.21.15.38:0/3187504880 shutdown_connections 2024-09-21T03:41:11.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.379+0000 7fa53d494640 1 -- 172.21.15.38:0/3187504880 wait complete. 2024-09-21T03:41:11.393 DEBUG:teuthology.orchestra.run.smithi038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-09-21T03:41:11.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.449+0000 7fae7951bf40 1 Processor -- start 2024-09-21T03:41:11.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.449+0000 7fae7951bf40 1 -- start start 2024-09-21T03:41:11.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.450+0000 7fae7951bf40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5607335dbd00 0x560733620730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:11.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.450+0000 7fae7951bf40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560733607e30 0x560733694c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:11.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.450+0000 7fae7951bf40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560733696950 0x5607336b1540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:11.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.450+0000 7fae7951bf40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x560733588430 con 0x560733696950 2024-09-21T03:41:11.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.450+0000 7fae7951bf40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5607335951f0 con 0x560733607e30 2024-09-21T03:41:11.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.450+0000 7fae7951bf40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x560733529d60 con 0x5607335dbd00 2024-09-21T03:41:11.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.450+0000 7fae72cfc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560733696950 0x5607336b1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:11.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.450+0000 7fae72cfc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560733696950 0x5607336b1540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50590/0 (socket says 172.21.15.38:50590) 2024-09-21T03:41:11.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.450+0000 7fae72cfc640 1 -- 172.21.15.38:0/3898173419 learned_addr learned my addr 172.21.15.38:0/3898173419 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:41:11.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.450+0000 7fae71cfa640 1 --2- 172.21.15.38:0/3898173419 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560733607e30 0x560733694c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:11.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.451+0000 7fae724fb640 1 --2- 172.21.15.38:0/3898173419 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5607335dbd00 0x560733620730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:11.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.451+0000 7fae72cfc640 1 -- 172.21.15.38:0/3898173419 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5607335dbd00 msgr2=0x560733620730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:11.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.451+0000 7fae72cfc640 1 --2- 172.21.15.38:0/3898173419 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5607335dbd00 0x560733620730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:11.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.451+0000 7fae72cfc640 1 -- 172.21.15.38:0/3898173419 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560733607e30 msgr2=0x560733694c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:11.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.451+0000 7fae72cfc640 1 --2- 172.21.15.38:0/3898173419 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560733607e30 0x560733694c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:11.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.451+0000 7fae72cfc640 1 -- 172.21.15.38:0/3898173419 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560733695bb0 con 0x560733696950 2024-09-21T03:41:11.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.451+0000 7fae71cfa640 1 --2- 172.21.15.38:0/3898173419 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560733607e30 0x560733694c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:41:11.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.451+0000 7fae72cfc640 1 --2- 172.21.15.38:0/3898173419 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560733696950 0x5607336b1540 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7fae6c001d60 tx=0x7fae6c00b590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37a15fa4a897d10f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:11.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.451+0000 7fae70cf8640 1 -- 172.21.15.38:0/3898173419 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae6c016070 con 0x560733696950 2024-09-21T03:41:11.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.451+0000 7fae70cf8640 1 -- 172.21.15.38:0/3898173419 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fae6c00f6b0 con 0x560733696950 2024-09-21T03:41:11.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.451+0000 7fae70cf8640 1 -- 172.21.15.38:0/3898173419 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae6c00fb00 con 0x560733696950 2024-09-21T03:41:11.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.452+0000 7fae7951bf40 1 -- 172.21.15.38:0/3898173419 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560733696950 msgr2=0x5607336b1540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:11.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.452+0000 7fae7951bf40 1 --2- 172.21.15.38:0/3898173419 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560733696950 0x5607336b1540 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7fae6c001d60 tx=0x7fae6c00b590 comp rx=0 tx=0).stop 2024-09-21T03:41:11.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.452+0000 7fae7951bf40 1 -- 172.21.15.38:0/3898173419 shutdown_connections 2024-09-21T03:41:11.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.452+0000 7fae7951bf40 1 --2- 172.21.15.38:0/3898173419 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560733696950 0x5607336b1540 unknown :-1 s=CLOSED pgs=1419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:11.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.452+0000 7fae7951bf40 1 --2- 172.21.15.38:0/3898173419 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560733607e30 0x560733694c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:11.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.452+0000 7fae7951bf40 1 --2- 172.21.15.38:0/3898173419 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5607335dbd00 0x560733620730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:11.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.452+0000 7fae7951bf40 1 -- 172.21.15.38:0/3898173419 >> 172.21.15.38:0/3898173419 conn(0x5607334faac0 msgr2=0x5607336ad590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:11.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.452+0000 7fae7951bf40 1 -- 172.21.15.38:0/3898173419 shutdown_connections 2024-09-21T03:41:11.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.452+0000 7fae7951bf40 1 -- 172.21.15.38:0/3898173419 wait complete. 2024-09-21T03:41:11.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.453+0000 7fae7951bf40 1 Processor -- start 2024-09-21T03:41:11.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.453+0000 7fae7951bf40 1 -- start start 2024-09-21T03:41:11.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.454+0000 7fae7951bf40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5607335dbd00 0x56073368ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:11.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.454+0000 7fae7951bf40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560733607e30 0x5607336b1540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:11.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.454+0000 7fae7951bf40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560733696950 0x5607336b1a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:11.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.454+0000 7fae7951bf40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x560733695640 con 0x560733696950 2024-09-21T03:41:11.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.454+0000 7fae7951bf40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x560733695e90 con 0x560733607e30 2024-09-21T03:41:11.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.454+0000 7fae7951bf40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x560733609e70 con 0x5607335dbd00 2024-09-21T03:41:11.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.454+0000 7fae72cfc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560733696950 0x5607336b1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:11.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.454+0000 7fae72cfc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560733696950 0x5607336b1a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50606/0 (socket says 172.21.15.38:50606) 2024-09-21T03:41:11.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.454+0000 7fae72cfc640 1 -- 172.21.15.38:0/4179802055 learned_addr learned my addr 172.21.15.38:0/4179802055 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:41:11.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.454+0000 7fae71cfa640 1 --2- 172.21.15.38:0/4179802055 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560733607e30 0x5607336b1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:11.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.454+0000 7fae724fb640 1 --2- 172.21.15.38:0/4179802055 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5607335dbd00 0x56073368ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:11.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.454+0000 7fae72cfc640 1 -- 172.21.15.38:0/4179802055 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5607335dbd00 msgr2=0x56073368ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:11.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.454+0000 7fae72cfc640 1 --2- 172.21.15.38:0/4179802055 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5607335dbd00 0x56073368ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:11.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.454+0000 7fae72cfc640 1 -- 172.21.15.38:0/4179802055 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560733607e30 msgr2=0x5607336b1540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:11.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.454+0000 7fae72cfc640 1 --2- 172.21.15.38:0/4179802055 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560733607e30 0x5607336b1540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:11.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.454+0000 7fae72cfc640 1 -- 172.21.15.38:0/4179802055 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560733609850 con 0x560733696950 2024-09-21T03:41:11.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.455+0000 7fae72cfc640 1 --2- 172.21.15.38:0/4179802055 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560733696950 0x5607336b1a80 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x5607336091e0 tx=0x7fae6c001ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:11.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.455+0000 7fae637fe640 1 -- 172.21.15.38:0/4179802055 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae6c022070 con 0x560733696950 2024-09-21T03:41:11.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.455+0000 7fae637fe640 1 -- 172.21.15.38:0/4179802055 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fae6c017040 con 0x560733696950 2024-09-21T03:41:11.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.455+0000 7fae7951bf40 1 -- 172.21.15.38:0/4179802055 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5607336b08e0 con 0x560733696950 2024-09-21T03:41:11.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.455+0000 7fae637fe640 1 -- 172.21.15.38:0/4179802055 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae6c016070 con 0x560733696950 2024-09-21T03:41:11.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.455+0000 7fae7951bf40 1 -- 172.21.15.38:0/4179802055 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56073361f770 con 0x560733696950 2024-09-21T03:41:11.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.457+0000 7fae637fe640 1 -- 172.21.15.38:0/4179802055 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fae6c025070 con 0x560733696950 2024-09-21T03:41:11.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.457+0000 7fae637fe640 1 --2- 172.21.15.38:0/4179802055 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fae54077570 0x7fae54079a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:11.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.457+0000 7fae637fe640 1 -- 172.21.15.38:0/4179802055 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 6992+0+0 (secure 0 0 0) 0x7fae6c09a9e0 con 0x560733696950 2024-09-21T03:41:11.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.457+0000 7fae724fb640 1 --2- 172.21.15.38:0/4179802055 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fae54077570 0x7fae54079a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:11.462 INFO:teuthology.orchestra.run.smithi038.stdout:Importing pool 2024-09-21T03:41:11.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.457+0000 7fae7951bf40 1 --2- 172.21.15.38:0/4179802055 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x56073361ec10 0x56073361f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:11.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.457+0000 7fae7951bf40 1 -- 172.21.15.38:0/4179802055 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x5607336901b0 con 0x56073361ec10 2024-09-21T03:41:11.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.458+0000 7fae71cfa640 1 --2- 172.21.15.38:0/4179802055 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x56073361ec10 0x56073361f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:11.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.458+0000 7fae724fb640 1 --2- 172.21.15.38:0/4179802055 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fae54077570 0x7fae54079a80 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fae6400d5a0 tx=0x7fae64007040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:11.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.458+0000 7fae71cfa640 1 --2- 172.21.15.38:0/4179802055 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x56073361ec10 0x56073361f020 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:11.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.460+0000 7fae71cfa640 1 -- 172.21.15.38:0/4179802055 <== osd.7 v2:172.21.15.195:6818/1606106342 1 ==== osd_op_reply(1 601.00000000 [create] v104'226 uv200 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fae6804eb00 con 0x56073361ec10 2024-09-21T03:41:11.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.460+0000 7fae7951bf40 1 -- 172.21.15.38:0/4179802055 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e104) -- 0x560733754d60 con 0x56073361ec10 2024-09-21T03:41:11.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.461+0000 7fae71cfa640 1 -- 172.21.15.38:0/4179802055 <== osd.7 v2:172.21.15.195:6818/1606106342 2 ==== osd_op_reply(2 601.00000000 [delete] v104'227 uv227 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fae6804eb00 con 0x56073361ec10 2024-09-21T03:41:11.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.461+0000 7fae7951bf40 1 -- 172.21.15.38:0/4179802055 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x560733755100 con 0x56073361ec10 2024-09-21T03:41:11.464 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-09-21T03:41:11.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.463+0000 7fae71cfa640 1 -- 172.21.15.38:0/4179802055 <== osd.7 v2:172.21.15.195:6818/1606106342 3 ==== osd_op_reply(3 601.00000000 [create] v104'228 uv228 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fae6804eb00 con 0x56073361ec10 2024-09-21T03:41:11.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.463+0000 7fae7951bf40 1 -- 172.21.15.38:0/4179802055 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x5607337564b0 con 0x56073361ec10 2024-09-21T03:41:11.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.464+0000 7fae71cfa640 1 -- 172.21.15.38:0/4179802055 <== osd.7 v2:172.21.15.195:6818/1606106342 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v104'229 uv229 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fae6804eb00 con 0x56073361ec10 2024-09-21T03:41:11.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.464+0000 7fae7951bf40 1 -- 172.21.15.38:0/4179802055 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x560733756850 con 0x56073361ec10 2024-09-21T03:41:11.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.466+0000 7fae71cfa640 1 -- 172.21.15.38:0/4179802055 <== osd.7 v2:172.21.15.195:6818/1606106342 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v104'230 uv230 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fae6804eb00 con 0x56073361ec10 2024-09-21T03:41:11.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.466+0000 7fae7951bf40 1 -- 172.21.15.38:0/4179802055 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x5607337554a0 con 0x56073361ec10 2024-09-21T03:41:11.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.467+0000 7fae71cfa640 1 -- 172.21.15.38:0/4179802055 <== osd.7 v2:172.21.15.195:6818/1606106342 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v104'231 uv231 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fae6804eb00 con 0x56073361ec10 2024-09-21T03:41:11.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.467+0000 7fae7951bf40 1 -- 172.21.15.38:0/4179802055 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x560733755890 con 0x56073361ec10 2024-09-21T03:41:11.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.469+0000 7fae71cfa640 1 -- 172.21.15.38:0/4179802055 <== osd.7 v2:172.21.15.195:6818/1606106342 7 ==== osd_op_reply(7 602.00000000 [create] v104'232 uv206 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fae6804eb00 con 0x56073361ec10 2024-09-21T03:41:11.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.469+0000 7fae7951bf40 1 -- 172.21.15.38:0/4179802055 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e104) -- 0x560733755890 con 0x56073361ec10 2024-09-21T03:41:11.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.470+0000 7fae71cfa640 1 -- 172.21.15.38:0/4179802055 <== osd.7 v2:172.21.15.195:6818/1606106342 8 ==== osd_op_reply(8 602.00000000 [delete] v104'233 uv233 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fae6804eb00 con 0x56073361ec10 2024-09-21T03:41:11.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.470+0000 7fae7951bf40 1 -- 172.21.15.38:0/4179802055 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x560733755890 con 0x56073361ec10 2024-09-21T03:41:11.473 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-09-21T03:41:11.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.471+0000 7fae71cfa640 1 -- 172.21.15.38:0/4179802055 <== osd.7 v2:172.21.15.195:6818/1606106342 9 ==== osd_op_reply(9 602.00000000 [create] v104'234 uv234 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fae6804eb00 con 0x56073361ec10 2024-09-21T03:41:11.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.472+0000 7fae7951bf40 1 -- 172.21.15.38:0/4179802055 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x560733755890 con 0x56073361ec10 2024-09-21T03:41:11.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.473+0000 7fae71cfa640 1 -- 172.21.15.38:0/4179802055 <== osd.7 v2:172.21.15.195:6818/1606106342 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v104'235 uv235 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fae6804eb00 con 0x56073361ec10 2024-09-21T03:41:11.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.473+0000 7fae7951bf40 1 -- 172.21.15.38:0/4179802055 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x560733755890 con 0x56073361ec10 2024-09-21T03:41:11.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.475+0000 7fae71cfa640 1 -- 172.21.15.38:0/4179802055 <== osd.7 v2:172.21.15.195:6818/1606106342 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v104'236 uv236 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fae6804eb00 con 0x56073361ec10 2024-09-21T03:41:11.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.475+0000 7fae7951bf40 1 -- 172.21.15.38:0/4179802055 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x560733755890 con 0x56073361ec10 2024-09-21T03:41:11.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.476+0000 7fae71cfa640 1 -- 172.21.15.38:0/4179802055 <== osd.7 v2:172.21.15.195:6818/1606106342 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v104'237 uv237 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fae6804eb00 con 0x56073361ec10 2024-09-21T03:41:11.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.476+0000 7fae7951bf40 1 -- 172.21.15.38:0/4179802055 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:13 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x560733755890 con 0x56073361ec10 2024-09-21T03:41:11.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.477+0000 7fae71cfa640 1 -- 172.21.15.38:0/4179802055 <== osd.7 v2:172.21.15.195:6818/1606106342 13 ==== osd_op_reply(13 10000000000.00000000 [create] v104'238 uv213 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7fae6804eb00 con 0x56073361ec10 2024-09-21T03:41:11.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.478+0000 7fae7951bf40 1 -- 172.21.15.38:0/4179802055 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:14 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e104) -- 0x560733755890 con 0x56073361ec10 2024-09-21T03:41:11.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.479+0000 7fae71cfa640 1 -- 172.21.15.38:0/4179802055 <== osd.7 v2:172.21.15.195:6818/1606106342 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v104'239 uv239 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fae6804eb00 con 0x56073361ec10 2024-09-21T03:41:11.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.479+0000 7fae7951bf40 1 -- 172.21.15.38:0/4179802055 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:15 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x560733755890 con 0x56073361ec10 2024-09-21T03:41:11.482 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-09-21T03:41:11.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.480+0000 7fae71cfa640 1 -- 172.21.15.38:0/4179802055 <== osd.7 v2:172.21.15.195:6818/1606106342 15 ==== osd_op_reply(15 10000000000.00000000 [create] v104'240 uv240 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fae6804eb00 con 0x56073361ec10 2024-09-21T03:41:11.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.480+0000 7fae7951bf40 1 -- 172.21.15.38:0/4179802055 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:16 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x560733755890 con 0x56073361ec10 2024-09-21T03:41:11.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.482+0000 7fae71cfa640 1 -- 172.21.15.38:0/4179802055 <== osd.7 v2:172.21.15.195:6818/1606106342 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v104'241 uv241 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fae6804eb00 con 0x56073361ec10 2024-09-21T03:41:11.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.482+0000 7fae7951bf40 1 -- 172.21.15.38:0/4179802055 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:17 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x560733755890 con 0x56073361ec10 2024-09-21T03:41:11.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.483+0000 7fae71cfa640 1 -- 172.21.15.38:0/4179802055 <== osd.7 v2:172.21.15.195:6818/1606106342 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v104'242 uv242 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fae6804eb00 con 0x56073361ec10 2024-09-21T03:41:11.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.483+0000 7fae7951bf40 1 -- 172.21.15.38:0/4179802055 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:18 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x560733755890 con 0x56073361ec10 2024-09-21T03:41:11.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.485+0000 7fae71cfa640 1 -- 172.21.15.38:0/4179802055 <== osd.7 v2:172.21.15.195:6818/1606106342 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v104'243 uv243 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fae6804eb00 con 0x56073361ec10 2024-09-21T03:41:11.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.485+0000 7fae7951bf40 1 -- 172.21.15.38:0/4179802055 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:19 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=585b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x560733755890 con 0x56073361ec10 2024-09-21T03:41:11.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.486+0000 7fae71cfa640 1 -- 172.21.15.38:0/4179802055 <== osd.7 v2:172.21.15.195:6818/1606106342 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v104'244 uv244 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fae6804eb00 con 0x56073361ec10 2024-09-21T03:41:11.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.486+0000 7fae7951bf40 1 -- 172.21.15.38:0/4179802055 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:20 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x560733755890 con 0x56073361ec10 2024-09-21T03:41:11.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.488+0000 7fae71cfa640 1 -- 172.21.15.38:0/4179802055 <== osd.7 v2:172.21.15.195:6818/1606106342 20 ==== osd_op_reply(20 600.00000000 [create] v104'245 uv219 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fae6804eb00 con 0x56073361ec10 2024-09-21T03:41:11.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.488+0000 7fae7951bf40 1 -- 172.21.15.38:0/4179802055 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:21 13.10 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e104) -- 0x560733755890 con 0x56073361ec10 2024-09-21T03:41:11.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.490+0000 7fae71cfa640 1 -- 172.21.15.38:0/4179802055 <== osd.7 v2:172.21.15.195:6818/1606106342 21 ==== osd_op_reply(21 600.00000000 [delete] v104'246 uv246 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fae6804eb00 con 0x56073361ec10 2024-09-21T03:41:11.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.490+0000 7fae7951bf40 1 -- 172.21.15.38:0/4179802055 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:22 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x560733755890 con 0x56073361ec10 2024-09-21T03:41:11.493 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-09-21T03:41:11.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.491+0000 7fae71cfa640 1 -- 172.21.15.38:0/4179802055 <== osd.7 v2:172.21.15.195:6818/1606106342 22 ==== osd_op_reply(22 600.00000000 [create] v104'247 uv247 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fae6804eb00 con 0x56073361ec10 2024-09-21T03:41:11.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.491+0000 7fae7951bf40 1 -- 172.21.15.38:0/4179802055 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:23 13.10 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x560733755890 con 0x56073361ec10 2024-09-21T03:41:11.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.493+0000 7fae71cfa640 1 -- 172.21.15.38:0/4179802055 <== osd.7 v2:172.21.15.195:6818/1606106342 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v104'248 uv248 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fae6804eb00 con 0x56073361ec10 2024-09-21T03:41:11.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.493+0000 7fae7951bf40 1 -- 172.21.15.38:0/4179802055 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:24 13.10 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x560733755890 con 0x56073361ec10 2024-09-21T03:41:11.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.494+0000 7fae71cfa640 1 -- 172.21.15.38:0/4179802055 <== osd.7 v2:172.21.15.195:6818/1606106342 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v104'249 uv249 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fae6804eb00 con 0x56073361ec10 2024-09-21T03:41:11.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.494+0000 7fae7951bf40 1 -- 172.21.15.38:0/4179802055 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:25 13.10 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x560733755890 con 0x56073361ec10 2024-09-21T03:41:11.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.496+0000 7fae71cfa640 1 -- 172.21.15.38:0/4179802055 <== osd.7 v2:172.21.15.195:6818/1606106342 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v104'250 uv250 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fae6804eb00 con 0x56073361ec10 2024-09-21T03:41:11.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.496+0000 7fae7951bf40 1 -- 172.21.15.38:0/4179802055 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:26 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x560733755890 con 0x56073361ec10 2024-09-21T03:41:11.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.497+0000 7fae71cfa640 1 -- 172.21.15.38:0/4179802055 <== osd.7 v2:172.21.15.195:6818/1606106342 26 ==== osd_op_reply(26 603.00000000 [create] v104'251 uv225 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fae6804eb00 con 0x56073361ec10 2024-09-21T03:41:11.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.497+0000 7fae7951bf40 1 -- 172.21.15.38:0/4179802055 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:27 13.10 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e104) -- 0x560733755890 con 0x56073361ec10 2024-09-21T03:41:11.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.498+0000 7fae71cfa640 1 -- 172.21.15.38:0/4179802055 <== osd.7 v2:172.21.15.195:6818/1606106342 27 ==== osd_op_reply(27 603.00000000 [delete] v104'252 uv252 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fae6804eb00 con 0x56073361ec10 2024-09-21T03:41:11.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.499+0000 7fae7951bf40 1 -- 172.21.15.38:0/4179802055 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:28 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x560733755890 con 0x56073361ec10 2024-09-21T03:41:11.502 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-09-21T03:41:11.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.500+0000 7fae71cfa640 1 -- 172.21.15.38:0/4179802055 <== osd.7 v2:172.21.15.195:6818/1606106342 28 ==== osd_op_reply(28 603.00000000 [create] v104'253 uv253 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fae6804eb00 con 0x56073361ec10 2024-09-21T03:41:11.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.500+0000 7fae7951bf40 1 -- 172.21.15.38:0/4179802055 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:29 13.10 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x560733755890 con 0x56073361ec10 2024-09-21T03:41:11.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.501+0000 7fae71cfa640 1 -- 172.21.15.38:0/4179802055 <== osd.7 v2:172.21.15.195:6818/1606106342 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v104'254 uv254 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fae6804eb00 con 0x56073361ec10 2024-09-21T03:41:11.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.501+0000 7fae7951bf40 1 -- 172.21.15.38:0/4179802055 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:30 13.10 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x560733755890 con 0x56073361ec10 2024-09-21T03:41:11.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.503+0000 7fae71cfa640 1 -- 172.21.15.38:0/4179802055 <== osd.7 v2:172.21.15.195:6818/1606106342 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v104'255 uv255 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fae6804eb00 con 0x56073361ec10 2024-09-21T03:41:11.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.503+0000 7fae7951bf40 1 -- 172.21.15.38:0/4179802055 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:31 13.10 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x560733755890 con 0x56073361ec10 2024-09-21T03:41:11.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.504+0000 7fae71cfa640 1 -- 172.21.15.38:0/4179802055 <== osd.7 v2:172.21.15.195:6818/1606106342 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v104'256 uv256 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fae6804eb00 con 0x56073361ec10 2024-09-21T03:41:11.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.504+0000 7fae7951bf40 1 --2- 172.21.15.38:0/4179802055 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x560733756bf0 0x560733759010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:11.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.504+0000 7fae7951bf40 1 -- 172.21.15.38:0/4179802055 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x560733755890 con 0x560733756bf0 2024-09-21T03:41:11.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.505+0000 7fae724fb640 1 --2- 172.21.15.38:0/4179802055 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x560733756bf0 0x560733759010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:11.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.505+0000 7fae724fb640 1 --2- 172.21.15.38:0/4179802055 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x560733756bf0 0x560733759010 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:11.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.510+0000 7fae724fb640 1 -- 172.21.15.38:0/4179802055 <== osd.6 v2:172.21.15.195:6826/4055825675 1 ==== osd_op_reply(32 1.00000000.inode [create] v104'176 uv155 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7fae640137d0 con 0x560733756bf0 2024-09-21T03:41:11.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.510+0000 7fae7951bf40 1 -- 172.21.15.38:0/4179802055 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e104) -- 0x560733759550 con 0x560733756bf0 2024-09-21T03:41:11.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.511+0000 7fae724fb640 1 -- 172.21.15.38:0/4179802055 <== osd.6 v2:172.21.15.195:6826/4055825675 2 ==== osd_op_reply(33 1.00000000.inode [delete] v104'177 uv177 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fae640137d0 con 0x560733756bf0 2024-09-21T03:41:11.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.511+0000 7fae7951bf40 1 -- 172.21.15.38:0/4179802055 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x5607337598f0 con 0x560733756bf0 2024-09-21T03:41:11.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.513+0000 7fae724fb640 1 -- 172.21.15.38:0/4179802055 <== osd.6 v2:172.21.15.195:6826/4055825675 3 ==== osd_op_reply(34 1.00000000.inode [create] v104'178 uv178 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fae640137d0 con 0x560733756bf0 2024-09-21T03:41:11.515 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-09-21T03:41:11.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.513+0000 7fae7951bf40 1 -- 172.21.15.38:0/4179802055 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x560733759c90 con 0x560733756bf0 2024-09-21T03:41:11.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.517+0000 7fae724fb640 1 -- 172.21.15.38:0/4179802055 <== osd.6 v2:172.21.15.195:6826/4055825675 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v104'179 uv179 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fae640137d0 con 0x560733756bf0 2024-09-21T03:41:11.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.517+0000 7fae7951bf40 1 -- 172.21.15.38:0/4179802055 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x56073375a030 con 0x560733756bf0 2024-09-21T03:41:11.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.518+0000 7fae724fb640 1 -- 172.21.15.38:0/4179802055 <== osd.6 v2:172.21.15.195:6826/4055825675 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v104'180 uv180 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fae640137d0 con 0x560733756bf0 2024-09-21T03:41:11.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.518+0000 7fae7951bf40 1 -- 172.21.15.38:0/4179802055 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x56073375a3d0 con 0x560733756bf0 2024-09-21T03:41:11.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.519+0000 7fae724fb640 1 -- 172.21.15.38:0/4179802055 <== osd.6 v2:172.21.15.195:6826/4055825675 6 ==== osd_op_reply(37 200.00000000 [create] v104'181 uv174 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fae640137d0 con 0x560733756bf0 2024-09-21T03:41:11.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.519+0000 7fae7951bf40 1 -- 172.21.15.38:0/4179802055 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e104) -- 0x56073375a770 con 0x560733756bf0 2024-09-21T03:41:11.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.520+0000 7fae724fb640 1 -- 172.21.15.38:0/4179802055 <== osd.6 v2:172.21.15.195:6826/4055825675 7 ==== osd_op_reply(38 200.00000000 [delete] v104'182 uv182 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fae640137d0 con 0x560733756bf0 2024-09-21T03:41:11.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.520+0000 7fae7951bf40 1 -- 172.21.15.38:0/4179802055 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x56073375ab10 con 0x560733756bf0 2024-09-21T03:41:11.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.522+0000 7fae724fb640 1 -- 172.21.15.38:0/4179802055 <== osd.6 v2:172.21.15.195:6826/4055825675 8 ==== osd_op_reply(39 200.00000000 [create] v104'183 uv183 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fae640137d0 con 0x560733756bf0 2024-09-21T03:41:11.524 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-09-21T03:41:11.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.522+0000 7fae7951bf40 1 -- 172.21.15.38:0/4179802055 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x56073368fd40 con 0x560733756bf0 2024-09-21T03:41:11.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.523+0000 7fae724fb640 1 -- 172.21.15.38:0/4179802055 <== osd.6 v2:172.21.15.195:6826/4055825675 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v104'184 uv184 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fae640137d0 con 0x560733756bf0 2024-09-21T03:41:11.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.523+0000 7fae7951bf40 1 -- 172.21.15.38:0/4179802055 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x56073368fd40 con 0x560733756bf0 2024-09-21T03:41:11.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.524+0000 7fae724fb640 1 -- 172.21.15.38:0/4179802055 <== osd.6 v2:172.21.15.195:6826/4055825675 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v104'185 uv185 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fae640137d0 con 0x560733756bf0 2024-09-21T03:41:11.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.524+0000 7fae7951bf40 1 -- 172.21.15.38:0/4179802055 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:42 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x56073368fd40 con 0x560733756bf0 2024-09-21T03:41:11.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.525+0000 7fae724fb640 1 -- 172.21.15.38:0/4179802055 <== osd.6 v2:172.21.15.195:6826/4055825675 11 ==== osd_op_reply(42 200.00000001 [create] v104'186 uv175 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fae640137d0 con 0x560733756bf0 2024-09-21T03:41:11.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.525+0000 7fae7951bf40 1 -- 172.21.15.38:0/4179802055 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:43 13.14 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e104) -- 0x56073368fd40 con 0x560733756bf0 2024-09-21T03:41:11.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.527+0000 7fae724fb640 1 -- 172.21.15.38:0/4179802055 <== osd.6 v2:172.21.15.195:6826/4055825675 12 ==== osd_op_reply(43 200.00000001 [delete] v104'187 uv187 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fae640137d0 con 0x560733756bf0 2024-09-21T03:41:11.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.527+0000 7fae7951bf40 1 -- 172.21.15.38:0/4179802055 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:44 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x56073368fd40 con 0x560733756bf0 2024-09-21T03:41:11.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.528+0000 7fae724fb640 1 -- 172.21.15.38:0/4179802055 <== osd.6 v2:172.21.15.195:6826/4055825675 13 ==== osd_op_reply(44 200.00000001 [create] v104'188 uv188 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fae640137d0 con 0x560733756bf0 2024-09-21T03:41:11.530 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-09-21T03:41:11.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.528+0000 7fae7951bf40 1 -- 172.21.15.38:0/4179802055 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:45 13.14 13:2e2fa760:::200.00000001:head [write 0~17490 in=17490b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x56073368fd40 con 0x560733756bf0 2024-09-21T03:41:11.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.529+0000 7fae724fb640 1 -- 172.21.15.38:0/4179802055 <== osd.6 v2:172.21.15.195:6826/4055825675 14 ==== osd_op_reply(45 200.00000001 [write 0~17490] v104'189 uv189 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fae640137d0 con 0x560733756bf0 2024-09-21T03:41:11.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.529+0000 7fae7951bf40 1 -- 172.21.15.38:0/4179802055 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:46 13.14 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x56073368fd40 con 0x560733756bf0 2024-09-21T03:41:11.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.530+0000 7fae724fb640 1 -- 172.21.15.38:0/4179802055 <== osd.6 v2:172.21.15.195:6826/4055825675 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v104'190 uv190 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fae640137d0 con 0x560733756bf0 2024-09-21T03:41:11.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.530+0000 7fae7951bf40 1 -- 172.21.15.38:0/4179802055 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:47 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x56073368fd40 con 0x560733756bf0 2024-09-21T03:41:11.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.531+0000 7fae724fb640 1 -- 172.21.15.38:0/4179802055 <== osd.6 v2:172.21.15.195:6826/4055825675 16 ==== osd_op_reply(47 606.00000000 [create] v104'191 uv171 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fae640137d0 con 0x560733756bf0 2024-09-21T03:41:11.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.531+0000 7fae7951bf40 1 -- 172.21.15.38:0/4179802055 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:48 13.14 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e104) -- 0x56073368fd40 con 0x560733756bf0 2024-09-21T03:41:11.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.533+0000 7fae724fb640 1 -- 172.21.15.38:0/4179802055 <== osd.6 v2:172.21.15.195:6826/4055825675 17 ==== osd_op_reply(48 606.00000000 [delete] v104'192 uv192 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fae640137d0 con 0x560733756bf0 2024-09-21T03:41:11.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.533+0000 7fae7951bf40 1 -- 172.21.15.38:0/4179802055 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:49 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x56073368fd40 con 0x560733756bf0 2024-09-21T03:41:11.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.534+0000 7fae724fb640 1 -- 172.21.15.38:0/4179802055 <== osd.6 v2:172.21.15.195:6826/4055825675 18 ==== osd_op_reply(49 606.00000000 [create] v104'193 uv193 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fae640137d0 con 0x560733756bf0 2024-09-21T03:41:11.536 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-09-21T03:41:11.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.534+0000 7fae7951bf40 1 -- 172.21.15.38:0/4179802055 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:50 13.14 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x56073368fd40 con 0x560733756bf0 2024-09-21T03:41:11.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.535+0000 7fae724fb640 1 -- 172.21.15.38:0/4179802055 <== osd.6 v2:172.21.15.195:6826/4055825675 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v104'194 uv194 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fae640137d0 con 0x560733756bf0 2024-09-21T03:41:11.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.535+0000 7fae7951bf40 1 -- 172.21.15.38:0/4179802055 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:51 13.14 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x56073368fd40 con 0x560733756bf0 2024-09-21T03:41:11.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.536+0000 7fae724fb640 1 -- 172.21.15.38:0/4179802055 <== osd.6 v2:172.21.15.195:6826/4055825675 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v104'195 uv195 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fae640137d0 con 0x560733756bf0 2024-09-21T03:41:11.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.536+0000 7fae7951bf40 1 -- 172.21.15.38:0/4179802055 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:52 13.14 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x56073368fd40 con 0x560733756bf0 2024-09-21T03:41:11.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.537+0000 7fae724fb640 1 -- 172.21.15.38:0/4179802055 <== osd.6 v2:172.21.15.195:6826/4055825675 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v104'196 uv196 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fae640137d0 con 0x560733756bf0 2024-09-21T03:41:11.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.537+0000 7fae7951bf40 1 --2- 172.21.15.38:0/4179802055 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x56073375b4d0 0x56073375d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:11.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.537+0000 7fae7951bf40 1 -- 172.21.15.38:0/4179802055 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:53 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x56073375de30 con 0x56073375b4d0 2024-09-21T03:41:11.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.537+0000 7fae71cfa640 1 --2- 172.21.15.38:0/4179802055 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x56073375b4d0 0x56073375d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:11.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.537+0000 7fae71cfa640 1 --2- 172.21.15.38:0/4179802055 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x56073375b4d0 0x56073375d8f0 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:11.541 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.539+0000 7fae71cfa640 1 -- 172.21.15.38:0/4179802055 <== osd.0 v2:172.21.15.38:6800/2381639433 1 ==== osd_op_reply(53 607.00000000 [create] v104'45 uv44 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fae6804eb00 con 0x56073375b4d0 2024-09-21T03:41:11.541 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.539+0000 7fae7951bf40 1 -- 172.21.15.38:0/4179802055 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:54 13.1c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e104) -- 0x56073375de30 con 0x56073375b4d0 2024-09-21T03:41:11.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.540+0000 7fae71cfa640 1 -- 172.21.15.38:0/4179802055 <== osd.0 v2:172.21.15.38:6800/2381639433 2 ==== osd_op_reply(54 607.00000000 [delete] v104'46 uv46 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fae6804eb00 con 0x56073375b4d0 2024-09-21T03:41:11.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.540+0000 7fae7951bf40 1 -- 172.21.15.38:0/4179802055 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:55 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x56073375e220 con 0x56073375b4d0 2024-09-21T03:41:11.543 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-09-21T03:41:11.543 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.541+0000 7fae71cfa640 1 -- 172.21.15.38:0/4179802055 <== osd.0 v2:172.21.15.38:6800/2381639433 3 ==== osd_op_reply(55 607.00000000 [create] v104'47 uv47 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fae6804eb00 con 0x56073375b4d0 2024-09-21T03:41:11.543 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.541+0000 7fae7951bf40 1 -- 172.21.15.38:0/4179802055 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:56 13.1c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x56073375f230 con 0x56073375b4d0 2024-09-21T03:41:11.544 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.542+0000 7fae71cfa640 1 -- 172.21.15.38:0/4179802055 <== osd.0 v2:172.21.15.38:6800/2381639433 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v104'48 uv48 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fae6804eb00 con 0x56073375b4d0 2024-09-21T03:41:11.544 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.542+0000 7fae7951bf40 1 -- 172.21.15.38:0/4179802055 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:57 13.1c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x56073375f230 con 0x56073375b4d0 2024-09-21T03:41:11.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.543+0000 7fae71cfa640 1 -- 172.21.15.38:0/4179802055 <== osd.0 v2:172.21.15.38:6800/2381639433 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v104'49 uv49 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fae6804eb00 con 0x56073375b4d0 2024-09-21T03:41:11.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.543+0000 7fae7951bf40 1 -- 172.21.15.38:0/4179802055 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:58 13.1c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x56073375e220 con 0x56073375b4d0 2024-09-21T03:41:11.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.544+0000 7fae71cfa640 1 -- 172.21.15.38:0/4179802055 <== osd.0 v2:172.21.15.38:6800/2381639433 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v104'50 uv50 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fae6804eb00 con 0x56073375b4d0 2024-09-21T03:41:11.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.544+0000 7fae7951bf40 1 --2- 172.21.15.38:0/4179802055 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x56073375e8a0 0x560733760cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:11.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.544+0000 7fae7951bf40 1 -- 172.21.15.38:0/4179802055 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:59 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x560733761340 con 0x56073375e8a0 2024-09-21T03:41:11.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.544+0000 7fae72cfc640 1 --2- 172.21.15.38:0/4179802055 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x56073375e8a0 0x560733760cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:11.547 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.545+0000 7fae72cfc640 1 --2- 172.21.15.38:0/4179802055 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x56073375e8a0 0x560733760cc0 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:11.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.546+0000 7fae72cfc640 1 -- 172.21.15.38:0/4179802055 <== osd.3 v2:172.21.15.38:6808/1286810294 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v104'36 uv35 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7fae6c024030 con 0x56073375e8a0 2024-09-21T03:41:11.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.546+0000 7fae7951bf40 1 -- 172.21.15.38:0/4179802055 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:60 13.6 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e104) -- 0x560733761850 con 0x56073375e8a0 2024-09-21T03:41:11.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.547+0000 7fae72cfc640 1 -- 172.21.15.38:0/4179802055 <== osd.3 v2:172.21.15.38:6808/1286810294 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v104'37 uv37 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fae6c024030 con 0x56073375e8a0 2024-09-21T03:41:11.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.547+0000 7fae7951bf40 1 -- 172.21.15.38:0/4179802055 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:61 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x560733761d10 con 0x56073375e8a0 2024-09-21T03:41:11.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.548+0000 7fae72cfc640 1 -- 172.21.15.38:0/4179802055 <== osd.3 v2:172.21.15.38:6808/1286810294 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v104'38 uv38 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fae6c024030 con 0x56073375e8a0 2024-09-21T03:41:11.550 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-09-21T03:41:11.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.548+0000 7fae7951bf40 1 -- 172.21.15.38:0/4179802055 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:62 13.6 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x560733762600 con 0x56073375e8a0 2024-09-21T03:41:11.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.549+0000 7fae72cfc640 1 -- 172.21.15.38:0/4179802055 <== osd.3 v2:172.21.15.38:6808/1286810294 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v104'39 uv39 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fae6c024030 con 0x56073375e8a0 2024-09-21T03:41:11.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.549+0000 7fae7951bf40 1 -- 172.21.15.38:0/4179802055 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x5607337629a0 con 0x56073375b4d0 2024-09-21T03:41:11.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.550+0000 7fae71cfa640 1 -- 172.21.15.38:0/4179802055 <== osd.0 v2:172.21.15.38:6800/2381639433 7 ==== osd_op_reply(63 608.00000000 [create] v104'45 uv44 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fae6804eb00 con 0x56073375b4d0 2024-09-21T03:41:11.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.550+0000 7fae7951bf40 1 -- 172.21.15.38:0/4179802055 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e104) -- 0x5607337629a0 con 0x56073375b4d0 2024-09-21T03:41:11.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.551+0000 7fae71cfa640 1 -- 172.21.15.38:0/4179802055 <== osd.0 v2:172.21.15.38:6800/2381639433 8 ==== osd_op_reply(64 608.00000000 [delete] v104'46 uv46 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fae6804eb00 con 0x56073375b4d0 2024-09-21T03:41:11.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.551+0000 7fae7951bf40 1 -- 172.21.15.38:0/4179802055 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x5607337629a0 con 0x56073375b4d0 2024-09-21T03:41:11.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.552+0000 7fae71cfa640 1 -- 172.21.15.38:0/4179802055 <== osd.0 v2:172.21.15.38:6800/2381639433 9 ==== osd_op_reply(65 608.00000000 [create] v104'47 uv47 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fae6804eb00 con 0x56073375b4d0 2024-09-21T03:41:11.554 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-09-21T03:41:11.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.552+0000 7fae7951bf40 1 -- 172.21.15.38:0/4179802055 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x5607337639b0 con 0x56073375b4d0 2024-09-21T03:41:11.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.554+0000 7fae71cfa640 1 -- 172.21.15.38:0/4179802055 <== osd.0 v2:172.21.15.38:6800/2381639433 10 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v104'48 uv48 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fae6804eb00 con 0x56073375b4d0 2024-09-21T03:41:11.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.554+0000 7fae7951bf40 1 -- 172.21.15.38:0/4179802055 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x5607337639b0 con 0x56073375b4d0 2024-09-21T03:41:11.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.555+0000 7fae71cfa640 1 -- 172.21.15.38:0/4179802055 <== osd.0 v2:172.21.15.38:6800/2381639433 11 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v104'49 uv49 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fae6804eb00 con 0x56073375b4d0 2024-09-21T03:41:11.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.555+0000 7fae7951bf40 1 -- 172.21.15.38:0/4179802055 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x5607337629a0 con 0x56073375b4d0 2024-09-21T03:41:11.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.556+0000 7fae71cfa640 1 -- 172.21.15.38:0/4179802055 <== osd.0 v2:172.21.15.38:6800/2381639433 12 ==== osd_op_reply(68 608.00000000 [omap-set-header] v104'50 uv50 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fae6804eb00 con 0x56073375b4d0 2024-09-21T03:41:11.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.556+0000 7fae7951bf40 1 -- 172.21.15.38:0/4179802055 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x5607337629a0 con 0x56073375e8a0 2024-09-21T03:41:11.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.557+0000 7fae72cfc640 1 -- 172.21.15.38:0/4179802055 <== osd.3 v2:172.21.15.38:6808/1286810294 5 ==== osd_op_reply(69 604.00000000 [create] v104'83 uv75 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fae6c024030 con 0x56073375e8a0 2024-09-21T03:41:11.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.557+0000 7fae7951bf40 1 -- 172.21.15.38:0/4179802055 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e104) -- 0x560733762d40 con 0x56073375e8a0 2024-09-21T03:41:11.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.558+0000 7fae72cfc640 1 -- 172.21.15.38:0/4179802055 <== osd.3 v2:172.21.15.38:6808/1286810294 6 ==== osd_op_reply(70 604.00000000 [delete] v104'84 uv84 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fae6c024030 con 0x56073375e8a0 2024-09-21T03:41:11.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.558+0000 7fae7951bf40 1 -- 172.21.15.38:0/4179802055 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x5607337630e0 con 0x56073375e8a0 2024-09-21T03:41:11.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.559+0000 7fae72cfc640 1 -- 172.21.15.38:0/4179802055 <== osd.3 v2:172.21.15.38:6808/1286810294 7 ==== osd_op_reply(71 604.00000000 [create] v104'85 uv85 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fae6c024030 con 0x56073375e8a0 2024-09-21T03:41:11.561 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-09-21T03:41:11.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.559+0000 7fae7951bf40 1 -- 172.21.15.38:0/4179802055 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x560733764490 con 0x56073375e8a0 2024-09-21T03:41:11.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.560+0000 7fae72cfc640 1 -- 172.21.15.38:0/4179802055 <== osd.3 v2:172.21.15.38:6808/1286810294 8 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v104'86 uv86 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fae6c024030 con 0x56073375e8a0 2024-09-21T03:41:11.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.560+0000 7fae7951bf40 1 -- 172.21.15.38:0/4179802055 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x560733764490 con 0x56073375e8a0 2024-09-21T03:41:11.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.561+0000 7fae72cfc640 1 -- 172.21.15.38:0/4179802055 <== osd.3 v2:172.21.15.38:6808/1286810294 9 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v104'87 uv87 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fae6c024030 con 0x56073375e8a0 2024-09-21T03:41:11.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.561+0000 7fae7951bf40 1 -- 172.21.15.38:0/4179802055 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x560733763480 con 0x56073375e8a0 2024-09-21T03:41:11.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.562+0000 7fae72cfc640 1 -- 172.21.15.38:0/4179802055 <== osd.3 v2:172.21.15.38:6808/1286810294 10 ==== osd_op_reply(74 604.00000000 [omap-set-header] v104'88 uv88 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fae6c024030 con 0x56073375e8a0 2024-09-21T03:41:11.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.562+0000 7fae7951bf40 1 -- 172.21.15.38:0/4179802055 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x560733763480 con 0x56073375e8a0 2024-09-21T03:41:11.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.563+0000 7fae72cfc640 1 -- 172.21.15.38:0/4179802055 <== osd.3 v2:172.21.15.38:6808/1286810294 11 ==== osd_op_reply(75 500.00000000 [create] v104'89 uv82 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fae6c024030 con 0x56073375e8a0 2024-09-21T03:41:11.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.563+0000 7fae7951bf40 1 -- 172.21.15.38:0/4179802055 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e104) -- 0x560733763480 con 0x56073375e8a0 2024-09-21T03:41:11.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.564+0000 7fae72cfc640 1 -- 172.21.15.38:0/4179802055 <== osd.3 v2:172.21.15.38:6808/1286810294 12 ==== osd_op_reply(76 500.00000000 [delete] v104'90 uv90 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fae6c024030 con 0x56073375e8a0 2024-09-21T03:41:11.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.564+0000 7fae7951bf40 1 -- 172.21.15.38:0/4179802055 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x560733763480 con 0x56073375e8a0 2024-09-21T03:41:11.567 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-09-21T03:41:11.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.565+0000 7fae72cfc640 1 -- 172.21.15.38:0/4179802055 <== osd.3 v2:172.21.15.38:6808/1286810294 13 ==== osd_op_reply(77 500.00000000 [create] v104'91 uv91 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fae6c024030 con 0x56073375e8a0 2024-09-21T03:41:11.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.565+0000 7fae7951bf40 1 -- 172.21.15.38:0/4179802055 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x560733763480 con 0x56073375e8a0 2024-09-21T03:41:11.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.566+0000 7fae72cfc640 1 -- 172.21.15.38:0/4179802055 <== osd.3 v2:172.21.15.38:6808/1286810294 14 ==== osd_op_reply(78 500.00000000 [write 0~90] v104'92 uv92 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fae6c024030 con 0x56073375e8a0 2024-09-21T03:41:11.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.567+0000 7fae7951bf40 1 -- 172.21.15.38:0/4179802055 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x560733763480 con 0x56073375e8a0 2024-09-21T03:41:11.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.568+0000 7fae72cfc640 1 -- 172.21.15.38:0/4179802055 <== osd.3 v2:172.21.15.38:6808/1286810294 15 ==== osd_op_reply(79 500.00000000 [omap-set-header] v104'93 uv93 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fae6c024030 con 0x56073375e8a0 2024-09-21T03:41:11.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.568+0000 7fae7951bf40 1 -- 172.21.15.38:0/4179802055 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:80 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x560733763480 con 0x56073375b4d0 2024-09-21T03:41:11.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.569+0000 7fae71cfa640 1 -- 172.21.15.38:0/4179802055 <== osd.0 v2:172.21.15.38:6800/2381639433 13 ==== osd_op_reply(80 mds_snaptable [create] v104'37 uv36 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7fae6804eb00 con 0x56073375b4d0 2024-09-21T03:41:11.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.569+0000 7fae7951bf40 1 -- 172.21.15.38:0/4179802055 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:81 13.d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e104) -- 0x560733763480 con 0x56073375b4d0 2024-09-21T03:41:11.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.570+0000 7fae71cfa640 1 -- 172.21.15.38:0/4179802055 <== osd.0 v2:172.21.15.38:6800/2381639433 14 ==== osd_op_reply(81 mds_snaptable [delete] v104'38 uv38 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fae6804eb00 con 0x56073375b4d0 2024-09-21T03:41:11.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.570+0000 7fae7951bf40 1 -- 172.21.15.38:0/4179802055 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:82 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x560733763480 con 0x56073375b4d0 2024-09-21T03:41:11.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.571+0000 7fae71cfa640 1 -- 172.21.15.38:0/4179802055 <== osd.0 v2:172.21.15.38:6800/2381639433 15 ==== osd_op_reply(82 mds_snaptable [create] v104'39 uv39 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fae6804eb00 con 0x56073375b4d0 2024-09-21T03:41:11.573 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-09-21T03:41:11.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.571+0000 7fae7951bf40 1 -- 172.21.15.38:0/4179802055 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:83 13.d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x560733763480 con 0x56073375b4d0 2024-09-21T03:41:11.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.572+0000 7fae71cfa640 1 -- 172.21.15.38:0/4179802055 <== osd.0 v2:172.21.15.38:6800/2381639433 16 ==== osd_op_reply(83 mds_snaptable [write 0~70] v104'40 uv40 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fae6804eb00 con 0x56073375b4d0 2024-09-21T03:41:11.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.572+0000 7fae7951bf40 1 -- 172.21.15.38:0/4179802055 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:84 13.d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x560733763480 con 0x56073375b4d0 2024-09-21T03:41:11.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.573+0000 7fae71cfa640 1 -- 172.21.15.38:0/4179802055 <== osd.0 v2:172.21.15.38:6800/2381639433 17 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v104'41 uv41 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fae6804eb00 con 0x56073375b4d0 2024-09-21T03:41:11.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.573+0000 7fae7951bf40 1 --2- 172.21.15.38:0/4179802055 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x560733763b00 0x560733765f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:11.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.573+0000 7fae7951bf40 1 -- 172.21.15.38:0/4179802055 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x5607337664f0 con 0x560733763b00 2024-09-21T03:41:11.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.573+0000 7fae724fb640 1 --2- 172.21.15.38:0/4179802055 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x560733763b00 0x560733765f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:11.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.574+0000 7fae724fb640 1 --2- 172.21.15.38:0/4179802055 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x560733763b00 0x560733765f20 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:11.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.575+0000 7fae724fb640 1 -- 172.21.15.38:0/4179802055 <== osd.1 v2:172.21.15.38:6816/1491818732 1 ==== osd_op_reply(85 605.00000000 [create] v104'45 uv44 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fae640137d0 con 0x560733763b00 2024-09-21T03:41:11.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.575+0000 7fae7951bf40 1 -- 172.21.15.38:0/4179802055 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e104) -- 0x5607337665b0 con 0x560733763b00 2024-09-21T03:41:11.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.576+0000 7fae724fb640 1 -- 172.21.15.38:0/4179802055 <== osd.1 v2:172.21.15.38:6816/1491818732 2 ==== osd_op_reply(86 605.00000000 [delete] v104'46 uv46 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fae640137d0 con 0x560733763b00 2024-09-21T03:41:11.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.576+0000 7fae7951bf40 1 -- 172.21.15.38:0/4179802055 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x5607337669d0 con 0x560733763b00 2024-09-21T03:41:11.579 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-09-21T03:41:11.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.577+0000 7fae724fb640 1 -- 172.21.15.38:0/4179802055 <== osd.1 v2:172.21.15.38:6816/1491818732 3 ==== osd_op_reply(87 605.00000000 [create] v104'47 uv47 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fae640137d0 con 0x560733763b00 2024-09-21T03:41:11.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.577+0000 7fae7951bf40 1 -- 172.21.15.38:0/4179802055 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x5607337679e0 con 0x560733763b00 2024-09-21T03:41:11.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.578+0000 7fae724fb640 1 -- 172.21.15.38:0/4179802055 <== osd.1 v2:172.21.15.38:6816/1491818732 4 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v104'48 uv48 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fae640137d0 con 0x560733763b00 2024-09-21T03:41:11.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.578+0000 7fae7951bf40 1 -- 172.21.15.38:0/4179802055 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x5607337679e0 con 0x560733763b00 2024-09-21T03:41:11.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.579+0000 7fae724fb640 1 -- 172.21.15.38:0/4179802055 <== osd.1 v2:172.21.15.38:6816/1491818732 5 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v104'49 uv49 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fae640137d0 con 0x560733763b00 2024-09-21T03:41:11.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.579+0000 7fae7951bf40 1 -- 172.21.15.38:0/4179802055 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x5607337669d0 con 0x560733763b00 2024-09-21T03:41:11.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.581+0000 7fae724fb640 1 -- 172.21.15.38:0/4179802055 <== osd.1 v2:172.21.15.38:6816/1491818732 6 ==== osd_op_reply(90 605.00000000 [omap-set-header] v104'50 uv50 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fae640137d0 con 0x560733763b00 2024-09-21T03:41:11.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.581+0000 7fae7951bf40 1 -- 172.21.15.38:0/4179802055 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x5607337669d0 con 0x560733763b00 2024-09-21T03:41:11.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.582+0000 7fae724fb640 1 -- 172.21.15.38:0/4179802055 <== osd.1 v2:172.21.15.38:6816/1491818732 7 ==== osd_op_reply(91 mds0_inotable [create] v104'74 uv68 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7fae640137d0 con 0x560733763b00 2024-09-21T03:41:11.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.582+0000 7fae7951bf40 1 -- 172.21.15.38:0/4179802055 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e104) -- 0x5607337669d0 con 0x560733763b00 2024-09-21T03:41:11.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.583+0000 7fae724fb640 1 -- 172.21.15.38:0/4179802055 <== osd.1 v2:172.21.15.38:6816/1491818732 8 ==== osd_op_reply(92 mds0_inotable [delete] v104'75 uv75 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fae640137d0 con 0x560733763b00 2024-09-21T03:41:11.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.583+0000 7fae7951bf40 1 -- 172.21.15.38:0/4179802055 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x5607337669d0 con 0x560733763b00 2024-09-21T03:41:11.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.584+0000 7fae724fb640 1 -- 172.21.15.38:0/4179802055 <== osd.1 v2:172.21.15.38:6816/1491818732 9 ==== osd_op_reply(93 mds0_inotable [create] v104'76 uv76 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fae640137d0 con 0x560733763b00 2024-09-21T03:41:11.586 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-09-21T03:41:11.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.584+0000 7fae7951bf40 1 -- 172.21.15.38:0/4179802055 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:94 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x5607337669d0 con 0x560733763b00 2024-09-21T03:41:11.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.585+0000 7fae724fb640 1 -- 172.21.15.38:0/4179802055 <== osd.1 v2:172.21.15.38:6816/1491818732 10 ==== osd_op_reply(94 mds0_inotable [write 0~34] v104'77 uv77 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fae640137d0 con 0x560733763b00 2024-09-21T03:41:11.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.585+0000 7fae7951bf40 1 -- 172.21.15.38:0/4179802055 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:95 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x5607337669d0 con 0x560733763b00 2024-09-21T03:41:11.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.586+0000 7fae724fb640 1 -- 172.21.15.38:0/4179802055 <== osd.1 v2:172.21.15.38:6816/1491818732 11 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v104'78 uv78 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fae640137d0 con 0x560733763b00 2024-09-21T03:41:11.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.586+0000 7fae7951bf40 1 -- 172.21.15.38:0/4179802055 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:96 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x5607337669d0 con 0x560733763b00 2024-09-21T03:41:11.589 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.587+0000 7fae724fb640 1 -- 172.21.15.38:0/4179802055 <== osd.1 v2:172.21.15.38:6816/1491818732 12 ==== osd_op_reply(96 100.00000000 [create] v104'79 uv73 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fae640137d0 con 0x560733763b00 2024-09-21T03:41:11.589 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.587+0000 7fae7951bf40 1 -- 172.21.15.38:0/4179802055 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:97 13.13 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e104) -- 0x5607337669d0 con 0x560733763b00 2024-09-21T03:41:11.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.588+0000 7fae724fb640 1 -- 172.21.15.38:0/4179802055 <== osd.1 v2:172.21.15.38:6816/1491818732 13 ==== osd_op_reply(97 100.00000000 [delete] v104'80 uv80 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fae640137d0 con 0x560733763b00 2024-09-21T03:41:11.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.588+0000 7fae7951bf40 1 -- 172.21.15.38:0/4179802055 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:98 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x5607337669d0 con 0x560733763b00 2024-09-21T03:41:11.591 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-09-21T03:41:11.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.589+0000 7fae724fb640 1 -- 172.21.15.38:0/4179802055 <== osd.1 v2:172.21.15.38:6816/1491818732 14 ==== osd_op_reply(98 100.00000000 [create] v104'81 uv81 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fae640137d0 con 0x560733763b00 2024-09-21T03:41:11.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.589+0000 7fae7951bf40 1 -- 172.21.15.38:0/4179802055 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:99 13.13 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x5607337669d0 con 0x560733763b00 2024-09-21T03:41:11.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.590+0000 7fae724fb640 1 -- 172.21.15.38:0/4179802055 <== osd.1 v2:172.21.15.38:6816/1491818732 15 ==== osd_op_reply(99 100.00000000 [omap-set-header] v104'82 uv82 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fae640137d0 con 0x560733763b00 2024-09-21T03:41:11.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.590+0000 7fae7951bf40 1 -- 172.21.15.38:0/4179802055 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:100 13.13 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x5607337697e0 con 0x560733763b00 2024-09-21T03:41:11.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.591+0000 7fae724fb640 1 -- 172.21.15.38:0/4179802055 <== osd.1 v2:172.21.15.38:6816/1491818732 16 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v104'83 uv83 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fae640137d0 con 0x560733763b00 2024-09-21T03:41:11.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.591+0000 7fae7951bf40 1 -- 172.21.15.38:0/4179802055 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x560733766e80 con 0x56073375e8a0 2024-09-21T03:41:11.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.592+0000 7fae72cfc640 1 -- 172.21.15.38:0/4179802055 <== osd.3 v2:172.21.15.38:6808/1286810294 16 ==== osd_op_reply(101 mds0_sessionmap [create] v104'36 uv34 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7fae6c024030 con 0x56073375e8a0 2024-09-21T03:41:11.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.592+0000 7fae7951bf40 1 -- 172.21.15.38:0/4179802055 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e104) -- 0x560733766e80 con 0x56073375e8a0 2024-09-21T03:41:11.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.594+0000 7fae72cfc640 1 -- 172.21.15.38:0/4179802055 <== osd.3 v2:172.21.15.38:6808/1286810294 17 ==== osd_op_reply(102 mds0_sessionmap [delete] v104'37 uv37 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fae6c024030 con 0x56073375e8a0 2024-09-21T03:41:11.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.594+0000 7fae7951bf40 1 -- 172.21.15.38:0/4179802055 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:103 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x560733766e80 con 0x56073375e8a0 2024-09-21T03:41:11.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.595+0000 7fae72cfc640 1 -- 172.21.15.38:0/4179802055 <== osd.3 v2:172.21.15.38:6808/1286810294 18 ==== osd_op_reply(103 mds0_sessionmap [create] v104'38 uv38 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fae6c024030 con 0x56073375e8a0 2024-09-21T03:41:11.597 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-09-21T03:41:11.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.595+0000 7fae7951bf40 1 -- 172.21.15.38:0/4179802055 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:104 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x560733766e80 con 0x56073375e8a0 2024-09-21T03:41:11.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.596+0000 7fae72cfc640 1 -- 172.21.15.38:0/4179802055 <== osd.3 v2:172.21.15.38:6808/1286810294 19 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v104'39 uv39 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fae6c024030 con 0x56073375e8a0 2024-09-21T03:41:11.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.596+0000 7fae7951bf40 1 -- 172.21.15.38:0/4179802055 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x560733766e80 con 0x56073375e8a0 2024-09-21T03:41:11.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.597+0000 7fae72cfc640 1 -- 172.21.15.38:0/4179802055 <== osd.3 v2:172.21.15.38:6808/1286810294 20 ==== osd_op_reply(105 609.00000000 [create] v104'45 uv44 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fae6c024030 con 0x56073375e8a0 2024-09-21T03:41:11.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.597+0000 7fae7951bf40 1 -- 172.21.15.38:0/4179802055 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e104) -- 0x560733766e80 con 0x56073375e8a0 2024-09-21T03:41:11.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.598+0000 7fae72cfc640 1 -- 172.21.15.38:0/4179802055 <== osd.3 v2:172.21.15.38:6808/1286810294 21 ==== osd_op_reply(106 609.00000000 [delete] v104'46 uv46 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fae6c024030 con 0x56073375e8a0 2024-09-21T03:41:11.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.598+0000 7fae7951bf40 1 -- 172.21.15.38:0/4179802055 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x560733766e80 con 0x56073375e8a0 2024-09-21T03:41:11.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.599+0000 7fae72cfc640 1 -- 172.21.15.38:0/4179802055 <== osd.3 v2:172.21.15.38:6808/1286810294 22 ==== osd_op_reply(107 609.00000000 [create] v104'47 uv47 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fae6c024030 con 0x56073375e8a0 2024-09-21T03:41:11.601 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-09-21T03:41:11.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.599+0000 7fae7951bf40 1 -- 172.21.15.38:0/4179802055 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x560733767e90 con 0x56073375e8a0 2024-09-21T03:41:11.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.600+0000 7fae72cfc640 1 -- 172.21.15.38:0/4179802055 <== osd.3 v2:172.21.15.38:6808/1286810294 23 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v104'48 uv48 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fae6c024030 con 0x56073375e8a0 2024-09-21T03:41:11.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.600+0000 7fae7951bf40 1 -- 172.21.15.38:0/4179802055 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:109 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x560733767e90 con 0x56073375e8a0 2024-09-21T03:41:11.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.601+0000 7fae72cfc640 1 -- 172.21.15.38:0/4179802055 <== osd.3 v2:172.21.15.38:6808/1286810294 24 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v104'49 uv49 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fae6c024030 con 0x56073375e8a0 2024-09-21T03:41:11.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.601+0000 7fae7951bf40 1 -- 172.21.15.38:0/4179802055 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:110 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x560733766e80 con 0x56073375e8a0 2024-09-21T03:41:11.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.602+0000 7fae72cfc640 1 -- 172.21.15.38:0/4179802055 <== osd.3 v2:172.21.15.38:6808/1286810294 25 ==== osd_op_reply(110 609.00000000 [omap-set-header] v104'50 uv50 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fae6c024030 con 0x56073375e8a0 2024-09-21T03:41:11.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.602+0000 7fae7951bf40 1 --2- 172.21.15.38:0/4179802055 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x560733767500 0x560733769e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:11.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.603+0000 7fae7951bf40 1 -- 172.21.15.38:0/4179802055 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x560733769920 con 0x560733767500 2024-09-21T03:41:11.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.603+0000 7fae71cfa640 1 --2- 172.21.15.38:0/4179802055 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x560733767500 0x560733769e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:11.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.603+0000 7fae71cfa640 1 --2- 172.21.15.38:0/4179802055 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x560733767500 0x560733769e20 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:11.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.604+0000 7fae71cfa640 1 -- 172.21.15.38:0/4179802055 <== osd.2 v2:172.21.15.38:6824/2577858902 1 ==== osd_op_reply(111 400.00000000 [create] v104'37 uv36 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fae6804eb00 con 0x560733767500 2024-09-21T03:41:11.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.604+0000 7fae7951bf40 1 -- 172.21.15.38:0/4179802055 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e104) -- 0x5607337669d0 con 0x560733767500 2024-09-21T03:41:11.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.605+0000 7fae71cfa640 1 -- 172.21.15.38:0/4179802055 <== osd.2 v2:172.21.15.38:6824/2577858902 2 ==== osd_op_reply(112 400.00000000 [delete] v104'38 uv38 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fae6804eb00 con 0x560733767500 2024-09-21T03:41:11.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.605+0000 7fae7951bf40 1 -- 172.21.15.38:0/4179802055 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x5607337669d0 con 0x560733767500 2024-09-21T03:41:11.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.606+0000 7fae71cfa640 1 -- 172.21.15.38:0/4179802055 <== osd.2 v2:172.21.15.38:6824/2577858902 3 ==== osd_op_reply(113 400.00000000 [create] v104'39 uv39 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fae6804eb00 con 0x560733767500 2024-09-21T03:41:11.608 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-09-21T03:41:11.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.606+0000 7fae7951bf40 1 -- 172.21.15.38:0/4179802055 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:114 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x56073376a360 con 0x560733767500 2024-09-21T03:41:11.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.608+0000 7fae71cfa640 1 -- 172.21.15.38:0/4179802055 <== osd.2 v2:172.21.15.38:6824/2577858902 4 ==== osd_op_reply(114 400.00000000 [write 0~22] v104'40 uv40 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fae6804eb00 con 0x560733767500 2024-09-21T03:41:11.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.608+0000 7fae7951bf40 1 -- 172.21.15.38:0/4179802055 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:115 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x56073376a360 con 0x560733767500 2024-09-21T03:41:11.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.609+0000 7fae71cfa640 1 -- 172.21.15.38:0/4179802055 <== osd.2 v2:172.21.15.38:6824/2577858902 5 ==== osd_op_reply(115 400.00000000 [omap-set-header] v104'41 uv41 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fae6804eb00 con 0x560733767500 2024-09-21T03:41:11.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.609+0000 7fae7951bf40 1 -- 172.21.15.38:0/4179802055 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:116 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x56073376a360 con 0x560733763b00 2024-09-21T03:41:11.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.610+0000 7fae724fb640 1 -- 172.21.15.38:0/4179802055 <== osd.1 v2:172.21.15.38:6816/1491818732 17 ==== osd_op_reply(116 100.00000000.inode [create] v104'95 uv86 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7fae640137d0 con 0x560733763b00 2024-09-21T03:41:11.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.610+0000 7fae7951bf40 1 -- 172.21.15.38:0/4179802055 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:117 13.1f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e104) -- 0x56073376a360 con 0x560733763b00 2024-09-21T03:41:11.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.611+0000 7fae724fb640 1 -- 172.21.15.38:0/4179802055 <== osd.1 v2:172.21.15.38:6816/1491818732 18 ==== osd_op_reply(117 100.00000000.inode [delete] v104'96 uv96 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fae640137d0 con 0x560733763b00 2024-09-21T03:41:11.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.611+0000 7fae7951bf40 1 -- 172.21.15.38:0/4179802055 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:118 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x56073376a360 con 0x560733763b00 2024-09-21T03:41:11.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.613+0000 7fae724fb640 1 -- 172.21.15.38:0/4179802055 <== osd.1 v2:172.21.15.38:6816/1491818732 19 ==== osd_op_reply(118 100.00000000.inode [create] v104'97 uv97 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fae640137d0 con 0x560733763b00 2024-09-21T03:41:11.615 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-09-21T03:41:11.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.613+0000 7fae7951bf40 1 -- 172.21.15.38:0/4179802055 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:119 13.1f 13:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x56073376a360 con 0x560733763b00 2024-09-21T03:41:11.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.614+0000 7fae724fb640 1 -- 172.21.15.38:0/4179802055 <== osd.1 v2:172.21.15.38:6816/1491818732 20 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v104'98 uv98 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fae640137d0 con 0x560733763b00 2024-09-21T03:41:11.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.614+0000 7fae7951bf40 1 -- 172.21.15.38:0/4179802055 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:120 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x56073376a360 con 0x560733763b00 2024-09-21T03:41:11.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.615+0000 7fae724fb640 1 -- 172.21.15.38:0/4179802055 <== osd.1 v2:172.21.15.38:6816/1491818732 21 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v104'99 uv99 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fae640137d0 con 0x560733763b00 2024-09-21T03:41:11.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.615+0000 7fae7951bf40 1 -- 172.21.15.38:0/4179802055 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:121 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x56073376a360 con 0x560733763b00 2024-09-21T03:41:11.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.616+0000 7fae724fb640 1 -- 172.21.15.38:0/4179802055 <== osd.1 v2:172.21.15.38:6816/1491818732 22 ==== osd_op_reply(121 1.00000000 [create] v104'100 uv93 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7fae640137d0 con 0x560733763b00 2024-09-21T03:41:11.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.616+0000 7fae7951bf40 1 -- 172.21.15.38:0/4179802055 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:122 13.1f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e104) -- 0x56073376a360 con 0x560733763b00 2024-09-21T03:41:11.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.618+0000 7fae724fb640 1 -- 172.21.15.38:0/4179802055 <== osd.1 v2:172.21.15.38:6816/1491818732 23 ==== osd_op_reply(122 1.00000000 [delete] v104'101 uv101 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fae640137d0 con 0x560733763b00 2024-09-21T03:41:11.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.618+0000 7fae7951bf40 1 -- 172.21.15.38:0/4179802055 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:123 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x56073376a360 con 0x560733763b00 2024-09-21T03:41:11.620 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-09-21T03:41:11.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.619+0000 7fae724fb640 1 -- 172.21.15.38:0/4179802055 <== osd.1 v2:172.21.15.38:6816/1491818732 24 ==== osd_op_reply(123 1.00000000 [create] v104'102 uv102 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fae640137d0 con 0x560733763b00 2024-09-21T03:41:11.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.619+0000 7fae7951bf40 1 -- 172.21.15.38:0/4179802055 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:124 13.1f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x56073376b370 con 0x560733763b00 2024-09-21T03:41:11.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.620+0000 7fae724fb640 1 -- 172.21.15.38:0/4179802055 <== osd.1 v2:172.21.15.38:6816/1491818732 25 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v104'103 uv103 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fae640137d0 con 0x560733763b00 2024-09-21T03:41:11.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.620+0000 7fae7951bf40 1 -- 172.21.15.38:0/4179802055 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:125 13.1f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x56073376b370 con 0x560733763b00 2024-09-21T03:41:11.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.621+0000 7fae724fb640 1 -- 172.21.15.38:0/4179802055 <== osd.1 v2:172.21.15.38:6816/1491818732 26 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v104'104 uv104 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fae640137d0 con 0x560733763b00 2024-09-21T03:41:11.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.621+0000 7fae7951bf40 1 -- 172.21.15.38:0/4179802055 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:126 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x56073376a360 con 0x560733763b00 2024-09-21T03:41:11.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.622+0000 7fae724fb640 1 -- 172.21.15.38:0/4179802055 <== osd.1 v2:172.21.15.38:6816/1491818732 27 ==== osd_op_reply(126 1.00000000 [omap-set-header] v104'105 uv105 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fae640137d0 con 0x560733763b00 2024-09-21T03:41:11.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.622+0000 7fae7951bf40 1 -- 172.21.15.38:0/4179802055 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:127 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=584b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x56073376a810 con 0x560733763b00 2024-09-21T03:41:11.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.623+0000 7fae724fb640 1 -- 172.21.15.38:0/4179802055 <== osd.1 v2:172.21.15.38:6816/1491818732 28 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v104'106 uv106 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fae640137d0 con 0x560733763b00 2024-09-21T03:41:11.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.623+0000 7fae7951bf40 1 -- 172.21.15.38:0/4179802055 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x56073375b4d0 msgr2=0x56073375d8f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:11.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.623+0000 7fae7951bf40 1 --2- 172.21.15.38:0/4179802055 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x56073375b4d0 0x56073375d8f0 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:11.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.623+0000 7fae7951bf40 1 -- 172.21.15.38:0/4179802055 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x560733763b00 msgr2=0x560733765f20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:11.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.623+0000 7fae7951bf40 1 --2- 172.21.15.38:0/4179802055 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x560733763b00 0x560733765f20 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:11.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.623+0000 7fae7951bf40 1 -- 172.21.15.38:0/4179802055 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x560733767500 msgr2=0x560733769e20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:11.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.623+0000 7fae7951bf40 1 --2- 172.21.15.38:0/4179802055 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x560733767500 0x560733769e20 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:11.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.623+0000 7fae7951bf40 1 -- 172.21.15.38:0/4179802055 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x56073375e8a0 msgr2=0x560733760cc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:11.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.623+0000 7fae7951bf40 1 --2- 172.21.15.38:0/4179802055 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x56073375e8a0 0x560733760cc0 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:11.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.623+0000 7fae72cfc640 1 -- 172.21.15.38:0/4179802055 reap_dead start 2024-09-21T03:41:11.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.623+0000 7fae7951bf40 1 -- 172.21.15.38:0/4179802055 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x560733756bf0 msgr2=0x560733759010 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:11.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.623+0000 7fae7951bf40 1 --2- 172.21.15.38:0/4179802055 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x560733756bf0 0x560733759010 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:11.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.623+0000 7fae72cfc640 1 -- 172.21.15.38:0/4179802055 reap_dead start 2024-09-21T03:41:11.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.623+0000 7fae7951bf40 1 -- 172.21.15.38:0/4179802055 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x56073361ec10 msgr2=0x56073361f020 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:11.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.623+0000 7fae7951bf40 1 --2- 172.21.15.38:0/4179802055 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x56073361ec10 0x56073361f020 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:11.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.623+0000 7fae7951bf40 1 -- 172.21.15.38:0/4179802055 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fae54077570 msgr2=0x7fae54079a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:11.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.623+0000 7fae7951bf40 1 --2- 172.21.15.38:0/4179802055 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fae54077570 0x7fae54079a80 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fae6400d5a0 tx=0x7fae64007040 comp rx=0 tx=0).stop 2024-09-21T03:41:11.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.623+0000 7fae7951bf40 1 -- 172.21.15.38:0/4179802055 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560733696950 msgr2=0x5607336b1a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:11.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.623+0000 7fae7951bf40 1 --2- 172.21.15.38:0/4179802055 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560733696950 0x5607336b1a80 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x5607336091e0 tx=0x7fae6c001ac0 comp rx=0 tx=0).stop 2024-09-21T03:41:11.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.624+0000 7fae7951bf40 1 -- 172.21.15.38:0/4179802055 shutdown_connections 2024-09-21T03:41:11.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.624+0000 7fae7951bf40 1 --2- 172.21.15.38:0/4179802055 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x56073361ec10 0x56073361f020 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:11.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.624+0000 7fae7951bf40 1 --2- 172.21.15.38:0/4179802055 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560733696950 0x5607336b1a80 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:11.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.624+0000 7fae7951bf40 1 --2- 172.21.15.38:0/4179802055 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fae54077570 0x7fae54079a80 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:11.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.624+0000 7fae7951bf40 1 -- 172.21.15.38:0/4179802055 >> 172.21.15.38:0/4179802055 conn(0x5607334faac0 msgr2=0x5607336ac660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:11.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.624+0000 7fae7951bf40 1 -- 172.21.15.38:0/4179802055 shutdown_connections 2024-09-21T03:41:11.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.624+0000 7fae7951bf40 1 -- 172.21.15.38:0/4179802055 wait complete. 2024-09-21T03:41:11.629 DEBUG:teuthology.orchestra.run.smithi038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rm mds_snaptable 2024-09-21T03:41:11.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.722+0000 7fa6fa07ff40 1 Processor -- start 2024-09-21T03:41:11.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.722+0000 7fa6fa07ff40 1 -- start start 2024-09-21T03:41:11.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.723+0000 7fa6fa07ff40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55d2ed969240 0x55d2eda23b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:11.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.723+0000 7fa6fa07ff40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d2ed9b0090 0x55d2eda36850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:11.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.723+0000 7fa6fa07ff40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d2eda25520 0x55d2eda36d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:11.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.723+0000 7fa6fa07ff40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55d2ed8b6d60 con 0x55d2eda25520 2024-09-21T03:41:11.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.723+0000 7fa6fa07ff40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55d2ed915430 con 0x55d2ed9b0090 2024-09-21T03:41:11.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.723+0000 7fa6fa07ff40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55d2eda3caa0 con 0x55d2ed969240 2024-09-21T03:41:11.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.723+0000 7fa6f433a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d2eda25520 0x55d2eda36d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:11.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.723+0000 7fa6f433a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d2eda25520 0x55d2eda36d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50618/0 (socket says 172.21.15.38:50618) 2024-09-21T03:41:11.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.723+0000 7fa6f433a640 1 -- 172.21.15.38:0/2837301358 learned_addr learned my addr 172.21.15.38:0/2837301358 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:41:11.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.723+0000 7fa6f3338640 1 --2- 172.21.15.38:0/2837301358 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d2ed9b0090 0x55d2eda36850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:11.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.723+0000 7fa6f3b39640 1 --2- 172.21.15.38:0/2837301358 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55d2ed969240 0x55d2eda23b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:11.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.724+0000 7fa6f433a640 1 -- 172.21.15.38:0/2837301358 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55d2ed969240 msgr2=0x55d2eda23b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:11.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.724+0000 7fa6f433a640 1 --2- 172.21.15.38:0/2837301358 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55d2ed969240 0x55d2eda23b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:11.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.724+0000 7fa6f433a640 1 -- 172.21.15.38:0/2837301358 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d2ed9b0090 msgr2=0x55d2eda36850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:11.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.724+0000 7fa6f433a640 1 --2- 172.21.15.38:0/2837301358 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d2ed9b0090 0x55d2eda36850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:11.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.724+0000 7fa6f433a640 1 -- 172.21.15.38:0/2837301358 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2eda24500 con 0x55d2eda25520 2024-09-21T03:41:11.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.724+0000 7fa6f433a640 1 --2- 172.21.15.38:0/2837301358 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d2eda25520 0x55d2eda36d90 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec001d40 tx=0x7fa6ec00b590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdbda0b5021076d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:11.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.724+0000 7fa6f2336640 1 -- 172.21.15.38:0/2837301358 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6ec016070 con 0x55d2eda25520 2024-09-21T03:41:11.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.724+0000 7fa6f2336640 1 -- 172.21.15.38:0/2837301358 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa6ec00f6b0 con 0x55d2eda25520 2024-09-21T03:41:11.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.724+0000 7fa6f2336640 1 -- 172.21.15.38:0/2837301358 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6ec00fb40 con 0x55d2eda25520 2024-09-21T03:41:11.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.725+0000 7fa6fa07ff40 1 -- 172.21.15.38:0/2837301358 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d2eda25520 msgr2=0x55d2eda36d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:11.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.725+0000 7fa6fa07ff40 1 --2- 172.21.15.38:0/2837301358 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d2eda25520 0x55d2eda36d90 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec001d40 tx=0x7fa6ec00b590 comp rx=0 tx=0).stop 2024-09-21T03:41:11.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.725+0000 7fa6fa07ff40 1 -- 172.21.15.38:0/2837301358 shutdown_connections 2024-09-21T03:41:11.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.725+0000 7fa6fa07ff40 1 --2- 172.21.15.38:0/2837301358 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d2eda25520 0x55d2eda36d90 unknown :-1 s=CLOSED pgs=1423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:11.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.725+0000 7fa6fa07ff40 1 --2- 172.21.15.38:0/2837301358 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d2ed9b0090 0x55d2eda36850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:11.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.725+0000 7fa6fa07ff40 1 --2- 172.21.15.38:0/2837301358 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55d2ed969240 0x55d2eda23b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:11.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.725+0000 7fa6fa07ff40 1 -- 172.21.15.38:0/2837301358 >> 172.21.15.38:0/2837301358 conn(0x55d2ed887ac0 msgr2=0x55d2ed967cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:11.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.725+0000 7fa6fa07ff40 1 -- 172.21.15.38:0/2837301358 shutdown_connections 2024-09-21T03:41:11.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.725+0000 7fa6fa07ff40 1 -- 172.21.15.38:0/2837301358 wait complete. 2024-09-21T03:41:11.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.726+0000 7fa6fa07ff40 1 Processor -- start 2024-09-21T03:41:11.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.726+0000 7fa6fa07ff40 1 -- start start 2024-09-21T03:41:11.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.727+0000 7fa6fa07ff40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55d2ed969240 0x55d2eda35570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:11.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.727+0000 7fa6fa07ff40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d2ed9b0090 0x55d2edac9e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:11.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.727+0000 7fa6fa07ff40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d2eda25520 0x55d2edaca3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:11.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.727+0000 7fa6fa07ff40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55d2ed992fa0 con 0x55d2eda25520 2024-09-21T03:41:11.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.727+0000 7fa6fa07ff40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55d2eda22260 con 0x55d2ed9b0090 2024-09-21T03:41:11.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.727+0000 7fa6fa07ff40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55d2ed992570 con 0x55d2ed969240 2024-09-21T03:41:11.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.727+0000 7fa6f433a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d2eda25520 0x55d2edaca3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:11.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.727+0000 7fa6f433a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d2eda25520 0x55d2edaca3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50634/0 (socket says 172.21.15.38:50634) 2024-09-21T03:41:11.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.727+0000 7fa6f433a640 1 -- 172.21.15.38:0/3144639276 learned_addr learned my addr 172.21.15.38:0/3144639276 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:41:11.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.727+0000 7fa6f3b39640 1 --2- 172.21.15.38:0/3144639276 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55d2ed969240 0x55d2eda35570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:11.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.727+0000 7fa6f3338640 1 --2- 172.21.15.38:0/3144639276 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d2ed9b0090 0x55d2edac9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:11.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.727+0000 7fa6f433a640 1 -- 172.21.15.38:0/3144639276 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55d2ed969240 msgr2=0x55d2eda35570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:11.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.727+0000 7fa6f433a640 1 --2- 172.21.15.38:0/3144639276 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55d2ed969240 0x55d2eda35570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:11.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.727+0000 7fa6f433a640 1 -- 172.21.15.38:0/3144639276 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d2ed9b0090 msgr2=0x55d2edac9e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:11.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.728+0000 7fa6f433a640 1 --2- 172.21.15.38:0/3144639276 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d2ed9b0090 0x55d2edac9e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:11.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.728+0000 7fa6f433a640 1 -- 172.21.15.38:0/3144639276 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2ed994bd0 con 0x55d2eda25520 2024-09-21T03:41:11.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.728+0000 7fa6f3b39640 1 --2- 172.21.15.38:0/3144639276 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55d2ed969240 0x55d2eda35570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:41:11.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.728+0000 7fa6f3338640 1 --2- 172.21.15.38:0/3144639276 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d2ed9b0090 0x55d2edac9e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:41:11.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.728+0000 7fa6f433a640 1 --2- 172.21.15.38:0/3144639276 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d2eda25520 0x55d2edaca3a0 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec00f000 tx=0x7fa6ec010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:11.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.728+0000 7fa6f1334640 1 -- 172.21.15.38:0/3144639276 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6ec016030 con 0x55d2eda25520 2024-09-21T03:41:11.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.728+0000 7fa6f1334640 1 -- 172.21.15.38:0/3144639276 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa6ec021070 con 0x55d2eda25520 2024-09-21T03:41:11.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.728+0000 7fa6f1334640 1 -- 172.21.15.38:0/3144639276 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6ec0109f0 con 0x55d2eda25520 2024-09-21T03:41:11.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.728+0000 7fa6fa07ff40 1 -- 172.21.15.38:0/3144639276 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55d2ed994eb0 con 0x55d2eda25520 2024-09-21T03:41:11.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.728+0000 7fa6fa07ff40 1 -- 172.21.15.38:0/3144639276 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2eda21fa0 con 0x55d2eda25520 2024-09-21T03:41:11.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.730+0000 7fa6f1334640 1 -- 172.21.15.38:0/3144639276 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa6ec010b90 con 0x55d2eda25520 2024-09-21T03:41:11.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.730+0000 7fa6f1334640 1 --2- 172.21.15.38:0/3144639276 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa6d40775a0 0x7fa6d4079ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:11.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.730+0000 7fa6f1334640 1 -- 172.21.15.38:0/3144639276 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 6992+0+0 (secure 0 0 0) 0x7fa6ec09b230 con 0x55d2eda25520 2024-09-21T03:41:11.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.730+0000 7fa6f3b39640 1 --2- 172.21.15.38:0/3144639276 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa6d40775a0 0x7fa6d4079ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:11.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.730+0000 7fa6fa07ff40 1 --2- 172.21.15.38:0/3144639276 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x55d2eda21740 0x55d2edac6680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:11.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.730+0000 7fa6fa07ff40 1 -- 172.21.15.38:0/3144639276 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:1 13.d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e104) -- 0x55d2ed995990 con 0x55d2eda21740 2024-09-21T03:41:11.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.730+0000 7fa6f3338640 1 --2- 172.21.15.38:0/3144639276 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x55d2eda21740 0x55d2edac6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:11.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.731+0000 7fa6f3b39640 1 --2- 172.21.15.38:0/3144639276 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa6d40775a0 0x7fa6d4079ab0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fa6e400f940 tx=0x7fa6e4004c40 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:11.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.731+0000 7fa6f3338640 1 --2- 172.21.15.38:0/3144639276 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x55d2eda21740 0x55d2edac6680 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:11.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.732+0000 7fa6f3338640 1 -- 172.21.15.38:0/3144639276 <== osd.0 v2:172.21.15.38:6800/2381639433 1 ==== osd_op_reply(1 mds_snaptable [delete] v104'42 uv42 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fa6dc04edb0 con 0x55d2eda21740 2024-09-21T03:41:11.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.732+0000 7fa6fa07ff40 1 -- 172.21.15.38:0/3144639276 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x55d2eda21740 msgr2=0x55d2edac6680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:11.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.732+0000 7fa6fa07ff40 1 --2- 172.21.15.38:0/3144639276 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x55d2eda21740 0x55d2edac6680 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:11.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.733+0000 7fa6fa07ff40 1 -- 172.21.15.38:0/3144639276 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa6d40775a0 msgr2=0x7fa6d4079ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:11.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.733+0000 7fa6fa07ff40 1 --2- 172.21.15.38:0/3144639276 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa6d40775a0 0x7fa6d4079ab0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fa6e400f940 tx=0x7fa6e4004c40 comp rx=0 tx=0).stop 2024-09-21T03:41:11.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.733+0000 7fa6fa07ff40 1 -- 172.21.15.38:0/3144639276 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d2eda25520 msgr2=0x55d2edaca3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:11.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.733+0000 7fa6fa07ff40 1 --2- 172.21.15.38:0/3144639276 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d2eda25520 0x55d2edaca3a0 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec00f000 tx=0x7fa6ec010000 comp rx=0 tx=0).stop 2024-09-21T03:41:11.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.733+0000 7fa6f433a640 1 -- 172.21.15.38:0/3144639276 reap_dead start 2024-09-21T03:41:11.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.733+0000 7fa6fa07ff40 1 -- 172.21.15.38:0/3144639276 shutdown_connections 2024-09-21T03:41:11.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.733+0000 7fa6fa07ff40 1 -- 172.21.15.38:0/3144639276 >> 172.21.15.38:0/3144639276 conn(0x55d2ed887ac0 msgr2=0x55d2ed967cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:11.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.733+0000 7fa6fa07ff40 1 -- 172.21.15.38:0/3144639276 shutdown_connections 2024-09-21T03:41:11.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.733+0000 7fa6fa07ff40 1 -- 172.21.15.38:0/3144639276 wait complete. 2024-09-21T03:41:11.738 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-21T03:41:11.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.855+0000 7fb0dade4640 1 -- 172.21.15.38:0/1141946149 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb0d40d1120 msgr2=0x7fb0d40ceb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:11.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.855+0000 7fb0dade4640 1 --2- 172.21.15.38:0/1141946149 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb0d40d1120 0x7fb0d40ceb90 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7fb0d40d2360 tx=0x7fb0cc02a4d0 comp rx=0 tx=0).stop 2024-09-21T03:41:11.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.855+0000 7fb0dade4640 1 -- 172.21.15.38:0/1141946149 shutdown_connections 2024-09-21T03:41:11.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.855+0000 7fb0dade4640 1 --2- 172.21.15.38:0/1141946149 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb0d40d1120 0x7fb0d40ceb90 unknown :-1 s=CLOSED pgs=1425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:11.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.855+0000 7fb0dade4640 1 --2- 172.21.15.38:0/1141946149 >> v2:172.21.15.195:3300/0 conn(0x7fb0d40d0170 0x7fb0d40d0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:11.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.855+0000 7fb0dade4640 1 -- 172.21.15.38:0/1141946149 >> 172.21.15.38:0/1141946149 conn(0x7fb0d409d830 msgr2=0x7fb0d40b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:11.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.855+0000 7fb0dade4640 1 -- 172.21.15.38:0/1141946149 shutdown_connections 2024-09-21T03:41:11.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.855+0000 7fb0dade4640 1 -- 172.21.15.38:0/1141946149 wait complete. 2024-09-21T03:41:11.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.857+0000 7fb0dade4640 1 Processor -- start 2024-09-21T03:41:11.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.857+0000 7fb0dade4640 1 -- start start 2024-09-21T03:41:11.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.858+0000 7fb0dade4640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb0d40d0170 0x7fb0d406f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:11.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.858+0000 7fb0dade4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb0d40d1120 0x7fb0d40cc850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:11.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.858+0000 7fb0dade4640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb0d415ee80 0x7fb0d406e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:11.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.858+0000 7fb0dade4640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fb0d40b6700 con 0x7fb0d40d1120 2024-09-21T03:41:11.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.858+0000 7fb0dade4640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fb0d40b6580 con 0x7fb0d40d0170 2024-09-21T03:41:11.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.858+0000 7fb0dade4640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fb0d40b6880 con 0x7fb0d415ee80 2024-09-21T03:41:11.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.858+0000 7fb0d3fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb0d40d1120 0x7fb0d40cc850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:11.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.858+0000 7fb0d3fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb0d40d1120 0x7fb0d40cc850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50640/0 (socket says 172.21.15.38:50640) 2024-09-21T03:41:11.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.858+0000 7fb0d3fff640 1 -- 172.21.15.38:0/2751549823 learned_addr learned my addr 172.21.15.38:0/2751549823 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:41:11.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.858+0000 7fb0d8b80640 1 --2- 172.21.15.38:0/2751549823 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb0d40d0170 0x7fb0d406f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:11.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.858+0000 7fb0d9381640 1 --2- 172.21.15.38:0/2751549823 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb0d415ee80 0x7fb0d406e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:11.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.859+0000 7fb0d3fff640 1 -- 172.21.15.38:0/2751549823 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb0d415ee80 msgr2=0x7fb0d406e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:11.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.859+0000 7fb0d3fff640 1 --2- 172.21.15.38:0/2751549823 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb0d415ee80 0x7fb0d406e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:11.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.859+0000 7fb0d3fff640 1 -- 172.21.15.38:0/2751549823 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb0d40d0170 msgr2=0x7fb0d406f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:11.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.859+0000 7fb0d3fff640 1 --2- 172.21.15.38:0/2751549823 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb0d40d0170 0x7fb0d406f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:11.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.859+0000 7fb0d3fff640 1 -- 172.21.15.38:0/2751549823 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0d40cb7b0 con 0x7fb0d40d1120 2024-09-21T03:41:11.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.859+0000 7fb0d3fff640 1 --2- 172.21.15.38:0/2751549823 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb0d40d1120 0x7fb0d40cc850 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7fb0c800e9f0 tx=0x7fb0c800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:11.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.859+0000 7fb0d17fa640 1 -- 172.21.15.38:0/2751549823 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0c800ccc0 con 0x7fb0d40d1120 2024-09-21T03:41:11.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.859+0000 7fb0d17fa640 1 -- 172.21.15.38:0/2751549823 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb0c8004020 con 0x7fb0d40d1120 2024-09-21T03:41:11.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.859+0000 7fb0dade4640 1 -- 172.21.15.38:0/2751549823 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0d40cba90 con 0x7fb0d40d1120 2024-09-21T03:41:11.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.859+0000 7fb0d17fa640 1 -- 172.21.15.38:0/2751549823 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0c800ce60 con 0x7fb0d40d1120 2024-09-21T03:41:11.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.859+0000 7fb0dade4640 1 -- 172.21.15.38:0/2751549823 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0d40cc050 con 0x7fb0d40d1120 2024-09-21T03:41:11.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.860+0000 7fb0dade4640 1 -- 172.21.15.38:0/2751549823 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0d40d1850 con 0x7fb0d40d1120 2024-09-21T03:41:11.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.861+0000 7fb0d17fa640 1 -- 172.21.15.38:0/2751549823 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb0c80035f0 con 0x7fb0d40d1120 2024-09-21T03:41:11.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.861+0000 7fb0d17fa640 1 --2- 172.21.15.38:0/2751549823 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb0c0078b90 0x7fb0c007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:11.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.861+0000 7fb0d17fa640 1 -- 172.21.15.38:0/2751549823 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 6992+0+0 (secure 0 0 0) 0x7fb0c809bad0 con 0x7fb0d40d1120 2024-09-21T03:41:11.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.862+0000 7fb0d8b80640 1 --2- 172.21.15.38:0/2751549823 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb0c0078b90 0x7fb0c007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:11.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.862+0000 7fb0d8b80640 1 --2- 172.21.15.38:0/2751549823 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb0c0078b90 0x7fb0c007b0a0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fb0c4001e20 tx=0x7fb0c4009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:11.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:11.866+0000 7fb0d17fa640 1 -- 172.21.15.38:0/2751549823 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fb0d40d1850 con 0x7fb0d40d1120 2024-09-21T03:41:12.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:12.097+0000 7fb0dade4640 1 -- 172.21.15.38:0/2751549823 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7fb0d40b6da0 con 0x7fb0d40d1120 2024-09-21T03:41:12.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:12.518+0000 7fb0d17fa640 1 -- 172.21.15.38:0/2751549823 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v134) ==== 162+0+0 (secure 0 0 0) 0x7fb0c8063460 con 0x7fb0d40d1120 2024-09-21T03:41:12.520 INFO:teuthology.orchestra.run.smithi038.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-21T03:41:12.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:12.520+0000 7fb0dade4640 1 -- 172.21.15.38:0/2751549823 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb0c0078b90 msgr2=0x7fb0c007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:12.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:12.520+0000 7fb0dade4640 1 --2- 172.21.15.38:0/2751549823 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb0c0078b90 0x7fb0c007b0a0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fb0c4001e20 tx=0x7fb0c4009040 comp rx=0 tx=0).stop 2024-09-21T03:41:12.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:12.520+0000 7fb0dade4640 1 -- 172.21.15.38:0/2751549823 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb0d40d1120 msgr2=0x7fb0d40cc850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:12.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:12.520+0000 7fb0dade4640 1 --2- 172.21.15.38:0/2751549823 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb0d40d1120 0x7fb0d40cc850 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7fb0c800e9f0 tx=0x7fb0c800eeb0 comp rx=0 tx=0).stop 2024-09-21T03:41:12.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:12.520+0000 7fb0dade4640 1 -- 172.21.15.38:0/2751549823 shutdown_connections 2024-09-21T03:41:12.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:12.520+0000 7fb0dade4640 1 --2- 172.21.15.38:0/2751549823 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb0c0078b90 0x7fb0c007b0a0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:12.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:12.520+0000 7fb0dade4640 1 --2- 172.21.15.38:0/2751549823 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb0d415ee80 0x7fb0d406e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:12.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:12.520+0000 7fb0dade4640 1 --2- 172.21.15.38:0/2751549823 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb0d40d1120 0x7fb0d40cc850 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:12.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:12.520+0000 7fb0dade4640 1 --2- 172.21.15.38:0/2751549823 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb0d40d0170 0x7fb0d406f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:12.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:12.520+0000 7fb0dade4640 1 -- 172.21.15.38:0/2751549823 >> 172.21.15.38:0/2751549823 conn(0x7fb0d409d830 msgr2=0x7fb0d415c9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:12.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:12.520+0000 7fb0dade4640 1 -- 172.21.15.38:0/2751549823 shutdown_connections 2024-09-21T03:41:12.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:12.520+0000 7fb0dade4640 1 -- 172.21.15.38:0/2751549823 wait complete. 2024-09-21T03:41:12.532 INFO:tasks.ceph.mds.c.smithi038.stderr:2024-09-21T03:41:12.531+0000 7f35096e3640 -1 mds.0.snaptable: load_2 could not read table: -2 2024-09-21T03:41:12.532 INFO:tasks.ceph.mds.c.smithi038.stderr:2024-09-21T03:41:12.531+0000 7f35096e3640 -1 log_channel(cluster) log [ERR] : error reading table object 'mds_snaptable' -2 ((2) No such file or directory) 2024-09-21T03:41:12.533 INFO:tasks.ceph.mon.a.smithi038.stderr:2024-09-21T03:41:12.531+0000 7f3f45713640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T03:41:12.533 INFO:tasks.ceph.mon.a.smithi038.stderr:2024-09-21T03:41:12.531+0000 7f3f45713640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-09-21T03:41:12.537 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:41:12.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:12.656+0000 7f1ab72e5640 1 -- 172.21.15.38:0/972033360 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1a9c0041c0 msgr2=0x7f1a9c004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:12.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:12.656+0000 7f1ab72e5640 1 --2- 172.21.15.38:0/972033360 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1a9c0041c0 0x7f1a9c004800 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f1aa0009b50 tx=0x7f1a9c005830 comp rx=0 tx=0).stop 2024-09-21T03:41:12.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:12.656+0000 7f1ab5882640 1 -- 172.21.15.38:0/972033360 reap_dead start 2024-09-21T03:41:12.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:12.656+0000 7f1ab72e5640 1 -- 172.21.15.38:0/972033360 shutdown_connections 2024-09-21T03:41:12.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:12.656+0000 7f1ab72e5640 1 -- 172.21.15.38:0/972033360 >> 172.21.15.38:0/972033360 conn(0x7f1ab00d4440 msgr2=0x7f1ab00d4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:12.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:12.656+0000 7f1ab72e5640 1 -- 172.21.15.38:0/972033360 shutdown_connections 2024-09-21T03:41:12.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:12.656+0000 7f1ab72e5640 1 -- 172.21.15.38:0/972033360 wait complete. 2024-09-21T03:41:12.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:12.658+0000 7f1ab72e5640 1 Processor -- start 2024-09-21T03:41:12.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:12.658+0000 7f1ab72e5640 1 -- start start 2024-09-21T03:41:12.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:12.659+0000 7f1ab72e5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1a9c0041c0 0x7f1ab00b2620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:12.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:12.659+0000 7f1ab72e5640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1ab015ddb0 0x7f1ab015e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:12.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:12.659+0000 7f1ab72e5640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1ab00c7f70 0x7f1ab00c8380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:12.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:12.659+0000 7f1ab72e5640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f1ab00cf5b0 con 0x7f1a9c0041c0 2024-09-21T03:41:12.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:12.659+0000 7f1ab72e5640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f1ab00b3b90 con 0x7f1ab00c7f70 2024-09-21T03:41:12.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:12.659+0000 7f1ab72e5640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f1ab00b4f70 con 0x7f1ab015ddb0 2024-09-21T03:41:12.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:12.659+0000 7f1ab5081640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1a9c0041c0 0x7f1ab00b2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:12.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:12.659+0000 7f1ab5081640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1a9c0041c0 0x7f1ab00b2620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46810/0 (socket says 172.21.15.38:46810) 2024-09-21T03:41:12.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:12.659+0000 7f1ab5081640 1 -- 172.21.15.38:0/1264108120 learned_addr learned my addr 172.21.15.38:0/1264108120 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:41:12.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:12.659+0000 7f1ab5081640 1 -- 172.21.15.38:0/1264108120 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1ab015ddb0 msgr2=0x7f1ab015e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:12.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:12.659+0000 7f1ab5081640 1 --2- 172.21.15.38:0/1264108120 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1ab015ddb0 0x7f1ab015e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:12.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:12.659+0000 7f1ab5081640 1 -- 172.21.15.38:0/1264108120 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1ab00c7f70 msgr2=0x7f1ab00c8380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:12.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:12.659+0000 7f1ab5081640 1 --2- 172.21.15.38:0/1264108120 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1ab00c7f70 0x7f1ab00c8380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:12.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:12.659+0000 7f1ab5081640 1 -- 172.21.15.38:0/1264108120 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ab015e700 con 0x7f1a9c0041c0 2024-09-21T03:41:12.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:12.660+0000 7f1ab5081640 1 --2- 172.21.15.38:0/1264108120 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1a9c0041c0 0x7f1ab00b2620 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f1aac006110 tx=0x7f1aac0065d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:12.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:12.660+0000 7f1aa5ffb640 1 -- 172.21.15.38:0/1264108120 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aac015070 con 0x7f1a9c0041c0 2024-09-21T03:41:12.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:12.660+0000 7f1aa5ffb640 1 -- 172.21.15.38:0/1264108120 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1aac00fcc0 con 0x7f1a9c0041c0 2024-09-21T03:41:12.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:12.660+0000 7f1ab72e5640 1 -- 172.21.15.38:0/1264108120 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ab00c8bf0 con 0x7f1a9c0041c0 2024-09-21T03:41:12.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:12.660+0000 7f1aa5ffb640 1 -- 172.21.15.38:0/1264108120 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aac011690 con 0x7f1a9c0041c0 2024-09-21T03:41:12.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:12.660+0000 7f1ab72e5640 1 -- 172.21.15.38:0/1264108120 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ab01f6230 con 0x7f1a9c0041c0 2024-09-21T03:41:12.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:12.661+0000 7f1ab72e5640 1 -- 172.21.15.38:0/1264108120 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ab01629f0 con 0x7f1a9c0041c0 2024-09-21T03:41:12.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:12.662+0000 7f1aa5ffb640 1 -- 172.21.15.38:0/1264108120 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f1aac011830 con 0x7f1a9c0041c0 2024-09-21T03:41:12.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:12.662+0000 7f1aa5ffb640 1 --2- 172.21.15.38:0/1264108120 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1a94078b90 0x7f1a9407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:12.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:12.662+0000 7f1aa5ffb640 1 -- 172.21.15.38:0/1264108120 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 7078+0+0 (secure 0 0 0) 0x7f1aac09c2e0 con 0x7f1a9c0041c0 2024-09-21T03:41:12.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:12.663+0000 7f1ab4880640 1 --2- 172.21.15.38:0/1264108120 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1a94078b90 0x7f1a9407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:12.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:12.663+0000 7f1ab4880640 1 --2- 172.21.15.38:0/1264108120 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1a94078b90 0x7f1a9407b0a0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f1aa0035000 tx=0x7f1aa0004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:12.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:12.667+0000 7f1aa5ffb640 1 -- 172.21.15.38:0/1264108120 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f1ab015e700 con 0x7f1a9c0041c0 2024-09-21T03:41:12.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:12.891+0000 7f1ab72e5640 1 -- 172.21.15.38:0/1264108120 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1ab01f68f0 con 0x7f1a9c0041c0 2024-09-21T03:41:12.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:12.891+0000 7f1aa5ffb640 1 -- 172.21.15.38:0/1264108120 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 136 v136) ==== 95+0+4392 (secure 0 0 0) 0x7f1aac064130 con 0x7f1a9c0041c0 2024-09-21T03:41:12.893 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:41:12.894 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":136,"btime":"2024-09-21T03:41:12:532631+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6086,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/965972663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":965972663},{"type":"v1","addr":"172.21.15.195:6836","nonce":965972663}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6700,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/3176122334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":3176122334},{"type":"v1","addr":"172.21.15.195:6837","nonce":3176122334}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":8061,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/2748959970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":2748959970},{"type":"v1","addr":"172.21.15.38:6835","nonce":2748959970}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":122}],"filesystems":[{"mdsmap":{"epoch":136,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:40:02.319282+0000","modified":"2024-09-21T03:41:12.532561+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":105,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-21T03:41:12.894 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 136 2024-09-21T03:41:12.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:12.893+0000 7f1ab72e5640 1 -- 172.21.15.38:0/1264108120 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1a94078b90 msgr2=0x7f1a9407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:12.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:12.893+0000 7f1ab72e5640 1 --2- 172.21.15.38:0/1264108120 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1a94078b90 0x7f1a9407b0a0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f1aa0035000 tx=0x7f1aa0004730 comp rx=0 tx=0).stop 2024-09-21T03:41:12.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:12.893+0000 7f1ab72e5640 1 -- 172.21.15.38:0/1264108120 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1a9c0041c0 msgr2=0x7f1ab00b2620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:12.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:12.893+0000 7f1ab72e5640 1 --2- 172.21.15.38:0/1264108120 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1a9c0041c0 0x7f1ab00b2620 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f1aac006110 tx=0x7f1aac0065d0 comp rx=0 tx=0).stop 2024-09-21T03:41:12.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:12.893+0000 7f1ab72e5640 1 -- 172.21.15.38:0/1264108120 shutdown_connections 2024-09-21T03:41:12.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:12.893+0000 7f1ab72e5640 1 --2- 172.21.15.38:0/1264108120 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1ab00c7f70 0x7f1ab00c8380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:12.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:12.893+0000 7f1ab72e5640 1 --2- 172.21.15.38:0/1264108120 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1a94078b90 0x7f1a9407b0a0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:12.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:12.893+0000 7f1ab72e5640 1 --2- 172.21.15.38:0/1264108120 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1ab015ddb0 0x7f1ab015e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:12.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:12.893+0000 7f1ab72e5640 1 --2- 172.21.15.38:0/1264108120 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1a9c0041c0 0x7f1ab00b2620 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:12.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:12.893+0000 7f1ab72e5640 1 -- 172.21.15.38:0/1264108120 >> 172.21.15.38:0/1264108120 conn(0x7f1ab00d4440 msgr2=0x7f1ab00d4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:12.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:12.894+0000 7f1ab72e5640 1 -- 172.21.15.38:0/1264108120 shutdown_connections 2024-09-21T03:41:12.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:12.894+0000 7f1ab72e5640 1 -- 172.21.15.38:0/1264108120 wait complete. 2024-09-21T03:41:12.907 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-21T03:41:12.907 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:41:13.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:13.021+0000 7f045f430640 1 -- 172.21.15.38:0/3132768625 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0458162f00 msgr2=0x7f04580d24d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:13.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:13.021+0000 7f045f430640 1 --2- 172.21.15.38:0/3132768625 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0458162f00 0x7f04580d24d0 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f04580b7460 tx=0x7f044802a820 comp rx=0 tx=0).stop 2024-09-21T03:41:13.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:13.021+0000 7f045f430640 1 -- 172.21.15.38:0/3132768625 shutdown_connections 2024-09-21T03:41:13.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:13.021+0000 7f045f430640 1 --2- 172.21.15.38:0/3132768625 >> v2:172.21.15.195:3300/0 conn(0x7f0458162280 0x7f0458162690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:13.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:13.021+0000 7f045f430640 1 --2- 172.21.15.38:0/3132768625 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0458162f00 0x7f04580d24d0 unknown :-1 s=CLOSED pgs=1431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:13.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:13.021+0000 7f045f430640 1 -- 172.21.15.38:0/3132768625 >> 172.21.15.38:0/3132768625 conn(0x7f04580b6b20 msgr2=0x7f04580d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:13.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:13.021+0000 7f045f430640 1 -- 172.21.15.38:0/3132768625 shutdown_connections 2024-09-21T03:41:13.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:13.021+0000 7f045f430640 1 -- 172.21.15.38:0/3132768625 wait complete. 2024-09-21T03:41:13.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:13.023+0000 7f045f430640 1 Processor -- start 2024-09-21T03:41:13.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:13.023+0000 7f045f430640 1 -- start start 2024-09-21T03:41:13.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:13.024+0000 7f045f430640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f04580b2e40 0x7f045814d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:13.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:13.024+0000 7f045f430640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0458162280 0x7f04580ce940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:13.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:13.024+0000 7f045f430640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f04581f60f0 0x7f04580aadc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:13.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:13.024+0000 7f045f430640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f045815e630 con 0x7f04581f60f0 2024-09-21T03:41:13.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:13.024+0000 7f045f430640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f045815e4b0 con 0x7f0458162280 2024-09-21T03:41:13.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:13.024+0000 7f045f430640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f045815e7b0 con 0x7f04580b2e40 2024-09-21T03:41:13.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:13.024+0000 7f045d9cd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f04581f60f0 0x7f04580aadc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:13.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:13.024+0000 7f045d9cd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f04581f60f0 0x7f04580aadc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46832/0 (socket says 172.21.15.38:46832) 2024-09-21T03:41:13.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:13.024+0000 7f045d9cd640 1 -- 172.21.15.38:0/2648265961 learned_addr learned my addr 172.21.15.38:0/2648265961 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:41:13.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:13.025+0000 7f045c9cb640 1 --2- 172.21.15.38:0/2648265961 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0458162280 0x7f04580ce940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:13.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:13.025+0000 7f045d9cd640 1 -- 172.21.15.38:0/2648265961 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f04580b2e40 msgr2=0x7f045814d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:13.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:13.025+0000 7f045d9cd640 1 --2- 172.21.15.38:0/2648265961 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f04580b2e40 0x7f045814d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:13.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:13.025+0000 7f045d9cd640 1 -- 172.21.15.38:0/2648265961 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0458162280 msgr2=0x7f04580ce940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:13.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:13.025+0000 7f045d9cd640 1 --2- 172.21.15.38:0/2648265961 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0458162280 0x7f04580ce940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:13.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:13.025+0000 7f045d9cd640 1 -- 172.21.15.38:0/2648265961 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f045814e220 con 0x7f04581f60f0 2024-09-21T03:41:13.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:13.025+0000 7f045d9cd640 1 --2- 172.21.15.38:0/2648265961 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f04581f60f0 0x7f04580aadc0 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f045000c3c0 tx=0x7f045000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:13.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:13.025+0000 7f0455ffb640 1 -- 172.21.15.38:0/2648265961 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0450017070 con 0x7f04581f60f0 2024-09-21T03:41:13.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:13.025+0000 7f0455ffb640 1 -- 172.21.15.38:0/2648265961 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f04500035f0 con 0x7f04581f60f0 2024-09-21T03:41:13.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:13.025+0000 7f045f430640 1 -- 172.21.15.38:0/2648265961 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f045814e500 con 0x7f04581f60f0 2024-09-21T03:41:13.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:13.025+0000 7f0455ffb640 1 -- 172.21.15.38:0/2648265961 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0450003a40 con 0x7f04581f60f0 2024-09-21T03:41:13.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:13.025+0000 7f045f430640 1 -- 172.21.15.38:0/2648265961 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04580b1c50 con 0x7f04581f60f0 2024-09-21T03:41:13.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:13.026+0000 7f045f430640 1 -- 172.21.15.38:0/2648265961 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f045814df90 con 0x7f04581f60f0 2024-09-21T03:41:13.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:13.027+0000 7f0455ffb640 1 -- 172.21.15.38:0/2648265961 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f045001a070 con 0x7f04581f60f0 2024-09-21T03:41:13.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:13.027+0000 7f0455ffb640 1 --2- 172.21.15.38:0/2648265961 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0444078b90 0x7f044407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:13.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:13.028+0000 7f0455ffb640 1 -- 172.21.15.38:0/2648265961 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 7078+0+0 (secure 0 0 0) 0x7f045009a4b0 con 0x7f04581f60f0 2024-09-21T03:41:13.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:13.028+0000 7f045d1cc640 1 --2- 172.21.15.38:0/2648265961 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0444078b90 0x7f044407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:13.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:13.028+0000 7f045d1cc640 1 --2- 172.21.15.38:0/2648265961 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0444078b90 0x7f044407b0a0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f0448034000 tx=0x7f0448004a40 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:13.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:13.032+0000 7f0455ffb640 1 -- 172.21.15.38:0/2648265961 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f0458162bd0 con 0x7f04581f60f0 2024-09-21T03:41:13.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:13.257+0000 7f045f430640 1 -- 172.21.15.38:0/2648265961 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0458162f00 con 0x7f04581f60f0 2024-09-21T03:41:13.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:13.257+0000 7f0455ffb640 1 -- 172.21.15.38:0/2648265961 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 136 v136) ==== 95+0+4392 (secure 0 0 0) 0x7f04500623a0 con 0x7f04581f60f0 2024-09-21T03:41:13.259 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:41:13.259 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":136,"btime":"2024-09-21T03:41:12:532631+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6086,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/965972663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":965972663},{"type":"v1","addr":"172.21.15.195:6836","nonce":965972663}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6700,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/3176122334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":3176122334},{"type":"v1","addr":"172.21.15.195:6837","nonce":3176122334}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":8061,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/2748959970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":2748959970},{"type":"v1","addr":"172.21.15.38:6835","nonce":2748959970}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":122}],"filesystems":[{"mdsmap":{"epoch":136,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:40:02.319282+0000","modified":"2024-09-21T03:41:12.532561+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":105,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-21T03:41:13.260 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 136 2024-09-21T03:41:13.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:13.259+0000 7f045f430640 1 -- 172.21.15.38:0/2648265961 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0444078b90 msgr2=0x7f044407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:13.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:13.259+0000 7f045f430640 1 --2- 172.21.15.38:0/2648265961 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0444078b90 0x7f044407b0a0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f0448034000 tx=0x7f0448004a40 comp rx=0 tx=0).stop 2024-09-21T03:41:13.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:13.259+0000 7f045f430640 1 -- 172.21.15.38:0/2648265961 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f04581f60f0 msgr2=0x7f04580aadc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:13.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:13.259+0000 7f045f430640 1 --2- 172.21.15.38:0/2648265961 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f04581f60f0 0x7f04580aadc0 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f045000c3c0 tx=0x7f045000c880 comp rx=0 tx=0).stop 2024-09-21T03:41:13.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:13.260+0000 7f045f430640 1 -- 172.21.15.38:0/2648265961 shutdown_connections 2024-09-21T03:41:13.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:13.260+0000 7f045f430640 1 --2- 172.21.15.38:0/2648265961 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f04581f60f0 0x7f04580aadc0 unknown :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:13.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:13.260+0000 7f045f430640 1 --2- 172.21.15.38:0/2648265961 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0458162280 0x7f04580ce940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:13.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:13.260+0000 7f045f430640 1 --2- 172.21.15.38:0/2648265961 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0444078b90 0x7f044407b0a0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:13.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:13.260+0000 7f045f430640 1 --2- 172.21.15.38:0/2648265961 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f04580b2e40 0x7f045814d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:13.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:13.260+0000 7f045f430640 1 -- 172.21.15.38:0/2648265961 >> 172.21.15.38:0/2648265961 conn(0x7f04580b6b20 msgr2=0x7f04580b3250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:13.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:13.260+0000 7f045f430640 1 -- 172.21.15.38:0/2648265961 shutdown_connections 2024-09-21T03:41:13.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:13.260+0000 7f045f430640 1 -- 172.21.15.38:0/2648265961 wait complete. 2024-09-21T03:41:13.272 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-21T03:41:13.273 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:41:13.390 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:13.387+0000 7f4b9c932640 1 -- 172.21.15.38:0/41363921 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4b84003c10 msgr2=0x7f4b84004060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:13.390 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:13.387+0000 7f4b9c932640 1 --2- 172.21.15.38:0/41363921 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4b84003c10 0x7f4b84004060 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f4b90001f30 tx=0x7f4b84005830 comp rx=0 tx=0).stop 2024-09-21T03:41:13.390 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:13.388+0000 7f4b96d9d640 1 -- 172.21.15.38:0/41363921 reap_dead start 2024-09-21T03:41:13.390 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:13.388+0000 7f4b9c932640 1 -- 172.21.15.38:0/41363921 shutdown_connections 2024-09-21T03:41:13.390 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:13.388+0000 7f4b9c932640 1 -- 172.21.15.38:0/41363921 >> 172.21.15.38:0/41363921 conn(0x7f4b980d4440 msgr2=0x7f4b980d4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:13.390 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:13.388+0000 7f4b9c932640 1 -- 172.21.15.38:0/41363921 shutdown_connections 2024-09-21T03:41:13.390 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:13.388+0000 7f4b9c932640 1 -- 172.21.15.38:0/41363921 wait complete. 2024-09-21T03:41:13.391 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:13.390+0000 7f4b9c932640 1 Processor -- start 2024-09-21T03:41:13.391 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:13.390+0000 7f4b9c932640 1 -- start start 2024-09-21T03:41:13.394 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:13.391+0000 7f4b9c932640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4b84003c10 0x7f4b9814bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:13.394 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:13.391+0000 7f4b9c932640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4b9814de20 0x7f4b9814c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:13.394 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:13.391+0000 7f4b9c932640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4b9814f820 0x7f4b9814fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:13.394 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:13.391+0000 7f4b9c932640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f4b980cf5e0 con 0x7f4b84003c10 2024-09-21T03:41:13.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:13.391+0000 7f4b9c932640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f4b980b3b90 con 0x7f4b9814f820 2024-09-21T03:41:13.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:13.391+0000 7f4b9c932640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f4b980b4f70 con 0x7f4b9814de20 2024-09-21T03:41:13.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:13.391+0000 7f4b9659c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4b84003c10 0x7f4b9814bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:13.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:13.391+0000 7f4b9659c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4b84003c10 0x7f4b9814bb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46864/0 (socket says 172.21.15.38:46864) 2024-09-21T03:41:13.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:13.391+0000 7f4b9659c640 1 -- 172.21.15.38:0/640812735 learned_addr learned my addr 172.21.15.38:0/640812735 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:41:13.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:13.391+0000 7f4b96d9d640 1 --2- 172.21.15.38:0/640812735 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4b9814f820 0x7f4b9814fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:13.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:13.391+0000 7f4b95d9b640 1 --2- 172.21.15.38:0/640812735 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4b9814de20 0x7f4b9814c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:13.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:13.391+0000 7f4b9659c640 1 -- 172.21.15.38:0/640812735 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4b9814de20 msgr2=0x7f4b9814c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:13.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:13.392+0000 7f4b9659c640 1 --2- 172.21.15.38:0/640812735 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4b9814de20 0x7f4b9814c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:13.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:13.392+0000 7f4b9659c640 1 -- 172.21.15.38:0/640812735 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4b9814f820 msgr2=0x7f4b9814fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:13.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:13.392+0000 7f4b9659c640 1 --2- 172.21.15.38:0/640812735 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4b9814f820 0x7f4b9814fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:13.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:13.392+0000 7f4b9659c640 1 -- 172.21.15.38:0/640812735 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b9814c870 con 0x7f4b84003c10 2024-09-21T03:41:13.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:13.392+0000 7f4b9659c640 1 --2- 172.21.15.38:0/640812735 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4b84003c10 0x7f4b9814bb70 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f4b9002ade0 tx=0x7f4b90006830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:13.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:13.392+0000 7f4b8effd640 1 -- 172.21.15.38:0/640812735 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b90006c40 con 0x7f4b84003c10 2024-09-21T03:41:13.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:13.392+0000 7f4b8effd640 1 -- 172.21.15.38:0/640812735 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4b90003d40 con 0x7f4b84003c10 2024-09-21T03:41:13.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:13.392+0000 7f4b8effd640 1 -- 172.21.15.38:0/640812735 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b900346c0 con 0x7f4b84003c10 2024-09-21T03:41:13.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:13.392+0000 7f4b9c932640 1 -- 172.21.15.38:0/640812735 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b98150170 con 0x7f4b84003c10 2024-09-21T03:41:13.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:13.392+0000 7f4b9c932640 1 -- 172.21.15.38:0/640812735 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b9814d060 con 0x7f4b84003c10 2024-09-21T03:41:13.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:13.393+0000 7f4b9c932640 1 -- 172.21.15.38:0/640812735 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b981629f0 con 0x7f4b84003c10 2024-09-21T03:41:13.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:13.394+0000 7f4b8effd640 1 -- 172.21.15.38:0/640812735 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f4b90034a70 con 0x7f4b84003c10 2024-09-21T03:41:13.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:13.394+0000 7f4b8effd640 1 --2- 172.21.15.38:0/640812735 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4b78078bc0 0x7f4b7807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:13.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:13.394+0000 7f4b8effd640 1 -- 172.21.15.38:0/640812735 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 7078+0+0 (secure 0 0 0) 0x7f4b900bb7b0 con 0x7f4b84003c10 2024-09-21T03:41:13.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:13.394+0000 7f4b95d9b640 1 --2- 172.21.15.38:0/640812735 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4b78078bc0 0x7f4b7807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:13.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:13.395+0000 7f4b95d9b640 1 --2- 172.21.15.38:0/640812735 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4b78078bc0 0x7f4b7807b0d0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f4b980d4f10 tx=0x7f4b80005f90 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:13.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:13.399+0000 7f4b8effd640 1 -- 172.21.15.38:0/640812735 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f4b900830d0 con 0x7f4b84003c10 2024-09-21T03:41:13.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:13.625+0000 7f4b9c932640 1 -- 172.21.15.38:0/640812735 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4b98076180 con 0x7f4b84003c10 2024-09-21T03:41:13.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:13.626+0000 7f4b8effd640 1 -- 172.21.15.38:0/640812735 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 136 v136) ==== 95+0+4392 (secure 0 0 0) 0x7f4b90087b00 con 0x7f4b84003c10 2024-09-21T03:41:13.628 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:41:13.628 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":136,"btime":"2024-09-21T03:41:12:532631+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6086,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/965972663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":965972663},{"type":"v1","addr":"172.21.15.195:6836","nonce":965972663}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6700,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/3176122334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":3176122334},{"type":"v1","addr":"172.21.15.195:6837","nonce":3176122334}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":8061,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/2748959970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":2748959970},{"type":"v1","addr":"172.21.15.38:6835","nonce":2748959970}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":122}],"filesystems":[{"mdsmap":{"epoch":136,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:40:02.319282+0000","modified":"2024-09-21T03:41:12.532561+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":105,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-21T03:41:13.628 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 136 2024-09-21T03:41:13.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:13.628+0000 7f4b9c932640 1 -- 172.21.15.38:0/640812735 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4b78078bc0 msgr2=0x7f4b7807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:13.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:13.628+0000 7f4b9c932640 1 --2- 172.21.15.38:0/640812735 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4b78078bc0 0x7f4b7807b0d0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f4b980d4f10 tx=0x7f4b80005f90 comp rx=0 tx=0).stop 2024-09-21T03:41:13.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:13.628+0000 7f4b9c932640 1 -- 172.21.15.38:0/640812735 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4b84003c10 msgr2=0x7f4b9814bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:13.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:13.628+0000 7f4b9c932640 1 --2- 172.21.15.38:0/640812735 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4b84003c10 0x7f4b9814bb70 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f4b9002ade0 tx=0x7f4b90006830 comp rx=0 tx=0).stop 2024-09-21T03:41:13.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:13.628+0000 7f4b9c932640 1 -- 172.21.15.38:0/640812735 shutdown_connections 2024-09-21T03:41:13.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:13.628+0000 7f4b9c932640 1 --2- 172.21.15.38:0/640812735 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4b9814f820 0x7f4b9814fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:13.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:13.628+0000 7f4b9c932640 1 --2- 172.21.15.38:0/640812735 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4b78078bc0 0x7f4b7807b0d0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:13.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:13.628+0000 7f4b9c932640 1 --2- 172.21.15.38:0/640812735 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4b9814de20 0x7f4b9814c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:13.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:13.628+0000 7f4b9c932640 1 --2- 172.21.15.38:0/640812735 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4b84003c10 0x7f4b9814bb70 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:13.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:13.628+0000 7f4b9c932640 1 -- 172.21.15.38:0/640812735 >> 172.21.15.38:0/640812735 conn(0x7f4b980d4440 msgr2=0x7f4b9815e4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:13.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:13.628+0000 7f4b9c932640 1 -- 172.21.15.38:0/640812735 shutdown_connections 2024-09-21T03:41:13.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:13.628+0000 7f4b9c932640 1 -- 172.21.15.38:0/640812735 wait complete. 2024-09-21T03:41:13.641 INFO:tasks.cephfs.test_damage:Result: Mutation 'Delete mds_snaptable' led to DAMAGED state 2024-09-21T03:41:13.642 INFO:tasks.cephfs.test_damage:Applying mutation 'Delete mds0_inotable' 2024-09-21T03:41:13.642 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-21T03:41:13.642 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:41:13.658 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056795,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2354 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T03:41:13.658 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi038.front.sepia.ceph.com 2024-09-21T03:41:13.658 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:41:13.658 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T03:41:13.714 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T03:41:13.714 INFO:teuthology.orchestra.run.smithi038.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-09-21T03:41:13.715 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-21T03:41:13.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:13.870+0000 7fb69b0b4640 1 -- 172.21.15.38:0/685522024 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb6940b5260 msgr2=0x7fb69415ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:13.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:13.870+0000 7fb69b0b4640 1 --2- 172.21.15.38:0/685522024 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb6940b5260 0x7fb69415ba80 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7fb694073190 tx=0x7fb67c02aac0 comp rx=0 tx=0).stop 2024-09-21T03:41:13.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:13.870+0000 7fb69b0b4640 1 -- 172.21.15.38:0/685522024 shutdown_connections 2024-09-21T03:41:13.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:13.870+0000 7fb69b0b4640 1 --2- 172.21.15.38:0/685522024 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb6940b5260 0x7fb69415ba80 unknown :-1 s=CLOSED pgs=1436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:13.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:13.870+0000 7fb69b0b4640 1 --2- 172.21.15.38:0/685522024 >> v2:172.21.15.195:3300/0 conn(0x7fb69406def0 0x7fb69406f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:13.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:13.870+0000 7fb69b0b4640 1 -- 172.21.15.38:0/685522024 >> 172.21.15.38:0/685522024 conn(0x7fb6940cdca0 msgr2=0x7fb6940ce0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:13.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:13.870+0000 7fb69b0b4640 1 -- 172.21.15.38:0/685522024 shutdown_connections 2024-09-21T03:41:13.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:13.870+0000 7fb69b0b4640 1 -- 172.21.15.38:0/685522024 wait complete. 2024-09-21T03:41:13.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:13.872+0000 7fb69b0b4640 1 Processor -- start 2024-09-21T03:41:13.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:13.872+0000 7fb69b0b4640 1 -- start start 2024-09-21T03:41:13.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:13.874+0000 7fb69b0b4640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb69406def0 0x7fb694155770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:13.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:13.874+0000 7fb69b0b4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb6940b5260 0x7fb6940cb640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:13.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:13.874+0000 7fb69b0b4640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb69415bfc0 0x7fb6940cfe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:13.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:13.874+0000 7fb69b0b4640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fb6940b5a00 con 0x7fb6940b5260 2024-09-21T03:41:13.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:13.874+0000 7fb69b0b4640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fb6940b5880 con 0x7fb69415bfc0 2024-09-21T03:41:13.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:13.874+0000 7fb69b0b4640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fb694071300 con 0x7fb69406def0 2024-09-21T03:41:13.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:13.874+0000 7fb693fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb6940b5260 0x7fb6940cb640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:13.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:13.874+0000 7fb693fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb6940b5260 0x7fb6940cb640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46890/0 (socket says 172.21.15.38:46890) 2024-09-21T03:41:13.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:13.874+0000 7fb693fff640 1 -- 172.21.15.38:0/655198685 learned_addr learned my addr 172.21.15.38:0/655198685 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:41:13.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:13.874+0000 7fb698e50640 1 --2- 172.21.15.38:0/655198685 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb69406def0 0x7fb694155770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:13.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:13.874+0000 7fb699651640 1 --2- 172.21.15.38:0/655198685 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb69415bfc0 0x7fb6940cfe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:13.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:13.874+0000 7fb693fff640 1 -- 172.21.15.38:0/655198685 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb69406def0 msgr2=0x7fb694155770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:13.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:13.874+0000 7fb693fff640 1 --2- 172.21.15.38:0/655198685 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb69406def0 0x7fb694155770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:13.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:13.874+0000 7fb693fff640 1 -- 172.21.15.38:0/655198685 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb69415bfc0 msgr2=0x7fb6940cfe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:13.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:13.874+0000 7fb693fff640 1 --2- 172.21.15.38:0/655198685 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb69415bfc0 0x7fb6940cfe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:13.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:13.874+0000 7fb693fff640 1 -- 172.21.15.38:0/655198685 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb694155cb0 con 0x7fb6940b5260 2024-09-21T03:41:13.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:13.874+0000 7fb693fff640 1 --2- 172.21.15.38:0/655198685 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb6940b5260 0x7fb6940cb640 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7fb67c009bf0 tx=0x7fb67c003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:13.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:13.875+0000 7fb6917fa640 1 -- 172.21.15.38:0/655198685 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb67c044070 con 0x7fb6940b5260 2024-09-21T03:41:13.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:13.875+0000 7fb6917fa640 1 -- 172.21.15.38:0/655198685 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb67c004250 con 0x7fb6940b5260 2024-09-21T03:41:13.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:13.875+0000 7fb69b0b4640 1 -- 172.21.15.38:0/655198685 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6940cc0a0 con 0x7fb6940b5260 2024-09-21T03:41:13.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:13.875+0000 7fb6917fa640 1 -- 172.21.15.38:0/655198685 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb67c004680 con 0x7fb6940b5260 2024-09-21T03:41:13.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:13.875+0000 7fb69b0b4640 1 -- 172.21.15.38:0/655198685 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb694156350 con 0x7fb6940b5260 2024-09-21T03:41:13.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:13.876+0000 7fb69b0b4640 1 -- 172.21.15.38:0/655198685 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb694156140 con 0x7fb6940b5260 2024-09-21T03:41:13.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:13.877+0000 7fb6917fa640 1 -- 172.21.15.38:0/655198685 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb67c036400 con 0x7fb6940b5260 2024-09-21T03:41:13.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:13.877+0000 7fb6917fa640 1 --2- 172.21.15.38:0/655198685 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb674078b90 0x7fb67407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:13.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:13.877+0000 7fb6917fa640 1 -- 172.21.15.38:0/655198685 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 7078+0+0 (secure 0 0 0) 0x7fb67c0bbbd0 con 0x7fb6940b5260 2024-09-21T03:41:13.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:13.877+0000 7fb698e50640 1 --2- 172.21.15.38:0/655198685 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb674078b90 0x7fb67407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:13.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:13.878+0000 7fb698e50640 1 --2- 172.21.15.38:0/655198685 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb674078b90 0x7fb67407b0a0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fb69415efd0 tx=0x7fb68400b2e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:13.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:13.881+0000 7fb6917fa640 1 -- 172.21.15.38:0/655198685 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fb694155cb0 con 0x7fb6940b5260 2024-09-21T03:41:14.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:14.108+0000 7fb69b0b4640 1 -- 172.21.15.38:0/655198685 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fb69406f410 con 0x7fb6940b5260 2024-09-21T03:41:14.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:14.544+0000 7fb6917fa640 1 -- 172.21.15.38:0/655198685 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v137) ==== 161+0+0 (secure 0 0 0) 0x7fb67c032030 con 0x7fb6940b5260 2024-09-21T03:41:14.546 INFO:teuthology.orchestra.run.smithi038.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-21T03:41:14.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:14.547+0000 7fb69b0b4640 1 -- 172.21.15.38:0/655198685 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb674078b90 msgr2=0x7fb67407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:14.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:14.547+0000 7fb69b0b4640 1 --2- 172.21.15.38:0/655198685 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb674078b90 0x7fb67407b0a0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fb69415efd0 tx=0x7fb68400b2e0 comp rx=0 tx=0).stop 2024-09-21T03:41:14.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:14.547+0000 7fb69b0b4640 1 -- 172.21.15.38:0/655198685 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb6940b5260 msgr2=0x7fb6940cb640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:14.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:14.547+0000 7fb69b0b4640 1 --2- 172.21.15.38:0/655198685 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb6940b5260 0x7fb6940cb640 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7fb67c009bf0 tx=0x7fb67c003f10 comp rx=0 tx=0).stop 2024-09-21T03:41:14.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:14.547+0000 7fb69b0b4640 1 -- 172.21.15.38:0/655198685 shutdown_connections 2024-09-21T03:41:14.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:14.547+0000 7fb69b0b4640 1 --2- 172.21.15.38:0/655198685 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb69415bfc0 0x7fb6940cfe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:14.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:14.547+0000 7fb69b0b4640 1 --2- 172.21.15.38:0/655198685 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb6940b5260 0x7fb6940cb640 unknown :-1 s=CLOSED pgs=1437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:14.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:14.547+0000 7fb69b0b4640 1 --2- 172.21.15.38:0/655198685 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb674078b90 0x7fb67407b0a0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:14.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:14.547+0000 7fb69b0b4640 1 --2- 172.21.15.38:0/655198685 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb69406def0 0x7fb694155770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:14.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:14.547+0000 7fb69b0b4640 1 -- 172.21.15.38:0/655198685 >> 172.21.15.38:0/655198685 conn(0x7fb6940cdca0 msgr2=0x7fb6940aeb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:14.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:14.548+0000 7fb69b0b4640 1 -- 172.21.15.38:0/655198685 shutdown_connections 2024-09-21T03:41:14.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:14.548+0000 7fb69b0b4640 1 -- 172.21.15.38:0/655198685 wait complete. 2024-09-21T03:41:14.562 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-09-21T03:41:14.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:14.680+0000 7f2480027640 1 -- 172.21.15.38:0/2362566882 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f24780b9ad0 msgr2=0x7f24780d0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:14.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:14.680+0000 7f2480027640 1 --2- 172.21.15.38:0/2362566882 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f24780b9ad0 0x7f24780d0be0 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f24780cfba0 tx=0x7f246802a4b0 comp rx=0 tx=0).stop 2024-09-21T03:41:14.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:14.680+0000 7f2480027640 1 -- 172.21.15.38:0/2362566882 shutdown_connections 2024-09-21T03:41:14.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:14.680+0000 7f2480027640 1 --2- 172.21.15.38:0/2362566882 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f24780d1120 0x7f24780ceb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:14.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:14.680+0000 7f2480027640 1 --2- 172.21.15.38:0/2362566882 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f24780b9ad0 0x7f24780d0be0 unknown :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:14.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:14.680+0000 7f2480027640 1 --2- 172.21.15.38:0/2362566882 >> v2:172.21.15.195:3300/0 conn(0x7f247815ee80 0x7f24780d0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:14.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:14.680+0000 7f2480027640 1 -- 172.21.15.38:0/2362566882 >> 172.21.15.38:0/2362566882 conn(0x7f247809d830 msgr2=0x7f24780b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:14.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:14.680+0000 7f2480027640 1 -- 172.21.15.38:0/2362566882 shutdown_connections 2024-09-21T03:41:14.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:14.680+0000 7f2480027640 1 -- 172.21.15.38:0/2362566882 wait complete. 2024-09-21T03:41:14.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:14.682+0000 7f2480027640 1 Processor -- start 2024-09-21T03:41:14.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:14.682+0000 7f2480027640 1 -- start start 2024-09-21T03:41:14.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:14.683+0000 7f2480027640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f24780b9ad0 0x7f247815c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:14.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:14.683+0000 7f2480027640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f24780d1120 0x7f24781577a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:14.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:14.684+0000 7f2480027640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f247815ee80 0x7f24780b27a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:14.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:14.684+0000 7f2480027640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f24780cf520 con 0x7f247815ee80 2024-09-21T03:41:14.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:14.684+0000 7f2480027640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f24780cf3a0 con 0x7f24780d1120 2024-09-21T03:41:14.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:14.684+0000 7f2480027640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f24780b6580 con 0x7f24780b9ad0 2024-09-21T03:41:14.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:14.684+0000 7f247e5c4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f247815ee80 0x7f24780b27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:14.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:14.684+0000 7f247e5c4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f247815ee80 0x7f24780b27a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46908/0 (socket says 172.21.15.38:46908) 2024-09-21T03:41:14.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:14.684+0000 7f247e5c4640 1 -- 172.21.15.38:0/1175303020 learned_addr learned my addr 172.21.15.38:0/1175303020 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:41:14.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:14.684+0000 7f247d5c2640 1 --2- 172.21.15.38:0/1175303020 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f24780d1120 0x7f24781577a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:14.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:14.684+0000 7f247ddc3640 1 --2- 172.21.15.38:0/1175303020 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f24780b9ad0 0x7f247815c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:14.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:14.684+0000 7f247e5c4640 1 -- 172.21.15.38:0/1175303020 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f24780b9ad0 msgr2=0x7f247815c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:14.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:14.684+0000 7f247e5c4640 1 --2- 172.21.15.38:0/1175303020 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f24780b9ad0 0x7f247815c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:14.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:14.684+0000 7f247e5c4640 1 -- 172.21.15.38:0/1175303020 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f24780d1120 msgr2=0x7f24781577a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:14.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:14.684+0000 7f247e5c4640 1 --2- 172.21.15.38:0/1175303020 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f24780d1120 0x7f24781577a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:14.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:14.684+0000 7f247e5c4640 1 -- 172.21.15.38:0/1175303020 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24781f5e00 con 0x7f247815ee80 2024-09-21T03:41:14.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:14.684+0000 7f247ddc3640 1 --2- 172.21.15.38:0/1175303020 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f24780b9ad0 0x7f247815c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:41:14.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:14.684+0000 7f247d5c2640 1 --2- 172.21.15.38:0/1175303020 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f24780d1120 0x7f24781577a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:41:14.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:14.684+0000 7f247e5c4640 1 --2- 172.21.15.38:0/1175303020 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f247815ee80 0x7f24780b27a0 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f247000e8e0 tx=0x7f247000eda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:14.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:14.684+0000 7f246e7fc640 1 -- 172.21.15.38:0/1175303020 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2470004d20 con 0x7f247815ee80 2024-09-21T03:41:14.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:14.684+0000 7f246e7fc640 1 -- 172.21.15.38:0/1175303020 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f247000ca60 con 0x7f247815ee80 2024-09-21T03:41:14.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:14.684+0000 7f246e7fc640 1 -- 172.21.15.38:0/1175303020 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2470005020 con 0x7f247815ee80 2024-09-21T03:41:14.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:14.684+0000 7f2480027640 1 -- 172.21.15.38:0/1175303020 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24781f60e0 con 0x7f247815ee80 2024-09-21T03:41:14.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:14.684+0000 7f2480027640 1 -- 172.21.15.38:0/1175303020 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24781f66a0 con 0x7f247815ee80 2024-09-21T03:41:14.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:14.685+0000 7f2480027640 1 -- 172.21.15.38:0/1175303020 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f247806acf0 con 0x7f247815ee80 2024-09-21T03:41:14.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:14.686+0000 7f246e7fc640 1 -- 172.21.15.38:0/1175303020 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f24700051c0 con 0x7f247815ee80 2024-09-21T03:41:14.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:14.687+0000 7f246e7fc640 1 --2- 172.21.15.38:0/1175303020 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f245c078b90 0x7f245c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:14.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:14.687+0000 7f246e7fc640 1 -- 172.21.15.38:0/1175303020 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 7078+0+0 (secure 0 0 0) 0x7f247009bb50 con 0x7f247815ee80 2024-09-21T03:41:14.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:14.687+0000 7f247ddc3640 1 --2- 172.21.15.38:0/1175303020 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f245c078b90 0x7f245c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:14.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:14.688+0000 7f247ddc3640 1 --2- 172.21.15.38:0/1175303020 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f245c078b90 0x7f245c07b0a0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f2474005fd0 tx=0x7f247400f360 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:14.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:14.691+0000 7f246e7fc640 1 -- 172.21.15.38:0/1175303020 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f247806acf0 con 0x7f247815ee80 2024-09-21T03:41:14.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:14.960+0000 7f2480027640 1 -- 172.21.15.38:0/1175303020 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7f24780ceb90 con 0x7f247815ee80 2024-09-21T03:41:15.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.550+0000 7f246e7fc640 1 -- 172.21.15.38:0/1175303020 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v138) ==== 101+0+0 (secure 0 0 0) 0x7f2470016030 con 0x7f247815ee80 2024-09-21T03:41:15.552 INFO:teuthology.orchestra.run.smithi038.stderr:repaired: restoring rank 6:0 2024-09-21T03:41:15.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.552+0000 7f2480027640 1 -- 172.21.15.38:0/1175303020 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f245c078b90 msgr2=0x7f245c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:15.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.552+0000 7f2480027640 1 --2- 172.21.15.38:0/1175303020 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f245c078b90 0x7f245c07b0a0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f2474005fd0 tx=0x7f247400f360 comp rx=0 tx=0).stop 2024-09-21T03:41:15.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.553+0000 7f2480027640 1 -- 172.21.15.38:0/1175303020 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f247815ee80 msgr2=0x7f24780b27a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:15.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.553+0000 7f2480027640 1 --2- 172.21.15.38:0/1175303020 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f247815ee80 0x7f24780b27a0 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f247000e8e0 tx=0x7f247000eda0 comp rx=0 tx=0).stop 2024-09-21T03:41:15.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.553+0000 7f2480027640 1 -- 172.21.15.38:0/1175303020 shutdown_connections 2024-09-21T03:41:15.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.553+0000 7f2480027640 1 --2- 172.21.15.38:0/1175303020 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f247815ee80 0x7f24780b27a0 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:15.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.553+0000 7f2480027640 1 --2- 172.21.15.38:0/1175303020 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f24780d1120 0x7f24781577a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:15.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.553+0000 7f2480027640 1 --2- 172.21.15.38:0/1175303020 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f245c078b90 0x7f245c07b0a0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:15.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.553+0000 7f2480027640 1 --2- 172.21.15.38:0/1175303020 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f24780b9ad0 0x7f247815c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:15.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.553+0000 7f2480027640 1 -- 172.21.15.38:0/1175303020 >> 172.21.15.38:0/1175303020 conn(0x7f247809d830 msgr2=0x7f24780d2ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:15.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.553+0000 7f2480027640 1 -- 172.21.15.38:0/1175303020 shutdown_connections 2024-09-21T03:41:15.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.553+0000 7f2480027640 1 -- 172.21.15.38:0/1175303020 wait complete. 2024-09-21T03:41:15.567 DEBUG:teuthology.orchestra.run.smithi038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-09-21T03:41:15.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.623+0000 7fb21defaf40 1 Processor -- start 2024-09-21T03:41:15.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.623+0000 7fb21defaf40 1 -- start start 2024-09-21T03:41:15.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.624+0000 7fb21defaf40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb2000028f0 0x55cf7a848e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:15.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.624+0000 7fb21defaf40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cf7a78d660 0x55cf7a78da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:15.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.624+0000 7fb21defaf40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cf7a862db0 0x55cf7a8631c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:15.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.624+0000 7fb21defaf40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55cf7a73e430 con 0x55cf7a862db0 2024-09-21T03:41:15.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.624+0000 7fb21defaf40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55cf7a74b1f0 con 0x55cf7a78d660 2024-09-21T03:41:15.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.624+0000 7fb21defaf40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55cf7a6dfd60 con 0x7fb2000028f0 2024-09-21T03:41:15.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.625+0000 7fb2181b5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cf7a862db0 0x55cf7a8631c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:15.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.625+0000 7fb2181b5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cf7a862db0 0x55cf7a8631c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46916/0 (socket says 172.21.15.38:46916) 2024-09-21T03:41:15.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.625+0000 7fb2181b5640 1 -- 172.21.15.38:0/1321444298 learned_addr learned my addr 172.21.15.38:0/1321444298 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:41:15.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.625+0000 7fb2179b4640 1 --2- 172.21.15.38:0/1321444298 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb2000028f0 0x55cf7a848e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:15.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.625+0000 7fb2171b3640 1 --2- 172.21.15.38:0/1321444298 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cf7a78d660 0x55cf7a78da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:15.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.625+0000 7fb2181b5640 1 -- 172.21.15.38:0/1321444298 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb2000028f0 msgr2=0x55cf7a848e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:15.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.625+0000 7fb2181b5640 1 --2- 172.21.15.38:0/1321444298 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb2000028f0 0x55cf7a848e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:15.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.625+0000 7fb2181b5640 1 -- 172.21.15.38:0/1321444298 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cf7a78d660 msgr2=0x55cf7a78da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:15.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.625+0000 7fb2181b5640 1 --2- 172.21.15.38:0/1321444298 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cf7a78d660 0x55cf7a78da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:15.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.625+0000 7fb2181b5640 1 -- 172.21.15.38:0/1321444298 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf7a78dfb0 con 0x55cf7a862db0 2024-09-21T03:41:15.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.625+0000 7fb2179b4640 1 --2- 172.21.15.38:0/1321444298 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb2000028f0 0x55cf7a848e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:41:15.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.625+0000 7fb2181b5640 1 --2- 172.21.15.38:0/1321444298 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cf7a862db0 0x55cf7a8631c0 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x55cf7a7ba5e0 tx=0x7fb210009ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8afeab1c7f2f65db server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:15.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.625+0000 7fb2161b1640 1 -- 172.21.15.38:0/1321444298 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb210009070 con 0x55cf7a862db0 2024-09-21T03:41:15.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.625+0000 7fb2161b1640 1 -- 172.21.15.38:0/1321444298 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb210009210 con 0x55cf7a862db0 2024-09-21T03:41:15.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.625+0000 7fb2161b1640 1 -- 172.21.15.38:0/1321444298 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2100077e0 con 0x55cf7a862db0 2024-09-21T03:41:15.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.626+0000 7fb21defaf40 1 -- 172.21.15.38:0/1321444298 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cf7a862db0 msgr2=0x55cf7a8631c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:15.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.626+0000 7fb21defaf40 1 --2- 172.21.15.38:0/1321444298 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cf7a862db0 0x55cf7a8631c0 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x55cf7a7ba5e0 tx=0x7fb210009ae0 comp rx=0 tx=0).stop 2024-09-21T03:41:15.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.626+0000 7fb21defaf40 1 -- 172.21.15.38:0/1321444298 shutdown_connections 2024-09-21T03:41:15.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.626+0000 7fb21defaf40 1 --2- 172.21.15.38:0/1321444298 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cf7a862db0 0x55cf7a8631c0 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:15.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.626+0000 7fb21defaf40 1 --2- 172.21.15.38:0/1321444298 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cf7a78d660 0x55cf7a78da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:15.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.626+0000 7fb21defaf40 1 --2- 172.21.15.38:0/1321444298 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb2000028f0 0x55cf7a848e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:15.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.626+0000 7fb21defaf40 1 -- 172.21.15.38:0/1321444298 >> 172.21.15.38:0/1321444298 conn(0x55cf7a6b0ac0 msgr2=0x55cf7a85ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:15.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.626+0000 7fb21defaf40 1 -- 172.21.15.38:0/1321444298 shutdown_connections 2024-09-21T03:41:15.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.626+0000 7fb21defaf40 1 -- 172.21.15.38:0/1321444298 wait complete. 2024-09-21T03:41:15.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.627+0000 7fb21defaf40 1 Processor -- start 2024-09-21T03:41:15.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.627+0000 7fb21defaf40 1 -- start start 2024-09-21T03:41:15.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.628+0000 7fb21defaf40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cf7a78d660 0x55cf7a843ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:15.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.628+0000 7fb21defaf40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cf7a862db0 0x55cf7a85f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:15.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.628+0000 7fb21defaf40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb2000028f0 0x55cf7a85fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:15.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.628+0000 7fb21defaf40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55cf7a84d950 con 0x55cf7a862db0 2024-09-21T03:41:15.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.628+0000 7fb21defaf40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55cf7a7d8980 con 0x55cf7a78d660 2024-09-21T03:41:15.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.628+0000 7fb21defaf40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55cf7a7bb860 con 0x7fb2000028f0 2024-09-21T03:41:15.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.628+0000 7fb2171b3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cf7a862db0 0x55cf7a85f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:15.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.628+0000 7fb2171b3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cf7a862db0 0x55cf7a85f850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46932/0 (socket says 172.21.15.38:46932) 2024-09-21T03:41:15.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.628+0000 7fb2171b3640 1 -- 172.21.15.38:0/2189246990 learned_addr learned my addr 172.21.15.38:0/2189246990 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:41:15.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.629+0000 7fb2179b4640 1 --2- 172.21.15.38:0/2189246990 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cf7a78d660 0x55cf7a843ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:15.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.629+0000 7fb2181b5640 1 --2- 172.21.15.38:0/2189246990 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb2000028f0 0x55cf7a85fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:15.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.629+0000 7fb2171b3640 1 -- 172.21.15.38:0/2189246990 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb2000028f0 msgr2=0x55cf7a85fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:15.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.629+0000 7fb2171b3640 1 --2- 172.21.15.38:0/2189246990 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb2000028f0 0x55cf7a85fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:15.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.629+0000 7fb2171b3640 1 -- 172.21.15.38:0/2189246990 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cf7a78d660 msgr2=0x55cf7a843ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:15.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.629+0000 7fb2171b3640 1 --2- 172.21.15.38:0/2189246990 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cf7a78d660 0x55cf7a843ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:15.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.629+0000 7fb2171b3640 1 -- 172.21.15.38:0/2189246990 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf7a8446e0 con 0x55cf7a862db0 2024-09-21T03:41:15.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.629+0000 7fb2181b5640 1 --2- 172.21.15.38:0/2189246990 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb2000028f0 0x55cf7a85fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:41:15.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.629+0000 7fb2171b3640 1 --2- 172.21.15.38:0/2189246990 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cf7a862db0 0x55cf7a85f850 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7fb20c002040 tx=0x7fb20c0559d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:15.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.629+0000 7fb2151af640 1 -- 172.21.15.38:0/2189246990 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb20c004e30 con 0x55cf7a862db0 2024-09-21T03:41:15.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.629+0000 7fb2151af640 1 -- 172.21.15.38:0/2189246990 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb20c04da90 con 0x55cf7a862db0 2024-09-21T03:41:15.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.629+0000 7fb21defaf40 1 -- 172.21.15.38:0/2189246990 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55cf7a863900 con 0x55cf7a862db0 2024-09-21T03:41:15.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.629+0000 7fb2151af640 1 -- 172.21.15.38:0/2189246990 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb20c055e20 con 0x55cf7a862db0 2024-09-21T03:41:15.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.629+0000 7fb21defaf40 1 -- 172.21.15.38:0/2189246990 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf7a7bde40 con 0x55cf7a862db0 2024-09-21T03:41:15.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.631+0000 7fb2151af640 1 -- 172.21.15.38:0/2189246990 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb20c05e440 con 0x55cf7a862db0 2024-09-21T03:41:15.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.631+0000 7fb2151af640 1 --2- 172.21.15.38:0/2189246990 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb204077570 0x7fb204079a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:15.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.632+0000 7fb2151af640 1 -- 172.21.15.38:0/2189246990 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 7078+0+0 (secure 0 0 0) 0x7fb20c0de3d0 con 0x55cf7a862db0 2024-09-21T03:41:15.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.632+0000 7fb2179b4640 1 --2- 172.21.15.38:0/2189246990 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb204077570 0x7fb204079a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:15.635 INFO:teuthology.orchestra.run.smithi038.stdout:Importing pool 2024-09-21T03:41:15.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.632+0000 7fb21defaf40 1 --2- 172.21.15.38:0/2189246990 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x55cf7a845860 0x55cf7a90b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:15.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.632+0000 7fb21defaf40 1 -- 172.21.15.38:0/2189246990 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55cf7a845c70 con 0x55cf7a845860 2024-09-21T03:41:15.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.632+0000 7fb2181b5640 1 --2- 172.21.15.38:0/2189246990 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x55cf7a845860 0x55cf7a90b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:15.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.632+0000 7fb2179b4640 1 --2- 172.21.15.38:0/2189246990 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb204077570 0x7fb204079a80 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fb208000c00 tx=0x7fb208003040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:15.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.633+0000 7fb2181b5640 1 --2- 172.21.15.38:0/2189246990 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x55cf7a845860 0x55cf7a90b330 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:15.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.634+0000 7fb2181b5640 1 -- 172.21.15.38:0/2189246990 <== osd.7 v2:172.21.15.195:6818/1606106342 1 ==== osd_op_reply(1 601.00000000 [create] v105'257 uv231 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb210007700 con 0x55cf7a845860 2024-09-21T03:41:15.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.634+0000 7fb21defaf40 1 -- 172.21.15.38:0/2189246990 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e105) -- 0x55cf7a90b870 con 0x55cf7a845860 2024-09-21T03:41:15.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.636+0000 7fb2181b5640 1 -- 172.21.15.38:0/2189246990 <== osd.7 v2:172.21.15.195:6818/1606106342 2 ==== osd_op_reply(2 601.00000000 [delete] v105'258 uv258 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb210007700 con 0x55cf7a845860 2024-09-21T03:41:15.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.636+0000 7fb21defaf40 1 -- 172.21.15.38:0/2189246990 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55cf7a90bc10 con 0x55cf7a845860 2024-09-21T03:41:15.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.637+0000 7fb2181b5640 1 -- 172.21.15.38:0/2189246990 <== osd.7 v2:172.21.15.195:6818/1606106342 3 ==== osd_op_reply(3 601.00000000 [create] v105'259 uv259 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb210007700 con 0x55cf7a845860 2024-09-21T03:41:15.640 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-09-21T03:41:15.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.638+0000 7fb21defaf40 1 -- 172.21.15.38:0/2189246990 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55cf7a90cfc0 con 0x55cf7a845860 2024-09-21T03:41:15.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.639+0000 7fb2181b5640 1 -- 172.21.15.38:0/2189246990 <== osd.7 v2:172.21.15.195:6818/1606106342 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v105'260 uv260 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb210007700 con 0x55cf7a845860 2024-09-21T03:41:15.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.639+0000 7fb21defaf40 1 -- 172.21.15.38:0/2189246990 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55cf7a90d360 con 0x55cf7a845860 2024-09-21T03:41:15.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.641+0000 7fb2181b5640 1 -- 172.21.15.38:0/2189246990 <== osd.7 v2:172.21.15.195:6818/1606106342 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v105'261 uv261 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb210007700 con 0x55cf7a845860 2024-09-21T03:41:15.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.641+0000 7fb21defaf40 1 -- 172.21.15.38:0/2189246990 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55cf7a90bfb0 con 0x55cf7a845860 2024-09-21T03:41:15.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.642+0000 7fb2181b5640 1 -- 172.21.15.38:0/2189246990 <== osd.7 v2:172.21.15.195:6818/1606106342 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v105'262 uv262 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb210007700 con 0x55cf7a845860 2024-09-21T03:41:15.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.642+0000 7fb21defaf40 1 -- 172.21.15.38:0/2189246990 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55cf7a90c350 con 0x55cf7a845860 2024-09-21T03:41:15.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.643+0000 7fb2181b5640 1 -- 172.21.15.38:0/2189246990 <== osd.7 v2:172.21.15.195:6818/1606106342 7 ==== osd_op_reply(7 602.00000000 [create] v105'263 uv237 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb210007700 con 0x55cf7a845860 2024-09-21T03:41:15.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.643+0000 7fb21defaf40 1 -- 172.21.15.38:0/2189246990 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e105) -- 0x55cf7a90c740 con 0x55cf7a845860 2024-09-21T03:41:15.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.645+0000 7fb2181b5640 1 -- 172.21.15.38:0/2189246990 <== osd.7 v2:172.21.15.195:6818/1606106342 8 ==== osd_op_reply(8 602.00000000 [delete] v105'264 uv264 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb210007700 con 0x55cf7a845860 2024-09-21T03:41:15.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.645+0000 7fb21defaf40 1 -- 172.21.15.38:0/2189246990 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55cf7a90c740 con 0x55cf7a845860 2024-09-21T03:41:15.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.646+0000 7fb2181b5640 1 -- 172.21.15.38:0/2189246990 <== osd.7 v2:172.21.15.195:6818/1606106342 9 ==== osd_op_reply(9 602.00000000 [create] v105'265 uv265 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb210007700 con 0x55cf7a845860 2024-09-21T03:41:15.648 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-09-21T03:41:15.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.646+0000 7fb21defaf40 1 -- 172.21.15.38:0/2189246990 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55cf7a90c6f0 con 0x55cf7a845860 2024-09-21T03:41:15.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.648+0000 7fb2181b5640 1 -- 172.21.15.38:0/2189246990 <== osd.7 v2:172.21.15.195:6818/1606106342 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v105'266 uv266 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb210007700 con 0x55cf7a845860 2024-09-21T03:41:15.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.648+0000 7fb21defaf40 1 -- 172.21.15.38:0/2189246990 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55cf7a90c6f0 con 0x55cf7a845860 2024-09-21T03:41:15.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.649+0000 7fb2181b5640 1 -- 172.21.15.38:0/2189246990 <== osd.7 v2:172.21.15.195:6818/1606106342 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v105'267 uv267 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb210007700 con 0x55cf7a845860 2024-09-21T03:41:15.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.649+0000 7fb21defaf40 1 -- 172.21.15.38:0/2189246990 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55cf7a90c6f0 con 0x55cf7a845860 2024-09-21T03:41:15.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.651+0000 7fb2181b5640 1 -- 172.21.15.38:0/2189246990 <== osd.7 v2:172.21.15.195:6818/1606106342 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v105'268 uv268 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb210007700 con 0x55cf7a845860 2024-09-21T03:41:15.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.651+0000 7fb21defaf40 1 -- 172.21.15.38:0/2189246990 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:13 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55cf7a90c6f0 con 0x55cf7a845860 2024-09-21T03:41:15.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.652+0000 7fb2181b5640 1 -- 172.21.15.38:0/2189246990 <== osd.7 v2:172.21.15.195:6818/1606106342 13 ==== osd_op_reply(13 10000000000.00000000 [create] v105'269 uv244 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7fb210007700 con 0x55cf7a845860 2024-09-21T03:41:15.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.652+0000 7fb21defaf40 1 -- 172.21.15.38:0/2189246990 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:14 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e105) -- 0x55cf7a90c6f0 con 0x55cf7a845860 2024-09-21T03:41:15.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.654+0000 7fb2181b5640 1 -- 172.21.15.38:0/2189246990 <== osd.7 v2:172.21.15.195:6818/1606106342 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v105'270 uv270 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fb210007700 con 0x55cf7a845860 2024-09-21T03:41:15.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.654+0000 7fb21defaf40 1 -- 172.21.15.38:0/2189246990 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:15 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55cf7a90c6f0 con 0x55cf7a845860 2024-09-21T03:41:15.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.655+0000 7fb2181b5640 1 -- 172.21.15.38:0/2189246990 <== osd.7 v2:172.21.15.195:6818/1606106342 15 ==== osd_op_reply(15 10000000000.00000000 [create] v105'271 uv271 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fb210007700 con 0x55cf7a845860 2024-09-21T03:41:15.657 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-09-21T03:41:15.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.655+0000 7fb21defaf40 1 -- 172.21.15.38:0/2189246990 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:16 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55cf7a90c6f0 con 0x55cf7a845860 2024-09-21T03:41:15.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.657+0000 7fb2181b5640 1 -- 172.21.15.38:0/2189246990 <== osd.7 v2:172.21.15.195:6818/1606106342 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v105'272 uv272 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fb210007700 con 0x55cf7a845860 2024-09-21T03:41:15.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.657+0000 7fb21defaf40 1 -- 172.21.15.38:0/2189246990 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:17 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55cf7a90c6f0 con 0x55cf7a845860 2024-09-21T03:41:15.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.658+0000 7fb2181b5640 1 -- 172.21.15.38:0/2189246990 <== osd.7 v2:172.21.15.195:6818/1606106342 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v105'273 uv273 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fb210007700 con 0x55cf7a845860 2024-09-21T03:41:15.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.658+0000 7fb21defaf40 1 -- 172.21.15.38:0/2189246990 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:18 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55cf7a90c6f0 con 0x55cf7a845860 2024-09-21T03:41:15.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.660+0000 7fb2181b5640 1 -- 172.21.15.38:0/2189246990 <== osd.7 v2:172.21.15.195:6818/1606106342 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v105'274 uv274 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fb210007700 con 0x55cf7a845860 2024-09-21T03:41:15.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.660+0000 7fb21defaf40 1 -- 172.21.15.38:0/2189246990 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:19 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=585b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55cf7a90c6f0 con 0x55cf7a845860 2024-09-21T03:41:15.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.661+0000 7fb2181b5640 1 -- 172.21.15.38:0/2189246990 <== osd.7 v2:172.21.15.195:6818/1606106342 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v105'275 uv275 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fb210007700 con 0x55cf7a845860 2024-09-21T03:41:15.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.661+0000 7fb21defaf40 1 -- 172.21.15.38:0/2189246990 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:20 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55cf7a90c6f0 con 0x55cf7a845860 2024-09-21T03:41:15.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.662+0000 7fb2181b5640 1 -- 172.21.15.38:0/2189246990 <== osd.7 v2:172.21.15.195:6818/1606106342 20 ==== osd_op_reply(20 600.00000000 [create] v105'276 uv250 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb210007700 con 0x55cf7a845860 2024-09-21T03:41:15.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.663+0000 7fb21defaf40 1 -- 172.21.15.38:0/2189246990 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:21 13.10 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e105) -- 0x55cf7a90c6f0 con 0x55cf7a845860 2024-09-21T03:41:15.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.664+0000 7fb2181b5640 1 -- 172.21.15.38:0/2189246990 <== osd.7 v2:172.21.15.195:6818/1606106342 21 ==== osd_op_reply(21 600.00000000 [delete] v105'277 uv277 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb210007700 con 0x55cf7a845860 2024-09-21T03:41:15.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.664+0000 7fb21defaf40 1 -- 172.21.15.38:0/2189246990 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:22 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55cf7a90c6f0 con 0x55cf7a845860 2024-09-21T03:41:15.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.666+0000 7fb2181b5640 1 -- 172.21.15.38:0/2189246990 <== osd.7 v2:172.21.15.195:6818/1606106342 22 ==== osd_op_reply(22 600.00000000 [create] v105'278 uv278 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb210007700 con 0x55cf7a845860 2024-09-21T03:41:15.668 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-09-21T03:41:15.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.666+0000 7fb21defaf40 1 -- 172.21.15.38:0/2189246990 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:23 13.10 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55cf7a90c6f0 con 0x55cf7a845860 2024-09-21T03:41:15.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.667+0000 7fb2181b5640 1 -- 172.21.15.38:0/2189246990 <== osd.7 v2:172.21.15.195:6818/1606106342 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v105'279 uv279 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb210007700 con 0x55cf7a845860 2024-09-21T03:41:15.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.667+0000 7fb21defaf40 1 -- 172.21.15.38:0/2189246990 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:24 13.10 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55cf7a90c6f0 con 0x55cf7a845860 2024-09-21T03:41:15.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.668+0000 7fb2181b5640 1 -- 172.21.15.38:0/2189246990 <== osd.7 v2:172.21.15.195:6818/1606106342 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v105'280 uv280 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb210007700 con 0x55cf7a845860 2024-09-21T03:41:15.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.668+0000 7fb21defaf40 1 -- 172.21.15.38:0/2189246990 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:25 13.10 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55cf7a90c6f0 con 0x55cf7a845860 2024-09-21T03:41:15.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.670+0000 7fb2181b5640 1 -- 172.21.15.38:0/2189246990 <== osd.7 v2:172.21.15.195:6818/1606106342 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v105'281 uv281 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb210007700 con 0x55cf7a845860 2024-09-21T03:41:15.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.670+0000 7fb21defaf40 1 -- 172.21.15.38:0/2189246990 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:26 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55cf7a90c6f0 con 0x55cf7a845860 2024-09-21T03:41:15.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.671+0000 7fb2181b5640 1 -- 172.21.15.38:0/2189246990 <== osd.7 v2:172.21.15.195:6818/1606106342 26 ==== osd_op_reply(26 603.00000000 [create] v105'282 uv256 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb210007700 con 0x55cf7a845860 2024-09-21T03:41:15.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.671+0000 7fb21defaf40 1 -- 172.21.15.38:0/2189246990 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:27 13.10 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e105) -- 0x55cf7a90c6f0 con 0x55cf7a845860 2024-09-21T03:41:15.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.672+0000 7fb2181b5640 1 -- 172.21.15.38:0/2189246990 <== osd.7 v2:172.21.15.195:6818/1606106342 27 ==== osd_op_reply(27 603.00000000 [delete] v105'283 uv283 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb210007700 con 0x55cf7a845860 2024-09-21T03:41:15.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.672+0000 7fb21defaf40 1 -- 172.21.15.38:0/2189246990 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:28 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55cf7a90c6f0 con 0x55cf7a845860 2024-09-21T03:41:15.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.674+0000 7fb2181b5640 1 -- 172.21.15.38:0/2189246990 <== osd.7 v2:172.21.15.195:6818/1606106342 28 ==== osd_op_reply(28 603.00000000 [create] v105'284 uv284 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb210007700 con 0x55cf7a845860 2024-09-21T03:41:15.676 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-09-21T03:41:15.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.674+0000 7fb21defaf40 1 -- 172.21.15.38:0/2189246990 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:29 13.10 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55cf7a90c6f0 con 0x55cf7a845860 2024-09-21T03:41:15.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.675+0000 7fb2181b5640 1 -- 172.21.15.38:0/2189246990 <== osd.7 v2:172.21.15.195:6818/1606106342 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v105'285 uv285 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb210007700 con 0x55cf7a845860 2024-09-21T03:41:15.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.675+0000 7fb21defaf40 1 -- 172.21.15.38:0/2189246990 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:30 13.10 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55cf7a90c6f0 con 0x55cf7a845860 2024-09-21T03:41:15.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.676+0000 7fb2181b5640 1 -- 172.21.15.38:0/2189246990 <== osd.7 v2:172.21.15.195:6818/1606106342 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v105'286 uv286 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb210007700 con 0x55cf7a845860 2024-09-21T03:41:15.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.676+0000 7fb21defaf40 1 -- 172.21.15.38:0/2189246990 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:31 13.10 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55cf7a90c6f0 con 0x55cf7a845860 2024-09-21T03:41:15.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.678+0000 7fb2181b5640 1 -- 172.21.15.38:0/2189246990 <== osd.7 v2:172.21.15.195:6818/1606106342 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v105'287 uv287 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb210007700 con 0x55cf7a845860 2024-09-21T03:41:15.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.678+0000 7fb21defaf40 1 --2- 172.21.15.38:0/2189246990 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x55cf7a90cb00 0x55cf7a90fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:15.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.678+0000 7fb21defaf40 1 -- 172.21.15.38:0/2189246990 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55cf7a90c6f0 con 0x55cf7a90cb00 2024-09-21T03:41:15.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.678+0000 7fb2179b4640 1 --2- 172.21.15.38:0/2189246990 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x55cf7a90cb00 0x55cf7a90fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:15.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.679+0000 7fb2179b4640 1 --2- 172.21.15.38:0/2189246990 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x55cf7a90cb00 0x55cf7a90fc20 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:15.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.680+0000 7fb2179b4640 1 -- 172.21.15.38:0/2189246990 <== osd.6 v2:172.21.15.195:6826/4055825675 1 ==== osd_op_reply(32 1.00000000.inode [create] v105'197 uv180 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7fb20800b040 con 0x55cf7a90cb00 2024-09-21T03:41:15.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.680+0000 7fb21defaf40 1 -- 172.21.15.38:0/2189246990 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e105) -- 0x55cf7a910160 con 0x55cf7a90cb00 2024-09-21T03:41:15.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.681+0000 7fb2179b4640 1 -- 172.21.15.38:0/2189246990 <== osd.6 v2:172.21.15.195:6826/4055825675 2 ==== osd_op_reply(33 1.00000000.inode [delete] v105'198 uv198 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fb20800b040 con 0x55cf7a90cb00 2024-09-21T03:41:15.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.682+0000 7fb21defaf40 1 -- 172.21.15.38:0/2189246990 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55cf7a910500 con 0x55cf7a90cb00 2024-09-21T03:41:15.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.683+0000 7fb2179b4640 1 -- 172.21.15.38:0/2189246990 <== osd.6 v2:172.21.15.195:6826/4055825675 3 ==== osd_op_reply(34 1.00000000.inode [create] v105'199 uv199 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fb20800b040 con 0x55cf7a90cb00 2024-09-21T03:41:15.685 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-09-21T03:41:15.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.683+0000 7fb21defaf40 1 -- 172.21.15.38:0/2189246990 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55cf7a9108a0 con 0x55cf7a90cb00 2024-09-21T03:41:15.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.684+0000 7fb2179b4640 1 -- 172.21.15.38:0/2189246990 <== osd.6 v2:172.21.15.195:6826/4055825675 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v105'200 uv200 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fb20800b040 con 0x55cf7a90cb00 2024-09-21T03:41:15.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.684+0000 7fb21defaf40 1 -- 172.21.15.38:0/2189246990 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55cf7a910c40 con 0x55cf7a90cb00 2024-09-21T03:41:15.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.685+0000 7fb2179b4640 1 -- 172.21.15.38:0/2189246990 <== osd.6 v2:172.21.15.195:6826/4055825675 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v105'201 uv201 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fb20800b040 con 0x55cf7a90cb00 2024-09-21T03:41:15.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.685+0000 7fb21defaf40 1 -- 172.21.15.38:0/2189246990 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55cf7a910fe0 con 0x55cf7a90cb00 2024-09-21T03:41:15.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.687+0000 7fb2179b4640 1 -- 172.21.15.38:0/2189246990 <== osd.6 v2:172.21.15.195:6826/4055825675 6 ==== osd_op_reply(37 200.00000000 [create] v105'202 uv185 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb20800b040 con 0x55cf7a90cb00 2024-09-21T03:41:15.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.687+0000 7fb21defaf40 1 -- 172.21.15.38:0/2189246990 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e105) -- 0x55cf7a911380 con 0x55cf7a90cb00 2024-09-21T03:41:15.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.688+0000 7fb2179b4640 1 -- 172.21.15.38:0/2189246990 <== osd.6 v2:172.21.15.195:6826/4055825675 7 ==== osd_op_reply(38 200.00000000 [delete] v105'203 uv203 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb20800b040 con 0x55cf7a90cb00 2024-09-21T03:41:15.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.688+0000 7fb21defaf40 1 -- 172.21.15.38:0/2189246990 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55cf7a911720 con 0x55cf7a90cb00 2024-09-21T03:41:15.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.689+0000 7fb2179b4640 1 -- 172.21.15.38:0/2189246990 <== osd.6 v2:172.21.15.195:6826/4055825675 8 ==== osd_op_reply(39 200.00000000 [create] v105'204 uv204 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb20800b040 con 0x55cf7a90cb00 2024-09-21T03:41:15.691 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-09-21T03:41:15.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.689+0000 7fb21defaf40 1 -- 172.21.15.38:0/2189246990 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55cf7a844f80 con 0x55cf7a90cb00 2024-09-21T03:41:15.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.691+0000 7fb2179b4640 1 -- 172.21.15.38:0/2189246990 <== osd.6 v2:172.21.15.195:6826/4055825675 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v105'205 uv205 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb20800b040 con 0x55cf7a90cb00 2024-09-21T03:41:15.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.691+0000 7fb21defaf40 1 -- 172.21.15.38:0/2189246990 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55cf7a844f80 con 0x55cf7a90cb00 2024-09-21T03:41:15.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.692+0000 7fb2179b4640 1 -- 172.21.15.38:0/2189246990 <== osd.6 v2:172.21.15.195:6826/4055825675 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v105'206 uv206 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb20800b040 con 0x55cf7a90cb00 2024-09-21T03:41:15.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.692+0000 7fb21defaf40 1 -- 172.21.15.38:0/2189246990 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:42 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55cf7a844f80 con 0x55cf7a90cb00 2024-09-21T03:41:15.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.693+0000 7fb2179b4640 1 -- 172.21.15.38:0/2189246990 <== osd.6 v2:172.21.15.195:6826/4055825675 11 ==== osd_op_reply(42 200.00000001 [create] v105'207 uv190 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb20800b040 con 0x55cf7a90cb00 2024-09-21T03:41:15.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.693+0000 7fb21defaf40 1 -- 172.21.15.38:0/2189246990 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:43 13.14 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e105) -- 0x55cf7a844f80 con 0x55cf7a90cb00 2024-09-21T03:41:15.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.695+0000 7fb2179b4640 1 -- 172.21.15.38:0/2189246990 <== osd.6 v2:172.21.15.195:6826/4055825675 12 ==== osd_op_reply(43 200.00000001 [delete] v105'208 uv208 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb20800b040 con 0x55cf7a90cb00 2024-09-21T03:41:15.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.695+0000 7fb21defaf40 1 -- 172.21.15.38:0/2189246990 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:44 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55cf7a844f80 con 0x55cf7a90cb00 2024-09-21T03:41:15.698 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-09-21T03:41:15.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.696+0000 7fb2179b4640 1 -- 172.21.15.38:0/2189246990 <== osd.6 v2:172.21.15.195:6826/4055825675 13 ==== osd_op_reply(44 200.00000001 [create] v105'209 uv209 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb20800b040 con 0x55cf7a90cb00 2024-09-21T03:41:15.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.696+0000 7fb21defaf40 1 -- 172.21.15.38:0/2189246990 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:45 13.14 13:2e2fa760:::200.00000001:head [write 0~17490 in=17490b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55cf7a844f80 con 0x55cf7a90cb00 2024-09-21T03:41:15.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.697+0000 7fb2179b4640 1 -- 172.21.15.38:0/2189246990 <== osd.6 v2:172.21.15.195:6826/4055825675 14 ==== osd_op_reply(45 200.00000001 [write 0~17490] v105'210 uv210 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb20800b040 con 0x55cf7a90cb00 2024-09-21T03:41:15.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.697+0000 7fb21defaf40 1 -- 172.21.15.38:0/2189246990 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:46 13.14 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55cf7a844f80 con 0x55cf7a90cb00 2024-09-21T03:41:15.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.699+0000 7fb2179b4640 1 -- 172.21.15.38:0/2189246990 <== osd.6 v2:172.21.15.195:6826/4055825675 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v105'211 uv211 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb20800b040 con 0x55cf7a90cb00 2024-09-21T03:41:15.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.699+0000 7fb21defaf40 1 -- 172.21.15.38:0/2189246990 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:47 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55cf7a844f80 con 0x55cf7a90cb00 2024-09-21T03:41:15.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.700+0000 7fb2179b4640 1 -- 172.21.15.38:0/2189246990 <== osd.6 v2:172.21.15.195:6826/4055825675 16 ==== osd_op_reply(47 606.00000000 [create] v105'212 uv196 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb20800b040 con 0x55cf7a90cb00 2024-09-21T03:41:15.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.700+0000 7fb21defaf40 1 -- 172.21.15.38:0/2189246990 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:48 13.14 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e105) -- 0x55cf7a844f80 con 0x55cf7a90cb00 2024-09-21T03:41:15.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.702+0000 7fb2179b4640 1 -- 172.21.15.38:0/2189246990 <== osd.6 v2:172.21.15.195:6826/4055825675 17 ==== osd_op_reply(48 606.00000000 [delete] v105'213 uv213 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb20800b040 con 0x55cf7a90cb00 2024-09-21T03:41:15.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.702+0000 7fb21defaf40 1 -- 172.21.15.38:0/2189246990 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:49 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55cf7a844f80 con 0x55cf7a90cb00 2024-09-21T03:41:15.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.703+0000 7fb2179b4640 1 -- 172.21.15.38:0/2189246990 <== osd.6 v2:172.21.15.195:6826/4055825675 18 ==== osd_op_reply(49 606.00000000 [create] v105'214 uv214 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb20800b040 con 0x55cf7a90cb00 2024-09-21T03:41:15.705 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-09-21T03:41:15.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.703+0000 7fb21defaf40 1 -- 172.21.15.38:0/2189246990 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:50 13.14 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55cf7a844f80 con 0x55cf7a90cb00 2024-09-21T03:41:15.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.704+0000 7fb2179b4640 1 -- 172.21.15.38:0/2189246990 <== osd.6 v2:172.21.15.195:6826/4055825675 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v105'215 uv215 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb20800b040 con 0x55cf7a90cb00 2024-09-21T03:41:15.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.704+0000 7fb21defaf40 1 -- 172.21.15.38:0/2189246990 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:51 13.14 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55cf7a844f80 con 0x55cf7a90cb00 2024-09-21T03:41:15.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.706+0000 7fb2179b4640 1 -- 172.21.15.38:0/2189246990 <== osd.6 v2:172.21.15.195:6826/4055825675 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v105'216 uv216 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb20800b040 con 0x55cf7a90cb00 2024-09-21T03:41:15.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.706+0000 7fb21defaf40 1 -- 172.21.15.38:0/2189246990 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:52 13.14 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55cf7a844f80 con 0x55cf7a90cb00 2024-09-21T03:41:15.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.707+0000 7fb2179b4640 1 -- 172.21.15.38:0/2189246990 <== osd.6 v2:172.21.15.195:6826/4055825675 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v105'217 uv217 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb20800b040 con 0x55cf7a90cb00 2024-09-21T03:41:15.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.707+0000 7fb21defaf40 1 --2- 172.21.15.38:0/2189246990 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x55cf7a9120e0 0x55cf7a914500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:15.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.707+0000 7fb21defaf40 1 -- 172.21.15.38:0/2189246990 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:53 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55cf7a914a40 con 0x55cf7a9120e0 2024-09-21T03:41:15.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.707+0000 7fb2171b3640 1 --2- 172.21.15.38:0/2189246990 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x55cf7a9120e0 0x55cf7a914500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:15.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.708+0000 7fb2171b3640 1 --2- 172.21.15.38:0/2189246990 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x55cf7a9120e0 0x55cf7a914500 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:15.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.709+0000 7fb2171b3640 1 -- 172.21.15.38:0/2189246990 <== osd.0 v2:172.21.15.38:6800/2381639433 1 ==== osd_op_reply(53 607.00000000 [create] v105'51 uv50 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb20c06d070 con 0x55cf7a9120e0 2024-09-21T03:41:15.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.709+0000 7fb21defaf40 1 -- 172.21.15.38:0/2189246990 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:54 13.1c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e105) -- 0x55cf7a914de0 con 0x55cf7a9120e0 2024-09-21T03:41:15.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.710+0000 7fb2171b3640 1 -- 172.21.15.38:0/2189246990 <== osd.0 v2:172.21.15.38:6800/2381639433 2 ==== osd_op_reply(54 607.00000000 [delete] v105'52 uv52 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb20c06d070 con 0x55cf7a9120e0 2024-09-21T03:41:15.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.710+0000 7fb21defaf40 1 -- 172.21.15.38:0/2189246990 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:55 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55cf7a915290 con 0x55cf7a9120e0 2024-09-21T03:41:15.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.711+0000 7fb2171b3640 1 -- 172.21.15.38:0/2189246990 <== osd.0 v2:172.21.15.38:6800/2381639433 3 ==== osd_op_reply(55 607.00000000 [create] v105'53 uv53 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb20c06d070 con 0x55cf7a9120e0 2024-09-21T03:41:15.713 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-09-21T03:41:15.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.711+0000 7fb21defaf40 1 -- 172.21.15.38:0/2189246990 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:56 13.1c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55cf7a916b40 con 0x55cf7a9120e0 2024-09-21T03:41:15.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.712+0000 7fb2171b3640 1 -- 172.21.15.38:0/2189246990 <== osd.0 v2:172.21.15.38:6800/2381639433 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v105'54 uv54 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb20c06d070 con 0x55cf7a9120e0 2024-09-21T03:41:15.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.712+0000 7fb21defaf40 1 -- 172.21.15.38:0/2189246990 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:57 13.1c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55cf7a916ee0 con 0x55cf7a9120e0 2024-09-21T03:41:15.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.713+0000 7fb2171b3640 1 -- 172.21.15.38:0/2189246990 <== osd.0 v2:172.21.15.38:6800/2381639433 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v105'55 uv55 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb20c06d070 con 0x55cf7a9120e0 2024-09-21T03:41:15.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.713+0000 7fb21defaf40 1 -- 172.21.15.38:0/2189246990 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:58 13.1c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55cf7a915b30 con 0x55cf7a9120e0 2024-09-21T03:41:15.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.714+0000 7fb2171b3640 1 -- 172.21.15.38:0/2189246990 <== osd.0 v2:172.21.15.38:6800/2381639433 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v105'56 uv56 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb20c06d070 con 0x55cf7a9120e0 2024-09-21T03:41:15.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.714+0000 7fb21defaf40 1 --2- 172.21.15.38:0/2189246990 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x55cf7a916430 0x55cf7a919290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:15.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.714+0000 7fb21defaf40 1 -- 172.21.15.38:0/2189246990 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:59 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55cf7a9197d0 con 0x55cf7a916430 2024-09-21T03:41:15.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.714+0000 7fb2181b5640 1 --2- 172.21.15.38:0/2189246990 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x55cf7a916430 0x55cf7a919290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:15.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.715+0000 7fb2181b5640 1 --2- 172.21.15.38:0/2189246990 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x55cf7a916430 0x55cf7a919290 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:15.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.716+0000 7fb2181b5640 1 -- 172.21.15.38:0/2189246990 <== osd.3 v2:172.21.15.38:6808/1286810294 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v105'40 uv39 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7fb210007700 con 0x55cf7a916430 2024-09-21T03:41:15.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.716+0000 7fb21defaf40 1 -- 172.21.15.38:0/2189246990 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:60 13.6 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e105) -- 0x55cf7a9197d0 con 0x55cf7a916430 2024-09-21T03:41:15.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.717+0000 7fb2181b5640 1 -- 172.21.15.38:0/2189246990 <== osd.3 v2:172.21.15.38:6808/1286810294 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v105'41 uv41 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fb210007700 con 0x55cf7a916430 2024-09-21T03:41:15.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.717+0000 7fb21defaf40 1 -- 172.21.15.38:0/2189246990 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:61 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55cf7a919bc0 con 0x55cf7a916430 2024-09-21T03:41:15.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.718+0000 7fb2181b5640 1 -- 172.21.15.38:0/2189246990 <== osd.3 v2:172.21.15.38:6808/1286810294 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v105'42 uv42 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fb210007700 con 0x55cf7a916430 2024-09-21T03:41:15.720 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-09-21T03:41:15.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.718+0000 7fb21defaf40 1 -- 172.21.15.38:0/2189246990 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:62 13.6 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55cf7a919bc0 con 0x55cf7a916430 2024-09-21T03:41:15.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.719+0000 7fb2181b5640 1 -- 172.21.15.38:0/2189246990 <== osd.3 v2:172.21.15.38:6808/1286810294 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v105'43 uv43 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fb210007700 con 0x55cf7a916430 2024-09-21T03:41:15.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.719+0000 7fb21defaf40 1 -- 172.21.15.38:0/2189246990 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55cf7a919bc0 con 0x55cf7a9120e0 2024-09-21T03:41:15.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.720+0000 7fb2171b3640 1 -- 172.21.15.38:0/2189246990 <== osd.0 v2:172.21.15.38:6800/2381639433 7 ==== osd_op_reply(63 608.00000000 [create] v105'51 uv50 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb20c06d070 con 0x55cf7a9120e0 2024-09-21T03:41:15.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.720+0000 7fb21defaf40 1 -- 172.21.15.38:0/2189246990 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e105) -- 0x55cf7a919f60 con 0x55cf7a9120e0 2024-09-21T03:41:15.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.723+0000 7fb2171b3640 1 -- 172.21.15.38:0/2189246990 <== osd.0 v2:172.21.15.38:6800/2381639433 8 ==== osd_op_reply(64 608.00000000 [delete] v105'52 uv52 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb20c06d070 con 0x55cf7a9120e0 2024-09-21T03:41:15.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.723+0000 7fb21defaf40 1 -- 172.21.15.38:0/2189246990 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55cf7a919f60 con 0x55cf7a9120e0 2024-09-21T03:41:15.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.724+0000 7fb2171b3640 1 -- 172.21.15.38:0/2189246990 <== osd.0 v2:172.21.15.38:6800/2381639433 9 ==== osd_op_reply(65 608.00000000 [create] v105'53 uv53 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb20c06d070 con 0x55cf7a9120e0 2024-09-21T03:41:15.726 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-09-21T03:41:15.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.724+0000 7fb21defaf40 1 -- 172.21.15.38:0/2189246990 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55cf7a91af70 con 0x55cf7a9120e0 2024-09-21T03:41:15.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.726+0000 7fb2171b3640 1 -- 172.21.15.38:0/2189246990 <== osd.0 v2:172.21.15.38:6800/2381639433 10 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v105'54 uv54 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb20c06d070 con 0x55cf7a9120e0 2024-09-21T03:41:15.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.726+0000 7fb21defaf40 1 -- 172.21.15.38:0/2189246990 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55cf7a91af70 con 0x55cf7a9120e0 2024-09-21T03:41:15.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.727+0000 7fb2171b3640 1 -- 172.21.15.38:0/2189246990 <== osd.0 v2:172.21.15.38:6800/2381639433 11 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v105'55 uv55 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb20c06d070 con 0x55cf7a9120e0 2024-09-21T03:41:15.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.727+0000 7fb21defaf40 1 -- 172.21.15.38:0/2189246990 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55cf7a919f60 con 0x55cf7a9120e0 2024-09-21T03:41:15.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.729+0000 7fb2171b3640 1 -- 172.21.15.38:0/2189246990 <== osd.0 v2:172.21.15.38:6800/2381639433 12 ==== osd_op_reply(68 608.00000000 [omap-set-header] v105'56 uv56 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb20c06d070 con 0x55cf7a9120e0 2024-09-21T03:41:15.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.729+0000 7fb21defaf40 1 -- 172.21.15.38:0/2189246990 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55cf7a919f60 con 0x55cf7a916430 2024-09-21T03:41:15.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.730+0000 7fb2181b5640 1 -- 172.21.15.38:0/2189246990 <== osd.3 v2:172.21.15.38:6808/1286810294 5 ==== osd_op_reply(69 604.00000000 [create] v105'94 uv88 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb210007700 con 0x55cf7a916430 2024-09-21T03:41:15.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.730+0000 7fb21defaf40 1 -- 172.21.15.38:0/2189246990 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e105) -- 0x55cf7a919f60 con 0x55cf7a916430 2024-09-21T03:41:15.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.731+0000 7fb2181b5640 1 -- 172.21.15.38:0/2189246990 <== osd.3 v2:172.21.15.38:6808/1286810294 6 ==== osd_op_reply(70 604.00000000 [delete] v105'95 uv95 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb210007700 con 0x55cf7a916430 2024-09-21T03:41:15.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.731+0000 7fb21defaf40 1 -- 172.21.15.38:0/2189246990 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55cf7a919f60 con 0x55cf7a916430 2024-09-21T03:41:15.734 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-09-21T03:41:15.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.732+0000 7fb2181b5640 1 -- 172.21.15.38:0/2189246990 <== osd.3 v2:172.21.15.38:6808/1286810294 7 ==== osd_op_reply(71 604.00000000 [create] v105'96 uv96 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb210007700 con 0x55cf7a916430 2024-09-21T03:41:15.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.732+0000 7fb21defaf40 1 -- 172.21.15.38:0/2189246990 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55cf7a91af70 con 0x55cf7a916430 2024-09-21T03:41:15.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.733+0000 7fb2181b5640 1 -- 172.21.15.38:0/2189246990 <== osd.3 v2:172.21.15.38:6808/1286810294 8 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v105'97 uv97 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb210007700 con 0x55cf7a916430 2024-09-21T03:41:15.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.733+0000 7fb21defaf40 1 -- 172.21.15.38:0/2189246990 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55cf7a91af70 con 0x55cf7a916430 2024-09-21T03:41:15.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.734+0000 7fb2181b5640 1 -- 172.21.15.38:0/2189246990 <== osd.3 v2:172.21.15.38:6808/1286810294 9 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v105'98 uv98 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb210007700 con 0x55cf7a916430 2024-09-21T03:41:15.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.734+0000 7fb21defaf40 1 -- 172.21.15.38:0/2189246990 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55cf7a919f60 con 0x55cf7a916430 2024-09-21T03:41:15.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.735+0000 7fb2181b5640 1 -- 172.21.15.38:0/2189246990 <== osd.3 v2:172.21.15.38:6808/1286810294 10 ==== osd_op_reply(74 604.00000000 [omap-set-header] v105'99 uv99 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb210007700 con 0x55cf7a916430 2024-09-21T03:41:15.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.735+0000 7fb21defaf40 1 -- 172.21.15.38:0/2189246990 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55cf7a919f60 con 0x55cf7a916430 2024-09-21T03:41:15.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.736+0000 7fb2181b5640 1 -- 172.21.15.38:0/2189246990 <== osd.3 v2:172.21.15.38:6808/1286810294 11 ==== osd_op_reply(75 500.00000000 [create] v105'100 uv93 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb210007700 con 0x55cf7a916430 2024-09-21T03:41:15.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.737+0000 7fb21defaf40 1 -- 172.21.15.38:0/2189246990 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e105) -- 0x55cf7a919f60 con 0x55cf7a916430 2024-09-21T03:41:15.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.738+0000 7fb2181b5640 1 -- 172.21.15.38:0/2189246990 <== osd.3 v2:172.21.15.38:6808/1286810294 12 ==== osd_op_reply(76 500.00000000 [delete] v105'101 uv101 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb210007700 con 0x55cf7a916430 2024-09-21T03:41:15.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.738+0000 7fb21defaf40 1 -- 172.21.15.38:0/2189246990 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55cf7a919f60 con 0x55cf7a916430 2024-09-21T03:41:15.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.739+0000 7fb2181b5640 1 -- 172.21.15.38:0/2189246990 <== osd.3 v2:172.21.15.38:6808/1286810294 13 ==== osd_op_reply(77 500.00000000 [create] v105'102 uv102 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb210007700 con 0x55cf7a916430 2024-09-21T03:41:15.741 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-09-21T03:41:15.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.739+0000 7fb21defaf40 1 -- 172.21.15.38:0/2189246990 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55cf7a919f60 con 0x55cf7a916430 2024-09-21T03:41:15.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.740+0000 7fb2181b5640 1 -- 172.21.15.38:0/2189246990 <== osd.3 v2:172.21.15.38:6808/1286810294 14 ==== osd_op_reply(78 500.00000000 [write 0~90] v105'103 uv103 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb210007700 con 0x55cf7a916430 2024-09-21T03:41:15.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.740+0000 7fb21defaf40 1 -- 172.21.15.38:0/2189246990 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55cf7a919f60 con 0x55cf7a916430 2024-09-21T03:41:15.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.741+0000 7fb2181b5640 1 -- 172.21.15.38:0/2189246990 <== osd.3 v2:172.21.15.38:6808/1286810294 15 ==== osd_op_reply(79 500.00000000 [omap-set-header] v105'104 uv104 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb210007700 con 0x55cf7a916430 2024-09-21T03:41:15.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.741+0000 7fb21defaf40 1 -- 172.21.15.38:0/2189246990 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:80 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55cf7a919f60 con 0x55cf7a9120e0 2024-09-21T03:41:15.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.742+0000 7fb2171b3640 1 -- 172.21.15.38:0/2189246990 <== osd.0 v2:172.21.15.38:6800/2381639433 13 ==== osd_op_reply(80 mds_snaptable [create] v105'43 uv43 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fb20c06d070 con 0x55cf7a9120e0 2024-09-21T03:41:15.744 INFO:teuthology.orchestra.run.smithi038.stdout:Write #-9223372036854775808:00000000:::mds_snaptable:head# 2024-09-21T03:41:15.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.742+0000 7fb21defaf40 1 -- 172.21.15.38:0/2189246990 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:81 13.d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55cf7a919f60 con 0x55cf7a9120e0 2024-09-21T03:41:15.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.744+0000 7fb2171b3640 1 -- 172.21.15.38:0/2189246990 <== osd.0 v2:172.21.15.38:6800/2381639433 14 ==== osd_op_reply(81 mds_snaptable [write 0~70] v105'44 uv44 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fb20c06d070 con 0x55cf7a9120e0 2024-09-21T03:41:15.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.744+0000 7fb21defaf40 1 -- 172.21.15.38:0/2189246990 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:82 13.d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55cf7a919f60 con 0x55cf7a9120e0 2024-09-21T03:41:15.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.745+0000 7fb2171b3640 1 -- 172.21.15.38:0/2189246990 <== osd.0 v2:172.21.15.38:6800/2381639433 15 ==== osd_op_reply(82 mds_snaptable [omap-set-header] v105'45 uv45 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fb20c06d070 con 0x55cf7a9120e0 2024-09-21T03:41:15.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.745+0000 7fb21defaf40 1 --2- 172.21.15.38:0/2189246990 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x55cf7a91a620 0x55cf7a91aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:15.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.745+0000 7fb21defaf40 1 -- 172.21.15.38:0/2189246990 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:83 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55cf7a91d360 con 0x55cf7a91a620 2024-09-21T03:41:15.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.745+0000 7fb2179b4640 1 --2- 172.21.15.38:0/2189246990 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x55cf7a91a620 0x55cf7a91aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:15.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.746+0000 7fb2179b4640 1 --2- 172.21.15.38:0/2189246990 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x55cf7a91a620 0x55cf7a91aaf0 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:15.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.747+0000 7fb2179b4640 1 -- 172.21.15.38:0/2189246990 <== osd.1 v2:172.21.15.38:6816/1491818732 1 ==== osd_op_reply(83 605.00000000 [create] v105'51 uv50 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb20800b040 con 0x55cf7a91a620 2024-09-21T03:41:15.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.747+0000 7fb21defaf40 1 -- 172.21.15.38:0/2189246990 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:84 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e105) -- 0x55cf7a91d360 con 0x55cf7a91a620 2024-09-21T03:41:15.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.748+0000 7fb2179b4640 1 -- 172.21.15.38:0/2189246990 <== osd.1 v2:172.21.15.38:6816/1491818732 2 ==== osd_op_reply(84 605.00000000 [delete] v105'52 uv52 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb20800b040 con 0x55cf7a91a620 2024-09-21T03:41:15.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.748+0000 7fb21defaf40 1 -- 172.21.15.38:0/2189246990 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55cf7a91d360 con 0x55cf7a91a620 2024-09-21T03:41:15.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.749+0000 7fb2179b4640 1 -- 172.21.15.38:0/2189246990 <== osd.1 v2:172.21.15.38:6816/1491818732 3 ==== osd_op_reply(85 605.00000000 [create] v105'53 uv53 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb20800b040 con 0x55cf7a91a620 2024-09-21T03:41:15.751 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-09-21T03:41:15.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.749+0000 7fb21defaf40 1 -- 172.21.15.38:0/2189246990 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55cf7a91e760 con 0x55cf7a91a620 2024-09-21T03:41:15.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.751+0000 7fb2179b4640 1 -- 172.21.15.38:0/2189246990 <== osd.1 v2:172.21.15.38:6816/1491818732 4 ==== osd_op_reply(86 605.00000000 [setxattr (30)] v105'54 uv54 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb20800b040 con 0x55cf7a91a620 2024-09-21T03:41:15.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.751+0000 7fb21defaf40 1 -- 172.21.15.38:0/2189246990 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55cf7a91e760 con 0x55cf7a91a620 2024-09-21T03:41:15.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.752+0000 7fb2179b4640 1 -- 172.21.15.38:0/2189246990 <== osd.1 v2:172.21.15.38:6816/1491818732 5 ==== osd_op_reply(87 605.00000000 [setxattr (62)] v105'55 uv55 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb20800b040 con 0x55cf7a91a620 2024-09-21T03:41:15.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.752+0000 7fb21defaf40 1 -- 172.21.15.38:0/2189246990 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55cf7a91d750 con 0x55cf7a91a620 2024-09-21T03:41:15.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.753+0000 7fb2179b4640 1 -- 172.21.15.38:0/2189246990 <== osd.1 v2:172.21.15.38:6816/1491818732 6 ==== osd_op_reply(88 605.00000000 [omap-set-header] v105'56 uv56 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb20800b040 con 0x55cf7a91a620 2024-09-21T03:41:15.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.753+0000 7fb21defaf40 1 -- 172.21.15.38:0/2189246990 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:89 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55cf7a91d750 con 0x55cf7a91a620 2024-09-21T03:41:15.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.754+0000 7fb2179b4640 1 -- 172.21.15.38:0/2189246990 <== osd.1 v2:172.21.15.38:6816/1491818732 7 ==== osd_op_reply(89 mds0_inotable [create] v105'84 uv78 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7fb20800b040 con 0x55cf7a91a620 2024-09-21T03:41:15.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.754+0000 7fb21defaf40 1 -- 172.21.15.38:0/2189246990 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:90 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e105) -- 0x55cf7a91d750 con 0x55cf7a91a620 2024-09-21T03:41:15.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.755+0000 7fb2179b4640 1 -- 172.21.15.38:0/2189246990 <== osd.1 v2:172.21.15.38:6816/1491818732 8 ==== osd_op_reply(90 mds0_inotable [delete] v105'85 uv85 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fb20800b040 con 0x55cf7a91a620 2024-09-21T03:41:15.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.755+0000 7fb21defaf40 1 -- 172.21.15.38:0/2189246990 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55cf7a91d750 con 0x55cf7a91a620 2024-09-21T03:41:15.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.756+0000 7fb2179b4640 1 -- 172.21.15.38:0/2189246990 <== osd.1 v2:172.21.15.38:6816/1491818732 9 ==== osd_op_reply(91 mds0_inotable [create] v105'86 uv86 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fb20800b040 con 0x55cf7a91a620 2024-09-21T03:41:15.758 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-09-21T03:41:15.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.756+0000 7fb21defaf40 1 -- 172.21.15.38:0/2189246990 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55cf7a91d750 con 0x55cf7a91a620 2024-09-21T03:41:15.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.757+0000 7fb2179b4640 1 -- 172.21.15.38:0/2189246990 <== osd.1 v2:172.21.15.38:6816/1491818732 10 ==== osd_op_reply(92 mds0_inotable [write 0~34] v105'87 uv87 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fb20800b040 con 0x55cf7a91a620 2024-09-21T03:41:15.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.757+0000 7fb21defaf40 1 -- 172.21.15.38:0/2189246990 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55cf7a91d750 con 0x55cf7a91a620 2024-09-21T03:41:15.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.758+0000 7fb2179b4640 1 -- 172.21.15.38:0/2189246990 <== osd.1 v2:172.21.15.38:6816/1491818732 11 ==== osd_op_reply(93 mds0_inotable [omap-set-header] v105'88 uv88 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fb20800b040 con 0x55cf7a91a620 2024-09-21T03:41:15.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.758+0000 7fb21defaf40 1 -- 172.21.15.38:0/2189246990 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:94 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55cf7a91d750 con 0x55cf7a91a620 2024-09-21T03:41:15.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.759+0000 7fb2179b4640 1 -- 172.21.15.38:0/2189246990 <== osd.1 v2:172.21.15.38:6816/1491818732 12 ==== osd_op_reply(94 100.00000000 [create] v105'89 uv83 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb20800b040 con 0x55cf7a91a620 2024-09-21T03:41:15.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.759+0000 7fb21defaf40 1 -- 172.21.15.38:0/2189246990 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:95 13.13 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e105) -- 0x55cf7a91d750 con 0x55cf7a91a620 2024-09-21T03:41:15.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.760+0000 7fb2179b4640 1 -- 172.21.15.38:0/2189246990 <== osd.1 v2:172.21.15.38:6816/1491818732 13 ==== osd_op_reply(95 100.00000000 [delete] v105'90 uv90 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb20800b040 con 0x55cf7a91a620 2024-09-21T03:41:15.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.760+0000 7fb21defaf40 1 -- 172.21.15.38:0/2189246990 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:96 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55cf7a91d750 con 0x55cf7a91a620 2024-09-21T03:41:15.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.761+0000 7fb2179b4640 1 -- 172.21.15.38:0/2189246990 <== osd.1 v2:172.21.15.38:6816/1491818732 14 ==== osd_op_reply(96 100.00000000 [create] v105'91 uv91 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb20800b040 con 0x55cf7a91a620 2024-09-21T03:41:15.762 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-09-21T03:41:15.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.761+0000 7fb21defaf40 1 -- 172.21.15.38:0/2189246990 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:97 13.13 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55cf7a91d750 con 0x55cf7a91a620 2024-09-21T03:41:15.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.762+0000 7fb2179b4640 1 -- 172.21.15.38:0/2189246990 <== osd.1 v2:172.21.15.38:6816/1491818732 15 ==== osd_op_reply(97 100.00000000 [omap-set-header] v105'92 uv92 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb20800b040 con 0x55cf7a91a620 2024-09-21T03:41:15.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.762+0000 7fb21defaf40 1 -- 172.21.15.38:0/2189246990 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:98 13.13 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55cf7a920560 con 0x55cf7a91a620 2024-09-21T03:41:15.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.763+0000 7fb2179b4640 1 -- 172.21.15.38:0/2189246990 <== osd.1 v2:172.21.15.38:6816/1491818732 16 ==== osd_op_reply(98 100.00000000 [omap-set-vals] v105'93 uv93 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb20800b040 con 0x55cf7a91a620 2024-09-21T03:41:15.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.763+0000 7fb21defaf40 1 -- 172.21.15.38:0/2189246990 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:99 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55cf7a91dc00 con 0x55cf7a916430 2024-09-21T03:41:15.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.764+0000 7fb2181b5640 1 -- 172.21.15.38:0/2189246990 <== osd.3 v2:172.21.15.38:6808/1286810294 16 ==== osd_op_reply(99 mds0_sessionmap [create] v105'40 uv39 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7fb210007700 con 0x55cf7a916430 2024-09-21T03:41:15.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.764+0000 7fb21defaf40 1 -- 172.21.15.38:0/2189246990 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:100 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e105) -- 0x55cf7a91dc00 con 0x55cf7a916430 2024-09-21T03:41:15.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.765+0000 7fb2181b5640 1 -- 172.21.15.38:0/2189246990 <== osd.3 v2:172.21.15.38:6808/1286810294 17 ==== osd_op_reply(100 mds0_sessionmap [delete] v105'41 uv41 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fb210007700 con 0x55cf7a916430 2024-09-21T03:41:15.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.765+0000 7fb21defaf40 1 -- 172.21.15.38:0/2189246990 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55cf7a91dc00 con 0x55cf7a916430 2024-09-21T03:41:15.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.766+0000 7fb2181b5640 1 -- 172.21.15.38:0/2189246990 <== osd.3 v2:172.21.15.38:6808/1286810294 18 ==== osd_op_reply(101 mds0_sessionmap [create] v105'42 uv42 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fb210007700 con 0x55cf7a916430 2024-09-21T03:41:15.768 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-09-21T03:41:15.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.766+0000 7fb21defaf40 1 -- 172.21.15.38:0/2189246990 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55cf7a91dc00 con 0x55cf7a916430 2024-09-21T03:41:15.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.767+0000 7fb2181b5640 1 -- 172.21.15.38:0/2189246990 <== osd.3 v2:172.21.15.38:6808/1286810294 19 ==== osd_op_reply(102 mds0_sessionmap [omap-set-header] v105'43 uv43 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fb210007700 con 0x55cf7a916430 2024-09-21T03:41:15.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.767+0000 7fb21defaf40 1 -- 172.21.15.38:0/2189246990 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:103 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55cf7a91dc00 con 0x55cf7a916430 2024-09-21T03:41:15.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.768+0000 7fb2181b5640 1 -- 172.21.15.38:0/2189246990 <== osd.3 v2:172.21.15.38:6808/1286810294 20 ==== osd_op_reply(103 609.00000000 [create] v105'51 uv50 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb210007700 con 0x55cf7a916430 2024-09-21T03:41:15.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.768+0000 7fb21defaf40 1 -- 172.21.15.38:0/2189246990 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:104 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e105) -- 0x55cf7a91dc00 con 0x55cf7a916430 2024-09-21T03:41:15.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.769+0000 7fb2181b5640 1 -- 172.21.15.38:0/2189246990 <== osd.3 v2:172.21.15.38:6808/1286810294 21 ==== osd_op_reply(104 609.00000000 [delete] v105'52 uv52 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb210007700 con 0x55cf7a916430 2024-09-21T03:41:15.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.769+0000 7fb21defaf40 1 -- 172.21.15.38:0/2189246990 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55cf7a91dc00 con 0x55cf7a916430 2024-09-21T03:41:15.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.770+0000 7fb2181b5640 1 -- 172.21.15.38:0/2189246990 <== osd.3 v2:172.21.15.38:6808/1286810294 22 ==== osd_op_reply(105 609.00000000 [create] v105'53 uv53 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb210007700 con 0x55cf7a916430 2024-09-21T03:41:15.772 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-09-21T03:41:15.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.770+0000 7fb21defaf40 1 -- 172.21.15.38:0/2189246990 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55cf7a91ec10 con 0x55cf7a916430 2024-09-21T03:41:15.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.771+0000 7fb2181b5640 1 -- 172.21.15.38:0/2189246990 <== osd.3 v2:172.21.15.38:6808/1286810294 23 ==== osd_op_reply(106 609.00000000 [setxattr (30)] v105'54 uv54 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb210007700 con 0x55cf7a916430 2024-09-21T03:41:15.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.771+0000 7fb21defaf40 1 -- 172.21.15.38:0/2189246990 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55cf7a91ec10 con 0x55cf7a916430 2024-09-21T03:41:15.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.772+0000 7fb2181b5640 1 -- 172.21.15.38:0/2189246990 <== osd.3 v2:172.21.15.38:6808/1286810294 24 ==== osd_op_reply(107 609.00000000 [setxattr (62)] v105'55 uv55 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb210007700 con 0x55cf7a916430 2024-09-21T03:41:15.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.772+0000 7fb21defaf40 1 -- 172.21.15.38:0/2189246990 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55cf7a91dc00 con 0x55cf7a916430 2024-09-21T03:41:15.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.773+0000 7fb2181b5640 1 -- 172.21.15.38:0/2189246990 <== osd.3 v2:172.21.15.38:6808/1286810294 25 ==== osd_op_reply(108 609.00000000 [omap-set-header] v105'56 uv56 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb210007700 con 0x55cf7a916430 2024-09-21T03:41:15.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.773+0000 7fb21defaf40 1 --2- 172.21.15.38:0/2189246990 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x55cf7a91e280 0x55cf7a91e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:15.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.773+0000 7fb21defaf40 1 -- 172.21.15.38:0/2189246990 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:109 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55cf7a91ebd0 con 0x55cf7a91e280 2024-09-21T03:41:15.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.773+0000 7fb2171b3640 1 --2- 172.21.15.38:0/2189246990 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x55cf7a91e280 0x55cf7a91e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:15.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.773+0000 7fb2171b3640 1 --2- 172.21.15.38:0/2189246990 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x55cf7a91e280 0x55cf7a91e690 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:15.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.774+0000 7fb2171b3640 1 -- 172.21.15.38:0/2189246990 <== osd.2 v2:172.21.15.38:6824/2577858902 1 ==== osd_op_reply(109 400.00000000 [create] v105'42 uv41 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb20c06d070 con 0x55cf7a91e280 2024-09-21T03:41:15.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.775+0000 7fb21defaf40 1 -- 172.21.15.38:0/2189246990 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:110 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e105) -- 0x55cf7a91ebd0 con 0x55cf7a91e280 2024-09-21T03:41:15.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.776+0000 7fb2171b3640 1 -- 172.21.15.38:0/2189246990 <== osd.2 v2:172.21.15.38:6824/2577858902 2 ==== osd_op_reply(110 400.00000000 [delete] v105'43 uv43 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb20c06d070 con 0x55cf7a91e280 2024-09-21T03:41:15.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.776+0000 7fb21defaf40 1 -- 172.21.15.38:0/2189246990 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55cf7a91ebd0 con 0x55cf7a91e280 2024-09-21T03:41:15.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.777+0000 7fb2171b3640 1 -- 172.21.15.38:0/2189246990 <== osd.2 v2:172.21.15.38:6824/2577858902 3 ==== osd_op_reply(111 400.00000000 [create] v105'44 uv44 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb20c06d070 con 0x55cf7a91e280 2024-09-21T03:41:15.779 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-09-21T03:41:15.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.777+0000 7fb21defaf40 1 -- 172.21.15.38:0/2189246990 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55cf7a91ebd0 con 0x55cf7a91e280 2024-09-21T03:41:15.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.778+0000 7fb2171b3640 1 -- 172.21.15.38:0/2189246990 <== osd.2 v2:172.21.15.38:6824/2577858902 4 ==== osd_op_reply(112 400.00000000 [write 0~22] v105'45 uv45 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb20c06d070 con 0x55cf7a91e280 2024-09-21T03:41:15.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.778+0000 7fb21defaf40 1 -- 172.21.15.38:0/2189246990 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55cf7a91ebd0 con 0x55cf7a91e280 2024-09-21T03:41:15.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.779+0000 7fb2171b3640 1 -- 172.21.15.38:0/2189246990 <== osd.2 v2:172.21.15.38:6824/2577858902 5 ==== osd_op_reply(113 400.00000000 [omap-set-header] v105'46 uv46 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb20c06d070 con 0x55cf7a91e280 2024-09-21T03:41:15.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.779+0000 7fb21defaf40 1 -- 172.21.15.38:0/2189246990 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:114 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55cf7a91ebd0 con 0x55cf7a91a620 2024-09-21T03:41:15.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.780+0000 7fb2179b4640 1 -- 172.21.15.38:0/2189246990 <== osd.1 v2:172.21.15.38:6816/1491818732 17 ==== osd_op_reply(114 100.00000000.inode [create] v105'107 uv99 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7fb20800b040 con 0x55cf7a91a620 2024-09-21T03:41:15.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.780+0000 7fb21defaf40 1 -- 172.21.15.38:0/2189246990 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:115 13.1f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e105) -- 0x55cf7a91ebd0 con 0x55cf7a91a620 2024-09-21T03:41:15.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.781+0000 7fb2179b4640 1 -- 172.21.15.38:0/2189246990 <== osd.1 v2:172.21.15.38:6816/1491818732 18 ==== osd_op_reply(115 100.00000000.inode [delete] v105'108 uv108 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fb20800b040 con 0x55cf7a91a620 2024-09-21T03:41:15.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.781+0000 7fb21defaf40 1 -- 172.21.15.38:0/2189246990 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:116 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55cf7a91ebd0 con 0x55cf7a91a620 2024-09-21T03:41:15.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.782+0000 7fb2179b4640 1 -- 172.21.15.38:0/2189246990 <== osd.1 v2:172.21.15.38:6816/1491818732 19 ==== osd_op_reply(116 100.00000000.inode [create] v105'109 uv109 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fb20800b040 con 0x55cf7a91a620 2024-09-21T03:41:15.784 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-09-21T03:41:15.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.782+0000 7fb21defaf40 1 -- 172.21.15.38:0/2189246990 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:117 13.1f 13:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55cf7a91ebd0 con 0x55cf7a91a620 2024-09-21T03:41:15.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.783+0000 7fb2179b4640 1 -- 172.21.15.38:0/2189246990 <== osd.1 v2:172.21.15.38:6816/1491818732 20 ==== osd_op_reply(117 100.00000000.inode [write 0~582] v105'110 uv110 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fb20800b040 con 0x55cf7a91a620 2024-09-21T03:41:15.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.783+0000 7fb21defaf40 1 -- 172.21.15.38:0/2189246990 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:118 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55cf7a91ebd0 con 0x55cf7a91a620 2024-09-21T03:41:15.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.784+0000 7fb2179b4640 1 -- 172.21.15.38:0/2189246990 <== osd.1 v2:172.21.15.38:6816/1491818732 21 ==== osd_op_reply(118 100.00000000.inode [omap-set-header] v105'111 uv111 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fb20800b040 con 0x55cf7a91a620 2024-09-21T03:41:15.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.785+0000 7fb21defaf40 1 -- 172.21.15.38:0/2189246990 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:119 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55cf7a91ebd0 con 0x55cf7a91a620 2024-09-21T03:41:15.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.786+0000 7fb2179b4640 1 -- 172.21.15.38:0/2189246990 <== osd.1 v2:172.21.15.38:6816/1491818732 22 ==== osd_op_reply(119 1.00000000 [create] v105'112 uv106 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7fb20800b040 con 0x55cf7a91a620 2024-09-21T03:41:15.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.786+0000 7fb21defaf40 1 -- 172.21.15.38:0/2189246990 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:120 13.1f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e105) -- 0x55cf7a91ebd0 con 0x55cf7a91a620 2024-09-21T03:41:15.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.787+0000 7fb2179b4640 1 -- 172.21.15.38:0/2189246990 <== osd.1 v2:172.21.15.38:6816/1491818732 23 ==== osd_op_reply(120 1.00000000 [delete] v105'113 uv113 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fb20800b040 con 0x55cf7a91a620 2024-09-21T03:41:15.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.787+0000 7fb21defaf40 1 -- 172.21.15.38:0/2189246990 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:121 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55cf7a91ebd0 con 0x55cf7a91a620 2024-09-21T03:41:15.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.788+0000 7fb2179b4640 1 -- 172.21.15.38:0/2189246990 <== osd.1 v2:172.21.15.38:6816/1491818732 24 ==== osd_op_reply(121 1.00000000 [create] v105'114 uv114 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fb20800b040 con 0x55cf7a91a620 2024-09-21T03:41:15.790 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-09-21T03:41:15.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.788+0000 7fb21defaf40 1 -- 172.21.15.38:0/2189246990 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:122 13.1f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55cf7a9200c0 con 0x55cf7a91a620 2024-09-21T03:41:15.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.789+0000 7fb2179b4640 1 -- 172.21.15.38:0/2189246990 <== osd.1 v2:172.21.15.38:6816/1491818732 25 ==== osd_op_reply(122 1.00000000 [setxattr (30)] v105'115 uv115 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fb20800b040 con 0x55cf7a91a620 2024-09-21T03:41:15.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.789+0000 7fb21defaf40 1 -- 172.21.15.38:0/2189246990 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:123 13.1f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55cf7a91ebd0 con 0x55cf7a91a620 2024-09-21T03:41:15.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.790+0000 7fb2179b4640 1 -- 172.21.15.38:0/2189246990 <== osd.1 v2:172.21.15.38:6816/1491818732 26 ==== osd_op_reply(123 1.00000000 [setxattr (30)] v105'116 uv116 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fb20800b040 con 0x55cf7a91a620 2024-09-21T03:41:15.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.790+0000 7fb21defaf40 1 -- 172.21.15.38:0/2189246990 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:124 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55cf7a91ebd0 con 0x55cf7a91a620 2024-09-21T03:41:15.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.791+0000 7fb2179b4640 1 -- 172.21.15.38:0/2189246990 <== osd.1 v2:172.21.15.38:6816/1491818732 27 ==== osd_op_reply(124 1.00000000 [omap-set-header] v105'117 uv117 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fb20800b040 con 0x55cf7a91a620 2024-09-21T03:41:15.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.791+0000 7fb21defaf40 1 -- 172.21.15.38:0/2189246990 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:125 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=584b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55cf7a9204c0 con 0x55cf7a91a620 2024-09-21T03:41:15.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.792+0000 7fb2179b4640 1 -- 172.21.15.38:0/2189246990 <== osd.1 v2:172.21.15.38:6816/1491818732 28 ==== osd_op_reply(125 1.00000000 [omap-set-vals] v105'118 uv118 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fb20800b040 con 0x55cf7a91a620 2024-09-21T03:41:15.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.792+0000 7fb21defaf40 1 -- 172.21.15.38:0/2189246990 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x55cf7a9120e0 msgr2=0x55cf7a914500 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:15.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.792+0000 7fb21defaf40 1 --2- 172.21.15.38:0/2189246990 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x55cf7a9120e0 0x55cf7a914500 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:15.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.792+0000 7fb21defaf40 1 -- 172.21.15.38:0/2189246990 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x55cf7a91a620 msgr2=0x55cf7a91aaf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:15.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.792+0000 7fb21defaf40 1 --2- 172.21.15.38:0/2189246990 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x55cf7a91a620 0x55cf7a91aaf0 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:15.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.792+0000 7fb21defaf40 1 -- 172.21.15.38:0/2189246990 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x55cf7a91e280 msgr2=0x55cf7a91e690 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:15.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.792+0000 7fb21defaf40 1 --2- 172.21.15.38:0/2189246990 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x55cf7a91e280 0x55cf7a91e690 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:15.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.792+0000 7fb21defaf40 1 -- 172.21.15.38:0/2189246990 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x55cf7a916430 msgr2=0x55cf7a919290 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:15.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.792+0000 7fb21defaf40 1 --2- 172.21.15.38:0/2189246990 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x55cf7a916430 0x55cf7a919290 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:15.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.792+0000 7fb2181b5640 1 -- 172.21.15.38:0/2189246990 reap_dead start 2024-09-21T03:41:15.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.792+0000 7fb21defaf40 1 -- 172.21.15.38:0/2189246990 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x55cf7a90cb00 msgr2=0x55cf7a90fc20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:15.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.792+0000 7fb21defaf40 1 --2- 172.21.15.38:0/2189246990 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x55cf7a90cb00 0x55cf7a90fc20 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:15.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.792+0000 7fb2181b5640 1 -- 172.21.15.38:0/2189246990 reap_dead start 2024-09-21T03:41:15.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.792+0000 7fb21defaf40 1 -- 172.21.15.38:0/2189246990 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x55cf7a845860 msgr2=0x55cf7a90b330 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:15.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.792+0000 7fb21defaf40 1 --2- 172.21.15.38:0/2189246990 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x55cf7a845860 0x55cf7a90b330 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:15.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.792+0000 7fb21defaf40 1 -- 172.21.15.38:0/2189246990 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb204077570 msgr2=0x7fb204079a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:15.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.792+0000 7fb21defaf40 1 --2- 172.21.15.38:0/2189246990 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb204077570 0x7fb204079a80 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fb208000c00 tx=0x7fb208003040 comp rx=0 tx=0).stop 2024-09-21T03:41:15.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.792+0000 7fb21defaf40 1 -- 172.21.15.38:0/2189246990 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cf7a862db0 msgr2=0x55cf7a85f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:15.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.792+0000 7fb21defaf40 1 --2- 172.21.15.38:0/2189246990 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cf7a862db0 0x55cf7a85f850 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7fb20c002040 tx=0x7fb20c0559d0 comp rx=0 tx=0).stop 2024-09-21T03:41:15.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.792+0000 7fb21defaf40 1 -- 172.21.15.38:0/2189246990 shutdown_connections 2024-09-21T03:41:15.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.792+0000 7fb21defaf40 1 --2- 172.21.15.38:0/2189246990 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x55cf7a845860 0x55cf7a90b330 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:15.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.792+0000 7fb21defaf40 1 --2- 172.21.15.38:0/2189246990 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb204077570 0x7fb204079a80 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:15.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.792+0000 7fb21defaf40 1 --2- 172.21.15.38:0/2189246990 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cf7a862db0 0x55cf7a85f850 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:15.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.793+0000 7fb21defaf40 1 -- 172.21.15.38:0/2189246990 >> 172.21.15.38:0/2189246990 conn(0x55cf7a6b0ac0 msgr2=0x55cf7a7b9e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:15.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.793+0000 7fb21defaf40 1 -- 172.21.15.38:0/2189246990 shutdown_connections 2024-09-21T03:41:15.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.793+0000 7fb21defaf40 1 -- 172.21.15.38:0/2189246990 wait complete. 2024-09-21T03:41:15.798 DEBUG:teuthology.orchestra.run.smithi038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rm mds0_inotable 2024-09-21T03:41:15.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.892+0000 7fd450b75f40 1 Processor -- start 2024-09-21T03:41:15.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.892+0000 7fd450b75f40 1 -- start start 2024-09-21T03:41:15.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.893+0000 7fd450b75f40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ee4d572900 0x55ee4d63c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:15.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.893+0000 7fd450b75f40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ee4d576d00 0x55ee4d5a2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:15.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.893+0000 7fd450b75f40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55ee4d5a45a0 0x55ee4d5a30b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:15.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.894+0000 7fd450b75f40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55ee4d4c4d60 con 0x55ee4d572900 2024-09-21T03:41:15.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.894+0000 7fd450b75f40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55ee4d523430 con 0x55ee4d576d00 2024-09-21T03:41:15.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.894+0000 7fd450b75f40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55ee4d5a5810 con 0x55ee4d5a45a0 2024-09-21T03:41:15.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.894+0000 7fd44a62f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ee4d572900 0x55ee4d63c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:15.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.894+0000 7fd44a62f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ee4d572900 0x55ee4d63c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46940/0 (socket says 172.21.15.38:46940) 2024-09-21T03:41:15.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.894+0000 7fd44a62f640 1 -- 172.21.15.38:0/1558838043 learned_addr learned my addr 172.21.15.38:0/1558838043 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:41:15.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.894+0000 7fd449e2e640 1 --2- 172.21.15.38:0/1558838043 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ee4d576d00 0x55ee4d5a2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:15.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.894+0000 7fd44ae30640 1 --2- 172.21.15.38:0/1558838043 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55ee4d5a45a0 0x55ee4d5a30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:15.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.894+0000 7fd44a62f640 1 -- 172.21.15.38:0/1558838043 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55ee4d5a45a0 msgr2=0x55ee4d5a30b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:15.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.894+0000 7fd44a62f640 1 --2- 172.21.15.38:0/1558838043 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55ee4d5a45a0 0x55ee4d5a30b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:15.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.894+0000 7fd44a62f640 1 -- 172.21.15.38:0/1558838043 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ee4d576d00 msgr2=0x55ee4d5a2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:15.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.894+0000 7fd44a62f640 1 --2- 172.21.15.38:0/1558838043 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ee4d576d00 0x55ee4d5a2b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:15.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.894+0000 7fd44a62f640 1 -- 172.21.15.38:0/1558838043 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee4d5a40e0 con 0x55ee4d572900 2024-09-21T03:41:15.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.894+0000 7fd44a62f640 1 --2- 172.21.15.38:0/1558838043 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ee4d572900 0x55ee4d63c440 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7fd43c025f40 tx=0x7fd43c002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c95cea63c0d00a4c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:15.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.894+0000 7fd448e2c640 1 -- 172.21.15.38:0/1558838043 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd43c056470 con 0x55ee4d572900 2024-09-21T03:41:15.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.894+0000 7fd448e2c640 1 -- 172.21.15.38:0/1558838043 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd43c056610 con 0x55ee4d572900 2024-09-21T03:41:15.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.894+0000 7fd448e2c640 1 -- 172.21.15.38:0/1558838043 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd43c056a40 con 0x55ee4d572900 2024-09-21T03:41:15.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.895+0000 7fd450b75f40 1 -- 172.21.15.38:0/1558838043 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ee4d572900 msgr2=0x55ee4d63c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:15.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.895+0000 7fd450b75f40 1 --2- 172.21.15.38:0/1558838043 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ee4d572900 0x55ee4d63c440 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7fd43c025f40 tx=0x7fd43c002fb0 comp rx=0 tx=0).stop 2024-09-21T03:41:15.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.895+0000 7fd450b75f40 1 -- 172.21.15.38:0/1558838043 shutdown_connections 2024-09-21T03:41:15.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.895+0000 7fd450b75f40 1 --2- 172.21.15.38:0/1558838043 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55ee4d5a45a0 0x55ee4d5a30b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:15.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.895+0000 7fd450b75f40 1 --2- 172.21.15.38:0/1558838043 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ee4d576d00 0x55ee4d5a2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:15.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.895+0000 7fd450b75f40 1 --2- 172.21.15.38:0/1558838043 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ee4d572900 0x55ee4d63c440 unknown :-1 s=CLOSED pgs=1445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:15.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.895+0000 7fd450b75f40 1 -- 172.21.15.38:0/1558838043 >> 172.21.15.38:0/1558838043 conn(0x55ee4d495ac0 msgr2=0x55ee4d644f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:15.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.895+0000 7fd450b75f40 1 -- 172.21.15.38:0/1558838043 shutdown_connections 2024-09-21T03:41:15.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.895+0000 7fd450b75f40 1 -- 172.21.15.38:0/1558838043 wait complete. 2024-09-21T03:41:15.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.896+0000 7fd450b75f40 1 Processor -- start 2024-09-21T03:41:15.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.896+0000 7fd450b75f40 1 -- start start 2024-09-21T03:41:15.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.897+0000 7fd450b75f40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ee4d572900 0x55ee4d6369a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:15.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.897+0000 7fd450b75f40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55ee4d576d00 0x55ee4d5bde40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:15.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.897+0000 7fd450b75f40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ee4d5a45a0 0x55ee4d5be380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:15.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.897+0000 7fd450b75f40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55ee4d5a43c0 con 0x55ee4d572900 2024-09-21T03:41:15.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.897+0000 7fd450b75f40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55ee4d63d620 con 0x55ee4d5a45a0 2024-09-21T03:41:15.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.897+0000 7fd450b75f40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55ee4d5ba720 con 0x55ee4d576d00 2024-09-21T03:41:15.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.897+0000 7fd44a62f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ee4d572900 0x55ee4d6369a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:15.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.897+0000 7fd44a62f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ee4d572900 0x55ee4d6369a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46946/0 (socket says 172.21.15.38:46946) 2024-09-21T03:41:15.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.897+0000 7fd44a62f640 1 -- 172.21.15.38:0/1998109338 learned_addr learned my addr 172.21.15.38:0/1998109338 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:41:15.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.897+0000 7fd449e2e640 1 --2- 172.21.15.38:0/1998109338 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55ee4d576d00 0x55ee4d5bde40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:15.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.897+0000 7fd44ae30640 1 --2- 172.21.15.38:0/1998109338 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ee4d5a45a0 0x55ee4d5be380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:15.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.898+0000 7fd44a62f640 1 -- 172.21.15.38:0/1998109338 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55ee4d576d00 msgr2=0x55ee4d5bde40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:15.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.898+0000 7fd44a62f640 1 --2- 172.21.15.38:0/1998109338 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55ee4d576d00 0x55ee4d5bde40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:15.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.898+0000 7fd44a62f640 1 -- 172.21.15.38:0/1998109338 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ee4d5a45a0 msgr2=0x55ee4d5be380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:15.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.898+0000 7fd44a62f640 1 --2- 172.21.15.38:0/1998109338 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ee4d5a45a0 0x55ee4d5be380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:15.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.898+0000 7fd44a62f640 1 -- 172.21.15.38:0/1998109338 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee4d637280 con 0x55ee4d572900 2024-09-21T03:41:15.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.898+0000 7fd44a62f640 1 --2- 172.21.15.38:0/1998109338 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ee4d572900 0x55ee4d6369a0 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7fd43c025f10 tx=0x7fd43c057a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:15.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.898+0000 7fd43b7fe640 1 -- 172.21.15.38:0/1998109338 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd43c067040 con 0x55ee4d572900 2024-09-21T03:41:15.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.898+0000 7fd43b7fe640 1 -- 172.21.15.38:0/1998109338 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd43c057c00 con 0x55ee4d572900 2024-09-21T03:41:15.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.898+0000 7fd450b75f40 1 -- 172.21.15.38:0/1998109338 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55ee4d5732b0 con 0x55ee4d572900 2024-09-21T03:41:15.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.898+0000 7fd43b7fe640 1 -- 172.21.15.38:0/1998109338 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd43c06b430 con 0x55ee4d572900 2024-09-21T03:41:15.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.898+0000 7fd450b75f40 1 -- 172.21.15.38:0/1998109338 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee4d637c90 con 0x55ee4d572900 2024-09-21T03:41:15.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.900+0000 7fd43b7fe640 1 -- 172.21.15.38:0/1998109338 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fd43c072030 con 0x55ee4d572900 2024-09-21T03:41:15.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.900+0000 7fd43b7fe640 1 --2- 172.21.15.38:0/1998109338 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd42c0775a0 0x7fd42c079ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:15.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.900+0000 7fd43b7fe640 1 -- 172.21.15.38:0/1998109338 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 7078+0+0 (secure 0 0 0) 0x7fd43c0def10 con 0x55ee4d572900 2024-09-21T03:41:15.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.900+0000 7fd449e2e640 1 --2- 172.21.15.38:0/1998109338 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd42c0775a0 0x7fd42c079ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:15.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.901+0000 7fd450b75f40 1 --2- 172.21.15.38:0/1998109338 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x55ee4d5b9da0 0x55ee4d5ba1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:15.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.901+0000 7fd450b75f40 1 -- 172.21.15.38:0/1998109338 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:1 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e105) -- 0x55ee4d637580 con 0x55ee4d5b9da0 2024-09-21T03:41:15.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.901+0000 7fd44ae30640 1 --2- 172.21.15.38:0/1998109338 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x55ee4d5b9da0 0x55ee4d5ba1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:15.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.901+0000 7fd449e2e640 1 --2- 172.21.15.38:0/1998109338 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd42c0775a0 0x7fd42c079ab0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x55ee4d577620 tx=0x7fd44000f040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:15.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.901+0000 7fd44ae30640 1 --2- 172.21.15.38:0/1998109338 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x55ee4d5b9da0 0x55ee4d5ba1b0 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:15.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.902+0000 7fd44ae30640 1 -- 172.21.15.38:0/1998109338 <== osd.1 v2:172.21.15.38:6816/1491818732 1 ==== osd_op_reply(1 mds0_inotable [delete] v105'94 uv94 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fd444002620 con 0x55ee4d5b9da0 2024-09-21T03:41:15.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.902+0000 7fd450b75f40 1 -- 172.21.15.38:0/1998109338 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x55ee4d5b9da0 msgr2=0x55ee4d5ba1b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:15.906 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.902+0000 7fd450b75f40 1 --2- 172.21.15.38:0/1998109338 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x55ee4d5b9da0 0x55ee4d5ba1b0 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:15.906 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.902+0000 7fd450b75f40 1 -- 172.21.15.38:0/1998109338 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd42c0775a0 msgr2=0x7fd42c079ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:15.906 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.902+0000 7fd450b75f40 1 --2- 172.21.15.38:0/1998109338 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd42c0775a0 0x7fd42c079ab0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x55ee4d577620 tx=0x7fd44000f040 comp rx=0 tx=0).stop 2024-09-21T03:41:15.906 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.902+0000 7fd450b75f40 1 -- 172.21.15.38:0/1998109338 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ee4d572900 msgr2=0x55ee4d6369a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:15.906 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.902+0000 7fd450b75f40 1 --2- 172.21.15.38:0/1998109338 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ee4d572900 0x55ee4d6369a0 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7fd43c025f10 tx=0x7fd43c057a60 comp rx=0 tx=0).stop 2024-09-21T03:41:15.906 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.902+0000 7fd44ae30640 1 -- 172.21.15.38:0/1998109338 reap_dead start 2024-09-21T03:41:15.906 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.902+0000 7fd450b75f40 1 -- 172.21.15.38:0/1998109338 shutdown_connections 2024-09-21T03:41:15.906 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.902+0000 7fd450b75f40 1 -- 172.21.15.38:0/1998109338 >> 172.21.15.38:0/1998109338 conn(0x55ee4d495ac0 msgr2=0x55ee4d63ba20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:15.906 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.902+0000 7fd450b75f40 1 -- 172.21.15.38:0/1998109338 shutdown_connections 2024-09-21T03:41:15.906 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:15.902+0000 7fd450b75f40 1 -- 172.21.15.38:0/1998109338 wait complete. 2024-09-21T03:41:15.907 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-21T03:41:16.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:16.025+0000 7fbcd1724640 1 -- 172.21.15.38:0/2120563071 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbccc15ee80 msgr2=0x7fbccc0d0170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:16.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:16.025+0000 7fbcd1724640 1 --2- 172.21.15.38:0/2120563071 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbccc15ee80 0x7fbccc0d0170 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7fbccc0d06b0 tx=0x7fbcbc02a4d0 comp rx=0 tx=0).stop 2024-09-21T03:41:16.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:16.025+0000 7fbcd1724640 1 -- 172.21.15.38:0/2120563071 shutdown_connections 2024-09-21T03:41:16.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:16.025+0000 7fbcd1724640 1 --2- 172.21.15.38:0/2120563071 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbccc0d1120 0x7fbccc0ceb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:16.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:16.025+0000 7fbcd1724640 1 --2- 172.21.15.38:0/2120563071 >> v2:172.21.15.195:3300/0 conn(0x7fbccc0b9ad0 0x7fbccc0d0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:16.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:16.025+0000 7fbcd1724640 1 --2- 172.21.15.38:0/2120563071 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbccc15ee80 0x7fbccc0d0170 unknown :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:16.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:16.025+0000 7fbcd1724640 1 -- 172.21.15.38:0/2120563071 >> 172.21.15.38:0/2120563071 conn(0x7fbccc09d830 msgr2=0x7fbccc0b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:16.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:16.025+0000 7fbcd1724640 1 -- 172.21.15.38:0/2120563071 shutdown_connections 2024-09-21T03:41:16.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:16.025+0000 7fbcd1724640 1 -- 172.21.15.38:0/2120563071 wait complete. 2024-09-21T03:41:16.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:16.026+0000 7fbcd1724640 1 Processor -- start 2024-09-21T03:41:16.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:16.027+0000 7fbcd1724640 1 -- start start 2024-09-21T03:41:16.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:16.028+0000 7fbcd1724640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbccc0b9ad0 0x7fbccc06f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:16.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:16.028+0000 7fbcd1724640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbccc0d1120 0x7fbccc153a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:16.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:16.028+0000 7fbcd1724640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbccc15ee80 0x7fbccc1f8860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:16.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:16.028+0000 7fbcd1724640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fbccc0cf520 con 0x7fbccc15ee80 2024-09-21T03:41:16.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:16.028+0000 7fbcd1724640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fbccc0cf3a0 con 0x7fbccc0b9ad0 2024-09-21T03:41:16.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:16.028+0000 7fbcd1724640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fbccc0b6580 con 0x7fbccc0d1120 2024-09-21T03:41:16.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:16.028+0000 7fbccb7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbccc15ee80 0x7fbccc1f8860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:16.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:16.028+0000 7fbccaffd640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbccc0b9ad0 0x7fbccc06f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:16.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:16.028+0000 7fbccb7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbccc15ee80 0x7fbccc1f8860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46962/0 (socket says 172.21.15.38:46962) 2024-09-21T03:41:16.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:16.028+0000 7fbcc3fff640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbccc0d1120 0x7fbccc153a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:16.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:16.028+0000 7fbccb7fe640 1 -- 172.21.15.38:0/1647685355 learned_addr learned my addr 172.21.15.38:0/1647685355 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:41:16.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:16.028+0000 7fbccb7fe640 1 -- 172.21.15.38:0/1647685355 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbccc0d1120 msgr2=0x7fbccc153a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:16.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:16.028+0000 7fbccb7fe640 1 --2- 172.21.15.38:0/1647685355 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbccc0d1120 0x7fbccc153a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:16.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:16.028+0000 7fbccb7fe640 1 -- 172.21.15.38:0/1647685355 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbccc0b9ad0 msgr2=0x7fbccc06f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:16.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:16.028+0000 7fbccb7fe640 1 --2- 172.21.15.38:0/1647685355 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbccc0b9ad0 0x7fbccc06f8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:16.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:16.028+0000 7fbccb7fe640 1 -- 172.21.15.38:0/1647685355 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbccc06d7f0 con 0x7fbccc15ee80 2024-09-21T03:41:16.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:16.029+0000 7fbccb7fe640 1 --2- 172.21.15.38:0/1647685355 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbccc15ee80 0x7fbccc1f8860 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7fbcc400d9e0 tx=0x7fbcc400dea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:16.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:16.029+0000 7fbcc37fe640 1 -- 172.21.15.38:0/1647685355 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcc4017070 con 0x7fbccc15ee80 2024-09-21T03:41:16.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:16.029+0000 7fbcc37fe640 1 -- 172.21.15.38:0/1647685355 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbcc400be30 con 0x7fbccc15ee80 2024-09-21T03:41:16.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:16.029+0000 7fbcd1724640 1 -- 172.21.15.38:0/1647685355 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbccc1f9670 con 0x7fbccc15ee80 2024-09-21T03:41:16.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:16.029+0000 7fbcc37fe640 1 -- 172.21.15.38:0/1647685355 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcc4003bb0 con 0x7fbccc15ee80 2024-09-21T03:41:16.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:16.029+0000 7fbcd1724640 1 -- 172.21.15.38:0/1647685355 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbccc06dd70 con 0x7fbccc15ee80 2024-09-21T03:41:16.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:16.030+0000 7fbcd1724640 1 -- 172.21.15.38:0/1647685355 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbccc0ceb90 con 0x7fbccc15ee80 2024-09-21T03:41:16.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:16.031+0000 7fbcc37fe640 1 -- 172.21.15.38:0/1647685355 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fbcc4010460 con 0x7fbccc15ee80 2024-09-21T03:41:16.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:16.031+0000 7fbcc37fe640 1 --2- 172.21.15.38:0/1647685355 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fbcac078bc0 0x7fbcac07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:16.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:16.032+0000 7fbcc37fe640 1 -- 172.21.15.38:0/1647685355 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 7078+0+0 (secure 0 0 0) 0x7fbcc409b620 con 0x7fbccc15ee80 2024-09-21T03:41:16.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:16.032+0000 7fbccaffd640 1 --2- 172.21.15.38:0/1647685355 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fbcac078bc0 0x7fbcac07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:16.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:16.032+0000 7fbccaffd640 1 --2- 172.21.15.38:0/1647685355 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fbcac078bc0 0x7fbcac07b0d0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc033000 tx=0x7fbcbc004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:16.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:16.036+0000 7fbcc37fe640 1 -- 172.21.15.38:0/1647685355 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fbcc4062f40 con 0x7fbccc15ee80 2024-09-21T03:41:16.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:16.263+0000 7fbcd1724640 1 -- 172.21.15.38:0/1647685355 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7fbccc0762a0 con 0x7fbccc15ee80 2024-09-21T03:41:16.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:16.551+0000 7fbcc37fe640 1 -- 172.21.15.38:0/1647685355 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v139) ==== 162+0+0 (secure 0 0 0) 0x7fbcc4067970 con 0x7fbccc15ee80 2024-09-21T03:41:16.553 INFO:teuthology.orchestra.run.smithi038.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-21T03:41:16.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:16.554+0000 7fbcd1724640 1 -- 172.21.15.38:0/1647685355 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fbcac078bc0 msgr2=0x7fbcac07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:16.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:16.554+0000 7fbcd1724640 1 --2- 172.21.15.38:0/1647685355 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fbcac078bc0 0x7fbcac07b0d0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc033000 tx=0x7fbcbc004730 comp rx=0 tx=0).stop 2024-09-21T03:41:16.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:16.554+0000 7fbcd1724640 1 -- 172.21.15.38:0/1647685355 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbccc15ee80 msgr2=0x7fbccc1f8860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:16.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:16.554+0000 7fbcd1724640 1 --2- 172.21.15.38:0/1647685355 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbccc15ee80 0x7fbccc1f8860 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7fbcc400d9e0 tx=0x7fbcc400dea0 comp rx=0 tx=0).stop 2024-09-21T03:41:16.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:16.554+0000 7fbcd1724640 1 -- 172.21.15.38:0/1647685355 shutdown_connections 2024-09-21T03:41:16.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:16.554+0000 7fbcd1724640 1 --2- 172.21.15.38:0/1647685355 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbccc15ee80 0x7fbccc1f8860 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:16.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:16.554+0000 7fbcd1724640 1 --2- 172.21.15.38:0/1647685355 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fbcac078bc0 0x7fbcac07b0d0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:16.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:16.554+0000 7fbcd1724640 1 --2- 172.21.15.38:0/1647685355 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbccc0d1120 0x7fbccc153a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:16.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:16.554+0000 7fbcd1724640 1 --2- 172.21.15.38:0/1647685355 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbccc0b9ad0 0x7fbccc06f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:16.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:16.554+0000 7fbcd1724640 1 -- 172.21.15.38:0/1647685355 >> 172.21.15.38:0/1647685355 conn(0x7fbccc09d830 msgr2=0x7fbccc160570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:16.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:16.554+0000 7fbcd1724640 1 -- 172.21.15.38:0/1647685355 shutdown_connections 2024-09-21T03:41:16.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:16.554+0000 7fbcd1724640 1 -- 172.21.15.38:0/1647685355 wait complete. 2024-09-21T03:41:16.566 INFO:tasks.ceph.mds.a.smithi038.stderr:2024-09-21T03:41:16.564+0000 7f84565dc640 -1 mds.0.inotable: load_2 could not read table: -2 2024-09-21T03:41:16.566 INFO:tasks.ceph.mds.a.smithi038.stderr:2024-09-21T03:41:16.564+0000 7f84565dc640 -1 log_channel(cluster) log [ERR] : error reading table object 'mds0_inotable' -2 ((2) No such file or directory) 2024-09-21T03:41:16.567 INFO:tasks.ceph.mon.a.smithi038.stderr:2024-09-21T03:41:16.565+0000 7f3f45713640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T03:41:16.567 INFO:tasks.ceph.mon.a.smithi038.stderr:2024-09-21T03:41:16.565+0000 7f3f45713640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-09-21T03:41:16.569 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:41:16.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:16.684+0000 7fcea93ad640 1 -- 172.21.15.38:0/3209471886 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcea40cfe00 msgr2=0x7fcea40d2a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:16.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:16.684+0000 7fcea93ad640 1 --2- 172.21.15.38:0/3209471886 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcea40cfe00 0x7fcea40d2a20 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7fce94005fd0 tx=0x7fce9402a8b0 comp rx=0 tx=0).stop 2024-09-21T03:41:16.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:16.684+0000 7fcea93ad640 1 -- 172.21.15.38:0/3209471886 shutdown_connections 2024-09-21T03:41:16.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:16.684+0000 7fcea93ad640 1 --2- 172.21.15.38:0/3209471886 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcea4162590 0x7fcea41629a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:16.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:16.684+0000 7fcea93ad640 1 --2- 172.21.15.38:0/3209471886 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcea40cfe00 0x7fcea40d2a20 unknown :-1 s=CLOSED pgs=1450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:16.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:16.684+0000 7fcea93ad640 1 -- 172.21.15.38:0/3209471886 >> 172.21.15.38:0/3209471886 conn(0x7fcea40d4440 msgr2=0x7fcea40d4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:16.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:16.684+0000 7fcea93ad640 1 -- 172.21.15.38:0/3209471886 shutdown_connections 2024-09-21T03:41:16.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:16.684+0000 7fcea93ad640 1 -- 172.21.15.38:0/3209471886 wait complete. 2024-09-21T03:41:16.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:16.686+0000 7fcea93ad640 1 Processor -- start 2024-09-21T03:41:16.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:16.686+0000 7fcea93ad640 1 -- start start 2024-09-21T03:41:16.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:16.687+0000 7fcea93ad640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcea40b2e60 0x7fcea4152070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:16.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:16.687+0000 7fcea93ad640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcea40cfe00 0x7fcea41525b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:16.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:16.687+0000 7fcea93ad640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcea4162590 0x7fcea4152af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:16.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:16.687+0000 7fcea93ad640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fcea40b3980 con 0x7fcea40cfe00 2024-09-21T03:41:16.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:16.687+0000 7fcea93ad640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fcea40cf5a0 con 0x7fcea4162590 2024-09-21T03:41:16.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:16.687+0000 7fcea93ad640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fcea40b3b00 con 0x7fcea40b2e60 2024-09-21T03:41:16.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:16.687+0000 7fcea27fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcea40cfe00 0x7fcea41525b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:16.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:16.687+0000 7fcea27fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcea40cfe00 0x7fcea41525b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46974/0 (socket says 172.21.15.38:46974) 2024-09-21T03:41:16.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:16.687+0000 7fcea27fc640 1 -- 172.21.15.38:0/494168180 learned_addr learned my addr 172.21.15.38:0/494168180 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:41:16.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:16.688+0000 7fcea2ffd640 1 --2- 172.21.15.38:0/494168180 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcea40b2e60 0x7fcea4152070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:16.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:16.688+0000 7fcea37fe640 1 --2- 172.21.15.38:0/494168180 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcea4162590 0x7fcea4152af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:16.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:16.688+0000 7fcea27fc640 1 -- 172.21.15.38:0/494168180 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcea40b2e60 msgr2=0x7fcea4152070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:16.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:16.688+0000 7fcea27fc640 1 --2- 172.21.15.38:0/494168180 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcea40b2e60 0x7fcea4152070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:16.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:16.688+0000 7fcea27fc640 1 -- 172.21.15.38:0/494168180 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcea4162590 msgr2=0x7fcea4152af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:16.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:16.688+0000 7fcea27fc640 1 --2- 172.21.15.38:0/494168180 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcea4162590 0x7fcea4152af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:16.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:16.688+0000 7fcea27fc640 1 -- 172.21.15.38:0/494168180 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcea41f57f0 con 0x7fcea40cfe00 2024-09-21T03:41:16.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:16.688+0000 7fcea37fe640 1 --2- 172.21.15.38:0/494168180 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcea4162590 0x7fcea4152af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:41:16.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:16.688+0000 7fcea27fc640 1 --2- 172.21.15.38:0/494168180 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcea40cfe00 0x7fcea41525b0 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7fce8c00d740 tx=0x7fce8c00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:16.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:16.688+0000 7fce9b7fe640 1 -- 172.21.15.38:0/494168180 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce8c017070 con 0x7fcea40cfe00 2024-09-21T03:41:16.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:16.688+0000 7fce9b7fe640 1 -- 172.21.15.38:0/494168180 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fce8c009d20 con 0x7fcea40cfe00 2024-09-21T03:41:16.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:16.688+0000 7fce9b7fe640 1 -- 172.21.15.38:0/494168180 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce8c0106c0 con 0x7fcea40cfe00 2024-09-21T03:41:16.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:16.688+0000 7fcea93ad640 1 -- 172.21.15.38:0/494168180 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcea4157ba0 con 0x7fcea40cfe00 2024-09-21T03:41:16.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:16.688+0000 7fcea93ad640 1 -- 172.21.15.38:0/494168180 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcea4151870 con 0x7fcea40cfe00 2024-09-21T03:41:16.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:16.689+0000 7fcea93ad640 1 -- 172.21.15.38:0/494168180 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcea40d2a20 con 0x7fcea40cfe00 2024-09-21T03:41:16.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:16.690+0000 7fce9b7fe640 1 -- 172.21.15.38:0/494168180 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fce8c01a070 con 0x7fcea40cfe00 2024-09-21T03:41:16.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:16.690+0000 7fce9b7fe640 1 --2- 172.21.15.38:0/494168180 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fce84078bc0 0x7fce8407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:16.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:16.691+0000 7fce9b7fe640 1 -- 172.21.15.38:0/494168180 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 7164+0+0 (secure 0 0 0) 0x7fce8c09b340 con 0x7fcea40cfe00 2024-09-21T03:41:16.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:16.691+0000 7fcea2ffd640 1 --2- 172.21.15.38:0/494168180 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fce84078bc0 0x7fce8407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:16.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:16.691+0000 7fcea2ffd640 1 --2- 172.21.15.38:0/494168180 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fce84078bc0 0x7fce8407b0d0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fce94034000 tx=0x7fce94004ab0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:16.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:16.695+0000 7fce9b7fe640 1 -- 172.21.15.38:0/494168180 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fce8c062b80 con 0x7fcea40cfe00 2024-09-21T03:41:16.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:16.921+0000 7fcea93ad640 1 -- 172.21.15.38:0/494168180 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcea40cea70 con 0x7fcea40cfe00 2024-09-21T03:41:16.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:16.922+0000 7fce9b7fe640 1 -- 172.21.15.38:0/494168180 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 141 v141) ==== 95+0+3566 (secure 0 0 0) 0x7fce8c0675b0 con 0x7fcea40cfe00 2024-09-21T03:41:16.924 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:41:16.924 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":141,"btime":"2024-09-21T03:41:16:566084+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6086,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/965972663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":965972663},{"type":"v1","addr":"172.21.15.195:6836","nonce":965972663}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6700,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/3176122334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":3176122334},{"type":"v1","addr":"172.21.15.195:6837","nonce":3176122334}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":141,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:40:02.319282+0000","modified":"2024-09-21T03:41:16.566009+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":106,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-21T03:41:16.924 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 141 2024-09-21T03:41:16.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:16.924+0000 7fcea93ad640 1 -- 172.21.15.38:0/494168180 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fce84078bc0 msgr2=0x7fce8407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:16.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:16.924+0000 7fcea93ad640 1 --2- 172.21.15.38:0/494168180 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fce84078bc0 0x7fce8407b0d0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fce94034000 tx=0x7fce94004ab0 comp rx=0 tx=0).stop 2024-09-21T03:41:16.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:16.924+0000 7fcea93ad640 1 -- 172.21.15.38:0/494168180 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcea40cfe00 msgr2=0x7fcea41525b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:16.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:16.924+0000 7fcea93ad640 1 --2- 172.21.15.38:0/494168180 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcea40cfe00 0x7fcea41525b0 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7fce8c00d740 tx=0x7fce8c00dc00 comp rx=0 tx=0).stop 2024-09-21T03:41:16.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:16.924+0000 7fcea93ad640 1 -- 172.21.15.38:0/494168180 shutdown_connections 2024-09-21T03:41:16.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:16.924+0000 7fcea93ad640 1 --2- 172.21.15.38:0/494168180 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcea4162590 0x7fcea4152af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:16.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:16.924+0000 7fcea93ad640 1 --2- 172.21.15.38:0/494168180 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcea40cfe00 0x7fcea41525b0 unknown :-1 s=CLOSED pgs=1451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:16.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:16.924+0000 7fcea93ad640 1 --2- 172.21.15.38:0/494168180 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fce84078bc0 0x7fce8407b0d0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:16.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:16.924+0000 7fcea93ad640 1 --2- 172.21.15.38:0/494168180 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcea40b2e60 0x7fcea4152070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:16.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:16.924+0000 7fcea93ad640 1 -- 172.21.15.38:0/494168180 >> 172.21.15.38:0/494168180 conn(0x7fcea40d4440 msgr2=0x7fcea40b3270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:16.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:16.924+0000 7fcea93ad640 1 -- 172.21.15.38:0/494168180 shutdown_connections 2024-09-21T03:41:16.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:16.924+0000 7fcea93ad640 1 -- 172.21.15.38:0/494168180 wait complete. 2024-09-21T03:41:16.937 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-21T03:41:16.937 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:41:17.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:17.052+0000 7ff17f0c4640 1 -- 172.21.15.38:0/1298105910 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff1780b2e40 msgr2=0x7ff1780b3250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:17.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:17.052+0000 7ff17f0c4640 1 --2- 172.21.15.38:0/1298105910 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff1780b2e40 0x7ff1780b3250 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7ff170005fd0 tx=0x7ff17002a8b0 comp rx=0 tx=0).stop 2024-09-21T03:41:17.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:17.052+0000 7ff17f0c4640 1 -- 172.21.15.38:0/1298105910 shutdown_connections 2024-09-21T03:41:17.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:17.052+0000 7ff17f0c4640 1 --2- 172.21.15.38:0/1298105910 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff1780b2e40 0x7ff1780b3250 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:17.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:17.052+0000 7ff17f0c4640 1 --2- 172.21.15.38:0/1298105910 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff178162f00 0x7ff1780d24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:17.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:17.052+0000 7ff17f0c4640 1 -- 172.21.15.38:0/1298105910 >> 172.21.15.38:0/1298105910 conn(0x7ff1780b6b20 msgr2=0x7ff1780d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:17.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:17.052+0000 7ff17f0c4640 1 -- 172.21.15.38:0/1298105910 shutdown_connections 2024-09-21T03:41:17.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:17.052+0000 7ff17f0c4640 1 -- 172.21.15.38:0/1298105910 wait complete. 2024-09-21T03:41:17.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:17.053+0000 7ff17f0c4640 1 Processor -- start 2024-09-21T03:41:17.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:17.054+0000 7ff17f0c4640 1 -- start start 2024-09-21T03:41:17.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:17.055+0000 7ff17f0c4640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff1780b2e40 0x7ff1780ac1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:17.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:17.055+0000 7ff17f0c4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff178162280 0x7ff1781f4d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:17.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:17.055+0000 7ff17f0c4640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff1781f5250 0x7ff1780d0c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:17.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:17.055+0000 7ff17f0c4640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ff17815e4b0 con 0x7ff178162280 2024-09-21T03:41:17.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:17.055+0000 7ff17f0c4640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ff1780b39a0 con 0x7ff1780b2e40 2024-09-21T03:41:17.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:17.055+0000 7ff17f0c4640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7ff17815e630 con 0x7ff1781f5250 2024-09-21T03:41:17.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:17.055+0000 7ff177fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff178162280 0x7ff1781f4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:17.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:17.055+0000 7ff177fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff178162280 0x7ff1781f4d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46994/0 (socket says 172.21.15.38:46994) 2024-09-21T03:41:17.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:17.055+0000 7ff177fff640 1 -- 172.21.15.38:0/1466906664 learned_addr learned my addr 172.21.15.38:0/1466906664 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:41:17.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:17.055+0000 7ff17d661640 1 --2- 172.21.15.38:0/1466906664 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff1781f5250 0x7ff1780d0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:17.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:17.055+0000 7ff17ce60640 1 --2- 172.21.15.38:0/1466906664 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff1780b2e40 0x7ff1780ac1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:17.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:17.055+0000 7ff177fff640 1 -- 172.21.15.38:0/1466906664 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff1781f5250 msgr2=0x7ff1780d0c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:17.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:17.055+0000 7ff177fff640 1 --2- 172.21.15.38:0/1466906664 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff1781f5250 0x7ff1780d0c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:17.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:17.055+0000 7ff177fff640 1 -- 172.21.15.38:0/1466906664 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff1780b2e40 msgr2=0x7ff1780ac1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:17.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:17.055+0000 7ff177fff640 1 --2- 172.21.15.38:0/1466906664 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff1780b2e40 0x7ff1780ac1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:17.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:17.055+0000 7ff177fff640 1 -- 172.21.15.38:0/1466906664 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1780cbc50 con 0x7ff178162280 2024-09-21T03:41:17.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:17.056+0000 7ff177fff640 1 --2- 172.21.15.38:0/1466906664 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff178162280 0x7ff1781f4d10 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7ff16c00c460 tx=0x7ff16c00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:17.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:17.056+0000 7ff1757fa640 1 -- 172.21.15.38:0/1466906664 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff16c016020 con 0x7ff178162280 2024-09-21T03:41:17.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:17.056+0000 7ff17f0c4640 1 -- 172.21.15.38:0/1466906664 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1781f5a40 con 0x7ff178162280 2024-09-21T03:41:17.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:17.056+0000 7ff1757fa640 1 -- 172.21.15.38:0/1466906664 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff16c012940 con 0x7ff178162280 2024-09-21T03:41:17.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:17.056+0000 7ff1757fa640 1 -- 172.21.15.38:0/1466906664 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff16c012d90 con 0x7ff178162280 2024-09-21T03:41:17.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:17.056+0000 7ff17f0c4640 1 -- 172.21.15.38:0/1466906664 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1780cc1d0 con 0x7ff178162280 2024-09-21T03:41:17.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:17.057+0000 7ff17f0c4640 1 -- 172.21.15.38:0/1466906664 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff178162f00 con 0x7ff178162280 2024-09-21T03:41:17.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:17.058+0000 7ff1757fa640 1 -- 172.21.15.38:0/1466906664 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ff16c019020 con 0x7ff178162280 2024-09-21T03:41:17.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:17.058+0000 7ff1757fa640 1 --2- 172.21.15.38:0/1466906664 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff164078b90 0x7ff16407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:17.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:17.058+0000 7ff1757fa640 1 -- 172.21.15.38:0/1466906664 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 7164+0+0 (secure 0 0 0) 0x7ff16c09afb0 con 0x7ff178162280 2024-09-21T03:41:17.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:17.058+0000 7ff17ce60640 1 --2- 172.21.15.38:0/1466906664 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff164078b90 0x7ff16407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:17.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:17.059+0000 7ff17ce60640 1 --2- 172.21.15.38:0/1466906664 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff164078b90 0x7ff16407b0a0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7ff168005fd0 tx=0x7ff168005850 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:17.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:17.063+0000 7ff1757fa640 1 -- 172.21.15.38:0/1466906664 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7ff16c020030 con 0x7ff178162280 2024-09-21T03:41:17.284 INFO:tasks.ceph.mds.c.smithi038.stderr: -44> 2024-09-21T03:41:12.531+0000 7f35096e3640 -1 mds.0.snaptable: load_2 could not read table: -2 2024-09-21T03:41:17.284 INFO:tasks.ceph.mds.c.smithi038.stderr: -43> 2024-09-21T03:41:12.531+0000 7f35096e3640 -1 log_channel(cluster) log [ERR] : error reading table object 'mds_snaptable' -2 ((2) No such file or directory) 2024-09-21T03:41:17.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:17.290+0000 7ff17f0c4640 1 -- 172.21.15.38:0/1466906664 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff1781f57d0 con 0x7ff178162280 2024-09-21T03:41:17.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:17.291+0000 7ff1757fa640 1 -- 172.21.15.38:0/1466906664 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 141 v141) ==== 95+0+3566 (secure 0 0 0) 0x7ff16c062e50 con 0x7ff178162280 2024-09-21T03:41:17.293 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:41:17.293 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":141,"btime":"2024-09-21T03:41:16:566084+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6086,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/965972663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":965972663},{"type":"v1","addr":"172.21.15.195:6836","nonce":965972663}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6700,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/3176122334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":3176122334},{"type":"v1","addr":"172.21.15.195:6837","nonce":3176122334}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":141,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:40:02.319282+0000","modified":"2024-09-21T03:41:16.566009+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":106,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-21T03:41:17.293 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 141 2024-09-21T03:41:17.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:17.294+0000 7ff17f0c4640 1 -- 172.21.15.38:0/1466906664 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff164078b90 msgr2=0x7ff16407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:17.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:17.294+0000 7ff17f0c4640 1 --2- 172.21.15.38:0/1466906664 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff164078b90 0x7ff16407b0a0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7ff168005fd0 tx=0x7ff168005850 comp rx=0 tx=0).stop 2024-09-21T03:41:17.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:17.294+0000 7ff17f0c4640 1 -- 172.21.15.38:0/1466906664 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff178162280 msgr2=0x7ff1781f4d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:17.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:17.294+0000 7ff17f0c4640 1 --2- 172.21.15.38:0/1466906664 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff178162280 0x7ff1781f4d10 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7ff16c00c460 tx=0x7ff16c00c920 comp rx=0 tx=0).stop 2024-09-21T03:41:17.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:17.295+0000 7ff17f0c4640 1 -- 172.21.15.38:0/1466906664 shutdown_connections 2024-09-21T03:41:17.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:17.295+0000 7ff17f0c4640 1 --2- 172.21.15.38:0/1466906664 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff164078b90 0x7ff16407b0a0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:17.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:17.295+0000 7ff17f0c4640 1 --2- 172.21.15.38:0/1466906664 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff1781f5250 0x7ff1780d0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:17.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:17.295+0000 7ff17f0c4640 1 --2- 172.21.15.38:0/1466906664 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff178162280 0x7ff1781f4d10 unknown :-1 s=CLOSED pgs=1454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:17.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:17.295+0000 7ff17f0c4640 1 --2- 172.21.15.38:0/1466906664 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff1780b2e40 0x7ff1780ac1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:17.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:17.295+0000 7ff17f0c4640 1 -- 172.21.15.38:0/1466906664 >> 172.21.15.38:0/1466906664 conn(0x7ff1780b6b20 msgr2=0x7ff17806df40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:17.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:17.295+0000 7ff17f0c4640 1 -- 172.21.15.38:0/1466906664 shutdown_connections 2024-09-21T03:41:17.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:17.295+0000 7ff17f0c4640 1 -- 172.21.15.38:0/1466906664 wait complete. 2024-09-21T03:41:17.312 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-21T03:41:17.312 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:41:17.346 INFO:tasks.ceph.mds.c.smithi038.stdout:starting mds.c at 2024-09-21T03:41:17.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:17.429+0000 7f0bc8d41640 1 -- 172.21.15.38:0/2850573814 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0bc4162590 msgr2=0x7f0bc41629a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:17.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:17.429+0000 7f0bc8d41640 1 --2- 172.21.15.38:0/2850573814 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0bc4162590 0x7f0bc41629a0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f0bac009830 tx=0x7f0bac02a540 comp rx=0 tx=0).stop 2024-09-21T03:41:17.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:17.429+0000 7f0bc8d41640 1 -- 172.21.15.38:0/2850573814 shutdown_connections 2024-09-21T03:41:17.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:17.429+0000 7f0bc8d41640 1 --2- 172.21.15.38:0/2850573814 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0bc4162590 0x7f0bc41629a0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:17.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:17.429+0000 7f0bc8d41640 1 -- 172.21.15.38:0/2850573814 >> 172.21.15.38:0/2850573814 conn(0x7f0bc40d4440 msgr2=0x7f0bc40d4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:17.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:17.429+0000 7f0bc8d41640 1 -- 172.21.15.38:0/2850573814 shutdown_connections 2024-09-21T03:41:17.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:17.429+0000 7f0bc8d41640 1 -- 172.21.15.38:0/2850573814 wait complete. 2024-09-21T03:41:17.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:17.431+0000 7f0bc8d41640 1 Processor -- start 2024-09-21T03:41:17.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:17.431+0000 7f0bc8d41640 1 -- start start 2024-09-21T03:41:17.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:17.432+0000 7f0bc8d41640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0bc40b2e60 0x7f0bc4154bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:17.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:17.432+0000 7f0bc8d41640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0bc40cfe00 0x7f0bc41550f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:17.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:17.432+0000 7f0bc8d41640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0bc4162590 0x7f0bc4155630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:17.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:17.432+0000 7f0bc8d41640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f0bc40b3b00 con 0x7f0bc4162590 2024-09-21T03:41:17.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:17.432+0000 7f0bc8d41640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f0bc40b3980 con 0x7f0bc40b2e60 2024-09-21T03:41:17.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:17.432+0000 7f0bc8d41640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f0bc40b4f70 con 0x7f0bc40cfe00 2024-09-21T03:41:17.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:17.432+0000 7f0bc2d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0bc4162590 0x7f0bc4155630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:17.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:17.432+0000 7f0bc2d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0bc4162590 0x7f0bc4155630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47034/0 (socket says 172.21.15.38:47034) 2024-09-21T03:41:17.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:17.432+0000 7f0bc2d9d640 1 -- 172.21.15.38:0/3510196857 learned_addr learned my addr 172.21.15.38:0/3510196857 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:41:17.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:17.432+0000 7f0bb9d9b640 1 --2- 172.21.15.38:0/3510196857 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0bc40cfe00 0x7f0bc41550f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:17.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:17.432+0000 7f0bc259c640 1 --2- 172.21.15.38:0/3510196857 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0bc40b2e60 0x7f0bc4154bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:17.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:17.433+0000 7f0bc2d9d640 1 -- 172.21.15.38:0/3510196857 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0bc40cfe00 msgr2=0x7f0bc41550f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:17.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:17.433+0000 7f0bc2d9d640 1 --2- 172.21.15.38:0/3510196857 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0bc40cfe00 0x7f0bc41550f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:17.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:17.433+0000 7f0bc2d9d640 1 -- 172.21.15.38:0/3510196857 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0bc40b2e60 msgr2=0x7f0bc4154bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:17.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:17.433+0000 7f0bc2d9d640 1 --2- 172.21.15.38:0/3510196857 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0bc40b2e60 0x7f0bc4154bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:17.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:17.433+0000 7f0bc2d9d640 1 -- 172.21.15.38:0/3510196857 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bc41e94a0 con 0x7f0bc4162590 2024-09-21T03:41:17.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:17.433+0000 7f0bc2d9d640 1 --2- 172.21.15.38:0/3510196857 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0bc4162590 0x7f0bc4155630 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc00cbc0 tx=0x7f0bbc00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:17.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:17.433+0000 7f0bbb7fe640 1 -- 172.21.15.38:0/3510196857 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bbc016020 con 0x7f0bc4162590 2024-09-21T03:41:17.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:17.433+0000 7f0bc8d41640 1 -- 172.21.15.38:0/3510196857 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bc4153180 con 0x7f0bc4162590 2024-09-21T03:41:17.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:17.433+0000 7f0bbb7fe640 1 -- 172.21.15.38:0/3510196857 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0bbc00adf0 con 0x7f0bc4162590 2024-09-21T03:41:17.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:17.433+0000 7f0bbb7fe640 1 -- 172.21.15.38:0/3510196857 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bbc012660 con 0x7f0bc4162590 2024-09-21T03:41:17.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:17.433+0000 7f0bc8d41640 1 -- 172.21.15.38:0/3510196857 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bc41543b0 con 0x7f0bc4162590 2024-09-21T03:41:17.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:17.434+0000 7f0bc8d41640 1 -- 172.21.15.38:0/3510196857 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0bc41629a0 con 0x7f0bc4162590 2024-09-21T03:41:17.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:17.435+0000 7f0bbb7fe640 1 -- 172.21.15.38:0/3510196857 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f0bbc019020 con 0x7f0bc4162590 2024-09-21T03:41:17.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:17.436+0000 7f0bbb7fe640 1 --2- 172.21.15.38:0/3510196857 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0ba4078bc0 0x7f0ba407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:17.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:17.436+0000 7f0bbb7fe640 1 -- 172.21.15.38:0/3510196857 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 7164+0+0 (secure 0 0 0) 0x7f0bbc09b790 con 0x7f0bc4162590 2024-09-21T03:41:17.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:17.436+0000 7f0bc259c640 1 --2- 172.21.15.38:0/3510196857 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0ba4078bc0 0x7f0ba407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:17.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:17.436+0000 7f0bc259c640 1 --2- 172.21.15.38:0/3510196857 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0ba4078bc0 0x7f0ba407b0d0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f0bb4000f60 tx=0x7f0bb4009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:17.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:17.440+0000 7f0bbb7fe640 1 -- 172.21.15.38:0/3510196857 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f0bbc020030 con 0x7f0bc4162590 2024-09-21T03:41:17.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:17.666+0000 7f0bc8d41640 1 -- 172.21.15.38:0/3510196857 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0bc4076180 con 0x7f0bc4162590 2024-09-21T03:41:17.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:17.666+0000 7f0bbb7fe640 1 -- 172.21.15.38:0/3510196857 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 142 v142) ==== 95+0+4392 (secure 0 0 0) 0x7f0bc41e94a0 con 0x7f0bc4162590 2024-09-21T03:41:17.668 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:41:17.668 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":142,"btime":"2024-09-21T03:41:17:567966+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6086,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/965972663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":965972663},{"type":"v1","addr":"172.21.15.195:6836","nonce":965972663}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6700,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/3176122334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":3176122334},{"type":"v1","addr":"172.21.15.195:6837","nonce":3176122334}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":8319,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/3402963947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":3402963947},{"type":"v1","addr":"172.21.15.38:6833","nonce":3402963947}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":142}],"filesystems":[{"mdsmap":{"epoch":141,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:40:02.319282+0000","modified":"2024-09-21T03:41:16.566009+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":106,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-21T03:41:17.669 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 142 2024-09-21T03:41:17.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:17.668+0000 7f0bc8d41640 1 -- 172.21.15.38:0/3510196857 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0ba4078bc0 msgr2=0x7f0ba407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:17.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:17.668+0000 7f0bc8d41640 1 --2- 172.21.15.38:0/3510196857 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0ba4078bc0 0x7f0ba407b0d0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f0bb4000f60 tx=0x7f0bb4009040 comp rx=0 tx=0).stop 2024-09-21T03:41:17.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:17.668+0000 7f0bc8d41640 1 -- 172.21.15.38:0/3510196857 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0bc4162590 msgr2=0x7f0bc4155630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:17.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:17.668+0000 7f0bc8d41640 1 --2- 172.21.15.38:0/3510196857 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0bc4162590 0x7f0bc4155630 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc00cbc0 tx=0x7f0bbc00a430 comp rx=0 tx=0).stop 2024-09-21T03:41:17.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:17.668+0000 7f0bc8d41640 1 -- 172.21.15.38:0/3510196857 shutdown_connections 2024-09-21T03:41:17.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:17.668+0000 7f0bc8d41640 1 --2- 172.21.15.38:0/3510196857 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0bc4162590 0x7f0bc4155630 unknown :-1 s=CLOSED pgs=1459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:17.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:17.668+0000 7f0bc8d41640 1 --2- 172.21.15.38:0/3510196857 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0ba4078bc0 0x7f0ba407b0d0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:17.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:17.668+0000 7f0bc8d41640 1 --2- 172.21.15.38:0/3510196857 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0bc40cfe00 0x7f0bc41550f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:17.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:17.668+0000 7f0bc8d41640 1 --2- 172.21.15.38:0/3510196857 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0bc40b2e60 0x7f0bc4154bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:17.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:17.668+0000 7f0bc8d41640 1 -- 172.21.15.38:0/3510196857 >> 172.21.15.38:0/3510196857 conn(0x7f0bc40d4440 msgr2=0x7f0bc40b3270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:17.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:17.668+0000 7f0bc8d41640 1 -- 172.21.15.38:0/3510196857 shutdown_connections 2024-09-21T03:41:17.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:17.668+0000 7f0bc8d41640 1 -- 172.21.15.38:0/3510196857 wait complete. 2024-09-21T03:41:17.681 INFO:tasks.cephfs.test_damage:Result: Mutation 'Delete mds0_inotable' led to DAMAGED state 2024-09-21T03:41:17.681 INFO:tasks.cephfs.test_damage:Applying mutation 'Delete 100.00000000' 2024-09-21T03:41:17.682 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-21T03:41:17.682 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:41:17.738 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056795,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2354 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T03:41:17.738 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi038.front.sepia.ceph.com 2024-09-21T03:41:17.738 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:41:17.738 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T03:41:17.793 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T03:41:17.793 INFO:teuthology.orchestra.run.smithi038.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-09-21T03:41:17.793 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-21T03:41:17.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:17.949+0000 7fd160589640 1 -- 172.21.15.38:0/433277068 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd140003a30 msgr2=0x7fd140003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:17.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:17.949+0000 7fd160589640 1 --2- 172.21.15.38:0/433277068 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd140003a30 0x7fd140003ea0 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7fd148009b50 tx=0x7fd140005830 comp rx=0 tx=0).stop 2024-09-21T03:41:17.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:17.949+0000 7fd15eb26640 1 -- 172.21.15.38:0/433277068 reap_dead start 2024-09-21T03:41:17.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:17.949+0000 7fd160589640 1 -- 172.21.15.38:0/433277068 shutdown_connections 2024-09-21T03:41:17.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:17.949+0000 7fd160589640 1 -- 172.21.15.38:0/433277068 >> 172.21.15.38:0/433277068 conn(0x7fd1580cdca0 msgr2=0x7fd1580ce0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:17.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:17.949+0000 7fd160589640 1 -- 172.21.15.38:0/433277068 shutdown_connections 2024-09-21T03:41:17.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:17.949+0000 7fd160589640 1 -- 172.21.15.38:0/433277068 wait complete. 2024-09-21T03:41:17.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:17.951+0000 7fd160589640 1 Processor -- start 2024-09-21T03:41:17.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:17.951+0000 7fd160589640 1 -- start start 2024-09-21T03:41:17.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:17.952+0000 7fd160589640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd140003a30 0x7fd1581f5c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:17.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:17.952+0000 7fd160589640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd1581f2d30 0x7fd1581f6bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:17.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:17.952+0000 7fd160589640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd1580b23a0 0x7fd1580b27b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:17.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:17.952+0000 7fd160589640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fd1580aed00 con 0x7fd1581f2d30 2024-09-21T03:41:17.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:17.952+0000 7fd160589640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd1580aeb80 con 0x7fd140003a30 2024-09-21T03:41:17.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:17.952+0000 7fd160589640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fd158070610 con 0x7fd1580b23a0 2024-09-21T03:41:17.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:17.952+0000 7fd15db24640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd1581f2d30 0x7fd1581f6bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:17.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:17.952+0000 7fd15db24640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd1581f2d30 0x7fd1581f6bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47056/0 (socket says 172.21.15.38:47056) 2024-09-21T03:41:17.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:17.952+0000 7fd15db24640 1 -- 172.21.15.38:0/1345249259 learned_addr learned my addr 172.21.15.38:0/1345249259 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:41:17.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:17.952+0000 7fd15eb26640 1 --2- 172.21.15.38:0/1345249259 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd1580b23a0 0x7fd1580b27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:17.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:17.952+0000 7fd15e325640 1 --2- 172.21.15.38:0/1345249259 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd140003a30 0x7fd1581f5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:17.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:17.953+0000 7fd15db24640 1 -- 172.21.15.38:0/1345249259 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd1580b23a0 msgr2=0x7fd1580b27b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:17.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:17.953+0000 7fd15db24640 1 --2- 172.21.15.38:0/1345249259 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd1580b23a0 0x7fd1580b27b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:17.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:17.953+0000 7fd15db24640 1 -- 172.21.15.38:0/1345249259 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd140003a30 msgr2=0x7fd1581f5c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:17.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:17.953+0000 7fd15db24640 1 --2- 172.21.15.38:0/1345249259 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd140003a30 0x7fd1581f5c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:17.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:17.953+0000 7fd15db24640 1 -- 172.21.15.38:0/1345249259 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1580b2cf0 con 0x7fd1581f2d30 2024-09-21T03:41:17.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:17.953+0000 7fd15db24640 1 --2- 172.21.15.38:0/1345249259 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd1581f2d30 0x7fd1581f6bf0 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7fd14c0018c0 tx=0x7fd14c001cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:17.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:17.953+0000 7fd156ffd640 1 -- 172.21.15.38:0/1345249259 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd14c015070 con 0x7fd1581f2d30 2024-09-21T03:41:17.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:17.953+0000 7fd156ffd640 1 -- 172.21.15.38:0/1345249259 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd14c010980 con 0x7fd1581f2d30 2024-09-21T03:41:17.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:17.953+0000 7fd160589640 1 -- 172.21.15.38:0/1345249259 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1580caf60 con 0x7fd1581f2d30 2024-09-21T03:41:17.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:17.953+0000 7fd156ffd640 1 -- 172.21.15.38:0/1345249259 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd14c010d70 con 0x7fd1581f2d30 2024-09-21T03:41:17.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:17.953+0000 7fd160589640 1 -- 172.21.15.38:0/1345249259 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1581f34e0 con 0x7fd1581f2d30 2024-09-21T03:41:17.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:17.954+0000 7fd160589640 1 -- 172.21.15.38:0/1345249259 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd158070b70 con 0x7fd1581f2d30 2024-09-21T03:41:17.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:17.955+0000 7fd156ffd640 1 -- 172.21.15.38:0/1345249259 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fd14c018070 con 0x7fd1581f2d30 2024-09-21T03:41:17.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:17.955+0000 7fd156ffd640 1 --2- 172.21.15.38:0/1345249259 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd144078bc0 0x7fd14407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:17.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:17.956+0000 7fd156ffd640 1 -- 172.21.15.38:0/1345249259 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 7164+0+0 (secure 0 0 0) 0x7fd14c09b9d0 con 0x7fd1581f2d30 2024-09-21T03:41:17.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:17.956+0000 7fd15e325640 1 --2- 172.21.15.38:0/1345249259 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd144078bc0 0x7fd14407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:17.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:17.956+0000 7fd15e325640 1 --2- 172.21.15.38:0/1345249259 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd144078bc0 0x7fd14407b0d0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fd148035000 tx=0x7fd14802d040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:17.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:17.960+0000 7fd156ffd640 1 -- 172.21.15.38:0/1345249259 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fd1580b2cf0 con 0x7fd1581f2d30 2024-09-21T03:41:18.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:18.183+0000 7fd160589640 1 -- 172.21.15.38:0/1345249259 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fd158159ed0 con 0x7fd1581f2d30 2024-09-21T03:41:18.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:18.580+0000 7fd156ffd640 1 -- 172.21.15.38:0/1345249259 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v143) ==== 161+0+0 (secure 0 0 0) 0x7fd14c0632a0 con 0x7fd1581f2d30 2024-09-21T03:41:18.583 INFO:teuthology.orchestra.run.smithi038.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-21T03:41:18.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:18.583+0000 7fd160589640 1 -- 172.21.15.38:0/1345249259 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd144078bc0 msgr2=0x7fd14407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:18.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:18.583+0000 7fd160589640 1 --2- 172.21.15.38:0/1345249259 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd144078bc0 0x7fd14407b0d0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fd148035000 tx=0x7fd14802d040 comp rx=0 tx=0).stop 2024-09-21T03:41:18.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:18.583+0000 7fd160589640 1 -- 172.21.15.38:0/1345249259 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd1581f2d30 msgr2=0x7fd1581f6bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:18.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:18.583+0000 7fd160589640 1 --2- 172.21.15.38:0/1345249259 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd1581f2d30 0x7fd1581f6bf0 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7fd14c0018c0 tx=0x7fd14c001cc0 comp rx=0 tx=0).stop 2024-09-21T03:41:18.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:18.583+0000 7fd160589640 1 -- 172.21.15.38:0/1345249259 shutdown_connections 2024-09-21T03:41:18.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:18.583+0000 7fd160589640 1 --2- 172.21.15.38:0/1345249259 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd144078bc0 0x7fd14407b0d0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:18.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:18.583+0000 7fd160589640 1 --2- 172.21.15.38:0/1345249259 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd1580b23a0 0x7fd1580b27b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:18.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:18.583+0000 7fd160589640 1 --2- 172.21.15.38:0/1345249259 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd1581f2d30 0x7fd1581f6bf0 unknown :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:18.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:18.583+0000 7fd160589640 1 --2- 172.21.15.38:0/1345249259 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd140003a30 0x7fd1581f5c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:18.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:18.583+0000 7fd160589640 1 -- 172.21.15.38:0/1345249259 >> 172.21.15.38:0/1345249259 conn(0x7fd1580cdca0 msgr2=0x7fd15806ff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:18.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:18.583+0000 7fd160589640 1 -- 172.21.15.38:0/1345249259 shutdown_connections 2024-09-21T03:41:18.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:18.583+0000 7fd160589640 1 -- 172.21.15.38:0/1345249259 wait complete. 2024-09-21T03:41:18.597 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-09-21T03:41:18.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:18.715+0000 7feae4f12640 1 -- 172.21.15.38:0/3788633274 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feac80053a0 msgr2=0x7feac800ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:18.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:18.715+0000 7feae4f12640 1 --2- 172.21.15.38:0/3788633274 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feac80053a0 0x7feac800ba50 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7fead8009ed0 tx=0x7feac8005830 comp rx=0 tx=0).stop 2024-09-21T03:41:18.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:18.715+0000 7feae4f12640 1 -- 172.21.15.38:0/3788633274 shutdown_connections 2024-09-21T03:41:18.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:18.715+0000 7feae4f12640 1 --2- 172.21.15.38:0/3788633274 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feac80053a0 0x7feac800ba50 unknown :-1 s=CLOSED pgs=1464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:18.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:18.715+0000 7feae4f12640 1 -- 172.21.15.38:0/3788633274 >> 172.21.15.38:0/3788633274 conn(0x7feae009d830 msgr2=0x7feae00b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:18.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:18.715+0000 7feae4f12640 1 -- 172.21.15.38:0/3788633274 shutdown_connections 2024-09-21T03:41:18.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:18.715+0000 7feae4f12640 1 -- 172.21.15.38:0/3788633274 wait complete. 2024-09-21T03:41:18.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:18.717+0000 7feae4f12640 1 Processor -- start 2024-09-21T03:41:18.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:18.717+0000 7feae4f12640 1 -- start start 2024-09-21T03:41:18.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:18.718+0000 7feae4f12640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7feac80053a0 0x7feae006e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:18.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:18.718+0000 7feae4f12640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feae0157990 0x7feae015bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:18.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:18.718+0000 7feae4f12640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feae01573d0 0x7feae00cb610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:18.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:18.718+0000 7feae4f12640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7feae00b9c50 con 0x7feae01573d0 2024-09-21T03:41:18.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:18.718+0000 7feae4f12640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7feae00b9ad0 con 0x7feae0157990 2024-09-21T03:41:18.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:18.718+0000 7feae4f12640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7feae00b9dd0 con 0x7feac80053a0 2024-09-21T03:41:18.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:18.719+0000 7feaded9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feae01573d0 0x7feae00cb610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:18.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:18.719+0000 7feaded9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feae01573d0 0x7feae00cb610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47074/0 (socket says 172.21.15.38:47074) 2024-09-21T03:41:18.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:18.719+0000 7feaded9d640 1 -- 172.21.15.38:0/838283885 learned_addr learned my addr 172.21.15.38:0/838283885 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:41:18.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:18.719+0000 7feaddd9b640 1 --2- 172.21.15.38:0/838283885 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feae0157990 0x7feae015bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:18.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:18.719+0000 7feade59c640 1 --2- 172.21.15.38:0/838283885 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7feac80053a0 0x7feae006e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:18.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:18.719+0000 7feaded9d640 1 -- 172.21.15.38:0/838283885 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7feac80053a0 msgr2=0x7feae006e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:18.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:18.719+0000 7feaded9d640 1 --2- 172.21.15.38:0/838283885 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7feac80053a0 0x7feae006e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:18.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:18.719+0000 7feaded9d640 1 -- 172.21.15.38:0/838283885 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feae0157990 msgr2=0x7feae015bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:18.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:18.719+0000 7feaded9d640 1 --2- 172.21.15.38:0/838283885 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feae0157990 0x7feae015bc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:18.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:18.719+0000 7feaded9d640 1 -- 172.21.15.38:0/838283885 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feae006dbd0 con 0x7feae01573d0 2024-09-21T03:41:18.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:18.719+0000 7feaded9d640 1 --2- 172.21.15.38:0/838283885 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feae01573d0 0x7feae00cb610 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7fead802af20 tx=0x7fead8004230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:18.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:18.719+0000 7feaceffd640 1 -- 172.21.15.38:0/838283885 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fead8009200 con 0x7feae01573d0 2024-09-21T03:41:18.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:18.719+0000 7feaceffd640 1 -- 172.21.15.38:0/838283885 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fead8004740 con 0x7feae01573d0 2024-09-21T03:41:18.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:18.719+0000 7feae4f12640 1 -- 172.21.15.38:0/838283885 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feae006f620 con 0x7feae01573d0 2024-09-21T03:41:18.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:18.719+0000 7feaceffd640 1 -- 172.21.15.38:0/838283885 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fead803c6e0 con 0x7feae01573d0 2024-09-21T03:41:18.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:18.719+0000 7feae4f12640 1 -- 172.21.15.38:0/838283885 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feae00cc030 con 0x7feae01573d0 2024-09-21T03:41:18.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:18.721+0000 7feae4f12640 1 -- 172.21.15.38:0/838283885 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feae006d940 con 0x7feae01573d0 2024-09-21T03:41:18.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:18.721+0000 7feaceffd640 1 -- 172.21.15.38:0/838283885 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fead8044020 con 0x7feae01573d0 2024-09-21T03:41:18.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:18.722+0000 7feaceffd640 1 --2- 172.21.15.38:0/838283885 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7feac0078b90 0x7feac007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:18.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:18.722+0000 7feaceffd640 1 -- 172.21.15.38:0/838283885 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 7164+0+0 (secure 0 0 0) 0x7fead80bc1c0 con 0x7feae01573d0 2024-09-21T03:41:18.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:18.722+0000 7feade59c640 1 --2- 172.21.15.38:0/838283885 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7feac0078b90 0x7feac007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:18.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:18.723+0000 7feade59c640 1 --2- 172.21.15.38:0/838283885 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7feac0078b90 0x7feac007b0a0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fead0006680 tx=0x7fead0006b70 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:18.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:18.727+0000 7feaceffd640 1 -- 172.21.15.38:0/838283885 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fead804b020 con 0x7feae01573d0 2024-09-21T03:41:18.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:18.989+0000 7feae4f12640 1 -- 172.21.15.38:0/838283885 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7feae00cc290 con 0x7feae01573d0 2024-09-21T03:41:19.589 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.587+0000 7feaceffd640 1 -- 172.21.15.38:0/838283885 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v144) ==== 101+0+0 (secure 0 0 0) 0x7fead80840f0 con 0x7feae01573d0 2024-09-21T03:41:19.589 INFO:teuthology.orchestra.run.smithi038.stderr:repaired: restoring rank 6:0 2024-09-21T03:41:19.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.590+0000 7feae4f12640 1 -- 172.21.15.38:0/838283885 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7feac0078b90 msgr2=0x7feac007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:19.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.590+0000 7feae4f12640 1 --2- 172.21.15.38:0/838283885 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7feac0078b90 0x7feac007b0a0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fead0006680 tx=0x7fead0006b70 comp rx=0 tx=0).stop 2024-09-21T03:41:19.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.590+0000 7feae4f12640 1 -- 172.21.15.38:0/838283885 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feae01573d0 msgr2=0x7feae00cb610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:19.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.590+0000 7feae4f12640 1 --2- 172.21.15.38:0/838283885 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feae01573d0 0x7feae00cb610 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7fead802af20 tx=0x7fead8004230 comp rx=0 tx=0).stop 2024-09-21T03:41:19.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.590+0000 7feae4f12640 1 -- 172.21.15.38:0/838283885 shutdown_connections 2024-09-21T03:41:19.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.590+0000 7feae4f12640 1 --2- 172.21.15.38:0/838283885 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feae01573d0 0x7feae00cb610 unknown :-1 s=CLOSED pgs=1465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:19.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.590+0000 7feae4f12640 1 --2- 172.21.15.38:0/838283885 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feae0157990 0x7feae015bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:19.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.590+0000 7feae4f12640 1 --2- 172.21.15.38:0/838283885 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7feac0078b90 0x7feac007b0a0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:19.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.590+0000 7feae4f12640 1 --2- 172.21.15.38:0/838283885 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7feac80053a0 0x7feae006e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:19.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.590+0000 7feae4f12640 1 -- 172.21.15.38:0/838283885 >> 172.21.15.38:0/838283885 conn(0x7feae009d830 msgr2=0x7feae00b31a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:19.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.590+0000 7feae4f12640 1 -- 172.21.15.38:0/838283885 shutdown_connections 2024-09-21T03:41:19.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.590+0000 7feae4f12640 1 -- 172.21.15.38:0/838283885 wait complete. 2024-09-21T03:41:19.605 DEBUG:teuthology.orchestra.run.smithi038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-09-21T03:41:19.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.661+0000 7f5aa19dcf40 1 Processor -- start 2024-09-21T03:41:19.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.661+0000 7f5aa19dcf40 1 -- start start 2024-09-21T03:41:19.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.662+0000 7f5aa19dcf40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561489109d00 0x561489106310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:19.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.662+0000 7f5aa19dcf40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5614891c4950 0x5614891c2a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:19.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.662+0000 7f5aa19dcf40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5614891df670 0x5614891dfc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:19.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.662+0000 7f5aa19dcf40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x561489057d60 con 0x5614891df670 2024-09-21T03:41:19.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.662+0000 7f5aa19dcf40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5614890b6430 con 0x561489109d00 2024-09-21T03:41:19.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.662+0000 7f5aa19dcf40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x5614891521d0 con 0x5614891c4950 2024-09-21T03:41:19.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.662+0000 7f5a9bc99640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5614891df670 0x5614891dfc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:19.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.662+0000 7f5a9bc99640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5614891df670 0x5614891dfc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47100/0 (socket says 172.21.15.38:47100) 2024-09-21T03:41:19.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.662+0000 7f5a9bc99640 1 -- 172.21.15.38:0/2750972926 learned_addr learned my addr 172.21.15.38:0/2750972926 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:41:19.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.663+0000 7f5a9ac97640 1 --2- 172.21.15.38:0/2750972926 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5614891c4950 0x5614891c2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:19.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.663+0000 7f5a9b498640 1 --2- 172.21.15.38:0/2750972926 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561489109d00 0x561489106310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:19.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.663+0000 7f5a9bc99640 1 -- 172.21.15.38:0/2750972926 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5614891c4950 msgr2=0x5614891c2a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:19.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.663+0000 7f5a9bc99640 1 --2- 172.21.15.38:0/2750972926 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5614891c4950 0x5614891c2a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:19.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.663+0000 7f5a9bc99640 1 -- 172.21.15.38:0/2750972926 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561489109d00 msgr2=0x561489106310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:19.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.663+0000 7f5a9bc99640 1 --2- 172.21.15.38:0/2750972926 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561489109d00 0x561489106310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:19.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.663+0000 7f5a9bc99640 1 -- 172.21.15.38:0/2750972926 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561489131ef0 con 0x5614891df670 2024-09-21T03:41:19.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.663+0000 7f5a9ac97640 1 --2- 172.21.15.38:0/2750972926 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5614891c4950 0x5614891c2a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:41:19.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.663+0000 7f5a9b498640 1 --2- 172.21.15.38:0/2750972926 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561489109d00 0x561489106310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:41:19.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.663+0000 7f5a9bc99640 1 --2- 172.21.15.38:0/2750972926 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5614891df670 0x5614891dfc70 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f5a94001510 tx=0x7f5a940019d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=434d7bb21ab32316 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:19.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.663+0000 7f5a99c95640 1 -- 172.21.15.38:0/2750972926 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a94017070 con 0x5614891df670 2024-09-21T03:41:19.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.663+0000 7f5a99c95640 1 -- 172.21.15.38:0/2750972926 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5a940027a0 con 0x5614891df670 2024-09-21T03:41:19.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.663+0000 7f5a99c95640 1 -- 172.21.15.38:0/2750972926 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a94002c30 con 0x5614891df670 2024-09-21T03:41:19.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.664+0000 7f5aa19dcf40 1 -- 172.21.15.38:0/2750972926 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5614891df670 msgr2=0x5614891dfc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:19.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.664+0000 7f5aa19dcf40 1 --2- 172.21.15.38:0/2750972926 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5614891df670 0x5614891dfc70 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f5a94001510 tx=0x7f5a940019d0 comp rx=0 tx=0).stop 2024-09-21T03:41:19.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.664+0000 7f5aa19dcf40 1 -- 172.21.15.38:0/2750972926 shutdown_connections 2024-09-21T03:41:19.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.664+0000 7f5aa19dcf40 1 --2- 172.21.15.38:0/2750972926 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5614891df670 0x5614891dfc70 unknown :-1 s=CLOSED pgs=1468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:19.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.664+0000 7f5aa19dcf40 1 --2- 172.21.15.38:0/2750972926 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5614891c4950 0x5614891c2a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:19.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.664+0000 7f5aa19dcf40 1 --2- 172.21.15.38:0/2750972926 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561489109d00 0x561489106310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:19.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.664+0000 7f5aa19dcf40 1 -- 172.21.15.38:0/2750972926 >> 172.21.15.38:0/2750972926 conn(0x561489028ac0 msgr2=0x5614891c05c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:19.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.664+0000 7f5aa19dcf40 1 -- 172.21.15.38:0/2750972926 shutdown_connections 2024-09-21T03:41:19.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.664+0000 7f5aa19dcf40 1 -- 172.21.15.38:0/2750972926 wait complete. 2024-09-21T03:41:19.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.665+0000 7f5aa19dcf40 1 Processor -- start 2024-09-21T03:41:19.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.665+0000 7f5aa19dcf40 1 -- start start 2024-09-21T03:41:19.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.666+0000 7f5aa19dcf40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561489109d00 0x5614891c0e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:19.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.666+0000 7f5aa19dcf40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5614891c4950 0x5614891c13d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:19.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.666+0000 7f5aa19dcf40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5614891df670 0x56148914d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:19.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.666+0000 7f5aa19dcf40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5614891dbc10 con 0x561489109d00 2024-09-21T03:41:19.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.666+0000 7f5aa19dcf40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5614891363d0 con 0x5614891c4950 2024-09-21T03:41:19.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.666+0000 7f5aa19dcf40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x5614891dd9f0 con 0x5614891df670 2024-09-21T03:41:19.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.666+0000 7f5a9b498640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561489109d00 0x5614891c0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:19.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.666+0000 7f5a9b498640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561489109d00 0x5614891c0e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47110/0 (socket says 172.21.15.38:47110) 2024-09-21T03:41:19.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.666+0000 7f5a9b498640 1 -- 172.21.15.38:0/1789342087 learned_addr learned my addr 172.21.15.38:0/1789342087 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:41:19.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.666+0000 7f5a9bc99640 1 --2- 172.21.15.38:0/1789342087 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5614891df670 0x56148914d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:19.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.666+0000 7f5a9ac97640 1 --2- 172.21.15.38:0/1789342087 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5614891c4950 0x5614891c13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:19.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.666+0000 7f5a9b498640 1 -- 172.21.15.38:0/1789342087 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5614891df670 msgr2=0x56148914d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:19.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.666+0000 7f5a9b498640 1 --2- 172.21.15.38:0/1789342087 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5614891df670 0x56148914d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:19.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.666+0000 7f5a9b498640 1 -- 172.21.15.38:0/1789342087 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5614891c4950 msgr2=0x5614891c13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:19.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.666+0000 7f5a9b498640 1 --2- 172.21.15.38:0/1789342087 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5614891c4950 0x5614891c13d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:19.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.666+0000 7f5a9b498640 1 -- 172.21.15.38:0/1789342087 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561489137dc0 con 0x561489109d00 2024-09-21T03:41:19.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.666+0000 7f5a9ac97640 1 --2- 172.21.15.38:0/1789342087 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5614891c4950 0x5614891c13d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:41:19.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.667+0000 7f5a9b498640 1 --2- 172.21.15.38:0/1789342087 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561489109d00 0x5614891c0e90 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f5a8c025f60 tx=0x7f5a8c04d9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:19.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.667+0000 7f5a98c93640 1 -- 172.21.15.38:0/1789342087 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a8c066070 con 0x561489109d00 2024-09-21T03:41:19.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.667+0000 7f5a98c93640 1 -- 172.21.15.38:0/1789342087 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5a8c0567e0 con 0x561489109d00 2024-09-21T03:41:19.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.667+0000 7f5aa19dcf40 1 -- 172.21.15.38:0/1789342087 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56148914bf70 con 0x561489109d00 2024-09-21T03:41:19.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.667+0000 7f5a98c93640 1 -- 172.21.15.38:0/1789342087 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a8c056bf0 con 0x561489109d00 2024-09-21T03:41:19.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.667+0000 7f5aa19dcf40 1 -- 172.21.15.38:0/1789342087 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614891dc190 con 0x561489109d00 2024-09-21T03:41:19.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.669+0000 7f5a98c93640 1 -- 172.21.15.38:0/1789342087 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f5a8c069070 con 0x561489109d00 2024-09-21T03:41:19.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.669+0000 7f5a98c93640 1 --2- 172.21.15.38:0/1789342087 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5a880775a0 0x7f5a88079ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:19.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.669+0000 7f5a98c93640 1 -- 172.21.15.38:0/1789342087 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 7164+0+0 (secure 0 0 0) 0x7f5a8c0dd960 con 0x561489109d00 2024-09-21T03:41:19.673 INFO:teuthology.orchestra.run.smithi038.stdout:Importing pool 2024-09-21T03:41:19.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.669+0000 7f5aa19dcf40 1 --2- 172.21.15.38:0/1789342087 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x56148914e390 0x561489282510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:19.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.669+0000 7f5aa19dcf40 1 -- 172.21.15.38:0/1789342087 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x561489132790 con 0x56148914e390 2024-09-21T03:41:19.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.670+0000 7f5a9ac97640 1 --2- 172.21.15.38:0/1789342087 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5a880775a0 0x7f5a88079ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:19.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.670+0000 7f5a9bc99640 1 --2- 172.21.15.38:0/1789342087 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x56148914e390 0x561489282510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:19.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.670+0000 7f5a9ac97640 1 --2- 172.21.15.38:0/1789342087 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5a880775a0 0x7f5a88079ab0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x56148914e9c0 tx=0x7f5a90006700 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:19.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.670+0000 7f5a9bc99640 1 --2- 172.21.15.38:0/1789342087 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x56148914e390 0x561489282510 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:19.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.672+0000 7f5a9bc99640 1 -- 172.21.15.38:0/1789342087 <== osd.7 v2:172.21.15.195:6818/1606106342 1 ==== osd_op_reply(1 601.00000000 [create] v106'288 uv262 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f5a9400f070 con 0x56148914e390 2024-09-21T03:41:19.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.672+0000 7f5aa19dcf40 1 -- 172.21.15.38:0/1789342087 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e106) -- 0x561489282a50 con 0x56148914e390 2024-09-21T03:41:19.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.673+0000 7f5a9bc99640 1 -- 172.21.15.38:0/1789342087 <== osd.7 v2:172.21.15.195:6818/1606106342 2 ==== osd_op_reply(2 601.00000000 [delete] v106'289 uv289 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5a9400f070 con 0x56148914e390 2024-09-21T03:41:19.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.673+0000 7f5aa19dcf40 1 -- 172.21.15.38:0/1789342087 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x561489282df0 con 0x56148914e390 2024-09-21T03:41:19.677 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-09-21T03:41:19.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.675+0000 7f5a9bc99640 1 -- 172.21.15.38:0/1789342087 <== osd.7 v2:172.21.15.195:6818/1606106342 3 ==== osd_op_reply(3 601.00000000 [create] v106'290 uv290 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5a9400f070 con 0x56148914e390 2024-09-21T03:41:19.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.675+0000 7f5aa19dcf40 1 -- 172.21.15.38:0/1789342087 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x5614892841a0 con 0x56148914e390 2024-09-21T03:41:19.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.676+0000 7f5a9bc99640 1 -- 172.21.15.38:0/1789342087 <== osd.7 v2:172.21.15.195:6818/1606106342 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v106'291 uv291 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5a9400f070 con 0x56148914e390 2024-09-21T03:41:19.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.677+0000 7f5aa19dcf40 1 -- 172.21.15.38:0/1789342087 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x561489284540 con 0x56148914e390 2024-09-21T03:41:19.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.678+0000 7f5a9bc99640 1 -- 172.21.15.38:0/1789342087 <== osd.7 v2:172.21.15.195:6818/1606106342 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v106'292 uv292 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5a9400f070 con 0x56148914e390 2024-09-21T03:41:19.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.678+0000 7f5aa19dcf40 1 -- 172.21.15.38:0/1789342087 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x561489283190 con 0x56148914e390 2024-09-21T03:41:19.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.679+0000 7f5a9bc99640 1 -- 172.21.15.38:0/1789342087 <== osd.7 v2:172.21.15.195:6818/1606106342 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v106'293 uv293 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5a9400f070 con 0x56148914e390 2024-09-21T03:41:19.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.679+0000 7f5aa19dcf40 1 -- 172.21.15.38:0/1789342087 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x561489283580 con 0x56148914e390 2024-09-21T03:41:19.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.681+0000 7f5a9bc99640 1 -- 172.21.15.38:0/1789342087 <== osd.7 v2:172.21.15.195:6818/1606106342 7 ==== osd_op_reply(7 602.00000000 [create] v106'294 uv268 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f5a9400f070 con 0x56148914e390 2024-09-21T03:41:19.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.681+0000 7f5aa19dcf40 1 -- 172.21.15.38:0/1789342087 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e106) -- 0x561489283970 con 0x56148914e390 2024-09-21T03:41:19.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.682+0000 7f5a9bc99640 1 -- 172.21.15.38:0/1789342087 <== osd.7 v2:172.21.15.195:6818/1606106342 8 ==== osd_op_reply(8 602.00000000 [delete] v106'295 uv295 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5a9400f070 con 0x56148914e390 2024-09-21T03:41:19.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.682+0000 7f5aa19dcf40 1 -- 172.21.15.38:0/1789342087 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x561489283970 con 0x56148914e390 2024-09-21T03:41:19.685 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-09-21T03:41:19.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.683+0000 7f5a9bc99640 1 -- 172.21.15.38:0/1789342087 <== osd.7 v2:172.21.15.195:6818/1606106342 9 ==== osd_op_reply(9 602.00000000 [create] v106'296 uv296 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5a9400f070 con 0x56148914e390 2024-09-21T03:41:19.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.684+0000 7f5aa19dcf40 1 -- 172.21.15.38:0/1789342087 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x561489283920 con 0x56148914e390 2024-09-21T03:41:19.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.685+0000 7f5a9bc99640 1 -- 172.21.15.38:0/1789342087 <== osd.7 v2:172.21.15.195:6818/1606106342 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v106'297 uv297 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5a9400f070 con 0x56148914e390 2024-09-21T03:41:19.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.685+0000 7f5aa19dcf40 1 -- 172.21.15.38:0/1789342087 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x561489283920 con 0x56148914e390 2024-09-21T03:41:19.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.686+0000 7f5a9bc99640 1 -- 172.21.15.38:0/1789342087 <== osd.7 v2:172.21.15.195:6818/1606106342 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v106'298 uv298 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5a9400f070 con 0x56148914e390 2024-09-21T03:41:19.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.686+0000 7f5aa19dcf40 1 -- 172.21.15.38:0/1789342087 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x561489283920 con 0x56148914e390 2024-09-21T03:41:19.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.688+0000 7f5a9bc99640 1 -- 172.21.15.38:0/1789342087 <== osd.7 v2:172.21.15.195:6818/1606106342 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v106'299 uv299 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5a9400f070 con 0x56148914e390 2024-09-21T03:41:19.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.688+0000 7f5aa19dcf40 1 -- 172.21.15.38:0/1789342087 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:13 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x561489283920 con 0x56148914e390 2024-09-21T03:41:19.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.689+0000 7f5a9bc99640 1 -- 172.21.15.38:0/1789342087 <== osd.7 v2:172.21.15.195:6818/1606106342 13 ==== osd_op_reply(13 10000000000.00000000 [create] v106'300 uv275 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7f5a9400f070 con 0x56148914e390 2024-09-21T03:41:19.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.689+0000 7f5aa19dcf40 1 -- 172.21.15.38:0/1789342087 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:14 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e106) -- 0x561489283920 con 0x56148914e390 2024-09-21T03:41:19.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.691+0000 7f5a9bc99640 1 -- 172.21.15.38:0/1789342087 <== osd.7 v2:172.21.15.195:6818/1606106342 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v106'301 uv301 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f5a9400f070 con 0x56148914e390 2024-09-21T03:41:19.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.691+0000 7f5aa19dcf40 1 -- 172.21.15.38:0/1789342087 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:15 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x561489283920 con 0x56148914e390 2024-09-21T03:41:19.694 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-09-21T03:41:19.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.692+0000 7f5a9bc99640 1 -- 172.21.15.38:0/1789342087 <== osd.7 v2:172.21.15.195:6818/1606106342 15 ==== osd_op_reply(15 10000000000.00000000 [create] v106'302 uv302 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f5a9400f070 con 0x56148914e390 2024-09-21T03:41:19.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.692+0000 7f5aa19dcf40 1 -- 172.21.15.38:0/1789342087 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:16 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x561489283920 con 0x56148914e390 2024-09-21T03:41:19.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.694+0000 7f5a9bc99640 1 -- 172.21.15.38:0/1789342087 <== osd.7 v2:172.21.15.195:6818/1606106342 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v106'303 uv303 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f5a9400f070 con 0x56148914e390 2024-09-21T03:41:19.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.694+0000 7f5aa19dcf40 1 -- 172.21.15.38:0/1789342087 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:17 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x561489283920 con 0x56148914e390 2024-09-21T03:41:19.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.696+0000 7f5a9bc99640 1 -- 172.21.15.38:0/1789342087 <== osd.7 v2:172.21.15.195:6818/1606106342 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v106'304 uv304 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f5a9400f070 con 0x56148914e390 2024-09-21T03:41:19.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.696+0000 7f5aa19dcf40 1 -- 172.21.15.38:0/1789342087 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:18 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x561489283920 con 0x56148914e390 2024-09-21T03:41:19.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.697+0000 7f5a9bc99640 1 -- 172.21.15.38:0/1789342087 <== osd.7 v2:172.21.15.195:6818/1606106342 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v106'305 uv305 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f5a9400f070 con 0x56148914e390 2024-09-21T03:41:19.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.697+0000 7f5aa19dcf40 1 -- 172.21.15.38:0/1789342087 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:19 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=585b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x561489283920 con 0x56148914e390 2024-09-21T03:41:19.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.698+0000 7f5a9bc99640 1 -- 172.21.15.38:0/1789342087 <== osd.7 v2:172.21.15.195:6818/1606106342 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v106'306 uv306 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f5a9400f070 con 0x56148914e390 2024-09-21T03:41:19.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.698+0000 7f5aa19dcf40 1 -- 172.21.15.38:0/1789342087 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:20 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x561489283920 con 0x56148914e390 2024-09-21T03:41:19.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.700+0000 7f5a9bc99640 1 -- 172.21.15.38:0/1789342087 <== osd.7 v2:172.21.15.195:6818/1606106342 20 ==== osd_op_reply(20 600.00000000 [create] v106'307 uv281 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f5a9400f070 con 0x56148914e390 2024-09-21T03:41:19.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.700+0000 7f5aa19dcf40 1 -- 172.21.15.38:0/1789342087 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:21 13.10 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e106) -- 0x561489283920 con 0x56148914e390 2024-09-21T03:41:19.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.701+0000 7f5a9bc99640 1 -- 172.21.15.38:0/1789342087 <== osd.7 v2:172.21.15.195:6818/1606106342 21 ==== osd_op_reply(21 600.00000000 [delete] v106'308 uv308 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5a9400f070 con 0x56148914e390 2024-09-21T03:41:19.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.701+0000 7f5aa19dcf40 1 -- 172.21.15.38:0/1789342087 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:22 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x561489283920 con 0x56148914e390 2024-09-21T03:41:19.704 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-09-21T03:41:19.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.703+0000 7f5a9bc99640 1 -- 172.21.15.38:0/1789342087 <== osd.7 v2:172.21.15.195:6818/1606106342 22 ==== osd_op_reply(22 600.00000000 [create] v106'309 uv309 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5a9400f070 con 0x56148914e390 2024-09-21T03:41:19.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.703+0000 7f5aa19dcf40 1 -- 172.21.15.38:0/1789342087 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:23 13.10 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x561489283920 con 0x56148914e390 2024-09-21T03:41:19.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.704+0000 7f5a9bc99640 1 -- 172.21.15.38:0/1789342087 <== osd.7 v2:172.21.15.195:6818/1606106342 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v106'310 uv310 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5a9400f070 con 0x56148914e390 2024-09-21T03:41:19.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.704+0000 7f5aa19dcf40 1 -- 172.21.15.38:0/1789342087 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:24 13.10 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x561489283920 con 0x56148914e390 2024-09-21T03:41:19.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.705+0000 7f5a9bc99640 1 -- 172.21.15.38:0/1789342087 <== osd.7 v2:172.21.15.195:6818/1606106342 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v106'311 uv311 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5a9400f070 con 0x56148914e390 2024-09-21T03:41:19.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.706+0000 7f5aa19dcf40 1 -- 172.21.15.38:0/1789342087 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:25 13.10 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x561489283920 con 0x56148914e390 2024-09-21T03:41:19.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.707+0000 7f5a9bc99640 1 -- 172.21.15.38:0/1789342087 <== osd.7 v2:172.21.15.195:6818/1606106342 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v106'312 uv312 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5a9400f070 con 0x56148914e390 2024-09-21T03:41:19.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.707+0000 7f5aa19dcf40 1 -- 172.21.15.38:0/1789342087 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:26 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x561489283920 con 0x56148914e390 2024-09-21T03:41:19.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.708+0000 7f5a9bc99640 1 -- 172.21.15.38:0/1789342087 <== osd.7 v2:172.21.15.195:6818/1606106342 26 ==== osd_op_reply(26 603.00000000 [create] v106'313 uv287 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f5a9400f070 con 0x56148914e390 2024-09-21T03:41:19.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.708+0000 7f5aa19dcf40 1 -- 172.21.15.38:0/1789342087 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:27 13.10 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e106) -- 0x561489283920 con 0x56148914e390 2024-09-21T03:41:19.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.710+0000 7f5a9bc99640 1 -- 172.21.15.38:0/1789342087 <== osd.7 v2:172.21.15.195:6818/1606106342 27 ==== osd_op_reply(27 603.00000000 [delete] v106'314 uv314 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5a9400f070 con 0x56148914e390 2024-09-21T03:41:19.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.710+0000 7f5aa19dcf40 1 -- 172.21.15.38:0/1789342087 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:28 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x561489283920 con 0x56148914e390 2024-09-21T03:41:19.713 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-09-21T03:41:19.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.711+0000 7f5a9bc99640 1 -- 172.21.15.38:0/1789342087 <== osd.7 v2:172.21.15.195:6818/1606106342 28 ==== osd_op_reply(28 603.00000000 [create] v106'315 uv315 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5a9400f070 con 0x56148914e390 2024-09-21T03:41:19.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.711+0000 7f5aa19dcf40 1 -- 172.21.15.38:0/1789342087 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:29 13.10 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x561489283920 con 0x56148914e390 2024-09-21T03:41:19.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.713+0000 7f5a9bc99640 1 -- 172.21.15.38:0/1789342087 <== osd.7 v2:172.21.15.195:6818/1606106342 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v106'316 uv316 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5a9400f070 con 0x56148914e390 2024-09-21T03:41:19.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.713+0000 7f5aa19dcf40 1 -- 172.21.15.38:0/1789342087 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:30 13.10 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x561489283920 con 0x56148914e390 2024-09-21T03:41:19.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.714+0000 7f5a9bc99640 1 -- 172.21.15.38:0/1789342087 <== osd.7 v2:172.21.15.195:6818/1606106342 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v106'317 uv317 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5a9400f070 con 0x56148914e390 2024-09-21T03:41:19.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.714+0000 7f5aa19dcf40 1 -- 172.21.15.38:0/1789342087 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:31 13.10 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x561489283920 con 0x56148914e390 2024-09-21T03:41:19.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.715+0000 7f5a9bc99640 1 -- 172.21.15.38:0/1789342087 <== osd.7 v2:172.21.15.195:6818/1606106342 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v106'318 uv318 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5a9400f070 con 0x56148914e390 2024-09-21T03:41:19.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.715+0000 7f5aa19dcf40 1 --2- 172.21.15.38:0/1789342087 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x561489283d20 0x561489286e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:19.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.716+0000 7f5aa19dcf40 1 -- 172.21.15.38:0/1789342087 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x561489283920 con 0x561489283d20 2024-09-21T03:41:19.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.716+0000 7f5a9b498640 1 --2- 172.21.15.38:0/1789342087 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x561489283d20 0x561489286e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:19.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.716+0000 7f5a9b498640 1 --2- 172.21.15.38:0/1789342087 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x561489283d20 0x561489286e00 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:19.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.718+0000 7f5a9b498640 1 -- 172.21.15.38:0/1789342087 <== osd.6 v2:172.21.15.195:6826/4055825675 1 ==== osd_op_reply(32 1.00000000.inode [create] v106'218 uv201 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f5a8c068030 con 0x561489283d20 2024-09-21T03:41:19.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.718+0000 7f5aa19dcf40 1 -- 172.21.15.38:0/1789342087 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e106) -- 0x561489287340 con 0x561489283d20 2024-09-21T03:41:19.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.719+0000 7f5a9b498640 1 -- 172.21.15.38:0/1789342087 <== osd.6 v2:172.21.15.195:6826/4055825675 2 ==== osd_op_reply(33 1.00000000.inode [delete] v106'219 uv219 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f5a8c068030 con 0x561489283d20 2024-09-21T03:41:19.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.719+0000 7f5aa19dcf40 1 -- 172.21.15.38:0/1789342087 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x5614892876e0 con 0x561489283d20 2024-09-21T03:41:19.722 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-09-21T03:41:19.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.720+0000 7f5a9b498640 1 -- 172.21.15.38:0/1789342087 <== osd.6 v2:172.21.15.195:6826/4055825675 3 ==== osd_op_reply(34 1.00000000.inode [create] v106'220 uv220 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f5a8c068030 con 0x561489283d20 2024-09-21T03:41:19.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.720+0000 7f5aa19dcf40 1 -- 172.21.15.38:0/1789342087 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x561489287f80 con 0x561489283d20 2024-09-21T03:41:19.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.722+0000 7f5a9b498640 1 -- 172.21.15.38:0/1789342087 <== osd.6 v2:172.21.15.195:6826/4055825675 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v106'221 uv221 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f5a8c068030 con 0x561489283d20 2024-09-21T03:41:19.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.722+0000 7f5aa19dcf40 1 -- 172.21.15.38:0/1789342087 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x561489288320 con 0x561489283d20 2024-09-21T03:41:19.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.723+0000 7f5a9b498640 1 -- 172.21.15.38:0/1789342087 <== osd.6 v2:172.21.15.195:6826/4055825675 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v106'222 uv222 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f5a8c068030 con 0x561489283d20 2024-09-21T03:41:19.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.723+0000 7f5aa19dcf40 1 -- 172.21.15.38:0/1789342087 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x5614892886c0 con 0x561489283d20 2024-09-21T03:41:19.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.724+0000 7f5a9b498640 1 -- 172.21.15.38:0/1789342087 <== osd.6 v2:172.21.15.195:6826/4055825675 6 ==== osd_op_reply(37 200.00000000 [create] v106'223 uv206 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f5a8c068030 con 0x561489283d20 2024-09-21T03:41:19.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.724+0000 7f5aa19dcf40 1 -- 172.21.15.38:0/1789342087 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e106) -- 0x561489288a60 con 0x561489283d20 2024-09-21T03:41:19.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.726+0000 7f5a9b498640 1 -- 172.21.15.38:0/1789342087 <== osd.6 v2:172.21.15.195:6826/4055825675 7 ==== osd_op_reply(38 200.00000000 [delete] v106'224 uv224 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5a8c068030 con 0x561489283d20 2024-09-21T03:41:19.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.726+0000 7f5aa19dcf40 1 -- 172.21.15.38:0/1789342087 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x561489288a60 con 0x561489283d20 2024-09-21T03:41:19.728 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-09-21T03:41:19.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.727+0000 7f5a9b498640 1 -- 172.21.15.38:0/1789342087 <== osd.6 v2:172.21.15.195:6826/4055825675 8 ==== osd_op_reply(39 200.00000000 [create] v106'225 uv225 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5a8c068030 con 0x561489283d20 2024-09-21T03:41:19.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.727+0000 7f5aa19dcf40 1 -- 172.21.15.38:0/1789342087 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x561489288ea0 con 0x561489283d20 2024-09-21T03:41:19.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.728+0000 7f5a9b498640 1 -- 172.21.15.38:0/1789342087 <== osd.6 v2:172.21.15.195:6826/4055825675 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v106'226 uv226 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5a8c068030 con 0x561489283d20 2024-09-21T03:41:19.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.728+0000 7f5aa19dcf40 1 -- 172.21.15.38:0/1789342087 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x561489288ea0 con 0x561489283d20 2024-09-21T03:41:19.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.729+0000 7f5a9b498640 1 -- 172.21.15.38:0/1789342087 <== osd.6 v2:172.21.15.195:6826/4055825675 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v106'227 uv227 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5a8c068030 con 0x561489283d20 2024-09-21T03:41:19.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.729+0000 7f5aa19dcf40 1 -- 172.21.15.38:0/1789342087 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:42 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x561489288be0 con 0x561489283d20 2024-09-21T03:41:19.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.731+0000 7f5a9b498640 1 -- 172.21.15.38:0/1789342087 <== osd.6 v2:172.21.15.195:6826/4055825675 11 ==== osd_op_reply(42 200.00000001 [create] v106'228 uv211 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f5a8c068030 con 0x561489283d20 2024-09-21T03:41:19.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.731+0000 7f5aa19dcf40 1 -- 172.21.15.38:0/1789342087 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:43 13.14 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e106) -- 0x561489288be0 con 0x561489283d20 2024-09-21T03:41:19.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.732+0000 7f5a9b498640 1 -- 172.21.15.38:0/1789342087 <== osd.6 v2:172.21.15.195:6826/4055825675 12 ==== osd_op_reply(43 200.00000001 [delete] v106'229 uv229 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5a8c068030 con 0x561489283d20 2024-09-21T03:41:19.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.732+0000 7f5aa19dcf40 1 -- 172.21.15.38:0/1789342087 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:44 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x561489288b90 con 0x561489283d20 2024-09-21T03:41:19.736 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-09-21T03:41:19.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.734+0000 7f5a9b498640 1 -- 172.21.15.38:0/1789342087 <== osd.6 v2:172.21.15.195:6826/4055825675 13 ==== osd_op_reply(44 200.00000001 [create] v106'230 uv230 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5a8c068030 con 0x561489283d20 2024-09-21T03:41:19.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.734+0000 7f5aa19dcf40 1 -- 172.21.15.38:0/1789342087 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:45 13.14 13:2e2fa760:::200.00000001:head [write 0~17490 in=17490b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x56148928d400 con 0x561489283d20 2024-09-21T03:41:19.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.735+0000 7f5a9b498640 1 -- 172.21.15.38:0/1789342087 <== osd.6 v2:172.21.15.195:6826/4055825675 14 ==== osd_op_reply(45 200.00000001 [write 0~17490] v106'231 uv231 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5a8c068030 con 0x561489283d20 2024-09-21T03:41:19.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.735+0000 7f5aa19dcf40 1 -- 172.21.15.38:0/1789342087 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:46 13.14 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x56148928d400 con 0x561489283d20 2024-09-21T03:41:19.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.737+0000 7f5a9b498640 1 -- 172.21.15.38:0/1789342087 <== osd.6 v2:172.21.15.195:6826/4055825675 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v106'232 uv232 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5a8c068030 con 0x561489283d20 2024-09-21T03:41:19.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.737+0000 7f5aa19dcf40 1 -- 172.21.15.38:0/1789342087 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:47 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x561489288c20 con 0x561489283d20 2024-09-21T03:41:19.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.738+0000 7f5a9b498640 1 -- 172.21.15.38:0/1789342087 <== osd.6 v2:172.21.15.195:6826/4055825675 16 ==== osd_op_reply(47 606.00000000 [create] v106'233 uv217 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f5a8c068030 con 0x561489283d20 2024-09-21T03:41:19.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.738+0000 7f5aa19dcf40 1 -- 172.21.15.38:0/1789342087 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:48 13.14 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e106) -- 0x561489288bd0 con 0x561489283d20 2024-09-21T03:41:19.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.739+0000 7f5a9b498640 1 -- 172.21.15.38:0/1789342087 <== osd.6 v2:172.21.15.195:6826/4055825675 17 ==== osd_op_reply(48 606.00000000 [delete] v106'234 uv234 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5a8c068030 con 0x561489283d20 2024-09-21T03:41:19.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.739+0000 7f5aa19dcf40 1 -- 172.21.15.38:0/1789342087 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:49 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x561489288bd0 con 0x561489283d20 2024-09-21T03:41:19.742 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-09-21T03:41:19.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.740+0000 7f5a9b498640 1 -- 172.21.15.38:0/1789342087 <== osd.6 v2:172.21.15.195:6826/4055825675 18 ==== osd_op_reply(49 606.00000000 [create] v106'235 uv235 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5a8c068030 con 0x561489283d20 2024-09-21T03:41:19.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.740+0000 7f5aa19dcf40 1 -- 172.21.15.38:0/1789342087 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:50 13.14 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x561489288bd0 con 0x561489283d20 2024-09-21T03:41:19.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.741+0000 7f5a9b498640 1 -- 172.21.15.38:0/1789342087 <== osd.6 v2:172.21.15.195:6826/4055825675 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v106'236 uv236 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5a8c068030 con 0x561489283d20 2024-09-21T03:41:19.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.741+0000 7f5aa19dcf40 1 -- 172.21.15.38:0/1789342087 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:51 13.14 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x561489289be0 con 0x561489283d20 2024-09-21T03:41:19.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.743+0000 7f5a9b498640 1 -- 172.21.15.38:0/1789342087 <== osd.6 v2:172.21.15.195:6826/4055825675 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v106'237 uv237 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5a8c068030 con 0x561489283d20 2024-09-21T03:41:19.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.743+0000 7f5aa19dcf40 1 -- 172.21.15.38:0/1789342087 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:52 13.14 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x561489288bd0 con 0x561489283d20 2024-09-21T03:41:19.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.744+0000 7f5a9b498640 1 -- 172.21.15.38:0/1789342087 <== osd.6 v2:172.21.15.195:6826/4055825675 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v106'238 uv238 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5a8c068030 con 0x561489283d20 2024-09-21T03:41:19.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.744+0000 7f5aa19dcf40 1 --2- 172.21.15.38:0/1789342087 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x5614892892a0 0x5614892896b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:19.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.744+0000 7f5aa19dcf40 1 -- 172.21.15.38:0/1789342087 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:53 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x56148928bfe0 con 0x5614892892a0 2024-09-21T03:41:19.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.744+0000 7f5a9ac97640 1 --2- 172.21.15.38:0/1789342087 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x5614892892a0 0x5614892896b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:19.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.744+0000 7f5a9ac97640 1 --2- 172.21.15.38:0/1789342087 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x5614892892a0 0x5614892896b0 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:19.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.746+0000 7f5a9ac97640 1 -- 172.21.15.38:0/1789342087 <== osd.0 v2:172.21.15.38:6800/2381639433 1 ==== osd_op_reply(53 607.00000000 [create] v106'57 uv56 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f5a90011070 con 0x5614892892a0 2024-09-21T03:41:19.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.746+0000 7f5aa19dcf40 1 -- 172.21.15.38:0/1789342087 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:54 13.1c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e106) -- 0x56148928c380 con 0x5614892892a0 2024-09-21T03:41:19.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.747+0000 7f5a9ac97640 1 -- 172.21.15.38:0/1789342087 <== osd.0 v2:172.21.15.38:6800/2381639433 2 ==== osd_op_reply(54 607.00000000 [delete] v106'58 uv58 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5a90011070 con 0x5614892892a0 2024-09-21T03:41:19.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.747+0000 7f5aa19dcf40 1 -- 172.21.15.38:0/1789342087 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:55 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x56148928c720 con 0x5614892892a0 2024-09-21T03:41:19.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.748+0000 7f5a9ac97640 1 -- 172.21.15.38:0/1789342087 <== osd.0 v2:172.21.15.38:6800/2381639433 3 ==== osd_op_reply(55 607.00000000 [create] v106'59 uv59 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5a90011070 con 0x5614892892a0 2024-09-21T03:41:19.750 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-09-21T03:41:19.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.748+0000 7f5aa19dcf40 1 -- 172.21.15.38:0/1789342087 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:56 13.1c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x56148928e000 con 0x5614892892a0 2024-09-21T03:41:19.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.749+0000 7f5a9ac97640 1 -- 172.21.15.38:0/1789342087 <== osd.0 v2:172.21.15.38:6800/2381639433 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v106'60 uv60 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5a90011070 con 0x5614892892a0 2024-09-21T03:41:19.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.749+0000 7f5aa19dcf40 1 -- 172.21.15.38:0/1789342087 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:57 13.1c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x56148928e3a0 con 0x5614892892a0 2024-09-21T03:41:19.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.750+0000 7f5a9ac97640 1 -- 172.21.15.38:0/1789342087 <== osd.0 v2:172.21.15.38:6800/2381639433 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v106'61 uv61 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5a90011070 con 0x5614892892a0 2024-09-21T03:41:19.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.750+0000 7f5aa19dcf40 1 -- 172.21.15.38:0/1789342087 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:58 13.1c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x56148928cff0 con 0x5614892892a0 2024-09-21T03:41:19.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.751+0000 7f5a9ac97640 1 -- 172.21.15.38:0/1789342087 <== osd.0 v2:172.21.15.38:6800/2381639433 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v106'62 uv62 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5a90011070 con 0x5614892892a0 2024-09-21T03:41:19.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.751+0000 7f5aa19dcf40 1 --2- 172.21.15.38:0/1789342087 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x56148928d8f0 0x561489290750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:19.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.751+0000 7f5aa19dcf40 1 -- 172.21.15.38:0/1789342087 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:59 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x561489290c90 con 0x56148928d8f0 2024-09-21T03:41:19.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.751+0000 7f5a9bc99640 1 --2- 172.21.15.38:0/1789342087 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x56148928d8f0 0x561489290750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:19.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.752+0000 7f5a9bc99640 1 --2- 172.21.15.38:0/1789342087 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x56148928d8f0 0x561489290750 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:19.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.753+0000 7f5a9bc99640 1 -- 172.21.15.38:0/1789342087 <== osd.3 v2:172.21.15.38:6808/1286810294 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v106'44 uv43 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f5a9400f070 con 0x56148928d8f0 2024-09-21T03:41:19.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.753+0000 7f5aa19dcf40 1 -- 172.21.15.38:0/1789342087 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:60 13.6 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e106) -- 0x561489290c90 con 0x56148928d8f0 2024-09-21T03:41:19.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.754+0000 7f5a9bc99640 1 -- 172.21.15.38:0/1789342087 <== osd.3 v2:172.21.15.38:6808/1286810294 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v106'45 uv45 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f5a9400f070 con 0x56148928d8f0 2024-09-21T03:41:19.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.754+0000 7f5aa19dcf40 1 -- 172.21.15.38:0/1789342087 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:61 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x561489291080 con 0x56148928d8f0 2024-09-21T03:41:19.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.755+0000 7f5a9bc99640 1 -- 172.21.15.38:0/1789342087 <== osd.3 v2:172.21.15.38:6808/1286810294 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v106'46 uv46 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f5a9400f070 con 0x56148928d8f0 2024-09-21T03:41:19.757 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-09-21T03:41:19.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.755+0000 7f5aa19dcf40 1 -- 172.21.15.38:0/1789342087 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:62 13.6 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x561489291080 con 0x56148928d8f0 2024-09-21T03:41:19.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.756+0000 7f5a9bc99640 1 -- 172.21.15.38:0/1789342087 <== osd.3 v2:172.21.15.38:6808/1286810294 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v106'47 uv47 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f5a9400f070 con 0x56148928d8f0 2024-09-21T03:41:19.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.756+0000 7f5aa19dcf40 1 -- 172.21.15.38:0/1789342087 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x561489291080 con 0x5614892892a0 2024-09-21T03:41:19.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.757+0000 7f5a9ac97640 1 -- 172.21.15.38:0/1789342087 <== osd.0 v2:172.21.15.38:6800/2381639433 7 ==== osd_op_reply(63 608.00000000 [create] v106'57 uv56 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f5a90011070 con 0x5614892892a0 2024-09-21T03:41:19.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.757+0000 7f5aa19dcf40 1 -- 172.21.15.38:0/1789342087 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e106) -- 0x561489291420 con 0x5614892892a0 2024-09-21T03:41:19.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.759+0000 7f5a9ac97640 1 -- 172.21.15.38:0/1789342087 <== osd.0 v2:172.21.15.38:6800/2381639433 8 ==== osd_op_reply(64 608.00000000 [delete] v106'58 uv58 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5a90011070 con 0x5614892892a0 2024-09-21T03:41:19.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.759+0000 7f5aa19dcf40 1 -- 172.21.15.38:0/1789342087 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x561489291420 con 0x5614892892a0 2024-09-21T03:41:19.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.760+0000 7f5a9ac97640 1 -- 172.21.15.38:0/1789342087 <== osd.0 v2:172.21.15.38:6800/2381639433 9 ==== osd_op_reply(65 608.00000000 [create] v106'59 uv59 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5a90011070 con 0x5614892892a0 2024-09-21T03:41:19.761 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-09-21T03:41:19.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.760+0000 7f5aa19dcf40 1 -- 172.21.15.38:0/1789342087 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x561489292430 con 0x5614892892a0 2024-09-21T03:41:19.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.761+0000 7f5a9ac97640 1 -- 172.21.15.38:0/1789342087 <== osd.0 v2:172.21.15.38:6800/2381639433 10 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v106'60 uv60 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5a90011070 con 0x5614892892a0 2024-09-21T03:41:19.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.761+0000 7f5aa19dcf40 1 -- 172.21.15.38:0/1789342087 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x561489292430 con 0x5614892892a0 2024-09-21T03:41:19.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.762+0000 7f5a9ac97640 1 -- 172.21.15.38:0/1789342087 <== osd.0 v2:172.21.15.38:6800/2381639433 11 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v106'61 uv61 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5a90011070 con 0x5614892892a0 2024-09-21T03:41:19.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.762+0000 7f5aa19dcf40 1 -- 172.21.15.38:0/1789342087 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x561489291420 con 0x5614892892a0 2024-09-21T03:41:19.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.763+0000 7f5a9ac97640 1 -- 172.21.15.38:0/1789342087 <== osd.0 v2:172.21.15.38:6800/2381639433 12 ==== osd_op_reply(68 608.00000000 [omap-set-header] v106'62 uv62 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5a90011070 con 0x5614892892a0 2024-09-21T03:41:19.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.763+0000 7f5aa19dcf40 1 -- 172.21.15.38:0/1789342087 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x561489291420 con 0x56148928d8f0 2024-09-21T03:41:19.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.764+0000 7f5a9bc99640 1 -- 172.21.15.38:0/1789342087 <== osd.3 v2:172.21.15.38:6808/1286810294 5 ==== osd_op_reply(69 604.00000000 [create] v106'105 uv99 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f5a9400f070 con 0x56148928d8f0 2024-09-21T03:41:19.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.764+0000 7f5aa19dcf40 1 -- 172.21.15.38:0/1789342087 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e106) -- 0x561489291420 con 0x56148928d8f0 2024-09-21T03:41:19.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.765+0000 7f5a9bc99640 1 -- 172.21.15.38:0/1789342087 <== osd.3 v2:172.21.15.38:6808/1286810294 6 ==== osd_op_reply(70 604.00000000 [delete] v106'106 uv106 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5a9400f070 con 0x56148928d8f0 2024-09-21T03:41:19.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.765+0000 7f5aa19dcf40 1 -- 172.21.15.38:0/1789342087 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x561489291420 con 0x56148928d8f0 2024-09-21T03:41:19.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.766+0000 7f5a9bc99640 1 -- 172.21.15.38:0/1789342087 <== osd.3 v2:172.21.15.38:6808/1286810294 7 ==== osd_op_reply(71 604.00000000 [create] v106'107 uv107 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5a9400f070 con 0x56148928d8f0 2024-09-21T03:41:19.768 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-09-21T03:41:19.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.766+0000 7f5aa19dcf40 1 -- 172.21.15.38:0/1789342087 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x561489292430 con 0x56148928d8f0 2024-09-21T03:41:19.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.768+0000 7f5a9bc99640 1 -- 172.21.15.38:0/1789342087 <== osd.3 v2:172.21.15.38:6808/1286810294 8 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v106'108 uv108 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5a9400f070 con 0x56148928d8f0 2024-09-21T03:41:19.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.768+0000 7f5aa19dcf40 1 -- 172.21.15.38:0/1789342087 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x561489292430 con 0x56148928d8f0 2024-09-21T03:41:19.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.769+0000 7f5a9bc99640 1 -- 172.21.15.38:0/1789342087 <== osd.3 v2:172.21.15.38:6808/1286810294 9 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v106'109 uv109 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5a9400f070 con 0x56148928d8f0 2024-09-21T03:41:19.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.769+0000 7f5aa19dcf40 1 -- 172.21.15.38:0/1789342087 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x561489291420 con 0x56148928d8f0 2024-09-21T03:41:19.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.770+0000 7f5a9bc99640 1 -- 172.21.15.38:0/1789342087 <== osd.3 v2:172.21.15.38:6808/1286810294 10 ==== osd_op_reply(74 604.00000000 [omap-set-header] v106'110 uv110 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5a9400f070 con 0x56148928d8f0 2024-09-21T03:41:19.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.770+0000 7f5aa19dcf40 1 -- 172.21.15.38:0/1789342087 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x561489291420 con 0x56148928d8f0 2024-09-21T03:41:19.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.771+0000 7f5a9bc99640 1 -- 172.21.15.38:0/1789342087 <== osd.3 v2:172.21.15.38:6808/1286810294 11 ==== osd_op_reply(75 500.00000000 [create] v106'111 uv104 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f5a9400f070 con 0x56148928d8f0 2024-09-21T03:41:19.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.771+0000 7f5aa19dcf40 1 -- 172.21.15.38:0/1789342087 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e106) -- 0x561489291420 con 0x56148928d8f0 2024-09-21T03:41:19.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.772+0000 7f5a9bc99640 1 -- 172.21.15.38:0/1789342087 <== osd.3 v2:172.21.15.38:6808/1286810294 12 ==== osd_op_reply(76 500.00000000 [delete] v106'112 uv112 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5a9400f070 con 0x56148928d8f0 2024-09-21T03:41:19.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.772+0000 7f5aa19dcf40 1 -- 172.21.15.38:0/1789342087 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x561489291420 con 0x56148928d8f0 2024-09-21T03:41:19.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.773+0000 7f5a9bc99640 1 -- 172.21.15.38:0/1789342087 <== osd.3 v2:172.21.15.38:6808/1286810294 13 ==== osd_op_reply(77 500.00000000 [create] v106'113 uv113 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5a9400f070 con 0x56148928d8f0 2024-09-21T03:41:19.775 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-09-21T03:41:19.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.773+0000 7f5aa19dcf40 1 -- 172.21.15.38:0/1789342087 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x561489291420 con 0x56148928d8f0 2024-09-21T03:41:19.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.774+0000 7f5a9bc99640 1 -- 172.21.15.38:0/1789342087 <== osd.3 v2:172.21.15.38:6808/1286810294 14 ==== osd_op_reply(78 500.00000000 [write 0~90] v106'114 uv114 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5a9400f070 con 0x56148928d8f0 2024-09-21T03:41:19.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.774+0000 7f5aa19dcf40 1 -- 172.21.15.38:0/1789342087 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x561489291420 con 0x56148928d8f0 2024-09-21T03:41:19.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.775+0000 7f5a9bc99640 1 -- 172.21.15.38:0/1789342087 <== osd.3 v2:172.21.15.38:6808/1286810294 15 ==== osd_op_reply(79 500.00000000 [omap-set-header] v106'115 uv115 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5a9400f070 con 0x56148928d8f0 2024-09-21T03:41:19.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.775+0000 7f5aa19dcf40 1 -- 172.21.15.38:0/1789342087 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:80 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x561489291420 con 0x5614892892a0 2024-09-21T03:41:19.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.776+0000 7f5a9ac97640 1 -- 172.21.15.38:0/1789342087 <== osd.0 v2:172.21.15.38:6800/2381639433 13 ==== osd_op_reply(80 mds_snaptable [create] v106'46 uv45 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f5a90011070 con 0x5614892892a0 2024-09-21T03:41:19.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.776+0000 7f5aa19dcf40 1 -- 172.21.15.38:0/1789342087 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:81 13.d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e106) -- 0x561489291420 con 0x5614892892a0 2024-09-21T03:41:19.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.778+0000 7f5a9ac97640 1 -- 172.21.15.38:0/1789342087 <== osd.0 v2:172.21.15.38:6800/2381639433 14 ==== osd_op_reply(81 mds_snaptable [delete] v106'47 uv47 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f5a90011070 con 0x5614892892a0 2024-09-21T03:41:19.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.778+0000 7f5aa19dcf40 1 -- 172.21.15.38:0/1789342087 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:82 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x561489291420 con 0x5614892892a0 2024-09-21T03:41:19.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.779+0000 7f5a9ac97640 1 -- 172.21.15.38:0/1789342087 <== osd.0 v2:172.21.15.38:6800/2381639433 15 ==== osd_op_reply(82 mds_snaptable [create] v106'48 uv48 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f5a90011070 con 0x5614892892a0 2024-09-21T03:41:19.780 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-09-21T03:41:19.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.779+0000 7f5aa19dcf40 1 -- 172.21.15.38:0/1789342087 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:83 13.d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x561489291420 con 0x5614892892a0 2024-09-21T03:41:19.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.780+0000 7f5a9ac97640 1 -- 172.21.15.38:0/1789342087 <== osd.0 v2:172.21.15.38:6800/2381639433 16 ==== osd_op_reply(83 mds_snaptable [write 0~70] v106'49 uv49 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f5a90011070 con 0x5614892892a0 2024-09-21T03:41:19.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.780+0000 7f5aa19dcf40 1 -- 172.21.15.38:0/1789342087 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:84 13.d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x561489291420 con 0x5614892892a0 2024-09-21T03:41:19.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.781+0000 7f5a9ac97640 1 -- 172.21.15.38:0/1789342087 <== osd.0 v2:172.21.15.38:6800/2381639433 17 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v106'50 uv50 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f5a90011070 con 0x5614892892a0 2024-09-21T03:41:19.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.781+0000 7f5aa19dcf40 1 --2- 172.21.15.38:0/1789342087 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x561489291d20 0x561489294140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:19.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.781+0000 7f5aa19dcf40 1 -- 172.21.15.38:0/1789342087 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x561489294680 con 0x561489291d20 2024-09-21T03:41:19.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.781+0000 7f5a9b498640 1 --2- 172.21.15.38:0/1789342087 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x561489291d20 0x561489294140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:19.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.781+0000 7f5a9b498640 1 --2- 172.21.15.38:0/1789342087 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x561489291d20 0x561489294140 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:19.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.783+0000 7f5a9b498640 1 -- 172.21.15.38:0/1789342087 <== osd.1 v2:172.21.15.38:6816/1491818732 1 ==== osd_op_reply(85 605.00000000 [create] v106'57 uv56 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f5a8c068030 con 0x561489291d20 2024-09-21T03:41:19.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.783+0000 7f5aa19dcf40 1 -- 172.21.15.38:0/1789342087 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e106) -- 0x561489294680 con 0x561489291d20 2024-09-21T03:41:19.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.784+0000 7f5a9b498640 1 -- 172.21.15.38:0/1789342087 <== osd.1 v2:172.21.15.38:6816/1491818732 2 ==== osd_op_reply(86 605.00000000 [delete] v106'58 uv58 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5a8c068030 con 0x561489291d20 2024-09-21T03:41:19.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.784+0000 7f5aa19dcf40 1 -- 172.21.15.38:0/1789342087 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x561489294aa0 con 0x561489291d20 2024-09-21T03:41:19.786 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-09-21T03:41:19.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.785+0000 7f5a9b498640 1 -- 172.21.15.38:0/1789342087 <== osd.1 v2:172.21.15.38:6816/1491818732 3 ==== osd_op_reply(87 605.00000000 [create] v106'59 uv59 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5a8c068030 con 0x561489291d20 2024-09-21T03:41:19.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.785+0000 7f5aa19dcf40 1 -- 172.21.15.38:0/1789342087 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x561489295ab0 con 0x561489291d20 2024-09-21T03:41:19.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.786+0000 7f5a9b498640 1 -- 172.21.15.38:0/1789342087 <== osd.1 v2:172.21.15.38:6816/1491818732 4 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v106'60 uv60 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5a8c068030 con 0x561489291d20 2024-09-21T03:41:19.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.786+0000 7f5aa19dcf40 1 -- 172.21.15.38:0/1789342087 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x561489295ab0 con 0x561489291d20 2024-09-21T03:41:19.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.787+0000 7f5a9b498640 1 -- 172.21.15.38:0/1789342087 <== osd.1 v2:172.21.15.38:6816/1491818732 5 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v106'61 uv61 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5a8c068030 con 0x561489291d20 2024-09-21T03:41:19.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.787+0000 7f5aa19dcf40 1 -- 172.21.15.38:0/1789342087 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x561489294aa0 con 0x561489291d20 2024-09-21T03:41:19.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.788+0000 7f5a9b498640 1 -- 172.21.15.38:0/1789342087 <== osd.1 v2:172.21.15.38:6816/1491818732 6 ==== osd_op_reply(90 605.00000000 [omap-set-header] v106'62 uv62 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5a8c068030 con 0x561489291d20 2024-09-21T03:41:19.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.788+0000 7f5aa19dcf40 1 -- 172.21.15.38:0/1789342087 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x561489294aa0 con 0x561489291d20 2024-09-21T03:41:19.791 INFO:teuthology.orchestra.run.smithi038.stdout:Write #-9223372036854775808:00000000:::mds0_inotable:head# 2024-09-21T03:41:19.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.789+0000 7f5a9b498640 1 -- 172.21.15.38:0/1789342087 <== osd.1 v2:172.21.15.38:6816/1491818732 7 ==== osd_op_reply(91 mds0_inotable [create] v106'95 uv95 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f5a8c068030 con 0x561489291d20 2024-09-21T03:41:19.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.789+0000 7f5aa19dcf40 1 -- 172.21.15.38:0/1789342087 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x561489294aa0 con 0x561489291d20 2024-09-21T03:41:19.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.790+0000 7f5a9b498640 1 -- 172.21.15.38:0/1789342087 <== osd.1 v2:172.21.15.38:6816/1491818732 8 ==== osd_op_reply(92 mds0_inotable [write 0~34] v106'96 uv96 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f5a8c068030 con 0x561489291d20 2024-09-21T03:41:19.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.790+0000 7f5aa19dcf40 1 -- 172.21.15.38:0/1789342087 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x561489294aa0 con 0x561489291d20 2024-09-21T03:41:19.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.791+0000 7f5a9b498640 1 -- 172.21.15.38:0/1789342087 <== osd.1 v2:172.21.15.38:6816/1491818732 9 ==== osd_op_reply(93 mds0_inotable [omap-set-header] v106'97 uv97 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f5a8c068030 con 0x561489291d20 2024-09-21T03:41:19.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.792+0000 7f5aa19dcf40 1 -- 172.21.15.38:0/1789342087 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:94 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x561489294aa0 con 0x561489291d20 2024-09-21T03:41:19.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.792+0000 7f5a9b498640 1 -- 172.21.15.38:0/1789342087 <== osd.1 v2:172.21.15.38:6816/1491818732 10 ==== osd_op_reply(94 100.00000000 [create] v106'98 uv93 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f5a8c068030 con 0x561489291d20 2024-09-21T03:41:19.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.793+0000 7f5aa19dcf40 1 -- 172.21.15.38:0/1789342087 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:95 13.13 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e106) -- 0x561489294aa0 con 0x561489291d20 2024-09-21T03:41:19.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.793+0000 7f5a9b498640 1 -- 172.21.15.38:0/1789342087 <== osd.1 v2:172.21.15.38:6816/1491818732 11 ==== osd_op_reply(95 100.00000000 [delete] v106'99 uv99 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5a8c068030 con 0x561489291d20 2024-09-21T03:41:19.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.794+0000 7f5aa19dcf40 1 -- 172.21.15.38:0/1789342087 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:96 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x561489294aa0 con 0x561489291d20 2024-09-21T03:41:19.796 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-09-21T03:41:19.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.795+0000 7f5a9b498640 1 -- 172.21.15.38:0/1789342087 <== osd.1 v2:172.21.15.38:6816/1491818732 12 ==== osd_op_reply(96 100.00000000 [create] v106'100 uv100 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5a8c068030 con 0x561489291d20 2024-09-21T03:41:19.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.795+0000 7f5aa19dcf40 1 -- 172.21.15.38:0/1789342087 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:97 13.13 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x561489294aa0 con 0x561489291d20 2024-09-21T03:41:19.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.796+0000 7f5a9b498640 1 -- 172.21.15.38:0/1789342087 <== osd.1 v2:172.21.15.38:6816/1491818732 13 ==== osd_op_reply(97 100.00000000 [omap-set-header] v106'101 uv101 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5a8c068030 con 0x561489291d20 2024-09-21T03:41:19.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.796+0000 7f5aa19dcf40 1 -- 172.21.15.38:0/1789342087 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:98 13.13 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x5614892978b0 con 0x561489291d20 2024-09-21T03:41:19.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.797+0000 7f5a9b498640 1 -- 172.21.15.38:0/1789342087 <== osd.1 v2:172.21.15.38:6816/1491818732 14 ==== osd_op_reply(98 100.00000000 [omap-set-vals] v106'102 uv102 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5a8c068030 con 0x561489291d20 2024-09-21T03:41:19.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.797+0000 7f5aa19dcf40 1 -- 172.21.15.38:0/1789342087 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:99 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x561489294f50 con 0x56148928d8f0 2024-09-21T03:41:19.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.798+0000 7f5a9bc99640 1 -- 172.21.15.38:0/1789342087 <== osd.3 v2:172.21.15.38:6808/1286810294 16 ==== osd_op_reply(99 mds0_sessionmap [create] v106'44 uv43 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7f5a9400f070 con 0x56148928d8f0 2024-09-21T03:41:19.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.798+0000 7f5aa19dcf40 1 -- 172.21.15.38:0/1789342087 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:100 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e106) -- 0x561489294f50 con 0x56148928d8f0 2024-09-21T03:41:19.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.799+0000 7f5a9bc99640 1 -- 172.21.15.38:0/1789342087 <== osd.3 v2:172.21.15.38:6808/1286810294 17 ==== osd_op_reply(100 mds0_sessionmap [delete] v106'45 uv45 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f5a9400f070 con 0x56148928d8f0 2024-09-21T03:41:19.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.799+0000 7f5aa19dcf40 1 -- 172.21.15.38:0/1789342087 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x561489294f50 con 0x56148928d8f0 2024-09-21T03:41:19.802 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-09-21T03:41:19.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.800+0000 7f5a9bc99640 1 -- 172.21.15.38:0/1789342087 <== osd.3 v2:172.21.15.38:6808/1286810294 18 ==== osd_op_reply(101 mds0_sessionmap [create] v106'46 uv46 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f5a9400f070 con 0x56148928d8f0 2024-09-21T03:41:19.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.800+0000 7f5aa19dcf40 1 -- 172.21.15.38:0/1789342087 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x561489294f50 con 0x56148928d8f0 2024-09-21T03:41:19.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.801+0000 7f5a9bc99640 1 -- 172.21.15.38:0/1789342087 <== osd.3 v2:172.21.15.38:6808/1286810294 19 ==== osd_op_reply(102 mds0_sessionmap [omap-set-header] v106'47 uv47 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f5a9400f070 con 0x56148928d8f0 2024-09-21T03:41:19.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.801+0000 7f5aa19dcf40 1 -- 172.21.15.38:0/1789342087 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:103 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x561489294f50 con 0x56148928d8f0 2024-09-21T03:41:19.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.802+0000 7f5a9bc99640 1 -- 172.21.15.38:0/1789342087 <== osd.3 v2:172.21.15.38:6808/1286810294 20 ==== osd_op_reply(103 609.00000000 [create] v106'57 uv56 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f5a9400f070 con 0x56148928d8f0 2024-09-21T03:41:19.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.802+0000 7f5aa19dcf40 1 -- 172.21.15.38:0/1789342087 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:104 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e106) -- 0x561489294f50 con 0x56148928d8f0 2024-09-21T03:41:19.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.803+0000 7f5a9bc99640 1 -- 172.21.15.38:0/1789342087 <== osd.3 v2:172.21.15.38:6808/1286810294 21 ==== osd_op_reply(104 609.00000000 [delete] v106'58 uv58 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5a9400f070 con 0x56148928d8f0 2024-09-21T03:41:19.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.803+0000 7f5aa19dcf40 1 -- 172.21.15.38:0/1789342087 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x561489294f50 con 0x56148928d8f0 2024-09-21T03:41:19.806 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-09-21T03:41:19.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.804+0000 7f5a9bc99640 1 -- 172.21.15.38:0/1789342087 <== osd.3 v2:172.21.15.38:6808/1286810294 22 ==== osd_op_reply(105 609.00000000 [create] v106'59 uv59 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5a9400f070 con 0x56148928d8f0 2024-09-21T03:41:19.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.804+0000 7f5aa19dcf40 1 -- 172.21.15.38:0/1789342087 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x561489295f60 con 0x56148928d8f0 2024-09-21T03:41:19.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.805+0000 7f5a9bc99640 1 -- 172.21.15.38:0/1789342087 <== osd.3 v2:172.21.15.38:6808/1286810294 23 ==== osd_op_reply(106 609.00000000 [setxattr (30)] v106'60 uv60 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5a9400f070 con 0x56148928d8f0 2024-09-21T03:41:19.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.805+0000 7f5aa19dcf40 1 -- 172.21.15.38:0/1789342087 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x561489295f60 con 0x56148928d8f0 2024-09-21T03:41:19.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.807+0000 7f5a9bc99640 1 -- 172.21.15.38:0/1789342087 <== osd.3 v2:172.21.15.38:6808/1286810294 24 ==== osd_op_reply(107 609.00000000 [setxattr (62)] v106'61 uv61 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5a9400f070 con 0x56148928d8f0 2024-09-21T03:41:19.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.807+0000 7f5aa19dcf40 1 -- 172.21.15.38:0/1789342087 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x561489294f50 con 0x56148928d8f0 2024-09-21T03:41:19.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.808+0000 7f5a9bc99640 1 -- 172.21.15.38:0/1789342087 <== osd.3 v2:172.21.15.38:6808/1286810294 25 ==== osd_op_reply(108 609.00000000 [omap-set-header] v106'62 uv62 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5a9400f070 con 0x56148928d8f0 2024-09-21T03:41:19.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.808+0000 7f5aa19dcf40 1 --2- 172.21.15.38:0/1789342087 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x5614892955d0 0x5614892959e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:19.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.808+0000 7f5aa19dcf40 1 -- 172.21.15.38:0/1789342087 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:109 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x561489295f20 con 0x5614892955d0 2024-09-21T03:41:19.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.808+0000 7f5a9ac97640 1 --2- 172.21.15.38:0/1789342087 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x5614892955d0 0x5614892959e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:19.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.809+0000 7f5a9ac97640 1 --2- 172.21.15.38:0/1789342087 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x5614892955d0 0x5614892959e0 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:19.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.810+0000 7f5a9ac97640 1 -- 172.21.15.38:0/1789342087 <== osd.2 v2:172.21.15.38:6824/2577858902 1 ==== osd_op_reply(109 400.00000000 [create] v106'47 uv46 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f5a90011070 con 0x5614892955d0 2024-09-21T03:41:19.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.810+0000 7f5aa19dcf40 1 -- 172.21.15.38:0/1789342087 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:110 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e106) -- 0x561489295f20 con 0x5614892955d0 2024-09-21T03:41:19.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.811+0000 7f5a9ac97640 1 -- 172.21.15.38:0/1789342087 <== osd.2 v2:172.21.15.38:6824/2577858902 2 ==== osd_op_reply(110 400.00000000 [delete] v106'48 uv48 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5a90011070 con 0x5614892955d0 2024-09-21T03:41:19.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.811+0000 7f5aa19dcf40 1 -- 172.21.15.38:0/1789342087 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x561489295f20 con 0x5614892955d0 2024-09-21T03:41:19.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.812+0000 7f5a9ac97640 1 -- 172.21.15.38:0/1789342087 <== osd.2 v2:172.21.15.38:6824/2577858902 3 ==== osd_op_reply(111 400.00000000 [create] v106'49 uv49 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5a90011070 con 0x5614892955d0 2024-09-21T03:41:19.814 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-09-21T03:41:19.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.812+0000 7f5aa19dcf40 1 -- 172.21.15.38:0/1789342087 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x561489296370 con 0x5614892955d0 2024-09-21T03:41:19.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.813+0000 7f5a9ac97640 1 -- 172.21.15.38:0/1789342087 <== osd.2 v2:172.21.15.38:6824/2577858902 4 ==== osd_op_reply(112 400.00000000 [write 0~22] v106'50 uv50 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5a90011070 con 0x5614892955d0 2024-09-21T03:41:19.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.813+0000 7f5aa19dcf40 1 -- 172.21.15.38:0/1789342087 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x561489296370 con 0x5614892955d0 2024-09-21T03:41:19.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.814+0000 7f5a9ac97640 1 -- 172.21.15.38:0/1789342087 <== osd.2 v2:172.21.15.38:6824/2577858902 5 ==== osd_op_reply(113 400.00000000 [omap-set-header] v106'51 uv51 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5a90011070 con 0x5614892955d0 2024-09-21T03:41:19.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.814+0000 7f5aa19dcf40 1 -- 172.21.15.38:0/1789342087 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:114 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x561489296320 con 0x561489291d20 2024-09-21T03:41:19.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.815+0000 7f5a9b498640 1 -- 172.21.15.38:0/1789342087 <== osd.1 v2:172.21.15.38:6816/1491818732 15 ==== osd_op_reply(114 100.00000000.inode [create] v106'119 uv111 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7f5a8c068030 con 0x561489291d20 2024-09-21T03:41:19.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.815+0000 7f5aa19dcf40 1 -- 172.21.15.38:0/1789342087 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:115 13.1f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e106) -- 0x561489296320 con 0x561489291d20 2024-09-21T03:41:19.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.816+0000 7f5a9b498640 1 -- 172.21.15.38:0/1789342087 <== osd.1 v2:172.21.15.38:6816/1491818732 16 ==== osd_op_reply(115 100.00000000.inode [delete] v106'120 uv120 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f5a8c068030 con 0x561489291d20 2024-09-21T03:41:19.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.816+0000 7f5aa19dcf40 1 -- 172.21.15.38:0/1789342087 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:116 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x561489296320 con 0x561489291d20 2024-09-21T03:41:19.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.817+0000 7f5a9b498640 1 -- 172.21.15.38:0/1789342087 <== osd.1 v2:172.21.15.38:6816/1491818732 17 ==== osd_op_reply(116 100.00000000.inode [create] v106'121 uv121 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f5a8c068030 con 0x561489291d20 2024-09-21T03:41:19.819 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-09-21T03:41:19.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.817+0000 7f5aa19dcf40 1 -- 172.21.15.38:0/1789342087 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:117 13.1f 13:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x561489296320 con 0x561489291d20 2024-09-21T03:41:19.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.818+0000 7f5a9b498640 1 -- 172.21.15.38:0/1789342087 <== osd.1 v2:172.21.15.38:6816/1491818732 18 ==== osd_op_reply(117 100.00000000.inode [write 0~582] v106'122 uv122 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f5a8c068030 con 0x561489291d20 2024-09-21T03:41:19.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.818+0000 7f5aa19dcf40 1 -- 172.21.15.38:0/1789342087 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:118 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x561489296320 con 0x561489291d20 2024-09-21T03:41:19.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.820+0000 7f5a9b498640 1 -- 172.21.15.38:0/1789342087 <== osd.1 v2:172.21.15.38:6816/1491818732 19 ==== osd_op_reply(118 100.00000000.inode [omap-set-header] v106'123 uv123 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f5a8c068030 con 0x561489291d20 2024-09-21T03:41:19.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.820+0000 7f5aa19dcf40 1 -- 172.21.15.38:0/1789342087 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:119 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x561489296320 con 0x561489291d20 2024-09-21T03:41:19.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.821+0000 7f5a9b498640 1 -- 172.21.15.38:0/1789342087 <== osd.1 v2:172.21.15.38:6816/1491818732 20 ==== osd_op_reply(119 1.00000000 [create] v106'124 uv118 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7f5a8c068030 con 0x561489291d20 2024-09-21T03:41:19.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.821+0000 7f5aa19dcf40 1 -- 172.21.15.38:0/1789342087 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:120 13.1f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e106) -- 0x561489296320 con 0x561489291d20 2024-09-21T03:41:19.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.822+0000 7f5a9b498640 1 -- 172.21.15.38:0/1789342087 <== osd.1 v2:172.21.15.38:6816/1491818732 21 ==== osd_op_reply(120 1.00000000 [delete] v106'125 uv125 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f5a8c068030 con 0x561489291d20 2024-09-21T03:41:19.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.822+0000 7f5aa19dcf40 1 -- 172.21.15.38:0/1789342087 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:121 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x561489296320 con 0x561489291d20 2024-09-21T03:41:19.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.823+0000 7f5a9b498640 1 -- 172.21.15.38:0/1789342087 <== osd.1 v2:172.21.15.38:6816/1491818732 22 ==== osd_op_reply(121 1.00000000 [create] v106'126 uv126 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f5a8c068030 con 0x561489291d20 2024-09-21T03:41:19.825 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-09-21T03:41:19.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.823+0000 7f5aa19dcf40 1 -- 172.21.15.38:0/1789342087 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:122 13.1f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x561489296320 con 0x561489291d20 2024-09-21T03:41:19.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.824+0000 7f5a9b498640 1 -- 172.21.15.38:0/1789342087 <== osd.1 v2:172.21.15.38:6816/1491818732 23 ==== osd_op_reply(122 1.00000000 [setxattr (30)] v106'127 uv127 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f5a8c068030 con 0x561489291d20 2024-09-21T03:41:19.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.824+0000 7f5aa19dcf40 1 -- 172.21.15.38:0/1789342087 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:123 13.1f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x561489296320 con 0x561489291d20 2024-09-21T03:41:19.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.825+0000 7f5a9b498640 1 -- 172.21.15.38:0/1789342087 <== osd.1 v2:172.21.15.38:6816/1491818732 24 ==== osd_op_reply(123 1.00000000 [setxattr (30)] v106'128 uv128 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f5a8c068030 con 0x561489291d20 2024-09-21T03:41:19.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.825+0000 7f5aa19dcf40 1 -- 172.21.15.38:0/1789342087 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:124 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x561489296320 con 0x561489291d20 2024-09-21T03:41:19.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.826+0000 7f5a9b498640 1 -- 172.21.15.38:0/1789342087 <== osd.1 v2:172.21.15.38:6816/1491818732 25 ==== osd_op_reply(124 1.00000000 [omap-set-header] v106'129 uv129 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f5a8c068030 con 0x561489291d20 2024-09-21T03:41:19.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.826+0000 7f5aa19dcf40 1 -- 172.21.15.38:0/1789342087 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:125 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=584b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x561489296780 con 0x561489291d20 2024-09-21T03:41:19.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.827+0000 7f5a9b498640 1 -- 172.21.15.38:0/1789342087 <== osd.1 v2:172.21.15.38:6816/1491818732 26 ==== osd_op_reply(125 1.00000000 [omap-set-vals] v106'130 uv130 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f5a8c068030 con 0x561489291d20 2024-09-21T03:41:19.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.828+0000 7f5aa19dcf40 1 -- 172.21.15.38:0/1789342087 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x5614892892a0 msgr2=0x5614892896b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:19.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.828+0000 7f5aa19dcf40 1 --2- 172.21.15.38:0/1789342087 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x5614892892a0 0x5614892896b0 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:19.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.828+0000 7f5aa19dcf40 1 -- 172.21.15.38:0/1789342087 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x561489291d20 msgr2=0x561489294140 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:19.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.828+0000 7f5aa19dcf40 1 --2- 172.21.15.38:0/1789342087 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x561489291d20 0x561489294140 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:19.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.828+0000 7f5aa19dcf40 1 -- 172.21.15.38:0/1789342087 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x5614892955d0 msgr2=0x5614892959e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:19.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.828+0000 7f5aa19dcf40 1 --2- 172.21.15.38:0/1789342087 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x5614892955d0 0x5614892959e0 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:19.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.828+0000 7f5aa19dcf40 1 -- 172.21.15.38:0/1789342087 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x56148928d8f0 msgr2=0x561489290750 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:19.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.828+0000 7f5aa19dcf40 1 --2- 172.21.15.38:0/1789342087 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x56148928d8f0 0x561489290750 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:19.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.828+0000 7f5a9bc99640 1 -- 172.21.15.38:0/1789342087 reap_dead start 2024-09-21T03:41:19.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.828+0000 7f5aa19dcf40 1 -- 172.21.15.38:0/1789342087 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x561489283d20 msgr2=0x561489286e00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:19.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.828+0000 7f5aa19dcf40 1 --2- 172.21.15.38:0/1789342087 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x561489283d20 0x561489286e00 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:19.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.828+0000 7f5a9bc99640 1 -- 172.21.15.38:0/1789342087 reap_dead start 2024-09-21T03:41:19.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.828+0000 7f5aa19dcf40 1 -- 172.21.15.38:0/1789342087 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x56148914e390 msgr2=0x561489282510 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:19.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.828+0000 7f5aa19dcf40 1 --2- 172.21.15.38:0/1789342087 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x56148914e390 0x561489282510 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:19.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.828+0000 7f5aa19dcf40 1 -- 172.21.15.38:0/1789342087 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5a880775a0 msgr2=0x7f5a88079ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:19.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.828+0000 7f5aa19dcf40 1 --2- 172.21.15.38:0/1789342087 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5a880775a0 0x7f5a88079ab0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x56148914e9c0 tx=0x7f5a90006700 comp rx=0 tx=0).stop 2024-09-21T03:41:19.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.828+0000 7f5aa19dcf40 1 -- 172.21.15.38:0/1789342087 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561489109d00 msgr2=0x5614891c0e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:19.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.828+0000 7f5aa19dcf40 1 --2- 172.21.15.38:0/1789342087 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561489109d00 0x5614891c0e90 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f5a8c025f60 tx=0x7f5a8c04d9a0 comp rx=0 tx=0).stop 2024-09-21T03:41:19.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.828+0000 7f5aa19dcf40 1 -- 172.21.15.38:0/1789342087 shutdown_connections 2024-09-21T03:41:19.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.828+0000 7f5aa19dcf40 1 --2- 172.21.15.38:0/1789342087 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x56148914e390 0x561489282510 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:19.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.828+0000 7f5aa19dcf40 1 --2- 172.21.15.38:0/1789342087 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5a880775a0 0x7f5a88079ab0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:19.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.828+0000 7f5aa19dcf40 1 --2- 172.21.15.38:0/1789342087 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561489109d00 0x5614891c0e90 unknown :-1 s=CLOSED pgs=1469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:19.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.828+0000 7f5aa19dcf40 1 -- 172.21.15.38:0/1789342087 >> 172.21.15.38:0/1789342087 conn(0x561489028ac0 msgr2=0x561489138690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:19.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.828+0000 7f5aa19dcf40 1 -- 172.21.15.38:0/1789342087 shutdown_connections 2024-09-21T03:41:19.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.828+0000 7f5aa19dcf40 1 -- 172.21.15.38:0/1789342087 wait complete. 2024-09-21T03:41:19.833 DEBUG:teuthology.orchestra.run.smithi038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rm 100.00000000 2024-09-21T03:41:19.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.927+0000 7f708aa48f40 1 Processor -- start 2024-09-21T03:41:19.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.928+0000 7f708aa48f40 1 -- start start 2024-09-21T03:41:19.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.929+0000 7f708aa48f40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f706c003630 0x560f9d8dee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:19.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.929+0000 7f708aa48f40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560f9d98a7d0 0x560f9d98b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:19.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.929+0000 7f708aa48f40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560f9d8b6dd0 0x560f9d8df610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:19.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.929+0000 7f708aa48f40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x560f9d804d60 con 0x560f9d98a7d0 2024-09-21T03:41:19.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.929+0000 7f708aa48f40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x560f9d863430 con 0x560f9d8b6dd0 2024-09-21T03:41:19.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.929+0000 7f708aa48f40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x560f9d8fe7b0 con 0x7f706c003630 2024-09-21T03:41:19.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.929+0000 7f7083d01640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560f9d98a7d0 0x560f9d98b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:19.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.929+0000 7f7083d01640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560f9d98a7d0 0x560f9d98b440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47134/0 (socket says 172.21.15.38:47134) 2024-09-21T03:41:19.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.929+0000 7f7083d01640 1 -- 172.21.15.38:0/3829512831 learned_addr learned my addr 172.21.15.38:0/3829512831 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:41:19.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.929+0000 7f7083d01640 1 -- 172.21.15.38:0/3829512831 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f706c003630 msgr2=0x560f9d8dee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:19.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.929+0000 7f7083d01640 1 --2- 172.21.15.38:0/3829512831 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f706c003630 0x560f9d8dee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:19.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.929+0000 7f7083d01640 1 -- 172.21.15.38:0/3829512831 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560f9d8b6dd0 msgr2=0x560f9d8df610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:19.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.929+0000 7f7083d01640 1 --2- 172.21.15.38:0/3829512831 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560f9d8b6dd0 0x560f9d8df610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:19.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.929+0000 7f7083d01640 1 -- 172.21.15.38:0/3829512831 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f9d8fbd40 con 0x560f9d98a7d0 2024-09-21T03:41:19.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.929+0000 7f7083d01640 1 --2- 172.21.15.38:0/3829512831 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560f9d98a7d0 0x560f9d98b440 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7f7078009630 tx=0x7f7078009af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42ccddc9a4b783dd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:19.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.929+0000 7f7082cff640 1 -- 172.21.15.38:0/3829512831 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f707800d780 con 0x560f9d98a7d0 2024-09-21T03:41:19.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.930+0000 7f7082cff640 1 -- 172.21.15.38:0/3829512831 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7078014020 con 0x560f9d98a7d0 2024-09-21T03:41:19.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.930+0000 7f7082cff640 1 -- 172.21.15.38:0/3829512831 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f707800dba0 con 0x560f9d98a7d0 2024-09-21T03:41:19.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.930+0000 7f708aa48f40 1 -- 172.21.15.38:0/3829512831 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560f9d98a7d0 msgr2=0x560f9d98b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:19.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.930+0000 7f708aa48f40 1 --2- 172.21.15.38:0/3829512831 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560f9d98a7d0 0x560f9d98b440 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7f7078009630 tx=0x7f7078009af0 comp rx=0 tx=0).stop 2024-09-21T03:41:19.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.930+0000 7f708aa48f40 1 -- 172.21.15.38:0/3829512831 shutdown_connections 2024-09-21T03:41:19.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.930+0000 7f708aa48f40 1 --2- 172.21.15.38:0/3829512831 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560f9d8b6dd0 0x560f9d8df610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:19.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.930+0000 7f708aa48f40 1 --2- 172.21.15.38:0/3829512831 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560f9d98a7d0 0x560f9d98b440 unknown :-1 s=CLOSED pgs=1472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:19.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.930+0000 7f708aa48f40 1 --2- 172.21.15.38:0/3829512831 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f706c003630 0x560f9d8dee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:19.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.930+0000 7f708aa48f40 1 -- 172.21.15.38:0/3829512831 >> 172.21.15.38:0/3829512831 conn(0x560f9d7d5ac0 msgr2=0x560f9d9890f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:19.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.930+0000 7f708aa48f40 1 -- 172.21.15.38:0/3829512831 shutdown_connections 2024-09-21T03:41:19.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.930+0000 7f708aa48f40 1 -- 172.21.15.38:0/3829512831 wait complete. 2024-09-21T03:41:19.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.931+0000 7f708aa48f40 1 Processor -- start 2024-09-21T03:41:19.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.931+0000 7f708aa48f40 1 -- start start 2024-09-21T03:41:19.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.932+0000 7f708aa48f40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560f9d8b6dd0 0x560f9d9652d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:19.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.932+0000 7f708aa48f40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560f9d98a7d0 0x560f9d98b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:19.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.932+0000 7f708aa48f40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f706c003630 0x560f9d98b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:19.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.932+0000 7f708aa48f40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x560f9d8fbaf0 con 0x560f9d8b6dd0 2024-09-21T03:41:19.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.932+0000 7f708aa48f40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x560f9d8df390 con 0x7f706c003630 2024-09-21T03:41:19.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.932+0000 7f708aa48f40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x560f9d96c560 con 0x560f9d98a7d0 2024-09-21T03:41:19.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.932+0000 7f7084502640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560f9d8b6dd0 0x560f9d9652d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:19.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.932+0000 7f7084502640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560f9d8b6dd0 0x560f9d9652d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47136/0 (socket says 172.21.15.38:47136) 2024-09-21T03:41:19.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.932+0000 7f7084502640 1 -- 172.21.15.38:0/1161339382 learned_addr learned my addr 172.21.15.38:0/1161339382 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:41:19.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.933+0000 7f7083d01640 1 --2- 172.21.15.38:0/1161339382 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560f9d98a7d0 0x560f9d98b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:19.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.933+0000 7f7084d03640 1 --2- 172.21.15.38:0/1161339382 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f706c003630 0x560f9d98b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:19.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.933+0000 7f7084502640 1 -- 172.21.15.38:0/1161339382 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560f9d98a7d0 msgr2=0x560f9d98b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:19.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.933+0000 7f7084502640 1 --2- 172.21.15.38:0/1161339382 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560f9d98a7d0 0x560f9d98b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:19.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.933+0000 7f7084502640 1 -- 172.21.15.38:0/1161339382 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f706c003630 msgr2=0x560f9d98b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:19.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.933+0000 7f7084502640 1 --2- 172.21.15.38:0/1161339382 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f706c003630 0x560f9d98b910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:19.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.933+0000 7f7084502640 1 -- 172.21.15.38:0/1161339382 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f9da24b40 con 0x560f9d8b6dd0 2024-09-21T03:41:19.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.933+0000 7f7084d03640 1 --2- 172.21.15.38:0/1161339382 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f706c003630 0x560f9d98b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:41:19.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.933+0000 7f7084502640 1 --2- 172.21.15.38:0/1161339382 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560f9d8b6dd0 0x560f9d9652d0 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f70740044b0 tx=0x7f707400bc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:19.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.933+0000 7f7081cfd640 1 -- 172.21.15.38:0/1161339382 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f707400ad70 con 0x560f9d8b6dd0 2024-09-21T03:41:19.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.933+0000 7f7081cfd640 1 -- 172.21.15.38:0/1161339382 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f70740183f0 con 0x560f9d8b6dd0 2024-09-21T03:41:19.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.933+0000 7f708aa48f40 1 -- 172.21.15.38:0/1161339382 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x560f9da24e20 con 0x560f9d8b6dd0 2024-09-21T03:41:19.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.933+0000 7f7081cfd640 1 -- 172.21.15.38:0/1161339382 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7074018840 con 0x560f9d8b6dd0 2024-09-21T03:41:19.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.933+0000 7f708aa48f40 1 -- 172.21.15.38:0/1161339382 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f9d982940 con 0x560f9d8b6dd0 2024-09-21T03:41:19.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.935+0000 7f7081cfd640 1 -- 172.21.15.38:0/1161339382 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f70740189e0 con 0x560f9d8b6dd0 2024-09-21T03:41:19.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.935+0000 7f7081cfd640 1 --2- 172.21.15.38:0/1161339382 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f70700775a0 0x7f7070079ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:19.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.936+0000 7f7081cfd640 1 -- 172.21.15.38:0/1161339382 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 7164+0+0 (secure 0 0 0) 0x7f707409c150 con 0x560f9d8b6dd0 2024-09-21T03:41:19.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.936+0000 7f7083d01640 1 --2- 172.21.15.38:0/1161339382 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f70700775a0 0x7f7070079ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:19.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.936+0000 7f708aa48f40 1 --2- 172.21.15.38:0/1161339382 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x560f9d984320 0x560f9d984730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:19.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.936+0000 7f708aa48f40 1 -- 172.21.15.38:0/1161339382 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:1 13.13 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e106) -- 0x560f9d981660 con 0x560f9d984320 2024-09-21T03:41:19.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.936+0000 7f7084d03640 1 --2- 172.21.15.38:0/1161339382 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x560f9d984320 0x560f9d984730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:19.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.936+0000 7f7083d01640 1 --2- 172.21.15.38:0/1161339382 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f70700775a0 0x7f7070079ab0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f7078009630 tx=0x7f70780092a0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:19.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.936+0000 7f7084d03640 1 --2- 172.21.15.38:0/1161339382 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x560f9d984320 0x560f9d984730 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:19.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.938+0000 7f7084d03640 1 -- 172.21.15.38:0/1161339382 <== osd.1 v2:172.21.15.38:6816/1491818732 1 ==== osd_op_reply(1 100.00000000 [delete] v106'103 uv103 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f707c00b040 con 0x560f9d984320 2024-09-21T03:41:19.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.938+0000 7f708aa48f40 1 -- 172.21.15.38:0/1161339382 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x560f9d984320 msgr2=0x560f9d984730 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:19.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.938+0000 7f708aa48f40 1 --2- 172.21.15.38:0/1161339382 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x560f9d984320 0x560f9d984730 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:19.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.938+0000 7f708aa48f40 1 -- 172.21.15.38:0/1161339382 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f70700775a0 msgr2=0x7f7070079ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:19.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.938+0000 7f708aa48f40 1 --2- 172.21.15.38:0/1161339382 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f70700775a0 0x7f7070079ab0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f7078009630 tx=0x7f70780092a0 comp rx=0 tx=0).stop 2024-09-21T03:41:19.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.938+0000 7f708aa48f40 1 -- 172.21.15.38:0/1161339382 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560f9d8b6dd0 msgr2=0x560f9d9652d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:19.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.938+0000 7f708aa48f40 1 --2- 172.21.15.38:0/1161339382 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560f9d8b6dd0 0x560f9d9652d0 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f70740044b0 tx=0x7f707400bc90 comp rx=0 tx=0).stop 2024-09-21T03:41:19.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.938+0000 7f7084d03640 1 -- 172.21.15.38:0/1161339382 reap_dead start 2024-09-21T03:41:19.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.938+0000 7f708aa48f40 1 -- 172.21.15.38:0/1161339382 shutdown_connections 2024-09-21T03:41:19.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.938+0000 7f708aa48f40 1 -- 172.21.15.38:0/1161339382 >> 172.21.15.38:0/1161339382 conn(0x560f9d7d5ac0 msgr2=0x560f9d96c7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:19.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.938+0000 7f708aa48f40 1 -- 172.21.15.38:0/1161339382 shutdown_connections 2024-09-21T03:41:19.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:19.938+0000 7f708aa48f40 1 -- 172.21.15.38:0/1161339382 wait complete. 2024-09-21T03:41:19.943 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-21T03:41:20.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:20.060+0000 7ff149ff1640 1 -- 172.21.15.38:0/3713824624 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff1440ccc40 msgr2=0x7ff14415ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:20.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:20.060+0000 7ff149ff1640 1 --2- 172.21.15.38:0/3713824624 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff1440ccc40 0x7ff14415ca90 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ff134009860 tx=0x7ff13402a4a0 comp rx=0 tx=0).stop 2024-09-21T03:41:20.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:20.060+0000 7ff149ff1640 1 -- 172.21.15.38:0/3713824624 shutdown_connections 2024-09-21T03:41:20.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:20.060+0000 7ff149ff1640 1 --2- 172.21.15.38:0/3713824624 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff1440ccc40 0x7ff14415ca90 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:20.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:20.060+0000 7ff149ff1640 1 -- 172.21.15.38:0/3713824624 >> 172.21.15.38:0/3713824624 conn(0x7ff1440b46e0 msgr2=0x7ff1440d23e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:20.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:20.060+0000 7ff149ff1640 1 -- 172.21.15.38:0/3713824624 shutdown_connections 2024-09-21T03:41:20.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:20.060+0000 7ff149ff1640 1 -- 172.21.15.38:0/3713824624 wait complete. 2024-09-21T03:41:20.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:20.062+0000 7ff149ff1640 1 Processor -- start 2024-09-21T03:41:20.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:20.062+0000 7ff149ff1640 1 -- start start 2024-09-21T03:41:20.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:20.064+0000 7ff149ff1640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff1440ccc40 0x7ff1441f3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:20.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:20.064+0000 7ff149ff1640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff1440cd440 0x7ff1440aa120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:20.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:20.064+0000 7ff149ff1640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff14415f1e0 0x7ff14415f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:20.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:20.064+0000 7ff149ff1640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ff1440b6890 con 0x7ff1440cd440 2024-09-21T03:41:20.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:20.064+0000 7ff149ff1640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ff1440b6710 con 0x7ff14415f1e0 2024-09-21T03:41:20.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:20.064+0000 7ff149ff1640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7ff1440b6a10 con 0x7ff1440ccc40 2024-09-21T03:41:20.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:20.064+0000 7ff142ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff1440cd440 0x7ff1440aa120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:20.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:20.064+0000 7ff142ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff1440cd440 0x7ff1440aa120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47140/0 (socket says 172.21.15.38:47140) 2024-09-21T03:41:20.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:20.064+0000 7ff142ffd640 1 -- 172.21.15.38:0/3229134602 learned_addr learned my addr 172.21.15.38:0/3229134602 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:41:20.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:20.064+0000 7ff143fff640 1 --2- 172.21.15.38:0/3229134602 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff14415f1e0 0x7ff14415f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:20.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:20.064+0000 7ff1437fe640 1 --2- 172.21.15.38:0/3229134602 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff1440ccc40 0x7ff1441f3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:20.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:20.064+0000 7ff143fff640 1 -- 172.21.15.38:0/3229134602 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff1440ccc40 msgr2=0x7ff1441f3a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:20.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:20.064+0000 7ff143fff640 1 --2- 172.21.15.38:0/3229134602 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff1440ccc40 0x7ff1441f3a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:20.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:20.064+0000 7ff143fff640 1 -- 172.21.15.38:0/3229134602 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff1440cd440 msgr2=0x7ff1440aa120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:20.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:20.064+0000 7ff143fff640 1 --2- 172.21.15.38:0/3229134602 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff1440cd440 0x7ff1440aa120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:20.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:20.064+0000 7ff143fff640 1 -- 172.21.15.38:0/3229134602 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff14415fb30 con 0x7ff14415f1e0 2024-09-21T03:41:20.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:20.064+0000 7ff142ffd640 1 --2- 172.21.15.38:0/3229134602 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff1440cd440 0x7ff1440aa120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:41:20.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:20.064+0000 7ff143fff640 1 --2- 172.21.15.38:0/3229134602 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff14415f1e0 0x7ff14415f5f0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7ff13c00c700 tx=0x7ff13c00cbc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:20.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:20.065+0000 7ff12bfff640 1 -- 172.21.15.38:0/3229134602 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff13c003b70 con 0x7ff14415f1e0 2024-09-21T03:41:20.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:20.065+0000 7ff12bfff640 1 -- 172.21.15.38:0/3229134602 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff13c003d10 con 0x7ff14415f1e0 2024-09-21T03:41:20.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:20.065+0000 7ff12bfff640 1 -- 172.21.15.38:0/3229134602 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff13c005330 con 0x7ff14415f1e0 2024-09-21T03:41:20.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:20.065+0000 7ff149ff1640 1 -- 172.21.15.38:0/3229134602 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1440c95c0 con 0x7ff14415f1e0 2024-09-21T03:41:20.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:20.065+0000 7ff149ff1640 1 -- 172.21.15.38:0/3229134602 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1440c9b80 con 0x7ff14415f1e0 2024-09-21T03:41:20.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:20.066+0000 7ff149ff1640 1 -- 172.21.15.38:0/3229134602 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff14415b190 con 0x7ff14415f1e0 2024-09-21T03:41:20.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:20.067+0000 7ff12bfff640 1 -- 172.21.15.38:0/3229134602 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ff13c0054d0 con 0x7ff14415f1e0 2024-09-21T03:41:20.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:20.067+0000 7ff12bfff640 1 --2- 172.21.15.38:0/3229134602 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff130078b90 0x7ff13007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:20.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:20.067+0000 7ff12bfff640 1 -- 172.21.15.38:0/3229134602 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 7164+0+0 (secure 0 0 0) 0x7ff13c09b0e0 con 0x7ff14415f1e0 2024-09-21T03:41:20.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:20.067+0000 7ff1437fe640 1 --2- 172.21.15.38:0/3229134602 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff130078b90 0x7ff13007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:20.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:20.068+0000 7ff1437fe640 1 --2- 172.21.15.38:0/3229134602 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff130078b90 0x7ff13007b0a0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7ff134034000 tx=0x7ff134004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:20.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:20.071+0000 7ff12bfff640 1 -- 172.21.15.38:0/3229134602 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7ff14415b190 con 0x7ff14415f1e0 2024-09-21T03:41:20.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:20.297+0000 7ff149ff1640 1 -- 172.21.15.38:0/3229134602 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7ff1440d2a50 con 0x7ff14415f1e0 2024-09-21T03:41:20.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:20.545+0000 7ff12bfff640 1 -- 172.21.15.38:0/3229134602 <== mon.1 v2:172.21.15.195:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v145) ==== 162+0+0 (secure 0 0 0) 0x7ff14415fb30 con 0x7ff14415f1e0 2024-09-21T03:41:20.548 INFO:teuthology.orchestra.run.smithi038.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-21T03:41:20.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:20.549+0000 7ff149ff1640 1 -- 172.21.15.38:0/3229134602 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff130078b90 msgr2=0x7ff13007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:20.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:20.549+0000 7ff149ff1640 1 --2- 172.21.15.38:0/3229134602 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff130078b90 0x7ff13007b0a0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7ff134034000 tx=0x7ff134004730 comp rx=0 tx=0).stop 2024-09-21T03:41:20.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:20.549+0000 7ff149ff1640 1 -- 172.21.15.38:0/3229134602 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff14415f1e0 msgr2=0x7ff14415f5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:20.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:20.549+0000 7ff149ff1640 1 --2- 172.21.15.38:0/3229134602 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff14415f1e0 0x7ff14415f5f0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7ff13c00c700 tx=0x7ff13c00cbc0 comp rx=0 tx=0).stop 2024-09-21T03:41:20.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:20.549+0000 7ff149ff1640 1 -- 172.21.15.38:0/3229134602 shutdown_connections 2024-09-21T03:41:20.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:20.549+0000 7ff149ff1640 1 --2- 172.21.15.38:0/3229134602 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff14415f1e0 0x7ff14415f5f0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:20.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:20.549+0000 7ff149ff1640 1 --2- 172.21.15.38:0/3229134602 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff1440cd440 0x7ff1440aa120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:20.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:20.549+0000 7ff149ff1640 1 --2- 172.21.15.38:0/3229134602 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff130078b90 0x7ff13007b0a0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:20.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:20.549+0000 7ff149ff1640 1 --2- 172.21.15.38:0/3229134602 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff1440ccc40 0x7ff1441f3a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:20.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:20.549+0000 7ff149ff1640 1 -- 172.21.15.38:0/3229134602 >> 172.21.15.38:0/3229134602 conn(0x7ff1440b46e0 msgr2=0x7ff144070cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:20.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:20.549+0000 7ff149ff1640 1 -- 172.21.15.38:0/3229134602 shutdown_connections 2024-09-21T03:41:20.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:20.549+0000 7ff149ff1640 1 -- 172.21.15.38:0/3229134602 wait complete. 2024-09-21T03:41:20.564 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:41:20.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:20.680+0000 7fd7c1bc3640 1 -- 172.21.15.38:0/1277628210 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd7bc0d1ed0 msgr2=0x7fd7bc0cc930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:20.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:20.680+0000 7fd7c1bc3640 1 --2- 172.21.15.38:0/1277628210 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd7bc0d1ed0 0x7fd7bc0cc930 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fd7b0009830 tx=0x7fd7b002a500 comp rx=0 tx=0).stop 2024-09-21T03:41:20.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:20.680+0000 7fd7c1bc3640 1 -- 172.21.15.38:0/1277628210 shutdown_connections 2024-09-21T03:41:20.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:20.680+0000 7fd7c1bc3640 1 --2- 172.21.15.38:0/1277628210 >> v2:172.21.15.195:3300/0 conn(0x7fd7bc0cce70 0x7fd7bc0d04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:20.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:20.680+0000 7fd7c1bc3640 1 --2- 172.21.15.38:0/1277628210 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd7bc0d1ed0 0x7fd7bc0cc930 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:20.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:20.680+0000 7fd7c1bc3640 1 -- 172.21.15.38:0/1277628210 >> 172.21.15.38:0/1277628210 conn(0x7fd7bc0cef40 msgr2=0x7fd7bc0cf350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:20.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:20.680+0000 7fd7c1bc3640 1 -- 172.21.15.38:0/1277628210 shutdown_connections 2024-09-21T03:41:20.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:20.680+0000 7fd7c1bc3640 1 -- 172.21.15.38:0/1277628210 wait complete. 2024-09-21T03:41:20.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:20.682+0000 7fd7c1bc3640 1 Processor -- start 2024-09-21T03:41:20.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:20.682+0000 7fd7c1bc3640 1 -- start start 2024-09-21T03:41:20.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:20.683+0000 7fd7c1bc3640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd7bc0cce70 0x7fd7bc1f3d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:20.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:20.683+0000 7fd7c1bc3640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd7bc0d1ed0 0x7fd7bc152220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:20.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:20.683+0000 7fd7c1bc3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7bc1536e0 0x7fd7bc152760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:20.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:20.683+0000 7fd7c1bc3640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fd7bc0d0d90 con 0x7fd7bc1536e0 2024-09-21T03:41:20.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:20.683+0000 7fd7c1bc3640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd7bc0d0c10 con 0x7fd7bc0d1ed0 2024-09-21T03:41:20.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:20.683+0000 7fd7c1bc3640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fd7bc0d0f10 con 0x7fd7bc0cce70 2024-09-21T03:41:20.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:20.683+0000 7fd7bbfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7bc1536e0 0x7fd7bc152760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:20.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:20.683+0000 7fd7bbfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7bc1536e0 0x7fd7bc152760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47152/0 (socket says 172.21.15.38:47152) 2024-09-21T03:41:20.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:20.683+0000 7fd7bbfff640 1 -- 172.21.15.38:0/1613410764 learned_addr learned my addr 172.21.15.38:0/1613410764 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:41:20.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:20.683+0000 7fd7baffd640 1 --2- 172.21.15.38:0/1613410764 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd7bc0d1ed0 0x7fd7bc152220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:20.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:20.683+0000 7fd7bb7fe640 1 --2- 172.21.15.38:0/1613410764 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd7bc0cce70 0x7fd7bc1f3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:20.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:20.684+0000 7fd7bbfff640 1 -- 172.21.15.38:0/1613410764 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd7bc0cce70 msgr2=0x7fd7bc1f3d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:20.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:20.684+0000 7fd7bbfff640 1 --2- 172.21.15.38:0/1613410764 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd7bc0cce70 0x7fd7bc1f3d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:20.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:20.684+0000 7fd7bbfff640 1 -- 172.21.15.38:0/1613410764 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd7bc0d1ed0 msgr2=0x7fd7bc152220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:20.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:20.684+0000 7fd7bbfff640 1 --2- 172.21.15.38:0/1613410764 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd7bc0d1ed0 0x7fd7bc152220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:20.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:20.684+0000 7fd7bbfff640 1 -- 172.21.15.38:0/1613410764 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7bc1e5270 con 0x7fd7bc1536e0 2024-09-21T03:41:20.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:20.684+0000 7fd7baffd640 1 --2- 172.21.15.38:0/1613410764 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd7bc0d1ed0 0x7fd7bc152220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:41:20.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:20.684+0000 7fd7bb7fe640 1 --2- 172.21.15.38:0/1613410764 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd7bc0cce70 0x7fd7bc1f3d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:41:20.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:20.684+0000 7fd7bbfff640 1 --2- 172.21.15.38:0/1613410764 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7bc1536e0 0x7fd7bc152760 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7fd7b400ec20 tx=0x7fd7b400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:20.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:20.684+0000 7fd7a3fff640 1 -- 172.21.15.38:0/1613410764 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7b4003b70 con 0x7fd7bc1536e0 2024-09-21T03:41:20.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:20.684+0000 7fd7a3fff640 1 -- 172.21.15.38:0/1613410764 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd7b4003e00 con 0x7fd7bc1536e0 2024-09-21T03:41:20.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:20.684+0000 7fd7c1bc3640 1 -- 172.21.15.38:0/1613410764 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7bc1e5550 con 0x7fd7bc1536e0 2024-09-21T03:41:20.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:20.684+0000 7fd7a3fff640 1 -- 172.21.15.38:0/1613410764 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7b4004280 con 0x7fd7bc1536e0 2024-09-21T03:41:20.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:20.684+0000 7fd7c1bc3640 1 -- 172.21.15.38:0/1613410764 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7bc1e5b10 con 0x7fd7bc1536e0 2024-09-21T03:41:20.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:20.685+0000 7fd7c1bc3640 1 -- 172.21.15.38:0/1613410764 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7bc0d04c0 con 0x7fd7bc1536e0 2024-09-21T03:41:20.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:20.686+0000 7fd7a3fff640 1 -- 172.21.15.38:0/1613410764 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fd7b4004420 con 0x7fd7bc1536e0 2024-09-21T03:41:20.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:20.686+0000 7fd7a3fff640 1 --2- 172.21.15.38:0/1613410764 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd7a8078b90 0x7fd7a807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:20.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:20.687+0000 7fd7a3fff640 1 -- 172.21.15.38:0/1613410764 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 7164+0+0 (secure 0 0 0) 0x7fd7b409bfc0 con 0x7fd7bc1536e0 2024-09-21T03:41:20.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:20.687+0000 7fd7bb7fe640 1 --2- 172.21.15.38:0/1613410764 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd7a8078b90 0x7fd7a807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:20.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:20.687+0000 7fd7bb7fe640 1 --2- 172.21.15.38:0/1613410764 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd7a8078b90 0x7fd7a807b0a0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fd7ac001b10 tx=0x7fd7ac007e90 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:20.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:20.691+0000 7fd7a3fff640 1 -- 172.21.15.38:0/1613410764 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fd7b4063920 con 0x7fd7bc1536e0 2024-09-21T03:41:20.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:20.914+0000 7fd7c1bc3640 1 -- 172.21.15.38:0/1613410764 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd7bc0b7400 con 0x7fd7bc1536e0 2024-09-21T03:41:20.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:20.914+0000 7fd7a3fff640 1 -- 172.21.15.38:0/1613410764 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 145 v145) ==== 95+0+4401 (secure 0 0 0) 0x7fd7b4068350 con 0x7fd7bc1536e0 2024-09-21T03:41:20.916 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:41:20.916 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":145,"btime":"2024-09-21T03:41:20:541901+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6086,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/965972663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":965972663},{"type":"v1","addr":"172.21.15.195:6836","nonce":965972663}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6700,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/3176122334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":3176122334},{"type":"v1","addr":"172.21.15.195:6837","nonce":3176122334}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":145,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:40:02.319282+0000","modified":"2024-09-21T03:41:20.541887+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":106,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8319},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8319":{"gid":8319,"name":"c","rank":0,"incarnation":145,"state":"up:replay","state_seq":1,"addr":"172.21.15.38:6833/3402963947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":3402963947},{"type":"v1","addr":"172.21.15.38:6833","nonce":3402963947}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-21T03:41:20.917 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 145 2024-09-21T03:41:20.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:20.917+0000 7fd7c1bc3640 1 -- 172.21.15.38:0/1613410764 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd7a8078b90 msgr2=0x7fd7a807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:20.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:20.917+0000 7fd7c1bc3640 1 --2- 172.21.15.38:0/1613410764 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd7a8078b90 0x7fd7a807b0a0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fd7ac001b10 tx=0x7fd7ac007e90 comp rx=0 tx=0).stop 2024-09-21T03:41:20.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:20.917+0000 7fd7c1bc3640 1 -- 172.21.15.38:0/1613410764 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7bc1536e0 msgr2=0x7fd7bc152760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:20.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:20.917+0000 7fd7c1bc3640 1 --2- 172.21.15.38:0/1613410764 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7bc1536e0 0x7fd7bc152760 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7fd7b400ec20 tx=0x7fd7b400c6a0 comp rx=0 tx=0).stop 2024-09-21T03:41:20.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:20.917+0000 7fd7c1bc3640 1 -- 172.21.15.38:0/1613410764 shutdown_connections 2024-09-21T03:41:20.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:20.917+0000 7fd7c1bc3640 1 --2- 172.21.15.38:0/1613410764 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7bc1536e0 0x7fd7bc152760 unknown :-1 s=CLOSED pgs=1475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:20.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:20.917+0000 7fd7c1bc3640 1 --2- 172.21.15.38:0/1613410764 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd7bc0d1ed0 0x7fd7bc152220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:20.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:20.917+0000 7fd7c1bc3640 1 --2- 172.21.15.38:0/1613410764 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd7a8078b90 0x7fd7a807b0a0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:20.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:20.917+0000 7fd7c1bc3640 1 --2- 172.21.15.38:0/1613410764 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd7bc0cce70 0x7fd7bc1f3d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:20.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:20.917+0000 7fd7c1bc3640 1 -- 172.21.15.38:0/1613410764 >> 172.21.15.38:0/1613410764 conn(0x7fd7bc0cef40 msgr2=0x7fd7bc0cc0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:20.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:20.917+0000 7fd7c1bc3640 1 -- 172.21.15.38:0/1613410764 shutdown_connections 2024-09-21T03:41:20.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:20.917+0000 7fd7c1bc3640 1 -- 172.21.15.38:0/1613410764 wait complete. 2024-09-21T03:41:20.930 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2024-09-21T03:41:21.317 INFO:tasks.ceph.mds.a.smithi038.stderr: -55> 2024-09-21T03:41:16.564+0000 7f84565dc640 -1 mds.0.inotable: load_2 could not read table: -2 2024-09-21T03:41:21.318 INFO:tasks.ceph.mds.a.smithi038.stderr: -54> 2024-09-21T03:41:16.564+0000 7f84565dc640 -1 log_channel(cluster) log [ERR] : error reading table object 'mds0_inotable' -2 ((2) No such file or directory) 2024-09-21T03:41:21.375 INFO:tasks.ceph.mds.a.smithi038.stdout:starting mds.a at 2024-09-21T03:41:23.553 INFO:tasks.ceph.mds.c.smithi038.stderr:2024-09-21T03:41:23.551+0000 7ff41bcf4640 -1 log_channel(cluster) log [ERR] : dir 0x100 object missing on disk; some files may be lost (~mds0) 2024-09-21T03:41:23.556 INFO:tasks.ceph.mon.a.smithi038.stderr:2024-09-21T03:41:23.554+0000 7f3f45713640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T03:41:23.557 INFO:tasks.ceph.mon.a.smithi038.stderr:2024-09-21T03:41:23.554+0000 7f3f45713640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-09-21T03:41:25.930 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:41:26.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:26.060+0000 7f5c3d827640 1 -- 172.21.15.38:0/3519121073 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5c380d0980 msgr2=0x7f5c380d0d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:26.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:26.060+0000 7f5c3d827640 1 --2- 172.21.15.38:0/3519121073 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5c380d0980 0x7f5c380d0d90 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f5c38073e10 tx=0x7f5c3002a540 comp rx=0 tx=0).stop 2024-09-21T03:41:26.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:26.060+0000 7f5c3d827640 1 -- 172.21.15.38:0/3519121073 shutdown_connections 2024-09-21T03:41:26.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:26.060+0000 7f5c3d827640 1 --2- 172.21.15.38:0/3519121073 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5c380d0980 0x7f5c380d0d90 unknown :-1 s=CLOSED pgs=1477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:26.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:26.060+0000 7f5c3d827640 1 --2- 172.21.15.38:0/3519121073 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5c3815a6f0 0x7f5c3815e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:26.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:26.060+0000 7f5c3d827640 1 --2- 172.21.15.38:0/3519121073 >> v2:172.21.15.195:3300/0 conn(0x7f5c380d01f0 0x7f5c3815daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:26.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:26.060+0000 7f5c3d827640 1 -- 172.21.15.38:0/3519121073 >> 172.21.15.38:0/3519121073 conn(0x7f5c38072990 msgr2=0x7f5c38072da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:26.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:26.060+0000 7f5c3d827640 1 -- 172.21.15.38:0/3519121073 shutdown_connections 2024-09-21T03:41:26.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:26.060+0000 7f5c3d827640 1 -- 172.21.15.38:0/3519121073 wait complete. 2024-09-21T03:41:26.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:26.062+0000 7f5c3d827640 1 Processor -- start 2024-09-21T03:41:26.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:26.062+0000 7f5c3d827640 1 -- start start 2024-09-21T03:41:26.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:26.064+0000 7f5c3d827640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5c380d01f0 0x7f5c3806ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:26.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:26.064+0000 7f5c3d827640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5c380d0980 0x7f5c3806f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:26.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:26.064+0000 7f5c3d827640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5c3815a6f0 0x7f5c380c7280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:26.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:26.064+0000 7f5c3d827640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f5c380ccfb0 con 0x7f5c380d0980 2024-09-21T03:41:26.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:26.064+0000 7f5c3d827640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f5c3815e980 con 0x7f5c380d01f0 2024-09-21T03:41:26.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:26.064+0000 7f5c3d827640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f5c380cd130 con 0x7f5c3815a6f0 2024-09-21T03:41:26.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:26.064+0000 7f5c367fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5c380d0980 0x7f5c3806f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:26.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:26.064+0000 7f5c367fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5c380d0980 0x7f5c3806f440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:52568/0 (socket says 172.21.15.38:52568) 2024-09-21T03:41:26.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:26.064+0000 7f5c367fc640 1 -- 172.21.15.38:0/1447112167 learned_addr learned my addr 172.21.15.38:0/1447112167 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:41:26.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:26.064+0000 7f5c36ffd640 1 --2- 172.21.15.38:0/1447112167 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5c380d01f0 0x7f5c3806ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:26.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:26.064+0000 7f5c377fe640 1 --2- 172.21.15.38:0/1447112167 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5c3815a6f0 0x7f5c380c7280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:26.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:26.064+0000 7f5c367fc640 1 -- 172.21.15.38:0/1447112167 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5c3815a6f0 msgr2=0x7f5c380c7280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:26.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:26.064+0000 7f5c367fc640 1 --2- 172.21.15.38:0/1447112167 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5c3815a6f0 0x7f5c380c7280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:26.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:26.064+0000 7f5c367fc640 1 -- 172.21.15.38:0/1447112167 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5c380d01f0 msgr2=0x7f5c3806ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:26.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:26.064+0000 7f5c367fc640 1 --2- 172.21.15.38:0/1447112167 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5c380d01f0 0x7f5c3806ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:26.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:26.064+0000 7f5c367fc640 1 -- 172.21.15.38:0/1447112167 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c380c57a0 con 0x7f5c380d0980 2024-09-21T03:41:26.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:26.064+0000 7f5c367fc640 1 --2- 172.21.15.38:0/1447112167 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5c380d0980 0x7f5c3806f440 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f5c2c00c970 tx=0x7f5c2c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:26.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:26.065+0000 7f5c1ffff640 1 -- 172.21.15.38:0/1447112167 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c2c007bb0 con 0x7f5c380d0980 2024-09-21T03:41:26.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:26.065+0000 7f5c1ffff640 1 -- 172.21.15.38:0/1447112167 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5c2c007e40 con 0x7f5c380d0980 2024-09-21T03:41:26.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:26.065+0000 7f5c3d827640 1 -- 172.21.15.38:0/1447112167 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c380c54c0 con 0x7f5c380d0980 2024-09-21T03:41:26.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:26.065+0000 7f5c1ffff640 1 -- 172.21.15.38:0/1447112167 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c2c00f6b0 con 0x7f5c380d0980 2024-09-21T03:41:26.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:26.065+0000 7f5c3d827640 1 -- 172.21.15.38:0/1447112167 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c381572f0 con 0x7f5c380d0980 2024-09-21T03:41:26.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:26.066+0000 7f5c3d827640 1 -- 172.21.15.38:0/1447112167 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c380d0d90 con 0x7f5c380d0980 2024-09-21T03:41:26.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:26.067+0000 7f5c1ffff640 1 -- 172.21.15.38:0/1447112167 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f5c2c00f850 con 0x7f5c380d0980 2024-09-21T03:41:26.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:26.067+0000 7f5c1ffff640 1 --2- 172.21.15.38:0/1447112167 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5c24078b90 0x7f5c2407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:26.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:26.067+0000 7f5c1ffff640 1 -- 172.21.15.38:0/1447112167 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 7250+0+0 (secure 0 0 0) 0x7f5c2c09b940 con 0x7f5c380d0980 2024-09-21T03:41:26.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:26.068+0000 7f5c36ffd640 1 --2- 172.21.15.38:0/1447112167 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5c24078b90 0x7f5c2407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:26.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:26.068+0000 7f5c36ffd640 1 --2- 172.21.15.38:0/1447112167 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5c24078b90 0x7f5c2407b0a0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f5c28005cc0 tx=0x7f5c2800a570 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:26.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:26.072+0000 7f5c1ffff640 1 -- 172.21.15.38:0/1447112167 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f5c2c014030 con 0x7f5c380d0980 2024-09-21T03:41:26.299 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:26.297+0000 7f5c3d827640 1 -- 172.21.15.38:0/1447112167 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5c3815a2b0 con 0x7f5c380d0980 2024-09-21T03:41:26.299 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:26.297+0000 7f5c1ffff640 1 -- 172.21.15.38:0/1447112167 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 149 v149) ==== 95+0+4392 (secure 0 0 0) 0x7f5c2c0631b0 con 0x7f5c380d0980 2024-09-21T03:41:26.299 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:41:26.300 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":149,"btime":"2024-09-21T03:41:23:555294+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6086,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/965972663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":965972663},{"type":"v1","addr":"172.21.15.195:6836","nonce":965972663}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6700,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/3176122334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":3176122334},{"type":"v1","addr":"172.21.15.195:6837","nonce":3176122334}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":8370,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/2916238332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":2916238332},{"type":"v1","addr":"172.21.15.38:6835","nonce":2916238332}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":146}],"filesystems":[{"mdsmap":{"epoch":149,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:40:02.319282+0000","modified":"2024-09-21T03:41:23.555289+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":107,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-21T03:41:26.300 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 149 2024-09-21T03:41:26.302 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:26.299+0000 7f5c3d827640 1 -- 172.21.15.38:0/1447112167 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5c24078b90 msgr2=0x7f5c2407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:26.302 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:26.299+0000 7f5c3d827640 1 --2- 172.21.15.38:0/1447112167 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5c24078b90 0x7f5c2407b0a0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f5c28005cc0 tx=0x7f5c2800a570 comp rx=0 tx=0).stop 2024-09-21T03:41:26.302 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:26.299+0000 7f5c3d827640 1 -- 172.21.15.38:0/1447112167 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5c380d0980 msgr2=0x7f5c3806f440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:26.302 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:26.299+0000 7f5c3d827640 1 --2- 172.21.15.38:0/1447112167 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5c380d0980 0x7f5c3806f440 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f5c2c00c970 tx=0x7f5c2c00ce30 comp rx=0 tx=0).stop 2024-09-21T03:41:26.302 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:26.300+0000 7f5c3d827640 1 -- 172.21.15.38:0/1447112167 shutdown_connections 2024-09-21T03:41:26.302 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:26.300+0000 7f5c3d827640 1 --2- 172.21.15.38:0/1447112167 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5c24078b90 0x7f5c2407b0a0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:26.302 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:26.300+0000 7f5c3d827640 1 --2- 172.21.15.38:0/1447112167 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5c3815a6f0 0x7f5c380c7280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:26.302 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:26.300+0000 7f5c3d827640 1 --2- 172.21.15.38:0/1447112167 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5c380d0980 0x7f5c3806f440 unknown :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:26.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:26.300+0000 7f5c3d827640 1 --2- 172.21.15.38:0/1447112167 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5c380d01f0 0x7f5c3806ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:26.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:26.300+0000 7f5c3d827640 1 -- 172.21.15.38:0/1447112167 >> 172.21.15.38:0/1447112167 conn(0x7f5c38072990 msgr2=0x7f5c38068d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:26.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:26.300+0000 7f5c3d827640 1 -- 172.21.15.38:0/1447112167 shutdown_connections 2024-09-21T03:41:26.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:26.300+0000 7f5c3d827640 1 -- 172.21.15.38:0/1447112167 wait complete. 2024-09-21T03:41:26.313 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-09-21T03:41:26.313 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:41:26.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:26.428+0000 7f94557b0640 1 -- 172.21.15.38:0/677572919 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f945015f5a0 msgr2=0x7f945015f9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:26.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:26.428+0000 7f94557b0640 1 --2- 172.21.15.38:0/677572919 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f945015f5a0 0x7f945015f9b0 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f9450072560 tx=0x7f944802a530 comp rx=0 tx=0).stop 2024-09-21T03:41:26.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:26.428+0000 7f94557b0640 1 -- 172.21.15.38:0/677572919 shutdown_connections 2024-09-21T03:41:26.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:26.428+0000 7f94557b0640 1 --2- 172.21.15.38:0/677572919 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f945015f5a0 0x7f945015f9b0 unknown :-1 s=CLOSED pgs=1480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:26.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:26.428+0000 7f94557b0640 1 --2- 172.21.15.38:0/677572919 >> v2:172.21.15.195:3300/0 conn(0x7f945015b9f0 0x7f94500cf930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:26.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:26.428+0000 7f94557b0640 1 -- 172.21.15.38:0/677572919 >> 172.21.15.38:0/677572919 conn(0x7f94500cca40 msgr2=0x7f94500cce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:26.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:26.428+0000 7f94557b0640 1 -- 172.21.15.38:0/677572919 shutdown_connections 2024-09-21T03:41:26.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:26.428+0000 7f94557b0640 1 -- 172.21.15.38:0/677572919 wait complete. 2024-09-21T03:41:26.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:26.430+0000 7f94557b0640 1 Processor -- start 2024-09-21T03:41:26.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:26.430+0000 7f94557b0640 1 -- start start 2024-09-21T03:41:26.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:26.431+0000 7f94557b0640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f94500cefa0 0x7f94500c97c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:26.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:26.431+0000 7f94557b0640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f945015b9f0 0x7f9450152a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:26.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:26.431+0000 7f94557b0640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f945015f5a0 0x7f94500c3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:26.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:26.431+0000 7f94557b0640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f945006df70 con 0x7f945015f5a0 2024-09-21T03:41:26.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:26.431+0000 7f94557b0640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f945006ddf0 con 0x7f945015b9f0 2024-09-21T03:41:26.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:26.431+0000 7f94557b0640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f945006e0f0 con 0x7f94500cefa0 2024-09-21T03:41:26.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:26.431+0000 7f944f7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f945015f5a0 0x7f94500c3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:26.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:26.431+0000 7f944f7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f945015f5a0 0x7f94500c3cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:52596/0 (socket says 172.21.15.38:52596) 2024-09-21T03:41:26.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:26.431+0000 7f944f7fe640 1 -- 172.21.15.38:0/2610418706 learned_addr learned my addr 172.21.15.38:0/2610418706 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:41:26.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:26.432+0000 7f944e7fc640 1 --2- 172.21.15.38:0/2610418706 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f945015b9f0 0x7f9450152a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:26.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:26.432+0000 7f944effd640 1 --2- 172.21.15.38:0/2610418706 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f94500cefa0 0x7f94500c97c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:26.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:26.432+0000 7f944f7fe640 1 -- 172.21.15.38:0/2610418706 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f94500cefa0 msgr2=0x7f94500c97c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:26.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:26.432+0000 7f944f7fe640 1 --2- 172.21.15.38:0/2610418706 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f94500cefa0 0x7f94500c97c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:26.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:26.432+0000 7f944f7fe640 1 -- 172.21.15.38:0/2610418706 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f945015b9f0 msgr2=0x7f9450152a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:26.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:26.432+0000 7f944f7fe640 1 --2- 172.21.15.38:0/2610418706 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f945015b9f0 0x7f9450152a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:26.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:26.432+0000 7f944f7fe640 1 -- 172.21.15.38:0/2610418706 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94500b0100 con 0x7f945015f5a0 2024-09-21T03:41:26.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:26.432+0000 7f944e7fc640 1 --2- 172.21.15.38:0/2610418706 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f945015b9f0 0x7f9450152a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:41:26.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:26.432+0000 7f944f7fe640 1 --2- 172.21.15.38:0/2610418706 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f945015f5a0 0x7f94500c3cd0 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f94480098f0 tx=0x7f9448033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:26.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:26.432+0000 7f94477fe640 1 -- 172.21.15.38:0/2610418706 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9448031d60 con 0x7f945015f5a0 2024-09-21T03:41:26.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:26.432+0000 7f94477fe640 1 -- 172.21.15.38:0/2610418706 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9448033e10 con 0x7f945015f5a0 2024-09-21T03:41:26.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:26.432+0000 7f94557b0640 1 -- 172.21.15.38:0/2610418706 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f945015a280 con 0x7f945015f5a0 2024-09-21T03:41:26.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:26.432+0000 7f94477fe640 1 -- 172.21.15.38:0/2610418706 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9448005260 con 0x7f945015f5a0 2024-09-21T03:41:26.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:26.432+0000 7f94557b0640 1 -- 172.21.15.38:0/2610418706 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94500c92a0 con 0x7f945015f5a0 2024-09-21T03:41:26.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:26.433+0000 7f94557b0640 1 -- 172.21.15.38:0/2610418706 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f945006e5a0 con 0x7f945015f5a0 2024-09-21T03:41:26.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:26.434+0000 7f94477fe640 1 -- 172.21.15.38:0/2610418706 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f9448005400 con 0x7f945015f5a0 2024-09-21T03:41:26.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:26.435+0000 7f94477fe640 1 --2- 172.21.15.38:0/2610418706 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9430078bc0 0x7f943007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:26.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:26.435+0000 7f94477fe640 1 -- 172.21.15.38:0/2610418706 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 7250+0+0 (secure 0 0 0) 0x7f94480bc8a0 con 0x7f945015f5a0 2024-09-21T03:41:26.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:26.435+0000 7f944effd640 1 --2- 172.21.15.38:0/2610418706 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9430078bc0 0x7f943007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:26.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:26.435+0000 7f944effd640 1 --2- 172.21.15.38:0/2610418706 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9430078bc0 0x7f943007b0d0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f9440005fd0 tx=0x7f9440009810 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:26.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:26.438+0000 7f94477fe640 1 -- 172.21.15.38:0/2610418706 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f944803f030 con 0x7f945015f5a0 2024-09-21T03:41:26.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:26.666+0000 7f94557b0640 1 -- 172.21.15.38:0/2610418706 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f945006f970 con 0x7f945015f5a0 2024-09-21T03:41:26.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:26.667+0000 7f94477fe640 1 -- 172.21.15.38:0/2610418706 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 149 v149) ==== 95+0+4392 (secure 0 0 0) 0x7f94480847b0 con 0x7f945015f5a0 2024-09-21T03:41:26.669 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:41:26.669 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":149,"btime":"2024-09-21T03:41:23:555294+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6086,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/965972663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":965972663},{"type":"v1","addr":"172.21.15.195:6836","nonce":965972663}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6700,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/3176122334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":3176122334},{"type":"v1","addr":"172.21.15.195:6837","nonce":3176122334}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":8370,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/2916238332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":2916238332},{"type":"v1","addr":"172.21.15.38:6835","nonce":2916238332}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":146}],"filesystems":[{"mdsmap":{"epoch":149,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:40:02.319282+0000","modified":"2024-09-21T03:41:23.555289+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":107,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-21T03:41:26.669 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 149 2024-09-21T03:41:26.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:26.669+0000 7f94557b0640 1 -- 172.21.15.38:0/2610418706 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9430078bc0 msgr2=0x7f943007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:26.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:26.669+0000 7f94557b0640 1 --2- 172.21.15.38:0/2610418706 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9430078bc0 0x7f943007b0d0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f9440005fd0 tx=0x7f9440009810 comp rx=0 tx=0).stop 2024-09-21T03:41:26.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:26.669+0000 7f94557b0640 1 -- 172.21.15.38:0/2610418706 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f945015f5a0 msgr2=0x7f94500c3cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:26.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:26.669+0000 7f94557b0640 1 --2- 172.21.15.38:0/2610418706 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f945015f5a0 0x7f94500c3cd0 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f94480098f0 tx=0x7f9448033c20 comp rx=0 tx=0).stop 2024-09-21T03:41:26.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:26.669+0000 7f94557b0640 1 -- 172.21.15.38:0/2610418706 shutdown_connections 2024-09-21T03:41:26.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:26.669+0000 7f94557b0640 1 --2- 172.21.15.38:0/2610418706 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f945015f5a0 0x7f94500c3cd0 unknown :-1 s=CLOSED pgs=1481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:26.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:26.669+0000 7f94557b0640 1 --2- 172.21.15.38:0/2610418706 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f945015b9f0 0x7f9450152a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:26.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:26.669+0000 7f94557b0640 1 --2- 172.21.15.38:0/2610418706 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9430078bc0 0x7f943007b0d0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:26.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:26.669+0000 7f94557b0640 1 --2- 172.21.15.38:0/2610418706 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f94500cefa0 0x7f94500c97c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:26.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:26.669+0000 7f94557b0640 1 -- 172.21.15.38:0/2610418706 >> 172.21.15.38:0/2610418706 conn(0x7f94500cca40 msgr2=0x7f94500732f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:26.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:26.670+0000 7f94557b0640 1 -- 172.21.15.38:0/2610418706 shutdown_connections 2024-09-21T03:41:26.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:26.670+0000 7f94557b0640 1 -- 172.21.15.38:0/2610418706 wait complete. 2024-09-21T03:41:26.683 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-21T03:41:26.683 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:41:26.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:26.798+0000 7f9dc4a0a640 1 -- 172.21.15.38:0/1928357263 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9dc0162f00 msgr2=0x7f9dc00d24d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:26.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:26.798+0000 7f9dc4a0a640 1 --2- 172.21.15.38:0/1928357263 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9dc0162f00 0x7f9dc00d24d0 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f9db0005fd0 tx=0x7f9db002aa00 comp rx=0 tx=0).stop 2024-09-21T03:41:26.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:26.798+0000 7f9dc4a0a640 1 -- 172.21.15.38:0/1928357263 shutdown_connections 2024-09-21T03:41:26.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:26.798+0000 7f9dc4a0a640 1 --2- 172.21.15.38:0/1928357263 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9dc00b2e40 0x7f9dc00b3250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:26.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:26.798+0000 7f9dc4a0a640 1 --2- 172.21.15.38:0/1928357263 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9dc0162f00 0x7f9dc00d24d0 unknown :-1 s=CLOSED pgs=1482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:26.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:26.798+0000 7f9dc4a0a640 1 -- 172.21.15.38:0/1928357263 >> 172.21.15.38:0/1928357263 conn(0x7f9dc00b6b20 msgr2=0x7f9dc00d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:26.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:26.798+0000 7f9dc4a0a640 1 -- 172.21.15.38:0/1928357263 shutdown_connections 2024-09-21T03:41:26.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:26.798+0000 7f9dc4a0a640 1 -- 172.21.15.38:0/1928357263 wait complete. 2024-09-21T03:41:26.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:26.800+0000 7f9dc4a0a640 1 Processor -- start 2024-09-21T03:41:26.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:26.800+0000 7f9dc4a0a640 1 -- start start 2024-09-21T03:41:26.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:26.801+0000 7f9dc4a0a640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9dc00b2e40 0x7f9dc00ce290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:26.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:26.801+0000 7f9dc4a0a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9dc0162280 0x7f9dc00aadc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:26.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:26.801+0000 7f9dc4a0a640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9dc00ab300 0x7f9dc015a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:26.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:26.801+0000 7f9dc4a0a640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f9dc015e4b0 con 0x7f9dc0162280 2024-09-21T03:41:26.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:26.802+0000 7f9dc4a0a640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f9dc00b3a20 con 0x7f9dc00b2e40 2024-09-21T03:41:26.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:26.802+0000 7f9dc4a0a640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f9dc015e630 con 0x7f9dc00ab300 2024-09-21T03:41:26.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:26.802+0000 7f9dbdd9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9dc0162280 0x7f9dc00aadc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:26.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:26.802+0000 7f9dbdd9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9dc0162280 0x7f9dc00aadc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:52612/0 (socket says 172.21.15.38:52612) 2024-09-21T03:41:26.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:26.802+0000 7f9dbdd9b640 1 -- 172.21.15.38:0/1474606955 learned_addr learned my addr 172.21.15.38:0/1474606955 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:41:26.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:26.802+0000 7f9dbe59c640 1 --2- 172.21.15.38:0/1474606955 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9dc00b2e40 0x7f9dc00ce290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:26.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:26.802+0000 7f9dbed9d640 1 --2- 172.21.15.38:0/1474606955 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9dc00ab300 0x7f9dc015a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:26.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:26.802+0000 7f9dbdd9b640 1 -- 172.21.15.38:0/1474606955 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9dc00ab300 msgr2=0x7f9dc015a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:26.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:26.802+0000 7f9dbdd9b640 1 --2- 172.21.15.38:0/1474606955 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9dc00ab300 0x7f9dc015a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:26.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:26.802+0000 7f9dbdd9b640 1 -- 172.21.15.38:0/1474606955 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9dc00b2e40 msgr2=0x7f9dc00ce290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:26.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:26.802+0000 7f9dbdd9b640 1 --2- 172.21.15.38:0/1474606955 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9dc00b2e40 0x7f9dc00ce290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:26.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:26.802+0000 7f9dbdd9b640 1 -- 172.21.15.38:0/1474606955 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9dc00ac2b0 con 0x7f9dc0162280 2024-09-21T03:41:26.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:26.802+0000 7f9dbdd9b640 1 --2- 172.21.15.38:0/1474606955 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9dc0162280 0x7f9dc00aadc0 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7f9da800c3f0 tx=0x7f9da800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:26.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:26.802+0000 7f9db6ffd640 1 -- 172.21.15.38:0/1474606955 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9da8011790 con 0x7f9dc0162280 2024-09-21T03:41:26.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:26.802+0000 7f9db6ffd640 1 -- 172.21.15.38:0/1474606955 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9da8017070 con 0x7f9dc0162280 2024-09-21T03:41:26.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:26.802+0000 7f9dc4a0a640 1 -- 172.21.15.38:0/1474606955 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9dc00aba10 con 0x7f9dc0162280 2024-09-21T03:41:26.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:26.802+0000 7f9db6ffd640 1 -- 172.21.15.38:0/1474606955 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9da8011ce0 con 0x7f9dc0162280 2024-09-21T03:41:26.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:26.803+0000 7f9dc4a0a640 1 -- 172.21.15.38:0/1474606955 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9dc00abfd0 con 0x7f9dc0162280 2024-09-21T03:41:26.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:26.804+0000 7f9dc4a0a640 1 -- 172.21.15.38:0/1474606955 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9dc00ab780 con 0x7f9dc0162280 2024-09-21T03:41:26.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:26.804+0000 7f9db6ffd640 1 -- 172.21.15.38:0/1474606955 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f9da801c460 con 0x7f9dc0162280 2024-09-21T03:41:26.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:26.805+0000 7f9db6ffd640 1 --2- 172.21.15.38:0/1474606955 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9da0078bc0 0x7f9da007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:26.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:26.805+0000 7f9db6ffd640 1 -- 172.21.15.38:0/1474606955 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 7250+0+0 (secure 0 0 0) 0x7f9da809b040 con 0x7f9dc0162280 2024-09-21T03:41:26.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:26.805+0000 7f9dbe59c640 1 --2- 172.21.15.38:0/1474606955 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9da0078bc0 0x7f9da007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:26.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:26.806+0000 7f9dbe59c640 1 --2- 172.21.15.38:0/1474606955 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9da0078bc0 0x7f9da007b0d0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f9db0035000 tx=0x7f9db003b040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:26.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:26.810+0000 7f9db6ffd640 1 -- 172.21.15.38:0/1474606955 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f9da8062820 con 0x7f9dc0162280 2024-09-21T03:41:27.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:27.034+0000 7f9dc4a0a640 1 -- 172.21.15.38:0/1474606955 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9dc0162f00 con 0x7f9dc0162280 2024-09-21T03:41:27.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:27.035+0000 7f9db6ffd640 1 -- 172.21.15.38:0/1474606955 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 149 v149) ==== 95+0+4392 (secure 0 0 0) 0x7f9da8067250 con 0x7f9dc0162280 2024-09-21T03:41:27.038 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:41:27.038 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":149,"btime":"2024-09-21T03:41:23:555294+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6086,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/965972663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":965972663},{"type":"v1","addr":"172.21.15.195:6836","nonce":965972663}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6700,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/3176122334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":3176122334},{"type":"v1","addr":"172.21.15.195:6837","nonce":3176122334}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":8370,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/2916238332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":2916238332},{"type":"v1","addr":"172.21.15.38:6835","nonce":2916238332}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":146}],"filesystems":[{"mdsmap":{"epoch":149,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:40:02.319282+0000","modified":"2024-09-21T03:41:23.555289+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":107,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-21T03:41:27.039 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 149 2024-09-21T03:41:27.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:27.037+0000 7f9dc4a0a640 1 -- 172.21.15.38:0/1474606955 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9da0078bc0 msgr2=0x7f9da007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:27.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:27.037+0000 7f9dc4a0a640 1 --2- 172.21.15.38:0/1474606955 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9da0078bc0 0x7f9da007b0d0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f9db0035000 tx=0x7f9db003b040 comp rx=0 tx=0).stop 2024-09-21T03:41:27.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:27.037+0000 7f9dc4a0a640 1 -- 172.21.15.38:0/1474606955 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9dc0162280 msgr2=0x7f9dc00aadc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:27.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:27.037+0000 7f9dc4a0a640 1 --2- 172.21.15.38:0/1474606955 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9dc0162280 0x7f9dc00aadc0 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7f9da800c3f0 tx=0x7f9da800c8b0 comp rx=0 tx=0).stop 2024-09-21T03:41:27.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:27.037+0000 7f9dc4a0a640 1 -- 172.21.15.38:0/1474606955 shutdown_connections 2024-09-21T03:41:27.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:27.037+0000 7f9dc4a0a640 1 --2- 172.21.15.38:0/1474606955 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9da0078bc0 0x7f9da007b0d0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:27.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:27.037+0000 7f9dc4a0a640 1 --2- 172.21.15.38:0/1474606955 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9dc00ab300 0x7f9dc015a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:27.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:27.037+0000 7f9dc4a0a640 1 --2- 172.21.15.38:0/1474606955 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9dc0162280 0x7f9dc00aadc0 unknown :-1 s=CLOSED pgs=1483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:27.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:27.037+0000 7f9dc4a0a640 1 --2- 172.21.15.38:0/1474606955 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9dc00b2e40 0x7f9dc00ce290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:27.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:27.037+0000 7f9dc4a0a640 1 -- 172.21.15.38:0/1474606955 >> 172.21.15.38:0/1474606955 conn(0x7f9dc00b6b20 msgr2=0x7f9dc00ceb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:27.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:27.037+0000 7f9dc4a0a640 1 -- 172.21.15.38:0/1474606955 shutdown_connections 2024-09-21T03:41:27.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:27.037+0000 7f9dc4a0a640 1 -- 172.21.15.38:0/1474606955 wait complete. 2024-09-21T03:41:27.050 INFO:tasks.cephfs.test_damage:Result: Mutation 'Delete 100.00000000' led to DAMAGED state 2024-09-21T03:41:27.050 INFO:tasks.cephfs.test_damage:Applying mutation 'Delete mds0_sessionmap' 2024-09-21T03:41:27.050 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-21T03:41:27.050 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:41:27.068 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056795,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2354 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T03:41:27.068 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi038.front.sepia.ceph.com 2024-09-21T03:41:27.068 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:41:27.068 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T03:41:27.124 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T03:41:27.124 INFO:teuthology.orchestra.run.smithi038.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-09-21T03:41:27.125 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-21T03:41:27.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:27.280+0000 7f1eb96bd640 1 -- 172.21.15.38:0/3602588526 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1eb40ceb90 msgr2=0x7f1eb40cefa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:27.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:27.280+0000 7f1eb96bd640 1 --2- 172.21.15.38:0/3602588526 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1eb40ceb90 0x7f1eb40cefa0 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f1ea4005fd0 tx=0x7f1ea402a820 comp rx=0 tx=0).stop 2024-09-21T03:41:27.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:27.280+0000 7f1eb96bd640 1 -- 172.21.15.38:0/3602588526 shutdown_connections 2024-09-21T03:41:27.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:27.280+0000 7f1eb96bd640 1 --2- 172.21.15.38:0/3602588526 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1eb40ceb90 0x7f1eb40cefa0 unknown :-1 s=CLOSED pgs=1485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:27.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:27.280+0000 7f1eb96bd640 1 --2- 172.21.15.38:0/3602588526 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1eb415ee80 0x7f1eb40d0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:27.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:27.280+0000 7f1eb96bd640 1 -- 172.21.15.38:0/3602588526 >> 172.21.15.38:0/3602588526 conn(0x7f1eb409d830 msgr2=0x7f1eb40b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:27.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:27.280+0000 7f1eb96bd640 1 -- 172.21.15.38:0/3602588526 shutdown_connections 2024-09-21T03:41:27.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:27.280+0000 7f1eb96bd640 1 -- 172.21.15.38:0/3602588526 wait complete. 2024-09-21T03:41:27.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:27.282+0000 7f1eb96bd640 1 Processor -- start 2024-09-21T03:41:27.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:27.282+0000 7f1eb96bd640 1 -- start start 2024-09-21T03:41:27.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:27.283+0000 7f1eb96bd640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1eb40b9ad0 0x7f1eb409e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:27.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:27.283+0000 7f1eb96bd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1eb40ceb90 0x7f1eb414f900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:27.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:27.283+0000 7f1eb96bd640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1eb415ee80 0x7f1eb406e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:27.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:27.283+0000 7f1eb96bd640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f1eb40b6700 con 0x7f1eb40ceb90 2024-09-21T03:41:27.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:27.283+0000 7f1eb96bd640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f1eb40b6580 con 0x7f1eb415ee80 2024-09-21T03:41:27.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:27.283+0000 7f1eb96bd640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f1eb40b6880 con 0x7f1eb40b9ad0 2024-09-21T03:41:27.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:27.284+0000 7f1eb27fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1eb40ceb90 0x7f1eb414f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:27.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:27.284+0000 7f1eb27fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1eb40ceb90 0x7f1eb414f900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:52618/0 (socket says 172.21.15.38:52618) 2024-09-21T03:41:27.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:27.284+0000 7f1eb27fc640 1 -- 172.21.15.38:0/1267616666 learned_addr learned my addr 172.21.15.38:0/1267616666 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:41:27.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:27.284+0000 7f1eb2ffd640 1 --2- 172.21.15.38:0/1267616666 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1eb40b9ad0 0x7f1eb409e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:27.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:27.284+0000 7f1eb37fe640 1 --2- 172.21.15.38:0/1267616666 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1eb415ee80 0x7f1eb406e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:27.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:27.284+0000 7f1eb27fc640 1 -- 172.21.15.38:0/1267616666 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1eb40b9ad0 msgr2=0x7f1eb409e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:27.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:27.284+0000 7f1eb27fc640 1 --2- 172.21.15.38:0/1267616666 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1eb40b9ad0 0x7f1eb409e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:27.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:27.284+0000 7f1eb27fc640 1 -- 172.21.15.38:0/1267616666 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1eb415ee80 msgr2=0x7f1eb406e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:27.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:27.284+0000 7f1eb27fc640 1 --2- 172.21.15.38:0/1267616666 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1eb415ee80 0x7f1eb406e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:27.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:27.284+0000 7f1eb27fc640 1 -- 172.21.15.38:0/1267616666 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1eb41f5980 con 0x7f1eb40ceb90 2024-09-21T03:41:27.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:27.284+0000 7f1eb2ffd640 1 --2- 172.21.15.38:0/1267616666 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1eb40b9ad0 0x7f1eb409e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:41:27.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:27.284+0000 7f1eb27fc640 1 --2- 172.21.15.38:0/1267616666 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1eb40ceb90 0x7f1eb414f900 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f1e9c001eb0 tx=0x7f1e9c00c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:27.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:27.284+0000 7f1eab7fe640 1 -- 172.21.15.38:0/1267616666 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e9c017070 con 0x7f1eb40ceb90 2024-09-21T03:41:27.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:27.284+0000 7f1eab7fe640 1 -- 172.21.15.38:0/1267616666 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1e9c0035f0 con 0x7f1eb40ceb90 2024-09-21T03:41:27.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:27.284+0000 7f1eab7fe640 1 -- 172.21.15.38:0/1267616666 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e9c003a00 con 0x7f1eb40ceb90 2024-09-21T03:41:27.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:27.284+0000 7f1eb96bd640 1 -- 172.21.15.38:0/1267616666 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1eb41f5c60 con 0x7f1eb40ceb90 2024-09-21T03:41:27.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:27.284+0000 7f1eb96bd640 1 -- 172.21.15.38:0/1267616666 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1eb41f6550 con 0x7f1eb40ceb90 2024-09-21T03:41:27.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:27.285+0000 7f1eb96bd640 1 -- 172.21.15.38:0/1267616666 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1eb41f6180 con 0x7f1eb40ceb90 2024-09-21T03:41:27.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:27.286+0000 7f1eab7fe640 1 -- 172.21.15.38:0/1267616666 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f1e9c01a070 con 0x7f1eb40ceb90 2024-09-21T03:41:27.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:27.287+0000 7f1eab7fe640 1 --2- 172.21.15.38:0/1267616666 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1e94078b90 0x7f1e9407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:27.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:27.287+0000 7f1eab7fe640 1 -- 172.21.15.38:0/1267616666 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 7250+0+0 (secure 0 0 0) 0x7f1e9c09a5b0 con 0x7f1eb40ceb90 2024-09-21T03:41:27.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:27.287+0000 7f1eb2ffd640 1 --2- 172.21.15.38:0/1267616666 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1e94078b90 0x7f1e9407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:27.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:27.288+0000 7f1eb2ffd640 1 --2- 172.21.15.38:0/1267616666 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1e94078b90 0x7f1e9407b0a0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f1eac005fd0 tx=0x7f1eac00f360 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:27.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:27.292+0000 7f1eab7fe640 1 -- 172.21.15.38:0/1267616666 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f1eb41f6180 con 0x7f1eb40ceb90 2024-09-21T03:41:27.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:27.519+0000 7f1eb96bd640 1 -- 172.21.15.38:0/1267616666 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f1eb41f5710 con 0x7f1eb40ceb90 2024-09-21T03:41:27.581 INFO:tasks.ceph.mds.c.smithi038.stderr:2024-09-21T03:41:27.579+0000 7ff41fcfc640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T03:41:27.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:27.614+0000 7f1eab7fe640 1 -- 172.21.15.38:0/1267616666 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v150) ==== 161+0+0 (secure 0 0 0) 0x7f1eb41592d0 con 0x7f1eb40ceb90 2024-09-21T03:41:27.616 INFO:teuthology.orchestra.run.smithi038.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-21T03:41:27.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:27.616+0000 7f1eb96bd640 1 -- 172.21.15.38:0/1267616666 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1e94078b90 msgr2=0x7f1e9407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:27.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:27.616+0000 7f1eb96bd640 1 --2- 172.21.15.38:0/1267616666 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1e94078b90 0x7f1e9407b0a0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f1eac005fd0 tx=0x7f1eac00f360 comp rx=0 tx=0).stop 2024-09-21T03:41:27.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:27.616+0000 7f1eb96bd640 1 -- 172.21.15.38:0/1267616666 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1eb40ceb90 msgr2=0x7f1eb414f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:27.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:27.616+0000 7f1eb96bd640 1 --2- 172.21.15.38:0/1267616666 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1eb40ceb90 0x7f1eb414f900 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f1e9c001eb0 tx=0x7f1e9c00c820 comp rx=0 tx=0).stop 2024-09-21T03:41:27.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:27.616+0000 7f1eb96bd640 1 -- 172.21.15.38:0/1267616666 shutdown_connections 2024-09-21T03:41:27.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:27.616+0000 7f1eb96bd640 1 --2- 172.21.15.38:0/1267616666 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1eb415ee80 0x7f1eb406e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:27.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:27.616+0000 7f1eb96bd640 1 --2- 172.21.15.38:0/1267616666 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1eb40ceb90 0x7f1eb414f900 unknown :-1 s=CLOSED pgs=1486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:27.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:27.616+0000 7f1eb96bd640 1 --2- 172.21.15.38:0/1267616666 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1e94078b90 0x7f1e9407b0a0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:27.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:27.616+0000 7f1eb96bd640 1 --2- 172.21.15.38:0/1267616666 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1eb40b9ad0 0x7f1eb409e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:27.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:27.616+0000 7f1eb96bd640 1 -- 172.21.15.38:0/1267616666 >> 172.21.15.38:0/1267616666 conn(0x7f1eb409d830 msgr2=0x7f1eb40b48d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:27.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:27.616+0000 7f1eb96bd640 1 -- 172.21.15.38:0/1267616666 shutdown_connections 2024-09-21T03:41:27.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:27.616+0000 7f1eb96bd640 1 -- 172.21.15.38:0/1267616666 wait complete. 2024-09-21T03:41:27.630 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-09-21T03:41:27.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:27.746+0000 7f87f94ce640 1 -- 172.21.15.38:0/3891029545 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f87f40ceb90 msgr2=0x7f87f40cefa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:27.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:27.746+0000 7f87f94ce640 1 --2- 172.21.15.38:0/3891029545 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f87f40ceb90 0x7f87f40cefa0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f87ec005fd0 tx=0x7f87ec02a810 comp rx=0 tx=0).stop 2024-09-21T03:41:27.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:27.746+0000 7f87f94ce640 1 -- 172.21.15.38:0/3891029545 shutdown_connections 2024-09-21T03:41:27.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:27.746+0000 7f87f94ce640 1 --2- 172.21.15.38:0/3891029545 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f87f40ceb90 0x7f87f40cefa0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:27.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:27.746+0000 7f87f94ce640 1 --2- 172.21.15.38:0/3891029545 >> v2:172.21.15.195:3300/0 conn(0x7f87f415ee80 0x7f87f40d0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:27.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:27.746+0000 7f87f94ce640 1 -- 172.21.15.38:0/3891029545 >> 172.21.15.38:0/3891029545 conn(0x7f87f409d830 msgr2=0x7f87f40b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:27.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:27.746+0000 7f87f94ce640 1 -- 172.21.15.38:0/3891029545 shutdown_connections 2024-09-21T03:41:27.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:27.746+0000 7f87f94ce640 1 -- 172.21.15.38:0/3891029545 wait complete. 2024-09-21T03:41:27.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:27.748+0000 7f87f94ce640 1 Processor -- start 2024-09-21T03:41:27.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:27.748+0000 7f87f94ce640 1 -- start start 2024-09-21T03:41:27.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:27.749+0000 7f87f94ce640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f87f40b9ad0 0x7f87f406f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:27.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:27.749+0000 7f87f94ce640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f87f40ceb90 0x7f87f40cd480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:27.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:27.749+0000 7f87f94ce640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f87f415ee80 0x7f87f40cd9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:27.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:27.749+0000 7f87f94ce640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f87f40b6580 con 0x7f87f415ee80 2024-09-21T03:41:27.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:27.749+0000 7f87f94ce640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f87f40ba0f0 con 0x7f87f40b9ad0 2024-09-21T03:41:27.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:27.749+0000 7f87f94ce640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f87f40b6700 con 0x7f87f40ceb90 2024-09-21T03:41:27.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:27.749+0000 7f87f37fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f87f415ee80 0x7f87f40cd9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:27.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:27.749+0000 7f87f37fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f87f415ee80 0x7f87f40cd9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:52632/0 (socket says 172.21.15.38:52632) 2024-09-21T03:41:27.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:27.749+0000 7f87f37fe640 1 -- 172.21.15.38:0/4290022188 learned_addr learned my addr 172.21.15.38:0/4290022188 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:41:27.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:27.750+0000 7f87f2ffd640 1 --2- 172.21.15.38:0/4290022188 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f87f40b9ad0 0x7f87f406f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:27.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:27.749+0000 7f87f27fc640 1 --2- 172.21.15.38:0/4290022188 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f87f40ceb90 0x7f87f40cd480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:27.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:27.750+0000 7f87f37fe640 1 -- 172.21.15.38:0/4290022188 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f87f40ceb90 msgr2=0x7f87f40cd480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:27.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:27.750+0000 7f87f37fe640 1 --2- 172.21.15.38:0/4290022188 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f87f40ceb90 0x7f87f40cd480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:27.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:27.750+0000 7f87f37fe640 1 -- 172.21.15.38:0/4290022188 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f87f40b9ad0 msgr2=0x7f87f406f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:27.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:27.750+0000 7f87f37fe640 1 --2- 172.21.15.38:0/4290022188 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f87f40b9ad0 0x7f87f406f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:27.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:27.750+0000 7f87f37fe640 1 -- 172.21.15.38:0/4290022188 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87f40cc590 con 0x7f87f415ee80 2024-09-21T03:41:27.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:27.750+0000 7f87f37fe640 1 --2- 172.21.15.38:0/4290022188 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f87f415ee80 0x7f87f40cd9c0 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f87ec035040 tx=0x7f87ec031650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:27.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:27.750+0000 7f87db7fe640 1 -- 172.21.15.38:0/4290022188 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87ec041070 con 0x7f87f415ee80 2024-09-21T03:41:27.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:27.750+0000 7f87db7fe640 1 -- 172.21.15.38:0/4290022188 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f87ec031a50 con 0x7f87f415ee80 2024-09-21T03:41:27.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:27.750+0000 7f87f94ce640 1 -- 172.21.15.38:0/4290022188 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87f40cc870 con 0x7f87f415ee80 2024-09-21T03:41:27.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:27.750+0000 7f87db7fe640 1 -- 172.21.15.38:0/4290022188 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87ec037040 con 0x7f87f415ee80 2024-09-21T03:41:27.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:27.750+0000 7f87f94ce640 1 -- 172.21.15.38:0/4290022188 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87f40cc020 con 0x7f87f415ee80 2024-09-21T03:41:27.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:27.751+0000 7f87f94ce640 1 -- 172.21.15.38:0/4290022188 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87f406a580 con 0x7f87f415ee80 2024-09-21T03:41:27.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:27.752+0000 7f87db7fe640 1 -- 172.21.15.38:0/4290022188 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f87ec044070 con 0x7f87f415ee80 2024-09-21T03:41:27.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:27.752+0000 7f87db7fe640 1 --2- 172.21.15.38:0/4290022188 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f87e0078b90 0x7f87e007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:27.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:27.753+0000 7f87db7fe640 1 -- 172.21.15.38:0/4290022188 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 7250+0+0 (secure 0 0 0) 0x7f87ec0bc210 con 0x7f87f415ee80 2024-09-21T03:41:27.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:27.753+0000 7f87f2ffd640 1 --2- 172.21.15.38:0/4290022188 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f87e0078b90 0x7f87e007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:27.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:27.753+0000 7f87f2ffd640 1 --2- 172.21.15.38:0/4290022188 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f87e0078b90 0x7f87e007b0a0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f87e40095f0 tx=0x7f87e4005f90 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:27.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:27.757+0000 7f87db7fe640 1 -- 172.21.15.38:0/4290022188 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f87f406a580 con 0x7f87f415ee80 2024-09-21T03:41:28.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.021+0000 7f87f94ce640 1 -- 172.21.15.38:0/4290022188 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7f87f40b6be0 con 0x7f87f415ee80 2024-09-21T03:41:28.305 INFO:tasks.ceph.mds.c.smithi038.stderr: -54> 2024-09-21T03:41:23.551+0000 7ff41bcf4640 -1 log_channel(cluster) log [ERR] : dir 0x100 object missing on disk; some files may be lost (~mds0) 2024-09-21T03:41:28.306 INFO:tasks.ceph.mds.c.smithi038.stderr: -8> 2024-09-21T03:41:27.579+0000 7ff41fcfc640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T03:41:28.360 INFO:tasks.ceph.mds.c.smithi038.stdout:starting mds.c at 2024-09-21T03:41:28.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.620+0000 7f87db7fe640 1 -- 172.21.15.38:0/4290022188 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v151) ==== 101+0+0 (secure 0 0 0) 0x7f87ec083a80 con 0x7f87f415ee80 2024-09-21T03:41:28.622 INFO:teuthology.orchestra.run.smithi038.stderr:repaired: restoring rank 6:0 2024-09-21T03:41:28.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.623+0000 7f87f94ce640 1 -- 172.21.15.38:0/4290022188 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f87e0078b90 msgr2=0x7f87e007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:28.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.623+0000 7f87f94ce640 1 --2- 172.21.15.38:0/4290022188 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f87e0078b90 0x7f87e007b0a0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f87e40095f0 tx=0x7f87e4005f90 comp rx=0 tx=0).stop 2024-09-21T03:41:28.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.623+0000 7f87f94ce640 1 -- 172.21.15.38:0/4290022188 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f87f415ee80 msgr2=0x7f87f40cd9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:28.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.623+0000 7f87f94ce640 1 --2- 172.21.15.38:0/4290022188 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f87f415ee80 0x7f87f40cd9c0 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f87ec035040 tx=0x7f87ec031650 comp rx=0 tx=0).stop 2024-09-21T03:41:28.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.623+0000 7f87f94ce640 1 -- 172.21.15.38:0/4290022188 shutdown_connections 2024-09-21T03:41:28.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.623+0000 7f87f94ce640 1 --2- 172.21.15.38:0/4290022188 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f87f415ee80 0x7f87f40cd9c0 unknown :-1 s=CLOSED pgs=1487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:28.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.623+0000 7f87f94ce640 1 --2- 172.21.15.38:0/4290022188 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f87e0078b90 0x7f87e007b0a0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:28.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.623+0000 7f87f94ce640 1 --2- 172.21.15.38:0/4290022188 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f87f40ceb90 0x7f87f40cd480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:28.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.623+0000 7f87f94ce640 1 --2- 172.21.15.38:0/4290022188 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f87f40b9ad0 0x7f87f406f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:28.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.623+0000 7f87f94ce640 1 -- 172.21.15.38:0/4290022188 >> 172.21.15.38:0/4290022188 conn(0x7f87f409d830 msgr2=0x7f87f4160a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:28.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.623+0000 7f87f94ce640 1 -- 172.21.15.38:0/4290022188 shutdown_connections 2024-09-21T03:41:28.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.623+0000 7f87f94ce640 1 -- 172.21.15.38:0/4290022188 wait complete. 2024-09-21T03:41:28.637 DEBUG:teuthology.orchestra.run.smithi038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-09-21T03:41:28.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.691+0000 7f97c5125f40 1 Processor -- start 2024-09-21T03:41:28.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.691+0000 7f97c5125f40 1 -- start start 2024-09-21T03:41:28.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.692+0000 7f97c5125f40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5605bcc61d00 0x5605bcd37c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:28.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.692+0000 7f97c5125f40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5605bcd1c950 0x5605bcc8dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:28.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.692+0000 7f97c5125f40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5605bcd37670 0x5605bcc8e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:28.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.692+0000 7f97c5125f40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5605bcca9ef0 con 0x5605bcd37670 2024-09-21T03:41:28.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.692+0000 7f97c5125f40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5605bcbafd60 con 0x5605bcd1c950 2024-09-21T03:41:28.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.692+0000 7f97c5125f40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x5605bccaa460 con 0x5605bcc61d00 2024-09-21T03:41:28.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.693+0000 7f97beac8640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5605bcd37670 0x5605bcc8e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:28.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.693+0000 7f97beac8640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5605bcd37670 0x5605bcc8e2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:52682/0 (socket says 172.21.15.38:52682) 2024-09-21T03:41:28.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.693+0000 7f97beac8640 1 -- 172.21.15.38:0/1241826743 learned_addr learned my addr 172.21.15.38:0/1241826743 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:41:28.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.693+0000 7f97be2c7640 1 --2- 172.21.15.38:0/1241826743 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5605bcc61d00 0x5605bcd37c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:28.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.693+0000 7f97bdac6640 1 --2- 172.21.15.38:0/1241826743 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5605bcd1c950 0x5605bcc8dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:28.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.693+0000 7f97beac8640 1 -- 172.21.15.38:0/1241826743 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5605bcc61d00 msgr2=0x5605bcd37c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:28.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.693+0000 7f97beac8640 1 --2- 172.21.15.38:0/1241826743 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5605bcc61d00 0x5605bcd37c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:28.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.693+0000 7f97beac8640 1 -- 172.21.15.38:0/1241826743 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5605bcd1c950 msgr2=0x5605bcc8dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:28.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.693+0000 7f97beac8640 1 --2- 172.21.15.38:0/1241826743 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5605bcd1c950 0x5605bcc8dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:28.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.693+0000 7f97beac8640 1 -- 172.21.15.38:0/1241826743 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605bcc8ed70 con 0x5605bcd37670 2024-09-21T03:41:28.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.693+0000 7f97bdac6640 1 --2- 172.21.15.38:0/1241826743 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5605bcd1c950 0x5605bcc8dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:41:28.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.693+0000 7f97beac8640 1 --2- 172.21.15.38:0/1241826743 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5605bcd37670 0x5605bcc8e2e0 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f97b800c4d0 tx=0x7f97b800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef5b805903246c62 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:28.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.693+0000 7f97bcac4640 1 -- 172.21.15.38:0/1241826743 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97b800a9e0 con 0x5605bcd37670 2024-09-21T03:41:28.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.693+0000 7f97bcac4640 1 -- 172.21.15.38:0/1241826743 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f97b800ab80 con 0x5605bcd37670 2024-09-21T03:41:28.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.693+0000 7f97bcac4640 1 -- 172.21.15.38:0/1241826743 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97b8008420 con 0x5605bcd37670 2024-09-21T03:41:28.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.694+0000 7f97c5125f40 1 -- 172.21.15.38:0/1241826743 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5605bcd37670 msgr2=0x5605bcc8e2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:28.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.694+0000 7f97c5125f40 1 --2- 172.21.15.38:0/1241826743 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5605bcd37670 0x5605bcc8e2e0 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f97b800c4d0 tx=0x7f97b800c990 comp rx=0 tx=0).stop 2024-09-21T03:41:28.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.694+0000 7f97c5125f40 1 -- 172.21.15.38:0/1241826743 shutdown_connections 2024-09-21T03:41:28.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.694+0000 7f97c5125f40 1 --2- 172.21.15.38:0/1241826743 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5605bcd37670 0x5605bcc8e2e0 unknown :-1 s=CLOSED pgs=1491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:28.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.694+0000 7f97c5125f40 1 --2- 172.21.15.38:0/1241826743 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5605bcd1c950 0x5605bcc8dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:28.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.694+0000 7f97c5125f40 1 --2- 172.21.15.38:0/1241826743 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5605bcc61d00 0x5605bcd37c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:28.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.694+0000 7f97c5125f40 1 -- 172.21.15.38:0/1241826743 >> 172.21.15.38:0/1241826743 conn(0x5605bcb80ac0 msgr2=0x5605bcd18660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:28.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.694+0000 7f97c5125f40 1 -- 172.21.15.38:0/1241826743 shutdown_connections 2024-09-21T03:41:28.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.694+0000 7f97c5125f40 1 -- 172.21.15.38:0/1241826743 wait complete. 2024-09-21T03:41:28.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.695+0000 7f97c5125f40 1 Processor -- start 2024-09-21T03:41:28.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.695+0000 7f97c5125f40 1 -- start start 2024-09-21T03:41:28.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.696+0000 7f97c5125f40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5605bcc61d00 0x5605bcd13e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:28.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.696+0000 7f97c5125f40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5605bcd1c950 0x5605bcc8f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:28.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.696+0000 7f97c5125f40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5605bcd37670 0x5605bcc8f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:28.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.696+0000 7f97c5125f40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5605bcc8a3e0 con 0x5605bcd1c950 2024-09-21T03:41:28.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.696+0000 7f97c5125f40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5605bcc8c4b0 con 0x5605bcc61d00 2024-09-21T03:41:28.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.696+0000 7f97c5125f40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x5605bcca1fd0 con 0x5605bcd37670 2024-09-21T03:41:28.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.696+0000 7f97bdac6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5605bcd1c950 0x5605bcc8f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:28.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.696+0000 7f97bdac6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5605bcd1c950 0x5605bcc8f110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:52694/0 (socket says 172.21.15.38:52694) 2024-09-21T03:41:28.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.696+0000 7f97bdac6640 1 -- 172.21.15.38:0/3696445974 learned_addr learned my addr 172.21.15.38:0/3696445974 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:41:28.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.696+0000 7f97be2c7640 1 --2- 172.21.15.38:0/3696445974 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5605bcc61d00 0x5605bcd13e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:28.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.696+0000 7f97beac8640 1 --2- 172.21.15.38:0/3696445974 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5605bcd37670 0x5605bcc8f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:28.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.697+0000 7f97bdac6640 1 -- 172.21.15.38:0/3696445974 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5605bcd37670 msgr2=0x5605bcc8f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:28.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.697+0000 7f97bdac6640 1 --2- 172.21.15.38:0/3696445974 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5605bcd37670 0x5605bcc8f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:28.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.697+0000 7f97bdac6640 1 -- 172.21.15.38:0/3696445974 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5605bcc61d00 msgr2=0x5605bcd13e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:28.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.697+0000 7f97bdac6640 1 --2- 172.21.15.38:0/3696445974 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5605bcc61d00 0x5605bcd13e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:28.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.697+0000 7f97bdac6640 1 -- 172.21.15.38:0/3696445974 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605bcca6480 con 0x5605bcd1c950 2024-09-21T03:41:28.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.697+0000 7f97bdac6640 1 --2- 172.21.15.38:0/3696445974 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5605bcd1c950 0x5605bcc8f110 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f97b400a680 tx=0x7f97b400ab40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:28.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.697+0000 7f97af7fe640 1 -- 172.21.15.38:0/3696445974 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97b4008990 con 0x5605bcd1c950 2024-09-21T03:41:28.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.697+0000 7f97af7fe640 1 -- 172.21.15.38:0/3696445974 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f97b4008b30 con 0x5605bcd1c950 2024-09-21T03:41:28.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.697+0000 7f97c5125f40 1 -- 172.21.15.38:0/3696445974 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5605bcdd0680 con 0x5605bcd1c950 2024-09-21T03:41:28.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.697+0000 7f97af7fe640 1 -- 172.21.15.38:0/3696445974 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97b4015410 con 0x5605bcd1c950 2024-09-21T03:41:28.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.697+0000 7f97c5125f40 1 -- 172.21.15.38:0/3696445974 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5605bcd176b0 con 0x5605bcd1c950 2024-09-21T03:41:28.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.699+0000 7f97af7fe640 1 -- 172.21.15.38:0/3696445974 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f97b40155b0 con 0x5605bcd1c950 2024-09-21T03:41:28.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.699+0000 7f97af7fe640 1 --2- 172.21.15.38:0/3696445974 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f97a0077570 0x7f97a0079a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:28.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.699+0000 7f97af7fe640 1 -- 172.21.15.38:0/3696445974 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 7250+0+0 (secure 0 0 0) 0x7f97b4099d60 con 0x5605bcd1c950 2024-09-21T03:41:28.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.699+0000 7f97be2c7640 1 --2- 172.21.15.38:0/3696445974 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f97a0077570 0x7f97a0079a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:28.703 INFO:teuthology.orchestra.run.smithi038.stdout:Importing pool 2024-09-21T03:41:28.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.699+0000 7f97c5125f40 1 --2- 172.21.15.38:0/3696445974 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x5605bcd19e00 0x5605bcca1650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:28.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.699+0000 7f97c5125f40 1 -- 172.21.15.38:0/3696445974 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5605bcdd8bb0 con 0x5605bcd19e00 2024-09-21T03:41:28.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.700+0000 7f97beac8640 1 --2- 172.21.15.38:0/3696445974 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x5605bcd19e00 0x5605bcca1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:28.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.700+0000 7f97be2c7640 1 --2- 172.21.15.38:0/3696445974 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f97a0077570 0x7f97a0079a80 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f97b0056000 tx=0x7f97b00033a0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:28.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.700+0000 7f97beac8640 1 --2- 172.21.15.38:0/3696445974 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x5605bcd19e00 0x5605bcca1650 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:28.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.702+0000 7f97beac8640 1 -- 172.21.15.38:0/3696445974 <== osd.7 v2:172.21.15.195:6818/1606106342 1 ==== osd_op_reply(1 601.00000000 [create] v107'319 uv293 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f97b800aa00 con 0x5605bcd19e00 2024-09-21T03:41:28.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.702+0000 7f97c5125f40 1 -- 172.21.15.38:0/3696445974 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e107) -- 0x5605bcdd8f50 con 0x5605bcd19e00 2024-09-21T03:41:28.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.703+0000 7f97beac8640 1 -- 172.21.15.38:0/3696445974 <== osd.7 v2:172.21.15.195:6818/1606106342 2 ==== osd_op_reply(2 601.00000000 [delete] v107'320 uv320 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f97b800aa00 con 0x5605bcd19e00 2024-09-21T03:41:28.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.703+0000 7f97c5125f40 1 -- 172.21.15.38:0/3696445974 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5605bcdd92f0 con 0x5605bcd19e00 2024-09-21T03:41:28.706 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-09-21T03:41:28.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.705+0000 7f97beac8640 1 -- 172.21.15.38:0/3696445974 <== osd.7 v2:172.21.15.195:6818/1606106342 3 ==== osd_op_reply(3 601.00000000 [create] v107'321 uv321 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f97b800aa00 con 0x5605bcd19e00 2024-09-21T03:41:28.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.705+0000 7f97c5125f40 1 -- 172.21.15.38:0/3696445974 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5605bcdda6a0 con 0x5605bcd19e00 2024-09-21T03:41:28.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.706+0000 7f97beac8640 1 -- 172.21.15.38:0/3696445974 <== osd.7 v2:172.21.15.195:6818/1606106342 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v107'322 uv322 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f97b800aa00 con 0x5605bcd19e00 2024-09-21T03:41:28.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.706+0000 7f97c5125f40 1 -- 172.21.15.38:0/3696445974 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5605bcddaa40 con 0x5605bcd19e00 2024-09-21T03:41:28.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.708+0000 7f97beac8640 1 -- 172.21.15.38:0/3696445974 <== osd.7 v2:172.21.15.195:6818/1606106342 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v107'323 uv323 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f97b800aa00 con 0x5605bcd19e00 2024-09-21T03:41:28.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.708+0000 7f97c5125f40 1 -- 172.21.15.38:0/3696445974 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5605bcdd9690 con 0x5605bcd19e00 2024-09-21T03:41:28.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.709+0000 7f97beac8640 1 -- 172.21.15.38:0/3696445974 <== osd.7 v2:172.21.15.195:6818/1606106342 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v107'324 uv324 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f97b800aa00 con 0x5605bcd19e00 2024-09-21T03:41:28.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.709+0000 7f97c5125f40 1 -- 172.21.15.38:0/3696445974 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5605bcdd9a30 con 0x5605bcd19e00 2024-09-21T03:41:28.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.710+0000 7f97beac8640 1 -- 172.21.15.38:0/3696445974 <== osd.7 v2:172.21.15.195:6818/1606106342 7 ==== osd_op_reply(7 602.00000000 [create] v107'325 uv299 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f97b800aa00 con 0x5605bcd19e00 2024-09-21T03:41:28.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.710+0000 7f97c5125f40 1 -- 172.21.15.38:0/3696445974 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e107) -- 0x5605bcdd9dd0 con 0x5605bcd19e00 2024-09-21T03:41:28.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.712+0000 7f97beac8640 1 -- 172.21.15.38:0/3696445974 <== osd.7 v2:172.21.15.195:6818/1606106342 8 ==== osd_op_reply(8 602.00000000 [delete] v107'326 uv326 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f97b800aa00 con 0x5605bcd19e00 2024-09-21T03:41:28.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.712+0000 7f97c5125f40 1 -- 172.21.15.38:0/3696445974 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5605bcdd9dd0 con 0x5605bcd19e00 2024-09-21T03:41:28.715 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-09-21T03:41:28.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.713+0000 7f97beac8640 1 -- 172.21.15.38:0/3696445974 <== osd.7 v2:172.21.15.195:6818/1606106342 9 ==== osd_op_reply(9 602.00000000 [create] v107'327 uv327 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f97b800aa00 con 0x5605bcd19e00 2024-09-21T03:41:28.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.713+0000 7f97c5125f40 1 -- 172.21.15.38:0/3696445974 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5605bcdd9dd0 con 0x5605bcd19e00 2024-09-21T03:41:28.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.715+0000 7f97beac8640 1 -- 172.21.15.38:0/3696445974 <== osd.7 v2:172.21.15.195:6818/1606106342 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v107'328 uv328 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f97b800aa00 con 0x5605bcd19e00 2024-09-21T03:41:28.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.715+0000 7f97c5125f40 1 -- 172.21.15.38:0/3696445974 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5605bcdd9dd0 con 0x5605bcd19e00 2024-09-21T03:41:28.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.716+0000 7f97beac8640 1 -- 172.21.15.38:0/3696445974 <== osd.7 v2:172.21.15.195:6818/1606106342 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v107'329 uv329 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f97b800aa00 con 0x5605bcd19e00 2024-09-21T03:41:28.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.716+0000 7f97c5125f40 1 -- 172.21.15.38:0/3696445974 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5605bcdd9dd0 con 0x5605bcd19e00 2024-09-21T03:41:28.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.717+0000 7f97beac8640 1 -- 172.21.15.38:0/3696445974 <== osd.7 v2:172.21.15.195:6818/1606106342 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v107'330 uv330 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f97b800aa00 con 0x5605bcd19e00 2024-09-21T03:41:28.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.717+0000 7f97c5125f40 1 -- 172.21.15.38:0/3696445974 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:13 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5605bcdd9dd0 con 0x5605bcd19e00 2024-09-21T03:41:28.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.719+0000 7f97beac8640 1 -- 172.21.15.38:0/3696445974 <== osd.7 v2:172.21.15.195:6818/1606106342 13 ==== osd_op_reply(13 10000000000.00000000 [create] v107'331 uv306 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7f97b800aa00 con 0x5605bcd19e00 2024-09-21T03:41:28.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.719+0000 7f97c5125f40 1 -- 172.21.15.38:0/3696445974 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:14 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e107) -- 0x5605bcdd9dd0 con 0x5605bcd19e00 2024-09-21T03:41:28.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.721+0000 7f97beac8640 1 -- 172.21.15.38:0/3696445974 <== osd.7 v2:172.21.15.195:6818/1606106342 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v107'332 uv332 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f97b800aa00 con 0x5605bcd19e00 2024-09-21T03:41:28.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.721+0000 7f97c5125f40 1 -- 172.21.15.38:0/3696445974 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:15 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5605bcdd9dd0 con 0x5605bcd19e00 2024-09-21T03:41:28.723 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-09-21T03:41:28.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.722+0000 7f97beac8640 1 -- 172.21.15.38:0/3696445974 <== osd.7 v2:172.21.15.195:6818/1606106342 15 ==== osd_op_reply(15 10000000000.00000000 [create] v107'333 uv333 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f97b800aa00 con 0x5605bcd19e00 2024-09-21T03:41:28.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.722+0000 7f97c5125f40 1 -- 172.21.15.38:0/3696445974 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:16 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5605bcdd9dd0 con 0x5605bcd19e00 2024-09-21T03:41:28.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.723+0000 7f97beac8640 1 -- 172.21.15.38:0/3696445974 <== osd.7 v2:172.21.15.195:6818/1606106342 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v107'334 uv334 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f97b800aa00 con 0x5605bcd19e00 2024-09-21T03:41:28.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.723+0000 7f97c5125f40 1 -- 172.21.15.38:0/3696445974 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:17 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5605bcdd9dd0 con 0x5605bcd19e00 2024-09-21T03:41:28.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.725+0000 7f97beac8640 1 -- 172.21.15.38:0/3696445974 <== osd.7 v2:172.21.15.195:6818/1606106342 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v107'335 uv335 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f97b800aa00 con 0x5605bcd19e00 2024-09-21T03:41:28.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.725+0000 7f97c5125f40 1 -- 172.21.15.38:0/3696445974 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:18 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5605bcdd9dd0 con 0x5605bcd19e00 2024-09-21T03:41:28.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.726+0000 7f97beac8640 1 -- 172.21.15.38:0/3696445974 <== osd.7 v2:172.21.15.195:6818/1606106342 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v107'336 uv336 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f97b800aa00 con 0x5605bcd19e00 2024-09-21T03:41:28.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.726+0000 7f97c5125f40 1 -- 172.21.15.38:0/3696445974 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:19 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=585b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5605bcddade0 con 0x5605bcd19e00 2024-09-21T03:41:28.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.727+0000 7f97beac8640 1 -- 172.21.15.38:0/3696445974 <== osd.7 v2:172.21.15.195:6818/1606106342 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v107'337 uv337 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f97b800aa00 con 0x5605bcd19e00 2024-09-21T03:41:28.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.727+0000 7f97c5125f40 1 -- 172.21.15.38:0/3696445974 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:20 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5605bcddade0 con 0x5605bcd19e00 2024-09-21T03:41:28.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.729+0000 7f97beac8640 1 -- 172.21.15.38:0/3696445974 <== osd.7 v2:172.21.15.195:6818/1606106342 20 ==== osd_op_reply(20 600.00000000 [create] v107'338 uv312 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f97b800aa00 con 0x5605bcd19e00 2024-09-21T03:41:28.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.729+0000 7f97c5125f40 1 -- 172.21.15.38:0/3696445974 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:21 13.10 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e107) -- 0x5605bcddade0 con 0x5605bcd19e00 2024-09-21T03:41:28.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.730+0000 7f97beac8640 1 -- 172.21.15.38:0/3696445974 <== osd.7 v2:172.21.15.195:6818/1606106342 21 ==== osd_op_reply(21 600.00000000 [delete] v107'339 uv339 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f97b800aa00 con 0x5605bcd19e00 2024-09-21T03:41:28.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.730+0000 7f97c5125f40 1 -- 172.21.15.38:0/3696445974 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:22 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5605bcddade0 con 0x5605bcd19e00 2024-09-21T03:41:28.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.731+0000 7f97beac8640 1 -- 172.21.15.38:0/3696445974 <== osd.7 v2:172.21.15.195:6818/1606106342 22 ==== osd_op_reply(22 600.00000000 [create] v107'340 uv340 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f97b800aa00 con 0x5605bcd19e00 2024-09-21T03:41:28.734 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-09-21T03:41:28.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.732+0000 7f97c5125f40 1 -- 172.21.15.38:0/3696445974 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:23 13.10 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5605bcddbdf0 con 0x5605bcd19e00 2024-09-21T03:41:28.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.733+0000 7f97beac8640 1 -- 172.21.15.38:0/3696445974 <== osd.7 v2:172.21.15.195:6818/1606106342 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v107'341 uv341 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f97b800aa00 con 0x5605bcd19e00 2024-09-21T03:41:28.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.733+0000 7f97c5125f40 1 -- 172.21.15.38:0/3696445974 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:24 13.10 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5605bcddbdf0 con 0x5605bcd19e00 2024-09-21T03:41:28.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.734+0000 7f97beac8640 1 -- 172.21.15.38:0/3696445974 <== osd.7 v2:172.21.15.195:6818/1606106342 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v107'342 uv342 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f97b800aa00 con 0x5605bcd19e00 2024-09-21T03:41:28.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.734+0000 7f97c5125f40 1 -- 172.21.15.38:0/3696445974 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:25 13.10 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5605bcddade0 con 0x5605bcd19e00 2024-09-21T03:41:28.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.735+0000 7f97beac8640 1 -- 172.21.15.38:0/3696445974 <== osd.7 v2:172.21.15.195:6818/1606106342 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v107'343 uv343 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f97b800aa00 con 0x5605bcd19e00 2024-09-21T03:41:28.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.736+0000 7f97c5125f40 1 -- 172.21.15.38:0/3696445974 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:26 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5605bcddade0 con 0x5605bcd19e00 2024-09-21T03:41:28.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.737+0000 7f97beac8640 1 -- 172.21.15.38:0/3696445974 <== osd.7 v2:172.21.15.195:6818/1606106342 26 ==== osd_op_reply(26 603.00000000 [create] v107'344 uv318 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f97b800aa00 con 0x5605bcd19e00 2024-09-21T03:41:28.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.737+0000 7f97c5125f40 1 -- 172.21.15.38:0/3696445974 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:27 13.10 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e107) -- 0x5605bcddade0 con 0x5605bcd19e00 2024-09-21T03:41:28.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.738+0000 7f97beac8640 1 -- 172.21.15.38:0/3696445974 <== osd.7 v2:172.21.15.195:6818/1606106342 27 ==== osd_op_reply(27 603.00000000 [delete] v107'345 uv345 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f97b800aa00 con 0x5605bcd19e00 2024-09-21T03:41:28.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.738+0000 7f97c5125f40 1 -- 172.21.15.38:0/3696445974 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:28 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5605bcddade0 con 0x5605bcd19e00 2024-09-21T03:41:28.741 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-09-21T03:41:28.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.740+0000 7f97beac8640 1 -- 172.21.15.38:0/3696445974 <== osd.7 v2:172.21.15.195:6818/1606106342 28 ==== osd_op_reply(28 603.00000000 [create] v107'346 uv346 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f97b800aa00 con 0x5605bcd19e00 2024-09-21T03:41:28.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.740+0000 7f97c5125f40 1 -- 172.21.15.38:0/3696445974 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:29 13.10 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5605bcddbdf0 con 0x5605bcd19e00 2024-09-21T03:41:28.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.741+0000 7f97beac8640 1 -- 172.21.15.38:0/3696445974 <== osd.7 v2:172.21.15.195:6818/1606106342 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v107'347 uv347 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f97b800aa00 con 0x5605bcd19e00 2024-09-21T03:41:28.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.741+0000 7f97c5125f40 1 -- 172.21.15.38:0/3696445974 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:30 13.10 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5605bcddbdf0 con 0x5605bcd19e00 2024-09-21T03:41:28.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.743+0000 7f97beac8640 1 -- 172.21.15.38:0/3696445974 <== osd.7 v2:172.21.15.195:6818/1606106342 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v107'348 uv348 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f97b800aa00 con 0x5605bcd19e00 2024-09-21T03:41:28.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.743+0000 7f97c5125f40 1 -- 172.21.15.38:0/3696445974 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:31 13.10 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5605bcddade0 con 0x5605bcd19e00 2024-09-21T03:41:28.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.744+0000 7f97beac8640 1 -- 172.21.15.38:0/3696445974 <== osd.7 v2:172.21.15.195:6818/1606106342 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v107'349 uv349 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f97b800aa00 con 0x5605bcd19e00 2024-09-21T03:41:28.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.744+0000 7f97c5125f40 1 --2- 172.21.15.38:0/3696445974 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x5605bcddb2f0 0x5605bcddd710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:28.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.744+0000 7f97c5125f40 1 -- 172.21.15.38:0/3696445974 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5605bcdddc50 con 0x5605bcddb2f0 2024-09-21T03:41:28.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.744+0000 7f97be2c7640 1 --2- 172.21.15.38:0/3696445974 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x5605bcddb2f0 0x5605bcddd710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:28.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.745+0000 7f97be2c7640 1 --2- 172.21.15.38:0/3696445974 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x5605bcddb2f0 0x5605bcddd710 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:28.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.746+0000 7f97be2c7640 1 -- 172.21.15.38:0/3696445974 <== osd.6 v2:172.21.15.195:6826/4055825675 1 ==== osd_op_reply(32 1.00000000.inode [create] v107'240 uv222 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f97b000b040 con 0x5605bcddb2f0 2024-09-21T03:41:28.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.746+0000 7f97c5125f40 1 -- 172.21.15.38:0/3696445974 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e107) -- 0x5605bcdddff0 con 0x5605bcddb2f0 2024-09-21T03:41:28.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.748+0000 7f97be2c7640 1 -- 172.21.15.38:0/3696445974 <== osd.6 v2:172.21.15.195:6826/4055825675 2 ==== osd_op_reply(33 1.00000000.inode [delete] v107'241 uv241 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f97b000b040 con 0x5605bcddb2f0 2024-09-21T03:41:28.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.748+0000 7f97c5125f40 1 -- 172.21.15.38:0/3696445974 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5605bcdde390 con 0x5605bcddb2f0 2024-09-21T03:41:28.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.749+0000 7f97be2c7640 1 -- 172.21.15.38:0/3696445974 <== osd.6 v2:172.21.15.195:6826/4055825675 3 ==== osd_op_reply(34 1.00000000.inode [create] v107'242 uv242 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f97b000b040 con 0x5605bcddb2f0 2024-09-21T03:41:28.751 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-09-21T03:41:28.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.749+0000 7f97c5125f40 1 -- 172.21.15.38:0/3696445974 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5605bcddebe0 con 0x5605bcddb2f0 2024-09-21T03:41:28.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.750+0000 7f97be2c7640 1 -- 172.21.15.38:0/3696445974 <== osd.6 v2:172.21.15.195:6826/4055825675 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v107'243 uv243 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f97b000b040 con 0x5605bcddb2f0 2024-09-21T03:41:28.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.750+0000 7f97c5125f40 1 -- 172.21.15.38:0/3696445974 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5605bcddef80 con 0x5605bcddb2f0 2024-09-21T03:41:28.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.751+0000 7f97be2c7640 1 -- 172.21.15.38:0/3696445974 <== osd.6 v2:172.21.15.195:6826/4055825675 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v107'244 uv244 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f97b000b040 con 0x5605bcddb2f0 2024-09-21T03:41:28.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.751+0000 7f97c5125f40 1 -- 172.21.15.38:0/3696445974 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5605bcddf320 con 0x5605bcddb2f0 2024-09-21T03:41:28.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.753+0000 7f97be2c7640 1 -- 172.21.15.38:0/3696445974 <== osd.6 v2:172.21.15.195:6826/4055825675 6 ==== osd_op_reply(37 200.00000000 [create] v107'245 uv227 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f97b000b040 con 0x5605bcddb2f0 2024-09-21T03:41:28.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.753+0000 7f97c5125f40 1 -- 172.21.15.38:0/3696445974 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e107) -- 0x5605bcddf6c0 con 0x5605bcddb2f0 2024-09-21T03:41:28.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.754+0000 7f97be2c7640 1 -- 172.21.15.38:0/3696445974 <== osd.6 v2:172.21.15.195:6826/4055825675 7 ==== osd_op_reply(38 200.00000000 [delete] v107'246 uv246 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f97b000b040 con 0x5605bcddb2f0 2024-09-21T03:41:28.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.754+0000 7f97c5125f40 1 -- 172.21.15.38:0/3696445974 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5605bcddfa60 con 0x5605bcddb2f0 2024-09-21T03:41:28.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.755+0000 7f97be2c7640 1 -- 172.21.15.38:0/3696445974 <== osd.6 v2:172.21.15.195:6826/4055825675 8 ==== osd_op_reply(39 200.00000000 [create] v107'247 uv247 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f97b000b040 con 0x5605bcddb2f0 2024-09-21T03:41:28.757 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-09-21T03:41:28.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.755+0000 7f97c5125f40 1 -- 172.21.15.38:0/3696445974 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5605bcddfa60 con 0x5605bcddb2f0 2024-09-21T03:41:28.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.756+0000 7f97be2c7640 1 -- 172.21.15.38:0/3696445974 <== osd.6 v2:172.21.15.195:6826/4055825675 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v107'248 uv248 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f97b000b040 con 0x5605bcddb2f0 2024-09-21T03:41:28.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.756+0000 7f97c5125f40 1 -- 172.21.15.38:0/3696445974 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5605bcddfa60 con 0x5605bcddb2f0 2024-09-21T03:41:28.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.758+0000 7f97be2c7640 1 -- 172.21.15.38:0/3696445974 <== osd.6 v2:172.21.15.195:6826/4055825675 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v107'249 uv249 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f97b000b040 con 0x5605bcddb2f0 2024-09-21T03:41:28.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.758+0000 7f97c5125f40 1 -- 172.21.15.38:0/3696445974 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:42 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5605bcddfa60 con 0x5605bcddb2f0 2024-09-21T03:41:28.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.759+0000 7f97be2c7640 1 -- 172.21.15.38:0/3696445974 <== osd.6 v2:172.21.15.195:6826/4055825675 11 ==== osd_op_reply(42 200.00000001 [create] v107'250 uv232 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f97b000b040 con 0x5605bcddb2f0 2024-09-21T03:41:28.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.759+0000 7f97c5125f40 1 -- 172.21.15.38:0/3696445974 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:43 13.14 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e107) -- 0x5605bcddfa60 con 0x5605bcddb2f0 2024-09-21T03:41:28.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.760+0000 7f97be2c7640 1 -- 172.21.15.38:0/3696445974 <== osd.6 v2:172.21.15.195:6826/4055825675 12 ==== osd_op_reply(43 200.00000001 [delete] v107'251 uv251 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f97b000b040 con 0x5605bcddb2f0 2024-09-21T03:41:28.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.760+0000 7f97c5125f40 1 -- 172.21.15.38:0/3696445974 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:44 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5605bcddfa60 con 0x5605bcddb2f0 2024-09-21T03:41:28.763 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-09-21T03:41:28.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.761+0000 7f97be2c7640 1 -- 172.21.15.38:0/3696445974 <== osd.6 v2:172.21.15.195:6826/4055825675 13 ==== osd_op_reply(44 200.00000001 [create] v107'252 uv252 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f97b000b040 con 0x5605bcddb2f0 2024-09-21T03:41:28.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.761+0000 7f97c5125f40 1 -- 172.21.15.38:0/3696445974 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:45 13.14 13:2e2fa760:::200.00000001:head [write 0~17490 in=17490b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5605bcde3ee0 con 0x5605bcddb2f0 2024-09-21T03:41:28.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.763+0000 7f97be2c7640 1 -- 172.21.15.38:0/3696445974 <== osd.6 v2:172.21.15.195:6826/4055825675 14 ==== osd_op_reply(45 200.00000001 [write 0~17490] v107'253 uv253 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f97b000b040 con 0x5605bcddb2f0 2024-09-21T03:41:28.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.763+0000 7f97c5125f40 1 -- 172.21.15.38:0/3696445974 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:46 13.14 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5605bcde3ee0 con 0x5605bcddb2f0 2024-09-21T03:41:28.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.764+0000 7f97be2c7640 1 -- 172.21.15.38:0/3696445974 <== osd.6 v2:172.21.15.195:6826/4055825675 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v107'254 uv254 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f97b000b040 con 0x5605bcddb2f0 2024-09-21T03:41:28.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.764+0000 7f97c5125f40 1 -- 172.21.15.38:0/3696445974 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:47 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5605bcddfa60 con 0x5605bcddb2f0 2024-09-21T03:41:28.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.765+0000 7f97be2c7640 1 -- 172.21.15.38:0/3696445974 <== osd.6 v2:172.21.15.195:6826/4055825675 16 ==== osd_op_reply(47 606.00000000 [create] v107'255 uv238 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f97b000b040 con 0x5605bcddb2f0 2024-09-21T03:41:28.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.765+0000 7f97c5125f40 1 -- 172.21.15.38:0/3696445974 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:48 13.14 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e107) -- 0x5605bcddfa60 con 0x5605bcddb2f0 2024-09-21T03:41:28.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.766+0000 7f97be2c7640 1 -- 172.21.15.38:0/3696445974 <== osd.6 v2:172.21.15.195:6826/4055825675 17 ==== osd_op_reply(48 606.00000000 [delete] v107'256 uv256 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f97b000b040 con 0x5605bcddb2f0 2024-09-21T03:41:28.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.766+0000 7f97c5125f40 1 -- 172.21.15.38:0/3696445974 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:49 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5605bcddfa60 con 0x5605bcddb2f0 2024-09-21T03:41:28.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.767+0000 7f97be2c7640 1 -- 172.21.15.38:0/3696445974 <== osd.6 v2:172.21.15.195:6826/4055825675 18 ==== osd_op_reply(49 606.00000000 [create] v107'257 uv257 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f97b000b040 con 0x5605bcddb2f0 2024-09-21T03:41:28.769 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-09-21T03:41:28.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.767+0000 7f97c5125f40 1 -- 172.21.15.38:0/3696445974 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:50 13.14 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5605bcde0a70 con 0x5605bcddb2f0 2024-09-21T03:41:28.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.768+0000 7f97be2c7640 1 -- 172.21.15.38:0/3696445974 <== osd.6 v2:172.21.15.195:6826/4055825675 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v107'258 uv258 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f97b000b040 con 0x5605bcddb2f0 2024-09-21T03:41:28.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.768+0000 7f97c5125f40 1 -- 172.21.15.38:0/3696445974 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:51 13.14 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5605bcde0a70 con 0x5605bcddb2f0 2024-09-21T03:41:28.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.769+0000 7f97be2c7640 1 -- 172.21.15.38:0/3696445974 <== osd.6 v2:172.21.15.195:6826/4055825675 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v107'259 uv259 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f97b000b040 con 0x5605bcddb2f0 2024-09-21T03:41:28.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.769+0000 7f97c5125f40 1 -- 172.21.15.38:0/3696445974 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:52 13.14 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5605bcddfa60 con 0x5605bcddb2f0 2024-09-21T03:41:28.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.770+0000 7f97be2c7640 1 -- 172.21.15.38:0/3696445974 <== osd.6 v2:172.21.15.195:6826/4055825675 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v107'260 uv260 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f97b000b040 con 0x5605bcddb2f0 2024-09-21T03:41:28.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.771+0000 7f97c5125f40 1 --2- 172.21.15.38:0/3696445974 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x5605bcddff70 0x5605bcde2390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:28.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.771+0000 7f97c5125f40 1 -- 172.21.15.38:0/3696445974 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:53 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5605bcde28d0 con 0x5605bcddff70 2024-09-21T03:41:28.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.771+0000 7f97bdac6640 1 --2- 172.21.15.38:0/3696445974 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x5605bcddff70 0x5605bcde2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:28.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.771+0000 7f97bdac6640 1 --2- 172.21.15.38:0/3696445974 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x5605bcddff70 0x5605bcde2390 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:28.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.772+0000 7f97bdac6640 1 -- 172.21.15.38:0/3696445974 <== osd.0 v2:172.21.15.38:6800/2381639433 1 ==== osd_op_reply(53 607.00000000 [create] v107'63 uv62 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f97b405d850 con 0x5605bcddff70 2024-09-21T03:41:28.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.772+0000 7f97c5125f40 1 -- 172.21.15.38:0/3696445974 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:54 13.1c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e107) -- 0x5605bcde2c70 con 0x5605bcddff70 2024-09-21T03:41:28.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.773+0000 7f97bdac6640 1 -- 172.21.15.38:0/3696445974 <== osd.0 v2:172.21.15.38:6800/2381639433 2 ==== osd_op_reply(54 607.00000000 [delete] v107'64 uv64 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f97b405d850 con 0x5605bcddff70 2024-09-21T03:41:28.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.773+0000 7f97c5125f40 1 -- 172.21.15.38:0/3696445974 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:55 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5605bcde3010 con 0x5605bcddff70 2024-09-21T03:41:28.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.774+0000 7f97bdac6640 1 -- 172.21.15.38:0/3696445974 <== osd.0 v2:172.21.15.38:6800/2381639433 3 ==== osd_op_reply(55 607.00000000 [create] v107'65 uv65 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f97b405d850 con 0x5605bcddff70 2024-09-21T03:41:28.776 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-09-21T03:41:28.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.774+0000 7f97c5125f40 1 -- 172.21.15.38:0/3696445974 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:56 13.1c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5605bcde43c0 con 0x5605bcddff70 2024-09-21T03:41:28.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.775+0000 7f97bdac6640 1 -- 172.21.15.38:0/3696445974 <== osd.0 v2:172.21.15.38:6800/2381639433 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v107'66 uv66 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f97b405d850 con 0x5605bcddff70 2024-09-21T03:41:28.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.775+0000 7f97c5125f40 1 -- 172.21.15.38:0/3696445974 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:57 13.1c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5605bcde4820 con 0x5605bcddff70 2024-09-21T03:41:28.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.776+0000 7f97bdac6640 1 -- 172.21.15.38:0/3696445974 <== osd.0 v2:172.21.15.38:6800/2381639433 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v107'67 uv67 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f97b405d850 con 0x5605bcddff70 2024-09-21T03:41:28.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.776+0000 7f97c5125f40 1 -- 172.21.15.38:0/3696445974 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:58 13.1c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5605bcde33b0 con 0x5605bcddff70 2024-09-21T03:41:28.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.777+0000 7f97bdac6640 1 -- 172.21.15.38:0/3696445974 <== osd.0 v2:172.21.15.38:6800/2381639433 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v107'68 uv68 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f97b405d850 con 0x5605bcddff70 2024-09-21T03:41:28.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.777+0000 7f97c5125f40 1 --2- 172.21.15.38:0/3696445974 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x5605bcde3e70 0x5605bcde6c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:28.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.777+0000 7f97c5125f40 1 -- 172.21.15.38:0/3696445974 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:59 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5605bcde7200 con 0x5605bcde3e70 2024-09-21T03:41:28.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.777+0000 7f97beac8640 1 --2- 172.21.15.38:0/3696445974 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x5605bcde3e70 0x5605bcde6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:28.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.778+0000 7f97beac8640 1 --2- 172.21.15.38:0/3696445974 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x5605bcde3e70 0x5605bcde6c60 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:28.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.779+0000 7f97beac8640 1 -- 172.21.15.38:0/3696445974 <== osd.3 v2:172.21.15.38:6808/1286810294 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v107'48 uv47 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f97b800aa00 con 0x5605bcde3e70 2024-09-21T03:41:28.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.779+0000 7f97c5125f40 1 -- 172.21.15.38:0/3696445974 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:60 13.6 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e107) -- 0x5605bcde75f0 con 0x5605bcde3e70 2024-09-21T03:41:28.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.780+0000 7f97beac8640 1 -- 172.21.15.38:0/3696445974 <== osd.3 v2:172.21.15.38:6808/1286810294 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v107'49 uv49 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f97b800aa00 con 0x5605bcde3e70 2024-09-21T03:41:28.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.780+0000 7f97c5125f40 1 -- 172.21.15.38:0/3696445974 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:61 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5605bcde75f0 con 0x5605bcde3e70 2024-09-21T03:41:28.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.781+0000 7f97beac8640 1 -- 172.21.15.38:0/3696445974 <== osd.3 v2:172.21.15.38:6808/1286810294 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v107'50 uv50 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f97b800aa00 con 0x5605bcde3e70 2024-09-21T03:41:28.783 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-09-21T03:41:28.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.781+0000 7f97c5125f40 1 -- 172.21.15.38:0/3696445974 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:62 13.6 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5605bcde7610 con 0x5605bcde3e70 2024-09-21T03:41:28.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.782+0000 7f97beac8640 1 -- 172.21.15.38:0/3696445974 <== osd.3 v2:172.21.15.38:6808/1286810294 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v107'51 uv51 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f97b800aa00 con 0x5605bcde3e70 2024-09-21T03:41:28.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.782+0000 7f97c5125f40 1 -- 172.21.15.38:0/3696445974 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5605bcde7610 con 0x5605bcddff70 2024-09-21T03:41:28.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.784+0000 7f97bdac6640 1 -- 172.21.15.38:0/3696445974 <== osd.0 v2:172.21.15.38:6800/2381639433 7 ==== osd_op_reply(63 608.00000000 [create] v107'63 uv62 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f97b405d850 con 0x5605bcddff70 2024-09-21T03:41:28.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.784+0000 7f97c5125f40 1 -- 172.21.15.38:0/3696445974 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e107) -- 0x5605bcde7610 con 0x5605bcddff70 2024-09-21T03:41:28.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.785+0000 7f97bdac6640 1 -- 172.21.15.38:0/3696445974 <== osd.0 v2:172.21.15.38:6800/2381639433 8 ==== osd_op_reply(64 608.00000000 [delete] v107'64 uv64 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f97b405d850 con 0x5605bcddff70 2024-09-21T03:41:28.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.785+0000 7f97c5125f40 1 -- 172.21.15.38:0/3696445974 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5605bcde75f0 con 0x5605bcddff70 2024-09-21T03:41:28.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.786+0000 7f97bdac6640 1 -- 172.21.15.38:0/3696445974 <== osd.0 v2:172.21.15.38:6800/2381639433 9 ==== osd_op_reply(65 608.00000000 [create] v107'65 uv65 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f97b405d850 con 0x5605bcddff70 2024-09-21T03:41:28.788 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-09-21T03:41:28.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.786+0000 7f97c5125f40 1 -- 172.21.15.38:0/3696445974 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5605bcde7620 con 0x5605bcddff70 2024-09-21T03:41:28.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.787+0000 7f97bdac6640 1 -- 172.21.15.38:0/3696445974 <== osd.0 v2:172.21.15.38:6800/2381639433 10 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v107'66 uv66 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f97b405d850 con 0x5605bcddff70 2024-09-21T03:41:28.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.787+0000 7f97c5125f40 1 -- 172.21.15.38:0/3696445974 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5605bcde75e0 con 0x5605bcddff70 2024-09-21T03:41:28.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.788+0000 7f97bdac6640 1 -- 172.21.15.38:0/3696445974 <== osd.0 v2:172.21.15.38:6800/2381639433 11 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v107'67 uv67 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f97b405d850 con 0x5605bcddff70 2024-09-21T03:41:28.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.788+0000 7f97c5125f40 1 -- 172.21.15.38:0/3696445974 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5605bcde7a70 con 0x5605bcddff70 2024-09-21T03:41:28.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.790+0000 7f97bdac6640 1 -- 172.21.15.38:0/3696445974 <== osd.0 v2:172.21.15.38:6800/2381639433 12 ==== osd_op_reply(68 608.00000000 [omap-set-header] v107'68 uv68 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f97b405d850 con 0x5605bcddff70 2024-09-21T03:41:28.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.790+0000 7f97c5125f40 1 -- 172.21.15.38:0/3696445974 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5605bcde75e0 con 0x5605bcde3e70 2024-09-21T03:41:28.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.791+0000 7f97beac8640 1 -- 172.21.15.38:0/3696445974 <== osd.3 v2:172.21.15.38:6808/1286810294 5 ==== osd_op_reply(69 604.00000000 [create] v107'116 uv110 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f97b800aa00 con 0x5605bcde3e70 2024-09-21T03:41:28.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.791+0000 7f97c5125f40 1 -- 172.21.15.38:0/3696445974 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e107) -- 0x5605bcde75e0 con 0x5605bcde3e70 2024-09-21T03:41:28.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.792+0000 7f97beac8640 1 -- 172.21.15.38:0/3696445974 <== osd.3 v2:172.21.15.38:6808/1286810294 6 ==== osd_op_reply(70 604.00000000 [delete] v107'117 uv117 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f97b800aa00 con 0x5605bcde3e70 2024-09-21T03:41:28.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.792+0000 7f97c5125f40 1 -- 172.21.15.38:0/3696445974 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5605bcde75e0 con 0x5605bcde3e70 2024-09-21T03:41:28.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.793+0000 7f97beac8640 1 -- 172.21.15.38:0/3696445974 <== osd.3 v2:172.21.15.38:6808/1286810294 7 ==== osd_op_reply(71 604.00000000 [create] v107'118 uv118 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f97b800aa00 con 0x5605bcde3e70 2024-09-21T03:41:28.795 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-09-21T03:41:28.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.793+0000 7f97c5125f40 1 -- 172.21.15.38:0/3696445974 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5605bcde75e0 con 0x5605bcde3e70 2024-09-21T03:41:28.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.794+0000 7f97beac8640 1 -- 172.21.15.38:0/3696445974 <== osd.3 v2:172.21.15.38:6808/1286810294 8 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v107'119 uv119 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f97b800aa00 con 0x5605bcde3e70 2024-09-21T03:41:28.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.794+0000 7f97c5125f40 1 -- 172.21.15.38:0/3696445974 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5605bcde75e0 con 0x5605bcde3e70 2024-09-21T03:41:28.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.795+0000 7f97beac8640 1 -- 172.21.15.38:0/3696445974 <== osd.3 v2:172.21.15.38:6808/1286810294 9 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v107'120 uv120 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f97b800aa00 con 0x5605bcde3e70 2024-09-21T03:41:28.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.795+0000 7f97c5125f40 1 -- 172.21.15.38:0/3696445974 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5605bcde75e0 con 0x5605bcde3e70 2024-09-21T03:41:28.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.796+0000 7f97beac8640 1 -- 172.21.15.38:0/3696445974 <== osd.3 v2:172.21.15.38:6808/1286810294 10 ==== osd_op_reply(74 604.00000000 [omap-set-header] v107'121 uv121 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f97b800aa00 con 0x5605bcde3e70 2024-09-21T03:41:28.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.796+0000 7f97c5125f40 1 -- 172.21.15.38:0/3696445974 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5605bcde75e0 con 0x5605bcde3e70 2024-09-21T03:41:28.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.797+0000 7f97beac8640 1 -- 172.21.15.38:0/3696445974 <== osd.3 v2:172.21.15.38:6808/1286810294 11 ==== osd_op_reply(75 500.00000000 [create] v107'122 uv115 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f97b800aa00 con 0x5605bcde3e70 2024-09-21T03:41:28.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.797+0000 7f97c5125f40 1 -- 172.21.15.38:0/3696445974 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e107) -- 0x5605bcde75e0 con 0x5605bcde3e70 2024-09-21T03:41:28.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.798+0000 7f97beac8640 1 -- 172.21.15.38:0/3696445974 <== osd.3 v2:172.21.15.38:6808/1286810294 12 ==== osd_op_reply(76 500.00000000 [delete] v107'123 uv123 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f97b800aa00 con 0x5605bcde3e70 2024-09-21T03:41:28.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.798+0000 7f97c5125f40 1 -- 172.21.15.38:0/3696445974 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5605bcde75e0 con 0x5605bcde3e70 2024-09-21T03:41:28.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.799+0000 7f97beac8640 1 -- 172.21.15.38:0/3696445974 <== osd.3 v2:172.21.15.38:6808/1286810294 13 ==== osd_op_reply(77 500.00000000 [create] v107'124 uv124 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f97b800aa00 con 0x5605bcde3e70 2024-09-21T03:41:28.802 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-09-21T03:41:28.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.799+0000 7f97c5125f40 1 -- 172.21.15.38:0/3696445974 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5605bcde75e0 con 0x5605bcde3e70 2024-09-21T03:41:28.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.801+0000 7f97beac8640 1 -- 172.21.15.38:0/3696445974 <== osd.3 v2:172.21.15.38:6808/1286810294 14 ==== osd_op_reply(78 500.00000000 [write 0~90] v107'125 uv125 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f97b800aa00 con 0x5605bcde3e70 2024-09-21T03:41:28.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.801+0000 7f97c5125f40 1 -- 172.21.15.38:0/3696445974 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5605bcde7a70 con 0x5605bcde3e70 2024-09-21T03:41:28.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.802+0000 7f97beac8640 1 -- 172.21.15.38:0/3696445974 <== osd.3 v2:172.21.15.38:6808/1286810294 15 ==== osd_op_reply(79 500.00000000 [omap-set-header] v107'126 uv126 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f97b800aa00 con 0x5605bcde3e70 2024-09-21T03:41:28.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.802+0000 7f97c5125f40 1 -- 172.21.15.38:0/3696445974 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:80 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5605bcde75e0 con 0x5605bcddff70 2024-09-21T03:41:28.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.803+0000 7f97bdac6640 1 -- 172.21.15.38:0/3696445974 <== osd.0 v2:172.21.15.38:6800/2381639433 13 ==== osd_op_reply(80 mds_snaptable [create] v107'51 uv50 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f97b405d850 con 0x5605bcddff70 2024-09-21T03:41:28.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.803+0000 7f97c5125f40 1 -- 172.21.15.38:0/3696445974 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:81 13.d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e107) -- 0x5605bcde7a70 con 0x5605bcddff70 2024-09-21T03:41:28.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.804+0000 7f97bdac6640 1 -- 172.21.15.38:0/3696445974 <== osd.0 v2:172.21.15.38:6800/2381639433 14 ==== osd_op_reply(81 mds_snaptable [delete] v107'52 uv52 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f97b405d850 con 0x5605bcddff70 2024-09-21T03:41:28.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.804+0000 7f97c5125f40 1 -- 172.21.15.38:0/3696445974 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:82 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5605bcde75e0 con 0x5605bcddff70 2024-09-21T03:41:28.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.805+0000 7f97bdac6640 1 -- 172.21.15.38:0/3696445974 <== osd.0 v2:172.21.15.38:6800/2381639433 15 ==== osd_op_reply(82 mds_snaptable [create] v107'53 uv53 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f97b405d850 con 0x5605bcddff70 2024-09-21T03:41:28.807 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-09-21T03:41:28.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.805+0000 7f97c5125f40 1 -- 172.21.15.38:0/3696445974 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:83 13.d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5605bcde7a70 con 0x5605bcddff70 2024-09-21T03:41:28.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.806+0000 7f97bdac6640 1 -- 172.21.15.38:0/3696445974 <== osd.0 v2:172.21.15.38:6800/2381639433 16 ==== osd_op_reply(83 mds_snaptable [write 0~70] v107'54 uv54 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f97b405d850 con 0x5605bcddff70 2024-09-21T03:41:28.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.807+0000 7f97c5125f40 1 -- 172.21.15.38:0/3696445974 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:84 13.d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5605bcde7a70 con 0x5605bcddff70 2024-09-21T03:41:28.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.808+0000 7f97bdac6640 1 -- 172.21.15.38:0/3696445974 <== osd.0 v2:172.21.15.38:6800/2381639433 17 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v107'55 uv55 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f97b405d850 con 0x5605bcddff70 2024-09-21T03:41:28.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.808+0000 7f97c5125f40 1 --2- 172.21.15.38:0/3696445974 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x5605bcde83e0 0x5605bcdea800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:28.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.808+0000 7f97c5125f40 1 -- 172.21.15.38:0/3696445974 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5605bcde7a70 con 0x5605bcde83e0 2024-09-21T03:41:28.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.808+0000 7f97be2c7640 1 --2- 172.21.15.38:0/3696445974 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x5605bcde83e0 0x5605bcdea800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:28.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.808+0000 7f97be2c7640 1 --2- 172.21.15.38:0/3696445974 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x5605bcde83e0 0x5605bcdea800 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:28.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.809+0000 7f97be2c7640 1 -- 172.21.15.38:0/3696445974 <== osd.1 v2:172.21.15.38:6816/1491818732 1 ==== osd_op_reply(85 605.00000000 [create] v107'63 uv62 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f97b000b040 con 0x5605bcde83e0 2024-09-21T03:41:28.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.809+0000 7f97c5125f40 1 -- 172.21.15.38:0/3696445974 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e107) -- 0x5605bcdead40 con 0x5605bcde83e0 2024-09-21T03:41:28.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.810+0000 7f97be2c7640 1 -- 172.21.15.38:0/3696445974 <== osd.1 v2:172.21.15.38:6816/1491818732 2 ==== osd_op_reply(86 605.00000000 [delete] v107'64 uv64 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f97b000b040 con 0x5605bcde83e0 2024-09-21T03:41:28.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.810+0000 7f97c5125f40 1 -- 172.21.15.38:0/3696445974 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5605bcdead40 con 0x5605bcde83e0 2024-09-21T03:41:28.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.811+0000 7f97be2c7640 1 -- 172.21.15.38:0/3696445974 <== osd.1 v2:172.21.15.38:6816/1491818732 3 ==== osd_op_reply(87 605.00000000 [create] v107'65 uv65 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f97b000b040 con 0x5605bcde83e0 2024-09-21T03:41:28.814 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-09-21T03:41:28.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.811+0000 7f97c5125f40 1 -- 172.21.15.38:0/3696445974 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5605bcdebd50 con 0x5605bcde83e0 2024-09-21T03:41:28.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.812+0000 7f97be2c7640 1 -- 172.21.15.38:0/3696445974 <== osd.1 v2:172.21.15.38:6816/1491818732 4 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v107'66 uv66 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f97b000b040 con 0x5605bcde83e0 2024-09-21T03:41:28.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.812+0000 7f97c5125f40 1 -- 172.21.15.38:0/3696445974 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5605bcdebd50 con 0x5605bcde83e0 2024-09-21T03:41:28.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.814+0000 7f97be2c7640 1 -- 172.21.15.38:0/3696445974 <== osd.1 v2:172.21.15.38:6816/1491818732 5 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v107'67 uv67 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f97b000b040 con 0x5605bcde83e0 2024-09-21T03:41:28.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.814+0000 7f97c5125f40 1 -- 172.21.15.38:0/3696445974 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5605bcdead40 con 0x5605bcde83e0 2024-09-21T03:41:28.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.815+0000 7f97be2c7640 1 -- 172.21.15.38:0/3696445974 <== osd.1 v2:172.21.15.38:6816/1491818732 6 ==== osd_op_reply(90 605.00000000 [omap-set-header] v107'68 uv68 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f97b000b040 con 0x5605bcde83e0 2024-09-21T03:41:28.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.815+0000 7f97c5125f40 1 -- 172.21.15.38:0/3696445974 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5605bcdead40 con 0x5605bcde83e0 2024-09-21T03:41:28.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.817+0000 7f97be2c7640 1 -- 172.21.15.38:0/3696445974 <== osd.1 v2:172.21.15.38:6816/1491818732 7 ==== osd_op_reply(91 mds0_inotable [create] v107'104 uv97 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f97b000b040 con 0x5605bcde83e0 2024-09-21T03:41:28.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.817+0000 7f97c5125f40 1 -- 172.21.15.38:0/3696445974 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e107) -- 0x5605bcdead40 con 0x5605bcde83e0 2024-09-21T03:41:28.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.818+0000 7f97be2c7640 1 -- 172.21.15.38:0/3696445974 <== osd.1 v2:172.21.15.38:6816/1491818732 8 ==== osd_op_reply(92 mds0_inotable [delete] v107'105 uv105 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f97b000b040 con 0x5605bcde83e0 2024-09-21T03:41:28.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.818+0000 7f97c5125f40 1 -- 172.21.15.38:0/3696445974 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5605bcdead40 con 0x5605bcde83e0 2024-09-21T03:41:28.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.819+0000 7f97be2c7640 1 -- 172.21.15.38:0/3696445974 <== osd.1 v2:172.21.15.38:6816/1491818732 9 ==== osd_op_reply(93 mds0_inotable [create] v107'106 uv106 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f97b000b040 con 0x5605bcde83e0 2024-09-21T03:41:28.821 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-09-21T03:41:28.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.819+0000 7f97c5125f40 1 -- 172.21.15.38:0/3696445974 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:94 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5605bcdde7d0 con 0x5605bcde83e0 2024-09-21T03:41:28.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.822+0000 7f97be2c7640 1 -- 172.21.15.38:0/3696445974 <== osd.1 v2:172.21.15.38:6816/1491818732 10 ==== osd_op_reply(94 mds0_inotable [write 0~34] v107'107 uv107 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f97b000b040 con 0x5605bcde83e0 2024-09-21T03:41:28.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.822+0000 7f97c5125f40 1 -- 172.21.15.38:0/3696445974 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:95 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5605bcdde7d0 con 0x5605bcde83e0 2024-09-21T03:41:28.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.823+0000 7f97be2c7640 1 -- 172.21.15.38:0/3696445974 <== osd.1 v2:172.21.15.38:6816/1491818732 11 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v107'108 uv108 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f97b000b040 con 0x5605bcde83e0 2024-09-21T03:41:28.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.823+0000 7f97c5125f40 1 -- 172.21.15.38:0/3696445974 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:96 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5605bcdde7d0 con 0x5605bcde83e0 2024-09-21T03:41:28.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.824+0000 7f97be2c7640 1 -- 172.21.15.38:0/3696445974 <== osd.1 v2:172.21.15.38:6816/1491818732 12 ==== osd_op_reply(96 100.00000000 [create] v107'109 uv109 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f97b000b040 con 0x5605bcde83e0 2024-09-21T03:41:28.826 INFO:teuthology.orchestra.run.smithi038.stdout:Write #-9223372036854775808:00000000:::100.00000000:head# 2024-09-21T03:41:28.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.824+0000 7f97c5125f40 1 -- 172.21.15.38:0/3696445974 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:97 13.13 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5605bcdde7d0 con 0x5605bcde83e0 2024-09-21T03:41:28.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.826+0000 7f97be2c7640 1 -- 172.21.15.38:0/3696445974 <== osd.1 v2:172.21.15.38:6816/1491818732 13 ==== osd_op_reply(97 100.00000000 [omap-set-header] v107'110 uv110 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f97b000b040 con 0x5605bcde83e0 2024-09-21T03:41:28.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.826+0000 7f97c5125f40 1 -- 172.21.15.38:0/3696445974 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:98 13.13 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5605bcdedb50 con 0x5605bcde83e0 2024-09-21T03:41:28.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.827+0000 7f97be2c7640 1 -- 172.21.15.38:0/3696445974 <== osd.1 v2:172.21.15.38:6816/1491818732 14 ==== osd_op_reply(98 100.00000000 [omap-set-vals] v107'111 uv111 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f97b000b040 con 0x5605bcde83e0 2024-09-21T03:41:28.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.827+0000 7f97c5125f40 1 -- 172.21.15.38:0/3696445974 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:99 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5605bcdeb1f0 con 0x5605bcde3e70 2024-09-21T03:41:28.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.829+0000 7f97beac8640 1 -- 172.21.15.38:0/3696445974 <== osd.3 v2:172.21.15.38:6808/1286810294 16 ==== osd_op_reply(99 mds0_sessionmap [create] v107'49 uv47 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7f97b800aa00 con 0x5605bcde3e70 2024-09-21T03:41:28.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.829+0000 7f97c5125f40 1 -- 172.21.15.38:0/3696445974 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:100 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e107) -- 0x5605bcdeb1f0 con 0x5605bcde3e70 2024-09-21T03:41:28.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.830+0000 7f97beac8640 1 -- 172.21.15.38:0/3696445974 <== osd.3 v2:172.21.15.38:6808/1286810294 17 ==== osd_op_reply(100 mds0_sessionmap [delete] v107'50 uv50 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f97b800aa00 con 0x5605bcde3e70 2024-09-21T03:41:28.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.830+0000 7f97c5125f40 1 -- 172.21.15.38:0/3696445974 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5605bcdeb1f0 con 0x5605bcde3e70 2024-09-21T03:41:28.833 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-09-21T03:41:28.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.831+0000 7f97beac8640 1 -- 172.21.15.38:0/3696445974 <== osd.3 v2:172.21.15.38:6808/1286810294 18 ==== osd_op_reply(101 mds0_sessionmap [create] v107'51 uv51 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f97b800aa00 con 0x5605bcde3e70 2024-09-21T03:41:28.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.831+0000 7f97c5125f40 1 -- 172.21.15.38:0/3696445974 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5605bcdeb1f0 con 0x5605bcde3e70 2024-09-21T03:41:28.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.832+0000 7f97beac8640 1 -- 172.21.15.38:0/3696445974 <== osd.3 v2:172.21.15.38:6808/1286810294 19 ==== osd_op_reply(102 mds0_sessionmap [omap-set-header] v107'52 uv52 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f97b800aa00 con 0x5605bcde3e70 2024-09-21T03:41:28.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.832+0000 7f97c5125f40 1 -- 172.21.15.38:0/3696445974 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:103 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5605bcdeb1f0 con 0x5605bcde3e70 2024-09-21T03:41:28.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.833+0000 7f97beac8640 1 -- 172.21.15.38:0/3696445974 <== osd.3 v2:172.21.15.38:6808/1286810294 20 ==== osd_op_reply(103 609.00000000 [create] v107'63 uv62 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f97b800aa00 con 0x5605bcde3e70 2024-09-21T03:41:28.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.833+0000 7f97c5125f40 1 -- 172.21.15.38:0/3696445974 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:104 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e107) -- 0x5605bcdeb1f0 con 0x5605bcde3e70 2024-09-21T03:41:28.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.834+0000 7f97beac8640 1 -- 172.21.15.38:0/3696445974 <== osd.3 v2:172.21.15.38:6808/1286810294 21 ==== osd_op_reply(104 609.00000000 [delete] v107'64 uv64 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f97b800aa00 con 0x5605bcde3e70 2024-09-21T03:41:28.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.834+0000 7f97c5125f40 1 -- 172.21.15.38:0/3696445974 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5605bcdeb1f0 con 0x5605bcde3e70 2024-09-21T03:41:28.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.835+0000 7f97beac8640 1 -- 172.21.15.38:0/3696445974 <== osd.3 v2:172.21.15.38:6808/1286810294 22 ==== osd_op_reply(105 609.00000000 [create] v107'65 uv65 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f97b800aa00 con 0x5605bcde3e70 2024-09-21T03:41:28.837 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-09-21T03:41:28.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.835+0000 7f97c5125f40 1 -- 172.21.15.38:0/3696445974 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5605bcdeb1f0 con 0x5605bcde3e70 2024-09-21T03:41:28.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.836+0000 7f97beac8640 1 -- 172.21.15.38:0/3696445974 <== osd.3 v2:172.21.15.38:6808/1286810294 23 ==== osd_op_reply(106 609.00000000 [setxattr (30)] v107'66 uv66 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f97b800aa00 con 0x5605bcde3e70 2024-09-21T03:41:28.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.836+0000 7f97c5125f40 1 -- 172.21.15.38:0/3696445974 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5605bcdeb1f0 con 0x5605bcde3e70 2024-09-21T03:41:28.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.837+0000 7f97beac8640 1 -- 172.21.15.38:0/3696445974 <== osd.3 v2:172.21.15.38:6808/1286810294 24 ==== osd_op_reply(107 609.00000000 [setxattr (62)] v107'67 uv67 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f97b800aa00 con 0x5605bcde3e70 2024-09-21T03:41:28.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.837+0000 7f97c5125f40 1 -- 172.21.15.38:0/3696445974 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5605bcdeb1f0 con 0x5605bcde3e70 2024-09-21T03:41:28.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.839+0000 7f97beac8640 1 -- 172.21.15.38:0/3696445974 <== osd.3 v2:172.21.15.38:6808/1286810294 25 ==== osd_op_reply(108 609.00000000 [omap-set-header] v107'68 uv68 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f97b800aa00 con 0x5605bcde3e70 2024-09-21T03:41:28.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.839+0000 7f97c5125f40 1 --2- 172.21.15.38:0/3696445974 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x5605bcdeb700 0x5605bcdedb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:28.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.839+0000 7f97c5125f40 1 -- 172.21.15.38:0/3696445974 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:109 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5605bcdee210 con 0x5605bcdeb700 2024-09-21T03:41:28.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.839+0000 7f97bdac6640 1 --2- 172.21.15.38:0/3696445974 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x5605bcdeb700 0x5605bcdedb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:28.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.839+0000 7f97bdac6640 1 --2- 172.21.15.38:0/3696445974 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x5605bcdeb700 0x5605bcdedb20 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:28.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.840+0000 7f97bdac6640 1 -- 172.21.15.38:0/3696445974 <== osd.2 v2:172.21.15.38:6824/2577858902 1 ==== osd_op_reply(109 400.00000000 [create] v107'52 uv51 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f97b405d850 con 0x5605bcdeb700 2024-09-21T03:41:28.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.840+0000 7f97c5125f40 1 -- 172.21.15.38:0/3696445974 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:110 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e107) -- 0x5605bcdee210 con 0x5605bcdeb700 2024-09-21T03:41:28.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.841+0000 7f97bdac6640 1 -- 172.21.15.38:0/3696445974 <== osd.2 v2:172.21.15.38:6824/2577858902 2 ==== osd_op_reply(110 400.00000000 [delete] v107'53 uv53 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f97b405d850 con 0x5605bcdeb700 2024-09-21T03:41:28.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.841+0000 7f97c5125f40 1 -- 172.21.15.38:0/3696445974 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5605bcdee210 con 0x5605bcdeb700 2024-09-21T03:41:28.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.842+0000 7f97bdac6640 1 -- 172.21.15.38:0/3696445974 <== osd.2 v2:172.21.15.38:6824/2577858902 3 ==== osd_op_reply(111 400.00000000 [create] v107'54 uv54 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f97b405d850 con 0x5605bcdeb700 2024-09-21T03:41:28.844 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-09-21T03:41:28.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.842+0000 7f97c5125f40 1 -- 172.21.15.38:0/3696445974 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5605bcdeae20 con 0x5605bcdeb700 2024-09-21T03:41:28.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.843+0000 7f97bdac6640 1 -- 172.21.15.38:0/3696445974 <== osd.2 v2:172.21.15.38:6824/2577858902 4 ==== osd_op_reply(112 400.00000000 [write 0~22] v107'55 uv55 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f97b405d850 con 0x5605bcdeb700 2024-09-21T03:41:28.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.843+0000 7f97c5125f40 1 -- 172.21.15.38:0/3696445974 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5605bcdeade0 con 0x5605bcdeb700 2024-09-21T03:41:28.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.844+0000 7f97bdac6640 1 -- 172.21.15.38:0/3696445974 <== osd.2 v2:172.21.15.38:6824/2577858902 5 ==== osd_op_reply(113 400.00000000 [omap-set-header] v107'56 uv56 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f97b405d850 con 0x5605bcdeb700 2024-09-21T03:41:28.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.844+0000 7f97c5125f40 1 -- 172.21.15.38:0/3696445974 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:114 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5605bcdeade0 con 0x5605bcde83e0 2024-09-21T03:41:28.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.845+0000 7f97be2c7640 1 -- 172.21.15.38:0/3696445974 <== osd.1 v2:172.21.15.38:6816/1491818732 15 ==== osd_op_reply(114 100.00000000.inode [create] v107'132 uv123 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7f97b000b040 con 0x5605bcde83e0 2024-09-21T03:41:28.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.845+0000 7f97c5125f40 1 -- 172.21.15.38:0/3696445974 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:115 13.1f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e107) -- 0x5605bcdeae00 con 0x5605bcde83e0 2024-09-21T03:41:28.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.846+0000 7f97be2c7640 1 -- 172.21.15.38:0/3696445974 <== osd.1 v2:172.21.15.38:6816/1491818732 16 ==== osd_op_reply(115 100.00000000.inode [delete] v107'133 uv133 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f97b000b040 con 0x5605bcde83e0 2024-09-21T03:41:28.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.846+0000 7f97c5125f40 1 -- 172.21.15.38:0/3696445974 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:116 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5605bcdeae00 con 0x5605bcde83e0 2024-09-21T03:41:28.849 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-09-21T03:41:28.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.848+0000 7f97be2c7640 1 -- 172.21.15.38:0/3696445974 <== osd.1 v2:172.21.15.38:6816/1491818732 17 ==== osd_op_reply(116 100.00000000.inode [create] v107'134 uv134 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f97b000b040 con 0x5605bcde83e0 2024-09-21T03:41:28.850 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.848+0000 7f97c5125f40 1 -- 172.21.15.38:0/3696445974 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:117 13.1f 13:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5605bcdee820 con 0x5605bcde83e0 2024-09-21T03:41:28.851 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.849+0000 7f97be2c7640 1 -- 172.21.15.38:0/3696445974 <== osd.1 v2:172.21.15.38:6816/1491818732 18 ==== osd_op_reply(117 100.00000000.inode [write 0~582] v107'135 uv135 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f97b000b040 con 0x5605bcde83e0 2024-09-21T03:41:28.851 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.849+0000 7f97c5125f40 1 -- 172.21.15.38:0/3696445974 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:118 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5605bcdee820 con 0x5605bcde83e0 2024-09-21T03:41:28.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.850+0000 7f97be2c7640 1 -- 172.21.15.38:0/3696445974 <== osd.1 v2:172.21.15.38:6816/1491818732 19 ==== osd_op_reply(118 100.00000000.inode [omap-set-header] v107'136 uv136 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f97b000b040 con 0x5605bcde83e0 2024-09-21T03:41:28.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.850+0000 7f97c5125f40 1 -- 172.21.15.38:0/3696445974 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:119 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5605bcdee7b0 con 0x5605bcde83e0 2024-09-21T03:41:28.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.851+0000 7f97be2c7640 1 -- 172.21.15.38:0/3696445974 <== osd.1 v2:172.21.15.38:6816/1491818732 20 ==== osd_op_reply(119 1.00000000 [create] v107'137 uv130 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7f97b000b040 con 0x5605bcde83e0 2024-09-21T03:41:28.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.851+0000 7f97c5125f40 1 -- 172.21.15.38:0/3696445974 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:120 13.1f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e107) -- 0x5605bcdee7b0 con 0x5605bcde83e0 2024-09-21T03:41:28.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.852+0000 7f97be2c7640 1 -- 172.21.15.38:0/3696445974 <== osd.1 v2:172.21.15.38:6816/1491818732 21 ==== osd_op_reply(120 1.00000000 [delete] v107'138 uv138 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f97b000b040 con 0x5605bcde83e0 2024-09-21T03:41:28.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.852+0000 7f97c5125f40 1 -- 172.21.15.38:0/3696445974 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:121 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5605bcdee760 con 0x5605bcde83e0 2024-09-21T03:41:28.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.853+0000 7f97be2c7640 1 -- 172.21.15.38:0/3696445974 <== osd.1 v2:172.21.15.38:6816/1491818732 22 ==== osd_op_reply(121 1.00000000 [create] v107'139 uv139 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f97b000b040 con 0x5605bcde83e0 2024-09-21T03:41:28.855 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-09-21T03:41:28.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.853+0000 7f97c5125f40 1 -- 172.21.15.38:0/3696445974 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:122 13.1f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5605bcdef770 con 0x5605bcde83e0 2024-09-21T03:41:28.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.854+0000 7f97be2c7640 1 -- 172.21.15.38:0/3696445974 <== osd.1 v2:172.21.15.38:6816/1491818732 23 ==== osd_op_reply(122 1.00000000 [setxattr (30)] v107'140 uv140 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f97b000b040 con 0x5605bcde83e0 2024-09-21T03:41:28.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.854+0000 7f97c5125f40 1 -- 172.21.15.38:0/3696445974 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:123 13.1f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5605bcdef770 con 0x5605bcde83e0 2024-09-21T03:41:28.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.856+0000 7f97be2c7640 1 -- 172.21.15.38:0/3696445974 <== osd.1 v2:172.21.15.38:6816/1491818732 24 ==== osd_op_reply(123 1.00000000 [setxattr (30)] v107'141 uv141 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f97b000b040 con 0x5605bcde83e0 2024-09-21T03:41:28.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.856+0000 7f97c5125f40 1 -- 172.21.15.38:0/3696445974 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:124 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5605bcdee760 con 0x5605bcde83e0 2024-09-21T03:41:28.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.857+0000 7f97be2c7640 1 -- 172.21.15.38:0/3696445974 <== osd.1 v2:172.21.15.38:6816/1491818732 25 ==== osd_op_reply(124 1.00000000 [omap-set-header] v107'142 uv142 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f97b000b040 con 0x5605bcde83e0 2024-09-21T03:41:28.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.857+0000 7f97c5125f40 1 -- 172.21.15.38:0/3696445974 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:125 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=584b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5605bcdee760 con 0x5605bcde83e0 2024-09-21T03:41:28.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.859+0000 7f97be2c7640 1 -- 172.21.15.38:0/3696445974 <== osd.1 v2:172.21.15.38:6816/1491818732 26 ==== osd_op_reply(125 1.00000000 [omap-set-vals] v107'143 uv143 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f97b000b040 con 0x5605bcde83e0 2024-09-21T03:41:28.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.859+0000 7f97c5125f40 1 -- 172.21.15.38:0/3696445974 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x5605bcddff70 msgr2=0x5605bcde2390 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:28.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.859+0000 7f97c5125f40 1 --2- 172.21.15.38:0/3696445974 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x5605bcddff70 0x5605bcde2390 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:28.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.859+0000 7f97c5125f40 1 -- 172.21.15.38:0/3696445974 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x5605bcde83e0 msgr2=0x5605bcdea800 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:28.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.859+0000 7f97c5125f40 1 --2- 172.21.15.38:0/3696445974 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x5605bcde83e0 0x5605bcdea800 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:28.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.859+0000 7f97c5125f40 1 -- 172.21.15.38:0/3696445974 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x5605bcdeb700 msgr2=0x5605bcdedb20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:28.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.859+0000 7f97c5125f40 1 --2- 172.21.15.38:0/3696445974 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x5605bcdeb700 0x5605bcdedb20 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:28.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.859+0000 7f97c5125f40 1 -- 172.21.15.38:0/3696445974 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x5605bcde3e70 msgr2=0x5605bcde6c60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:28.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.859+0000 7f97c5125f40 1 --2- 172.21.15.38:0/3696445974 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x5605bcde3e70 0x5605bcde6c60 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:28.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.859+0000 7f97beac8640 1 -- 172.21.15.38:0/3696445974 reap_dead start 2024-09-21T03:41:28.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.859+0000 7f97c5125f40 1 -- 172.21.15.38:0/3696445974 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x5605bcddb2f0 msgr2=0x5605bcddd710 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:28.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.859+0000 7f97c5125f40 1 --2- 172.21.15.38:0/3696445974 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x5605bcddb2f0 0x5605bcddd710 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:28.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.859+0000 7f97beac8640 1 -- 172.21.15.38:0/3696445974 reap_dead start 2024-09-21T03:41:28.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.859+0000 7f97c5125f40 1 -- 172.21.15.38:0/3696445974 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x5605bcd19e00 msgr2=0x5605bcca1650 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:28.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.859+0000 7f97c5125f40 1 --2- 172.21.15.38:0/3696445974 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x5605bcd19e00 0x5605bcca1650 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:28.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.859+0000 7f97c5125f40 1 -- 172.21.15.38:0/3696445974 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f97a0077570 msgr2=0x7f97a0079a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:28.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.859+0000 7f97c5125f40 1 --2- 172.21.15.38:0/3696445974 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f97a0077570 0x7f97a0079a80 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f97b0056000 tx=0x7f97b00033a0 comp rx=0 tx=0).stop 2024-09-21T03:41:28.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.859+0000 7f97c5125f40 1 -- 172.21.15.38:0/3696445974 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5605bcd1c950 msgr2=0x5605bcc8f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:28.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.859+0000 7f97c5125f40 1 --2- 172.21.15.38:0/3696445974 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5605bcd1c950 0x5605bcc8f110 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f97b400a680 tx=0x7f97b400ab40 comp rx=0 tx=0).stop 2024-09-21T03:41:28.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.859+0000 7f97c5125f40 1 -- 172.21.15.38:0/3696445974 shutdown_connections 2024-09-21T03:41:28.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.859+0000 7f97c5125f40 1 --2- 172.21.15.38:0/3696445974 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x5605bcd19e00 0x5605bcca1650 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:28.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.859+0000 7f97c5125f40 1 --2- 172.21.15.38:0/3696445974 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f97a0077570 0x7f97a0079a80 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:28.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.859+0000 7f97c5125f40 1 --2- 172.21.15.38:0/3696445974 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5605bcd1c950 0x5605bcc8f110 unknown :-1 s=CLOSED pgs=1492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:28.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.859+0000 7f97c5125f40 1 -- 172.21.15.38:0/3696445974 >> 172.21.15.38:0/3696445974 conn(0x5605bcb80ac0 msgr2=0x5605bcd37d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:28.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.859+0000 7f97c5125f40 1 -- 172.21.15.38:0/3696445974 shutdown_connections 2024-09-21T03:41:28.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.859+0000 7f97c5125f40 1 -- 172.21.15.38:0/3696445974 wait complete. 2024-09-21T03:41:28.865 DEBUG:teuthology.orchestra.run.smithi038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rm mds0_sessionmap 2024-09-21T03:41:28.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.960+0000 7f6319f20f40 1 Processor -- start 2024-09-21T03:41:28.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.960+0000 7f6319f20f40 1 -- start start 2024-09-21T03:41:28.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.961+0000 7f6319f20f40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5578b0efd240 0x5578b105dee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:28.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.961+0000 7f6319f20f40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5578b0f44090 0x5578b0f259d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:28.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.961+0000 7f6319f20f40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5578b0fb9520 0x5578b0f25f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:28.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.961+0000 7f6319f20f40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5578b0ea9430 con 0x5578b0fb9520 2024-09-21T03:41:28.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.961+0000 7f6319f20f40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5578b0eb61f0 con 0x5578b0f44090 2024-09-21T03:41:28.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.961+0000 7f6319f20f40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x5578b0e4ad60 con 0x5578b0efd240 2024-09-21T03:41:28.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.961+0000 7f63141db640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5578b0fb9520 0x5578b0f25f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:28.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.961+0000 7f63141db640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5578b0fb9520 0x5578b0f25f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:52726/0 (socket says 172.21.15.38:52726) 2024-09-21T03:41:28.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.961+0000 7f63141db640 1 -- 172.21.15.38:0/1564228292 learned_addr learned my addr 172.21.15.38:0/1564228292 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:41:28.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.961+0000 7f630bfff640 1 --2- 172.21.15.38:0/1564228292 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5578b0f44090 0x5578b0f259d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:28.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.961+0000 7f63139da640 1 --2- 172.21.15.38:0/1564228292 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5578b0efd240 0x5578b105dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:28.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.961+0000 7f63141db640 1 -- 172.21.15.38:0/1564228292 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5578b0efd240 msgr2=0x5578b105dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:28.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.961+0000 7f63141db640 1 --2- 172.21.15.38:0/1564228292 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5578b0efd240 0x5578b105dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:28.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.961+0000 7f63141db640 1 -- 172.21.15.38:0/1564228292 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5578b0f44090 msgr2=0x5578b0f259d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:28.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.961+0000 7f63141db640 1 --2- 172.21.15.38:0/1564228292 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5578b0f44090 0x5578b0f259d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:28.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.961+0000 7f63141db640 1 -- 172.21.15.38:0/1564228292 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578b0fb3ec0 con 0x5578b0fb9520 2024-09-21T03:41:28.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.962+0000 7f63139da640 1 --2- 172.21.15.38:0/1564228292 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5578b0efd240 0x5578b105dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:41:28.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.962+0000 7f63141db640 1 --2- 172.21.15.38:0/1564228292 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5578b0fb9520 0x5578b0f25f10 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f630c001570 tx=0x7f630c001a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae0a97b03ab1a896 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:28.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.962+0000 7f63129d8640 1 -- 172.21.15.38:0/1564228292 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f630c017070 con 0x5578b0fb9520 2024-09-21T03:41:28.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.962+0000 7f63129d8640 1 -- 172.21.15.38:0/1564228292 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f630c002780 con 0x5578b0fb9520 2024-09-21T03:41:28.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.962+0000 7f63129d8640 1 -- 172.21.15.38:0/1564228292 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f630c002bd0 con 0x5578b0fb9520 2024-09-21T03:41:28.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.962+0000 7f6319f20f40 1 -- 172.21.15.38:0/1564228292 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5578b0fb9520 msgr2=0x5578b0f25f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:28.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.962+0000 7f6319f20f40 1 --2- 172.21.15.38:0/1564228292 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5578b0fb9520 0x5578b0f25f10 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f630c001570 tx=0x7f630c001a30 comp rx=0 tx=0).stop 2024-09-21T03:41:28.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.962+0000 7f6319f20f40 1 -- 172.21.15.38:0/1564228292 shutdown_connections 2024-09-21T03:41:28.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.962+0000 7f6319f20f40 1 --2- 172.21.15.38:0/1564228292 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5578b0fb9520 0x5578b0f25f10 unknown :-1 s=CLOSED pgs=1495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:28.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.962+0000 7f6319f20f40 1 --2- 172.21.15.38:0/1564228292 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5578b0f44090 0x5578b0f259d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:28.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.962+0000 7f6319f20f40 1 --2- 172.21.15.38:0/1564228292 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5578b0efd240 0x5578b105dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:28.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.963+0000 7f6319f20f40 1 -- 172.21.15.38:0/1564228292 >> 172.21.15.38:0/1564228292 conn(0x5578b0e1bac0 msgr2=0x5578b0fc9eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:28.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.963+0000 7f6319f20f40 1 -- 172.21.15.38:0/1564228292 shutdown_connections 2024-09-21T03:41:28.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.963+0000 7f6319f20f40 1 -- 172.21.15.38:0/1564228292 wait complete. 2024-09-21T03:41:28.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.963+0000 7f6319f20f40 1 Processor -- start 2024-09-21T03:41:28.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.963+0000 7f6319f20f40 1 -- start start 2024-09-21T03:41:28.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.965+0000 7f6319f20f40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5578b0efd240 0x5578b0f25b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:28.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.965+0000 7f6319f20f40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5578b0f44090 0x5578b0fb5f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:28.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.965+0000 7f6319f20f40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5578b0fb9520 0x5578b0fb6450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:28.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.965+0000 7f6319f20f40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5578b0f42b80 con 0x5578b0efd240 2024-09-21T03:41:28.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.965+0000 7f6319f20f40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5578b0fce450 con 0x5578b0f44090 2024-09-21T03:41:28.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.965+0000 7f6319f20f40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x5578b0f438b0 con 0x5578b0fb9520 2024-09-21T03:41:28.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.965+0000 7f63139da640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5578b0efd240 0x5578b0f25b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:28.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.965+0000 7f63139da640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5578b0efd240 0x5578b0f25b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:52738/0 (socket says 172.21.15.38:52738) 2024-09-21T03:41:28.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.965+0000 7f63139da640 1 -- 172.21.15.38:0/2322343153 learned_addr learned my addr 172.21.15.38:0/2322343153 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:41:28.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.965+0000 7f630bfff640 1 --2- 172.21.15.38:0/2322343153 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5578b0f44090 0x5578b0fb5f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:28.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.965+0000 7f63141db640 1 --2- 172.21.15.38:0/2322343153 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5578b0fb9520 0x5578b0fb6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:28.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.965+0000 7f63139da640 1 -- 172.21.15.38:0/2322343153 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5578b0fb9520 msgr2=0x5578b0fb6450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:28.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.965+0000 7f63139da640 1 --2- 172.21.15.38:0/2322343153 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5578b0fb9520 0x5578b0fb6450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:28.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.965+0000 7f63139da640 1 -- 172.21.15.38:0/2322343153 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5578b0f44090 msgr2=0x5578b0fb5f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:28.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.965+0000 7f63139da640 1 --2- 172.21.15.38:0/2322343153 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5578b0f44090 0x5578b0fb5f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:28.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.965+0000 7f63139da640 1 -- 172.21.15.38:0/2322343153 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578b1068520 con 0x5578b0efd240 2024-09-21T03:41:28.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.965+0000 7f630bfff640 1 --2- 172.21.15.38:0/2322343153 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5578b0f44090 0x5578b0fb5f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:41:28.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.965+0000 7f63141db640 1 --2- 172.21.15.38:0/2322343153 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5578b0fb9520 0x5578b0fb6450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:41:28.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.965+0000 7f63139da640 1 --2- 172.21.15.38:0/2322343153 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5578b0efd240 0x5578b0f25b50 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f630400df70 tx=0x7f630400b5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:28.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.966+0000 7f63119d6640 1 -- 172.21.15.38:0/2322343153 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63040032d0 con 0x5578b0efd240 2024-09-21T03:41:28.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.966+0000 7f63119d6640 1 -- 172.21.15.38:0/2322343153 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6304016070 con 0x5578b0efd240 2024-09-21T03:41:28.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.966+0000 7f6319f20f40 1 -- 172.21.15.38:0/2322343153 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5578b1067760 con 0x5578b0efd240 2024-09-21T03:41:28.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.966+0000 7f63119d6640 1 -- 172.21.15.38:0/2322343153 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63040116c0 con 0x5578b0efd240 2024-09-21T03:41:28.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.966+0000 7f6319f20f40 1 -- 172.21.15.38:0/2322343153 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578b1067d20 con 0x5578b0efd240 2024-09-21T03:41:28.972 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.967+0000 7f63119d6640 1 -- 172.21.15.38:0/2322343153 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f6304011860 con 0x5578b0efd240 2024-09-21T03:41:28.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.968+0000 7f63119d6640 1 --2- 172.21.15.38:0/2322343153 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f62f4077570 0x7f62f4079a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:28.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.968+0000 7f63119d6640 1 -- 172.21.15.38:0/2322343153 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 7250+0+0 (secure 0 0 0) 0x7f6304099f60 con 0x5578b0efd240 2024-09-21T03:41:28.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.968+0000 7f6319f20f40 1 --2- 172.21.15.38:0/2322343153 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x5578b0fb4a50 0x5578b105cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:28.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.968+0000 7f6319f20f40 1 -- 172.21.15.38:0/2322343153 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:1 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e107) -- 0x5578b105e230 con 0x5578b0fb4a50 2024-09-21T03:41:28.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.968+0000 7f630bfff640 1 --2- 172.21.15.38:0/2322343153 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f62f4077570 0x7f62f4079a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:28.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.968+0000 7f63141db640 1 --2- 172.21.15.38:0/2322343153 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x5578b0fb4a50 0x5578b105cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:28.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.969+0000 7f63141db640 1 --2- 172.21.15.38:0/2322343153 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x5578b0fb4a50 0x5578b105cae0 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:28.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.969+0000 7f630bfff640 1 --2- 172.21.15.38:0/2322343153 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f62f4077570 0x7f62f4079a80 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f62fc055000 tx=0x7f62fc058040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:28.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.970+0000 7f63141db640 1 -- 172.21.15.38:0/2322343153 <== osd.3 v2:172.21.15.38:6808/1286810294 1 ==== osd_op_reply(1 mds0_sessionmap [delete] v107'53 uv53 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f630c00f070 con 0x5578b0fb4a50 2024-09-21T03:41:28.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.970+0000 7f6319f20f40 1 -- 172.21.15.38:0/2322343153 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x5578b0fb4a50 msgr2=0x5578b105cae0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:28.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.970+0000 7f6319f20f40 1 --2- 172.21.15.38:0/2322343153 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x5578b0fb4a50 0x5578b105cae0 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:28.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.970+0000 7f6319f20f40 1 -- 172.21.15.38:0/2322343153 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f62f4077570 msgr2=0x7f62f4079a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:28.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.970+0000 7f6319f20f40 1 --2- 172.21.15.38:0/2322343153 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f62f4077570 0x7f62f4079a80 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f62fc055000 tx=0x7f62fc058040 comp rx=0 tx=0).stop 2024-09-21T03:41:28.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.970+0000 7f6319f20f40 1 -- 172.21.15.38:0/2322343153 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5578b0efd240 msgr2=0x5578b0f25b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:28.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.970+0000 7f6319f20f40 1 --2- 172.21.15.38:0/2322343153 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5578b0efd240 0x5578b0f25b50 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f630400df70 tx=0x7f630400b5b0 comp rx=0 tx=0).stop 2024-09-21T03:41:28.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.970+0000 7f63141db640 1 -- 172.21.15.38:0/2322343153 reap_dead start 2024-09-21T03:41:28.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.970+0000 7f6319f20f40 1 -- 172.21.15.38:0/2322343153 shutdown_connections 2024-09-21T03:41:28.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.970+0000 7f6319f20f40 1 -- 172.21.15.38:0/2322343153 >> 172.21.15.38:0/2322343153 conn(0x5578b0e1bac0 msgr2=0x5578b0fc9eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:28.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.970+0000 7f6319f20f40 1 -- 172.21.15.38:0/2322343153 shutdown_connections 2024-09-21T03:41:28.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:28.970+0000 7f6319f20f40 1 -- 172.21.15.38:0/2322343153 wait complete. 2024-09-21T03:41:28.975 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-21T03:41:29.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:29.094+0000 7f17d5d50640 1 -- 172.21.15.38:0/4148987965 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f17b8003a50 msgr2=0x7f17b8003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:29.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:29.094+0000 7f17d5d50640 1 --2- 172.21.15.38:0/4148987965 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f17b8003a50 0x7f17b8003ec0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f17c00016c0 tx=0x7f17b8005830 comp rx=0 tx=0).stop 2024-09-21T03:41:29.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:29.094+0000 7f17d5d50640 1 -- 172.21.15.38:0/4148987965 shutdown_connections 2024-09-21T03:41:29.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:29.094+0000 7f17d5d50640 1 --2- 172.21.15.38:0/4148987965 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f17b8003a50 0x7f17b8003ec0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:29.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:29.094+0000 7f17d5d50640 1 -- 172.21.15.38:0/4148987965 >> 172.21.15.38:0/4148987965 conn(0x7f17d009d830 msgr2=0x7f17d00b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:29.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:29.094+0000 7f17d5d50640 1 -- 172.21.15.38:0/4148987965 shutdown_connections 2024-09-21T03:41:29.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:29.094+0000 7f17d5d50640 1 -- 172.21.15.38:0/4148987965 wait complete. 2024-09-21T03:41:29.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:29.095+0000 7f17d5d50640 1 Processor -- start 2024-09-21T03:41:29.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:29.096+0000 7f17d5d50640 1 -- start start 2024-09-21T03:41:29.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:29.097+0000 7f17d5d50640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f17b8003a50 0x7f17d006d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:29.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:29.097+0000 7f17d5d50640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f17d014f600 0x7f17d00b2780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:29.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:29.097+0000 7f17d5d50640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f17d014fa10 0x7f17d00b2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:29.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:29.097+0000 7f17d5d50640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f17d00b9c50 con 0x7f17d014fa10 2024-09-21T03:41:29.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:29.097+0000 7f17d5d50640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f17d00b9ad0 con 0x7f17d014f600 2024-09-21T03:41:29.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:29.097+0000 7f17d5d50640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f17d00b9dd0 con 0x7f17b8003a50 2024-09-21T03:41:29.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:29.097+0000 7f17cffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f17d014fa10 0x7f17d00b2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:29.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:29.097+0000 7f17cffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f17d014fa10 0x7f17d00b2cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:52750/0 (socket says 172.21.15.38:52750) 2024-09-21T03:41:29.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:29.097+0000 7f17cffff640 1 -- 172.21.15.38:0/3532851216 learned_addr learned my addr 172.21.15.38:0/3532851216 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:41:29.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:29.097+0000 7f17cf7fe640 1 --2- 172.21.15.38:0/3532851216 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f17b8003a50 0x7f17d006d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:29.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:29.097+0000 7f17ceffd640 1 --2- 172.21.15.38:0/3532851216 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f17d014f600 0x7f17d00b2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:29.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:29.097+0000 7f17cffff640 1 -- 172.21.15.38:0/3532851216 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f17b8003a50 msgr2=0x7f17d006d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:29.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:29.097+0000 7f17cffff640 1 --2- 172.21.15.38:0/3532851216 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f17b8003a50 0x7f17d006d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:29.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:29.097+0000 7f17cffff640 1 -- 172.21.15.38:0/3532851216 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f17d014f600 msgr2=0x7f17d00b2780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:29.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:29.097+0000 7f17cffff640 1 --2- 172.21.15.38:0/3532851216 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f17d014f600 0x7f17d00b2780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:29.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:29.097+0000 7f17cffff640 1 -- 172.21.15.38:0/3532851216 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17d006ebd0 con 0x7f17d014fa10 2024-09-21T03:41:29.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:29.098+0000 7f17cffff640 1 --2- 172.21.15.38:0/3532851216 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f17d014fa10 0x7f17d00b2cc0 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f17d015c9c0 tx=0x7f17c80064f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:29.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:29.098+0000 7f17b7fff640 1 -- 172.21.15.38:0/3532851216 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17c80119c0 con 0x7f17d014fa10 2024-09-21T03:41:29.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:29.098+0000 7f17b7fff640 1 -- 172.21.15.38:0/3532851216 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f17c8017070 con 0x7f17d014fa10 2024-09-21T03:41:29.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:29.098+0000 7f17b7fff640 1 -- 172.21.15.38:0/3532851216 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17c8011e40 con 0x7f17d014fa10 2024-09-21T03:41:29.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:29.098+0000 7f17d5d50640 1 -- 172.21.15.38:0/3532851216 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17d006eeb0 con 0x7f17d014fa10 2024-09-21T03:41:29.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:29.098+0000 7f17d5d50640 1 -- 172.21.15.38:0/3532851216 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17d006f470 con 0x7f17d014fa10 2024-09-21T03:41:29.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:29.099+0000 7f17d5d50640 1 -- 172.21.15.38:0/3532851216 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17d0051050 con 0x7f17d014fa10 2024-09-21T03:41:29.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:29.100+0000 7f17b7fff640 1 -- 172.21.15.38:0/3532851216 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f17c801c460 con 0x7f17d014fa10 2024-09-21T03:41:29.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:29.100+0000 7f17b7fff640 1 --2- 172.21.15.38:0/3532851216 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f17bc078b90 0x7f17bc07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:29.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:29.100+0000 7f17b7fff640 1 -- 172.21.15.38:0/3532851216 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 7250+0+0 (secure 0 0 0) 0x7f17c809b9f0 con 0x7f17d014fa10 2024-09-21T03:41:29.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:29.100+0000 7f17cf7fe640 1 --2- 172.21.15.38:0/3532851216 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f17bc078b90 0x7f17bc07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:29.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:29.101+0000 7f17cf7fe640 1 --2- 172.21.15.38:0/3532851216 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f17bc078b90 0x7f17bc07b0a0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f17c0032000 tx=0x7f17c0001190 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:29.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:29.105+0000 7f17b7fff640 1 -- 172.21.15.38:0/3532851216 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f17d0051050 con 0x7f17d014fa10 2024-09-21T03:41:29.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:29.331+0000 7f17d5d50640 1 -- 172.21.15.38:0/3532851216 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f17d00cec70 con 0x7f17d014fa10 2024-09-21T03:41:29.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:29.628+0000 7f17b7fff640 1 -- 172.21.15.38:0/3532851216 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v152) ==== 162+0+0 (secure 0 0 0) 0x7f17c8068e20 con 0x7f17d014fa10 2024-09-21T03:41:29.630 INFO:teuthology.orchestra.run.smithi038.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-21T03:41:29.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:29.630+0000 7f17d5d50640 1 -- 172.21.15.38:0/3532851216 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f17bc078b90 msgr2=0x7f17bc07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:29.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:29.630+0000 7f17d5d50640 1 --2- 172.21.15.38:0/3532851216 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f17bc078b90 0x7f17bc07b0a0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f17c0032000 tx=0x7f17c0001190 comp rx=0 tx=0).stop 2024-09-21T03:41:29.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:29.631+0000 7f17d5d50640 1 -- 172.21.15.38:0/3532851216 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f17d014fa10 msgr2=0x7f17d00b2cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:29.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:29.631+0000 7f17d5d50640 1 --2- 172.21.15.38:0/3532851216 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f17d014fa10 0x7f17d00b2cc0 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f17d015c9c0 tx=0x7f17c80064f0 comp rx=0 tx=0).stop 2024-09-21T03:41:29.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:29.631+0000 7f17d5d50640 1 -- 172.21.15.38:0/3532851216 shutdown_connections 2024-09-21T03:41:29.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:29.631+0000 7f17d5d50640 1 --2- 172.21.15.38:0/3532851216 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f17d014fa10 0x7f17d00b2cc0 unknown :-1 s=CLOSED pgs=1498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:29.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:29.631+0000 7f17d5d50640 1 --2- 172.21.15.38:0/3532851216 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f17d014f600 0x7f17d00b2780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:29.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:29.631+0000 7f17d5d50640 1 --2- 172.21.15.38:0/3532851216 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f17bc078b90 0x7f17bc07b0a0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:29.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:29.631+0000 7f17d5d50640 1 --2- 172.21.15.38:0/3532851216 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f17b8003a50 0x7f17d006d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:29.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:29.631+0000 7f17d5d50640 1 -- 172.21.15.38:0/3532851216 >> 172.21.15.38:0/3532851216 conn(0x7f17d009d830 msgr2=0x7f17d015df60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:29.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:29.631+0000 7f17d5d50640 1 -- 172.21.15.38:0/3532851216 shutdown_connections 2024-09-21T03:41:29.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:29.631+0000 7f17d5d50640 1 -- 172.21.15.38:0/3532851216 wait complete. 2024-09-21T03:41:29.639 INFO:tasks.ceph.mds.c.smithi038.stderr:2024-09-21T03:41:29.638+0000 7f4d93216640 -1 mds.0.sessionmap _load_finish got (2) No such file or directory 2024-09-21T03:41:29.639 INFO:tasks.ceph.mds.c.smithi038.stderr:2024-09-21T03:41:29.638+0000 7f4d93216640 -1 log_channel(cluster) log [ERR] : error reading sessionmap 'mds0_sessionmap' -2 ((2) No such file or directory) 2024-09-21T03:41:29.640 INFO:tasks.ceph.mon.a.smithi038.stderr:2024-09-21T03:41:29.639+0000 7f3f45713640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T03:41:29.640 INFO:tasks.ceph.mon.a.smithi038.stderr:2024-09-21T03:41:29.639+0000 7f3f45713640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-09-21T03:41:29.647 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:41:29.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:29.764+0000 7f13b5c4d640 1 -- 172.21.15.38:0/4142724913 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f13b015ecf0 msgr2=0x7f13b00cf480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:29.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:29.764+0000 7f13b5c4d640 1 --2- 172.21.15.38:0/4142724913 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f13b015ecf0 0x7f13b00cf480 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f13a0009830 tx=0x7f13a002a510 comp rx=0 tx=0).stop 2024-09-21T03:41:29.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:29.764+0000 7f13b5c4d640 1 -- 172.21.15.38:0/4142724913 shutdown_connections 2024-09-21T03:41:29.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:29.764+0000 7f13b5c4d640 1 --2- 172.21.15.38:0/4142724913 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f13b015ecf0 0x7f13b00cf480 unknown :-1 s=CLOSED pgs=1500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:29.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:29.764+0000 7f13b5c4d640 1 -- 172.21.15.38:0/4142724913 >> 172.21.15.38:0/4142724913 conn(0x7f13b00d4600 msgr2=0x7f13b00d4a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:29.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:29.764+0000 7f13b5c4d640 1 -- 172.21.15.38:0/4142724913 shutdown_connections 2024-09-21T03:41:29.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:29.765+0000 7f13b5c4d640 1 -- 172.21.15.38:0/4142724913 wait complete. 2024-09-21T03:41:29.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:29.766+0000 7f13b5c4d640 1 Processor -- start 2024-09-21T03:41:29.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:29.767+0000 7f13b5c4d640 1 -- start start 2024-09-21T03:41:29.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:29.768+0000 7f13b5c4d640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f13b00b43a0 0x7f13b01514f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:29.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:29.768+0000 7f13b5c4d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f13b015ecf0 0x7f13b0156a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:29.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:29.768+0000 7f13b5c4d640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f13b01627b0 0x7f13b0152d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:29.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:29.768+0000 7f13b5c4d640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f13b00b5250 con 0x7f13b015ecf0 2024-09-21T03:41:29.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:29.768+0000 7f13b5c4d640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f13b00b50d0 con 0x7f13b00b43a0 2024-09-21T03:41:29.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:29.768+0000 7f13b5c4d640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f13b00b53d0 con 0x7f13b01627b0 2024-09-21T03:41:29.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:29.768+0000 7f13aeffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f13b015ecf0 0x7f13b0156a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:29.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:29.768+0000 7f13aeffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f13b015ecf0 0x7f13b0156a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:52768/0 (socket says 172.21.15.38:52768) 2024-09-21T03:41:29.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:29.768+0000 7f13aeffd640 1 -- 172.21.15.38:0/2976283553 learned_addr learned my addr 172.21.15.38:0/2976283553 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:41:29.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:29.768+0000 7f13af7fe640 1 --2- 172.21.15.38:0/2976283553 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f13b00b43a0 0x7f13b01514f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:29.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:29.768+0000 7f13affff640 1 --2- 172.21.15.38:0/2976283553 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f13b01627b0 0x7f13b0152d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:29.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:29.768+0000 7f13aeffd640 1 -- 172.21.15.38:0/2976283553 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f13b01627b0 msgr2=0x7f13b0152d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:29.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:29.768+0000 7f13aeffd640 1 --2- 172.21.15.38:0/2976283553 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f13b01627b0 0x7f13b0152d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:29.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:29.768+0000 7f13aeffd640 1 -- 172.21.15.38:0/2976283553 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f13b00b43a0 msgr2=0x7f13b01514f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:29.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:29.768+0000 7f13aeffd640 1 --2- 172.21.15.38:0/2976283553 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f13b00b43a0 0x7f13b01514f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:29.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:29.768+0000 7f13aeffd640 1 -- 172.21.15.38:0/2976283553 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13b0157400 con 0x7f13b015ecf0 2024-09-21T03:41:29.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:29.769+0000 7f13aeffd640 1 --2- 172.21.15.38:0/2976283553 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f13b015ecf0 0x7f13b0156a10 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7f13a400b3f0 tx=0x7f13a400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:29.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:29.769+0000 7f1397fff640 1 -- 172.21.15.38:0/2976283553 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13a4015020 con 0x7f13b015ecf0 2024-09-21T03:41:29.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:29.769+0000 7f13b5c4d640 1 -- 172.21.15.38:0/2976283553 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13b01576e0 con 0x7f13b015ecf0 2024-09-21T03:41:29.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:29.769+0000 7f1397fff640 1 -- 172.21.15.38:0/2976283553 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f13a4011920 con 0x7f13b015ecf0 2024-09-21T03:41:29.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:29.769+0000 7f1397fff640 1 -- 172.21.15.38:0/2976283553 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13a4011d70 con 0x7f13b015ecf0 2024-09-21T03:41:29.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:29.769+0000 7f13b5c4d640 1 -- 172.21.15.38:0/2976283553 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13b0153760 con 0x7f13b015ecf0 2024-09-21T03:41:29.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:29.770+0000 7f13b5c4d640 1 -- 172.21.15.38:0/2976283553 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13b00cf480 con 0x7f13b015ecf0 2024-09-21T03:41:29.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:29.771+0000 7f1397fff640 1 -- 172.21.15.38:0/2976283553 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f13a4018020 con 0x7f13b015ecf0 2024-09-21T03:41:29.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:29.771+0000 7f1397fff640 1 --2- 172.21.15.38:0/2976283553 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f139c078bc0 0x7f139c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:29.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:29.771+0000 7f1397fff640 1 -- 172.21.15.38:0/2976283553 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 7336+0+0 (secure 0 0 0) 0x7f13a409b250 con 0x7f13b015ecf0 2024-09-21T03:41:29.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:29.771+0000 7f13af7fe640 1 --2- 172.21.15.38:0/2976283553 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f139c078bc0 0x7f139c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:29.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:29.772+0000 7f13af7fe640 1 --2- 172.21.15.38:0/2976283553 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f139c078bc0 0x7f139c07b0d0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f13a0034000 tx=0x7f13a0004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:29.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:29.776+0000 7f1397fff640 1 -- 172.21.15.38:0/2976283553 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f13a401f020 con 0x7f13b015ecf0 2024-09-21T03:41:30.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:30.005+0000 7f13b5c4d640 1 -- 172.21.15.38:0/2976283553 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f13b014b900 con 0x7f13b015ecf0 2024-09-21T03:41:30.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:30.006+0000 7f1397fff640 1 -- 172.21.15.38:0/2976283553 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 154 v154) ==== 95+0+4392 (secure 0 0 0) 0x7f13b014b900 con 0x7f13b015ecf0 2024-09-21T03:41:30.008 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:41:30.008 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":154,"btime":"2024-09-21T03:41:29:639761+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6086,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/965972663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":965972663},{"type":"v1","addr":"172.21.15.195:6836","nonce":965972663}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6700,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/3176122334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":3176122334},{"type":"v1","addr":"172.21.15.195:6837","nonce":3176122334}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":8370,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/2916238332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":2916238332},{"type":"v1","addr":"172.21.15.38:6835","nonce":2916238332}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":146}],"filesystems":[{"mdsmap":{"epoch":154,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:40:02.319282+0000","modified":"2024-09-21T03:41:29.639674+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":108,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-21T03:41:30.008 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 154 2024-09-21T03:41:30.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:30.008+0000 7f13b5c4d640 1 -- 172.21.15.38:0/2976283553 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f139c078bc0 msgr2=0x7f139c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:30.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:30.008+0000 7f13b5c4d640 1 --2- 172.21.15.38:0/2976283553 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f139c078bc0 0x7f139c07b0d0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f13a0034000 tx=0x7f13a0004730 comp rx=0 tx=0).stop 2024-09-21T03:41:30.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:30.008+0000 7f13b5c4d640 1 -- 172.21.15.38:0/2976283553 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f13b015ecf0 msgr2=0x7f13b0156a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:30.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:30.008+0000 7f13b5c4d640 1 --2- 172.21.15.38:0/2976283553 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f13b015ecf0 0x7f13b0156a10 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7f13a400b3f0 tx=0x7f13a400b8b0 comp rx=0 tx=0).stop 2024-09-21T03:41:30.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:30.008+0000 7f13b5c4d640 1 -- 172.21.15.38:0/2976283553 shutdown_connections 2024-09-21T03:41:30.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:30.008+0000 7f13b5c4d640 1 --2- 172.21.15.38:0/2976283553 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f139c078bc0 0x7f139c07b0d0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:30.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:30.008+0000 7f13b5c4d640 1 --2- 172.21.15.38:0/2976283553 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f13b01627b0 0x7f13b0152d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:30.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:30.008+0000 7f13b5c4d640 1 --2- 172.21.15.38:0/2976283553 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f13b015ecf0 0x7f13b0156a10 unknown :-1 s=CLOSED pgs=1501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:30.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:30.008+0000 7f13b5c4d640 1 --2- 172.21.15.38:0/2976283553 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f13b00b43a0 0x7f13b01514f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:30.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:30.008+0000 7f13b5c4d640 1 -- 172.21.15.38:0/2976283553 >> 172.21.15.38:0/2976283553 conn(0x7f13b00d4600 msgr2=0x7f13b00d31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:30.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:30.009+0000 7f13b5c4d640 1 -- 172.21.15.38:0/2976283553 shutdown_connections 2024-09-21T03:41:30.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:30.009+0000 7f13b5c4d640 1 -- 172.21.15.38:0/2976283553 wait complete. 2024-09-21T03:41:30.022 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-21T03:41:30.022 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:41:30.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:30.138+0000 7f2f3bf7b640 1 -- 172.21.15.38:0/1902748726 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2f20005150 msgr2=0x7f2f2000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:30.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:30.138+0000 7f2f3bf7b640 1 --2- 172.21.15.38:0/1902748726 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2f20005150 0x7f2f2000b9c0 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f2f2c00c2e0 tx=0x7f2f20005830 comp rx=0 tx=0).stop 2024-09-21T03:41:30.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:30.138+0000 7f2f3a518640 1 -- 172.21.15.38:0/1902748726 reap_dead start 2024-09-21T03:41:30.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:30.138+0000 7f2f3bf7b640 1 -- 172.21.15.38:0/1902748726 shutdown_connections 2024-09-21T03:41:30.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:30.138+0000 7f2f3bf7b640 1 -- 172.21.15.38:0/1902748726 >> 172.21.15.38:0/1902748726 conn(0x7f2f340b6b20 msgr2=0x7f2f340d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:30.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:30.138+0000 7f2f3bf7b640 1 -- 172.21.15.38:0/1902748726 shutdown_connections 2024-09-21T03:41:30.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:30.138+0000 7f2f3bf7b640 1 -- 172.21.15.38:0/1902748726 wait complete. 2024-09-21T03:41:30.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:30.140+0000 7f2f3bf7b640 1 Processor -- start 2024-09-21T03:41:30.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:30.140+0000 7f2f3bf7b640 1 -- start start 2024-09-21T03:41:30.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:30.141+0000 7f2f3bf7b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2f20005150 0x7f2f341f4490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:30.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:30.141+0000 7f2f3bf7b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2f340c98a0 0x7f2f341f83c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:30.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:30.141+0000 7f2f3bf7b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2f341f49d0 0x7f2f341f8900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:30.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:30.141+0000 7f2f3bf7b640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f2f3415f640 con 0x7f2f341f49d0 2024-09-21T03:41:30.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:30.141+0000 7f2f3bf7b640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f2f3415f4c0 con 0x7f2f20005150 2024-09-21T03:41:30.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:30.141+0000 7f2f3bf7b640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f2f3415f7c0 con 0x7f2f340c98a0 2024-09-21T03:41:30.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:30.141+0000 7f2f3a518640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2f341f49d0 0x7f2f341f8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:30.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:30.141+0000 7f2f3a518640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2f341f49d0 0x7f2f341f8900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:52784/0 (socket says 172.21.15.38:52784) 2024-09-21T03:41:30.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:30.141+0000 7f2f3a518640 1 -- 172.21.15.38:0/2931253120 learned_addr learned my addr 172.21.15.38:0/2931253120 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:41:30.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:30.141+0000 7f2f39d17640 1 --2- 172.21.15.38:0/2931253120 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2f20005150 0x7f2f341f4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:30.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:30.141+0000 7f2f39516640 1 --2- 172.21.15.38:0/2931253120 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2f340c98a0 0x7f2f341f83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:30.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:30.142+0000 7f2f3a518640 1 -- 172.21.15.38:0/2931253120 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2f340c98a0 msgr2=0x7f2f341f83c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:30.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:30.142+0000 7f2f3a518640 1 --2- 172.21.15.38:0/2931253120 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2f340c98a0 0x7f2f341f83c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:30.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:30.142+0000 7f2f3a518640 1 -- 172.21.15.38:0/2931253120 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2f20005150 msgr2=0x7f2f341f4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:30.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:30.142+0000 7f2f3a518640 1 --2- 172.21.15.38:0/2931253120 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2f20005150 0x7f2f341f4490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:30.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:30.142+0000 7f2f3a518640 1 -- 172.21.15.38:0/2931253120 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f3414c500 con 0x7f2f341f49d0 2024-09-21T03:41:30.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:30.142+0000 7f2f3a518640 1 --2- 172.21.15.38:0/2931253120 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2f341f49d0 0x7f2f341f8900 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7f2f2c02b160 tx=0x7f2f2c02ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:30.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:30.142+0000 7f2f2a7fc640 1 -- 172.21.15.38:0/2931253120 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f2c031050 con 0x7f2f341f49d0 2024-09-21T03:41:30.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:30.142+0000 7f2f2a7fc640 1 -- 172.21.15.38:0/2931253120 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2f2c006070 con 0x7f2f341f49d0 2024-09-21T03:41:30.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:30.142+0000 7f2f3bf7b640 1 -- 172.21.15.38:0/2931253120 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f340c9f40 con 0x7f2f341f49d0 2024-09-21T03:41:30.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:30.142+0000 7f2f2a7fc640 1 -- 172.21.15.38:0/2931253120 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f2c006c90 con 0x7f2f341f49d0 2024-09-21T03:41:30.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:30.142+0000 7f2f3bf7b640 1 -- 172.21.15.38:0/2931253120 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f340ca500 con 0x7f2f341f49d0 2024-09-21T03:41:30.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:30.143+0000 7f2f3bf7b640 1 -- 172.21.15.38:0/2931253120 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f340c9cb0 con 0x7f2f341f49d0 2024-09-21T03:41:30.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:30.144+0000 7f2f2a7fc640 1 -- 172.21.15.38:0/2931253120 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f2f2c03e030 con 0x7f2f341f49d0 2024-09-21T03:41:30.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:30.145+0000 7f2f2a7fc640 1 --2- 172.21.15.38:0/2931253120 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2f18078bc0 0x7f2f1807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:30.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:30.145+0000 7f2f2a7fc640 1 -- 172.21.15.38:0/2931253120 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 7336+0+0 (secure 0 0 0) 0x7f2f2c0bae80 con 0x7f2f341f49d0 2024-09-21T03:41:30.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:30.145+0000 7f2f39d17640 1 --2- 172.21.15.38:0/2931253120 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2f18078bc0 0x7f2f1807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:30.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:30.146+0000 7f2f39d17640 1 --2- 172.21.15.38:0/2931253120 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2f18078bc0 0x7f2f1807b0d0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f2f30005fd0 tx=0x7f2f30009500 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:30.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:30.149+0000 7f2f2a7fc640 1 -- 172.21.15.38:0/2931253120 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f2f2c006260 con 0x7f2f341f49d0 2024-09-21T03:41:30.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:30.375+0000 7f2f3bf7b640 1 -- 172.21.15.38:0/2931253120 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2f3415df20 con 0x7f2f341f49d0 2024-09-21T03:41:30.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:30.376+0000 7f2f2a7fc640 1 -- 172.21.15.38:0/2931253120 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 154 v154) ==== 95+0+4392 (secure 0 0 0) 0x7f2f2c082600 con 0x7f2f341f49d0 2024-09-21T03:41:30.378 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:41:30.378 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":154,"btime":"2024-09-21T03:41:29:639761+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6086,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/965972663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":965972663},{"type":"v1","addr":"172.21.15.195:6836","nonce":965972663}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6700,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/3176122334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":3176122334},{"type":"v1","addr":"172.21.15.195:6837","nonce":3176122334}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":8370,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/2916238332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":2916238332},{"type":"v1","addr":"172.21.15.38:6835","nonce":2916238332}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":146}],"filesystems":[{"mdsmap":{"epoch":154,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:40:02.319282+0000","modified":"2024-09-21T03:41:29.639674+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":108,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-21T03:41:30.378 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 154 2024-09-21T03:41:30.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:30.378+0000 7f2f3bf7b640 1 -- 172.21.15.38:0/2931253120 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2f18078bc0 msgr2=0x7f2f1807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:30.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:30.378+0000 7f2f3bf7b640 1 --2- 172.21.15.38:0/2931253120 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2f18078bc0 0x7f2f1807b0d0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f2f30005fd0 tx=0x7f2f30009500 comp rx=0 tx=0).stop 2024-09-21T03:41:30.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:30.378+0000 7f2f3bf7b640 1 -- 172.21.15.38:0/2931253120 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2f341f49d0 msgr2=0x7f2f341f8900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:30.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:30.378+0000 7f2f3bf7b640 1 --2- 172.21.15.38:0/2931253120 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2f341f49d0 0x7f2f341f8900 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7f2f2c02b160 tx=0x7f2f2c02ec40 comp rx=0 tx=0).stop 2024-09-21T03:41:30.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:30.378+0000 7f2f3bf7b640 1 -- 172.21.15.38:0/2931253120 shutdown_connections 2024-09-21T03:41:30.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:30.378+0000 7f2f3bf7b640 1 --2- 172.21.15.38:0/2931253120 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2f341f49d0 0x7f2f341f8900 unknown :-1 s=CLOSED pgs=1504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:30.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:30.378+0000 7f2f3bf7b640 1 --2- 172.21.15.38:0/2931253120 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2f18078bc0 0x7f2f1807b0d0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:30.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:30.378+0000 7f2f3bf7b640 1 --2- 172.21.15.38:0/2931253120 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2f340c98a0 0x7f2f341f83c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:30.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:30.378+0000 7f2f3bf7b640 1 --2- 172.21.15.38:0/2931253120 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2f20005150 0x7f2f341f4490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:30.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:30.378+0000 7f2f3bf7b640 1 -- 172.21.15.38:0/2931253120 >> 172.21.15.38:0/2931253120 conn(0x7f2f340b6b20 msgr2=0x7f2f340cf730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:30.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:30.378+0000 7f2f3bf7b640 1 -- 172.21.15.38:0/2931253120 shutdown_connections 2024-09-21T03:41:30.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:30.378+0000 7f2f3bf7b640 1 -- 172.21.15.38:0/2931253120 wait complete. 2024-09-21T03:41:30.391 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-21T03:41:30.391 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:41:30.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:30.505+0000 7fb760694640 1 -- 172.21.15.38:0/1589542826 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb7580b2e60 msgr2=0x7fb7580b3270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:30.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:30.505+0000 7fb760694640 1 --2- 172.21.15.38:0/1589542826 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb7580b2e60 0x7fb7580b3270 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7fb7580cef90 tx=0x7fb75002aac0 comp rx=0 tx=0).stop 2024-09-21T03:41:30.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:30.505+0000 7fb760694640 1 -- 172.21.15.38:0/1589542826 shutdown_connections 2024-09-21T03:41:30.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:30.506+0000 7fb760694640 1 --2- 172.21.15.38:0/1589542826 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb7580b2e60 0x7fb7580b3270 unknown :-1 s=CLOSED pgs=1505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:30.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:30.506+0000 7fb760694640 1 --2- 172.21.15.38:0/1589542826 >> v2:172.21.15.195:3300/0 conn(0x7fb758162590 0x7fb7581629a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:30.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:30.506+0000 7fb760694640 1 -- 172.21.15.38:0/1589542826 >> 172.21.15.38:0/1589542826 conn(0x7fb7580d4440 msgr2=0x7fb7580d4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:30.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:30.506+0000 7fb760694640 1 -- 172.21.15.38:0/1589542826 shutdown_connections 2024-09-21T03:41:30.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:30.506+0000 7fb760694640 1 -- 172.21.15.38:0/1589542826 wait complete. 2024-09-21T03:41:30.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:30.507+0000 7fb760694640 1 Processor -- start 2024-09-21T03:41:30.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:30.507+0000 7fb760694640 1 -- start start 2024-09-21T03:41:30.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:30.509+0000 7fb760694640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb7580b2e60 0x7fb758153130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:30.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:30.509+0000 7fb760694640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb7580cfe00 0x7fb7581607e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:30.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:30.509+0000 7fb760694640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb758162590 0x7fb758160d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:30.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:30.509+0000 7fb760694640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fb7580b3930 con 0x7fb7580b2e60 2024-09-21T03:41:30.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:30.509+0000 7fb760694640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fb7580b37b0 con 0x7fb7580cfe00 2024-09-21T03:41:30.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:30.509+0000 7fb760694640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fb7580b3ab0 con 0x7fb758162590 2024-09-21T03:41:30.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:30.509+0000 7fb75e430640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb7580b2e60 0x7fb758153130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:30.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:30.509+0000 7fb75e430640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb7580b2e60 0x7fb758153130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:52812/0 (socket says 172.21.15.38:52812) 2024-09-21T03:41:30.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:30.509+0000 7fb75e430640 1 -- 172.21.15.38:0/1345356457 learned_addr learned my addr 172.21.15.38:0/1345356457 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:41:30.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:30.509+0000 7fb75dc2f640 1 --2- 172.21.15.38:0/1345356457 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb7580cfe00 0x7fb7581607e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:30.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:30.509+0000 7fb75ec31640 1 --2- 172.21.15.38:0/1345356457 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb758162590 0x7fb758160d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:30.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:30.509+0000 7fb75e430640 1 -- 172.21.15.38:0/1345356457 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb758162590 msgr2=0x7fb758160d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:30.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:30.509+0000 7fb75e430640 1 --2- 172.21.15.38:0/1345356457 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb758162590 0x7fb758160d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:30.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:30.509+0000 7fb75e430640 1 -- 172.21.15.38:0/1345356457 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb7580cfe00 msgr2=0x7fb7581607e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:30.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:30.509+0000 7fb75e430640 1 --2- 172.21.15.38:0/1345356457 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb7580cfe00 0x7fb7581607e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:30.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:30.509+0000 7fb75e430640 1 -- 172.21.15.38:0/1345356457 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7581e8830 con 0x7fb7580b2e60 2024-09-21T03:41:30.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:30.509+0000 7fb75dc2f640 1 --2- 172.21.15.38:0/1345356457 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb7580cfe00 0x7fb7581607e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:41:30.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:30.509+0000 7fb75e430640 1 --2- 172.21.15.38:0/1345356457 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb7580b2e60 0x7fb758153130 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7fb75400c3a0 tx=0x7fb75400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:30.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:30.509+0000 7fb74effd640 1 -- 172.21.15.38:0/1345356457 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb754017070 con 0x7fb7580b2e60 2024-09-21T03:41:30.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:30.509+0000 7fb74effd640 1 -- 172.21.15.38:0/1345356457 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb7540053c0 con 0x7fb7580b2e60 2024-09-21T03:41:30.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:30.509+0000 7fb760694640 1 -- 172.21.15.38:0/1345356457 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7581e8ad0 con 0x7fb7580b2e60 2024-09-21T03:41:30.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:30.509+0000 7fb74effd640 1 -- 172.21.15.38:0/1345356457 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb754003850 con 0x7fb7580b2e60 2024-09-21T03:41:30.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:30.509+0000 7fb760694640 1 -- 172.21.15.38:0/1345356457 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7581e9090 con 0x7fb7580b2e60 2024-09-21T03:41:30.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:30.510+0000 7fb760694640 1 -- 172.21.15.38:0/1345356457 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7580b3270 con 0x7fb7580b2e60 2024-09-21T03:41:30.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:30.511+0000 7fb74effd640 1 -- 172.21.15.38:0/1345356457 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb75401a070 con 0x7fb7580b2e60 2024-09-21T03:41:30.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:30.512+0000 7fb74effd640 1 --2- 172.21.15.38:0/1345356457 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb73c078bc0 0x7fb73c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:30.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:30.512+0000 7fb74effd640 1 -- 172.21.15.38:0/1345356457 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 7336+0+0 (secure 0 0 0) 0x7fb75409a6b0 con 0x7fb7580b2e60 2024-09-21T03:41:30.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:30.512+0000 7fb75dc2f640 1 --2- 172.21.15.38:0/1345356457 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb73c078bc0 0x7fb73c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:30.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:30.512+0000 7fb75dc2f640 1 --2- 172.21.15.38:0/1345356457 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb73c078bc0 0x7fb73c07b0d0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fb7581565f0 tx=0x7fb748005ef0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:30.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:30.516+0000 7fb74effd640 1 -- 172.21.15.38:0/1345356457 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fb758162ee0 con 0x7fb7580b2e60 2024-09-21T03:41:30.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:30.745+0000 7fb760694640 1 -- 172.21.15.38:0/1345356457 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb7581e8610 con 0x7fb7580b2e60 2024-09-21T03:41:30.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:30.745+0000 7fb74effd640 1 -- 172.21.15.38:0/1345356457 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 154 v154) ==== 95+0+4392 (secure 0 0 0) 0x7fb754062520 con 0x7fb7580b2e60 2024-09-21T03:41:30.747 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:41:30.747 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":154,"btime":"2024-09-21T03:41:29:639761+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6086,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/965972663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":965972663},{"type":"v1","addr":"172.21.15.195:6836","nonce":965972663}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6700,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/3176122334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":3176122334},{"type":"v1","addr":"172.21.15.195:6837","nonce":3176122334}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":8370,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/2916238332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":2916238332},{"type":"v1","addr":"172.21.15.38:6835","nonce":2916238332}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":146}],"filesystems":[{"mdsmap":{"epoch":154,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:40:02.319282+0000","modified":"2024-09-21T03:41:29.639674+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":108,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-21T03:41:30.748 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 154 2024-09-21T03:41:30.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:30.747+0000 7fb760694640 1 -- 172.21.15.38:0/1345356457 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb73c078bc0 msgr2=0x7fb73c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:30.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:30.747+0000 7fb760694640 1 --2- 172.21.15.38:0/1345356457 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb73c078bc0 0x7fb73c07b0d0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fb7581565f0 tx=0x7fb748005ef0 comp rx=0 tx=0).stop 2024-09-21T03:41:30.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:30.747+0000 7fb760694640 1 -- 172.21.15.38:0/1345356457 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb7580b2e60 msgr2=0x7fb758153130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:30.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:30.747+0000 7fb760694640 1 --2- 172.21.15.38:0/1345356457 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb7580b2e60 0x7fb758153130 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7fb75400c3a0 tx=0x7fb75400c860 comp rx=0 tx=0).stop 2024-09-21T03:41:30.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:30.748+0000 7fb760694640 1 -- 172.21.15.38:0/1345356457 shutdown_connections 2024-09-21T03:41:30.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:30.748+0000 7fb760694640 1 --2- 172.21.15.38:0/1345356457 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb73c078bc0 0x7fb73c07b0d0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:30.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:30.748+0000 7fb760694640 1 --2- 172.21.15.38:0/1345356457 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb758162590 0x7fb758160d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:30.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:30.748+0000 7fb760694640 1 --2- 172.21.15.38:0/1345356457 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb7580cfe00 0x7fb7581607e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:30.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:30.748+0000 7fb760694640 1 --2- 172.21.15.38:0/1345356457 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb7580b2e60 0x7fb758153130 unknown :-1 s=CLOSED pgs=1506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:30.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:30.748+0000 7fb760694640 1 -- 172.21.15.38:0/1345356457 >> 172.21.15.38:0/1345356457 conn(0x7fb7580d4440 msgr2=0x7fb7580d4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:30.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:30.748+0000 7fb760694640 1 -- 172.21.15.38:0/1345356457 shutdown_connections 2024-09-21T03:41:30.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:30.748+0000 7fb760694640 1 -- 172.21.15.38:0/1345356457 wait complete. 2024-09-21T03:41:30.761 INFO:tasks.cephfs.test_damage:Result: Mutation 'Delete mds0_sessionmap' led to DAMAGED state 2024-09-21T03:41:30.761 INFO:tasks.cephfs.test_damage:Applying mutation 'Delete 400.00000000' 2024-09-21T03:41:30.761 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-21T03:41:30.761 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:41:30.778 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056795,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2354 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T03:41:30.778 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi038.front.sepia.ceph.com 2024-09-21T03:41:30.778 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:41:30.778 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T03:41:30.833 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T03:41:30.834 INFO:teuthology.orchestra.run.smithi038.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-09-21T03:41:30.834 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-21T03:41:30.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:30.992+0000 7eff7dc1d640 1 -- 172.21.15.38:0/1083418841 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7eff7806e500 msgr2=0x7eff780cef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:30.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:30.992+0000 7eff7dc1d640 1 --2- 172.21.15.38:0/1083418841 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7eff7806e500 0x7eff780cef00 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7eff70005fd0 tx=0x7eff7002a8b0 comp rx=0 tx=0).stop 2024-09-21T03:41:30.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:30.992+0000 7eff7dc1d640 1 -- 172.21.15.38:0/1083418841 shutdown_connections 2024-09-21T03:41:30.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:30.992+0000 7eff7dc1d640 1 --2- 172.21.15.38:0/1083418841 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7eff7806e500 0x7eff780cef00 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:30.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:30.992+0000 7eff7dc1d640 1 --2- 172.21.15.38:0/1083418841 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7eff78156a50 0x7eff78074620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:30.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:30.992+0000 7eff7dc1d640 1 -- 172.21.15.38:0/1083418841 >> 172.21.15.38:0/1083418841 conn(0x7eff780b1b90 msgr2=0x7eff780b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:30.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:30.992+0000 7eff7dc1d640 1 -- 172.21.15.38:0/1083418841 shutdown_connections 2024-09-21T03:41:30.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:30.992+0000 7eff7dc1d640 1 -- 172.21.15.38:0/1083418841 wait complete. 2024-09-21T03:41:30.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:30.994+0000 7eff7dc1d640 1 Processor -- start 2024-09-21T03:41:30.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:30.994+0000 7eff7dc1d640 1 -- start start 2024-09-21T03:41:30.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:30.995+0000 7eff7dc1d640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7eff78074b60 0x7eff781562b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:30.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:30.995+0000 7eff7dc1d640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7eff78156a50 0x7eff780bb420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:30.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:30.995+0000 7eff7dc1d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7eff780ba860 0x7eff780bac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:30.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:30.995+0000 7eff7dc1d640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7eff780cf7d0 con 0x7eff780ba860 2024-09-21T03:41:30.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:30.995+0000 7eff7dc1d640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7eff780cf650 con 0x7eff78074b60 2024-09-21T03:41:30.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:30.995+0000 7eff7dc1d640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7eff780cf950 con 0x7eff78156a50 2024-09-21T03:41:30.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:30.996+0000 7eff77fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7eff780ba860 0x7eff780bac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:30.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:30.996+0000 7eff77fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7eff780ba860 0x7eff780bac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:52828/0 (socket says 172.21.15.38:52828) 2024-09-21T03:41:30.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:30.996+0000 7eff77fff640 1 -- 172.21.15.38:0/2489273947 learned_addr learned my addr 172.21.15.38:0/2489273947 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:41:30.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:30.996+0000 7eff777fe640 1 --2- 172.21.15.38:0/2489273947 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7eff78074b60 0x7eff781562b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:30.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:30.996+0000 7eff6effd640 1 --2- 172.21.15.38:0/2489273947 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7eff78156a50 0x7eff780bb420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:30.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:30.996+0000 7eff77fff640 1 -- 172.21.15.38:0/2489273947 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7eff78156a50 msgr2=0x7eff780bb420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:30.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:30.996+0000 7eff77fff640 1 --2- 172.21.15.38:0/2489273947 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7eff78156a50 0x7eff780bb420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:30.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:30.996+0000 7eff77fff640 1 -- 172.21.15.38:0/2489273947 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7eff78074b60 msgr2=0x7eff781562b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:30.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:30.996+0000 7eff77fff640 1 --2- 172.21.15.38:0/2489273947 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7eff78074b60 0x7eff781562b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:31.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:30.996+0000 7eff77fff640 1 -- 172.21.15.38:0/2489273947 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff780bb1b0 con 0x7eff780ba860 2024-09-21T03:41:31.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:30.996+0000 7eff77fff640 1 --2- 172.21.15.38:0/2489273947 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7eff780ba860 0x7eff780bac70 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7eff7002aea0 tx=0x7eff700316f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:31.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:30.996+0000 7eff74ff9640 1 -- 172.21.15.38:0/2489273947 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff70031da0 con 0x7eff780ba860 2024-09-21T03:41:31.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:30.996+0000 7eff74ff9640 1 -- 172.21.15.38:0/2489273947 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7eff70003a60 con 0x7eff780ba860 2024-09-21T03:41:31.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:30.996+0000 7eff7dc1d640 1 -- 172.21.15.38:0/2489273947 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff780cb6c0 con 0x7eff780ba860 2024-09-21T03:41:31.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:30.996+0000 7eff74ff9640 1 -- 172.21.15.38:0/2489273947 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff700330d0 con 0x7eff780ba860 2024-09-21T03:41:31.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:30.996+0000 7eff7dc1d640 1 -- 172.21.15.38:0/2489273947 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff780cbc80 con 0x7eff780ba860 2024-09-21T03:41:31.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:30.998+0000 7eff7dc1d640 1 -- 172.21.15.38:0/2489273947 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff780cef00 con 0x7eff780ba860 2024-09-21T03:41:31.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:30.998+0000 7eff74ff9640 1 -- 172.21.15.38:0/2489273947 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7eff70033270 con 0x7eff780ba860 2024-09-21T03:41:31.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:30.999+0000 7eff74ff9640 1 --2- 172.21.15.38:0/2489273947 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7eff58078bc0 0x7eff5807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:31.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:30.999+0000 7eff74ff9640 1 -- 172.21.15.38:0/2489273947 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 7336+0+0 (secure 0 0 0) 0x7eff700bc660 con 0x7eff780ba860 2024-09-21T03:41:31.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:30.999+0000 7eff777fe640 1 --2- 172.21.15.38:0/2489273947 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7eff58078bc0 0x7eff5807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:31.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:31.000+0000 7eff777fe640 1 --2- 172.21.15.38:0/2489273947 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7eff58078bc0 0x7eff5807b0d0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7eff680095f0 tx=0x7eff68005f90 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:31.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:31.004+0000 7eff74ff9640 1 -- 172.21.15.38:0/2489273947 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7eff780bb1b0 con 0x7eff780ba860 2024-09-21T03:41:31.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:31.229+0000 7eff7dc1d640 1 -- 172.21.15.38:0/2489273947 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7eff780c68f0 con 0x7eff780ba860 2024-09-21T03:41:31.265 DEBUG:teuthology.orchestra.run.smithi038:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T03:41:31.268 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T03:41:31.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:31.637+0000 7eff74ff9640 1 -- 172.21.15.38:0/2489273947 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v155) ==== 161+0+0 (secure 0 0 0) 0x7eff7003f030 con 0x7eff780ba860 2024-09-21T03:41:31.639 INFO:teuthology.orchestra.run.smithi038.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-21T03:41:31.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:31.639+0000 7eff7dc1d640 1 -- 172.21.15.38:0/2489273947 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7eff58078bc0 msgr2=0x7eff5807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:31.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:31.639+0000 7eff7dc1d640 1 --2- 172.21.15.38:0/2489273947 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7eff58078bc0 0x7eff5807b0d0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7eff680095f0 tx=0x7eff68005f90 comp rx=0 tx=0).stop 2024-09-21T03:41:31.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:31.639+0000 7eff7dc1d640 1 -- 172.21.15.38:0/2489273947 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7eff780ba860 msgr2=0x7eff780bac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:31.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:31.639+0000 7eff7dc1d640 1 --2- 172.21.15.38:0/2489273947 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7eff780ba860 0x7eff780bac70 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7eff7002aea0 tx=0x7eff700316f0 comp rx=0 tx=0).stop 2024-09-21T03:41:31.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:31.639+0000 7eff7dc1d640 1 -- 172.21.15.38:0/2489273947 shutdown_connections 2024-09-21T03:41:31.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:31.639+0000 7eff7dc1d640 1 --2- 172.21.15.38:0/2489273947 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7eff780ba860 0x7eff780bac70 unknown :-1 s=CLOSED pgs=1507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:31.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:31.639+0000 7eff7dc1d640 1 --2- 172.21.15.38:0/2489273947 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7eff58078bc0 0x7eff5807b0d0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:31.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:31.639+0000 7eff7dc1d640 1 --2- 172.21.15.38:0/2489273947 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7eff78156a50 0x7eff780bb420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:31.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:31.639+0000 7eff7dc1d640 1 --2- 172.21.15.38:0/2489273947 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7eff78074b60 0x7eff781562b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:31.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:31.639+0000 7eff7dc1d640 1 -- 172.21.15.38:0/2489273947 >> 172.21.15.38:0/2489273947 conn(0x7eff780b1b90 msgr2=0x7eff780c61e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:31.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:31.639+0000 7eff7dc1d640 1 -- 172.21.15.38:0/2489273947 shutdown_connections 2024-09-21T03:41:31.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:31.639+0000 7eff7dc1d640 1 -- 172.21.15.38:0/2489273947 wait complete. 2024-09-21T03:41:31.653 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-09-21T03:41:31.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:31.769+0000 7f7f78e15640 1 -- 172.21.15.38:0/2644943209 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7f740d1120 msgr2=0x7f7f740ceb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:31.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:31.769+0000 7f7f78e15640 1 --2- 172.21.15.38:0/2644943209 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7f740d1120 0x7f7f740ceb90 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f7f6c005fd0 tx=0x7f7f6c02aa10 comp rx=0 tx=0).stop 2024-09-21T03:41:31.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:31.769+0000 7f7f78e15640 1 -- 172.21.15.38:0/2644943209 shutdown_connections 2024-09-21T03:41:31.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:31.769+0000 7f7f78e15640 1 --2- 172.21.15.38:0/2644943209 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7f740d1120 0x7f7f740ceb90 unknown :-1 s=CLOSED pgs=1508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:31.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:31.769+0000 7f7f78e15640 1 --2- 172.21.15.38:0/2644943209 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7f740d0170 0x7f7f740d0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:31.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:31.769+0000 7f7f78e15640 1 -- 172.21.15.38:0/2644943209 >> 172.21.15.38:0/2644943209 conn(0x7f7f7409d830 msgr2=0x7f7f740b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:31.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:31.769+0000 7f7f78e15640 1 -- 172.21.15.38:0/2644943209 shutdown_connections 2024-09-21T03:41:31.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:31.769+0000 7f7f78e15640 1 -- 172.21.15.38:0/2644943209 wait complete. 2024-09-21T03:41:31.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:31.771+0000 7f7f78e15640 1 Processor -- start 2024-09-21T03:41:31.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:31.771+0000 7f7f78e15640 1 -- start start 2024-09-21T03:41:31.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:31.772+0000 7f7f78e15640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7f740d0170 0x7f7f740cb610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:31.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:31.772+0000 7f7f78e15640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7f740d1120 0x7f7f7406e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:31.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:31.772+0000 7f7f78e15640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7f7415ee80 0x7f7f7406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:31.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:31.772+0000 7f7f78e15640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f7f740b6700 con 0x7f7f740d0170 2024-09-21T03:41:31.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:31.772+0000 7f7f78e15640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f7f740b6580 con 0x7f7f7415ee80 2024-09-21T03:41:31.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:31.772+0000 7f7f78e15640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f7f740b6880 con 0x7f7f740d1120 2024-09-21T03:41:31.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:31.772+0000 7f7f7259c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7f740d0170 0x7f7f740cb610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:31.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:31.772+0000 7f7f7259c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7f740d0170 0x7f7f740cb610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:52838/0 (socket says 172.21.15.38:52838) 2024-09-21T03:41:31.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:31.772+0000 7f7f7259c640 1 -- 172.21.15.38:0/177010924 learned_addr learned my addr 172.21.15.38:0/177010924 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:41:31.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:31.772+0000 7f7f71d9b640 1 --2- 172.21.15.38:0/177010924 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7f740d1120 0x7f7f7406e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:31.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:31.772+0000 7f7f72d9d640 1 --2- 172.21.15.38:0/177010924 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7f7415ee80 0x7f7f7406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:31.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:31.773+0000 7f7f7259c640 1 -- 172.21.15.38:0/177010924 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7f740d1120 msgr2=0x7f7f7406e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:31.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:31.773+0000 7f7f7259c640 1 --2- 172.21.15.38:0/177010924 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7f740d1120 0x7f7f7406e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:31.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:31.773+0000 7f7f7259c640 1 -- 172.21.15.38:0/177010924 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7f7415ee80 msgr2=0x7f7f7406e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:31.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:31.773+0000 7f7f7259c640 1 --2- 172.21.15.38:0/177010924 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7f7415ee80 0x7f7f7406e9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:31.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:31.773+0000 7f7f7259c640 1 -- 172.21.15.38:0/177010924 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f741488d0 con 0x7f7f740d0170 2024-09-21T03:41:31.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:31.773+0000 7f7f7259c640 1 --2- 172.21.15.38:0/177010924 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7f740d0170 0x7f7f740cb610 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7f7f6400c3a0 tx=0x7f7f6400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:31.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:31.773+0000 7f7f62ffd640 1 -- 172.21.15.38:0/177010924 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f64017070 con 0x7f7f740d0170 2024-09-21T03:41:31.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:31.773+0000 7f7f78e15640 1 -- 172.21.15.38:0/177010924 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f740cbd20 con 0x7f7f740d0170 2024-09-21T03:41:31.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:31.773+0000 7f7f62ffd640 1 -- 172.21.15.38:0/177010924 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7f640053c0 con 0x7f7f740d0170 2024-09-21T03:41:31.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:31.773+0000 7f7f62ffd640 1 -- 172.21.15.38:0/177010924 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f64003850 con 0x7f7f740d0170 2024-09-21T03:41:31.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:31.773+0000 7f7f78e15640 1 -- 172.21.15.38:0/177010924 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f74148e70 con 0x7f7f740d0170 2024-09-21T03:41:31.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:31.774+0000 7f7f78e15640 1 -- 172.21.15.38:0/177010924 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f740b6da0 con 0x7f7f740d0170 2024-09-21T03:41:31.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:31.775+0000 7f7f62ffd640 1 -- 172.21.15.38:0/177010924 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f7f6401a070 con 0x7f7f740d0170 2024-09-21T03:41:31.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:31.776+0000 7f7f62ffd640 1 --2- 172.21.15.38:0/177010924 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7f54078bc0 0x7f7f5407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:31.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:31.776+0000 7f7f62ffd640 1 -- 172.21.15.38:0/177010924 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 7336+0+0 (secure 0 0 0) 0x7f7f6409a6f0 con 0x7f7f740d0170 2024-09-21T03:41:31.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:31.776+0000 7f7f71d9b640 1 --2- 172.21.15.38:0/177010924 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7f54078bc0 0x7f7f5407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:31.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:31.776+0000 7f7f71d9b640 1 --2- 172.21.15.38:0/177010924 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7f54078bc0 0x7f7f5407b0d0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f7f74146780 tx=0x7f7f68005f30 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:31.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:31.780+0000 7f7f62ffd640 1 -- 172.21.15.38:0/177010924 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f7f741592d0 con 0x7f7f740d0170 2024-09-21T03:41:32.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.042+0000 7f7f78e15640 1 -- 172.21.15.38:0/177010924 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7f7f740cb1b0 con 0x7f7f740d0170 2024-09-21T03:41:32.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.645+0000 7f7f62ffd640 1 -- 172.21.15.38:0/177010924 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v156) ==== 101+0+0 (secure 0 0 0) 0x7f7f64061e00 con 0x7f7f740d0170 2024-09-21T03:41:32.647 INFO:teuthology.orchestra.run.smithi038.stderr:repaired: restoring rank 6:0 2024-09-21T03:41:32.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.648+0000 7f7f78e15640 1 -- 172.21.15.38:0/177010924 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7f54078bc0 msgr2=0x7f7f5407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:32.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.648+0000 7f7f78e15640 1 --2- 172.21.15.38:0/177010924 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7f54078bc0 0x7f7f5407b0d0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f7f74146780 tx=0x7f7f68005f30 comp rx=0 tx=0).stop 2024-09-21T03:41:32.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.648+0000 7f7f78e15640 1 -- 172.21.15.38:0/177010924 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7f740d0170 msgr2=0x7f7f740cb610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:32.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.648+0000 7f7f78e15640 1 --2- 172.21.15.38:0/177010924 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7f740d0170 0x7f7f740cb610 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7f7f6400c3a0 tx=0x7f7f6400c860 comp rx=0 tx=0).stop 2024-09-21T03:41:32.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.648+0000 7f7f78e15640 1 -- 172.21.15.38:0/177010924 shutdown_connections 2024-09-21T03:41:32.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.648+0000 7f7f78e15640 1 --2- 172.21.15.38:0/177010924 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7f7415ee80 0x7f7f7406e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:32.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.648+0000 7f7f78e15640 1 --2- 172.21.15.38:0/177010924 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7f54078bc0 0x7f7f5407b0d0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:32.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.648+0000 7f7f78e15640 1 --2- 172.21.15.38:0/177010924 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7f740d1120 0x7f7f7406e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:32.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.648+0000 7f7f78e15640 1 --2- 172.21.15.38:0/177010924 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7f740d0170 0x7f7f740cb610 unknown :-1 s=CLOSED pgs=1509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:32.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.648+0000 7f7f78e15640 1 -- 172.21.15.38:0/177010924 >> 172.21.15.38:0/177010924 conn(0x7f7f7409d830 msgr2=0x7f7f740b5490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:32.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.648+0000 7f7f78e15640 1 -- 172.21.15.38:0/177010924 shutdown_connections 2024-09-21T03:41:32.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.648+0000 7f7f78e15640 1 -- 172.21.15.38:0/177010924 wait complete. 2024-09-21T03:41:32.663 DEBUG:teuthology.orchestra.run.smithi038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-09-21T03:41:32.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.721+0000 7ff7c201cf40 1 Processor -- start 2024-09-21T03:41:32.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.721+0000 7ff7c201cf40 1 -- start start 2024-09-21T03:41:32.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.722+0000 7ff7c201cf40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560ade655d00 0x560ade70e200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:32.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.722+0000 7ff7c201cf40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560ade681e30 0x560ade70d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:32.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.722+0000 7ff7c201cf40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560ade710950 0x560ade726950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:32.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.722+0000 7ff7c201cf40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x560ade5a3d60 con 0x560ade681e30 2024-09-21T03:41:32.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.722+0000 7ff7c201cf40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x560ade602430 con 0x560ade710950 2024-09-21T03:41:32.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.722+0000 7ff7c201cf40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x560ade69d7b0 con 0x560ade655d00 2024-09-21T03:41:32.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.722+0000 7ff7ba666640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560ade681e30 0x560ade70d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:32.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.722+0000 7ff7ba666640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560ade681e30 0x560ade70d080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54170/0 (socket says 172.21.15.38:54170) 2024-09-21T03:41:32.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.722+0000 7ff7ba666640 1 -- 172.21.15.38:0/994982632 learned_addr learned my addr 172.21.15.38:0/994982632 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:41:32.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.722+0000 7ff7bae67640 1 --2- 172.21.15.38:0/994982632 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560ade655d00 0x560ade70e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:32.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.722+0000 7ff7bb668640 1 --2- 172.21.15.38:0/994982632 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560ade710950 0x560ade726950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:32.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.723+0000 7ff7ba666640 1 -- 172.21.15.38:0/994982632 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560ade655d00 msgr2=0x560ade70e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:32.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.723+0000 7ff7ba666640 1 --2- 172.21.15.38:0/994982632 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560ade655d00 0x560ade70e200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:32.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.723+0000 7ff7ba666640 1 -- 172.21.15.38:0/994982632 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560ade710950 msgr2=0x560ade726950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:32.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.723+0000 7ff7ba666640 1 --2- 172.21.15.38:0/994982632 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560ade710950 0x560ade726950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:32.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.723+0000 7ff7ba666640 1 -- 172.21.15.38:0/994982632 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ade651a80 con 0x560ade681e30 2024-09-21T03:41:32.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.723+0000 7ff7ba666640 1 --2- 172.21.15.38:0/994982632 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560ade681e30 0x560ade70d080 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7ff7b0001e60 tx=0x7ff7b000b730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0899f73f463778 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:32.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.723+0000 7ff7b9664640 1 -- 172.21.15.38:0/994982632 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7b000e830 con 0x560ade681e30 2024-09-21T03:41:32.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.723+0000 7ff7b9664640 1 -- 172.21.15.38:0/994982632 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff7b0006050 con 0x560ade681e30 2024-09-21T03:41:32.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.723+0000 7ff7b9664640 1 -- 172.21.15.38:0/994982632 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7b000ed80 con 0x560ade681e30 2024-09-21T03:41:32.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.724+0000 7ff7c201cf40 1 -- 172.21.15.38:0/994982632 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560ade681e30 msgr2=0x560ade70d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:32.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.724+0000 7ff7c201cf40 1 --2- 172.21.15.38:0/994982632 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560ade681e30 0x560ade70d080 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7ff7b0001e60 tx=0x7ff7b000b730 comp rx=0 tx=0).stop 2024-09-21T03:41:32.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.724+0000 7ff7c201cf40 1 -- 172.21.15.38:0/994982632 shutdown_connections 2024-09-21T03:41:32.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.724+0000 7ff7c201cf40 1 --2- 172.21.15.38:0/994982632 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560ade710950 0x560ade726950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:32.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.724+0000 7ff7c201cf40 1 --2- 172.21.15.38:0/994982632 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560ade681e30 0x560ade70d080 unknown :-1 s=CLOSED pgs=1511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:32.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.724+0000 7ff7c201cf40 1 --2- 172.21.15.38:0/994982632 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560ade655d00 0x560ade70e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:32.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.724+0000 7ff7c201cf40 1 -- 172.21.15.38:0/994982632 >> 172.21.15.38:0/994982632 conn(0x560ade574ac0 msgr2=0x560ade653a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:32.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.724+0000 7ff7c201cf40 1 -- 172.21.15.38:0/994982632 shutdown_connections 2024-09-21T03:41:32.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.724+0000 7ff7c201cf40 1 -- 172.21.15.38:0/994982632 wait complete. 2024-09-21T03:41:32.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.725+0000 7ff7c201cf40 1 Processor -- start 2024-09-21T03:41:32.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.725+0000 7ff7c201cf40 1 -- start start 2024-09-21T03:41:32.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.726+0000 7ff7c201cf40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560ade655d00 0x560ade7280f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:32.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.726+0000 7ff7c201cf40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560ade681e30 0x560ade7c3e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:32.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.726+0000 7ff7c201cf40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560ade710950 0x560ade7c0d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:32.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.726+0000 7ff7c201cf40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x560ade651d60 con 0x560ade710950 2024-09-21T03:41:32.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.726+0000 7ff7c201cf40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x560ade652040 con 0x560ade655d00 2024-09-21T03:41:32.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.726+0000 7ff7c201cf40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x560ade652320 con 0x560ade681e30 2024-09-21T03:41:32.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.726+0000 7ff7bb668640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560ade710950 0x560ade7c0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:32.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.726+0000 7ff7bae67640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560ade655d00 0x560ade7280f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:32.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.726+0000 7ff7bb668640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560ade710950 0x560ade7c0d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54182/0 (socket says 172.21.15.38:54182) 2024-09-21T03:41:32.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.726+0000 7ff7bb668640 1 -- 172.21.15.38:0/1449055308 learned_addr learned my addr 172.21.15.38:0/1449055308 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:41:32.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.726+0000 7ff7ba666640 1 --2- 172.21.15.38:0/1449055308 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560ade681e30 0x560ade7c3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:32.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.726+0000 7ff7bb668640 1 -- 172.21.15.38:0/1449055308 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560ade681e30 msgr2=0x560ade7c3e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:32.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.726+0000 7ff7bb668640 1 --2- 172.21.15.38:0/1449055308 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560ade681e30 0x560ade7c3e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:32.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.726+0000 7ff7bb668640 1 -- 172.21.15.38:0/1449055308 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560ade655d00 msgr2=0x560ade7280f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:32.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.726+0000 7ff7bb668640 1 --2- 172.21.15.38:0/1449055308 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560ade655d00 0x560ade7280f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:32.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.726+0000 7ff7bb668640 1 -- 172.21.15.38:0/1449055308 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ade722ae0 con 0x560ade710950 2024-09-21T03:41:32.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.727+0000 7ff7bb668640 1 --2- 172.21.15.38:0/1449055308 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560ade710950 0x560ade7c0d20 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7ff7b4046470 tx=0x7ff7b4004570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:32.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.727+0000 7ff7a3fff640 1 -- 172.21.15.38:0/1449055308 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7b404d690 con 0x560ade710950 2024-09-21T03:41:32.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.727+0000 7ff7a3fff640 1 -- 172.21.15.38:0/1449055308 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff7b4004020 con 0x560ade710950 2024-09-21T03:41:32.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.727+0000 7ff7c201cf40 1 -- 172.21.15.38:0/1449055308 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x560ade67e050 con 0x560ade710950 2024-09-21T03:41:32.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.727+0000 7ff7a3fff640 1 -- 172.21.15.38:0/1449055308 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7b4060440 con 0x560ade710950 2024-09-21T03:41:32.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.727+0000 7ff7c201cf40 1 -- 172.21.15.38:0/1449055308 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ade7c45c0 con 0x560ade710950 2024-09-21T03:41:32.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.729+0000 7ff7a3fff640 1 -- 172.21.15.38:0/1449055308 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ff7b405e040 con 0x560ade710950 2024-09-21T03:41:32.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.729+0000 7ff7a3fff640 1 --2- 172.21.15.38:0/1449055308 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff7a8077570 0x7ff7a8079a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:32.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.729+0000 7ff7a3fff640 1 -- 172.21.15.38:0/1449055308 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 7336+0+0 (secure 0 0 0) 0x7ff7b40de3b0 con 0x560ade710950 2024-09-21T03:41:32.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.729+0000 7ff7bae67640 1 --2- 172.21.15.38:0/1449055308 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff7a8077570 0x7ff7a8079a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:32.733 INFO:teuthology.orchestra.run.smithi038.stdout:Importing pool 2024-09-21T03:41:32.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.730+0000 7ff7c201cf40 1 --2- 172.21.15.38:0/1449055308 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x560ade7b2630 0x560ade70ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:32.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.730+0000 7ff7c201cf40 1 -- 172.21.15.38:0/1449055308 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x560ade709d30 con 0x560ade7b2630 2024-09-21T03:41:32.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.730+0000 7ff7ba666640 1 --2- 172.21.15.38:0/1449055308 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x560ade7b2630 0x560ade70ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:32.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.730+0000 7ff7bae67640 1 --2- 172.21.15.38:0/1449055308 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff7a8077570 0x7ff7a8079a80 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7ff7ac00c780 tx=0x7ff7ac00f040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:32.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.730+0000 7ff7ba666640 1 --2- 172.21.15.38:0/1449055308 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x560ade7b2630 0x560ade70ecc0 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:32.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.732+0000 7ff7ba666640 1 -- 172.21.15.38:0/1449055308 <== osd.7 v2:172.21.15.195:6818/1606106342 1 ==== osd_op_reply(1 601.00000000 [create] v108'350 uv324 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ff7b0014070 con 0x560ade7b2630 2024-09-21T03:41:32.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.732+0000 7ff7c201cf40 1 -- 172.21.15.38:0/1449055308 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e108) -- 0x560ade70f200 con 0x560ade7b2630 2024-09-21T03:41:32.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.733+0000 7ff7ba666640 1 -- 172.21.15.38:0/1449055308 <== osd.7 v2:172.21.15.195:6818/1606106342 2 ==== osd_op_reply(2 601.00000000 [delete] v108'351 uv351 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff7b0014070 con 0x560ade7b2630 2024-09-21T03:41:32.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.733+0000 7ff7c201cf40 1 -- 172.21.15.38:0/1449055308 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x560ade7ccb30 con 0x560ade7b2630 2024-09-21T03:41:32.736 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-09-21T03:41:32.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.735+0000 7ff7ba666640 1 -- 172.21.15.38:0/1449055308 <== osd.7 v2:172.21.15.195:6818/1606106342 3 ==== osd_op_reply(3 601.00000000 [create] v108'352 uv352 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff7b0014070 con 0x560ade7b2630 2024-09-21T03:41:32.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.735+0000 7ff7c201cf40 1 -- 172.21.15.38:0/1449055308 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x560ade7cdee0 con 0x560ade7b2630 2024-09-21T03:41:32.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.736+0000 7ff7ba666640 1 -- 172.21.15.38:0/1449055308 <== osd.7 v2:172.21.15.195:6818/1606106342 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v108'353 uv353 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff7b0014070 con 0x560ade7b2630 2024-09-21T03:41:32.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.736+0000 7ff7c201cf40 1 -- 172.21.15.38:0/1449055308 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x560ade7ce280 con 0x560ade7b2630 2024-09-21T03:41:32.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.737+0000 7ff7ba666640 1 -- 172.21.15.38:0/1449055308 <== osd.7 v2:172.21.15.195:6818/1606106342 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v108'354 uv354 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff7b0014070 con 0x560ade7b2630 2024-09-21T03:41:32.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.737+0000 7ff7c201cf40 1 -- 172.21.15.38:0/1449055308 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x560ade7cced0 con 0x560ade7b2630 2024-09-21T03:41:32.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.738+0000 7ff7ba666640 1 -- 172.21.15.38:0/1449055308 <== osd.7 v2:172.21.15.195:6818/1606106342 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v108'355 uv355 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff7b0014070 con 0x560ade7b2630 2024-09-21T03:41:32.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.738+0000 7ff7c201cf40 1 -- 172.21.15.38:0/1449055308 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x560ade7cd270 con 0x560ade7b2630 2024-09-21T03:41:32.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.739+0000 7ff7ba666640 1 -- 172.21.15.38:0/1449055308 <== osd.7 v2:172.21.15.195:6818/1606106342 7 ==== osd_op_reply(7 602.00000000 [create] v108'356 uv330 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ff7b0014070 con 0x560ade7b2630 2024-09-21T03:41:32.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.740+0000 7ff7c201cf40 1 -- 172.21.15.38:0/1449055308 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e108) -- 0x560ade7cd270 con 0x560ade7b2630 2024-09-21T03:41:32.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.741+0000 7ff7ba666640 1 -- 172.21.15.38:0/1449055308 <== osd.7 v2:172.21.15.195:6818/1606106342 8 ==== osd_op_reply(8 602.00000000 [delete] v108'357 uv357 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff7b0014070 con 0x560ade7b2630 2024-09-21T03:41:32.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.741+0000 7ff7c201cf40 1 -- 172.21.15.38:0/1449055308 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x560ade7cd270 con 0x560ade7b2630 2024-09-21T03:41:32.744 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-09-21T03:41:32.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.742+0000 7ff7ba666640 1 -- 172.21.15.38:0/1449055308 <== osd.7 v2:172.21.15.195:6818/1606106342 9 ==== osd_op_reply(9 602.00000000 [create] v108'358 uv358 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff7b0014070 con 0x560ade7b2630 2024-09-21T03:41:32.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.742+0000 7ff7c201cf40 1 -- 172.21.15.38:0/1449055308 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x560ade7cd270 con 0x560ade7b2630 2024-09-21T03:41:32.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.743+0000 7ff7ba666640 1 -- 172.21.15.38:0/1449055308 <== osd.7 v2:172.21.15.195:6818/1606106342 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v108'359 uv359 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff7b0014070 con 0x560ade7b2630 2024-09-21T03:41:32.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.743+0000 7ff7c201cf40 1 -- 172.21.15.38:0/1449055308 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x560ade7cd270 con 0x560ade7b2630 2024-09-21T03:41:32.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.745+0000 7ff7ba666640 1 -- 172.21.15.38:0/1449055308 <== osd.7 v2:172.21.15.195:6818/1606106342 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v108'360 uv360 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff7b0014070 con 0x560ade7b2630 2024-09-21T03:41:32.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.745+0000 7ff7c201cf40 1 -- 172.21.15.38:0/1449055308 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x560ade7cd270 con 0x560ade7b2630 2024-09-21T03:41:32.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.746+0000 7ff7ba666640 1 -- 172.21.15.38:0/1449055308 <== osd.7 v2:172.21.15.195:6818/1606106342 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v108'361 uv361 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff7b0014070 con 0x560ade7b2630 2024-09-21T03:41:32.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.746+0000 7ff7c201cf40 1 -- 172.21.15.38:0/1449055308 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:13 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x560ade7cd270 con 0x560ade7b2630 2024-09-21T03:41:32.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.748+0000 7ff7ba666640 1 -- 172.21.15.38:0/1449055308 <== osd.7 v2:172.21.15.195:6818/1606106342 13 ==== osd_op_reply(13 10000000000.00000000 [create] v108'362 uv337 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7ff7b0014070 con 0x560ade7b2630 2024-09-21T03:41:32.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.748+0000 7ff7c201cf40 1 -- 172.21.15.38:0/1449055308 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:14 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e108) -- 0x560ade7cd270 con 0x560ade7b2630 2024-09-21T03:41:32.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.749+0000 7ff7ba666640 1 -- 172.21.15.38:0/1449055308 <== osd.7 v2:172.21.15.195:6818/1606106342 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v108'363 uv363 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7ff7b0014070 con 0x560ade7b2630 2024-09-21T03:41:32.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.749+0000 7ff7c201cf40 1 -- 172.21.15.38:0/1449055308 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:15 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x560ade7cd270 con 0x560ade7b2630 2024-09-21T03:41:32.752 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-09-21T03:41:32.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.750+0000 7ff7ba666640 1 -- 172.21.15.38:0/1449055308 <== osd.7 v2:172.21.15.195:6818/1606106342 15 ==== osd_op_reply(15 10000000000.00000000 [create] v108'364 uv364 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7ff7b0014070 con 0x560ade7b2630 2024-09-21T03:41:32.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.750+0000 7ff7c201cf40 1 -- 172.21.15.38:0/1449055308 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:16 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x560ade7cd270 con 0x560ade7b2630 2024-09-21T03:41:32.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.751+0000 7ff7ba666640 1 -- 172.21.15.38:0/1449055308 <== osd.7 v2:172.21.15.195:6818/1606106342 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v108'365 uv365 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7ff7b0014070 con 0x560ade7b2630 2024-09-21T03:41:32.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.752+0000 7ff7c201cf40 1 -- 172.21.15.38:0/1449055308 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:17 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x560ade7cd270 con 0x560ade7b2630 2024-09-21T03:41:32.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.753+0000 7ff7ba666640 1 -- 172.21.15.38:0/1449055308 <== osd.7 v2:172.21.15.195:6818/1606106342 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v108'366 uv366 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7ff7b0014070 con 0x560ade7b2630 2024-09-21T03:41:32.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.753+0000 7ff7c201cf40 1 -- 172.21.15.38:0/1449055308 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:18 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x560ade7cd270 con 0x560ade7b2630 2024-09-21T03:41:32.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.754+0000 7ff7ba666640 1 -- 172.21.15.38:0/1449055308 <== osd.7 v2:172.21.15.195:6818/1606106342 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v108'367 uv367 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7ff7b0014070 con 0x560ade7b2630 2024-09-21T03:41:32.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.754+0000 7ff7c201cf40 1 -- 172.21.15.38:0/1449055308 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:19 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=585b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x560ade7cdb30 con 0x560ade7b2630 2024-09-21T03:41:32.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.756+0000 7ff7ba666640 1 -- 172.21.15.38:0/1449055308 <== osd.7 v2:172.21.15.195:6818/1606106342 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v108'368 uv368 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7ff7b0014070 con 0x560ade7b2630 2024-09-21T03:41:32.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.756+0000 7ff7c201cf40 1 -- 172.21.15.38:0/1449055308 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:20 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x560ade7ce620 con 0x560ade7b2630 2024-09-21T03:41:32.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.757+0000 7ff7ba666640 1 -- 172.21.15.38:0/1449055308 <== osd.7 v2:172.21.15.195:6818/1606106342 20 ==== osd_op_reply(20 600.00000000 [create] v108'369 uv343 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ff7b0014070 con 0x560ade7b2630 2024-09-21T03:41:32.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.757+0000 7ff7c201cf40 1 -- 172.21.15.38:0/1449055308 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:21 13.10 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e108) -- 0x560ade7ce620 con 0x560ade7b2630 2024-09-21T03:41:32.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.758+0000 7ff7ba666640 1 -- 172.21.15.38:0/1449055308 <== osd.7 v2:172.21.15.195:6818/1606106342 21 ==== osd_op_reply(21 600.00000000 [delete] v108'370 uv370 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff7b0014070 con 0x560ade7b2630 2024-09-21T03:41:32.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.758+0000 7ff7c201cf40 1 -- 172.21.15.38:0/1449055308 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:22 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x560ade7ce620 con 0x560ade7b2630 2024-09-21T03:41:32.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.759+0000 7ff7ba666640 1 -- 172.21.15.38:0/1449055308 <== osd.7 v2:172.21.15.195:6818/1606106342 22 ==== osd_op_reply(22 600.00000000 [create] v108'371 uv371 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff7b0014070 con 0x560ade7b2630 2024-09-21T03:41:32.761 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-09-21T03:41:32.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.759+0000 7ff7c201cf40 1 -- 172.21.15.38:0/1449055308 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:23 13.10 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x560ade7cf630 con 0x560ade7b2630 2024-09-21T03:41:32.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.760+0000 7ff7ba666640 1 -- 172.21.15.38:0/1449055308 <== osd.7 v2:172.21.15.195:6818/1606106342 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v108'372 uv372 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff7b0014070 con 0x560ade7b2630 2024-09-21T03:41:32.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.760+0000 7ff7c201cf40 1 -- 172.21.15.38:0/1449055308 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:24 13.10 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x560ade7cf630 con 0x560ade7b2630 2024-09-21T03:41:32.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.761+0000 7ff7ba666640 1 -- 172.21.15.38:0/1449055308 <== osd.7 v2:172.21.15.195:6818/1606106342 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v108'373 uv373 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff7b0014070 con 0x560ade7b2630 2024-09-21T03:41:32.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.761+0000 7ff7c201cf40 1 -- 172.21.15.38:0/1449055308 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:25 13.10 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x560ade7ce620 con 0x560ade7b2630 2024-09-21T03:41:32.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.762+0000 7ff7ba666640 1 -- 172.21.15.38:0/1449055308 <== osd.7 v2:172.21.15.195:6818/1606106342 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v108'374 uv374 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff7b0014070 con 0x560ade7b2630 2024-09-21T03:41:32.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.763+0000 7ff7c201cf40 1 -- 172.21.15.38:0/1449055308 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:26 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x560ade7ce620 con 0x560ade7b2630 2024-09-21T03:41:32.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.764+0000 7ff7ba666640 1 -- 172.21.15.38:0/1449055308 <== osd.7 v2:172.21.15.195:6818/1606106342 26 ==== osd_op_reply(26 603.00000000 [create] v108'375 uv349 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ff7b0014070 con 0x560ade7b2630 2024-09-21T03:41:32.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.764+0000 7ff7c201cf40 1 -- 172.21.15.38:0/1449055308 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:27 13.10 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e108) -- 0x560ade7ce620 con 0x560ade7b2630 2024-09-21T03:41:32.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.765+0000 7ff7ba666640 1 -- 172.21.15.38:0/1449055308 <== osd.7 v2:172.21.15.195:6818/1606106342 27 ==== osd_op_reply(27 603.00000000 [delete] v108'376 uv376 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff7b0014070 con 0x560ade7b2630 2024-09-21T03:41:32.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.765+0000 7ff7c201cf40 1 -- 172.21.15.38:0/1449055308 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:28 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x560ade7ce620 con 0x560ade7b2630 2024-09-21T03:41:32.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.766+0000 7ff7ba666640 1 -- 172.21.15.38:0/1449055308 <== osd.7 v2:172.21.15.195:6818/1606106342 28 ==== osd_op_reply(28 603.00000000 [create] v108'377 uv377 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff7b0014070 con 0x560ade7b2630 2024-09-21T03:41:32.768 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-09-21T03:41:32.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.766+0000 7ff7c201cf40 1 -- 172.21.15.38:0/1449055308 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:29 13.10 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x560ade7cf630 con 0x560ade7b2630 2024-09-21T03:41:32.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.767+0000 7ff7ba666640 1 -- 172.21.15.38:0/1449055308 <== osd.7 v2:172.21.15.195:6818/1606106342 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v108'378 uv378 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff7b0014070 con 0x560ade7b2630 2024-09-21T03:41:32.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.767+0000 7ff7c201cf40 1 -- 172.21.15.38:0/1449055308 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:30 13.10 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x560ade7cf630 con 0x560ade7b2630 2024-09-21T03:41:32.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.768+0000 7ff7ba666640 1 -- 172.21.15.38:0/1449055308 <== osd.7 v2:172.21.15.195:6818/1606106342 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v108'379 uv379 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff7b0014070 con 0x560ade7b2630 2024-09-21T03:41:32.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.768+0000 7ff7c201cf40 1 -- 172.21.15.38:0/1449055308 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:31 13.10 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x560ade7ce620 con 0x560ade7b2630 2024-09-21T03:41:32.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.769+0000 7ff7ba666640 1 -- 172.21.15.38:0/1449055308 <== osd.7 v2:172.21.15.195:6818/1606106342 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v108'380 uv380 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff7b0014070 con 0x560ade7b2630 2024-09-21T03:41:32.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.769+0000 7ff7c201cf40 1 --2- 172.21.15.38:0/1449055308 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x560ade7ceb30 0x560ade7d0f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:32.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.769+0000 7ff7c201cf40 1 -- 172.21.15.38:0/1449055308 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x560ade7d1490 con 0x560ade7ceb30 2024-09-21T03:41:32.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.770+0000 7ff7bae67640 1 --2- 172.21.15.38:0/1449055308 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x560ade7ceb30 0x560ade7d0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:32.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.770+0000 7ff7bae67640 1 --2- 172.21.15.38:0/1449055308 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x560ade7ceb30 0x560ade7d0f50 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:32.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.771+0000 7ff7bae67640 1 -- 172.21.15.38:0/1449055308 <== osd.6 v2:172.21.15.195:6826/4055825675 1 ==== osd_op_reply(32 1.00000000.inode [create] v108'261 uv244 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7ff7ac00c2b0 con 0x560ade7ceb30 2024-09-21T03:41:32.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.771+0000 7ff7c201cf40 1 -- 172.21.15.38:0/1449055308 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e108) -- 0x560ade7d1830 con 0x560ade7ceb30 2024-09-21T03:41:32.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.773+0000 7ff7bae67640 1 -- 172.21.15.38:0/1449055308 <== osd.6 v2:172.21.15.195:6826/4055825675 2 ==== osd_op_reply(33 1.00000000.inode [delete] v108'262 uv262 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7ff7ac00c2b0 con 0x560ade7ceb30 2024-09-21T03:41:32.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.773+0000 7ff7c201cf40 1 -- 172.21.15.38:0/1449055308 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x560ade7d1bd0 con 0x560ade7ceb30 2024-09-21T03:41:32.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.774+0000 7ff7bae67640 1 -- 172.21.15.38:0/1449055308 <== osd.6 v2:172.21.15.195:6826/4055825675 3 ==== osd_op_reply(34 1.00000000.inode [create] v108'263 uv263 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7ff7ac00c2b0 con 0x560ade7ceb30 2024-09-21T03:41:32.776 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-09-21T03:41:32.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.774+0000 7ff7c201cf40 1 -- 172.21.15.38:0/1449055308 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x560ade7d2420 con 0x560ade7ceb30 2024-09-21T03:41:32.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.776+0000 7ff7bae67640 1 -- 172.21.15.38:0/1449055308 <== osd.6 v2:172.21.15.195:6826/4055825675 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v108'264 uv264 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7ff7ac00c2b0 con 0x560ade7ceb30 2024-09-21T03:41:32.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.776+0000 7ff7c201cf40 1 -- 172.21.15.38:0/1449055308 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x560ade7d27c0 con 0x560ade7ceb30 2024-09-21T03:41:32.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.777+0000 7ff7bae67640 1 -- 172.21.15.38:0/1449055308 <== osd.6 v2:172.21.15.195:6826/4055825675 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v108'265 uv265 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7ff7ac00c2b0 con 0x560ade7ceb30 2024-09-21T03:41:32.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.777+0000 7ff7c201cf40 1 -- 172.21.15.38:0/1449055308 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x560ade7d2b60 con 0x560ade7ceb30 2024-09-21T03:41:32.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.778+0000 7ff7bae67640 1 -- 172.21.15.38:0/1449055308 <== osd.6 v2:172.21.15.195:6826/4055825675 6 ==== osd_op_reply(37 200.00000000 [create] v108'266 uv249 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ff7ac00c2b0 con 0x560ade7ceb30 2024-09-21T03:41:32.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.778+0000 7ff7c201cf40 1 -- 172.21.15.38:0/1449055308 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e108) -- 0x560ade7d2f00 con 0x560ade7ceb30 2024-09-21T03:41:32.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.779+0000 7ff7bae67640 1 -- 172.21.15.38:0/1449055308 <== osd.6 v2:172.21.15.195:6826/4055825675 7 ==== osd_op_reply(38 200.00000000 [delete] v108'267 uv267 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff7ac00c2b0 con 0x560ade7ceb30 2024-09-21T03:41:32.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.779+0000 7ff7c201cf40 1 -- 172.21.15.38:0/1449055308 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x560ade7d32a0 con 0x560ade7ceb30 2024-09-21T03:41:32.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.780+0000 7ff7bae67640 1 -- 172.21.15.38:0/1449055308 <== osd.6 v2:172.21.15.195:6826/4055825675 8 ==== osd_op_reply(39 200.00000000 [create] v108'268 uv268 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff7ac00c2b0 con 0x560ade7ceb30 2024-09-21T03:41:32.782 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-09-21T03:41:32.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.780+0000 7ff7c201cf40 1 -- 172.21.15.38:0/1449055308 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x560ade7d32a0 con 0x560ade7ceb30 2024-09-21T03:41:32.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.782+0000 7ff7bae67640 1 -- 172.21.15.38:0/1449055308 <== osd.6 v2:172.21.15.195:6826/4055825675 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v108'269 uv269 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff7ac00c2b0 con 0x560ade7ceb30 2024-09-21T03:41:32.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.782+0000 7ff7c201cf40 1 -- 172.21.15.38:0/1449055308 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x560ade7d32a0 con 0x560ade7ceb30 2024-09-21T03:41:32.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.783+0000 7ff7bae67640 1 -- 172.21.15.38:0/1449055308 <== osd.6 v2:172.21.15.195:6826/4055825675 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v108'270 uv270 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff7ac00c2b0 con 0x560ade7ceb30 2024-09-21T03:41:32.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.783+0000 7ff7c201cf40 1 -- 172.21.15.38:0/1449055308 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:42 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x560ade7d32a0 con 0x560ade7ceb30 2024-09-21T03:41:32.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.784+0000 7ff7bae67640 1 -- 172.21.15.38:0/1449055308 <== osd.6 v2:172.21.15.195:6826/4055825675 11 ==== osd_op_reply(42 200.00000001 [create] v108'271 uv254 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ff7ac00c2b0 con 0x560ade7ceb30 2024-09-21T03:41:32.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.784+0000 7ff7c201cf40 1 -- 172.21.15.38:0/1449055308 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:43 13.14 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e108) -- 0x560ade7d32a0 con 0x560ade7ceb30 2024-09-21T03:41:32.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.785+0000 7ff7bae67640 1 -- 172.21.15.38:0/1449055308 <== osd.6 v2:172.21.15.195:6826/4055825675 12 ==== osd_op_reply(43 200.00000001 [delete] v108'272 uv272 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff7ac00c2b0 con 0x560ade7ceb30 2024-09-21T03:41:32.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.786+0000 7ff7c201cf40 1 -- 172.21.15.38:0/1449055308 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:44 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x560ade7d32a0 con 0x560ade7ceb30 2024-09-21T03:41:32.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.787+0000 7ff7bae67640 1 -- 172.21.15.38:0/1449055308 <== osd.6 v2:172.21.15.195:6826/4055825675 13 ==== osd_op_reply(44 200.00000001 [create] v108'273 uv273 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff7ac00c2b0 con 0x560ade7ceb30 2024-09-21T03:41:32.789 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-09-21T03:41:32.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.787+0000 7ff7c201cf40 1 -- 172.21.15.38:0/1449055308 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:45 13.14 13:2e2fa760:::200.00000001:head [write 0~17490 in=17490b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x560ade7d7720 con 0x560ade7ceb30 2024-09-21T03:41:32.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.788+0000 7ff7bae67640 1 -- 172.21.15.38:0/1449055308 <== osd.6 v2:172.21.15.195:6826/4055825675 14 ==== osd_op_reply(45 200.00000001 [write 0~17490] v108'274 uv274 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff7ac00c2b0 con 0x560ade7ceb30 2024-09-21T03:41:32.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.788+0000 7ff7c201cf40 1 -- 172.21.15.38:0/1449055308 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:46 13.14 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x560ade7d7720 con 0x560ade7ceb30 2024-09-21T03:41:32.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.789+0000 7ff7bae67640 1 -- 172.21.15.38:0/1449055308 <== osd.6 v2:172.21.15.195:6826/4055825675 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v108'275 uv275 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff7ac00c2b0 con 0x560ade7ceb30 2024-09-21T03:41:32.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.789+0000 7ff7c201cf40 1 -- 172.21.15.38:0/1449055308 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:47 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x560ade7d32a0 con 0x560ade7ceb30 2024-09-21T03:41:32.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.790+0000 7ff7bae67640 1 -- 172.21.15.38:0/1449055308 <== osd.6 v2:172.21.15.195:6826/4055825675 16 ==== osd_op_reply(47 606.00000000 [create] v108'276 uv260 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ff7ac00c2b0 con 0x560ade7ceb30 2024-09-21T03:41:32.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.790+0000 7ff7c201cf40 1 -- 172.21.15.38:0/1449055308 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:48 13.14 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e108) -- 0x560ade7d32a0 con 0x560ade7ceb30 2024-09-21T03:41:32.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.792+0000 7ff7bae67640 1 -- 172.21.15.38:0/1449055308 <== osd.6 v2:172.21.15.195:6826/4055825675 17 ==== osd_op_reply(48 606.00000000 [delete] v108'277 uv277 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff7ac00c2b0 con 0x560ade7ceb30 2024-09-21T03:41:32.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.792+0000 7ff7c201cf40 1 -- 172.21.15.38:0/1449055308 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:49 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x560ade7d32a0 con 0x560ade7ceb30 2024-09-21T03:41:32.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.793+0000 7ff7bae67640 1 -- 172.21.15.38:0/1449055308 <== osd.6 v2:172.21.15.195:6826/4055825675 18 ==== osd_op_reply(49 606.00000000 [create] v108'278 uv278 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff7ac00c2b0 con 0x560ade7ceb30 2024-09-21T03:41:32.795 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-09-21T03:41:32.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.793+0000 7ff7c201cf40 1 -- 172.21.15.38:0/1449055308 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:50 13.14 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x560ade7d42b0 con 0x560ade7ceb30 2024-09-21T03:41:32.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.794+0000 7ff7bae67640 1 -- 172.21.15.38:0/1449055308 <== osd.6 v2:172.21.15.195:6826/4055825675 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v108'279 uv279 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff7ac00c2b0 con 0x560ade7ceb30 2024-09-21T03:41:32.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.794+0000 7ff7c201cf40 1 -- 172.21.15.38:0/1449055308 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:51 13.14 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x560ade7d42b0 con 0x560ade7ceb30 2024-09-21T03:41:32.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.795+0000 7ff7bae67640 1 -- 172.21.15.38:0/1449055308 <== osd.6 v2:172.21.15.195:6826/4055825675 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v108'280 uv280 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff7ac00c2b0 con 0x560ade7ceb30 2024-09-21T03:41:32.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.795+0000 7ff7c201cf40 1 -- 172.21.15.38:0/1449055308 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:52 13.14 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x560ade7d32a0 con 0x560ade7ceb30 2024-09-21T03:41:32.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.796+0000 7ff7bae67640 1 -- 172.21.15.38:0/1449055308 <== osd.6 v2:172.21.15.195:6826/4055825675 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v108'281 uv281 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff7ac00c2b0 con 0x560ade7ceb30 2024-09-21T03:41:32.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.796+0000 7ff7c201cf40 1 --2- 172.21.15.38:0/1449055308 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x560ade7d37b0 0x560ade7d5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:32.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.796+0000 7ff7c201cf40 1 -- 172.21.15.38:0/1449055308 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:53 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x560ade7d6110 con 0x560ade7d37b0 2024-09-21T03:41:32.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.797+0000 7ff7ba666640 1 --2- 172.21.15.38:0/1449055308 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x560ade7d37b0 0x560ade7d5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:32.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.797+0000 7ff7ba666640 1 --2- 172.21.15.38:0/1449055308 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x560ade7d37b0 0x560ade7d5bd0 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:32.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.798+0000 7ff7ba666640 1 -- 172.21.15.38:0/1449055308 <== osd.0 v2:172.21.15.38:6800/2381639433 1 ==== osd_op_reply(53 607.00000000 [create] v108'69 uv68 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ff7b0014070 con 0x560ade7d37b0 2024-09-21T03:41:32.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.798+0000 7ff7c201cf40 1 -- 172.21.15.38:0/1449055308 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:54 13.1c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e108) -- 0x560ade7d6110 con 0x560ade7d37b0 2024-09-21T03:41:32.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.800+0000 7ff7ba666640 1 -- 172.21.15.38:0/1449055308 <== osd.0 v2:172.21.15.38:6800/2381639433 2 ==== osd_op_reply(54 607.00000000 [delete] v108'70 uv70 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff7b0014070 con 0x560ade7d37b0 2024-09-21T03:41:32.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.800+0000 7ff7c201cf40 1 -- 172.21.15.38:0/1449055308 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:55 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x560ade7d6530 con 0x560ade7d37b0 2024-09-21T03:41:32.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.801+0000 7ff7ba666640 1 -- 172.21.15.38:0/1449055308 <== osd.0 v2:172.21.15.38:6800/2381639433 3 ==== osd_op_reply(55 607.00000000 [create] v108'71 uv71 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff7b0014070 con 0x560ade7d37b0 2024-09-21T03:41:32.803 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-09-21T03:41:32.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.801+0000 7ff7c201cf40 1 -- 172.21.15.38:0/1449055308 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:56 13.1c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x560ade7d7510 con 0x560ade7d37b0 2024-09-21T03:41:32.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.802+0000 7ff7ba666640 1 -- 172.21.15.38:0/1449055308 <== osd.0 v2:172.21.15.38:6800/2381639433 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v108'72 uv72 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff7b0014070 con 0x560ade7d37b0 2024-09-21T03:41:32.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.802+0000 7ff7c201cf40 1 -- 172.21.15.38:0/1449055308 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:57 13.1c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x560ade7d7510 con 0x560ade7d37b0 2024-09-21T03:41:32.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.803+0000 7ff7ba666640 1 -- 172.21.15.38:0/1449055308 <== osd.0 v2:172.21.15.38:6800/2381639433 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v108'73 uv73 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff7b0014070 con 0x560ade7d37b0 2024-09-21T03:41:32.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.803+0000 7ff7c201cf40 1 -- 172.21.15.38:0/1449055308 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:58 13.1c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x560ade7d6500 con 0x560ade7d37b0 2024-09-21T03:41:32.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.804+0000 7ff7ba666640 1 -- 172.21.15.38:0/1449055308 <== osd.0 v2:172.21.15.38:6800/2381639433 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v108'74 uv74 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff7b0014070 con 0x560ade7d37b0 2024-09-21T03:41:32.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.804+0000 7ff7c201cf40 1 --2- 172.21.15.38:0/1449055308 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x560ade7d6a10 0x560ade7d8e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:32.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.804+0000 7ff7c201cf40 1 -- 172.21.15.38:0/1449055308 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:59 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x560ade7d9520 con 0x560ade7d6a10 2024-09-21T03:41:32.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.804+0000 7ff7bb668640 1 --2- 172.21.15.38:0/1449055308 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x560ade7d6a10 0x560ade7d8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:32.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.804+0000 7ff7bb668640 1 --2- 172.21.15.38:0/1449055308 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x560ade7d6a10 0x560ade7d8e50 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:32.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.805+0000 7ff7bb668640 1 -- 172.21.15.38:0/1449055308 <== osd.3 v2:172.21.15.38:6808/1286810294 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v108'52 uv51 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7ff7b4071070 con 0x560ade7d6a10 2024-09-21T03:41:32.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.805+0000 7ff7c201cf40 1 -- 172.21.15.38:0/1449055308 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:60 13.6 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e108) -- 0x560ade7d9a30 con 0x560ade7d6a10 2024-09-21T03:41:32.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.807+0000 7ff7bb668640 1 -- 172.21.15.38:0/1449055308 <== osd.3 v2:172.21.15.38:6808/1286810294 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v108'53 uv53 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7ff7b4071070 con 0x560ade7d6a10 2024-09-21T03:41:32.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.807+0000 7ff7c201cf40 1 -- 172.21.15.38:0/1449055308 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:61 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x560ade7d9f30 con 0x560ade7d6a10 2024-09-21T03:41:32.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.808+0000 7ff7bb668640 1 -- 172.21.15.38:0/1449055308 <== osd.3 v2:172.21.15.38:6808/1286810294 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v108'54 uv54 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7ff7b4071070 con 0x560ade7d6a10 2024-09-21T03:41:32.810 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-09-21T03:41:32.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.808+0000 7ff7c201cf40 1 -- 172.21.15.38:0/1449055308 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:62 13.6 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x560ade7da7f0 con 0x560ade7d6a10 2024-09-21T03:41:32.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.810+0000 7ff7bb668640 1 -- 172.21.15.38:0/1449055308 <== osd.3 v2:172.21.15.38:6808/1286810294 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v108'55 uv55 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7ff7b4071070 con 0x560ade7d6a10 2024-09-21T03:41:32.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.810+0000 7ff7c201cf40 1 -- 172.21.15.38:0/1449055308 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x560ade7dab90 con 0x560ade7d37b0 2024-09-21T03:41:32.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.811+0000 7ff7ba666640 1 -- 172.21.15.38:0/1449055308 <== osd.0 v2:172.21.15.38:6800/2381639433 7 ==== osd_op_reply(63 608.00000000 [create] v108'69 uv68 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ff7b0014070 con 0x560ade7d37b0 2024-09-21T03:41:32.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.811+0000 7ff7c201cf40 1 -- 172.21.15.38:0/1449055308 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e108) -- 0x560ade7dab90 con 0x560ade7d37b0 2024-09-21T03:41:32.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.812+0000 7ff7ba666640 1 -- 172.21.15.38:0/1449055308 <== osd.0 v2:172.21.15.38:6800/2381639433 8 ==== osd_op_reply(64 608.00000000 [delete] v108'70 uv70 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff7b0014070 con 0x560ade7d37b0 2024-09-21T03:41:32.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.812+0000 7ff7c201cf40 1 -- 172.21.15.38:0/1449055308 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x560ade7dab90 con 0x560ade7d37b0 2024-09-21T03:41:32.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.813+0000 7ff7ba666640 1 -- 172.21.15.38:0/1449055308 <== osd.0 v2:172.21.15.38:6800/2381639433 9 ==== osd_op_reply(65 608.00000000 [create] v108'71 uv71 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff7b0014070 con 0x560ade7d37b0 2024-09-21T03:41:32.815 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-09-21T03:41:32.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.813+0000 7ff7c201cf40 1 -- 172.21.15.38:0/1449055308 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x560ade7dbba0 con 0x560ade7d37b0 2024-09-21T03:41:32.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.814+0000 7ff7ba666640 1 -- 172.21.15.38:0/1449055308 <== osd.0 v2:172.21.15.38:6800/2381639433 10 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v108'72 uv72 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff7b0014070 con 0x560ade7d37b0 2024-09-21T03:41:32.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.814+0000 7ff7c201cf40 1 -- 172.21.15.38:0/1449055308 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x560ade7dbba0 con 0x560ade7d37b0 2024-09-21T03:41:32.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.815+0000 7ff7ba666640 1 -- 172.21.15.38:0/1449055308 <== osd.0 v2:172.21.15.38:6800/2381639433 11 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v108'73 uv73 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff7b0014070 con 0x560ade7d37b0 2024-09-21T03:41:32.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.815+0000 7ff7c201cf40 1 -- 172.21.15.38:0/1449055308 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x560ade7dab90 con 0x560ade7d37b0 2024-09-21T03:41:32.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.816+0000 7ff7ba666640 1 -- 172.21.15.38:0/1449055308 <== osd.0 v2:172.21.15.38:6800/2381639433 12 ==== osd_op_reply(68 608.00000000 [omap-set-header] v108'74 uv74 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff7b0014070 con 0x560ade7d37b0 2024-09-21T03:41:32.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.816+0000 7ff7c201cf40 1 -- 172.21.15.38:0/1449055308 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x560ade7dab90 con 0x560ade7d6a10 2024-09-21T03:41:32.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.817+0000 7ff7bb668640 1 -- 172.21.15.38:0/1449055308 <== osd.3 v2:172.21.15.38:6808/1286810294 5 ==== osd_op_reply(69 604.00000000 [create] v108'127 uv121 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ff7b4071070 con 0x560ade7d6a10 2024-09-21T03:41:32.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.818+0000 7ff7c201cf40 1 -- 172.21.15.38:0/1449055308 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e108) -- 0x560ade7daf30 con 0x560ade7d6a10 2024-09-21T03:41:32.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.819+0000 7ff7bb668640 1 -- 172.21.15.38:0/1449055308 <== osd.3 v2:172.21.15.38:6808/1286810294 6 ==== osd_op_reply(70 604.00000000 [delete] v108'128 uv128 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff7b4071070 con 0x560ade7d6a10 2024-09-21T03:41:32.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.819+0000 7ff7c201cf40 1 -- 172.21.15.38:0/1449055308 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x560ade7db2d0 con 0x560ade7d6a10 2024-09-21T03:41:32.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.820+0000 7ff7bb668640 1 -- 172.21.15.38:0/1449055308 <== osd.3 v2:172.21.15.38:6808/1286810294 7 ==== osd_op_reply(71 604.00000000 [create] v108'129 uv129 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff7b40c3040 con 0x560ade7d6a10 2024-09-21T03:41:32.822 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-09-21T03:41:32.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.820+0000 7ff7c201cf40 1 -- 172.21.15.38:0/1449055308 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x560ade7dc680 con 0x560ade7d6a10 2024-09-21T03:41:32.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.821+0000 7ff7bb668640 1 -- 172.21.15.38:0/1449055308 <== osd.3 v2:172.21.15.38:6808/1286810294 8 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v108'130 uv130 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff7b40c3040 con 0x560ade7d6a10 2024-09-21T03:41:32.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.821+0000 7ff7c201cf40 1 -- 172.21.15.38:0/1449055308 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x560ade7dc680 con 0x560ade7d6a10 2024-09-21T03:41:32.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.822+0000 7ff7bb668640 1 -- 172.21.15.38:0/1449055308 <== osd.3 v2:172.21.15.38:6808/1286810294 9 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v108'131 uv131 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff7b40c3040 con 0x560ade7d6a10 2024-09-21T03:41:32.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.822+0000 7ff7c201cf40 1 -- 172.21.15.38:0/1449055308 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x560ade7db670 con 0x560ade7d6a10 2024-09-21T03:41:32.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.823+0000 7ff7bb668640 1 -- 172.21.15.38:0/1449055308 <== osd.3 v2:172.21.15.38:6808/1286810294 10 ==== osd_op_reply(74 604.00000000 [omap-set-header] v108'132 uv132 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff7b40c3040 con 0x560ade7d6a10 2024-09-21T03:41:32.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.823+0000 7ff7c201cf40 1 -- 172.21.15.38:0/1449055308 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x560ade7db670 con 0x560ade7d6a10 2024-09-21T03:41:32.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.824+0000 7ff7bb668640 1 -- 172.21.15.38:0/1449055308 <== osd.3 v2:172.21.15.38:6808/1286810294 11 ==== osd_op_reply(75 500.00000000 [create] v108'133 uv126 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ff7b40c3040 con 0x560ade7d6a10 2024-09-21T03:41:32.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.824+0000 7ff7c201cf40 1 -- 172.21.15.38:0/1449055308 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e108) -- 0x560ade7db670 con 0x560ade7d6a10 2024-09-21T03:41:32.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.825+0000 7ff7bb668640 1 -- 172.21.15.38:0/1449055308 <== osd.3 v2:172.21.15.38:6808/1286810294 12 ==== osd_op_reply(76 500.00000000 [delete] v108'134 uv134 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff7b40c3040 con 0x560ade7d6a10 2024-09-21T03:41:32.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.825+0000 7ff7c201cf40 1 -- 172.21.15.38:0/1449055308 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x560ade7db670 con 0x560ade7d6a10 2024-09-21T03:41:32.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.826+0000 7ff7bb668640 1 -- 172.21.15.38:0/1449055308 <== osd.3 v2:172.21.15.38:6808/1286810294 13 ==== osd_op_reply(77 500.00000000 [create] v108'135 uv135 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff7b40c3040 con 0x560ade7d6a10 2024-09-21T03:41:32.828 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-09-21T03:41:32.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.826+0000 7ff7c201cf40 1 -- 172.21.15.38:0/1449055308 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x560ade7db670 con 0x560ade7d6a10 2024-09-21T03:41:32.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.827+0000 7ff7bb668640 1 -- 172.21.15.38:0/1449055308 <== osd.3 v2:172.21.15.38:6808/1286810294 14 ==== osd_op_reply(78 500.00000000 [write 0~90] v108'136 uv136 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff7b40c3040 con 0x560ade7d6a10 2024-09-21T03:41:32.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.827+0000 7ff7c201cf40 1 -- 172.21.15.38:0/1449055308 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x560ade7db670 con 0x560ade7d6a10 2024-09-21T03:41:32.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.828+0000 7ff7bb668640 1 -- 172.21.15.38:0/1449055308 <== osd.3 v2:172.21.15.38:6808/1286810294 15 ==== osd_op_reply(79 500.00000000 [omap-set-header] v108'137 uv137 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff7b40c3040 con 0x560ade7d6a10 2024-09-21T03:41:32.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.828+0000 7ff7c201cf40 1 -- 172.21.15.38:0/1449055308 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:80 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x560ade7db670 con 0x560ade7d37b0 2024-09-21T03:41:32.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.829+0000 7ff7ba666640 1 -- 172.21.15.38:0/1449055308 <== osd.0 v2:172.21.15.38:6800/2381639433 13 ==== osd_op_reply(80 mds_snaptable [create] v108'56 uv55 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7ff7b0014070 con 0x560ade7d37b0 2024-09-21T03:41:32.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.829+0000 7ff7c201cf40 1 -- 172.21.15.38:0/1449055308 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:81 13.d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e108) -- 0x560ade7db670 con 0x560ade7d37b0 2024-09-21T03:41:32.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.830+0000 7ff7ba666640 1 -- 172.21.15.38:0/1449055308 <== osd.0 v2:172.21.15.38:6800/2381639433 14 ==== osd_op_reply(81 mds_snaptable [delete] v108'57 uv57 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7ff7b0014070 con 0x560ade7d37b0 2024-09-21T03:41:32.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.830+0000 7ff7c201cf40 1 -- 172.21.15.38:0/1449055308 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:82 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x560ade7db670 con 0x560ade7d37b0 2024-09-21T03:41:32.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.831+0000 7ff7ba666640 1 -- 172.21.15.38:0/1449055308 <== osd.0 v2:172.21.15.38:6800/2381639433 15 ==== osd_op_reply(82 mds_snaptable [create] v108'58 uv58 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7ff7b0014070 con 0x560ade7d37b0 2024-09-21T03:41:32.833 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-09-21T03:41:32.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.831+0000 7ff7c201cf40 1 -- 172.21.15.38:0/1449055308 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:83 13.d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x560ade7db670 con 0x560ade7d37b0 2024-09-21T03:41:32.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.832+0000 7ff7ba666640 1 -- 172.21.15.38:0/1449055308 <== osd.0 v2:172.21.15.38:6800/2381639433 16 ==== osd_op_reply(83 mds_snaptable [write 0~70] v108'59 uv59 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7ff7b0014070 con 0x560ade7d37b0 2024-09-21T03:41:32.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.832+0000 7ff7c201cf40 1 -- 172.21.15.38:0/1449055308 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:84 13.d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x560ade7db670 con 0x560ade7d37b0 2024-09-21T03:41:32.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.833+0000 7ff7ba666640 1 -- 172.21.15.38:0/1449055308 <== osd.0 v2:172.21.15.38:6800/2381639433 17 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v108'60 uv60 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7ff7b0014070 con 0x560ade7d37b0 2024-09-21T03:41:32.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.833+0000 7ff7c201cf40 1 --2- 172.21.15.38:0/1449055308 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x560ade7dbcf0 0x560ade7de110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:32.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.833+0000 7ff7c201cf40 1 -- 172.21.15.38:0/1449055308 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x560ade7de6e0 con 0x560ade7dbcf0 2024-09-21T03:41:32.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.834+0000 7ff7bae67640 1 --2- 172.21.15.38:0/1449055308 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x560ade7dbcf0 0x560ade7de110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:32.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.834+0000 7ff7bae67640 1 --2- 172.21.15.38:0/1449055308 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x560ade7dbcf0 0x560ade7de110 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:32.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.835+0000 7ff7bae67640 1 -- 172.21.15.38:0/1449055308 <== osd.1 v2:172.21.15.38:6816/1491818732 1 ==== osd_op_reply(85 605.00000000 [create] v108'69 uv68 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ff7ac00c2b0 con 0x560ade7dbcf0 2024-09-21T03:41:32.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.835+0000 7ff7c201cf40 1 -- 172.21.15.38:0/1449055308 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e108) -- 0x560ade7deab0 con 0x560ade7dbcf0 2024-09-21T03:41:32.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.836+0000 7ff7bae67640 1 -- 172.21.15.38:0/1449055308 <== osd.1 v2:172.21.15.38:6816/1491818732 2 ==== osd_op_reply(86 605.00000000 [delete] v108'70 uv70 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff7ac00c2b0 con 0x560ade7dbcf0 2024-09-21T03:41:32.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.836+0000 7ff7c201cf40 1 -- 172.21.15.38:0/1449055308 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x560ade7deab0 con 0x560ade7dbcf0 2024-09-21T03:41:32.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.837+0000 7ff7bae67640 1 -- 172.21.15.38:0/1449055308 <== osd.1 v2:172.21.15.38:6816/1491818732 3 ==== osd_op_reply(87 605.00000000 [create] v108'71 uv71 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff7ac00c2b0 con 0x560ade7dbcf0 2024-09-21T03:41:32.839 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-09-21T03:41:32.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.837+0000 7ff7c201cf40 1 -- 172.21.15.38:0/1449055308 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x560ade7dfac0 con 0x560ade7dbcf0 2024-09-21T03:41:32.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.838+0000 7ff7bae67640 1 -- 172.21.15.38:0/1449055308 <== osd.1 v2:172.21.15.38:6816/1491818732 4 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v108'72 uv72 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff7ac00c2b0 con 0x560ade7dbcf0 2024-09-21T03:41:32.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.838+0000 7ff7c201cf40 1 -- 172.21.15.38:0/1449055308 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x560ade7dfac0 con 0x560ade7dbcf0 2024-09-21T03:41:32.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.839+0000 7ff7bae67640 1 -- 172.21.15.38:0/1449055308 <== osd.1 v2:172.21.15.38:6816/1491818732 5 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v108'73 uv73 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff7ac00c2b0 con 0x560ade7dbcf0 2024-09-21T03:41:32.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.839+0000 7ff7c201cf40 1 -- 172.21.15.38:0/1449055308 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x560ade7deab0 con 0x560ade7dbcf0 2024-09-21T03:41:32.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.840+0000 7ff7bae67640 1 -- 172.21.15.38:0/1449055308 <== osd.1 v2:172.21.15.38:6816/1491818732 6 ==== osd_op_reply(90 605.00000000 [omap-set-header] v108'74 uv74 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff7ac00c2b0 con 0x560ade7dbcf0 2024-09-21T03:41:32.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.840+0000 7ff7c201cf40 1 -- 172.21.15.38:0/1449055308 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x560ade7deea0 con 0x560ade7dbcf0 2024-09-21T03:41:32.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.841+0000 7ff7bae67640 1 -- 172.21.15.38:0/1449055308 <== osd.1 v2:172.21.15.38:6816/1491818732 7 ==== osd_op_reply(91 mds0_inotable [create] v108'112 uv108 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7ff7ac00c2b0 con 0x560ade7dbcf0 2024-09-21T03:41:32.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.841+0000 7ff7c201cf40 1 -- 172.21.15.38:0/1449055308 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e108) -- 0x560ade7deea0 con 0x560ade7dbcf0 2024-09-21T03:41:32.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.842+0000 7ff7bae67640 1 -- 172.21.15.38:0/1449055308 <== osd.1 v2:172.21.15.38:6816/1491818732 8 ==== osd_op_reply(92 mds0_inotable [delete] v108'113 uv113 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7ff7ac00c2b0 con 0x560ade7dbcf0 2024-09-21T03:41:32.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.842+0000 7ff7c201cf40 1 -- 172.21.15.38:0/1449055308 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x560ade7deb30 con 0x560ade7dbcf0 2024-09-21T03:41:32.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.844+0000 7ff7bae67640 1 -- 172.21.15.38:0/1449055308 <== osd.1 v2:172.21.15.38:6816/1491818732 9 ==== osd_op_reply(93 mds0_inotable [create] v108'114 uv114 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7ff7ac00c2b0 con 0x560ade7dbcf0 2024-09-21T03:41:32.845 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-09-21T03:41:32.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.844+0000 7ff7c201cf40 1 -- 172.21.15.38:0/1449055308 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:94 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x560ade7da3e0 con 0x560ade7dbcf0 2024-09-21T03:41:32.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.845+0000 7ff7bae67640 1 -- 172.21.15.38:0/1449055308 <== osd.1 v2:172.21.15.38:6816/1491818732 10 ==== osd_op_reply(94 mds0_inotable [write 0~34] v108'115 uv115 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7ff7ac00c2b0 con 0x560ade7dbcf0 2024-09-21T03:41:32.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.845+0000 7ff7c201cf40 1 -- 172.21.15.38:0/1449055308 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:95 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x560ade7da3e0 con 0x560ade7dbcf0 2024-09-21T03:41:32.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.846+0000 7ff7bae67640 1 -- 172.21.15.38:0/1449055308 <== osd.1 v2:172.21.15.38:6816/1491818732 11 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v108'116 uv116 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7ff7ac00c2b0 con 0x560ade7dbcf0 2024-09-21T03:41:32.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.846+0000 7ff7c201cf40 1 -- 172.21.15.38:0/1449055308 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:96 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x560ade7da3e0 con 0x560ade7dbcf0 2024-09-21T03:41:32.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.847+0000 7ff7bae67640 1 -- 172.21.15.38:0/1449055308 <== osd.1 v2:172.21.15.38:6816/1491818732 12 ==== osd_op_reply(96 100.00000000 [create] v108'117 uv111 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ff7ac00c2b0 con 0x560ade7dbcf0 2024-09-21T03:41:32.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.847+0000 7ff7c201cf40 1 -- 172.21.15.38:0/1449055308 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:97 13.13 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e108) -- 0x560ade7def60 con 0x560ade7dbcf0 2024-09-21T03:41:32.850 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.848+0000 7ff7bae67640 1 -- 172.21.15.38:0/1449055308 <== osd.1 v2:172.21.15.38:6816/1491818732 13 ==== osd_op_reply(97 100.00000000 [delete] v108'118 uv118 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff7ac00c2b0 con 0x560ade7dbcf0 2024-09-21T03:41:32.850 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.848+0000 7ff7c201cf40 1 -- 172.21.15.38:0/1449055308 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:98 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x560ade7def60 con 0x560ade7dbcf0 2024-09-21T03:41:32.851 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-09-21T03:41:32.851 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.849+0000 7ff7bae67640 1 -- 172.21.15.38:0/1449055308 <== osd.1 v2:172.21.15.38:6816/1491818732 14 ==== osd_op_reply(98 100.00000000 [create] v108'119 uv119 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff7ac00c2b0 con 0x560ade7dbcf0 2024-09-21T03:41:32.851 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.849+0000 7ff7c201cf40 1 -- 172.21.15.38:0/1449055308 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:99 13.13 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x560ade7def60 con 0x560ade7dbcf0 2024-09-21T03:41:32.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.850+0000 7ff7bae67640 1 -- 172.21.15.38:0/1449055308 <== osd.1 v2:172.21.15.38:6816/1491818732 15 ==== osd_op_reply(99 100.00000000 [omap-set-header] v108'120 uv120 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff7ac00c2b0 con 0x560ade7dbcf0 2024-09-21T03:41:32.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.850+0000 7ff7c201cf40 1 -- 172.21.15.38:0/1449055308 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:100 13.13 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x560ade7e1a30 con 0x560ade7dbcf0 2024-09-21T03:41:32.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.852+0000 7ff7bae67640 1 -- 172.21.15.38:0/1449055308 <== osd.1 v2:172.21.15.38:6816/1491818732 16 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v108'121 uv121 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff7ac00c2b0 con 0x560ade7dbcf0 2024-09-21T03:41:32.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.852+0000 7ff7c201cf40 1 -- 172.21.15.38:0/1449055308 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x560ade7e19f0 con 0x560ade7d6a10 2024-09-21T03:41:32.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.853+0000 7ff7bb668640 1 -- 172.21.15.38:0/1449055308 <== osd.3 v2:172.21.15.38:6808/1286810294 16 ==== osd_op_reply(101 mds0_sessionmap [create] v108'54 uv54 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7ff7b40c3040 con 0x560ade7d6a10 2024-09-21T03:41:32.855 INFO:teuthology.orchestra.run.smithi038.stdout:Write #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-09-21T03:41:32.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.853+0000 7ff7c201cf40 1 -- 172.21.15.38:0/1449055308 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x560ade7e19f0 con 0x560ade7d6a10 2024-09-21T03:41:32.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.854+0000 7ff7bb668640 1 -- 172.21.15.38:0/1449055308 <== osd.3 v2:172.21.15.38:6808/1286810294 17 ==== osd_op_reply(102 mds0_sessionmap [omap-set-header] v108'55 uv55 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7ff7b40c3040 con 0x560ade7d6a10 2024-09-21T03:41:32.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.854+0000 7ff7c201cf40 1 -- 172.21.15.38:0/1449055308 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:103 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x560ade7e19f0 con 0x560ade7d6a10 2024-09-21T03:41:32.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.855+0000 7ff7bb668640 1 -- 172.21.15.38:0/1449055308 <== osd.3 v2:172.21.15.38:6808/1286810294 18 ==== osd_op_reply(103 609.00000000 [create] v108'69 uv68 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ff7b40c3040 con 0x560ade7d6a10 2024-09-21T03:41:32.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.855+0000 7ff7c201cf40 1 -- 172.21.15.38:0/1449055308 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:104 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e108) -- 0x560ade7e19f0 con 0x560ade7d6a10 2024-09-21T03:41:32.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.856+0000 7ff7bb668640 1 -- 172.21.15.38:0/1449055308 <== osd.3 v2:172.21.15.38:6808/1286810294 19 ==== osd_op_reply(104 609.00000000 [delete] v108'70 uv70 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff7b40c3040 con 0x560ade7d6a10 2024-09-21T03:41:32.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.856+0000 7ff7c201cf40 1 -- 172.21.15.38:0/1449055308 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x560ade7e19f0 con 0x560ade7d6a10 2024-09-21T03:41:32.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.857+0000 7ff7bb668640 1 -- 172.21.15.38:0/1449055308 <== osd.3 v2:172.21.15.38:6808/1286810294 20 ==== osd_op_reply(105 609.00000000 [create] v108'71 uv71 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff7b40c3040 con 0x560ade7d6a10 2024-09-21T03:41:32.859 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-09-21T03:41:32.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.857+0000 7ff7c201cf40 1 -- 172.21.15.38:0/1449055308 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x560ade7e19f0 con 0x560ade7d6a10 2024-09-21T03:41:32.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.858+0000 7ff7bb668640 1 -- 172.21.15.38:0/1449055308 <== osd.3 v2:172.21.15.38:6808/1286810294 21 ==== osd_op_reply(106 609.00000000 [setxattr (30)] v108'72 uv72 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff7b40c3040 con 0x560ade7d6a10 2024-09-21T03:41:32.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.858+0000 7ff7c201cf40 1 -- 172.21.15.38:0/1449055308 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x560ade7e19d0 con 0x560ade7d6a10 2024-09-21T03:41:32.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.859+0000 7ff7bb668640 1 -- 172.21.15.38:0/1449055308 <== osd.3 v2:172.21.15.38:6808/1286810294 22 ==== osd_op_reply(107 609.00000000 [setxattr (62)] v108'73 uv73 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff7b40c3040 con 0x560ade7d6a10 2024-09-21T03:41:32.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.859+0000 7ff7c201cf40 1 -- 172.21.15.38:0/1449055308 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x560ade7e19d0 con 0x560ade7d6a10 2024-09-21T03:41:32.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.860+0000 7ff7bb668640 1 -- 172.21.15.38:0/1449055308 <== osd.3 v2:172.21.15.38:6808/1286810294 23 ==== osd_op_reply(108 609.00000000 [omap-set-header] v108'74 uv74 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff7b40c3040 con 0x560ade7d6a10 2024-09-21T03:41:32.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.860+0000 7ff7c201cf40 1 --2- 172.21.15.38:0/1449055308 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x560ade7defc0 0x560ade7e20f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:32.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.860+0000 7ff7c201cf40 1 -- 172.21.15.38:0/1449055308 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:109 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x560ade7e19b0 con 0x560ade7defc0 2024-09-21T03:41:32.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.860+0000 7ff7ba666640 1 --2- 172.21.15.38:0/1449055308 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x560ade7defc0 0x560ade7e20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:32.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.861+0000 7ff7ba666640 1 --2- 172.21.15.38:0/1449055308 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x560ade7defc0 0x560ade7e20f0 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:32.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.862+0000 7ff7ba666640 1 -- 172.21.15.38:0/1449055308 <== osd.2 v2:172.21.15.38:6824/2577858902 1 ==== osd_op_reply(109 400.00000000 [create] v108'57 uv56 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ff7b0014070 con 0x560ade7defc0 2024-09-21T03:41:32.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.862+0000 7ff7c201cf40 1 -- 172.21.15.38:0/1449055308 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:110 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e108) -- 0x560ade7e19b0 con 0x560ade7defc0 2024-09-21T03:41:32.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.863+0000 7ff7ba666640 1 -- 172.21.15.38:0/1449055308 <== osd.2 v2:172.21.15.38:6824/2577858902 2 ==== osd_op_reply(110 400.00000000 [delete] v108'58 uv58 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff7b0014070 con 0x560ade7defc0 2024-09-21T03:41:32.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.863+0000 7ff7c201cf40 1 -- 172.21.15.38:0/1449055308 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x560ade7e19b0 con 0x560ade7defc0 2024-09-21T03:41:32.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.864+0000 7ff7ba666640 1 -- 172.21.15.38:0/1449055308 <== osd.2 v2:172.21.15.38:6824/2577858902 3 ==== osd_op_reply(111 400.00000000 [create] v108'59 uv59 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff7b0014070 con 0x560ade7defc0 2024-09-21T03:41:32.866 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-09-21T03:41:32.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.864+0000 7ff7c201cf40 1 -- 172.21.15.38:0/1449055308 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x560ade7e2630 con 0x560ade7defc0 2024-09-21T03:41:32.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.865+0000 7ff7ba666640 1 -- 172.21.15.38:0/1449055308 <== osd.2 v2:172.21.15.38:6824/2577858902 4 ==== osd_op_reply(112 400.00000000 [write 0~22] v108'60 uv60 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff7b0014070 con 0x560ade7defc0 2024-09-21T03:41:32.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.865+0000 7ff7c201cf40 1 -- 172.21.15.38:0/1449055308 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x560ade7e2630 con 0x560ade7defc0 2024-09-21T03:41:32.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.866+0000 7ff7ba666640 1 -- 172.21.15.38:0/1449055308 <== osd.2 v2:172.21.15.38:6824/2577858902 5 ==== osd_op_reply(113 400.00000000 [omap-set-header] v108'61 uv61 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff7b0014070 con 0x560ade7defc0 2024-09-21T03:41:32.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.866+0000 7ff7c201cf40 1 -- 172.21.15.38:0/1449055308 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:114 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x560ade7e2630 con 0x560ade7dbcf0 2024-09-21T03:41:32.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.867+0000 7ff7bae67640 1 -- 172.21.15.38:0/1449055308 <== osd.1 v2:172.21.15.38:6816/1491818732 17 ==== osd_op_reply(114 100.00000000.inode [create] v108'144 uv136 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7ff7ac00c2b0 con 0x560ade7dbcf0 2024-09-21T03:41:32.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.867+0000 7ff7c201cf40 1 -- 172.21.15.38:0/1449055308 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:115 13.1f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e108) -- 0x560ade7e2630 con 0x560ade7dbcf0 2024-09-21T03:41:32.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.869+0000 7ff7bae67640 1 -- 172.21.15.38:0/1449055308 <== osd.1 v2:172.21.15.38:6816/1491818732 18 ==== osd_op_reply(115 100.00000000.inode [delete] v108'145 uv145 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7ff7ac00c2b0 con 0x560ade7dbcf0 2024-09-21T03:41:32.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.869+0000 7ff7c201cf40 1 -- 172.21.15.38:0/1449055308 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:116 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x560ade7e2630 con 0x560ade7dbcf0 2024-09-21T03:41:32.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.870+0000 7ff7bae67640 1 -- 172.21.15.38:0/1449055308 <== osd.1 v2:172.21.15.38:6816/1491818732 19 ==== osd_op_reply(116 100.00000000.inode [create] v108'146 uv146 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7ff7ac00c2b0 con 0x560ade7dbcf0 2024-09-21T03:41:32.872 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-09-21T03:41:32.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.870+0000 7ff7c201cf40 1 -- 172.21.15.38:0/1449055308 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:117 13.1f 13:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x560ade7e2630 con 0x560ade7dbcf0 2024-09-21T03:41:32.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.871+0000 7ff7bae67640 1 -- 172.21.15.38:0/1449055308 <== osd.1 v2:172.21.15.38:6816/1491818732 20 ==== osd_op_reply(117 100.00000000.inode [write 0~582] v108'147 uv147 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7ff7ac00c2b0 con 0x560ade7dbcf0 2024-09-21T03:41:32.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.871+0000 7ff7c201cf40 1 -- 172.21.15.38:0/1449055308 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:118 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x560ade7e2630 con 0x560ade7dbcf0 2024-09-21T03:41:32.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.872+0000 7ff7bae67640 1 -- 172.21.15.38:0/1449055308 <== osd.1 v2:172.21.15.38:6816/1491818732 21 ==== osd_op_reply(118 100.00000000.inode [omap-set-header] v108'148 uv148 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7ff7ac00c2b0 con 0x560ade7dbcf0 2024-09-21T03:41:32.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.872+0000 7ff7c201cf40 1 -- 172.21.15.38:0/1449055308 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:119 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x560ade7e2630 con 0x560ade7dbcf0 2024-09-21T03:41:32.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.873+0000 7ff7bae67640 1 -- 172.21.15.38:0/1449055308 <== osd.1 v2:172.21.15.38:6816/1491818732 22 ==== osd_op_reply(119 1.00000000 [create] v108'149 uv143 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7ff7ac00c2b0 con 0x560ade7dbcf0 2024-09-21T03:41:32.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.873+0000 7ff7c201cf40 1 -- 172.21.15.38:0/1449055308 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:120 13.1f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e108) -- 0x560ade7e2630 con 0x560ade7dbcf0 2024-09-21T03:41:32.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.874+0000 7ff7bae67640 1 -- 172.21.15.38:0/1449055308 <== osd.1 v2:172.21.15.38:6816/1491818732 23 ==== osd_op_reply(120 1.00000000 [delete] v108'150 uv150 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7ff7ac00c2b0 con 0x560ade7dbcf0 2024-09-21T03:41:32.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.874+0000 7ff7c201cf40 1 -- 172.21.15.38:0/1449055308 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:121 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x560ade7e2630 con 0x560ade7dbcf0 2024-09-21T03:41:32.877 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-09-21T03:41:32.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.875+0000 7ff7bae67640 1 -- 172.21.15.38:0/1449055308 <== osd.1 v2:172.21.15.38:6816/1491818732 24 ==== osd_op_reply(121 1.00000000 [create] v108'151 uv151 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7ff7ac00c2b0 con 0x560ade7dbcf0 2024-09-21T03:41:32.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.875+0000 7ff7c201cf40 1 -- 172.21.15.38:0/1449055308 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:122 13.1f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x560ade7e3640 con 0x560ade7dbcf0 2024-09-21T03:41:32.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.876+0000 7ff7bae67640 1 -- 172.21.15.38:0/1449055308 <== osd.1 v2:172.21.15.38:6816/1491818732 25 ==== osd_op_reply(122 1.00000000 [setxattr (30)] v108'152 uv152 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7ff7ac00c2b0 con 0x560ade7dbcf0 2024-09-21T03:41:32.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.877+0000 7ff7c201cf40 1 -- 172.21.15.38:0/1449055308 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:123 13.1f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x560ade7e3640 con 0x560ade7dbcf0 2024-09-21T03:41:32.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.877+0000 7ff7bae67640 1 -- 172.21.15.38:0/1449055308 <== osd.1 v2:172.21.15.38:6816/1491818732 26 ==== osd_op_reply(123 1.00000000 [setxattr (30)] v108'153 uv153 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7ff7ac00c2b0 con 0x560ade7dbcf0 2024-09-21T03:41:32.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.878+0000 7ff7c201cf40 1 -- 172.21.15.38:0/1449055308 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:124 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x560ade7e2630 con 0x560ade7dbcf0 2024-09-21T03:41:32.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.879+0000 7ff7bae67640 1 -- 172.21.15.38:0/1449055308 <== osd.1 v2:172.21.15.38:6816/1491818732 27 ==== osd_op_reply(124 1.00000000 [omap-set-header] v108'154 uv154 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7ff7ac00c2b0 con 0x560ade7dbcf0 2024-09-21T03:41:32.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.879+0000 7ff7c201cf40 1 -- 172.21.15.38:0/1449055308 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:125 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=584b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x560ade7e2ae0 con 0x560ade7dbcf0 2024-09-21T03:41:32.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.880+0000 7ff7bae67640 1 -- 172.21.15.38:0/1449055308 <== osd.1 v2:172.21.15.38:6816/1491818732 28 ==== osd_op_reply(125 1.00000000 [omap-set-vals] v108'155 uv155 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7ff7ac00c2b0 con 0x560ade7dbcf0 2024-09-21T03:41:32.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.880+0000 7ff7c201cf40 1 -- 172.21.15.38:0/1449055308 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x560ade7d37b0 msgr2=0x560ade7d5bd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:32.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.880+0000 7ff7c201cf40 1 --2- 172.21.15.38:0/1449055308 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x560ade7d37b0 0x560ade7d5bd0 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:32.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.880+0000 7ff7c201cf40 1 -- 172.21.15.38:0/1449055308 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x560ade7dbcf0 msgr2=0x560ade7de110 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:32.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.880+0000 7ff7c201cf40 1 --2- 172.21.15.38:0/1449055308 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x560ade7dbcf0 0x560ade7de110 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:32.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.880+0000 7ff7c201cf40 1 -- 172.21.15.38:0/1449055308 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x560ade7defc0 msgr2=0x560ade7e20f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:32.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.880+0000 7ff7c201cf40 1 --2- 172.21.15.38:0/1449055308 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x560ade7defc0 0x560ade7e20f0 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:32.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.880+0000 7ff7c201cf40 1 -- 172.21.15.38:0/1449055308 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x560ade7d6a10 msgr2=0x560ade7d8e50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:32.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.880+0000 7ff7c201cf40 1 --2- 172.21.15.38:0/1449055308 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x560ade7d6a10 0x560ade7d8e50 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:32.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.880+0000 7ff7bb668640 1 -- 172.21.15.38:0/1449055308 reap_dead start 2024-09-21T03:41:32.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.880+0000 7ff7c201cf40 1 -- 172.21.15.38:0/1449055308 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x560ade7ceb30 msgr2=0x560ade7d0f50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:32.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.880+0000 7ff7c201cf40 1 --2- 172.21.15.38:0/1449055308 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x560ade7ceb30 0x560ade7d0f50 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:32.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.880+0000 7ff7bb668640 1 -- 172.21.15.38:0/1449055308 reap_dead start 2024-09-21T03:41:32.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.880+0000 7ff7c201cf40 1 -- 172.21.15.38:0/1449055308 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x560ade7b2630 msgr2=0x560ade70ecc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:32.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.880+0000 7ff7c201cf40 1 --2- 172.21.15.38:0/1449055308 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x560ade7b2630 0x560ade70ecc0 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:32.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.880+0000 7ff7c201cf40 1 -- 172.21.15.38:0/1449055308 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff7a8077570 msgr2=0x7ff7a8079a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:32.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.880+0000 7ff7c201cf40 1 --2- 172.21.15.38:0/1449055308 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff7a8077570 0x7ff7a8079a80 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7ff7ac00c780 tx=0x7ff7ac00f040 comp rx=0 tx=0).stop 2024-09-21T03:41:32.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.880+0000 7ff7c201cf40 1 -- 172.21.15.38:0/1449055308 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560ade710950 msgr2=0x560ade7c0d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:32.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.880+0000 7ff7c201cf40 1 --2- 172.21.15.38:0/1449055308 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560ade710950 0x560ade7c0d20 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7ff7b4046470 tx=0x7ff7b4004570 comp rx=0 tx=0).stop 2024-09-21T03:41:32.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.880+0000 7ff7c201cf40 1 -- 172.21.15.38:0/1449055308 shutdown_connections 2024-09-21T03:41:32.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.880+0000 7ff7c201cf40 1 --2- 172.21.15.38:0/1449055308 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x560ade7b2630 0x560ade70ecc0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:32.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.880+0000 7ff7c201cf40 1 --2- 172.21.15.38:0/1449055308 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560ade710950 0x560ade7c0d20 unknown :-1 s=CLOSED pgs=1512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:32.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.880+0000 7ff7c201cf40 1 --2- 172.21.15.38:0/1449055308 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff7a8077570 0x7ff7a8079a80 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:32.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.880+0000 7ff7c201cf40 1 -- 172.21.15.38:0/1449055308 >> 172.21.15.38:0/1449055308 conn(0x560ade574ac0 msgr2=0x560ade70d080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:32.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.880+0000 7ff7c201cf40 1 -- 172.21.15.38:0/1449055308 shutdown_connections 2024-09-21T03:41:32.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.880+0000 7ff7c201cf40 1 -- 172.21.15.38:0/1449055308 wait complete. 2024-09-21T03:41:32.886 DEBUG:teuthology.orchestra.run.smithi038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rm 400.00000000 2024-09-21T03:41:32.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.980+0000 7f2bb8fd9f40 1 Processor -- start 2024-09-21T03:41:32.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.980+0000 7f2bb8fd9f40 1 -- start start 2024-09-21T03:41:32.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.981+0000 7f2bb8fd9f40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2b9c003bd0 0x55b74e3fa2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:32.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.981+0000 7f2bb8fd9f40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b74e3db950 0x55b74e34e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:32.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.981+0000 7f2bb8fd9f40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b74e3dbd60 0x55b74e3218e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:32.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.981+0000 7f2bb8fd9f40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55b74e2d2430 con 0x7f2b9c003bd0 2024-09-21T03:41:32.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.981+0000 7f2bb8fd9f40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55b74e2df1f0 con 0x55b74e3dbd60 2024-09-21T03:41:32.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.981+0000 7f2bb8fd9f40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55b74e273d60 con 0x55b74e3db950 2024-09-21T03:41:32.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.981+0000 7f2bb2a93640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2b9c003bd0 0x55b74e3fa2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:32.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.981+0000 7f2bb2a93640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2b9c003bd0 0x55b74e3fa2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54202/0 (socket says 172.21.15.38:54202) 2024-09-21T03:41:32.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.981+0000 7f2bb2a93640 1 -- 172.21.15.38:0/2097964583 learned_addr learned my addr 172.21.15.38:0/2097964583 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:41:32.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.981+0000 7f2bb2292640 1 --2- 172.21.15.38:0/2097964583 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b74e3db950 0x55b74e34e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:32.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.981+0000 7f2bb3294640 1 --2- 172.21.15.38:0/2097964583 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b74e3dbd60 0x55b74e3218e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:32.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.981+0000 7f2bb2a93640 1 -- 172.21.15.38:0/2097964583 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b74e3db950 msgr2=0x55b74e34e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:32.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.981+0000 7f2bb2a93640 1 --2- 172.21.15.38:0/2097964583 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b74e3db950 0x55b74e34e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:32.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.981+0000 7f2bb2a93640 1 -- 172.21.15.38:0/2097964583 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b74e3dbd60 msgr2=0x55b74e3218e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:32.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.981+0000 7f2bb2a93640 1 --2- 172.21.15.38:0/2097964583 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b74e3dbd60 0x55b74e3218e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:32.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.981+0000 7f2bb2a93640 1 -- 172.21.15.38:0/2097964583 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b74e36a5b0 con 0x7f2b9c003bd0 2024-09-21T03:41:32.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.981+0000 7f2bb2292640 1 --2- 172.21.15.38:0/2097964583 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b74e3db950 0x55b74e34e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:41:32.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.981+0000 7f2bb3294640 1 --2- 172.21.15.38:0/2097964583 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b74e3dbd60 0x55b74e3218e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:41:32.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.982+0000 7f2bb2a93640 1 --2- 172.21.15.38:0/2097964583 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2b9c003bd0 0x55b74e3fa2f0 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f2ba400b7c0 tx=0x7f2ba4004880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e07cd002c2af14a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:32.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.982+0000 7f2bb1290640 1 -- 172.21.15.38:0/2097964583 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ba400ac20 con 0x7f2b9c003bd0 2024-09-21T03:41:32.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.982+0000 7f2bb1290640 1 -- 172.21.15.38:0/2097964583 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2ba400adc0 con 0x7f2b9c003bd0 2024-09-21T03:41:32.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.982+0000 7f2bb1290640 1 -- 172.21.15.38:0/2097964583 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ba4017670 con 0x7f2b9c003bd0 2024-09-21T03:41:32.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.982+0000 7f2bb8fd9f40 1 -- 172.21.15.38:0/2097964583 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2b9c003bd0 msgr2=0x55b74e3fa2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:32.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.982+0000 7f2bb8fd9f40 1 --2- 172.21.15.38:0/2097964583 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2b9c003bd0 0x55b74e3fa2f0 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f2ba400b7c0 tx=0x7f2ba4004880 comp rx=0 tx=0).stop 2024-09-21T03:41:32.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.982+0000 7f2bb8fd9f40 1 -- 172.21.15.38:0/2097964583 shutdown_connections 2024-09-21T03:41:32.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.982+0000 7f2bb8fd9f40 1 --2- 172.21.15.38:0/2097964583 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b74e3dbd60 0x55b74e3218e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:32.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.982+0000 7f2bb8fd9f40 1 --2- 172.21.15.38:0/2097964583 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b74e3db950 0x55b74e34e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:32.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.982+0000 7f2bb8fd9f40 1 --2- 172.21.15.38:0/2097964583 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2b9c003bd0 0x55b74e3fa2f0 unknown :-1 s=CLOSED pgs=1515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:32.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.982+0000 7f2bb8fd9f40 1 -- 172.21.15.38:0/2097964583 >> 172.21.15.38:0/2097964583 conn(0x55b74e244ac0 msgr2=0x55b74e325be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:32.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.982+0000 7f2bb8fd9f40 1 -- 172.21.15.38:0/2097964583 shutdown_connections 2024-09-21T03:41:32.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.982+0000 7f2bb8fd9f40 1 -- 172.21.15.38:0/2097964583 wait complete. 2024-09-21T03:41:32.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.983+0000 7f2bb8fd9f40 1 Processor -- start 2024-09-21T03:41:32.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.983+0000 7f2bb8fd9f40 1 -- start start 2024-09-21T03:41:32.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.985+0000 7f2bb8fd9f40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b74e3db950 0x55b74e3df790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:32.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.985+0000 7f2bb8fd9f40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b74e3dbd60 0x55b74e496700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:32.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.985+0000 7f2bb8fd9f40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2b9c003bd0 0x55b74e34e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:32.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.985+0000 7f2bb8fd9f40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55b74e3500f0 con 0x55b74e3dbd60 2024-09-21T03:41:32.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.985+0000 7f2bb8fd9f40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55b74e3f8200 con 0x7f2b9c003bd0 2024-09-21T03:41:32.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.985+0000 7f2bb8fd9f40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55b74e3d72c0 con 0x55b74e3db950 2024-09-21T03:41:32.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.985+0000 7f2bb2292640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b74e3dbd60 0x55b74e496700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:32.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.985+0000 7f2bb2292640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b74e3dbd60 0x55b74e496700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54216/0 (socket says 172.21.15.38:54216) 2024-09-21T03:41:32.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.985+0000 7f2bb2292640 1 -- 172.21.15.38:0/2486467375 learned_addr learned my addr 172.21.15.38:0/2486467375 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:41:32.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.985+0000 7f2bb2a93640 1 --2- 172.21.15.38:0/2486467375 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b74e3db950 0x55b74e3df790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:32.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.985+0000 7f2bb3294640 1 --2- 172.21.15.38:0/2486467375 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2b9c003bd0 0x55b74e34e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:32.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.985+0000 7f2bb2292640 1 -- 172.21.15.38:0/2486467375 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b74e3db950 msgr2=0x55b74e3df790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:32.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.985+0000 7f2bb2292640 1 --2- 172.21.15.38:0/2486467375 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b74e3db950 0x55b74e3df790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:32.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.985+0000 7f2bb2292640 1 -- 172.21.15.38:0/2486467375 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2b9c003bd0 msgr2=0x55b74e34e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:32.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.985+0000 7f2bb2292640 1 --2- 172.21.15.38:0/2486467375 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2b9c003bd0 0x55b74e34e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:32.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.985+0000 7f2bb2292640 1 -- 172.21.15.38:0/2486467375 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b74e492ad0 con 0x55b74e3dbd60 2024-09-21T03:41:32.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.985+0000 7f2bb2292640 1 --2- 172.21.15.38:0/2486467375 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b74e3dbd60 0x55b74e496700 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f2ba800e3c0 tx=0x7f2ba800e880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:32.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.985+0000 7f2b9bfff640 1 -- 172.21.15.38:0/2486467375 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ba800f8d0 con 0x55b74e3dbd60 2024-09-21T03:41:32.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.985+0000 7f2b9bfff640 1 -- 172.21.15.38:0/2486467375 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2ba8013030 con 0x55b74e3dbd60 2024-09-21T03:41:32.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.985+0000 7f2bb8fd9f40 1 -- 172.21.15.38:0/2486467375 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55b74e492db0 con 0x55b74e3dbd60 2024-09-21T03:41:32.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.985+0000 7f2b9bfff640 1 -- 172.21.15.38:0/2486467375 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ba800fcb0 con 0x55b74e3dbd60 2024-09-21T03:41:32.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.985+0000 7f2bb8fd9f40 1 -- 172.21.15.38:0/2486467375 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b74e493370 con 0x55b74e3dbd60 2024-09-21T03:41:32.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.987+0000 7f2b9bfff640 1 -- 172.21.15.38:0/2486467375 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f2ba80173d0 con 0x55b74e3dbd60 2024-09-21T03:41:32.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.988+0000 7f2b9bfff640 1 --2- 172.21.15.38:0/2486467375 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2ba0077570 0x7f2ba0079a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:32.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.988+0000 7f2b9bfff640 1 -- 172.21.15.38:0/2486467375 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 7336+0+0 (secure 0 0 0) 0x7f2ba809a2a0 con 0x55b74e3dbd60 2024-09-21T03:41:32.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.988+0000 7f2bb2a93640 1 --2- 172.21.15.38:0/2486467375 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2ba0077570 0x7f2ba0079a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:32.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.988+0000 7f2bb8fd9f40 1 --2- 172.21.15.38:0/2486467375 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x55b74e496c40 0x55b74e3ece00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:32.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.988+0000 7f2bb8fd9f40 1 -- 172.21.15.38:0/2486467375 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:1 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e108) -- 0x55b74e3dfcd0 con 0x55b74e496c40 2024-09-21T03:41:32.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.988+0000 7f2bb3294640 1 --2- 172.21.15.38:0/2486467375 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x55b74e496c40 0x55b74e3ece00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:32.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.988+0000 7f2bb2a93640 1 --2- 172.21.15.38:0/2486467375 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2ba0077570 0x7f2ba0079a80 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f2ba4000c00 tx=0x7f2ba4019040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:32.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.988+0000 7f2bb3294640 1 --2- 172.21.15.38:0/2486467375 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x55b74e496c40 0x55b74e3ece00 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:32.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.990+0000 7f2bb3294640 1 -- 172.21.15.38:0/2486467375 <== osd.2 v2:172.21.15.38:6824/2577858902 1 ==== osd_op_reply(1 400.00000000 [delete] v108'62 uv62 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2bac003070 con 0x55b74e496c40 2024-09-21T03:41:32.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.990+0000 7f2bb8fd9f40 1 -- 172.21.15.38:0/2486467375 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x55b74e496c40 msgr2=0x55b74e3ece00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:32.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.990+0000 7f2bb8fd9f40 1 --2- 172.21.15.38:0/2486467375 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x55b74e496c40 0x55b74e3ece00 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:32.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.990+0000 7f2bb8fd9f40 1 -- 172.21.15.38:0/2486467375 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2ba0077570 msgr2=0x7f2ba0079a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:32.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.990+0000 7f2bb8fd9f40 1 --2- 172.21.15.38:0/2486467375 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2ba0077570 0x7f2ba0079a80 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f2ba4000c00 tx=0x7f2ba4019040 comp rx=0 tx=0).stop 2024-09-21T03:41:32.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.990+0000 7f2bb8fd9f40 1 -- 172.21.15.38:0/2486467375 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b74e3dbd60 msgr2=0x55b74e496700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:32.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.990+0000 7f2bb8fd9f40 1 --2- 172.21.15.38:0/2486467375 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b74e3dbd60 0x55b74e496700 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f2ba800e3c0 tx=0x7f2ba800e880 comp rx=0 tx=0).stop 2024-09-21T03:41:32.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.990+0000 7f2bb3294640 1 -- 172.21.15.38:0/2486467375 reap_dead start 2024-09-21T03:41:32.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.990+0000 7f2bb8fd9f40 1 -- 172.21.15.38:0/2486467375 shutdown_connections 2024-09-21T03:41:32.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.990+0000 7f2bb8fd9f40 1 -- 172.21.15.38:0/2486467375 >> 172.21.15.38:0/2486467375 conn(0x55b74e244ac0 msgr2=0x55b74e3f89a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:32.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.990+0000 7f2bb8fd9f40 1 -- 172.21.15.38:0/2486467375 shutdown_connections 2024-09-21T03:41:32.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:32.990+0000 7f2bb8fd9f40 1 -- 172.21.15.38:0/2486467375 wait complete. 2024-09-21T03:41:32.995 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-21T03:41:33.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:33.113+0000 7f477a99d640 1 -- 172.21.15.38:0/3388309933 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4774162710 msgr2=0x7f47740cf300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:33.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:33.113+0000 7f477a99d640 1 --2- 172.21.15.38:0/3388309933 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4774162710 0x7f47740cf300 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f4764005fd0 tx=0x7f476402a540 comp rx=0 tx=0).stop 2024-09-21T03:41:33.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:33.113+0000 7f477a99d640 1 -- 172.21.15.38:0/3388309933 shutdown_connections 2024-09-21T03:41:33.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:33.113+0000 7f477a99d640 1 --2- 172.21.15.38:0/3388309933 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4774162710 0x7f47740cf300 unknown :-1 s=CLOSED pgs=1517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:33.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:33.113+0000 7f477a99d640 1 --2- 172.21.15.38:0/3388309933 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f477406d2b0 0x7f47741621d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:33.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:33.113+0000 7f477a99d640 1 -- 172.21.15.38:0/3388309933 >> 172.21.15.38:0/3388309933 conn(0x7f47740d1960 msgr2=0x7f47740d42e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:33.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:33.113+0000 7f477a99d640 1 -- 172.21.15.38:0/3388309933 shutdown_connections 2024-09-21T03:41:33.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:33.113+0000 7f477a99d640 1 -- 172.21.15.38:0/3388309933 wait complete. 2024-09-21T03:41:33.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:33.115+0000 7f477a99d640 1 Processor -- start 2024-09-21T03:41:33.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:33.115+0000 7f477a99d640 1 -- start start 2024-09-21T03:41:33.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:33.116+0000 7f477a99d640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f477406d2b0 0x7f477415b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:33.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:33.116+0000 7f477a99d640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f47740cf840 0x7f47740ca790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:33.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:33.116+0000 7f477a99d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4774162710 0x7f47740d2430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:33.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:33.116+0000 7f477a99d640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f4774155940 con 0x7f4774162710 2024-09-21T03:41:33.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:33.116+0000 7f477a99d640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f47741557c0 con 0x7f477406d2b0 2024-09-21T03:41:33.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:33.116+0000 7f477a99d640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f4774155ac0 con 0x7f47740cf840 2024-09-21T03:41:33.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:33.117+0000 7f477a19c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4774162710 0x7f47740d2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:33.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:33.117+0000 7f477999b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f477406d2b0 0x7f477415b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:33.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:33.117+0000 7f477919a640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f47740cf840 0x7f47740ca790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:33.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:33.117+0000 7f477919a640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f47740cf840 0x7f47740ca790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.38:44700/0 (socket says 172.21.15.38:44700) 2024-09-21T03:41:33.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:33.117+0000 7f477999b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f477406d2b0 0x7f477415b7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.38:37014/0 (socket says 172.21.15.38:37014) 2024-09-21T03:41:33.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:33.117+0000 7f477919a640 1 -- 172.21.15.38:0/1285645117 learned_addr learned my addr 172.21.15.38:0/1285645117 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:41:33.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:33.117+0000 7f477919a640 1 -- 172.21.15.38:0/1285645117 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f477406d2b0 msgr2=0x7f477415b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:33.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:33.117+0000 7f477919a640 1 --2- 172.21.15.38:0/1285645117 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f477406d2b0 0x7f477415b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:33.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:33.117+0000 7f477919a640 1 -- 172.21.15.38:0/1285645117 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4774162710 msgr2=0x7f47740d2430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:33.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:33.117+0000 7f477919a640 1 --2- 172.21.15.38:0/1285645117 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4774162710 0x7f47740d2430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:33.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:33.117+0000 7f477919a640 1 -- 172.21.15.38:0/1285645117 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4774156d20 con 0x7f47740cf840 2024-09-21T03:41:33.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:33.117+0000 7f477999b640 1 --2- 172.21.15.38:0/1285645117 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f477406d2b0 0x7f477415b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:41:33.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:33.117+0000 7f477a19c640 1 --2- 172.21.15.38:0/1285645117 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4774162710 0x7f47740d2430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:41:33.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:33.117+0000 7f477919a640 1 --2- 172.21.15.38:0/1285645117 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f47740cf840 0x7f47740ca790 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f4764000c00 tx=0x7f47640038a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:33.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:33.117+0000 7f476a7fc640 1 -- 172.21.15.38:0/1285645117 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f476403e030 con 0x7f47740cf840 2024-09-21T03:41:33.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:33.117+0000 7f476a7fc640 1 -- 172.21.15.38:0/1285645117 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4764003db0 con 0x7f47740cf840 2024-09-21T03:41:33.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:33.117+0000 7f477a99d640 1 -- 172.21.15.38:0/1285645117 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4774157000 con 0x7f47740cf840 2024-09-21T03:41:33.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:33.117+0000 7f476a7fc640 1 -- 172.21.15.38:0/1285645117 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4764035040 con 0x7f47740cf840 2024-09-21T03:41:33.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:33.117+0000 7f477a99d640 1 -- 172.21.15.38:0/1285645117 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f47741575c0 con 0x7f47740cf840 2024-09-21T03:41:33.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:33.118+0000 7f477a99d640 1 -- 172.21.15.38:0/1285645117 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47741621d0 con 0x7f47740cf840 2024-09-21T03:41:33.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:33.119+0000 7f476a7fc640 1 -- 172.21.15.38:0/1285645117 <== mon.2 v2:172.21.15.195:3301/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f4764041030 con 0x7f47740cf840 2024-09-21T03:41:33.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:33.120+0000 7f476a7fc640 1 --2- 172.21.15.38:0/1285645117 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4758078b90 0x7f475807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:33.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:33.120+0000 7f476a7fc640 1 -- 172.21.15.38:0/1285645117 <== mon.2 v2:172.21.15.195:3301/0 5 ==== osd_map(108..108 src has 1..108) ==== 7336+0+0 (secure 0 0 0) 0x7f47640bbb60 con 0x7f47740cf840 2024-09-21T03:41:33.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:33.120+0000 7f477999b640 1 --2- 172.21.15.38:0/1285645117 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4758078b90 0x7f475807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:33.123 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:33.120+0000 7f477999b640 1 --2- 172.21.15.38:0/1285645117 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4758078b90 0x7f475807b0a0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f4770009f20 tx=0x7f4770005f90 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:33.126 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:33.125+0000 7f476a7fc640 1 -- 172.21.15.38:0/1285645117 <== mon.2 v2:172.21.15.195:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f47640831d0 con 0x7f47740cf840 2024-09-21T03:41:33.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:33.350+0000 7f477a99d640 1 -- 172.21.15.38:0/1285645117 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f477415c040 con 0x7f47740cf840 2024-09-21T03:41:33.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:33.664+0000 7f476a7fc640 1 -- 172.21.15.38:0/1285645117 <== mon.2 v2:172.21.15.195:3301/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v157) ==== 162+0+0 (secure 0 0 0) 0x7f4764087c00 con 0x7f47740cf840 2024-09-21T03:41:33.667 INFO:teuthology.orchestra.run.smithi038.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-21T03:41:33.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:33.667+0000 7f477a99d640 1 -- 172.21.15.38:0/1285645117 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4758078b90 msgr2=0x7f475807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:33.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:33.667+0000 7f477a99d640 1 --2- 172.21.15.38:0/1285645117 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4758078b90 0x7f475807b0a0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f4770009f20 tx=0x7f4770005f90 comp rx=0 tx=0).stop 2024-09-21T03:41:33.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:33.667+0000 7f477a99d640 1 -- 172.21.15.38:0/1285645117 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f47740cf840 msgr2=0x7f47740ca790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:33.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:33.667+0000 7f477a99d640 1 --2- 172.21.15.38:0/1285645117 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f47740cf840 0x7f47740ca790 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f4764000c00 tx=0x7f47640038a0 comp rx=0 tx=0).stop 2024-09-21T03:41:33.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:33.667+0000 7f477a99d640 1 -- 172.21.15.38:0/1285645117 shutdown_connections 2024-09-21T03:41:33.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:33.667+0000 7f477a99d640 1 --2- 172.21.15.38:0/1285645117 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4774162710 0x7f47740d2430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:33.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:33.667+0000 7f477a99d640 1 --2- 172.21.15.38:0/1285645117 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4758078b90 0x7f475807b0a0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:33.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:33.667+0000 7f477a99d640 1 --2- 172.21.15.38:0/1285645117 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f47740cf840 0x7f47740ca790 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:33.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:33.667+0000 7f477a99d640 1 --2- 172.21.15.38:0/1285645117 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f477406d2b0 0x7f477415b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:33.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:33.667+0000 7f477a99d640 1 -- 172.21.15.38:0/1285645117 >> 172.21.15.38:0/1285645117 conn(0x7f47740d1960 msgr2=0x7f47740b7dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:33.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:33.667+0000 7f477a99d640 1 -- 172.21.15.38:0/1285645117 shutdown_connections 2024-09-21T03:41:33.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:33.667+0000 7f477a99d640 1 -- 172.21.15.38:0/1285645117 wait complete. 2024-09-21T03:41:34.086 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:41:34.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:34.209+0000 7f1f34ff1640 1 -- 172.21.15.38:0/989570677 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1f301536e0 msgr2=0x7f1f300d17c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:34.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:34.209+0000 7f1f34ff1640 1 --2- 172.21.15.38:0/989570677 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1f301536e0 0x7f1f300d17c0 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f1f300b29d0 tx=0x7f1f2002a560 comp rx=0 tx=0).stop 2024-09-21T03:41:34.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:34.210+0000 7f1f34ff1640 1 -- 172.21.15.38:0/989570677 shutdown_connections 2024-09-21T03:41:34.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:34.210+0000 7f1f34ff1640 1 --2- 172.21.15.38:0/989570677 >> v2:172.21.15.195:3300/0 conn(0x7f1f300cce70 0x7f1f30160190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:34.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:34.210+0000 7f1f34ff1640 1 --2- 172.21.15.38:0/989570677 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1f300d1d00 0x7f1f300cc930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:34.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:34.210+0000 7f1f34ff1640 1 --2- 172.21.15.38:0/989570677 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1f301536e0 0x7f1f300d17c0 unknown :-1 s=CLOSED pgs=1518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:34.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:34.210+0000 7f1f34ff1640 1 -- 172.21.15.38:0/989570677 >> 172.21.15.38:0/989570677 conn(0x7f1f300cef40 msgr2=0x7f1f300cf350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:34.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:34.210+0000 7f1f34ff1640 1 -- 172.21.15.38:0/989570677 shutdown_connections 2024-09-21T03:41:34.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:34.210+0000 7f1f34ff1640 1 -- 172.21.15.38:0/989570677 wait complete. 2024-09-21T03:41:34.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:34.212+0000 7f1f34ff1640 1 Processor -- start 2024-09-21T03:41:34.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:34.212+0000 7f1f34ff1640 1 -- start start 2024-09-21T03:41:34.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:34.213+0000 7f1f34ff1640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1f300cce70 0x7f1f301550a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:34.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:34.213+0000 7f1f34ff1640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1f300d1d00 0x7f1f300b68b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:34.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:34.213+0000 7f1f34ff1640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1f301536e0 0x7f1f300b6df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:34.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:34.213+0000 7f1f34ff1640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f1f301606d0 con 0x7f1f300d1d00 2024-09-21T03:41:34.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:34.213+0000 7f1f34ff1640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f1f30160850 con 0x7f1f301536e0 2024-09-21T03:41:34.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:34.213+0000 7f1f34ff1640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f1f301609d0 con 0x7f1f300cce70 2024-09-21T03:41:34.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:34.213+0000 7f1f27fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1f300d1d00 0x7f1f300b68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:34.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:34.213+0000 7f1f27fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1f300d1d00 0x7f1f300b68b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54252/0 (socket says 172.21.15.38:54252) 2024-09-21T03:41:34.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:34.213+0000 7f1f27fff640 1 -- 172.21.15.38:0/3641509986 learned_addr learned my addr 172.21.15.38:0/3641509986 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:41:34.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:34.213+0000 7f1f2ed9d640 1 --2- 172.21.15.38:0/3641509986 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1f301536e0 0x7f1f300b6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:34.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:34.213+0000 7f1f2e59c640 1 --2- 172.21.15.38:0/3641509986 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1f300cce70 0x7f1f301550a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:34.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:34.213+0000 7f1f27fff640 1 -- 172.21.15.38:0/3641509986 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1f300cce70 msgr2=0x7f1f301550a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:34.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:34.213+0000 7f1f27fff640 1 --2- 172.21.15.38:0/3641509986 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1f300cce70 0x7f1f301550a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:34.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:34.213+0000 7f1f27fff640 1 -- 172.21.15.38:0/3641509986 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1f301536e0 msgr2=0x7f1f300b6df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:34.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:34.213+0000 7f1f27fff640 1 --2- 172.21.15.38:0/3641509986 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1f301536e0 0x7f1f300b6df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:34.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:34.213+0000 7f1f27fff640 1 -- 172.21.15.38:0/3641509986 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f301555e0 con 0x7f1f300d1d00 2024-09-21T03:41:34.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:34.214+0000 7f1f27fff640 1 --2- 172.21.15.38:0/3641509986 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1f300d1d00 0x7f1f300b68b0 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f1f1800c940 tx=0x7f1f1800ce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:34.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:34.214+0000 7f1f26ffd640 1 -- 172.21.15.38:0/3641509986 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f18004220 con 0x7f1f300d1d00 2024-09-21T03:41:34.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:34.214+0000 7f1f26ffd640 1 -- 172.21.15.38:0/3641509986 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1f18007950 con 0x7f1f300d1d00 2024-09-21T03:41:34.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:34.214+0000 7f1f26ffd640 1 -- 172.21.15.38:0/3641509986 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f18007d80 con 0x7f1f300d1d00 2024-09-21T03:41:34.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:34.214+0000 7f1f34ff1640 1 -- 172.21.15.38:0/3641509986 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f30156ad0 con 0x7f1f300d1d00 2024-09-21T03:41:34.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:34.214+0000 7f1f34ff1640 1 -- 172.21.15.38:0/3641509986 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f30156230 con 0x7f1f300d1d00 2024-09-21T03:41:34.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:34.215+0000 7f1f34ff1640 1 -- 172.21.15.38:0/3641509986 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f30160ef0 con 0x7f1f300d1d00 2024-09-21T03:41:34.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:34.216+0000 7f1f26ffd640 1 -- 172.21.15.38:0/3641509986 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f1f18005020 con 0x7f1f300d1d00 2024-09-21T03:41:34.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:34.216+0000 7f1f26ffd640 1 --2- 172.21.15.38:0/3641509986 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1f10078bc0 0x7f1f1007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:34.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:34.216+0000 7f1f26ffd640 1 -- 172.21.15.38:0/3641509986 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 7336+0+0 (secure 0 0 0) 0x7f1f1809b620 con 0x7f1f300d1d00 2024-09-21T03:41:34.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:34.216+0000 7f1f2e59c640 1 --2- 172.21.15.38:0/3641509986 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1f10078bc0 0x7f1f1007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:34.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:34.217+0000 7f1f2e59c640 1 --2- 172.21.15.38:0/3641509986 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1f10078bc0 0x7f1f1007b0d0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f1f20033000 tx=0x7f1f20004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:34.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:34.221+0000 7f1f26ffd640 1 -- 172.21.15.38:0/3641509986 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f1f18014030 con 0x7f1f300d1d00 2024-09-21T03:41:34.391 INFO:tasks.ceph.mds.c.smithi038.stderr: -54> 2024-09-21T03:41:29.638+0000 7f4d93216640 -1 mds.0.sessionmap _load_finish got (2) No such file or directory 2024-09-21T03:41:34.391 INFO:tasks.ceph.mds.c.smithi038.stderr: -53> 2024-09-21T03:41:29.638+0000 7f4d93216640 -1 log_channel(cluster) log [ERR] : error reading sessionmap 'mds0_sessionmap' -2 ((2) No such file or directory) 2024-09-21T03:41:34.445 INFO:tasks.ceph.mds.c.smithi038.stdout:starting mds.c at 2024-09-21T03:41:34.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:34.445+0000 7f1f34ff1640 1 -- 172.21.15.38:0/3641509986 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1f300701a0 con 0x7f1f300d1d00 2024-09-21T03:41:34.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:34.446+0000 7f1f26ffd640 1 -- 172.21.15.38:0/3641509986 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 158 v158) ==== 95+0+4401 (secure 0 0 0) 0x7f1f18062dc0 con 0x7f1f300d1d00 2024-09-21T03:41:34.448 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:41:34.448 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":158,"btime":"2024-09-21T03:41:33:664652+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6086,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/965972663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":965972663},{"type":"v1","addr":"172.21.15.195:6836","nonce":965972663}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6700,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/3176122334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":3176122334},{"type":"v1","addr":"172.21.15.195:6837","nonce":3176122334}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":158,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:40:02.319282+0000","modified":"2024-09-21T03:41:33.664640+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":108,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8370},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8370":{"gid":8370,"name":"a","rank":0,"incarnation":158,"state":"up:replay","state_seq":1,"addr":"172.21.15.38:6835/2916238332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":2916238332},{"type":"v1","addr":"172.21.15.38:6835","nonce":2916238332}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-21T03:41:34.448 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 158 2024-09-21T03:41:34.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:34.449+0000 7f1f34ff1640 1 -- 172.21.15.38:0/3641509986 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1f10078bc0 msgr2=0x7f1f1007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:34.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:34.449+0000 7f1f34ff1640 1 --2- 172.21.15.38:0/3641509986 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1f10078bc0 0x7f1f1007b0d0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f1f20033000 tx=0x7f1f20004730 comp rx=0 tx=0).stop 2024-09-21T03:41:34.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:34.449+0000 7f1f34ff1640 1 -- 172.21.15.38:0/3641509986 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1f300d1d00 msgr2=0x7f1f300b68b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:34.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:34.449+0000 7f1f34ff1640 1 --2- 172.21.15.38:0/3641509986 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1f300d1d00 0x7f1f300b68b0 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f1f1800c940 tx=0x7f1f1800ce00 comp rx=0 tx=0).stop 2024-09-21T03:41:34.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:34.449+0000 7f1f34ff1640 1 -- 172.21.15.38:0/3641509986 shutdown_connections 2024-09-21T03:41:34.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:34.449+0000 7f1f34ff1640 1 --2- 172.21.15.38:0/3641509986 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1f301536e0 0x7f1f300b6df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:34.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:34.449+0000 7f1f34ff1640 1 --2- 172.21.15.38:0/3641509986 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1f300d1d00 0x7f1f300b68b0 unknown :-1 s=CLOSED pgs=1519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:34.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:34.449+0000 7f1f34ff1640 1 --2- 172.21.15.38:0/3641509986 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1f10078bc0 0x7f1f1007b0d0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:34.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:34.449+0000 7f1f34ff1640 1 --2- 172.21.15.38:0/3641509986 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1f300cce70 0x7f1f301550a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:34.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:34.449+0000 7f1f34ff1640 1 -- 172.21.15.38:0/3641509986 >> 172.21.15.38:0/3641509986 conn(0x7f1f300cef40 msgr2=0x7f1f300cf350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:34.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:34.449+0000 7f1f34ff1640 1 -- 172.21.15.38:0/3641509986 shutdown_connections 2024-09-21T03:41:34.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:34.449+0000 7f1f34ff1640 1 -- 172.21.15.38:0/3641509986 wait complete. 2024-09-21T03:41:34.462 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 158, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T03:40:02.319282+0000', 'modified': '2024-09-21T03:41:33.664640+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 108, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8370}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8370': {'gid': 8370, 'name': 'a', 'rank': 0, 'incarnation': 158, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.38:6835/2916238332', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.38:6834', 'nonce': 2916238332}, {'type': 'v1', 'addr': '172.21.15.38:6835', 'nonce': 2916238332}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-09-21T03:41:34.462 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8370:up:replay 2024-09-21T03:41:34.462 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2024-09-21T03:41:39.866 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:41:39.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:39.995+0000 7f5c6989b640 1 -- 172.21.15.38:0/1893543159 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5c640cd040 msgr2=0x7f5c640d04c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:39.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:39.995+0000 7f5c6989b640 1 --2- 172.21.15.38:0/1893543159 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5c640cd040 0x7f5c640d04c0 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7f5c640b4ab0 tx=0x7f5c5402a560 comp rx=0 tx=0).stop 2024-09-21T03:41:39.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:39.995+0000 7f5c6989b640 1 -- 172.21.15.38:0/1893543159 shutdown_connections 2024-09-21T03:41:39.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:39.995+0000 7f5c6989b640 1 --2- 172.21.15.38:0/1893543159 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5c640cd040 0x7f5c640d04c0 unknown :-1 s=CLOSED pgs=1523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:39.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:39.995+0000 7f5c6989b640 1 --2- 172.21.15.38:0/1893543159 >> v2:172.21.15.195:3300/0 conn(0x7f5c641536e0 0x7f5c640d17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:39.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:39.995+0000 7f5c6989b640 1 -- 172.21.15.38:0/1893543159 >> 172.21.15.38:0/1893543159 conn(0x7f5c640cef40 msgr2=0x7f5c640cf350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:39.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:39.995+0000 7f5c6989b640 1 -- 172.21.15.38:0/1893543159 shutdown_connections 2024-09-21T03:41:39.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:39.995+0000 7f5c6989b640 1 -- 172.21.15.38:0/1893543159 wait complete. 2024-09-21T03:41:39.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:39.997+0000 7f5c6989b640 1 Processor -- start 2024-09-21T03:41:39.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:39.998+0000 7f5c6989b640 1 -- start start 2024-09-21T03:41:40.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:39.999+0000 7f5c6989b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5c640d1d00 0x7f5c640c33e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:40.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:39.999+0000 7f5c6989b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5c641536e0 0x7f5c641f49c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:40.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:39.999+0000 7f5c6989b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5c640cb770 0x7f5c640cbb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:40.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:39.999+0000 7f5c6989b640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f5c640d0d90 con 0x7f5c641536e0 2024-09-21T03:41:40.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:39.999+0000 7f5c6989b640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f5c640d0c10 con 0x7f5c640cb770 2024-09-21T03:41:40.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:39.999+0000 7f5c6989b640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f5c640d0f10 con 0x7f5c640d1d00 2024-09-21T03:41:40.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:39.999+0000 7f5c627fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5c641536e0 0x7f5c641f49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:40.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:39.999+0000 7f5c627fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5c641536e0 0x7f5c641f49c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54286/0 (socket says 172.21.15.38:54286) 2024-09-21T03:41:40.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:39.999+0000 7f5c627fc640 1 -- 172.21.15.38:0/3032925947 learned_addr learned my addr 172.21.15.38:0/3032925947 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:41:40.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:39.999+0000 7f5c637fe640 1 --2- 172.21.15.38:0/3032925947 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5c640cb770 0x7f5c640cbb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:40.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:39.999+0000 7f5c62ffd640 1 --2- 172.21.15.38:0/3032925947 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5c640d1d00 0x7f5c640c33e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:40.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:39.999+0000 7f5c627fc640 1 -- 172.21.15.38:0/3032925947 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5c640d1d00 msgr2=0x7f5c640c33e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:40.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:39.999+0000 7f5c627fc640 1 --2- 172.21.15.38:0/3032925947 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5c640d1d00 0x7f5c640c33e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:40.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:39.999+0000 7f5c627fc640 1 -- 172.21.15.38:0/3032925947 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5c640cb770 msgr2=0x7f5c640cbb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:40.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:39.999+0000 7f5c627fc640 1 --2- 172.21.15.38:0/3032925947 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5c640cb770 0x7f5c640cbb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:40.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:39.999+0000 7f5c627fc640 1 -- 172.21.15.38:0/3032925947 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c640c3920 con 0x7f5c641536e0 2024-09-21T03:41:40.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:39.999+0000 7f5c62ffd640 1 --2- 172.21.15.38:0/3032925947 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5c640d1d00 0x7f5c640c33e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:41:40.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.000+0000 7f5c627fc640 1 --2- 172.21.15.38:0/3032925947 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5c641536e0 0x7f5c641f49c0 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7f5c4c00b3f0 tx=0x7f5c4c00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:40.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.000+0000 7f5c5bfff640 1 -- 172.21.15.38:0/3032925947 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c4c015020 con 0x7f5c641536e0 2024-09-21T03:41:40.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.000+0000 7f5c5bfff640 1 -- 172.21.15.38:0/3032925947 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5c4c011920 con 0x7f5c641536e0 2024-09-21T03:41:40.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.000+0000 7f5c6989b640 1 -- 172.21.15.38:0/3032925947 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c640cc0c0 con 0x7f5c641536e0 2024-09-21T03:41:40.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.000+0000 7f5c5bfff640 1 -- 172.21.15.38:0/3032925947 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c4c011d70 con 0x7f5c641536e0 2024-09-21T03:41:40.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.000+0000 7f5c6989b640 1 -- 172.21.15.38:0/3032925947 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c6414c200 con 0x7f5c641536e0 2024-09-21T03:41:40.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.001+0000 7f5c6989b640 1 -- 172.21.15.38:0/3032925947 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c640d04c0 con 0x7f5c641536e0 2024-09-21T03:41:40.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.002+0000 7f5c5bfff640 1 -- 172.21.15.38:0/3032925947 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f5c4c018020 con 0x7f5c641536e0 2024-09-21T03:41:40.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.002+0000 7f5c5bfff640 1 --2- 172.21.15.38:0/3032925947 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5c44078b90 0x7f5c4407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:40.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.002+0000 7f5c5bfff640 1 -- 172.21.15.38:0/3032925947 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 7336+0+0 (secure 0 0 0) 0x7f5c4c09b170 con 0x7f5c641536e0 2024-09-21T03:41:40.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.002+0000 7f5c62ffd640 1 --2- 172.21.15.38:0/3032925947 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5c44078b90 0x7f5c4407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:40.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.003+0000 7f5c62ffd640 1 --2- 172.21.15.38:0/3032925947 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5c44078b90 0x7f5c4407b0a0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f5c5c005fd0 tx=0x7f5c5c009340 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:40.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.007+0000 7f5c5bfff640 1 -- 172.21.15.38:0/3032925947 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f5c4c01f030 con 0x7f5c641536e0 2024-09-21T03:41:40.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.231+0000 7f5c6989b640 1 -- 172.21.15.38:0/3032925947 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5c64070860 con 0x7f5c641536e0 2024-09-21T03:41:40.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.232+0000 7f5c5bfff640 1 -- 172.21.15.38:0/3032925947 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 161 v161) ==== 95+0+5234 (secure 0 0 0) 0x7f5c640cc0c0 con 0x7f5c641536e0 2024-09-21T03:41:40.234 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:41:40.234 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":161,"btime":"2024-09-21T03:41:36:689921+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6086,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/965972663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":965972663},{"type":"v1","addr":"172.21.15.195:6836","nonce":965972663}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6700,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/3176122334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":3176122334},{"type":"v1","addr":"172.21.15.195:6837","nonce":3176122334}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":8505,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/1717955365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":1717955365},{"type":"v1","addr":"172.21.15.38:6833","nonce":1717955365}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":159}],"filesystems":[{"mdsmap":{"epoch":161,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:40:02.319282+0000","modified":"2024-09-21T03:41:36.689909+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":108,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8370},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8370":{"gid":8370,"name":"a","rank":0,"incarnation":158,"state":"up:active","state_seq":7,"addr":"172.21.15.38:6835/2916238332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":2916238332},{"type":"v1","addr":"172.21.15.38:6835","nonce":2916238332}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8370,"qdb_cluster":[8370]},"id":6}]} 2024-09-21T03:41:40.234 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 161 2024-09-21T03:41:40.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.234+0000 7f5c6989b640 1 -- 172.21.15.38:0/3032925947 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5c44078b90 msgr2=0x7f5c4407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:40.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.234+0000 7f5c6989b640 1 --2- 172.21.15.38:0/3032925947 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5c44078b90 0x7f5c4407b0a0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f5c5c005fd0 tx=0x7f5c5c009340 comp rx=0 tx=0).stop 2024-09-21T03:41:40.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.234+0000 7f5c6989b640 1 -- 172.21.15.38:0/3032925947 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5c641536e0 msgr2=0x7f5c641f49c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:40.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.234+0000 7f5c6989b640 1 --2- 172.21.15.38:0/3032925947 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5c641536e0 0x7f5c641f49c0 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7f5c4c00b3f0 tx=0x7f5c4c00b8b0 comp rx=0 tx=0).stop 2024-09-21T03:41:40.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.234+0000 7f5c6989b640 1 -- 172.21.15.38:0/3032925947 shutdown_connections 2024-09-21T03:41:40.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.234+0000 7f5c6989b640 1 --2- 172.21.15.38:0/3032925947 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5c640cb770 0x7f5c640cbb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:40.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.234+0000 7f5c6989b640 1 --2- 172.21.15.38:0/3032925947 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5c641536e0 0x7f5c641f49c0 unknown :-1 s=CLOSED pgs=1524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:40.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.234+0000 7f5c6989b640 1 --2- 172.21.15.38:0/3032925947 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5c44078b90 0x7f5c4407b0a0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:40.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.234+0000 7f5c6989b640 1 --2- 172.21.15.38:0/3032925947 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5c640d1d00 0x7f5c640c33e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:40.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.234+0000 7f5c6989b640 1 -- 172.21.15.38:0/3032925947 >> 172.21.15.38:0/3032925947 conn(0x7f5c640cef40 msgr2=0x7f5c640ccb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:40.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.234+0000 7f5c6989b640 1 -- 172.21.15.38:0/3032925947 shutdown_connections 2024-09-21T03:41:40.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.234+0000 7f5c6989b640 1 -- 172.21.15.38:0/3032925947 wait complete. 2024-09-21T03:41:40.247 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 161, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T03:40:02.319282+0000', 'modified': '2024-09-21T03:41:36.689909+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 108, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8370}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8370': {'gid': 8370, 'name': 'a', 'rank': 0, 'incarnation': 158, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.38:6835/2916238332', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.38:6834', 'nonce': 2916238332}, {'type': 'v1', 'addr': '172.21.15.38:6835', 'nonce': 2916238332}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 8370, 'qdb_cluster': [8370]} 2024-09-21T03:41:40.247 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-21T03:41:40.247 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-21T03:41:40.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.363+0000 7faeae3c3640 1 -- 172.21.15.38:0/1788398472 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faea815bfc0 msgr2=0x7faea815f630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:40.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.363+0000 7faeae3c3640 1 --2- 172.21.15.38:0/1788398472 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faea815bfc0 0x7faea815f630 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7faea0005fd0 tx=0x7faea002a8b0 comp rx=0 tx=0).stop 2024-09-21T03:41:40.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.363+0000 7faeae3c3640 1 -- 172.21.15.38:0/1788398472 shutdown_connections 2024-09-21T03:41:40.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.363+0000 7faeae3c3640 1 --2- 172.21.15.38:0/1788398472 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faea815bfc0 0x7faea815f630 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:40.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.363+0000 7faeae3c3640 1 --2- 172.21.15.38:0/1788398472 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faea80b5260 0x7faea815ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:40.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.363+0000 7faeae3c3640 1 -- 172.21.15.38:0/1788398472 >> 172.21.15.38:0/1788398472 conn(0x7faea80cdca0 msgr2=0x7faea80ce0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:40.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.363+0000 7faeae3c3640 1 -- 172.21.15.38:0/1788398472 shutdown_connections 2024-09-21T03:41:40.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.363+0000 7faeae3c3640 1 -- 172.21.15.38:0/1788398472 wait complete. 2024-09-21T03:41:40.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.365+0000 7faeae3c3640 1 Processor -- start 2024-09-21T03:41:40.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.365+0000 7faeae3c3640 1 -- start start 2024-09-21T03:41:40.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.366+0000 7faeae3c3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faea806def0 0x7faea80b1670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:40.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.366+0000 7faeae3c3640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faea80b5260 0x7faea81f6ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:40.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.366+0000 7faeae3c3640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faea815bfc0 0x7faea8155250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:40.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.366+0000 7faeae3c3640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7faea815ff40 con 0x7faea806def0 2024-09-21T03:41:40.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.366+0000 7faeae3c3640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7faea815fdc0 con 0x7faea815bfc0 2024-09-21T03:41:40.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.366+0000 7faeae3c3640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7faea8071300 con 0x7faea80b5260 2024-09-21T03:41:40.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.366+0000 7faea7fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faea806def0 0x7faea80b1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:40.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.366+0000 7faea7fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faea806def0 0x7faea80b1670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54296/0 (socket says 172.21.15.38:54296) 2024-09-21T03:41:40.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.366+0000 7faea7fff640 1 -- 172.21.15.38:0/2298216813 learned_addr learned my addr 172.21.15.38:0/2298216813 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:41:40.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.366+0000 7faeac960640 1 --2- 172.21.15.38:0/2298216813 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faea815bfc0 0x7faea8155250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:40.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.366+0000 7faea77fe640 1 --2- 172.21.15.38:0/2298216813 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faea80b5260 0x7faea81f6ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:40.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.367+0000 7faea7fff640 1 -- 172.21.15.38:0/2298216813 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faea80b5260 msgr2=0x7faea81f6ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:40.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.367+0000 7faea7fff640 1 --2- 172.21.15.38:0/2298216813 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faea80b5260 0x7faea81f6ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:40.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.367+0000 7faea7fff640 1 -- 172.21.15.38:0/2298216813 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faea815bfc0 msgr2=0x7faea8155250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:40.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.367+0000 7faea7fff640 1 --2- 172.21.15.38:0/2298216813 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faea815bfc0 0x7faea8155250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:40.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.367+0000 7faea7fff640 1 -- 172.21.15.38:0/2298216813 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faea80b1bb0 con 0x7faea806def0 2024-09-21T03:41:40.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.367+0000 7faea7fff640 1 --2- 172.21.15.38:0/2298216813 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faea806def0 0x7faea80b1670 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7fae9800c340 tx=0x7fae9800c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:40.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.367+0000 7faea4ff9640 1 -- 172.21.15.38:0/2298216813 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae98006890 con 0x7faea806def0 2024-09-21T03:41:40.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.367+0000 7faeae3c3640 1 -- 172.21.15.38:0/2298216813 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faea80b2030 con 0x7faea806def0 2024-09-21T03:41:40.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.367+0000 7faea4ff9640 1 -- 172.21.15.38:0/2298216813 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fae98006b40 con 0x7faea806def0 2024-09-21T03:41:40.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.367+0000 7faea4ff9640 1 -- 172.21.15.38:0/2298216813 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae98007020 con 0x7faea806def0 2024-09-21T03:41:40.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.367+0000 7faeae3c3640 1 -- 172.21.15.38:0/2298216813 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faea81f4540 con 0x7faea806def0 2024-09-21T03:41:40.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.369+0000 7faea4ff9640 1 -- 172.21.15.38:0/2298216813 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fae98019020 con 0x7faea806def0 2024-09-21T03:41:40.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.369+0000 7faea4ff9640 1 --2- 172.21.15.38:0/2298216813 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fae94078b90 0x7fae9407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:40.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.370+0000 7faea4ff9640 1 -- 172.21.15.38:0/2298216813 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 7336+0+0 (secure 0 0 0) 0x7fae9809a910 con 0x7faea806def0 2024-09-21T03:41:40.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.370+0000 7faea77fe640 1 --2- 172.21.15.38:0/2298216813 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fae94078b90 0x7fae9407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:40.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.370+0000 7faea77fe640 1 --2- 172.21.15.38:0/2298216813 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fae94078b90 0x7fae9407b0a0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7faea80cabc0 tx=0x7fae9c005f90 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:40.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.370+0000 7faeb5411740 1 Processor -- start 2024-09-21T03:41:40.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.370+0000 7faeb5411740 1 -- start start 2024-09-21T03:41:40.374 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.372+0000 7faeb5411740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f3d3e56270 0x55f3d4227fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:40.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.372+0000 7faeb5411740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f3d3e2d3b0 0x55f3d42284e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:40.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.372+0000 7faeb5411740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f3d3e35590 0x55f3d42453e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:40.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.372+0000 7faeb5411740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55f3d4170820 con 0x55f3d3e56270 2024-09-21T03:41:40.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.372+0000 7faeb5411740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55f3d415fc50 con 0x55f3d3e35590 2024-09-21T03:41:40.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.372+0000 7faeb5411740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55f3d4163a60 con 0x55f3d3e2d3b0 2024-09-21T03:41:40.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.372+0000 7faea7fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f3d3e56270 0x55f3d4227fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:40.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.372+0000 7faea7fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f3d3e56270 0x55f3d4227fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54306/0 (socket says 172.21.15.38:54306) 2024-09-21T03:41:40.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.372+0000 7faea7fff640 1 -- 172.21.15.38:0/824302710 learned_addr learned my addr 172.21.15.38:0/824302710 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:41:40.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.372+0000 7faea77fe640 1 --2- 172.21.15.38:0/824302710 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f3d3e2d3b0 0x55f3d42284e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:40.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.372+0000 7faea7fff640 1 -- 172.21.15.38:0/824302710 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f3d3e2d3b0 msgr2=0x55f3d42284e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:40.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.372+0000 7faea7fff640 1 --2- 172.21.15.38:0/824302710 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f3d3e2d3b0 0x55f3d42284e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:40.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.372+0000 7faea7fff640 1 -- 172.21.15.38:0/824302710 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f3d3e35590 msgr2=0x55f3d42453e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:40.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.372+0000 7faeac960640 1 --2- 172.21.15.38:0/824302710 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f3d3e35590 0x55f3d42453e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:40.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.372+0000 7faea7fff640 1 --2- 172.21.15.38:0/824302710 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f3d3e35590 0x55f3d42453e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:40.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.372+0000 7faea7fff640 1 -- 172.21.15.38:0/824302710 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3d3ec1850 con 0x55f3d3e56270 2024-09-21T03:41:40.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.372+0000 7faea7fff640 1 --2- 172.21.15.38:0/824302710 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f3d3e56270 0x55f3d4227fa0 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7fae9800cfd0 tx=0x7fae98061820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94ead3753618a4c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:40.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.372+0000 7fae8dffb640 1 -- 172.21.15.38:0/824302710 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae98066ae0 con 0x55f3d3e56270 2024-09-21T03:41:40.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.372+0000 7fae8dffb640 1 -- 172.21.15.38:0/824302710 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fae9806b990 con 0x55f3d3e56270 2024-09-21T03:41:40.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.372+0000 7fae8dffb640 1 -- 172.21.15.38:0/824302710 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae98061530 con 0x55f3d3e56270 2024-09-21T03:41:40.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.373+0000 7faeb5411740 1 -- 172.21.15.38:0/824302710 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f3d3e56270 msgr2=0x55f3d4227fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:40.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.373+0000 7faeb5411740 1 --2- 172.21.15.38:0/824302710 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f3d3e56270 0x55f3d4227fa0 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7fae9800cfd0 tx=0x7fae98061820 comp rx=0 tx=0).stop 2024-09-21T03:41:40.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.373+0000 7faeb5411740 1 -- 172.21.15.38:0/824302710 shutdown_connections 2024-09-21T03:41:40.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.373+0000 7faeb5411740 1 --2- 172.21.15.38:0/824302710 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f3d3e35590 0x55f3d42453e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:40.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.373+0000 7faeb5411740 1 --2- 172.21.15.38:0/824302710 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f3d3e2d3b0 0x55f3d42284e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:40.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.373+0000 7faeb5411740 1 --2- 172.21.15.38:0/824302710 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f3d3e56270 0x55f3d4227fa0 unknown :-1 s=CLOSED pgs=1526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:40.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.373+0000 7faeb5411740 1 -- 172.21.15.38:0/824302710 >> 172.21.15.38:0/824302710 conn(0x55f3d3e4a3c0 msgr2=0x55f3d41fe260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:40.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.373+0000 7faeb5411740 1 -- 172.21.15.38:0/824302710 shutdown_connections 2024-09-21T03:41:40.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.373+0000 7faeb5411740 1 -- 172.21.15.38:0/824302710 wait complete. 2024-09-21T03:41:40.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.374+0000 7faeb5411740 1 -- start start 2024-09-21T03:41:40.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.374+0000 7faeb5411740 1 Processor -- start 2024-09-21T03:41:40.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.376+0000 7faeb5411740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f3d3e2d3b0 0x55f3d4215630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:40.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.376+0000 7faeb5411740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f3d3e35590 0x55f3d4215b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:40.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.376+0000 7faeb5411740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f3d3e56270 0x55f3d42553c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:40.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.376+0000 7faeb5411740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55f3d3ed0620 con 0x55f3d3e2d3b0 2024-09-21T03:41:40.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.376+0000 7faeb5411740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55f3d41055f0 con 0x55f3d3e35590 2024-09-21T03:41:40.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.376+0000 7faeb5411740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55f3d4216660 con 0x55f3d3e56270 2024-09-21T03:41:40.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.376+0000 7faeb5411740 10 client.0 _ll_register_callbacks cb 0x55f3d41090a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:41:40.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.376+0000 7faea7fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f3d3e2d3b0 0x55f3d4215630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:40.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.376+0000 7faea7fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f3d3e2d3b0 0x55f3d4215630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54318/0 (socket says 172.21.15.38:54318) 2024-09-21T03:41:40.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.376+0000 7faea7fff640 1 -- 172.21.15.38:0/2046379808 learned_addr learned my addr 172.21.15.38:0/2046379808 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:41:40.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.376+0000 7faea77fe640 1 --2- 172.21.15.38:0/2046379808 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f3d3e35590 0x55f3d4215b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:40.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.376+0000 7faeac960640 1 --2- 172.21.15.38:0/2046379808 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f3d3e56270 0x55f3d42553c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:40.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.376+0000 7faea7fff640 1 -- 172.21.15.38:0/2046379808 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f3d3e56270 msgr2=0x55f3d42553c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:40.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.376+0000 7faea7fff640 1 --2- 172.21.15.38:0/2046379808 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f3d3e56270 0x55f3d42553c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:40.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.376+0000 7faea7fff640 1 -- 172.21.15.38:0/2046379808 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f3d3e35590 msgr2=0x55f3d4215b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:40.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.376+0000 7faea7fff640 1 --2- 172.21.15.38:0/2046379808 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f3d3e35590 0x55f3d4215b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:40.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.376+0000 7faea7fff640 1 -- 172.21.15.38:0/2046379808 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3d3f386b0 con 0x55f3d3e2d3b0 2024-09-21T03:41:40.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.376+0000 7faeac960640 1 --2- 172.21.15.38:0/2046379808 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f3d3e56270 0x55f3d42553c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:41:40.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.376+0000 7faea7fff640 1 --2- 172.21.15.38:0/2046379808 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f3d3e2d3b0 0x55f3d4215630 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7fae98000fb0 tx=0x7fae9805e5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0fb984e47ada975 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:40.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.376+0000 7fae86ffd640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-21T03:41:40.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.376+0000 7fae86ffd640 1 -- 172.21.15.38:0/2046379808 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae98061510 con 0x55f3d3e2d3b0 2024-09-21T03:41:40.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.376+0000 7fae86ffd640 1 -- 172.21.15.38:0/2046379808 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 7336+0+0 (secure 0 0 0) 0x7fae98014d80 con 0x55f3d3e2d3b0 2024-09-21T03:41:40.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.376+0000 7faeb5411740 1 -- 172.21.15.38:0/2046379808 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae9806b990 con 0x55f3d3e2d3b0 2024-09-21T03:41:40.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.377+0000 7fae86ffd640 1 -- 172.21.15.38:0/2046379808 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=161) ==== 24+0+0 (secure 0 0 0) 0x7fae980883f0 con 0x55f3d3e2d3b0 2024-09-21T03:41:40.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.377+0000 7faeb5411740 10 client.8520 fetch_fsmap learned FSMap version 161 2024-09-21T03:41:40.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.377+0000 7faeb5411740 1 -- 172.21.15.38:0/2046379808 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=161,monmap=2+}) -- 0x55f3d423ce10 con 0x55f3d3e2d3b0 2024-09-21T03:41:40.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.377+0000 7fae86ffd640 1 -- 172.21.15.38:0/2046379808 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fae9806b990 con 0x55f3d3e2d3b0 2024-09-21T03:41:40.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.377+0000 7fae86ffd640 1 -- 172.21.15.38:0/2046379808 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 161) ==== 3502+0+0 (secure 0 0 0) 0x7fae9805da00 con 0x55f3d3e2d3b0 2024-09-21T03:41:40.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.377+0000 7faeb5411740 10 client.8520 fetch_fsmap finished waiting for FSMap version 161 2024-09-21T03:41:40.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.377+0000 7faeb5411740 10 client.8520 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-21T03:41:40.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.377+0000 7faeb5411740 20 client.8520 populate_metadata read hostname 'smithi038' 2024-09-21T03:41:40.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.377+0000 7faeb5411740 1 --2- 172.21.15.38:0/2046379808 >> [v2:172.21.15.38:6834/2916238332,v1:172.21.15.38:6835/2916238332] conn(0x55f3d3e14e40 0x55f3d4251d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:40.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.377+0000 7faeb5411740 4 client.8520 mds_command: new command op to 8370 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:41:40.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.377+0000 7faeb5411740 1 -- 172.21.15.38:0/2046379808 --> [v2:172.21.15.38:6834/2916238332,v1:172.21.15.38:6835/2916238332] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f3d4172770 con 0x55f3d3e14e40 2024-09-21T03:41:40.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.377+0000 7faea77fe640 1 --2- 172.21.15.38:0/2046379808 >> [v2:172.21.15.38:6834/2916238332,v1:172.21.15.38:6835/2916238332] conn(0x55f3d3e14e40 0x55f3d4251d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:40.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.378+0000 7fae86ffd640 10 client.8520 ms_handle_connect on v2:172.21.15.38:6834/2916238332 2024-09-21T03:41:40.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.378+0000 7faea77fe640 1 --2- 172.21.15.38:0/2046379808 >> [v2:172.21.15.38:6834/2916238332,v1:172.21.15.38:6835/2916238332] conn(0x55f3d3e14e40 0x55f3d4251d80 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd058c757a7e9023 server_cookie=ccae2c1acaedb199 in_seq=0 out_seq=0 2024-09-21T03:41:40.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.378+0000 7fae86ffd640 1 -- 172.21.15.38:0/2046379808 <== mds.0 v2:172.21.15.38:6834/2916238332 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f3d4172770 con 0x55f3d3e14e40 2024-09-21T03:41:40.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.378+0000 7fae86ffd640 10 client.8520 handle_command_reply: tid=0 2024-09-21T03:41:40.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.378+0000 7faeb5411740 1 client.8520 shutdown 2024-09-21T03:41:40.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.378+0000 7faeb5411740 1 -- 172.21.15.38:0/2046379808 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f3d3e2d3b0 msgr2=0x55f3d4215630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:40.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.378+0000 7faeb5411740 1 --2- 172.21.15.38:0/2046379808 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f3d3e2d3b0 0x55f3d4215630 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7fae98000fb0 tx=0x7fae9805e5b0 comp rx=0 tx=0).stop 2024-09-21T03:41:40.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.379+0000 7faeb5411740 1 -- 172.21.15.38:0/2046379808 shutdown_connections 2024-09-21T03:41:40.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.379+0000 7faeb5411740 1 --2- 172.21.15.38:0/2046379808 >> [v2:172.21.15.38:6834/2916238332,v1:172.21.15.38:6835/2916238332] conn(0x55f3d3e14e40 0x55f3d4251d80 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:40.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.379+0000 7faeb5411740 1 --2- 172.21.15.38:0/2046379808 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f3d3e56270 0x55f3d42553c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:40.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.379+0000 7faeb5411740 1 --2- 172.21.15.38:0/2046379808 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f3d3e35590 0x55f3d4215b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:40.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.379+0000 7fae86ffd640 0 client.8520 ms_handle_reset on v2:172.21.15.38:6834/2916238332 2024-09-21T03:41:40.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.379+0000 7faeb5411740 1 --2- 172.21.15.38:0/2046379808 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f3d3e2d3b0 0x55f3d4215630 unknown :-1 s=CLOSED pgs=1527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:40.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.379+0000 7faeb5411740 1 -- 172.21.15.38:0/2046379808 >> 172.21.15.38:0/2046379808 conn(0x55f3d3e4a3c0 msgr2=0x55f3d42449b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:40.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.379+0000 7faeb5411740 1 -- 172.21.15.38:0/2046379808 shutdown_connections 2024-09-21T03:41:40.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.379+0000 7faeb5411740 1 -- 172.21.15.38:0/2046379808 wait complete. 2024-09-21T03:41:40.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.379+0000 7faeb5411740 20 client.8520 trim_cache size 0 max 16384 2024-09-21T03:41:40.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.398+0000 7faeb5411740 1 Processor -- start 2024-09-21T03:41:40.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.398+0000 7faeb5411740 1 -- start start 2024-09-21T03:41:40.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.399+0000 7faeb5411740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f3d3e2d3b0 0x55f3d423a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:40.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.399+0000 7faeb5411740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f3d3e35590 0x55f3d4265880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:40.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.399+0000 7faeb5411740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f3d3e56270 0x55f3d4265dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:40.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.399+0000 7faeb5411740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7faea8200080 con 0x55f3d3e56270 2024-09-21T03:41:40.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.399+0000 7faeb5411740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7faea81f58c0 con 0x55f3d3e2d3b0 2024-09-21T03:41:40.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.399+0000 7faeb5411740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7faea81f5f10 con 0x55f3d3e35590 2024-09-21T03:41:40.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.399+0000 7faeac960640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f3d3e56270 0x55f3d4265dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:40.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.399+0000 7faeac960640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f3d3e56270 0x55f3d4265dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54332/0 (socket says 172.21.15.38:54332) 2024-09-21T03:41:40.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.399+0000 7faeac960640 1 -- 172.21.15.38:0/3915228636 learned_addr learned my addr 172.21.15.38:0/3915228636 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:41:40.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.400+0000 7faea7fff640 1 --2- 172.21.15.38:0/3915228636 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f3d3e2d3b0 0x55f3d423a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:40.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.400+0000 7faea77fe640 1 --2- 172.21.15.38:0/3915228636 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f3d3e35590 0x55f3d4265880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:40.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.400+0000 7faeac960640 1 -- 172.21.15.38:0/3915228636 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f3d3e35590 msgr2=0x55f3d4265880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:40.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.400+0000 7faeac960640 1 --2- 172.21.15.38:0/3915228636 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f3d3e35590 0x55f3d4265880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:40.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.400+0000 7faeac960640 1 -- 172.21.15.38:0/3915228636 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f3d3e2d3b0 msgr2=0x55f3d423a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:40.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.400+0000 7faeac960640 1 --2- 172.21.15.38:0/3915228636 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f3d3e2d3b0 0x55f3d423a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:40.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.400+0000 7faeac960640 1 -- 172.21.15.38:0/3915228636 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3d423c690 con 0x55f3d3e56270 2024-09-21T03:41:40.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.400+0000 7faea7fff640 1 --2- 172.21.15.38:0/3915228636 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f3d3e2d3b0 0x55f3d423a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:41:40.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.400+0000 7faeac960640 1 --2- 172.21.15.38:0/3915228636 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f3d3e56270 0x55f3d4265dc0 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7faea0009cd0 tx=0x7faea0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e09e5b5d768ab86b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:40.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.400+0000 7fae8d7fa640 1 -- 172.21.15.38:0/3915228636 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faea0003cd0 con 0x55f3d3e56270 2024-09-21T03:41:40.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.400+0000 7fae8d7fa640 1 -- 172.21.15.38:0/3915228636 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faea0004040 con 0x55f3d3e56270 2024-09-21T03:41:40.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.400+0000 7fae8d7fa640 1 -- 172.21.15.38:0/3915228636 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faea00043f0 con 0x55f3d3e56270 2024-09-21T03:41:40.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.401+0000 7faeb5411740 1 -- 172.21.15.38:0/3915228636 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f3d3e56270 msgr2=0x55f3d4265dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:40.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.401+0000 7faeb5411740 1 --2- 172.21.15.38:0/3915228636 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f3d3e56270 0x55f3d4265dc0 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7faea0009cd0 tx=0x7faea0004a40 comp rx=0 tx=0).stop 2024-09-21T03:41:40.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.401+0000 7faeb5411740 1 -- 172.21.15.38:0/3915228636 shutdown_connections 2024-09-21T03:41:40.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.401+0000 7faeb5411740 1 --2- 172.21.15.38:0/3915228636 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f3d3e56270 0x55f3d4265dc0 unknown :-1 s=CLOSED pgs=1528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:40.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.401+0000 7faeb5411740 1 --2- 172.21.15.38:0/3915228636 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f3d3e35590 0x55f3d4265880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:40.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.401+0000 7faeb5411740 1 --2- 172.21.15.38:0/3915228636 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f3d3e2d3b0 0x55f3d423a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:40.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.401+0000 7faeb5411740 1 -- 172.21.15.38:0/3915228636 >> 172.21.15.38:0/3915228636 conn(0x55f3d3e4a3c0 msgr2=0x55f3d41fd4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:40.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.401+0000 7faeb5411740 1 -- 172.21.15.38:0/3915228636 shutdown_connections 2024-09-21T03:41:40.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.401+0000 7faeb5411740 1 -- 172.21.15.38:0/3915228636 wait complete. 2024-09-21T03:41:40.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.402+0000 7faeb5411740 1 -- start start 2024-09-21T03:41:40.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.402+0000 7faeb5411740 1 Processor -- start 2024-09-21T03:41:40.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.404+0000 7faeb5411740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f3d3e2d3b0 0x55f3d41fc150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:40.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.404+0000 7faeb5411740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f3d3e35590 0x55f3d4227e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:40.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.404+0000 7faeb5411740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f3d3e56270 0x55f3d42c2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:40.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.404+0000 7faeb5411740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55f3d423ac00 con 0x55f3d3e35590 2024-09-21T03:41:40.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.404+0000 7faeb5411740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55f3d3ed0620 con 0x55f3d3e2d3b0 2024-09-21T03:41:40.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.404+0000 7faeb5411740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55f3d4255dd0 con 0x55f3d3e56270 2024-09-21T03:41:40.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.404+0000 7faeb5411740 10 client.0 _ll_register_callbacks cb 0x7fae98061f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:41:40.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.404+0000 7faea77fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f3d3e35590 0x55f3d4227e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:40.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.404+0000 7faea77fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f3d3e35590 0x55f3d4227e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54348/0 (socket says 172.21.15.38:54348) 2024-09-21T03:41:40.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.404+0000 7faea77fe640 1 -- 172.21.15.38:0/1962717109 learned_addr learned my addr 172.21.15.38:0/1962717109 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:41:40.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.404+0000 7faea7fff640 1 --2- 172.21.15.38:0/1962717109 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f3d3e2d3b0 0x55f3d41fc150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:40.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.404+0000 7faeac960640 1 --2- 172.21.15.38:0/1962717109 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f3d3e56270 0x55f3d42c2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:40.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.404+0000 7faea77fe640 1 -- 172.21.15.38:0/1962717109 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f3d3e56270 msgr2=0x55f3d42c2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:40.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.404+0000 7faea77fe640 1 --2- 172.21.15.38:0/1962717109 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f3d3e56270 0x55f3d42c2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:40.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.404+0000 7faea77fe640 1 -- 172.21.15.38:0/1962717109 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f3d3e2d3b0 msgr2=0x55f3d41fc150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:40.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.404+0000 7faea77fe640 1 --2- 172.21.15.38:0/1962717109 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f3d3e2d3b0 0x55f3d41fc150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:40.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.404+0000 7faea77fe640 1 -- 172.21.15.38:0/1962717109 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3d41fcc00 con 0x55f3d3e35590 2024-09-21T03:41:40.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.404+0000 7faea77fe640 1 --2- 172.21.15.38:0/1962717109 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f3d3e35590 0x55f3d4227e30 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x55f3d4225c00 tx=0x7fae9c013040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49288d34e5fc41f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:40.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.404+0000 7fae86ffd640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-21T03:41:40.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.404+0000 7fae86ffd640 1 -- 172.21.15.38:0/1962717109 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae9c0044d0 con 0x55f3d3e35590 2024-09-21T03:41:40.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.405+0000 7fae86ffd640 1 -- 172.21.15.38:0/1962717109 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 7336+0+0 (secure 0 0 0) 0x7fae9c029450 con 0x55f3d3e35590 2024-09-21T03:41:40.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.405+0000 7faeb5411740 1 -- 172.21.15.38:0/1962717109 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faea0004040 con 0x55f3d3e35590 2024-09-21T03:41:40.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.405+0000 7fae86ffd640 1 -- 172.21.15.38:0/1962717109 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=161) ==== 24+0+0 (secure 0 0 0) 0x7fae9c033e60 con 0x55f3d3e35590 2024-09-21T03:41:40.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.405+0000 7faeb5411740 10 client.8526 fetch_fsmap learned FSMap version 161 2024-09-21T03:41:40.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.405+0000 7faeb5411740 1 -- 172.21.15.38:0/1962717109 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=161,monmap=2+}) -- 0x55f3d4224280 con 0x55f3d3e35590 2024-09-21T03:41:40.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.405+0000 7fae86ffd640 1 -- 172.21.15.38:0/1962717109 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faea0004040 con 0x55f3d3e35590 2024-09-21T03:41:40.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.405+0000 7fae86ffd640 1 -- 172.21.15.38:0/1962717109 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 161) ==== 3502+0+0 (secure 0 0 0) 0x7fae9c026040 con 0x55f3d3e35590 2024-09-21T03:41:40.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.405+0000 7faeb5411740 10 client.8526 fetch_fsmap finished waiting for FSMap version 161 2024-09-21T03:41:40.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.405+0000 7faeb5411740 10 client.8526 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-21T03:41:40.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.405+0000 7faeb5411740 20 client.8526 populate_metadata read hostname 'smithi038' 2024-09-21T03:41:40.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.405+0000 7faeb5411740 1 --2- 172.21.15.38:0/1962717109 >> [v2:172.21.15.38:6834/2916238332,v1:172.21.15.38:6835/2916238332] conn(0x55f3d42658c0 0x55f3d42c6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:40.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.405+0000 7faeb5411740 4 client.8526 mds_command: new command op to 8370 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T03:41:40.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.405+0000 7faeb5411740 1 -- 172.21.15.38:0/1962717109 --> [v2:172.21.15.38:6834/2916238332,v1:172.21.15.38:6835/2916238332] -- command(tid 0: {"prefix": "status"}) -- 0x55f3d416e220 con 0x55f3d42658c0 2024-09-21T03:41:40.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.405+0000 7faea7fff640 1 --2- 172.21.15.38:0/1962717109 >> [v2:172.21.15.38:6834/2916238332,v1:172.21.15.38:6835/2916238332] conn(0x55f3d42658c0 0x55f3d42c6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:40.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.406+0000 7fae86ffd640 10 client.8526 ms_handle_connect on v2:172.21.15.38:6834/2916238332 2024-09-21T03:41:40.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.406+0000 7faea7fff640 1 --2- 172.21.15.38:0/1962717109 >> [v2:172.21.15.38:6834/2916238332,v1:172.21.15.38:6835/2916238332] conn(0x55f3d42658c0 0x55f3d42c6cf0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=525650a47838d81b server_cookie=ba75672ef9bfad09 in_seq=0 out_seq=0 2024-09-21T03:41:40.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.406+0000 7fae86ffd640 1 -- 172.21.15.38:0/1962717109 <== mds.0 v2:172.21.15.38:6834/2916238332 1 ==== command_reply(tid 0: 0 ) ==== 8+0+334 (crc 0 0 0) 0x55f3d416e220 con 0x55f3d42658c0 2024-09-21T03:41:40.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.406+0000 7fae86ffd640 10 client.8526 handle_command_reply: tid=0 2024-09-21T03:41:40.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.406+0000 7faeb5411740 1 client.8526 shutdown 2024-09-21T03:41:40.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.406+0000 7faeb5411740 1 -- 172.21.15.38:0/1962717109 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f3d3e35590 msgr2=0x55f3d4227e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:40.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.406+0000 7faeb5411740 1 --2- 172.21.15.38:0/1962717109 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f3d3e35590 0x55f3d4227e30 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x55f3d4225c00 tx=0x7fae9c013040 comp rx=0 tx=0).stop 2024-09-21T03:41:40.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.406+0000 7faeb5411740 1 -- 172.21.15.38:0/1962717109 shutdown_connections 2024-09-21T03:41:40.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.406+0000 7faeb5411740 1 --2- 172.21.15.38:0/1962717109 >> [v2:172.21.15.38:6834/2916238332,v1:172.21.15.38:6835/2916238332] conn(0x55f3d42658c0 0x55f3d42c6cf0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:40.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.406+0000 7faeb5411740 1 --2- 172.21.15.38:0/1962717109 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f3d3e56270 0x55f3d42c2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:40.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.406+0000 7faeb5411740 1 --2- 172.21.15.38:0/1962717109 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f3d3e35590 0x55f3d4227e30 unknown :-1 s=CLOSED pgs=1529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:40.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.406+0000 7fae86ffd640 0 client.8526 ms_handle_reset on v2:172.21.15.38:6834/2916238332 2024-09-21T03:41:40.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.406+0000 7faeb5411740 1 --2- 172.21.15.38:0/1962717109 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f3d3e2d3b0 0x55f3d41fc150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:40.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.406+0000 7faeb5411740 1 -- 172.21.15.38:0/1962717109 >> 172.21.15.38:0/1962717109 conn(0x55f3d3e4a3c0 msgr2=0x55f3d41f9ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:40.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.406+0000 7faeb5411740 1 -- 172.21.15.38:0/1962717109 shutdown_connections 2024-09-21T03:41:40.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.406+0000 7faeb5411740 1 -- 172.21.15.38:0/1962717109 wait complete. 2024-09-21T03:41:40.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.406+0000 7faeb5411740 20 client.8526 trim_cache size 0 max 16384 2024-09-21T03:41:40.413 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-09-21T03:41:40.413 INFO:teuthology.orchestra.run.smithi038.stdout: "cluster_fsid": "9d50d66f-656b-4fe2-8e3b-fe53b79c4d43", 2024-09-21T03:41:40.413 INFO:teuthology.orchestra.run.smithi038.stdout: "whoami": 0, 2024-09-21T03:41:40.413 INFO:teuthology.orchestra.run.smithi038.stdout: "id": 8370, 2024-09-21T03:41:40.413 INFO:teuthology.orchestra.run.smithi038.stdout: "want_state": "up:active", 2024-09-21T03:41:40.414 INFO:teuthology.orchestra.run.smithi038.stdout: "state": "up:active", 2024-09-21T03:41:40.414 INFO:teuthology.orchestra.run.smithi038.stdout: "fs_name": "cephfs", 2024-09-21T03:41:40.414 INFO:teuthology.orchestra.run.smithi038.stdout: "rank_uptime": 6.7401135859999997, 2024-09-21T03:41:40.414 INFO:teuthology.orchestra.run.smithi038.stdout: "mdsmap_epoch": 161, 2024-09-21T03:41:40.414 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch": 108, 2024-09-21T03:41:40.414 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch_barrier": 108, 2024-09-21T03:41:40.414 INFO:teuthology.orchestra.run.smithi038.stdout: "uptime": 19.030885222999999 2024-09-21T03:41:40.414 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-09-21T03:41:40.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.407+0000 7faeae3c3640 1 -- 172.21.15.38:0/2298216813 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fae94078b90 msgr2=0x7fae9407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:40.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.407+0000 7faeae3c3640 1 --2- 172.21.15.38:0/2298216813 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fae94078b90 0x7fae9407b0a0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7faea80cabc0 tx=0x7fae9c005f90 comp rx=0 tx=0).stop 2024-09-21T03:41:40.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.407+0000 7faeae3c3640 1 -- 172.21.15.38:0/2298216813 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faea806def0 msgr2=0x7faea80b1670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:40.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.407+0000 7faeae3c3640 1 --2- 172.21.15.38:0/2298216813 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faea806def0 0x7faea80b1670 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7fae9800c340 tx=0x7fae9800c800 comp rx=0 tx=0).stop 2024-09-21T03:41:40.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.407+0000 7faeae3c3640 1 -- 172.21.15.38:0/2298216813 shutdown_connections 2024-09-21T03:41:40.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.407+0000 7faeae3c3640 1 --2- 172.21.15.38:0/2298216813 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faea815bfc0 0x7faea8155250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:40.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.407+0000 7faeae3c3640 1 --2- 172.21.15.38:0/2298216813 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fae94078b90 0x7fae9407b0a0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:40.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.407+0000 7faeae3c3640 1 --2- 172.21.15.38:0/2298216813 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faea80b5260 0x7faea81f6ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:40.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.407+0000 7faeae3c3640 1 --2- 172.21.15.38:0/2298216813 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faea806def0 0x7faea80b1670 unknown :-1 s=CLOSED pgs=1525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:40.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.407+0000 7faeae3c3640 1 -- 172.21.15.38:0/2298216813 >> 172.21.15.38:0/2298216813 conn(0x7faea80cdca0 msgr2=0x7faea80ce0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:40.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.407+0000 7faeae3c3640 1 -- 172.21.15.38:0/2298216813 shutdown_connections 2024-09-21T03:41:40.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.407+0000 7faeae3c3640 1 -- 172.21.15.38:0/2298216813 wait complete. 2024-09-21T03:41:40.420 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-09-21T03:41:40.420 INFO:tasks.cephfs.test_damage:Daemons came up after mutation 'Delete 400.00000000', proceeding to ls 2024-09-21T03:41:40.420 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T03:41:40.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.534+0000 7fa9225a1640 1 -- 172.21.15.38:0/2171502089 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa91c162f00 msgr2=0x7fa91c0d24d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:40.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.534+0000 7fa9225a1640 1 --2- 172.21.15.38:0/2171502089 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa91c162f00 0x7fa91c0d24d0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fa90c005fd0 tx=0x7fa90c02a5d0 comp rx=0 tx=0).stop 2024-09-21T03:41:40.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.534+0000 7fa9225a1640 1 -- 172.21.15.38:0/2171502089 shutdown_connections 2024-09-21T03:41:40.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.534+0000 7fa9225a1640 1 --2- 172.21.15.38:0/2171502089 >> v2:172.21.15.195:3300/0 conn(0x7fa91c0b2e40 0x7fa91c0b3250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:40.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.534+0000 7fa9225a1640 1 --2- 172.21.15.38:0/2171502089 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa91c162f00 0x7fa91c0d24d0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:40.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.534+0000 7fa9225a1640 1 -- 172.21.15.38:0/2171502089 >> 172.21.15.38:0/2171502089 conn(0x7fa91c0b6b20 msgr2=0x7fa91c0d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:40.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.534+0000 7fa9225a1640 1 -- 172.21.15.38:0/2171502089 shutdown_connections 2024-09-21T03:41:40.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.534+0000 7fa9225a1640 1 -- 172.21.15.38:0/2171502089 wait complete. 2024-09-21T03:41:40.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.536+0000 7fa9225a1640 1 Processor -- start 2024-09-21T03:41:40.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.536+0000 7fa9225a1640 1 -- start start 2024-09-21T03:41:40.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.537+0000 7fa9225a1640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa91c0b2e40 0x7fa91c0acf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:40.541 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.537+0000 7fa9225a1640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa91c162280 0x7fa91c0afb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:40.541 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.537+0000 7fa9225a1640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa91c14d6e0 0x7fa91c1f4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:40.541 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.537+0000 7fa9225a1640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa91c15e4b0 con 0x7fa91c0b2e40 2024-09-21T03:41:40.541 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.537+0000 7fa9225a1640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fa91c0b39a0 con 0x7fa91c162280 2024-09-21T03:41:40.541 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.537+0000 7fa9225a1640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fa91c15e630 con 0x7fa91c14d6e0 2024-09-21T03:41:40.541 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.538+0000 7fa91bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa91c0b2e40 0x7fa91c0acf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:40.541 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.538+0000 7fa91bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa91c0b2e40 0x7fa91c0acf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54350/0 (socket says 172.21.15.38:54350) 2024-09-21T03:41:40.541 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.538+0000 7fa91bfff640 1 -- 172.21.15.38:0/2351112511 learned_addr learned my addr 172.21.15.38:0/2351112511 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:41:40.541 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.538+0000 7fa91b7fe640 1 --2- 172.21.15.38:0/2351112511 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa91c162280 0x7fa91c0afb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:40.541 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.538+0000 7fa920b3e640 1 --2- 172.21.15.38:0/2351112511 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa91c14d6e0 0x7fa91c1f4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:40.541 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.538+0000 7fa91bfff640 1 -- 172.21.15.38:0/2351112511 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa91c14d6e0 msgr2=0x7fa91c1f4fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:40.541 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.538+0000 7fa91bfff640 1 --2- 172.21.15.38:0/2351112511 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa91c14d6e0 0x7fa91c1f4fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:40.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.538+0000 7fa91bfff640 1 -- 172.21.15.38:0/2351112511 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa91c162280 msgr2=0x7fa91c0afb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:40.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.538+0000 7fa91bfff640 1 --2- 172.21.15.38:0/2351112511 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa91c162280 0x7fa91c0afb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:40.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.538+0000 7fa91bfff640 1 -- 172.21.15.38:0/2351112511 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa91c14dec0 con 0x7fa91c0b2e40 2024-09-21T03:41:40.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.538+0000 7fa91bfff640 1 --2- 172.21.15.38:0/2351112511 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa91c0b2e40 0x7fa91c0acf00 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7fa90c034040 tx=0x7fa90c005770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:40.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.538+0000 7fa918ff9640 1 -- 172.21.15.38:0/2351112511 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa90c03e030 con 0x7fa91c0b2e40 2024-09-21T03:41:40.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.538+0000 7fa9225a1640 1 -- 172.21.15.38:0/2351112511 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa91c14e1a0 con 0x7fa91c0b2e40 2024-09-21T03:41:40.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.538+0000 7fa918ff9640 1 -- 172.21.15.38:0/2351112511 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa90c003a90 con 0x7fa91c0b2e40 2024-09-21T03:41:40.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.539+0000 7fa918ff9640 1 -- 172.21.15.38:0/2351112511 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa90c035040 con 0x7fa91c0b2e40 2024-09-21T03:41:40.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.539+0000 7fa9225a1640 1 -- 172.21.15.38:0/2351112511 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa91c0b1700 con 0x7fa91c0b2e40 2024-09-21T03:41:40.543 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.540+0000 7fa9225a1640 1 -- 172.21.15.38:0/2351112511 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa91c162f00 con 0x7fa91c0b2e40 2024-09-21T03:41:40.543 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.540+0000 7fa918ff9640 1 -- 172.21.15.38:0/2351112511 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa90c041030 con 0x7fa91c0b2e40 2024-09-21T03:41:40.543 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.541+0000 7fa918ff9640 1 --2- 172.21.15.38:0/2351112511 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa908078bc0 0x7fa90807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:40.543 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.541+0000 7fa918ff9640 1 -- 172.21.15.38:0/2351112511 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 7336+0+0 (secure 0 0 0) 0x7fa90c0bb600 con 0x7fa91c0b2e40 2024-09-21T03:41:40.543 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.541+0000 7fa91b7fe640 1 --2- 172.21.15.38:0/2351112511 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa908078bc0 0x7fa90807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:40.544 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.542+0000 7fa91b7fe640 1 --2- 172.21.15.38:0/2351112511 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa908078bc0 0x7fa90807b0d0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fa910007fd0 tx=0x7fa910009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:40.547 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.546+0000 7fa918ff9640 1 -- 172.21.15.38:0/2351112511 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fa90c082e30 con 0x7fa91c0b2e40 2024-09-21T03:41:40.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.771+0000 7fa9225a1640 1 -- 172.21.15.38:0/2351112511 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fa91c0b7af0 con 0x7fa91c0b2e40 2024-09-21T03:41:40.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.771+0000 7fa918ff9640 1 -- 172.21.15.38:0/2351112511 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v161) ==== 53+0+73 (secure 0 0 0) 0x7fa90c087860 con 0x7fa91c0b2e40 2024-09-21T03:41:40.774 INFO:teuthology.orchestra.run.smithi038.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T03:41:40.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.773+0000 7fa9225a1640 1 -- 172.21.15.38:0/2351112511 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa908078bc0 msgr2=0x7fa90807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:40.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.773+0000 7fa9225a1640 1 --2- 172.21.15.38:0/2351112511 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa908078bc0 0x7fa90807b0d0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fa910007fd0 tx=0x7fa910009040 comp rx=0 tx=0).stop 2024-09-21T03:41:40.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.773+0000 7fa9225a1640 1 -- 172.21.15.38:0/2351112511 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa91c0b2e40 msgr2=0x7fa91c0acf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:40.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.773+0000 7fa9225a1640 1 --2- 172.21.15.38:0/2351112511 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa91c0b2e40 0x7fa91c0acf00 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7fa90c034040 tx=0x7fa90c005770 comp rx=0 tx=0).stop 2024-09-21T03:41:40.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.773+0000 7fa9225a1640 1 -- 172.21.15.38:0/2351112511 shutdown_connections 2024-09-21T03:41:40.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.773+0000 7fa9225a1640 1 --2- 172.21.15.38:0/2351112511 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa908078bc0 0x7fa90807b0d0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:40.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.773+0000 7fa9225a1640 1 --2- 172.21.15.38:0/2351112511 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa91c14d6e0 0x7fa91c1f4fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:40.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.773+0000 7fa9225a1640 1 --2- 172.21.15.38:0/2351112511 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa91c162280 0x7fa91c0afb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:40.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.773+0000 7fa9225a1640 1 --2- 172.21.15.38:0/2351112511 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa91c0b2e40 0x7fa91c0acf00 unknown :-1 s=CLOSED pgs=1530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:40.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.774+0000 7fa9225a1640 1 -- 172.21.15.38:0/2351112511 >> 172.21.15.38:0/2351112511 conn(0x7fa91c0b6b20 msgr2=0x7fa91c06def0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:40.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.774+0000 7fa9225a1640 1 -- 172.21.15.38:0/2351112511 shutdown_connections 2024-09-21T03:41:40.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:40.774+0000 7fa9225a1640 1 -- 172.21.15.38:0/2351112511 wait complete. 2024-09-21T03:41:40.787 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-21T03:41:40.787 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T03:41:40.787 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi038.front.sepia.ceph.com 2024-09-21T03:41:40.787 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-21T03:41:40.787 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T03:41:40.787 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-21T03:41:40.787 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-21T03:41:40.787 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T03:41:40.787 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T03:41:40.787 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:41:40.788 DEBUG:teuthology.orchestra.run.smithi038:> ip addr 2024-09-21T03:41:40.804 INFO:teuthology.orchestra.run.smithi038.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T03:41:40.804 INFO:teuthology.orchestra.run.smithi038.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T03:41:40.804 INFO:teuthology.orchestra.run.smithi038.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T03:41:40.804 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:41:40.804 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 ::1/128 scope host 2024-09-21T03:41:40.804 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:41:40.804 INFO:teuthology.orchestra.run.smithi038.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T03:41:40.804 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:6b:fd:a2 brd ff:ff:ff:ff:ff:ff 2024-09-21T03:41:40.804 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp7s0f0 2024-09-21T03:41:40.804 INFO:teuthology.orchestra.run.smithi038.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T03:41:40.804 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:6b:fd:a3 brd ff:ff:ff:ff:ff:ff 2024-09-21T03:41:40.805 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp7s0f1 2024-09-21T03:41:40.805 INFO:teuthology.orchestra.run.smithi038.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T03:41:40.805 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:bd:15:fc brd ff:ff:ff:ff:ff:ff 2024-09-21T03:41:40.805 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp1s0f0 2024-09-21T03:41:40.805 INFO:teuthology.orchestra.run.smithi038.stdout: inet 172.21.15.38/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-21T03:41:40.805 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft 42308sec preferred_lft 42308sec 2024-09-21T03:41:40.805 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 fe80::ec4:7aff:febd:15fc/64 scope link 2024-09-21T03:41:40.805 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:41:40.805 INFO:teuthology.orchestra.run.smithi038.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T03:41:40.805 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:bd:15:fd brd ff:ff:ff:ff:ff:ff 2024-09-21T03:41:40.805 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp1s0f1 2024-09-21T03:41:40.805 INFO:teuthology.orchestra.run.smithi038.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T03:41:40.805 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 92:55:c7:97:6f:b5 brd ff:ff:ff:ff:ff:ff 2024-09-21T03:41:40.805 INFO:teuthology.orchestra.run.smithi038.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-21T03:41:40.806 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:41:40.806 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 fe80::c49f:c1ff:fe21:b12a/64 scope link 2024-09-21T03:41:40.806 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:41:40.806 INFO:teuthology.orchestra.run.smithi038.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T03:41:40.806 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether a6:3a:83:ad:db:d5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T03:41:40.806 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 fe80::a43a:83ff:fead:dbd5/64 scope link 2024-09-21T03:41:40.806 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:41:40.806 INFO:teuthology.orchestra.run.smithi038.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T03:41:40.806 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 92:55:c7:97:6f:b5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T03:41:40.806 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 fe80::9055:c7ff:fe97:6fb5/64 scope link 2024-09-21T03:41:40.806 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:41:40.807 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:41:40.807 DEBUG:teuthology.orchestra.run.smithi038:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T03:41:40.867 INFO:teuthology.orchestra.run.smithi038.stdout:1 2024-09-21T03:41:40.869 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:41:40.869 DEBUG:teuthology.orchestra.run.smithi038:> route 2024-09-21T03:41:40.926 INFO:teuthology.orchestra.run.smithi038.stdout:Kernel IP routing table 2024-09-21T03:41:40.926 INFO:teuthology.orchestra.run.smithi038.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T03:41:40.927 INFO:teuthology.orchestra.run.smithi038.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-21T03:41:40.927 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-21T03:41:40.927 INFO:teuthology.orchestra.run.smithi038.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T03:41:40.927 DEBUG:teuthology.orchestra.run.smithi038:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T03:41:40.927 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-09-21T03:41:40.927 DEBUG:teuthology.orchestra.run.smithi038:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-21T03:41:40.927 DEBUG:teuthology.orchestra.run.smithi038:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-21T03:41:40.927 DEBUG:teuthology.orchestra.run.smithi038:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-21T03:41:40.927 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-09-21T03:41:41.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:41 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T03:41:41.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:41:41.056 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:41:41.056 DEBUG:teuthology.orchestra.run.smithi038:> ip netns list 2024-09-21T03:41:41.072 INFO:teuthology.orchestra.run.smithi038.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-21T03:41:41.072 INFO:teuthology.orchestra.run.smithi038.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-21T03:41:41.072 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T03:41:41.073 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:41:41.073 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip link set brx.0 up 2024-09-21T03:41:41.138 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T03:41:41.139 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-21T03:41:41.194 INFO:teuthology.orchestra.run.smithi038.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-21T03:41:41.194 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T03:41:41.194 DEBUG:teuthology.orchestra.run.smithi038:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-21T03:41:41.250 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-21T03:41:41.250 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:41:41.250 DEBUG:teuthology.orchestra.run.smithi038:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T03:41:41.344 INFO:teuthology.orchestra.run.smithi038.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T03:41:41.345 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T03:41:41.345 INFO:teuthology.orchestra.run.smithi038.stdout:invalid new device string format 2024-09-21T03:41:41.356 INFO:teuthology.orchestra.run.smithi038.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T03:41:41.356 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T03:41:41.356 INFO:teuthology.orchestra.run.smithi038.stdout:invalid new device string format 2024-09-21T03:41:41.356 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: resolved to: "172.21.15.38:3300,172.21.15.195:3300,172.21.15.195:3301" 2024-09-21T03:41:41.356 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T03:41:41.362 INFO:teuthology.orchestra.run.smithi038.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T03:41:41.362 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T03:41:41.363 INFO:teuthology.orchestra.run.smithi038.stdout:invalid new device string format 2024-09-21T03:41:41.363 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: resolved to: "172.21.15.38:3300,172.21.15.195:3300,172.21.15.195:3301" 2024-09-21T03:41:41.363 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T03:41:41.363 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: resolved to: "172.21.15.38:6789,172.21.15.195:6789,172.21.15.195:6790" 2024-09-21T03:41:41.363 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.38:6789,172.21.15.195:6789,172.21.15.195:6790:/ 2024-09-21T03:41:41.363 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=9d50d66f-656b-4fe2-8e3b-fe53b79c4d43" will pass to kernel 2024-09-21T03:41:41.363 INFO:teuthology.orchestra.run.smithi038.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-21T03:41:41.364 INFO:teuthology.orchestra.run.smithi038.stdout: You just mounted a file system that supports labels which does not 2024-09-21T03:41:41.364 INFO:teuthology.orchestra.run.smithi038.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-21T03:41:41.364 INFO:teuthology.orchestra.run.smithi038.stdout: applications will generate AVC messages and not be allowed access to 2024-09-21T03:41:41.364 INFO:teuthology.orchestra.run.smithi038.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-21T03:41:41.365 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T03:41:41.365 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:41:41.366 DEBUG:teuthology.orchestra.run.smithi038:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-21T03:41:41.390 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T03:41:41.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:41 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T03:41:41.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:41:41.487 INFO:teuthology.orchestra.run.smithi038.stdout:client8532 2024-09-21T03:41:41.488 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-21T03:41:41.488 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:41:41.504 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:41:41.504 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls -R .) 2024-09-21T03:41:41.547 INFO:teuthology.orchestra.run:waiting for 20 2024-09-21T03:41:41.554 INFO:tasks.ceph.mds.a.smithi038.stderr:2024-09-21T03:41:41.553+0000 7f8aabc87640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T03:41:41.562 INFO:teuthology.orchestra.run.smithi038.stdout:.: 2024-09-21T03:41:41.562 INFO:teuthology.orchestra.run.smithi038.stdout:subdir 2024-09-21T03:41:41.562 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:41:41.562 INFO:teuthology.orchestra.run.smithi038.stdout:./subdir: 2024-09-21T03:41:41.562 INFO:teuthology.orchestra.run.smithi038.stdout:sixmegs 2024-09-21T03:41:41.563 INFO:tasks.cephfs.test_damage:Result: Mutation 'Delete 400.00000000' did not caused DAMAGED state 2024-09-21T03:41:41.563 INFO:tasks.cephfs.test_damage:Applying mutation 'Delete 100.00000000.inode' 2024-09-21T03:41:41.563 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-21T03:41:41.563 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:41:41.619 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-21T03:41:41.620 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:41:41.676 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T03:41:41.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:41.832+0000 7f2a468ec640 1 -- 172.21.15.38:0/1120596472 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2a2c003a30 msgr2=0x7f2a2c003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:41.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:41.832+0000 7f2a468ec640 1 --2- 172.21.15.38:0/1120596472 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2a2c003a30 0x7f2a2c003ec0 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f2a3000c640 tx=0x7f2a2c005830 comp rx=0 tx=0).stop 2024-09-21T03:41:41.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:41.832+0000 7f2a44e89640 1 -- 172.21.15.38:0/1120596472 reap_dead start 2024-09-21T03:41:41.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:41.832+0000 7f2a468ec640 1 -- 172.21.15.38:0/1120596472 shutdown_connections 2024-09-21T03:41:41.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:41.832+0000 7f2a468ec640 1 -- 172.21.15.38:0/1120596472 >> 172.21.15.38:0/1120596472 conn(0x7f2a400b46e0 msgr2=0x7f2a400d23e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:41.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:41.832+0000 7f2a468ec640 1 -- 172.21.15.38:0/1120596472 shutdown_connections 2024-09-21T03:41:41.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:41.832+0000 7f2a468ec640 1 -- 172.21.15.38:0/1120596472 wait complete. 2024-09-21T03:41:41.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:41.833+0000 7f2a468ec640 1 Processor -- start 2024-09-21T03:41:41.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:41.834+0000 7f2a468ec640 1 -- start start 2024-09-21T03:41:41.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:41.835+0000 7f2a468ec640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2a2c003a30 0x7f2a40151b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:41.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:41.835+0000 7f2a468ec640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2a40153e90 0x7f2a40152090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:41.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:41.835+0000 7f2a468ec640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2a401503f0 0x7f2a40152a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:41.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:41.835+0000 7f2a468ec640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f2a400b62f0 con 0x7f2a40153e90 2024-09-21T03:41:41.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:41.835+0000 7f2a468ec640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f2a400d09e0 con 0x7f2a2c003a30 2024-09-21T03:41:41.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:41.835+0000 7f2a468ec640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f2a400b6470 con 0x7f2a401503f0 2024-09-21T03:41:41.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:41.835+0000 7f2a3f7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2a40153e90 0x7f2a40152090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:41.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:41.835+0000 7f2a3f7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2a40153e90 0x7f2a40152090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54370/0 (socket says 172.21.15.38:54370) 2024-09-21T03:41:41.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:41.835+0000 7f2a3f7fe640 1 -- 172.21.15.38:0/2071288761 learned_addr learned my addr 172.21.15.38:0/2071288761 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:41:41.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:41.835+0000 7f2a3ffff640 1 --2- 172.21.15.38:0/2071288761 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2a2c003a30 0x7f2a40151b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:41.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:41.835+0000 7f2a44e89640 1 --2- 172.21.15.38:0/2071288761 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2a401503f0 0x7f2a40152a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:41.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:41.835+0000 7f2a3f7fe640 1 -- 172.21.15.38:0/2071288761 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2a401503f0 msgr2=0x7f2a40152a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:41.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:41.835+0000 7f2a3f7fe640 1 --2- 172.21.15.38:0/2071288761 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2a401503f0 0x7f2a40152a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:41.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:41.835+0000 7f2a3f7fe640 1 -- 172.21.15.38:0/2071288761 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2a2c003a30 msgr2=0x7f2a40151b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:41.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:41.835+0000 7f2a3f7fe640 1 --2- 172.21.15.38:0/2071288761 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2a2c003a30 0x7f2a40151b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:41.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:41.835+0000 7f2a3f7fe640 1 -- 172.21.15.38:0/2071288761 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a40150ad0 con 0x7f2a40153e90 2024-09-21T03:41:41.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:41.835+0000 7f2a3ffff640 1 --2- 172.21.15.38:0/2071288761 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2a2c003a30 0x7f2a40151b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:41:41.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:41.836+0000 7f2a3f7fe640 1 --2- 172.21.15.38:0/2071288761 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2a40153e90 0x7f2a40152090 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7f2a28003fa0 tx=0x7f2a2800ba60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:41.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:41.836+0000 7f2a3cff9640 1 -- 172.21.15.38:0/2071288761 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a280107f0 con 0x7f2a40153e90 2024-09-21T03:41:41.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:41.836+0000 7f2a3cff9640 1 -- 172.21.15.38:0/2071288761 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2a28010990 con 0x7f2a40153e90 2024-09-21T03:41:41.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:41.836+0000 7f2a468ec640 1 -- 172.21.15.38:0/2071288761 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a40150db0 con 0x7f2a40153e90 2024-09-21T03:41:41.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:41.836+0000 7f2a3cff9640 1 -- 172.21.15.38:0/2071288761 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a28010da0 con 0x7f2a40153e90 2024-09-21T03:41:41.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:41.836+0000 7f2a468ec640 1 -- 172.21.15.38:0/2071288761 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a40151370 con 0x7f2a40153e90 2024-09-21T03:41:41.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:41.837+0000 7f2a468ec640 1 -- 172.21.15.38:0/2071288761 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a400b69b0 con 0x7f2a40153e90 2024-09-21T03:41:41.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:41.838+0000 7f2a3cff9640 1 -- 172.21.15.38:0/2071288761 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f2a28023440 con 0x7f2a40153e90 2024-09-21T03:41:41.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:41.838+0000 7f2a3cff9640 1 --2- 172.21.15.38:0/2071288761 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2a20078b90 0x7f2a2007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:41.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:41.838+0000 7f2a3cff9640 1 -- 172.21.15.38:0/2071288761 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 7336+0+0 (secure 0 0 0) 0x7f2a2809c0c0 con 0x7f2a40153e90 2024-09-21T03:41:41.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:41.838+0000 7f2a3ffff640 1 --2- 172.21.15.38:0/2071288761 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2a20078b90 0x7f2a2007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:41.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:41.839+0000 7f2a3ffff640 1 --2- 172.21.15.38:0/2071288761 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2a20078b90 0x7f2a2007b0a0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f2a30033000 tx=0x7f2a30001bc0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:41.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:41.843+0000 7f2a3cff9640 1 -- 172.21.15.38:0/2071288761 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f2a28063ea0 con 0x7f2a40153e90 2024-09-21T03:41:42.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:42.044+0000 7f2a468ec640 1 -- 172.21.15.38:0/2071288761 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f2a400cd8e0 con 0x7f2a40153e90 2024-09-21T03:41:42.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:42.044+0000 7f2a3cff9640 1 -- 172.21.15.38:0/2071288761 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 24 entries v108) ==== 81+0+1458 (secure 0 0 0) 0x7f2a280637a0 con 0x7f2a40153e90 2024-09-21T03:41:42.046 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6833/3553953266 2024-09-22T03:41:29.639383+0000 2024-09-21T03:41:42.046 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6832/3553953266 2024-09-22T03:41:29.639383+0000 2024-09-21T03:41:42.046 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6835/2748959970 2024-09-22T03:41:16.565736+0000 2024-09-21T03:41:42.047 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6834/2748959970 2024-09-22T03:41:16.565736+0000 2024-09-21T03:41:42.047 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6832/3726830644 2024-09-22T03:41:12.532341+0000 2024-09-21T03:41:42.047 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6833/121492560 2024-09-22T03:41:10.371933+0000 2024-09-21T03:41:42.047 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6832/121492560 2024-09-22T03:41:10.371933+0000 2024-09-21T03:41:42.047 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6833/3710736169 2024-09-22T03:40:59.454480+0000 2024-09-21T03:41:42.047 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6832/751618334 2024-09-22T03:40:42.053279+0000 2024-09-21T03:41:42.047 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6835/2934569248 2024-09-22T03:40:26.934591+0000 2024-09-21T03:41:42.047 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6833/751618334 2024-09-22T03:40:42.053279+0000 2024-09-21T03:41:42.047 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6832/3402963947 2024-09-22T03:41:23.555005+0000 2024-09-21T03:41:42.047 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6835/3864487171 2024-09-22T03:40:31.989837+0000 2024-09-21T03:41:42.047 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6833/3726830644 2024-09-22T03:41:12.532341+0000 2024-09-21T03:41:42.047 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.195:6835/712370966 2024-09-22T03:40:11.633787+0000 2024-09-21T03:41:42.047 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6834/2934569248 2024-09-22T03:40:26.934591+0000 2024-09-21T03:41:42.048 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6834/3864487171 2024-09-22T03:40:31.989837+0000 2024-09-21T03:41:42.048 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6833/3402963947 2024-09-22T03:41:23.555005+0000 2024-09-21T03:41:42.048 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.195:6837/712370966 2024-09-22T03:40:11.633787+0000 2024-09-21T03:41:42.048 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6834/425672974 2024-09-22T03:40:46.105521+0000 2024-09-21T03:41:42.048 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6835/425672974 2024-09-22T03:40:46.105521+0000 2024-09-21T03:41:42.048 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6834/2751962827 2024-09-22T03:40:38.029580+0000 2024-09-21T03:41:42.048 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6835/2751962827 2024-09-22T03:40:38.029580+0000 2024-09-21T03:41:42.048 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6832/3710736169 2024-09-22T03:40:59.454480+0000 2024-09-21T03:41:42.049 INFO:teuthology.orchestra.run.smithi038.stderr:listed 24 entries 2024-09-21T03:41:42.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:42.046+0000 7f2a468ec640 1 -- 172.21.15.38:0/2071288761 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2a20078b90 msgr2=0x7f2a2007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:42.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:42.046+0000 7f2a468ec640 1 --2- 172.21.15.38:0/2071288761 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2a20078b90 0x7f2a2007b0a0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f2a30033000 tx=0x7f2a30001bc0 comp rx=0 tx=0).stop 2024-09-21T03:41:42.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:42.046+0000 7f2a468ec640 1 -- 172.21.15.38:0/2071288761 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2a40153e90 msgr2=0x7f2a40152090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:42.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:42.046+0000 7f2a468ec640 1 --2- 172.21.15.38:0/2071288761 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2a40153e90 0x7f2a40152090 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7f2a28003fa0 tx=0x7f2a2800ba60 comp rx=0 tx=0).stop 2024-09-21T03:41:42.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:42.046+0000 7f2a468ec640 1 -- 172.21.15.38:0/2071288761 shutdown_connections 2024-09-21T03:41:42.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:42.046+0000 7f2a468ec640 1 --2- 172.21.15.38:0/2071288761 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2a20078b90 0x7f2a2007b0a0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:42.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:42.046+0000 7f2a468ec640 1 --2- 172.21.15.38:0/2071288761 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2a401503f0 0x7f2a40152a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:42.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:42.046+0000 7f2a468ec640 1 --2- 172.21.15.38:0/2071288761 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2a40153e90 0x7f2a40152090 unknown :-1 s=CLOSED pgs=1534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:42.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:42.046+0000 7f2a468ec640 1 --2- 172.21.15.38:0/2071288761 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2a2c003a30 0x7f2a40151b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:42.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:42.046+0000 7f2a468ec640 1 -- 172.21.15.38:0/2071288761 >> 172.21.15.38:0/2071288761 conn(0x7f2a400b46e0 msgr2=0x7f2a400b1ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:42.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:42.046+0000 7f2a468ec640 1 -- 172.21.15.38:0/2071288761 shutdown_connections 2024-09-21T03:41:42.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:42.046+0000 7f2a468ec640 1 -- 172.21.15.38:0/2071288761 wait complete. 2024-09-21T03:41:42.060 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-21T03:41:42.060 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:41:42.060 DEBUG:teuthology.orchestra.run.smithi038:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-09-21T03:41:42.102 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi038.front.sepia.ceph.com 2024-09-21T03:41:42.102 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:41:42.102 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T03:41:42.159 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-21T03:41:42.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:42.314+0000 7f713516a640 1 -- 172.21.15.38:0/1622611935 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f713015ca90 msgr2=0x7f71300d0670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:42.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:42.314+0000 7f713516a640 1 --2- 172.21.15.38:0/1622611935 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f713015ca90 0x7f71300d0670 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f7118009830 tx=0x7f711802a480 comp rx=0 tx=0).stop 2024-09-21T03:41:42.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:42.314+0000 7f713516a640 1 -- 172.21.15.38:0/1622611935 shutdown_connections 2024-09-21T03:41:42.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:42.314+0000 7f713516a640 1 --2- 172.21.15.38:0/1622611935 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f71300d0bb0 0x7f71300b5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:42.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:42.314+0000 7f713516a640 1 --2- 172.21.15.38:0/1622611935 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f713015ca90 0x7f71300d0670 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:42.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:42.314+0000 7f713516a640 1 -- 172.21.15.38:0/1622611935 >> 172.21.15.38:0/1622611935 conn(0x7f71300b46e0 msgr2=0x7f71300d23e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:42.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:42.314+0000 7f713516a640 1 -- 172.21.15.38:0/1622611935 shutdown_connections 2024-09-21T03:41:42.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:42.314+0000 7f713516a640 1 -- 172.21.15.38:0/1622611935 wait complete. 2024-09-21T03:41:42.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:42.316+0000 7f713516a640 1 Processor -- start 2024-09-21T03:41:42.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:42.316+0000 7f713516a640 1 -- start start 2024-09-21T03:41:42.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:42.317+0000 7f713516a640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f71300ccc40 0x7f71300ad580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:42.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:42.317+0000 7f713516a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f71300d0bb0 0x7f713014f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:42.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:42.317+0000 7f713516a640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f713015ca90 0x7f71300c6850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:42.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:42.317+0000 7f713516a640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f71300b66a0 con 0x7f71300d0bb0 2024-09-21T03:41:42.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:42.317+0000 7f713516a640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f71300b6520 con 0x7f713015ca90 2024-09-21T03:41:42.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:42.317+0000 7f713516a640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f71300b6820 con 0x7f71300ccc40 2024-09-21T03:41:42.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:42.318+0000 7f7127fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f71300d0bb0 0x7f713014f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:42.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:42.318+0000 7f7127fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f71300d0bb0 0x7f713014f5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54392/0 (socket says 172.21.15.38:54392) 2024-09-21T03:41:42.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:42.318+0000 7f7127fff640 1 -- 172.21.15.38:0/2550300032 learned_addr learned my addr 172.21.15.38:0/2550300032 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:41:42.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:42.318+0000 7f712f59e640 1 --2- 172.21.15.38:0/2550300032 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f713015ca90 0x7f71300c6850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:42.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:42.318+0000 7f712ed9d640 1 --2- 172.21.15.38:0/2550300032 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f71300ccc40 0x7f71300ad580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:42.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:42.318+0000 7f7127fff640 1 -- 172.21.15.38:0/2550300032 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f71300ccc40 msgr2=0x7f71300ad580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:42.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:42.318+0000 7f7127fff640 1 --2- 172.21.15.38:0/2550300032 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f71300ccc40 0x7f71300ad580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:42.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:42.318+0000 7f7127fff640 1 -- 172.21.15.38:0/2550300032 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f713015ca90 msgr2=0x7f71300c6850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:42.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:42.318+0000 7f7127fff640 1 --2- 172.21.15.38:0/2550300032 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f713015ca90 0x7f71300c6850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:42.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:42.318+0000 7f7127fff640 1 -- 172.21.15.38:0/2550300032 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f713015b8a0 con 0x7f71300d0bb0 2024-09-21T03:41:42.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:42.318+0000 7f7127fff640 1 --2- 172.21.15.38:0/2550300032 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f71300d0bb0 0x7f713014f5a0 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f7118009950 tx=0x7f7118033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:42.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:42.318+0000 7f71277fe640 1 -- 172.21.15.38:0/2550300032 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7118005c90 con 0x7f71300d0bb0 2024-09-21T03:41:42.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:42.318+0000 7f713516a640 1 -- 172.21.15.38:0/2550300032 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7130150140 con 0x7f71300d0bb0 2024-09-21T03:41:42.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:42.318+0000 7f71277fe640 1 -- 172.21.15.38:0/2550300032 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7118031e50 con 0x7f71300d0bb0 2024-09-21T03:41:42.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:42.318+0000 7f71277fe640 1 -- 172.21.15.38:0/2550300032 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7118005260 con 0x7f71300d0bb0 2024-09-21T03:41:42.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:42.319+0000 7f713516a640 1 -- 172.21.15.38:0/2550300032 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f713014d540 con 0x7f71300d0bb0 2024-09-21T03:41:42.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:42.320+0000 7f713516a640 1 -- 172.21.15.38:0/2550300032 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f713014feb0 con 0x7f71300d0bb0 2024-09-21T03:41:42.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:42.320+0000 7f71277fe640 1 -- 172.21.15.38:0/2550300032 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f7118005400 con 0x7f71300d0bb0 2024-09-21T03:41:42.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:42.321+0000 7f71277fe640 1 --2- 172.21.15.38:0/2550300032 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7110078bc0 0x7f711007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:42.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:42.321+0000 7f71277fe640 1 -- 172.21.15.38:0/2550300032 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 7336+0+0 (secure 0 0 0) 0x7f71180bc2a0 con 0x7f71300d0bb0 2024-09-21T03:41:42.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:42.321+0000 7f712ed9d640 1 --2- 172.21.15.38:0/2550300032 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7110078bc0 0x7f711007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:42.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:42.322+0000 7f712ed9d640 1 --2- 172.21.15.38:0/2550300032 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7110078bc0 0x7f711007b0d0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f7120007d60 tx=0x7f7120007cf0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:42.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:42.325+0000 7f71277fe640 1 -- 172.21.15.38:0/2550300032 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f711803f030 con 0x7f71300d0bb0 2024-09-21T03:41:42.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:42.552+0000 7f713516a640 1 -- 172.21.15.38:0/2550300032 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f71300b6db0 con 0x7f71300d0bb0 2024-09-21T03:41:42.554 INFO:tasks.ceph.mon.a.smithi038.stderr:2024-09-21T03:41:42.553+0000 7f3f45713640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T03:41:42.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:42.554+0000 7f71277fe640 1 -- 172.21.15.38:0/2550300032 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v162) ==== 161+0+0 (secure 0 0 0) 0x7f7118083ad0 con 0x7f71300d0bb0 2024-09-21T03:41:42.556 INFO:teuthology.orchestra.run.smithi038.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-21T03:41:42.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:42.557+0000 7f713516a640 1 -- 172.21.15.38:0/2550300032 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7110078bc0 msgr2=0x7f711007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:42.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:42.557+0000 7f713516a640 1 --2- 172.21.15.38:0/2550300032 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7110078bc0 0x7f711007b0d0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f7120007d60 tx=0x7f7120007cf0 comp rx=0 tx=0).stop 2024-09-21T03:41:42.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:42.557+0000 7f713516a640 1 -- 172.21.15.38:0/2550300032 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f71300d0bb0 msgr2=0x7f713014f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:42.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:42.557+0000 7f713516a640 1 --2- 172.21.15.38:0/2550300032 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f71300d0bb0 0x7f713014f5a0 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f7118009950 tx=0x7f7118033c20 comp rx=0 tx=0).stop 2024-09-21T03:41:42.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:42.557+0000 7f713516a640 1 -- 172.21.15.38:0/2550300032 shutdown_connections 2024-09-21T03:41:42.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:42.557+0000 7f713516a640 1 --2- 172.21.15.38:0/2550300032 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f713015ca90 0x7f71300c6850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:42.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:42.557+0000 7f713516a640 1 --2- 172.21.15.38:0/2550300032 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f71300d0bb0 0x7f713014f5a0 unknown :-1 s=CLOSED pgs=1535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:42.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:42.557+0000 7f713516a640 1 --2- 172.21.15.38:0/2550300032 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7110078bc0 0x7f711007b0d0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:42.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:42.557+0000 7f713516a640 1 --2- 172.21.15.38:0/2550300032 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f71300ccc40 0x7f71300ad580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:42.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:42.557+0000 7f713516a640 1 -- 172.21.15.38:0/2550300032 >> 172.21.15.38:0/2550300032 conn(0x7f71300b46e0 msgr2=0x7f71300cd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:42.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:42.557+0000 7f713516a640 1 -- 172.21.15.38:0/2550300032 shutdown_connections 2024-09-21T03:41:42.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:42.557+0000 7f713516a640 1 -- 172.21.15.38:0/2550300032 wait complete. 2024-09-21T03:41:42.562 INFO:tasks.ceph.mds.a.smithi038.stderr: -297> 2024-09-21T03:41:41.553+0000 7f8aabc87640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T03:41:42.572 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-09-21T03:41:42.619 INFO:tasks.ceph.mds.a.smithi038.stdout:starting mds.a at 2024-09-21T03:41:42.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:42.692+0000 7f469aca9640 1 -- 172.21.15.38:0/2895775442 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4684003fd0 msgr2=0x7f4684004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:42.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:42.692+0000 7f469aca9640 1 --2- 172.21.15.38:0/2895775442 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4684003fd0 0x7f4684004800 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f468000c260 tx=0x7f4684005830 comp rx=0 tx=0).stop 2024-09-21T03:41:42.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:42.692+0000 7f4699246640 1 -- 172.21.15.38:0/2895775442 reap_dead start 2024-09-21T03:41:42.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:42.692+0000 7f469aca9640 1 -- 172.21.15.38:0/2895775442 shutdown_connections 2024-09-21T03:41:42.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:42.692+0000 7f469aca9640 1 -- 172.21.15.38:0/2895775442 >> 172.21.15.38:0/2895775442 conn(0x7f46940cdca0 msgr2=0x7f46940ce0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:42.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:42.692+0000 7f469aca9640 1 -- 172.21.15.38:0/2895775442 shutdown_connections 2024-09-21T03:41:42.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:42.692+0000 7f469aca9640 1 -- 172.21.15.38:0/2895775442 wait complete. 2024-09-21T03:41:42.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:42.694+0000 7f469aca9640 1 Processor -- start 2024-09-21T03:41:42.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:42.694+0000 7f469aca9640 1 -- start start 2024-09-21T03:41:42.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:42.695+0000 7f469aca9640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4684003fd0 0x7f46941f9380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:42.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:42.695+0000 7f469aca9640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f46941f53b0 0x7f46940cd3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:42.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:42.695+0000 7f469aca9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f46940cff70 0x7f46940d0380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:42.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:42.695+0000 7f469aca9640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f469415f9c0 con 0x7f46940cff70 2024-09-21T03:41:42.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:42.695+0000 7f469aca9640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f469415f840 con 0x7f46941f53b0 2024-09-21T03:41:42.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:42.695+0000 7f469aca9640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f469415fb40 con 0x7f4684003fd0 2024-09-21T03:41:42.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:42.695+0000 7f4699246640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f46940cff70 0x7f46940d0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:42.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:42.695+0000 7f4699246640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f46940cff70 0x7f46940d0380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54410/0 (socket says 172.21.15.38:54410) 2024-09-21T03:41:42.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:42.695+0000 7f4699246640 1 -- 172.21.15.38:0/1218992259 learned_addr learned my addr 172.21.15.38:0/1218992259 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:41:42.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:42.696+0000 7f4698a45640 1 --2- 172.21.15.38:0/1218992259 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4684003fd0 0x7f46941f9380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:42.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:42.696+0000 7f4693fff640 1 --2- 172.21.15.38:0/1218992259 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f46941f53b0 0x7f46940cd3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:42.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:42.696+0000 7f4699246640 1 -- 172.21.15.38:0/1218992259 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4684003fd0 msgr2=0x7f46941f9380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:42.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:42.696+0000 7f4699246640 1 --2- 172.21.15.38:0/1218992259 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4684003fd0 0x7f46941f9380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:42.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:42.696+0000 7f4699246640 1 -- 172.21.15.38:0/1218992259 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f46941f53b0 msgr2=0x7f46940cd3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:42.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:42.696+0000 7f4699246640 1 --2- 172.21.15.38:0/1218992259 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f46941f53b0 0x7f46940cd3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:42.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:42.696+0000 7f4699246640 1 -- 172.21.15.38:0/1218992259 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46940d08c0 con 0x7f46940cff70 2024-09-21T03:41:42.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:42.696+0000 7f4698a45640 1 --2- 172.21.15.38:0/1218992259 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4684003fd0 0x7f46941f9380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:41:42.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:42.696+0000 7f4699246640 1 --2- 172.21.15.38:0/1218992259 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f46940cff70 0x7f46940d0380 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7f468c005fd0 tx=0x7f468c011f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:42.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:42.696+0000 7f46917fa640 1 -- 172.21.15.38:0/1218992259 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f468c00fd10 con 0x7f46940cff70 2024-09-21T03:41:42.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:42.696+0000 7f46917fa640 1 -- 172.21.15.38:0/1218992259 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f468c004b20 con 0x7f46940cff70 2024-09-21T03:41:42.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:42.696+0000 7f469aca9640 1 -- 172.21.15.38:0/1218992259 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46941f98c0 con 0x7f46940cff70 2024-09-21T03:41:42.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:42.696+0000 7f46917fa640 1 -- 172.21.15.38:0/1218992259 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f468c003970 con 0x7f46940cff70 2024-09-21T03:41:42.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:42.696+0000 7f469aca9640 1 -- 172.21.15.38:0/1218992259 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46941f5b80 con 0x7f46940cff70 2024-09-21T03:41:42.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:42.697+0000 7f469aca9640 1 -- 172.21.15.38:0/1218992259 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f469406f410 con 0x7f46940cff70 2024-09-21T03:41:42.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:42.698+0000 7f46917fa640 1 -- 172.21.15.38:0/1218992259 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f468c020020 con 0x7f46940cff70 2024-09-21T03:41:42.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:42.698+0000 7f46917fa640 1 --2- 172.21.15.38:0/1218992259 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4678078b90 0x7f467807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:42.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:42.698+0000 7f46917fa640 1 -- 172.21.15.38:0/1218992259 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 7422+0+0 (secure 0 0 0) 0x7f468c09bba0 con 0x7f46940cff70 2024-09-21T03:41:42.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:42.699+0000 7f4698a45640 1 --2- 172.21.15.38:0/1218992259 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4678078b90 0x7f467807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:42.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:42.699+0000 7f4698a45640 1 --2- 172.21.15.38:0/1218992259 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4678078b90 0x7f467807b0a0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f469415efd0 tx=0x7f4688005f90 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:42.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:42.703+0000 7f46917fa640 1 -- 172.21.15.38:0/1218992259 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f46940d08c0 con 0x7f46940cff70 2024-09-21T03:41:42.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:42.965+0000 7f469aca9640 1 -- 172.21.15.38:0/1218992259 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7f469406f820 con 0x7f46940cff70 2024-09-21T03:41:43.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.556+0000 7f46917fa640 1 -- 172.21.15.38:0/1218992259 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 nothing to do: rank is not damaged v163) ==== 107+0+0 (secure 0 0 0) 0x7f468c0639c0 con 0x7f46940cff70 2024-09-21T03:41:43.558 INFO:teuthology.orchestra.run.smithi038.stderr:nothing to do: rank is not damaged 2024-09-21T03:41:43.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.559+0000 7f469aca9640 1 -- 172.21.15.38:0/1218992259 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4678078b90 msgr2=0x7f467807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:43.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.559+0000 7f469aca9640 1 --2- 172.21.15.38:0/1218992259 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4678078b90 0x7f467807b0a0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f469415efd0 tx=0x7f4688005f90 comp rx=0 tx=0).stop 2024-09-21T03:41:43.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.559+0000 7f469aca9640 1 -- 172.21.15.38:0/1218992259 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f46940cff70 msgr2=0x7f46940d0380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:43.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.559+0000 7f469aca9640 1 --2- 172.21.15.38:0/1218992259 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f46940cff70 0x7f46940d0380 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7f468c005fd0 tx=0x7f468c011f90 comp rx=0 tx=0).stop 2024-09-21T03:41:43.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.559+0000 7f469aca9640 1 -- 172.21.15.38:0/1218992259 shutdown_connections 2024-09-21T03:41:43.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.559+0000 7f469aca9640 1 --2- 172.21.15.38:0/1218992259 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f46940cff70 0x7f46940d0380 unknown :-1 s=CLOSED pgs=1539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:43.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.559+0000 7f469aca9640 1 --2- 172.21.15.38:0/1218992259 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f46941f53b0 0x7f46940cd3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:43.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.559+0000 7f469aca9640 1 --2- 172.21.15.38:0/1218992259 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4678078b90 0x7f467807b0a0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:43.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.559+0000 7f469aca9640 1 --2- 172.21.15.38:0/1218992259 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4684003fd0 0x7f46941f9380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:43.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.559+0000 7f469aca9640 1 -- 172.21.15.38:0/1218992259 >> 172.21.15.38:0/1218992259 conn(0x7f46940cdca0 msgr2=0x7f46940ce0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:43.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.560+0000 7f469aca9640 1 -- 172.21.15.38:0/1218992259 shutdown_connections 2024-09-21T03:41:43.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.560+0000 7f469aca9640 1 -- 172.21.15.38:0/1218992259 wait complete. 2024-09-21T03:41:43.575 DEBUG:teuthology.orchestra.run.smithi038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-09-21T03:41:43.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.631+0000 7f7c99d1df40 1 Processor -- start 2024-09-21T03:41:43.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.631+0000 7f7c99d1df40 1 -- start start 2024-09-21T03:41:43.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.632+0000 7f7c99d1df40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5598c0ffe090 0x5598c1117b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:43.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.632+0000 7f7c99d1df40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5598c1073520 0x5598c0fe0490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:43.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.632+0000 7f7c99d1df40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5598c1084ca0 0x5598c0fe09d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:43.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.632+0000 7f7c99d1df40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5598c0f04d60 con 0x5598c1073520 2024-09-21T03:41:43.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.632+0000 7f7c99d1df40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5598c0f63430 con 0x5598c1084ca0 2024-09-21T03:41:43.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.632+0000 7f7c99d1df40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x5598c108b350 con 0x5598c0ffe090 2024-09-21T03:41:43.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.632+0000 7f7c92592640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5598c1073520 0x5598c0fe0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:43.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.632+0000 7f7c92592640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5598c1073520 0x5598c0fe0490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54438/0 (socket says 172.21.15.38:54438) 2024-09-21T03:41:43.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.632+0000 7f7c92592640 1 -- 172.21.15.38:0/2495447042 learned_addr learned my addr 172.21.15.38:0/2495447042 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:41:43.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.632+0000 7f7c93594640 1 --2- 172.21.15.38:0/2495447042 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5598c1084ca0 0x5598c0fe09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:43.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.632+0000 7f7c92d93640 1 --2- 172.21.15.38:0/2495447042 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5598c0ffe090 0x5598c1117b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:43.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.632+0000 7f7c92592640 1 -- 172.21.15.38:0/2495447042 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5598c0ffe090 msgr2=0x5598c1117b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:43.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.632+0000 7f7c92592640 1 --2- 172.21.15.38:0/2495447042 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5598c0ffe090 0x5598c1117b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:43.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.632+0000 7f7c92592640 1 -- 172.21.15.38:0/2495447042 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5598c1084ca0 msgr2=0x5598c0fe09d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:43.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.632+0000 7f7c92592640 1 --2- 172.21.15.38:0/2495447042 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5598c1084ca0 0x5598c0fe09d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:43.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.632+0000 7f7c92592640 1 -- 172.21.15.38:0/2495447042 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5598c10899b0 con 0x5598c1073520 2024-09-21T03:41:43.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.632+0000 7f7c92d93640 1 --2- 172.21.15.38:0/2495447042 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5598c0ffe090 0x5598c1117b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:41:43.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.633+0000 7f7c92592640 1 --2- 172.21.15.38:0/2495447042 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5598c1073520 0x5598c0fe0490 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7f7c88001d20 tx=0x7f7c8800b5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=538d616b7fb987e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:43.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.633+0000 7f7c91590640 1 -- 172.21.15.38:0/2495447042 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c8800bd80 con 0x5598c1073520 2024-09-21T03:41:43.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.633+0000 7f7c91590640 1 -- 172.21.15.38:0/2495447042 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7c8800f400 con 0x5598c1073520 2024-09-21T03:41:43.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.633+0000 7f7c91590640 1 -- 172.21.15.38:0/2495447042 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c8800f850 con 0x5598c1073520 2024-09-21T03:41:43.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.633+0000 7f7c99d1df40 1 -- 172.21.15.38:0/2495447042 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5598c1073520 msgr2=0x5598c0fe0490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:43.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.633+0000 7f7c99d1df40 1 --2- 172.21.15.38:0/2495447042 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5598c1073520 0x5598c0fe0490 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7f7c88001d20 tx=0x7f7c8800b5b0 comp rx=0 tx=0).stop 2024-09-21T03:41:43.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.633+0000 7f7c99d1df40 1 -- 172.21.15.38:0/2495447042 shutdown_connections 2024-09-21T03:41:43.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.633+0000 7f7c99d1df40 1 --2- 172.21.15.38:0/2495447042 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5598c1084ca0 0x5598c0fe09d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:43.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.633+0000 7f7c99d1df40 1 --2- 172.21.15.38:0/2495447042 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5598c1073520 0x5598c0fe0490 unknown :-1 s=CLOSED pgs=1541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:43.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.633+0000 7f7c99d1df40 1 --2- 172.21.15.38:0/2495447042 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5598c0ffe090 0x5598c1117b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:43.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.634+0000 7f7c99d1df40 1 -- 172.21.15.38:0/2495447042 >> 172.21.15.38:0/2495447042 conn(0x5598c0ed5ac0 msgr2=0x5598c0ffc7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:43.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.634+0000 7f7c99d1df40 1 -- 172.21.15.38:0/2495447042 shutdown_connections 2024-09-21T03:41:43.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.634+0000 7f7c99d1df40 1 -- 172.21.15.38:0/2495447042 wait complete. 2024-09-21T03:41:43.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.634+0000 7f7c99d1df40 1 Processor -- start 2024-09-21T03:41:43.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.635+0000 7f7c99d1df40 1 -- start start 2024-09-21T03:41:43.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.636+0000 7f7c99d1df40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5598c0ffe090 0x5598c106bff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:43.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.636+0000 7f7c99d1df40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5598c1073520 0x5598c0fe0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:43.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.636+0000 7f7c99d1df40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5598c1084ca0 0x5598c11213d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:43.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.636+0000 7f7c99d1df40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5598c1072b40 con 0x5598c0ffe090 2024-09-21T03:41:43.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.636+0000 7f7c99d1df40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5598c1072860 con 0x5598c1084ca0 2024-09-21T03:41:43.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.636+0000 7f7c99d1df40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x5598c1122220 con 0x5598c1073520 2024-09-21T03:41:43.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.636+0000 7f7c92d93640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5598c0ffe090 0x5598c106bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:43.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.636+0000 7f7c92d93640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5598c0ffe090 0x5598c106bff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54442/0 (socket says 172.21.15.38:54442) 2024-09-21T03:41:43.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.636+0000 7f7c92d93640 1 -- 172.21.15.38:0/1459949514 learned_addr learned my addr 172.21.15.38:0/1459949514 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:41:43.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.636+0000 7f7c93594640 1 --2- 172.21.15.38:0/1459949514 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5598c1084ca0 0x5598c11213d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:43.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.636+0000 7f7c92592640 1 --2- 172.21.15.38:0/1459949514 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5598c1073520 0x5598c0fe0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:43.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.636+0000 7f7c92d93640 1 -- 172.21.15.38:0/1459949514 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5598c1073520 msgr2=0x5598c0fe0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:43.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.636+0000 7f7c92d93640 1 --2- 172.21.15.38:0/1459949514 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5598c1073520 0x5598c0fe0aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:43.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.636+0000 7f7c92d93640 1 -- 172.21.15.38:0/1459949514 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5598c1084ca0 msgr2=0x5598c11213d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:43.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.636+0000 7f7c92d93640 1 --2- 172.21.15.38:0/1459949514 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5598c1084ca0 0x5598c11213d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:43.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.636+0000 7f7c92d93640 1 -- 172.21.15.38:0/1459949514 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5598c1086ff0 con 0x5598c0ffe090 2024-09-21T03:41:43.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.636+0000 7f7c93594640 1 --2- 172.21.15.38:0/1459949514 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5598c1084ca0 0x5598c11213d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:41:43.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.636+0000 7f7c92d93640 1 --2- 172.21.15.38:0/1459949514 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5598c0ffe090 0x5598c106bff0 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7f7c8400b690 tx=0x7f7c8400bb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:43.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.637+0000 7f7c7bfff640 1 -- 172.21.15.38:0/1459949514 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c840099d0 con 0x5598c0ffe090 2024-09-21T03:41:43.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.637+0000 7f7c7bfff640 1 -- 172.21.15.38:0/1459949514 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7c8400be90 con 0x5598c0ffe090 2024-09-21T03:41:43.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.637+0000 7f7c99d1df40 1 -- 172.21.15.38:0/1459949514 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5598c10874f0 con 0x5598c0ffe090 2024-09-21T03:41:43.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.637+0000 7f7c7bfff640 1 -- 172.21.15.38:0/1459949514 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c8400f430 con 0x5598c0ffe090 2024-09-21T03:41:43.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.637+0000 7f7c99d1df40 1 -- 172.21.15.38:0/1459949514 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5598c10860c0 con 0x5598c0ffe090 2024-09-21T03:41:43.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.638+0000 7f7c7bfff640 1 -- 172.21.15.38:0/1459949514 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f7c8400f5d0 con 0x5598c0ffe090 2024-09-21T03:41:43.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.639+0000 7f7c7bfff640 1 --2- 172.21.15.38:0/1459949514 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7c800775a0 0x7f7c80079ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:43.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.639+0000 7f7c7bfff640 1 -- 172.21.15.38:0/1459949514 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 7422+0+0 (secure 0 0 0) 0x7f7c8409ae10 con 0x5598c0ffe090 2024-09-21T03:41:43.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.639+0000 7f7c92592640 1 --2- 172.21.15.38:0/1459949514 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7c800775a0 0x7f7c80079ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:43.643 INFO:teuthology.orchestra.run.smithi038.stdout:Importing pool 2024-09-21T03:41:43.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.639+0000 7f7c99d1df40 1 --2- 172.21.15.38:0/1459949514 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x5598c106eb70 0x5598c106ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:43.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.639+0000 7f7c99d1df40 1 -- 172.21.15.38:0/1459949514 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x5598c1121e20 con 0x5598c106eb70 2024-09-21T03:41:43.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.640+0000 7f7c93594640 1 --2- 172.21.15.38:0/1459949514 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x5598c106eb70 0x5598c106ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:43.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.640+0000 7f7c92592640 1 --2- 172.21.15.38:0/1459949514 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7c800775a0 0x7f7c80079ab0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x5598c0ffd2f0 tx=0x7f7c88000ed0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:43.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.640+0000 7f7c93594640 1 --2- 172.21.15.38:0/1459949514 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x5598c106eb70 0x5598c106ef80 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:43.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.644+0000 7f7c93594640 1 -- 172.21.15.38:0/1459949514 <== osd.7 v2:172.21.15.195:6818/1606106342 1 ==== osd_op_reply(1 601.00000000 [create] v109'381 uv355 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7c8c054370 con 0x5598c106eb70 2024-09-21T03:41:43.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.644+0000 7f7c99d1df40 1 -- 172.21.15.38:0/1459949514 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e109) -- 0x5598c112d990 con 0x5598c106eb70 2024-09-21T03:41:43.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.646+0000 7f7c93594640 1 -- 172.21.15.38:0/1459949514 <== osd.7 v2:172.21.15.195:6818/1606106342 2 ==== osd_op_reply(2 601.00000000 [delete] v109'382 uv382 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7c8c054370 con 0x5598c106eb70 2024-09-21T03:41:43.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.646+0000 7f7c99d1df40 1 -- 172.21.15.38:0/1459949514 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x5598c112dd30 con 0x5598c106eb70 2024-09-21T03:41:43.649 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-09-21T03:41:43.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.647+0000 7f7c93594640 1 -- 172.21.15.38:0/1459949514 <== osd.7 v2:172.21.15.195:6818/1606106342 3 ==== osd_op_reply(3 601.00000000 [create] v109'383 uv383 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7c8c054370 con 0x5598c106eb70 2024-09-21T03:41:43.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.647+0000 7f7c99d1df40 1 -- 172.21.15.38:0/1459949514 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x5598c112f0e0 con 0x5598c106eb70 2024-09-21T03:41:43.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.648+0000 7f7c93594640 1 -- 172.21.15.38:0/1459949514 <== osd.7 v2:172.21.15.195:6818/1606106342 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v109'384 uv384 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7c8c054370 con 0x5598c106eb70 2024-09-21T03:41:43.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.648+0000 7f7c99d1df40 1 -- 172.21.15.38:0/1459949514 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x5598c112f480 con 0x5598c106eb70 2024-09-21T03:41:43.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.650+0000 7f7c93594640 1 -- 172.21.15.38:0/1459949514 <== osd.7 v2:172.21.15.195:6818/1606106342 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v109'385 uv385 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7c8c054370 con 0x5598c106eb70 2024-09-21T03:41:43.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.650+0000 7f7c99d1df40 1 -- 172.21.15.38:0/1459949514 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x5598c112e0d0 con 0x5598c106eb70 2024-09-21T03:41:43.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.651+0000 7f7c93594640 1 -- 172.21.15.38:0/1459949514 <== osd.7 v2:172.21.15.195:6818/1606106342 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v109'386 uv386 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7c8c054370 con 0x5598c106eb70 2024-09-21T03:41:43.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.651+0000 7f7c99d1df40 1 -- 172.21.15.38:0/1459949514 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x5598c112e4c0 con 0x5598c106eb70 2024-09-21T03:41:43.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.653+0000 7f7c93594640 1 -- 172.21.15.38:0/1459949514 <== osd.7 v2:172.21.15.195:6818/1606106342 7 ==== osd_op_reply(7 602.00000000 [create] v109'387 uv361 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7c8c054370 con 0x5598c106eb70 2024-09-21T03:41:43.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.653+0000 7f7c99d1df40 1 -- 172.21.15.38:0/1459949514 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e109) -- 0x5598c112e4c0 con 0x5598c106eb70 2024-09-21T03:41:43.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.654+0000 7f7c93594640 1 -- 172.21.15.38:0/1459949514 <== osd.7 v2:172.21.15.195:6818/1606106342 8 ==== osd_op_reply(8 602.00000000 [delete] v109'388 uv388 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7c8c054370 con 0x5598c106eb70 2024-09-21T03:41:43.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.654+0000 7f7c99d1df40 1 -- 172.21.15.38:0/1459949514 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x5598c112e470 con 0x5598c106eb70 2024-09-21T03:41:43.657 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-09-21T03:41:43.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.656+0000 7f7c93594640 1 -- 172.21.15.38:0/1459949514 <== osd.7 v2:172.21.15.195:6818/1606106342 9 ==== osd_op_reply(9 602.00000000 [create] v109'389 uv389 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7c8c054370 con 0x5598c106eb70 2024-09-21T03:41:43.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.656+0000 7f7c99d1df40 1 -- 172.21.15.38:0/1459949514 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x5598c112e470 con 0x5598c106eb70 2024-09-21T03:41:43.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.657+0000 7f7c93594640 1 -- 172.21.15.38:0/1459949514 <== osd.7 v2:172.21.15.195:6818/1606106342 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v109'390 uv390 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7c8c054370 con 0x5598c106eb70 2024-09-21T03:41:43.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.657+0000 7f7c99d1df40 1 -- 172.21.15.38:0/1459949514 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x5598c112e470 con 0x5598c106eb70 2024-09-21T03:41:43.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.658+0000 7f7c93594640 1 -- 172.21.15.38:0/1459949514 <== osd.7 v2:172.21.15.195:6818/1606106342 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v109'391 uv391 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7c8c054370 con 0x5598c106eb70 2024-09-21T03:41:43.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.658+0000 7f7c99d1df40 1 -- 172.21.15.38:0/1459949514 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x5598c112e470 con 0x5598c106eb70 2024-09-21T03:41:43.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.660+0000 7f7c93594640 1 -- 172.21.15.38:0/1459949514 <== osd.7 v2:172.21.15.195:6818/1606106342 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v109'392 uv392 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7c8c054370 con 0x5598c106eb70 2024-09-21T03:41:43.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.660+0000 7f7c99d1df40 1 -- 172.21.15.38:0/1459949514 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:13 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x5598c112e470 con 0x5598c106eb70 2024-09-21T03:41:43.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.661+0000 7f7c93594640 1 -- 172.21.15.38:0/1459949514 <== osd.7 v2:172.21.15.195:6818/1606106342 13 ==== osd_op_reply(13 10000000000.00000000 [create] v109'393 uv368 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7f7c8c054370 con 0x5598c106eb70 2024-09-21T03:41:43.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.661+0000 7f7c99d1df40 1 -- 172.21.15.38:0/1459949514 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:14 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e109) -- 0x5598c112e470 con 0x5598c106eb70 2024-09-21T03:41:43.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.663+0000 7f7c93594640 1 -- 172.21.15.38:0/1459949514 <== osd.7 v2:172.21.15.195:6818/1606106342 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v109'394 uv394 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f7c8c054370 con 0x5598c106eb70 2024-09-21T03:41:43.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.663+0000 7f7c99d1df40 1 -- 172.21.15.38:0/1459949514 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:15 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x5598c112e470 con 0x5598c106eb70 2024-09-21T03:41:43.665 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-09-21T03:41:43.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.664+0000 7f7c93594640 1 -- 172.21.15.38:0/1459949514 <== osd.7 v2:172.21.15.195:6818/1606106342 15 ==== osd_op_reply(15 10000000000.00000000 [create] v109'395 uv395 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f7c8c054370 con 0x5598c106eb70 2024-09-21T03:41:43.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.664+0000 7f7c99d1df40 1 -- 172.21.15.38:0/1459949514 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:16 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x5598c112e470 con 0x5598c106eb70 2024-09-21T03:41:43.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.666+0000 7f7c93594640 1 -- 172.21.15.38:0/1459949514 <== osd.7 v2:172.21.15.195:6818/1606106342 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v109'396 uv396 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f7c8c054370 con 0x5598c106eb70 2024-09-21T03:41:43.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.666+0000 7f7c99d1df40 1 -- 172.21.15.38:0/1459949514 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:17 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x5598c112e470 con 0x5598c106eb70 2024-09-21T03:41:43.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.667+0000 7f7c93594640 1 -- 172.21.15.38:0/1459949514 <== osd.7 v2:172.21.15.195:6818/1606106342 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v109'397 uv397 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f7c8c054370 con 0x5598c106eb70 2024-09-21T03:41:43.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.667+0000 7f7c99d1df40 1 -- 172.21.15.38:0/1459949514 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:18 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x5598c112e470 con 0x5598c106eb70 2024-09-21T03:41:43.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.668+0000 7f7c93594640 1 -- 172.21.15.38:0/1459949514 <== osd.7 v2:172.21.15.195:6818/1606106342 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v109'398 uv398 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f7c8c054370 con 0x5598c106eb70 2024-09-21T03:41:43.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.668+0000 7f7c99d1df40 1 -- 172.21.15.38:0/1459949514 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:19 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=585b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x5598c112e860 con 0x5598c106eb70 2024-09-21T03:41:43.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.670+0000 7f7c93594640 1 -- 172.21.15.38:0/1459949514 <== osd.7 v2:172.21.15.195:6818/1606106342 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v109'399 uv399 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f7c8c054370 con 0x5598c106eb70 2024-09-21T03:41:43.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.670+0000 7f7c99d1df40 1 -- 172.21.15.38:0/1459949514 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:20 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x5598c112ecd0 con 0x5598c106eb70 2024-09-21T03:41:43.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.671+0000 7f7c93594640 1 -- 172.21.15.38:0/1459949514 <== osd.7 v2:172.21.15.195:6818/1606106342 20 ==== osd_op_reply(20 600.00000000 [create] v109'400 uv374 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7c8c054370 con 0x5598c106eb70 2024-09-21T03:41:43.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.671+0000 7f7c99d1df40 1 -- 172.21.15.38:0/1459949514 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:21 13.10 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e109) -- 0x5598c112ecd0 con 0x5598c106eb70 2024-09-21T03:41:43.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.673+0000 7f7c93594640 1 -- 172.21.15.38:0/1459949514 <== osd.7 v2:172.21.15.195:6818/1606106342 21 ==== osd_op_reply(21 600.00000000 [delete] v109'401 uv401 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7c8c054370 con 0x5598c106eb70 2024-09-21T03:41:43.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.673+0000 7f7c99d1df40 1 -- 172.21.15.38:0/1459949514 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:22 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x5598c112e720 con 0x5598c106eb70 2024-09-21T03:41:43.675 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-09-21T03:41:43.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.674+0000 7f7c93594640 1 -- 172.21.15.38:0/1459949514 <== osd.7 v2:172.21.15.195:6818/1606106342 22 ==== osd_op_reply(22 600.00000000 [create] v109'402 uv402 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7c8c054370 con 0x5598c106eb70 2024-09-21T03:41:43.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.674+0000 7f7c99d1df40 1 -- 172.21.15.38:0/1459949514 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:23 13.10 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x5598c112e720 con 0x5598c106eb70 2024-09-21T03:41:43.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.675+0000 7f7c93594640 1 -- 172.21.15.38:0/1459949514 <== osd.7 v2:172.21.15.195:6818/1606106342 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v109'403 uv403 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7c8c054370 con 0x5598c106eb70 2024-09-21T03:41:43.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.675+0000 7f7c99d1df40 1 -- 172.21.15.38:0/1459949514 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:24 13.10 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x5598c112e720 con 0x5598c106eb70 2024-09-21T03:41:43.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.676+0000 7f7c93594640 1 -- 172.21.15.38:0/1459949514 <== osd.7 v2:172.21.15.195:6818/1606106342 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v109'404 uv404 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7c8c054370 con 0x5598c106eb70 2024-09-21T03:41:43.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.677+0000 7f7c99d1df40 1 -- 172.21.15.38:0/1459949514 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:25 13.10 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x5598c112e720 con 0x5598c106eb70 2024-09-21T03:41:43.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.678+0000 7f7c93594640 1 -- 172.21.15.38:0/1459949514 <== osd.7 v2:172.21.15.195:6818/1606106342 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v109'405 uv405 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7c8c054370 con 0x5598c106eb70 2024-09-21T03:41:43.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.678+0000 7f7c99d1df40 1 -- 172.21.15.38:0/1459949514 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:26 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x5598c112e720 con 0x5598c106eb70 2024-09-21T03:41:43.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.680+0000 7f7c93594640 1 -- 172.21.15.38:0/1459949514 <== osd.7 v2:172.21.15.195:6818/1606106342 26 ==== osd_op_reply(26 603.00000000 [create] v109'406 uv380 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7c8c054370 con 0x5598c106eb70 2024-09-21T03:41:43.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.680+0000 7f7c99d1df40 1 -- 172.21.15.38:0/1459949514 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:27 13.10 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e109) -- 0x5598c112e720 con 0x5598c106eb70 2024-09-21T03:41:43.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.681+0000 7f7c93594640 1 -- 172.21.15.38:0/1459949514 <== osd.7 v2:172.21.15.195:6818/1606106342 27 ==== osd_op_reply(27 603.00000000 [delete] v109'407 uv407 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7c8c054370 con 0x5598c106eb70 2024-09-21T03:41:43.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.681+0000 7f7c99d1df40 1 -- 172.21.15.38:0/1459949514 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:28 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x5598c112e720 con 0x5598c106eb70 2024-09-21T03:41:43.684 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-09-21T03:41:43.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.683+0000 7f7c93594640 1 -- 172.21.15.38:0/1459949514 <== osd.7 v2:172.21.15.195:6818/1606106342 28 ==== osd_op_reply(28 603.00000000 [create] v109'408 uv408 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7c8c054370 con 0x5598c106eb70 2024-09-21T03:41:43.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.683+0000 7f7c99d1df40 1 -- 172.21.15.38:0/1459949514 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:29 13.10 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x5598c112e720 con 0x5598c106eb70 2024-09-21T03:41:43.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.683+0000 7f7c93594640 1 -- 172.21.15.38:0/1459949514 <== osd.7 v2:172.21.15.195:6818/1606106342 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v109'409 uv409 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7c8c054370 con 0x5598c106eb70 2024-09-21T03:41:43.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.684+0000 7f7c99d1df40 1 -- 172.21.15.38:0/1459949514 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:30 13.10 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x5598c112e720 con 0x5598c106eb70 2024-09-21T03:41:43.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.686+0000 7f7c93594640 1 -- 172.21.15.38:0/1459949514 <== osd.7 v2:172.21.15.195:6818/1606106342 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v109'410 uv410 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7c8c054370 con 0x5598c106eb70 2024-09-21T03:41:43.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.686+0000 7f7c99d1df40 1 -- 172.21.15.38:0/1459949514 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:31 13.10 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x5598c112e720 con 0x5598c106eb70 2024-09-21T03:41:43.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.686+0000 7f7c93594640 1 -- 172.21.15.38:0/1459949514 <== osd.7 v2:172.21.15.195:6818/1606106342 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v109'411 uv411 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7c8c054370 con 0x5598c106eb70 2024-09-21T03:41:43.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.686+0000 7f7c99d1df40 1 --2- 172.21.15.38:0/1459949514 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x5598c112ec30 0x5598c1131830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:43.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.687+0000 7f7c99d1df40 1 -- 172.21.15.38:0/1459949514 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x5598c1131d70 con 0x5598c112ec30 2024-09-21T03:41:43.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.687+0000 7f7c92d93640 1 --2- 172.21.15.38:0/1459949514 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x5598c112ec30 0x5598c1131830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:43.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.687+0000 7f7c92d93640 1 --2- 172.21.15.38:0/1459949514 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x5598c112ec30 0x5598c1131830 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:43.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.689+0000 7f7c92d93640 1 -- 172.21.15.38:0/1459949514 <== osd.6 v2:172.21.15.195:6826/4055825675 1 ==== osd_op_reply(32 1.00000000.inode [create] v109'286 uv265 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f7c8405e710 con 0x5598c112ec30 2024-09-21T03:41:43.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.689+0000 7f7c99d1df40 1 -- 172.21.15.38:0/1459949514 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e109) -- 0x5598c1132110 con 0x5598c112ec30 2024-09-21T03:41:43.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.691+0000 7f7c92d93640 1 -- 172.21.15.38:0/1459949514 <== osd.6 v2:172.21.15.195:6826/4055825675 2 ==== osd_op_reply(33 1.00000000.inode [delete] v109'287 uv287 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f7c8405e710 con 0x5598c112ec30 2024-09-21T03:41:43.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.691+0000 7f7c99d1df40 1 -- 172.21.15.38:0/1459949514 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x5598c11324b0 con 0x5598c112ec30 2024-09-21T03:41:43.694 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-09-21T03:41:43.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.692+0000 7f7c92d93640 1 -- 172.21.15.38:0/1459949514 <== osd.6 v2:172.21.15.195:6826/4055825675 3 ==== osd_op_reply(34 1.00000000.inode [create] v109'288 uv288 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f7c8405e710 con 0x5598c112ec30 2024-09-21T03:41:43.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.692+0000 7f7c99d1df40 1 -- 172.21.15.38:0/1459949514 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x5598c1132d00 con 0x5598c112ec30 2024-09-21T03:41:43.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.694+0000 7f7c92d93640 1 -- 172.21.15.38:0/1459949514 <== osd.6 v2:172.21.15.195:6826/4055825675 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v109'289 uv289 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f7c8405e710 con 0x5598c112ec30 2024-09-21T03:41:43.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.694+0000 7f7c99d1df40 1 -- 172.21.15.38:0/1459949514 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x5598c11330a0 con 0x5598c112ec30 2024-09-21T03:41:43.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.696+0000 7f7c92d93640 1 -- 172.21.15.38:0/1459949514 <== osd.6 v2:172.21.15.195:6826/4055825675 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v109'290 uv290 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f7c8405e710 con 0x5598c112ec30 2024-09-21T03:41:43.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.696+0000 7f7c99d1df40 1 -- 172.21.15.38:0/1459949514 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x5598c1133440 con 0x5598c112ec30 2024-09-21T03:41:43.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.697+0000 7f7c92d93640 1 -- 172.21.15.38:0/1459949514 <== osd.6 v2:172.21.15.195:6826/4055825675 6 ==== osd_op_reply(37 200.00000000 [create] v109'291 uv284 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7c8405e710 con 0x5598c112ec30 2024-09-21T03:41:43.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.697+0000 7f7c99d1df40 1 -- 172.21.15.38:0/1459949514 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e109) -- 0x5598c11337e0 con 0x5598c112ec30 2024-09-21T03:41:43.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.699+0000 7f7c92d93640 1 -- 172.21.15.38:0/1459949514 <== osd.6 v2:172.21.15.195:6826/4055825675 7 ==== osd_op_reply(38 200.00000000 [delete] v109'292 uv292 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7c8405e710 con 0x5598c112ec30 2024-09-21T03:41:43.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.699+0000 7f7c99d1df40 1 -- 172.21.15.38:0/1459949514 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x5598c1133b80 con 0x5598c112ec30 2024-09-21T03:41:43.702 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-09-21T03:41:43.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.700+0000 7f7c92d93640 1 -- 172.21.15.38:0/1459949514 <== osd.6 v2:172.21.15.195:6826/4055825675 8 ==== osd_op_reply(39 200.00000000 [create] v109'293 uv293 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7c8405e710 con 0x5598c112ec30 2024-09-21T03:41:43.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.701+0000 7f7c99d1df40 1 -- 172.21.15.38:0/1459949514 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x5598c1133b80 con 0x5598c112ec30 2024-09-21T03:41:43.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.702+0000 7f7c92d93640 1 -- 172.21.15.38:0/1459949514 <== osd.6 v2:172.21.15.195:6826/4055825675 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v109'294 uv294 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7c8405e710 con 0x5598c112ec30 2024-09-21T03:41:43.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.702+0000 7f7c99d1df40 1 -- 172.21.15.38:0/1459949514 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x5598c1133b80 con 0x5598c112ec30 2024-09-21T03:41:43.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.704+0000 7f7c92d93640 1 -- 172.21.15.38:0/1459949514 <== osd.6 v2:172.21.15.195:6826/4055825675 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v109'295 uv295 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7c8405e710 con 0x5598c112ec30 2024-09-21T03:41:43.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.704+0000 7f7c99d1df40 1 -- 172.21.15.38:0/1459949514 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:42 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x5598c1133b80 con 0x5598c112ec30 2024-09-21T03:41:43.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.705+0000 7f7c92d93640 1 -- 172.21.15.38:0/1459949514 <== osd.6 v2:172.21.15.195:6826/4055825675 11 ==== osd_op_reply(42 200.00000001 [create] v109'296 uv285 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7c8405e710 con 0x5598c112ec30 2024-09-21T03:41:43.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.705+0000 7f7c99d1df40 1 -- 172.21.15.38:0/1459949514 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:43 13.14 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e109) -- 0x5598c1133b80 con 0x5598c112ec30 2024-09-21T03:41:43.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.707+0000 7f7c92d93640 1 -- 172.21.15.38:0/1459949514 <== osd.6 v2:172.21.15.195:6826/4055825675 12 ==== osd_op_reply(43 200.00000001 [delete] v109'297 uv297 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7c8405e710 con 0x5598c112ec30 2024-09-21T03:41:43.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.707+0000 7f7c99d1df40 1 -- 172.21.15.38:0/1459949514 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:44 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x5598c1133b80 con 0x5598c112ec30 2024-09-21T03:41:43.711 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-09-21T03:41:43.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.709+0000 7f7c92d93640 1 -- 172.21.15.38:0/1459949514 <== osd.6 v2:172.21.15.195:6826/4055825675 13 ==== osd_op_reply(44 200.00000001 [create] v109'298 uv298 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7c8405e710 con 0x5598c112ec30 2024-09-21T03:41:43.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.709+0000 7f7c99d1df40 1 -- 172.21.15.38:0/1459949514 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:45 13.14 13:2e2fa760:::200.00000001:head [write 0~17490 in=17490b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x5598c1138000 con 0x5598c112ec30 2024-09-21T03:41:43.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.711+0000 7f7c92d93640 1 -- 172.21.15.38:0/1459949514 <== osd.6 v2:172.21.15.195:6826/4055825675 14 ==== osd_op_reply(45 200.00000001 [write 0~17490] v109'299 uv299 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7c8405e710 con 0x5598c112ec30 2024-09-21T03:41:43.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.711+0000 7f7c99d1df40 1 -- 172.21.15.38:0/1459949514 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:46 13.14 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x5598c1138000 con 0x5598c112ec30 2024-09-21T03:41:43.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.712+0000 7f7c92d93640 1 -- 172.21.15.38:0/1459949514 <== osd.6 v2:172.21.15.195:6826/4055825675 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v109'300 uv300 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7c8405e710 con 0x5598c112ec30 2024-09-21T03:41:43.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.712+0000 7f7c99d1df40 1 -- 172.21.15.38:0/1459949514 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:47 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x5598c1133b80 con 0x5598c112ec30 2024-09-21T03:41:43.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.714+0000 7f7c92d93640 1 -- 172.21.15.38:0/1459949514 <== osd.6 v2:172.21.15.195:6826/4055825675 16 ==== osd_op_reply(47 606.00000000 [create] v109'301 uv281 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7c8405e710 con 0x5598c112ec30 2024-09-21T03:41:43.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.714+0000 7f7c99d1df40 1 -- 172.21.15.38:0/1459949514 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:48 13.14 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e109) -- 0x5598c1133b80 con 0x5598c112ec30 2024-09-21T03:41:43.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.715+0000 7f7c92d93640 1 -- 172.21.15.38:0/1459949514 <== osd.6 v2:172.21.15.195:6826/4055825675 17 ==== osd_op_reply(48 606.00000000 [delete] v109'302 uv302 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7c8405e710 con 0x5598c112ec30 2024-09-21T03:41:43.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.715+0000 7f7c99d1df40 1 -- 172.21.15.38:0/1459949514 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:49 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x5598c1133b80 con 0x5598c112ec30 2024-09-21T03:41:43.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.716+0000 7f7c92d93640 1 -- 172.21.15.38:0/1459949514 <== osd.6 v2:172.21.15.195:6826/4055825675 18 ==== osd_op_reply(49 606.00000000 [create] v109'303 uv303 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7c8405e710 con 0x5598c112ec30 2024-09-21T03:41:43.719 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-09-21T03:41:43.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.716+0000 7f7c99d1df40 1 -- 172.21.15.38:0/1459949514 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:50 13.14 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x5598c1134b90 con 0x5598c112ec30 2024-09-21T03:41:43.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.718+0000 7f7c92d93640 1 -- 172.21.15.38:0/1459949514 <== osd.6 v2:172.21.15.195:6826/4055825675 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v109'304 uv304 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7c8405e710 con 0x5598c112ec30 2024-09-21T03:41:43.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.718+0000 7f7c99d1df40 1 -- 172.21.15.38:0/1459949514 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:51 13.14 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x5598c1134b90 con 0x5598c112ec30 2024-09-21T03:41:43.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.719+0000 7f7c92d93640 1 -- 172.21.15.38:0/1459949514 <== osd.6 v2:172.21.15.195:6826/4055825675 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v109'305 uv305 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7c8405e710 con 0x5598c112ec30 2024-09-21T03:41:43.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.719+0000 7f7c99d1df40 1 -- 172.21.15.38:0/1459949514 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:52 13.14 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x5598c1133b80 con 0x5598c112ec30 2024-09-21T03:41:43.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.720+0000 7f7c92d93640 1 -- 172.21.15.38:0/1459949514 <== osd.6 v2:172.21.15.195:6826/4055825675 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v109'306 uv306 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7c8405e710 con 0x5598c112ec30 2024-09-21T03:41:43.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.720+0000 7f7c99d1df40 1 --2- 172.21.15.38:0/1459949514 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x5598c1134450 0x5598c1136870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:43.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.720+0000 7f7c99d1df40 1 -- 172.21.15.38:0/1459949514 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:53 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x5598c1136db0 con 0x5598c1134450 2024-09-21T03:41:43.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.720+0000 7f7c92592640 1 --2- 172.21.15.38:0/1459949514 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x5598c1134450 0x5598c1136870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:43.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.721+0000 7f7c92592640 1 --2- 172.21.15.38:0/1459949514 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x5598c1134450 0x5598c1136870 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:43.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.724+0000 7f7c92592640 1 -- 172.21.15.38:0/1459949514 <== osd.0 v2:172.21.15.38:6800/2381639433 1 ==== osd_op_reply(53 607.00000000 [create] v109'75 uv74 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7c8800f750 con 0x5598c1134450 2024-09-21T03:41:43.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.724+0000 7f7c99d1df40 1 -- 172.21.15.38:0/1459949514 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:54 13.1c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e109) -- 0x5598c1137150 con 0x5598c1134450 2024-09-21T03:41:43.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.725+0000 7f7c92592640 1 -- 172.21.15.38:0/1459949514 <== osd.0 v2:172.21.15.38:6800/2381639433 2 ==== osd_op_reply(54 607.00000000 [delete] v109'76 uv76 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7c8800f750 con 0x5598c1134450 2024-09-21T03:41:43.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.725+0000 7f7c99d1df40 1 -- 172.21.15.38:0/1459949514 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:55 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x5598c1137630 con 0x5598c1134450 2024-09-21T03:41:43.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.726+0000 7f7c92592640 1 -- 172.21.15.38:0/1459949514 <== osd.0 v2:172.21.15.38:6800/2381639433 3 ==== osd_op_reply(55 607.00000000 [create] v109'77 uv77 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7c8800f750 con 0x5598c1134450 2024-09-21T03:41:43.728 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-09-21T03:41:43.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.726+0000 7f7c99d1df40 1 -- 172.21.15.38:0/1459949514 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:56 13.1c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x5598c1138ee0 con 0x5598c1134450 2024-09-21T03:41:43.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.727+0000 7f7c92592640 1 -- 172.21.15.38:0/1459949514 <== osd.0 v2:172.21.15.38:6800/2381639433 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v109'78 uv78 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7c8800f750 con 0x5598c1134450 2024-09-21T03:41:43.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.727+0000 7f7c99d1df40 1 -- 172.21.15.38:0/1459949514 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:57 13.1c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x5598c1139280 con 0x5598c1134450 2024-09-21T03:41:43.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.728+0000 7f7c92592640 1 -- 172.21.15.38:0/1459949514 <== osd.0 v2:172.21.15.38:6800/2381639433 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v109'79 uv79 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7c8800f750 con 0x5598c1134450 2024-09-21T03:41:43.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.728+0000 7f7c99d1df40 1 -- 172.21.15.38:0/1459949514 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:58 13.1c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x5598c1137ed0 con 0x5598c1134450 2024-09-21T03:41:43.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.729+0000 7f7c92592640 1 -- 172.21.15.38:0/1459949514 <== osd.0 v2:172.21.15.38:6800/2381639433 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v109'80 uv80 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7c8800f750 con 0x5598c1134450 2024-09-21T03:41:43.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.729+0000 7f7c99d1df40 1 --2- 172.21.15.38:0/1459949514 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x5598c11387d0 0x5598c113b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:43.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.729+0000 7f7c99d1df40 1 -- 172.21.15.38:0/1459949514 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:59 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x5598c113bb70 con 0x5598c11387d0 2024-09-21T03:41:43.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.729+0000 7f7c93594640 1 --2- 172.21.15.38:0/1459949514 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x5598c11387d0 0x5598c113b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:43.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.730+0000 7f7c93594640 1 --2- 172.21.15.38:0/1459949514 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x5598c11387d0 0x5598c113b630 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:43.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.731+0000 7f7c93594640 1 -- 172.21.15.38:0/1459949514 <== osd.3 v2:172.21.15.38:6808/1286810294 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v109'58 uv57 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f7c8c054370 con 0x5598c11387d0 2024-09-21T03:41:43.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.731+0000 7f7c99d1df40 1 -- 172.21.15.38:0/1459949514 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:60 13.6 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e109) -- 0x5598c113bb70 con 0x5598c11387d0 2024-09-21T03:41:43.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.732+0000 7f7c93594640 1 -- 172.21.15.38:0/1459949514 <== osd.3 v2:172.21.15.38:6808/1286810294 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v109'59 uv59 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f7c8c054370 con 0x5598c11387d0 2024-09-21T03:41:43.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.732+0000 7f7c99d1df40 1 -- 172.21.15.38:0/1459949514 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:61 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x5598c113bf60 con 0x5598c11387d0 2024-09-21T03:41:43.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.733+0000 7f7c93594640 1 -- 172.21.15.38:0/1459949514 <== osd.3 v2:172.21.15.38:6808/1286810294 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v109'60 uv60 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f7c8c054370 con 0x5598c11387d0 2024-09-21T03:41:43.735 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-09-21T03:41:43.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.733+0000 7f7c99d1df40 1 -- 172.21.15.38:0/1459949514 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:62 13.6 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x5598c113bf60 con 0x5598c11387d0 2024-09-21T03:41:43.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.734+0000 7f7c93594640 1 -- 172.21.15.38:0/1459949514 <== osd.3 v2:172.21.15.38:6808/1286810294 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v109'61 uv61 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f7c8c054370 con 0x5598c11387d0 2024-09-21T03:41:43.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.734+0000 7f7c99d1df40 1 -- 172.21.15.38:0/1459949514 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x5598c113bf60 con 0x5598c1134450 2024-09-21T03:41:43.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.735+0000 7f7c92592640 1 -- 172.21.15.38:0/1459949514 <== osd.0 v2:172.21.15.38:6800/2381639433 7 ==== osd_op_reply(63 608.00000000 [create] v109'75 uv74 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7c8800f750 con 0x5598c1134450 2024-09-21T03:41:43.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.735+0000 7f7c99d1df40 1 -- 172.21.15.38:0/1459949514 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e109) -- 0x5598c113c300 con 0x5598c1134450 2024-09-21T03:41:43.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.736+0000 7f7c92592640 1 -- 172.21.15.38:0/1459949514 <== osd.0 v2:172.21.15.38:6800/2381639433 8 ==== osd_op_reply(64 608.00000000 [delete] v109'76 uv76 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7c8800f750 con 0x5598c1134450 2024-09-21T03:41:43.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.736+0000 7f7c99d1df40 1 -- 172.21.15.38:0/1459949514 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x5598c113c300 con 0x5598c1134450 2024-09-21T03:41:43.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.738+0000 7f7c92592640 1 -- 172.21.15.38:0/1459949514 <== osd.0 v2:172.21.15.38:6800/2381639433 9 ==== osd_op_reply(65 608.00000000 [create] v109'77 uv77 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7c8800f750 con 0x5598c1134450 2024-09-21T03:41:43.739 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-09-21T03:41:43.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.738+0000 7f7c99d1df40 1 -- 172.21.15.38:0/1459949514 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x5598c113d310 con 0x5598c1134450 2024-09-21T03:41:43.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.739+0000 7f7c92592640 1 -- 172.21.15.38:0/1459949514 <== osd.0 v2:172.21.15.38:6800/2381639433 10 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v109'78 uv78 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7c8800f750 con 0x5598c1134450 2024-09-21T03:41:43.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.739+0000 7f7c99d1df40 1 -- 172.21.15.38:0/1459949514 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x5598c113d310 con 0x5598c1134450 2024-09-21T03:41:43.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.740+0000 7f7c92592640 1 -- 172.21.15.38:0/1459949514 <== osd.0 v2:172.21.15.38:6800/2381639433 11 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v109'79 uv79 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7c8800f750 con 0x5598c1134450 2024-09-21T03:41:43.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.740+0000 7f7c99d1df40 1 -- 172.21.15.38:0/1459949514 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x5598c113c300 con 0x5598c1134450 2024-09-21T03:41:43.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.741+0000 7f7c92592640 1 -- 172.21.15.38:0/1459949514 <== osd.0 v2:172.21.15.38:6800/2381639433 12 ==== osd_op_reply(68 608.00000000 [omap-set-header] v109'80 uv80 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7c8800f750 con 0x5598c1134450 2024-09-21T03:41:43.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.741+0000 7f7c99d1df40 1 -- 172.21.15.38:0/1459949514 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x5598c113c300 con 0x5598c11387d0 2024-09-21T03:41:43.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.743+0000 7f7c93594640 1 -- 172.21.15.38:0/1459949514 <== osd.3 v2:172.21.15.38:6808/1286810294 5 ==== osd_op_reply(69 604.00000000 [create] v109'138 uv132 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7c8c054370 con 0x5598c11387d0 2024-09-21T03:41:43.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.743+0000 7f7c99d1df40 1 -- 172.21.15.38:0/1459949514 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e109) -- 0x5598c113c300 con 0x5598c11387d0 2024-09-21T03:41:43.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.744+0000 7f7c93594640 1 -- 172.21.15.38:0/1459949514 <== osd.3 v2:172.21.15.38:6808/1286810294 6 ==== osd_op_reply(70 604.00000000 [delete] v109'139 uv139 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7c8c054370 con 0x5598c11387d0 2024-09-21T03:41:43.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.744+0000 7f7c99d1df40 1 -- 172.21.15.38:0/1459949514 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x5598c113c300 con 0x5598c11387d0 2024-09-21T03:41:43.747 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-09-21T03:41:43.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.746+0000 7f7c93594640 1 -- 172.21.15.38:0/1459949514 <== osd.3 v2:172.21.15.38:6808/1286810294 7 ==== osd_op_reply(71 604.00000000 [create] v109'140 uv140 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7c8c054370 con 0x5598c11387d0 2024-09-21T03:41:43.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.746+0000 7f7c99d1df40 1 -- 172.21.15.38:0/1459949514 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x5598c113d310 con 0x5598c11387d0 2024-09-21T03:41:43.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.747+0000 7f7c93594640 1 -- 172.21.15.38:0/1459949514 <== osd.3 v2:172.21.15.38:6808/1286810294 8 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v109'141 uv141 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7c8c054370 con 0x5598c11387d0 2024-09-21T03:41:43.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.747+0000 7f7c99d1df40 1 -- 172.21.15.38:0/1459949514 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x5598c113d310 con 0x5598c11387d0 2024-09-21T03:41:43.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.748+0000 7f7c93594640 1 -- 172.21.15.38:0/1459949514 <== osd.3 v2:172.21.15.38:6808/1286810294 9 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v109'142 uv142 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7c8c054370 con 0x5598c11387d0 2024-09-21T03:41:43.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.748+0000 7f7c99d1df40 1 -- 172.21.15.38:0/1459949514 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x5598c113c300 con 0x5598c11387d0 2024-09-21T03:41:43.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.749+0000 7f7c93594640 1 -- 172.21.15.38:0/1459949514 <== osd.3 v2:172.21.15.38:6808/1286810294 10 ==== osd_op_reply(74 604.00000000 [omap-set-header] v109'143 uv143 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7c8c054370 con 0x5598c11387d0 2024-09-21T03:41:43.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.749+0000 7f7c99d1df40 1 -- 172.21.15.38:0/1459949514 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x5598c113c300 con 0x5598c11387d0 2024-09-21T03:41:43.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.750+0000 7f7c93594640 1 -- 172.21.15.38:0/1459949514 <== osd.3 v2:172.21.15.38:6808/1286810294 11 ==== osd_op_reply(75 500.00000000 [create] v109'144 uv137 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7c8c054370 con 0x5598c11387d0 2024-09-21T03:41:43.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.750+0000 7f7c99d1df40 1 -- 172.21.15.38:0/1459949514 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e109) -- 0x5598c113c300 con 0x5598c11387d0 2024-09-21T03:41:43.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.751+0000 7f7c93594640 1 -- 172.21.15.38:0/1459949514 <== osd.3 v2:172.21.15.38:6808/1286810294 12 ==== osd_op_reply(76 500.00000000 [delete] v109'145 uv145 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7c8c054370 con 0x5598c11387d0 2024-09-21T03:41:43.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.751+0000 7f7c99d1df40 1 -- 172.21.15.38:0/1459949514 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x5598c113c300 con 0x5598c11387d0 2024-09-21T03:41:43.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.752+0000 7f7c93594640 1 -- 172.21.15.38:0/1459949514 <== osd.3 v2:172.21.15.38:6808/1286810294 13 ==== osd_op_reply(77 500.00000000 [create] v109'146 uv146 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7c8c054370 con 0x5598c11387d0 2024-09-21T03:41:43.754 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-09-21T03:41:43.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.752+0000 7f7c99d1df40 1 -- 172.21.15.38:0/1459949514 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x5598c113c300 con 0x5598c11387d0 2024-09-21T03:41:43.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.753+0000 7f7c93594640 1 -- 172.21.15.38:0/1459949514 <== osd.3 v2:172.21.15.38:6808/1286810294 14 ==== osd_op_reply(78 500.00000000 [write 0~90] v109'147 uv147 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7c8c054370 con 0x5598c11387d0 2024-09-21T03:41:43.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.753+0000 7f7c99d1df40 1 -- 172.21.15.38:0/1459949514 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x5598c113c300 con 0x5598c11387d0 2024-09-21T03:41:43.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.754+0000 7f7c93594640 1 -- 172.21.15.38:0/1459949514 <== osd.3 v2:172.21.15.38:6808/1286810294 15 ==== osd_op_reply(79 500.00000000 [omap-set-header] v109'148 uv148 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7c8c054370 con 0x5598c11387d0 2024-09-21T03:41:43.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.754+0000 7f7c99d1df40 1 -- 172.21.15.38:0/1459949514 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:80 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x5598c113c300 con 0x5598c1134450 2024-09-21T03:41:43.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.756+0000 7f7c92592640 1 -- 172.21.15.38:0/1459949514 <== osd.0 v2:172.21.15.38:6800/2381639433 13 ==== osd_op_reply(80 mds_snaptable [create] v109'61 uv60 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f7c8800f750 con 0x5598c1134450 2024-09-21T03:41:43.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.756+0000 7f7c99d1df40 1 -- 172.21.15.38:0/1459949514 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:81 13.d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e109) -- 0x5598c113c300 con 0x5598c1134450 2024-09-21T03:41:43.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.757+0000 7f7c92592640 1 -- 172.21.15.38:0/1459949514 <== osd.0 v2:172.21.15.38:6800/2381639433 14 ==== osd_op_reply(81 mds_snaptable [delete] v109'62 uv62 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f7c8800f750 con 0x5598c1134450 2024-09-21T03:41:43.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.757+0000 7f7c99d1df40 1 -- 172.21.15.38:0/1459949514 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:82 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x5598c113c300 con 0x5598c1134450 2024-09-21T03:41:43.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.758+0000 7f7c92592640 1 -- 172.21.15.38:0/1459949514 <== osd.0 v2:172.21.15.38:6800/2381639433 15 ==== osd_op_reply(82 mds_snaptable [create] v109'63 uv63 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f7c8800f750 con 0x5598c1134450 2024-09-21T03:41:43.760 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-09-21T03:41:43.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.758+0000 7f7c99d1df40 1 -- 172.21.15.38:0/1459949514 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:83 13.d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x5598c113c300 con 0x5598c1134450 2024-09-21T03:41:43.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.759+0000 7f7c92592640 1 -- 172.21.15.38:0/1459949514 <== osd.0 v2:172.21.15.38:6800/2381639433 16 ==== osd_op_reply(83 mds_snaptable [write 0~70] v109'64 uv64 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f7c8800f750 con 0x5598c1134450 2024-09-21T03:41:43.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.759+0000 7f7c99d1df40 1 -- 172.21.15.38:0/1459949514 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:84 13.d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x5598c113c300 con 0x5598c1134450 2024-09-21T03:41:43.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.760+0000 7f7c92592640 1 -- 172.21.15.38:0/1459949514 <== osd.0 v2:172.21.15.38:6800/2381639433 17 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v109'65 uv65 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f7c8800f750 con 0x5598c1134450 2024-09-21T03:41:43.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.760+0000 7f7c99d1df40 1 --2- 172.21.15.38:0/1459949514 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x5598c113c980 0x5598c113ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:43.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.760+0000 7f7c99d1df40 1 -- 172.21.15.38:0/1459949514 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x5598c113f440 con 0x5598c113c980 2024-09-21T03:41:43.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.761+0000 7f7c92d93640 1 --2- 172.21.15.38:0/1459949514 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x5598c113c980 0x5598c113ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:43.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.761+0000 7f7c92d93640 1 --2- 172.21.15.38:0/1459949514 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x5598c113c980 0x5598c113ee00 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:43.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.762+0000 7f7c92d93640 1 -- 172.21.15.38:0/1459949514 <== osd.1 v2:172.21.15.38:6816/1491818732 1 ==== osd_op_reply(85 605.00000000 [create] v109'75 uv74 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7c8405e710 con 0x5598c113c980 2024-09-21T03:41:43.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.762+0000 7f7c99d1df40 1 -- 172.21.15.38:0/1459949514 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e109) -- 0x5598c113f830 con 0x5598c113c980 2024-09-21T03:41:43.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.763+0000 7f7c92d93640 1 -- 172.21.15.38:0/1459949514 <== osd.1 v2:172.21.15.38:6816/1491818732 2 ==== osd_op_reply(86 605.00000000 [delete] v109'76 uv76 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7c8405e710 con 0x5598c113c980 2024-09-21T03:41:43.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.763+0000 7f7c99d1df40 1 -- 172.21.15.38:0/1459949514 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x5598c113f830 con 0x5598c113c980 2024-09-21T03:41:43.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.764+0000 7f7c92d93640 1 -- 172.21.15.38:0/1459949514 <== osd.1 v2:172.21.15.38:6816/1491818732 3 ==== osd_op_reply(87 605.00000000 [create] v109'77 uv77 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7c8405e710 con 0x5598c113c980 2024-09-21T03:41:43.766 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-09-21T03:41:43.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.764+0000 7f7c99d1df40 1 -- 172.21.15.38:0/1459949514 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x5598c1140840 con 0x5598c113c980 2024-09-21T03:41:43.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.765+0000 7f7c92d93640 1 -- 172.21.15.38:0/1459949514 <== osd.1 v2:172.21.15.38:6816/1491818732 4 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v109'78 uv78 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7c8405e710 con 0x5598c113c980 2024-09-21T03:41:43.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.765+0000 7f7c99d1df40 1 -- 172.21.15.38:0/1459949514 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x5598c1140840 con 0x5598c113c980 2024-09-21T03:41:43.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.766+0000 7f7c92d93640 1 -- 172.21.15.38:0/1459949514 <== osd.1 v2:172.21.15.38:6816/1491818732 5 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v109'79 uv79 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7c8405e710 con 0x5598c113c980 2024-09-21T03:41:43.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.766+0000 7f7c99d1df40 1 -- 172.21.15.38:0/1459949514 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x5598c113f830 con 0x5598c113c980 2024-09-21T03:41:43.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.768+0000 7f7c92d93640 1 -- 172.21.15.38:0/1459949514 <== osd.1 v2:172.21.15.38:6816/1491818732 6 ==== osd_op_reply(90 605.00000000 [omap-set-header] v109'80 uv80 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7c8405e710 con 0x5598c113c980 2024-09-21T03:41:43.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.768+0000 7f7c99d1df40 1 -- 172.21.15.38:0/1459949514 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x5598c113f830 con 0x5598c113c980 2024-09-21T03:41:43.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.769+0000 7f7c92d93640 1 -- 172.21.15.38:0/1459949514 <== osd.1 v2:172.21.15.38:6816/1491818732 7 ==== osd_op_reply(91 mds0_inotable [create] v109'122 uv116 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f7c8405e710 con 0x5598c113c980 2024-09-21T03:41:43.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.769+0000 7f7c99d1df40 1 -- 172.21.15.38:0/1459949514 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e109) -- 0x5598c113f830 con 0x5598c113c980 2024-09-21T03:41:43.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.770+0000 7f7c92d93640 1 -- 172.21.15.38:0/1459949514 <== osd.1 v2:172.21.15.38:6816/1491818732 8 ==== osd_op_reply(92 mds0_inotable [delete] v109'123 uv123 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f7c8405e710 con 0x5598c113c980 2024-09-21T03:41:43.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.770+0000 7f7c99d1df40 1 -- 172.21.15.38:0/1459949514 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x5598c113f830 con 0x5598c113c980 2024-09-21T03:41:43.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.771+0000 7f7c92d93640 1 -- 172.21.15.38:0/1459949514 <== osd.1 v2:172.21.15.38:6816/1491818732 9 ==== osd_op_reply(93 mds0_inotable [create] v109'124 uv124 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f7c8405e710 con 0x5598c113c980 2024-09-21T03:41:43.773 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-09-21T03:41:43.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.771+0000 7f7c99d1df40 1 -- 172.21.15.38:0/1459949514 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:94 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x5598c113f830 con 0x5598c113c980 2024-09-21T03:41:43.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.772+0000 7f7c92d93640 1 -- 172.21.15.38:0/1459949514 <== osd.1 v2:172.21.15.38:6816/1491818732 10 ==== osd_op_reply(94 mds0_inotable [write 0~34] v109'125 uv125 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f7c8405e710 con 0x5598c113c980 2024-09-21T03:41:43.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.772+0000 7f7c99d1df40 1 -- 172.21.15.38:0/1459949514 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:95 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x5598c113f830 con 0x5598c113c980 2024-09-21T03:41:43.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.773+0000 7f7c92d93640 1 -- 172.21.15.38:0/1459949514 <== osd.1 v2:172.21.15.38:6816/1491818732 11 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v109'126 uv126 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f7c8405e710 con 0x5598c113c980 2024-09-21T03:41:43.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.773+0000 7f7c99d1df40 1 -- 172.21.15.38:0/1459949514 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:96 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x5598c113f830 con 0x5598c113c980 2024-09-21T03:41:43.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.774+0000 7f7c92d93640 1 -- 172.21.15.38:0/1459949514 <== osd.1 v2:172.21.15.38:6816/1491818732 12 ==== osd_op_reply(96 100.00000000 [create] v109'127 uv121 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7c8405e710 con 0x5598c113c980 2024-09-21T03:41:43.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.774+0000 7f7c99d1df40 1 -- 172.21.15.38:0/1459949514 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:97 13.13 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e109) -- 0x5598c113f830 con 0x5598c113c980 2024-09-21T03:41:43.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.775+0000 7f7c92d93640 1 -- 172.21.15.38:0/1459949514 <== osd.1 v2:172.21.15.38:6816/1491818732 13 ==== osd_op_reply(97 100.00000000 [delete] v109'128 uv128 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7c8405e710 con 0x5598c113c980 2024-09-21T03:41:43.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.775+0000 7f7c99d1df40 1 -- 172.21.15.38:0/1459949514 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:98 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x5598c113f830 con 0x5598c113c980 2024-09-21T03:41:43.777 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-09-21T03:41:43.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.776+0000 7f7c92d93640 1 -- 172.21.15.38:0/1459949514 <== osd.1 v2:172.21.15.38:6816/1491818732 14 ==== osd_op_reply(98 100.00000000 [create] v109'129 uv129 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7c8405e710 con 0x5598c113c980 2024-09-21T03:41:43.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.776+0000 7f7c99d1df40 1 -- 172.21.15.38:0/1459949514 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:99 13.13 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x5598c113f830 con 0x5598c113c980 2024-09-21T03:41:43.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.777+0000 7f7c92d93640 1 -- 172.21.15.38:0/1459949514 <== osd.1 v2:172.21.15.38:6816/1491818732 15 ==== osd_op_reply(99 100.00000000 [omap-set-header] v109'130 uv130 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7c8405e710 con 0x5598c113c980 2024-09-21T03:41:43.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.777+0000 7f7c99d1df40 1 -- 172.21.15.38:0/1459949514 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:100 13.13 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x5598c1137af0 con 0x5598c113c980 2024-09-21T03:41:43.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.778+0000 7f7c92d93640 1 -- 172.21.15.38:0/1459949514 <== osd.1 v2:172.21.15.38:6816/1491818732 16 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v109'131 uv131 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7c8405e710 con 0x5598c113c980 2024-09-21T03:41:43.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.778+0000 7f7c99d1df40 1 -- 172.21.15.38:0/1459949514 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x5598c1137ab0 con 0x5598c11387d0 2024-09-21T03:41:43.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.779+0000 7f7c93594640 1 -- 172.21.15.38:0/1459949514 <== osd.3 v2:172.21.15.38:6808/1286810294 16 ==== osd_op_reply(101 mds0_sessionmap [create] v109'57 uv55 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7f7c8c054370 con 0x5598c11387d0 2024-09-21T03:41:43.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.779+0000 7f7c99d1df40 1 -- 172.21.15.38:0/1459949514 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e109) -- 0x5598c1137ab0 con 0x5598c11387d0 2024-09-21T03:41:43.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.780+0000 7f7c93594640 1 -- 172.21.15.38:0/1459949514 <== osd.3 v2:172.21.15.38:6808/1286810294 17 ==== osd_op_reply(102 mds0_sessionmap [delete] v109'58 uv58 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f7c8c054370 con 0x5598c11387d0 2024-09-21T03:41:43.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.780+0000 7f7c99d1df40 1 -- 172.21.15.38:0/1459949514 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:103 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x5598c1137ab0 con 0x5598c11387d0 2024-09-21T03:41:43.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.781+0000 7f7c93594640 1 -- 172.21.15.38:0/1459949514 <== osd.3 v2:172.21.15.38:6808/1286810294 18 ==== osd_op_reply(103 mds0_sessionmap [create] v109'59 uv59 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f7c8c054370 con 0x5598c11387d0 2024-09-21T03:41:43.783 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-09-21T03:41:43.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.781+0000 7f7c99d1df40 1 -- 172.21.15.38:0/1459949514 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:104 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x5598c1137ab0 con 0x5598c11387d0 2024-09-21T03:41:43.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.782+0000 7f7c93594640 1 -- 172.21.15.38:0/1459949514 <== osd.3 v2:172.21.15.38:6808/1286810294 19 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v109'60 uv60 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f7c8c054370 con 0x5598c11387d0 2024-09-21T03:41:43.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.782+0000 7f7c99d1df40 1 -- 172.21.15.38:0/1459949514 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x5598c1137ab0 con 0x5598c11387d0 2024-09-21T03:41:43.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.783+0000 7f7c93594640 1 -- 172.21.15.38:0/1459949514 <== osd.3 v2:172.21.15.38:6808/1286810294 20 ==== osd_op_reply(105 609.00000000 [create] v109'75 uv74 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7c8c054370 con 0x5598c11387d0 2024-09-21T03:41:43.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.783+0000 7f7c99d1df40 1 -- 172.21.15.38:0/1459949514 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e109) -- 0x5598c1137ab0 con 0x5598c11387d0 2024-09-21T03:41:43.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.784+0000 7f7c93594640 1 -- 172.21.15.38:0/1459949514 <== osd.3 v2:172.21.15.38:6808/1286810294 21 ==== osd_op_reply(106 609.00000000 [delete] v109'76 uv76 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7c8c054370 con 0x5598c11387d0 2024-09-21T03:41:43.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.784+0000 7f7c99d1df40 1 -- 172.21.15.38:0/1459949514 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x5598c1137ab0 con 0x5598c11387d0 2024-09-21T03:41:43.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.785+0000 7f7c93594640 1 -- 172.21.15.38:0/1459949514 <== osd.3 v2:172.21.15.38:6808/1286810294 22 ==== osd_op_reply(107 609.00000000 [create] v109'77 uv77 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7c8c054370 con 0x5598c11387d0 2024-09-21T03:41:43.787 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-09-21T03:41:43.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.785+0000 7f7c99d1df40 1 -- 172.21.15.38:0/1459949514 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x5598c1137ab0 con 0x5598c11387d0 2024-09-21T03:41:43.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.786+0000 7f7c93594640 1 -- 172.21.15.38:0/1459949514 <== osd.3 v2:172.21.15.38:6808/1286810294 23 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v109'78 uv78 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7c8c054370 con 0x5598c11387d0 2024-09-21T03:41:43.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.786+0000 7f7c99d1df40 1 -- 172.21.15.38:0/1459949514 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:109 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x5598c1137a90 con 0x5598c11387d0 2024-09-21T03:41:43.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.788+0000 7f7c93594640 1 -- 172.21.15.38:0/1459949514 <== osd.3 v2:172.21.15.38:6808/1286810294 24 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v109'79 uv79 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7c8c054370 con 0x5598c11387d0 2024-09-21T03:41:43.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.788+0000 7f7c99d1df40 1 -- 172.21.15.38:0/1459949514 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:110 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x5598c1137a90 con 0x5598c11387d0 2024-09-21T03:41:43.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.789+0000 7f7c93594640 1 -- 172.21.15.38:0/1459949514 <== osd.3 v2:172.21.15.38:6808/1286810294 25 ==== osd_op_reply(110 609.00000000 [omap-set-header] v109'80 uv80 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7c8c054370 con 0x5598c11387d0 2024-09-21T03:41:43.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.789+0000 7f7c99d1df40 1 --2- 172.21.15.38:0/1459949514 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x5598c11405e0 0x5598c1142d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:43.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.789+0000 7f7c99d1df40 1 -- 172.21.15.38:0/1459949514 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x5598c1143250 con 0x5598c11405e0 2024-09-21T03:41:43.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.789+0000 7f7c92592640 1 --2- 172.21.15.38:0/1459949514 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x5598c11405e0 0x5598c1142d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:43.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.789+0000 7f7c92592640 1 --2- 172.21.15.38:0/1459949514 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x5598c11405e0 0x5598c1142d10 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:43.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.790+0000 7f7c92592640 1 -- 172.21.15.38:0/1459949514 <== osd.2 v2:172.21.15.38:6824/2577858902 1 ==== osd_op_reply(111 400.00000000 [create] v109'64 uv63 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7c8800f750 con 0x5598c11405e0 2024-09-21T03:41:43.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.790+0000 7f7c99d1df40 1 -- 172.21.15.38:0/1459949514 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e109) -- 0x5598c1143250 con 0x5598c11405e0 2024-09-21T03:41:43.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.792+0000 7f7c92592640 1 -- 172.21.15.38:0/1459949514 <== osd.2 v2:172.21.15.38:6824/2577858902 2 ==== osd_op_reply(112 400.00000000 [delete] v109'65 uv65 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7c8800f750 con 0x5598c11405e0 2024-09-21T03:41:43.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.792+0000 7f7c99d1df40 1 -- 172.21.15.38:0/1459949514 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x5598c1143250 con 0x5598c11405e0 2024-09-21T03:41:43.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.793+0000 7f7c92592640 1 -- 172.21.15.38:0/1459949514 <== osd.2 v2:172.21.15.38:6824/2577858902 3 ==== osd_op_reply(113 400.00000000 [create] v109'66 uv66 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7c8800f750 con 0x5598c11405e0 2024-09-21T03:41:43.794 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-09-21T03:41:43.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.793+0000 7f7c99d1df40 1 -- 172.21.15.38:0/1459949514 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:114 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x5598c1143250 con 0x5598c11405e0 2024-09-21T03:41:43.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.794+0000 7f7c92592640 1 -- 172.21.15.38:0/1459949514 <== osd.2 v2:172.21.15.38:6824/2577858902 4 ==== osd_op_reply(114 400.00000000 [write 0~22] v109'67 uv67 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7c8800f750 con 0x5598c11405e0 2024-09-21T03:41:43.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.794+0000 7f7c99d1df40 1 -- 172.21.15.38:0/1459949514 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:115 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x5598c1143250 con 0x5598c11405e0 2024-09-21T03:41:43.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.795+0000 7f7c92592640 1 -- 172.21.15.38:0/1459949514 <== osd.2 v2:172.21.15.38:6824/2577858902 5 ==== osd_op_reply(115 400.00000000 [omap-set-header] v109'68 uv68 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7c8800f750 con 0x5598c11405e0 2024-09-21T03:41:43.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.795+0000 7f7c99d1df40 1 -- 172.21.15.38:0/1459949514 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:116 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x5598c1143250 con 0x5598c113c980 2024-09-21T03:41:43.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.796+0000 7f7c92d93640 1 -- 172.21.15.38:0/1459949514 <== osd.1 v2:172.21.15.38:6816/1491818732 17 ==== osd_op_reply(116 100.00000000.inode [create] v109'157 uv148 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7f7c8405e710 con 0x5598c113c980 2024-09-21T03:41:43.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.796+0000 7f7c99d1df40 1 -- 172.21.15.38:0/1459949514 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:117 13.1f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e109) -- 0x5598c1143250 con 0x5598c113c980 2024-09-21T03:41:43.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.797+0000 7f7c92d93640 1 -- 172.21.15.38:0/1459949514 <== osd.1 v2:172.21.15.38:6816/1491818732 18 ==== osd_op_reply(117 100.00000000.inode [delete] v109'158 uv158 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f7c8405e710 con 0x5598c113c980 2024-09-21T03:41:43.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.797+0000 7f7c99d1df40 1 -- 172.21.15.38:0/1459949514 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:118 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x5598c1143250 con 0x5598c113c980 2024-09-21T03:41:43.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.798+0000 7f7c92d93640 1 -- 172.21.15.38:0/1459949514 <== osd.1 v2:172.21.15.38:6816/1491818732 19 ==== osd_op_reply(118 100.00000000.inode [create] v109'159 uv159 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f7c8405e710 con 0x5598c113c980 2024-09-21T03:41:43.800 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-09-21T03:41:43.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.798+0000 7f7c99d1df40 1 -- 172.21.15.38:0/1459949514 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:119 13.1f 13:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x5598c1143250 con 0x5598c113c980 2024-09-21T03:41:43.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.800+0000 7f7c92d93640 1 -- 172.21.15.38:0/1459949514 <== osd.1 v2:172.21.15.38:6816/1491818732 20 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v109'160 uv160 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f7c8405e710 con 0x5598c113c980 2024-09-21T03:41:43.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.800+0000 7f7c99d1df40 1 -- 172.21.15.38:0/1459949514 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:120 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x5598c1143250 con 0x5598c113c980 2024-09-21T03:41:43.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.801+0000 7f7c92d93640 1 -- 172.21.15.38:0/1459949514 <== osd.1 v2:172.21.15.38:6816/1491818732 21 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v109'161 uv161 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f7c8405e710 con 0x5598c113c980 2024-09-21T03:41:43.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.801+0000 7f7c99d1df40 1 -- 172.21.15.38:0/1459949514 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:121 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x5598c1143250 con 0x5598c113c980 2024-09-21T03:41:43.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.802+0000 7f7c92d93640 1 -- 172.21.15.38:0/1459949514 <== osd.1 v2:172.21.15.38:6816/1491818732 22 ==== osd_op_reply(121 1.00000000 [create] v109'162 uv155 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7f7c8405e710 con 0x5598c113c980 2024-09-21T03:41:43.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.802+0000 7f7c99d1df40 1 -- 172.21.15.38:0/1459949514 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:122 13.1f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e109) -- 0x5598c1143250 con 0x5598c113c980 2024-09-21T03:41:43.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.803+0000 7f7c92d93640 1 -- 172.21.15.38:0/1459949514 <== osd.1 v2:172.21.15.38:6816/1491818732 23 ==== osd_op_reply(122 1.00000000 [delete] v109'163 uv163 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f7c8405e710 con 0x5598c113c980 2024-09-21T03:41:43.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.803+0000 7f7c99d1df40 1 -- 172.21.15.38:0/1459949514 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:123 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x5598c1143250 con 0x5598c113c980 2024-09-21T03:41:43.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.804+0000 7f7c92d93640 1 -- 172.21.15.38:0/1459949514 <== osd.1 v2:172.21.15.38:6816/1491818732 24 ==== osd_op_reply(123 1.00000000 [create] v109'164 uv164 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f7c8405e710 con 0x5598c113c980 2024-09-21T03:41:43.806 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-09-21T03:41:43.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.804+0000 7f7c99d1df40 1 -- 172.21.15.38:0/1459949514 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:124 13.1f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x5598c1144260 con 0x5598c113c980 2024-09-21T03:41:43.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.805+0000 7f7c92d93640 1 -- 172.21.15.38:0/1459949514 <== osd.1 v2:172.21.15.38:6816/1491818732 25 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v109'165 uv165 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f7c8405e710 con 0x5598c113c980 2024-09-21T03:41:43.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.805+0000 7f7c99d1df40 1 -- 172.21.15.38:0/1459949514 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:125 13.1f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x5598c1144260 con 0x5598c113c980 2024-09-21T03:41:43.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.806+0000 7f7c92d93640 1 -- 172.21.15.38:0/1459949514 <== osd.1 v2:172.21.15.38:6816/1491818732 26 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v109'166 uv166 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f7c8405e710 con 0x5598c113c980 2024-09-21T03:41:43.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.806+0000 7f7c99d1df40 1 -- 172.21.15.38:0/1459949514 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:126 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x5598c1143250 con 0x5598c113c980 2024-09-21T03:41:43.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.807+0000 7f7c92d93640 1 -- 172.21.15.38:0/1459949514 <== osd.1 v2:172.21.15.38:6816/1491818732 27 ==== osd_op_reply(126 1.00000000 [omap-set-header] v109'167 uv167 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f7c8405e710 con 0x5598c113c980 2024-09-21T03:41:43.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.807+0000 7f7c99d1df40 1 -- 172.21.15.38:0/1459949514 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:127 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=584b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x5598c1143250 con 0x5598c113c980 2024-09-21T03:41:43.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.808+0000 7f7c92d93640 1 -- 172.21.15.38:0/1459949514 <== osd.1 v2:172.21.15.38:6816/1491818732 28 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v109'168 uv168 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f7c8405e710 con 0x5598c113c980 2024-09-21T03:41:43.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.808+0000 7f7c99d1df40 1 -- 172.21.15.38:0/1459949514 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x5598c1134450 msgr2=0x5598c1136870 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:43.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.808+0000 7f7c99d1df40 1 --2- 172.21.15.38:0/1459949514 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x5598c1134450 0x5598c1136870 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:43.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.808+0000 7f7c99d1df40 1 -- 172.21.15.38:0/1459949514 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x5598c113c980 msgr2=0x5598c113ee00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:43.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.808+0000 7f7c99d1df40 1 --2- 172.21.15.38:0/1459949514 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x5598c113c980 0x5598c113ee00 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:43.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.808+0000 7f7c99d1df40 1 -- 172.21.15.38:0/1459949514 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x5598c11405e0 msgr2=0x5598c1142d10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:43.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.808+0000 7f7c99d1df40 1 --2- 172.21.15.38:0/1459949514 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x5598c11405e0 0x5598c1142d10 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:43.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.808+0000 7f7c99d1df40 1 -- 172.21.15.38:0/1459949514 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x5598c11387d0 msgr2=0x5598c113b630 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:43.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.808+0000 7f7c99d1df40 1 --2- 172.21.15.38:0/1459949514 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x5598c11387d0 0x5598c113b630 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:43.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.808+0000 7f7c93594640 1 -- 172.21.15.38:0/1459949514 reap_dead start 2024-09-21T03:41:43.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.808+0000 7f7c99d1df40 1 -- 172.21.15.38:0/1459949514 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x5598c112ec30 msgr2=0x5598c1131830 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:43.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.808+0000 7f7c99d1df40 1 --2- 172.21.15.38:0/1459949514 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x5598c112ec30 0x5598c1131830 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:43.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.808+0000 7f7c93594640 1 -- 172.21.15.38:0/1459949514 reap_dead start 2024-09-21T03:41:43.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.808+0000 7f7c99d1df40 1 -- 172.21.15.38:0/1459949514 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x5598c106eb70 msgr2=0x5598c106ef80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:43.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.808+0000 7f7c99d1df40 1 --2- 172.21.15.38:0/1459949514 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x5598c106eb70 0x5598c106ef80 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:43.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.808+0000 7f7c99d1df40 1 -- 172.21.15.38:0/1459949514 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7c800775a0 msgr2=0x7f7c80079ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:43.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.808+0000 7f7c99d1df40 1 --2- 172.21.15.38:0/1459949514 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7c800775a0 0x7f7c80079ab0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x5598c0ffd2f0 tx=0x7f7c88000ed0 comp rx=0 tx=0).stop 2024-09-21T03:41:43.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.808+0000 7f7c99d1df40 1 -- 172.21.15.38:0/1459949514 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5598c0ffe090 msgr2=0x5598c106bff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:43.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.808+0000 7f7c99d1df40 1 --2- 172.21.15.38:0/1459949514 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5598c0ffe090 0x5598c106bff0 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7f7c8400b690 tx=0x7f7c8400bb50 comp rx=0 tx=0).stop 2024-09-21T03:41:43.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.808+0000 7f7c99d1df40 1 -- 172.21.15.38:0/1459949514 shutdown_connections 2024-09-21T03:41:43.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.808+0000 7f7c99d1df40 1 --2- 172.21.15.38:0/1459949514 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x5598c106eb70 0x5598c106ef80 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:43.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.808+0000 7f7c99d1df40 1 --2- 172.21.15.38:0/1459949514 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7c800775a0 0x7f7c80079ab0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:43.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.808+0000 7f7c99d1df40 1 --2- 172.21.15.38:0/1459949514 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5598c0ffe090 0x5598c106bff0 unknown :-1 s=CLOSED pgs=1542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:43.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.808+0000 7f7c99d1df40 1 -- 172.21.15.38:0/1459949514 >> 172.21.15.38:0/1459949514 conn(0x5598c0ed5ac0 msgr2=0x5598c0ffc7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:43.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.809+0000 7f7c99d1df40 1 -- 172.21.15.38:0/1459949514 shutdown_connections 2024-09-21T03:41:43.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.809+0000 7f7c99d1df40 1 -- 172.21.15.38:0/1459949514 wait complete. 2024-09-21T03:41:43.814 DEBUG:teuthology.orchestra.run.smithi038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rm 100.00000000.inode 2024-09-21T03:41:43.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.867+0000 7fb6b66fef40 1 Processor -- start 2024-09-21T03:41:43.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.867+0000 7fb6b66fef40 1 -- start start 2024-09-21T03:41:43.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.868+0000 7fb6b66fef40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5598b467dd00 0x5598b4733d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:43.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.868+0000 7fb6b66fef40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5598b46a9e30 0x5598b4732ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:43.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.868+0000 7fb6b66fef40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5598b4738950 0x5598b4679af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:43.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.868+0000 7fb6b66fef40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5598b45cbd60 con 0x5598b4738950 2024-09-21T03:41:43.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.868+0000 7fb6b66fef40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5598b462a430 con 0x5598b467dd00 2024-09-21T03:41:43.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.868+0000 7fb6b66fef40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x5598b46c6460 con 0x5598b46a9e30 2024-09-21T03:41:43.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.868+0000 7fb6b09b9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5598b4738950 0x5598b4679af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:43.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.868+0000 7fb6b09b9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5598b4738950 0x5598b4679af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54462/0 (socket says 172.21.15.38:54462) 2024-09-21T03:41:43.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.868+0000 7fb6b09b9640 1 -- 172.21.15.38:0/3253912899 learned_addr learned my addr 172.21.15.38:0/3253912899 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:41:43.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.868+0000 7fb6b01b8640 1 --2- 172.21.15.38:0/3253912899 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5598b467dd00 0x5598b4733d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:43.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.868+0000 7fb6af9b7640 1 --2- 172.21.15.38:0/3253912899 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5598b46a9e30 0x5598b4732ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:43.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.868+0000 7fb6b09b9640 1 -- 172.21.15.38:0/3253912899 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5598b46a9e30 msgr2=0x5598b4732ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:43.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.868+0000 7fb6b09b9640 1 --2- 172.21.15.38:0/3253912899 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5598b46a9e30 0x5598b4732ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:43.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.868+0000 7fb6b09b9640 1 -- 172.21.15.38:0/3253912899 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5598b467dd00 msgr2=0x5598b4733d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:43.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.868+0000 7fb6b09b9640 1 --2- 172.21.15.38:0/3253912899 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5598b467dd00 0x5598b4733d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:43.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.868+0000 7fb6b09b9640 1 -- 172.21.15.38:0/3253912899 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5598b4731e80 con 0x5598b4738950 2024-09-21T03:41:43.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.869+0000 7fb6b09b9640 1 --2- 172.21.15.38:0/3253912899 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5598b4738950 0x5598b4679af0 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7fb6a8004fd0 tx=0x7fb6a800b9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56cd684133bb0712 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:43.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.869+0000 7fb6ae9b5640 1 -- 172.21.15.38:0/3253912899 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6a800a7c0 con 0x5598b4738950 2024-09-21T03:41:43.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.869+0000 7fb6ae9b5640 1 -- 172.21.15.38:0/3253912899 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb6a800a960 con 0x5598b4738950 2024-09-21T03:41:43.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.869+0000 7fb6ae9b5640 1 -- 172.21.15.38:0/3253912899 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6a800adb0 con 0x5598b4738950 2024-09-21T03:41:43.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.869+0000 7fb6b66fef40 1 -- 172.21.15.38:0/3253912899 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5598b4738950 msgr2=0x5598b4679af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:43.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.869+0000 7fb6b66fef40 1 --2- 172.21.15.38:0/3253912899 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5598b4738950 0x5598b4679af0 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7fb6a8004fd0 tx=0x7fb6a800b9d0 comp rx=0 tx=0).stop 2024-09-21T03:41:43.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.869+0000 7fb6b66fef40 1 -- 172.21.15.38:0/3253912899 shutdown_connections 2024-09-21T03:41:43.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.869+0000 7fb6b66fef40 1 --2- 172.21.15.38:0/3253912899 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5598b4738950 0x5598b4679af0 unknown :-1 s=CLOSED pgs=1545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:43.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.869+0000 7fb6b66fef40 1 --2- 172.21.15.38:0/3253912899 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5598b46a9e30 0x5598b4732ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:43.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.869+0000 7fb6b66fef40 1 --2- 172.21.15.38:0/3253912899 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5598b467dd00 0x5598b4733d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:43.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.869+0000 7fb6b66fef40 1 -- 172.21.15.38:0/3253912899 >> 172.21.15.38:0/3253912899 conn(0x5598b459cac0 msgr2=0x5598b4753350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:43.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.869+0000 7fb6b66fef40 1 -- 172.21.15.38:0/3253912899 shutdown_connections 2024-09-21T03:41:43.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.870+0000 7fb6b66fef40 1 -- 172.21.15.38:0/3253912899 wait complete. 2024-09-21T03:41:43.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.870+0000 7fb6b66fef40 1 Processor -- start 2024-09-21T03:41:43.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.870+0000 7fb6b66fef40 1 -- start start 2024-09-21T03:41:43.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.872+0000 7fb6b66fef40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5598b467dd00 0x5598b472e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:43.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.872+0000 7fb6b66fef40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5598b46a9e30 0x5598b4737400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:43.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.872+0000 7fb6b66fef40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5598b4738950 0x5598b4737940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:43.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.872+0000 7fb6b66fef40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5598b467a4d0 con 0x5598b4738950 2024-09-21T03:41:43.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.872+0000 7fb6b66fef40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5598b4751fb0 con 0x5598b46a9e30 2024-09-21T03:41:43.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.872+0000 7fb6b66fef40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x5598b4753700 con 0x5598b467dd00 2024-09-21T03:41:43.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.872+0000 7fb6b09b9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5598b4738950 0x5598b4737940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:43.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.872+0000 7fb6b09b9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5598b4738950 0x5598b4737940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54470/0 (socket says 172.21.15.38:54470) 2024-09-21T03:41:43.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.872+0000 7fb6b09b9640 1 -- 172.21.15.38:0/331211465 learned_addr learned my addr 172.21.15.38:0/331211465 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:41:43.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.872+0000 7fb6af9b7640 1 --2- 172.21.15.38:0/331211465 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5598b46a9e30 0x5598b4737400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:43.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.872+0000 7fb6b01b8640 1 --2- 172.21.15.38:0/331211465 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5598b467dd00 0x5598b472e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:43.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.872+0000 7fb6b09b9640 1 -- 172.21.15.38:0/331211465 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5598b467dd00 msgr2=0x5598b472e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:43.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.872+0000 7fb6b09b9640 1 --2- 172.21.15.38:0/331211465 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5598b467dd00 0x5598b472e1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:43.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.872+0000 7fb6b09b9640 1 -- 172.21.15.38:0/331211465 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5598b46a9e30 msgr2=0x5598b4737400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:43.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.872+0000 7fb6b09b9640 1 --2- 172.21.15.38:0/331211465 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5598b46a9e30 0x5598b4737400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:43.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.872+0000 7fb6b09b9640 1 -- 172.21.15.38:0/331211465 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5598b472e710 con 0x5598b4738950 2024-09-21T03:41:43.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.872+0000 7fb6b09b9640 1 --2- 172.21.15.38:0/331211465 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5598b4738950 0x5598b4737940 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7fb6a80176b0 tx=0x7fb6a80176e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:43.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.872+0000 7fb6ad9b3640 1 -- 172.21.15.38:0/331211465 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6a8017110 con 0x5598b4738950 2024-09-21T03:41:43.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.872+0000 7fb6ad9b3640 1 -- 172.21.15.38:0/331211465 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb6a80172b0 con 0x5598b4738950 2024-09-21T03:41:43.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.872+0000 7fb6ad9b3640 1 -- 172.21.15.38:0/331211465 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6a8017800 con 0x5598b4738950 2024-09-21T03:41:43.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.873+0000 7fb6b66fef40 1 -- 172.21.15.38:0/331211465 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5598b4734570 con 0x5598b4738950 2024-09-21T03:41:43.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.873+0000 7fb6b66fef40 1 -- 172.21.15.38:0/331211465 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5598b4730e30 con 0x5598b4738950 2024-09-21T03:41:43.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.874+0000 7fb6ad9b3640 1 -- 172.21.15.38:0/331211465 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb6a80179a0 con 0x5598b4738950 2024-09-21T03:41:43.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.875+0000 7fb6ad9b3640 1 --2- 172.21.15.38:0/331211465 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb69c077570 0x7fb69c079a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:43.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.875+0000 7fb6ad9b3640 1 -- 172.21.15.38:0/331211465 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 7422+0+0 (secure 0 0 0) 0x7fb6a809b3c0 con 0x5598b4738950 2024-09-21T03:41:43.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.875+0000 7fb6b66fef40 1 --2- 172.21.15.38:0/331211465 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x5598b4751630 0x5598b472fa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:43.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.875+0000 7fb6b66fef40 1 -- 172.21.15.38:0/331211465 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:1 13.1f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e109) -- 0x5598b46ac720 con 0x5598b4751630 2024-09-21T03:41:43.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.875+0000 7fb6b01b8640 1 --2- 172.21.15.38:0/331211465 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb69c077570 0x7fb69c079a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:43.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.875+0000 7fb6af9b7640 1 --2- 172.21.15.38:0/331211465 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x5598b4751630 0x5598b472fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:43.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.876+0000 7fb6af9b7640 1 --2- 172.21.15.38:0/331211465 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x5598b4751630 0x5598b472fa40 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:43.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.876+0000 7fb6b01b8640 1 --2- 172.21.15.38:0/331211465 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb69c077570 0x7fb69c079a80 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fb6a0000f80 tx=0x7fb6a0006a20 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:43.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.877+0000 7fb6af9b7640 1 -- 172.21.15.38:0/331211465 <== osd.1 v2:172.21.15.38:6816/1491818732 1 ==== osd_op_reply(1 100.00000000.inode [delete] v109'169 uv169 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fb6a404ec60 con 0x5598b4751630 2024-09-21T03:41:43.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.877+0000 7fb6b66fef40 1 -- 172.21.15.38:0/331211465 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x5598b4751630 msgr2=0x5598b472fa40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:43.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.877+0000 7fb6b66fef40 1 --2- 172.21.15.38:0/331211465 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x5598b4751630 0x5598b472fa40 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:43.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.877+0000 7fb6b66fef40 1 -- 172.21.15.38:0/331211465 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb69c077570 msgr2=0x7fb69c079a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:43.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.877+0000 7fb6b66fef40 1 --2- 172.21.15.38:0/331211465 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb69c077570 0x7fb69c079a80 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fb6a0000f80 tx=0x7fb6a0006a20 comp rx=0 tx=0).stop 2024-09-21T03:41:43.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.877+0000 7fb6b66fef40 1 -- 172.21.15.38:0/331211465 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5598b4738950 msgr2=0x5598b4737940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:43.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.877+0000 7fb6b66fef40 1 --2- 172.21.15.38:0/331211465 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5598b4738950 0x5598b4737940 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7fb6a80176b0 tx=0x7fb6a80176e0 comp rx=0 tx=0).stop 2024-09-21T03:41:43.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.877+0000 7fb6b09b9640 1 -- 172.21.15.38:0/331211465 reap_dead start 2024-09-21T03:41:43.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.877+0000 7fb6b66fef40 1 -- 172.21.15.38:0/331211465 shutdown_connections 2024-09-21T03:41:43.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.877+0000 7fb6b66fef40 1 -- 172.21.15.38:0/331211465 >> 172.21.15.38:0/331211465 conn(0x5598b459cac0 msgr2=0x5598b4679af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:43.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.877+0000 7fb6b66fef40 1 -- 172.21.15.38:0/331211465 shutdown_connections 2024-09-21T03:41:43.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.877+0000 7fb6b66fef40 1 -- 172.21.15.38:0/331211465 wait complete. 2024-09-21T03:41:43.882 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-21T03:41:44.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.999+0000 7f93ecfad640 1 -- 172.21.15.38:0/3064196428 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f93e815c740 msgr2=0x7f93e80b3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:44.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.999+0000 7f93ecfad640 1 --2- 172.21.15.38:0/3064196428 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f93e815c740 0x7f93e80b3740 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f93e0005fd0 tx=0x7f93e002a8b0 comp rx=0 tx=0).stop 2024-09-21T03:41:44.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.999+0000 7f93ecfad640 1 -- 172.21.15.38:0/3064196428 shutdown_connections 2024-09-21T03:41:44.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.999+0000 7f93ecfad640 1 --2- 172.21.15.38:0/3064196428 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f93e80740f0 0x7f93e8074500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:44.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.999+0000 7f93ecfad640 1 --2- 172.21.15.38:0/3064196428 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f93e815c740 0x7f93e80b3740 unknown :-1 s=CLOSED pgs=1548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:44.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.999+0000 7f93ecfad640 1 -- 172.21.15.38:0/3064196428 >> 172.21.15.38:0/3064196428 conn(0x7f93e80735d0 msgr2=0x7f93e80739e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:44.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.999+0000 7f93ecfad640 1 -- 172.21.15.38:0/3064196428 shutdown_connections 2024-09-21T03:41:44.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:43.999+0000 7f93ecfad640 1 -- 172.21.15.38:0/3064196428 wait complete. 2024-09-21T03:41:44.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:44.001+0000 7f93ecfad640 1 Processor -- start 2024-09-21T03:41:44.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:44.001+0000 7f93ecfad640 1 -- start start 2024-09-21T03:41:44.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:44.003+0000 7f93ecfad640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f93e806ddf0 0x7f93e8151fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:44.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:44.003+0000 7f93ecfad640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f93e80740f0 0x7f93e81e8c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:44.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:44.003+0000 7f93ecfad640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f93e815c740 0x7f93e8156db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:44.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:44.003+0000 7f93ecfad640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f93e80cfb10 con 0x7f93e815c740 2024-09-21T03:41:44.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:44.003+0000 7f93ecfad640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f93e80cf990 con 0x7f93e80740f0 2024-09-21T03:41:44.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:44.003+0000 7f93ecfad640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f93e80cfc90 con 0x7f93e806ddf0 2024-09-21T03:41:44.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:44.003+0000 7f93e6d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f93e815c740 0x7f93e8156db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:44.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:44.003+0000 7f93e6d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f93e815c740 0x7f93e8156db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54486/0 (socket says 172.21.15.38:54486) 2024-09-21T03:41:44.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:44.003+0000 7f93e659c640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f93e806ddf0 0x7f93e8151fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:44.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:44.003+0000 7f93e6d9d640 1 -- 172.21.15.38:0/3681169237 learned_addr learned my addr 172.21.15.38:0/3681169237 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:41:44.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:44.003+0000 7f93e5d9b640 1 --2- 172.21.15.38:0/3681169237 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f93e80740f0 0x7f93e81e8c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:44.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:44.003+0000 7f93e6d9d640 1 -- 172.21.15.38:0/3681169237 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f93e806ddf0 msgr2=0x7f93e8151fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:44.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:44.003+0000 7f93e6d9d640 1 --2- 172.21.15.38:0/3681169237 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f93e806ddf0 0x7f93e8151fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:44.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:44.003+0000 7f93e6d9d640 1 -- 172.21.15.38:0/3681169237 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f93e80740f0 msgr2=0x7f93e81e8c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:44.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:44.003+0000 7f93e6d9d640 1 --2- 172.21.15.38:0/3681169237 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f93e80740f0 0x7f93e81e8c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:44.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:44.003+0000 7f93e6d9d640 1 -- 172.21.15.38:0/3681169237 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93e81f6ee0 con 0x7f93e815c740 2024-09-21T03:41:44.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:44.003+0000 7f93e659c640 1 --2- 172.21.15.38:0/3681169237 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f93e806ddf0 0x7f93e8151fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:41:44.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:44.004+0000 7f93e6d9d640 1 --2- 172.21.15.38:0/3681169237 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f93e815c740 0x7f93e8156db0 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f93d800c340 tx=0x7f93d800c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:44.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:44.004+0000 7f93deffd640 1 -- 172.21.15.38:0/3681169237 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93d8017070 con 0x7f93e815c740 2024-09-21T03:41:44.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:44.004+0000 7f93deffd640 1 -- 172.21.15.38:0/3681169237 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f93d80035f0 con 0x7f93e815c740 2024-09-21T03:41:44.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:44.004+0000 7f93ecfad640 1 -- 172.21.15.38:0/3681169237 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93e81f71c0 con 0x7f93e815c740 2024-09-21T03:41:44.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:44.004+0000 7f93deffd640 1 -- 172.21.15.38:0/3681169237 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93d8003a80 con 0x7f93e815c740 2024-09-21T03:41:44.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:44.004+0000 7f93ecfad640 1 -- 172.21.15.38:0/3681169237 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93e81f7780 con 0x7f93e815c740 2024-09-21T03:41:44.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:44.005+0000 7f93ecfad640 1 -- 172.21.15.38:0/3681169237 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93e81f6c50 con 0x7f93e815c740 2024-09-21T03:41:44.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:44.006+0000 7f93deffd640 1 -- 172.21.15.38:0/3681169237 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f93d801a070 con 0x7f93e815c740 2024-09-21T03:41:44.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:44.006+0000 7f93deffd640 1 --2- 172.21.15.38:0/3681169237 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f93c8078bc0 0x7f93c807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:44.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:44.006+0000 7f93deffd640 1 -- 172.21.15.38:0/3681169237 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 7422+0+0 (secure 0 0 0) 0x7f93d809a420 con 0x7f93e815c740 2024-09-21T03:41:44.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:44.006+0000 7f93e659c640 1 --2- 172.21.15.38:0/3681169237 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f93c8078bc0 0x7f93c807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:44.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:44.007+0000 7f93e659c640 1 --2- 172.21.15.38:0/3681169237 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f93c8078bc0 0x7f93c807b0d0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f93e0034000 tx=0x7f93e0004ab0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:44.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:44.011+0000 7f93deffd640 1 -- 172.21.15.38:0/3681169237 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f93d8061c80 con 0x7f93e815c740 2024-09-21T03:41:44.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:44.239+0000 7f93ecfad640 1 -- 172.21.15.38:0/3681169237 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f93e80d0140 con 0x7f93e815c740 2024-09-21T03:41:44.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:44.717+0000 7f93deffd640 1 -- 172.21.15.38:0/3681169237 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v164) ==== 162+0+0 (secure 0 0 0) 0x7f93d80666b0 con 0x7f93e815c740 2024-09-21T03:41:44.719 INFO:teuthology.orchestra.run.smithi038.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-21T03:41:44.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:44.721+0000 7f93ecfad640 1 -- 172.21.15.38:0/3681169237 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f93c8078bc0 msgr2=0x7f93c807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:44.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:44.721+0000 7f93ecfad640 1 --2- 172.21.15.38:0/3681169237 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f93c8078bc0 0x7f93c807b0d0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f93e0034000 tx=0x7f93e0004ab0 comp rx=0 tx=0).stop 2024-09-21T03:41:44.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:44.721+0000 7f93ecfad640 1 -- 172.21.15.38:0/3681169237 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f93e815c740 msgr2=0x7f93e8156db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:44.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:44.721+0000 7f93ecfad640 1 --2- 172.21.15.38:0/3681169237 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f93e815c740 0x7f93e8156db0 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f93d800c340 tx=0x7f93d800c800 comp rx=0 tx=0).stop 2024-09-21T03:41:44.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:44.721+0000 7f93ecfad640 1 -- 172.21.15.38:0/3681169237 shutdown_connections 2024-09-21T03:41:44.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:44.721+0000 7f93ecfad640 1 --2- 172.21.15.38:0/3681169237 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f93e815c740 0x7f93e8156db0 unknown :-1 s=CLOSED pgs=1549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:44.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:44.721+0000 7f93ecfad640 1 --2- 172.21.15.38:0/3681169237 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f93e80740f0 0x7f93e81e8c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:44.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:44.721+0000 7f93ecfad640 1 --2- 172.21.15.38:0/3681169237 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f93c8078bc0 0x7f93c807b0d0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:44.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:44.721+0000 7f93ecfad640 1 --2- 172.21.15.38:0/3681169237 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f93e806ddf0 0x7f93e8151fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:44.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:44.721+0000 7f93ecfad640 1 -- 172.21.15.38:0/3681169237 >> 172.21.15.38:0/3681169237 conn(0x7f93e80735d0 msgr2=0x7f93e80b46f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:44.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:44.721+0000 7f93ecfad640 1 -- 172.21.15.38:0/3681169237 shutdown_connections 2024-09-21T03:41:44.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:44.721+0000 7f93ecfad640 1 -- 172.21.15.38:0/3681169237 wait complete. 2024-09-21T03:41:44.737 INFO:tasks.ceph.mds.a.smithi038.stderr:2024-09-21T03:41:44.735+0000 7fe6e6b6f640 -1 mds.0.cache.ino(0x100) No data while reading inode 0x100 2024-09-21T03:41:44.737 INFO:tasks.ceph.mds.a.smithi038.stderr:2024-09-21T03:41:44.736+0000 7fe6e6b6f640 -1 log_channel(cluster) log [ERR] : Error loading MDS rank 0: (2) No such file or directory 2024-09-21T03:41:44.739 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:41:44.739 INFO:tasks.ceph.mon.a.smithi038.stderr:2024-09-21T03:41:44.736+0000 7f3f45713640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T03:41:44.739 INFO:tasks.ceph.mon.a.smithi038.stderr:2024-09-21T03:41:44.736+0000 7f3f45713640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-09-21T03:41:44.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:44.857+0000 7f5ffccb6640 1 -- 172.21.15.38:0/3695994381 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5ff8073e20 msgr2=0x7f5ff8074230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:44.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:44.857+0000 7f5ffccb6640 1 --2- 172.21.15.38:0/3695994381 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5ff8073e20 0x7f5ff8074230 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7f5ff80ca5a0 tx=0x7f5fec02a5f0 comp rx=0 tx=0).stop 2024-09-21T03:41:44.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:44.857+0000 7f5ffccb6640 1 -- 172.21.15.38:0/3695994381 shutdown_connections 2024-09-21T03:41:44.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:44.857+0000 7f5ffccb6640 1 --2- 172.21.15.38:0/3695994381 >> v2:172.21.15.195:3300/0 conn(0x7f5ff815e800 0x7f5ff815ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:44.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:44.857+0000 7f5ffccb6640 1 --2- 172.21.15.38:0/3695994381 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5ff8073e20 0x7f5ff8074230 unknown :-1 s=CLOSED pgs=1551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:44.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:44.857+0000 7f5ffccb6640 1 -- 172.21.15.38:0/3695994381 >> 172.21.15.38:0/3695994381 conn(0x7f5ff80cca40 msgr2=0x7f5ff80cce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:44.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:44.857+0000 7f5ffccb6640 1 -- 172.21.15.38:0/3695994381 shutdown_connections 2024-09-21T03:41:44.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:44.857+0000 7f5ffccb6640 1 -- 172.21.15.38:0/3695994381 wait complete. 2024-09-21T03:41:44.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:44.859+0000 7f5ffccb6640 1 Processor -- start 2024-09-21T03:41:44.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:44.859+0000 7f5ffccb6640 1 -- start start 2024-09-21T03:41:44.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:44.860+0000 7f5ffccb6640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5ff8073e20 0x7f5ff81f4700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:44.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:44.860+0000 7f5ffccb6640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5ff80ca610 0x7f5ff81edd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:44.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:44.860+0000 7f5ffccb6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5ff815e800 0x7f5ff815b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:44.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:44.860+0000 7f5ffccb6640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f5ff815f2d0 con 0x7f5ff815e800 2024-09-21T03:41:44.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:44.860+0000 7f5ffccb6640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f5ff815f150 con 0x7f5ff8073e20 2024-09-21T03:41:44.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:44.860+0000 7f5ffccb6640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f5ff815f450 con 0x7f5ff80ca610 2024-09-21T03:41:44.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:44.860+0000 7f5ff6d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5ff815e800 0x7f5ff815b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:44.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:44.860+0000 7f5ff6d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5ff815e800 0x7f5ff815b6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54508/0 (socket says 172.21.15.38:54508) 2024-09-21T03:41:44.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:44.860+0000 7f5ff6d9d640 1 -- 172.21.15.38:0/3141905101 learned_addr learned my addr 172.21.15.38:0/3141905101 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:41:44.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:44.861+0000 7f5ff659c640 1 --2- 172.21.15.38:0/3141905101 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5ff8073e20 0x7f5ff81f4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:44.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:44.861+0000 7f5ff5d9b640 1 --2- 172.21.15.38:0/3141905101 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5ff80ca610 0x7f5ff81edd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:44.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:44.861+0000 7f5ff6d9d640 1 -- 172.21.15.38:0/3141905101 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5ff80ca610 msgr2=0x7f5ff81edd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:44.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:44.861+0000 7f5ff6d9d640 1 --2- 172.21.15.38:0/3141905101 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5ff80ca610 0x7f5ff81edd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:44.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:44.861+0000 7f5ff6d9d640 1 -- 172.21.15.38:0/3141905101 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5ff8073e20 msgr2=0x7f5ff81f4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:44.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:44.861+0000 7f5ff6d9d640 1 --2- 172.21.15.38:0/3141905101 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5ff8073e20 0x7f5ff81f4700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:44.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:44.861+0000 7f5ff6d9d640 1 -- 172.21.15.38:0/3141905101 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ff81f4e70 con 0x7f5ff815e800 2024-09-21T03:41:44.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:44.861+0000 7f5ff659c640 1 --2- 172.21.15.38:0/3141905101 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5ff8073e20 0x7f5ff81f4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:41:44.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:44.861+0000 7f5ff6d9d640 1 --2- 172.21.15.38:0/3141905101 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5ff815e800 0x7f5ff815b6f0 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f5ff000e9c0 tx=0x7f5ff000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:44.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:44.861+0000 7f5fe6ffd640 1 -- 172.21.15.38:0/3141905101 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ff0004d20 con 0x7f5ff815e800 2024-09-21T03:41:44.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:44.861+0000 7f5fe6ffd640 1 -- 172.21.15.38:0/3141905101 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5ff000cb20 con 0x7f5ff815e800 2024-09-21T03:41:44.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:44.861+0000 7f5ffccb6640 1 -- 172.21.15.38:0/3141905101 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ff8159730 con 0x7f5ff815e800 2024-09-21T03:41:44.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:44.861+0000 7f5fe6ffd640 1 -- 172.21.15.38:0/3141905101 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ff0005020 con 0x7f5ff815e800 2024-09-21T03:41:44.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:44.861+0000 7f5ffccb6640 1 -- 172.21.15.38:0/3141905101 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ff8159cf0 con 0x7f5ff815e800 2024-09-21T03:41:44.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:44.862+0000 7f5ffccb6640 1 -- 172.21.15.38:0/3141905101 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ff8071760 con 0x7f5ff815e800 2024-09-21T03:41:44.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:44.863+0000 7f5fe6ffd640 1 -- 172.21.15.38:0/3141905101 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f5ff00051c0 con 0x7f5ff815e800 2024-09-21T03:41:44.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:44.863+0000 7f5fe6ffd640 1 --2- 172.21.15.38:0/3141905101 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5fd8078b90 0x7f5fd807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:44.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:44.864+0000 7f5fe6ffd640 1 -- 172.21.15.38:0/3141905101 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 7508+0+0 (secure 0 0 0) 0x7f5ff009be90 con 0x7f5ff815e800 2024-09-21T03:41:44.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:44.864+0000 7f5ff659c640 1 --2- 172.21.15.38:0/3141905101 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5fd8078b90 0x7f5fd807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:44.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:44.864+0000 7f5ff659c640 1 --2- 172.21.15.38:0/3141905101 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5fd8078b90 0x7f5fd807b0a0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f5fe8000f60 tx=0x7f5fe8009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:44.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:44.868+0000 7f5fe6ffd640 1 -- 172.21.15.38:0/3141905101 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f5ff8071760 con 0x7f5ff815e800 2024-09-21T03:41:45.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:45.091+0000 7f5ffccb6640 1 -- 172.21.15.38:0/3141905101 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5ff81f4c80 con 0x7f5ff815e800 2024-09-21T03:41:45.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:45.091+0000 7f5fe6ffd640 1 -- 172.21.15.38:0/3141905101 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 166 v166) ==== 95+0+4392 (secure 0 0 0) 0x7f5ff009d090 con 0x7f5ff815e800 2024-09-21T03:41:45.094 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:41:45.094 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":166,"btime":"2024-09-21T03:41:44:737565+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6086,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/965972663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":965972663},{"type":"v1","addr":"172.21.15.195:6836","nonce":965972663}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6700,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/3176122334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":3176122334},{"type":"v1","addr":"172.21.15.195:6837","nonce":3176122334}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":8505,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/1717955365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":1717955365},{"type":"v1","addr":"172.21.15.38:6833","nonce":1717955365}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":159}],"filesystems":[{"mdsmap":{"epoch":166,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:40:02.319282+0000","modified":"2024-09-21T03:41:44.737494+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":110,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-21T03:41:45.094 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 166 2024-09-21T03:41:45.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:45.093+0000 7f5ffccb6640 1 -- 172.21.15.38:0/3141905101 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5fd8078b90 msgr2=0x7f5fd807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:45.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:45.093+0000 7f5ffccb6640 1 --2- 172.21.15.38:0/3141905101 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5fd8078b90 0x7f5fd807b0a0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f5fe8000f60 tx=0x7f5fe8009040 comp rx=0 tx=0).stop 2024-09-21T03:41:45.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:45.093+0000 7f5ffccb6640 1 -- 172.21.15.38:0/3141905101 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5ff815e800 msgr2=0x7f5ff815b6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:45.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:45.093+0000 7f5ffccb6640 1 --2- 172.21.15.38:0/3141905101 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5ff815e800 0x7f5ff815b6f0 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f5ff000e9c0 tx=0x7f5ff000ee80 comp rx=0 tx=0).stop 2024-09-21T03:41:45.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:45.093+0000 7f5ffccb6640 1 -- 172.21.15.38:0/3141905101 shutdown_connections 2024-09-21T03:41:45.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:45.093+0000 7f5ffccb6640 1 --2- 172.21.15.38:0/3141905101 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5ff815e800 0x7f5ff815b6f0 unknown :-1 s=CLOSED pgs=1552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:45.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:45.093+0000 7f5ffccb6640 1 --2- 172.21.15.38:0/3141905101 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5fd8078b90 0x7f5fd807b0a0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:45.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:45.093+0000 7f5ffccb6640 1 --2- 172.21.15.38:0/3141905101 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5ff80ca610 0x7f5ff81edd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:45.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:45.093+0000 7f5ffccb6640 1 --2- 172.21.15.38:0/3141905101 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5ff8073e20 0x7f5ff81f4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:45.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:45.094+0000 7f5ffccb6640 1 -- 172.21.15.38:0/3141905101 >> 172.21.15.38:0/3141905101 conn(0x7f5ff80cca40 msgr2=0x7f5ff8070380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:45.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:45.094+0000 7f5ffccb6640 1 -- 172.21.15.38:0/3141905101 shutdown_connections 2024-09-21T03:41:45.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:45.094+0000 7f5ffccb6640 1 -- 172.21.15.38:0/3141905101 wait complete. 2024-09-21T03:41:45.107 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-21T03:41:45.107 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:41:45.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:45.222+0000 7fa9e7158640 1 -- 172.21.15.38:0/1402390419 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa9e00b4740 msgr2=0x7fa9e006f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:45.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:45.222+0000 7fa9e7158640 1 --2- 172.21.15.38:0/1402390419 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa9e00b4740 0x7fa9e006f350 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7fa9e00cd2f0 tx=0x7fa9c802a560 comp rx=0 tx=0).stop 2024-09-21T03:41:45.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:45.222+0000 7fa9e7158640 1 -- 172.21.15.38:0/1402390419 shutdown_connections 2024-09-21T03:41:45.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:45.222+0000 7fa9e7158640 1 --2- 172.21.15.38:0/1402390419 >> v2:172.21.15.195:3300/0 conn(0x7fa9e00b4b50 0x7fa9e006f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:45.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:45.222+0000 7fa9e7158640 1 --2- 172.21.15.38:0/1402390419 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa9e00b4740 0x7fa9e006f350 unknown :-1 s=CLOSED pgs=1553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:45.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:45.222+0000 7fa9e7158640 1 --2- 172.21.15.38:0/1402390419 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa9e006e1d0 0x7fa9e00afd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:45.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:45.222+0000 7fa9e7158640 1 -- 172.21.15.38:0/1402390419 >> 172.21.15.38:0/1402390419 conn(0x7fa9e00d0a30 msgr2=0x7fa9e00d0e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:45.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:45.222+0000 7fa9e7158640 1 -- 172.21.15.38:0/1402390419 shutdown_connections 2024-09-21T03:41:45.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:45.222+0000 7fa9e7158640 1 -- 172.21.15.38:0/1402390419 wait complete. 2024-09-21T03:41:45.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:45.224+0000 7fa9e7158640 1 Processor -- start 2024-09-21T03:41:45.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:45.224+0000 7fa9e7158640 1 -- start start 2024-09-21T03:41:45.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:45.225+0000 7fa9e7158640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa9e006e1d0 0x7fa9e014a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:45.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:45.225+0000 7fa9e7158640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa9e00b4740 0x7fa9e0147df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:45.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:45.225+0000 7fa9e7158640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa9e00b4b50 0x7fa9e0158350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:45.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:45.225+0000 7fa9e7158640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa9e015b110 con 0x7fa9e00b4b50 2024-09-21T03:41:45.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:45.225+0000 7fa9e7158640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fa9e015af90 con 0x7fa9e006e1d0 2024-09-21T03:41:45.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:45.225+0000 7fa9e7158640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fa9e015b290 con 0x7fa9e00b4740 2024-09-21T03:41:45.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:45.225+0000 7fa9e56f5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa9e00b4b50 0x7fa9e0158350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:45.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:45.225+0000 7fa9e56f5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa9e00b4b50 0x7fa9e0158350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54526/0 (socket says 172.21.15.38:54526) 2024-09-21T03:41:45.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:45.225+0000 7fa9e56f5640 1 -- 172.21.15.38:0/1890885141 learned_addr learned my addr 172.21.15.38:0/1890885141 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:41:45.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:45.225+0000 7fa9e4ef4640 1 --2- 172.21.15.38:0/1890885141 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa9e006e1d0 0x7fa9e014a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:45.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:45.225+0000 7fa9dffff640 1 --2- 172.21.15.38:0/1890885141 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa9e00b4740 0x7fa9e0147df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:45.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:45.226+0000 7fa9e56f5640 1 -- 172.21.15.38:0/1890885141 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa9e00b4740 msgr2=0x7fa9e0147df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:45.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:45.226+0000 7fa9e56f5640 1 --2- 172.21.15.38:0/1890885141 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa9e00b4740 0x7fa9e0147df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:45.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:45.226+0000 7fa9e56f5640 1 -- 172.21.15.38:0/1890885141 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa9e006e1d0 msgr2=0x7fa9e014a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:45.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:45.226+0000 7fa9e56f5640 1 --2- 172.21.15.38:0/1890885141 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa9e006e1d0 0x7fa9e014a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:45.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:45.226+0000 7fa9e56f5640 1 -- 172.21.15.38:0/1890885141 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9e014b190 con 0x7fa9e00b4b50 2024-09-21T03:41:45.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:45.226+0000 7fa9e56f5640 1 --2- 172.21.15.38:0/1890885141 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa9e00b4b50 0x7fa9e0158350 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7fa9d800c940 tx=0x7fa9d800ce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:45.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:45.226+0000 7fa9dd7fa640 1 -- 172.21.15.38:0/1890885141 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9d8004460 con 0x7fa9e00b4b50 2024-09-21T03:41:45.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:45.226+0000 7fa9dd7fa640 1 -- 172.21.15.38:0/1890885141 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa9d80077a0 con 0x7fa9e00b4b50 2024-09-21T03:41:45.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:45.226+0000 7fa9dd7fa640 1 -- 172.21.15.38:0/1890885141 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9d8007c10 con 0x7fa9e00b4b50 2024-09-21T03:41:45.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:45.226+0000 7fa9e7158640 1 -- 172.21.15.38:0/1890885141 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9e014b470 con 0x7fa9e00b4b50 2024-09-21T03:41:45.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:45.226+0000 7fa9e7158640 1 -- 172.21.15.38:0/1890885141 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9e014aca0 con 0x7fa9e00b4b50 2024-09-21T03:41:45.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:45.227+0000 7fa9e7158640 1 -- 172.21.15.38:0/1890885141 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9e014aed0 con 0x7fa9e00b4b50 2024-09-21T03:41:45.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:45.228+0000 7fa9dd7fa640 1 -- 172.21.15.38:0/1890885141 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa9d80035f0 con 0x7fa9e00b4b50 2024-09-21T03:41:45.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:45.229+0000 7fa9dd7fa640 1 --2- 172.21.15.38:0/1890885141 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa9c0078bf0 0x7fa9c007b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:45.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:45.229+0000 7fa9dd7fa640 1 -- 172.21.15.38:0/1890885141 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 7508+0+0 (secure 0 0 0) 0x7fa9d809b960 con 0x7fa9e00b4b50 2024-09-21T03:41:45.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:45.229+0000 7fa9e4ef4640 1 --2- 172.21.15.38:0/1890885141 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa9c0078bf0 0x7fa9c007b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:45.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:45.229+0000 7fa9e4ef4640 1 --2- 172.21.15.38:0/1890885141 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa9c0078bf0 0x7fa9c007b100 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fa9d000ae50 tx=0x7fa9d000a540 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:45.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:45.233+0000 7fa9dd7fa640 1 -- 172.21.15.38:0/1890885141 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fa9d8013030 con 0x7fa9e00b4b50 2024-09-21T03:41:45.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:45.461+0000 7fa9e7158640 1 -- 172.21.15.38:0/1890885141 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa9e015b7b0 con 0x7fa9e00b4b50 2024-09-21T03:41:45.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:45.461+0000 7fa9dd7fa640 1 -- 172.21.15.38:0/1890885141 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 166 v166) ==== 95+0+4392 (secure 0 0 0) 0x7fa9d80630d0 con 0x7fa9e00b4b50 2024-09-21T03:41:45.464 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:41:45.464 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":166,"btime":"2024-09-21T03:41:44:737565+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6086,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/965972663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":965972663},{"type":"v1","addr":"172.21.15.195:6836","nonce":965972663}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6700,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/3176122334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":3176122334},{"type":"v1","addr":"172.21.15.195:6837","nonce":3176122334}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":8505,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/1717955365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":1717955365},{"type":"v1","addr":"172.21.15.38:6833","nonce":1717955365}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":159}],"filesystems":[{"mdsmap":{"epoch":166,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:40:02.319282+0000","modified":"2024-09-21T03:41:44.737494+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":110,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-21T03:41:45.464 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 166 2024-09-21T03:41:45.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:45.464+0000 7fa9e7158640 1 -- 172.21.15.38:0/1890885141 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa9c0078bf0 msgr2=0x7fa9c007b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:45.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:45.464+0000 7fa9e7158640 1 --2- 172.21.15.38:0/1890885141 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa9c0078bf0 0x7fa9c007b100 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fa9d000ae50 tx=0x7fa9d000a540 comp rx=0 tx=0).stop 2024-09-21T03:41:45.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:45.464+0000 7fa9e7158640 1 -- 172.21.15.38:0/1890885141 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa9e00b4b50 msgr2=0x7fa9e0158350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:45.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:45.464+0000 7fa9e7158640 1 --2- 172.21.15.38:0/1890885141 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa9e00b4b50 0x7fa9e0158350 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7fa9d800c940 tx=0x7fa9d800ce00 comp rx=0 tx=0).stop 2024-09-21T03:41:45.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:45.464+0000 7fa9e7158640 1 -- 172.21.15.38:0/1890885141 shutdown_connections 2024-09-21T03:41:45.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:45.464+0000 7fa9e7158640 1 --2- 172.21.15.38:0/1890885141 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa9e00b4b50 0x7fa9e0158350 unknown :-1 s=CLOSED pgs=1554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:45.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:45.464+0000 7fa9e7158640 1 --2- 172.21.15.38:0/1890885141 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa9c0078bf0 0x7fa9c007b100 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:45.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:45.464+0000 7fa9e7158640 1 --2- 172.21.15.38:0/1890885141 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa9e00b4740 0x7fa9e0147df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:45.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:45.464+0000 7fa9e7158640 1 --2- 172.21.15.38:0/1890885141 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa9e006e1d0 0x7fa9e014a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:45.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:45.464+0000 7fa9e7158640 1 -- 172.21.15.38:0/1890885141 >> 172.21.15.38:0/1890885141 conn(0x7fa9e00d0a30 msgr2=0x7fa9e00ce5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:45.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:45.464+0000 7fa9e7158640 1 -- 172.21.15.38:0/1890885141 shutdown_connections 2024-09-21T03:41:45.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:45.464+0000 7fa9e7158640 1 -- 172.21.15.38:0/1890885141 wait complete. 2024-09-21T03:41:45.477 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-21T03:41:45.477 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:41:45.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:45.591+0000 7f746e09b640 1 -- 172.21.15.38:0/2900106415 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f746815cf20 msgr2=0x7f74680b6960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:45.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:45.591+0000 7f746e09b640 1 --2- 172.21.15.38:0/2900106415 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f746815cf20 0x7f74680b6960 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f7460005fd0 tx=0x7f746002a940 comp rx=0 tx=0).stop 2024-09-21T03:41:45.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:45.591+0000 7f746e09b640 1 -- 172.21.15.38:0/2900106415 shutdown_connections 2024-09-21T03:41:45.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:45.591+0000 7f746e09b640 1 --2- 172.21.15.38:0/2900106415 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f746815cf20 0x7f74680b6960 unknown :-1 s=CLOSED pgs=1556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:45.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:45.591+0000 7f746e09b640 1 -- 172.21.15.38:0/2900106415 >> 172.21.15.38:0/2900106415 conn(0x7f74680d2350 msgr2=0x7f74680d2760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:45.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:45.591+0000 7f746e09b640 1 -- 172.21.15.38:0/2900106415 shutdown_connections 2024-09-21T03:41:45.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:45.591+0000 7f746e09b640 1 -- 172.21.15.38:0/2900106415 wait complete. 2024-09-21T03:41:45.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:45.593+0000 7f746e09b640 1 Processor -- start 2024-09-21T03:41:45.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:45.593+0000 7f746e09b640 1 -- start start 2024-09-21T03:41:45.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:45.594+0000 7f746e09b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f74680d19f0 0x7f746815b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:45.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:45.594+0000 7f746e09b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f746815cb10 0x7f746806f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:45.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:45.594+0000 7f746e09b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f746815cf20 0x7f74681557e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:45.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:45.594+0000 7f746e09b640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f74680b7020 con 0x7f746815cb10 2024-09-21T03:41:45.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:45.594+0000 7f746e09b640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f74680b6ea0 con 0x7f746815cf20 2024-09-21T03:41:45.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:45.594+0000 7f746e09b640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f74680b71a0 con 0x7f74680d19f0 2024-09-21T03:41:45.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:45.594+0000 7f7466ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f746815cb10 0x7f746806f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:45.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:45.594+0000 7f7466ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f746815cb10 0x7f746806f4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54546/0 (socket says 172.21.15.38:54546) 2024-09-21T03:41:45.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:45.594+0000 7f7466ffd640 1 -- 172.21.15.38:0/3376543981 learned_addr learned my addr 172.21.15.38:0/3376543981 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:41:45.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:45.595+0000 7f7467fff640 1 --2- 172.21.15.38:0/3376543981 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f746815cf20 0x7f74681557e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:45.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:45.595+0000 7f74677fe640 1 --2- 172.21.15.38:0/3376543981 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f74680d19f0 0x7f746815b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:45.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:45.595+0000 7f7466ffd640 1 -- 172.21.15.38:0/3376543981 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f74680d19f0 msgr2=0x7f746815b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:45.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:45.595+0000 7f7466ffd640 1 --2- 172.21.15.38:0/3376543981 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f74680d19f0 0x7f746815b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:45.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:45.595+0000 7f7466ffd640 1 -- 172.21.15.38:0/3376543981 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f746815cf20 msgr2=0x7f74681557e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:45.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:45.595+0000 7f7466ffd640 1 --2- 172.21.15.38:0/3376543981 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f746815cf20 0x7f74681557e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:45.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:45.595+0000 7f7466ffd640 1 -- 172.21.15.38:0/3376543981 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74681f5a80 con 0x7f746815cb10 2024-09-21T03:41:45.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:45.595+0000 7f7466ffd640 1 --2- 172.21.15.38:0/3376543981 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f746815cb10 0x7f746806f4d0 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7f745c00c340 tx=0x7f745c00c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:45.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:45.595+0000 7f744ffff640 1 -- 172.21.15.38:0/3376543981 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f745c011790 con 0x7f746815cb10 2024-09-21T03:41:45.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:45.595+0000 7f744ffff640 1 -- 172.21.15.38:0/3376543981 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f745c017070 con 0x7f746815cb10 2024-09-21T03:41:45.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:45.595+0000 7f746e09b640 1 -- 172.21.15.38:0/3376543981 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74681f5d60 con 0x7f746815cb10 2024-09-21T03:41:45.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:45.595+0000 7f744ffff640 1 -- 172.21.15.38:0/3376543981 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f745c011d00 con 0x7f746815cb10 2024-09-21T03:41:45.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:45.595+0000 7f746e09b640 1 -- 172.21.15.38:0/3376543981 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74681f6320 con 0x7f746815cb10 2024-09-21T03:41:45.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:45.596+0000 7f746e09b640 1 -- 172.21.15.38:0/3376543981 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74680d12e0 con 0x7f746815cb10 2024-09-21T03:41:45.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:45.597+0000 7f744ffff640 1 -- 172.21.15.38:0/3376543981 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f745c01c460 con 0x7f746815cb10 2024-09-21T03:41:45.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:45.598+0000 7f744ffff640 1 --2- 172.21.15.38:0/3376543981 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7454078b90 0x7f745407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:45.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:45.598+0000 7f744ffff640 1 -- 172.21.15.38:0/3376543981 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 7508+0+0 (secure 0 0 0) 0x7f745c09b0b0 con 0x7f746815cb10 2024-09-21T03:41:45.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:45.598+0000 7f74677fe640 1 --2- 172.21.15.38:0/3376543981 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7454078b90 0x7f745407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:45.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:45.599+0000 7f74677fe640 1 --2- 172.21.15.38:0/3376543981 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7454078b90 0x7f745407b0a0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f7458000f60 tx=0x7f7458009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:45.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:45.602+0000 7f744ffff640 1 -- 172.21.15.38:0/3376543981 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f745c062790 con 0x7f746815cb10 2024-09-21T03:41:45.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:45.827+0000 7f746e09b640 1 -- 172.21.15.38:0/3376543981 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f74680b6980 con 0x7f746815cb10 2024-09-21T03:41:45.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:45.827+0000 7f744ffff640 1 -- 172.21.15.38:0/3376543981 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 166 v166) ==== 95+0+4392 (secure 0 0 0) 0x7f745c0671c0 con 0x7f746815cb10 2024-09-21T03:41:45.829 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:41:45.830 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":166,"btime":"2024-09-21T03:41:44:737565+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6086,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/965972663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":965972663},{"type":"v1","addr":"172.21.15.195:6836","nonce":965972663}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6700,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/3176122334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":3176122334},{"type":"v1","addr":"172.21.15.195:6837","nonce":3176122334}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":8505,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/1717955365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":1717955365},{"type":"v1","addr":"172.21.15.38:6833","nonce":1717955365}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":159}],"filesystems":[{"mdsmap":{"epoch":166,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:40:02.319282+0000","modified":"2024-09-21T03:41:44.737494+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":110,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-21T03:41:45.830 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 166 2024-09-21T03:41:45.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:45.830+0000 7f746e09b640 1 -- 172.21.15.38:0/3376543981 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7454078b90 msgr2=0x7f745407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:45.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:45.830+0000 7f746e09b640 1 --2- 172.21.15.38:0/3376543981 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7454078b90 0x7f745407b0a0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f7458000f60 tx=0x7f7458009040 comp rx=0 tx=0).stop 2024-09-21T03:41:45.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:45.830+0000 7f746e09b640 1 -- 172.21.15.38:0/3376543981 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f746815cb10 msgr2=0x7f746806f4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:45.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:45.830+0000 7f746e09b640 1 --2- 172.21.15.38:0/3376543981 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f746815cb10 0x7f746806f4d0 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7f745c00c340 tx=0x7f745c00c800 comp rx=0 tx=0).stop 2024-09-21T03:41:45.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:45.830+0000 7f746e09b640 1 -- 172.21.15.38:0/3376543981 shutdown_connections 2024-09-21T03:41:45.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:45.830+0000 7f746e09b640 1 --2- 172.21.15.38:0/3376543981 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f746815cf20 0x7f74681557e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:45.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:45.830+0000 7f746e09b640 1 --2- 172.21.15.38:0/3376543981 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f746815cb10 0x7f746806f4d0 unknown :-1 s=CLOSED pgs=1557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:45.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:45.830+0000 7f746e09b640 1 --2- 172.21.15.38:0/3376543981 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7454078b90 0x7f745407b0a0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:45.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:45.830+0000 7f746e09b640 1 --2- 172.21.15.38:0/3376543981 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f74680d19f0 0x7f746815b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:45.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:45.830+0000 7f746e09b640 1 -- 172.21.15.38:0/3376543981 >> 172.21.15.38:0/3376543981 conn(0x7f74680d2350 msgr2=0x7f746815e190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:45.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:45.830+0000 7f746e09b640 1 -- 172.21.15.38:0/3376543981 shutdown_connections 2024-09-21T03:41:45.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:45.830+0000 7f746e09b640 1 -- 172.21.15.38:0/3376543981 wait complete. 2024-09-21T03:41:45.843 INFO:tasks.cephfs.test_damage:Result: Mutation 'Delete 100.00000000.inode' led to DAMAGED state 2024-09-21T03:41:45.843 INFO:tasks.cephfs.test_damage:Applying mutation 'Delete 1.00000000' 2024-09-21T03:41:45.843 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-21T03:41:45.843 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:41:45.860 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056795,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2354 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T03:41:45.860 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi038.front.sepia.ceph.com 2024-09-21T03:41:45.860 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:41:45.860 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T03:41:45.915 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T03:41:45.916 INFO:teuthology.orchestra.run.smithi038.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-09-21T03:41:45.916 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-21T03:41:46.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:46.071+0000 7f9847fed640 1 -- 172.21.15.38:0/2311671420 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f98400d0170 msgr2=0x7f98400d0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:46.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:46.071+0000 7f9847fed640 1 --2- 172.21.15.38:0/2311671420 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f98400d0170 0x7f98400d0be0 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f98400d2080 tx=0x7f983402a4d0 comp rx=0 tx=0).stop 2024-09-21T03:41:46.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:46.071+0000 7f9847fed640 1 -- 172.21.15.38:0/2311671420 shutdown_connections 2024-09-21T03:41:46.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:46.071+0000 7f9847fed640 1 --2- 172.21.15.38:0/2311671420 >> v2:172.21.15.195:3300/0 conn(0x7f98400d1120 0x7f98400ceb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:46.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:46.071+0000 7f9847fed640 1 --2- 172.21.15.38:0/2311671420 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f98400d0170 0x7f98400d0be0 unknown :-1 s=CLOSED pgs=1558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:46.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:46.071+0000 7f9847fed640 1 -- 172.21.15.38:0/2311671420 >> 172.21.15.38:0/2311671420 conn(0x7f984009d830 msgr2=0x7f98400b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:46.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:46.071+0000 7f9847fed640 1 -- 172.21.15.38:0/2311671420 shutdown_connections 2024-09-21T03:41:46.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:46.071+0000 7f9847fed640 1 -- 172.21.15.38:0/2311671420 wait complete. 2024-09-21T03:41:46.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:46.073+0000 7f9847fed640 1 Processor -- start 2024-09-21T03:41:46.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:46.073+0000 7f9847fed640 1 -- start start 2024-09-21T03:41:46.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:46.074+0000 7f9847fed640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f98400d0170 0x7f98401552c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:46.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:46.074+0000 7f9847fed640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f98400d1120 0x7f98400b2f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:46.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:46.074+0000 7f9847fed640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f984015ee80 0x7f984015dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:46.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:46.074+0000 7f9847fed640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f98400b6700 con 0x7f984015ee80 2024-09-21T03:41:46.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:46.074+0000 7f9847fed640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f98400b6580 con 0x7f98400d0170 2024-09-21T03:41:46.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:46.074+0000 7f9847fed640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f98400b6880 con 0x7f98400d1120 2024-09-21T03:41:46.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:46.075+0000 7f984658a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f984015ee80 0x7f984015dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:46.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:46.075+0000 7f984658a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f984015ee80 0x7f984015dd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54558/0 (socket says 172.21.15.38:54558) 2024-09-21T03:41:46.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:46.075+0000 7f984658a640 1 -- 172.21.15.38:0/3402726599 learned_addr learned my addr 172.21.15.38:0/3402726599 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:41:46.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:46.075+0000 7f9845d89640 1 --2- 172.21.15.38:0/3402726599 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f98400d0170 0x7f98401552c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:46.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:46.075+0000 7f9845588640 1 --2- 172.21.15.38:0/3402726599 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f98400d1120 0x7f98400b2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:46.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:46.075+0000 7f984658a640 1 -- 172.21.15.38:0/3402726599 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f98400d1120 msgr2=0x7f98400b2f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:46.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:46.075+0000 7f984658a640 1 --2- 172.21.15.38:0/3402726599 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f98400d1120 0x7f98400b2f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:46.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:46.075+0000 7f984658a640 1 -- 172.21.15.38:0/3402726599 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f98400d0170 msgr2=0x7f98401552c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:46.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:46.075+0000 7f984658a640 1 --2- 172.21.15.38:0/3402726599 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f98400d0170 0x7f98401552c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:46.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:46.075+0000 7f984658a640 1 -- 172.21.15.38:0/3402726599 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98401f9240 con 0x7f984015ee80 2024-09-21T03:41:46.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:46.075+0000 7f984658a640 1 --2- 172.21.15.38:0/3402726599 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f984015ee80 0x7f984015dd30 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f983000e990 tx=0x7f983000ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:46.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:46.075+0000 7f983e7fc640 1 -- 172.21.15.38:0/3402726599 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9830004d20 con 0x7f984015ee80 2024-09-21T03:41:46.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:46.075+0000 7f983e7fc640 1 -- 172.21.15.38:0/3402726599 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f983000cb00 con 0x7f984015ee80 2024-09-21T03:41:46.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:46.075+0000 7f983e7fc640 1 -- 172.21.15.38:0/3402726599 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9830005020 con 0x7f984015ee80 2024-09-21T03:41:46.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:46.075+0000 7f9847fed640 1 -- 172.21.15.38:0/3402726599 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98401f9520 con 0x7f984015ee80 2024-09-21T03:41:46.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:46.075+0000 7f9847fed640 1 -- 172.21.15.38:0/3402726599 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98401f9ae0 con 0x7f984015ee80 2024-09-21T03:41:46.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:46.076+0000 7f9847fed640 1 -- 172.21.15.38:0/3402726599 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98400ceec0 con 0x7f984015ee80 2024-09-21T03:41:46.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:46.077+0000 7f983e7fc640 1 -- 172.21.15.38:0/3402726599 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f98300051c0 con 0x7f984015ee80 2024-09-21T03:41:46.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:46.078+0000 7f983e7fc640 1 --2- 172.21.15.38:0/3402726599 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f982c078b90 0x7f982c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:46.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:46.078+0000 7f983e7fc640 1 -- 172.21.15.38:0/3402726599 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 7508+0+0 (secure 0 0 0) 0x7f983009bca0 con 0x7f984015ee80 2024-09-21T03:41:46.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:46.078+0000 7f9845d89640 1 --2- 172.21.15.38:0/3402726599 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f982c078b90 0x7f982c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:46.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:46.079+0000 7f9845d89640 1 --2- 172.21.15.38:0/3402726599 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f982c078b90 0x7f982c07b0a0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f9838001720 tx=0x7f9838009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:46.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:46.082+0000 7f983e7fc640 1 -- 172.21.15.38:0/3402726599 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f98400ceec0 con 0x7f984015ee80 2024-09-21T03:41:46.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:46.308+0000 7f9847fed640 1 -- 172.21.15.38:0/3402726599 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f98400b6da0 con 0x7f984015ee80 2024-09-21T03:41:46.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:46.741+0000 7f983e7fc640 1 -- 172.21.15.38:0/3402726599 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v167) ==== 161+0+0 (secure 0 0 0) 0x7f9830016030 con 0x7f984015ee80 2024-09-21T03:41:46.743 INFO:teuthology.orchestra.run.smithi038.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-21T03:41:46.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:46.744+0000 7f9847fed640 1 -- 172.21.15.38:0/3402726599 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f982c078b90 msgr2=0x7f982c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:46.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:46.744+0000 7f9847fed640 1 --2- 172.21.15.38:0/3402726599 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f982c078b90 0x7f982c07b0a0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f9838001720 tx=0x7f9838009040 comp rx=0 tx=0).stop 2024-09-21T03:41:46.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:46.744+0000 7f9847fed640 1 -- 172.21.15.38:0/3402726599 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f984015ee80 msgr2=0x7f984015dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:46.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:46.744+0000 7f9847fed640 1 --2- 172.21.15.38:0/3402726599 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f984015ee80 0x7f984015dd30 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f983000e990 tx=0x7f983000ee50 comp rx=0 tx=0).stop 2024-09-21T03:41:46.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:46.744+0000 7f9847fed640 1 -- 172.21.15.38:0/3402726599 shutdown_connections 2024-09-21T03:41:46.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:46.744+0000 7f9847fed640 1 --2- 172.21.15.38:0/3402726599 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f984015ee80 0x7f984015dd30 unknown :-1 s=CLOSED pgs=1559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:46.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:46.744+0000 7f9847fed640 1 --2- 172.21.15.38:0/3402726599 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f982c078b90 0x7f982c07b0a0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:46.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:46.744+0000 7f9847fed640 1 --2- 172.21.15.38:0/3402726599 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f98400d1120 0x7f98400b2f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:46.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:46.744+0000 7f9847fed640 1 --2- 172.21.15.38:0/3402726599 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f98400d0170 0x7f98401552c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:46.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:46.744+0000 7f9847fed640 1 -- 172.21.15.38:0/3402726599 >> 172.21.15.38:0/3402726599 conn(0x7f984009d830 msgr2=0x7f9840161130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:46.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:46.744+0000 7f9847fed640 1 -- 172.21.15.38:0/3402726599 shutdown_connections 2024-09-21T03:41:46.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:46.744+0000 7f9847fed640 1 -- 172.21.15.38:0/3402726599 wait complete. 2024-09-21T03:41:46.758 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-09-21T03:41:46.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:46.876+0000 7fb834030640 1 -- 172.21.15.38:0/4164511063 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb82c15ee80 msgr2=0x7fb82c0d0170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:46.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:46.876+0000 7fb834030640 1 --2- 172.21.15.38:0/4164511063 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb82c15ee80 0x7fb82c0d0170 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7fb82c0b3c20 tx=0x7fb82802a4d0 comp rx=0 tx=0).stop 2024-09-21T03:41:46.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:46.876+0000 7fb834030640 1 -- 172.21.15.38:0/4164511063 shutdown_connections 2024-09-21T03:41:46.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:46.876+0000 7fb834030640 1 --2- 172.21.15.38:0/4164511063 >> v2:172.21.15.195:3300/0 conn(0x7fb82c0ceb90 0x7fb82c0cefa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:46.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:46.876+0000 7fb834030640 1 --2- 172.21.15.38:0/4164511063 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb82c15ee80 0x7fb82c0d0170 unknown :-1 s=CLOSED pgs=1560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:46.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:46.876+0000 7fb834030640 1 -- 172.21.15.38:0/4164511063 >> 172.21.15.38:0/4164511063 conn(0x7fb82c09d830 msgr2=0x7fb82c0b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:46.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:46.876+0000 7fb834030640 1 -- 172.21.15.38:0/4164511063 shutdown_connections 2024-09-21T03:41:46.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:46.876+0000 7fb834030640 1 -- 172.21.15.38:0/4164511063 wait complete. 2024-09-21T03:41:46.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:46.878+0000 7fb834030640 1 Processor -- start 2024-09-21T03:41:46.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:46.878+0000 7fb834030640 1 -- start start 2024-09-21T03:41:46.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:46.879+0000 7fb834030640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb82c0b9ad0 0x7fb82c15a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:46.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:46.879+0000 7fb834030640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb82c0ceb90 0x7fb82c0d3d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:46.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:46.879+0000 7fb834030640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb82c15ee80 0x7fb82c15b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:46.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:46.879+0000 7fb834030640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fb82c0b6700 con 0x7fb82c15ee80 2024-09-21T03:41:46.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:46.879+0000 7fb834030640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fb82c0b6580 con 0x7fb82c0b9ad0 2024-09-21T03:41:46.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:46.879+0000 7fb834030640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fb82c0b6880 con 0x7fb82c0ceb90 2024-09-21T03:41:46.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:46.879+0000 7fb8325cd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb82c15ee80 0x7fb82c15b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:46.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:46.879+0000 7fb8325cd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb82c15ee80 0x7fb82c15b240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54576/0 (socket says 172.21.15.38:54576) 2024-09-21T03:41:46.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:46.879+0000 7fb8325cd640 1 -- 172.21.15.38:0/2310385887 learned_addr learned my addr 172.21.15.38:0/2310385887 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:41:46.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:46.879+0000 7fb831dcc640 1 --2- 172.21.15.38:0/2310385887 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb82c0b9ad0 0x7fb82c15a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:46.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:46.879+0000 7fb8315cb640 1 --2- 172.21.15.38:0/2310385887 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb82c0ceb90 0x7fb82c0d3d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:46.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:46.879+0000 7fb8325cd640 1 -- 172.21.15.38:0/2310385887 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb82c0ceb90 msgr2=0x7fb82c0d3d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:46.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:46.879+0000 7fb8325cd640 1 --2- 172.21.15.38:0/2310385887 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb82c0ceb90 0x7fb82c0d3d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:46.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:46.879+0000 7fb8325cd640 1 -- 172.21.15.38:0/2310385887 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb82c0b9ad0 msgr2=0x7fb82c15a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:46.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:46.879+0000 7fb8325cd640 1 --2- 172.21.15.38:0/2310385887 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb82c0b9ad0 0x7fb82c15a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:46.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:46.879+0000 7fb8325cd640 1 -- 172.21.15.38:0/2310385887 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb82c0d42a0 con 0x7fb82c15ee80 2024-09-21T03:41:46.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:46.880+0000 7fb8325cd640 1 --2- 172.21.15.38:0/2310385887 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb82c15ee80 0x7fb82c15b240 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7fb82400e990 tx=0x7fb82400ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:46.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:46.880+0000 7fb8227fc640 1 -- 172.21.15.38:0/2310385887 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb824004d20 con 0x7fb82c15ee80 2024-09-21T03:41:46.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:46.880+0000 7fb8227fc640 1 -- 172.21.15.38:0/2310385887 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb82400caa0 con 0x7fb82c15ee80 2024-09-21T03:41:46.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:46.880+0000 7fb834030640 1 -- 172.21.15.38:0/2310385887 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb82c159700 con 0x7fb82c15ee80 2024-09-21T03:41:46.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:46.880+0000 7fb8227fc640 1 -- 172.21.15.38:0/2310385887 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb824005020 con 0x7fb82c15ee80 2024-09-21T03:41:46.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:46.880+0000 7fb834030640 1 -- 172.21.15.38:0/2310385887 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb82c159cc0 con 0x7fb82c15ee80 2024-09-21T03:41:46.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:46.881+0000 7fb834030640 1 -- 172.21.15.38:0/2310385887 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb82c0d44e0 con 0x7fb82c15ee80 2024-09-21T03:41:46.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:46.882+0000 7fb8227fc640 1 -- 172.21.15.38:0/2310385887 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb8240051c0 con 0x7fb82c15ee80 2024-09-21T03:41:46.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:46.882+0000 7fb8227fc640 1 --2- 172.21.15.38:0/2310385887 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb810078bc0 0x7fb81007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:46.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:46.882+0000 7fb8227fc640 1 -- 172.21.15.38:0/2310385887 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 7508+0+0 (secure 0 0 0) 0x7fb82409bc10 con 0x7fb82c15ee80 2024-09-21T03:41:46.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:46.882+0000 7fb831dcc640 1 --2- 172.21.15.38:0/2310385887 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb810078bc0 0x7fb81007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:46.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:46.883+0000 7fb831dcc640 1 --2- 172.21.15.38:0/2310385887 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb810078bc0 0x7fb81007b0d0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fb828033000 tx=0x7fb828004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:46.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:46.887+0000 7fb8227fc640 1 -- 172.21.15.38:0/2310385887 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fb824016030 con 0x7fb82c15ee80 2024-09-21T03:41:47.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.152+0000 7fb834030640 1 -- 172.21.15.38:0/2310385887 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7fb82c0b6dd0 con 0x7fb82c15ee80 2024-09-21T03:41:47.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.755+0000 7fb8227fc640 1 -- 172.21.15.38:0/2310385887 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v168) ==== 101+0+0 (secure 0 0 0) 0x7fb824063380 con 0x7fb82c15ee80 2024-09-21T03:41:47.757 INFO:teuthology.orchestra.run.smithi038.stderr:repaired: restoring rank 6:0 2024-09-21T03:41:47.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.758+0000 7fb834030640 1 -- 172.21.15.38:0/2310385887 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb810078bc0 msgr2=0x7fb81007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:47.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.758+0000 7fb834030640 1 --2- 172.21.15.38:0/2310385887 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb810078bc0 0x7fb81007b0d0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fb828033000 tx=0x7fb828004730 comp rx=0 tx=0).stop 2024-09-21T03:41:47.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.758+0000 7fb834030640 1 -- 172.21.15.38:0/2310385887 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb82c15ee80 msgr2=0x7fb82c15b240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:47.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.758+0000 7fb834030640 1 --2- 172.21.15.38:0/2310385887 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb82c15ee80 0x7fb82c15b240 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7fb82400e990 tx=0x7fb82400ee50 comp rx=0 tx=0).stop 2024-09-21T03:41:47.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.759+0000 7fb834030640 1 -- 172.21.15.38:0/2310385887 shutdown_connections 2024-09-21T03:41:47.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.759+0000 7fb834030640 1 --2- 172.21.15.38:0/2310385887 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb82c15ee80 0x7fb82c15b240 unknown :-1 s=CLOSED pgs=1561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:47.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.759+0000 7fb834030640 1 --2- 172.21.15.38:0/2310385887 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb810078bc0 0x7fb81007b0d0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:47.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.759+0000 7fb834030640 1 --2- 172.21.15.38:0/2310385887 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb82c0ceb90 0x7fb82c0d3d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:47.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.759+0000 7fb834030640 1 --2- 172.21.15.38:0/2310385887 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb82c0b9ad0 0x7fb82c15a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:47.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.759+0000 7fb834030640 1 -- 172.21.15.38:0/2310385887 >> 172.21.15.38:0/2310385887 conn(0x7fb82c09d830 msgr2=0x7fb82c0d06e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:47.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.759+0000 7fb834030640 1 -- 172.21.15.38:0/2310385887 shutdown_connections 2024-09-21T03:41:47.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.759+0000 7fb834030640 1 -- 172.21.15.38:0/2310385887 wait complete. 2024-09-21T03:41:47.775 DEBUG:teuthology.orchestra.run.smithi038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-09-21T03:41:47.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.834+0000 7fe8ee745f40 1 Processor -- start 2024-09-21T03:41:47.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.834+0000 7fe8ee745f40 1 -- start start 2024-09-21T03:41:47.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.835+0000 7fe8ee745f40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564d4a93fd00 0x564d4aa08a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:47.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.835+0000 7fe8ee745f40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x564d4a9fa950 0x564d4a96d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:47.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.835+0000 7fe8ee745f40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564d4aa15670 0x564d4a96de00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:47.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.835+0000 7fe8ee745f40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x564d4a88dd60 con 0x564d4aa15670 2024-09-21T03:41:47.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.835+0000 7fe8ee745f40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x564d4a8ec430 con 0x564d4a93fd00 2024-09-21T03:41:47.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.835+0000 7fe8ee745f40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x564d4a9877b0 con 0x564d4a9fa950 2024-09-21T03:41:47.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.835+0000 7fe8e8a00640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564d4aa15670 0x564d4a96de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:47.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.835+0000 7fe8e81ff640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564d4a93fd00 0x564d4aa08a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:47.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.835+0000 7fe8e8a00640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564d4aa15670 0x564d4a96de00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54590/0 (socket says 172.21.15.38:54590) 2024-09-21T03:41:47.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.835+0000 7fe8e79fe640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x564d4a9fa950 0x564d4a96d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:47.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.835+0000 7fe8e8a00640 1 -- 172.21.15.38:0/1588836940 learned_addr learned my addr 172.21.15.38:0/1588836940 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:41:47.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.836+0000 7fe8e8a00640 1 -- 172.21.15.38:0/1588836940 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x564d4a9fa950 msgr2=0x564d4a96d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:47.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.836+0000 7fe8e8a00640 1 --2- 172.21.15.38:0/1588836940 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x564d4a9fa950 0x564d4a96d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:47.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.836+0000 7fe8e8a00640 1 -- 172.21.15.38:0/1588836940 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564d4a93fd00 msgr2=0x564d4aa08a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:47.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.836+0000 7fe8e8a00640 1 --2- 172.21.15.38:0/1588836940 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564d4a93fd00 0x564d4aa08a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:47.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.836+0000 7fe8e8a00640 1 -- 172.21.15.38:0/1588836940 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d4aa1f820 con 0x564d4aa15670 2024-09-21T03:41:47.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.836+0000 7fe8e8a00640 1 --2- 172.21.15.38:0/1588836940 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564d4aa15670 0x564d4a96de00 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7fe8d8001590 tx=0x7fe8d8001a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c796d763e1f61be server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:47.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.836+0000 7fe8e69fc640 1 -- 172.21.15.38:0/1588836940 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8d8016070 con 0x564d4aa15670 2024-09-21T03:41:47.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.836+0000 7fe8e69fc640 1 -- 172.21.15.38:0/1588836940 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe8d800f800 con 0x564d4aa15670 2024-09-21T03:41:47.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.836+0000 7fe8e69fc640 1 -- 172.21.15.38:0/1588836940 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8d800fc30 con 0x564d4aa15670 2024-09-21T03:41:47.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.837+0000 7fe8ee745f40 1 -- 172.21.15.38:0/1588836940 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564d4aa15670 msgr2=0x564d4a96de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:47.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.837+0000 7fe8ee745f40 1 --2- 172.21.15.38:0/1588836940 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564d4aa15670 0x564d4a96de00 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7fe8d8001590 tx=0x7fe8d8001a50 comp rx=0 tx=0).stop 2024-09-21T03:41:47.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.837+0000 7fe8ee745f40 1 -- 172.21.15.38:0/1588836940 shutdown_connections 2024-09-21T03:41:47.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.837+0000 7fe8ee745f40 1 --2- 172.21.15.38:0/1588836940 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564d4aa15670 0x564d4a96de00 unknown :-1 s=CLOSED pgs=1564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:47.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.837+0000 7fe8ee745f40 1 --2- 172.21.15.38:0/1588836940 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x564d4a9fa950 0x564d4a96d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:47.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.837+0000 7fe8ee745f40 1 --2- 172.21.15.38:0/1588836940 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564d4a93fd00 0x564d4aa08a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:47.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.837+0000 7fe8ee745f40 1 -- 172.21.15.38:0/1588836940 >> 172.21.15.38:0/1588836940 conn(0x564d4a85eac0 msgr2=0x564d4aa15a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:47.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.837+0000 7fe8ee745f40 1 -- 172.21.15.38:0/1588836940 shutdown_connections 2024-09-21T03:41:47.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.837+0000 7fe8ee745f40 1 -- 172.21.15.38:0/1588836940 wait complete. 2024-09-21T03:41:47.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.838+0000 7fe8ee745f40 1 Processor -- start 2024-09-21T03:41:47.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.838+0000 7fe8ee745f40 1 -- start start 2024-09-21T03:41:47.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.839+0000 7fe8ee745f40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x564d4a93fd00 0x564d4a9f35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:47.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.839+0000 7fe8ee745f40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564d4a9fa950 0x564d4aa14280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:47.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.839+0000 7fe8ee745f40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564d4aa15670 0x564d4aa147c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:47.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.839+0000 7fe8ee745f40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x564d4a9f5d20 con 0x564d4aa15670 2024-09-21T03:41:47.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.839+0000 7fe8ee745f40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x564d4a96d430 con 0x564d4a9fa950 2024-09-21T03:41:47.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.839+0000 7fe8ee745f40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x564d4aa0bb10 con 0x564d4a93fd00 2024-09-21T03:41:47.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.839+0000 7fe8e8a00640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564d4aa15670 0x564d4aa147c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:47.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.839+0000 7fe8e8a00640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564d4aa15670 0x564d4aa147c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54598/0 (socket says 172.21.15.38:54598) 2024-09-21T03:41:47.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.839+0000 7fe8e8a00640 1 -- 172.21.15.38:0/972509514 learned_addr learned my addr 172.21.15.38:0/972509514 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:41:47.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.840+0000 7fe8e79fe640 1 --2- 172.21.15.38:0/972509514 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564d4a9fa950 0x564d4aa14280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:47.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.840+0000 7fe8e81ff640 1 --2- 172.21.15.38:0/972509514 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x564d4a93fd00 0x564d4a9f35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:47.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.840+0000 7fe8e8a00640 1 -- 172.21.15.38:0/972509514 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x564d4a93fd00 msgr2=0x564d4a9f35e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:47.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.840+0000 7fe8e8a00640 1 --2- 172.21.15.38:0/972509514 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x564d4a93fd00 0x564d4a9f35e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:47.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.840+0000 7fe8e8a00640 1 -- 172.21.15.38:0/972509514 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564d4a9fa950 msgr2=0x564d4aa14280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:47.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.840+0000 7fe8e8a00640 1 --2- 172.21.15.38:0/972509514 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564d4a9fa950 0x564d4aa14280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:47.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.840+0000 7fe8e8a00640 1 -- 172.21.15.38:0/972509514 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d4aa11db0 con 0x564d4aa15670 2024-09-21T03:41:47.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.840+0000 7fe8e8a00640 1 --2- 172.21.15.38:0/972509514 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564d4aa15670 0x564d4aa147c0 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7fe8d8009040 tx=0x7fe8d8006ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:47.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.840+0000 7fe8e59fa640 1 -- 172.21.15.38:0/972509514 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8d8021020 con 0x564d4aa15670 2024-09-21T03:41:47.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.840+0000 7fe8e59fa640 1 -- 172.21.15.38:0/972509514 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe8d8017040 con 0x564d4aa15670 2024-09-21T03:41:47.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.840+0000 7fe8ee745f40 1 -- 172.21.15.38:0/972509514 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x564d4a9f41b0 con 0x564d4aa15670 2024-09-21T03:41:47.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.840+0000 7fe8e59fa640 1 -- 172.21.15.38:0/972509514 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8d8016070 con 0x564d4aa15670 2024-09-21T03:41:47.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.840+0000 7fe8ee745f40 1 -- 172.21.15.38:0/972509514 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d4aa0a790 con 0x564d4aa15670 2024-09-21T03:41:47.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.842+0000 7fe8e59fa640 1 -- 172.21.15.38:0/972509514 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fe8d8024020 con 0x564d4aa15670 2024-09-21T03:41:47.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.842+0000 7fe8e59fa640 1 --2- 172.21.15.38:0/972509514 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fe8c8077570 0x7fe8c8079a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:47.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.843+0000 7fe8e59fa640 1 -- 172.21.15.38:0/972509514 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 7508+0+0 (secure 0 0 0) 0x7fe8d809b820 con 0x564d4aa15670 2024-09-21T03:41:47.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.843+0000 7fe8e81ff640 1 --2- 172.21.15.38:0/972509514 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fe8c8077570 0x7fe8c8079a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:47.846 INFO:teuthology.orchestra.run.smithi038.stdout:Importing pool 2024-09-21T03:41:47.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.843+0000 7fe8ee745f40 1 --2- 172.21.15.38:0/972509514 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x564d4a9f9d90 0x564d4aa0b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:47.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.843+0000 7fe8ee745f40 1 -- 172.21.15.38:0/972509514 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x564d4aa0b640 con 0x564d4a9f9d90 2024-09-21T03:41:47.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.843+0000 7fe8e79fe640 1 --2- 172.21.15.38:0/972509514 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x564d4a9f9d90 0x564d4aa0b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:47.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.843+0000 7fe8e81ff640 1 --2- 172.21.15.38:0/972509514 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fe8c8077570 0x7fe8c8079a80 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fe8e00041b0 tx=0x7fe8e000d040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:47.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.844+0000 7fe8e79fe640 1 --2- 172.21.15.38:0/972509514 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x564d4a9f9d90 0x564d4aa0b100 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:47.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.846+0000 7fe8e79fe640 1 -- 172.21.15.38:0/972509514 <== osd.7 v2:172.21.15.195:6818/1606106342 1 ==== osd_op_reply(1 601.00000000 [create] v110'412 uv386 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe8d0056470 con 0x564d4a9f9d90 2024-09-21T03:41:47.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.846+0000 7fe8ee745f40 1 -- 172.21.15.38:0/972509514 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e110) -- 0x564d4aab8eb0 con 0x564d4a9f9d90 2024-09-21T03:41:47.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.847+0000 7fe8e79fe640 1 -- 172.21.15.38:0/972509514 <== osd.7 v2:172.21.15.195:6818/1606106342 2 ==== osd_op_reply(2 601.00000000 [delete] v110'413 uv413 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe8d0056470 con 0x564d4a9f9d90 2024-09-21T03:41:47.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.847+0000 7fe8ee745f40 1 -- 172.21.15.38:0/972509514 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x564d4aab9250 con 0x564d4a9f9d90 2024-09-21T03:41:47.850 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-09-21T03:41:47.851 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.849+0000 7fe8e79fe640 1 -- 172.21.15.38:0/972509514 <== osd.7 v2:172.21.15.195:6818/1606106342 3 ==== osd_op_reply(3 601.00000000 [create] v110'414 uv414 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe8d0056470 con 0x564d4a9f9d90 2024-09-21T03:41:47.851 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.849+0000 7fe8ee745f40 1 -- 172.21.15.38:0/972509514 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x564d4aaba600 con 0x564d4a9f9d90 2024-09-21T03:41:47.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.850+0000 7fe8e79fe640 1 -- 172.21.15.38:0/972509514 <== osd.7 v2:172.21.15.195:6818/1606106342 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v110'415 uv415 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe8d0056470 con 0x564d4a9f9d90 2024-09-21T03:41:47.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.850+0000 7fe8ee745f40 1 -- 172.21.15.38:0/972509514 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x564d4aaba9a0 con 0x564d4a9f9d90 2024-09-21T03:41:47.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.851+0000 7fe8e79fe640 1 -- 172.21.15.38:0/972509514 <== osd.7 v2:172.21.15.195:6818/1606106342 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v110'416 uv416 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe8d0056470 con 0x564d4a9f9d90 2024-09-21T03:41:47.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.852+0000 7fe8ee745f40 1 -- 172.21.15.38:0/972509514 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x564d4aab95f0 con 0x564d4a9f9d90 2024-09-21T03:41:47.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.854+0000 7fe8e79fe640 1 -- 172.21.15.38:0/972509514 <== osd.7 v2:172.21.15.195:6818/1606106342 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v110'417 uv417 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe8d0056470 con 0x564d4a9f9d90 2024-09-21T03:41:47.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.854+0000 7fe8ee745f40 1 -- 172.21.15.38:0/972509514 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x564d4aab99e0 con 0x564d4a9f9d90 2024-09-21T03:41:47.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.854+0000 7fe8e79fe640 1 -- 172.21.15.38:0/972509514 <== osd.7 v2:172.21.15.195:6818/1606106342 7 ==== osd_op_reply(7 602.00000000 [create] v110'418 uv392 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe8d0056470 con 0x564d4a9f9d90 2024-09-21T03:41:47.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.854+0000 7fe8ee745f40 1 -- 172.21.15.38:0/972509514 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e110) -- 0x564d4aab99e0 con 0x564d4a9f9d90 2024-09-21T03:41:47.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.856+0000 7fe8e79fe640 1 -- 172.21.15.38:0/972509514 <== osd.7 v2:172.21.15.195:6818/1606106342 8 ==== osd_op_reply(8 602.00000000 [delete] v110'419 uv419 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe8d0056470 con 0x564d4a9f9d90 2024-09-21T03:41:47.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.856+0000 7fe8ee745f40 1 -- 172.21.15.38:0/972509514 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x564d4aab99e0 con 0x564d4a9f9d90 2024-09-21T03:41:47.859 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-09-21T03:41:47.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.857+0000 7fe8e79fe640 1 -- 172.21.15.38:0/972509514 <== osd.7 v2:172.21.15.195:6818/1606106342 9 ==== osd_op_reply(9 602.00000000 [create] v110'420 uv420 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe8d0056470 con 0x564d4a9f9d90 2024-09-21T03:41:47.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.857+0000 7fe8ee745f40 1 -- 172.21.15.38:0/972509514 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x564d4aab99e0 con 0x564d4a9f9d90 2024-09-21T03:41:47.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.859+0000 7fe8e79fe640 1 -- 172.21.15.38:0/972509514 <== osd.7 v2:172.21.15.195:6818/1606106342 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v110'421 uv421 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe8d0056470 con 0x564d4a9f9d90 2024-09-21T03:41:47.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.859+0000 7fe8ee745f40 1 -- 172.21.15.38:0/972509514 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x564d4aab99e0 con 0x564d4a9f9d90 2024-09-21T03:41:47.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.860+0000 7fe8e79fe640 1 -- 172.21.15.38:0/972509514 <== osd.7 v2:172.21.15.195:6818/1606106342 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v110'422 uv422 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe8d0056470 con 0x564d4a9f9d90 2024-09-21T03:41:47.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.860+0000 7fe8ee745f40 1 -- 172.21.15.38:0/972509514 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x564d4aab99e0 con 0x564d4a9f9d90 2024-09-21T03:41:47.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.861+0000 7fe8e79fe640 1 -- 172.21.15.38:0/972509514 <== osd.7 v2:172.21.15.195:6818/1606106342 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v110'423 uv423 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe8d0056470 con 0x564d4a9f9d90 2024-09-21T03:41:47.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.861+0000 7fe8ee745f40 1 -- 172.21.15.38:0/972509514 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:13 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x564d4aab99e0 con 0x564d4a9f9d90 2024-09-21T03:41:47.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.864+0000 7fe8e79fe640 1 -- 172.21.15.38:0/972509514 <== osd.7 v2:172.21.15.195:6818/1606106342 13 ==== osd_op_reply(13 10000000000.00000000 [create] v110'424 uv399 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7fe8d0056470 con 0x564d4a9f9d90 2024-09-21T03:41:47.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.864+0000 7fe8ee745f40 1 -- 172.21.15.38:0/972509514 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:14 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e110) -- 0x564d4aab99e0 con 0x564d4a9f9d90 2024-09-21T03:41:47.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.865+0000 7fe8e79fe640 1 -- 172.21.15.38:0/972509514 <== osd.7 v2:172.21.15.195:6818/1606106342 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v110'425 uv425 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fe8d0056470 con 0x564d4a9f9d90 2024-09-21T03:41:47.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.865+0000 7fe8ee745f40 1 -- 172.21.15.38:0/972509514 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:15 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x564d4aab99e0 con 0x564d4a9f9d90 2024-09-21T03:41:47.868 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-09-21T03:41:47.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.866+0000 7fe8e79fe640 1 -- 172.21.15.38:0/972509514 <== osd.7 v2:172.21.15.195:6818/1606106342 15 ==== osd_op_reply(15 10000000000.00000000 [create] v110'426 uv426 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fe8d0056470 con 0x564d4a9f9d90 2024-09-21T03:41:47.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.866+0000 7fe8ee745f40 1 -- 172.21.15.38:0/972509514 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:16 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x564d4aab99e0 con 0x564d4a9f9d90 2024-09-21T03:41:47.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.868+0000 7fe8e79fe640 1 -- 172.21.15.38:0/972509514 <== osd.7 v2:172.21.15.195:6818/1606106342 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v110'427 uv427 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fe8d0056470 con 0x564d4a9f9d90 2024-09-21T03:41:47.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.868+0000 7fe8ee745f40 1 -- 172.21.15.38:0/972509514 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:17 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x564d4aab99e0 con 0x564d4a9f9d90 2024-09-21T03:41:47.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.870+0000 7fe8e79fe640 1 -- 172.21.15.38:0/972509514 <== osd.7 v2:172.21.15.195:6818/1606106342 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v110'428 uv428 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fe8d0056470 con 0x564d4a9f9d90 2024-09-21T03:41:47.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.870+0000 7fe8ee745f40 1 -- 172.21.15.38:0/972509514 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:18 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x564d4aab99e0 con 0x564d4a9f9d90 2024-09-21T03:41:47.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.871+0000 7fe8e79fe640 1 -- 172.21.15.38:0/972509514 <== osd.7 v2:172.21.15.195:6818/1606106342 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v110'429 uv429 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fe8d0056470 con 0x564d4a9f9d90 2024-09-21T03:41:47.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.871+0000 7fe8ee745f40 1 -- 172.21.15.38:0/972509514 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:19 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=585b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x564d4aab9da0 con 0x564d4a9f9d90 2024-09-21T03:41:47.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.873+0000 7fe8e79fe640 1 -- 172.21.15.38:0/972509514 <== osd.7 v2:172.21.15.195:6818/1606106342 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v110'430 uv430 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fe8d0056470 con 0x564d4a9f9d90 2024-09-21T03:41:47.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.873+0000 7fe8ee745f40 1 -- 172.21.15.38:0/972509514 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:20 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x564d4aab9da0 con 0x564d4a9f9d90 2024-09-21T03:41:47.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.873+0000 7fe8e79fe640 1 -- 172.21.15.38:0/972509514 <== osd.7 v2:172.21.15.195:6818/1606106342 20 ==== osd_op_reply(20 600.00000000 [create] v110'431 uv405 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe8d0056470 con 0x564d4a9f9d90 2024-09-21T03:41:47.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.873+0000 7fe8ee745f40 1 -- 172.21.15.38:0/972509514 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:21 13.10 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e110) -- 0x564d4aab9da0 con 0x564d4a9f9d90 2024-09-21T03:41:47.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.875+0000 7fe8e79fe640 1 -- 172.21.15.38:0/972509514 <== osd.7 v2:172.21.15.195:6818/1606106342 21 ==== osd_op_reply(21 600.00000000 [delete] v110'432 uv432 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe8d0056470 con 0x564d4a9f9d90 2024-09-21T03:41:47.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.875+0000 7fe8ee745f40 1 -- 172.21.15.38:0/972509514 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:22 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x564d4aab9c90 con 0x564d4a9f9d90 2024-09-21T03:41:47.878 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-09-21T03:41:47.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.876+0000 7fe8e79fe640 1 -- 172.21.15.38:0/972509514 <== osd.7 v2:172.21.15.195:6818/1606106342 22 ==== osd_op_reply(22 600.00000000 [create] v110'433 uv433 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe8d0056470 con 0x564d4a9f9d90 2024-09-21T03:41:47.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.876+0000 7fe8ee745f40 1 -- 172.21.15.38:0/972509514 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:23 13.10 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x564d4aab9c90 con 0x564d4a9f9d90 2024-09-21T03:41:47.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.877+0000 7fe8e79fe640 1 -- 172.21.15.38:0/972509514 <== osd.7 v2:172.21.15.195:6818/1606106342 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v110'434 uv434 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe8d0056470 con 0x564d4a9f9d90 2024-09-21T03:41:47.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.877+0000 7fe8ee745f40 1 -- 172.21.15.38:0/972509514 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:24 13.10 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x564d4aab9c90 con 0x564d4a9f9d90 2024-09-21T03:41:47.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.879+0000 7fe8e79fe640 1 -- 172.21.15.38:0/972509514 <== osd.7 v2:172.21.15.195:6818/1606106342 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v110'435 uv435 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe8d0056470 con 0x564d4a9f9d90 2024-09-21T03:41:47.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.879+0000 7fe8ee745f40 1 -- 172.21.15.38:0/972509514 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:25 13.10 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x564d4aab9c90 con 0x564d4a9f9d90 2024-09-21T03:41:47.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.880+0000 7fe8e79fe640 1 -- 172.21.15.38:0/972509514 <== osd.7 v2:172.21.15.195:6818/1606106342 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v110'436 uv436 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe8d0056470 con 0x564d4a9f9d90 2024-09-21T03:41:47.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.880+0000 7fe8ee745f40 1 -- 172.21.15.38:0/972509514 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:26 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x564d4aab9c90 con 0x564d4a9f9d90 2024-09-21T03:41:47.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.881+0000 7fe8e79fe640 1 -- 172.21.15.38:0/972509514 <== osd.7 v2:172.21.15.195:6818/1606106342 26 ==== osd_op_reply(26 603.00000000 [create] v110'437 uv411 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe8d0056470 con 0x564d4a9f9d90 2024-09-21T03:41:47.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.881+0000 7fe8ee745f40 1 -- 172.21.15.38:0/972509514 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:27 13.10 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e110) -- 0x564d4aab9c90 con 0x564d4a9f9d90 2024-09-21T03:41:47.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.883+0000 7fe8e79fe640 1 -- 172.21.15.38:0/972509514 <== osd.7 v2:172.21.15.195:6818/1606106342 27 ==== osd_op_reply(27 603.00000000 [delete] v110'438 uv438 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe8d0056470 con 0x564d4a9f9d90 2024-09-21T03:41:47.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.883+0000 7fe8ee745f40 1 -- 172.21.15.38:0/972509514 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:28 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x564d4aab9c90 con 0x564d4a9f9d90 2024-09-21T03:41:47.885 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-09-21T03:41:47.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.884+0000 7fe8e79fe640 1 -- 172.21.15.38:0/972509514 <== osd.7 v2:172.21.15.195:6818/1606106342 28 ==== osd_op_reply(28 603.00000000 [create] v110'439 uv439 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe8d0056470 con 0x564d4a9f9d90 2024-09-21T03:41:47.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.884+0000 7fe8ee745f40 1 -- 172.21.15.38:0/972509514 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:29 13.10 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x564d4aab9c90 con 0x564d4a9f9d90 2024-09-21T03:41:47.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.885+0000 7fe8e79fe640 1 -- 172.21.15.38:0/972509514 <== osd.7 v2:172.21.15.195:6818/1606106342 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v110'440 uv440 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe8d0056470 con 0x564d4a9f9d90 2024-09-21T03:41:47.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.885+0000 7fe8ee745f40 1 -- 172.21.15.38:0/972509514 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:30 13.10 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x564d4aab9c90 con 0x564d4a9f9d90 2024-09-21T03:41:47.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.886+0000 7fe8e79fe640 1 -- 172.21.15.38:0/972509514 <== osd.7 v2:172.21.15.195:6818/1606106342 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v110'441 uv441 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe8d0056470 con 0x564d4a9f9d90 2024-09-21T03:41:47.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.886+0000 7fe8ee745f40 1 -- 172.21.15.38:0/972509514 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:31 13.10 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x564d4aab9c90 con 0x564d4a9f9d90 2024-09-21T03:41:47.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.887+0000 7fe8e79fe640 1 -- 172.21.15.38:0/972509514 <== osd.7 v2:172.21.15.195:6818/1606106342 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v110'442 uv442 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe8d0056470 con 0x564d4a9f9d90 2024-09-21T03:41:47.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.887+0000 7fe8ee745f40 1 --2- 172.21.15.38:0/972509514 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x564d4aabad40 0x564d4aabd160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:47.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.888+0000 7fe8ee745f40 1 -- 172.21.15.38:0/972509514 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x564d4aab9c90 con 0x564d4aabad40 2024-09-21T03:41:47.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.888+0000 7fe8e81ff640 1 --2- 172.21.15.38:0/972509514 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x564d4aabad40 0x564d4aabd160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:47.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.888+0000 7fe8e81ff640 1 --2- 172.21.15.38:0/972509514 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x564d4aabad40 0x564d4aabd160 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:47.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.890+0000 7fe8e81ff640 1 -- 172.21.15.38:0/972509514 <== osd.6 v2:172.21.15.195:6826/4055825675 1 ==== osd_op_reply(32 1.00000000.inode [create] v110'307 uv290 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7fe8e0017040 con 0x564d4aabad40 2024-09-21T03:41:47.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.890+0000 7fe8ee745f40 1 -- 172.21.15.38:0/972509514 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e110) -- 0x564d4aabd6a0 con 0x564d4aabad40 2024-09-21T03:41:47.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.891+0000 7fe8e81ff640 1 -- 172.21.15.38:0/972509514 <== osd.6 v2:172.21.15.195:6826/4055825675 2 ==== osd_op_reply(33 1.00000000.inode [delete] v110'308 uv308 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fe8e0017040 con 0x564d4aabad40 2024-09-21T03:41:47.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.891+0000 7fe8ee745f40 1 -- 172.21.15.38:0/972509514 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x564d4aabda40 con 0x564d4aabad40 2024-09-21T03:41:47.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.892+0000 7fe8e81ff640 1 -- 172.21.15.38:0/972509514 <== osd.6 v2:172.21.15.195:6826/4055825675 3 ==== osd_op_reply(34 1.00000000.inode [create] v110'309 uv309 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fe8e0017040 con 0x564d4aabad40 2024-09-21T03:41:47.894 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-09-21T03:41:47.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.892+0000 7fe8ee745f40 1 -- 172.21.15.38:0/972509514 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x564d4aabe290 con 0x564d4aabad40 2024-09-21T03:41:47.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.894+0000 7fe8e81ff640 1 -- 172.21.15.38:0/972509514 <== osd.6 v2:172.21.15.195:6826/4055825675 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v110'310 uv310 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fe8e0017040 con 0x564d4aabad40 2024-09-21T03:41:47.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.894+0000 7fe8ee745f40 1 -- 172.21.15.38:0/972509514 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x564d4aabe630 con 0x564d4aabad40 2024-09-21T03:41:47.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.895+0000 7fe8e81ff640 1 -- 172.21.15.38:0/972509514 <== osd.6 v2:172.21.15.195:6826/4055825675 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v110'311 uv311 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fe8e0017040 con 0x564d4aabad40 2024-09-21T03:41:47.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.895+0000 7fe8ee745f40 1 -- 172.21.15.38:0/972509514 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x564d4aabe9d0 con 0x564d4aabad40 2024-09-21T03:41:47.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.897+0000 7fe8e81ff640 1 -- 172.21.15.38:0/972509514 <== osd.6 v2:172.21.15.195:6826/4055825675 6 ==== osd_op_reply(37 200.00000000 [create] v110'312 uv295 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe8e0017040 con 0x564d4aabad40 2024-09-21T03:41:47.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.897+0000 7fe8ee745f40 1 -- 172.21.15.38:0/972509514 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e110) -- 0x564d4aabed70 con 0x564d4aabad40 2024-09-21T03:41:47.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.898+0000 7fe8e81ff640 1 -- 172.21.15.38:0/972509514 <== osd.6 v2:172.21.15.195:6826/4055825675 7 ==== osd_op_reply(38 200.00000000 [delete] v110'313 uv313 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe8e0017040 con 0x564d4aabad40 2024-09-21T03:41:47.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.898+0000 7fe8ee745f40 1 -- 172.21.15.38:0/972509514 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x564d4aabf110 con 0x564d4aabad40 2024-09-21T03:41:47.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.900+0000 7fe8e81ff640 1 -- 172.21.15.38:0/972509514 <== osd.6 v2:172.21.15.195:6826/4055825675 8 ==== osd_op_reply(39 200.00000000 [create] v110'314 uv314 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe8e0017040 con 0x564d4aabad40 2024-09-21T03:41:47.901 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-09-21T03:41:47.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.900+0000 7fe8ee745f40 1 -- 172.21.15.38:0/972509514 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x564d4aabf110 con 0x564d4aabad40 2024-09-21T03:41:47.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.901+0000 7fe8e81ff640 1 -- 172.21.15.38:0/972509514 <== osd.6 v2:172.21.15.195:6826/4055825675 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v110'315 uv315 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe8e0017040 con 0x564d4aabad40 2024-09-21T03:41:47.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.901+0000 7fe8ee745f40 1 -- 172.21.15.38:0/972509514 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x564d4aabf110 con 0x564d4aabad40 2024-09-21T03:41:47.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.902+0000 7fe8e81ff640 1 -- 172.21.15.38:0/972509514 <== osd.6 v2:172.21.15.195:6826/4055825675 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v110'316 uv316 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe8e0017040 con 0x564d4aabad40 2024-09-21T03:41:47.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.902+0000 7fe8ee745f40 1 -- 172.21.15.38:0/972509514 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:42 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x564d4aabf110 con 0x564d4aabad40 2024-09-21T03:41:47.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.904+0000 7fe8e81ff640 1 -- 172.21.15.38:0/972509514 <== osd.6 v2:172.21.15.195:6826/4055825675 11 ==== osd_op_reply(42 200.00000001 [create] v110'317 uv300 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe8e0017040 con 0x564d4aabad40 2024-09-21T03:41:47.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.904+0000 7fe8ee745f40 1 -- 172.21.15.38:0/972509514 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:43 13.14 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e110) -- 0x564d4aabf110 con 0x564d4aabad40 2024-09-21T03:41:47.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.905+0000 7fe8e81ff640 1 -- 172.21.15.38:0/972509514 <== osd.6 v2:172.21.15.195:6826/4055825675 12 ==== osd_op_reply(43 200.00000001 [delete] v110'318 uv318 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe8e0017040 con 0x564d4aabad40 2024-09-21T03:41:47.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.905+0000 7fe8ee745f40 1 -- 172.21.15.38:0/972509514 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:44 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x564d4aabf110 con 0x564d4aabad40 2024-09-21T03:41:47.908 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-09-21T03:41:47.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.906+0000 7fe8e81ff640 1 -- 172.21.15.38:0/972509514 <== osd.6 v2:172.21.15.195:6826/4055825675 13 ==== osd_op_reply(44 200.00000001 [create] v110'319 uv319 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe8e0017040 con 0x564d4aabad40 2024-09-21T03:41:47.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.906+0000 7fe8ee745f40 1 -- 172.21.15.38:0/972509514 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:45 13.14 13:2e2fa760:::200.00000001:head [write 0~17490 in=17490b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x564d4aac3590 con 0x564d4aabad40 2024-09-21T03:41:47.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.908+0000 7fe8e81ff640 1 -- 172.21.15.38:0/972509514 <== osd.6 v2:172.21.15.195:6826/4055825675 14 ==== osd_op_reply(45 200.00000001 [write 0~17490] v110'320 uv320 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe8e0017040 con 0x564d4aabad40 2024-09-21T03:41:47.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.908+0000 7fe8ee745f40 1 -- 172.21.15.38:0/972509514 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:46 13.14 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x564d4aac3590 con 0x564d4aabad40 2024-09-21T03:41:47.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.909+0000 7fe8e81ff640 1 -- 172.21.15.38:0/972509514 <== osd.6 v2:172.21.15.195:6826/4055825675 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v110'321 uv321 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe8e0017040 con 0x564d4aabad40 2024-09-21T03:41:47.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.909+0000 7fe8ee745f40 1 -- 172.21.15.38:0/972509514 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:47 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x564d4aabf110 con 0x564d4aabad40 2024-09-21T03:41:47.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.910+0000 7fe8e81ff640 1 -- 172.21.15.38:0/972509514 <== osd.6 v2:172.21.15.195:6826/4055825675 16 ==== osd_op_reply(47 606.00000000 [create] v110'322 uv306 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe8e0017040 con 0x564d4aabad40 2024-09-21T03:41:47.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.910+0000 7fe8ee745f40 1 -- 172.21.15.38:0/972509514 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:48 13.14 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e110) -- 0x564d4aabf110 con 0x564d4aabad40 2024-09-21T03:41:47.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.911+0000 7fe8e81ff640 1 -- 172.21.15.38:0/972509514 <== osd.6 v2:172.21.15.195:6826/4055825675 17 ==== osd_op_reply(48 606.00000000 [delete] v110'323 uv323 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe8e0017040 con 0x564d4aabad40 2024-09-21T03:41:47.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.911+0000 7fe8ee745f40 1 -- 172.21.15.38:0/972509514 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:49 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x564d4aabf110 con 0x564d4aabad40 2024-09-21T03:41:47.914 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-09-21T03:41:47.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.913+0000 7fe8e81ff640 1 -- 172.21.15.38:0/972509514 <== osd.6 v2:172.21.15.195:6826/4055825675 18 ==== osd_op_reply(49 606.00000000 [create] v110'324 uv324 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe8e0017040 con 0x564d4aabad40 2024-09-21T03:41:47.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.913+0000 7fe8ee745f40 1 -- 172.21.15.38:0/972509514 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:50 13.14 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x564d4aac0120 con 0x564d4aabad40 2024-09-21T03:41:47.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.914+0000 7fe8e81ff640 1 -- 172.21.15.38:0/972509514 <== osd.6 v2:172.21.15.195:6826/4055825675 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v110'325 uv325 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe8e0017040 con 0x564d4aabad40 2024-09-21T03:41:47.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.914+0000 7fe8ee745f40 1 -- 172.21.15.38:0/972509514 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:51 13.14 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x564d4aac0120 con 0x564d4aabad40 2024-09-21T03:41:47.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.915+0000 7fe8e81ff640 1 -- 172.21.15.38:0/972509514 <== osd.6 v2:172.21.15.195:6826/4055825675 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v110'326 uv326 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe8e0017040 con 0x564d4aabad40 2024-09-21T03:41:47.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.915+0000 7fe8ee745f40 1 -- 172.21.15.38:0/972509514 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:52 13.14 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x564d4aabf110 con 0x564d4aabad40 2024-09-21T03:41:47.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.916+0000 7fe8e81ff640 1 -- 172.21.15.38:0/972509514 <== osd.6 v2:172.21.15.195:6826/4055825675 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v110'327 uv327 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe8e0017040 con 0x564d4aabad40 2024-09-21T03:41:47.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.916+0000 7fe8ee745f40 1 --2- 172.21.15.38:0/972509514 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x564d4aabf620 0x564d4aac1a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:47.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.916+0000 7fe8ee745f40 1 -- 172.21.15.38:0/972509514 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:53 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x564d4aac1f80 con 0x564d4aabf620 2024-09-21T03:41:47.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.916+0000 7fe8e79fe640 1 --2- 172.21.15.38:0/972509514 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x564d4aabf620 0x564d4aac1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:47.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.917+0000 7fe8e79fe640 1 --2- 172.21.15.38:0/972509514 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x564d4aabf620 0x564d4aac1a40 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:47.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.918+0000 7fe8e79fe640 1 -- 172.21.15.38:0/972509514 <== osd.0 v2:172.21.15.38:6800/2381639433 1 ==== osd_op_reply(53 607.00000000 [create] v110'81 uv80 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe8d0056470 con 0x564d4aabf620 2024-09-21T03:41:47.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.918+0000 7fe8ee745f40 1 -- 172.21.15.38:0/972509514 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:54 13.1c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e110) -- 0x564d4aac2000 con 0x564d4aabf620 2024-09-21T03:41:47.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.919+0000 7fe8e79fe640 1 -- 172.21.15.38:0/972509514 <== osd.0 v2:172.21.15.38:6800/2381639433 2 ==== osd_op_reply(54 607.00000000 [delete] v110'82 uv82 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe8d0056470 con 0x564d4aabf620 2024-09-21T03:41:47.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.919+0000 7fe8ee745f40 1 -- 172.21.15.38:0/972509514 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:55 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x564d4aac23f0 con 0x564d4aabf620 2024-09-21T03:41:47.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.921+0000 7fe8e79fe640 1 -- 172.21.15.38:0/972509514 <== osd.0 v2:172.21.15.38:6800/2381639433 3 ==== osd_op_reply(55 607.00000000 [create] v110'83 uv83 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe8d0056470 con 0x564d4aabf620 2024-09-21T03:41:47.923 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-09-21T03:41:47.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.921+0000 7fe8ee745f40 1 -- 172.21.15.38:0/972509514 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:56 13.1c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x564d4aac3400 con 0x564d4aabf620 2024-09-21T03:41:47.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.922+0000 7fe8e79fe640 1 -- 172.21.15.38:0/972509514 <== osd.0 v2:172.21.15.38:6800/2381639433 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v110'84 uv84 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe8d0056470 con 0x564d4aabf620 2024-09-21T03:41:47.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.922+0000 7fe8ee745f40 1 -- 172.21.15.38:0/972509514 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:57 13.1c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x564d4aac3400 con 0x564d4aabf620 2024-09-21T03:41:47.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.923+0000 7fe8e79fe640 1 -- 172.21.15.38:0/972509514 <== osd.0 v2:172.21.15.38:6800/2381639433 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v110'85 uv85 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe8d0056470 con 0x564d4aabf620 2024-09-21T03:41:47.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.923+0000 7fe8ee745f40 1 -- 172.21.15.38:0/972509514 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:58 13.1c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x564d4aac23f0 con 0x564d4aabf620 2024-09-21T03:41:47.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.925+0000 7fe8e79fe640 1 -- 172.21.15.38:0/972509514 <== osd.0 v2:172.21.15.38:6800/2381639433 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v110'86 uv86 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe8d0056470 con 0x564d4aabf620 2024-09-21T03:41:47.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.925+0000 7fe8ee745f40 1 --2- 172.21.15.38:0/972509514 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x564d4aac2a70 0x564d4aac4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:47.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.925+0000 7fe8ee745f40 1 -- 172.21.15.38:0/972509514 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:59 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x564d4aac54a0 con 0x564d4aac2a70 2024-09-21T03:41:47.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.925+0000 7fe8e8a00640 1 --2- 172.21.15.38:0/972509514 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x564d4aac2a70 0x564d4aac4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:47.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.925+0000 7fe8e8a00640 1 --2- 172.21.15.38:0/972509514 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x564d4aac2a70 0x564d4aac4e90 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:47.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.926+0000 7fe8e8a00640 1 -- 172.21.15.38:0/972509514 <== osd.3 v2:172.21.15.38:6808/1286810294 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v110'62 uv61 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7fe8d800f070 con 0x564d4aac2a70 2024-09-21T03:41:47.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.926+0000 7fe8ee745f40 1 -- 172.21.15.38:0/972509514 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:60 13.6 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e110) -- 0x564d4aac59b0 con 0x564d4aac2a70 2024-09-21T03:41:47.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.928+0000 7fe8e8a00640 1 -- 172.21.15.38:0/972509514 <== osd.3 v2:172.21.15.38:6808/1286810294 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v110'63 uv63 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fe8d800f070 con 0x564d4aac2a70 2024-09-21T03:41:47.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.928+0000 7fe8ee745f40 1 -- 172.21.15.38:0/972509514 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:61 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x564d4aac5eb0 con 0x564d4aac2a70 2024-09-21T03:41:47.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.929+0000 7fe8e8a00640 1 -- 172.21.15.38:0/972509514 <== osd.3 v2:172.21.15.38:6808/1286810294 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v110'64 uv64 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fe8d800f070 con 0x564d4aac2a70 2024-09-21T03:41:47.931 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-09-21T03:41:47.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.929+0000 7fe8ee745f40 1 -- 172.21.15.38:0/972509514 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:62 13.6 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x564d4aac6770 con 0x564d4aac2a70 2024-09-21T03:41:47.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.930+0000 7fe8e8a00640 1 -- 172.21.15.38:0/972509514 <== osd.3 v2:172.21.15.38:6808/1286810294 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v110'65 uv65 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fe8d800f070 con 0x564d4aac2a70 2024-09-21T03:41:47.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.930+0000 7fe8ee745f40 1 -- 172.21.15.38:0/972509514 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x564d4aac6b10 con 0x564d4aabf620 2024-09-21T03:41:47.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.931+0000 7fe8e79fe640 1 -- 172.21.15.38:0/972509514 <== osd.0 v2:172.21.15.38:6800/2381639433 7 ==== osd_op_reply(63 608.00000000 [create] v110'81 uv80 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe8d0056470 con 0x564d4aabf620 2024-09-21T03:41:47.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.931+0000 7fe8ee745f40 1 -- 172.21.15.38:0/972509514 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e110) -- 0x564d4aac6b10 con 0x564d4aabf620 2024-09-21T03:41:47.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.932+0000 7fe8e79fe640 1 -- 172.21.15.38:0/972509514 <== osd.0 v2:172.21.15.38:6800/2381639433 8 ==== osd_op_reply(64 608.00000000 [delete] v110'82 uv82 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe8d0056470 con 0x564d4aabf620 2024-09-21T03:41:47.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.932+0000 7fe8ee745f40 1 -- 172.21.15.38:0/972509514 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x564d4aac6b10 con 0x564d4aabf620 2024-09-21T03:41:47.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.933+0000 7fe8e79fe640 1 -- 172.21.15.38:0/972509514 <== osd.0 v2:172.21.15.38:6800/2381639433 9 ==== osd_op_reply(65 608.00000000 [create] v110'83 uv83 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe8d0056470 con 0x564d4aabf620 2024-09-21T03:41:47.935 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-09-21T03:41:47.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.933+0000 7fe8ee745f40 1 -- 172.21.15.38:0/972509514 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x564d4aac7b20 con 0x564d4aabf620 2024-09-21T03:41:47.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.934+0000 7fe8e79fe640 1 -- 172.21.15.38:0/972509514 <== osd.0 v2:172.21.15.38:6800/2381639433 10 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v110'84 uv84 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe8d0056470 con 0x564d4aabf620 2024-09-21T03:41:47.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.934+0000 7fe8ee745f40 1 -- 172.21.15.38:0/972509514 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x564d4aac7b20 con 0x564d4aabf620 2024-09-21T03:41:47.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.935+0000 7fe8e79fe640 1 -- 172.21.15.38:0/972509514 <== osd.0 v2:172.21.15.38:6800/2381639433 11 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v110'85 uv85 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe8d0056470 con 0x564d4aabf620 2024-09-21T03:41:47.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.935+0000 7fe8ee745f40 1 -- 172.21.15.38:0/972509514 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x564d4aac6b10 con 0x564d4aabf620 2024-09-21T03:41:47.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.936+0000 7fe8e79fe640 1 -- 172.21.15.38:0/972509514 <== osd.0 v2:172.21.15.38:6800/2381639433 12 ==== osd_op_reply(68 608.00000000 [omap-set-header] v110'86 uv86 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe8d0056470 con 0x564d4aabf620 2024-09-21T03:41:47.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.937+0000 7fe8ee745f40 1 -- 172.21.15.38:0/972509514 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x564d4aac6b10 con 0x564d4aac2a70 2024-09-21T03:41:47.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.938+0000 7fe8e8a00640 1 -- 172.21.15.38:0/972509514 <== osd.3 v2:172.21.15.38:6808/1286810294 5 ==== osd_op_reply(69 604.00000000 [create] v110'149 uv143 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe8d800f070 con 0x564d4aac2a70 2024-09-21T03:41:47.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.938+0000 7fe8ee745f40 1 -- 172.21.15.38:0/972509514 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e110) -- 0x564d4aac6eb0 con 0x564d4aac2a70 2024-09-21T03:41:47.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.939+0000 7fe8e8a00640 1 -- 172.21.15.38:0/972509514 <== osd.3 v2:172.21.15.38:6808/1286810294 6 ==== osd_op_reply(70 604.00000000 [delete] v110'150 uv150 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe8d800f070 con 0x564d4aac2a70 2024-09-21T03:41:47.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.939+0000 7fe8ee745f40 1 -- 172.21.15.38:0/972509514 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x564d4aac7250 con 0x564d4aac2a70 2024-09-21T03:41:47.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.940+0000 7fe8e8a00640 1 -- 172.21.15.38:0/972509514 <== osd.3 v2:172.21.15.38:6808/1286810294 7 ==== osd_op_reply(71 604.00000000 [create] v110'151 uv151 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe8d800f070 con 0x564d4aac2a70 2024-09-21T03:41:47.942 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-09-21T03:41:47.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.940+0000 7fe8ee745f40 1 -- 172.21.15.38:0/972509514 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x564d4aac8600 con 0x564d4aac2a70 2024-09-21T03:41:47.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.941+0000 7fe8e8a00640 1 -- 172.21.15.38:0/972509514 <== osd.3 v2:172.21.15.38:6808/1286810294 8 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v110'152 uv152 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe8d800f070 con 0x564d4aac2a70 2024-09-21T03:41:47.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.941+0000 7fe8ee745f40 1 -- 172.21.15.38:0/972509514 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x564d4aac8600 con 0x564d4aac2a70 2024-09-21T03:41:47.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.942+0000 7fe8e8a00640 1 -- 172.21.15.38:0/972509514 <== osd.3 v2:172.21.15.38:6808/1286810294 9 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v110'153 uv153 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe8d800f070 con 0x564d4aac2a70 2024-09-21T03:41:47.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.942+0000 7fe8ee745f40 1 -- 172.21.15.38:0/972509514 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x564d4aac75f0 con 0x564d4aac2a70 2024-09-21T03:41:47.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.944+0000 7fe8e8a00640 1 -- 172.21.15.38:0/972509514 <== osd.3 v2:172.21.15.38:6808/1286810294 10 ==== osd_op_reply(74 604.00000000 [omap-set-header] v110'154 uv154 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe8d800f070 con 0x564d4aac2a70 2024-09-21T03:41:47.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.944+0000 7fe8ee745f40 1 -- 172.21.15.38:0/972509514 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x564d4aac75f0 con 0x564d4aac2a70 2024-09-21T03:41:47.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.945+0000 7fe8e8a00640 1 -- 172.21.15.38:0/972509514 <== osd.3 v2:172.21.15.38:6808/1286810294 11 ==== osd_op_reply(75 500.00000000 [create] v110'155 uv148 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe8d800f070 con 0x564d4aac2a70 2024-09-21T03:41:47.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.945+0000 7fe8ee745f40 1 -- 172.21.15.38:0/972509514 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e110) -- 0x564d4aac75f0 con 0x564d4aac2a70 2024-09-21T03:41:47.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.946+0000 7fe8e8a00640 1 -- 172.21.15.38:0/972509514 <== osd.3 v2:172.21.15.38:6808/1286810294 12 ==== osd_op_reply(76 500.00000000 [delete] v110'156 uv156 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe8d800f070 con 0x564d4aac2a70 2024-09-21T03:41:47.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.946+0000 7fe8ee745f40 1 -- 172.21.15.38:0/972509514 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x564d4aac75f0 con 0x564d4aac2a70 2024-09-21T03:41:47.949 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-09-21T03:41:47.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.947+0000 7fe8e8a00640 1 -- 172.21.15.38:0/972509514 <== osd.3 v2:172.21.15.38:6808/1286810294 13 ==== osd_op_reply(77 500.00000000 [create] v110'157 uv157 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe8d800f070 con 0x564d4aac2a70 2024-09-21T03:41:47.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.947+0000 7fe8ee745f40 1 -- 172.21.15.38:0/972509514 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x564d4aac75f0 con 0x564d4aac2a70 2024-09-21T03:41:47.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.948+0000 7fe8e8a00640 1 -- 172.21.15.38:0/972509514 <== osd.3 v2:172.21.15.38:6808/1286810294 14 ==== osd_op_reply(78 500.00000000 [write 0~90] v110'158 uv158 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe8d800f070 con 0x564d4aac2a70 2024-09-21T03:41:47.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.948+0000 7fe8ee745f40 1 -- 172.21.15.38:0/972509514 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x564d4aac75f0 con 0x564d4aac2a70 2024-09-21T03:41:47.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.949+0000 7fe8e8a00640 1 -- 172.21.15.38:0/972509514 <== osd.3 v2:172.21.15.38:6808/1286810294 15 ==== osd_op_reply(79 500.00000000 [omap-set-header] v110'159 uv159 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe8d800f070 con 0x564d4aac2a70 2024-09-21T03:41:47.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.949+0000 7fe8ee745f40 1 -- 172.21.15.38:0/972509514 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:80 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x564d4aac75f0 con 0x564d4aabf620 2024-09-21T03:41:47.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.951+0000 7fe8e79fe640 1 -- 172.21.15.38:0/972509514 <== osd.0 v2:172.21.15.38:6800/2381639433 13 ==== osd_op_reply(80 mds_snaptable [create] v110'66 uv65 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7fe8d0056470 con 0x564d4aabf620 2024-09-21T03:41:47.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.951+0000 7fe8ee745f40 1 -- 172.21.15.38:0/972509514 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:81 13.d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e110) -- 0x564d4aac75f0 con 0x564d4aabf620 2024-09-21T03:41:47.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.952+0000 7fe8e79fe640 1 -- 172.21.15.38:0/972509514 <== osd.0 v2:172.21.15.38:6800/2381639433 14 ==== osd_op_reply(81 mds_snaptable [delete] v110'67 uv67 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fe8d0056470 con 0x564d4aabf620 2024-09-21T03:41:47.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.952+0000 7fe8ee745f40 1 -- 172.21.15.38:0/972509514 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:82 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x564d4aac75f0 con 0x564d4aabf620 2024-09-21T03:41:47.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.953+0000 7fe8e79fe640 1 -- 172.21.15.38:0/972509514 <== osd.0 v2:172.21.15.38:6800/2381639433 15 ==== osd_op_reply(82 mds_snaptable [create] v110'68 uv68 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fe8d0056470 con 0x564d4aabf620 2024-09-21T03:41:47.955 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-09-21T03:41:47.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.953+0000 7fe8ee745f40 1 -- 172.21.15.38:0/972509514 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:83 13.d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x564d4aac75f0 con 0x564d4aabf620 2024-09-21T03:41:47.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.954+0000 7fe8e79fe640 1 -- 172.21.15.38:0/972509514 <== osd.0 v2:172.21.15.38:6800/2381639433 16 ==== osd_op_reply(83 mds_snaptable [write 0~70] v110'69 uv69 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fe8d0056470 con 0x564d4aabf620 2024-09-21T03:41:47.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.954+0000 7fe8ee745f40 1 -- 172.21.15.38:0/972509514 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:84 13.d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x564d4aac75f0 con 0x564d4aabf620 2024-09-21T03:41:47.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.955+0000 7fe8e79fe640 1 -- 172.21.15.38:0/972509514 <== osd.0 v2:172.21.15.38:6800/2381639433 17 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v110'70 uv70 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fe8d0056470 con 0x564d4aabf620 2024-09-21T03:41:47.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.955+0000 7fe8ee745f40 1 --2- 172.21.15.38:0/972509514 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x564d4aac7c70 0x564d4aaca090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:47.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.955+0000 7fe8ee745f40 1 -- 172.21.15.38:0/972509514 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x564d4aaca6a0 con 0x564d4aac7c70 2024-09-21T03:41:47.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.955+0000 7fe8e81ff640 1 --2- 172.21.15.38:0/972509514 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x564d4aac7c70 0x564d4aaca090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:47.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.956+0000 7fe8e81ff640 1 --2- 172.21.15.38:0/972509514 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x564d4aac7c70 0x564d4aaca090 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:47.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.957+0000 7fe8e81ff640 1 -- 172.21.15.38:0/972509514 <== osd.1 v2:172.21.15.38:6816/1491818732 1 ==== osd_op_reply(85 605.00000000 [create] v110'81 uv80 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe8e0017040 con 0x564d4aac7c70 2024-09-21T03:41:47.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.957+0000 7fe8ee745f40 1 -- 172.21.15.38:0/972509514 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e110) -- 0x564d4aacaa70 con 0x564d4aac7c70 2024-09-21T03:41:47.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.958+0000 7fe8e81ff640 1 -- 172.21.15.38:0/972509514 <== osd.1 v2:172.21.15.38:6816/1491818732 2 ==== osd_op_reply(86 605.00000000 [delete] v110'82 uv82 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe8e0017040 con 0x564d4aac7c70 2024-09-21T03:41:47.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.958+0000 7fe8ee745f40 1 -- 172.21.15.38:0/972509514 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x564d4aacaa70 con 0x564d4aac7c70 2024-09-21T03:41:47.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.959+0000 7fe8e81ff640 1 -- 172.21.15.38:0/972509514 <== osd.1 v2:172.21.15.38:6816/1491818732 3 ==== osd_op_reply(87 605.00000000 [create] v110'83 uv83 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe8e0017040 con 0x564d4aac7c70 2024-09-21T03:41:47.961 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-09-21T03:41:47.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.959+0000 7fe8ee745f40 1 -- 172.21.15.38:0/972509514 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x564d4aacba80 con 0x564d4aac7c70 2024-09-21T03:41:47.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.960+0000 7fe8e81ff640 1 -- 172.21.15.38:0/972509514 <== osd.1 v2:172.21.15.38:6816/1491818732 4 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v110'84 uv84 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe8e0017040 con 0x564d4aac7c70 2024-09-21T03:41:47.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.960+0000 7fe8ee745f40 1 -- 172.21.15.38:0/972509514 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x564d4aacba80 con 0x564d4aac7c70 2024-09-21T03:41:47.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.961+0000 7fe8e81ff640 1 -- 172.21.15.38:0/972509514 <== osd.1 v2:172.21.15.38:6816/1491818732 5 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v110'85 uv85 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe8e0017040 con 0x564d4aac7c70 2024-09-21T03:41:47.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.961+0000 7fe8ee745f40 1 -- 172.21.15.38:0/972509514 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x564d4aacaa70 con 0x564d4aac7c70 2024-09-21T03:41:47.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.962+0000 7fe8e81ff640 1 -- 172.21.15.38:0/972509514 <== osd.1 v2:172.21.15.38:6816/1491818732 6 ==== osd_op_reply(90 605.00000000 [omap-set-header] v110'86 uv86 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe8e0017040 con 0x564d4aac7c70 2024-09-21T03:41:47.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.962+0000 7fe8ee745f40 1 -- 172.21.15.38:0/972509514 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x564d4aacaa70 con 0x564d4aac7c70 2024-09-21T03:41:47.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.963+0000 7fe8e81ff640 1 -- 172.21.15.38:0/972509514 <== osd.1 v2:172.21.15.38:6816/1491818732 7 ==== osd_op_reply(91 mds0_inotable [create] v110'132 uv126 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7fe8e0017040 con 0x564d4aac7c70 2024-09-21T03:41:47.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.963+0000 7fe8ee745f40 1 -- 172.21.15.38:0/972509514 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e110) -- 0x564d4aacaa70 con 0x564d4aac7c70 2024-09-21T03:41:47.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.964+0000 7fe8e81ff640 1 -- 172.21.15.38:0/972509514 <== osd.1 v2:172.21.15.38:6816/1491818732 8 ==== osd_op_reply(92 mds0_inotable [delete] v110'133 uv133 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fe8e0017040 con 0x564d4aac7c70 2024-09-21T03:41:47.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.965+0000 7fe8ee745f40 1 -- 172.21.15.38:0/972509514 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x564d4aacaa70 con 0x564d4aac7c70 2024-09-21T03:41:47.967 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-09-21T03:41:47.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.966+0000 7fe8e81ff640 1 -- 172.21.15.38:0/972509514 <== osd.1 v2:172.21.15.38:6816/1491818732 9 ==== osd_op_reply(93 mds0_inotable [create] v110'134 uv134 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fe8e0017040 con 0x564d4aac7c70 2024-09-21T03:41:47.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.966+0000 7fe8ee745f40 1 -- 172.21.15.38:0/972509514 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:94 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x564d4aac6360 con 0x564d4aac7c70 2024-09-21T03:41:47.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.967+0000 7fe8e81ff640 1 -- 172.21.15.38:0/972509514 <== osd.1 v2:172.21.15.38:6816/1491818732 10 ==== osd_op_reply(94 mds0_inotable [write 0~34] v110'135 uv135 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fe8e0017040 con 0x564d4aac7c70 2024-09-21T03:41:47.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.967+0000 7fe8ee745f40 1 -- 172.21.15.38:0/972509514 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:95 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x564d4aac6360 con 0x564d4aac7c70 2024-09-21T03:41:47.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.968+0000 7fe8e81ff640 1 -- 172.21.15.38:0/972509514 <== osd.1 v2:172.21.15.38:6816/1491818732 11 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v110'136 uv136 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fe8e0017040 con 0x564d4aac7c70 2024-09-21T03:41:47.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.968+0000 7fe8ee745f40 1 -- 172.21.15.38:0/972509514 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:96 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x564d4aac6360 con 0x564d4aac7c70 2024-09-21T03:41:47.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.969+0000 7fe8e81ff640 1 -- 172.21.15.38:0/972509514 <== osd.1 v2:172.21.15.38:6816/1491818732 12 ==== osd_op_reply(96 100.00000000 [create] v110'137 uv131 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe8e0017040 con 0x564d4aac7c70 2024-09-21T03:41:47.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.969+0000 7fe8ee745f40 1 -- 172.21.15.38:0/972509514 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:97 13.13 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e110) -- 0x564d4aac6360 con 0x564d4aac7c70 2024-09-21T03:41:47.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.970+0000 7fe8e81ff640 1 -- 172.21.15.38:0/972509514 <== osd.1 v2:172.21.15.38:6816/1491818732 13 ==== osd_op_reply(97 100.00000000 [delete] v110'138 uv138 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe8e0017040 con 0x564d4aac7c70 2024-09-21T03:41:47.972 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.970+0000 7fe8ee745f40 1 -- 172.21.15.38:0/972509514 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:98 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x564d4aac6360 con 0x564d4aac7c70 2024-09-21T03:41:47.972 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.971+0000 7fe8e81ff640 1 -- 172.21.15.38:0/972509514 <== osd.1 v2:172.21.15.38:6816/1491818732 14 ==== osd_op_reply(98 100.00000000 [create] v110'139 uv139 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe8e0017040 con 0x564d4aac7c70 2024-09-21T03:41:47.973 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-09-21T03:41:47.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.971+0000 7fe8ee745f40 1 -- 172.21.15.38:0/972509514 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:99 13.13 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x564d4aac6360 con 0x564d4aac7c70 2024-09-21T03:41:47.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.972+0000 7fe8e81ff640 1 -- 172.21.15.38:0/972509514 <== osd.1 v2:172.21.15.38:6816/1491818732 15 ==== osd_op_reply(99 100.00000000 [omap-set-header] v110'140 uv140 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe8e0017040 con 0x564d4aac7c70 2024-09-21T03:41:47.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.972+0000 7fe8ee745f40 1 -- 172.21.15.38:0/972509514 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:100 13.13 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x564d4aacd920 con 0x564d4aac7c70 2024-09-21T03:41:47.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.973+0000 7fe8e81ff640 1 -- 172.21.15.38:0/972509514 <== osd.1 v2:172.21.15.38:6816/1491818732 16 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v110'141 uv141 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe8e0017040 con 0x564d4aac7c70 2024-09-21T03:41:47.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.973+0000 7fe8ee745f40 1 -- 172.21.15.38:0/972509514 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x564d4aacaa70 con 0x564d4aac2a70 2024-09-21T03:41:47.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.974+0000 7fe8e8a00640 1 -- 172.21.15.38:0/972509514 <== osd.3 v2:172.21.15.38:6808/1286810294 16 ==== osd_op_reply(101 mds0_sessionmap [create] v110'61 uv60 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7fe8d800f070 con 0x564d4aac2a70 2024-09-21T03:41:47.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.974+0000 7fe8ee745f40 1 -- 172.21.15.38:0/972509514 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e110) -- 0x564d4aacaa70 con 0x564d4aac2a70 2024-09-21T03:41:47.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.975+0000 7fe8e8a00640 1 -- 172.21.15.38:0/972509514 <== osd.3 v2:172.21.15.38:6808/1286810294 17 ==== osd_op_reply(102 mds0_sessionmap [delete] v110'62 uv62 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fe8d800f070 con 0x564d4aac2a70 2024-09-21T03:41:47.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.975+0000 7fe8ee745f40 1 -- 172.21.15.38:0/972509514 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:103 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x564d4aacaa70 con 0x564d4aac2a70 2024-09-21T03:41:47.978 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-09-21T03:41:47.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.976+0000 7fe8e8a00640 1 -- 172.21.15.38:0/972509514 <== osd.3 v2:172.21.15.38:6808/1286810294 18 ==== osd_op_reply(103 mds0_sessionmap [create] v110'63 uv63 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fe8d800f070 con 0x564d4aac2a70 2024-09-21T03:41:47.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.976+0000 7fe8ee745f40 1 -- 172.21.15.38:0/972509514 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:104 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x564d4aacaa70 con 0x564d4aac2a70 2024-09-21T03:41:47.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.977+0000 7fe8e8a00640 1 -- 172.21.15.38:0/972509514 <== osd.3 v2:172.21.15.38:6808/1286810294 19 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v110'64 uv64 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fe8d800f070 con 0x564d4aac2a70 2024-09-21T03:41:47.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.977+0000 7fe8ee745f40 1 -- 172.21.15.38:0/972509514 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x564d4aacaa70 con 0x564d4aac2a70 2024-09-21T03:41:47.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.978+0000 7fe8e8a00640 1 -- 172.21.15.38:0/972509514 <== osd.3 v2:172.21.15.38:6808/1286810294 20 ==== osd_op_reply(105 609.00000000 [create] v110'81 uv80 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe8d800f070 con 0x564d4aac2a70 2024-09-21T03:41:47.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.978+0000 7fe8ee745f40 1 -- 172.21.15.38:0/972509514 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e110) -- 0x564d4aacaa70 con 0x564d4aac2a70 2024-09-21T03:41:47.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.979+0000 7fe8e8a00640 1 -- 172.21.15.38:0/972509514 <== osd.3 v2:172.21.15.38:6808/1286810294 21 ==== osd_op_reply(106 609.00000000 [delete] v110'82 uv82 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe8d800f070 con 0x564d4aac2a70 2024-09-21T03:41:47.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.979+0000 7fe8ee745f40 1 -- 172.21.15.38:0/972509514 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x564d4aacaa70 con 0x564d4aac2a70 2024-09-21T03:41:47.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.980+0000 7fe8e8a00640 1 -- 172.21.15.38:0/972509514 <== osd.3 v2:172.21.15.38:6808/1286810294 22 ==== osd_op_reply(107 609.00000000 [create] v110'83 uv83 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe8d800f070 con 0x564d4aac2a70 2024-09-21T03:41:47.982 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-09-21T03:41:47.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.980+0000 7fe8ee745f40 1 -- 172.21.15.38:0/972509514 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x564d4aacaa70 con 0x564d4aac2a70 2024-09-21T03:41:47.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.981+0000 7fe8e8a00640 1 -- 172.21.15.38:0/972509514 <== osd.3 v2:172.21.15.38:6808/1286810294 23 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v110'84 uv84 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe8d800f070 con 0x564d4aac2a70 2024-09-21T03:41:47.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.981+0000 7fe8ee745f40 1 -- 172.21.15.38:0/972509514 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:109 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x564d4aacaa70 con 0x564d4aac2a70 2024-09-21T03:41:47.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.982+0000 7fe8e8a00640 1 -- 172.21.15.38:0/972509514 <== osd.3 v2:172.21.15.38:6808/1286810294 24 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v110'85 uv85 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe8d800f070 con 0x564d4aac2a70 2024-09-21T03:41:47.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.982+0000 7fe8ee745f40 1 -- 172.21.15.38:0/972509514 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:110 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x564d4aacaa70 con 0x564d4aac2a70 2024-09-21T03:41:47.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.983+0000 7fe8e8a00640 1 -- 172.21.15.38:0/972509514 <== osd.3 v2:172.21.15.38:6808/1286810294 25 ==== osd_op_reply(110 609.00000000 [omap-set-header] v110'86 uv86 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe8d800f070 con 0x564d4aac2a70 2024-09-21T03:41:47.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.983+0000 7fe8ee745f40 1 --2- 172.21.15.38:0/972509514 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x564d4aacaf80 0x564d4aacdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:47.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.983+0000 7fe8ee745f40 1 -- 172.21.15.38:0/972509514 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x564d4aacd3a0 con 0x564d4aacaf80 2024-09-21T03:41:47.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.983+0000 7fe8e79fe640 1 --2- 172.21.15.38:0/972509514 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x564d4aacaf80 0x564d4aacdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:47.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.984+0000 7fe8e79fe640 1 --2- 172.21.15.38:0/972509514 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x564d4aacaf80 0x564d4aacdfe0 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:47.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.985+0000 7fe8e79fe640 1 -- 172.21.15.38:0/972509514 <== osd.2 v2:172.21.15.38:6824/2577858902 1 ==== osd_op_reply(111 400.00000000 [create] v110'69 uv68 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe8d0056470 con 0x564d4aacaf80 2024-09-21T03:41:47.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.985+0000 7fe8ee745f40 1 -- 172.21.15.38:0/972509514 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e110) -- 0x564d4aacd3a0 con 0x564d4aacaf80 2024-09-21T03:41:47.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.986+0000 7fe8e79fe640 1 -- 172.21.15.38:0/972509514 <== osd.2 v2:172.21.15.38:6824/2577858902 2 ==== osd_op_reply(112 400.00000000 [delete] v110'70 uv70 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe8d0056470 con 0x564d4aacaf80 2024-09-21T03:41:47.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.986+0000 7fe8ee745f40 1 -- 172.21.15.38:0/972509514 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x564d4aacd460 con 0x564d4aacaf80 2024-09-21T03:41:47.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.987+0000 7fe8e79fe640 1 -- 172.21.15.38:0/972509514 <== osd.2 v2:172.21.15.38:6824/2577858902 3 ==== osd_op_reply(113 400.00000000 [create] v110'71 uv71 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe8d0056470 con 0x564d4aacaf80 2024-09-21T03:41:47.989 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-09-21T03:41:47.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.987+0000 7fe8ee745f40 1 -- 172.21.15.38:0/972509514 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:114 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x564d4aace520 con 0x564d4aacaf80 2024-09-21T03:41:47.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.988+0000 7fe8e79fe640 1 -- 172.21.15.38:0/972509514 <== osd.2 v2:172.21.15.38:6824/2577858902 4 ==== osd_op_reply(114 400.00000000 [write 0~22] v110'72 uv72 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe8d0056470 con 0x564d4aacaf80 2024-09-21T03:41:47.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.988+0000 7fe8ee745f40 1 -- 172.21.15.38:0/972509514 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:115 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x564d4aace520 con 0x564d4aacaf80 2024-09-21T03:41:47.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.989+0000 7fe8e79fe640 1 -- 172.21.15.38:0/972509514 <== osd.2 v2:172.21.15.38:6824/2577858902 5 ==== osd_op_reply(115 400.00000000 [omap-set-header] v110'73 uv73 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe8d0056470 con 0x564d4aacaf80 2024-09-21T03:41:47.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.989+0000 7fe8ee745f40 1 -- 172.21.15.38:0/972509514 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:116 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x564d4aace520 con 0x564d4aac7c70 2024-09-21T03:41:47.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.990+0000 7fe8e81ff640 1 -- 172.21.15.38:0/972509514 <== osd.1 v2:172.21.15.38:6816/1491818732 17 ==== osd_op_reply(116 100.00000000.inode [create] v110'170 uv170 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fe8e0017040 con 0x564d4aac7c70 2024-09-21T03:41:47.992 INFO:teuthology.orchestra.run.smithi038.stdout:Write #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-09-21T03:41:47.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.990+0000 7fe8ee745f40 1 -- 172.21.15.38:0/972509514 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:117 13.1f 13:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x564d4aace520 con 0x564d4aac7c70 2024-09-21T03:41:47.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.992+0000 7fe8e81ff640 1 -- 172.21.15.38:0/972509514 <== osd.1 v2:172.21.15.38:6816/1491818732 18 ==== osd_op_reply(117 100.00000000.inode [write 0~582] v110'171 uv171 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fe8e0017040 con 0x564d4aac7c70 2024-09-21T03:41:47.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.992+0000 7fe8ee745f40 1 -- 172.21.15.38:0/972509514 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:118 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x564d4aace520 con 0x564d4aac7c70 2024-09-21T03:41:47.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.993+0000 7fe8e81ff640 1 -- 172.21.15.38:0/972509514 <== osd.1 v2:172.21.15.38:6816/1491818732 19 ==== osd_op_reply(118 100.00000000.inode [omap-set-header] v110'172 uv172 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fe8e0017040 con 0x564d4aac7c70 2024-09-21T03:41:47.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.993+0000 7fe8ee745f40 1 -- 172.21.15.38:0/972509514 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:119 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x564d4aace520 con 0x564d4aac7c70 2024-09-21T03:41:47.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.994+0000 7fe8e81ff640 1 -- 172.21.15.38:0/972509514 <== osd.1 v2:172.21.15.38:6816/1491818732 20 ==== osd_op_reply(119 1.00000000 [create] v110'173 uv168 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7fe8e0017040 con 0x564d4aac7c70 2024-09-21T03:41:47.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.994+0000 7fe8ee745f40 1 -- 172.21.15.38:0/972509514 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:120 13.1f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e110) -- 0x564d4aace520 con 0x564d4aac7c70 2024-09-21T03:41:47.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.995+0000 7fe8e81ff640 1 -- 172.21.15.38:0/972509514 <== osd.1 v2:172.21.15.38:6816/1491818732 21 ==== osd_op_reply(120 1.00000000 [delete] v110'174 uv174 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fe8e0017040 con 0x564d4aac7c70 2024-09-21T03:41:47.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.995+0000 7fe8ee745f40 1 -- 172.21.15.38:0/972509514 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:121 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x564d4aace520 con 0x564d4aac7c70 2024-09-21T03:41:47.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.996+0000 7fe8e81ff640 1 -- 172.21.15.38:0/972509514 <== osd.1 v2:172.21.15.38:6816/1491818732 22 ==== osd_op_reply(121 1.00000000 [create] v110'175 uv175 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fe8e0017040 con 0x564d4aac7c70 2024-09-21T03:41:47.998 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-09-21T03:41:47.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.996+0000 7fe8ee745f40 1 -- 172.21.15.38:0/972509514 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:122 13.1f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x564d4aacf530 con 0x564d4aac7c70 2024-09-21T03:41:47.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.997+0000 7fe8e81ff640 1 -- 172.21.15.38:0/972509514 <== osd.1 v2:172.21.15.38:6816/1491818732 23 ==== osd_op_reply(122 1.00000000 [setxattr (30)] v110'176 uv176 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fe8e0017040 con 0x564d4aac7c70 2024-09-21T03:41:47.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.997+0000 7fe8ee745f40 1 -- 172.21.15.38:0/972509514 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:123 13.1f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x564d4aacf530 con 0x564d4aac7c70 2024-09-21T03:41:48.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.998+0000 7fe8e81ff640 1 -- 172.21.15.38:0/972509514 <== osd.1 v2:172.21.15.38:6816/1491818732 24 ==== osd_op_reply(123 1.00000000 [setxattr (30)] v110'177 uv177 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fe8e0017040 con 0x564d4aac7c70 2024-09-21T03:41:48.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.998+0000 7fe8ee745f40 1 -- 172.21.15.38:0/972509514 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:124 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x564d4aace520 con 0x564d4aac7c70 2024-09-21T03:41:48.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.999+0000 7fe8e81ff640 1 -- 172.21.15.38:0/972509514 <== osd.1 v2:172.21.15.38:6816/1491818732 25 ==== osd_op_reply(124 1.00000000 [omap-set-header] v110'178 uv178 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fe8e0017040 con 0x564d4aac7c70 2024-09-21T03:41:48.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:47.999+0000 7fe8ee745f40 1 -- 172.21.15.38:0/972509514 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:125 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=584b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x564d4aace9d0 con 0x564d4aac7c70 2024-09-21T03:41:48.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.000+0000 7fe8e81ff640 1 -- 172.21.15.38:0/972509514 <== osd.1 v2:172.21.15.38:6816/1491818732 26 ==== osd_op_reply(125 1.00000000 [omap-set-vals] v110'179 uv179 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fe8e0017040 con 0x564d4aac7c70 2024-09-21T03:41:48.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.000+0000 7fe8ee745f40 1 -- 172.21.15.38:0/972509514 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x564d4aabf620 msgr2=0x564d4aac1a40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:48.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.000+0000 7fe8ee745f40 1 --2- 172.21.15.38:0/972509514 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x564d4aabf620 0x564d4aac1a40 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:48.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.000+0000 7fe8ee745f40 1 -- 172.21.15.38:0/972509514 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x564d4aac7c70 msgr2=0x564d4aaca090 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:48.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.000+0000 7fe8ee745f40 1 --2- 172.21.15.38:0/972509514 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x564d4aac7c70 0x564d4aaca090 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:48.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.000+0000 7fe8ee745f40 1 -- 172.21.15.38:0/972509514 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x564d4aacaf80 msgr2=0x564d4aacdfe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:48.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.000+0000 7fe8ee745f40 1 --2- 172.21.15.38:0/972509514 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x564d4aacaf80 0x564d4aacdfe0 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:48.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.000+0000 7fe8ee745f40 1 -- 172.21.15.38:0/972509514 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x564d4aac2a70 msgr2=0x564d4aac4e90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:48.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.000+0000 7fe8ee745f40 1 --2- 172.21.15.38:0/972509514 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x564d4aac2a70 0x564d4aac4e90 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:48.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.000+0000 7fe8ee745f40 1 -- 172.21.15.38:0/972509514 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x564d4aabad40 msgr2=0x564d4aabd160 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:48.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.000+0000 7fe8e8a00640 1 -- 172.21.15.38:0/972509514 reap_dead start 2024-09-21T03:41:48.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.000+0000 7fe8ee745f40 1 --2- 172.21.15.38:0/972509514 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x564d4aabad40 0x564d4aabd160 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:48.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.001+0000 7fe8e8a00640 1 -- 172.21.15.38:0/972509514 reap_dead start 2024-09-21T03:41:48.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.001+0000 7fe8ee745f40 1 -- 172.21.15.38:0/972509514 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x564d4a9f9d90 msgr2=0x564d4aa0b100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:48.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.001+0000 7fe8ee745f40 1 --2- 172.21.15.38:0/972509514 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x564d4a9f9d90 0x564d4aa0b100 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:48.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.001+0000 7fe8ee745f40 1 -- 172.21.15.38:0/972509514 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fe8c8077570 msgr2=0x7fe8c8079a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:48.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.001+0000 7fe8ee745f40 1 --2- 172.21.15.38:0/972509514 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fe8c8077570 0x7fe8c8079a80 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fe8e00041b0 tx=0x7fe8e000d040 comp rx=0 tx=0).stop 2024-09-21T03:41:48.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.001+0000 7fe8ee745f40 1 -- 172.21.15.38:0/972509514 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564d4aa15670 msgr2=0x564d4aa147c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:48.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.001+0000 7fe8ee745f40 1 --2- 172.21.15.38:0/972509514 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564d4aa15670 0x564d4aa147c0 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7fe8d8009040 tx=0x7fe8d8006ea0 comp rx=0 tx=0).stop 2024-09-21T03:41:48.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.001+0000 7fe8ee745f40 1 -- 172.21.15.38:0/972509514 shutdown_connections 2024-09-21T03:41:48.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.001+0000 7fe8ee745f40 1 --2- 172.21.15.38:0/972509514 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x564d4a9f9d90 0x564d4aa0b100 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:48.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.001+0000 7fe8ee745f40 1 --2- 172.21.15.38:0/972509514 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564d4aa15670 0x564d4aa147c0 unknown :-1 s=CLOSED pgs=1565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:48.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.001+0000 7fe8ee745f40 1 --2- 172.21.15.38:0/972509514 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fe8c8077570 0x7fe8c8079a80 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:48.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.001+0000 7fe8ee745f40 1 -- 172.21.15.38:0/972509514 >> 172.21.15.38:0/972509514 conn(0x564d4a85eac0 msgr2=0x564d4a96c040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:48.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.001+0000 7fe8ee745f40 1 -- 172.21.15.38:0/972509514 shutdown_connections 2024-09-21T03:41:48.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.001+0000 7fe8ee745f40 1 -- 172.21.15.38:0/972509514 wait complete. 2024-09-21T03:41:48.006 DEBUG:teuthology.orchestra.run.smithi038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rm 1.00000000 2024-09-21T03:41:48.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.059+0000 7fe27b387f40 1 Processor -- start 2024-09-21T03:41:48.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.059+0000 7fe27b387f40 1 -- start start 2024-09-21T03:41:48.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.060+0000 7fe27b387f40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f069dcdd00 0x55f069e82210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:48.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.060+0000 7fe27b387f40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f069df9e30 0x55f069dfb490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:48.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.060+0000 7fe27b387f40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f069e88950 0x55f069dfb9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:48.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.060+0000 7fe27b387f40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55f069d1bd60 con 0x55f069df9e30 2024-09-21T03:41:48.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.060+0000 7fe27b387f40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55f069d7a430 con 0x55f069e88950 2024-09-21T03:41:48.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.060+0000 7fe27b387f40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55f069e15f20 con 0x55f069dcdd00 2024-09-21T03:41:48.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.060+0000 7fe274640640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f069df9e30 0x55f069dfb490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:48.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.060+0000 7fe274640640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f069df9e30 0x55f069dfb490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54612/0 (socket says 172.21.15.38:54612) 2024-09-21T03:41:48.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.060+0000 7fe274640640 1 -- 172.21.15.38:0/3372836552 learned_addr learned my addr 172.21.15.38:0/3372836552 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:41:48.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.061+0000 7fe274e41640 1 --2- 172.21.15.38:0/3372836552 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f069dcdd00 0x55f069e82210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:48.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.061+0000 7fe275642640 1 --2- 172.21.15.38:0/3372836552 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f069e88950 0x55f069dfb9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:48.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.061+0000 7fe274640640 1 -- 172.21.15.38:0/3372836552 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f069dcdd00 msgr2=0x55f069e82210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:48.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.061+0000 7fe274640640 1 --2- 172.21.15.38:0/3372836552 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f069dcdd00 0x55f069e82210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:48.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.061+0000 7fe274640640 1 -- 172.21.15.38:0/3372836552 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f069e88950 msgr2=0x55f069dfb9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:48.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.061+0000 7fe274640640 1 --2- 172.21.15.38:0/3372836552 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f069e88950 0x55f069dfb9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:48.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.061+0000 7fe274640640 1 -- 172.21.15.38:0/3372836552 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f069ea3bd0 con 0x55f069df9e30 2024-09-21T03:41:48.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.061+0000 7fe274640640 1 --2- 172.21.15.38:0/3372836552 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f069df9e30 0x55f069dfb490 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7fe268004e90 tx=0x7fe268002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2cf523c6705d1a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:48.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.061+0000 7fe27363e640 1 -- 172.21.15.38:0/3372836552 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe268056670 con 0x55f069df9e30 2024-09-21T03:41:48.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.061+0000 7fe27363e640 1 -- 172.21.15.38:0/3372836552 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe268056830 con 0x55f069df9e30 2024-09-21T03:41:48.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.061+0000 7fe27363e640 1 -- 172.21.15.38:0/3372836552 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe268056c80 con 0x55f069df9e30 2024-09-21T03:41:48.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.062+0000 7fe27b387f40 1 -- 172.21.15.38:0/3372836552 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f069df9e30 msgr2=0x55f069dfb490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:48.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.062+0000 7fe27b387f40 1 --2- 172.21.15.38:0/3372836552 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f069df9e30 0x55f069dfb490 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7fe268004e90 tx=0x7fe268002fb0 comp rx=0 tx=0).stop 2024-09-21T03:41:48.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.062+0000 7fe27b387f40 1 -- 172.21.15.38:0/3372836552 shutdown_connections 2024-09-21T03:41:48.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.062+0000 7fe27b387f40 1 --2- 172.21.15.38:0/3372836552 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f069e88950 0x55f069dfb9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:48.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.062+0000 7fe27b387f40 1 --2- 172.21.15.38:0/3372836552 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f069df9e30 0x55f069dfb490 unknown :-1 s=CLOSED pgs=1568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:48.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.062+0000 7fe27b387f40 1 --2- 172.21.15.38:0/3372836552 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f069dcdd00 0x55f069e82210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:48.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.062+0000 7fe27b387f40 1 -- 172.21.15.38:0/3372836552 >> 172.21.15.38:0/3372836552 conn(0x55f069cecac0 msgr2=0x55f069e9f750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:48.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.062+0000 7fe27b387f40 1 -- 172.21.15.38:0/3372836552 shutdown_connections 2024-09-21T03:41:48.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.062+0000 7fe27b387f40 1 -- 172.21.15.38:0/3372836552 wait complete. 2024-09-21T03:41:48.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.063+0000 7fe27b387f40 1 Processor -- start 2024-09-21T03:41:48.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.063+0000 7fe27b387f40 1 -- start start 2024-09-21T03:41:48.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.064+0000 7fe27b387f40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f069dcdd00 0x55f069e9cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:48.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.064+0000 7fe27b387f40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f069df9e30 0x55f069e9e120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:48.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.064+0000 7fe27b387f40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f069e88950 0x55f069e9e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:48.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.064+0000 7fe27b387f40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55f069ea3350 con 0x55f069df9e30 2024-09-21T03:41:48.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.064+0000 7fe27b387f40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55f069ea35f0 con 0x55f069e88950 2024-09-21T03:41:48.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.064+0000 7fe27b387f40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55f069e84b90 con 0x55f069dcdd00 2024-09-21T03:41:48.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.064+0000 7fe274640640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f069df9e30 0x55f069e9e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:48.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.064+0000 7fe274640640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f069df9e30 0x55f069e9e120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54616/0 (socket says 172.21.15.38:54616) 2024-09-21T03:41:48.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.064+0000 7fe274640640 1 -- 172.21.15.38:0/587307580 learned_addr learned my addr 172.21.15.38:0/587307580 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:41:48.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.064+0000 7fe274e41640 1 --2- 172.21.15.38:0/587307580 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f069dcdd00 0x55f069e9cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:48.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.064+0000 7fe275642640 1 --2- 172.21.15.38:0/587307580 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f069e88950 0x55f069e9e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:48.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.064+0000 7fe274640640 1 -- 172.21.15.38:0/587307580 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f069dcdd00 msgr2=0x55f069e9cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:48.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.064+0000 7fe274640640 1 --2- 172.21.15.38:0/587307580 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f069dcdd00 0x55f069e9cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:48.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.064+0000 7fe274640640 1 -- 172.21.15.38:0/587307580 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f069e88950 msgr2=0x55f069e9e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:48.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.064+0000 7fe274640640 1 --2- 172.21.15.38:0/587307580 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f069e88950 0x55f069e9e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:48.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.064+0000 7fe274640640 1 -- 172.21.15.38:0/587307580 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f069e83dd0 con 0x55f069df9e30 2024-09-21T03:41:48.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.064+0000 7fe274e41640 1 --2- 172.21.15.38:0/587307580 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f069dcdd00 0x55f069e9cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:41:48.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.064+0000 7fe275642640 1 --2- 172.21.15.38:0/587307580 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f069e88950 0x55f069e9e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:41:48.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.065+0000 7fe274640640 1 --2- 172.21.15.38:0/587307580 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f069df9e30 0x55f069e9e120 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7fe268025f60 tx=0x7fe268057740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:48.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.065+0000 7fe27263c640 1 -- 172.21.15.38:0/587307580 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe26806f030 con 0x55f069df9e30 2024-09-21T03:41:48.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.065+0000 7fe27263c640 1 -- 172.21.15.38:0/587307580 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe268057a00 con 0x55f069df9e30 2024-09-21T03:41:48.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.065+0000 7fe27b387f40 1 -- 172.21.15.38:0/587307580 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55f069e840b0 con 0x55f069df9e30 2024-09-21T03:41:48.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.065+0000 7fe27263c640 1 -- 172.21.15.38:0/587307580 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe268067040 con 0x55f069df9e30 2024-09-21T03:41:48.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.065+0000 7fe27b387f40 1 -- 172.21.15.38:0/587307580 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f069e12c90 con 0x55f069df9e30 2024-09-21T03:41:48.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.066+0000 7fe27263c640 1 -- 172.21.15.38:0/587307580 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fe268072070 con 0x55f069df9e30 2024-09-21T03:41:48.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.067+0000 7fe27263c640 1 --2- 172.21.15.38:0/587307580 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fe2600775a0 0x7fe260079ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:48.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.067+0000 7fe27263c640 1 -- 172.21.15.38:0/587307580 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 7508+0+0 (secure 0 0 0) 0x7fe2680df040 con 0x55f069df9e30 2024-09-21T03:41:48.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.067+0000 7fe274e41640 1 --2- 172.21.15.38:0/587307580 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fe2600775a0 0x7fe260079ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:48.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.067+0000 7fe27b387f40 1 --2- 172.21.15.38:0/587307580 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x55f069f38a10 0x55f069e9bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:48.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.067+0000 7fe27b387f40 1 -- 172.21.15.38:0/587307580 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:1 13.1f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e110) -- 0x55f069e9c160 con 0x55f069f38a10 2024-09-21T03:41:48.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.067+0000 7fe275642640 1 --2- 172.21.15.38:0/587307580 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x55f069f38a10 0x55f069e9bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:48.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.068+0000 7fe274e41640 1 --2- 172.21.15.38:0/587307580 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fe2600775a0 0x7fe260079ab0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fe26400f460 tx=0x7fe2640025c0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:48.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.068+0000 7fe275642640 1 --2- 172.21.15.38:0/587307580 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x55f069f38a10 0x55f069e9bc20 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:48.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.069+0000 7fe275642640 1 -- 172.21.15.38:0/587307580 <== osd.1 v2:172.21.15.38:6816/1491818732 1 ==== osd_op_reply(1 1.00000000 [delete] v110'180 uv180 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fe26c00e900 con 0x55f069f38a10 2024-09-21T03:41:48.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.069+0000 7fe27b387f40 1 -- 172.21.15.38:0/587307580 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x55f069f38a10 msgr2=0x55f069e9bc20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:48.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.069+0000 7fe27b387f40 1 --2- 172.21.15.38:0/587307580 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x55f069f38a10 0x55f069e9bc20 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:48.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.069+0000 7fe27b387f40 1 -- 172.21.15.38:0/587307580 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fe2600775a0 msgr2=0x7fe260079ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:48.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.069+0000 7fe27b387f40 1 --2- 172.21.15.38:0/587307580 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fe2600775a0 0x7fe260079ab0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fe26400f460 tx=0x7fe2640025c0 comp rx=0 tx=0).stop 2024-09-21T03:41:48.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.069+0000 7fe27b387f40 1 -- 172.21.15.38:0/587307580 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f069df9e30 msgr2=0x55f069e9e120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:48.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.069+0000 7fe27b387f40 1 --2- 172.21.15.38:0/587307580 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f069df9e30 0x55f069e9e120 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7fe268025f60 tx=0x7fe268057740 comp rx=0 tx=0).stop 2024-09-21T03:41:48.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.069+0000 7fe275642640 1 -- 172.21.15.38:0/587307580 reap_dead start 2024-09-21T03:41:48.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.069+0000 7fe27b387f40 1 -- 172.21.15.38:0/587307580 shutdown_connections 2024-09-21T03:41:48.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.069+0000 7fe27b387f40 1 -- 172.21.15.38:0/587307580 >> 172.21.15.38:0/587307580 conn(0x55f069cecac0 msgr2=0x55f069e82210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:48.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.069+0000 7fe27b387f40 1 -- 172.21.15.38:0/587307580 shutdown_connections 2024-09-21T03:41:48.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.069+0000 7fe27b387f40 1 -- 172.21.15.38:0/587307580 wait complete. 2024-09-21T03:41:48.074 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-21T03:41:48.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.192+0000 7f5194607640 1 -- 172.21.15.38:0/3473023202 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5178003fd0 msgr2=0x7f5178004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:48.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.192+0000 7f5194607640 1 --2- 172.21.15.38:0/3473023202 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5178003fd0 0x7f5178004800 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7f517c00c260 tx=0x7f5178005830 comp rx=0 tx=0).stop 2024-09-21T03:41:48.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.192+0000 7f5192ba4640 1 -- 172.21.15.38:0/3473023202 reap_dead start 2024-09-21T03:41:48.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.192+0000 7f5194607640 1 -- 172.21.15.38:0/3473023202 shutdown_connections 2024-09-21T03:41:48.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.192+0000 7f5194607640 1 -- 172.21.15.38:0/3473023202 >> 172.21.15.38:0/3473023202 conn(0x7f518c0cdca0 msgr2=0x7f518c0ce0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:48.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.192+0000 7f5194607640 1 -- 172.21.15.38:0/3473023202 shutdown_connections 2024-09-21T03:41:48.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.192+0000 7f5194607640 1 -- 172.21.15.38:0/3473023202 wait complete. 2024-09-21T03:41:48.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.194+0000 7f5194607640 1 Processor -- start 2024-09-21T03:41:48.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.194+0000 7f5194607640 1 -- start start 2024-09-21T03:41:48.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.195+0000 7f5194607640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5178003fd0 0x7f518c1530b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:48.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.195+0000 7f5194607640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f518c15a550 0x7f518c151530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:48.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.195+0000 7f5194607640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f518c1f88f0 0x7f518c1460e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:48.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.195+0000 7f5194607640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f518c06f7a0 con 0x7f518c1f88f0 2024-09-21T03:41:48.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.195+0000 7f5194607640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f518c06f620 con 0x7f518c15a550 2024-09-21T03:41:48.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.196+0000 7f5194607640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f518c15fb70 con 0x7f5178003fd0 2024-09-21T03:41:48.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.196+0000 7f5192ba4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f518c1f88f0 0x7f518c1460e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:48.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.196+0000 7f5192ba4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f518c1f88f0 0x7f518c1460e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54632/0 (socket says 172.21.15.38:54632) 2024-09-21T03:41:48.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.196+0000 7f5192ba4640 1 -- 172.21.15.38:0/2729924167 learned_addr learned my addr 172.21.15.38:0/2729924167 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:41:48.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.196+0000 7f51923a3640 1 --2- 172.21.15.38:0/2729924167 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5178003fd0 0x7f518c1530b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:48.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.196+0000 7f5191ba2640 1 --2- 172.21.15.38:0/2729924167 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f518c15a550 0x7f518c151530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:48.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.196+0000 7f5192ba4640 1 -- 172.21.15.38:0/2729924167 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5178003fd0 msgr2=0x7f518c1530b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:48.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.196+0000 7f5192ba4640 1 --2- 172.21.15.38:0/2729924167 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5178003fd0 0x7f518c1530b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:48.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.196+0000 7f5192ba4640 1 -- 172.21.15.38:0/2729924167 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f518c15a550 msgr2=0x7f518c151530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:48.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.196+0000 7f5192ba4640 1 --2- 172.21.15.38:0/2729924167 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f518c15a550 0x7f518c151530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:48.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.196+0000 7f5192ba4640 1 -- 172.21.15.38:0/2729924167 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f518c15ad60 con 0x7f518c1f88f0 2024-09-21T03:41:48.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.196+0000 7f5192ba4640 1 --2- 172.21.15.38:0/2729924167 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f518c1f88f0 0x7f518c1460e0 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f518c0b0c30 tx=0x7f518400e350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:48.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.196+0000 7f5182ffd640 1 -- 172.21.15.38:0/2729924167 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5184003970 con 0x7f518c1f88f0 2024-09-21T03:41:48.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.196+0000 7f5182ffd640 1 -- 172.21.15.38:0/2729924167 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5184015070 con 0x7f518c1f88f0 2024-09-21T03:41:48.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.196+0000 7f5182ffd640 1 -- 172.21.15.38:0/2729924167 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5184004030 con 0x7f518c1f88f0 2024-09-21T03:41:48.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.196+0000 7f5194607640 1 -- 172.21.15.38:0/2729924167 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f518c15b040 con 0x7f518c1f88f0 2024-09-21T03:41:48.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.196+0000 7f5194607640 1 -- 172.21.15.38:0/2729924167 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f518c1458e0 con 0x7f518c1f88f0 2024-09-21T03:41:48.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.198+0000 7f5194607640 1 -- 172.21.15.38:0/2729924167 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f518c15f630 con 0x7f518c1f88f0 2024-09-21T03:41:48.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.198+0000 7f5182ffd640 1 -- 172.21.15.38:0/2729924167 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f51840041d0 con 0x7f518c1f88f0 2024-09-21T03:41:48.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.199+0000 7f5182ffd640 1 --2- 172.21.15.38:0/2729924167 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5170078bc0 0x7f517007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:48.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.199+0000 7f5182ffd640 1 -- 172.21.15.38:0/2729924167 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 7508+0+0 (secure 0 0 0) 0x7f518409be90 con 0x7f518c1f88f0 2024-09-21T03:41:48.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.199+0000 7f51923a3640 1 --2- 172.21.15.38:0/2729924167 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5170078bc0 0x7f517007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:48.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.200+0000 7f51923a3640 1 --2- 172.21.15.38:0/2729924167 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5170078bc0 0x7f517007b0d0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f518c15efd0 tx=0x7f5188005f90 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:48.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.203+0000 7f5182ffd640 1 -- 172.21.15.38:0/2729924167 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f5184063610 con 0x7f518c1f88f0 2024-09-21T03:41:48.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.428+0000 7f5194607640 1 -- 172.21.15.38:0/2729924167 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f518c155b50 con 0x7f518c1f88f0 2024-09-21T03:41:48.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.756+0000 7f5182ffd640 1 -- 172.21.15.38:0/2729924167 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v169) ==== 162+0+0 (secure 0 0 0) 0x7f5184068040 con 0x7f518c1f88f0 2024-09-21T03:41:48.758 INFO:teuthology.orchestra.run.smithi038.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-21T03:41:48.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.758+0000 7f5194607640 1 -- 172.21.15.38:0/2729924167 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5170078bc0 msgr2=0x7f517007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:48.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.758+0000 7f5194607640 1 --2- 172.21.15.38:0/2729924167 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5170078bc0 0x7f517007b0d0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f518c15efd0 tx=0x7f5188005f90 comp rx=0 tx=0).stop 2024-09-21T03:41:48.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.758+0000 7f5194607640 1 -- 172.21.15.38:0/2729924167 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f518c1f88f0 msgr2=0x7f518c1460e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:48.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.758+0000 7f5194607640 1 --2- 172.21.15.38:0/2729924167 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f518c1f88f0 0x7f518c1460e0 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f518c0b0c30 tx=0x7f518400e350 comp rx=0 tx=0).stop 2024-09-21T03:41:48.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.758+0000 7f5194607640 1 -- 172.21.15.38:0/2729924167 shutdown_connections 2024-09-21T03:41:48.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.758+0000 7f5194607640 1 --2- 172.21.15.38:0/2729924167 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f518c1f88f0 0x7f518c1460e0 unknown :-1 s=CLOSED pgs=1572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:48.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.758+0000 7f5194607640 1 --2- 172.21.15.38:0/2729924167 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f518c15a550 0x7f518c151530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:48.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.758+0000 7f5194607640 1 --2- 172.21.15.38:0/2729924167 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5170078bc0 0x7f517007b0d0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:48.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.758+0000 7f5194607640 1 --2- 172.21.15.38:0/2729924167 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5178003fd0 0x7f518c1530b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:48.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.758+0000 7f5194607640 1 -- 172.21.15.38:0/2729924167 >> 172.21.15.38:0/2729924167 conn(0x7f518c0cdca0 msgr2=0x7f518c073cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:48.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.759+0000 7f5194607640 1 -- 172.21.15.38:0/2729924167 shutdown_connections 2024-09-21T03:41:48.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.759+0000 7f5194607640 1 -- 172.21.15.38:0/2729924167 wait complete. 2024-09-21T03:41:48.775 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:41:48.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.891+0000 7f6a115c6640 1 -- 172.21.15.38:0/4001845210 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6a0c1536e0 msgr2=0x7f6a0c0d17c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:48.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.891+0000 7f6a115c6640 1 --2- 172.21.15.38:0/4001845210 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6a0c1536e0 0x7f6a0c0d17c0 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7f69fc005fd0 tx=0x7f69fc02a940 comp rx=0 tx=0).stop 2024-09-21T03:41:48.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.891+0000 7f6a115c6640 1 -- 172.21.15.38:0/4001845210 shutdown_connections 2024-09-21T03:41:48.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.891+0000 7f6a115c6640 1 --2- 172.21.15.38:0/4001845210 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6a0c1536e0 0x7f6a0c0d17c0 unknown :-1 s=CLOSED pgs=1574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:48.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.891+0000 7f6a115c6640 1 -- 172.21.15.38:0/4001845210 >> 172.21.15.38:0/4001845210 conn(0x7f6a0c0cef40 msgr2=0x7f6a0c0cf350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:48.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.891+0000 7f6a115c6640 1 -- 172.21.15.38:0/4001845210 shutdown_connections 2024-09-21T03:41:48.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.891+0000 7f6a115c6640 1 -- 172.21.15.38:0/4001845210 wait complete. 2024-09-21T03:41:48.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.893+0000 7f6a115c6640 1 Processor -- start 2024-09-21T03:41:48.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.893+0000 7f6a115c6640 1 -- start start 2024-09-21T03:41:48.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.894+0000 7f6a115c6640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6a0c0d1d00 0x7f6a0c1f0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:48.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.894+0000 7f6a115c6640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6a0c1536e0 0x7f6a0c1f37b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:48.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.894+0000 7f6a115c6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6a0c1f6400 0x7f6a0c1f6810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:48.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.894+0000 7f6a115c6640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f6a0c0d0f60 con 0x7f6a0c1f6400 2024-09-21T03:41:48.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.894+0000 7f6a115c6640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f6a0c0d0de0 con 0x7f6a0c1536e0 2024-09-21T03:41:48.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.894+0000 7f6a115c6640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f6a0c0d10e0 con 0x7f6a0c0d1d00 2024-09-21T03:41:48.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.894+0000 7f6a0b7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6a0c1f6400 0x7f6a0c1f6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:48.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.895+0000 7f6a0b7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6a0c1f6400 0x7f6a0c1f6810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54664/0 (socket says 172.21.15.38:54664) 2024-09-21T03:41:48.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.895+0000 7f6a0b7fe640 1 -- 172.21.15.38:0/3695061705 learned_addr learned my addr 172.21.15.38:0/3695061705 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:41:48.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.895+0000 7f6a0a7fc640 1 --2- 172.21.15.38:0/3695061705 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6a0c1536e0 0x7f6a0c1f37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:48.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.895+0000 7f6a0affd640 1 --2- 172.21.15.38:0/3695061705 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6a0c0d1d00 0x7f6a0c1f0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:48.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.895+0000 7f6a0b7fe640 1 -- 172.21.15.38:0/3695061705 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6a0c0d1d00 msgr2=0x7f6a0c1f0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:48.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.895+0000 7f6a0b7fe640 1 --2- 172.21.15.38:0/3695061705 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6a0c0d1d00 0x7f6a0c1f0280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:48.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.895+0000 7f6a0b7fe640 1 -- 172.21.15.38:0/3695061705 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6a0c1536e0 msgr2=0x7f6a0c1f37b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:48.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.895+0000 7f6a0b7fe640 1 --2- 172.21.15.38:0/3695061705 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6a0c1536e0 0x7f6a0c1f37b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:48.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.895+0000 7f6a0b7fe640 1 -- 172.21.15.38:0/3695061705 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a0c1f6d50 con 0x7f6a0c1f6400 2024-09-21T03:41:48.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.895+0000 7f6a0a7fc640 1 --2- 172.21.15.38:0/3695061705 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6a0c1536e0 0x7f6a0c1f37b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:41:48.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.895+0000 7f6a0b7fe640 1 --2- 172.21.15.38:0/3695061705 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6a0c1f6400 0x7f6a0c1f6810 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f6a0400c3f0 tx=0x7f6a0400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:48.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.895+0000 7f69f37fe640 1 -- 172.21.15.38:0/3695061705 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a04017070 con 0x7f6a0c1f6400 2024-09-21T03:41:48.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.895+0000 7f69f37fe640 1 -- 172.21.15.38:0/3695061705 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6a040035f0 con 0x7f6a0c1f6400 2024-09-21T03:41:48.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.895+0000 7f69f37fe640 1 -- 172.21.15.38:0/3695061705 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a04003a40 con 0x7f6a0c1f6400 2024-09-21T03:41:48.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.895+0000 7f6a115c6640 1 -- 172.21.15.38:0/3695061705 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a0c1502c0 con 0x7f6a0c1f6400 2024-09-21T03:41:48.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.895+0000 7f6a115c6640 1 -- 172.21.15.38:0/3695061705 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a0c150840 con 0x7f6a0c1f6400 2024-09-21T03:41:48.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.896+0000 7f6a115c6640 1 -- 172.21.15.38:0/3695061705 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a0c156cb0 con 0x7f6a0c1f6400 2024-09-21T03:41:48.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.897+0000 7f69f37fe640 1 -- 172.21.15.38:0/3695061705 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f6a0401a070 con 0x7f6a0c1f6400 2024-09-21T03:41:48.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.898+0000 7f69f37fe640 1 --2- 172.21.15.38:0/3695061705 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f69f8078bc0 0x7f69f807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:48.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.898+0000 7f69f37fe640 1 -- 172.21.15.38:0/3695061705 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 7508+0+0 (secure 0 0 0) 0x7f6a0409a700 con 0x7f6a0c1f6400 2024-09-21T03:41:48.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.898+0000 7f6a0affd640 1 --2- 172.21.15.38:0/3695061705 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f69f8078bc0 0x7f69f807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:48.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.899+0000 7f6a0affd640 1 --2- 172.21.15.38:0/3695061705 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f69f8078bc0 0x7f69f807b0d0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f69fc034000 tx=0x7f69fc004ab0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:48.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:48.902+0000 7f69f37fe640 1 -- 172.21.15.38:0/3695061705 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f6a0c1f6d50 con 0x7f6a0c1f6400 2024-09-21T03:41:49.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:49.129+0000 7f6a115c6640 1 -- 172.21.15.38:0/3695061705 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6a0c0b5640 con 0x7f6a0c1f6400 2024-09-21T03:41:49.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:49.130+0000 7f69f37fe640 1 -- 172.21.15.38:0/3695061705 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 170 v170) ==== 95+0+4401 (secure 0 0 0) 0x7f6a04062390 con 0x7f6a0c1f6400 2024-09-21T03:41:49.132 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:41:49.132 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":170,"btime":"2024-09-21T03:41:48:759682+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6086,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/965972663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":965972663},{"type":"v1","addr":"172.21.15.195:6836","nonce":965972663}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6700,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/3176122334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":3176122334},{"type":"v1","addr":"172.21.15.195:6837","nonce":3176122334}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":170,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:40:02.319282+0000","modified":"2024-09-21T03:41:48.759654+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":110,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8505},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8505":{"gid":8505,"name":"c","rank":0,"incarnation":170,"state":"up:replay","state_seq":1,"addr":"172.21.15.38:6833/1717955365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":1717955365},{"type":"v1","addr":"172.21.15.38:6833","nonce":1717955365}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-21T03:41:49.132 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 170 2024-09-21T03:41:49.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:49.132+0000 7f6a115c6640 1 -- 172.21.15.38:0/3695061705 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f69f8078bc0 msgr2=0x7f69f807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:49.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:49.132+0000 7f6a115c6640 1 --2- 172.21.15.38:0/3695061705 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f69f8078bc0 0x7f69f807b0d0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f69fc034000 tx=0x7f69fc004ab0 comp rx=0 tx=0).stop 2024-09-21T03:41:49.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:49.132+0000 7f6a115c6640 1 -- 172.21.15.38:0/3695061705 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6a0c1f6400 msgr2=0x7f6a0c1f6810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:49.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:49.132+0000 7f6a115c6640 1 --2- 172.21.15.38:0/3695061705 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6a0c1f6400 0x7f6a0c1f6810 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f6a0400c3f0 tx=0x7f6a0400c8b0 comp rx=0 tx=0).stop 2024-09-21T03:41:49.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:49.132+0000 7f6a115c6640 1 -- 172.21.15.38:0/3695061705 shutdown_connections 2024-09-21T03:41:49.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:49.132+0000 7f6a115c6640 1 --2- 172.21.15.38:0/3695061705 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6a0c1f6400 0x7f6a0c1f6810 unknown :-1 s=CLOSED pgs=1575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:49.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:49.132+0000 7f6a115c6640 1 --2- 172.21.15.38:0/3695061705 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6a0c1536e0 0x7f6a0c1f37b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:49.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:49.132+0000 7f6a115c6640 1 --2- 172.21.15.38:0/3695061705 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f69f8078bc0 0x7f69f807b0d0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:49.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:49.132+0000 7f6a115c6640 1 --2- 172.21.15.38:0/3695061705 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6a0c0d1d00 0x7f6a0c1f0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:49.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:49.132+0000 7f6a115c6640 1 -- 172.21.15.38:0/3695061705 >> 172.21.15.38:0/3695061705 conn(0x7f6a0c0cef40 msgr2=0x7f6a0c0b3ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:49.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:49.132+0000 7f6a115c6640 1 -- 172.21.15.38:0/3695061705 shutdown_connections 2024-09-21T03:41:49.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:49.132+0000 7f6a115c6640 1 -- 172.21.15.38:0/3695061705 wait complete. 2024-09-21T03:41:49.145 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2024-09-21T03:41:49.490 INFO:tasks.ceph.mds.a.smithi038.stderr: -48> 2024-09-21T03:41:44.735+0000 7fe6e6b6f640 -1 mds.0.cache.ino(0x100) No data while reading inode 0x100 2024-09-21T03:41:49.490 INFO:tasks.ceph.mds.a.smithi038.stderr: -38> 2024-09-21T03:41:44.736+0000 7fe6e6b6f640 -1 log_channel(cluster) log [ERR] : Error loading MDS rank 0: (2) No such file or directory 2024-09-21T03:41:49.547 INFO:tasks.ceph.mds.a.smithi038.stdout:starting mds.a at 2024-09-21T03:41:51.787 INFO:tasks.ceph.mds.c.smithi038.stderr:2024-09-21T03:41:51.784+0000 7fc3eb90e640 -1 log_channel(cluster) log [ERR] : dir 0x1 object missing on disk; some files may be lost () 2024-09-21T03:41:51.787 INFO:tasks.ceph.mds.c.smithi038.stderr:2024-09-21T03:41:51.784+0000 7fc3eb90e640 -1 mds.0.damage notify_dirfrag Damage to fragment * of ino 0x1 is fatal because it is a system directory for this rank 2024-09-21T03:41:51.789 INFO:tasks.ceph.mon.a.smithi038.stderr:2024-09-21T03:41:51.786+0000 7f3f45713640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T03:41:51.789 INFO:tasks.ceph.mon.a.smithi038.stderr:2024-09-21T03:41:51.786+0000 7f3f45713640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-09-21T03:41:54.146 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:41:54.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:54.276+0000 7f372b43f640 1 -- 172.21.15.38:0/2416126323 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f37240d3a70 msgr2=0x7f37240d00d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:54.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:54.276+0000 7f372b43f640 1 --2- 172.21.15.38:0/2416126323 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f37240d3a70 0x7f37240d00d0 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7f371c005fd0 tx=0x7f371c02a8b0 comp rx=0 tx=0).stop 2024-09-21T03:41:54.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:54.276+0000 7f372b43f640 1 -- 172.21.15.38:0/2416126323 shutdown_connections 2024-09-21T03:41:54.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:54.276+0000 7f372b43f640 1 --2- 172.21.15.38:0/2416126323 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f37240d3a70 0x7f37240d00d0 unknown :-1 s=CLOSED pgs=1580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:54.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:54.276+0000 7f372b43f640 1 -- 172.21.15.38:0/2416126323 >> 172.21.15.38:0/2416126323 conn(0x7f37240d4620 msgr2=0x7f37240d4a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:54.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:54.276+0000 7f372b43f640 1 -- 172.21.15.38:0/2416126323 shutdown_connections 2024-09-21T03:41:54.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:54.276+0000 7f372b43f640 1 -- 172.21.15.38:0/2416126323 wait complete. 2024-09-21T03:41:54.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:54.278+0000 7f372b43f640 1 Processor -- start 2024-09-21T03:41:54.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:54.278+0000 7f372b43f640 1 -- start start 2024-09-21T03:41:54.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:54.279+0000 7f372b43f640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f37240cea90 0x7f372414b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:54.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:54.279+0000 7f372b43f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f37240d3a70 0x7f372415dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:54.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:54.279+0000 7f372b43f640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f372415a100 0x7f372415e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:54.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:54.279+0000 7f372b43f640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f37240b4360 con 0x7f37240d3a70 2024-09-21T03:41:54.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:54.279+0000 7f372b43f640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f37240b41e0 con 0x7f37240cea90 2024-09-21T03:41:54.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:54.279+0000 7f372b43f640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f37240b44e0 con 0x7f372415a100 2024-09-21T03:41:54.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:54.279+0000 7f37289da640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f37240d3a70 0x7f372415dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:54.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:54.279+0000 7f37289da640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f37240d3a70 0x7f372415dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53056/0 (socket says 172.21.15.38:53056) 2024-09-21T03:41:54.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:54.279+0000 7f37289da640 1 -- 172.21.15.38:0/2621398911 learned_addr learned my addr 172.21.15.38:0/2621398911 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:41:54.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:54.280+0000 7f37291db640 1 --2- 172.21.15.38:0/2621398911 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f37240cea90 0x7f372414b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:54.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:54.280+0000 7f37299dc640 1 --2- 172.21.15.38:0/2621398911 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f372415a100 0x7f372415e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:54.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:54.280+0000 7f37289da640 1 -- 172.21.15.38:0/2621398911 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f372415a100 msgr2=0x7f372415e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:54.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:54.280+0000 7f37289da640 1 --2- 172.21.15.38:0/2621398911 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f372415a100 0x7f372415e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:54.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:54.280+0000 7f37289da640 1 -- 172.21.15.38:0/2621398911 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f37240cea90 msgr2=0x7f372414b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:54.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:54.280+0000 7f37289da640 1 --2- 172.21.15.38:0/2621398911 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f37240cea90 0x7f372414b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:54.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:54.280+0000 7f37289da640 1 -- 172.21.15.38:0/2621398911 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3724154ff0 con 0x7f37240d3a70 2024-09-21T03:41:54.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:54.280+0000 7f37289da640 1 --2- 172.21.15.38:0/2621398911 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f37240d3a70 0x7f372415dd80 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f371800c370 tx=0x7f371800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:54.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:54.280+0000 7f3721ffb640 1 -- 172.21.15.38:0/2621398911 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3718017070 con 0x7f37240d3a70 2024-09-21T03:41:54.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:54.280+0000 7f3721ffb640 1 -- 172.21.15.38:0/2621398911 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f37180053c0 con 0x7f37240d3a70 2024-09-21T03:41:54.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:54.280+0000 7f3721ffb640 1 -- 172.21.15.38:0/2621398911 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3718003850 con 0x7f37240d3a70 2024-09-21T03:41:54.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:54.280+0000 7f372b43f640 1 -- 172.21.15.38:0/2621398911 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f372415a7e0 con 0x7f37240d3a70 2024-09-21T03:41:54.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:54.281+0000 7f372b43f640 1 -- 172.21.15.38:0/2621398911 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37241577a0 con 0x7f37240d3a70 2024-09-21T03:41:54.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:54.282+0000 7f372b43f640 1 -- 172.21.15.38:0/2621398911 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37240d00d0 con 0x7f37240d3a70 2024-09-21T03:41:54.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:54.282+0000 7f3721ffb640 1 -- 172.21.15.38:0/2621398911 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f371801a070 con 0x7f37240d3a70 2024-09-21T03:41:54.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:54.283+0000 7f3721ffb640 1 --2- 172.21.15.38:0/2621398911 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3710078b90 0x7f371007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:54.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:54.283+0000 7f3721ffb640 1 -- 172.21.15.38:0/2621398911 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 7594+0+0 (secure 0 0 0) 0x7f371809a720 con 0x7f37240d3a70 2024-09-21T03:41:54.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:54.283+0000 7f37291db640 1 --2- 172.21.15.38:0/2621398911 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3710078b90 0x7f371007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:54.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:54.284+0000 7f37291db640 1 --2- 172.21.15.38:0/2621398911 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3710078b90 0x7f371007b0a0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f3714001090 tx=0x7f3714009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:54.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:54.287+0000 7f3721ffb640 1 -- 172.21.15.38:0/2621398911 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f3718062520 con 0x7f37240d3a70 2024-09-21T03:41:54.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:54.515+0000 7f372b43f640 1 -- 172.21.15.38:0/2621398911 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f372415a570 con 0x7f37240d3a70 2024-09-21T03:41:54.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:54.515+0000 7f3721ffb640 1 -- 172.21.15.38:0/2621398911 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 174 v174) ==== 95+0+4392 (secure 0 0 0) 0x7f3718061e20 con 0x7f37240d3a70 2024-09-21T03:41:54.518 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:41:54.518 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":174,"btime":"2024-09-21T03:41:51:788118+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6086,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/965972663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":965972663},{"type":"v1","addr":"172.21.15.195:6836","nonce":965972663}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6700,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/3176122334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":3176122334},{"type":"v1","addr":"172.21.15.195:6837","nonce":3176122334}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":8661,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/2418225210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":2418225210},{"type":"v1","addr":"172.21.15.38:6835","nonce":2418225210}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":171}],"filesystems":[{"mdsmap":{"epoch":174,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:40:02.319282+0000","modified":"2024-09-21T03:41:51.788113+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":111,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-21T03:41:54.518 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 174 2024-09-21T03:41:54.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:54.517+0000 7f372b43f640 1 -- 172.21.15.38:0/2621398911 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3710078b90 msgr2=0x7f371007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:54.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:54.517+0000 7f372b43f640 1 --2- 172.21.15.38:0/2621398911 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3710078b90 0x7f371007b0a0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f3714001090 tx=0x7f3714009040 comp rx=0 tx=0).stop 2024-09-21T03:41:54.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:54.517+0000 7f372b43f640 1 -- 172.21.15.38:0/2621398911 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f37240d3a70 msgr2=0x7f372415dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:54.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:54.517+0000 7f372b43f640 1 --2- 172.21.15.38:0/2621398911 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f37240d3a70 0x7f372415dd80 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f371800c370 tx=0x7f371800c830 comp rx=0 tx=0).stop 2024-09-21T03:41:54.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:54.517+0000 7f372b43f640 1 -- 172.21.15.38:0/2621398911 shutdown_connections 2024-09-21T03:41:54.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:54.517+0000 7f372b43f640 1 --2- 172.21.15.38:0/2621398911 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3710078b90 0x7f371007b0a0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:54.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:54.517+0000 7f372b43f640 1 --2- 172.21.15.38:0/2621398911 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f372415a100 0x7f372415e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:54.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:54.517+0000 7f372b43f640 1 --2- 172.21.15.38:0/2621398911 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f37240d3a70 0x7f372415dd80 unknown :-1 s=CLOSED pgs=1581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:54.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:54.517+0000 7f372b43f640 1 --2- 172.21.15.38:0/2621398911 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f37240cea90 0x7f372414b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:54.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:54.517+0000 7f372b43f640 1 -- 172.21.15.38:0/2621398911 >> 172.21.15.38:0/2621398911 conn(0x7f37240d4620 msgr2=0x7f372415f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:54.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:54.518+0000 7f372b43f640 1 -- 172.21.15.38:0/2621398911 shutdown_connections 2024-09-21T03:41:54.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:54.518+0000 7f372b43f640 1 -- 172.21.15.38:0/2621398911 wait complete. 2024-09-21T03:41:54.532 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-09-21T03:41:54.532 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:41:54.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:54.645+0000 7fd814ae5640 1 -- 172.21.15.38:0/1694592773 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd810162280 msgr2=0x7fd810162690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:54.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:54.645+0000 7fd814ae5640 1 --2- 172.21.15.38:0/1694592773 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd810162280 0x7fd810162690 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fd804009830 tx=0x7fd80402a510 comp rx=0 tx=0).stop 2024-09-21T03:41:54.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:54.645+0000 7fd814ae5640 1 -- 172.21.15.38:0/1694592773 shutdown_connections 2024-09-21T03:41:54.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:54.645+0000 7fd814ae5640 1 --2- 172.21.15.38:0/1694592773 >> v2:172.21.15.195:3300/0 conn(0x7fd8100b2e40 0x7fd8100b3250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:54.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:54.645+0000 7fd814ae5640 1 --2- 172.21.15.38:0/1694592773 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd810162280 0x7fd810162690 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:54.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:54.645+0000 7fd814ae5640 1 -- 172.21.15.38:0/1694592773 >> 172.21.15.38:0/1694592773 conn(0x7fd8100b6b20 msgr2=0x7fd8100d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:54.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:54.645+0000 7fd814ae5640 1 -- 172.21.15.38:0/1694592773 shutdown_connections 2024-09-21T03:41:54.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:54.645+0000 7fd814ae5640 1 -- 172.21.15.38:0/1694592773 wait complete. 2024-09-21T03:41:54.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:54.647+0000 7fd814ae5640 1 Processor -- start 2024-09-21T03:41:54.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:54.647+0000 7fd814ae5640 1 -- start start 2024-09-21T03:41:54.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:54.649+0000 7fd814ae5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd8100b2e40 0x7fd8100adad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:54.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:54.649+0000 7fd814ae5640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd810162280 0x7fd810149910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:54.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:54.649+0000 7fd814ae5640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd810149e50 0x7fd81014a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:54.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:54.649+0000 7fd814ae5640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fd81015e4b0 con 0x7fd8100b2e40 2024-09-21T03:41:54.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:54.649+0000 7fd814ae5640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd8100b39a0 con 0x7fd810149e50 2024-09-21T03:41:54.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:54.649+0000 7fd814ae5640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fd81015e630 con 0x7fd810162280 2024-09-21T03:41:54.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:54.649+0000 7fd80e59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd8100b2e40 0x7fd8100adad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:54.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:54.649+0000 7fd80e59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd8100b2e40 0x7fd8100adad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53058/0 (socket says 172.21.15.38:53058) 2024-09-21T03:41:54.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:54.649+0000 7fd80e59c640 1 -- 172.21.15.38:0/1612966091 learned_addr learned my addr 172.21.15.38:0/1612966091 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:41:54.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:54.649+0000 7fd80ed9d640 1 --2- 172.21.15.38:0/1612966091 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd810149e50 0x7fd81014a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:54.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:54.649+0000 7fd80dd9b640 1 --2- 172.21.15.38:0/1612966091 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd810162280 0x7fd810149910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:54.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:54.649+0000 7fd80e59c640 1 -- 172.21.15.38:0/1612966091 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd810162280 msgr2=0x7fd810149910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:54.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:54.649+0000 7fd80e59c640 1 --2- 172.21.15.38:0/1612966091 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd810162280 0x7fd810149910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:54.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:54.649+0000 7fd80e59c640 1 -- 172.21.15.38:0/1612966091 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd810149e50 msgr2=0x7fd81014a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:54.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:54.649+0000 7fd80e59c640 1 --2- 172.21.15.38:0/1612966091 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd810149e50 0x7fd81014a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:54.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:54.649+0000 7fd80e59c640 1 -- 172.21.15.38:0/1612966091 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd81014a7a0 con 0x7fd8100b2e40 2024-09-21T03:41:54.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:54.649+0000 7fd80e59c640 1 --2- 172.21.15.38:0/1612966091 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd8100b2e40 0x7fd8100adad0 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7fd80000c730 tx=0x7fd80000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:54.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:54.649+0000 7fd7feffd640 1 -- 172.21.15.38:0/1612966091 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd800003b70 con 0x7fd8100b2e40 2024-09-21T03:41:54.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:54.650+0000 7fd7feffd640 1 -- 172.21.15.38:0/1612966091 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd800003d30 con 0x7fd8100b2e40 2024-09-21T03:41:54.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:54.650+0000 7fd814ae5640 1 -- 172.21.15.38:0/1612966091 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8100aadc0 con 0x7fd8100b2e40 2024-09-21T03:41:54.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:54.650+0000 7fd7feffd640 1 -- 172.21.15.38:0/1612966091 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd80000aab0 con 0x7fd8100b2e40 2024-09-21T03:41:54.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:54.650+0000 7fd814ae5640 1 -- 172.21.15.38:0/1612966091 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd81015c320 con 0x7fd8100b2e40 2024-09-21T03:41:54.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:54.651+0000 7fd814ae5640 1 -- 172.21.15.38:0/1612966091 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd810162f00 con 0x7fd8100b2e40 2024-09-21T03:41:54.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:54.651+0000 7fd7feffd640 1 -- 172.21.15.38:0/1612966091 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fd80000ac50 con 0x7fd8100b2e40 2024-09-21T03:41:54.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:54.652+0000 7fd7feffd640 1 --2- 172.21.15.38:0/1612966091 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd7f0078b90 0x7fd7f007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:54.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:54.652+0000 7fd7feffd640 1 -- 172.21.15.38:0/1612966091 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 7594+0+0 (secure 0 0 0) 0x7fd80009b190 con 0x7fd8100b2e40 2024-09-21T03:41:54.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:54.652+0000 7fd80dd9b640 1 --2- 172.21.15.38:0/1612966091 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd7f0078b90 0x7fd7f007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:54.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:54.653+0000 7fd80dd9b640 1 --2- 172.21.15.38:0/1612966091 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd7f0078b90 0x7fd7f007b0a0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fd804034000 tx=0x7fd804004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:54.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:54.657+0000 7fd7feffd640 1 -- 172.21.15.38:0/1612966091 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fd810162bd0 con 0x7fd8100b2e40 2024-09-21T03:41:54.683 INFO:tasks.ceph.mds.c.smithi038.stderr:2024-09-21T03:41:54.680+0000 7fc3ef916640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T03:41:54.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:54.881+0000 7fd814ae5640 1 -- 172.21.15.38:0/1612966091 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd81015dbc0 con 0x7fd8100b2e40 2024-09-21T03:41:54.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:54.881+0000 7fd7feffd640 1 -- 172.21.15.38:0/1612966091 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 174 v174) ==== 95+0+4392 (secure 0 0 0) 0x7fd8000628b0 con 0x7fd8100b2e40 2024-09-21T03:41:54.884 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:41:54.884 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":174,"btime":"2024-09-21T03:41:51:788118+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6086,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/965972663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":965972663},{"type":"v1","addr":"172.21.15.195:6836","nonce":965972663}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6700,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/3176122334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":3176122334},{"type":"v1","addr":"172.21.15.195:6837","nonce":3176122334}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":8661,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/2418225210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":2418225210},{"type":"v1","addr":"172.21.15.38:6835","nonce":2418225210}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":171}],"filesystems":[{"mdsmap":{"epoch":174,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:40:02.319282+0000","modified":"2024-09-21T03:41:51.788113+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":111,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-21T03:41:54.885 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 174 2024-09-21T03:41:54.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:54.883+0000 7fd814ae5640 1 -- 172.21.15.38:0/1612966091 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd7f0078b90 msgr2=0x7fd7f007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:54.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:54.883+0000 7fd814ae5640 1 --2- 172.21.15.38:0/1612966091 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd7f0078b90 0x7fd7f007b0a0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fd804034000 tx=0x7fd804004730 comp rx=0 tx=0).stop 2024-09-21T03:41:54.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:54.883+0000 7fd814ae5640 1 -- 172.21.15.38:0/1612966091 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd8100b2e40 msgr2=0x7fd8100adad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:54.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:54.883+0000 7fd814ae5640 1 --2- 172.21.15.38:0/1612966091 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd8100b2e40 0x7fd8100adad0 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7fd80000c730 tx=0x7fd80000cbf0 comp rx=0 tx=0).stop 2024-09-21T03:41:54.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:54.883+0000 7fd814ae5640 1 -- 172.21.15.38:0/1612966091 shutdown_connections 2024-09-21T03:41:54.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:54.883+0000 7fd814ae5640 1 --2- 172.21.15.38:0/1612966091 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd810149e50 0x7fd81014a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:54.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:54.883+0000 7fd814ae5640 1 --2- 172.21.15.38:0/1612966091 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd7f0078b90 0x7fd7f007b0a0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:54.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:54.883+0000 7fd814ae5640 1 --2- 172.21.15.38:0/1612966091 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd810162280 0x7fd810149910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:54.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:54.883+0000 7fd814ae5640 1 --2- 172.21.15.38:0/1612966091 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd8100b2e40 0x7fd8100adad0 unknown :-1 s=CLOSED pgs=1582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:54.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:54.883+0000 7fd814ae5640 1 -- 172.21.15.38:0/1612966091 >> 172.21.15.38:0/1612966091 conn(0x7fd8100b6b20 msgr2=0x7fd810068ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:54.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:54.883+0000 7fd814ae5640 1 -- 172.21.15.38:0/1612966091 shutdown_connections 2024-09-21T03:41:54.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:54.883+0000 7fd814ae5640 1 -- 172.21.15.38:0/1612966091 wait complete. 2024-09-21T03:41:54.897 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-21T03:41:54.898 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:41:55.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:55.012+0000 7fac32630640 1 -- 172.21.15.38:0/4155175638 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fac2c0d19f0 msgr2=0x7fac2c15cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:55.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:55.012+0000 7fac32630640 1 --2- 172.21.15.38:0/4155175638 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fac2c0d19f0 0x7fac2c15cf20 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7fac20009830 tx=0x7fac2002a510 comp rx=0 tx=0).stop 2024-09-21T03:41:55.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:55.012+0000 7fac32630640 1 -- 172.21.15.38:0/4155175638 shutdown_connections 2024-09-21T03:41:55.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:55.012+0000 7fac32630640 1 --2- 172.21.15.38:0/4155175638 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fac2c0d19f0 0x7fac2c15cf20 unknown :-1 s=CLOSED pgs=1584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:55.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:55.012+0000 7fac32630640 1 -- 172.21.15.38:0/4155175638 >> 172.21.15.38:0/4155175638 conn(0x7fac2c0d2350 msgr2=0x7fac2c0d2760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:55.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:55.012+0000 7fac32630640 1 -- 172.21.15.38:0/4155175638 shutdown_connections 2024-09-21T03:41:55.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:55.012+0000 7fac32630640 1 -- 172.21.15.38:0/4155175638 wait complete. 2024-09-21T03:41:55.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:55.014+0000 7fac32630640 1 Processor -- start 2024-09-21T03:41:55.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:55.014+0000 7fac32630640 1 -- start start 2024-09-21T03:41:55.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:55.015+0000 7fac32630640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fac2c0b6250 0x7fac2c0c74f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:55.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:55.015+0000 7fac32630640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fac2c0d19f0 0x7fac2c151de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:55.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:55.015+0000 7fac32630640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fac2c15cb10 0x7fac2c0cda50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:55.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:55.015+0000 7fac32630640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fac2c0b7100 con 0x7fac2c15cb10 2024-09-21T03:41:55.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:55.015+0000 7fac32630640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fac2c0b6f80 con 0x7fac2c0b6250 2024-09-21T03:41:55.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:55.015+0000 7fac32630640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fac2c0b7280 con 0x7fac2c0d19f0 2024-09-21T03:41:55.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:55.015+0000 7fac30bcd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fac2c15cb10 0x7fac2c0cda50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:55.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:55.016+0000 7fac30bcd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fac2c15cb10 0x7fac2c0cda50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53080/0 (socket says 172.21.15.38:53080) 2024-09-21T03:41:55.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:55.016+0000 7fac30bcd640 1 -- 172.21.15.38:0/3041447187 learned_addr learned my addr 172.21.15.38:0/3041447187 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:41:55.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:55.016+0000 7fac2bfff640 1 --2- 172.21.15.38:0/3041447187 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fac2c0b6250 0x7fac2c0c74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:55.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:55.016+0000 7fac2b7fe640 1 --2- 172.21.15.38:0/3041447187 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fac2c0d19f0 0x7fac2c151de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:55.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:55.016+0000 7fac30bcd640 1 -- 172.21.15.38:0/3041447187 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fac2c0d19f0 msgr2=0x7fac2c151de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:55.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:55.016+0000 7fac30bcd640 1 --2- 172.21.15.38:0/3041447187 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fac2c0d19f0 0x7fac2c151de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:55.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:55.016+0000 7fac30bcd640 1 -- 172.21.15.38:0/3041447187 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fac2c0b6250 msgr2=0x7fac2c0c74f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:55.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:55.016+0000 7fac30bcd640 1 --2- 172.21.15.38:0/3041447187 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fac2c0b6250 0x7fac2c0c74f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:55.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:55.016+0000 7fac30bcd640 1 -- 172.21.15.38:0/3041447187 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac2c0c7bb0 con 0x7fac2c15cb10 2024-09-21T03:41:55.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:55.016+0000 7fac30bcd640 1 --2- 172.21.15.38:0/3041447187 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fac2c15cb10 0x7fac2c0cda50 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7fac2400ba10 tx=0x7fac2400bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:55.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:55.016+0000 7fac28ff9640 1 -- 172.21.15.38:0/3041447187 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac24019070 con 0x7fac2c15cb10 2024-09-21T03:41:55.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:55.016+0000 7fac28ff9640 1 -- 172.21.15.38:0/3041447187 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fac2400cc30 con 0x7fac2c15cb10 2024-09-21T03:41:55.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:55.016+0000 7fac32630640 1 -- 172.21.15.38:0/3041447187 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac2c0c7e90 con 0x7fac2c15cb10 2024-09-21T03:41:55.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:55.016+0000 7fac28ff9640 1 -- 172.21.15.38:0/3041447187 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac240146a0 con 0x7fac2c15cb10 2024-09-21T03:41:55.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:55.016+0000 7fac32630640 1 -- 172.21.15.38:0/3041447187 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac2c0c8450 con 0x7fac2c15cb10 2024-09-21T03:41:55.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:55.017+0000 7fac32630640 1 -- 172.21.15.38:0/3041447187 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac2c0c9d60 con 0x7fac2c15cb10 2024-09-21T03:41:55.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:55.018+0000 7fac28ff9640 1 -- 172.21.15.38:0/3041447187 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fac2401c070 con 0x7fac2c15cb10 2024-09-21T03:41:55.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:55.019+0000 7fac28ff9640 1 --2- 172.21.15.38:0/3041447187 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fac18078bc0 0x7fac1807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:55.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:55.019+0000 7fac28ff9640 1 -- 172.21.15.38:0/3041447187 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 7594+0+0 (secure 0 0 0) 0x7fac2409af70 con 0x7fac2c15cb10 2024-09-21T03:41:55.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:55.019+0000 7fac2bfff640 1 --2- 172.21.15.38:0/3041447187 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fac18078bc0 0x7fac1807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:55.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:55.020+0000 7fac2bfff640 1 --2- 172.21.15.38:0/3041447187 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fac18078bc0 0x7fac1807b0d0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fac1c000f60 tx=0x7fac1c009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:55.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:55.024+0000 7fac28ff9640 1 -- 172.21.15.38:0/3041447187 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fac2c0c9d60 con 0x7fac2c15cb10 2024-09-21T03:41:55.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:55.246+0000 7fac32630640 1 -- 172.21.15.38:0/3041447187 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fac2c15cf20 con 0x7fac2c15cb10 2024-09-21T03:41:55.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:55.247+0000 7fac28ff9640 1 -- 172.21.15.38:0/3041447187 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 174 v174) ==== 95+0+4392 (secure 0 0 0) 0x7fac2400ce20 con 0x7fac2c15cb10 2024-09-21T03:41:55.250 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:41:55.250 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":174,"btime":"2024-09-21T03:41:51:788118+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6086,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/965972663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":965972663},{"type":"v1","addr":"172.21.15.195:6836","nonce":965972663}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6700,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/3176122334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":3176122334},{"type":"v1","addr":"172.21.15.195:6837","nonce":3176122334}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":8661,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/2418225210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":2418225210},{"type":"v1","addr":"172.21.15.38:6835","nonce":2418225210}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":171}],"filesystems":[{"mdsmap":{"epoch":174,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:40:02.319282+0000","modified":"2024-09-21T03:41:51.788113+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":111,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-21T03:41:55.250 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 174 2024-09-21T03:41:55.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:55.249+0000 7fac32630640 1 -- 172.21.15.38:0/3041447187 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fac18078bc0 msgr2=0x7fac1807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:55.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:55.249+0000 7fac32630640 1 --2- 172.21.15.38:0/3041447187 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fac18078bc0 0x7fac1807b0d0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fac1c000f60 tx=0x7fac1c009040 comp rx=0 tx=0).stop 2024-09-21T03:41:55.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:55.249+0000 7fac32630640 1 -- 172.21.15.38:0/3041447187 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fac2c15cb10 msgr2=0x7fac2c0cda50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:55.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:55.249+0000 7fac32630640 1 --2- 172.21.15.38:0/3041447187 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fac2c15cb10 0x7fac2c0cda50 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7fac2400ba10 tx=0x7fac2400bed0 comp rx=0 tx=0).stop 2024-09-21T03:41:55.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:55.249+0000 7fac32630640 1 -- 172.21.15.38:0/3041447187 shutdown_connections 2024-09-21T03:41:55.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:55.249+0000 7fac32630640 1 --2- 172.21.15.38:0/3041447187 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fac2c15cb10 0x7fac2c0cda50 unknown :-1 s=CLOSED pgs=1585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:55.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:55.249+0000 7fac32630640 1 --2- 172.21.15.38:0/3041447187 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fac18078bc0 0x7fac1807b0d0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:55.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:55.249+0000 7fac32630640 1 --2- 172.21.15.38:0/3041447187 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fac2c0d19f0 0x7fac2c151de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:55.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:55.249+0000 7fac32630640 1 --2- 172.21.15.38:0/3041447187 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fac2c0b6250 0x7fac2c0c74f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:55.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:55.249+0000 7fac32630640 1 -- 172.21.15.38:0/3041447187 >> 172.21.15.38:0/3041447187 conn(0x7fac2c0d2350 msgr2=0x7fac2c160090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:55.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:55.249+0000 7fac32630640 1 -- 172.21.15.38:0/3041447187 shutdown_connections 2024-09-21T03:41:55.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:55.249+0000 7fac32630640 1 -- 172.21.15.38:0/3041447187 wait complete. 2024-09-21T03:41:55.263 INFO:tasks.cephfs.test_damage:Result: Mutation 'Delete 1.00000000' led to DAMAGED state 2024-09-21T03:41:55.263 INFO:tasks.cephfs.test_damage:Applying mutation 'Corrupt 1.00000000.inode' 2024-09-21T03:41:55.264 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-21T03:41:55.264 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:41:55.279 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056795,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2354 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T03:41:55.280 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi038.front.sepia.ceph.com 2024-09-21T03:41:55.280 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:41:55.280 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T03:41:55.335 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T03:41:55.335 INFO:teuthology.orchestra.run.smithi038.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-09-21T03:41:55.335 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-21T03:41:55.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:55.490+0000 7fd257dc6640 1 -- 172.21.15.38:0/4139398376 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd23c004800 msgr2=0x7fd23c004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:55.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:55.490+0000 7fd257dc6640 1 --2- 172.21.15.38:0/4139398376 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd23c004800 0x7fd23c004c10 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7fd2500d28f0 tx=0x7fd23c005830 comp rx=0 tx=0).stop 2024-09-21T03:41:55.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:55.490+0000 7fd257dc6640 1 -- 172.21.15.38:0/4139398376 shutdown_connections 2024-09-21T03:41:55.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:55.490+0000 7fd257dc6640 1 --2- 172.21.15.38:0/4139398376 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd23c004800 0x7fd23c004c10 unknown :-1 s=CLOSED pgs=1587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:55.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:55.490+0000 7fd257dc6640 1 -- 172.21.15.38:0/4139398376 >> 172.21.15.38:0/4139398376 conn(0x7fd25009d830 msgr2=0x7fd2500b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:55.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:55.490+0000 7fd257dc6640 1 -- 172.21.15.38:0/4139398376 shutdown_connections 2024-09-21T03:41:55.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:55.490+0000 7fd257dc6640 1 -- 172.21.15.38:0/4139398376 wait complete. 2024-09-21T03:41:55.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:55.492+0000 7fd257dc6640 1 Processor -- start 2024-09-21T03:41:55.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:55.492+0000 7fd257dc6640 1 -- start start 2024-09-21T03:41:55.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:55.493+0000 7fd257dc6640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd23c004800 0x7fd2501f51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:55.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:55.493+0000 7fd257dc6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd2500ce1c0 0x7fd250157ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:55.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:55.493+0000 7fd257dc6640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd25006e1e0 0x7fd25006d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:55.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:55.493+0000 7fd257dc6640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fd2500cf120 con 0x7fd2500ce1c0 2024-09-21T03:41:55.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:55.493+0000 7fd257dc6640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd2500cefa0 con 0x7fd23c004800 2024-09-21T03:41:55.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:55.493+0000 7fd257dc6640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fd2500cf2a0 con 0x7fd25006e1e0 2024-09-21T03:41:55.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:55.493+0000 7fd255361640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd2500ce1c0 0x7fd250157ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:55.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:55.493+0000 7fd255361640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd2500ce1c0 0x7fd250157ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53092/0 (socket says 172.21.15.38:53092) 2024-09-21T03:41:55.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:55.493+0000 7fd255361640 1 -- 172.21.15.38:0/600047206 learned_addr learned my addr 172.21.15.38:0/600047206 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:41:55.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:55.494+0000 7fd255b62640 1 --2- 172.21.15.38:0/600047206 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd23c004800 0x7fd2501f51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:55.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:55.494+0000 7fd256363640 1 --2- 172.21.15.38:0/600047206 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd25006e1e0 0x7fd25006d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:55.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:55.494+0000 7fd255361640 1 -- 172.21.15.38:0/600047206 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd25006e1e0 msgr2=0x7fd25006d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:55.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:55.494+0000 7fd255361640 1 --2- 172.21.15.38:0/600047206 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd25006e1e0 0x7fd25006d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:55.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:55.494+0000 7fd255361640 1 -- 172.21.15.38:0/600047206 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd23c004800 msgr2=0x7fd2501f51e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:55.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:55.494+0000 7fd255361640 1 --2- 172.21.15.38:0/600047206 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd23c004800 0x7fd2501f51e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:55.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:55.494+0000 7fd255361640 1 -- 172.21.15.38:0/600047206 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2501f5aa0 con 0x7fd2500ce1c0 2024-09-21T03:41:55.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:55.494+0000 7fd255b62640 1 --2- 172.21.15.38:0/600047206 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd23c004800 0x7fd2501f51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:41:55.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:55.494+0000 7fd256363640 1 --2- 172.21.15.38:0/600047206 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd25006e1e0 0x7fd25006d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:41:55.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:55.494+0000 7fd255361640 1 --2- 172.21.15.38:0/600047206 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd2500ce1c0 0x7fd250157ba0 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7fd244001eb0 tx=0x7fd244000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:55.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:55.494+0000 7fd2427fc640 1 -- 172.21.15.38:0/600047206 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2440019c0 con 0x7fd2500ce1c0 2024-09-21T03:41:55.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:55.494+0000 7fd2427fc640 1 -- 172.21.15.38:0/600047206 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd2440060d0 con 0x7fd2500ce1c0 2024-09-21T03:41:55.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:55.494+0000 7fd257dc6640 1 -- 172.21.15.38:0/600047206 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2501f5d80 con 0x7fd2500ce1c0 2024-09-21T03:41:55.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:55.494+0000 7fd2427fc640 1 -- 172.21.15.38:0/600047206 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2440035f0 con 0x7fd2500ce1c0 2024-09-21T03:41:55.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:55.494+0000 7fd257dc6640 1 -- 172.21.15.38:0/600047206 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd25006d160 con 0x7fd2500ce1c0 2024-09-21T03:41:55.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:55.495+0000 7fd257dc6640 1 -- 172.21.15.38:0/600047206 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2501f5810 con 0x7fd2500ce1c0 2024-09-21T03:41:55.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:55.496+0000 7fd2427fc640 1 -- 172.21.15.38:0/600047206 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fd24402b5a0 con 0x7fd2500ce1c0 2024-09-21T03:41:55.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:55.496+0000 7fd2427fc640 1 --2- 172.21.15.38:0/600047206 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd234078b90 0x7fd23407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:55.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:55.496+0000 7fd2427fc640 1 -- 172.21.15.38:0/600047206 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 7594+0+0 (secure 0 0 0) 0x7fd2440bb410 con 0x7fd2500ce1c0 2024-09-21T03:41:55.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:55.497+0000 7fd255b62640 1 --2- 172.21.15.38:0/600047206 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd234078b90 0x7fd23407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:55.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:55.497+0000 7fd255b62640 1 --2- 172.21.15.38:0/600047206 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd234078b90 0x7fd23407b0a0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fd24c009b10 tx=0x7fd24c005f90 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:55.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:55.501+0000 7fd2427fc640 1 -- 172.21.15.38:0/600047206 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fd244045060 con 0x7fd2500ce1c0 2024-09-21T03:41:55.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:55.727+0000 7fd257dc6640 1 -- 172.21.15.38:0/600047206 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fd2500d0c30 con 0x7fd2500ce1c0 2024-09-21T03:41:56.540 INFO:tasks.ceph.mds.c.smithi038.stderr: -55> 2024-09-21T03:41:51.784+0000 7fc3eb90e640 -1 log_channel(cluster) log [ERR] : dir 0x1 object missing on disk; some files may be lost () 2024-09-21T03:41:56.540 INFO:tasks.ceph.mds.c.smithi038.stderr: -53> 2024-09-21T03:41:51.784+0000 7fc3eb90e640 -1 mds.0.damage notify_dirfrag Damage to fragment * of ino 0x1 is fatal because it is a system directory for this rank 2024-09-21T03:41:56.540 INFO:tasks.ceph.mds.c.smithi038.stderr: -23> 2024-09-21T03:41:54.680+0000 7fc3ef916640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T03:41:56.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:56.554+0000 7fd2427fc640 1 -- 172.21.15.38:0/600047206 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v175) ==== 161+0+0 (secure 0 0 0) 0x7fd2501592d0 con 0x7fd2500ce1c0 2024-09-21T03:41:56.557 INFO:teuthology.orchestra.run.smithi038.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-21T03:41:56.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:56.559+0000 7fd257dc6640 1 -- 172.21.15.38:0/600047206 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd234078b90 msgr2=0x7fd23407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:56.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:56.559+0000 7fd257dc6640 1 --2- 172.21.15.38:0/600047206 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd234078b90 0x7fd23407b0a0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fd24c009b10 tx=0x7fd24c005f90 comp rx=0 tx=0).stop 2024-09-21T03:41:56.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:56.559+0000 7fd257dc6640 1 -- 172.21.15.38:0/600047206 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd2500ce1c0 msgr2=0x7fd250157ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:56.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:56.559+0000 7fd257dc6640 1 --2- 172.21.15.38:0/600047206 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd2500ce1c0 0x7fd250157ba0 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7fd244001eb0 tx=0x7fd244000f40 comp rx=0 tx=0).stop 2024-09-21T03:41:56.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:56.559+0000 7fd257dc6640 1 -- 172.21.15.38:0/600047206 shutdown_connections 2024-09-21T03:41:56.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:56.559+0000 7fd257dc6640 1 --2- 172.21.15.38:0/600047206 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd234078b90 0x7fd23407b0a0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:56.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:56.559+0000 7fd257dc6640 1 --2- 172.21.15.38:0/600047206 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd25006e1e0 0x7fd25006d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:56.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:56.559+0000 7fd257dc6640 1 --2- 172.21.15.38:0/600047206 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd2500ce1c0 0x7fd250157ba0 unknown :-1 s=CLOSED pgs=1588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:56.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:56.559+0000 7fd257dc6640 1 --2- 172.21.15.38:0/600047206 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd23c004800 0x7fd2501f51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:56.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:56.559+0000 7fd257dc6640 1 -- 172.21.15.38:0/600047206 >> 172.21.15.38:0/600047206 conn(0x7fd25009d830 msgr2=0x7fd250158a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:56.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:56.559+0000 7fd257dc6640 1 -- 172.21.15.38:0/600047206 shutdown_connections 2024-09-21T03:41:56.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:56.559+0000 7fd257dc6640 1 -- 172.21.15.38:0/600047206 wait complete. 2024-09-21T03:41:56.577 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-09-21T03:41:56.604 INFO:tasks.ceph.mds.c.smithi038.stdout:starting mds.c at 2024-09-21T03:41:56.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:56.700+0000 7f39b1fa1640 1 -- 172.21.15.38:0/1123002858 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f39ac15ee80 msgr2=0x7f39ac0d0170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:56.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:56.700+0000 7f39b1fa1640 1 --2- 172.21.15.38:0/1123002858 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f39ac15ee80 0x7f39ac0d0170 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7f39ac0d06b0 tx=0x7f399c02a4d0 comp rx=0 tx=0).stop 2024-09-21T03:41:56.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:56.700+0000 7f39b1fa1640 1 -- 172.21.15.38:0/1123002858 shutdown_connections 2024-09-21T03:41:56.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:56.700+0000 7f39b1fa1640 1 --2- 172.21.15.38:0/1123002858 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f39ac0d1120 0x7f39ac0ceb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:56.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:56.700+0000 7f39b1fa1640 1 --2- 172.21.15.38:0/1123002858 >> v2:172.21.15.195:3300/0 conn(0x7f39ac0b9ad0 0x7f39ac0d0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:56.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:56.700+0000 7f39b1fa1640 1 --2- 172.21.15.38:0/1123002858 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f39ac15ee80 0x7f39ac0d0170 unknown :-1 s=CLOSED pgs=1591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:56.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:56.700+0000 7f39b1fa1640 1 -- 172.21.15.38:0/1123002858 >> 172.21.15.38:0/1123002858 conn(0x7f39ac09d830 msgr2=0x7f39ac0b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:56.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:56.700+0000 7f39b1fa1640 1 -- 172.21.15.38:0/1123002858 shutdown_connections 2024-09-21T03:41:56.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:56.700+0000 7f39b1fa1640 1 -- 172.21.15.38:0/1123002858 wait complete. 2024-09-21T03:41:56.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:56.702+0000 7f39b1fa1640 1 Processor -- start 2024-09-21T03:41:56.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:56.702+0000 7f39b1fa1640 1 -- start start 2024-09-21T03:41:56.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:56.703+0000 7f39b1fa1640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f39ac0b9ad0 0x7f39ac0cc340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:56.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:56.703+0000 7f39b1fa1640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f39ac0d1120 0x7f39ac156b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:56.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:56.703+0000 7f39b1fa1640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f39ac15ee80 0x7f39ac1578a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:56.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:56.703+0000 7f39b1fa1640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f39ac0cf520 con 0x7f39ac15ee80 2024-09-21T03:41:56.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:56.703+0000 7f39b1fa1640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f39ac0cf3a0 con 0x7f39ac0b9ad0 2024-09-21T03:41:56.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:56.703+0000 7f39b1fa1640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f39ac0b6580 con 0x7f39ac0d1120 2024-09-21T03:41:56.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:56.703+0000 7f39abfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f39ac15ee80 0x7f39ac1578a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:56.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:56.703+0000 7f39abfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f39ac15ee80 0x7f39ac1578a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53140/0 (socket says 172.21.15.38:53140) 2024-09-21T03:41:56.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:56.703+0000 7f39abfff640 1 -- 172.21.15.38:0/3292650557 learned_addr learned my addr 172.21.15.38:0/3292650557 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:41:56.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:56.703+0000 7f39aaffd640 1 --2- 172.21.15.38:0/3292650557 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f39ac0d1120 0x7f39ac156b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:56.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:56.703+0000 7f39ab7fe640 1 --2- 172.21.15.38:0/3292650557 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f39ac0b9ad0 0x7f39ac0cc340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:56.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:56.704+0000 7f39abfff640 1 -- 172.21.15.38:0/3292650557 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f39ac0d1120 msgr2=0x7f39ac156b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:56.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:56.704+0000 7f39abfff640 1 --2- 172.21.15.38:0/3292650557 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f39ac0d1120 0x7f39ac156b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:56.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:56.704+0000 7f39abfff640 1 -- 172.21.15.38:0/3292650557 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f39ac0b9ad0 msgr2=0x7f39ac0cc340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:56.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:56.704+0000 7f39abfff640 1 --2- 172.21.15.38:0/3292650557 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f39ac0b9ad0 0x7f39ac0cc340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:56.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:56.704+0000 7f39abfff640 1 -- 172.21.15.38:0/3292650557 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39ac0cc880 con 0x7f39ac15ee80 2024-09-21T03:41:56.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:56.704+0000 7f39abfff640 1 --2- 172.21.15.38:0/3292650557 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f39ac15ee80 0x7f39ac1578a0 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7f39a400e960 tx=0x7f39a400ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:56.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:56.704+0000 7f3993fff640 1 -- 172.21.15.38:0/3292650557 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39a4004d20 con 0x7f39ac15ee80 2024-09-21T03:41:56.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:56.704+0000 7f3993fff640 1 -- 172.21.15.38:0/3292650557 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f39a400cb20 con 0x7f39ac15ee80 2024-09-21T03:41:56.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:56.704+0000 7f39b1fa1640 1 -- 172.21.15.38:0/3292650557 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39ac1f6630 con 0x7f39ac15ee80 2024-09-21T03:41:56.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:56.704+0000 7f3993fff640 1 -- 172.21.15.38:0/3292650557 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39a4005020 con 0x7f39ac15ee80 2024-09-21T03:41:56.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:56.704+0000 7f39b1fa1640 1 -- 172.21.15.38:0/3292650557 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39ac0cbb40 con 0x7f39ac15ee80 2024-09-21T03:41:56.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:56.705+0000 7f39b1fa1640 1 -- 172.21.15.38:0/3292650557 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39ac1f63a0 con 0x7f39ac15ee80 2024-09-21T03:41:56.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:56.706+0000 7f3993fff640 1 -- 172.21.15.38:0/3292650557 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f39a40051c0 con 0x7f39ac15ee80 2024-09-21T03:41:56.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:56.706+0000 7f3993fff640 1 --2- 172.21.15.38:0/3292650557 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3998078b90 0x7f399807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:56.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:56.706+0000 7f3993fff640 1 -- 172.21.15.38:0/3292650557 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 7594+0+0 (secure 0 0 0) 0x7f39a409bbf0 con 0x7f39ac15ee80 2024-09-21T03:41:56.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:56.706+0000 7f39ab7fe640 1 --2- 172.21.15.38:0/3292650557 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3998078b90 0x7f399807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:56.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:56.707+0000 7f39ab7fe640 1 --2- 172.21.15.38:0/3292650557 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3998078b90 0x7f399807b0a0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f399c033000 tx=0x7f399c004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:56.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:56.711+0000 7f3993fff640 1 -- 172.21.15.38:0/3292650557 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f39a4016030 con 0x7f39ac15ee80 2024-09-21T03:41:56.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:56.973+0000 7f39b1fa1640 1 -- 172.21.15.38:0/3292650557 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7f39ac0d4ce0 con 0x7f39ac15ee80 2024-09-21T03:41:57.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.556+0000 7f3993fff640 1 -- 172.21.15.38:0/3292650557 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v176) ==== 101+0+0 (secure 0 0 0) 0x7f39a40633a0 con 0x7f39ac15ee80 2024-09-21T03:41:57.559 INFO:teuthology.orchestra.run.smithi038.stderr:repaired: restoring rank 6:0 2024-09-21T03:41:57.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.559+0000 7f39b1fa1640 1 -- 172.21.15.38:0/3292650557 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3998078b90 msgr2=0x7f399807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:57.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.559+0000 7f39b1fa1640 1 --2- 172.21.15.38:0/3292650557 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3998078b90 0x7f399807b0a0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f399c033000 tx=0x7f399c004730 comp rx=0 tx=0).stop 2024-09-21T03:41:57.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.559+0000 7f39b1fa1640 1 -- 172.21.15.38:0/3292650557 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f39ac15ee80 msgr2=0x7f39ac1578a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:57.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.559+0000 7f39b1fa1640 1 --2- 172.21.15.38:0/3292650557 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f39ac15ee80 0x7f39ac1578a0 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7f39a400e960 tx=0x7f39a400ee20 comp rx=0 tx=0).stop 2024-09-21T03:41:57.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.559+0000 7f39b1fa1640 1 -- 172.21.15.38:0/3292650557 shutdown_connections 2024-09-21T03:41:57.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.559+0000 7f39b1fa1640 1 --2- 172.21.15.38:0/3292650557 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f39ac15ee80 0x7f39ac1578a0 unknown :-1 s=CLOSED pgs=1592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:57.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.559+0000 7f39b1fa1640 1 --2- 172.21.15.38:0/3292650557 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3998078b90 0x7f399807b0a0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:57.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.559+0000 7f39b1fa1640 1 --2- 172.21.15.38:0/3292650557 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f39ac0d1120 0x7f39ac156b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:57.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.559+0000 7f39b1fa1640 1 --2- 172.21.15.38:0/3292650557 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f39ac0b9ad0 0x7f39ac0cc340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:57.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.559+0000 7f39b1fa1640 1 -- 172.21.15.38:0/3292650557 >> 172.21.15.38:0/3292650557 conn(0x7f39ac09d830 msgr2=0x7f39ac160b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:57.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.559+0000 7f39b1fa1640 1 -- 172.21.15.38:0/3292650557 shutdown_connections 2024-09-21T03:41:57.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.559+0000 7f39b1fa1640 1 -- 172.21.15.38:0/3292650557 wait complete. 2024-09-21T03:41:57.576 DEBUG:teuthology.orchestra.run.smithi038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-09-21T03:41:57.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.677+0000 7fdc0d568f40 1 Processor -- start 2024-09-21T03:41:57.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.677+0000 7fdc0d568f40 1 -- start start 2024-09-21T03:41:57.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.678+0000 7fdc0d568f40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f8701fccc0 0x55f8701fc2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:57.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.678+0000 7fdc0d568f40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f8701ce1c0 0x55f8701ce5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:57.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.678+0000 7fdc0d568f40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f8701ca900 0x55f8701cad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:57.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.678+0000 7fdc0d568f40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55f87017b430 con 0x55f8701fccc0 2024-09-21T03:41:57.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.678+0000 7fdc0d568f40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55f8701881f0 con 0x55f8701ca900 2024-09-21T03:41:57.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.678+0000 7fdc0d568f40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55f87011cd60 con 0x55f8701ce1c0 2024-09-21T03:41:57.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.678+0000 7fdc07022640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f8701fccc0 0x55f8701fc2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:57.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.678+0000 7fdc07022640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f8701fccc0 0x55f8701fc2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53166/0 (socket says 172.21.15.38:53166) 2024-09-21T03:41:57.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.678+0000 7fdc07022640 1 -- 172.21.15.38:0/2418985487 learned_addr learned my addr 172.21.15.38:0/2418985487 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:41:57.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.678+0000 7fdc07823640 1 --2- 172.21.15.38:0/2418985487 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f8701ca900 0x55f8701cad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:57.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.678+0000 7fdc06821640 1 --2- 172.21.15.38:0/2418985487 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f8701ce1c0 0x55f8701ce5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:57.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.678+0000 7fdc07022640 1 -- 172.21.15.38:0/2418985487 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f8701ce1c0 msgr2=0x55f8701ce5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:57.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.678+0000 7fdc07022640 1 --2- 172.21.15.38:0/2418985487 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f8701ce1c0 0x55f8701ce5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:57.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.678+0000 7fdc07022640 1 -- 172.21.15.38:0/2418985487 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f8701ca900 msgr2=0x55f8701cad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:57.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.678+0000 7fdc07022640 1 --2- 172.21.15.38:0/2418985487 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f8701ca900 0x55f8701cad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:57.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.678+0000 7fdc07022640 1 -- 172.21.15.38:0/2418985487 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f8701f7100 con 0x55f8701fccc0 2024-09-21T03:41:57.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.678+0000 7fdc06821640 1 --2- 172.21.15.38:0/2418985487 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f8701ce1c0 0x55f8701ce5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:41:57.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.679+0000 7fdc07022640 1 --2- 172.21.15.38:0/2418985487 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f8701fccc0 0x55f8701fc2b0 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7fdbf8001ba0 tx=0x7fdbf8009dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4972fd2512ff4e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:57.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.679+0000 7fdc0581f640 1 -- 172.21.15.38:0/2418985487 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbf8019070 con 0x55f8701fccc0 2024-09-21T03:41:57.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.679+0000 7fdc0581f640 1 -- 172.21.15.38:0/2418985487 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdbf80059f0 con 0x55f8701fccc0 2024-09-21T03:41:57.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.679+0000 7fdc0581f640 1 -- 172.21.15.38:0/2418985487 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbf8005e40 con 0x55f8701fccc0 2024-09-21T03:41:57.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.679+0000 7fdc0d568f40 1 -- 172.21.15.38:0/2418985487 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f8701fccc0 msgr2=0x55f8701fc2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:57.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.679+0000 7fdc0d568f40 1 --2- 172.21.15.38:0/2418985487 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f8701fccc0 0x55f8701fc2b0 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7fdbf8001ba0 tx=0x7fdbf8009dc0 comp rx=0 tx=0).stop 2024-09-21T03:41:57.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.679+0000 7fdc0d568f40 1 -- 172.21.15.38:0/2418985487 shutdown_connections 2024-09-21T03:41:57.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.679+0000 7fdc0d568f40 1 --2- 172.21.15.38:0/2418985487 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f8701ca900 0x55f8701cad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:57.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.679+0000 7fdc0d568f40 1 --2- 172.21.15.38:0/2418985487 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f8701ce1c0 0x55f8701ce5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:57.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.679+0000 7fdc0d568f40 1 --2- 172.21.15.38:0/2418985487 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f8701fccc0 0x55f8701fc2b0 unknown :-1 s=CLOSED pgs=1595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:57.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.679+0000 7fdc0d568f40 1 -- 172.21.15.38:0/2418985487 >> 172.21.15.38:0/2418985487 conn(0x55f8700edac0 msgr2=0x55f8701cece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:57.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.680+0000 7fdc0d568f40 1 -- 172.21.15.38:0/2418985487 shutdown_connections 2024-09-21T03:41:57.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.680+0000 7fdc0d568f40 1 -- 172.21.15.38:0/2418985487 wait complete. 2024-09-21T03:41:57.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.680+0000 7fdc0d568f40 1 Processor -- start 2024-09-21T03:41:57.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.682+0000 7fdc0d568f40 1 -- start start 2024-09-21T03:41:57.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.683+0000 7fdc0d568f40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f8701ca900 0x55f870283a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:57.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.683+0000 7fdc0d568f40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f8701ce1c0 0x55f87029f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:57.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.683+0000 7fdc0d568f40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f8701fccc0 0x55f8702a1df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:57.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.683+0000 7fdc0d568f40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55f870282a60 con 0x55f8701fccc0 2024-09-21T03:41:57.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.683+0000 7fdc0d568f40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55f8701f73a0 con 0x55f8701ca900 2024-09-21T03:41:57.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.683+0000 7fdc0d568f40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55f8701fbc80 con 0x55f8701ce1c0 2024-09-21T03:41:57.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.683+0000 7fdc07823640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f8701fccc0 0x55f8702a1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:57.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.683+0000 7fdc07823640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f8701fccc0 0x55f8702a1df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53170/0 (socket says 172.21.15.38:53170) 2024-09-21T03:41:57.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.683+0000 7fdc07823640 1 -- 172.21.15.38:0/1575589723 learned_addr learned my addr 172.21.15.38:0/1575589723 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:41:57.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.683+0000 7fdc06821640 1 --2- 172.21.15.38:0/1575589723 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f8701ce1c0 0x55f87029f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:57.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.683+0000 7fdc07022640 1 --2- 172.21.15.38:0/1575589723 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f8701ca900 0x55f870283a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:57.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.683+0000 7fdc07823640 1 -- 172.21.15.38:0/1575589723 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f8701ce1c0 msgr2=0x55f87029f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:57.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.683+0000 7fdc07823640 1 --2- 172.21.15.38:0/1575589723 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f8701ce1c0 0x55f87029f9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:57.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.683+0000 7fdc07823640 1 -- 172.21.15.38:0/1575589723 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f8701ca900 msgr2=0x55f870283a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:57.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.683+0000 7fdc07823640 1 --2- 172.21.15.38:0/1575589723 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f8701ca900 0x55f870283a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:57.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.683+0000 7fdc07823640 1 -- 172.21.15.38:0/1575589723 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f8702a1630 con 0x55f8701fccc0 2024-09-21T03:41:57.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.683+0000 7fdc07823640 1 --2- 172.21.15.38:0/1575589723 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f8701fccc0 0x55f8702a1df0 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7fdc0000a640 tx=0x7fdc00051bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:57.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.683+0000 7fdc0481d640 1 -- 172.21.15.38:0/1575589723 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc0000a340 con 0x55f8701fccc0 2024-09-21T03:41:57.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.683+0000 7fdc0481d640 1 -- 172.21.15.38:0/1575589723 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdc00051db0 con 0x55f8701fccc0 2024-09-21T03:41:57.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.683+0000 7fdc0d568f40 1 -- 172.21.15.38:0/1575589723 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55f8701fc350 con 0x55f8701fccc0 2024-09-21T03:41:57.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.683+0000 7fdc0481d640 1 -- 172.21.15.38:0/1575589723 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc00001860 con 0x55f8701fccc0 2024-09-21T03:41:57.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.683+0000 7fdc0d568f40 1 -- 172.21.15.38:0/1575589723 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f8702a18d0 con 0x55f8701fccc0 2024-09-21T03:41:57.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.685+0000 7fdc0481d640 1 -- 172.21.15.38:0/1575589723 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fdc00059030 con 0x55f8701fccc0 2024-09-21T03:41:57.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.685+0000 7fdc0481d640 1 --2- 172.21.15.38:0/1575589723 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fdbf4077570 0x7fdbf4079a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:57.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.685+0000 7fdc0481d640 1 -- 172.21.15.38:0/1575589723 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 7594+0+0 (secure 0 0 0) 0x7fdc000dd130 con 0x55f8701fccc0 2024-09-21T03:41:57.692 INFO:teuthology.orchestra.run.smithi038.stdout:Importing pool 2024-09-21T03:41:57.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.686+0000 7fdc07022640 1 --2- 172.21.15.38:0/1575589723 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fdbf4077570 0x7fdbf4079a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:57.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.686+0000 7fdc0d568f40 1 --2- 172.21.15.38:0/1575589723 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x55f870284270 0x55f8701cad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:57.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.686+0000 7fdc0d568f40 1 -- 172.21.15.38:0/1575589723 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55f8701cb250 con 0x55f870284270 2024-09-21T03:41:57.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.686+0000 7fdc06821640 1 --2- 172.21.15.38:0/1575589723 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x55f870284270 0x55f8701cad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:57.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.686+0000 7fdc07022640 1 --2- 172.21.15.38:0/1575589723 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fdbf4077570 0x7fdbf4079a80 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fdbf8017040 tx=0x7fdbf80095a0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:57.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.686+0000 7fdc06821640 1 --2- 172.21.15.38:0/1575589723 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x55f870284270 0x55f8701cad10 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:57.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.688+0000 7fdc06821640 1 -- 172.21.15.38:0/1575589723 <== osd.7 v2:172.21.15.195:6818/1606106342 1 ==== osd_op_reply(1 601.00000000 [create] v111'443 uv417 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fdbfc00c140 con 0x55f870284270 2024-09-21T03:41:57.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.688+0000 7fdc0d568f40 1 -- 172.21.15.38:0/1575589723 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e111) -- 0x55f8703483e0 con 0x55f870284270 2024-09-21T03:41:57.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.691+0000 7fdc06821640 1 -- 172.21.15.38:0/1575589723 <== osd.7 v2:172.21.15.195:6818/1606106342 2 ==== osd_op_reply(2 601.00000000 [delete] v111'444 uv444 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbfc00c140 con 0x55f870284270 2024-09-21T03:41:57.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.691+0000 7fdc0d568f40 1 -- 172.21.15.38:0/1575589723 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55f870348780 con 0x55f870284270 2024-09-21T03:41:57.694 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-09-21T03:41:57.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.692+0000 7fdc06821640 1 -- 172.21.15.38:0/1575589723 <== osd.7 v2:172.21.15.195:6818/1606106342 3 ==== osd_op_reply(3 601.00000000 [create] v111'445 uv445 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbfc00c140 con 0x55f870284270 2024-09-21T03:41:57.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.692+0000 7fdc0d568f40 1 -- 172.21.15.38:0/1575589723 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55f870349b30 con 0x55f870284270 2024-09-21T03:41:57.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.694+0000 7fdc06821640 1 -- 172.21.15.38:0/1575589723 <== osd.7 v2:172.21.15.195:6818/1606106342 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v111'446 uv446 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbfc00c140 con 0x55f870284270 2024-09-21T03:41:57.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.694+0000 7fdc0d568f40 1 -- 172.21.15.38:0/1575589723 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55f870349ed0 con 0x55f870284270 2024-09-21T03:41:57.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.695+0000 7fdc06821640 1 -- 172.21.15.38:0/1575589723 <== osd.7 v2:172.21.15.195:6818/1606106342 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v111'447 uv447 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbfc00c140 con 0x55f870284270 2024-09-21T03:41:57.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.695+0000 7fdc0d568f40 1 -- 172.21.15.38:0/1575589723 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55f870348b20 con 0x55f870284270 2024-09-21T03:41:57.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.696+0000 7fdc06821640 1 -- 172.21.15.38:0/1575589723 <== osd.7 v2:172.21.15.195:6818/1606106342 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v111'448 uv448 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbfc00c140 con 0x55f870284270 2024-09-21T03:41:57.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.696+0000 7fdc0d568f40 1 -- 172.21.15.38:0/1575589723 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55f870348ec0 con 0x55f870284270 2024-09-21T03:41:57.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.698+0000 7fdc06821640 1 -- 172.21.15.38:0/1575589723 <== osd.7 v2:172.21.15.195:6818/1606106342 7 ==== osd_op_reply(7 602.00000000 [create] v111'449 uv423 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fdbfc00c140 con 0x55f870284270 2024-09-21T03:41:57.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.698+0000 7fdc0d568f40 1 -- 172.21.15.38:0/1575589723 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e111) -- 0x55f870349290 con 0x55f870284270 2024-09-21T03:41:57.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.699+0000 7fdc06821640 1 -- 172.21.15.38:0/1575589723 <== osd.7 v2:172.21.15.195:6818/1606106342 8 ==== osd_op_reply(8 602.00000000 [delete] v111'450 uv450 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbfc00c140 con 0x55f870284270 2024-09-21T03:41:57.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.699+0000 7fdc0d568f40 1 -- 172.21.15.38:0/1575589723 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55f870349290 con 0x55f870284270 2024-09-21T03:41:57.702 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-09-21T03:41:57.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.700+0000 7fdc06821640 1 -- 172.21.15.38:0/1575589723 <== osd.7 v2:172.21.15.195:6818/1606106342 9 ==== osd_op_reply(9 602.00000000 [create] v111'451 uv451 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbfc00c140 con 0x55f870284270 2024-09-21T03:41:57.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.700+0000 7fdc0d568f40 1 -- 172.21.15.38:0/1575589723 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55f870349260 con 0x55f870284270 2024-09-21T03:41:57.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.702+0000 7fdc06821640 1 -- 172.21.15.38:0/1575589723 <== osd.7 v2:172.21.15.195:6818/1606106342 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v111'452 uv452 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbfc00c140 con 0x55f870284270 2024-09-21T03:41:57.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.702+0000 7fdc0d568f40 1 -- 172.21.15.38:0/1575589723 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55f870349260 con 0x55f870284270 2024-09-21T03:41:57.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.703+0000 7fdc06821640 1 -- 172.21.15.38:0/1575589723 <== osd.7 v2:172.21.15.195:6818/1606106342 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v111'453 uv453 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbfc00c140 con 0x55f870284270 2024-09-21T03:41:57.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.703+0000 7fdc0d568f40 1 -- 172.21.15.38:0/1575589723 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55f870349260 con 0x55f870284270 2024-09-21T03:41:57.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.705+0000 7fdc06821640 1 -- 172.21.15.38:0/1575589723 <== osd.7 v2:172.21.15.195:6818/1606106342 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v111'454 uv454 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbfc00c140 con 0x55f870284270 2024-09-21T03:41:57.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.705+0000 7fdc0d568f40 1 -- 172.21.15.38:0/1575589723 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:13 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55f870349260 con 0x55f870284270 2024-09-21T03:41:57.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.706+0000 7fdc06821640 1 -- 172.21.15.38:0/1575589723 <== osd.7 v2:172.21.15.195:6818/1606106342 13 ==== osd_op_reply(13 10000000000.00000000 [create] v111'455 uv430 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7fdbfc00c140 con 0x55f870284270 2024-09-21T03:41:57.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.706+0000 7fdc0d568f40 1 -- 172.21.15.38:0/1575589723 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:14 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e111) -- 0x55f870349260 con 0x55f870284270 2024-09-21T03:41:57.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.708+0000 7fdc06821640 1 -- 172.21.15.38:0/1575589723 <== osd.7 v2:172.21.15.195:6818/1606106342 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v111'456 uv456 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fdbfc00c140 con 0x55f870284270 2024-09-21T03:41:57.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.708+0000 7fdc0d568f40 1 -- 172.21.15.38:0/1575589723 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:15 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55f870349260 con 0x55f870284270 2024-09-21T03:41:57.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.709+0000 7fdc06821640 1 -- 172.21.15.38:0/1575589723 <== osd.7 v2:172.21.15.195:6818/1606106342 15 ==== osd_op_reply(15 10000000000.00000000 [create] v111'457 uv457 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fdbfc00c140 con 0x55f870284270 2024-09-21T03:41:57.711 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-09-21T03:41:57.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.709+0000 7fdc0d568f40 1 -- 172.21.15.38:0/1575589723 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:16 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55f870349260 con 0x55f870284270 2024-09-21T03:41:57.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.711+0000 7fdc06821640 1 -- 172.21.15.38:0/1575589723 <== osd.7 v2:172.21.15.195:6818/1606106342 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v111'458 uv458 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fdbfc00c140 con 0x55f870284270 2024-09-21T03:41:57.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.711+0000 7fdc0d568f40 1 -- 172.21.15.38:0/1575589723 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:17 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55f870349260 con 0x55f870284270 2024-09-21T03:41:57.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.712+0000 7fdc06821640 1 -- 172.21.15.38:0/1575589723 <== osd.7 v2:172.21.15.195:6818/1606106342 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v111'459 uv459 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fdbfc00c140 con 0x55f870284270 2024-09-21T03:41:57.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.712+0000 7fdc0d568f40 1 -- 172.21.15.38:0/1575589723 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:18 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55f870349260 con 0x55f870284270 2024-09-21T03:41:57.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.714+0000 7fdc06821640 1 -- 172.21.15.38:0/1575589723 <== osd.7 v2:172.21.15.195:6818/1606106342 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v111'460 uv460 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fdbfc00c140 con 0x55f870284270 2024-09-21T03:41:57.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.714+0000 7fdc0d568f40 1 -- 172.21.15.38:0/1575589723 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:19 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=585b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55f870349260 con 0x55f870284270 2024-09-21T03:41:57.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.715+0000 7fdc06821640 1 -- 172.21.15.38:0/1575589723 <== osd.7 v2:172.21.15.195:6818/1606106342 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v111'461 uv461 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fdbfc00c140 con 0x55f870284270 2024-09-21T03:41:57.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.715+0000 7fdc0d568f40 1 -- 172.21.15.38:0/1575589723 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:20 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55f870349260 con 0x55f870284270 2024-09-21T03:41:57.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.717+0000 7fdc06821640 1 -- 172.21.15.38:0/1575589723 <== osd.7 v2:172.21.15.195:6818/1606106342 20 ==== osd_op_reply(20 600.00000000 [create] v111'462 uv436 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fdbfc00c140 con 0x55f870284270 2024-09-21T03:41:57.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.717+0000 7fdc0d568f40 1 -- 172.21.15.38:0/1575589723 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:21 13.10 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e111) -- 0x55f870349260 con 0x55f870284270 2024-09-21T03:41:57.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.718+0000 7fdc06821640 1 -- 172.21.15.38:0/1575589723 <== osd.7 v2:172.21.15.195:6818/1606106342 21 ==== osd_op_reply(21 600.00000000 [delete] v111'463 uv463 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbfc00c140 con 0x55f870284270 2024-09-21T03:41:57.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.718+0000 7fdc0d568f40 1 -- 172.21.15.38:0/1575589723 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:22 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55f870349260 con 0x55f870284270 2024-09-21T03:41:57.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.720+0000 7fdc06821640 1 -- 172.21.15.38:0/1575589723 <== osd.7 v2:172.21.15.195:6818/1606106342 22 ==== osd_op_reply(22 600.00000000 [create] v111'464 uv464 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbfc00c140 con 0x55f870284270 2024-09-21T03:41:57.722 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-09-21T03:41:57.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.720+0000 7fdc0d568f40 1 -- 172.21.15.38:0/1575589723 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:23 13.10 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55f870349260 con 0x55f870284270 2024-09-21T03:41:57.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.721+0000 7fdc06821640 1 -- 172.21.15.38:0/1575589723 <== osd.7 v2:172.21.15.195:6818/1606106342 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v111'465 uv465 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbfc00c140 con 0x55f870284270 2024-09-21T03:41:57.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.721+0000 7fdc0d568f40 1 -- 172.21.15.38:0/1575589723 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:24 13.10 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55f870349260 con 0x55f870284270 2024-09-21T03:41:57.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.722+0000 7fdc06821640 1 -- 172.21.15.38:0/1575589723 <== osd.7 v2:172.21.15.195:6818/1606106342 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v111'466 uv466 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbfc00c140 con 0x55f870284270 2024-09-21T03:41:57.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.722+0000 7fdc0d568f40 1 -- 172.21.15.38:0/1575589723 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:25 13.10 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55f870349260 con 0x55f870284270 2024-09-21T03:41:57.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.722+0000 7fdc06821640 1 -- 172.21.15.38:0/1575589723 <== osd.7 v2:172.21.15.195:6818/1606106342 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v111'467 uv467 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbfc00c140 con 0x55f870284270 2024-09-21T03:41:57.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.723+0000 7fdc0d568f40 1 -- 172.21.15.38:0/1575589723 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:26 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55f870349260 con 0x55f870284270 2024-09-21T03:41:57.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.725+0000 7fdc06821640 1 -- 172.21.15.38:0/1575589723 <== osd.7 v2:172.21.15.195:6818/1606106342 26 ==== osd_op_reply(26 603.00000000 [create] v111'468 uv442 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fdbfc00c140 con 0x55f870284270 2024-09-21T03:41:57.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.725+0000 7fdc0d568f40 1 -- 172.21.15.38:0/1575589723 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:27 13.10 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e111) -- 0x55f870349260 con 0x55f870284270 2024-09-21T03:41:57.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.727+0000 7fdc06821640 1 -- 172.21.15.38:0/1575589723 <== osd.7 v2:172.21.15.195:6818/1606106342 27 ==== osd_op_reply(27 603.00000000 [delete] v111'469 uv469 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbfc00c140 con 0x55f870284270 2024-09-21T03:41:57.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.727+0000 7fdc0d568f40 1 -- 172.21.15.38:0/1575589723 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:28 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55f870349260 con 0x55f870284270 2024-09-21T03:41:57.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.728+0000 7fdc06821640 1 -- 172.21.15.38:0/1575589723 <== osd.7 v2:172.21.15.195:6818/1606106342 28 ==== osd_op_reply(28 603.00000000 [create] v111'470 uv470 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbfc00c140 con 0x55f870284270 2024-09-21T03:41:57.730 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-09-21T03:41:57.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.728+0000 7fdc0d568f40 1 -- 172.21.15.38:0/1575589723 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:29 13.10 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55f870349260 con 0x55f870284270 2024-09-21T03:41:57.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.729+0000 7fdc06821640 1 -- 172.21.15.38:0/1575589723 <== osd.7 v2:172.21.15.195:6818/1606106342 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v111'471 uv471 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbfc00c140 con 0x55f870284270 2024-09-21T03:41:57.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.729+0000 7fdc0d568f40 1 -- 172.21.15.38:0/1575589723 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:30 13.10 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55f870349260 con 0x55f870284270 2024-09-21T03:41:57.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.729+0000 7fdc06821640 1 -- 172.21.15.38:0/1575589723 <== osd.7 v2:172.21.15.195:6818/1606106342 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v111'472 uv472 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbfc00c140 con 0x55f870284270 2024-09-21T03:41:57.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.730+0000 7fdc0d568f40 1 -- 172.21.15.38:0/1575589723 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:31 13.10 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55f870349260 con 0x55f870284270 2024-09-21T03:41:57.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.732+0000 7fdc06821640 1 -- 172.21.15.38:0/1575589723 <== osd.7 v2:172.21.15.195:6818/1606106342 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v111'473 uv473 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbfc00c140 con 0x55f870284270 2024-09-21T03:41:57.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.732+0000 7fdc0d568f40 1 --2- 172.21.15.38:0/1575589723 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x55f870349640 0x55f87034c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:57.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.732+0000 7fdc0d568f40 1 -- 172.21.15.38:0/1575589723 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55f870349260 con 0x55f870349640 2024-09-21T03:41:57.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.732+0000 7fdc07022640 1 --2- 172.21.15.38:0/1575589723 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x55f870349640 0x55f87034c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:57.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.732+0000 7fdc07022640 1 --2- 172.21.15.38:0/1575589723 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x55f870349640 0x55f87034c790 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:57.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.733+0000 7fdc07022640 1 -- 172.21.15.38:0/1575589723 <== osd.6 v2:172.21.15.195:6826/4055825675 1 ==== osd_op_reply(32 1.00000000.inode [create] v111'329 uv311 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7fdbf801c040 con 0x55f870349640 2024-09-21T03:41:57.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.733+0000 7fdc0d568f40 1 -- 172.21.15.38:0/1575589723 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e111) -- 0x55f87034ccd0 con 0x55f870349640 2024-09-21T03:41:57.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.735+0000 7fdc07022640 1 -- 172.21.15.38:0/1575589723 <== osd.6 v2:172.21.15.195:6826/4055825675 2 ==== osd_op_reply(33 1.00000000.inode [delete] v111'330 uv330 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fdbf801c040 con 0x55f870349640 2024-09-21T03:41:57.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.735+0000 7fdc0d568f40 1 -- 172.21.15.38:0/1575589723 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55f87034d070 con 0x55f870349640 2024-09-21T03:41:57.739 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-09-21T03:41:57.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.736+0000 7fdc07022640 1 -- 172.21.15.38:0/1575589723 <== osd.6 v2:172.21.15.195:6826/4055825675 3 ==== osd_op_reply(34 1.00000000.inode [create] v111'331 uv331 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fdbf801c040 con 0x55f870349640 2024-09-21T03:41:57.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.736+0000 7fdc0d568f40 1 -- 172.21.15.38:0/1575589723 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55f87034d8c0 con 0x55f870349640 2024-09-21T03:41:57.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.738+0000 7fdc07022640 1 -- 172.21.15.38:0/1575589723 <== osd.6 v2:172.21.15.195:6826/4055825675 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v111'332 uv332 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fdbf801c040 con 0x55f870349640 2024-09-21T03:41:57.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.738+0000 7fdc0d568f40 1 -- 172.21.15.38:0/1575589723 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55f87034dc60 con 0x55f870349640 2024-09-21T03:41:57.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.740+0000 7fdc07022640 1 -- 172.21.15.38:0/1575589723 <== osd.6 v2:172.21.15.195:6826/4055825675 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v111'333 uv333 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fdbf801c040 con 0x55f870349640 2024-09-21T03:41:57.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.740+0000 7fdc0d568f40 1 -- 172.21.15.38:0/1575589723 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55f87034e000 con 0x55f870349640 2024-09-21T03:41:57.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.741+0000 7fdc07022640 1 -- 172.21.15.38:0/1575589723 <== osd.6 v2:172.21.15.195:6826/4055825675 6 ==== osd_op_reply(37 200.00000000 [create] v111'334 uv316 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fdbf801c040 con 0x55f870349640 2024-09-21T03:41:57.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.741+0000 7fdc0d568f40 1 -- 172.21.15.38:0/1575589723 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e111) -- 0x55f87034e3a0 con 0x55f870349640 2024-09-21T03:41:57.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.742+0000 7fdc07022640 1 -- 172.21.15.38:0/1575589723 <== osd.6 v2:172.21.15.195:6826/4055825675 7 ==== osd_op_reply(38 200.00000000 [delete] v111'335 uv335 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbf801c040 con 0x55f870349640 2024-09-21T03:41:57.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.742+0000 7fdc0d568f40 1 -- 172.21.15.38:0/1575589723 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55f87034e3a0 con 0x55f870349640 2024-09-21T03:41:57.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.744+0000 7fdc07022640 1 -- 172.21.15.38:0/1575589723 <== osd.6 v2:172.21.15.195:6826/4055825675 8 ==== osd_op_reply(39 200.00000000 [create] v111'336 uv336 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbf801c040 con 0x55f870349640 2024-09-21T03:41:57.747 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-09-21T03:41:57.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.744+0000 7fdc0d568f40 1 -- 172.21.15.38:0/1575589723 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55f87034e3a0 con 0x55f870349640 2024-09-21T03:41:57.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.746+0000 7fdc07022640 1 -- 172.21.15.38:0/1575589723 <== osd.6 v2:172.21.15.195:6826/4055825675 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v111'337 uv337 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbf801c040 con 0x55f870349640 2024-09-21T03:41:57.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.746+0000 7fdc0d568f40 1 -- 172.21.15.38:0/1575589723 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55f87034e3a0 con 0x55f870349640 2024-09-21T03:41:57.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.747+0000 7fdc07022640 1 -- 172.21.15.38:0/1575589723 <== osd.6 v2:172.21.15.195:6826/4055825675 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v111'338 uv338 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbf801c040 con 0x55f870349640 2024-09-21T03:41:57.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.747+0000 7fdc0d568f40 1 -- 172.21.15.38:0/1575589723 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:42 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55f87034e3a0 con 0x55f870349640 2024-09-21T03:41:57.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.748+0000 7fdc07022640 1 -- 172.21.15.38:0/1575589723 <== osd.6 v2:172.21.15.195:6826/4055825675 11 ==== osd_op_reply(42 200.00000001 [create] v111'339 uv321 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fdbf801c040 con 0x55f870349640 2024-09-21T03:41:57.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.748+0000 7fdc0d568f40 1 -- 172.21.15.38:0/1575589723 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:43 13.14 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e111) -- 0x55f87034e3a0 con 0x55f870349640 2024-09-21T03:41:57.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.749+0000 7fdc07022640 1 -- 172.21.15.38:0/1575589723 <== osd.6 v2:172.21.15.195:6826/4055825675 12 ==== osd_op_reply(43 200.00000001 [delete] v111'340 uv340 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbf801c040 con 0x55f870349640 2024-09-21T03:41:57.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.749+0000 7fdc0d568f40 1 -- 172.21.15.38:0/1575589723 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:44 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55f87034e3a0 con 0x55f870349640 2024-09-21T03:41:57.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.751+0000 7fdc07022640 1 -- 172.21.15.38:0/1575589723 <== osd.6 v2:172.21.15.195:6826/4055825675 13 ==== osd_op_reply(44 200.00000001 [create] v111'341 uv341 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbf801c040 con 0x55f870349640 2024-09-21T03:41:57.754 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-09-21T03:41:57.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.751+0000 7fdc0d568f40 1 -- 172.21.15.38:0/1575589723 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:45 13.14 13:2e2fa760:::200.00000001:head [write 0~17490 in=17490b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55f870352820 con 0x55f870349640 2024-09-21T03:41:57.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.752+0000 7fdc07022640 1 -- 172.21.15.38:0/1575589723 <== osd.6 v2:172.21.15.195:6826/4055825675 14 ==== osd_op_reply(45 200.00000001 [write 0~17490] v111'342 uv342 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbf801c040 con 0x55f870349640 2024-09-21T03:41:57.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.752+0000 7fdc0d568f40 1 -- 172.21.15.38:0/1575589723 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:46 13.14 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55f870352820 con 0x55f870349640 2024-09-21T03:41:57.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.754+0000 7fdc07022640 1 -- 172.21.15.38:0/1575589723 <== osd.6 v2:172.21.15.195:6826/4055825675 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v111'343 uv343 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbf801c040 con 0x55f870349640 2024-09-21T03:41:57.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.754+0000 7fdc0d568f40 1 -- 172.21.15.38:0/1575589723 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:47 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55f87034e3a0 con 0x55f870349640 2024-09-21T03:41:57.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.755+0000 7fdc07022640 1 -- 172.21.15.38:0/1575589723 <== osd.6 v2:172.21.15.195:6826/4055825675 16 ==== osd_op_reply(47 606.00000000 [create] v111'344 uv327 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fdbf801c040 con 0x55f870349640 2024-09-21T03:41:57.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.755+0000 7fdc0d568f40 1 -- 172.21.15.38:0/1575589723 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:48 13.14 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e111) -- 0x55f87034e3a0 con 0x55f870349640 2024-09-21T03:41:57.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.757+0000 7fdc07022640 1 -- 172.21.15.38:0/1575589723 <== osd.6 v2:172.21.15.195:6826/4055825675 17 ==== osd_op_reply(48 606.00000000 [delete] v111'345 uv345 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbf801c040 con 0x55f870349640 2024-09-21T03:41:57.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.757+0000 7fdc0d568f40 1 -- 172.21.15.38:0/1575589723 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:49 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55f87034e3a0 con 0x55f870349640 2024-09-21T03:41:57.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.758+0000 7fdc07022640 1 -- 172.21.15.38:0/1575589723 <== osd.6 v2:172.21.15.195:6826/4055825675 18 ==== osd_op_reply(49 606.00000000 [create] v111'346 uv346 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbf801c040 con 0x55f870349640 2024-09-21T03:41:57.761 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-09-21T03:41:57.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.758+0000 7fdc0d568f40 1 -- 172.21.15.38:0/1575589723 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:50 13.14 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55f87034f3b0 con 0x55f870349640 2024-09-21T03:41:57.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.759+0000 7fdc07022640 1 -- 172.21.15.38:0/1575589723 <== osd.6 v2:172.21.15.195:6826/4055825675 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v111'347 uv347 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbf801c040 con 0x55f870349640 2024-09-21T03:41:57.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.759+0000 7fdc0d568f40 1 -- 172.21.15.38:0/1575589723 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:51 13.14 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55f87034f3b0 con 0x55f870349640 2024-09-21T03:41:57.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.760+0000 7fdc07022640 1 -- 172.21.15.38:0/1575589723 <== osd.6 v2:172.21.15.195:6826/4055825675 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v111'348 uv348 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbf801c040 con 0x55f870349640 2024-09-21T03:41:57.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.760+0000 7fdc0d568f40 1 -- 172.21.15.38:0/1575589723 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:52 13.14 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55f87034e3a0 con 0x55f870349640 2024-09-21T03:41:57.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.762+0000 7fdc07022640 1 -- 172.21.15.38:0/1575589723 <== osd.6 v2:172.21.15.195:6826/4055825675 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v111'349 uv349 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbf801c040 con 0x55f870349640 2024-09-21T03:41:57.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.762+0000 7fdc0d568f40 1 --2- 172.21.15.38:0/1575589723 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x55f87034ea20 0x55f870350e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:57.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.762+0000 7fdc0d568f40 1 -- 172.21.15.38:0/1575589723 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:53 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55f8703513c0 con 0x55f87034ea20 2024-09-21T03:41:57.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.762+0000 7fdc06821640 1 --2- 172.21.15.38:0/1575589723 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x55f87034ea20 0x55f870350e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:57.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.762+0000 7fdc06821640 1 --2- 172.21.15.38:0/1575589723 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x55f87034ea20 0x55f870350e40 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:57.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.763+0000 7fdc06821640 1 -- 172.21.15.38:0/1575589723 <== osd.0 v2:172.21.15.38:6800/2381639433 1 ==== osd_op_reply(53 607.00000000 [create] v111'87 uv86 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fdbfc00c140 con 0x55f87034ea20 2024-09-21T03:41:57.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.763+0000 7fdc0d568f40 1 -- 172.21.15.38:0/1575589723 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:54 13.1c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e111) -- 0x55f870351480 con 0x55f87034ea20 2024-09-21T03:41:57.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.765+0000 7fdc06821640 1 -- 172.21.15.38:0/1575589723 <== osd.0 v2:172.21.15.38:6800/2381639433 2 ==== osd_op_reply(54 607.00000000 [delete] v111'88 uv88 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbfc00c140 con 0x55f87034ea20 2024-09-21T03:41:57.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.765+0000 7fdc0d568f40 1 -- 172.21.15.38:0/1575589723 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:55 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55f8703518a0 con 0x55f87034ea20 2024-09-21T03:41:57.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.766+0000 7fdc06821640 1 -- 172.21.15.38:0/1575589723 <== osd.0 v2:172.21.15.38:6800/2381639433 3 ==== osd_op_reply(55 607.00000000 [create] v111'89 uv89 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbfc00c140 con 0x55f87034ea20 2024-09-21T03:41:57.769 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-09-21T03:41:57.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.766+0000 7fdc0d568f40 1 -- 172.21.15.38:0/1575589723 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:56 13.1c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55f8703528b0 con 0x55f87034ea20 2024-09-21T03:41:57.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.767+0000 7fdc06821640 1 -- 172.21.15.38:0/1575589723 <== osd.0 v2:172.21.15.38:6800/2381639433 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v111'90 uv90 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbfc00c140 con 0x55f87034ea20 2024-09-21T03:41:57.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.767+0000 7fdc0d568f40 1 -- 172.21.15.38:0/1575589723 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:57 13.1c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55f8703528b0 con 0x55f87034ea20 2024-09-21T03:41:57.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.768+0000 7fdc06821640 1 -- 172.21.15.38:0/1575589723 <== osd.0 v2:172.21.15.38:6800/2381639433 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v111'91 uv91 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbfc00c140 con 0x55f87034ea20 2024-09-21T03:41:57.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.768+0000 7fdc0d568f40 1 -- 172.21.15.38:0/1575589723 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:58 13.1c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55f8703518a0 con 0x55f87034ea20 2024-09-21T03:41:57.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.769+0000 7fdc06821640 1 -- 172.21.15.38:0/1575589723 <== osd.0 v2:172.21.15.38:6800/2381639433 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v111'92 uv92 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbfc00c140 con 0x55f87034ea20 2024-09-21T03:41:57.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.769+0000 7fdc0d568f40 1 --2- 172.21.15.38:0/1575589723 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x55f870351f20 0x55f870354340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:57.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.769+0000 7fdc0d568f40 1 -- 172.21.15.38:0/1575589723 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:59 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55f870354970 con 0x55f870351f20 2024-09-21T03:41:57.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.769+0000 7fdc07823640 1 --2- 172.21.15.38:0/1575589723 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x55f870351f20 0x55f870354340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:57.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.769+0000 7fdc07823640 1 --2- 172.21.15.38:0/1575589723 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x55f870351f20 0x55f870354340 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:57.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.771+0000 7fdc07823640 1 -- 172.21.15.38:0/1575589723 <== osd.3 v2:172.21.15.38:6808/1286810294 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v111'66 uv65 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7fdc0000b040 con 0x55f870351f20 2024-09-21T03:41:57.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.771+0000 7fdc0d568f40 1 -- 172.21.15.38:0/1575589723 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:60 13.6 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e111) -- 0x55f870354e80 con 0x55f870351f20 2024-09-21T03:41:57.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.772+0000 7fdc07823640 1 -- 172.21.15.38:0/1575589723 <== osd.3 v2:172.21.15.38:6808/1286810294 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v111'67 uv67 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fdc0000b040 con 0x55f870351f20 2024-09-21T03:41:57.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.772+0000 7fdc0d568f40 1 -- 172.21.15.38:0/1575589723 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:61 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55f870355380 con 0x55f870351f20 2024-09-21T03:41:57.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.773+0000 7fdc07823640 1 -- 172.21.15.38:0/1575589723 <== osd.3 v2:172.21.15.38:6808/1286810294 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v111'68 uv68 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fdc0000b040 con 0x55f870351f20 2024-09-21T03:41:57.776 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-09-21T03:41:57.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.773+0000 7fdc0d568f40 1 -- 172.21.15.38:0/1575589723 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:62 13.6 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55f870355c40 con 0x55f870351f20 2024-09-21T03:41:57.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.774+0000 7fdc07823640 1 -- 172.21.15.38:0/1575589723 <== osd.3 v2:172.21.15.38:6808/1286810294 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v111'69 uv69 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fdc0000b040 con 0x55f870351f20 2024-09-21T03:41:57.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.774+0000 7fdc0d568f40 1 -- 172.21.15.38:0/1575589723 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55f870355fe0 con 0x55f87034ea20 2024-09-21T03:41:57.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.775+0000 7fdc06821640 1 -- 172.21.15.38:0/1575589723 <== osd.0 v2:172.21.15.38:6800/2381639433 7 ==== osd_op_reply(63 608.00000000 [create] v111'87 uv86 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fdbfc00c140 con 0x55f87034ea20 2024-09-21T03:41:57.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.775+0000 7fdc0d568f40 1 -- 172.21.15.38:0/1575589723 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e111) -- 0x55f870355fe0 con 0x55f87034ea20 2024-09-21T03:41:57.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.777+0000 7fdc06821640 1 -- 172.21.15.38:0/1575589723 <== osd.0 v2:172.21.15.38:6800/2381639433 8 ==== osd_op_reply(64 608.00000000 [delete] v111'88 uv88 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbfc00c140 con 0x55f87034ea20 2024-09-21T03:41:57.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.777+0000 7fdc0d568f40 1 -- 172.21.15.38:0/1575589723 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55f870355fe0 con 0x55f87034ea20 2024-09-21T03:41:57.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.778+0000 7fdc06821640 1 -- 172.21.15.38:0/1575589723 <== osd.0 v2:172.21.15.38:6800/2381639433 9 ==== osd_op_reply(65 608.00000000 [create] v111'89 uv89 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbfc00c140 con 0x55f87034ea20 2024-09-21T03:41:57.781 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-09-21T03:41:57.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.778+0000 7fdc0d568f40 1 -- 172.21.15.38:0/1575589723 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55f870356ff0 con 0x55f87034ea20 2024-09-21T03:41:57.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.779+0000 7fdc06821640 1 -- 172.21.15.38:0/1575589723 <== osd.0 v2:172.21.15.38:6800/2381639433 10 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v111'90 uv90 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbfc00c140 con 0x55f87034ea20 2024-09-21T03:41:57.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.779+0000 7fdc0d568f40 1 -- 172.21.15.38:0/1575589723 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55f870356ff0 con 0x55f87034ea20 2024-09-21T03:41:57.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.780+0000 7fdc06821640 1 -- 172.21.15.38:0/1575589723 <== osd.0 v2:172.21.15.38:6800/2381639433 11 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v111'91 uv91 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbfc00c140 con 0x55f87034ea20 2024-09-21T03:41:57.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.780+0000 7fdc0d568f40 1 -- 172.21.15.38:0/1575589723 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55f870355fe0 con 0x55f87034ea20 2024-09-21T03:41:57.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.781+0000 7fdc06821640 1 -- 172.21.15.38:0/1575589723 <== osd.0 v2:172.21.15.38:6800/2381639433 12 ==== osd_op_reply(68 608.00000000 [omap-set-header] v111'92 uv92 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbfc00c140 con 0x55f87034ea20 2024-09-21T03:41:57.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.781+0000 7fdc0d568f40 1 -- 172.21.15.38:0/1575589723 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55f870355fe0 con 0x55f870351f20 2024-09-21T03:41:57.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.784+0000 7fdc07823640 1 -- 172.21.15.38:0/1575589723 <== osd.3 v2:172.21.15.38:6808/1286810294 5 ==== osd_op_reply(69 604.00000000 [create] v111'160 uv154 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fdc0000b040 con 0x55f870351f20 2024-09-21T03:41:57.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.784+0000 7fdc0d568f40 1 -- 172.21.15.38:0/1575589723 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e111) -- 0x55f870356380 con 0x55f870351f20 2024-09-21T03:41:57.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.784+0000 7fdc07823640 1 -- 172.21.15.38:0/1575589723 <== osd.3 v2:172.21.15.38:6808/1286810294 6 ==== osd_op_reply(70 604.00000000 [delete] v111'161 uv161 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdc0000b040 con 0x55f870351f20 2024-09-21T03:41:57.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.784+0000 7fdc0d568f40 1 -- 172.21.15.38:0/1575589723 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55f870356720 con 0x55f870351f20 2024-09-21T03:41:57.788 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-09-21T03:41:57.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.785+0000 7fdc07823640 1 -- 172.21.15.38:0/1575589723 <== osd.3 v2:172.21.15.38:6808/1286810294 7 ==== osd_op_reply(71 604.00000000 [create] v111'162 uv162 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdc0000b040 con 0x55f870351f20 2024-09-21T03:41:57.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.785+0000 7fdc0d568f40 1 -- 172.21.15.38:0/1575589723 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55f870357ad0 con 0x55f870351f20 2024-09-21T03:41:57.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.786+0000 7fdc07823640 1 -- 172.21.15.38:0/1575589723 <== osd.3 v2:172.21.15.38:6808/1286810294 8 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v111'163 uv163 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdc0000b040 con 0x55f870351f20 2024-09-21T03:41:57.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.786+0000 7fdc0d568f40 1 -- 172.21.15.38:0/1575589723 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55f870357ad0 con 0x55f870351f20 2024-09-21T03:41:57.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.787+0000 7fdc07823640 1 -- 172.21.15.38:0/1575589723 <== osd.3 v2:172.21.15.38:6808/1286810294 9 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v111'164 uv164 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdc0000b040 con 0x55f870351f20 2024-09-21T03:41:57.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.787+0000 7fdc0d568f40 1 -- 172.21.15.38:0/1575589723 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55f870356ac0 con 0x55f870351f20 2024-09-21T03:41:57.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.788+0000 7fdc07823640 1 -- 172.21.15.38:0/1575589723 <== osd.3 v2:172.21.15.38:6808/1286810294 10 ==== osd_op_reply(74 604.00000000 [omap-set-header] v111'165 uv165 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdc0000b040 con 0x55f870351f20 2024-09-21T03:41:57.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.788+0000 7fdc0d568f40 1 -- 172.21.15.38:0/1575589723 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55f870356ac0 con 0x55f870351f20 2024-09-21T03:41:57.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.789+0000 7fdc07823640 1 -- 172.21.15.38:0/1575589723 <== osd.3 v2:172.21.15.38:6808/1286810294 11 ==== osd_op_reply(75 500.00000000 [create] v111'166 uv159 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fdc0000b040 con 0x55f870351f20 2024-09-21T03:41:57.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.789+0000 7fdc0d568f40 1 -- 172.21.15.38:0/1575589723 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e111) -- 0x55f870356ac0 con 0x55f870351f20 2024-09-21T03:41:57.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.790+0000 7fdc07823640 1 -- 172.21.15.38:0/1575589723 <== osd.3 v2:172.21.15.38:6808/1286810294 12 ==== osd_op_reply(76 500.00000000 [delete] v111'167 uv167 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdc0000b040 con 0x55f870351f20 2024-09-21T03:41:57.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.790+0000 7fdc0d568f40 1 -- 172.21.15.38:0/1575589723 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55f870356ac0 con 0x55f870351f20 2024-09-21T03:41:57.794 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-09-21T03:41:57.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.791+0000 7fdc07823640 1 -- 172.21.15.38:0/1575589723 <== osd.3 v2:172.21.15.38:6808/1286810294 13 ==== osd_op_reply(77 500.00000000 [create] v111'168 uv168 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdc0000b040 con 0x55f870351f20 2024-09-21T03:41:57.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.791+0000 7fdc0d568f40 1 -- 172.21.15.38:0/1575589723 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55f870356ac0 con 0x55f870351f20 2024-09-21T03:41:57.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.792+0000 7fdc07823640 1 -- 172.21.15.38:0/1575589723 <== osd.3 v2:172.21.15.38:6808/1286810294 14 ==== osd_op_reply(78 500.00000000 [write 0~90] v111'169 uv169 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdc0000b040 con 0x55f870351f20 2024-09-21T03:41:57.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.792+0000 7fdc0d568f40 1 -- 172.21.15.38:0/1575589723 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55f870356ac0 con 0x55f870351f20 2024-09-21T03:41:57.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.793+0000 7fdc07823640 1 -- 172.21.15.38:0/1575589723 <== osd.3 v2:172.21.15.38:6808/1286810294 15 ==== osd_op_reply(79 500.00000000 [omap-set-header] v111'170 uv170 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdc0000b040 con 0x55f870351f20 2024-09-21T03:41:57.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.793+0000 7fdc0d568f40 1 -- 172.21.15.38:0/1575589723 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:80 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55f870356ac0 con 0x55f87034ea20 2024-09-21T03:41:57.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.794+0000 7fdc06821640 1 -- 172.21.15.38:0/1575589723 <== osd.0 v2:172.21.15.38:6800/2381639433 13 ==== osd_op_reply(80 mds_snaptable [create] v111'71 uv70 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7fdbfc00c140 con 0x55f87034ea20 2024-09-21T03:41:57.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.794+0000 7fdc0d568f40 1 -- 172.21.15.38:0/1575589723 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:81 13.d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e111) -- 0x55f870356ac0 con 0x55f87034ea20 2024-09-21T03:41:57.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.795+0000 7fdc06821640 1 -- 172.21.15.38:0/1575589723 <== osd.0 v2:172.21.15.38:6800/2381639433 14 ==== osd_op_reply(81 mds_snaptable [delete] v111'72 uv72 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fdbfc00c140 con 0x55f87034ea20 2024-09-21T03:41:57.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.795+0000 7fdc0d568f40 1 -- 172.21.15.38:0/1575589723 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:82 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55f870356ac0 con 0x55f87034ea20 2024-09-21T03:41:57.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.796+0000 7fdc06821640 1 -- 172.21.15.38:0/1575589723 <== osd.0 v2:172.21.15.38:6800/2381639433 15 ==== osd_op_reply(82 mds_snaptable [create] v111'73 uv73 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fdbfc00c140 con 0x55f87034ea20 2024-09-21T03:41:57.799 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-09-21T03:41:57.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.797+0000 7fdc0d568f40 1 -- 172.21.15.38:0/1575589723 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:83 13.d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55f8703557d0 con 0x55f87034ea20 2024-09-21T03:41:57.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.798+0000 7fdc06821640 1 -- 172.21.15.38:0/1575589723 <== osd.0 v2:172.21.15.38:6800/2381639433 16 ==== osd_op_reply(83 mds_snaptable [write 0~70] v111'74 uv74 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fdbfc00c140 con 0x55f87034ea20 2024-09-21T03:41:57.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.798+0000 7fdc0d568f40 1 -- 172.21.15.38:0/1575589723 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:84 13.d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55f8703557d0 con 0x55f87034ea20 2024-09-21T03:41:57.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.799+0000 7fdc06821640 1 -- 172.21.15.38:0/1575589723 <== osd.0 v2:172.21.15.38:6800/2381639433 17 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v111'75 uv75 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fdbfc00c140 con 0x55f87034ea20 2024-09-21T03:41:57.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.799+0000 7fdc0d568f40 1 --2- 172.21.15.38:0/1575589723 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x55f870357480 0x55f8703598a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:57.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.799+0000 7fdc0d568f40 1 -- 172.21.15.38:0/1575589723 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55f870359de0 con 0x55f870357480 2024-09-21T03:41:57.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.799+0000 7fdc07022640 1 --2- 172.21.15.38:0/1575589723 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x55f870357480 0x55f8703598a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:57.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.800+0000 7fdc07022640 1 --2- 172.21.15.38:0/1575589723 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x55f870357480 0x55f8703598a0 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:57.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.801+0000 7fdc07022640 1 -- 172.21.15.38:0/1575589723 <== osd.1 v2:172.21.15.38:6816/1491818732 1 ==== osd_op_reply(85 605.00000000 [create] v111'87 uv86 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fdbf801c040 con 0x55f870357480 2024-09-21T03:41:57.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.801+0000 7fdc0d568f40 1 -- 172.21.15.38:0/1575589723 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e111) -- 0x55f870359de0 con 0x55f870357480 2024-09-21T03:41:57.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.803+0000 7fdc07022640 1 -- 172.21.15.38:0/1575589723 <== osd.1 v2:172.21.15.38:6816/1491818732 2 ==== osd_op_reply(86 605.00000000 [delete] v111'88 uv88 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbf801c040 con 0x55f870357480 2024-09-21T03:41:57.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.803+0000 7fdc0d568f40 1 -- 172.21.15.38:0/1575589723 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55f870359e60 con 0x55f870357480 2024-09-21T03:41:57.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.804+0000 7fdc07022640 1 -- 172.21.15.38:0/1575589723 <== osd.1 v2:172.21.15.38:6816/1491818732 3 ==== osd_op_reply(87 605.00000000 [create] v111'89 uv89 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbf801c040 con 0x55f870357480 2024-09-21T03:41:57.807 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-09-21T03:41:57.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.804+0000 7fdc0d568f40 1 -- 172.21.15.38:0/1575589723 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55f87035b260 con 0x55f870357480 2024-09-21T03:41:57.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.805+0000 7fdc07022640 1 -- 172.21.15.38:0/1575589723 <== osd.1 v2:172.21.15.38:6816/1491818732 4 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v111'90 uv90 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbf801c040 con 0x55f870357480 2024-09-21T03:41:57.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.806+0000 7fdc0d568f40 1 -- 172.21.15.38:0/1575589723 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55f87035b260 con 0x55f870357480 2024-09-21T03:41:57.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.807+0000 7fdc07022640 1 -- 172.21.15.38:0/1575589723 <== osd.1 v2:172.21.15.38:6816/1491818732 5 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v111'91 uv91 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbf801c040 con 0x55f870357480 2024-09-21T03:41:57.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.807+0000 7fdc0d568f40 1 -- 172.21.15.38:0/1575589723 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55f87035a250 con 0x55f870357480 2024-09-21T03:41:57.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.807+0000 7fdc07022640 1 -- 172.21.15.38:0/1575589723 <== osd.1 v2:172.21.15.38:6816/1491818732 6 ==== osd_op_reply(90 605.00000000 [omap-set-header] v111'92 uv92 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbf801c040 con 0x55f870357480 2024-09-21T03:41:57.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.808+0000 7fdc0d568f40 1 -- 172.21.15.38:0/1575589723 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55f87035a250 con 0x55f870357480 2024-09-21T03:41:57.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.809+0000 7fdc07022640 1 -- 172.21.15.38:0/1575589723 <== osd.1 v2:172.21.15.38:6816/1491818732 7 ==== osd_op_reply(91 mds0_inotable [create] v111'142 uv136 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7fdbf801c040 con 0x55f870357480 2024-09-21T03:41:57.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.809+0000 7fdc0d568f40 1 -- 172.21.15.38:0/1575589723 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e111) -- 0x55f87035a250 con 0x55f870357480 2024-09-21T03:41:57.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.810+0000 7fdc07022640 1 -- 172.21.15.38:0/1575589723 <== osd.1 v2:172.21.15.38:6816/1491818732 8 ==== osd_op_reply(92 mds0_inotable [delete] v111'143 uv143 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fdbf801c040 con 0x55f870357480 2024-09-21T03:41:57.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.810+0000 7fdc0d568f40 1 -- 172.21.15.38:0/1575589723 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55f87035a250 con 0x55f870357480 2024-09-21T03:41:57.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.811+0000 7fdc07022640 1 -- 172.21.15.38:0/1575589723 <== osd.1 v2:172.21.15.38:6816/1491818732 9 ==== osd_op_reply(93 mds0_inotable [create] v111'144 uv144 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fdbf801c040 con 0x55f870357480 2024-09-21T03:41:57.814 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-09-21T03:41:57.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.811+0000 7fdc0d568f40 1 -- 172.21.15.38:0/1575589723 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:94 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55f87035a250 con 0x55f870357480 2024-09-21T03:41:57.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.812+0000 7fdc07022640 1 -- 172.21.15.38:0/1575589723 <== osd.1 v2:172.21.15.38:6816/1491818732 10 ==== osd_op_reply(94 mds0_inotable [write 0~34] v111'145 uv145 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fdbf801c040 con 0x55f870357480 2024-09-21T03:41:57.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.812+0000 7fdc0d568f40 1 -- 172.21.15.38:0/1575589723 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:95 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55f87035a250 con 0x55f870357480 2024-09-21T03:41:57.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.813+0000 7fdc07022640 1 -- 172.21.15.38:0/1575589723 <== osd.1 v2:172.21.15.38:6816/1491818732 11 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v111'146 uv146 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fdbf801c040 con 0x55f870357480 2024-09-21T03:41:57.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.813+0000 7fdc0d568f40 1 -- 172.21.15.38:0/1575589723 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:96 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55f87035a250 con 0x55f870357480 2024-09-21T03:41:57.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.814+0000 7fdc07022640 1 -- 172.21.15.38:0/1575589723 <== osd.1 v2:172.21.15.38:6816/1491818732 12 ==== osd_op_reply(96 100.00000000 [create] v111'147 uv141 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fdbf801c040 con 0x55f870357480 2024-09-21T03:41:57.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.814+0000 7fdc0d568f40 1 -- 172.21.15.38:0/1575589723 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:97 13.13 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e111) -- 0x55f87035a250 con 0x55f870357480 2024-09-21T03:41:57.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.815+0000 7fdc07022640 1 -- 172.21.15.38:0/1575589723 <== osd.1 v2:172.21.15.38:6816/1491818732 13 ==== osd_op_reply(97 100.00000000 [delete] v111'148 uv148 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbf801c040 con 0x55f870357480 2024-09-21T03:41:57.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.815+0000 7fdc0d568f40 1 -- 172.21.15.38:0/1575589723 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:98 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55f87035a250 con 0x55f870357480 2024-09-21T03:41:57.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.817+0000 7fdc07022640 1 -- 172.21.15.38:0/1575589723 <== osd.1 v2:172.21.15.38:6816/1491818732 14 ==== osd_op_reply(98 100.00000000 [create] v111'149 uv149 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbf801c040 con 0x55f870357480 2024-09-21T03:41:57.819 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-09-21T03:41:57.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.817+0000 7fdc0d568f40 1 -- 172.21.15.38:0/1575589723 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:99 13.13 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55f87035a250 con 0x55f870357480 2024-09-21T03:41:57.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.818+0000 7fdc07022640 1 -- 172.21.15.38:0/1575589723 <== osd.1 v2:172.21.15.38:6816/1491818732 15 ==== osd_op_reply(99 100.00000000 [omap-set-header] v111'150 uv150 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbf801c040 con 0x55f870357480 2024-09-21T03:41:57.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.818+0000 7fdc0d568f40 1 -- 172.21.15.38:0/1575589723 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:100 13.13 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55f87035d060 con 0x55f870357480 2024-09-21T03:41:57.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.819+0000 7fdc07022640 1 -- 172.21.15.38:0/1575589723 <== osd.1 v2:172.21.15.38:6816/1491818732 16 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v111'151 uv151 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbf801c040 con 0x55f870357480 2024-09-21T03:41:57.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.819+0000 7fdc0d568f40 1 -- 172.21.15.38:0/1575589723 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55f87035a700 con 0x55f870351f20 2024-09-21T03:41:57.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.820+0000 7fdc07823640 1 -- 172.21.15.38:0/1575589723 <== osd.3 v2:172.21.15.38:6808/1286810294 16 ==== osd_op_reply(101 mds0_sessionmap [create] v111'66 uv64 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7fdc0000b040 con 0x55f870351f20 2024-09-21T03:41:57.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.820+0000 7fdc0d568f40 1 -- 172.21.15.38:0/1575589723 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e111) -- 0x55f87035a700 con 0x55f870351f20 2024-09-21T03:41:57.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.821+0000 7fdc07823640 1 -- 172.21.15.38:0/1575589723 <== osd.3 v2:172.21.15.38:6808/1286810294 17 ==== osd_op_reply(102 mds0_sessionmap [delete] v111'67 uv67 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fdc0000b040 con 0x55f870351f20 2024-09-21T03:41:57.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.821+0000 7fdc0d568f40 1 -- 172.21.15.38:0/1575589723 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:103 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55f87035a700 con 0x55f870351f20 2024-09-21T03:41:57.825 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-09-21T03:41:57.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.822+0000 7fdc07823640 1 -- 172.21.15.38:0/1575589723 <== osd.3 v2:172.21.15.38:6808/1286810294 18 ==== osd_op_reply(103 mds0_sessionmap [create] v111'68 uv68 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fdc0000b040 con 0x55f870351f20 2024-09-21T03:41:57.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.822+0000 7fdc0d568f40 1 -- 172.21.15.38:0/1575589723 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:104 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55f87035a700 con 0x55f870351f20 2024-09-21T03:41:57.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.823+0000 7fdc07823640 1 -- 172.21.15.38:0/1575589723 <== osd.3 v2:172.21.15.38:6808/1286810294 19 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v111'69 uv69 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fdc0000b040 con 0x55f870351f20 2024-09-21T03:41:57.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.823+0000 7fdc0d568f40 1 -- 172.21.15.38:0/1575589723 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55f87035a700 con 0x55f870351f20 2024-09-21T03:41:57.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.824+0000 7fdc07823640 1 -- 172.21.15.38:0/1575589723 <== osd.3 v2:172.21.15.38:6808/1286810294 20 ==== osd_op_reply(105 609.00000000 [create] v111'87 uv86 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fdc0000b040 con 0x55f870351f20 2024-09-21T03:41:57.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.824+0000 7fdc0d568f40 1 -- 172.21.15.38:0/1575589723 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e111) -- 0x55f87035a700 con 0x55f870351f20 2024-09-21T03:41:57.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.825+0000 7fdc07823640 1 -- 172.21.15.38:0/1575589723 <== osd.3 v2:172.21.15.38:6808/1286810294 21 ==== osd_op_reply(106 609.00000000 [delete] v111'88 uv88 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdc0000b040 con 0x55f870351f20 2024-09-21T03:41:57.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.825+0000 7fdc0d568f40 1 -- 172.21.15.38:0/1575589723 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55f87035a700 con 0x55f870351f20 2024-09-21T03:41:57.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.826+0000 7fdc07823640 1 -- 172.21.15.38:0/1575589723 <== osd.3 v2:172.21.15.38:6808/1286810294 22 ==== osd_op_reply(107 609.00000000 [create] v111'89 uv89 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdc0000b040 con 0x55f870351f20 2024-09-21T03:41:57.829 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-09-21T03:41:57.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.826+0000 7fdc0d568f40 1 -- 172.21.15.38:0/1575589723 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55f87035b710 con 0x55f870351f20 2024-09-21T03:41:57.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.827+0000 7fdc07823640 1 -- 172.21.15.38:0/1575589723 <== osd.3 v2:172.21.15.38:6808/1286810294 23 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v111'90 uv90 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdc0000b040 con 0x55f870351f20 2024-09-21T03:41:57.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.827+0000 7fdc0d568f40 1 -- 172.21.15.38:0/1575589723 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:109 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55f87035b710 con 0x55f870351f20 2024-09-21T03:41:57.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.828+0000 7fdc07823640 1 -- 172.21.15.38:0/1575589723 <== osd.3 v2:172.21.15.38:6808/1286810294 24 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v111'91 uv91 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdc0000b040 con 0x55f870351f20 2024-09-21T03:41:57.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.828+0000 7fdc0d568f40 1 -- 172.21.15.38:0/1575589723 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:110 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55f87035a700 con 0x55f870351f20 2024-09-21T03:41:57.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.829+0000 7fdc07823640 1 -- 172.21.15.38:0/1575589723 <== osd.3 v2:172.21.15.38:6808/1286810294 25 ==== osd_op_reply(110 609.00000000 [omap-set-header] v111'92 uv92 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdc0000b040 con 0x55f870351f20 2024-09-21T03:41:57.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.830+0000 7fdc0d568f40 1 --2- 172.21.15.38:0/1575589723 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x55f87035ac10 0x55f87035d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:57.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.830+0000 7fdc0d568f40 1 -- 172.21.15.38:0/1575589723 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55f87035d030 con 0x55f87035ac10 2024-09-21T03:41:57.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.830+0000 7fdc06821640 1 --2- 172.21.15.38:0/1575589723 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x55f87035ac10 0x55f87035d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:57.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.830+0000 7fdc06821640 1 --2- 172.21.15.38:0/1575589723 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x55f87035ac10 0x55f87035d560 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:57.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.831+0000 7fdc06821640 1 -- 172.21.15.38:0/1575589723 <== osd.2 v2:172.21.15.38:6824/2577858902 1 ==== osd_op_reply(111 400.00000000 [create] v111'74 uv73 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fdbfc00c140 con 0x55f87035ac10 2024-09-21T03:41:57.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.831+0000 7fdc0d568f40 1 -- 172.21.15.38:0/1575589723 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e111) -- 0x55f87035d030 con 0x55f87035ac10 2024-09-21T03:41:57.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.832+0000 7fdc06821640 1 -- 172.21.15.38:0/1575589723 <== osd.2 v2:172.21.15.38:6824/2577858902 2 ==== osd_op_reply(112 400.00000000 [delete] v111'75 uv75 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbfc00c140 con 0x55f87035ac10 2024-09-21T03:41:57.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.832+0000 7fdc0d568f40 1 -- 172.21.15.38:0/1575589723 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55f87035d030 con 0x55f87035ac10 2024-09-21T03:41:57.836 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-09-21T03:41:57.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.833+0000 7fdc06821640 1 -- 172.21.15.38:0/1575589723 <== osd.2 v2:172.21.15.38:6824/2577858902 3 ==== osd_op_reply(113 400.00000000 [create] v111'76 uv76 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbfc00c140 con 0x55f87035ac10 2024-09-21T03:41:57.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.833+0000 7fdc0d568f40 1 -- 172.21.15.38:0/1575589723 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:114 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55f87035d030 con 0x55f87035ac10 2024-09-21T03:41:57.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.835+0000 7fdc06821640 1 -- 172.21.15.38:0/1575589723 <== osd.2 v2:172.21.15.38:6824/2577858902 4 ==== osd_op_reply(114 400.00000000 [write 0~22] v111'77 uv77 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbfc00c140 con 0x55f87035ac10 2024-09-21T03:41:57.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.835+0000 7fdc0d568f40 1 -- 172.21.15.38:0/1575589723 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:115 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55f87035d030 con 0x55f87035ac10 2024-09-21T03:41:57.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.836+0000 7fdc06821640 1 -- 172.21.15.38:0/1575589723 <== osd.2 v2:172.21.15.38:6824/2577858902 5 ==== osd_op_reply(115 400.00000000 [omap-set-header] v111'78 uv78 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbfc00c140 con 0x55f87035ac10 2024-09-21T03:41:57.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.836+0000 7fdc0d568f40 1 -- 172.21.15.38:0/1575589723 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:116 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55f87035d030 con 0x55f870357480 2024-09-21T03:41:57.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.837+0000 7fdc07022640 1 -- 172.21.15.38:0/1575589723 <== osd.1 v2:172.21.15.38:6816/1491818732 17 ==== osd_op_reply(116 100.00000000.inode [create] v111'182 uv172 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7fdbf801c040 con 0x55f870357480 2024-09-21T03:41:57.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.837+0000 7fdc0d568f40 1 -- 172.21.15.38:0/1575589723 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:117 13.1f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e111) -- 0x55f87035d030 con 0x55f870357480 2024-09-21T03:41:57.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.838+0000 7fdc07022640 1 -- 172.21.15.38:0/1575589723 <== osd.1 v2:172.21.15.38:6816/1491818732 18 ==== osd_op_reply(117 100.00000000.inode [delete] v111'183 uv183 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fdbf801c040 con 0x55f870357480 2024-09-21T03:41:57.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.838+0000 7fdc0d568f40 1 -- 172.21.15.38:0/1575589723 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:118 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55f87035d030 con 0x55f870357480 2024-09-21T03:41:57.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.839+0000 7fdc07022640 1 -- 172.21.15.38:0/1575589723 <== osd.1 v2:172.21.15.38:6816/1491818732 19 ==== osd_op_reply(118 100.00000000.inode [create] v111'184 uv184 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fdbf801c040 con 0x55f870357480 2024-09-21T03:41:57.842 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-09-21T03:41:57.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.839+0000 7fdc0d568f40 1 -- 172.21.15.38:0/1575589723 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:119 13.1f 13:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55f87035d030 con 0x55f870357480 2024-09-21T03:41:57.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.841+0000 7fdc07022640 1 -- 172.21.15.38:0/1575589723 <== osd.1 v2:172.21.15.38:6816/1491818732 20 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v111'185 uv185 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fdbf801c040 con 0x55f870357480 2024-09-21T03:41:57.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.841+0000 7fdc0d568f40 1 -- 172.21.15.38:0/1575589723 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:120 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55f87035d030 con 0x55f870357480 2024-09-21T03:41:57.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.842+0000 7fdc07022640 1 -- 172.21.15.38:0/1575589723 <== osd.1 v2:172.21.15.38:6816/1491818732 21 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v111'186 uv186 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fdbf801c040 con 0x55f870357480 2024-09-21T03:41:57.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.842+0000 7fdc0d568f40 1 -- 172.21.15.38:0/1575589723 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:121 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55f87035d030 con 0x55f870357480 2024-09-21T03:41:57.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.843+0000 7fdc07022640 1 -- 172.21.15.38:0/1575589723 <== osd.1 v2:172.21.15.38:6816/1491818732 22 ==== osd_op_reply(121 1.00000000 [create] v111'187 uv187 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fdbf801c040 con 0x55f870357480 2024-09-21T03:41:57.846 INFO:teuthology.orchestra.run.smithi038.stdout:Write #-9223372036854775808:00000000:::1.00000000:head# 2024-09-21T03:41:57.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.843+0000 7fdc0d568f40 1 -- 172.21.15.38:0/1575589723 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:122 13.1f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55f87035d030 con 0x55f870357480 2024-09-21T03:41:57.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.844+0000 7fdc07022640 1 -- 172.21.15.38:0/1575589723 <== osd.1 v2:172.21.15.38:6816/1491818732 23 ==== osd_op_reply(122 1.00000000 [setxattr (30)] v111'188 uv188 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fdbf801c040 con 0x55f870357480 2024-09-21T03:41:57.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.844+0000 7fdc0d568f40 1 -- 172.21.15.38:0/1575589723 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:123 13.1f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55f87035d030 con 0x55f870357480 2024-09-21T03:41:57.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.846+0000 7fdc07022640 1 -- 172.21.15.38:0/1575589723 <== osd.1 v2:172.21.15.38:6816/1491818732 24 ==== osd_op_reply(123 1.00000000 [setxattr (30)] v111'189 uv189 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fdbf801c040 con 0x55f870357480 2024-09-21T03:41:57.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.846+0000 7fdc0d568f40 1 -- 172.21.15.38:0/1575589723 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:124 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55f87035d030 con 0x55f870357480 2024-09-21T03:41:57.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.847+0000 7fdc07022640 1 -- 172.21.15.38:0/1575589723 <== osd.1 v2:172.21.15.38:6816/1491818732 25 ==== osd_op_reply(124 1.00000000 [omap-set-header] v111'190 uv190 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fdbf801c040 con 0x55f870357480 2024-09-21T03:41:57.850 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.847+0000 7fdc0d568f40 1 -- 172.21.15.38:0/1575589723 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:125 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=584b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55f87035dc00 con 0x55f870357480 2024-09-21T03:41:57.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.848+0000 7fdc07022640 1 -- 172.21.15.38:0/1575589723 <== osd.1 v2:172.21.15.38:6816/1491818732 26 ==== osd_op_reply(125 1.00000000 [omap-set-vals] v111'191 uv191 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fdbf801c040 con 0x55f870357480 2024-09-21T03:41:57.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.848+0000 7fdc0d568f40 1 -- 172.21.15.38:0/1575589723 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x55f87034ea20 msgr2=0x55f870350e40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:57.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.848+0000 7fdc0d568f40 1 --2- 172.21.15.38:0/1575589723 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x55f87034ea20 0x55f870350e40 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:57.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.848+0000 7fdc0d568f40 1 -- 172.21.15.38:0/1575589723 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x55f870357480 msgr2=0x55f8703598a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:57.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.848+0000 7fdc0d568f40 1 --2- 172.21.15.38:0/1575589723 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x55f870357480 0x55f8703598a0 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:57.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.848+0000 7fdc0d568f40 1 -- 172.21.15.38:0/1575589723 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x55f87035ac10 msgr2=0x55f87035d560 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:57.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.848+0000 7fdc0d568f40 1 --2- 172.21.15.38:0/1575589723 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x55f87035ac10 0x55f87035d560 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:57.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.848+0000 7fdc0d568f40 1 -- 172.21.15.38:0/1575589723 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x55f870351f20 msgr2=0x55f870354340 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:57.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.848+0000 7fdc0d568f40 1 --2- 172.21.15.38:0/1575589723 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x55f870351f20 0x55f870354340 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:57.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.848+0000 7fdc07823640 1 -- 172.21.15.38:0/1575589723 reap_dead start 2024-09-21T03:41:57.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.848+0000 7fdc0d568f40 1 -- 172.21.15.38:0/1575589723 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x55f870349640 msgr2=0x55f87034c790 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:57.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.848+0000 7fdc0d568f40 1 --2- 172.21.15.38:0/1575589723 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x55f870349640 0x55f87034c790 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:57.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.848+0000 7fdc07823640 1 -- 172.21.15.38:0/1575589723 reap_dead start 2024-09-21T03:41:57.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.848+0000 7fdc0d568f40 1 -- 172.21.15.38:0/1575589723 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x55f870284270 msgr2=0x55f8701cad10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:57.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.848+0000 7fdc0d568f40 1 --2- 172.21.15.38:0/1575589723 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x55f870284270 0x55f8701cad10 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:57.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.848+0000 7fdc0d568f40 1 -- 172.21.15.38:0/1575589723 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fdbf4077570 msgr2=0x7fdbf4079a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:57.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.848+0000 7fdc0d568f40 1 --2- 172.21.15.38:0/1575589723 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fdbf4077570 0x7fdbf4079a80 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fdbf8017040 tx=0x7fdbf80095a0 comp rx=0 tx=0).stop 2024-09-21T03:41:57.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.848+0000 7fdc0d568f40 1 -- 172.21.15.38:0/1575589723 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f8701fccc0 msgr2=0x55f8702a1df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:57.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.848+0000 7fdc0d568f40 1 --2- 172.21.15.38:0/1575589723 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f8701fccc0 0x55f8702a1df0 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7fdc0000a640 tx=0x7fdc00051bd0 comp rx=0 tx=0).stop 2024-09-21T03:41:57.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.848+0000 7fdc0d568f40 1 -- 172.21.15.38:0/1575589723 shutdown_connections 2024-09-21T03:41:57.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.848+0000 7fdc0d568f40 1 --2- 172.21.15.38:0/1575589723 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x55f870284270 0x55f8701cad10 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:57.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.848+0000 7fdc0d568f40 1 --2- 172.21.15.38:0/1575589723 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f8701fccc0 0x55f8702a1df0 unknown :-1 s=CLOSED pgs=1596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:57.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.848+0000 7fdc0d568f40 1 --2- 172.21.15.38:0/1575589723 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fdbf4077570 0x7fdbf4079a80 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:57.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.848+0000 7fdc0d568f40 1 -- 172.21.15.38:0/1575589723 >> 172.21.15.38:0/1575589723 conn(0x55f8700edac0 msgr2=0x55f8701ceca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:57.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.848+0000 7fdc0d568f40 1 -- 172.21.15.38:0/1575589723 shutdown_connections 2024-09-21T03:41:57.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.848+0000 7fdc0d568f40 1 -- 172.21.15.38:0/1575589723 wait complete. 2024-09-21T03:41:57.855 DEBUG:teuthology.orchestra.run.smithi038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata put 1.00000000.inode - 2024-09-21T03:41:57.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.909+0000 7f9b2abdaf40 1 Processor -- start 2024-09-21T03:41:57.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.909+0000 7f9b2abdaf40 1 -- start start 2024-09-21T03:41:57.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.911+0000 7f9b2abdaf40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556000ee3d00 0x556000fb8390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:57.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.911+0000 7f9b2abdaf40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x556000f115a0 0x556000fb7130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:57.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.911+0000 7f9b2abdaf40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556000f2ae60 0x556000fb7670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:57.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.911+0000 7f9b2abdaf40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x556000e90430 con 0x556000f2ae60 2024-09-21T03:41:57.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.911+0000 7f9b2abdaf40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x556000e9d1f0 con 0x556000ee3d00 2024-09-21T03:41:57.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.911+0000 7f9b2abdaf40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x556000e31d60 con 0x556000f115a0 2024-09-21T03:41:57.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.911+0000 7f9b24e95640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556000f2ae60 0x556000fb7670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:57.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.911+0000 7f9b24e95640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556000f2ae60 0x556000fb7670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53184/0 (socket says 172.21.15.38:53184) 2024-09-21T03:41:57.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.911+0000 7f9b24e95640 1 -- 172.21.15.38:0/2743869373 learned_addr learned my addr 172.21.15.38:0/2743869373 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:41:57.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.911+0000 7f9b24694640 1 --2- 172.21.15.38:0/2743869373 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556000ee3d00 0x556000fb8390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:57.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.911+0000 7f9b24e95640 1 -- 172.21.15.38:0/2743869373 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x556000f115a0 msgr2=0x556000fb7130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:57.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.911+0000 7f9b23e93640 1 --2- 172.21.15.38:0/2743869373 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x556000f115a0 0x556000fb7130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:57.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.911+0000 7f9b24e95640 1 --2- 172.21.15.38:0/2743869373 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x556000f115a0 0x556000fb7130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:57.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.911+0000 7f9b24e95640 1 -- 172.21.15.38:0/2743869373 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556000ee3d00 msgr2=0x556000fb8390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:57.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.911+0000 7f9b24e95640 1 --2- 172.21.15.38:0/2743869373 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556000ee3d00 0x556000fb8390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:57.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.911+0000 7f9b24e95640 1 -- 172.21.15.38:0/2743869373 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556001044470 con 0x556000f2ae60 2024-09-21T03:41:57.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.911+0000 7f9b24694640 1 --2- 172.21.15.38:0/2743869373 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556000ee3d00 0x556000fb8390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:41:57.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.911+0000 7f9b23e93640 1 --2- 172.21.15.38:0/2743869373 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x556000f115a0 0x556000fb7130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:41:57.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.911+0000 7f9b24e95640 1 --2- 172.21.15.38:0/2743869373 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556000f2ae60 0x556000fb7670 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7f9b1c001540 tx=0x7f9b1c001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5263e729702c699b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:57.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.911+0000 7f9b22e91640 1 -- 172.21.15.38:0/2743869373 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b1c017070 con 0x556000f2ae60 2024-09-21T03:41:57.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.911+0000 7f9b22e91640 1 -- 172.21.15.38:0/2743869373 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9b1c0027a0 con 0x556000f2ae60 2024-09-21T03:41:57.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.911+0000 7f9b22e91640 1 -- 172.21.15.38:0/2743869373 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b1c002c30 con 0x556000f2ae60 2024-09-21T03:41:57.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.912+0000 7f9b2abdaf40 1 -- 172.21.15.38:0/2743869373 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556000f2ae60 msgr2=0x556000fb7670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:57.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.912+0000 7f9b2abdaf40 1 --2- 172.21.15.38:0/2743869373 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556000f2ae60 0x556000fb7670 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7f9b1c001540 tx=0x7f9b1c001a00 comp rx=0 tx=0).stop 2024-09-21T03:41:57.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.912+0000 7f9b2abdaf40 1 -- 172.21.15.38:0/2743869373 shutdown_connections 2024-09-21T03:41:57.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.912+0000 7f9b2abdaf40 1 --2- 172.21.15.38:0/2743869373 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556000f2ae60 0x556000fb7670 unknown :-1 s=CLOSED pgs=1599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:57.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.912+0000 7f9b2abdaf40 1 --2- 172.21.15.38:0/2743869373 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x556000f115a0 0x556000fb7130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:57.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.912+0000 7f9b2abdaf40 1 --2- 172.21.15.38:0/2743869373 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556000ee3d00 0x556000fb8390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:57.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.912+0000 7f9b2abdaf40 1 -- 172.21.15.38:0/2743869373 >> 172.21.15.38:0/2743869373 conn(0x556000e02ac0 msgr2=0x556000fb1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:57.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.912+0000 7f9b2abdaf40 1 -- 172.21.15.38:0/2743869373 shutdown_connections 2024-09-21T03:41:57.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.912+0000 7f9b2abdaf40 1 -- 172.21.15.38:0/2743869373 wait complete. 2024-09-21T03:41:57.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.913+0000 7f9b2abdaf40 1 Processor -- start 2024-09-21T03:41:57.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.913+0000 7f9b2abdaf40 1 -- start start 2024-09-21T03:41:57.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.914+0000 7f9b2abdaf40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x556000ee3d00 0x556000fa9800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:57.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.914+0000 7f9b2abdaf40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556000f115a0 0x556001043730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:57.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.914+0000 7f9b2abdaf40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556000f2ae60 0x556001043c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:57.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.914+0000 7f9b2abdaf40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x556000fba900 con 0x556000f115a0 2024-09-21T03:41:57.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.914+0000 7f9b2abdaf40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x556000fbabe0 con 0x556000f2ae60 2024-09-21T03:41:57.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.914+0000 7f9b2abdaf40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x556000fb5780 con 0x556000ee3d00 2024-09-21T03:41:57.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.914+0000 7f9b23e93640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556000f115a0 0x556001043730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:57.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.914+0000 7f9b23e93640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556000f115a0 0x556001043730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53194/0 (socket says 172.21.15.38:53194) 2024-09-21T03:41:57.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.914+0000 7f9b23e93640 1 -- 172.21.15.38:0/194767291 learned_addr learned my addr 172.21.15.38:0/194767291 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:41:57.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.914+0000 7f9b24e95640 1 --2- 172.21.15.38:0/194767291 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556000f2ae60 0x556001043c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:57.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.914+0000 7f9b24694640 1 --2- 172.21.15.38:0/194767291 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x556000ee3d00 0x556000fa9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:57.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.915+0000 7f9b23e93640 1 -- 172.21.15.38:0/194767291 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x556000ee3d00 msgr2=0x556000fa9800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:57.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.915+0000 7f9b23e93640 1 --2- 172.21.15.38:0/194767291 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x556000ee3d00 0x556000fa9800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:57.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.915+0000 7f9b23e93640 1 -- 172.21.15.38:0/194767291 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556000f2ae60 msgr2=0x556001043c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:57.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.915+0000 7f9b23e93640 1 --2- 172.21.15.38:0/194767291 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556000f2ae60 0x556001043c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:57.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.915+0000 7f9b23e93640 1 -- 172.21.15.38:0/194767291 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556000f0e460 con 0x556000f115a0 2024-09-21T03:41:57.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.915+0000 7f9b23e93640 1 --2- 172.21.15.38:0/194767291 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556000f115a0 0x556001043730 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7f9b18025fd0 tx=0x7f9b18056c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:57.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.915+0000 7f9b21e8f640 1 -- 172.21.15.38:0/194767291 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b18067070 con 0x556000f115a0 2024-09-21T03:41:57.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.915+0000 7f9b21e8f640 1 -- 172.21.15.38:0/194767291 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9b18062470 con 0x556000f115a0 2024-09-21T03:41:57.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.915+0000 7f9b2abdaf40 1 -- 172.21.15.38:0/194767291 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x556000fa9fb0 con 0x556000f115a0 2024-09-21T03:41:57.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.915+0000 7f9b21e8f640 1 -- 172.21.15.38:0/194767291 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b18004580 con 0x556000f115a0 2024-09-21T03:41:57.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.915+0000 7f9b2abdaf40 1 -- 172.21.15.38:0/194767291 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556000faa6b0 con 0x556000f115a0 2024-09-21T03:41:57.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.917+0000 7f9b21e8f640 1 -- 172.21.15.38:0/194767291 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f9b1806a070 con 0x556000f115a0 2024-09-21T03:41:57.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.917+0000 7f9b21e8f640 1 --2- 172.21.15.38:0/194767291 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9b10077570 0x7f9b10079a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:57.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.917+0000 7f9b21e8f640 1 -- 172.21.15.38:0/194767291 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 7594+0+0 (secure 0 0 0) 0x7f9b180de540 con 0x556000f115a0 2024-09-21T03:41:57.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.917+0000 7f9b2abdaf40 1 --2- 172.21.15.38:0/194767291 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x556000fbb8c0 0x556000fb3ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:57.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.917+0000 7f9b2abdaf40 1 -- 172.21.15.38:0/194767291 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:1 13.14 13:287034c4:::1.00000000.inode:head [writefull 0~80 in=80b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x556000fb8640 con 0x556000fbb8c0 2024-09-21T03:41:57.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.917+0000 7f9b24694640 1 --2- 172.21.15.38:0/194767291 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9b10077570 0x7f9b10079a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:57.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.918+0000 7f9b24e95640 1 --2- 172.21.15.38:0/194767291 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x556000fbb8c0 0x556000fb3ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:57.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.918+0000 7f9b24694640 1 --2- 172.21.15.38:0/194767291 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9b10077570 0x7f9b10079a80 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f9b14000c00 tx=0x7f9b14001820 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:57.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.918+0000 7f9b24e95640 1 --2- 172.21.15.38:0/194767291 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x556000fbb8c0 0x556000fb3ea0 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:57.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.920+0000 7f9b24e95640 1 -- 172.21.15.38:0/194767291 <== osd.6 v2:172.21.15.195:6826/4055825675 1 ==== osd_op_reply(1 1.00000000.inode [writefull 0~80] v111'350 uv350 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f9b1c00f070 con 0x556000fbb8c0 2024-09-21T03:41:57.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.920+0000 7f9b2abdaf40 1 -- 172.21.15.38:0/194767291 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x556000fbb8c0 msgr2=0x556000fb3ea0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:57.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.920+0000 7f9b2abdaf40 1 --2- 172.21.15.38:0/194767291 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x556000fbb8c0 0x556000fb3ea0 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:57.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.920+0000 7f9b2abdaf40 1 -- 172.21.15.38:0/194767291 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9b10077570 msgr2=0x7f9b10079a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:57.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.920+0000 7f9b2abdaf40 1 --2- 172.21.15.38:0/194767291 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9b10077570 0x7f9b10079a80 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f9b14000c00 tx=0x7f9b14001820 comp rx=0 tx=0).stop 2024-09-21T03:41:57.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.920+0000 7f9b2abdaf40 1 -- 172.21.15.38:0/194767291 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556000f115a0 msgr2=0x556001043730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:57.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.920+0000 7f9b2abdaf40 1 --2- 172.21.15.38:0/194767291 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556000f115a0 0x556001043730 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7f9b18025fd0 tx=0x7f9b18056c50 comp rx=0 tx=0).stop 2024-09-21T03:41:57.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.920+0000 7f9b24e95640 1 -- 172.21.15.38:0/194767291 reap_dead start 2024-09-21T03:41:57.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.920+0000 7f9b2abdaf40 1 -- 172.21.15.38:0/194767291 shutdown_connections 2024-09-21T03:41:57.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.920+0000 7f9b2abdaf40 1 -- 172.21.15.38:0/194767291 >> 172.21.15.38:0/194767291 conn(0x556000e02ac0 msgr2=0x556000edff70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:57.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.920+0000 7f9b2abdaf40 1 -- 172.21.15.38:0/194767291 shutdown_connections 2024-09-21T03:41:57.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:57.920+0000 7f9b2abdaf40 1 -- 172.21.15.38:0/194767291 wait complete. 2024-09-21T03:41:57.926 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-21T03:41:58.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:58.084+0000 7f56a32ec640 1 -- 172.21.15.38:0/4075423425 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f569c0ccc40 msgr2=0x7f569c15ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:58.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:58.084+0000 7f56a32ec640 1 --2- 172.21.15.38:0/4075423425 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f569c0ccc40 0x7f569c15ca90 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7f569c0cd050 tx=0x7f569802a560 comp rx=0 tx=0).stop 2024-09-21T03:41:58.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:58.084+0000 7f56a32ec640 1 -- 172.21.15.38:0/4075423425 shutdown_connections 2024-09-21T03:41:58.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:58.084+0000 7f56a32ec640 1 --2- 172.21.15.38:0/4075423425 >> v2:172.21.15.195:3300/0 conn(0x7f569c0d0d80 0x7f569c0b5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:58.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:58.084+0000 7f56a32ec640 1 --2- 172.21.15.38:0/4075423425 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f569c0ccc40 0x7f569c15ca90 unknown :-1 s=CLOSED pgs=1601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:58.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:58.084+0000 7f56a32ec640 1 -- 172.21.15.38:0/4075423425 >> 172.21.15.38:0/4075423425 conn(0x7f569c0b46e0 msgr2=0x7f569c0d23e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:58.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:58.084+0000 7f56a32ec640 1 -- 172.21.15.38:0/4075423425 shutdown_connections 2024-09-21T03:41:58.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:58.085+0000 7f56a32ec640 1 -- 172.21.15.38:0/4075423425 wait complete. 2024-09-21T03:41:58.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:58.086+0000 7f56a32ec640 1 Processor -- start 2024-09-21T03:41:58.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:58.086+0000 7f56a32ec640 1 -- start start 2024-09-21T03:41:58.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:58.087+0000 7f56a32ec640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f569c0ccc40 0x7f569c0c7f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:58.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:58.088+0000 7f56a32ec640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f569c0cd440 0x7f569c0c8460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:58.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:58.088+0000 7f56a32ec640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f569c1f3530 0x7f569c1f3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:58.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:58.088+0000 7f56a32ec640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f569c0b6650 con 0x7f569c0ccc40 2024-09-21T03:41:58.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:58.088+0000 7f56a32ec640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f569c0b64d0 con 0x7f569c0cd440 2024-09-21T03:41:58.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:58.088+0000 7f56a32ec640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f569c0b67d0 con 0x7f569c1f3530 2024-09-21T03:41:58.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:58.088+0000 7f56a1088640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f569c0ccc40 0x7f569c0c7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:58.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:58.088+0000 7f56a1088640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f569c0ccc40 0x7f569c0c7f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53220/0 (socket says 172.21.15.38:53220) 2024-09-21T03:41:58.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:58.088+0000 7f56a1088640 1 -- 172.21.15.38:0/3359297191 learned_addr learned my addr 172.21.15.38:0/3359297191 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:41:58.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:58.088+0000 7f56a0887640 1 --2- 172.21.15.38:0/3359297191 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f569c0cd440 0x7f569c0c8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:58.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:58.088+0000 7f56a1889640 1 --2- 172.21.15.38:0/3359297191 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f569c1f3530 0x7f569c1f3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:58.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:58.088+0000 7f56a1088640 1 -- 172.21.15.38:0/3359297191 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f569c1f3530 msgr2=0x7f569c1f3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:58.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:58.088+0000 7f56a1088640 1 --2- 172.21.15.38:0/3359297191 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f569c1f3530 0x7f569c1f3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:58.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:58.088+0000 7f56a1088640 1 -- 172.21.15.38:0/3359297191 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f569c0cd440 msgr2=0x7f569c0c8460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:58.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:58.088+0000 7f56a1088640 1 --2- 172.21.15.38:0/3359297191 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f569c0cd440 0x7f569c0c8460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:58.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:58.088+0000 7f56a1088640 1 -- 172.21.15.38:0/3359297191 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f569c0c76f0 con 0x7f569c0ccc40 2024-09-21T03:41:58.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:58.088+0000 7f56a1088640 1 --2- 172.21.15.38:0/3359297191 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f569c0ccc40 0x7f569c0c7f20 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f5698034040 tx=0x7f56980038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:58.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:58.088+0000 7f568dffb640 1 -- 172.21.15.38:0/3359297191 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f569803e030 con 0x7f569c0ccc40 2024-09-21T03:41:58.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:58.088+0000 7f568dffb640 1 -- 172.21.15.38:0/3359297191 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5698003db0 con 0x7f569c0ccc40 2024-09-21T03:41:58.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:58.088+0000 7f56a32ec640 1 -- 172.21.15.38:0/3359297191 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f569c0aa430 con 0x7f569c0ccc40 2024-09-21T03:41:58.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:58.088+0000 7f568dffb640 1 -- 172.21.15.38:0/3359297191 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5698035040 con 0x7f569c0ccc40 2024-09-21T03:41:58.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:58.089+0000 7f56a32ec640 1 -- 172.21.15.38:0/3359297191 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f569c1f4380 con 0x7f569c0ccc40 2024-09-21T03:41:58.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:58.090+0000 7f56a32ec640 1 -- 172.21.15.38:0/3359297191 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f569c0b74b0 con 0x7f569c0ccc40 2024-09-21T03:41:58.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:58.091+0000 7f568dffb640 1 -- 172.21.15.38:0/3359297191 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f5698041030 con 0x7f569c0ccc40 2024-09-21T03:41:58.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:58.091+0000 7f568dffb640 1 --2- 172.21.15.38:0/3359297191 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5680078b90 0x7f568007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:58.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:58.091+0000 7f568dffb640 1 -- 172.21.15.38:0/3359297191 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 7594+0+0 (secure 0 0 0) 0x7f56980bc0b0 con 0x7f569c0ccc40 2024-09-21T03:41:58.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:58.091+0000 7f56a0887640 1 --2- 172.21.15.38:0/3359297191 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5680078b90 0x7f568007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:58.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:58.092+0000 7f56a0887640 1 --2- 172.21.15.38:0/3359297191 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5680078b90 0x7f568007b0a0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f569c15c760 tx=0x7f5690009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:58.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:58.095+0000 7f568dffb640 1 -- 172.21.15.38:0/3359297191 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f5698083820 con 0x7f569c0ccc40 2024-09-21T03:41:58.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:58.321+0000 7f56a32ec640 1 -- 172.21.15.38:0/3359297191 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f569c070e40 con 0x7f569c0ccc40 2024-09-21T03:41:58.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:58.817+0000 7f568dffb640 1 -- 172.21.15.38:0/3359297191 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v177) ==== 162+0+0 (secure 0 0 0) 0x7f5698088250 con 0x7f569c0ccc40 2024-09-21T03:41:58.819 INFO:teuthology.orchestra.run.smithi038.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-21T03:41:58.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:58.820+0000 7f56a32ec640 1 -- 172.21.15.38:0/3359297191 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5680078b90 msgr2=0x7f568007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:58.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:58.820+0000 7f56a32ec640 1 --2- 172.21.15.38:0/3359297191 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5680078b90 0x7f568007b0a0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f569c15c760 tx=0x7f5690009040 comp rx=0 tx=0).stop 2024-09-21T03:41:58.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:58.820+0000 7f56a32ec640 1 -- 172.21.15.38:0/3359297191 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f569c0ccc40 msgr2=0x7f569c0c7f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:58.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:58.820+0000 7f56a32ec640 1 --2- 172.21.15.38:0/3359297191 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f569c0ccc40 0x7f569c0c7f20 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f5698034040 tx=0x7f56980038a0 comp rx=0 tx=0).stop 2024-09-21T03:41:58.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:58.820+0000 7f56a32ec640 1 -- 172.21.15.38:0/3359297191 shutdown_connections 2024-09-21T03:41:58.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:58.820+0000 7f56a32ec640 1 --2- 172.21.15.38:0/3359297191 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5680078b90 0x7f568007b0a0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:58.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:58.820+0000 7f56a32ec640 1 --2- 172.21.15.38:0/3359297191 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f569c1f3530 0x7f569c1f3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:58.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:58.820+0000 7f56a32ec640 1 --2- 172.21.15.38:0/3359297191 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f569c0cd440 0x7f569c0c8460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:58.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:58.820+0000 7f56a32ec640 1 --2- 172.21.15.38:0/3359297191 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f569c0ccc40 0x7f569c0c7f20 unknown :-1 s=CLOSED pgs=1602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:58.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:58.820+0000 7f56a32ec640 1 -- 172.21.15.38:0/3359297191 >> 172.21.15.38:0/3359297191 conn(0x7f569c0b46e0 msgr2=0x7f569c0b2630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:58.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:58.820+0000 7f56a32ec640 1 -- 172.21.15.38:0/3359297191 shutdown_connections 2024-09-21T03:41:58.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:58.820+0000 7f56a32ec640 1 -- 172.21.15.38:0/3359297191 wait complete. 2024-09-21T03:41:58.836 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:41:58.839 INFO:tasks.ceph.mon.a.smithi038.stderr:2024-09-21T03:41:58.836+0000 7f3f45713640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T03:41:58.839 INFO:tasks.ceph.mon.a.smithi038.stderr:2024-09-21T03:41:58.836+0000 7f3f45713640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-09-21T03:41:58.839 INFO:tasks.ceph.mds.c.smithi038.stderr:2024-09-21T03:41:58.835+0000 7f34dd612640 -1 mds.0.cache.ino(0x1) Corrupt inode 0x1: End of buffer [buffer:2] 2024-09-21T03:41:58.839 INFO:tasks.ceph.mds.c.smithi038.stderr:2024-09-21T03:41:58.835+0000 7f34dd612640 -1 log_channel(cluster) log [ERR] : Error loading MDS rank 0: (22) Invalid argument 2024-09-21T03:41:58.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:58.953+0000 7fd4d952b640 1 -- 172.21.15.38:0/4185863768 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd4d40d19f0 msgr2=0x7fd4d415cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:58.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:58.953+0000 7fd4d952b640 1 --2- 172.21.15.38:0/4185863768 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd4d40d19f0 0x7fd4d415cf20 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7fd4d40b3cf0 tx=0x7fd4bc02a5f0 comp rx=0 tx=0).stop 2024-09-21T03:41:58.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:58.953+0000 7fd4d952b640 1 -- 172.21.15.38:0/4185863768 shutdown_connections 2024-09-21T03:41:58.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:58.953+0000 7fd4d952b640 1 --2- 172.21.15.38:0/4185863768 >> v2:172.21.15.195:3300/0 conn(0x7fd4d40b6250 0x7fd4d40b6660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:58.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:58.953+0000 7fd4d952b640 1 --2- 172.21.15.38:0/4185863768 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd4d40d19f0 0x7fd4d415cf20 unknown :-1 s=CLOSED pgs=1603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:58.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:58.953+0000 7fd4d952b640 1 -- 172.21.15.38:0/4185863768 >> 172.21.15.38:0/4185863768 conn(0x7fd4d40d2350 msgr2=0x7fd4d40d2760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:58.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:58.953+0000 7fd4d952b640 1 -- 172.21.15.38:0/4185863768 shutdown_connections 2024-09-21T03:41:58.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:58.953+0000 7fd4d952b640 1 -- 172.21.15.38:0/4185863768 wait complete. 2024-09-21T03:41:58.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:58.955+0000 7fd4d952b640 1 Processor -- start 2024-09-21T03:41:58.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:58.955+0000 7fd4d952b640 1 -- start start 2024-09-21T03:41:58.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:58.956+0000 7fd4d952b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd4d40b6250 0x7fd4d4149d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:58.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:58.956+0000 7fd4d952b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd4d40d19f0 0x7fd4d415bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:58.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:58.956+0000 7fd4d952b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd4d415cb10 0x7fd4d406eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:58.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:58.956+0000 7fd4d952b640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fd4d40b6d20 con 0x7fd4d40d19f0 2024-09-21T03:41:58.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:58.956+0000 7fd4d952b640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd4d40b6ba0 con 0x7fd4d40b6250 2024-09-21T03:41:58.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:58.956+0000 7fd4d952b640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fd4d40b6ea0 con 0x7fd4d415cb10 2024-09-21T03:41:58.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:58.956+0000 7fd4d27fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd4d40d19f0 0x7fd4d415bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:58.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:58.956+0000 7fd4d27fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd4d40d19f0 0x7fd4d415bdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53234/0 (socket says 172.21.15.38:53234) 2024-09-21T03:41:58.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:58.956+0000 7fd4d27fc640 1 -- 172.21.15.38:0/2135609483 learned_addr learned my addr 172.21.15.38:0/2135609483 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:41:58.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:58.956+0000 7fd4d2ffd640 1 --2- 172.21.15.38:0/2135609483 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd4d40b6250 0x7fd4d4149d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:58.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:58.956+0000 7fd4d37fe640 1 --2- 172.21.15.38:0/2135609483 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd4d415cb10 0x7fd4d406eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:58.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:58.957+0000 7fd4d27fc640 1 -- 172.21.15.38:0/2135609483 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd4d415cb10 msgr2=0x7fd4d406eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:58.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:58.957+0000 7fd4d27fc640 1 --2- 172.21.15.38:0/2135609483 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd4d415cb10 0x7fd4d406eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:58.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:58.957+0000 7fd4d27fc640 1 -- 172.21.15.38:0/2135609483 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd4d40b6250 msgr2=0x7fd4d4149d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:58.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:58.957+0000 7fd4d27fc640 1 --2- 172.21.15.38:0/2135609483 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd4d40b6250 0x7fd4d4149d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:58.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:58.957+0000 7fd4d27fc640 1 -- 172.21.15.38:0/2135609483 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4d406e100 con 0x7fd4d40d19f0 2024-09-21T03:41:58.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:58.957+0000 7fd4d27fc640 1 --2- 172.21.15.38:0/2135609483 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd4d40d19f0 0x7fd4d415bdb0 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7fd4bc0099c0 tx=0x7fd4bc0038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:58.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:58.957+0000 7fd4cb7fe640 1 -- 172.21.15.38:0/2135609483 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4bc03e030 con 0x7fd4d40d19f0 2024-09-21T03:41:58.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:58.957+0000 7fd4cb7fe640 1 -- 172.21.15.38:0/2135609483 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd4bc003db0 con 0x7fd4d40d19f0 2024-09-21T03:41:58.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:58.957+0000 7fd4d952b640 1 -- 172.21.15.38:0/2135609483 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4d406e3e0 con 0x7fd4d40d19f0 2024-09-21T03:41:58.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:58.957+0000 7fd4cb7fe640 1 -- 172.21.15.38:0/2135609483 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4bc035040 con 0x7fd4d40d19f0 2024-09-21T03:41:58.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:58.957+0000 7fd4d952b640 1 -- 172.21.15.38:0/2135609483 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4d414a2b0 con 0x7fd4d40d19f0 2024-09-21T03:41:58.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:58.958+0000 7fd4d952b640 1 -- 172.21.15.38:0/2135609483 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4d40b7350 con 0x7fd4d40d19f0 2024-09-21T03:41:58.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:58.959+0000 7fd4cb7fe640 1 -- 172.21.15.38:0/2135609483 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fd4bc041030 con 0x7fd4d40d19f0 2024-09-21T03:41:58.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:58.960+0000 7fd4cb7fe640 1 --2- 172.21.15.38:0/2135609483 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd4b4078b90 0x7fd4b407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:58.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:58.960+0000 7fd4cb7fe640 1 -- 172.21.15.38:0/2135609483 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 7680+0+0 (secure 0 0 0) 0x7fd4bc0bbf20 con 0x7fd4d40d19f0 2024-09-21T03:41:58.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:58.960+0000 7fd4d2ffd640 1 --2- 172.21.15.38:0/2135609483 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd4b4078b90 0x7fd4b407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:58.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:58.960+0000 7fd4d2ffd640 1 --2- 172.21.15.38:0/2135609483 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd4b4078b90 0x7fd4b407b0a0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fd4c4001e20 tx=0x7fd4c4009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:58.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:58.964+0000 7fd4cb7fe640 1 -- 172.21.15.38:0/2135609483 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fd4bc0834e0 con 0x7fd4d40d19f0 2024-09-21T03:41:59.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:59.193+0000 7fd4d952b640 1 -- 172.21.15.38:0/2135609483 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd4d40b56b0 con 0x7fd4d40d19f0 2024-09-21T03:41:59.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:59.194+0000 7fd4cb7fe640 1 -- 172.21.15.38:0/2135609483 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 179 v179) ==== 95+0+4392 (secure 0 0 0) 0x7fd4bc087f10 con 0x7fd4d40d19f0 2024-09-21T03:41:59.196 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:41:59.196 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":179,"btime":"2024-09-21T03:41:58:837196+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6086,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/965972663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":965972663},{"type":"v1","addr":"172.21.15.195:6836","nonce":965972663}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6700,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/3176122334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":3176122334},{"type":"v1","addr":"172.21.15.195:6837","nonce":3176122334}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":8661,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/2418225210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":2418225210},{"type":"v1","addr":"172.21.15.38:6835","nonce":2418225210}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":171}],"filesystems":[{"mdsmap":{"epoch":179,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:40:02.319282+0000","modified":"2024-09-21T03:41:58.837120+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":112,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-21T03:41:59.196 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 179 2024-09-21T03:41:59.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:59.196+0000 7fd4d952b640 1 -- 172.21.15.38:0/2135609483 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd4b4078b90 msgr2=0x7fd4b407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:59.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:59.196+0000 7fd4d952b640 1 --2- 172.21.15.38:0/2135609483 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd4b4078b90 0x7fd4b407b0a0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fd4c4001e20 tx=0x7fd4c4009040 comp rx=0 tx=0).stop 2024-09-21T03:41:59.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:59.196+0000 7fd4d952b640 1 -- 172.21.15.38:0/2135609483 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd4d40d19f0 msgr2=0x7fd4d415bdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:59.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:59.196+0000 7fd4d952b640 1 --2- 172.21.15.38:0/2135609483 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd4d40d19f0 0x7fd4d415bdb0 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7fd4bc0099c0 tx=0x7fd4bc0038a0 comp rx=0 tx=0).stop 2024-09-21T03:41:59.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:59.196+0000 7fd4d952b640 1 -- 172.21.15.38:0/2135609483 shutdown_connections 2024-09-21T03:41:59.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:59.196+0000 7fd4d952b640 1 --2- 172.21.15.38:0/2135609483 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd4b4078b90 0x7fd4b407b0a0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:59.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:59.196+0000 7fd4d952b640 1 --2- 172.21.15.38:0/2135609483 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd4d415cb10 0x7fd4d406eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:59.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:59.196+0000 7fd4d952b640 1 --2- 172.21.15.38:0/2135609483 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd4d40d19f0 0x7fd4d415bdb0 unknown :-1 s=CLOSED pgs=1604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:59.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:59.196+0000 7fd4d952b640 1 --2- 172.21.15.38:0/2135609483 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd4d40b6250 0x7fd4d4149d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:59.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:59.196+0000 7fd4d952b640 1 -- 172.21.15.38:0/2135609483 >> 172.21.15.38:0/2135609483 conn(0x7fd4d40d2350 msgr2=0x7fd4d415e190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:59.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:59.196+0000 7fd4d952b640 1 -- 172.21.15.38:0/2135609483 shutdown_connections 2024-09-21T03:41:59.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:59.196+0000 7fd4d952b640 1 -- 172.21.15.38:0/2135609483 wait complete. 2024-09-21T03:41:59.209 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-21T03:41:59.210 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:41:59.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:59.324+0000 7f9fafbc2640 1 -- 172.21.15.38:0/55216680 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9fa8162f00 msgr2=0x7f9fa80d24d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:59.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:59.324+0000 7f9fafbc2640 1 --2- 172.21.15.38:0/55216680 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9fa8162f00 0x7f9fa80d24d0 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7f9fa80b7460 tx=0x7f9f9802a820 comp rx=0 tx=0).stop 2024-09-21T03:41:59.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:59.324+0000 7f9fafbc2640 1 -- 172.21.15.38:0/55216680 shutdown_connections 2024-09-21T03:41:59.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:59.324+0000 7f9fafbc2640 1 --2- 172.21.15.38:0/55216680 >> v2:172.21.15.195:3300/0 conn(0x7f9fa8162280 0x7f9fa8162690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:59.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:59.324+0000 7f9fafbc2640 1 --2- 172.21.15.38:0/55216680 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9fa8162f00 0x7f9fa80d24d0 unknown :-1 s=CLOSED pgs=1606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:59.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:59.324+0000 7f9fafbc2640 1 -- 172.21.15.38:0/55216680 >> 172.21.15.38:0/55216680 conn(0x7f9fa80b6b20 msgr2=0x7f9fa80d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:59.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:59.324+0000 7f9fafbc2640 1 -- 172.21.15.38:0/55216680 shutdown_connections 2024-09-21T03:41:59.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:59.324+0000 7f9fafbc2640 1 -- 172.21.15.38:0/55216680 wait complete. 2024-09-21T03:41:59.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:59.326+0000 7f9fafbc2640 1 Processor -- start 2024-09-21T03:41:59.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:59.326+0000 7f9fafbc2640 1 -- start start 2024-09-21T03:41:59.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:59.327+0000 7f9fafbc2640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9fa80b2e40 0x7f9fa80cf040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:59.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:59.327+0000 7f9fafbc2640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9fa8162280 0x7f9fa815a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:59.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:59.327+0000 7f9fafbc2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9fa80b25a0 0x7f9fa80b7950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:59.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:59.327+0000 7f9fafbc2640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f9fa815e630 con 0x7f9fa80b25a0 2024-09-21T03:41:59.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:59.327+0000 7f9fafbc2640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f9fa815e4b0 con 0x7f9fa80b2e40 2024-09-21T03:41:59.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:59.327+0000 7f9fafbc2640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f9fa815e7b0 con 0x7f9fa8162280 2024-09-21T03:41:59.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:59.327+0000 7f9fae15f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9fa80b25a0 0x7f9fa80b7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:59.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:59.327+0000 7f9fae15f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9fa80b25a0 0x7f9fa80b7950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53254/0 (socket says 172.21.15.38:53254) 2024-09-21T03:41:59.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:59.327+0000 7f9fae15f640 1 -- 172.21.15.38:0/3802612438 learned_addr learned my addr 172.21.15.38:0/3802612438 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:41:59.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:59.328+0000 7f9fad95e640 1 --2- 172.21.15.38:0/3802612438 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9fa80b2e40 0x7f9fa80cf040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:59.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:59.328+0000 7f9fad15d640 1 --2- 172.21.15.38:0/3802612438 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9fa8162280 0x7f9fa815a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:59.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:59.328+0000 7f9fae15f640 1 -- 172.21.15.38:0/3802612438 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9fa8162280 msgr2=0x7f9fa815a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:59.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:59.328+0000 7f9fae15f640 1 --2- 172.21.15.38:0/3802612438 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9fa8162280 0x7f9fa815a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:59.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:59.328+0000 7f9fae15f640 1 -- 172.21.15.38:0/3802612438 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9fa80b2e40 msgr2=0x7f9fa80cf040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:59.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:59.328+0000 7f9fae15f640 1 --2- 172.21.15.38:0/3802612438 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9fa80b2e40 0x7f9fa80cf040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:59.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:59.328+0000 7f9fae15f640 1 -- 172.21.15.38:0/3802612438 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fa815aad0 con 0x7f9fa80b25a0 2024-09-21T03:41:59.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:59.328+0000 7f9fae15f640 1 --2- 172.21.15.38:0/3802612438 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9fa80b25a0 0x7f9fa80b7950 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7f9fa000c3c0 tx=0x7f9fa000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:59.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:59.328+0000 7f9fa67fc640 1 -- 172.21.15.38:0/3802612438 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fa0011790 con 0x7f9fa80b25a0 2024-09-21T03:41:59.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:59.328+0000 7f9fa67fc640 1 -- 172.21.15.38:0/3802612438 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9fa0017070 con 0x7f9fa80b25a0 2024-09-21T03:41:59.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:59.328+0000 7f9fa67fc640 1 -- 172.21.15.38:0/3802612438 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fa0011c40 con 0x7f9fa80b25a0 2024-09-21T03:41:59.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:59.328+0000 7f9fafbc2640 1 -- 172.21.15.38:0/3802612438 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fa81f5130 con 0x7f9fa80b25a0 2024-09-21T03:41:59.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:59.328+0000 7f9fafbc2640 1 -- 172.21.15.38:0/3802612438 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fa81f56f0 con 0x7f9fa80b25a0 2024-09-21T03:41:59.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:59.329+0000 7f9fafbc2640 1 -- 172.21.15.38:0/3802612438 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9fa8162f00 con 0x7f9fa80b25a0 2024-09-21T03:41:59.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:59.330+0000 7f9fa67fc640 1 -- 172.21.15.38:0/3802612438 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f9fa001c460 con 0x7f9fa80b25a0 2024-09-21T03:41:59.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:59.330+0000 7f9fa67fc640 1 --2- 172.21.15.38:0/3802612438 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9f94078b90 0x7f9f9407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:59.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:59.331+0000 7f9fa67fc640 1 -- 172.21.15.38:0/3802612438 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 7680+0+0 (secure 0 0 0) 0x7f9fa009b1e0 con 0x7f9fa80b25a0 2024-09-21T03:41:59.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:59.331+0000 7f9fad95e640 1 --2- 172.21.15.38:0/3802612438 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9f94078b90 0x7f9f9407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:59.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:59.331+0000 7f9fad95e640 1 --2- 172.21.15.38:0/3802612438 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9f94078b90 0x7f9f9407b0a0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f9f98034000 tx=0x7f9f98004ab0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:59.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:59.335+0000 7f9fa67fc640 1 -- 172.21.15.38:0/3802612438 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f9fa8162bd0 con 0x7f9fa80b25a0 2024-09-21T03:41:59.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:59.559+0000 7f9fafbc2640 1 -- 172.21.15.38:0/3802612438 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9fa815def0 con 0x7f9fa80b25a0 2024-09-21T03:41:59.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:59.559+0000 7f9fa67fc640 1 -- 172.21.15.38:0/3802612438 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 179 v179) ==== 95+0+4392 (secure 0 0 0) 0x7f9fa0062f00 con 0x7f9fa80b25a0 2024-09-21T03:41:59.561 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:41:59.561 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":179,"btime":"2024-09-21T03:41:58:837196+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6086,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/965972663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":965972663},{"type":"v1","addr":"172.21.15.195:6836","nonce":965972663}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6700,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/3176122334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":3176122334},{"type":"v1","addr":"172.21.15.195:6837","nonce":3176122334}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":8661,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/2418225210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":2418225210},{"type":"v1","addr":"172.21.15.38:6835","nonce":2418225210}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":171}],"filesystems":[{"mdsmap":{"epoch":179,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:40:02.319282+0000","modified":"2024-09-21T03:41:58.837120+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":112,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-21T03:41:59.562 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 179 2024-09-21T03:41:59.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:59.561+0000 7f9fafbc2640 1 -- 172.21.15.38:0/3802612438 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9f94078b90 msgr2=0x7f9f9407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:59.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:59.561+0000 7f9fafbc2640 1 --2- 172.21.15.38:0/3802612438 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9f94078b90 0x7f9f9407b0a0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f9f98034000 tx=0x7f9f98004ab0 comp rx=0 tx=0).stop 2024-09-21T03:41:59.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:59.561+0000 7f9fafbc2640 1 -- 172.21.15.38:0/3802612438 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9fa80b25a0 msgr2=0x7f9fa80b7950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:59.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:59.561+0000 7f9fafbc2640 1 --2- 172.21.15.38:0/3802612438 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9fa80b25a0 0x7f9fa80b7950 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7f9fa000c3c0 tx=0x7f9fa000c880 comp rx=0 tx=0).stop 2024-09-21T03:41:59.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:59.561+0000 7f9fafbc2640 1 -- 172.21.15.38:0/3802612438 shutdown_connections 2024-09-21T03:41:59.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:59.561+0000 7f9fafbc2640 1 --2- 172.21.15.38:0/3802612438 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9fa80b25a0 0x7f9fa80b7950 unknown :-1 s=CLOSED pgs=1607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:59.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:59.561+0000 7f9fafbc2640 1 --2- 172.21.15.38:0/3802612438 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9f94078b90 0x7f9f9407b0a0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:59.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:59.561+0000 7f9fafbc2640 1 --2- 172.21.15.38:0/3802612438 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9fa8162280 0x7f9fa815a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:59.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:59.561+0000 7f9fafbc2640 1 --2- 172.21.15.38:0/3802612438 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9fa80b2e40 0x7f9fa80cf040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:59.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:59.561+0000 7f9fafbc2640 1 -- 172.21.15.38:0/3802612438 >> 172.21.15.38:0/3802612438 conn(0x7f9fa80b6b20 msgr2=0x7f9fa806d9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:59.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:59.561+0000 7f9fafbc2640 1 -- 172.21.15.38:0/3802612438 shutdown_connections 2024-09-21T03:41:59.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:59.561+0000 7f9fafbc2640 1 -- 172.21.15.38:0/3802612438 wait complete. 2024-09-21T03:41:59.575 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-21T03:41:59.575 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:41:59.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:59.690+0000 7fa9a8721640 1 -- 172.21.15.38:0/1949942617 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa988003a50 msgr2=0x7fa988003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:59.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:59.690+0000 7fa9a8721640 1 --2- 172.21.15.38:0/1949942617 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa988003a50 0x7fa988003ec0 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7fa990009ed0 tx=0x7fa988005830 comp rx=0 tx=0).stop 2024-09-21T03:41:59.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:59.690+0000 7fa9a8721640 1 -- 172.21.15.38:0/1949942617 shutdown_connections 2024-09-21T03:41:59.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:59.690+0000 7fa9a8721640 1 --2- 172.21.15.38:0/1949942617 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa988003a50 0x7fa988003ec0 unknown :-1 s=CLOSED pgs=1609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:59.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:59.690+0000 7fa9a8721640 1 -- 172.21.15.38:0/1949942617 >> 172.21.15.38:0/1949942617 conn(0x7fa9a00b6b20 msgr2=0x7fa9a00d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:59.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:59.690+0000 7fa9a8721640 1 -- 172.21.15.38:0/1949942617 shutdown_connections 2024-09-21T03:41:59.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:59.690+0000 7fa9a8721640 1 -- 172.21.15.38:0/1949942617 wait complete. 2024-09-21T03:41:59.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:59.693+0000 7fa9a8721640 1 Processor -- start 2024-09-21T03:41:59.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:59.693+0000 7fa9a8721640 1 -- start start 2024-09-21T03:41:59.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:59.694+0000 7fa9a8721640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa988003a50 0x7fa9a01537e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:59.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:59.694+0000 7fa9a8721640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa9a01564a0 0x7fa9a0158400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:59.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:59.695+0000 7fa9a8721640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa9a0154270 0x7fa9a01f8860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:59.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:59.695+0000 7fa9a8721640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa9a0162b80 con 0x7fa9a0154270 2024-09-21T03:41:59.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:59.695+0000 7fa9a8721640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fa9a0162a00 con 0x7fa9a01564a0 2024-09-21T03:41:59.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:59.695+0000 7fa9a8721640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fa9a00d24d0 con 0x7fa988003a50 2024-09-21T03:41:59.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:59.695+0000 7fa9a6cbe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa9a0154270 0x7fa9a01f8860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:59.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:59.695+0000 7fa9a6cbe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa9a0154270 0x7fa9a01f8860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53280/0 (socket says 172.21.15.38:53280) 2024-09-21T03:41:59.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:59.695+0000 7fa9a6cbe640 1 -- 172.21.15.38:0/4240726446 learned_addr learned my addr 172.21.15.38:0/4240726446 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:41:59.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:59.695+0000 7fa9a64bd640 1 --2- 172.21.15.38:0/4240726446 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa988003a50 0x7fa9a01537e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:59.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:59.695+0000 7fa9a5cbc640 1 --2- 172.21.15.38:0/4240726446 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa9a01564a0 0x7fa9a0158400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:59.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:59.695+0000 7fa9a6cbe640 1 -- 172.21.15.38:0/4240726446 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa988003a50 msgr2=0x7fa9a01537e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:59.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:59.695+0000 7fa9a6cbe640 1 --2- 172.21.15.38:0/4240726446 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa988003a50 0x7fa9a01537e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:59.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:59.695+0000 7fa9a6cbe640 1 -- 172.21.15.38:0/4240726446 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa9a01564a0 msgr2=0x7fa9a0158400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:59.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:59.695+0000 7fa9a6cbe640 1 --2- 172.21.15.38:0/4240726446 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa9a01564a0 0x7fa9a0158400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:59.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:59.695+0000 7fa9a6cbe640 1 -- 172.21.15.38:0/4240726446 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9a0158940 con 0x7fa9a0154270 2024-09-21T03:41:59.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:59.695+0000 7fa9a6cbe640 1 --2- 172.21.15.38:0/4240726446 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa9a0154270 0x7fa9a01f8860 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7fa9a00d0960 tx=0x7fa998001630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:59.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:59.695+0000 7fa99effd640 1 -- 172.21.15.38:0/4240726446 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa998001cc0 con 0x7fa9a0154270 2024-09-21T03:41:59.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:59.695+0000 7fa9a8721640 1 -- 172.21.15.38:0/4240726446 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9a01572f0 con 0x7fa9a0154270 2024-09-21T03:41:59.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:59.695+0000 7fa99effd640 1 -- 172.21.15.38:0/4240726446 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa998016020 con 0x7fa9a0154270 2024-09-21T03:41:59.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:59.695+0000 7fa99effd640 1 -- 172.21.15.38:0/4240726446 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa998011690 con 0x7fa9a0154270 2024-09-21T03:41:59.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:59.696+0000 7fa9a8721640 1 -- 172.21.15.38:0/4240726446 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9a0157c00 con 0x7fa9a0154270 2024-09-21T03:41:59.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:59.697+0000 7fa9a8721640 1 -- 172.21.15.38:0/4240726446 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9a01575d0 con 0x7fa9a0154270 2024-09-21T03:41:59.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:59.697+0000 7fa99effd640 1 -- 172.21.15.38:0/4240726446 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa998011830 con 0x7fa9a0154270 2024-09-21T03:41:59.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:59.698+0000 7fa99effd640 1 --2- 172.21.15.38:0/4240726446 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa98c078bc0 0x7fa98c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:59.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:59.698+0000 7fa99effd640 1 -- 172.21.15.38:0/4240726446 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 7680+0+0 (secure 0 0 0) 0x7fa99809bd50 con 0x7fa9a0154270 2024-09-21T03:41:59.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:59.698+0000 7fa9a64bd640 1 --2- 172.21.15.38:0/4240726446 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa98c078bc0 0x7fa98c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:59.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:59.699+0000 7fa9a64bd640 1 --2- 172.21.15.38:0/4240726446 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa98c078bc0 0x7fa98c07b0d0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fa990035000 tx=0x7fa990004a40 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:59.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:59.702+0000 7fa99effd640 1 -- 172.21.15.38:0/4240726446 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fa99809f030 con 0x7fa9a0154270 2024-09-21T03:41:59.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:59.928+0000 7fa9a8721640 1 -- 172.21.15.38:0/4240726446 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa9a00d2a30 con 0x7fa9a0154270 2024-09-21T03:41:59.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:59.928+0000 7fa99effd640 1 -- 172.21.15.38:0/4240726446 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 179 v179) ==== 95+0+4392 (secure 0 0 0) 0x7fa998020020 con 0x7fa9a0154270 2024-09-21T03:41:59.931 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:41:59.931 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":179,"btime":"2024-09-21T03:41:58:837196+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6086,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/965972663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":965972663},{"type":"v1","addr":"172.21.15.195:6836","nonce":965972663}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6700,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/3176122334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":3176122334},{"type":"v1","addr":"172.21.15.195:6837","nonce":3176122334}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":8661,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/2418225210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":2418225210},{"type":"v1","addr":"172.21.15.38:6835","nonce":2418225210}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":171}],"filesystems":[{"mdsmap":{"epoch":179,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:40:02.319282+0000","modified":"2024-09-21T03:41:58.837120+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":112,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-21T03:41:59.931 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 179 2024-09-21T03:41:59.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:59.930+0000 7fa9a8721640 1 -- 172.21.15.38:0/4240726446 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa98c078bc0 msgr2=0x7fa98c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:59.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:59.930+0000 7fa9a8721640 1 --2- 172.21.15.38:0/4240726446 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa98c078bc0 0x7fa98c07b0d0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fa990035000 tx=0x7fa990004a40 comp rx=0 tx=0).stop 2024-09-21T03:41:59.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:59.930+0000 7fa9a8721640 1 -- 172.21.15.38:0/4240726446 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa9a0154270 msgr2=0x7fa9a01f8860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:59.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:59.930+0000 7fa9a8721640 1 --2- 172.21.15.38:0/4240726446 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa9a0154270 0x7fa9a01f8860 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7fa9a00d0960 tx=0x7fa998001630 comp rx=0 tx=0).stop 2024-09-21T03:41:59.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:59.931+0000 7fa9a8721640 1 -- 172.21.15.38:0/4240726446 shutdown_connections 2024-09-21T03:41:59.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:59.931+0000 7fa9a8721640 1 --2- 172.21.15.38:0/4240726446 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa9a0154270 0x7fa9a01f8860 unknown :-1 s=CLOSED pgs=1610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:59.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:59.931+0000 7fa9a8721640 1 --2- 172.21.15.38:0/4240726446 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa9a01564a0 0x7fa9a0158400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:59.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:59.931+0000 7fa9a8721640 1 --2- 172.21.15.38:0/4240726446 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa98c078bc0 0x7fa98c07b0d0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:59.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:59.931+0000 7fa9a8721640 1 --2- 172.21.15.38:0/4240726446 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa988003a50 0x7fa9a01537e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:59.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:59.931+0000 7fa9a8721640 1 -- 172.21.15.38:0/4240726446 >> 172.21.15.38:0/4240726446 conn(0x7fa9a00b6b20 msgr2=0x7fa9a00b3df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:59.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:59.931+0000 7fa9a8721640 1 -- 172.21.15.38:0/4240726446 shutdown_connections 2024-09-21T03:41:59.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:41:59.931+0000 7fa9a8721640 1 -- 172.21.15.38:0/4240726446 wait complete. 2024-09-21T03:41:59.944 INFO:tasks.cephfs.test_damage:Result: Mutation 'Corrupt 1.00000000.inode' led to DAMAGED state 2024-09-21T03:41:59.944 INFO:tasks.cephfs.test_damage:Applying mutation 'Corrupt 200.00000000' 2024-09-21T03:41:59.944 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-21T03:41:59.944 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:41:59.960 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056795,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2354 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T03:41:59.961 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi038.front.sepia.ceph.com 2024-09-21T03:41:59.961 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:41:59.961 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T03:42:00.016 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T03:42:00.016 INFO:teuthology.orchestra.run.smithi038.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-09-21T03:42:00.016 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-21T03:42:00.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:00.172+0000 7ffa17a27640 1 -- 172.21.15.38:0/2994304450 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffa1015fc50 msgr2=0x7ffa1006ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:00.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:00.172+0000 7ffa17a27640 1 --2- 172.21.15.38:0/2994304450 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffa1015fc50 0x7ffa1006ddf0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ffa0c009830 tx=0x7ffa0c02a500 comp rx=0 tx=0).stop 2024-09-21T03:42:00.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:00.172+0000 7ffa17a27640 1 -- 172.21.15.38:0/2994304450 shutdown_connections 2024-09-21T03:42:00.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:00.172+0000 7ffa17a27640 1 --2- 172.21.15.38:0/2994304450 >> v2:172.21.15.195:3300/0 conn(0x7ffa100cf9a0 0x7ffa10072d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:00.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:00.172+0000 7ffa17a27640 1 --2- 172.21.15.38:0/2994304450 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffa1015fc50 0x7ffa1006ddf0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:00.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:00.172+0000 7ffa17a27640 1 -- 172.21.15.38:0/2994304450 >> 172.21.15.38:0/2994304450 conn(0x7ffa100b1b90 msgr2=0x7ffa100b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:00.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:00.172+0000 7ffa17a27640 1 -- 172.21.15.38:0/2994304450 shutdown_connections 2024-09-21T03:42:00.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:00.172+0000 7ffa17a27640 1 -- 172.21.15.38:0/2994304450 wait complete. 2024-09-21T03:42:00.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:00.174+0000 7ffa17a27640 1 Processor -- start 2024-09-21T03:42:00.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:00.174+0000 7ffa17a27640 1 -- start start 2024-09-21T03:42:00.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:00.175+0000 7ffa17a27640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffa1006e330 0x7ffa100c70e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:00.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:00.175+0000 7ffa17a27640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffa100cf9a0 0x7ffa100c7620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:00.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:00.176+0000 7ffa17a27640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffa1015fc50 0x7ffa100c9380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:00.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:00.176+0000 7ffa17a27640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ffa100732d0 con 0x7ffa1006e330 2024-09-21T03:42:00.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:00.176+0000 7ffa17a27640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ffa100ca9f0 con 0x7ffa100cf9a0 2024-09-21T03:42:00.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:00.176+0000 7ffa17a27640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7ffa10073450 con 0x7ffa1015fc50 2024-09-21T03:42:00.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:00.176+0000 7ffa157c3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffa1006e330 0x7ffa100c70e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:00.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:00.176+0000 7ffa157c3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffa1006e330 0x7ffa100c70e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53290/0 (socket says 172.21.15.38:53290) 2024-09-21T03:42:00.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:00.176+0000 7ffa157c3640 1 -- 172.21.15.38:0/817360663 learned_addr learned my addr 172.21.15.38:0/817360663 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:42:00.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:00.176+0000 7ffa14fc2640 1 --2- 172.21.15.38:0/817360663 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffa100cf9a0 0x7ffa100c7620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:00.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:00.176+0000 7ffa15fc4640 1 --2- 172.21.15.38:0/817360663 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffa1015fc50 0x7ffa100c9380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:00.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:00.176+0000 7ffa157c3640 1 -- 172.21.15.38:0/817360663 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffa1015fc50 msgr2=0x7ffa100c9380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:00.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:00.176+0000 7ffa157c3640 1 --2- 172.21.15.38:0/817360663 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffa1015fc50 0x7ffa100c9380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:00.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:00.176+0000 7ffa157c3640 1 -- 172.21.15.38:0/817360663 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffa100cf9a0 msgr2=0x7ffa100c7620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:00.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:00.176+0000 7ffa157c3640 1 --2- 172.21.15.38:0/817360663 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffa100cf9a0 0x7ffa100c7620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:00.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:00.176+0000 7ffa157c3640 1 -- 172.21.15.38:0/817360663 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa100c6470 con 0x7ffa1006e330 2024-09-21T03:42:00.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:00.176+0000 7ffa14fc2640 1 --2- 172.21.15.38:0/817360663 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffa100cf9a0 0x7ffa100c7620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:42:00.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:00.176+0000 7ffa157c3640 1 --2- 172.21.15.38:0/817360663 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffa1006e330 0x7ffa100c70e0 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7ffa0c000c00 tx=0x7ffa0c033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:00.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:00.176+0000 7ffa05ffb640 1 -- 172.21.15.38:0/817360663 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa0c031d60 con 0x7ffa1006e330 2024-09-21T03:42:00.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:00.177+0000 7ffa05ffb640 1 -- 172.21.15.38:0/817360663 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffa0c033e10 con 0x7ffa1006e330 2024-09-21T03:42:00.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:00.177+0000 7ffa17a27640 1 -- 172.21.15.38:0/817360663 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa100b4b50 con 0x7ffa1006e330 2024-09-21T03:42:00.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:00.177+0000 7ffa05ffb640 1 -- 172.21.15.38:0/817360663 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa0c0332b0 con 0x7ffa1006e330 2024-09-21T03:42:00.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:00.177+0000 7ffa17a27640 1 -- 172.21.15.38:0/817360663 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa101f92b0 con 0x7ffa1006e330 2024-09-21T03:42:00.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:00.178+0000 7ffa17a27640 1 -- 172.21.15.38:0/817360663 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa100739e0 con 0x7ffa1006e330 2024-09-21T03:42:00.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:00.178+0000 7ffa05ffb640 1 -- 172.21.15.38:0/817360663 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ffa0c033450 con 0x7ffa1006e330 2024-09-21T03:42:00.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:00.179+0000 7ffa05ffb640 1 --2- 172.21.15.38:0/817360663 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff9f4078b90 0x7ff9f407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:00.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:00.179+0000 7ffa05ffb640 1 -- 172.21.15.38:0/817360663 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 7680+0+0 (secure 0 0 0) 0x7ffa0c0bc980 con 0x7ffa1006e330 2024-09-21T03:42:00.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:00.179+0000 7ffa14fc2640 1 --2- 172.21.15.38:0/817360663 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff9f4078b90 0x7ff9f407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:00.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:00.180+0000 7ffa14fc2640 1 --2- 172.21.15.38:0/817360663 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff9f4078b90 0x7ff9f407b0a0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7ffa100c61d0 tx=0x7ffa00009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:00.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:00.184+0000 7ffa05ffb640 1 -- 172.21.15.38:0/817360663 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7ffa0c03f030 con 0x7ffa1006e330 2024-09-21T03:42:00.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:00.409+0000 7ffa17a27640 1 -- 172.21.15.38:0/817360663 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7ffa100b4170 con 0x7ffa1006e330 2024-09-21T03:42:00.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:00.841+0000 7ffa05ffb640 1 -- 172.21.15.38:0/817360663 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v180) ==== 161+0+0 (secure 0 0 0) 0x7ffa0c0845a0 con 0x7ffa1006e330 2024-09-21T03:42:00.843 INFO:teuthology.orchestra.run.smithi038.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-21T03:42:00.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:00.844+0000 7ffa17a27640 1 -- 172.21.15.38:0/817360663 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff9f4078b90 msgr2=0x7ff9f407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:00.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:00.844+0000 7ffa17a27640 1 --2- 172.21.15.38:0/817360663 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff9f4078b90 0x7ff9f407b0a0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7ffa100c61d0 tx=0x7ffa00009040 comp rx=0 tx=0).stop 2024-09-21T03:42:00.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:00.844+0000 7ffa17a27640 1 -- 172.21.15.38:0/817360663 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffa1006e330 msgr2=0x7ffa100c70e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:00.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:00.844+0000 7ffa17a27640 1 --2- 172.21.15.38:0/817360663 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffa1006e330 0x7ffa100c70e0 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7ffa0c000c00 tx=0x7ffa0c033c20 comp rx=0 tx=0).stop 2024-09-21T03:42:00.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:00.844+0000 7ffa17a27640 1 -- 172.21.15.38:0/817360663 shutdown_connections 2024-09-21T03:42:00.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:00.844+0000 7ffa17a27640 1 --2- 172.21.15.38:0/817360663 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff9f4078b90 0x7ff9f407b0a0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:00.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:00.844+0000 7ffa17a27640 1 --2- 172.21.15.38:0/817360663 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffa1015fc50 0x7ffa100c9380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:00.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:00.844+0000 7ffa17a27640 1 --2- 172.21.15.38:0/817360663 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffa100cf9a0 0x7ffa100c7620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:00.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:00.844+0000 7ffa17a27640 1 --2- 172.21.15.38:0/817360663 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffa1006e330 0x7ffa100c70e0 unknown :-1 s=CLOSED pgs=1611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:00.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:00.844+0000 7ffa17a27640 1 -- 172.21.15.38:0/817360663 >> 172.21.15.38:0/817360663 conn(0x7ffa100b1b90 msgr2=0x7ffa1006ef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:00.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:00.844+0000 7ffa17a27640 1 -- 172.21.15.38:0/817360663 shutdown_connections 2024-09-21T03:42:00.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:00.844+0000 7ffa17a27640 1 -- 172.21.15.38:0/817360663 wait complete. 2024-09-21T03:42:00.859 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-09-21T03:42:00.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:00.978+0000 7f337b3c3640 1 -- 172.21.15.38:0/4202704890 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3360004800 msgr2=0x7f3360004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:00.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:00.978+0000 7f337b3c3640 1 --2- 172.21.15.38:0/4202704890 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3360004800 0x7f3360004c10 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7f3364009b10 tx=0x7f3360005830 comp rx=0 tx=0).stop 2024-09-21T03:42:00.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:00.978+0000 7f3379960640 1 -- 172.21.15.38:0/4202704890 reap_dead start 2024-09-21T03:42:00.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:00.978+0000 7f337b3c3640 1 -- 172.21.15.38:0/4202704890 shutdown_connections 2024-09-21T03:42:00.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:00.978+0000 7f337b3c3640 1 -- 172.21.15.38:0/4202704890 >> 172.21.15.38:0/4202704890 conn(0x7f337409d830 msgr2=0x7f33740b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:00.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:00.978+0000 7f337b3c3640 1 -- 172.21.15.38:0/4202704890 shutdown_connections 2024-09-21T03:42:00.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:00.979+0000 7f337b3c3640 1 -- 172.21.15.38:0/4202704890 wait complete. 2024-09-21T03:42:00.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:00.980+0000 7f337b3c3640 1 Processor -- start 2024-09-21T03:42:00.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:00.980+0000 7f337b3c3640 1 -- start start 2024-09-21T03:42:00.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:00.982+0000 7f337b3c3640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3360004800 0x7f33740a1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:00.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:00.982+0000 7f337b3c3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f33741f5c30 0x7f33741f8530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:00.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:00.982+0000 7f337b3c3640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f33741f8a70 0x7f33740b4760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:00.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:00.982+0000 7f337b3c3640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f33740d0df0 con 0x7f33741f5c30 2024-09-21T03:42:00.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:00.982+0000 7f337b3c3640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f33740cf540 con 0x7f33741f8a70 2024-09-21T03:42:00.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:00.982+0000 7f337b3c3640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f33740d0f70 con 0x7f3360004800 2024-09-21T03:42:00.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:00.982+0000 7f337895e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f33741f5c30 0x7f33741f8530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:00.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:00.982+0000 7f337895e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f33741f5c30 0x7f33741f8530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53304/0 (socket says 172.21.15.38:53304) 2024-09-21T03:42:00.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:00.982+0000 7f337895e640 1 -- 172.21.15.38:0/1023641876 learned_addr learned my addr 172.21.15.38:0/1023641876 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:42:00.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:00.982+0000 7f3379960640 1 --2- 172.21.15.38:0/1023641876 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f33741f8a70 0x7f33740b4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:00.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:00.982+0000 7f337915f640 1 --2- 172.21.15.38:0/1023641876 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3360004800 0x7f33740a1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:00.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:00.982+0000 7f337895e640 1 -- 172.21.15.38:0/1023641876 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3360004800 msgr2=0x7f33740a1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:00.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:00.982+0000 7f337895e640 1 --2- 172.21.15.38:0/1023641876 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3360004800 0x7f33740a1cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:00.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:00.982+0000 7f337895e640 1 -- 172.21.15.38:0/1023641876 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f33741f8a70 msgr2=0x7f33740b4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:00.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:00.982+0000 7f337895e640 1 --2- 172.21.15.38:0/1023641876 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f33741f8a70 0x7f33740b4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:00.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:00.982+0000 7f337895e640 1 -- 172.21.15.38:0/1023641876 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33741f91b0 con 0x7f33741f5c30 2024-09-21T03:42:00.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:00.982+0000 7f337895e640 1 --2- 172.21.15.38:0/1023641876 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f33741f5c30 0x7f33741f8530 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7f3364036040 tx=0x7f33640336f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:00.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:00.983+0000 7f3369ffb640 1 -- 172.21.15.38:0/1023641876 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3364043070 con 0x7f33741f5c30 2024-09-21T03:42:00.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:00.983+0000 7f3369ffb640 1 -- 172.21.15.38:0/1023641876 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3364004030 con 0x7f33741f5c30 2024-09-21T03:42:00.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:00.983+0000 7f3369ffb640 1 -- 172.21.15.38:0/1023641876 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3364038040 con 0x7f33741f5c30 2024-09-21T03:42:00.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:00.983+0000 7f337b3c3640 1 -- 172.21.15.38:0/1023641876 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33741f5150 con 0x7f33741f5c30 2024-09-21T03:42:00.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:00.983+0000 7f337b3c3640 1 -- 172.21.15.38:0/1023641876 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33740a2210 con 0x7f33741f5c30 2024-09-21T03:42:00.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:00.984+0000 7f337b3c3640 1 -- 172.21.15.38:0/1023641876 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33740cbc20 con 0x7f33741f5c30 2024-09-21T03:42:00.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:00.985+0000 7f3369ffb640 1 -- 172.21.15.38:0/1023641876 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f33640338e0 con 0x7f33741f5c30 2024-09-21T03:42:00.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:00.985+0000 7f3369ffb640 1 --2- 172.21.15.38:0/1023641876 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3358078b90 0x7f335807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:00.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:00.985+0000 7f3369ffb640 1 -- 172.21.15.38:0/1023641876 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 7680+0+0 (secure 0 0 0) 0x7f33640bba40 con 0x7f33741f5c30 2024-09-21T03:42:00.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:00.985+0000 7f337915f640 1 --2- 172.21.15.38:0/1023641876 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3358078b90 0x7f335807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:00.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:00.986+0000 7f337915f640 1 --2- 172.21.15.38:0/1023641876 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3358078b90 0x7f335807b0a0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f3370006590 tx=0x7f3370006f70 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:00.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:00.990+0000 7f3369ffb640 1 -- 172.21.15.38:0/1023641876 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f3364083010 con 0x7f33741f5c30 2024-09-21T03:42:01.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.251+0000 7f337b3c3640 1 -- 172.21.15.38:0/1023641876 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7f33741f8f40 con 0x7f33741f5c30 2024-09-21T03:42:01.430 DEBUG:teuthology.orchestra.run.smithi038:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T03:42:01.433 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T03:42:01.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.853+0000 7f3369ffb640 1 -- 172.21.15.38:0/1023641876 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v181) ==== 101+0+0 (secure 0 0 0) 0x7f3364087a40 con 0x7f33741f5c30 2024-09-21T03:42:01.856 INFO:teuthology.orchestra.run.smithi038.stderr:repaired: restoring rank 6:0 2024-09-21T03:42:01.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.856+0000 7f337b3c3640 1 -- 172.21.15.38:0/1023641876 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3358078b90 msgr2=0x7f335807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:01.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.856+0000 7f337b3c3640 1 --2- 172.21.15.38:0/1023641876 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3358078b90 0x7f335807b0a0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f3370006590 tx=0x7f3370006f70 comp rx=0 tx=0).stop 2024-09-21T03:42:01.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.856+0000 7f337b3c3640 1 -- 172.21.15.38:0/1023641876 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f33741f5c30 msgr2=0x7f33741f8530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:01.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.856+0000 7f337b3c3640 1 --2- 172.21.15.38:0/1023641876 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f33741f5c30 0x7f33741f8530 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7f3364036040 tx=0x7f33640336f0 comp rx=0 tx=0).stop 2024-09-21T03:42:01.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.856+0000 7f337b3c3640 1 -- 172.21.15.38:0/1023641876 shutdown_connections 2024-09-21T03:42:01.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.856+0000 7f337b3c3640 1 --2- 172.21.15.38:0/1023641876 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f33741f8a70 0x7f33740b4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:01.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.856+0000 7f337b3c3640 1 --2- 172.21.15.38:0/1023641876 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f33741f5c30 0x7f33741f8530 unknown :-1 s=CLOSED pgs=1614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:01.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.856+0000 7f337b3c3640 1 --2- 172.21.15.38:0/1023641876 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3358078b90 0x7f335807b0a0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:01.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.856+0000 7f337b3c3640 1 --2- 172.21.15.38:0/1023641876 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3360004800 0x7f33740a1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:01.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.856+0000 7f337b3c3640 1 -- 172.21.15.38:0/1023641876 >> 172.21.15.38:0/1023641876 conn(0x7f337409d830 msgr2=0x7f33740b31a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:01.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.856+0000 7f337b3c3640 1 -- 172.21.15.38:0/1023641876 shutdown_connections 2024-09-21T03:42:01.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.856+0000 7f337b3c3640 1 -- 172.21.15.38:0/1023641876 wait complete. 2024-09-21T03:42:01.871 DEBUG:teuthology.orchestra.run.smithi038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-09-21T03:42:01.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.927+0000 7f0215336f40 1 Processor -- start 2024-09-21T03:42:01.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.927+0000 7f0215336f40 1 -- start start 2024-09-21T03:42:01.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.928+0000 7f0215336f40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559a3efa9790 0x559a3efa90a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:01.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.928+0000 7f0215336f40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x559a3ef1dbf0 0x559a3ef1e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:01.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.928+0000 7f0215336f40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559a3efaa880 0x559a3efaac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:01.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.928+0000 7f0215336f40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x559a3ee3ed60 con 0x559a3efaa880 2024-09-21T03:42:01.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.928+0000 7f0215336f40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x559a3ee9d430 con 0x559a3efa9790 2024-09-21T03:42:01.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.928+0000 7f0215336f40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x559a3ef39180 con 0x559a3ef1dbf0 2024-09-21T03:42:01.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.928+0000 7f020f5f1640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559a3efaa880 0x559a3efaac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:01.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.928+0000 7f020f5f1640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559a3efaa880 0x559a3efaac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53324/0 (socket says 172.21.15.38:53324) 2024-09-21T03:42:01.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.928+0000 7f020f5f1640 1 -- 172.21.15.38:0/4156675984 learned_addr learned my addr 172.21.15.38:0/4156675984 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:42:01.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.929+0000 7f020edf0640 1 --2- 172.21.15.38:0/4156675984 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559a3efa9790 0x559a3efa90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:01.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.929+0000 7f0207fff640 1 --2- 172.21.15.38:0/4156675984 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x559a3ef1dbf0 0x559a3ef1e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:01.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.929+0000 7f020f5f1640 1 -- 172.21.15.38:0/4156675984 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x559a3ef1dbf0 msgr2=0x559a3ef1e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:01.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.929+0000 7f020f5f1640 1 --2- 172.21.15.38:0/4156675984 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x559a3ef1dbf0 0x559a3ef1e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:01.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.929+0000 7f020f5f1640 1 -- 172.21.15.38:0/4156675984 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559a3efa9790 msgr2=0x559a3efa90a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:01.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.929+0000 7f020f5f1640 1 --2- 172.21.15.38:0/4156675984 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559a3efa9790 0x559a3efa90a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:01.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.929+0000 7f020f5f1640 1 -- 172.21.15.38:0/4156675984 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a3eef1360 con 0x559a3efaa880 2024-09-21T03:42:01.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.929+0000 7f020f5f1640 1 --2- 172.21.15.38:0/4156675984 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559a3efaa880 0x559a3efaac90 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f02080032c0 tx=0x7f02080033a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76c859285b8b414b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:01.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.929+0000 7f020ddee640 1 -- 172.21.15.38:0/4156675984 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0208064030 con 0x559a3efaa880 2024-09-21T03:42:01.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.929+0000 7f020ddee640 1 -- 172.21.15.38:0/4156675984 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0208055430 con 0x559a3efaa880 2024-09-21T03:42:01.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.929+0000 7f020ddee640 1 -- 172.21.15.38:0/4156675984 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f020800b480 con 0x559a3efaa880 2024-09-21T03:42:01.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.930+0000 7f0215336f40 1 -- 172.21.15.38:0/4156675984 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559a3efaa880 msgr2=0x559a3efaac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:01.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.930+0000 7f0215336f40 1 --2- 172.21.15.38:0/4156675984 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559a3efaa880 0x559a3efaac90 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f02080032c0 tx=0x7f02080033a0 comp rx=0 tx=0).stop 2024-09-21T03:42:01.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.930+0000 7f0215336f40 1 -- 172.21.15.38:0/4156675984 shutdown_connections 2024-09-21T03:42:01.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.930+0000 7f0215336f40 1 --2- 172.21.15.38:0/4156675984 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559a3efaa880 0x559a3efaac90 unknown :-1 s=CLOSED pgs=1617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:01.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.930+0000 7f0215336f40 1 --2- 172.21.15.38:0/4156675984 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x559a3ef1dbf0 0x559a3ef1e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:01.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.930+0000 7f0215336f40 1 --2- 172.21.15.38:0/4156675984 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559a3efa9790 0x559a3efa90a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:01.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.930+0000 7f0215336f40 1 -- 172.21.15.38:0/4156675984 >> 172.21.15.38:0/4156675984 conn(0x559a3ee0fac0 msgr2=0x559a3eeefcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:01.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.930+0000 7f0215336f40 1 -- 172.21.15.38:0/4156675984 shutdown_connections 2024-09-21T03:42:01.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.930+0000 7f0215336f40 1 -- 172.21.15.38:0/4156675984 wait complete. 2024-09-21T03:42:01.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.931+0000 7f0215336f40 1 Processor -- start 2024-09-21T03:42:01.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.931+0000 7f0215336f40 1 -- start start 2024-09-21T03:42:01.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.932+0000 7f0215336f40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x559a3ef1dbf0 0x559a3efc40d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:01.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.932+0000 7f0215336f40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559a3efa9790 0x559a3f05d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:01.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.932+0000 7f0215336f40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559a3efaa880 0x559a3efa7010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:01.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.932+0000 7f0215336f40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x559a3ef36b90 con 0x559a3efa9790 2024-09-21T03:42:01.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.932+0000 7f0215336f40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x559a3eef0510 con 0x559a3efaa880 2024-09-21T03:42:01.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.932+0000 7f0215336f40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x559a3efc2c60 con 0x559a3ef1dbf0 2024-09-21T03:42:01.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.932+0000 7f0207fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559a3efa9790 0x559a3f05d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:01.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.932+0000 7f0207fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559a3efa9790 0x559a3f05d7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53334/0 (socket says 172.21.15.38:53334) 2024-09-21T03:42:01.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.932+0000 7f0207fff640 1 -- 172.21.15.38:0/2278462177 learned_addr learned my addr 172.21.15.38:0/2278462177 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:42:01.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.933+0000 7f020f5f1640 1 --2- 172.21.15.38:0/2278462177 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559a3efaa880 0x559a3efa7010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:01.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.933+0000 7f0207fff640 1 -- 172.21.15.38:0/2278462177 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x559a3ef1dbf0 msgr2=0x559a3efc40d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:01.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.933+0000 7f020edf0640 1 --2- 172.21.15.38:0/2278462177 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x559a3ef1dbf0 0x559a3efc40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:01.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.933+0000 7f0207fff640 1 --2- 172.21.15.38:0/2278462177 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x559a3ef1dbf0 0x559a3efc40d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:01.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.933+0000 7f0207fff640 1 -- 172.21.15.38:0/2278462177 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559a3efaa880 msgr2=0x559a3efa7010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:01.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.933+0000 7f0207fff640 1 --2- 172.21.15.38:0/2278462177 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559a3efaa880 0x559a3efa7010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:01.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.933+0000 7f0207fff640 1 -- 172.21.15.38:0/2278462177 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a3efc4610 con 0x559a3efa9790 2024-09-21T03:42:01.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.933+0000 7f020f5f1640 1 --2- 172.21.15.38:0/2278462177 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559a3efaa880 0x559a3efa7010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:42:01.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.933+0000 7f020edf0640 1 --2- 172.21.15.38:0/2278462177 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x559a3ef1dbf0 0x559a3efc40d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:42:01.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.933+0000 7f0207fff640 1 --2- 172.21.15.38:0/2278462177 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559a3efa9790 0x559a3f05d7b0 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7f01f800aa20 tx=0x7f01f800aee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:01.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.933+0000 7f020cdec640 1 -- 172.21.15.38:0/2278462177 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01f801b070 con 0x559a3efa9790 2024-09-21T03:42:01.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.933+0000 7f020cdec640 1 -- 172.21.15.38:0/2278462177 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f01f80099a0 con 0x559a3efa9790 2024-09-21T03:42:01.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.933+0000 7f0215336f40 1 -- 172.21.15.38:0/2278462177 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x559a3f061620 con 0x559a3efa9790 2024-09-21T03:42:01.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.933+0000 7f020cdec640 1 -- 172.21.15.38:0/2278462177 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01f8009d90 con 0x559a3efa9790 2024-09-21T03:42:01.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.933+0000 7f0215336f40 1 -- 172.21.15.38:0/2278462177 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a3f061e00 con 0x559a3efa9790 2024-09-21T03:42:01.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.935+0000 7f020cdec640 1 -- 172.21.15.38:0/2278462177 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f01f801e070 con 0x559a3efa9790 2024-09-21T03:42:01.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.935+0000 7f020cdec640 1 --2- 172.21.15.38:0/2278462177 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f01f00775a0 0x7f01f0079ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:01.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.935+0000 7f020cdec640 1 -- 172.21.15.38:0/2278462177 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 7680+0+0 (secure 0 0 0) 0x7f01f809ac40 con 0x559a3efa9790 2024-09-21T03:42:01.940 INFO:teuthology.orchestra.run.smithi038.stdout:Importing pool 2024-09-21T03:42:01.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.935+0000 7f0215336f40 1 --2- 172.21.15.38:0/2278462177 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x559a3f05dcf0 0x559a3efbe0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:01.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.935+0000 7f0215336f40 1 -- 172.21.15.38:0/2278462177 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x559a3efbe5f0 con 0x559a3f05dcf0 2024-09-21T03:42:01.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.935+0000 7f020edf0640 1 --2- 172.21.15.38:0/2278462177 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f01f00775a0 0x7f01f0079ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:01.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.936+0000 7f020f5f1640 1 --2- 172.21.15.38:0/2278462177 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x559a3f05dcf0 0x559a3efbe0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:01.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.936+0000 7f020edf0640 1 --2- 172.21.15.38:0/2278462177 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f01f00775a0 0x7f01f0079ab0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f0200001d50 tx=0x7f0200003040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:01.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.936+0000 7f020f5f1640 1 --2- 172.21.15.38:0/2278462177 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x559a3f05dcf0 0x559a3efbe0b0 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:01.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.938+0000 7f020f5f1640 1 -- 172.21.15.38:0/2278462177 <== osd.7 v2:172.21.15.195:6818/1606106342 1 ==== osd_op_reply(1 601.00000000 [create] v112'474 uv448 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f020800b040 con 0x559a3f05dcf0 2024-09-21T03:42:01.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.938+0000 7f0215336f40 1 -- 172.21.15.38:0/2278462177 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e112) -- 0x559a3f06a7b0 con 0x559a3f05dcf0 2024-09-21T03:42:01.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.939+0000 7f020f5f1640 1 -- 172.21.15.38:0/2278462177 <== osd.7 v2:172.21.15.195:6818/1606106342 2 ==== osd_op_reply(2 601.00000000 [delete] v112'475 uv475 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f020800b040 con 0x559a3f05dcf0 2024-09-21T03:42:01.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.940+0000 7f0215336f40 1 -- 172.21.15.38:0/2278462177 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x559a3f06ab50 con 0x559a3f05dcf0 2024-09-21T03:42:01.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.941+0000 7f020f5f1640 1 -- 172.21.15.38:0/2278462177 <== osd.7 v2:172.21.15.195:6818/1606106342 3 ==== osd_op_reply(3 601.00000000 [create] v112'476 uv476 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f020800b040 con 0x559a3f05dcf0 2024-09-21T03:42:01.943 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-09-21T03:42:01.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.941+0000 7f0215336f40 1 -- 172.21.15.38:0/2278462177 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x559a3f06bf00 con 0x559a3f05dcf0 2024-09-21T03:42:01.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.943+0000 7f020f5f1640 1 -- 172.21.15.38:0/2278462177 <== osd.7 v2:172.21.15.195:6818/1606106342 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v112'477 uv477 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f020800b040 con 0x559a3f05dcf0 2024-09-21T03:42:01.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.943+0000 7f0215336f40 1 -- 172.21.15.38:0/2278462177 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x559a3efa3200 con 0x559a3f05dcf0 2024-09-21T03:42:01.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.944+0000 7f020f5f1640 1 -- 172.21.15.38:0/2278462177 <== osd.7 v2:172.21.15.195:6818/1606106342 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v112'478 uv478 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f020800b040 con 0x559a3f05dcf0 2024-09-21T03:42:01.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.944+0000 7f0215336f40 1 -- 172.21.15.38:0/2278462177 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x559a3f06aef0 con 0x559a3f05dcf0 2024-09-21T03:42:01.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.946+0000 7f020f5f1640 1 -- 172.21.15.38:0/2278462177 <== osd.7 v2:172.21.15.195:6818/1606106342 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v112'479 uv479 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f020800b040 con 0x559a3f05dcf0 2024-09-21T03:42:01.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.946+0000 7f0215336f40 1 -- 172.21.15.38:0/2278462177 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x559a3f06b290 con 0x559a3f05dcf0 2024-09-21T03:42:01.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.947+0000 7f020f5f1640 1 -- 172.21.15.38:0/2278462177 <== osd.7 v2:172.21.15.195:6818/1606106342 7 ==== osd_op_reply(7 602.00000000 [create] v112'480 uv454 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f020800b040 con 0x559a3f05dcf0 2024-09-21T03:42:01.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.947+0000 7f0215336f40 1 -- 172.21.15.38:0/2278462177 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e112) -- 0x559a3f06b680 con 0x559a3f05dcf0 2024-09-21T03:42:01.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.948+0000 7f020f5f1640 1 -- 172.21.15.38:0/2278462177 <== osd.7 v2:172.21.15.195:6818/1606106342 8 ==== osd_op_reply(8 602.00000000 [delete] v112'481 uv481 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f020800b040 con 0x559a3f05dcf0 2024-09-21T03:42:01.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.949+0000 7f0215336f40 1 -- 172.21.15.38:0/2278462177 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x559a3f06b680 con 0x559a3f05dcf0 2024-09-21T03:42:01.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.950+0000 7f020f5f1640 1 -- 172.21.15.38:0/2278462177 <== osd.7 v2:172.21.15.195:6818/1606106342 9 ==== osd_op_reply(9 602.00000000 [create] v112'482 uv482 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f020800b040 con 0x559a3f05dcf0 2024-09-21T03:42:01.952 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-09-21T03:42:01.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.950+0000 7f0215336f40 1 -- 172.21.15.38:0/2278462177 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x559a3f06b630 con 0x559a3f05dcf0 2024-09-21T03:42:01.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.951+0000 7f020f5f1640 1 -- 172.21.15.38:0/2278462177 <== osd.7 v2:172.21.15.195:6818/1606106342 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v112'483 uv483 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f020800b040 con 0x559a3f05dcf0 2024-09-21T03:42:01.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.951+0000 7f0215336f40 1 -- 172.21.15.38:0/2278462177 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x559a3f06b630 con 0x559a3f05dcf0 2024-09-21T03:42:01.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.953+0000 7f020f5f1640 1 -- 172.21.15.38:0/2278462177 <== osd.7 v2:172.21.15.195:6818/1606106342 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v112'484 uv484 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f020800b040 con 0x559a3f05dcf0 2024-09-21T03:42:01.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.953+0000 7f0215336f40 1 -- 172.21.15.38:0/2278462177 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x559a3f06b630 con 0x559a3f05dcf0 2024-09-21T03:42:01.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.954+0000 7f020f5f1640 1 -- 172.21.15.38:0/2278462177 <== osd.7 v2:172.21.15.195:6818/1606106342 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v112'485 uv485 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f020800b040 con 0x559a3f05dcf0 2024-09-21T03:42:01.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.954+0000 7f0215336f40 1 -- 172.21.15.38:0/2278462177 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:13 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x559a3f06b630 con 0x559a3f05dcf0 2024-09-21T03:42:01.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.956+0000 7f020f5f1640 1 -- 172.21.15.38:0/2278462177 <== osd.7 v2:172.21.15.195:6818/1606106342 13 ==== osd_op_reply(13 10000000000.00000000 [create] v112'486 uv461 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7f020800b040 con 0x559a3f05dcf0 2024-09-21T03:42:01.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.956+0000 7f0215336f40 1 -- 172.21.15.38:0/2278462177 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:14 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e112) -- 0x559a3f06b630 con 0x559a3f05dcf0 2024-09-21T03:42:01.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.957+0000 7f020f5f1640 1 -- 172.21.15.38:0/2278462177 <== osd.7 v2:172.21.15.195:6818/1606106342 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v112'487 uv487 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f020800b040 con 0x559a3f05dcf0 2024-09-21T03:42:01.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.957+0000 7f0215336f40 1 -- 172.21.15.38:0/2278462177 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:15 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x559a3f06b630 con 0x559a3f05dcf0 2024-09-21T03:42:01.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.958+0000 7f020f5f1640 1 -- 172.21.15.38:0/2278462177 <== osd.7 v2:172.21.15.195:6818/1606106342 15 ==== osd_op_reply(15 10000000000.00000000 [create] v112'488 uv488 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f020800b040 con 0x559a3f05dcf0 2024-09-21T03:42:01.960 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-09-21T03:42:01.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.958+0000 7f0215336f40 1 -- 172.21.15.38:0/2278462177 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:16 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x559a3f06b630 con 0x559a3f05dcf0 2024-09-21T03:42:01.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.960+0000 7f020f5f1640 1 -- 172.21.15.38:0/2278462177 <== osd.7 v2:172.21.15.195:6818/1606106342 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v112'489 uv489 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f020800b040 con 0x559a3f05dcf0 2024-09-21T03:42:01.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.960+0000 7f0215336f40 1 -- 172.21.15.38:0/2278462177 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:17 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x559a3f06b630 con 0x559a3f05dcf0 2024-09-21T03:42:01.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.962+0000 7f020f5f1640 1 -- 172.21.15.38:0/2278462177 <== osd.7 v2:172.21.15.195:6818/1606106342 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v112'490 uv490 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f020800b040 con 0x559a3f05dcf0 2024-09-21T03:42:01.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.962+0000 7f0215336f40 1 -- 172.21.15.38:0/2278462177 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:18 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x559a3f06b630 con 0x559a3f05dcf0 2024-09-21T03:42:01.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.963+0000 7f020f5f1640 1 -- 172.21.15.38:0/2278462177 <== osd.7 v2:172.21.15.195:6818/1606106342 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v112'491 uv491 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f020800b040 con 0x559a3f05dcf0 2024-09-21T03:42:01.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.963+0000 7f0215336f40 1 -- 172.21.15.38:0/2278462177 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:19 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=585b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x559a3f06c2a0 con 0x559a3f05dcf0 2024-09-21T03:42:01.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.965+0000 7f020f5f1640 1 -- 172.21.15.38:0/2278462177 <== osd.7 v2:172.21.15.195:6818/1606106342 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v112'492 uv492 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f020800b040 con 0x559a3f05dcf0 2024-09-21T03:42:01.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.965+0000 7f0215336f40 1 -- 172.21.15.38:0/2278462177 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:20 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x559a3f06c2a0 con 0x559a3f05dcf0 2024-09-21T03:42:01.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.966+0000 7f020f5f1640 1 -- 172.21.15.38:0/2278462177 <== osd.7 v2:172.21.15.195:6818/1606106342 20 ==== osd_op_reply(20 600.00000000 [create] v112'493 uv467 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f020800b040 con 0x559a3f05dcf0 2024-09-21T03:42:01.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.966+0000 7f0215336f40 1 -- 172.21.15.38:0/2278462177 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:21 13.10 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e112) -- 0x559a3f06c2a0 con 0x559a3f05dcf0 2024-09-21T03:42:01.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.968+0000 7f020f5f1640 1 -- 172.21.15.38:0/2278462177 <== osd.7 v2:172.21.15.195:6818/1606106342 21 ==== osd_op_reply(21 600.00000000 [delete] v112'494 uv494 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f020800b040 con 0x559a3f05dcf0 2024-09-21T03:42:01.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.968+0000 7f0215336f40 1 -- 172.21.15.38:0/2278462177 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:22 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x559a3f06c2a0 con 0x559a3f05dcf0 2024-09-21T03:42:01.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.969+0000 7f020f5f1640 1 -- 172.21.15.38:0/2278462177 <== osd.7 v2:172.21.15.195:6818/1606106342 22 ==== osd_op_reply(22 600.00000000 [create] v112'495 uv495 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f020800b040 con 0x559a3f05dcf0 2024-09-21T03:42:01.971 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-09-21T03:42:01.972 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.970+0000 7f0215336f40 1 -- 172.21.15.38:0/2278462177 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:23 13.10 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x559a3f06d2b0 con 0x559a3f05dcf0 2024-09-21T03:42:01.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.971+0000 7f020f5f1640 1 -- 172.21.15.38:0/2278462177 <== osd.7 v2:172.21.15.195:6818/1606106342 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v112'496 uv496 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f020800b040 con 0x559a3f05dcf0 2024-09-21T03:42:01.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.971+0000 7f0215336f40 1 -- 172.21.15.38:0/2278462177 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:24 13.10 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x559a3f06d2b0 con 0x559a3f05dcf0 2024-09-21T03:42:01.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.973+0000 7f020f5f1640 1 -- 172.21.15.38:0/2278462177 <== osd.7 v2:172.21.15.195:6818/1606106342 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v112'497 uv497 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f020800b040 con 0x559a3f05dcf0 2024-09-21T03:42:01.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.973+0000 7f0215336f40 1 -- 172.21.15.38:0/2278462177 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:25 13.10 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x559a3f06c2a0 con 0x559a3f05dcf0 2024-09-21T03:42:01.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.974+0000 7f020f5f1640 1 -- 172.21.15.38:0/2278462177 <== osd.7 v2:172.21.15.195:6818/1606106342 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v112'498 uv498 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f020800b040 con 0x559a3f05dcf0 2024-09-21T03:42:01.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.974+0000 7f0215336f40 1 -- 172.21.15.38:0/2278462177 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:26 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x559a3f06c2a0 con 0x559a3f05dcf0 2024-09-21T03:42:01.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.976+0000 7f020f5f1640 1 -- 172.21.15.38:0/2278462177 <== osd.7 v2:172.21.15.195:6818/1606106342 26 ==== osd_op_reply(26 603.00000000 [create] v112'499 uv473 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f020800b040 con 0x559a3f05dcf0 2024-09-21T03:42:01.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.976+0000 7f0215336f40 1 -- 172.21.15.38:0/2278462177 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:27 13.10 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e112) -- 0x559a3f06c2a0 con 0x559a3f05dcf0 2024-09-21T03:42:01.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.977+0000 7f020f5f1640 1 -- 172.21.15.38:0/2278462177 <== osd.7 v2:172.21.15.195:6818/1606106342 27 ==== osd_op_reply(27 603.00000000 [delete] v112'500 uv500 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f020800b040 con 0x559a3f05dcf0 2024-09-21T03:42:01.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.977+0000 7f0215336f40 1 -- 172.21.15.38:0/2278462177 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:28 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x559a3f06c2a0 con 0x559a3f05dcf0 2024-09-21T03:42:01.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.978+0000 7f020f5f1640 1 -- 172.21.15.38:0/2278462177 <== osd.7 v2:172.21.15.195:6818/1606106342 28 ==== osd_op_reply(28 603.00000000 [create] v112'501 uv501 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f020800b040 con 0x559a3f05dcf0 2024-09-21T03:42:01.980 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-09-21T03:42:01.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.978+0000 7f0215336f40 1 -- 172.21.15.38:0/2278462177 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:29 13.10 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x559a3f06d2b0 con 0x559a3f05dcf0 2024-09-21T03:42:01.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.980+0000 7f020f5f1640 1 -- 172.21.15.38:0/2278462177 <== osd.7 v2:172.21.15.195:6818/1606106342 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v112'502 uv502 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f020800b040 con 0x559a3f05dcf0 2024-09-21T03:42:01.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.980+0000 7f0215336f40 1 -- 172.21.15.38:0/2278462177 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:30 13.10 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x559a3f06d2b0 con 0x559a3f05dcf0 2024-09-21T03:42:01.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.981+0000 7f020f5f1640 1 -- 172.21.15.38:0/2278462177 <== osd.7 v2:172.21.15.195:6818/1606106342 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v112'503 uv503 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f020800b040 con 0x559a3f05dcf0 2024-09-21T03:42:01.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.981+0000 7f0215336f40 1 -- 172.21.15.38:0/2278462177 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:31 13.10 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x559a3f06c2a0 con 0x559a3f05dcf0 2024-09-21T03:42:01.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.982+0000 7f020f5f1640 1 -- 172.21.15.38:0/2278462177 <== osd.7 v2:172.21.15.195:6818/1606106342 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v112'504 uv504 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f020800b040 con 0x559a3f05dcf0 2024-09-21T03:42:01.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.983+0000 7f0215336f40 1 --2- 172.21.15.38:0/2278462177 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x559a3f06c7b0 0x559a3f06ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:01.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.983+0000 7f0215336f40 1 -- 172.21.15.38:0/2278462177 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x559a3f06f110 con 0x559a3f06c7b0 2024-09-21T03:42:01.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.983+0000 7f020edf0640 1 --2- 172.21.15.38:0/2278462177 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x559a3f06c7b0 0x559a3f06ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:01.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.983+0000 7f020edf0640 1 --2- 172.21.15.38:0/2278462177 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x559a3f06c7b0 0x559a3f06ebd0 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:01.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.985+0000 7f020edf0640 1 -- 172.21.15.38:0/2278462177 <== osd.6 v2:172.21.15.195:6826/4055825675 1 ==== osd_op_reply(32 1.00000000.inode [create] v112'351 uv350 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f02000116a0 con 0x559a3f06c7b0 2024-09-21T03:42:01.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.985+0000 7f0215336f40 1 -- 172.21.15.38:0/2278462177 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e112) -- 0x559a3f06f4b0 con 0x559a3f06c7b0 2024-09-21T03:42:01.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.986+0000 7f020edf0640 1 -- 172.21.15.38:0/2278462177 <== osd.6 v2:172.21.15.195:6826/4055825675 2 ==== osd_op_reply(33 1.00000000.inode [delete] v112'352 uv352 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f02000116a0 con 0x559a3f06c7b0 2024-09-21T03:42:01.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.986+0000 7f0215336f40 1 -- 172.21.15.38:0/2278462177 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x559a3f06f850 con 0x559a3f06c7b0 2024-09-21T03:42:01.989 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-09-21T03:42:01.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.987+0000 7f020edf0640 1 -- 172.21.15.38:0/2278462177 <== osd.6 v2:172.21.15.195:6826/4055825675 3 ==== osd_op_reply(34 1.00000000.inode [create] v112'353 uv353 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f02000116a0 con 0x559a3f06c7b0 2024-09-21T03:42:01.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.988+0000 7f0215336f40 1 -- 172.21.15.38:0/2278462177 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x559a3f0700a0 con 0x559a3f06c7b0 2024-09-21T03:42:01.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.989+0000 7f020edf0640 1 -- 172.21.15.38:0/2278462177 <== osd.6 v2:172.21.15.195:6826/4055825675 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v112'354 uv354 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f02000116a0 con 0x559a3f06c7b0 2024-09-21T03:42:01.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.989+0000 7f0215336f40 1 -- 172.21.15.38:0/2278462177 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x559a3f070440 con 0x559a3f06c7b0 2024-09-21T03:42:01.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.991+0000 7f020edf0640 1 -- 172.21.15.38:0/2278462177 <== osd.6 v2:172.21.15.195:6826/4055825675 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v112'355 uv355 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f02000116a0 con 0x559a3f06c7b0 2024-09-21T03:42:01.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.991+0000 7f0215336f40 1 -- 172.21.15.38:0/2278462177 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x559a3f0707e0 con 0x559a3f06c7b0 2024-09-21T03:42:01.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.992+0000 7f020edf0640 1 -- 172.21.15.38:0/2278462177 <== osd.6 v2:172.21.15.195:6826/4055825675 6 ==== osd_op_reply(37 200.00000000 [create] v112'356 uv338 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f02000116a0 con 0x559a3f06c7b0 2024-09-21T03:42:01.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.992+0000 7f0215336f40 1 -- 172.21.15.38:0/2278462177 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e112) -- 0x559a3f070b80 con 0x559a3f06c7b0 2024-09-21T03:42:01.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.993+0000 7f020edf0640 1 -- 172.21.15.38:0/2278462177 <== osd.6 v2:172.21.15.195:6826/4055825675 7 ==== osd_op_reply(38 200.00000000 [delete] v112'357 uv357 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f02000116a0 con 0x559a3f06c7b0 2024-09-21T03:42:01.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.993+0000 7f0215336f40 1 -- 172.21.15.38:0/2278462177 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x559a3f070f20 con 0x559a3f06c7b0 2024-09-21T03:42:01.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.995+0000 7f020edf0640 1 -- 172.21.15.38:0/2278462177 <== osd.6 v2:172.21.15.195:6826/4055825675 8 ==== osd_op_reply(39 200.00000000 [create] v112'358 uv358 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0200017040 con 0x559a3f06c7b0 2024-09-21T03:42:01.997 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-09-21T03:42:01.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.995+0000 7f0215336f40 1 -- 172.21.15.38:0/2278462177 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x559a3f070f20 con 0x559a3f06c7b0 2024-09-21T03:42:01.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.996+0000 7f020edf0640 1 -- 172.21.15.38:0/2278462177 <== osd.6 v2:172.21.15.195:6826/4055825675 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v112'359 uv359 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0200017040 con 0x559a3f06c7b0 2024-09-21T03:42:01.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.996+0000 7f0215336f40 1 -- 172.21.15.38:0/2278462177 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x559a3f070f20 con 0x559a3f06c7b0 2024-09-21T03:42:01.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.997+0000 7f020edf0640 1 -- 172.21.15.38:0/2278462177 <== osd.6 v2:172.21.15.195:6826/4055825675 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v112'360 uv360 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0200017040 con 0x559a3f06c7b0 2024-09-21T03:42:01.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.997+0000 7f0215336f40 1 -- 172.21.15.38:0/2278462177 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:42 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x559a3f070f20 con 0x559a3f06c7b0 2024-09-21T03:42:02.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.999+0000 7f020edf0640 1 -- 172.21.15.38:0/2278462177 <== osd.6 v2:172.21.15.195:6826/4055825675 11 ==== osd_op_reply(42 200.00000001 [create] v112'361 uv343 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f0200017040 con 0x559a3f06c7b0 2024-09-21T03:42:02.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:01.999+0000 7f0215336f40 1 -- 172.21.15.38:0/2278462177 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:43 13.14 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e112) -- 0x559a3f070f20 con 0x559a3f06c7b0 2024-09-21T03:42:02.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.000+0000 7f020edf0640 1 -- 172.21.15.38:0/2278462177 <== osd.6 v2:172.21.15.195:6826/4055825675 12 ==== osd_op_reply(43 200.00000001 [delete] v112'362 uv362 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0200017040 con 0x559a3f06c7b0 2024-09-21T03:42:02.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.000+0000 7f0215336f40 1 -- 172.21.15.38:0/2278462177 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:44 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x559a3f070f20 con 0x559a3f06c7b0 2024-09-21T03:42:02.003 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-09-21T03:42:02.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.001+0000 7f020edf0640 1 -- 172.21.15.38:0/2278462177 <== osd.6 v2:172.21.15.195:6826/4055825675 13 ==== osd_op_reply(44 200.00000001 [create] v112'363 uv363 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0200017040 con 0x559a3f06c7b0 2024-09-21T03:42:02.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.001+0000 7f0215336f40 1 -- 172.21.15.38:0/2278462177 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:45 13.14 13:2e2fa760:::200.00000001:head [write 0~17490 in=17490b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x559a3f0753a0 con 0x559a3f06c7b0 2024-09-21T03:42:02.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.003+0000 7f020edf0640 1 -- 172.21.15.38:0/2278462177 <== osd.6 v2:172.21.15.195:6826/4055825675 14 ==== osd_op_reply(45 200.00000001 [write 0~17490] v112'364 uv364 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0200017040 con 0x559a3f06c7b0 2024-09-21T03:42:02.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.003+0000 7f0215336f40 1 -- 172.21.15.38:0/2278462177 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:46 13.14 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x559a3f0753a0 con 0x559a3f06c7b0 2024-09-21T03:42:02.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.004+0000 7f020edf0640 1 -- 172.21.15.38:0/2278462177 <== osd.6 v2:172.21.15.195:6826/4055825675 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v112'365 uv365 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0200017040 con 0x559a3f06c7b0 2024-09-21T03:42:02.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.004+0000 7f0215336f40 1 -- 172.21.15.38:0/2278462177 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:47 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x559a3f070f20 con 0x559a3f06c7b0 2024-09-21T03:42:02.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.006+0000 7f020edf0640 1 -- 172.21.15.38:0/2278462177 <== osd.6 v2:172.21.15.195:6826/4055825675 16 ==== osd_op_reply(47 606.00000000 [create] v112'366 uv349 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f0200017040 con 0x559a3f06c7b0 2024-09-21T03:42:02.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.006+0000 7f0215336f40 1 -- 172.21.15.38:0/2278462177 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:48 13.14 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e112) -- 0x559a3f070f20 con 0x559a3f06c7b0 2024-09-21T03:42:02.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.007+0000 7f020edf0640 1 -- 172.21.15.38:0/2278462177 <== osd.6 v2:172.21.15.195:6826/4055825675 17 ==== osd_op_reply(48 606.00000000 [delete] v112'367 uv367 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0200017040 con 0x559a3f06c7b0 2024-09-21T03:42:02.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.007+0000 7f0215336f40 1 -- 172.21.15.38:0/2278462177 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:49 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x559a3f070f20 con 0x559a3f06c7b0 2024-09-21T03:42:02.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.008+0000 7f020edf0640 1 -- 172.21.15.38:0/2278462177 <== osd.6 v2:172.21.15.195:6826/4055825675 18 ==== osd_op_reply(49 606.00000000 [create] v112'368 uv368 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0200017040 con 0x559a3f06c7b0 2024-09-21T03:42:02.010 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-09-21T03:42:02.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.008+0000 7f0215336f40 1 -- 172.21.15.38:0/2278462177 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:50 13.14 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x559a3f071f30 con 0x559a3f06c7b0 2024-09-21T03:42:02.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.010+0000 7f020edf0640 1 -- 172.21.15.38:0/2278462177 <== osd.6 v2:172.21.15.195:6826/4055825675 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v112'369 uv369 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0200017040 con 0x559a3f06c7b0 2024-09-21T03:42:02.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.010+0000 7f0215336f40 1 -- 172.21.15.38:0/2278462177 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:51 13.14 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x559a3f071f30 con 0x559a3f06c7b0 2024-09-21T03:42:02.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.011+0000 7f020edf0640 1 -- 172.21.15.38:0/2278462177 <== osd.6 v2:172.21.15.195:6826/4055825675 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v112'370 uv370 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0200017040 con 0x559a3f06c7b0 2024-09-21T03:42:02.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.011+0000 7f0215336f40 1 -- 172.21.15.38:0/2278462177 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:52 13.14 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x559a3f070f20 con 0x559a3f06c7b0 2024-09-21T03:42:02.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.012+0000 7f020edf0640 1 -- 172.21.15.38:0/2278462177 <== osd.6 v2:172.21.15.195:6826/4055825675 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v112'371 uv371 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0200017040 con 0x559a3f06c7b0 2024-09-21T03:42:02.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.012+0000 7f0215336f40 1 --2- 172.21.15.38:0/2278462177 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x559a3f071430 0x559a3f073850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:02.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.012+0000 7f0215336f40 1 -- 172.21.15.38:0/2278462177 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:53 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x559a3f073e20 con 0x559a3f071430 2024-09-21T03:42:02.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.012+0000 7f0207fff640 1 --2- 172.21.15.38:0/2278462177 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x559a3f071430 0x559a3f073850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:02.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.013+0000 7f0207fff640 1 --2- 172.21.15.38:0/2278462177 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x559a3f071430 0x559a3f073850 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:02.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.014+0000 7f0207fff640 1 -- 172.21.15.38:0/2278462177 <== osd.0 v2:172.21.15.38:6800/2381639433 1 ==== osd_op_reply(53 607.00000000 [create] v112'93 uv92 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f01f801d030 con 0x559a3f071430 2024-09-21T03:42:02.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.014+0000 7f0215336f40 1 -- 172.21.15.38:0/2278462177 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:54 13.1c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e112) -- 0x559a3f0742c0 con 0x559a3f071430 2024-09-21T03:42:02.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.015+0000 7f0207fff640 1 -- 172.21.15.38:0/2278462177 <== osd.0 v2:172.21.15.38:6800/2381639433 2 ==== osd_op_reply(54 607.00000000 [delete] v112'94 uv94 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f01f801d030 con 0x559a3f071430 2024-09-21T03:42:02.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.015+0000 7f0215336f40 1 -- 172.21.15.38:0/2278462177 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:55 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x559a3f0747a0 con 0x559a3f071430 2024-09-21T03:42:02.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.016+0000 7f0207fff640 1 -- 172.21.15.38:0/2278462177 <== osd.0 v2:172.21.15.38:6800/2381639433 3 ==== osd_op_reply(55 607.00000000 [create] v112'95 uv95 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f01f801d030 con 0x559a3f071430 2024-09-21T03:42:02.018 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-09-21T03:42:02.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.016+0000 7f0215336f40 1 -- 172.21.15.38:0/2278462177 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:56 13.1c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x559a3f076050 con 0x559a3f071430 2024-09-21T03:42:02.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.017+0000 7f0207fff640 1 -- 172.21.15.38:0/2278462177 <== osd.0 v2:172.21.15.38:6800/2381639433 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v112'96 uv96 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f01f801d030 con 0x559a3f071430 2024-09-21T03:42:02.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.017+0000 7f0215336f40 1 -- 172.21.15.38:0/2278462177 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:57 13.1c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x559a3f0763f0 con 0x559a3f071430 2024-09-21T03:42:02.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.019+0000 7f0207fff640 1 -- 172.21.15.38:0/2278462177 <== osd.0 v2:172.21.15.38:6800/2381639433 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v112'97 uv97 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f01f801d030 con 0x559a3f071430 2024-09-21T03:42:02.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.019+0000 7f0215336f40 1 -- 172.21.15.38:0/2278462177 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:58 13.1c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x559a3f075040 con 0x559a3f071430 2024-09-21T03:42:02.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.020+0000 7f0207fff640 1 -- 172.21.15.38:0/2278462177 <== osd.0 v2:172.21.15.38:6800/2381639433 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v112'98 uv98 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f01f801d030 con 0x559a3f071430 2024-09-21T03:42:02.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.020+0000 7f0215336f40 1 --2- 172.21.15.38:0/2278462177 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x559a3f075940 0x559a3f0787a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:02.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.020+0000 7f0215336f40 1 -- 172.21.15.38:0/2278462177 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:59 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x559a3f078ce0 con 0x559a3f075940 2024-09-21T03:42:02.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.020+0000 7f020f5f1640 1 --2- 172.21.15.38:0/2278462177 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x559a3f075940 0x559a3f0787a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:02.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.020+0000 7f020f5f1640 1 --2- 172.21.15.38:0/2278462177 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x559a3f075940 0x559a3f0787a0 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:02.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.021+0000 7f020f5f1640 1 -- 172.21.15.38:0/2278462177 <== osd.3 v2:172.21.15.38:6808/1286810294 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v112'70 uv69 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f020800b040 con 0x559a3f075940 2024-09-21T03:42:02.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.021+0000 7f0215336f40 1 -- 172.21.15.38:0/2278462177 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:60 13.6 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e112) -- 0x559a3f078ce0 con 0x559a3f075940 2024-09-21T03:42:02.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.022+0000 7f020f5f1640 1 -- 172.21.15.38:0/2278462177 <== osd.3 v2:172.21.15.38:6808/1286810294 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v112'71 uv71 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f020800b040 con 0x559a3f075940 2024-09-21T03:42:02.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.022+0000 7f0215336f40 1 -- 172.21.15.38:0/2278462177 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:61 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x559a3f078d60 con 0x559a3f075940 2024-09-21T03:42:02.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.023+0000 7f020f5f1640 1 -- 172.21.15.38:0/2278462177 <== osd.3 v2:172.21.15.38:6808/1286810294 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v112'72 uv72 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f020800b040 con 0x559a3f075940 2024-09-21T03:42:02.025 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-09-21T03:42:02.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.023+0000 7f0215336f40 1 -- 172.21.15.38:0/2278462177 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:62 13.6 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x559a3f0791a0 con 0x559a3f075940 2024-09-21T03:42:02.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.025+0000 7f020f5f1640 1 -- 172.21.15.38:0/2278462177 <== osd.3 v2:172.21.15.38:6808/1286810294 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v112'73 uv73 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f020800b040 con 0x559a3f075940 2024-09-21T03:42:02.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.025+0000 7f0215336f40 1 -- 172.21.15.38:0/2278462177 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x559a3f0791a0 con 0x559a3f071430 2024-09-21T03:42:02.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.026+0000 7f0207fff640 1 -- 172.21.15.38:0/2278462177 <== osd.0 v2:172.21.15.38:6800/2381639433 7 ==== osd_op_reply(63 608.00000000 [create] v112'93 uv92 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f01f801d030 con 0x559a3f071430 2024-09-21T03:42:02.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.026+0000 7f0215336f40 1 -- 172.21.15.38:0/2278462177 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e112) -- 0x559a3f079540 con 0x559a3f071430 2024-09-21T03:42:02.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.027+0000 7f0207fff640 1 -- 172.21.15.38:0/2278462177 <== osd.0 v2:172.21.15.38:6800/2381639433 8 ==== osd_op_reply(64 608.00000000 [delete] v112'94 uv94 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f01f801d030 con 0x559a3f071430 2024-09-21T03:42:02.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.027+0000 7f0215336f40 1 -- 172.21.15.38:0/2278462177 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x559a3f079540 con 0x559a3f071430 2024-09-21T03:42:02.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.028+0000 7f0207fff640 1 -- 172.21.15.38:0/2278462177 <== osd.0 v2:172.21.15.38:6800/2381639433 9 ==== osd_op_reply(65 608.00000000 [create] v112'95 uv95 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f01f801d030 con 0x559a3f071430 2024-09-21T03:42:02.030 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-09-21T03:42:02.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.028+0000 7f0215336f40 1 -- 172.21.15.38:0/2278462177 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x559a3f07a550 con 0x559a3f071430 2024-09-21T03:42:02.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.029+0000 7f0207fff640 1 -- 172.21.15.38:0/2278462177 <== osd.0 v2:172.21.15.38:6800/2381639433 10 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v112'96 uv96 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f01f801d030 con 0x559a3f071430 2024-09-21T03:42:02.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.029+0000 7f0215336f40 1 -- 172.21.15.38:0/2278462177 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x559a3f07a550 con 0x559a3f071430 2024-09-21T03:42:02.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.030+0000 7f0207fff640 1 -- 172.21.15.38:0/2278462177 <== osd.0 v2:172.21.15.38:6800/2381639433 11 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v112'97 uv97 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f01f801d030 con 0x559a3f071430 2024-09-21T03:42:02.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.030+0000 7f0215336f40 1 -- 172.21.15.38:0/2278462177 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x559a3f079540 con 0x559a3f071430 2024-09-21T03:42:02.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.031+0000 7f0207fff640 1 -- 172.21.15.38:0/2278462177 <== osd.0 v2:172.21.15.38:6800/2381639433 12 ==== osd_op_reply(68 608.00000000 [omap-set-header] v112'98 uv98 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f01f801d030 con 0x559a3f071430 2024-09-21T03:42:02.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.031+0000 7f0215336f40 1 -- 172.21.15.38:0/2278462177 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x559a3f079540 con 0x559a3f075940 2024-09-21T03:42:02.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.033+0000 7f020f5f1640 1 -- 172.21.15.38:0/2278462177 <== osd.3 v2:172.21.15.38:6808/1286810294 5 ==== osd_op_reply(69 604.00000000 [create] v112'171 uv165 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f020800b040 con 0x559a3f075940 2024-09-21T03:42:02.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.033+0000 7f0215336f40 1 -- 172.21.15.38:0/2278462177 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e112) -- 0x559a3f079540 con 0x559a3f075940 2024-09-21T03:42:02.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.034+0000 7f020f5f1640 1 -- 172.21.15.38:0/2278462177 <== osd.3 v2:172.21.15.38:6808/1286810294 6 ==== osd_op_reply(70 604.00000000 [delete] v112'172 uv172 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f020800b040 con 0x559a3f075940 2024-09-21T03:42:02.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.034+0000 7f0215336f40 1 -- 172.21.15.38:0/2278462177 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x559a3f079540 con 0x559a3f075940 2024-09-21T03:42:02.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.035+0000 7f020f5f1640 1 -- 172.21.15.38:0/2278462177 <== osd.3 v2:172.21.15.38:6808/1286810294 7 ==== osd_op_reply(71 604.00000000 [create] v112'173 uv173 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f020800b040 con 0x559a3f075940 2024-09-21T03:42:02.037 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-09-21T03:42:02.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.035+0000 7f0215336f40 1 -- 172.21.15.38:0/2278462177 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x559a3f07a550 con 0x559a3f075940 2024-09-21T03:42:02.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.036+0000 7f020f5f1640 1 -- 172.21.15.38:0/2278462177 <== osd.3 v2:172.21.15.38:6808/1286810294 8 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v112'174 uv174 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f020800b040 con 0x559a3f075940 2024-09-21T03:42:02.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.036+0000 7f0215336f40 1 -- 172.21.15.38:0/2278462177 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x559a3f07a550 con 0x559a3f075940 2024-09-21T03:42:02.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.037+0000 7f020f5f1640 1 -- 172.21.15.38:0/2278462177 <== osd.3 v2:172.21.15.38:6808/1286810294 9 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v112'175 uv175 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f020800b040 con 0x559a3f075940 2024-09-21T03:42:02.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.037+0000 7f0215336f40 1 -- 172.21.15.38:0/2278462177 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x559a3f079540 con 0x559a3f075940 2024-09-21T03:42:02.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.038+0000 7f020f5f1640 1 -- 172.21.15.38:0/2278462177 <== osd.3 v2:172.21.15.38:6808/1286810294 10 ==== osd_op_reply(74 604.00000000 [omap-set-header] v112'176 uv176 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f020800b040 con 0x559a3f075940 2024-09-21T03:42:02.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.038+0000 7f0215336f40 1 -- 172.21.15.38:0/2278462177 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x559a3f079540 con 0x559a3f075940 2024-09-21T03:42:02.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.039+0000 7f020f5f1640 1 -- 172.21.15.38:0/2278462177 <== osd.3 v2:172.21.15.38:6808/1286810294 11 ==== osd_op_reply(75 500.00000000 [create] v112'177 uv170 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f020800b040 con 0x559a3f075940 2024-09-21T03:42:02.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.039+0000 7f0215336f40 1 -- 172.21.15.38:0/2278462177 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e112) -- 0x559a3f079540 con 0x559a3f075940 2024-09-21T03:42:02.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.040+0000 7f020f5f1640 1 -- 172.21.15.38:0/2278462177 <== osd.3 v2:172.21.15.38:6808/1286810294 12 ==== osd_op_reply(76 500.00000000 [delete] v112'178 uv178 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f020800b040 con 0x559a3f075940 2024-09-21T03:42:02.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.040+0000 7f0215336f40 1 -- 172.21.15.38:0/2278462177 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x559a3f079540 con 0x559a3f075940 2024-09-21T03:42:02.043 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-09-21T03:42:02.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.041+0000 7f020f5f1640 1 -- 172.21.15.38:0/2278462177 <== osd.3 v2:172.21.15.38:6808/1286810294 13 ==== osd_op_reply(77 500.00000000 [create] v112'179 uv179 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f020800b040 con 0x559a3f075940 2024-09-21T03:42:02.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.041+0000 7f0215336f40 1 -- 172.21.15.38:0/2278462177 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x559a3f079540 con 0x559a3f075940 2024-09-21T03:42:02.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.042+0000 7f020f5f1640 1 -- 172.21.15.38:0/2278462177 <== osd.3 v2:172.21.15.38:6808/1286810294 14 ==== osd_op_reply(78 500.00000000 [write 0~90] v112'180 uv180 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f020800b040 con 0x559a3f075940 2024-09-21T03:42:02.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.042+0000 7f0215336f40 1 -- 172.21.15.38:0/2278462177 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x559a3f079540 con 0x559a3f075940 2024-09-21T03:42:02.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.043+0000 7f020f5f1640 1 -- 172.21.15.38:0/2278462177 <== osd.3 v2:172.21.15.38:6808/1286810294 15 ==== osd_op_reply(79 500.00000000 [omap-set-header] v112'181 uv181 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f020800b040 con 0x559a3f075940 2024-09-21T03:42:02.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.043+0000 7f0215336f40 1 -- 172.21.15.38:0/2278462177 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:80 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x559a3f079540 con 0x559a3f071430 2024-09-21T03:42:02.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.044+0000 7f0207fff640 1 -- 172.21.15.38:0/2278462177 <== osd.0 v2:172.21.15.38:6800/2381639433 13 ==== osd_op_reply(80 mds_snaptable [create] v112'76 uv75 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f01f801d030 con 0x559a3f071430 2024-09-21T03:42:02.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.045+0000 7f0215336f40 1 -- 172.21.15.38:0/2278462177 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:81 13.d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e112) -- 0x559a3f079540 con 0x559a3f071430 2024-09-21T03:42:02.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.046+0000 7f0207fff640 1 -- 172.21.15.38:0/2278462177 <== osd.0 v2:172.21.15.38:6800/2381639433 14 ==== osd_op_reply(81 mds_snaptable [delete] v112'77 uv77 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f01f801d030 con 0x559a3f071430 2024-09-21T03:42:02.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.046+0000 7f0215336f40 1 -- 172.21.15.38:0/2278462177 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:82 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x559a3f079540 con 0x559a3f071430 2024-09-21T03:42:02.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.047+0000 7f0207fff640 1 -- 172.21.15.38:0/2278462177 <== osd.0 v2:172.21.15.38:6800/2381639433 15 ==== osd_op_reply(82 mds_snaptable [create] v112'78 uv78 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f01f801d030 con 0x559a3f071430 2024-09-21T03:42:02.049 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-09-21T03:42:02.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.047+0000 7f0215336f40 1 -- 172.21.15.38:0/2278462177 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:83 13.d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x559a3f074bd0 con 0x559a3f071430 2024-09-21T03:42:02.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.048+0000 7f0207fff640 1 -- 172.21.15.38:0/2278462177 <== osd.0 v2:172.21.15.38:6800/2381639433 16 ==== osd_op_reply(83 mds_snaptable [write 0~70] v112'79 uv79 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f01f801d030 con 0x559a3f071430 2024-09-21T03:42:02.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.048+0000 7f0215336f40 1 -- 172.21.15.38:0/2278462177 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:84 13.d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x559a3f074bd0 con 0x559a3f071430 2024-09-21T03:42:02.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.049+0000 7f0207fff640 1 -- 172.21.15.38:0/2278462177 <== osd.0 v2:172.21.15.38:6800/2381639433 17 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v112'80 uv80 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f01f801d030 con 0x559a3f071430 2024-09-21T03:42:02.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.049+0000 7f0215336f40 1 --2- 172.21.15.38:0/2278462177 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x559a3f079f00 0x559a3f07c320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:02.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.049+0000 7f0215336f40 1 -- 172.21.15.38:0/2278462177 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x559a3f07c860 con 0x559a3f079f00 2024-09-21T03:42:02.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.049+0000 7f020edf0640 1 --2- 172.21.15.38:0/2278462177 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x559a3f079f00 0x559a3f07c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:02.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.050+0000 7f020edf0640 1 --2- 172.21.15.38:0/2278462177 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x559a3f079f00 0x559a3f07c320 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:02.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.051+0000 7f020edf0640 1 -- 172.21.15.38:0/2278462177 <== osd.1 v2:172.21.15.38:6816/1491818732 1 ==== osd_op_reply(85 605.00000000 [create] v112'93 uv92 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f0200017040 con 0x559a3f079f00 2024-09-21T03:42:02.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.051+0000 7f0215336f40 1 -- 172.21.15.38:0/2278462177 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e112) -- 0x559a3f07c860 con 0x559a3f079f00 2024-09-21T03:42:02.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.052+0000 7f020edf0640 1 -- 172.21.15.38:0/2278462177 <== osd.1 v2:172.21.15.38:6816/1491818732 2 ==== osd_op_reply(86 605.00000000 [delete] v112'94 uv94 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0200017040 con 0x559a3f079f00 2024-09-21T03:42:02.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.052+0000 7f0215336f40 1 -- 172.21.15.38:0/2278462177 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x559a3f07c8e0 con 0x559a3f079f00 2024-09-21T03:42:02.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.053+0000 7f020edf0640 1 -- 172.21.15.38:0/2278462177 <== osd.1 v2:172.21.15.38:6816/1491818732 3 ==== osd_op_reply(87 605.00000000 [create] v112'95 uv95 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0200017040 con 0x559a3f079f00 2024-09-21T03:42:02.055 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-09-21T03:42:02.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.053+0000 7f0215336f40 1 -- 172.21.15.38:0/2278462177 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x559a3f07dd10 con 0x559a3f079f00 2024-09-21T03:42:02.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.054+0000 7f020edf0640 1 -- 172.21.15.38:0/2278462177 <== osd.1 v2:172.21.15.38:6816/1491818732 4 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v112'96 uv96 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0200017040 con 0x559a3f079f00 2024-09-21T03:42:02.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.054+0000 7f0215336f40 1 -- 172.21.15.38:0/2278462177 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x559a3f07dd10 con 0x559a3f079f00 2024-09-21T03:42:02.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.055+0000 7f020edf0640 1 -- 172.21.15.38:0/2278462177 <== osd.1 v2:172.21.15.38:6816/1491818732 5 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v112'97 uv97 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0200017040 con 0x559a3f079f00 2024-09-21T03:42:02.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.055+0000 7f0215336f40 1 -- 172.21.15.38:0/2278462177 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x559a3f07cd00 con 0x559a3f079f00 2024-09-21T03:42:02.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.056+0000 7f020edf0640 1 -- 172.21.15.38:0/2278462177 <== osd.1 v2:172.21.15.38:6816/1491818732 6 ==== osd_op_reply(90 605.00000000 [omap-set-header] v112'98 uv98 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0200017040 con 0x559a3f079f00 2024-09-21T03:42:02.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.056+0000 7f0215336f40 1 -- 172.21.15.38:0/2278462177 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x559a3f07cd00 con 0x559a3f079f00 2024-09-21T03:42:02.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.057+0000 7f020edf0640 1 -- 172.21.15.38:0/2278462177 <== osd.1 v2:172.21.15.38:6816/1491818732 7 ==== osd_op_reply(91 mds0_inotable [create] v112'152 uv146 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f0200017040 con 0x559a3f079f00 2024-09-21T03:42:02.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.058+0000 7f0215336f40 1 -- 172.21.15.38:0/2278462177 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e112) -- 0x559a3f07cd00 con 0x559a3f079f00 2024-09-21T03:42:02.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.059+0000 7f020edf0640 1 -- 172.21.15.38:0/2278462177 <== osd.1 v2:172.21.15.38:6816/1491818732 8 ==== osd_op_reply(92 mds0_inotable [delete] v112'153 uv153 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f0200017040 con 0x559a3f079f00 2024-09-21T03:42:02.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.059+0000 7f0215336f40 1 -- 172.21.15.38:0/2278462177 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x559a3f07cd00 con 0x559a3f079f00 2024-09-21T03:42:02.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.060+0000 7f020edf0640 1 -- 172.21.15.38:0/2278462177 <== osd.1 v2:172.21.15.38:6816/1491818732 9 ==== osd_op_reply(93 mds0_inotable [create] v112'154 uv154 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f0200017040 con 0x559a3f079f00 2024-09-21T03:42:02.061 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-09-21T03:42:02.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.060+0000 7f0215336f40 1 -- 172.21.15.38:0/2278462177 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:94 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x559a3f07cd00 con 0x559a3f079f00 2024-09-21T03:42:02.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.061+0000 7f020edf0640 1 -- 172.21.15.38:0/2278462177 <== osd.1 v2:172.21.15.38:6816/1491818732 10 ==== osd_op_reply(94 mds0_inotable [write 0~34] v112'155 uv155 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f0200017040 con 0x559a3f079f00 2024-09-21T03:42:02.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.061+0000 7f0215336f40 1 -- 172.21.15.38:0/2278462177 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:95 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x559a3f07cd00 con 0x559a3f079f00 2024-09-21T03:42:02.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.062+0000 7f020edf0640 1 -- 172.21.15.38:0/2278462177 <== osd.1 v2:172.21.15.38:6816/1491818732 11 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v112'156 uv156 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f0200017040 con 0x559a3f079f00 2024-09-21T03:42:02.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.062+0000 7f0215336f40 1 -- 172.21.15.38:0/2278462177 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:96 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x559a3f07cd00 con 0x559a3f079f00 2024-09-21T03:42:02.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.063+0000 7f020edf0640 1 -- 172.21.15.38:0/2278462177 <== osd.1 v2:172.21.15.38:6816/1491818732 12 ==== osd_op_reply(96 100.00000000 [create] v112'157 uv151 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f0200017040 con 0x559a3f079f00 2024-09-21T03:42:02.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.063+0000 7f0215336f40 1 -- 172.21.15.38:0/2278462177 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:97 13.13 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e112) -- 0x559a3f07cd00 con 0x559a3f079f00 2024-09-21T03:42:02.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.064+0000 7f020edf0640 1 -- 172.21.15.38:0/2278462177 <== osd.1 v2:172.21.15.38:6816/1491818732 13 ==== osd_op_reply(97 100.00000000 [delete] v112'158 uv158 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0200017040 con 0x559a3f079f00 2024-09-21T03:42:02.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.064+0000 7f0215336f40 1 -- 172.21.15.38:0/2278462177 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:98 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x559a3f07cd00 con 0x559a3f079f00 2024-09-21T03:42:02.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.065+0000 7f020edf0640 1 -- 172.21.15.38:0/2278462177 <== osd.1 v2:172.21.15.38:6816/1491818732 14 ==== osd_op_reply(98 100.00000000 [create] v112'159 uv159 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0200017040 con 0x559a3f079f00 2024-09-21T03:42:02.067 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-09-21T03:42:02.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.065+0000 7f0215336f40 1 -- 172.21.15.38:0/2278462177 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:99 13.13 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x559a3f07cd00 con 0x559a3f079f00 2024-09-21T03:42:02.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.066+0000 7f020edf0640 1 -- 172.21.15.38:0/2278462177 <== osd.1 v2:172.21.15.38:6816/1491818732 15 ==== osd_op_reply(99 100.00000000 [omap-set-header] v112'160 uv160 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0200017040 con 0x559a3f079f00 2024-09-21T03:42:02.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.066+0000 7f0215336f40 1 -- 172.21.15.38:0/2278462177 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:100 13.13 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x559a3f07fb10 con 0x559a3f079f00 2024-09-21T03:42:02.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.068+0000 7f020edf0640 1 -- 172.21.15.38:0/2278462177 <== osd.1 v2:172.21.15.38:6816/1491818732 16 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v112'161 uv161 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0200017040 con 0x559a3f079f00 2024-09-21T03:42:02.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.068+0000 7f0215336f40 1 -- 172.21.15.38:0/2278462177 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x559a3f07d1b0 con 0x559a3f075940 2024-09-21T03:42:02.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.069+0000 7f020f5f1640 1 -- 172.21.15.38:0/2278462177 <== osd.3 v2:172.21.15.38:6808/1286810294 16 ==== osd_op_reply(101 mds0_sessionmap [create] v112'70 uv69 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7f020800b040 con 0x559a3f075940 2024-09-21T03:42:02.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.069+0000 7f0215336f40 1 -- 172.21.15.38:0/2278462177 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e112) -- 0x559a3f07d1b0 con 0x559a3f075940 2024-09-21T03:42:02.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.070+0000 7f020f5f1640 1 -- 172.21.15.38:0/2278462177 <== osd.3 v2:172.21.15.38:6808/1286810294 17 ==== osd_op_reply(102 mds0_sessionmap [delete] v112'71 uv71 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f020800b040 con 0x559a3f075940 2024-09-21T03:42:02.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.070+0000 7f0215336f40 1 -- 172.21.15.38:0/2278462177 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:103 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x559a3f07d1b0 con 0x559a3f075940 2024-09-21T03:42:02.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.071+0000 7f020f5f1640 1 -- 172.21.15.38:0/2278462177 <== osd.3 v2:172.21.15.38:6808/1286810294 18 ==== osd_op_reply(103 mds0_sessionmap [create] v112'72 uv72 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f020800b040 con 0x559a3f075940 2024-09-21T03:42:02.073 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-09-21T03:42:02.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.071+0000 7f0215336f40 1 -- 172.21.15.38:0/2278462177 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:104 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x559a3f07d1b0 con 0x559a3f075940 2024-09-21T03:42:02.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.072+0000 7f020f5f1640 1 -- 172.21.15.38:0/2278462177 <== osd.3 v2:172.21.15.38:6808/1286810294 19 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v112'73 uv73 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f020800b040 con 0x559a3f075940 2024-09-21T03:42:02.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.072+0000 7f0215336f40 1 -- 172.21.15.38:0/2278462177 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x559a3f07d1b0 con 0x559a3f075940 2024-09-21T03:42:02.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.073+0000 7f020f5f1640 1 -- 172.21.15.38:0/2278462177 <== osd.3 v2:172.21.15.38:6808/1286810294 20 ==== osd_op_reply(105 609.00000000 [create] v112'93 uv92 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f020800b040 con 0x559a3f075940 2024-09-21T03:42:02.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.073+0000 7f0215336f40 1 -- 172.21.15.38:0/2278462177 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e112) -- 0x559a3f07d1b0 con 0x559a3f075940 2024-09-21T03:42:02.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.074+0000 7f020f5f1640 1 -- 172.21.15.38:0/2278462177 <== osd.3 v2:172.21.15.38:6808/1286810294 21 ==== osd_op_reply(106 609.00000000 [delete] v112'94 uv94 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f020800b040 con 0x559a3f075940 2024-09-21T03:42:02.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.074+0000 7f0215336f40 1 -- 172.21.15.38:0/2278462177 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x559a3f07d1b0 con 0x559a3f075940 2024-09-21T03:42:02.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.075+0000 7f020f5f1640 1 -- 172.21.15.38:0/2278462177 <== osd.3 v2:172.21.15.38:6808/1286810294 22 ==== osd_op_reply(107 609.00000000 [create] v112'95 uv95 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f020800b040 con 0x559a3f075940 2024-09-21T03:42:02.077 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-09-21T03:42:02.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.075+0000 7f0215336f40 1 -- 172.21.15.38:0/2278462177 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x559a3f07e1c0 con 0x559a3f075940 2024-09-21T03:42:02.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.076+0000 7f020f5f1640 1 -- 172.21.15.38:0/2278462177 <== osd.3 v2:172.21.15.38:6808/1286810294 23 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v112'96 uv96 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f020800b040 con 0x559a3f075940 2024-09-21T03:42:02.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.076+0000 7f0215336f40 1 -- 172.21.15.38:0/2278462177 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:109 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x559a3f07e1c0 con 0x559a3f075940 2024-09-21T03:42:02.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.077+0000 7f020f5f1640 1 -- 172.21.15.38:0/2278462177 <== osd.3 v2:172.21.15.38:6808/1286810294 24 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v112'97 uv97 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f020800b040 con 0x559a3f075940 2024-09-21T03:42:02.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.077+0000 7f0215336f40 1 -- 172.21.15.38:0/2278462177 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:110 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x559a3f07d1b0 con 0x559a3f075940 2024-09-21T03:42:02.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.078+0000 7f020f5f1640 1 -- 172.21.15.38:0/2278462177 <== osd.3 v2:172.21.15.38:6808/1286810294 25 ==== osd_op_reply(110 609.00000000 [omap-set-header] v112'98 uv98 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f020800b040 con 0x559a3f075940 2024-09-21T03:42:02.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.079+0000 7f0215336f40 1 --2- 172.21.15.38:0/2278462177 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x559a3f07d6c0 0x559a3f07dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:02.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.079+0000 7f0215336f40 1 -- 172.21.15.38:0/2278462177 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x559a3f07e010 con 0x559a3f07d6c0 2024-09-21T03:42:02.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.079+0000 7f0207fff640 1 --2- 172.21.15.38:0/2278462177 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x559a3f07d6c0 0x559a3f07dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:02.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.079+0000 7f0207fff640 1 --2- 172.21.15.38:0/2278462177 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x559a3f07d6c0 0x559a3f07dad0 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:02.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.080+0000 7f0207fff640 1 -- 172.21.15.38:0/2278462177 <== osd.2 v2:172.21.15.38:6824/2577858902 1 ==== osd_op_reply(111 400.00000000 [create] v112'79 uv78 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f01f801d030 con 0x559a3f07d6c0 2024-09-21T03:42:02.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.080+0000 7f0215336f40 1 -- 172.21.15.38:0/2278462177 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e112) -- 0x559a3f07e0d0 con 0x559a3f07d6c0 2024-09-21T03:42:02.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.081+0000 7f0207fff640 1 -- 172.21.15.38:0/2278462177 <== osd.2 v2:172.21.15.38:6824/2577858902 2 ==== osd_op_reply(112 400.00000000 [delete] v112'80 uv80 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f01f801d030 con 0x559a3f07d6c0 2024-09-21T03:42:02.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.081+0000 7f0215336f40 1 -- 172.21.15.38:0/2278462177 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x559a3f07e090 con 0x559a3f07d6c0 2024-09-21T03:42:02.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.082+0000 7f0207fff640 1 -- 172.21.15.38:0/2278462177 <== osd.2 v2:172.21.15.38:6824/2577858902 3 ==== osd_op_reply(113 400.00000000 [create] v112'81 uv81 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f01f801d030 con 0x559a3f07d6c0 2024-09-21T03:42:02.084 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-09-21T03:42:02.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.082+0000 7f0215336f40 1 -- 172.21.15.38:0/2278462177 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:114 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x559a3f07e5c0 con 0x559a3f07d6c0 2024-09-21T03:42:02.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.084+0000 7f0207fff640 1 -- 172.21.15.38:0/2278462177 <== osd.2 v2:172.21.15.38:6824/2577858902 4 ==== osd_op_reply(114 400.00000000 [write 0~22] v112'82 uv82 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f01f801d030 con 0x559a3f07d6c0 2024-09-21T03:42:02.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.084+0000 7f0215336f40 1 -- 172.21.15.38:0/2278462177 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:115 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x559a3f07e5c0 con 0x559a3f07d6c0 2024-09-21T03:42:02.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.085+0000 7f0207fff640 1 -- 172.21.15.38:0/2278462177 <== osd.2 v2:172.21.15.38:6824/2577858902 5 ==== osd_op_reply(115 400.00000000 [omap-set-header] v112'83 uv83 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f01f801d030 con 0x559a3f07d6c0 2024-09-21T03:42:02.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.085+0000 7f0215336f40 1 -- 172.21.15.38:0/2278462177 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:116 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x559a3f07e5c0 con 0x559a3f079f00 2024-09-21T03:42:02.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.086+0000 7f020edf0640 1 -- 172.21.15.38:0/2278462177 <== osd.1 v2:172.21.15.38:6816/1491818732 17 ==== osd_op_reply(116 100.00000000.inode [create] v112'192 uv186 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7f0200017040 con 0x559a3f079f00 2024-09-21T03:42:02.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.086+0000 7f0215336f40 1 -- 172.21.15.38:0/2278462177 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:117 13.1f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e112) -- 0x559a3f07e5c0 con 0x559a3f079f00 2024-09-21T03:42:02.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.087+0000 7f020edf0640 1 -- 172.21.15.38:0/2278462177 <== osd.1 v2:172.21.15.38:6816/1491818732 18 ==== osd_op_reply(117 100.00000000.inode [delete] v112'193 uv193 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f0200017040 con 0x559a3f079f00 2024-09-21T03:42:02.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.087+0000 7f0215336f40 1 -- 172.21.15.38:0/2278462177 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:118 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x559a3f07e5c0 con 0x559a3f079f00 2024-09-21T03:42:02.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.088+0000 7f020edf0640 1 -- 172.21.15.38:0/2278462177 <== osd.1 v2:172.21.15.38:6816/1491818732 19 ==== osd_op_reply(118 100.00000000.inode [create] v112'194 uv194 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f0200017040 con 0x559a3f079f00 2024-09-21T03:42:02.090 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-09-21T03:42:02.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.088+0000 7f0215336f40 1 -- 172.21.15.38:0/2278462177 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:119 13.1f 13:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x559a3f07e5c0 con 0x559a3f079f00 2024-09-21T03:42:02.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.089+0000 7f020edf0640 1 -- 172.21.15.38:0/2278462177 <== osd.1 v2:172.21.15.38:6816/1491818732 20 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v112'195 uv195 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f0200017040 con 0x559a3f079f00 2024-09-21T03:42:02.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.089+0000 7f0215336f40 1 -- 172.21.15.38:0/2278462177 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:120 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x559a3f07e5c0 con 0x559a3f079f00 2024-09-21T03:42:02.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.090+0000 7f020edf0640 1 -- 172.21.15.38:0/2278462177 <== osd.1 v2:172.21.15.38:6816/1491818732 21 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v112'196 uv196 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f0200017040 con 0x559a3f079f00 2024-09-21T03:42:02.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.090+0000 7f0215336f40 1 -- 172.21.15.38:0/2278462177 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:121 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x559a3f07e5c0 con 0x559a3f079f00 2024-09-21T03:42:02.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.091+0000 7f020edf0640 1 -- 172.21.15.38:0/2278462177 <== osd.1 v2:172.21.15.38:6816/1491818732 22 ==== osd_op_reply(121 1.00000000 [create] v112'197 uv191 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7f0200017040 con 0x559a3f079f00 2024-09-21T03:42:02.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.091+0000 7f0215336f40 1 -- 172.21.15.38:0/2278462177 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:122 13.1f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e112) -- 0x559a3f07e5c0 con 0x559a3f079f00 2024-09-21T03:42:02.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.092+0000 7f020edf0640 1 -- 172.21.15.38:0/2278462177 <== osd.1 v2:172.21.15.38:6816/1491818732 23 ==== osd_op_reply(122 1.00000000 [delete] v112'198 uv198 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f0200017040 con 0x559a3f079f00 2024-09-21T03:42:02.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.092+0000 7f0215336f40 1 -- 172.21.15.38:0/2278462177 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:123 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x559a3f07e5c0 con 0x559a3f079f00 2024-09-21T03:42:02.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.093+0000 7f020edf0640 1 -- 172.21.15.38:0/2278462177 <== osd.1 v2:172.21.15.38:6816/1491818732 24 ==== osd_op_reply(123 1.00000000 [create] v112'199 uv199 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f0200017040 con 0x559a3f079f00 2024-09-21T03:42:02.095 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-09-21T03:42:02.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.093+0000 7f0215336f40 1 -- 172.21.15.38:0/2278462177 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:124 13.1f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x559a3f07e5c0 con 0x559a3f079f00 2024-09-21T03:42:02.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.094+0000 7f020edf0640 1 -- 172.21.15.38:0/2278462177 <== osd.1 v2:172.21.15.38:6816/1491818732 25 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v112'200 uv200 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f0200017040 con 0x559a3f079f00 2024-09-21T03:42:02.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.094+0000 7f0215336f40 1 -- 172.21.15.38:0/2278462177 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:125 13.1f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x559a3f07e5c0 con 0x559a3f079f00 2024-09-21T03:42:02.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.095+0000 7f020edf0640 1 -- 172.21.15.38:0/2278462177 <== osd.1 v2:172.21.15.38:6816/1491818732 26 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v112'201 uv201 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f0200017040 con 0x559a3f079f00 2024-09-21T03:42:02.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.095+0000 7f0215336f40 1 -- 172.21.15.38:0/2278462177 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:126 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x559a3f07e5c0 con 0x559a3f079f00 2024-09-21T03:42:02.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.096+0000 7f020edf0640 1 -- 172.21.15.38:0/2278462177 <== osd.1 v2:172.21.15.38:6816/1491818732 27 ==== osd_op_reply(126 1.00000000 [omap-set-header] v112'202 uv202 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f0200017040 con 0x559a3f079f00 2024-09-21T03:42:02.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.096+0000 7f0215336f40 1 -- 172.21.15.38:0/2278462177 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:127 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=584b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x559a3f07e9d0 con 0x559a3f079f00 2024-09-21T03:42:02.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.097+0000 7f020edf0640 1 -- 172.21.15.38:0/2278462177 <== osd.1 v2:172.21.15.38:6816/1491818732 28 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v112'203 uv203 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f0200017040 con 0x559a3f079f00 2024-09-21T03:42:02.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.098+0000 7f0215336f40 1 -- 172.21.15.38:0/2278462177 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x559a3f071430 msgr2=0x559a3f073850 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:02.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.098+0000 7f0215336f40 1 --2- 172.21.15.38:0/2278462177 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x559a3f071430 0x559a3f073850 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:02.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.098+0000 7f0215336f40 1 -- 172.21.15.38:0/2278462177 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x559a3f079f00 msgr2=0x559a3f07c320 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:02.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.098+0000 7f0215336f40 1 --2- 172.21.15.38:0/2278462177 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x559a3f079f00 0x559a3f07c320 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:02.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.098+0000 7f0215336f40 1 -- 172.21.15.38:0/2278462177 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x559a3f07d6c0 msgr2=0x559a3f07dad0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:02.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.098+0000 7f0215336f40 1 --2- 172.21.15.38:0/2278462177 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x559a3f07d6c0 0x559a3f07dad0 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:02.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.098+0000 7f0215336f40 1 -- 172.21.15.38:0/2278462177 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x559a3f075940 msgr2=0x559a3f0787a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:02.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.098+0000 7f0215336f40 1 --2- 172.21.15.38:0/2278462177 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x559a3f075940 0x559a3f0787a0 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:02.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.098+0000 7f020f5f1640 1 -- 172.21.15.38:0/2278462177 reap_dead start 2024-09-21T03:42:02.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.098+0000 7f0215336f40 1 -- 172.21.15.38:0/2278462177 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x559a3f06c7b0 msgr2=0x559a3f06ebd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:02.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.098+0000 7f0215336f40 1 --2- 172.21.15.38:0/2278462177 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x559a3f06c7b0 0x559a3f06ebd0 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:02.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.098+0000 7f020f5f1640 1 -- 172.21.15.38:0/2278462177 reap_dead start 2024-09-21T03:42:02.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.098+0000 7f0215336f40 1 -- 172.21.15.38:0/2278462177 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x559a3f05dcf0 msgr2=0x559a3efbe0b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:02.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.098+0000 7f0215336f40 1 --2- 172.21.15.38:0/2278462177 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x559a3f05dcf0 0x559a3efbe0b0 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:02.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.098+0000 7f0215336f40 1 -- 172.21.15.38:0/2278462177 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f01f00775a0 msgr2=0x7f01f0079ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:02.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.098+0000 7f0215336f40 1 --2- 172.21.15.38:0/2278462177 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f01f00775a0 0x7f01f0079ab0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f0200001d50 tx=0x7f0200003040 comp rx=0 tx=0).stop 2024-09-21T03:42:02.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.098+0000 7f0215336f40 1 -- 172.21.15.38:0/2278462177 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559a3efa9790 msgr2=0x559a3f05d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:02.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.098+0000 7f0215336f40 1 --2- 172.21.15.38:0/2278462177 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559a3efa9790 0x559a3f05d7b0 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7f01f800aa20 tx=0x7f01f800aee0 comp rx=0 tx=0).stop 2024-09-21T03:42:02.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.098+0000 7f0215336f40 1 -- 172.21.15.38:0/2278462177 shutdown_connections 2024-09-21T03:42:02.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.098+0000 7f0215336f40 1 --2- 172.21.15.38:0/2278462177 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x559a3f06c7b0 0x559a3f06ebd0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:02.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.098+0000 7f0215336f40 1 --2- 172.21.15.38:0/2278462177 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x559a3f05dcf0 0x559a3efbe0b0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:02.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.098+0000 7f0215336f40 1 --2- 172.21.15.38:0/2278462177 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559a3efa9790 0x559a3f05d7b0 unknown :-1 s=CLOSED pgs=1618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:02.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.098+0000 7f0215336f40 1 --2- 172.21.15.38:0/2278462177 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f01f00775a0 0x7f01f0079ab0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:02.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.098+0000 7f0215336f40 1 -- 172.21.15.38:0/2278462177 >> 172.21.15.38:0/2278462177 conn(0x559a3ee0fac0 msgr2=0x559a3ef1d2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:02.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.098+0000 7f0215336f40 1 -- 172.21.15.38:0/2278462177 shutdown_connections 2024-09-21T03:42:02.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.098+0000 7f0215336f40 1 -- 172.21.15.38:0/2278462177 wait complete. 2024-09-21T03:42:02.103 DEBUG:teuthology.orchestra.run.smithi038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata put 200.00000000 - 2024-09-21T03:42:02.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.157+0000 7fc248a86f40 1 Processor -- start 2024-09-21T03:42:02.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.157+0000 7fc248a86f40 1 -- start start 2024-09-21T03:42:02.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.158+0000 7fc248a86f40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e145842090 0x55e1458ccae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:02.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.158+0000 7fc248a86f40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e1458b7520 0x55e1458234a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:02.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.158+0000 7fc248a86f40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e145824080 0x55e145824490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:02.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.158+0000 7fc248a86f40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55e1457a7430 con 0x55e145824080 2024-09-21T03:42:02.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.158+0000 7fc248a86f40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55e1457b41f0 con 0x55e1458b7520 2024-09-21T03:42:02.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.158+0000 7fc248a86f40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55e145748d60 con 0x55e145842090 2024-09-21T03:42:02.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.158+0000 7fc242d41640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e145824080 0x55e145824490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:02.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.158+0000 7fc242d41640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e145824080 0x55e145824490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53344/0 (socket says 172.21.15.38:53344) 2024-09-21T03:42:02.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.158+0000 7fc242d41640 1 -- 172.21.15.38:0/4024428152 learned_addr learned my addr 172.21.15.38:0/4024428152 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:42:02.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.158+0000 7fc241d3f640 1 --2- 172.21.15.38:0/4024428152 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e1458b7520 0x55e1458234a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:02.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.159+0000 7fc242540640 1 --2- 172.21.15.38:0/4024428152 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e145842090 0x55e1458ccae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:02.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.159+0000 7fc242d41640 1 -- 172.21.15.38:0/4024428152 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e145842090 msgr2=0x55e1458ccae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:02.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.159+0000 7fc242d41640 1 --2- 172.21.15.38:0/4024428152 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e145842090 0x55e1458ccae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:02.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.159+0000 7fc242d41640 1 -- 172.21.15.38:0/4024428152 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e1458b7520 msgr2=0x55e1458234a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:02.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.159+0000 7fc242d41640 1 --2- 172.21.15.38:0/4024428152 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e1458b7520 0x55e1458234a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:02.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.159+0000 7fc242d41640 1 -- 172.21.15.38:0/4024428152 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e1457f9e40 con 0x55e145824080 2024-09-21T03:42:02.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.159+0000 7fc242d41640 1 --2- 172.21.15.38:0/4024428152 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e145824080 0x55e145824490 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7fc23c057040 tx=0x7fc23c003330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94d21332848ec84b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:02.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.159+0000 7fc240d3d640 1 -- 172.21.15.38:0/4024428152 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc23c065020 con 0x55e145824080 2024-09-21T03:42:02.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.159+0000 7fc240d3d640 1 -- 172.21.15.38:0/4024428152 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc23c061770 con 0x55e145824080 2024-09-21T03:42:02.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.159+0000 7fc240d3d640 1 -- 172.21.15.38:0/4024428152 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc23c061ba0 con 0x55e145824080 2024-09-21T03:42:02.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.160+0000 7fc248a86f40 1 -- 172.21.15.38:0/4024428152 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e145824080 msgr2=0x55e145824490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:02.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.160+0000 7fc248a86f40 1 --2- 172.21.15.38:0/4024428152 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e145824080 0x55e145824490 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7fc23c057040 tx=0x7fc23c003330 comp rx=0 tx=0).stop 2024-09-21T03:42:02.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.160+0000 7fc248a86f40 1 -- 172.21.15.38:0/4024428152 shutdown_connections 2024-09-21T03:42:02.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.160+0000 7fc248a86f40 1 --2- 172.21.15.38:0/4024428152 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e145824080 0x55e145824490 unknown :-1 s=CLOSED pgs=1621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:02.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.160+0000 7fc248a86f40 1 --2- 172.21.15.38:0/4024428152 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e1458b7520 0x55e1458234a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:02.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.160+0000 7fc248a86f40 1 --2- 172.21.15.38:0/4024428152 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e145842090 0x55e1458ccae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:02.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.160+0000 7fc248a86f40 1 -- 172.21.15.38:0/4024428152 >> 172.21.15.38:0/4024428152 conn(0x55e145719ac0 msgr2=0x55e1458cd410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:02.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.160+0000 7fc248a86f40 1 -- 172.21.15.38:0/4024428152 shutdown_connections 2024-09-21T03:42:02.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.160+0000 7fc248a86f40 1 -- 172.21.15.38:0/4024428152 wait complete. 2024-09-21T03:42:02.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.161+0000 7fc248a86f40 1 Processor -- start 2024-09-21T03:42:02.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.161+0000 7fc248a86f40 1 -- start start 2024-09-21T03:42:02.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.162+0000 7fc248a86f40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e145824080 0x55e1458ae500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:02.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.162+0000 7fc248a86f40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e145842090 0x55e1459673b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:02.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.162+0000 7fc248a86f40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e1458b7520 0x55e1458b4220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:02.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.162+0000 7fc248a86f40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55e1458cd020 con 0x55e1458b7520 2024-09-21T03:42:02.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.162+0000 7fc248a86f40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55e1458b4b50 con 0x55e145824080 2024-09-21T03:42:02.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.162+0000 7fc248a86f40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55e1458c8850 con 0x55e145842090 2024-09-21T03:42:02.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.162+0000 7fc242d41640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e1458b7520 0x55e1458b4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:02.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.162+0000 7fc242d41640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e1458b7520 0x55e1458b4220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53348/0 (socket says 172.21.15.38:53348) 2024-09-21T03:42:02.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.162+0000 7fc242d41640 1 -- 172.21.15.38:0/902589249 learned_addr learned my addr 172.21.15.38:0/902589249 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:42:02.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.162+0000 7fc242540640 1 --2- 172.21.15.38:0/902589249 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e145824080 0x55e1458ae500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:02.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.162+0000 7fc241d3f640 1 --2- 172.21.15.38:0/902589249 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e145842090 0x55e1459673b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:02.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.162+0000 7fc242d41640 1 -- 172.21.15.38:0/902589249 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e145842090 msgr2=0x55e1459673b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:02.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.162+0000 7fc242d41640 1 --2- 172.21.15.38:0/902589249 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e145842090 0x55e1459673b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:02.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.162+0000 7fc242d41640 1 -- 172.21.15.38:0/902589249 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e145824080 msgr2=0x55e1458ae500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:02.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.162+0000 7fc242d41640 1 --2- 172.21.15.38:0/902589249 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e145824080 0x55e1458ae500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:02.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.162+0000 7fc242d41640 1 -- 172.21.15.38:0/902589249 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e1458ccb20 con 0x55e1458b7520 2024-09-21T03:42:02.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.162+0000 7fc241d3f640 1 --2- 172.21.15.38:0/902589249 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e145842090 0x55e1459673b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:42:02.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.163+0000 7fc242d41640 1 --2- 172.21.15.38:0/902589249 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e1458b7520 0x55e1458b4220 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7fc23c046610 tx=0x7fc23c061ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:02.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.163+0000 7fc2337fe640 1 -- 172.21.15.38:0/902589249 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc23c075430 con 0x55e1458b7520 2024-09-21T03:42:02.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.163+0000 7fc2337fe640 1 -- 172.21.15.38:0/902589249 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc23c06e070 con 0x55e1458b7520 2024-09-21T03:42:02.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.163+0000 7fc248a86f40 1 -- 172.21.15.38:0/902589249 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55e14583f9b0 con 0x55e1458b7520 2024-09-21T03:42:02.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.163+0000 7fc2337fe640 1 -- 172.21.15.38:0/902589249 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc23c075940 con 0x55e1458b7520 2024-09-21T03:42:02.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.163+0000 7fc248a86f40 1 -- 172.21.15.38:0/902589249 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e14583ea20 con 0x55e1458b7520 2024-09-21T03:42:02.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.165+0000 7fc2337fe640 1 -- 172.21.15.38:0/902589249 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc23c068020 con 0x55e1458b7520 2024-09-21T03:42:02.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.165+0000 7fc2337fe640 1 --2- 172.21.15.38:0/902589249 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc224077570 0x7fc224079a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:02.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.165+0000 7fc2337fe640 1 -- 172.21.15.38:0/902589249 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 7680+0+0 (secure 0 0 0) 0x7fc23c0e03d0 con 0x55e1458b7520 2024-09-21T03:42:02.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.165+0000 7fc242540640 1 --2- 172.21.15.38:0/902589249 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc224077570 0x7fc224079a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:02.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.165+0000 7fc248a86f40 1 --2- 172.21.15.38:0/902589249 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x55e1458c8370 0x55e14583ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:02.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.165+0000 7fc248a86f40 1 -- 172.21.15.38:0/902589249 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:1 13.14 13:292cf221:::200.00000000:head [writefull 0~80 in=80b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55e1458b2840 con 0x55e1458c8370 2024-09-21T03:42:02.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.166+0000 7fc241d3f640 1 --2- 172.21.15.38:0/902589249 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x55e1458c8370 0x55e14583ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:02.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.166+0000 7fc242540640 1 --2- 172.21.15.38:0/902589249 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc224077570 0x7fc224079a80 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fc234000f60 tx=0x7fc234002960 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:02.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.166+0000 7fc241d3f640 1 --2- 172.21.15.38:0/902589249 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x55e1458c8370 0x55e14583ce60 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:02.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.168+0000 7fc241d3f640 1 -- 172.21.15.38:0/902589249 <== osd.6 v2:172.21.15.195:6826/4055825675 1 ==== osd_op_reply(1 200.00000000 [writefull 0~80] v112'372 uv372 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc23800e040 con 0x55e1458c8370 2024-09-21T03:42:02.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.168+0000 7fc248a86f40 1 -- 172.21.15.38:0/902589249 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x55e1458c8370 msgr2=0x55e14583ce60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:02.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.168+0000 7fc248a86f40 1 --2- 172.21.15.38:0/902589249 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x55e1458c8370 0x55e14583ce60 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:02.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.168+0000 7fc248a86f40 1 -- 172.21.15.38:0/902589249 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc224077570 msgr2=0x7fc224079a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:02.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.168+0000 7fc248a86f40 1 --2- 172.21.15.38:0/902589249 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc224077570 0x7fc224079a80 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fc234000f60 tx=0x7fc234002960 comp rx=0 tx=0).stop 2024-09-21T03:42:02.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.168+0000 7fc248a86f40 1 -- 172.21.15.38:0/902589249 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e1458b7520 msgr2=0x55e1458b4220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:02.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.168+0000 7fc248a86f40 1 --2- 172.21.15.38:0/902589249 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e1458b7520 0x55e1458b4220 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7fc23c046610 tx=0x7fc23c061ca0 comp rx=0 tx=0).stop 2024-09-21T03:42:02.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.168+0000 7fc242d41640 1 -- 172.21.15.38:0/902589249 reap_dead start 2024-09-21T03:42:02.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.168+0000 7fc248a86f40 1 -- 172.21.15.38:0/902589249 shutdown_connections 2024-09-21T03:42:02.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.168+0000 7fc248a86f40 1 -- 172.21.15.38:0/902589249 >> 172.21.15.38:0/902589249 conn(0x55e145719ac0 msgr2=0x55e145824490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:02.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.168+0000 7fc248a86f40 1 -- 172.21.15.38:0/902589249 shutdown_connections 2024-09-21T03:42:02.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.168+0000 7fc248a86f40 1 -- 172.21.15.38:0/902589249 wait complete. 2024-09-21T03:42:02.173 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-21T03:42:02.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.330+0000 7fd15fa27640 1 -- 172.21.15.38:0/472201597 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd1580d1120 msgr2=0x7fd1580ceb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:02.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.330+0000 7fd15fa27640 1 --2- 172.21.15.38:0/472201597 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd1580d1120 0x7fd1580ceb90 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7fd1580d2360 tx=0x7fd15002a790 comp rx=0 tx=0).stop 2024-09-21T03:42:02.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.330+0000 7fd15fa27640 1 -- 172.21.15.38:0/472201597 shutdown_connections 2024-09-21T03:42:02.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.330+0000 7fd15fa27640 1 --2- 172.21.15.38:0/472201597 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd1580d1120 0x7fd1580ceb90 unknown :-1 s=CLOSED pgs=1624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:02.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.330+0000 7fd15fa27640 1 --2- 172.21.15.38:0/472201597 >> v2:172.21.15.195:3300/0 conn(0x7fd1580d0170 0x7fd1580d0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:02.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.330+0000 7fd15fa27640 1 -- 172.21.15.38:0/472201597 >> 172.21.15.38:0/472201597 conn(0x7fd15809d830 msgr2=0x7fd1580b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:02.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.331+0000 7fd15fa27640 1 -- 172.21.15.38:0/472201597 shutdown_connections 2024-09-21T03:42:02.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.331+0000 7fd15fa27640 1 -- 172.21.15.38:0/472201597 wait complete. 2024-09-21T03:42:02.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.332+0000 7fd15fa27640 1 Processor -- start 2024-09-21T03:42:02.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.332+0000 7fd15fa27640 1 -- start start 2024-09-21T03:42:02.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.334+0000 7fd15fa27640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd1580d0170 0x7fd158149570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:02.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.334+0000 7fd15fa27640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd1580d1120 0x7fd1580ccb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:02.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.334+0000 7fd15fa27640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd15815ee80 0x7fd1581f5b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:02.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.334+0000 7fd15fa27640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fd1580b6700 con 0x7fd1580d0170 2024-09-21T03:42:02.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.334+0000 7fd15fa27640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd1580b6580 con 0x7fd15815ee80 2024-09-21T03:42:02.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.334+0000 7fd15fa27640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fd1580b6880 con 0x7fd1580d1120 2024-09-21T03:42:02.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.334+0000 7fd15d7c3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd1580d0170 0x7fd158149570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:02.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.334+0000 7fd15d7c3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd1580d0170 0x7fd158149570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49342/0 (socket says 172.21.15.38:49342) 2024-09-21T03:42:02.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.334+0000 7fd15d7c3640 1 -- 172.21.15.38:0/3428736735 learned_addr learned my addr 172.21.15.38:0/3428736735 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:42:02.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.334+0000 7fd15dfc4640 1 --2- 172.21.15.38:0/3428736735 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd15815ee80 0x7fd1581f5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:02.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.334+0000 7fd15cfc2640 1 --2- 172.21.15.38:0/3428736735 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd1580d1120 0x7fd1580ccb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:02.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.334+0000 7fd15d7c3640 1 -- 172.21.15.38:0/3428736735 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd1580d1120 msgr2=0x7fd1580ccb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:02.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.334+0000 7fd15d7c3640 1 --2- 172.21.15.38:0/3428736735 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd1580d1120 0x7fd1580ccb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:02.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.334+0000 7fd15d7c3640 1 -- 172.21.15.38:0/3428736735 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd15815ee80 msgr2=0x7fd1581f5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:02.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.334+0000 7fd15d7c3640 1 --2- 172.21.15.38:0/3428736735 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd15815ee80 0x7fd1581f5b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:02.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.334+0000 7fd15d7c3640 1 -- 172.21.15.38:0/3428736735 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1581f53a0 con 0x7fd1580d0170 2024-09-21T03:42:02.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.334+0000 7fd15dfc4640 1 --2- 172.21.15.38:0/3428736735 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd15815ee80 0x7fd1581f5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:42:02.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.334+0000 7fd15d7c3640 1 --2- 172.21.15.38:0/3428736735 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd1580d0170 0x7fd158149570 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7fd15400c730 tx=0x7fd15400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:02.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.335+0000 7fd14dffb640 1 -- 172.21.15.38:0/3428736735 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd154016020 con 0x7fd1580d0170 2024-09-21T03:42:02.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.335+0000 7fd14dffb640 1 -- 172.21.15.38:0/3428736735 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd15400a930 con 0x7fd1580d0170 2024-09-21T03:42:02.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.335+0000 7fd15fa27640 1 -- 172.21.15.38:0/3428736735 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1581f9170 con 0x7fd1580d0170 2024-09-21T03:42:02.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.335+0000 7fd14dffb640 1 -- 172.21.15.38:0/3428736735 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd15400ad60 con 0x7fd1580d0170 2024-09-21T03:42:02.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.335+0000 7fd15fa27640 1 -- 172.21.15.38:0/3428736735 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1581f9730 con 0x7fd1580d0170 2024-09-21T03:42:02.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.336+0000 7fd15fa27640 1 -- 172.21.15.38:0/3428736735 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1580b6da0 con 0x7fd1580d0170 2024-09-21T03:42:02.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.337+0000 7fd14dffb640 1 -- 172.21.15.38:0/3428736735 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fd154019020 con 0x7fd1580d0170 2024-09-21T03:42:02.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.337+0000 7fd14dffb640 1 --2- 172.21.15.38:0/3428736735 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd13c078b90 0x7fd13c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:02.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.337+0000 7fd14dffb640 1 -- 172.21.15.38:0/3428736735 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 7680+0+0 (secure 0 0 0) 0x7fd15409b020 con 0x7fd1580d0170 2024-09-21T03:42:02.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.338+0000 7fd15cfc2640 1 --2- 172.21.15.38:0/3428736735 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd13c078b90 0x7fd13c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:02.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.338+0000 7fd15cfc2640 1 --2- 172.21.15.38:0/3428736735 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd13c078b90 0x7fd13c07b0a0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fd148005860 tx=0x7fd1480057f0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:02.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.342+0000 7fd14dffb640 1 -- 172.21.15.38:0/3428736735 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fd154020030 con 0x7fd1580d0170 2024-09-21T03:42:02.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.571+0000 7fd15fa27640 1 -- 172.21.15.38:0/3428736735 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7fd158158d80 con 0x7fd1580d0170 2024-09-21T03:42:02.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.855+0000 7fd14dffb640 1 -- 172.21.15.38:0/3428736735 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v182) ==== 162+0+0 (secure 0 0 0) 0x7fd1581f53a0 con 0x7fd1580d0170 2024-09-21T03:42:02.857 INFO:teuthology.orchestra.run.smithi038.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-21T03:42:02.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.858+0000 7fd15fa27640 1 -- 172.21.15.38:0/3428736735 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd13c078b90 msgr2=0x7fd13c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:02.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.858+0000 7fd15fa27640 1 --2- 172.21.15.38:0/3428736735 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd13c078b90 0x7fd13c07b0a0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fd148005860 tx=0x7fd1480057f0 comp rx=0 tx=0).stop 2024-09-21T03:42:02.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.858+0000 7fd15fa27640 1 -- 172.21.15.38:0/3428736735 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd1580d0170 msgr2=0x7fd158149570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:02.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.858+0000 7fd15fa27640 1 --2- 172.21.15.38:0/3428736735 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd1580d0170 0x7fd158149570 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7fd15400c730 tx=0x7fd15400cbf0 comp rx=0 tx=0).stop 2024-09-21T03:42:02.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.858+0000 7fd15fa27640 1 -- 172.21.15.38:0/3428736735 shutdown_connections 2024-09-21T03:42:02.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.858+0000 7fd15fa27640 1 --2- 172.21.15.38:0/3428736735 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd15815ee80 0x7fd1581f5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:02.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.858+0000 7fd15fa27640 1 --2- 172.21.15.38:0/3428736735 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd13c078b90 0x7fd13c07b0a0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:02.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.858+0000 7fd15fa27640 1 --2- 172.21.15.38:0/3428736735 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd1580d1120 0x7fd1580ccb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:02.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.858+0000 7fd15fa27640 1 --2- 172.21.15.38:0/3428736735 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd1580d0170 0x7fd158149570 unknown :-1 s=CLOSED pgs=1625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:02.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.858+0000 7fd15fa27640 1 -- 172.21.15.38:0/3428736735 >> 172.21.15.38:0/3428736735 conn(0x7fd15809d830 msgr2=0x7fd1580b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:02.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.858+0000 7fd15fa27640 1 -- 172.21.15.38:0/3428736735 shutdown_connections 2024-09-21T03:42:02.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.858+0000 7fd15fa27640 1 -- 172.21.15.38:0/3428736735 wait complete. 2024-09-21T03:42:02.870 INFO:tasks.ceph.mds.a.smithi038.stderr:2024-09-21T03:42:02.868+0000 7fd1e32e2640 -1 log_channel(cluster) log [ERR] : Error recovering journal 0x200: (22) Invalid argument 2024-09-21T03:42:02.870 INFO:tasks.ceph.mon.a.smithi038.stderr:2024-09-21T03:42:02.869+0000 7f3f45713640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T03:42:02.870 INFO:tasks.ceph.mon.a.smithi038.stderr:2024-09-21T03:42:02.869+0000 7f3f45713640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-09-21T03:42:02.874 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:42:02.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.993+0000 7f1113972640 1 -- 172.21.15.38:0/2302409917 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f110c0cd1d0 msgr2=0x7f110c0b5590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:02.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.993+0000 7f1113972640 1 --2- 172.21.15.38:0/2302409917 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f110c0cd1d0 0x7f110c0b5590 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f1104005fd0 tx=0x7f110402a8b0 comp rx=0 tx=0).stop 2024-09-21T03:42:02.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.993+0000 7f1113972640 1 -- 172.21.15.38:0/2302409917 shutdown_connections 2024-09-21T03:42:02.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.993+0000 7f1113972640 1 --2- 172.21.15.38:0/2302409917 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f110c0cd1d0 0x7f110c0b5590 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:02.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.993+0000 7f1113972640 1 --2- 172.21.15.38:0/2302409917 >> v2:172.21.15.195:3300/0 conn(0x7f110c0d19d0 0x7f110c15c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:02.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.993+0000 7f1113972640 1 -- 172.21.15.38:0/2302409917 >> 172.21.15.38:0/2302409917 conn(0x7f110c0b41d0 msgr2=0x7f110c0b45e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:02.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.993+0000 7f1113972640 1 -- 172.21.15.38:0/2302409917 shutdown_connections 2024-09-21T03:42:02.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.993+0000 7f1113972640 1 -- 172.21.15.38:0/2302409917 wait complete. 2024-09-21T03:42:02.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.995+0000 7f1113972640 1 Processor -- start 2024-09-21T03:42:02.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.995+0000 7f1113972640 1 -- start start 2024-09-21T03:42:03.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.996+0000 7f1113972640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f110c0b2500 0x7f110c1f65b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:03.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.996+0000 7f1113972640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f110c0d19d0 0x7f110c1f6af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:03.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.996+0000 7f1113972640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f110c0c8dd0 0x7f110c0b08b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:03.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.996+0000 7f1113972640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f110c0b6020 con 0x7f110c0d19d0 2024-09-21T03:42:03.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.996+0000 7f1113972640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f110c0b5ea0 con 0x7f110c0c8dd0 2024-09-21T03:42:03.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.996+0000 7f1113972640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f110c0b61a0 con 0x7f110c0b2500 2024-09-21T03:42:03.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.997+0000 7f1110f0d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f110c0d19d0 0x7f110c1f6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:03.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.997+0000 7f1110f0d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f110c0d19d0 0x7f110c1f6af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49354/0 (socket says 172.21.15.38:49354) 2024-09-21T03:42:03.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.997+0000 7f1110f0d640 1 -- 172.21.15.38:0/2298107206 learned_addr learned my addr 172.21.15.38:0/2298107206 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:42:03.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.997+0000 7f1111f0f640 1 --2- 172.21.15.38:0/2298107206 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f110c0c8dd0 0x7f110c0b08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:03.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.997+0000 7f111170e640 1 --2- 172.21.15.38:0/2298107206 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f110c0b2500 0x7f110c1f65b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:03.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.997+0000 7f1110f0d640 1 -- 172.21.15.38:0/2298107206 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f110c0b2500 msgr2=0x7f110c1f65b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:03.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.997+0000 7f1110f0d640 1 --2- 172.21.15.38:0/2298107206 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f110c0b2500 0x7f110c1f65b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:03.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.997+0000 7f1110f0d640 1 -- 172.21.15.38:0/2298107206 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f110c0c8dd0 msgr2=0x7f110c0b08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:03.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.997+0000 7f1110f0d640 1 --2- 172.21.15.38:0/2298107206 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f110c0c8dd0 0x7f110c0b08b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:03.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.997+0000 7f1110f0d640 1 -- 172.21.15.38:0/2298107206 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f110c15ef10 con 0x7f110c0d19d0 2024-09-21T03:42:03.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.997+0000 7f1110f0d640 1 --2- 172.21.15.38:0/2298107206 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f110c0d19d0 0x7f110c1f6af0 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7f10fc00c3d0 tx=0x7f10fc00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:03.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.997+0000 7f1101ffb640 1 -- 172.21.15.38:0/2298107206 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10fc011790 con 0x7f110c0d19d0 2024-09-21T03:42:03.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.997+0000 7f1101ffb640 1 -- 172.21.15.38:0/2298107206 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f10fc017070 con 0x7f110c0d19d0 2024-09-21T03:42:03.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.997+0000 7f1113972640 1 -- 172.21.15.38:0/2298107206 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f110c15f1f0 con 0x7f110c0d19d0 2024-09-21T03:42:03.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.997+0000 7f1101ffb640 1 -- 172.21.15.38:0/2298107206 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10fc011cc0 con 0x7f110c0d19d0 2024-09-21T03:42:03.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.998+0000 7f1113972640 1 -- 172.21.15.38:0/2298107206 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f110c15f7b0 con 0x7f110c0d19d0 2024-09-21T03:42:03.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.999+0000 7f1113972640 1 -- 172.21.15.38:0/2298107206 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f110c0b6e10 con 0x7f110c0d19d0 2024-09-21T03:42:03.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:02.999+0000 7f1101ffb640 1 -- 172.21.15.38:0/2298107206 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f10fc01c460 con 0x7f110c0d19d0 2024-09-21T03:42:03.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:03.000+0000 7f1101ffb640 1 --2- 172.21.15.38:0/2298107206 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f10f0078bc0 0x7f10f007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:03.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:03.000+0000 7f1101ffb640 1 -- 172.21.15.38:0/2298107206 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 7766+0+0 (secure 0 0 0) 0x7f10fc09b270 con 0x7f110c0d19d0 2024-09-21T03:42:03.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:03.000+0000 7f111170e640 1 --2- 172.21.15.38:0/2298107206 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f10f0078bc0 0x7f10f007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:03.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:03.001+0000 7f111170e640 1 --2- 172.21.15.38:0/2298107206 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f10f0078bc0 0x7f10f007b0d0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f11080095f0 tx=0x7f1108005f90 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:03.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:03.004+0000 7f1101ffb640 1 -- 172.21.15.38:0/2298107206 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f10fc062730 con 0x7f110c0d19d0 2024-09-21T03:42:03.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:03.230+0000 7f1113972640 1 -- 172.21.15.38:0/2298107206 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f110c0b7cb0 con 0x7f110c0d19d0 2024-09-21T03:42:03.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:03.231+0000 7f1101ffb640 1 -- 172.21.15.38:0/2298107206 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 184 v184) ==== 95+0+3566 (secure 0 0 0) 0x7f10fc067160 con 0x7f110c0d19d0 2024-09-21T03:42:03.233 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:42:03.233 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":184,"btime":"2024-09-21T03:42:02:869889+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6086,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/965972663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":965972663},{"type":"v1","addr":"172.21.15.195:6836","nonce":965972663}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6700,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/3176122334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":3176122334},{"type":"v1","addr":"172.21.15.195:6837","nonce":3176122334}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":184,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:40:02.319282+0000","modified":"2024-09-21T03:42:02.869817+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":113,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-21T03:42:03.233 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 184 2024-09-21T03:42:03.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:03.233+0000 7f1113972640 1 -- 172.21.15.38:0/2298107206 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f10f0078bc0 msgr2=0x7f10f007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:03.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:03.233+0000 7f1113972640 1 --2- 172.21.15.38:0/2298107206 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f10f0078bc0 0x7f10f007b0d0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f11080095f0 tx=0x7f1108005f90 comp rx=0 tx=0).stop 2024-09-21T03:42:03.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:03.233+0000 7f1113972640 1 -- 172.21.15.38:0/2298107206 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f110c0d19d0 msgr2=0x7f110c1f6af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:03.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:03.233+0000 7f1113972640 1 --2- 172.21.15.38:0/2298107206 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f110c0d19d0 0x7f110c1f6af0 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7f10fc00c3d0 tx=0x7f10fc00c890 comp rx=0 tx=0).stop 2024-09-21T03:42:03.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:03.233+0000 7f1113972640 1 -- 172.21.15.38:0/2298107206 shutdown_connections 2024-09-21T03:42:03.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:03.233+0000 7f1113972640 1 --2- 172.21.15.38:0/2298107206 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f110c0c8dd0 0x7f110c0b08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:03.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:03.233+0000 7f1113972640 1 --2- 172.21.15.38:0/2298107206 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f110c0d19d0 0x7f110c1f6af0 unknown :-1 s=CLOSED pgs=1626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:03.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:03.233+0000 7f1113972640 1 --2- 172.21.15.38:0/2298107206 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f10f0078bc0 0x7f10f007b0d0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:03.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:03.233+0000 7f1113972640 1 --2- 172.21.15.38:0/2298107206 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f110c0b2500 0x7f110c1f65b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:03.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:03.233+0000 7f1113972640 1 -- 172.21.15.38:0/2298107206 >> 172.21.15.38:0/2298107206 conn(0x7f110c0b41d0 msgr2=0x7f110c15b8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:03.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:03.233+0000 7f1113972640 1 -- 172.21.15.38:0/2298107206 shutdown_connections 2024-09-21T03:42:03.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:03.233+0000 7f1113972640 1 -- 172.21.15.38:0/2298107206 wait complete. 2024-09-21T03:42:03.246 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-21T03:42:03.246 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:42:03.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:03.361+0000 7fd82c939640 1 -- 172.21.15.38:0/297064041 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd82815cb10 msgr2=0x7fd82815cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:03.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:03.361+0000 7fd82c939640 1 --2- 172.21.15.38:0/297064041 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd82815cb10 0x7fd82815cf20 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7fd8280d2320 tx=0x7fd82002a530 comp rx=0 tx=0).stop 2024-09-21T03:42:03.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:03.361+0000 7fd82c939640 1 -- 172.21.15.38:0/297064041 shutdown_connections 2024-09-21T03:42:03.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:03.361+0000 7fd82c939640 1 --2- 172.21.15.38:0/297064041 >> v2:172.21.15.195:3300/0 conn(0x7fd8280d12e0 0x7fd8280d16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:03.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:03.361+0000 7fd82c939640 1 --2- 172.21.15.38:0/297064041 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd82815cb10 0x7fd82815cf20 unknown :-1 s=CLOSED pgs=1628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:03.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:03.361+0000 7fd82c939640 1 -- 172.21.15.38:0/297064041 >> 172.21.15.38:0/297064041 conn(0x7fd8280d2350 msgr2=0x7fd8280d2760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:03.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:03.361+0000 7fd82c939640 1 -- 172.21.15.38:0/297064041 shutdown_connections 2024-09-21T03:42:03.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:03.361+0000 7fd82c939640 1 -- 172.21.15.38:0/297064041 wait complete. 2024-09-21T03:42:03.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:03.363+0000 7fd82c939640 1 Processor -- start 2024-09-21T03:42:03.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:03.363+0000 7fd82c939640 1 -- start start 2024-09-21T03:42:03.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:03.365+0000 7fd82c939640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd8280d12e0 0x7fd828151bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:03.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:03.365+0000 7fd82c939640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd82815cb10 0x7fd82806f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:03.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:03.365+0000 7fd82c939640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd828160100 0x7fd82815b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:03.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:03.365+0000 7fd82c939640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fd8280b73e0 con 0x7fd82815cb10 2024-09-21T03:42:03.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:03.365+0000 7fd82c939640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd8280b7260 con 0x7fd828160100 2024-09-21T03:42:03.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:03.365+0000 7fd82c939640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fd828160c20 con 0x7fd8280d12e0 2024-09-21T03:42:03.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:03.365+0000 7fd825d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd82815cb10 0x7fd82806f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:03.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:03.365+0000 7fd825d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd82815cb10 0x7fd82806f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49362/0 (socket says 172.21.15.38:49362) 2024-09-21T03:42:03.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:03.365+0000 7fd825d9b640 1 -- 172.21.15.38:0/1314078881 learned_addr learned my addr 172.21.15.38:0/1314078881 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:42:03.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:03.365+0000 7fd82659c640 1 --2- 172.21.15.38:0/1314078881 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd8280d12e0 0x7fd828151bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:03.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:03.365+0000 7fd826d9d640 1 --2- 172.21.15.38:0/1314078881 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd828160100 0x7fd82815b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:03.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:03.365+0000 7fd825d9b640 1 -- 172.21.15.38:0/1314078881 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd8280d12e0 msgr2=0x7fd828151bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:03.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:03.365+0000 7fd825d9b640 1 --2- 172.21.15.38:0/1314078881 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd8280d12e0 0x7fd828151bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:03.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:03.365+0000 7fd825d9b640 1 -- 172.21.15.38:0/1314078881 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd828160100 msgr2=0x7fd82815b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:03.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:03.365+0000 7fd825d9b640 1 --2- 172.21.15.38:0/1314078881 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd828160100 0x7fd82815b5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:03.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:03.365+0000 7fd825d9b640 1 -- 172.21.15.38:0/1314078881 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8281f90d0 con 0x7fd82815cb10 2024-09-21T03:42:03.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:03.365+0000 7fd825d9b640 1 --2- 172.21.15.38:0/1314078881 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd82815cb10 0x7fd82806f6b0 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7fd81000c9a0 tx=0x7fd81000ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:03.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:03.366+0000 7fd81effd640 1 -- 172.21.15.38:0/1314078881 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd810007bb0 con 0x7fd82815cb10 2024-09-21T03:42:03.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:03.366+0000 7fd81effd640 1 -- 172.21.15.38:0/1314078881 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd810007e40 con 0x7fd82815cb10 2024-09-21T03:42:03.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:03.366+0000 7fd82c939640 1 -- 172.21.15.38:0/1314078881 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8281f93b0 con 0x7fd82815cb10 2024-09-21T03:42:03.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:03.366+0000 7fd81effd640 1 -- 172.21.15.38:0/1314078881 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd810005280 con 0x7fd82815cb10 2024-09-21T03:42:03.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:03.366+0000 7fd82c939640 1 -- 172.21.15.38:0/1314078881 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8281f9970 con 0x7fd82815cb10 2024-09-21T03:42:03.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:03.367+0000 7fd82c939640 1 -- 172.21.15.38:0/1314078881 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd82806fee0 con 0x7fd82815cb10 2024-09-21T03:42:03.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:03.368+0000 7fd81effd640 1 -- 172.21.15.38:0/1314078881 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fd810005420 con 0x7fd82815cb10 2024-09-21T03:42:03.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:03.368+0000 7fd81effd640 1 --2- 172.21.15.38:0/1314078881 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd808078bc0 0x7fd80807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:03.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:03.368+0000 7fd81effd640 1 -- 172.21.15.38:0/1314078881 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 7766+0+0 (secure 0 0 0) 0x7fd81009be70 con 0x7fd82815cb10 2024-09-21T03:42:03.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:03.368+0000 7fd82659c640 1 --2- 172.21.15.38:0/1314078881 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd808078bc0 0x7fd80807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:03.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:03.369+0000 7fd82659c640 1 --2- 172.21.15.38:0/1314078881 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd808078bc0 0x7fd80807b0d0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fd820034000 tx=0x7fd820004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:03.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:03.372+0000 7fd81effd640 1 -- 172.21.15.38:0/1314078881 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fd810014030 con 0x7fd82815cb10 2024-09-21T03:42:03.589 INFO:tasks.ceph.mds.c.smithi038.stderr: -40> 2024-09-21T03:41:58.835+0000 7f34dd612640 -1 mds.0.cache.ino(0x1) Corrupt inode 0x1: End of buffer [buffer:2] 2024-09-21T03:42:03.590 INFO:tasks.ceph.mds.c.smithi038.stderr: -38> 2024-09-21T03:41:58.835+0000 7f34dd612640 -1 log_channel(cluster) log [ERR] : Error loading MDS rank 0: (22) Invalid argument 2024-09-21T03:42:03.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:03.600+0000 7fd82c939640 1 -- 172.21.15.38:0/1314078881 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd8280d16f0 con 0x7fd82815cb10 2024-09-21T03:42:03.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:03.601+0000 7fd81effd640 1 -- 172.21.15.38:0/1314078881 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 184 v184) ==== 95+0+3566 (secure 0 0 0) 0x7fd810063470 con 0x7fd82815cb10 2024-09-21T03:42:03.603 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:42:03.603 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":184,"btime":"2024-09-21T03:42:02:869889+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6086,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/965972663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":965972663},{"type":"v1","addr":"172.21.15.195:6836","nonce":965972663}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6700,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/3176122334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":3176122334},{"type":"v1","addr":"172.21.15.195:6837","nonce":3176122334}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":184,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:40:02.319282+0000","modified":"2024-09-21T03:42:02.869817+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":113,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-21T03:42:03.603 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 184 2024-09-21T03:42:03.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:03.603+0000 7fd82c939640 1 -- 172.21.15.38:0/1314078881 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd808078bc0 msgr2=0x7fd80807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:03.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:03.603+0000 7fd82c939640 1 --2- 172.21.15.38:0/1314078881 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd808078bc0 0x7fd80807b0d0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fd820034000 tx=0x7fd820004730 comp rx=0 tx=0).stop 2024-09-21T03:42:03.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:03.603+0000 7fd82c939640 1 -- 172.21.15.38:0/1314078881 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd82815cb10 msgr2=0x7fd82806f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:03.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:03.603+0000 7fd82c939640 1 --2- 172.21.15.38:0/1314078881 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd82815cb10 0x7fd82806f6b0 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7fd81000c9a0 tx=0x7fd81000ce60 comp rx=0 tx=0).stop 2024-09-21T03:42:03.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:03.603+0000 7fd82c939640 1 -- 172.21.15.38:0/1314078881 shutdown_connections 2024-09-21T03:42:03.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:03.603+0000 7fd82c939640 1 --2- 172.21.15.38:0/1314078881 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd828160100 0x7fd82815b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:03.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:03.603+0000 7fd82c939640 1 --2- 172.21.15.38:0/1314078881 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd82815cb10 0x7fd82806f6b0 unknown :-1 s=CLOSED pgs=1629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:03.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:03.603+0000 7fd82c939640 1 --2- 172.21.15.38:0/1314078881 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd808078bc0 0x7fd80807b0d0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:03.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:03.603+0000 7fd82c939640 1 --2- 172.21.15.38:0/1314078881 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd8280d12e0 0x7fd828151bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:03.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:03.603+0000 7fd82c939640 1 -- 172.21.15.38:0/1314078881 >> 172.21.15.38:0/1314078881 conn(0x7fd8280d2350 msgr2=0x7fd8280d2760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:03.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:03.603+0000 7fd82c939640 1 -- 172.21.15.38:0/1314078881 shutdown_connections 2024-09-21T03:42:03.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:03.603+0000 7fd82c939640 1 -- 172.21.15.38:0/1314078881 wait complete. 2024-09-21T03:42:03.618 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-21T03:42:03.618 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:42:03.646 INFO:tasks.ceph.mds.c.smithi038.stdout:starting mds.c at 2024-09-21T03:42:03.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:03.734+0000 7f22afb2b640 1 -- 172.21.15.38:0/1301816893 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f22a8162280 msgr2=0x7f22a8162690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:03.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:03.734+0000 7f22afb2b640 1 --2- 172.21.15.38:0/1301816893 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f22a8162280 0x7f22a8162690 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f229c009830 tx=0x7f229c02a510 comp rx=0 tx=0).stop 2024-09-21T03:42:03.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:03.734+0000 7f22afb2b640 1 -- 172.21.15.38:0/1301816893 shutdown_connections 2024-09-21T03:42:03.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:03.734+0000 7f22afb2b640 1 --2- 172.21.15.38:0/1301816893 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f22a80b2e40 0x7f22a80b3250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:03.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:03.734+0000 7f22afb2b640 1 --2- 172.21.15.38:0/1301816893 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f22a8162280 0x7f22a8162690 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:03.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:03.734+0000 7f22afb2b640 1 -- 172.21.15.38:0/1301816893 >> 172.21.15.38:0/1301816893 conn(0x7f22a80b6b20 msgr2=0x7f22a80d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:03.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:03.734+0000 7f22afb2b640 1 -- 172.21.15.38:0/1301816893 shutdown_connections 2024-09-21T03:42:03.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:03.734+0000 7f22afb2b640 1 -- 172.21.15.38:0/1301816893 wait complete. 2024-09-21T03:42:03.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:03.736+0000 7f22afb2b640 1 Processor -- start 2024-09-21T03:42:03.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:03.736+0000 7f22afb2b640 1 -- start start 2024-09-21T03:42:03.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:03.737+0000 7f22afb2b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f22a80b2e40 0x7f22a80ce940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:03.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:03.737+0000 7f22afb2b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f22a8162280 0x7f22a81f84e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:03.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:03.737+0000 7f22afb2b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f22a81f8a20 0x7f22a81f8e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:03.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:03.737+0000 7f22afb2b640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f22a815e4b0 con 0x7f22a80b2e40 2024-09-21T03:42:03.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:03.737+0000 7f22afb2b640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f22a80b39d0 con 0x7f22a8162280 2024-09-21T03:42:03.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:03.737+0000 7f22afb2b640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f22a815e630 con 0x7f22a81f8a20 2024-09-21T03:42:03.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:03.738+0000 7f22ad8c7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f22a80b2e40 0x7f22a80ce940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:03.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:03.738+0000 7f22ad8c7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f22a80b2e40 0x7f22a80ce940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49376/0 (socket says 172.21.15.38:49376) 2024-09-21T03:42:03.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:03.738+0000 7f22ad8c7640 1 -- 172.21.15.38:0/3355536728 learned_addr learned my addr 172.21.15.38:0/3355536728 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:42:03.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:03.738+0000 7f22ae0c8640 1 --2- 172.21.15.38:0/3355536728 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f22a81f8a20 0x7f22a81f8e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:03.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:03.738+0000 7f22ad0c6640 1 --2- 172.21.15.38:0/3355536728 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f22a8162280 0x7f22a81f84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:03.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:03.738+0000 7f22ad8c7640 1 -- 172.21.15.38:0/3355536728 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f22a81f8a20 msgr2=0x7f22a81f8e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:03.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:03.738+0000 7f22ad8c7640 1 --2- 172.21.15.38:0/3355536728 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f22a81f8a20 0x7f22a81f8e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:03.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:03.738+0000 7f22ad8c7640 1 -- 172.21.15.38:0/3355536728 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f22a8162280 msgr2=0x7f22a81f84e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:03.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:03.738+0000 7f22ad8c7640 1 --2- 172.21.15.38:0/3355536728 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f22a8162280 0x7f22a81f84e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:03.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:03.738+0000 7f22ad8c7640 1 -- 172.21.15.38:0/3355536728 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22a80aadc0 con 0x7f22a80b2e40 2024-09-21T03:42:03.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:03.738+0000 7f22ad8c7640 1 --2- 172.21.15.38:0/3355536728 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f22a80b2e40 0x7f22a80ce940 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7f22a000c3a0 tx=0x7f22a000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:03.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:03.738+0000 7f22a67fc640 1 -- 172.21.15.38:0/3355536728 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22a0011790 con 0x7f22a80b2e40 2024-09-21T03:42:03.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:03.738+0000 7f22afb2b640 1 -- 172.21.15.38:0/3355536728 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22a80cee80 con 0x7f22a80b2e40 2024-09-21T03:42:03.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:03.738+0000 7f22a67fc640 1 -- 172.21.15.38:0/3355536728 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f22a0017070 con 0x7f22a80b2e40 2024-09-21T03:42:03.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:03.738+0000 7f22a67fc640 1 -- 172.21.15.38:0/3355536728 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22a0011cc0 con 0x7f22a80b2e40 2024-09-21T03:42:03.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:03.738+0000 7f22afb2b640 1 -- 172.21.15.38:0/3355536728 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22a814df40 con 0x7f22a80b2e40 2024-09-21T03:42:03.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:03.740+0000 7f22afb2b640 1 -- 172.21.15.38:0/3355536728 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22a80aaf90 con 0x7f22a80b2e40 2024-09-21T03:42:03.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:03.740+0000 7f22a67fc640 1 -- 172.21.15.38:0/3355536728 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f22a001c460 con 0x7f22a80b2e40 2024-09-21T03:42:03.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:03.741+0000 7f22a67fc640 1 --2- 172.21.15.38:0/3355536728 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2294078bc0 0x7f229407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:03.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:03.741+0000 7f22a67fc640 1 -- 172.21.15.38:0/3355536728 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 7766+0+0 (secure 0 0 0) 0x7f22a009b1e0 con 0x7f22a80b2e40 2024-09-21T03:42:03.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:03.741+0000 7f22ad0c6640 1 --2- 172.21.15.38:0/3355536728 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2294078bc0 0x7f229407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:03.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:03.741+0000 7f22ad0c6640 1 --2- 172.21.15.38:0/3355536728 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2294078bc0 0x7f229407b0d0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f229c034000 tx=0x7f229c004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:03.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:03.745+0000 7f22a67fc640 1 -- 172.21.15.38:0/3355536728 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f22a8162bd0 con 0x7f22a80b2e40 2024-09-21T03:42:03.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:03.977+0000 7f22afb2b640 1 -- 172.21.15.38:0/3355536728 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f22a8162f00 con 0x7f22a80b2e40 2024-09-21T03:42:03.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:03.978+0000 7f22a67fc640 1 -- 172.21.15.38:0/3355536728 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 185 v185) ==== 95+0+4392 (secure 0 0 0) 0x7f22a00627d0 con 0x7f22a80b2e40 2024-09-21T03:42:03.980 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:42:03.980 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":185,"btime":"2024-09-21T03:42:03:871556+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6086,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/965972663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":965972663},{"type":"v1","addr":"172.21.15.195:6836","nonce":965972663}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6700,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/3176122334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":3176122334},{"type":"v1","addr":"172.21.15.195:6837","nonce":3176122334}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":8805,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/3537258034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":3537258034},{"type":"v1","addr":"172.21.15.38:6833","nonce":3537258034}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":185}],"filesystems":[{"mdsmap":{"epoch":184,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:40:02.319282+0000","modified":"2024-09-21T03:42:02.869817+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":113,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-21T03:42:03.980 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 185 2024-09-21T03:42:03.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:03.980+0000 7f22afb2b640 1 -- 172.21.15.38:0/3355536728 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2294078bc0 msgr2=0x7f229407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:03.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:03.980+0000 7f22afb2b640 1 --2- 172.21.15.38:0/3355536728 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2294078bc0 0x7f229407b0d0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f229c034000 tx=0x7f229c004730 comp rx=0 tx=0).stop 2024-09-21T03:42:03.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:03.980+0000 7f22afb2b640 1 -- 172.21.15.38:0/3355536728 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f22a80b2e40 msgr2=0x7f22a80ce940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:03.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:03.980+0000 7f22afb2b640 1 --2- 172.21.15.38:0/3355536728 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f22a80b2e40 0x7f22a80ce940 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7f22a000c3a0 tx=0x7f22a000c860 comp rx=0 tx=0).stop 2024-09-21T03:42:03.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:03.980+0000 7f22afb2b640 1 -- 172.21.15.38:0/3355536728 shutdown_connections 2024-09-21T03:42:03.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:03.980+0000 7f22afb2b640 1 --2- 172.21.15.38:0/3355536728 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2294078bc0 0x7f229407b0d0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:03.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:03.980+0000 7f22afb2b640 1 --2- 172.21.15.38:0/3355536728 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f22a81f8a20 0x7f22a81f8e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:03.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:03.980+0000 7f22afb2b640 1 --2- 172.21.15.38:0/3355536728 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f22a8162280 0x7f22a81f84e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:03.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:03.980+0000 7f22afb2b640 1 --2- 172.21.15.38:0/3355536728 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f22a80b2e40 0x7f22a80ce940 unknown :-1 s=CLOSED pgs=1632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:03.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:03.980+0000 7f22afb2b640 1 -- 172.21.15.38:0/3355536728 >> 172.21.15.38:0/3355536728 conn(0x7f22a80b6b20 msgr2=0x7f22a815daa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:03.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:03.980+0000 7f22afb2b640 1 -- 172.21.15.38:0/3355536728 shutdown_connections 2024-09-21T03:42:03.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:03.980+0000 7f22afb2b640 1 -- 172.21.15.38:0/3355536728 wait complete. 2024-09-21T03:42:03.994 INFO:tasks.cephfs.test_damage:Result: Mutation 'Corrupt 200.00000000' led to DAMAGED state 2024-09-21T03:42:03.994 INFO:tasks.cephfs.test_damage:Applying mutation 'Corrupt 200.00000001' 2024-09-21T03:42:03.994 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-21T03:42:03.994 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:42:04.050 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056795,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2354 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T03:42:04.050 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi038.front.sepia.ceph.com 2024-09-21T03:42:04.050 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:42:04.050 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T03:42:04.105 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T03:42:04.106 INFO:teuthology.orchestra.run.smithi038.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-09-21T03:42:04.106 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-21T03:42:04.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:04.261+0000 7f66a448b640 1 -- 172.21.15.38:0/3924161890 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6684005250 msgr2=0x7f668400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:04.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:04.261+0000 7f66a448b640 1 --2- 172.21.15.38:0/3924161890 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6684005250 0x7f668400b9c0 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7f6694001690 tx=0x7f6684005830 comp rx=0 tx=0).stop 2024-09-21T03:42:04.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:04.261+0000 7f66a448b640 1 -- 172.21.15.38:0/3924161890 shutdown_connections 2024-09-21T03:42:04.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:04.261+0000 7f66a448b640 1 --2- 172.21.15.38:0/3924161890 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6684005250 0x7f668400b9c0 unknown :-1 s=CLOSED pgs=1634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:04.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:04.261+0000 7f66a448b640 1 -- 172.21.15.38:0/3924161890 >> 172.21.15.38:0/3924161890 conn(0x7f669c0cdca0 msgr2=0x7f669c0ce0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:04.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:04.261+0000 7f66a448b640 1 -- 172.21.15.38:0/3924161890 shutdown_connections 2024-09-21T03:42:04.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:04.261+0000 7f66a448b640 1 -- 172.21.15.38:0/3924161890 wait complete. 2024-09-21T03:42:04.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:04.263+0000 7f66a448b640 1 Processor -- start 2024-09-21T03:42:04.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:04.263+0000 7f66a448b640 1 -- start start 2024-09-21T03:42:04.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:04.264+0000 7f66a448b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6684005250 0x7f669c14f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:04.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:04.264+0000 7f66a448b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f669c1536a0 0x7f669c153ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:04.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:04.264+0000 7f66a448b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f669c1f6e00 0x7f669c1f7210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:04.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:04.264+0000 7f66a448b640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f669c15bc00 con 0x7f6684005250 2024-09-21T03:42:04.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:04.264+0000 7f66a448b640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f669c15ba80 con 0x7f669c1536a0 2024-09-21T03:42:04.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:04.264+0000 7f66a448b640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f669c15bd80 con 0x7f669c1f6e00 2024-09-21T03:42:04.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:04.264+0000 7f66a2227640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6684005250 0x7f669c14f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:04.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:04.264+0000 7f66a2227640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6684005250 0x7f669c14f030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49394/0 (socket says 172.21.15.38:49394) 2024-09-21T03:42:04.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:04.264+0000 7f66a2227640 1 -- 172.21.15.38:0/3799201110 learned_addr learned my addr 172.21.15.38:0/3799201110 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:42:04.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:04.265+0000 7f66a1a26640 1 --2- 172.21.15.38:0/3799201110 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f669c1536a0 0x7f669c153ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:04.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:04.265+0000 7f66a2a28640 1 --2- 172.21.15.38:0/3799201110 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f669c1f6e00 0x7f669c1f7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:04.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:04.265+0000 7f66a2227640 1 -- 172.21.15.38:0/3799201110 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f669c1f6e00 msgr2=0x7f669c1f7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:04.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:04.265+0000 7f66a2227640 1 --2- 172.21.15.38:0/3799201110 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f669c1f6e00 0x7f669c1f7210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:04.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:04.265+0000 7f66a2227640 1 -- 172.21.15.38:0/3799201110 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f669c1536a0 msgr2=0x7f669c153ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:04.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:04.265+0000 7f66a2227640 1 --2- 172.21.15.38:0/3799201110 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f669c1536a0 0x7f669c153ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:04.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:04.265+0000 7f66a2227640 1 -- 172.21.15.38:0/3799201110 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f669c1f38c0 con 0x7f6684005250 2024-09-21T03:42:04.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:04.265+0000 7f66a2a28640 1 --2- 172.21.15.38:0/3799201110 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f669c1f6e00 0x7f669c1f7210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:42:04.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:04.265+0000 7f66a2227640 1 --2- 172.21.15.38:0/3799201110 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6684005250 0x7f669c14f030 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7f668c011c00 tx=0x7f668c00f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:04.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:04.265+0000 7f669affd640 1 -- 172.21.15.38:0/3799201110 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f668c003e40 con 0x7f6684005250 2024-09-21T03:42:04.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:04.265+0000 7f669affd640 1 -- 172.21.15.38:0/3799201110 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f668c004050 con 0x7f6684005250 2024-09-21T03:42:04.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:04.265+0000 7f669affd640 1 -- 172.21.15.38:0/3799201110 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f668c004460 con 0x7f6684005250 2024-09-21T03:42:04.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:04.265+0000 7f66a448b640 1 -- 172.21.15.38:0/3799201110 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f669c1f3ba0 con 0x7f6684005250 2024-09-21T03:42:04.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:04.265+0000 7f66a448b640 1 -- 172.21.15.38:0/3799201110 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f669c14f570 con 0x7f6684005250 2024-09-21T03:42:04.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:04.266+0000 7f66a448b640 1 -- 172.21.15.38:0/3799201110 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f669c15f630 con 0x7f6684005250 2024-09-21T03:42:04.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:04.267+0000 7f669affd640 1 -- 172.21.15.38:0/3799201110 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f668c004600 con 0x7f6684005250 2024-09-21T03:42:04.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:04.267+0000 7f669affd640 1 --2- 172.21.15.38:0/3799201110 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6688078b90 0x7f668807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:04.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:04.268+0000 7f669affd640 1 -- 172.21.15.38:0/3799201110 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 7766+0+0 (secure 0 0 0) 0x7f668c09cc00 con 0x7f6684005250 2024-09-21T03:42:04.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:04.268+0000 7f66a1a26640 1 --2- 172.21.15.38:0/3799201110 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6688078b90 0x7f668807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:04.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:04.268+0000 7f66a1a26640 1 --2- 172.21.15.38:0/3799201110 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6688078b90 0x7f668807b0a0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f669c1529a0 tx=0x7f66900067a0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:04.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:04.272+0000 7f669affd640 1 -- 172.21.15.38:0/3799201110 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f668c064820 con 0x7f6684005250 2024-09-21T03:42:04.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:04.497+0000 7f66a448b640 1 -- 172.21.15.38:0/3799201110 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f669c1f3d70 con 0x7f6684005250 2024-09-21T03:42:04.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:04.878+0000 7f669affd640 1 -- 172.21.15.38:0/3799201110 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v186) ==== 161+0+0 (secure 0 0 0) 0x7f668c064120 con 0x7f6684005250 2024-09-21T03:42:04.880 INFO:teuthology.orchestra.run.smithi038.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-21T03:42:04.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:04.881+0000 7f66a448b640 1 -- 172.21.15.38:0/3799201110 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6688078b90 msgr2=0x7f668807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:04.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:04.881+0000 7f66a448b640 1 --2- 172.21.15.38:0/3799201110 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6688078b90 0x7f668807b0a0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f669c1529a0 tx=0x7f66900067a0 comp rx=0 tx=0).stop 2024-09-21T03:42:04.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:04.881+0000 7f66a448b640 1 -- 172.21.15.38:0/3799201110 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6684005250 msgr2=0x7f669c14f030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:04.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:04.881+0000 7f66a448b640 1 --2- 172.21.15.38:0/3799201110 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6684005250 0x7f669c14f030 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7f668c011c00 tx=0x7f668c00f790 comp rx=0 tx=0).stop 2024-09-21T03:42:04.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:04.881+0000 7f66a448b640 1 -- 172.21.15.38:0/3799201110 shutdown_connections 2024-09-21T03:42:04.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:04.881+0000 7f66a448b640 1 --2- 172.21.15.38:0/3799201110 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6688078b90 0x7f668807b0a0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:04.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:04.881+0000 7f66a448b640 1 --2- 172.21.15.38:0/3799201110 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f669c1f6e00 0x7f669c1f7210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:04.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:04.881+0000 7f66a448b640 1 --2- 172.21.15.38:0/3799201110 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f669c1536a0 0x7f669c153ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:04.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:04.881+0000 7f66a448b640 1 --2- 172.21.15.38:0/3799201110 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6684005250 0x7f669c14f030 unknown :-1 s=CLOSED pgs=1635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:04.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:04.881+0000 7f66a448b640 1 -- 172.21.15.38:0/3799201110 >> 172.21.15.38:0/3799201110 conn(0x7f669c0cdca0 msgr2=0x7f669c0aebc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:04.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:04.881+0000 7f66a448b640 1 -- 172.21.15.38:0/3799201110 shutdown_connections 2024-09-21T03:42:04.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:04.881+0000 7f66a448b640 1 -- 172.21.15.38:0/3799201110 wait complete. 2024-09-21T03:42:04.895 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-09-21T03:42:05.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:05.013+0000 7fbfcd5cf640 1 -- 172.21.15.38:0/1942282311 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbfb4004800 msgr2=0x7fbfb4004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:05.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:05.013+0000 7fbfcd5cf640 1 --2- 172.21.15.38:0/1942282311 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbfb4004800 0x7fbfb4004c10 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7fbfb0009b50 tx=0x7fbfb4005830 comp rx=0 tx=0).stop 2024-09-21T03:42:05.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:05.013+0000 7fbfc77fe640 1 -- 172.21.15.38:0/1942282311 reap_dead start 2024-09-21T03:42:05.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:05.013+0000 7fbfcd5cf640 1 -- 172.21.15.38:0/1942282311 shutdown_connections 2024-09-21T03:42:05.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:05.013+0000 7fbfcd5cf640 1 -- 172.21.15.38:0/1942282311 >> 172.21.15.38:0/1942282311 conn(0x7fbfc80cdca0 msgr2=0x7fbfc80ce0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:05.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:05.013+0000 7fbfcd5cf640 1 -- 172.21.15.38:0/1942282311 shutdown_connections 2024-09-21T03:42:05.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:05.013+0000 7fbfcd5cf640 1 -- 172.21.15.38:0/1942282311 wait complete. 2024-09-21T03:42:05.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:05.015+0000 7fbfcd5cf640 1 Processor -- start 2024-09-21T03:42:05.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:05.015+0000 7fbfcd5cf640 1 -- start start 2024-09-21T03:42:05.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:05.016+0000 7fbfcd5cf640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbfb4004800 0x7fbfc8153200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:05.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:05.016+0000 7fbfcd5cf640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbfc806dea0 0x7fbfc806e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:05.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:05.016+0000 7fbfcd5cf640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbfc80c7560 0x7fbfc80c7970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:05.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:05.016+0000 7fbfcd5cf640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fbfc80aed00 con 0x7fbfc806dea0 2024-09-21T03:42:05.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:05.016+0000 7fbfcd5cf640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fbfc80aeb80 con 0x7fbfc80c7560 2024-09-21T03:42:05.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:05.016+0000 7fbfcd5cf640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fbfc81601a0 con 0x7fbfb4004800 2024-09-21T03:42:05.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:05.016+0000 7fbfbffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbfc806dea0 0x7fbfc806e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:05.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:05.016+0000 7fbfbffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbfc806dea0 0x7fbfc806e2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49414/0 (socket says 172.21.15.38:49414) 2024-09-21T03:42:05.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:05.016+0000 7fbfbffff640 1 -- 172.21.15.38:0/2330471129 learned_addr learned my addr 172.21.15.38:0/2330471129 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:42:05.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:05.016+0000 7fbfc6ffd640 1 --2- 172.21.15.38:0/2330471129 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbfb4004800 0x7fbfc8153200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:05.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:05.016+0000 7fbfc77fe640 1 --2- 172.21.15.38:0/2330471129 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbfc80c7560 0x7fbfc80c7970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:05.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:05.017+0000 7fbfbffff640 1 -- 172.21.15.38:0/2330471129 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbfb4004800 msgr2=0x7fbfc8153200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:05.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:05.017+0000 7fbfbffff640 1 --2- 172.21.15.38:0/2330471129 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbfb4004800 0x7fbfc8153200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:05.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:05.017+0000 7fbfbffff640 1 -- 172.21.15.38:0/2330471129 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbfc80c7560 msgr2=0x7fbfc80c7970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:05.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:05.017+0000 7fbfbffff640 1 --2- 172.21.15.38:0/2330471129 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbfc80c7560 0x7fbfc80c7970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:05.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:05.017+0000 7fbfbffff640 1 -- 172.21.15.38:0/2330471129 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfc80c7eb0 con 0x7fbfc806dea0 2024-09-21T03:42:05.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:05.017+0000 7fbfbffff640 1 --2- 172.21.15.38:0/2330471129 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbfc806dea0 0x7fbfc806e2b0 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7fbfb0009a70 tx=0x7fbfb0009850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:05.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:05.017+0000 7fbfbf7fe640 1 -- 172.21.15.38:0/2330471129 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfb0038050 con 0x7fbfc806dea0 2024-09-21T03:42:05.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:05.017+0000 7fbfcd5cf640 1 -- 172.21.15.38:0/2330471129 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfc81f6530 con 0x7fbfc806dea0 2024-09-21T03:42:05.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:05.017+0000 7fbfbf7fe640 1 -- 172.21.15.38:0/2330471129 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbfb0039040 con 0x7fbfc806dea0 2024-09-21T03:42:05.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:05.017+0000 7fbfbf7fe640 1 -- 172.21.15.38:0/2330471129 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfb0033e40 con 0x7fbfc806dea0 2024-09-21T03:42:05.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:05.017+0000 7fbfcd5cf640 1 -- 172.21.15.38:0/2330471129 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfc81f6af0 con 0x7fbfc806dea0 2024-09-21T03:42:05.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:05.018+0000 7fbfcd5cf640 1 -- 172.21.15.38:0/2330471129 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbfc80b5a40 con 0x7fbfc806dea0 2024-09-21T03:42:05.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:05.019+0000 7fbfbf7fe640 1 -- 172.21.15.38:0/2330471129 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fbfb002b8c0 con 0x7fbfc806dea0 2024-09-21T03:42:05.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:05.019+0000 7fbfbf7fe640 1 --2- 172.21.15.38:0/2330471129 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fbfa8078bc0 0x7fbfa807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:05.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:05.019+0000 7fbfbf7fe640 1 -- 172.21.15.38:0/2330471129 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 7766+0+0 (secure 0 0 0) 0x7fbfb00bc680 con 0x7fbfc806dea0 2024-09-21T03:42:05.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:05.020+0000 7fbfc6ffd640 1 --2- 172.21.15.38:0/2330471129 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fbfa8078bc0 0x7fbfa807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:05.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:05.020+0000 7fbfc6ffd640 1 --2- 172.21.15.38:0/2330471129 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fbfa8078bc0 0x7fbfa807b0d0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fbfb8005eb0 tx=0x7fbfb8005f90 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:05.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:05.024+0000 7fbfbf7fe640 1 -- 172.21.15.38:0/2330471129 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fbfc80c7eb0 con 0x7fbfc806dea0 2024-09-21T03:42:05.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:05.290+0000 7fbfcd5cf640 1 -- 172.21.15.38:0/2330471129 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7fbfc80ce3c0 con 0x7fbfc806dea0 2024-09-21T03:42:05.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:05.885+0000 7fbfbf7fe640 1 -- 172.21.15.38:0/2330471129 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v187) ==== 101+0+0 (secure 0 0 0) 0x7fbfb0083c70 con 0x7fbfc806dea0 2024-09-21T03:42:05.887 INFO:teuthology.orchestra.run.smithi038.stderr:repaired: restoring rank 6:0 2024-09-21T03:42:05.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:05.887+0000 7fbfcd5cf640 1 -- 172.21.15.38:0/2330471129 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fbfa8078bc0 msgr2=0x7fbfa807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:05.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:05.887+0000 7fbfcd5cf640 1 --2- 172.21.15.38:0/2330471129 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fbfa8078bc0 0x7fbfa807b0d0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fbfb8005eb0 tx=0x7fbfb8005f90 comp rx=0 tx=0).stop 2024-09-21T03:42:05.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:05.887+0000 7fbfcd5cf640 1 -- 172.21.15.38:0/2330471129 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbfc806dea0 msgr2=0x7fbfc806e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:05.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:05.887+0000 7fbfcd5cf640 1 --2- 172.21.15.38:0/2330471129 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbfc806dea0 0x7fbfc806e2b0 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7fbfb0009a70 tx=0x7fbfb0009850 comp rx=0 tx=0).stop 2024-09-21T03:42:05.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:05.887+0000 7fbfcd5cf640 1 -- 172.21.15.38:0/2330471129 shutdown_connections 2024-09-21T03:42:05.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:05.887+0000 7fbfcd5cf640 1 --2- 172.21.15.38:0/2330471129 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbfc80c7560 0x7fbfc80c7970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:05.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:05.887+0000 7fbfcd5cf640 1 --2- 172.21.15.38:0/2330471129 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbfc806dea0 0x7fbfc806e2b0 unknown :-1 s=CLOSED pgs=1638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:05.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:05.887+0000 7fbfcd5cf640 1 --2- 172.21.15.38:0/2330471129 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fbfa8078bc0 0x7fbfa807b0d0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:05.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:05.887+0000 7fbfcd5cf640 1 --2- 172.21.15.38:0/2330471129 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbfb4004800 0x7fbfc8153200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:05.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:05.887+0000 7fbfcd5cf640 1 -- 172.21.15.38:0/2330471129 >> 172.21.15.38:0/2330471129 conn(0x7fbfc80cdca0 msgr2=0x7fbfc80b30b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:05.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:05.887+0000 7fbfcd5cf640 1 -- 172.21.15.38:0/2330471129 shutdown_connections 2024-09-21T03:42:05.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:05.887+0000 7fbfcd5cf640 1 -- 172.21.15.38:0/2330471129 wait complete. 2024-09-21T03:42:05.903 DEBUG:teuthology.orchestra.run.smithi038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-09-21T03:42:05.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:05.961+0000 7f58ec03cf40 1 Processor -- start 2024-09-21T03:42:05.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:05.961+0000 7f58ec03cf40 1 -- start start 2024-09-21T03:42:05.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:05.962+0000 7f58ec03cf40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5588334cf090 0x5588334b0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:05.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:05.962+0000 7f58ec03cf40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558833544520 0x55883355ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:05.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:05.962+0000 7f58ec03cf40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55883353ca40 0x558833488010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:05.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:05.962+0000 7f58ec03cf40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5588333d5d60 con 0x558833544520 2024-09-21T03:42:05.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:05.962+0000 7f58ec03cf40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x558833434430 con 0x5588334cf090 2024-09-21T03:42:05.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:05.962+0000 7f58ec03cf40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55883355c350 con 0x55883353ca40 2024-09-21T03:42:05.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:05.962+0000 7f58e52f7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558833544520 0x55883355ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:05.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:05.962+0000 7f58e52f7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558833544520 0x55883355ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49416/0 (socket says 172.21.15.38:49416) 2024-09-21T03:42:05.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:05.962+0000 7f58e52f7640 1 -- 172.21.15.38:0/1303138368 learned_addr learned my addr 172.21.15.38:0/1303138368 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:42:05.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:05.962+0000 7f58e5af8640 1 --2- 172.21.15.38:0/1303138368 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5588334cf090 0x5588334b0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:05.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:05.962+0000 7f58e62f9640 1 --2- 172.21.15.38:0/1303138368 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55883353ca40 0x558833488010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:05.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:05.963+0000 7f58e52f7640 1 -- 172.21.15.38:0/1303138368 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55883353ca40 msgr2=0x558833488010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:05.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:05.963+0000 7f58e52f7640 1 --2- 172.21.15.38:0/1303138368 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55883353ca40 0x558833488010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:05.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:05.963+0000 7f58e52f7640 1 -- 172.21.15.38:0/1303138368 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5588334cf090 msgr2=0x5588334b0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:05.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:05.963+0000 7f58e52f7640 1 --2- 172.21.15.38:0/1303138368 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5588334cf090 0x5588334b0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:05.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:05.963+0000 7f58e52f7640 1 -- 172.21.15.38:0/1303138368 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588334b1940 con 0x558833544520 2024-09-21T03:42:05.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:05.963+0000 7f58e5af8640 1 --2- 172.21.15.38:0/1303138368 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5588334cf090 0x5588334b0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:42:05.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:05.963+0000 7f58e52f7640 1 --2- 172.21.15.38:0/1303138368 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558833544520 0x55883355ce60 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f58dc001d00 tx=0x7f58dc00b5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a49c8c42b10948c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:05.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:05.963+0000 7f58d7fff640 1 -- 172.21.15.38:0/1303138368 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58dc00bd80 con 0x558833544520 2024-09-21T03:42:05.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:05.963+0000 7f58d7fff640 1 -- 172.21.15.38:0/1303138368 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f58dc001e50 con 0x558833544520 2024-09-21T03:42:05.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:05.963+0000 7f58d7fff640 1 -- 172.21.15.38:0/1303138368 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58dc00f660 con 0x558833544520 2024-09-21T03:42:05.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:05.964+0000 7f58ec03cf40 1 -- 172.21.15.38:0/1303138368 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558833544520 msgr2=0x55883355ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:05.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:05.964+0000 7f58ec03cf40 1 --2- 172.21.15.38:0/1303138368 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558833544520 0x55883355ce60 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f58dc001d00 tx=0x7f58dc00b5b0 comp rx=0 tx=0).stop 2024-09-21T03:42:05.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:05.964+0000 7f58ec03cf40 1 -- 172.21.15.38:0/1303138368 shutdown_connections 2024-09-21T03:42:05.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:05.964+0000 7f58ec03cf40 1 --2- 172.21.15.38:0/1303138368 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55883353ca40 0x558833488010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:05.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:05.964+0000 7f58ec03cf40 1 --2- 172.21.15.38:0/1303138368 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558833544520 0x55883355ce60 unknown :-1 s=CLOSED pgs=1639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:05.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:05.964+0000 7f58ec03cf40 1 --2- 172.21.15.38:0/1303138368 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5588334cf090 0x5588334b0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:05.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:05.964+0000 7f58ec03cf40 1 -- 172.21.15.38:0/1303138368 >> 172.21.15.38:0/1303138368 conn(0x5588333a6ac0 msgr2=0x5588334cd7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:05.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:05.964+0000 7f58ec03cf40 1 -- 172.21.15.38:0/1303138368 shutdown_connections 2024-09-21T03:42:05.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:05.964+0000 7f58ec03cf40 1 -- 172.21.15.38:0/1303138368 wait complete. 2024-09-21T03:42:05.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:05.965+0000 7f58ec03cf40 1 Processor -- start 2024-09-21T03:42:05.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:05.965+0000 7f58ec03cf40 1 -- start start 2024-09-21T03:42:05.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:05.966+0000 7f58ec03cf40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5588334cf090 0x55883353c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:05.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:05.966+0000 7f58ec03cf40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55883353ca40 0x5588335f8570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:05.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:05.966+0000 7f58ec03cf40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558833544520 0x55883353ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:05.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:05.966+0000 7f58ec03cf40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5588334b2160 con 0x558833544520 2024-09-21T03:42:05.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:05.966+0000 7f58ec03cf40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5588334cdf00 con 0x55883353ca40 2024-09-21T03:42:05.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:05.966+0000 7f58ec03cf40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x5588334b66e0 con 0x5588334cf090 2024-09-21T03:42:05.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:05.966+0000 7f58e62f9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558833544520 0x55883353ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:05.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:05.966+0000 7f58e62f9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558833544520 0x55883353ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49426/0 (socket says 172.21.15.38:49426) 2024-09-21T03:42:05.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:05.966+0000 7f58e62f9640 1 -- 172.21.15.38:0/1849620275 learned_addr learned my addr 172.21.15.38:0/1849620275 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:42:05.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:05.966+0000 7f58e52f7640 1 --2- 172.21.15.38:0/1849620275 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55883353ca40 0x5588335f8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:05.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:05.966+0000 7f58e5af8640 1 --2- 172.21.15.38:0/1849620275 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5588334cf090 0x55883353c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:05.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:05.966+0000 7f58e62f9640 1 -- 172.21.15.38:0/1849620275 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5588334cf090 msgr2=0x55883353c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:05.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:05.966+0000 7f58e62f9640 1 --2- 172.21.15.38:0/1849620275 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5588334cf090 0x55883353c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:05.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:05.967+0000 7f58e62f9640 1 -- 172.21.15.38:0/1849620275 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55883353ca40 msgr2=0x5588335f8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:05.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:05.967+0000 7f58e62f9640 1 --2- 172.21.15.38:0/1849620275 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55883353ca40 0x5588335f8570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:05.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:05.967+0000 7f58e62f9640 1 -- 172.21.15.38:0/1849620275 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588335e7b00 con 0x558833544520 2024-09-21T03:42:05.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:05.967+0000 7f58e52f7640 1 --2- 172.21.15.38:0/1849620275 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55883353ca40 0x5588335f8570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:42:05.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:05.967+0000 7f58e62f9640 1 --2- 172.21.15.38:0/1849620275 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558833544520 0x55883353ce50 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7f58e000f680 tx=0x7f58e000df90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:05.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:05.967+0000 7f58d6ffd640 1 -- 172.21.15.38:0/1849620275 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58e0010970 con 0x558833544520 2024-09-21T03:42:05.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:05.967+0000 7f58d6ffd640 1 -- 172.21.15.38:0/1849620275 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f58e0014070 con 0x558833544520 2024-09-21T03:42:05.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:05.967+0000 7f58ec03cf40 1 -- 172.21.15.38:0/1849620275 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55883355cfe0 con 0x558833544520 2024-09-21T03:42:05.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:05.967+0000 7f58d6ffd640 1 -- 172.21.15.38:0/1849620275 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58e0018410 con 0x558833544520 2024-09-21T03:42:05.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:05.967+0000 7f58ec03cf40 1 -- 172.21.15.38:0/1849620275 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5588335f7d70 con 0x558833544520 2024-09-21T03:42:05.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:05.969+0000 7f58d6ffd640 1 -- 172.21.15.38:0/1849620275 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f58e0016030 con 0x558833544520 2024-09-21T03:42:05.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:05.969+0000 7f58d6ffd640 1 --2- 172.21.15.38:0/1849620275 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f58c80775a0 0x7f58c8079ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:05.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:05.969+0000 7f58d6ffd640 1 -- 172.21.15.38:0/1849620275 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 7766+0+0 (secure 0 0 0) 0x7f58e009aab0 con 0x558833544520 2024-09-21T03:42:05.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:05.969+0000 7f58e5af8640 1 --2- 172.21.15.38:0/1849620275 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f58c80775a0 0x7f58c8079ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:05.976 INFO:teuthology.orchestra.run.smithi038.stdout:Importing pool 2024-09-21T03:42:05.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:05.970+0000 7f58ec03cf40 1 --2- 172.21.15.38:0/1849620275 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x5588335e7d70 0x558833600dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:05.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:05.970+0000 7f58ec03cf40 1 -- 172.21.15.38:0/1849620275 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x5588335580f0 con 0x5588335e7d70 2024-09-21T03:42:05.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:05.970+0000 7f58e52f7640 1 --2- 172.21.15.38:0/1849620275 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x5588335e7d70 0x558833600dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:05.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:05.970+0000 7f58e5af8640 1 --2- 172.21.15.38:0/1849620275 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f58c80775a0 0x7f58c8079ab0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f58d8055000 tx=0x7f58d8058040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:05.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:05.970+0000 7f58e52f7640 1 --2- 172.21.15.38:0/1849620275 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x5588335e7d70 0x558833600dd0 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:05.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:05.972+0000 7f58e52f7640 1 -- 172.21.15.38:0/1849620275 <== osd.7 v2:172.21.15.195:6818/1606106342 1 ==== osd_op_reply(1 601.00000000 [create] v113'505 uv479 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f58dc01c440 con 0x5588335e7d70 2024-09-21T03:42:05.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:05.972+0000 7f58ec03cf40 1 -- 172.21.15.38:0/1849620275 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e113) -- 0x558833601310 con 0x5588335e7d70 2024-09-21T03:42:05.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:05.974+0000 7f58e52f7640 1 -- 172.21.15.38:0/1849620275 <== osd.7 v2:172.21.15.195:6818/1606106342 2 ==== osd_op_reply(2 601.00000000 [delete] v113'506 uv506 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f58dc01c440 con 0x5588335e7d70 2024-09-21T03:42:05.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:05.974+0000 7f58ec03cf40 1 -- 172.21.15.38:0/1849620275 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x5588336016b0 con 0x5588335e7d70 2024-09-21T03:42:05.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:05.975+0000 7f58e52f7640 1 -- 172.21.15.38:0/1849620275 <== osd.7 v2:172.21.15.195:6818/1606106342 3 ==== osd_op_reply(3 601.00000000 [create] v113'507 uv507 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f58dc01c440 con 0x5588335e7d70 2024-09-21T03:42:05.978 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-09-21T03:42:05.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:05.975+0000 7f58ec03cf40 1 -- 172.21.15.38:0/1849620275 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x558833602a60 con 0x5588335e7d70 2024-09-21T03:42:05.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:05.977+0000 7f58e52f7640 1 -- 172.21.15.38:0/1849620275 <== osd.7 v2:172.21.15.195:6818/1606106342 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v113'508 uv508 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f58dc01c440 con 0x5588335e7d70 2024-09-21T03:42:05.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:05.977+0000 7f58ec03cf40 1 -- 172.21.15.38:0/1849620275 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55883353d730 con 0x5588335e7d70 2024-09-21T03:42:05.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:05.978+0000 7f58e52f7640 1 -- 172.21.15.38:0/1849620275 <== osd.7 v2:172.21.15.195:6818/1606106342 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v113'509 uv509 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f58dc01c440 con 0x5588335e7d70 2024-09-21T03:42:05.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:05.978+0000 7f58ec03cf40 1 -- 172.21.15.38:0/1849620275 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x558833601a50 con 0x5588335e7d70 2024-09-21T03:42:05.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:05.980+0000 7f58e52f7640 1 -- 172.21.15.38:0/1849620275 <== osd.7 v2:172.21.15.195:6818/1606106342 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v113'510 uv510 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f58dc01c440 con 0x5588335e7d70 2024-09-21T03:42:05.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:05.980+0000 7f58ec03cf40 1 -- 172.21.15.38:0/1849620275 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x558833601e10 con 0x5588335e7d70 2024-09-21T03:42:05.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:05.981+0000 7f58e52f7640 1 -- 172.21.15.38:0/1849620275 <== osd.7 v2:172.21.15.195:6818/1606106342 7 ==== osd_op_reply(7 602.00000000 [create] v113'511 uv485 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f58dc01c440 con 0x5588335e7d70 2024-09-21T03:42:05.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:05.981+0000 7f58ec03cf40 1 -- 172.21.15.38:0/1849620275 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e113) -- 0x5588336021b0 con 0x5588335e7d70 2024-09-21T03:42:05.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:05.983+0000 7f58e52f7640 1 -- 172.21.15.38:0/1849620275 <== osd.7 v2:172.21.15.195:6818/1606106342 8 ==== osd_op_reply(8 602.00000000 [delete] v113'512 uv512 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f58dc01c440 con 0x5588335e7d70 2024-09-21T03:42:05.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:05.983+0000 7f58ec03cf40 1 -- 172.21.15.38:0/1849620275 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x5588336021b0 con 0x5588335e7d70 2024-09-21T03:42:05.986 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-09-21T03:42:05.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:05.984+0000 7f58e52f7640 1 -- 172.21.15.38:0/1849620275 <== osd.7 v2:172.21.15.195:6818/1606106342 9 ==== osd_op_reply(9 602.00000000 [create] v113'513 uv513 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f58dc01c440 con 0x5588335e7d70 2024-09-21T03:42:05.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:05.984+0000 7f58ec03cf40 1 -- 172.21.15.38:0/1849620275 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x5588336021b0 con 0x5588335e7d70 2024-09-21T03:42:05.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:05.986+0000 7f58e52f7640 1 -- 172.21.15.38:0/1849620275 <== osd.7 v2:172.21.15.195:6818/1606106342 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v113'514 uv514 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f58dc01c440 con 0x5588335e7d70 2024-09-21T03:42:05.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:05.986+0000 7f58ec03cf40 1 -- 172.21.15.38:0/1849620275 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x5588336021b0 con 0x5588335e7d70 2024-09-21T03:42:05.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:05.987+0000 7f58e52f7640 1 -- 172.21.15.38:0/1849620275 <== osd.7 v2:172.21.15.195:6818/1606106342 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v113'515 uv515 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f58dc01c440 con 0x5588335e7d70 2024-09-21T03:42:05.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:05.987+0000 7f58ec03cf40 1 -- 172.21.15.38:0/1849620275 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x5588336021b0 con 0x5588335e7d70 2024-09-21T03:42:05.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:05.989+0000 7f58e52f7640 1 -- 172.21.15.38:0/1849620275 <== osd.7 v2:172.21.15.195:6818/1606106342 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v113'516 uv516 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f58dc01c440 con 0x5588335e7d70 2024-09-21T03:42:05.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:05.989+0000 7f58ec03cf40 1 -- 172.21.15.38:0/1849620275 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:13 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x5588336021b0 con 0x5588335e7d70 2024-09-21T03:42:05.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:05.990+0000 7f58e52f7640 1 -- 172.21.15.38:0/1849620275 <== osd.7 v2:172.21.15.195:6818/1606106342 13 ==== osd_op_reply(13 10000000000.00000000 [create] v113'517 uv492 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7f58dc01c440 con 0x5588335e7d70 2024-09-21T03:42:05.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:05.990+0000 7f58ec03cf40 1 -- 172.21.15.38:0/1849620275 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:14 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e113) -- 0x5588336021b0 con 0x5588335e7d70 2024-09-21T03:42:05.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:05.992+0000 7f58e52f7640 1 -- 172.21.15.38:0/1849620275 <== osd.7 v2:172.21.15.195:6818/1606106342 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v113'518 uv518 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f58dc01c440 con 0x5588335e7d70 2024-09-21T03:42:05.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:05.992+0000 7f58ec03cf40 1 -- 172.21.15.38:0/1849620275 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:15 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x5588336021b0 con 0x5588335e7d70 2024-09-21T03:42:05.995 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-09-21T03:42:05.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:05.994+0000 7f58e52f7640 1 -- 172.21.15.38:0/1849620275 <== osd.7 v2:172.21.15.195:6818/1606106342 15 ==== osd_op_reply(15 10000000000.00000000 [create] v113'519 uv519 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f58dc01c440 con 0x5588335e7d70 2024-09-21T03:42:05.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:05.994+0000 7f58ec03cf40 1 -- 172.21.15.38:0/1849620275 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:16 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x5588336021b0 con 0x5588335e7d70 2024-09-21T03:42:05.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:05.995+0000 7f58e52f7640 1 -- 172.21.15.38:0/1849620275 <== osd.7 v2:172.21.15.195:6818/1606106342 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v113'520 uv520 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f58dc01c440 con 0x5588335e7d70 2024-09-21T03:42:05.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:05.995+0000 7f58ec03cf40 1 -- 172.21.15.38:0/1849620275 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:17 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x5588336021b0 con 0x5588335e7d70 2024-09-21T03:42:05.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:05.996+0000 7f58e52f7640 1 -- 172.21.15.38:0/1849620275 <== osd.7 v2:172.21.15.195:6818/1606106342 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v113'521 uv521 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f58dc01c440 con 0x5588335e7d70 2024-09-21T03:42:05.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:05.996+0000 7f58ec03cf40 1 -- 172.21.15.38:0/1849620275 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:18 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x5588336021b0 con 0x5588335e7d70 2024-09-21T03:42:06.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:05.998+0000 7f58e52f7640 1 -- 172.21.15.38:0/1849620275 <== osd.7 v2:172.21.15.195:6818/1606106342 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v113'522 uv522 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f58dc01c440 con 0x5588335e7d70 2024-09-21T03:42:06.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:05.998+0000 7f58ec03cf40 1 -- 172.21.15.38:0/1849620275 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:19 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=585b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x5588336021b0 con 0x5588335e7d70 2024-09-21T03:42:06.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:05.999+0000 7f58e52f7640 1 -- 172.21.15.38:0/1849620275 <== osd.7 v2:172.21.15.195:6818/1606106342 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v113'523 uv523 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f58dc01c440 con 0x5588335e7d70 2024-09-21T03:42:06.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:05.999+0000 7f58ec03cf40 1 -- 172.21.15.38:0/1849620275 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:20 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x5588336021b0 con 0x5588335e7d70 2024-09-21T03:42:06.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.001+0000 7f58e52f7640 1 -- 172.21.15.38:0/1849620275 <== osd.7 v2:172.21.15.195:6818/1606106342 20 ==== osd_op_reply(20 600.00000000 [create] v113'524 uv498 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f58dc01c440 con 0x5588335e7d70 2024-09-21T03:42:06.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.001+0000 7f58ec03cf40 1 -- 172.21.15.38:0/1849620275 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:21 13.10 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e113) -- 0x5588336021b0 con 0x5588335e7d70 2024-09-21T03:42:06.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.002+0000 7f58e52f7640 1 -- 172.21.15.38:0/1849620275 <== osd.7 v2:172.21.15.195:6818/1606106342 21 ==== osd_op_reply(21 600.00000000 [delete] v113'525 uv525 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f58dc01c440 con 0x5588335e7d70 2024-09-21T03:42:06.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.002+0000 7f58ec03cf40 1 -- 172.21.15.38:0/1849620275 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:22 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x5588336021b0 con 0x5588335e7d70 2024-09-21T03:42:06.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.004+0000 7f58e52f7640 1 -- 172.21.15.38:0/1849620275 <== osd.7 v2:172.21.15.195:6818/1606106342 22 ==== osd_op_reply(22 600.00000000 [create] v113'526 uv526 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f58dc01c440 con 0x5588335e7d70 2024-09-21T03:42:06.006 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-09-21T03:42:06.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.004+0000 7f58ec03cf40 1 -- 172.21.15.38:0/1849620275 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:23 13.10 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x5588336021b0 con 0x5588335e7d70 2024-09-21T03:42:06.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.005+0000 7f58e52f7640 1 -- 172.21.15.38:0/1849620275 <== osd.7 v2:172.21.15.195:6818/1606106342 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v113'527 uv527 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f58dc01c440 con 0x5588335e7d70 2024-09-21T03:42:06.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.005+0000 7f58ec03cf40 1 -- 172.21.15.38:0/1849620275 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:24 13.10 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x5588336021b0 con 0x5588335e7d70 2024-09-21T03:42:06.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.007+0000 7f58e52f7640 1 -- 172.21.15.38:0/1849620275 <== osd.7 v2:172.21.15.195:6818/1606106342 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v113'528 uv528 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f58dc01c440 con 0x5588335e7d70 2024-09-21T03:42:06.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.007+0000 7f58ec03cf40 1 -- 172.21.15.38:0/1849620275 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:25 13.10 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x5588336021b0 con 0x5588335e7d70 2024-09-21T03:42:06.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.008+0000 7f58e52f7640 1 -- 172.21.15.38:0/1849620275 <== osd.7 v2:172.21.15.195:6818/1606106342 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v113'529 uv529 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f58dc01c440 con 0x5588335e7d70 2024-09-21T03:42:06.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.008+0000 7f58ec03cf40 1 -- 172.21.15.38:0/1849620275 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:26 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x5588336021b0 con 0x5588335e7d70 2024-09-21T03:42:06.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.010+0000 7f58e52f7640 1 -- 172.21.15.38:0/1849620275 <== osd.7 v2:172.21.15.195:6818/1606106342 26 ==== osd_op_reply(26 603.00000000 [create] v113'530 uv504 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f58dc01c440 con 0x5588335e7d70 2024-09-21T03:42:06.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.010+0000 7f58ec03cf40 1 -- 172.21.15.38:0/1849620275 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:27 13.10 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e113) -- 0x5588336021b0 con 0x5588335e7d70 2024-09-21T03:42:06.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.011+0000 7f58e52f7640 1 -- 172.21.15.38:0/1849620275 <== osd.7 v2:172.21.15.195:6818/1606106342 27 ==== osd_op_reply(27 603.00000000 [delete] v113'531 uv531 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f58dc01c440 con 0x5588335e7d70 2024-09-21T03:42:06.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.011+0000 7f58ec03cf40 1 -- 172.21.15.38:0/1849620275 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:28 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x5588336021b0 con 0x5588335e7d70 2024-09-21T03:42:06.014 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-09-21T03:42:06.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.013+0000 7f58e52f7640 1 -- 172.21.15.38:0/1849620275 <== osd.7 v2:172.21.15.195:6818/1606106342 28 ==== osd_op_reply(28 603.00000000 [create] v113'532 uv532 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f58dc01c440 con 0x5588335e7d70 2024-09-21T03:42:06.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.013+0000 7f58ec03cf40 1 -- 172.21.15.38:0/1849620275 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:29 13.10 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x5588336021b0 con 0x5588335e7d70 2024-09-21T03:42:06.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.014+0000 7f58e52f7640 1 -- 172.21.15.38:0/1849620275 <== osd.7 v2:172.21.15.195:6818/1606106342 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v113'533 uv533 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f58dc01c440 con 0x5588335e7d70 2024-09-21T03:42:06.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.014+0000 7f58ec03cf40 1 -- 172.21.15.38:0/1849620275 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:30 13.10 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x5588336021b0 con 0x5588335e7d70 2024-09-21T03:42:06.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.016+0000 7f58e52f7640 1 -- 172.21.15.38:0/1849620275 <== osd.7 v2:172.21.15.195:6818/1606106342 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v113'534 uv534 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f58dc01c440 con 0x5588335e7d70 2024-09-21T03:42:06.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.016+0000 7f58ec03cf40 1 -- 172.21.15.38:0/1849620275 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:31 13.10 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x5588336021b0 con 0x5588335e7d70 2024-09-21T03:42:06.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.017+0000 7f58e52f7640 1 -- 172.21.15.38:0/1849620275 <== osd.7 v2:172.21.15.195:6818/1606106342 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v113'535 uv535 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f58dc01c440 con 0x5588335e7d70 2024-09-21T03:42:06.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.017+0000 7f58ec03cf40 1 --2- 172.21.15.38:0/1849620275 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x558833602e00 0x558833605220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:06.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.017+0000 7f58ec03cf40 1 -- 172.21.15.38:0/1849620275 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x5588336026c0 con 0x558833602e00 2024-09-21T03:42:06.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.018+0000 7f58e5af8640 1 --2- 172.21.15.38:0/1849620275 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x558833602e00 0x558833605220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:06.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.018+0000 7f58e5af8640 1 --2- 172.21.15.38:0/1849620275 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x558833602e00 0x558833605220 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:06.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.020+0000 7f58e5af8640 1 -- 172.21.15.38:0/1849620275 <== osd.6 v2:172.21.15.195:6826/4055825675 1 ==== osd_op_reply(32 1.00000000.inode [create] v113'373 uv355 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f58d80564f0 con 0x558833602e00 2024-09-21T03:42:06.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.020+0000 7f58ec03cf40 1 -- 172.21.15.38:0/1849620275 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e113) -- 0x558833605760 con 0x558833602e00 2024-09-21T03:42:06.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.021+0000 7f58e5af8640 1 -- 172.21.15.38:0/1849620275 <== osd.6 v2:172.21.15.195:6826/4055825675 2 ==== osd_op_reply(33 1.00000000.inode [delete] v113'374 uv374 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f58d80564f0 con 0x558833602e00 2024-09-21T03:42:06.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.021+0000 7f58ec03cf40 1 -- 172.21.15.38:0/1849620275 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x558833605b00 con 0x558833602e00 2024-09-21T03:42:06.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.023+0000 7f58e5af8640 1 -- 172.21.15.38:0/1849620275 <== osd.6 v2:172.21.15.195:6826/4055825675 3 ==== osd_op_reply(34 1.00000000.inode [create] v113'375 uv375 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f58d80564f0 con 0x558833602e00 2024-09-21T03:42:06.025 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-09-21T03:42:06.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.023+0000 7f58ec03cf40 1 -- 172.21.15.38:0/1849620275 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x558833605ea0 con 0x558833602e00 2024-09-21T03:42:06.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.024+0000 7f58e5af8640 1 -- 172.21.15.38:0/1849620275 <== osd.6 v2:172.21.15.195:6826/4055825675 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v113'376 uv376 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f58d80564f0 con 0x558833602e00 2024-09-21T03:42:06.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.024+0000 7f58ec03cf40 1 -- 172.21.15.38:0/1849620275 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x558833606240 con 0x558833602e00 2024-09-21T03:42:06.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.025+0000 7f58e5af8640 1 -- 172.21.15.38:0/1849620275 <== osd.6 v2:172.21.15.195:6826/4055825675 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v113'377 uv377 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f58d80564f0 con 0x558833602e00 2024-09-21T03:42:06.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.025+0000 7f58ec03cf40 1 -- 172.21.15.38:0/1849620275 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x5588336065e0 con 0x558833602e00 2024-09-21T03:42:06.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.027+0000 7f58e5af8640 1 -- 172.21.15.38:0/1849620275 <== osd.6 v2:172.21.15.195:6826/4055825675 6 ==== osd_op_reply(37 200.00000000 [create] v113'378 uv372 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f58d80564f0 con 0x558833602e00 2024-09-21T03:42:06.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.027+0000 7f58ec03cf40 1 -- 172.21.15.38:0/1849620275 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e113) -- 0x558833606980 con 0x558833602e00 2024-09-21T03:42:06.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.028+0000 7f58e5af8640 1 -- 172.21.15.38:0/1849620275 <== osd.6 v2:172.21.15.195:6826/4055825675 7 ==== osd_op_reply(38 200.00000000 [delete] v113'379 uv379 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f58d80564f0 con 0x558833602e00 2024-09-21T03:42:06.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.028+0000 7f58ec03cf40 1 -- 172.21.15.38:0/1849620275 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x558833606980 con 0x558833602e00 2024-09-21T03:42:06.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.029+0000 7f58e5af8640 1 -- 172.21.15.38:0/1849620275 <== osd.6 v2:172.21.15.195:6826/4055825675 8 ==== osd_op_reply(39 200.00000000 [create] v113'380 uv380 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f58d80564f0 con 0x558833602e00 2024-09-21T03:42:06.031 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-09-21T03:42:06.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.029+0000 7f58ec03cf40 1 -- 172.21.15.38:0/1849620275 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x5588334b4b20 con 0x558833602e00 2024-09-21T03:42:06.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.031+0000 7f58e5af8640 1 -- 172.21.15.38:0/1849620275 <== osd.6 v2:172.21.15.195:6826/4055825675 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v113'381 uv381 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f58d80564f0 con 0x558833602e00 2024-09-21T03:42:06.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.031+0000 7f58ec03cf40 1 -- 172.21.15.38:0/1849620275 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x5588334b4b20 con 0x558833602e00 2024-09-21T03:42:06.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.032+0000 7f58e5af8640 1 -- 172.21.15.38:0/1849620275 <== osd.6 v2:172.21.15.195:6826/4055825675 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v113'382 uv382 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f58d80564f0 con 0x558833602e00 2024-09-21T03:42:06.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.032+0000 7f58ec03cf40 1 -- 172.21.15.38:0/1849620275 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:42 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x5588334b4b20 con 0x558833602e00 2024-09-21T03:42:06.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.033+0000 7f58e5af8640 1 -- 172.21.15.38:0/1849620275 <== osd.6 v2:172.21.15.195:6826/4055825675 11 ==== osd_op_reply(42 200.00000001 [create] v113'383 uv365 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f58d80564f0 con 0x558833602e00 2024-09-21T03:42:06.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.034+0000 7f58ec03cf40 1 -- 172.21.15.38:0/1849620275 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:43 13.14 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e113) -- 0x5588334b4b20 con 0x558833602e00 2024-09-21T03:42:06.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.035+0000 7f58e5af8640 1 -- 172.21.15.38:0/1849620275 <== osd.6 v2:172.21.15.195:6826/4055825675 12 ==== osd_op_reply(43 200.00000001 [delete] v113'384 uv384 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f58d80564f0 con 0x558833602e00 2024-09-21T03:42:06.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.035+0000 7f58ec03cf40 1 -- 172.21.15.38:0/1849620275 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:44 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x558833606e30 con 0x558833602e00 2024-09-21T03:42:06.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.036+0000 7f58e5af8640 1 -- 172.21.15.38:0/1849620275 <== osd.6 v2:172.21.15.195:6826/4055825675 13 ==== osd_op_reply(44 200.00000001 [create] v113'385 uv385 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f58d80564f0 con 0x558833602e00 2024-09-21T03:42:06.038 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-09-21T03:42:06.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.036+0000 7f58ec03cf40 1 -- 172.21.15.38:0/1849620275 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:45 13.14 13:2e2fa760:::200.00000001:head [write 0~17490 in=17490b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55883360b2b0 con 0x558833602e00 2024-09-21T03:42:06.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.038+0000 7f58e5af8640 1 -- 172.21.15.38:0/1849620275 <== osd.6 v2:172.21.15.195:6826/4055825675 14 ==== osd_op_reply(45 200.00000001 [write 0~17490] v113'386 uv386 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f58d80564f0 con 0x558833602e00 2024-09-21T03:42:06.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.038+0000 7f58ec03cf40 1 -- 172.21.15.38:0/1849620275 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:46 13.14 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55883360b2b0 con 0x558833602e00 2024-09-21T03:42:06.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.039+0000 7f58e5af8640 1 -- 172.21.15.38:0/1849620275 <== osd.6 v2:172.21.15.195:6826/4055825675 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v113'387 uv387 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f58d80564f0 con 0x558833602e00 2024-09-21T03:42:06.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.039+0000 7f58ec03cf40 1 -- 172.21.15.38:0/1849620275 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:47 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x558833606e30 con 0x558833602e00 2024-09-21T03:42:06.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.040+0000 7f58e5af8640 1 -- 172.21.15.38:0/1849620275 <== osd.6 v2:172.21.15.195:6826/4055825675 16 ==== osd_op_reply(47 606.00000000 [create] v113'388 uv371 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f58d80564f0 con 0x558833602e00 2024-09-21T03:42:06.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.040+0000 7f58ec03cf40 1 -- 172.21.15.38:0/1849620275 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:48 13.14 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e113) -- 0x558833606e30 con 0x558833602e00 2024-09-21T03:42:06.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.042+0000 7f58e5af8640 1 -- 172.21.15.38:0/1849620275 <== osd.6 v2:172.21.15.195:6826/4055825675 17 ==== osd_op_reply(48 606.00000000 [delete] v113'389 uv389 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f58d80564f0 con 0x558833602e00 2024-09-21T03:42:06.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.042+0000 7f58ec03cf40 1 -- 172.21.15.38:0/1849620275 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:49 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x558833606e30 con 0x558833602e00 2024-09-21T03:42:06.045 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-09-21T03:42:06.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.043+0000 7f58e5af8640 1 -- 172.21.15.38:0/1849620275 <== osd.6 v2:172.21.15.195:6826/4055825675 18 ==== osd_op_reply(49 606.00000000 [create] v113'390 uv390 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f58d80564f0 con 0x558833602e00 2024-09-21T03:42:06.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.043+0000 7f58ec03cf40 1 -- 172.21.15.38:0/1849620275 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:50 13.14 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x558833607e40 con 0x558833602e00 2024-09-21T03:42:06.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.044+0000 7f58e5af8640 1 -- 172.21.15.38:0/1849620275 <== osd.6 v2:172.21.15.195:6826/4055825675 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v113'391 uv391 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f58d80564f0 con 0x558833602e00 2024-09-21T03:42:06.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.044+0000 7f58ec03cf40 1 -- 172.21.15.38:0/1849620275 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:51 13.14 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x558833607e40 con 0x558833602e00 2024-09-21T03:42:06.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.046+0000 7f58e5af8640 1 -- 172.21.15.38:0/1849620275 <== osd.6 v2:172.21.15.195:6826/4055825675 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v113'392 uv392 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f58d80564f0 con 0x558833602e00 2024-09-21T03:42:06.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.046+0000 7f58ec03cf40 1 -- 172.21.15.38:0/1849620275 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:52 13.14 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x558833606e30 con 0x558833602e00 2024-09-21T03:42:06.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.047+0000 7f58e5af8640 1 -- 172.21.15.38:0/1849620275 <== osd.6 v2:172.21.15.195:6826/4055825675 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v113'393 uv393 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f58d80564f0 con 0x558833602e00 2024-09-21T03:42:06.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.047+0000 7f58ec03cf40 1 --2- 172.21.15.38:0/1849620275 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x558833607340 0x558833609760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:06.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.047+0000 7f58ec03cf40 1 -- 172.21.15.38:0/1849620275 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:53 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x558833609ca0 con 0x558833607340 2024-09-21T03:42:06.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.047+0000 7f58e52f7640 1 --2- 172.21.15.38:0/1849620275 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x558833607340 0x558833609760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:06.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.048+0000 7f58e52f7640 1 --2- 172.21.15.38:0/1849620275 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x558833607340 0x558833609760 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:06.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.049+0000 7f58e52f7640 1 -- 172.21.15.38:0/1849620275 <== osd.0 v2:172.21.15.38:6800/2381639433 1 ==== osd_op_reply(53 607.00000000 [create] v113'99 uv98 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f58dc01c440 con 0x558833607340 2024-09-21T03:42:06.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.049+0000 7f58ec03cf40 1 -- 172.21.15.38:0/1849620275 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:54 13.1c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e113) -- 0x558833609d60 con 0x558833607340 2024-09-21T03:42:06.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.050+0000 7f58e52f7640 1 -- 172.21.15.38:0/1849620275 <== osd.0 v2:172.21.15.38:6800/2381639433 2 ==== osd_op_reply(54 607.00000000 [delete] v113'100 uv100 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f58dc01c440 con 0x558833607340 2024-09-21T03:42:06.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.050+0000 7f58ec03cf40 1 -- 172.21.15.38:0/1849620275 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:55 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55883360a150 con 0x558833607340 2024-09-21T03:42:06.053 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-09-21T03:42:06.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.051+0000 7f58e52f7640 1 -- 172.21.15.38:0/1849620275 <== osd.0 v2:172.21.15.38:6800/2381639433 3 ==== osd_op_reply(55 607.00000000 [create] v113'101 uv101 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f58dc01c440 con 0x558833607340 2024-09-21T03:42:06.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.051+0000 7f58ec03cf40 1 -- 172.21.15.38:0/1849620275 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:56 13.1c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55883360b160 con 0x558833607340 2024-09-21T03:42:06.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.052+0000 7f58e52f7640 1 -- 172.21.15.38:0/1849620275 <== osd.0 v2:172.21.15.38:6800/2381639433 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v113'102 uv102 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f58dc01c440 con 0x558833607340 2024-09-21T03:42:06.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.052+0000 7f58ec03cf40 1 -- 172.21.15.38:0/1849620275 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:57 13.1c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55883360b160 con 0x558833607340 2024-09-21T03:42:06.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.054+0000 7f58e52f7640 1 -- 172.21.15.38:0/1849620275 <== osd.0 v2:172.21.15.38:6800/2381639433 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v113'103 uv103 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f58dc01c440 con 0x558833607340 2024-09-21T03:42:06.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.054+0000 7f58ec03cf40 1 -- 172.21.15.38:0/1849620275 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:58 13.1c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55883360a150 con 0x558833607340 2024-09-21T03:42:06.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.055+0000 7f58e52f7640 1 -- 172.21.15.38:0/1849620275 <== osd.0 v2:172.21.15.38:6800/2381639433 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v113'104 uv104 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f58dc01c440 con 0x558833607340 2024-09-21T03:42:06.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.055+0000 7f58ec03cf40 1 --2- 172.21.15.38:0/1849620275 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x55883360a7d0 0x55883360cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:06.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.055+0000 7f58ec03cf40 1 -- 172.21.15.38:0/1849620275 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:59 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55883360d290 con 0x55883360a7d0 2024-09-21T03:42:06.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.055+0000 7f58e62f9640 1 --2- 172.21.15.38:0/1849620275 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x55883360a7d0 0x55883360cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:06.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.055+0000 7f58e62f9640 1 --2- 172.21.15.38:0/1849620275 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x55883360a7d0 0x55883360cbf0 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:06.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.056+0000 7f58e62f9640 1 -- 172.21.15.38:0/1849620275 <== osd.3 v2:172.21.15.38:6808/1286810294 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v113'74 uv73 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f58e000b040 con 0x55883360a7d0 2024-09-21T03:42:06.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.056+0000 7f58ec03cf40 1 -- 172.21.15.38:0/1849620275 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:60 13.6 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e113) -- 0x55883360d7a0 con 0x55883360a7d0 2024-09-21T03:42:06.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.057+0000 7f58e62f9640 1 -- 172.21.15.38:0/1849620275 <== osd.3 v2:172.21.15.38:6808/1286810294 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v113'75 uv75 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f58e000b040 con 0x55883360a7d0 2024-09-21T03:42:06.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.057+0000 7f58ec03cf40 1 -- 172.21.15.38:0/1849620275 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:61 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55883360dca0 con 0x55883360a7d0 2024-09-21T03:42:06.060 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-09-21T03:42:06.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.058+0000 7f58e62f9640 1 -- 172.21.15.38:0/1849620275 <== osd.3 v2:172.21.15.38:6808/1286810294 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v113'76 uv76 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f58e000b040 con 0x55883360a7d0 2024-09-21T03:42:06.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.058+0000 7f58ec03cf40 1 -- 172.21.15.38:0/1849620275 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:62 13.6 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55883360e560 con 0x55883360a7d0 2024-09-21T03:42:06.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.060+0000 7f58e62f9640 1 -- 172.21.15.38:0/1849620275 <== osd.3 v2:172.21.15.38:6808/1286810294 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v113'77 uv77 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f58e000b040 con 0x55883360a7d0 2024-09-21T03:42:06.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.060+0000 7f58ec03cf40 1 -- 172.21.15.38:0/1849620275 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55883360e900 con 0x558833607340 2024-09-21T03:42:06.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.061+0000 7f58e52f7640 1 -- 172.21.15.38:0/1849620275 <== osd.0 v2:172.21.15.38:6800/2381639433 7 ==== osd_op_reply(63 608.00000000 [create] v113'99 uv98 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f58dc01c440 con 0x558833607340 2024-09-21T03:42:06.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.061+0000 7f58ec03cf40 1 -- 172.21.15.38:0/1849620275 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e113) -- 0x55883360e900 con 0x558833607340 2024-09-21T03:42:06.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.062+0000 7f58e52f7640 1 -- 172.21.15.38:0/1849620275 <== osd.0 v2:172.21.15.38:6800/2381639433 8 ==== osd_op_reply(64 608.00000000 [delete] v113'100 uv100 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f58dc01c440 con 0x558833607340 2024-09-21T03:42:06.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.062+0000 7f58ec03cf40 1 -- 172.21.15.38:0/1849620275 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55883360e900 con 0x558833607340 2024-09-21T03:42:06.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.063+0000 7f58e52f7640 1 -- 172.21.15.38:0/1849620275 <== osd.0 v2:172.21.15.38:6800/2381639433 9 ==== osd_op_reply(65 608.00000000 [create] v113'101 uv101 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f58dc01c440 con 0x558833607340 2024-09-21T03:42:06.065 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-09-21T03:42:06.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.063+0000 7f58ec03cf40 1 -- 172.21.15.38:0/1849620275 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55883360f910 con 0x558833607340 2024-09-21T03:42:06.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.064+0000 7f58e52f7640 1 -- 172.21.15.38:0/1849620275 <== osd.0 v2:172.21.15.38:6800/2381639433 10 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v113'102 uv102 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f58dc01c440 con 0x558833607340 2024-09-21T03:42:06.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.064+0000 7f58ec03cf40 1 -- 172.21.15.38:0/1849620275 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55883360f910 con 0x558833607340 2024-09-21T03:42:06.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.065+0000 7f58e52f7640 1 -- 172.21.15.38:0/1849620275 <== osd.0 v2:172.21.15.38:6800/2381639433 11 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v113'103 uv103 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f58dc01c440 con 0x558833607340 2024-09-21T03:42:06.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.065+0000 7f58ec03cf40 1 -- 172.21.15.38:0/1849620275 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55883360e900 con 0x558833607340 2024-09-21T03:42:06.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.066+0000 7f58e52f7640 1 -- 172.21.15.38:0/1849620275 <== osd.0 v2:172.21.15.38:6800/2381639433 12 ==== osd_op_reply(68 608.00000000 [omap-set-header] v113'104 uv104 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f58dc01c440 con 0x558833607340 2024-09-21T03:42:06.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.066+0000 7f58ec03cf40 1 -- 172.21.15.38:0/1849620275 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55883360e900 con 0x55883360a7d0 2024-09-21T03:42:06.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.068+0000 7f58e62f9640 1 -- 172.21.15.38:0/1849620275 <== osd.3 v2:172.21.15.38:6808/1286810294 5 ==== osd_op_reply(69 604.00000000 [create] v113'182 uv176 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f58e000b040 con 0x55883360a7d0 2024-09-21T03:42:06.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.068+0000 7f58ec03cf40 1 -- 172.21.15.38:0/1849620275 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e113) -- 0x55883360eca0 con 0x55883360a7d0 2024-09-21T03:42:06.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.069+0000 7f58e62f9640 1 -- 172.21.15.38:0/1849620275 <== osd.3 v2:172.21.15.38:6808/1286810294 6 ==== osd_op_reply(70 604.00000000 [delete] v113'183 uv183 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f58e000b040 con 0x55883360a7d0 2024-09-21T03:42:06.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.069+0000 7f58ec03cf40 1 -- 172.21.15.38:0/1849620275 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55883360f040 con 0x55883360a7d0 2024-09-21T03:42:06.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.070+0000 7f58e62f9640 1 -- 172.21.15.38:0/1849620275 <== osd.3 v2:172.21.15.38:6808/1286810294 7 ==== osd_op_reply(71 604.00000000 [create] v113'184 uv184 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f58e000b040 con 0x55883360a7d0 2024-09-21T03:42:06.072 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-09-21T03:42:06.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.070+0000 7f58ec03cf40 1 -- 172.21.15.38:0/1849620275 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x5588336103f0 con 0x55883360a7d0 2024-09-21T03:42:06.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.071+0000 7f58e62f9640 1 -- 172.21.15.38:0/1849620275 <== osd.3 v2:172.21.15.38:6808/1286810294 8 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v113'185 uv185 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f58e000b040 con 0x55883360a7d0 2024-09-21T03:42:06.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.071+0000 7f58ec03cf40 1 -- 172.21.15.38:0/1849620275 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x558833610430 con 0x55883360a7d0 2024-09-21T03:42:06.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.072+0000 7f58e62f9640 1 -- 172.21.15.38:0/1849620275 <== osd.3 v2:172.21.15.38:6808/1286810294 9 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v113'186 uv186 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f58e000b040 con 0x55883360a7d0 2024-09-21T03:42:06.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.072+0000 7f58ec03cf40 1 -- 172.21.15.38:0/1849620275 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55883360f3e0 con 0x55883360a7d0 2024-09-21T03:42:06.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.073+0000 7f58e62f9640 1 -- 172.21.15.38:0/1849620275 <== osd.3 v2:172.21.15.38:6808/1286810294 10 ==== osd_op_reply(74 604.00000000 [omap-set-header] v113'187 uv187 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f58e000b040 con 0x55883360a7d0 2024-09-21T03:42:06.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.073+0000 7f58ec03cf40 1 -- 172.21.15.38:0/1849620275 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55883360f3e0 con 0x55883360a7d0 2024-09-21T03:42:06.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.074+0000 7f58e62f9640 1 -- 172.21.15.38:0/1849620275 <== osd.3 v2:172.21.15.38:6808/1286810294 11 ==== osd_op_reply(75 500.00000000 [create] v113'188 uv181 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f58e000b040 con 0x55883360a7d0 2024-09-21T03:42:06.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.074+0000 7f58ec03cf40 1 -- 172.21.15.38:0/1849620275 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e113) -- 0x55883360f3e0 con 0x55883360a7d0 2024-09-21T03:42:06.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.075+0000 7f58e62f9640 1 -- 172.21.15.38:0/1849620275 <== osd.3 v2:172.21.15.38:6808/1286810294 12 ==== osd_op_reply(76 500.00000000 [delete] v113'189 uv189 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f58e000b040 con 0x55883360a7d0 2024-09-21T03:42:06.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.075+0000 7f58ec03cf40 1 -- 172.21.15.38:0/1849620275 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55883360f3e0 con 0x55883360a7d0 2024-09-21T03:42:06.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.076+0000 7f58e62f9640 1 -- 172.21.15.38:0/1849620275 <== osd.3 v2:172.21.15.38:6808/1286810294 13 ==== osd_op_reply(77 500.00000000 [create] v113'190 uv190 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f58e000b040 con 0x55883360a7d0 2024-09-21T03:42:06.078 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-09-21T03:42:06.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.076+0000 7f58ec03cf40 1 -- 172.21.15.38:0/1849620275 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55883360f3e0 con 0x55883360a7d0 2024-09-21T03:42:06.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.077+0000 7f58e62f9640 1 -- 172.21.15.38:0/1849620275 <== osd.3 v2:172.21.15.38:6808/1286810294 14 ==== osd_op_reply(78 500.00000000 [write 0~90] v113'191 uv191 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f58e000b040 con 0x55883360a7d0 2024-09-21T03:42:06.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.078+0000 7f58ec03cf40 1 -- 172.21.15.38:0/1849620275 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55883360f3e0 con 0x55883360a7d0 2024-09-21T03:42:06.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.078+0000 7f58e62f9640 1 -- 172.21.15.38:0/1849620275 <== osd.3 v2:172.21.15.38:6808/1286810294 15 ==== osd_op_reply(79 500.00000000 [omap-set-header] v113'192 uv192 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f58e000b040 con 0x55883360a7d0 2024-09-21T03:42:06.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.079+0000 7f58ec03cf40 1 -- 172.21.15.38:0/1849620275 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:80 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55883360f3e0 con 0x558833607340 2024-09-21T03:42:06.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.080+0000 7f58e52f7640 1 -- 172.21.15.38:0/1849620275 <== osd.0 v2:172.21.15.38:6800/2381639433 13 ==== osd_op_reply(80 mds_snaptable [create] v113'81 uv80 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f58dc01c440 con 0x558833607340 2024-09-21T03:42:06.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.080+0000 7f58ec03cf40 1 -- 172.21.15.38:0/1849620275 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:81 13.d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e113) -- 0x55883360f3e0 con 0x558833607340 2024-09-21T03:42:06.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.081+0000 7f58e52f7640 1 -- 172.21.15.38:0/1849620275 <== osd.0 v2:172.21.15.38:6800/2381639433 14 ==== osd_op_reply(81 mds_snaptable [delete] v113'82 uv82 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f58dc01c440 con 0x558833607340 2024-09-21T03:42:06.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.081+0000 7f58ec03cf40 1 -- 172.21.15.38:0/1849620275 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:82 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55883360f3e0 con 0x558833607340 2024-09-21T03:42:06.084 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-09-21T03:42:06.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.082+0000 7f58e52f7640 1 -- 172.21.15.38:0/1849620275 <== osd.0 v2:172.21.15.38:6800/2381639433 15 ==== osd_op_reply(82 mds_snaptable [create] v113'83 uv83 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f58dc01c440 con 0x558833607340 2024-09-21T03:42:06.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.082+0000 7f58ec03cf40 1 -- 172.21.15.38:0/1849620275 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:83 13.d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55883360f3e0 con 0x558833607340 2024-09-21T03:42:06.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.083+0000 7f58e52f7640 1 -- 172.21.15.38:0/1849620275 <== osd.0 v2:172.21.15.38:6800/2381639433 16 ==== osd_op_reply(83 mds_snaptable [write 0~70] v113'84 uv84 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f58dc01c440 con 0x558833607340 2024-09-21T03:42:06.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.083+0000 7f58ec03cf40 1 -- 172.21.15.38:0/1849620275 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:84 13.d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55883360f3e0 con 0x558833607340 2024-09-21T03:42:06.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.084+0000 7f58e52f7640 1 -- 172.21.15.38:0/1849620275 <== osd.0 v2:172.21.15.38:6800/2381639433 17 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v113'85 uv85 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f58dc01c440 con 0x558833607340 2024-09-21T03:42:06.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.084+0000 7f58ec03cf40 1 --2- 172.21.15.38:0/1849620275 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x55883360fa60 0x558833611e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:06.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.084+0000 7f58ec03cf40 1 -- 172.21.15.38:0/1849620275 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x5588336124e0 con 0x55883360fa60 2024-09-21T03:42:06.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.085+0000 7f58e5af8640 1 --2- 172.21.15.38:0/1849620275 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x55883360fa60 0x558833611e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:06.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.085+0000 7f58e5af8640 1 --2- 172.21.15.38:0/1849620275 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x55883360fa60 0x558833611e80 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:06.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.086+0000 7f58e5af8640 1 -- 172.21.15.38:0/1849620275 <== osd.1 v2:172.21.15.38:6816/1491818732 1 ==== osd_op_reply(85 605.00000000 [create] v113'99 uv98 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f58d80564f0 con 0x55883360fa60 2024-09-21T03:42:06.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.086+0000 7f58ec03cf40 1 -- 172.21.15.38:0/1849620275 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e113) -- 0x5588336128b0 con 0x55883360fa60 2024-09-21T03:42:06.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.087+0000 7f58e5af8640 1 -- 172.21.15.38:0/1849620275 <== osd.1 v2:172.21.15.38:6816/1491818732 2 ==== osd_op_reply(86 605.00000000 [delete] v113'100 uv100 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f58d80564f0 con 0x55883360fa60 2024-09-21T03:42:06.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.087+0000 7f58ec03cf40 1 -- 172.21.15.38:0/1849620275 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x5588336128b0 con 0x55883360fa60 2024-09-21T03:42:06.090 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-09-21T03:42:06.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.088+0000 7f58e5af8640 1 -- 172.21.15.38:0/1849620275 <== osd.1 v2:172.21.15.38:6816/1491818732 3 ==== osd_op_reply(87 605.00000000 [create] v113'101 uv101 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f58d80564f0 con 0x55883360fa60 2024-09-21T03:42:06.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.088+0000 7f58ec03cf40 1 -- 172.21.15.38:0/1849620275 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x5588336138c0 con 0x55883360fa60 2024-09-21T03:42:06.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.090+0000 7f58e5af8640 1 -- 172.21.15.38:0/1849620275 <== osd.1 v2:172.21.15.38:6816/1491818732 4 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v113'102 uv102 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f58d80564f0 con 0x55883360fa60 2024-09-21T03:42:06.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.090+0000 7f58ec03cf40 1 -- 172.21.15.38:0/1849620275 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x5588336138c0 con 0x55883360fa60 2024-09-21T03:42:06.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.091+0000 7f58e5af8640 1 -- 172.21.15.38:0/1849620275 <== osd.1 v2:172.21.15.38:6816/1491818732 5 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v113'103 uv103 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f58d80564f0 con 0x55883360fa60 2024-09-21T03:42:06.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.091+0000 7f58ec03cf40 1 -- 172.21.15.38:0/1849620275 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x5588336128b0 con 0x55883360fa60 2024-09-21T03:42:06.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.092+0000 7f58e5af8640 1 -- 172.21.15.38:0/1849620275 <== osd.1 v2:172.21.15.38:6816/1491818732 6 ==== osd_op_reply(90 605.00000000 [omap-set-header] v113'104 uv104 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f58d80564f0 con 0x55883360fa60 2024-09-21T03:42:06.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.092+0000 7f58ec03cf40 1 -- 172.21.15.38:0/1849620275 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x5588336128b0 con 0x55883360fa60 2024-09-21T03:42:06.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.093+0000 7f58e5af8640 1 -- 172.21.15.38:0/1849620275 <== osd.1 v2:172.21.15.38:6816/1491818732 7 ==== osd_op_reply(91 mds0_inotable [create] v113'162 uv156 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f58d80564f0 con 0x55883360fa60 2024-09-21T03:42:06.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.093+0000 7f58ec03cf40 1 -- 172.21.15.38:0/1849620275 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e113) -- 0x5588336128b0 con 0x55883360fa60 2024-09-21T03:42:06.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.094+0000 7f58e5af8640 1 -- 172.21.15.38:0/1849620275 <== osd.1 v2:172.21.15.38:6816/1491818732 8 ==== osd_op_reply(92 mds0_inotable [delete] v113'163 uv163 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f58d80564f0 con 0x55883360fa60 2024-09-21T03:42:06.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.094+0000 7f58ec03cf40 1 -- 172.21.15.38:0/1849620275 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x5588336128b0 con 0x55883360fa60 2024-09-21T03:42:06.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.095+0000 7f58e5af8640 1 -- 172.21.15.38:0/1849620275 <== osd.1 v2:172.21.15.38:6816/1491818732 9 ==== osd_op_reply(93 mds0_inotable [create] v113'164 uv164 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f58d80564f0 con 0x55883360fa60 2024-09-21T03:42:06.097 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-09-21T03:42:06.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.095+0000 7f58ec03cf40 1 -- 172.21.15.38:0/1849620275 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:94 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x558833612950 con 0x55883360fa60 2024-09-21T03:42:06.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.097+0000 7f58e5af8640 1 -- 172.21.15.38:0/1849620275 <== osd.1 v2:172.21.15.38:6816/1491818732 10 ==== osd_op_reply(94 mds0_inotable [write 0~34] v113'165 uv165 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f58d80564f0 con 0x55883360fa60 2024-09-21T03:42:06.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.097+0000 7f58ec03cf40 1 -- 172.21.15.38:0/1849620275 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:95 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x558833612950 con 0x55883360fa60 2024-09-21T03:42:06.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.098+0000 7f58e5af8640 1 -- 172.21.15.38:0/1849620275 <== osd.1 v2:172.21.15.38:6816/1491818732 11 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v113'166 uv166 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f58d80564f0 con 0x55883360fa60 2024-09-21T03:42:06.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.098+0000 7f58ec03cf40 1 -- 172.21.15.38:0/1849620275 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:96 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x558833612950 con 0x55883360fa60 2024-09-21T03:42:06.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.099+0000 7f58e5af8640 1 -- 172.21.15.38:0/1849620275 <== osd.1 v2:172.21.15.38:6816/1491818732 12 ==== osd_op_reply(96 100.00000000 [create] v113'167 uv161 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f58d80564f0 con 0x55883360fa60 2024-09-21T03:42:06.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.099+0000 7f58ec03cf40 1 -- 172.21.15.38:0/1849620275 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:97 13.13 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e113) -- 0x558833612950 con 0x55883360fa60 2024-09-21T03:42:06.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.100+0000 7f58e5af8640 1 -- 172.21.15.38:0/1849620275 <== osd.1 v2:172.21.15.38:6816/1491818732 13 ==== osd_op_reply(97 100.00000000 [delete] v113'168 uv168 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f58d80564f0 con 0x55883360fa60 2024-09-21T03:42:06.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.100+0000 7f58ec03cf40 1 -- 172.21.15.38:0/1849620275 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:98 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x558833612950 con 0x55883360fa60 2024-09-21T03:42:06.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.101+0000 7f58e5af8640 1 -- 172.21.15.38:0/1849620275 <== osd.1 v2:172.21.15.38:6816/1491818732 14 ==== osd_op_reply(98 100.00000000 [create] v113'169 uv169 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f58d80564f0 con 0x55883360fa60 2024-09-21T03:42:06.103 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-09-21T03:42:06.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.101+0000 7f58ec03cf40 1 -- 172.21.15.38:0/1849620275 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:99 13.13 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x558833612950 con 0x55883360fa60 2024-09-21T03:42:06.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.102+0000 7f58e5af8640 1 -- 172.21.15.38:0/1849620275 <== osd.1 v2:172.21.15.38:6816/1491818732 15 ==== osd_op_reply(99 100.00000000 [omap-set-header] v113'170 uv170 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f58d80564f0 con 0x55883360fa60 2024-09-21T03:42:06.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.102+0000 7f58ec03cf40 1 -- 172.21.15.38:0/1849620275 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:100 13.13 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55883360e1a0 con 0x55883360fa60 2024-09-21T03:42:06.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.103+0000 7f58e5af8640 1 -- 172.21.15.38:0/1849620275 <== osd.1 v2:172.21.15.38:6816/1491818732 16 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v113'171 uv171 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f58d80564f0 con 0x55883360fa60 2024-09-21T03:42:06.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.103+0000 7f58ec03cf40 1 -- 172.21.15.38:0/1849620275 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55883360e160 con 0x55883360a7d0 2024-09-21T03:42:06.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.104+0000 7f58e62f9640 1 -- 172.21.15.38:0/1849620275 <== osd.3 v2:172.21.15.38:6808/1286810294 16 ==== osd_op_reply(101 mds0_sessionmap [create] v113'74 uv73 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7f58e000b040 con 0x55883360a7d0 2024-09-21T03:42:06.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.104+0000 7f58ec03cf40 1 -- 172.21.15.38:0/1849620275 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e113) -- 0x55883360e160 con 0x55883360a7d0 2024-09-21T03:42:06.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.105+0000 7f58e62f9640 1 -- 172.21.15.38:0/1849620275 <== osd.3 v2:172.21.15.38:6808/1286810294 17 ==== osd_op_reply(102 mds0_sessionmap [delete] v113'75 uv75 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f58e000b040 con 0x55883360a7d0 2024-09-21T03:42:06.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.105+0000 7f58ec03cf40 1 -- 172.21.15.38:0/1849620275 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:103 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55883360e160 con 0x55883360a7d0 2024-09-21T03:42:06.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.106+0000 7f58e62f9640 1 -- 172.21.15.38:0/1849620275 <== osd.3 v2:172.21.15.38:6808/1286810294 18 ==== osd_op_reply(103 mds0_sessionmap [create] v113'76 uv76 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f58e000b040 con 0x55883360a7d0 2024-09-21T03:42:06.108 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-09-21T03:42:06.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.106+0000 7f58ec03cf40 1 -- 172.21.15.38:0/1849620275 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:104 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55883360e160 con 0x55883360a7d0 2024-09-21T03:42:06.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.107+0000 7f58e62f9640 1 -- 172.21.15.38:0/1849620275 <== osd.3 v2:172.21.15.38:6808/1286810294 19 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v113'77 uv77 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f58e000b040 con 0x55883360a7d0 2024-09-21T03:42:06.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.107+0000 7f58ec03cf40 1 -- 172.21.15.38:0/1849620275 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55883360e140 con 0x55883360a7d0 2024-09-21T03:42:06.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.108+0000 7f58e62f9640 1 -- 172.21.15.38:0/1849620275 <== osd.3 v2:172.21.15.38:6808/1286810294 20 ==== osd_op_reply(105 609.00000000 [create] v113'99 uv98 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f58e000b040 con 0x55883360a7d0 2024-09-21T03:42:06.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.108+0000 7f58ec03cf40 1 -- 172.21.15.38:0/1849620275 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e113) -- 0x55883360e140 con 0x55883360a7d0 2024-09-21T03:42:06.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.109+0000 7f58e62f9640 1 -- 172.21.15.38:0/1849620275 <== osd.3 v2:172.21.15.38:6808/1286810294 21 ==== osd_op_reply(106 609.00000000 [delete] v113'100 uv100 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f58e000b040 con 0x55883360a7d0 2024-09-21T03:42:06.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.109+0000 7f58ec03cf40 1 -- 172.21.15.38:0/1849620275 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55883360e140 con 0x55883360a7d0 2024-09-21T03:42:06.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.110+0000 7f58e62f9640 1 -- 172.21.15.38:0/1849620275 <== osd.3 v2:172.21.15.38:6808/1286810294 22 ==== osd_op_reply(107 609.00000000 [create] v113'101 uv101 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f58e000b040 con 0x55883360a7d0 2024-09-21T03:42:06.112 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-09-21T03:42:06.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.110+0000 7f58ec03cf40 1 -- 172.21.15.38:0/1849620275 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55883360e120 con 0x55883360a7d0 2024-09-21T03:42:06.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.111+0000 7f58e62f9640 1 -- 172.21.15.38:0/1849620275 <== osd.3 v2:172.21.15.38:6808/1286810294 23 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v113'102 uv102 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f58e000b040 con 0x55883360a7d0 2024-09-21T03:42:06.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.111+0000 7f58ec03cf40 1 -- 172.21.15.38:0/1849620275 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:109 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55883360e120 con 0x55883360a7d0 2024-09-21T03:42:06.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.112+0000 7f58e62f9640 1 -- 172.21.15.38:0/1849620275 <== osd.3 v2:172.21.15.38:6808/1286810294 24 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v113'103 uv103 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f58e000b040 con 0x55883360a7d0 2024-09-21T03:42:06.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.112+0000 7f58ec03cf40 1 -- 172.21.15.38:0/1849620275 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:110 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55883360e120 con 0x55883360a7d0 2024-09-21T03:42:06.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.113+0000 7f58e62f9640 1 -- 172.21.15.38:0/1849620275 <== osd.3 v2:172.21.15.38:6808/1286810294 25 ==== osd_op_reply(110 609.00000000 [omap-set-header] v113'104 uv104 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f58e000b040 con 0x55883360a7d0 2024-09-21T03:42:06.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.113+0000 7f58ec03cf40 1 --2- 172.21.15.38:0/1849620275 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x558833613660 0x558833615e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:06.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.113+0000 7f58ec03cf40 1 -- 172.21.15.38:0/1849620275 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x558833616360 con 0x558833613660 2024-09-21T03:42:06.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.113+0000 7f58e52f7640 1 --2- 172.21.15.38:0/1849620275 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x558833613660 0x558833615e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:06.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.113+0000 7f58e52f7640 1 --2- 172.21.15.38:0/1849620275 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x558833613660 0x558833615e20 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:06.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.115+0000 7f58e52f7640 1 -- 172.21.15.38:0/1849620275 <== osd.2 v2:172.21.15.38:6824/2577858902 1 ==== osd_op_reply(111 400.00000000 [create] v113'84 uv83 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f58dc033040 con 0x558833613660 2024-09-21T03:42:06.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.115+0000 7f58ec03cf40 1 -- 172.21.15.38:0/1849620275 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e113) -- 0x558833616360 con 0x558833613660 2024-09-21T03:42:06.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.116+0000 7f58e52f7640 1 -- 172.21.15.38:0/1849620275 <== osd.2 v2:172.21.15.38:6824/2577858902 2 ==== osd_op_reply(112 400.00000000 [delete] v113'85 uv85 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f58dc033040 con 0x558833613660 2024-09-21T03:42:06.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.116+0000 7f58ec03cf40 1 -- 172.21.15.38:0/1849620275 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x558833616360 con 0x558833613660 2024-09-21T03:42:06.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.117+0000 7f58e52f7640 1 -- 172.21.15.38:0/1849620275 <== osd.2 v2:172.21.15.38:6824/2577858902 3 ==== osd_op_reply(113 400.00000000 [create] v113'86 uv86 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f58dc033040 con 0x558833613660 2024-09-21T03:42:06.119 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-09-21T03:42:06.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.117+0000 7f58ec03cf40 1 -- 172.21.15.38:0/1849620275 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:114 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x558833616360 con 0x558833613660 2024-09-21T03:42:06.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.118+0000 7f58e52f7640 1 -- 172.21.15.38:0/1849620275 <== osd.2 v2:172.21.15.38:6824/2577858902 4 ==== osd_op_reply(114 400.00000000 [write 0~22] v113'87 uv87 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f58dc033040 con 0x558833613660 2024-09-21T03:42:06.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.118+0000 7f58ec03cf40 1 -- 172.21.15.38:0/1849620275 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:115 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x558833616360 con 0x558833613660 2024-09-21T03:42:06.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.119+0000 7f58e52f7640 1 -- 172.21.15.38:0/1849620275 <== osd.2 v2:172.21.15.38:6824/2577858902 5 ==== osd_op_reply(115 400.00000000 [omap-set-header] v113'88 uv88 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f58dc033040 con 0x558833613660 2024-09-21T03:42:06.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.119+0000 7f58ec03cf40 1 -- 172.21.15.38:0/1849620275 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:116 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x558833616360 con 0x55883360fa60 2024-09-21T03:42:06.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.120+0000 7f58e5af8640 1 -- 172.21.15.38:0/1849620275 <== osd.1 v2:172.21.15.38:6816/1491818732 17 ==== osd_op_reply(116 100.00000000.inode [create] v113'204 uv196 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7f58d80564f0 con 0x55883360fa60 2024-09-21T03:42:06.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.120+0000 7f58ec03cf40 1 -- 172.21.15.38:0/1849620275 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:117 13.1f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e113) -- 0x558833616360 con 0x55883360fa60 2024-09-21T03:42:06.123 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.121+0000 7f58e5af8640 1 -- 172.21.15.38:0/1849620275 <== osd.1 v2:172.21.15.38:6816/1491818732 18 ==== osd_op_reply(117 100.00000000.inode [delete] v113'205 uv205 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f58d80564f0 con 0x55883360fa60 2024-09-21T03:42:06.123 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.121+0000 7f58ec03cf40 1 -- 172.21.15.38:0/1849620275 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:118 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x558833616360 con 0x55883360fa60 2024-09-21T03:42:06.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.122+0000 7f58e5af8640 1 -- 172.21.15.38:0/1849620275 <== osd.1 v2:172.21.15.38:6816/1491818732 19 ==== osd_op_reply(118 100.00000000.inode [create] v113'206 uv206 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f58d80564f0 con 0x55883360fa60 2024-09-21T03:42:06.124 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-09-21T03:42:06.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.122+0000 7f58ec03cf40 1 -- 172.21.15.38:0/1849620275 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:119 13.1f 13:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x558833616360 con 0x55883360fa60 2024-09-21T03:42:06.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.124+0000 7f58e5af8640 1 -- 172.21.15.38:0/1849620275 <== osd.1 v2:172.21.15.38:6816/1491818732 20 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v113'207 uv207 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f58d80564f0 con 0x55883360fa60 2024-09-21T03:42:06.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.124+0000 7f58ec03cf40 1 -- 172.21.15.38:0/1849620275 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:120 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x558833616360 con 0x55883360fa60 2024-09-21T03:42:06.126 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.125+0000 7f58e5af8640 1 -- 172.21.15.38:0/1849620275 <== osd.1 v2:172.21.15.38:6816/1491818732 21 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v113'208 uv208 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f58d80564f0 con 0x55883360fa60 2024-09-21T03:42:06.126 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.125+0000 7f58ec03cf40 1 -- 172.21.15.38:0/1849620275 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:121 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x558833616360 con 0x55883360fa60 2024-09-21T03:42:06.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.125+0000 7f58e5af8640 1 -- 172.21.15.38:0/1849620275 <== osd.1 v2:172.21.15.38:6816/1491818732 22 ==== osd_op_reply(121 1.00000000 [create] v113'209 uv203 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7f58d80564f0 con 0x55883360fa60 2024-09-21T03:42:06.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.126+0000 7f58ec03cf40 1 -- 172.21.15.38:0/1849620275 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:122 13.1f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e113) -- 0x558833616360 con 0x55883360fa60 2024-09-21T03:42:06.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.127+0000 7f58e5af8640 1 -- 172.21.15.38:0/1849620275 <== osd.1 v2:172.21.15.38:6816/1491818732 23 ==== osd_op_reply(122 1.00000000 [delete] v113'210 uv210 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f58d80564f0 con 0x55883360fa60 2024-09-21T03:42:06.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.127+0000 7f58ec03cf40 1 -- 172.21.15.38:0/1849620275 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:123 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x558833616360 con 0x55883360fa60 2024-09-21T03:42:06.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.128+0000 7f58e5af8640 1 -- 172.21.15.38:0/1849620275 <== osd.1 v2:172.21.15.38:6816/1491818732 24 ==== osd_op_reply(123 1.00000000 [create] v113'211 uv211 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f58d80564f0 con 0x55883360fa60 2024-09-21T03:42:06.130 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-09-21T03:42:06.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.128+0000 7f58ec03cf40 1 -- 172.21.15.38:0/1849620275 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:124 13.1f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x558833617370 con 0x55883360fa60 2024-09-21T03:42:06.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.129+0000 7f58e5af8640 1 -- 172.21.15.38:0/1849620275 <== osd.1 v2:172.21.15.38:6816/1491818732 25 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v113'212 uv212 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f58d80564f0 con 0x55883360fa60 2024-09-21T03:42:06.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.129+0000 7f58ec03cf40 1 -- 172.21.15.38:0/1849620275 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:125 13.1f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x558833617370 con 0x55883360fa60 2024-09-21T03:42:06.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.130+0000 7f58e5af8640 1 -- 172.21.15.38:0/1849620275 <== osd.1 v2:172.21.15.38:6816/1491818732 26 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v113'213 uv213 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f58d80564f0 con 0x55883360fa60 2024-09-21T03:42:06.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.130+0000 7f58ec03cf40 1 -- 172.21.15.38:0/1849620275 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:126 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x558833616360 con 0x55883360fa60 2024-09-21T03:42:06.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.131+0000 7f58e5af8640 1 -- 172.21.15.38:0/1849620275 <== osd.1 v2:172.21.15.38:6816/1491818732 27 ==== osd_op_reply(126 1.00000000 [omap-set-header] v113'214 uv214 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f58d80564f0 con 0x55883360fa60 2024-09-21T03:42:06.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.131+0000 7f58ec03cf40 1 -- 172.21.15.38:0/1849620275 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:127 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=584b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x558833616810 con 0x55883360fa60 2024-09-21T03:42:06.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.132+0000 7f58e5af8640 1 -- 172.21.15.38:0/1849620275 <== osd.1 v2:172.21.15.38:6816/1491818732 28 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v113'215 uv215 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f58d80564f0 con 0x55883360fa60 2024-09-21T03:42:06.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.132+0000 7f58ec03cf40 1 -- 172.21.15.38:0/1849620275 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x558833607340 msgr2=0x558833609760 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:06.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.132+0000 7f58ec03cf40 1 --2- 172.21.15.38:0/1849620275 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x558833607340 0x558833609760 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:06.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.132+0000 7f58ec03cf40 1 -- 172.21.15.38:0/1849620275 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x55883360fa60 msgr2=0x558833611e80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:06.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.132+0000 7f58ec03cf40 1 --2- 172.21.15.38:0/1849620275 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x55883360fa60 0x558833611e80 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:06.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.132+0000 7f58ec03cf40 1 -- 172.21.15.38:0/1849620275 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x558833613660 msgr2=0x558833615e20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:06.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.132+0000 7f58ec03cf40 1 --2- 172.21.15.38:0/1849620275 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x558833613660 0x558833615e20 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:06.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.132+0000 7f58ec03cf40 1 -- 172.21.15.38:0/1849620275 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x55883360a7d0 msgr2=0x55883360cbf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:06.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.132+0000 7f58ec03cf40 1 --2- 172.21.15.38:0/1849620275 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x55883360a7d0 0x55883360cbf0 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:06.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.132+0000 7f58e62f9640 1 -- 172.21.15.38:0/1849620275 reap_dead start 2024-09-21T03:42:06.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.132+0000 7f58ec03cf40 1 -- 172.21.15.38:0/1849620275 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x558833602e00 msgr2=0x558833605220 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:06.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.132+0000 7f58ec03cf40 1 --2- 172.21.15.38:0/1849620275 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x558833602e00 0x558833605220 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:06.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.132+0000 7f58e62f9640 1 -- 172.21.15.38:0/1849620275 reap_dead start 2024-09-21T03:42:06.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.132+0000 7f58ec03cf40 1 -- 172.21.15.38:0/1849620275 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x5588335e7d70 msgr2=0x558833600dd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:06.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.132+0000 7f58ec03cf40 1 --2- 172.21.15.38:0/1849620275 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x5588335e7d70 0x558833600dd0 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:06.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.132+0000 7f58ec03cf40 1 -- 172.21.15.38:0/1849620275 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f58c80775a0 msgr2=0x7f58c8079ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:06.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.132+0000 7f58ec03cf40 1 --2- 172.21.15.38:0/1849620275 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f58c80775a0 0x7f58c8079ab0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f58d8055000 tx=0x7f58d8058040 comp rx=0 tx=0).stop 2024-09-21T03:42:06.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.132+0000 7f58ec03cf40 1 -- 172.21.15.38:0/1849620275 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558833544520 msgr2=0x55883353ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:06.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.132+0000 7f58ec03cf40 1 --2- 172.21.15.38:0/1849620275 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558833544520 0x55883353ce50 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7f58e000f680 tx=0x7f58e000df90 comp rx=0 tx=0).stop 2024-09-21T03:42:06.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.132+0000 7f58ec03cf40 1 -- 172.21.15.38:0/1849620275 shutdown_connections 2024-09-21T03:42:06.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.132+0000 7f58ec03cf40 1 --2- 172.21.15.38:0/1849620275 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x5588335e7d70 0x558833600dd0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:06.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.132+0000 7f58ec03cf40 1 --2- 172.21.15.38:0/1849620275 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558833544520 0x55883353ce50 unknown :-1 s=CLOSED pgs=1640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:06.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.132+0000 7f58ec03cf40 1 --2- 172.21.15.38:0/1849620275 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f58c80775a0 0x7f58c8079ab0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:06.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.132+0000 7f58ec03cf40 1 -- 172.21.15.38:0/1849620275 >> 172.21.15.38:0/1849620275 conn(0x5588333a6ac0 msgr2=0x5588334cd7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:06.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.132+0000 7f58ec03cf40 1 -- 172.21.15.38:0/1849620275 shutdown_connections 2024-09-21T03:42:06.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.132+0000 7f58ec03cf40 1 -- 172.21.15.38:0/1849620275 wait complete. 2024-09-21T03:42:06.137 DEBUG:teuthology.orchestra.run.smithi038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata put 200.00000001 - 2024-09-21T03:42:06.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.192+0000 7fb805cf3f40 1 Processor -- start 2024-09-21T03:42:06.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.192+0000 7fb805cf3f40 1 -- start start 2024-09-21T03:42:06.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.193+0000 7fb805cf3f40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fc529ecd00 0x55fc52ac3180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:06.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.193+0000 7fb805cf3f40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55fc52a18e30 0x55fc52a1aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:06.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.193+0000 7fb805cf3f40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fc52aa7950 0x55fc52ac2350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:06.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.193+0000 7fb805cf3f40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55fc52999430 con 0x55fc52aa7950 2024-09-21T03:42:06.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.193+0000 7fb805cf3f40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55fc529a61f0 con 0x55fc529ecd00 2024-09-21T03:42:06.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.193+0000 7fb805cf3f40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55fc5293ad60 con 0x55fc52a18e30 2024-09-21T03:42:06.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.193+0000 7fb7fffae640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fc52aa7950 0x55fc52ac2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:06.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.193+0000 7fb7fffae640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fc52aa7950 0x55fc52ac2350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49440/0 (socket says 172.21.15.38:49440) 2024-09-21T03:42:06.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.193+0000 7fb7fffae640 1 -- 172.21.15.38:0/2640773102 learned_addr learned my addr 172.21.15.38:0/2640773102 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:42:06.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.193+0000 7fb7ff7ad640 1 --2- 172.21.15.38:0/2640773102 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fc529ecd00 0x55fc52ac3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:06.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.193+0000 7fb7fffae640 1 -- 172.21.15.38:0/2640773102 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55fc52a18e30 msgr2=0x55fc52a1aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:06.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.193+0000 7fb7f7fff640 1 --2- 172.21.15.38:0/2640773102 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55fc52a18e30 0x55fc52a1aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:06.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.193+0000 7fb7fffae640 1 --2- 172.21.15.38:0/2640773102 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55fc52a18e30 0x55fc52a1aca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:06.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.193+0000 7fb7fffae640 1 -- 172.21.15.38:0/2640773102 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fc529ecd00 msgr2=0x55fc52ac3180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:06.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.193+0000 7fb7fffae640 1 --2- 172.21.15.38:0/2640773102 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fc529ecd00 0x55fc52ac3180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:06.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.193+0000 7fb7fffae640 1 -- 172.21.15.38:0/2640773102 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc52a14fb0 con 0x55fc52aa7950 2024-09-21T03:42:06.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.194+0000 7fb7fffae640 1 --2- 172.21.15.38:0/2640773102 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fc52aa7950 0x55fc52ac2350 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7fb7f80020e0 tx=0x7fb7f8006820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e95afdc3842c2683 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:06.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.194+0000 7fb7fe7ab640 1 -- 172.21.15.38:0/2640773102 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7f8016070 con 0x55fc52aa7950 2024-09-21T03:42:06.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.194+0000 7fb7fe7ab640 1 -- 172.21.15.38:0/2640773102 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb7f800f800 con 0x55fc52aa7950 2024-09-21T03:42:06.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.194+0000 7fb7fe7ab640 1 -- 172.21.15.38:0/2640773102 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7f800fc50 con 0x55fc52aa7950 2024-09-21T03:42:06.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.194+0000 7fb805cf3f40 1 -- 172.21.15.38:0/2640773102 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fc52aa7950 msgr2=0x55fc52ac2350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:06.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.194+0000 7fb805cf3f40 1 --2- 172.21.15.38:0/2640773102 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fc52aa7950 0x55fc52ac2350 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7fb7f80020e0 tx=0x7fb7f8006820 comp rx=0 tx=0).stop 2024-09-21T03:42:06.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.194+0000 7fb805cf3f40 1 -- 172.21.15.38:0/2640773102 shutdown_connections 2024-09-21T03:42:06.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.194+0000 7fb805cf3f40 1 --2- 172.21.15.38:0/2640773102 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fc52aa7950 0x55fc52ac2350 unknown :-1 s=CLOSED pgs=1642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:06.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.194+0000 7fb805cf3f40 1 --2- 172.21.15.38:0/2640773102 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55fc52a18e30 0x55fc52a1aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:06.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.194+0000 7fb805cf3f40 1 --2- 172.21.15.38:0/2640773102 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fc529ecd00 0x55fc52ac3180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:06.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.194+0000 7fb805cf3f40 1 -- 172.21.15.38:0/2640773102 >> 172.21.15.38:0/2640773102 conn(0x55fc5290bac0 msgr2=0x55fc52abe750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:06.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.194+0000 7fb805cf3f40 1 -- 172.21.15.38:0/2640773102 shutdown_connections 2024-09-21T03:42:06.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.194+0000 7fb805cf3f40 1 -- 172.21.15.38:0/2640773102 wait complete. 2024-09-21T03:42:06.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.195+0000 7fb805cf3f40 1 Processor -- start 2024-09-21T03:42:06.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.195+0000 7fb805cf3f40 1 -- start start 2024-09-21T03:42:06.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.197+0000 7fb805cf3f40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fc529ecd00 0x55fc52ab91b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:06.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.197+0000 7fb805cf3f40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55fc52a18e30 0x55fc52ac2350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:06.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.197+0000 7fb805cf3f40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fc52aa7950 0x55fc52ac2890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:06.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.197+0000 7fb805cf3f40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55fc52aa6c00 con 0x55fc52aa7950 2024-09-21T03:42:06.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.197+0000 7fb805cf3f40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55fc52ac13a0 con 0x55fc529ecd00 2024-09-21T03:42:06.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.197+0000 7fb805cf3f40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55fc52ac1680 con 0x55fc52a18e30 2024-09-21T03:42:06.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.197+0000 7fb7fffae640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fc52aa7950 0x55fc52ac2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:06.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.197+0000 7fb7fffae640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fc52aa7950 0x55fc52ac2890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49444/0 (socket says 172.21.15.38:49444) 2024-09-21T03:42:06.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.197+0000 7fb7fffae640 1 -- 172.21.15.38:0/2296683739 learned_addr learned my addr 172.21.15.38:0/2296683739 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:42:06.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.197+0000 7fb7f7fff640 1 --2- 172.21.15.38:0/2296683739 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55fc52a18e30 0x55fc52ac2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:06.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.197+0000 7fb7ff7ad640 1 --2- 172.21.15.38:0/2296683739 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fc529ecd00 0x55fc52ab91b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:06.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.197+0000 7fb7fffae640 1 -- 172.21.15.38:0/2296683739 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55fc52a18e30 msgr2=0x55fc52ac2350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:06.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.197+0000 7fb7fffae640 1 --2- 172.21.15.38:0/2296683739 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55fc52a18e30 0x55fc52ac2350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:06.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.197+0000 7fb7fffae640 1 -- 172.21.15.38:0/2296683739 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fc529ecd00 msgr2=0x55fc52ab91b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:06.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.197+0000 7fb7fffae640 1 --2- 172.21.15.38:0/2296683739 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fc529ecd00 0x55fc52ab91b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:06.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.197+0000 7fb7fffae640 1 -- 172.21.15.38:0/2296683739 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc52abda00 con 0x55fc52aa7950 2024-09-21T03:42:06.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.197+0000 7fb7ff7ad640 1 --2- 172.21.15.38:0/2296683739 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fc529ecd00 0x55fc52ab91b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:42:06.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.197+0000 7fb7fffae640 1 --2- 172.21.15.38:0/2296683739 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fc52aa7950 0x55fc52ac2890 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7fb7f8001330 tx=0x7fb7f8011670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:06.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.197+0000 7fb7fd7a9640 1 -- 172.21.15.38:0/2296683739 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7f8020030 con 0x55fc52aa7950 2024-09-21T03:42:06.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.198+0000 7fb7fd7a9640 1 -- 172.21.15.38:0/2296683739 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb7f800f510 con 0x55fc52aa7950 2024-09-21T03:42:06.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.198+0000 7fb805cf3f40 1 -- 172.21.15.38:0/2296683739 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55fc52aa1270 con 0x55fc52aa7950 2024-09-21T03:42:06.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.198+0000 7fb7fd7a9640 1 -- 172.21.15.38:0/2296683739 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7f8016040 con 0x55fc52aa7950 2024-09-21T03:42:06.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.198+0000 7fb805cf3f40 1 -- 172.21.15.38:0/2296683739 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc52ab96f0 con 0x55fc52aa7950 2024-09-21T03:42:06.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.199+0000 7fb7fd7a9640 1 -- 172.21.15.38:0/2296683739 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb7f8023030 con 0x55fc52aa7950 2024-09-21T03:42:06.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.200+0000 7fb7fd7a9640 1 --2- 172.21.15.38:0/2296683739 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb7e0077570 0x7fb7e0079a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:06.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.200+0000 7fb7fd7a9640 1 -- 172.21.15.38:0/2296683739 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 7766+0+0 (secure 0 0 0) 0x7fb7f809afc0 con 0x55fc52aa7950 2024-09-21T03:42:06.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.200+0000 7fb7ff7ad640 1 --2- 172.21.15.38:0/2296683739 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb7e0077570 0x7fb7e0079a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:06.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.200+0000 7fb805cf3f40 1 --2- 172.21.15.38:0/2296683739 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x55fc52a1ba50 0x55fc52b501d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:06.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.200+0000 7fb805cf3f40 1 -- 172.21.15.38:0/2296683739 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:1 13.14 13:2e2fa760:::200.00000001:head [writefull 0~80 in=80b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55fc52abd3e0 con 0x55fc52a1ba50 2024-09-21T03:42:06.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.200+0000 7fb7f7fff640 1 --2- 172.21.15.38:0/2296683739 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x55fc52a1ba50 0x55fc52b501d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:06.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.200+0000 7fb7ff7ad640 1 --2- 172.21.15.38:0/2296683739 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb7e0077570 0x7fb7e0079a80 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fb7f0000c00 tx=0x7fb7f00029a0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:06.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.201+0000 7fb7f7fff640 1 --2- 172.21.15.38:0/2296683739 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x55fc52a1ba50 0x55fc52b501d0 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:06.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.202+0000 7fb7f7fff640 1 -- 172.21.15.38:0/2296683739 <== osd.6 v2:172.21.15.195:6826/4055825675 1 ==== osd_op_reply(1 200.00000001 [writefull 0~80] v113'394 uv394 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb7e804e940 con 0x55fc52a1ba50 2024-09-21T03:42:06.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.202+0000 7fb805cf3f40 1 -- 172.21.15.38:0/2296683739 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x55fc52a1ba50 msgr2=0x55fc52b501d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:06.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.202+0000 7fb805cf3f40 1 --2- 172.21.15.38:0/2296683739 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x55fc52a1ba50 0x55fc52b501d0 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:06.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.202+0000 7fb805cf3f40 1 -- 172.21.15.38:0/2296683739 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb7e0077570 msgr2=0x7fb7e0079a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:06.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.202+0000 7fb805cf3f40 1 --2- 172.21.15.38:0/2296683739 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb7e0077570 0x7fb7e0079a80 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fb7f0000c00 tx=0x7fb7f00029a0 comp rx=0 tx=0).stop 2024-09-21T03:42:06.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.202+0000 7fb805cf3f40 1 -- 172.21.15.38:0/2296683739 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fc52aa7950 msgr2=0x55fc52ac2890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:06.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.202+0000 7fb805cf3f40 1 --2- 172.21.15.38:0/2296683739 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fc52aa7950 0x55fc52ac2890 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7fb7f8001330 tx=0x7fb7f8011670 comp rx=0 tx=0).stop 2024-09-21T03:42:06.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.202+0000 7fb7fffae640 1 -- 172.21.15.38:0/2296683739 reap_dead start 2024-09-21T03:42:06.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.202+0000 7fb805cf3f40 1 -- 172.21.15.38:0/2296683739 shutdown_connections 2024-09-21T03:42:06.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.203+0000 7fb805cf3f40 1 -- 172.21.15.38:0/2296683739 >> 172.21.15.38:0/2296683739 conn(0x55fc5290bac0 msgr2=0x55fc52a1aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:06.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.203+0000 7fb805cf3f40 1 -- 172.21.15.38:0/2296683739 shutdown_connections 2024-09-21T03:42:06.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.203+0000 7fb805cf3f40 1 -- 172.21.15.38:0/2296683739 wait complete. 2024-09-21T03:42:06.208 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-21T03:42:06.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.365+0000 7fcf901a1640 1 -- 172.21.15.38:0/2733960759 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcf880d0170 msgr2=0x7fcf880d0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:06.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.365+0000 7fcf901a1640 1 --2- 172.21.15.38:0/2733960759 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcf880d0170 0x7fcf880d0be0 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7fcf78009820 tx=0x7fcf7802a480 comp rx=0 tx=0).stop 2024-09-21T03:42:06.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.365+0000 7fcf901a1640 1 -- 172.21.15.38:0/2733960759 shutdown_connections 2024-09-21T03:42:06.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.365+0000 7fcf901a1640 1 --2- 172.21.15.38:0/2733960759 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcf880d0170 0x7fcf880d0be0 unknown :-1 s=CLOSED pgs=1645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:06.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.365+0000 7fcf901a1640 1 -- 172.21.15.38:0/2733960759 >> 172.21.15.38:0/2733960759 conn(0x7fcf8809d830 msgr2=0x7fcf880b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:06.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.365+0000 7fcf901a1640 1 -- 172.21.15.38:0/2733960759 shutdown_connections 2024-09-21T03:42:06.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.365+0000 7fcf901a1640 1 -- 172.21.15.38:0/2733960759 wait complete. 2024-09-21T03:42:06.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.367+0000 7fcf901a1640 1 Processor -- start 2024-09-21T03:42:06.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.367+0000 7fcf901a1640 1 -- start start 2024-09-21T03:42:06.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.368+0000 7fcf901a1640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcf880d0170 0x7fcf88150aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:06.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.368+0000 7fcf901a1640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcf880d1120 0x7fcf8806e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:06.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.368+0000 7fcf901a1640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcf8815ee80 0x7fcf880cc240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:06.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.368+0000 7fcf901a1640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fcf880b6910 con 0x7fcf880d1120 2024-09-21T03:42:06.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.368+0000 7fcf901a1640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fcf880b6790 con 0x7fcf8815ee80 2024-09-21T03:42:06.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.368+0000 7fcf901a1640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fcf880b6a90 con 0x7fcf880d0170 2024-09-21T03:42:06.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.368+0000 7fcf8d73c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcf880d1120 0x7fcf8806e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:06.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.368+0000 7fcf8d73c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcf880d1120 0x7fcf8806e560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49462/0 (socket says 172.21.15.38:49462) 2024-09-21T03:42:06.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.368+0000 7fcf8d73c640 1 -- 172.21.15.38:0/178428913 learned_addr learned my addr 172.21.15.38:0/178428913 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:42:06.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.368+0000 7fcf8df3d640 1 --2- 172.21.15.38:0/178428913 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcf880d0170 0x7fcf88150aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:06.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.368+0000 7fcf8e73e640 1 --2- 172.21.15.38:0/178428913 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcf8815ee80 0x7fcf880cc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:06.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.369+0000 7fcf8d73c640 1 -- 172.21.15.38:0/178428913 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcf880d0170 msgr2=0x7fcf88150aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:06.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.369+0000 7fcf8d73c640 1 --2- 172.21.15.38:0/178428913 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcf880d0170 0x7fcf88150aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:06.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.369+0000 7fcf8d73c640 1 -- 172.21.15.38:0/178428913 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcf8815ee80 msgr2=0x7fcf880cc240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:06.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.369+0000 7fcf8d73c640 1 --2- 172.21.15.38:0/178428913 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcf8815ee80 0x7fcf880cc240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:06.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.369+0000 7fcf8d73c640 1 -- 172.21.15.38:0/178428913 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf8806d4c0 con 0x7fcf880d1120 2024-09-21T03:42:06.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.369+0000 7fcf8d73c640 1 --2- 172.21.15.38:0/178428913 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcf880d1120 0x7fcf8806e560 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7fcf78009940 tx=0x7fcf78005360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:06.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.369+0000 7fcf7e7fc640 1 -- 172.21.15.38:0/178428913 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf7803f070 con 0x7fcf880d1120 2024-09-21T03:42:06.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.369+0000 7fcf7e7fc640 1 -- 172.21.15.38:0/178428913 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcf78004340 con 0x7fcf880d1120 2024-09-21T03:42:06.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.369+0000 7fcf7e7fc640 1 -- 172.21.15.38:0/178428913 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf78031bb0 con 0x7fcf880d1120 2024-09-21T03:42:06.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.369+0000 7fcf901a1640 1 -- 172.21.15.38:0/178428913 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf8806d7a0 con 0x7fcf880d1120 2024-09-21T03:42:06.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.369+0000 7fcf901a1640 1 -- 172.21.15.38:0/178428913 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf8806dd60 con 0x7fcf880d1120 2024-09-21T03:42:06.374 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.370+0000 7fcf901a1640 1 -- 172.21.15.38:0/178428913 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf880d0be0 con 0x7fcf880d1120 2024-09-21T03:42:06.374 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.371+0000 7fcf7e7fc640 1 -- 172.21.15.38:0/178428913 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fcf78042070 con 0x7fcf880d1120 2024-09-21T03:42:06.374 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.372+0000 7fcf7e7fc640 1 --2- 172.21.15.38:0/178428913 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fcf6c078bc0 0x7fcf6c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:06.374 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.372+0000 7fcf7e7fc640 1 -- 172.21.15.38:0/178428913 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 7766+0+0 (secure 0 0 0) 0x7fcf780bc4a0 con 0x7fcf880d1120 2024-09-21T03:42:06.374 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.372+0000 7fcf8df3d640 1 --2- 172.21.15.38:0/178428913 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fcf6c078bc0 0x7fcf6c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:06.374 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.373+0000 7fcf8df3d640 1 --2- 172.21.15.38:0/178428913 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fcf6c078bc0 0x7fcf6c07b0d0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fcf84000f60 tx=0x7fcf84009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:06.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.377+0000 7fcf7e7fc640 1 -- 172.21.15.38:0/178428913 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fcf78083b10 con 0x7fcf880d1120 2024-09-21T03:42:06.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.604+0000 7fcf901a1640 1 -- 172.21.15.38:0/178428913 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7fcf8815db30 con 0x7fcf880d1120 2024-09-21T03:42:06.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.886+0000 7fcf7e7fc640 1 -- 172.21.15.38:0/178428913 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v188) ==== 162+0+0 (secure 0 0 0) 0x7fcf78088540 con 0x7fcf880d1120 2024-09-21T03:42:06.888 INFO:teuthology.orchestra.run.smithi038.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-21T03:42:06.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.889+0000 7fcf901a1640 1 -- 172.21.15.38:0/178428913 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fcf6c078bc0 msgr2=0x7fcf6c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:06.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.889+0000 7fcf901a1640 1 --2- 172.21.15.38:0/178428913 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fcf6c078bc0 0x7fcf6c07b0d0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fcf84000f60 tx=0x7fcf84009040 comp rx=0 tx=0).stop 2024-09-21T03:42:06.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.889+0000 7fcf901a1640 1 -- 172.21.15.38:0/178428913 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcf880d1120 msgr2=0x7fcf8806e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:06.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.889+0000 7fcf901a1640 1 --2- 172.21.15.38:0/178428913 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcf880d1120 0x7fcf8806e560 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7fcf78009940 tx=0x7fcf78005360 comp rx=0 tx=0).stop 2024-09-21T03:42:06.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.889+0000 7fcf901a1640 1 -- 172.21.15.38:0/178428913 shutdown_connections 2024-09-21T03:42:06.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.889+0000 7fcf901a1640 1 --2- 172.21.15.38:0/178428913 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcf8815ee80 0x7fcf880cc240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:06.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.889+0000 7fcf901a1640 1 --2- 172.21.15.38:0/178428913 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcf880d1120 0x7fcf8806e560 unknown :-1 s=CLOSED pgs=1646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:06.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.889+0000 7fcf901a1640 1 --2- 172.21.15.38:0/178428913 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fcf6c078bc0 0x7fcf6c07b0d0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:06.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.889+0000 7fcf901a1640 1 --2- 172.21.15.38:0/178428913 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcf880d0170 0x7fcf88150aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:06.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.889+0000 7fcf901a1640 1 -- 172.21.15.38:0/178428913 >> 172.21.15.38:0/178428913 conn(0x7fcf8809d830 msgr2=0x7fcf88160670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:06.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.889+0000 7fcf901a1640 1 -- 172.21.15.38:0/178428913 shutdown_connections 2024-09-21T03:42:06.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:06.889+0000 7fcf901a1640 1 -- 172.21.15.38:0/178428913 wait complete. 2024-09-21T03:42:06.901 INFO:tasks.ceph.mds.c.smithi038.stderr:2024-09-21T03:42:06.900+0000 7ffab9f77640 -1 log_channel(cluster) log [ERR] : missing journal object 2024-09-21T03:42:06.902 INFO:tasks.ceph.mon.a.smithi038.stderr:2024-09-21T03:42:06.900+0000 7f3f45713640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T03:42:06.902 INFO:tasks.ceph.mon.a.smithi038.stderr:2024-09-21T03:42:06.900+0000 7f3f45713640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-09-21T03:42:06.905 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:42:07.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:07.020+0000 7f4cf796e640 1 -- 172.21.15.38:0/2972168857 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4cf00b2e60 msgr2=0x7f4cf00b3270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:07.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:07.020+0000 7f4cf796e640 1 --2- 172.21.15.38:0/2972168857 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4cf00b2e60 0x7f4cf00b3270 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f4ce8005fd0 tx=0x7f4ce802a8a0 comp rx=0 tx=0).stop 2024-09-21T03:42:07.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:07.021+0000 7f4cf796e640 1 -- 172.21.15.38:0/2972168857 shutdown_connections 2024-09-21T03:42:07.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:07.021+0000 7f4cf796e640 1 --2- 172.21.15.38:0/2972168857 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4cf00b2e60 0x7f4cf00b3270 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:07.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:07.021+0000 7f4cf796e640 1 --2- 172.21.15.38:0/2972168857 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4cf0162590 0x7f4cf01629a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:07.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:07.021+0000 7f4cf796e640 1 -- 172.21.15.38:0/2972168857 >> 172.21.15.38:0/2972168857 conn(0x7f4cf00d4440 msgr2=0x7f4cf00d4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:07.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:07.021+0000 7f4cf796e640 1 -- 172.21.15.38:0/2972168857 shutdown_connections 2024-09-21T03:42:07.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:07.021+0000 7f4cf796e640 1 -- 172.21.15.38:0/2972168857 wait complete. 2024-09-21T03:42:07.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:07.022+0000 7f4cf796e640 1 Processor -- start 2024-09-21T03:42:07.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:07.022+0000 7f4cf796e640 1 -- start start 2024-09-21T03:42:07.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:07.024+0000 7f4cf796e640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4cf00b2e60 0x7f4cf01442b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:07.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:07.024+0000 7f4cf796e640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4cf00cfe00 0x7f4cf0153e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:07.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:07.024+0000 7f4cf796e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4cf0162590 0x7f4cf01f59d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:07.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:07.024+0000 7f4cf796e640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f4cf00b3930 con 0x7f4cf0162590 2024-09-21T03:42:07.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:07.024+0000 7f4cf796e640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f4cf00b37b0 con 0x7f4cf00cfe00 2024-09-21T03:42:07.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:07.024+0000 7f4cf796e640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f4cf00b3ab0 con 0x7f4cf00b2e60 2024-09-21T03:42:07.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:07.024+0000 7f4cf5f0b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4cf0162590 0x7f4cf01f59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:07.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:07.024+0000 7f4cf4f09640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4cf00cfe00 0x7f4cf0153e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:07.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:07.024+0000 7f4cf4f09640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4cf00cfe00 0x7f4cf0153e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.38:56048/0 (socket says 172.21.15.38:56048) 2024-09-21T03:42:07.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:07.024+0000 7f4cf4f09640 1 -- 172.21.15.38:0/3842078790 learned_addr learned my addr 172.21.15.38:0/3842078790 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:42:07.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:07.024+0000 7f4cf570a640 1 --2- 172.21.15.38:0/3842078790 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4cf00b2e60 0x7f4cf01442b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:07.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:07.024+0000 7f4cf5f0b640 1 -- 172.21.15.38:0/3842078790 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4cf00b2e60 msgr2=0x7f4cf01442b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:07.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:07.024+0000 7f4cf5f0b640 1 --2- 172.21.15.38:0/3842078790 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4cf00b2e60 0x7f4cf01442b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:07.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:07.024+0000 7f4cf5f0b640 1 -- 172.21.15.38:0/3842078790 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4cf00cfe00 msgr2=0x7f4cf0153e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:07.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:07.024+0000 7f4cf5f0b640 1 --2- 172.21.15.38:0/3842078790 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4cf00cfe00 0x7f4cf0153e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:07.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:07.024+0000 7f4cf5f0b640 1 -- 172.21.15.38:0/3842078790 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cf0159870 con 0x7f4cf0162590 2024-09-21T03:42:07.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:07.024+0000 7f4cf4f09640 1 --2- 172.21.15.38:0/3842078790 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4cf00cfe00 0x7f4cf0153e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:42:07.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:07.025+0000 7f4cf5f0b640 1 --2- 172.21.15.38:0/3842078790 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4cf0162590 0x7f4cf01f59d0 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7f4ce8009000 tx=0x7f4ce8032db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:07.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:07.025+0000 7f4ce5ffb640 1 -- 172.21.15.38:0/3842078790 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ce8041070 con 0x7f4cf0162590 2024-09-21T03:42:07.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:07.025+0000 7f4cf796e640 1 -- 172.21.15.38:0/3842078790 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cf01447f0 con 0x7f4cf0162590 2024-09-21T03:42:07.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:07.025+0000 7f4ce5ffb640 1 -- 172.21.15.38:0/3842078790 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4ce8004450 con 0x7f4cf0162590 2024-09-21T03:42:07.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:07.025+0000 7f4ce5ffb640 1 -- 172.21.15.38:0/3842078790 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ce8031430 con 0x7f4cf0162590 2024-09-21T03:42:07.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:07.025+0000 7f4cf796e640 1 -- 172.21.15.38:0/3842078790 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cf01565b0 con 0x7f4cf0162590 2024-09-21T03:42:07.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:07.026+0000 7f4cf796e640 1 -- 172.21.15.38:0/3842078790 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4cf00b3270 con 0x7f4cf0162590 2024-09-21T03:42:07.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:07.027+0000 7f4ce5ffb640 1 -- 172.21.15.38:0/3842078790 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f4ce8044070 con 0x7f4cf0162590 2024-09-21T03:42:07.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:07.027+0000 7f4ce5ffb640 1 --2- 172.21.15.38:0/3842078790 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4cd4078b90 0x7f4cd407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:07.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:07.027+0000 7f4ce5ffb640 1 -- 172.21.15.38:0/3842078790 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 7852+0+0 (secure 0 0 0) 0x7f4ce80bc410 con 0x7f4cf0162590 2024-09-21T03:42:07.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:07.028+0000 7f4cf570a640 1 --2- 172.21.15.38:0/3842078790 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4cd4078b90 0x7f4cd407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:07.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:07.028+0000 7f4cf570a640 1 --2- 172.21.15.38:0/3842078790 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4cd4078b90 0x7f4cd407b0a0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f4cec007d60 tx=0x7f4cec007cf0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:07.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:07.032+0000 7f4ce5ffb640 1 -- 172.21.15.38:0/3842078790 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f4ce8083880 con 0x7f4cf0162590 2024-09-21T03:42:07.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:07.258+0000 7f4cf796e640 1 -- 172.21.15.38:0/3842078790 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4cf00b4e50 con 0x7f4cf0162590 2024-09-21T03:42:07.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:07.258+0000 7f4ce5ffb640 1 -- 172.21.15.38:0/3842078790 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 190 v190) ==== 95+0+3566 (secure 0 0 0) 0x7f4ce80882b0 con 0x7f4cf0162590 2024-09-21T03:42:07.261 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:42:07.261 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":190,"btime":"2024-09-21T03:42:06:901463+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6086,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/965972663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":965972663},{"type":"v1","addr":"172.21.15.195:6836","nonce":965972663}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6700,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/3176122334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":3176122334},{"type":"v1","addr":"172.21.15.195:6837","nonce":3176122334}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":190,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:40:02.319282+0000","modified":"2024-09-21T03:42:06.901398+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":114,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-21T03:42:07.261 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 190 2024-09-21T03:42:07.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:07.261+0000 7f4cf796e640 1 -- 172.21.15.38:0/3842078790 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4cd4078b90 msgr2=0x7f4cd407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:07.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:07.261+0000 7f4cf796e640 1 --2- 172.21.15.38:0/3842078790 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4cd4078b90 0x7f4cd407b0a0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f4cec007d60 tx=0x7f4cec007cf0 comp rx=0 tx=0).stop 2024-09-21T03:42:07.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:07.261+0000 7f4cf796e640 1 -- 172.21.15.38:0/3842078790 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4cf0162590 msgr2=0x7f4cf01f59d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:07.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:07.261+0000 7f4cf796e640 1 --2- 172.21.15.38:0/3842078790 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4cf0162590 0x7f4cf01f59d0 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7f4ce8009000 tx=0x7f4ce8032db0 comp rx=0 tx=0).stop 2024-09-21T03:42:07.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:07.261+0000 7f4cf796e640 1 -- 172.21.15.38:0/3842078790 shutdown_connections 2024-09-21T03:42:07.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:07.261+0000 7f4cf796e640 1 --2- 172.21.15.38:0/3842078790 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4cf0162590 0x7f4cf01f59d0 unknown :-1 s=CLOSED pgs=1647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:07.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:07.261+0000 7f4cf796e640 1 --2- 172.21.15.38:0/3842078790 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4cf00cfe00 0x7f4cf0153e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:07.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:07.261+0000 7f4cf796e640 1 --2- 172.21.15.38:0/3842078790 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4cd4078b90 0x7f4cd407b0a0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:07.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:07.261+0000 7f4cf796e640 1 --2- 172.21.15.38:0/3842078790 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4cf00b2e60 0x7f4cf01442b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:07.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:07.261+0000 7f4cf796e640 1 -- 172.21.15.38:0/3842078790 >> 172.21.15.38:0/3842078790 conn(0x7f4cf00d4440 msgr2=0x7f4cf0158cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:07.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:07.261+0000 7f4cf796e640 1 -- 172.21.15.38:0/3842078790 shutdown_connections 2024-09-21T03:42:07.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:07.261+0000 7f4cf796e640 1 -- 172.21.15.38:0/3842078790 wait complete. 2024-09-21T03:42:07.275 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-21T03:42:07.275 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:42:07.391 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:07.389+0000 7f4433969640 1 -- 172.21.15.38:0/3200369895 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f442c0cfe00 msgr2=0x7f442c0d2a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:07.391 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:07.389+0000 7f4433969640 1 --2- 172.21.15.38:0/3200369895 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f442c0cfe00 0x7f442c0d2a20 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f442c0b5a40 tx=0x7f442802a530 comp rx=0 tx=0).stop 2024-09-21T03:42:07.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:07.389+0000 7f4433969640 1 -- 172.21.15.38:0/3200369895 shutdown_connections 2024-09-21T03:42:07.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:07.389+0000 7f4433969640 1 --2- 172.21.15.38:0/3200369895 >> v2:172.21.15.195:3300/0 conn(0x7f442c0b2e60 0x7f442c0b3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:07.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:07.389+0000 7f4433969640 1 --2- 172.21.15.38:0/3200369895 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f442c0cfe00 0x7f442c0d2a20 unknown :-1 s=CLOSED pgs=1649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:07.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:07.389+0000 7f4433969640 1 -- 172.21.15.38:0/3200369895 >> 172.21.15.38:0/3200369895 conn(0x7f442c0d4440 msgr2=0x7f442c0d4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:07.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:07.389+0000 7f4433969640 1 -- 172.21.15.38:0/3200369895 shutdown_connections 2024-09-21T03:42:07.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:07.389+0000 7f4433969640 1 -- 172.21.15.38:0/3200369895 wait complete. 2024-09-21T03:42:07.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:07.391+0000 7f4433969640 1 Processor -- start 2024-09-21T03:42:07.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:07.391+0000 7f4433969640 1 -- start start 2024-09-21T03:42:07.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:07.392+0000 7f4433969640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f442c0b2e60 0x7f442c15a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:07.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:07.392+0000 7f4433969640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f442c0cfe00 0x7f442c15db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:07.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:07.392+0000 7f4433969640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f442c162590 0x7f442c146740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:07.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:07.392+0000 7f4433969640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f442c0b3930 con 0x7f442c0cfe00 2024-09-21T03:42:07.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:07.392+0000 7f4433969640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f442c0b37b0 con 0x7f442c162590 2024-09-21T03:42:07.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:07.392+0000 7f4433969640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f442c0b3ab0 con 0x7f442c0b2e60 2024-09-21T03:42:07.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:07.393+0000 7f4430f04640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f442c0cfe00 0x7f442c15db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:07.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:07.393+0000 7f4430f04640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f442c0cfe00 0x7f442c15db40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49494/0 (socket says 172.21.15.38:49494) 2024-09-21T03:42:07.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:07.393+0000 7f4430f04640 1 -- 172.21.15.38:0/2836138339 learned_addr learned my addr 172.21.15.38:0/2836138339 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:42:07.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:07.393+0000 7f4431f06640 1 --2- 172.21.15.38:0/2836138339 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f442c162590 0x7f442c146740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:07.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:07.393+0000 7f4431705640 1 --2- 172.21.15.38:0/2836138339 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f442c0b2e60 0x7f442c15a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:07.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:07.393+0000 7f4430f04640 1 -- 172.21.15.38:0/2836138339 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f442c0b2e60 msgr2=0x7f442c15a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:07.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:07.393+0000 7f4430f04640 1 --2- 172.21.15.38:0/2836138339 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f442c0b2e60 0x7f442c15a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:07.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:07.393+0000 7f4430f04640 1 -- 172.21.15.38:0/2836138339 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f442c162590 msgr2=0x7f442c146740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:07.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:07.393+0000 7f4430f04640 1 --2- 172.21.15.38:0/2836138339 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f442c162590 0x7f442c146740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:07.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:07.393+0000 7f4430f04640 1 -- 172.21.15.38:0/2836138339 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f442c159670 con 0x7f442c0cfe00 2024-09-21T03:42:07.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:07.393+0000 7f4430f04640 1 --2- 172.21.15.38:0/2836138339 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f442c0cfe00 0x7f442c15db40 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7f441c00c400 tx=0x7f441c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:07.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:07.393+0000 7f4421ffb640 1 -- 172.21.15.38:0/2836138339 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f441c011790 con 0x7f442c0cfe00 2024-09-21T03:42:07.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:07.393+0000 7f4421ffb640 1 -- 172.21.15.38:0/2836138339 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f441c017070 con 0x7f442c0cfe00 2024-09-21T03:42:07.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:07.393+0000 7f4421ffb640 1 -- 172.21.15.38:0/2836138339 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f441c011ce0 con 0x7f442c0cfe00 2024-09-21T03:42:07.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:07.393+0000 7f4433969640 1 -- 172.21.15.38:0/2836138339 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f442c155b00 con 0x7f442c0cfe00 2024-09-21T03:42:07.398 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:07.394+0000 7f4433969640 1 -- 172.21.15.38:0/2836138339 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f442c1598e0 con 0x7f442c0cfe00 2024-09-21T03:42:07.398 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:07.394+0000 7f4433969640 1 -- 172.21.15.38:0/2836138339 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f442c0b3270 con 0x7f442c0cfe00 2024-09-21T03:42:07.398 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:07.395+0000 7f4421ffb640 1 -- 172.21.15.38:0/2836138339 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f441c01c460 con 0x7f442c0cfe00 2024-09-21T03:42:07.399 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:07.396+0000 7f4421ffb640 1 --2- 172.21.15.38:0/2836138339 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4410078b90 0x7f441007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:07.399 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:07.396+0000 7f4421ffb640 1 -- 172.21.15.38:0/2836138339 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 7852+0+0 (secure 0 0 0) 0x7f441c09b3a0 con 0x7f442c0cfe00 2024-09-21T03:42:07.399 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:07.396+0000 7f4431705640 1 --2- 172.21.15.38:0/2836138339 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4410078b90 0x7f441007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:07.399 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:07.397+0000 7f4431705640 1 --2- 172.21.15.38:0/2836138339 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4410078b90 0x7f441007b0a0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f4428034000 tx=0x7f4428004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:07.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:07.400+0000 7f4421ffb640 1 -- 172.21.15.38:0/2836138339 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f442c155b00 con 0x7f442c0cfe00 2024-09-21T03:42:07.621 INFO:tasks.ceph.mds.a.smithi038.stderr: -41> 2024-09-21T03:42:02.868+0000 7fd1e32e2640 -1 log_channel(cluster) log [ERR] : Error recovering journal 0x200: (22) Invalid argument 2024-09-21T03:42:07.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:07.626+0000 7f4433969640 1 -- 172.21.15.38:0/2836138339 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f442c158f00 con 0x7f442c0cfe00 2024-09-21T03:42:07.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:07.627+0000 7f4421ffb640 1 -- 172.21.15.38:0/2836138339 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 190 v190) ==== 95+0+3566 (secure 0 0 0) 0x7f441c011e80 con 0x7f442c0cfe00 2024-09-21T03:42:07.629 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:42:07.629 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":190,"btime":"2024-09-21T03:42:06:901463+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6086,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/965972663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":965972663},{"type":"v1","addr":"172.21.15.195:6836","nonce":965972663}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6700,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/3176122334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":3176122334},{"type":"v1","addr":"172.21.15.195:6837","nonce":3176122334}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":190,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:40:02.319282+0000","modified":"2024-09-21T03:42:06.901398+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":114,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-21T03:42:07.629 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 190 2024-09-21T03:42:07.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:07.629+0000 7f4433969640 1 -- 172.21.15.38:0/2836138339 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4410078b90 msgr2=0x7f441007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:07.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:07.629+0000 7f4433969640 1 --2- 172.21.15.38:0/2836138339 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4410078b90 0x7f441007b0a0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f4428034000 tx=0x7f4428004730 comp rx=0 tx=0).stop 2024-09-21T03:42:07.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:07.629+0000 7f4433969640 1 -- 172.21.15.38:0/2836138339 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f442c0cfe00 msgr2=0x7f442c15db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:07.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:07.629+0000 7f4433969640 1 --2- 172.21.15.38:0/2836138339 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f442c0cfe00 0x7f442c15db40 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7f441c00c400 tx=0x7f441c00c8c0 comp rx=0 tx=0).stop 2024-09-21T03:42:07.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:07.629+0000 7f4433969640 1 -- 172.21.15.38:0/2836138339 shutdown_connections 2024-09-21T03:42:07.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:07.629+0000 7f4433969640 1 --2- 172.21.15.38:0/2836138339 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f442c162590 0x7f442c146740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:07.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:07.629+0000 7f4433969640 1 --2- 172.21.15.38:0/2836138339 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f442c0cfe00 0x7f442c15db40 unknown :-1 s=CLOSED pgs=1650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:07.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:07.629+0000 7f4433969640 1 --2- 172.21.15.38:0/2836138339 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4410078b90 0x7f441007b0a0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:07.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:07.629+0000 7f4433969640 1 --2- 172.21.15.38:0/2836138339 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f442c0b2e60 0x7f442c15a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:07.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:07.629+0000 7f4433969640 1 -- 172.21.15.38:0/2836138339 >> 172.21.15.38:0/2836138339 conn(0x7f442c0d4440 msgr2=0x7f442c0d4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:07.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:07.630+0000 7f4433969640 1 -- 172.21.15.38:0/2836138339 shutdown_connections 2024-09-21T03:42:07.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:07.630+0000 7f4433969640 1 -- 172.21.15.38:0/2836138339 wait complete. 2024-09-21T03:42:07.643 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-21T03:42:07.643 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:42:07.677 INFO:tasks.ceph.mds.a.smithi038.stdout:starting mds.a at 2024-09-21T03:42:07.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:07.759+0000 7fa9eb8d3640 1 -- 172.21.15.38:0/2395294978 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa9e40cfe00 msgr2=0x7fa9e40d2a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:07.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:07.759+0000 7fa9eb8d3640 1 --2- 172.21.15.38:0/2395294978 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa9e40cfe00 0x7fa9e40d2a20 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7fa9e0009830 tx=0x7fa9e002a510 comp rx=0 tx=0).stop 2024-09-21T03:42:07.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:07.760+0000 7fa9eb8d3640 1 -- 172.21.15.38:0/2395294978 shutdown_connections 2024-09-21T03:42:07.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:07.760+0000 7fa9eb8d3640 1 --2- 172.21.15.38:0/2395294978 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa9e4162590 0x7fa9e41629a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:07.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:07.760+0000 7fa9eb8d3640 1 --2- 172.21.15.38:0/2395294978 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa9e40cfe00 0x7fa9e40d2a20 unknown :-1 s=CLOSED pgs=1655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:07.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:07.760+0000 7fa9eb8d3640 1 -- 172.21.15.38:0/2395294978 >> 172.21.15.38:0/2395294978 conn(0x7fa9e40d4440 msgr2=0x7fa9e40d4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:07.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:07.760+0000 7fa9eb8d3640 1 -- 172.21.15.38:0/2395294978 shutdown_connections 2024-09-21T03:42:07.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:07.760+0000 7fa9eb8d3640 1 -- 172.21.15.38:0/2395294978 wait complete. 2024-09-21T03:42:07.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:07.761+0000 7fa9eb8d3640 1 Processor -- start 2024-09-21T03:42:07.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:07.762+0000 7fa9eb8d3640 1 -- start start 2024-09-21T03:42:07.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:07.763+0000 7fa9eb8d3640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa9e40b2e60 0x7fa9e40c71f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:07.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:07.763+0000 7fa9eb8d3640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa9e40cfe00 0x7fa9e415b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:07.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:07.763+0000 7fa9eb8d3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa9e4162590 0x7fa9e415bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:07.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:07.763+0000 7fa9eb8d3640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa9e40b3980 con 0x7fa9e4162590 2024-09-21T03:42:07.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:07.763+0000 7fa9eb8d3640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fa9e40cf5a0 con 0x7fa9e40b2e60 2024-09-21T03:42:07.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:07.763+0000 7fa9eb8d3640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fa9e40b3b00 con 0x7fa9e40cfe00 2024-09-21T03:42:07.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:07.763+0000 7fa9e9e70640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa9e4162590 0x7fa9e415bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:07.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:07.763+0000 7fa9e9e70640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa9e4162590 0x7fa9e415bdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49536/0 (socket says 172.21.15.38:49536) 2024-09-21T03:42:07.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:07.763+0000 7fa9e9e70640 1 -- 172.21.15.38:0/2471255842 learned_addr learned my addr 172.21.15.38:0/2471255842 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:42:07.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:07.763+0000 7fa9e966f640 1 --2- 172.21.15.38:0/2471255842 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa9e40b2e60 0x7fa9e40c71f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:07.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:07.763+0000 7fa9e9e70640 1 -- 172.21.15.38:0/2471255842 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa9e40cfe00 msgr2=0x7fa9e415b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:07.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:07.763+0000 7fa9e9e70640 1 --2- 172.21.15.38:0/2471255842 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa9e40cfe00 0x7fa9e415b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:07.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:07.763+0000 7fa9e9e70640 1 -- 172.21.15.38:0/2471255842 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa9e40b2e60 msgr2=0x7fa9e40c71f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:07.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:07.763+0000 7fa9e9e70640 1 --2- 172.21.15.38:0/2471255842 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa9e40b2e60 0x7fa9e40c71f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:07.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:07.763+0000 7fa9e9e70640 1 -- 172.21.15.38:0/2471255842 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9e41ec100 con 0x7fa9e4162590 2024-09-21T03:42:07.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:07.763+0000 7fa9e9e70640 1 --2- 172.21.15.38:0/2471255842 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa9e4162590 0x7fa9e415bdb0 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7fa9dc00cab0 tx=0x7fa9dc00cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:07.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:07.764+0000 7fa9d5ffb640 1 -- 172.21.15.38:0/2471255842 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9dc016020 con 0x7fa9e4162590 2024-09-21T03:42:07.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:07.764+0000 7fa9d5ffb640 1 -- 172.21.15.38:0/2471255842 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa9dc00ad20 con 0x7fa9e4162590 2024-09-21T03:42:07.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:07.764+0000 7fa9d5ffb640 1 -- 172.21.15.38:0/2471255842 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9dc012680 con 0x7fa9e4162590 2024-09-21T03:42:07.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:07.764+0000 7fa9eb8d3640 1 -- 172.21.15.38:0/2471255842 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9e40c7730 con 0x7fa9e4162590 2024-09-21T03:42:07.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:07.764+0000 7fa9eb8d3640 1 -- 172.21.15.38:0/2471255842 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9e41f6570 con 0x7fa9e4162590 2024-09-21T03:42:07.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:07.765+0000 7fa9eb8d3640 1 -- 172.21.15.38:0/2471255842 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9e41f6000 con 0x7fa9e4162590 2024-09-21T03:42:07.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:07.766+0000 7fa9d5ffb640 1 -- 172.21.15.38:0/2471255842 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa9dc019020 con 0x7fa9e4162590 2024-09-21T03:42:07.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:07.766+0000 7fa9d5ffb640 1 --2- 172.21.15.38:0/2471255842 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa9c8078b90 0x7fa9c807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:07.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:07.766+0000 7fa9d5ffb640 1 -- 172.21.15.38:0/2471255842 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 7852+0+0 (secure 0 0 0) 0x7fa9dc09b8c0 con 0x7fa9e4162590 2024-09-21T03:42:07.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:07.766+0000 7fa9e966f640 1 --2- 172.21.15.38:0/2471255842 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa9c8078b90 0x7fa9c807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:07.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:07.767+0000 7fa9e966f640 1 --2- 172.21.15.38:0/2471255842 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa9c8078b90 0x7fa9c807b0a0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fa9e0034000 tx=0x7fa9e0004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:07.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:07.771+0000 7fa9d5ffb640 1 -- 172.21.15.38:0/2471255842 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fa9dc020030 con 0x7fa9e4162590 2024-09-21T03:42:08.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:07.998+0000 7fa9eb8d3640 1 -- 172.21.15.38:0/2471255842 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa9e40d2a20 con 0x7fa9e4162590 2024-09-21T03:42:08.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:07.998+0000 7fa9d5ffb640 1 -- 172.21.15.38:0/2471255842 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 191 v191) ==== 95+0+4392 (secure 0 0 0) 0x7fa9e4162ee0 con 0x7fa9e4162590 2024-09-21T03:42:08.000 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:42:08.001 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":191,"btime":"2024-09-21T03:42:07:903055+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6086,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/965972663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":965972663},{"type":"v1","addr":"172.21.15.195:6836","nonce":965972663}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6700,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/3176122334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":3176122334},{"type":"v1","addr":"172.21.15.195:6837","nonce":3176122334}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":8862,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/2712360296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":2712360296},{"type":"v1","addr":"172.21.15.38:6835","nonce":2712360296}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":191}],"filesystems":[{"mdsmap":{"epoch":190,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:40:02.319282+0000","modified":"2024-09-21T03:42:06.901398+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":114,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-21T03:42:08.001 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 191 2024-09-21T03:42:08.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:08.001+0000 7fa9eb8d3640 1 -- 172.21.15.38:0/2471255842 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa9c8078b90 msgr2=0x7fa9c807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:08.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:08.001+0000 7fa9eb8d3640 1 --2- 172.21.15.38:0/2471255842 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa9c8078b90 0x7fa9c807b0a0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fa9e0034000 tx=0x7fa9e0004730 comp rx=0 tx=0).stop 2024-09-21T03:42:08.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:08.001+0000 7fa9eb8d3640 1 -- 172.21.15.38:0/2471255842 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa9e4162590 msgr2=0x7fa9e415bdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:08.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:08.001+0000 7fa9eb8d3640 1 --2- 172.21.15.38:0/2471255842 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa9e4162590 0x7fa9e415bdb0 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7fa9dc00cab0 tx=0x7fa9dc00cf70 comp rx=0 tx=0).stop 2024-09-21T03:42:08.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:08.001+0000 7fa9eb8d3640 1 -- 172.21.15.38:0/2471255842 shutdown_connections 2024-09-21T03:42:08.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:08.001+0000 7fa9eb8d3640 1 --2- 172.21.15.38:0/2471255842 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa9e4162590 0x7fa9e415bdb0 unknown :-1 s=CLOSED pgs=1656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:08.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:08.001+0000 7fa9eb8d3640 1 --2- 172.21.15.38:0/2471255842 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa9c8078b90 0x7fa9c807b0a0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:08.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:08.001+0000 7fa9eb8d3640 1 --2- 172.21.15.38:0/2471255842 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa9e40cfe00 0x7fa9e415b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:08.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:08.001+0000 7fa9eb8d3640 1 --2- 172.21.15.38:0/2471255842 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa9e40b2e60 0x7fa9e40c71f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:08.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:08.001+0000 7fa9eb8d3640 1 -- 172.21.15.38:0/2471255842 >> 172.21.15.38:0/2471255842 conn(0x7fa9e40d4440 msgr2=0x7fa9e406cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:08.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:08.001+0000 7fa9eb8d3640 1 -- 172.21.15.38:0/2471255842 shutdown_connections 2024-09-21T03:42:08.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:08.001+0000 7fa9eb8d3640 1 -- 172.21.15.38:0/2471255842 wait complete. 2024-09-21T03:42:08.014 INFO:tasks.cephfs.test_damage:Result: Mutation 'Corrupt 200.00000001' led to DAMAGED state 2024-09-21T03:42:08.014 INFO:tasks.cephfs.test_damage:Applying mutation 'Corrupt 500.00000000' 2024-09-21T03:42:08.014 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-21T03:42:08.014 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:42:08.070 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056795,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2354 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T03:42:08.071 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi038.front.sepia.ceph.com 2024-09-21T03:42:08.071 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:42:08.071 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T03:42:08.126 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T03:42:08.126 INFO:teuthology.orchestra.run.smithi038.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-09-21T03:42:08.127 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-21T03:42:08.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:08.283+0000 7fb8cb069640 1 -- 172.21.15.38:0/3337116428 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb8ac003a00 msgr2=0x7fb8ac003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:08.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:08.283+0000 7fb8cb069640 1 --2- 172.21.15.38:0/3337116428 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb8ac003a00 0x7fb8ac003e70 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7fb8c0005fd0 tx=0x7fb8ac005830 comp rx=0 tx=0).stop 2024-09-21T03:42:08.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:08.283+0000 7fb8c9606640 1 -- 172.21.15.38:0/3337116428 reap_dead start 2024-09-21T03:42:08.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:08.283+0000 7fb8cb069640 1 -- 172.21.15.38:0/3337116428 shutdown_connections 2024-09-21T03:42:08.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:08.283+0000 7fb8cb069640 1 -- 172.21.15.38:0/3337116428 >> 172.21.15.38:0/3337116428 conn(0x7fb8c40cdca0 msgr2=0x7fb8c40ce0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:08.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:08.283+0000 7fb8cb069640 1 -- 172.21.15.38:0/3337116428 shutdown_connections 2024-09-21T03:42:08.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:08.283+0000 7fb8cb069640 1 -- 172.21.15.38:0/3337116428 wait complete. 2024-09-21T03:42:08.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:08.285+0000 7fb8cb069640 1 Processor -- start 2024-09-21T03:42:08.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:08.285+0000 7fb8cb069640 1 -- start start 2024-09-21T03:42:08.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:08.287+0000 7fb8cb069640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb8ac003a00 0x7fb8c4156600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:08.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:08.287+0000 7fb8cb069640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb8c41f8980 0x7fb8c41f8d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:08.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:08.287+0000 7fb8cb069640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb8c41f92d0 0x7fb8c40b1320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:08.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:08.287+0000 7fb8cb069640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fb8c40b5a00 con 0x7fb8c41f8980 2024-09-21T03:42:08.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:08.287+0000 7fb8cb069640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fb8c40b5880 con 0x7fb8c41f92d0 2024-09-21T03:42:08.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:08.287+0000 7fb8cb069640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fb8c415ba80 con 0x7fb8ac003a00 2024-09-21T03:42:08.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:08.287+0000 7fb8bbfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb8c41f8980 0x7fb8c41f8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:08.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:08.287+0000 7fb8bbfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb8c41f8980 0x7fb8c41f8d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49554/0 (socket says 172.21.15.38:49554) 2024-09-21T03:42:08.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:08.287+0000 7fb8bbfff640 1 -- 172.21.15.38:0/4208406950 learned_addr learned my addr 172.21.15.38:0/4208406950 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:42:08.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:08.287+0000 7fb8c8e05640 1 --2- 172.21.15.38:0/4208406950 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb8ac003a00 0x7fb8c4156600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:08.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:08.287+0000 7fb8c9606640 1 --2- 172.21.15.38:0/4208406950 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb8c41f92d0 0x7fb8c40b1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:08.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:08.287+0000 7fb8bbfff640 1 -- 172.21.15.38:0/4208406950 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb8ac003a00 msgr2=0x7fb8c4156600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:08.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:08.287+0000 7fb8bbfff640 1 --2- 172.21.15.38:0/4208406950 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb8ac003a00 0x7fb8c4156600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:08.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:08.287+0000 7fb8bbfff640 1 -- 172.21.15.38:0/4208406950 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb8c41f92d0 msgr2=0x7fb8c40b1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:08.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:08.287+0000 7fb8bbfff640 1 --2- 172.21.15.38:0/4208406950 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb8c41f92d0 0x7fb8c40b1320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:08.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:08.287+0000 7fb8bbfff640 1 -- 172.21.15.38:0/4208406950 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8c4146ec0 con 0x7fb8c41f8980 2024-09-21T03:42:08.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:08.287+0000 7fb8bbfff640 1 --2- 172.21.15.38:0/4208406950 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb8c41f8980 0x7fb8c41f8d90 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7fb8b4011820 tx=0x7fb8b4011c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:08.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:08.287+0000 7fb8b97fa640 1 -- 172.21.15.38:0/4208406950 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8b400fcd0 con 0x7fb8c41f8980 2024-09-21T03:42:08.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:08.287+0000 7fb8b97fa640 1 -- 172.21.15.38:0/4208406950 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb8b4016070 con 0x7fb8c41f8980 2024-09-21T03:42:08.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:08.287+0000 7fb8cb069640 1 -- 172.21.15.38:0/4208406950 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8c41471a0 con 0x7fb8c41f8980 2024-09-21T03:42:08.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:08.288+0000 7fb8b97fa640 1 -- 172.21.15.38:0/4208406950 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8b40126b0 con 0x7fb8c41f8980 2024-09-21T03:42:08.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:08.288+0000 7fb8cb069640 1 -- 172.21.15.38:0/4208406950 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8c4156b40 con 0x7fb8c41f8980 2024-09-21T03:42:08.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:08.288+0000 7fb8cb069640 1 -- 172.21.15.38:0/4208406950 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8c4146c30 con 0x7fb8c41f8980 2024-09-21T03:42:08.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:08.289+0000 7fb8b97fa640 1 -- 172.21.15.38:0/4208406950 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb8b4018030 con 0x7fb8c41f8980 2024-09-21T03:42:08.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:08.290+0000 7fb8b97fa640 1 --2- 172.21.15.38:0/4208406950 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb8b0078bc0 0x7fb8b007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:08.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:08.290+0000 7fb8b97fa640 1 -- 172.21.15.38:0/4208406950 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 7852+0+0 (secure 0 0 0) 0x7fb8b409bb90 con 0x7fb8c41f8980 2024-09-21T03:42:08.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:08.290+0000 7fb8c8e05640 1 --2- 172.21.15.38:0/4208406950 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb8b0078bc0 0x7fb8b007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:08.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:08.291+0000 7fb8c8e05640 1 --2- 172.21.15.38:0/4208406950 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb8b0078bc0 0x7fb8b007b0d0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fb8c0005fd0 tx=0x7fb8c002d040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:08.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:08.296+0000 7fb8b97fa640 1 -- 172.21.15.38:0/4208406950 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fb8b4063120 con 0x7fb8c41f8980 2024-09-21T03:42:08.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:08.522+0000 7fb8cb069640 1 -- 172.21.15.38:0/4208406950 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fb8c406f410 con 0x7fb8c41f8980 2024-09-21T03:42:08.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:08.912+0000 7fb8b97fa640 1 -- 172.21.15.38:0/4208406950 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v192) ==== 161+0+0 (secure 0 0 0) 0x7fb8b4067b50 con 0x7fb8c41f8980 2024-09-21T03:42:08.914 INFO:teuthology.orchestra.run.smithi038.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-21T03:42:08.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:08.914+0000 7fb8cb069640 1 -- 172.21.15.38:0/4208406950 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb8b0078bc0 msgr2=0x7fb8b007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:08.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:08.914+0000 7fb8cb069640 1 --2- 172.21.15.38:0/4208406950 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb8b0078bc0 0x7fb8b007b0d0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fb8c0005fd0 tx=0x7fb8c002d040 comp rx=0 tx=0).stop 2024-09-21T03:42:08.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:08.914+0000 7fb8cb069640 1 -- 172.21.15.38:0/4208406950 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb8c41f8980 msgr2=0x7fb8c41f8d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:08.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:08.914+0000 7fb8cb069640 1 --2- 172.21.15.38:0/4208406950 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb8c41f8980 0x7fb8c41f8d90 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7fb8b4011820 tx=0x7fb8b4011c20 comp rx=0 tx=0).stop 2024-09-21T03:42:08.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:08.914+0000 7fb8cb069640 1 -- 172.21.15.38:0/4208406950 shutdown_connections 2024-09-21T03:42:08.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:08.914+0000 7fb8cb069640 1 --2- 172.21.15.38:0/4208406950 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb8c41f92d0 0x7fb8c40b1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:08.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:08.914+0000 7fb8cb069640 1 --2- 172.21.15.38:0/4208406950 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb8c41f8980 0x7fb8c41f8d90 unknown :-1 s=CLOSED pgs=1659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:08.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:08.914+0000 7fb8cb069640 1 --2- 172.21.15.38:0/4208406950 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb8b0078bc0 0x7fb8b007b0d0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:08.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:08.914+0000 7fb8cb069640 1 --2- 172.21.15.38:0/4208406950 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb8ac003a00 0x7fb8c4156600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:08.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:08.914+0000 7fb8cb069640 1 -- 172.21.15.38:0/4208406950 >> 172.21.15.38:0/4208406950 conn(0x7fb8c40cdca0 msgr2=0x7fb8c41558b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:08.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:08.915+0000 7fb8cb069640 1 -- 172.21.15.38:0/4208406950 shutdown_connections 2024-09-21T03:42:08.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:08.915+0000 7fb8cb069640 1 -- 172.21.15.38:0/4208406950 wait complete. 2024-09-21T03:42:08.929 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-09-21T03:42:09.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:09.048+0000 7f3a03c84640 1 -- 172.21.15.38:0/3605114930 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f39fc162710 msgr2=0x7f39fc162b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:09.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:09.048+0000 7f3a03c84640 1 --2- 172.21.15.38:0/3605114930 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f39fc162710 0x7f39fc162b20 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f39ec005fd0 tx=0x7f39ec02a540 comp rx=0 tx=0).stop 2024-09-21T03:42:09.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:09.048+0000 7f3a03c84640 1 -- 172.21.15.38:0/3605114930 shutdown_connections 2024-09-21T03:42:09.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:09.048+0000 7f3a03c84640 1 --2- 172.21.15.38:0/3605114930 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f39fc162710 0x7f39fc162b20 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:09.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:09.048+0000 7f3a03c84640 1 --2- 172.21.15.38:0/3605114930 >> v2:172.21.15.195:3300/0 conn(0x7f39fc0b5bd0 0x7f39fc15ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:09.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:09.048+0000 7f3a03c84640 1 -- 172.21.15.38:0/3605114930 >> 172.21.15.38:0/3605114930 conn(0x7f39fc0b46e0 msgr2=0x7f39fc0d23e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:09.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:09.048+0000 7f3a03c84640 1 -- 172.21.15.38:0/3605114930 shutdown_connections 2024-09-21T03:42:09.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:09.048+0000 7f3a03c84640 1 -- 172.21.15.38:0/3605114930 wait complete. 2024-09-21T03:42:09.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:09.050+0000 7f3a03c84640 1 Processor -- start 2024-09-21T03:42:09.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:09.050+0000 7f3a03c84640 1 -- start start 2024-09-21T03:42:09.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:09.051+0000 7f3a03c84640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f39fc0b5bd0 0x7f39fc154800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:09.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:09.051+0000 7f3a03c84640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f39fc162710 0x7f39fc15fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:09.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:09.051+0000 7f3a03c84640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f39fc163060 0x7f39fc151460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:09.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:09.051+0000 7f3a03c84640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f39fc0b7b90 con 0x7f39fc162710 2024-09-21T03:42:09.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:09.051+0000 7f3a03c84640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f39fc0ce670 con 0x7f39fc163060 2024-09-21T03:42:09.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:09.051+0000 7f3a03c84640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f39fc0b7d10 con 0x7f39fc0b5bd0 2024-09-21T03:42:09.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:09.051+0000 7f3a0121f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f39fc162710 0x7f39fc15fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:09.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:09.051+0000 7f3a0121f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f39fc162710 0x7f39fc15fc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49568/0 (socket says 172.21.15.38:49568) 2024-09-21T03:42:09.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:09.051+0000 7f3a0121f640 1 -- 172.21.15.38:0/172003045 learned_addr learned my addr 172.21.15.38:0/172003045 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:42:09.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:09.052+0000 7f3a02221640 1 --2- 172.21.15.38:0/172003045 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f39fc163060 0x7f39fc151460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:09.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:09.052+0000 7f3a01a20640 1 --2- 172.21.15.38:0/172003045 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f39fc0b5bd0 0x7f39fc154800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:09.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:09.052+0000 7f3a0121f640 1 -- 172.21.15.38:0/172003045 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f39fc0b5bd0 msgr2=0x7f39fc154800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:09.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:09.052+0000 7f3a0121f640 1 --2- 172.21.15.38:0/172003045 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f39fc0b5bd0 0x7f39fc154800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:09.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:09.052+0000 7f3a0121f640 1 -- 172.21.15.38:0/172003045 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f39fc163060 msgr2=0x7f39fc151460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:09.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:09.052+0000 7f3a0121f640 1 --2- 172.21.15.38:0/172003045 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f39fc163060 0x7f39fc151460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:09.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:09.052+0000 7f3a0121f640 1 -- 172.21.15.38:0/172003045 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39fc1559d0 con 0x7f39fc162710 2024-09-21T03:42:09.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:09.052+0000 7f3a02221640 1 --2- 172.21.15.38:0/172003045 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f39fc163060 0x7f39fc151460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:42:09.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:09.052+0000 7f3a01a20640 1 --2- 172.21.15.38:0/172003045 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f39fc0b5bd0 0x7f39fc154800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:42:09.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:09.052+0000 7f3a0121f640 1 --2- 172.21.15.38:0/172003045 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f39fc162710 0x7f39fc15fc80 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7f39ec034040 tx=0x7f39ec0038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:09.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:09.052+0000 7f39f27fc640 1 -- 172.21.15.38:0/172003045 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39ec03e030 con 0x7f39fc162710 2024-09-21T03:42:09.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:09.052+0000 7f39f27fc640 1 -- 172.21.15.38:0/172003045 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f39ec003db0 con 0x7f39fc162710 2024-09-21T03:42:09.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:09.052+0000 7f3a03c84640 1 -- 172.21.15.38:0/172003045 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39fc1f5e40 con 0x7f39fc162710 2024-09-21T03:42:09.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:09.052+0000 7f39f27fc640 1 -- 172.21.15.38:0/172003045 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39ec035040 con 0x7f39fc162710 2024-09-21T03:42:09.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:09.052+0000 7f3a03c84640 1 -- 172.21.15.38:0/172003045 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39fc1f6400 con 0x7f39fc162710 2024-09-21T03:42:09.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:09.053+0000 7f3a03c84640 1 -- 172.21.15.38:0/172003045 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39fc0b81e0 con 0x7f39fc162710 2024-09-21T03:42:09.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:09.054+0000 7f39f27fc640 1 -- 172.21.15.38:0/172003045 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f39ec041030 con 0x7f39fc162710 2024-09-21T03:42:09.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:09.055+0000 7f39f27fc640 1 --2- 172.21.15.38:0/172003045 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f39e0078b90 0x7f39e007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:09.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:09.055+0000 7f39f27fc640 1 -- 172.21.15.38:0/172003045 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 7852+0+0 (secure 0 0 0) 0x7f39ec0bbc00 con 0x7f39fc162710 2024-09-21T03:42:09.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:09.055+0000 7f3a01a20640 1 --2- 172.21.15.38:0/172003045 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f39e0078b90 0x7f39e007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:09.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:09.056+0000 7f3a01a20640 1 --2- 172.21.15.38:0/172003045 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f39e0078b90 0x7f39e007b0a0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f39f8009b80 tx=0x7f39f8005f90 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:09.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:09.059+0000 7f39f27fc640 1 -- 172.21.15.38:0/172003045 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f39ec0c2080 con 0x7f39fc162710 2024-09-21T03:42:09.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:09.320+0000 7f3a03c84640 1 -- 172.21.15.38:0/172003045 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7f39fc0b75c0 con 0x7f39fc162710 2024-09-21T03:42:09.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:09.918+0000 7f39f27fc640 1 -- 172.21.15.38:0/172003045 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v193) ==== 101+0+0 (secure 0 0 0) 0x7f39ec083870 con 0x7f39fc162710 2024-09-21T03:42:09.921 INFO:teuthology.orchestra.run.smithi038.stderr:repaired: restoring rank 6:0 2024-09-21T03:42:09.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:09.921+0000 7f3a03c84640 1 -- 172.21.15.38:0/172003045 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f39e0078b90 msgr2=0x7f39e007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:09.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:09.921+0000 7f3a03c84640 1 --2- 172.21.15.38:0/172003045 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f39e0078b90 0x7f39e007b0a0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f39f8009b80 tx=0x7f39f8005f90 comp rx=0 tx=0).stop 2024-09-21T03:42:09.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:09.921+0000 7f3a03c84640 1 -- 172.21.15.38:0/172003045 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f39fc162710 msgr2=0x7f39fc15fc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:09.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:09.921+0000 7f3a03c84640 1 --2- 172.21.15.38:0/172003045 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f39fc162710 0x7f39fc15fc80 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7f39ec034040 tx=0x7f39ec0038a0 comp rx=0 tx=0).stop 2024-09-21T03:42:09.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:09.921+0000 7f3a03c84640 1 -- 172.21.15.38:0/172003045 shutdown_connections 2024-09-21T03:42:09.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:09.921+0000 7f3a03c84640 1 --2- 172.21.15.38:0/172003045 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f39fc163060 0x7f39fc151460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:09.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:09.921+0000 7f3a03c84640 1 --2- 172.21.15.38:0/172003045 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f39fc162710 0x7f39fc15fc80 unknown :-1 s=CLOSED pgs=1660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:09.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:09.921+0000 7f3a03c84640 1 --2- 172.21.15.38:0/172003045 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f39e0078b90 0x7f39e007b0a0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:09.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:09.921+0000 7f3a03c84640 1 --2- 172.21.15.38:0/172003045 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f39fc0b5bd0 0x7f39fc154800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:09.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:09.921+0000 7f3a03c84640 1 -- 172.21.15.38:0/172003045 >> 172.21.15.38:0/172003045 conn(0x7f39fc0b46e0 msgr2=0x7f39fc0b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:09.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:09.921+0000 7f3a03c84640 1 -- 172.21.15.38:0/172003045 shutdown_connections 2024-09-21T03:42:09.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:09.921+0000 7f3a03c84640 1 -- 172.21.15.38:0/172003045 wait complete. 2024-09-21T03:42:09.935 DEBUG:teuthology.orchestra.run.smithi038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-09-21T03:42:09.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:09.990+0000 7f25c4979f40 1 Processor -- start 2024-09-21T03:42:09.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:09.990+0000 7f25c4979f40 1 -- start start 2024-09-21T03:42:09.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:09.991+0000 7f25c4979f40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d730981d00 0x55d730a37ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:09.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:09.991+0000 7f25c4979f40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55d7309ade30 0x55d730a360a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:09.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:09.991+0000 7f25c4979f40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d730a3c950 0x55d730a3a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:09.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:09.991+0000 7f25c4979f40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55d7308cfd60 con 0x55d730a3c950 2024-09-21T03:42:09.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:09.991+0000 7f25c4979f40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55d73092e430 con 0x55d730981d00 2024-09-21T03:42:09.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:09.991+0000 7f25c4979f40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55d7309ca460 con 0x55d7309ade30 2024-09-21T03:42:09.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:09.992+0000 7f25bec34640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d730a3c950 0x55d730a3a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:09.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:09.992+0000 7f25bec34640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d730a3c950 0x55d730a3a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49582/0 (socket says 172.21.15.38:49582) 2024-09-21T03:42:09.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:09.992+0000 7f25bec34640 1 -- 172.21.15.38:0/3852849296 learned_addr learned my addr 172.21.15.38:0/3852849296 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:42:09.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:09.992+0000 7f25be433640 1 --2- 172.21.15.38:0/3852849296 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d730981d00 0x55d730a37ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:09.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:09.992+0000 7f25bdc32640 1 --2- 172.21.15.38:0/3852849296 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55d7309ade30 0x55d730a360a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:09.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:09.992+0000 7f25bec34640 1 -- 172.21.15.38:0/3852849296 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55d7309ade30 msgr2=0x55d730a360a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:09.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:09.992+0000 7f25bec34640 1 --2- 172.21.15.38:0/3852849296 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55d7309ade30 0x55d730a360a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:09.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:09.992+0000 7f25bec34640 1 -- 172.21.15.38:0/3852849296 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d730981d00 msgr2=0x55d730a37ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:09.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:09.992+0000 7f25bec34640 1 --2- 172.21.15.38:0/3852849296 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d730981d00 0x55d730a37ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:09.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:09.992+0000 7f25bec34640 1 -- 172.21.15.38:0/3852849296 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d730a3c1a0 con 0x55d730a3c950 2024-09-21T03:42:09.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:09.992+0000 7f25bdc32640 1 --2- 172.21.15.38:0/3852849296 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55d7309ade30 0x55d730a360a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:42:09.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:09.992+0000 7f25bec34640 1 --2- 172.21.15.38:0/3852849296 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d730a3c950 0x55d730a3a2f0 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7f25b8008670 tx=0x7f25b8008b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc5e8f4416a824f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:09.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:09.992+0000 7f25bcc30640 1 -- 172.21.15.38:0/3852849296 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25b800d0d0 con 0x55d730a3c950 2024-09-21T03:42:09.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:09.992+0000 7f25bcc30640 1 -- 172.21.15.38:0/3852849296 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f25b800d270 con 0x55d730a3c950 2024-09-21T03:42:09.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:09.992+0000 7f25bcc30640 1 -- 172.21.15.38:0/3852849296 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25b800dbc0 con 0x55d730a3c950 2024-09-21T03:42:09.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:09.993+0000 7f25c4979f40 1 -- 172.21.15.38:0/3852849296 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d730a3c950 msgr2=0x55d730a3a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:09.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:09.993+0000 7f25c4979f40 1 --2- 172.21.15.38:0/3852849296 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d730a3c950 0x55d730a3a2f0 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7f25b8008670 tx=0x7f25b8008b30 comp rx=0 tx=0).stop 2024-09-21T03:42:09.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:09.993+0000 7f25c4979f40 1 -- 172.21.15.38:0/3852849296 shutdown_connections 2024-09-21T03:42:09.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:09.993+0000 7f25c4979f40 1 --2- 172.21.15.38:0/3852849296 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d730a3c950 0x55d730a3a2f0 unknown :-1 s=CLOSED pgs=1662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:09.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:09.993+0000 7f25c4979f40 1 --2- 172.21.15.38:0/3852849296 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55d7309ade30 0x55d730a360a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:09.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:09.993+0000 7f25c4979f40 1 --2- 172.21.15.38:0/3852849296 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d730981d00 0x55d730a37ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:09.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:09.993+0000 7f25c4979f40 1 -- 172.21.15.38:0/3852849296 >> 172.21.15.38:0/3852849296 conn(0x55d7308a0ac0 msgr2=0x55d730a537c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:09.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:09.993+0000 7f25c4979f40 1 -- 172.21.15.38:0/3852849296 shutdown_connections 2024-09-21T03:42:09.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:09.993+0000 7f25c4979f40 1 -- 172.21.15.38:0/3852849296 wait complete. 2024-09-21T03:42:09.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:09.994+0000 7f25c4979f40 1 Processor -- start 2024-09-21T03:42:09.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:09.994+0000 7f25c4979f40 1 -- start start 2024-09-21T03:42:09.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:09.995+0000 7f25c4979f40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d730981d00 0x55d730a50490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:09.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:09.995+0000 7f25c4979f40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d7309ade30 0x55d730a57540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:09.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:09.995+0000 7f25c4979f40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55d730a3c950 0x55d730a52190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:09.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:09.995+0000 7f25c4979f40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55d730a530a0 con 0x55d730981d00 2024-09-21T03:42:09.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:09.995+0000 7f25c4979f40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55d7309aed60 con 0x55d7309ade30 2024-09-21T03:42:09.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:09.995+0000 7f25c4979f40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55d730a3bf00 con 0x55d730a3c950 2024-09-21T03:42:09.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:09.995+0000 7f25be433640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d730981d00 0x55d730a50490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:09.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:09.995+0000 7f25be433640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d730981d00 0x55d730a50490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49584/0 (socket says 172.21.15.38:49584) 2024-09-21T03:42:09.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:09.995+0000 7f25be433640 1 -- 172.21.15.38:0/2350447563 learned_addr learned my addr 172.21.15.38:0/2350447563 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:42:09.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:09.995+0000 7f25bec34640 1 --2- 172.21.15.38:0/2350447563 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55d730a3c950 0x55d730a52190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:10.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:09.995+0000 7f25bdc32640 1 --2- 172.21.15.38:0/2350447563 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d7309ade30 0x55d730a57540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:10.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:09.996+0000 7f25be433640 1 -- 172.21.15.38:0/2350447563 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55d730a3c950 msgr2=0x55d730a52190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:10.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:09.996+0000 7f25be433640 1 --2- 172.21.15.38:0/2350447563 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55d730a3c950 0x55d730a52190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:10.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:09.996+0000 7f25be433640 1 -- 172.21.15.38:0/2350447563 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d7309ade30 msgr2=0x55d730a57540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:10.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:09.996+0000 7f25be433640 1 --2- 172.21.15.38:0/2350447563 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d7309ade30 0x55d730a57540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:10.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:09.996+0000 7f25be433640 1 -- 172.21.15.38:0/2350447563 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d730a564c0 con 0x55d730981d00 2024-09-21T03:42:10.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:09.996+0000 7f25bdc32640 1 --2- 172.21.15.38:0/2350447563 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d7309ade30 0x55d730a57540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:42:10.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:09.996+0000 7f25bec34640 1 --2- 172.21.15.38:0/2350447563 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55d730a3c950 0x55d730a52190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:42:10.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:09.996+0000 7f25be433640 1 --2- 172.21.15.38:0/2350447563 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d730981d00 0x55d730a50490 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7f25b000d3f0 tx=0x7f25b000d8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:10.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:09.996+0000 7f25af7fe640 1 -- 172.21.15.38:0/2350447563 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25b0011590 con 0x55d730981d00 2024-09-21T03:42:10.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:09.996+0000 7f25af7fe640 1 -- 172.21.15.38:0/2350447563 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f25b0017070 con 0x55d730981d00 2024-09-21T03:42:10.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:09.996+0000 7f25c4979f40 1 -- 172.21.15.38:0/2350447563 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55d730a561e0 con 0x55d730981d00 2024-09-21T03:42:10.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:09.996+0000 7f25af7fe640 1 -- 172.21.15.38:0/2350447563 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25b0011aa0 con 0x55d730981d00 2024-09-21T03:42:10.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:09.996+0000 7f25c4979f40 1 -- 172.21.15.38:0/2350447563 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d730a4ef70 con 0x55d730981d00 2024-09-21T03:42:10.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:09.998+0000 7f25af7fe640 1 -- 172.21.15.38:0/2350447563 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f25b0011c40 con 0x55d730981d00 2024-09-21T03:42:10.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:09.998+0000 7f25af7fe640 1 --2- 172.21.15.38:0/2350447563 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f25a0077570 0x7f25a0079a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:10.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:09.998+0000 7f25af7fe640 1 -- 172.21.15.38:0/2350447563 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 7852+0+0 (secure 0 0 0) 0x7f25b0099bc0 con 0x55d730981d00 2024-09-21T03:42:10.002 INFO:teuthology.orchestra.run.smithi038.stdout:Importing pool 2024-09-21T03:42:10.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:09.999+0000 7f25bdc32640 1 --2- 172.21.15.38:0/2350447563 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f25a0077570 0x7f25a0079a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:10.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:09.999+0000 7f25c4979f40 1 --2- 172.21.15.38:0/2350447563 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x55d730aee1a0 0x55d730aee5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:10.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:09.999+0000 7f25c4979f40 1 -- 172.21.15.38:0/2350447563 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55d730a35ac0 con 0x55d730aee1a0 2024-09-21T03:42:10.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:09.999+0000 7f25bec34640 1 --2- 172.21.15.38:0/2350447563 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x55d730aee1a0 0x55d730aee5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:10.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:09.999+0000 7f25bdc32640 1 --2- 172.21.15.38:0/2350447563 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f25a0077570 0x7f25a0079a80 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f25b4055000 tx=0x7f25b4058040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:10.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:09.999+0000 7f25bec34640 1 --2- 172.21.15.38:0/2350447563 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x55d730aee1a0 0x55d730aee5b0 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:10.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.001+0000 7f25bec34640 1 -- 172.21.15.38:0/2350447563 <== osd.7 v2:172.21.15.195:6818/1606106342 1 ==== osd_op_reply(1 601.00000000 [create] v114'536 uv510 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f25b800d960 con 0x55d730aee1a0 2024-09-21T03:42:10.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.001+0000 7f25c4979f40 1 -- 172.21.15.38:0/2350447563 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e114) -- 0x55d730a4f780 con 0x55d730aee1a0 2024-09-21T03:42:10.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.002+0000 7f25bec34640 1 -- 172.21.15.38:0/2350447563 <== osd.7 v2:172.21.15.195:6818/1606106342 2 ==== osd_op_reply(2 601.00000000 [delete] v114'537 uv537 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f25b800d960 con 0x55d730aee1a0 2024-09-21T03:42:10.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.002+0000 7f25c4979f40 1 -- 172.21.15.38:0/2350447563 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55d730a509d0 con 0x55d730aee1a0 2024-09-21T03:42:10.009 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-09-21T03:42:10.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.004+0000 7f25bec34640 1 -- 172.21.15.38:0/2350447563 <== osd.7 v2:172.21.15.195:6818/1606106342 3 ==== osd_op_reply(3 601.00000000 [create] v114'538 uv538 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f25b800d960 con 0x55d730aee1a0 2024-09-21T03:42:10.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.004+0000 7f25c4979f40 1 -- 172.21.15.38:0/2350447563 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55d730a57a80 con 0x55d730aee1a0 2024-09-21T03:42:10.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.005+0000 7f25bec34640 1 -- 172.21.15.38:0/2350447563 <== osd.7 v2:172.21.15.195:6818/1606106342 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v114'539 uv539 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f25b800d960 con 0x55d730aee1a0 2024-09-21T03:42:10.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.005+0000 7f25c4979f40 1 -- 172.21.15.38:0/2350447563 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55d730af7f70 con 0x55d730aee1a0 2024-09-21T03:42:10.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.007+0000 7f25bec34640 1 -- 172.21.15.38:0/2350447563 <== osd.7 v2:172.21.15.195:6818/1606106342 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v114'540 uv540 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f25b800d960 con 0x55d730aee1a0 2024-09-21T03:42:10.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.007+0000 7f25c4979f40 1 -- 172.21.15.38:0/2350447563 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55d730af6f60 con 0x55d730aee1a0 2024-09-21T03:42:10.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.008+0000 7f25bec34640 1 -- 172.21.15.38:0/2350447563 <== osd.7 v2:172.21.15.195:6818/1606106342 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v114'541 uv541 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f25b800d960 con 0x55d730aee1a0 2024-09-21T03:42:10.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.008+0000 7f25c4979f40 1 -- 172.21.15.38:0/2350447563 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55d730af7350 con 0x55d730aee1a0 2024-09-21T03:42:10.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.009+0000 7f25bec34640 1 -- 172.21.15.38:0/2350447563 <== osd.7 v2:172.21.15.195:6818/1606106342 7 ==== osd_op_reply(7 602.00000000 [create] v114'542 uv516 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f25b800d960 con 0x55d730aee1a0 2024-09-21T03:42:10.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.009+0000 7f25c4979f40 1 -- 172.21.15.38:0/2350447563 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e114) -- 0x55d730af76f0 con 0x55d730aee1a0 2024-09-21T03:42:10.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.011+0000 7f25bec34640 1 -- 172.21.15.38:0/2350447563 <== osd.7 v2:172.21.15.195:6818/1606106342 8 ==== osd_op_reply(8 602.00000000 [delete] v114'543 uv543 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f25b800d960 con 0x55d730aee1a0 2024-09-21T03:42:10.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.011+0000 7f25c4979f40 1 -- 172.21.15.38:0/2350447563 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55d730af76f0 con 0x55d730aee1a0 2024-09-21T03:42:10.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.012+0000 7f25bec34640 1 -- 172.21.15.38:0/2350447563 <== osd.7 v2:172.21.15.195:6818/1606106342 9 ==== osd_op_reply(9 602.00000000 [create] v114'544 uv544 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f25b800d960 con 0x55d730aee1a0 2024-09-21T03:42:10.015 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-09-21T03:42:10.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.012+0000 7f25c4979f40 1 -- 172.21.15.38:0/2350447563 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55d730af76f0 con 0x55d730aee1a0 2024-09-21T03:42:10.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.014+0000 7f25bec34640 1 -- 172.21.15.38:0/2350447563 <== osd.7 v2:172.21.15.195:6818/1606106342 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v114'545 uv545 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f25b800d960 con 0x55d730aee1a0 2024-09-21T03:42:10.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.014+0000 7f25c4979f40 1 -- 172.21.15.38:0/2350447563 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55d730af76f0 con 0x55d730aee1a0 2024-09-21T03:42:10.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.015+0000 7f25bec34640 1 -- 172.21.15.38:0/2350447563 <== osd.7 v2:172.21.15.195:6818/1606106342 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v114'546 uv546 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f25b800d960 con 0x55d730aee1a0 2024-09-21T03:42:10.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.015+0000 7f25c4979f40 1 -- 172.21.15.38:0/2350447563 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55d730af76f0 con 0x55d730aee1a0 2024-09-21T03:42:10.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.017+0000 7f25bec34640 1 -- 172.21.15.38:0/2350447563 <== osd.7 v2:172.21.15.195:6818/1606106342 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v114'547 uv547 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f25b800d960 con 0x55d730aee1a0 2024-09-21T03:42:10.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.017+0000 7f25c4979f40 1 -- 172.21.15.38:0/2350447563 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:13 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55d730af76f0 con 0x55d730aee1a0 2024-09-21T03:42:10.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.018+0000 7f25bec34640 1 -- 172.21.15.38:0/2350447563 <== osd.7 v2:172.21.15.195:6818/1606106342 13 ==== osd_op_reply(13 10000000000.00000000 [create] v114'548 uv523 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7f25b800d960 con 0x55d730aee1a0 2024-09-21T03:42:10.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.018+0000 7f25c4979f40 1 -- 172.21.15.38:0/2350447563 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:14 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e114) -- 0x55d730af76f0 con 0x55d730aee1a0 2024-09-21T03:42:10.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.020+0000 7f25bec34640 1 -- 172.21.15.38:0/2350447563 <== osd.7 v2:172.21.15.195:6818/1606106342 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v114'549 uv549 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f25b800d960 con 0x55d730aee1a0 2024-09-21T03:42:10.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.020+0000 7f25c4979f40 1 -- 172.21.15.38:0/2350447563 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:15 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55d730af76f0 con 0x55d730aee1a0 2024-09-21T03:42:10.023 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-09-21T03:42:10.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.021+0000 7f25bec34640 1 -- 172.21.15.38:0/2350447563 <== osd.7 v2:172.21.15.195:6818/1606106342 15 ==== osd_op_reply(15 10000000000.00000000 [create] v114'550 uv550 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f25b800d960 con 0x55d730aee1a0 2024-09-21T03:42:10.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.021+0000 7f25c4979f40 1 -- 172.21.15.38:0/2350447563 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:16 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55d730af76f0 con 0x55d730aee1a0 2024-09-21T03:42:10.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.022+0000 7f25bec34640 1 -- 172.21.15.38:0/2350447563 <== osd.7 v2:172.21.15.195:6818/1606106342 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v114'551 uv551 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f25b800d960 con 0x55d730aee1a0 2024-09-21T03:42:10.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.023+0000 7f25c4979f40 1 -- 172.21.15.38:0/2350447563 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:17 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55d730af76f0 con 0x55d730aee1a0 2024-09-21T03:42:10.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.024+0000 7f25bec34640 1 -- 172.21.15.38:0/2350447563 <== osd.7 v2:172.21.15.195:6818/1606106342 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v114'552 uv552 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f25b800d960 con 0x55d730aee1a0 2024-09-21T03:42:10.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.024+0000 7f25c4979f40 1 -- 172.21.15.38:0/2350447563 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:18 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55d730af76f0 con 0x55d730aee1a0 2024-09-21T03:42:10.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.025+0000 7f25bec34640 1 -- 172.21.15.38:0/2350447563 <== osd.7 v2:172.21.15.195:6818/1606106342 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v114'553 uv553 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f25b800d960 con 0x55d730aee1a0 2024-09-21T03:42:10.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.026+0000 7f25c4979f40 1 -- 172.21.15.38:0/2350447563 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:19 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=585b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55d730af8310 con 0x55d730aee1a0 2024-09-21T03:42:10.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.027+0000 7f25bec34640 1 -- 172.21.15.38:0/2350447563 <== osd.7 v2:172.21.15.195:6818/1606106342 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v114'554 uv554 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f25b800d960 con 0x55d730aee1a0 2024-09-21T03:42:10.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.027+0000 7f25c4979f40 1 -- 172.21.15.38:0/2350447563 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:20 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55d730af8310 con 0x55d730aee1a0 2024-09-21T03:42:10.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.029+0000 7f25bec34640 1 -- 172.21.15.38:0/2350447563 <== osd.7 v2:172.21.15.195:6818/1606106342 20 ==== osd_op_reply(20 600.00000000 [create] v114'555 uv529 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f25b800d960 con 0x55d730aee1a0 2024-09-21T03:42:10.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.029+0000 7f25c4979f40 1 -- 172.21.15.38:0/2350447563 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:21 13.10 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e114) -- 0x55d730af8310 con 0x55d730aee1a0 2024-09-21T03:42:10.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.030+0000 7f25bec34640 1 -- 172.21.15.38:0/2350447563 <== osd.7 v2:172.21.15.195:6818/1606106342 21 ==== osd_op_reply(21 600.00000000 [delete] v114'556 uv556 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f25b800d960 con 0x55d730aee1a0 2024-09-21T03:42:10.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.030+0000 7f25c4979f40 1 -- 172.21.15.38:0/2350447563 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:22 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55d730af8310 con 0x55d730aee1a0 2024-09-21T03:42:10.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.031+0000 7f25bec34640 1 -- 172.21.15.38:0/2350447563 <== osd.7 v2:172.21.15.195:6818/1606106342 22 ==== osd_op_reply(22 600.00000000 [create] v114'557 uv557 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f25b800d960 con 0x55d730aee1a0 2024-09-21T03:42:10.033 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-09-21T03:42:10.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.031+0000 7f25c4979f40 1 -- 172.21.15.38:0/2350447563 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:23 13.10 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55d730af9320 con 0x55d730aee1a0 2024-09-21T03:42:10.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.032+0000 7f25bec34640 1 -- 172.21.15.38:0/2350447563 <== osd.7 v2:172.21.15.195:6818/1606106342 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v114'558 uv558 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f25b800d960 con 0x55d730aee1a0 2024-09-21T03:42:10.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.032+0000 7f25c4979f40 1 -- 172.21.15.38:0/2350447563 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:24 13.10 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55d730af9320 con 0x55d730aee1a0 2024-09-21T03:42:10.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.034+0000 7f25bec34640 1 -- 172.21.15.38:0/2350447563 <== osd.7 v2:172.21.15.195:6818/1606106342 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v114'559 uv559 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f25b800d960 con 0x55d730aee1a0 2024-09-21T03:42:10.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.034+0000 7f25c4979f40 1 -- 172.21.15.38:0/2350447563 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:25 13.10 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55d730af8310 con 0x55d730aee1a0 2024-09-21T03:42:10.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.035+0000 7f25bec34640 1 -- 172.21.15.38:0/2350447563 <== osd.7 v2:172.21.15.195:6818/1606106342 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v114'560 uv560 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f25b800d960 con 0x55d730aee1a0 2024-09-21T03:42:10.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.035+0000 7f25c4979f40 1 -- 172.21.15.38:0/2350447563 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:26 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55d730af8310 con 0x55d730aee1a0 2024-09-21T03:42:10.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.037+0000 7f25bec34640 1 -- 172.21.15.38:0/2350447563 <== osd.7 v2:172.21.15.195:6818/1606106342 26 ==== osd_op_reply(26 603.00000000 [create] v114'561 uv535 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f25b800d960 con 0x55d730aee1a0 2024-09-21T03:42:10.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.037+0000 7f25c4979f40 1 -- 172.21.15.38:0/2350447563 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:27 13.10 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e114) -- 0x55d730af8310 con 0x55d730aee1a0 2024-09-21T03:42:10.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.038+0000 7f25bec34640 1 -- 172.21.15.38:0/2350447563 <== osd.7 v2:172.21.15.195:6818/1606106342 27 ==== osd_op_reply(27 603.00000000 [delete] v114'562 uv562 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f25b800d960 con 0x55d730aee1a0 2024-09-21T03:42:10.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.038+0000 7f25c4979f40 1 -- 172.21.15.38:0/2350447563 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:28 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55d730af8310 con 0x55d730aee1a0 2024-09-21T03:42:10.041 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-09-21T03:42:10.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.039+0000 7f25bec34640 1 -- 172.21.15.38:0/2350447563 <== osd.7 v2:172.21.15.195:6818/1606106342 28 ==== osd_op_reply(28 603.00000000 [create] v114'563 uv563 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f25b800d960 con 0x55d730aee1a0 2024-09-21T03:42:10.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.040+0000 7f25c4979f40 1 -- 172.21.15.38:0/2350447563 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:29 13.10 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55d730af9320 con 0x55d730aee1a0 2024-09-21T03:42:10.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.041+0000 7f25bec34640 1 -- 172.21.15.38:0/2350447563 <== osd.7 v2:172.21.15.195:6818/1606106342 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v114'564 uv564 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f25b800d960 con 0x55d730aee1a0 2024-09-21T03:42:10.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.041+0000 7f25c4979f40 1 -- 172.21.15.38:0/2350447563 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:30 13.10 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55d730af9320 con 0x55d730aee1a0 2024-09-21T03:42:10.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.042+0000 7f25bec34640 1 -- 172.21.15.38:0/2350447563 <== osd.7 v2:172.21.15.195:6818/1606106342 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v114'565 uv565 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f25b800d960 con 0x55d730aee1a0 2024-09-21T03:42:10.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.042+0000 7f25c4979f40 1 -- 172.21.15.38:0/2350447563 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:31 13.10 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55d730af8310 con 0x55d730aee1a0 2024-09-21T03:42:10.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.044+0000 7f25bec34640 1 -- 172.21.15.38:0/2350447563 <== osd.7 v2:172.21.15.195:6818/1606106342 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v114'566 uv566 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f25b800d960 con 0x55d730aee1a0 2024-09-21T03:42:10.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.044+0000 7f25c4979f40 1 --2- 172.21.15.38:0/2350447563 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x55d730af8820 0x55d730afac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:10.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.044+0000 7f25c4979f40 1 -- 172.21.15.38:0/2350447563 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55d730afb180 con 0x55d730af8820 2024-09-21T03:42:10.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.044+0000 7f25be433640 1 --2- 172.21.15.38:0/2350447563 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x55d730af8820 0x55d730afac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:10.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.045+0000 7f25be433640 1 --2- 172.21.15.38:0/2350447563 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x55d730af8820 0x55d730afac40 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:10.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.046+0000 7f25be433640 1 -- 172.21.15.38:0/2350447563 <== osd.6 v2:172.21.15.195:6826/4055825675 1 ==== osd_op_reply(32 1.00000000.inode [create] v114'395 uv377 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f25b00680b0 con 0x55d730af8820 2024-09-21T03:42:10.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.046+0000 7f25c4979f40 1 -- 172.21.15.38:0/2350447563 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e114) -- 0x55d730afb520 con 0x55d730af8820 2024-09-21T03:42:10.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.048+0000 7f25be433640 1 -- 172.21.15.38:0/2350447563 <== osd.6 v2:172.21.15.195:6826/4055825675 2 ==== osd_op_reply(33 1.00000000.inode [delete] v114'396 uv396 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f25b00680b0 con 0x55d730af8820 2024-09-21T03:42:10.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.048+0000 7f25c4979f40 1 -- 172.21.15.38:0/2350447563 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55d730afb8c0 con 0x55d730af8820 2024-09-21T03:42:10.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.050+0000 7f25be433640 1 -- 172.21.15.38:0/2350447563 <== osd.6 v2:172.21.15.195:6826/4055825675 3 ==== osd_op_reply(34 1.00000000.inode [create] v114'397 uv397 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f25b00680b0 con 0x55d730af8820 2024-09-21T03:42:10.052 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-09-21T03:42:10.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.050+0000 7f25c4979f40 1 -- 172.21.15.38:0/2350447563 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55d730a52990 con 0x55d730af8820 2024-09-21T03:42:10.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.051+0000 7f25be433640 1 -- 172.21.15.38:0/2350447563 <== osd.6 v2:172.21.15.195:6826/4055825675 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v114'398 uv398 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f25b00680b0 con 0x55d730af8820 2024-09-21T03:42:10.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.051+0000 7f25c4979f40 1 -- 172.21.15.38:0/2350447563 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55d730afc110 con 0x55d730af8820 2024-09-21T03:42:10.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.053+0000 7f25be433640 1 -- 172.21.15.38:0/2350447563 <== osd.6 v2:172.21.15.195:6826/4055825675 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v114'399 uv399 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f25b00680b0 con 0x55d730af8820 2024-09-21T03:42:10.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.053+0000 7f25c4979f40 1 -- 172.21.15.38:0/2350447563 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55d730afc4b0 con 0x55d730af8820 2024-09-21T03:42:10.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.054+0000 7f25be433640 1 -- 172.21.15.38:0/2350447563 <== osd.6 v2:172.21.15.195:6826/4055825675 6 ==== osd_op_reply(37 200.00000000 [create] v114'400 uv382 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f25b00680b0 con 0x55d730af8820 2024-09-21T03:42:10.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.054+0000 7f25c4979f40 1 -- 172.21.15.38:0/2350447563 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e114) -- 0x55d730afc850 con 0x55d730af8820 2024-09-21T03:42:10.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.055+0000 7f25be433640 1 -- 172.21.15.38:0/2350447563 <== osd.6 v2:172.21.15.195:6826/4055825675 7 ==== osd_op_reply(38 200.00000000 [delete] v114'401 uv401 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f25b00680b0 con 0x55d730af8820 2024-09-21T03:42:10.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.055+0000 7f25c4979f40 1 -- 172.21.15.38:0/2350447563 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55d730afcbf0 con 0x55d730af8820 2024-09-21T03:42:10.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.057+0000 7f25be433640 1 -- 172.21.15.38:0/2350447563 <== osd.6 v2:172.21.15.195:6826/4055825675 8 ==== osd_op_reply(39 200.00000000 [create] v114'402 uv402 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f25b00680b0 con 0x55d730af8820 2024-09-21T03:42:10.059 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-09-21T03:42:10.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.057+0000 7f25c4979f40 1 -- 172.21.15.38:0/2350447563 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55d730afcbf0 con 0x55d730af8820 2024-09-21T03:42:10.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.058+0000 7f25be433640 1 -- 172.21.15.38:0/2350447563 <== osd.6 v2:172.21.15.195:6826/4055825675 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v114'403 uv403 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f25b00680b0 con 0x55d730af8820 2024-09-21T03:42:10.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.058+0000 7f25c4979f40 1 -- 172.21.15.38:0/2350447563 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55d730afcbf0 con 0x55d730af8820 2024-09-21T03:42:10.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.059+0000 7f25be433640 1 -- 172.21.15.38:0/2350447563 <== osd.6 v2:172.21.15.195:6826/4055825675 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v114'404 uv404 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f25b00680b0 con 0x55d730af8820 2024-09-21T03:42:10.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.059+0000 7f25c4979f40 1 -- 172.21.15.38:0/2350447563 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:42 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55d730afcbf0 con 0x55d730af8820 2024-09-21T03:42:10.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.061+0000 7f25be433640 1 -- 172.21.15.38:0/2350447563 <== osd.6 v2:172.21.15.195:6826/4055825675 11 ==== osd_op_reply(42 200.00000001 [create] v114'405 uv394 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f25b00680b0 con 0x55d730af8820 2024-09-21T03:42:10.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.061+0000 7f25c4979f40 1 -- 172.21.15.38:0/2350447563 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:43 13.14 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e114) -- 0x55d730afcbf0 con 0x55d730af8820 2024-09-21T03:42:10.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.062+0000 7f25be433640 1 -- 172.21.15.38:0/2350447563 <== osd.6 v2:172.21.15.195:6826/4055825675 12 ==== osd_op_reply(43 200.00000001 [delete] v114'406 uv406 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f25b00680b0 con 0x55d730af8820 2024-09-21T03:42:10.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.062+0000 7f25c4979f40 1 -- 172.21.15.38:0/2350447563 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:44 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55d730afcbf0 con 0x55d730af8820 2024-09-21T03:42:10.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.063+0000 7f25be433640 1 -- 172.21.15.38:0/2350447563 <== osd.6 v2:172.21.15.195:6826/4055825675 13 ==== osd_op_reply(44 200.00000001 [create] v114'407 uv407 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f25b00680b0 con 0x55d730af8820 2024-09-21T03:42:10.065 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-09-21T03:42:10.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.063+0000 7f25c4979f40 1 -- 172.21.15.38:0/2350447563 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:45 13.14 13:2e2fa760:::200.00000001:head [write 0~17490 in=17490b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55d730b01070 con 0x55d730af8820 2024-09-21T03:42:10.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.065+0000 7f25be433640 1 -- 172.21.15.38:0/2350447563 <== osd.6 v2:172.21.15.195:6826/4055825675 14 ==== osd_op_reply(45 200.00000001 [write 0~17490] v114'408 uv408 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f25b00680b0 con 0x55d730af8820 2024-09-21T03:42:10.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.065+0000 7f25c4979f40 1 -- 172.21.15.38:0/2350447563 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:46 13.14 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55d730b01070 con 0x55d730af8820 2024-09-21T03:42:10.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.066+0000 7f25be433640 1 -- 172.21.15.38:0/2350447563 <== osd.6 v2:172.21.15.195:6826/4055825675 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v114'409 uv409 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f25b00680b0 con 0x55d730af8820 2024-09-21T03:42:10.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.066+0000 7f25c4979f40 1 -- 172.21.15.38:0/2350447563 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:47 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55d730afcbf0 con 0x55d730af8820 2024-09-21T03:42:10.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.068+0000 7f25be433640 1 -- 172.21.15.38:0/2350447563 <== osd.6 v2:172.21.15.195:6826/4055825675 16 ==== osd_op_reply(47 606.00000000 [create] v114'410 uv393 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f25b00680b0 con 0x55d730af8820 2024-09-21T03:42:10.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.068+0000 7f25c4979f40 1 -- 172.21.15.38:0/2350447563 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:48 13.14 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e114) -- 0x55d730afcbf0 con 0x55d730af8820 2024-09-21T03:42:10.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.069+0000 7f25be433640 1 -- 172.21.15.38:0/2350447563 <== osd.6 v2:172.21.15.195:6826/4055825675 17 ==== osd_op_reply(48 606.00000000 [delete] v114'411 uv411 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f25b00680b0 con 0x55d730af8820 2024-09-21T03:42:10.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.069+0000 7f25c4979f40 1 -- 172.21.15.38:0/2350447563 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:49 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55d730afcbf0 con 0x55d730af8820 2024-09-21T03:42:10.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.070+0000 7f25be433640 1 -- 172.21.15.38:0/2350447563 <== osd.6 v2:172.21.15.195:6826/4055825675 18 ==== osd_op_reply(49 606.00000000 [create] v114'412 uv412 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f25b00680b0 con 0x55d730af8820 2024-09-21T03:42:10.072 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-09-21T03:42:10.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.070+0000 7f25c4979f40 1 -- 172.21.15.38:0/2350447563 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:50 13.14 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55d730afdc00 con 0x55d730af8820 2024-09-21T03:42:10.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.071+0000 7f25be433640 1 -- 172.21.15.38:0/2350447563 <== osd.6 v2:172.21.15.195:6826/4055825675 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v114'413 uv413 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f25b00680b0 con 0x55d730af8820 2024-09-21T03:42:10.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.071+0000 7f25c4979f40 1 -- 172.21.15.38:0/2350447563 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:51 13.14 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55d730afdc00 con 0x55d730af8820 2024-09-21T03:42:10.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.073+0000 7f25be433640 1 -- 172.21.15.38:0/2350447563 <== osd.6 v2:172.21.15.195:6826/4055825675 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v114'414 uv414 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f25b00680b0 con 0x55d730af8820 2024-09-21T03:42:10.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.073+0000 7f25c4979f40 1 -- 172.21.15.38:0/2350447563 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:52 13.14 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55d730afcbf0 con 0x55d730af8820 2024-09-21T03:42:10.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.074+0000 7f25be433640 1 -- 172.21.15.38:0/2350447563 <== osd.6 v2:172.21.15.195:6826/4055825675 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v114'415 uv415 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f25b00680b0 con 0x55d730af8820 2024-09-21T03:42:10.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.074+0000 7f25c4979f40 1 --2- 172.21.15.38:0/2350447563 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x55d730afd270 0x55d730aff690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:10.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.074+0000 7f25c4979f40 1 -- 172.21.15.38:0/2350447563 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:53 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55d730affc60 con 0x55d730afd270 2024-09-21T03:42:10.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.075+0000 7f25bdc32640 1 --2- 172.21.15.38:0/2350447563 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x55d730afd270 0x55d730aff690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:10.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.075+0000 7f25bdc32640 1 --2- 172.21.15.38:0/2350447563 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x55d730afd270 0x55d730aff690 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:10.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.076+0000 7f25bdc32640 1 -- 172.21.15.38:0/2350447563 <== osd.0 v2:172.21.15.38:6800/2381639433 1 ==== osd_op_reply(53 607.00000000 [create] v114'105 uv104 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f25b40564f0 con 0x55d730afd270 2024-09-21T03:42:10.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.076+0000 7f25c4979f40 1 -- 172.21.15.38:0/2350447563 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:54 13.1c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e114) -- 0x55d730b00150 con 0x55d730afd270 2024-09-21T03:42:10.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.078+0000 7f25bdc32640 1 -- 172.21.15.38:0/2350447563 <== osd.0 v2:172.21.15.38:6800/2381639433 2 ==== osd_op_reply(54 607.00000000 [delete] v114'106 uv106 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f25b40564f0 con 0x55d730afd270 2024-09-21T03:42:10.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.078+0000 7f25c4979f40 1 -- 172.21.15.38:0/2350447563 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:55 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55d730b00630 con 0x55d730afd270 2024-09-21T03:42:10.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.079+0000 7f25bdc32640 1 -- 172.21.15.38:0/2350447563 <== osd.0 v2:172.21.15.38:6800/2381639433 3 ==== osd_op_reply(55 607.00000000 [create] v114'107 uv107 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f25b40564f0 con 0x55d730afd270 2024-09-21T03:42:10.080 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-09-21T03:42:10.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.079+0000 7f25c4979f40 1 -- 172.21.15.38:0/2350447563 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:56 13.1c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55d730b01ee0 con 0x55d730afd270 2024-09-21T03:42:10.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.080+0000 7f25bdc32640 1 -- 172.21.15.38:0/2350447563 <== osd.0 v2:172.21.15.38:6800/2381639433 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v114'108 uv108 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f25b40564f0 con 0x55d730afd270 2024-09-21T03:42:10.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.080+0000 7f25c4979f40 1 -- 172.21.15.38:0/2350447563 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:57 13.1c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55d730b02280 con 0x55d730afd270 2024-09-21T03:42:10.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.081+0000 7f25bdc32640 1 -- 172.21.15.38:0/2350447563 <== osd.0 v2:172.21.15.38:6800/2381639433 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v114'109 uv109 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f25b40564f0 con 0x55d730afd270 2024-09-21T03:42:10.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.081+0000 7f25c4979f40 1 -- 172.21.15.38:0/2350447563 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:58 13.1c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55d730b00ed0 con 0x55d730afd270 2024-09-21T03:42:10.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.082+0000 7f25bdc32640 1 -- 172.21.15.38:0/2350447563 <== osd.0 v2:172.21.15.38:6800/2381639433 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v114'110 uv110 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f25b40564f0 con 0x55d730afd270 2024-09-21T03:42:10.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.082+0000 7f25c4979f40 1 --2- 172.21.15.38:0/2350447563 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x55d730b017d0 0x55d730b04630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:10.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.082+0000 7f25c4979f40 1 -- 172.21.15.38:0/2350447563 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:59 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55d730b04b70 con 0x55d730b017d0 2024-09-21T03:42:10.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.082+0000 7f25bec34640 1 --2- 172.21.15.38:0/2350447563 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x55d730b017d0 0x55d730b04630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:10.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.082+0000 7f25bec34640 1 --2- 172.21.15.38:0/2350447563 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x55d730b017d0 0x55d730b04630 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:10.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.083+0000 7f25bec34640 1 -- 172.21.15.38:0/2350447563 <== osd.3 v2:172.21.15.38:6808/1286810294 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v114'78 uv77 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f25b800d960 con 0x55d730b017d0 2024-09-21T03:42:10.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.083+0000 7f25c4979f40 1 -- 172.21.15.38:0/2350447563 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:60 13.6 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e114) -- 0x55d730b04b70 con 0x55d730b017d0 2024-09-21T03:42:10.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.084+0000 7f25bec34640 1 -- 172.21.15.38:0/2350447563 <== osd.3 v2:172.21.15.38:6808/1286810294 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v114'79 uv79 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f25b800d960 con 0x55d730b017d0 2024-09-21T03:42:10.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.085+0000 7f25c4979f40 1 -- 172.21.15.38:0/2350447563 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:61 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55d730b04bf0 con 0x55d730b017d0 2024-09-21T03:42:10.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.086+0000 7f25bec34640 1 -- 172.21.15.38:0/2350447563 <== osd.3 v2:172.21.15.38:6808/1286810294 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v114'80 uv80 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f25b800d960 con 0x55d730b017d0 2024-09-21T03:42:10.088 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-09-21T03:42:10.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.086+0000 7f25c4979f40 1 -- 172.21.15.38:0/2350447563 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:62 13.6 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55d730b04fe0 con 0x55d730b017d0 2024-09-21T03:42:10.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.087+0000 7f25bec34640 1 -- 172.21.15.38:0/2350447563 <== osd.3 v2:172.21.15.38:6808/1286810294 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v114'81 uv81 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f25b800d960 con 0x55d730b017d0 2024-09-21T03:42:10.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.087+0000 7f25c4979f40 1 -- 172.21.15.38:0/2350447563 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55d730b04fe0 con 0x55d730afd270 2024-09-21T03:42:10.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.088+0000 7f25bdc32640 1 -- 172.21.15.38:0/2350447563 <== osd.0 v2:172.21.15.38:6800/2381639433 7 ==== osd_op_reply(63 608.00000000 [create] v114'105 uv104 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f25b40564f0 con 0x55d730afd270 2024-09-21T03:42:10.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.088+0000 7f25c4979f40 1 -- 172.21.15.38:0/2350447563 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e114) -- 0x55d730b04fe0 con 0x55d730afd270 2024-09-21T03:42:10.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.089+0000 7f25bdc32640 1 -- 172.21.15.38:0/2350447563 <== osd.0 v2:172.21.15.38:6800/2381639433 8 ==== osd_op_reply(64 608.00000000 [delete] v114'106 uv106 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f25b40564f0 con 0x55d730afd270 2024-09-21T03:42:10.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.089+0000 7f25c4979f40 1 -- 172.21.15.38:0/2350447563 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55d730b04fe0 con 0x55d730afd270 2024-09-21T03:42:10.092 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-09-21T03:42:10.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.090+0000 7f25bdc32640 1 -- 172.21.15.38:0/2350447563 <== osd.0 v2:172.21.15.38:6800/2381639433 9 ==== osd_op_reply(65 608.00000000 [create] v114'107 uv107 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f25b40564f0 con 0x55d730afd270 2024-09-21T03:42:10.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.090+0000 7f25c4979f40 1 -- 172.21.15.38:0/2350447563 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55d730b05ff0 con 0x55d730afd270 2024-09-21T03:42:10.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.092+0000 7f25bdc32640 1 -- 172.21.15.38:0/2350447563 <== osd.0 v2:172.21.15.38:6800/2381639433 10 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v114'108 uv108 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f25b40564f0 con 0x55d730afd270 2024-09-21T03:42:10.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.092+0000 7f25c4979f40 1 -- 172.21.15.38:0/2350447563 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55d730b05ff0 con 0x55d730afd270 2024-09-21T03:42:10.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.093+0000 7f25bdc32640 1 -- 172.21.15.38:0/2350447563 <== osd.0 v2:172.21.15.38:6800/2381639433 11 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v114'109 uv109 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f25b40564f0 con 0x55d730afd270 2024-09-21T03:42:10.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.093+0000 7f25c4979f40 1 -- 172.21.15.38:0/2350447563 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55d730b04fe0 con 0x55d730afd270 2024-09-21T03:42:10.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.094+0000 7f25bdc32640 1 -- 172.21.15.38:0/2350447563 <== osd.0 v2:172.21.15.38:6800/2381639433 12 ==== osd_op_reply(68 608.00000000 [omap-set-header] v114'110 uv110 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f25b40564f0 con 0x55d730afd270 2024-09-21T03:42:10.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.094+0000 7f25c4979f40 1 -- 172.21.15.38:0/2350447563 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55d730b04fe0 con 0x55d730b017d0 2024-09-21T03:42:10.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.095+0000 7f25bec34640 1 -- 172.21.15.38:0/2350447563 <== osd.3 v2:172.21.15.38:6808/1286810294 5 ==== osd_op_reply(69 604.00000000 [create] v114'193 uv187 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f25b800d960 con 0x55d730b017d0 2024-09-21T03:42:10.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.095+0000 7f25c4979f40 1 -- 172.21.15.38:0/2350447563 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e114) -- 0x55d730b04fe0 con 0x55d730b017d0 2024-09-21T03:42:10.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.096+0000 7f25bec34640 1 -- 172.21.15.38:0/2350447563 <== osd.3 v2:172.21.15.38:6808/1286810294 6 ==== osd_op_reply(70 604.00000000 [delete] v114'194 uv194 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f25b800d960 con 0x55d730b017d0 2024-09-21T03:42:10.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.096+0000 7f25c4979f40 1 -- 172.21.15.38:0/2350447563 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55d730b04fe0 con 0x55d730b017d0 2024-09-21T03:42:10.099 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-09-21T03:42:10.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.097+0000 7f25bec34640 1 -- 172.21.15.38:0/2350447563 <== osd.3 v2:172.21.15.38:6808/1286810294 7 ==== osd_op_reply(71 604.00000000 [create] v114'195 uv195 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f25b800d960 con 0x55d730b017d0 2024-09-21T03:42:10.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.097+0000 7f25c4979f40 1 -- 172.21.15.38:0/2350447563 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55d730b05ff0 con 0x55d730b017d0 2024-09-21T03:42:10.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.098+0000 7f25bec34640 1 -- 172.21.15.38:0/2350447563 <== osd.3 v2:172.21.15.38:6808/1286810294 8 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v114'196 uv196 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f25b800d960 con 0x55d730b017d0 2024-09-21T03:42:10.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.098+0000 7f25c4979f40 1 -- 172.21.15.38:0/2350447563 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55d730b05ff0 con 0x55d730b017d0 2024-09-21T03:42:10.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.099+0000 7f25bec34640 1 -- 172.21.15.38:0/2350447563 <== osd.3 v2:172.21.15.38:6808/1286810294 9 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v114'197 uv197 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f25b800d960 con 0x55d730b017d0 2024-09-21T03:42:10.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.099+0000 7f25c4979f40 1 -- 172.21.15.38:0/2350447563 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55d730b04fe0 con 0x55d730b017d0 2024-09-21T03:42:10.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.101+0000 7f25bec34640 1 -- 172.21.15.38:0/2350447563 <== osd.3 v2:172.21.15.38:6808/1286810294 10 ==== osd_op_reply(74 604.00000000 [omap-set-header] v114'198 uv198 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f25b800d960 con 0x55d730b017d0 2024-09-21T03:42:10.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.101+0000 7f25c4979f40 1 -- 172.21.15.38:0/2350447563 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55d730b04fe0 con 0x55d730b017d0 2024-09-21T03:42:10.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.102+0000 7f25bec34640 1 -- 172.21.15.38:0/2350447563 <== osd.3 v2:172.21.15.38:6808/1286810294 11 ==== osd_op_reply(75 500.00000000 [create] v114'199 uv192 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f25b800d960 con 0x55d730b017d0 2024-09-21T03:42:10.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.102+0000 7f25c4979f40 1 -- 172.21.15.38:0/2350447563 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e114) -- 0x55d730b04fe0 con 0x55d730b017d0 2024-09-21T03:42:10.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.103+0000 7f25bec34640 1 -- 172.21.15.38:0/2350447563 <== osd.3 v2:172.21.15.38:6808/1286810294 12 ==== osd_op_reply(76 500.00000000 [delete] v114'200 uv200 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f25b800d960 con 0x55d730b017d0 2024-09-21T03:42:10.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.103+0000 7f25c4979f40 1 -- 172.21.15.38:0/2350447563 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55d730b04fe0 con 0x55d730b017d0 2024-09-21T03:42:10.106 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-09-21T03:42:10.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.104+0000 7f25bec34640 1 -- 172.21.15.38:0/2350447563 <== osd.3 v2:172.21.15.38:6808/1286810294 13 ==== osd_op_reply(77 500.00000000 [create] v114'201 uv201 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f25b800d960 con 0x55d730b017d0 2024-09-21T03:42:10.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.104+0000 7f25c4979f40 1 -- 172.21.15.38:0/2350447563 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55d730b04fe0 con 0x55d730b017d0 2024-09-21T03:42:10.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.105+0000 7f25bec34640 1 -- 172.21.15.38:0/2350447563 <== osd.3 v2:172.21.15.38:6808/1286810294 14 ==== osd_op_reply(78 500.00000000 [write 0~90] v114'202 uv202 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f25b800d960 con 0x55d730b017d0 2024-09-21T03:42:10.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.105+0000 7f25c4979f40 1 -- 172.21.15.38:0/2350447563 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55d730b04fe0 con 0x55d730b017d0 2024-09-21T03:42:10.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.106+0000 7f25bec34640 1 -- 172.21.15.38:0/2350447563 <== osd.3 v2:172.21.15.38:6808/1286810294 15 ==== osd_op_reply(79 500.00000000 [omap-set-header] v114'203 uv203 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f25b800d960 con 0x55d730b017d0 2024-09-21T03:42:10.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.106+0000 7f25c4979f40 1 -- 172.21.15.38:0/2350447563 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:80 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55d730b04fe0 con 0x55d730afd270 2024-09-21T03:42:10.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.107+0000 7f25bdc32640 1 -- 172.21.15.38:0/2350447563 <== osd.0 v2:172.21.15.38:6800/2381639433 13 ==== osd_op_reply(80 mds_snaptable [create] v114'86 uv85 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f25b40564f0 con 0x55d730afd270 2024-09-21T03:42:10.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.107+0000 7f25c4979f40 1 -- 172.21.15.38:0/2350447563 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:81 13.d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e114) -- 0x55d730b04fe0 con 0x55d730afd270 2024-09-21T03:42:10.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.109+0000 7f25bdc32640 1 -- 172.21.15.38:0/2350447563 <== osd.0 v2:172.21.15.38:6800/2381639433 14 ==== osd_op_reply(81 mds_snaptable [delete] v114'87 uv87 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f25b40564f0 con 0x55d730afd270 2024-09-21T03:42:10.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.109+0000 7f25c4979f40 1 -- 172.21.15.38:0/2350447563 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:82 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55d730b04fe0 con 0x55d730afd270 2024-09-21T03:42:10.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.110+0000 7f25bdc32640 1 -- 172.21.15.38:0/2350447563 <== osd.0 v2:172.21.15.38:6800/2381639433 15 ==== osd_op_reply(82 mds_snaptable [create] v114'88 uv88 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f25b40564f0 con 0x55d730afd270 2024-09-21T03:42:10.112 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-09-21T03:42:10.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.110+0000 7f25c4979f40 1 -- 172.21.15.38:0/2350447563 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:83 13.d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55d730b04fe0 con 0x55d730afd270 2024-09-21T03:42:10.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.111+0000 7f25bdc32640 1 -- 172.21.15.38:0/2350447563 <== osd.0 v2:172.21.15.38:6800/2381639433 16 ==== osd_op_reply(83 mds_snaptable [write 0~70] v114'89 uv89 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f25b40564f0 con 0x55d730afd270 2024-09-21T03:42:10.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.111+0000 7f25c4979f40 1 -- 172.21.15.38:0/2350447563 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:84 13.d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55d730b04fe0 con 0x55d730afd270 2024-09-21T03:42:10.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.113+0000 7f25bdc32640 1 -- 172.21.15.38:0/2350447563 <== osd.0 v2:172.21.15.38:6800/2381639433 17 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v114'90 uv90 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f25b40564f0 con 0x55d730afd270 2024-09-21T03:42:10.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.113+0000 7f25c4979f40 1 --2- 172.21.15.38:0/2350447563 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x55d730b05660 0x55d730b07a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:10.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.113+0000 7f25c4979f40 1 -- 172.21.15.38:0/2350447563 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55d730b08130 con 0x55d730b05660 2024-09-21T03:42:10.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.113+0000 7f25be433640 1 --2- 172.21.15.38:0/2350447563 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x55d730b05660 0x55d730b07a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:10.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.113+0000 7f25be433640 1 --2- 172.21.15.38:0/2350447563 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x55d730b05660 0x55d730b07a80 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:10.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.115+0000 7f25be433640 1 -- 172.21.15.38:0/2350447563 <== osd.1 v2:172.21.15.38:6816/1491818732 1 ==== osd_op_reply(85 605.00000000 [create] v114'105 uv104 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f25b00680b0 con 0x55d730b05660 2024-09-21T03:42:10.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.115+0000 7f25c4979f40 1 -- 172.21.15.38:0/2350447563 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e114) -- 0x55d730b08500 con 0x55d730b05660 2024-09-21T03:42:10.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.116+0000 7f25be433640 1 -- 172.21.15.38:0/2350447563 <== osd.1 v2:172.21.15.38:6816/1491818732 2 ==== osd_op_reply(86 605.00000000 [delete] v114'106 uv106 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f25b00680b0 con 0x55d730b05660 2024-09-21T03:42:10.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.116+0000 7f25c4979f40 1 -- 172.21.15.38:0/2350447563 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55d730b08500 con 0x55d730b05660 2024-09-21T03:42:10.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.117+0000 7f25be433640 1 -- 172.21.15.38:0/2350447563 <== osd.1 v2:172.21.15.38:6816/1491818732 3 ==== osd_op_reply(87 605.00000000 [create] v114'107 uv107 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f25b00680b0 con 0x55d730b05660 2024-09-21T03:42:10.119 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-09-21T03:42:10.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.117+0000 7f25c4979f40 1 -- 172.21.15.38:0/2350447563 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55d730b09510 con 0x55d730b05660 2024-09-21T03:42:10.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.118+0000 7f25be433640 1 -- 172.21.15.38:0/2350447563 <== osd.1 v2:172.21.15.38:6816/1491818732 4 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v114'108 uv108 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f25b00680b0 con 0x55d730b05660 2024-09-21T03:42:10.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.118+0000 7f25c4979f40 1 -- 172.21.15.38:0/2350447563 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55d730b09510 con 0x55d730b05660 2024-09-21T03:42:10.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.119+0000 7f25be433640 1 -- 172.21.15.38:0/2350447563 <== osd.1 v2:172.21.15.38:6816/1491818732 5 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v114'109 uv109 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f25b00680b0 con 0x55d730b05660 2024-09-21T03:42:10.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.119+0000 7f25c4979f40 1 -- 172.21.15.38:0/2350447563 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55d730b08500 con 0x55d730b05660 2024-09-21T03:42:10.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.120+0000 7f25be433640 1 -- 172.21.15.38:0/2350447563 <== osd.1 v2:172.21.15.38:6816/1491818732 6 ==== osd_op_reply(90 605.00000000 [omap-set-header] v114'110 uv110 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f25b00680b0 con 0x55d730b05660 2024-09-21T03:42:10.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.120+0000 7f25c4979f40 1 -- 172.21.15.38:0/2350447563 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55d730b08500 con 0x55d730b05660 2024-09-21T03:42:10.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.122+0000 7f25be433640 1 -- 172.21.15.38:0/2350447563 <== osd.1 v2:172.21.15.38:6816/1491818732 7 ==== osd_op_reply(91 mds0_inotable [create] v114'172 uv166 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f25b00680b0 con 0x55d730b05660 2024-09-21T03:42:10.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.122+0000 7f25c4979f40 1 -- 172.21.15.38:0/2350447563 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e114) -- 0x55d730b08500 con 0x55d730b05660 2024-09-21T03:42:10.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.123+0000 7f25be433640 1 -- 172.21.15.38:0/2350447563 <== osd.1 v2:172.21.15.38:6816/1491818732 8 ==== osd_op_reply(92 mds0_inotable [delete] v114'173 uv173 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f25b00680b0 con 0x55d730b05660 2024-09-21T03:42:10.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.123+0000 7f25c4979f40 1 -- 172.21.15.38:0/2350447563 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55d730b08500 con 0x55d730b05660 2024-09-21T03:42:10.126 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.124+0000 7f25be433640 1 -- 172.21.15.38:0/2350447563 <== osd.1 v2:172.21.15.38:6816/1491818732 9 ==== osd_op_reply(93 mds0_inotable [create] v114'174 uv174 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f25b00680b0 con 0x55d730b05660 2024-09-21T03:42:10.126 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-09-21T03:42:10.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.124+0000 7f25c4979f40 1 -- 172.21.15.38:0/2350447563 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:94 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55d730b085a0 con 0x55d730b05660 2024-09-21T03:42:10.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.125+0000 7f25be433640 1 -- 172.21.15.38:0/2350447563 <== osd.1 v2:172.21.15.38:6816/1491818732 10 ==== osd_op_reply(94 mds0_inotable [write 0~34] v114'175 uv175 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f25b00680b0 con 0x55d730b05660 2024-09-21T03:42:10.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.125+0000 7f25c4979f40 1 -- 172.21.15.38:0/2350447563 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:95 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55d730b085a0 con 0x55d730b05660 2024-09-21T03:42:10.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.126+0000 7f25be433640 1 -- 172.21.15.38:0/2350447563 <== osd.1 v2:172.21.15.38:6816/1491818732 11 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v114'176 uv176 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f25b00680b0 con 0x55d730b05660 2024-09-21T03:42:10.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.126+0000 7f25c4979f40 1 -- 172.21.15.38:0/2350447563 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:96 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55d730b085a0 con 0x55d730b05660 2024-09-21T03:42:10.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.128+0000 7f25be433640 1 -- 172.21.15.38:0/2350447563 <== osd.1 v2:172.21.15.38:6816/1491818732 12 ==== osd_op_reply(96 100.00000000 [create] v114'177 uv171 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f25b00680b0 con 0x55d730b05660 2024-09-21T03:42:10.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.128+0000 7f25c4979f40 1 -- 172.21.15.38:0/2350447563 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:97 13.13 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e114) -- 0x55d730b085a0 con 0x55d730b05660 2024-09-21T03:42:10.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.129+0000 7f25be433640 1 -- 172.21.15.38:0/2350447563 <== osd.1 v2:172.21.15.38:6816/1491818732 13 ==== osd_op_reply(97 100.00000000 [delete] v114'178 uv178 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f25b00680b0 con 0x55d730b05660 2024-09-21T03:42:10.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.129+0000 7f25c4979f40 1 -- 172.21.15.38:0/2350447563 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:98 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55d730b085a0 con 0x55d730b05660 2024-09-21T03:42:10.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.130+0000 7f25be433640 1 -- 172.21.15.38:0/2350447563 <== osd.1 v2:172.21.15.38:6816/1491818732 14 ==== osd_op_reply(98 100.00000000 [create] v114'179 uv179 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f25b00680b0 con 0x55d730b05660 2024-09-21T03:42:10.132 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-09-21T03:42:10.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.130+0000 7f25c4979f40 1 -- 172.21.15.38:0/2350447563 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:99 13.13 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55d730b085a0 con 0x55d730b05660 2024-09-21T03:42:10.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.131+0000 7f25be433640 1 -- 172.21.15.38:0/2350447563 <== osd.1 v2:172.21.15.38:6816/1491818732 15 ==== osd_op_reply(99 100.00000000 [omap-set-header] v114'180 uv180 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f25b00680b0 con 0x55d730b05660 2024-09-21T03:42:10.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.131+0000 7f25c4979f40 1 -- 172.21.15.38:0/2350447563 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:100 13.13 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55d730b0b3c0 con 0x55d730b05660 2024-09-21T03:42:10.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.132+0000 7f25be433640 1 -- 172.21.15.38:0/2350447563 <== osd.1 v2:172.21.15.38:6816/1491818732 16 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v114'181 uv181 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f25b00680b0 con 0x55d730b05660 2024-09-21T03:42:10.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.132+0000 7f25c4979f40 1 -- 172.21.15.38:0/2350447563 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55d730b0b380 con 0x55d730b017d0 2024-09-21T03:42:10.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.134+0000 7f25bec34640 1 -- 172.21.15.38:0/2350447563 <== osd.3 v2:172.21.15.38:6808/1286810294 16 ==== osd_op_reply(101 mds0_sessionmap [create] v114'78 uv77 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7f25b800d960 con 0x55d730b017d0 2024-09-21T03:42:10.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.134+0000 7f25c4979f40 1 -- 172.21.15.38:0/2350447563 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e114) -- 0x55d730b0b380 con 0x55d730b017d0 2024-09-21T03:42:10.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.135+0000 7f25bec34640 1 -- 172.21.15.38:0/2350447563 <== osd.3 v2:172.21.15.38:6808/1286810294 17 ==== osd_op_reply(102 mds0_sessionmap [delete] v114'79 uv79 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f25b800d960 con 0x55d730b017d0 2024-09-21T03:42:10.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.135+0000 7f25c4979f40 1 -- 172.21.15.38:0/2350447563 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:103 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55d730b0b380 con 0x55d730b017d0 2024-09-21T03:42:10.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.136+0000 7f25bec34640 1 -- 172.21.15.38:0/2350447563 <== osd.3 v2:172.21.15.38:6808/1286810294 18 ==== osd_op_reply(103 mds0_sessionmap [create] v114'80 uv80 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f25b800d960 con 0x55d730b017d0 2024-09-21T03:42:10.138 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-09-21T03:42:10.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.136+0000 7f25c4979f40 1 -- 172.21.15.38:0/2350447563 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:104 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55d730b0b380 con 0x55d730b017d0 2024-09-21T03:42:10.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.137+0000 7f25bec34640 1 -- 172.21.15.38:0/2350447563 <== osd.3 v2:172.21.15.38:6808/1286810294 19 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v114'81 uv81 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f25b800d960 con 0x55d730b017d0 2024-09-21T03:42:10.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.137+0000 7f25c4979f40 1 -- 172.21.15.38:0/2350447563 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55d730b089e0 con 0x55d730b017d0 2024-09-21T03:42:10.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.138+0000 7f25bec34640 1 -- 172.21.15.38:0/2350447563 <== osd.3 v2:172.21.15.38:6808/1286810294 20 ==== osd_op_reply(105 609.00000000 [create] v114'105 uv104 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f25b800d960 con 0x55d730b017d0 2024-09-21T03:42:10.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.138+0000 7f25c4979f40 1 -- 172.21.15.38:0/2350447563 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e114) -- 0x55d730b089e0 con 0x55d730b017d0 2024-09-21T03:42:10.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.139+0000 7f25bec34640 1 -- 172.21.15.38:0/2350447563 <== osd.3 v2:172.21.15.38:6808/1286810294 21 ==== osd_op_reply(106 609.00000000 [delete] v114'106 uv106 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f25b800d960 con 0x55d730b017d0 2024-09-21T03:42:10.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.139+0000 7f25c4979f40 1 -- 172.21.15.38:0/2350447563 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55d730b089e0 con 0x55d730b017d0 2024-09-21T03:42:10.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.140+0000 7f25bec34640 1 -- 172.21.15.38:0/2350447563 <== osd.3 v2:172.21.15.38:6808/1286810294 22 ==== osd_op_reply(107 609.00000000 [create] v114'107 uv107 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f25b800d960 con 0x55d730b017d0 2024-09-21T03:42:10.142 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-09-21T03:42:10.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.140+0000 7f25c4979f40 1 -- 172.21.15.38:0/2350447563 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55d730b099f0 con 0x55d730b017d0 2024-09-21T03:42:10.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.141+0000 7f25bec34640 1 -- 172.21.15.38:0/2350447563 <== osd.3 v2:172.21.15.38:6808/1286810294 23 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v114'108 uv108 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f25b800d960 con 0x55d730b017d0 2024-09-21T03:42:10.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.141+0000 7f25c4979f40 1 -- 172.21.15.38:0/2350447563 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:109 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55d730b099f0 con 0x55d730b017d0 2024-09-21T03:42:10.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.142+0000 7f25bec34640 1 -- 172.21.15.38:0/2350447563 <== osd.3 v2:172.21.15.38:6808/1286810294 24 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v114'109 uv109 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f25b800d960 con 0x55d730b017d0 2024-09-21T03:42:10.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.142+0000 7f25c4979f40 1 -- 172.21.15.38:0/2350447563 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:110 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55d730b089e0 con 0x55d730b017d0 2024-09-21T03:42:10.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.143+0000 7f25bec34640 1 -- 172.21.15.38:0/2350447563 <== osd.3 v2:172.21.15.38:6808/1286810294 25 ==== osd_op_reply(110 609.00000000 [omap-set-header] v114'110 uv110 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f25b800d960 con 0x55d730b017d0 2024-09-21T03:42:10.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.143+0000 7f25c4979f40 1 --2- 172.21.15.38:0/2350447563 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x55d730b09060 0x55d730b09470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:10.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.143+0000 7f25c4979f40 1 -- 172.21.15.38:0/2350447563 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55d730b099b0 con 0x55d730b09060 2024-09-21T03:42:10.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.144+0000 7f25bdc32640 1 --2- 172.21.15.38:0/2350447563 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x55d730b09060 0x55d730b09470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:10.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.144+0000 7f25bdc32640 1 --2- 172.21.15.38:0/2350447563 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x55d730b09060 0x55d730b09470 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:10.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.145+0000 7f25bdc32640 1 -- 172.21.15.38:0/2350447563 <== osd.2 v2:172.21.15.38:6824/2577858902 1 ==== osd_op_reply(111 400.00000000 [create] v114'89 uv88 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f25b40564f0 con 0x55d730b09060 2024-09-21T03:42:10.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.145+0000 7f25c4979f40 1 -- 172.21.15.38:0/2350447563 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e114) -- 0x55d730b099b0 con 0x55d730b09060 2024-09-21T03:42:10.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.146+0000 7f25bdc32640 1 -- 172.21.15.38:0/2350447563 <== osd.2 v2:172.21.15.38:6824/2577858902 2 ==== osd_op_reply(112 400.00000000 [delete] v114'90 uv90 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f25b40564f0 con 0x55d730b09060 2024-09-21T03:42:10.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.146+0000 7f25c4979f40 1 -- 172.21.15.38:0/2350447563 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55d730b099b0 con 0x55d730b09060 2024-09-21T03:42:10.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.147+0000 7f25bdc32640 1 -- 172.21.15.38:0/2350447563 <== osd.2 v2:172.21.15.38:6824/2577858902 3 ==== osd_op_reply(113 400.00000000 [create] v114'91 uv91 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f25b40564f0 con 0x55d730b09060 2024-09-21T03:42:10.149 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-09-21T03:42:10.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.148+0000 7f25c4979f40 1 -- 172.21.15.38:0/2350447563 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:114 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55d730b09dd0 con 0x55d730b09060 2024-09-21T03:42:10.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.149+0000 7f25bdc32640 1 -- 172.21.15.38:0/2350447563 <== osd.2 v2:172.21.15.38:6824/2577858902 4 ==== osd_op_reply(114 400.00000000 [write 0~22] v114'92 uv92 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f25b40564f0 con 0x55d730b09060 2024-09-21T03:42:10.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.149+0000 7f25c4979f40 1 -- 172.21.15.38:0/2350447563 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:115 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55d730b09dd0 con 0x55d730b09060 2024-09-21T03:42:10.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.150+0000 7f25bdc32640 1 -- 172.21.15.38:0/2350447563 <== osd.2 v2:172.21.15.38:6824/2577858902 5 ==== osd_op_reply(115 400.00000000 [omap-set-header] v114'93 uv93 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f25b40564f0 con 0x55d730b09060 2024-09-21T03:42:10.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.150+0000 7f25c4979f40 1 -- 172.21.15.38:0/2350447563 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:116 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55d730b09dd0 con 0x55d730b05660 2024-09-21T03:42:10.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.151+0000 7f25be433640 1 -- 172.21.15.38:0/2350447563 <== osd.1 v2:172.21.15.38:6816/1491818732 17 ==== osd_op_reply(116 100.00000000.inode [create] v114'216 uv208 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7f25b00680b0 con 0x55d730b05660 2024-09-21T03:42:10.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.151+0000 7f25c4979f40 1 -- 172.21.15.38:0/2350447563 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:117 13.1f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e114) -- 0x55d730b09dd0 con 0x55d730b05660 2024-09-21T03:42:10.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.152+0000 7f25be433640 1 -- 172.21.15.38:0/2350447563 <== osd.1 v2:172.21.15.38:6816/1491818732 18 ==== osd_op_reply(117 100.00000000.inode [delete] v114'217 uv217 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f25b00680b0 con 0x55d730b05660 2024-09-21T03:42:10.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.152+0000 7f25c4979f40 1 -- 172.21.15.38:0/2350447563 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:118 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55d730b09dd0 con 0x55d730b05660 2024-09-21T03:42:10.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.153+0000 7f25be433640 1 -- 172.21.15.38:0/2350447563 <== osd.1 v2:172.21.15.38:6816/1491818732 19 ==== osd_op_reply(118 100.00000000.inode [create] v114'218 uv218 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f25b00680b0 con 0x55d730b05660 2024-09-21T03:42:10.155 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-09-21T03:42:10.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.153+0000 7f25c4979f40 1 -- 172.21.15.38:0/2350447563 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:119 13.1f 13:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55d730b09dd0 con 0x55d730b05660 2024-09-21T03:42:10.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.154+0000 7f25be433640 1 -- 172.21.15.38:0/2350447563 <== osd.1 v2:172.21.15.38:6816/1491818732 20 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v114'219 uv219 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f25b00680b0 con 0x55d730b05660 2024-09-21T03:42:10.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.154+0000 7f25c4979f40 1 -- 172.21.15.38:0/2350447563 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:120 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55d730b09dd0 con 0x55d730b05660 2024-09-21T03:42:10.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.155+0000 7f25be433640 1 -- 172.21.15.38:0/2350447563 <== osd.1 v2:172.21.15.38:6816/1491818732 21 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v114'220 uv220 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f25b00680b0 con 0x55d730b05660 2024-09-21T03:42:10.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.155+0000 7f25c4979f40 1 -- 172.21.15.38:0/2350447563 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:121 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55d730b09dd0 con 0x55d730b05660 2024-09-21T03:42:10.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.156+0000 7f25be433640 1 -- 172.21.15.38:0/2350447563 <== osd.1 v2:172.21.15.38:6816/1491818732 22 ==== osd_op_reply(121 1.00000000 [create] v114'221 uv215 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7f25b00680b0 con 0x55d730b05660 2024-09-21T03:42:10.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.156+0000 7f25c4979f40 1 -- 172.21.15.38:0/2350447563 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:122 13.1f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e114) -- 0x55d730b09dd0 con 0x55d730b05660 2024-09-21T03:42:10.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.157+0000 7f25be433640 1 -- 172.21.15.38:0/2350447563 <== osd.1 v2:172.21.15.38:6816/1491818732 23 ==== osd_op_reply(122 1.00000000 [delete] v114'222 uv222 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f25b00680b0 con 0x55d730b05660 2024-09-21T03:42:10.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.157+0000 7f25c4979f40 1 -- 172.21.15.38:0/2350447563 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:123 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55d730b09dd0 con 0x55d730b05660 2024-09-21T03:42:10.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.158+0000 7f25be433640 1 -- 172.21.15.38:0/2350447563 <== osd.1 v2:172.21.15.38:6816/1491818732 24 ==== osd_op_reply(123 1.00000000 [create] v114'223 uv223 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f25b00680b0 con 0x55d730b05660 2024-09-21T03:42:10.160 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-09-21T03:42:10.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.158+0000 7f25c4979f40 1 -- 172.21.15.38:0/2350447563 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:124 13.1f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55d730b09dd0 con 0x55d730b05660 2024-09-21T03:42:10.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.159+0000 7f25be433640 1 -- 172.21.15.38:0/2350447563 <== osd.1 v2:172.21.15.38:6816/1491818732 25 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v114'224 uv224 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f25b00680b0 con 0x55d730b05660 2024-09-21T03:42:10.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.159+0000 7f25c4979f40 1 -- 172.21.15.38:0/2350447563 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:125 13.1f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55d730b09dd0 con 0x55d730b05660 2024-09-21T03:42:10.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.160+0000 7f25be433640 1 -- 172.21.15.38:0/2350447563 <== osd.1 v2:172.21.15.38:6816/1491818732 26 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v114'225 uv225 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f25b00680b0 con 0x55d730b05660 2024-09-21T03:42:10.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.160+0000 7f25c4979f40 1 -- 172.21.15.38:0/2350447563 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:126 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55d730b09dd0 con 0x55d730b05660 2024-09-21T03:42:10.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.161+0000 7f25be433640 1 -- 172.21.15.38:0/2350447563 <== osd.1 v2:172.21.15.38:6816/1491818732 27 ==== osd_op_reply(126 1.00000000 [omap-set-header] v114'226 uv226 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f25b00680b0 con 0x55d730b05660 2024-09-21T03:42:10.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.161+0000 7f25c4979f40 1 -- 172.21.15.38:0/2350447563 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:127 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=584b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55d730b0a1c0 con 0x55d730b05660 2024-09-21T03:42:10.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.162+0000 7f25be433640 1 -- 172.21.15.38:0/2350447563 <== osd.1 v2:172.21.15.38:6816/1491818732 28 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v114'227 uv227 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f25b00680b0 con 0x55d730b05660 2024-09-21T03:42:10.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.162+0000 7f25c4979f40 1 -- 172.21.15.38:0/2350447563 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x55d730afd270 msgr2=0x55d730aff690 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:10.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.162+0000 7f25c4979f40 1 --2- 172.21.15.38:0/2350447563 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x55d730afd270 0x55d730aff690 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:10.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.162+0000 7f25c4979f40 1 -- 172.21.15.38:0/2350447563 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x55d730b05660 msgr2=0x55d730b07a80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:10.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.162+0000 7f25c4979f40 1 --2- 172.21.15.38:0/2350447563 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x55d730b05660 0x55d730b07a80 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:10.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.162+0000 7f25c4979f40 1 -- 172.21.15.38:0/2350447563 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x55d730b09060 msgr2=0x55d730b09470 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:10.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.162+0000 7f25c4979f40 1 --2- 172.21.15.38:0/2350447563 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x55d730b09060 0x55d730b09470 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:10.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.162+0000 7f25c4979f40 1 -- 172.21.15.38:0/2350447563 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x55d730b017d0 msgr2=0x55d730b04630 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:10.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.162+0000 7f25c4979f40 1 --2- 172.21.15.38:0/2350447563 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x55d730b017d0 0x55d730b04630 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:10.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.162+0000 7f25c4979f40 1 -- 172.21.15.38:0/2350447563 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x55d730af8820 msgr2=0x55d730afac40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:10.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.162+0000 7f25c4979f40 1 --2- 172.21.15.38:0/2350447563 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x55d730af8820 0x55d730afac40 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:10.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.162+0000 7f25bec34640 1 -- 172.21.15.38:0/2350447563 reap_dead start 2024-09-21T03:42:10.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.162+0000 7f25bec34640 1 -- 172.21.15.38:0/2350447563 reap_dead start 2024-09-21T03:42:10.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.162+0000 7f25c4979f40 1 -- 172.21.15.38:0/2350447563 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x55d730aee1a0 msgr2=0x55d730aee5b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:10.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.162+0000 7f25c4979f40 1 --2- 172.21.15.38:0/2350447563 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x55d730aee1a0 0x55d730aee5b0 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:10.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.162+0000 7f25c4979f40 1 -- 172.21.15.38:0/2350447563 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f25a0077570 msgr2=0x7f25a0079a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:10.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.162+0000 7f25c4979f40 1 --2- 172.21.15.38:0/2350447563 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f25a0077570 0x7f25a0079a80 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f25b4055000 tx=0x7f25b4058040 comp rx=0 tx=0).stop 2024-09-21T03:42:10.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.162+0000 7f25c4979f40 1 -- 172.21.15.38:0/2350447563 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d730981d00 msgr2=0x55d730a50490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:10.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.162+0000 7f25c4979f40 1 --2- 172.21.15.38:0/2350447563 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d730981d00 0x55d730a50490 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7f25b000d3f0 tx=0x7f25b000d8b0 comp rx=0 tx=0).stop 2024-09-21T03:42:10.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.162+0000 7f25c4979f40 1 -- 172.21.15.38:0/2350447563 shutdown_connections 2024-09-21T03:42:10.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.162+0000 7f25c4979f40 1 --2- 172.21.15.38:0/2350447563 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x55d730aee1a0 0x55d730aee5b0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:10.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.162+0000 7f25c4979f40 1 --2- 172.21.15.38:0/2350447563 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f25a0077570 0x7f25a0079a80 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:10.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.162+0000 7f25c4979f40 1 --2- 172.21.15.38:0/2350447563 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d730981d00 0x55d730a50490 unknown :-1 s=CLOSED pgs=1663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:10.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.162+0000 7f25c4979f40 1 -- 172.21.15.38:0/2350447563 >> 172.21.15.38:0/2350447563 conn(0x55d7308a0ac0 msgr2=0x55d730a360a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:10.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.163+0000 7f25c4979f40 1 -- 172.21.15.38:0/2350447563 shutdown_connections 2024-09-21T03:42:10.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.163+0000 7f25c4979f40 1 -- 172.21.15.38:0/2350447563 wait complete. 2024-09-21T03:42:10.168 DEBUG:teuthology.orchestra.run.smithi038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata put 500.00000000 - 2024-09-21T03:42:10.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.262+0000 7f4453d22f40 1 Processor -- start 2024-09-21T03:42:10.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.262+0000 7f4453d22f40 1 -- start start 2024-09-21T03:42:10.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.264+0000 7f4453d22f40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55936a002240 0x55936a0d2630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:10.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.264+0000 7f4453d22f40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55936a049090 0x55936a047a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:10.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.264+0000 7f4453d22f40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55936a0be520 0x55936a0cf850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:10.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.264+0000 7f4453d22f40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x559369f4fd60 con 0x55936a049090 2024-09-21T03:42:10.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.264+0000 7f4453d22f40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x559369fae430 con 0x55936a0be520 2024-09-21T03:42:10.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.264+0000 7f4453d22f40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55936a0d6350 con 0x55936a002240 2024-09-21T03:42:10.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.264+0000 7f444cfdb640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55936a049090 0x55936a047a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:10.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.264+0000 7f444cfdb640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55936a049090 0x55936a047a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49598/0 (socket says 172.21.15.38:49598) 2024-09-21T03:42:10.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.264+0000 7f444cfdb640 1 -- 172.21.15.38:0/100436521 learned_addr learned my addr 172.21.15.38:0/100436521 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:42:10.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.264+0000 7f444d7dc640 1 --2- 172.21.15.38:0/100436521 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55936a002240 0x55936a0d2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:10.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.264+0000 7f444dfdd640 1 --2- 172.21.15.38:0/100436521 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55936a0be520 0x55936a0cf850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:10.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.264+0000 7f444cfdb640 1 -- 172.21.15.38:0/100436521 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55936a002240 msgr2=0x55936a0d2630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:10.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.264+0000 7f444cfdb640 1 --2- 172.21.15.38:0/100436521 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55936a002240 0x55936a0d2630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:10.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.264+0000 7f444cfdb640 1 -- 172.21.15.38:0/100436521 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55936a0be520 msgr2=0x55936a0cf850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:10.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.264+0000 7f444cfdb640 1 --2- 172.21.15.38:0/100436521 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55936a0be520 0x55936a0cf850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:10.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.264+0000 7f444cfdb640 1 -- 172.21.15.38:0/100436521 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55936a0bdcf0 con 0x55936a049090 2024-09-21T03:42:10.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.264+0000 7f444d7dc640 1 --2- 172.21.15.38:0/100436521 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55936a002240 0x55936a0d2630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:42:10.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.264+0000 7f444dfdd640 1 --2- 172.21.15.38:0/100436521 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55936a0be520 0x55936a0cf850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:42:10.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.264+0000 7f444cfdb640 1 --2- 172.21.15.38:0/100436521 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55936a049090 0x55936a047a40 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7f444400b550 tx=0x7f444400ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff8a2c26128c4ee3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:10.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.264+0000 7f443f7fe640 1 -- 172.21.15.38:0/100436521 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f444400a8c0 con 0x55936a049090 2024-09-21T03:42:10.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.264+0000 7f443f7fe640 1 -- 172.21.15.38:0/100436521 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f444400aa60 con 0x55936a049090 2024-09-21T03:42:10.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.264+0000 7f443f7fe640 1 -- 172.21.15.38:0/100436521 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4444008450 con 0x55936a049090 2024-09-21T03:42:10.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.265+0000 7f4453d22f40 1 -- 172.21.15.38:0/100436521 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55936a049090 msgr2=0x55936a047a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:10.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.265+0000 7f4453d22f40 1 --2- 172.21.15.38:0/100436521 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55936a049090 0x55936a047a40 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7f444400b550 tx=0x7f444400ba10 comp rx=0 tx=0).stop 2024-09-21T03:42:10.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.265+0000 7f4453d22f40 1 -- 172.21.15.38:0/100436521 shutdown_connections 2024-09-21T03:42:10.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.265+0000 7f4453d22f40 1 --2- 172.21.15.38:0/100436521 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55936a0be520 0x55936a0cf850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:10.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.265+0000 7f4453d22f40 1 --2- 172.21.15.38:0/100436521 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55936a049090 0x55936a047a40 unknown :-1 s=CLOSED pgs=1666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:10.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.265+0000 7f4453d22f40 1 --2- 172.21.15.38:0/100436521 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55936a002240 0x55936a0d2630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:10.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.265+0000 7f4453d22f40 1 -- 172.21.15.38:0/100436521 >> 172.21.15.38:0/100436521 conn(0x559369f20ac0 msgr2=0x55936a02bc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:10.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.265+0000 7f4453d22f40 1 -- 172.21.15.38:0/100436521 shutdown_connections 2024-09-21T03:42:10.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.265+0000 7f4453d22f40 1 -- 172.21.15.38:0/100436521 wait complete. 2024-09-21T03:42:10.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.266+0000 7f4453d22f40 1 Processor -- start 2024-09-21T03:42:10.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.266+0000 7f4453d22f40 1 -- start start 2024-09-21T03:42:10.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.267+0000 7f4453d22f40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55936a002240 0x55936a0bc550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:10.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.267+0000 7f4453d22f40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55936a049090 0x55936a02ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:10.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.267+0000 7f4453d22f40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55936a0be520 0x55936a040810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:10.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.267+0000 7f4453d22f40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55936a0bd310 con 0x55936a049090 2024-09-21T03:42:10.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.267+0000 7f4453d22f40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55936a0ceeb0 con 0x55936a002240 2024-09-21T03:42:10.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.267+0000 7f4453d22f40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55936a0cf490 con 0x55936a0be520 2024-09-21T03:42:10.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.267+0000 7f444cfdb640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55936a049090 0x55936a02ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:10.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.267+0000 7f444cfdb640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55936a049090 0x55936a02ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49602/0 (socket says 172.21.15.38:49602) 2024-09-21T03:42:10.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.267+0000 7f444cfdb640 1 -- 172.21.15.38:0/2168755176 learned_addr learned my addr 172.21.15.38:0/2168755176 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:42:10.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.267+0000 7f444d7dc640 1 --2- 172.21.15.38:0/2168755176 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55936a002240 0x55936a0bc550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:10.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.267+0000 7f444dfdd640 1 --2- 172.21.15.38:0/2168755176 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55936a0be520 0x55936a040810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:10.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.268+0000 7f444cfdb640 1 -- 172.21.15.38:0/2168755176 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55936a0be520 msgr2=0x55936a040810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:10.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.268+0000 7f444cfdb640 1 --2- 172.21.15.38:0/2168755176 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55936a0be520 0x55936a040810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:10.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.268+0000 7f444cfdb640 1 -- 172.21.15.38:0/2168755176 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55936a002240 msgr2=0x55936a0bc550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:10.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.268+0000 7f444cfdb640 1 --2- 172.21.15.38:0/2168755176 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55936a002240 0x55936a0bc550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:10.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.268+0000 7f444cfdb640 1 -- 172.21.15.38:0/2168755176 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55936a0d1ea0 con 0x55936a049090 2024-09-21T03:42:10.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.268+0000 7f444cfdb640 1 --2- 172.21.15.38:0/2168755176 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55936a049090 0x55936a02ed40 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7f444400bf10 tx=0x7f44440169c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:10.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.268+0000 7f443e7fc640 1 -- 172.21.15.38:0/2168755176 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4444016be0 con 0x55936a049090 2024-09-21T03:42:10.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.268+0000 7f443e7fc640 1 -- 172.21.15.38:0/2168755176 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4444016d80 con 0x55936a049090 2024-09-21T03:42:10.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.268+0000 7f4453d22f40 1 -- 172.21.15.38:0/2168755176 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55936a0d2140 con 0x55936a049090 2024-09-21T03:42:10.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.268+0000 7f443e7fc640 1 -- 172.21.15.38:0/2168755176 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4444020420 con 0x55936a049090 2024-09-21T03:42:10.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.268+0000 7f4453d22f40 1 -- 172.21.15.38:0/2168755176 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55936a161df0 con 0x55936a049090 2024-09-21T03:42:10.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.270+0000 7f443e7fc640 1 -- 172.21.15.38:0/2168755176 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f4444008450 con 0x55936a049090 2024-09-21T03:42:10.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.270+0000 7f443e7fc640 1 --2- 172.21.15.38:0/2168755176 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4430077570 0x7f4430079a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:10.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.271+0000 7f443e7fc640 1 -- 172.21.15.38:0/2168755176 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 7852+0+0 (secure 0 0 0) 0x7f444409bf60 con 0x55936a049090 2024-09-21T03:42:10.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.271+0000 7f444d7dc640 1 --2- 172.21.15.38:0/2168755176 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4430077570 0x7f4430079a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:10.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.271+0000 7f4453d22f40 1 --2- 172.21.15.38:0/2168755176 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x55936a040d50 0x55936a02a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:10.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.271+0000 7f4453d22f40 1 -- 172.21.15.38:0/2168755176 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:1 13.15 13:aa448500:::500.00000000:head [writefull 0~80 in=80b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55936a02daa0 con 0x55936a040d50 2024-09-21T03:42:10.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.271+0000 7f444dfdd640 1 --2- 172.21.15.38:0/2168755176 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x55936a040d50 0x55936a02a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:10.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.271+0000 7f444d7dc640 1 --2- 172.21.15.38:0/2168755176 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4430077570 0x7f4430079a80 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f444000f4e0 tx=0x7f44400017e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:10.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.271+0000 7f444dfdd640 1 --2- 172.21.15.38:0/2168755176 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x55936a040d50 0x55936a02a970 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:10.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.273+0000 7f444dfdd640 1 -- 172.21.15.38:0/2168755176 <== osd.3 v2:172.21.15.38:6808/1286810294 1 ==== osd_op_reply(1 500.00000000 [writefull 0~80] v114'204 uv204 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f444800b040 con 0x55936a040d50 2024-09-21T03:42:10.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.273+0000 7f4453d22f40 1 -- 172.21.15.38:0/2168755176 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x55936a040d50 msgr2=0x55936a02a970 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:10.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.273+0000 7f4453d22f40 1 --2- 172.21.15.38:0/2168755176 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x55936a040d50 0x55936a02a970 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:10.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.273+0000 7f4453d22f40 1 -- 172.21.15.38:0/2168755176 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4430077570 msgr2=0x7f4430079a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:10.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.273+0000 7f4453d22f40 1 --2- 172.21.15.38:0/2168755176 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4430077570 0x7f4430079a80 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f444000f4e0 tx=0x7f44400017e0 comp rx=0 tx=0).stop 2024-09-21T03:42:10.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.273+0000 7f4453d22f40 1 -- 172.21.15.38:0/2168755176 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55936a049090 msgr2=0x55936a02ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:10.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.273+0000 7f4453d22f40 1 --2- 172.21.15.38:0/2168755176 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55936a049090 0x55936a02ed40 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7f444400bf10 tx=0x7f44440169c0 comp rx=0 tx=0).stop 2024-09-21T03:42:10.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.273+0000 7f444dfdd640 1 -- 172.21.15.38:0/2168755176 reap_dead start 2024-09-21T03:42:10.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.273+0000 7f4453d22f40 1 -- 172.21.15.38:0/2168755176 shutdown_connections 2024-09-21T03:42:10.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.273+0000 7f4453d22f40 1 -- 172.21.15.38:0/2168755176 >> 172.21.15.38:0/2168755176 conn(0x559369f20ac0 msgr2=0x55936a0cf9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:10.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.273+0000 7f4453d22f40 1 -- 172.21.15.38:0/2168755176 shutdown_connections 2024-09-21T03:42:10.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.273+0000 7f4453d22f40 1 -- 172.21.15.38:0/2168755176 wait complete. 2024-09-21T03:42:10.278 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-21T03:42:10.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.436+0000 7fd29cb39640 1 -- 172.21.15.38:0/4289105358 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd2980d0170 msgr2=0x7fd2980d0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:10.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.436+0000 7fd29cb39640 1 --2- 172.21.15.38:0/4289105358 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd2980d0170 0x7fd2980d0be0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fd28c009850 tx=0x7fd28c02a4f0 comp rx=0 tx=0).stop 2024-09-21T03:42:10.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.436+0000 7fd29cb39640 1 -- 172.21.15.38:0/4289105358 shutdown_connections 2024-09-21T03:42:10.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.436+0000 7fd29cb39640 1 --2- 172.21.15.38:0/4289105358 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd2980d0170 0x7fd2980d0be0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:10.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.436+0000 7fd29cb39640 1 -- 172.21.15.38:0/4289105358 >> 172.21.15.38:0/4289105358 conn(0x7fd29809d830 msgr2=0x7fd2980b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:10.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.436+0000 7fd29cb39640 1 -- 172.21.15.38:0/4289105358 shutdown_connections 2024-09-21T03:42:10.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.436+0000 7fd29cb39640 1 -- 172.21.15.38:0/4289105358 wait complete. 2024-09-21T03:42:10.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.438+0000 7fd29cb39640 1 Processor -- start 2024-09-21T03:42:10.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.438+0000 7fd29cb39640 1 -- start start 2024-09-21T03:42:10.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.439+0000 7fd29cb39640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd2980d0170 0x7fd2981561d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:10.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.439+0000 7fd29cb39640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd2980d1120 0x7fd2981f9370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:10.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.439+0000 7fd29cb39640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd29815ee80 0x7fd2981f54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:10.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.439+0000 7fd29cb39640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fd2980b6910 con 0x7fd2980d0170 2024-09-21T03:42:10.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.439+0000 7fd29cb39640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd2980b6790 con 0x7fd29815ee80 2024-09-21T03:42:10.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.439+0000 7fd29cb39640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fd2980b6a90 con 0x7fd2980d1120 2024-09-21T03:42:10.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.439+0000 7fd29659c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd2980d0170 0x7fd2981561d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:10.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.439+0000 7fd29659c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd2980d0170 0x7fd2981561d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49610/0 (socket says 172.21.15.38:49610) 2024-09-21T03:42:10.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.439+0000 7fd29659c640 1 -- 172.21.15.38:0/3636156615 learned_addr learned my addr 172.21.15.38:0/3636156615 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:42:10.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.440+0000 7fd295d9b640 1 --2- 172.21.15.38:0/3636156615 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd2980d1120 0x7fd2981f9370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:10.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.440+0000 7fd296d9d640 1 --2- 172.21.15.38:0/3636156615 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd29815ee80 0x7fd2981f54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:10.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.440+0000 7fd29659c640 1 -- 172.21.15.38:0/3636156615 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd2980d1120 msgr2=0x7fd2981f9370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:10.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.440+0000 7fd29659c640 1 --2- 172.21.15.38:0/3636156615 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd2980d1120 0x7fd2981f9370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:10.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.440+0000 7fd29659c640 1 -- 172.21.15.38:0/3636156615 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd29815ee80 msgr2=0x7fd2981f54e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:10.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.440+0000 7fd29659c640 1 --2- 172.21.15.38:0/3636156615 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd29815ee80 0x7fd2981f54e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:10.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.440+0000 7fd29659c640 1 -- 172.21.15.38:0/3636156615 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd29815bec0 con 0x7fd2980d0170 2024-09-21T03:42:10.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.440+0000 7fd29659c640 1 --2- 172.21.15.38:0/3636156615 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd2980d0170 0x7fd2981561d0 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7fd28800c460 tx=0x7fd28800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:10.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.440+0000 7fd286ffd640 1 -- 172.21.15.38:0/3636156615 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2880052d0 con 0x7fd2980d0170 2024-09-21T03:42:10.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.440+0000 7fd286ffd640 1 -- 172.21.15.38:0/3636156615 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd288005490 con 0x7fd2980d0170 2024-09-21T03:42:10.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.440+0000 7fd286ffd640 1 -- 172.21.15.38:0/3636156615 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd288012870 con 0x7fd2980d0170 2024-09-21T03:42:10.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.440+0000 7fd29cb39640 1 -- 172.21.15.38:0/3636156615 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd29815c1a0 con 0x7fd2980d0170 2024-09-21T03:42:10.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.440+0000 7fd29cb39640 1 -- 172.21.15.38:0/3636156615 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2981f8e50 con 0x7fd2980d0170 2024-09-21T03:42:10.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.441+0000 7fd29cb39640 1 -- 172.21.15.38:0/3636156615 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2980d0be0 con 0x7fd2980d0170 2024-09-21T03:42:10.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.442+0000 7fd286ffd640 1 -- 172.21.15.38:0/3636156615 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fd288012a10 con 0x7fd2980d0170 2024-09-21T03:42:10.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.443+0000 7fd286ffd640 1 --2- 172.21.15.38:0/3636156615 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd278078b90 0x7fd27807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:10.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.443+0000 7fd286ffd640 1 -- 172.21.15.38:0/3636156615 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 7852+0+0 (secure 0 0 0) 0x7fd28809af00 con 0x7fd2980d0170 2024-09-21T03:42:10.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.443+0000 7fd295d9b640 1 --2- 172.21.15.38:0/3636156615 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd278078b90 0x7fd27807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:10.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.444+0000 7fd295d9b640 1 --2- 172.21.15.38:0/3636156615 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd278078b90 0x7fd27807b0a0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fd28c034000 tx=0x7fd28c004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:10.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.448+0000 7fd286ffd640 1 -- 172.21.15.38:0/3636156615 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fd288062520 con 0x7fd2980d0170 2024-09-21T03:42:10.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.674+0000 7fd29cb39640 1 -- 172.21.15.38:0/3636156615 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7fd2980b3220 con 0x7fd2980d0170 2024-09-21T03:42:10.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.921+0000 7fd286ffd640 1 -- 172.21.15.38:0/3636156615 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v194) ==== 162+0+0 (secure 0 0 0) 0x7fd288066f50 con 0x7fd2980d0170 2024-09-21T03:42:10.923 INFO:teuthology.orchestra.run.smithi038.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-21T03:42:10.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.924+0000 7fd29cb39640 1 -- 172.21.15.38:0/3636156615 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd278078b90 msgr2=0x7fd27807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:10.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.924+0000 7fd29cb39640 1 --2- 172.21.15.38:0/3636156615 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd278078b90 0x7fd27807b0a0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fd28c034000 tx=0x7fd28c004730 comp rx=0 tx=0).stop 2024-09-21T03:42:10.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.924+0000 7fd29cb39640 1 -- 172.21.15.38:0/3636156615 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd2980d0170 msgr2=0x7fd2981561d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:10.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.924+0000 7fd29cb39640 1 --2- 172.21.15.38:0/3636156615 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd2980d0170 0x7fd2981561d0 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7fd28800c460 tx=0x7fd28800c920 comp rx=0 tx=0).stop 2024-09-21T03:42:10.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.924+0000 7fd29cb39640 1 -- 172.21.15.38:0/3636156615 shutdown_connections 2024-09-21T03:42:10.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.924+0000 7fd29cb39640 1 --2- 172.21.15.38:0/3636156615 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd29815ee80 0x7fd2981f54e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:10.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.924+0000 7fd29cb39640 1 --2- 172.21.15.38:0/3636156615 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd278078b90 0x7fd27807b0a0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:10.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.924+0000 7fd29cb39640 1 --2- 172.21.15.38:0/3636156615 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd2980d1120 0x7fd2981f9370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:10.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.924+0000 7fd29cb39640 1 --2- 172.21.15.38:0/3636156615 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd2980d0170 0x7fd2981561d0 unknown :-1 s=CLOSED pgs=1669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:10.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.924+0000 7fd29cb39640 1 -- 172.21.15.38:0/3636156615 >> 172.21.15.38:0/3636156615 conn(0x7fd29809d830 msgr2=0x7fd2981589a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:10.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.924+0000 7fd29cb39640 1 -- 172.21.15.38:0/3636156615 shutdown_connections 2024-09-21T03:42:10.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:10.924+0000 7fd29cb39640 1 -- 172.21.15.38:0/3636156615 wait complete. 2024-09-21T03:42:10.934 INFO:tasks.ceph.mds.a.smithi038.stderr:2024-09-21T03:42:10.933+0000 7fc16453c640 -1 mds.0.purge_queue operator(): Error -22 loading Journaler 2024-09-21T03:42:10.934 INFO:tasks.ceph.mds.a.smithi038.stderr:2024-09-21T03:42:10.933+0000 7fc16453c640 -1 mds.0.195 unhandled write error (22) Invalid argument, force readonly... 2024-09-21T03:42:10.940 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:42:11.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:11.055+0000 7fd1c2da8640 1 -- 172.21.15.38:0/1742267218 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd1bc15f4c0 msgr2=0x7fd1bc0b2e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:11.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:11.055+0000 7fd1c2da8640 1 --2- 172.21.15.38:0/1742267218 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd1bc15f4c0 0x7fd1bc0b2e40 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7fd1b4005fd0 tx=0x7fd1b402a8b0 comp rx=0 tx=0).stop 2024-09-21T03:42:11.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:11.055+0000 7fd1c2da8640 1 -- 172.21.15.38:0/1742267218 shutdown_connections 2024-09-21T03:42:11.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:11.055+0000 7fd1c2da8640 1 --2- 172.21.15.38:0/1742267218 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd1bc15f4c0 0x7fd1bc0b2e40 unknown :-1 s=CLOSED pgs=1671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:11.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:11.055+0000 7fd1c2da8640 1 -- 172.21.15.38:0/1742267218 >> 172.21.15.38:0/1742267218 conn(0x7fd1bc0b6b20 msgr2=0x7fd1bc0d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:11.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:11.055+0000 7fd1c2da8640 1 -- 172.21.15.38:0/1742267218 shutdown_connections 2024-09-21T03:42:11.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:11.055+0000 7fd1c2da8640 1 -- 172.21.15.38:0/1742267218 wait complete. 2024-09-21T03:42:11.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:11.057+0000 7fd1c2da8640 1 Processor -- start 2024-09-21T03:42:11.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:11.057+0000 7fd1c2da8640 1 -- start start 2024-09-21T03:42:11.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:11.058+0000 7fd1c2da8640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd1bc15f4c0 0x7fd1bc14c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:11.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:11.058+0000 7fd1c2da8640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd1bc162280 0x7fd1bc14da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:11.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:11.058+0000 7fd1c2da8640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd1bc14dfd0 0x7fd1bc14e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:11.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:11.058+0000 7fd1c2da8640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fd1bc0b3710 con 0x7fd1bc14dfd0 2024-09-21T03:42:11.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:11.058+0000 7fd1c2da8640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd1bc0b3590 con 0x7fd1bc162280 2024-09-21T03:42:11.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:11.058+0000 7fd1c2da8640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fd1bc0b3890 con 0x7fd1bc15f4c0 2024-09-21T03:42:11.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:11.058+0000 7fd1c1345640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd1bc14dfd0 0x7fd1bc14e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:11.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:11.059+0000 7fd1c1345640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd1bc14dfd0 0x7fd1bc14e3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49620/0 (socket says 172.21.15.38:49620) 2024-09-21T03:42:11.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:11.059+0000 7fd1c1345640 1 -- 172.21.15.38:0/2352618910 learned_addr learned my addr 172.21.15.38:0/2352618910 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:42:11.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:11.059+0000 7fd1bbfff640 1 --2- 172.21.15.38:0/2352618910 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd1bc162280 0x7fd1bc14da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:11.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:11.059+0000 7fd1c0b44640 1 --2- 172.21.15.38:0/2352618910 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd1bc15f4c0 0x7fd1bc14c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:11.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:11.059+0000 7fd1c1345640 1 -- 172.21.15.38:0/2352618910 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd1bc15f4c0 msgr2=0x7fd1bc14c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:11.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:11.059+0000 7fd1c1345640 1 --2- 172.21.15.38:0/2352618910 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd1bc15f4c0 0x7fd1bc14c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:11.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:11.059+0000 7fd1c1345640 1 -- 172.21.15.38:0/2352618910 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd1bc162280 msgr2=0x7fd1bc14da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:11.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:11.059+0000 7fd1c1345640 1 --2- 172.21.15.38:0/2352618910 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd1bc162280 0x7fd1bc14da90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:11.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:11.059+0000 7fd1c1345640 1 -- 172.21.15.38:0/2352618910 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1bc14e920 con 0x7fd1bc14dfd0 2024-09-21T03:42:11.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:11.059+0000 7fd1c1345640 1 --2- 172.21.15.38:0/2352618910 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd1bc14dfd0 0x7fd1bc14e3e0 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7fd1b4009cd0 tx=0x7fd1b4031700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:11.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:11.059+0000 7fd1b97fa640 1 -- 172.21.15.38:0/2352618910 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1b4031990 con 0x7fd1bc14dfd0 2024-09-21T03:42:11.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:11.059+0000 7fd1b97fa640 1 -- 172.21.15.38:0/2352618910 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd1b4031c20 con 0x7fd1bc14dfd0 2024-09-21T03:42:11.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:11.059+0000 7fd1c2da8640 1 -- 172.21.15.38:0/2352618910 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1bc0ad840 con 0x7fd1bc14dfd0 2024-09-21T03:42:11.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:11.059+0000 7fd1b97fa640 1 -- 172.21.15.38:0/2352618910 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1b40330d0 con 0x7fd1bc14dfd0 2024-09-21T03:42:11.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:11.059+0000 7fd1c2da8640 1 -- 172.21.15.38:0/2352618910 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1bc14d290 con 0x7fd1bc14dfd0 2024-09-21T03:42:11.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:11.061+0000 7fd1c2da8640 1 -- 172.21.15.38:0/2352618910 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1bc162f00 con 0x7fd1bc14dfd0 2024-09-21T03:42:11.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:11.061+0000 7fd1b97fa640 1 -- 172.21.15.38:0/2352618910 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fd1b4033270 con 0x7fd1bc14dfd0 2024-09-21T03:42:11.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:11.061+0000 7fd1b97fa640 1 --2- 172.21.15.38:0/2352618910 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd1a8078b90 0x7fd1a807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:11.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:11.062+0000 7fd1b97fa640 1 -- 172.21.15.38:0/2352618910 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 7852+0+0 (secure 0 0 0) 0x7fd1b40bca10 con 0x7fd1bc14dfd0 2024-09-21T03:42:11.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:11.062+0000 7fd1c0b44640 1 --2- 172.21.15.38:0/2352618910 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd1a8078b90 0x7fd1a807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:11.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:11.062+0000 7fd1c0b44640 1 --2- 172.21.15.38:0/2352618910 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd1a8078b90 0x7fd1a807b0a0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fd1ac007fd0 tx=0x7fd1ac001370 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:11.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:11.066+0000 7fd1b97fa640 1 -- 172.21.15.38:0/2352618910 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fd1b403f030 con 0x7fd1bc14dfd0 2024-09-21T03:42:11.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:11.291+0000 7fd1c2da8640 1 -- 172.21.15.38:0/2352618910 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd1bc0ad5d0 con 0x7fd1bc14dfd0 2024-09-21T03:42:11.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:11.291+0000 7fd1b97fa640 1 -- 172.21.15.38:0/2352618910 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 195 v195) ==== 95+0+4401 (secure 0 0 0) 0x7fd1b4083f00 con 0x7fd1bc14dfd0 2024-09-21T03:42:11.293 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:42:11.293 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":195,"btime":"2024-09-21T03:42:10:925454+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6086,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/965972663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":965972663},{"type":"v1","addr":"172.21.15.195:6836","nonce":965972663}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6700,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/3176122334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":3176122334},{"type":"v1","addr":"172.21.15.195:6837","nonce":3176122334}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":195,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:40:02.319282+0000","modified":"2024-09-21T03:42:10.925439+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":114,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8862},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8862":{"gid":8862,"name":"a","rank":0,"incarnation":195,"state":"up:replay","state_seq":1,"addr":"172.21.15.38:6835/2712360296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":2712360296},{"type":"v1","addr":"172.21.15.38:6835","nonce":2712360296}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-21T03:42:11.294 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 195 2024-09-21T03:42:11.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:11.293+0000 7fd1c2da8640 1 -- 172.21.15.38:0/2352618910 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd1a8078b90 msgr2=0x7fd1a807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:11.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:11.293+0000 7fd1c2da8640 1 --2- 172.21.15.38:0/2352618910 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd1a8078b90 0x7fd1a807b0a0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fd1ac007fd0 tx=0x7fd1ac001370 comp rx=0 tx=0).stop 2024-09-21T03:42:11.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:11.293+0000 7fd1c2da8640 1 -- 172.21.15.38:0/2352618910 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd1bc14dfd0 msgr2=0x7fd1bc14e3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:11.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:11.293+0000 7fd1c2da8640 1 --2- 172.21.15.38:0/2352618910 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd1bc14dfd0 0x7fd1bc14e3e0 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7fd1b4009cd0 tx=0x7fd1b4031700 comp rx=0 tx=0).stop 2024-09-21T03:42:11.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:11.294+0000 7fd1c2da8640 1 -- 172.21.15.38:0/2352618910 shutdown_connections 2024-09-21T03:42:11.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:11.294+0000 7fd1c2da8640 1 --2- 172.21.15.38:0/2352618910 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd1bc14dfd0 0x7fd1bc14e3e0 unknown :-1 s=CLOSED pgs=1672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:11.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:11.294+0000 7fd1c2da8640 1 --2- 172.21.15.38:0/2352618910 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd1bc162280 0x7fd1bc14da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:11.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:11.294+0000 7fd1c2da8640 1 --2- 172.21.15.38:0/2352618910 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd1a8078b90 0x7fd1a807b0a0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:11.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:11.294+0000 7fd1c2da8640 1 --2- 172.21.15.38:0/2352618910 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd1bc15f4c0 0x7fd1bc14c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:11.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:11.294+0000 7fd1c2da8640 1 -- 172.21.15.38:0/2352618910 >> 172.21.15.38:0/2352618910 conn(0x7fd1bc0b6b20 msgr2=0x7fd1bc15e4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:11.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:11.294+0000 7fd1c2da8640 1 -- 172.21.15.38:0/2352618910 shutdown_connections 2024-09-21T03:42:11.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:11.294+0000 7fd1c2da8640 1 -- 172.21.15.38:0/2352618910 wait complete. 2024-09-21T03:42:11.654 INFO:tasks.ceph.mds.c.smithi038.stderr: -43> 2024-09-21T03:42:06.900+0000 7ffab9f77640 -1 log_channel(cluster) log [ERR] : missing journal object 2024-09-21T03:42:11.708 INFO:tasks.ceph.mds.c.smithi038.stdout:starting mds.c at 2024-09-21T03:42:11.710 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:42:11.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:11.824+0000 7fc39e5c3640 1 -- 172.21.15.38:0/1624517883 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc398074290 msgr2=0x7fc3980afb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:11.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:11.824+0000 7fc39e5c3640 1 --2- 172.21.15.38:0/1624517883 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc398074290 0x7fc3980afb50 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7fc390005fd0 tx=0x7fc39002aa70 comp rx=0 tx=0).stop 2024-09-21T03:42:11.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:11.824+0000 7fc39e5c3640 1 -- 172.21.15.38:0/1624517883 shutdown_connections 2024-09-21T03:42:11.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:11.824+0000 7fc39e5c3640 1 --2- 172.21.15.38:0/1624517883 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc398074290 0x7fc3980afb50 unknown :-1 s=CLOSED pgs=1676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:11.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:11.824+0000 7fc39e5c3640 1 --2- 172.21.15.38:0/1624517883 >> v2:172.21.15.195:3300/0 conn(0x7fc398158ba0 0x7fc398073d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:11.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:11.824+0000 7fc39e5c3640 1 -- 172.21.15.38:0/1624517883 >> 172.21.15.38:0/1624517883 conn(0x7fc3980b2cc0 msgr2=0x7fc3980b30d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:11.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:11.824+0000 7fc39e5c3640 1 -- 172.21.15.38:0/1624517883 shutdown_connections 2024-09-21T03:42:11.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:11.824+0000 7fc39e5c3640 1 -- 172.21.15.38:0/1624517883 wait complete. 2024-09-21T03:42:11.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:11.826+0000 7fc39e5c3640 1 Processor -- start 2024-09-21T03:42:11.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:11.826+0000 7fc39e5c3640 1 -- start start 2024-09-21T03:42:11.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:11.828+0000 7fc39e5c3640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc398074290 0x7fc398144b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:11.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:11.828+0000 7fc39e5c3640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc398158ba0 0x7fc3981f8c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:11.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:11.828+0000 7fc39e5c3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc3981f91b0 0x7fc3980d1060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:11.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:11.828+0000 7fc39e5c3640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fc3980b04e0 con 0x7fc3981f91b0 2024-09-21T03:42:11.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:11.828+0000 7fc39e5c3640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fc3980b0360 con 0x7fc398158ba0 2024-09-21T03:42:11.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:11.828+0000 7fc39e5c3640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fc3980b0660 con 0x7fc398074290 2024-09-21T03:42:11.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:11.828+0000 7fc39cb60640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc3981f91b0 0x7fc3980d1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:11.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:11.828+0000 7fc39cb60640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc3981f91b0 0x7fc3980d1060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49664/0 (socket says 172.21.15.38:49664) 2024-09-21T03:42:11.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:11.828+0000 7fc39cb60640 1 -- 172.21.15.38:0/1851509637 learned_addr learned my addr 172.21.15.38:0/1851509637 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:42:11.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:11.828+0000 7fc3977fe640 1 --2- 172.21.15.38:0/1851509637 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc398158ba0 0x7fc3981f8c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:11.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:11.828+0000 7fc397fff640 1 --2- 172.21.15.38:0/1851509637 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc398074290 0x7fc398144b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:11.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:11.828+0000 7fc39cb60640 1 -- 172.21.15.38:0/1851509637 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc398074290 msgr2=0x7fc398144b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:11.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:11.828+0000 7fc39cb60640 1 --2- 172.21.15.38:0/1851509637 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc398074290 0x7fc398144b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:11.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:11.828+0000 7fc39cb60640 1 -- 172.21.15.38:0/1851509637 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc398158ba0 msgr2=0x7fc3981f8c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:11.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:11.828+0000 7fc39cb60640 1 --2- 172.21.15.38:0/1851509637 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc398158ba0 0x7fc3981f8c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:11.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:11.828+0000 7fc39cb60640 1 -- 172.21.15.38:0/1851509637 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3981f5680 con 0x7fc3981f91b0 2024-09-21T03:42:11.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:11.828+0000 7fc39cb60640 1 --2- 172.21.15.38:0/1851509637 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc3981f91b0 0x7fc3980d1060 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7fc390006010 tx=0x7fc3900093d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:11.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:11.829+0000 7fc394ff9640 1 -- 172.21.15.38:0/1851509637 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc390046070 con 0x7fc3981f91b0 2024-09-21T03:42:11.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:11.829+0000 7fc39e5c3640 1 -- 172.21.15.38:0/1851509637 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3981f9960 con 0x7fc3981f91b0 2024-09-21T03:42:11.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:11.829+0000 7fc394ff9640 1 -- 172.21.15.38:0/1851509637 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc390004540 con 0x7fc3981f91b0 2024-09-21T03:42:11.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:11.829+0000 7fc394ff9640 1 -- 172.21.15.38:0/1851509637 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc390033070 con 0x7fc3981f91b0 2024-09-21T03:42:11.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:11.829+0000 7fc39e5c3640 1 -- 172.21.15.38:0/1851509637 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3981f5be0 con 0x7fc3981f91b0 2024-09-21T03:42:11.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:11.830+0000 7fc39e5c3640 1 -- 172.21.15.38:0/1851509637 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc39815ec70 con 0x7fc3981f91b0 2024-09-21T03:42:11.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:11.831+0000 7fc394ff9640 1 -- 172.21.15.38:0/1851509637 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc390049070 con 0x7fc3981f91b0 2024-09-21T03:42:11.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:11.831+0000 7fc394ff9640 1 --2- 172.21.15.38:0/1851509637 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc384078b90 0x7fc38407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:11.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:11.831+0000 7fc394ff9640 1 -- 172.21.15.38:0/1851509637 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 7852+0+0 (secure 0 0 0) 0x7fc3900bc9d0 con 0x7fc3981f91b0 2024-09-21T03:42:11.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:11.831+0000 7fc397fff640 1 --2- 172.21.15.38:0/1851509637 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc384078b90 0x7fc38407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:11.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:11.832+0000 7fc397fff640 1 --2- 172.21.15.38:0/1851509637 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc384078b90 0x7fc38407b0a0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fc398075c30 tx=0x7fc388009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:11.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:11.835+0000 7fc394ff9640 1 -- 172.21.15.38:0/1851509637 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fc3981f5680 con 0x7fc3981f91b0 2024-09-21T03:42:12.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:12.062+0000 7fc39e5c3640 1 -- 172.21.15.38:0/1851509637 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc3980b35c0 con 0x7fc3981f91b0 2024-09-21T03:42:12.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:12.063+0000 7fc394ff9640 1 -- 172.21.15.38:0/1851509637 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 196 v196) ==== 95+0+5230 (secure 0 0 0) 0x7fc39003e070 con 0x7fc3981f91b0 2024-09-21T03:42:12.065 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:42:12.065 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":196,"btime":"2024-09-21T03:42:11:927185+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6086,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/965972663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":965972663},{"type":"v1","addr":"172.21.15.195:6836","nonce":965972663}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6700,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/3176122334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":3176122334},{"type":"v1","addr":"172.21.15.195:6837","nonce":3176122334}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":8925,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/4221150556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":4221150556},{"type":"v1","addr":"172.21.15.38:6833","nonce":4221150556}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":196}],"filesystems":[{"mdsmap":{"epoch":196,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:40:02.319282+0000","modified":"2024-09-21T03:42:10.938507+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":114,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8862},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8862":{"gid":8862,"name":"a","rank":0,"incarnation":195,"state":"up:reconnect","state_seq":2,"addr":"172.21.15.38:6835/2712360296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":2712360296},{"type":"v1","addr":"172.21.15.38:6835","nonce":2712360296}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-21T03:42:12.065 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 196 2024-09-21T03:42:12.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:12.065+0000 7fc39e5c3640 1 -- 172.21.15.38:0/1851509637 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc384078b90 msgr2=0x7fc38407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:12.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:12.065+0000 7fc39e5c3640 1 --2- 172.21.15.38:0/1851509637 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc384078b90 0x7fc38407b0a0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fc398075c30 tx=0x7fc388009040 comp rx=0 tx=0).stop 2024-09-21T03:42:12.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:12.065+0000 7fc39e5c3640 1 -- 172.21.15.38:0/1851509637 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc3981f91b0 msgr2=0x7fc3980d1060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:12.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:12.065+0000 7fc39e5c3640 1 --2- 172.21.15.38:0/1851509637 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc3981f91b0 0x7fc3980d1060 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7fc390006010 tx=0x7fc3900093d0 comp rx=0 tx=0).stop 2024-09-21T03:42:12.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:12.065+0000 7fc39e5c3640 1 -- 172.21.15.38:0/1851509637 shutdown_connections 2024-09-21T03:42:12.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:12.065+0000 7fc39e5c3640 1 --2- 172.21.15.38:0/1851509637 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc3981f91b0 0x7fc3980d1060 unknown :-1 s=CLOSED pgs=1677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:12.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:12.065+0000 7fc39e5c3640 1 --2- 172.21.15.38:0/1851509637 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc398158ba0 0x7fc3981f8c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:12.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:12.065+0000 7fc39e5c3640 1 --2- 172.21.15.38:0/1851509637 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc384078b90 0x7fc38407b0a0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:12.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:12.065+0000 7fc39e5c3640 1 --2- 172.21.15.38:0/1851509637 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc398074290 0x7fc398144b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:12.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:12.065+0000 7fc39e5c3640 1 -- 172.21.15.38:0/1851509637 >> 172.21.15.38:0/1851509637 conn(0x7fc3980b2cc0 msgr2=0x7fc3980af5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:12.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:12.065+0000 7fc39e5c3640 1 -- 172.21.15.38:0/1851509637 shutdown_connections 2024-09-21T03:42:12.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:12.065+0000 7fc39e5c3640 1 -- 172.21.15.38:0/1851509637 wait complete. 2024-09-21T03:42:12.079 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 196, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T03:40:02.319282+0000', 'modified': '2024-09-21T03:42:10.938507+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 114, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8862}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8862': {'gid': 8862, 'name': 'a', 'rank': 0, 'incarnation': 195, 'state': 'up:reconnect', 'state_seq': 2, 'addr': '172.21.15.38:6835/2712360296', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.38:6834', 'nonce': 2712360296}, {'type': 'v1', 'addr': '172.21.15.38:6835', 'nonce': 2712360296}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-09-21T03:42:12.079 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8862:up:reconnect 2024-09-21T03:42:12.079 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2024-09-21T03:42:17.080 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:42:17.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:17.211+0000 7f06de4d5640 1 -- 172.21.15.38:0/3129058808 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f06d80b2e40 msgr2=0x7f06d80b3250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:17.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:17.211+0000 7f06de4d5640 1 --2- 172.21.15.38:0/3129058808 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f06d80b2e40 0x7f06d80b3250 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7f06d0005fd0 tx=0x7f06d002aa90 comp rx=0 tx=0).stop 2024-09-21T03:42:17.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:17.211+0000 7f06de4d5640 1 -- 172.21.15.38:0/3129058808 shutdown_connections 2024-09-21T03:42:17.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:17.211+0000 7f06de4d5640 1 --2- 172.21.15.38:0/3129058808 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f06d80b2e40 0x7f06d80b3250 unknown :-1 s=CLOSED pgs=1678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:17.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:17.211+0000 7f06de4d5640 1 --2- 172.21.15.38:0/3129058808 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f06d8162f00 0x7f06d80d24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:17.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:17.211+0000 7f06de4d5640 1 -- 172.21.15.38:0/3129058808 >> 172.21.15.38:0/3129058808 conn(0x7f06d80b6b20 msgr2=0x7f06d80d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:17.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:17.211+0000 7f06de4d5640 1 -- 172.21.15.38:0/3129058808 shutdown_connections 2024-09-21T03:42:17.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:17.211+0000 7f06de4d5640 1 -- 172.21.15.38:0/3129058808 wait complete. 2024-09-21T03:42:17.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:17.213+0000 7f06de4d5640 1 Processor -- start 2024-09-21T03:42:17.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:17.213+0000 7f06de4d5640 1 -- start start 2024-09-21T03:42:17.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:17.215+0000 7f06de4d5640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f06d80b2e40 0x7f06d814e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:17.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:17.215+0000 7f06de4d5640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f06d8162280 0x7f06d81f53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:17.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:17.215+0000 7f06de4d5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f06d80b2380 0x7f06d80b2790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:17.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:17.215+0000 7f06de4d5640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f06d815e4b0 con 0x7f06d80b2380 2024-09-21T03:42:17.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:17.215+0000 7f06de4d5640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f06d80b39a0 con 0x7f06d8162280 2024-09-21T03:42:17.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:17.215+0000 7f06de4d5640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f06d815e630 con 0x7f06d80b2e40 2024-09-21T03:42:17.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:17.215+0000 7f06dca72640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f06d80b2380 0x7f06d80b2790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:17.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:17.215+0000 7f06dca72640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f06d80b2380 0x7f06d80b2790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34248/0 (socket says 172.21.15.38:34248) 2024-09-21T03:42:17.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:17.215+0000 7f06dca72640 1 -- 172.21.15.38:0/1237460246 learned_addr learned my addr 172.21.15.38:0/1237460246 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:42:17.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:17.215+0000 7f06d7fff640 1 --2- 172.21.15.38:0/1237460246 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f06d80b2e40 0x7f06d814e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:17.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:17.215+0000 7f06d77fe640 1 --2- 172.21.15.38:0/1237460246 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f06d8162280 0x7f06d81f53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:17.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:17.215+0000 7f06dca72640 1 -- 172.21.15.38:0/1237460246 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f06d80b2e40 msgr2=0x7f06d814e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:17.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:17.215+0000 7f06dca72640 1 --2- 172.21.15.38:0/1237460246 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f06d80b2e40 0x7f06d814e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:17.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:17.215+0000 7f06dca72640 1 -- 172.21.15.38:0/1237460246 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f06d8162280 msgr2=0x7f06d81f53d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:17.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:17.215+0000 7f06dca72640 1 --2- 172.21.15.38:0/1237460246 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f06d8162280 0x7f06d81f53d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:17.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:17.215+0000 7f06dca72640 1 -- 172.21.15.38:0/1237460246 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06d81f5f80 con 0x7f06d80b2380 2024-09-21T03:42:17.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:17.215+0000 7f06d7fff640 1 --2- 172.21.15.38:0/1237460246 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f06d80b2e40 0x7f06d814e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:42:17.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:17.215+0000 7f06d77fe640 1 --2- 172.21.15.38:0/1237460246 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f06d8162280 0x7f06d81f53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:42:17.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:17.215+0000 7f06dca72640 1 --2- 172.21.15.38:0/1237460246 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f06d80b2380 0x7f06d80b2790 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7f06d0009bf0 tx=0x7f06d0003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:17.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:17.216+0000 7f06d4ff9640 1 -- 172.21.15.38:0/1237460246 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06d0044070 con 0x7f06d80b2380 2024-09-21T03:42:17.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:17.216+0000 7f06d4ff9640 1 -- 172.21.15.38:0/1237460246 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f06d0004250 con 0x7f06d80b2380 2024-09-21T03:42:17.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:17.216+0000 7f06de4d5640 1 -- 172.21.15.38:0/1237460246 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06d81f6260 con 0x7f06d80b2380 2024-09-21T03:42:17.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:17.216+0000 7f06d4ff9640 1 -- 172.21.15.38:0/1237460246 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06d0004660 con 0x7f06d80b2380 2024-09-21T03:42:17.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:17.216+0000 7f06de4d5640 1 -- 172.21.15.38:0/1237460246 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06d815ae60 con 0x7f06d80b2380 2024-09-21T03:42:17.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:17.217+0000 7f06de4d5640 1 -- 172.21.15.38:0/1237460246 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06d814dba0 con 0x7f06d80b2380 2024-09-21T03:42:17.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:17.218+0000 7f06d4ff9640 1 -- 172.21.15.38:0/1237460246 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f06d0047070 con 0x7f06d80b2380 2024-09-21T03:42:17.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:17.218+0000 7f06d4ff9640 1 --2- 172.21.15.38:0/1237460246 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f06c4078b90 0x7f06c407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:17.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:17.218+0000 7f06d4ff9640 1 -- 172.21.15.38:0/1237460246 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 7852+0+0 (secure 0 0 0) 0x7f06d00bc6b0 con 0x7f06d80b2380 2024-09-21T03:42:17.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:17.218+0000 7f06d7fff640 1 --2- 172.21.15.38:0/1237460246 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f06c4078b90 0x7f06c407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:17.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:17.219+0000 7f06d7fff640 1 --2- 172.21.15.38:0/1237460246 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f06c4078b90 0x7f06c407b0a0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f06c8005fd0 tx=0x7f06c800f360 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:17.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:17.222+0000 7f06d4ff9640 1 -- 172.21.15.38:0/1237460246 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f06d004e080 con 0x7f06d80b2380 2024-09-21T03:42:17.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:17.449+0000 7f06de4d5640 1 -- 172.21.15.38:0/1237460246 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f06d81f5d10 con 0x7f06d80b2380 2024-09-21T03:42:17.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:17.449+0000 7f06d4ff9640 1 -- 172.21.15.38:0/1237460246 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 198 v198) ==== 95+0+5234 (secure 0 0 0) 0x7f06d0032030 con 0x7f06d80b2380 2024-09-21T03:42:17.451 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:42:17.452 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":198,"btime":"2024-09-21T03:42:13:945248+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6086,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/965972663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":965972663},{"type":"v1","addr":"172.21.15.195:6836","nonce":965972663}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6700,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/3176122334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":3176122334},{"type":"v1","addr":"172.21.15.195:6837","nonce":3176122334}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":8925,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/4221150556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":4221150556},{"type":"v1","addr":"172.21.15.38:6833","nonce":4221150556}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":196}],"filesystems":[{"mdsmap":{"epoch":198,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:40:02.319282+0000","modified":"2024-09-21T03:42:13.945241+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":114,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8862},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8862":{"gid":8862,"name":"a","rank":0,"incarnation":195,"state":"up:active","state_seq":4,"addr":"172.21.15.38:6835/2712360296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":2712360296},{"type":"v1","addr":"172.21.15.38:6835","nonce":2712360296}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8862,"qdb_cluster":[8862]},"id":6}]} 2024-09-21T03:42:17.452 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 198 2024-09-21T03:42:17.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:17.451+0000 7f06de4d5640 1 -- 172.21.15.38:0/1237460246 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f06c4078b90 msgr2=0x7f06c407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:17.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:17.451+0000 7f06de4d5640 1 --2- 172.21.15.38:0/1237460246 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f06c4078b90 0x7f06c407b0a0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f06c8005fd0 tx=0x7f06c800f360 comp rx=0 tx=0).stop 2024-09-21T03:42:17.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:17.451+0000 7f06de4d5640 1 -- 172.21.15.38:0/1237460246 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f06d80b2380 msgr2=0x7f06d80b2790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:17.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:17.451+0000 7f06de4d5640 1 --2- 172.21.15.38:0/1237460246 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f06d80b2380 0x7f06d80b2790 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7f06d0009bf0 tx=0x7f06d0003ce0 comp rx=0 tx=0).stop 2024-09-21T03:42:17.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:17.452+0000 7f06de4d5640 1 -- 172.21.15.38:0/1237460246 shutdown_connections 2024-09-21T03:42:17.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:17.452+0000 7f06de4d5640 1 --2- 172.21.15.38:0/1237460246 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f06d80b2380 0x7f06d80b2790 unknown :-1 s=CLOSED pgs=1679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:17.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:17.452+0000 7f06de4d5640 1 --2- 172.21.15.38:0/1237460246 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f06d8162280 0x7f06d81f53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:17.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:17.452+0000 7f06de4d5640 1 --2- 172.21.15.38:0/1237460246 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f06c4078b90 0x7f06c407b0a0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:17.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:17.452+0000 7f06de4d5640 1 --2- 172.21.15.38:0/1237460246 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f06d80b2e40 0x7f06d814e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:17.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:17.452+0000 7f06de4d5640 1 -- 172.21.15.38:0/1237460246 >> 172.21.15.38:0/1237460246 conn(0x7f06d80b6b20 msgr2=0x7f06d806df30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:17.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:17.452+0000 7f06de4d5640 1 -- 172.21.15.38:0/1237460246 shutdown_connections 2024-09-21T03:42:17.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:17.452+0000 7f06de4d5640 1 -- 172.21.15.38:0/1237460246 wait complete. 2024-09-21T03:42:17.867 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:42:17.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:17.987+0000 7f80ef48b640 1 -- 172.21.15.38:0/3446798314 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f80e80b2e40 msgr2=0x7f80e80b3250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:17.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:17.987+0000 7f80ef48b640 1 --2- 172.21.15.38:0/3446798314 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f80e80b2e40 0x7f80e80b3250 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7f80e80b7460 tx=0x7f80e002aa30 comp rx=0 tx=0).stop 2024-09-21T03:42:17.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:17.987+0000 7f80ef48b640 1 -- 172.21.15.38:0/3446798314 shutdown_connections 2024-09-21T03:42:17.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:17.987+0000 7f80ef48b640 1 --2- 172.21.15.38:0/3446798314 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f80e80b2e40 0x7f80e80b3250 unknown :-1 s=CLOSED pgs=1680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:17.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:17.987+0000 7f80ef48b640 1 --2- 172.21.15.38:0/3446798314 >> v2:172.21.15.195:3300/0 conn(0x7f80e8162280 0x7f80e8162690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:17.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:17.987+0000 7f80ef48b640 1 -- 172.21.15.38:0/3446798314 >> 172.21.15.38:0/3446798314 conn(0x7f80e80b6b20 msgr2=0x7f80e80d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:17.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:17.987+0000 7f80ef48b640 1 -- 172.21.15.38:0/3446798314 shutdown_connections 2024-09-21T03:42:17.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:17.987+0000 7f80ef48b640 1 -- 172.21.15.38:0/3446798314 wait complete. 2024-09-21T03:42:17.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:17.989+0000 7f80ef48b640 1 Processor -- start 2024-09-21T03:42:17.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:17.989+0000 7f80ef48b640 1 -- start start 2024-09-21T03:42:17.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:17.990+0000 7f80ef48b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f80e80b2e40 0x7f80e80ced50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:17.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:17.990+0000 7f80ef48b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f80e8162280 0x7f80e80b1a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:17.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:17.990+0000 7f80ef48b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f80e81f8330 0x7f80e81f8740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:17.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:17.990+0000 7f80ef48b640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f80e815e4b0 con 0x7f80e8162280 2024-09-21T03:42:17.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:17.990+0000 7f80ef48b640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f80e80b39a0 con 0x7f80e80b2e40 2024-09-21T03:42:17.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:17.990+0000 7f80ef48b640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f80e815e630 con 0x7f80e81f8330 2024-09-21T03:42:17.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:17.990+0000 7f80eca26640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f80e8162280 0x7f80e80b1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:17.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:17.990+0000 7f80eca26640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f80e8162280 0x7f80e80b1a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34278/0 (socket says 172.21.15.38:34278) 2024-09-21T03:42:17.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:17.990+0000 7f80eca26640 1 -- 172.21.15.38:0/1625169693 learned_addr learned my addr 172.21.15.38:0/1625169693 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:42:17.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:17.990+0000 7f80ed227640 1 --2- 172.21.15.38:0/1625169693 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f80e80b2e40 0x7f80e80ced50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:17.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:17.990+0000 7f80eda28640 1 --2- 172.21.15.38:0/1625169693 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f80e81f8330 0x7f80e81f8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:17.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:17.991+0000 7f80eca26640 1 -- 172.21.15.38:0/1625169693 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f80e81f8330 msgr2=0x7f80e81f8740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:17.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:17.991+0000 7f80eca26640 1 --2- 172.21.15.38:0/1625169693 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f80e81f8330 0x7f80e81f8740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:17.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:17.991+0000 7f80eca26640 1 -- 172.21.15.38:0/1625169693 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f80e80b2e40 msgr2=0x7f80e80ced50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:17.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:17.991+0000 7f80eca26640 1 --2- 172.21.15.38:0/1625169693 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f80e80b2e40 0x7f80e80ced50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:17.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:17.991+0000 7f80eca26640 1 -- 172.21.15.38:0/1625169693 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80e81f8c80 con 0x7f80e8162280 2024-09-21T03:42:17.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:17.991+0000 7f80eca26640 1 --2- 172.21.15.38:0/1625169693 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f80e8162280 0x7f80e80b1a00 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7f80d800d740 tx=0x7f80d800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:17.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:17.991+0000 7f80ddffb640 1 -- 172.21.15.38:0/1625169693 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80d8017070 con 0x7f80e8162280 2024-09-21T03:42:17.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:17.991+0000 7f80ddffb640 1 -- 172.21.15.38:0/1625169693 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f80d8009d20 con 0x7f80e8162280 2024-09-21T03:42:17.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:17.991+0000 7f80ef48b640 1 -- 172.21.15.38:0/1625169693 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80e80b25e0 con 0x7f80e8162280 2024-09-21T03:42:17.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:17.991+0000 7f80ddffb640 1 -- 172.21.15.38:0/1625169693 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80d80106c0 con 0x7f80e8162280 2024-09-21T03:42:17.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:17.991+0000 7f80ef48b640 1 -- 172.21.15.38:0/1625169693 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80e80b2ba0 con 0x7f80e8162280 2024-09-21T03:42:17.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:17.992+0000 7f80ef48b640 1 -- 172.21.15.38:0/1625169693 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80e80b2350 con 0x7f80e8162280 2024-09-21T03:42:17.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:17.993+0000 7f80ddffb640 1 -- 172.21.15.38:0/1625169693 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f80d801a070 con 0x7f80e8162280 2024-09-21T03:42:17.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:17.994+0000 7f80ddffb640 1 --2- 172.21.15.38:0/1625169693 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f80cc078b90 0x7f80cc07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:17.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:17.994+0000 7f80ddffb640 1 -- 172.21.15.38:0/1625169693 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 7852+0+0 (secure 0 0 0) 0x7f80d809b5a0 con 0x7f80e8162280 2024-09-21T03:42:17.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:17.994+0000 7f80ed227640 1 --2- 172.21.15.38:0/1625169693 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f80cc078b90 0x7f80cc07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:17.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:17.994+0000 7f80ed227640 1 --2- 172.21.15.38:0/1625169693 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f80cc078b90 0x7f80cc07b0a0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f80e4001720 tx=0x7f80e4009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:18.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:17.998+0000 7f80ddffb640 1 -- 172.21.15.38:0/1625169693 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f80e81f8c80 con 0x7f80e8162280 2024-09-21T03:42:18.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.223+0000 7f80ef48b640 1 -- 172.21.15.38:0/1625169693 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f80e8162f00 con 0x7f80e8162280 2024-09-21T03:42:18.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.224+0000 7f80ddffb640 1 -- 172.21.15.38:0/1625169693 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 198 v198) ==== 95+0+5234 (secure 0 0 0) 0x7f80d8062c50 con 0x7f80e8162280 2024-09-21T03:42:18.226 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:42:18.226 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":198,"btime":"2024-09-21T03:42:13:945248+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6086,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/965972663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":965972663},{"type":"v1","addr":"172.21.15.195:6836","nonce":965972663}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6700,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/3176122334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":3176122334},{"type":"v1","addr":"172.21.15.195:6837","nonce":3176122334}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":8925,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/4221150556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":4221150556},{"type":"v1","addr":"172.21.15.38:6833","nonce":4221150556}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":196}],"filesystems":[{"mdsmap":{"epoch":198,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:40:02.319282+0000","modified":"2024-09-21T03:42:13.945241+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":114,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8862},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8862":{"gid":8862,"name":"a","rank":0,"incarnation":195,"state":"up:active","state_seq":4,"addr":"172.21.15.38:6835/2712360296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":2712360296},{"type":"v1","addr":"172.21.15.38:6835","nonce":2712360296}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8862,"qdb_cluster":[8862]},"id":6}]} 2024-09-21T03:42:18.226 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 198 2024-09-21T03:42:18.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.226+0000 7f80ef48b640 1 -- 172.21.15.38:0/1625169693 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f80cc078b90 msgr2=0x7f80cc07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:18.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.226+0000 7f80ef48b640 1 --2- 172.21.15.38:0/1625169693 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f80cc078b90 0x7f80cc07b0a0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f80e4001720 tx=0x7f80e4009040 comp rx=0 tx=0).stop 2024-09-21T03:42:18.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.226+0000 7f80ef48b640 1 -- 172.21.15.38:0/1625169693 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f80e8162280 msgr2=0x7f80e80b1a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:18.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.226+0000 7f80ef48b640 1 --2- 172.21.15.38:0/1625169693 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f80e8162280 0x7f80e80b1a00 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7f80d800d740 tx=0x7f80d800dc00 comp rx=0 tx=0).stop 2024-09-21T03:42:18.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.226+0000 7f80ef48b640 1 -- 172.21.15.38:0/1625169693 shutdown_connections 2024-09-21T03:42:18.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.226+0000 7f80ef48b640 1 --2- 172.21.15.38:0/1625169693 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f80cc078b90 0x7f80cc07b0a0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:18.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.226+0000 7f80ef48b640 1 --2- 172.21.15.38:0/1625169693 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f80e81f8330 0x7f80e81f8740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:18.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.226+0000 7f80ef48b640 1 --2- 172.21.15.38:0/1625169693 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f80e8162280 0x7f80e80b1a00 unknown :-1 s=CLOSED pgs=1681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:18.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.226+0000 7f80ef48b640 1 --2- 172.21.15.38:0/1625169693 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f80e80b2e40 0x7f80e80ced50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:18.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.226+0000 7f80ef48b640 1 -- 172.21.15.38:0/1625169693 >> 172.21.15.38:0/1625169693 conn(0x7f80e80b6b20 msgr2=0x7f80e815ea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:18.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.226+0000 7f80ef48b640 1 -- 172.21.15.38:0/1625169693 shutdown_connections 2024-09-21T03:42:18.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.226+0000 7f80ef48b640 1 -- 172.21.15.38:0/1625169693 wait complete. 2024-09-21T03:42:18.239 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 198, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T03:40:02.319282+0000', 'modified': '2024-09-21T03:42:13.945241+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 114, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8862}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8862': {'gid': 8862, 'name': 'a', 'rank': 0, 'incarnation': 195, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.38:6835/2712360296', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.38:6834', 'nonce': 2712360296}, {'type': 'v1', 'addr': '172.21.15.38:6835', 'nonce': 2712360296}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 8862, 'qdb_cluster': [8862]} 2024-09-21T03:42:18.239 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-21T03:42:18.240 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-21T03:42:18.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.356+0000 7f2d83dc3640 1 -- 172.21.15.38:0/834850613 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2d640099b0 msgr2=0x7f2d6400bdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:18.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.356+0000 7f2d83dc3640 1 --2- 172.21.15.38:0/834850613 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2d640099b0 0x7f2d6400bdd0 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7f2d740016c0 tx=0x7f2d64005830 comp rx=0 tx=0).stop 2024-09-21T03:42:18.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.356+0000 7f2d83dc3640 1 -- 172.21.15.38:0/834850613 shutdown_connections 2024-09-21T03:42:18.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.356+0000 7f2d83dc3640 1 --2- 172.21.15.38:0/834850613 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2d640099b0 0x7f2d6400bdd0 unknown :-1 s=CLOSED pgs=1683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:18.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.356+0000 7f2d83dc3640 1 -- 172.21.15.38:0/834850613 >> 172.21.15.38:0/834850613 conn(0x7f2d7c09d830 msgr2=0x7f2d7c0b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:18.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.356+0000 7f2d83dc3640 1 -- 172.21.15.38:0/834850613 shutdown_connections 2024-09-21T03:42:18.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.356+0000 7f2d83dc3640 1 -- 172.21.15.38:0/834850613 wait complete. 2024-09-21T03:42:18.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.358+0000 7f2d83dc3640 1 Processor -- start 2024-09-21T03:42:18.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.358+0000 7f2d83dc3640 1 -- start start 2024-09-21T03:42:18.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.359+0000 7f2d83dc3640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2d640099b0 0x7f2d7c06cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:18.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.359+0000 7f2d83dc3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2d7c1f62d0 0x7f2d7c14c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:18.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.359+0000 7f2d83dc3640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2d7c159ff0 0x7f2d7c15a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:18.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.359+0000 7f2d83dc3640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f2d7c0b9c50 con 0x7f2d7c1f62d0 2024-09-21T03:42:18.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.359+0000 7f2d83dc3640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f2d7c0b9ad0 con 0x7f2d640099b0 2024-09-21T03:42:18.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.359+0000 7f2d83dc3640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f2d7c0b9dd0 con 0x7f2d7c159ff0 2024-09-21T03:42:18.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.359+0000 7f2d8135e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2d7c1f62d0 0x7f2d7c14c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:18.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.359+0000 7f2d8135e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2d7c1f62d0 0x7f2d7c14c940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34290/0 (socket says 172.21.15.38:34290) 2024-09-21T03:42:18.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.359+0000 7f2d8135e640 1 -- 172.21.15.38:0/4004053647 learned_addr learned my addr 172.21.15.38:0/4004053647 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:42:18.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.359+0000 7f2d8135e640 1 -- 172.21.15.38:0/4004053647 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2d7c159ff0 msgr2=0x7f2d7c15a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:18.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.359+0000 7f2d8135e640 1 --2- 172.21.15.38:0/4004053647 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2d7c159ff0 0x7f2d7c15a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:18.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.359+0000 7f2d8135e640 1 -- 172.21.15.38:0/4004053647 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2d640099b0 msgr2=0x7f2d7c06cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:18.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.359+0000 7f2d8135e640 1 --2- 172.21.15.38:0/4004053647 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2d640099b0 0x7f2d7c06cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:18.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.359+0000 7f2d8135e640 1 -- 172.21.15.38:0/4004053647 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d7c159550 con 0x7f2d7c1f62d0 2024-09-21T03:42:18.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.360+0000 7f2d8135e640 1 --2- 172.21.15.38:0/4004053647 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2d7c1f62d0 0x7f2d7c14c940 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7f2d70004e90 tx=0x7f2d7000b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:18.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.360+0000 7f2d7a7fc640 1 -- 172.21.15.38:0/4004053647 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d70004440 con 0x7f2d7c1f62d0 2024-09-21T03:42:18.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.360+0000 7f2d7a7fc640 1 -- 172.21.15.38:0/4004053647 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2d70015030 con 0x7f2d7c1f62d0 2024-09-21T03:42:18.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.360+0000 7f2d83dc3640 1 -- 172.21.15.38:0/4004053647 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d7c14ce80 con 0x7f2d7c1f62d0 2024-09-21T03:42:18.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.360+0000 7f2d7a7fc640 1 -- 172.21.15.38:0/4004053647 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d70003850 con 0x7f2d7c1f62d0 2024-09-21T03:42:18.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.360+0000 7f2d83dc3640 1 -- 172.21.15.38:0/4004053647 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d7c15af70 con 0x7f2d7c1f62d0 2024-09-21T03:42:18.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.362+0000 7f2d7a7fc640 1 -- 172.21.15.38:0/4004053647 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f2d700039f0 con 0x7f2d7c1f62d0 2024-09-21T03:42:18.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.363+0000 7f2d7a7fc640 1 --2- 172.21.15.38:0/4004053647 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2d68078b90 0x7f2d6807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:18.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.363+0000 7f2d7a7fc640 1 -- 172.21.15.38:0/4004053647 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 7852+0+0 (secure 0 0 0) 0x7f2d7009c100 con 0x7f2d7c1f62d0 2024-09-21T03:42:18.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.363+0000 7f2d81b5f640 1 --2- 172.21.15.38:0/4004053647 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2d68078b90 0x7f2d6807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:18.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.363+0000 7f2d8ae13740 1 Processor -- start 2024-09-21T03:42:18.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.363+0000 7f2d8ae13740 1 -- start start 2024-09-21T03:42:18.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.364+0000 7f2d81b5f640 1 --2- 172.21.15.38:0/4004053647 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2d68078b90 0x7f2d6807b0a0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c001c00 tx=0x7f2d6c0014a0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:18.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.365+0000 7f2d8ae13740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e6c6f50270 0x55e6c7321fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:18.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.365+0000 7f2d8ae13740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e6c6f273b0 0x55e6c73224e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:18.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.365+0000 7f2d8ae13740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e6c6f2f590 0x55e6c733f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:18.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.365+0000 7f2d8ae13740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55e6c726a820 con 0x55e6c6f50270 2024-09-21T03:42:18.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.365+0000 7f2d8ae13740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55e6c7259c50 con 0x55e6c6f2f590 2024-09-21T03:42:18.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.365+0000 7f2d8ae13740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55e6c725da60 con 0x55e6c6f273b0 2024-09-21T03:42:18.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.365+0000 7f2d81b5f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e6c6f50270 0x55e6c7321fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:18.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.365+0000 7f2d81b5f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e6c6f50270 0x55e6c7321fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34292/0 (socket says 172.21.15.38:34292) 2024-09-21T03:42:18.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.365+0000 7f2d81b5f640 1 -- 172.21.15.38:0/4226206157 learned_addr learned my addr 172.21.15.38:0/4226206157 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:42:18.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.365+0000 7f2d8135e640 1 --2- 172.21.15.38:0/4226206157 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e6c6f273b0 0x55e6c73224e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:18.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.365+0000 7f2d82360640 1 --2- 172.21.15.38:0/4226206157 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e6c6f2f590 0x55e6c733f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:18.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.365+0000 7f2d81b5f640 1 -- 172.21.15.38:0/4226206157 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e6c6f273b0 msgr2=0x55e6c73224e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:18.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.365+0000 7f2d81b5f640 1 --2- 172.21.15.38:0/4226206157 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e6c6f273b0 0x55e6c73224e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:18.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.365+0000 7f2d81b5f640 1 -- 172.21.15.38:0/4226206157 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e6c6f2f590 msgr2=0x55e6c733f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:18.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.365+0000 7f2d81b5f640 1 --2- 172.21.15.38:0/4226206157 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e6c6f2f590 0x55e6c733f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:18.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.365+0000 7f2d81b5f640 1 -- 172.21.15.38:0/4226206157 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6c6fbb850 con 0x55e6c6f50270 2024-09-21T03:42:18.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.365+0000 7f2d81b5f640 1 --2- 172.21.15.38:0/4226206157 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e6c6f50270 0x55e6c7321fa0 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c006600 tx=0x7f2d6c012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af15d977ca9c0579 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:18.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.366+0000 7f2d637fe640 1 -- 172.21.15.38:0/4226206157 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d6c011c70 con 0x55e6c6f50270 2024-09-21T03:42:18.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.366+0000 7f2d637fe640 1 -- 172.21.15.38:0/4226206157 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2d6c011e10 con 0x55e6c6f50270 2024-09-21T03:42:18.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.366+0000 7f2d637fe640 1 -- 172.21.15.38:0/4226206157 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d6c01a640 con 0x55e6c6f50270 2024-09-21T03:42:18.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.366+0000 7f2d8ae13740 1 -- 172.21.15.38:0/4226206157 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e6c6f50270 msgr2=0x55e6c7321fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:18.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.366+0000 7f2d8ae13740 1 --2- 172.21.15.38:0/4226206157 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e6c6f50270 0x55e6c7321fa0 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c006600 tx=0x7f2d6c012000 comp rx=0 tx=0).stop 2024-09-21T03:42:18.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.366+0000 7f2d8ae13740 1 -- 172.21.15.38:0/4226206157 shutdown_connections 2024-09-21T03:42:18.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.366+0000 7f2d8ae13740 1 --2- 172.21.15.38:0/4226206157 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e6c6f2f590 0x55e6c733f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:18.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.366+0000 7f2d8ae13740 1 --2- 172.21.15.38:0/4226206157 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e6c6f273b0 0x55e6c73224e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:18.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.366+0000 7f2d8ae13740 1 --2- 172.21.15.38:0/4226206157 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e6c6f50270 0x55e6c7321fa0 unknown :-1 s=CLOSED pgs=1685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:18.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.366+0000 7f2d8ae13740 1 -- 172.21.15.38:0/4226206157 >> 172.21.15.38:0/4226206157 conn(0x55e6c6f443c0 msgr2=0x55e6c72f8260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:18.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.367+0000 7f2d8ae13740 1 -- 172.21.15.38:0/4226206157 shutdown_connections 2024-09-21T03:42:18.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.367+0000 7f2d8ae13740 1 -- 172.21.15.38:0/4226206157 wait complete. 2024-09-21T03:42:18.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.367+0000 7f2d8ae13740 1 -- start start 2024-09-21T03:42:18.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.367+0000 7f2d8ae13740 1 Processor -- start 2024-09-21T03:42:18.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.369+0000 7f2d8ae13740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e6c6f273b0 0x55e6c7333b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:18.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.369+0000 7f2d8ae13740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e6c6f2f590 0x55e6c733d770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:18.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.369+0000 7f2d8ae13740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e6c6f50270 0x55e6c731e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:18.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.369+0000 7f2d8ae13740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55e6c6fca620 con 0x55e6c6f2f590 2024-09-21T03:42:18.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.369+0000 7f2d8ae13740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55e6c71ff5f0 con 0x55e6c6f273b0 2024-09-21T03:42:18.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.369+0000 7f2d8ae13740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55e6c7322a20 con 0x55e6c6f50270 2024-09-21T03:42:18.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.369+0000 7f2d8ae13740 10 client.0 _ll_register_callbacks cb 0x55e6c72030a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:42:18.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.369+0000 7f2d8135e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e6c6f2f590 0x55e6c733d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:18.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.369+0000 7f2d8135e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e6c6f2f590 0x55e6c733d770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34298/0 (socket says 172.21.15.38:34298) 2024-09-21T03:42:18.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.369+0000 7f2d8135e640 1 -- 172.21.15.38:0/1788710056 learned_addr learned my addr 172.21.15.38:0/1788710056 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:42:18.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.369+0000 7f2d81b5f640 1 --2- 172.21.15.38:0/1788710056 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e6c6f273b0 0x55e6c7333b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:18.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.369+0000 7f2d82360640 1 --2- 172.21.15.38:0/1788710056 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e6c6f50270 0x55e6c731e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:18.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.369+0000 7f2d8135e640 1 -- 172.21.15.38:0/1788710056 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e6c6f50270 msgr2=0x55e6c731e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:18.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.369+0000 7f2d8135e640 1 --2- 172.21.15.38:0/1788710056 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e6c6f50270 0x55e6c731e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:18.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.369+0000 7f2d8135e640 1 -- 172.21.15.38:0/1788710056 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e6c6f273b0 msgr2=0x55e6c7333b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:18.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.369+0000 7f2d8135e640 1 --2- 172.21.15.38:0/1788710056 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e6c6f273b0 0x55e6c7333b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:18.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.369+0000 7f2d8135e640 1 -- 172.21.15.38:0/1788710056 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e6c70326b0 con 0x55e6c6f2f590 2024-09-21T03:42:18.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.370+0000 7f2d8135e640 1 --2- 172.21.15.38:0/1788710056 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e6c6f2f590 0x55e6c733d770 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7f2d70063570 tx=0x7f2d70062e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bde378ceca95a8ce server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:18.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.370+0000 7f2d60ff9640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-21T03:42:18.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.370+0000 7f2d60ff9640 1 -- 172.21.15.38:0/1788710056 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d70082060 con 0x55e6c6f2f590 2024-09-21T03:42:18.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.370+0000 7f2d60ff9640 1 -- 172.21.15.38:0/1788710056 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 7852+0+0 (secure 0 0 0) 0x7f2d7007db30 con 0x55e6c6f2f590 2024-09-21T03:42:18.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.370+0000 7f2d8ae13740 1 -- 172.21.15.38:0/1788710056 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d6c011e10 con 0x55e6c6f2f590 2024-09-21T03:42:18.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.370+0000 7f2d60ff9640 1 -- 172.21.15.38:0/1788710056 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=198) ==== 24+0+0 (secure 0 0 0) 0x7f2d7007d070 con 0x55e6c6f2f590 2024-09-21T03:42:18.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.370+0000 7f2d8ae13740 10 client.8955 fetch_fsmap learned FSMap version 198 2024-09-21T03:42:18.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.370+0000 7f2d8ae13740 1 -- 172.21.15.38:0/1788710056 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=198,monmap=2+}) -- 0x55e6c734fdc0 con 0x55e6c6f2f590 2024-09-21T03:42:18.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.370+0000 7f2d60ff9640 1 -- 172.21.15.38:0/1788710056 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2d6c011e10 con 0x55e6c6f2f590 2024-09-21T03:42:18.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.370+0000 7f2d60ff9640 1 -- 172.21.15.38:0/1788710056 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 198) ==== 3502+0+0 (secure 0 0 0) 0x7f2d70062840 con 0x55e6c6f2f590 2024-09-21T03:42:18.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.370+0000 7f2d8ae13740 10 client.8955 fetch_fsmap finished waiting for FSMap version 198 2024-09-21T03:42:18.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.370+0000 7f2d8ae13740 10 client.8955 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-21T03:42:18.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.370+0000 7f2d8ae13740 20 client.8955 populate_metadata read hostname 'smithi038' 2024-09-21T03:42:18.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.370+0000 7f2d8ae13740 1 --2- 172.21.15.38:0/1788710056 >> [v2:172.21.15.38:6834/2712360296,v1:172.21.15.38:6835/2712360296] conn(0x55e6c6f0ee40 0x55e6c7320d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:18.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.370+0000 7f2d8ae13740 4 client.8955 mds_command: new command op to 8862 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:42:18.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.370+0000 7f2d8ae13740 1 -- 172.21.15.38:0/1788710056 --> [v2:172.21.15.38:6834/2712360296,v1:172.21.15.38:6835/2712360296] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e6c726c770 con 0x55e6c6f0ee40 2024-09-21T03:42:18.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.370+0000 7f2d81b5f640 1 --2- 172.21.15.38:0/1788710056 >> [v2:172.21.15.38:6834/2712360296,v1:172.21.15.38:6835/2712360296] conn(0x55e6c6f0ee40 0x55e6c7320d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:18.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.371+0000 7f2d60ff9640 10 client.8955 ms_handle_connect on v2:172.21.15.38:6834/2712360296 2024-09-21T03:42:18.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.371+0000 7f2d81b5f640 1 --2- 172.21.15.38:0/1788710056 >> [v2:172.21.15.38:6834/2712360296,v1:172.21.15.38:6835/2712360296] conn(0x55e6c6f0ee40 0x55e6c7320d20 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=271b1de92e258bc3 server_cookie=76d6e79752a33a70 in_seq=0 out_seq=0 2024-09-21T03:42:18.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.372+0000 7f2d60ff9640 1 -- 172.21.15.38:0/1788710056 <== mds.0 v2:172.21.15.38:6834/2712360296 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e6c726c770 con 0x55e6c6f0ee40 2024-09-21T03:42:18.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.372+0000 7f2d60ff9640 10 client.8955 handle_command_reply: tid=0 2024-09-21T03:42:18.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.372+0000 7f2d8ae13740 1 client.8955 shutdown 2024-09-21T03:42:18.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.372+0000 7f2d8ae13740 1 -- 172.21.15.38:0/1788710056 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e6c6f2f590 msgr2=0x55e6c733d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:18.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.372+0000 7f2d8ae13740 1 --2- 172.21.15.38:0/1788710056 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e6c6f2f590 0x55e6c733d770 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7f2d70063570 tx=0x7f2d70062e70 comp rx=0 tx=0).stop 2024-09-21T03:42:18.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.372+0000 7f2d8ae13740 1 -- 172.21.15.38:0/1788710056 shutdown_connections 2024-09-21T03:42:18.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.372+0000 7f2d8ae13740 1 --2- 172.21.15.38:0/1788710056 >> [v2:172.21.15.38:6834/2712360296,v1:172.21.15.38:6835/2712360296] conn(0x55e6c6f0ee40 0x55e6c7320d20 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:18.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.372+0000 7f2d8ae13740 1 --2- 172.21.15.38:0/1788710056 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e6c6f50270 0x55e6c731e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:18.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.372+0000 7f2d60ff9640 0 client.8955 ms_handle_reset on v2:172.21.15.38:6834/2712360296 2024-09-21T03:42:18.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.372+0000 7f2d8ae13740 1 --2- 172.21.15.38:0/1788710056 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e6c6f2f590 0x55e6c733d770 unknown :-1 s=CLOSED pgs=1686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:18.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.372+0000 7f2d8ae13740 1 --2- 172.21.15.38:0/1788710056 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e6c6f273b0 0x55e6c7333b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:18.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.372+0000 7f2d8ae13740 1 -- 172.21.15.38:0/1788710056 >> 172.21.15.38:0/1788710056 conn(0x55e6c6f443c0 msgr2=0x55e6c733e9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:18.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.372+0000 7f2d8ae13740 1 -- 172.21.15.38:0/1788710056 shutdown_connections 2024-09-21T03:42:18.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.372+0000 7f2d8ae13740 1 -- 172.21.15.38:0/1788710056 wait complete. 2024-09-21T03:42:18.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.372+0000 7f2d8ae13740 20 client.8955 trim_cache size 0 max 16384 2024-09-21T03:42:18.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.391+0000 7f2d8ae13740 1 Processor -- start 2024-09-21T03:42:18.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.391+0000 7f2d8ae13740 1 -- start start 2024-09-21T03:42:18.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.392+0000 7f2d8ae13740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e6c6f273b0 0x55e6c731e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:18.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.392+0000 7f2d8ae13740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e6c6f2f590 0x55e6c735f630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:18.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.392+0000 7f2d8ae13740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e6c6f50270 0x55e6c735fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:18.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.392+0000 7f2d8ae13740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f2d7c202e30 con 0x55e6c6f273b0 2024-09-21T03:42:18.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.392+0000 7f2d8ae13740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f2d7c202110 con 0x55e6c6f2f590 2024-09-21T03:42:18.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.392+0000 7f2d8ae13740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f2d7c202760 con 0x55e6c6f50270 2024-09-21T03:42:18.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.392+0000 7f2d81b5f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e6c6f273b0 0x55e6c731e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:18.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.392+0000 7f2d81b5f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e6c6f273b0 0x55e6c731e4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34306/0 (socket says 172.21.15.38:34306) 2024-09-21T03:42:18.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.392+0000 7f2d81b5f640 1 -- 172.21.15.38:0/2871434415 learned_addr learned my addr 172.21.15.38:0/2871434415 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:42:18.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.392+0000 7f2d81b5f640 1 -- 172.21.15.38:0/2871434415 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e6c6f50270 msgr2=0x55e6c735fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:18.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.392+0000 7f2d81b5f640 1 --2- 172.21.15.38:0/2871434415 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e6c6f50270 0x55e6c735fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:18.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.392+0000 7f2d8135e640 1 --2- 172.21.15.38:0/2871434415 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e6c6f2f590 0x55e6c735f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:18.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.392+0000 7f2d81b5f640 1 -- 172.21.15.38:0/2871434415 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e6c6f2f590 msgr2=0x55e6c735f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:18.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.392+0000 7f2d81b5f640 1 --2- 172.21.15.38:0/2871434415 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e6c6f2f590 0x55e6c735f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:18.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.392+0000 7f2d81b5f640 1 -- 172.21.15.38:0/2871434415 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6c7333f00 con 0x55e6c6f273b0 2024-09-21T03:42:18.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.392+0000 7f2d8135e640 1 --2- 172.21.15.38:0/2871434415 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e6c6f2f590 0x55e6c735f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:42:18.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.393+0000 7f2d81b5f640 1 --2- 172.21.15.38:0/2871434415 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e6c6f273b0 0x55e6c731e4a0 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x55e6c72f9e30 tx=0x7f2d6c015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9d319d0bd3e3537 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:18.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.393+0000 7f2d62ffd640 1 -- 172.21.15.38:0/2871434415 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d6c011d70 con 0x55e6c6f273b0 2024-09-21T03:42:18.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.393+0000 7f2d62ffd640 1 -- 172.21.15.38:0/2871434415 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2d6c035420 con 0x55e6c6f273b0 2024-09-21T03:42:18.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.393+0000 7f2d62ffd640 1 -- 172.21.15.38:0/2871434415 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d6c0357f0 con 0x55e6c6f273b0 2024-09-21T03:42:18.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.394+0000 7f2d8ae13740 1 -- 172.21.15.38:0/2871434415 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e6c6f273b0 msgr2=0x55e6c731e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:18.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.394+0000 7f2d8ae13740 1 --2- 172.21.15.38:0/2871434415 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e6c6f273b0 0x55e6c731e4a0 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x55e6c72f9e30 tx=0x7f2d6c015000 comp rx=0 tx=0).stop 2024-09-21T03:42:18.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.394+0000 7f2d8ae13740 1 -- 172.21.15.38:0/2871434415 shutdown_connections 2024-09-21T03:42:18.398 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.394+0000 7f2d8ae13740 1 --2- 172.21.15.38:0/2871434415 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e6c6f50270 0x55e6c735fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:18.398 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.394+0000 7f2d8ae13740 1 --2- 172.21.15.38:0/2871434415 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e6c6f2f590 0x55e6c735f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:18.398 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.394+0000 7f2d8ae13740 1 --2- 172.21.15.38:0/2871434415 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e6c6f273b0 0x55e6c731e4a0 unknown :-1 s=CLOSED pgs=1687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:18.398 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.394+0000 7f2d8ae13740 1 -- 172.21.15.38:0/2871434415 >> 172.21.15.38:0/2871434415 conn(0x55e6c6f443c0 msgr2=0x55e6c73106e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:18.398 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.394+0000 7f2d8ae13740 1 -- 172.21.15.38:0/2871434415 shutdown_connections 2024-09-21T03:42:18.398 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.394+0000 7f2d8ae13740 1 -- 172.21.15.38:0/2871434415 wait complete. 2024-09-21T03:42:18.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.394+0000 7f2d8ae13740 1 -- start start 2024-09-21T03:42:18.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.395+0000 7f2d8ae13740 1 Processor -- start 2024-09-21T03:42:18.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.396+0000 7f2d8ae13740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e6c6f273b0 0x55e6c73be150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:18.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.396+0000 7f2d8ae13740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e6c6f2f590 0x55e6c735e620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:18.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.396+0000 7f2d8ae13740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e6c6f50270 0x55e6c735eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:18.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.396+0000 7f2d8ae13740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55e6c734f2d0 con 0x55e6c6f273b0 2024-09-21T03:42:18.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.396+0000 7f2d8ae13740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55e6c7334160 con 0x55e6c6f2f590 2024-09-21T03:42:18.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.396+0000 7f2d8ae13740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55e6c730f9c0 con 0x55e6c6f50270 2024-09-21T03:42:18.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.396+0000 7f2d8ae13740 10 client.0 _ll_register_callbacks cb 0x7f2d700630f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:42:18.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.396+0000 7f2d81b5f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e6c6f273b0 0x55e6c73be150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:18.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.396+0000 7f2d81b5f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e6c6f273b0 0x55e6c73be150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34318/0 (socket says 172.21.15.38:34318) 2024-09-21T03:42:18.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.396+0000 7f2d81b5f640 1 -- 172.21.15.38:0/1499522721 learned_addr learned my addr 172.21.15.38:0/1499522721 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:42:18.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.396+0000 7f2d82360640 1 --2- 172.21.15.38:0/1499522721 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e6c6f50270 0x55e6c735eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:18.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.396+0000 7f2d8135e640 1 --2- 172.21.15.38:0/1499522721 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e6c6f2f590 0x55e6c735e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:18.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.396+0000 7f2d81b5f640 1 -- 172.21.15.38:0/1499522721 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e6c6f50270 msgr2=0x55e6c735eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:18.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.396+0000 7f2d81b5f640 1 --2- 172.21.15.38:0/1499522721 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e6c6f50270 0x55e6c735eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:18.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.396+0000 7f2d81b5f640 1 -- 172.21.15.38:0/1499522721 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e6c6f2f590 msgr2=0x55e6c735e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:18.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.396+0000 7f2d81b5f640 1 --2- 172.21.15.38:0/1499522721 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e6c6f2f590 0x55e6c735e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:18.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.396+0000 7f2d81b5f640 1 -- 172.21.15.38:0/1499522721 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e6c733dac0 con 0x55e6c6f273b0 2024-09-21T03:42:18.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.397+0000 7f2d81b5f640 1 --2- 172.21.15.38:0/1499522721 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e6c6f273b0 0x55e6c73be150 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c0013b0 tx=0x7f2d6c012b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c099b18bd24b0adf server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:18.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.397+0000 7f2d60ff9640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-21T03:42:18.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.397+0000 7f2d60ff9640 1 -- 172.21.15.38:0/1499522721 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d6c0355d0 con 0x55e6c6f273b0 2024-09-21T03:42:18.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.397+0000 7f2d8ae13740 1 -- 172.21.15.38:0/1499522721 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d6c035420 con 0x55e6c6f273b0 2024-09-21T03:42:18.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.397+0000 7f2d60ff9640 1 -- 172.21.15.38:0/1499522721 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 7852+0+0 (secure 0 0 0) 0x7f2d6c035770 con 0x55e6c6f273b0 2024-09-21T03:42:18.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.397+0000 7f2d60ff9640 1 -- 172.21.15.38:0/1499522721 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=198) ==== 24+0+0 (secure 0 0 0) 0x7f2d6c035970 con 0x55e6c6f273b0 2024-09-21T03:42:18.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.397+0000 7f2d8ae13740 10 client.8961 fetch_fsmap learned FSMap version 198 2024-09-21T03:42:18.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.397+0000 7f2d8ae13740 1 -- 172.21.15.38:0/1499522721 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=198,monmap=2+}) -- 0x55e6c735f390 con 0x55e6c6f273b0 2024-09-21T03:42:18.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.397+0000 7f2d60ff9640 1 -- 172.21.15.38:0/1499522721 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2d6c035420 con 0x55e6c6f273b0 2024-09-21T03:42:18.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.397+0000 7f2d60ff9640 1 -- 172.21.15.38:0/1499522721 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 198) ==== 3502+0+0 (secure 0 0 0) 0x7f2d6c017030 con 0x55e6c6f273b0 2024-09-21T03:42:18.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.397+0000 7f2d8ae13740 10 client.8961 fetch_fsmap finished waiting for FSMap version 198 2024-09-21T03:42:18.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.397+0000 7f2d8ae13740 10 client.8961 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-21T03:42:18.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.397+0000 7f2d8ae13740 20 client.8961 populate_metadata read hostname 'smithi038' 2024-09-21T03:42:18.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.397+0000 7f2d8ae13740 1 --2- 172.21.15.38:0/1499522721 >> [v2:172.21.15.38:6834/2712360296,v1:172.21.15.38:6835/2712360296] conn(0x55e6c7335400 0x55e6c73c7810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:18.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.397+0000 7f2d8ae13740 4 client.8961 mds_command: new command op to 8862 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T03:42:18.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.397+0000 7f2d8ae13740 1 -- 172.21.15.38:0/1499522721 --> [v2:172.21.15.38:6834/2712360296,v1:172.21.15.38:6835/2712360296] -- command(tid 0: {"prefix": "status"}) -- 0x55e6c7268220 con 0x55e6c7335400 2024-09-21T03:42:18.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.398+0000 7f2d8135e640 1 --2- 172.21.15.38:0/1499522721 >> [v2:172.21.15.38:6834/2712360296,v1:172.21.15.38:6835/2712360296] conn(0x55e6c7335400 0x55e6c73c7810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:18.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.398+0000 7f2d60ff9640 10 client.8961 ms_handle_connect on v2:172.21.15.38:6834/2712360296 2024-09-21T03:42:18.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.398+0000 7f2d8135e640 1 --2- 172.21.15.38:0/1499522721 >> [v2:172.21.15.38:6834/2712360296,v1:172.21.15.38:6835/2712360296] conn(0x55e6c7335400 0x55e6c73c7810 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=594f4519e147a9b9 server_cookie=88a77ef3366362f5 in_seq=0 out_seq=0 2024-09-21T03:42:18.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.398+0000 7f2d60ff9640 1 -- 172.21.15.38:0/1499522721 <== mds.0 v2:172.21.15.38:6834/2712360296 1 ==== command_reply(tid 0: 0 ) ==== 8+0+328 (crc 0 0 0) 0x55e6c7268220 con 0x55e6c7335400 2024-09-21T03:42:18.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.398+0000 7f2d60ff9640 10 client.8961 handle_command_reply: tid=0 2024-09-21T03:42:18.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.398+0000 7f2d8ae13740 1 client.8961 shutdown 2024-09-21T03:42:18.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.398+0000 7f2d8ae13740 1 -- 172.21.15.38:0/1499522721 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e6c6f273b0 msgr2=0x55e6c73be150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:18.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.398+0000 7f2d8ae13740 1 --2- 172.21.15.38:0/1499522721 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e6c6f273b0 0x55e6c73be150 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c0013b0 tx=0x7f2d6c012b80 comp rx=0 tx=0).stop 2024-09-21T03:42:18.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.398+0000 7f2d8ae13740 1 -- 172.21.15.38:0/1499522721 shutdown_connections 2024-09-21T03:42:18.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.398+0000 7f2d8ae13740 1 --2- 172.21.15.38:0/1499522721 >> [v2:172.21.15.38:6834/2712360296,v1:172.21.15.38:6835/2712360296] conn(0x55e6c7335400 0x55e6c73c7810 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:18.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.398+0000 7f2d8ae13740 1 --2- 172.21.15.38:0/1499522721 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e6c6f50270 0x55e6c735eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:18.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.398+0000 7f2d8ae13740 1 --2- 172.21.15.38:0/1499522721 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e6c6f2f590 0x55e6c735e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:18.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.398+0000 7f2d8ae13740 1 --2- 172.21.15.38:0/1499522721 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e6c6f273b0 0x55e6c73be150 unknown :-1 s=CLOSED pgs=1688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:18.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.398+0000 7f2d60ff9640 0 client.8961 ms_handle_reset on v2:172.21.15.38:6834/2712360296 2024-09-21T03:42:18.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.398+0000 7f2d8ae13740 1 -- 172.21.15.38:0/1499522721 >> 172.21.15.38:0/1499522721 conn(0x55e6c6f443c0 msgr2=0x55e6c73106e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:18.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.399+0000 7f2d8ae13740 1 -- 172.21.15.38:0/1499522721 shutdown_connections 2024-09-21T03:42:18.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.399+0000 7f2d8ae13740 1 -- 172.21.15.38:0/1499522721 wait complete. 2024-09-21T03:42:18.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.399+0000 7f2d8ae13740 20 client.8961 trim_cache size 0 max 16384 2024-09-21T03:42:18.406 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-09-21T03:42:18.406 INFO:teuthology.orchestra.run.smithi038.stdout: "cluster_fsid": "9d50d66f-656b-4fe2-8e3b-fe53b79c4d43", 2024-09-21T03:42:18.406 INFO:teuthology.orchestra.run.smithi038.stdout: "whoami": 0, 2024-09-21T03:42:18.406 INFO:teuthology.orchestra.run.smithi038.stdout: "id": 8862, 2024-09-21T03:42:18.406 INFO:teuthology.orchestra.run.smithi038.stdout: "want_state": "up:active", 2024-09-21T03:42:18.406 INFO:teuthology.orchestra.run.smithi038.stdout: "state": "up:active", 2024-09-21T03:42:18.406 INFO:teuthology.orchestra.run.smithi038.stdout: "fs_name": "cephfs", 2024-09-21T03:42:18.406 INFO:teuthology.orchestra.run.smithi038.stdout: "rank_uptime": 7.4721534700000003, 2024-09-21T03:42:18.407 INFO:teuthology.orchestra.run.smithi038.stdout: "mdsmap_epoch": 198, 2024-09-21T03:42:18.407 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch": 114, 2024-09-21T03:42:18.407 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch_barrier": 114, 2024-09-21T03:42:18.407 INFO:teuthology.orchestra.run.smithi038.stdout: "uptime": 10.721059639 2024-09-21T03:42:18.407 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-09-21T03:42:18.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.399+0000 7f2d83dc3640 1 -- 172.21.15.38:0/4004053647 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2d68078b90 msgr2=0x7f2d6807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:18.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.399+0000 7f2d83dc3640 1 --2- 172.21.15.38:0/4004053647 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2d68078b90 0x7f2d6807b0a0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c001c00 tx=0x7f2d6c0014a0 comp rx=0 tx=0).stop 2024-09-21T03:42:18.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.399+0000 7f2d83dc3640 1 -- 172.21.15.38:0/4004053647 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2d7c1f62d0 msgr2=0x7f2d7c14c940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:18.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.399+0000 7f2d83dc3640 1 --2- 172.21.15.38:0/4004053647 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2d7c1f62d0 0x7f2d7c14c940 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7f2d70004e90 tx=0x7f2d7000b9a0 comp rx=0 tx=0).stop 2024-09-21T03:42:18.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.399+0000 7f2d83dc3640 1 -- 172.21.15.38:0/4004053647 shutdown_connections 2024-09-21T03:42:18.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.399+0000 7f2d83dc3640 1 --2- 172.21.15.38:0/4004053647 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2d68078b90 0x7f2d6807b0a0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:18.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.399+0000 7f2d83dc3640 1 --2- 172.21.15.38:0/4004053647 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2d7c159ff0 0x7f2d7c15a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:18.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.399+0000 7f2d83dc3640 1 --2- 172.21.15.38:0/4004053647 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2d7c1f62d0 0x7f2d7c14c940 unknown :-1 s=CLOSED pgs=1684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:18.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.399+0000 7f2d83dc3640 1 --2- 172.21.15.38:0/4004053647 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2d640099b0 0x7f2d7c06cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:18.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.399+0000 7f2d83dc3640 1 -- 172.21.15.38:0/4004053647 >> 172.21.15.38:0/4004053647 conn(0x7f2d7c09d830 msgr2=0x7f2d7c15cec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:18.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.399+0000 7f2d83dc3640 1 -- 172.21.15.38:0/4004053647 shutdown_connections 2024-09-21T03:42:18.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.399+0000 7f2d83dc3640 1 -- 172.21.15.38:0/4004053647 wait complete. 2024-09-21T03:42:18.412 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-09-21T03:42:18.412 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:42:18.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.527+0000 7fcb38da8640 1 -- 172.21.15.38:0/2686576148 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcb340b2e40 msgr2=0x7fcb340b3250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:18.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.527+0000 7fcb38da8640 1 --2- 172.21.15.38:0/2686576148 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcb340b2e40 0x7fcb340b3250 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fcb24005fd0 tx=0x7fcb2402a8b0 comp rx=0 tx=0).stop 2024-09-21T03:42:18.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.527+0000 7fcb38da8640 1 -- 172.21.15.38:0/2686576148 shutdown_connections 2024-09-21T03:42:18.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.527+0000 7fcb38da8640 1 --2- 172.21.15.38:0/2686576148 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcb340b2e40 0x7fcb340b3250 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:18.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.527+0000 7fcb38da8640 1 --2- 172.21.15.38:0/2686576148 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcb34162f00 0x7fcb340d24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:18.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.527+0000 7fcb38da8640 1 -- 172.21.15.38:0/2686576148 >> 172.21.15.38:0/2686576148 conn(0x7fcb340b6b20 msgr2=0x7fcb340d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:18.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.527+0000 7fcb38da8640 1 -- 172.21.15.38:0/2686576148 shutdown_connections 2024-09-21T03:42:18.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.527+0000 7fcb38da8640 1 -- 172.21.15.38:0/2686576148 wait complete. 2024-09-21T03:42:18.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.529+0000 7fcb38da8640 1 Processor -- start 2024-09-21T03:42:18.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.529+0000 7fcb38da8640 1 -- start start 2024-09-21T03:42:18.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.530+0000 7fcb38da8640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcb340b2e40 0x7fcb340cede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:18.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.530+0000 7fcb38da8640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcb34162280 0x7fcb3414d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:18.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.530+0000 7fcb38da8640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcb34160e10 0x7fcb34161220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:18.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.530+0000 7fcb38da8640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fcb3415e4b0 con 0x7fcb34160e10 2024-09-21T03:42:18.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.531+0000 7fcb38da8640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fcb340b39a0 con 0x7fcb34162280 2024-09-21T03:42:18.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.531+0000 7fcb38da8640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fcb3415e630 con 0x7fcb340b2e40 2024-09-21T03:42:18.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.531+0000 7fcb32d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcb34160e10 0x7fcb34161220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:18.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.531+0000 7fcb32d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcb34160e10 0x7fcb34161220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34328/0 (socket says 172.21.15.38:34328) 2024-09-21T03:42:18.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.531+0000 7fcb32d9d640 1 -- 172.21.15.38:0/3530925488 learned_addr learned my addr 172.21.15.38:0/3530925488 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:42:18.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.531+0000 7fcb3259c640 1 --2- 172.21.15.38:0/3530925488 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcb340b2e40 0x7fcb340cede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:18.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.531+0000 7fcb31d9b640 1 --2- 172.21.15.38:0/3530925488 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcb34162280 0x7fcb3414d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:18.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.531+0000 7fcb32d9d640 1 -- 172.21.15.38:0/3530925488 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcb340b2e40 msgr2=0x7fcb340cede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:18.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.531+0000 7fcb32d9d640 1 --2- 172.21.15.38:0/3530925488 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcb340b2e40 0x7fcb340cede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:18.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.531+0000 7fcb32d9d640 1 -- 172.21.15.38:0/3530925488 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcb34162280 msgr2=0x7fcb3414d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:18.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.531+0000 7fcb32d9d640 1 --2- 172.21.15.38:0/3530925488 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcb34162280 0x7fcb3414d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:18.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.531+0000 7fcb32d9d640 1 -- 172.21.15.38:0/3530925488 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb34161760 con 0x7fcb34160e10 2024-09-21T03:42:18.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.531+0000 7fcb32d9d640 1 --2- 172.21.15.38:0/3530925488 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcb34160e10 0x7fcb34161220 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7fcb24006010 tx=0x7fcb24031700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:18.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.531+0000 7fcb2affd640 1 -- 172.21.15.38:0/3530925488 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb24031990 con 0x7fcb34160e10 2024-09-21T03:42:18.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.532+0000 7fcb2affd640 1 -- 172.21.15.38:0/3530925488 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcb24031c20 con 0x7fcb34160e10 2024-09-21T03:42:18.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.532+0000 7fcb38da8640 1 -- 172.21.15.38:0/3530925488 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb3414dad0 con 0x7fcb34160e10 2024-09-21T03:42:18.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.532+0000 7fcb2affd640 1 -- 172.21.15.38:0/3530925488 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb240330d0 con 0x7fcb34160e10 2024-09-21T03:42:18.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.532+0000 7fcb38da8640 1 -- 172.21.15.38:0/3530925488 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb340cc800 con 0x7fcb34160e10 2024-09-21T03:42:18.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.533+0000 7fcb38da8640 1 -- 172.21.15.38:0/3530925488 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb34162f00 con 0x7fcb34160e10 2024-09-21T03:42:18.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.534+0000 7fcb2affd640 1 -- 172.21.15.38:0/3530925488 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fcb24033270 con 0x7fcb34160e10 2024-09-21T03:42:18.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.534+0000 7fcb2affd640 1 --2- 172.21.15.38:0/3530925488 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fcb14078b90 0x7fcb1407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:18.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.534+0000 7fcb2affd640 1 -- 172.21.15.38:0/3530925488 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 7852+0+0 (secure 0 0 0) 0x7fcb240bc910 con 0x7fcb34160e10 2024-09-21T03:42:18.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.534+0000 7fcb3259c640 1 --2- 172.21.15.38:0/3530925488 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fcb14078b90 0x7fcb1407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:18.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.535+0000 7fcb3259c640 1 --2- 172.21.15.38:0/3530925488 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fcb14078b90 0x7fcb1407b0a0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fcb2c009b30 tx=0x7fcb2c005f90 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:18.541 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.539+0000 7fcb2affd640 1 -- 172.21.15.38:0/3530925488 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fcb34161760 con 0x7fcb34160e10 2024-09-21T03:42:18.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.765+0000 7fcb38da8640 1 -- 172.21.15.38:0/3530925488 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcb3415de50 con 0x7fcb34160e10 2024-09-21T03:42:18.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.766+0000 7fcb2affd640 1 -- 172.21.15.38:0/3530925488 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 198 v198) ==== 95+0+5234 (secure 0 0 0) 0x7fcb2403f030 con 0x7fcb34160e10 2024-09-21T03:42:18.768 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:42:18.768 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":198,"btime":"2024-09-21T03:42:13:945248+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6086,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/965972663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":965972663},{"type":"v1","addr":"172.21.15.195:6836","nonce":965972663}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6700,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/3176122334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":3176122334},{"type":"v1","addr":"172.21.15.195:6837","nonce":3176122334}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":8925,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/4221150556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":4221150556},{"type":"v1","addr":"172.21.15.38:6833","nonce":4221150556}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":196}],"filesystems":[{"mdsmap":{"epoch":198,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:40:02.319282+0000","modified":"2024-09-21T03:42:13.945241+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":114,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8862},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8862":{"gid":8862,"name":"a","rank":0,"incarnation":195,"state":"up:active","state_seq":4,"addr":"172.21.15.38:6835/2712360296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":2712360296},{"type":"v1","addr":"172.21.15.38:6835","nonce":2712360296}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8862,"qdb_cluster":[8862]},"id":6}]} 2024-09-21T03:42:18.768 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 198 2024-09-21T03:42:18.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.768+0000 7fcb38da8640 1 -- 172.21.15.38:0/3530925488 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fcb14078b90 msgr2=0x7fcb1407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:18.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.768+0000 7fcb38da8640 1 --2- 172.21.15.38:0/3530925488 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fcb14078b90 0x7fcb1407b0a0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fcb2c009b30 tx=0x7fcb2c005f90 comp rx=0 tx=0).stop 2024-09-21T03:42:18.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.768+0000 7fcb38da8640 1 -- 172.21.15.38:0/3530925488 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcb34160e10 msgr2=0x7fcb34161220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:18.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.768+0000 7fcb38da8640 1 --2- 172.21.15.38:0/3530925488 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcb34160e10 0x7fcb34161220 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7fcb24006010 tx=0x7fcb24031700 comp rx=0 tx=0).stop 2024-09-21T03:42:18.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.768+0000 7fcb38da8640 1 -- 172.21.15.38:0/3530925488 shutdown_connections 2024-09-21T03:42:18.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.768+0000 7fcb38da8640 1 --2- 172.21.15.38:0/3530925488 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcb34160e10 0x7fcb34161220 unknown :-1 s=CLOSED pgs=1689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:18.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.768+0000 7fcb38da8640 1 --2- 172.21.15.38:0/3530925488 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcb34162280 0x7fcb3414d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:18.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.768+0000 7fcb38da8640 1 --2- 172.21.15.38:0/3530925488 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fcb14078b90 0x7fcb1407b0a0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:18.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.768+0000 7fcb38da8640 1 --2- 172.21.15.38:0/3530925488 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcb340b2e40 0x7fcb340cede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:18.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.768+0000 7fcb38da8640 1 -- 172.21.15.38:0/3530925488 >> 172.21.15.38:0/3530925488 conn(0x7fcb340b6b20 msgr2=0x7fcb34068e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:18.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.768+0000 7fcb38da8640 1 -- 172.21.15.38:0/3530925488 shutdown_connections 2024-09-21T03:42:18.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.768+0000 7fcb38da8640 1 -- 172.21.15.38:0/3530925488 wait complete. 2024-09-21T03:42:18.782 INFO:tasks.cephfs.test_damage:Mutation 'Corrupt 500.00000000' did not prevent MDS startup, attempting ls... 2024-09-21T03:42:18.782 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T03:42:18.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.896+0000 7f9132720640 1 -- 172.21.15.38:0/2280668432 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f912c1536e0 msgr2=0x7f912c0d17c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:18.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.896+0000 7f9132720640 1 --2- 172.21.15.38:0/2280668432 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f912c1536e0 0x7f912c0d17c0 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7f912c0b2db0 tx=0x7f911c02a5f0 comp rx=0 tx=0).stop 2024-09-21T03:42:18.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.896+0000 7f9132720640 1 -- 172.21.15.38:0/2280668432 shutdown_connections 2024-09-21T03:42:18.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.896+0000 7f9132720640 1 --2- 172.21.15.38:0/2280668432 >> v2:172.21.15.195:3300/0 conn(0x7f912c0cd040 0x7f912c0d04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:18.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.896+0000 7f9132720640 1 --2- 172.21.15.38:0/2280668432 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f912c1536e0 0x7f912c0d17c0 unknown :-1 s=CLOSED pgs=1690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:18.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.896+0000 7f9132720640 1 -- 172.21.15.38:0/2280668432 >> 172.21.15.38:0/2280668432 conn(0x7f912c0cef40 msgr2=0x7f912c0cf350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:18.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.896+0000 7f9132720640 1 -- 172.21.15.38:0/2280668432 shutdown_connections 2024-09-21T03:42:18.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.896+0000 7f9132720640 1 -- 172.21.15.38:0/2280668432 wait complete. 2024-09-21T03:42:18.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.898+0000 7f9132720640 1 Processor -- start 2024-09-21T03:42:18.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.898+0000 7f9132720640 1 -- start start 2024-09-21T03:42:18.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.900+0000 7f9132720640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f912c0d1d00 0x7f912c14efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:18.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.900+0000 7f9132720640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f912c1536e0 0x7f912c06def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:18.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.900+0000 7f9132720640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f912c14f6e0 0x7f912c06e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:18.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.900+0000 7f9132720640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f912c0d0d90 con 0x7f912c0d1d00 2024-09-21T03:42:18.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.900+0000 7f9132720640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f912c0d0c10 con 0x7f912c1536e0 2024-09-21T03:42:18.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.900+0000 7f9132720640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f912c0d0f10 con 0x7f912c14f6e0 2024-09-21T03:42:18.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.900+0000 7f912bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f912c0d1d00 0x7f912c14efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:18.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.900+0000 7f912bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f912c0d1d00 0x7f912c14efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34344/0 (socket says 172.21.15.38:34344) 2024-09-21T03:42:18.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.900+0000 7f912bfff640 1 -- 172.21.15.38:0/3968054353 learned_addr learned my addr 172.21.15.38:0/3968054353 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:42:18.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.900+0000 7f9130cbd640 1 --2- 172.21.15.38:0/3968054353 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f912c14f6e0 0x7f912c06e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:18.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.900+0000 7f912b7fe640 1 --2- 172.21.15.38:0/3968054353 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f912c1536e0 0x7f912c06def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:18.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.900+0000 7f912bfff640 1 -- 172.21.15.38:0/3968054353 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f912c14f6e0 msgr2=0x7f912c06e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:18.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.900+0000 7f912bfff640 1 --2- 172.21.15.38:0/3968054353 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f912c14f6e0 0x7f912c06e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:18.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.900+0000 7f912bfff640 1 -- 172.21.15.38:0/3968054353 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f912c1536e0 msgr2=0x7f912c06def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:18.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.900+0000 7f912bfff640 1 --2- 172.21.15.38:0/3968054353 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f912c1536e0 0x7f912c06def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:18.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.900+0000 7f912bfff640 1 -- 172.21.15.38:0/3968054353 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f912c151d90 con 0x7f912c0d1d00 2024-09-21T03:42:18.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.900+0000 7f912bfff640 1 --2- 172.21.15.38:0/3968054353 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f912c0d1d00 0x7f912c14efc0 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7f911c034040 tx=0x7f911c003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:18.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.901+0000 7f9128ff9640 1 -- 172.21.15.38:0/3968054353 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f911c03e030 con 0x7f912c0d1d00 2024-09-21T03:42:18.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.901+0000 7f9128ff9640 1 -- 172.21.15.38:0/3968054353 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f911c003e00 con 0x7f912c0d1d00 2024-09-21T03:42:18.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.901+0000 7f9132720640 1 -- 172.21.15.38:0/3968054353 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f912c152070 con 0x7f912c0d1d00 2024-09-21T03:42:18.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.901+0000 7f9128ff9640 1 -- 172.21.15.38:0/3968054353 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f911c035040 con 0x7f912c0d1d00 2024-09-21T03:42:18.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.901+0000 7f9132720640 1 -- 172.21.15.38:0/3968054353 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f912c152630 con 0x7f912c0d1d00 2024-09-21T03:42:18.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.902+0000 7f9132720640 1 -- 172.21.15.38:0/3968054353 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f912c152c60 con 0x7f912c0d1d00 2024-09-21T03:42:18.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.903+0000 7f9128ff9640 1 -- 172.21.15.38:0/3968054353 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f911c041030 con 0x7f912c0d1d00 2024-09-21T03:42:18.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.903+0000 7f9128ff9640 1 --2- 172.21.15.38:0/3968054353 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9118078b90 0x7f911807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:18.906 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.903+0000 7f9128ff9640 1 -- 172.21.15.38:0/3968054353 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 7852+0+0 (secure 0 0 0) 0x7f911c0bbce0 con 0x7f912c0d1d00 2024-09-21T03:42:18.906 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.903+0000 7f912b7fe640 1 --2- 172.21.15.38:0/3968054353 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9118078b90 0x7f911807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:18.906 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.904+0000 7f912b7fe640 1 --2- 172.21.15.38:0/3968054353 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9118078b90 0x7f911807b0a0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f912c06fa00 tx=0x7f9120009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:18.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:18.908+0000 7f9128ff9640 1 -- 172.21.15.38:0/3968054353 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f911c083270 con 0x7f912c0d1d00 2024-09-21T03:42:18.911 INFO:tasks.ceph.mds.a.smithi038.stderr:2024-09-21T03:42:18.910+0000 7fc167542640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T03:42:19.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:19.136+0000 7f9132720640 1 -- 172.21.15.38:0/3968054353 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f912c0b5590 con 0x7f912c0d1d00 2024-09-21T03:42:19.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:19.136+0000 7f9128ff9640 1 -- 172.21.15.38:0/3968054353 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v198) ==== 53+0+73 (secure 0 0 0) 0x7f911c087ca0 con 0x7f912c0d1d00 2024-09-21T03:42:19.138 INFO:teuthology.orchestra.run.smithi038.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T03:42:19.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:19.138+0000 7f9132720640 1 -- 172.21.15.38:0/3968054353 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9118078b90 msgr2=0x7f911807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:19.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:19.138+0000 7f9132720640 1 --2- 172.21.15.38:0/3968054353 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9118078b90 0x7f911807b0a0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f912c06fa00 tx=0x7f9120009040 comp rx=0 tx=0).stop 2024-09-21T03:42:19.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:19.138+0000 7f9132720640 1 -- 172.21.15.38:0/3968054353 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f912c0d1d00 msgr2=0x7f912c14efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:19.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:19.138+0000 7f9132720640 1 --2- 172.21.15.38:0/3968054353 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f912c0d1d00 0x7f912c14efc0 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7f911c034040 tx=0x7f911c003910 comp rx=0 tx=0).stop 2024-09-21T03:42:19.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:19.138+0000 7f9132720640 1 -- 172.21.15.38:0/3968054353 shutdown_connections 2024-09-21T03:42:19.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:19.138+0000 7f9132720640 1 --2- 172.21.15.38:0/3968054353 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9118078b90 0x7f911807b0a0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:19.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:19.139+0000 7f9132720640 1 --2- 172.21.15.38:0/3968054353 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f912c14f6e0 0x7f912c06e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:19.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:19.139+0000 7f9132720640 1 --2- 172.21.15.38:0/3968054353 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f912c1536e0 0x7f912c06def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:19.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:19.139+0000 7f9132720640 1 --2- 172.21.15.38:0/3968054353 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f912c0d1d00 0x7f912c14efc0 unknown :-1 s=CLOSED pgs=1691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:19.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:19.139+0000 7f9132720640 1 -- 172.21.15.38:0/3968054353 >> 172.21.15.38:0/3968054353 conn(0x7f912c0cef40 msgr2=0x7f912c0cf350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:19.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:19.139+0000 7f9132720640 1 -- 172.21.15.38:0/3968054353 shutdown_connections 2024-09-21T03:42:19.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:19.139+0000 7f9132720640 1 -- 172.21.15.38:0/3968054353 wait complete. 2024-09-21T03:42:19.152 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-21T03:42:19.152 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T03:42:19.152 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi038.front.sepia.ceph.com 2024-09-21T03:42:19.152 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-21T03:42:19.152 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T03:42:19.152 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-21T03:42:19.152 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-21T03:42:19.152 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T03:42:19.152 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T03:42:19.153 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:42:19.153 DEBUG:teuthology.orchestra.run.smithi038:> ip addr 2024-09-21T03:42:19.169 INFO:teuthology.orchestra.run.smithi038.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T03:42:19.169 INFO:teuthology.orchestra.run.smithi038.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T03:42:19.169 INFO:teuthology.orchestra.run.smithi038.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T03:42:19.169 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:42:19.169 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 ::1/128 scope host 2024-09-21T03:42:19.169 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:42:19.169 INFO:teuthology.orchestra.run.smithi038.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T03:42:19.169 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:6b:fd:a2 brd ff:ff:ff:ff:ff:ff 2024-09-21T03:42:19.170 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp7s0f0 2024-09-21T03:42:19.170 INFO:teuthology.orchestra.run.smithi038.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T03:42:19.170 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:6b:fd:a3 brd ff:ff:ff:ff:ff:ff 2024-09-21T03:42:19.170 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp7s0f1 2024-09-21T03:42:19.170 INFO:teuthology.orchestra.run.smithi038.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T03:42:19.170 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:bd:15:fc brd ff:ff:ff:ff:ff:ff 2024-09-21T03:42:19.170 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp1s0f0 2024-09-21T03:42:19.170 INFO:teuthology.orchestra.run.smithi038.stdout: inet 172.21.15.38/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-21T03:42:19.170 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft 42270sec preferred_lft 42270sec 2024-09-21T03:42:19.170 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 fe80::ec4:7aff:febd:15fc/64 scope link 2024-09-21T03:42:19.170 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:42:19.170 INFO:teuthology.orchestra.run.smithi038.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T03:42:19.170 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:bd:15:fd brd ff:ff:ff:ff:ff:ff 2024-09-21T03:42:19.171 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp1s0f1 2024-09-21T03:42:19.171 INFO:teuthology.orchestra.run.smithi038.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T03:42:19.171 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 92:55:c7:97:6f:b5 brd ff:ff:ff:ff:ff:ff 2024-09-21T03:42:19.171 INFO:teuthology.orchestra.run.smithi038.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-21T03:42:19.171 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:42:19.171 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 fe80::c49f:c1ff:fe21:b12a/64 scope link 2024-09-21T03:42:19.171 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:42:19.171 INFO:teuthology.orchestra.run.smithi038.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T03:42:19.171 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether a6:3a:83:ad:db:d5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T03:42:19.171 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 fe80::a43a:83ff:fead:dbd5/64 scope link 2024-09-21T03:42:19.171 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:42:19.171 INFO:teuthology.orchestra.run.smithi038.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T03:42:19.171 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 92:55:c7:97:6f:b5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T03:42:19.171 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 fe80::9055:c7ff:fe97:6fb5/64 scope link 2024-09-21T03:42:19.171 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:42:19.172 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:42:19.172 DEBUG:teuthology.orchestra.run.smithi038:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T03:42:19.234 INFO:teuthology.orchestra.run.smithi038.stdout:1 2024-09-21T03:42:19.235 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:42:19.235 DEBUG:teuthology.orchestra.run.smithi038:> route 2024-09-21T03:42:19.292 INFO:teuthology.orchestra.run.smithi038.stdout:Kernel IP routing table 2024-09-21T03:42:19.292 INFO:teuthology.orchestra.run.smithi038.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T03:42:19.292 INFO:teuthology.orchestra.run.smithi038.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-21T03:42:19.293 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-21T03:42:19.293 INFO:teuthology.orchestra.run.smithi038.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T03:42:19.293 DEBUG:teuthology.orchestra.run.smithi038:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T03:42:19.293 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-09-21T03:42:19.293 DEBUG:teuthology.orchestra.run.smithi038:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-21T03:42:19.293 DEBUG:teuthology.orchestra.run.smithi038:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-21T03:42:19.293 DEBUG:teuthology.orchestra.run.smithi038:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-21T03:42:19.293 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-09-21T03:42:19.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:19 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T03:42:19.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:42:19.421 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:42:19.422 DEBUG:teuthology.orchestra.run.smithi038:> ip netns list 2024-09-21T03:42:19.438 INFO:teuthology.orchestra.run.smithi038.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-21T03:42:19.438 INFO:teuthology.orchestra.run.smithi038.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-21T03:42:19.438 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T03:42:19.438 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:42:19.438 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip link set brx.0 up 2024-09-21T03:42:19.503 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T03:42:19.503 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-21T03:42:19.559 INFO:teuthology.orchestra.run.smithi038.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-21T03:42:19.560 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T03:42:19.560 DEBUG:teuthology.orchestra.run.smithi038:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-21T03:42:19.615 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-21T03:42:19.615 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:42:19.615 DEBUG:teuthology.orchestra.run.smithi038:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T03:42:19.711 INFO:teuthology.orchestra.run.smithi038.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T03:42:19.711 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T03:42:19.711 INFO:teuthology.orchestra.run.smithi038.stdout:invalid new device string format 2024-09-21T03:42:19.722 INFO:teuthology.orchestra.run.smithi038.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T03:42:19.722 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T03:42:19.722 INFO:teuthology.orchestra.run.smithi038.stdout:invalid new device string format 2024-09-21T03:42:19.722 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: resolved to: "172.21.15.38:3300,172.21.15.195:3300,172.21.15.195:3301" 2024-09-21T03:42:19.722 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T03:42:19.729 INFO:teuthology.orchestra.run.smithi038.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T03:42:19.730 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T03:42:19.730 INFO:teuthology.orchestra.run.smithi038.stdout:invalid new device string format 2024-09-21T03:42:19.730 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: resolved to: "172.21.15.38:3300,172.21.15.195:3300,172.21.15.195:3301" 2024-09-21T03:42:19.730 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T03:42:19.730 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: resolved to: "172.21.15.38:6789,172.21.15.195:6789,172.21.15.195:6790" 2024-09-21T03:42:19.730 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.38:6789,172.21.15.195:6789,172.21.15.195:6790:/ 2024-09-21T03:42:19.730 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=9d50d66f-656b-4fe2-8e3b-fe53b79c4d43" will pass to kernel 2024-09-21T03:42:19.730 INFO:teuthology.orchestra.run.smithi038.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-21T03:42:19.730 INFO:teuthology.orchestra.run.smithi038.stdout: You just mounted a file system that supports labels which does not 2024-09-21T03:42:19.730 INFO:teuthology.orchestra.run.smithi038.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-21T03:42:19.731 INFO:teuthology.orchestra.run.smithi038.stdout: applications will generate AVC messages and not be allowed access to 2024-09-21T03:42:19.731 INFO:teuthology.orchestra.run.smithi038.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-21T03:42:19.732 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T03:42:19.732 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:42:19.732 DEBUG:teuthology.orchestra.run.smithi038:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-21T03:42:19.758 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T03:42:19.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:19 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T03:42:19.850 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:42:19.856 INFO:teuthology.orchestra.run.smithi038.stdout:client8194 2024-09-21T03:42:19.857 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-21T03:42:19.857 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:42:19.873 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:42:19.873 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls -R .) 2024-09-21T03:42:19.915 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:42:19.915 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir foo) 2024-09-21T03:42:19.930 INFO:teuthology.orchestra.run.smithi038.stdout:.: 2024-09-21T03:42:19.931 INFO:teuthology.orchestra.run.smithi038.stdout:subdir 2024-09-21T03:42:19.931 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:42:19.931 INFO:teuthology.orchestra.run.smithi038.stdout:./subdir: 2024-09-21T03:42:19.931 INFO:teuthology.orchestra.run.smithi038.stdout:sixmegs 2024-09-21T03:42:19.945 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T03:42:19.945 INFO:teuthology.orchestra.run.smithi038.stderr:mkdir: cannot create directory ‘foo’: Read-only file system 2024-09-21T03:42:19.945 INFO:tasks.cephfs.test_damage:mkdir: cannot create directory ‘foo’: Read-only file system 2024-09-21T03:42:19.945 INFO:tasks.cephfs.test_damage:Applying mutation 'Corrupt mds_snaptable' 2024-09-21T03:42:19.946 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-21T03:42:19.946 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:42:20.002 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-21T03:42:20.002 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:42:20.058 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T03:42:20.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:20.213+0000 7f76c14b6640 1 -- 172.21.15.38:0/111174407 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f76bc0d1120 msgr2=0x7f76bc0ceb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:20.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:20.213+0000 7f76c14b6640 1 --2- 172.21.15.38:0/111174407 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f76bc0d1120 0x7f76bc0ceb90 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7f76bc0757e0 tx=0x7f76b402a980 comp rx=0 tx=0).stop 2024-09-21T03:42:20.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:20.214+0000 7f76c14b6640 1 -- 172.21.15.38:0/111174407 shutdown_connections 2024-09-21T03:42:20.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:20.214+0000 7f76c14b6640 1 --2- 172.21.15.38:0/111174407 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f76bc0d1120 0x7f76bc0ceb90 unknown :-1 s=CLOSED pgs=1692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:20.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:20.214+0000 7f76c14b6640 1 --2- 172.21.15.38:0/111174407 >> v2:172.21.15.195:3300/0 conn(0x7f76bc0b9ad0 0x7f76bc0d0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:20.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:20.214+0000 7f76c14b6640 1 --2- 172.21.15.38:0/111174407 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f76bc15ee80 0x7f76bc0d0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:20.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:20.214+0000 7f76c14b6640 1 -- 172.21.15.38:0/111174407 >> 172.21.15.38:0/111174407 conn(0x7f76bc09d830 msgr2=0x7f76bc0b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:20.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:20.214+0000 7f76c14b6640 1 -- 172.21.15.38:0/111174407 shutdown_connections 2024-09-21T03:42:20.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:20.214+0000 7f76c14b6640 1 -- 172.21.15.38:0/111174407 wait complete. 2024-09-21T03:42:20.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:20.216+0000 7f76c14b6640 1 Processor -- start 2024-09-21T03:42:20.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:20.216+0000 7f76c14b6640 1 -- start start 2024-09-21T03:42:20.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:20.217+0000 7f76c14b6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f76bc0b9ad0 0x7f76bc0ce100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:20.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:20.217+0000 7f76c14b6640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f76bc0d1120 0x7f76bc09e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:20.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:20.217+0000 7f76c14b6640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f76bc15ee80 0x7f76bc09e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:20.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:20.217+0000 7f76c14b6640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f76bc0cf460 con 0x7f76bc0b9ad0 2024-09-21T03:42:20.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:20.217+0000 7f76c14b6640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f76bc0cf2e0 con 0x7f76bc15ee80 2024-09-21T03:42:20.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:20.217+0000 7f76c14b6640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f76bc0b6580 con 0x7f76bc0d1120 2024-09-21T03:42:20.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:20.217+0000 7f76baffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f76bc0b9ad0 0x7f76bc0ce100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:20.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:20.217+0000 7f76baffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f76bc0b9ad0 0x7f76bc0ce100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34374/0 (socket says 172.21.15.38:34374) 2024-09-21T03:42:20.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:20.217+0000 7f76baffd640 1 -- 172.21.15.38:0/187786336 learned_addr learned my addr 172.21.15.38:0/187786336 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:42:20.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:20.217+0000 7f76ba7fc640 1 --2- 172.21.15.38:0/187786336 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f76bc0d1120 0x7f76bc09e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:20.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:20.217+0000 7f76bb7fe640 1 --2- 172.21.15.38:0/187786336 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f76bc15ee80 0x7f76bc09e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:20.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:20.217+0000 7f76baffd640 1 -- 172.21.15.38:0/187786336 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f76bc0d1120 msgr2=0x7f76bc09e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:20.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:20.217+0000 7f76baffd640 1 --2- 172.21.15.38:0/187786336 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f76bc0d1120 0x7f76bc09e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:20.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:20.217+0000 7f76baffd640 1 -- 172.21.15.38:0/187786336 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f76bc15ee80 msgr2=0x7f76bc09e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:20.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:20.217+0000 7f76baffd640 1 --2- 172.21.15.38:0/187786336 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f76bc15ee80 0x7f76bc09e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:20.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:20.217+0000 7f76baffd640 1 -- 172.21.15.38:0/187786336 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76bc1f56a0 con 0x7f76bc0b9ad0 2024-09-21T03:42:20.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:20.218+0000 7f76baffd640 1 --2- 172.21.15.38:0/187786336 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f76bc0b9ad0 0x7f76bc0ce100 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7f76ac00d750 tx=0x7f76ac00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:20.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:20.218+0000 7f76a37fe640 1 -- 172.21.15.38:0/187786336 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76ac017070 con 0x7f76bc0b9ad0 2024-09-21T03:42:20.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:20.218+0000 7f76a37fe640 1 -- 172.21.15.38:0/187786336 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f76ac004020 con 0x7f76bc0b9ad0 2024-09-21T03:42:20.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:20.218+0000 7f76c14b6640 1 -- 172.21.15.38:0/187786336 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76bc1f5980 con 0x7f76bc0b9ad0 2024-09-21T03:42:20.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:20.218+0000 7f76a37fe640 1 -- 172.21.15.38:0/187786336 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76ac004450 con 0x7f76bc0b9ad0 2024-09-21T03:42:20.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:20.218+0000 7f76c14b6640 1 -- 172.21.15.38:0/187786336 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76bc1f5f40 con 0x7f76bc0b9ad0 2024-09-21T03:42:20.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:20.219+0000 7f76c14b6640 1 -- 172.21.15.38:0/187786336 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76bc0ceb70 con 0x7f76bc0b9ad0 2024-09-21T03:42:20.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:20.220+0000 7f76a37fe640 1 -- 172.21.15.38:0/187786336 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f76ac0099d0 con 0x7f76bc0b9ad0 2024-09-21T03:42:20.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:20.220+0000 7f76a37fe640 1 --2- 172.21.15.38:0/187786336 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f76a8078bc0 0x7f76a807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:20.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:20.221+0000 7f76a37fe640 1 -- 172.21.15.38:0/187786336 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 7852+0+0 (secure 0 0 0) 0x7f76ac09b710 con 0x7f76bc0b9ad0 2024-09-21T03:42:20.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:20.221+0000 7f76ba7fc640 1 --2- 172.21.15.38:0/187786336 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f76a8078bc0 0x7f76a807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:20.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:20.221+0000 7f76ba7fc640 1 --2- 172.21.15.38:0/187786336 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f76a8078bc0 0x7f76a807b0d0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f76bc0cfb40 tx=0x7f76b0005f30 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:20.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:20.225+0000 7f76a37fe640 1 -- 172.21.15.38:0/187786336 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f76ac020020 con 0x7f76bc0b9ad0 2024-09-21T03:42:20.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:20.427+0000 7f76c14b6640 1 -- 172.21.15.38:0/187786336 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f76bc15e710 con 0x7f76bc0b9ad0 2024-09-21T03:42:20.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:20.428+0000 7f76a37fe640 1 -- 172.21.15.38:0/187786336 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 36 entries v114) ==== 81+0+2190 (secure 0 0 0) 0x7f76ac062d30 con 0x7f76bc0b9ad0 2024-09-21T03:42:20.430 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6832/3537258034 2024-09-22T03:42:06.901189+0000 2024-09-21T03:42:20.430 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6833/3537258034 2024-09-22T03:42:06.901189+0000 2024-09-21T03:42:20.430 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6833/1272005650 2024-09-22T03:41:58.836862+0000 2024-09-21T03:42:20.431 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6832/1272005650 2024-09-22T03:41:58.836862+0000 2024-09-21T03:42:20.431 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6833/1717955365 2024-09-22T03:41:51.787804+0000 2024-09-21T03:42:20.431 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6832/3710736169 2024-09-22T03:40:59.454480+0000 2024-09-21T03:42:20.431 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6835/2751962827 2024-09-22T03:40:38.029580+0000 2024-09-21T03:42:20.431 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6835/425672974 2024-09-22T03:40:46.105521+0000 2024-09-21T03:42:20.431 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6834/425672974 2024-09-22T03:40:46.105521+0000 2024-09-21T03:42:20.431 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6833/3402963947 2024-09-22T03:41:23.555005+0000 2024-09-21T03:42:20.431 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6835/2916238332 2024-09-22T03:41:42.553770+0000 2024-09-21T03:42:20.431 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6834/3572946698 2024-09-22T03:41:44.737273+0000 2024-09-21T03:42:20.431 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6834/3864487171 2024-09-22T03:40:31.989837+0000 2024-09-21T03:42:20.431 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.195:6835/712370966 2024-09-22T03:40:11.633787+0000 2024-09-21T03:42:20.431 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6833/3726830644 2024-09-22T03:41:12.532341+0000 2024-09-21T03:42:20.432 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6832/3402963947 2024-09-22T03:41:23.555005+0000 2024-09-21T03:42:20.432 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6833/751618334 2024-09-22T03:40:42.053279+0000 2024-09-21T03:42:20.432 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6832/751618334 2024-09-22T03:40:42.053279+0000 2024-09-21T03:42:20.432 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6835/2418225210 2024-09-22T03:42:02.869612+0000 2024-09-21T03:42:20.432 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6834/2418225210 2024-09-22T03:42:02.869612+0000 2024-09-21T03:42:20.432 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6832/1717955365 2024-09-22T03:41:51.787804+0000 2024-09-21T03:42:20.432 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6832/121492560 2024-09-22T03:41:10.371933+0000 2024-09-21T03:42:20.432 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6835/3864487171 2024-09-22T03:40:31.989837+0000 2024-09-21T03:42:20.432 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6835/2934569248 2024-09-22T03:40:26.934591+0000 2024-09-21T03:42:20.432 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6833/121492560 2024-09-22T03:41:10.371933+0000 2024-09-21T03:42:20.432 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6832/3726830644 2024-09-22T03:41:12.532341+0000 2024-09-21T03:42:20.432 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6834/2934569248 2024-09-22T03:40:26.934591+0000 2024-09-21T03:42:20.432 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6834/2748959970 2024-09-22T03:41:16.565736+0000 2024-09-21T03:42:20.433 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.195:6837/712370966 2024-09-22T03:40:11.633787+0000 2024-09-21T03:42:20.433 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6833/3710736169 2024-09-22T03:40:59.454480+0000 2024-09-21T03:42:20.433 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6835/2748959970 2024-09-22T03:41:16.565736+0000 2024-09-21T03:42:20.433 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6832/3553953266 2024-09-22T03:41:29.639383+0000 2024-09-21T03:42:20.433 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6834/2751962827 2024-09-22T03:40:38.029580+0000 2024-09-21T03:42:20.433 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6833/3553953266 2024-09-22T03:41:29.639383+0000 2024-09-21T03:42:20.433 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6834/2916238332 2024-09-22T03:41:42.553770+0000 2024-09-21T03:42:20.433 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6835/3572946698 2024-09-22T03:41:44.737273+0000 2024-09-21T03:42:20.434 INFO:teuthology.orchestra.run.smithi038.stderr:listed 36 entries 2024-09-21T03:42:20.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:20.430+0000 7f76c14b6640 1 -- 172.21.15.38:0/187786336 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f76a8078bc0 msgr2=0x7f76a807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:20.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:20.430+0000 7f76c14b6640 1 --2- 172.21.15.38:0/187786336 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f76a8078bc0 0x7f76a807b0d0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f76bc0cfb40 tx=0x7f76b0005f30 comp rx=0 tx=0).stop 2024-09-21T03:42:20.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:20.430+0000 7f76c14b6640 1 -- 172.21.15.38:0/187786336 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f76bc0b9ad0 msgr2=0x7f76bc0ce100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:20.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:20.430+0000 7f76c14b6640 1 --2- 172.21.15.38:0/187786336 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f76bc0b9ad0 0x7f76bc0ce100 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7f76ac00d750 tx=0x7f76ac00dc10 comp rx=0 tx=0).stop 2024-09-21T03:42:20.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:20.430+0000 7f76c14b6640 1 -- 172.21.15.38:0/187786336 shutdown_connections 2024-09-21T03:42:20.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:20.430+0000 7f76c14b6640 1 --2- 172.21.15.38:0/187786336 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f76bc15ee80 0x7f76bc09e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:20.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:20.430+0000 7f76c14b6640 1 --2- 172.21.15.38:0/187786336 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f76a8078bc0 0x7f76a807b0d0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:20.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:20.430+0000 7f76c14b6640 1 --2- 172.21.15.38:0/187786336 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f76bc0d1120 0x7f76bc09e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:20.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:20.430+0000 7f76c14b6640 1 --2- 172.21.15.38:0/187786336 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f76bc0b9ad0 0x7f76bc0ce100 unknown :-1 s=CLOSED pgs=1693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:20.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:20.430+0000 7f76c14b6640 1 -- 172.21.15.38:0/187786336 >> 172.21.15.38:0/187786336 conn(0x7f76bc09d830 msgr2=0x7f76bc161270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:20.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:20.430+0000 7f76c14b6640 1 -- 172.21.15.38:0/187786336 shutdown_connections 2024-09-21T03:42:20.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:20.430+0000 7f76c14b6640 1 -- 172.21.15.38:0/187786336 wait complete. 2024-09-21T03:42:20.443 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-21T03:42:20.444 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:42:20.444 DEBUG:teuthology.orchestra.run.smithi038:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-09-21T03:42:20.478 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi038.front.sepia.ceph.com 2024-09-21T03:42:20.478 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:42:20.478 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T03:42:20.535 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-21T03:42:20.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:20.691+0000 7f38b18b4640 1 -- 172.21.15.38:0/607602578 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f38ac15ee80 msgr2=0x7f38ac0d0170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:20.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:20.691+0000 7f38b18b4640 1 --2- 172.21.15.38:0/607602578 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f38ac15ee80 0x7f38ac0d0170 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7f38ac0d06b0 tx=0x7f38a402a410 comp rx=0 tx=0).stop 2024-09-21T03:42:20.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:20.691+0000 7f38b18b4640 1 -- 172.21.15.38:0/607602578 shutdown_connections 2024-09-21T03:42:20.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:20.691+0000 7f38b18b4640 1 --2- 172.21.15.38:0/607602578 >> v2:172.21.15.195:3300/0 conn(0x7f38ac0b9ad0 0x7f38ac0d0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:20.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:20.691+0000 7f38b18b4640 1 --2- 172.21.15.38:0/607602578 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f38ac15ee80 0x7f38ac0d0170 unknown :-1 s=CLOSED pgs=1695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:20.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:20.691+0000 7f38b18b4640 1 -- 172.21.15.38:0/607602578 >> 172.21.15.38:0/607602578 conn(0x7f38ac09d830 msgr2=0x7f38ac0b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:20.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:20.691+0000 7f38b18b4640 1 -- 172.21.15.38:0/607602578 shutdown_connections 2024-09-21T03:42:20.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:20.691+0000 7f38b18b4640 1 -- 172.21.15.38:0/607602578 wait complete. 2024-09-21T03:42:20.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:20.693+0000 7f38b18b4640 1 Processor -- start 2024-09-21T03:42:20.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:20.693+0000 7f38b18b4640 1 -- start start 2024-09-21T03:42:20.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:20.694+0000 7f38b18b4640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f38ac0b9ad0 0x7f38ac14f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:20.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:20.694+0000 7f38b18b4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f38ac0d1120 0x7f38ac0b21e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:20.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:20.694+0000 7f38b18b4640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f38ac15ee80 0x7f38ac0b2720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:20.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:20.694+0000 7f38b18b4640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f38ac0b6700 con 0x7f38ac0d1120 2024-09-21T03:42:20.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:20.694+0000 7f38b18b4640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f38ac0b6580 con 0x7f38ac15ee80 2024-09-21T03:42:20.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:20.694+0000 7f38b18b4640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f38ac0b6880 con 0x7f38ac0b9ad0 2024-09-21T03:42:20.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:20.694+0000 7f38aa7fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f38ac0d1120 0x7f38ac0b21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:20.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:20.694+0000 7f38aa7fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f38ac0d1120 0x7f38ac0b21e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34384/0 (socket says 172.21.15.38:34384) 2024-09-21T03:42:20.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:20.694+0000 7f38aa7fc640 1 -- 172.21.15.38:0/392110932 learned_addr learned my addr 172.21.15.38:0/392110932 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:42:20.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:20.695+0000 7f38ab7fe640 1 --2- 172.21.15.38:0/392110932 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f38ac15ee80 0x7f38ac0b2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:20.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:20.695+0000 7f38aaffd640 1 --2- 172.21.15.38:0/392110932 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f38ac0b9ad0 0x7f38ac14f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:20.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:20.695+0000 7f38aa7fc640 1 -- 172.21.15.38:0/392110932 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f38ac0b9ad0 msgr2=0x7f38ac14f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:20.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:20.695+0000 7f38aa7fc640 1 --2- 172.21.15.38:0/392110932 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f38ac0b9ad0 0x7f38ac14f890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:20.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:20.695+0000 7f38aa7fc640 1 -- 172.21.15.38:0/392110932 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f38ac15ee80 msgr2=0x7f38ac0b2720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:20.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:20.695+0000 7f38aa7fc640 1 --2- 172.21.15.38:0/392110932 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f38ac15ee80 0x7f38ac0b2720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:20.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:20.695+0000 7f38aa7fc640 1 -- 172.21.15.38:0/392110932 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38ac06e160 con 0x7f38ac0d1120 2024-09-21T03:42:20.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:20.695+0000 7f38aa7fc640 1 --2- 172.21.15.38:0/392110932 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f38ac0d1120 0x7f38ac0b21e0 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7f3894005ca0 tx=0x7f389400dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:20.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:20.695+0000 7f38a3fff640 1 -- 172.21.15.38:0/392110932 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3894017070 con 0x7f38ac0d1120 2024-09-21T03:42:20.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:20.695+0000 7f38a3fff640 1 -- 172.21.15.38:0/392110932 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3894004110 con 0x7f38ac0d1120 2024-09-21T03:42:20.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:20.695+0000 7f38b18b4640 1 -- 172.21.15.38:0/392110932 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38ac06e440 con 0x7f38ac0d1120 2024-09-21T03:42:20.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:20.695+0000 7f38a3fff640 1 -- 172.21.15.38:0/392110932 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38940099d0 con 0x7f38ac0d1120 2024-09-21T03:42:20.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:20.695+0000 7f38b18b4640 1 -- 172.21.15.38:0/392110932 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38ac06ea00 con 0x7f38ac0d1120 2024-09-21T03:42:20.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:20.696+0000 7f38b18b4640 1 -- 172.21.15.38:0/392110932 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38ac0b6d30 con 0x7f38ac0d1120 2024-09-21T03:42:20.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:20.697+0000 7f38a3fff640 1 -- 172.21.15.38:0/392110932 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f3894009b70 con 0x7f38ac0d1120 2024-09-21T03:42:20.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:20.697+0000 7f38a3fff640 1 --2- 172.21.15.38:0/392110932 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f388c078bc0 0x7f388c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:20.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:20.698+0000 7f38a3fff640 1 -- 172.21.15.38:0/392110932 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 7852+0+0 (secure 0 0 0) 0x7f389409b580 con 0x7f38ac0d1120 2024-09-21T03:42:20.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:20.698+0000 7f38aaffd640 1 --2- 172.21.15.38:0/392110932 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f388c078bc0 0x7f388c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:20.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:20.698+0000 7f38aaffd640 1 --2- 172.21.15.38:0/392110932 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f388c078bc0 0x7f388c07b0d0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f38a4034000 tx=0x7f38a4004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:20.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:20.702+0000 7f38a3fff640 1 -- 172.21.15.38:0/392110932 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f38940631f0 con 0x7f38ac0d1120 2024-09-21T03:42:20.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:20.929+0000 7f38b18b4640 1 -- 172.21.15.38:0/392110932 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f38ac15c3c0 con 0x7f38ac0d1120 2024-09-21T03:42:20.932 INFO:tasks.ceph.mon.a.smithi038.stderr:2024-09-21T03:42:20.930+0000 7f3f45713640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T03:42:20.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:20.936+0000 7f38a3fff640 1 -- 172.21.15.38:0/392110932 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v199) ==== 161+0+0 (secure 0 0 0) 0x7f3894062af0 con 0x7f38ac0d1120 2024-09-21T03:42:20.938 INFO:teuthology.orchestra.run.smithi038.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-21T03:42:20.940 INFO:tasks.ceph.mds.a.smithi038.stderr: -1880> 2024-09-21T03:42:10.933+0000 7fc16453c640 -1 mds.0.purge_queue operator(): Error -22 loading Journaler 2024-09-21T03:42:20.940 INFO:tasks.ceph.mds.a.smithi038.stderr: -1877> 2024-09-21T03:42:10.933+0000 7fc16453c640 -1 mds.0.195 unhandled write error (22) Invalid argument, force readonly... 2024-09-21T03:42:20.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:20.940+0000 7f38b18b4640 1 -- 172.21.15.38:0/392110932 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f388c078bc0 msgr2=0x7f388c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:20.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:20.940+0000 7f38b18b4640 1 --2- 172.21.15.38:0/392110932 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f388c078bc0 0x7f388c07b0d0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f38a4034000 tx=0x7f38a4004730 comp rx=0 tx=0).stop 2024-09-21T03:42:20.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:20.940+0000 7f38b18b4640 1 -- 172.21.15.38:0/392110932 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f38ac0d1120 msgr2=0x7f38ac0b21e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:20.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:20.940+0000 7f38b18b4640 1 --2- 172.21.15.38:0/392110932 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f38ac0d1120 0x7f38ac0b21e0 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7f3894005ca0 tx=0x7f389400dbb0 comp rx=0 tx=0).stop 2024-09-21T03:42:20.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:20.940+0000 7f38b18b4640 1 -- 172.21.15.38:0/392110932 shutdown_connections 2024-09-21T03:42:20.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:20.940+0000 7f38b18b4640 1 --2- 172.21.15.38:0/392110932 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f38ac15ee80 0x7f38ac0b2720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:20.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:20.940+0000 7f38b18b4640 1 --2- 172.21.15.38:0/392110932 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f38ac0d1120 0x7f38ac0b21e0 unknown :-1 s=CLOSED pgs=1696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:20.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:20.940+0000 7f38b18b4640 1 --2- 172.21.15.38:0/392110932 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f388c078bc0 0x7f388c07b0d0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:20.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:20.940+0000 7f38b18b4640 1 --2- 172.21.15.38:0/392110932 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f38ac0b9ad0 0x7f38ac14f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:20.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:20.940+0000 7f38b18b4640 1 -- 172.21.15.38:0/392110932 >> 172.21.15.38:0/392110932 conn(0x7f38ac09d830 msgr2=0x7f38ac0b4480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:20.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:20.941+0000 7f38b18b4640 1 -- 172.21.15.38:0/392110932 shutdown_connections 2024-09-21T03:42:20.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:20.941+0000 7f38b18b4640 1 -- 172.21.15.38:0/392110932 wait complete. 2024-09-21T03:42:20.943 INFO:tasks.ceph.mds.a.smithi038.stderr: -569> 2024-09-21T03:42:18.910+0000 7fc167542640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T03:42:20.955 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-09-21T03:42:21.000 INFO:tasks.ceph.mds.a.smithi038.stdout:starting mds.a at 2024-09-21T03:42:21.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:21.073+0000 7f2cbb025640 1 -- 172.21.15.38:0/3254974835 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2ca0004800 msgr2=0x7f2ca0004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:21.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:21.073+0000 7f2cbb025640 1 --2- 172.21.15.38:0/3254974835 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2ca0004800 0x7f2ca0004c10 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7f2cb408d980 tx=0x7f2ca0005830 comp rx=0 tx=0).stop 2024-09-21T03:42:21.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:21.073+0000 7f2cbb025640 1 -- 172.21.15.38:0/3254974835 shutdown_connections 2024-09-21T03:42:21.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:21.074+0000 7f2cbb025640 1 --2- 172.21.15.38:0/3254974835 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2ca0004800 0x7f2ca0004c10 unknown :-1 s=CLOSED pgs=1700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:21.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:21.074+0000 7f2cbb025640 1 -- 172.21.15.38:0/3254974835 >> 172.21.15.38:0/3254974835 conn(0x7f2cb409d830 msgr2=0x7f2cb40b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:21.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:21.074+0000 7f2cbb025640 1 -- 172.21.15.38:0/3254974835 shutdown_connections 2024-09-21T03:42:21.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:21.074+0000 7f2cbb025640 1 -- 172.21.15.38:0/3254974835 wait complete. 2024-09-21T03:42:21.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:21.075+0000 7f2cbb025640 1 Processor -- start 2024-09-21T03:42:21.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:21.075+0000 7f2cbb025640 1 -- start start 2024-09-21T03:42:21.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:21.077+0000 7f2cbb025640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2ca0004800 0x7f2cb41579d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:21.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:21.077+0000 7f2cbb025640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2cb415c2b0 0x7f2cb4161000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:21.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:21.077+0000 7f2cbb025640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2cb4161540 0x7f2cb406cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:21.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:21.077+0000 7f2cbb025640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f2cb40cf120 con 0x7f2cb4161540 2024-09-21T03:42:21.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:21.077+0000 7f2cbb025640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f2cb40cefa0 con 0x7f2ca0004800 2024-09-21T03:42:21.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:21.077+0000 7f2cbb025640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f2cb40cf2a0 con 0x7f2cb415c2b0 2024-09-21T03:42:21.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:21.077+0000 7f2cb95c2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2cb4161540 0x7f2cb406cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:21.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:21.077+0000 7f2cb95c2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2cb4161540 0x7f2cb406cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34436/0 (socket says 172.21.15.38:34436) 2024-09-21T03:42:21.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:21.077+0000 7f2cb95c2640 1 -- 172.21.15.38:0/2060962993 learned_addr learned my addr 172.21.15.38:0/2060962993 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:42:21.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:21.077+0000 7f2cb8dc1640 1 --2- 172.21.15.38:0/2060962993 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2ca0004800 0x7f2cb41579d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:21.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:21.077+0000 7f2cb3fff640 1 --2- 172.21.15.38:0/2060962993 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2cb415c2b0 0x7f2cb4161000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:21.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:21.077+0000 7f2cb95c2640 1 -- 172.21.15.38:0/2060962993 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2cb415c2b0 msgr2=0x7f2cb4161000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:21.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:21.077+0000 7f2cb95c2640 1 --2- 172.21.15.38:0/2060962993 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2cb415c2b0 0x7f2cb4161000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:21.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:21.077+0000 7f2cb95c2640 1 -- 172.21.15.38:0/2060962993 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2ca0004800 msgr2=0x7f2cb41579d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:21.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:21.077+0000 7f2cb95c2640 1 --2- 172.21.15.38:0/2060962993 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2ca0004800 0x7f2cb41579d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:21.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:21.077+0000 7f2cb95c2640 1 -- 172.21.15.38:0/2060962993 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cb4161950 con 0x7f2cb4161540 2024-09-21T03:42:21.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:21.077+0000 7f2cb8dc1640 1 --2- 172.21.15.38:0/2060962993 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2ca0004800 0x7f2cb41579d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:42:21.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:21.078+0000 7f2cb95c2640 1 --2- 172.21.15.38:0/2060962993 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2cb4161540 0x7f2cb406cde0 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7f2cb415c9c0 tx=0x7f2cac006e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:21.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:21.078+0000 7f2cb17fa640 1 -- 172.21.15.38:0/2060962993 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cac0105d0 con 0x7f2cb4161540 2024-09-21T03:42:21.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:21.078+0000 7f2cb17fa640 1 -- 172.21.15.38:0/2060962993 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2cac0108a0 con 0x7f2cb4161540 2024-09-21T03:42:21.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:21.078+0000 7f2cbb025640 1 -- 172.21.15.38:0/2060962993 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cb406ed80 con 0x7f2cb4161540 2024-09-21T03:42:21.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:21.078+0000 7f2cb17fa640 1 -- 172.21.15.38:0/2060962993 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cac010cf0 con 0x7f2cb4161540 2024-09-21T03:42:21.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:21.078+0000 7f2cbb025640 1 -- 172.21.15.38:0/2060962993 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cb406f340 con 0x7f2cb4161540 2024-09-21T03:42:21.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:21.079+0000 7f2cbb025640 1 -- 172.21.15.38:0/2060962993 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2cb406e750 con 0x7f2cb4161540 2024-09-21T03:42:21.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:21.080+0000 7f2cb17fa640 1 -- 172.21.15.38:0/2060962993 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f2cac018020 con 0x7f2cb4161540 2024-09-21T03:42:21.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:21.080+0000 7f2cb17fa640 1 --2- 172.21.15.38:0/2060962993 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2c94078bc0 0x7f2c9407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:21.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:21.080+0000 7f2cb17fa640 1 -- 172.21.15.38:0/2060962993 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 7938+0+0 (secure 0 0 0) 0x7f2cac09b850 con 0x7f2cb4161540 2024-09-21T03:42:21.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:21.081+0000 7f2cb8dc1640 1 --2- 172.21.15.38:0/2060962993 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2c94078bc0 0x7f2c9407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:21.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:21.081+0000 7f2cb8dc1640 1 --2- 172.21.15.38:0/2060962993 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2c94078bc0 0x7f2c9407b0d0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f2c9c009cd0 tx=0x7f2c9c005f90 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:21.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:21.085+0000 7f2cb17fa640 1 -- 172.21.15.38:0/2060962993 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f2cb4161950 con 0x7f2cb4161540 2024-09-21T03:42:21.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:21.353+0000 7f2cbb025640 1 -- 172.21.15.38:0/2060962993 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7f2cb40d13a0 con 0x7f2cb4161540 2024-09-21T03:42:21.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:21.943+0000 7f2cb17fa640 1 -- 172.21.15.38:0/2060962993 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 nothing to do: rank is not damaged v200) ==== 107+0+0 (secure 0 0 0) 0x7f2cb41592d0 con 0x7f2cb4161540 2024-09-21T03:42:21.946 INFO:teuthology.orchestra.run.smithi038.stderr:nothing to do: rank is not damaged 2024-09-21T03:42:21.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:21.946+0000 7f2cbb025640 1 -- 172.21.15.38:0/2060962993 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2c94078bc0 msgr2=0x7f2c9407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:21.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:21.946+0000 7f2cbb025640 1 --2- 172.21.15.38:0/2060962993 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2c94078bc0 0x7f2c9407b0d0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f2c9c009cd0 tx=0x7f2c9c005f90 comp rx=0 tx=0).stop 2024-09-21T03:42:21.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:21.946+0000 7f2cbb025640 1 -- 172.21.15.38:0/2060962993 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2cb4161540 msgr2=0x7f2cb406cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:21.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:21.946+0000 7f2cbb025640 1 --2- 172.21.15.38:0/2060962993 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2cb4161540 0x7f2cb406cde0 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7f2cb415c9c0 tx=0x7f2cac006e60 comp rx=0 tx=0).stop 2024-09-21T03:42:21.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:21.946+0000 7f2cbb025640 1 -- 172.21.15.38:0/2060962993 shutdown_connections 2024-09-21T03:42:21.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:21.946+0000 7f2cbb025640 1 --2- 172.21.15.38:0/2060962993 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2cb4161540 0x7f2cb406cde0 unknown :-1 s=CLOSED pgs=1701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:21.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:21.946+0000 7f2cbb025640 1 --2- 172.21.15.38:0/2060962993 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2c94078bc0 0x7f2c9407b0d0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:21.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:21.946+0000 7f2cbb025640 1 --2- 172.21.15.38:0/2060962993 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2cb415c2b0 0x7f2cb4161000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:21.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:21.946+0000 7f2cbb025640 1 --2- 172.21.15.38:0/2060962993 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2ca0004800 0x7f2cb41579d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:21.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:21.946+0000 7f2cbb025640 1 -- 172.21.15.38:0/2060962993 >> 172.21.15.38:0/2060962993 conn(0x7f2cb409d830 msgr2=0x7f2cb40b3fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:21.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:21.946+0000 7f2cbb025640 1 -- 172.21.15.38:0/2060962993 shutdown_connections 2024-09-21T03:42:21.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:21.947+0000 7f2cbb025640 1 -- 172.21.15.38:0/2060962993 wait complete. 2024-09-21T03:42:21.963 DEBUG:teuthology.orchestra.run.smithi038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-09-21T03:42:22.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.022+0000 7f59ac068f40 1 Processor -- start 2024-09-21T03:42:22.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.022+0000 7f59ac068f40 1 -- start start 2024-09-21T03:42:22.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.023+0000 7f59ac068f40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55746fe14090 0x55746fdcd280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:22.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.023+0000 7f59ac068f40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55746fe89520 0x55746fea21e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:22.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.023+0000 7f59ac068f40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55746fe88a90 0x55746fea2b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:22.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.023+0000 7f59ac068f40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55746fd1ad60 con 0x55746fe88a90 2024-09-21T03:42:22.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.023+0000 7f59ac068f40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55746fd79430 con 0x55746fe89520 2024-09-21T03:42:22.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.023+0000 7f59ac068f40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55746fea0af0 con 0x55746fe14090 2024-09-21T03:42:22.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.023+0000 7f59a6325640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55746fe88a90 0x55746fea2b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:22.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.023+0000 7f59a6325640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55746fe88a90 0x55746fea2b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34458/0 (socket says 172.21.15.38:34458) 2024-09-21T03:42:22.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.023+0000 7f59a6325640 1 -- 172.21.15.38:0/2812684569 learned_addr learned my addr 172.21.15.38:0/2812684569 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:42:22.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.023+0000 7f59a5323640 1 --2- 172.21.15.38:0/2812684569 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55746fe89520 0x55746fea21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:22.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.023+0000 7f59a5b24640 1 --2- 172.21.15.38:0/2812684569 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55746fe14090 0x55746fdcd280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:22.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.023+0000 7f59a6325640 1 -- 172.21.15.38:0/2812684569 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55746fe14090 msgr2=0x55746fdcd280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:22.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.023+0000 7f59a6325640 1 --2- 172.21.15.38:0/2812684569 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55746fe14090 0x55746fdcd280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:22.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.023+0000 7f59a6325640 1 -- 172.21.15.38:0/2812684569 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55746fe89520 msgr2=0x55746fea21e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:22.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.023+0000 7f59a6325640 1 --2- 172.21.15.38:0/2812684569 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55746fe89520 0x55746fea21e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:22.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.023+0000 7f59a6325640 1 -- 172.21.15.38:0/2812684569 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55746fea2720 con 0x55746fe88a90 2024-09-21T03:42:22.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.024+0000 7f59a6325640 1 --2- 172.21.15.38:0/2812684569 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55746fe88a90 0x55746fea2b30 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7f59a0025db0 tx=0x7f59a0003330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52571ef862097864 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:22.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.024+0000 7f5997fff640 1 -- 172.21.15.38:0/2812684569 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59a0064030 con 0x55746fe88a90 2024-09-21T03:42:22.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.024+0000 7f5997fff640 1 -- 172.21.15.38:0/2812684569 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f59a0004110 con 0x55746fe88a90 2024-09-21T03:42:22.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.024+0000 7f5997fff640 1 -- 172.21.15.38:0/2812684569 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59a004daa0 con 0x55746fe88a90 2024-09-21T03:42:22.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.024+0000 7f59ac068f40 1 -- 172.21.15.38:0/2812684569 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55746fe88a90 msgr2=0x55746fea2b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:22.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.024+0000 7f59ac068f40 1 --2- 172.21.15.38:0/2812684569 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55746fe88a90 0x55746fea2b30 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7f59a0025db0 tx=0x7f59a0003330 comp rx=0 tx=0).stop 2024-09-21T03:42:22.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.025+0000 7f59ac068f40 1 -- 172.21.15.38:0/2812684569 shutdown_connections 2024-09-21T03:42:22.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.025+0000 7f59ac068f40 1 --2- 172.21.15.38:0/2812684569 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55746fe88a90 0x55746fea2b30 unknown :-1 s=CLOSED pgs=1704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:22.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.025+0000 7f59ac068f40 1 --2- 172.21.15.38:0/2812684569 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55746fe89520 0x55746fea21e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:22.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.025+0000 7f59ac068f40 1 --2- 172.21.15.38:0/2812684569 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55746fe14090 0x55746fdcd280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:22.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.025+0000 7f59ac068f40 1 -- 172.21.15.38:0/2812684569 >> 172.21.15.38:0/2812684569 conn(0x55746fcebac0 msgr2=0x55746fe9e060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:22.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.025+0000 7f59ac068f40 1 -- 172.21.15.38:0/2812684569 shutdown_connections 2024-09-21T03:42:22.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.025+0000 7f59ac068f40 1 -- 172.21.15.38:0/2812684569 wait complete. 2024-09-21T03:42:22.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.026+0000 7f59ac068f40 1 Processor -- start 2024-09-21T03:42:22.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.026+0000 7f59ac068f40 1 -- start start 2024-09-21T03:42:22.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.027+0000 7f59ac068f40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55746fe14090 0x55746fe0aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:22.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.027+0000 7f59ac068f40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55746fe88a90 0x55746ff3af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:22.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.027+0000 7f59ac068f40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55746fe89520 0x55746fe0b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:22.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.027+0000 7f59ac068f40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55746fea15a0 con 0x55746fe88a90 2024-09-21T03:42:22.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.027+0000 7f59ac068f40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55746fdf71c0 con 0x55746fe14090 2024-09-21T03:42:22.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.027+0000 7f59ac068f40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55746fdf6600 con 0x55746fe89520 2024-09-21T03:42:22.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.027+0000 7f59a5323640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55746fe88a90 0x55746ff3af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:22.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.027+0000 7f59a5323640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55746fe88a90 0x55746ff3af80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34474/0 (socket says 172.21.15.38:34474) 2024-09-21T03:42:22.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.027+0000 7f59a5323640 1 -- 172.21.15.38:0/4230160286 learned_addr learned my addr 172.21.15.38:0/4230160286 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:42:22.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.027+0000 7f59a5b24640 1 --2- 172.21.15.38:0/4230160286 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55746fe14090 0x55746fe0aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:22.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.027+0000 7f59a6325640 1 --2- 172.21.15.38:0/4230160286 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55746fe89520 0x55746fe0b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:22.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.027+0000 7f59a5323640 1 -- 172.21.15.38:0/4230160286 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55746fe89520 msgr2=0x55746fe0b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:22.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.027+0000 7f59a5323640 1 --2- 172.21.15.38:0/4230160286 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55746fe89520 0x55746fe0b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:22.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.027+0000 7f59a5323640 1 -- 172.21.15.38:0/4230160286 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55746fe14090 msgr2=0x55746fe0aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:22.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.027+0000 7f59a5323640 1 --2- 172.21.15.38:0/4230160286 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55746fe14090 0x55746fe0aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:22.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.027+0000 7f59a5323640 1 -- 172.21.15.38:0/4230160286 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55746ff3b4c0 con 0x55746fe88a90 2024-09-21T03:42:22.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.027+0000 7f59a5b24640 1 --2- 172.21.15.38:0/4230160286 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55746fe14090 0x55746fe0aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:42:22.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.028+0000 7f59a5323640 1 --2- 172.21.15.38:0/4230160286 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55746fe88a90 0x55746ff3af80 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7f599c003e70 tx=0x7f599c00b590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:22.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.028+0000 7f5996ffd640 1 -- 172.21.15.38:0/4230160286 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f599c0032d0 con 0x55746fe88a90 2024-09-21T03:42:22.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.028+0000 7f5996ffd640 1 -- 172.21.15.38:0/4230160286 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f599c016070 con 0x55746fe88a90 2024-09-21T03:42:22.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.028+0000 7f59ac068f40 1 -- 172.21.15.38:0/4230160286 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55746fe0c6c0 con 0x55746fe88a90 2024-09-21T03:42:22.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.028+0000 7f5996ffd640 1 -- 172.21.15.38:0/4230160286 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f599c0016d0 con 0x55746fe88a90 2024-09-21T03:42:22.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.028+0000 7f59ac068f40 1 -- 172.21.15.38:0/4230160286 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55746fe9ba20 con 0x55746fe88a90 2024-09-21T03:42:22.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.030+0000 7f5996ffd640 1 -- 172.21.15.38:0/4230160286 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f599c001870 con 0x55746fe88a90 2024-09-21T03:42:22.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.030+0000 7f5996ffd640 1 --2- 172.21.15.38:0/4230160286 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5988077570 0x7f5988079a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:22.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.030+0000 7f5996ffd640 1 -- 172.21.15.38:0/4230160286 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 7938+0+0 (secure 0 0 0) 0x7f599c09a360 con 0x55746fe88a90 2024-09-21T03:42:22.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.030+0000 7f59a5b24640 1 --2- 172.21.15.38:0/4230160286 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5988077570 0x7f5988079a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:22.034 INFO:teuthology.orchestra.run.smithi038.stdout:Importing pool 2024-09-21T03:42:22.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.030+0000 7f59ac068f40 1 --2- 172.21.15.38:0/4230160286 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x55746fe9c6d0 0x55746fe9f520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:22.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.030+0000 7f59ac068f40 1 -- 172.21.15.38:0/4230160286 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x55746ff43730 con 0x55746fe9c6d0 2024-09-21T03:42:22.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.031+0000 7f59a6325640 1 --2- 172.21.15.38:0/4230160286 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x55746fe9c6d0 0x55746fe9f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:22.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.031+0000 7f59a5b24640 1 --2- 172.21.15.38:0/4230160286 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5988077570 0x7f5988079a80 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f599800c640 tx=0x7f5998004bb0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:22.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.031+0000 7f59a6325640 1 --2- 172.21.15.38:0/4230160286 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x55746fe9c6d0 0x55746fe9f520 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:22.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.033+0000 7f59a6325640 1 -- 172.21.15.38:0/4230160286 <== osd.7 v2:172.21.15.195:6818/1606106342 1 ==== osd_op_reply(1 601.00000000 [create] v115'567 uv541 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f59a000b040 con 0x55746fe9c6d0 2024-09-21T03:42:22.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.033+0000 7f59ac068f40 1 -- 172.21.15.38:0/4230160286 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e115) -- 0x55746ff43ad0 con 0x55746fe9c6d0 2024-09-21T03:42:22.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.035+0000 7f59a6325640 1 -- 172.21.15.38:0/4230160286 <== osd.7 v2:172.21.15.195:6818/1606106342 2 ==== osd_op_reply(2 601.00000000 [delete] v115'568 uv568 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f59a000b040 con 0x55746fe9c6d0 2024-09-21T03:42:22.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.035+0000 7f59ac068f40 1 -- 172.21.15.38:0/4230160286 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x55746ff43e70 con 0x55746fe9c6d0 2024-09-21T03:42:22.038 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-09-21T03:42:22.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.036+0000 7f59a6325640 1 -- 172.21.15.38:0/4230160286 <== osd.7 v2:172.21.15.195:6818/1606106342 3 ==== osd_op_reply(3 601.00000000 [create] v115'569 uv569 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f59a000b040 con 0x55746fe9c6d0 2024-09-21T03:42:22.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.037+0000 7f59ac068f40 1 -- 172.21.15.38:0/4230160286 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x55746ff45220 con 0x55746fe9c6d0 2024-09-21T03:42:22.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.038+0000 7f59a6325640 1 -- 172.21.15.38:0/4230160286 <== osd.7 v2:172.21.15.195:6818/1606106342 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v115'570 uv570 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f59a000b040 con 0x55746fe9c6d0 2024-09-21T03:42:22.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.038+0000 7f59ac068f40 1 -- 172.21.15.38:0/4230160286 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x55746ff455c0 con 0x55746fe9c6d0 2024-09-21T03:42:22.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.040+0000 7f59a6325640 1 -- 172.21.15.38:0/4230160286 <== osd.7 v2:172.21.15.195:6818/1606106342 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v115'571 uv571 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f59a000b040 con 0x55746fe9c6d0 2024-09-21T03:42:22.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.040+0000 7f59ac068f40 1 -- 172.21.15.38:0/4230160286 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x55746ff44210 con 0x55746fe9c6d0 2024-09-21T03:42:22.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.041+0000 7f59a6325640 1 -- 172.21.15.38:0/4230160286 <== osd.7 v2:172.21.15.195:6818/1606106342 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v115'572 uv572 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f59a000b040 con 0x55746fe9c6d0 2024-09-21T03:42:22.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.041+0000 7f59ac068f40 1 -- 172.21.15.38:0/4230160286 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x55746ff44600 con 0x55746fe9c6d0 2024-09-21T03:42:22.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.043+0000 7f59a6325640 1 -- 172.21.15.38:0/4230160286 <== osd.7 v2:172.21.15.195:6818/1606106342 7 ==== osd_op_reply(7 602.00000000 [create] v115'573 uv547 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f59a000b040 con 0x55746fe9c6d0 2024-09-21T03:42:22.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.043+0000 7f59ac068f40 1 -- 172.21.15.38:0/4230160286 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e115) -- 0x55746ff44600 con 0x55746fe9c6d0 2024-09-21T03:42:22.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.044+0000 7f59a6325640 1 -- 172.21.15.38:0/4230160286 <== osd.7 v2:172.21.15.195:6818/1606106342 8 ==== osd_op_reply(8 602.00000000 [delete] v115'574 uv574 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f59a000b040 con 0x55746fe9c6d0 2024-09-21T03:42:22.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.044+0000 7f59ac068f40 1 -- 172.21.15.38:0/4230160286 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x55746ff44600 con 0x55746fe9c6d0 2024-09-21T03:42:22.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.046+0000 7f59a6325640 1 -- 172.21.15.38:0/4230160286 <== osd.7 v2:172.21.15.195:6818/1606106342 9 ==== osd_op_reply(9 602.00000000 [create] v115'575 uv575 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f59a000b040 con 0x55746fe9c6d0 2024-09-21T03:42:22.048 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-09-21T03:42:22.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.046+0000 7f59ac068f40 1 -- 172.21.15.38:0/4230160286 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x55746ff44600 con 0x55746fe9c6d0 2024-09-21T03:42:22.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.047+0000 7f59a6325640 1 -- 172.21.15.38:0/4230160286 <== osd.7 v2:172.21.15.195:6818/1606106342 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v115'576 uv576 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f59a000b040 con 0x55746fe9c6d0 2024-09-21T03:42:22.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.047+0000 7f59ac068f40 1 -- 172.21.15.38:0/4230160286 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x55746ff44600 con 0x55746fe9c6d0 2024-09-21T03:42:22.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.049+0000 7f59a6325640 1 -- 172.21.15.38:0/4230160286 <== osd.7 v2:172.21.15.195:6818/1606106342 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v115'577 uv577 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f59a000b040 con 0x55746fe9c6d0 2024-09-21T03:42:22.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.049+0000 7f59ac068f40 1 -- 172.21.15.38:0/4230160286 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x55746ff44600 con 0x55746fe9c6d0 2024-09-21T03:42:22.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.050+0000 7f59a6325640 1 -- 172.21.15.38:0/4230160286 <== osd.7 v2:172.21.15.195:6818/1606106342 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v115'578 uv578 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f59a000b040 con 0x55746fe9c6d0 2024-09-21T03:42:22.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.050+0000 7f59ac068f40 1 -- 172.21.15.38:0/4230160286 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:13 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x55746ff44600 con 0x55746fe9c6d0 2024-09-21T03:42:22.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.052+0000 7f59a6325640 1 -- 172.21.15.38:0/4230160286 <== osd.7 v2:172.21.15.195:6818/1606106342 13 ==== osd_op_reply(13 10000000000.00000000 [create] v115'579 uv554 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7f59a000b040 con 0x55746fe9c6d0 2024-09-21T03:42:22.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.052+0000 7f59ac068f40 1 -- 172.21.15.38:0/4230160286 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:14 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e115) -- 0x55746ff44600 con 0x55746fe9c6d0 2024-09-21T03:42:22.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.054+0000 7f59a6325640 1 -- 172.21.15.38:0/4230160286 <== osd.7 v2:172.21.15.195:6818/1606106342 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v115'580 uv580 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f59a000b040 con 0x55746fe9c6d0 2024-09-21T03:42:22.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.054+0000 7f59ac068f40 1 -- 172.21.15.38:0/4230160286 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:15 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x55746ff44600 con 0x55746fe9c6d0 2024-09-21T03:42:22.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.056+0000 7f59a6325640 1 -- 172.21.15.38:0/4230160286 <== osd.7 v2:172.21.15.195:6818/1606106342 15 ==== osd_op_reply(15 10000000000.00000000 [create] v115'581 uv581 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f59a000b040 con 0x55746fe9c6d0 2024-09-21T03:42:22.057 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-09-21T03:42:22.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.056+0000 7f59ac068f40 1 -- 172.21.15.38:0/4230160286 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:16 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x55746ff44600 con 0x55746fe9c6d0 2024-09-21T03:42:22.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.057+0000 7f59a6325640 1 -- 172.21.15.38:0/4230160286 <== osd.7 v2:172.21.15.195:6818/1606106342 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v115'582 uv582 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f59a000b040 con 0x55746fe9c6d0 2024-09-21T03:42:22.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.057+0000 7f59ac068f40 1 -- 172.21.15.38:0/4230160286 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:17 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x55746ff44600 con 0x55746fe9c6d0 2024-09-21T03:42:22.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.059+0000 7f59a6325640 1 -- 172.21.15.38:0/4230160286 <== osd.7 v2:172.21.15.195:6818/1606106342 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v115'583 uv583 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f59a000b040 con 0x55746fe9c6d0 2024-09-21T03:42:22.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.059+0000 7f59ac068f40 1 -- 172.21.15.38:0/4230160286 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:18 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x55746ff44600 con 0x55746fe9c6d0 2024-09-21T03:42:22.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.060+0000 7f59a6325640 1 -- 172.21.15.38:0/4230160286 <== osd.7 v2:172.21.15.195:6818/1606106342 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v115'584 uv584 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f59a000b040 con 0x55746fe9c6d0 2024-09-21T03:42:22.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.060+0000 7f59ac068f40 1 -- 172.21.15.38:0/4230160286 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:19 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=585b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x55746ff44600 con 0x55746fe9c6d0 2024-09-21T03:42:22.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.061+0000 7f59a6325640 1 -- 172.21.15.38:0/4230160286 <== osd.7 v2:172.21.15.195:6818/1606106342 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v115'585 uv585 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f59a000b040 con 0x55746fe9c6d0 2024-09-21T03:42:22.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.062+0000 7f59ac068f40 1 -- 172.21.15.38:0/4230160286 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:20 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x55746ff44600 con 0x55746fe9c6d0 2024-09-21T03:42:22.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.063+0000 7f59a6325640 1 -- 172.21.15.38:0/4230160286 <== osd.7 v2:172.21.15.195:6818/1606106342 20 ==== osd_op_reply(20 600.00000000 [create] v115'586 uv560 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f59a000b040 con 0x55746fe9c6d0 2024-09-21T03:42:22.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.063+0000 7f59ac068f40 1 -- 172.21.15.38:0/4230160286 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:21 13.10 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e115) -- 0x55746ff44600 con 0x55746fe9c6d0 2024-09-21T03:42:22.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.065+0000 7f59a6325640 1 -- 172.21.15.38:0/4230160286 <== osd.7 v2:172.21.15.195:6818/1606106342 21 ==== osd_op_reply(21 600.00000000 [delete] v115'587 uv587 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f59a000b040 con 0x55746fe9c6d0 2024-09-21T03:42:22.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.065+0000 7f59ac068f40 1 -- 172.21.15.38:0/4230160286 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:22 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x55746ff44600 con 0x55746fe9c6d0 2024-09-21T03:42:22.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.066+0000 7f59a6325640 1 -- 172.21.15.38:0/4230160286 <== osd.7 v2:172.21.15.195:6818/1606106342 22 ==== osd_op_reply(22 600.00000000 [create] v115'588 uv588 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f59a000b040 con 0x55746fe9c6d0 2024-09-21T03:42:22.068 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-09-21T03:42:22.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.066+0000 7f59ac068f40 1 -- 172.21.15.38:0/4230160286 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:23 13.10 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x55746ff44600 con 0x55746fe9c6d0 2024-09-21T03:42:22.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.068+0000 7f59a6325640 1 -- 172.21.15.38:0/4230160286 <== osd.7 v2:172.21.15.195:6818/1606106342 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v115'589 uv589 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f59a000b040 con 0x55746fe9c6d0 2024-09-21T03:42:22.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.068+0000 7f59ac068f40 1 -- 172.21.15.38:0/4230160286 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:24 13.10 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x55746ff44600 con 0x55746fe9c6d0 2024-09-21T03:42:22.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.069+0000 7f59a6325640 1 -- 172.21.15.38:0/4230160286 <== osd.7 v2:172.21.15.195:6818/1606106342 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v115'590 uv590 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f59a000b040 con 0x55746fe9c6d0 2024-09-21T03:42:22.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.069+0000 7f59ac068f40 1 -- 172.21.15.38:0/4230160286 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:25 13.10 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x55746ff44600 con 0x55746fe9c6d0 2024-09-21T03:42:22.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.071+0000 7f59a6325640 1 -- 172.21.15.38:0/4230160286 <== osd.7 v2:172.21.15.195:6818/1606106342 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v115'591 uv591 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f59a000b040 con 0x55746fe9c6d0 2024-09-21T03:42:22.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.071+0000 7f59ac068f40 1 -- 172.21.15.38:0/4230160286 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:26 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x55746ff44600 con 0x55746fe9c6d0 2024-09-21T03:42:22.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.072+0000 7f59a6325640 1 -- 172.21.15.38:0/4230160286 <== osd.7 v2:172.21.15.195:6818/1606106342 26 ==== osd_op_reply(26 603.00000000 [create] v115'592 uv566 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f59a000b040 con 0x55746fe9c6d0 2024-09-21T03:42:22.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.072+0000 7f59ac068f40 1 -- 172.21.15.38:0/4230160286 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:27 13.10 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e115) -- 0x55746ff44600 con 0x55746fe9c6d0 2024-09-21T03:42:22.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.074+0000 7f59a6325640 1 -- 172.21.15.38:0/4230160286 <== osd.7 v2:172.21.15.195:6818/1606106342 27 ==== osd_op_reply(27 603.00000000 [delete] v115'593 uv593 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f59a000b040 con 0x55746fe9c6d0 2024-09-21T03:42:22.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.074+0000 7f59ac068f40 1 -- 172.21.15.38:0/4230160286 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:28 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x55746ff44600 con 0x55746fe9c6d0 2024-09-21T03:42:22.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.075+0000 7f59a6325640 1 -- 172.21.15.38:0/4230160286 <== osd.7 v2:172.21.15.195:6818/1606106342 28 ==== osd_op_reply(28 603.00000000 [create] v115'594 uv594 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f59a000b040 con 0x55746fe9c6d0 2024-09-21T03:42:22.077 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-09-21T03:42:22.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.075+0000 7f59ac068f40 1 -- 172.21.15.38:0/4230160286 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:29 13.10 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x55746ff44600 con 0x55746fe9c6d0 2024-09-21T03:42:22.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.076+0000 7f59a6325640 1 -- 172.21.15.38:0/4230160286 <== osd.7 v2:172.21.15.195:6818/1606106342 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v115'595 uv595 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f59a000b040 con 0x55746fe9c6d0 2024-09-21T03:42:22.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.076+0000 7f59ac068f40 1 -- 172.21.15.38:0/4230160286 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:30 13.10 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x55746ff44600 con 0x55746fe9c6d0 2024-09-21T03:42:22.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.078+0000 7f59a6325640 1 -- 172.21.15.38:0/4230160286 <== osd.7 v2:172.21.15.195:6818/1606106342 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v115'596 uv596 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f59a000b040 con 0x55746fe9c6d0 2024-09-21T03:42:22.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.078+0000 7f59ac068f40 1 -- 172.21.15.38:0/4230160286 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:31 13.10 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x55746ff44600 con 0x55746fe9c6d0 2024-09-21T03:42:22.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.079+0000 7f59a6325640 1 -- 172.21.15.38:0/4230160286 <== osd.7 v2:172.21.15.195:6818/1606106342 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v115'597 uv597 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f59a000b040 con 0x55746fe9c6d0 2024-09-21T03:42:22.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.079+0000 7f59ac068f40 1 --2- 172.21.15.38:0/4230160286 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x55746ff45960 0x55746ff47d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:22.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.079+0000 7f59ac068f40 1 -- 172.21.15.38:0/4230160286 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x55746ff44600 con 0x55746ff45960 2024-09-21T03:42:22.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.080+0000 7f59a5b24640 1 --2- 172.21.15.38:0/4230160286 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x55746ff45960 0x55746ff47d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:22.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.080+0000 7f59a5b24640 1 --2- 172.21.15.38:0/4230160286 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x55746ff45960 0x55746ff47d80 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:22.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.082+0000 7f59a5b24640 1 -- 172.21.15.38:0/4230160286 <== osd.6 v2:172.21.15.195:6826/4055825675 1 ==== osd_op_reply(32 1.00000000.inode [create] v115'420 uv399 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f599800a6c0 con 0x55746ff45960 2024-09-21T03:42:22.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.082+0000 7f59ac068f40 1 -- 172.21.15.38:0/4230160286 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e115) -- 0x55746ff482c0 con 0x55746ff45960 2024-09-21T03:42:22.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.083+0000 7f59a5b24640 1 -- 172.21.15.38:0/4230160286 <== osd.6 v2:172.21.15.195:6826/4055825675 2 ==== osd_op_reply(33 1.00000000.inode [delete] v115'421 uv421 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f599800a6c0 con 0x55746ff45960 2024-09-21T03:42:22.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.083+0000 7f59ac068f40 1 -- 172.21.15.38:0/4230160286 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x55746ff48660 con 0x55746ff45960 2024-09-21T03:42:22.086 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-09-21T03:42:22.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.084+0000 7f59a5b24640 1 -- 172.21.15.38:0/4230160286 <== osd.6 v2:172.21.15.195:6826/4055825675 3 ==== osd_op_reply(34 1.00000000.inode [create] v115'422 uv422 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f599800a6c0 con 0x55746ff45960 2024-09-21T03:42:22.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.085+0000 7f59ac068f40 1 -- 172.21.15.38:0/4230160286 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x55746ff48a00 con 0x55746ff45960 2024-09-21T03:42:22.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.086+0000 7f59a5b24640 1 -- 172.21.15.38:0/4230160286 <== osd.6 v2:172.21.15.195:6826/4055825675 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v115'423 uv423 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f599800a6c0 con 0x55746ff45960 2024-09-21T03:42:22.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.086+0000 7f59ac068f40 1 -- 172.21.15.38:0/4230160286 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x55746ff48da0 con 0x55746ff45960 2024-09-21T03:42:22.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.087+0000 7f59a5b24640 1 -- 172.21.15.38:0/4230160286 <== osd.6 v2:172.21.15.195:6826/4055825675 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v115'424 uv424 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f599800a6c0 con 0x55746ff45960 2024-09-21T03:42:22.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.087+0000 7f59ac068f40 1 -- 172.21.15.38:0/4230160286 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x55746ff49140 con 0x55746ff45960 2024-09-21T03:42:22.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.088+0000 7f59a5b24640 1 -- 172.21.15.38:0/4230160286 <== osd.6 v2:172.21.15.195:6826/4055825675 6 ==== osd_op_reply(37 200.00000000 [create] v115'425 uv418 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f599800a6c0 con 0x55746ff45960 2024-09-21T03:42:22.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.088+0000 7f59ac068f40 1 -- 172.21.15.38:0/4230160286 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e115) -- 0x55746ff494e0 con 0x55746ff45960 2024-09-21T03:42:22.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.090+0000 7f59a5b24640 1 -- 172.21.15.38:0/4230160286 <== osd.6 v2:172.21.15.195:6826/4055825675 7 ==== osd_op_reply(38 200.00000000 [delete] v115'426 uv426 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f599800a6c0 con 0x55746ff45960 2024-09-21T03:42:22.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.090+0000 7f59ac068f40 1 -- 172.21.15.38:0/4230160286 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x55746ff49880 con 0x55746ff45960 2024-09-21T03:42:22.092 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-09-21T03:42:22.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.091+0000 7f59a5b24640 1 -- 172.21.15.38:0/4230160286 <== osd.6 v2:172.21.15.195:6826/4055825675 8 ==== osd_op_reply(39 200.00000000 [create] v115'427 uv427 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f599800a6c0 con 0x55746ff45960 2024-09-21T03:42:22.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.091+0000 7f59ac068f40 1 -- 172.21.15.38:0/4230160286 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x55746fe9c300 con 0x55746ff45960 2024-09-21T03:42:22.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.092+0000 7f59a5b24640 1 -- 172.21.15.38:0/4230160286 <== osd.6 v2:172.21.15.195:6826/4055825675 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v115'428 uv428 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f599800a6c0 con 0x55746ff45960 2024-09-21T03:42:22.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.092+0000 7f59ac068f40 1 -- 172.21.15.38:0/4230160286 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x55746fe9c300 con 0x55746ff45960 2024-09-21T03:42:22.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.093+0000 7f59a5b24640 1 -- 172.21.15.38:0/4230160286 <== osd.6 v2:172.21.15.195:6826/4055825675 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v115'429 uv429 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f599800a6c0 con 0x55746ff45960 2024-09-21T03:42:22.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.093+0000 7f59ac068f40 1 -- 172.21.15.38:0/4230160286 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:42 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x55746fe9c300 con 0x55746ff45960 2024-09-21T03:42:22.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.095+0000 7f59a5b24640 1 -- 172.21.15.38:0/4230160286 <== osd.6 v2:172.21.15.195:6826/4055825675 11 ==== osd_op_reply(42 200.00000001 [create] v115'430 uv419 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f599800a6c0 con 0x55746ff45960 2024-09-21T03:42:22.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.095+0000 7f59ac068f40 1 -- 172.21.15.38:0/4230160286 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:43 13.14 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e115) -- 0x55746fe9c300 con 0x55746ff45960 2024-09-21T03:42:22.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.096+0000 7f59a5b24640 1 -- 172.21.15.38:0/4230160286 <== osd.6 v2:172.21.15.195:6826/4055825675 12 ==== osd_op_reply(43 200.00000001 [delete] v115'431 uv431 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f599800a6c0 con 0x55746ff45960 2024-09-21T03:42:22.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.096+0000 7f59ac068f40 1 -- 172.21.15.38:0/4230160286 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:44 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x55746fe9c300 con 0x55746ff45960 2024-09-21T03:42:22.099 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-09-21T03:42:22.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.097+0000 7f59a5b24640 1 -- 172.21.15.38:0/4230160286 <== osd.6 v2:172.21.15.195:6826/4055825675 13 ==== osd_op_reply(44 200.00000001 [create] v115'432 uv432 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f599800a6c0 con 0x55746ff45960 2024-09-21T03:42:22.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.097+0000 7f59ac068f40 1 -- 172.21.15.38:0/4230160286 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:45 13.14 13:2e2fa760:::200.00000001:head [write 0~17490 in=17490b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x55746fe9c300 con 0x55746ff45960 2024-09-21T03:42:22.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.099+0000 7f59a5b24640 1 -- 172.21.15.38:0/4230160286 <== osd.6 v2:172.21.15.195:6826/4055825675 14 ==== osd_op_reply(45 200.00000001 [write 0~17490] v115'433 uv433 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f599800a6c0 con 0x55746ff45960 2024-09-21T03:42:22.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.099+0000 7f59ac068f40 1 -- 172.21.15.38:0/4230160286 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:46 13.14 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x55746fe9c300 con 0x55746ff45960 2024-09-21T03:42:22.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.100+0000 7f59a5b24640 1 -- 172.21.15.38:0/4230160286 <== osd.6 v2:172.21.15.195:6826/4055825675 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v115'434 uv434 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f599800a6c0 con 0x55746ff45960 2024-09-21T03:42:22.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.100+0000 7f59ac068f40 1 -- 172.21.15.38:0/4230160286 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:47 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x55746fe9c300 con 0x55746ff45960 2024-09-21T03:42:22.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.102+0000 7f59a5b24640 1 -- 172.21.15.38:0/4230160286 <== osd.6 v2:172.21.15.195:6826/4055825675 16 ==== osd_op_reply(47 606.00000000 [create] v115'435 uv415 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f599800a6c0 con 0x55746ff45960 2024-09-21T03:42:22.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.102+0000 7f59ac068f40 1 -- 172.21.15.38:0/4230160286 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:48 13.14 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e115) -- 0x55746fe9c300 con 0x55746ff45960 2024-09-21T03:42:22.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.103+0000 7f59a5b24640 1 -- 172.21.15.38:0/4230160286 <== osd.6 v2:172.21.15.195:6826/4055825675 17 ==== osd_op_reply(48 606.00000000 [delete] v115'436 uv436 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f599800a6c0 con 0x55746ff45960 2024-09-21T03:42:22.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.103+0000 7f59ac068f40 1 -- 172.21.15.38:0/4230160286 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:49 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x55746fe9c300 con 0x55746ff45960 2024-09-21T03:42:22.106 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-09-21T03:42:22.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.104+0000 7f59a5b24640 1 -- 172.21.15.38:0/4230160286 <== osd.6 v2:172.21.15.195:6826/4055825675 18 ==== osd_op_reply(49 606.00000000 [create] v115'437 uv437 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f599800a6c0 con 0x55746ff45960 2024-09-21T03:42:22.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.104+0000 7f59ac068f40 1 -- 172.21.15.38:0/4230160286 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:50 13.14 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x55746fe9c300 con 0x55746ff45960 2024-09-21T03:42:22.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.105+0000 7f59a5b24640 1 -- 172.21.15.38:0/4230160286 <== osd.6 v2:172.21.15.195:6826/4055825675 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v115'438 uv438 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f599800a6c0 con 0x55746ff45960 2024-09-21T03:42:22.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.105+0000 7f59ac068f40 1 -- 172.21.15.38:0/4230160286 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:51 13.14 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x55746fe9c300 con 0x55746ff45960 2024-09-21T03:42:22.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.106+0000 7f59a5b24640 1 -- 172.21.15.38:0/4230160286 <== osd.6 v2:172.21.15.195:6826/4055825675 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v115'439 uv439 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f599800a6c0 con 0x55746ff45960 2024-09-21T03:42:22.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.107+0000 7f59ac068f40 1 -- 172.21.15.38:0/4230160286 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:52 13.14 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x55746fe9c300 con 0x55746ff45960 2024-09-21T03:42:22.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.108+0000 7f59a5b24640 1 -- 172.21.15.38:0/4230160286 <== osd.6 v2:172.21.15.195:6826/4055825675 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v115'440 uv440 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f599800a6c0 con 0x55746ff45960 2024-09-21T03:42:22.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.108+0000 7f59ac068f40 1 --2- 172.21.15.38:0/4230160286 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x55746ff4a240 0x55746ff4c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:22.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.108+0000 7f59ac068f40 1 -- 172.21.15.38:0/4230160286 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:53 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x55746ff4cba0 con 0x55746ff4a240 2024-09-21T03:42:22.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.108+0000 7f59a5323640 1 --2- 172.21.15.38:0/4230160286 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x55746ff4a240 0x55746ff4c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:22.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.108+0000 7f59a5323640 1 --2- 172.21.15.38:0/4230160286 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x55746ff4a240 0x55746ff4c660 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:22.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.110+0000 7f59a5323640 1 -- 172.21.15.38:0/4230160286 <== osd.0 v2:172.21.15.38:6800/2381639433 1 ==== osd_op_reply(53 607.00000000 [create] v115'111 uv110 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f599c09a050 con 0x55746ff4a240 2024-09-21T03:42:22.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.110+0000 7f59ac068f40 1 -- 172.21.15.38:0/4230160286 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:54 13.1c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e115) -- 0x55746ff4d000 con 0x55746ff4a240 2024-09-21T03:42:22.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.111+0000 7f59a5323640 1 -- 172.21.15.38:0/4230160286 <== osd.0 v2:172.21.15.38:6800/2381639433 2 ==== osd_op_reply(54 607.00000000 [delete] v115'112 uv112 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f599c09a050 con 0x55746ff4a240 2024-09-21T03:42:22.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.111+0000 7f59ac068f40 1 -- 172.21.15.38:0/4230160286 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:55 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x55746ff4d4e0 con 0x55746ff4a240 2024-09-21T03:42:22.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.112+0000 7f59a5323640 1 -- 172.21.15.38:0/4230160286 <== osd.0 v2:172.21.15.38:6800/2381639433 3 ==== osd_op_reply(55 607.00000000 [create] v115'113 uv113 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f599c09a050 con 0x55746ff4a240 2024-09-21T03:42:22.114 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-09-21T03:42:22.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.112+0000 7f59ac068f40 1 -- 172.21.15.38:0/4230160286 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:56 13.1c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x55746ff49980 con 0x55746ff4a240 2024-09-21T03:42:22.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.113+0000 7f59a5323640 1 -- 172.21.15.38:0/4230160286 <== osd.0 v2:172.21.15.38:6800/2381639433 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v115'114 uv114 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f599c09a050 con 0x55746ff4a240 2024-09-21T03:42:22.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.113+0000 7f59ac068f40 1 -- 172.21.15.38:0/4230160286 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:57 13.1c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x55746ff4ee70 con 0x55746ff4a240 2024-09-21T03:42:22.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.114+0000 7f59a5323640 1 -- 172.21.15.38:0/4230160286 <== osd.0 v2:172.21.15.38:6800/2381639433 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v115'115 uv115 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f599c09a050 con 0x55746ff4a240 2024-09-21T03:42:22.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.114+0000 7f59ac068f40 1 -- 172.21.15.38:0/4230160286 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:58 13.1c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x55746ff4dd80 con 0x55746ff4a240 2024-09-21T03:42:22.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.115+0000 7f59a5323640 1 -- 172.21.15.38:0/4230160286 <== osd.0 v2:172.21.15.38:6800/2381639433 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v115'116 uv116 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f599c09a050 con 0x55746ff4a240 2024-09-21T03:42:22.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.115+0000 7f59ac068f40 1 --2- 172.21.15.38:0/4230160286 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x55746ff4e860 0x55746ff512d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:22.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.115+0000 7f59ac068f40 1 -- 172.21.15.38:0/4230160286 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:59 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x55746ff51870 con 0x55746ff4e860 2024-09-21T03:42:22.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.116+0000 7f59a6325640 1 --2- 172.21.15.38:0/4230160286 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x55746ff4e860 0x55746ff512d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:22.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.116+0000 7f59a6325640 1 --2- 172.21.15.38:0/4230160286 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x55746ff4e860 0x55746ff512d0 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:22.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.117+0000 7f59a6325640 1 -- 172.21.15.38:0/4230160286 <== osd.3 v2:172.21.15.38:6808/1286810294 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v115'82 uv81 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f59a000b040 con 0x55746ff4e860 2024-09-21T03:42:22.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.117+0000 7f59ac068f40 1 -- 172.21.15.38:0/4230160286 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:60 13.6 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e115) -- 0x55746ff51c60 con 0x55746ff4e860 2024-09-21T03:42:22.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.118+0000 7f59a6325640 1 -- 172.21.15.38:0/4230160286 <== osd.3 v2:172.21.15.38:6808/1286810294 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v115'83 uv83 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f59a000b040 con 0x55746ff4e860 2024-09-21T03:42:22.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.118+0000 7f59ac068f40 1 -- 172.21.15.38:0/4230160286 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:61 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x55746ff51c60 con 0x55746ff4e860 2024-09-21T03:42:22.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.120+0000 7f59a6325640 1 -- 172.21.15.38:0/4230160286 <== osd.3 v2:172.21.15.38:6808/1286810294 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v115'84 uv84 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f59a000b040 con 0x55746ff4e860 2024-09-21T03:42:22.122 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-09-21T03:42:22.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.120+0000 7f59ac068f40 1 -- 172.21.15.38:0/4230160286 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:62 13.6 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x55746ff51c60 con 0x55746ff4e860 2024-09-21T03:42:22.123 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.121+0000 7f59a6325640 1 -- 172.21.15.38:0/4230160286 <== osd.3 v2:172.21.15.38:6808/1286810294 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v115'85 uv85 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f59a000b040 con 0x55746ff4e860 2024-09-21T03:42:22.123 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.121+0000 7f59ac068f40 1 -- 172.21.15.38:0/4230160286 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x55746ff51b80 con 0x55746ff4a240 2024-09-21T03:42:22.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.122+0000 7f59a5323640 1 -- 172.21.15.38:0/4230160286 <== osd.0 v2:172.21.15.38:6800/2381639433 7 ==== osd_op_reply(63 608.00000000 [create] v115'111 uv110 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f599c09a050 con 0x55746ff4a240 2024-09-21T03:42:22.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.122+0000 7f59ac068f40 1 -- 172.21.15.38:0/4230160286 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e115) -- 0x55746ff52050 con 0x55746ff4a240 2024-09-21T03:42:22.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.124+0000 7f59a5323640 1 -- 172.21.15.38:0/4230160286 <== osd.0 v2:172.21.15.38:6800/2381639433 8 ==== osd_op_reply(64 608.00000000 [delete] v115'112 uv112 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f599c09a050 con 0x55746ff4a240 2024-09-21T03:42:22.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.124+0000 7f59ac068f40 1 -- 172.21.15.38:0/4230160286 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x55746ff523f0 con 0x55746ff4a240 2024-09-21T03:42:22.126 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.125+0000 7f59a5323640 1 -- 172.21.15.38:0/4230160286 <== osd.0 v2:172.21.15.38:6800/2381639433 9 ==== osd_op_reply(65 608.00000000 [create] v115'113 uv113 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f599c09a050 con 0x55746ff4a240 2024-09-21T03:42:22.127 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-09-21T03:42:22.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.125+0000 7f59ac068f40 1 -- 172.21.15.38:0/4230160286 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x55746ff53400 con 0x55746ff4a240 2024-09-21T03:42:22.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.126+0000 7f59a5323640 1 -- 172.21.15.38:0/4230160286 <== osd.0 v2:172.21.15.38:6800/2381639433 10 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v115'114 uv114 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f599c09a050 con 0x55746ff4a240 2024-09-21T03:42:22.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.126+0000 7f59ac068f40 1 -- 172.21.15.38:0/4230160286 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x55746ff53400 con 0x55746ff4a240 2024-09-21T03:42:22.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.127+0000 7f59a5323640 1 -- 172.21.15.38:0/4230160286 <== osd.0 v2:172.21.15.38:6800/2381639433 11 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v115'115 uv115 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f599c09a050 con 0x55746ff4a240 2024-09-21T03:42:22.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.127+0000 7f59ac068f40 1 -- 172.21.15.38:0/4230160286 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x55746ff523f0 con 0x55746ff4a240 2024-09-21T03:42:22.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.129+0000 7f59a5323640 1 -- 172.21.15.38:0/4230160286 <== osd.0 v2:172.21.15.38:6800/2381639433 12 ==== osd_op_reply(68 608.00000000 [omap-set-header] v115'116 uv116 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f599c09a050 con 0x55746ff4a240 2024-09-21T03:42:22.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.129+0000 7f59ac068f40 1 -- 172.21.15.38:0/4230160286 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x55746ff523f0 con 0x55746ff4e860 2024-09-21T03:42:22.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.130+0000 7f59a6325640 1 -- 172.21.15.38:0/4230160286 <== osd.3 v2:172.21.15.38:6808/1286810294 5 ==== osd_op_reply(69 604.00000000 [create] v115'205 uv198 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f59a000b040 con 0x55746ff4e860 2024-09-21T03:42:22.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.130+0000 7f59ac068f40 1 -- 172.21.15.38:0/4230160286 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e115) -- 0x55746ff523f0 con 0x55746ff4e860 2024-09-21T03:42:22.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.131+0000 7f59a6325640 1 -- 172.21.15.38:0/4230160286 <== osd.3 v2:172.21.15.38:6808/1286810294 6 ==== osd_op_reply(70 604.00000000 [delete] v115'206 uv206 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f59a000b040 con 0x55746ff4e860 2024-09-21T03:42:22.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.131+0000 7f59ac068f40 1 -- 172.21.15.38:0/4230160286 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x55746ff523f0 con 0x55746ff4e860 2024-09-21T03:42:22.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.132+0000 7f59a6325640 1 -- 172.21.15.38:0/4230160286 <== osd.3 v2:172.21.15.38:6808/1286810294 7 ==== osd_op_reply(71 604.00000000 [create] v115'207 uv207 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f59a000b040 con 0x55746ff4e860 2024-09-21T03:42:22.134 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-09-21T03:42:22.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.132+0000 7f59ac068f40 1 -- 172.21.15.38:0/4230160286 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x55746ff523f0 con 0x55746ff4e860 2024-09-21T03:42:22.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.133+0000 7f59a6325640 1 -- 172.21.15.38:0/4230160286 <== osd.3 v2:172.21.15.38:6808/1286810294 8 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v115'208 uv208 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f59a000b040 con 0x55746ff4e860 2024-09-21T03:42:22.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.133+0000 7f59ac068f40 1 -- 172.21.15.38:0/4230160286 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x55746ff523f0 con 0x55746ff4e860 2024-09-21T03:42:22.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.134+0000 7f59a6325640 1 -- 172.21.15.38:0/4230160286 <== osd.3 v2:172.21.15.38:6808/1286810294 9 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v115'209 uv209 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f59a000b040 con 0x55746ff4e860 2024-09-21T03:42:22.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.134+0000 7f59ac068f40 1 -- 172.21.15.38:0/4230160286 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x55746ff523f0 con 0x55746ff4e860 2024-09-21T03:42:22.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.135+0000 7f59a6325640 1 -- 172.21.15.38:0/4230160286 <== osd.3 v2:172.21.15.38:6808/1286810294 10 ==== osd_op_reply(74 604.00000000 [omap-set-header] v115'210 uv210 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f59a000b040 con 0x55746ff4e860 2024-09-21T03:42:22.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.135+0000 7f59ac068f40 1 -- 172.21.15.38:0/4230160286 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x55746ff523f0 con 0x55746ff4e860 2024-09-21T03:42:22.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.136+0000 7f59a6325640 1 -- 172.21.15.38:0/4230160286 <== osd.3 v2:172.21.15.38:6808/1286810294 11 ==== osd_op_reply(75 500.00000000 [create] v115'211 uv204 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f59a000b040 con 0x55746ff4e860 2024-09-21T03:42:22.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.136+0000 7f59ac068f40 1 -- 172.21.15.38:0/4230160286 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e115) -- 0x55746ff523f0 con 0x55746ff4e860 2024-09-21T03:42:22.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.137+0000 7f59a6325640 1 -- 172.21.15.38:0/4230160286 <== osd.3 v2:172.21.15.38:6808/1286810294 12 ==== osd_op_reply(76 500.00000000 [delete] v115'212 uv212 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f59a000b040 con 0x55746ff4e860 2024-09-21T03:42:22.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.137+0000 7f59ac068f40 1 -- 172.21.15.38:0/4230160286 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x55746ff523f0 con 0x55746ff4e860 2024-09-21T03:42:22.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.138+0000 7f59a6325640 1 -- 172.21.15.38:0/4230160286 <== osd.3 v2:172.21.15.38:6808/1286810294 13 ==== osd_op_reply(77 500.00000000 [create] v115'213 uv213 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f59a000b040 con 0x55746ff4e860 2024-09-21T03:42:22.140 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-09-21T03:42:22.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.138+0000 7f59ac068f40 1 -- 172.21.15.38:0/4230160286 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x55746ff523f0 con 0x55746ff4e860 2024-09-21T03:42:22.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.139+0000 7f59a6325640 1 -- 172.21.15.38:0/4230160286 <== osd.3 v2:172.21.15.38:6808/1286810294 14 ==== osd_op_reply(78 500.00000000 [write 0~90] v115'214 uv214 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f59a000b040 con 0x55746ff4e860 2024-09-21T03:42:22.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.139+0000 7f59ac068f40 1 -- 172.21.15.38:0/4230160286 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x55746ff523f0 con 0x55746ff4e860 2024-09-21T03:42:22.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.140+0000 7f59a6325640 1 -- 172.21.15.38:0/4230160286 <== osd.3 v2:172.21.15.38:6808/1286810294 15 ==== osd_op_reply(79 500.00000000 [omap-set-header] v115'215 uv215 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f59a000b040 con 0x55746ff4e860 2024-09-21T03:42:22.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.140+0000 7f59ac068f40 1 -- 172.21.15.38:0/4230160286 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:80 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x55746ff523f0 con 0x55746ff4a240 2024-09-21T03:42:22.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.141+0000 7f59a5323640 1 -- 172.21.15.38:0/4230160286 <== osd.0 v2:172.21.15.38:6800/2381639433 13 ==== osd_op_reply(80 mds_snaptable [create] v115'91 uv90 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f599c09a050 con 0x55746ff4a240 2024-09-21T03:42:22.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.141+0000 7f59ac068f40 1 -- 172.21.15.38:0/4230160286 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:81 13.d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e115) -- 0x55746ff523f0 con 0x55746ff4a240 2024-09-21T03:42:22.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.142+0000 7f59a5323640 1 -- 172.21.15.38:0/4230160286 <== osd.0 v2:172.21.15.38:6800/2381639433 14 ==== osd_op_reply(81 mds_snaptable [delete] v115'92 uv92 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f599c09a050 con 0x55746ff4a240 2024-09-21T03:42:22.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.142+0000 7f59ac068f40 1 -- 172.21.15.38:0/4230160286 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:82 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x55746ff523f0 con 0x55746ff4a240 2024-09-21T03:42:22.145 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-09-21T03:42:22.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.143+0000 7f59a5323640 1 -- 172.21.15.38:0/4230160286 <== osd.0 v2:172.21.15.38:6800/2381639433 15 ==== osd_op_reply(82 mds_snaptable [create] v115'93 uv93 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f599c09a050 con 0x55746ff4a240 2024-09-21T03:42:22.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.143+0000 7f59ac068f40 1 -- 172.21.15.38:0/4230160286 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:83 13.d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x55746ff523f0 con 0x55746ff4a240 2024-09-21T03:42:22.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.144+0000 7f59a5323640 1 -- 172.21.15.38:0/4230160286 <== osd.0 v2:172.21.15.38:6800/2381639433 16 ==== osd_op_reply(83 mds_snaptable [write 0~70] v115'94 uv94 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f599c09a050 con 0x55746ff4a240 2024-09-21T03:42:22.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.145+0000 7f59ac068f40 1 -- 172.21.15.38:0/4230160286 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:84 13.d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x55746ff523f0 con 0x55746ff4a240 2024-09-21T03:42:22.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.146+0000 7f59a5323640 1 -- 172.21.15.38:0/4230160286 <== osd.0 v2:172.21.15.38:6800/2381639433 17 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v115'95 uv95 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f599c09a050 con 0x55746ff4a240 2024-09-21T03:42:22.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.146+0000 7f59ac068f40 1 --2- 172.21.15.38:0/4230160286 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x55746ff52d40 0x55746ff55160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:22.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.146+0000 7f59ac068f40 1 -- 172.21.15.38:0/4230160286 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x55746ff556a0 con 0x55746ff52d40 2024-09-21T03:42:22.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.146+0000 7f59a5b24640 1 --2- 172.21.15.38:0/4230160286 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x55746ff52d40 0x55746ff55160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:22.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.146+0000 7f59a5b24640 1 --2- 172.21.15.38:0/4230160286 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x55746ff52d40 0x55746ff55160 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:22.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.148+0000 7f59a5b24640 1 -- 172.21.15.38:0/4230160286 <== osd.1 v2:172.21.15.38:6816/1491818732 1 ==== osd_op_reply(85 605.00000000 [create] v115'111 uv110 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f599800a6c0 con 0x55746ff52d40 2024-09-21T03:42:22.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.148+0000 7f59ac068f40 1 -- 172.21.15.38:0/4230160286 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e115) -- 0x55746ff55720 con 0x55746ff52d40 2024-09-21T03:42:22.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.149+0000 7f59a5b24640 1 -- 172.21.15.38:0/4230160286 <== osd.1 v2:172.21.15.38:6816/1491818732 2 ==== osd_op_reply(86 605.00000000 [delete] v115'112 uv112 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f599800a6c0 con 0x55746ff52d40 2024-09-21T03:42:22.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.149+0000 7f59ac068f40 1 -- 172.21.15.38:0/4230160286 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x55746ff55b40 con 0x55746ff52d40 2024-09-21T03:42:22.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.150+0000 7f59a5b24640 1 -- 172.21.15.38:0/4230160286 <== osd.1 v2:172.21.15.38:6816/1491818732 3 ==== osd_op_reply(87 605.00000000 [create] v115'113 uv113 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f599800a6c0 con 0x55746ff52d40 2024-09-21T03:42:22.152 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-09-21T03:42:22.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.150+0000 7f59ac068f40 1 -- 172.21.15.38:0/4230160286 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x55746ff56b50 con 0x55746ff52d40 2024-09-21T03:42:22.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.151+0000 7f59a5b24640 1 -- 172.21.15.38:0/4230160286 <== osd.1 v2:172.21.15.38:6816/1491818732 4 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v115'114 uv114 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f599800a6c0 con 0x55746ff52d40 2024-09-21T03:42:22.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.151+0000 7f59ac068f40 1 -- 172.21.15.38:0/4230160286 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x55746ff56b50 con 0x55746ff52d40 2024-09-21T03:42:22.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.152+0000 7f59a5b24640 1 -- 172.21.15.38:0/4230160286 <== osd.1 v2:172.21.15.38:6816/1491818732 5 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v115'115 uv115 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f599800a6c0 con 0x55746ff52d40 2024-09-21T03:42:22.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.152+0000 7f59ac068f40 1 -- 172.21.15.38:0/4230160286 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x55746ff55b40 con 0x55746ff52d40 2024-09-21T03:42:22.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.153+0000 7f59a5b24640 1 -- 172.21.15.38:0/4230160286 <== osd.1 v2:172.21.15.38:6816/1491818732 6 ==== osd_op_reply(90 605.00000000 [omap-set-header] v115'116 uv116 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f599800a6c0 con 0x55746ff52d40 2024-09-21T03:42:22.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.153+0000 7f59ac068f40 1 -- 172.21.15.38:0/4230160286 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x55746ff55b40 con 0x55746ff52d40 2024-09-21T03:42:22.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.154+0000 7f59a5b24640 1 -- 172.21.15.38:0/4230160286 <== osd.1 v2:172.21.15.38:6816/1491818732 7 ==== osd_op_reply(91 mds0_inotable [create] v115'182 uv176 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f599800a6c0 con 0x55746ff52d40 2024-09-21T03:42:22.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.154+0000 7f59ac068f40 1 -- 172.21.15.38:0/4230160286 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e115) -- 0x55746ff55b40 con 0x55746ff52d40 2024-09-21T03:42:22.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.155+0000 7f59a5b24640 1 -- 172.21.15.38:0/4230160286 <== osd.1 v2:172.21.15.38:6816/1491818732 8 ==== osd_op_reply(92 mds0_inotable [delete] v115'183 uv183 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f599800a6c0 con 0x55746ff52d40 2024-09-21T03:42:22.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.155+0000 7f59ac068f40 1 -- 172.21.15.38:0/4230160286 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x55746ff55b40 con 0x55746ff52d40 2024-09-21T03:42:22.158 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-09-21T03:42:22.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.157+0000 7f59a5b24640 1 -- 172.21.15.38:0/4230160286 <== osd.1 v2:172.21.15.38:6816/1491818732 9 ==== osd_op_reply(93 mds0_inotable [create] v115'184 uv184 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f599800a6c0 con 0x55746ff52d40 2024-09-21T03:42:22.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.157+0000 7f59ac068f40 1 -- 172.21.15.38:0/4230160286 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:94 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x55746ff55b40 con 0x55746ff52d40 2024-09-21T03:42:22.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.158+0000 7f59a5b24640 1 -- 172.21.15.38:0/4230160286 <== osd.1 v2:172.21.15.38:6816/1491818732 10 ==== osd_op_reply(94 mds0_inotable [write 0~34] v115'185 uv185 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f599800a6c0 con 0x55746ff52d40 2024-09-21T03:42:22.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.158+0000 7f59ac068f40 1 -- 172.21.15.38:0/4230160286 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:95 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x55746ff55b40 con 0x55746ff52d40 2024-09-21T03:42:22.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.159+0000 7f59a5b24640 1 -- 172.21.15.38:0/4230160286 <== osd.1 v2:172.21.15.38:6816/1491818732 11 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v115'186 uv186 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f599800a6c0 con 0x55746ff52d40 2024-09-21T03:42:22.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.159+0000 7f59ac068f40 1 -- 172.21.15.38:0/4230160286 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:96 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x55746ff55b40 con 0x55746ff52d40 2024-09-21T03:42:22.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.160+0000 7f59a5b24640 1 -- 172.21.15.38:0/4230160286 <== osd.1 v2:172.21.15.38:6816/1491818732 12 ==== osd_op_reply(96 100.00000000 [create] v115'187 uv181 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f599800a6c0 con 0x55746ff52d40 2024-09-21T03:42:22.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.160+0000 7f59ac068f40 1 -- 172.21.15.38:0/4230160286 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:97 13.13 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e115) -- 0x55746ff55b40 con 0x55746ff52d40 2024-09-21T03:42:22.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.161+0000 7f59a5b24640 1 -- 172.21.15.38:0/4230160286 <== osd.1 v2:172.21.15.38:6816/1491818732 13 ==== osd_op_reply(97 100.00000000 [delete] v115'188 uv188 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f599800a6c0 con 0x55746ff52d40 2024-09-21T03:42:22.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.161+0000 7f59ac068f40 1 -- 172.21.15.38:0/4230160286 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:98 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x55746ff55b40 con 0x55746ff52d40 2024-09-21T03:42:22.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.162+0000 7f59a5b24640 1 -- 172.21.15.38:0/4230160286 <== osd.1 v2:172.21.15.38:6816/1491818732 14 ==== osd_op_reply(98 100.00000000 [create] v115'189 uv189 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f599800a6c0 con 0x55746ff52d40 2024-09-21T03:42:22.163 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-09-21T03:42:22.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.162+0000 7f59ac068f40 1 -- 172.21.15.38:0/4230160286 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:99 13.13 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x55746ff55b40 con 0x55746ff52d40 2024-09-21T03:42:22.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.163+0000 7f59a5b24640 1 -- 172.21.15.38:0/4230160286 <== osd.1 v2:172.21.15.38:6816/1491818732 15 ==== osd_op_reply(99 100.00000000 [omap-set-header] v115'190 uv190 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f599800a6c0 con 0x55746ff52d40 2024-09-21T03:42:22.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.163+0000 7f59ac068f40 1 -- 172.21.15.38:0/4230160286 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:100 13.13 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x55746ff58950 con 0x55746ff52d40 2024-09-21T03:42:22.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.164+0000 7f59a5b24640 1 -- 172.21.15.38:0/4230160286 <== osd.1 v2:172.21.15.38:6816/1491818732 16 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v115'191 uv191 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f599800a6c0 con 0x55746ff52d40 2024-09-21T03:42:22.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.164+0000 7f59ac068f40 1 -- 172.21.15.38:0/4230160286 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x55746ff55ff0 con 0x55746ff4e860 2024-09-21T03:42:22.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.165+0000 7f59a6325640 1 -- 172.21.15.38:0/4230160286 <== osd.3 v2:172.21.15.38:6808/1286810294 16 ==== osd_op_reply(101 mds0_sessionmap [create] v115'83 uv81 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7f59a000b040 con 0x55746ff4e860 2024-09-21T03:42:22.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.165+0000 7f59ac068f40 1 -- 172.21.15.38:0/4230160286 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e115) -- 0x55746ff55ff0 con 0x55746ff4e860 2024-09-21T03:42:22.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.166+0000 7f59a6325640 1 -- 172.21.15.38:0/4230160286 <== osd.3 v2:172.21.15.38:6808/1286810294 17 ==== osd_op_reply(102 mds0_sessionmap [delete] v115'84 uv84 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f59a000b040 con 0x55746ff4e860 2024-09-21T03:42:22.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.166+0000 7f59ac068f40 1 -- 172.21.15.38:0/4230160286 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:103 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x55746ff55ff0 con 0x55746ff4e860 2024-09-21T03:42:22.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.167+0000 7f59a6325640 1 -- 172.21.15.38:0/4230160286 <== osd.3 v2:172.21.15.38:6808/1286810294 18 ==== osd_op_reply(103 mds0_sessionmap [create] v115'85 uv85 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f59a000b040 con 0x55746ff4e860 2024-09-21T03:42:22.169 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-09-21T03:42:22.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.167+0000 7f59ac068f40 1 -- 172.21.15.38:0/4230160286 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:104 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x55746ff55ff0 con 0x55746ff4e860 2024-09-21T03:42:22.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.168+0000 7f59a6325640 1 -- 172.21.15.38:0/4230160286 <== osd.3 v2:172.21.15.38:6808/1286810294 19 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v115'86 uv86 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f59a000b040 con 0x55746ff4e860 2024-09-21T03:42:22.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.168+0000 7f59ac068f40 1 -- 172.21.15.38:0/4230160286 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x55746ff55ff0 con 0x55746ff4e860 2024-09-21T03:42:22.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.169+0000 7f59a6325640 1 -- 172.21.15.38:0/4230160286 <== osd.3 v2:172.21.15.38:6808/1286810294 20 ==== osd_op_reply(105 609.00000000 [create] v115'111 uv110 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f59a000b040 con 0x55746ff4e860 2024-09-21T03:42:22.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.169+0000 7f59ac068f40 1 -- 172.21.15.38:0/4230160286 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e115) -- 0x55746ff55ff0 con 0x55746ff4e860 2024-09-21T03:42:22.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.170+0000 7f59a6325640 1 -- 172.21.15.38:0/4230160286 <== osd.3 v2:172.21.15.38:6808/1286810294 21 ==== osd_op_reply(106 609.00000000 [delete] v115'112 uv112 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f59a000b040 con 0x55746ff4e860 2024-09-21T03:42:22.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.170+0000 7f59ac068f40 1 -- 172.21.15.38:0/4230160286 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x55746ff55ff0 con 0x55746ff4e860 2024-09-21T03:42:22.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.171+0000 7f59a6325640 1 -- 172.21.15.38:0/4230160286 <== osd.3 v2:172.21.15.38:6808/1286810294 22 ==== osd_op_reply(107 609.00000000 [create] v115'113 uv113 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f59a000b040 con 0x55746ff4e860 2024-09-21T03:42:22.173 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-09-21T03:42:22.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.171+0000 7f59ac068f40 1 -- 172.21.15.38:0/4230160286 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x55746ff57000 con 0x55746ff4e860 2024-09-21T03:42:22.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.172+0000 7f59a6325640 1 -- 172.21.15.38:0/4230160286 <== osd.3 v2:172.21.15.38:6808/1286810294 23 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v115'114 uv114 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f59a000b040 con 0x55746ff4e860 2024-09-21T03:42:22.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.172+0000 7f59ac068f40 1 -- 172.21.15.38:0/4230160286 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:109 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x55746ff57000 con 0x55746ff4e860 2024-09-21T03:42:22.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.173+0000 7f59a6325640 1 -- 172.21.15.38:0/4230160286 <== osd.3 v2:172.21.15.38:6808/1286810294 24 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v115'115 uv115 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f59a000b040 con 0x55746ff4e860 2024-09-21T03:42:22.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.173+0000 7f59ac068f40 1 -- 172.21.15.38:0/4230160286 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:110 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x55746ff55ff0 con 0x55746ff4e860 2024-09-21T03:42:22.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.174+0000 7f59a6325640 1 -- 172.21.15.38:0/4230160286 <== osd.3 v2:172.21.15.38:6808/1286810294 25 ==== osd_op_reply(110 609.00000000 [omap-set-header] v115'116 uv116 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f59a000b040 con 0x55746ff4e860 2024-09-21T03:42:22.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.174+0000 7f59ac068f40 1 --2- 172.21.15.38:0/4230160286 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x55746ff56500 0x55746ff56910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:22.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.174+0000 7f59ac068f40 1 -- 172.21.15.38:0/4230160286 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x55746ff56e50 con 0x55746ff56500 2024-09-21T03:42:22.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.174+0000 7f59a5323640 1 --2- 172.21.15.38:0/4230160286 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x55746ff56500 0x55746ff56910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:22.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.175+0000 7f59a5323640 1 --2- 172.21.15.38:0/4230160286 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x55746ff56500 0x55746ff56910 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:22.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.176+0000 7f59a5323640 1 -- 172.21.15.38:0/4230160286 <== osd.2 v2:172.21.15.38:6824/2577858902 1 ==== osd_op_reply(111 400.00000000 [create] v115'94 uv93 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f599c09a050 con 0x55746ff56500 2024-09-21T03:42:22.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.176+0000 7f59ac068f40 1 -- 172.21.15.38:0/4230160286 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e115) -- 0x55746ff56e50 con 0x55746ff56500 2024-09-21T03:42:22.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.177+0000 7f59a5323640 1 -- 172.21.15.38:0/4230160286 <== osd.2 v2:172.21.15.38:6824/2577858902 2 ==== osd_op_reply(112 400.00000000 [delete] v115'95 uv95 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f599c09a050 con 0x55746ff56500 2024-09-21T03:42:22.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.177+0000 7f59ac068f40 1 -- 172.21.15.38:0/4230160286 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x55746ff56e50 con 0x55746ff56500 2024-09-21T03:42:22.180 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-09-21T03:42:22.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.178+0000 7f59a5323640 1 -- 172.21.15.38:0/4230160286 <== osd.2 v2:172.21.15.38:6824/2577858902 3 ==== osd_op_reply(113 400.00000000 [create] v115'96 uv96 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f599c09a050 con 0x55746ff56500 2024-09-21T03:42:22.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.178+0000 7f59ac068f40 1 -- 172.21.15.38:0/4230160286 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:114 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x55746ff56e50 con 0x55746ff56500 2024-09-21T03:42:22.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.179+0000 7f59a5323640 1 -- 172.21.15.38:0/4230160286 <== osd.2 v2:172.21.15.38:6824/2577858902 4 ==== osd_op_reply(114 400.00000000 [write 0~22] v115'97 uv97 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f599c09a050 con 0x55746ff56500 2024-09-21T03:42:22.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.179+0000 7f59ac068f40 1 -- 172.21.15.38:0/4230160286 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:115 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x55746ff56e50 con 0x55746ff56500 2024-09-21T03:42:22.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.180+0000 7f59a5323640 1 -- 172.21.15.38:0/4230160286 <== osd.2 v2:172.21.15.38:6824/2577858902 5 ==== osd_op_reply(115 400.00000000 [omap-set-header] v115'98 uv98 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f599c09a050 con 0x55746ff56500 2024-09-21T03:42:22.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.180+0000 7f59ac068f40 1 -- 172.21.15.38:0/4230160286 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:116 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x55746ff56e50 con 0x55746ff52d40 2024-09-21T03:42:22.183 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.182+0000 7f59a5b24640 1 -- 172.21.15.38:0/4230160286 <== osd.1 v2:172.21.15.38:6816/1491818732 17 ==== osd_op_reply(116 100.00000000.inode [create] v115'229 uv220 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7f599800a6c0 con 0x55746ff52d40 2024-09-21T03:42:22.183 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.182+0000 7f59ac068f40 1 -- 172.21.15.38:0/4230160286 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:117 13.1f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e115) -- 0x55746ff56e50 con 0x55746ff52d40 2024-09-21T03:42:22.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.183+0000 7f59a5b24640 1 -- 172.21.15.38:0/4230160286 <== osd.1 v2:172.21.15.38:6816/1491818732 18 ==== osd_op_reply(117 100.00000000.inode [delete] v115'230 uv230 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f599800a6c0 con 0x55746ff52d40 2024-09-21T03:42:22.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.183+0000 7f59ac068f40 1 -- 172.21.15.38:0/4230160286 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:118 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x55746ff56e50 con 0x55746ff52d40 2024-09-21T03:42:22.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.184+0000 7f59a5b24640 1 -- 172.21.15.38:0/4230160286 <== osd.1 v2:172.21.15.38:6816/1491818732 19 ==== osd_op_reply(118 100.00000000.inode [create] v115'231 uv231 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f599800a6c0 con 0x55746ff52d40 2024-09-21T03:42:22.186 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-09-21T03:42:22.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.184+0000 7f59ac068f40 1 -- 172.21.15.38:0/4230160286 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:119 13.1f 13:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x55746ff57430 con 0x55746ff52d40 2024-09-21T03:42:22.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.185+0000 7f59a5b24640 1 -- 172.21.15.38:0/4230160286 <== osd.1 v2:172.21.15.38:6816/1491818732 20 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v115'232 uv232 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f599800a6c0 con 0x55746ff52d40 2024-09-21T03:42:22.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.185+0000 7f59ac068f40 1 -- 172.21.15.38:0/4230160286 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:120 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x55746ff57430 con 0x55746ff52d40 2024-09-21T03:42:22.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.186+0000 7f59a5b24640 1 -- 172.21.15.38:0/4230160286 <== osd.1 v2:172.21.15.38:6816/1491818732 21 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v115'233 uv233 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f599800a6c0 con 0x55746ff52d40 2024-09-21T03:42:22.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.186+0000 7f59ac068f40 1 -- 172.21.15.38:0/4230160286 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:121 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x55746ff573e0 con 0x55746ff52d40 2024-09-21T03:42:22.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.187+0000 7f59a5b24640 1 -- 172.21.15.38:0/4230160286 <== osd.1 v2:172.21.15.38:6816/1491818732 22 ==== osd_op_reply(121 1.00000000 [create] v115'234 uv227 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7f599800a6c0 con 0x55746ff52d40 2024-09-21T03:42:22.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.187+0000 7f59ac068f40 1 -- 172.21.15.38:0/4230160286 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:122 13.1f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e115) -- 0x55746ff573e0 con 0x55746ff52d40 2024-09-21T03:42:22.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.188+0000 7f59a5b24640 1 -- 172.21.15.38:0/4230160286 <== osd.1 v2:172.21.15.38:6816/1491818732 23 ==== osd_op_reply(122 1.00000000 [delete] v115'235 uv235 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f599800a6c0 con 0x55746ff52d40 2024-09-21T03:42:22.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.188+0000 7f59ac068f40 1 -- 172.21.15.38:0/4230160286 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:123 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x55746ff56e50 con 0x55746ff52d40 2024-09-21T03:42:22.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.190+0000 7f59a5b24640 1 -- 172.21.15.38:0/4230160286 <== osd.1 v2:172.21.15.38:6816/1491818732 24 ==== osd_op_reply(123 1.00000000 [create] v115'236 uv236 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f599800a6c0 con 0x55746ff52d40 2024-09-21T03:42:22.191 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-09-21T03:42:22.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.190+0000 7f59ac068f40 1 -- 172.21.15.38:0/4230160286 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:124 13.1f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x55746ff573e0 con 0x55746ff52d40 2024-09-21T03:42:22.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.191+0000 7f59a5b24640 1 -- 172.21.15.38:0/4230160286 <== osd.1 v2:172.21.15.38:6816/1491818732 25 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v115'237 uv237 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f599800a6c0 con 0x55746ff52d40 2024-09-21T03:42:22.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.191+0000 7f59ac068f40 1 -- 172.21.15.38:0/4230160286 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:125 13.1f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x55746ff583f0 con 0x55746ff52d40 2024-09-21T03:42:22.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.192+0000 7f59a5b24640 1 -- 172.21.15.38:0/4230160286 <== osd.1 v2:172.21.15.38:6816/1491818732 26 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v115'238 uv238 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f599800a6c0 con 0x55746ff52d40 2024-09-21T03:42:22.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.192+0000 7f59ac068f40 1 -- 172.21.15.38:0/4230160286 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:126 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x55746ff58800 con 0x55746ff52d40 2024-09-21T03:42:22.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.193+0000 7f59a5b24640 1 -- 172.21.15.38:0/4230160286 <== osd.1 v2:172.21.15.38:6816/1491818732 27 ==== osd_op_reply(126 1.00000000 [omap-set-header] v115'239 uv239 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f599800a6c0 con 0x55746ff52d40 2024-09-21T03:42:22.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.193+0000 7f59ac068f40 1 -- 172.21.15.38:0/4230160286 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:127 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=584b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x55746ff58800 con 0x55746ff52d40 2024-09-21T03:42:22.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.194+0000 7f59a5b24640 1 -- 172.21.15.38:0/4230160286 <== osd.1 v2:172.21.15.38:6816/1491818732 28 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v115'240 uv240 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f599800a6c0 con 0x55746ff52d40 2024-09-21T03:42:22.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.194+0000 7f59ac068f40 1 -- 172.21.15.38:0/4230160286 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x55746ff4a240 msgr2=0x55746ff4c660 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:22.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.194+0000 7f59ac068f40 1 --2- 172.21.15.38:0/4230160286 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x55746ff4a240 0x55746ff4c660 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:22.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.194+0000 7f59ac068f40 1 -- 172.21.15.38:0/4230160286 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x55746ff52d40 msgr2=0x55746ff55160 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:22.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.194+0000 7f59ac068f40 1 --2- 172.21.15.38:0/4230160286 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x55746ff52d40 0x55746ff55160 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:22.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.194+0000 7f59ac068f40 1 -- 172.21.15.38:0/4230160286 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x55746ff56500 msgr2=0x55746ff56910 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:22.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.194+0000 7f59ac068f40 1 --2- 172.21.15.38:0/4230160286 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x55746ff56500 0x55746ff56910 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:22.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.194+0000 7f59ac068f40 1 -- 172.21.15.38:0/4230160286 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x55746ff4e860 msgr2=0x55746ff512d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:22.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.194+0000 7f59ac068f40 1 --2- 172.21.15.38:0/4230160286 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x55746ff4e860 0x55746ff512d0 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:22.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.194+0000 7f59a6325640 1 -- 172.21.15.38:0/4230160286 reap_dead start 2024-09-21T03:42:22.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.194+0000 7f59ac068f40 1 -- 172.21.15.38:0/4230160286 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x55746ff45960 msgr2=0x55746ff47d80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:22.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.194+0000 7f59ac068f40 1 --2- 172.21.15.38:0/4230160286 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x55746ff45960 0x55746ff47d80 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:22.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.194+0000 7f59a6325640 1 -- 172.21.15.38:0/4230160286 reap_dead start 2024-09-21T03:42:22.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.194+0000 7f59ac068f40 1 -- 172.21.15.38:0/4230160286 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x55746fe9c6d0 msgr2=0x55746fe9f520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:22.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.194+0000 7f59ac068f40 1 --2- 172.21.15.38:0/4230160286 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x55746fe9c6d0 0x55746fe9f520 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:22.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.194+0000 7f59ac068f40 1 -- 172.21.15.38:0/4230160286 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5988077570 msgr2=0x7f5988079a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:22.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.194+0000 7f59ac068f40 1 --2- 172.21.15.38:0/4230160286 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5988077570 0x7f5988079a80 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f599800c640 tx=0x7f5998004bb0 comp rx=0 tx=0).stop 2024-09-21T03:42:22.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.194+0000 7f59ac068f40 1 -- 172.21.15.38:0/4230160286 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55746fe88a90 msgr2=0x55746ff3af80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:22.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.194+0000 7f59ac068f40 1 --2- 172.21.15.38:0/4230160286 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55746fe88a90 0x55746ff3af80 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7f599c003e70 tx=0x7f599c00b590 comp rx=0 tx=0).stop 2024-09-21T03:42:22.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.194+0000 7f59ac068f40 1 -- 172.21.15.38:0/4230160286 shutdown_connections 2024-09-21T03:42:22.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.194+0000 7f59ac068f40 1 --2- 172.21.15.38:0/4230160286 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x55746fe9c6d0 0x55746fe9f520 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:22.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.194+0000 7f59ac068f40 1 --2- 172.21.15.38:0/4230160286 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5988077570 0x7f5988079a80 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:22.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.194+0000 7f59ac068f40 1 --2- 172.21.15.38:0/4230160286 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55746fe88a90 0x55746ff3af80 unknown :-1 s=CLOSED pgs=1705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:22.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.194+0000 7f59ac068f40 1 -- 172.21.15.38:0/4230160286 >> 172.21.15.38:0/4230160286 conn(0x55746fcebac0 msgr2=0x55746fe88070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:22.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.194+0000 7f59ac068f40 1 -- 172.21.15.38:0/4230160286 shutdown_connections 2024-09-21T03:42:22.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.194+0000 7f59ac068f40 1 -- 172.21.15.38:0/4230160286 wait complete. 2024-09-21T03:42:22.200 DEBUG:teuthology.orchestra.run.smithi038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata put mds_snaptable - 2024-09-21T03:42:22.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.254+0000 7f589161ff40 1 Processor -- start 2024-09-21T03:42:22.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.254+0000 7f589161ff40 1 -- start start 2024-09-21T03:42:22.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.255+0000 7f589161ff40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55dbf7462240 0x55dbf75069c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:22.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.255+0000 7f589161ff40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55dbf74a9090 0x55dbf751abf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:22.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.255+0000 7f589161ff40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dbf751e520 0x55dbf75365d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:22.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.255+0000 7f589161ff40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55dbf73afd60 con 0x55dbf751e520 2024-09-21T03:42:22.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.255+0000 7f589161ff40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55dbf740e430 con 0x55dbf74a9090 2024-09-21T03:42:22.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.255+0000 7f589161ff40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55dbf7535aa0 con 0x55dbf7462240 2024-09-21T03:42:22.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.255+0000 7f588b8da640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dbf751e520 0x55dbf75365d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:22.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.255+0000 7f588b8da640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dbf751e520 0x55dbf75365d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34496/0 (socket says 172.21.15.38:34496) 2024-09-21T03:42:22.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.255+0000 7f588b0d9640 1 --2- 172.21.15.38:0/1397967734 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55dbf7462240 0x55dbf75069c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:22.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.255+0000 7f588b8da640 1 -- 172.21.15.38:0/1397967734 learned_addr learned my addr 172.21.15.38:0/1397967734 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:42:22.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.256+0000 7f588a8d8640 1 --2- 172.21.15.38:0/1397967734 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55dbf74a9090 0x55dbf751abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:22.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.256+0000 7f588b8da640 1 -- 172.21.15.38:0/1397967734 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55dbf7462240 msgr2=0x55dbf75069c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:22.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.256+0000 7f588b8da640 1 --2- 172.21.15.38:0/1397967734 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55dbf7462240 0x55dbf75069c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:22.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.256+0000 7f588b8da640 1 -- 172.21.15.38:0/1397967734 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55dbf74a9090 msgr2=0x55dbf751abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:22.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.256+0000 7f588b8da640 1 --2- 172.21.15.38:0/1397967734 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55dbf74a9090 0x55dbf751abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:22.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.256+0000 7f588b8da640 1 -- 172.21.15.38:0/1397967734 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dbf7509a00 con 0x55dbf751e520 2024-09-21T03:42:22.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.256+0000 7f588b8da640 1 --2- 172.21.15.38:0/1397967734 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dbf751e520 0x55dbf75365d0 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7f5884001540 tx=0x7f5884001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec786518768c8747 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:22.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.256+0000 7f58898d6640 1 -- 172.21.15.38:0/1397967734 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5884016070 con 0x55dbf751e520 2024-09-21T03:42:22.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.256+0000 7f58898d6640 1 -- 172.21.15.38:0/1397967734 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f588400f800 con 0x55dbf751e520 2024-09-21T03:42:22.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.256+0000 7f58898d6640 1 -- 172.21.15.38:0/1397967734 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f588400fc10 con 0x55dbf751e520 2024-09-21T03:42:22.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.257+0000 7f589161ff40 1 -- 172.21.15.38:0/1397967734 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dbf751e520 msgr2=0x55dbf75365d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:22.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.257+0000 7f589161ff40 1 --2- 172.21.15.38:0/1397967734 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dbf751e520 0x55dbf75365d0 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7f5884001540 tx=0x7f5884001a00 comp rx=0 tx=0).stop 2024-09-21T03:42:22.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.257+0000 7f589161ff40 1 -- 172.21.15.38:0/1397967734 shutdown_connections 2024-09-21T03:42:22.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.257+0000 7f589161ff40 1 --2- 172.21.15.38:0/1397967734 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dbf751e520 0x55dbf75365d0 unknown :-1 s=CLOSED pgs=1707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:22.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.257+0000 7f589161ff40 1 --2- 172.21.15.38:0/1397967734 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55dbf74a9090 0x55dbf751abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:22.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.257+0000 7f589161ff40 1 --2- 172.21.15.38:0/1397967734 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55dbf7462240 0x55dbf75069c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:22.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.257+0000 7f589161ff40 1 -- 172.21.15.38:0/1397967734 >> 172.21.15.38:0/1397967734 conn(0x55dbf7380ac0 msgr2=0x55dbf7460cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:22.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.257+0000 7f589161ff40 1 -- 172.21.15.38:0/1397967734 shutdown_connections 2024-09-21T03:42:22.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.257+0000 7f589161ff40 1 -- 172.21.15.38:0/1397967734 wait complete. 2024-09-21T03:42:22.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.258+0000 7f589161ff40 1 Processor -- start 2024-09-21T03:42:22.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.258+0000 7f589161ff40 1 -- start start 2024-09-21T03:42:22.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.259+0000 7f589161ff40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55dbf7462240 0x55dbf748b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:22.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.259+0000 7f589161ff40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55dbf74a9090 0x55dbf748b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:22.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.259+0000 7f589161ff40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dbf751e520 0x55dbf7521440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:22.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.259+0000 7f589161ff40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55dbf7509570 con 0x55dbf751e520 2024-09-21T03:42:22.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.259+0000 7f589161ff40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55dbf7461740 con 0x55dbf74a9090 2024-09-21T03:42:22.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.259+0000 7f589161ff40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55dbf7522780 con 0x55dbf7462240 2024-09-21T03:42:22.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.259+0000 7f588b8da640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dbf751e520 0x55dbf7521440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:22.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.259+0000 7f588b8da640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dbf751e520 0x55dbf7521440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34510/0 (socket says 172.21.15.38:34510) 2024-09-21T03:42:22.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.259+0000 7f588b8da640 1 -- 172.21.15.38:0/377503801 learned_addr learned my addr 172.21.15.38:0/377503801 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:42:22.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.259+0000 7f588b0d9640 1 --2- 172.21.15.38:0/377503801 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55dbf7462240 0x55dbf748b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:22.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.259+0000 7f588a8d8640 1 --2- 172.21.15.38:0/377503801 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55dbf74a9090 0x55dbf748b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:22.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.260+0000 7f588b8da640 1 -- 172.21.15.38:0/377503801 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55dbf7462240 msgr2=0x55dbf748b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:22.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.260+0000 7f588b8da640 1 --2- 172.21.15.38:0/377503801 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55dbf7462240 0x55dbf748b190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:22.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.260+0000 7f588b8da640 1 -- 172.21.15.38:0/377503801 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55dbf74a9090 msgr2=0x55dbf748b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:22.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.260+0000 7f588b8da640 1 --2- 172.21.15.38:0/377503801 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55dbf74a9090 0x55dbf748b6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:22.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.260+0000 7f588b8da640 1 -- 172.21.15.38:0/377503801 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dbf7523510 con 0x55dbf751e520 2024-09-21T03:42:22.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.260+0000 7f588b8da640 1 --2- 172.21.15.38:0/377503801 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dbf751e520 0x55dbf7521440 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7f58840064a0 tx=0x7f588400fab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:22.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.260+0000 7f58888d4640 1 -- 172.21.15.38:0/377503801 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5884016030 con 0x55dbf751e520 2024-09-21T03:42:22.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.260+0000 7f58888d4640 1 -- 172.21.15.38:0/377503801 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5884011440 con 0x55dbf751e520 2024-09-21T03:42:22.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.260+0000 7f589161ff40 1 -- 172.21.15.38:0/377503801 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55dbf75237f0 con 0x55dbf751e520 2024-09-21T03:42:22.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.260+0000 7f58888d4640 1 -- 172.21.15.38:0/377503801 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5884011870 con 0x55dbf751e520 2024-09-21T03:42:22.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.260+0000 7f589161ff40 1 -- 172.21.15.38:0/377503801 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dbf7520f20 con 0x55dbf751e520 2024-09-21T03:42:22.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.262+0000 7f58888d4640 1 -- 172.21.15.38:0/377503801 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f5884011a10 con 0x55dbf751e520 2024-09-21T03:42:22.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.262+0000 7f58888d4640 1 --2- 172.21.15.38:0/377503801 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5878077570 0x7f5878079a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:22.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.262+0000 7f58888d4640 1 -- 172.21.15.38:0/377503801 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 7938+0+0 (secure 0 0 0) 0x7f588409b890 con 0x55dbf751e520 2024-09-21T03:42:22.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.262+0000 7f588b0d9640 1 --2- 172.21.15.38:0/377503801 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5878077570 0x7f5878079a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:22.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.262+0000 7f589161ff40 1 --2- 172.21.15.38:0/377503801 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x55dbf7521e90 0x55dbf75cbb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:22.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.262+0000 7f589161ff40 1 -- 172.21.15.38:0/377503801 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:1 13.d 13:b50e409b:::mds_snaptable:head [writefull 0~80 in=80b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x55dbf75cc080 con 0x55dbf7521e90 2024-09-21T03:42:22.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.263+0000 7f588a8d8640 1 --2- 172.21.15.38:0/377503801 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x55dbf7521e90 0x55dbf75cbb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:22.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.263+0000 7f588b0d9640 1 --2- 172.21.15.38:0/377503801 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5878077570 0x7f5878079a80 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f587c00c700 tx=0x7f587c00e040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:22.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.263+0000 7f588a8d8640 1 --2- 172.21.15.38:0/377503801 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x55dbf7521e90 0x55dbf75cbb40 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:22.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.264+0000 7f588a8d8640 1 -- 172.21.15.38:0/377503801 <== osd.0 v2:172.21.15.38:6800/2381639433 1 ==== osd_op_reply(1 mds_snaptable [writefull 0~80] v115'96 uv96 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f588004ec80 con 0x55dbf7521e90 2024-09-21T03:42:22.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.264+0000 7f589161ff40 1 -- 172.21.15.38:0/377503801 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x55dbf7521e90 msgr2=0x55dbf75cbb40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:22.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.264+0000 7f589161ff40 1 --2- 172.21.15.38:0/377503801 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x55dbf7521e90 0x55dbf75cbb40 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:22.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.264+0000 7f589161ff40 1 -- 172.21.15.38:0/377503801 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5878077570 msgr2=0x7f5878079a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:22.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.264+0000 7f589161ff40 1 --2- 172.21.15.38:0/377503801 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5878077570 0x7f5878079a80 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f587c00c700 tx=0x7f587c00e040 comp rx=0 tx=0).stop 2024-09-21T03:42:22.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.264+0000 7f589161ff40 1 -- 172.21.15.38:0/377503801 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dbf751e520 msgr2=0x55dbf7521440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:22.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.264+0000 7f589161ff40 1 --2- 172.21.15.38:0/377503801 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dbf751e520 0x55dbf7521440 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7f58840064a0 tx=0x7f588400fab0 comp rx=0 tx=0).stop 2024-09-21T03:42:22.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.264+0000 7f588b8da640 1 -- 172.21.15.38:0/377503801 reap_dead start 2024-09-21T03:42:22.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.265+0000 7f589161ff40 1 -- 172.21.15.38:0/377503801 shutdown_connections 2024-09-21T03:42:22.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.265+0000 7f589161ff40 1 -- 172.21.15.38:0/377503801 >> 172.21.15.38:0/377503801 conn(0x55dbf7380ac0 msgr2=0x55dbf7460cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:22.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.265+0000 7f589161ff40 1 -- 172.21.15.38:0/377503801 shutdown_connections 2024-09-21T03:42:22.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.265+0000 7f589161ff40 1 -- 172.21.15.38:0/377503801 wait complete. 2024-09-21T03:42:22.269 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-21T03:42:22.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.428+0000 7f1f5fa64640 1 -- 172.21.15.38:0/2628943174 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1f580d0bb0 msgr2=0x7f1f580b5be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:22.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.428+0000 7f1f5fa64640 1 --2- 172.21.15.38:0/2628943174 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1f580d0bb0 0x7f1f580b5be0 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7f1f50005fd0 tx=0x7f1f5002aa10 comp rx=0 tx=0).stop 2024-09-21T03:42:22.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.428+0000 7f1f5fa64640 1 -- 172.21.15.38:0/2628943174 shutdown_connections 2024-09-21T03:42:22.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.428+0000 7f1f5fa64640 1 --2- 172.21.15.38:0/2628943174 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1f580d0bb0 0x7f1f580b5be0 unknown :-1 s=CLOSED pgs=1709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:22.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.428+0000 7f1f5fa64640 1 --2- 172.21.15.38:0/2628943174 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1f5815ca90 0x7f1f580d0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:22.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.428+0000 7f1f5fa64640 1 -- 172.21.15.38:0/2628943174 >> 172.21.15.38:0/2628943174 conn(0x7f1f580b46e0 msgr2=0x7f1f580d23e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:22.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.428+0000 7f1f5fa64640 1 -- 172.21.15.38:0/2628943174 shutdown_connections 2024-09-21T03:42:22.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.428+0000 7f1f5fa64640 1 -- 172.21.15.38:0/2628943174 wait complete. 2024-09-21T03:42:22.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.430+0000 7f1f5fa64640 1 Processor -- start 2024-09-21T03:42:22.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.430+0000 7f1f5fa64640 1 -- start start 2024-09-21T03:42:22.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.431+0000 7f1f5fa64640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1f580ccc40 0x7f1f5814c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:22.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.431+0000 7f1f5fa64640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1f580d0bb0 0x7f1f581f2730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:22.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.431+0000 7f1f5fa64640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1f5815ca90 0x7f1f581f2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:22.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.431+0000 7f1f5fa64640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f1f580b6620 con 0x7f1f580d0bb0 2024-09-21T03:42:22.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.431+0000 7f1f5fa64640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f1f580b64a0 con 0x7f1f580ccc40 2024-09-21T03:42:22.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.431+0000 7f1f5fa64640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f1f580b67a0 con 0x7f1f5815ca90 2024-09-21T03:42:22.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.431+0000 7f1f5cfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1f580d0bb0 0x7f1f581f2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:22.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.431+0000 7f1f5cfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1f580d0bb0 0x7f1f581f2730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:59722/0 (socket says 172.21.15.38:59722) 2024-09-21T03:42:22.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.431+0000 7f1f5cfff640 1 -- 172.21.15.38:0/1050272764 learned_addr learned my addr 172.21.15.38:0/1050272764 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:42:22.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.431+0000 7f1f5d800640 1 --2- 172.21.15.38:0/1050272764 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1f580ccc40 0x7f1f5814c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:22.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.431+0000 7f1f5e001640 1 --2- 172.21.15.38:0/1050272764 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1f5815ca90 0x7f1f581f2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:22.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.432+0000 7f1f5cfff640 1 -- 172.21.15.38:0/1050272764 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1f5815ca90 msgr2=0x7f1f581f2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:22.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.432+0000 7f1f5cfff640 1 --2- 172.21.15.38:0/1050272764 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1f5815ca90 0x7f1f581f2c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:22.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.432+0000 7f1f5cfff640 1 -- 172.21.15.38:0/1050272764 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1f580ccc40 msgr2=0x7f1f5814c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:22.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.432+0000 7f1f5cfff640 1 --2- 172.21.15.38:0/1050272764 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1f580ccc40 0x7f1f5814c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:22.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.432+0000 7f1f5cfff640 1 -- 172.21.15.38:0/1050272764 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f58146bc0 con 0x7f1f580d0bb0 2024-09-21T03:42:22.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.432+0000 7f1f5cfff640 1 --2- 172.21.15.38:0/1050272764 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1f580d0bb0 0x7f1f581f2730 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7f1f48009fd0 tx=0x7f1f4800ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:22.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.432+0000 7f1f4dffb640 1 -- 172.21.15.38:0/1050272764 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f4800cc60 con 0x7f1f580d0bb0 2024-09-21T03:42:22.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.432+0000 7f1f4dffb640 1 -- 172.21.15.38:0/1050272764 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1f4800ce00 con 0x7f1f580d0bb0 2024-09-21T03:42:22.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.432+0000 7f1f5fa64640 1 -- 172.21.15.38:0/1050272764 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f5814c980 con 0x7f1f580d0bb0 2024-09-21T03:42:22.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.432+0000 7f1f4dffb640 1 -- 172.21.15.38:0/1050272764 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f48004280 con 0x7f1f580d0bb0 2024-09-21T03:42:22.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.432+0000 7f1f5fa64640 1 -- 172.21.15.38:0/1050272764 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f581e3140 con 0x7f1f580d0bb0 2024-09-21T03:42:22.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.433+0000 7f1f5fa64640 1 -- 172.21.15.38:0/1050272764 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f58146850 con 0x7f1f580d0bb0 2024-09-21T03:42:22.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.434+0000 7f1f4dffb640 1 -- 172.21.15.38:0/1050272764 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f1f480035f0 con 0x7f1f580d0bb0 2024-09-21T03:42:22.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.435+0000 7f1f4dffb640 1 --2- 172.21.15.38:0/1050272764 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1f3c078b90 0x7f1f3c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:22.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.435+0000 7f1f4dffb640 1 -- 172.21.15.38:0/1050272764 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 7938+0+0 (secure 0 0 0) 0x7f1f4809bec0 con 0x7f1f580d0bb0 2024-09-21T03:42:22.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.435+0000 7f1f5d800640 1 --2- 172.21.15.38:0/1050272764 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1f3c078b90 0x7f1f3c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:22.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.436+0000 7f1f5d800640 1 --2- 172.21.15.38:0/1050272764 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1f3c078b90 0x7f1f3c07b0a0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f1f54000f60 tx=0x7f1f54009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:22.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.439+0000 7f1f4dffb640 1 -- 172.21.15.38:0/1050272764 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f1f480633f0 con 0x7f1f580d0bb0 2024-09-21T03:42:22.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:22.665+0000 7f1f5fa64640 1 -- 172.21.15.38:0/1050272764 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f1f580b6d30 con 0x7f1f580d0bb0 2024-09-21T03:42:23.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:23.203+0000 7f1f4dffb640 1 -- 172.21.15.38:0/1050272764 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v201) ==== 162+0+0 (secure 0 0 0) 0x7f1f48067e20 con 0x7f1f580d0bb0 2024-09-21T03:42:23.206 INFO:teuthology.orchestra.run.smithi038.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-21T03:42:23.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:23.206+0000 7f1f5fa64640 1 -- 172.21.15.38:0/1050272764 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1f3c078b90 msgr2=0x7f1f3c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:23.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:23.206+0000 7f1f5fa64640 1 --2- 172.21.15.38:0/1050272764 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1f3c078b90 0x7f1f3c07b0a0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f1f54000f60 tx=0x7f1f54009040 comp rx=0 tx=0).stop 2024-09-21T03:42:23.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:23.206+0000 7f1f5fa64640 1 -- 172.21.15.38:0/1050272764 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1f580d0bb0 msgr2=0x7f1f581f2730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:23.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:23.206+0000 7f1f5fa64640 1 --2- 172.21.15.38:0/1050272764 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1f580d0bb0 0x7f1f581f2730 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7f1f48009fd0 tx=0x7f1f4800ec20 comp rx=0 tx=0).stop 2024-09-21T03:42:23.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:23.206+0000 7f1f5fa64640 1 -- 172.21.15.38:0/1050272764 shutdown_connections 2024-09-21T03:42:23.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:23.206+0000 7f1f5fa64640 1 --2- 172.21.15.38:0/1050272764 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1f3c078b90 0x7f1f3c07b0a0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:23.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:23.206+0000 7f1f5fa64640 1 --2- 172.21.15.38:0/1050272764 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1f5815ca90 0x7f1f581f2c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:23.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:23.206+0000 7f1f5fa64640 1 --2- 172.21.15.38:0/1050272764 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1f580d0bb0 0x7f1f581f2730 unknown :-1 s=CLOSED pgs=1710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:23.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:23.206+0000 7f1f5fa64640 1 --2- 172.21.15.38:0/1050272764 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1f580ccc40 0x7f1f5814c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:23.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:23.206+0000 7f1f5fa64640 1 -- 172.21.15.38:0/1050272764 >> 172.21.15.38:0/1050272764 conn(0x7f1f580b46e0 msgr2=0x7f1f5815af90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:23.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:23.207+0000 7f1f5fa64640 1 -- 172.21.15.38:0/1050272764 shutdown_connections 2024-09-21T03:42:23.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:23.207+0000 7f1f5fa64640 1 -- 172.21.15.38:0/1050272764 wait complete. 2024-09-21T03:42:23.215 INFO:tasks.ceph.mds.a.smithi038.stderr:2024-09-21T03:42:23.214+0000 7fdf0b9d8640 -1 log_channel(cluster) log [ERR] : error decoding table object 'mds_snaptable': Decoder at 'virtual void SnapServer::decode_server_state(ceph::buffer::v15_2_0::list::const_iterator&)' v=5 cannot decode v=100 minimal_decoder=101: Malformed input [buffer:3] 2024-09-21T03:42:23.216 INFO:tasks.ceph.mon.a.smithi038.stderr:2024-09-21T03:42:23.214+0000 7f3f45713640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T03:42:23.216 INFO:tasks.ceph.mon.a.smithi038.stderr:2024-09-21T03:42:23.214+0000 7f3f45713640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-09-21T03:42:23.226 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:42:23.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:23.346+0000 7f61c0694640 1 -- 172.21.15.38:0/3338720545 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f61b80b2e40 msgr2=0x7f61b80b3250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:23.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:23.346+0000 7f61c0694640 1 --2- 172.21.15.38:0/3338720545 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f61b80b2e40 0x7f61b80b3250 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7f61b80b7460 tx=0x7f61b002a820 comp rx=0 tx=0).stop 2024-09-21T03:42:23.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:23.347+0000 7f61c0694640 1 -- 172.21.15.38:0/3338720545 shutdown_connections 2024-09-21T03:42:23.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:23.347+0000 7f61c0694640 1 --2- 172.21.15.38:0/3338720545 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f61b80b2e40 0x7f61b80b3250 unknown :-1 s=CLOSED pgs=1712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:23.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:23.347+0000 7f61c0694640 1 --2- 172.21.15.38:0/3338720545 >> v2:172.21.15.195:3300/0 conn(0x7f61b8162280 0x7f61b8162690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:23.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:23.347+0000 7f61c0694640 1 -- 172.21.15.38:0/3338720545 >> 172.21.15.38:0/3338720545 conn(0x7f61b80b6b20 msgr2=0x7f61b80d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:23.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:23.347+0000 7f61c0694640 1 -- 172.21.15.38:0/3338720545 shutdown_connections 2024-09-21T03:42:23.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:23.347+0000 7f61c0694640 1 -- 172.21.15.38:0/3338720545 wait complete. 2024-09-21T03:42:23.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:23.348+0000 7f61c0694640 1 Processor -- start 2024-09-21T03:42:23.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:23.348+0000 7f61c0694640 1 -- start start 2024-09-21T03:42:23.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:23.350+0000 7f61c0694640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f61b80b2e40 0x7f61b80aadc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:23.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:23.350+0000 7f61c0694640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f61b8162280 0x7f61b81618b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:23.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:23.350+0000 7f61c0694640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f61b814d6d0 0x7f61b814b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:23.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:23.350+0000 7f61c0694640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f61b815e4b0 con 0x7f61b80b2e40 2024-09-21T03:42:23.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:23.350+0000 7f61c0694640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f61b80b39a0 con 0x7f61b814d6d0 2024-09-21T03:42:23.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:23.350+0000 7f61c0694640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f61b815e630 con 0x7f61b8162280 2024-09-21T03:42:23.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:23.350+0000 7f61be430640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f61b80b2e40 0x7f61b80aadc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:23.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:23.350+0000 7f61be430640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f61b80b2e40 0x7f61b80aadc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:59736/0 (socket says 172.21.15.38:59736) 2024-09-21T03:42:23.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:23.350+0000 7f61be430640 1 -- 172.21.15.38:0/557043203 learned_addr learned my addr 172.21.15.38:0/557043203 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:42:23.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:23.350+0000 7f61bdc2f640 1 --2- 172.21.15.38:0/557043203 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f61b8162280 0x7f61b81618b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:23.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:23.350+0000 7f61bec31640 1 --2- 172.21.15.38:0/557043203 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f61b814d6d0 0x7f61b814b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:23.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:23.350+0000 7f61be430640 1 -- 172.21.15.38:0/557043203 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f61b8162280 msgr2=0x7f61b81618b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:23.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:23.350+0000 7f61be430640 1 --2- 172.21.15.38:0/557043203 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f61b8162280 0x7f61b81618b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:23.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:23.350+0000 7f61be430640 1 -- 172.21.15.38:0/557043203 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f61b814d6d0 msgr2=0x7f61b814b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:23.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:23.350+0000 7f61be430640 1 --2- 172.21.15.38:0/557043203 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f61b814d6d0 0x7f61b814b070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:23.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:23.350+0000 7f61be430640 1 -- 172.21.15.38:0/557043203 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61b814df70 con 0x7f61b80b2e40 2024-09-21T03:42:23.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:23.350+0000 7f61bdc2f640 1 --2- 172.21.15.38:0/557043203 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f61b8162280 0x7f61b81618b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:42:23.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:23.350+0000 7f61bec31640 1 --2- 172.21.15.38:0/557043203 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f61b814d6d0 0x7f61b814b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:42:23.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:23.350+0000 7f61be430640 1 --2- 172.21.15.38:0/557043203 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f61b80b2e40 0x7f61b80aadc0 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7f61b400c760 tx=0x7f61b400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:23.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:23.350+0000 7f61aeffd640 1 -- 172.21.15.38:0/557043203 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61b4016020 con 0x7f61b80b2e40 2024-09-21T03:42:23.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:23.350+0000 7f61aeffd640 1 -- 172.21.15.38:0/557043203 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f61b400a950 con 0x7f61b80b2e40 2024-09-21T03:42:23.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:23.351+0000 7f61c0694640 1 -- 172.21.15.38:0/557043203 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61b80abe30 con 0x7f61b80b2e40 2024-09-21T03:42:23.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:23.351+0000 7f61aeffd640 1 -- 172.21.15.38:0/557043203 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61b400ada0 con 0x7f61b80b2e40 2024-09-21T03:42:23.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:23.351+0000 7f61c0694640 1 -- 172.21.15.38:0/557043203 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61b81610b0 con 0x7f61b80b2e40 2024-09-21T03:42:23.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:23.351+0000 7f61c0694640 1 -- 172.21.15.38:0/557043203 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61b8162f00 con 0x7f61b80b2e40 2024-09-21T03:42:23.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:23.352+0000 7f61aeffd640 1 -- 172.21.15.38:0/557043203 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f61b4019020 con 0x7f61b80b2e40 2024-09-21T03:42:23.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:23.353+0000 7f61aeffd640 1 --2- 172.21.15.38:0/557043203 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f619c078bc0 0x7f619c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:23.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:23.353+0000 7f61aeffd640 1 -- 172.21.15.38:0/557043203 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 8024+0+0 (secure 0 0 0) 0x7f61b409b8d0 con 0x7f61b80b2e40 2024-09-21T03:42:23.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:23.353+0000 7f61bdc2f640 1 --2- 172.21.15.38:0/557043203 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f619c078bc0 0x7f619c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:23.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:23.354+0000 7f61bdc2f640 1 --2- 172.21.15.38:0/557043203 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f619c078bc0 0x7f619c07b0d0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f61b815dc80 tx=0x7f61a8005f90 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:23.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:23.357+0000 7f61aeffd640 1 -- 172.21.15.38:0/557043203 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f61b4020020 con 0x7f61b80b2e40 2024-09-21T03:42:23.589 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:23.587+0000 7f61c0694640 1 -- 172.21.15.38:0/557043203 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f61b815dcb0 con 0x7f61b80b2e40 2024-09-21T03:42:23.589 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:23.587+0000 7f61aeffd640 1 -- 172.21.15.38:0/557043203 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 203 v203) ==== 95+0+4392 (secure 0 0 0) 0x7f61b814df70 con 0x7f61b80b2e40 2024-09-21T03:42:23.589 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:42:23.590 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":203,"btime":"2024-09-21T03:42:23:215613+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6086,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/965972663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":965972663},{"type":"v1","addr":"172.21.15.195:6836","nonce":965972663}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6700,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/3176122334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":3176122334},{"type":"v1","addr":"172.21.15.195:6837","nonce":3176122334}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":8925,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/4221150556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":4221150556},{"type":"v1","addr":"172.21.15.38:6833","nonce":4221150556}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":196}],"filesystems":[{"mdsmap":{"epoch":203,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:40:02.319282+0000","modified":"2024-09-21T03:42:23.215514+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":116,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-21T03:42:23.590 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 203 2024-09-21T03:42:23.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:23.590+0000 7f61c0694640 1 -- 172.21.15.38:0/557043203 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f619c078bc0 msgr2=0x7f619c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:23.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:23.590+0000 7f61c0694640 1 --2- 172.21.15.38:0/557043203 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f619c078bc0 0x7f619c07b0d0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f61b815dc80 tx=0x7f61a8005f90 comp rx=0 tx=0).stop 2024-09-21T03:42:23.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:23.590+0000 7f61c0694640 1 -- 172.21.15.38:0/557043203 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f61b80b2e40 msgr2=0x7f61b80aadc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:23.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:23.590+0000 7f61c0694640 1 --2- 172.21.15.38:0/557043203 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f61b80b2e40 0x7f61b80aadc0 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7f61b400c760 tx=0x7f61b400cc20 comp rx=0 tx=0).stop 2024-09-21T03:42:23.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:23.590+0000 7f61c0694640 1 -- 172.21.15.38:0/557043203 shutdown_connections 2024-09-21T03:42:23.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:23.590+0000 7f61c0694640 1 --2- 172.21.15.38:0/557043203 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f61b814d6d0 0x7f61b814b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:23.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:23.590+0000 7f61c0694640 1 --2- 172.21.15.38:0/557043203 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f619c078bc0 0x7f619c07b0d0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:23.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:23.590+0000 7f61c0694640 1 --2- 172.21.15.38:0/557043203 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f61b8162280 0x7f61b81618b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:23.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:23.590+0000 7f61c0694640 1 --2- 172.21.15.38:0/557043203 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f61b80b2e40 0x7f61b80aadc0 unknown :-1 s=CLOSED pgs=1713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:23.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:23.590+0000 7f61c0694640 1 -- 172.21.15.38:0/557043203 >> 172.21.15.38:0/557043203 conn(0x7f61b80b6b20 msgr2=0x7f61b806e640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:23.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:23.590+0000 7f61c0694640 1 -- 172.21.15.38:0/557043203 shutdown_connections 2024-09-21T03:42:23.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:23.590+0000 7f61c0694640 1 -- 172.21.15.38:0/557043203 wait complete. 2024-09-21T03:42:23.603 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-21T03:42:23.603 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:42:23.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:23.718+0000 7ff6f1438640 1 -- 172.21.15.38:0/3772568927 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff6ec06ddf0 msgr2=0x7ff6ec06e200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:23.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:23.718+0000 7ff6f1438640 1 --2- 172.21.15.38:0/3772568927 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff6ec06ddf0 0x7ff6ec06e200 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7ff6ec15c8d0 tx=0x7ff6dc02a4b0 comp rx=0 tx=0).stop 2024-09-21T03:42:23.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:23.718+0000 7ff6f1438640 1 -- 172.21.15.38:0/3772568927 shutdown_connections 2024-09-21T03:42:23.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:23.718+0000 7ff6f1438640 1 --2- 172.21.15.38:0/3772568927 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff6ec06ddf0 0x7ff6ec06e200 unknown :-1 s=CLOSED pgs=1715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:23.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:23.718+0000 7ff6f1438640 1 --2- 172.21.15.38:0/3772568927 >> v2:172.21.15.195:3300/0 conn(0x7ff6ec06f4c0 0x7ff6ec15c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:23.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:23.718+0000 7ff6f1438640 1 -- 172.21.15.38:0/3772568927 >> 172.21.15.38:0/3772568927 conn(0x7ff6ec073510 msgr2=0x7ff6ec073920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:23.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:23.718+0000 7ff6f1438640 1 -- 172.21.15.38:0/3772568927 shutdown_connections 2024-09-21T03:42:23.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:23.718+0000 7ff6f1438640 1 -- 172.21.15.38:0/3772568927 wait complete. 2024-09-21T03:42:23.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:23.720+0000 7ff6f1438640 1 Processor -- start 2024-09-21T03:42:23.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:23.720+0000 7ff6f1438640 1 -- start start 2024-09-21T03:42:23.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:23.721+0000 7ff6f1438640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff6ec06ddf0 0x7ff6ec1f71a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:23.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:23.721+0000 7ff6f1438640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff6ec06f4c0 0x7ff6ec159ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:23.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:23.721+0000 7ff6f1438640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff6ec15eff0 0x7ff6ec0b58b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:23.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:23.721+0000 7ff6f1438640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ff6ec15e630 con 0x7ff6ec06f4c0 2024-09-21T03:42:23.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:23.721+0000 7ff6f1438640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ff6ec15e4b0 con 0x7ff6ec15eff0 2024-09-21T03:42:23.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:23.721+0000 7ff6f1438640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7ff6ec15e7b0 con 0x7ff6ec06ddf0 2024-09-21T03:42:23.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:23.722+0000 7ff6ea7fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff6ec06f4c0 0x7ff6ec159ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:23.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:23.722+0000 7ff6ea7fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff6ec06f4c0 0x7ff6ec159ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:59760/0 (socket says 172.21.15.38:59760) 2024-09-21T03:42:23.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:23.722+0000 7ff6ea7fc640 1 -- 172.21.15.38:0/643318536 learned_addr learned my addr 172.21.15.38:0/643318536 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:42:23.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:23.722+0000 7ff6eb7fe640 1 --2- 172.21.15.38:0/643318536 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff6ec15eff0 0x7ff6ec0b58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:23.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:23.722+0000 7ff6eaffd640 1 --2- 172.21.15.38:0/643318536 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff6ec06ddf0 0x7ff6ec1f71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:23.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:23.722+0000 7ff6ea7fc640 1 -- 172.21.15.38:0/643318536 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff6ec06ddf0 msgr2=0x7ff6ec1f71a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:23.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:23.722+0000 7ff6ea7fc640 1 --2- 172.21.15.38:0/643318536 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff6ec06ddf0 0x7ff6ec1f71a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:23.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:23.722+0000 7ff6ea7fc640 1 -- 172.21.15.38:0/643318536 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff6ec15eff0 msgr2=0x7ff6ec0b58b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:23.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:23.722+0000 7ff6ea7fc640 1 --2- 172.21.15.38:0/643318536 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff6ec15eff0 0x7ff6ec0b58b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:23.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:23.722+0000 7ff6ea7fc640 1 -- 172.21.15.38:0/643318536 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6ec1f3650 con 0x7ff6ec06f4c0 2024-09-21T03:42:23.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:23.722+0000 7ff6ea7fc640 1 --2- 172.21.15.38:0/643318536 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff6ec06f4c0 0x7ff6ec159ac0 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7ff6d400b440 tx=0x7ff6d4009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:23.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:23.722+0000 7ff6e37fe640 1 -- 172.21.15.38:0/643318536 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6d4015020 con 0x7ff6ec06f4c0 2024-09-21T03:42:23.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:23.722+0000 7ff6e37fe640 1 -- 172.21.15.38:0/643318536 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff6d4011920 con 0x7ff6ec06f4c0 2024-09-21T03:42:23.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:23.722+0000 7ff6f1438640 1 -- 172.21.15.38:0/643318536 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6ec1f3930 con 0x7ff6ec06f4c0 2024-09-21T03:42:23.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:23.722+0000 7ff6e37fe640 1 -- 172.21.15.38:0/643318536 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6d4011d70 con 0x7ff6ec06f4c0 2024-09-21T03:42:23.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:23.722+0000 7ff6f1438640 1 -- 172.21.15.38:0/643318536 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6ec1f3ef0 con 0x7ff6ec06f4c0 2024-09-21T03:42:23.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:23.723+0000 7ff6f1438640 1 -- 172.21.15.38:0/643318536 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6ec15ed50 con 0x7ff6ec06f4c0 2024-09-21T03:42:23.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:23.724+0000 7ff6e37fe640 1 -- 172.21.15.38:0/643318536 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ff6d4018020 con 0x7ff6ec06f4c0 2024-09-21T03:42:23.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:23.725+0000 7ff6e37fe640 1 --2- 172.21.15.38:0/643318536 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff6cc078b90 0x7ff6cc07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:23.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:23.725+0000 7ff6e37fe640 1 -- 172.21.15.38:0/643318536 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 8024+0+0 (secure 0 0 0) 0x7ff6d409b3f0 con 0x7ff6ec06f4c0 2024-09-21T03:42:23.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:23.725+0000 7ff6eaffd640 1 --2- 172.21.15.38:0/643318536 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff6cc078b90 0x7ff6cc07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:23.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:23.726+0000 7ff6eaffd640 1 --2- 172.21.15.38:0/643318536 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff6cc078b90 0x7ff6cc07b0a0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7ff6e40012e0 tx=0x7ff6e4009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:23.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:23.729+0000 7ff6e37fe640 1 -- 172.21.15.38:0/643318536 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7ff6d401f020 con 0x7ff6ec06f4c0 2024-09-21T03:42:23.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:23.959+0000 7ff6f1438640 1 -- 172.21.15.38:0/643318536 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff6ec074030 con 0x7ff6ec06f4c0 2024-09-21T03:42:23.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:23.959+0000 7ff6e37fe640 1 -- 172.21.15.38:0/643318536 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 203 v203) ==== 95+0+4392 (secure 0 0 0) 0x7ff6d4062960 con 0x7ff6ec06f4c0 2024-09-21T03:42:23.961 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:42:23.961 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":203,"btime":"2024-09-21T03:42:23:215613+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6086,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/965972663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":965972663},{"type":"v1","addr":"172.21.15.195:6836","nonce":965972663}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6700,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/3176122334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":3176122334},{"type":"v1","addr":"172.21.15.195:6837","nonce":3176122334}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":8925,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/4221150556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":4221150556},{"type":"v1","addr":"172.21.15.38:6833","nonce":4221150556}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":196}],"filesystems":[{"mdsmap":{"epoch":203,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:40:02.319282+0000","modified":"2024-09-21T03:42:23.215514+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":116,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-21T03:42:23.962 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 203 2024-09-21T03:42:23.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:23.961+0000 7ff6f1438640 1 -- 172.21.15.38:0/643318536 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff6cc078b90 msgr2=0x7ff6cc07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:23.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:23.961+0000 7ff6f1438640 1 --2- 172.21.15.38:0/643318536 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff6cc078b90 0x7ff6cc07b0a0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7ff6e40012e0 tx=0x7ff6e4009040 comp rx=0 tx=0).stop 2024-09-21T03:42:23.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:23.961+0000 7ff6f1438640 1 -- 172.21.15.38:0/643318536 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff6ec06f4c0 msgr2=0x7ff6ec159ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:23.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:23.961+0000 7ff6f1438640 1 --2- 172.21.15.38:0/643318536 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff6ec06f4c0 0x7ff6ec159ac0 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7ff6d400b440 tx=0x7ff6d4009f90 comp rx=0 tx=0).stop 2024-09-21T03:42:23.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:23.962+0000 7ff6f1438640 1 -- 172.21.15.38:0/643318536 shutdown_connections 2024-09-21T03:42:23.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:23.962+0000 7ff6f1438640 1 --2- 172.21.15.38:0/643318536 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff6ec15eff0 0x7ff6ec0b58b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:23.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:23.962+0000 7ff6f1438640 1 --2- 172.21.15.38:0/643318536 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff6ec06f4c0 0x7ff6ec159ac0 unknown :-1 s=CLOSED pgs=1716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:23.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:23.962+0000 7ff6f1438640 1 --2- 172.21.15.38:0/643318536 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff6cc078b90 0x7ff6cc07b0a0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:23.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:23.962+0000 7ff6f1438640 1 --2- 172.21.15.38:0/643318536 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff6ec06ddf0 0x7ff6ec1f71a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:23.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:23.962+0000 7ff6f1438640 1 -- 172.21.15.38:0/643318536 >> 172.21.15.38:0/643318536 conn(0x7ff6ec073510 msgr2=0x7ff6ec0b0850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:23.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:23.962+0000 7ff6f1438640 1 -- 172.21.15.38:0/643318536 shutdown_connections 2024-09-21T03:42:23.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:23.962+0000 7ff6f1438640 1 -- 172.21.15.38:0/643318536 wait complete. 2024-09-21T03:42:23.975 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-21T03:42:23.975 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:42:24.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:24.090+0000 7ff424350640 1 -- 172.21.15.38:0/925449176 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff41c0d01f0 msgr2=0x7ff41c15daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:24.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:24.090+0000 7ff424350640 1 --2- 172.21.15.38:0/925449176 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff41c0d01f0 0x7ff41c15daf0 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7ff418009830 tx=0x7ff41802a5c0 comp rx=0 tx=0).stop 2024-09-21T03:42:24.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:24.090+0000 7ff424350640 1 -- 172.21.15.38:0/925449176 shutdown_connections 2024-09-21T03:42:24.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:24.090+0000 7ff424350640 1 --2- 172.21.15.38:0/925449176 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff41c0d01f0 0x7ff41c15daf0 unknown :-1 s=CLOSED pgs=1718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:24.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:24.090+0000 7ff424350640 1 -- 172.21.15.38:0/925449176 >> 172.21.15.38:0/925449176 conn(0x7ff41c072990 msgr2=0x7ff41c072da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:24.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:24.090+0000 7ff424350640 1 -- 172.21.15.38:0/925449176 shutdown_connections 2024-09-21T03:42:24.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:24.090+0000 7ff424350640 1 -- 172.21.15.38:0/925449176 wait complete. 2024-09-21T03:42:24.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:24.092+0000 7ff424350640 1 Processor -- start 2024-09-21T03:42:24.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:24.092+0000 7ff424350640 1 -- start start 2024-09-21T03:42:24.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:24.093+0000 7ff424350640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff41c0d01f0 0x7ff41c0c8830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:24.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:24.093+0000 7ff424350640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff41c0d0980 0x7ff41c0c8f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:24.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:24.093+0000 7ff424350640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff41c15a6f0 0x7ff41c0cba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:24.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:24.093+0000 7ff424350640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ff41c0cd3c0 con 0x7ff41c15a6f0 2024-09-21T03:42:24.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:24.093+0000 7ff424350640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ff41c0cd240 con 0x7ff41c0d01f0 2024-09-21T03:42:24.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:24.093+0000 7ff424350640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7ff41c0cd540 con 0x7ff41c0d0980 2024-09-21T03:42:24.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:24.093+0000 7ff4228ed640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff41c15a6f0 0x7ff41c0cba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:24.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:24.093+0000 7ff4228ed640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff41c15a6f0 0x7ff41c0cba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:59782/0 (socket says 172.21.15.38:59782) 2024-09-21T03:42:24.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:24.093+0000 7ff4228ed640 1 -- 172.21.15.38:0/2112563046 learned_addr learned my addr 172.21.15.38:0/2112563046 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:42:24.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:24.093+0000 7ff4220ec640 1 --2- 172.21.15.38:0/2112563046 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff41c0d01f0 0x7ff41c0c8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:24.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:24.093+0000 7ff4218eb640 1 --2- 172.21.15.38:0/2112563046 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff41c0d0980 0x7ff41c0c8f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:24.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:24.093+0000 7ff4228ed640 1 -- 172.21.15.38:0/2112563046 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff41c0d0980 msgr2=0x7ff41c0c8f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:24.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:24.093+0000 7ff4228ed640 1 --2- 172.21.15.38:0/2112563046 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff41c0d0980 0x7ff41c0c8f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:24.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:24.093+0000 7ff4228ed640 1 -- 172.21.15.38:0/2112563046 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff41c0d01f0 msgr2=0x7ff41c0c8830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:24.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:24.093+0000 7ff4228ed640 1 --2- 172.21.15.38:0/2112563046 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff41c0d01f0 0x7ff41c0c8830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:24.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:24.093+0000 7ff4228ed640 1 -- 172.21.15.38:0/2112563046 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff41c0c9620 con 0x7ff41c15a6f0 2024-09-21T03:42:24.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:24.093+0000 7ff4220ec640 1 --2- 172.21.15.38:0/2112563046 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff41c0d01f0 0x7ff41c0c8830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:42:24.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:24.094+0000 7ff4228ed640 1 --2- 172.21.15.38:0/2112563046 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff41c15a6f0 0x7ff41c0cba60 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7ff414002230 tx=0x7ff41400cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:24.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:24.094+0000 7ff412ffd640 1 -- 172.21.15.38:0/2112563046 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff414003b70 con 0x7ff41c15a6f0 2024-09-21T03:42:24.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:24.094+0000 7ff412ffd640 1 -- 172.21.15.38:0/2112563046 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff414003d10 con 0x7ff41c15a6f0 2024-09-21T03:42:24.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:24.094+0000 7ff424350640 1 -- 172.21.15.38:0/2112563046 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff41c0cb280 con 0x7ff41c15a6f0 2024-09-21T03:42:24.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:24.094+0000 7ff412ffd640 1 -- 172.21.15.38:0/2112563046 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff414005330 con 0x7ff41c15a6f0 2024-09-21T03:42:24.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:24.094+0000 7ff424350640 1 -- 172.21.15.38:0/2112563046 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff41c1f9970 con 0x7ff41c15a6f0 2024-09-21T03:42:24.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:24.095+0000 7ff424350640 1 -- 172.21.15.38:0/2112563046 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff41c1f9bd0 con 0x7ff41c15a6f0 2024-09-21T03:42:24.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:24.096+0000 7ff412ffd640 1 -- 172.21.15.38:0/2112563046 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ff4140054d0 con 0x7ff41c15a6f0 2024-09-21T03:42:24.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:24.096+0000 7ff412ffd640 1 --2- 172.21.15.38:0/2112563046 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff400078bc0 0x7ff40007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:24.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:24.096+0000 7ff412ffd640 1 -- 172.21.15.38:0/2112563046 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 8024+0+0 (secure 0 0 0) 0x7ff41409b400 con 0x7ff41c15a6f0 2024-09-21T03:42:24.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:24.096+0000 7ff4220ec640 1 --2- 172.21.15.38:0/2112563046 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff400078bc0 0x7ff40007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:24.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:24.097+0000 7ff4220ec640 1 --2- 172.21.15.38:0/2112563046 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff400078bc0 0x7ff40007b0d0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7ff418034000 tx=0x7ff418004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:24.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:24.101+0000 7ff412ffd640 1 -- 172.21.15.38:0/2112563046 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7ff414062fd0 con 0x7ff41c15a6f0 2024-09-21T03:42:24.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:24.327+0000 7ff424350640 1 -- 172.21.15.38:0/2112563046 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff41c15daf0 con 0x7ff41c15a6f0 2024-09-21T03:42:24.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:24.327+0000 7ff412ffd640 1 -- 172.21.15.38:0/2112563046 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 203 v203) ==== 95+0+4392 (secure 0 0 0) 0x7ff4140628d0 con 0x7ff41c15a6f0 2024-09-21T03:42:24.329 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:42:24.330 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":203,"btime":"2024-09-21T03:42:23:215613+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6086,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/965972663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":965972663},{"type":"v1","addr":"172.21.15.195:6836","nonce":965972663}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6700,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/3176122334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":3176122334},{"type":"v1","addr":"172.21.15.195:6837","nonce":3176122334}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":8925,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/4221150556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":4221150556},{"type":"v1","addr":"172.21.15.38:6833","nonce":4221150556}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":196}],"filesystems":[{"mdsmap":{"epoch":203,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:40:02.319282+0000","modified":"2024-09-21T03:42:23.215514+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":116,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-21T03:42:24.330 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 203 2024-09-21T03:42:24.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:24.330+0000 7ff424350640 1 -- 172.21.15.38:0/2112563046 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff400078bc0 msgr2=0x7ff40007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:24.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:24.330+0000 7ff424350640 1 --2- 172.21.15.38:0/2112563046 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff400078bc0 0x7ff40007b0d0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7ff418034000 tx=0x7ff418004730 comp rx=0 tx=0).stop 2024-09-21T03:42:24.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:24.330+0000 7ff424350640 1 -- 172.21.15.38:0/2112563046 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff41c15a6f0 msgr2=0x7ff41c0cba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:24.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:24.330+0000 7ff424350640 1 --2- 172.21.15.38:0/2112563046 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff41c15a6f0 0x7ff41c0cba60 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7ff414002230 tx=0x7ff41400cb90 comp rx=0 tx=0).stop 2024-09-21T03:42:24.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:24.330+0000 7ff424350640 1 -- 172.21.15.38:0/2112563046 shutdown_connections 2024-09-21T03:42:24.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:24.330+0000 7ff424350640 1 --2- 172.21.15.38:0/2112563046 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff41c15a6f0 0x7ff41c0cba60 unknown :-1 s=CLOSED pgs=1719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:24.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:24.330+0000 7ff424350640 1 --2- 172.21.15.38:0/2112563046 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff400078bc0 0x7ff40007b0d0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:24.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:24.330+0000 7ff424350640 1 --2- 172.21.15.38:0/2112563046 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff41c0d0980 0x7ff41c0c8f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:24.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:24.330+0000 7ff424350640 1 --2- 172.21.15.38:0/2112563046 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff41c0d01f0 0x7ff41c0c8830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:24.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:24.330+0000 7ff424350640 1 -- 172.21.15.38:0/2112563046 >> 172.21.15.38:0/2112563046 conn(0x7ff41c072990 msgr2=0x7ff41c15b830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:24.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:24.330+0000 7ff424350640 1 -- 172.21.15.38:0/2112563046 shutdown_connections 2024-09-21T03:42:24.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:24.330+0000 7ff424350640 1 -- 172.21.15.38:0/2112563046 wait complete. 2024-09-21T03:42:24.343 INFO:tasks.cephfs.test_damage:Result: Mutation 'Corrupt mds_snaptable' led to DAMAGED state 2024-09-21T03:42:24.343 INFO:tasks.cephfs.test_damage:Applying mutation 'Corrupt mds0_inotable' 2024-09-21T03:42:24.343 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-21T03:42:24.343 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:42:24.359 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056795,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2354 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T03:42:24.359 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi038.front.sepia.ceph.com 2024-09-21T03:42:24.360 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:42:24.360 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T03:42:24.415 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T03:42:24.415 INFO:teuthology.orchestra.run.smithi038.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-09-21T03:42:24.415 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-21T03:42:24.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:24.571+0000 7fe4d7c0a640 1 -- 172.21.15.38:0/2786738238 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe4d00ceb90 msgr2=0x7fe4d00cefa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:24.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:24.571+0000 7fe4d7c0a640 1 --2- 172.21.15.38:0/2786738238 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe4d00ceb90 0x7fe4d00cefa0 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7fe4c8005fd0 tx=0x7fe4c802a820 comp rx=0 tx=0).stop 2024-09-21T03:42:24.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:24.571+0000 7fe4d7c0a640 1 -- 172.21.15.38:0/2786738238 shutdown_connections 2024-09-21T03:42:24.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:24.571+0000 7fe4d7c0a640 1 --2- 172.21.15.38:0/2786738238 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe4d00ceb90 0x7fe4d00cefa0 unknown :-1 s=CLOSED pgs=1721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:24.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:24.571+0000 7fe4d7c0a640 1 -- 172.21.15.38:0/2786738238 >> 172.21.15.38:0/2786738238 conn(0x7fe4d009d830 msgr2=0x7fe4d00b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:24.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:24.571+0000 7fe4d7c0a640 1 -- 172.21.15.38:0/2786738238 shutdown_connections 2024-09-21T03:42:24.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:24.571+0000 7fe4d7c0a640 1 -- 172.21.15.38:0/2786738238 wait complete. 2024-09-21T03:42:24.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:24.573+0000 7fe4d7c0a640 1 Processor -- start 2024-09-21T03:42:24.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:24.573+0000 7fe4d7c0a640 1 -- start start 2024-09-21T03:42:24.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:24.574+0000 7fe4d7c0a640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe4d00ceb90 0x7fe4d006e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:24.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:24.574+0000 7fe4d7c0a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe4d00d0170 0x7fe4d0157300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:24.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:24.574+0000 7fe4d7c0a640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe4d015ee80 0x7fe4d015be30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:24.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:24.574+0000 7fe4d7c0a640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fe4d00b6790 con 0x7fe4d00d0170 2024-09-21T03:42:24.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:24.574+0000 7fe4d7c0a640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fe4d00d14b0 con 0x7fe4d00ceb90 2024-09-21T03:42:24.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:24.574+0000 7fe4d7c0a640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fe4d00b6910 con 0x7fe4d015ee80 2024-09-21T03:42:24.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:24.575+0000 7fe4d51a5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe4d00d0170 0x7fe4d0157300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:24.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:24.575+0000 7fe4d51a5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe4d00d0170 0x7fe4d0157300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:59802/0 (socket says 172.21.15.38:59802) 2024-09-21T03:42:24.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:24.575+0000 7fe4d51a5640 1 -- 172.21.15.38:0/279055459 learned_addr learned my addr 172.21.15.38:0/279055459 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:42:24.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:24.575+0000 7fe4d59a6640 1 --2- 172.21.15.38:0/279055459 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe4d00ceb90 0x7fe4d006e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:24.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:24.575+0000 7fe4d61a7640 1 --2- 172.21.15.38:0/279055459 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe4d015ee80 0x7fe4d015be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:24.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:24.575+0000 7fe4d51a5640 1 -- 172.21.15.38:0/279055459 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe4d015ee80 msgr2=0x7fe4d015be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:24.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:24.575+0000 7fe4d51a5640 1 --2- 172.21.15.38:0/279055459 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe4d015ee80 0x7fe4d015be30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:24.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:24.575+0000 7fe4d51a5640 1 -- 172.21.15.38:0/279055459 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe4d00ceb90 msgr2=0x7fe4d006e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:24.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:24.575+0000 7fe4d51a5640 1 --2- 172.21.15.38:0/279055459 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe4d00ceb90 0x7fe4d006e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:24.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:24.575+0000 7fe4d51a5640 1 -- 172.21.15.38:0/279055459 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4d006cf80 con 0x7fe4d00d0170 2024-09-21T03:42:24.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:24.575+0000 7fe4d51a5640 1 --2- 172.21.15.38:0/279055459 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe4d00d0170 0x7fe4d0157300 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7fe4c000c3c0 tx=0x7fe4c000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:24.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:24.575+0000 7fe4c67fc640 1 -- 172.21.15.38:0/279055459 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4c0011790 con 0x7fe4d00d0170 2024-09-21T03:42:24.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:24.575+0000 7fe4c67fc640 1 -- 172.21.15.38:0/279055459 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe4c0017070 con 0x7fe4d00d0170 2024-09-21T03:42:24.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:24.575+0000 7fe4c67fc640 1 -- 172.21.15.38:0/279055459 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4c0011cc0 con 0x7fe4d00d0170 2024-09-21T03:42:24.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:24.575+0000 7fe4d7c0a640 1 -- 172.21.15.38:0/279055459 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4d006f2a0 con 0x7fe4d00d0170 2024-09-21T03:42:24.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:24.575+0000 7fe4d7c0a640 1 -- 172.21.15.38:0/279055459 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4d006d500 con 0x7fe4d00d0170 2024-09-21T03:42:24.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:24.576+0000 7fe4d7c0a640 1 -- 172.21.15.38:0/279055459 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4d006f010 con 0x7fe4d00d0170 2024-09-21T03:42:24.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:24.577+0000 7fe4c67fc640 1 -- 172.21.15.38:0/279055459 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fe4c001c460 con 0x7fe4d00d0170 2024-09-21T03:42:24.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:24.578+0000 7fe4c67fc640 1 --2- 172.21.15.38:0/279055459 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fe4b4078b90 0x7fe4b407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:24.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:24.578+0000 7fe4c67fc640 1 -- 172.21.15.38:0/279055459 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 8024+0+0 (secure 0 0 0) 0x7fe4c009b0f0 con 0x7fe4d00d0170 2024-09-21T03:42:24.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:24.578+0000 7fe4d59a6640 1 --2- 172.21.15.38:0/279055459 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fe4b4078b90 0x7fe4b407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:24.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:24.579+0000 7fe4d59a6640 1 --2- 172.21.15.38:0/279055459 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fe4b4078b90 0x7fe4b407b0a0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc001720 tx=0x7fe4cc009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:24.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:24.583+0000 7fe4c67fc640 1 -- 172.21.15.38:0/279055459 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fe4d01592d0 con 0x7fe4d00d0170 2024-09-21T03:42:24.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:24.808+0000 7fe4d7c0a640 1 -- 172.21.15.38:0/279055459 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fe4d00b6e50 con 0x7fe4d00d0170 2024-09-21T03:42:25.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:25.213+0000 7fe4c67fc640 1 -- 172.21.15.38:0/279055459 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v204) ==== 161+0+0 (secure 0 0 0) 0x7fe4c00626f0 con 0x7fe4d00d0170 2024-09-21T03:42:25.215 INFO:teuthology.orchestra.run.smithi038.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-21T03:42:25.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:25.217+0000 7fe4d7c0a640 1 -- 172.21.15.38:0/279055459 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fe4b4078b90 msgr2=0x7fe4b407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:25.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:25.217+0000 7fe4d7c0a640 1 --2- 172.21.15.38:0/279055459 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fe4b4078b90 0x7fe4b407b0a0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc001720 tx=0x7fe4cc009040 comp rx=0 tx=0).stop 2024-09-21T03:42:25.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:25.217+0000 7fe4d7c0a640 1 -- 172.21.15.38:0/279055459 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe4d00d0170 msgr2=0x7fe4d0157300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:25.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:25.217+0000 7fe4d7c0a640 1 --2- 172.21.15.38:0/279055459 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe4d00d0170 0x7fe4d0157300 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7fe4c000c3c0 tx=0x7fe4c000c880 comp rx=0 tx=0).stop 2024-09-21T03:42:25.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:25.217+0000 7fe4d7c0a640 1 -- 172.21.15.38:0/279055459 shutdown_connections 2024-09-21T03:42:25.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:25.217+0000 7fe4d7c0a640 1 --2- 172.21.15.38:0/279055459 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fe4b4078b90 0x7fe4b407b0a0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:25.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:25.217+0000 7fe4d7c0a640 1 --2- 172.21.15.38:0/279055459 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe4d015ee80 0x7fe4d015be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:25.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:25.217+0000 7fe4d7c0a640 1 --2- 172.21.15.38:0/279055459 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe4d00d0170 0x7fe4d0157300 unknown :-1 s=CLOSED pgs=1722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:25.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:25.217+0000 7fe4d7c0a640 1 --2- 172.21.15.38:0/279055459 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe4d00ceb90 0x7fe4d006e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:25.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:25.217+0000 7fe4d7c0a640 1 -- 172.21.15.38:0/279055459 >> 172.21.15.38:0/279055459 conn(0x7fe4d009d830 msgr2=0x7fe4d00b4460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:25.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:25.217+0000 7fe4d7c0a640 1 -- 172.21.15.38:0/279055459 shutdown_connections 2024-09-21T03:42:25.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:25.217+0000 7fe4d7c0a640 1 -- 172.21.15.38:0/279055459 wait complete. 2024-09-21T03:42:25.231 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-09-21T03:42:25.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:25.347+0000 7fc05e5c6640 1 -- 172.21.15.38:0/3103110212 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc05815ee80 msgr2=0x7fc0580d0170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:25.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:25.347+0000 7fc05e5c6640 1 --2- 172.21.15.38:0/3103110212 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc05815ee80 0x7fc0580d0170 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7fc0580b3c20 tx=0x7fc04802a320 comp rx=0 tx=0).stop 2024-09-21T03:42:25.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:25.347+0000 7fc05e5c6640 1 -- 172.21.15.38:0/3103110212 shutdown_connections 2024-09-21T03:42:25.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:25.347+0000 7fc05e5c6640 1 --2- 172.21.15.38:0/3103110212 >> v2:172.21.15.195:3300/0 conn(0x7fc0580ceb90 0x7fc0580cefa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:25.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:25.347+0000 7fc05e5c6640 1 --2- 172.21.15.38:0/3103110212 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc05815ee80 0x7fc0580d0170 unknown :-1 s=CLOSED pgs=1724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:25.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:25.347+0000 7fc05e5c6640 1 -- 172.21.15.38:0/3103110212 >> 172.21.15.38:0/3103110212 conn(0x7fc05809d830 msgr2=0x7fc0580b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:25.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:25.347+0000 7fc05e5c6640 1 -- 172.21.15.38:0/3103110212 shutdown_connections 2024-09-21T03:42:25.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:25.347+0000 7fc05e5c6640 1 -- 172.21.15.38:0/3103110212 wait complete. 2024-09-21T03:42:25.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:25.349+0000 7fc05e5c6640 1 Processor -- start 2024-09-21T03:42:25.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:25.349+0000 7fc05e5c6640 1 -- start start 2024-09-21T03:42:25.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:25.350+0000 7fc05e5c6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc0580b9ad0 0x7fc0581f8c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:25.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:25.350+0000 7fc05e5c6640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc0580ceb90 0x7fc058158540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:25.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:25.351+0000 7fc05e5c6640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc05815ee80 0x7fc05806cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:25.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:25.351+0000 7fc05e5c6640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fc0580b6580 con 0x7fc0580b9ad0 2024-09-21T03:42:25.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:25.351+0000 7fc05e5c6640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fc0580ba0f0 con 0x7fc0580ceb90 2024-09-21T03:42:25.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:25.351+0000 7fc05e5c6640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fc0580b6700 con 0x7fc05815ee80 2024-09-21T03:42:25.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:25.351+0000 7fc057fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc0580b9ad0 0x7fc0581f8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:25.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:25.351+0000 7fc057fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc0580b9ad0 0x7fc0581f8c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:59810/0 (socket says 172.21.15.38:59810) 2024-09-21T03:42:25.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:25.351+0000 7fc057fff640 1 -- 172.21.15.38:0/2744788829 learned_addr learned my addr 172.21.15.38:0/2744788829 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:42:25.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:25.351+0000 7fc0577fe640 1 --2- 172.21.15.38:0/2744788829 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc0580ceb90 0x7fc058158540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:25.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:25.351+0000 7fc05cb63640 1 --2- 172.21.15.38:0/2744788829 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc05815ee80 0x7fc05806cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:25.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:25.351+0000 7fc057fff640 1 -- 172.21.15.38:0/2744788829 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc05815ee80 msgr2=0x7fc05806cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:25.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:25.351+0000 7fc057fff640 1 --2- 172.21.15.38:0/2744788829 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc05815ee80 0x7fc05806cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:25.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:25.351+0000 7fc057fff640 1 -- 172.21.15.38:0/2744788829 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc0580ceb90 msgr2=0x7fc058158540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:25.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:25.351+0000 7fc057fff640 1 --2- 172.21.15.38:0/2744788829 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc0580ceb90 0x7fc058158540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:25.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:25.351+0000 7fc057fff640 1 -- 172.21.15.38:0/2744788829 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc058158a80 con 0x7fc0580b9ad0 2024-09-21T03:42:25.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:25.351+0000 7fc057fff640 1 --2- 172.21.15.38:0/2744788829 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc0580b9ad0 0x7fc0581f8c30 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7fc048000c00 tx=0x7fc048033ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:25.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:25.351+0000 7fc054ff9640 1 -- 172.21.15.38:0/2744788829 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc04803f020 con 0x7fc0580b9ad0 2024-09-21T03:42:25.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:25.351+0000 7fc054ff9640 1 -- 172.21.15.38:0/2744788829 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc048033d90 con 0x7fc0580b9ad0 2024-09-21T03:42:25.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:25.352+0000 7fc05e5c6640 1 -- 172.21.15.38:0/2744788829 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0581f9b40 con 0x7fc0580b9ad0 2024-09-21T03:42:25.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:25.352+0000 7fc054ff9640 1 -- 172.21.15.38:0/2744788829 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc048037040 con 0x7fc0580b9ad0 2024-09-21T03:42:25.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:25.352+0000 7fc05e5c6640 1 -- 172.21.15.38:0/2744788829 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0581f9860 con 0x7fc0580b9ad0 2024-09-21T03:42:25.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:25.353+0000 7fc05e5c6640 1 -- 172.21.15.38:0/2744788829 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0580b6bf0 con 0x7fc0580b9ad0 2024-09-21T03:42:25.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:25.353+0000 7fc054ff9640 1 -- 172.21.15.38:0/2744788829 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc048042020 con 0x7fc0580b9ad0 2024-09-21T03:42:25.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:25.354+0000 7fc054ff9640 1 --2- 172.21.15.38:0/2744788829 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc044078b90 0x7fc04407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:25.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:25.354+0000 7fc054ff9640 1 -- 172.21.15.38:0/2744788829 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 8024+0+0 (secure 0 0 0) 0x7fc0480bb3e0 con 0x7fc0580b9ad0 2024-09-21T03:42:25.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:25.354+0000 7fc0577fe640 1 --2- 172.21.15.38:0/2744788829 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc044078b90 0x7fc04407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:25.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:25.355+0000 7fc0577fe640 1 --2- 172.21.15.38:0/2744788829 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc044078b90 0x7fc04407b0a0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fc0580cac80 tx=0x7fc04c009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:25.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:25.359+0000 7fc054ff9640 1 -- 172.21.15.38:0/2744788829 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fc048049070 con 0x7fc0580b9ad0 2024-09-21T03:42:25.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:25.620+0000 7fc05e5c6640 1 -- 172.21.15.38:0/2744788829 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7fc0580b96a0 con 0x7fc0580b9ad0 2024-09-21T03:42:26.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.216+0000 7fc054ff9640 1 -- 172.21.15.38:0/2744788829 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v205) ==== 101+0+0 (secure 0 0 0) 0x7fc058158a80 con 0x7fc0580b9ad0 2024-09-21T03:42:26.218 INFO:teuthology.orchestra.run.smithi038.stderr:repaired: restoring rank 6:0 2024-09-21T03:42:26.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.219+0000 7fc05e5c6640 1 -- 172.21.15.38:0/2744788829 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc044078b90 msgr2=0x7fc04407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:26.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.219+0000 7fc05e5c6640 1 --2- 172.21.15.38:0/2744788829 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc044078b90 0x7fc04407b0a0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fc0580cac80 tx=0x7fc04c009040 comp rx=0 tx=0).stop 2024-09-21T03:42:26.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.219+0000 7fc05e5c6640 1 -- 172.21.15.38:0/2744788829 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc0580b9ad0 msgr2=0x7fc0581f8c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:26.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.219+0000 7fc05e5c6640 1 --2- 172.21.15.38:0/2744788829 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc0580b9ad0 0x7fc0581f8c30 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7fc048000c00 tx=0x7fc048033ba0 comp rx=0 tx=0).stop 2024-09-21T03:42:26.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.219+0000 7fc05e5c6640 1 -- 172.21.15.38:0/2744788829 shutdown_connections 2024-09-21T03:42:26.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.219+0000 7fc05e5c6640 1 --2- 172.21.15.38:0/2744788829 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc044078b90 0x7fc04407b0a0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:26.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.219+0000 7fc05e5c6640 1 --2- 172.21.15.38:0/2744788829 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc05815ee80 0x7fc05806cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:26.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.219+0000 7fc05e5c6640 1 --2- 172.21.15.38:0/2744788829 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc0580ceb90 0x7fc058158540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:26.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.219+0000 7fc05e5c6640 1 --2- 172.21.15.38:0/2744788829 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc0580b9ad0 0x7fc0581f8c30 unknown :-1 s=CLOSED pgs=1725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:26.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.219+0000 7fc05e5c6640 1 -- 172.21.15.38:0/2744788829 >> 172.21.15.38:0/2744788829 conn(0x7fc05809d830 msgr2=0x7fc0580b3fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:26.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.219+0000 7fc05e5c6640 1 -- 172.21.15.38:0/2744788829 shutdown_connections 2024-09-21T03:42:26.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.219+0000 7fc05e5c6640 1 -- 172.21.15.38:0/2744788829 wait complete. 2024-09-21T03:42:26.234 DEBUG:teuthology.orchestra.run.smithi038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-09-21T03:42:26.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.291+0000 7fa53f0ccf40 1 Processor -- start 2024-09-21T03:42:26.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.291+0000 7fa53f0ccf40 1 -- start start 2024-09-21T03:42:26.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.293+0000 7fa53f0ccf40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa5240030f0 0x55dee90f1660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:26.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.293+0000 7fa53f0ccf40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dee91af610 0x55dee90f1ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:26.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.293+0000 7fa53f0ccf40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55dee91ca110 0x55dee91ca520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:26.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.293+0000 7fa53f0ccf40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55dee9043d60 con 0x55dee91af610 2024-09-21T03:42:26.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.293+0000 7fa53f0ccf40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55dee90a2430 con 0x7fa5240030f0 2024-09-21T03:42:26.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.293+0000 7fa53f0ccf40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55dee9122500 con 0x55dee91ca110 2024-09-21T03:42:26.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.293+0000 7fa530385640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dee91af610 0x55dee90f1ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:26.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.293+0000 7fa530385640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dee91af610 0x55dee90f1ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:59828/0 (socket says 172.21.15.38:59828) 2024-09-21T03:42:26.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.293+0000 7fa530385640 1 -- 172.21.15.38:0/2039351869 learned_addr learned my addr 172.21.15.38:0/2039351869 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:42:26.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.293+0000 7fa538b86640 1 --2- 172.21.15.38:0/2039351869 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa5240030f0 0x55dee90f1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:26.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.293+0000 7fa539387640 1 --2- 172.21.15.38:0/2039351869 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55dee91ca110 0x55dee91ca520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:26.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.293+0000 7fa530385640 1 -- 172.21.15.38:0/2039351869 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55dee91ca110 msgr2=0x55dee91ca520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:26.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.293+0000 7fa530385640 1 --2- 172.21.15.38:0/2039351869 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55dee91ca110 0x55dee91ca520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:26.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.293+0000 7fa530385640 1 -- 172.21.15.38:0/2039351869 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa5240030f0 msgr2=0x55dee90f1660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:26.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.293+0000 7fa530385640 1 --2- 172.21.15.38:0/2039351869 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa5240030f0 0x55dee90f1660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:26.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.293+0000 7fa530385640 1 -- 172.21.15.38:0/2039351869 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dee91ac6f0 con 0x55dee91af610 2024-09-21T03:42:26.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.293+0000 7fa538b86640 1 --2- 172.21.15.38:0/2039351869 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa5240030f0 0x55dee90f1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:42:26.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.293+0000 7fa530385640 1 --2- 172.21.15.38:0/2039351869 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dee91af610 0x55dee90f1ba0 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7fa5200260c0 tx=0x7fa520001a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38edd34827ddea27 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:26.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.293+0000 7fa5337fe640 1 -- 172.21.15.38:0/2039351869 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5200014b0 con 0x55dee91af610 2024-09-21T03:42:26.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.294+0000 7fa5337fe640 1 -- 172.21.15.38:0/2039351869 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa520065070 con 0x55dee91af610 2024-09-21T03:42:26.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.294+0000 7fa5337fe640 1 -- 172.21.15.38:0/2039351869 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa52004db80 con 0x55dee91af610 2024-09-21T03:42:26.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.294+0000 7fa53f0ccf40 1 -- 172.21.15.38:0/2039351869 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dee91af610 msgr2=0x55dee90f1ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:26.299 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.294+0000 7fa53f0ccf40 1 --2- 172.21.15.38:0/2039351869 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dee91af610 0x55dee90f1ba0 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7fa5200260c0 tx=0x7fa520001a30 comp rx=0 tx=0).stop 2024-09-21T03:42:26.299 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.294+0000 7fa53f0ccf40 1 -- 172.21.15.38:0/2039351869 shutdown_connections 2024-09-21T03:42:26.299 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.294+0000 7fa53f0ccf40 1 --2- 172.21.15.38:0/2039351869 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55dee91ca110 0x55dee91ca520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:26.299 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.294+0000 7fa53f0ccf40 1 --2- 172.21.15.38:0/2039351869 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dee91af610 0x55dee90f1ba0 unknown :-1 s=CLOSED pgs=1728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:26.299 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.294+0000 7fa53f0ccf40 1 --2- 172.21.15.38:0/2039351869 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa5240030f0 0x55dee90f1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:26.299 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.294+0000 7fa53f0ccf40 1 -- 172.21.15.38:0/2039351869 >> 172.21.15.38:0/2039351869 conn(0x55dee9014ac0 msgr2=0x55dee91c74a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:26.299 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.294+0000 7fa53f0ccf40 1 -- 172.21.15.38:0/2039351869 shutdown_connections 2024-09-21T03:42:26.299 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.294+0000 7fa53f0ccf40 1 -- 172.21.15.38:0/2039351869 wait complete. 2024-09-21T03:42:26.301 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.295+0000 7fa53f0ccf40 1 Processor -- start 2024-09-21T03:42:26.301 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.295+0000 7fa53f0ccf40 1 -- start start 2024-09-21T03:42:26.302 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.296+0000 7fa53f0ccf40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dee91ca110 0x55dee91a9c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:26.302 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.296+0000 7fa53f0ccf40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa5240030f0 0x55dee91a9100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:26.302 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.296+0000 7fa53f0ccf40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55dee9256d70 0x55dee9266c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:26.302 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.296+0000 7fa53f0ccf40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55dee91c9560 con 0x55dee91ca110 2024-09-21T03:42:26.302 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.296+0000 7fa53f0ccf40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55dee91c9780 con 0x7fa5240030f0 2024-09-21T03:42:26.302 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.296+0000 7fa53f0ccf40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55dee91c8750 con 0x55dee9256d70 2024-09-21T03:42:26.302 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.297+0000 7fa538b86640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dee91ca110 0x55dee91a9c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:26.302 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.297+0000 7fa538b86640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dee91ca110 0x55dee91a9c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:59838/0 (socket says 172.21.15.38:59838) 2024-09-21T03:42:26.302 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.297+0000 7fa538b86640 1 -- 172.21.15.38:0/3818166296 learned_addr learned my addr 172.21.15.38:0/3818166296 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:42:26.302 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.297+0000 7fa539387640 1 --2- 172.21.15.38:0/3818166296 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55dee9256d70 0x55dee9266c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:26.302 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.297+0000 7fa530385640 1 --2- 172.21.15.38:0/3818166296 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa5240030f0 0x55dee91a9100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:26.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.297+0000 7fa538b86640 1 -- 172.21.15.38:0/3818166296 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55dee9256d70 msgr2=0x55dee9266c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:26.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.297+0000 7fa538b86640 1 --2- 172.21.15.38:0/3818166296 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55dee9256d70 0x55dee9266c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:26.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.297+0000 7fa538b86640 1 -- 172.21.15.38:0/3818166296 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa5240030f0 msgr2=0x55dee91a9100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:26.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.297+0000 7fa538b86640 1 --2- 172.21.15.38:0/3818166296 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa5240030f0 0x55dee91a9100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:26.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.297+0000 7fa538b86640 1 -- 172.21.15.38:0/3818166296 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dee92636b0 con 0x55dee91ca110 2024-09-21T03:42:26.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.297+0000 7fa538b86640 1 --2- 172.21.15.38:0/3818166296 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dee91ca110 0x55dee91a9c40 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7fa52800a7a0 tx=0x7fa52800ac60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:26.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.297+0000 7fa5327fc640 1 -- 172.21.15.38:0/3818166296 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa52801b070 con 0x55dee91ca110 2024-09-21T03:42:26.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.297+0000 7fa5327fc640 1 -- 172.21.15.38:0/3818166296 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa528009ac0 con 0x55dee91ca110 2024-09-21T03:42:26.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.297+0000 7fa53f0ccf40 1 -- 172.21.15.38:0/3818166296 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55dee9263990 con 0x55dee91ca110 2024-09-21T03:42:26.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.297+0000 7fa5327fc640 1 -- 172.21.15.38:0/3818166296 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa528016460 con 0x55dee91ca110 2024-09-21T03:42:26.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.297+0000 7fa53f0ccf40 1 -- 172.21.15.38:0/3818166296 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dee91aa280 con 0x55dee91ca110 2024-09-21T03:42:26.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.299+0000 7fa5327fc640 1 -- 172.21.15.38:0/3818166296 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa528014070 con 0x55dee91ca110 2024-09-21T03:42:26.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.300+0000 7fa5327fc640 1 --2- 172.21.15.38:0/3818166296 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa518077570 0x7fa518079a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:26.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.300+0000 7fa5327fc640 1 -- 172.21.15.38:0/3818166296 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 8024+0+0 (secure 0 0 0) 0x7fa52809a8c0 con 0x55dee91ca110 2024-09-21T03:42:26.304 INFO:teuthology.orchestra.run.smithi038.stdout:Importing pool 2024-09-21T03:42:26.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.300+0000 7fa530385640 1 --2- 172.21.15.38:0/3818166296 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa518077570 0x7fa518079a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:26.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.300+0000 7fa53f0ccf40 1 --2- 172.21.15.38:0/3818166296 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x55dee91caa30 0x55dee926f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:26.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.300+0000 7fa53f0ccf40 1 -- 172.21.15.38:0/3818166296 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55dee926fd00 con 0x55dee91caa30 2024-09-21T03:42:26.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.300+0000 7fa539387640 1 --2- 172.21.15.38:0/3818166296 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x55dee91caa30 0x55dee926f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:26.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.301+0000 7fa530385640 1 --2- 172.21.15.38:0/3818166296 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa518077570 0x7fa518079a80 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fa520064000 tx=0x7fa5200092d0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:26.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.301+0000 7fa539387640 1 --2- 172.21.15.38:0/3818166296 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x55dee91caa30 0x55dee926f7c0 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:26.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.303+0000 7fa539387640 1 -- 172.21.15.38:0/3818166296 <== osd.7 v2:172.21.15.195:6818/1606106342 1 ==== osd_op_reply(1 601.00000000 [create] v116'598 uv572 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fa534003040 con 0x55dee91caa30 2024-09-21T03:42:26.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.303+0000 7fa53f0ccf40 1 -- 172.21.15.38:0/3818166296 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e116) -- 0x55dee92700a0 con 0x55dee91caa30 2024-09-21T03:42:26.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.305+0000 7fa539387640 1 -- 172.21.15.38:0/3818166296 <== osd.7 v2:172.21.15.195:6818/1606106342 2 ==== osd_op_reply(2 601.00000000 [delete] v116'599 uv599 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa534003040 con 0x55dee91caa30 2024-09-21T03:42:26.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.305+0000 7fa53f0ccf40 1 -- 172.21.15.38:0/3818166296 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55dee9270440 con 0x55dee91caa30 2024-09-21T03:42:26.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.306+0000 7fa539387640 1 -- 172.21.15.38:0/3818166296 <== osd.7 v2:172.21.15.195:6818/1606106342 3 ==== osd_op_reply(3 601.00000000 [create] v116'600 uv600 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa534003040 con 0x55dee91caa30 2024-09-21T03:42:26.308 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-09-21T03:42:26.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.306+0000 7fa53f0ccf40 1 -- 172.21.15.38:0/3818166296 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55dee92717f0 con 0x55dee91caa30 2024-09-21T03:42:26.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.308+0000 7fa539387640 1 -- 172.21.15.38:0/3818166296 <== osd.7 v2:172.21.15.195:6818/1606106342 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v116'601 uv601 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa534003040 con 0x55dee91caa30 2024-09-21T03:42:26.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.308+0000 7fa53f0ccf40 1 -- 172.21.15.38:0/3818166296 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55dee9271b90 con 0x55dee91caa30 2024-09-21T03:42:26.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.309+0000 7fa539387640 1 -- 172.21.15.38:0/3818166296 <== osd.7 v2:172.21.15.195:6818/1606106342 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v116'602 uv602 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa534003040 con 0x55dee91caa30 2024-09-21T03:42:26.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.309+0000 7fa53f0ccf40 1 -- 172.21.15.38:0/3818166296 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55dee92707e0 con 0x55dee91caa30 2024-09-21T03:42:26.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.311+0000 7fa539387640 1 -- 172.21.15.38:0/3818166296 <== osd.7 v2:172.21.15.195:6818/1606106342 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v116'603 uv603 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa534003040 con 0x55dee91caa30 2024-09-21T03:42:26.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.311+0000 7fa53f0ccf40 1 -- 172.21.15.38:0/3818166296 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55dee9270b80 con 0x55dee91caa30 2024-09-21T03:42:26.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.313+0000 7fa539387640 1 -- 172.21.15.38:0/3818166296 <== osd.7 v2:172.21.15.195:6818/1606106342 7 ==== osd_op_reply(7 602.00000000 [create] v116'604 uv578 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fa534003040 con 0x55dee91caa30 2024-09-21T03:42:26.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.313+0000 7fa53f0ccf40 1 -- 172.21.15.38:0/3818166296 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e116) -- 0x55dee9270b80 con 0x55dee91caa30 2024-09-21T03:42:26.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.314+0000 7fa539387640 1 -- 172.21.15.38:0/3818166296 <== osd.7 v2:172.21.15.195:6818/1606106342 8 ==== osd_op_reply(8 602.00000000 [delete] v116'605 uv605 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa534003040 con 0x55dee91caa30 2024-09-21T03:42:26.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.314+0000 7fa53f0ccf40 1 -- 172.21.15.38:0/3818166296 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55dee9270b80 con 0x55dee91caa30 2024-09-21T03:42:26.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.316+0000 7fa539387640 1 -- 172.21.15.38:0/3818166296 <== osd.7 v2:172.21.15.195:6818/1606106342 9 ==== osd_op_reply(9 602.00000000 [create] v116'606 uv606 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa534003040 con 0x55dee91caa30 2024-09-21T03:42:26.318 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-09-21T03:42:26.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.316+0000 7fa53f0ccf40 1 -- 172.21.15.38:0/3818166296 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55dee9270b80 con 0x55dee91caa30 2024-09-21T03:42:26.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.317+0000 7fa539387640 1 -- 172.21.15.38:0/3818166296 <== osd.7 v2:172.21.15.195:6818/1606106342 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v116'607 uv607 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa534003040 con 0x55dee91caa30 2024-09-21T03:42:26.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.317+0000 7fa53f0ccf40 1 -- 172.21.15.38:0/3818166296 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55dee9270b80 con 0x55dee91caa30 2024-09-21T03:42:26.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.319+0000 7fa539387640 1 -- 172.21.15.38:0/3818166296 <== osd.7 v2:172.21.15.195:6818/1606106342 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v116'608 uv608 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa534003040 con 0x55dee91caa30 2024-09-21T03:42:26.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.319+0000 7fa53f0ccf40 1 -- 172.21.15.38:0/3818166296 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55dee9270b80 con 0x55dee91caa30 2024-09-21T03:42:26.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.320+0000 7fa539387640 1 -- 172.21.15.38:0/3818166296 <== osd.7 v2:172.21.15.195:6818/1606106342 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v116'609 uv609 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa534003040 con 0x55dee91caa30 2024-09-21T03:42:26.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.320+0000 7fa53f0ccf40 1 -- 172.21.15.38:0/3818166296 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:13 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55dee9270b80 con 0x55dee91caa30 2024-09-21T03:42:26.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.322+0000 7fa539387640 1 -- 172.21.15.38:0/3818166296 <== osd.7 v2:172.21.15.195:6818/1606106342 13 ==== osd_op_reply(13 10000000000.00000000 [create] v116'610 uv585 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7fa534003040 con 0x55dee91caa30 2024-09-21T03:42:26.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.322+0000 7fa53f0ccf40 1 -- 172.21.15.38:0/3818166296 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:14 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e116) -- 0x55dee9270b80 con 0x55dee91caa30 2024-09-21T03:42:26.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.323+0000 7fa539387640 1 -- 172.21.15.38:0/3818166296 <== osd.7 v2:172.21.15.195:6818/1606106342 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v116'611 uv611 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fa534003040 con 0x55dee91caa30 2024-09-21T03:42:26.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.324+0000 7fa53f0ccf40 1 -- 172.21.15.38:0/3818166296 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:15 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55dee9270b80 con 0x55dee91caa30 2024-09-21T03:42:26.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.325+0000 7fa539387640 1 -- 172.21.15.38:0/3818166296 <== osd.7 v2:172.21.15.195:6818/1606106342 15 ==== osd_op_reply(15 10000000000.00000000 [create] v116'612 uv612 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fa534003040 con 0x55dee91caa30 2024-09-21T03:42:26.327 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-09-21T03:42:26.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.325+0000 7fa53f0ccf40 1 -- 172.21.15.38:0/3818166296 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:16 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55dee9270b80 con 0x55dee91caa30 2024-09-21T03:42:26.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.327+0000 7fa539387640 1 -- 172.21.15.38:0/3818166296 <== osd.7 v2:172.21.15.195:6818/1606106342 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v116'613 uv613 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fa534003040 con 0x55dee91caa30 2024-09-21T03:42:26.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.327+0000 7fa53f0ccf40 1 -- 172.21.15.38:0/3818166296 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:17 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55dee9270b80 con 0x55dee91caa30 2024-09-21T03:42:26.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.328+0000 7fa539387640 1 -- 172.21.15.38:0/3818166296 <== osd.7 v2:172.21.15.195:6818/1606106342 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v116'614 uv614 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fa534003040 con 0x55dee91caa30 2024-09-21T03:42:26.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.328+0000 7fa53f0ccf40 1 -- 172.21.15.38:0/3818166296 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:18 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55dee9270b80 con 0x55dee91caa30 2024-09-21T03:42:26.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.330+0000 7fa539387640 1 -- 172.21.15.38:0/3818166296 <== osd.7 v2:172.21.15.195:6818/1606106342 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v116'615 uv615 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fa534003040 con 0x55dee91caa30 2024-09-21T03:42:26.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.330+0000 7fa53f0ccf40 1 -- 172.21.15.38:0/3818166296 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:19 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=585b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55dee9270b80 con 0x55dee91caa30 2024-09-21T03:42:26.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.331+0000 7fa539387640 1 -- 172.21.15.38:0/3818166296 <== osd.7 v2:172.21.15.195:6818/1606106342 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v116'616 uv616 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fa534003040 con 0x55dee91caa30 2024-09-21T03:42:26.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.331+0000 7fa53f0ccf40 1 -- 172.21.15.38:0/3818166296 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:20 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55dee9270b80 con 0x55dee91caa30 2024-09-21T03:42:26.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.333+0000 7fa539387640 1 -- 172.21.15.38:0/3818166296 <== osd.7 v2:172.21.15.195:6818/1606106342 20 ==== osd_op_reply(20 600.00000000 [create] v116'617 uv591 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fa534003040 con 0x55dee91caa30 2024-09-21T03:42:26.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.333+0000 7fa53f0ccf40 1 -- 172.21.15.38:0/3818166296 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:21 13.10 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e116) -- 0x55dee9270b80 con 0x55dee91caa30 2024-09-21T03:42:26.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.334+0000 7fa539387640 1 -- 172.21.15.38:0/3818166296 <== osd.7 v2:172.21.15.195:6818/1606106342 21 ==== osd_op_reply(21 600.00000000 [delete] v116'618 uv618 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa534003040 con 0x55dee91caa30 2024-09-21T03:42:26.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.334+0000 7fa53f0ccf40 1 -- 172.21.15.38:0/3818166296 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:22 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55dee9270b80 con 0x55dee91caa30 2024-09-21T03:42:26.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.336+0000 7fa539387640 1 -- 172.21.15.38:0/3818166296 <== osd.7 v2:172.21.15.195:6818/1606106342 22 ==== osd_op_reply(22 600.00000000 [create] v116'619 uv619 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa534003040 con 0x55dee91caa30 2024-09-21T03:42:26.338 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-09-21T03:42:26.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.336+0000 7fa53f0ccf40 1 -- 172.21.15.38:0/3818166296 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:23 13.10 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55dee9270b80 con 0x55dee91caa30 2024-09-21T03:42:26.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.337+0000 7fa539387640 1 -- 172.21.15.38:0/3818166296 <== osd.7 v2:172.21.15.195:6818/1606106342 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v116'620 uv620 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa534003040 con 0x55dee91caa30 2024-09-21T03:42:26.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.337+0000 7fa53f0ccf40 1 -- 172.21.15.38:0/3818166296 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:24 13.10 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55dee9270b80 con 0x55dee91caa30 2024-09-21T03:42:26.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.338+0000 7fa539387640 1 -- 172.21.15.38:0/3818166296 <== osd.7 v2:172.21.15.195:6818/1606106342 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v116'621 uv621 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa534003040 con 0x55dee91caa30 2024-09-21T03:42:26.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.338+0000 7fa53f0ccf40 1 -- 172.21.15.38:0/3818166296 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:25 13.10 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55dee9270b80 con 0x55dee91caa30 2024-09-21T03:42:26.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.340+0000 7fa539387640 1 -- 172.21.15.38:0/3818166296 <== osd.7 v2:172.21.15.195:6818/1606106342 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v116'622 uv622 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa534003040 con 0x55dee91caa30 2024-09-21T03:42:26.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.340+0000 7fa53f0ccf40 1 -- 172.21.15.38:0/3818166296 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:26 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55dee9270b80 con 0x55dee91caa30 2024-09-21T03:42:26.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.341+0000 7fa539387640 1 -- 172.21.15.38:0/3818166296 <== osd.7 v2:172.21.15.195:6818/1606106342 26 ==== osd_op_reply(26 603.00000000 [create] v116'623 uv597 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fa534003040 con 0x55dee91caa30 2024-09-21T03:42:26.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.341+0000 7fa53f0ccf40 1 -- 172.21.15.38:0/3818166296 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:27 13.10 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e116) -- 0x55dee9270b80 con 0x55dee91caa30 2024-09-21T03:42:26.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.343+0000 7fa539387640 1 -- 172.21.15.38:0/3818166296 <== osd.7 v2:172.21.15.195:6818/1606106342 27 ==== osd_op_reply(27 603.00000000 [delete] v116'624 uv624 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa534003040 con 0x55dee91caa30 2024-09-21T03:42:26.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.343+0000 7fa53f0ccf40 1 -- 172.21.15.38:0/3818166296 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:28 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55dee9270b80 con 0x55dee91caa30 2024-09-21T03:42:26.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.344+0000 7fa539387640 1 -- 172.21.15.38:0/3818166296 <== osd.7 v2:172.21.15.195:6818/1606106342 28 ==== osd_op_reply(28 603.00000000 [create] v116'625 uv625 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa534003040 con 0x55dee91caa30 2024-09-21T03:42:26.346 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-09-21T03:42:26.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.344+0000 7fa53f0ccf40 1 -- 172.21.15.38:0/3818166296 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:29 13.10 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55dee9270b80 con 0x55dee91caa30 2024-09-21T03:42:26.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.346+0000 7fa539387640 1 -- 172.21.15.38:0/3818166296 <== osd.7 v2:172.21.15.195:6818/1606106342 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v116'626 uv626 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa534003040 con 0x55dee91caa30 2024-09-21T03:42:26.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.346+0000 7fa53f0ccf40 1 -- 172.21.15.38:0/3818166296 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:30 13.10 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55dee9270b80 con 0x55dee91caa30 2024-09-21T03:42:26.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.348+0000 7fa539387640 1 -- 172.21.15.38:0/3818166296 <== osd.7 v2:172.21.15.195:6818/1606106342 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v116'627 uv627 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa534003040 con 0x55dee91caa30 2024-09-21T03:42:26.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.348+0000 7fa53f0ccf40 1 -- 172.21.15.38:0/3818166296 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:31 13.10 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55dee9270b80 con 0x55dee91caa30 2024-09-21T03:42:26.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.349+0000 7fa539387640 1 -- 172.21.15.38:0/3818166296 <== osd.7 v2:172.21.15.195:6818/1606106342 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v116'628 uv628 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa534003040 con 0x55dee91caa30 2024-09-21T03:42:26.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.349+0000 7fa53f0ccf40 1 --2- 172.21.15.38:0/3818166296 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x55dee9271f30 0x55dee9274350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:26.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.349+0000 7fa53f0ccf40 1 -- 172.21.15.38:0/3818166296 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55dee9270b80 con 0x55dee9271f30 2024-09-21T03:42:26.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.349+0000 7fa538b86640 1 --2- 172.21.15.38:0/3818166296 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x55dee9271f30 0x55dee9274350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:26.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.350+0000 7fa538b86640 1 --2- 172.21.15.38:0/3818166296 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x55dee9271f30 0x55dee9274350 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:26.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.351+0000 7fa538b86640 1 -- 172.21.15.38:0/3818166296 <== osd.6 v2:172.21.15.195:6826/4055825675 1 ==== osd_op_reply(32 1.00000000.inode [create] v116'441 uv424 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7fa52805e170 con 0x55dee9271f30 2024-09-21T03:42:26.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.351+0000 7fa53f0ccf40 1 -- 172.21.15.38:0/3818166296 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e116) -- 0x55dee9274890 con 0x55dee9271f30 2024-09-21T03:42:26.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.353+0000 7fa538b86640 1 -- 172.21.15.38:0/3818166296 <== osd.6 v2:172.21.15.195:6826/4055825675 2 ==== osd_op_reply(33 1.00000000.inode [delete] v116'442 uv442 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fa52805e170 con 0x55dee9271f30 2024-09-21T03:42:26.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.353+0000 7fa53f0ccf40 1 -- 172.21.15.38:0/3818166296 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55dee9274c30 con 0x55dee9271f30 2024-09-21T03:42:26.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.354+0000 7fa538b86640 1 -- 172.21.15.38:0/3818166296 <== osd.6 v2:172.21.15.195:6826/4055825675 3 ==== osd_op_reply(34 1.00000000.inode [create] v116'443 uv443 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fa52805e170 con 0x55dee9271f30 2024-09-21T03:42:26.356 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-09-21T03:42:26.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.354+0000 7fa53f0ccf40 1 -- 172.21.15.38:0/3818166296 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55dee9274fd0 con 0x55dee9271f30 2024-09-21T03:42:26.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.355+0000 7fa538b86640 1 -- 172.21.15.38:0/3818166296 <== osd.6 v2:172.21.15.195:6826/4055825675 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v116'444 uv444 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fa52805e170 con 0x55dee9271f30 2024-09-21T03:42:26.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.355+0000 7fa53f0ccf40 1 -- 172.21.15.38:0/3818166296 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55dee9275370 con 0x55dee9271f30 2024-09-21T03:42:26.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.357+0000 7fa538b86640 1 -- 172.21.15.38:0/3818166296 <== osd.6 v2:172.21.15.195:6826/4055825675 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v116'445 uv445 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fa52805e170 con 0x55dee9271f30 2024-09-21T03:42:26.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.357+0000 7fa53f0ccf40 1 -- 172.21.15.38:0/3818166296 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55dee9275710 con 0x55dee9271f30 2024-09-21T03:42:26.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.358+0000 7fa538b86640 1 -- 172.21.15.38:0/3818166296 <== osd.6 v2:172.21.15.195:6826/4055825675 6 ==== osd_op_reply(37 200.00000000 [create] v116'446 uv429 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fa52805e170 con 0x55dee9271f30 2024-09-21T03:42:26.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.358+0000 7fa53f0ccf40 1 -- 172.21.15.38:0/3818166296 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e116) -- 0x55dee9275ab0 con 0x55dee9271f30 2024-09-21T03:42:26.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.360+0000 7fa538b86640 1 -- 172.21.15.38:0/3818166296 <== osd.6 v2:172.21.15.195:6826/4055825675 7 ==== osd_op_reply(38 200.00000000 [delete] v116'447 uv447 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa52805e170 con 0x55dee9271f30 2024-09-21T03:42:26.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.360+0000 7fa53f0ccf40 1 -- 172.21.15.38:0/3818166296 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55dee9275e50 con 0x55dee9271f30 2024-09-21T03:42:26.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.361+0000 7fa538b86640 1 -- 172.21.15.38:0/3818166296 <== osd.6 v2:172.21.15.195:6826/4055825675 8 ==== osd_op_reply(39 200.00000000 [create] v116'448 uv448 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa52805e170 con 0x55dee9271f30 2024-09-21T03:42:26.363 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-09-21T03:42:26.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.361+0000 7fa53f0ccf40 1 -- 172.21.15.38:0/3818166296 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55dee9275e50 con 0x55dee9271f30 2024-09-21T03:42:26.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.362+0000 7fa538b86640 1 -- 172.21.15.38:0/3818166296 <== osd.6 v2:172.21.15.195:6826/4055825675 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v116'449 uv449 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa52805e170 con 0x55dee9271f30 2024-09-21T03:42:26.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.362+0000 7fa53f0ccf40 1 -- 172.21.15.38:0/3818166296 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55dee9275e50 con 0x55dee9271f30 2024-09-21T03:42:26.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.363+0000 7fa538b86640 1 -- 172.21.15.38:0/3818166296 <== osd.6 v2:172.21.15.195:6826/4055825675 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v116'450 uv450 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa52805e170 con 0x55dee9271f30 2024-09-21T03:42:26.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.364+0000 7fa53f0ccf40 1 -- 172.21.15.38:0/3818166296 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:42 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55dee9275e50 con 0x55dee9271f30 2024-09-21T03:42:26.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.365+0000 7fa538b86640 1 -- 172.21.15.38:0/3818166296 <== osd.6 v2:172.21.15.195:6826/4055825675 11 ==== osd_op_reply(42 200.00000001 [create] v116'451 uv434 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fa52805e170 con 0x55dee9271f30 2024-09-21T03:42:26.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.365+0000 7fa53f0ccf40 1 -- 172.21.15.38:0/3818166296 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:43 13.14 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e116) -- 0x55dee9275e50 con 0x55dee9271f30 2024-09-21T03:42:26.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.366+0000 7fa538b86640 1 -- 172.21.15.38:0/3818166296 <== osd.6 v2:172.21.15.195:6826/4055825675 12 ==== osd_op_reply(43 200.00000001 [delete] v116'452 uv452 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa52805e170 con 0x55dee9271f30 2024-09-21T03:42:26.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.366+0000 7fa53f0ccf40 1 -- 172.21.15.38:0/3818166296 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:44 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55dee9275e50 con 0x55dee9271f30 2024-09-21T03:42:26.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.367+0000 7fa538b86640 1 -- 172.21.15.38:0/3818166296 <== osd.6 v2:172.21.15.195:6826/4055825675 13 ==== osd_op_reply(44 200.00000001 [create] v116'453 uv453 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa52805e170 con 0x55dee9271f30 2024-09-21T03:42:26.369 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-09-21T03:42:26.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.367+0000 7fa53f0ccf40 1 -- 172.21.15.38:0/3818166296 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:45 13.14 13:2e2fa760:::200.00000001:head [write 0~17490 in=17490b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55dee927a2d0 con 0x55dee9271f30 2024-09-21T03:42:26.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.369+0000 7fa538b86640 1 -- 172.21.15.38:0/3818166296 <== osd.6 v2:172.21.15.195:6826/4055825675 14 ==== osd_op_reply(45 200.00000001 [write 0~17490] v116'454 uv454 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa52805e170 con 0x55dee9271f30 2024-09-21T03:42:26.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.369+0000 7fa53f0ccf40 1 -- 172.21.15.38:0/3818166296 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:46 13.14 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55dee927a2d0 con 0x55dee9271f30 2024-09-21T03:42:26.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.370+0000 7fa538b86640 1 -- 172.21.15.38:0/3818166296 <== osd.6 v2:172.21.15.195:6826/4055825675 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v116'455 uv455 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa52805e170 con 0x55dee9271f30 2024-09-21T03:42:26.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.370+0000 7fa53f0ccf40 1 -- 172.21.15.38:0/3818166296 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:47 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55dee9275e50 con 0x55dee9271f30 2024-09-21T03:42:26.374 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.372+0000 7fa538b86640 1 -- 172.21.15.38:0/3818166296 <== osd.6 v2:172.21.15.195:6826/4055825675 16 ==== osd_op_reply(47 606.00000000 [create] v116'456 uv440 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fa52805e170 con 0x55dee9271f30 2024-09-21T03:42:26.374 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.372+0000 7fa53f0ccf40 1 -- 172.21.15.38:0/3818166296 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:48 13.14 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e116) -- 0x55dee9275e50 con 0x55dee9271f30 2024-09-21T03:42:26.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.373+0000 7fa538b86640 1 -- 172.21.15.38:0/3818166296 <== osd.6 v2:172.21.15.195:6826/4055825675 17 ==== osd_op_reply(48 606.00000000 [delete] v116'457 uv457 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa52805e170 con 0x55dee9271f30 2024-09-21T03:42:26.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.373+0000 7fa53f0ccf40 1 -- 172.21.15.38:0/3818166296 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:49 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55dee9275e50 con 0x55dee9271f30 2024-09-21T03:42:26.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.374+0000 7fa538b86640 1 -- 172.21.15.38:0/3818166296 <== osd.6 v2:172.21.15.195:6826/4055825675 18 ==== osd_op_reply(49 606.00000000 [create] v116'458 uv458 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa52805e170 con 0x55dee9271f30 2024-09-21T03:42:26.376 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-09-21T03:42:26.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.374+0000 7fa53f0ccf40 1 -- 172.21.15.38:0/3818166296 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:50 13.14 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55dee9276e60 con 0x55dee9271f30 2024-09-21T03:42:26.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.375+0000 7fa538b86640 1 -- 172.21.15.38:0/3818166296 <== osd.6 v2:172.21.15.195:6826/4055825675 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v116'459 uv459 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa52805e170 con 0x55dee9271f30 2024-09-21T03:42:26.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.376+0000 7fa53f0ccf40 1 -- 172.21.15.38:0/3818166296 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:51 13.14 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55dee9276e60 con 0x55dee9271f30 2024-09-21T03:42:26.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.377+0000 7fa538b86640 1 -- 172.21.15.38:0/3818166296 <== osd.6 v2:172.21.15.195:6826/4055825675 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v116'460 uv460 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa52805e170 con 0x55dee9271f30 2024-09-21T03:42:26.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.377+0000 7fa53f0ccf40 1 -- 172.21.15.38:0/3818166296 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:52 13.14 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55dee9275e50 con 0x55dee9271f30 2024-09-21T03:42:26.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.378+0000 7fa538b86640 1 -- 172.21.15.38:0/3818166296 <== osd.6 v2:172.21.15.195:6826/4055825675 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v116'461 uv461 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa52805e170 con 0x55dee9271f30 2024-09-21T03:42:26.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.378+0000 7fa53f0ccf40 1 --2- 172.21.15.38:0/3818166296 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x55dee9276360 0x55dee9278780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:26.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.378+0000 7fa53f0ccf40 1 -- 172.21.15.38:0/3818166296 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:53 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55dee9278d00 con 0x55dee9276360 2024-09-21T03:42:26.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.378+0000 7fa530385640 1 --2- 172.21.15.38:0/3818166296 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x55dee9276360 0x55dee9278780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:26.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.379+0000 7fa530385640 1 --2- 172.21.15.38:0/3818166296 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x55dee9276360 0x55dee9278780 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:26.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.380+0000 7fa530385640 1 -- 172.21.15.38:0/3818166296 <== osd.0 v2:172.21.15.38:6800/2381639433 1 ==== osd_op_reply(53 607.00000000 [create] v116'117 uv116 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fa52004da70 con 0x55dee9276360 2024-09-21T03:42:26.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.380+0000 7fa53f0ccf40 1 -- 172.21.15.38:0/3818166296 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:54 13.1c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e116) -- 0x55dee92791c0 con 0x55dee9276360 2024-09-21T03:42:26.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.381+0000 7fa530385640 1 -- 172.21.15.38:0/3818166296 <== osd.0 v2:172.21.15.38:6800/2381639433 2 ==== osd_op_reply(54 607.00000000 [delete] v116'118 uv118 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa52004da70 con 0x55dee9276360 2024-09-21T03:42:26.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.381+0000 7fa53f0ccf40 1 -- 172.21.15.38:0/3818166296 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:55 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55dee92796a0 con 0x55dee9276360 2024-09-21T03:42:26.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.382+0000 7fa530385640 1 -- 172.21.15.38:0/3818166296 <== osd.0 v2:172.21.15.38:6800/2381639433 3 ==== osd_op_reply(55 607.00000000 [create] v116'119 uv119 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa52004da70 con 0x55dee9276360 2024-09-21T03:42:26.384 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-09-21T03:42:26.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.382+0000 7fa53f0ccf40 1 -- 172.21.15.38:0/3818166296 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:56 13.1c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55dee927af50 con 0x55dee9276360 2024-09-21T03:42:26.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.383+0000 7fa530385640 1 -- 172.21.15.38:0/3818166296 <== osd.0 v2:172.21.15.38:6800/2381639433 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v116'120 uv120 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa52004da70 con 0x55dee9276360 2024-09-21T03:42:26.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.383+0000 7fa53f0ccf40 1 -- 172.21.15.38:0/3818166296 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:57 13.1c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55dee927b2f0 con 0x55dee9276360 2024-09-21T03:42:26.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.384+0000 7fa530385640 1 -- 172.21.15.38:0/3818166296 <== osd.0 v2:172.21.15.38:6800/2381639433 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v116'121 uv121 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa52004da70 con 0x55dee9276360 2024-09-21T03:42:26.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.384+0000 7fa53f0ccf40 1 -- 172.21.15.38:0/3818166296 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:58 13.1c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55dee9279f40 con 0x55dee9276360 2024-09-21T03:42:26.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.385+0000 7fa530385640 1 -- 172.21.15.38:0/3818166296 <== osd.0 v2:172.21.15.38:6800/2381639433 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v116'122 uv122 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa52004da70 con 0x55dee9276360 2024-09-21T03:42:26.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.385+0000 7fa53f0ccf40 1 --2- 172.21.15.38:0/3818166296 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x55dee927a840 0x55dee927d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:26.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.385+0000 7fa53f0ccf40 1 -- 172.21.15.38:0/3818166296 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:59 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55dee927dbe0 con 0x55dee927a840 2024-09-21T03:42:26.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.386+0000 7fa539387640 1 --2- 172.21.15.38:0/3818166296 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x55dee927a840 0x55dee927d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:26.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.386+0000 7fa539387640 1 --2- 172.21.15.38:0/3818166296 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x55dee927a840 0x55dee927d6a0 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:26.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.387+0000 7fa539387640 1 -- 172.21.15.38:0/3818166296 <== osd.3 v2:172.21.15.38:6808/1286810294 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v116'86 uv85 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7fa534003040 con 0x55dee927a840 2024-09-21T03:42:26.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.387+0000 7fa53f0ccf40 1 -- 172.21.15.38:0/3818166296 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:60 13.6 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e116) -- 0x55dee927dbe0 con 0x55dee927a840 2024-09-21T03:42:26.390 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.388+0000 7fa539387640 1 -- 172.21.15.38:0/3818166296 <== osd.3 v2:172.21.15.38:6808/1286810294 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v116'87 uv87 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fa534003040 con 0x55dee927a840 2024-09-21T03:42:26.390 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.388+0000 7fa53f0ccf40 1 -- 172.21.15.38:0/3818166296 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:61 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55dee927dfd0 con 0x55dee927a840 2024-09-21T03:42:26.391 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.389+0000 7fa539387640 1 -- 172.21.15.38:0/3818166296 <== osd.3 v2:172.21.15.38:6808/1286810294 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v116'88 uv88 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fa534003040 con 0x55dee927a840 2024-09-21T03:42:26.391 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-09-21T03:42:26.391 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.389+0000 7fa53f0ccf40 1 -- 172.21.15.38:0/3818166296 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:62 13.6 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55dee927dfd0 con 0x55dee927a840 2024-09-21T03:42:26.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.390+0000 7fa539387640 1 -- 172.21.15.38:0/3818166296 <== osd.3 v2:172.21.15.38:6808/1286810294 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v116'89 uv89 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fa534003040 con 0x55dee927a840 2024-09-21T03:42:26.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.390+0000 7fa53f0ccf40 1 -- 172.21.15.38:0/3818166296 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55dee927dfd0 con 0x55dee9276360 2024-09-21T03:42:26.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.391+0000 7fa530385640 1 -- 172.21.15.38:0/3818166296 <== osd.0 v2:172.21.15.38:6800/2381639433 7 ==== osd_op_reply(63 608.00000000 [create] v116'117 uv116 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fa52004da70 con 0x55dee9276360 2024-09-21T03:42:26.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.391+0000 7fa53f0ccf40 1 -- 172.21.15.38:0/3818166296 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e116) -- 0x55dee927e370 con 0x55dee9276360 2024-09-21T03:42:26.394 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.393+0000 7fa530385640 1 -- 172.21.15.38:0/3818166296 <== osd.0 v2:172.21.15.38:6800/2381639433 8 ==== osd_op_reply(64 608.00000000 [delete] v116'118 uv118 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa52004da70 con 0x55dee9276360 2024-09-21T03:42:26.394 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.393+0000 7fa53f0ccf40 1 -- 172.21.15.38:0/3818166296 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55dee927e370 con 0x55dee9276360 2024-09-21T03:42:26.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.394+0000 7fa530385640 1 -- 172.21.15.38:0/3818166296 <== osd.0 v2:172.21.15.38:6800/2381639433 9 ==== osd_op_reply(65 608.00000000 [create] v116'119 uv119 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa52004da70 con 0x55dee9276360 2024-09-21T03:42:26.396 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-09-21T03:42:26.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.394+0000 7fa53f0ccf40 1 -- 172.21.15.38:0/3818166296 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55dee927f380 con 0x55dee9276360 2024-09-21T03:42:26.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.395+0000 7fa530385640 1 -- 172.21.15.38:0/3818166296 <== osd.0 v2:172.21.15.38:6800/2381639433 10 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v116'120 uv120 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa52004da70 con 0x55dee9276360 2024-09-21T03:42:26.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.395+0000 7fa53f0ccf40 1 -- 172.21.15.38:0/3818166296 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55dee927f380 con 0x55dee9276360 2024-09-21T03:42:26.398 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.396+0000 7fa530385640 1 -- 172.21.15.38:0/3818166296 <== osd.0 v2:172.21.15.38:6800/2381639433 11 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v116'121 uv121 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa52004da70 con 0x55dee9276360 2024-09-21T03:42:26.398 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.396+0000 7fa53f0ccf40 1 -- 172.21.15.38:0/3818166296 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55dee927e370 con 0x55dee9276360 2024-09-21T03:42:26.399 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.397+0000 7fa530385640 1 -- 172.21.15.38:0/3818166296 <== osd.0 v2:172.21.15.38:6800/2381639433 12 ==== osd_op_reply(68 608.00000000 [omap-set-header] v116'122 uv122 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa52004da70 con 0x55dee9276360 2024-09-21T03:42:26.399 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.397+0000 7fa53f0ccf40 1 -- 172.21.15.38:0/3818166296 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55dee927e370 con 0x55dee927a840 2024-09-21T03:42:26.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.399+0000 7fa539387640 1 -- 172.21.15.38:0/3818166296 <== osd.3 v2:172.21.15.38:6808/1286810294 5 ==== osd_op_reply(69 604.00000000 [create] v116'216 uv210 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fa534003040 con 0x55dee927a840 2024-09-21T03:42:26.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.399+0000 7fa53f0ccf40 1 -- 172.21.15.38:0/3818166296 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e116) -- 0x55dee927e370 con 0x55dee927a840 2024-09-21T03:42:26.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.400+0000 7fa539387640 1 -- 172.21.15.38:0/3818166296 <== osd.3 v2:172.21.15.38:6808/1286810294 6 ==== osd_op_reply(70 604.00000000 [delete] v116'217 uv217 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa534003040 con 0x55dee927a840 2024-09-21T03:42:26.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.400+0000 7fa53f0ccf40 1 -- 172.21.15.38:0/3818166296 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55dee927e370 con 0x55dee927a840 2024-09-21T03:42:26.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.401+0000 7fa539387640 1 -- 172.21.15.38:0/3818166296 <== osd.3 v2:172.21.15.38:6808/1286810294 7 ==== osd_op_reply(71 604.00000000 [create] v116'218 uv218 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa534003040 con 0x55dee927a840 2024-09-21T03:42:26.403 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-09-21T03:42:26.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.401+0000 7fa53f0ccf40 1 -- 172.21.15.38:0/3818166296 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55dee927e370 con 0x55dee927a840 2024-09-21T03:42:26.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.402+0000 7fa539387640 1 -- 172.21.15.38:0/3818166296 <== osd.3 v2:172.21.15.38:6808/1286810294 8 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v116'219 uv219 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa534003040 con 0x55dee927a840 2024-09-21T03:42:26.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.402+0000 7fa53f0ccf40 1 -- 172.21.15.38:0/3818166296 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55dee927e370 con 0x55dee927a840 2024-09-21T03:42:26.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.403+0000 7fa539387640 1 -- 172.21.15.38:0/3818166296 <== osd.3 v2:172.21.15.38:6808/1286810294 9 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v116'220 uv220 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa534003040 con 0x55dee927a840 2024-09-21T03:42:26.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.403+0000 7fa53f0ccf40 1 -- 172.21.15.38:0/3818166296 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55dee927e370 con 0x55dee927a840 2024-09-21T03:42:26.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.404+0000 7fa539387640 1 -- 172.21.15.38:0/3818166296 <== osd.3 v2:172.21.15.38:6808/1286810294 10 ==== osd_op_reply(74 604.00000000 [omap-set-header] v116'221 uv221 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa534003040 con 0x55dee927a840 2024-09-21T03:42:26.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.404+0000 7fa53f0ccf40 1 -- 172.21.15.38:0/3818166296 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55dee927e370 con 0x55dee927a840 2024-09-21T03:42:26.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.405+0000 7fa539387640 1 -- 172.21.15.38:0/3818166296 <== osd.3 v2:172.21.15.38:6808/1286810294 11 ==== osd_op_reply(75 500.00000000 [create] v116'222 uv215 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fa534003040 con 0x55dee927a840 2024-09-21T03:42:26.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.405+0000 7fa53f0ccf40 1 -- 172.21.15.38:0/3818166296 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e116) -- 0x55dee927e370 con 0x55dee927a840 2024-09-21T03:42:26.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.406+0000 7fa539387640 1 -- 172.21.15.38:0/3818166296 <== osd.3 v2:172.21.15.38:6808/1286810294 12 ==== osd_op_reply(76 500.00000000 [delete] v116'223 uv223 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa534003040 con 0x55dee927a840 2024-09-21T03:42:26.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.406+0000 7fa53f0ccf40 1 -- 172.21.15.38:0/3818166296 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55dee927e370 con 0x55dee927a840 2024-09-21T03:42:26.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.407+0000 7fa539387640 1 -- 172.21.15.38:0/3818166296 <== osd.3 v2:172.21.15.38:6808/1286810294 13 ==== osd_op_reply(77 500.00000000 [create] v116'224 uv224 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa534003040 con 0x55dee927a840 2024-09-21T03:42:26.409 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-09-21T03:42:26.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.407+0000 7fa53f0ccf40 1 -- 172.21.15.38:0/3818166296 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55dee927e370 con 0x55dee927a840 2024-09-21T03:42:26.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.408+0000 7fa539387640 1 -- 172.21.15.38:0/3818166296 <== osd.3 v2:172.21.15.38:6808/1286810294 14 ==== osd_op_reply(78 500.00000000 [write 0~90] v116'225 uv225 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa534003040 con 0x55dee927a840 2024-09-21T03:42:26.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.408+0000 7fa53f0ccf40 1 -- 172.21.15.38:0/3818166296 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55dee927e370 con 0x55dee927a840 2024-09-21T03:42:26.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.409+0000 7fa539387640 1 -- 172.21.15.38:0/3818166296 <== osd.3 v2:172.21.15.38:6808/1286810294 15 ==== osd_op_reply(79 500.00000000 [omap-set-header] v116'226 uv226 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa534003040 con 0x55dee927a840 2024-09-21T03:42:26.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.409+0000 7fa53f0ccf40 1 -- 172.21.15.38:0/3818166296 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:80 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55dee927e370 con 0x55dee9276360 2024-09-21T03:42:26.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.410+0000 7fa530385640 1 -- 172.21.15.38:0/3818166296 <== osd.0 v2:172.21.15.38:6800/2381639433 13 ==== osd_op_reply(80 mds_snaptable [create] v116'97 uv96 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7fa52004da70 con 0x55dee9276360 2024-09-21T03:42:26.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.410+0000 7fa53f0ccf40 1 -- 172.21.15.38:0/3818166296 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:81 13.d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e116) -- 0x55dee927e370 con 0x55dee9276360 2024-09-21T03:42:26.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.411+0000 7fa530385640 1 -- 172.21.15.38:0/3818166296 <== osd.0 v2:172.21.15.38:6800/2381639433 14 ==== osd_op_reply(81 mds_snaptable [delete] v116'98 uv98 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fa52004da70 con 0x55dee9276360 2024-09-21T03:42:26.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.411+0000 7fa53f0ccf40 1 -- 172.21.15.38:0/3818166296 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:82 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55dee927e370 con 0x55dee9276360 2024-09-21T03:42:26.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.412+0000 7fa530385640 1 -- 172.21.15.38:0/3818166296 <== osd.0 v2:172.21.15.38:6800/2381639433 15 ==== osd_op_reply(82 mds_snaptable [create] v116'99 uv99 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fa52004da70 con 0x55dee9276360 2024-09-21T03:42:26.414 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-09-21T03:42:26.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.412+0000 7fa53f0ccf40 1 -- 172.21.15.38:0/3818166296 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:83 13.d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55dee927e370 con 0x55dee9276360 2024-09-21T03:42:26.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.413+0000 7fa530385640 1 -- 172.21.15.38:0/3818166296 <== osd.0 v2:172.21.15.38:6800/2381639433 16 ==== osd_op_reply(83 mds_snaptable [write 0~70] v116'100 uv100 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fa52004da70 con 0x55dee9276360 2024-09-21T03:42:26.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.413+0000 7fa53f0ccf40 1 -- 172.21.15.38:0/3818166296 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:84 13.d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55dee927e370 con 0x55dee9276360 2024-09-21T03:42:26.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.414+0000 7fa530385640 1 -- 172.21.15.38:0/3818166296 <== osd.0 v2:172.21.15.38:6800/2381639433 17 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v116'101 uv101 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fa52004da70 con 0x55dee9276360 2024-09-21T03:42:26.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.414+0000 7fa53f0ccf40 1 --2- 172.21.15.38:0/3818166296 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x55dee927ed10 0x55dee9281130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:26.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.414+0000 7fa53f0ccf40 1 -- 172.21.15.38:0/3818166296 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55dee9281670 con 0x55dee927ed10 2024-09-21T03:42:26.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.414+0000 7fa538b86640 1 --2- 172.21.15.38:0/3818166296 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x55dee927ed10 0x55dee9281130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:26.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.415+0000 7fa538b86640 1 --2- 172.21.15.38:0/3818166296 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x55dee927ed10 0x55dee9281130 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:26.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.416+0000 7fa538b86640 1 -- 172.21.15.38:0/3818166296 <== osd.1 v2:172.21.15.38:6816/1491818732 1 ==== osd_op_reply(85 605.00000000 [create] v116'117 uv116 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fa52805e170 con 0x55dee927ed10 2024-09-21T03:42:26.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.416+0000 7fa53f0ccf40 1 -- 172.21.15.38:0/3818166296 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e116) -- 0x55dee92816f0 con 0x55dee927ed10 2024-09-21T03:42:26.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.417+0000 7fa538b86640 1 -- 172.21.15.38:0/3818166296 <== osd.1 v2:172.21.15.38:6816/1491818732 2 ==== osd_op_reply(86 605.00000000 [delete] v116'118 uv118 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa52805e170 con 0x55dee927ed10 2024-09-21T03:42:26.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.417+0000 7fa53f0ccf40 1 -- 172.21.15.38:0/3818166296 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55dee9281b10 con 0x55dee927ed10 2024-09-21T03:42:26.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.418+0000 7fa538b86640 1 -- 172.21.15.38:0/3818166296 <== osd.1 v2:172.21.15.38:6816/1491818732 3 ==== osd_op_reply(87 605.00000000 [create] v116'119 uv119 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa52805e170 con 0x55dee927ed10 2024-09-21T03:42:26.420 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-09-21T03:42:26.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.418+0000 7fa53f0ccf40 1 -- 172.21.15.38:0/3818166296 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55dee9282b20 con 0x55dee927ed10 2024-09-21T03:42:26.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.419+0000 7fa538b86640 1 -- 172.21.15.38:0/3818166296 <== osd.1 v2:172.21.15.38:6816/1491818732 4 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v116'120 uv120 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa52805e170 con 0x55dee927ed10 2024-09-21T03:42:26.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.419+0000 7fa53f0ccf40 1 -- 172.21.15.38:0/3818166296 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55dee9282b20 con 0x55dee927ed10 2024-09-21T03:42:26.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.421+0000 7fa538b86640 1 -- 172.21.15.38:0/3818166296 <== osd.1 v2:172.21.15.38:6816/1491818732 5 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v116'121 uv121 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa52805e170 con 0x55dee927ed10 2024-09-21T03:42:26.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.421+0000 7fa53f0ccf40 1 -- 172.21.15.38:0/3818166296 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55dee9281b10 con 0x55dee927ed10 2024-09-21T03:42:26.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.422+0000 7fa538b86640 1 -- 172.21.15.38:0/3818166296 <== osd.1 v2:172.21.15.38:6816/1491818732 6 ==== osd_op_reply(90 605.00000000 [omap-set-header] v116'122 uv122 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa52805e170 con 0x55dee927ed10 2024-09-21T03:42:26.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.422+0000 7fa53f0ccf40 1 -- 172.21.15.38:0/3818166296 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55dee9281b10 con 0x55dee927ed10 2024-09-21T03:42:26.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.423+0000 7fa538b86640 1 -- 172.21.15.38:0/3818166296 <== osd.1 v2:172.21.15.38:6816/1491818732 7 ==== osd_op_reply(91 mds0_inotable [create] v116'192 uv186 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7fa52805e170 con 0x55dee927ed10 2024-09-21T03:42:26.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.423+0000 7fa53f0ccf40 1 -- 172.21.15.38:0/3818166296 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e116) -- 0x55dee9281b10 con 0x55dee927ed10 2024-09-21T03:42:26.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.424+0000 7fa538b86640 1 -- 172.21.15.38:0/3818166296 <== osd.1 v2:172.21.15.38:6816/1491818732 8 ==== osd_op_reply(92 mds0_inotable [delete] v116'193 uv193 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fa52805e170 con 0x55dee927ed10 2024-09-21T03:42:26.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.424+0000 7fa53f0ccf40 1 -- 172.21.15.38:0/3818166296 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55dee9281b10 con 0x55dee927ed10 2024-09-21T03:42:26.426 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-09-21T03:42:26.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.425+0000 7fa538b86640 1 -- 172.21.15.38:0/3818166296 <== osd.1 v2:172.21.15.38:6816/1491818732 9 ==== osd_op_reply(93 mds0_inotable [create] v116'194 uv194 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fa52805e170 con 0x55dee927ed10 2024-09-21T03:42:26.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.425+0000 7fa53f0ccf40 1 -- 172.21.15.38:0/3818166296 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:94 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55dee9281b10 con 0x55dee927ed10 2024-09-21T03:42:26.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.426+0000 7fa538b86640 1 -- 172.21.15.38:0/3818166296 <== osd.1 v2:172.21.15.38:6816/1491818732 10 ==== osd_op_reply(94 mds0_inotable [write 0~34] v116'195 uv195 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fa52805e170 con 0x55dee927ed10 2024-09-21T03:42:26.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.426+0000 7fa53f0ccf40 1 -- 172.21.15.38:0/3818166296 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:95 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55dee9281b10 con 0x55dee927ed10 2024-09-21T03:42:26.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.427+0000 7fa538b86640 1 -- 172.21.15.38:0/3818166296 <== osd.1 v2:172.21.15.38:6816/1491818732 11 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v116'196 uv196 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fa52805e170 con 0x55dee927ed10 2024-09-21T03:42:26.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.427+0000 7fa53f0ccf40 1 -- 172.21.15.38:0/3818166296 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:96 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55dee9281b10 con 0x55dee927ed10 2024-09-21T03:42:26.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.428+0000 7fa538b86640 1 -- 172.21.15.38:0/3818166296 <== osd.1 v2:172.21.15.38:6816/1491818732 12 ==== osd_op_reply(96 100.00000000 [create] v116'197 uv191 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fa52805e170 con 0x55dee927ed10 2024-09-21T03:42:26.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.428+0000 7fa53f0ccf40 1 -- 172.21.15.38:0/3818166296 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:97 13.13 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e116) -- 0x55dee9281b10 con 0x55dee927ed10 2024-09-21T03:42:26.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.429+0000 7fa538b86640 1 -- 172.21.15.38:0/3818166296 <== osd.1 v2:172.21.15.38:6816/1491818732 13 ==== osd_op_reply(97 100.00000000 [delete] v116'198 uv198 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa52805e170 con 0x55dee927ed10 2024-09-21T03:42:26.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.429+0000 7fa53f0ccf40 1 -- 172.21.15.38:0/3818166296 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:98 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55dee9281b10 con 0x55dee927ed10 2024-09-21T03:42:26.431 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-09-21T03:42:26.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.430+0000 7fa538b86640 1 -- 172.21.15.38:0/3818166296 <== osd.1 v2:172.21.15.38:6816/1491818732 14 ==== osd_op_reply(98 100.00000000 [create] v116'199 uv199 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa52805e170 con 0x55dee927ed10 2024-09-21T03:42:26.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.430+0000 7fa53f0ccf40 1 -- 172.21.15.38:0/3818166296 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:99 13.13 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55dee9281b10 con 0x55dee927ed10 2024-09-21T03:42:26.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.431+0000 7fa538b86640 1 -- 172.21.15.38:0/3818166296 <== osd.1 v2:172.21.15.38:6816/1491818732 15 ==== osd_op_reply(99 100.00000000 [omap-set-header] v116'200 uv200 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa52805e170 con 0x55dee927ed10 2024-09-21T03:42:26.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.431+0000 7fa53f0ccf40 1 -- 172.21.15.38:0/3818166296 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:100 13.13 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55dee9284920 con 0x55dee927ed10 2024-09-21T03:42:26.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.432+0000 7fa538b86640 1 -- 172.21.15.38:0/3818166296 <== osd.1 v2:172.21.15.38:6816/1491818732 16 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v116'201 uv201 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa52805e170 con 0x55dee927ed10 2024-09-21T03:42:26.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.432+0000 7fa53f0ccf40 1 -- 172.21.15.38:0/3818166296 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55dee9281fc0 con 0x55dee927a840 2024-09-21T03:42:26.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.433+0000 7fa539387640 1 -- 172.21.15.38:0/3818166296 <== osd.3 v2:172.21.15.38:6808/1286810294 16 ==== osd_op_reply(101 mds0_sessionmap [create] v116'87 uv86 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7fa534003040 con 0x55dee927a840 2024-09-21T03:42:26.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.433+0000 7fa53f0ccf40 1 -- 172.21.15.38:0/3818166296 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e116) -- 0x55dee9281fc0 con 0x55dee927a840 2024-09-21T03:42:26.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.434+0000 7fa539387640 1 -- 172.21.15.38:0/3818166296 <== osd.3 v2:172.21.15.38:6808/1286810294 17 ==== osd_op_reply(102 mds0_sessionmap [delete] v116'88 uv88 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fa534003040 con 0x55dee927a840 2024-09-21T03:42:26.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.434+0000 7fa53f0ccf40 1 -- 172.21.15.38:0/3818166296 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:103 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55dee9281fc0 con 0x55dee927a840 2024-09-21T03:42:26.436 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-09-21T03:42:26.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.434+0000 7fa539387640 1 -- 172.21.15.38:0/3818166296 <== osd.3 v2:172.21.15.38:6808/1286810294 18 ==== osd_op_reply(103 mds0_sessionmap [create] v116'89 uv89 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fa534003040 con 0x55dee927a840 2024-09-21T03:42:26.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.435+0000 7fa53f0ccf40 1 -- 172.21.15.38:0/3818166296 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:104 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55dee9281fc0 con 0x55dee927a840 2024-09-21T03:42:26.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.435+0000 7fa539387640 1 -- 172.21.15.38:0/3818166296 <== osd.3 v2:172.21.15.38:6808/1286810294 19 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v116'90 uv90 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fa534003040 con 0x55dee927a840 2024-09-21T03:42:26.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.436+0000 7fa53f0ccf40 1 -- 172.21.15.38:0/3818166296 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55dee9281fc0 con 0x55dee927a840 2024-09-21T03:42:26.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.436+0000 7fa539387640 1 -- 172.21.15.38:0/3818166296 <== osd.3 v2:172.21.15.38:6808/1286810294 20 ==== osd_op_reply(105 609.00000000 [create] v116'117 uv116 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fa534003040 con 0x55dee927a840 2024-09-21T03:42:26.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.437+0000 7fa53f0ccf40 1 -- 172.21.15.38:0/3818166296 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e116) -- 0x55dee9281fc0 con 0x55dee927a840 2024-09-21T03:42:26.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.437+0000 7fa539387640 1 -- 172.21.15.38:0/3818166296 <== osd.3 v2:172.21.15.38:6808/1286810294 21 ==== osd_op_reply(106 609.00000000 [delete] v116'118 uv118 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa534003040 con 0x55dee927a840 2024-09-21T03:42:26.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.438+0000 7fa53f0ccf40 1 -- 172.21.15.38:0/3818166296 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55dee9281fc0 con 0x55dee927a840 2024-09-21T03:42:26.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.439+0000 7fa539387640 1 -- 172.21.15.38:0/3818166296 <== osd.3 v2:172.21.15.38:6808/1286810294 22 ==== osd_op_reply(107 609.00000000 [create] v116'119 uv119 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa534003040 con 0x55dee927a840 2024-09-21T03:42:26.440 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-09-21T03:42:26.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.439+0000 7fa53f0ccf40 1 -- 172.21.15.38:0/3818166296 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55dee9282fd0 con 0x55dee927a840 2024-09-21T03:42:26.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.440+0000 7fa539387640 1 -- 172.21.15.38:0/3818166296 <== osd.3 v2:172.21.15.38:6808/1286810294 23 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v116'120 uv120 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa534003040 con 0x55dee927a840 2024-09-21T03:42:26.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.440+0000 7fa53f0ccf40 1 -- 172.21.15.38:0/3818166296 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:109 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55dee9282fd0 con 0x55dee927a840 2024-09-21T03:42:26.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.440+0000 7fa539387640 1 -- 172.21.15.38:0/3818166296 <== osd.3 v2:172.21.15.38:6808/1286810294 24 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v116'121 uv121 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa534003040 con 0x55dee927a840 2024-09-21T03:42:26.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.441+0000 7fa53f0ccf40 1 -- 172.21.15.38:0/3818166296 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:110 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55dee9281fc0 con 0x55dee927a840 2024-09-21T03:42:26.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.442+0000 7fa539387640 1 -- 172.21.15.38:0/3818166296 <== osd.3 v2:172.21.15.38:6808/1286810294 25 ==== osd_op_reply(110 609.00000000 [omap-set-header] v116'122 uv122 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa534003040 con 0x55dee927a840 2024-09-21T03:42:26.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.442+0000 7fa53f0ccf40 1 --2- 172.21.15.38:0/3818166296 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x55dee92824d0 0x55dee92828e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:26.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.442+0000 7fa53f0ccf40 1 -- 172.21.15.38:0/3818166296 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55dee9282e20 con 0x55dee92824d0 2024-09-21T03:42:26.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.442+0000 7fa530385640 1 --2- 172.21.15.38:0/3818166296 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x55dee92824d0 0x55dee92828e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:26.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.442+0000 7fa530385640 1 --2- 172.21.15.38:0/3818166296 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x55dee92824d0 0x55dee92828e0 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:26.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.443+0000 7fa530385640 1 -- 172.21.15.38:0/3818166296 <== osd.2 v2:172.21.15.38:6824/2577858902 1 ==== osd_op_reply(111 400.00000000 [create] v116'99 uv98 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fa52004da70 con 0x55dee92824d0 2024-09-21T03:42:26.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.443+0000 7fa53f0ccf40 1 -- 172.21.15.38:0/3818166296 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e116) -- 0x55dee9282e20 con 0x55dee92824d0 2024-09-21T03:42:26.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.445+0000 7fa530385640 1 -- 172.21.15.38:0/3818166296 <== osd.2 v2:172.21.15.38:6824/2577858902 2 ==== osd_op_reply(112 400.00000000 [delete] v116'100 uv100 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa52004da70 con 0x55dee92824d0 2024-09-21T03:42:26.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.445+0000 7fa53f0ccf40 1 -- 172.21.15.38:0/3818166296 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55dee9282e20 con 0x55dee92824d0 2024-09-21T03:42:26.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.446+0000 7fa530385640 1 -- 172.21.15.38:0/3818166296 <== osd.2 v2:172.21.15.38:6824/2577858902 3 ==== osd_op_reply(113 400.00000000 [create] v116'101 uv101 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa52004da70 con 0x55dee92824d0 2024-09-21T03:42:26.448 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-09-21T03:42:26.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.446+0000 7fa53f0ccf40 1 -- 172.21.15.38:0/3818166296 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:114 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55dee9282e20 con 0x55dee92824d0 2024-09-21T03:42:26.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.447+0000 7fa530385640 1 -- 172.21.15.38:0/3818166296 <== osd.2 v2:172.21.15.38:6824/2577858902 4 ==== osd_op_reply(114 400.00000000 [write 0~22] v116'102 uv102 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa52004da70 con 0x55dee92824d0 2024-09-21T03:42:26.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.447+0000 7fa53f0ccf40 1 -- 172.21.15.38:0/3818166296 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:115 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55dee9282e20 con 0x55dee92824d0 2024-09-21T03:42:26.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.448+0000 7fa530385640 1 -- 172.21.15.38:0/3818166296 <== osd.2 v2:172.21.15.38:6824/2577858902 5 ==== osd_op_reply(115 400.00000000 [omap-set-header] v116'103 uv103 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa52004da70 con 0x55dee92824d0 2024-09-21T03:42:26.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.448+0000 7fa53f0ccf40 1 -- 172.21.15.38:0/3818166296 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:116 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55dee9282e20 con 0x55dee927ed10 2024-09-21T03:42:26.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.449+0000 7fa538b86640 1 -- 172.21.15.38:0/3818166296 <== osd.1 v2:172.21.15.38:6816/1491818732 17 ==== osd_op_reply(116 100.00000000.inode [create] v116'241 uv233 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7fa52805e170 con 0x55dee927ed10 2024-09-21T03:42:26.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.449+0000 7fa53f0ccf40 1 -- 172.21.15.38:0/3818166296 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:117 13.1f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e116) -- 0x55dee9282e20 con 0x55dee927ed10 2024-09-21T03:42:26.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.450+0000 7fa538b86640 1 -- 172.21.15.38:0/3818166296 <== osd.1 v2:172.21.15.38:6816/1491818732 18 ==== osd_op_reply(117 100.00000000.inode [delete] v116'242 uv242 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fa52805e170 con 0x55dee927ed10 2024-09-21T03:42:26.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.451+0000 7fa53f0ccf40 1 -- 172.21.15.38:0/3818166296 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:118 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55dee9282e20 con 0x55dee927ed10 2024-09-21T03:42:26.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.452+0000 7fa538b86640 1 -- 172.21.15.38:0/3818166296 <== osd.1 v2:172.21.15.38:6816/1491818732 19 ==== osd_op_reply(118 100.00000000.inode [create] v116'243 uv243 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fa52805e170 con 0x55dee927ed10 2024-09-21T03:42:26.453 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-09-21T03:42:26.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.452+0000 7fa53f0ccf40 1 -- 172.21.15.38:0/3818166296 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:119 13.1f 13:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55dee9282e20 con 0x55dee927ed10 2024-09-21T03:42:26.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.453+0000 7fa538b86640 1 -- 172.21.15.38:0/3818166296 <== osd.1 v2:172.21.15.38:6816/1491818732 20 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v116'244 uv244 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fa52805e170 con 0x55dee927ed10 2024-09-21T03:42:26.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.453+0000 7fa53f0ccf40 1 -- 172.21.15.38:0/3818166296 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:120 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55dee9282e20 con 0x55dee927ed10 2024-09-21T03:42:26.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.454+0000 7fa538b86640 1 -- 172.21.15.38:0/3818166296 <== osd.1 v2:172.21.15.38:6816/1491818732 21 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v116'245 uv245 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fa52805e170 con 0x55dee927ed10 2024-09-21T03:42:26.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.454+0000 7fa53f0ccf40 1 -- 172.21.15.38:0/3818166296 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:121 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55dee9282e20 con 0x55dee927ed10 2024-09-21T03:42:26.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.455+0000 7fa538b86640 1 -- 172.21.15.38:0/3818166296 <== osd.1 v2:172.21.15.38:6816/1491818732 22 ==== osd_op_reply(121 1.00000000 [create] v116'246 uv240 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7fa52805e170 con 0x55dee927ed10 2024-09-21T03:42:26.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.455+0000 7fa53f0ccf40 1 -- 172.21.15.38:0/3818166296 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:122 13.1f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e116) -- 0x55dee9282e20 con 0x55dee927ed10 2024-09-21T03:42:26.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.456+0000 7fa538b86640 1 -- 172.21.15.38:0/3818166296 <== osd.1 v2:172.21.15.38:6816/1491818732 23 ==== osd_op_reply(122 1.00000000 [delete] v116'247 uv247 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fa52805e170 con 0x55dee927ed10 2024-09-21T03:42:26.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.456+0000 7fa53f0ccf40 1 -- 172.21.15.38:0/3818166296 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:123 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55dee9282e20 con 0x55dee927ed10 2024-09-21T03:42:26.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.457+0000 7fa538b86640 1 -- 172.21.15.38:0/3818166296 <== osd.1 v2:172.21.15.38:6816/1491818732 24 ==== osd_op_reply(123 1.00000000 [create] v116'248 uv248 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fa52805e170 con 0x55dee927ed10 2024-09-21T03:42:26.459 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-09-21T03:42:26.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.457+0000 7fa53f0ccf40 1 -- 172.21.15.38:0/3818166296 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:124 13.1f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55dee9282e20 con 0x55dee927ed10 2024-09-21T03:42:26.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.458+0000 7fa538b86640 1 -- 172.21.15.38:0/3818166296 <== osd.1 v2:172.21.15.38:6816/1491818732 25 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v116'249 uv249 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fa52805e170 con 0x55dee927ed10 2024-09-21T03:42:26.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.458+0000 7fa53f0ccf40 1 -- 172.21.15.38:0/3818166296 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:125 13.1f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55dee9282e20 con 0x55dee927ed10 2024-09-21T03:42:26.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.460+0000 7fa538b86640 1 -- 172.21.15.38:0/3818166296 <== osd.1 v2:172.21.15.38:6816/1491818732 26 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v116'250 uv250 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fa52805e170 con 0x55dee927ed10 2024-09-21T03:42:26.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.460+0000 7fa53f0ccf40 1 -- 172.21.15.38:0/3818166296 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:126 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55dee9282e20 con 0x55dee927ed10 2024-09-21T03:42:26.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.461+0000 7fa538b86640 1 -- 172.21.15.38:0/3818166296 <== osd.1 v2:172.21.15.38:6816/1491818732 27 ==== osd_op_reply(126 1.00000000 [omap-set-header] v116'251 uv251 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fa52805e170 con 0x55dee927ed10 2024-09-21T03:42:26.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.461+0000 7fa53f0ccf40 1 -- 172.21.15.38:0/3818166296 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:127 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=584b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55dee92833d0 con 0x55dee927ed10 2024-09-21T03:42:26.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.462+0000 7fa538b86640 1 -- 172.21.15.38:0/3818166296 <== osd.1 v2:172.21.15.38:6816/1491818732 28 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v116'252 uv252 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fa52805e170 con 0x55dee927ed10 2024-09-21T03:42:26.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.462+0000 7fa53f0ccf40 1 -- 172.21.15.38:0/3818166296 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x55dee9276360 msgr2=0x55dee9278780 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:26.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.462+0000 7fa53f0ccf40 1 --2- 172.21.15.38:0/3818166296 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x55dee9276360 0x55dee9278780 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:26.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.462+0000 7fa53f0ccf40 1 -- 172.21.15.38:0/3818166296 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x55dee927ed10 msgr2=0x55dee9281130 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:26.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.462+0000 7fa53f0ccf40 1 --2- 172.21.15.38:0/3818166296 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x55dee927ed10 0x55dee9281130 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:26.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.462+0000 7fa53f0ccf40 1 -- 172.21.15.38:0/3818166296 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x55dee92824d0 msgr2=0x55dee92828e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:26.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.462+0000 7fa53f0ccf40 1 --2- 172.21.15.38:0/3818166296 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x55dee92824d0 0x55dee92828e0 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:26.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.462+0000 7fa53f0ccf40 1 -- 172.21.15.38:0/3818166296 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x55dee927a840 msgr2=0x55dee927d6a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:26.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.462+0000 7fa53f0ccf40 1 --2- 172.21.15.38:0/3818166296 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x55dee927a840 0x55dee927d6a0 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:26.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.462+0000 7fa539387640 1 -- 172.21.15.38:0/3818166296 reap_dead start 2024-09-21T03:42:26.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.462+0000 7fa53f0ccf40 1 -- 172.21.15.38:0/3818166296 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x55dee9271f30 msgr2=0x55dee9274350 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:26.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.462+0000 7fa53f0ccf40 1 --2- 172.21.15.38:0/3818166296 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x55dee9271f30 0x55dee9274350 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:26.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.462+0000 7fa539387640 1 -- 172.21.15.38:0/3818166296 reap_dead start 2024-09-21T03:42:26.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.462+0000 7fa53f0ccf40 1 -- 172.21.15.38:0/3818166296 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x55dee91caa30 msgr2=0x55dee926f7c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:26.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.462+0000 7fa53f0ccf40 1 --2- 172.21.15.38:0/3818166296 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x55dee91caa30 0x55dee926f7c0 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:26.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.462+0000 7fa53f0ccf40 1 -- 172.21.15.38:0/3818166296 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa518077570 msgr2=0x7fa518079a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:26.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.462+0000 7fa53f0ccf40 1 --2- 172.21.15.38:0/3818166296 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa518077570 0x7fa518079a80 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fa520064000 tx=0x7fa5200092d0 comp rx=0 tx=0).stop 2024-09-21T03:42:26.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.462+0000 7fa53f0ccf40 1 -- 172.21.15.38:0/3818166296 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dee91ca110 msgr2=0x55dee91a9c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:26.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.462+0000 7fa53f0ccf40 1 --2- 172.21.15.38:0/3818166296 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dee91ca110 0x55dee91a9c40 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7fa52800a7a0 tx=0x7fa52800ac60 comp rx=0 tx=0).stop 2024-09-21T03:42:26.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.462+0000 7fa53f0ccf40 1 -- 172.21.15.38:0/3818166296 shutdown_connections 2024-09-21T03:42:26.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.462+0000 7fa53f0ccf40 1 --2- 172.21.15.38:0/3818166296 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x55dee9271f30 0x55dee9274350 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:26.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.462+0000 7fa53f0ccf40 1 --2- 172.21.15.38:0/3818166296 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x55dee91caa30 0x55dee926f7c0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:26.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.462+0000 7fa53f0ccf40 1 --2- 172.21.15.38:0/3818166296 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa518077570 0x7fa518079a80 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:26.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.462+0000 7fa53f0ccf40 1 --2- 172.21.15.38:0/3818166296 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dee91ca110 0x55dee91a9c40 unknown :-1 s=CLOSED pgs=1729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:26.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.462+0000 7fa53f0ccf40 1 -- 172.21.15.38:0/3818166296 >> 172.21.15.38:0/3818166296 conn(0x55dee9014ac0 msgr2=0x55dee91c8a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:26.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.462+0000 7fa53f0ccf40 1 -- 172.21.15.38:0/3818166296 shutdown_connections 2024-09-21T03:42:26.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.463+0000 7fa53f0ccf40 1 -- 172.21.15.38:0/3818166296 wait complete. 2024-09-21T03:42:26.468 DEBUG:teuthology.orchestra.run.smithi038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata put mds0_inotable - 2024-09-21T03:42:26.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.563+0000 7f9669ffdf40 1 Processor -- start 2024-09-21T03:42:26.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.563+0000 7f9669ffdf40 1 -- start start 2024-09-21T03:42:26.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.564+0000 7f9669ffdf40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56078b0481a0 0x56078afc0240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:26.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.564+0000 7f9669ffdf40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56078b0485b0 0x56078afc0780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:26.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.564+0000 7f9669ffdf40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56078af8f900 0x56078af8fd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:26.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.564+0000 7f9669ffdf40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x56078af40430 con 0x56078af8f900 2024-09-21T03:42:26.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.564+0000 7f9669ffdf40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x56078af4d1f0 con 0x56078b0481a0 2024-09-21T03:42:26.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.564+0000 7f9669ffdf40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x56078aee1d60 con 0x56078b0485b0 2024-09-21T03:42:26.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.564+0000 7f96642b8640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56078af8f900 0x56078af8fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:26.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.564+0000 7f96642b8640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56078af8f900 0x56078af8fd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:59852/0 (socket says 172.21.15.38:59852) 2024-09-21T03:42:26.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.564+0000 7f96642b8640 1 -- 172.21.15.38:0/3746281397 learned_addr learned my addr 172.21.15.38:0/3746281397 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:42:26.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.565+0000 7f9663ab7640 1 --2- 172.21.15.38:0/3746281397 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56078b0481a0 0x56078afc0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:26.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.565+0000 7f96632b6640 1 --2- 172.21.15.38:0/3746281397 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56078b0485b0 0x56078afc0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:26.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.565+0000 7f96642b8640 1 -- 172.21.15.38:0/3746281397 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56078b0485b0 msgr2=0x56078afc0780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:26.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.565+0000 7f96642b8640 1 --2- 172.21.15.38:0/3746281397 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56078b0485b0 0x56078afc0780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:26.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.565+0000 7f96642b8640 1 -- 172.21.15.38:0/3746281397 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56078b0481a0 msgr2=0x56078afc0240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:26.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.565+0000 7f96642b8640 1 --2- 172.21.15.38:0/3746281397 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56078b0481a0 0x56078afc0240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:26.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.565+0000 7f96642b8640 1 -- 172.21.15.38:0/3746281397 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56078b069ee0 con 0x56078af8f900 2024-09-21T03:42:26.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.565+0000 7f96642b8640 1 --2- 172.21.15.38:0/3746281397 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56078af8f900 0x56078af8fd10 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7f965c025b90 tx=0x7f965c0018b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcd4abf4697ba7a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:26.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.565+0000 7f96622b4640 1 -- 172.21.15.38:0/3746281397 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f965c009330 con 0x56078af8f900 2024-09-21T03:42:26.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.565+0000 7f96622b4640 1 -- 172.21.15.38:0/3746281397 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f965c001490 con 0x56078af8f900 2024-09-21T03:42:26.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.565+0000 7f96622b4640 1 -- 172.21.15.38:0/3746281397 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f965c007720 con 0x56078af8f900 2024-09-21T03:42:26.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.566+0000 7f9669ffdf40 1 -- 172.21.15.38:0/3746281397 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56078af8f900 msgr2=0x56078af8fd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:26.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.566+0000 7f9669ffdf40 1 --2- 172.21.15.38:0/3746281397 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56078af8f900 0x56078af8fd10 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7f965c025b90 tx=0x7f965c0018b0 comp rx=0 tx=0).stop 2024-09-21T03:42:26.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.566+0000 7f9669ffdf40 1 -- 172.21.15.38:0/3746281397 shutdown_connections 2024-09-21T03:42:26.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.566+0000 7f9669ffdf40 1 --2- 172.21.15.38:0/3746281397 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56078af8f900 0x56078af8fd10 unknown :-1 s=CLOSED pgs=1732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:26.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.566+0000 7f9669ffdf40 1 --2- 172.21.15.38:0/3746281397 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56078b0485b0 0x56078afc0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:26.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.566+0000 7f9669ffdf40 1 --2- 172.21.15.38:0/3746281397 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56078b0481a0 0x56078afc0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:26.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.566+0000 7f9669ffdf40 1 -- 172.21.15.38:0/3746281397 >> 172.21.15.38:0/3746281397 conn(0x56078aeb2ac0 msgr2=0x56078b0657d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:26.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.566+0000 7f9669ffdf40 1 -- 172.21.15.38:0/3746281397 shutdown_connections 2024-09-21T03:42:26.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.566+0000 7f9669ffdf40 1 -- 172.21.15.38:0/3746281397 wait complete. 2024-09-21T03:42:26.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.567+0000 7f9669ffdf40 1 Processor -- start 2024-09-21T03:42:26.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.567+0000 7f9669ffdf40 1 -- start start 2024-09-21T03:42:26.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.568+0000 7f9669ffdf40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56078af8f900 0x56078b047100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:26.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.568+0000 7f9669ffdf40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56078b0481a0 0x56078af8fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:26.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.568+0000 7f9669ffdf40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56078b0485b0 0x56078af90250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:26.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.568+0000 7f9669ffdf40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x56078b06a180 con 0x56078b0485b0 2024-09-21T03:42:26.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.568+0000 7f9669ffdf40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x56078b04cb70 con 0x56078af8f900 2024-09-21T03:42:26.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.568+0000 7f9669ffdf40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x56078b04a440 con 0x56078b0481a0 2024-09-21T03:42:26.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.568+0000 7f96642b8640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56078b0485b0 0x56078af90250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:26.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.568+0000 7f96642b8640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56078b0485b0 0x56078af90250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:59858/0 (socket says 172.21.15.38:59858) 2024-09-21T03:42:26.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.568+0000 7f96642b8640 1 -- 172.21.15.38:0/1603371891 learned_addr learned my addr 172.21.15.38:0/1603371891 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:42:26.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.568+0000 7f96632b6640 1 --2- 172.21.15.38:0/1603371891 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56078b0481a0 0x56078af8fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:26.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.568+0000 7f9663ab7640 1 --2- 172.21.15.38:0/1603371891 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56078af8f900 0x56078b047100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:26.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.569+0000 7f96642b8640 1 -- 172.21.15.38:0/1603371891 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56078b0481a0 msgr2=0x56078af8fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:26.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.569+0000 7f96642b8640 1 --2- 172.21.15.38:0/1603371891 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56078b0481a0 0x56078af8fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:26.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.569+0000 7f96642b8640 1 -- 172.21.15.38:0/1603371891 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56078af8f900 msgr2=0x56078b047100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:26.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.569+0000 7f96642b8640 1 --2- 172.21.15.38:0/1603371891 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56078af8f900 0x56078b047100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:26.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.569+0000 7f96642b8640 1 -- 172.21.15.38:0/1603371891 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56078afc0a40 con 0x56078b0485b0 2024-09-21T03:42:26.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.569+0000 7f9663ab7640 1 --2- 172.21.15.38:0/1603371891 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56078af8f900 0x56078b047100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:42:26.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.569+0000 7f96642b8640 1 --2- 172.21.15.38:0/1603371891 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56078b0485b0 0x56078af90250 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7f965c0098f0 tx=0x7f965c001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:26.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.569+0000 7f96612b2640 1 -- 172.21.15.38:0/1603371891 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f965c0091f0 con 0x56078b0485b0 2024-09-21T03:42:26.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.569+0000 7f96612b2640 1 -- 172.21.15.38:0/1603371891 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f965c009390 con 0x56078b0485b0 2024-09-21T03:42:26.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.569+0000 7f96612b2640 1 -- 172.21.15.38:0/1603371891 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f965c007700 con 0x56078b0485b0 2024-09-21T03:42:26.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.569+0000 7f9669ffdf40 1 -- 172.21.15.38:0/1603371891 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56078b060b60 con 0x56078b0485b0 2024-09-21T03:42:26.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.569+0000 7f9669ffdf40 1 -- 172.21.15.38:0/1603371891 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56078b061240 con 0x56078b0485b0 2024-09-21T03:42:26.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.571+0000 7f96612b2640 1 -- 172.21.15.38:0/1603371891 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f965c053040 con 0x56078b0485b0 2024-09-21T03:42:26.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.571+0000 7f96612b2640 1 --2- 172.21.15.38:0/1603371891 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9650077570 0x7f9650079a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:26.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.571+0000 7f96612b2640 1 -- 172.21.15.38:0/1603371891 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 8024+0+0 (secure 0 0 0) 0x7f965c0ddd10 con 0x56078b0485b0 2024-09-21T03:42:26.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.571+0000 7f9663ab7640 1 --2- 172.21.15.38:0/1603371891 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9650077570 0x7f9650079a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:26.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.571+0000 7f9669ffdf40 1 --2- 172.21.15.38:0/1603371891 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x56078b05e450 0x56078b104f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:26.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.571+0000 7f9669ffdf40 1 -- 172.21.15.38:0/1603371891 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:1 13.13 13:c91d4a1d:::mds0_inotable:head [writefull 0~80 in=80b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x56078b049100 con 0x56078b05e450 2024-09-21T03:42:26.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.572+0000 7f96632b6640 1 --2- 172.21.15.38:0/1603371891 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x56078b05e450 0x56078b104f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:26.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.572+0000 7f9663ab7640 1 --2- 172.21.15.38:0/1603371891 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9650077570 0x7f9650079a80 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f9654000fd0 tx=0x7f9654003040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:26.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.572+0000 7f96632b6640 1 --2- 172.21.15.38:0/1603371891 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x56078b05e450 0x56078b104f90 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:26.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.573+0000 7f96632b6640 1 -- 172.21.15.38:0/1603371891 <== osd.1 v2:172.21.15.38:6816/1491818732 1 ==== osd_op_reply(1 mds0_inotable [writefull 0~80] v116'202 uv202 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f965800ca40 con 0x56078b05e450 2024-09-21T03:42:26.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.574+0000 7f9669ffdf40 1 -- 172.21.15.38:0/1603371891 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x56078b05e450 msgr2=0x56078b104f90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:26.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.574+0000 7f9669ffdf40 1 --2- 172.21.15.38:0/1603371891 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x56078b05e450 0x56078b104f90 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:26.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.574+0000 7f9669ffdf40 1 -- 172.21.15.38:0/1603371891 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9650077570 msgr2=0x7f9650079a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:26.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.574+0000 7f9669ffdf40 1 --2- 172.21.15.38:0/1603371891 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9650077570 0x7f9650079a80 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f9654000fd0 tx=0x7f9654003040 comp rx=0 tx=0).stop 2024-09-21T03:42:26.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.574+0000 7f9669ffdf40 1 -- 172.21.15.38:0/1603371891 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56078b0485b0 msgr2=0x56078af90250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:26.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.574+0000 7f9669ffdf40 1 --2- 172.21.15.38:0/1603371891 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56078b0485b0 0x56078af90250 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7f965c0098f0 tx=0x7f965c001250 comp rx=0 tx=0).stop 2024-09-21T03:42:26.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.574+0000 7f96642b8640 1 -- 172.21.15.38:0/1603371891 reap_dead start 2024-09-21T03:42:26.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.574+0000 7f9669ffdf40 1 -- 172.21.15.38:0/1603371891 shutdown_connections 2024-09-21T03:42:26.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.574+0000 7f9669ffdf40 1 -- 172.21.15.38:0/1603371891 >> 172.21.15.38:0/1603371891 conn(0x56078aeb2ac0 msgr2=0x56078b0657b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:26.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.574+0000 7f9669ffdf40 1 -- 172.21.15.38:0/1603371891 shutdown_connections 2024-09-21T03:42:26.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.574+0000 7f9669ffdf40 1 -- 172.21.15.38:0/1603371891 wait complete. 2024-09-21T03:42:26.578 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-21T03:42:26.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.736+0000 7f5ce497b640 1 -- 172.21.15.38:0/3891128005 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5ce00ce6b0 msgr2=0x7f5ce00736e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:26.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.736+0000 7f5ce497b640 1 --2- 172.21.15.38:0/3891128005 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5ce00ce6b0 0x7f5ce00736e0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f5cc8009830 tx=0x7f5cc802a480 comp rx=0 tx=0).stop 2024-09-21T03:42:26.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.736+0000 7f5ce497b640 1 -- 172.21.15.38:0/3891128005 shutdown_connections 2024-09-21T03:42:26.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.736+0000 7f5ce497b640 1 --2- 172.21.15.38:0/3891128005 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5ce00ce6b0 0x7f5ce00736e0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:26.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.736+0000 7f5ce497b640 1 --2- 172.21.15.38:0/3891128005 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5ce015c4b0 0x7f5ce00ce170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:26.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.736+0000 7f5ce497b640 1 -- 172.21.15.38:0/3891128005 >> 172.21.15.38:0/3891128005 conn(0x7f5ce00cd460 msgr2=0x7f5ce00cd870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:26.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.736+0000 7f5ce497b640 1 -- 172.21.15.38:0/3891128005 shutdown_connections 2024-09-21T03:42:26.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.736+0000 7f5ce497b640 1 -- 172.21.15.38:0/3891128005 wait complete. 2024-09-21T03:42:26.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.738+0000 7f5ce497b640 1 Processor -- start 2024-09-21T03:42:26.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.738+0000 7f5ce497b640 1 -- start start 2024-09-21T03:42:26.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.740+0000 7f5ce497b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5ce0073c20 0x7f5ce00aa5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:26.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.740+0000 7f5ce497b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5ce00ce6b0 0x7f5ce015b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:26.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.740+0000 7f5ce497b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5ce015c4b0 0x7f5ce01e8390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:26.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.740+0000 7f5ce497b640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f5ce00b5510 con 0x7f5ce0073c20 2024-09-21T03:42:26.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.740+0000 7f5ce497b640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f5ce00b5390 con 0x7f5ce00ce6b0 2024-09-21T03:42:26.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.740+0000 7f5ce497b640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f5ce015f770 con 0x7f5ce015c4b0 2024-09-21T03:42:26.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.740+0000 7f5cde59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5ce0073c20 0x7f5ce00aa5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:26.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.740+0000 7f5cde59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5ce0073c20 0x7f5ce00aa5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:59874/0 (socket says 172.21.15.38:59874) 2024-09-21T03:42:26.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.740+0000 7f5cde59c640 1 -- 172.21.15.38:0/3979025203 learned_addr learned my addr 172.21.15.38:0/3979025203 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:42:26.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.740+0000 7f5cddd9b640 1 --2- 172.21.15.38:0/3979025203 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5ce00ce6b0 0x7f5ce015b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:26.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.740+0000 7f5cded9d640 1 --2- 172.21.15.38:0/3979025203 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5ce015c4b0 0x7f5ce01e8390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:26.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.740+0000 7f5cde59c640 1 -- 172.21.15.38:0/3979025203 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5ce015c4b0 msgr2=0x7f5ce01e8390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:26.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.740+0000 7f5cde59c640 1 --2- 172.21.15.38:0/3979025203 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5ce015c4b0 0x7f5ce01e8390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:26.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.740+0000 7f5cde59c640 1 -- 172.21.15.38:0/3979025203 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5ce00ce6b0 msgr2=0x7f5ce015b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:26.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.740+0000 7f5cde59c640 1 --2- 172.21.15.38:0/3979025203 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5ce00ce6b0 0x7f5ce015b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:26.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.740+0000 7f5cde59c640 1 -- 172.21.15.38:0/3979025203 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ce00aab20 con 0x7f5ce0073c20 2024-09-21T03:42:26.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.740+0000 7f5cde59c640 1 --2- 172.21.15.38:0/3979025203 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5ce0073c20 0x7f5ce00aa5e0 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7f5cd000ecb0 tx=0x7f5cd000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:26.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.740+0000 7f5cd6ffd640 1 -- 172.21.15.38:0/3979025203 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cd0003b70 con 0x7f5ce0073c20 2024-09-21T03:42:26.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.741+0000 7f5cd6ffd640 1 -- 172.21.15.38:0/3979025203 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5cd0003d10 con 0x7f5ce0073c20 2024-09-21T03:42:26.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.741+0000 7f5ce497b640 1 -- 172.21.15.38:0/3979025203 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ce00ab030 con 0x7f5ce0073c20 2024-09-21T03:42:26.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.741+0000 7f5cd6ffd640 1 -- 172.21.15.38:0/3979025203 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cd0004280 con 0x7f5ce0073c20 2024-09-21T03:42:26.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.741+0000 7f5ce497b640 1 -- 172.21.15.38:0/3979025203 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ce00ab5f0 con 0x7f5ce0073c20 2024-09-21T03:42:26.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.742+0000 7f5ce497b640 1 -- 172.21.15.38:0/3979025203 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ce015fc90 con 0x7f5ce0073c20 2024-09-21T03:42:26.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.743+0000 7f5cd6ffd640 1 -- 172.21.15.38:0/3979025203 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f5cd0004420 con 0x7f5ce0073c20 2024-09-21T03:42:26.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.743+0000 7f5cd6ffd640 1 --2- 172.21.15.38:0/3979025203 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5cc0078b90 0x7f5cc007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:26.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.743+0000 7f5cd6ffd640 1 -- 172.21.15.38:0/3979025203 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 8024+0+0 (secure 0 0 0) 0x7f5cd009c300 con 0x7f5ce0073c20 2024-09-21T03:42:26.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.743+0000 7f5cddd9b640 1 --2- 172.21.15.38:0/3979025203 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5cc0078b90 0x7f5cc007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:26.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.744+0000 7f5cddd9b640 1 --2- 172.21.15.38:0/3979025203 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5cc0078b90 0x7f5cc007b0a0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f5cc8034000 tx=0x7f5cc8004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:26.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.747+0000 7f5cd6ffd640 1 -- 172.21.15.38:0/3979025203 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f5cd0063900 con 0x7f5ce0073c20 2024-09-21T03:42:26.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:26.978+0000 7f5ce497b640 1 -- 172.21.15.38:0/3979025203 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f5ce006e460 con 0x7f5ce0073c20 2024-09-21T03:42:27.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:27.229+0000 7f5cd6ffd640 1 -- 172.21.15.38:0/3979025203 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v206) ==== 162+0+0 (secure 0 0 0) 0x7f5cd0068330 con 0x7f5ce0073c20 2024-09-21T03:42:27.231 INFO:teuthology.orchestra.run.smithi038.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-21T03:42:27.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:27.231+0000 7f5ce497b640 1 -- 172.21.15.38:0/3979025203 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5cc0078b90 msgr2=0x7f5cc007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:27.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:27.231+0000 7f5ce497b640 1 --2- 172.21.15.38:0/3979025203 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5cc0078b90 0x7f5cc007b0a0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f5cc8034000 tx=0x7f5cc8004730 comp rx=0 tx=0).stop 2024-09-21T03:42:27.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:27.231+0000 7f5ce497b640 1 -- 172.21.15.38:0/3979025203 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5ce0073c20 msgr2=0x7f5ce00aa5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:27.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:27.231+0000 7f5ce497b640 1 --2- 172.21.15.38:0/3979025203 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5ce0073c20 0x7f5ce00aa5e0 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7f5cd000ecb0 tx=0x7f5cd000c6a0 comp rx=0 tx=0).stop 2024-09-21T03:42:27.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:27.231+0000 7f5ce497b640 1 -- 172.21.15.38:0/3979025203 shutdown_connections 2024-09-21T03:42:27.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:27.231+0000 7f5ce497b640 1 --2- 172.21.15.38:0/3979025203 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5cc0078b90 0x7f5cc007b0a0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:27.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:27.231+0000 7f5ce497b640 1 --2- 172.21.15.38:0/3979025203 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5ce015c4b0 0x7f5ce01e8390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:27.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:27.231+0000 7f5ce497b640 1 --2- 172.21.15.38:0/3979025203 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5ce00ce6b0 0x7f5ce015b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:27.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:27.231+0000 7f5ce497b640 1 --2- 172.21.15.38:0/3979025203 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5ce0073c20 0x7f5ce00aa5e0 unknown :-1 s=CLOSED pgs=1735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:27.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:27.231+0000 7f5ce497b640 1 -- 172.21.15.38:0/3979025203 >> 172.21.15.38:0/3979025203 conn(0x7f5ce00cd460 msgr2=0x7f5ce00cd870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:27.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:27.231+0000 7f5ce497b640 1 -- 172.21.15.38:0/3979025203 shutdown_connections 2024-09-21T03:42:27.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:27.231+0000 7f5ce497b640 1 -- 172.21.15.38:0/3979025203 wait complete. 2024-09-21T03:42:27.241 INFO:tasks.ceph.mds.c.smithi038.stderr:2024-09-21T03:42:27.239+0000 7f1eda49e640 -1 log_channel(cluster) log [ERR] : error decoding table object 'mds0_inotable': Decoder at 'virtual void InoTable::decode_state(ceph::buffer::v15_2_0::list::const_iterator&)' v=2 cannot decode v=100 minimal_decoder=101: Malformed input [buffer:3] 2024-09-21T03:42:27.242 INFO:tasks.ceph.mon.a.smithi038.stderr:2024-09-21T03:42:27.240+0000 7f3f45713640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T03:42:27.242 INFO:tasks.ceph.mon.a.smithi038.stderr:2024-09-21T03:42:27.240+0000 7f3f45713640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-09-21T03:42:27.248 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:42:27.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:27.363+0000 7efd6a1ad640 1 -- 172.21.15.38:0/1712394285 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efd6415d4c0 msgr2=0x7efd6415db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:27.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:27.363+0000 7efd6a1ad640 1 --2- 172.21.15.38:0/1712394285 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efd6415d4c0 0x7efd6415db30 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7efd5c005fd0 tx=0x7efd5c02a540 comp rx=0 tx=0).stop 2024-09-21T03:42:27.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:27.363+0000 7efd6a1ad640 1 -- 172.21.15.38:0/1712394285 shutdown_connections 2024-09-21T03:42:27.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:27.363+0000 7efd6a1ad640 1 --2- 172.21.15.38:0/1712394285 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efd6415d4c0 0x7efd6415db30 unknown :-1 s=CLOSED pgs=1736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:27.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:27.363+0000 7efd6a1ad640 1 --2- 172.21.15.38:0/1712394285 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efd6406f560 0x7efd64073d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:27.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:27.363+0000 7efd6a1ad640 1 -- 172.21.15.38:0/1712394285 >> 172.21.15.38:0/1712394285 conn(0x7efd640b2cc0 msgr2=0x7efd640b30d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:27.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:27.363+0000 7efd6a1ad640 1 -- 172.21.15.38:0/1712394285 shutdown_connections 2024-09-21T03:42:27.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:27.363+0000 7efd6a1ad640 1 -- 172.21.15.38:0/1712394285 wait complete. 2024-09-21T03:42:27.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:27.365+0000 7efd6a1ad640 1 Processor -- start 2024-09-21T03:42:27.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:27.365+0000 7efd6a1ad640 1 -- start start 2024-09-21T03:42:27.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:27.367+0000 7efd6a1ad640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efd6406f560 0x7efd6415be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:27.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:27.367+0000 7efd6a1ad640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efd64074290 0x7efd6406dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:27.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:27.367+0000 7efd6a1ad640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efd6415d4c0 0x7efd641476c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:27.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:27.367+0000 7efd6a1ad640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7efd6415e540 con 0x7efd6415d4c0 2024-09-21T03:42:27.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:27.367+0000 7efd6a1ad640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7efd6415e3c0 con 0x7efd64074290 2024-09-21T03:42:27.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:27.367+0000 7efd6a1ad640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7efd6415e6c0 con 0x7efd6406f560 2024-09-21T03:42:27.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:27.367+0000 7efd63fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efd6415d4c0 0x7efd641476c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:27.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:27.367+0000 7efd63fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efd6415d4c0 0x7efd641476c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:59894/0 (socket says 172.21.15.38:59894) 2024-09-21T03:42:27.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:27.367+0000 7efd63fff640 1 -- 172.21.15.38:0/4098959214 learned_addr learned my addr 172.21.15.38:0/4098959214 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:42:27.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:27.367+0000 7efd637fe640 1 --2- 172.21.15.38:0/4098959214 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efd6406f560 0x7efd6415be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:27.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:27.367+0000 7efd62ffd640 1 --2- 172.21.15.38:0/4098959214 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efd64074290 0x7efd6406dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:27.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:27.367+0000 7efd63fff640 1 -- 172.21.15.38:0/4098959214 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efd6406f560 msgr2=0x7efd6415be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:27.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:27.367+0000 7efd63fff640 1 --2- 172.21.15.38:0/4098959214 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efd6406f560 0x7efd6415be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:27.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:27.367+0000 7efd63fff640 1 -- 172.21.15.38:0/4098959214 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efd64074290 msgr2=0x7efd6406dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:27.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:27.367+0000 7efd63fff640 1 --2- 172.21.15.38:0/4098959214 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efd64074290 0x7efd6406dc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:27.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:27.367+0000 7efd63fff640 1 -- 172.21.15.38:0/4098959214 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd641f7a30 con 0x7efd6415d4c0 2024-09-21T03:42:27.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:27.367+0000 7efd63fff640 1 --2- 172.21.15.38:0/4098959214 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efd6415d4c0 0x7efd641476c0 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7efd5c006010 tx=0x7efd5c003890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:27.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:27.368+0000 7efd4bfff640 1 -- 172.21.15.38:0/4098959214 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd5c03e030 con 0x7efd6415d4c0 2024-09-21T03:42:27.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:27.368+0000 7efd6a1ad640 1 -- 172.21.15.38:0/4098959214 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd641f3790 con 0x7efd6415d4c0 2024-09-21T03:42:27.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:27.368+0000 7efd4bfff640 1 -- 172.21.15.38:0/4098959214 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efd5c003d80 con 0x7efd6415d4c0 2024-09-21T03:42:27.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:27.368+0000 7efd4bfff640 1 -- 172.21.15.38:0/4098959214 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd5c035040 con 0x7efd6415d4c0 2024-09-21T03:42:27.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:27.368+0000 7efd6a1ad640 1 -- 172.21.15.38:0/4098959214 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd641f3d50 con 0x7efd6415d4c0 2024-09-21T03:42:27.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:27.369+0000 7efd6a1ad640 1 -- 172.21.15.38:0/4098959214 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd640a9490 con 0x7efd6415d4c0 2024-09-21T03:42:27.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:27.369+0000 7efd4bfff640 1 -- 172.21.15.38:0/4098959214 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7efd5c041030 con 0x7efd6415d4c0 2024-09-21T03:42:27.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:27.370+0000 7efd4bfff640 1 --2- 172.21.15.38:0/4098959214 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7efd50078b90 0x7efd5007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:27.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:27.370+0000 7efd4bfff640 1 -- 172.21.15.38:0/4098959214 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 8110+0+0 (secure 0 0 0) 0x7efd5c0bc230 con 0x7efd6415d4c0 2024-09-21T03:42:27.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:27.370+0000 7efd637fe640 1 --2- 172.21.15.38:0/4098959214 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7efd50078b90 0x7efd5007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:27.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:27.371+0000 7efd637fe640 1 --2- 172.21.15.38:0/4098959214 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7efd50078b90 0x7efd5007b0a0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7efd54005fd0 tx=0x7efd540095e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:27.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:27.374+0000 7efd4bfff640 1 -- 172.21.15.38:0/4098959214 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7efd5c083640 con 0x7efd6415d4c0 2024-09-21T03:42:27.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:27.599+0000 7efd6a1ad640 1 -- 172.21.15.38:0/4098959214 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efd6415c7c0 con 0x7efd6415d4c0 2024-09-21T03:42:27.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:27.600+0000 7efd4bfff640 1 -- 172.21.15.38:0/4098959214 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 208 v208) ==== 95+0+3566 (secure 0 0 0) 0x7efd5c088070 con 0x7efd6415d4c0 2024-09-21T03:42:27.602 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:42:27.602 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":208,"btime":"2024-09-21T03:42:27:241044+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6086,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/965972663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":965972663},{"type":"v1","addr":"172.21.15.195:6836","nonce":965972663}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6700,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/3176122334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":3176122334},{"type":"v1","addr":"172.21.15.195:6837","nonce":3176122334}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":208,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:40:02.319282+0000","modified":"2024-09-21T03:42:27.240948+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":117,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-21T03:42:27.603 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 208 2024-09-21T03:42:27.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:27.602+0000 7efd6a1ad640 1 -- 172.21.15.38:0/4098959214 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7efd50078b90 msgr2=0x7efd5007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:27.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:27.602+0000 7efd6a1ad640 1 --2- 172.21.15.38:0/4098959214 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7efd50078b90 0x7efd5007b0a0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7efd54005fd0 tx=0x7efd540095e0 comp rx=0 tx=0).stop 2024-09-21T03:42:27.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:27.602+0000 7efd6a1ad640 1 -- 172.21.15.38:0/4098959214 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efd6415d4c0 msgr2=0x7efd641476c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:27.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:27.602+0000 7efd6a1ad640 1 --2- 172.21.15.38:0/4098959214 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efd6415d4c0 0x7efd641476c0 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7efd5c006010 tx=0x7efd5c003890 comp rx=0 tx=0).stop 2024-09-21T03:42:27.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:27.602+0000 7efd6a1ad640 1 -- 172.21.15.38:0/4098959214 shutdown_connections 2024-09-21T03:42:27.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:27.602+0000 7efd6a1ad640 1 --2- 172.21.15.38:0/4098959214 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efd6415d4c0 0x7efd641476c0 unknown :-1 s=CLOSED pgs=1737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:27.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:27.602+0000 7efd6a1ad640 1 --2- 172.21.15.38:0/4098959214 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efd64074290 0x7efd6406dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:27.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:27.602+0000 7efd6a1ad640 1 --2- 172.21.15.38:0/4098959214 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7efd50078b90 0x7efd5007b0a0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:27.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:27.602+0000 7efd6a1ad640 1 --2- 172.21.15.38:0/4098959214 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efd6406f560 0x7efd6415be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:27.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:27.602+0000 7efd6a1ad640 1 -- 172.21.15.38:0/4098959214 >> 172.21.15.38:0/4098959214 conn(0x7efd640b2cc0 msgr2=0x7efd640b30d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:27.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:27.602+0000 7efd6a1ad640 1 -- 172.21.15.38:0/4098959214 shutdown_connections 2024-09-21T03:42:27.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:27.602+0000 7efd6a1ad640 1 -- 172.21.15.38:0/4098959214 wait complete. 2024-09-21T03:42:27.616 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-21T03:42:27.616 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:42:27.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:27.730+0000 7f3702101640 1 -- 172.21.15.38:0/1464719283 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f36fc0d01f0 msgr2=0x7f36fc15daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:27.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:27.730+0000 7f3702101640 1 --2- 172.21.15.38:0/1464719283 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f36fc0d01f0 0x7f36fc15daf0 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7f36fc1605a0 tx=0x7f36ec02a5f0 comp rx=0 tx=0).stop 2024-09-21T03:42:27.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:27.730+0000 7f3702101640 1 -- 172.21.15.38:0/1464719283 shutdown_connections 2024-09-21T03:42:27.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:27.730+0000 7f3702101640 1 --2- 172.21.15.38:0/1464719283 >> v2:172.21.15.195:3300/0 conn(0x7f36fc0d0980 0x7f36fc0d0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:27.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:27.730+0000 7f3702101640 1 --2- 172.21.15.38:0/1464719283 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f36fc0d01f0 0x7f36fc15daf0 unknown :-1 s=CLOSED pgs=1738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:27.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:27.730+0000 7f3702101640 1 -- 172.21.15.38:0/1464719283 >> 172.21.15.38:0/1464719283 conn(0x7f36fc072990 msgr2=0x7f36fc072da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:27.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:27.730+0000 7f3702101640 1 -- 172.21.15.38:0/1464719283 shutdown_connections 2024-09-21T03:42:27.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:27.730+0000 7f3702101640 1 -- 172.21.15.38:0/1464719283 wait complete. 2024-09-21T03:42:27.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:27.732+0000 7f3702101640 1 Processor -- start 2024-09-21T03:42:27.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:27.732+0000 7f3702101640 1 -- start start 2024-09-21T03:42:27.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:27.734+0000 7f3702101640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f36fc0d01f0 0x7f36fc0c8260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:27.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:27.734+0000 7f3702101640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f36fc0d0980 0x7f36fc1e55e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:27.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:27.734+0000 7f3702101640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f36fc15a6f0 0x7f36fc146710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:27.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:27.734+0000 7f3702101640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f36fc0cd340 con 0x7f36fc15a6f0 2024-09-21T03:42:27.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:27.734+0000 7f3702101640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f36fc0cd1c0 con 0x7f36fc0d01f0 2024-09-21T03:42:27.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:27.734+0000 7f3702101640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f36fc0cd4c0 con 0x7f36fc0d0980 2024-09-21T03:42:27.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:27.734+0000 7f36fbfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f36fc15a6f0 0x7f36fc146710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:27.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:27.734+0000 7f36fbfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f36fc15a6f0 0x7f36fc146710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:59904/0 (socket says 172.21.15.38:59904) 2024-09-21T03:42:27.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:27.734+0000 7f36fbfff640 1 -- 172.21.15.38:0/3566048869 learned_addr learned my addr 172.21.15.38:0/3566048869 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:42:27.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:27.734+0000 7f36fb7fe640 1 --2- 172.21.15.38:0/3566048869 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f36fc0d01f0 0x7f36fc0c8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:27.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:27.734+0000 7f36faffd640 1 --2- 172.21.15.38:0/3566048869 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f36fc0d0980 0x7f36fc1e55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:27.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:27.734+0000 7f36fbfff640 1 -- 172.21.15.38:0/3566048869 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f36fc0d0980 msgr2=0x7f36fc1e55e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:27.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:27.734+0000 7f36fbfff640 1 --2- 172.21.15.38:0/3566048869 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f36fc0d0980 0x7f36fc1e55e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:27.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:27.734+0000 7f36fbfff640 1 -- 172.21.15.38:0/3566048869 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f36fc0d01f0 msgr2=0x7f36fc0c8260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:27.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:27.734+0000 7f36fbfff640 1 --2- 172.21.15.38:0/3566048869 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f36fc0d01f0 0x7f36fc0c8260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:27.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:27.734+0000 7f36fbfff640 1 -- 172.21.15.38:0/3566048869 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36fc0c71c0 con 0x7f36fc15a6f0 2024-09-21T03:42:27.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:27.734+0000 7f36fbfff640 1 --2- 172.21.15.38:0/3566048869 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f36fc15a6f0 0x7f36fc146710 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7f36f400ea70 tx=0x7f36f400ef30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:27.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:27.734+0000 7f36e3fff640 1 -- 172.21.15.38:0/3566048869 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36f400cde0 con 0x7f36fc15a6f0 2024-09-21T03:42:27.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:27.735+0000 7f36e3fff640 1 -- 172.21.15.38:0/3566048869 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f36f4004c10 con 0x7f36fc15a6f0 2024-09-21T03:42:27.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:27.735+0000 7f36e3fff640 1 -- 172.21.15.38:0/3566048869 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36f4005020 con 0x7f36fc15a6f0 2024-09-21T03:42:27.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:27.735+0000 7f3702101640 1 -- 172.21.15.38:0/3566048869 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36fc0c74a0 con 0x7f36fc15a6f0 2024-09-21T03:42:27.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:27.735+0000 7f3702101640 1 -- 172.21.15.38:0/3566048869 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36fc0c7a60 con 0x7f36fc15a6f0 2024-09-21T03:42:27.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:27.736+0000 7f3702101640 1 -- 172.21.15.38:0/3566048869 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36fc06ac50 con 0x7f36fc15a6f0 2024-09-21T03:42:27.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:27.736+0000 7f36e3fff640 1 -- 172.21.15.38:0/3566048869 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f36f40051c0 con 0x7f36fc15a6f0 2024-09-21T03:42:27.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:27.737+0000 7f36e3fff640 1 --2- 172.21.15.38:0/3566048869 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f36e8078bc0 0x7f36e807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:27.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:27.737+0000 7f36e3fff640 1 -- 172.21.15.38:0/3566048869 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 8110+0+0 (secure 0 0 0) 0x7f36f409c260 con 0x7f36fc15a6f0 2024-09-21T03:42:27.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:27.737+0000 7f36fb7fe640 1 --2- 172.21.15.38:0/3566048869 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f36e8078bc0 0x7f36e807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:27.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:27.738+0000 7f36fb7fe640 1 --2- 172.21.15.38:0/3566048869 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f36e8078bc0 0x7f36e807b0d0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f36ec033000 tx=0x7f36ec004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:27.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:27.742+0000 7f36e3fff640 1 -- 172.21.15.38:0/3566048869 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f36fc06ac50 con 0x7f36fc15a6f0 2024-09-21T03:42:27.968 INFO:tasks.ceph.mds.a.smithi038.stderr: -52> 2024-09-21T03:42:23.214+0000 7fdf0b9d8640 -1 log_channel(cluster) log [ERR] : error decoding table object 'mds_snaptable': Decoder at 'virtual void SnapServer::decode_server_state(ceph::buffer::v15_2_0::list::const_iterator&)' v=5 cannot decode v=100 minimal_decoder=101: Malformed input [buffer:3] 2024-09-21T03:42:27.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:27.971+0000 7f3702101640 1 -- 172.21.15.38:0/3566048869 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f36fc15daf0 con 0x7f36fc15a6f0 2024-09-21T03:42:27.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:27.971+0000 7f36e3fff640 1 -- 172.21.15.38:0/3566048869 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 208 v208) ==== 95+0+3566 (secure 0 0 0) 0x7f36f4063780 con 0x7f36fc15a6f0 2024-09-21T03:42:27.973 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:42:27.973 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":208,"btime":"2024-09-21T03:42:27:241044+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6086,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/965972663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":965972663},{"type":"v1","addr":"172.21.15.195:6836","nonce":965972663}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6700,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/3176122334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":3176122334},{"type":"v1","addr":"172.21.15.195:6837","nonce":3176122334}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":208,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:40:02.319282+0000","modified":"2024-09-21T03:42:27.240948+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":117,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-21T03:42:27.974 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 208 2024-09-21T03:42:27.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:27.974+0000 7f3702101640 1 -- 172.21.15.38:0/3566048869 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f36e8078bc0 msgr2=0x7f36e807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:27.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:27.974+0000 7f3702101640 1 --2- 172.21.15.38:0/3566048869 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f36e8078bc0 0x7f36e807b0d0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f36ec033000 tx=0x7f36ec004730 comp rx=0 tx=0).stop 2024-09-21T03:42:27.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:27.974+0000 7f3702101640 1 -- 172.21.15.38:0/3566048869 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f36fc15a6f0 msgr2=0x7f36fc146710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:27.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:27.974+0000 7f3702101640 1 --2- 172.21.15.38:0/3566048869 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f36fc15a6f0 0x7f36fc146710 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7f36f400ea70 tx=0x7f36f400ef30 comp rx=0 tx=0).stop 2024-09-21T03:42:27.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:27.974+0000 7f3702101640 1 -- 172.21.15.38:0/3566048869 shutdown_connections 2024-09-21T03:42:27.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:27.974+0000 7f3702101640 1 --2- 172.21.15.38:0/3566048869 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f36fc15a6f0 0x7f36fc146710 unknown :-1 s=CLOSED pgs=1739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:27.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:27.974+0000 7f3702101640 1 --2- 172.21.15.38:0/3566048869 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f36e8078bc0 0x7f36e807b0d0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:27.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:27.974+0000 7f3702101640 1 --2- 172.21.15.38:0/3566048869 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f36fc0d0980 0x7f36fc1e55e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:27.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:27.974+0000 7f3702101640 1 --2- 172.21.15.38:0/3566048869 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f36fc0d01f0 0x7f36fc0c8260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:27.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:27.974+0000 7f3702101640 1 -- 172.21.15.38:0/3566048869 >> 172.21.15.38:0/3566048869 conn(0x7f36fc072990 msgr2=0x7f36fc15eb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:27.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:27.974+0000 7f3702101640 1 -- 172.21.15.38:0/3566048869 shutdown_connections 2024-09-21T03:42:27.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:27.974+0000 7f3702101640 1 -- 172.21.15.38:0/3566048869 wait complete. 2024-09-21T03:42:27.988 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-21T03:42:27.988 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:42:28.025 INFO:tasks.ceph.mds.a.smithi038.stdout:starting mds.a at 2024-09-21T03:42:28.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:28.106+0000 7f5c3dcd5640 1 -- 172.21.15.38:0/1377471840 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5c38162280 msgr2=0x7f5c38162690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:28.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:28.106+0000 7f5c3dcd5640 1 --2- 172.21.15.38:0/1377471840 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5c38162280 0x7f5c38162690 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7f5c2c005fd0 tx=0x7f5c2c02a5d0 comp rx=0 tx=0).stop 2024-09-21T03:42:28.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:28.106+0000 7f5c3dcd5640 1 -- 172.21.15.38:0/1377471840 shutdown_connections 2024-09-21T03:42:28.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:28.106+0000 7f5c3dcd5640 1 --2- 172.21.15.38:0/1377471840 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5c38162280 0x7f5c38162690 unknown :-1 s=CLOSED pgs=1743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:28.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:28.106+0000 7f5c3dcd5640 1 --2- 172.21.15.38:0/1377471840 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5c38162f00 0x7f5c380d24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:28.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:28.106+0000 7f5c3dcd5640 1 -- 172.21.15.38:0/1377471840 >> 172.21.15.38:0/1377471840 conn(0x7f5c380b6b20 msgr2=0x7f5c380d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:28.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:28.106+0000 7f5c3dcd5640 1 -- 172.21.15.38:0/1377471840 shutdown_connections 2024-09-21T03:42:28.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:28.106+0000 7f5c3dcd5640 1 -- 172.21.15.38:0/1377471840 wait complete. 2024-09-21T03:42:28.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:28.108+0000 7f5c3dcd5640 1 Processor -- start 2024-09-21T03:42:28.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:28.108+0000 7f5c3dcd5640 1 -- start start 2024-09-21T03:42:28.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:28.109+0000 7f5c3dcd5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5c380b2e40 0x7f5c380ce880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:28.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:28.109+0000 7f5c3dcd5640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5c38162280 0x7f5c380abd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:28.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:28.109+0000 7f5c3dcd5640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5c380acae0 0x7f5c3815c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:28.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:28.109+0000 7f5c3dcd5640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f5c3815e630 con 0x7f5c380b2e40 2024-09-21T03:42:28.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:28.109+0000 7f5c3dcd5640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f5c3815e4b0 con 0x7f5c38162280 2024-09-21T03:42:28.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:28.109+0000 7f5c3dcd5640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f5c3815e7b0 con 0x7f5c380acae0 2024-09-21T03:42:28.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:28.109+0000 7f5c377fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5c380b2e40 0x7f5c380ce880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:28.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:28.109+0000 7f5c377fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5c380b2e40 0x7f5c380ce880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:59932/0 (socket says 172.21.15.38:59932) 2024-09-21T03:42:28.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:28.109+0000 7f5c377fe640 1 -- 172.21.15.38:0/3698353252 learned_addr learned my addr 172.21.15.38:0/3698353252 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:42:28.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:28.110+0000 7f5c36ffd640 1 --2- 172.21.15.38:0/3698353252 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5c38162280 0x7f5c380abd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:28.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:28.110+0000 7f5c37fff640 1 --2- 172.21.15.38:0/3698353252 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5c380acae0 0x7f5c3815c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:28.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:28.110+0000 7f5c377fe640 1 -- 172.21.15.38:0/3698353252 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5c380acae0 msgr2=0x7f5c3815c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:28.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:28.110+0000 7f5c377fe640 1 --2- 172.21.15.38:0/3698353252 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5c380acae0 0x7f5c3815c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:28.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:28.110+0000 7f5c377fe640 1 -- 172.21.15.38:0/3698353252 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5c38162280 msgr2=0x7f5c380abd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:28.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:28.110+0000 7f5c377fe640 1 --2- 172.21.15.38:0/3698353252 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5c38162280 0x7f5c380abd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:28.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:28.110+0000 7f5c377fe640 1 -- 172.21.15.38:0/3698353252 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c380ac240 con 0x7f5c380b2e40 2024-09-21T03:42:28.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:28.110+0000 7f5c377fe640 1 --2- 172.21.15.38:0/3698353252 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5c380b2e40 0x7f5c380ce880 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7f5c2800d740 tx=0x7f5c2800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:28.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:28.110+0000 7f5c1ffff640 1 -- 172.21.15.38:0/3698353252 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c28017070 con 0x7f5c380b2e40 2024-09-21T03:42:28.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:28.110+0000 7f5c1ffff640 1 -- 172.21.15.38:0/3698353252 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5c28009d20 con 0x7f5c380b2e40 2024-09-21T03:42:28.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:28.110+0000 7f5c3dcd5640 1 -- 172.21.15.38:0/3698353252 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c380aaf40 con 0x7f5c380b2e40 2024-09-21T03:42:28.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:28.110+0000 7f5c1ffff640 1 -- 172.21.15.38:0/3698353252 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c280106c0 con 0x7f5c380b2e40 2024-09-21T03:42:28.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:28.110+0000 7f5c3dcd5640 1 -- 172.21.15.38:0/3698353252 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c380ab500 con 0x7f5c380b2e40 2024-09-21T03:42:28.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:28.112+0000 7f5c3dcd5640 1 -- 172.21.15.38:0/3698353252 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c38162f00 con 0x7f5c380b2e40 2024-09-21T03:42:28.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:28.112+0000 7f5c1ffff640 1 -- 172.21.15.38:0/3698353252 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f5c2801a070 con 0x7f5c380b2e40 2024-09-21T03:42:28.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:28.113+0000 7f5c1ffff640 1 --2- 172.21.15.38:0/3698353252 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5c24078b90 0x7f5c2407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:28.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:28.113+0000 7f5c1ffff640 1 -- 172.21.15.38:0/3698353252 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 8110+0+0 (secure 0 0 0) 0x7f5c2809b7e0 con 0x7f5c380b2e40 2024-09-21T03:42:28.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:28.113+0000 7f5c36ffd640 1 --2- 172.21.15.38:0/3698353252 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5c24078b90 0x7f5c2407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:28.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:28.113+0000 7f5c36ffd640 1 --2- 172.21.15.38:0/3698353252 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5c24078b90 0x7f5c2407b0a0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f5c2c033000 tx=0x7f5c2c004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:28.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:28.117+0000 7f5c1ffff640 1 -- 172.21.15.38:0/3698353252 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f5c28062c60 con 0x7f5c380b2e40 2024-09-21T03:42:28.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:28.343+0000 7f5c3dcd5640 1 -- 172.21.15.38:0/3698353252 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5c380b9460 con 0x7f5c380b2e40 2024-09-21T03:42:28.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:28.344+0000 7f5c1ffff640 1 -- 172.21.15.38:0/3698353252 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 209 v209) ==== 95+0+4392 (secure 0 0 0) 0x7f5c28067690 con 0x7f5c380b2e40 2024-09-21T03:42:28.346 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:42:28.346 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":209,"btime":"2024-09-21T03:42:28:242508+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6086,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/965972663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":965972663},{"type":"v1","addr":"172.21.15.195:6836","nonce":965972663}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6700,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/3176122334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":3176122334},{"type":"v1","addr":"172.21.15.195:6837","nonce":3176122334}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":9111,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/3951934534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":3951934534},{"type":"v1","addr":"172.21.15.38:6835","nonce":3951934534}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":209}],"filesystems":[{"mdsmap":{"epoch":208,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:40:02.319282+0000","modified":"2024-09-21T03:42:27.240948+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":117,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-21T03:42:28.346 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 209 2024-09-21T03:42:28.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:28.346+0000 7f5c3dcd5640 1 -- 172.21.15.38:0/3698353252 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5c24078b90 msgr2=0x7f5c2407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:28.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:28.346+0000 7f5c3dcd5640 1 --2- 172.21.15.38:0/3698353252 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5c24078b90 0x7f5c2407b0a0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f5c2c033000 tx=0x7f5c2c004730 comp rx=0 tx=0).stop 2024-09-21T03:42:28.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:28.346+0000 7f5c3dcd5640 1 -- 172.21.15.38:0/3698353252 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5c380b2e40 msgr2=0x7f5c380ce880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:28.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:28.346+0000 7f5c3dcd5640 1 --2- 172.21.15.38:0/3698353252 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5c380b2e40 0x7f5c380ce880 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7f5c2800d740 tx=0x7f5c2800dc00 comp rx=0 tx=0).stop 2024-09-21T03:42:28.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:28.346+0000 7f5c3dcd5640 1 -- 172.21.15.38:0/3698353252 shutdown_connections 2024-09-21T03:42:28.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:28.346+0000 7f5c3dcd5640 1 --2- 172.21.15.38:0/3698353252 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5c24078b90 0x7f5c2407b0a0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:28.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:28.346+0000 7f5c3dcd5640 1 --2- 172.21.15.38:0/3698353252 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5c380acae0 0x7f5c3815c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:28.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:28.346+0000 7f5c3dcd5640 1 --2- 172.21.15.38:0/3698353252 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5c38162280 0x7f5c380abd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:28.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:28.346+0000 7f5c3dcd5640 1 --2- 172.21.15.38:0/3698353252 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5c380b2e40 0x7f5c380ce880 unknown :-1 s=CLOSED pgs=1744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:28.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:28.346+0000 7f5c3dcd5640 1 -- 172.21.15.38:0/3698353252 >> 172.21.15.38:0/3698353252 conn(0x7f5c380b6b20 msgr2=0x7f5c380b3250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:28.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:28.346+0000 7f5c3dcd5640 1 -- 172.21.15.38:0/3698353252 shutdown_connections 2024-09-21T03:42:28.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:28.346+0000 7f5c3dcd5640 1 -- 172.21.15.38:0/3698353252 wait complete. 2024-09-21T03:42:28.359 INFO:tasks.cephfs.test_damage:Result: Mutation 'Corrupt mds0_inotable' led to DAMAGED state 2024-09-21T03:42:28.359 INFO:tasks.cephfs.test_damage:Applying mutation 'Corrupt 400.00000000' 2024-09-21T03:42:28.359 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-21T03:42:28.359 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:42:28.416 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056795,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2354 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T03:42:28.416 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi038.front.sepia.ceph.com 2024-09-21T03:42:28.416 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:42:28.416 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T03:42:28.471 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T03:42:28.471 INFO:teuthology.orchestra.run.smithi038.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-09-21T03:42:28.471 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-21T03:42:28.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:28.627+0000 7fa948fed640 1 -- 172.21.15.38:0/50166556 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa94415ee80 msgr2=0x7fa9440d0170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:28.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:28.627+0000 7fa948fed640 1 --2- 172.21.15.38:0/50166556 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa94415ee80 0x7fa9440d0170 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7fa934005fd0 tx=0x7fa93402a680 comp rx=0 tx=0).stop 2024-09-21T03:42:28.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:28.627+0000 7fa948fed640 1 -- 172.21.15.38:0/50166556 shutdown_connections 2024-09-21T03:42:28.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:28.627+0000 7fa948fed640 1 --2- 172.21.15.38:0/50166556 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa94415ee80 0x7fa9440d0170 unknown :-1 s=CLOSED pgs=1746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:28.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:28.627+0000 7fa948fed640 1 -- 172.21.15.38:0/50166556 >> 172.21.15.38:0/50166556 conn(0x7fa94409d830 msgr2=0x7fa9440b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:28.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:28.627+0000 7fa948fed640 1 -- 172.21.15.38:0/50166556 shutdown_connections 2024-09-21T03:42:28.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:28.627+0000 7fa948fed640 1 -- 172.21.15.38:0/50166556 wait complete. 2024-09-21T03:42:28.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:28.629+0000 7fa948fed640 1 Processor -- start 2024-09-21T03:42:28.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:28.629+0000 7fa948fed640 1 -- start start 2024-09-21T03:42:28.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:28.630+0000 7fa948fed640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa9440b9ad0 0x7fa94414a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:28.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:28.630+0000 7fa948fed640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa9440ceb90 0x7fa94414a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:28.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:28.630+0000 7fa948fed640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa94415ee80 0x7fa944150120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:28.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:28.630+0000 7fa948fed640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa9440b6910 con 0x7fa9440b9ad0 2024-09-21T03:42:28.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:28.630+0000 7fa948fed640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fa9440b6790 con 0x7fa9440ceb90 2024-09-21T03:42:28.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:28.630+0000 7fa948fed640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fa9440b6a90 con 0x7fa94415ee80 2024-09-21T03:42:28.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:28.630+0000 7fa94259c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa9440b9ad0 0x7fa94414a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:28.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:28.630+0000 7fa94259c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa9440b9ad0 0x7fa94414a050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:59940/0 (socket says 172.21.15.38:59940) 2024-09-21T03:42:28.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:28.630+0000 7fa94259c640 1 -- 172.21.15.38:0/4118520701 learned_addr learned my addr 172.21.15.38:0/4118520701 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:42:28.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:28.631+0000 7fa941d9b640 1 --2- 172.21.15.38:0/4118520701 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa9440ceb90 0x7fa94414a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:28.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:28.631+0000 7fa942d9d640 1 --2- 172.21.15.38:0/4118520701 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa94415ee80 0x7fa944150120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:28.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:28.631+0000 7fa94259c640 1 -- 172.21.15.38:0/4118520701 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa94415ee80 msgr2=0x7fa944150120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:28.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:28.631+0000 7fa94259c640 1 --2- 172.21.15.38:0/4118520701 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa94415ee80 0x7fa944150120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:28.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:28.631+0000 7fa94259c640 1 -- 172.21.15.38:0/4118520701 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa9440ceb90 msgr2=0x7fa94414a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:28.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:28.631+0000 7fa94259c640 1 --2- 172.21.15.38:0/4118520701 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa9440ceb90 0x7fa94414a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:28.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:28.631+0000 7fa94259c640 1 -- 172.21.15.38:0/4118520701 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa94415c360 con 0x7fa9440b9ad0 2024-09-21T03:42:28.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:28.631+0000 7fa942d9d640 1 --2- 172.21.15.38:0/4118520701 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa94415ee80 0x7fa944150120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:42:28.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:28.631+0000 7fa94259c640 1 --2- 172.21.15.38:0/4118520701 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa9440b9ad0 0x7fa94414a050 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7fa934006010 tx=0x7fa934032f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:28.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:28.631+0000 7fa93affd640 1 -- 172.21.15.38:0/4118520701 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa934041020 con 0x7fa9440b9ad0 2024-09-21T03:42:28.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:28.631+0000 7fa93affd640 1 -- 172.21.15.38:0/4118520701 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa934004330 con 0x7fa9440b9ad0 2024-09-21T03:42:28.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:28.631+0000 7fa93affd640 1 -- 172.21.15.38:0/4118520701 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa934036040 con 0x7fa9440b9ad0 2024-09-21T03:42:28.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:28.631+0000 7fa948fed640 1 -- 172.21.15.38:0/4118520701 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa94406e0d0 con 0x7fa9440b9ad0 2024-09-21T03:42:28.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:28.631+0000 7fa948fed640 1 -- 172.21.15.38:0/4118520701 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa94406e690 con 0x7fa9440b9ad0 2024-09-21T03:42:28.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:28.632+0000 7fa948fed640 1 -- 172.21.15.38:0/4118520701 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9440d0170 con 0x7fa9440b9ad0 2024-09-21T03:42:28.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:28.633+0000 7fa93affd640 1 -- 172.21.15.38:0/4118520701 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa934044020 con 0x7fa9440b9ad0 2024-09-21T03:42:28.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:28.633+0000 7fa93affd640 1 --2- 172.21.15.38:0/4118520701 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa924078b90 0x7fa92407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:28.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:28.633+0000 7fa93affd640 1 -- 172.21.15.38:0/4118520701 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 8110+0+0 (secure 0 0 0) 0x7fa9340bc9b0 con 0x7fa9440b9ad0 2024-09-21T03:42:28.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:28.634+0000 7fa941d9b640 1 --2- 172.21.15.38:0/4118520701 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa924078b90 0x7fa92407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:28.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:28.634+0000 7fa941d9b640 1 --2- 172.21.15.38:0/4118520701 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa924078b90 0x7fa92407b0a0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fa94415ba40 tx=0x7fa92c009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:28.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:28.638+0000 7fa93affd640 1 -- 172.21.15.38:0/4118520701 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fa93404b030 con 0x7fa9440b9ad0 2024-09-21T03:42:28.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:28.864+0000 7fa948fed640 1 -- 172.21.15.38:0/4118520701 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fa9440b82f0 con 0x7fa9440b9ad0 2024-09-21T03:42:29.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:29.250+0000 7fa93affd640 1 -- 172.21.15.38:0/4118520701 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v210) ==== 161+0+0 (secure 0 0 0) 0x7fa934083e30 con 0x7fa9440b9ad0 2024-09-21T03:42:29.252 INFO:teuthology.orchestra.run.smithi038.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-21T03:42:29.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:29.252+0000 7fa948fed640 1 -- 172.21.15.38:0/4118520701 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa924078b90 msgr2=0x7fa92407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:29.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:29.252+0000 7fa948fed640 1 --2- 172.21.15.38:0/4118520701 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa924078b90 0x7fa92407b0a0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fa94415ba40 tx=0x7fa92c009040 comp rx=0 tx=0).stop 2024-09-21T03:42:29.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:29.252+0000 7fa948fed640 1 -- 172.21.15.38:0/4118520701 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa9440b9ad0 msgr2=0x7fa94414a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:29.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:29.252+0000 7fa948fed640 1 --2- 172.21.15.38:0/4118520701 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa9440b9ad0 0x7fa94414a050 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7fa934006010 tx=0x7fa934032f10 comp rx=0 tx=0).stop 2024-09-21T03:42:29.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:29.252+0000 7fa948fed640 1 -- 172.21.15.38:0/4118520701 shutdown_connections 2024-09-21T03:42:29.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:29.252+0000 7fa948fed640 1 --2- 172.21.15.38:0/4118520701 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa924078b90 0x7fa92407b0a0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:29.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:29.252+0000 7fa948fed640 1 --2- 172.21.15.38:0/4118520701 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa94415ee80 0x7fa944150120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:29.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:29.252+0000 7fa948fed640 1 --2- 172.21.15.38:0/4118520701 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa9440ceb90 0x7fa94414a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:29.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:29.252+0000 7fa948fed640 1 --2- 172.21.15.38:0/4118520701 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa9440b9ad0 0x7fa94414a050 unknown :-1 s=CLOSED pgs=1747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:29.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:29.252+0000 7fa948fed640 1 -- 172.21.15.38:0/4118520701 >> 172.21.15.38:0/4118520701 conn(0x7fa94409d830 msgr2=0x7fa9441608a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:29.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:29.252+0000 7fa948fed640 1 -- 172.21.15.38:0/4118520701 shutdown_connections 2024-09-21T03:42:29.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:29.252+0000 7fa948fed640 1 -- 172.21.15.38:0/4118520701 wait complete. 2024-09-21T03:42:29.266 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-09-21T03:42:29.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:29.383+0000 7f9110038640 1 -- 172.21.15.38:0/2370646834 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f91080d0170 msgr2=0x7f91080d0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:29.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:29.383+0000 7f9110038640 1 --2- 172.21.15.38:0/2370646834 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f91080d0170 0x7f91080d0be0 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7f90f80098e0 tx=0x7f90f802a540 comp rx=0 tx=0).stop 2024-09-21T03:42:29.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:29.383+0000 7f9110038640 1 -- 172.21.15.38:0/2370646834 shutdown_connections 2024-09-21T03:42:29.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:29.383+0000 7f9110038640 1 --2- 172.21.15.38:0/2370646834 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f91080d0170 0x7f91080d0be0 unknown :-1 s=CLOSED pgs=1749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:29.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:29.383+0000 7f9110038640 1 -- 172.21.15.38:0/2370646834 >> 172.21.15.38:0/2370646834 conn(0x7f910809d830 msgr2=0x7f91080b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:29.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:29.383+0000 7f9110038640 1 -- 172.21.15.38:0/2370646834 shutdown_connections 2024-09-21T03:42:29.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:29.383+0000 7f9110038640 1 -- 172.21.15.38:0/2370646834 wait complete. 2024-09-21T03:42:29.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:29.385+0000 7f9110038640 1 Processor -- start 2024-09-21T03:42:29.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:29.385+0000 7f9110038640 1 -- start start 2024-09-21T03:42:29.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:29.386+0000 7f9110038640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f91080d0170 0x7f910806e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:29.390 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:29.386+0000 7f9110038640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f91080d1120 0x7f91081f5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:29.390 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:29.386+0000 7f9110038640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f910815ee80 0x7f91080b2810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:29.390 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:29.386+0000 7f9110038640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f91080b6910 con 0x7f91080d0170 2024-09-21T03:42:29.390 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:29.386+0000 7f9110038640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f91080b6790 con 0x7f91080d1120 2024-09-21T03:42:29.390 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:29.386+0000 7f9110038640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f91080b6a90 con 0x7f910815ee80 2024-09-21T03:42:29.390 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:29.387+0000 7f910ddd4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f91080d0170 0x7f910806e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:29.390 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:29.387+0000 7f910ddd4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f91080d0170 0x7f910806e730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:59958/0 (socket says 172.21.15.38:59958) 2024-09-21T03:42:29.390 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:29.387+0000 7f910ddd4640 1 -- 172.21.15.38:0/556525179 learned_addr learned my addr 172.21.15.38:0/556525179 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:42:29.390 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:29.387+0000 7f910d5d3640 1 --2- 172.21.15.38:0/556525179 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f91080d1120 0x7f91081f5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:29.390 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:29.387+0000 7f910e5d5640 1 --2- 172.21.15.38:0/556525179 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f910815ee80 0x7f91080b2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:29.390 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:29.387+0000 7f910ddd4640 1 -- 172.21.15.38:0/556525179 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f910815ee80 msgr2=0x7f91080b2810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:29.391 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:29.387+0000 7f910ddd4640 1 --2- 172.21.15.38:0/556525179 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f910815ee80 0x7f91080b2810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:29.391 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:29.387+0000 7f910ddd4640 1 -- 172.21.15.38:0/556525179 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f91080d1120 msgr2=0x7f91081f5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:29.391 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:29.387+0000 7f910ddd4640 1 --2- 172.21.15.38:0/556525179 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f91080d1120 0x7f91081f5ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:29.391 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:29.387+0000 7f910ddd4640 1 -- 172.21.15.38:0/556525179 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f910806ce80 con 0x7f91080d0170 2024-09-21T03:42:29.391 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:29.387+0000 7f910d5d3640 1 --2- 172.21.15.38:0/556525179 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f91080d1120 0x7f91081f5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:42:29.391 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:29.387+0000 7f910ddd4640 1 --2- 172.21.15.38:0/556525179 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f91080d0170 0x7f910806e730 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7f910400c400 tx=0x7f910400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:29.391 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:29.387+0000 7f90fe7fc640 1 -- 172.21.15.38:0/556525179 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9104017070 con 0x7f91080d0170 2024-09-21T03:42:29.391 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:29.387+0000 7f90fe7fc640 1 -- 172.21.15.38:0/556525179 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f91040053c0 con 0x7f91080d0170 2024-09-21T03:42:29.391 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:29.387+0000 7f90fe7fc640 1 -- 172.21.15.38:0/556525179 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9104003850 con 0x7f91080d0170 2024-09-21T03:42:29.391 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:29.387+0000 7f9110038640 1 -- 172.21.15.38:0/556525179 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f910806d120 con 0x7f91080d0170 2024-09-21T03:42:29.391 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:29.387+0000 7f9110038640 1 -- 172.21.15.38:0/556525179 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f910806d6e0 con 0x7f91080d0170 2024-09-21T03:42:29.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:29.388+0000 7f9110038640 1 -- 172.21.15.38:0/556525179 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91080d0be0 con 0x7f91080d0170 2024-09-21T03:42:29.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:29.389+0000 7f90fe7fc640 1 -- 172.21.15.38:0/556525179 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f910401a070 con 0x7f91080d0170 2024-09-21T03:42:29.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:29.390+0000 7f90fe7fc640 1 --2- 172.21.15.38:0/556525179 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f90ec078b90 0x7f90ec07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:29.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:29.390+0000 7f90fe7fc640 1 -- 172.21.15.38:0/556525179 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 8110+0+0 (secure 0 0 0) 0x7f910409abc0 con 0x7f91080d0170 2024-09-21T03:42:29.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:29.390+0000 7f910d5d3640 1 --2- 172.21.15.38:0/556525179 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f90ec078b90 0x7f90ec07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:29.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:29.390+0000 7f910d5d3640 1 --2- 172.21.15.38:0/556525179 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f90ec078b90 0x7f90ec07b0a0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f90f8034000 tx=0x7f90f8004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:29.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:29.394+0000 7f90fe7fc640 1 -- 172.21.15.38:0/556525179 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f91081592d0 con 0x7f91080d0170 2024-09-21T03:42:29.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:29.655+0000 7f9110038640 1 -- 172.21.15.38:0/556525179 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7f91080cb070 con 0x7f91080d0170 2024-09-21T03:42:30.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.252+0000 7f90fe7fc640 1 -- 172.21.15.38:0/556525179 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v211) ==== 101+0+0 (secure 0 0 0) 0x7f9104062580 con 0x7f91080d0170 2024-09-21T03:42:30.254 INFO:teuthology.orchestra.run.smithi038.stderr:repaired: restoring rank 6:0 2024-09-21T03:42:30.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.254+0000 7f9110038640 1 -- 172.21.15.38:0/556525179 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f90ec078b90 msgr2=0x7f90ec07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:30.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.254+0000 7f9110038640 1 --2- 172.21.15.38:0/556525179 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f90ec078b90 0x7f90ec07b0a0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f90f8034000 tx=0x7f90f8004730 comp rx=0 tx=0).stop 2024-09-21T03:42:30.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.254+0000 7f9110038640 1 -- 172.21.15.38:0/556525179 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f91080d0170 msgr2=0x7f910806e730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:30.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.254+0000 7f9110038640 1 --2- 172.21.15.38:0/556525179 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f91080d0170 0x7f910806e730 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7f910400c400 tx=0x7f910400c8c0 comp rx=0 tx=0).stop 2024-09-21T03:42:30.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.254+0000 7f9110038640 1 -- 172.21.15.38:0/556525179 shutdown_connections 2024-09-21T03:42:30.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.254+0000 7f9110038640 1 --2- 172.21.15.38:0/556525179 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f90ec078b90 0x7f90ec07b0a0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:30.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.254+0000 7f9110038640 1 --2- 172.21.15.38:0/556525179 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f910815ee80 0x7f91080b2810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:30.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.254+0000 7f9110038640 1 --2- 172.21.15.38:0/556525179 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f91080d1120 0x7f91081f5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:30.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.255+0000 7f9110038640 1 --2- 172.21.15.38:0/556525179 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f91080d0170 0x7f910806e730 unknown :-1 s=CLOSED pgs=1750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:30.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.255+0000 7f9110038640 1 -- 172.21.15.38:0/556525179 >> 172.21.15.38:0/556525179 conn(0x7f910809d830 msgr2=0x7f91080ceb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:30.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.255+0000 7f9110038640 1 -- 172.21.15.38:0/556525179 shutdown_connections 2024-09-21T03:42:30.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.255+0000 7f9110038640 1 -- 172.21.15.38:0/556525179 wait complete. 2024-09-21T03:42:30.270 DEBUG:teuthology.orchestra.run.smithi038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-09-21T03:42:30.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.329+0000 7fc809536f40 1 Processor -- start 2024-09-21T03:42:30.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.329+0000 7fc809536f40 1 -- start start 2024-09-21T03:42:30.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.330+0000 7fc809536f40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a528961d00 0x55a528a35ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:30.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.330+0000 7fc809536f40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a52898f5a0 0x55a528a365d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:30.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.330+0000 7fc809536f40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a5289a8e60 0x55a52895d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:30.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.330+0000 7fc809536f40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55a5288afd60 con 0x55a52898f5a0 2024-09-21T03:42:30.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.330+0000 7fc809536f40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55a52890e430 con 0x55a5289a8e60 2024-09-21T03:42:30.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.331+0000 7fc809536f40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55a52898fef0 con 0x55a528961d00 2024-09-21T03:42:30.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.331+0000 7fc8027ef640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a52898f5a0 0x55a528a365d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:30.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.331+0000 7fc8027ef640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a52898f5a0 0x55a528a365d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:59976/0 (socket says 172.21.15.38:59976) 2024-09-21T03:42:30.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.331+0000 7fc8027ef640 1 -- 172.21.15.38:0/4169287702 learned_addr learned my addr 172.21.15.38:0/4169287702 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:42:30.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.331+0000 7fc802ff0640 1 --2- 172.21.15.38:0/4169287702 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a528961d00 0x55a528a35ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:30.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.331+0000 7fc8037f1640 1 --2- 172.21.15.38:0/4169287702 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a5289a8e60 0x55a52895d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:30.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.331+0000 7fc8027ef640 1 -- 172.21.15.38:0/4169287702 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a528961d00 msgr2=0x55a528a35ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:30.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.331+0000 7fc8027ef640 1 --2- 172.21.15.38:0/4169287702 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a528961d00 0x55a528a35ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:30.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.331+0000 7fc8027ef640 1 -- 172.21.15.38:0/4169287702 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a5289a8e60 msgr2=0x55a52895d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:30.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.331+0000 7fc8027ef640 1 --2- 172.21.15.38:0/4169287702 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a5289a8e60 0x55a52895d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:30.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.331+0000 7fc8027ef640 1 -- 172.21.15.38:0/4169287702 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a528a2ed70 con 0x55a52898f5a0 2024-09-21T03:42:30.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.331+0000 7fc8027ef640 1 --2- 172.21.15.38:0/4169287702 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a52898f5a0 0x55a528a365d0 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7fc7f8000c00 tx=0x7fc7f8002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8887b8daead5d7e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:30.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.331+0000 7fc8017ed640 1 -- 172.21.15.38:0/4169287702 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7f804dd50 con 0x55a52898f5a0 2024-09-21T03:42:30.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.332+0000 7fc8017ed640 1 -- 172.21.15.38:0/4169287702 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc7f8056470 con 0x55a52898f5a0 2024-09-21T03:42:30.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.332+0000 7fc8017ed640 1 -- 172.21.15.38:0/4169287702 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7f8056860 con 0x55a52898f5a0 2024-09-21T03:42:30.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.332+0000 7fc809536f40 1 -- 172.21.15.38:0/4169287702 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a52898f5a0 msgr2=0x55a528a365d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:30.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.332+0000 7fc809536f40 1 --2- 172.21.15.38:0/4169287702 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a52898f5a0 0x55a528a365d0 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7fc7f8000c00 tx=0x7fc7f8002fb0 comp rx=0 tx=0).stop 2024-09-21T03:42:30.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.332+0000 7fc809536f40 1 -- 172.21.15.38:0/4169287702 shutdown_connections 2024-09-21T03:42:30.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.332+0000 7fc809536f40 1 --2- 172.21.15.38:0/4169287702 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a5289a8e60 0x55a52895d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:30.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.332+0000 7fc809536f40 1 --2- 172.21.15.38:0/4169287702 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a52898f5a0 0x55a528a365d0 unknown :-1 s=CLOSED pgs=1753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:30.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.332+0000 7fc809536f40 1 --2- 172.21.15.38:0/4169287702 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a528961d00 0x55a528a35ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:30.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.332+0000 7fc809536f40 1 -- 172.21.15.38:0/4169287702 >> 172.21.15.38:0/4169287702 conn(0x55a528880ac0 msgr2=0x55a528a357d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:30.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.332+0000 7fc809536f40 1 -- 172.21.15.38:0/4169287702 shutdown_connections 2024-09-21T03:42:30.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.332+0000 7fc809536f40 1 -- 172.21.15.38:0/4169287702 wait complete. 2024-09-21T03:42:30.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.333+0000 7fc809536f40 1 Processor -- start 2024-09-21T03:42:30.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.333+0000 7fc809536f40 1 -- start start 2024-09-21T03:42:30.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.335+0000 7fc809536f40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a528961d00 0x55a528a2b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:30.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.335+0000 7fc809536f40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a52898f5a0 0x55a528a2bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:30.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.335+0000 7fc809536f40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a5289a8e60 0x55a528a2c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:30.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.335+0000 7fc809536f40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55a528a39360 con 0x55a5289a8e60 2024-09-21T03:42:30.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.335+0000 7fc809536f40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55a5289a6140 con 0x55a528961d00 2024-09-21T03:42:30.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.335+0000 7fc809536f40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55a528a36b10 con 0x55a52898f5a0 2024-09-21T03:42:30.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.335+0000 7fc8037f1640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a5289a8e60 0x55a528a2c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:30.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.335+0000 7fc8037f1640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a5289a8e60 0x55a528a2c0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:59982/0 (socket says 172.21.15.38:59982) 2024-09-21T03:42:30.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.335+0000 7fc8037f1640 1 -- 172.21.15.38:0/175752509 learned_addr learned my addr 172.21.15.38:0/175752509 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:42:30.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.335+0000 7fc802ff0640 1 --2- 172.21.15.38:0/175752509 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a528961d00 0x55a528a2b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:30.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.335+0000 7fc8027ef640 1 --2- 172.21.15.38:0/175752509 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a52898f5a0 0x55a528a2bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:30.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.335+0000 7fc8037f1640 1 -- 172.21.15.38:0/175752509 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a52898f5a0 msgr2=0x55a528a2bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:30.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.335+0000 7fc8037f1640 1 --2- 172.21.15.38:0/175752509 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a52898f5a0 0x55a528a2bb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:30.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.335+0000 7fc8037f1640 1 -- 172.21.15.38:0/175752509 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a528961d00 msgr2=0x55a528a2b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:30.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.335+0000 7fc8037f1640 1 --2- 172.21.15.38:0/175752509 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a528961d00 0x55a528a2b480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:30.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.335+0000 7fc8037f1640 1 -- 172.21.15.38:0/175752509 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5289a68d0 con 0x55a5289a8e60 2024-09-21T03:42:30.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.335+0000 7fc8027ef640 1 --2- 172.21.15.38:0/175752509 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a52898f5a0 0x55a528a2bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:42:30.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.335+0000 7fc8037f1640 1 --2- 172.21.15.38:0/175752509 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a5289a8e60 0x55a528a2c0d0 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7fc7fc00d600 tx=0x7fc7fc00dac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:30.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.335+0000 7fc7ebfff640 1 -- 172.21.15.38:0/175752509 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7fc010a00 con 0x55a5289a8e60 2024-09-21T03:42:30.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.335+0000 7fc7ebfff640 1 -- 172.21.15.38:0/175752509 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc7fc014070 con 0x55a5289a8e60 2024-09-21T03:42:30.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.335+0000 7fc809536f40 1 -- 172.21.15.38:0/175752509 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55a528ace9a0 con 0x55a5289a8e60 2024-09-21T03:42:30.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.336+0000 7fc7ebfff640 1 -- 172.21.15.38:0/175752509 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7fc019470 con 0x55a5289a8e60 2024-09-21T03:42:30.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.336+0000 7fc809536f40 1 -- 172.21.15.38:0/175752509 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5289a4710 con 0x55a5289a8e60 2024-09-21T03:42:30.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.337+0000 7fc7ebfff640 1 -- 172.21.15.38:0/175752509 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc7fc0223f0 con 0x55a5289a8e60 2024-09-21T03:42:30.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.338+0000 7fc7ebfff640 1 --2- 172.21.15.38:0/175752509 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc7f0077570 0x7fc7f0079a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:30.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.338+0000 7fc7ebfff640 1 -- 172.21.15.38:0/175752509 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 8110+0+0 (secure 0 0 0) 0x7fc7fc09b4a0 con 0x55a5289a8e60 2024-09-21T03:42:30.341 INFO:teuthology.orchestra.run.smithi038.stdout:Importing pool 2024-09-21T03:42:30.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.338+0000 7fc809536f40 1 --2- 172.21.15.38:0/175752509 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x55a528ad8e70 0x55a528adb290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:30.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.338+0000 7fc809536f40 1 -- 172.21.15.38:0/175752509 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x55a52898c2b0 con 0x55a528ad8e70 2024-09-21T03:42:30.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.338+0000 7fc802ff0640 1 --2- 172.21.15.38:0/175752509 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc7f0077570 0x7fc7f0079a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:30.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.338+0000 7fc8027ef640 1 --2- 172.21.15.38:0/175752509 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x55a528ad8e70 0x55a528adb290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:30.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.339+0000 7fc802ff0640 1 --2- 172.21.15.38:0/175752509 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc7f0077570 0x7fc7f0079a80 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fc7f4000c00 tx=0x7fc7f4001690 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:30.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.339+0000 7fc8027ef640 1 --2- 172.21.15.38:0/175752509 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x55a528ad8e70 0x55a528adb290 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:30.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.341+0000 7fc8027ef640 1 -- 172.21.15.38:0/175752509 <== osd.7 v2:172.21.15.195:6818/1606106342 1 ==== osd_op_reply(1 601.00000000 [create] v117'629 uv603 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fc7f805f030 con 0x55a528ad8e70 2024-09-21T03:42:30.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.341+0000 7fc809536f40 1 -- 172.21.15.38:0/175752509 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e117) -- 0x55a528adb7d0 con 0x55a528ad8e70 2024-09-21T03:42:30.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.342+0000 7fc8027ef640 1 -- 172.21.15.38:0/175752509 <== osd.7 v2:172.21.15.195:6818/1606106342 2 ==== osd_op_reply(2 601.00000000 [delete] v117'630 uv630 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc7f805f030 con 0x55a528ad8e70 2024-09-21T03:42:30.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.342+0000 7fc809536f40 1 -- 172.21.15.38:0/175752509 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x55a528adbb70 con 0x55a528ad8e70 2024-09-21T03:42:30.345 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-09-21T03:42:30.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.344+0000 7fc8027ef640 1 -- 172.21.15.38:0/175752509 <== osd.7 v2:172.21.15.195:6818/1606106342 3 ==== osd_op_reply(3 601.00000000 [create] v117'631 uv631 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc7f805f030 con 0x55a528ad8e70 2024-09-21T03:42:30.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.344+0000 7fc809536f40 1 -- 172.21.15.38:0/175752509 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x55a528adcf20 con 0x55a528ad8e70 2024-09-21T03:42:30.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.345+0000 7fc8027ef640 1 -- 172.21.15.38:0/175752509 <== osd.7 v2:172.21.15.195:6818/1606106342 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v117'632 uv632 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc7f805f030 con 0x55a528ad8e70 2024-09-21T03:42:30.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.345+0000 7fc809536f40 1 -- 172.21.15.38:0/175752509 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x55a528a31cf0 con 0x55a528ad8e70 2024-09-21T03:42:30.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.347+0000 7fc8027ef640 1 -- 172.21.15.38:0/175752509 <== osd.7 v2:172.21.15.195:6818/1606106342 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v117'633 uv633 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc7f805f030 con 0x55a528ad8e70 2024-09-21T03:42:30.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.347+0000 7fc809536f40 1 -- 172.21.15.38:0/175752509 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x55a528adbf10 con 0x55a528ad8e70 2024-09-21T03:42:30.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.348+0000 7fc8027ef640 1 -- 172.21.15.38:0/175752509 <== osd.7 v2:172.21.15.195:6818/1606106342 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v117'634 uv634 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc7f805f030 con 0x55a528ad8e70 2024-09-21T03:42:30.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.348+0000 7fc809536f40 1 -- 172.21.15.38:0/175752509 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x55a528adc2b0 con 0x55a528ad8e70 2024-09-21T03:42:30.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.350+0000 7fc8027ef640 1 -- 172.21.15.38:0/175752509 <== osd.7 v2:172.21.15.195:6818/1606106342 7 ==== osd_op_reply(7 602.00000000 [create] v117'635 uv609 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fc7f805f030 con 0x55a528ad8e70 2024-09-21T03:42:30.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.350+0000 7fc809536f40 1 -- 172.21.15.38:0/175752509 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e117) -- 0x55a528adc2b0 con 0x55a528ad8e70 2024-09-21T03:42:30.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.351+0000 7fc8027ef640 1 -- 172.21.15.38:0/175752509 <== osd.7 v2:172.21.15.195:6818/1606106342 8 ==== osd_op_reply(8 602.00000000 [delete] v117'636 uv636 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc7f805f030 con 0x55a528ad8e70 2024-09-21T03:42:30.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.351+0000 7fc809536f40 1 -- 172.21.15.38:0/175752509 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x55a528adc2b0 con 0x55a528ad8e70 2024-09-21T03:42:30.354 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-09-21T03:42:30.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.352+0000 7fc8027ef640 1 -- 172.21.15.38:0/175752509 <== osd.7 v2:172.21.15.195:6818/1606106342 9 ==== osd_op_reply(9 602.00000000 [create] v117'637 uv637 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc7f805f030 con 0x55a528ad8e70 2024-09-21T03:42:30.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.353+0000 7fc809536f40 1 -- 172.21.15.38:0/175752509 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x55a528adc2b0 con 0x55a528ad8e70 2024-09-21T03:42:30.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.354+0000 7fc8027ef640 1 -- 172.21.15.38:0/175752509 <== osd.7 v2:172.21.15.195:6818/1606106342 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v117'638 uv638 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc7f805f030 con 0x55a528ad8e70 2024-09-21T03:42:30.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.354+0000 7fc809536f40 1 -- 172.21.15.38:0/175752509 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x55a528adc2b0 con 0x55a528ad8e70 2024-09-21T03:42:30.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.356+0000 7fc8027ef640 1 -- 172.21.15.38:0/175752509 <== osd.7 v2:172.21.15.195:6818/1606106342 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v117'639 uv639 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc7f805f030 con 0x55a528ad8e70 2024-09-21T03:42:30.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.356+0000 7fc809536f40 1 -- 172.21.15.38:0/175752509 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x55a528adc2b0 con 0x55a528ad8e70 2024-09-21T03:42:30.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.357+0000 7fc8027ef640 1 -- 172.21.15.38:0/175752509 <== osd.7 v2:172.21.15.195:6818/1606106342 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v117'640 uv640 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc7f805f030 con 0x55a528ad8e70 2024-09-21T03:42:30.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.357+0000 7fc809536f40 1 -- 172.21.15.38:0/175752509 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:13 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x55a528adc2b0 con 0x55a528ad8e70 2024-09-21T03:42:30.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.359+0000 7fc8027ef640 1 -- 172.21.15.38:0/175752509 <== osd.7 v2:172.21.15.195:6818/1606106342 13 ==== osd_op_reply(13 10000000000.00000000 [create] v117'641 uv616 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7fc7f805f030 con 0x55a528ad8e70 2024-09-21T03:42:30.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.359+0000 7fc809536f40 1 -- 172.21.15.38:0/175752509 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:14 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e117) -- 0x55a528adc2b0 con 0x55a528ad8e70 2024-09-21T03:42:30.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.360+0000 7fc8027ef640 1 -- 172.21.15.38:0/175752509 <== osd.7 v2:172.21.15.195:6818/1606106342 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v117'642 uv642 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fc7f805f030 con 0x55a528ad8e70 2024-09-21T03:42:30.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.360+0000 7fc809536f40 1 -- 172.21.15.38:0/175752509 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:15 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x55a528adc2b0 con 0x55a528ad8e70 2024-09-21T03:42:30.363 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-09-21T03:42:30.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.362+0000 7fc8027ef640 1 -- 172.21.15.38:0/175752509 <== osd.7 v2:172.21.15.195:6818/1606106342 15 ==== osd_op_reply(15 10000000000.00000000 [create] v117'643 uv643 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fc7f805f030 con 0x55a528ad8e70 2024-09-21T03:42:30.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.362+0000 7fc809536f40 1 -- 172.21.15.38:0/175752509 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:16 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x55a528adc2b0 con 0x55a528ad8e70 2024-09-21T03:42:30.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.363+0000 7fc8027ef640 1 -- 172.21.15.38:0/175752509 <== osd.7 v2:172.21.15.195:6818/1606106342 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v117'644 uv644 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fc7f805f030 con 0x55a528ad8e70 2024-09-21T03:42:30.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.363+0000 7fc809536f40 1 -- 172.21.15.38:0/175752509 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:17 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x55a528adc2b0 con 0x55a528ad8e70 2024-09-21T03:42:30.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.364+0000 7fc8027ef640 1 -- 172.21.15.38:0/175752509 <== osd.7 v2:172.21.15.195:6818/1606106342 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v117'645 uv645 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fc7f805f030 con 0x55a528ad8e70 2024-09-21T03:42:30.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.364+0000 7fc809536f40 1 -- 172.21.15.38:0/175752509 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:18 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x55a528adc2b0 con 0x55a528ad8e70 2024-09-21T03:42:30.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.366+0000 7fc8027ef640 1 -- 172.21.15.38:0/175752509 <== osd.7 v2:172.21.15.195:6818/1606106342 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v117'646 uv646 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fc7f805f030 con 0x55a528ad8e70 2024-09-21T03:42:30.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.366+0000 7fc809536f40 1 -- 172.21.15.38:0/175752509 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:19 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=585b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x55a528adc2b0 con 0x55a528ad8e70 2024-09-21T03:42:30.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.367+0000 7fc8027ef640 1 -- 172.21.15.38:0/175752509 <== osd.7 v2:172.21.15.195:6818/1606106342 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v117'647 uv647 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fc7f805f030 con 0x55a528ad8e70 2024-09-21T03:42:30.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.367+0000 7fc809536f40 1 -- 172.21.15.38:0/175752509 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:20 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x55a528adc2b0 con 0x55a528ad8e70 2024-09-21T03:42:30.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.369+0000 7fc8027ef640 1 -- 172.21.15.38:0/175752509 <== osd.7 v2:172.21.15.195:6818/1606106342 20 ==== osd_op_reply(20 600.00000000 [create] v117'648 uv622 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fc7f805f030 con 0x55a528ad8e70 2024-09-21T03:42:30.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.369+0000 7fc809536f40 1 -- 172.21.15.38:0/175752509 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:21 13.10 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e117) -- 0x55a528adc2b0 con 0x55a528ad8e70 2024-09-21T03:42:30.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.370+0000 7fc8027ef640 1 -- 172.21.15.38:0/175752509 <== osd.7 v2:172.21.15.195:6818/1606106342 21 ==== osd_op_reply(21 600.00000000 [delete] v117'649 uv649 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc7f805f030 con 0x55a528ad8e70 2024-09-21T03:42:30.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.370+0000 7fc809536f40 1 -- 172.21.15.38:0/175752509 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:22 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x55a528adc2b0 con 0x55a528ad8e70 2024-09-21T03:42:30.373 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-09-21T03:42:30.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.371+0000 7fc8027ef640 1 -- 172.21.15.38:0/175752509 <== osd.7 v2:172.21.15.195:6818/1606106342 22 ==== osd_op_reply(22 600.00000000 [create] v117'650 uv650 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc7f805f030 con 0x55a528ad8e70 2024-09-21T03:42:30.374 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.372+0000 7fc809536f40 1 -- 172.21.15.38:0/175752509 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:23 13.10 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x55a528adc2b0 con 0x55a528ad8e70 2024-09-21T03:42:30.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.373+0000 7fc8027ef640 1 -- 172.21.15.38:0/175752509 <== osd.7 v2:172.21.15.195:6818/1606106342 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v117'651 uv651 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc7f805f030 con 0x55a528ad8e70 2024-09-21T03:42:30.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.373+0000 7fc809536f40 1 -- 172.21.15.38:0/175752509 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:24 13.10 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x55a528adc2b0 con 0x55a528ad8e70 2024-09-21T03:42:30.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.374+0000 7fc8027ef640 1 -- 172.21.15.38:0/175752509 <== osd.7 v2:172.21.15.195:6818/1606106342 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v117'652 uv652 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc7f805f030 con 0x55a528ad8e70 2024-09-21T03:42:30.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.374+0000 7fc809536f40 1 -- 172.21.15.38:0/175752509 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:25 13.10 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x55a528adc2b0 con 0x55a528ad8e70 2024-09-21T03:42:30.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.376+0000 7fc8027ef640 1 -- 172.21.15.38:0/175752509 <== osd.7 v2:172.21.15.195:6818/1606106342 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v117'653 uv653 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc7f805f030 con 0x55a528ad8e70 2024-09-21T03:42:30.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.376+0000 7fc809536f40 1 -- 172.21.15.38:0/175752509 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:26 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x55a528adc2b0 con 0x55a528ad8e70 2024-09-21T03:42:30.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.378+0000 7fc8027ef640 1 -- 172.21.15.38:0/175752509 <== osd.7 v2:172.21.15.195:6818/1606106342 26 ==== osd_op_reply(26 603.00000000 [create] v117'654 uv628 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fc7f805f030 con 0x55a528ad8e70 2024-09-21T03:42:30.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.378+0000 7fc809536f40 1 -- 172.21.15.38:0/175752509 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:27 13.10 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e117) -- 0x55a528adc2b0 con 0x55a528ad8e70 2024-09-21T03:42:30.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.380+0000 7fc8027ef640 1 -- 172.21.15.38:0/175752509 <== osd.7 v2:172.21.15.195:6818/1606106342 27 ==== osd_op_reply(27 603.00000000 [delete] v117'655 uv655 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc7f805f030 con 0x55a528ad8e70 2024-09-21T03:42:30.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.380+0000 7fc809536f40 1 -- 172.21.15.38:0/175752509 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:28 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x55a528adc2b0 con 0x55a528ad8e70 2024-09-21T03:42:30.382 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-09-21T03:42:30.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.380+0000 7fc8027ef640 1 -- 172.21.15.38:0/175752509 <== osd.7 v2:172.21.15.195:6818/1606106342 28 ==== osd_op_reply(28 603.00000000 [create] v117'656 uv656 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc7f805f030 con 0x55a528ad8e70 2024-09-21T03:42:30.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.381+0000 7fc809536f40 1 -- 172.21.15.38:0/175752509 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:29 13.10 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x55a528adc2b0 con 0x55a528ad8e70 2024-09-21T03:42:30.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.383+0000 7fc8027ef640 1 -- 172.21.15.38:0/175752509 <== osd.7 v2:172.21.15.195:6818/1606106342 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v117'657 uv657 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc7f805f030 con 0x55a528ad8e70 2024-09-21T03:42:30.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.383+0000 7fc809536f40 1 -- 172.21.15.38:0/175752509 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:30 13.10 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x55a528adc2b0 con 0x55a528ad8e70 2024-09-21T03:42:30.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.384+0000 7fc8027ef640 1 -- 172.21.15.38:0/175752509 <== osd.7 v2:172.21.15.195:6818/1606106342 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v117'658 uv658 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc7f805f030 con 0x55a528ad8e70 2024-09-21T03:42:30.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.384+0000 7fc809536f40 1 -- 172.21.15.38:0/175752509 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:31 13.10 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x55a528adc2b0 con 0x55a528ad8e70 2024-09-21T03:42:30.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.385+0000 7fc8027ef640 1 -- 172.21.15.38:0/175752509 <== osd.7 v2:172.21.15.195:6818/1606106342 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v117'659 uv659 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc7f805f030 con 0x55a528ad8e70 2024-09-21T03:42:30.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.385+0000 7fc809536f40 1 --2- 172.21.15.38:0/175752509 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x55a528add2c0 0x55a528adf6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:30.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.386+0000 7fc809536f40 1 -- 172.21.15.38:0/175752509 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x55a528adc2b0 con 0x55a528add2c0 2024-09-21T03:42:30.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.386+0000 7fc802ff0640 1 --2- 172.21.15.38:0/175752509 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x55a528add2c0 0x55a528adf6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:30.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.386+0000 7fc802ff0640 1 --2- 172.21.15.38:0/175752509 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x55a528add2c0 0x55a528adf6e0 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:30.390 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.388+0000 7fc802ff0640 1 -- 172.21.15.38:0/175752509 <== osd.6 v2:172.21.15.195:6826/4055825675 1 ==== osd_op_reply(32 1.00000000.inode [create] v117'462 uv445 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7fc7f401b040 con 0x55a528add2c0 2024-09-21T03:42:30.390 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.388+0000 7fc809536f40 1 -- 172.21.15.38:0/175752509 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e117) -- 0x55a528adcb80 con 0x55a528add2c0 2024-09-21T03:42:30.391 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.389+0000 7fc802ff0640 1 -- 172.21.15.38:0/175752509 <== osd.6 v2:172.21.15.195:6826/4055825675 2 ==== osd_op_reply(33 1.00000000.inode [delete] v117'463 uv463 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fc7f401b040 con 0x55a528add2c0 2024-09-21T03:42:30.391 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.389+0000 7fc809536f40 1 -- 172.21.15.38:0/175752509 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x55a528adfc20 con 0x55a528add2c0 2024-09-21T03:42:30.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.391+0000 7fc802ff0640 1 -- 172.21.15.38:0/175752509 <== osd.6 v2:172.21.15.195:6826/4055825675 3 ==== osd_op_reply(34 1.00000000.inode [create] v117'464 uv464 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fc7f401b040 con 0x55a528add2c0 2024-09-21T03:42:30.393 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-09-21T03:42:30.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.391+0000 7fc809536f40 1 -- 172.21.15.38:0/175752509 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x55a528adffc0 con 0x55a528add2c0 2024-09-21T03:42:30.394 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.393+0000 7fc802ff0640 1 -- 172.21.15.38:0/175752509 <== osd.6 v2:172.21.15.195:6826/4055825675 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v117'465 uv465 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fc7f401b040 con 0x55a528add2c0 2024-09-21T03:42:30.394 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.393+0000 7fc809536f40 1 -- 172.21.15.38:0/175752509 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x55a528ae0360 con 0x55a528add2c0 2024-09-21T03:42:30.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.394+0000 7fc802ff0640 1 -- 172.21.15.38:0/175752509 <== osd.6 v2:172.21.15.195:6826/4055825675 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v117'466 uv466 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fc7f401b040 con 0x55a528add2c0 2024-09-21T03:42:30.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.394+0000 7fc809536f40 1 -- 172.21.15.38:0/175752509 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x55a528ae0700 con 0x55a528add2c0 2024-09-21T03:42:30.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.396+0000 7fc802ff0640 1 -- 172.21.15.38:0/175752509 <== osd.6 v2:172.21.15.195:6826/4055825675 6 ==== osd_op_reply(37 200.00000000 [create] v117'467 uv450 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fc7f401b040 con 0x55a528add2c0 2024-09-21T03:42:30.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.396+0000 7fc809536f40 1 -- 172.21.15.38:0/175752509 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e117) -- 0x55a528ae0aa0 con 0x55a528add2c0 2024-09-21T03:42:30.399 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.397+0000 7fc802ff0640 1 -- 172.21.15.38:0/175752509 <== osd.6 v2:172.21.15.195:6826/4055825675 7 ==== osd_op_reply(38 200.00000000 [delete] v117'468 uv468 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc7f401b040 con 0x55a528add2c0 2024-09-21T03:42:30.399 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.397+0000 7fc809536f40 1 -- 172.21.15.38:0/175752509 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x55a528ae0e40 con 0x55a528add2c0 2024-09-21T03:42:30.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.398+0000 7fc802ff0640 1 -- 172.21.15.38:0/175752509 <== osd.6 v2:172.21.15.195:6826/4055825675 8 ==== osd_op_reply(39 200.00000000 [create] v117'469 uv469 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc7f401b040 con 0x55a528add2c0 2024-09-21T03:42:30.400 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-09-21T03:42:30.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.398+0000 7fc809536f40 1 -- 172.21.15.38:0/175752509 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x55a528ae0e40 con 0x55a528add2c0 2024-09-21T03:42:30.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.400+0000 7fc802ff0640 1 -- 172.21.15.38:0/175752509 <== osd.6 v2:172.21.15.195:6826/4055825675 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v117'470 uv470 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc7f401b040 con 0x55a528add2c0 2024-09-21T03:42:30.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.400+0000 7fc809536f40 1 -- 172.21.15.38:0/175752509 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x55a528ae0e40 con 0x55a528add2c0 2024-09-21T03:42:30.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.401+0000 7fc802ff0640 1 -- 172.21.15.38:0/175752509 <== osd.6 v2:172.21.15.195:6826/4055825675 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v117'471 uv471 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc7f401b040 con 0x55a528add2c0 2024-09-21T03:42:30.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.401+0000 7fc809536f40 1 -- 172.21.15.38:0/175752509 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:42 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x55a528ae0e40 con 0x55a528add2c0 2024-09-21T03:42:30.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.403+0000 7fc802ff0640 1 -- 172.21.15.38:0/175752509 <== osd.6 v2:172.21.15.195:6826/4055825675 11 ==== osd_op_reply(42 200.00000001 [create] v117'472 uv455 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fc7f401b040 con 0x55a528add2c0 2024-09-21T03:42:30.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.403+0000 7fc809536f40 1 -- 172.21.15.38:0/175752509 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:43 13.14 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e117) -- 0x55a528ae0e40 con 0x55a528add2c0 2024-09-21T03:42:30.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.404+0000 7fc802ff0640 1 -- 172.21.15.38:0/175752509 <== osd.6 v2:172.21.15.195:6826/4055825675 12 ==== osd_op_reply(43 200.00000001 [delete] v117'473 uv473 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc7f401b040 con 0x55a528add2c0 2024-09-21T03:42:30.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.404+0000 7fc809536f40 1 -- 172.21.15.38:0/175752509 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:44 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x55a528ae0e40 con 0x55a528add2c0 2024-09-21T03:42:30.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.406+0000 7fc802ff0640 1 -- 172.21.15.38:0/175752509 <== osd.6 v2:172.21.15.195:6826/4055825675 13 ==== osd_op_reply(44 200.00000001 [create] v117'474 uv474 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc7f401b040 con 0x55a528add2c0 2024-09-21T03:42:30.408 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-09-21T03:42:30.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.406+0000 7fc809536f40 1 -- 172.21.15.38:0/175752509 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:45 13.14 13:2e2fa760:::200.00000001:head [write 0~17490 in=17490b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x55a528ae52c0 con 0x55a528add2c0 2024-09-21T03:42:30.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.408+0000 7fc802ff0640 1 -- 172.21.15.38:0/175752509 <== osd.6 v2:172.21.15.195:6826/4055825675 14 ==== osd_op_reply(45 200.00000001 [write 0~17490] v117'475 uv475 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc7f401b040 con 0x55a528add2c0 2024-09-21T03:42:30.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.408+0000 7fc809536f40 1 -- 172.21.15.38:0/175752509 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:46 13.14 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x55a528ae52c0 con 0x55a528add2c0 2024-09-21T03:42:30.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.409+0000 7fc802ff0640 1 -- 172.21.15.38:0/175752509 <== osd.6 v2:172.21.15.195:6826/4055825675 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v117'476 uv476 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc7f401b040 con 0x55a528add2c0 2024-09-21T03:42:30.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.409+0000 7fc809536f40 1 -- 172.21.15.38:0/175752509 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:47 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x55a528ae0e40 con 0x55a528add2c0 2024-09-21T03:42:30.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.410+0000 7fc802ff0640 1 -- 172.21.15.38:0/175752509 <== osd.6 v2:172.21.15.195:6826/4055825675 16 ==== osd_op_reply(47 606.00000000 [create] v117'477 uv461 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fc7f401b040 con 0x55a528add2c0 2024-09-21T03:42:30.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.410+0000 7fc809536f40 1 -- 172.21.15.38:0/175752509 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:48 13.14 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e117) -- 0x55a528ae0e40 con 0x55a528add2c0 2024-09-21T03:42:30.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.412+0000 7fc802ff0640 1 -- 172.21.15.38:0/175752509 <== osd.6 v2:172.21.15.195:6826/4055825675 17 ==== osd_op_reply(48 606.00000000 [delete] v117'478 uv478 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc7f401b040 con 0x55a528add2c0 2024-09-21T03:42:30.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.412+0000 7fc809536f40 1 -- 172.21.15.38:0/175752509 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:49 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x55a528ae0e40 con 0x55a528add2c0 2024-09-21T03:42:30.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.413+0000 7fc802ff0640 1 -- 172.21.15.38:0/175752509 <== osd.6 v2:172.21.15.195:6826/4055825675 18 ==== osd_op_reply(49 606.00000000 [create] v117'479 uv479 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc7f401b040 con 0x55a528add2c0 2024-09-21T03:42:30.415 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-09-21T03:42:30.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.413+0000 7fc809536f40 1 -- 172.21.15.38:0/175752509 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:50 13.14 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x55a528ae1e50 con 0x55a528add2c0 2024-09-21T03:42:30.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.415+0000 7fc802ff0640 1 -- 172.21.15.38:0/175752509 <== osd.6 v2:172.21.15.195:6826/4055825675 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v117'480 uv480 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc7f401b040 con 0x55a528add2c0 2024-09-21T03:42:30.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.415+0000 7fc809536f40 1 -- 172.21.15.38:0/175752509 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:51 13.14 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x55a528ae1e50 con 0x55a528add2c0 2024-09-21T03:42:30.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.416+0000 7fc802ff0640 1 -- 172.21.15.38:0/175752509 <== osd.6 v2:172.21.15.195:6826/4055825675 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v117'481 uv481 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc7f401b040 con 0x55a528add2c0 2024-09-21T03:42:30.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.416+0000 7fc809536f40 1 -- 172.21.15.38:0/175752509 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:52 13.14 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x55a528ae0e40 con 0x55a528add2c0 2024-09-21T03:42:30.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.418+0000 7fc802ff0640 1 -- 172.21.15.38:0/175752509 <== osd.6 v2:172.21.15.195:6826/4055825675 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v117'482 uv482 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc7f401b040 con 0x55a528add2c0 2024-09-21T03:42:30.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.418+0000 7fc809536f40 1 --2- 172.21.15.38:0/175752509 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x55a528ae14c0 0x55a528ae38e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:30.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.418+0000 7fc809536f40 1 -- 172.21.15.38:0/175752509 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:53 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x55a528ae3ef0 con 0x55a528ae14c0 2024-09-21T03:42:30.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.418+0000 7fc8027ef640 1 --2- 172.21.15.38:0/175752509 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x55a528ae14c0 0x55a528ae38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:30.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.418+0000 7fc8027ef640 1 --2- 172.21.15.38:0/175752509 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x55a528ae14c0 0x55a528ae38e0 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:30.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.420+0000 7fc8027ef640 1 -- 172.21.15.38:0/175752509 <== osd.0 v2:172.21.15.38:6800/2381639433 1 ==== osd_op_reply(53 607.00000000 [create] v117'123 uv122 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fc7f805f030 con 0x55a528ae14c0 2024-09-21T03:42:30.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.420+0000 7fc809536f40 1 -- 172.21.15.38:0/175752509 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:54 13.1c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e117) -- 0x55a528ae3fb0 con 0x55a528ae14c0 2024-09-21T03:42:30.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.421+0000 7fc8027ef640 1 -- 172.21.15.38:0/175752509 <== osd.0 v2:172.21.15.38:6800/2381639433 2 ==== osd_op_reply(54 607.00000000 [delete] v117'124 uv124 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc7f805f030 con 0x55a528ae14c0 2024-09-21T03:42:30.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.421+0000 7fc809536f40 1 -- 172.21.15.38:0/175752509 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:55 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x55a528ae43d0 con 0x55a528ae14c0 2024-09-21T03:42:30.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.422+0000 7fc8027ef640 1 -- 172.21.15.38:0/175752509 <== osd.0 v2:172.21.15.38:6800/2381639433 3 ==== osd_op_reply(55 607.00000000 [create] v117'125 uv125 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc7f805f030 con 0x55a528ae14c0 2024-09-21T03:42:30.424 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-09-21T03:42:30.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.422+0000 7fc809536f40 1 -- 172.21.15.38:0/175752509 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:56 13.1c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x55a528ae53e0 con 0x55a528ae14c0 2024-09-21T03:42:30.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.423+0000 7fc8027ef640 1 -- 172.21.15.38:0/175752509 <== osd.0 v2:172.21.15.38:6800/2381639433 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v117'126 uv126 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc7f805f030 con 0x55a528ae14c0 2024-09-21T03:42:30.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.423+0000 7fc809536f40 1 -- 172.21.15.38:0/175752509 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:57 13.1c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x55a528ae53e0 con 0x55a528ae14c0 2024-09-21T03:42:30.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.424+0000 7fc8027ef640 1 -- 172.21.15.38:0/175752509 <== osd.0 v2:172.21.15.38:6800/2381639433 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v117'127 uv127 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc7f805f030 con 0x55a528ae14c0 2024-09-21T03:42:30.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.424+0000 7fc809536f40 1 -- 172.21.15.38:0/175752509 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:58 13.1c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x55a528ae43d0 con 0x55a528ae14c0 2024-09-21T03:42:30.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.425+0000 7fc8027ef640 1 -- 172.21.15.38:0/175752509 <== osd.0 v2:172.21.15.38:6800/2381639433 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v117'128 uv128 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc7f805f030 con 0x55a528ae14c0 2024-09-21T03:42:30.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.425+0000 7fc809536f40 1 --2- 172.21.15.38:0/175752509 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x55a528ae48e0 0x55a528ae6d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:30.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.425+0000 7fc809536f40 1 -- 172.21.15.38:0/175752509 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:59 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x55a528ae7450 con 0x55a528ae48e0 2024-09-21T03:42:30.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.425+0000 7fc8037f1640 1 --2- 172.21.15.38:0/175752509 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x55a528ae48e0 0x55a528ae6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:30.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.425+0000 7fc8037f1640 1 --2- 172.21.15.38:0/175752509 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x55a528ae48e0 0x55a528ae6d80 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:30.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.427+0000 7fc8037f1640 1 -- 172.21.15.38:0/175752509 <== osd.3 v2:172.21.15.38:6808/1286810294 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v117'90 uv89 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7fc7fc00b040 con 0x55a528ae48e0 2024-09-21T03:42:30.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.427+0000 7fc809536f40 1 -- 172.21.15.38:0/175752509 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:60 13.6 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e117) -- 0x55a528ae7960 con 0x55a528ae48e0 2024-09-21T03:42:30.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.428+0000 7fc8037f1640 1 -- 172.21.15.38:0/175752509 <== osd.3 v2:172.21.15.38:6808/1286810294 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v117'91 uv91 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fc7fc00b040 con 0x55a528ae48e0 2024-09-21T03:42:30.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.428+0000 7fc809536f40 1 -- 172.21.15.38:0/175752509 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:61 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x55a528ae7e20 con 0x55a528ae48e0 2024-09-21T03:42:30.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.429+0000 7fc8037f1640 1 -- 172.21.15.38:0/175752509 <== osd.3 v2:172.21.15.38:6808/1286810294 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v117'92 uv92 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fc7fc00b040 con 0x55a528ae48e0 2024-09-21T03:42:30.431 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-09-21T03:42:30.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.429+0000 7fc809536f40 1 -- 172.21.15.38:0/175752509 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:62 13.6 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x55a5289a5010 con 0x55a528ae48e0 2024-09-21T03:42:30.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.430+0000 7fc8037f1640 1 -- 172.21.15.38:0/175752509 <== osd.3 v2:172.21.15.38:6808/1286810294 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v117'93 uv93 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fc7fc00b040 con 0x55a528ae48e0 2024-09-21T03:42:30.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.430+0000 7fc809536f40 1 -- 172.21.15.38:0/175752509 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x55a528ae87a0 con 0x55a528ae14c0 2024-09-21T03:42:30.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.431+0000 7fc8027ef640 1 -- 172.21.15.38:0/175752509 <== osd.0 v2:172.21.15.38:6800/2381639433 7 ==== osd_op_reply(63 608.00000000 [create] v117'123 uv122 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fc7f805f030 con 0x55a528ae14c0 2024-09-21T03:42:30.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.432+0000 7fc809536f40 1 -- 172.21.15.38:0/175752509 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e117) -- 0x55a528ae87a0 con 0x55a528ae14c0 2024-09-21T03:42:30.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.433+0000 7fc8027ef640 1 -- 172.21.15.38:0/175752509 <== osd.0 v2:172.21.15.38:6800/2381639433 8 ==== osd_op_reply(64 608.00000000 [delete] v117'124 uv124 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc7f805f030 con 0x55a528ae14c0 2024-09-21T03:42:30.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.433+0000 7fc809536f40 1 -- 172.21.15.38:0/175752509 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x55a528ae87a0 con 0x55a528ae14c0 2024-09-21T03:42:30.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.434+0000 7fc8027ef640 1 -- 172.21.15.38:0/175752509 <== osd.0 v2:172.21.15.38:6800/2381639433 9 ==== osd_op_reply(65 608.00000000 [create] v117'125 uv125 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc7f805f030 con 0x55a528ae14c0 2024-09-21T03:42:30.436 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-09-21T03:42:30.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.434+0000 7fc809536f40 1 -- 172.21.15.38:0/175752509 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x55a528ae97b0 con 0x55a528ae14c0 2024-09-21T03:42:30.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.435+0000 7fc8027ef640 1 -- 172.21.15.38:0/175752509 <== osd.0 v2:172.21.15.38:6800/2381639433 10 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v117'126 uv126 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc7f805f030 con 0x55a528ae14c0 2024-09-21T03:42:30.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.435+0000 7fc809536f40 1 -- 172.21.15.38:0/175752509 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x55a528ae97b0 con 0x55a528ae14c0 2024-09-21T03:42:30.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.436+0000 7fc8027ef640 1 -- 172.21.15.38:0/175752509 <== osd.0 v2:172.21.15.38:6800/2381639433 11 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v117'127 uv127 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc7f805f030 con 0x55a528ae14c0 2024-09-21T03:42:30.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.436+0000 7fc809536f40 1 -- 172.21.15.38:0/175752509 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x55a528ae87a0 con 0x55a528ae14c0 2024-09-21T03:42:30.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.437+0000 7fc8027ef640 1 -- 172.21.15.38:0/175752509 <== osd.0 v2:172.21.15.38:6800/2381639433 12 ==== osd_op_reply(68 608.00000000 [omap-set-header] v117'128 uv128 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc7f805f030 con 0x55a528ae14c0 2024-09-21T03:42:30.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.437+0000 7fc809536f40 1 -- 172.21.15.38:0/175752509 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x55a528ae87a0 con 0x55a528ae48e0 2024-09-21T03:42:30.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.438+0000 7fc8037f1640 1 -- 172.21.15.38:0/175752509 <== osd.3 v2:172.21.15.38:6808/1286810294 5 ==== osd_op_reply(69 604.00000000 [create] v117'227 uv221 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fc7fc00b040 con 0x55a528ae48e0 2024-09-21T03:42:30.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.438+0000 7fc809536f40 1 -- 172.21.15.38:0/175752509 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e117) -- 0x55a528ae8b60 con 0x55a528ae48e0 2024-09-21T03:42:30.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.439+0000 7fc8037f1640 1 -- 172.21.15.38:0/175752509 <== osd.3 v2:172.21.15.38:6808/1286810294 6 ==== osd_op_reply(70 604.00000000 [delete] v117'228 uv228 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc7fc00b040 con 0x55a528ae48e0 2024-09-21T03:42:30.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.439+0000 7fc809536f40 1 -- 172.21.15.38:0/175752509 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x55a528ae8f50 con 0x55a528ae48e0 2024-09-21T03:42:30.442 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-09-21T03:42:30.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.440+0000 7fc8037f1640 1 -- 172.21.15.38:0/175752509 <== osd.3 v2:172.21.15.38:6808/1286810294 7 ==== osd_op_reply(71 604.00000000 [create] v117'229 uv229 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc7fc00b040 con 0x55a528ae48e0 2024-09-21T03:42:30.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.440+0000 7fc809536f40 1 -- 172.21.15.38:0/175752509 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x55a528aea340 con 0x55a528ae48e0 2024-09-21T03:42:30.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.441+0000 7fc8037f1640 1 -- 172.21.15.38:0/175752509 <== osd.3 v2:172.21.15.38:6808/1286810294 8 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v117'230 uv230 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc7fc00b040 con 0x55a528ae48e0 2024-09-21T03:42:30.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.441+0000 7fc809536f40 1 -- 172.21.15.38:0/175752509 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x55a528aea300 con 0x55a528ae48e0 2024-09-21T03:42:30.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.443+0000 7fc8037f1640 1 -- 172.21.15.38:0/175752509 <== osd.3 v2:172.21.15.38:6808/1286810294 9 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v117'231 uv231 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc7fc00b040 con 0x55a528ae48e0 2024-09-21T03:42:30.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.443+0000 7fc809536f40 1 -- 172.21.15.38:0/175752509 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x55a528ae92f0 con 0x55a528ae48e0 2024-09-21T03:42:30.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.444+0000 7fc8037f1640 1 -- 172.21.15.38:0/175752509 <== osd.3 v2:172.21.15.38:6808/1286810294 10 ==== osd_op_reply(74 604.00000000 [omap-set-header] v117'232 uv232 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc7fc00b040 con 0x55a528ae48e0 2024-09-21T03:42:30.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.444+0000 7fc809536f40 1 -- 172.21.15.38:0/175752509 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x55a528ae92f0 con 0x55a528ae48e0 2024-09-21T03:42:30.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.445+0000 7fc8037f1640 1 -- 172.21.15.38:0/175752509 <== osd.3 v2:172.21.15.38:6808/1286810294 11 ==== osd_op_reply(75 500.00000000 [create] v117'233 uv226 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fc7fc00b040 con 0x55a528ae48e0 2024-09-21T03:42:30.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.445+0000 7fc809536f40 1 -- 172.21.15.38:0/175752509 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e117) -- 0x55a528ae92f0 con 0x55a528ae48e0 2024-09-21T03:42:30.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.446+0000 7fc8037f1640 1 -- 172.21.15.38:0/175752509 <== osd.3 v2:172.21.15.38:6808/1286810294 12 ==== osd_op_reply(76 500.00000000 [delete] v117'234 uv234 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc7fc00b040 con 0x55a528ae48e0 2024-09-21T03:42:30.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.446+0000 7fc809536f40 1 -- 172.21.15.38:0/175752509 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x55a528ae92f0 con 0x55a528ae48e0 2024-09-21T03:42:30.448 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-09-21T03:42:30.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.447+0000 7fc8037f1640 1 -- 172.21.15.38:0/175752509 <== osd.3 v2:172.21.15.38:6808/1286810294 13 ==== osd_op_reply(77 500.00000000 [create] v117'235 uv235 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc7fc00b040 con 0x55a528ae48e0 2024-09-21T03:42:30.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.447+0000 7fc809536f40 1 -- 172.21.15.38:0/175752509 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x55a528ae92f0 con 0x55a528ae48e0 2024-09-21T03:42:30.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.448+0000 7fc8037f1640 1 -- 172.21.15.38:0/175752509 <== osd.3 v2:172.21.15.38:6808/1286810294 14 ==== osd_op_reply(78 500.00000000 [write 0~90] v117'236 uv236 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc7fc00b040 con 0x55a528ae48e0 2024-09-21T03:42:30.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.448+0000 7fc809536f40 1 -- 172.21.15.38:0/175752509 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x55a528ae92f0 con 0x55a528ae48e0 2024-09-21T03:42:30.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.449+0000 7fc8037f1640 1 -- 172.21.15.38:0/175752509 <== osd.3 v2:172.21.15.38:6808/1286810294 15 ==== osd_op_reply(79 500.00000000 [omap-set-header] v117'237 uv237 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc7fc00b040 con 0x55a528ae48e0 2024-09-21T03:42:30.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.449+0000 7fc809536f40 1 -- 172.21.15.38:0/175752509 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:80 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x55a528ae92f0 con 0x55a528ae14c0 2024-09-21T03:42:30.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.450+0000 7fc8027ef640 1 -- 172.21.15.38:0/175752509 <== osd.0 v2:172.21.15.38:6800/2381639433 13 ==== osd_op_reply(80 mds_snaptable [create] v117'102 uv101 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7fc7f805f030 con 0x55a528ae14c0 2024-09-21T03:42:30.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.450+0000 7fc809536f40 1 -- 172.21.15.38:0/175752509 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:81 13.d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e117) -- 0x55a528ae92f0 con 0x55a528ae14c0 2024-09-21T03:42:30.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.451+0000 7fc8027ef640 1 -- 172.21.15.38:0/175752509 <== osd.0 v2:172.21.15.38:6800/2381639433 14 ==== osd_op_reply(81 mds_snaptable [delete] v117'103 uv103 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fc7f805f030 con 0x55a528ae14c0 2024-09-21T03:42:30.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.452+0000 7fc809536f40 1 -- 172.21.15.38:0/175752509 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:82 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x55a528ae92f0 con 0x55a528ae14c0 2024-09-21T03:42:30.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.452+0000 7fc8027ef640 1 -- 172.21.15.38:0/175752509 <== osd.0 v2:172.21.15.38:6800/2381639433 15 ==== osd_op_reply(82 mds_snaptable [create] v117'104 uv104 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fc7f805f030 con 0x55a528ae14c0 2024-09-21T03:42:30.454 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-09-21T03:42:30.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.453+0000 7fc809536f40 1 -- 172.21.15.38:0/175752509 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:83 13.d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x55a528ae92f0 con 0x55a528ae14c0 2024-09-21T03:42:30.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.454+0000 7fc8027ef640 1 -- 172.21.15.38:0/175752509 <== osd.0 v2:172.21.15.38:6800/2381639433 16 ==== osd_op_reply(83 mds_snaptable [write 0~70] v117'105 uv105 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fc7f805f030 con 0x55a528ae14c0 2024-09-21T03:42:30.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.454+0000 7fc809536f40 1 -- 172.21.15.38:0/175752509 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:84 13.d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x55a528ae92f0 con 0x55a528ae14c0 2024-09-21T03:42:30.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.455+0000 7fc8027ef640 1 -- 172.21.15.38:0/175752509 <== osd.0 v2:172.21.15.38:6800/2381639433 17 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v117'106 uv106 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fc7f805f030 con 0x55a528ae14c0 2024-09-21T03:42:30.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.455+0000 7fc809536f40 1 --2- 172.21.15.38:0/175752509 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x55a528ae9c80 0x55a528aea090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:30.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.455+0000 7fc809536f40 1 -- 172.21.15.38:0/175752509 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x55a528aec760 con 0x55a528ae9c80 2024-09-21T03:42:30.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.455+0000 7fc802ff0640 1 --2- 172.21.15.38:0/175752509 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x55a528ae9c80 0x55a528aea090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:30.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.455+0000 7fc802ff0640 1 --2- 172.21.15.38:0/175752509 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x55a528ae9c80 0x55a528aea090 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:30.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.456+0000 7fc802ff0640 1 -- 172.21.15.38:0/175752509 <== osd.1 v2:172.21.15.38:6816/1491818732 1 ==== osd_op_reply(85 605.00000000 [create] v117'123 uv122 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fc7f401b040 con 0x55a528ae9c80 2024-09-21T03:42:30.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.457+0000 7fc809536f40 1 -- 172.21.15.38:0/175752509 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e117) -- 0x55a528aec760 con 0x55a528ae9c80 2024-09-21T03:42:30.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.458+0000 7fc802ff0640 1 -- 172.21.15.38:0/175752509 <== osd.1 v2:172.21.15.38:6816/1491818732 2 ==== osd_op_reply(86 605.00000000 [delete] v117'124 uv124 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc7f401b040 con 0x55a528ae9c80 2024-09-21T03:42:30.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.458+0000 7fc809536f40 1 -- 172.21.15.38:0/175752509 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x55a528aecb80 con 0x55a528ae9c80 2024-09-21T03:42:30.460 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-09-21T03:42:30.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.459+0000 7fc802ff0640 1 -- 172.21.15.38:0/175752509 <== osd.1 v2:172.21.15.38:6816/1491818732 3 ==== osd_op_reply(87 605.00000000 [create] v117'125 uv125 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc7f401b040 con 0x55a528ae9c80 2024-09-21T03:42:30.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.459+0000 7fc809536f40 1 -- 172.21.15.38:0/175752509 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x55a528aedb60 con 0x55a528ae9c80 2024-09-21T03:42:30.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.460+0000 7fc802ff0640 1 -- 172.21.15.38:0/175752509 <== osd.1 v2:172.21.15.38:6816/1491818732 4 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v117'126 uv126 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc7f401b040 con 0x55a528ae9c80 2024-09-21T03:42:30.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.460+0000 7fc809536f40 1 -- 172.21.15.38:0/175752509 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x55a528aedb60 con 0x55a528ae9c80 2024-09-21T03:42:30.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.461+0000 7fc802ff0640 1 -- 172.21.15.38:0/175752509 <== osd.1 v2:172.21.15.38:6816/1491818732 5 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v117'127 uv127 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc7f401b040 con 0x55a528ae9c80 2024-09-21T03:42:30.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.461+0000 7fc809536f40 1 -- 172.21.15.38:0/175752509 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x55a528aecb50 con 0x55a528ae9c80 2024-09-21T03:42:30.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.462+0000 7fc802ff0640 1 -- 172.21.15.38:0/175752509 <== osd.1 v2:172.21.15.38:6816/1491818732 6 ==== osd_op_reply(90 605.00000000 [omap-set-header] v117'128 uv128 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc7f401b040 con 0x55a528ae9c80 2024-09-21T03:42:30.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.462+0000 7fc809536f40 1 -- 172.21.15.38:0/175752509 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x55a528aecb50 con 0x55a528ae9c80 2024-09-21T03:42:30.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.463+0000 7fc802ff0640 1 -- 172.21.15.38:0/175752509 <== osd.1 v2:172.21.15.38:6816/1491818732 7 ==== osd_op_reply(91 mds0_inotable [create] v117'203 uv202 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7fc7f401b040 con 0x55a528ae9c80 2024-09-21T03:42:30.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.463+0000 7fc809536f40 1 -- 172.21.15.38:0/175752509 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e117) -- 0x55a528aecb50 con 0x55a528ae9c80 2024-09-21T03:42:30.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.464+0000 7fc802ff0640 1 -- 172.21.15.38:0/175752509 <== osd.1 v2:172.21.15.38:6816/1491818732 8 ==== osd_op_reply(92 mds0_inotable [delete] v117'204 uv204 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fc7f401b040 con 0x55a528ae9c80 2024-09-21T03:42:30.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.464+0000 7fc809536f40 1 -- 172.21.15.38:0/175752509 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x55a528aecb50 con 0x55a528ae9c80 2024-09-21T03:42:30.467 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-09-21T03:42:30.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.465+0000 7fc802ff0640 1 -- 172.21.15.38:0/175752509 <== osd.1 v2:172.21.15.38:6816/1491818732 9 ==== osd_op_reply(93 mds0_inotable [create] v117'205 uv205 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fc7f401b040 con 0x55a528ae9c80 2024-09-21T03:42:30.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.465+0000 7fc809536f40 1 -- 172.21.15.38:0/175752509 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:94 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x55a528aecb50 con 0x55a528ae9c80 2024-09-21T03:42:30.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.466+0000 7fc802ff0640 1 -- 172.21.15.38:0/175752509 <== osd.1 v2:172.21.15.38:6816/1491818732 10 ==== osd_op_reply(94 mds0_inotable [write 0~34] v117'206 uv206 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fc7f401b040 con 0x55a528ae9c80 2024-09-21T03:42:30.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.466+0000 7fc809536f40 1 -- 172.21.15.38:0/175752509 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:95 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x55a528aecb50 con 0x55a528ae9c80 2024-09-21T03:42:30.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.467+0000 7fc802ff0640 1 -- 172.21.15.38:0/175752509 <== osd.1 v2:172.21.15.38:6816/1491818732 11 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v117'207 uv207 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fc7f401b040 con 0x55a528ae9c80 2024-09-21T03:42:30.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.467+0000 7fc809536f40 1 -- 172.21.15.38:0/175752509 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:96 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x55a528aecb50 con 0x55a528ae9c80 2024-09-21T03:42:30.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.468+0000 7fc802ff0640 1 -- 172.21.15.38:0/175752509 <== osd.1 v2:172.21.15.38:6816/1491818732 12 ==== osd_op_reply(96 100.00000000 [create] v117'208 uv201 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fc7f401b040 con 0x55a528ae9c80 2024-09-21T03:42:30.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.468+0000 7fc809536f40 1 -- 172.21.15.38:0/175752509 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:97 13.13 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e117) -- 0x55a528aecb50 con 0x55a528ae9c80 2024-09-21T03:42:30.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.469+0000 7fc802ff0640 1 -- 172.21.15.38:0/175752509 <== osd.1 v2:172.21.15.38:6816/1491818732 13 ==== osd_op_reply(97 100.00000000 [delete] v117'209 uv209 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc7f401b040 con 0x55a528ae9c80 2024-09-21T03:42:30.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.469+0000 7fc809536f40 1 -- 172.21.15.38:0/175752509 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:98 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x55a528aecb50 con 0x55a528ae9c80 2024-09-21T03:42:30.472 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-09-21T03:42:30.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.470+0000 7fc802ff0640 1 -- 172.21.15.38:0/175752509 <== osd.1 v2:172.21.15.38:6816/1491818732 14 ==== osd_op_reply(98 100.00000000 [create] v117'210 uv210 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc7f401b040 con 0x55a528ae9c80 2024-09-21T03:42:30.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.470+0000 7fc809536f40 1 -- 172.21.15.38:0/175752509 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:99 13.13 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x55a528aecb50 con 0x55a528ae9c80 2024-09-21T03:42:30.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.471+0000 7fc802ff0640 1 -- 172.21.15.38:0/175752509 <== osd.1 v2:172.21.15.38:6816/1491818732 15 ==== osd_op_reply(99 100.00000000 [omap-set-header] v117'211 uv211 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc7f401b040 con 0x55a528ae9c80 2024-09-21T03:42:30.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.471+0000 7fc809536f40 1 -- 172.21.15.38:0/175752509 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:100 13.13 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x55a528aef960 con 0x55a528ae9c80 2024-09-21T03:42:30.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.472+0000 7fc802ff0640 1 -- 172.21.15.38:0/175752509 <== osd.1 v2:172.21.15.38:6816/1491818732 16 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v117'212 uv212 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc7f401b040 con 0x55a528ae9c80 2024-09-21T03:42:30.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.472+0000 7fc809536f40 1 -- 172.21.15.38:0/175752509 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x55a528aed000 con 0x55a528ae48e0 2024-09-21T03:42:30.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.473+0000 7fc8037f1640 1 -- 172.21.15.38:0/175752509 <== osd.3 v2:172.21.15.38:6808/1286810294 16 ==== osd_op_reply(101 mds0_sessionmap [create] v117'91 uv90 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7fc7fc00b040 con 0x55a528ae48e0 2024-09-21T03:42:30.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.473+0000 7fc809536f40 1 -- 172.21.15.38:0/175752509 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e117) -- 0x55a528aed000 con 0x55a528ae48e0 2024-09-21T03:42:30.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.474+0000 7fc8037f1640 1 -- 172.21.15.38:0/175752509 <== osd.3 v2:172.21.15.38:6808/1286810294 17 ==== osd_op_reply(102 mds0_sessionmap [delete] v117'92 uv92 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fc7fc00b040 con 0x55a528ae48e0 2024-09-21T03:42:30.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.474+0000 7fc809536f40 1 -- 172.21.15.38:0/175752509 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:103 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x55a528aed000 con 0x55a528ae48e0 2024-09-21T03:42:30.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.475+0000 7fc8037f1640 1 -- 172.21.15.38:0/175752509 <== osd.3 v2:172.21.15.38:6808/1286810294 18 ==== osd_op_reply(103 mds0_sessionmap [create] v117'93 uv93 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fc7fc00b040 con 0x55a528ae48e0 2024-09-21T03:42:30.477 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-09-21T03:42:30.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.475+0000 7fc809536f40 1 -- 172.21.15.38:0/175752509 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:104 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x55a528aed000 con 0x55a528ae48e0 2024-09-21T03:42:30.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.476+0000 7fc8037f1640 1 -- 172.21.15.38:0/175752509 <== osd.3 v2:172.21.15.38:6808/1286810294 19 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v117'94 uv94 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fc7fc00b040 con 0x55a528ae48e0 2024-09-21T03:42:30.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.476+0000 7fc809536f40 1 -- 172.21.15.38:0/175752509 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x55a528aed000 con 0x55a528ae48e0 2024-09-21T03:42:30.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.477+0000 7fc8037f1640 1 -- 172.21.15.38:0/175752509 <== osd.3 v2:172.21.15.38:6808/1286810294 20 ==== osd_op_reply(105 609.00000000 [create] v117'123 uv122 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fc7fc00b040 con 0x55a528ae48e0 2024-09-21T03:42:30.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.477+0000 7fc809536f40 1 -- 172.21.15.38:0/175752509 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e117) -- 0x55a528aed000 con 0x55a528ae48e0 2024-09-21T03:42:30.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.478+0000 7fc8037f1640 1 -- 172.21.15.38:0/175752509 <== osd.3 v2:172.21.15.38:6808/1286810294 21 ==== osd_op_reply(106 609.00000000 [delete] v117'124 uv124 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc7fc00b040 con 0x55a528ae48e0 2024-09-21T03:42:30.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.478+0000 7fc809536f40 1 -- 172.21.15.38:0/175752509 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x55a528aed000 con 0x55a528ae48e0 2024-09-21T03:42:30.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.479+0000 7fc8037f1640 1 -- 172.21.15.38:0/175752509 <== osd.3 v2:172.21.15.38:6808/1286810294 22 ==== osd_op_reply(107 609.00000000 [create] v117'125 uv125 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc7fc00b040 con 0x55a528ae48e0 2024-09-21T03:42:30.481 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-09-21T03:42:30.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.479+0000 7fc809536f40 1 -- 172.21.15.38:0/175752509 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x55a528aee010 con 0x55a528ae48e0 2024-09-21T03:42:30.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.480+0000 7fc8037f1640 1 -- 172.21.15.38:0/175752509 <== osd.3 v2:172.21.15.38:6808/1286810294 23 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v117'126 uv126 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc7fc00b040 con 0x55a528ae48e0 2024-09-21T03:42:30.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.480+0000 7fc809536f40 1 -- 172.21.15.38:0/175752509 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:109 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x55a528aee010 con 0x55a528ae48e0 2024-09-21T03:42:30.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.481+0000 7fc8037f1640 1 -- 172.21.15.38:0/175752509 <== osd.3 v2:172.21.15.38:6808/1286810294 24 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v117'127 uv127 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc7fc00b040 con 0x55a528ae48e0 2024-09-21T03:42:30.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.481+0000 7fc809536f40 1 -- 172.21.15.38:0/175752509 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:110 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x55a528aed000 con 0x55a528ae48e0 2024-09-21T03:42:30.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.482+0000 7fc8037f1640 1 -- 172.21.15.38:0/175752509 <== osd.3 v2:172.21.15.38:6808/1286810294 25 ==== osd_op_reply(110 609.00000000 [omap-set-header] v117'128 uv128 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc7fc00b040 con 0x55a528ae48e0 2024-09-21T03:42:30.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.482+0000 7fc809536f40 1 --2- 172.21.15.38:0/175752509 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x55a528aed510 0x55a528aed920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:30.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.482+0000 7fc809536f40 1 -- 172.21.15.38:0/175752509 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x55a528aede60 con 0x55a528aed510 2024-09-21T03:42:30.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.482+0000 7fc8027ef640 1 --2- 172.21.15.38:0/175752509 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x55a528aed510 0x55a528aed920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:30.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.483+0000 7fc8027ef640 1 --2- 172.21.15.38:0/175752509 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x55a528aed510 0x55a528aed920 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:30.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.484+0000 7fc8027ef640 1 -- 172.21.15.38:0/175752509 <== osd.2 v2:172.21.15.38:6824/2577858902 1 ==== osd_op_reply(111 400.00000000 [create] v117'104 uv103 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fc7f805f030 con 0x55a528aed510 2024-09-21T03:42:30.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.484+0000 7fc809536f40 1 -- 172.21.15.38:0/175752509 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e117) -- 0x55a528aede60 con 0x55a528aed510 2024-09-21T03:42:30.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.485+0000 7fc8027ef640 1 -- 172.21.15.38:0/175752509 <== osd.2 v2:172.21.15.38:6824/2577858902 2 ==== osd_op_reply(112 400.00000000 [delete] v117'105 uv105 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc7f805f030 con 0x55a528aed510 2024-09-21T03:42:30.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.485+0000 7fc809536f40 1 -- 172.21.15.38:0/175752509 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x55a528aede60 con 0x55a528aed510 2024-09-21T03:42:30.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.486+0000 7fc8027ef640 1 -- 172.21.15.38:0/175752509 <== osd.2 v2:172.21.15.38:6824/2577858902 3 ==== osd_op_reply(113 400.00000000 [create] v117'106 uv106 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc7f805f030 con 0x55a528aed510 2024-09-21T03:42:30.488 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-09-21T03:42:30.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.486+0000 7fc809536f40 1 -- 172.21.15.38:0/175752509 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:114 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x55a528aede60 con 0x55a528aed510 2024-09-21T03:42:30.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.487+0000 7fc8027ef640 1 -- 172.21.15.38:0/175752509 <== osd.2 v2:172.21.15.38:6824/2577858902 4 ==== osd_op_reply(114 400.00000000 [write 0~22] v117'107 uv107 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc7f805f030 con 0x55a528aed510 2024-09-21T03:42:30.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.487+0000 7fc809536f40 1 -- 172.21.15.38:0/175752509 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:115 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x55a528aede60 con 0x55a528aed510 2024-09-21T03:42:30.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.488+0000 7fc8027ef640 1 -- 172.21.15.38:0/175752509 <== osd.2 v2:172.21.15.38:6824/2577858902 5 ==== osd_op_reply(115 400.00000000 [omap-set-header] v117'108 uv108 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc7f805f030 con 0x55a528aed510 2024-09-21T03:42:30.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.488+0000 7fc809536f40 1 -- 172.21.15.38:0/175752509 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:116 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x55a528aede60 con 0x55a528ae9c80 2024-09-21T03:42:30.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.489+0000 7fc802ff0640 1 -- 172.21.15.38:0/175752509 <== osd.1 v2:172.21.15.38:6816/1491818732 17 ==== osd_op_reply(116 100.00000000.inode [create] v117'253 uv245 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7fc7f401b040 con 0x55a528ae9c80 2024-09-21T03:42:30.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.489+0000 7fc809536f40 1 -- 172.21.15.38:0/175752509 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:117 13.1f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e117) -- 0x55a528aede60 con 0x55a528ae9c80 2024-09-21T03:42:30.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.491+0000 7fc802ff0640 1 -- 172.21.15.38:0/175752509 <== osd.1 v2:172.21.15.38:6816/1491818732 18 ==== osd_op_reply(117 100.00000000.inode [delete] v117'254 uv254 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fc7f401b040 con 0x55a528ae9c80 2024-09-21T03:42:30.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.491+0000 7fc809536f40 1 -- 172.21.15.38:0/175752509 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:118 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x55a528aede60 con 0x55a528ae9c80 2024-09-21T03:42:30.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.492+0000 7fc802ff0640 1 -- 172.21.15.38:0/175752509 <== osd.1 v2:172.21.15.38:6816/1491818732 19 ==== osd_op_reply(118 100.00000000.inode [create] v117'255 uv255 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fc7f401b040 con 0x55a528ae9c80 2024-09-21T03:42:30.494 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-09-21T03:42:30.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.492+0000 7fc809536f40 1 -- 172.21.15.38:0/175752509 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:119 13.1f 13:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x55a528aede60 con 0x55a528ae9c80 2024-09-21T03:42:30.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.493+0000 7fc802ff0640 1 -- 172.21.15.38:0/175752509 <== osd.1 v2:172.21.15.38:6816/1491818732 20 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v117'256 uv256 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fc7f401b040 con 0x55a528ae9c80 2024-09-21T03:42:30.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.493+0000 7fc809536f40 1 -- 172.21.15.38:0/175752509 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:120 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x55a528aede60 con 0x55a528ae9c80 2024-09-21T03:42:30.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.494+0000 7fc802ff0640 1 -- 172.21.15.38:0/175752509 <== osd.1 v2:172.21.15.38:6816/1491818732 21 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v117'257 uv257 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fc7f401b040 con 0x55a528ae9c80 2024-09-21T03:42:30.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.494+0000 7fc809536f40 1 -- 172.21.15.38:0/175752509 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:121 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x55a528aede60 con 0x55a528ae9c80 2024-09-21T03:42:30.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.495+0000 7fc802ff0640 1 -- 172.21.15.38:0/175752509 <== osd.1 v2:172.21.15.38:6816/1491818732 22 ==== osd_op_reply(121 1.00000000 [create] v117'258 uv252 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7fc7f401b040 con 0x55a528ae9c80 2024-09-21T03:42:30.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.495+0000 7fc809536f40 1 -- 172.21.15.38:0/175752509 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:122 13.1f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e117) -- 0x55a528aede60 con 0x55a528ae9c80 2024-09-21T03:42:30.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.496+0000 7fc802ff0640 1 -- 172.21.15.38:0/175752509 <== osd.1 v2:172.21.15.38:6816/1491818732 23 ==== osd_op_reply(122 1.00000000 [delete] v117'259 uv259 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fc7f401b040 con 0x55a528ae9c80 2024-09-21T03:42:30.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.496+0000 7fc809536f40 1 -- 172.21.15.38:0/175752509 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:123 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x55a528aede60 con 0x55a528ae9c80 2024-09-21T03:42:30.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.497+0000 7fc802ff0640 1 -- 172.21.15.38:0/175752509 <== osd.1 v2:172.21.15.38:6816/1491818732 24 ==== osd_op_reply(123 1.00000000 [create] v117'260 uv260 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fc7f401b040 con 0x55a528ae9c80 2024-09-21T03:42:30.499 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-09-21T03:42:30.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.498+0000 7fc809536f40 1 -- 172.21.15.38:0/175752509 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:124 13.1f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x55a528aede60 con 0x55a528ae9c80 2024-09-21T03:42:30.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.499+0000 7fc802ff0640 1 -- 172.21.15.38:0/175752509 <== osd.1 v2:172.21.15.38:6816/1491818732 25 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v117'261 uv261 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fc7f401b040 con 0x55a528ae9c80 2024-09-21T03:42:30.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.499+0000 7fc809536f40 1 -- 172.21.15.38:0/175752509 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:125 13.1f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x55a528aede60 con 0x55a528ae9c80 2024-09-21T03:42:30.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.500+0000 7fc802ff0640 1 -- 172.21.15.38:0/175752509 <== osd.1 v2:172.21.15.38:6816/1491818732 26 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v117'262 uv262 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fc7f401b040 con 0x55a528ae9c80 2024-09-21T03:42:30.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.500+0000 7fc809536f40 1 -- 172.21.15.38:0/175752509 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:126 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x55a528aede60 con 0x55a528ae9c80 2024-09-21T03:42:30.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.501+0000 7fc802ff0640 1 -- 172.21.15.38:0/175752509 <== osd.1 v2:172.21.15.38:6816/1491818732 27 ==== osd_op_reply(126 1.00000000 [omap-set-header] v117'263 uv263 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fc7f401b040 con 0x55a528ae9c80 2024-09-21T03:42:30.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.501+0000 7fc809536f40 1 -- 172.21.15.38:0/175752509 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:127 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=584b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x55a528aef4b0 con 0x55a528ae9c80 2024-09-21T03:42:30.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.502+0000 7fc802ff0640 1 -- 172.21.15.38:0/175752509 <== osd.1 v2:172.21.15.38:6816/1491818732 28 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v117'264 uv264 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fc7f401b040 con 0x55a528ae9c80 2024-09-21T03:42:30.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.502+0000 7fc809536f40 1 -- 172.21.15.38:0/175752509 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x55a528ae14c0 msgr2=0x55a528ae38e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:30.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.502+0000 7fc809536f40 1 --2- 172.21.15.38:0/175752509 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x55a528ae14c0 0x55a528ae38e0 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:30.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.502+0000 7fc809536f40 1 -- 172.21.15.38:0/175752509 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x55a528ae9c80 msgr2=0x55a528aea090 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:30.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.502+0000 7fc809536f40 1 --2- 172.21.15.38:0/175752509 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x55a528ae9c80 0x55a528aea090 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:30.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.502+0000 7fc809536f40 1 -- 172.21.15.38:0/175752509 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x55a528aed510 msgr2=0x55a528aed920 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:30.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.502+0000 7fc809536f40 1 --2- 172.21.15.38:0/175752509 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x55a528aed510 0x55a528aed920 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:30.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.502+0000 7fc809536f40 1 -- 172.21.15.38:0/175752509 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x55a528ae48e0 msgr2=0x55a528ae6d80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:30.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.502+0000 7fc809536f40 1 --2- 172.21.15.38:0/175752509 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x55a528ae48e0 0x55a528ae6d80 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:30.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.502+0000 7fc8037f1640 1 -- 172.21.15.38:0/175752509 reap_dead start 2024-09-21T03:42:30.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.502+0000 7fc809536f40 1 -- 172.21.15.38:0/175752509 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x55a528add2c0 msgr2=0x55a528adf6e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:30.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.502+0000 7fc809536f40 1 --2- 172.21.15.38:0/175752509 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x55a528add2c0 0x55a528adf6e0 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:30.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.502+0000 7fc8037f1640 1 -- 172.21.15.38:0/175752509 reap_dead start 2024-09-21T03:42:30.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.502+0000 7fc809536f40 1 -- 172.21.15.38:0/175752509 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x55a528ad8e70 msgr2=0x55a528adb290 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:30.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.502+0000 7fc809536f40 1 --2- 172.21.15.38:0/175752509 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x55a528ad8e70 0x55a528adb290 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:30.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.503+0000 7fc809536f40 1 -- 172.21.15.38:0/175752509 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc7f0077570 msgr2=0x7fc7f0079a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:30.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.503+0000 7fc809536f40 1 --2- 172.21.15.38:0/175752509 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc7f0077570 0x7fc7f0079a80 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fc7f4000c00 tx=0x7fc7f4001690 comp rx=0 tx=0).stop 2024-09-21T03:42:30.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.503+0000 7fc809536f40 1 -- 172.21.15.38:0/175752509 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a5289a8e60 msgr2=0x55a528a2c0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:30.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.503+0000 7fc809536f40 1 --2- 172.21.15.38:0/175752509 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a5289a8e60 0x55a528a2c0d0 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7fc7fc00d600 tx=0x7fc7fc00dac0 comp rx=0 tx=0).stop 2024-09-21T03:42:30.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.503+0000 7fc809536f40 1 -- 172.21.15.38:0/175752509 shutdown_connections 2024-09-21T03:42:30.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.503+0000 7fc809536f40 1 --2- 172.21.15.38:0/175752509 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x55a528ad8e70 0x55a528adb290 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:30.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.503+0000 7fc809536f40 1 --2- 172.21.15.38:0/175752509 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a5289a8e60 0x55a528a2c0d0 unknown :-1 s=CLOSED pgs=1754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:30.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.503+0000 7fc809536f40 1 --2- 172.21.15.38:0/175752509 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc7f0077570 0x7fc7f0079a80 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:30.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.503+0000 7fc809536f40 1 -- 172.21.15.38:0/175752509 >> 172.21.15.38:0/175752509 conn(0x55a528880ac0 msgr2=0x55a528960b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:30.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.503+0000 7fc809536f40 1 -- 172.21.15.38:0/175752509 shutdown_connections 2024-09-21T03:42:30.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.503+0000 7fc809536f40 1 -- 172.21.15.38:0/175752509 wait complete. 2024-09-21T03:42:30.508 DEBUG:teuthology.orchestra.run.smithi038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata put 400.00000000 - 2024-09-21T03:42:30.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.562+0000 7fb039945f40 1 Processor -- start 2024-09-21T03:42:30.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.562+0000 7fb039945f40 1 -- start start 2024-09-21T03:42:30.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.563+0000 7fb039945f40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e5d57b7d00 0x55e5d57fc250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:30.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.563+0000 7fb039945f40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e5d57e3e30 0x55e5d5870460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:30.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.563+0000 7fb039945f40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e5d5872950 0x55e5d57b3af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:30.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.563+0000 7fb039945f40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55e5d5764430 con 0x55e5d57b7d00 2024-09-21T03:42:30.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.563+0000 7fb039945f40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55e5d57711f0 con 0x55e5d5872950 2024-09-21T03:42:30.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.563+0000 7fb039945f40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55e5d5705d60 con 0x55e5d57e3e30 2024-09-21T03:42:30.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.563+0000 7fb0333ff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e5d57b7d00 0x55e5d57fc250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:30.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.563+0000 7fb0333ff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e5d57b7d00 0x55e5d57fc250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:59996/0 (socket says 172.21.15.38:59996) 2024-09-21T03:42:30.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.563+0000 7fb0333ff640 1 -- 172.21.15.38:0/2020845888 learned_addr learned my addr 172.21.15.38:0/2020845888 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:42:30.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.563+0000 7fb033c00640 1 --2- 172.21.15.38:0/2020845888 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e5d5872950 0x55e5d57b3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:30.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.563+0000 7fb02bfff640 1 --2- 172.21.15.38:0/2020845888 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e5d57e3e30 0x55e5d5870460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:30.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.563+0000 7fb0333ff640 1 -- 172.21.15.38:0/2020845888 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e5d57e3e30 msgr2=0x55e5d5870460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:30.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.563+0000 7fb0333ff640 1 --2- 172.21.15.38:0/2020845888 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e5d57e3e30 0x55e5d5870460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:30.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.563+0000 7fb0333ff640 1 -- 172.21.15.38:0/2020845888 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e5d5872950 msgr2=0x55e5d57b3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:30.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.563+0000 7fb0333ff640 1 --2- 172.21.15.38:0/2020845888 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e5d5872950 0x55e5d57b3af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:30.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.564+0000 7fb0333ff640 1 -- 172.21.15.38:0/2020845888 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5d57e4720 con 0x55e5d57b7d00 2024-09-21T03:42:30.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.564+0000 7fb0333ff640 1 --2- 172.21.15.38:0/2020845888 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e5d57b7d00 0x55e5d57fc250 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7fb024004e90 tx=0x7fb024002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43bfc733385ff9d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:30.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.564+0000 7fb0323fd640 1 -- 172.21.15.38:0/2020845888 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb024066070 con 0x55e5d57b7d00 2024-09-21T03:42:30.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.564+0000 7fb0323fd640 1 -- 172.21.15.38:0/2020845888 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb024056760 con 0x55e5d57b7d00 2024-09-21T03:42:30.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.564+0000 7fb0323fd640 1 -- 172.21.15.38:0/2020845888 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb024056b50 con 0x55e5d57b7d00 2024-09-21T03:42:30.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.565+0000 7fb039945f40 1 -- 172.21.15.38:0/2020845888 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e5d57b7d00 msgr2=0x55e5d57fc250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:30.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.565+0000 7fb039945f40 1 --2- 172.21.15.38:0/2020845888 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e5d57b7d00 0x55e5d57fc250 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7fb024004e90 tx=0x7fb024002fb0 comp rx=0 tx=0).stop 2024-09-21T03:42:30.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.565+0000 7fb039945f40 1 -- 172.21.15.38:0/2020845888 shutdown_connections 2024-09-21T03:42:30.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.565+0000 7fb039945f40 1 --2- 172.21.15.38:0/2020845888 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e5d5872950 0x55e5d57b3af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:30.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.565+0000 7fb039945f40 1 --2- 172.21.15.38:0/2020845888 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e5d57e3e30 0x55e5d5870460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:30.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.565+0000 7fb039945f40 1 --2- 172.21.15.38:0/2020845888 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e5d57b7d00 0x55e5d57fc250 unknown :-1 s=CLOSED pgs=1757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:30.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.565+0000 7fb039945f40 1 -- 172.21.15.38:0/2020845888 >> 172.21.15.38:0/2020845888 conn(0x55e5d56d6ac0 msgr2=0x55e5d586ee90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:30.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.565+0000 7fb039945f40 1 -- 172.21.15.38:0/2020845888 shutdown_connections 2024-09-21T03:42:30.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.565+0000 7fb039945f40 1 -- 172.21.15.38:0/2020845888 wait complete. 2024-09-21T03:42:30.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.566+0000 7fb039945f40 1 Processor -- start 2024-09-21T03:42:30.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.566+0000 7fb039945f40 1 -- start start 2024-09-21T03:42:30.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.567+0000 7fb039945f40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e5d57b7d00 0x55e5d586cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:30.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.567+0000 7fb039945f40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e5d57e3e30 0x55e5d5871b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:30.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.567+0000 7fb039945f40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e5d5872950 0x55e5d58720a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:30.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.567+0000 7fb039945f40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55e5d586f640 con 0x55e5d57b7d00 2024-09-21T03:42:30.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.567+0000 7fb039945f40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55e5d57fc7e0 con 0x55e5d5872950 2024-09-21T03:42:30.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.567+0000 7fb039945f40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55e5d59277d0 con 0x55e5d57e3e30 2024-09-21T03:42:30.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.567+0000 7fb0333ff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e5d57b7d00 0x55e5d586cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:30.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.567+0000 7fb0333ff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e5d57b7d00 0x55e5d586cbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60008/0 (socket says 172.21.15.38:60008) 2024-09-21T03:42:30.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.567+0000 7fb0333ff640 1 -- 172.21.15.38:0/127835073 learned_addr learned my addr 172.21.15.38:0/127835073 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:42:30.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.567+0000 7fb033c00640 1 --2- 172.21.15.38:0/127835073 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e5d5872950 0x55e5d58720a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:30.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.567+0000 7fb02bfff640 1 --2- 172.21.15.38:0/127835073 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e5d57e3e30 0x55e5d5871b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:30.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.567+0000 7fb0333ff640 1 -- 172.21.15.38:0/127835073 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e5d57e3e30 msgr2=0x55e5d5871b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:30.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.567+0000 7fb0333ff640 1 --2- 172.21.15.38:0/127835073 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e5d57e3e30 0x55e5d5871b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:30.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.567+0000 7fb0333ff640 1 -- 172.21.15.38:0/127835073 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e5d5872950 msgr2=0x55e5d58720a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:30.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.567+0000 7fb0333ff640 1 --2- 172.21.15.38:0/127835073 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e5d5872950 0x55e5d58720a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:30.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.567+0000 7fb0333ff640 1 -- 172.21.15.38:0/127835073 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5d588a640 con 0x55e5d57b7d00 2024-09-21T03:42:30.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.567+0000 7fb0333ff640 1 --2- 172.21.15.38:0/127835073 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e5d57b7d00 0x55e5d586cbe0 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7fb024004e90 tx=0x7fb024058780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:30.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.568+0000 7fb0313fb640 1 -- 172.21.15.38:0/127835073 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb024066040 con 0x55e5d57b7d00 2024-09-21T03:42:30.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.568+0000 7fb0313fb640 1 -- 172.21.15.38:0/127835073 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb024058480 con 0x55e5d57b7d00 2024-09-21T03:42:30.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.568+0000 7fb0313fb640 1 -- 172.21.15.38:0/127835073 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb024058bd0 con 0x55e5d57b7d00 2024-09-21T03:42:30.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.568+0000 7fb039945f40 1 -- 172.21.15.38:0/127835073 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55e5d586e3f0 con 0x55e5d57b7d00 2024-09-21T03:42:30.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.568+0000 7fb039945f40 1 -- 172.21.15.38:0/127835073 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5d588ab90 con 0x55e5d57b7d00 2024-09-21T03:42:30.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.569+0000 7fb0313fb640 1 -- 172.21.15.38:0/127835073 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb024058d70 con 0x55e5d57b7d00 2024-09-21T03:42:30.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.570+0000 7fb0313fb640 1 --2- 172.21.15.38:0/127835073 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb0140775a0 0x7fb014079ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:30.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.570+0000 7fb0313fb640 1 -- 172.21.15.38:0/127835073 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 8110+0+0 (secure 0 0 0) 0x7fb0240df9e0 con 0x55e5d57b7d00 2024-09-21T03:42:30.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.570+0000 7fb02bfff640 1 --2- 172.21.15.38:0/127835073 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb0140775a0 0x7fb014079ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:30.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.570+0000 7fb039945f40 1 --2- 172.21.15.38:0/127835073 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x55e5d5927950 0x55e5d57e1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:30.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.570+0000 7fb039945f40 1 -- 172.21.15.38:0/127835073 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:1 13.f 13:f1f69726:::400.00000000:head [writefull 0~80 in=80b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x55e5d57e21e0 con 0x55e5d5927950 2024-09-21T03:42:30.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.570+0000 7fb033c00640 1 --2- 172.21.15.38:0/127835073 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x55e5d5927950 0x55e5d57e1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:30.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.571+0000 7fb02bfff640 1 --2- 172.21.15.38:0/127835073 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb0140775a0 0x7fb014079ab0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x55e5d57b4400 tx=0x7fb01c003750 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:30.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.571+0000 7fb033c00640 1 --2- 172.21.15.38:0/127835073 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x55e5d5927950 0x55e5d57e1ca0 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:30.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.572+0000 7fb033c00640 1 -- 172.21.15.38:0/127835073 <== osd.2 v2:172.21.15.38:6824/2577858902 1 ==== osd_op_reply(1 400.00000000 [writefull 0~80] v117'109 uv109 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb02c00d730 con 0x55e5d5927950 2024-09-21T03:42:30.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.572+0000 7fb039945f40 1 -- 172.21.15.38:0/127835073 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x55e5d5927950 msgr2=0x55e5d57e1ca0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:30.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.572+0000 7fb039945f40 1 --2- 172.21.15.38:0/127835073 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x55e5d5927950 0x55e5d57e1ca0 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:30.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.572+0000 7fb039945f40 1 -- 172.21.15.38:0/127835073 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb0140775a0 msgr2=0x7fb014079ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:30.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.572+0000 7fb039945f40 1 --2- 172.21.15.38:0/127835073 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb0140775a0 0x7fb014079ab0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x55e5d57b4400 tx=0x7fb01c003750 comp rx=0 tx=0).stop 2024-09-21T03:42:30.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.572+0000 7fb039945f40 1 -- 172.21.15.38:0/127835073 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e5d57b7d00 msgr2=0x55e5d586cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:30.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.572+0000 7fb039945f40 1 --2- 172.21.15.38:0/127835073 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e5d57b7d00 0x55e5d586cbe0 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7fb024004e90 tx=0x7fb024058780 comp rx=0 tx=0).stop 2024-09-21T03:42:30.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.572+0000 7fb033c00640 1 -- 172.21.15.38:0/127835073 reap_dead start 2024-09-21T03:42:30.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.572+0000 7fb039945f40 1 -- 172.21.15.38:0/127835073 shutdown_connections 2024-09-21T03:42:30.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.572+0000 7fb039945f40 1 -- 172.21.15.38:0/127835073 >> 172.21.15.38:0/127835073 conn(0x55e5d56d6ac0 msgr2=0x55e5d586ee90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:30.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.572+0000 7fb039945f40 1 -- 172.21.15.38:0/127835073 shutdown_connections 2024-09-21T03:42:30.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.572+0000 7fb039945f40 1 -- 172.21.15.38:0/127835073 wait complete. 2024-09-21T03:42:30.577 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-21T03:42:30.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.736+0000 7f524da0e640 1 -- 172.21.15.38:0/3041732031 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f52480ceb90 msgr2=0x7f52480cefa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:30.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.736+0000 7f524da0e640 1 --2- 172.21.15.38:0/3041732031 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f52480ceb90 0x7f52480cefa0 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7f5240005fd0 tx=0x7f524002a820 comp rx=0 tx=0).stop 2024-09-21T03:42:30.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.736+0000 7f524da0e640 1 -- 172.21.15.38:0/3041732031 shutdown_connections 2024-09-21T03:42:30.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.736+0000 7f524da0e640 1 --2- 172.21.15.38:0/3041732031 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f52480ceb90 0x7f52480cefa0 unknown :-1 s=CLOSED pgs=1760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:30.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.736+0000 7f524da0e640 1 -- 172.21.15.38:0/3041732031 >> 172.21.15.38:0/3041732031 conn(0x7f524809d830 msgr2=0x7f52480b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:30.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.736+0000 7f524da0e640 1 -- 172.21.15.38:0/3041732031 shutdown_connections 2024-09-21T03:42:30.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.736+0000 7f524da0e640 1 -- 172.21.15.38:0/3041732031 wait complete. 2024-09-21T03:42:30.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.738+0000 7f524da0e640 1 Processor -- start 2024-09-21T03:42:30.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.738+0000 7f524da0e640 1 -- start start 2024-09-21T03:42:30.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.739+0000 7f524da0e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f52480ceb90 0x7f524814a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:30.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.739+0000 7f524da0e640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f52480d0170 0x7f52481567c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:30.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.739+0000 7f524da0e640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f524815ee80 0x7f52480b1190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:30.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.739+0000 7f524da0e640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f52480b6790 con 0x7f52480ceb90 2024-09-21T03:42:30.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.739+0000 7f524da0e640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f52480d14b0 con 0x7f524815ee80 2024-09-21T03:42:30.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.739+0000 7f524da0e640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f52480b6910 con 0x7f52480d0170 2024-09-21T03:42:30.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.739+0000 7f5246ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f52480ceb90 0x7f524814a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:30.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.739+0000 7f5246ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f52480ceb90 0x7f524814a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60030/0 (socket says 172.21.15.38:60030) 2024-09-21T03:42:30.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.739+0000 7f5246ffd640 1 -- 172.21.15.38:0/2202181228 learned_addr learned my addr 172.21.15.38:0/2202181228 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:42:30.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.739+0000 7f52477fe640 1 --2- 172.21.15.38:0/2202181228 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f524815ee80 0x7f52480b1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:30.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.739+0000 7f52467fc640 1 --2- 172.21.15.38:0/2202181228 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f52480d0170 0x7f52481567c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:30.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.740+0000 7f5246ffd640 1 -- 172.21.15.38:0/2202181228 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f52480d0170 msgr2=0x7f52481567c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:30.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.740+0000 7f5246ffd640 1 --2- 172.21.15.38:0/2202181228 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f52480d0170 0x7f52481567c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:30.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.740+0000 7f5246ffd640 1 -- 172.21.15.38:0/2202181228 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f524815ee80 msgr2=0x7f52480b1190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:30.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.740+0000 7f5246ffd640 1 --2- 172.21.15.38:0/2202181228 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f524815ee80 0x7f52480b1190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:30.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.740+0000 7f5246ffd640 1 -- 172.21.15.38:0/2202181228 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f524814af30 con 0x7f52480ceb90 2024-09-21T03:42:30.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.740+0000 7f52467fc640 1 --2- 172.21.15.38:0/2202181228 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f52480d0170 0x7f52481567c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:42:30.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.740+0000 7f5246ffd640 1 --2- 172.21.15.38:0/2202181228 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f52480ceb90 0x7f524814a9f0 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7f523000c760 tx=0x7f523000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:30.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.740+0000 7f523ffff640 1 -- 172.21.15.38:0/2202181228 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5230006a90 con 0x7f52480ceb90 2024-09-21T03:42:30.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.740+0000 7f523ffff640 1 -- 172.21.15.38:0/2202181228 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5230017070 con 0x7f52480ceb90 2024-09-21T03:42:30.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.740+0000 7f524da0e640 1 -- 172.21.15.38:0/2202181228 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52480cd690 con 0x7f52480ceb90 2024-09-21T03:42:30.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.740+0000 7f523ffff640 1 -- 172.21.15.38:0/2202181228 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f523000ce10 con 0x7f52480ceb90 2024-09-21T03:42:30.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.740+0000 7f524da0e640 1 -- 172.21.15.38:0/2202181228 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f524814b440 con 0x7f52480ceb90 2024-09-21T03:42:30.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.741+0000 7f524da0e640 1 -- 172.21.15.38:0/2202181228 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52480b6e50 con 0x7f52480ceb90 2024-09-21T03:42:30.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.742+0000 7f523ffff640 1 -- 172.21.15.38:0/2202181228 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f5230007050 con 0x7f52480ceb90 2024-09-21T03:42:30.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.742+0000 7f523ffff640 1 --2- 172.21.15.38:0/2202181228 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5228078bc0 0x7f522807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:30.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.743+0000 7f523ffff640 1 -- 172.21.15.38:0/2202181228 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 8110+0+0 (secure 0 0 0) 0x7f523009b340 con 0x7f52480ceb90 2024-09-21T03:42:30.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.743+0000 7f52467fc640 1 --2- 172.21.15.38:0/2202181228 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5228078bc0 0x7f522807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:30.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.743+0000 7f52467fc640 1 --2- 172.21.15.38:0/2202181228 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5228078bc0 0x7f522807b0d0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f524806fde0 tx=0x7f5238009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:30.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.747+0000 7f523ffff640 1 -- 172.21.15.38:0/2202181228 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f52481592d0 con 0x7f52480ceb90 2024-09-21T03:42:30.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:30.975+0000 7f524da0e640 1 -- 172.21.15.38:0/2202181228 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f5248158080 con 0x7f52480ceb90 2024-09-21T03:42:31.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:31.263+0000 7f523ffff640 1 -- 172.21.15.38:0/2202181228 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v212) ==== 162+0+0 (secure 0 0 0) 0x7f5230062ec0 con 0x7f52480ceb90 2024-09-21T03:42:31.266 INFO:teuthology.orchestra.run.smithi038.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-21T03:42:31.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:31.266+0000 7f524da0e640 1 -- 172.21.15.38:0/2202181228 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5228078bc0 msgr2=0x7f522807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:31.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:31.266+0000 7f524da0e640 1 --2- 172.21.15.38:0/2202181228 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5228078bc0 0x7f522807b0d0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f524806fde0 tx=0x7f5238009040 comp rx=0 tx=0).stop 2024-09-21T03:42:31.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:31.266+0000 7f524da0e640 1 -- 172.21.15.38:0/2202181228 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f52480ceb90 msgr2=0x7f524814a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:31.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:31.266+0000 7f524da0e640 1 --2- 172.21.15.38:0/2202181228 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f52480ceb90 0x7f524814a9f0 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7f523000c760 tx=0x7f523000cc20 comp rx=0 tx=0).stop 2024-09-21T03:42:31.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:31.266+0000 7f524da0e640 1 -- 172.21.15.38:0/2202181228 shutdown_connections 2024-09-21T03:42:31.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:31.266+0000 7f524da0e640 1 --2- 172.21.15.38:0/2202181228 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f524815ee80 0x7f52480b1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:31.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:31.266+0000 7f524da0e640 1 --2- 172.21.15.38:0/2202181228 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5228078bc0 0x7f522807b0d0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:31.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:31.266+0000 7f524da0e640 1 --2- 172.21.15.38:0/2202181228 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f52480d0170 0x7f52481567c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:31.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:31.266+0000 7f524da0e640 1 --2- 172.21.15.38:0/2202181228 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f52480ceb90 0x7f524814a9f0 unknown :-1 s=CLOSED pgs=1761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:31.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:31.266+0000 7f524da0e640 1 -- 172.21.15.38:0/2202181228 >> 172.21.15.38:0/2202181228 conn(0x7f524809d830 msgr2=0x7f5248157340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:31.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:31.266+0000 7f524da0e640 1 -- 172.21.15.38:0/2202181228 shutdown_connections 2024-09-21T03:42:31.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:31.266+0000 7f524da0e640 1 -- 172.21.15.38:0/2202181228 wait complete. 2024-09-21T03:42:31.276 INFO:tasks.ceph.mds.a.smithi038.stderr:2024-09-21T03:42:31.274+0000 7ff816d2a640 -1 log_channel(cluster) log [ERR] : failed to read JournalPointer: -22 ((22) Invalid argument) 2024-09-21T03:42:31.277 INFO:tasks.ceph.mon.a.smithi038.stderr:2024-09-21T03:42:31.275+0000 7f3f45713640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T03:42:31.277 INFO:tasks.ceph.mon.a.smithi038.stderr:2024-09-21T03:42:31.275+0000 7f3f45713640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-09-21T03:42:31.282 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:42:31.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:31.398+0000 7f974b562640 1 -- 172.21.15.38:0/2488830247 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f97440cea90 msgr2=0x7f97440d00d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:31.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:31.398+0000 7f974b562640 1 --2- 172.21.15.38:0/2488830247 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f97440cea90 0x7f97440d00d0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f9740009850 tx=0x7f974002a4f0 comp rx=0 tx=0).stop 2024-09-21T03:42:31.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:31.398+0000 7f974b562640 1 -- 172.21.15.38:0/2488830247 shutdown_connections 2024-09-21T03:42:31.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:31.398+0000 7f974b562640 1 --2- 172.21.15.38:0/2488830247 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f97440cea90 0x7f97440d00d0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:31.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:31.398+0000 7f974b562640 1 -- 172.21.15.38:0/2488830247 >> 172.21.15.38:0/2488830247 conn(0x7f97440d4620 msgr2=0x7f97440d4a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:31.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:31.398+0000 7f974b562640 1 -- 172.21.15.38:0/2488830247 shutdown_connections 2024-09-21T03:42:31.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:31.398+0000 7f974b562640 1 -- 172.21.15.38:0/2488830247 wait complete. 2024-09-21T03:42:31.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:31.400+0000 7f974b562640 1 Processor -- start 2024-09-21T03:42:31.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:31.400+0000 7f974b562640 1 -- start start 2024-09-21T03:42:31.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:31.401+0000 7f974b562640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f97440cea90 0x7f97441551e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:31.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:31.401+0000 7f974b562640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f97440d3360 0x7f974415a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:31.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:31.401+0000 7f974b562640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9744162790 0x7f9744156230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:31.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:31.401+0000 7f974b562640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f97440b43c0 con 0x7f9744162790 2024-09-21T03:42:31.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:31.401+0000 7f974b562640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f97440b4240 con 0x7f97440cea90 2024-09-21T03:42:31.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:31.401+0000 7f974b562640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f97440b4540 con 0x7f97440d3360 2024-09-21T03:42:31.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:31.401+0000 7f9749aff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9744162790 0x7f9744156230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:31.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:31.401+0000 7f9749aff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9744162790 0x7f9744156230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60046/0 (socket says 172.21.15.38:60046) 2024-09-21T03:42:31.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:31.401+0000 7f9749aff640 1 -- 172.21.15.38:0/3448676781 learned_addr learned my addr 172.21.15.38:0/3448676781 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:42:31.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:31.401+0000 7f97492fe640 1 --2- 172.21.15.38:0/3448676781 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f97440cea90 0x7f97441551e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:31.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:31.401+0000 7f9748afd640 1 --2- 172.21.15.38:0/3448676781 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f97440d3360 0x7f974415a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:31.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:31.402+0000 7f9749aff640 1 -- 172.21.15.38:0/3448676781 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f97440d3360 msgr2=0x7f974415a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:31.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:31.402+0000 7f9749aff640 1 --2- 172.21.15.38:0/3448676781 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f97440d3360 0x7f974415a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:31.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:31.402+0000 7f9749aff640 1 -- 172.21.15.38:0/3448676781 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f97440cea90 msgr2=0x7f97441551e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:31.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:31.402+0000 7f9749aff640 1 --2- 172.21.15.38:0/3448676781 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f97440cea90 0x7f97441551e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:31.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:31.402+0000 7f9749aff640 1 -- 172.21.15.38:0/3448676781 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9744159870 con 0x7f9744162790 2024-09-21T03:42:31.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:31.402+0000 7f9749aff640 1 --2- 172.21.15.38:0/3448676781 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9744162790 0x7f9744156230 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7f973c00cac0 tx=0x7f973c00cf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:31.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:31.402+0000 7f9739ffb640 1 -- 172.21.15.38:0/3448676781 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f973c016020 con 0x7f9744162790 2024-09-21T03:42:31.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:31.402+0000 7f9739ffb640 1 -- 172.21.15.38:0/3448676781 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f973c00ad40 con 0x7f9744162790 2024-09-21T03:42:31.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:31.402+0000 7f9739ffb640 1 -- 172.21.15.38:0/3448676781 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f973c012680 con 0x7f9744162790 2024-09-21T03:42:31.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:31.402+0000 7f974b562640 1 -- 172.21.15.38:0/3448676781 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9744159b50 con 0x7f9744162790 2024-09-21T03:42:31.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:31.402+0000 7f974b562640 1 -- 172.21.15.38:0/3448676781 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f974415a110 con 0x7f9744162790 2024-09-21T03:42:31.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:31.403+0000 7f974b562640 1 -- 172.21.15.38:0/3448676781 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97440d00d0 con 0x7f9744162790 2024-09-21T03:42:31.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:31.404+0000 7f9739ffb640 1 -- 172.21.15.38:0/3448676781 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f973c019020 con 0x7f9744162790 2024-09-21T03:42:31.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:31.405+0000 7f9739ffb640 1 --2- 172.21.15.38:0/3448676781 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9728078b90 0x7f972807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:31.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:31.405+0000 7f9739ffb640 1 -- 172.21.15.38:0/3448676781 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 8196+0+0 (secure 0 0 0) 0x7f973c09bd50 con 0x7f9744162790 2024-09-21T03:42:31.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:31.405+0000 7f97492fe640 1 --2- 172.21.15.38:0/3448676781 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9728078b90 0x7f972807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:31.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:31.405+0000 7f97492fe640 1 --2- 172.21.15.38:0/3448676781 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9728078b90 0x7f972807b0a0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f9740034000 tx=0x7f9740004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:31.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:31.409+0000 7f9739ffb640 1 -- 172.21.15.38:0/3448676781 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f973c020020 con 0x7f9744162790 2024-09-21T03:42:31.519 DEBUG:teuthology.orchestra.run.smithi038:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T03:42:31.521 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T03:42:31.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:31.638+0000 7f974b562640 1 -- 172.21.15.38:0/3448676781 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f97441f65d0 con 0x7f9744162790 2024-09-21T03:42:31.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:31.639+0000 7f9739ffb640 1 -- 172.21.15.38:0/3448676781 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 214 v214) ==== 95+0+3566 (secure 0 0 0) 0x7f973c063050 con 0x7f9744162790 2024-09-21T03:42:31.641 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:42:31.641 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":214,"btime":"2024-09-21T03:42:31:276459+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6086,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/965972663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":965972663},{"type":"v1","addr":"172.21.15.195:6836","nonce":965972663}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6700,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/3176122334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":3176122334},{"type":"v1","addr":"172.21.15.195:6837","nonce":3176122334}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":214,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:40:02.319282+0000","modified":"2024-09-21T03:42:31.276380+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":118,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-21T03:42:31.641 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 214 2024-09-21T03:42:31.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:31.641+0000 7f974b562640 1 -- 172.21.15.38:0/3448676781 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9728078b90 msgr2=0x7f972807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:31.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:31.641+0000 7f974b562640 1 --2- 172.21.15.38:0/3448676781 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9728078b90 0x7f972807b0a0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f9740034000 tx=0x7f9740004730 comp rx=0 tx=0).stop 2024-09-21T03:42:31.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:31.641+0000 7f974b562640 1 -- 172.21.15.38:0/3448676781 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9744162790 msgr2=0x7f9744156230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:31.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:31.641+0000 7f974b562640 1 --2- 172.21.15.38:0/3448676781 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9744162790 0x7f9744156230 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7f973c00cac0 tx=0x7f973c00cf80 comp rx=0 tx=0).stop 2024-09-21T03:42:31.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:31.641+0000 7f974b562640 1 -- 172.21.15.38:0/3448676781 shutdown_connections 2024-09-21T03:42:31.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:31.641+0000 7f974b562640 1 --2- 172.21.15.38:0/3448676781 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9744162790 0x7f9744156230 unknown :-1 s=CLOSED pgs=1763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:31.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:31.641+0000 7f974b562640 1 --2- 172.21.15.38:0/3448676781 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9728078b90 0x7f972807b0a0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:31.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:31.641+0000 7f974b562640 1 --2- 172.21.15.38:0/3448676781 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f97440d3360 0x7f974415a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:31.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:31.641+0000 7f974b562640 1 --2- 172.21.15.38:0/3448676781 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f97440cea90 0x7f97441551e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:31.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:31.641+0000 7f974b562640 1 -- 172.21.15.38:0/3448676781 >> 172.21.15.38:0/3448676781 conn(0x7f97440d4620 msgr2=0x7f9744160ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:31.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:31.641+0000 7f974b562640 1 -- 172.21.15.38:0/3448676781 shutdown_connections 2024-09-21T03:42:31.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:31.641+0000 7f974b562640 1 -- 172.21.15.38:0/3448676781 wait complete. 2024-09-21T03:42:31.654 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-21T03:42:31.654 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:42:31.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:31.769+0000 7ffbe6c94640 1 -- 172.21.15.38:0/705212030 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffbe0162280 msgr2=0x7ffbe0162690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:31.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:31.769+0000 7ffbe6c94640 1 --2- 172.21.15.38:0/705212030 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffbe0162280 0x7ffbe0162690 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7ffbe00d5040 tx=0x7ffbd002aa30 comp rx=0 tx=0).stop 2024-09-21T03:42:31.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:31.769+0000 7ffbe6c94640 1 -- 172.21.15.38:0/705212030 shutdown_connections 2024-09-21T03:42:31.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:31.769+0000 7ffbe6c94640 1 --2- 172.21.15.38:0/705212030 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffbe00b2e40 0x7ffbe00b3250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:31.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:31.769+0000 7ffbe6c94640 1 --2- 172.21.15.38:0/705212030 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffbe0162280 0x7ffbe0162690 unknown :-1 s=CLOSED pgs=1764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:31.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:31.769+0000 7ffbe6c94640 1 --2- 172.21.15.38:0/705212030 >> v2:172.21.15.195:3300/0 conn(0x7ffbe0162f00 0x7ffbe00d24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:31.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:31.769+0000 7ffbe6c94640 1 -- 172.21.15.38:0/705212030 >> 172.21.15.38:0/705212030 conn(0x7ffbe00b6b20 msgr2=0x7ffbe00d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:31.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:31.769+0000 7ffbe6c94640 1 -- 172.21.15.38:0/705212030 shutdown_connections 2024-09-21T03:42:31.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:31.769+0000 7ffbe6c94640 1 -- 172.21.15.38:0/705212030 wait complete. 2024-09-21T03:42:31.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:31.771+0000 7ffbe6c94640 1 Processor -- start 2024-09-21T03:42:31.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:31.771+0000 7ffbe6c94640 1 -- start start 2024-09-21T03:42:31.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:31.772+0000 7ffbe6c94640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffbe00b2e40 0x7ffbe00cee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:31.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:31.772+0000 7ffbe6c94640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffbe0162280 0x7ffbe014f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:31.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:31.772+0000 7ffbe6c94640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffbe014ff00 0x7ffbe014e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:31.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:31.772+0000 7ffbe6c94640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ffbe00b3dd0 con 0x7ffbe014ff00 2024-09-21T03:42:31.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:31.772+0000 7ffbe6c94640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ffbe00b3a20 con 0x7ffbe0162280 2024-09-21T03:42:31.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:31.772+0000 7ffbe6c94640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7ffbe00b3f50 con 0x7ffbe00b2e40 2024-09-21T03:42:31.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:31.772+0000 7ffbe5231640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffbe014ff00 0x7ffbe014e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:31.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:31.772+0000 7ffbdffff640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffbe0162280 0x7ffbe014f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:31.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:31.772+0000 7ffbe4a30640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffbe00b2e40 0x7ffbe00cee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:31.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:31.772+0000 7ffbe5231640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffbe014ff00 0x7ffbe014e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60064/0 (socket says 172.21.15.38:60064) 2024-09-21T03:42:31.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:31.772+0000 7ffbe5231640 1 -- 172.21.15.38:0/3886695461 learned_addr learned my addr 172.21.15.38:0/3886695461 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:42:31.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:31.772+0000 7ffbdffff640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffbe0162280 0x7ffbe014f9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.38:58590/0 (socket says 172.21.15.38:58590) 2024-09-21T03:42:31.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:31.773+0000 7ffbe5231640 1 -- 172.21.15.38:0/3886695461 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffbe00b2e40 msgr2=0x7ffbe00cee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:31.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:31.773+0000 7ffbe5231640 1 --2- 172.21.15.38:0/3886695461 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffbe00b2e40 0x7ffbe00cee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:31.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:31.773+0000 7ffbe5231640 1 -- 172.21.15.38:0/3886695461 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffbe0162280 msgr2=0x7ffbe014f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:31.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:31.773+0000 7ffbe5231640 1 --2- 172.21.15.38:0/3886695461 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffbe0162280 0x7ffbe014f9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:31.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:31.773+0000 7ffbe5231640 1 -- 172.21.15.38:0/3886695461 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbe0150310 con 0x7ffbe014ff00 2024-09-21T03:42:31.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:31.773+0000 7ffbe5231640 1 --2- 172.21.15.38:0/3886695461 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffbe014ff00 0x7ffbe014e750 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7ffbc80058c0 tx=0x7ffbc800da70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:31.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:31.773+0000 7ffbdd7fa640 1 -- 172.21.15.38:0/3886695461 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbc8017070 con 0x7ffbe014ff00 2024-09-21T03:42:31.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:31.773+0000 7ffbdd7fa640 1 -- 172.21.15.38:0/3886695461 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffbc8005310 con 0x7ffbe014ff00 2024-09-21T03:42:31.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:31.773+0000 7ffbe6c94640 1 -- 172.21.15.38:0/3886695461 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbe014d9d0 con 0x7ffbe014ff00 2024-09-21T03:42:31.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:31.773+0000 7ffbdd7fa640 1 -- 172.21.15.38:0/3886695461 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbc80047a0 con 0x7ffbe014ff00 2024-09-21T03:42:31.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:31.773+0000 7ffbe6c94640 1 -- 172.21.15.38:0/3886695461 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbe014dc70 con 0x7ffbe014ff00 2024-09-21T03:42:31.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:31.774+0000 7ffbe6c94640 1 -- 172.21.15.38:0/3886695461 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffbe00b3ba0 con 0x7ffbe014ff00 2024-09-21T03:42:31.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:31.775+0000 7ffbdd7fa640 1 -- 172.21.15.38:0/3886695461 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ffbc8004940 con 0x7ffbe014ff00 2024-09-21T03:42:31.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:31.775+0000 7ffbdd7fa640 1 --2- 172.21.15.38:0/3886695461 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ffbc0078b90 0x7ffbc007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:31.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:31.775+0000 7ffbdd7fa640 1 -- 172.21.15.38:0/3886695461 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 8196+0+0 (secure 0 0 0) 0x7ffbc809b7a0 con 0x7ffbe014ff00 2024-09-21T03:42:31.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:31.776+0000 7ffbe4a30640 1 --2- 172.21.15.38:0/3886695461 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ffbc0078b90 0x7ffbc007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:31.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:31.776+0000 7ffbe4a30640 1 --2- 172.21.15.38:0/3886695461 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ffbc0078b90 0x7ffbc007b0a0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7ffbd800bb50 tx=0x7ffbd800b2e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:31.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:31.780+0000 7ffbdd7fa640 1 -- 172.21.15.38:0/3886695461 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7ffbe00b3ba0 con 0x7ffbe014ff00 2024-09-21T03:42:31.993 INFO:tasks.ceph.mds.c.smithi038.stderr: -52> 2024-09-21T03:42:27.239+0000 7f1eda49e640 -1 log_channel(cluster) log [ERR] : error decoding table object 'mds0_inotable': Decoder at 'virtual void InoTable::decode_state(ceph::buffer::v15_2_0::list::const_iterator&)' v=2 cannot decode v=100 minimal_decoder=101: Malformed input [buffer:3] 2024-09-21T03:42:32.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:32.010+0000 7ffbe6c94640 1 -- 172.21.15.38:0/3886695461 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffbe0162f00 con 0x7ffbe014ff00 2024-09-21T03:42:32.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:32.010+0000 7ffbdd7fa640 1 -- 172.21.15.38:0/3886695461 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 214 v214) ==== 95+0+3566 (secure 0 0 0) 0x7ffbe0150310 con 0x7ffbe014ff00 2024-09-21T03:42:32.013 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:42:32.013 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":214,"btime":"2024-09-21T03:42:31:276459+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6086,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/965972663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":965972663},{"type":"v1","addr":"172.21.15.195:6836","nonce":965972663}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6700,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/3176122334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":3176122334},{"type":"v1","addr":"172.21.15.195:6837","nonce":3176122334}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":214,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:40:02.319282+0000","modified":"2024-09-21T03:42:31.276380+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":118,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-21T03:42:32.013 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 214 2024-09-21T03:42:32.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:32.013+0000 7ffbe6c94640 1 -- 172.21.15.38:0/3886695461 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ffbc0078b90 msgr2=0x7ffbc007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:32.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:32.013+0000 7ffbe6c94640 1 --2- 172.21.15.38:0/3886695461 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ffbc0078b90 0x7ffbc007b0a0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7ffbd800bb50 tx=0x7ffbd800b2e0 comp rx=0 tx=0).stop 2024-09-21T03:42:32.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:32.013+0000 7ffbe6c94640 1 -- 172.21.15.38:0/3886695461 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffbe014ff00 msgr2=0x7ffbe014e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:32.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:32.013+0000 7ffbe6c94640 1 --2- 172.21.15.38:0/3886695461 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffbe014ff00 0x7ffbe014e750 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7ffbc80058c0 tx=0x7ffbc800da70 comp rx=0 tx=0).stop 2024-09-21T03:42:32.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:32.013+0000 7ffbe6c94640 1 -- 172.21.15.38:0/3886695461 shutdown_connections 2024-09-21T03:42:32.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:32.013+0000 7ffbe6c94640 1 --2- 172.21.15.38:0/3886695461 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffbe014ff00 0x7ffbe014e750 unknown :-1 s=CLOSED pgs=1765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:32.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:32.013+0000 7ffbe6c94640 1 --2- 172.21.15.38:0/3886695461 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffbe0162280 0x7ffbe014f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:32.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:32.013+0000 7ffbe6c94640 1 --2- 172.21.15.38:0/3886695461 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ffbc0078b90 0x7ffbc007b0a0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:32.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:32.013+0000 7ffbe6c94640 1 --2- 172.21.15.38:0/3886695461 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffbe00b2e40 0x7ffbe00cee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:32.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:32.013+0000 7ffbe6c94640 1 -- 172.21.15.38:0/3886695461 >> 172.21.15.38:0/3886695461 conn(0x7ffbe00b6b20 msgr2=0x7ffbe006d7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:32.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:32.013+0000 7ffbe6c94640 1 -- 172.21.15.38:0/3886695461 shutdown_connections 2024-09-21T03:42:32.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:32.013+0000 7ffbe6c94640 1 -- 172.21.15.38:0/3886695461 wait complete. 2024-09-21T03:42:32.027 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-21T03:42:32.027 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:42:32.047 INFO:tasks.ceph.mds.c.smithi038.stdout:starting mds.c at 2024-09-21T03:42:32.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:32.143+0000 7f20645ad640 1 -- 172.21.15.38:0/2109600153 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f205c15a6f0 msgr2=0x7f205c15e230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:32.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:32.143+0000 7f20645ad640 1 --2- 172.21.15.38:0/2109600153 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f205c15a6f0 0x7f205c15e230 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7f2050005fd0 tx=0x7f205002a5d0 comp rx=0 tx=0).stop 2024-09-21T03:42:32.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:32.143+0000 7f20645ad640 1 -- 172.21.15.38:0/2109600153 shutdown_connections 2024-09-21T03:42:32.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:32.143+0000 7f20645ad640 1 --2- 172.21.15.38:0/2109600153 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f205c15a6f0 0x7f205c15e230 unknown :-1 s=CLOSED pgs=1769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:32.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:32.143+0000 7f20645ad640 1 --2- 172.21.15.38:0/2109600153 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f205c0d01f0 0x7f205c15daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:32.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:32.143+0000 7f20645ad640 1 -- 172.21.15.38:0/2109600153 >> 172.21.15.38:0/2109600153 conn(0x7f205c072990 msgr2=0x7f205c072da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:32.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:32.143+0000 7f20645ad640 1 -- 172.21.15.38:0/2109600153 shutdown_connections 2024-09-21T03:42:32.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:32.143+0000 7f20645ad640 1 -- 172.21.15.38:0/2109600153 wait complete. 2024-09-21T03:42:32.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:32.145+0000 7f20645ad640 1 Processor -- start 2024-09-21T03:42:32.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:32.145+0000 7f20645ad640 1 -- start start 2024-09-21T03:42:32.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:32.146+0000 7f20645ad640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f205c0d01f0 0x7f205c15c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:32.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:32.146+0000 7f20645ad640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f205c0d0980 0x7f205c1456b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:32.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:32.146+0000 7f20645ad640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f205c15a6f0 0x7f205c145bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:32.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:32.146+0000 7f20645ad640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f205c0cd340 con 0x7f205c15a6f0 2024-09-21T03:42:32.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:32.146+0000 7f20645ad640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f205c0cd1c0 con 0x7f205c0d0980 2024-09-21T03:42:32.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:32.146+0000 7f20645ad640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f205c0cd4c0 con 0x7f205c0d01f0 2024-09-21T03:42:32.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:32.146+0000 7f2062b4a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f205c15a6f0 0x7f205c145bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:32.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:32.146+0000 7f2062b4a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f205c15a6f0 0x7f205c145bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60086/0 (socket says 172.21.15.38:60086) 2024-09-21T03:42:32.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:32.146+0000 7f2062b4a640 1 -- 172.21.15.38:0/494838523 learned_addr learned my addr 172.21.15.38:0/494838523 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:42:32.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:32.146+0000 7f2062349640 1 --2- 172.21.15.38:0/494838523 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f205c0d01f0 0x7f205c15c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:32.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:32.146+0000 7f2061b48640 1 --2- 172.21.15.38:0/494838523 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f205c0d0980 0x7f205c1456b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:32.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:32.147+0000 7f2062b4a640 1 -- 172.21.15.38:0/494838523 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f205c0d01f0 msgr2=0x7f205c15c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:32.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:32.147+0000 7f2062b4a640 1 --2- 172.21.15.38:0/494838523 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f205c0d01f0 0x7f205c15c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:32.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:32.147+0000 7f2062b4a640 1 -- 172.21.15.38:0/494838523 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f205c0d0980 msgr2=0x7f205c1456b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:32.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:32.147+0000 7f2062b4a640 1 --2- 172.21.15.38:0/494838523 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f205c0d0980 0x7f205c1456b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:32.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:32.147+0000 7f2062b4a640 1 -- 172.21.15.38:0/494838523 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f205c1551f0 con 0x7f205c15a6f0 2024-09-21T03:42:32.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:32.147+0000 7f2062b4a640 1 --2- 172.21.15.38:0/494838523 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f205c15a6f0 0x7f205c145bf0 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7f205400c3f0 tx=0x7f205400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:32.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:32.147+0000 7f205affd640 1 -- 172.21.15.38:0/494838523 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2054017070 con 0x7f205c15a6f0 2024-09-21T03:42:32.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:32.147+0000 7f205affd640 1 -- 172.21.15.38:0/494838523 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f20540053c0 con 0x7f205c15a6f0 2024-09-21T03:42:32.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:32.147+0000 7f20645ad640 1 -- 172.21.15.38:0/494838523 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f205c1554b0 con 0x7f205c15a6f0 2024-09-21T03:42:32.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:32.147+0000 7f205affd640 1 -- 172.21.15.38:0/494838523 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2054003850 con 0x7f205c15a6f0 2024-09-21T03:42:32.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:32.147+0000 7f20645ad640 1 -- 172.21.15.38:0/494838523 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f205c155a70 con 0x7f205c15a6f0 2024-09-21T03:42:32.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:32.148+0000 7f20645ad640 1 -- 172.21.15.38:0/494838523 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f205c0cd9b0 con 0x7f205c15a6f0 2024-09-21T03:42:32.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:32.149+0000 7f205affd640 1 -- 172.21.15.38:0/494838523 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f205401a070 con 0x7f205c15a6f0 2024-09-21T03:42:32.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:32.150+0000 7f205affd640 1 --2- 172.21.15.38:0/494838523 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2048078b90 0x7f204807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:32.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:32.150+0000 7f205affd640 1 -- 172.21.15.38:0/494838523 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 8196+0+0 (secure 0 0 0) 0x7f205409aa90 con 0x7f205c15a6f0 2024-09-21T03:42:32.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:32.150+0000 7f2062349640 1 --2- 172.21.15.38:0/494838523 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2048078b90 0x7f204807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:32.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:32.150+0000 7f2062349640 1 --2- 172.21.15.38:0/494838523 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2048078b90 0x7f204807b0a0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f204c005fd0 tx=0x7f204c0095e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:32.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:32.154+0000 7f205affd640 1 -- 172.21.15.38:0/494838523 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f2054061ed0 con 0x7f205c15a6f0 2024-09-21T03:42:32.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:32.382+0000 7f20645ad640 1 -- 172.21.15.38:0/494838523 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f205c1f7c80 con 0x7f205c15a6f0 2024-09-21T03:42:32.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:32.383+0000 7f205affd640 1 -- 172.21.15.38:0/494838523 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 215 v215) ==== 95+0+4389 (secure 0 0 0) 0x7f2054066900 con 0x7f205c15a6f0 2024-09-21T03:42:32.385 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:42:32.385 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":215,"btime":"2024-09-21T03:42:32:279452+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6086,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/965972663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":965972663},{"type":"v1","addr":"172.21.15.195:6836","nonce":965972663}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6700,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/3176122334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":3176122334},{"type":"v1","addr":"172.21.15.195:6837","nonce":3176122334}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":9189,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/245538755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":245538755},{"type":"v1","addr":"172.21.15.38:6833","nonce":245538755}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":215}],"filesystems":[{"mdsmap":{"epoch":214,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:40:02.319282+0000","modified":"2024-09-21T03:42:31.276380+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":118,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-21T03:42:32.385 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 215 2024-09-21T03:42:32.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:32.385+0000 7f20645ad640 1 -- 172.21.15.38:0/494838523 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2048078b90 msgr2=0x7f204807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:32.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:32.385+0000 7f20645ad640 1 --2- 172.21.15.38:0/494838523 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2048078b90 0x7f204807b0a0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f204c005fd0 tx=0x7f204c0095e0 comp rx=0 tx=0).stop 2024-09-21T03:42:32.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:32.385+0000 7f20645ad640 1 -- 172.21.15.38:0/494838523 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f205c15a6f0 msgr2=0x7f205c145bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:32.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:32.385+0000 7f20645ad640 1 --2- 172.21.15.38:0/494838523 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f205c15a6f0 0x7f205c145bf0 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7f205400c3f0 tx=0x7f205400c8b0 comp rx=0 tx=0).stop 2024-09-21T03:42:32.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:32.385+0000 7f20645ad640 1 -- 172.21.15.38:0/494838523 shutdown_connections 2024-09-21T03:42:32.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:32.385+0000 7f20645ad640 1 --2- 172.21.15.38:0/494838523 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f205c15a6f0 0x7f205c145bf0 unknown :-1 s=CLOSED pgs=1770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:32.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:32.385+0000 7f20645ad640 1 --2- 172.21.15.38:0/494838523 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f205c0d0980 0x7f205c1456b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:32.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:32.385+0000 7f20645ad640 1 --2- 172.21.15.38:0/494838523 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2048078b90 0x7f204807b0a0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:32.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:32.385+0000 7f20645ad640 1 --2- 172.21.15.38:0/494838523 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f205c0d01f0 0x7f205c15c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:32.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:32.385+0000 7f20645ad640 1 -- 172.21.15.38:0/494838523 >> 172.21.15.38:0/494838523 conn(0x7f205c072990 msgr2=0x7f205c15b810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:32.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:32.385+0000 7f20645ad640 1 -- 172.21.15.38:0/494838523 shutdown_connections 2024-09-21T03:42:32.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:32.385+0000 7f20645ad640 1 -- 172.21.15.38:0/494838523 wait complete. 2024-09-21T03:42:32.398 INFO:tasks.cephfs.test_damage:Result: Mutation 'Corrupt 400.00000000' led to DAMAGED state 2024-09-21T03:42:32.398 INFO:tasks.cephfs.test_damage:Applying mutation 'Corrupt 100.00000000.inode' 2024-09-21T03:42:32.398 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-21T03:42:32.398 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:42:32.454 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056795,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2354 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T03:42:32.454 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi038.front.sepia.ceph.com 2024-09-21T03:42:32.454 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:42:32.455 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T03:42:32.510 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T03:42:32.510 INFO:teuthology.orchestra.run.smithi038.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-09-21T03:42:32.511 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-21T03:42:32.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:32.666+0000 7f197c638640 1 -- 172.21.15.38:0/3078776719 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f19740cd440 msgr2=0x7f19740d0670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:32.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:32.666+0000 7f197c638640 1 --2- 172.21.15.38:0/3078776719 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f19740cd440 0x7f19740d0670 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f1968009830 tx=0x7f196802a480 comp rx=0 tx=0).stop 2024-09-21T03:42:32.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:32.666+0000 7f197c638640 1 -- 172.21.15.38:0/3078776719 shutdown_connections 2024-09-21T03:42:32.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:32.666+0000 7f197c638640 1 --2- 172.21.15.38:0/3078776719 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f19740cd440 0x7f19740d0670 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:32.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:32.666+0000 7f197c638640 1 --2- 172.21.15.38:0/3078776719 >> v2:172.21.15.195:3300/0 conn(0x7f19740ccc40 0x7f197415ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:32.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:32.666+0000 7f197c638640 1 -- 172.21.15.38:0/3078776719 >> 172.21.15.38:0/3078776719 conn(0x7f19740b46e0 msgr2=0x7f19740d23e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:32.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:32.666+0000 7f197c638640 1 -- 172.21.15.38:0/3078776719 shutdown_connections 2024-09-21T03:42:32.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:32.666+0000 7f197c638640 1 -- 172.21.15.38:0/3078776719 wait complete. 2024-09-21T03:42:32.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:32.668+0000 7f197c638640 1 Processor -- start 2024-09-21T03:42:32.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:32.668+0000 7f197c638640 1 -- start start 2024-09-21T03:42:32.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:32.669+0000 7f197c638640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f19740ccc40 0x7f197415eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:32.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:32.669+0000 7f197c638640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f19740cd440 0x7f19740cb840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:32.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:32.669+0000 7f197c638640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f19740d0bb0 0x7f19740a9450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:32.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:32.669+0000 7f197c638640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f1974070810 con 0x7f19740d0bb0 2024-09-21T03:42:32.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:32.669+0000 7f197c638640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f19740b6e00 con 0x7f19740ccc40 2024-09-21T03:42:32.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:32.669+0000 7f197c638640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f1974070990 con 0x7f19740cd440 2024-09-21T03:42:32.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:32.669+0000 7f197a3d4640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f19740ccc40 0x7f197415eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:32.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:32.669+0000 7f197abd5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f19740d0bb0 0x7f19740a9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:32.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:32.669+0000 7f197a3d4640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f19740ccc40 0x7f197415eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.38:47894/0 (socket says 172.21.15.38:47894) 2024-09-21T03:42:32.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:32.669+0000 7f197a3d4640 1 -- 172.21.15.38:0/2990333293 learned_addr learned my addr 172.21.15.38:0/2990333293 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:42:32.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:32.669+0000 7f1979bd3640 1 --2- 172.21.15.38:0/2990333293 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f19740cd440 0x7f19740cb840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:32.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:32.670+0000 7f197a3d4640 1 -- 172.21.15.38:0/2990333293 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f19740cd440 msgr2=0x7f19740cb840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:32.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:32.670+0000 7f197a3d4640 1 --2- 172.21.15.38:0/2990333293 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f19740cd440 0x7f19740cb840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:32.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:32.670+0000 7f197a3d4640 1 -- 172.21.15.38:0/2990333293 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f19740d0bb0 msgr2=0x7f19740a9450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:32.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:32.670+0000 7f197a3d4640 1 --2- 172.21.15.38:0/2990333293 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f19740d0bb0 0x7f19740a9450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:32.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:32.670+0000 7f197a3d4640 1 -- 172.21.15.38:0/2990333293 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f197415f620 con 0x7f19740ccc40 2024-09-21T03:42:32.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:32.670+0000 7f197a3d4640 1 --2- 172.21.15.38:0/2990333293 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f19740ccc40 0x7f197415eb60 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f196400ecb0 tx=0x7f196400c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:32.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:32.670+0000 7f1972ffd640 1 -- 172.21.15.38:0/2990333293 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1964003b70 con 0x7f19740ccc40 2024-09-21T03:42:32.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:32.670+0000 7f1972ffd640 1 -- 172.21.15.38:0/2990333293 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1964003d10 con 0x7f19740ccc40 2024-09-21T03:42:32.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:32.670+0000 7f197c638640 1 -- 172.21.15.38:0/2990333293 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f197415d120 con 0x7f19740ccc40 2024-09-21T03:42:32.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:32.670+0000 7f1972ffd640 1 -- 172.21.15.38:0/2990333293 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1964004280 con 0x7f19740ccc40 2024-09-21T03:42:32.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:32.670+0000 7f197c638640 1 -- 172.21.15.38:0/2990333293 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f197415d6e0 con 0x7f19740ccc40 2024-09-21T03:42:32.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:32.671+0000 7f197c638640 1 -- 172.21.15.38:0/2990333293 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1974070ed0 con 0x7f19740ccc40 2024-09-21T03:42:32.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:32.672+0000 7f1972ffd640 1 -- 172.21.15.38:0/2990333293 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f1964004b40 con 0x7f19740ccc40 2024-09-21T03:42:32.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:32.672+0000 7f1972ffd640 1 --2- 172.21.15.38:0/2990333293 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1960078bc0 0x7f196007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:32.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:32.673+0000 7f1972ffd640 1 -- 172.21.15.38:0/2990333293 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 8196+0+0 (secure 0 0 0) 0x7f196409c5b0 con 0x7f19740ccc40 2024-09-21T03:42:32.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:32.673+0000 7f1979bd3640 1 --2- 172.21.15.38:0/2990333293 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1960078bc0 0x7f196007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:32.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:32.673+0000 7f1979bd3640 1 --2- 172.21.15.38:0/2990333293 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1960078bc0 0x7f196007b0d0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f1968034000 tx=0x7f1968004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:32.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:32.676+0000 7f1972ffd640 1 -- 172.21.15.38:0/2990333293 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f1964063a40 con 0x7f19740ccc40 2024-09-21T03:42:32.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:32.900+0000 7f197c638640 1 -- 172.21.15.38:0/2990333293 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f1974070350 con 0x7f19740ccc40 2024-09-21T03:42:33.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:33.291+0000 7f1972ffd640 1 -- 172.21.15.38:0/2990333293 <== mon.1 v2:172.21.15.195:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v216) ==== 161+0+0 (secure 0 0 0) 0x7f1964068470 con 0x7f19740ccc40 2024-09-21T03:42:33.293 INFO:teuthology.orchestra.run.smithi038.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-21T03:42:33.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:33.294+0000 7f197c638640 1 -- 172.21.15.38:0/2990333293 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1960078bc0 msgr2=0x7f196007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:33.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:33.294+0000 7f197c638640 1 --2- 172.21.15.38:0/2990333293 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1960078bc0 0x7f196007b0d0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f1968034000 tx=0x7f1968004730 comp rx=0 tx=0).stop 2024-09-21T03:42:33.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:33.294+0000 7f197c638640 1 -- 172.21.15.38:0/2990333293 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f19740ccc40 msgr2=0x7f197415eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:33.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:33.294+0000 7f197c638640 1 --2- 172.21.15.38:0/2990333293 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f19740ccc40 0x7f197415eb60 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f196400ecb0 tx=0x7f196400c6a0 comp rx=0 tx=0).stop 2024-09-21T03:42:33.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:33.294+0000 7f197c638640 1 -- 172.21.15.38:0/2990333293 shutdown_connections 2024-09-21T03:42:33.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:33.294+0000 7f197c638640 1 --2- 172.21.15.38:0/2990333293 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f19740d0bb0 0x7f19740a9450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:33.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:33.294+0000 7f197c638640 1 --2- 172.21.15.38:0/2990333293 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1960078bc0 0x7f196007b0d0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:33.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:33.294+0000 7f197c638640 1 --2- 172.21.15.38:0/2990333293 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f19740cd440 0x7f19740cb840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:33.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:33.294+0000 7f197c638640 1 --2- 172.21.15.38:0/2990333293 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f19740ccc40 0x7f197415eb60 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:33.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:33.294+0000 7f197c638640 1 -- 172.21.15.38:0/2990333293 >> 172.21.15.38:0/2990333293 conn(0x7f19740b46e0 msgr2=0x7f19740b2af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:33.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:33.294+0000 7f197c638640 1 -- 172.21.15.38:0/2990333293 shutdown_connections 2024-09-21T03:42:33.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:33.294+0000 7f197c638640 1 -- 172.21.15.38:0/2990333293 wait complete. 2024-09-21T03:42:33.309 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-09-21T03:42:33.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:33.427+0000 7f8a46d20640 1 -- 172.21.15.38:0/3639118282 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8a28004800 msgr2=0x7f8a28004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:33.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:33.427+0000 7f8a46d20640 1 --2- 172.21.15.38:0/3639118282 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8a28004800 0x7f8a28004c10 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7f8a400d1410 tx=0x7f8a28005830 comp rx=0 tx=0).stop 2024-09-21T03:42:33.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:33.427+0000 7f8a452bd640 1 -- 172.21.15.38:0/3639118282 reap_dead start 2024-09-21T03:42:33.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:33.427+0000 7f8a46d20640 1 -- 172.21.15.38:0/3639118282 shutdown_connections 2024-09-21T03:42:33.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:33.427+0000 7f8a46d20640 1 -- 172.21.15.38:0/3639118282 >> 172.21.15.38:0/3639118282 conn(0x7f8a400b46e0 msgr2=0x7f8a400d23e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:33.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:33.427+0000 7f8a46d20640 1 -- 172.21.15.38:0/3639118282 shutdown_connections 2024-09-21T03:42:33.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:33.427+0000 7f8a46d20640 1 -- 172.21.15.38:0/3639118282 wait complete. 2024-09-21T03:42:33.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:33.429+0000 7f8a46d20640 1 Processor -- start 2024-09-21T03:42:33.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:33.429+0000 7f8a46d20640 1 -- start start 2024-09-21T03:42:33.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:33.431+0000 7f8a46d20640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8a28004800 0x7f8a401507d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:33.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:33.431+0000 7f8a46d20640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8a40154a90 0x7f8a40154ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:33.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:33.431+0000 7f8a46d20640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8a4015f400 0x7f8a4015f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:33.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:33.431+0000 7f8a46d20640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f8a400b5d60 con 0x7f8a40154a90 2024-09-21T03:42:33.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:33.431+0000 7f8a46d20640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f8a400b5be0 con 0x7f8a28004800 2024-09-21T03:42:33.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:33.431+0000 7f8a46d20640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f8a400b5ee0 con 0x7f8a4015f400 2024-09-21T03:42:33.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:33.431+0000 7f8a37fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8a40154a90 0x7f8a40154ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:33.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:33.431+0000 7f8a37fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8a40154a90 0x7f8a40154ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:52074/0 (socket says 172.21.15.38:52074) 2024-09-21T03:42:33.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:33.431+0000 7f8a37fff640 1 -- 172.21.15.38:0/2944789999 learned_addr learned my addr 172.21.15.38:0/2944789999 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:42:33.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:33.431+0000 7f8a44abc640 1 --2- 172.21.15.38:0/2944789999 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8a28004800 0x7f8a401507d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:33.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:33.431+0000 7f8a452bd640 1 --2- 172.21.15.38:0/2944789999 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8a4015f400 0x7f8a4015f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:33.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:33.431+0000 7f8a37fff640 1 -- 172.21.15.38:0/2944789999 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8a4015f400 msgr2=0x7f8a4015f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:33.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:33.431+0000 7f8a37fff640 1 --2- 172.21.15.38:0/2944789999 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8a4015f400 0x7f8a4015f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:33.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:33.431+0000 7f8a37fff640 1 -- 172.21.15.38:0/2944789999 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8a28004800 msgr2=0x7f8a401507d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:33.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:33.431+0000 7f8a37fff640 1 --2- 172.21.15.38:0/2944789999 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8a28004800 0x7f8a401507d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:33.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:33.431+0000 7f8a37fff640 1 -- 172.21.15.38:0/2944789999 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a4015fd50 con 0x7f8a40154a90 2024-09-21T03:42:33.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:33.431+0000 7f8a37fff640 1 --2- 172.21.15.38:0/2944789999 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8a40154a90 0x7f8a40154ea0 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7f8a3000c3a0 tx=0x7f8a30006d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:33.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:33.432+0000 7f8a357fa640 1 -- 172.21.15.38:0/2944789999 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a30031040 con 0x7f8a40154a90 2024-09-21T03:42:33.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:33.432+0000 7f8a357fa640 1 -- 172.21.15.38:0/2944789999 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8a30003610 con 0x7f8a40154a90 2024-09-21T03:42:33.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:33.432+0000 7f8a46d20640 1 -- 172.21.15.38:0/2944789999 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a401f32d0 con 0x7f8a40154a90 2024-09-21T03:42:33.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:33.432+0000 7f8a357fa640 1 -- 172.21.15.38:0/2944789999 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a30003a60 con 0x7f8a40154a90 2024-09-21T03:42:33.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:33.432+0000 7f8a46d20640 1 -- 172.21.15.38:0/2944789999 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a401f3870 con 0x7f8a40154a90 2024-09-21T03:42:33.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:33.433+0000 7f8a46d20640 1 -- 172.21.15.38:0/2944789999 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a40155570 con 0x7f8a40154a90 2024-09-21T03:42:33.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:33.434+0000 7f8a357fa640 1 -- 172.21.15.38:0/2944789999 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f8a30034070 con 0x7f8a40154a90 2024-09-21T03:42:33.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:33.434+0000 7f8a357fa640 1 --2- 172.21.15.38:0/2944789999 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8a2c078b90 0x7f8a2c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:33.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:33.434+0000 7f8a357fa640 1 -- 172.21.15.38:0/2944789999 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 8196+0+0 (secure 0 0 0) 0x7f8a300bb660 con 0x7f8a40154a90 2024-09-21T03:42:33.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:33.435+0000 7f8a44abc640 1 --2- 172.21.15.38:0/2944789999 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8a2c078b90 0x7f8a2c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:33.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:33.435+0000 7f8a44abc640 1 --2- 172.21.15.38:0/2944789999 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8a2c078b90 0x7f8a2c07b0a0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c009c50 tx=0x7f8a3c005f90 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:33.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:33.439+0000 7f8a357fa640 1 -- 172.21.15.38:0/2944789999 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f8a4015fd50 con 0x7f8a40154a90 2024-09-21T03:42:33.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:33.702+0000 7f8a46d20640 1 -- 172.21.15.38:0/2944789999 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7f8a40071110 con 0x7f8a40154a90 2024-09-21T03:42:34.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.291+0000 7f8a357fa640 1 -- 172.21.15.38:0/2944789999 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v217) ==== 101+0+0 (secure 0 0 0) 0x7f8a30083090 con 0x7f8a40154a90 2024-09-21T03:42:34.293 INFO:teuthology.orchestra.run.smithi038.stderr:repaired: restoring rank 6:0 2024-09-21T03:42:34.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.294+0000 7f8a46d20640 1 -- 172.21.15.38:0/2944789999 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8a2c078b90 msgr2=0x7f8a2c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:34.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.294+0000 7f8a46d20640 1 --2- 172.21.15.38:0/2944789999 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8a2c078b90 0x7f8a2c07b0a0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c009c50 tx=0x7f8a3c005f90 comp rx=0 tx=0).stop 2024-09-21T03:42:34.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.294+0000 7f8a46d20640 1 -- 172.21.15.38:0/2944789999 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8a40154a90 msgr2=0x7f8a40154ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:34.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.294+0000 7f8a46d20640 1 --2- 172.21.15.38:0/2944789999 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8a40154a90 0x7f8a40154ea0 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7f8a3000c3a0 tx=0x7f8a30006d00 comp rx=0 tx=0).stop 2024-09-21T03:42:34.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.294+0000 7f8a46d20640 1 -- 172.21.15.38:0/2944789999 shutdown_connections 2024-09-21T03:42:34.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.294+0000 7f8a46d20640 1 --2- 172.21.15.38:0/2944789999 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8a2c078b90 0x7f8a2c07b0a0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:34.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.294+0000 7f8a46d20640 1 --2- 172.21.15.38:0/2944789999 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8a4015f400 0x7f8a4015f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:34.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.294+0000 7f8a46d20640 1 --2- 172.21.15.38:0/2944789999 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8a40154a90 0x7f8a40154ea0 unknown :-1 s=CLOSED pgs=1773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:34.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.294+0000 7f8a46d20640 1 --2- 172.21.15.38:0/2944789999 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8a28004800 0x7f8a401507d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:34.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.294+0000 7f8a46d20640 1 -- 172.21.15.38:0/2944789999 >> 172.21.15.38:0/2944789999 conn(0x7f8a400b46e0 msgr2=0x7f8a40068e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:34.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.294+0000 7f8a46d20640 1 -- 172.21.15.38:0/2944789999 shutdown_connections 2024-09-21T03:42:34.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.295+0000 7f8a46d20640 1 -- 172.21.15.38:0/2944789999 wait complete. 2024-09-21T03:42:34.310 DEBUG:teuthology.orchestra.run.smithi038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-09-21T03:42:34.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.369+0000 7f7382f20f40 1 Processor -- start 2024-09-21T03:42:34.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.369+0000 7f7382f20f40 1 -- start start 2024-09-21T03:42:34.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.370+0000 7f7382f20f40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5606f23cc240 0x5606f23caec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:34.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.370+0000 7f7382f20f40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5606f2413090 0x5606f24a1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:34.374 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.370+0000 7f7382f20f40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5606f2488520 0x5606f24a1750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:34.374 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.370+0000 7f7382f20f40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5606f2319d60 con 0x5606f2413090 2024-09-21T03:42:34.374 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.370+0000 7f7382f20f40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5606f2378430 con 0x5606f2488520 2024-09-21T03:42:34.374 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.370+0000 7f7382f20f40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x5606f249faa0 con 0x5606f23cc240 2024-09-21T03:42:34.374 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.370+0000 7f737b8bf640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5606f2413090 0x5606f24a1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:34.374 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.370+0000 7f737b8bf640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5606f2413090 0x5606f24a1210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:52092/0 (socket says 172.21.15.38:52092) 2024-09-21T03:42:34.374 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.370+0000 7f737b8bf640 1 -- 172.21.15.38:0/1236086874 learned_addr learned my addr 172.21.15.38:0/1236086874 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:42:34.374 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.370+0000 7f737c0c0640 1 --2- 172.21.15.38:0/1236086874 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5606f23cc240 0x5606f23caec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:34.374 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.370+0000 7f737c8c1640 1 --2- 172.21.15.38:0/1236086874 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5606f2488520 0x5606f24a1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:34.374 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.370+0000 7f737b8bf640 1 -- 172.21.15.38:0/1236086874 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5606f23cc240 msgr2=0x5606f23caec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:34.374 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.370+0000 7f737b8bf640 1 --2- 172.21.15.38:0/1236086874 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5606f23cc240 0x5606f23caec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:34.374 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.370+0000 7f737b8bf640 1 -- 172.21.15.38:0/1236086874 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5606f2488520 msgr2=0x5606f24a1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:34.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.370+0000 7f737b8bf640 1 --2- 172.21.15.38:0/1236086874 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5606f2488520 0x5606f24a1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:34.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.370+0000 7f737b8bf640 1 -- 172.21.15.38:0/1236086874 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5606f2499340 con 0x5606f2413090 2024-09-21T03:42:34.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.371+0000 7f737b8bf640 1 --2- 172.21.15.38:0/1236086874 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5606f2413090 0x5606f24a1210 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7f737000c5d0 tx=0x7f737000ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f23adf5625515b76 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:34.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.371+0000 7f737a8bd640 1 -- 172.21.15.38:0/1236086874 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f737000aae0 con 0x5606f2413090 2024-09-21T03:42:34.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.371+0000 7f737a8bd640 1 -- 172.21.15.38:0/1236086874 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f737000ac80 con 0x5606f2413090 2024-09-21T03:42:34.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.371+0000 7f737a8bd640 1 -- 172.21.15.38:0/1236086874 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73700156a0 con 0x5606f2413090 2024-09-21T03:42:34.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.371+0000 7f7382f20f40 1 -- 172.21.15.38:0/1236086874 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5606f2413090 msgr2=0x5606f24a1210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:34.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.371+0000 7f7382f20f40 1 --2- 172.21.15.38:0/1236086874 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5606f2413090 0x5606f24a1210 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7f737000c5d0 tx=0x7f737000ca90 comp rx=0 tx=0).stop 2024-09-21T03:42:34.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.372+0000 7f7382f20f40 1 -- 172.21.15.38:0/1236086874 shutdown_connections 2024-09-21T03:42:34.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.372+0000 7f7382f20f40 1 --2- 172.21.15.38:0/1236086874 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5606f2488520 0x5606f24a1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:34.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.372+0000 7f7382f20f40 1 --2- 172.21.15.38:0/1236086874 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5606f2413090 0x5606f24a1210 unknown :-1 s=CLOSED pgs=1775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:34.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.372+0000 7f7382f20f40 1 --2- 172.21.15.38:0/1236086874 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5606f23cc240 0x5606f23caec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:34.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.372+0000 7f7382f20f40 1 -- 172.21.15.38:0/1236086874 >> 172.21.15.38:0/1236086874 conn(0x5606f22eaac0 msgr2=0x5606f249dec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:34.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.372+0000 7f7382f20f40 1 -- 172.21.15.38:0/1236086874 shutdown_connections 2024-09-21T03:42:34.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.372+0000 7f7382f20f40 1 -- 172.21.15.38:0/1236086874 wait complete. 2024-09-21T03:42:34.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.372+0000 7f7382f20f40 1 Processor -- start 2024-09-21T03:42:34.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.373+0000 7f7382f20f40 1 -- start start 2024-09-21T03:42:34.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.374+0000 7f7382f20f40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5606f23cc240 0x5606f249bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:34.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.374+0000 7f7382f20f40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5606f2413090 0x5606f252c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:34.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.374+0000 7f7382f20f40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5606f2488520 0x5606f247ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:34.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.374+0000 7f7382f20f40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5606f24a0cd0 con 0x5606f2413090 2024-09-21T03:42:34.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.374+0000 7f7382f20f40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5606f24a0560 con 0x5606f2488520 2024-09-21T03:42:34.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.374+0000 7f7382f20f40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x5606f249c0b0 con 0x5606f23cc240 2024-09-21T03:42:34.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.374+0000 7f737b8bf640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5606f2413090 0x5606f252c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:34.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.374+0000 7f737b8bf640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5606f2413090 0x5606f252c210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:52106/0 (socket says 172.21.15.38:52106) 2024-09-21T03:42:34.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.374+0000 7f737b8bf640 1 -- 172.21.15.38:0/3284913117 learned_addr learned my addr 172.21.15.38:0/3284913117 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:42:34.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.374+0000 7f737c8c1640 1 --2- 172.21.15.38:0/3284913117 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5606f2488520 0x5606f247ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:34.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.374+0000 7f737c0c0640 1 --2- 172.21.15.38:0/3284913117 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5606f23cc240 0x5606f249bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:34.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.374+0000 7f737b8bf640 1 -- 172.21.15.38:0/3284913117 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5606f23cc240 msgr2=0x5606f249bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:34.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.374+0000 7f737b8bf640 1 --2- 172.21.15.38:0/3284913117 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5606f23cc240 0x5606f249bae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:34.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.374+0000 7f737b8bf640 1 -- 172.21.15.38:0/3284913117 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5606f2488520 msgr2=0x5606f247ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:34.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.374+0000 7f737b8bf640 1 --2- 172.21.15.38:0/3284913117 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5606f2488520 0x5606f247ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:34.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.374+0000 7f737b8bf640 1 -- 172.21.15.38:0/3284913117 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5606f253bf40 con 0x5606f2413090 2024-09-21T03:42:34.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.375+0000 7f737b8bf640 1 --2- 172.21.15.38:0/3284913117 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5606f2413090 0x5606f252c210 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7f7370004160 tx=0x7f7370015440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:34.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.375+0000 7f73798bb640 1 -- 172.21.15.38:0/3284913117 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7370015a40 con 0x5606f2413090 2024-09-21T03:42:34.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.375+0000 7f73798bb640 1 -- 172.21.15.38:0/3284913117 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7370015cd0 con 0x5606f2413090 2024-09-21T03:42:34.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.375+0000 7f73798bb640 1 -- 172.21.15.38:0/3284913117 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f737001f690 con 0x5606f2413090 2024-09-21T03:42:34.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.375+0000 7f7382f20f40 1 -- 172.21.15.38:0/3284913117 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5606f253c220 con 0x5606f2413090 2024-09-21T03:42:34.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.375+0000 7f7382f20f40 1 -- 172.21.15.38:0/3284913117 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5606f253c7e0 con 0x5606f2413090 2024-09-21T03:42:34.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.377+0000 7f73798bb640 1 -- 172.21.15.38:0/3284913117 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f7370026060 con 0x5606f2413090 2024-09-21T03:42:34.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.377+0000 7f73798bb640 1 --2- 172.21.15.38:0/3284913117 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f73680775a0 0x7f7368079ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:34.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.377+0000 7f73798bb640 1 -- 172.21.15.38:0/3284913117 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 8196+0+0 (secure 0 0 0) 0x7f737009bb00 con 0x5606f2413090 2024-09-21T03:42:34.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.377+0000 7f737c0c0640 1 --2- 172.21.15.38:0/3284913117 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f73680775a0 0x7f7368079ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:34.380 INFO:teuthology.orchestra.run.smithi038.stdout:Importing pool 2024-09-21T03:42:34.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.377+0000 7f7382f20f40 1 --2- 172.21.15.38:0/3284913117 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x5606f2538930 0x5606f2538d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:34.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.377+0000 7f7382f20f40 1 -- 172.21.15.38:0/3284913117 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x5606f2539280 con 0x5606f2538930 2024-09-21T03:42:34.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.378+0000 7f737c8c1640 1 --2- 172.21.15.38:0/3284913117 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x5606f2538930 0x5606f2538d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:34.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.378+0000 7f737c0c0640 1 --2- 172.21.15.38:0/3284913117 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f73680775a0 0x7f7368079ab0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f736c056000 tx=0x7f736c0033a0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:34.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.378+0000 7f737c8c1640 1 --2- 172.21.15.38:0/3284913117 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x5606f2538930 0x5606f2538d40 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:34.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.380+0000 7f737c8c1640 1 -- 172.21.15.38:0/3284913117 <== osd.7 v2:172.21.15.195:6818/1606106342 1 ==== osd_op_reply(1 601.00000000 [create] v118'660 uv634 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7374011520 con 0x5606f2538930 2024-09-21T03:42:34.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.380+0000 7f7382f20f40 1 -- 172.21.15.38:0/3284913117 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e118) -- 0x5606f2544c80 con 0x5606f2538930 2024-09-21T03:42:34.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.382+0000 7f737c8c1640 1 -- 172.21.15.38:0/3284913117 <== osd.7 v2:172.21.15.195:6818/1606106342 2 ==== osd_op_reply(2 601.00000000 [delete] v118'661 uv661 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7374011520 con 0x5606f2538930 2024-09-21T03:42:34.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.382+0000 7f7382f20f40 1 -- 172.21.15.38:0/3284913117 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x5606f2545020 con 0x5606f2538930 2024-09-21T03:42:34.385 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-09-21T03:42:34.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.383+0000 7f737c8c1640 1 -- 172.21.15.38:0/3284913117 <== osd.7 v2:172.21.15.195:6818/1606106342 3 ==== osd_op_reply(3 601.00000000 [create] v118'662 uv662 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7374011520 con 0x5606f2538930 2024-09-21T03:42:34.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.383+0000 7f7382f20f40 1 -- 172.21.15.38:0/3284913117 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x5606f25463d0 con 0x5606f2538930 2024-09-21T03:42:34.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.385+0000 7f737c8c1640 1 -- 172.21.15.38:0/3284913117 <== osd.7 v2:172.21.15.195:6818/1606106342 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v118'663 uv663 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7374011520 con 0x5606f2538930 2024-09-21T03:42:34.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.385+0000 7f7382f20f40 1 -- 172.21.15.38:0/3284913117 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x5606f2546770 con 0x5606f2538930 2024-09-21T03:42:34.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.386+0000 7f737c8c1640 1 -- 172.21.15.38:0/3284913117 <== osd.7 v2:172.21.15.195:6818/1606106342 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v118'664 uv664 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7374011520 con 0x5606f2538930 2024-09-21T03:42:34.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.386+0000 7f7382f20f40 1 -- 172.21.15.38:0/3284913117 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x5606f25453c0 con 0x5606f2538930 2024-09-21T03:42:34.390 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.388+0000 7f737c8c1640 1 -- 172.21.15.38:0/3284913117 <== osd.7 v2:172.21.15.195:6818/1606106342 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v118'665 uv665 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7374011520 con 0x5606f2538930 2024-09-21T03:42:34.390 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.388+0000 7f7382f20f40 1 -- 172.21.15.38:0/3284913117 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x5606f25457b0 con 0x5606f2538930 2024-09-21T03:42:34.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.390+0000 7f737c8c1640 1 -- 172.21.15.38:0/3284913117 <== osd.7 v2:172.21.15.195:6818/1606106342 7 ==== osd_op_reply(7 602.00000000 [create] v118'666 uv640 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7374011520 con 0x5606f2538930 2024-09-21T03:42:34.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.390+0000 7f7382f20f40 1 -- 172.21.15.38:0/3284913117 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e118) -- 0x5606f2545ba0 con 0x5606f2538930 2024-09-21T03:42:34.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.391+0000 7f737c8c1640 1 -- 172.21.15.38:0/3284913117 <== osd.7 v2:172.21.15.195:6818/1606106342 8 ==== osd_op_reply(8 602.00000000 [delete] v118'667 uv667 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7374011520 con 0x5606f2538930 2024-09-21T03:42:34.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.391+0000 7f7382f20f40 1 -- 172.21.15.38:0/3284913117 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x5606f2545ba0 con 0x5606f2538930 2024-09-21T03:42:34.395 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-09-21T03:42:34.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.393+0000 7f737c8c1640 1 -- 172.21.15.38:0/3284913117 <== osd.7 v2:172.21.15.195:6818/1606106342 9 ==== osd_op_reply(9 602.00000000 [create] v118'668 uv668 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7374011520 con 0x5606f2538930 2024-09-21T03:42:34.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.393+0000 7f7382f20f40 1 -- 172.21.15.38:0/3284913117 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x5606f2545b50 con 0x5606f2538930 2024-09-21T03:42:34.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.394+0000 7f737c8c1640 1 -- 172.21.15.38:0/3284913117 <== osd.7 v2:172.21.15.195:6818/1606106342 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v118'669 uv669 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7374011520 con 0x5606f2538930 2024-09-21T03:42:34.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.394+0000 7f7382f20f40 1 -- 172.21.15.38:0/3284913117 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x5606f2545b50 con 0x5606f2538930 2024-09-21T03:42:34.398 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.396+0000 7f737c8c1640 1 -- 172.21.15.38:0/3284913117 <== osd.7 v2:172.21.15.195:6818/1606106342 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v118'670 uv670 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7374011520 con 0x5606f2538930 2024-09-21T03:42:34.398 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.396+0000 7f7382f20f40 1 -- 172.21.15.38:0/3284913117 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x5606f2545b50 con 0x5606f2538930 2024-09-21T03:42:34.399 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.398+0000 7f737c8c1640 1 -- 172.21.15.38:0/3284913117 <== osd.7 v2:172.21.15.195:6818/1606106342 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v118'671 uv671 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7374011520 con 0x5606f2538930 2024-09-21T03:42:34.399 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.398+0000 7f7382f20f40 1 -- 172.21.15.38:0/3284913117 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:13 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x5606f2545b50 con 0x5606f2538930 2024-09-21T03:42:34.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.400+0000 7f737c8c1640 1 -- 172.21.15.38:0/3284913117 <== osd.7 v2:172.21.15.195:6818/1606106342 13 ==== osd_op_reply(13 10000000000.00000000 [create] v118'672 uv647 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7f7374011520 con 0x5606f2538930 2024-09-21T03:42:34.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.400+0000 7f7382f20f40 1 -- 172.21.15.38:0/3284913117 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:14 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e118) -- 0x5606f2545b50 con 0x5606f2538930 2024-09-21T03:42:34.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.401+0000 7f737c8c1640 1 -- 172.21.15.38:0/3284913117 <== osd.7 v2:172.21.15.195:6818/1606106342 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v118'673 uv673 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f7374011520 con 0x5606f2538930 2024-09-21T03:42:34.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.401+0000 7f7382f20f40 1 -- 172.21.15.38:0/3284913117 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:15 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x5606f2545b50 con 0x5606f2538930 2024-09-21T03:42:34.404 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-09-21T03:42:34.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.402+0000 7f737c8c1640 1 -- 172.21.15.38:0/3284913117 <== osd.7 v2:172.21.15.195:6818/1606106342 15 ==== osd_op_reply(15 10000000000.00000000 [create] v118'674 uv674 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f7374011520 con 0x5606f2538930 2024-09-21T03:42:34.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.402+0000 7f7382f20f40 1 -- 172.21.15.38:0/3284913117 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:16 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x5606f2545b50 con 0x5606f2538930 2024-09-21T03:42:34.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.403+0000 7f737c8c1640 1 -- 172.21.15.38:0/3284913117 <== osd.7 v2:172.21.15.195:6818/1606106342 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v118'675 uv675 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f7374011520 con 0x5606f2538930 2024-09-21T03:42:34.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.403+0000 7f7382f20f40 1 -- 172.21.15.38:0/3284913117 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:17 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x5606f2545b50 con 0x5606f2538930 2024-09-21T03:42:34.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.405+0000 7f737c8c1640 1 -- 172.21.15.38:0/3284913117 <== osd.7 v2:172.21.15.195:6818/1606106342 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v118'676 uv676 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f7374011520 con 0x5606f2538930 2024-09-21T03:42:34.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.405+0000 7f7382f20f40 1 -- 172.21.15.38:0/3284913117 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:18 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x5606f2545b50 con 0x5606f2538930 2024-09-21T03:42:34.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.407+0000 7f737c8c1640 1 -- 172.21.15.38:0/3284913117 <== osd.7 v2:172.21.15.195:6818/1606106342 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v118'677 uv677 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f7374011520 con 0x5606f2538930 2024-09-21T03:42:34.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.407+0000 7f7382f20f40 1 -- 172.21.15.38:0/3284913117 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:19 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=585b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x5606f2545fd0 con 0x5606f2538930 2024-09-21T03:42:34.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.409+0000 7f737c8c1640 1 -- 172.21.15.38:0/3284913117 <== osd.7 v2:172.21.15.195:6818/1606106342 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v118'678 uv678 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f7374011520 con 0x5606f2538930 2024-09-21T03:42:34.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.409+0000 7f7382f20f40 1 -- 172.21.15.38:0/3284913117 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:20 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x5606f2545e00 con 0x5606f2538930 2024-09-21T03:42:34.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.410+0000 7f737c8c1640 1 -- 172.21.15.38:0/3284913117 <== osd.7 v2:172.21.15.195:6818/1606106342 20 ==== osd_op_reply(20 600.00000000 [create] v118'679 uv653 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7374011520 con 0x5606f2538930 2024-09-21T03:42:34.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.410+0000 7f7382f20f40 1 -- 172.21.15.38:0/3284913117 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:21 13.10 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e118) -- 0x5606f2545e00 con 0x5606f2538930 2024-09-21T03:42:34.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.412+0000 7f737c8c1640 1 -- 172.21.15.38:0/3284913117 <== osd.7 v2:172.21.15.195:6818/1606106342 21 ==== osd_op_reply(21 600.00000000 [delete] v118'680 uv680 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7374011520 con 0x5606f2538930 2024-09-21T03:42:34.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.412+0000 7f7382f20f40 1 -- 172.21.15.38:0/3284913117 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:22 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x5606f2545e00 con 0x5606f2538930 2024-09-21T03:42:34.415 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-09-21T03:42:34.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.413+0000 7f737c8c1640 1 -- 172.21.15.38:0/3284913117 <== osd.7 v2:172.21.15.195:6818/1606106342 22 ==== osd_op_reply(22 600.00000000 [create] v118'681 uv681 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7374011520 con 0x5606f2538930 2024-09-21T03:42:34.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.413+0000 7f7382f20f40 1 -- 172.21.15.38:0/3284913117 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:23 13.10 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x5606f2545e00 con 0x5606f2538930 2024-09-21T03:42:34.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.415+0000 7f737c8c1640 1 -- 172.21.15.38:0/3284913117 <== osd.7 v2:172.21.15.195:6818/1606106342 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v118'682 uv682 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7374011520 con 0x5606f2538930 2024-09-21T03:42:34.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.415+0000 7f7382f20f40 1 -- 172.21.15.38:0/3284913117 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:24 13.10 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x5606f2545e00 con 0x5606f2538930 2024-09-21T03:42:34.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.416+0000 7f737c8c1640 1 -- 172.21.15.38:0/3284913117 <== osd.7 v2:172.21.15.195:6818/1606106342 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v118'683 uv683 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7374011520 con 0x5606f2538930 2024-09-21T03:42:34.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.416+0000 7f7382f20f40 1 -- 172.21.15.38:0/3284913117 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:25 13.10 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x5606f2545e00 con 0x5606f2538930 2024-09-21T03:42:34.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.418+0000 7f737c8c1640 1 -- 172.21.15.38:0/3284913117 <== osd.7 v2:172.21.15.195:6818/1606106342 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v118'684 uv684 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7374011520 con 0x5606f2538930 2024-09-21T03:42:34.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.418+0000 7f7382f20f40 1 -- 172.21.15.38:0/3284913117 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:26 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x5606f2545e00 con 0x5606f2538930 2024-09-21T03:42:34.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.419+0000 7f737c8c1640 1 -- 172.21.15.38:0/3284913117 <== osd.7 v2:172.21.15.195:6818/1606106342 26 ==== osd_op_reply(26 603.00000000 [create] v118'685 uv659 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7374011520 con 0x5606f2538930 2024-09-21T03:42:34.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.419+0000 7f7382f20f40 1 -- 172.21.15.38:0/3284913117 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:27 13.10 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e118) -- 0x5606f2545e00 con 0x5606f2538930 2024-09-21T03:42:34.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.421+0000 7f737c8c1640 1 -- 172.21.15.38:0/3284913117 <== osd.7 v2:172.21.15.195:6818/1606106342 27 ==== osd_op_reply(27 603.00000000 [delete] v118'686 uv686 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7374011520 con 0x5606f2538930 2024-09-21T03:42:34.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.421+0000 7f7382f20f40 1 -- 172.21.15.38:0/3284913117 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:28 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x5606f2545e00 con 0x5606f2538930 2024-09-21T03:42:34.424 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-09-21T03:42:34.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.422+0000 7f737c8c1640 1 -- 172.21.15.38:0/3284913117 <== osd.7 v2:172.21.15.195:6818/1606106342 28 ==== osd_op_reply(28 603.00000000 [create] v118'687 uv687 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7374011520 con 0x5606f2538930 2024-09-21T03:42:34.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.422+0000 7f7382f20f40 1 -- 172.21.15.38:0/3284913117 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:29 13.10 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x5606f2545e00 con 0x5606f2538930 2024-09-21T03:42:34.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.424+0000 7f737c8c1640 1 -- 172.21.15.38:0/3284913117 <== osd.7 v2:172.21.15.195:6818/1606106342 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v118'688 uv688 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7374011520 con 0x5606f2538930 2024-09-21T03:42:34.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.424+0000 7f7382f20f40 1 -- 172.21.15.38:0/3284913117 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:30 13.10 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x5606f2545e00 con 0x5606f2538930 2024-09-21T03:42:34.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.425+0000 7f737c8c1640 1 -- 172.21.15.38:0/3284913117 <== osd.7 v2:172.21.15.195:6818/1606106342 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v118'689 uv689 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7374011520 con 0x5606f2538930 2024-09-21T03:42:34.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.425+0000 7f7382f20f40 1 -- 172.21.15.38:0/3284913117 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:31 13.10 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x5606f2545e00 con 0x5606f2538930 2024-09-21T03:42:34.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.427+0000 7f737c8c1640 1 -- 172.21.15.38:0/3284913117 <== osd.7 v2:172.21.15.195:6818/1606106342 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v118'690 uv690 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7374011520 con 0x5606f2538930 2024-09-21T03:42:34.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.427+0000 7f7382f20f40 1 --2- 172.21.15.38:0/3284913117 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x5606f2547020 0x5606f2549440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:34.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.427+0000 7f7382f20f40 1 -- 172.21.15.38:0/3284913117 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x5606f2545e00 con 0x5606f2547020 2024-09-21T03:42:34.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.427+0000 7f737c0c0640 1 --2- 172.21.15.38:0/3284913117 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x5606f2547020 0x5606f2549440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:34.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.427+0000 7f737c0c0640 1 --2- 172.21.15.38:0/3284913117 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x5606f2547020 0x5606f2549440 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:34.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.430+0000 7f737c0c0640 1 -- 172.21.15.38:0/3284913117 <== osd.6 v2:172.21.15.195:6826/4055825675 1 ==== osd_op_reply(32 1.00000000.inode [create] v118'483 uv466 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f736c00b040 con 0x5606f2547020 2024-09-21T03:42:34.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.430+0000 7f7382f20f40 1 -- 172.21.15.38:0/3284913117 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e118) -- 0x5606f2549980 con 0x5606f2547020 2024-09-21T03:42:34.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.431+0000 7f737c0c0640 1 -- 172.21.15.38:0/3284913117 <== osd.6 v2:172.21.15.195:6826/4055825675 2 ==== osd_op_reply(33 1.00000000.inode [delete] v118'484 uv484 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f736c00b040 con 0x5606f2547020 2024-09-21T03:42:34.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.431+0000 7f7382f20f40 1 -- 172.21.15.38:0/3284913117 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x5606f2549d20 con 0x5606f2547020 2024-09-21T03:42:34.434 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-09-21T03:42:34.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.432+0000 7f737c0c0640 1 -- 172.21.15.38:0/3284913117 <== osd.6 v2:172.21.15.195:6826/4055825675 3 ==== osd_op_reply(34 1.00000000.inode [create] v118'485 uv485 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f736c00b040 con 0x5606f2547020 2024-09-21T03:42:34.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.432+0000 7f7382f20f40 1 -- 172.21.15.38:0/3284913117 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x5606f254a570 con 0x5606f2547020 2024-09-21T03:42:34.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.434+0000 7f737c0c0640 1 -- 172.21.15.38:0/3284913117 <== osd.6 v2:172.21.15.195:6826/4055825675 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v118'486 uv486 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f736c00b040 con 0x5606f2547020 2024-09-21T03:42:34.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.434+0000 7f7382f20f40 1 -- 172.21.15.38:0/3284913117 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x5606f254a910 con 0x5606f2547020 2024-09-21T03:42:34.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.435+0000 7f737c0c0640 1 -- 172.21.15.38:0/3284913117 <== osd.6 v2:172.21.15.195:6826/4055825675 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v118'487 uv487 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f736c00b040 con 0x5606f2547020 2024-09-21T03:42:34.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.435+0000 7f7382f20f40 1 -- 172.21.15.38:0/3284913117 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x5606f254acb0 con 0x5606f2547020 2024-09-21T03:42:34.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.437+0000 7f737c0c0640 1 -- 172.21.15.38:0/3284913117 <== osd.6 v2:172.21.15.195:6826/4055825675 6 ==== osd_op_reply(37 200.00000000 [create] v118'488 uv471 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f736c00b040 con 0x5606f2547020 2024-09-21T03:42:34.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.437+0000 7f7382f20f40 1 -- 172.21.15.38:0/3284913117 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e118) -- 0x5606f254b050 con 0x5606f2547020 2024-09-21T03:42:34.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.438+0000 7f737c0c0640 1 -- 172.21.15.38:0/3284913117 <== osd.6 v2:172.21.15.195:6826/4055825675 7 ==== osd_op_reply(38 200.00000000 [delete] v118'489 uv489 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f736c00b040 con 0x5606f2547020 2024-09-21T03:42:34.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.438+0000 7f7382f20f40 1 -- 172.21.15.38:0/3284913117 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x5606f254b050 con 0x5606f2547020 2024-09-21T03:42:34.441 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-09-21T03:42:34.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.439+0000 7f737c0c0640 1 -- 172.21.15.38:0/3284913117 <== osd.6 v2:172.21.15.195:6826/4055825675 8 ==== osd_op_reply(39 200.00000000 [create] v118'490 uv490 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f736c00b040 con 0x5606f2547020 2024-09-21T03:42:34.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.439+0000 7f7382f20f40 1 -- 172.21.15.38:0/3284913117 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x5606f254b050 con 0x5606f2547020 2024-09-21T03:42:34.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.441+0000 7f737c0c0640 1 -- 172.21.15.38:0/3284913117 <== osd.6 v2:172.21.15.195:6826/4055825675 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v118'491 uv491 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f736c00b040 con 0x5606f2547020 2024-09-21T03:42:34.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.441+0000 7f7382f20f40 1 -- 172.21.15.38:0/3284913117 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x5606f254b050 con 0x5606f2547020 2024-09-21T03:42:34.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.442+0000 7f737c0c0640 1 -- 172.21.15.38:0/3284913117 <== osd.6 v2:172.21.15.195:6826/4055825675 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v118'492 uv492 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f736c00b040 con 0x5606f2547020 2024-09-21T03:42:34.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.442+0000 7f7382f20f40 1 -- 172.21.15.38:0/3284913117 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:42 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x5606f254b050 con 0x5606f2547020 2024-09-21T03:42:34.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.443+0000 7f737c0c0640 1 -- 172.21.15.38:0/3284913117 <== osd.6 v2:172.21.15.195:6826/4055825675 11 ==== osd_op_reply(42 200.00000001 [create] v118'493 uv476 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f736c00b040 con 0x5606f2547020 2024-09-21T03:42:34.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.444+0000 7f7382f20f40 1 -- 172.21.15.38:0/3284913117 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:43 13.14 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e118) -- 0x5606f254b050 con 0x5606f2547020 2024-09-21T03:42:34.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.445+0000 7f737c0c0640 1 -- 172.21.15.38:0/3284913117 <== osd.6 v2:172.21.15.195:6826/4055825675 12 ==== osd_op_reply(43 200.00000001 [delete] v118'494 uv494 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f736c00b040 con 0x5606f2547020 2024-09-21T03:42:34.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.445+0000 7f7382f20f40 1 -- 172.21.15.38:0/3284913117 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:44 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x5606f254b050 con 0x5606f2547020 2024-09-21T03:42:34.448 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-09-21T03:42:34.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.446+0000 7f737c0c0640 1 -- 172.21.15.38:0/3284913117 <== osd.6 v2:172.21.15.195:6826/4055825675 13 ==== osd_op_reply(44 200.00000001 [create] v118'495 uv495 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f736c00b040 con 0x5606f2547020 2024-09-21T03:42:34.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.446+0000 7f7382f20f40 1 -- 172.21.15.38:0/3284913117 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:45 13.14 13:2e2fa760:::200.00000001:head [write 0~17490 in=17490b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x5606f254f4d0 con 0x5606f2547020 2024-09-21T03:42:34.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.448+0000 7f737c0c0640 1 -- 172.21.15.38:0/3284913117 <== osd.6 v2:172.21.15.195:6826/4055825675 14 ==== osd_op_reply(45 200.00000001 [write 0~17490] v118'496 uv496 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f736c00b040 con 0x5606f2547020 2024-09-21T03:42:34.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.448+0000 7f7382f20f40 1 -- 172.21.15.38:0/3284913117 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:46 13.14 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x5606f254f4d0 con 0x5606f2547020 2024-09-21T03:42:34.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.450+0000 7f737c0c0640 1 -- 172.21.15.38:0/3284913117 <== osd.6 v2:172.21.15.195:6826/4055825675 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v118'497 uv497 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f736c00b040 con 0x5606f2547020 2024-09-21T03:42:34.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.450+0000 7f7382f20f40 1 -- 172.21.15.38:0/3284913117 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:47 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x5606f254b050 con 0x5606f2547020 2024-09-21T03:42:34.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.451+0000 7f737c0c0640 1 -- 172.21.15.38:0/3284913117 <== osd.6 v2:172.21.15.195:6826/4055825675 16 ==== osd_op_reply(47 606.00000000 [create] v118'498 uv482 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f736c00b040 con 0x5606f2547020 2024-09-21T03:42:34.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.451+0000 7f7382f20f40 1 -- 172.21.15.38:0/3284913117 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:48 13.14 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e118) -- 0x5606f254b050 con 0x5606f2547020 2024-09-21T03:42:34.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.452+0000 7f737c0c0640 1 -- 172.21.15.38:0/3284913117 <== osd.6 v2:172.21.15.195:6826/4055825675 17 ==== osd_op_reply(48 606.00000000 [delete] v118'499 uv499 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f736c00b040 con 0x5606f2547020 2024-09-21T03:42:34.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.452+0000 7f7382f20f40 1 -- 172.21.15.38:0/3284913117 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:49 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x5606f254b050 con 0x5606f2547020 2024-09-21T03:42:34.455 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-09-21T03:42:34.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.454+0000 7f737c0c0640 1 -- 172.21.15.38:0/3284913117 <== osd.6 v2:172.21.15.195:6826/4055825675 18 ==== osd_op_reply(49 606.00000000 [create] v118'500 uv500 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f736c00b040 con 0x5606f2547020 2024-09-21T03:42:34.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.454+0000 7f7382f20f40 1 -- 172.21.15.38:0/3284913117 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:50 13.14 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x5606f254c060 con 0x5606f2547020 2024-09-21T03:42:34.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.455+0000 7f737c0c0640 1 -- 172.21.15.38:0/3284913117 <== osd.6 v2:172.21.15.195:6826/4055825675 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v118'501 uv501 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f736c00b040 con 0x5606f2547020 2024-09-21T03:42:34.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.455+0000 7f7382f20f40 1 -- 172.21.15.38:0/3284913117 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:51 13.14 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x5606f254c060 con 0x5606f2547020 2024-09-21T03:42:34.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.456+0000 7f737c0c0640 1 -- 172.21.15.38:0/3284913117 <== osd.6 v2:172.21.15.195:6826/4055825675 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v118'502 uv502 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f736c00b040 con 0x5606f2547020 2024-09-21T03:42:34.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.456+0000 7f7382f20f40 1 -- 172.21.15.38:0/3284913117 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:52 13.14 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x5606f254b050 con 0x5606f2547020 2024-09-21T03:42:34.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.457+0000 7f737c0c0640 1 -- 172.21.15.38:0/3284913117 <== osd.6 v2:172.21.15.195:6826/4055825675 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v118'503 uv503 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f736c00b040 con 0x5606f2547020 2024-09-21T03:42:34.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.457+0000 7f7382f20f40 1 --2- 172.21.15.38:0/3284913117 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x5606f254b560 0x5606f254d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:34.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.457+0000 7f7382f20f40 1 -- 172.21.15.38:0/3284913117 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:53 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x5606f254df90 con 0x5606f254b560 2024-09-21T03:42:34.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.457+0000 7f737b8bf640 1 --2- 172.21.15.38:0/3284913117 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x5606f254b560 0x5606f254d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:34.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.458+0000 7f737b8bf640 1 --2- 172.21.15.38:0/3284913117 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x5606f254b560 0x5606f254d980 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:34.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.459+0000 7f737b8bf640 1 -- 172.21.15.38:0/3284913117 <== osd.0 v2:172.21.15.38:6800/2381639433 1 ==== osd_op_reply(53 607.00000000 [create] v118'129 uv128 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f737001b070 con 0x5606f254b560 2024-09-21T03:42:34.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.459+0000 7f7382f20f40 1 -- 172.21.15.38:0/3284913117 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:54 13.1c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e118) -- 0x5606f254e480 con 0x5606f254b560 2024-09-21T03:42:34.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.460+0000 7f737b8bf640 1 -- 172.21.15.38:0/3284913117 <== osd.0 v2:172.21.15.38:6800/2381639433 2 ==== osd_op_reply(54 607.00000000 [delete] v118'130 uv130 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f737001b070 con 0x5606f254b560 2024-09-21T03:42:34.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.460+0000 7f7382f20f40 1 -- 172.21.15.38:0/3284913117 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:55 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x5606f254e960 con 0x5606f254b560 2024-09-21T03:42:34.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.461+0000 7f737b8bf640 1 -- 172.21.15.38:0/3284913117 <== osd.0 v2:172.21.15.38:6800/2381639433 3 ==== osd_op_reply(55 607.00000000 [create] v118'131 uv131 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f737001b070 con 0x5606f254b560 2024-09-21T03:42:34.463 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-09-21T03:42:34.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.461+0000 7f7382f20f40 1 -- 172.21.15.38:0/3284913117 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:56 13.1c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x5606f2550210 con 0x5606f254b560 2024-09-21T03:42:34.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.462+0000 7f737b8bf640 1 -- 172.21.15.38:0/3284913117 <== osd.0 v2:172.21.15.38:6800/2381639433 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v118'132 uv132 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f737001b070 con 0x5606f254b560 2024-09-21T03:42:34.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.463+0000 7f7382f20f40 1 -- 172.21.15.38:0/3284913117 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:57 13.1c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x5606f25505b0 con 0x5606f254b560 2024-09-21T03:42:34.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.464+0000 7f737b8bf640 1 -- 172.21.15.38:0/3284913117 <== osd.0 v2:172.21.15.38:6800/2381639433 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v118'133 uv133 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f737001b070 con 0x5606f254b560 2024-09-21T03:42:34.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.464+0000 7f7382f20f40 1 -- 172.21.15.38:0/3284913117 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:58 13.1c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x5606f254f200 con 0x5606f254b560 2024-09-21T03:42:34.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.465+0000 7f737b8bf640 1 -- 172.21.15.38:0/3284913117 <== osd.0 v2:172.21.15.38:6800/2381639433 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v118'134 uv134 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f737001b070 con 0x5606f254b560 2024-09-21T03:42:34.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.465+0000 7f7382f20f40 1 --2- 172.21.15.38:0/3284913117 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x5606f254fb20 0x5606f2552960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:34.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.465+0000 7f7382f20f40 1 -- 172.21.15.38:0/3284913117 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:59 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x5606f2552ea0 con 0x5606f254fb20 2024-09-21T03:42:34.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.465+0000 7f737c8c1640 1 --2- 172.21.15.38:0/3284913117 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x5606f254fb20 0x5606f2552960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:34.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.465+0000 7f737c8c1640 1 --2- 172.21.15.38:0/3284913117 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x5606f254fb20 0x5606f2552960 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:34.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.466+0000 7f737c8c1640 1 -- 172.21.15.38:0/3284913117 <== osd.3 v2:172.21.15.38:6808/1286810294 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v118'94 uv93 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f7374011520 con 0x5606f254fb20 2024-09-21T03:42:34.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.466+0000 7f7382f20f40 1 -- 172.21.15.38:0/3284913117 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:60 13.6 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e118) -- 0x5606f2552ea0 con 0x5606f254fb20 2024-09-21T03:42:34.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.467+0000 7f737c8c1640 1 -- 172.21.15.38:0/3284913117 <== osd.3 v2:172.21.15.38:6808/1286810294 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v118'95 uv95 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f7374011520 con 0x5606f254fb20 2024-09-21T03:42:34.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.467+0000 7f7382f20f40 1 -- 172.21.15.38:0/3284913117 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:61 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x5606f2552f20 con 0x5606f254fb20 2024-09-21T03:42:34.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.468+0000 7f737c8c1640 1 -- 172.21.15.38:0/3284913117 <== osd.3 v2:172.21.15.38:6808/1286810294 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v118'96 uv96 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f7374011520 con 0x5606f254fb20 2024-09-21T03:42:34.470 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-09-21T03:42:34.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.468+0000 7f7382f20f40 1 -- 172.21.15.38:0/3284913117 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:62 13.6 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x5606f2553330 con 0x5606f254fb20 2024-09-21T03:42:34.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.469+0000 7f737c8c1640 1 -- 172.21.15.38:0/3284913117 <== osd.3 v2:172.21.15.38:6808/1286810294 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v118'97 uv97 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f7374011520 con 0x5606f254fb20 2024-09-21T03:42:34.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.469+0000 7f7382f20f40 1 -- 172.21.15.38:0/3284913117 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x5606f2553330 con 0x5606f254b560 2024-09-21T03:42:34.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.470+0000 7f737b8bf640 1 -- 172.21.15.38:0/3284913117 <== osd.0 v2:172.21.15.38:6800/2381639433 7 ==== osd_op_reply(63 608.00000000 [create] v118'129 uv128 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f737001b070 con 0x5606f254b560 2024-09-21T03:42:34.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.470+0000 7f7382f20f40 1 -- 172.21.15.38:0/3284913117 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e118) -- 0x5606f25536d0 con 0x5606f254b560 2024-09-21T03:42:34.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.472+0000 7f737b8bf640 1 -- 172.21.15.38:0/3284913117 <== osd.0 v2:172.21.15.38:6800/2381639433 8 ==== osd_op_reply(64 608.00000000 [delete] v118'130 uv130 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f737001b070 con 0x5606f254b560 2024-09-21T03:42:34.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.472+0000 7f7382f20f40 1 -- 172.21.15.38:0/3284913117 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x5606f25536d0 con 0x5606f254b560 2024-09-21T03:42:34.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.473+0000 7f737b8bf640 1 -- 172.21.15.38:0/3284913117 <== osd.0 v2:172.21.15.38:6800/2381639433 9 ==== osd_op_reply(65 608.00000000 [create] v118'131 uv131 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f737001b070 con 0x5606f254b560 2024-09-21T03:42:34.475 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-09-21T03:42:34.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.473+0000 7f7382f20f40 1 -- 172.21.15.38:0/3284913117 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x5606f25546e0 con 0x5606f254b560 2024-09-21T03:42:34.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.474+0000 7f737b8bf640 1 -- 172.21.15.38:0/3284913117 <== osd.0 v2:172.21.15.38:6800/2381639433 10 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v118'132 uv132 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f737001b070 con 0x5606f254b560 2024-09-21T03:42:34.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.474+0000 7f7382f20f40 1 -- 172.21.15.38:0/3284913117 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x5606f25546e0 con 0x5606f254b560 2024-09-21T03:42:34.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.475+0000 7f737b8bf640 1 -- 172.21.15.38:0/3284913117 <== osd.0 v2:172.21.15.38:6800/2381639433 11 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v118'133 uv133 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f737001b070 con 0x5606f254b560 2024-09-21T03:42:34.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.475+0000 7f7382f20f40 1 -- 172.21.15.38:0/3284913117 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x5606f25536d0 con 0x5606f254b560 2024-09-21T03:42:34.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.476+0000 7f737b8bf640 1 -- 172.21.15.38:0/3284913117 <== osd.0 v2:172.21.15.38:6800/2381639433 12 ==== osd_op_reply(68 608.00000000 [omap-set-header] v118'134 uv134 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f737001b070 con 0x5606f254b560 2024-09-21T03:42:34.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.476+0000 7f7382f20f40 1 -- 172.21.15.38:0/3284913117 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x5606f25536d0 con 0x5606f254fb20 2024-09-21T03:42:34.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.477+0000 7f737c8c1640 1 -- 172.21.15.38:0/3284913117 <== osd.3 v2:172.21.15.38:6808/1286810294 5 ==== osd_op_reply(69 604.00000000 [create] v118'238 uv232 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7374011520 con 0x5606f254fb20 2024-09-21T03:42:34.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.477+0000 7f7382f20f40 1 -- 172.21.15.38:0/3284913117 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e118) -- 0x5606f25536d0 con 0x5606f254fb20 2024-09-21T03:42:34.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.478+0000 7f737c8c1640 1 -- 172.21.15.38:0/3284913117 <== osd.3 v2:172.21.15.38:6808/1286810294 6 ==== osd_op_reply(70 604.00000000 [delete] v118'239 uv239 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7374011520 con 0x5606f254fb20 2024-09-21T03:42:34.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.478+0000 7f7382f20f40 1 -- 172.21.15.38:0/3284913117 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x5606f25536d0 con 0x5606f254fb20 2024-09-21T03:42:34.481 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-09-21T03:42:34.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.479+0000 7f737c8c1640 1 -- 172.21.15.38:0/3284913117 <== osd.3 v2:172.21.15.38:6808/1286810294 7 ==== osd_op_reply(71 604.00000000 [create] v118'240 uv240 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7374011520 con 0x5606f254fb20 2024-09-21T03:42:34.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.479+0000 7f7382f20f40 1 -- 172.21.15.38:0/3284913117 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x5606f25546e0 con 0x5606f254fb20 2024-09-21T03:42:34.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.481+0000 7f737c8c1640 1 -- 172.21.15.38:0/3284913117 <== osd.3 v2:172.21.15.38:6808/1286810294 8 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v118'241 uv241 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7374011520 con 0x5606f254fb20 2024-09-21T03:42:34.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.481+0000 7f7382f20f40 1 -- 172.21.15.38:0/3284913117 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x5606f25546e0 con 0x5606f254fb20 2024-09-21T03:42:34.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.482+0000 7f737c8c1640 1 -- 172.21.15.38:0/3284913117 <== osd.3 v2:172.21.15.38:6808/1286810294 9 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v118'242 uv242 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7374011520 con 0x5606f254fb20 2024-09-21T03:42:34.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.482+0000 7f7382f20f40 1 -- 172.21.15.38:0/3284913117 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x5606f25536d0 con 0x5606f254fb20 2024-09-21T03:42:34.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.484+0000 7f737c8c1640 1 -- 172.21.15.38:0/3284913117 <== osd.3 v2:172.21.15.38:6808/1286810294 10 ==== osd_op_reply(74 604.00000000 [omap-set-header] v118'243 uv243 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7374011520 con 0x5606f254fb20 2024-09-21T03:42:34.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.484+0000 7f7382f20f40 1 -- 172.21.15.38:0/3284913117 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x5606f25536d0 con 0x5606f254fb20 2024-09-21T03:42:34.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.485+0000 7f737c8c1640 1 -- 172.21.15.38:0/3284913117 <== osd.3 v2:172.21.15.38:6808/1286810294 11 ==== osd_op_reply(75 500.00000000 [create] v118'244 uv237 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7374011520 con 0x5606f254fb20 2024-09-21T03:42:34.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.485+0000 7f7382f20f40 1 -- 172.21.15.38:0/3284913117 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e118) -- 0x5606f25536d0 con 0x5606f254fb20 2024-09-21T03:42:34.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.486+0000 7f737c8c1640 1 -- 172.21.15.38:0/3284913117 <== osd.3 v2:172.21.15.38:6808/1286810294 12 ==== osd_op_reply(76 500.00000000 [delete] v118'245 uv245 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7374011520 con 0x5606f254fb20 2024-09-21T03:42:34.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.486+0000 7f7382f20f40 1 -- 172.21.15.38:0/3284913117 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x5606f25536d0 con 0x5606f254fb20 2024-09-21T03:42:34.489 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-09-21T03:42:34.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.487+0000 7f737c8c1640 1 -- 172.21.15.38:0/3284913117 <== osd.3 v2:172.21.15.38:6808/1286810294 13 ==== osd_op_reply(77 500.00000000 [create] v118'246 uv246 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7374011520 con 0x5606f254fb20 2024-09-21T03:42:34.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.487+0000 7f7382f20f40 1 -- 172.21.15.38:0/3284913117 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x5606f25536d0 con 0x5606f254fb20 2024-09-21T03:42:34.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.489+0000 7f737c8c1640 1 -- 172.21.15.38:0/3284913117 <== osd.3 v2:172.21.15.38:6808/1286810294 14 ==== osd_op_reply(78 500.00000000 [write 0~90] v118'247 uv247 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7374011520 con 0x5606f254fb20 2024-09-21T03:42:34.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.489+0000 7f7382f20f40 1 -- 172.21.15.38:0/3284913117 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x5606f25536d0 con 0x5606f254fb20 2024-09-21T03:42:34.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.490+0000 7f737c8c1640 1 -- 172.21.15.38:0/3284913117 <== osd.3 v2:172.21.15.38:6808/1286810294 15 ==== osd_op_reply(79 500.00000000 [omap-set-header] v118'248 uv248 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7374011520 con 0x5606f254fb20 2024-09-21T03:42:34.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.490+0000 7f7382f20f40 1 -- 172.21.15.38:0/3284913117 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:80 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x5606f25536d0 con 0x5606f254b560 2024-09-21T03:42:34.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.491+0000 7f737b8bf640 1 -- 172.21.15.38:0/3284913117 <== osd.0 v2:172.21.15.38:6800/2381639433 13 ==== osd_op_reply(80 mds_snaptable [create] v118'107 uv106 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f737001b070 con 0x5606f254b560 2024-09-21T03:42:34.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.491+0000 7f7382f20f40 1 -- 172.21.15.38:0/3284913117 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:81 13.d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e118) -- 0x5606f25536d0 con 0x5606f254b560 2024-09-21T03:42:34.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.492+0000 7f737b8bf640 1 -- 172.21.15.38:0/3284913117 <== osd.0 v2:172.21.15.38:6800/2381639433 14 ==== osd_op_reply(81 mds_snaptable [delete] v118'108 uv108 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f737001b070 con 0x5606f254b560 2024-09-21T03:42:34.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.492+0000 7f7382f20f40 1 -- 172.21.15.38:0/3284913117 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:82 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x5606f25536d0 con 0x5606f254b560 2024-09-21T03:42:34.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.493+0000 7f737b8bf640 1 -- 172.21.15.38:0/3284913117 <== osd.0 v2:172.21.15.38:6800/2381639433 15 ==== osd_op_reply(82 mds_snaptable [create] v118'109 uv109 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f737001b070 con 0x5606f254b560 2024-09-21T03:42:34.495 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-09-21T03:42:34.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.493+0000 7f7382f20f40 1 -- 172.21.15.38:0/3284913117 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:83 13.d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x5606f25536d0 con 0x5606f254b560 2024-09-21T03:42:34.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.494+0000 7f737b8bf640 1 -- 172.21.15.38:0/3284913117 <== osd.0 v2:172.21.15.38:6800/2381639433 16 ==== osd_op_reply(83 mds_snaptable [write 0~70] v118'110 uv110 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f737001b070 con 0x5606f254b560 2024-09-21T03:42:34.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.494+0000 7f7382f20f40 1 -- 172.21.15.38:0/3284913117 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:84 13.d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x5606f25536d0 con 0x5606f254b560 2024-09-21T03:42:34.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.495+0000 7f737b8bf640 1 -- 172.21.15.38:0/3284913117 <== osd.0 v2:172.21.15.38:6800/2381639433 17 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v118'111 uv111 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f737001b070 con 0x5606f254b560 2024-09-21T03:42:34.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.495+0000 7f7382f20f40 1 --2- 172.21.15.38:0/3284913117 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x5606f2553d50 0x5606f2556170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:34.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.495+0000 7f7382f20f40 1 -- 172.21.15.38:0/3284913117 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x5606f25567c0 con 0x5606f2553d50 2024-09-21T03:42:34.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.495+0000 7f737c0c0640 1 --2- 172.21.15.38:0/3284913117 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x5606f2553d50 0x5606f2556170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:34.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.496+0000 7f737c0c0640 1 --2- 172.21.15.38:0/3284913117 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x5606f2553d50 0x5606f2556170 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:34.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.497+0000 7f737c0c0640 1 -- 172.21.15.38:0/3284913117 <== osd.1 v2:172.21.15.38:6816/1491818732 1 ==== osd_op_reply(85 605.00000000 [create] v118'129 uv128 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f736c00b040 con 0x5606f2553d50 2024-09-21T03:42:34.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.497+0000 7f7382f20f40 1 -- 172.21.15.38:0/3284913117 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e118) -- 0x5606f2556b90 con 0x5606f2553d50 2024-09-21T03:42:34.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.498+0000 7f737c0c0640 1 -- 172.21.15.38:0/3284913117 <== osd.1 v2:172.21.15.38:6816/1491818732 2 ==== osd_op_reply(86 605.00000000 [delete] v118'130 uv130 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f736c00b040 con 0x5606f2553d50 2024-09-21T03:42:34.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.499+0000 7f7382f20f40 1 -- 172.21.15.38:0/3284913117 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x5606f2556b90 con 0x5606f2553d50 2024-09-21T03:42:34.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.500+0000 7f737c0c0640 1 -- 172.21.15.38:0/3284913117 <== osd.1 v2:172.21.15.38:6816/1491818732 3 ==== osd_op_reply(87 605.00000000 [create] v118'131 uv131 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f736c00b040 con 0x5606f2553d50 2024-09-21T03:42:34.502 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-09-21T03:42:34.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.500+0000 7f7382f20f40 1 -- 172.21.15.38:0/3284913117 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x5606f2557ba0 con 0x5606f2553d50 2024-09-21T03:42:34.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.501+0000 7f737c0c0640 1 -- 172.21.15.38:0/3284913117 <== osd.1 v2:172.21.15.38:6816/1491818732 4 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v118'132 uv132 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f736c00b040 con 0x5606f2553d50 2024-09-21T03:42:34.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.501+0000 7f7382f20f40 1 -- 172.21.15.38:0/3284913117 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x5606f2557ba0 con 0x5606f2553d50 2024-09-21T03:42:34.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.502+0000 7f737c0c0640 1 -- 172.21.15.38:0/3284913117 <== osd.1 v2:172.21.15.38:6816/1491818732 5 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v118'133 uv133 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f736c00b040 con 0x5606f2553d50 2024-09-21T03:42:34.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.502+0000 7f7382f20f40 1 -- 172.21.15.38:0/3284913117 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x5606f2556b90 con 0x5606f2553d50 2024-09-21T03:42:34.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.503+0000 7f737c0c0640 1 -- 172.21.15.38:0/3284913117 <== osd.1 v2:172.21.15.38:6816/1491818732 6 ==== osd_op_reply(90 605.00000000 [omap-set-header] v118'134 uv134 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f736c00b040 con 0x5606f2553d50 2024-09-21T03:42:34.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.503+0000 7f7382f20f40 1 -- 172.21.15.38:0/3284913117 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x5606f2556b90 con 0x5606f2553d50 2024-09-21T03:42:34.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.504+0000 7f737c0c0640 1 -- 172.21.15.38:0/3284913117 <== osd.1 v2:172.21.15.38:6816/1491818732 7 ==== osd_op_reply(91 mds0_inotable [create] v118'213 uv207 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f736c00b040 con 0x5606f2553d50 2024-09-21T03:42:34.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.504+0000 7f7382f20f40 1 -- 172.21.15.38:0/3284913117 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e118) -- 0x5606f2556b90 con 0x5606f2553d50 2024-09-21T03:42:34.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.506+0000 7f737c0c0640 1 -- 172.21.15.38:0/3284913117 <== osd.1 v2:172.21.15.38:6816/1491818732 8 ==== osd_op_reply(92 mds0_inotable [delete] v118'214 uv214 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f736c00b040 con 0x5606f2553d50 2024-09-21T03:42:34.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.506+0000 7f7382f20f40 1 -- 172.21.15.38:0/3284913117 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x5606f2556b90 con 0x5606f2553d50 2024-09-21T03:42:34.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.507+0000 7f737c0c0640 1 -- 172.21.15.38:0/3284913117 <== osd.1 v2:172.21.15.38:6816/1491818732 9 ==== osd_op_reply(93 mds0_inotable [create] v118'215 uv215 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f736c00b040 con 0x5606f2553d50 2024-09-21T03:42:34.509 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-09-21T03:42:34.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.507+0000 7f7382f20f40 1 -- 172.21.15.38:0/3284913117 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:94 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x5606f254ee10 con 0x5606f2553d50 2024-09-21T03:42:34.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.508+0000 7f737c0c0640 1 -- 172.21.15.38:0/3284913117 <== osd.1 v2:172.21.15.38:6816/1491818732 10 ==== osd_op_reply(94 mds0_inotable [write 0~34] v118'216 uv216 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f736c00b040 con 0x5606f2553d50 2024-09-21T03:42:34.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.508+0000 7f7382f20f40 1 -- 172.21.15.38:0/3284913117 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:95 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x5606f254ee10 con 0x5606f2553d50 2024-09-21T03:42:34.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.509+0000 7f737c0c0640 1 -- 172.21.15.38:0/3284913117 <== osd.1 v2:172.21.15.38:6816/1491818732 11 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v118'217 uv217 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f736c00b040 con 0x5606f2553d50 2024-09-21T03:42:34.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.509+0000 7f7382f20f40 1 -- 172.21.15.38:0/3284913117 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:96 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x5606f254edf0 con 0x5606f2553d50 2024-09-21T03:42:34.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.510+0000 7f737c0c0640 1 -- 172.21.15.38:0/3284913117 <== osd.1 v2:172.21.15.38:6816/1491818732 12 ==== osd_op_reply(96 100.00000000 [create] v118'218 uv212 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f736c00b040 con 0x5606f2553d50 2024-09-21T03:42:34.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.510+0000 7f7382f20f40 1 -- 172.21.15.38:0/3284913117 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:97 13.13 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e118) -- 0x5606f254edf0 con 0x5606f2553d50 2024-09-21T03:42:34.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.511+0000 7f737c0c0640 1 -- 172.21.15.38:0/3284913117 <== osd.1 v2:172.21.15.38:6816/1491818732 13 ==== osd_op_reply(97 100.00000000 [delete] v118'219 uv219 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f736c00b040 con 0x5606f2553d50 2024-09-21T03:42:34.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.511+0000 7f7382f20f40 1 -- 172.21.15.38:0/3284913117 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:98 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x5606f254edf0 con 0x5606f2553d50 2024-09-21T03:42:34.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.513+0000 7f737c0c0640 1 -- 172.21.15.38:0/3284913117 <== osd.1 v2:172.21.15.38:6816/1491818732 14 ==== osd_op_reply(98 100.00000000 [create] v118'220 uv220 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f736c00b040 con 0x5606f2553d50 2024-09-21T03:42:34.515 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-09-21T03:42:34.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.513+0000 7f7382f20f40 1 -- 172.21.15.38:0/3284913117 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:99 13.13 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x5606f254edf0 con 0x5606f2553d50 2024-09-21T03:42:34.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.514+0000 7f737c0c0640 1 -- 172.21.15.38:0/3284913117 <== osd.1 v2:172.21.15.38:6816/1491818732 15 ==== osd_op_reply(99 100.00000000 [omap-set-header] v118'221 uv221 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f736c00b040 con 0x5606f2553d50 2024-09-21T03:42:34.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.514+0000 7f7382f20f40 1 -- 172.21.15.38:0/3284913117 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:100 13.13 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x5606f25599a0 con 0x5606f2553d50 2024-09-21T03:42:34.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.515+0000 7f737c0c0640 1 -- 172.21.15.38:0/3284913117 <== osd.1 v2:172.21.15.38:6816/1491818732 16 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v118'222 uv222 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f736c00b040 con 0x5606f2553d50 2024-09-21T03:42:34.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.516+0000 7f7382f20f40 1 -- 172.21.15.38:0/3284913117 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x5606f2557040 con 0x5606f254fb20 2024-09-21T03:42:34.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.517+0000 7f737c8c1640 1 -- 172.21.15.38:0/3284913117 <== osd.3 v2:172.21.15.38:6808/1286810294 16 ==== osd_op_reply(101 mds0_sessionmap [create] v118'95 uv94 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7f7374011520 con 0x5606f254fb20 2024-09-21T03:42:34.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.517+0000 7f7382f20f40 1 -- 172.21.15.38:0/3284913117 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e118) -- 0x5606f2557040 con 0x5606f254fb20 2024-09-21T03:42:34.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.518+0000 7f737c8c1640 1 -- 172.21.15.38:0/3284913117 <== osd.3 v2:172.21.15.38:6808/1286810294 17 ==== osd_op_reply(102 mds0_sessionmap [delete] v118'96 uv96 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f7374011520 con 0x5606f254fb20 2024-09-21T03:42:34.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.518+0000 7f7382f20f40 1 -- 172.21.15.38:0/3284913117 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:103 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x5606f2557040 con 0x5606f254fb20 2024-09-21T03:42:34.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.519+0000 7f737c8c1640 1 -- 172.21.15.38:0/3284913117 <== osd.3 v2:172.21.15.38:6808/1286810294 18 ==== osd_op_reply(103 mds0_sessionmap [create] v118'97 uv97 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f7374011520 con 0x5606f254fb20 2024-09-21T03:42:34.521 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-09-21T03:42:34.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.519+0000 7f7382f20f40 1 -- 172.21.15.38:0/3284913117 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:104 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x5606f2557040 con 0x5606f254fb20 2024-09-21T03:42:34.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.520+0000 7f737c8c1640 1 -- 172.21.15.38:0/3284913117 <== osd.3 v2:172.21.15.38:6808/1286810294 19 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v118'98 uv98 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f7374011520 con 0x5606f254fb20 2024-09-21T03:42:34.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.520+0000 7f7382f20f40 1 -- 172.21.15.38:0/3284913117 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x5606f2557040 con 0x5606f254fb20 2024-09-21T03:42:34.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.521+0000 7f737c8c1640 1 -- 172.21.15.38:0/3284913117 <== osd.3 v2:172.21.15.38:6808/1286810294 20 ==== osd_op_reply(105 609.00000000 [create] v118'129 uv128 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7374011520 con 0x5606f254fb20 2024-09-21T03:42:34.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.521+0000 7f7382f20f40 1 -- 172.21.15.38:0/3284913117 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e118) -- 0x5606f2557040 con 0x5606f254fb20 2024-09-21T03:42:34.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.522+0000 7f737c8c1640 1 -- 172.21.15.38:0/3284913117 <== osd.3 v2:172.21.15.38:6808/1286810294 21 ==== osd_op_reply(106 609.00000000 [delete] v118'130 uv130 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7374011520 con 0x5606f254fb20 2024-09-21T03:42:34.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.522+0000 7f7382f20f40 1 -- 172.21.15.38:0/3284913117 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x5606f2557040 con 0x5606f254fb20 2024-09-21T03:42:34.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.523+0000 7f737c8c1640 1 -- 172.21.15.38:0/3284913117 <== osd.3 v2:172.21.15.38:6808/1286810294 22 ==== osd_op_reply(107 609.00000000 [create] v118'131 uv131 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7374011520 con 0x5606f254fb20 2024-09-21T03:42:34.525 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-09-21T03:42:34.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.523+0000 7f7382f20f40 1 -- 172.21.15.38:0/3284913117 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x5606f2557040 con 0x5606f254fb20 2024-09-21T03:42:34.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.525+0000 7f737c8c1640 1 -- 172.21.15.38:0/3284913117 <== osd.3 v2:172.21.15.38:6808/1286810294 23 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v118'132 uv132 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7374011520 con 0x5606f254fb20 2024-09-21T03:42:34.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.525+0000 7f7382f20f40 1 -- 172.21.15.38:0/3284913117 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:109 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x5606f2557040 con 0x5606f254fb20 2024-09-21T03:42:34.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.526+0000 7f737c8c1640 1 -- 172.21.15.38:0/3284913117 <== osd.3 v2:172.21.15.38:6808/1286810294 24 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v118'133 uv133 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7374011520 con 0x5606f254fb20 2024-09-21T03:42:34.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.526+0000 7f7382f20f40 1 -- 172.21.15.38:0/3284913117 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:110 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x5606f2557040 con 0x5606f254fb20 2024-09-21T03:42:34.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.527+0000 7f737c8c1640 1 -- 172.21.15.38:0/3284913117 <== osd.3 v2:172.21.15.38:6808/1286810294 25 ==== osd_op_reply(110 609.00000000 [omap-set-header] v118'134 uv134 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7374011520 con 0x5606f254fb20 2024-09-21T03:42:34.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.527+0000 7f7382f20f40 1 --2- 172.21.15.38:0/3284913117 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x5606f2557940 0x5606f255a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:34.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.527+0000 7f7382f20f40 1 -- 172.21.15.38:0/3284913117 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x5606f255a580 con 0x5606f2557940 2024-09-21T03:42:34.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.527+0000 7f737b8bf640 1 --2- 172.21.15.38:0/3284913117 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x5606f2557940 0x5606f255a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:34.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.527+0000 7f737b8bf640 1 --2- 172.21.15.38:0/3284913117 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x5606f2557940 0x5606f255a040 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:34.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.528+0000 7f737b8bf640 1 -- 172.21.15.38:0/3284913117 <== osd.2 v2:172.21.15.38:6824/2577858902 1 ==== osd_op_reply(111 400.00000000 [create] v118'110 uv109 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f737001b070 con 0x5606f2557940 2024-09-21T03:42:34.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.528+0000 7f7382f20f40 1 -- 172.21.15.38:0/3284913117 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e118) -- 0x5606f255a580 con 0x5606f2557940 2024-09-21T03:42:34.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.530+0000 7f737b8bf640 1 -- 172.21.15.38:0/3284913117 <== osd.2 v2:172.21.15.38:6824/2577858902 2 ==== osd_op_reply(112 400.00000000 [delete] v118'111 uv111 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f737001b070 con 0x5606f2557940 2024-09-21T03:42:34.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.530+0000 7f7382f20f40 1 -- 172.21.15.38:0/3284913117 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x5606f255a580 con 0x5606f2557940 2024-09-21T03:42:34.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.531+0000 7f737b8bf640 1 -- 172.21.15.38:0/3284913117 <== osd.2 v2:172.21.15.38:6824/2577858902 3 ==== osd_op_reply(113 400.00000000 [create] v118'112 uv112 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f737001b070 con 0x5606f2557940 2024-09-21T03:42:34.533 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-09-21T03:42:34.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.531+0000 7f7382f20f40 1 -- 172.21.15.38:0/3284913117 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:114 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x5606f255a580 con 0x5606f2557940 2024-09-21T03:42:34.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.532+0000 7f737b8bf640 1 -- 172.21.15.38:0/3284913117 <== osd.2 v2:172.21.15.38:6824/2577858902 4 ==== osd_op_reply(114 400.00000000 [write 0~22] v118'113 uv113 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f737001b070 con 0x5606f2557940 2024-09-21T03:42:34.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.532+0000 7f7382f20f40 1 -- 172.21.15.38:0/3284913117 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:115 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x5606f255a580 con 0x5606f2557940 2024-09-21T03:42:34.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.533+0000 7f737b8bf640 1 -- 172.21.15.38:0/3284913117 <== osd.2 v2:172.21.15.38:6824/2577858902 5 ==== osd_op_reply(115 400.00000000 [omap-set-header] v118'114 uv114 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f737001b070 con 0x5606f2557940 2024-09-21T03:42:34.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.533+0000 7f7382f20f40 1 -- 172.21.15.38:0/3284913117 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:116 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x5606f255a580 con 0x5606f2553d50 2024-09-21T03:42:34.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.534+0000 7f737c0c0640 1 -- 172.21.15.38:0/3284913117 <== osd.1 v2:172.21.15.38:6816/1491818732 17 ==== osd_op_reply(116 100.00000000.inode [create] v118'265 uv257 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7f736c00b040 con 0x5606f2553d50 2024-09-21T03:42:34.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.534+0000 7f7382f20f40 1 -- 172.21.15.38:0/3284913117 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:117 13.1f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e118) -- 0x5606f255a580 con 0x5606f2553d50 2024-09-21T03:42:34.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.536+0000 7f737c0c0640 1 -- 172.21.15.38:0/3284913117 <== osd.1 v2:172.21.15.38:6816/1491818732 18 ==== osd_op_reply(117 100.00000000.inode [delete] v118'266 uv266 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f736c00b040 con 0x5606f2553d50 2024-09-21T03:42:34.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.536+0000 7f7382f20f40 1 -- 172.21.15.38:0/3284913117 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:118 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x5606f255a580 con 0x5606f2553d50 2024-09-21T03:42:34.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.537+0000 7f737c0c0640 1 -- 172.21.15.38:0/3284913117 <== osd.1 v2:172.21.15.38:6816/1491818732 19 ==== osd_op_reply(118 100.00000000.inode [create] v118'267 uv267 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f736c00b040 con 0x5606f2553d50 2024-09-21T03:42:34.539 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-09-21T03:42:34.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.537+0000 7f7382f20f40 1 -- 172.21.15.38:0/3284913117 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:119 13.1f 13:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x5606f255a580 con 0x5606f2553d50 2024-09-21T03:42:34.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.538+0000 7f737c0c0640 1 -- 172.21.15.38:0/3284913117 <== osd.1 v2:172.21.15.38:6816/1491818732 20 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v118'268 uv268 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f736c00b040 con 0x5606f2553d50 2024-09-21T03:42:34.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.538+0000 7f7382f20f40 1 -- 172.21.15.38:0/3284913117 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:120 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x5606f255a580 con 0x5606f2553d50 2024-09-21T03:42:34.541 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.539+0000 7f737c0c0640 1 -- 172.21.15.38:0/3284913117 <== osd.1 v2:172.21.15.38:6816/1491818732 21 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v118'269 uv269 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f736c00b040 con 0x5606f2553d50 2024-09-21T03:42:34.541 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.539+0000 7f7382f20f40 1 -- 172.21.15.38:0/3284913117 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:121 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x5606f255a580 con 0x5606f2553d50 2024-09-21T03:42:34.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.541+0000 7f737c0c0640 1 -- 172.21.15.38:0/3284913117 <== osd.1 v2:172.21.15.38:6816/1491818732 22 ==== osd_op_reply(121 1.00000000 [create] v118'270 uv264 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7f736c00b040 con 0x5606f2553d50 2024-09-21T03:42:34.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.541+0000 7f7382f20f40 1 -- 172.21.15.38:0/3284913117 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:122 13.1f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e118) -- 0x5606f255a580 con 0x5606f2553d50 2024-09-21T03:42:34.543 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.542+0000 7f737c0c0640 1 -- 172.21.15.38:0/3284913117 <== osd.1 v2:172.21.15.38:6816/1491818732 23 ==== osd_op_reply(122 1.00000000 [delete] v118'271 uv271 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f736c00b040 con 0x5606f2553d50 2024-09-21T03:42:34.544 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.542+0000 7f7382f20f40 1 -- 172.21.15.38:0/3284913117 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:123 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x5606f255a580 con 0x5606f2553d50 2024-09-21T03:42:34.544 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-09-21T03:42:34.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.543+0000 7f737c0c0640 1 -- 172.21.15.38:0/3284913117 <== osd.1 v2:172.21.15.38:6816/1491818732 24 ==== osd_op_reply(123 1.00000000 [create] v118'272 uv272 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f736c00b040 con 0x5606f2553d50 2024-09-21T03:42:34.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.543+0000 7f7382f20f40 1 -- 172.21.15.38:0/3284913117 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:124 13.1f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x5606f255b590 con 0x5606f2553d50 2024-09-21T03:42:34.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.544+0000 7f737c0c0640 1 -- 172.21.15.38:0/3284913117 <== osd.1 v2:172.21.15.38:6816/1491818732 25 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v118'273 uv273 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f736c00b040 con 0x5606f2553d50 2024-09-21T03:42:34.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.544+0000 7f7382f20f40 1 -- 172.21.15.38:0/3284913117 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:125 13.1f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x5606f255b590 con 0x5606f2553d50 2024-09-21T03:42:34.547 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.545+0000 7f737c0c0640 1 -- 172.21.15.38:0/3284913117 <== osd.1 v2:172.21.15.38:6816/1491818732 26 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v118'274 uv274 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f736c00b040 con 0x5606f2553d50 2024-09-21T03:42:34.547 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.545+0000 7f7382f20f40 1 -- 172.21.15.38:0/3284913117 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:126 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x5606f255a580 con 0x5606f2553d50 2024-09-21T03:42:34.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.546+0000 7f737c0c0640 1 -- 172.21.15.38:0/3284913117 <== osd.1 v2:172.21.15.38:6816/1491818732 27 ==== osd_op_reply(126 1.00000000 [omap-set-header] v118'275 uv275 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f736c00b040 con 0x5606f2553d50 2024-09-21T03:42:34.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.546+0000 7f7382f20f40 1 -- 172.21.15.38:0/3284913117 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:127 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=584b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x5606f255aa30 con 0x5606f2553d50 2024-09-21T03:42:34.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.548+0000 7f737c0c0640 1 -- 172.21.15.38:0/3284913117 <== osd.1 v2:172.21.15.38:6816/1491818732 28 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v118'276 uv276 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f736c00b040 con 0x5606f2553d50 2024-09-21T03:42:34.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.548+0000 7f7382f20f40 1 -- 172.21.15.38:0/3284913117 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x5606f254b560 msgr2=0x5606f254d980 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:34.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.548+0000 7f7382f20f40 1 --2- 172.21.15.38:0/3284913117 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x5606f254b560 0x5606f254d980 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:34.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.548+0000 7f7382f20f40 1 -- 172.21.15.38:0/3284913117 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x5606f2553d50 msgr2=0x5606f2556170 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:34.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.548+0000 7f7382f20f40 1 --2- 172.21.15.38:0/3284913117 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x5606f2553d50 0x5606f2556170 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:34.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.548+0000 7f7382f20f40 1 -- 172.21.15.38:0/3284913117 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x5606f2557940 msgr2=0x5606f255a040 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:34.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.548+0000 7f7382f20f40 1 --2- 172.21.15.38:0/3284913117 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x5606f2557940 0x5606f255a040 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:34.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.548+0000 7f7382f20f40 1 -- 172.21.15.38:0/3284913117 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x5606f254fb20 msgr2=0x5606f2552960 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:34.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.548+0000 7f7382f20f40 1 --2- 172.21.15.38:0/3284913117 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x5606f254fb20 0x5606f2552960 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:34.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.548+0000 7f7382f20f40 1 -- 172.21.15.38:0/3284913117 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x5606f2547020 msgr2=0x5606f2549440 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:34.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.548+0000 7f7382f20f40 1 --2- 172.21.15.38:0/3284913117 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x5606f2547020 0x5606f2549440 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:34.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.548+0000 7f737c8c1640 1 -- 172.21.15.38:0/3284913117 reap_dead start 2024-09-21T03:42:34.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.548+0000 7f7382f20f40 1 -- 172.21.15.38:0/3284913117 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x5606f2538930 msgr2=0x5606f2538d40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:34.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.548+0000 7f7382f20f40 1 --2- 172.21.15.38:0/3284913117 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x5606f2538930 0x5606f2538d40 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:34.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.548+0000 7f737c8c1640 1 -- 172.21.15.38:0/3284913117 reap_dead start 2024-09-21T03:42:34.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.548+0000 7f737c8c1640 1 -- 172.21.15.38:0/3284913117 reap_dead start 2024-09-21T03:42:34.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.548+0000 7f7382f20f40 1 -- 172.21.15.38:0/3284913117 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f73680775a0 msgr2=0x7f7368079ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:34.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.548+0000 7f7382f20f40 1 --2- 172.21.15.38:0/3284913117 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f73680775a0 0x7f7368079ab0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f736c056000 tx=0x7f736c0033a0 comp rx=0 tx=0).stop 2024-09-21T03:42:34.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.548+0000 7f7382f20f40 1 -- 172.21.15.38:0/3284913117 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5606f2413090 msgr2=0x5606f252c210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:34.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.548+0000 7f7382f20f40 1 --2- 172.21.15.38:0/3284913117 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5606f2413090 0x5606f252c210 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7f7370004160 tx=0x7f7370015440 comp rx=0 tx=0).stop 2024-09-21T03:42:34.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.548+0000 7f7382f20f40 1 -- 172.21.15.38:0/3284913117 shutdown_connections 2024-09-21T03:42:34.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.548+0000 7f7382f20f40 1 --2- 172.21.15.38:0/3284913117 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5606f2413090 0x5606f252c210 unknown :-1 s=CLOSED pgs=1776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:34.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.548+0000 7f7382f20f40 1 --2- 172.21.15.38:0/3284913117 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f73680775a0 0x7f7368079ab0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:34.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.548+0000 7f7382f20f40 1 -- 172.21.15.38:0/3284913117 >> 172.21.15.38:0/3284913117 conn(0x5606f22eaac0 msgr2=0x5606f247f930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:34.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.548+0000 7f7382f20f40 1 -- 172.21.15.38:0/3284913117 shutdown_connections 2024-09-21T03:42:34.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.548+0000 7f7382f20f40 1 -- 172.21.15.38:0/3284913117 wait complete. 2024-09-21T03:42:34.553 DEBUG:teuthology.orchestra.run.smithi038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata put 100.00000000.inode - 2024-09-21T03:42:34.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.648+0000 7fa12f71ef40 1 Processor -- start 2024-09-21T03:42:34.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.648+0000 7fa12f71ef40 1 -- start start 2024-09-21T03:42:34.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.649+0000 7fa12f71ef40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55ac25ea6240 0x55ac25f7a650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:34.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.649+0000 7fa12f71ef40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ac25eed090 0x55ac25ed1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:34.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.649+0000 7fa12f71ef40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ac25f62520 0x55ac25f5aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:34.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.649+0000 7fa12f71ef40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55ac25df3d60 con 0x55ac25eed090 2024-09-21T03:42:34.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.649+0000 7fa12f71ef40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55ac25e52430 con 0x55ac25f62520 2024-09-21T03:42:34.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.649+0000 7fa12f71ef40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55ac25f7a350 con 0x55ac25ea6240 2024-09-21T03:42:34.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.649+0000 7fa127f0a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ac25eed090 0x55ac25ed1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:34.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.649+0000 7fa127f0a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ac25eed090 0x55ac25ed1aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:52116/0 (socket says 172.21.15.38:52116) 2024-09-21T03:42:34.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.649+0000 7fa127f0a640 1 -- 172.21.15.38:0/3487908303 learned_addr learned my addr 172.21.15.38:0/3487908303 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:42:34.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.649+0000 7fa12870b640 1 --2- 172.21.15.38:0/3487908303 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55ac25ea6240 0x55ac25f7a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:34.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.649+0000 7fa128f0c640 1 --2- 172.21.15.38:0/3487908303 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ac25f62520 0x55ac25f5aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:34.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.650+0000 7fa127f0a640 1 -- 172.21.15.38:0/3487908303 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55ac25ea6240 msgr2=0x55ac25f7a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:34.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.650+0000 7fa127f0a640 1 --2- 172.21.15.38:0/3487908303 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55ac25ea6240 0x55ac25f7a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:34.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.650+0000 7fa127f0a640 1 -- 172.21.15.38:0/3487908303 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ac25f62520 msgr2=0x55ac25f5aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:34.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.650+0000 7fa127f0a640 1 --2- 172.21.15.38:0/3487908303 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ac25f62520 0x55ac25f5aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:34.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.650+0000 7fa127f0a640 1 -- 172.21.15.38:0/3487908303 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac25f72f80 con 0x55ac25eed090 2024-09-21T03:42:34.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.650+0000 7fa128f0c640 1 --2- 172.21.15.38:0/3487908303 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ac25f62520 0x55ac25f5aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:42:34.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.650+0000 7fa127f0a640 1 --2- 172.21.15.38:0/3487908303 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ac25eed090 0x55ac25ed1aa0 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7fa11c025f30 tx=0x7fa11c002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=657f9966a2637b7c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:34.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.650+0000 7fa126f08640 1 -- 172.21.15.38:0/3487908303 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa11c056670 con 0x55ac25eed090 2024-09-21T03:42:34.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.650+0000 7fa126f08640 1 -- 172.21.15.38:0/3487908303 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa11c056830 con 0x55ac25eed090 2024-09-21T03:42:34.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.650+0000 7fa126f08640 1 -- 172.21.15.38:0/3487908303 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa11c056c80 con 0x55ac25eed090 2024-09-21T03:42:34.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.650+0000 7fa12f71ef40 1 -- 172.21.15.38:0/3487908303 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ac25eed090 msgr2=0x55ac25ed1aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:34.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.650+0000 7fa12f71ef40 1 --2- 172.21.15.38:0/3487908303 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ac25eed090 0x55ac25ed1aa0 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7fa11c025f30 tx=0x7fa11c002fb0 comp rx=0 tx=0).stop 2024-09-21T03:42:34.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.651+0000 7fa12f71ef40 1 -- 172.21.15.38:0/3487908303 shutdown_connections 2024-09-21T03:42:34.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.651+0000 7fa12f71ef40 1 --2- 172.21.15.38:0/3487908303 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ac25f62520 0x55ac25f5aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:34.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.651+0000 7fa12f71ef40 1 --2- 172.21.15.38:0/3487908303 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ac25eed090 0x55ac25ed1aa0 unknown :-1 s=CLOSED pgs=1778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:34.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.651+0000 7fa12f71ef40 1 --2- 172.21.15.38:0/3487908303 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55ac25ea6240 0x55ac25f7a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:34.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.651+0000 7fa12f71ef40 1 -- 172.21.15.38:0/3487908303 >> 172.21.15.38:0/3487908303 conn(0x55ac25dc4ac0 msgr2=0x55ac25eeb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:34.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.651+0000 7fa12f71ef40 1 -- 172.21.15.38:0/3487908303 shutdown_connections 2024-09-21T03:42:34.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.651+0000 7fa12f71ef40 1 -- 172.21.15.38:0/3487908303 wait complete. 2024-09-21T03:42:34.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.651+0000 7fa12f71ef40 1 Processor -- start 2024-09-21T03:42:34.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.652+0000 7fa12f71ef40 1 -- start start 2024-09-21T03:42:34.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.653+0000 7fa12f71ef40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ac25ea6240 0x55ac25f6e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:34.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.653+0000 7fa12f71ef40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55ac25eed090 0x55ac25eeb7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:34.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.653+0000 7fa12f71ef40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ac25f62520 0x55ac25eebd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:34.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.653+0000 7fa12f71ef40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55ac25ed3940 con 0x55ac25f62520 2024-09-21T03:42:34.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.653+0000 7fa12f71ef40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55ac25f7adb0 con 0x55ac25ea6240 2024-09-21T03:42:34.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.653+0000 7fa12f71ef40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55ac2600ed00 con 0x55ac25eed090 2024-09-21T03:42:34.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.653+0000 7fa128f0c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ac25f62520 0x55ac25eebd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:34.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.653+0000 7fa128f0c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ac25f62520 0x55ac25eebd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:52128/0 (socket says 172.21.15.38:52128) 2024-09-21T03:42:34.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.653+0000 7fa128f0c640 1 -- 172.21.15.38:0/1687512686 learned_addr learned my addr 172.21.15.38:0/1687512686 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:42:34.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.653+0000 7fa12870b640 1 --2- 172.21.15.38:0/1687512686 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ac25ea6240 0x55ac25f6e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:34.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.653+0000 7fa127f0a640 1 --2- 172.21.15.38:0/1687512686 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55ac25eed090 0x55ac25eeb7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:34.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.653+0000 7fa128f0c640 1 -- 172.21.15.38:0/1687512686 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55ac25eed090 msgr2=0x55ac25eeb7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:34.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.653+0000 7fa128f0c640 1 --2- 172.21.15.38:0/1687512686 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55ac25eed090 0x55ac25eeb7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:34.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.653+0000 7fa128f0c640 1 -- 172.21.15.38:0/1687512686 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ac25ea6240 msgr2=0x55ac25f6e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:34.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.653+0000 7fa128f0c640 1 --2- 172.21.15.38:0/1687512686 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ac25ea6240 0x55ac25f6e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:34.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.653+0000 7fa128f0c640 1 -- 172.21.15.38:0/1687512686 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac25f7ab20 con 0x55ac25f62520 2024-09-21T03:42:34.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.653+0000 7fa128f0c640 1 --2- 172.21.15.38:0/1687512686 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ac25f62520 0x55ac25eebd30 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7fa12000b6a0 tx=0x7fa12000bb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:34.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.654+0000 7fa125f06640 1 -- 172.21.15.38:0/1687512686 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa120004220 con 0x55ac25f62520 2024-09-21T03:42:34.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.654+0000 7fa125f06640 1 -- 172.21.15.38:0/1687512686 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa120009aa0 con 0x55ac25f62520 2024-09-21T03:42:34.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.654+0000 7fa125f06640 1 -- 172.21.15.38:0/1687512686 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa120017450 con 0x55ac25f62520 2024-09-21T03:42:34.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.654+0000 7fa12f71ef40 1 -- 172.21.15.38:0/1687512686 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55ac25f59ec0 con 0x55ac25f62520 2024-09-21T03:42:34.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.654+0000 7fa12f71ef40 1 -- 172.21.15.38:0/1687512686 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac25f74ea0 con 0x55ac25f62520 2024-09-21T03:42:34.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.655+0000 7fa125f06640 1 -- 172.21.15.38:0/1687512686 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa12000f430 con 0x55ac25f62520 2024-09-21T03:42:34.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.656+0000 7fa125f06640 1 --2- 172.21.15.38:0/1687512686 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa1140775a0 0x7fa114079ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:34.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.656+0000 7fa125f06640 1 -- 172.21.15.38:0/1687512686 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 8196+0+0 (secure 0 0 0) 0x7fa12009b0f0 con 0x55ac25f62520 2024-09-21T03:42:34.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.656+0000 7fa12f71ef40 1 --2- 172.21.15.38:0/1687512686 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x55ac2600d750 0x55ac25f74210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:34.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.656+0000 7fa12870b640 1 --2- 172.21.15.38:0/1687512686 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa1140775a0 0x7fa114079ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:34.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.656+0000 7fa12f71ef40 1 -- 172.21.15.38:0/1687512686 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:1 13.1f 13:fe07bba1:::100.00000000.inode:head [writefull 0~80 in=80b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x55ac25f7c2e0 con 0x55ac2600d750 2024-09-21T03:42:34.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.656+0000 7fa127f0a640 1 --2- 172.21.15.38:0/1687512686 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x55ac2600d750 0x55ac25f74210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:34.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.657+0000 7fa127f0a640 1 --2- 172.21.15.38:0/1687512686 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x55ac2600d750 0x55ac25f74210 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:34.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.657+0000 7fa12870b640 1 --2- 172.21.15.38:0/1687512686 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa1140775a0 0x7fa114079ab0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fa11800b6a0 tx=0x7fa11800d040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:34.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.658+0000 7fa127f0a640 1 -- 172.21.15.38:0/1687512686 <== osd.1 v2:172.21.15.38:6816/1491818732 1 ==== osd_op_reply(1 100.00000000.inode [writefull 0~80] v118'277 uv277 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fa11c06a040 con 0x55ac2600d750 2024-09-21T03:42:34.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.658+0000 7fa12f71ef40 1 -- 172.21.15.38:0/1687512686 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x55ac2600d750 msgr2=0x55ac25f74210 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:34.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.658+0000 7fa12f71ef40 1 --2- 172.21.15.38:0/1687512686 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x55ac2600d750 0x55ac25f74210 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:34.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.658+0000 7fa12f71ef40 1 -- 172.21.15.38:0/1687512686 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa1140775a0 msgr2=0x7fa114079ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:34.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.658+0000 7fa12f71ef40 1 --2- 172.21.15.38:0/1687512686 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa1140775a0 0x7fa114079ab0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fa11800b6a0 tx=0x7fa11800d040 comp rx=0 tx=0).stop 2024-09-21T03:42:34.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.658+0000 7fa12f71ef40 1 -- 172.21.15.38:0/1687512686 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ac25f62520 msgr2=0x55ac25eebd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:34.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.658+0000 7fa12f71ef40 1 --2- 172.21.15.38:0/1687512686 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ac25f62520 0x55ac25eebd30 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7fa12000b6a0 tx=0x7fa12000bb60 comp rx=0 tx=0).stop 2024-09-21T03:42:34.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.658+0000 7fa128f0c640 1 -- 172.21.15.38:0/1687512686 reap_dead start 2024-09-21T03:42:34.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.658+0000 7fa12f71ef40 1 -- 172.21.15.38:0/1687512686 shutdown_connections 2024-09-21T03:42:34.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.658+0000 7fa12f71ef40 1 -- 172.21.15.38:0/1687512686 >> 172.21.15.38:0/1687512686 conn(0x55ac25dc4ac0 msgr2=0x55ac25f7bbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:34.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.659+0000 7fa12f71ef40 1 -- 172.21.15.38:0/1687512686 shutdown_connections 2024-09-21T03:42:34.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.659+0000 7fa12f71ef40 1 -- 172.21.15.38:0/1687512686 wait complete. 2024-09-21T03:42:34.664 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-21T03:42:34.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.823+0000 7f1e12aa4640 1 -- 172.21.15.38:0/1646877225 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1df4003a50 msgr2=0x7f1df4003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:34.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.823+0000 7f1e12aa4640 1 --2- 172.21.15.38:0/1646877225 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1df4003a50 0x7f1df4003ec0 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7f1e04001dc0 tx=0x7f1df4005830 comp rx=0 tx=0).stop 2024-09-21T03:42:34.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.823+0000 7f1e12aa4640 1 -- 172.21.15.38:0/1646877225 shutdown_connections 2024-09-21T03:42:34.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.823+0000 7f1e12aa4640 1 --2- 172.21.15.38:0/1646877225 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1df4003a50 0x7f1df4003ec0 unknown :-1 s=CLOSED pgs=1781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:34.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.823+0000 7f1e12aa4640 1 -- 172.21.15.38:0/1646877225 >> 172.21.15.38:0/1646877225 conn(0x7f1e0c0b46e0 msgr2=0x7f1e0c0d23e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:34.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.823+0000 7f1e12aa4640 1 -- 172.21.15.38:0/1646877225 shutdown_connections 2024-09-21T03:42:34.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.823+0000 7f1e12aa4640 1 -- 172.21.15.38:0/1646877225 wait complete. 2024-09-21T03:42:34.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.824+0000 7f1e12aa4640 1 Processor -- start 2024-09-21T03:42:34.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.824+0000 7f1e12aa4640 1 -- start start 2024-09-21T03:42:34.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.826+0000 7f1e12aa4640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1df4003a50 0x7f1e0c1f9090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:34.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.826+0000 7f1e12aa4640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1e0c06f9e0 0x7f1e0c152980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:34.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.826+0000 7f1e12aa4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1e0c155260 0x7f1e0c155670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:34.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.826+0000 7f1e12aa4640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f1e0c0d0670 con 0x7f1e0c155260 2024-09-21T03:42:34.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.826+0000 7f1e12aa4640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f1e0c0cda80 con 0x7f1df4003a50 2024-09-21T03:42:34.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.826+0000 7f1e12aa4640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f1e0c0d07f0 con 0x7f1e0c06f9e0 2024-09-21T03:42:34.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.826+0000 7f1e11041640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1e0c155260 0x7f1e0c155670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:34.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.826+0000 7f1e11041640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1e0c155260 0x7f1e0c155670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:52158/0 (socket says 172.21.15.38:52158) 2024-09-21T03:42:34.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.826+0000 7f1e11041640 1 -- 172.21.15.38:0/695067461 learned_addr learned my addr 172.21.15.38:0/695067461 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:42:34.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.826+0000 7f1e10840640 1 --2- 172.21.15.38:0/695067461 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1df4003a50 0x7f1e0c1f9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:34.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.826+0000 7f1e0bfff640 1 --2- 172.21.15.38:0/695067461 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1e0c06f9e0 0x7f1e0c152980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:34.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.826+0000 7f1e11041640 1 -- 172.21.15.38:0/695067461 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1e0c06f9e0 msgr2=0x7f1e0c152980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:34.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.826+0000 7f1e11041640 1 --2- 172.21.15.38:0/695067461 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1e0c06f9e0 0x7f1e0c152980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:34.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.826+0000 7f1e11041640 1 -- 172.21.15.38:0/695067461 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1df4003a50 msgr2=0x7f1e0c1f9090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:34.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.826+0000 7f1e11041640 1 --2- 172.21.15.38:0/695067461 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1df4003a50 0x7f1e0c1f9090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:34.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.826+0000 7f1e11041640 1 -- 172.21.15.38:0/695067461 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e0c155bb0 con 0x7f1e0c155260 2024-09-21T03:42:34.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.827+0000 7f1e11041640 1 --2- 172.21.15.38:0/695067461 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1e0c155260 0x7f1e0c155670 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7f1e0c0d34b0 tx=0x7f1dfc011b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:34.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.827+0000 7f1e097fa640 1 -- 172.21.15.38:0/695067461 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dfc01f070 con 0x7f1e0c155260 2024-09-21T03:42:34.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.827+0000 7f1e097fa640 1 -- 172.21.15.38:0/695067461 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1dfc00fad0 con 0x7f1e0c155260 2024-09-21T03:42:34.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.827+0000 7f1e12aa4640 1 -- 172.21.15.38:0/695067461 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e0c0c95c0 con 0x7f1e0c155260 2024-09-21T03:42:34.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.827+0000 7f1e097fa640 1 -- 172.21.15.38:0/695067461 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dfc012450 con 0x7f1e0c155260 2024-09-21T03:42:34.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.827+0000 7f1e12aa4640 1 -- 172.21.15.38:0/695067461 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e0c0cb2e0 con 0x7f1e0c155260 2024-09-21T03:42:34.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.828+0000 7f1e12aa4640 1 -- 172.21.15.38:0/695067461 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e0c0c9ae0 con 0x7f1e0c155260 2024-09-21T03:42:34.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.829+0000 7f1e097fa640 1 -- 172.21.15.38:0/695067461 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f1dfc022070 con 0x7f1e0c155260 2024-09-21T03:42:34.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.829+0000 7f1e097fa640 1 --2- 172.21.15.38:0/695067461 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1df8078bc0 0x7f1df807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:34.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.829+0000 7f1e097fa640 1 -- 172.21.15.38:0/695067461 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 8196+0+0 (secure 0 0 0) 0x7f1dfc09bea0 con 0x7f1e0c155260 2024-09-21T03:42:34.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.829+0000 7f1e10840640 1 --2- 172.21.15.38:0/695067461 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1df8078bc0 0x7f1df807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:34.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.830+0000 7f1e10840640 1 --2- 172.21.15.38:0/695067461 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1df8078bc0 0x7f1df807b0d0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f1e04032000 tx=0x7f1e04001510 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:34.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:34.834+0000 7f1e097fa640 1 -- 172.21.15.38:0/695067461 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f1e0c0c9ae0 con 0x7f1e0c155260 2024-09-21T03:42:35.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:35.062+0000 7f1e12aa4640 1 -- 172.21.15.38:0/695067461 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f1e0c0d0d00 con 0x7f1e0c155260 2024-09-21T03:42:35.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:35.302+0000 7f1e097fa640 1 -- 172.21.15.38:0/695067461 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v218) ==== 162+0+0 (secure 0 0 0) 0x7f1e0c155bb0 con 0x7f1e0c155260 2024-09-21T03:42:35.304 INFO:teuthology.orchestra.run.smithi038.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-21T03:42:35.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:35.305+0000 7f1e12aa4640 1 -- 172.21.15.38:0/695067461 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1df8078bc0 msgr2=0x7f1df807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:35.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:35.305+0000 7f1e12aa4640 1 --2- 172.21.15.38:0/695067461 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1df8078bc0 0x7f1df807b0d0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f1e04032000 tx=0x7f1e04001510 comp rx=0 tx=0).stop 2024-09-21T03:42:35.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:35.305+0000 7f1e12aa4640 1 -- 172.21.15.38:0/695067461 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1e0c155260 msgr2=0x7f1e0c155670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:35.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:35.305+0000 7f1e12aa4640 1 --2- 172.21.15.38:0/695067461 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1e0c155260 0x7f1e0c155670 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7f1e0c0d34b0 tx=0x7f1dfc011b30 comp rx=0 tx=0).stop 2024-09-21T03:42:35.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:35.305+0000 7f1e12aa4640 1 -- 172.21.15.38:0/695067461 shutdown_connections 2024-09-21T03:42:35.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:35.305+0000 7f1e12aa4640 1 --2- 172.21.15.38:0/695067461 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1e0c155260 0x7f1e0c155670 unknown :-1 s=CLOSED pgs=1782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:35.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:35.305+0000 7f1e12aa4640 1 --2- 172.21.15.38:0/695067461 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1df8078bc0 0x7f1df807b0d0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:35.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:35.305+0000 7f1e12aa4640 1 --2- 172.21.15.38:0/695067461 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1e0c06f9e0 0x7f1e0c152980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:35.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:35.305+0000 7f1e12aa4640 1 --2- 172.21.15.38:0/695067461 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1df4003a50 0x7f1e0c1f9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:35.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:35.305+0000 7f1e12aa4640 1 -- 172.21.15.38:0/695067461 >> 172.21.15.38:0/695067461 conn(0x7f1e0c0b46e0 msgr2=0x7f1e0c0712a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:35.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:35.305+0000 7f1e12aa4640 1 -- 172.21.15.38:0/695067461 shutdown_connections 2024-09-21T03:42:35.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:35.305+0000 7f1e12aa4640 1 -- 172.21.15.38:0/695067461 wait complete. 2024-09-21T03:42:35.319 INFO:tasks.ceph.mds.c.smithi038.stderr:2024-09-21T03:42:35.317+0000 7f6e0eba1640 -1 mds.0.cache.ino(0x100) Corrupt inode 0x100: End of buffer [buffer:2] 2024-09-21T03:42:35.319 INFO:tasks.ceph.mds.c.smithi038.stderr:2024-09-21T03:42:35.317+0000 7f6e0eba1640 -1 log_channel(cluster) log [ERR] : Error loading MDS rank 0: (22) Invalid argument 2024-09-21T03:42:35.319 INFO:tasks.ceph.mon.a.smithi038.stderr:2024-09-21T03:42:35.318+0000 7f3f45713640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T03:42:35.320 INFO:tasks.ceph.mon.a.smithi038.stderr:2024-09-21T03:42:35.318+0000 7f3f45713640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-09-21T03:42:35.321 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:42:35.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:35.436+0000 7f348d6de640 1 -- 172.21.15.38:0/1027263578 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3488074290 msgr2=0x7f348815cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:35.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:35.436+0000 7f348d6de640 1 --2- 172.21.15.38:0/1027263578 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3488074290 0x7f348815cdb0 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7f3488073800 tx=0x7f347c02a4d0 comp rx=0 tx=0).stop 2024-09-21T03:42:35.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:35.436+0000 7f348d6de640 1 -- 172.21.15.38:0/1027263578 shutdown_connections 2024-09-21T03:42:35.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:35.436+0000 7f348d6de640 1 --2- 172.21.15.38:0/1027263578 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3488074290 0x7f348815cdb0 unknown :-1 s=CLOSED pgs=1783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:35.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:35.436+0000 7f348d6de640 1 --2- 172.21.15.38:0/1027263578 >> v2:172.21.15.195:3300/0 conn(0x7f348806f560 0x7f3488073d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:35.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:35.436+0000 7f348d6de640 1 -- 172.21.15.38:0/1027263578 >> 172.21.15.38:0/1027263578 conn(0x7f34880b2cc0 msgr2=0x7f34880b30d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:35.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:35.436+0000 7f348d6de640 1 -- 172.21.15.38:0/1027263578 shutdown_connections 2024-09-21T03:42:35.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:35.436+0000 7f348d6de640 1 -- 172.21.15.38:0/1027263578 wait complete. 2024-09-21T03:42:35.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:35.438+0000 7f348d6de640 1 Processor -- start 2024-09-21T03:42:35.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:35.438+0000 7f348d6de640 1 -- start start 2024-09-21T03:42:35.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:35.440+0000 7f348d6de640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f348806f560 0x7f34880cc600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:35.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:35.440+0000 7f348d6de640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3488074290 0x7f34881591a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:35.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:35.440+0000 7f348d6de640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f348815d2f0 0x7f34881596e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:35.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:35.440+0000 7f348d6de640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f3488160860 con 0x7f348815d2f0 2024-09-21T03:42:35.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:35.440+0000 7f348d6de640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f34880ceab0 con 0x7f348806f560 2024-09-21T03:42:35.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:35.440+0000 7f348d6de640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f34881609e0 con 0x7f3488074290 2024-09-21T03:42:35.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:35.440+0000 7f34877fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f348815d2f0 0x7f34881596e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:35.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:35.440+0000 7f34877fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f348815d2f0 0x7f34881596e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:52176/0 (socket says 172.21.15.38:52176) 2024-09-21T03:42:35.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:35.440+0000 7f34877fe640 1 -- 172.21.15.38:0/548549231 learned_addr learned my addr 172.21.15.38:0/548549231 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:42:35.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:35.440+0000 7f3486ffd640 1 --2- 172.21.15.38:0/548549231 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f348806f560 0x7f34880cc600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:35.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:35.440+0000 7f34867fc640 1 --2- 172.21.15.38:0/548549231 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3488074290 0x7f34881591a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:35.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:35.440+0000 7f34877fe640 1 -- 172.21.15.38:0/548549231 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3488074290 msgr2=0x7f34881591a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:35.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:35.440+0000 7f34877fe640 1 --2- 172.21.15.38:0/548549231 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3488074290 0x7f34881591a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:35.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:35.440+0000 7f34877fe640 1 -- 172.21.15.38:0/548549231 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f348806f560 msgr2=0x7f34880cc600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:35.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:35.440+0000 7f34877fe640 1 --2- 172.21.15.38:0/548549231 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f348806f560 0x7f34880cc600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:35.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:35.440+0000 7f34877fe640 1 -- 172.21.15.38:0/548549231 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34880c5f30 con 0x7f348815d2f0 2024-09-21T03:42:35.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:35.440+0000 7f34877fe640 1 --2- 172.21.15.38:0/548549231 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f348815d2f0 0x7f34881596e0 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7f348000b990 tx=0x7f348000be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:35.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:35.440+0000 7f346f7fe640 1 -- 172.21.15.38:0/548549231 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3480019070 con 0x7f348815d2f0 2024-09-21T03:42:35.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:35.441+0000 7f346f7fe640 1 -- 172.21.15.38:0/548549231 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f348000cc90 con 0x7f348815d2f0 2024-09-21T03:42:35.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:35.441+0000 7f346f7fe640 1 -- 172.21.15.38:0/548549231 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34800146a0 con 0x7f348815d2f0 2024-09-21T03:42:35.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:35.441+0000 7f348d6de640 1 -- 172.21.15.38:0/548549231 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34880c6210 con 0x7f348815d2f0 2024-09-21T03:42:35.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:35.441+0000 7f348d6de640 1 -- 172.21.15.38:0/548549231 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34880c67d0 con 0x7f348815d2f0 2024-09-21T03:42:35.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:35.442+0000 7f348d6de640 1 -- 172.21.15.38:0/548549231 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3488073d50 con 0x7f348815d2f0 2024-09-21T03:42:35.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:35.442+0000 7f346f7fe640 1 -- 172.21.15.38:0/548549231 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f348001c070 con 0x7f348815d2f0 2024-09-21T03:42:35.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:35.443+0000 7f346f7fe640 1 --2- 172.21.15.38:0/548549231 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3474078b90 0x7f347407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:35.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:35.443+0000 7f346f7fe640 1 -- 172.21.15.38:0/548549231 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 8282+0+0 (secure 0 0 0) 0x7f348009b4c0 con 0x7f348815d2f0 2024-09-21T03:42:35.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:35.443+0000 7f3486ffd640 1 --2- 172.21.15.38:0/548549231 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3474078b90 0x7f347407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:35.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:35.444+0000 7f3486ffd640 1 --2- 172.21.15.38:0/548549231 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3474078b90 0x7f347407b0a0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f34780098e0 tx=0x7f3478009060 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:35.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:35.448+0000 7f346f7fe640 1 -- 172.21.15.38:0/548549231 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f3480062880 con 0x7f348815d2f0 2024-09-21T03:42:35.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:35.677+0000 7f348d6de640 1 -- 172.21.15.38:0/548549231 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f348815c1c0 con 0x7f348815d2f0 2024-09-21T03:42:35.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:35.677+0000 7f346f7fe640 1 -- 172.21.15.38:0/548549231 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 220 v220) ==== 95+0+3566 (secure 0 0 0) 0x7f34800672b0 con 0x7f348815d2f0 2024-09-21T03:42:35.680 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:42:35.680 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":220,"btime":"2024-09-21T03:42:35:319034+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6086,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/965972663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":965972663},{"type":"v1","addr":"172.21.15.195:6836","nonce":965972663}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6700,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/3176122334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":3176122334},{"type":"v1","addr":"172.21.15.195:6837","nonce":3176122334}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":220,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:40:02.319282+0000","modified":"2024-09-21T03:42:35.318972+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":119,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-21T03:42:35.680 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 220 2024-09-21T03:42:35.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:35.680+0000 7f348d6de640 1 -- 172.21.15.38:0/548549231 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3474078b90 msgr2=0x7f347407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:35.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:35.680+0000 7f348d6de640 1 --2- 172.21.15.38:0/548549231 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3474078b90 0x7f347407b0a0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f34780098e0 tx=0x7f3478009060 comp rx=0 tx=0).stop 2024-09-21T03:42:35.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:35.680+0000 7f348d6de640 1 -- 172.21.15.38:0/548549231 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f348815d2f0 msgr2=0x7f34881596e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:35.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:35.680+0000 7f348d6de640 1 --2- 172.21.15.38:0/548549231 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f348815d2f0 0x7f34881596e0 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7f348000b990 tx=0x7f348000be50 comp rx=0 tx=0).stop 2024-09-21T03:42:35.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:35.680+0000 7f348d6de640 1 -- 172.21.15.38:0/548549231 shutdown_connections 2024-09-21T03:42:35.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:35.680+0000 7f348d6de640 1 --2- 172.21.15.38:0/548549231 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f348815d2f0 0x7f34881596e0 unknown :-1 s=CLOSED pgs=1784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:35.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:35.680+0000 7f348d6de640 1 --2- 172.21.15.38:0/548549231 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3474078b90 0x7f347407b0a0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:35.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:35.680+0000 7f348d6de640 1 --2- 172.21.15.38:0/548549231 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3488074290 0x7f34881591a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:35.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:35.680+0000 7f348d6de640 1 --2- 172.21.15.38:0/548549231 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f348806f560 0x7f34880cc600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:35.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:35.680+0000 7f348d6de640 1 -- 172.21.15.38:0/548549231 >> 172.21.15.38:0/548549231 conn(0x7f34880b2cc0 msgr2=0x7f34880b0b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:35.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:35.680+0000 7f348d6de640 1 -- 172.21.15.38:0/548549231 shutdown_connections 2024-09-21T03:42:35.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:35.680+0000 7f348d6de640 1 -- 172.21.15.38:0/548549231 wait complete. 2024-09-21T03:42:35.693 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-21T03:42:35.693 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:42:35.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:35.810+0000 7f30ab77b640 1 -- 172.21.15.38:0/3288568014 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f308c004d40 msgr2=0x7f308c005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:35.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:35.810+0000 7f30ab77b640 1 --2- 172.21.15.38:0/3288568014 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f308c004d40 0x7f308c005150 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7f3094005fd0 tx=0x7f308c005830 comp rx=0 tx=0).stop 2024-09-21T03:42:35.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:35.810+0000 7f30a9d18640 1 -- 172.21.15.38:0/3288568014 reap_dead start 2024-09-21T03:42:35.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:35.810+0000 7f30ab77b640 1 -- 172.21.15.38:0/3288568014 shutdown_connections 2024-09-21T03:42:35.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:35.810+0000 7f30ab77b640 1 -- 172.21.15.38:0/3288568014 >> 172.21.15.38:0/3288568014 conn(0x7f30a40b6b20 msgr2=0x7f30a40d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:35.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:35.810+0000 7f30ab77b640 1 -- 172.21.15.38:0/3288568014 shutdown_connections 2024-09-21T03:42:35.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:35.810+0000 7f30ab77b640 1 -- 172.21.15.38:0/3288568014 wait complete. 2024-09-21T03:42:35.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:35.811+0000 7f30ab77b640 1 Processor -- start 2024-09-21T03:42:35.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:35.812+0000 7f30ab77b640 1 -- start start 2024-09-21T03:42:35.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:35.813+0000 7f30ab77b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f30a41e9af0 0x7f30a415a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:35.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:35.813+0000 7f30ab77b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f30a40ceae0 0x7f30a40b3b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:35.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:35.813+0000 7f30ab77b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f30a40b40c0 0x7f30a41f4f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:35.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:35.813+0000 7f30ab77b640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f30a40b35e0 con 0x7f30a40ceae0 2024-09-21T03:42:35.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:35.813+0000 7f30ab77b640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f30a40b3460 con 0x7f30a40b40c0 2024-09-21T03:42:35.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:35.813+0000 7f30ab77b640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f30a40b3760 con 0x7f30a41e9af0 2024-09-21T03:42:35.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:35.813+0000 7f30a8d16640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f30a40ceae0 0x7f30a40b3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:35.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:35.813+0000 7f30a8d16640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f30a40ceae0 0x7f30a40b3b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:52194/0 (socket says 172.21.15.38:52194) 2024-09-21T03:42:35.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:35.813+0000 7f30a8d16640 1 -- 172.21.15.38:0/2673871850 learned_addr learned my addr 172.21.15.38:0/2673871850 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:42:35.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:35.813+0000 7f30a9517640 1 --2- 172.21.15.38:0/2673871850 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f30a41e9af0 0x7f30a415a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:35.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:35.813+0000 7f30a9d18640 1 --2- 172.21.15.38:0/2673871850 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f30a40b40c0 0x7f30a41f4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:35.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:35.813+0000 7f30a8d16640 1 -- 172.21.15.38:0/2673871850 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f30a41e9af0 msgr2=0x7f30a415a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:35.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:35.813+0000 7f30a8d16640 1 --2- 172.21.15.38:0/2673871850 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f30a41e9af0 0x7f30a415a830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:35.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:35.813+0000 7f30a8d16640 1 -- 172.21.15.38:0/2673871850 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f30a40b40c0 msgr2=0x7f30a41f4f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:35.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:35.813+0000 7f30a8d16640 1 --2- 172.21.15.38:0/2673871850 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f30a40b40c0 0x7f30a41f4f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:35.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:35.813+0000 7f30a8d16640 1 -- 172.21.15.38:0/2673871850 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30a40b44d0 con 0x7f30a40ceae0 2024-09-21T03:42:35.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:35.813+0000 7f30a8d16640 1 --2- 172.21.15.38:0/2673871850 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f30a40ceae0 0x7f30a40b3b80 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7f3098006600 tx=0x7f3098003aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:35.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:35.814+0000 7f30a1ffb640 1 -- 172.21.15.38:0/2673871850 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3098015070 con 0x7f30a40ceae0 2024-09-21T03:42:35.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:35.814+0000 7f30a1ffb640 1 -- 172.21.15.38:0/2673871850 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3098011900 con 0x7f30a40ceae0 2024-09-21T03:42:35.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:35.814+0000 7f30a1ffb640 1 -- 172.21.15.38:0/2673871850 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3098011d50 con 0x7f30a40ceae0 2024-09-21T03:42:35.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:35.814+0000 7f30ab77b640 1 -- 172.21.15.38:0/2673871850 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30a41f93a0 con 0x7f30a40ceae0 2024-09-21T03:42:35.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:35.814+0000 7f30ab77b640 1 -- 172.21.15.38:0/2673871850 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30a41f8620 con 0x7f30a40ceae0 2024-09-21T03:42:35.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:35.815+0000 7f30ab77b640 1 -- 172.21.15.38:0/2673871850 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30a4162f00 con 0x7f30a40ceae0 2024-09-21T03:42:35.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:35.816+0000 7f30a1ffb640 1 -- 172.21.15.38:0/2673871850 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f309800f430 con 0x7f30a40ceae0 2024-09-21T03:42:35.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:35.816+0000 7f30a1ffb640 1 --2- 172.21.15.38:0/2673871850 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3090078b90 0x7f309007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:35.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:35.816+0000 7f30a1ffb640 1 -- 172.21.15.38:0/2673871850 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 8282+0+0 (secure 0 0 0) 0x7f309809c550 con 0x7f30a40ceae0 2024-09-21T03:42:35.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:35.817+0000 7f30a9517640 1 --2- 172.21.15.38:0/2673871850 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3090078b90 0x7f309007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:35.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:35.817+0000 7f30a9517640 1 --2- 172.21.15.38:0/2673871850 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3090078b90 0x7f309007b0a0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f30a40ce450 tx=0x7f309c005f90 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:35.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:35.821+0000 7f30a1ffb640 1 -- 172.21.15.38:0/2673871850 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f30a40b44d0 con 0x7f30a40ceae0 2024-09-21T03:42:36.029 INFO:tasks.ceph.mds.a.smithi038.stderr: -49> 2024-09-21T03:42:31.274+0000 7ff816d2a640 -1 log_channel(cluster) log [ERR] : failed to read JournalPointer: -22 ((22) Invalid argument) 2024-09-21T03:42:36.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:36.046+0000 7f30ab77b640 1 -- 172.21.15.38:0/2673871850 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f30a4069440 con 0x7f30a40ceae0 2024-09-21T03:42:36.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:36.047+0000 7f30a1ffb640 1 -- 172.21.15.38:0/2673871850 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 220 v220) ==== 95+0+3566 (secure 0 0 0) 0x7f3098063fd0 con 0x7f30a40ceae0 2024-09-21T03:42:36.049 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:42:36.049 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":220,"btime":"2024-09-21T03:42:35:319034+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6086,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/965972663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":965972663},{"type":"v1","addr":"172.21.15.195:6836","nonce":965972663}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6700,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/3176122334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":3176122334},{"type":"v1","addr":"172.21.15.195:6837","nonce":3176122334}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":220,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:40:02.319282+0000","modified":"2024-09-21T03:42:35.318972+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":119,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-21T03:42:36.049 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 220 2024-09-21T03:42:36.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:36.049+0000 7f30ab77b640 1 -- 172.21.15.38:0/2673871850 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3090078b90 msgr2=0x7f309007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:36.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:36.049+0000 7f30ab77b640 1 --2- 172.21.15.38:0/2673871850 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3090078b90 0x7f309007b0a0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f30a40ce450 tx=0x7f309c005f90 comp rx=0 tx=0).stop 2024-09-21T03:42:36.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:36.049+0000 7f30ab77b640 1 -- 172.21.15.38:0/2673871850 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f30a40ceae0 msgr2=0x7f30a40b3b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:36.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:36.049+0000 7f30ab77b640 1 --2- 172.21.15.38:0/2673871850 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f30a40ceae0 0x7f30a40b3b80 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7f3098006600 tx=0x7f3098003aa0 comp rx=0 tx=0).stop 2024-09-21T03:42:36.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:36.049+0000 7f30ab77b640 1 -- 172.21.15.38:0/2673871850 shutdown_connections 2024-09-21T03:42:36.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:36.049+0000 7f30ab77b640 1 --2- 172.21.15.38:0/2673871850 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f30a40b40c0 0x7f30a41f4f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:36.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:36.049+0000 7f30ab77b640 1 --2- 172.21.15.38:0/2673871850 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f30a40ceae0 0x7f30a40b3b80 unknown :-1 s=CLOSED pgs=1787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:36.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:36.049+0000 7f30ab77b640 1 --2- 172.21.15.38:0/2673871850 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3090078b90 0x7f309007b0a0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:36.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:36.049+0000 7f30ab77b640 1 --2- 172.21.15.38:0/2673871850 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f30a41e9af0 0x7f30a415a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:36.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:36.049+0000 7f30ab77b640 1 -- 172.21.15.38:0/2673871850 >> 172.21.15.38:0/2673871850 conn(0x7f30a40b6b20 msgr2=0x7f30a40cf8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:36.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:36.049+0000 7f30ab77b640 1 -- 172.21.15.38:0/2673871850 shutdown_connections 2024-09-21T03:42:36.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:36.049+0000 7f30ab77b640 1 -- 172.21.15.38:0/2673871850 wait complete. 2024-09-21T03:42:36.063 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-21T03:42:36.063 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:42:36.085 INFO:tasks.ceph.mds.a.smithi038.stdout:starting mds.a at 2024-09-21T03:42:36.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:36.180+0000 7fa33656e640 1 -- 172.21.15.38:0/2787137975 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa318004800 msgr2=0x7fa318004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:36.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:36.180+0000 7fa33656e640 1 --2- 172.21.15.38:0/2787137975 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa318004800 0x7fa318004c10 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7fa3240016c0 tx=0x7fa318005830 comp rx=0 tx=0).stop 2024-09-21T03:42:36.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:36.180+0000 7fa33656e640 1 -- 172.21.15.38:0/2787137975 shutdown_connections 2024-09-21T03:42:36.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:36.180+0000 7fa33656e640 1 --2- 172.21.15.38:0/2787137975 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa318004800 0x7fa318004c10 unknown :-1 s=CLOSED pgs=1791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:36.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:36.180+0000 7fa33656e640 1 -- 172.21.15.38:0/2787137975 >> 172.21.15.38:0/2787137975 conn(0x7fa3300b6b20 msgr2=0x7fa3300d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:36.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:36.180+0000 7fa33656e640 1 -- 172.21.15.38:0/2787137975 shutdown_connections 2024-09-21T03:42:36.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:36.180+0000 7fa33656e640 1 -- 172.21.15.38:0/2787137975 wait complete. 2024-09-21T03:42:36.183 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:36.182+0000 7fa33656e640 1 Processor -- start 2024-09-21T03:42:36.183 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:36.182+0000 7fa33656e640 1 -- start start 2024-09-21T03:42:36.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:36.183+0000 7fa33656e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa318004800 0x7fa3300c9f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:36.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:36.183+0000 7fa33656e640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa3301596b0 0x7fa330159ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:36.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:36.183+0000 7fa33656e640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa3301575f0 0x7fa330157a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:36.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:36.183+0000 7fa33656e640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa330162b80 con 0x7fa318004800 2024-09-21T03:42:36.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:36.183+0000 7fa33656e640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fa330162a00 con 0x7fa3301596b0 2024-09-21T03:42:36.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:36.183+0000 7fa33656e640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fa3300d24d0 con 0x7fa3301575f0 2024-09-21T03:42:36.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:36.183+0000 7fa32ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa318004800 0x7fa3300c9f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:36.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:36.183+0000 7fa32ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa318004800 0x7fa3300c9f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:52210/0 (socket says 172.21.15.38:52210) 2024-09-21T03:42:36.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:36.183+0000 7fa32ffff640 1 -- 172.21.15.38:0/2245911934 learned_addr learned my addr 172.21.15.38:0/2245911934 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:42:36.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:36.183+0000 7fa334b0b640 1 --2- 172.21.15.38:0/2245911934 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa3301575f0 0x7fa330157a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:36.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:36.183+0000 7fa32f7fe640 1 --2- 172.21.15.38:0/2245911934 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa3301596b0 0x7fa330159ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:36.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:36.183+0000 7fa32ffff640 1 -- 172.21.15.38:0/2245911934 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa3301575f0 msgr2=0x7fa330157a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:36.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:36.183+0000 7fa32ffff640 1 --2- 172.21.15.38:0/2245911934 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa3301575f0 0x7fa330157a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:36.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:36.183+0000 7fa32ffff640 1 -- 172.21.15.38:0/2245911934 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa3301596b0 msgr2=0x7fa330159ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:36.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:36.183+0000 7fa32ffff640 1 --2- 172.21.15.38:0/2245911934 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa3301596b0 0x7fa330159ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:36.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:36.183+0000 7fa32ffff640 1 -- 172.21.15.38:0/2245911934 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3301f52b0 con 0x7fa318004800 2024-09-21T03:42:36.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:36.184+0000 7fa334b0b640 1 --2- 172.21.15.38:0/2245911934 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa3301575f0 0x7fa330157a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:42:36.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:36.184+0000 7fa32ffff640 1 --2- 172.21.15.38:0/2245911934 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa318004800 0x7fa3300c9f20 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7fa320011a40 tx=0x7fa320011f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:36.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:36.184+0000 7fa32cff9640 1 -- 172.21.15.38:0/2245911934 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa32001e070 con 0x7fa318004800 2024-09-21T03:42:36.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:36.184+0000 7fa32cff9640 1 -- 172.21.15.38:0/2245911934 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa320004460 con 0x7fa318004800 2024-09-21T03:42:36.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:36.184+0000 7fa33656e640 1 -- 172.21.15.38:0/2245911934 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa330157f40 con 0x7fa318004800 2024-09-21T03:42:36.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:36.184+0000 7fa32cff9640 1 -- 172.21.15.38:0/2245911934 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa32000fc00 con 0x7fa318004800 2024-09-21T03:42:36.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:36.184+0000 7fa33656e640 1 -- 172.21.15.38:0/2245911934 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa330160f40 con 0x7fa318004800 2024-09-21T03:42:36.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:36.185+0000 7fa33656e640 1 -- 172.21.15.38:0/2245911934 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3300d2a30 con 0x7fa318004800 2024-09-21T03:42:36.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:36.186+0000 7fa32cff9640 1 -- 172.21.15.38:0/2245911934 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa320021070 con 0x7fa318004800 2024-09-21T03:42:36.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:36.186+0000 7fa32cff9640 1 --2- 172.21.15.38:0/2245911934 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa31c078bc0 0x7fa31c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:36.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:36.186+0000 7fa32cff9640 1 -- 172.21.15.38:0/2245911934 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 8282+0+0 (secure 0 0 0) 0x7fa32009c140 con 0x7fa318004800 2024-09-21T03:42:36.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:36.186+0000 7fa32f7fe640 1 --2- 172.21.15.38:0/2245911934 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa31c078bc0 0x7fa31c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:36.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:36.187+0000 7fa32f7fe640 1 --2- 172.21.15.38:0/2245911934 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa31c078bc0 0x7fa31c07b0d0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fa324032000 tx=0x7fa324001190 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:36.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:36.191+0000 7fa32cff9640 1 -- 172.21.15.38:0/2245911934 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fa330157f40 con 0x7fa318004800 2024-09-21T03:42:36.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:36.419+0000 7fa33656e640 1 -- 172.21.15.38:0/2245911934 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa33015a370 con 0x7fa318004800 2024-09-21T03:42:36.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:36.420+0000 7fa32cff9640 1 -- 172.21.15.38:0/2245911934 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 221 v221) ==== 95+0+4392 (secure 0 0 0) 0x7fa3301f52b0 con 0x7fa318004800 2024-09-21T03:42:36.422 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:42:36.422 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":221,"btime":"2024-09-21T03:42:36:320405+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6086,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/965972663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":965972663},{"type":"v1","addr":"172.21.15.195:6836","nonce":965972663}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6700,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/3176122334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":3176122334},{"type":"v1","addr":"172.21.15.195:6837","nonce":3176122334}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":9249,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/3055025811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":3055025811},{"type":"v1","addr":"172.21.15.38:6835","nonce":3055025811}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":221}],"filesystems":[{"mdsmap":{"epoch":220,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:40:02.319282+0000","modified":"2024-09-21T03:42:35.318972+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":119,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-21T03:42:36.422 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 221 2024-09-21T03:42:36.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:36.422+0000 7fa33656e640 1 -- 172.21.15.38:0/2245911934 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa31c078bc0 msgr2=0x7fa31c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:36.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:36.422+0000 7fa33656e640 1 --2- 172.21.15.38:0/2245911934 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa31c078bc0 0x7fa31c07b0d0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fa324032000 tx=0x7fa324001190 comp rx=0 tx=0).stop 2024-09-21T03:42:36.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:36.422+0000 7fa33656e640 1 -- 172.21.15.38:0/2245911934 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa318004800 msgr2=0x7fa3300c9f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:36.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:36.422+0000 7fa33656e640 1 --2- 172.21.15.38:0/2245911934 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa318004800 0x7fa3300c9f20 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7fa320011a40 tx=0x7fa320011f00 comp rx=0 tx=0).stop 2024-09-21T03:42:36.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:36.422+0000 7fa33656e640 1 -- 172.21.15.38:0/2245911934 shutdown_connections 2024-09-21T03:42:36.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:36.422+0000 7fa33656e640 1 --2- 172.21.15.38:0/2245911934 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa31c078bc0 0x7fa31c07b0d0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:36.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:36.422+0000 7fa33656e640 1 --2- 172.21.15.38:0/2245911934 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa3301575f0 0x7fa330157a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:36.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:36.422+0000 7fa33656e640 1 --2- 172.21.15.38:0/2245911934 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa3301596b0 0x7fa330159ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:36.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:36.422+0000 7fa33656e640 1 --2- 172.21.15.38:0/2245911934 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa318004800 0x7fa3300c9f20 unknown :-1 s=CLOSED pgs=1792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:36.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:36.422+0000 7fa33656e640 1 -- 172.21.15.38:0/2245911934 >> 172.21.15.38:0/2245911934 conn(0x7fa3300b6b20 msgr2=0x7fa3300b3fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:36.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:36.422+0000 7fa33656e640 1 -- 172.21.15.38:0/2245911934 shutdown_connections 2024-09-21T03:42:36.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:36.422+0000 7fa33656e640 1 -- 172.21.15.38:0/2245911934 wait complete. 2024-09-21T03:42:36.435 INFO:tasks.cephfs.test_damage:Result: Mutation 'Corrupt 100.00000000.inode' led to DAMAGED state 2024-09-21T03:42:36.435 INFO:tasks.cephfs.test_damage:Applying mutation 'Truncate 1.00000000.inode' 2024-09-21T03:42:36.436 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-21T03:42:36.436 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:42:36.492 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056795,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2354 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T03:42:36.492 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi038.front.sepia.ceph.com 2024-09-21T03:42:36.492 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:42:36.492 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T03:42:36.547 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T03:42:36.547 INFO:teuthology.orchestra.run.smithi038.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-09-21T03:42:36.548 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-21T03:42:36.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:36.703+0000 7f12dc269640 1 -- 172.21.15.38:0/635644203 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f12d40ceb90 msgr2=0x7f12d40cefa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:36.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:36.703+0000 7f12dc269640 1 --2- 172.21.15.38:0/635644203 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f12d40ceb90 0x7f12d40cefa0 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7f12d40cfba0 tx=0x7f12cc02a9a0 comp rx=0 tx=0).stop 2024-09-21T03:42:36.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:36.703+0000 7f12dc269640 1 -- 172.21.15.38:0/635644203 shutdown_connections 2024-09-21T03:42:36.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:36.703+0000 7f12dc269640 1 --2- 172.21.15.38:0/635644203 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f12d40ceb90 0x7f12d40cefa0 unknown :-1 s=CLOSED pgs=1794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:36.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:36.703+0000 7f12dc269640 1 --2- 172.21.15.38:0/635644203 >> v2:172.21.15.195:3300/0 conn(0x7f12d415ee80 0x7f12d40d0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:36.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:36.703+0000 7f12dc269640 1 -- 172.21.15.38:0/635644203 >> 172.21.15.38:0/635644203 conn(0x7f12d409d830 msgr2=0x7f12d40b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:36.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:36.703+0000 7f12dc269640 1 -- 172.21.15.38:0/635644203 shutdown_connections 2024-09-21T03:42:36.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:36.703+0000 7f12dc269640 1 -- 172.21.15.38:0/635644203 wait complete. 2024-09-21T03:42:36.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:36.705+0000 7f12dc269640 1 Processor -- start 2024-09-21T03:42:36.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:36.705+0000 7f12dc269640 1 -- start start 2024-09-21T03:42:36.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:36.706+0000 7f12dc269640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f12d40b9ad0 0x7f12d406f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:36.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:36.706+0000 7f12dc269640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f12d40ceb90 0x7f12d41f52a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:36.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:36.706+0000 7f12dc269640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f12d415ee80 0x7f12d406d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:36.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:36.706+0000 7f12dc269640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f12d40b6700 con 0x7f12d40b9ad0 2024-09-21T03:42:36.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:36.706+0000 7f12dc269640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f12d40b6580 con 0x7f12d40ceb90 2024-09-21T03:42:36.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:36.706+0000 7f12dc269640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f12d40b6880 con 0x7f12d415ee80 2024-09-21T03:42:36.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:36.707+0000 7f12da005640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f12d40b9ad0 0x7f12d406f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:36.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:36.707+0000 7f12da005640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f12d40b9ad0 0x7f12d406f7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:52222/0 (socket says 172.21.15.38:52222) 2024-09-21T03:42:36.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:36.707+0000 7f12da005640 1 -- 172.21.15.38:0/639970460 learned_addr learned my addr 172.21.15.38:0/639970460 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:42:36.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:36.707+0000 7f12d9804640 1 --2- 172.21.15.38:0/639970460 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f12d40ceb90 0x7f12d41f52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:36.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:36.707+0000 7f12da005640 1 -- 172.21.15.38:0/639970460 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f12d415ee80 msgr2=0x7f12d406d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:36.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:36.707+0000 7f12da806640 1 --2- 172.21.15.38:0/639970460 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f12d415ee80 0x7f12d406d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:36.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:36.707+0000 7f12da005640 1 --2- 172.21.15.38:0/639970460 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f12d415ee80 0x7f12d406d780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:36.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:36.707+0000 7f12da005640 1 -- 172.21.15.38:0/639970460 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f12d40ceb90 msgr2=0x7f12d41f52a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:36.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:36.707+0000 7f12da005640 1 --2- 172.21.15.38:0/639970460 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f12d40ceb90 0x7f12d41f52a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:36.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:36.707+0000 7f12da005640 1 -- 172.21.15.38:0/639970460 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12d41f57e0 con 0x7f12d40b9ad0 2024-09-21T03:42:36.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:36.707+0000 7f12da005640 1 --2- 172.21.15.38:0/639970460 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f12d40b9ad0 0x7f12d406f7e0 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7f12d000d740 tx=0x7f12d000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:36.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:36.707+0000 7f12caffd640 1 -- 172.21.15.38:0/639970460 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12d0017070 con 0x7f12d40b9ad0 2024-09-21T03:42:36.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:36.707+0000 7f12caffd640 1 -- 172.21.15.38:0/639970460 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f12d0009d20 con 0x7f12d40b9ad0 2024-09-21T03:42:36.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:36.707+0000 7f12dc269640 1 -- 172.21.15.38:0/639970460 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12d41f5cd0 con 0x7f12d40b9ad0 2024-09-21T03:42:36.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:36.707+0000 7f12caffd640 1 -- 172.21.15.38:0/639970460 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12d00106c0 con 0x7f12d40b9ad0 2024-09-21T03:42:36.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:36.707+0000 7f12dc269640 1 -- 172.21.15.38:0/639970460 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12d406cf80 con 0x7f12d40b9ad0 2024-09-21T03:42:36.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:36.708+0000 7f12dc269640 1 -- 172.21.15.38:0/639970460 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12d41f61f0 con 0x7f12d40b9ad0 2024-09-21T03:42:36.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:36.709+0000 7f12caffd640 1 -- 172.21.15.38:0/639970460 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f12d001a070 con 0x7f12d40b9ad0 2024-09-21T03:42:36.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:36.710+0000 7f12caffd640 1 --2- 172.21.15.38:0/639970460 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f12b8078b90 0x7f12b807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:36.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:36.710+0000 7f12caffd640 1 -- 172.21.15.38:0/639970460 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 8282+0+0 (secure 0 0 0) 0x7f12d009b960 con 0x7f12d40b9ad0 2024-09-21T03:42:36.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:36.710+0000 7f12d9804640 1 --2- 172.21.15.38:0/639970460 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f12b8078b90 0x7f12b807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:36.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:36.711+0000 7f12d9804640 1 --2- 172.21.15.38:0/639970460 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f12b8078b90 0x7f12b807b0a0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f12d415c930 tx=0x7f12c4009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:36.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:36.714+0000 7f12caffd640 1 -- 172.21.15.38:0/639970460 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f12d41f61f0 con 0x7f12d40b9ad0 2024-09-21T03:42:36.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:36.939+0000 7f12dc269640 1 -- 172.21.15.38:0/639970460 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f12d40b6dd0 con 0x7f12d40b9ad0 2024-09-21T03:42:37.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:37.327+0000 7f12caffd640 1 -- 172.21.15.38:0/639970460 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v222) ==== 161+0+0 (secure 0 0 0) 0x7f12d00632f0 con 0x7f12d40b9ad0 2024-09-21T03:42:37.330 INFO:teuthology.orchestra.run.smithi038.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-21T03:42:37.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:37.330+0000 7f12dc269640 1 -- 172.21.15.38:0/639970460 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f12b8078b90 msgr2=0x7f12b807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:37.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:37.330+0000 7f12dc269640 1 --2- 172.21.15.38:0/639970460 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f12b8078b90 0x7f12b807b0a0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f12d415c930 tx=0x7f12c4009040 comp rx=0 tx=0).stop 2024-09-21T03:42:37.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:37.330+0000 7f12dc269640 1 -- 172.21.15.38:0/639970460 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f12d40b9ad0 msgr2=0x7f12d406f7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:37.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:37.330+0000 7f12dc269640 1 --2- 172.21.15.38:0/639970460 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f12d40b9ad0 0x7f12d406f7e0 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7f12d000d740 tx=0x7f12d000dc00 comp rx=0 tx=0).stop 2024-09-21T03:42:37.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:37.330+0000 7f12dc269640 1 -- 172.21.15.38:0/639970460 shutdown_connections 2024-09-21T03:42:37.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:37.330+0000 7f12dc269640 1 --2- 172.21.15.38:0/639970460 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f12b8078b90 0x7f12b807b0a0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:37.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:37.330+0000 7f12dc269640 1 --2- 172.21.15.38:0/639970460 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f12d415ee80 0x7f12d406d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:37.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:37.330+0000 7f12dc269640 1 --2- 172.21.15.38:0/639970460 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f12d40ceb90 0x7f12d41f52a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:37.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:37.330+0000 7f12dc269640 1 --2- 172.21.15.38:0/639970460 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f12d40b9ad0 0x7f12d406f7e0 unknown :-1 s=CLOSED pgs=1795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:37.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:37.330+0000 7f12dc269640 1 -- 172.21.15.38:0/639970460 >> 172.21.15.38:0/639970460 conn(0x7f12d409d830 msgr2=0x7f12d415e0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:37.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:37.330+0000 7f12dc269640 1 -- 172.21.15.38:0/639970460 shutdown_connections 2024-09-21T03:42:37.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:37.330+0000 7f12dc269640 1 -- 172.21.15.38:0/639970460 wait complete. 2024-09-21T03:42:37.346 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-09-21T03:42:37.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:37.465+0000 7f1157757640 1 -- 172.21.15.38:0/266369293 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f11500d0bb0 msgr2=0x7f11500b5be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:37.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:37.465+0000 7f1157757640 1 --2- 172.21.15.38:0/266369293 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f11500d0bb0 0x7f11500b5be0 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7f1148009830 tx=0x7f114802a480 comp rx=0 tx=0).stop 2024-09-21T03:42:37.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:37.465+0000 7f1157757640 1 -- 172.21.15.38:0/266369293 shutdown_connections 2024-09-21T03:42:37.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:37.465+0000 7f1157757640 1 --2- 172.21.15.38:0/266369293 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f11500d0bb0 0x7f11500b5be0 unknown :-1 s=CLOSED pgs=1797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:37.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:37.465+0000 7f1157757640 1 --2- 172.21.15.38:0/266369293 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f115015ca90 0x7f11500d0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:37.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:37.465+0000 7f1157757640 1 -- 172.21.15.38:0/266369293 >> 172.21.15.38:0/266369293 conn(0x7f11500b46e0 msgr2=0x7f11500d23e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:37.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:37.465+0000 7f1157757640 1 -- 172.21.15.38:0/266369293 shutdown_connections 2024-09-21T03:42:37.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:37.465+0000 7f1157757640 1 -- 172.21.15.38:0/266369293 wait complete. 2024-09-21T03:42:37.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:37.467+0000 7f1157757640 1 Processor -- start 2024-09-21T03:42:37.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:37.467+0000 7f1157757640 1 -- start start 2024-09-21T03:42:37.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:37.468+0000 7f1157757640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f11500ccc40 0x7f11500b26b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:37.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:37.468+0000 7f1157757640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f11500d0bb0 0x7f11501f45c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:37.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:37.468+0000 7f1157757640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f115015ca90 0x7f11501f4b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:37.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:37.468+0000 7f1157757640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f11500b6620 con 0x7f115015ca90 2024-09-21T03:42:37.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:37.468+0000 7f1157757640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f11500b64a0 con 0x7f11500d0bb0 2024-09-21T03:42:37.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:37.468+0000 7f1157757640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f11500b67a0 con 0x7f11500ccc40 2024-09-21T03:42:37.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:37.468+0000 7f1155cf4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f115015ca90 0x7f11501f4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:37.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:37.468+0000 7f1155cf4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f115015ca90 0x7f11501f4b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:52238/0 (socket says 172.21.15.38:52238) 2024-09-21T03:42:37.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:37.468+0000 7f1155cf4640 1 -- 172.21.15.38:0/4185268610 learned_addr learned my addr 172.21.15.38:0/4185268610 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:42:37.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:37.468+0000 7f1154cf2640 1 --2- 172.21.15.38:0/4185268610 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f11500d0bb0 0x7f11501f45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:37.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:37.468+0000 7f11554f3640 1 --2- 172.21.15.38:0/4185268610 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f11500ccc40 0x7f11500b26b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:37.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:37.469+0000 7f1155cf4640 1 -- 172.21.15.38:0/4185268610 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f11500ccc40 msgr2=0x7f11500b26b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:37.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:37.469+0000 7f1155cf4640 1 --2- 172.21.15.38:0/4185268610 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f11500ccc40 0x7f11500b26b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:37.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:37.469+0000 7f1155cf4640 1 -- 172.21.15.38:0/4185268610 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f11500d0bb0 msgr2=0x7f11501f45c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:37.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:37.469+0000 7f1155cf4640 1 --2- 172.21.15.38:0/4185268610 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f11500d0bb0 0x7f11501f45c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:37.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:37.469+0000 7f1155cf4640 1 -- 172.21.15.38:0/4185268610 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11500b2bf0 con 0x7f115015ca90 2024-09-21T03:42:37.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:37.469+0000 7f1155cf4640 1 --2- 172.21.15.38:0/4185268610 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f115015ca90 0x7f11501f4b00 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7f114802aa70 tx=0x7f1148033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:37.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:37.469+0000 7f114dffb640 1 -- 172.21.15.38:0/4185268610 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1148031d60 con 0x7f115015ca90 2024-09-21T03:42:37.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:37.469+0000 7f114dffb640 1 -- 172.21.15.38:0/4185268610 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1148033e10 con 0x7f115015ca90 2024-09-21T03:42:37.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:37.469+0000 7f1157757640 1 -- 172.21.15.38:0/4185268610 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11501f5040 con 0x7f115015ca90 2024-09-21T03:42:37.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:37.469+0000 7f114dffb640 1 -- 172.21.15.38:0/4185268610 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1148005260 con 0x7f115015ca90 2024-09-21T03:42:37.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:37.469+0000 7f1157757640 1 -- 172.21.15.38:0/4185268610 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11500a9f10 con 0x7f115015ca90 2024-09-21T03:42:37.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:37.471+0000 7f1157757640 1 -- 172.21.15.38:0/4185268610 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11500a99a0 con 0x7f115015ca90 2024-09-21T03:42:37.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:37.471+0000 7f114dffb640 1 -- 172.21.15.38:0/4185268610 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f1148005400 con 0x7f115015ca90 2024-09-21T03:42:37.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:37.472+0000 7f114dffb640 1 --2- 172.21.15.38:0/4185268610 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f113c078bc0 0x7f113c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:37.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:37.472+0000 7f114dffb640 1 -- 172.21.15.38:0/4185268610 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 8282+0+0 (secure 0 0 0) 0x7f11480bc930 con 0x7f115015ca90 2024-09-21T03:42:37.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:37.472+0000 7f11554f3640 1 --2- 172.21.15.38:0/4185268610 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f113c078bc0 0x7f113c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:37.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:37.473+0000 7f11554f3640 1 --2- 172.21.15.38:0/4185268610 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f113c078bc0 0x7f113c07b0d0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f11400012e0 tx=0x7f1140009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:37.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:37.476+0000 7f114dffb640 1 -- 172.21.15.38:0/4185268610 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f114803f030 con 0x7f115015ca90 2024-09-21T03:42:37.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:37.739+0000 7f1157757640 1 -- 172.21.15.38:0/4185268610 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7f11500c76e0 con 0x7f115015ca90 2024-09-21T03:42:38.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.330+0000 7f114dffb640 1 -- 172.21.15.38:0/4185268610 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v223) ==== 101+0+0 (secure 0 0 0) 0x7f11500c76e0 con 0x7f115015ca90 2024-09-21T03:42:38.332 INFO:teuthology.orchestra.run.smithi038.stderr:repaired: restoring rank 6:0 2024-09-21T03:42:38.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.333+0000 7f1157757640 1 -- 172.21.15.38:0/4185268610 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f113c078bc0 msgr2=0x7f113c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:38.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.333+0000 7f1157757640 1 --2- 172.21.15.38:0/4185268610 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f113c078bc0 0x7f113c07b0d0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f11400012e0 tx=0x7f1140009040 comp rx=0 tx=0).stop 2024-09-21T03:42:38.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.333+0000 7f1157757640 1 -- 172.21.15.38:0/4185268610 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f115015ca90 msgr2=0x7f11501f4b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:38.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.333+0000 7f1157757640 1 --2- 172.21.15.38:0/4185268610 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f115015ca90 0x7f11501f4b00 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7f114802aa70 tx=0x7f1148033c20 comp rx=0 tx=0).stop 2024-09-21T03:42:38.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.333+0000 7f1157757640 1 -- 172.21.15.38:0/4185268610 shutdown_connections 2024-09-21T03:42:38.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.333+0000 7f1157757640 1 --2- 172.21.15.38:0/4185268610 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f115015ca90 0x7f11501f4b00 unknown :-1 s=CLOSED pgs=1798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:38.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.333+0000 7f1157757640 1 --2- 172.21.15.38:0/4185268610 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f11500d0bb0 0x7f11501f45c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:38.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.333+0000 7f1157757640 1 --2- 172.21.15.38:0/4185268610 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f113c078bc0 0x7f113c07b0d0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:38.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.333+0000 7f1157757640 1 --2- 172.21.15.38:0/4185268610 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f11500ccc40 0x7f11500b26b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:38.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.333+0000 7f1157757640 1 -- 172.21.15.38:0/4185268610 >> 172.21.15.38:0/4185268610 conn(0x7f11500b46e0 msgr2=0x7f11500cf790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:38.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.333+0000 7f1157757640 1 -- 172.21.15.38:0/4185268610 shutdown_connections 2024-09-21T03:42:38.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.333+0000 7f1157757640 1 -- 172.21.15.38:0/4185268610 wait complete. 2024-09-21T03:42:38.350 DEBUG:teuthology.orchestra.run.smithi038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-09-21T03:42:38.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.409+0000 7fd82d60cf40 1 Processor -- start 2024-09-21T03:42:38.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.409+0000 7fd82d60cf40 1 -- start start 2024-09-21T03:42:38.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.411+0000 7fd82d60cf40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd8140030f0 0x562543150a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:38.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.411+0000 7fd82d60cf40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56254314e830 0x56254314ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:38.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.411+0000 7fd82d60cf40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562543129a00 0x562543129e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:38.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.411+0000 7fd82d60cf40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x56254302d430 con 0x7fd8140030f0 2024-09-21T03:42:38.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.411+0000 7fd82d60cf40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x56254303a1f0 con 0x56254314e830 2024-09-21T03:42:38.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.411+0000 7fd82d60cf40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x562542fced60 con 0x562543129a00 2024-09-21T03:42:38.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.411+0000 7fd82635b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd8140030f0 0x562543150a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:38.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.411+0000 7fd82635b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd8140030f0 0x562543150a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:52268/0 (socket says 172.21.15.38:52268) 2024-09-21T03:42:38.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.411+0000 7fd82635b640 1 -- 172.21.15.38:0/3166800663 learned_addr learned my addr 172.21.15.38:0/3166800663 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:42:38.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.411+0000 7fd82635b640 1 -- 172.21.15.38:0/3166800663 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562543129a00 msgr2=0x562543129e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:38.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.411+0000 7fd82635b640 1 --2- 172.21.15.38:0/3166800663 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562543129a00 0x562543129e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:38.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.411+0000 7fd82635b640 1 -- 172.21.15.38:0/3166800663 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56254314e830 msgr2=0x56254314ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:38.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.411+0000 7fd82635b640 1 --2- 172.21.15.38:0/3166800663 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56254314e830 0x56254314ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:38.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.411+0000 7fd82635b640 1 -- 172.21.15.38:0/3166800663 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56254314f180 con 0x7fd8140030f0 2024-09-21T03:42:38.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.411+0000 7fd82635b640 1 --2- 172.21.15.38:0/3166800663 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd8140030f0 0x562543150a00 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7fd818001b70 tx=0x7fd818001ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cafebdc188c03f25 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:38.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.412+0000 7fd824b58640 1 -- 172.21.15.38:0/3166800663 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd818019070 con 0x7fd8140030f0 2024-09-21T03:42:38.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.412+0000 7fd824b58640 1 -- 172.21.15.38:0/3166800663 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd818005890 con 0x7fd8140030f0 2024-09-21T03:42:38.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.412+0000 7fd824b58640 1 -- 172.21.15.38:0/3166800663 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd818005ce0 con 0x7fd8140030f0 2024-09-21T03:42:38.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.412+0000 7fd82d60cf40 1 -- 172.21.15.38:0/3166800663 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd8140030f0 msgr2=0x562543150a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:38.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.412+0000 7fd82d60cf40 1 --2- 172.21.15.38:0/3166800663 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd8140030f0 0x562543150a00 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7fd818001b70 tx=0x7fd818001ba0 comp rx=0 tx=0).stop 2024-09-21T03:42:38.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.412+0000 7fd82d60cf40 1 -- 172.21.15.38:0/3166800663 shutdown_connections 2024-09-21T03:42:38.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.412+0000 7fd82d60cf40 1 --2- 172.21.15.38:0/3166800663 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562543129a00 0x562543129e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:38.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.412+0000 7fd82d60cf40 1 --2- 172.21.15.38:0/3166800663 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56254314e830 0x56254314ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:38.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.412+0000 7fd82d60cf40 1 --2- 172.21.15.38:0/3166800663 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd8140030f0 0x562543150a00 unknown :-1 s=CLOSED pgs=1801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:38.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.412+0000 7fd82d60cf40 1 -- 172.21.15.38:0/3166800663 >> 172.21.15.38:0/3166800663 conn(0x562542f9fac0 msgr2=0x56254307c660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:38.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.412+0000 7fd82d60cf40 1 -- 172.21.15.38:0/3166800663 shutdown_connections 2024-09-21T03:42:38.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.412+0000 7fd82d60cf40 1 -- 172.21.15.38:0/3166800663 wait complete. 2024-09-21T03:42:38.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.413+0000 7fd82d60cf40 1 Processor -- start 2024-09-21T03:42:38.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.413+0000 7fd82d60cf40 1 -- start start 2024-09-21T03:42:38.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.415+0000 7fd82d60cf40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562543129a00 0x562543122010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:38.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.415+0000 7fd82d60cf40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56254314e830 0x56254314d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:38.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.415+0000 7fd82d60cf40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd8140030f0 0x56254314fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:38.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.415+0000 7fd82d60cf40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5625430ab390 con 0x56254314e830 2024-09-21T03:42:38.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.415+0000 7fd82d60cf40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x56254313bdd0 con 0x562543129a00 2024-09-21T03:42:38.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.415+0000 7fd82d60cf40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x5625431228a0 con 0x7fd8140030f0 2024-09-21T03:42:38.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.415+0000 7fd825b5a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56254314e830 0x56254314d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:38.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.415+0000 7fd825b5a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56254314e830 0x56254314d830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:52272/0 (socket says 172.21.15.38:52272) 2024-09-21T03:42:38.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.415+0000 7fd825b5a640 1 -- 172.21.15.38:0/57432031 learned_addr learned my addr 172.21.15.38:0/57432031 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:42:38.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.415+0000 7fd82635b640 1 --2- 172.21.15.38:0/57432031 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562543129a00 0x562543122010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:38.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.415+0000 7fd826b5c640 1 --2- 172.21.15.38:0/57432031 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd8140030f0 0x56254314fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:38.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.415+0000 7fd825b5a640 1 -- 172.21.15.38:0/57432031 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd8140030f0 msgr2=0x56254314fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:38.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.415+0000 7fd825b5a640 1 --2- 172.21.15.38:0/57432031 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd8140030f0 0x56254314fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:38.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.415+0000 7fd825b5a640 1 -- 172.21.15.38:0/57432031 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562543129a00 msgr2=0x562543122010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:38.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.415+0000 7fd825b5a640 1 --2- 172.21.15.38:0/57432031 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562543129a00 0x562543122010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:38.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.415+0000 7fd825b5a640 1 -- 172.21.15.38:0/57432031 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56254314dd70 con 0x56254314e830 2024-09-21T03:42:38.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.415+0000 7fd82635b640 1 --2- 172.21.15.38:0/57432031 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562543129a00 0x562543122010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:42:38.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.415+0000 7fd825b5a640 1 --2- 172.21.15.38:0/57432031 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56254314e830 0x56254314d830 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7fd810026270 tx=0x7fd810056a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:38.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.416+0000 7fd81f7fe640 1 -- 172.21.15.38:0/57432031 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd810063030 con 0x56254314e830 2024-09-21T03:42:38.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.416+0000 7fd81f7fe640 1 -- 172.21.15.38:0/57432031 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd81004dad0 con 0x56254314e830 2024-09-21T03:42:38.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.416+0000 7fd82d60cf40 1 -- 172.21.15.38:0/57432031 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x562543129f30 con 0x56254314e830 2024-09-21T03:42:38.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.416+0000 7fd81f7fe640 1 -- 172.21.15.38:0/57432031 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd810004e30 con 0x56254314e830 2024-09-21T03:42:38.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.416+0000 7fd82d60cf40 1 -- 172.21.15.38:0/57432031 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56254312a3a0 con 0x56254314e830 2024-09-21T03:42:38.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.417+0000 7fd81f7fe640 1 -- 172.21.15.38:0/57432031 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fd810066070 con 0x56254314e830 2024-09-21T03:42:38.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.418+0000 7fd81f7fe640 1 --2- 172.21.15.38:0/57432031 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd8080775a0 0x7fd808079ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:38.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.418+0000 7fd81f7fe640 1 -- 172.21.15.38:0/57432031 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 8282+0+0 (secure 0 0 0) 0x7fd8100de6e0 con 0x56254314e830 2024-09-21T03:42:38.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.418+0000 7fd82635b640 1 --2- 172.21.15.38:0/57432031 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd8080775a0 0x7fd808079ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:38.422 INFO:teuthology.orchestra.run.smithi038.stdout:Importing pool 2024-09-21T03:42:38.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.418+0000 7fd82d60cf40 1 --2- 172.21.15.38:0/57432031 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x56254314df60 0x56254313a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:38.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.418+0000 7fd82d60cf40 1 -- 172.21.15.38:0/57432031 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x56254313ab00 con 0x56254314df60 2024-09-21T03:42:38.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.419+0000 7fd826b5c640 1 --2- 172.21.15.38:0/57432031 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x56254314df60 0x56254313a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:38.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.419+0000 7fd82635b640 1 --2- 172.21.15.38:0/57432031 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd8080775a0 0x7fd808079ab0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fd818000f80 tx=0x562543154d80 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:38.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.419+0000 7fd826b5c640 1 --2- 172.21.15.38:0/57432031 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x56254314df60 0x56254313a5c0 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:38.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.421+0000 7fd826b5c640 1 -- 172.21.15.38:0/57432031 <== osd.7 v2:172.21.15.195:6818/1606106342 1 ==== osd_op_reply(1 601.00000000 [create] v119'691 uv665 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fd82000b040 con 0x56254314df60 2024-09-21T03:42:38.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.421+0000 7fd82d60cf40 1 -- 172.21.15.38:0/57432031 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e119) -- 0x5625431f8390 con 0x56254314df60 2024-09-21T03:42:38.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.423+0000 7fd826b5c640 1 -- 172.21.15.38:0/57432031 <== osd.7 v2:172.21.15.195:6818/1606106342 2 ==== osd_op_reply(2 601.00000000 [delete] v119'692 uv692 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd82000b040 con 0x56254314df60 2024-09-21T03:42:38.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.423+0000 7fd82d60cf40 1 -- 172.21.15.38:0/57432031 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x5625431f8730 con 0x56254314df60 2024-09-21T03:42:38.426 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-09-21T03:42:38.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.425+0000 7fd826b5c640 1 -- 172.21.15.38:0/57432031 <== osd.7 v2:172.21.15.195:6818/1606106342 3 ==== osd_op_reply(3 601.00000000 [create] v119'693 uv693 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd82000b040 con 0x56254314df60 2024-09-21T03:42:38.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.425+0000 7fd82d60cf40 1 -- 172.21.15.38:0/57432031 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x5625431f9ae0 con 0x56254314df60 2024-09-21T03:42:38.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.426+0000 7fd826b5c640 1 -- 172.21.15.38:0/57432031 <== osd.7 v2:172.21.15.195:6818/1606106342 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v119'694 uv694 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd82000b040 con 0x56254314df60 2024-09-21T03:42:38.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.427+0000 7fd82d60cf40 1 -- 172.21.15.38:0/57432031 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x5625431f9e80 con 0x56254314df60 2024-09-21T03:42:38.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.428+0000 7fd826b5c640 1 -- 172.21.15.38:0/57432031 <== osd.7 v2:172.21.15.195:6818/1606106342 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v119'695 uv695 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd82000b040 con 0x56254314df60 2024-09-21T03:42:38.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.428+0000 7fd82d60cf40 1 -- 172.21.15.38:0/57432031 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x5625431f8ad0 con 0x56254314df60 2024-09-21T03:42:38.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.430+0000 7fd826b5c640 1 -- 172.21.15.38:0/57432031 <== osd.7 v2:172.21.15.195:6818/1606106342 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v119'696 uv696 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd82000b040 con 0x56254314df60 2024-09-21T03:42:38.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.430+0000 7fd82d60cf40 1 -- 172.21.15.38:0/57432031 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x5625431f8e70 con 0x56254314df60 2024-09-21T03:42:38.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.432+0000 7fd826b5c640 1 -- 172.21.15.38:0/57432031 <== osd.7 v2:172.21.15.195:6818/1606106342 7 ==== osd_op_reply(7 602.00000000 [create] v119'697 uv671 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fd82000b040 con 0x56254314df60 2024-09-21T03:42:38.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.432+0000 7fd82d60cf40 1 -- 172.21.15.38:0/57432031 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e119) -- 0x5625431f9260 con 0x56254314df60 2024-09-21T03:42:38.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.433+0000 7fd826b5c640 1 -- 172.21.15.38:0/57432031 <== osd.7 v2:172.21.15.195:6818/1606106342 8 ==== osd_op_reply(8 602.00000000 [delete] v119'698 uv698 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd82000b040 con 0x56254314df60 2024-09-21T03:42:38.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.433+0000 7fd82d60cf40 1 -- 172.21.15.38:0/57432031 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x5625431f9260 con 0x56254314df60 2024-09-21T03:42:38.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.435+0000 7fd826b5c640 1 -- 172.21.15.38:0/57432031 <== osd.7 v2:172.21.15.195:6818/1606106342 9 ==== osd_op_reply(9 602.00000000 [create] v119'699 uv699 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd82000b040 con 0x56254314df60 2024-09-21T03:42:38.437 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-09-21T03:42:38.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.435+0000 7fd82d60cf40 1 -- 172.21.15.38:0/57432031 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x5625431f9210 con 0x56254314df60 2024-09-21T03:42:38.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.436+0000 7fd826b5c640 1 -- 172.21.15.38:0/57432031 <== osd.7 v2:172.21.15.195:6818/1606106342 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v119'700 uv700 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd82000b040 con 0x56254314df60 2024-09-21T03:42:38.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.437+0000 7fd82d60cf40 1 -- 172.21.15.38:0/57432031 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x5625431f9210 con 0x56254314df60 2024-09-21T03:42:38.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.438+0000 7fd826b5c640 1 -- 172.21.15.38:0/57432031 <== osd.7 v2:172.21.15.195:6818/1606106342 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v119'701 uv701 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd82000b040 con 0x56254314df60 2024-09-21T03:42:38.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.438+0000 7fd82d60cf40 1 -- 172.21.15.38:0/57432031 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x5625431f9210 con 0x56254314df60 2024-09-21T03:42:38.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.440+0000 7fd826b5c640 1 -- 172.21.15.38:0/57432031 <== osd.7 v2:172.21.15.195:6818/1606106342 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v119'702 uv702 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd82000b040 con 0x56254314df60 2024-09-21T03:42:38.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.440+0000 7fd82d60cf40 1 -- 172.21.15.38:0/57432031 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:13 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x5625431f9210 con 0x56254314df60 2024-09-21T03:42:38.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.441+0000 7fd826b5c640 1 -- 172.21.15.38:0/57432031 <== osd.7 v2:172.21.15.195:6818/1606106342 13 ==== osd_op_reply(13 10000000000.00000000 [create] v119'703 uv678 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7fd82000b040 con 0x56254314df60 2024-09-21T03:42:38.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.441+0000 7fd82d60cf40 1 -- 172.21.15.38:0/57432031 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:14 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e119) -- 0x5625431f9210 con 0x56254314df60 2024-09-21T03:42:38.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.443+0000 7fd826b5c640 1 -- 172.21.15.38:0/57432031 <== osd.7 v2:172.21.15.195:6818/1606106342 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v119'704 uv704 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fd82000b040 con 0x56254314df60 2024-09-21T03:42:38.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.443+0000 7fd82d60cf40 1 -- 172.21.15.38:0/57432031 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:15 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x5625431f9210 con 0x56254314df60 2024-09-21T03:42:38.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.445+0000 7fd826b5c640 1 -- 172.21.15.38:0/57432031 <== osd.7 v2:172.21.15.195:6818/1606106342 15 ==== osd_op_reply(15 10000000000.00000000 [create] v119'705 uv705 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fd82000b040 con 0x56254314df60 2024-09-21T03:42:38.447 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-09-21T03:42:38.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.445+0000 7fd82d60cf40 1 -- 172.21.15.38:0/57432031 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:16 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x5625431f9210 con 0x56254314df60 2024-09-21T03:42:38.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.446+0000 7fd826b5c640 1 -- 172.21.15.38:0/57432031 <== osd.7 v2:172.21.15.195:6818/1606106342 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v119'706 uv706 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fd82000b040 con 0x56254314df60 2024-09-21T03:42:38.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.446+0000 7fd82d60cf40 1 -- 172.21.15.38:0/57432031 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:17 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x5625431f9210 con 0x56254314df60 2024-09-21T03:42:38.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.449+0000 7fd826b5c640 1 -- 172.21.15.38:0/57432031 <== osd.7 v2:172.21.15.195:6818/1606106342 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v119'707 uv707 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fd82000b040 con 0x56254314df60 2024-09-21T03:42:38.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.449+0000 7fd82d60cf40 1 -- 172.21.15.38:0/57432031 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:18 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x5625431f9210 con 0x56254314df60 2024-09-21T03:42:38.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.450+0000 7fd826b5c640 1 -- 172.21.15.38:0/57432031 <== osd.7 v2:172.21.15.195:6818/1606106342 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v119'708 uv708 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fd82000b040 con 0x56254314df60 2024-09-21T03:42:38.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.450+0000 7fd82d60cf40 1 -- 172.21.15.38:0/57432031 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:19 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=585b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x5625431fa220 con 0x56254314df60 2024-09-21T03:42:38.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.451+0000 7fd826b5c640 1 -- 172.21.15.38:0/57432031 <== osd.7 v2:172.21.15.195:6818/1606106342 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v119'709 uv709 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fd82000b040 con 0x56254314df60 2024-09-21T03:42:38.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.452+0000 7fd82d60cf40 1 -- 172.21.15.38:0/57432031 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:20 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x5625431fa220 con 0x56254314df60 2024-09-21T03:42:38.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.454+0000 7fd826b5c640 1 -- 172.21.15.38:0/57432031 <== osd.7 v2:172.21.15.195:6818/1606106342 20 ==== osd_op_reply(20 600.00000000 [create] v119'710 uv684 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fd82000b040 con 0x56254314df60 2024-09-21T03:42:38.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.454+0000 7fd82d60cf40 1 -- 172.21.15.38:0/57432031 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:21 13.10 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e119) -- 0x5625431fa220 con 0x56254314df60 2024-09-21T03:42:38.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.455+0000 7fd826b5c640 1 -- 172.21.15.38:0/57432031 <== osd.7 v2:172.21.15.195:6818/1606106342 21 ==== osd_op_reply(21 600.00000000 [delete] v119'711 uv711 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd82000b040 con 0x56254314df60 2024-09-21T03:42:38.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.455+0000 7fd82d60cf40 1 -- 172.21.15.38:0/57432031 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:22 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x5625431fa220 con 0x56254314df60 2024-09-21T03:42:38.458 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-09-21T03:42:38.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.456+0000 7fd826b5c640 1 -- 172.21.15.38:0/57432031 <== osd.7 v2:172.21.15.195:6818/1606106342 22 ==== osd_op_reply(22 600.00000000 [create] v119'712 uv712 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd82000b040 con 0x56254314df60 2024-09-21T03:42:38.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.456+0000 7fd82d60cf40 1 -- 172.21.15.38:0/57432031 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:23 13.10 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x5625431fb230 con 0x56254314df60 2024-09-21T03:42:38.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.458+0000 7fd826b5c640 1 -- 172.21.15.38:0/57432031 <== osd.7 v2:172.21.15.195:6818/1606106342 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v119'713 uv713 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd82000b040 con 0x56254314df60 2024-09-21T03:42:38.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.458+0000 7fd82d60cf40 1 -- 172.21.15.38:0/57432031 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:24 13.10 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x5625431fb230 con 0x56254314df60 2024-09-21T03:42:38.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.460+0000 7fd826b5c640 1 -- 172.21.15.38:0/57432031 <== osd.7 v2:172.21.15.195:6818/1606106342 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v119'714 uv714 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd82000b040 con 0x56254314df60 2024-09-21T03:42:38.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.460+0000 7fd82d60cf40 1 -- 172.21.15.38:0/57432031 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:25 13.10 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x5625431fa220 con 0x56254314df60 2024-09-21T03:42:38.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.461+0000 7fd826b5c640 1 -- 172.21.15.38:0/57432031 <== osd.7 v2:172.21.15.195:6818/1606106342 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v119'715 uv715 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd82000b040 con 0x56254314df60 2024-09-21T03:42:38.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.461+0000 7fd82d60cf40 1 -- 172.21.15.38:0/57432031 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:26 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x5625431fa220 con 0x56254314df60 2024-09-21T03:42:38.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.463+0000 7fd826b5c640 1 -- 172.21.15.38:0/57432031 <== osd.7 v2:172.21.15.195:6818/1606106342 26 ==== osd_op_reply(26 603.00000000 [create] v119'716 uv690 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fd82000b040 con 0x56254314df60 2024-09-21T03:42:38.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.463+0000 7fd82d60cf40 1 -- 172.21.15.38:0/57432031 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:27 13.10 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e119) -- 0x5625431fa220 con 0x56254314df60 2024-09-21T03:42:38.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.464+0000 7fd826b5c640 1 -- 172.21.15.38:0/57432031 <== osd.7 v2:172.21.15.195:6818/1606106342 27 ==== osd_op_reply(27 603.00000000 [delete] v119'717 uv717 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd82000b040 con 0x56254314df60 2024-09-21T03:42:38.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.465+0000 7fd82d60cf40 1 -- 172.21.15.38:0/57432031 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:28 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x5625431fa220 con 0x56254314df60 2024-09-21T03:42:38.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.466+0000 7fd826b5c640 1 -- 172.21.15.38:0/57432031 <== osd.7 v2:172.21.15.195:6818/1606106342 28 ==== osd_op_reply(28 603.00000000 [create] v119'718 uv718 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd82000b040 con 0x56254314df60 2024-09-21T03:42:38.468 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-09-21T03:42:38.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.466+0000 7fd82d60cf40 1 -- 172.21.15.38:0/57432031 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:29 13.10 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x5625431fb230 con 0x56254314df60 2024-09-21T03:42:38.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.468+0000 7fd826b5c640 1 -- 172.21.15.38:0/57432031 <== osd.7 v2:172.21.15.195:6818/1606106342 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v119'719 uv719 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd82000b040 con 0x56254314df60 2024-09-21T03:42:38.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.468+0000 7fd82d60cf40 1 -- 172.21.15.38:0/57432031 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:30 13.10 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x5625431fb230 con 0x56254314df60 2024-09-21T03:42:38.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.469+0000 7fd826b5c640 1 -- 172.21.15.38:0/57432031 <== osd.7 v2:172.21.15.195:6818/1606106342 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v119'720 uv720 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd82000b040 con 0x56254314df60 2024-09-21T03:42:38.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.469+0000 7fd82d60cf40 1 -- 172.21.15.38:0/57432031 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:31 13.10 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x5625431fa220 con 0x56254314df60 2024-09-21T03:42:38.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.471+0000 7fd826b5c640 1 -- 172.21.15.38:0/57432031 <== osd.7 v2:172.21.15.195:6818/1606106342 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v119'721 uv721 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd82000b040 con 0x56254314df60 2024-09-21T03:42:38.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.471+0000 7fd82d60cf40 1 --2- 172.21.15.38:0/57432031 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x5625431fa730 0x5625431fcb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:38.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.471+0000 7fd82d60cf40 1 -- 172.21.15.38:0/57432031 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x5625431fd090 con 0x5625431fa730 2024-09-21T03:42:38.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.472+0000 7fd82635b640 1 --2- 172.21.15.38:0/57432031 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x5625431fa730 0x5625431fcb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:38.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.472+0000 7fd82635b640 1 --2- 172.21.15.38:0/57432031 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x5625431fa730 0x5625431fcb50 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:38.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.474+0000 7fd82635b640 1 -- 172.21.15.38:0/57432031 <== osd.6 v2:172.21.15.195:6826/4055825675 1 ==== osd_op_reply(32 1.00000000.inode [create] v119'504 uv487 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7fd818003040 con 0x5625431fa730 2024-09-21T03:42:38.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.474+0000 7fd82d60cf40 1 -- 172.21.15.38:0/57432031 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e119) -- 0x5625431fd430 con 0x5625431fa730 2024-09-21T03:42:38.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.475+0000 7fd82635b640 1 -- 172.21.15.38:0/57432031 <== osd.6 v2:172.21.15.195:6826/4055825675 2 ==== osd_op_reply(33 1.00000000.inode [delete] v119'505 uv505 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fd818003040 con 0x5625431fa730 2024-09-21T03:42:38.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.475+0000 7fd82d60cf40 1 -- 172.21.15.38:0/57432031 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x5625431fd7d0 con 0x5625431fa730 2024-09-21T03:42:38.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.477+0000 7fd82635b640 1 -- 172.21.15.38:0/57432031 <== osd.6 v2:172.21.15.195:6826/4055825675 3 ==== osd_op_reply(34 1.00000000.inode [create] v119'506 uv506 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fd818003040 con 0x5625431fa730 2024-09-21T03:42:38.479 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-09-21T03:42:38.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.477+0000 7fd82d60cf40 1 -- 172.21.15.38:0/57432031 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x5625431fe020 con 0x5625431fa730 2024-09-21T03:42:38.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.479+0000 7fd82635b640 1 -- 172.21.15.38:0/57432031 <== osd.6 v2:172.21.15.195:6826/4055825675 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v119'507 uv507 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fd818003040 con 0x5625431fa730 2024-09-21T03:42:38.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.479+0000 7fd82d60cf40 1 -- 172.21.15.38:0/57432031 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x5625431fe3c0 con 0x5625431fa730 2024-09-21T03:42:38.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.480+0000 7fd82635b640 1 -- 172.21.15.38:0/57432031 <== osd.6 v2:172.21.15.195:6826/4055825675 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v119'508 uv508 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fd818003040 con 0x5625431fa730 2024-09-21T03:42:38.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.480+0000 7fd82d60cf40 1 -- 172.21.15.38:0/57432031 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x5625431fe760 con 0x5625431fa730 2024-09-21T03:42:38.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.481+0000 7fd82635b640 1 -- 172.21.15.38:0/57432031 <== osd.6 v2:172.21.15.195:6826/4055825675 6 ==== osd_op_reply(37 200.00000000 [create] v119'509 uv492 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fd818003040 con 0x5625431fa730 2024-09-21T03:42:38.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.481+0000 7fd82d60cf40 1 -- 172.21.15.38:0/57432031 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e119) -- 0x5625431feb00 con 0x5625431fa730 2024-09-21T03:42:38.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.483+0000 7fd82635b640 1 -- 172.21.15.38:0/57432031 <== osd.6 v2:172.21.15.195:6826/4055825675 7 ==== osd_op_reply(38 200.00000000 [delete] v119'510 uv510 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd818003040 con 0x5625431fa730 2024-09-21T03:42:38.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.483+0000 7fd82d60cf40 1 -- 172.21.15.38:0/57432031 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x5625431feea0 con 0x5625431fa730 2024-09-21T03:42:38.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.484+0000 7fd82635b640 1 -- 172.21.15.38:0/57432031 <== osd.6 v2:172.21.15.195:6826/4055825675 8 ==== osd_op_reply(39 200.00000000 [create] v119'511 uv511 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd818003040 con 0x5625431fa730 2024-09-21T03:42:38.486 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-09-21T03:42:38.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.484+0000 7fd82d60cf40 1 -- 172.21.15.38:0/57432031 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x5625431feea0 con 0x5625431fa730 2024-09-21T03:42:38.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.485+0000 7fd82635b640 1 -- 172.21.15.38:0/57432031 <== osd.6 v2:172.21.15.195:6826/4055825675 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v119'512 uv512 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd818003040 con 0x5625431fa730 2024-09-21T03:42:38.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.485+0000 7fd82d60cf40 1 -- 172.21.15.38:0/57432031 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x5625431feea0 con 0x5625431fa730 2024-09-21T03:42:38.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.487+0000 7fd82635b640 1 -- 172.21.15.38:0/57432031 <== osd.6 v2:172.21.15.195:6826/4055825675 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v119'513 uv513 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd818003040 con 0x5625431fa730 2024-09-21T03:42:38.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.487+0000 7fd82d60cf40 1 -- 172.21.15.38:0/57432031 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:42 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x5625431feea0 con 0x5625431fa730 2024-09-21T03:42:38.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.488+0000 7fd82635b640 1 -- 172.21.15.38:0/57432031 <== osd.6 v2:172.21.15.195:6826/4055825675 11 ==== osd_op_reply(42 200.00000001 [create] v119'514 uv497 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fd818003040 con 0x5625431fa730 2024-09-21T03:42:38.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.488+0000 7fd82d60cf40 1 -- 172.21.15.38:0/57432031 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:43 13.14 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e119) -- 0x5625431feea0 con 0x5625431fa730 2024-09-21T03:42:38.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.490+0000 7fd82635b640 1 -- 172.21.15.38:0/57432031 <== osd.6 v2:172.21.15.195:6826/4055825675 12 ==== osd_op_reply(43 200.00000001 [delete] v119'515 uv515 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd818003040 con 0x5625431fa730 2024-09-21T03:42:38.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.490+0000 7fd82d60cf40 1 -- 172.21.15.38:0/57432031 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:44 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x5625431feea0 con 0x5625431fa730 2024-09-21T03:42:38.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.491+0000 7fd82635b640 1 -- 172.21.15.38:0/57432031 <== osd.6 v2:172.21.15.195:6826/4055825675 13 ==== osd_op_reply(44 200.00000001 [create] v119'516 uv516 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd818003040 con 0x5625431fa730 2024-09-21T03:42:38.493 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-09-21T03:42:38.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.491+0000 7fd82d60cf40 1 -- 172.21.15.38:0/57432031 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:45 13.14 13:2e2fa760:::200.00000001:head [write 0~17490 in=17490b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x562543203320 con 0x5625431fa730 2024-09-21T03:42:38.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.492+0000 7fd82635b640 1 -- 172.21.15.38:0/57432031 <== osd.6 v2:172.21.15.195:6826/4055825675 14 ==== osd_op_reply(45 200.00000001 [write 0~17490] v119'517 uv517 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd818003040 con 0x5625431fa730 2024-09-21T03:42:38.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.492+0000 7fd82d60cf40 1 -- 172.21.15.38:0/57432031 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:46 13.14 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x562543203320 con 0x5625431fa730 2024-09-21T03:42:38.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.494+0000 7fd82635b640 1 -- 172.21.15.38:0/57432031 <== osd.6 v2:172.21.15.195:6826/4055825675 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v119'518 uv518 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd818003040 con 0x5625431fa730 2024-09-21T03:42:38.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.494+0000 7fd82d60cf40 1 -- 172.21.15.38:0/57432031 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:47 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x5625431feea0 con 0x5625431fa730 2024-09-21T03:42:38.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.495+0000 7fd82635b640 1 -- 172.21.15.38:0/57432031 <== osd.6 v2:172.21.15.195:6826/4055825675 16 ==== osd_op_reply(47 606.00000000 [create] v119'519 uv503 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fd818003040 con 0x5625431fa730 2024-09-21T03:42:38.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.495+0000 7fd82d60cf40 1 -- 172.21.15.38:0/57432031 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:48 13.14 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e119) -- 0x5625431feea0 con 0x5625431fa730 2024-09-21T03:42:38.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.496+0000 7fd82635b640 1 -- 172.21.15.38:0/57432031 <== osd.6 v2:172.21.15.195:6826/4055825675 17 ==== osd_op_reply(48 606.00000000 [delete] v119'520 uv520 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd818003040 con 0x5625431fa730 2024-09-21T03:42:38.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.496+0000 7fd82d60cf40 1 -- 172.21.15.38:0/57432031 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:49 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x5625431feea0 con 0x5625431fa730 2024-09-21T03:42:38.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.498+0000 7fd82635b640 1 -- 172.21.15.38:0/57432031 <== osd.6 v2:172.21.15.195:6826/4055825675 18 ==== osd_op_reply(49 606.00000000 [create] v119'521 uv521 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd818003040 con 0x5625431fa730 2024-09-21T03:42:38.500 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-09-21T03:42:38.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.498+0000 7fd82d60cf40 1 -- 172.21.15.38:0/57432031 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:50 13.14 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x5625431ffeb0 con 0x5625431fa730 2024-09-21T03:42:38.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.499+0000 7fd82635b640 1 -- 172.21.15.38:0/57432031 <== osd.6 v2:172.21.15.195:6826/4055825675 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v119'522 uv522 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd818003040 con 0x5625431fa730 2024-09-21T03:42:38.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.499+0000 7fd82d60cf40 1 -- 172.21.15.38:0/57432031 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:51 13.14 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x5625431ffeb0 con 0x5625431fa730 2024-09-21T03:42:38.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.500+0000 7fd82635b640 1 -- 172.21.15.38:0/57432031 <== osd.6 v2:172.21.15.195:6826/4055825675 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v119'523 uv523 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd818003040 con 0x5625431fa730 2024-09-21T03:42:38.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.500+0000 7fd82d60cf40 1 -- 172.21.15.38:0/57432031 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:52 13.14 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x5625431feea0 con 0x5625431fa730 2024-09-21T03:42:38.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.502+0000 7fd82635b640 1 -- 172.21.15.38:0/57432031 <== osd.6 v2:172.21.15.195:6826/4055825675 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v119'524 uv524 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd818003040 con 0x5625431fa730 2024-09-21T03:42:38.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.502+0000 7fd82d60cf40 1 --2- 172.21.15.38:0/57432031 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x5625431ff3b0 0x5625432017d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:38.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.502+0000 7fd82d60cf40 1 -- 172.21.15.38:0/57432031 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:53 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x562543201d10 con 0x5625431ff3b0 2024-09-21T03:42:38.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.502+0000 7fd825b5a640 1 --2- 172.21.15.38:0/57432031 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x5625431ff3b0 0x5625432017d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:38.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.502+0000 7fd825b5a640 1 --2- 172.21.15.38:0/57432031 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x5625431ff3b0 0x5625432017d0 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:38.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.503+0000 7fd825b5a640 1 -- 172.21.15.38:0/57432031 <== osd.0 v2:172.21.15.38:6800/2381639433 1 ==== osd_op_reply(53 607.00000000 [create] v119'135 uv134 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fd8100a1d80 con 0x5625431ff3b0 2024-09-21T03:42:38.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.503+0000 7fd82d60cf40 1 -- 172.21.15.38:0/57432031 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:54 13.1c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e119) -- 0x5625432020b0 con 0x5625431ff3b0 2024-09-21T03:42:38.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.504+0000 7fd825b5a640 1 -- 172.21.15.38:0/57432031 <== osd.0 v2:172.21.15.38:6800/2381639433 2 ==== osd_op_reply(54 607.00000000 [delete] v119'136 uv136 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd8100a1d80 con 0x5625431ff3b0 2024-09-21T03:42:38.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.505+0000 7fd82d60cf40 1 -- 172.21.15.38:0/57432031 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:55 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x562543202450 con 0x5625431ff3b0 2024-09-21T03:42:38.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.506+0000 7fd825b5a640 1 -- 172.21.15.38:0/57432031 <== osd.0 v2:172.21.15.38:6800/2381639433 3 ==== osd_op_reply(55 607.00000000 [create] v119'137 uv137 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd8100a1d80 con 0x5625431ff3b0 2024-09-21T03:42:38.507 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-09-21T03:42:38.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.506+0000 7fd82d60cf40 1 -- 172.21.15.38:0/57432031 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:56 13.1c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x562543203860 con 0x5625431ff3b0 2024-09-21T03:42:38.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.507+0000 7fd825b5a640 1 -- 172.21.15.38:0/57432031 <== osd.0 v2:172.21.15.38:6800/2381639433 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v119'138 uv138 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd8100a1d80 con 0x5625431ff3b0 2024-09-21T03:42:38.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.507+0000 7fd82d60cf40 1 -- 172.21.15.38:0/57432031 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:57 13.1c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x562543203ce0 con 0x5625431ff3b0 2024-09-21T03:42:38.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.508+0000 7fd825b5a640 1 -- 172.21.15.38:0/57432031 <== osd.0 v2:172.21.15.38:6800/2381639433 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v119'139 uv139 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd8100a1d80 con 0x5625431ff3b0 2024-09-21T03:42:38.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.508+0000 7fd82d60cf40 1 -- 172.21.15.38:0/57432031 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:58 13.1c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x5625432027f0 con 0x5625431ff3b0 2024-09-21T03:42:38.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.509+0000 7fd825b5a640 1 -- 172.21.15.38:0/57432031 <== osd.0 v2:172.21.15.38:6800/2381639433 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v119'140 uv140 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd8100a1d80 con 0x5625431ff3b0 2024-09-21T03:42:38.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.509+0000 7fd82d60cf40 1 --2- 172.21.15.38:0/57432031 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x562543203260 0x5625432060e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:38.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.509+0000 7fd82d60cf40 1 -- 172.21.15.38:0/57432031 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:59 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x562543206640 con 0x562543203260 2024-09-21T03:42:38.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.509+0000 7fd826b5c640 1 --2- 172.21.15.38:0/57432031 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x562543203260 0x5625432060e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:38.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.509+0000 7fd826b5c640 1 --2- 172.21.15.38:0/57432031 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x562543203260 0x5625432060e0 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:38.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.510+0000 7fd826b5c640 1 -- 172.21.15.38:0/57432031 <== osd.3 v2:172.21.15.38:6808/1286810294 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v119'98 uv97 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7fd82000b040 con 0x562543203260 2024-09-21T03:42:38.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.510+0000 7fd82d60cf40 1 -- 172.21.15.38:0/57432031 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:60 13.6 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e119) -- 0x562543206a30 con 0x562543203260 2024-09-21T03:42:38.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.512+0000 7fd826b5c640 1 -- 172.21.15.38:0/57432031 <== osd.3 v2:172.21.15.38:6808/1286810294 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v119'99 uv99 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fd82000b040 con 0x562543203260 2024-09-21T03:42:38.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.512+0000 7fd82d60cf40 1 -- 172.21.15.38:0/57432031 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:61 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x562543206a30 con 0x562543203260 2024-09-21T03:42:38.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.513+0000 7fd826b5c640 1 -- 172.21.15.38:0/57432031 <== osd.3 v2:172.21.15.38:6808/1286810294 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v119'100 uv100 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fd82000b040 con 0x562543203260 2024-09-21T03:42:38.515 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-09-21T03:42:38.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.513+0000 7fd82d60cf40 1 -- 172.21.15.38:0/57432031 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:62 13.6 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x562543206a30 con 0x562543203260 2024-09-21T03:42:38.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.514+0000 7fd826b5c640 1 -- 172.21.15.38:0/57432031 <== osd.3 v2:172.21.15.38:6808/1286810294 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v119'101 uv101 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fd82000b040 con 0x562543203260 2024-09-21T03:42:38.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.514+0000 7fd82d60cf40 1 -- 172.21.15.38:0/57432031 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x562543206a30 con 0x5625431ff3b0 2024-09-21T03:42:38.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.515+0000 7fd825b5a640 1 -- 172.21.15.38:0/57432031 <== osd.0 v2:172.21.15.38:6800/2381639433 7 ==== osd_op_reply(63 608.00000000 [create] v119'135 uv134 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fd8100a1d80 con 0x5625431ff3b0 2024-09-21T03:42:38.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.515+0000 7fd82d60cf40 1 -- 172.21.15.38:0/57432031 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e119) -- 0x562543206a30 con 0x5625431ff3b0 2024-09-21T03:42:38.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.516+0000 7fd825b5a640 1 -- 172.21.15.38:0/57432031 <== osd.0 v2:172.21.15.38:6800/2381639433 8 ==== osd_op_reply(64 608.00000000 [delete] v119'136 uv136 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd8100a1d80 con 0x5625431ff3b0 2024-09-21T03:42:38.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.516+0000 7fd82d60cf40 1 -- 172.21.15.38:0/57432031 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x562543206a30 con 0x5625431ff3b0 2024-09-21T03:42:38.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.518+0000 7fd825b5a640 1 -- 172.21.15.38:0/57432031 <== osd.0 v2:172.21.15.38:6800/2381639433 9 ==== osd_op_reply(65 608.00000000 [create] v119'137 uv137 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd8100a1d80 con 0x5625431ff3b0 2024-09-21T03:42:38.519 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-09-21T03:42:38.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.518+0000 7fd82d60cf40 1 -- 172.21.15.38:0/57432031 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x562543206a30 con 0x5625431ff3b0 2024-09-21T03:42:38.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.519+0000 7fd825b5a640 1 -- 172.21.15.38:0/57432031 <== osd.0 v2:172.21.15.38:6800/2381639433 10 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v119'138 uv138 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd8100a1d80 con 0x5625431ff3b0 2024-09-21T03:42:38.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.519+0000 7fd82d60cf40 1 -- 172.21.15.38:0/57432031 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x562543206a30 con 0x5625431ff3b0 2024-09-21T03:42:38.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.520+0000 7fd825b5a640 1 -- 172.21.15.38:0/57432031 <== osd.0 v2:172.21.15.38:6800/2381639433 11 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v119'139 uv139 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd8100a1d80 con 0x5625431ff3b0 2024-09-21T03:42:38.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.520+0000 7fd82d60cf40 1 -- 172.21.15.38:0/57432031 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x562543206a30 con 0x5625431ff3b0 2024-09-21T03:42:38.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.521+0000 7fd825b5a640 1 -- 172.21.15.38:0/57432031 <== osd.0 v2:172.21.15.38:6800/2381639433 12 ==== osd_op_reply(68 608.00000000 [omap-set-header] v119'140 uv140 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd8100a1d80 con 0x5625431ff3b0 2024-09-21T03:42:38.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.521+0000 7fd82d60cf40 1 -- 172.21.15.38:0/57432031 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x562543206a30 con 0x562543203260 2024-09-21T03:42:38.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.522+0000 7fd826b5c640 1 -- 172.21.15.38:0/57432031 <== osd.3 v2:172.21.15.38:6808/1286810294 5 ==== osd_op_reply(69 604.00000000 [create] v119'249 uv243 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fd82000b040 con 0x562543203260 2024-09-21T03:42:38.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.522+0000 7fd82d60cf40 1 -- 172.21.15.38:0/57432031 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e119) -- 0x562543206a30 con 0x562543203260 2024-09-21T03:42:38.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.524+0000 7fd826b5c640 1 -- 172.21.15.38:0/57432031 <== osd.3 v2:172.21.15.38:6808/1286810294 6 ==== osd_op_reply(70 604.00000000 [delete] v119'250 uv250 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd82000b040 con 0x562543203260 2024-09-21T03:42:38.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.524+0000 7fd82d60cf40 1 -- 172.21.15.38:0/57432031 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x562543206a30 con 0x562543203260 2024-09-21T03:42:38.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.525+0000 7fd826b5c640 1 -- 172.21.15.38:0/57432031 <== osd.3 v2:172.21.15.38:6808/1286810294 7 ==== osd_op_reply(71 604.00000000 [create] v119'251 uv251 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd82000b040 con 0x562543203260 2024-09-21T03:42:38.527 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-09-21T03:42:38.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.525+0000 7fd82d60cf40 1 -- 172.21.15.38:0/57432031 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x562543207e30 con 0x562543203260 2024-09-21T03:42:38.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.526+0000 7fd826b5c640 1 -- 172.21.15.38:0/57432031 <== osd.3 v2:172.21.15.38:6808/1286810294 8 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v119'252 uv252 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd82000b040 con 0x562543203260 2024-09-21T03:42:38.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.526+0000 7fd82d60cf40 1 -- 172.21.15.38:0/57432031 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x562543207e30 con 0x562543203260 2024-09-21T03:42:38.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.527+0000 7fd826b5c640 1 -- 172.21.15.38:0/57432031 <== osd.3 v2:172.21.15.38:6808/1286810294 9 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v119'253 uv253 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd82000b040 con 0x562543203260 2024-09-21T03:42:38.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.527+0000 7fd82d60cf40 1 -- 172.21.15.38:0/57432031 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x562543206b40 con 0x562543203260 2024-09-21T03:42:38.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.528+0000 7fd826b5c640 1 -- 172.21.15.38:0/57432031 <== osd.3 v2:172.21.15.38:6808/1286810294 10 ==== osd_op_reply(74 604.00000000 [omap-set-header] v119'254 uv254 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd82000b040 con 0x562543203260 2024-09-21T03:42:38.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.528+0000 7fd82d60cf40 1 -- 172.21.15.38:0/57432031 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x562543206af0 con 0x562543203260 2024-09-21T03:42:38.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.529+0000 7fd826b5c640 1 -- 172.21.15.38:0/57432031 <== osd.3 v2:172.21.15.38:6808/1286810294 11 ==== osd_op_reply(75 500.00000000 [create] v119'255 uv248 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fd82000b040 con 0x562543203260 2024-09-21T03:42:38.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.529+0000 7fd82d60cf40 1 -- 172.21.15.38:0/57432031 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e119) -- 0x562543206ad0 con 0x562543203260 2024-09-21T03:42:38.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.530+0000 7fd826b5c640 1 -- 172.21.15.38:0/57432031 <== osd.3 v2:172.21.15.38:6808/1286810294 12 ==== osd_op_reply(76 500.00000000 [delete] v119'256 uv256 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd82000b040 con 0x562543203260 2024-09-21T03:42:38.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.530+0000 7fd82d60cf40 1 -- 172.21.15.38:0/57432031 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x562543206ad0 con 0x562543203260 2024-09-21T03:42:38.533 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-09-21T03:42:38.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.531+0000 7fd826b5c640 1 -- 172.21.15.38:0/57432031 <== osd.3 v2:172.21.15.38:6808/1286810294 13 ==== osd_op_reply(77 500.00000000 [create] v119'257 uv257 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd82000b040 con 0x562543203260 2024-09-21T03:42:38.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.531+0000 7fd82d60cf40 1 -- 172.21.15.38:0/57432031 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x562543206ad0 con 0x562543203260 2024-09-21T03:42:38.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.532+0000 7fd826b5c640 1 -- 172.21.15.38:0/57432031 <== osd.3 v2:172.21.15.38:6808/1286810294 14 ==== osd_op_reply(78 500.00000000 [write 0~90] v119'258 uv258 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd82000b040 con 0x562543203260 2024-09-21T03:42:38.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.532+0000 7fd82d60cf40 1 -- 172.21.15.38:0/57432031 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x562543206ad0 con 0x562543203260 2024-09-21T03:42:38.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.533+0000 7fd826b5c640 1 -- 172.21.15.38:0/57432031 <== osd.3 v2:172.21.15.38:6808/1286810294 15 ==== osd_op_reply(79 500.00000000 [omap-set-header] v119'259 uv259 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd82000b040 con 0x562543203260 2024-09-21T03:42:38.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.534+0000 7fd82d60cf40 1 -- 172.21.15.38:0/57432031 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:80 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x562543206ad0 con 0x5625431ff3b0 2024-09-21T03:42:38.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.535+0000 7fd825b5a640 1 -- 172.21.15.38:0/57432031 <== osd.0 v2:172.21.15.38:6800/2381639433 13 ==== osd_op_reply(80 mds_snaptable [create] v119'112 uv111 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7fd8100a1d80 con 0x5625431ff3b0 2024-09-21T03:42:38.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.535+0000 7fd82d60cf40 1 -- 172.21.15.38:0/57432031 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:81 13.d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e119) -- 0x562543206ad0 con 0x5625431ff3b0 2024-09-21T03:42:38.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.536+0000 7fd825b5a640 1 -- 172.21.15.38:0/57432031 <== osd.0 v2:172.21.15.38:6800/2381639433 14 ==== osd_op_reply(81 mds_snaptable [delete] v119'113 uv113 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fd8100a1d80 con 0x5625431ff3b0 2024-09-21T03:42:38.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.536+0000 7fd82d60cf40 1 -- 172.21.15.38:0/57432031 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:82 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x562543206ad0 con 0x5625431ff3b0 2024-09-21T03:42:38.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.537+0000 7fd825b5a640 1 -- 172.21.15.38:0/57432031 <== osd.0 v2:172.21.15.38:6800/2381639433 15 ==== osd_op_reply(82 mds_snaptable [create] v119'114 uv114 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fd8100a1d80 con 0x5625431ff3b0 2024-09-21T03:42:38.539 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-09-21T03:42:38.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.537+0000 7fd82d60cf40 1 -- 172.21.15.38:0/57432031 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:83 13.d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x562543206ad0 con 0x5625431ff3b0 2024-09-21T03:42:38.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.538+0000 7fd825b5a640 1 -- 172.21.15.38:0/57432031 <== osd.0 v2:172.21.15.38:6800/2381639433 16 ==== osd_op_reply(83 mds_snaptable [write 0~70] v119'115 uv115 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fd8100a1d80 con 0x5625431ff3b0 2024-09-21T03:42:38.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.538+0000 7fd82d60cf40 1 -- 172.21.15.38:0/57432031 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:84 13.d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x562543206ad0 con 0x5625431ff3b0 2024-09-21T03:42:38.541 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.539+0000 7fd825b5a640 1 -- 172.21.15.38:0/57432031 <== osd.0 v2:172.21.15.38:6800/2381639433 17 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v119'116 uv116 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fd8100a1d80 con 0x5625431ff3b0 2024-09-21T03:42:38.541 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.539+0000 7fd82d60cf40 1 --2- 172.21.15.38:0/57432031 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x5625432073f0 0x562543209810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:38.541 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.539+0000 7fd82d60cf40 1 -- 172.21.15.38:0/57432031 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x562543209d50 con 0x5625432073f0 2024-09-21T03:42:38.541 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.539+0000 7fd82635b640 1 --2- 172.21.15.38:0/57432031 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x5625432073f0 0x562543209810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:38.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.540+0000 7fd82635b640 1 --2- 172.21.15.38:0/57432031 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x5625432073f0 0x562543209810 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:38.543 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.541+0000 7fd82635b640 1 -- 172.21.15.38:0/57432031 <== osd.1 v2:172.21.15.38:6816/1491818732 1 ==== osd_op_reply(85 605.00000000 [create] v119'135 uv134 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fd818003040 con 0x5625432073f0 2024-09-21T03:42:38.543 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.541+0000 7fd82d60cf40 1 -- 172.21.15.38:0/57432031 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e119) -- 0x562543209db0 con 0x5625432073f0 2024-09-21T03:42:38.544 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.542+0000 7fd82635b640 1 -- 172.21.15.38:0/57432031 <== osd.1 v2:172.21.15.38:6816/1491818732 2 ==== osd_op_reply(86 605.00000000 [delete] v119'136 uv136 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd818003040 con 0x5625432073f0 2024-09-21T03:42:38.544 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.542+0000 7fd82d60cf40 1 -- 172.21.15.38:0/57432031 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x56254320a1d0 con 0x5625432073f0 2024-09-21T03:42:38.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.543+0000 7fd82635b640 1 -- 172.21.15.38:0/57432031 <== osd.1 v2:172.21.15.38:6816/1491818732 3 ==== osd_op_reply(87 605.00000000 [create] v119'137 uv137 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd818003040 con 0x5625432073f0 2024-09-21T03:42:38.545 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-09-21T03:42:38.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.543+0000 7fd82d60cf40 1 -- 172.21.15.38:0/57432031 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x56254320b1e0 con 0x5625432073f0 2024-09-21T03:42:38.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.545+0000 7fd82635b640 1 -- 172.21.15.38:0/57432031 <== osd.1 v2:172.21.15.38:6816/1491818732 4 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v119'138 uv138 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd818003040 con 0x5625432073f0 2024-09-21T03:42:38.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.545+0000 7fd82d60cf40 1 -- 172.21.15.38:0/57432031 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x56254320b1e0 con 0x5625432073f0 2024-09-21T03:42:38.547 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.546+0000 7fd82635b640 1 -- 172.21.15.38:0/57432031 <== osd.1 v2:172.21.15.38:6816/1491818732 5 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v119'139 uv139 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd818003040 con 0x5625432073f0 2024-09-21T03:42:38.547 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.546+0000 7fd82d60cf40 1 -- 172.21.15.38:0/57432031 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x56254320a1d0 con 0x5625432073f0 2024-09-21T03:42:38.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.547+0000 7fd82635b640 1 -- 172.21.15.38:0/57432031 <== osd.1 v2:172.21.15.38:6816/1491818732 6 ==== osd_op_reply(90 605.00000000 [omap-set-header] v119'140 uv140 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd818003040 con 0x5625432073f0 2024-09-21T03:42:38.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.547+0000 7fd82d60cf40 1 -- 172.21.15.38:0/57432031 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x56254320a1d0 con 0x5625432073f0 2024-09-21T03:42:38.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.548+0000 7fd82635b640 1 -- 172.21.15.38:0/57432031 <== osd.1 v2:172.21.15.38:6816/1491818732 7 ==== osd_op_reply(91 mds0_inotable [create] v119'223 uv217 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7fd818003040 con 0x5625432073f0 2024-09-21T03:42:38.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.548+0000 7fd82d60cf40 1 -- 172.21.15.38:0/57432031 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e119) -- 0x56254320a1d0 con 0x5625432073f0 2024-09-21T03:42:38.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.549+0000 7fd82635b640 1 -- 172.21.15.38:0/57432031 <== osd.1 v2:172.21.15.38:6816/1491818732 8 ==== osd_op_reply(92 mds0_inotable [delete] v119'224 uv224 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fd818003040 con 0x5625432073f0 2024-09-21T03:42:38.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.549+0000 7fd82d60cf40 1 -- 172.21.15.38:0/57432031 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x56254320a1d0 con 0x5625432073f0 2024-09-21T03:42:38.552 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-09-21T03:42:38.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.551+0000 7fd82635b640 1 -- 172.21.15.38:0/57432031 <== osd.1 v2:172.21.15.38:6816/1491818732 9 ==== osd_op_reply(93 mds0_inotable [create] v119'225 uv225 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fd818003040 con 0x5625432073f0 2024-09-21T03:42:38.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.551+0000 7fd82d60cf40 1 -- 172.21.15.38:0/57432031 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:94 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x56254320a1d0 con 0x5625432073f0 2024-09-21T03:42:38.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.552+0000 7fd82635b640 1 -- 172.21.15.38:0/57432031 <== osd.1 v2:172.21.15.38:6816/1491818732 10 ==== osd_op_reply(94 mds0_inotable [write 0~34] v119'226 uv226 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fd818003040 con 0x5625432073f0 2024-09-21T03:42:38.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.552+0000 7fd82d60cf40 1 -- 172.21.15.38:0/57432031 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:95 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x56254320a1d0 con 0x5625432073f0 2024-09-21T03:42:38.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.553+0000 7fd82635b640 1 -- 172.21.15.38:0/57432031 <== osd.1 v2:172.21.15.38:6816/1491818732 11 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v119'227 uv227 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fd818003040 con 0x5625432073f0 2024-09-21T03:42:38.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.553+0000 7fd82d60cf40 1 -- 172.21.15.38:0/57432031 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:96 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x56254320a1d0 con 0x5625432073f0 2024-09-21T03:42:38.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.554+0000 7fd82635b640 1 -- 172.21.15.38:0/57432031 <== osd.1 v2:172.21.15.38:6816/1491818732 12 ==== osd_op_reply(96 100.00000000 [create] v119'228 uv222 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fd818003040 con 0x5625432073f0 2024-09-21T03:42:38.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.554+0000 7fd82d60cf40 1 -- 172.21.15.38:0/57432031 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:97 13.13 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e119) -- 0x56254320a1d0 con 0x5625432073f0 2024-09-21T03:42:38.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.555+0000 7fd82635b640 1 -- 172.21.15.38:0/57432031 <== osd.1 v2:172.21.15.38:6816/1491818732 13 ==== osd_op_reply(97 100.00000000 [delete] v119'229 uv229 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd818003040 con 0x5625432073f0 2024-09-21T03:42:38.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.555+0000 7fd82d60cf40 1 -- 172.21.15.38:0/57432031 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:98 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x56254320a1d0 con 0x5625432073f0 2024-09-21T03:42:38.557 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-09-21T03:42:38.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.556+0000 7fd82635b640 1 -- 172.21.15.38:0/57432031 <== osd.1 v2:172.21.15.38:6816/1491818732 14 ==== osd_op_reply(98 100.00000000 [create] v119'230 uv230 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd818003040 con 0x5625432073f0 2024-09-21T03:42:38.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.556+0000 7fd82d60cf40 1 -- 172.21.15.38:0/57432031 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:99 13.13 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x56254320a1d0 con 0x5625432073f0 2024-09-21T03:42:38.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.557+0000 7fd82635b640 1 -- 172.21.15.38:0/57432031 <== osd.1 v2:172.21.15.38:6816/1491818732 15 ==== osd_op_reply(99 100.00000000 [omap-set-header] v119'231 uv231 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd818003040 con 0x5625432073f0 2024-09-21T03:42:38.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.557+0000 7fd82d60cf40 1 -- 172.21.15.38:0/57432031 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:100 13.13 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x5625431fdc40 con 0x5625432073f0 2024-09-21T03:42:38.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.558+0000 7fd82635b640 1 -- 172.21.15.38:0/57432031 <== osd.1 v2:172.21.15.38:6816/1491818732 16 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v119'232 uv232 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd818003040 con 0x5625432073f0 2024-09-21T03:42:38.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.558+0000 7fd82d60cf40 1 -- 172.21.15.38:0/57432031 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x5625431fdc00 con 0x562543203260 2024-09-21T03:42:38.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.559+0000 7fd826b5c640 1 -- 172.21.15.38:0/57432031 <== osd.3 v2:172.21.15.38:6808/1286810294 16 ==== osd_op_reply(101 mds0_sessionmap [create] v119'99 uv98 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7fd82000b040 con 0x562543203260 2024-09-21T03:42:38.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.559+0000 7fd82d60cf40 1 -- 172.21.15.38:0/57432031 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e119) -- 0x5625431fdc00 con 0x562543203260 2024-09-21T03:42:38.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.560+0000 7fd826b5c640 1 -- 172.21.15.38:0/57432031 <== osd.3 v2:172.21.15.38:6808/1286810294 17 ==== osd_op_reply(102 mds0_sessionmap [delete] v119'100 uv100 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fd82000b040 con 0x562543203260 2024-09-21T03:42:38.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.560+0000 7fd82d60cf40 1 -- 172.21.15.38:0/57432031 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:103 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x5625431fdc00 con 0x562543203260 2024-09-21T03:42:38.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.561+0000 7fd826b5c640 1 -- 172.21.15.38:0/57432031 <== osd.3 v2:172.21.15.38:6808/1286810294 18 ==== osd_op_reply(103 mds0_sessionmap [create] v119'101 uv101 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fd82000b040 con 0x562543203260 2024-09-21T03:42:38.563 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-09-21T03:42:38.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.561+0000 7fd82d60cf40 1 -- 172.21.15.38:0/57432031 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:104 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x5625431fdc00 con 0x562543203260 2024-09-21T03:42:38.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.563+0000 7fd826b5c640 1 -- 172.21.15.38:0/57432031 <== osd.3 v2:172.21.15.38:6808/1286810294 19 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v119'102 uv102 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fd82000b040 con 0x562543203260 2024-09-21T03:42:38.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.563+0000 7fd82d60cf40 1 -- 172.21.15.38:0/57432031 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x5625431fdc00 con 0x562543203260 2024-09-21T03:42:38.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.564+0000 7fd826b5c640 1 -- 172.21.15.38:0/57432031 <== osd.3 v2:172.21.15.38:6808/1286810294 20 ==== osd_op_reply(105 609.00000000 [create] v119'135 uv134 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fd82000b040 con 0x562543203260 2024-09-21T03:42:38.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.564+0000 7fd82d60cf40 1 -- 172.21.15.38:0/57432031 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e119) -- 0x5625431fdc00 con 0x562543203260 2024-09-21T03:42:38.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.565+0000 7fd826b5c640 1 -- 172.21.15.38:0/57432031 <== osd.3 v2:172.21.15.38:6808/1286810294 21 ==== osd_op_reply(106 609.00000000 [delete] v119'136 uv136 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd82000b040 con 0x562543203260 2024-09-21T03:42:38.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.565+0000 7fd82d60cf40 1 -- 172.21.15.38:0/57432031 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x5625431fdc00 con 0x562543203260 2024-09-21T03:42:38.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.566+0000 7fd826b5c640 1 -- 172.21.15.38:0/57432031 <== osd.3 v2:172.21.15.38:6808/1286810294 22 ==== osd_op_reply(107 609.00000000 [create] v119'137 uv137 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd82000b040 con 0x562543203260 2024-09-21T03:42:38.568 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-09-21T03:42:38.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.566+0000 7fd82d60cf40 1 -- 172.21.15.38:0/57432031 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x5625431fdc00 con 0x562543203260 2024-09-21T03:42:38.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.567+0000 7fd826b5c640 1 -- 172.21.15.38:0/57432031 <== osd.3 v2:172.21.15.38:6808/1286810294 23 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v119'138 uv138 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd82000b040 con 0x562543203260 2024-09-21T03:42:38.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.567+0000 7fd82d60cf40 1 -- 172.21.15.38:0/57432031 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:109 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x5625431fdc00 con 0x562543203260 2024-09-21T03:42:38.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.568+0000 7fd826b5c640 1 -- 172.21.15.38:0/57432031 <== osd.3 v2:172.21.15.38:6808/1286810294 24 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v119'139 uv139 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd82000b040 con 0x562543203260 2024-09-21T03:42:38.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.568+0000 7fd82d60cf40 1 -- 172.21.15.38:0/57432031 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:110 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x5625431fdc00 con 0x562543203260 2024-09-21T03:42:38.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.569+0000 7fd826b5c640 1 -- 172.21.15.38:0/57432031 <== osd.3 v2:172.21.15.38:6808/1286810294 25 ==== osd_op_reply(110 609.00000000 [omap-set-header] v119'140 uv140 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd82000b040 con 0x562543203260 2024-09-21T03:42:38.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.569+0000 7fd82d60cf40 1 --2- 172.21.15.38:0/57432031 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x56254320af80 0x56254320d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:38.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.569+0000 7fd82d60cf40 1 -- 172.21.15.38:0/57432031 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x56254320dbf0 con 0x56254320af80 2024-09-21T03:42:38.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.569+0000 7fd825b5a640 1 --2- 172.21.15.38:0/57432031 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x56254320af80 0x56254320d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:38.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.570+0000 7fd825b5a640 1 --2- 172.21.15.38:0/57432031 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x56254320af80 0x56254320d6b0 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:38.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.571+0000 7fd825b5a640 1 -- 172.21.15.38:0/57432031 <== osd.2 v2:172.21.15.38:6824/2577858902 1 ==== osd_op_reply(111 400.00000000 [create] v119'115 uv114 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fd8100a1d80 con 0x56254320af80 2024-09-21T03:42:38.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.571+0000 7fd82d60cf40 1 -- 172.21.15.38:0/57432031 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e119) -- 0x56254320dbf0 con 0x56254320af80 2024-09-21T03:42:38.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.572+0000 7fd825b5a640 1 -- 172.21.15.38:0/57432031 <== osd.2 v2:172.21.15.38:6824/2577858902 2 ==== osd_op_reply(112 400.00000000 [delete] v119'116 uv116 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd8100a1d80 con 0x56254320af80 2024-09-21T03:42:38.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.572+0000 7fd82d60cf40 1 -- 172.21.15.38:0/57432031 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x56254320dbf0 con 0x56254320af80 2024-09-21T03:42:38.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.573+0000 7fd825b5a640 1 -- 172.21.15.38:0/57432031 <== osd.2 v2:172.21.15.38:6824/2577858902 3 ==== osd_op_reply(113 400.00000000 [create] v119'117 uv117 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd8100a1d80 con 0x56254320af80 2024-09-21T03:42:38.575 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-09-21T03:42:38.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.573+0000 7fd82d60cf40 1 -- 172.21.15.38:0/57432031 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:114 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x56254320dbf0 con 0x56254320af80 2024-09-21T03:42:38.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.574+0000 7fd825b5a640 1 -- 172.21.15.38:0/57432031 <== osd.2 v2:172.21.15.38:6824/2577858902 4 ==== osd_op_reply(114 400.00000000 [write 0~22] v119'118 uv118 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd8100a1d80 con 0x56254320af80 2024-09-21T03:42:38.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.574+0000 7fd82d60cf40 1 -- 172.21.15.38:0/57432031 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:115 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x56254320dbf0 con 0x56254320af80 2024-09-21T03:42:38.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.575+0000 7fd825b5a640 1 -- 172.21.15.38:0/57432031 <== osd.2 v2:172.21.15.38:6824/2577858902 5 ==== osd_op_reply(115 400.00000000 [omap-set-header] v119'119 uv119 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd8100a1d80 con 0x56254320af80 2024-09-21T03:42:38.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.575+0000 7fd82d60cf40 1 -- 172.21.15.38:0/57432031 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:116 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x56254320dbf0 con 0x5625432073f0 2024-09-21T03:42:38.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.577+0000 7fd82635b640 1 -- 172.21.15.38:0/57432031 <== osd.1 v2:172.21.15.38:6816/1491818732 17 ==== osd_op_reply(116 100.00000000.inode [create] v119'278 uv277 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7fd818003040 con 0x5625432073f0 2024-09-21T03:42:38.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.577+0000 7fd82d60cf40 1 -- 172.21.15.38:0/57432031 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:117 13.1f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e119) -- 0x56254320dbf0 con 0x5625432073f0 2024-09-21T03:42:38.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.578+0000 7fd82635b640 1 -- 172.21.15.38:0/57432031 <== osd.1 v2:172.21.15.38:6816/1491818732 18 ==== osd_op_reply(117 100.00000000.inode [delete] v119'279 uv279 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fd818003040 con 0x5625432073f0 2024-09-21T03:42:38.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.578+0000 7fd82d60cf40 1 -- 172.21.15.38:0/57432031 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:118 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x56254320dbf0 con 0x5625432073f0 2024-09-21T03:42:38.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.579+0000 7fd82635b640 1 -- 172.21.15.38:0/57432031 <== osd.1 v2:172.21.15.38:6816/1491818732 19 ==== osd_op_reply(118 100.00000000.inode [create] v119'280 uv280 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fd818003040 con 0x5625432073f0 2024-09-21T03:42:38.581 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-09-21T03:42:38.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.579+0000 7fd82d60cf40 1 -- 172.21.15.38:0/57432031 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:119 13.1f 13:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x56254320dbf0 con 0x5625432073f0 2024-09-21T03:42:38.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.580+0000 7fd82635b640 1 -- 172.21.15.38:0/57432031 <== osd.1 v2:172.21.15.38:6816/1491818732 20 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v119'281 uv281 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fd818003040 con 0x5625432073f0 2024-09-21T03:42:38.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.580+0000 7fd82d60cf40 1 -- 172.21.15.38:0/57432031 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:120 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x56254320dbf0 con 0x5625432073f0 2024-09-21T03:42:38.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.581+0000 7fd82635b640 1 -- 172.21.15.38:0/57432031 <== osd.1 v2:172.21.15.38:6816/1491818732 21 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v119'282 uv282 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fd818003040 con 0x5625432073f0 2024-09-21T03:42:38.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.582+0000 7fd82d60cf40 1 -- 172.21.15.38:0/57432031 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:121 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x56254320dbf0 con 0x5625432073f0 2024-09-21T03:42:38.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.582+0000 7fd82635b640 1 -- 172.21.15.38:0/57432031 <== osd.1 v2:172.21.15.38:6816/1491818732 22 ==== osd_op_reply(121 1.00000000 [create] v119'283 uv276 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7fd818003040 con 0x5625432073f0 2024-09-21T03:42:38.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.583+0000 7fd82d60cf40 1 -- 172.21.15.38:0/57432031 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:122 13.1f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e119) -- 0x56254320dbf0 con 0x5625432073f0 2024-09-21T03:42:38.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.584+0000 7fd82635b640 1 -- 172.21.15.38:0/57432031 <== osd.1 v2:172.21.15.38:6816/1491818732 23 ==== osd_op_reply(122 1.00000000 [delete] v119'284 uv284 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fd818003040 con 0x5625432073f0 2024-09-21T03:42:38.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.584+0000 7fd82d60cf40 1 -- 172.21.15.38:0/57432031 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:123 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x56254320dbf0 con 0x5625432073f0 2024-09-21T03:42:38.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.584+0000 7fd82635b640 1 -- 172.21.15.38:0/57432031 <== osd.1 v2:172.21.15.38:6816/1491818732 24 ==== osd_op_reply(123 1.00000000 [create] v119'285 uv285 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fd818003040 con 0x5625432073f0 2024-09-21T03:42:38.586 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-09-21T03:42:38.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.585+0000 7fd82d60cf40 1 -- 172.21.15.38:0/57432031 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:124 13.1f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x56254320ec00 con 0x5625432073f0 2024-09-21T03:42:38.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.586+0000 7fd82635b640 1 -- 172.21.15.38:0/57432031 <== osd.1 v2:172.21.15.38:6816/1491818732 25 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v119'286 uv286 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fd818003040 con 0x5625432073f0 2024-09-21T03:42:38.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.586+0000 7fd82d60cf40 1 -- 172.21.15.38:0/57432031 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:125 13.1f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x56254320ec00 con 0x5625432073f0 2024-09-21T03:42:38.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.587+0000 7fd82635b640 1 -- 172.21.15.38:0/57432031 <== osd.1 v2:172.21.15.38:6816/1491818732 26 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v119'287 uv287 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fd818003040 con 0x5625432073f0 2024-09-21T03:42:38.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.587+0000 7fd82d60cf40 1 -- 172.21.15.38:0/57432031 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:126 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x56254320dbf0 con 0x5625432073f0 2024-09-21T03:42:38.589 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.588+0000 7fd82635b640 1 -- 172.21.15.38:0/57432031 <== osd.1 v2:172.21.15.38:6816/1491818732 27 ==== osd_op_reply(126 1.00000000 [omap-set-header] v119'288 uv288 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fd818003040 con 0x5625432073f0 2024-09-21T03:42:38.589 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.588+0000 7fd82d60cf40 1 -- 172.21.15.38:0/57432031 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:127 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=584b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x56254320dbf0 con 0x5625432073f0 2024-09-21T03:42:38.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.591+0000 7fd82635b640 1 -- 172.21.15.38:0/57432031 <== osd.1 v2:172.21.15.38:6816/1491818732 28 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v119'289 uv289 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fd818003040 con 0x5625432073f0 2024-09-21T03:42:38.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.591+0000 7fd82d60cf40 1 -- 172.21.15.38:0/57432031 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x5625431ff3b0 msgr2=0x5625432017d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:38.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.591+0000 7fd82d60cf40 1 --2- 172.21.15.38:0/57432031 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x5625431ff3b0 0x5625432017d0 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:38.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.591+0000 7fd82d60cf40 1 -- 172.21.15.38:0/57432031 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x5625432073f0 msgr2=0x562543209810 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:38.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.591+0000 7fd82d60cf40 1 --2- 172.21.15.38:0/57432031 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x5625432073f0 0x562543209810 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:38.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.591+0000 7fd82d60cf40 1 -- 172.21.15.38:0/57432031 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x56254320af80 msgr2=0x56254320d6b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:38.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.591+0000 7fd82d60cf40 1 --2- 172.21.15.38:0/57432031 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x56254320af80 0x56254320d6b0 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:38.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.591+0000 7fd82d60cf40 1 -- 172.21.15.38:0/57432031 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x562543203260 msgr2=0x5625432060e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:38.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.591+0000 7fd82d60cf40 1 --2- 172.21.15.38:0/57432031 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x562543203260 0x5625432060e0 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:38.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.591+0000 7fd82d60cf40 1 -- 172.21.15.38:0/57432031 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x5625431fa730 msgr2=0x5625431fcb50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:38.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.591+0000 7fd826b5c640 1 -- 172.21.15.38:0/57432031 reap_dead start 2024-09-21T03:42:38.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.591+0000 7fd82d60cf40 1 --2- 172.21.15.38:0/57432031 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x5625431fa730 0x5625431fcb50 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:38.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.591+0000 7fd826b5c640 1 -- 172.21.15.38:0/57432031 reap_dead start 2024-09-21T03:42:38.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.591+0000 7fd82d60cf40 1 -- 172.21.15.38:0/57432031 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x56254314df60 msgr2=0x56254313a5c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:38.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.591+0000 7fd82d60cf40 1 --2- 172.21.15.38:0/57432031 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x56254314df60 0x56254313a5c0 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:38.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.591+0000 7fd82d60cf40 1 -- 172.21.15.38:0/57432031 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd8080775a0 msgr2=0x7fd808079ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:38.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.591+0000 7fd82d60cf40 1 --2- 172.21.15.38:0/57432031 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd8080775a0 0x7fd808079ab0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fd818000f80 tx=0x562543154d80 comp rx=0 tx=0).stop 2024-09-21T03:42:38.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.591+0000 7fd82d60cf40 1 -- 172.21.15.38:0/57432031 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56254314e830 msgr2=0x56254314d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:38.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.591+0000 7fd82d60cf40 1 --2- 172.21.15.38:0/57432031 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56254314e830 0x56254314d830 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7fd810026270 tx=0x7fd810056a70 comp rx=0 tx=0).stop 2024-09-21T03:42:38.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.591+0000 7fd82d60cf40 1 -- 172.21.15.38:0/57432031 shutdown_connections 2024-09-21T03:42:38.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.591+0000 7fd82d60cf40 1 --2- 172.21.15.38:0/57432031 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x56254314df60 0x56254313a5c0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:38.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.591+0000 7fd82d60cf40 1 --2- 172.21.15.38:0/57432031 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd8080775a0 0x7fd808079ab0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:38.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.591+0000 7fd82d60cf40 1 --2- 172.21.15.38:0/57432031 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56254314e830 0x56254314d830 unknown :-1 s=CLOSED pgs=1802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:38.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.591+0000 7fd82d60cf40 1 -- 172.21.15.38:0/57432031 >> 172.21.15.38:0/57432031 conn(0x562542f9fac0 msgr2=0x56254307d1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:38.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.591+0000 7fd82d60cf40 1 -- 172.21.15.38:0/57432031 shutdown_connections 2024-09-21T03:42:38.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.591+0000 7fd82d60cf40 1 -- 172.21.15.38:0/57432031 wait complete. 2024-09-21T03:42:38.596 DEBUG:teuthology.orchestra.run.smithi038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata truncate 1.00000000.inode 0 2024-09-21T03:42:38.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.650+0000 7fd7f7ef1f40 1 Processor -- start 2024-09-21T03:42:38.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.651+0000 7fd7f7ef1f40 1 -- start start 2024-09-21T03:42:38.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.652+0000 7fd7f7ef1f40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5640b6036090 0x5640b60c4180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:38.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.652+0000 7fd7f7ef1f40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5640b60ab520 0x5640b60bc800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:38.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.652+0000 7fd7f7ef1f40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5640b60c46c0 0x5640b60bcd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:38.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.652+0000 7fd7f7ef1f40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5640b5f3cd60 con 0x5640b60ab520 2024-09-21T03:42:38.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.652+0000 7fd7f7ef1f40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5640b5f9b430 con 0x5640b6036090 2024-09-21T03:42:38.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.652+0000 7fd7f7ef1f40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x5640b60c3350 con 0x5640b60c46c0 2024-09-21T03:42:38.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.652+0000 7fd7f11aa640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5640b60ab520 0x5640b60bc800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:38.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.652+0000 7fd7f19ab640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5640b6036090 0x5640b60c4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:38.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.652+0000 7fd7f19ab640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5640b6036090 0x5640b60c4180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.38:48140/0 (socket says 172.21.15.38:48140) 2024-09-21T03:42:38.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.652+0000 7fd7f11aa640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5640b60ab520 0x5640b60bc800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:52284/0 (socket says 172.21.15.38:52284) 2024-09-21T03:42:38.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.652+0000 7fd7f19ab640 1 -- 172.21.15.38:0/2762584143 learned_addr learned my addr 172.21.15.38:0/2762584143 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:42:38.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.652+0000 7fd7f21ac640 1 --2- 172.21.15.38:0/2762584143 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5640b60c46c0 0x5640b60bcd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:38.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.652+0000 7fd7f19ab640 1 -- 172.21.15.38:0/2762584143 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5640b60c46c0 msgr2=0x5640b60bcd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:38.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.652+0000 7fd7f19ab640 1 --2- 172.21.15.38:0/2762584143 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5640b60c46c0 0x5640b60bcd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:38.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.652+0000 7fd7f19ab640 1 -- 172.21.15.38:0/2762584143 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5640b60ab520 msgr2=0x5640b60bc800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:38.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.652+0000 7fd7f19ab640 1 --2- 172.21.15.38:0/2762584143 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5640b60ab520 0x5640b60bc800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:38.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.652+0000 7fd7f19ab640 1 -- 172.21.15.38:0/2762584143 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5640b60c3c20 con 0x5640b6036090 2024-09-21T03:42:38.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.652+0000 7fd7f19ab640 1 --2- 172.21.15.38:0/2762584143 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5640b6036090 0x5640b60c4180 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fd7e4057040 tx=0x7fd7e4003330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a7bc62e76313c6aa server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:38.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.653+0000 7fd7e3fff640 1 -- 172.21.15.38:0/2762584143 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7e4064030 con 0x5640b6036090 2024-09-21T03:42:38.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.653+0000 7fd7e3fff640 1 -- 172.21.15.38:0/2762584143 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd7e4004020 con 0x5640b6036090 2024-09-21T03:42:38.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.653+0000 7fd7e3fff640 1 -- 172.21.15.38:0/2762584143 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7e4004270 con 0x5640b6036090 2024-09-21T03:42:38.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.653+0000 7fd7f7ef1f40 1 -- 172.21.15.38:0/2762584143 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5640b6036090 msgr2=0x5640b60c4180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:38.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.653+0000 7fd7f7ef1f40 1 --2- 172.21.15.38:0/2762584143 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5640b6036090 0x5640b60c4180 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fd7e4057040 tx=0x7fd7e4003330 comp rx=0 tx=0).stop 2024-09-21T03:42:38.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.653+0000 7fd7f7ef1f40 1 -- 172.21.15.38:0/2762584143 shutdown_connections 2024-09-21T03:42:38.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.653+0000 7fd7f7ef1f40 1 --2- 172.21.15.38:0/2762584143 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5640b60c46c0 0x5640b60bcd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:38.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.653+0000 7fd7f7ef1f40 1 --2- 172.21.15.38:0/2762584143 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5640b60ab520 0x5640b60bc800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:38.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.653+0000 7fd7f7ef1f40 1 --2- 172.21.15.38:0/2762584143 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5640b6036090 0x5640b60c4180 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:38.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.653+0000 7fd7f7ef1f40 1 -- 172.21.15.38:0/2762584143 >> 172.21.15.38:0/2762584143 conn(0x5640b5f0dac0 msgr2=0x5640b60347f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:38.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.653+0000 7fd7f7ef1f40 1 -- 172.21.15.38:0/2762584143 shutdown_connections 2024-09-21T03:42:38.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.653+0000 7fd7f7ef1f40 1 -- 172.21.15.38:0/2762584143 wait complete. 2024-09-21T03:42:38.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.654+0000 7fd7f7ef1f40 1 Processor -- start 2024-09-21T03:42:38.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.654+0000 7fd7f7ef1f40 1 -- start start 2024-09-21T03:42:38.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.656+0000 7fd7f7ef1f40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5640b6036090 0x5640b60a5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:38.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.656+0000 7fd7f7ef1f40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5640b60ab520 0x5640b5feee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:38.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.656+0000 7fd7f7ef1f40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5640b60c46c0 0x5640b5fef390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:38.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.656+0000 7fd7f7ef1f40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5640b6159320 con 0x5640b6036090 2024-09-21T03:42:38.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.656+0000 7fd7f7ef1f40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5640b6033b00 con 0x5640b60c46c0 2024-09-21T03:42:38.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.656+0000 7fd7f7ef1f40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x5640b6033540 con 0x5640b60ab520 2024-09-21T03:42:38.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.656+0000 7fd7f19ab640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5640b6036090 0x5640b60a5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:38.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.656+0000 7fd7f19ab640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5640b6036090 0x5640b60a5cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:52294/0 (socket says 172.21.15.38:52294) 2024-09-21T03:42:38.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.656+0000 7fd7f19ab640 1 -- 172.21.15.38:0/323725014 learned_addr learned my addr 172.21.15.38:0/323725014 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:42:38.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.656+0000 7fd7f21ac640 1 --2- 172.21.15.38:0/323725014 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5640b60c46c0 0x5640b5fef390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:38.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.656+0000 7fd7f11aa640 1 --2- 172.21.15.38:0/323725014 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5640b60ab520 0x5640b5feee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:38.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.656+0000 7fd7f19ab640 1 -- 172.21.15.38:0/323725014 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5640b60ab520 msgr2=0x5640b5feee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:38.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.656+0000 7fd7f19ab640 1 --2- 172.21.15.38:0/323725014 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5640b60ab520 0x5640b5feee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:38.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.656+0000 7fd7f19ab640 1 -- 172.21.15.38:0/323725014 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5640b60c46c0 msgr2=0x5640b5fef390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:38.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.656+0000 7fd7f19ab640 1 --2- 172.21.15.38:0/323725014 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5640b60c46c0 0x5640b5fef390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:38.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.656+0000 7fd7f19ab640 1 -- 172.21.15.38:0/323725014 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5640b6033820 con 0x5640b6036090 2024-09-21T03:42:38.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.656+0000 7fd7f19ab640 1 --2- 172.21.15.38:0/323725014 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5640b6036090 0x5640b60a5cd0 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x7fd7e4046530 tx=0x7fd7e4046440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:38.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.656+0000 7fd7e2ffd640 1 -- 172.21.15.38:0/323725014 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7e406d070 con 0x5640b6036090 2024-09-21T03:42:38.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.656+0000 7fd7e2ffd640 1 -- 172.21.15.38:0/323725014 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd7e4065040 con 0x5640b6036090 2024-09-21T03:42:38.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.657+0000 7fd7f7ef1f40 1 -- 172.21.15.38:0/323725014 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5640b60a9760 con 0x5640b6036090 2024-09-21T03:42:38.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.657+0000 7fd7e2ffd640 1 -- 172.21.15.38:0/323725014 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7e4064070 con 0x5640b6036090 2024-09-21T03:42:38.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.657+0000 7fd7f7ef1f40 1 -- 172.21.15.38:0/323725014 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5640b60a6210 con 0x5640b6036090 2024-09-21T03:42:38.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.658+0000 7fd7e2ffd640 1 -- 172.21.15.38:0/323725014 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fd7e4070070 con 0x5640b6036090 2024-09-21T03:42:38.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.659+0000 7fd7e2ffd640 1 --2- 172.21.15.38:0/323725014 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd7d40775a0 0x7fd7d4079ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:38.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.659+0000 7fd7e2ffd640 1 -- 172.21.15.38:0/323725014 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 8282+0+0 (secure 0 0 0) 0x7fd7e40dfc90 con 0x5640b6036090 2024-09-21T03:42:38.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.659+0000 7fd7f11aa640 1 --2- 172.21.15.38:0/323725014 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd7d40775a0 0x7fd7d4079ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:38.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.659+0000 7fd7f7ef1f40 1 --2- 172.21.15.38:0/323725014 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x5640b60bcd10 0x5640b60a8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:38.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.659+0000 7fd7f7ef1f40 1 -- 172.21.15.38:0/323725014 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:1 13.14 13:287034c4:::1.00000000.inode:head [truncate 0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x5640b60a9100 con 0x5640b60bcd10 2024-09-21T03:42:38.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.659+0000 7fd7f21ac640 1 --2- 172.21.15.38:0/323725014 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x5640b60bcd10 0x5640b60a8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:38.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.660+0000 7fd7f11aa640 1 --2- 172.21.15.38:0/323725014 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd7d40775a0 0x7fd7d4079ab0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x5640b60c3950 tx=0x7fd7e8001730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:38.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.660+0000 7fd7f21ac640 1 --2- 172.21.15.38:0/323725014 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x5640b60bcd10 0x5640b60a8bc0 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:38.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.661+0000 7fd7f21ac640 1 -- 172.21.15.38:0/323725014 <== osd.6 v2:172.21.15.195:6826/4055825675 1 ==== osd_op_reply(1 1.00000000.inode [truncate 0] v119'525 uv525 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fd7ec00cb90 con 0x5640b60bcd10 2024-09-21T03:42:38.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.661+0000 7fd7f7ef1f40 1 -- 172.21.15.38:0/323725014 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x5640b60bcd10 msgr2=0x5640b60a8bc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:38.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.661+0000 7fd7f7ef1f40 1 --2- 172.21.15.38:0/323725014 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x5640b60bcd10 0x5640b60a8bc0 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:38.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.661+0000 7fd7f7ef1f40 1 -- 172.21.15.38:0/323725014 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd7d40775a0 msgr2=0x7fd7d4079ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:38.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.661+0000 7fd7f7ef1f40 1 --2- 172.21.15.38:0/323725014 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd7d40775a0 0x7fd7d4079ab0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x5640b60c3950 tx=0x7fd7e8001730 comp rx=0 tx=0).stop 2024-09-21T03:42:38.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.661+0000 7fd7f7ef1f40 1 -- 172.21.15.38:0/323725014 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5640b6036090 msgr2=0x5640b60a5cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:38.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.661+0000 7fd7f7ef1f40 1 --2- 172.21.15.38:0/323725014 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5640b6036090 0x5640b60a5cd0 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x7fd7e4046530 tx=0x7fd7e4046440 comp rx=0 tx=0).stop 2024-09-21T03:42:38.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.661+0000 7fd7f21ac640 1 -- 172.21.15.38:0/323725014 reap_dead start 2024-09-21T03:42:38.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.661+0000 7fd7f7ef1f40 1 -- 172.21.15.38:0/323725014 shutdown_connections 2024-09-21T03:42:38.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.661+0000 7fd7f7ef1f40 1 -- 172.21.15.38:0/323725014 >> 172.21.15.38:0/323725014 conn(0x5640b5f0dac0 msgr2=0x5640b60347f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:38.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.661+0000 7fd7f7ef1f40 1 -- 172.21.15.38:0/323725014 shutdown_connections 2024-09-21T03:42:38.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.661+0000 7fd7f7ef1f40 1 -- 172.21.15.38:0/323725014 wait complete. 2024-09-21T03:42:38.666 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-21T03:42:38.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.784+0000 7f4e079c2640 1 -- 172.21.15.38:0/3040885204 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4de8004210 msgr2=0x7f4de8004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:38.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.784+0000 7f4e079c2640 1 --2- 172.21.15.38:0/3040885204 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4de8004210 0x7f4de8004800 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7f4df4009b10 tx=0x7f4de8005830 comp rx=0 tx=0).stop 2024-09-21T03:42:38.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.784+0000 7f4e05f5f640 1 -- 172.21.15.38:0/3040885204 reap_dead start 2024-09-21T03:42:38.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.784+0000 7f4e079c2640 1 -- 172.21.15.38:0/3040885204 shutdown_connections 2024-09-21T03:42:38.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.784+0000 7f4e079c2640 1 -- 172.21.15.38:0/3040885204 >> 172.21.15.38:0/3040885204 conn(0x7f4e0009d830 msgr2=0x7f4e000b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:38.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.784+0000 7f4e079c2640 1 -- 172.21.15.38:0/3040885204 shutdown_connections 2024-09-21T03:42:38.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.784+0000 7f4e079c2640 1 -- 172.21.15.38:0/3040885204 wait complete. 2024-09-21T03:42:38.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.786+0000 7f4e079c2640 1 Processor -- start 2024-09-21T03:42:38.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.786+0000 7f4e079c2640 1 -- start start 2024-09-21T03:42:38.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.787+0000 7f4e079c2640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4de8004210 0x7f4e0009e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:38.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.787+0000 7f4e079c2640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4e001496e0 0x7f4e0006efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:38.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.787+0000 7f4e079c2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4e0014b950 0x7f4e0014bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:38.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.787+0000 7f4e079c2640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f4e000d0df0 con 0x7f4e0014b950 2024-09-21T03:42:38.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.787+0000 7f4e079c2640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f4e000cf540 con 0x7f4de8004210 2024-09-21T03:42:38.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.787+0000 7f4e079c2640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f4e000d0f70 con 0x7f4e001496e0 2024-09-21T03:42:38.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.787+0000 7f4e05f5f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4e0014b950 0x7f4e0014bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:38.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.787+0000 7f4e05f5f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4e0014b950 0x7f4e0014bd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:52304/0 (socket says 172.21.15.38:52304) 2024-09-21T03:42:38.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.787+0000 7f4e05f5f640 1 -- 172.21.15.38:0/1606435525 learned_addr learned my addr 172.21.15.38:0/1606435525 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:42:38.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.787+0000 7f4e04f5d640 1 --2- 172.21.15.38:0/1606435525 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4e001496e0 0x7f4e0006efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:38.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.787+0000 7f4e0575e640 1 --2- 172.21.15.38:0/1606435525 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4de8004210 0x7f4e0009e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:38.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.788+0000 7f4e05f5f640 1 -- 172.21.15.38:0/1606435525 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4e001496e0 msgr2=0x7f4e0006efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:38.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.788+0000 7f4e05f5f640 1 --2- 172.21.15.38:0/1606435525 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4e001496e0 0x7f4e0006efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:38.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.788+0000 7f4e05f5f640 1 -- 172.21.15.38:0/1606435525 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4de8004210 msgr2=0x7f4e0009e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:38.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.788+0000 7f4e05f5f640 1 --2- 172.21.15.38:0/1606435525 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4de8004210 0x7f4e0009e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:38.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.788+0000 7f4e05f5f640 1 -- 172.21.15.38:0/1606435525 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e0009e8e0 con 0x7f4e0014b950 2024-09-21T03:42:38.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.788+0000 7f4e05f5f640 1 --2- 172.21.15.38:0/1606435525 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4e0014b950 0x7f4e0014bd60 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7f4df0000f80 tx=0x7f4df0006720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:38.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.788+0000 7f4dfdffb640 1 -- 172.21.15.38:0/1606435525 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4df000fb30 con 0x7f4e0014b950 2024-09-21T03:42:38.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.788+0000 7f4e079c2640 1 -- 172.21.15.38:0/1606435525 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e0006df60 con 0x7f4e0014b950 2024-09-21T03:42:38.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.788+0000 7f4dfdffb640 1 -- 172.21.15.38:0/1606435525 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4df0018070 con 0x7f4e0014b950 2024-09-21T03:42:38.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.788+0000 7f4dfdffb640 1 -- 172.21.15.38:0/1606435525 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4df0011430 con 0x7f4e0014b950 2024-09-21T03:42:38.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.788+0000 7f4e079c2640 1 -- 172.21.15.38:0/1606435525 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e0006e4e0 con 0x7f4e0014b950 2024-09-21T03:42:38.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.789+0000 7f4e079c2640 1 -- 172.21.15.38:0/1606435525 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e000ceb90 con 0x7f4e0014b950 2024-09-21T03:42:38.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.790+0000 7f4dfdffb640 1 -- 172.21.15.38:0/1606435525 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f4df00115d0 con 0x7f4e0014b950 2024-09-21T03:42:38.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.791+0000 7f4dfdffb640 1 --2- 172.21.15.38:0/1606435525 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4dec078b90 0x7f4dec07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:38.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.791+0000 7f4dfdffb640 1 -- 172.21.15.38:0/1606435525 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 8282+0+0 (secure 0 0 0) 0x7f4df009bfe0 con 0x7f4e0014b950 2024-09-21T03:42:38.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.791+0000 7f4e0575e640 1 --2- 172.21.15.38:0/1606435525 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4dec078b90 0x7f4dec07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:38.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.791+0000 7f4e0575e640 1 --2- 172.21.15.38:0/1606435525 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4dec078b90 0x7f4dec07b0a0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f4df8006fd0 tx=0x7f4df8006b10 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:38.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:38.795+0000 7f4dfdffb640 1 -- 172.21.15.38:0/1606435525 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f4e001592d0 con 0x7f4e0014b950 2024-09-21T03:42:39.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:39.020+0000 7f4e079c2640 1 -- 172.21.15.38:0/1606435525 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f4e00069080 con 0x7f4e0014b950 2024-09-21T03:42:39.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:39.342+0000 7f4dfdffb640 1 -- 172.21.15.38:0/1606435525 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v224) ==== 162+0+0 (secure 0 0 0) 0x7f4df00632d0 con 0x7f4e0014b950 2024-09-21T03:42:39.344 INFO:teuthology.orchestra.run.smithi038.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-21T03:42:39.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:39.344+0000 7f4e079c2640 1 -- 172.21.15.38:0/1606435525 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4dec078b90 msgr2=0x7f4dec07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:39.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:39.344+0000 7f4e079c2640 1 --2- 172.21.15.38:0/1606435525 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4dec078b90 0x7f4dec07b0a0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f4df8006fd0 tx=0x7f4df8006b10 comp rx=0 tx=0).stop 2024-09-21T03:42:39.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:39.344+0000 7f4e079c2640 1 -- 172.21.15.38:0/1606435525 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4e0014b950 msgr2=0x7f4e0014bd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:39.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:39.344+0000 7f4e079c2640 1 --2- 172.21.15.38:0/1606435525 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4e0014b950 0x7f4e0014bd60 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7f4df0000f80 tx=0x7f4df0006720 comp rx=0 tx=0).stop 2024-09-21T03:42:39.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:39.344+0000 7f4e079c2640 1 -- 172.21.15.38:0/1606435525 shutdown_connections 2024-09-21T03:42:39.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:39.344+0000 7f4e079c2640 1 --2- 172.21.15.38:0/1606435525 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4e0014b950 0x7f4e0014bd60 unknown :-1 s=CLOSED pgs=1806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:39.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:39.344+0000 7f4e079c2640 1 --2- 172.21.15.38:0/1606435525 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4dec078b90 0x7f4dec07b0a0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:39.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:39.344+0000 7f4e079c2640 1 --2- 172.21.15.38:0/1606435525 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4e001496e0 0x7f4e0006efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:39.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:39.344+0000 7f4e079c2640 1 --2- 172.21.15.38:0/1606435525 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4de8004210 0x7f4e0009e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:39.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:39.344+0000 7f4e079c2640 1 -- 172.21.15.38:0/1606435525 >> 172.21.15.38:0/1606435525 conn(0x7f4e0009d830 msgr2=0x7f4e00161a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:39.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:39.344+0000 7f4e079c2640 1 -- 172.21.15.38:0/1606435525 shutdown_connections 2024-09-21T03:42:39.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:39.345+0000 7f4e079c2640 1 -- 172.21.15.38:0/1606435525 wait complete. 2024-09-21T03:42:39.359 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:42:39.362 INFO:tasks.ceph.mds.a.smithi038.stderr:2024-09-21T03:42:39.360+0000 7f7ed144c640 -1 mds.0.cache.ino(0x1) No data while reading inode 0x1 2024-09-21T03:42:39.362 INFO:tasks.ceph.mds.a.smithi038.stderr:2024-09-21T03:42:39.360+0000 7f7ed144c640 -1 log_channel(cluster) log [ERR] : Error loading MDS rank 0: (2) No such file or directory 2024-09-21T03:42:39.362 INFO:tasks.ceph.mon.a.smithi038.stderr:2024-09-21T03:42:39.360+0000 7f3f45713640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T03:42:39.363 INFO:tasks.ceph.mon.a.smithi038.stderr:2024-09-21T03:42:39.360+0000 7f3f45713640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-09-21T03:42:39.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:39.475+0000 7f2b71a30640 1 -- 172.21.15.38:0/251070996 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2b6c162590 msgr2=0x7f2b6c1629a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:39.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:39.475+0000 7f2b71a30640 1 --2- 172.21.15.38:0/251070996 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2b6c162590 0x7f2b6c1629a0 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7f2b60009900 tx=0x7f2b6002a560 comp rx=0 tx=0).stop 2024-09-21T03:42:39.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:39.475+0000 7f2b71a30640 1 -- 172.21.15.38:0/251070996 shutdown_connections 2024-09-21T03:42:39.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:39.475+0000 7f2b71a30640 1 --2- 172.21.15.38:0/251070996 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2b6c0b2e60 0x7f2b6c0b3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:39.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:39.475+0000 7f2b71a30640 1 --2- 172.21.15.38:0/251070996 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2b6c162590 0x7f2b6c1629a0 unknown :-1 s=CLOSED pgs=1807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:39.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:39.475+0000 7f2b71a30640 1 -- 172.21.15.38:0/251070996 >> 172.21.15.38:0/251070996 conn(0x7f2b6c0d4440 msgr2=0x7f2b6c0d4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:39.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:39.475+0000 7f2b71a30640 1 -- 172.21.15.38:0/251070996 shutdown_connections 2024-09-21T03:42:39.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:39.475+0000 7f2b71a30640 1 -- 172.21.15.38:0/251070996 wait complete. 2024-09-21T03:42:39.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:39.477+0000 7f2b71a30640 1 Processor -- start 2024-09-21T03:42:39.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:39.477+0000 7f2b71a30640 1 -- start start 2024-09-21T03:42:39.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:39.478+0000 7f2b71a30640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2b6c0b2e60 0x7f2b6c15de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:39.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:39.478+0000 7f2b71a30640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2b6c0cfe00 0x7f2b6c145350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:39.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:39.478+0000 7f2b71a30640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2b6c162590 0x7f2b6c145890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:39.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:39.478+0000 7f2b71a30640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f2b6c0b3b40 con 0x7f2b6c162590 2024-09-21T03:42:39.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:39.478+0000 7f2b71a30640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f2b6c0b39c0 con 0x7f2b6c0b2e60 2024-09-21T03:42:39.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:39.478+0000 7f2b71a30640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f2b6c0cf590 con 0x7f2b6c0cfe00 2024-09-21T03:42:39.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:39.479+0000 7f2b6b7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2b6c162590 0x7f2b6c145890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:39.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:39.479+0000 7f2b6b7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2b6c162590 0x7f2b6c145890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:52322/0 (socket says 172.21.15.38:52322) 2024-09-21T03:42:39.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:39.479+0000 7f2b6b7fe640 1 -- 172.21.15.38:0/2776215252 learned_addr learned my addr 172.21.15.38:0/2776215252 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:42:39.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:39.479+0000 7f2b6affd640 1 --2- 172.21.15.38:0/2776215252 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2b6c0b2e60 0x7f2b6c15de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:39.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:39.479+0000 7f2b6a7fc640 1 --2- 172.21.15.38:0/2776215252 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2b6c0cfe00 0x7f2b6c145350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:39.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:39.479+0000 7f2b6b7fe640 1 -- 172.21.15.38:0/2776215252 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2b6c0cfe00 msgr2=0x7f2b6c145350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:39.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:39.479+0000 7f2b6b7fe640 1 --2- 172.21.15.38:0/2776215252 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2b6c0cfe00 0x7f2b6c145350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:39.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:39.479+0000 7f2b6b7fe640 1 -- 172.21.15.38:0/2776215252 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2b6c0b2e60 msgr2=0x7f2b6c15de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:39.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:39.479+0000 7f2b6b7fe640 1 --2- 172.21.15.38:0/2776215252 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2b6c0b2e60 0x7f2b6c15de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:39.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:39.479+0000 7f2b6b7fe640 1 -- 172.21.15.38:0/2776215252 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b6c158cd0 con 0x7f2b6c162590 2024-09-21T03:42:39.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:39.479+0000 7f2b6b7fe640 1 --2- 172.21.15.38:0/2776215252 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2b6c162590 0x7f2b6c145890 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7f2b6400c920 tx=0x7f2b6400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:39.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:39.479+0000 7f2b53fff640 1 -- 172.21.15.38:0/2776215252 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b64007bb0 con 0x7f2b6c162590 2024-09-21T03:42:39.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:39.479+0000 7f2b53fff640 1 -- 172.21.15.38:0/2776215252 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2b64007e40 con 0x7f2b6c162590 2024-09-21T03:42:39.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:39.479+0000 7f2b71a30640 1 -- 172.21.15.38:0/2776215252 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b6c1e9fc0 con 0x7f2b6c162590 2024-09-21T03:42:39.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:39.479+0000 7f2b53fff640 1 -- 172.21.15.38:0/2776215252 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b64005280 con 0x7f2b6c162590 2024-09-21T03:42:39.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:39.480+0000 7f2b71a30640 1 -- 172.21.15.38:0/2776215252 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b6c1ea580 con 0x7f2b6c162590 2024-09-21T03:42:39.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:39.480+0000 7f2b71a30640 1 -- 172.21.15.38:0/2776215252 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b6c1e9d30 con 0x7f2b6c162590 2024-09-21T03:42:39.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:39.481+0000 7f2b53fff640 1 -- 172.21.15.38:0/2776215252 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f2b64005420 con 0x7f2b6c162590 2024-09-21T03:42:39.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:39.482+0000 7f2b53fff640 1 --2- 172.21.15.38:0/2776215252 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2b58078b90 0x7f2b5807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:39.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:39.482+0000 7f2b53fff640 1 -- 172.21.15.38:0/2776215252 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 8368+0+0 (secure 0 0 0) 0x7f2b6409bb80 con 0x7f2b6c162590 2024-09-21T03:42:39.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:39.482+0000 7f2b6affd640 1 --2- 172.21.15.38:0/2776215252 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2b58078b90 0x7f2b5807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:39.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:39.483+0000 7f2b6affd640 1 --2- 172.21.15.38:0/2776215252 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2b58078b90 0x7f2b5807b0a0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f2b5c009fd0 tx=0x7f2b5c00d040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:39.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:39.486+0000 7f2b53fff640 1 -- 172.21.15.38:0/2776215252 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f2b6c158cd0 con 0x7f2b6c162590 2024-09-21T03:42:39.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:39.713+0000 7f2b71a30640 1 -- 172.21.15.38:0/2776215252 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2b6c0ce710 con 0x7f2b6c162590 2024-09-21T03:42:39.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:39.713+0000 7f2b53fff640 1 -- 172.21.15.38:0/2776215252 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 226 v226) ==== 95+0+3566 (secure 0 0 0) 0x7f2b64014030 con 0x7f2b6c162590 2024-09-21T03:42:39.715 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:42:39.715 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":226,"btime":"2024-09-21T03:42:39:361564+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6086,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/965972663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":965972663},{"type":"v1","addr":"172.21.15.195:6836","nonce":965972663}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6700,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/3176122334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":3176122334},{"type":"v1","addr":"172.21.15.195:6837","nonce":3176122334}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":226,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:40:02.319282+0000","modified":"2024-09-21T03:42:39.361494+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":120,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-21T03:42:39.716 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 226 2024-09-21T03:42:39.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:39.715+0000 7f2b71a30640 1 -- 172.21.15.38:0/2776215252 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2b58078b90 msgr2=0x7f2b5807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:39.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:39.715+0000 7f2b71a30640 1 --2- 172.21.15.38:0/2776215252 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2b58078b90 0x7f2b5807b0a0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f2b5c009fd0 tx=0x7f2b5c00d040 comp rx=0 tx=0).stop 2024-09-21T03:42:39.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:39.715+0000 7f2b71a30640 1 -- 172.21.15.38:0/2776215252 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2b6c162590 msgr2=0x7f2b6c145890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:39.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:39.715+0000 7f2b71a30640 1 --2- 172.21.15.38:0/2776215252 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2b6c162590 0x7f2b6c145890 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7f2b6400c920 tx=0x7f2b6400cde0 comp rx=0 tx=0).stop 2024-09-21T03:42:39.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:39.715+0000 7f2b71a30640 1 -- 172.21.15.38:0/2776215252 shutdown_connections 2024-09-21T03:42:39.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:39.715+0000 7f2b71a30640 1 --2- 172.21.15.38:0/2776215252 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2b6c162590 0x7f2b6c145890 unknown :-1 s=CLOSED pgs=1808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:39.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:39.715+0000 7f2b71a30640 1 --2- 172.21.15.38:0/2776215252 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2b58078b90 0x7f2b5807b0a0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:39.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:39.715+0000 7f2b71a30640 1 --2- 172.21.15.38:0/2776215252 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2b6c0cfe00 0x7f2b6c145350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:39.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:39.715+0000 7f2b71a30640 1 --2- 172.21.15.38:0/2776215252 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2b6c0b2e60 0x7f2b6c15de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:39.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:39.715+0000 7f2b71a30640 1 -- 172.21.15.38:0/2776215252 >> 172.21.15.38:0/2776215252 conn(0x7f2b6c0d4440 msgr2=0x7f2b6c0d4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:39.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:39.715+0000 7f2b71a30640 1 -- 172.21.15.38:0/2776215252 shutdown_connections 2024-09-21T03:42:39.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:39.715+0000 7f2b71a30640 1 -- 172.21.15.38:0/2776215252 wait complete. 2024-09-21T03:42:39.729 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-21T03:42:39.729 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:42:39.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:39.845+0000 7f4ed2b3a640 1 -- 172.21.15.38:0/3423397940 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4eb40041c0 msgr2=0x7f4eb4004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:39.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:39.845+0000 7f4ed2b3a640 1 --2- 172.21.15.38:0/3423397940 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4eb40041c0 0x7f4eb4004800 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7f4ec0006fb0 tx=0x7f4eb4005830 comp rx=0 tx=0).stop 2024-09-21T03:42:39.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:39.845+0000 7f4ed10d7640 1 -- 172.21.15.38:0/3423397940 reap_dead start 2024-09-21T03:42:39.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:39.846+0000 7f4ed2b3a640 1 -- 172.21.15.38:0/3423397940 shutdown_connections 2024-09-21T03:42:39.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:39.846+0000 7f4ed2b3a640 1 -- 172.21.15.38:0/3423397940 >> 172.21.15.38:0/3423397940 conn(0x7f4ecc0b6b20 msgr2=0x7f4ecc0d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:39.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:39.846+0000 7f4ed2b3a640 1 -- 172.21.15.38:0/3423397940 shutdown_connections 2024-09-21T03:42:39.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:39.846+0000 7f4ed2b3a640 1 -- 172.21.15.38:0/3423397940 wait complete. 2024-09-21T03:42:39.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:39.847+0000 7f4ed2b3a640 1 Processor -- start 2024-09-21T03:42:39.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:39.847+0000 7f4ed2b3a640 1 -- start start 2024-09-21T03:42:39.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:39.849+0000 7f4ed2b3a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4eb40041c0 0x7f4ecc0ceb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:39.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:39.849+0000 7f4ed2b3a640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4ecc147e80 0x7f4ecc1f86a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:39.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:39.849+0000 7f4ed2b3a640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4ecc1f8be0 0x7f4ecc1f8ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:39.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:39.849+0000 7f4ed2b3a640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f4ecc0b31d0 con 0x7f4eb40041c0 2024-09-21T03:42:39.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:39.849+0000 7f4ed2b3a640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f4ecc0b3050 con 0x7f4ecc1f8be0 2024-09-21T03:42:39.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:39.849+0000 7f4ed2b3a640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f4ecc0b3350 con 0x7f4ecc147e80 2024-09-21T03:42:39.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:39.849+0000 7f4ed08d6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4eb40041c0 0x7f4ecc0ceb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:39.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:39.849+0000 7f4ed08d6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4eb40041c0 0x7f4ecc0ceb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:52334/0 (socket says 172.21.15.38:52334) 2024-09-21T03:42:39.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:39.849+0000 7f4ed08d6640 1 -- 172.21.15.38:0/2408259450 learned_addr learned my addr 172.21.15.38:0/2408259450 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:42:39.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:39.849+0000 7f4ecbfff640 1 --2- 172.21.15.38:0/2408259450 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4ecc147e80 0x7f4ecc1f86a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:39.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:39.849+0000 7f4ed10d7640 1 --2- 172.21.15.38:0/2408259450 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4ecc1f8be0 0x7f4ecc1f8ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:39.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:39.849+0000 7f4ed08d6640 1 -- 172.21.15.38:0/2408259450 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4ecc147e80 msgr2=0x7f4ecc1f86a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:39.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:39.849+0000 7f4ed08d6640 1 --2- 172.21.15.38:0/2408259450 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4ecc147e80 0x7f4ecc1f86a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:39.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:39.849+0000 7f4ed08d6640 1 -- 172.21.15.38:0/2408259450 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4ecc1f8be0 msgr2=0x7f4ecc1f8ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:39.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:39.849+0000 7f4ed08d6640 1 --2- 172.21.15.38:0/2408259450 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4ecc1f8be0 0x7f4ecc1f8ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:39.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:39.849+0000 7f4ed08d6640 1 -- 172.21.15.38:0/2408259450 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ecc1e8c10 con 0x7f4eb40041c0 2024-09-21T03:42:39.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:39.849+0000 7f4ed10d7640 1 --2- 172.21.15.38:0/2408259450 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4ecc1f8be0 0x7f4ecc1f8ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:42:39.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:39.850+0000 7f4ed08d6640 1 --2- 172.21.15.38:0/2408259450 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4eb40041c0 0x7f4ecc0ceb20 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7f4ebc004cf0 tx=0x7f4ebc0017a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:39.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:39.850+0000 7f4ec97fa640 1 -- 172.21.15.38:0/2408259450 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ebc015020 con 0x7f4eb40041c0 2024-09-21T03:42:39.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:39.850+0000 7f4ec97fa640 1 -- 172.21.15.38:0/2408259450 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4ebc0119a0 con 0x7f4eb40041c0 2024-09-21T03:42:39.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:39.850+0000 7f4ed2b3a640 1 -- 172.21.15.38:0/2408259450 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ecc15b2f0 con 0x7f4eb40041c0 2024-09-21T03:42:39.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:39.850+0000 7f4ec97fa640 1 -- 172.21.15.38:0/2408259450 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ebc011df0 con 0x7f4eb40041c0 2024-09-21T03:42:39.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:39.850+0000 7f4ed2b3a640 1 -- 172.21.15.38:0/2408259450 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ecc1e9190 con 0x7f4eb40041c0 2024-09-21T03:42:39.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:39.851+0000 7f4ed2b3a640 1 -- 172.21.15.38:0/2408259450 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ecc0b38e0 con 0x7f4eb40041c0 2024-09-21T03:42:39.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:39.852+0000 7f4ec97fa640 1 -- 172.21.15.38:0/2408259450 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f4ebc018020 con 0x7f4eb40041c0 2024-09-21T03:42:39.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:39.852+0000 7f4ec97fa640 1 --2- 172.21.15.38:0/2408259450 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4eb8078b90 0x7f4eb807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:39.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:39.852+0000 7f4ec97fa640 1 -- 172.21.15.38:0/2408259450 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 8368+0+0 (secure 0 0 0) 0x7f4ebc09c2a0 con 0x7f4eb40041c0 2024-09-21T03:42:39.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:39.852+0000 7f4ecbfff640 1 --2- 172.21.15.38:0/2408259450 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4eb8078b90 0x7f4eb807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:39.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:39.853+0000 7f4ecbfff640 1 --2- 172.21.15.38:0/2408259450 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4eb8078b90 0x7f4eb807b0a0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f4ec0033000 tx=0x7f4ec002d000 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:39.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:39.857+0000 7f4ec97fa640 1 -- 172.21.15.38:0/2408259450 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f4ebc01f020 con 0x7f4eb40041c0 2024-09-21T03:42:40.071 INFO:tasks.ceph.mds.c.smithi038.stderr: -51> 2024-09-21T03:42:35.317+0000 7f6e0eba1640 -1 mds.0.cache.ino(0x100) Corrupt inode 0x100: End of buffer [buffer:2] 2024-09-21T03:42:40.071 INFO:tasks.ceph.mds.c.smithi038.stderr: -41> 2024-09-21T03:42:35.317+0000 7f6e0eba1640 -1 log_channel(cluster) log [ERR] : Error loading MDS rank 0: (22) Invalid argument 2024-09-21T03:42:40.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:40.083+0000 7f4ed2b3a640 1 -- 172.21.15.38:0/2408259450 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4ecc0b8a50 con 0x7f4eb40041c0 2024-09-21T03:42:40.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:40.084+0000 7f4ec97fa640 1 -- 172.21.15.38:0/2408259450 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 226 v226) ==== 95+0+3566 (secure 0 0 0) 0x7f4ebc063c30 con 0x7f4eb40041c0 2024-09-21T03:42:40.086 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:42:40.086 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":226,"btime":"2024-09-21T03:42:39:361564+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6086,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/965972663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":965972663},{"type":"v1","addr":"172.21.15.195:6836","nonce":965972663}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6700,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/3176122334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":3176122334},{"type":"v1","addr":"172.21.15.195:6837","nonce":3176122334}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":226,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:40:02.319282+0000","modified":"2024-09-21T03:42:39.361494+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":120,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-21T03:42:40.086 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 226 2024-09-21T03:42:40.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:40.086+0000 7f4ed2b3a640 1 -- 172.21.15.38:0/2408259450 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4eb8078b90 msgr2=0x7f4eb807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:40.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:40.086+0000 7f4ed2b3a640 1 --2- 172.21.15.38:0/2408259450 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4eb8078b90 0x7f4eb807b0a0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f4ec0033000 tx=0x7f4ec002d000 comp rx=0 tx=0).stop 2024-09-21T03:42:40.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:40.086+0000 7f4ed2b3a640 1 -- 172.21.15.38:0/2408259450 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4eb40041c0 msgr2=0x7f4ecc0ceb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:40.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:40.086+0000 7f4ed2b3a640 1 --2- 172.21.15.38:0/2408259450 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4eb40041c0 0x7f4ecc0ceb20 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7f4ebc004cf0 tx=0x7f4ebc0017a0 comp rx=0 tx=0).stop 2024-09-21T03:42:40.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:40.086+0000 7f4ed2b3a640 1 -- 172.21.15.38:0/2408259450 shutdown_connections 2024-09-21T03:42:40.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:40.086+0000 7f4ed2b3a640 1 --2- 172.21.15.38:0/2408259450 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4ecc1f8be0 0x7f4ecc1f8ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:40.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:40.086+0000 7f4ed2b3a640 1 --2- 172.21.15.38:0/2408259450 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4eb8078b90 0x7f4eb807b0a0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:40.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:40.086+0000 7f4ed2b3a640 1 --2- 172.21.15.38:0/2408259450 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4ecc147e80 0x7f4ecc1f86a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:40.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:40.086+0000 7f4ed2b3a640 1 --2- 172.21.15.38:0/2408259450 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4eb40041c0 0x7f4ecc0ceb20 unknown :-1 s=CLOSED pgs=1811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:40.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:40.086+0000 7f4ed2b3a640 1 -- 172.21.15.38:0/2408259450 >> 172.21.15.38:0/2408259450 conn(0x7f4ecc0b6b20 msgr2=0x7f4ecc0d0c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:40.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:40.086+0000 7f4ed2b3a640 1 -- 172.21.15.38:0/2408259450 shutdown_connections 2024-09-21T03:42:40.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:40.086+0000 7f4ed2b3a640 1 -- 172.21.15.38:0/2408259450 wait complete. 2024-09-21T03:42:40.101 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-21T03:42:40.101 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:42:40.124 INFO:tasks.ceph.mds.c.smithi038.stdout:starting mds.c at 2024-09-21T03:42:40.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:40.217+0000 7f6cbcdad640 1 -- 172.21.15.38:0/43677328 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6ca4003c10 msgr2=0x7f6ca4004060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:40.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:40.217+0000 7f6cbcdad640 1 --2- 172.21.15.38:0/43677328 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6ca4003c10 0x7f6ca4004060 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7f6cb80d25a0 tx=0x7f6ca4005830 comp rx=0 tx=0).stop 2024-09-21T03:42:40.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:40.217+0000 7f6cb6d9d640 1 -- 172.21.15.38:0/43677328 reap_dead start 2024-09-21T03:42:40.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:40.217+0000 7f6cbcdad640 1 -- 172.21.15.38:0/43677328 shutdown_connections 2024-09-21T03:42:40.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:40.217+0000 7f6cbcdad640 1 -- 172.21.15.38:0/43677328 >> 172.21.15.38:0/43677328 conn(0x7f6cb80d4440 msgr2=0x7f6cb80d4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:40.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:40.217+0000 7f6cbcdad640 1 -- 172.21.15.38:0/43677328 shutdown_connections 2024-09-21T03:42:40.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:40.217+0000 7f6cbcdad640 1 -- 172.21.15.38:0/43677328 wait complete. 2024-09-21T03:42:40.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:40.219+0000 7f6cbcdad640 1 Processor -- start 2024-09-21T03:42:40.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:40.219+0000 7f6cbcdad640 1 -- start start 2024-09-21T03:42:40.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:40.220+0000 7f6cbcdad640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6ca4003c10 0x7f6cb815bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:40.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:40.220+0000 7f6cbcdad640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6cb815e570 0x7f6cb815a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:40.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:40.220+0000 7f6cbcdad640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6cb815ab60 0x7f6cb80c7f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:40.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:40.220+0000 7f6cbcdad640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f6cb80b50f0 con 0x7f6cb815ab60 2024-09-21T03:42:40.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:40.220+0000 7f6cbcdad640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f6cb80b4f70 con 0x7f6ca4003c10 2024-09-21T03:42:40.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:40.220+0000 7f6cbcdad640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f6cb80b5270 con 0x7f6cb815e570 2024-09-21T03:42:40.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:40.220+0000 7f6cb6d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6cb815ab60 0x7f6cb80c7f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:40.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:40.220+0000 7f6cb6d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6cb815ab60 0x7f6cb80c7f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:52372/0 (socket says 172.21.15.38:52372) 2024-09-21T03:42:40.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:40.220+0000 7f6cb6d9d640 1 -- 172.21.15.38:0/4166128865 learned_addr learned my addr 172.21.15.38:0/4166128865 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:42:40.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:40.220+0000 7f6cb659c640 1 --2- 172.21.15.38:0/4166128865 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6ca4003c10 0x7f6cb815bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:40.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:40.220+0000 7f6cb5d9b640 1 --2- 172.21.15.38:0/4166128865 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6cb815e570 0x7f6cb815a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:40.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:40.221+0000 7f6cb6d9d640 1 -- 172.21.15.38:0/4166128865 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6cb815e570 msgr2=0x7f6cb815a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:40.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:40.221+0000 7f6cb6d9d640 1 --2- 172.21.15.38:0/4166128865 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6cb815e570 0x7f6cb815a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:40.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:40.221+0000 7f6cb6d9d640 1 -- 172.21.15.38:0/4166128865 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6ca4003c10 msgr2=0x7f6cb815bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:40.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:40.221+0000 7f6cb6d9d640 1 --2- 172.21.15.38:0/4166128865 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6ca4003c10 0x7f6cb815bc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:40.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:40.221+0000 7f6cb6d9d640 1 -- 172.21.15.38:0/4166128865 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cb815d4f0 con 0x7f6cb815ab60 2024-09-21T03:42:40.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:40.221+0000 7f6cb659c640 1 --2- 172.21.15.38:0/4166128865 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6ca4003c10 0x7f6cb815bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:42:40.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:40.221+0000 7f6cb6d9d640 1 --2- 172.21.15.38:0/4166128865 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6cb815ab60 0x7f6cb80c7f70 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7f6cb80b5fc0 tx=0x7f6cb0001800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:40.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:40.221+0000 7f6caeffd640 1 -- 172.21.15.38:0/4166128865 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cb0015070 con 0x7f6cb815ab60 2024-09-21T03:42:40.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:40.221+0000 7f6caeffd640 1 -- 172.21.15.38:0/4166128865 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6cb0003e50 con 0x7f6cb815ab60 2024-09-21T03:42:40.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:40.221+0000 7f6cbcdad640 1 -- 172.21.15.38:0/4166128865 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cb815d7d0 con 0x7f6cb815ab60 2024-09-21T03:42:40.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:40.221+0000 7f6caeffd640 1 -- 172.21.15.38:0/4166128865 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cb0011ac0 con 0x7f6cb815ab60 2024-09-21T03:42:40.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:40.221+0000 7f6cbcdad640 1 -- 172.21.15.38:0/4166128865 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cb815dd90 con 0x7f6cb815ab60 2024-09-21T03:42:40.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:40.222+0000 7f6cbcdad640 1 -- 172.21.15.38:0/4166128865 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6cb815d260 con 0x7f6cb815ab60 2024-09-21T03:42:40.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:40.223+0000 7f6caeffd640 1 -- 172.21.15.38:0/4166128865 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f6cb0018070 con 0x7f6cb815ab60 2024-09-21T03:42:40.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:40.223+0000 7f6caeffd640 1 --2- 172.21.15.38:0/4166128865 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6c98078bc0 0x7f6c9807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:40.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:40.224+0000 7f6caeffd640 1 -- 172.21.15.38:0/4166128865 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 8368+0+0 (secure 0 0 0) 0x7f6cb009c140 con 0x7f6cb815ab60 2024-09-21T03:42:40.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:40.224+0000 7f6cb659c640 1 --2- 172.21.15.38:0/4166128865 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6c98078bc0 0x7f6c9807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:40.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:40.224+0000 7f6cb659c640 1 --2- 172.21.15.38:0/4166128865 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6c98078bc0 0x7f6c9807b0d0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f6ca8034000 tx=0x7f6ca802d040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:40.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:40.228+0000 7f6caeffd640 1 -- 172.21.15.38:0/4166128865 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f6cb0063bf0 con 0x7f6cb815ab60 2024-09-21T03:42:40.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:40.453+0000 7f6cbcdad640 1 -- 172.21.15.38:0/4166128865 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6cb815e170 con 0x7f6cb815ab60 2024-09-21T03:42:40.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:40.453+0000 7f6caeffd640 1 -- 172.21.15.38:0/4166128865 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 227 v227) ==== 95+0+4392 (secure 0 0 0) 0x7f6cb00634f0 con 0x7f6cb815ab60 2024-09-21T03:42:40.456 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:42:40.456 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":227,"btime":"2024-09-21T03:42:40:363219+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6086,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/965972663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":965972663},{"type":"v1","addr":"172.21.15.195:6836","nonce":965972663}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6700,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/3176122334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":3176122334},{"type":"v1","addr":"172.21.15.195:6837","nonce":3176122334}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":9309,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/2459386631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":2459386631},{"type":"v1","addr":"172.21.15.38:6833","nonce":2459386631}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":227}],"filesystems":[{"mdsmap":{"epoch":226,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:40:02.319282+0000","modified":"2024-09-21T03:42:39.361494+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":120,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-21T03:42:40.456 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 227 2024-09-21T03:42:40.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:40.456+0000 7f6cbcdad640 1 -- 172.21.15.38:0/4166128865 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6c98078bc0 msgr2=0x7f6c9807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:40.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:40.456+0000 7f6cbcdad640 1 --2- 172.21.15.38:0/4166128865 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6c98078bc0 0x7f6c9807b0d0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f6ca8034000 tx=0x7f6ca802d040 comp rx=0 tx=0).stop 2024-09-21T03:42:40.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:40.456+0000 7f6cbcdad640 1 -- 172.21.15.38:0/4166128865 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6cb815ab60 msgr2=0x7f6cb80c7f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:40.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:40.456+0000 7f6cbcdad640 1 --2- 172.21.15.38:0/4166128865 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6cb815ab60 0x7f6cb80c7f70 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7f6cb80b5fc0 tx=0x7f6cb0001800 comp rx=0 tx=0).stop 2024-09-21T03:42:40.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:40.456+0000 7f6cbcdad640 1 -- 172.21.15.38:0/4166128865 shutdown_connections 2024-09-21T03:42:40.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:40.456+0000 7f6cbcdad640 1 --2- 172.21.15.38:0/4166128865 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6cb815ab60 0x7f6cb80c7f70 unknown :-1 s=CLOSED pgs=1815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:40.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:40.456+0000 7f6cbcdad640 1 --2- 172.21.15.38:0/4166128865 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6c98078bc0 0x7f6c9807b0d0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:40.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:40.456+0000 7f6cbcdad640 1 --2- 172.21.15.38:0/4166128865 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6cb815e570 0x7f6cb815a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:40.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:40.456+0000 7f6cbcdad640 1 --2- 172.21.15.38:0/4166128865 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6ca4003c10 0x7f6cb815bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:40.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:40.456+0000 7f6cbcdad640 1 -- 172.21.15.38:0/4166128865 >> 172.21.15.38:0/4166128865 conn(0x7f6cb80d4440 msgr2=0x7f6cb80d4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:40.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:40.456+0000 7f6cbcdad640 1 -- 172.21.15.38:0/4166128865 shutdown_connections 2024-09-21T03:42:40.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:40.456+0000 7f6cbcdad640 1 -- 172.21.15.38:0/4166128865 wait complete. 2024-09-21T03:42:40.469 INFO:tasks.cephfs.test_damage:Result: Mutation 'Truncate 1.00000000.inode' led to DAMAGED state 2024-09-21T03:42:40.469 INFO:tasks.cephfs.test_damage:Applying mutation 'Truncate 200.00000000' 2024-09-21T03:42:40.469 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-21T03:42:40.469 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:42:40.525 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056795,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2354 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T03:42:40.525 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi038.front.sepia.ceph.com 2024-09-21T03:42:40.525 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:42:40.525 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T03:42:40.581 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T03:42:40.581 INFO:teuthology.orchestra.run.smithi038.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-09-21T03:42:40.582 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-21T03:42:40.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:40.737+0000 7f51e7be4640 1 -- 172.21.15.38:0/340392365 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f51e00b9ad0 msgr2=0x7f51e00d0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:40.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:40.737+0000 7f51e7be4640 1 --2- 172.21.15.38:0/340392365 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f51e00b9ad0 0x7f51e00d0be0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f51d0009820 tx=0x7f51d002a480 comp rx=0 tx=0).stop 2024-09-21T03:42:40.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:40.737+0000 7f51e7be4640 1 -- 172.21.15.38:0/340392365 shutdown_connections 2024-09-21T03:42:40.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:40.737+0000 7f51e7be4640 1 --2- 172.21.15.38:0/340392365 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f51e00b9ad0 0x7f51e00d0be0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:40.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:40.737+0000 7f51e7be4640 1 --2- 172.21.15.38:0/340392365 >> v2:172.21.15.195:3300/0 conn(0x7f51e015ee80 0x7f51e00d0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:40.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:40.737+0000 7f51e7be4640 1 -- 172.21.15.38:0/340392365 >> 172.21.15.38:0/340392365 conn(0x7f51e009d830 msgr2=0x7f51e00b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:40.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:40.737+0000 7f51e7be4640 1 -- 172.21.15.38:0/340392365 shutdown_connections 2024-09-21T03:42:40.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:40.737+0000 7f51e7be4640 1 -- 172.21.15.38:0/340392365 wait complete. 2024-09-21T03:42:40.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:40.739+0000 7f51e7be4640 1 Processor -- start 2024-09-21T03:42:40.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:40.739+0000 7f51e7be4640 1 -- start start 2024-09-21T03:42:40.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:40.740+0000 7f51e7be4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f51e00b9ad0 0x7f51e01f53b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:40.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:40.740+0000 7f51e7be4640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f51e00d1120 0x7f51e006e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:40.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:40.741+0000 7f51e7be4640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f51e015ee80 0x7f51e014f900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:40.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:40.741+0000 7f51e7be4640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f51e00b6700 con 0x7f51e00b9ad0 2024-09-21T03:42:40.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:40.741+0000 7f51e7be4640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f51e00b6580 con 0x7f51e015ee80 2024-09-21T03:42:40.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:40.741+0000 7f51e7be4640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f51e00b6880 con 0x7f51e00d1120 2024-09-21T03:42:40.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:40.741+0000 7f51e5980640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f51e00b9ad0 0x7f51e01f53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:40.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:40.741+0000 7f51e5980640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f51e00b9ad0 0x7f51e01f53b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:52376/0 (socket says 172.21.15.38:52376) 2024-09-21T03:42:40.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:40.741+0000 7f51e5980640 1 -- 172.21.15.38:0/3737998816 learned_addr learned my addr 172.21.15.38:0/3737998816 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:42:40.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:40.741+0000 7f51e517f640 1 --2- 172.21.15.38:0/3737998816 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f51e00d1120 0x7f51e006e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:40.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:40.741+0000 7f51e6181640 1 --2- 172.21.15.38:0/3737998816 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f51e015ee80 0x7f51e014f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:40.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:40.741+0000 7f51e5980640 1 -- 172.21.15.38:0/3737998816 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f51e00d1120 msgr2=0x7f51e006e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:40.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:40.741+0000 7f51e5980640 1 --2- 172.21.15.38:0/3737998816 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f51e00d1120 0x7f51e006e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:40.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:40.741+0000 7f51e5980640 1 -- 172.21.15.38:0/3737998816 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f51e015ee80 msgr2=0x7f51e014f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:40.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:40.741+0000 7f51e5980640 1 --2- 172.21.15.38:0/3737998816 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f51e015ee80 0x7f51e014f900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:40.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:40.741+0000 7f51e5980640 1 -- 172.21.15.38:0/3737998816 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51e006e6b0 con 0x7f51e00b9ad0 2024-09-21T03:42:40.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:40.741+0000 7f51e5980640 1 --2- 172.21.15.38:0/3737998816 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f51e00b9ad0 0x7f51e01f53b0 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7f51dc00ece0 tx=0x7f51dc00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:40.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:40.741+0000 7f51d67fc640 1 -- 172.21.15.38:0/3737998816 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51dc003b70 con 0x7f51e00b9ad0 2024-09-21T03:42:40.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:40.741+0000 7f51d67fc640 1 -- 172.21.15.38:0/3737998816 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f51dc003d10 con 0x7f51e00b9ad0 2024-09-21T03:42:40.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:40.741+0000 7f51e7be4640 1 -- 172.21.15.38:0/3737998816 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51e014fe40 con 0x7f51e00b9ad0 2024-09-21T03:42:40.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:40.741+0000 7f51d67fc640 1 -- 172.21.15.38:0/3737998816 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51dc004280 con 0x7f51e00b9ad0 2024-09-21T03:42:40.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:40.741+0000 7f51e7be4640 1 -- 172.21.15.38:0/3737998816 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51e01f5f90 con 0x7f51e00b9ad0 2024-09-21T03:42:40.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:40.742+0000 7f51e7be4640 1 -- 172.21.15.38:0/3737998816 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51e00b6d30 con 0x7f51e00b9ad0 2024-09-21T03:42:40.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:40.743+0000 7f51d67fc640 1 -- 172.21.15.38:0/3737998816 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f51dc004420 con 0x7f51e00b9ad0 2024-09-21T03:42:40.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:40.744+0000 7f51d67fc640 1 --2- 172.21.15.38:0/3737998816 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f51c4078bc0 0x7f51c407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:40.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:40.744+0000 7f51d67fc640 1 -- 172.21.15.38:0/3737998816 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 8368+0+0 (secure 0 0 0) 0x7f51dc09c4a0 con 0x7f51e00b9ad0 2024-09-21T03:42:40.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:40.744+0000 7f51e517f640 1 --2- 172.21.15.38:0/3737998816 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f51c4078bc0 0x7f51c407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:40.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:40.745+0000 7f51e517f640 1 --2- 172.21.15.38:0/3737998816 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f51c4078bc0 0x7f51c407b0d0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f51d0034000 tx=0x7f51d0004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:40.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:40.748+0000 7f51d67fc640 1 -- 172.21.15.38:0/3737998816 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f51dc063df0 con 0x7f51e00b9ad0 2024-09-21T03:42:40.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:40.976+0000 7f51e7be4640 1 -- 172.21.15.38:0/3737998816 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f51e00ced30 con 0x7f51e00b9ad0 2024-09-21T03:42:41.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:41.369+0000 7f51d67fc640 1 -- 172.21.15.38:0/3737998816 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v228) ==== 161+0+0 (secure 0 0 0) 0x7f51dc0636f0 con 0x7f51e00b9ad0 2024-09-21T03:42:41.372 INFO:teuthology.orchestra.run.smithi038.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-21T03:42:41.374 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:41.372+0000 7f51e7be4640 1 -- 172.21.15.38:0/3737998816 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f51c4078bc0 msgr2=0x7f51c407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:41.374 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:41.372+0000 7f51e7be4640 1 --2- 172.21.15.38:0/3737998816 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f51c4078bc0 0x7f51c407b0d0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f51d0034000 tx=0x7f51d0004730 comp rx=0 tx=0).stop 2024-09-21T03:42:41.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:41.372+0000 7f51e7be4640 1 -- 172.21.15.38:0/3737998816 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f51e00b9ad0 msgr2=0x7f51e01f53b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:41.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:41.372+0000 7f51e7be4640 1 --2- 172.21.15.38:0/3737998816 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f51e00b9ad0 0x7f51e01f53b0 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7f51dc00ece0 tx=0x7f51dc00c6a0 comp rx=0 tx=0).stop 2024-09-21T03:42:41.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:41.372+0000 7f51e7be4640 1 -- 172.21.15.38:0/3737998816 shutdown_connections 2024-09-21T03:42:41.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:41.372+0000 7f51e7be4640 1 --2- 172.21.15.38:0/3737998816 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f51e015ee80 0x7f51e014f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:41.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:41.372+0000 7f51e7be4640 1 --2- 172.21.15.38:0/3737998816 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f51c4078bc0 0x7f51c407b0d0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:41.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:41.372+0000 7f51e7be4640 1 --2- 172.21.15.38:0/3737998816 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f51e00d1120 0x7f51e006e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:41.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:41.372+0000 7f51e7be4640 1 --2- 172.21.15.38:0/3737998816 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f51e00b9ad0 0x7f51e01f53b0 unknown :-1 s=CLOSED pgs=1816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:41.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:41.372+0000 7f51e7be4640 1 -- 172.21.15.38:0/3737998816 >> 172.21.15.38:0/3737998816 conn(0x7f51e009d830 msgr2=0x7f51e00b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:41.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:41.372+0000 7f51e7be4640 1 -- 172.21.15.38:0/3737998816 shutdown_connections 2024-09-21T03:42:41.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:41.372+0000 7f51e7be4640 1 -- 172.21.15.38:0/3737998816 wait complete. 2024-09-21T03:42:41.387 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-09-21T03:42:41.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:41.504+0000 7f988d58b640 1 -- 172.21.15.38:0/912660600 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f988806e500 msgr2=0x7f98880b5d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:41.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:41.504+0000 7f988d58b640 1 --2- 172.21.15.38:0/912660600 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f988806e500 0x7f98880b5d20 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7f9880005fd0 tx=0x7f988002a820 comp rx=0 tx=0).stop 2024-09-21T03:42:41.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:41.504+0000 7f988d58b640 1 -- 172.21.15.38:0/912660600 shutdown_connections 2024-09-21T03:42:41.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:41.504+0000 7f988d58b640 1 --2- 172.21.15.38:0/912660600 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f988806e500 0x7f98880b5d20 unknown :-1 s=CLOSED pgs=1818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:41.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:41.504+0000 7f988d58b640 1 -- 172.21.15.38:0/912660600 >> 172.21.15.38:0/912660600 conn(0x7f98880735d0 msgr2=0x7f98880739e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:41.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:41.504+0000 7f988d58b640 1 -- 172.21.15.38:0/912660600 shutdown_connections 2024-09-21T03:42:41.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:41.504+0000 7f988d58b640 1 -- 172.21.15.38:0/912660600 wait complete. 2024-09-21T03:42:41.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:41.506+0000 7f988d58b640 1 Processor -- start 2024-09-21T03:42:41.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:41.506+0000 7f988d58b640 1 -- start start 2024-09-21T03:42:41.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:41.508+0000 7f988d58b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f98880b6290 0x7f9888151d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:41.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:41.508+0000 7f988d58b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f988815b730 0x7f98880c4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:41.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:41.508+0000 7f988d58b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f98880ad5b0 0x7f98880c5270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:41.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:41.508+0000 7f988d58b640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f9888071a60 con 0x7f98880b6290 2024-09-21T03:42:41.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:41.508+0000 7f988d58b640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f98880718e0 con 0x7f988815b730 2024-09-21T03:42:41.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:41.508+0000 7f988d58b640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f9888071be0 con 0x7f98880ad5b0 2024-09-21T03:42:41.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:41.508+0000 7f9886ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f98880b6290 0x7f9888151d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:41.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:41.508+0000 7f9886ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f98880b6290 0x7f9888151d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:52396/0 (socket says 172.21.15.38:52396) 2024-09-21T03:42:41.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:41.508+0000 7f9886ffd640 1 -- 172.21.15.38:0/2066109098 learned_addr learned my addr 172.21.15.38:0/2066109098 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:42:41.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:41.508+0000 7f98867fc640 1 --2- 172.21.15.38:0/2066109098 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f988815b730 0x7f98880c4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:41.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:41.508+0000 7f98877fe640 1 --2- 172.21.15.38:0/2066109098 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f98880ad5b0 0x7f98880c5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:41.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:41.508+0000 7f9886ffd640 1 -- 172.21.15.38:0/2066109098 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f98880ad5b0 msgr2=0x7f98880c5270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:41.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:41.508+0000 7f9886ffd640 1 --2- 172.21.15.38:0/2066109098 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f98880ad5b0 0x7f98880c5270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:41.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:41.508+0000 7f9886ffd640 1 -- 172.21.15.38:0/2066109098 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f988815b730 msgr2=0x7f98880c4d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:41.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:41.508+0000 7f9886ffd640 1 --2- 172.21.15.38:0/2066109098 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f988815b730 0x7f98880c4d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:41.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:41.508+0000 7f9886ffd640 1 -- 172.21.15.38:0/2066109098 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98880ae750 con 0x7f98880b6290 2024-09-21T03:42:41.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:41.508+0000 7f98877fe640 1 --2- 172.21.15.38:0/2066109098 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f98880ad5b0 0x7f98880c5270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:42:41.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:41.508+0000 7f9886ffd640 1 --2- 172.21.15.38:0/2066109098 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f98880b6290 0x7f9888151d10 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7f987800b410 tx=0x7f987800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:41.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:41.508+0000 7f986f7fe640 1 -- 172.21.15.38:0/2066109098 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9878015020 con 0x7f98880b6290 2024-09-21T03:42:41.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:41.508+0000 7f988d58b640 1 -- 172.21.15.38:0/2066109098 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98880aea30 con 0x7f98880b6290 2024-09-21T03:42:41.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:41.508+0000 7f986f7fe640 1 -- 172.21.15.38:0/2066109098 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9878011920 con 0x7f98880b6290 2024-09-21T03:42:41.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:41.509+0000 7f986f7fe640 1 -- 172.21.15.38:0/2066109098 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9878011d70 con 0x7f98880b6290 2024-09-21T03:42:41.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:41.509+0000 7f988d58b640 1 -- 172.21.15.38:0/2066109098 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98880aeff0 con 0x7f98880b6290 2024-09-21T03:42:41.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:41.510+0000 7f988d58b640 1 -- 172.21.15.38:0/2066109098 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98880b5d20 con 0x7f98880b6290 2024-09-21T03:42:41.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:41.510+0000 7f986f7fe640 1 -- 172.21.15.38:0/2066109098 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f9878018020 con 0x7f98880b6290 2024-09-21T03:42:41.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:41.511+0000 7f986f7fe640 1 --2- 172.21.15.38:0/2066109098 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9874078bc0 0x7f987407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:41.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:41.511+0000 7f986f7fe640 1 -- 172.21.15.38:0/2066109098 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 8368+0+0 (secure 0 0 0) 0x7f987809b130 con 0x7f98880b6290 2024-09-21T03:42:41.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:41.511+0000 7f98867fc640 1 --2- 172.21.15.38:0/2066109098 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9874078bc0 0x7f987407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:41.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:41.512+0000 7f98867fc640 1 --2- 172.21.15.38:0/2066109098 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9874078bc0 0x7f987407b0d0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f98880c4290 tx=0x7f987c007e90 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:41.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:41.515+0000 7f986f7fe640 1 -- 172.21.15.38:0/2066109098 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f987801f070 con 0x7f98880b6290 2024-09-21T03:42:41.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:41.783+0000 7f988d58b640 1 -- 172.21.15.38:0/2066109098 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7f98880b0a70 con 0x7f98880b6290 2024-09-21T03:42:42.374 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.371+0000 7f986f7fe640 1 -- 172.21.15.38:0/2066109098 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v229) ==== 101+0+0 (secure 0 0 0) 0x7f9878062ad0 con 0x7f98880b6290 2024-09-21T03:42:42.375 INFO:teuthology.orchestra.run.smithi038.stderr:repaired: restoring rank 6:0 2024-09-21T03:42:42.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.374+0000 7f988d58b640 1 -- 172.21.15.38:0/2066109098 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9874078bc0 msgr2=0x7f987407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:42.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.375+0000 7f988d58b640 1 --2- 172.21.15.38:0/2066109098 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9874078bc0 0x7f987407b0d0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f98880c4290 tx=0x7f987c007e90 comp rx=0 tx=0).stop 2024-09-21T03:42:42.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.375+0000 7f988d58b640 1 -- 172.21.15.38:0/2066109098 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f98880b6290 msgr2=0x7f9888151d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:42.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.375+0000 7f988d58b640 1 --2- 172.21.15.38:0/2066109098 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f98880b6290 0x7f9888151d10 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7f987800b410 tx=0x7f987800b8d0 comp rx=0 tx=0).stop 2024-09-21T03:42:42.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.375+0000 7f988d58b640 1 -- 172.21.15.38:0/2066109098 shutdown_connections 2024-09-21T03:42:42.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.375+0000 7f988d58b640 1 --2- 172.21.15.38:0/2066109098 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9874078bc0 0x7f987407b0d0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:42.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.375+0000 7f988d58b640 1 --2- 172.21.15.38:0/2066109098 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f98880ad5b0 0x7f98880c5270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:42.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.375+0000 7f988d58b640 1 --2- 172.21.15.38:0/2066109098 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f988815b730 0x7f98880c4d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:42.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.375+0000 7f988d58b640 1 --2- 172.21.15.38:0/2066109098 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f98880b6290 0x7f9888151d10 unknown :-1 s=CLOSED pgs=1819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:42.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.375+0000 7f988d58b640 1 -- 172.21.15.38:0/2066109098 >> 172.21.15.38:0/2066109098 conn(0x7f98880735d0 msgr2=0x7f98880ce4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:42.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.375+0000 7f988d58b640 1 -- 172.21.15.38:0/2066109098 shutdown_connections 2024-09-21T03:42:42.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.375+0000 7f988d58b640 1 -- 172.21.15.38:0/2066109098 wait complete. 2024-09-21T03:42:42.392 DEBUG:teuthology.orchestra.run.smithi038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-09-21T03:42:42.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.450+0000 7fe1ebb8bf40 1 Processor -- start 2024-09-21T03:42:42.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.450+0000 7fe1ebb8bf40 1 -- start start 2024-09-21T03:42:42.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.451+0000 7fe1ebb8bf40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5626e99ac090 0x5626e9a32850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:42.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.451+0000 7fe1ebb8bf40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5626e9a21520 0x5626e998e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:42.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.451+0000 7fe1ebb8bf40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5626e9a19bb0 0x5626e9a35db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:42.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.451+0000 7fe1ebb8bf40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5626e98b2d60 con 0x5626e9a19bb0 2024-09-21T03:42:42.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.451+0000 7fe1ebb8bf40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5626e9911430 con 0x5626e9a21520 2024-09-21T03:42:42.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.451+0000 7fe1ebb8bf40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x5626e9a33010 con 0x5626e99ac090 2024-09-21T03:42:42.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.451+0000 7fe1e5e46640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5626e9a19bb0 0x5626e9a35db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:42.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.451+0000 7fe1e5e46640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5626e9a19bb0 0x5626e9a35db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38126/0 (socket says 172.21.15.38:38126) 2024-09-21T03:42:42.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.451+0000 7fe1e5e46640 1 -- 172.21.15.38:0/2988128438 learned_addr learned my addr 172.21.15.38:0/2988128438 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:42:42.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.451+0000 7fe1e4e44640 1 --2- 172.21.15.38:0/2988128438 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5626e9a21520 0x5626e998e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:42.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.451+0000 7fe1e5e46640 1 -- 172.21.15.38:0/2988128438 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5626e99ac090 msgr2=0x5626e9a32850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:42.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.451+0000 7fe1e5645640 1 --2- 172.21.15.38:0/2988128438 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5626e99ac090 0x5626e9a32850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:42.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.451+0000 7fe1e5e46640 1 --2- 172.21.15.38:0/2988128438 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5626e99ac090 0x5626e9a32850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:42.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.451+0000 7fe1e5e46640 1 -- 172.21.15.38:0/2988128438 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5626e9a21520 msgr2=0x5626e998e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:42.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.451+0000 7fe1e5e46640 1 --2- 172.21.15.38:0/2988128438 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5626e9a21520 0x5626e998e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:42.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.451+0000 7fe1e5e46640 1 -- 172.21.15.38:0/2988128438 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5626e9a31df0 con 0x5626e9a19bb0 2024-09-21T03:42:42.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.451+0000 7fe1e4e44640 1 --2- 172.21.15.38:0/2988128438 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5626e9a21520 0x5626e998e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:42:42.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.452+0000 7fe1e5e46640 1 --2- 172.21.15.38:0/2988128438 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5626e9a19bb0 0x5626e9a35db0 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7fe1e000c500 tx=0x7fe1e000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f67b9f1f022caaed server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:42.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.452+0000 7fe1d77fe640 1 -- 172.21.15.38:0/2988128438 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1e000a980 con 0x5626e9a19bb0 2024-09-21T03:42:42.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.452+0000 7fe1d77fe640 1 -- 172.21.15.38:0/2988128438 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe1e000ab20 con 0x5626e9a19bb0 2024-09-21T03:42:42.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.452+0000 7fe1d77fe640 1 -- 172.21.15.38:0/2988128438 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1e0008420 con 0x5626e9a19bb0 2024-09-21T03:42:42.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.452+0000 7fe1ebb8bf40 1 -- 172.21.15.38:0/2988128438 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5626e9a19bb0 msgr2=0x5626e9a35db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:42.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.452+0000 7fe1ebb8bf40 1 --2- 172.21.15.38:0/2988128438 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5626e9a19bb0 0x5626e9a35db0 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7fe1e000c500 tx=0x7fe1e000c9c0 comp rx=0 tx=0).stop 2024-09-21T03:42:42.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.453+0000 7fe1ebb8bf40 1 -- 172.21.15.38:0/2988128438 shutdown_connections 2024-09-21T03:42:42.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.453+0000 7fe1ebb8bf40 1 --2- 172.21.15.38:0/2988128438 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5626e9a19bb0 0x5626e9a35db0 unknown :-1 s=CLOSED pgs=1821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:42.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.453+0000 7fe1ebb8bf40 1 --2- 172.21.15.38:0/2988128438 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5626e9a21520 0x5626e998e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:42.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.453+0000 7fe1ebb8bf40 1 --2- 172.21.15.38:0/2988128438 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5626e99ac090 0x5626e9a32850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:42.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.453+0000 7fe1ebb8bf40 1 -- 172.21.15.38:0/2988128438 >> 172.21.15.38:0/2988128438 conn(0x5626e9883ac0 msgr2=0x5626e99aa760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:42.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.453+0000 7fe1ebb8bf40 1 -- 172.21.15.38:0/2988128438 shutdown_connections 2024-09-21T03:42:42.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.453+0000 7fe1ebb8bf40 1 -- 172.21.15.38:0/2988128438 wait complete. 2024-09-21T03:42:42.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.453+0000 7fe1ebb8bf40 1 Processor -- start 2024-09-21T03:42:42.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.454+0000 7fe1ebb8bf40 1 -- start start 2024-09-21T03:42:42.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.455+0000 7fe1ebb8bf40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5626e99ac090 0x5626e9ad5890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:42.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.455+0000 7fe1ebb8bf40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5626e9a19bb0 0x5626e9a18f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:42.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.455+0000 7fe1ebb8bf40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5626e9a21520 0x5626e9ad1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:42.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.455+0000 7fe1ebb8bf40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5626e9a39810 con 0x5626e9a19bb0 2024-09-21T03:42:42.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.455+0000 7fe1ebb8bf40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5626e9a39af0 con 0x5626e99ac090 2024-09-21T03:42:42.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.455+0000 7fe1ebb8bf40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x5626e9ad2520 con 0x5626e9a21520 2024-09-21T03:42:42.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.455+0000 7fe1e4e44640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5626e9a19bb0 0x5626e9a18f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:42.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.455+0000 7fe1e4e44640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5626e9a19bb0 0x5626e9a18f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38140/0 (socket says 172.21.15.38:38140) 2024-09-21T03:42:42.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.455+0000 7fe1e4e44640 1 -- 172.21.15.38:0/1811344108 learned_addr learned my addr 172.21.15.38:0/1811344108 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:42:42.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.455+0000 7fe1e5645640 1 --2- 172.21.15.38:0/1811344108 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5626e99ac090 0x5626e9ad5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:42.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.455+0000 7fe1e5e46640 1 --2- 172.21.15.38:0/1811344108 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5626e9a21520 0x5626e9ad1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:42.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.455+0000 7fe1e4e44640 1 -- 172.21.15.38:0/1811344108 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5626e9a21520 msgr2=0x5626e9ad1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:42.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.455+0000 7fe1e4e44640 1 --2- 172.21.15.38:0/1811344108 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5626e9a21520 0x5626e9ad1720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:42.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.455+0000 7fe1e4e44640 1 -- 172.21.15.38:0/1811344108 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5626e99ac090 msgr2=0x5626e9ad5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:42.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.455+0000 7fe1e4e44640 1 --2- 172.21.15.38:0/1811344108 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5626e99ac090 0x5626e9ad5890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:42.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.455+0000 7fe1e4e44640 1 -- 172.21.15.38:0/1811344108 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5626e99917d0 con 0x5626e9a19bb0 2024-09-21T03:42:42.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.455+0000 7fe1e4e44640 1 --2- 172.21.15.38:0/1811344108 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5626e9a19bb0 0x5626e9a18f50 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7fe1dc001eb0 tx=0x7fe1dc00a5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:42.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.456+0000 7fe1d67fc640 1 -- 172.21.15.38:0/1811344108 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1dc015030 con 0x5626e9a19bb0 2024-09-21T03:42:42.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.456+0000 7fe1d67fc640 1 -- 172.21.15.38:0/1811344108 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe1dc010470 con 0x5626e9a19bb0 2024-09-21T03:42:42.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.456+0000 7fe1d67fc640 1 -- 172.21.15.38:0/1811344108 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1dc010680 con 0x5626e9a19bb0 2024-09-21T03:42:42.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.456+0000 7fe1ebb8bf40 1 -- 172.21.15.38:0/1811344108 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5626e9ad4db0 con 0x5626e9a19bb0 2024-09-21T03:42:42.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.456+0000 7fe1ebb8bf40 1 -- 172.21.15.38:0/1811344108 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5626e9ad5370 con 0x5626e9a19bb0 2024-09-21T03:42:42.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.458+0000 7fe1d67fc640 1 -- 172.21.15.38:0/1811344108 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fe1dc018070 con 0x5626e9a19bb0 2024-09-21T03:42:42.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.458+0000 7fe1d67fc640 1 --2- 172.21.15.38:0/1811344108 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fe1c8077570 0x7fe1c8079a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:42.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.458+0000 7fe1d67fc640 1 -- 172.21.15.38:0/1811344108 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 8368+0+0 (secure 0 0 0) 0x7fe1dc099ec0 con 0x5626e9a19bb0 2024-09-21T03:42:42.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.458+0000 7fe1e5645640 1 --2- 172.21.15.38:0/1811344108 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fe1c8077570 0x7fe1c8079a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:42.463 INFO:teuthology.orchestra.run.smithi038.stdout:Importing pool 2024-09-21T03:42:42.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.458+0000 7fe1ebb8bf40 1 --2- 172.21.15.38:0/1811344108 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x5626e9a32300 0x5626e9a35db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:42.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.458+0000 7fe1ebb8bf40 1 -- 172.21.15.38:0/1811344108 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x5626e9a362f0 con 0x5626e9a32300 2024-09-21T03:42:42.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.459+0000 7fe1e5e46640 1 --2- 172.21.15.38:0/1811344108 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x5626e9a32300 0x5626e9a35db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:42.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.459+0000 7fe1e5645640 1 --2- 172.21.15.38:0/1811344108 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fe1c8077570 0x7fe1c8079a80 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fe1d8056000 tx=0x7fe1d80033a0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:42.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.459+0000 7fe1e5e46640 1 --2- 172.21.15.38:0/1811344108 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x5626e9a32300 0x5626e9a35db0 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:42.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.461+0000 7fe1e5e46640 1 -- 172.21.15.38:0/1811344108 <== osd.7 v2:172.21.15.195:6818/1606106342 1 ==== osd_op_reply(1 601.00000000 [create] v120'722 uv696 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe1e000aa20 con 0x5626e9a32300 2024-09-21T03:42:42.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.461+0000 7fe1ebb8bf40 1 -- 172.21.15.38:0/1811344108 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e120) -- 0x5626e9addea0 con 0x5626e9a32300 2024-09-21T03:42:42.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.463+0000 7fe1e5e46640 1 -- 172.21.15.38:0/1811344108 <== osd.7 v2:172.21.15.195:6818/1606106342 2 ==== osd_op_reply(2 601.00000000 [delete] v120'723 uv723 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe1e000aa20 con 0x5626e9a32300 2024-09-21T03:42:42.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.463+0000 7fe1ebb8bf40 1 -- 172.21.15.38:0/1811344108 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x5626e9ade240 con 0x5626e9a32300 2024-09-21T03:42:42.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.465+0000 7fe1e5e46640 1 -- 172.21.15.38:0/1811344108 <== osd.7 v2:172.21.15.195:6818/1606106342 3 ==== osd_op_reply(3 601.00000000 [create] v120'724 uv724 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe1e000aa20 con 0x5626e9a32300 2024-09-21T03:42:42.468 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-09-21T03:42:42.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.465+0000 7fe1ebb8bf40 1 -- 172.21.15.38:0/1811344108 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x5626e9adf5f0 con 0x5626e9a32300 2024-09-21T03:42:42.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.466+0000 7fe1e5e46640 1 -- 172.21.15.38:0/1811344108 <== osd.7 v2:172.21.15.195:6818/1606106342 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v120'725 uv725 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe1e000aa20 con 0x5626e9a32300 2024-09-21T03:42:42.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.466+0000 7fe1ebb8bf40 1 -- 172.21.15.38:0/1811344108 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x5626e9adf990 con 0x5626e9a32300 2024-09-21T03:42:42.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.468+0000 7fe1e5e46640 1 -- 172.21.15.38:0/1811344108 <== osd.7 v2:172.21.15.195:6818/1606106342 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v120'726 uv726 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe1e000aa20 con 0x5626e9a32300 2024-09-21T03:42:42.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.468+0000 7fe1ebb8bf40 1 -- 172.21.15.38:0/1811344108 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x5626e9ade5e0 con 0x5626e9a32300 2024-09-21T03:42:42.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.470+0000 7fe1e5e46640 1 -- 172.21.15.38:0/1811344108 <== osd.7 v2:172.21.15.195:6818/1606106342 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v120'727 uv727 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe1e000aa20 con 0x5626e9a32300 2024-09-21T03:42:42.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.470+0000 7fe1ebb8bf40 1 -- 172.21.15.38:0/1811344108 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x5626e9ade9d0 con 0x5626e9a32300 2024-09-21T03:42:42.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.471+0000 7fe1e5e46640 1 -- 172.21.15.38:0/1811344108 <== osd.7 v2:172.21.15.195:6818/1606106342 7 ==== osd_op_reply(7 602.00000000 [create] v120'728 uv702 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe1e000aa20 con 0x5626e9a32300 2024-09-21T03:42:42.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.471+0000 7fe1ebb8bf40 1 -- 172.21.15.38:0/1811344108 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e120) -- 0x5626e9adedc0 con 0x5626e9a32300 2024-09-21T03:42:42.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.473+0000 7fe1e5e46640 1 -- 172.21.15.38:0/1811344108 <== osd.7 v2:172.21.15.195:6818/1606106342 8 ==== osd_op_reply(8 602.00000000 [delete] v120'729 uv729 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe1e000aa20 con 0x5626e9a32300 2024-09-21T03:42:42.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.473+0000 7fe1ebb8bf40 1 -- 172.21.15.38:0/1811344108 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x5626e9adedc0 con 0x5626e9a32300 2024-09-21T03:42:42.477 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-09-21T03:42:42.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.475+0000 7fe1e5e46640 1 -- 172.21.15.38:0/1811344108 <== osd.7 v2:172.21.15.195:6818/1606106342 9 ==== osd_op_reply(9 602.00000000 [create] v120'730 uv730 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe1e000aa20 con 0x5626e9a32300 2024-09-21T03:42:42.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.475+0000 7fe1ebb8bf40 1 -- 172.21.15.38:0/1811344108 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x5626e9aded70 con 0x5626e9a32300 2024-09-21T03:42:42.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.476+0000 7fe1e5e46640 1 -- 172.21.15.38:0/1811344108 <== osd.7 v2:172.21.15.195:6818/1606106342 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v120'731 uv731 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe1e000aa20 con 0x5626e9a32300 2024-09-21T03:42:42.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.476+0000 7fe1ebb8bf40 1 -- 172.21.15.38:0/1811344108 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x5626e9aded70 con 0x5626e9a32300 2024-09-21T03:42:42.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.479+0000 7fe1e5e46640 1 -- 172.21.15.38:0/1811344108 <== osd.7 v2:172.21.15.195:6818/1606106342 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v120'732 uv732 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe1e000aa20 con 0x5626e9a32300 2024-09-21T03:42:42.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.479+0000 7fe1ebb8bf40 1 -- 172.21.15.38:0/1811344108 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x5626e9aded70 con 0x5626e9a32300 2024-09-21T03:42:42.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.480+0000 7fe1e5e46640 1 -- 172.21.15.38:0/1811344108 <== osd.7 v2:172.21.15.195:6818/1606106342 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v120'733 uv733 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe1e000aa20 con 0x5626e9a32300 2024-09-21T03:42:42.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.480+0000 7fe1ebb8bf40 1 -- 172.21.15.38:0/1811344108 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:13 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x5626e9aded70 con 0x5626e9a32300 2024-09-21T03:42:42.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.481+0000 7fe1e5e46640 1 -- 172.21.15.38:0/1811344108 <== osd.7 v2:172.21.15.195:6818/1606106342 13 ==== osd_op_reply(13 10000000000.00000000 [create] v120'734 uv709 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7fe1e000aa20 con 0x5626e9a32300 2024-09-21T03:42:42.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.482+0000 7fe1ebb8bf40 1 -- 172.21.15.38:0/1811344108 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:14 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e120) -- 0x5626e9aded70 con 0x5626e9a32300 2024-09-21T03:42:42.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.484+0000 7fe1e5e46640 1 -- 172.21.15.38:0/1811344108 <== osd.7 v2:172.21.15.195:6818/1606106342 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v120'735 uv735 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fe1e000aa20 con 0x5626e9a32300 2024-09-21T03:42:42.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.484+0000 7fe1ebb8bf40 1 -- 172.21.15.38:0/1811344108 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:15 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x5626e9aded70 con 0x5626e9a32300 2024-09-21T03:42:42.487 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-09-21T03:42:42.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.485+0000 7fe1e5e46640 1 -- 172.21.15.38:0/1811344108 <== osd.7 v2:172.21.15.195:6818/1606106342 15 ==== osd_op_reply(15 10000000000.00000000 [create] v120'736 uv736 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fe1e000aa20 con 0x5626e9a32300 2024-09-21T03:42:42.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.485+0000 7fe1ebb8bf40 1 -- 172.21.15.38:0/1811344108 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:16 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x5626e9aded70 con 0x5626e9a32300 2024-09-21T03:42:42.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.486+0000 7fe1e5e46640 1 -- 172.21.15.38:0/1811344108 <== osd.7 v2:172.21.15.195:6818/1606106342 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v120'737 uv737 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fe1e000aa20 con 0x5626e9a32300 2024-09-21T03:42:42.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.486+0000 7fe1ebb8bf40 1 -- 172.21.15.38:0/1811344108 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:17 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x5626e9aded70 con 0x5626e9a32300 2024-09-21T03:42:42.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.488+0000 7fe1e5e46640 1 -- 172.21.15.38:0/1811344108 <== osd.7 v2:172.21.15.195:6818/1606106342 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v120'738 uv738 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fe1e000aa20 con 0x5626e9a32300 2024-09-21T03:42:42.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.488+0000 7fe1ebb8bf40 1 -- 172.21.15.38:0/1811344108 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:18 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x5626e9aded70 con 0x5626e9a32300 2024-09-21T03:42:42.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.489+0000 7fe1e5e46640 1 -- 172.21.15.38:0/1811344108 <== osd.7 v2:172.21.15.195:6818/1606106342 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v120'739 uv739 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fe1e000aa20 con 0x5626e9a32300 2024-09-21T03:42:42.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.489+0000 7fe1ebb8bf40 1 -- 172.21.15.38:0/1811344108 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:19 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=585b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x5626e9aded70 con 0x5626e9a32300 2024-09-21T03:42:42.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.491+0000 7fe1e5e46640 1 -- 172.21.15.38:0/1811344108 <== osd.7 v2:172.21.15.195:6818/1606106342 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v120'740 uv740 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fe1e000aa20 con 0x5626e9a32300 2024-09-21T03:42:42.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.491+0000 7fe1ebb8bf40 1 -- 172.21.15.38:0/1811344108 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:20 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x5626e9aded70 con 0x5626e9a32300 2024-09-21T03:42:42.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.492+0000 7fe1e5e46640 1 -- 172.21.15.38:0/1811344108 <== osd.7 v2:172.21.15.195:6818/1606106342 20 ==== osd_op_reply(20 600.00000000 [create] v120'741 uv715 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe1e000aa20 con 0x5626e9a32300 2024-09-21T03:42:42.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.492+0000 7fe1ebb8bf40 1 -- 172.21.15.38:0/1811344108 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:21 13.10 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e120) -- 0x5626e9aded70 con 0x5626e9a32300 2024-09-21T03:42:42.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.494+0000 7fe1e5e46640 1 -- 172.21.15.38:0/1811344108 <== osd.7 v2:172.21.15.195:6818/1606106342 21 ==== osd_op_reply(21 600.00000000 [delete] v120'742 uv742 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe1e000aa20 con 0x5626e9a32300 2024-09-21T03:42:42.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.494+0000 7fe1ebb8bf40 1 -- 172.21.15.38:0/1811344108 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:22 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x5626e9aded70 con 0x5626e9a32300 2024-09-21T03:42:42.498 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-09-21T03:42:42.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.495+0000 7fe1e5e46640 1 -- 172.21.15.38:0/1811344108 <== osd.7 v2:172.21.15.195:6818/1606106342 22 ==== osd_op_reply(22 600.00000000 [create] v120'743 uv743 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe1e000aa20 con 0x5626e9a32300 2024-09-21T03:42:42.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.495+0000 7fe1ebb8bf40 1 -- 172.21.15.38:0/1811344108 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:23 13.10 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x5626e9aded70 con 0x5626e9a32300 2024-09-21T03:42:42.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.497+0000 7fe1e5e46640 1 -- 172.21.15.38:0/1811344108 <== osd.7 v2:172.21.15.195:6818/1606106342 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v120'744 uv744 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe1e000aa20 con 0x5626e9a32300 2024-09-21T03:42:42.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.497+0000 7fe1ebb8bf40 1 -- 172.21.15.38:0/1811344108 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:24 13.10 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x5626e9aded70 con 0x5626e9a32300 2024-09-21T03:42:42.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.498+0000 7fe1e5e46640 1 -- 172.21.15.38:0/1811344108 <== osd.7 v2:172.21.15.195:6818/1606106342 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v120'745 uv745 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe1e000aa20 con 0x5626e9a32300 2024-09-21T03:42:42.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.498+0000 7fe1ebb8bf40 1 -- 172.21.15.38:0/1811344108 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:25 13.10 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x5626e9aded70 con 0x5626e9a32300 2024-09-21T03:42:42.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.500+0000 7fe1e5e46640 1 -- 172.21.15.38:0/1811344108 <== osd.7 v2:172.21.15.195:6818/1606106342 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v120'746 uv746 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe1e000aa20 con 0x5626e9a32300 2024-09-21T03:42:42.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.500+0000 7fe1ebb8bf40 1 -- 172.21.15.38:0/1811344108 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:26 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x5626e9aded70 con 0x5626e9a32300 2024-09-21T03:42:42.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.502+0000 7fe1e5e46640 1 -- 172.21.15.38:0/1811344108 <== osd.7 v2:172.21.15.195:6818/1606106342 26 ==== osd_op_reply(26 603.00000000 [create] v120'747 uv721 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe1e000aa20 con 0x5626e9a32300 2024-09-21T03:42:42.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.502+0000 7fe1ebb8bf40 1 -- 172.21.15.38:0/1811344108 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:27 13.10 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e120) -- 0x5626e9aded70 con 0x5626e9a32300 2024-09-21T03:42:42.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.504+0000 7fe1e5e46640 1 -- 172.21.15.38:0/1811344108 <== osd.7 v2:172.21.15.195:6818/1606106342 27 ==== osd_op_reply(27 603.00000000 [delete] v120'748 uv748 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe1e000aa20 con 0x5626e9a32300 2024-09-21T03:42:42.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.504+0000 7fe1ebb8bf40 1 -- 172.21.15.38:0/1811344108 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:28 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x5626e9aded70 con 0x5626e9a32300 2024-09-21T03:42:42.508 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-09-21T03:42:42.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.505+0000 7fe1e5e46640 1 -- 172.21.15.38:0/1811344108 <== osd.7 v2:172.21.15.195:6818/1606106342 28 ==== osd_op_reply(28 603.00000000 [create] v120'749 uv749 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe1e000aa20 con 0x5626e9a32300 2024-09-21T03:42:42.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.505+0000 7fe1ebb8bf40 1 -- 172.21.15.38:0/1811344108 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:29 13.10 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x5626e9aded70 con 0x5626e9a32300 2024-09-21T03:42:42.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.507+0000 7fe1e5e46640 1 -- 172.21.15.38:0/1811344108 <== osd.7 v2:172.21.15.195:6818/1606106342 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v120'750 uv750 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe1e000aa20 con 0x5626e9a32300 2024-09-21T03:42:42.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.507+0000 7fe1ebb8bf40 1 -- 172.21.15.38:0/1811344108 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:30 13.10 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x5626e9aded70 con 0x5626e9a32300 2024-09-21T03:42:42.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.509+0000 7fe1e5e46640 1 -- 172.21.15.38:0/1811344108 <== osd.7 v2:172.21.15.195:6818/1606106342 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v120'751 uv751 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe1e000aa20 con 0x5626e9a32300 2024-09-21T03:42:42.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.509+0000 7fe1ebb8bf40 1 -- 172.21.15.38:0/1811344108 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:31 13.10 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x5626e9aded70 con 0x5626e9a32300 2024-09-21T03:42:42.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.510+0000 7fe1e5e46640 1 -- 172.21.15.38:0/1811344108 <== osd.7 v2:172.21.15.195:6818/1606106342 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v120'752 uv752 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe1e000aa20 con 0x5626e9a32300 2024-09-21T03:42:42.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.510+0000 7fe1ebb8bf40 1 --2- 172.21.15.38:0/1811344108 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x5626e9adf150 0x5626e9ae2250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:42.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.510+0000 7fe1ebb8bf40 1 -- 172.21.15.38:0/1811344108 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x5626e9aded70 con 0x5626e9adf150 2024-09-21T03:42:42.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.510+0000 7fe1e5645640 1 --2- 172.21.15.38:0/1811344108 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x5626e9adf150 0x5626e9ae2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:42.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.510+0000 7fe1e5645640 1 --2- 172.21.15.38:0/1811344108 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x5626e9adf150 0x5626e9ae2250 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:42.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.512+0000 7fe1e5645640 1 -- 172.21.15.38:0/1811344108 <== osd.6 v2:172.21.15.195:6826/4055825675 1 ==== osd_op_reply(32 1.00000000.inode [create] v120'526 uv525 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7fe1d800b040 con 0x5626e9adf150 2024-09-21T03:42:42.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.512+0000 7fe1ebb8bf40 1 -- 172.21.15.38:0/1811344108 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e120) -- 0x5626e9ae2790 con 0x5626e9adf150 2024-09-21T03:42:42.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.514+0000 7fe1e5645640 1 -- 172.21.15.38:0/1811344108 <== osd.6 v2:172.21.15.195:6826/4055825675 2 ==== osd_op_reply(33 1.00000000.inode [delete] v120'527 uv527 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fe1d800b040 con 0x5626e9adf150 2024-09-21T03:42:42.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.514+0000 7fe1ebb8bf40 1 -- 172.21.15.38:0/1811344108 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x5626e9ae2b30 con 0x5626e9adf150 2024-09-21T03:42:42.518 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-09-21T03:42:42.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.515+0000 7fe1e5645640 1 -- 172.21.15.38:0/1811344108 <== osd.6 v2:172.21.15.195:6826/4055825675 3 ==== osd_op_reply(34 1.00000000.inode [create] v120'528 uv528 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fe1d800b040 con 0x5626e9adf150 2024-09-21T03:42:42.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.515+0000 7fe1ebb8bf40 1 -- 172.21.15.38:0/1811344108 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x5626e9ae2ef0 con 0x5626e9adf150 2024-09-21T03:42:42.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.516+0000 7fe1e5645640 1 -- 172.21.15.38:0/1811344108 <== osd.6 v2:172.21.15.195:6826/4055825675 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v120'529 uv529 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fe1d800b040 con 0x5626e9adf150 2024-09-21T03:42:42.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.517+0000 7fe1ebb8bf40 1 -- 172.21.15.38:0/1811344108 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x5626e9ae3320 con 0x5626e9adf150 2024-09-21T03:42:42.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.518+0000 7fe1e5645640 1 -- 172.21.15.38:0/1811344108 <== osd.6 v2:172.21.15.195:6826/4055825675 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v120'530 uv530 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fe1d800b040 con 0x5626e9adf150 2024-09-21T03:42:42.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.518+0000 7fe1ebb8bf40 1 -- 172.21.15.38:0/1811344108 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x5626e9ae36c0 con 0x5626e9adf150 2024-09-21T03:42:42.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.519+0000 7fe1e5645640 1 -- 172.21.15.38:0/1811344108 <== osd.6 v2:172.21.15.195:6826/4055825675 6 ==== osd_op_reply(37 200.00000000 [create] v120'531 uv513 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe1d800b040 con 0x5626e9adf150 2024-09-21T03:42:42.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.519+0000 7fe1ebb8bf40 1 -- 172.21.15.38:0/1811344108 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e120) -- 0x5626e9ae3ab0 con 0x5626e9adf150 2024-09-21T03:42:42.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.521+0000 7fe1e5645640 1 -- 172.21.15.38:0/1811344108 <== osd.6 v2:172.21.15.195:6826/4055825675 7 ==== osd_op_reply(38 200.00000000 [delete] v120'532 uv532 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe1d800b040 con 0x5626e9adf150 2024-09-21T03:42:42.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.521+0000 7fe1ebb8bf40 1 -- 172.21.15.38:0/1811344108 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x5626e9ae3ea0 con 0x5626e9adf150 2024-09-21T03:42:42.525 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-09-21T03:42:42.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.522+0000 7fe1e5645640 1 -- 172.21.15.38:0/1811344108 <== osd.6 v2:172.21.15.195:6826/4055825675 8 ==== osd_op_reply(39 200.00000000 [create] v120'533 uv533 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe1d800b040 con 0x5626e9adf150 2024-09-21T03:42:42.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.522+0000 7fe1ebb8bf40 1 -- 172.21.15.38:0/1811344108 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x5626e9ae4290 con 0x5626e9adf150 2024-09-21T03:42:42.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.524+0000 7fe1e5645640 1 -- 172.21.15.38:0/1811344108 <== osd.6 v2:172.21.15.195:6826/4055825675 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v120'534 uv534 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe1d800b040 con 0x5626e9adf150 2024-09-21T03:42:42.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.524+0000 7fe1ebb8bf40 1 -- 172.21.15.38:0/1811344108 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x5626e9ae4290 con 0x5626e9adf150 2024-09-21T03:42:42.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.525+0000 7fe1e5645640 1 -- 172.21.15.38:0/1811344108 <== osd.6 v2:172.21.15.195:6826/4055825675 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v120'535 uv535 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe1d800b040 con 0x5626e9adf150 2024-09-21T03:42:42.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.525+0000 7fe1ebb8bf40 1 -- 172.21.15.38:0/1811344108 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:42 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x5626e9ae3fd0 con 0x5626e9adf150 2024-09-21T03:42:42.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.526+0000 7fe1e5645640 1 -- 172.21.15.38:0/1811344108 <== osd.6 v2:172.21.15.195:6826/4055825675 11 ==== osd_op_reply(42 200.00000001 [create] v120'536 uv518 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe1d800b040 con 0x5626e9adf150 2024-09-21T03:42:42.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.526+0000 7fe1ebb8bf40 1 -- 172.21.15.38:0/1811344108 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:43 13.14 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e120) -- 0x5626e9ae3f80 con 0x5626e9adf150 2024-09-21T03:42:42.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.528+0000 7fe1e5645640 1 -- 172.21.15.38:0/1811344108 <== osd.6 v2:172.21.15.195:6826/4055825675 12 ==== osd_op_reply(43 200.00000001 [delete] v120'537 uv537 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe1d800b040 con 0x5626e9adf150 2024-09-21T03:42:42.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.528+0000 7fe1ebb8bf40 1 -- 172.21.15.38:0/1811344108 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:44 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x5626e9ae3f80 con 0x5626e9adf150 2024-09-21T03:42:42.532 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-09-21T03:42:42.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.529+0000 7fe1e5645640 1 -- 172.21.15.38:0/1811344108 <== osd.6 v2:172.21.15.195:6826/4055825675 13 ==== osd_op_reply(44 200.00000001 [create] v120'538 uv538 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe1d800b040 con 0x5626e9adf150 2024-09-21T03:42:42.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.529+0000 7fe1ebb8bf40 1 -- 172.21.15.38:0/1811344108 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:45 13.14 13:2e2fa760:::200.00000001:head [write 0~17490 in=17490b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x5626e9ae3f80 con 0x5626e9adf150 2024-09-21T03:42:42.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.531+0000 7fe1e5645640 1 -- 172.21.15.38:0/1811344108 <== osd.6 v2:172.21.15.195:6826/4055825675 14 ==== osd_op_reply(45 200.00000001 [write 0~17490] v120'539 uv539 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe1d800b040 con 0x5626e9adf150 2024-09-21T03:42:42.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.531+0000 7fe1ebb8bf40 1 -- 172.21.15.38:0/1811344108 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:46 13.14 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x5626e9ae3f80 con 0x5626e9adf150 2024-09-21T03:42:42.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.532+0000 7fe1e5645640 1 -- 172.21.15.38:0/1811344108 <== osd.6 v2:172.21.15.195:6826/4055825675 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v120'540 uv540 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe1d800b040 con 0x5626e9adf150 2024-09-21T03:42:42.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.532+0000 7fe1ebb8bf40 1 -- 172.21.15.38:0/1811344108 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:47 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x5626e9ae3f80 con 0x5626e9adf150 2024-09-21T03:42:42.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.534+0000 7fe1e5645640 1 -- 172.21.15.38:0/1811344108 <== osd.6 v2:172.21.15.195:6826/4055825675 16 ==== osd_op_reply(47 606.00000000 [create] v120'541 uv524 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe1d800b040 con 0x5626e9adf150 2024-09-21T03:42:42.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.534+0000 7fe1ebb8bf40 1 -- 172.21.15.38:0/1811344108 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:48 13.14 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e120) -- 0x5626e9ae3f80 con 0x5626e9adf150 2024-09-21T03:42:42.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.535+0000 7fe1e5645640 1 -- 172.21.15.38:0/1811344108 <== osd.6 v2:172.21.15.195:6826/4055825675 17 ==== osd_op_reply(48 606.00000000 [delete] v120'542 uv542 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe1d800b040 con 0x5626e9adf150 2024-09-21T03:42:42.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.535+0000 7fe1ebb8bf40 1 -- 172.21.15.38:0/1811344108 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:49 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x5626e9ae3f80 con 0x5626e9adf150 2024-09-21T03:42:42.539 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-09-21T03:42:42.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.536+0000 7fe1e5645640 1 -- 172.21.15.38:0/1811344108 <== osd.6 v2:172.21.15.195:6826/4055825675 18 ==== osd_op_reply(49 606.00000000 [create] v120'543 uv543 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe1d800b040 con 0x5626e9adf150 2024-09-21T03:42:42.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.536+0000 7fe1ebb8bf40 1 -- 172.21.15.38:0/1811344108 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:50 13.14 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x5626e9ae3f80 con 0x5626e9adf150 2024-09-21T03:42:42.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.537+0000 7fe1e5645640 1 -- 172.21.15.38:0/1811344108 <== osd.6 v2:172.21.15.195:6826/4055825675 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v120'544 uv544 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe1d800b040 con 0x5626e9adf150 2024-09-21T03:42:42.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.537+0000 7fe1ebb8bf40 1 -- 172.21.15.38:0/1811344108 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:51 13.14 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x5626e9ae3f80 con 0x5626e9adf150 2024-09-21T03:42:42.541 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.539+0000 7fe1e5645640 1 -- 172.21.15.38:0/1811344108 <== osd.6 v2:172.21.15.195:6826/4055825675 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v120'545 uv545 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe1d800b040 con 0x5626e9adf150 2024-09-21T03:42:42.541 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.539+0000 7fe1ebb8bf40 1 -- 172.21.15.38:0/1811344108 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:52 13.14 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x5626e9ae3f80 con 0x5626e9adf150 2024-09-21T03:42:42.543 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.540+0000 7fe1e5645640 1 -- 172.21.15.38:0/1811344108 <== osd.6 v2:172.21.15.195:6826/4055825675 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v120'546 uv546 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe1d800b040 con 0x5626e9adf150 2024-09-21T03:42:42.543 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.540+0000 7fe1ebb8bf40 1 --2- 172.21.15.38:0/1811344108 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x5626e9ae48a0 0x5626e9ae4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:42.543 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.540+0000 7fe1ebb8bf40 1 -- 172.21.15.38:0/1811344108 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:53 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x5626e9ae73d0 con 0x5626e9ae48a0 2024-09-21T03:42:42.543 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.540+0000 7fe1e4e44640 1 --2- 172.21.15.38:0/1811344108 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x5626e9ae48a0 0x5626e9ae4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:42.544 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.541+0000 7fe1e4e44640 1 --2- 172.21.15.38:0/1811344108 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x5626e9ae48a0 0x5626e9ae4cb0 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:42.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.542+0000 7fe1e4e44640 1 -- 172.21.15.38:0/1811344108 <== osd.0 v2:172.21.15.38:6800/2381639433 1 ==== osd_op_reply(53 607.00000000 [create] v120'141 uv140 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe1dc05d550 con 0x5626e9ae48a0 2024-09-21T03:42:42.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.542+0000 7fe1ebb8bf40 1 -- 172.21.15.38:0/1811344108 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:54 13.1c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e120) -- 0x5626e9ae7770 con 0x5626e9ae48a0 2024-09-21T03:42:42.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.544+0000 7fe1e4e44640 1 -- 172.21.15.38:0/1811344108 <== osd.0 v2:172.21.15.38:6800/2381639433 2 ==== osd_op_reply(54 607.00000000 [delete] v120'142 uv142 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe1dc05d550 con 0x5626e9ae48a0 2024-09-21T03:42:42.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.544+0000 7fe1ebb8bf40 1 -- 172.21.15.38:0/1811344108 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:55 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x5626e9ae7b90 con 0x5626e9ae48a0 2024-09-21T03:42:42.547 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.545+0000 7fe1e4e44640 1 -- 172.21.15.38:0/1811344108 <== osd.0 v2:172.21.15.38:6800/2381639433 3 ==== osd_op_reply(55 607.00000000 [create] v120'143 uv143 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe1dc05d550 con 0x5626e9ae48a0 2024-09-21T03:42:42.548 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-09-21T03:42:42.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.545+0000 7fe1ebb8bf40 1 -- 172.21.15.38:0/1811344108 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:56 13.1c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x5626e9ae9440 con 0x5626e9ae48a0 2024-09-21T03:42:42.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.546+0000 7fe1e4e44640 1 -- 172.21.15.38:0/1811344108 <== osd.0 v2:172.21.15.38:6800/2381639433 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v120'144 uv144 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe1dc05d550 con 0x5626e9ae48a0 2024-09-21T03:42:42.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.546+0000 7fe1ebb8bf40 1 -- 172.21.15.38:0/1811344108 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:57 13.1c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x5626e9ae97e0 con 0x5626e9ae48a0 2024-09-21T03:42:42.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.547+0000 7fe1e4e44640 1 -- 172.21.15.38:0/1811344108 <== osd.0 v2:172.21.15.38:6800/2381639433 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v120'145 uv145 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe1dc05d550 con 0x5626e9ae48a0 2024-09-21T03:42:42.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.547+0000 7fe1ebb8bf40 1 -- 172.21.15.38:0/1811344108 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:58 13.1c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x5626e9ae8430 con 0x5626e9ae48a0 2024-09-21T03:42:42.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.548+0000 7fe1e4e44640 1 -- 172.21.15.38:0/1811344108 <== osd.0 v2:172.21.15.38:6800/2381639433 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v120'146 uv146 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe1dc05d550 con 0x5626e9ae48a0 2024-09-21T03:42:42.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.548+0000 7fe1ebb8bf40 1 --2- 172.21.15.38:0/1811344108 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x5626e9ae8d30 0x5626e9aebb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:42.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.548+0000 7fe1ebb8bf40 1 -- 172.21.15.38:0/1811344108 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:59 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x5626e9aec0d0 con 0x5626e9ae8d30 2024-09-21T03:42:42.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.548+0000 7fe1e5e46640 1 --2- 172.21.15.38:0/1811344108 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x5626e9ae8d30 0x5626e9aebb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:42.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.548+0000 7fe1e5e46640 1 --2- 172.21.15.38:0/1811344108 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x5626e9ae8d30 0x5626e9aebb90 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:42.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.550+0000 7fe1e5e46640 1 -- 172.21.15.38:0/1811344108 <== osd.3 v2:172.21.15.38:6808/1286810294 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v120'102 uv101 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7fe1e000aa20 con 0x5626e9ae8d30 2024-09-21T03:42:42.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.550+0000 7fe1ebb8bf40 1 -- 172.21.15.38:0/1811344108 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:60 13.6 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e120) -- 0x5626e9aec0d0 con 0x5626e9ae8d30 2024-09-21T03:42:42.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.551+0000 7fe1e5e46640 1 -- 172.21.15.38:0/1811344108 <== osd.3 v2:172.21.15.38:6808/1286810294 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v120'103 uv103 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fe1e000aa20 con 0x5626e9ae8d30 2024-09-21T03:42:42.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.551+0000 7fe1ebb8bf40 1 -- 172.21.15.38:0/1811344108 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:61 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x5626e9aec4f0 con 0x5626e9ae8d30 2024-09-21T03:42:42.555 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-09-21T03:42:42.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.552+0000 7fe1e5e46640 1 -- 172.21.15.38:0/1811344108 <== osd.3 v2:172.21.15.38:6808/1286810294 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v120'104 uv104 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fe1e000aa20 con 0x5626e9ae8d30 2024-09-21T03:42:42.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.552+0000 7fe1ebb8bf40 1 -- 172.21.15.38:0/1811344108 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:62 13.6 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x5626e9aec4f0 con 0x5626e9ae8d30 2024-09-21T03:42:42.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.553+0000 7fe1e5e46640 1 -- 172.21.15.38:0/1811344108 <== osd.3 v2:172.21.15.38:6808/1286810294 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v120'105 uv105 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fe1e000aa20 con 0x5626e9ae8d30 2024-09-21T03:42:42.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.553+0000 7fe1ebb8bf40 1 -- 172.21.15.38:0/1811344108 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x5626e9aec4f0 con 0x5626e9ae48a0 2024-09-21T03:42:42.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.554+0000 7fe1e4e44640 1 -- 172.21.15.38:0/1811344108 <== osd.0 v2:172.21.15.38:6800/2381639433 7 ==== osd_op_reply(63 608.00000000 [create] v120'141 uv140 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe1dc05d550 con 0x5626e9ae48a0 2024-09-21T03:42:42.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.554+0000 7fe1ebb8bf40 1 -- 172.21.15.38:0/1811344108 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e120) -- 0x5626e9aec890 con 0x5626e9ae48a0 2024-09-21T03:42:42.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.556+0000 7fe1e4e44640 1 -- 172.21.15.38:0/1811344108 <== osd.0 v2:172.21.15.38:6800/2381639433 8 ==== osd_op_reply(64 608.00000000 [delete] v120'142 uv142 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe1dc05d550 con 0x5626e9ae48a0 2024-09-21T03:42:42.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.556+0000 7fe1ebb8bf40 1 -- 172.21.15.38:0/1811344108 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x5626e9aec890 con 0x5626e9ae48a0 2024-09-21T03:42:42.559 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-09-21T03:42:42.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.557+0000 7fe1e4e44640 1 -- 172.21.15.38:0/1811344108 <== osd.0 v2:172.21.15.38:6800/2381639433 9 ==== osd_op_reply(65 608.00000000 [create] v120'143 uv143 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe1dc05d550 con 0x5626e9ae48a0 2024-09-21T03:42:42.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.557+0000 7fe1ebb8bf40 1 -- 172.21.15.38:0/1811344108 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x5626e9aed8a0 con 0x5626e9ae48a0 2024-09-21T03:42:42.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.558+0000 7fe1e4e44640 1 -- 172.21.15.38:0/1811344108 <== osd.0 v2:172.21.15.38:6800/2381639433 10 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v120'144 uv144 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe1dc05d550 con 0x5626e9ae48a0 2024-09-21T03:42:42.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.558+0000 7fe1ebb8bf40 1 -- 172.21.15.38:0/1811344108 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x5626e9aed8a0 con 0x5626e9ae48a0 2024-09-21T03:42:42.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.559+0000 7fe1e4e44640 1 -- 172.21.15.38:0/1811344108 <== osd.0 v2:172.21.15.38:6800/2381639433 11 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v120'145 uv145 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe1dc05d550 con 0x5626e9ae48a0 2024-09-21T03:42:42.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.559+0000 7fe1ebb8bf40 1 -- 172.21.15.38:0/1811344108 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x5626e9aec890 con 0x5626e9ae48a0 2024-09-21T03:42:42.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.560+0000 7fe1e4e44640 1 -- 172.21.15.38:0/1811344108 <== osd.0 v2:172.21.15.38:6800/2381639433 12 ==== osd_op_reply(68 608.00000000 [omap-set-header] v120'146 uv146 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe1dc05d550 con 0x5626e9ae48a0 2024-09-21T03:42:42.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.560+0000 7fe1ebb8bf40 1 -- 172.21.15.38:0/1811344108 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x5626e9aec890 con 0x5626e9ae8d30 2024-09-21T03:42:42.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.562+0000 7fe1e5e46640 1 -- 172.21.15.38:0/1811344108 <== osd.3 v2:172.21.15.38:6808/1286810294 5 ==== osd_op_reply(69 604.00000000 [create] v120'260 uv254 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe1e000aa20 con 0x5626e9ae8d30 2024-09-21T03:42:42.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.562+0000 7fe1ebb8bf40 1 -- 172.21.15.38:0/1811344108 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e120) -- 0x5626e9aec890 con 0x5626e9ae8d30 2024-09-21T03:42:42.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.563+0000 7fe1e5e46640 1 -- 172.21.15.38:0/1811344108 <== osd.3 v2:172.21.15.38:6808/1286810294 6 ==== osd_op_reply(70 604.00000000 [delete] v120'261 uv261 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe1e000aa20 con 0x5626e9ae8d30 2024-09-21T03:42:42.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.563+0000 7fe1ebb8bf40 1 -- 172.21.15.38:0/1811344108 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x5626e9aec890 con 0x5626e9ae8d30 2024-09-21T03:42:42.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.565+0000 7fe1e5e46640 1 -- 172.21.15.38:0/1811344108 <== osd.3 v2:172.21.15.38:6808/1286810294 7 ==== osd_op_reply(71 604.00000000 [create] v120'262 uv262 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe1e000aa20 con 0x5626e9ae8d30 2024-09-21T03:42:42.568 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-09-21T03:42:42.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.565+0000 7fe1ebb8bf40 1 -- 172.21.15.38:0/1811344108 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x5626e9aed8a0 con 0x5626e9ae8d30 2024-09-21T03:42:42.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.566+0000 7fe1e5e46640 1 -- 172.21.15.38:0/1811344108 <== osd.3 v2:172.21.15.38:6808/1286810294 8 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v120'263 uv263 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe1e000aa20 con 0x5626e9ae8d30 2024-09-21T03:42:42.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.566+0000 7fe1ebb8bf40 1 -- 172.21.15.38:0/1811344108 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x5626e9aed8a0 con 0x5626e9ae8d30 2024-09-21T03:42:42.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.567+0000 7fe1e5e46640 1 -- 172.21.15.38:0/1811344108 <== osd.3 v2:172.21.15.38:6808/1286810294 9 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v120'264 uv264 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe1e000aa20 con 0x5626e9ae8d30 2024-09-21T03:42:42.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.567+0000 7fe1ebb8bf40 1 -- 172.21.15.38:0/1811344108 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x5626e9aec890 con 0x5626e9ae8d30 2024-09-21T03:42:42.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.568+0000 7fe1e5e46640 1 -- 172.21.15.38:0/1811344108 <== osd.3 v2:172.21.15.38:6808/1286810294 10 ==== osd_op_reply(74 604.00000000 [omap-set-header] v120'265 uv265 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe1e000aa20 con 0x5626e9ae8d30 2024-09-21T03:42:42.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.568+0000 7fe1ebb8bf40 1 -- 172.21.15.38:0/1811344108 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x5626e9aec890 con 0x5626e9ae8d30 2024-09-21T03:42:42.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.569+0000 7fe1e5e46640 1 -- 172.21.15.38:0/1811344108 <== osd.3 v2:172.21.15.38:6808/1286810294 11 ==== osd_op_reply(75 500.00000000 [create] v120'266 uv259 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe1e000aa20 con 0x5626e9ae8d30 2024-09-21T03:42:42.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.569+0000 7fe1ebb8bf40 1 -- 172.21.15.38:0/1811344108 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e120) -- 0x5626e9aec890 con 0x5626e9ae8d30 2024-09-21T03:42:42.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.571+0000 7fe1e5e46640 1 -- 172.21.15.38:0/1811344108 <== osd.3 v2:172.21.15.38:6808/1286810294 12 ==== osd_op_reply(76 500.00000000 [delete] v120'267 uv267 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe1e000aa20 con 0x5626e9ae8d30 2024-09-21T03:42:42.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.571+0000 7fe1ebb8bf40 1 -- 172.21.15.38:0/1811344108 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x5626e9aec890 con 0x5626e9ae8d30 2024-09-21T03:42:42.575 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-09-21T03:42:42.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.572+0000 7fe1e5e46640 1 -- 172.21.15.38:0/1811344108 <== osd.3 v2:172.21.15.38:6808/1286810294 13 ==== osd_op_reply(77 500.00000000 [create] v120'268 uv268 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe1e000aa20 con 0x5626e9ae8d30 2024-09-21T03:42:42.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.572+0000 7fe1ebb8bf40 1 -- 172.21.15.38:0/1811344108 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x5626e9aec890 con 0x5626e9ae8d30 2024-09-21T03:42:42.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.573+0000 7fe1e5e46640 1 -- 172.21.15.38:0/1811344108 <== osd.3 v2:172.21.15.38:6808/1286810294 14 ==== osd_op_reply(78 500.00000000 [write 0~90] v120'269 uv269 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe1e000aa20 con 0x5626e9ae8d30 2024-09-21T03:42:42.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.573+0000 7fe1ebb8bf40 1 -- 172.21.15.38:0/1811344108 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x5626e9aec890 con 0x5626e9ae8d30 2024-09-21T03:42:42.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.574+0000 7fe1e5e46640 1 -- 172.21.15.38:0/1811344108 <== osd.3 v2:172.21.15.38:6808/1286810294 15 ==== osd_op_reply(79 500.00000000 [omap-set-header] v120'270 uv270 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe1e000aa20 con 0x5626e9ae8d30 2024-09-21T03:42:42.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.575+0000 7fe1ebb8bf40 1 -- 172.21.15.38:0/1811344108 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:80 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x5626e9aec890 con 0x5626e9ae48a0 2024-09-21T03:42:42.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.576+0000 7fe1e4e44640 1 -- 172.21.15.38:0/1811344108 <== osd.0 v2:172.21.15.38:6800/2381639433 13 ==== osd_op_reply(80 mds_snaptable [create] v120'117 uv116 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7fe1dc05d550 con 0x5626e9ae48a0 2024-09-21T03:42:42.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.576+0000 7fe1ebb8bf40 1 -- 172.21.15.38:0/1811344108 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:81 13.d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e120) -- 0x5626e9aec890 con 0x5626e9ae48a0 2024-09-21T03:42:42.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.577+0000 7fe1e4e44640 1 -- 172.21.15.38:0/1811344108 <== osd.0 v2:172.21.15.38:6800/2381639433 14 ==== osd_op_reply(81 mds_snaptable [delete] v120'118 uv118 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fe1dc05d550 con 0x5626e9ae48a0 2024-09-21T03:42:42.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.577+0000 7fe1ebb8bf40 1 -- 172.21.15.38:0/1811344108 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:82 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x5626e9aec890 con 0x5626e9ae48a0 2024-09-21T03:42:42.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.578+0000 7fe1e4e44640 1 -- 172.21.15.38:0/1811344108 <== osd.0 v2:172.21.15.38:6800/2381639433 15 ==== osd_op_reply(82 mds_snaptable [create] v120'119 uv119 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fe1dc05d550 con 0x5626e9ae48a0 2024-09-21T03:42:42.581 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-09-21T03:42:42.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.578+0000 7fe1ebb8bf40 1 -- 172.21.15.38:0/1811344108 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:83 13.d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x5626e9aec890 con 0x5626e9ae48a0 2024-09-21T03:42:42.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.579+0000 7fe1e4e44640 1 -- 172.21.15.38:0/1811344108 <== osd.0 v2:172.21.15.38:6800/2381639433 16 ==== osd_op_reply(83 mds_snaptable [write 0~70] v120'120 uv120 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fe1dc05d550 con 0x5626e9ae48a0 2024-09-21T03:42:42.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.579+0000 7fe1ebb8bf40 1 -- 172.21.15.38:0/1811344108 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:84 13.d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x5626e9aec890 con 0x5626e9ae48a0 2024-09-21T03:42:42.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.580+0000 7fe1e4e44640 1 -- 172.21.15.38:0/1811344108 <== osd.0 v2:172.21.15.38:6800/2381639433 17 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v120'121 uv121 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fe1dc05d550 con 0x5626e9ae48a0 2024-09-21T03:42:42.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.580+0000 7fe1ebb8bf40 1 --2- 172.21.15.38:0/1811344108 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x5626e9aecf10 0x5626e9aef330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:42.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.580+0000 7fe1ebb8bf40 1 -- 172.21.15.38:0/1811344108 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x5626e9aef900 con 0x5626e9aecf10 2024-09-21T03:42:42.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.580+0000 7fe1e5645640 1 --2- 172.21.15.38:0/1811344108 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x5626e9aecf10 0x5626e9aef330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:42.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.581+0000 7fe1e5645640 1 --2- 172.21.15.38:0/1811344108 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x5626e9aecf10 0x5626e9aef330 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:42.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.582+0000 7fe1e5645640 1 -- 172.21.15.38:0/1811344108 <== osd.1 v2:172.21.15.38:6816/1491818732 1 ==== osd_op_reply(85 605.00000000 [create] v120'141 uv140 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe1d800b040 con 0x5626e9aecf10 2024-09-21T03:42:42.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.582+0000 7fe1ebb8bf40 1 -- 172.21.15.38:0/1811344108 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e120) -- 0x5626e9aefcd0 con 0x5626e9aecf10 2024-09-21T03:42:42.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.583+0000 7fe1e5645640 1 -- 172.21.15.38:0/1811344108 <== osd.1 v2:172.21.15.38:6816/1491818732 2 ==== osd_op_reply(86 605.00000000 [delete] v120'142 uv142 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe1d800b040 con 0x5626e9aecf10 2024-09-21T03:42:42.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.583+0000 7fe1ebb8bf40 1 -- 172.21.15.38:0/1811344108 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x5626e9aefcd0 con 0x5626e9aecf10 2024-09-21T03:42:42.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.585+0000 7fe1e5645640 1 -- 172.21.15.38:0/1811344108 <== osd.1 v2:172.21.15.38:6816/1491818732 3 ==== osd_op_reply(87 605.00000000 [create] v120'143 uv143 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe1d800b040 con 0x5626e9aecf10 2024-09-21T03:42:42.588 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-09-21T03:42:42.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.585+0000 7fe1ebb8bf40 1 -- 172.21.15.38:0/1811344108 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x5626e9af0ce0 con 0x5626e9aecf10 2024-09-21T03:42:42.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.586+0000 7fe1e5645640 1 -- 172.21.15.38:0/1811344108 <== osd.1 v2:172.21.15.38:6816/1491818732 4 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v120'144 uv144 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe1d800b040 con 0x5626e9aecf10 2024-09-21T03:42:42.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.586+0000 7fe1ebb8bf40 1 -- 172.21.15.38:0/1811344108 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x5626e9af0ce0 con 0x5626e9aecf10 2024-09-21T03:42:42.589 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.587+0000 7fe1e5645640 1 -- 172.21.15.38:0/1811344108 <== osd.1 v2:172.21.15.38:6816/1491818732 5 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v120'145 uv145 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe1d800b040 con 0x5626e9aecf10 2024-09-21T03:42:42.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.587+0000 7fe1ebb8bf40 1 -- 172.21.15.38:0/1811344108 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x5626e9aefcd0 con 0x5626e9aecf10 2024-09-21T03:42:42.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.588+0000 7fe1e5645640 1 -- 172.21.15.38:0/1811344108 <== osd.1 v2:172.21.15.38:6816/1491818732 6 ==== osd_op_reply(90 605.00000000 [omap-set-header] v120'146 uv146 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe1d800b040 con 0x5626e9aecf10 2024-09-21T03:42:42.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.588+0000 7fe1ebb8bf40 1 -- 172.21.15.38:0/1811344108 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x5626e9aefcd0 con 0x5626e9aecf10 2024-09-21T03:42:42.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.589+0000 7fe1e5645640 1 -- 172.21.15.38:0/1811344108 <== osd.1 v2:172.21.15.38:6816/1491818732 7 ==== osd_op_reply(91 mds0_inotable [create] v120'233 uv227 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7fe1d800b040 con 0x5626e9aecf10 2024-09-21T03:42:42.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.589+0000 7fe1ebb8bf40 1 -- 172.21.15.38:0/1811344108 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e120) -- 0x5626e9aefcd0 con 0x5626e9aecf10 2024-09-21T03:42:42.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.590+0000 7fe1e5645640 1 -- 172.21.15.38:0/1811344108 <== osd.1 v2:172.21.15.38:6816/1491818732 8 ==== osd_op_reply(92 mds0_inotable [delete] v120'234 uv234 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fe1d800b040 con 0x5626e9aecf10 2024-09-21T03:42:42.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.590+0000 7fe1ebb8bf40 1 -- 172.21.15.38:0/1811344108 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x5626e9aefcd0 con 0x5626e9aecf10 2024-09-21T03:42:42.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.592+0000 7fe1e5645640 1 -- 172.21.15.38:0/1811344108 <== osd.1 v2:172.21.15.38:6816/1491818732 9 ==== osd_op_reply(93 mds0_inotable [create] v120'235 uv235 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fe1d800b040 con 0x5626e9aecf10 2024-09-21T03:42:42.595 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-09-21T03:42:42.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.592+0000 7fe1ebb8bf40 1 -- 172.21.15.38:0/1811344108 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:94 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x5626e9af00c0 con 0x5626e9aecf10 2024-09-21T03:42:42.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.593+0000 7fe1e5645640 1 -- 172.21.15.38:0/1811344108 <== osd.1 v2:172.21.15.38:6816/1491818732 10 ==== osd_op_reply(94 mds0_inotable [write 0~34] v120'236 uv236 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fe1d800b040 con 0x5626e9aecf10 2024-09-21T03:42:42.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.593+0000 7fe1ebb8bf40 1 -- 172.21.15.38:0/1811344108 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:95 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x5626e9af00c0 con 0x5626e9aecf10 2024-09-21T03:42:42.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.594+0000 7fe1e5645640 1 -- 172.21.15.38:0/1811344108 <== osd.1 v2:172.21.15.38:6816/1491818732 11 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v120'237 uv237 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fe1d800b040 con 0x5626e9aecf10 2024-09-21T03:42:42.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.594+0000 7fe1ebb8bf40 1 -- 172.21.15.38:0/1811344108 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:96 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x5626e9aefde0 con 0x5626e9aecf10 2024-09-21T03:42:42.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.595+0000 7fe1e5645640 1 -- 172.21.15.38:0/1811344108 <== osd.1 v2:172.21.15.38:6816/1491818732 12 ==== osd_op_reply(96 100.00000000 [create] v120'238 uv232 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe1d800b040 con 0x5626e9aecf10 2024-09-21T03:42:42.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.595+0000 7fe1ebb8bf40 1 -- 172.21.15.38:0/1811344108 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:97 13.13 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e120) -- 0x5626e9aefd70 con 0x5626e9aecf10 2024-09-21T03:42:42.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.596+0000 7fe1e5645640 1 -- 172.21.15.38:0/1811344108 <== osd.1 v2:172.21.15.38:6816/1491818732 13 ==== osd_op_reply(97 100.00000000 [delete] v120'239 uv239 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe1d800b040 con 0x5626e9aecf10 2024-09-21T03:42:42.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.596+0000 7fe1ebb8bf40 1 -- 172.21.15.38:0/1811344108 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:98 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x5626e9aefd70 con 0x5626e9aecf10 2024-09-21T03:42:42.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.597+0000 7fe1e5645640 1 -- 172.21.15.38:0/1811344108 <== osd.1 v2:172.21.15.38:6816/1491818732 14 ==== osd_op_reply(98 100.00000000 [create] v120'240 uv240 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe1d800b040 con 0x5626e9aecf10 2024-09-21T03:42:42.600 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-09-21T03:42:42.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.598+0000 7fe1ebb8bf40 1 -- 172.21.15.38:0/1811344108 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:99 13.13 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x5626e9af0160 con 0x5626e9aecf10 2024-09-21T03:42:42.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.599+0000 7fe1e5645640 1 -- 172.21.15.38:0/1811344108 <== osd.1 v2:172.21.15.38:6816/1491818732 15 ==== osd_op_reply(99 100.00000000 [omap-set-header] v120'241 uv241 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe1d800b040 con 0x5626e9aecf10 2024-09-21T03:42:42.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.599+0000 7fe1ebb8bf40 1 -- 172.21.15.38:0/1811344108 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:100 13.13 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x5626e9af2c10 con 0x5626e9aecf10 2024-09-21T03:42:42.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.600+0000 7fe1e5645640 1 -- 172.21.15.38:0/1811344108 <== osd.1 v2:172.21.15.38:6816/1491818732 16 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v120'242 uv242 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe1d800b040 con 0x5626e9aecf10 2024-09-21T03:42:42.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.600+0000 7fe1ebb8bf40 1 -- 172.21.15.38:0/1811344108 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x5626e9af02b0 con 0x5626e9ae8d30 2024-09-21T03:42:42.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.601+0000 7fe1e5e46640 1 -- 172.21.15.38:0/1811344108 <== osd.3 v2:172.21.15.38:6808/1286810294 16 ==== osd_op_reply(101 mds0_sessionmap [create] v120'103 uv102 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7fe1e000aa20 con 0x5626e9ae8d30 2024-09-21T03:42:42.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.601+0000 7fe1ebb8bf40 1 -- 172.21.15.38:0/1811344108 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e120) -- 0x5626e9af02b0 con 0x5626e9ae8d30 2024-09-21T03:42:42.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.602+0000 7fe1e5e46640 1 -- 172.21.15.38:0/1811344108 <== osd.3 v2:172.21.15.38:6808/1286810294 17 ==== osd_op_reply(102 mds0_sessionmap [delete] v120'104 uv104 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fe1e000aa20 con 0x5626e9ae8d30 2024-09-21T03:42:42.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.602+0000 7fe1ebb8bf40 1 -- 172.21.15.38:0/1811344108 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:103 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x5626e9af02b0 con 0x5626e9ae8d30 2024-09-21T03:42:42.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.603+0000 7fe1e5e46640 1 -- 172.21.15.38:0/1811344108 <== osd.3 v2:172.21.15.38:6808/1286810294 18 ==== osd_op_reply(103 mds0_sessionmap [create] v120'105 uv105 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fe1e000aa20 con 0x5626e9ae8d30 2024-09-21T03:42:42.606 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-09-21T03:42:42.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.603+0000 7fe1ebb8bf40 1 -- 172.21.15.38:0/1811344108 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:104 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x5626e9af02b0 con 0x5626e9ae8d30 2024-09-21T03:42:42.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.604+0000 7fe1e5e46640 1 -- 172.21.15.38:0/1811344108 <== osd.3 v2:172.21.15.38:6808/1286810294 19 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v120'106 uv106 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fe1e000aa20 con 0x5626e9ae8d30 2024-09-21T03:42:42.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.604+0000 7fe1ebb8bf40 1 -- 172.21.15.38:0/1811344108 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x5626e9af02b0 con 0x5626e9ae8d30 2024-09-21T03:42:42.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.605+0000 7fe1e5e46640 1 -- 172.21.15.38:0/1811344108 <== osd.3 v2:172.21.15.38:6808/1286810294 20 ==== osd_op_reply(105 609.00000000 [create] v120'141 uv140 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe1e000aa20 con 0x5626e9ae8d30 2024-09-21T03:42:42.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.605+0000 7fe1ebb8bf40 1 -- 172.21.15.38:0/1811344108 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e120) -- 0x5626e9af02b0 con 0x5626e9ae8d30 2024-09-21T03:42:42.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.606+0000 7fe1e5e46640 1 -- 172.21.15.38:0/1811344108 <== osd.3 v2:172.21.15.38:6808/1286810294 21 ==== osd_op_reply(106 609.00000000 [delete] v120'142 uv142 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe1e000aa20 con 0x5626e9ae8d30 2024-09-21T03:42:42.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.606+0000 7fe1ebb8bf40 1 -- 172.21.15.38:0/1811344108 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x5626e9af02b0 con 0x5626e9ae8d30 2024-09-21T03:42:42.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.607+0000 7fe1e5e46640 1 -- 172.21.15.38:0/1811344108 <== osd.3 v2:172.21.15.38:6808/1286810294 22 ==== osd_op_reply(107 609.00000000 [create] v120'143 uv143 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe1e000aa20 con 0x5626e9ae8d30 2024-09-21T03:42:42.610 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-09-21T03:42:42.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.607+0000 7fe1ebb8bf40 1 -- 172.21.15.38:0/1811344108 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x5626e9af12c0 con 0x5626e9ae8d30 2024-09-21T03:42:42.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.608+0000 7fe1e5e46640 1 -- 172.21.15.38:0/1811344108 <== osd.3 v2:172.21.15.38:6808/1286810294 23 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v120'144 uv144 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe1e000aa20 con 0x5626e9ae8d30 2024-09-21T03:42:42.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.608+0000 7fe1ebb8bf40 1 -- 172.21.15.38:0/1811344108 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:109 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x5626e9af12c0 con 0x5626e9ae8d30 2024-09-21T03:42:42.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.609+0000 7fe1e5e46640 1 -- 172.21.15.38:0/1811344108 <== osd.3 v2:172.21.15.38:6808/1286810294 24 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v120'145 uv145 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe1e000aa20 con 0x5626e9ae8d30 2024-09-21T03:42:42.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.609+0000 7fe1ebb8bf40 1 -- 172.21.15.38:0/1811344108 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:110 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x5626e9af02b0 con 0x5626e9ae8d30 2024-09-21T03:42:42.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.610+0000 7fe1e5e46640 1 -- 172.21.15.38:0/1811344108 <== osd.3 v2:172.21.15.38:6808/1286810294 25 ==== osd_op_reply(110 609.00000000 [omap-set-header] v120'146 uv146 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe1e000aa20 con 0x5626e9ae8d30 2024-09-21T03:42:42.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.610+0000 7fe1ebb8bf40 1 --2- 172.21.15.38:0/1811344108 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x5626e9af07c0 0x5626e9af2be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:42.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.610+0000 7fe1ebb8bf40 1 -- 172.21.15.38:0/1811344108 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x5626e9af3250 con 0x5626e9af07c0 2024-09-21T03:42:42.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.610+0000 7fe1e4e44640 1 --2- 172.21.15.38:0/1811344108 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x5626e9af07c0 0x5626e9af2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:42.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.611+0000 7fe1e4e44640 1 --2- 172.21.15.38:0/1811344108 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x5626e9af07c0 0x5626e9af2be0 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:42.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.612+0000 7fe1e4e44640 1 -- 172.21.15.38:0/1811344108 <== osd.2 v2:172.21.15.38:6824/2577858902 1 ==== osd_op_reply(111 400.00000000 [create] v120'120 uv119 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe1dc05d550 con 0x5626e9af07c0 2024-09-21T03:42:42.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.612+0000 7fe1ebb8bf40 1 -- 172.21.15.38:0/1811344108 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e120) -- 0x5626e9af3250 con 0x5626e9af07c0 2024-09-21T03:42:42.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.613+0000 7fe1e4e44640 1 -- 172.21.15.38:0/1811344108 <== osd.2 v2:172.21.15.38:6824/2577858902 2 ==== osd_op_reply(112 400.00000000 [delete] v120'121 uv121 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe1dc05d550 con 0x5626e9af07c0 2024-09-21T03:42:42.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.613+0000 7fe1ebb8bf40 1 -- 172.21.15.38:0/1811344108 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x5626e9af3250 con 0x5626e9af07c0 2024-09-21T03:42:42.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.614+0000 7fe1e4e44640 1 -- 172.21.15.38:0/1811344108 <== osd.2 v2:172.21.15.38:6824/2577858902 3 ==== osd_op_reply(113 400.00000000 [create] v120'122 uv122 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe1dc05d550 con 0x5626e9af07c0 2024-09-21T03:42:42.617 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-09-21T03:42:42.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.614+0000 7fe1ebb8bf40 1 -- 172.21.15.38:0/1811344108 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:114 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x5626e9af3640 con 0x5626e9af07c0 2024-09-21T03:42:42.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.615+0000 7fe1e4e44640 1 -- 172.21.15.38:0/1811344108 <== osd.2 v2:172.21.15.38:6824/2577858902 4 ==== osd_op_reply(114 400.00000000 [write 0~22] v120'123 uv123 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe1dc05d550 con 0x5626e9af07c0 2024-09-21T03:42:42.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.615+0000 7fe1ebb8bf40 1 -- 172.21.15.38:0/1811344108 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:115 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x5626e9af3640 con 0x5626e9af07c0 2024-09-21T03:42:42.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.616+0000 7fe1e4e44640 1 -- 172.21.15.38:0/1811344108 <== osd.2 v2:172.21.15.38:6824/2577858902 5 ==== osd_op_reply(115 400.00000000 [omap-set-header] v120'124 uv124 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe1dc05d550 con 0x5626e9af07c0 2024-09-21T03:42:42.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.616+0000 7fe1ebb8bf40 1 -- 172.21.15.38:0/1811344108 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:116 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x5626e9af3640 con 0x5626e9aecf10 2024-09-21T03:42:42.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.617+0000 7fe1e5645640 1 -- 172.21.15.38:0/1811344108 <== osd.1 v2:172.21.15.38:6816/1491818732 17 ==== osd_op_reply(116 100.00000000.inode [create] v120'290 uv282 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7fe1d800b040 con 0x5626e9aecf10 2024-09-21T03:42:42.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.617+0000 7fe1ebb8bf40 1 -- 172.21.15.38:0/1811344108 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:117 13.1f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e120) -- 0x5626e9af3640 con 0x5626e9aecf10 2024-09-21T03:42:42.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.619+0000 7fe1e5645640 1 -- 172.21.15.38:0/1811344108 <== osd.1 v2:172.21.15.38:6816/1491818732 18 ==== osd_op_reply(117 100.00000000.inode [delete] v120'291 uv291 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fe1d800b040 con 0x5626e9aecf10 2024-09-21T03:42:42.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.619+0000 7fe1ebb8bf40 1 -- 172.21.15.38:0/1811344108 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:118 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x5626e9af3640 con 0x5626e9aecf10 2024-09-21T03:42:42.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.620+0000 7fe1e5645640 1 -- 172.21.15.38:0/1811344108 <== osd.1 v2:172.21.15.38:6816/1491818732 19 ==== osd_op_reply(118 100.00000000.inode [create] v120'292 uv292 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fe1d800b040 con 0x5626e9aecf10 2024-09-21T03:42:42.623 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-09-21T03:42:42.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.620+0000 7fe1ebb8bf40 1 -- 172.21.15.38:0/1811344108 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:119 13.1f 13:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x5626e9af3640 con 0x5626e9aecf10 2024-09-21T03:42:42.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.621+0000 7fe1e5645640 1 -- 172.21.15.38:0/1811344108 <== osd.1 v2:172.21.15.38:6816/1491818732 20 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v120'293 uv293 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fe1d800b040 con 0x5626e9aecf10 2024-09-21T03:42:42.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.621+0000 7fe1ebb8bf40 1 -- 172.21.15.38:0/1811344108 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:120 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x5626e9af3640 con 0x5626e9aecf10 2024-09-21T03:42:42.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.622+0000 7fe1e5645640 1 -- 172.21.15.38:0/1811344108 <== osd.1 v2:172.21.15.38:6816/1491818732 21 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v120'294 uv294 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fe1d800b040 con 0x5626e9aecf10 2024-09-21T03:42:42.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.622+0000 7fe1ebb8bf40 1 -- 172.21.15.38:0/1811344108 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:121 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x5626e9af3640 con 0x5626e9aecf10 2024-09-21T03:42:42.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.623+0000 7fe1e5645640 1 -- 172.21.15.38:0/1811344108 <== osd.1 v2:172.21.15.38:6816/1491818732 22 ==== osd_op_reply(121 1.00000000 [create] v120'295 uv289 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7fe1d800b040 con 0x5626e9aecf10 2024-09-21T03:42:42.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.623+0000 7fe1ebb8bf40 1 -- 172.21.15.38:0/1811344108 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:122 13.1f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e120) -- 0x5626e9af3640 con 0x5626e9aecf10 2024-09-21T03:42:42.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.625+0000 7fe1e5645640 1 -- 172.21.15.38:0/1811344108 <== osd.1 v2:172.21.15.38:6816/1491818732 23 ==== osd_op_reply(122 1.00000000 [delete] v120'296 uv296 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fe1d800b040 con 0x5626e9aecf10 2024-09-21T03:42:42.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.625+0000 7fe1ebb8bf40 1 -- 172.21.15.38:0/1811344108 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:123 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x5626e9af3640 con 0x5626e9aecf10 2024-09-21T03:42:42.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.626+0000 7fe1e5645640 1 -- 172.21.15.38:0/1811344108 <== osd.1 v2:172.21.15.38:6816/1491818732 24 ==== osd_op_reply(123 1.00000000 [create] v120'297 uv297 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fe1d800b040 con 0x5626e9aecf10 2024-09-21T03:42:42.629 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-09-21T03:42:42.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.626+0000 7fe1ebb8bf40 1 -- 172.21.15.38:0/1811344108 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:124 13.1f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x5626e9af4650 con 0x5626e9aecf10 2024-09-21T03:42:42.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.627+0000 7fe1e5645640 1 -- 172.21.15.38:0/1811344108 <== osd.1 v2:172.21.15.38:6816/1491818732 25 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v120'298 uv298 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fe1d800b040 con 0x5626e9aecf10 2024-09-21T03:42:42.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.627+0000 7fe1ebb8bf40 1 -- 172.21.15.38:0/1811344108 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:125 13.1f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x5626e9af4650 con 0x5626e9aecf10 2024-09-21T03:42:42.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.628+0000 7fe1e5645640 1 -- 172.21.15.38:0/1811344108 <== osd.1 v2:172.21.15.38:6816/1491818732 26 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v120'299 uv299 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fe1d800b040 con 0x5626e9aecf10 2024-09-21T03:42:42.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.628+0000 7fe1ebb8bf40 1 -- 172.21.15.38:0/1811344108 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:126 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x5626e9af3640 con 0x5626e9aecf10 2024-09-21T03:42:42.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.630+0000 7fe1e5645640 1 -- 172.21.15.38:0/1811344108 <== osd.1 v2:172.21.15.38:6816/1491818732 27 ==== osd_op_reply(126 1.00000000 [omap-set-header] v120'300 uv300 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fe1d800b040 con 0x5626e9aecf10 2024-09-21T03:42:42.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.630+0000 7fe1ebb8bf40 1 -- 172.21.15.38:0/1811344108 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:127 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=584b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x5626e9af3da0 con 0x5626e9aecf10 2024-09-21T03:42:42.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.631+0000 7fe1e5645640 1 -- 172.21.15.38:0/1811344108 <== osd.1 v2:172.21.15.38:6816/1491818732 28 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v120'301 uv301 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fe1d800b040 con 0x5626e9aecf10 2024-09-21T03:42:42.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.631+0000 7fe1ebb8bf40 1 -- 172.21.15.38:0/1811344108 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x5626e9ae48a0 msgr2=0x5626e9ae4cb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:42.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.631+0000 7fe1ebb8bf40 1 --2- 172.21.15.38:0/1811344108 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x5626e9ae48a0 0x5626e9ae4cb0 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:42.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.631+0000 7fe1ebb8bf40 1 -- 172.21.15.38:0/1811344108 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x5626e9aecf10 msgr2=0x5626e9aef330 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:42.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.631+0000 7fe1ebb8bf40 1 --2- 172.21.15.38:0/1811344108 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x5626e9aecf10 0x5626e9aef330 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:42.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.631+0000 7fe1ebb8bf40 1 -- 172.21.15.38:0/1811344108 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x5626e9af07c0 msgr2=0x5626e9af2be0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:42.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.631+0000 7fe1ebb8bf40 1 --2- 172.21.15.38:0/1811344108 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x5626e9af07c0 0x5626e9af2be0 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:42.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.631+0000 7fe1ebb8bf40 1 -- 172.21.15.38:0/1811344108 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x5626e9ae8d30 msgr2=0x5626e9aebb90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:42.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.631+0000 7fe1ebb8bf40 1 --2- 172.21.15.38:0/1811344108 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x5626e9ae8d30 0x5626e9aebb90 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:42.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.631+0000 7fe1e5e46640 1 -- 172.21.15.38:0/1811344108 reap_dead start 2024-09-21T03:42:42.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.631+0000 7fe1ebb8bf40 1 -- 172.21.15.38:0/1811344108 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x5626e9adf150 msgr2=0x5626e9ae2250 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:42.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.631+0000 7fe1ebb8bf40 1 --2- 172.21.15.38:0/1811344108 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x5626e9adf150 0x5626e9ae2250 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:42.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.631+0000 7fe1e5e46640 1 -- 172.21.15.38:0/1811344108 reap_dead start 2024-09-21T03:42:42.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.631+0000 7fe1ebb8bf40 1 -- 172.21.15.38:0/1811344108 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x5626e9a32300 msgr2=0x5626e9a35db0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:42.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.631+0000 7fe1ebb8bf40 1 --2- 172.21.15.38:0/1811344108 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x5626e9a32300 0x5626e9a35db0 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:42.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.631+0000 7fe1ebb8bf40 1 -- 172.21.15.38:0/1811344108 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fe1c8077570 msgr2=0x7fe1c8079a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:42.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.631+0000 7fe1ebb8bf40 1 --2- 172.21.15.38:0/1811344108 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fe1c8077570 0x7fe1c8079a80 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fe1d8056000 tx=0x7fe1d80033a0 comp rx=0 tx=0).stop 2024-09-21T03:42:42.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.631+0000 7fe1ebb8bf40 1 -- 172.21.15.38:0/1811344108 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5626e9a19bb0 msgr2=0x5626e9a18f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:42.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.631+0000 7fe1ebb8bf40 1 --2- 172.21.15.38:0/1811344108 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5626e9a19bb0 0x5626e9a18f50 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7fe1dc001eb0 tx=0x7fe1dc00a5a0 comp rx=0 tx=0).stop 2024-09-21T03:42:42.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.631+0000 7fe1ebb8bf40 1 -- 172.21.15.38:0/1811344108 shutdown_connections 2024-09-21T03:42:42.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.631+0000 7fe1ebb8bf40 1 --2- 172.21.15.38:0/1811344108 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x5626e9a32300 0x5626e9a35db0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:42.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.631+0000 7fe1ebb8bf40 1 --2- 172.21.15.38:0/1811344108 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fe1c8077570 0x7fe1c8079a80 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:42.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.631+0000 7fe1ebb8bf40 1 --2- 172.21.15.38:0/1811344108 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5626e9a19bb0 0x5626e9a18f50 unknown :-1 s=CLOSED pgs=1822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:42.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.631+0000 7fe1ebb8bf40 1 -- 172.21.15.38:0/1811344108 >> 172.21.15.38:0/1811344108 conn(0x5626e9883ac0 msgr2=0x5626e9a32850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:42.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.631+0000 7fe1ebb8bf40 1 -- 172.21.15.38:0/1811344108 shutdown_connections 2024-09-21T03:42:42.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.631+0000 7fe1ebb8bf40 1 -- 172.21.15.38:0/1811344108 wait complete. 2024-09-21T03:42:42.638 DEBUG:teuthology.orchestra.run.smithi038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata truncate 200.00000000 0 2024-09-21T03:42:42.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.731+0000 7f0b9251ff40 1 Processor -- start 2024-09-21T03:42:42.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.731+0000 7f0b9251ff40 1 -- start start 2024-09-21T03:42:42.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.733+0000 7f0b9251ff40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5650fa072240 0x5650fa06d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:42.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.733+0000 7f0b9251ff40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5650fa0b9090 0x5650fa06dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:42.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.733+0000 7f0b9251ff40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5650fa12e520 0x5650fa1d77a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:42.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.733+0000 7f0b9251ff40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5650fa145aa0 con 0x5650fa12e520 2024-09-21T03:42:42.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.733+0000 7f0b9251ff40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5650f9fbfd60 con 0x5650fa0b9090 2024-09-21T03:42:42.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.733+0000 7f0b9251ff40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x5650fa0a09b0 con 0x5650fa072240 2024-09-21T03:42:42.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.733+0000 7f0b8bd55640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5650fa12e520 0x5650fa1d77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:42.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.733+0000 7f0b8bd55640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5650fa12e520 0x5650fa1d77a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38154/0 (socket says 172.21.15.38:38154) 2024-09-21T03:42:42.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.733+0000 7f0b8bd55640 1 -- 172.21.15.38:0/1068835422 learned_addr learned my addr 172.21.15.38:0/1068835422 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:42:42.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.733+0000 7f0b8ad53640 1 --2- 172.21.15.38:0/1068835422 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5650fa0b9090 0x5650fa06dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:42.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.733+0000 7f0b8b554640 1 --2- 172.21.15.38:0/1068835422 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5650fa072240 0x5650fa06d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:42.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.733+0000 7f0b8bd55640 1 -- 172.21.15.38:0/1068835422 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5650fa072240 msgr2=0x5650fa06d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:42.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.733+0000 7f0b8bd55640 1 --2- 172.21.15.38:0/1068835422 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5650fa072240 0x5650fa06d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:42.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.733+0000 7f0b8bd55640 1 -- 172.21.15.38:0/1068835422 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5650fa0b9090 msgr2=0x5650fa06dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:42.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.733+0000 7f0b8bd55640 1 --2- 172.21.15.38:0/1068835422 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5650fa0b9090 0x5650fa06dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:42.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.733+0000 7f0b8bd55640 1 -- 172.21.15.38:0/1068835422 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650fa13f970 con 0x5650fa12e520 2024-09-21T03:42:42.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.733+0000 7f0b8bd55640 1 --2- 172.21.15.38:0/1068835422 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5650fa12e520 0x5650fa1d77a0 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7f0b8404d760 tx=0x7f0b84002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b919c9cfb3bc912f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:42.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.734+0000 7f0b89d51640 1 -- 172.21.15.38:0/1068835422 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b8404dd70 con 0x5650fa12e520 2024-09-21T03:42:42.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.734+0000 7f0b89d51640 1 -- 172.21.15.38:0/1068835422 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0b84056470 con 0x5650fa12e520 2024-09-21T03:42:42.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.734+0000 7f0b89d51640 1 -- 172.21.15.38:0/1068835422 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b840568c0 con 0x5650fa12e520 2024-09-21T03:42:42.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.734+0000 7f0b9251ff40 1 -- 172.21.15.38:0/1068835422 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5650fa12e520 msgr2=0x5650fa1d77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:42.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.734+0000 7f0b9251ff40 1 --2- 172.21.15.38:0/1068835422 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5650fa12e520 0x5650fa1d77a0 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7f0b8404d760 tx=0x7f0b84002fb0 comp rx=0 tx=0).stop 2024-09-21T03:42:42.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.734+0000 7f0b9251ff40 1 -- 172.21.15.38:0/1068835422 shutdown_connections 2024-09-21T03:42:42.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.734+0000 7f0b9251ff40 1 --2- 172.21.15.38:0/1068835422 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5650fa12e520 0x5650fa1d77a0 unknown :-1 s=CLOSED pgs=1824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:42.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.734+0000 7f0b9251ff40 1 --2- 172.21.15.38:0/1068835422 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5650fa0b9090 0x5650fa06dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:42.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.734+0000 7f0b9251ff40 1 --2- 172.21.15.38:0/1068835422 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5650fa072240 0x5650fa06d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:42.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.734+0000 7f0b9251ff40 1 -- 172.21.15.38:0/1068835422 >> 172.21.15.38:0/1068835422 conn(0x5650f9f90ac0 msgr2=0x5650fa13f0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:42.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.734+0000 7f0b9251ff40 1 -- 172.21.15.38:0/1068835422 shutdown_connections 2024-09-21T03:42:42.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.734+0000 7f0b9251ff40 1 -- 172.21.15.38:0/1068835422 wait complete. 2024-09-21T03:42:42.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.735+0000 7f0b9251ff40 1 Processor -- start 2024-09-21T03:42:42.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.735+0000 7f0b9251ff40 1 -- start start 2024-09-21T03:42:42.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.736+0000 7f0b9251ff40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5650fa072240 0x5650fa12bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:42.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.736+0000 7f0b9251ff40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5650fa0b9090 0x5650fa13e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:42.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.737+0000 7f0b9251ff40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5650fa12e520 0x5650fa120970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:42.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.737+0000 7f0b9251ff40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5650fa13fc50 con 0x5650fa072240 2024-09-21T03:42:42.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.737+0000 7f0b9251ff40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5650fa13ff30 con 0x5650fa12e520 2024-09-21T03:42:42.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.737+0000 7f0b9251ff40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x5650fa121700 con 0x5650fa0b9090 2024-09-21T03:42:42.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.737+0000 7f0b8b554640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5650fa072240 0x5650fa12bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:42.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.737+0000 7f0b8b554640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5650fa072240 0x5650fa12bad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38164/0 (socket says 172.21.15.38:38164) 2024-09-21T03:42:42.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.737+0000 7f0b8b554640 1 -- 172.21.15.38:0/1793792657 learned_addr learned my addr 172.21.15.38:0/1793792657 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:42:42.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.737+0000 7f0b8ad53640 1 --2- 172.21.15.38:0/1793792657 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5650fa0b9090 0x5650fa13e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:42.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.737+0000 7f0b8bd55640 1 --2- 172.21.15.38:0/1793792657 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5650fa12e520 0x5650fa120970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:42.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.737+0000 7f0b8b554640 1 -- 172.21.15.38:0/1793792657 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5650fa0b9090 msgr2=0x5650fa13e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:42.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.737+0000 7f0b8b554640 1 --2- 172.21.15.38:0/1793792657 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5650fa0b9090 0x5650fa13e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:42.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.737+0000 7f0b8b554640 1 -- 172.21.15.38:0/1793792657 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5650fa12e520 msgr2=0x5650fa120970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:42.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.737+0000 7f0b8b554640 1 --2- 172.21.15.38:0/1793792657 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5650fa12e520 0x5650fa120970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:42.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.737+0000 7f0b8b554640 1 -- 172.21.15.38:0/1793792657 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650fa12dc40 con 0x5650fa072240 2024-09-21T03:42:42.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.737+0000 7f0b8b554640 1 --2- 172.21.15.38:0/1793792657 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5650fa072240 0x5650fa12bad0 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7f0b7c00f700 tx=0x7f0b7c00fbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:42.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.737+0000 7f0b88d4f640 1 -- 172.21.15.38:0/1793792657 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b7c00cac0 con 0x5650fa072240 2024-09-21T03:42:42.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.737+0000 7f0b88d4f640 1 -- 172.21.15.38:0/1793792657 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0b7c014070 con 0x5650fa072240 2024-09-21T03:42:42.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.737+0000 7f0b9251ff40 1 -- 172.21.15.38:0/1793792657 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5650fa12d960 con 0x5650fa072240 2024-09-21T03:42:42.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.737+0000 7f0b88d4f640 1 -- 172.21.15.38:0/1793792657 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b7c019470 con 0x5650fa072240 2024-09-21T03:42:42.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.738+0000 7f0b9251ff40 1 -- 172.21.15.38:0/1793792657 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650fa129080 con 0x5650fa072240 2024-09-21T03:42:42.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.739+0000 7f0b88d4f640 1 -- 172.21.15.38:0/1793792657 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f0b7c019610 con 0x5650fa072240 2024-09-21T03:42:42.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.740+0000 7f0b88d4f640 1 --2- 172.21.15.38:0/1793792657 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0b78077570 0x7f0b78079a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:42.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.740+0000 7f0b88d4f640 1 -- 172.21.15.38:0/1793792657 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 8368+0+0 (secure 0 0 0) 0x7f0b7c09b1d0 con 0x5650fa072240 2024-09-21T03:42:42.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.740+0000 7f0b8ad53640 1 --2- 172.21.15.38:0/1793792657 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0b78077570 0x7f0b78079a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:42.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.740+0000 7f0b9251ff40 1 --2- 172.21.15.38:0/1793792657 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x5650fa126290 0x5650fa1266a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:42.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.740+0000 7f0b9251ff40 1 -- 172.21.15.38:0/1793792657 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:1 13.14 13:292cf221:::200.00000000:head [truncate 0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x5650fa124a90 con 0x5650fa126290 2024-09-21T03:42:42.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.740+0000 7f0b8bd55640 1 --2- 172.21.15.38:0/1793792657 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x5650fa126290 0x5650fa1266a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:42.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.740+0000 7f0b8ad53640 1 --2- 172.21.15.38:0/1793792657 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0b78077570 0x7f0b78079a80 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f0b80004fd0 tx=0x7f0b8000e040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:42.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.741+0000 7f0b8bd55640 1 --2- 172.21.15.38:0/1793792657 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x5650fa126290 0x5650fa1266a0 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:42.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.742+0000 7f0b8bd55640 1 -- 172.21.15.38:0/1793792657 <== osd.6 v2:172.21.15.195:6826/4055825675 1 ==== osd_op_reply(1 200.00000000 [truncate 0] v120'547 uv547 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0b8405f030 con 0x5650fa126290 2024-09-21T03:42:42.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.742+0000 7f0b9251ff40 1 -- 172.21.15.38:0/1793792657 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x5650fa126290 msgr2=0x5650fa1266a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:42.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.742+0000 7f0b9251ff40 1 --2- 172.21.15.38:0/1793792657 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x5650fa126290 0x5650fa1266a0 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:42.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.742+0000 7f0b9251ff40 1 -- 172.21.15.38:0/1793792657 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0b78077570 msgr2=0x7f0b78079a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:42.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.742+0000 7f0b9251ff40 1 --2- 172.21.15.38:0/1793792657 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0b78077570 0x7f0b78079a80 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f0b80004fd0 tx=0x7f0b8000e040 comp rx=0 tx=0).stop 2024-09-21T03:42:42.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.742+0000 7f0b9251ff40 1 -- 172.21.15.38:0/1793792657 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5650fa072240 msgr2=0x5650fa12bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:42.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.742+0000 7f0b9251ff40 1 --2- 172.21.15.38:0/1793792657 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5650fa072240 0x5650fa12bad0 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7f0b7c00f700 tx=0x7f0b7c00fbc0 comp rx=0 tx=0).stop 2024-09-21T03:42:42.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.742+0000 7f0b8bd55640 1 -- 172.21.15.38:0/1793792657 reap_dead start 2024-09-21T03:42:42.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.742+0000 7f0b9251ff40 1 -- 172.21.15.38:0/1793792657 shutdown_connections 2024-09-21T03:42:42.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.742+0000 7f0b9251ff40 1 -- 172.21.15.38:0/1793792657 >> 172.21.15.38:0/1793792657 conn(0x5650f9f90ac0 msgr2=0x5650fa09bd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:42.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.742+0000 7f0b9251ff40 1 -- 172.21.15.38:0/1793792657 shutdown_connections 2024-09-21T03:42:42.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.742+0000 7f0b9251ff40 1 -- 172.21.15.38:0/1793792657 wait complete. 2024-09-21T03:42:42.748 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-21T03:42:42.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.864+0000 7f7a44d19640 1 -- 172.21.15.38:0/1952264894 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7a400b9ad0 msgr2=0x7f7a400d0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:42.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.864+0000 7f7a44d19640 1 --2- 172.21.15.38:0/1952264894 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7a400b9ad0 0x7f7a400d0be0 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7f7a400cfba0 tx=0x7f7a2802a4b0 comp rx=0 tx=0).stop 2024-09-21T03:42:42.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.864+0000 7f7a44d19640 1 -- 172.21.15.38:0/1952264894 shutdown_connections 2024-09-21T03:42:42.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.864+0000 7f7a44d19640 1 --2- 172.21.15.38:0/1952264894 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7a400d1120 0x7f7a400ceb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:42.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.864+0000 7f7a44d19640 1 --2- 172.21.15.38:0/1952264894 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7a400b9ad0 0x7f7a400d0be0 unknown :-1 s=CLOSED pgs=1826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:42.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.864+0000 7f7a44d19640 1 --2- 172.21.15.38:0/1952264894 >> v2:172.21.15.195:3300/0 conn(0x7f7a4015ee80 0x7f7a400d0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:42.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.864+0000 7f7a44d19640 1 -- 172.21.15.38:0/1952264894 >> 172.21.15.38:0/1952264894 conn(0x7f7a4009d830 msgr2=0x7f7a400b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:42.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.864+0000 7f7a44d19640 1 -- 172.21.15.38:0/1952264894 shutdown_connections 2024-09-21T03:42:42.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.864+0000 7f7a44d19640 1 -- 172.21.15.38:0/1952264894 wait complete. 2024-09-21T03:42:42.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.867+0000 7f7a44d19640 1 Processor -- start 2024-09-21T03:42:42.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.867+0000 7f7a44d19640 1 -- start start 2024-09-21T03:42:42.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.868+0000 7f7a44d19640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7a400b9ad0 0x7f7a4014fa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:42.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.868+0000 7f7a44d19640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7a400d1120 0x7f7a400d3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:42.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.868+0000 7f7a44d19640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7a4015ee80 0x7f7a400d4130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:42.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.868+0000 7f7a44d19640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f7a400cf520 con 0x7f7a400d1120 2024-09-21T03:42:42.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.868+0000 7f7a44d19640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f7a400cf3a0 con 0x7f7a400b9ad0 2024-09-21T03:42:42.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.868+0000 7f7a44d19640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f7a400b6580 con 0x7f7a4015ee80 2024-09-21T03:42:42.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.868+0000 7f7a35d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7a400d1120 0x7f7a400d3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:42.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.868+0000 7f7a35d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7a400d1120 0x7f7a400d3bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38184/0 (socket says 172.21.15.38:38184) 2024-09-21T03:42:42.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.868+0000 7f7a35d9b640 1 -- 172.21.15.38:0/768653395 learned_addr learned my addr 172.21.15.38:0/768653395 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:42:42.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.868+0000 7f7a3e59c640 1 --2- 172.21.15.38:0/768653395 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7a400b9ad0 0x7f7a4014fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:42.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.868+0000 7f7a3ed9d640 1 --2- 172.21.15.38:0/768653395 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7a4015ee80 0x7f7a400d4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:42.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.868+0000 7f7a35d9b640 1 -- 172.21.15.38:0/768653395 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7a4015ee80 msgr2=0x7f7a400d4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:42.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.868+0000 7f7a35d9b640 1 --2- 172.21.15.38:0/768653395 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7a4015ee80 0x7f7a400d4130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:42.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.868+0000 7f7a35d9b640 1 -- 172.21.15.38:0/768653395 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7a400b9ad0 msgr2=0x7f7a4014fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:42.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.868+0000 7f7a35d9b640 1 --2- 172.21.15.38:0/768653395 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7a400b9ad0 0x7f7a4014fa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:42.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.868+0000 7f7a35d9b640 1 -- 172.21.15.38:0/768653395 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a4006e750 con 0x7f7a400d1120 2024-09-21T03:42:42.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.869+0000 7f7a35d9b640 1 --2- 172.21.15.38:0/768653395 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7a400d1120 0x7f7a400d3bf0 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7f7a280099d0 tx=0x7f7a280052f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:42.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.869+0000 7f7a377fe640 1 -- 172.21.15.38:0/768653395 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a28033e60 con 0x7f7a400d1120 2024-09-21T03:42:42.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.869+0000 7f7a377fe640 1 -- 172.21.15.38:0/768653395 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7a28031c40 con 0x7f7a400d1120 2024-09-21T03:42:42.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.869+0000 7f7a44d19640 1 -- 172.21.15.38:0/768653395 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a4006ea30 con 0x7f7a400d1120 2024-09-21T03:42:42.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.869+0000 7f7a377fe640 1 -- 172.21.15.38:0/768653395 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a28005cf0 con 0x7f7a400d1120 2024-09-21T03:42:42.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.869+0000 7f7a44d19640 1 -- 172.21.15.38:0/768653395 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a4006eff0 con 0x7f7a400d1120 2024-09-21T03:42:42.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.870+0000 7f7a44d19640 1 -- 172.21.15.38:0/768653395 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a4006e4c0 con 0x7f7a400d1120 2024-09-21T03:42:42.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.871+0000 7f7a377fe640 1 -- 172.21.15.38:0/768653395 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f7a28033330 con 0x7f7a400d1120 2024-09-21T03:42:42.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.871+0000 7f7a377fe640 1 --2- 172.21.15.38:0/768653395 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7a20078bc0 0x7f7a2007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:42.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.871+0000 7f7a377fe640 1 -- 172.21.15.38:0/768653395 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 8368+0+0 (secure 0 0 0) 0x7f7a280bc8b0 con 0x7f7a400d1120 2024-09-21T03:42:42.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.871+0000 7f7a3e59c640 1 --2- 172.21.15.38:0/768653395 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7a20078bc0 0x7f7a2007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:42.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.872+0000 7f7a3e59c640 1 --2- 172.21.15.38:0/768653395 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7a20078bc0 0x7f7a2007b0d0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f7a30009b30 tx=0x7f7a30005f90 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:42.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:42.876+0000 7f7a377fe640 1 -- 172.21.15.38:0/768653395 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f7a2803f030 con 0x7f7a400d1120 2024-09-21T03:42:43.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:43.104+0000 7f7a44d19640 1 -- 172.21.15.38:0/768653395 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f7a400b5bd0 con 0x7f7a400d1120 2024-09-21T03:42:43.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:43.381+0000 7f7a377fe640 1 -- 172.21.15.38:0/768653395 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v230) ==== 162+0+0 (secure 0 0 0) 0x7f7a400b5bd0 con 0x7f7a400d1120 2024-09-21T03:42:43.384 INFO:teuthology.orchestra.run.smithi038.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-21T03:42:43.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:43.384+0000 7f7a44d19640 1 -- 172.21.15.38:0/768653395 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7a20078bc0 msgr2=0x7f7a2007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:43.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:43.384+0000 7f7a44d19640 1 --2- 172.21.15.38:0/768653395 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7a20078bc0 0x7f7a2007b0d0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f7a30009b30 tx=0x7f7a30005f90 comp rx=0 tx=0).stop 2024-09-21T03:42:43.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:43.384+0000 7f7a44d19640 1 -- 172.21.15.38:0/768653395 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7a400d1120 msgr2=0x7f7a400d3bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:43.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:43.384+0000 7f7a44d19640 1 --2- 172.21.15.38:0/768653395 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7a400d1120 0x7f7a400d3bf0 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7f7a280099d0 tx=0x7f7a280052f0 comp rx=0 tx=0).stop 2024-09-21T03:42:43.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:43.384+0000 7f7a44d19640 1 -- 172.21.15.38:0/768653395 shutdown_connections 2024-09-21T03:42:43.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:43.384+0000 7f7a44d19640 1 --2- 172.21.15.38:0/768653395 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7a20078bc0 0x7f7a2007b0d0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:43.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:43.384+0000 7f7a44d19640 1 --2- 172.21.15.38:0/768653395 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7a4015ee80 0x7f7a400d4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:43.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:43.384+0000 7f7a44d19640 1 --2- 172.21.15.38:0/768653395 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7a400d1120 0x7f7a400d3bf0 unknown :-1 s=CLOSED pgs=1827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:43.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:43.384+0000 7f7a44d19640 1 --2- 172.21.15.38:0/768653395 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7a400b9ad0 0x7f7a4014fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:43.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:43.384+0000 7f7a44d19640 1 -- 172.21.15.38:0/768653395 >> 172.21.15.38:0/768653395 conn(0x7f7a4009d830 msgr2=0x7f7a40161b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:43.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:43.384+0000 7f7a44d19640 1 -- 172.21.15.38:0/768653395 shutdown_connections 2024-09-21T03:42:43.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:43.384+0000 7f7a44d19640 1 -- 172.21.15.38:0/768653395 wait complete. 2024-09-21T03:42:43.396 INFO:tasks.ceph.mds.c.smithi038.stderr:2024-09-21T03:42:43.393+0000 7f7bcc751640 -1 log_channel(cluster) log [ERR] : Error loading MDS rank 0: (22) Invalid argument 2024-09-21T03:42:43.396 INFO:tasks.ceph.mon.a.smithi038.stderr:2024-09-21T03:42:43.394+0000 7f3f45713640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T03:42:43.397 INFO:tasks.ceph.mon.a.smithi038.stderr:2024-09-21T03:42:43.394+0000 7f3f45713640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-09-21T03:42:43.403 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:42:43.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:43.517+0000 7f14bda69640 1 -- 172.21.15.38:0/3958828093 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f14b80b6250 msgr2=0x7f14b80b6660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:43.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:43.517+0000 7f14bda69640 1 --2- 172.21.15.38:0/3958828093 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f14b80b6250 0x7f14b80b6660 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7f14b8074a40 tx=0x7f14b002aa30 comp rx=0 tx=0).stop 2024-09-21T03:42:43.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:43.517+0000 7f14bda69640 1 -- 172.21.15.38:0/3958828093 shutdown_connections 2024-09-21T03:42:43.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:43.517+0000 7f14bda69640 1 --2- 172.21.15.38:0/3958828093 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f14b80b6250 0x7f14b80b6660 unknown :-1 s=CLOSED pgs=1828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:43.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:43.517+0000 7f14bda69640 1 --2- 172.21.15.38:0/3958828093 >> v2:172.21.15.195:3300/0 conn(0x7f14b815cb10 0x7f14b815cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:43.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:43.517+0000 7f14bda69640 1 -- 172.21.15.38:0/3958828093 >> 172.21.15.38:0/3958828093 conn(0x7f14b80d2350 msgr2=0x7f14b80d2760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:43.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:43.517+0000 7f14bda69640 1 -- 172.21.15.38:0/3958828093 shutdown_connections 2024-09-21T03:42:43.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:43.517+0000 7f14bda69640 1 -- 172.21.15.38:0/3958828093 wait complete. 2024-09-21T03:42:43.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:43.519+0000 7f14bda69640 1 Processor -- start 2024-09-21T03:42:43.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:43.519+0000 7f14bda69640 1 -- start start 2024-09-21T03:42:43.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:43.521+0000 7f14bda69640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f14b80b6250 0x7f14b81e9690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:43.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:43.521+0000 7f14bda69640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f14b80d12e0 0x7f14b81f3520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:43.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:43.521+0000 7f14bda69640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f14b815cb10 0x7f14b815bee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:43.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:43.521+0000 7f14bda69640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f14b80b6d20 con 0x7f14b815cb10 2024-09-21T03:42:43.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:43.521+0000 7f14bda69640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f14b80b6ba0 con 0x7f14b80d12e0 2024-09-21T03:42:43.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:43.521+0000 7f14bda69640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f14b80b6ea0 con 0x7f14b80b6250 2024-09-21T03:42:43.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:43.521+0000 7f14b7fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f14b815cb10 0x7f14b815bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:43.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:43.521+0000 7f14b7fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f14b815cb10 0x7f14b815bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38204/0 (socket says 172.21.15.38:38204) 2024-09-21T03:42:43.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:43.521+0000 7f14b7fff640 1 -- 172.21.15.38:0/1264098537 learned_addr learned my addr 172.21.15.38:0/1264098537 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:42:43.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:43.521+0000 7f14b77fe640 1 --2- 172.21.15.38:0/1264098537 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f14b80b6250 0x7f14b81e9690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:43.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:43.521+0000 7f14b6ffd640 1 --2- 172.21.15.38:0/1264098537 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f14b80d12e0 0x7f14b81f3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:43.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:43.521+0000 7f14b7fff640 1 -- 172.21.15.38:0/1264098537 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f14b80b6250 msgr2=0x7f14b81e9690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:43.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:43.521+0000 7f14b7fff640 1 --2- 172.21.15.38:0/1264098537 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f14b80b6250 0x7f14b81e9690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:43.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:43.521+0000 7f14b7fff640 1 -- 172.21.15.38:0/1264098537 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f14b80d12e0 msgr2=0x7f14b81f3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:43.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:43.521+0000 7f14b7fff640 1 --2- 172.21.15.38:0/1264098537 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f14b80d12e0 0x7f14b81f3520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:43.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:43.521+0000 7f14b7fff640 1 -- 172.21.15.38:0/1264098537 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14b81e85f0 con 0x7f14b815cb10 2024-09-21T03:42:43.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:43.521+0000 7f14b7fff640 1 --2- 172.21.15.38:0/1264098537 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f14b815cb10 0x7f14b815bee0 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7f14b0009d60 tx=0x7f14b0003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:43.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:43.521+0000 7f149ffff640 1 -- 172.21.15.38:0/1264098537 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14b0033050 con 0x7f14b815cb10 2024-09-21T03:42:43.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:43.521+0000 7f149ffff640 1 -- 172.21.15.38:0/1264098537 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f14b0004450 con 0x7f14b815cb10 2024-09-21T03:42:43.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:43.522+0000 7f14bda69640 1 -- 172.21.15.38:0/1264098537 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14b81e88d0 con 0x7f14b815cb10 2024-09-21T03:42:43.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:43.522+0000 7f149ffff640 1 -- 172.21.15.38:0/1264098537 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14b0038450 con 0x7f14b815cb10 2024-09-21T03:42:43.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:43.522+0000 7f14bda69640 1 -- 172.21.15.38:0/1264098537 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14b81e8e90 con 0x7f14b815cb10 2024-09-21T03:42:43.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:43.523+0000 7f14bda69640 1 -- 172.21.15.38:0/1264098537 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14b80b7350 con 0x7f14b815cb10 2024-09-21T03:42:43.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:43.523+0000 7f149ffff640 1 -- 172.21.15.38:0/1264098537 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f14b0036070 con 0x7f14b815cb10 2024-09-21T03:42:43.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:43.524+0000 7f149ffff640 1 --2- 172.21.15.38:0/1264098537 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f14a4078b90 0x7f14a407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:43.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:43.524+0000 7f149ffff640 1 -- 172.21.15.38:0/1264098537 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 8454+0+0 (secure 0 0 0) 0x7f14b00bcc00 con 0x7f14b815cb10 2024-09-21T03:42:43.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:43.524+0000 7f14b77fe640 1 --2- 172.21.15.38:0/1264098537 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f14a4078b90 0x7f14a407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:43.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:43.525+0000 7f14b77fe640 1 --2- 172.21.15.38:0/1264098537 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f14a4078b90 0x7f14a407b0a0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f14a8004630 tx=0x7f14a800b2e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:43.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:43.529+0000 7f149ffff640 1 -- 172.21.15.38:0/1264098537 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f14b0083f10 con 0x7f14b815cb10 2024-09-21T03:42:43.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:43.754+0000 7f14bda69640 1 -- 172.21.15.38:0/1264098537 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f14b80b56b0 con 0x7f14b815cb10 2024-09-21T03:42:43.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:43.755+0000 7f149ffff640 1 -- 172.21.15.38:0/1264098537 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 232 v232) ==== 95+0+3566 (secure 0 0 0) 0x7f14b0088940 con 0x7f14b815cb10 2024-09-21T03:42:43.758 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:42:43.758 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":232,"btime":"2024-09-21T03:42:43:395891+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6086,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/965972663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":965972663},{"type":"v1","addr":"172.21.15.195:6836","nonce":965972663}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6700,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/3176122334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":3176122334},{"type":"v1","addr":"172.21.15.195:6837","nonce":3176122334}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":232,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:40:02.319282+0000","modified":"2024-09-21T03:42:43.395821+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":121,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-21T03:42:43.758 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 232 2024-09-21T03:42:43.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:43.757+0000 7f14bda69640 1 -- 172.21.15.38:0/1264098537 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f14a4078b90 msgr2=0x7f14a407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:43.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:43.757+0000 7f14bda69640 1 --2- 172.21.15.38:0/1264098537 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f14a4078b90 0x7f14a407b0a0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f14a8004630 tx=0x7f14a800b2e0 comp rx=0 tx=0).stop 2024-09-21T03:42:43.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:43.757+0000 7f14bda69640 1 -- 172.21.15.38:0/1264098537 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f14b815cb10 msgr2=0x7f14b815bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:43.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:43.757+0000 7f14bda69640 1 --2- 172.21.15.38:0/1264098537 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f14b815cb10 0x7f14b815bee0 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7f14b0009d60 tx=0x7f14b0003f10 comp rx=0 tx=0).stop 2024-09-21T03:42:43.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:43.757+0000 7f14bda69640 1 -- 172.21.15.38:0/1264098537 shutdown_connections 2024-09-21T03:42:43.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:43.757+0000 7f14bda69640 1 --2- 172.21.15.38:0/1264098537 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f14b815cb10 0x7f14b815bee0 unknown :-1 s=CLOSED pgs=1829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:43.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:43.757+0000 7f14bda69640 1 --2- 172.21.15.38:0/1264098537 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f14b80d12e0 0x7f14b81f3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:43.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:43.757+0000 7f14bda69640 1 --2- 172.21.15.38:0/1264098537 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f14a4078b90 0x7f14a407b0a0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:43.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:43.757+0000 7f14bda69640 1 --2- 172.21.15.38:0/1264098537 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f14b80b6250 0x7f14b81e9690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:43.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:43.757+0000 7f14bda69640 1 -- 172.21.15.38:0/1264098537 >> 172.21.15.38:0/1264098537 conn(0x7f14b80d2350 msgr2=0x7f14b80d2760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:43.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:43.757+0000 7f14bda69640 1 -- 172.21.15.38:0/1264098537 shutdown_connections 2024-09-21T03:42:43.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:43.757+0000 7f14bda69640 1 -- 172.21.15.38:0/1264098537 wait complete. 2024-09-21T03:42:43.771 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-21T03:42:43.772 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:42:43.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:43.886+0000 7fb4a75ba640 1 -- 172.21.15.38:0/2845133682 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb4a0074290 msgr2=0x7fb4a015ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:43.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:43.886+0000 7fb4a75ba640 1 --2- 172.21.15.38:0/2845133682 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb4a0074290 0x7fb4a015ce90 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 crypto rx=0x7fb494005fd0 tx=0x7fb49402a5d0 comp rx=0 tx=0).stop 2024-09-21T03:42:43.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:43.886+0000 7fb4a75ba640 1 -- 172.21.15.38:0/2845133682 shutdown_connections 2024-09-21T03:42:43.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:43.886+0000 7fb4a75ba640 1 --2- 172.21.15.38:0/2845133682 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb4a0074290 0x7fb4a015ce90 unknown :-1 s=CLOSED pgs=1830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:43.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:43.886+0000 7fb4a75ba640 1 --2- 172.21.15.38:0/2845133682 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb4a00d0be0 0x7fb4a0073d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:43.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:43.886+0000 7fb4a75ba640 1 -- 172.21.15.38:0/2845133682 >> 172.21.15.38:0/2845133682 conn(0x7fb4a00b2cc0 msgr2=0x7fb4a00b30d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:43.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:43.886+0000 7fb4a75ba640 1 -- 172.21.15.38:0/2845133682 shutdown_connections 2024-09-21T03:42:43.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:43.886+0000 7fb4a75ba640 1 -- 172.21.15.38:0/2845133682 wait complete. 2024-09-21T03:42:43.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:43.888+0000 7fb4a75ba640 1 Processor -- start 2024-09-21T03:42:43.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:43.888+0000 7fb4a75ba640 1 -- start start 2024-09-21T03:42:43.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:43.889+0000 7fb4a75ba640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb4a0074290 0x7fb4a01f3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:43.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:43.889+0000 7fb4a75ba640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb4a00d0be0 0x7fb4a014df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:43.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:43.889+0000 7fb4a75ba640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb4a015d3d0 0x7fb4a01592f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:43.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:43.889+0000 7fb4a75ba640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a015d960 con 0x7fb4a0074290 2024-09-21T03:42:43.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:43.889+0000 7fb4a75ba640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a015d7e0 con 0x7fb4a015d3d0 2024-09-21T03:42:43.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:43.889+0000 7fb4a75ba640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fb4a0160880 con 0x7fb4a00d0be0 2024-09-21T03:42:43.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:43.889+0000 7fb4a5356640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb4a0074290 0x7fb4a01f3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:43.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:43.889+0000 7fb4a5356640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb4a0074290 0x7fb4a01f3460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38226/0 (socket says 172.21.15.38:38226) 2024-09-21T03:42:43.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:43.889+0000 7fb4a5356640 1 -- 172.21.15.38:0/2264539939 learned_addr learned my addr 172.21.15.38:0/2264539939 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:42:43.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:43.889+0000 7fb4a4b55640 1 --2- 172.21.15.38:0/2264539939 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb4a00d0be0 0x7fb4a014df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:43.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:43.889+0000 7fb4a5b57640 1 --2- 172.21.15.38:0/2264539939 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb4a015d3d0 0x7fb4a01592f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:43.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:43.890+0000 7fb4a5356640 1 -- 172.21.15.38:0/2264539939 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb4a00d0be0 msgr2=0x7fb4a014df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:43.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:43.890+0000 7fb4a5356640 1 --2- 172.21.15.38:0/2264539939 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb4a00d0be0 0x7fb4a014df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:43.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:43.890+0000 7fb4a5356640 1 -- 172.21.15.38:0/2264539939 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb4a015d3d0 msgr2=0x7fb4a01592f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:43.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:43.890+0000 7fb4a5356640 1 --2- 172.21.15.38:0/2264539939 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb4a015d3d0 0x7fb4a01592f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:43.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:43.890+0000 7fb4a5356640 1 -- 172.21.15.38:0/2264539939 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4a01f3dd0 con 0x7fb4a0074290 2024-09-21T03:42:43.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:43.890+0000 7fb4a5356640 1 --2- 172.21.15.38:0/2264539939 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb4a0074290 0x7fb4a01f3460 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7fb49800d7a0 tx=0x7fb49800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:43.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:43.890+0000 7fb49dffb640 1 -- 172.21.15.38:0/2264539939 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb498017070 con 0x7fb4a0074290 2024-09-21T03:42:43.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:43.890+0000 7fb49dffb640 1 -- 172.21.15.38:0/2264539939 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb498009d20 con 0x7fb4a0074290 2024-09-21T03:42:43.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:43.890+0000 7fb4a75ba640 1 -- 172.21.15.38:0/2264539939 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4a01f40b0 con 0x7fb4a0074290 2024-09-21T03:42:43.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:43.890+0000 7fb49dffb640 1 -- 172.21.15.38:0/2264539939 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb498004280 con 0x7fb4a0074290 2024-09-21T03:42:43.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:43.890+0000 7fb4a75ba640 1 -- 172.21.15.38:0/2264539939 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4a0157cb0 con 0x7fb4a0074290 2024-09-21T03:42:43.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:43.891+0000 7fb4a75ba640 1 -- 172.21.15.38:0/2264539939 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4a0160da0 con 0x7fb4a0074290 2024-09-21T03:42:43.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:43.892+0000 7fb49dffb640 1 -- 172.21.15.38:0/2264539939 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb498005020 con 0x7fb4a0074290 2024-09-21T03:42:43.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:43.892+0000 7fb49dffb640 1 --2- 172.21.15.38:0/2264539939 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb48c078b90 0x7fb48c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:43.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:43.893+0000 7fb49dffb640 1 -- 172.21.15.38:0/2264539939 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 8454+0+0 (secure 0 0 0) 0x7fb49809bb30 con 0x7fb4a0074290 2024-09-21T03:42:43.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:43.893+0000 7fb4a4b55640 1 --2- 172.21.15.38:0/2264539939 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb48c078b90 0x7fb48c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:43.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:43.893+0000 7fb4a4b55640 1 --2- 172.21.15.38:0/2264539939 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb48c078b90 0x7fb48c07b0a0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fb494033000 tx=0x7fb494004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:43.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:43.897+0000 7fb49dffb640 1 -- 172.21.15.38:0/2264539939 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fb498020020 con 0x7fb4a0074290 2024-09-21T03:42:44.113 INFO:tasks.ceph.mds.a.smithi038.stderr: -43> 2024-09-21T03:42:39.360+0000 7f7ed144c640 -1 mds.0.cache.ino(0x1) No data while reading inode 0x1 2024-09-21T03:42:44.114 INFO:tasks.ceph.mds.a.smithi038.stderr: -41> 2024-09-21T03:42:39.360+0000 7f7ed144c640 -1 log_channel(cluster) log [ERR] : Error loading MDS rank 0: (2) No such file or directory 2024-09-21T03:42:44.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:44.125+0000 7fb4a75ba640 1 -- 172.21.15.38:0/2264539939 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb4a015c6d0 con 0x7fb4a0074290 2024-09-21T03:42:44.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:44.126+0000 7fb49dffb640 1 -- 172.21.15.38:0/2264539939 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 232 v232) ==== 95+0+3566 (secure 0 0 0) 0x7fb498062e10 con 0x7fb4a0074290 2024-09-21T03:42:44.129 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:42:44.129 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":232,"btime":"2024-09-21T03:42:43:395891+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6086,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/965972663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":965972663},{"type":"v1","addr":"172.21.15.195:6836","nonce":965972663}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6700,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/3176122334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":3176122334},{"type":"v1","addr":"172.21.15.195:6837","nonce":3176122334}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":232,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:40:02.319282+0000","modified":"2024-09-21T03:42:43.395821+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":121,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-21T03:42:44.129 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 232 2024-09-21T03:42:44.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:44.128+0000 7fb4a75ba640 1 -- 172.21.15.38:0/2264539939 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb48c078b90 msgr2=0x7fb48c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:44.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:44.128+0000 7fb4a75ba640 1 --2- 172.21.15.38:0/2264539939 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb48c078b90 0x7fb48c07b0a0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fb494033000 tx=0x7fb494004730 comp rx=0 tx=0).stop 2024-09-21T03:42:44.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:44.128+0000 7fb4a75ba640 1 -- 172.21.15.38:0/2264539939 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb4a0074290 msgr2=0x7fb4a01f3460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:44.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:44.128+0000 7fb4a75ba640 1 --2- 172.21.15.38:0/2264539939 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb4a0074290 0x7fb4a01f3460 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7fb49800d7a0 tx=0x7fb49800dc60 comp rx=0 tx=0).stop 2024-09-21T03:42:44.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:44.128+0000 7fb4a75ba640 1 -- 172.21.15.38:0/2264539939 shutdown_connections 2024-09-21T03:42:44.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:44.128+0000 7fb4a75ba640 1 --2- 172.21.15.38:0/2264539939 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb4a015d3d0 0x7fb4a01592f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:44.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:44.128+0000 7fb4a75ba640 1 --2- 172.21.15.38:0/2264539939 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb48c078b90 0x7fb48c07b0a0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:44.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:44.128+0000 7fb4a75ba640 1 --2- 172.21.15.38:0/2264539939 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb4a00d0be0 0x7fb4a014df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:44.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:44.128+0000 7fb4a75ba640 1 --2- 172.21.15.38:0/2264539939 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb4a0074290 0x7fb4a01f3460 unknown :-1 s=CLOSED pgs=1831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:44.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:44.128+0000 7fb4a75ba640 1 -- 172.21.15.38:0/2264539939 >> 172.21.15.38:0/2264539939 conn(0x7fb4a00b2cc0 msgr2=0x7fb4a015a7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:44.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:44.129+0000 7fb4a75ba640 1 -- 172.21.15.38:0/2264539939 shutdown_connections 2024-09-21T03:42:44.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:44.129+0000 7fb4a75ba640 1 -- 172.21.15.38:0/2264539939 wait complete. 2024-09-21T03:42:44.144 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-21T03:42:44.144 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:42:44.168 INFO:tasks.ceph.mds.a.smithi038.stdout:starting mds.a at 2024-09-21T03:42:44.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:44.259+0000 7f91da2a9640 1 -- 172.21.15.38:0/3055963754 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f91d4162280 msgr2=0x7f91d4162690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:44.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:44.259+0000 7f91da2a9640 1 --2- 172.21.15.38:0/3055963754 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f91d4162280 0x7f91d4162690 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7f91c8005fd0 tx=0x7f91c802a5d0 comp rx=0 tx=0).stop 2024-09-21T03:42:44.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:44.259+0000 7f91da2a9640 1 -- 172.21.15.38:0/3055963754 shutdown_connections 2024-09-21T03:42:44.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:44.259+0000 7f91da2a9640 1 --2- 172.21.15.38:0/3055963754 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f91d4162280 0x7f91d4162690 unknown :-1 s=CLOSED pgs=1835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:44.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:44.259+0000 7f91da2a9640 1 --2- 172.21.15.38:0/3055963754 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f91d4162f00 0x7f91d40d24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:44.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:44.259+0000 7f91da2a9640 1 -- 172.21.15.38:0/3055963754 >> 172.21.15.38:0/3055963754 conn(0x7f91d40b6b20 msgr2=0x7f91d40d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:44.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:44.259+0000 7f91da2a9640 1 -- 172.21.15.38:0/3055963754 shutdown_connections 2024-09-21T03:42:44.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:44.259+0000 7f91da2a9640 1 -- 172.21.15.38:0/3055963754 wait complete. 2024-09-21T03:42:44.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:44.261+0000 7f91da2a9640 1 Processor -- start 2024-09-21T03:42:44.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:44.261+0000 7f91da2a9640 1 -- start start 2024-09-21T03:42:44.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:44.262+0000 7f91da2a9640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f91d40b2e40 0x7f91d415a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:44.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:44.262+0000 7f91da2a9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f91d4162280 0x7f91d40ac6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:44.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:44.262+0000 7f91da2a9640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f91d41f90b0 0x7f91d41f84a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:44.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:44.262+0000 7f91da2a9640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f91d415e630 con 0x7f91d4162280 2024-09-21T03:42:44.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:44.262+0000 7f91da2a9640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f91d415e4b0 con 0x7f91d41f90b0 2024-09-21T03:42:44.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:44.262+0000 7f91da2a9640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f91d415e7b0 con 0x7f91d40b2e40 2024-09-21T03:42:44.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:44.262+0000 7f91d37fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f91d4162280 0x7f91d40ac6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:44.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:44.262+0000 7f91d37fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f91d4162280 0x7f91d40ac6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38266/0 (socket says 172.21.15.38:38266) 2024-09-21T03:42:44.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:44.262+0000 7f91d37fe640 1 -- 172.21.15.38:0/2407181499 learned_addr learned my addr 172.21.15.38:0/2407181499 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:42:44.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:44.262+0000 7f91d3fff640 1 --2- 172.21.15.38:0/2407181499 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f91d40b2e40 0x7f91d415a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:44.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:44.262+0000 7f91d8846640 1 --2- 172.21.15.38:0/2407181499 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f91d41f90b0 0x7f91d41f84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:44.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:44.263+0000 7f91d37fe640 1 -- 172.21.15.38:0/2407181499 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f91d40b2e40 msgr2=0x7f91d415a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:44.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:44.263+0000 7f91d37fe640 1 --2- 172.21.15.38:0/2407181499 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f91d40b2e40 0x7f91d415a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:44.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:44.263+0000 7f91d37fe640 1 -- 172.21.15.38:0/2407181499 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f91d41f90b0 msgr2=0x7f91d41f84a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:44.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:44.263+0000 7f91d37fe640 1 --2- 172.21.15.38:0/2407181499 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f91d41f90b0 0x7f91d41f84a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:44.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:44.263+0000 7f91d37fe640 1 -- 172.21.15.38:0/2407181499 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91d4158d30 con 0x7f91d4162280 2024-09-21T03:42:44.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:44.263+0000 7f91d37fe640 1 --2- 172.21.15.38:0/2407181499 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f91d4162280 0x7f91d40ac6b0 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7f91c8006010 tx=0x7f91c80038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:44.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:44.263+0000 7f91d0ff9640 1 -- 172.21.15.38:0/2407181499 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91c803e030 con 0x7f91d4162280 2024-09-21T03:42:44.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:44.263+0000 7f91d0ff9640 1 -- 172.21.15.38:0/2407181499 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f91c8003ca0 con 0x7f91d4162280 2024-09-21T03:42:44.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:44.263+0000 7f91da2a9640 1 -- 172.21.15.38:0/2407181499 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91d4159010 con 0x7f91d4162280 2024-09-21T03:42:44.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:44.263+0000 7f91d0ff9640 1 -- 172.21.15.38:0/2407181499 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91c8035040 con 0x7f91d4162280 2024-09-21T03:42:44.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:44.263+0000 7f91da2a9640 1 -- 172.21.15.38:0/2407181499 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91d41595d0 con 0x7f91d4162280 2024-09-21T03:42:44.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:44.264+0000 7f91da2a9640 1 -- 172.21.15.38:0/2407181499 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91d4162f00 con 0x7f91d4162280 2024-09-21T03:42:44.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:44.265+0000 7f91d0ff9640 1 -- 172.21.15.38:0/2407181499 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f91c8041030 con 0x7f91d4162280 2024-09-21T03:42:44.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:44.266+0000 7f91d0ff9640 1 --2- 172.21.15.38:0/2407181499 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f91c0078b90 0x7f91c007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:44.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:44.266+0000 7f91d0ff9640 1 -- 172.21.15.38:0/2407181499 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 8454+0+0 (secure 0 0 0) 0x7f91c80bbcd0 con 0x7f91d4162280 2024-09-21T03:42:44.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:44.266+0000 7f91d3fff640 1 --2- 172.21.15.38:0/2407181499 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f91c0078b90 0x7f91c007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:44.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:44.266+0000 7f91d3fff640 1 --2- 172.21.15.38:0/2407181499 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f91c0078b90 0x7f91c007b0a0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f91c4004630 tx=0x7f91c400b2e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:44.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:44.271+0000 7f91d0ff9640 1 -- 172.21.15.38:0/2407181499 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f91c80830d0 con 0x7f91d4162280 2024-09-21T03:42:44.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:44.498+0000 7f91da2a9640 1 -- 172.21.15.38:0/2407181499 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f91d40b8f70 con 0x7f91d4162280 2024-09-21T03:42:44.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:44.498+0000 7f91d0ff9640 1 -- 172.21.15.38:0/2407181499 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 233 v233) ==== 95+0+4392 (secure 0 0 0) 0x7f91c8087b00 con 0x7f91d4162280 2024-09-21T03:42:44.501 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:42:44.501 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":233,"btime":"2024-09-21T03:42:44:397318+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6086,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/965972663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":965972663},{"type":"v1","addr":"172.21.15.195:6836","nonce":965972663}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6700,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/3176122334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":3176122334},{"type":"v1","addr":"172.21.15.195:6837","nonce":3176122334}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":9372,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/2798021708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":2798021708},{"type":"v1","addr":"172.21.15.38:6835","nonce":2798021708}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":233}],"filesystems":[{"mdsmap":{"epoch":232,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:40:02.319282+0000","modified":"2024-09-21T03:42:43.395821+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":121,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-21T03:42:44.502 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 233 2024-09-21T03:42:44.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:44.500+0000 7f91da2a9640 1 -- 172.21.15.38:0/2407181499 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f91c0078b90 msgr2=0x7f91c007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:44.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:44.500+0000 7f91da2a9640 1 --2- 172.21.15.38:0/2407181499 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f91c0078b90 0x7f91c007b0a0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f91c4004630 tx=0x7f91c400b2e0 comp rx=0 tx=0).stop 2024-09-21T03:42:44.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:44.500+0000 7f91da2a9640 1 -- 172.21.15.38:0/2407181499 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f91d4162280 msgr2=0x7f91d40ac6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:44.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:44.500+0000 7f91da2a9640 1 --2- 172.21.15.38:0/2407181499 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f91d4162280 0x7f91d40ac6b0 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7f91c8006010 tx=0x7f91c80038a0 comp rx=0 tx=0).stop 2024-09-21T03:42:44.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:44.500+0000 7f91da2a9640 1 -- 172.21.15.38:0/2407181499 shutdown_connections 2024-09-21T03:42:44.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:44.500+0000 7f91da2a9640 1 --2- 172.21.15.38:0/2407181499 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f91d41f90b0 0x7f91d41f84a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:44.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:44.500+0000 7f91da2a9640 1 --2- 172.21.15.38:0/2407181499 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f91d4162280 0x7f91d40ac6b0 unknown :-1 s=CLOSED pgs=1836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:44.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:44.500+0000 7f91da2a9640 1 --2- 172.21.15.38:0/2407181499 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f91c0078b90 0x7f91c007b0a0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:44.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:44.500+0000 7f91da2a9640 1 --2- 172.21.15.38:0/2407181499 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f91d40b2e40 0x7f91d415a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:44.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:44.500+0000 7f91da2a9640 1 -- 172.21.15.38:0/2407181499 >> 172.21.15.38:0/2407181499 conn(0x7f91d40b6b20 msgr2=0x7f91d415da30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:44.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:44.501+0000 7f91da2a9640 1 -- 172.21.15.38:0/2407181499 shutdown_connections 2024-09-21T03:42:44.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:44.501+0000 7f91da2a9640 1 -- 172.21.15.38:0/2407181499 wait complete. 2024-09-21T03:42:44.515 INFO:tasks.cephfs.test_damage:Result: Mutation 'Truncate 200.00000000' led to DAMAGED state 2024-09-21T03:42:44.515 INFO:tasks.cephfs.test_damage:Applying mutation 'Truncate 200.00000001' 2024-09-21T03:42:44.515 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-21T03:42:44.515 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:42:44.571 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056795,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2354 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T03:42:44.571 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi038.front.sepia.ceph.com 2024-09-21T03:42:44.571 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:42:44.572 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T03:42:44.627 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T03:42:44.627 INFO:teuthology.orchestra.run.smithi038.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-09-21T03:42:44.627 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-21T03:42:44.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:44.782+0000 7fce891da640 1 -- 172.21.15.38:0/1413035473 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fce840d0bb0 msgr2=0x7fce84070100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:44.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:44.782+0000 7fce891da640 1 --2- 172.21.15.38:0/1413035473 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fce840d0bb0 0x7fce84070100 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7fce84071ec0 tx=0x7fce7402a980 comp rx=0 tx=0).stop 2024-09-21T03:42:44.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:44.782+0000 7fce891da640 1 -- 172.21.15.38:0/1413035473 shutdown_connections 2024-09-21T03:42:44.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:44.782+0000 7fce891da640 1 --2- 172.21.15.38:0/1413035473 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fce840d0bb0 0x7fce84070100 unknown :-1 s=CLOSED pgs=1837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:44.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:44.782+0000 7fce891da640 1 --2- 172.21.15.38:0/1413035473 >> v2:172.21.15.195:3300/0 conn(0x7fce840cd440 0x7fce840d0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:44.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:44.782+0000 7fce891da640 1 --2- 172.21.15.38:0/1413035473 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fce840ccc40 0x7fce8415ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:44.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:44.782+0000 7fce891da640 1 -- 172.21.15.38:0/1413035473 >> 172.21.15.38:0/1413035473 conn(0x7fce840b46e0 msgr2=0x7fce840d23e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:44.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:44.782+0000 7fce891da640 1 -- 172.21.15.38:0/1413035473 shutdown_connections 2024-09-21T03:42:44.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:44.782+0000 7fce891da640 1 -- 172.21.15.38:0/1413035473 wait complete. 2024-09-21T03:42:44.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:44.784+0000 7fce891da640 1 Processor -- start 2024-09-21T03:42:44.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:44.784+0000 7fce891da640 1 -- start start 2024-09-21T03:42:44.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:44.785+0000 7fce891da640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fce840ccc40 0x7fce8414c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:44.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:44.785+0000 7fce891da640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fce840cd440 0x7fce840cbe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:44.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:44.785+0000 7fce891da640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fce840d0bb0 0x7fce841f6e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:44.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:44.785+0000 7fce891da640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fce84070a10 con 0x7fce840ccc40 2024-09-21T03:42:44.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:44.785+0000 7fce891da640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fce84070890 con 0x7fce840cd440 2024-09-21T03:42:44.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:44.785+0000 7fce891da640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fce84070b90 con 0x7fce840d0bb0 2024-09-21T03:42:44.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:44.786+0000 7fce82d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fce840ccc40 0x7fce8414c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:44.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:44.786+0000 7fce82d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fce840ccc40 0x7fce8414c340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38296/0 (socket says 172.21.15.38:38296) 2024-09-21T03:42:44.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:44.786+0000 7fce82d9d640 1 -- 172.21.15.38:0/2873259102 learned_addr learned my addr 172.21.15.38:0/2873259102 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:42:44.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:44.786+0000 7fce8259c640 1 --2- 172.21.15.38:0/2873259102 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fce840cd440 0x7fce840cbe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:44.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:44.786+0000 7fce8359e640 1 --2- 172.21.15.38:0/2873259102 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fce840d0bb0 0x7fce841f6e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:44.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:44.786+0000 7fce82d9d640 1 -- 172.21.15.38:0/2873259102 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fce840d0bb0 msgr2=0x7fce841f6e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:44.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:44.786+0000 7fce82d9d640 1 --2- 172.21.15.38:0/2873259102 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fce840d0bb0 0x7fce841f6e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:44.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:44.786+0000 7fce82d9d640 1 -- 172.21.15.38:0/2873259102 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fce840cd440 msgr2=0x7fce840cbe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:44.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:44.786+0000 7fce82d9d640 1 --2- 172.21.15.38:0/2873259102 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fce840cd440 0x7fce840cbe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:44.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:44.786+0000 7fce82d9d640 1 -- 172.21.15.38:0/2873259102 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce840cc840 con 0x7fce840ccc40 2024-09-21T03:42:44.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:44.786+0000 7fce82d9d640 1 --2- 172.21.15.38:0/2873259102 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fce840ccc40 0x7fce8414c340 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7fce7c00d660 tx=0x7fce7c00db20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:44.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:44.786+0000 7fce7b7fe640 1 -- 172.21.15.38:0/2873259102 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce7c017070 con 0x7fce840ccc40 2024-09-21T03:42:44.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:44.786+0000 7fce7b7fe640 1 -- 172.21.15.38:0/2873259102 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fce7c005310 con 0x7fce840ccc40 2024-09-21T03:42:44.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:44.786+0000 7fce891da640 1 -- 172.21.15.38:0/2873259102 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce840c96e0 con 0x7fce840ccc40 2024-09-21T03:42:44.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:44.786+0000 7fce7b7fe640 1 -- 172.21.15.38:0/2873259102 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce7c009820 con 0x7fce840ccc40 2024-09-21T03:42:44.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:44.786+0000 7fce891da640 1 -- 172.21.15.38:0/2873259102 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce840c9ca0 con 0x7fce840ccc40 2024-09-21T03:42:44.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:44.787+0000 7fce891da640 1 -- 172.21.15.38:0/2873259102 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce840d0670 con 0x7fce840ccc40 2024-09-21T03:42:44.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:44.788+0000 7fce7b7fe640 1 -- 172.21.15.38:0/2873259102 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fce7c0099c0 con 0x7fce840ccc40 2024-09-21T03:42:44.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:44.789+0000 7fce7b7fe640 1 --2- 172.21.15.38:0/2873259102 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fce64078bc0 0x7fce6407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:44.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:44.789+0000 7fce7b7fe640 1 -- 172.21.15.38:0/2873259102 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 8454+0+0 (secure 0 0 0) 0x7fce7c09b7c0 con 0x7fce840ccc40 2024-09-21T03:42:44.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:44.789+0000 7fce8259c640 1 --2- 172.21.15.38:0/2873259102 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fce64078bc0 0x7fce6407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:44.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:44.790+0000 7fce8259c640 1 --2- 172.21.15.38:0/2873259102 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fce64078bc0 0x7fce6407b0d0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fce8415bc90 tx=0x7fce6c00b2e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:44.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:44.793+0000 7fce7b7fe640 1 -- 172.21.15.38:0/2873259102 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fce7c062ad0 con 0x7fce840ccc40 2024-09-21T03:42:45.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:45.016+0000 7fce891da640 1 -- 172.21.15.38:0/2873259102 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fce840d2680 con 0x7fce840ccc40 2024-09-21T03:42:45.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:45.406+0000 7fce7b7fe640 1 -- 172.21.15.38:0/2873259102 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v234) ==== 161+0+0 (secure 0 0 0) 0x7fce7c067500 con 0x7fce840ccc40 2024-09-21T03:42:45.408 INFO:teuthology.orchestra.run.smithi038.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-21T03:42:45.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:45.408+0000 7fce891da640 1 -- 172.21.15.38:0/2873259102 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fce64078bc0 msgr2=0x7fce6407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:45.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:45.408+0000 7fce891da640 1 --2- 172.21.15.38:0/2873259102 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fce64078bc0 0x7fce6407b0d0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fce8415bc90 tx=0x7fce6c00b2e0 comp rx=0 tx=0).stop 2024-09-21T03:42:45.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:45.408+0000 7fce891da640 1 -- 172.21.15.38:0/2873259102 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fce840ccc40 msgr2=0x7fce8414c340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:45.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:45.408+0000 7fce891da640 1 --2- 172.21.15.38:0/2873259102 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fce840ccc40 0x7fce8414c340 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7fce7c00d660 tx=0x7fce7c00db20 comp rx=0 tx=0).stop 2024-09-21T03:42:45.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:45.408+0000 7fce891da640 1 -- 172.21.15.38:0/2873259102 shutdown_connections 2024-09-21T03:42:45.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:45.408+0000 7fce891da640 1 --2- 172.21.15.38:0/2873259102 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fce64078bc0 0x7fce6407b0d0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:45.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:45.408+0000 7fce891da640 1 --2- 172.21.15.38:0/2873259102 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fce840d0bb0 0x7fce841f6e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:45.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:45.408+0000 7fce891da640 1 --2- 172.21.15.38:0/2873259102 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fce840cd440 0x7fce840cbe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:45.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:45.408+0000 7fce891da640 1 --2- 172.21.15.38:0/2873259102 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fce840ccc40 0x7fce8414c340 unknown :-1 s=CLOSED pgs=1838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:45.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:45.408+0000 7fce891da640 1 -- 172.21.15.38:0/2873259102 >> 172.21.15.38:0/2873259102 conn(0x7fce840b46e0 msgr2=0x7fce840c80b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:45.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:45.408+0000 7fce891da640 1 -- 172.21.15.38:0/2873259102 shutdown_connections 2024-09-21T03:42:45.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:45.408+0000 7fce891da640 1 -- 172.21.15.38:0/2873259102 wait complete. 2024-09-21T03:42:45.424 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-09-21T03:42:45.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:45.542+0000 7fce3b15e640 1 -- 172.21.15.38:0/2124331833 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fce20003a30 msgr2=0x7fce20003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:45.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:45.542+0000 7fce3b15e640 1 --2- 172.21.15.38:0/2124331833 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fce20003a30 0x7fce20003ea0 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7fce2c009b50 tx=0x7fce20005830 comp rx=0 tx=0).stop 2024-09-21T03:42:45.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:45.542+0000 7fce396fb640 1 -- 172.21.15.38:0/2124331833 reap_dead start 2024-09-21T03:42:45.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:45.542+0000 7fce3b15e640 1 -- 172.21.15.38:0/2124331833 shutdown_connections 2024-09-21T03:42:45.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:45.542+0000 7fce3b15e640 1 -- 172.21.15.38:0/2124331833 >> 172.21.15.38:0/2124331833 conn(0x7fce3409d830 msgr2=0x7fce340b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:45.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:45.542+0000 7fce3b15e640 1 -- 172.21.15.38:0/2124331833 shutdown_connections 2024-09-21T03:42:45.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:45.543+0000 7fce3b15e640 1 -- 172.21.15.38:0/2124331833 wait complete. 2024-09-21T03:42:45.547 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:45.544+0000 7fce3b15e640 1 Processor -- start 2024-09-21T03:42:45.547 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:45.544+0000 7fce3b15e640 1 -- start start 2024-09-21T03:42:45.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:45.546+0000 7fce3b15e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fce20003a30 0x7fce341594b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:45.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:45.546+0000 7fce3b15e640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fce3406f300 0x7fce340cb710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:45.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:45.546+0000 7fce3b15e640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fce3415ae50 0x7fce34157c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:45.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:45.546+0000 7fce3b15e640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fce340d0500 con 0x7fce20003a30 2024-09-21T03:42:45.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:45.546+0000 7fce3b15e640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fce340d0380 con 0x7fce3406f300 2024-09-21T03:42:45.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:45.546+0000 7fce3b15e640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fce340b6580 con 0x7fce3415ae50 2024-09-21T03:42:45.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:45.546+0000 7fce38efa640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fce20003a30 0x7fce341594b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:45.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:45.546+0000 7fce38efa640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fce20003a30 0x7fce341594b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38322/0 (socket says 172.21.15.38:38322) 2024-09-21T03:42:45.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:45.546+0000 7fce38efa640 1 -- 172.21.15.38:0/1790772157 learned_addr learned my addr 172.21.15.38:0/1790772157 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:42:45.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:45.546+0000 7fce33fff640 1 --2- 172.21.15.38:0/1790772157 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fce3406f300 0x7fce340cb710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:45.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:45.546+0000 7fce396fb640 1 --2- 172.21.15.38:0/1790772157 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fce3415ae50 0x7fce34157c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:45.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:45.546+0000 7fce38efa640 1 -- 172.21.15.38:0/1790772157 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fce3415ae50 msgr2=0x7fce34157c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:45.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:45.546+0000 7fce38efa640 1 --2- 172.21.15.38:0/1790772157 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fce3415ae50 0x7fce34157c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:45.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:45.546+0000 7fce38efa640 1 -- 172.21.15.38:0/1790772157 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fce3406f300 msgr2=0x7fce340cb710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:45.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:45.546+0000 7fce38efa640 1 --2- 172.21.15.38:0/1790772157 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fce3406f300 0x7fce340cb710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:45.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:45.546+0000 7fce38efa640 1 -- 172.21.15.38:0/1790772157 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce34159db0 con 0x7fce20003a30 2024-09-21T03:42:45.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:45.546+0000 7fce396fb640 1 --2- 172.21.15.38:0/1790772157 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fce3415ae50 0x7fce34157c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:42:45.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:45.546+0000 7fce38efa640 1 --2- 172.21.15.38:0/1790772157 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fce20003a30 0x7fce341594b0 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7fce2c009c70 tx=0x7fce2c033670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:45.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:45.547+0000 7fce317fa640 1 -- 172.21.15.38:0/1790772157 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce2c038040 con 0x7fce20003a30 2024-09-21T03:42:45.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:45.547+0000 7fce317fa640 1 -- 172.21.15.38:0/1790772157 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fce2c004020 con 0x7fce20003a30 2024-09-21T03:42:45.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:45.547+0000 7fce3b15e640 1 -- 172.21.15.38:0/1790772157 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce3415a090 con 0x7fce20003a30 2024-09-21T03:42:45.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:45.547+0000 7fce317fa640 1 -- 172.21.15.38:0/1790772157 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce2c004470 con 0x7fce20003a30 2024-09-21T03:42:45.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:45.547+0000 7fce3b15e640 1 -- 172.21.15.38:0/1790772157 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce3415a650 con 0x7fce20003a30 2024-09-21T03:42:45.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:45.548+0000 7fce3b15e640 1 -- 172.21.15.38:0/1790772157 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce340b6ae0 con 0x7fce20003a30 2024-09-21T03:42:45.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:45.548+0000 7fce317fa640 1 -- 172.21.15.38:0/1790772157 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fce2c045020 con 0x7fce20003a30 2024-09-21T03:42:45.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:45.549+0000 7fce317fa640 1 --2- 172.21.15.38:0/1790772157 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fce14078b90 0x7fce1407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:45.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:45.549+0000 7fce317fa640 1 -- 172.21.15.38:0/1790772157 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 8454+0+0 (secure 0 0 0) 0x7fce2c0bc0a0 con 0x7fce20003a30 2024-09-21T03:42:45.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:45.549+0000 7fce33fff640 1 --2- 172.21.15.38:0/1790772157 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fce14078b90 0x7fce1407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:45.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:45.550+0000 7fce33fff640 1 --2- 172.21.15.38:0/1790772157 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fce14078b90 0x7fce1407b0a0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fce3406eb10 tx=0x7fce24006c90 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:45.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:45.554+0000 7fce317fa640 1 -- 172.21.15.38:0/1790772157 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fce2c083420 con 0x7fce20003a30 2024-09-21T03:42:45.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:45.818+0000 7fce3b15e640 1 -- 172.21.15.38:0/1790772157 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7fce341600e0 con 0x7fce20003a30 2024-09-21T03:42:46.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.412+0000 7fce317fa640 1 -- 172.21.15.38:0/1790772157 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v235) ==== 101+0+0 (secure 0 0 0) 0x7fce2c087e50 con 0x7fce20003a30 2024-09-21T03:42:46.416 INFO:teuthology.orchestra.run.smithi038.stderr:repaired: restoring rank 6:0 2024-09-21T03:42:46.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.416+0000 7fce3b15e640 1 -- 172.21.15.38:0/1790772157 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fce14078b90 msgr2=0x7fce1407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:46.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.416+0000 7fce3b15e640 1 --2- 172.21.15.38:0/1790772157 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fce14078b90 0x7fce1407b0a0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fce3406eb10 tx=0x7fce24006c90 comp rx=0 tx=0).stop 2024-09-21T03:42:46.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.416+0000 7fce3b15e640 1 -- 172.21.15.38:0/1790772157 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fce20003a30 msgr2=0x7fce341594b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:46.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.416+0000 7fce3b15e640 1 --2- 172.21.15.38:0/1790772157 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fce20003a30 0x7fce341594b0 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7fce2c009c70 tx=0x7fce2c033670 comp rx=0 tx=0).stop 2024-09-21T03:42:46.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.416+0000 7fce3b15e640 1 -- 172.21.15.38:0/1790772157 shutdown_connections 2024-09-21T03:42:46.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.416+0000 7fce3b15e640 1 --2- 172.21.15.38:0/1790772157 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fce14078b90 0x7fce1407b0a0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:46.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.416+0000 7fce3b15e640 1 --2- 172.21.15.38:0/1790772157 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fce3415ae50 0x7fce34157c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:46.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.416+0000 7fce3b15e640 1 --2- 172.21.15.38:0/1790772157 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fce3406f300 0x7fce340cb710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:46.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.416+0000 7fce3b15e640 1 --2- 172.21.15.38:0/1790772157 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fce20003a30 0x7fce341594b0 unknown :-1 s=CLOSED pgs=1841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:46.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.416+0000 7fce3b15e640 1 -- 172.21.15.38:0/1790772157 >> 172.21.15.38:0/1790772157 conn(0x7fce3409d830 msgr2=0x7fce340cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:46.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.416+0000 7fce3b15e640 1 -- 172.21.15.38:0/1790772157 shutdown_connections 2024-09-21T03:42:46.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.416+0000 7fce3b15e640 1 -- 172.21.15.38:0/1790772157 wait complete. 2024-09-21T03:42:46.433 DEBUG:teuthology.orchestra.run.smithi038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-09-21T03:42:46.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.490+0000 7f981e6eaf40 1 Processor -- start 2024-09-21T03:42:46.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.490+0000 7f981e6eaf40 1 -- start start 2024-09-21T03:42:46.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.491+0000 7f981e6eaf40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e725ac7d00 0x55e725b97250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:46.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.491+0000 7f981e6eaf40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e725b82950 0x55e725b9da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:46.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.491+0000 7f981e6eaf40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e725b9d670 0x55e725b9dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:46.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.491+0000 7f981e6eaf40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55e725a15d60 con 0x55e725b82950 2024-09-21T03:42:46.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.491+0000 7f981e6eaf40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55e725b10460 con 0x55e725b9d670 2024-09-21T03:42:46.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.491+0000 7f981e6eaf40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55e725b7eb70 con 0x55e725ac7d00 2024-09-21T03:42:46.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.492+0000 7f98179a3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e725b82950 0x55e725b9da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:46.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.492+0000 7f98179a3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e725b82950 0x55e725b9da80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38332/0 (socket says 172.21.15.38:38332) 2024-09-21T03:42:46.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.492+0000 7f98179a3640 1 -- 172.21.15.38:0/3318785914 learned_addr learned my addr 172.21.15.38:0/3318785914 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:42:46.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.492+0000 7f98189a5640 1 --2- 172.21.15.38:0/3318785914 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e725b9d670 0x55e725b9dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:46.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.492+0000 7f98181a4640 1 --2- 172.21.15.38:0/3318785914 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e725ac7d00 0x55e725b97250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:46.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.492+0000 7f98179a3640 1 -- 172.21.15.38:0/3318785914 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e725ac7d00 msgr2=0x55e725b97250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:46.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.492+0000 7f98179a3640 1 --2- 172.21.15.38:0/3318785914 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e725ac7d00 0x55e725b97250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:46.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.492+0000 7f98179a3640 1 -- 172.21.15.38:0/3318785914 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e725b9d670 msgr2=0x55e725b9dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:46.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.492+0000 7f98179a3640 1 --2- 172.21.15.38:0/3318785914 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e725b9d670 0x55e725b9dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:46.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.492+0000 7f98179a3640 1 -- 172.21.15.38:0/3318785914 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e725b9af50 con 0x55e725b82950 2024-09-21T03:42:46.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.492+0000 7f98189a5640 1 --2- 172.21.15.38:0/3318785914 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e725b9d670 0x55e725b9dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:42:46.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.492+0000 7f98179a3640 1 --2- 172.21.15.38:0/3318785914 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e725b82950 0x55e725b9da80 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7f980c00b5f0 tx=0x7f980c00bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7c4a5f61fc89afb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:46.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.492+0000 7f98169a1640 1 -- 172.21.15.38:0/3318785914 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f980c018070 con 0x55e725b82950 2024-09-21T03:42:46.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.492+0000 7f98169a1640 1 -- 172.21.15.38:0/3318785914 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f980c009850 con 0x55e725b82950 2024-09-21T03:42:46.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.492+0000 7f98169a1640 1 -- 172.21.15.38:0/3318785914 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f980c009c80 con 0x55e725b82950 2024-09-21T03:42:46.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.493+0000 7f981e6eaf40 1 -- 172.21.15.38:0/3318785914 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e725b82950 msgr2=0x55e725b9da80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:46.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.493+0000 7f981e6eaf40 1 --2- 172.21.15.38:0/3318785914 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e725b82950 0x55e725b9da80 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7f980c00b5f0 tx=0x7f980c00bab0 comp rx=0 tx=0).stop 2024-09-21T03:42:46.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.493+0000 7f981e6eaf40 1 -- 172.21.15.38:0/3318785914 shutdown_connections 2024-09-21T03:42:46.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.493+0000 7f981e6eaf40 1 --2- 172.21.15.38:0/3318785914 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e725b9d670 0x55e725b9dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:46.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.493+0000 7f981e6eaf40 1 --2- 172.21.15.38:0/3318785914 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e725b82950 0x55e725b9da80 unknown :-1 s=CLOSED pgs=1843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:46.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.493+0000 7f981e6eaf40 1 --2- 172.21.15.38:0/3318785914 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e725ac7d00 0x55e725b97250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:46.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.493+0000 7f981e6eaf40 1 -- 172.21.15.38:0/3318785914 >> 172.21.15.38:0/3318785914 conn(0x55e7259e6ac0 msgr2=0x55e725b7d3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:46.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.493+0000 7f981e6eaf40 1 -- 172.21.15.38:0/3318785914 shutdown_connections 2024-09-21T03:42:46.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.493+0000 7f981e6eaf40 1 -- 172.21.15.38:0/3318785914 wait complete. 2024-09-21T03:42:46.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.494+0000 7f981e6eaf40 1 Processor -- start 2024-09-21T03:42:46.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.494+0000 7f981e6eaf40 1 -- start start 2024-09-21T03:42:46.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.495+0000 7f981e6eaf40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e725ac7d00 0x55e725b968b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:46.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.495+0000 7f981e6eaf40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e725b82950 0x55e725b7cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:46.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.495+0000 7f981e6eaf40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e725b9d670 0x55e725b7d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:46.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.495+0000 7f981e6eaf40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55e725b9b1e0 con 0x55e725b9d670 2024-09-21T03:42:46.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.495+0000 7f981e6eaf40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55e725b9a290 con 0x55e725b82950 2024-09-21T03:42:46.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.495+0000 7f981e6eaf40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55e725b9b4c0 con 0x55e725ac7d00 2024-09-21T03:42:46.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.495+0000 7f98189a5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e725b9d670 0x55e725b7d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:46.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.496+0000 7f98181a4640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e725ac7d00 0x55e725b968b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:46.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.496+0000 7f98189a5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e725b9d670 0x55e725b7d310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38334/0 (socket says 172.21.15.38:38334) 2024-09-21T03:42:46.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.496+0000 7f98189a5640 1 -- 172.21.15.38:0/667171432 learned_addr learned my addr 172.21.15.38:0/667171432 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:42:46.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.496+0000 7f98179a3640 1 --2- 172.21.15.38:0/667171432 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e725b82950 0x55e725b7cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:46.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.496+0000 7f98189a5640 1 -- 172.21.15.38:0/667171432 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e725ac7d00 msgr2=0x55e725b968b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:46.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.496+0000 7f98189a5640 1 --2- 172.21.15.38:0/667171432 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e725ac7d00 0x55e725b968b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:46.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.496+0000 7f98189a5640 1 -- 172.21.15.38:0/667171432 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e725b82950 msgr2=0x55e725b7cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:46.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.496+0000 7f98189a5640 1 --2- 172.21.15.38:0/667171432 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e725b82950 0x55e725b7cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:46.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.496+0000 7f98189a5640 1 -- 172.21.15.38:0/667171432 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e725b7df30 con 0x55e725b9d670 2024-09-21T03:42:46.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.496+0000 7f98189a5640 1 --2- 172.21.15.38:0/667171432 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e725b9d670 0x55e725b7d310 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7f98100465b0 tx=0x7f981004d870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:46.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.496+0000 7f981599f640 1 -- 172.21.15.38:0/667171432 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9810066070 con 0x55e725b9d670 2024-09-21T03:42:46.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.496+0000 7f981599f640 1 -- 172.21.15.38:0/667171432 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9810004110 con 0x55e725b9d670 2024-09-21T03:42:46.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.496+0000 7f981e6eaf40 1 -- 172.21.15.38:0/667171432 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55e725b7e210 con 0x55e725b9d670 2024-09-21T03:42:46.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.496+0000 7f981599f640 1 -- 172.21.15.38:0/667171432 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98100042c0 con 0x55e725b9d670 2024-09-21T03:42:46.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.496+0000 7f981e6eaf40 1 -- 172.21.15.38:0/667171432 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e725b7e7d0 con 0x55e725b9d670 2024-09-21T03:42:46.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.498+0000 7f981599f640 1 -- 172.21.15.38:0/667171432 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f981005e040 con 0x55e725b9d670 2024-09-21T03:42:46.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.499+0000 7f981599f640 1 --2- 172.21.15.38:0/667171432 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f98040775a0 0x7f9804079ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:46.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.499+0000 7f981599f640 1 -- 172.21.15.38:0/667171432 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 8454+0+0 (secure 0 0 0) 0x7f98100de910 con 0x55e725b9d670 2024-09-21T03:42:46.502 INFO:teuthology.orchestra.run.smithi038.stdout:Importing pool 2024-09-21T03:42:46.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.499+0000 7f98181a4640 1 --2- 172.21.15.38:0/667171432 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f98040775a0 0x7f9804079ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:46.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.499+0000 7f981e6eaf40 1 --2- 172.21.15.38:0/667171432 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x55e725b76a00 0x55e725b7a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:46.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.499+0000 7f981e6eaf40 1 -- 172.21.15.38:0/667171432 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55e725b7abe0 con 0x55e725b76a00 2024-09-21T03:42:46.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.499+0000 7f98179a3640 1 --2- 172.21.15.38:0/667171432 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x55e725b76a00 0x55e725b7a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:46.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.500+0000 7f98181a4640 1 --2- 172.21.15.38:0/667171432 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f98040775a0 0x7f9804079ab0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f98080041b0 tx=0x7f980800d040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:46.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.500+0000 7f98179a3640 1 --2- 172.21.15.38:0/667171432 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x55e725b76a00 0x55e725b7a6a0 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:46.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.502+0000 7f98179a3640 1 -- 172.21.15.38:0/667171432 <== osd.7 v2:172.21.15.195:6818/1606106342 1 ==== osd_op_reply(1 601.00000000 [create] v121'753 uv727 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f980c01b040 con 0x55e725b76a00 2024-09-21T03:42:46.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.502+0000 7f981e6eaf40 1 -- 172.21.15.38:0/667171432 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e121) -- 0x55e725c3eaf0 con 0x55e725b76a00 2024-09-21T03:42:46.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.504+0000 7f98179a3640 1 -- 172.21.15.38:0/667171432 <== osd.7 v2:172.21.15.195:6818/1606106342 2 ==== osd_op_reply(2 601.00000000 [delete] v121'754 uv754 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f980c01b040 con 0x55e725b76a00 2024-09-21T03:42:46.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.504+0000 7f981e6eaf40 1 -- 172.21.15.38:0/667171432 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55e725c3ee90 con 0x55e725b76a00 2024-09-21T03:42:46.508 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-09-21T03:42:46.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.506+0000 7f98179a3640 1 -- 172.21.15.38:0/667171432 <== osd.7 v2:172.21.15.195:6818/1606106342 3 ==== osd_op_reply(3 601.00000000 [create] v121'755 uv755 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f980c01b040 con 0x55e725b76a00 2024-09-21T03:42:46.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.506+0000 7f981e6eaf40 1 -- 172.21.15.38:0/667171432 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55e725c40240 con 0x55e725b76a00 2024-09-21T03:42:46.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.507+0000 7f98179a3640 1 -- 172.21.15.38:0/667171432 <== osd.7 v2:172.21.15.195:6818/1606106342 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v121'756 uv756 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f980c01b040 con 0x55e725b76a00 2024-09-21T03:42:46.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.507+0000 7f981e6eaf40 1 -- 172.21.15.38:0/667171432 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55e725b7af80 con 0x55e725b76a00 2024-09-21T03:42:46.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.509+0000 7f98179a3640 1 -- 172.21.15.38:0/667171432 <== osd.7 v2:172.21.15.195:6818/1606106342 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v121'757 uv757 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f980c01b040 con 0x55e725b76a00 2024-09-21T03:42:46.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.509+0000 7f981e6eaf40 1 -- 172.21.15.38:0/667171432 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55e725c3f230 con 0x55e725b76a00 2024-09-21T03:42:46.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.511+0000 7f98179a3640 1 -- 172.21.15.38:0/667171432 <== osd.7 v2:172.21.15.195:6818/1606106342 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v121'758 uv758 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f980c01b040 con 0x55e725b76a00 2024-09-21T03:42:46.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.511+0000 7f981e6eaf40 1 -- 172.21.15.38:0/667171432 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55e725c3f5d0 con 0x55e725b76a00 2024-09-21T03:42:46.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.512+0000 7f98179a3640 1 -- 172.21.15.38:0/667171432 <== osd.7 v2:172.21.15.195:6818/1606106342 7 ==== osd_op_reply(7 602.00000000 [create] v121'759 uv733 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f980c01b040 con 0x55e725b76a00 2024-09-21T03:42:46.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.513+0000 7f981e6eaf40 1 -- 172.21.15.38:0/667171432 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e121) -- 0x55e725c3f970 con 0x55e725b76a00 2024-09-21T03:42:46.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.514+0000 7f98179a3640 1 -- 172.21.15.38:0/667171432 <== osd.7 v2:172.21.15.195:6818/1606106342 8 ==== osd_op_reply(8 602.00000000 [delete] v121'760 uv760 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f980c01b040 con 0x55e725b76a00 2024-09-21T03:42:46.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.514+0000 7f981e6eaf40 1 -- 172.21.15.38:0/667171432 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55e725c3f970 con 0x55e725b76a00 2024-09-21T03:42:46.518 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-09-21T03:42:46.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.516+0000 7f98179a3640 1 -- 172.21.15.38:0/667171432 <== osd.7 v2:172.21.15.195:6818/1606106342 9 ==== osd_op_reply(9 602.00000000 [create] v121'761 uv761 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f980c01b040 con 0x55e725b76a00 2024-09-21T03:42:46.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.516+0000 7f981e6eaf40 1 -- 172.21.15.38:0/667171432 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55e725c3f970 con 0x55e725b76a00 2024-09-21T03:42:46.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.517+0000 7f98179a3640 1 -- 172.21.15.38:0/667171432 <== osd.7 v2:172.21.15.195:6818/1606106342 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v121'762 uv762 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f980c01b040 con 0x55e725b76a00 2024-09-21T03:42:46.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.518+0000 7f981e6eaf40 1 -- 172.21.15.38:0/667171432 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55e725c3f970 con 0x55e725b76a00 2024-09-21T03:42:46.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.520+0000 7f98179a3640 1 -- 172.21.15.38:0/667171432 <== osd.7 v2:172.21.15.195:6818/1606106342 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v121'763 uv763 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f980c01b040 con 0x55e725b76a00 2024-09-21T03:42:46.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.520+0000 7f981e6eaf40 1 -- 172.21.15.38:0/667171432 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55e725c3f970 con 0x55e725b76a00 2024-09-21T03:42:46.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.522+0000 7f98179a3640 1 -- 172.21.15.38:0/667171432 <== osd.7 v2:172.21.15.195:6818/1606106342 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v121'764 uv764 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f980c01b040 con 0x55e725b76a00 2024-09-21T03:42:46.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.522+0000 7f981e6eaf40 1 -- 172.21.15.38:0/667171432 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:13 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55e725c3f970 con 0x55e725b76a00 2024-09-21T03:42:46.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.524+0000 7f98179a3640 1 -- 172.21.15.38:0/667171432 <== osd.7 v2:172.21.15.195:6818/1606106342 13 ==== osd_op_reply(13 10000000000.00000000 [create] v121'765 uv740 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7f980c01b040 con 0x55e725b76a00 2024-09-21T03:42:46.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.524+0000 7f981e6eaf40 1 -- 172.21.15.38:0/667171432 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:14 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e121) -- 0x55e725c3f970 con 0x55e725b76a00 2024-09-21T03:42:46.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.524+0000 7f98179a3640 1 -- 172.21.15.38:0/667171432 <== osd.7 v2:172.21.15.195:6818/1606106342 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v121'766 uv766 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f980c01b040 con 0x55e725b76a00 2024-09-21T03:42:46.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.525+0000 7f981e6eaf40 1 -- 172.21.15.38:0/667171432 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:15 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55e725c3f970 con 0x55e725b76a00 2024-09-21T03:42:46.529 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-09-21T03:42:46.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.526+0000 7f98179a3640 1 -- 172.21.15.38:0/667171432 <== osd.7 v2:172.21.15.195:6818/1606106342 15 ==== osd_op_reply(15 10000000000.00000000 [create] v121'767 uv767 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f980c01b040 con 0x55e725b76a00 2024-09-21T03:42:46.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.527+0000 7f981e6eaf40 1 -- 172.21.15.38:0/667171432 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:16 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55e725c3f970 con 0x55e725b76a00 2024-09-21T03:42:46.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.528+0000 7f98179a3640 1 -- 172.21.15.38:0/667171432 <== osd.7 v2:172.21.15.195:6818/1606106342 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v121'768 uv768 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f980c01b040 con 0x55e725b76a00 2024-09-21T03:42:46.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.528+0000 7f981e6eaf40 1 -- 172.21.15.38:0/667171432 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:17 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55e725c3f970 con 0x55e725b76a00 2024-09-21T03:42:46.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.531+0000 7f98179a3640 1 -- 172.21.15.38:0/667171432 <== osd.7 v2:172.21.15.195:6818/1606106342 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v121'769 uv769 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f980c01b040 con 0x55e725b76a00 2024-09-21T03:42:46.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.531+0000 7f981e6eaf40 1 -- 172.21.15.38:0/667171432 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:18 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55e725c3f970 con 0x55e725b76a00 2024-09-21T03:42:46.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.531+0000 7f98179a3640 1 -- 172.21.15.38:0/667171432 <== osd.7 v2:172.21.15.195:6818/1606106342 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v121'770 uv770 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f980c01b040 con 0x55e725b76a00 2024-09-21T03:42:46.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.532+0000 7f981e6eaf40 1 -- 172.21.15.38:0/667171432 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:19 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=585b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55e725c3fde0 con 0x55e725b76a00 2024-09-21T03:42:46.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.533+0000 7f98179a3640 1 -- 172.21.15.38:0/667171432 <== osd.7 v2:172.21.15.195:6818/1606106342 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v121'771 uv771 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f980c01b040 con 0x55e725b76a00 2024-09-21T03:42:46.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.533+0000 7f981e6eaf40 1 -- 172.21.15.38:0/667171432 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:20 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55e725c3fd50 con 0x55e725b76a00 2024-09-21T03:42:46.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.536+0000 7f98179a3640 1 -- 172.21.15.38:0/667171432 <== osd.7 v2:172.21.15.195:6818/1606106342 20 ==== osd_op_reply(20 600.00000000 [create] v121'772 uv746 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f980c01b040 con 0x55e725b76a00 2024-09-21T03:42:46.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.536+0000 7f981e6eaf40 1 -- 172.21.15.38:0/667171432 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:21 13.10 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e121) -- 0x55e725c3fd50 con 0x55e725b76a00 2024-09-21T03:42:46.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.537+0000 7f98179a3640 1 -- 172.21.15.38:0/667171432 <== osd.7 v2:172.21.15.195:6818/1606106342 21 ==== osd_op_reply(21 600.00000000 [delete] v121'773 uv773 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f980c01b040 con 0x55e725b76a00 2024-09-21T03:42:46.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.537+0000 7f981e6eaf40 1 -- 172.21.15.38:0/667171432 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:22 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55e725c3fd50 con 0x55e725b76a00 2024-09-21T03:42:46.541 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-09-21T03:42:46.541 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.538+0000 7f98179a3640 1 -- 172.21.15.38:0/667171432 <== osd.7 v2:172.21.15.195:6818/1606106342 22 ==== osd_op_reply(22 600.00000000 [create] v121'774 uv774 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f980c01b040 con 0x55e725b76a00 2024-09-21T03:42:46.541 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.538+0000 7f981e6eaf40 1 -- 172.21.15.38:0/667171432 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:23 13.10 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55e725c3fc20 con 0x55e725b76a00 2024-09-21T03:42:46.543 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.541+0000 7f98179a3640 1 -- 172.21.15.38:0/667171432 <== osd.7 v2:172.21.15.195:6818/1606106342 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v121'775 uv775 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f980c01b040 con 0x55e725b76a00 2024-09-21T03:42:46.543 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.541+0000 7f981e6eaf40 1 -- 172.21.15.38:0/667171432 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:24 13.10 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55e725c3fc20 con 0x55e725b76a00 2024-09-21T03:42:46.544 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.541+0000 7f98179a3640 1 -- 172.21.15.38:0/667171432 <== osd.7 v2:172.21.15.195:6818/1606106342 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v121'776 uv776 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f980c01b040 con 0x55e725b76a00 2024-09-21T03:42:46.544 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.541+0000 7f981e6eaf40 1 -- 172.21.15.38:0/667171432 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:25 13.10 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55e725c3fc20 con 0x55e725b76a00 2024-09-21T03:42:46.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.543+0000 7f98179a3640 1 -- 172.21.15.38:0/667171432 <== osd.7 v2:172.21.15.195:6818/1606106342 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v121'777 uv777 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f980c01b040 con 0x55e725b76a00 2024-09-21T03:42:46.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.543+0000 7f981e6eaf40 1 -- 172.21.15.38:0/667171432 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:26 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55e725c3fc20 con 0x55e725b76a00 2024-09-21T03:42:46.547 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.544+0000 7f98179a3640 1 -- 172.21.15.38:0/667171432 <== osd.7 v2:172.21.15.195:6818/1606106342 26 ==== osd_op_reply(26 603.00000000 [create] v121'778 uv752 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f980c01b040 con 0x55e725b76a00 2024-09-21T03:42:46.547 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.545+0000 7f981e6eaf40 1 -- 172.21.15.38:0/667171432 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:27 13.10 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e121) -- 0x55e725c3fc20 con 0x55e725b76a00 2024-09-21T03:42:46.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.546+0000 7f98179a3640 1 -- 172.21.15.38:0/667171432 <== osd.7 v2:172.21.15.195:6818/1606106342 27 ==== osd_op_reply(27 603.00000000 [delete] v121'779 uv779 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f980c01b040 con 0x55e725b76a00 2024-09-21T03:42:46.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.546+0000 7f981e6eaf40 1 -- 172.21.15.38:0/667171432 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:28 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55e725c3fc20 con 0x55e725b76a00 2024-09-21T03:42:46.550 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-09-21T03:42:46.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.547+0000 7f98179a3640 1 -- 172.21.15.38:0/667171432 <== osd.7 v2:172.21.15.195:6818/1606106342 28 ==== osd_op_reply(28 603.00000000 [create] v121'780 uv780 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f980c01b040 con 0x55e725b76a00 2024-09-21T03:42:46.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.548+0000 7f981e6eaf40 1 -- 172.21.15.38:0/667171432 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:29 13.10 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55e725c3fc20 con 0x55e725b76a00 2024-09-21T03:42:46.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.549+0000 7f98179a3640 1 -- 172.21.15.38:0/667171432 <== osd.7 v2:172.21.15.195:6818/1606106342 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v121'781 uv781 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f980c01b040 con 0x55e725b76a00 2024-09-21T03:42:46.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.549+0000 7f981e6eaf40 1 -- 172.21.15.38:0/667171432 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:30 13.10 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55e725c3fc20 con 0x55e725b76a00 2024-09-21T03:42:46.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.551+0000 7f98179a3640 1 -- 172.21.15.38:0/667171432 <== osd.7 v2:172.21.15.195:6818/1606106342 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v121'782 uv782 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f980c01b040 con 0x55e725b76a00 2024-09-21T03:42:46.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.551+0000 7f981e6eaf40 1 -- 172.21.15.38:0/667171432 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:31 13.10 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55e725c3fc20 con 0x55e725b76a00 2024-09-21T03:42:46.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.553+0000 7f98179a3640 1 -- 172.21.15.38:0/667171432 <== osd.7 v2:172.21.15.195:6818/1606106342 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v121'783 uv783 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f980c01b040 con 0x55e725b76a00 2024-09-21T03:42:46.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.553+0000 7f981e6eaf40 1 --2- 172.21.15.38:0/667171432 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x55e725c40af0 0x55e725c42f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:46.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.553+0000 7f981e6eaf40 1 -- 172.21.15.38:0/667171432 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55e725c3fc20 con 0x55e725c40af0 2024-09-21T03:42:46.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.553+0000 7f98181a4640 1 --2- 172.21.15.38:0/667171432 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x55e725c40af0 0x55e725c42f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:46.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.553+0000 7f98181a4640 1 --2- 172.21.15.38:0/667171432 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x55e725c40af0 0x55e725c42f10 crc :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:46.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.555+0000 7f98181a4640 1 -- 172.21.15.38:0/667171432 <== osd.6 v2:172.21.15.195:6826/4055825675 1 ==== osd_op_reply(32 1.00000000.inode [create] v121'548 uv530 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f9808019140 con 0x55e725c40af0 2024-09-21T03:42:46.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.555+0000 7f981e6eaf40 1 -- 172.21.15.38:0/667171432 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e121) -- 0x55e725c43450 con 0x55e725c40af0 2024-09-21T03:42:46.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.556+0000 7f98181a4640 1 -- 172.21.15.38:0/667171432 <== osd.6 v2:172.21.15.195:6826/4055825675 2 ==== osd_op_reply(33 1.00000000.inode [delete] v121'549 uv549 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f9808019140 con 0x55e725c40af0 2024-09-21T03:42:46.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.556+0000 7f981e6eaf40 1 -- 172.21.15.38:0/667171432 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55e725c437f0 con 0x55e725c40af0 2024-09-21T03:42:46.561 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-09-21T03:42:46.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.558+0000 7f98181a4640 1 -- 172.21.15.38:0/667171432 <== osd.6 v2:172.21.15.195:6826/4055825675 3 ==== osd_op_reply(34 1.00000000.inode [create] v121'550 uv550 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f9808019140 con 0x55e725c40af0 2024-09-21T03:42:46.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.558+0000 7f981e6eaf40 1 -- 172.21.15.38:0/667171432 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55e725c44040 con 0x55e725c40af0 2024-09-21T03:42:46.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.559+0000 7f98181a4640 1 -- 172.21.15.38:0/667171432 <== osd.6 v2:172.21.15.195:6826/4055825675 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v121'551 uv551 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f9808019140 con 0x55e725c40af0 2024-09-21T03:42:46.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.559+0000 7f981e6eaf40 1 -- 172.21.15.38:0/667171432 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55e725c443e0 con 0x55e725c40af0 2024-09-21T03:42:46.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.561+0000 7f98181a4640 1 -- 172.21.15.38:0/667171432 <== osd.6 v2:172.21.15.195:6826/4055825675 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v121'552 uv552 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f9808019140 con 0x55e725c40af0 2024-09-21T03:42:46.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.561+0000 7f981e6eaf40 1 -- 172.21.15.38:0/667171432 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55e725c44780 con 0x55e725c40af0 2024-09-21T03:42:46.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.562+0000 7f98181a4640 1 -- 172.21.15.38:0/667171432 <== osd.6 v2:172.21.15.195:6826/4055825675 6 ==== osd_op_reply(37 200.00000000 [create] v121'553 uv547 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9808019140 con 0x55e725c40af0 2024-09-21T03:42:46.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.562+0000 7f981e6eaf40 1 -- 172.21.15.38:0/667171432 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e121) -- 0x55e725c44b20 con 0x55e725c40af0 2024-09-21T03:42:46.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.564+0000 7f98181a4640 1 -- 172.21.15.38:0/667171432 <== osd.6 v2:172.21.15.195:6826/4055825675 7 ==== osd_op_reply(38 200.00000000 [delete] v121'554 uv554 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9808019140 con 0x55e725c40af0 2024-09-21T03:42:46.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.564+0000 7f981e6eaf40 1 -- 172.21.15.38:0/667171432 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55e725c44b20 con 0x55e725c40af0 2024-09-21T03:42:46.567 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-09-21T03:42:46.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.565+0000 7f98181a4640 1 -- 172.21.15.38:0/667171432 <== osd.6 v2:172.21.15.195:6826/4055825675 8 ==== osd_op_reply(39 200.00000000 [create] v121'555 uv555 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9808019140 con 0x55e725c40af0 2024-09-21T03:42:46.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.565+0000 7f981e6eaf40 1 -- 172.21.15.38:0/667171432 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55e725c44b20 con 0x55e725c40af0 2024-09-21T03:42:46.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.567+0000 7f98181a4640 1 -- 172.21.15.38:0/667171432 <== osd.6 v2:172.21.15.195:6826/4055825675 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v121'556 uv556 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9808019140 con 0x55e725c40af0 2024-09-21T03:42:46.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.567+0000 7f981e6eaf40 1 -- 172.21.15.38:0/667171432 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55e725c44b20 con 0x55e725c40af0 2024-09-21T03:42:46.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.568+0000 7f98181a4640 1 -- 172.21.15.38:0/667171432 <== osd.6 v2:172.21.15.195:6826/4055825675 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v121'557 uv557 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9808019140 con 0x55e725c40af0 2024-09-21T03:42:46.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.568+0000 7f981e6eaf40 1 -- 172.21.15.38:0/667171432 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:42 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55e725c44b20 con 0x55e725c40af0 2024-09-21T03:42:46.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.569+0000 7f98181a4640 1 -- 172.21.15.38:0/667171432 <== osd.6 v2:172.21.15.195:6826/4055825675 11 ==== osd_op_reply(42 200.00000001 [create] v121'558 uv540 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9808019140 con 0x55e725c40af0 2024-09-21T03:42:46.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.569+0000 7f981e6eaf40 1 -- 172.21.15.38:0/667171432 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:43 13.14 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e121) -- 0x55e725c44b20 con 0x55e725c40af0 2024-09-21T03:42:46.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.571+0000 7f98181a4640 1 -- 172.21.15.38:0/667171432 <== osd.6 v2:172.21.15.195:6826/4055825675 12 ==== osd_op_reply(43 200.00000001 [delete] v121'559 uv559 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9808019140 con 0x55e725c40af0 2024-09-21T03:42:46.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.571+0000 7f981e6eaf40 1 -- 172.21.15.38:0/667171432 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:44 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55e725c44b20 con 0x55e725c40af0 2024-09-21T03:42:46.575 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-09-21T03:42:46.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.572+0000 7f98181a4640 1 -- 172.21.15.38:0/667171432 <== osd.6 v2:172.21.15.195:6826/4055825675 13 ==== osd_op_reply(44 200.00000001 [create] v121'560 uv560 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9808019140 con 0x55e725c40af0 2024-09-21T03:42:46.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.573+0000 7f981e6eaf40 1 -- 172.21.15.38:0/667171432 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:45 13.14 13:2e2fa760:::200.00000001:head [write 0~17490 in=17490b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55e725c48fa0 con 0x55e725c40af0 2024-09-21T03:42:46.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.574+0000 7f98181a4640 1 -- 172.21.15.38:0/667171432 <== osd.6 v2:172.21.15.195:6826/4055825675 14 ==== osd_op_reply(45 200.00000001 [write 0~17490] v121'561 uv561 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9808019140 con 0x55e725c40af0 2024-09-21T03:42:46.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.574+0000 7f981e6eaf40 1 -- 172.21.15.38:0/667171432 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:46 13.14 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55e725c48fa0 con 0x55e725c40af0 2024-09-21T03:42:46.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.576+0000 7f98181a4640 1 -- 172.21.15.38:0/667171432 <== osd.6 v2:172.21.15.195:6826/4055825675 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v121'562 uv562 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9808019140 con 0x55e725c40af0 2024-09-21T03:42:46.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.576+0000 7f981e6eaf40 1 -- 172.21.15.38:0/667171432 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:47 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55e725c44b20 con 0x55e725c40af0 2024-09-21T03:42:46.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.577+0000 7f98181a4640 1 -- 172.21.15.38:0/667171432 <== osd.6 v2:172.21.15.195:6826/4055825675 16 ==== osd_op_reply(47 606.00000000 [create] v121'563 uv546 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9808019140 con 0x55e725c40af0 2024-09-21T03:42:46.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.577+0000 7f981e6eaf40 1 -- 172.21.15.38:0/667171432 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:48 13.14 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e121) -- 0x55e725c44b20 con 0x55e725c40af0 2024-09-21T03:42:46.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.578+0000 7f98181a4640 1 -- 172.21.15.38:0/667171432 <== osd.6 v2:172.21.15.195:6826/4055825675 17 ==== osd_op_reply(48 606.00000000 [delete] v121'564 uv564 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9808019140 con 0x55e725c40af0 2024-09-21T03:42:46.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.578+0000 7f981e6eaf40 1 -- 172.21.15.38:0/667171432 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:49 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55e725c44b20 con 0x55e725c40af0 2024-09-21T03:42:46.582 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-09-21T03:42:46.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.580+0000 7f98181a4640 1 -- 172.21.15.38:0/667171432 <== osd.6 v2:172.21.15.195:6826/4055825675 18 ==== osd_op_reply(49 606.00000000 [create] v121'565 uv565 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9808019140 con 0x55e725c40af0 2024-09-21T03:42:46.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.580+0000 7f981e6eaf40 1 -- 172.21.15.38:0/667171432 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:50 13.14 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55e725c45b30 con 0x55e725c40af0 2024-09-21T03:42:46.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.581+0000 7f98181a4640 1 -- 172.21.15.38:0/667171432 <== osd.6 v2:172.21.15.195:6826/4055825675 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v121'566 uv566 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9808019140 con 0x55e725c40af0 2024-09-21T03:42:46.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.581+0000 7f981e6eaf40 1 -- 172.21.15.38:0/667171432 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:51 13.14 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55e725c45b30 con 0x55e725c40af0 2024-09-21T03:42:46.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.582+0000 7f98181a4640 1 -- 172.21.15.38:0/667171432 <== osd.6 v2:172.21.15.195:6826/4055825675 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v121'567 uv567 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9808019140 con 0x55e725c40af0 2024-09-21T03:42:46.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.582+0000 7f981e6eaf40 1 -- 172.21.15.38:0/667171432 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:52 13.14 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55e725c44b20 con 0x55e725c40af0 2024-09-21T03:42:46.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.584+0000 7f98181a4640 1 -- 172.21.15.38:0/667171432 <== osd.6 v2:172.21.15.195:6826/4055825675 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v121'568 uv568 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9808019140 con 0x55e725c40af0 2024-09-21T03:42:46.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.584+0000 7f981e6eaf40 1 --2- 172.21.15.38:0/667171432 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x55e725c451a0 0x55e725c475c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:46.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.584+0000 7f981e6eaf40 1 -- 172.21.15.38:0/667171432 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:53 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55e725c47b40 con 0x55e725c451a0 2024-09-21T03:42:46.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.584+0000 7f98179a3640 1 --2- 172.21.15.38:0/667171432 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x55e725c451a0 0x55e725c475c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:46.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.584+0000 7f98179a3640 1 --2- 172.21.15.38:0/667171432 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x55e725c451a0 0x55e725c475c0 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:46.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.586+0000 7f98179a3640 1 -- 172.21.15.38:0/667171432 <== osd.0 v2:172.21.15.38:6800/2381639433 1 ==== osd_op_reply(53 607.00000000 [create] v121'147 uv146 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f980c01b040 con 0x55e725c451a0 2024-09-21T03:42:46.589 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.586+0000 7f981e6eaf40 1 -- 172.21.15.38:0/667171432 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:54 13.1c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e121) -- 0x55e725c47f10 con 0x55e725c451a0 2024-09-21T03:42:46.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.587+0000 7f98179a3640 1 -- 172.21.15.38:0/667171432 <== osd.0 v2:172.21.15.38:6800/2381639433 2 ==== osd_op_reply(54 607.00000000 [delete] v121'148 uv148 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f980c01b040 con 0x55e725c451a0 2024-09-21T03:42:46.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.587+0000 7f981e6eaf40 1 -- 172.21.15.38:0/667171432 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:55 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55e725c47f10 con 0x55e725c451a0 2024-09-21T03:42:46.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.588+0000 7f98179a3640 1 -- 172.21.15.38:0/667171432 <== osd.0 v2:172.21.15.38:6800/2381639433 3 ==== osd_op_reply(55 607.00000000 [create] v121'149 uv149 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f980c01b040 con 0x55e725c451a0 2024-09-21T03:42:46.591 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-09-21T03:42:46.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.588+0000 7f981e6eaf40 1 -- 172.21.15.38:0/667171432 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:56 13.1c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55e725c48f20 con 0x55e725c451a0 2024-09-21T03:42:46.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.589+0000 7f98179a3640 1 -- 172.21.15.38:0/667171432 <== osd.0 v2:172.21.15.38:6800/2381639433 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v121'150 uv150 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f980c01b040 con 0x55e725c451a0 2024-09-21T03:42:46.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.589+0000 7f981e6eaf40 1 -- 172.21.15.38:0/667171432 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:57 13.1c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55e725c48f20 con 0x55e725c451a0 2024-09-21T03:42:46.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.591+0000 7f98179a3640 1 -- 172.21.15.38:0/667171432 <== osd.0 v2:172.21.15.38:6800/2381639433 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v121'151 uv151 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f980c01b040 con 0x55e725c451a0 2024-09-21T03:42:46.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.591+0000 7f981e6eaf40 1 -- 172.21.15.38:0/667171432 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:58 13.1c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55e725c47f10 con 0x55e725c451a0 2024-09-21T03:42:46.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.592+0000 7f98179a3640 1 -- 172.21.15.38:0/667171432 <== osd.0 v2:172.21.15.38:6800/2381639433 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v121'152 uv152 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f980c01b040 con 0x55e725c451a0 2024-09-21T03:42:46.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.592+0000 7f981e6eaf40 1 --2- 172.21.15.38:0/667171432 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x55e725c48590 0x55e725c48a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:46.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.592+0000 7f981e6eaf40 1 -- 172.21.15.38:0/667171432 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:59 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55e725c4b370 con 0x55e725c48590 2024-09-21T03:42:46.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.592+0000 7f98189a5640 1 --2- 172.21.15.38:0/667171432 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x55e725c48590 0x55e725c48a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:46.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.592+0000 7f98189a5640 1 --2- 172.21.15.38:0/667171432 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x55e725c48590 0x55e725c48a00 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:46.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.593+0000 7f98189a5640 1 -- 172.21.15.38:0/667171432 <== osd.3 v2:172.21.15.38:6808/1286810294 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v121'106 uv105 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f9810068070 con 0x55e725c48590 2024-09-21T03:42:46.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.593+0000 7f981e6eaf40 1 -- 172.21.15.38:0/667171432 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:60 13.6 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e121) -- 0x55e725c4b710 con 0x55e725c48590 2024-09-21T03:42:46.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.595+0000 7f98189a5640 1 -- 172.21.15.38:0/667171432 <== osd.3 v2:172.21.15.38:6808/1286810294 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v121'107 uv107 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f9810068070 con 0x55e725c48590 2024-09-21T03:42:46.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.595+0000 7f981e6eaf40 1 -- 172.21.15.38:0/667171432 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:61 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55e725c4bb30 con 0x55e725c48590 2024-09-21T03:42:46.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.596+0000 7f98189a5640 1 -- 172.21.15.38:0/667171432 <== osd.3 v2:172.21.15.38:6808/1286810294 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v121'108 uv108 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f9810068070 con 0x55e725c48590 2024-09-21T03:42:46.599 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-09-21T03:42:46.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.596+0000 7f981e6eaf40 1 -- 172.21.15.38:0/667171432 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:62 13.6 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55e725c4c3d0 con 0x55e725c48590 2024-09-21T03:42:46.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.597+0000 7f98189a5640 1 -- 172.21.15.38:0/667171432 <== osd.3 v2:172.21.15.38:6808/1286810294 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v121'109 uv109 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f9810068070 con 0x55e725c48590 2024-09-21T03:42:46.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.597+0000 7f981e6eaf40 1 -- 172.21.15.38:0/667171432 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55e725c4c770 con 0x55e725c451a0 2024-09-21T03:42:46.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.598+0000 7f98179a3640 1 -- 172.21.15.38:0/667171432 <== osd.0 v2:172.21.15.38:6800/2381639433 7 ==== osd_op_reply(63 608.00000000 [create] v121'147 uv146 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f980c01b040 con 0x55e725c451a0 2024-09-21T03:42:46.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.598+0000 7f981e6eaf40 1 -- 172.21.15.38:0/667171432 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e121) -- 0x55e725c4c770 con 0x55e725c451a0 2024-09-21T03:42:46.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.599+0000 7f98179a3640 1 -- 172.21.15.38:0/667171432 <== osd.0 v2:172.21.15.38:6800/2381639433 8 ==== osd_op_reply(64 608.00000000 [delete] v121'148 uv148 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f980c01b040 con 0x55e725c451a0 2024-09-21T03:42:46.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.599+0000 7f981e6eaf40 1 -- 172.21.15.38:0/667171432 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55e725c4c770 con 0x55e725c451a0 2024-09-21T03:42:46.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.600+0000 7f98179a3640 1 -- 172.21.15.38:0/667171432 <== osd.0 v2:172.21.15.38:6800/2381639433 9 ==== osd_op_reply(65 608.00000000 [create] v121'149 uv149 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f980c01b040 con 0x55e725c451a0 2024-09-21T03:42:46.603 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-09-21T03:42:46.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.600+0000 7f981e6eaf40 1 -- 172.21.15.38:0/667171432 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55e725c4d780 con 0x55e725c451a0 2024-09-21T03:42:46.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.601+0000 7f98179a3640 1 -- 172.21.15.38:0/667171432 <== osd.0 v2:172.21.15.38:6800/2381639433 10 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v121'150 uv150 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f980c01b040 con 0x55e725c451a0 2024-09-21T03:42:46.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.601+0000 7f981e6eaf40 1 -- 172.21.15.38:0/667171432 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55e725c4d780 con 0x55e725c451a0 2024-09-21T03:42:46.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.603+0000 7f98179a3640 1 -- 172.21.15.38:0/667171432 <== osd.0 v2:172.21.15.38:6800/2381639433 11 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v121'151 uv151 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f980c01b040 con 0x55e725c451a0 2024-09-21T03:42:46.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.603+0000 7f981e6eaf40 1 -- 172.21.15.38:0/667171432 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55e725c4c770 con 0x55e725c451a0 2024-09-21T03:42:46.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.605+0000 7f98179a3640 1 -- 172.21.15.38:0/667171432 <== osd.0 v2:172.21.15.38:6800/2381639433 12 ==== osd_op_reply(68 608.00000000 [omap-set-header] v121'152 uv152 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f980c01b040 con 0x55e725c451a0 2024-09-21T03:42:46.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.605+0000 7f981e6eaf40 1 -- 172.21.15.38:0/667171432 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55e725c4c770 con 0x55e725c48590 2024-09-21T03:42:46.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.606+0000 7f98189a5640 1 -- 172.21.15.38:0/667171432 <== osd.3 v2:172.21.15.38:6808/1286810294 5 ==== osd_op_reply(69 604.00000000 [create] v121'271 uv265 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9810068070 con 0x55e725c48590 2024-09-21T03:42:46.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.607+0000 7f981e6eaf40 1 -- 172.21.15.38:0/667171432 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e121) -- 0x55e725c4cb10 con 0x55e725c48590 2024-09-21T03:42:46.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.608+0000 7f98189a5640 1 -- 172.21.15.38:0/667171432 <== osd.3 v2:172.21.15.38:6808/1286810294 6 ==== osd_op_reply(70 604.00000000 [delete] v121'272 uv272 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9810068070 con 0x55e725c48590 2024-09-21T03:42:46.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.608+0000 7f981e6eaf40 1 -- 172.21.15.38:0/667171432 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55e725c4ceb0 con 0x55e725c48590 2024-09-21T03:42:46.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.609+0000 7f98189a5640 1 -- 172.21.15.38:0/667171432 <== osd.3 v2:172.21.15.38:6808/1286810294 7 ==== osd_op_reply(71 604.00000000 [create] v121'273 uv273 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9810068070 con 0x55e725c48590 2024-09-21T03:42:46.612 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-09-21T03:42:46.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.609+0000 7f981e6eaf40 1 -- 172.21.15.38:0/667171432 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55e725c4e260 con 0x55e725c48590 2024-09-21T03:42:46.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.610+0000 7f98189a5640 1 -- 172.21.15.38:0/667171432 <== osd.3 v2:172.21.15.38:6808/1286810294 8 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v121'274 uv274 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9810068070 con 0x55e725c48590 2024-09-21T03:42:46.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.610+0000 7f981e6eaf40 1 -- 172.21.15.38:0/667171432 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55e725c4e260 con 0x55e725c48590 2024-09-21T03:42:46.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.611+0000 7f98189a5640 1 -- 172.21.15.38:0/667171432 <== osd.3 v2:172.21.15.38:6808/1286810294 9 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v121'275 uv275 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9810068070 con 0x55e725c48590 2024-09-21T03:42:46.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.611+0000 7f981e6eaf40 1 -- 172.21.15.38:0/667171432 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55e725c4d250 con 0x55e725c48590 2024-09-21T03:42:46.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.612+0000 7f98189a5640 1 -- 172.21.15.38:0/667171432 <== osd.3 v2:172.21.15.38:6808/1286810294 10 ==== osd_op_reply(74 604.00000000 [omap-set-header] v121'276 uv276 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9810068070 con 0x55e725c48590 2024-09-21T03:42:46.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.612+0000 7f981e6eaf40 1 -- 172.21.15.38:0/667171432 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55e725c4d250 con 0x55e725c48590 2024-09-21T03:42:46.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.614+0000 7f98189a5640 1 -- 172.21.15.38:0/667171432 <== osd.3 v2:172.21.15.38:6808/1286810294 11 ==== osd_op_reply(75 500.00000000 [create] v121'277 uv270 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9810068070 con 0x55e725c48590 2024-09-21T03:42:46.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.614+0000 7f981e6eaf40 1 -- 172.21.15.38:0/667171432 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e121) -- 0x55e725c4d250 con 0x55e725c48590 2024-09-21T03:42:46.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.615+0000 7f98189a5640 1 -- 172.21.15.38:0/667171432 <== osd.3 v2:172.21.15.38:6808/1286810294 12 ==== osd_op_reply(76 500.00000000 [delete] v121'278 uv278 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9810068070 con 0x55e725c48590 2024-09-21T03:42:46.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.615+0000 7f981e6eaf40 1 -- 172.21.15.38:0/667171432 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55e725c4d250 con 0x55e725c48590 2024-09-21T03:42:46.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.616+0000 7f98189a5640 1 -- 172.21.15.38:0/667171432 <== osd.3 v2:172.21.15.38:6808/1286810294 13 ==== osd_op_reply(77 500.00000000 [create] v121'279 uv279 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9810068070 con 0x55e725c48590 2024-09-21T03:42:46.619 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-09-21T03:42:46.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.616+0000 7f981e6eaf40 1 -- 172.21.15.38:0/667171432 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55e725c4d250 con 0x55e725c48590 2024-09-21T03:42:46.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.617+0000 7f98189a5640 1 -- 172.21.15.38:0/667171432 <== osd.3 v2:172.21.15.38:6808/1286810294 14 ==== osd_op_reply(78 500.00000000 [write 0~90] v121'280 uv280 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9810068070 con 0x55e725c48590 2024-09-21T03:42:46.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.617+0000 7f981e6eaf40 1 -- 172.21.15.38:0/667171432 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55e725c4d250 con 0x55e725c48590 2024-09-21T03:42:46.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.619+0000 7f98189a5640 1 -- 172.21.15.38:0/667171432 <== osd.3 v2:172.21.15.38:6808/1286810294 15 ==== osd_op_reply(79 500.00000000 [omap-set-header] v121'281 uv281 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9810068070 con 0x55e725c48590 2024-09-21T03:42:46.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.619+0000 7f981e6eaf40 1 -- 172.21.15.38:0/667171432 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:80 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55e725c4d250 con 0x55e725c451a0 2024-09-21T03:42:46.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.620+0000 7f98179a3640 1 -- 172.21.15.38:0/667171432 <== osd.0 v2:172.21.15.38:6800/2381639433 13 ==== osd_op_reply(80 mds_snaptable [create] v121'122 uv121 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f980c01b040 con 0x55e725c451a0 2024-09-21T03:42:46.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.620+0000 7f981e6eaf40 1 -- 172.21.15.38:0/667171432 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:81 13.d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e121) -- 0x55e725c4d250 con 0x55e725c451a0 2024-09-21T03:42:46.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.621+0000 7f98179a3640 1 -- 172.21.15.38:0/667171432 <== osd.0 v2:172.21.15.38:6800/2381639433 14 ==== osd_op_reply(81 mds_snaptable [delete] v121'123 uv123 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f980c01b040 con 0x55e725c451a0 2024-09-21T03:42:46.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.621+0000 7f981e6eaf40 1 -- 172.21.15.38:0/667171432 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:82 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55e725c4d250 con 0x55e725c451a0 2024-09-21T03:42:46.625 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-09-21T03:42:46.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.622+0000 7f98179a3640 1 -- 172.21.15.38:0/667171432 <== osd.0 v2:172.21.15.38:6800/2381639433 15 ==== osd_op_reply(82 mds_snaptable [create] v121'124 uv124 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f980c01b040 con 0x55e725c451a0 2024-09-21T03:42:46.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.622+0000 7f981e6eaf40 1 -- 172.21.15.38:0/667171432 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:83 13.d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55e725c4d250 con 0x55e725c451a0 2024-09-21T03:42:46.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.623+0000 7f98179a3640 1 -- 172.21.15.38:0/667171432 <== osd.0 v2:172.21.15.38:6800/2381639433 16 ==== osd_op_reply(83 mds_snaptable [write 0~70] v121'125 uv125 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f980c01b040 con 0x55e725c451a0 2024-09-21T03:42:46.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.623+0000 7f981e6eaf40 1 -- 172.21.15.38:0/667171432 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:84 13.d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55e725c4d250 con 0x55e725c451a0 2024-09-21T03:42:46.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.624+0000 7f98179a3640 1 -- 172.21.15.38:0/667171432 <== osd.0 v2:172.21.15.38:6800/2381639433 17 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v121'126 uv126 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f980c01b040 con 0x55e725c451a0 2024-09-21T03:42:46.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.624+0000 7f981e6eaf40 1 --2- 172.21.15.38:0/667171432 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x55e725c4d8d0 0x55e725c4fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:46.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.624+0000 7f981e6eaf40 1 -- 172.21.15.38:0/667171432 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55e725c50380 con 0x55e725c4d8d0 2024-09-21T03:42:46.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.624+0000 7f98181a4640 1 --2- 172.21.15.38:0/667171432 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x55e725c4d8d0 0x55e725c4fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:46.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.625+0000 7f98181a4640 1 --2- 172.21.15.38:0/667171432 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x55e725c4d8d0 0x55e725c4fcf0 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:46.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.626+0000 7f98181a4640 1 -- 172.21.15.38:0/667171432 <== osd.1 v2:172.21.15.38:6816/1491818732 1 ==== osd_op_reply(85 605.00000000 [create] v121'147 uv146 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9808019140 con 0x55e725c4d8d0 2024-09-21T03:42:46.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.626+0000 7f981e6eaf40 1 -- 172.21.15.38:0/667171432 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e121) -- 0x55e725c50750 con 0x55e725c4d8d0 2024-09-21T03:42:46.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.628+0000 7f98181a4640 1 -- 172.21.15.38:0/667171432 <== osd.1 v2:172.21.15.38:6816/1491818732 2 ==== osd_op_reply(86 605.00000000 [delete] v121'148 uv148 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9808019140 con 0x55e725c4d8d0 2024-09-21T03:42:46.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.628+0000 7f981e6eaf40 1 -- 172.21.15.38:0/667171432 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55e725c50750 con 0x55e725c4d8d0 2024-09-21T03:42:46.632 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-09-21T03:42:46.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.629+0000 7f98181a4640 1 -- 172.21.15.38:0/667171432 <== osd.1 v2:172.21.15.38:6816/1491818732 3 ==== osd_op_reply(87 605.00000000 [create] v121'149 uv149 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9808019140 con 0x55e725c4d8d0 2024-09-21T03:42:46.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.629+0000 7f981e6eaf40 1 -- 172.21.15.38:0/667171432 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55e725c51760 con 0x55e725c4d8d0 2024-09-21T03:42:46.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.630+0000 7f98181a4640 1 -- 172.21.15.38:0/667171432 <== osd.1 v2:172.21.15.38:6816/1491818732 4 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v121'150 uv150 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9808019140 con 0x55e725c4d8d0 2024-09-21T03:42:46.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.631+0000 7f981e6eaf40 1 -- 172.21.15.38:0/667171432 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55e725c51760 con 0x55e725c4d8d0 2024-09-21T03:42:46.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.632+0000 7f98181a4640 1 -- 172.21.15.38:0/667171432 <== osd.1 v2:172.21.15.38:6816/1491818732 5 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v121'151 uv151 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9808019140 con 0x55e725c4d8d0 2024-09-21T03:42:46.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.632+0000 7f981e6eaf40 1 -- 172.21.15.38:0/667171432 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55e725c50750 con 0x55e725c4d8d0 2024-09-21T03:42:46.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.633+0000 7f98181a4640 1 -- 172.21.15.38:0/667171432 <== osd.1 v2:172.21.15.38:6816/1491818732 6 ==== osd_op_reply(90 605.00000000 [omap-set-header] v121'152 uv152 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9808019140 con 0x55e725c4d8d0 2024-09-21T03:42:46.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.633+0000 7f981e6eaf40 1 -- 172.21.15.38:0/667171432 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55e725c50750 con 0x55e725c4d8d0 2024-09-21T03:42:46.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.634+0000 7f98181a4640 1 -- 172.21.15.38:0/667171432 <== osd.1 v2:172.21.15.38:6816/1491818732 7 ==== osd_op_reply(91 mds0_inotable [create] v121'243 uv237 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f9808019140 con 0x55e725c4d8d0 2024-09-21T03:42:46.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.634+0000 7f981e6eaf40 1 -- 172.21.15.38:0/667171432 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e121) -- 0x55e725c50750 con 0x55e725c4d8d0 2024-09-21T03:42:46.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.635+0000 7f98181a4640 1 -- 172.21.15.38:0/667171432 <== osd.1 v2:172.21.15.38:6816/1491818732 8 ==== osd_op_reply(92 mds0_inotable [delete] v121'244 uv244 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f9808019140 con 0x55e725c4d8d0 2024-09-21T03:42:46.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.635+0000 7f981e6eaf40 1 -- 172.21.15.38:0/667171432 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55e725c50750 con 0x55e725c4d8d0 2024-09-21T03:42:46.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.636+0000 7f98181a4640 1 -- 172.21.15.38:0/667171432 <== osd.1 v2:172.21.15.38:6816/1491818732 9 ==== osd_op_reply(93 mds0_inotable [create] v121'245 uv245 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f9808019140 con 0x55e725c4d8d0 2024-09-21T03:42:46.639 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-09-21T03:42:46.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.636+0000 7f981e6eaf40 1 -- 172.21.15.38:0/667171432 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:94 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55e725c4bfc0 con 0x55e725c4d8d0 2024-09-21T03:42:46.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.638+0000 7f98181a4640 1 -- 172.21.15.38:0/667171432 <== osd.1 v2:172.21.15.38:6816/1491818732 10 ==== osd_op_reply(94 mds0_inotable [write 0~34] v121'246 uv246 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f9808019140 con 0x55e725c4d8d0 2024-09-21T03:42:46.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.638+0000 7f981e6eaf40 1 -- 172.21.15.38:0/667171432 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:95 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55e725c4bfc0 con 0x55e725c4d8d0 2024-09-21T03:42:46.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.639+0000 7f98181a4640 1 -- 172.21.15.38:0/667171432 <== osd.1 v2:172.21.15.38:6816/1491818732 11 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v121'247 uv247 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f9808019140 con 0x55e725c4d8d0 2024-09-21T03:42:46.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.639+0000 7f981e6eaf40 1 -- 172.21.15.38:0/667171432 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:96 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55e725c4bfc0 con 0x55e725c4d8d0 2024-09-21T03:42:46.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.640+0000 7f98181a4640 1 -- 172.21.15.38:0/667171432 <== osd.1 v2:172.21.15.38:6816/1491818732 12 ==== osd_op_reply(96 100.00000000 [create] v121'248 uv242 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9808019140 con 0x55e725c4d8d0 2024-09-21T03:42:46.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.640+0000 7f981e6eaf40 1 -- 172.21.15.38:0/667171432 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:97 13.13 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e121) -- 0x55e725c4bfc0 con 0x55e725c4d8d0 2024-09-21T03:42:46.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.641+0000 7f98181a4640 1 -- 172.21.15.38:0/667171432 <== osd.1 v2:172.21.15.38:6816/1491818732 13 ==== osd_op_reply(97 100.00000000 [delete] v121'249 uv249 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9808019140 con 0x55e725c4d8d0 2024-09-21T03:42:46.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.641+0000 7f981e6eaf40 1 -- 172.21.15.38:0/667171432 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:98 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55e725c4bfc0 con 0x55e725c4d8d0 2024-09-21T03:42:46.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.642+0000 7f98181a4640 1 -- 172.21.15.38:0/667171432 <== osd.1 v2:172.21.15.38:6816/1491818732 14 ==== osd_op_reply(98 100.00000000 [create] v121'250 uv250 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9808019140 con 0x55e725c4d8d0 2024-09-21T03:42:46.645 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-09-21T03:42:46.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.642+0000 7f981e6eaf40 1 -- 172.21.15.38:0/667171432 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:99 13.13 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55e725c4bfc0 con 0x55e725c4d8d0 2024-09-21T03:42:46.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.643+0000 7f98181a4640 1 -- 172.21.15.38:0/667171432 <== osd.1 v2:172.21.15.38:6816/1491818732 15 ==== osd_op_reply(99 100.00000000 [omap-set-header] v121'251 uv251 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9808019140 con 0x55e725c4d8d0 2024-09-21T03:42:46.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.643+0000 7f981e6eaf40 1 -- 172.21.15.38:0/667171432 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:100 13.13 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55e725c50cb0 con 0x55e725c4d8d0 2024-09-21T03:42:46.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.644+0000 7f98181a4640 1 -- 172.21.15.38:0/667171432 <== osd.1 v2:172.21.15.38:6816/1491818732 16 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v121'252 uv252 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9808019140 con 0x55e725c4d8d0 2024-09-21T03:42:46.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.644+0000 7f981e6eaf40 1 -- 172.21.15.38:0/667171432 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55e725c50c70 con 0x55e725c48590 2024-09-21T03:42:46.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.645+0000 7f98189a5640 1 -- 172.21.15.38:0/667171432 <== osd.3 v2:172.21.15.38:6808/1286810294 16 ==== osd_op_reply(101 mds0_sessionmap [create] v121'107 uv106 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7f9810068070 con 0x55e725c48590 2024-09-21T03:42:46.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.645+0000 7f981e6eaf40 1 -- 172.21.15.38:0/667171432 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e121) -- 0x55e725c50c70 con 0x55e725c48590 2024-09-21T03:42:46.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.646+0000 7f98189a5640 1 -- 172.21.15.38:0/667171432 <== osd.3 v2:172.21.15.38:6808/1286810294 17 ==== osd_op_reply(102 mds0_sessionmap [delete] v121'108 uv108 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f9810068070 con 0x55e725c48590 2024-09-21T03:42:46.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.646+0000 7f981e6eaf40 1 -- 172.21.15.38:0/667171432 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:103 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55e725c50c70 con 0x55e725c48590 2024-09-21T03:42:46.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.647+0000 7f98189a5640 1 -- 172.21.15.38:0/667171432 <== osd.3 v2:172.21.15.38:6808/1286810294 18 ==== osd_op_reply(103 mds0_sessionmap [create] v121'109 uv109 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f9810068070 con 0x55e725c48590 2024-09-21T03:42:46.650 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-09-21T03:42:46.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.647+0000 7f981e6eaf40 1 -- 172.21.15.38:0/667171432 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:104 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55e725c50c70 con 0x55e725c48590 2024-09-21T03:42:46.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.648+0000 7f98189a5640 1 -- 172.21.15.38:0/667171432 <== osd.3 v2:172.21.15.38:6808/1286810294 19 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v121'110 uv110 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f9810068070 con 0x55e725c48590 2024-09-21T03:42:46.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.648+0000 7f981e6eaf40 1 -- 172.21.15.38:0/667171432 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55e725c50c30 con 0x55e725c48590 2024-09-21T03:42:46.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.649+0000 7f98189a5640 1 -- 172.21.15.38:0/667171432 <== osd.3 v2:172.21.15.38:6808/1286810294 20 ==== osd_op_reply(105 609.00000000 [create] v121'147 uv146 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9810068070 con 0x55e725c48590 2024-09-21T03:42:46.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.649+0000 7f981e6eaf40 1 -- 172.21.15.38:0/667171432 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e121) -- 0x55e725c50c30 con 0x55e725c48590 2024-09-21T03:42:46.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.650+0000 7f98189a5640 1 -- 172.21.15.38:0/667171432 <== osd.3 v2:172.21.15.38:6808/1286810294 21 ==== osd_op_reply(106 609.00000000 [delete] v121'148 uv148 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9810068070 con 0x55e725c48590 2024-09-21T03:42:46.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.650+0000 7f981e6eaf40 1 -- 172.21.15.38:0/667171432 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55e725c50c30 con 0x55e725c48590 2024-09-21T03:42:46.654 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-09-21T03:42:46.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.651+0000 7f98189a5640 1 -- 172.21.15.38:0/667171432 <== osd.3 v2:172.21.15.38:6808/1286810294 22 ==== osd_op_reply(107 609.00000000 [create] v121'149 uv149 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9810068070 con 0x55e725c48590 2024-09-21T03:42:46.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.651+0000 7f981e6eaf40 1 -- 172.21.15.38:0/667171432 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55e725c50c30 con 0x55e725c48590 2024-09-21T03:42:46.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.653+0000 7f98189a5640 1 -- 172.21.15.38:0/667171432 <== osd.3 v2:172.21.15.38:6808/1286810294 23 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v121'150 uv150 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9810068070 con 0x55e725c48590 2024-09-21T03:42:46.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.653+0000 7f981e6eaf40 1 -- 172.21.15.38:0/667171432 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:109 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55e725c50c00 con 0x55e725c48590 2024-09-21T03:42:46.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.654+0000 7f98189a5640 1 -- 172.21.15.38:0/667171432 <== osd.3 v2:172.21.15.38:6808/1286810294 24 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v121'151 uv151 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9810068070 con 0x55e725c48590 2024-09-21T03:42:46.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.654+0000 7f981e6eaf40 1 -- 172.21.15.38:0/667171432 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:110 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55e725c50c00 con 0x55e725c48590 2024-09-21T03:42:46.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.655+0000 7f98189a5640 1 -- 172.21.15.38:0/667171432 <== osd.3 v2:172.21.15.38:6808/1286810294 25 ==== osd_op_reply(110 609.00000000 [omap-set-header] v121'152 uv152 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9810068070 con 0x55e725c48590 2024-09-21T03:42:46.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.655+0000 7f981e6eaf40 1 --2- 172.21.15.38:0/667171432 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x55e725c51880 0x55e725c53ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:46.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.655+0000 7f981e6eaf40 1 -- 172.21.15.38:0/667171432 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55e725c50c00 con 0x55e725c51880 2024-09-21T03:42:46.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.655+0000 7f98179a3640 1 --2- 172.21.15.38:0/667171432 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x55e725c51880 0x55e725c53ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:46.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.656+0000 7f98179a3640 1 --2- 172.21.15.38:0/667171432 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x55e725c51880 0x55e725c53ca0 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:46.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.657+0000 7f98179a3640 1 -- 172.21.15.38:0/667171432 <== osd.2 v2:172.21.15.38:6824/2577858902 1 ==== osd_op_reply(111 400.00000000 [create] v121'125 uv124 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f980c01b040 con 0x55e725c51880 2024-09-21T03:42:46.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.657+0000 7f981e6eaf40 1 -- 172.21.15.38:0/667171432 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e121) -- 0x55e725c50c00 con 0x55e725c51880 2024-09-21T03:42:46.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.658+0000 7f98179a3640 1 -- 172.21.15.38:0/667171432 <== osd.2 v2:172.21.15.38:6824/2577858902 2 ==== osd_op_reply(112 400.00000000 [delete] v121'126 uv126 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f980c01b040 con 0x55e725c51880 2024-09-21T03:42:46.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.658+0000 7f981e6eaf40 1 -- 172.21.15.38:0/667171432 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55e725c50c00 con 0x55e725c51880 2024-09-21T03:42:46.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.659+0000 7f98179a3640 1 -- 172.21.15.38:0/667171432 <== osd.2 v2:172.21.15.38:6824/2577858902 3 ==== osd_op_reply(113 400.00000000 [create] v121'127 uv127 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f980c01b040 con 0x55e725c51880 2024-09-21T03:42:46.662 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-09-21T03:42:46.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.659+0000 7f981e6eaf40 1 -- 172.21.15.38:0/667171432 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:114 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55e725c50c00 con 0x55e725c51880 2024-09-21T03:42:46.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.660+0000 7f98179a3640 1 -- 172.21.15.38:0/667171432 <== osd.2 v2:172.21.15.38:6824/2577858902 4 ==== osd_op_reply(114 400.00000000 [write 0~22] v121'128 uv128 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f980c01b040 con 0x55e725c51880 2024-09-21T03:42:46.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.660+0000 7f981e6eaf40 1 -- 172.21.15.38:0/667171432 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:115 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55e725c50c00 con 0x55e725c51880 2024-09-21T03:42:46.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.661+0000 7f98179a3640 1 -- 172.21.15.38:0/667171432 <== osd.2 v2:172.21.15.38:6824/2577858902 5 ==== osd_op_reply(115 400.00000000 [omap-set-header] v121'129 uv129 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f980c01b040 con 0x55e725c51880 2024-09-21T03:42:46.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.661+0000 7f981e6eaf40 1 -- 172.21.15.38:0/667171432 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:116 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55e725c50c00 con 0x55e725c4d8d0 2024-09-21T03:42:46.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.662+0000 7f98181a4640 1 -- 172.21.15.38:0/667171432 <== osd.1 v2:172.21.15.38:6816/1491818732 17 ==== osd_op_reply(116 100.00000000.inode [create] v121'302 uv294 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7f9808019140 con 0x55e725c4d8d0 2024-09-21T03:42:46.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.662+0000 7f981e6eaf40 1 -- 172.21.15.38:0/667171432 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:117 13.1f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e121) -- 0x55e725c50c00 con 0x55e725c4d8d0 2024-09-21T03:42:46.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.663+0000 7f98181a4640 1 -- 172.21.15.38:0/667171432 <== osd.1 v2:172.21.15.38:6816/1491818732 18 ==== osd_op_reply(117 100.00000000.inode [delete] v121'303 uv303 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f9808019140 con 0x55e725c4d8d0 2024-09-21T03:42:46.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.663+0000 7f981e6eaf40 1 -- 172.21.15.38:0/667171432 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:118 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55e725c50c00 con 0x55e725c4d8d0 2024-09-21T03:42:46.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.665+0000 7f98181a4640 1 -- 172.21.15.38:0/667171432 <== osd.1 v2:172.21.15.38:6816/1491818732 19 ==== osd_op_reply(118 100.00000000.inode [create] v121'304 uv304 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f9808019140 con 0x55e725c4d8d0 2024-09-21T03:42:46.668 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-09-21T03:42:46.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.665+0000 7f981e6eaf40 1 -- 172.21.15.38:0/667171432 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:119 13.1f 13:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55e725c50c00 con 0x55e725c4d8d0 2024-09-21T03:42:46.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.666+0000 7f98181a4640 1 -- 172.21.15.38:0/667171432 <== osd.1 v2:172.21.15.38:6816/1491818732 20 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v121'305 uv305 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f9808019140 con 0x55e725c4d8d0 2024-09-21T03:42:46.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.666+0000 7f981e6eaf40 1 -- 172.21.15.38:0/667171432 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:120 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55e725c50c00 con 0x55e725c4d8d0 2024-09-21T03:42:46.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.667+0000 7f98181a4640 1 -- 172.21.15.38:0/667171432 <== osd.1 v2:172.21.15.38:6816/1491818732 21 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v121'306 uv306 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f9808019140 con 0x55e725c4d8d0 2024-09-21T03:42:46.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.667+0000 7f981e6eaf40 1 -- 172.21.15.38:0/667171432 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:121 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55e725c50c00 con 0x55e725c4d8d0 2024-09-21T03:42:46.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.668+0000 7f98181a4640 1 -- 172.21.15.38:0/667171432 <== osd.1 v2:172.21.15.38:6816/1491818732 22 ==== osd_op_reply(121 1.00000000 [create] v121'307 uv301 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7f9808019140 con 0x55e725c4d8d0 2024-09-21T03:42:46.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.668+0000 7f981e6eaf40 1 -- 172.21.15.38:0/667171432 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:122 13.1f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e121) -- 0x55e725c50c00 con 0x55e725c4d8d0 2024-09-21T03:42:46.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.669+0000 7f98181a4640 1 -- 172.21.15.38:0/667171432 <== osd.1 v2:172.21.15.38:6816/1491818732 23 ==== osd_op_reply(122 1.00000000 [delete] v121'308 uv308 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f9808019140 con 0x55e725c4d8d0 2024-09-21T03:42:46.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.669+0000 7f981e6eaf40 1 -- 172.21.15.38:0/667171432 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:123 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55e725c50c00 con 0x55e725c4d8d0 2024-09-21T03:42:46.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.670+0000 7f98181a4640 1 -- 172.21.15.38:0/667171432 <== osd.1 v2:172.21.15.38:6816/1491818732 24 ==== osd_op_reply(123 1.00000000 [create] v121'309 uv309 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f9808019140 con 0x55e725c4d8d0 2024-09-21T03:42:46.673 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-09-21T03:42:46.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.670+0000 7f981e6eaf40 1 -- 172.21.15.38:0/667171432 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:124 13.1f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55e725c50c00 con 0x55e725c4d8d0 2024-09-21T03:42:46.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.672+0000 7f98181a4640 1 -- 172.21.15.38:0/667171432 <== osd.1 v2:172.21.15.38:6816/1491818732 25 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v121'310 uv310 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f9808019140 con 0x55e725c4d8d0 2024-09-21T03:42:46.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.672+0000 7f981e6eaf40 1 -- 172.21.15.38:0/667171432 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:125 13.1f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55e725c50c00 con 0x55e725c4d8d0 2024-09-21T03:42:46.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.673+0000 7f98181a4640 1 -- 172.21.15.38:0/667171432 <== osd.1 v2:172.21.15.38:6816/1491818732 26 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v121'311 uv311 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f9808019140 con 0x55e725c4d8d0 2024-09-21T03:42:46.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.673+0000 7f981e6eaf40 1 -- 172.21.15.38:0/667171432 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:126 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55e725c50c00 con 0x55e725c4d8d0 2024-09-21T03:42:46.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.674+0000 7f98181a4640 1 -- 172.21.15.38:0/667171432 <== osd.1 v2:172.21.15.38:6816/1491818732 27 ==== osd_op_reply(126 1.00000000 [omap-set-header] v121'312 uv312 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f9808019140 con 0x55e725c4d8d0 2024-09-21T03:42:46.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.674+0000 7f981e6eaf40 1 -- 172.21.15.38:0/667171432 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:127 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=584b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55e725c542a0 con 0x55e725c4d8d0 2024-09-21T03:42:46.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.675+0000 7f98181a4640 1 -- 172.21.15.38:0/667171432 <== osd.1 v2:172.21.15.38:6816/1491818732 28 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v121'313 uv313 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f9808019140 con 0x55e725c4d8d0 2024-09-21T03:42:46.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.675+0000 7f981e6eaf40 1 -- 172.21.15.38:0/667171432 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x55e725c451a0 msgr2=0x55e725c475c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:46.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.675+0000 7f981e6eaf40 1 --2- 172.21.15.38:0/667171432 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x55e725c451a0 0x55e725c475c0 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:46.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.675+0000 7f981e6eaf40 1 -- 172.21.15.38:0/667171432 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x55e725c4d8d0 msgr2=0x55e725c4fcf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:46.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.675+0000 7f981e6eaf40 1 --2- 172.21.15.38:0/667171432 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x55e725c4d8d0 0x55e725c4fcf0 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:46.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.675+0000 7f981e6eaf40 1 -- 172.21.15.38:0/667171432 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x55e725c51880 msgr2=0x55e725c53ca0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:46.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.675+0000 7f981e6eaf40 1 --2- 172.21.15.38:0/667171432 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x55e725c51880 0x55e725c53ca0 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:46.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.675+0000 7f981e6eaf40 1 -- 172.21.15.38:0/667171432 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x55e725c48590 msgr2=0x55e725c48a00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:46.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.675+0000 7f981e6eaf40 1 --2- 172.21.15.38:0/667171432 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x55e725c48590 0x55e725c48a00 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:46.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.675+0000 7f981e6eaf40 1 -- 172.21.15.38:0/667171432 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x55e725c40af0 msgr2=0x55e725c42f10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:46.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.675+0000 7f98189a5640 1 -- 172.21.15.38:0/667171432 reap_dead start 2024-09-21T03:42:46.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.675+0000 7f981e6eaf40 1 --2- 172.21.15.38:0/667171432 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x55e725c40af0 0x55e725c42f10 crc :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:46.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.675+0000 7f98189a5640 1 -- 172.21.15.38:0/667171432 reap_dead start 2024-09-21T03:42:46.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.675+0000 7f981e6eaf40 1 -- 172.21.15.38:0/667171432 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x55e725b76a00 msgr2=0x55e725b7a6a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:46.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.675+0000 7f981e6eaf40 1 --2- 172.21.15.38:0/667171432 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x55e725b76a00 0x55e725b7a6a0 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:46.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.675+0000 7f981e6eaf40 1 -- 172.21.15.38:0/667171432 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f98040775a0 msgr2=0x7f9804079ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:46.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.675+0000 7f981e6eaf40 1 --2- 172.21.15.38:0/667171432 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f98040775a0 0x7f9804079ab0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f98080041b0 tx=0x7f980800d040 comp rx=0 tx=0).stop 2024-09-21T03:42:46.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.675+0000 7f981e6eaf40 1 -- 172.21.15.38:0/667171432 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e725b9d670 msgr2=0x55e725b7d310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:46.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.675+0000 7f981e6eaf40 1 --2- 172.21.15.38:0/667171432 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e725b9d670 0x55e725b7d310 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7f98100465b0 tx=0x7f981004d870 comp rx=0 tx=0).stop 2024-09-21T03:42:46.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.675+0000 7f981e6eaf40 1 -- 172.21.15.38:0/667171432 shutdown_connections 2024-09-21T03:42:46.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.675+0000 7f981e6eaf40 1 --2- 172.21.15.38:0/667171432 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x55e725b76a00 0x55e725b7a6a0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:46.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.675+0000 7f981e6eaf40 1 --2- 172.21.15.38:0/667171432 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e725b9d670 0x55e725b7d310 unknown :-1 s=CLOSED pgs=1844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:46.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.675+0000 7f981e6eaf40 1 --2- 172.21.15.38:0/667171432 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f98040775a0 0x7f9804079ab0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:46.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.675+0000 7f981e6eaf40 1 -- 172.21.15.38:0/667171432 >> 172.21.15.38:0/667171432 conn(0x55e7259e6ac0 msgr2=0x55e725b7c5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:46.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.676+0000 7f981e6eaf40 1 -- 172.21.15.38:0/667171432 shutdown_connections 2024-09-21T03:42:46.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.676+0000 7f981e6eaf40 1 -- 172.21.15.38:0/667171432 wait complete. 2024-09-21T03:42:46.682 DEBUG:teuthology.orchestra.run.smithi038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata truncate 200.00000001 0 2024-09-21T03:42:46.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.776+0000 7f977071cf40 1 Processor -- start 2024-09-21T03:42:46.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.776+0000 7f977071cf40 1 -- start start 2024-09-21T03:42:46.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.777+0000 7f977071cf40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561549777240 0x5615497bd610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:46.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.777+0000 7f977071cf40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5615497be090 0x56154982bc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:46.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.777+0000 7f977071cf40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561549833520 0x56154984b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:46.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.777+0000 7f977071cf40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x561549723430 con 0x561549777240 2024-09-21T03:42:46.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.777+0000 7f977071cf40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5615497301f0 con 0x561549833520 2024-09-21T03:42:46.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.777+0000 7f977071cf40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x5615496c4d60 con 0x5615497be090 2024-09-21T03:42:46.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.777+0000 7f9769771640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561549777240 0x5615497bd610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:46.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.777+0000 7f9769771640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561549777240 0x5615497bd610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38348/0 (socket says 172.21.15.38:38348) 2024-09-21T03:42:46.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.777+0000 7f9769771640 1 -- 172.21.15.38:0/562698445 learned_addr learned my addr 172.21.15.38:0/562698445 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:42:46.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.778+0000 7f9769f72640 1 --2- 172.21.15.38:0/562698445 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561549833520 0x56154984b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:46.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.778+0000 7f9769771640 1 -- 172.21.15.38:0/562698445 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5615497be090 msgr2=0x56154982bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:46.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.778+0000 7f9769771640 1 --2- 172.21.15.38:0/562698445 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5615497be090 0x56154982bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:46.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.778+0000 7f9769771640 1 -- 172.21.15.38:0/562698445 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561549833520 msgr2=0x56154984b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:46.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.778+0000 7f9769771640 1 --2- 172.21.15.38:0/562698445 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561549833520 0x56154984b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:46.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.778+0000 7f9769771640 1 -- 172.21.15.38:0/562698445 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561549844080 con 0x561549777240 2024-09-21T03:42:46.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.778+0000 7f9769f72640 1 --2- 172.21.15.38:0/562698445 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561549833520 0x56154984b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:42:46.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.778+0000 7f9769771640 1 --2- 172.21.15.38:0/562698445 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561549777240 0x5615497bd610 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7f975c001dc0 tx=0x7f975c00b590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0a0a88c44b3607b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:46.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.778+0000 7f975b7fe640 1 -- 172.21.15.38:0/562698445 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f975c016070 con 0x561549777240 2024-09-21T03:42:46.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.778+0000 7f975b7fe640 1 -- 172.21.15.38:0/562698445 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f975c00f690 con 0x561549777240 2024-09-21T03:42:46.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.778+0000 7f975b7fe640 1 -- 172.21.15.38:0/562698445 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f975c00fb20 con 0x561549777240 2024-09-21T03:42:46.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.779+0000 7f977071cf40 1 -- 172.21.15.38:0/562698445 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561549777240 msgr2=0x5615497bd610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:46.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.779+0000 7f977071cf40 1 --2- 172.21.15.38:0/562698445 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561549777240 0x5615497bd610 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7f975c001dc0 tx=0x7f975c00b590 comp rx=0 tx=0).stop 2024-09-21T03:42:46.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.779+0000 7f977071cf40 1 -- 172.21.15.38:0/562698445 shutdown_connections 2024-09-21T03:42:46.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.779+0000 7f977071cf40 1 --2- 172.21.15.38:0/562698445 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561549833520 0x56154984b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:46.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.779+0000 7f977071cf40 1 --2- 172.21.15.38:0/562698445 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5615497be090 0x56154982bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:46.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.779+0000 7f977071cf40 1 --2- 172.21.15.38:0/562698445 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561549777240 0x5615497bd610 unknown :-1 s=CLOSED pgs=1847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:46.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.779+0000 7f977071cf40 1 -- 172.21.15.38:0/562698445 >> 172.21.15.38:0/562698445 conn(0x561549695ac0 msgr2=0x5615497bca50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:46.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.779+0000 7f977071cf40 1 -- 172.21.15.38:0/562698445 shutdown_connections 2024-09-21T03:42:46.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.779+0000 7f977071cf40 1 -- 172.21.15.38:0/562698445 wait complete. 2024-09-21T03:42:46.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.780+0000 7f977071cf40 1 Processor -- start 2024-09-21T03:42:46.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.780+0000 7f977071cf40 1 -- start start 2024-09-21T03:42:46.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.781+0000 7f977071cf40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x561549777240 0x5615498e1e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:46.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.781+0000 7f977071cf40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5615497be090 0x56154982b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:46.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.781+0000 7f977071cf40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561549833520 0x56154982c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:46.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.781+0000 7f977071cf40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x561549849060 con 0x5615497be090 2024-09-21T03:42:46.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.781+0000 7f977071cf40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5615497a0ed0 con 0x561549833520 2024-09-21T03:42:46.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.781+0000 7f977071cf40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x56154982ad60 con 0x561549777240 2024-09-21T03:42:46.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.781+0000 7f9768f70640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5615497be090 0x56154982b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:46.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.781+0000 7f9768f70640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5615497be090 0x56154982b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38358/0 (socket says 172.21.15.38:38358) 2024-09-21T03:42:46.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.781+0000 7f9768f70640 1 -- 172.21.15.38:0/2838240106 learned_addr learned my addr 172.21.15.38:0/2838240106 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:42:46.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.781+0000 7f9769771640 1 --2- 172.21.15.38:0/2838240106 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x561549777240 0x5615498e1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:46.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.781+0000 7f9769f72640 1 --2- 172.21.15.38:0/2838240106 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561549833520 0x56154982c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:46.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.781+0000 7f9768f70640 1 -- 172.21.15.38:0/2838240106 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x561549777240 msgr2=0x5615498e1e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:46.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.781+0000 7f9768f70640 1 --2- 172.21.15.38:0/2838240106 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x561549777240 0x5615498e1e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:46.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.781+0000 7f9768f70640 1 -- 172.21.15.38:0/2838240106 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561549833520 msgr2=0x56154982c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:46.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.781+0000 7f9768f70640 1 --2- 172.21.15.38:0/2838240106 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561549833520 0x56154982c570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:46.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.781+0000 7f9768f70640 1 -- 172.21.15.38:0/2838240106 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56154982b850 con 0x5615497be090 2024-09-21T03:42:46.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.781+0000 7f9768f70640 1 --2- 172.21.15.38:0/2838240106 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5615497be090 0x56154982b310 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7f9760046700 tx=0x7f9760056c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:46.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.782+0000 7f975a7fc640 1 -- 172.21.15.38:0/2838240106 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9760066070 con 0x5615497be090 2024-09-21T03:42:46.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.782+0000 7f977071cf40 1 -- 172.21.15.38:0/2838240106 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56154982dbe0 con 0x5615497be090 2024-09-21T03:42:46.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.782+0000 7f975a7fc640 1 -- 172.21.15.38:0/2838240106 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f976004d860 con 0x5615497be090 2024-09-21T03:42:46.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.782+0000 7f975a7fc640 1 -- 172.21.15.38:0/2838240106 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9760004580 con 0x5615497be090 2024-09-21T03:42:46.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.782+0000 7f977071cf40 1 -- 172.21.15.38:0/2838240106 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56154982bd70 con 0x5615497be090 2024-09-21T03:42:46.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.783+0000 7f975a7fc640 1 -- 172.21.15.38:0/2838240106 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f9760069070 con 0x5615497be090 2024-09-21T03:42:46.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.784+0000 7f975a7fc640 1 --2- 172.21.15.38:0/2838240106 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f974c077570 0x7f974c079a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:46.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.784+0000 7f975a7fc640 1 -- 172.21.15.38:0/2838240106 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 8454+0+0 (secure 0 0 0) 0x7f97600de340 con 0x5615497be090 2024-09-21T03:42:46.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.784+0000 7f9769771640 1 --2- 172.21.15.38:0/2838240106 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f974c077570 0x7f974c079a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:46.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.784+0000 7f977071cf40 1 --2- 172.21.15.38:0/2838240106 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x5615498453b0 0x56154982a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:46.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.784+0000 7f977071cf40 1 -- 172.21.15.38:0/2838240106 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:1 13.14 13:2e2fa760:::200.00000001:head [truncate 0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x56154982a720 con 0x5615498453b0 2024-09-21T03:42:46.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.784+0000 7f9769f72640 1 --2- 172.21.15.38:0/2838240106 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x5615498453b0 0x56154982a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:46.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.785+0000 7f9769771640 1 --2- 172.21.15.38:0/2838240106 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f974c077570 0x7f974c079a80 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f975c014040 tx=0x7f975c003040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:46.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.785+0000 7f9769f72640 1 --2- 172.21.15.38:0/2838240106 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x5615498453b0 0x56154982a1e0 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:46.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.786+0000 7f9769f72640 1 -- 172.21.15.38:0/2838240106 <== osd.6 v2:172.21.15.195:6826/4055825675 1 ==== osd_op_reply(1 200.00000001 [truncate 0] v121'569 uv569 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9764003680 con 0x5615498453b0 2024-09-21T03:42:46.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.786+0000 7f977071cf40 1 -- 172.21.15.38:0/2838240106 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x5615498453b0 msgr2=0x56154982a1e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:46.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.786+0000 7f977071cf40 1 --2- 172.21.15.38:0/2838240106 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x5615498453b0 0x56154982a1e0 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:46.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.786+0000 7f977071cf40 1 -- 172.21.15.38:0/2838240106 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f974c077570 msgr2=0x7f974c079a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:46.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.786+0000 7f977071cf40 1 --2- 172.21.15.38:0/2838240106 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f974c077570 0x7f974c079a80 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f975c014040 tx=0x7f975c003040 comp rx=0 tx=0).stop 2024-09-21T03:42:46.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.786+0000 7f977071cf40 1 -- 172.21.15.38:0/2838240106 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5615497be090 msgr2=0x56154982b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:46.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.786+0000 7f977071cf40 1 --2- 172.21.15.38:0/2838240106 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5615497be090 0x56154982b310 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7f9760046700 tx=0x7f9760056c20 comp rx=0 tx=0).stop 2024-09-21T03:42:46.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.786+0000 7f9769f72640 1 -- 172.21.15.38:0/2838240106 reap_dead start 2024-09-21T03:42:46.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.787+0000 7f977071cf40 1 -- 172.21.15.38:0/2838240106 shutdown_connections 2024-09-21T03:42:46.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.787+0000 7f977071cf40 1 -- 172.21.15.38:0/2838240106 >> 172.21.15.38:0/2838240106 conn(0x561549695ac0 msgr2=0x5615497bca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:46.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.787+0000 7f977071cf40 1 -- 172.21.15.38:0/2838240106 shutdown_connections 2024-09-21T03:42:46.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.787+0000 7f977071cf40 1 -- 172.21.15.38:0/2838240106 wait complete. 2024-09-21T03:42:46.792 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-21T03:42:46.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.909+0000 7fce5708b640 1 -- 172.21.15.38:0/2354278988 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fce40004db0 msgr2=0x7fce40005280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:46.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.909+0000 7fce5708b640 1 --2- 172.21.15.38:0/2354278988 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fce40004db0 0x7fce40005280 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7fce48009ed0 tx=0x7fce40005830 comp rx=0 tx=0).stop 2024-09-21T03:42:46.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.909+0000 7fce55628640 1 -- 172.21.15.38:0/2354278988 reap_dead start 2024-09-21T03:42:46.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.909+0000 7fce5708b640 1 -- 172.21.15.38:0/2354278988 shutdown_connections 2024-09-21T03:42:46.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.909+0000 7fce5708b640 1 -- 172.21.15.38:0/2354278988 >> 172.21.15.38:0/2354278988 conn(0x7fce5009d830 msgr2=0x7fce500b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:46.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.909+0000 7fce5708b640 1 -- 172.21.15.38:0/2354278988 shutdown_connections 2024-09-21T03:42:46.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.909+0000 7fce5708b640 1 -- 172.21.15.38:0/2354278988 wait complete. 2024-09-21T03:42:46.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.911+0000 7fce5708b640 1 Processor -- start 2024-09-21T03:42:46.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.911+0000 7fce5708b640 1 -- start start 2024-09-21T03:42:46.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.913+0000 7fce5708b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fce40004db0 0x7fce500cbc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:46.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.913+0000 7fce5708b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fce5006f890 0x7fce501569c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:46.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.913+0000 7fce5708b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fce501f8ab0 0x7fce501f8ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:46.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.913+0000 7fce5708b640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fce500cf330 con 0x7fce501f8ab0 2024-09-21T03:42:46.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.913+0000 7fce5708b640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fce500cf1b0 con 0x7fce40004db0 2024-09-21T03:42:46.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.913+0000 7fce5708b640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fce500b6580 con 0x7fce5006f890 2024-09-21T03:42:46.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.913+0000 7fce55628640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fce501f8ab0 0x7fce501f8ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:46.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.913+0000 7fce55628640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fce501f8ab0 0x7fce501f8ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38368/0 (socket says 172.21.15.38:38368) 2024-09-21T03:42:46.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.913+0000 7fce55628640 1 -- 172.21.15.38:0/1006617966 learned_addr learned my addr 172.21.15.38:0/1006617966 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:42:46.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.913+0000 7fce54e27640 1 --2- 172.21.15.38:0/1006617966 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fce40004db0 0x7fce500cbc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:46.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.913+0000 7fce4ffff640 1 --2- 172.21.15.38:0/1006617966 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fce5006f890 0x7fce501569c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:46.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.913+0000 7fce55628640 1 -- 172.21.15.38:0/1006617966 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fce5006f890 msgr2=0x7fce501569c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:46.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.913+0000 7fce55628640 1 --2- 172.21.15.38:0/1006617966 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fce5006f890 0x7fce501569c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:46.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.913+0000 7fce55628640 1 -- 172.21.15.38:0/1006617966 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fce40004db0 msgr2=0x7fce500cbc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:46.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.913+0000 7fce55628640 1 --2- 172.21.15.38:0/1006617966 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fce40004db0 0x7fce500cbc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:46.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.913+0000 7fce55628640 1 -- 172.21.15.38:0/1006617966 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce500ce360 con 0x7fce501f8ab0 2024-09-21T03:42:46.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.913+0000 7fce55628640 1 --2- 172.21.15.38:0/1006617966 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fce501f8ab0 0x7fce501f8ec0 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7fce4802aed0 tx=0x7fce48033c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:46.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.914+0000 7fce4d7fa640 1 -- 172.21.15.38:0/1006617966 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce48042070 con 0x7fce501f8ab0 2024-09-21T03:42:46.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.914+0000 7fce4d7fa640 1 -- 172.21.15.38:0/1006617966 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fce48003970 con 0x7fce501f8ab0 2024-09-21T03:42:46.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.914+0000 7fce5708b640 1 -- 172.21.15.38:0/1006617966 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce501f9400 con 0x7fce501f8ab0 2024-09-21T03:42:46.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.914+0000 7fce4d7fa640 1 -- 172.21.15.38:0/1006617966 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce48038040 con 0x7fce501f8ab0 2024-09-21T03:42:46.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.914+0000 7fce5708b640 1 -- 172.21.15.38:0/1006617966 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce500b14d0 con 0x7fce501f8ab0 2024-09-21T03:42:46.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.915+0000 7fce5708b640 1 -- 172.21.15.38:0/1006617966 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce500cc250 con 0x7fce501f8ab0 2024-09-21T03:42:46.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.915+0000 7fce4d7fa640 1 -- 172.21.15.38:0/1006617966 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fce48045070 con 0x7fce501f8ab0 2024-09-21T03:42:46.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.916+0000 7fce4d7fa640 1 --2- 172.21.15.38:0/1006617966 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fce34078b90 0x7fce3407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:46.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.916+0000 7fce4d7fa640 1 -- 172.21.15.38:0/1006617966 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 8454+0+0 (secure 0 0 0) 0x7fce480bbb00 con 0x7fce501f8ab0 2024-09-21T03:42:46.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.916+0000 7fce54e27640 1 --2- 172.21.15.38:0/1006617966 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fce34078b90 0x7fce3407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:46.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.917+0000 7fce54e27640 1 --2- 172.21.15.38:0/1006617966 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fce34078b90 0x7fce3407b0a0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fce44006bd0 tx=0x7fce440067a0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:46.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:46.920+0000 7fce4d7fa640 1 -- 172.21.15.38:0/1006617966 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fce501f9400 con 0x7fce501f8ab0 2024-09-21T03:42:47.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:47.145+0000 7fce5708b640 1 -- 172.21.15.38:0/1006617966 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7fce500b6ae0 con 0x7fce501f8ab0 2024-09-21T03:42:47.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:47.416+0000 7fce4d7fa640 1 -- 172.21.15.38:0/1006617966 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v236) ==== 162+0+0 (secure 0 0 0) 0x7fce48082de0 con 0x7fce501f8ab0 2024-09-21T03:42:47.419 INFO:teuthology.orchestra.run.smithi038.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-21T03:42:47.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:47.419+0000 7fce5708b640 1 -- 172.21.15.38:0/1006617966 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fce34078b90 msgr2=0x7fce3407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:47.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:47.419+0000 7fce5708b640 1 --2- 172.21.15.38:0/1006617966 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fce34078b90 0x7fce3407b0a0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fce44006bd0 tx=0x7fce440067a0 comp rx=0 tx=0).stop 2024-09-21T03:42:47.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:47.419+0000 7fce5708b640 1 -- 172.21.15.38:0/1006617966 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fce501f8ab0 msgr2=0x7fce501f8ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:47.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:47.419+0000 7fce5708b640 1 --2- 172.21.15.38:0/1006617966 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fce501f8ab0 0x7fce501f8ec0 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7fce4802aed0 tx=0x7fce48033c10 comp rx=0 tx=0).stop 2024-09-21T03:42:47.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:47.419+0000 7fce5708b640 1 -- 172.21.15.38:0/1006617966 shutdown_connections 2024-09-21T03:42:47.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:47.419+0000 7fce5708b640 1 --2- 172.21.15.38:0/1006617966 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fce501f8ab0 0x7fce501f8ec0 unknown :-1 s=CLOSED pgs=1851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:47.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:47.419+0000 7fce5708b640 1 --2- 172.21.15.38:0/1006617966 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fce34078b90 0x7fce3407b0a0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:47.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:47.419+0000 7fce5708b640 1 --2- 172.21.15.38:0/1006617966 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fce5006f890 0x7fce501569c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:47.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:47.419+0000 7fce5708b640 1 --2- 172.21.15.38:0/1006617966 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fce40004db0 0x7fce500cbc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:47.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:47.419+0000 7fce5708b640 1 -- 172.21.15.38:0/1006617966 >> 172.21.15.38:0/1006617966 conn(0x7fce5009d830 msgr2=0x7fce500b5440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:47.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:47.419+0000 7fce5708b640 1 -- 172.21.15.38:0/1006617966 shutdown_connections 2024-09-21T03:42:47.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:47.419+0000 7fce5708b640 1 -- 172.21.15.38:0/1006617966 wait complete. 2024-09-21T03:42:47.435 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:42:47.437 INFO:tasks.ceph.mds.a.smithi038.stderr:2024-09-21T03:42:47.434+0000 7f24e1eef640 -1 log_channel(cluster) log [ERR] : missing journal object 2024-09-21T03:42:47.438 INFO:tasks.ceph.mon.a.smithi038.stderr:2024-09-21T03:42:47.435+0000 7f3f45713640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T03:42:47.438 INFO:tasks.ceph.mon.a.smithi038.stderr:2024-09-21T03:42:47.435+0000 7f3f45713640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-09-21T03:42:47.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:47.549+0000 7f9e21f4a640 1 -- 172.21.15.38:0/349172533 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9e1c162f00 msgr2=0x7f9e1c0d24d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:47.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:47.549+0000 7f9e21f4a640 1 --2- 172.21.15.38:0/349172533 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9e1c162f00 0x7f9e1c0d24d0 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7f9e0c005fd0 tx=0x7f9e0c02a5d0 comp rx=0 tx=0).stop 2024-09-21T03:42:47.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:47.549+0000 7f9e21f4a640 1 -- 172.21.15.38:0/349172533 shutdown_connections 2024-09-21T03:42:47.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:47.549+0000 7f9e21f4a640 1 --2- 172.21.15.38:0/349172533 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9e1c0b2e40 0x7f9e1c0b3250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:47.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:47.549+0000 7f9e21f4a640 1 --2- 172.21.15.38:0/349172533 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9e1c162f00 0x7f9e1c0d24d0 unknown :-1 s=CLOSED pgs=1852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:47.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:47.549+0000 7f9e21f4a640 1 -- 172.21.15.38:0/349172533 >> 172.21.15.38:0/349172533 conn(0x7f9e1c0b6b20 msgr2=0x7f9e1c0d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:47.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:47.549+0000 7f9e21f4a640 1 -- 172.21.15.38:0/349172533 shutdown_connections 2024-09-21T03:42:47.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:47.549+0000 7f9e21f4a640 1 -- 172.21.15.38:0/349172533 wait complete. 2024-09-21T03:42:47.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:47.551+0000 7f9e21f4a640 1 Processor -- start 2024-09-21T03:42:47.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:47.551+0000 7f9e21f4a640 1 -- start start 2024-09-21T03:42:47.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:47.552+0000 7f9e21f4a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9e1c0b2e40 0x7f9e1c0b0040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:47.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:47.552+0000 7f9e21f4a640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9e1c162280 0x7f9e1c0ceb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:47.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:47.552+0000 7f9e21f4a640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9e1c0cf0b0 0x7f9e1c0b1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:47.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:47.552+0000 7f9e21f4a640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f9e1c15e4b0 con 0x7f9e1c0b2e40 2024-09-21T03:42:47.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:47.552+0000 7f9e21f4a640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f9e1c0b3a20 con 0x7f9e1c0cf0b0 2024-09-21T03:42:47.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:47.552+0000 7f9e21f4a640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f9e1c15e630 con 0x7f9e1c162280 2024-09-21T03:42:47.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:47.552+0000 7f9e1b7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9e1c0b2e40 0x7f9e1c0b0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:47.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:47.553+0000 7f9e1b7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9e1c0b2e40 0x7f9e1c0b0040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38376/0 (socket says 172.21.15.38:38376) 2024-09-21T03:42:47.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:47.553+0000 7f9e1b7fe640 1 -- 172.21.15.38:0/3743209278 learned_addr learned my addr 172.21.15.38:0/3743209278 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:42:47.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:47.553+0000 7f9e1affd640 1 --2- 172.21.15.38:0/3743209278 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9e1c162280 0x7f9e1c0ceb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:47.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:47.553+0000 7f9e1bfff640 1 --2- 172.21.15.38:0/3743209278 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9e1c0cf0b0 0x7f9e1c0b1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:47.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:47.553+0000 7f9e1b7fe640 1 -- 172.21.15.38:0/3743209278 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9e1c162280 msgr2=0x7f9e1c0ceb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:47.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:47.553+0000 7f9e1b7fe640 1 --2- 172.21.15.38:0/3743209278 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9e1c162280 0x7f9e1c0ceb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:47.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:47.553+0000 7f9e1b7fe640 1 -- 172.21.15.38:0/3743209278 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9e1c0cf0b0 msgr2=0x7f9e1c0b1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:47.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:47.553+0000 7f9e1b7fe640 1 --2- 172.21.15.38:0/3743209278 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9e1c0cf0b0 0x7f9e1c0b1ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:47.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:47.553+0000 7f9e1b7fe640 1 -- 172.21.15.38:0/3743209278 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e1c14dad0 con 0x7f9e1c0b2e40 2024-09-21T03:42:47.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:47.553+0000 7f9e1affd640 1 --2- 172.21.15.38:0/3743209278 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9e1c162280 0x7f9e1c0ceb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:42:47.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:47.553+0000 7f9e1b7fe640 1 --2- 172.21.15.38:0/3743209278 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9e1c0b2e40 0x7f9e1c0b0040 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7f9e0c034040 tx=0x7f9e0c0038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:47.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:47.553+0000 7f9e03fff640 1 -- 172.21.15.38:0/3743209278 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e0c03e030 con 0x7f9e1c0b2e40 2024-09-21T03:42:47.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:47.553+0000 7f9e03fff640 1 -- 172.21.15.38:0/3743209278 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9e0c003ca0 con 0x7f9e1c0b2e40 2024-09-21T03:42:47.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:47.553+0000 7f9e21f4a640 1 -- 172.21.15.38:0/3743209278 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e1c14ddb0 con 0x7f9e1c0b2e40 2024-09-21T03:42:47.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:47.553+0000 7f9e03fff640 1 -- 172.21.15.38:0/3743209278 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e0c035040 con 0x7f9e1c0b2e40 2024-09-21T03:42:47.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:47.553+0000 7f9e21f4a640 1 -- 172.21.15.38:0/3743209278 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e1c0abcc0 con 0x7f9e1c0b2e40 2024-09-21T03:42:47.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:47.554+0000 7f9e21f4a640 1 -- 172.21.15.38:0/3743209278 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e1c162f00 con 0x7f9e1c0b2e40 2024-09-21T03:42:47.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:47.555+0000 7f9e03fff640 1 -- 172.21.15.38:0/3743209278 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f9e0c041030 con 0x7f9e1c0b2e40 2024-09-21T03:42:47.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:47.555+0000 7f9e03fff640 1 --2- 172.21.15.38:0/3743209278 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9e08078bc0 0x7f9e0807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:47.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:47.556+0000 7f9e03fff640 1 -- 172.21.15.38:0/3743209278 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 8540+0+0 (secure 0 0 0) 0x7f9e0c0bbd80 con 0x7f9e1c0b2e40 2024-09-21T03:42:47.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:47.556+0000 7f9e1affd640 1 --2- 172.21.15.38:0/3743209278 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9e08078bc0 0x7f9e0807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:47.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:47.556+0000 7f9e1affd640 1 --2- 172.21.15.38:0/3743209278 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9e08078bc0 0x7f9e0807b0d0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f9e1c0b9240 tx=0x7f9e10009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:47.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:47.560+0000 7f9e03fff640 1 -- 172.21.15.38:0/3743209278 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f9e0c083780 con 0x7f9e1c0b2e40 2024-09-21T03:42:47.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:47.784+0000 7f9e21f4a640 1 -- 172.21.15.38:0/3743209278 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9e1c0cfdb0 con 0x7f9e1c0b2e40 2024-09-21T03:42:47.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:47.785+0000 7f9e03fff640 1 -- 172.21.15.38:0/3743209278 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 238 v238) ==== 95+0+3566 (secure 0 0 0) 0x7f9e0c083080 con 0x7f9e1c0b2e40 2024-09-21T03:42:47.788 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:42:47.788 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":238,"btime":"2024-09-21T03:42:47:436932+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6086,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/965972663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":965972663},{"type":"v1","addr":"172.21.15.195:6836","nonce":965972663}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6700,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/3176122334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":3176122334},{"type":"v1","addr":"172.21.15.195:6837","nonce":3176122334}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":238,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:40:02.319282+0000","modified":"2024-09-21T03:42:47.436872+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":122,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-21T03:42:47.788 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 238 2024-09-21T03:42:47.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:47.787+0000 7f9e21f4a640 1 -- 172.21.15.38:0/3743209278 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9e08078bc0 msgr2=0x7f9e0807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:47.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:47.787+0000 7f9e21f4a640 1 --2- 172.21.15.38:0/3743209278 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9e08078bc0 0x7f9e0807b0d0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f9e1c0b9240 tx=0x7f9e10009040 comp rx=0 tx=0).stop 2024-09-21T03:42:47.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:47.787+0000 7f9e21f4a640 1 -- 172.21.15.38:0/3743209278 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9e1c0b2e40 msgr2=0x7f9e1c0b0040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:47.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:47.787+0000 7f9e21f4a640 1 --2- 172.21.15.38:0/3743209278 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9e1c0b2e40 0x7f9e1c0b0040 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7f9e0c034040 tx=0x7f9e0c0038a0 comp rx=0 tx=0).stop 2024-09-21T03:42:47.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:47.787+0000 7f9e21f4a640 1 -- 172.21.15.38:0/3743209278 shutdown_connections 2024-09-21T03:42:47.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:47.787+0000 7f9e21f4a640 1 --2- 172.21.15.38:0/3743209278 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9e1c0cf0b0 0x7f9e1c0b1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:47.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:47.787+0000 7f9e21f4a640 1 --2- 172.21.15.38:0/3743209278 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9e08078bc0 0x7f9e0807b0d0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:47.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:47.787+0000 7f9e21f4a640 1 --2- 172.21.15.38:0/3743209278 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9e1c162280 0x7f9e1c0ceb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:47.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:47.787+0000 7f9e21f4a640 1 --2- 172.21.15.38:0/3743209278 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9e1c0b2e40 0x7f9e1c0b0040 unknown :-1 s=CLOSED pgs=1853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:47.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:47.787+0000 7f9e21f4a640 1 -- 172.21.15.38:0/3743209278 >> 172.21.15.38:0/3743209278 conn(0x7f9e1c0b6b20 msgr2=0x7f9e1c06e210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:47.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:47.787+0000 7f9e21f4a640 1 -- 172.21.15.38:0/3743209278 shutdown_connections 2024-09-21T03:42:47.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:47.787+0000 7f9e21f4a640 1 -- 172.21.15.38:0/3743209278 wait complete. 2024-09-21T03:42:47.801 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-21T03:42:47.802 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:42:47.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:47.915+0000 7f270c721640 1 -- 172.21.15.38:0/1401415517 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f270415cb10 msgr2=0x7f270415cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:47.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:47.915+0000 7f270c721640 1 --2- 172.21.15.38:0/1401415517 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f270415cb10 0x7f270415cf20 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7f26f4005fd0 tx=0x7f26f402a8b0 comp rx=0 tx=0).stop 2024-09-21T03:42:47.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:47.915+0000 7f270c721640 1 -- 172.21.15.38:0/1401415517 shutdown_connections 2024-09-21T03:42:47.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:47.915+0000 7f270c721640 1 --2- 172.21.15.38:0/1401415517 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f27040b6250 0x7f27040b6660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:47.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:47.915+0000 7f270c721640 1 --2- 172.21.15.38:0/1401415517 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f270415cb10 0x7f270415cf20 unknown :-1 s=CLOSED pgs=1855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:47.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:47.915+0000 7f270c721640 1 -- 172.21.15.38:0/1401415517 >> 172.21.15.38:0/1401415517 conn(0x7f27040d2350 msgr2=0x7f27040d2760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:47.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:47.915+0000 7f270c721640 1 -- 172.21.15.38:0/1401415517 shutdown_connections 2024-09-21T03:42:47.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:47.915+0000 7f270c721640 1 -- 172.21.15.38:0/1401415517 wait complete. 2024-09-21T03:42:47.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:47.917+0000 7f270c721640 1 Processor -- start 2024-09-21T03:42:47.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:47.917+0000 7f270c721640 1 -- start start 2024-09-21T03:42:47.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:47.918+0000 7f270c721640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f27040b6250 0x7f270415a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:47.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:47.918+0000 7f270c721640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f27040d12e0 0x7f27041e9000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:47.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:47.918+0000 7f270c721640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f270415cb10 0x7f27041e9540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:47.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:47.918+0000 7f270c721640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f27040b6f30 con 0x7f270415cb10 2024-09-21T03:42:47.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:47.918+0000 7f270c721640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f27040b6db0 con 0x7f27040d12e0 2024-09-21T03:42:47.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:47.918+0000 7f270c721640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f27040b70b0 con 0x7f27040b6250 2024-09-21T03:42:47.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:47.918+0000 7f270acbe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f270415cb10 0x7f27041e9540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:47.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:47.918+0000 7f270acbe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f270415cb10 0x7f27041e9540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38400/0 (socket says 172.21.15.38:38400) 2024-09-21T03:42:47.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:47.918+0000 7f270acbe640 1 -- 172.21.15.38:0/3593662613 learned_addr learned my addr 172.21.15.38:0/3593662613 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:42:47.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:47.919+0000 7f270a4bd640 1 --2- 172.21.15.38:0/3593662613 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f27040b6250 0x7f270415a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:47.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:47.919+0000 7f2709cbc640 1 --2- 172.21.15.38:0/3593662613 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f27040d12e0 0x7f27041e9000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:47.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:47.919+0000 7f270acbe640 1 -- 172.21.15.38:0/3593662613 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f27040b6250 msgr2=0x7f270415a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:47.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:47.919+0000 7f270acbe640 1 --2- 172.21.15.38:0/3593662613 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f27040b6250 0x7f270415a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:47.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:47.919+0000 7f270acbe640 1 -- 172.21.15.38:0/3593662613 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f27040d12e0 msgr2=0x7f27041e9000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:47.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:47.919+0000 7f270acbe640 1 --2- 172.21.15.38:0/3593662613 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f27040d12e0 0x7f27041e9000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:47.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:47.919+0000 7f270acbe640 1 -- 172.21.15.38:0/3593662613 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27041e9a80 con 0x7f270415cb10 2024-09-21T03:42:47.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:47.919+0000 7f2709cbc640 1 --2- 172.21.15.38:0/3593662613 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f27040d12e0 0x7f27041e9000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:42:47.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:47.919+0000 7f270acbe640 1 --2- 172.21.15.38:0/3593662613 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f270415cb10 0x7f27041e9540 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7f26fc00d620 tx=0x7f26fc00dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:47.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:47.919+0000 7f2702ffd640 1 -- 172.21.15.38:0/3593662613 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26fc017070 con 0x7f270415cb10 2024-09-21T03:42:47.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:47.919+0000 7f2702ffd640 1 -- 172.21.15.38:0/3593662613 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f26fc005310 con 0x7f270415cb10 2024-09-21T03:42:47.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:47.919+0000 7f270c721640 1 -- 172.21.15.38:0/3593662613 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f270415c4e0 con 0x7f270415cb10 2024-09-21T03:42:47.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:47.919+0000 7f2702ffd640 1 -- 172.21.15.38:0/3593662613 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26fc0047a0 con 0x7f270415cb10 2024-09-21T03:42:47.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:47.919+0000 7f270c721640 1 -- 172.21.15.38:0/3593662613 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27041e8240 con 0x7f270415cb10 2024-09-21T03:42:47.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:47.920+0000 7f270c721640 1 -- 172.21.15.38:0/3593662613 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f270415c250 con 0x7f270415cb10 2024-09-21T03:42:47.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:47.921+0000 7f2702ffd640 1 -- 172.21.15.38:0/3593662613 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f26fc004940 con 0x7f270415cb10 2024-09-21T03:42:47.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:47.922+0000 7f2702ffd640 1 --2- 172.21.15.38:0/3593662613 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f26f0078b90 0x7f26f007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:47.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:47.922+0000 7f2702ffd640 1 -- 172.21.15.38:0/3593662613 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 8540+0+0 (secure 0 0 0) 0x7f26fc09bc50 con 0x7f270415cb10 2024-09-21T03:42:47.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:47.922+0000 7f270a4bd640 1 --2- 172.21.15.38:0/3593662613 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f26f0078b90 0x7f26f007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:47.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:47.922+0000 7f270a4bd640 1 --2- 172.21.15.38:0/3593662613 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f26f0078b90 0x7f26f007b0a0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f26f4034000 tx=0x7f26f4004ab0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:47.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:47.925+0000 7f2702ffd640 1 -- 172.21.15.38:0/3593662613 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f270415c4e0 con 0x7f270415cb10 2024-09-21T03:42:48.147 INFO:tasks.ceph.mds.c.smithi038.stderr: -41> 2024-09-21T03:42:43.393+0000 7f7bcc751640 -1 log_channel(cluster) log [ERR] : Error loading MDS rank 0: (22) Invalid argument 2024-09-21T03:42:48.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:48.148+0000 7f270c721640 1 -- 172.21.15.38:0/3593662613 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f27040b6660 con 0x7f270415cb10 2024-09-21T03:42:48.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:48.148+0000 7f2702ffd640 1 -- 172.21.15.38:0/3593662613 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 238 v238) ==== 95+0+3566 (secure 0 0 0) 0x7f26fc062ff0 con 0x7f270415cb10 2024-09-21T03:42:48.151 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:42:48.151 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":238,"btime":"2024-09-21T03:42:47:436932+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6086,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/965972663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":965972663},{"type":"v1","addr":"172.21.15.195:6836","nonce":965972663}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6700,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/3176122334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":3176122334},{"type":"v1","addr":"172.21.15.195:6837","nonce":3176122334}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":238,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:40:02.319282+0000","modified":"2024-09-21T03:42:47.436872+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":122,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-21T03:42:48.151 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 238 2024-09-21T03:42:48.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:48.150+0000 7f270c721640 1 -- 172.21.15.38:0/3593662613 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f26f0078b90 msgr2=0x7f26f007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:48.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:48.150+0000 7f270c721640 1 --2- 172.21.15.38:0/3593662613 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f26f0078b90 0x7f26f007b0a0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f26f4034000 tx=0x7f26f4004ab0 comp rx=0 tx=0).stop 2024-09-21T03:42:48.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:48.150+0000 7f270c721640 1 -- 172.21.15.38:0/3593662613 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f270415cb10 msgr2=0x7f27041e9540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:48.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:48.150+0000 7f270c721640 1 --2- 172.21.15.38:0/3593662613 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f270415cb10 0x7f27041e9540 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7f26fc00d620 tx=0x7f26fc00dae0 comp rx=0 tx=0).stop 2024-09-21T03:42:48.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:48.150+0000 7f270c721640 1 -- 172.21.15.38:0/3593662613 shutdown_connections 2024-09-21T03:42:48.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:48.150+0000 7f270c721640 1 --2- 172.21.15.38:0/3593662613 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f270415cb10 0x7f27041e9540 unknown :-1 s=CLOSED pgs=1856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:48.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:48.150+0000 7f270c721640 1 --2- 172.21.15.38:0/3593662613 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f27040d12e0 0x7f27041e9000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:48.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:48.150+0000 7f270c721640 1 --2- 172.21.15.38:0/3593662613 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f26f0078b90 0x7f26f007b0a0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:48.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:48.150+0000 7f270c721640 1 --2- 172.21.15.38:0/3593662613 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f27040b6250 0x7f270415a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:48.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:48.150+0000 7f270c721640 1 -- 172.21.15.38:0/3593662613 >> 172.21.15.38:0/3593662613 conn(0x7f27040d2350 msgr2=0x7f270415e190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:48.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:48.150+0000 7f270c721640 1 -- 172.21.15.38:0/3593662613 shutdown_connections 2024-09-21T03:42:48.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:48.150+0000 7f270c721640 1 -- 172.21.15.38:0/3593662613 wait complete. 2024-09-21T03:42:48.165 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-21T03:42:48.165 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:42:48.203 INFO:tasks.ceph.mds.c.smithi038.stdout:starting mds.c at 2024-09-21T03:42:48.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:48.282+0000 7f7c13c77640 1 -- 172.21.15.38:0/4020584647 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7bf8004d40 msgr2=0x7f7bf80051f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:48.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:48.282+0000 7f7c13c77640 1 --2- 172.21.15.38:0/4020584647 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7bf8004d40 0x7f7bf80051f0 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x7f7c0c0d00c0 tx=0x7f7bf8005830 comp rx=0 tx=0).stop 2024-09-21T03:42:48.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:48.282+0000 7f7c12214640 1 -- 172.21.15.38:0/4020584647 reap_dead start 2024-09-21T03:42:48.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:48.282+0000 7f7c13c77640 1 -- 172.21.15.38:0/4020584647 shutdown_connections 2024-09-21T03:42:48.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:48.282+0000 7f7c13c77640 1 -- 172.21.15.38:0/4020584647 >> 172.21.15.38:0/4020584647 conn(0x7f7c0c0b6b20 msgr2=0x7f7c0c0d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:48.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:48.282+0000 7f7c13c77640 1 -- 172.21.15.38:0/4020584647 shutdown_connections 2024-09-21T03:42:48.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:48.282+0000 7f7c13c77640 1 -- 172.21.15.38:0/4020584647 wait complete. 2024-09-21T03:42:48.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:48.284+0000 7f7c13c77640 1 Processor -- start 2024-09-21T03:42:48.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:48.284+0000 7f7c13c77640 1 -- start start 2024-09-21T03:42:48.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:48.285+0000 7f7c13c77640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7bf8004d40 0x7f7c0c1497f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:48.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:48.285+0000 7f7c13c77640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7c0c1f4f60 0x7f7c0c1f8f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:48.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:48.285+0000 7f7c13c77640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7c0c14bf20 0x7f7c0c14c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:48.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:48.285+0000 7f7c13c77640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f7c0c162810 con 0x7f7c0c1f4f60 2024-09-21T03:42:48.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:48.285+0000 7f7c13c77640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f7c0c162690 con 0x7f7c0c14bf20 2024-09-21T03:42:48.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:48.285+0000 7f7c13c77640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f7c0c162990 con 0x7f7bf8004d40 2024-09-21T03:42:48.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:48.285+0000 7f7c11212640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7c0c1f4f60 0x7f7c0c1f8f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:48.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:48.285+0000 7f7c11212640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7c0c1f4f60 0x7f7c0c1f8f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38438/0 (socket says 172.21.15.38:38438) 2024-09-21T03:42:48.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:48.285+0000 7f7c11212640 1 -- 172.21.15.38:0/2833823027 learned_addr learned my addr 172.21.15.38:0/2833823027 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:42:48.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:48.286+0000 7f7c12214640 1 --2- 172.21.15.38:0/2833823027 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7c0c14bf20 0x7f7c0c14c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:48.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:48.286+0000 7f7c11a13640 1 --2- 172.21.15.38:0/2833823027 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7bf8004d40 0x7f7c0c1497f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:48.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:48.286+0000 7f7c11212640 1 -- 172.21.15.38:0/2833823027 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7bf8004d40 msgr2=0x7f7c0c1497f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:48.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:48.286+0000 7f7c11212640 1 --2- 172.21.15.38:0/2833823027 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7bf8004d40 0x7f7c0c1497f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:48.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:48.286+0000 7f7c11212640 1 -- 172.21.15.38:0/2833823027 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7c0c14bf20 msgr2=0x7f7c0c14c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:48.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:48.286+0000 7f7c11212640 1 --2- 172.21.15.38:0/2833823027 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7c0c14bf20 0x7f7c0c14c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:48.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:48.286+0000 7f7c11212640 1 -- 172.21.15.38:0/2833823027 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c0c1f56d0 con 0x7f7c0c1f4f60 2024-09-21T03:42:48.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:48.286+0000 7f7c11212640 1 --2- 172.21.15.38:0/2833823027 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7c0c1f4f60 0x7f7c0c1f8f60 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7f7bfc011ab0 tx=0x7f7bfc011f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:48.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:48.286+0000 7f7c027fc640 1 -- 172.21.15.38:0/2833823027 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bfc01e070 con 0x7f7c0c1f4f60 2024-09-21T03:42:48.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:48.286+0000 7f7c027fc640 1 -- 172.21.15.38:0/2833823027 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7bfc00fc70 con 0x7f7c0c1f4f60 2024-09-21T03:42:48.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:48.286+0000 7f7c13c77640 1 -- 172.21.15.38:0/2833823027 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c0c1f59b0 con 0x7f7c0c1f4f60 2024-09-21T03:42:48.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:48.286+0000 7f7c027fc640 1 -- 172.21.15.38:0/2833823027 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bfc019670 con 0x7f7c0c1f4f60 2024-09-21T03:42:48.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:48.286+0000 7f7c13c77640 1 -- 172.21.15.38:0/2833823027 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c0c14ba00 con 0x7f7c0c1f4f60 2024-09-21T03:42:48.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:48.287+0000 7f7c13c77640 1 -- 172.21.15.38:0/2833823027 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c0c0d0050 con 0x7f7c0c1f4f60 2024-09-21T03:42:48.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:48.288+0000 7f7c027fc640 1 -- 172.21.15.38:0/2833823027 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f7bfc021070 con 0x7f7c0c1f4f60 2024-09-21T03:42:48.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:48.289+0000 7f7c027fc640 1 --2- 172.21.15.38:0/2833823027 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7bf0078bc0 0x7f7bf007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:48.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:48.289+0000 7f7c027fc640 1 -- 172.21.15.38:0/2833823027 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 8540+0+0 (secure 0 0 0) 0x7f7bfc09c140 con 0x7f7c0c1f4f60 2024-09-21T03:42:48.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:48.289+0000 7f7c11a13640 1 --2- 172.21.15.38:0/2833823027 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7bf0078bc0 0x7f7bf007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:48.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:48.290+0000 7f7c11a13640 1 --2- 172.21.15.38:0/2833823027 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7bf0078bc0 0x7f7bf007b0d0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f7c08000c00 tx=0x7f7c08006b70 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:48.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:48.294+0000 7f7c027fc640 1 -- 172.21.15.38:0/2833823027 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f7c0c0d0050 con 0x7f7c0c1f4f60 2024-09-21T03:42:48.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:48.524+0000 7f7c13c77640 1 -- 172.21.15.38:0/2833823027 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7c0c1f5460 con 0x7f7c0c1f4f60 2024-09-21T03:42:48.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:48.525+0000 7f7c027fc640 1 -- 172.21.15.38:0/2833823027 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 239 v239) ==== 95+0+4392 (secure 0 0 0) 0x7f7bfc00fe60 con 0x7f7c0c1f4f60 2024-09-21T03:42:48.528 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:42:48.528 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":239,"btime":"2024-09-21T03:42:48:438528+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6086,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/965972663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":965972663},{"type":"v1","addr":"172.21.15.195:6836","nonce":965972663}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6700,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/3176122334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":3176122334},{"type":"v1","addr":"172.21.15.195:6837","nonce":3176122334}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":9438,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/1995903506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":1995903506},{"type":"v1","addr":"172.21.15.38:6833","nonce":1995903506}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":239}],"filesystems":[{"mdsmap":{"epoch":238,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:40:02.319282+0000","modified":"2024-09-21T03:42:47.436872+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":122,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-21T03:42:48.528 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 239 2024-09-21T03:42:48.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:48.527+0000 7f7c13c77640 1 -- 172.21.15.38:0/2833823027 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7bf0078bc0 msgr2=0x7f7bf007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:48.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:48.527+0000 7f7c13c77640 1 --2- 172.21.15.38:0/2833823027 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7bf0078bc0 0x7f7bf007b0d0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f7c08000c00 tx=0x7f7c08006b70 comp rx=0 tx=0).stop 2024-09-21T03:42:48.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:48.527+0000 7f7c13c77640 1 -- 172.21.15.38:0/2833823027 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7c0c1f4f60 msgr2=0x7f7c0c1f8f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:48.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:48.527+0000 7f7c13c77640 1 --2- 172.21.15.38:0/2833823027 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7c0c1f4f60 0x7f7c0c1f8f60 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7f7bfc011ab0 tx=0x7f7bfc011f70 comp rx=0 tx=0).stop 2024-09-21T03:42:48.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:48.527+0000 7f7c13c77640 1 -- 172.21.15.38:0/2833823027 shutdown_connections 2024-09-21T03:42:48.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:48.527+0000 7f7c13c77640 1 --2- 172.21.15.38:0/2833823027 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7c0c14bf20 0x7f7c0c14c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:48.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:48.527+0000 7f7c13c77640 1 --2- 172.21.15.38:0/2833823027 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7c0c1f4f60 0x7f7c0c1f8f60 unknown :-1 s=CLOSED pgs=1861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:48.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:48.527+0000 7f7c13c77640 1 --2- 172.21.15.38:0/2833823027 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7bf0078bc0 0x7f7bf007b0d0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:48.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:48.527+0000 7f7c13c77640 1 --2- 172.21.15.38:0/2833823027 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7bf8004d40 0x7f7c0c1497f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:48.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:48.527+0000 7f7c13c77640 1 -- 172.21.15.38:0/2833823027 >> 172.21.15.38:0/2833823027 conn(0x7f7c0c0b6b20 msgr2=0x7f7c0c0b3250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:48.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:48.527+0000 7f7c13c77640 1 -- 172.21.15.38:0/2833823027 shutdown_connections 2024-09-21T03:42:48.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:48.527+0000 7f7c13c77640 1 -- 172.21.15.38:0/2833823027 wait complete. 2024-09-21T03:42:48.542 INFO:tasks.cephfs.test_damage:Result: Mutation 'Truncate 200.00000001' led to DAMAGED state 2024-09-21T03:42:48.542 INFO:tasks.cephfs.test_damage:Applying mutation 'Truncate 500.00000000' 2024-09-21T03:42:48.542 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-21T03:42:48.542 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:42:48.598 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056795,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2354 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T03:42:48.598 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi038.front.sepia.ceph.com 2024-09-21T03:42:48.598 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:42:48.598 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T03:42:48.653 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T03:42:48.653 INFO:teuthology.orchestra.run.smithi038.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-09-21T03:42:48.654 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-21T03:42:48.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:48.808+0000 7fa994263640 1 -- 172.21.15.38:0/854215078 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa98c15ee80 msgr2=0x7fa98c0d0170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:48.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:48.808+0000 7fa994263640 1 --2- 172.21.15.38:0/854215078 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa98c15ee80 0x7fa98c0d0170 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fa988009830 tx=0x7fa98802a420 comp rx=0 tx=0).stop 2024-09-21T03:42:48.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:48.809+0000 7fa994263640 1 -- 172.21.15.38:0/854215078 shutdown_connections 2024-09-21T03:42:48.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:48.809+0000 7fa994263640 1 --2- 172.21.15.38:0/854215078 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa98c15ee80 0x7fa98c0d0170 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:48.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:48.809+0000 7fa994263640 1 -- 172.21.15.38:0/854215078 >> 172.21.15.38:0/854215078 conn(0x7fa98c09d830 msgr2=0x7fa98c0b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:48.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:48.809+0000 7fa994263640 1 -- 172.21.15.38:0/854215078 shutdown_connections 2024-09-21T03:42:48.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:48.809+0000 7fa994263640 1 -- 172.21.15.38:0/854215078 wait complete. 2024-09-21T03:42:48.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:48.810+0000 7fa994263640 1 Processor -- start 2024-09-21T03:42:48.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:48.811+0000 7fa994263640 1 -- start start 2024-09-21T03:42:48.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:48.812+0000 7fa994263640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa98c0b9ad0 0x7fa98c0b21e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:48.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:48.812+0000 7fa994263640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa98c0ceb90 0x7fa98c0b2720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:48.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:48.812+0000 7fa994263640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa98c15ee80 0x7fa98c1f5890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:48.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:48.812+0000 7fa994263640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa98c0b6910 con 0x7fa98c15ee80 2024-09-21T03:42:48.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:48.812+0000 7fa994263640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fa98c0b6790 con 0x7fa98c0b9ad0 2024-09-21T03:42:48.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:48.812+0000 7fa994263640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fa98c0b6a90 con 0x7fa98c0ceb90 2024-09-21T03:42:48.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:48.812+0000 7fa992800640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa98c15ee80 0x7fa98c1f5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:48.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:48.812+0000 7fa992800640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa98c15ee80 0x7fa98c1f5890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38444/0 (socket says 172.21.15.38:38444) 2024-09-21T03:42:48.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:48.812+0000 7fa992800640 1 -- 172.21.15.38:0/1508421532 learned_addr learned my addr 172.21.15.38:0/1508421532 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:42:48.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:48.812+0000 7fa991fff640 1 --2- 172.21.15.38:0/1508421532 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa98c0b9ad0 0x7fa98c0b21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:48.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:48.812+0000 7fa9917fe640 1 --2- 172.21.15.38:0/1508421532 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa98c0ceb90 0x7fa98c0b2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:48.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:48.812+0000 7fa992800640 1 -- 172.21.15.38:0/1508421532 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa98c0ceb90 msgr2=0x7fa98c0b2720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:48.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:48.812+0000 7fa992800640 1 --2- 172.21.15.38:0/1508421532 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa98c0ceb90 0x7fa98c0b2720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:48.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:48.812+0000 7fa992800640 1 -- 172.21.15.38:0/1508421532 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa98c0b9ad0 msgr2=0x7fa98c0b21e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:48.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:48.812+0000 7fa992800640 1 --2- 172.21.15.38:0/1508421532 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa98c0b9ad0 0x7fa98c0b21e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:48.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:48.812+0000 7fa992800640 1 -- 172.21.15.38:0/1508421532 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa98c06f4a0 con 0x7fa98c15ee80 2024-09-21T03:42:48.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:48.812+0000 7fa991fff640 1 --2- 172.21.15.38:0/1508421532 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa98c0b9ad0 0x7fa98c0b21e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:42:48.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:48.813+0000 7fa992800640 1 --2- 172.21.15.38:0/1508421532 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa98c15ee80 0x7fa98c1f5890 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7fa98400c700 tx=0x7fa98400cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:48.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:48.813+0000 7fa9827fc640 1 -- 172.21.15.38:0/1508421532 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa984003b70 con 0x7fa98c15ee80 2024-09-21T03:42:48.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:48.813+0000 7fa994263640 1 -- 172.21.15.38:0/1508421532 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa98c159650 con 0x7fa98c15ee80 2024-09-21T03:42:48.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:48.813+0000 7fa9827fc640 1 -- 172.21.15.38:0/1508421532 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa984003d10 con 0x7fa98c15ee80 2024-09-21T03:42:48.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:48.813+0000 7fa9827fc640 1 -- 172.21.15.38:0/1508421532 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa98400aab0 con 0x7fa98c15ee80 2024-09-21T03:42:48.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:48.813+0000 7fa994263640 1 -- 172.21.15.38:0/1508421532 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa98c159c10 con 0x7fa98c15ee80 2024-09-21T03:42:48.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:48.814+0000 7fa994263640 1 -- 172.21.15.38:0/1508421532 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa98c0d0170 con 0x7fa98c15ee80 2024-09-21T03:42:48.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:48.815+0000 7fa9827fc640 1 -- 172.21.15.38:0/1508421532 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa98400ac50 con 0x7fa98c15ee80 2024-09-21T03:42:48.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:48.815+0000 7fa9827fc640 1 --2- 172.21.15.38:0/1508421532 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa970078b90 0x7fa97007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:48.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:48.815+0000 7fa9827fc640 1 -- 172.21.15.38:0/1508421532 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 8540+0+0 (secure 0 0 0) 0x7fa98409b6e0 con 0x7fa98c15ee80 2024-09-21T03:42:48.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:48.816+0000 7fa991fff640 1 --2- 172.21.15.38:0/1508421532 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa970078b90 0x7fa97007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:48.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:48.816+0000 7fa991fff640 1 --2- 172.21.15.38:0/1508421532 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa970078b90 0x7fa97007b0a0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fa988034000 tx=0x7fa988004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:48.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:48.820+0000 7fa9827fc640 1 -- 172.21.15.38:0/1508421532 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fa984062f40 con 0x7fa98c15ee80 2024-09-21T03:42:49.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:49.046+0000 7fa994263640 1 -- 172.21.15.38:0/1508421532 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fa98c15ca90 con 0x7fa98c15ee80 2024-09-21T03:42:49.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:49.453+0000 7fa9827fc640 1 -- 172.21.15.38:0/1508421532 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v240) ==== 161+0+0 (secure 0 0 0) 0x7fa984062840 con 0x7fa98c15ee80 2024-09-21T03:42:49.456 INFO:teuthology.orchestra.run.smithi038.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-21T03:42:49.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:49.456+0000 7fa994263640 1 -- 172.21.15.38:0/1508421532 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa970078b90 msgr2=0x7fa97007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:49.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:49.456+0000 7fa994263640 1 --2- 172.21.15.38:0/1508421532 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa970078b90 0x7fa97007b0a0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fa988034000 tx=0x7fa988004730 comp rx=0 tx=0).stop 2024-09-21T03:42:49.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:49.456+0000 7fa994263640 1 -- 172.21.15.38:0/1508421532 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa98c15ee80 msgr2=0x7fa98c1f5890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:49.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:49.456+0000 7fa994263640 1 --2- 172.21.15.38:0/1508421532 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa98c15ee80 0x7fa98c1f5890 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7fa98400c700 tx=0x7fa98400cbc0 comp rx=0 tx=0).stop 2024-09-21T03:42:49.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:49.456+0000 7fa994263640 1 -- 172.21.15.38:0/1508421532 shutdown_connections 2024-09-21T03:42:49.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:49.456+0000 7fa994263640 1 --2- 172.21.15.38:0/1508421532 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa98c15ee80 0x7fa98c1f5890 unknown :-1 s=CLOSED pgs=1863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:49.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:49.456+0000 7fa994263640 1 --2- 172.21.15.38:0/1508421532 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa970078b90 0x7fa97007b0a0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:49.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:49.456+0000 7fa994263640 1 --2- 172.21.15.38:0/1508421532 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa98c0ceb90 0x7fa98c0b2720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:49.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:49.456+0000 7fa994263640 1 --2- 172.21.15.38:0/1508421532 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa98c0b9ad0 0x7fa98c0b21e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:49.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:49.456+0000 7fa994263640 1 -- 172.21.15.38:0/1508421532 >> 172.21.15.38:0/1508421532 conn(0x7fa98c09d830 msgr2=0x7fa98c1588f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:49.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:49.456+0000 7fa994263640 1 -- 172.21.15.38:0/1508421532 shutdown_connections 2024-09-21T03:42:49.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:49.456+0000 7fa994263640 1 -- 172.21.15.38:0/1508421532 wait complete. 2024-09-21T03:42:49.471 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-09-21T03:42:49.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:49.588+0000 7f5092e71640 1 -- 172.21.15.38:0/3879946231 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f507c004d40 msgr2=0x7f507c005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:49.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:49.588+0000 7f5092e71640 1 --2- 172.21.15.38:0/3879946231 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f507c004d40 0x7f507c005150 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7f5084005fd0 tx=0x7f507c005830 comp rx=0 tx=0).stop 2024-09-21T03:42:49.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:49.588+0000 7f509140e640 1 -- 172.21.15.38:0/3879946231 reap_dead start 2024-09-21T03:42:49.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:49.588+0000 7f5092e71640 1 -- 172.21.15.38:0/3879946231 shutdown_connections 2024-09-21T03:42:49.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:49.588+0000 7f5092e71640 1 -- 172.21.15.38:0/3879946231 >> 172.21.15.38:0/3879946231 conn(0x7f508c09d830 msgr2=0x7f508c0b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:49.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:49.588+0000 7f5092e71640 1 -- 172.21.15.38:0/3879946231 shutdown_connections 2024-09-21T03:42:49.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:49.588+0000 7f5092e71640 1 -- 172.21.15.38:0/3879946231 wait complete. 2024-09-21T03:42:49.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:49.590+0000 7f5092e71640 1 Processor -- start 2024-09-21T03:42:49.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:49.590+0000 7f5092e71640 1 -- start start 2024-09-21T03:42:49.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:49.591+0000 7f5092e71640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f507c004d40 0x7f508c0cc150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:49.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:49.591+0000 7f5092e71640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f508c0cd220 0x7f508c0a1230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:49.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:49.591+0000 7f5092e71640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f508c0cd630 0x7f508c06efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:49.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:49.591+0000 7f5092e71640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f508c0d0170 con 0x7f508c0cd630 2024-09-21T03:42:49.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:49.591+0000 7f5092e71640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f508c0ba0f0 con 0x7f508c0cd220 2024-09-21T03:42:49.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:49.591+0000 7f5092e71640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f508c0d02f0 con 0x7f507c004d40 2024-09-21T03:42:49.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:49.592+0000 7f509140e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f508c0cd630 0x7f508c06efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:49.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:49.592+0000 7f509140e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f508c0cd630 0x7f508c06efb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38458/0 (socket says 172.21.15.38:38458) 2024-09-21T03:42:49.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:49.592+0000 7f509140e640 1 -- 172.21.15.38:0/2043842927 learned_addr learned my addr 172.21.15.38:0/2043842927 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:42:49.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:49.592+0000 7f5090c0d640 1 --2- 172.21.15.38:0/2043842927 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f507c004d40 0x7f508c0cc150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:49.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:49.592+0000 7f508bfff640 1 --2- 172.21.15.38:0/2043842927 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f508c0cd220 0x7f508c0a1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:49.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:49.592+0000 7f509140e640 1 -- 172.21.15.38:0/2043842927 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f507c004d40 msgr2=0x7f508c0cc150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:49.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:49.592+0000 7f509140e640 1 --2- 172.21.15.38:0/2043842927 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f507c004d40 0x7f508c0cc150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:49.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:49.592+0000 7f509140e640 1 -- 172.21.15.38:0/2043842927 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f508c0cd220 msgr2=0x7f508c0a1230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:49.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:49.592+0000 7f509140e640 1 --2- 172.21.15.38:0/2043842927 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f508c0cd220 0x7f508c0a1230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:49.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:49.592+0000 7f509140e640 1 -- 172.21.15.38:0/2043842927 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f508c0cb6b0 con 0x7f508c0cd630 2024-09-21T03:42:49.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:49.592+0000 7f509140e640 1 --2- 172.21.15.38:0/2043842927 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f508c0cd630 0x7f508c06efb0 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7f508402aec0 tx=0x7f5084034bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:49.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:49.592+0000 7f50897fa640 1 -- 172.21.15.38:0/2043842927 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5084038040 con 0x7f508c0cd630 2024-09-21T03:42:49.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:49.592+0000 7f50897fa640 1 -- 172.21.15.38:0/2043842927 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f508402bc30 con 0x7f508c0cd630 2024-09-21T03:42:49.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:49.592+0000 7f5092e71640 1 -- 172.21.15.38:0/2043842927 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f508c0cc690 con 0x7f508c0cd630 2024-09-21T03:42:49.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:49.592+0000 7f50897fa640 1 -- 172.21.15.38:0/2043842927 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5084009200 con 0x7f508c0cd630 2024-09-21T03:42:49.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:49.592+0000 7f5092e71640 1 -- 172.21.15.38:0/2043842927 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f508c0cb950 con 0x7f508c0cd630 2024-09-21T03:42:49.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:49.593+0000 7f5092e71640 1 -- 172.21.15.38:0/2043842927 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f508c0cefa0 con 0x7f508c0cd630 2024-09-21T03:42:49.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:49.594+0000 7f50897fa640 1 -- 172.21.15.38:0/2043842927 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f5084044020 con 0x7f508c0cd630 2024-09-21T03:42:49.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:49.595+0000 7f50897fa640 1 --2- 172.21.15.38:0/2043842927 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5070078bc0 0x7f507007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:49.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:49.595+0000 7f50897fa640 1 -- 172.21.15.38:0/2043842927 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 8540+0+0 (secure 0 0 0) 0x7f50840bbb40 con 0x7f508c0cd630 2024-09-21T03:42:49.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:49.595+0000 7f5090c0d640 1 --2- 172.21.15.38:0/2043842927 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5070078bc0 0x7f507007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:49.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:49.596+0000 7f5090c0d640 1 --2- 172.21.15.38:0/2043842927 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5070078bc0 0x7f507007b0d0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f5080009bd0 tx=0x7f5080005f90 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:49.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:49.599+0000 7f50897fa640 1 -- 172.21.15.38:0/2043842927 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f508404b020 con 0x7f508c0cd630 2024-09-21T03:42:49.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:49.864+0000 7f5092e71640 1 -- 172.21.15.38:0/2043842927 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7f508c160ef0 con 0x7f508c0cd630 2024-09-21T03:42:50.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.473+0000 7f50897fa640 1 -- 172.21.15.38:0/2043842927 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v241) ==== 101+0+0 (secure 0 0 0) 0x7f5084082dd0 con 0x7f508c0cd630 2024-09-21T03:42:50.475 INFO:teuthology.orchestra.run.smithi038.stderr:repaired: restoring rank 6:0 2024-09-21T03:42:50.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.476+0000 7f5092e71640 1 -- 172.21.15.38:0/2043842927 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5070078bc0 msgr2=0x7f507007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:50.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.476+0000 7f5092e71640 1 --2- 172.21.15.38:0/2043842927 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5070078bc0 0x7f507007b0d0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f5080009bd0 tx=0x7f5080005f90 comp rx=0 tx=0).stop 2024-09-21T03:42:50.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.476+0000 7f5092e71640 1 -- 172.21.15.38:0/2043842927 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f508c0cd630 msgr2=0x7f508c06efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:50.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.476+0000 7f5092e71640 1 --2- 172.21.15.38:0/2043842927 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f508c0cd630 0x7f508c06efb0 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7f508402aec0 tx=0x7f5084034bb0 comp rx=0 tx=0).stop 2024-09-21T03:42:50.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.476+0000 7f5092e71640 1 -- 172.21.15.38:0/2043842927 shutdown_connections 2024-09-21T03:42:50.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.476+0000 7f5092e71640 1 --2- 172.21.15.38:0/2043842927 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f508c0cd630 0x7f508c06efb0 unknown :-1 s=CLOSED pgs=1866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:50.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.476+0000 7f5092e71640 1 --2- 172.21.15.38:0/2043842927 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f508c0cd220 0x7f508c0a1230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:50.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.476+0000 7f5092e71640 1 --2- 172.21.15.38:0/2043842927 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5070078bc0 0x7f507007b0d0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:50.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.476+0000 7f5092e71640 1 --2- 172.21.15.38:0/2043842927 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f507c004d40 0x7f508c0cc150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:50.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.476+0000 7f5092e71640 1 -- 172.21.15.38:0/2043842927 >> 172.21.15.38:0/2043842927 conn(0x7f508c09d830 msgr2=0x7f508c0b4740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:50.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.477+0000 7f5092e71640 1 -- 172.21.15.38:0/2043842927 shutdown_connections 2024-09-21T03:42:50.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.477+0000 7f5092e71640 1 -- 172.21.15.38:0/2043842927 wait complete. 2024-09-21T03:42:50.492 DEBUG:teuthology.orchestra.run.smithi038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-09-21T03:42:50.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.551+0000 7f4812d05f40 1 Processor -- start 2024-09-21T03:42:50.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.551+0000 7f4812d05f40 1 -- start start 2024-09-21T03:42:50.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.552+0000 7f4812d05f40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557e54610d00 0x557e546c5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:50.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.552+0000 7f4812d05f40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x557e5463ce30 0x557e546e6540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:50.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.552+0000 7f4812d05f40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557e546cb950 0x557e546c4b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:50.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.552+0000 7f4812d05f40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x557e5455ed60 con 0x557e546cb950 2024-09-21T03:42:50.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.552+0000 7f4812d05f40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x557e545bd430 con 0x557e54610d00 2024-09-21T03:42:50.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.552+0000 7f4812d05f40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x557e54659460 con 0x557e5463ce30 2024-09-21T03:42:50.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.552+0000 7f480cfc0640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557e546cb950 0x557e546c4b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:50.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.552+0000 7f480cfc0640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557e546cb950 0x557e546c4b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38466/0 (socket says 172.21.15.38:38466) 2024-09-21T03:42:50.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.552+0000 7f480cfc0640 1 -- 172.21.15.38:0/29471796 learned_addr learned my addr 172.21.15.38:0/29471796 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:42:50.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.552+0000 7f480c7bf640 1 --2- 172.21.15.38:0/29471796 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557e54610d00 0x557e546c5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:50.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.552+0000 7f480bfbe640 1 --2- 172.21.15.38:0/29471796 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x557e5463ce30 0x557e546e6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:50.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.552+0000 7f480cfc0640 1 -- 172.21.15.38:0/29471796 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x557e5463ce30 msgr2=0x557e546e6540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:50.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.552+0000 7f480cfc0640 1 --2- 172.21.15.38:0/29471796 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x557e5463ce30 0x557e546e6540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:50.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.552+0000 7f480cfc0640 1 -- 172.21.15.38:0/29471796 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557e54610d00 msgr2=0x557e546c5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:50.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.552+0000 7f480cfc0640 1 --2- 172.21.15.38:0/29471796 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557e54610d00 0x557e546c5dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:50.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.552+0000 7f480cfc0640 1 -- 172.21.15.38:0/29471796 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e546c6f60 con 0x557e546cb950 2024-09-21T03:42:50.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.552+0000 7f480c7bf640 1 --2- 172.21.15.38:0/29471796 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557e54610d00 0x557e546c5dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:42:50.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.552+0000 7f480bfbe640 1 --2- 172.21.15.38:0/29471796 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x557e5463ce30 0x557e546e6540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:42:50.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.553+0000 7f480cfc0640 1 --2- 172.21.15.38:0/29471796 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557e546cb950 0x557e546c4b60 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7f4804008640 tx=0x7f4804008b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4b140906f4961ce server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:50.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.553+0000 7f480afbc640 1 -- 172.21.15.38:0/29471796 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f480400d710 con 0x557e546cb950 2024-09-21T03:42:50.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.553+0000 7f480afbc640 1 -- 172.21.15.38:0/29471796 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f480400d8b0 con 0x557e546cb950 2024-09-21T03:42:50.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.553+0000 7f480afbc640 1 -- 172.21.15.38:0/29471796 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f480400dd00 con 0x557e546cb950 2024-09-21T03:42:50.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.554+0000 7f4812d05f40 1 -- 172.21.15.38:0/29471796 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557e546cb950 msgr2=0x557e546c4b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:50.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.554+0000 7f4812d05f40 1 --2- 172.21.15.38:0/29471796 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557e546cb950 0x557e546c4b60 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7f4804008640 tx=0x7f4804008b00 comp rx=0 tx=0).stop 2024-09-21T03:42:50.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.554+0000 7f4812d05f40 1 -- 172.21.15.38:0/29471796 shutdown_connections 2024-09-21T03:42:50.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.554+0000 7f4812d05f40 1 --2- 172.21.15.38:0/29471796 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557e546cb950 0x557e546c4b60 unknown :-1 s=CLOSED pgs=1869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:50.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.554+0000 7f4812d05f40 1 --2- 172.21.15.38:0/29471796 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x557e5463ce30 0x557e546e6540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:50.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.554+0000 7f4812d05f40 1 --2- 172.21.15.38:0/29471796 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557e54610d00 0x557e546c5dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:50.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.554+0000 7f4812d05f40 1 -- 172.21.15.38:0/29471796 >> 172.21.15.38:0/29471796 conn(0x557e5452fac0 msgr2=0x557e546ca000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:50.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.554+0000 7f4812d05f40 1 -- 172.21.15.38:0/29471796 shutdown_connections 2024-09-21T03:42:50.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.554+0000 7f4812d05f40 1 -- 172.21.15.38:0/29471796 wait complete. 2024-09-21T03:42:50.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.555+0000 7f4812d05f40 1 Processor -- start 2024-09-21T03:42:50.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.555+0000 7f4812d05f40 1 -- start start 2024-09-21T03:42:50.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.556+0000 7f4812d05f40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557e54610d00 0x557e546553b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:50.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.556+0000 7f4812d05f40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x557e5463ce30 0x557e546b6be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:50.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.556+0000 7f4812d05f40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557e546cb950 0x557e546b7120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:50.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.556+0000 7f4812d05f40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x557e546e30f0 con 0x557e54610d00 2024-09-21T03:42:50.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.556+0000 7f4812d05f40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x557e546c7200 con 0x557e546cb950 2024-09-21T03:42:50.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.556+0000 7f4812d05f40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x557e546b7700 con 0x557e5463ce30 2024-09-21T03:42:50.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.556+0000 7f480c7bf640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557e54610d00 0x557e546553b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:50.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.556+0000 7f480c7bf640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557e54610d00 0x557e546553b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38478/0 (socket says 172.21.15.38:38478) 2024-09-21T03:42:50.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.556+0000 7f480c7bf640 1 -- 172.21.15.38:0/240085233 learned_addr learned my addr 172.21.15.38:0/240085233 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:42:50.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.556+0000 7f480cfc0640 1 --2- 172.21.15.38:0/240085233 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557e546cb950 0x557e546b7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:50.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.556+0000 7f480bfbe640 1 --2- 172.21.15.38:0/240085233 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x557e5463ce30 0x557e546b6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:50.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.557+0000 7f480c7bf640 1 -- 172.21.15.38:0/240085233 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x557e5463ce30 msgr2=0x557e546b6be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:50.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.557+0000 7f480c7bf640 1 --2- 172.21.15.38:0/240085233 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x557e5463ce30 0x557e546b6be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:50.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.557+0000 7f480c7bf640 1 -- 172.21.15.38:0/240085233 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557e546cb950 msgr2=0x557e546b7120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:50.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.557+0000 7f480c7bf640 1 --2- 172.21.15.38:0/240085233 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557e546cb950 0x557e546b7120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:50.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.557+0000 7f480c7bf640 1 -- 172.21.15.38:0/240085233 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e546b7950 con 0x557e54610d00 2024-09-21T03:42:50.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.557+0000 7f480c7bf640 1 --2- 172.21.15.38:0/240085233 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557e54610d00 0x557e546553b0 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7f47fc00c1d0 tx=0x7f47fc00c690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:50.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.557+0000 7f4809fba640 1 -- 172.21.15.38:0/240085233 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47fc015020 con 0x557e54610d00 2024-09-21T03:42:50.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.557+0000 7f4809fba640 1 -- 172.21.15.38:0/240085233 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f47fc0118f0 con 0x557e54610d00 2024-09-21T03:42:50.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.557+0000 7f4812d05f40 1 -- 172.21.15.38:0/240085233 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x557e5477d6b0 con 0x557e54610d00 2024-09-21T03:42:50.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.557+0000 7f4809fba640 1 -- 172.21.15.38:0/240085233 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47fc00ce60 con 0x557e54610d00 2024-09-21T03:42:50.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.557+0000 7f4812d05f40 1 -- 172.21.15.38:0/240085233 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e5477dc70 con 0x557e54610d00 2024-09-21T03:42:50.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.559+0000 7f4809fba640 1 -- 172.21.15.38:0/240085233 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f47fc018020 con 0x557e54610d00 2024-09-21T03:42:50.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.559+0000 7f4809fba640 1 --2- 172.21.15.38:0/240085233 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f47ec0775a0 0x7f47ec079ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:50.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.559+0000 7f4809fba640 1 -- 172.21.15.38:0/240085233 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 8540+0+0 (secure 0 0 0) 0x7f47fc09a630 con 0x557e54610d00 2024-09-21T03:42:50.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.560+0000 7f480bfbe640 1 --2- 172.21.15.38:0/240085233 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f47ec0775a0 0x7f47ec079ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:50.563 INFO:teuthology.orchestra.run.smithi038.stdout:Importing pool 2024-09-21T03:42:50.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.560+0000 7f4812d05f40 1 --2- 172.21.15.38:0/240085233 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x557e5477ea90 0x557e5477eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:50.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.560+0000 7f4812d05f40 1 -- 172.21.15.38:0/240085233 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x557e5477f3e0 con 0x557e5477ea90 2024-09-21T03:42:50.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.560+0000 7f480cfc0640 1 --2- 172.21.15.38:0/240085233 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x557e5477ea90 0x557e5477eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:50.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.560+0000 7f480bfbe640 1 --2- 172.21.15.38:0/240085233 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f47ec0775a0 0x7f47ec079ab0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f47f4055000 tx=0x7f47f4058040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:50.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.560+0000 7f480cfc0640 1 --2- 172.21.15.38:0/240085233 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x557e5477ea90 0x557e5477eea0 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:50.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.563+0000 7f480cfc0640 1 -- 172.21.15.38:0/240085233 <== osd.7 v2:172.21.15.195:6818/1606106342 1 ==== osd_op_reply(1 601.00000000 [create] v122'784 uv758 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f480400d890 con 0x557e5477ea90 2024-09-21T03:42:50.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.563+0000 7f4812d05f40 1 -- 172.21.15.38:0/240085233 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e122) -- 0x557e5477f7d0 con 0x557e5477ea90 2024-09-21T03:42:50.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.564+0000 7f480cfc0640 1 -- 172.21.15.38:0/240085233 <== osd.7 v2:172.21.15.195:6818/1606106342 2 ==== osd_op_reply(2 601.00000000 [delete] v122'785 uv785 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f480400d890 con 0x557e5477ea90 2024-09-21T03:42:50.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.564+0000 7f4812d05f40 1 -- 172.21.15.38:0/240085233 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x557e5477fba0 con 0x557e5477ea90 2024-09-21T03:42:50.567 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-09-21T03:42:50.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.566+0000 7f480cfc0640 1 -- 172.21.15.38:0/240085233 <== osd.7 v2:172.21.15.195:6818/1606106342 3 ==== osd_op_reply(3 601.00000000 [create] v122'786 uv786 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f480400d890 con 0x557e5477ea90 2024-09-21T03:42:50.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.566+0000 7f4812d05f40 1 -- 172.21.15.38:0/240085233 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x557e5477ff40 con 0x557e5477ea90 2024-09-21T03:42:50.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.567+0000 7f480cfc0640 1 -- 172.21.15.38:0/240085233 <== osd.7 v2:172.21.15.195:6818/1606106342 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v122'787 uv787 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f480400d890 con 0x557e5477ea90 2024-09-21T03:42:50.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.567+0000 7f4812d05f40 1 -- 172.21.15.38:0/240085233 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x557e54780340 con 0x557e5477ea90 2024-09-21T03:42:50.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.569+0000 7f480cfc0640 1 -- 172.21.15.38:0/240085233 <== osd.7 v2:172.21.15.195:6818/1606106342 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v122'788 uv788 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f480400d890 con 0x557e5477ea90 2024-09-21T03:42:50.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.569+0000 7f4812d05f40 1 -- 172.21.15.38:0/240085233 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x557e54780720 con 0x557e5477ea90 2024-09-21T03:42:50.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.571+0000 7f480cfc0640 1 -- 172.21.15.38:0/240085233 <== osd.7 v2:172.21.15.195:6818/1606106342 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v122'789 uv789 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f480400d890 con 0x557e5477ea90 2024-09-21T03:42:50.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.571+0000 7f4812d05f40 1 -- 172.21.15.38:0/240085233 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x557e5478abf0 con 0x557e5477ea90 2024-09-21T03:42:50.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.572+0000 7f480cfc0640 1 -- 172.21.15.38:0/240085233 <== osd.7 v2:172.21.15.195:6818/1606106342 7 ==== osd_op_reply(7 602.00000000 [create] v122'790 uv764 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f480400d890 con 0x557e5477ea90 2024-09-21T03:42:50.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.572+0000 7f4812d05f40 1 -- 172.21.15.38:0/240085233 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e122) -- 0x557e5478af90 con 0x557e5477ea90 2024-09-21T03:42:50.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.574+0000 7f480cfc0640 1 -- 172.21.15.38:0/240085233 <== osd.7 v2:172.21.15.195:6818/1606106342 8 ==== osd_op_reply(8 602.00000000 [delete] v122'791 uv791 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f480400d890 con 0x557e5477ea90 2024-09-21T03:42:50.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.574+0000 7f4812d05f40 1 -- 172.21.15.38:0/240085233 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x557e5478af90 con 0x557e5477ea90 2024-09-21T03:42:50.578 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-09-21T03:42:50.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.576+0000 7f480cfc0640 1 -- 172.21.15.38:0/240085233 <== osd.7 v2:172.21.15.195:6818/1606106342 9 ==== osd_op_reply(9 602.00000000 [create] v122'792 uv792 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f480400d890 con 0x557e5477ea90 2024-09-21T03:42:50.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.576+0000 7f4812d05f40 1 -- 172.21.15.38:0/240085233 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x557e5478bfa0 con 0x557e5477ea90 2024-09-21T03:42:50.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.577+0000 7f480cfc0640 1 -- 172.21.15.38:0/240085233 <== osd.7 v2:172.21.15.195:6818/1606106342 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v122'793 uv793 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f480400d890 con 0x557e5477ea90 2024-09-21T03:42:50.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.578+0000 7f4812d05f40 1 -- 172.21.15.38:0/240085233 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x557e5478bfa0 con 0x557e5477ea90 2024-09-21T03:42:50.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.579+0000 7f480cfc0640 1 -- 172.21.15.38:0/240085233 <== osd.7 v2:172.21.15.195:6818/1606106342 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v122'794 uv794 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f480400d890 con 0x557e5477ea90 2024-09-21T03:42:50.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.579+0000 7f4812d05f40 1 -- 172.21.15.38:0/240085233 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x557e5478af90 con 0x557e5477ea90 2024-09-21T03:42:50.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.581+0000 7f480cfc0640 1 -- 172.21.15.38:0/240085233 <== osd.7 v2:172.21.15.195:6818/1606106342 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v122'795 uv795 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f480400d890 con 0x557e5477ea90 2024-09-21T03:42:50.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.581+0000 7f4812d05f40 1 -- 172.21.15.38:0/240085233 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:13 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x557e5478af90 con 0x557e5477ea90 2024-09-21T03:42:50.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.582+0000 7f480cfc0640 1 -- 172.21.15.38:0/240085233 <== osd.7 v2:172.21.15.195:6818/1606106342 13 ==== osd_op_reply(13 10000000000.00000000 [create] v122'796 uv771 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7f480400d890 con 0x557e5477ea90 2024-09-21T03:42:50.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.582+0000 7f4812d05f40 1 -- 172.21.15.38:0/240085233 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:14 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e122) -- 0x557e5478af90 con 0x557e5477ea90 2024-09-21T03:42:50.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.584+0000 7f480cfc0640 1 -- 172.21.15.38:0/240085233 <== osd.7 v2:172.21.15.195:6818/1606106342 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v122'797 uv797 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f480400d890 con 0x557e5477ea90 2024-09-21T03:42:50.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.584+0000 7f4812d05f40 1 -- 172.21.15.38:0/240085233 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:15 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x557e5478af90 con 0x557e5477ea90 2024-09-21T03:42:50.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.586+0000 7f480cfc0640 1 -- 172.21.15.38:0/240085233 <== osd.7 v2:172.21.15.195:6818/1606106342 15 ==== osd_op_reply(15 10000000000.00000000 [create] v122'798 uv798 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f480400d890 con 0x557e5477ea90 2024-09-21T03:42:50.588 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-09-21T03:42:50.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.586+0000 7f4812d05f40 1 -- 172.21.15.38:0/240085233 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:16 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x557e5478bfa0 con 0x557e5477ea90 2024-09-21T03:42:50.589 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.587+0000 7f480cfc0640 1 -- 172.21.15.38:0/240085233 <== osd.7 v2:172.21.15.195:6818/1606106342 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v122'799 uv799 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f480400d890 con 0x557e5477ea90 2024-09-21T03:42:50.589 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.588+0000 7f4812d05f40 1 -- 172.21.15.38:0/240085233 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:17 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x557e5478bfa0 con 0x557e5477ea90 2024-09-21T03:42:50.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.589+0000 7f480cfc0640 1 -- 172.21.15.38:0/240085233 <== osd.7 v2:172.21.15.195:6818/1606106342 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v122'800 uv800 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f480400d890 con 0x557e5477ea90 2024-09-21T03:42:50.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.589+0000 7f4812d05f40 1 -- 172.21.15.38:0/240085233 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:18 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x557e5478af90 con 0x557e5477ea90 2024-09-21T03:42:50.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.591+0000 7f480cfc0640 1 -- 172.21.15.38:0/240085233 <== osd.7 v2:172.21.15.195:6818/1606106342 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v122'801 uv801 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f480400d890 con 0x557e5477ea90 2024-09-21T03:42:50.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.591+0000 7f4812d05f40 1 -- 172.21.15.38:0/240085233 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:19 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=585b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x557e5478b440 con 0x557e5477ea90 2024-09-21T03:42:50.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.592+0000 7f480cfc0640 1 -- 172.21.15.38:0/240085233 <== osd.7 v2:172.21.15.195:6818/1606106342 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v122'802 uv802 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f480400d890 con 0x557e5477ea90 2024-09-21T03:42:50.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.592+0000 7f4812d05f40 1 -- 172.21.15.38:0/240085233 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:20 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x557e5478b440 con 0x557e5477ea90 2024-09-21T03:42:50.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.594+0000 7f480cfc0640 1 -- 172.21.15.38:0/240085233 <== osd.7 v2:172.21.15.195:6818/1606106342 20 ==== osd_op_reply(20 600.00000000 [create] v122'803 uv777 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f480400d890 con 0x557e5477ea90 2024-09-21T03:42:50.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.594+0000 7f4812d05f40 1 -- 172.21.15.38:0/240085233 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:21 13.10 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e122) -- 0x557e5478b440 con 0x557e5477ea90 2024-09-21T03:42:50.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.596+0000 7f480cfc0640 1 -- 172.21.15.38:0/240085233 <== osd.7 v2:172.21.15.195:6818/1606106342 21 ==== osd_op_reply(21 600.00000000 [delete] v122'804 uv804 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f480400d890 con 0x557e5477ea90 2024-09-21T03:42:50.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.596+0000 7f4812d05f40 1 -- 172.21.15.38:0/240085233 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:22 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x557e5478b440 con 0x557e5477ea90 2024-09-21T03:42:50.599 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-09-21T03:42:50.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.597+0000 7f480cfc0640 1 -- 172.21.15.38:0/240085233 <== osd.7 v2:172.21.15.195:6818/1606106342 22 ==== osd_op_reply(22 600.00000000 [create] v122'805 uv805 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f480400d890 con 0x557e5477ea90 2024-09-21T03:42:50.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.597+0000 7f4812d05f40 1 -- 172.21.15.38:0/240085233 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:23 13.10 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x557e5478c450 con 0x557e5477ea90 2024-09-21T03:42:50.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.599+0000 7f480cfc0640 1 -- 172.21.15.38:0/240085233 <== osd.7 v2:172.21.15.195:6818/1606106342 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v122'806 uv806 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f480400d890 con 0x557e5477ea90 2024-09-21T03:42:50.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.599+0000 7f4812d05f40 1 -- 172.21.15.38:0/240085233 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:24 13.10 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x557e5478c450 con 0x557e5477ea90 2024-09-21T03:42:50.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.600+0000 7f480cfc0640 1 -- 172.21.15.38:0/240085233 <== osd.7 v2:172.21.15.195:6818/1606106342 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v122'807 uv807 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f480400d890 con 0x557e5477ea90 2024-09-21T03:42:50.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.600+0000 7f4812d05f40 1 -- 172.21.15.38:0/240085233 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:25 13.10 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x557e5478b440 con 0x557e5477ea90 2024-09-21T03:42:50.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.602+0000 7f480cfc0640 1 -- 172.21.15.38:0/240085233 <== osd.7 v2:172.21.15.195:6818/1606106342 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v122'808 uv808 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f480400d890 con 0x557e5477ea90 2024-09-21T03:42:50.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.602+0000 7f4812d05f40 1 -- 172.21.15.38:0/240085233 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:26 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x557e5478b440 con 0x557e5477ea90 2024-09-21T03:42:50.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.604+0000 7f480cfc0640 1 -- 172.21.15.38:0/240085233 <== osd.7 v2:172.21.15.195:6818/1606106342 26 ==== osd_op_reply(26 603.00000000 [create] v122'809 uv783 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f480400d890 con 0x557e5477ea90 2024-09-21T03:42:50.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.604+0000 7f4812d05f40 1 -- 172.21.15.38:0/240085233 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:27 13.10 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e122) -- 0x557e5478b440 con 0x557e5477ea90 2024-09-21T03:42:50.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.606+0000 7f480cfc0640 1 -- 172.21.15.38:0/240085233 <== osd.7 v2:172.21.15.195:6818/1606106342 27 ==== osd_op_reply(27 603.00000000 [delete] v122'810 uv810 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f480400d890 con 0x557e5477ea90 2024-09-21T03:42:50.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.606+0000 7f4812d05f40 1 -- 172.21.15.38:0/240085233 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:28 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x557e5478b440 con 0x557e5477ea90 2024-09-21T03:42:50.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.607+0000 7f480cfc0640 1 -- 172.21.15.38:0/240085233 <== osd.7 v2:172.21.15.195:6818/1606106342 28 ==== osd_op_reply(28 603.00000000 [create] v122'811 uv811 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f480400d890 con 0x557e5477ea90 2024-09-21T03:42:50.609 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-09-21T03:42:50.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.607+0000 7f4812d05f40 1 -- 172.21.15.38:0/240085233 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:29 13.10 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x557e5478c450 con 0x557e5477ea90 2024-09-21T03:42:50.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.609+0000 7f480cfc0640 1 -- 172.21.15.38:0/240085233 <== osd.7 v2:172.21.15.195:6818/1606106342 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v122'812 uv812 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f480400d890 con 0x557e5477ea90 2024-09-21T03:42:50.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.609+0000 7f4812d05f40 1 -- 172.21.15.38:0/240085233 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:30 13.10 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x557e5478c450 con 0x557e5477ea90 2024-09-21T03:42:50.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.610+0000 7f480cfc0640 1 -- 172.21.15.38:0/240085233 <== osd.7 v2:172.21.15.195:6818/1606106342 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v122'813 uv813 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f480400d890 con 0x557e5477ea90 2024-09-21T03:42:50.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.610+0000 7f4812d05f40 1 -- 172.21.15.38:0/240085233 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:31 13.10 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x557e5478b440 con 0x557e5477ea90 2024-09-21T03:42:50.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.612+0000 7f480cfc0640 1 -- 172.21.15.38:0/240085233 <== osd.7 v2:172.21.15.195:6818/1606106342 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v122'814 uv814 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f480400d890 con 0x557e5477ea90 2024-09-21T03:42:50.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.612+0000 7f4812d05f40 1 --2- 172.21.15.38:0/240085233 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x557e5478b950 0x557e5478dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:50.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.612+0000 7f4812d05f40 1 -- 172.21.15.38:0/240085233 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x557e5478e2b0 con 0x557e5478b950 2024-09-21T03:42:50.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.612+0000 7f480c7bf640 1 --2- 172.21.15.38:0/240085233 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x557e5478b950 0x557e5478dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:50.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.613+0000 7f480c7bf640 1 --2- 172.21.15.38:0/240085233 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x557e5478b950 0x557e5478dd70 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:50.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.614+0000 7f480c7bf640 1 -- 172.21.15.38:0/240085233 <== osd.6 v2:172.21.15.195:6826/4055825675 1 ==== osd_op_reply(32 1.00000000.inode [create] v122'570 uv552 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f47fc021070 con 0x557e5478b950 2024-09-21T03:42:50.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.614+0000 7f4812d05f40 1 -- 172.21.15.38:0/240085233 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e122) -- 0x557e5478e650 con 0x557e5478b950 2024-09-21T03:42:50.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.616+0000 7f480c7bf640 1 -- 172.21.15.38:0/240085233 <== osd.6 v2:172.21.15.195:6826/4055825675 2 ==== osd_op_reply(33 1.00000000.inode [delete] v122'571 uv571 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f47fc021070 con 0x557e5478b950 2024-09-21T03:42:50.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.616+0000 7f4812d05f40 1 -- 172.21.15.38:0/240085233 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x557e5478e9f0 con 0x557e5478b950 2024-09-21T03:42:50.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.617+0000 7f480c7bf640 1 -- 172.21.15.38:0/240085233 <== osd.6 v2:172.21.15.195:6826/4055825675 3 ==== osd_op_reply(34 1.00000000.inode [create] v122'572 uv572 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f47fc021070 con 0x557e5478b950 2024-09-21T03:42:50.619 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-09-21T03:42:50.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.617+0000 7f4812d05f40 1 -- 172.21.15.38:0/240085233 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x557e5478f240 con 0x557e5478b950 2024-09-21T03:42:50.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.619+0000 7f480c7bf640 1 -- 172.21.15.38:0/240085233 <== osd.6 v2:172.21.15.195:6826/4055825675 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v122'573 uv573 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f47fc021070 con 0x557e5478b950 2024-09-21T03:42:50.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.619+0000 7f4812d05f40 1 -- 172.21.15.38:0/240085233 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x557e5478f5e0 con 0x557e5478b950 2024-09-21T03:42:50.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.620+0000 7f480c7bf640 1 -- 172.21.15.38:0/240085233 <== osd.6 v2:172.21.15.195:6826/4055825675 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v122'574 uv574 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f47fc021070 con 0x557e5478b950 2024-09-21T03:42:50.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.620+0000 7f4812d05f40 1 -- 172.21.15.38:0/240085233 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x557e5478f980 con 0x557e5478b950 2024-09-21T03:42:50.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.622+0000 7f480c7bf640 1 -- 172.21.15.38:0/240085233 <== osd.6 v2:172.21.15.195:6826/4055825675 6 ==== osd_op_reply(37 200.00000000 [create] v122'575 uv557 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f47fc021070 con 0x557e5478b950 2024-09-21T03:42:50.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.622+0000 7f4812d05f40 1 -- 172.21.15.38:0/240085233 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e122) -- 0x557e5478fd20 con 0x557e5478b950 2024-09-21T03:42:50.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.623+0000 7f480c7bf640 1 -- 172.21.15.38:0/240085233 <== osd.6 v2:172.21.15.195:6826/4055825675 7 ==== osd_op_reply(38 200.00000000 [delete] v122'576 uv576 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f47fc021070 con 0x557e5478b950 2024-09-21T03:42:50.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.623+0000 7f4812d05f40 1 -- 172.21.15.38:0/240085233 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x557e5478fd20 con 0x557e5478b950 2024-09-21T03:42:50.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.624+0000 7f480c7bf640 1 -- 172.21.15.38:0/240085233 <== osd.6 v2:172.21.15.195:6826/4055825675 8 ==== osd_op_reply(39 200.00000000 [create] v122'577 uv577 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f47fc021070 con 0x557e5478b950 2024-09-21T03:42:50.626 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-09-21T03:42:50.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.624+0000 7f4812d05f40 1 -- 172.21.15.38:0/240085233 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x557e5478ee90 con 0x557e5478b950 2024-09-21T03:42:50.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.626+0000 7f480c7bf640 1 -- 172.21.15.38:0/240085233 <== osd.6 v2:172.21.15.195:6826/4055825675 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v122'578 uv578 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f47fc021070 con 0x557e5478b950 2024-09-21T03:42:50.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.626+0000 7f4812d05f40 1 -- 172.21.15.38:0/240085233 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x557e547901d0 con 0x557e5478b950 2024-09-21T03:42:50.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.627+0000 7f480c7bf640 1 -- 172.21.15.38:0/240085233 <== osd.6 v2:172.21.15.195:6826/4055825675 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v122'579 uv579 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f47fc021070 con 0x557e5478b950 2024-09-21T03:42:50.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.627+0000 7f4812d05f40 1 -- 172.21.15.38:0/240085233 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:42 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x557e547901d0 con 0x557e5478b950 2024-09-21T03:42:50.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.629+0000 7f480c7bf640 1 -- 172.21.15.38:0/240085233 <== osd.6 v2:172.21.15.195:6826/4055825675 11 ==== osd_op_reply(42 200.00000001 [create] v122'580 uv569 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f47fc021070 con 0x557e5478b950 2024-09-21T03:42:50.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.629+0000 7f4812d05f40 1 -- 172.21.15.38:0/240085233 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:43 13.14 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e122) -- 0x557e547901d0 con 0x557e5478b950 2024-09-21T03:42:50.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.630+0000 7f480c7bf640 1 -- 172.21.15.38:0/240085233 <== osd.6 v2:172.21.15.195:6826/4055825675 12 ==== osd_op_reply(43 200.00000001 [delete] v122'581 uv581 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f47fc021070 con 0x557e5478b950 2024-09-21T03:42:50.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.630+0000 7f4812d05f40 1 -- 172.21.15.38:0/240085233 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:44 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x557e547901d0 con 0x557e5478b950 2024-09-21T03:42:50.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.632+0000 7f480c7bf640 1 -- 172.21.15.38:0/240085233 <== osd.6 v2:172.21.15.195:6826/4055825675 13 ==== osd_op_reply(44 200.00000001 [create] v122'582 uv582 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f47fc021070 con 0x557e5478b950 2024-09-21T03:42:50.633 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-09-21T03:42:50.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.632+0000 7f4812d05f40 1 -- 172.21.15.38:0/240085233 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:45 13.14 13:2e2fa760:::200.00000001:head [write 0~17490 in=17490b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x557e54794650 con 0x557e5478b950 2024-09-21T03:42:50.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.633+0000 7f480c7bf640 1 -- 172.21.15.38:0/240085233 <== osd.6 v2:172.21.15.195:6826/4055825675 14 ==== osd_op_reply(45 200.00000001 [write 0~17490] v122'583 uv583 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f47fc021070 con 0x557e5478b950 2024-09-21T03:42:50.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.633+0000 7f4812d05f40 1 -- 172.21.15.38:0/240085233 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:46 13.14 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x557e54794650 con 0x557e5478b950 2024-09-21T03:42:50.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.635+0000 7f480c7bf640 1 -- 172.21.15.38:0/240085233 <== osd.6 v2:172.21.15.195:6826/4055825675 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v122'584 uv584 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f47fc021070 con 0x557e5478b950 2024-09-21T03:42:50.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.635+0000 7f4812d05f40 1 -- 172.21.15.38:0/240085233 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:47 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x557e547901d0 con 0x557e5478b950 2024-09-21T03:42:50.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.636+0000 7f480c7bf640 1 -- 172.21.15.38:0/240085233 <== osd.6 v2:172.21.15.195:6826/4055825675 16 ==== osd_op_reply(47 606.00000000 [create] v122'585 uv568 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f47fc021070 con 0x557e5478b950 2024-09-21T03:42:50.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.636+0000 7f4812d05f40 1 -- 172.21.15.38:0/240085233 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:48 13.14 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e122) -- 0x557e547901d0 con 0x557e5478b950 2024-09-21T03:42:50.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.637+0000 7f480c7bf640 1 -- 172.21.15.38:0/240085233 <== osd.6 v2:172.21.15.195:6826/4055825675 17 ==== osd_op_reply(48 606.00000000 [delete] v122'586 uv586 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f47fc021070 con 0x557e5478b950 2024-09-21T03:42:50.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.637+0000 7f4812d05f40 1 -- 172.21.15.38:0/240085233 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:49 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x557e547901d0 con 0x557e5478b950 2024-09-21T03:42:50.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.639+0000 7f480c7bf640 1 -- 172.21.15.38:0/240085233 <== osd.6 v2:172.21.15.195:6826/4055825675 18 ==== osd_op_reply(49 606.00000000 [create] v122'587 uv587 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f47fc021070 con 0x557e5478b950 2024-09-21T03:42:50.641 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-09-21T03:42:50.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.639+0000 7f4812d05f40 1 -- 172.21.15.38:0/240085233 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:50 13.14 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x557e547911e0 con 0x557e5478b950 2024-09-21T03:42:50.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.640+0000 7f480c7bf640 1 -- 172.21.15.38:0/240085233 <== osd.6 v2:172.21.15.195:6826/4055825675 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v122'588 uv588 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f47fc021070 con 0x557e5478b950 2024-09-21T03:42:50.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.640+0000 7f4812d05f40 1 -- 172.21.15.38:0/240085233 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:51 13.14 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x557e547911e0 con 0x557e5478b950 2024-09-21T03:42:50.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.641+0000 7f480c7bf640 1 -- 172.21.15.38:0/240085233 <== osd.6 v2:172.21.15.195:6826/4055825675 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v122'589 uv589 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f47fc021070 con 0x557e5478b950 2024-09-21T03:42:50.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.641+0000 7f4812d05f40 1 -- 172.21.15.38:0/240085233 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:52 13.14 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x557e547901d0 con 0x557e5478b950 2024-09-21T03:42:50.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.642+0000 7f480c7bf640 1 -- 172.21.15.38:0/240085233 <== osd.6 v2:172.21.15.195:6826/4055825675 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v122'590 uv590 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f47fc021070 con 0x557e5478b950 2024-09-21T03:42:50.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.642+0000 7f4812d05f40 1 --2- 172.21.15.38:0/240085233 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x557e54790ad0 0x557e54792ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:50.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.642+0000 7f4812d05f40 1 -- 172.21.15.38:0/240085233 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:53 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x557e54793430 con 0x557e54790ad0 2024-09-21T03:42:50.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.642+0000 7f480bfbe640 1 --2- 172.21.15.38:0/240085233 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x557e54790ad0 0x557e54792ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:50.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.643+0000 7f480bfbe640 1 --2- 172.21.15.38:0/240085233 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x557e54790ad0 0x557e54792ef0 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:50.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.644+0000 7f480bfbe640 1 -- 172.21.15.38:0/240085233 <== osd.0 v2:172.21.15.38:6800/2381639433 1 ==== osd_op_reply(53 607.00000000 [create] v122'153 uv152 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f47f4063480 con 0x557e54790ad0 2024-09-21T03:42:50.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.644+0000 7f4812d05f40 1 -- 172.21.15.38:0/240085233 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:54 13.1c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e122) -- 0x557e547937d0 con 0x557e54790ad0 2024-09-21T03:42:50.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.646+0000 7f480bfbe640 1 -- 172.21.15.38:0/240085233 <== osd.0 v2:172.21.15.38:6800/2381639433 2 ==== osd_op_reply(54 607.00000000 [delete] v122'154 uv154 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f47f4063480 con 0x557e54790ad0 2024-09-21T03:42:50.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.646+0000 7f4812d05f40 1 -- 172.21.15.38:0/240085233 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:55 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x557e54793cb0 con 0x557e54790ad0 2024-09-21T03:42:50.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.647+0000 7f480bfbe640 1 -- 172.21.15.38:0/240085233 <== osd.0 v2:172.21.15.38:6800/2381639433 3 ==== osd_op_reply(55 607.00000000 [create] v122'155 uv155 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f47f4063480 con 0x557e54790ad0 2024-09-21T03:42:50.649 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-09-21T03:42:50.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.647+0000 7f4812d05f40 1 -- 172.21.15.38:0/240085233 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:56 13.1c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x557e54795560 con 0x557e54790ad0 2024-09-21T03:42:50.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.648+0000 7f480bfbe640 1 -- 172.21.15.38:0/240085233 <== osd.0 v2:172.21.15.38:6800/2381639433 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v122'156 uv156 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f47f4063480 con 0x557e54790ad0 2024-09-21T03:42:50.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.648+0000 7f4812d05f40 1 -- 172.21.15.38:0/240085233 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:57 13.1c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x557e54795900 con 0x557e54790ad0 2024-09-21T03:42:50.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.649+0000 7f480bfbe640 1 -- 172.21.15.38:0/240085233 <== osd.0 v2:172.21.15.38:6800/2381639433 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v122'157 uv157 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f47f4063480 con 0x557e54790ad0 2024-09-21T03:42:50.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.649+0000 7f4812d05f40 1 -- 172.21.15.38:0/240085233 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:58 13.1c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x557e54794550 con 0x557e54790ad0 2024-09-21T03:42:50.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.650+0000 7f480bfbe640 1 -- 172.21.15.38:0/240085233 <== osd.0 v2:172.21.15.38:6800/2381639433 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v122'158 uv158 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f47f4063480 con 0x557e54790ad0 2024-09-21T03:42:50.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.650+0000 7f4812d05f40 1 --2- 172.21.15.38:0/240085233 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x557e54794e50 0x557e54797cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:50.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.650+0000 7f4812d05f40 1 -- 172.21.15.38:0/240085233 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:59 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x557e547981f0 con 0x557e54794e50 2024-09-21T03:42:50.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.650+0000 7f480cfc0640 1 --2- 172.21.15.38:0/240085233 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x557e54794e50 0x557e54797cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:50.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.650+0000 7f480cfc0640 1 --2- 172.21.15.38:0/240085233 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x557e54794e50 0x557e54797cb0 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:50.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.652+0000 7f480cfc0640 1 -- 172.21.15.38:0/240085233 <== osd.3 v2:172.21.15.38:6808/1286810294 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v122'110 uv109 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f480400d890 con 0x557e54794e50 2024-09-21T03:42:50.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.652+0000 7f4812d05f40 1 -- 172.21.15.38:0/240085233 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:60 13.6 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e122) -- 0x557e547981f0 con 0x557e54794e50 2024-09-21T03:42:50.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.653+0000 7f480cfc0640 1 -- 172.21.15.38:0/240085233 <== osd.3 v2:172.21.15.38:6808/1286810294 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v122'111 uv111 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f480400d890 con 0x557e54794e50 2024-09-21T03:42:50.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.653+0000 7f4812d05f40 1 -- 172.21.15.38:0/240085233 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:61 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x557e547985e0 con 0x557e54794e50 2024-09-21T03:42:50.656 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-09-21T03:42:50.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.654+0000 7f480cfc0640 1 -- 172.21.15.38:0/240085233 <== osd.3 v2:172.21.15.38:6808/1286810294 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v122'112 uv112 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f480400d890 con 0x557e54794e50 2024-09-21T03:42:50.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.654+0000 7f4812d05f40 1 -- 172.21.15.38:0/240085233 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:62 13.6 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x557e547985e0 con 0x557e54794e50 2024-09-21T03:42:50.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.655+0000 7f480cfc0640 1 -- 172.21.15.38:0/240085233 <== osd.3 v2:172.21.15.38:6808/1286810294 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v122'113 uv113 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f480400d890 con 0x557e54794e50 2024-09-21T03:42:50.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.655+0000 7f4812d05f40 1 -- 172.21.15.38:0/240085233 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x557e547985e0 con 0x557e54790ad0 2024-09-21T03:42:50.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.657+0000 7f480bfbe640 1 -- 172.21.15.38:0/240085233 <== osd.0 v2:172.21.15.38:6800/2381639433 7 ==== osd_op_reply(63 608.00000000 [create] v122'153 uv152 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f47f4063480 con 0x557e54790ad0 2024-09-21T03:42:50.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.657+0000 7f4812d05f40 1 -- 172.21.15.38:0/240085233 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e122) -- 0x557e547985e0 con 0x557e54790ad0 2024-09-21T03:42:50.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.658+0000 7f480bfbe640 1 -- 172.21.15.38:0/240085233 <== osd.0 v2:172.21.15.38:6800/2381639433 8 ==== osd_op_reply(64 608.00000000 [delete] v122'154 uv154 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f47f4063480 con 0x557e54790ad0 2024-09-21T03:42:50.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.658+0000 7f4812d05f40 1 -- 172.21.15.38:0/240085233 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x557e547985e0 con 0x557e54790ad0 2024-09-21T03:42:50.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.659+0000 7f480bfbe640 1 -- 172.21.15.38:0/240085233 <== osd.0 v2:172.21.15.38:6800/2381639433 9 ==== osd_op_reply(65 608.00000000 [create] v122'155 uv155 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f47f4063480 con 0x557e54790ad0 2024-09-21T03:42:50.661 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-09-21T03:42:50.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.659+0000 7f4812d05f40 1 -- 172.21.15.38:0/240085233 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x557e547995f0 con 0x557e54790ad0 2024-09-21T03:42:50.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.660+0000 7f480bfbe640 1 -- 172.21.15.38:0/240085233 <== osd.0 v2:172.21.15.38:6800/2381639433 10 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v122'156 uv156 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f47f4063480 con 0x557e54790ad0 2024-09-21T03:42:50.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.660+0000 7f4812d05f40 1 -- 172.21.15.38:0/240085233 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x557e547995f0 con 0x557e54790ad0 2024-09-21T03:42:50.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.661+0000 7f480bfbe640 1 -- 172.21.15.38:0/240085233 <== osd.0 v2:172.21.15.38:6800/2381639433 11 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v122'157 uv157 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f47f4063480 con 0x557e54790ad0 2024-09-21T03:42:50.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.661+0000 7f4812d05f40 1 -- 172.21.15.38:0/240085233 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x557e547985e0 con 0x557e54790ad0 2024-09-21T03:42:50.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.663+0000 7f480bfbe640 1 -- 172.21.15.38:0/240085233 <== osd.0 v2:172.21.15.38:6800/2381639433 12 ==== osd_op_reply(68 608.00000000 [omap-set-header] v122'158 uv158 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f47f4063480 con 0x557e54790ad0 2024-09-21T03:42:50.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.663+0000 7f4812d05f40 1 -- 172.21.15.38:0/240085233 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x557e547985e0 con 0x557e54794e50 2024-09-21T03:42:50.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.664+0000 7f480cfc0640 1 -- 172.21.15.38:0/240085233 <== osd.3 v2:172.21.15.38:6808/1286810294 5 ==== osd_op_reply(69 604.00000000 [create] v122'282 uv276 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f480400d890 con 0x557e54794e50 2024-09-21T03:42:50.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.664+0000 7f4812d05f40 1 -- 172.21.15.38:0/240085233 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e122) -- 0x557e547985e0 con 0x557e54794e50 2024-09-21T03:42:50.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.665+0000 7f480cfc0640 1 -- 172.21.15.38:0/240085233 <== osd.3 v2:172.21.15.38:6808/1286810294 6 ==== osd_op_reply(70 604.00000000 [delete] v122'283 uv283 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f480400d890 con 0x557e54794e50 2024-09-21T03:42:50.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.665+0000 7f4812d05f40 1 -- 172.21.15.38:0/240085233 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x557e547985e0 con 0x557e54794e50 2024-09-21T03:42:50.668 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-09-21T03:42:50.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.667+0000 7f480cfc0640 1 -- 172.21.15.38:0/240085233 <== osd.3 v2:172.21.15.38:6808/1286810294 7 ==== osd_op_reply(71 604.00000000 [create] v122'284 uv284 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f480400d890 con 0x557e54794e50 2024-09-21T03:42:50.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.667+0000 7f4812d05f40 1 -- 172.21.15.38:0/240085233 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x557e547995f0 con 0x557e54794e50 2024-09-21T03:42:50.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.668+0000 7f480cfc0640 1 -- 172.21.15.38:0/240085233 <== osd.3 v2:172.21.15.38:6808/1286810294 8 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v122'285 uv285 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f480400d890 con 0x557e54794e50 2024-09-21T03:42:50.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.668+0000 7f4812d05f40 1 -- 172.21.15.38:0/240085233 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x557e547995f0 con 0x557e54794e50 2024-09-21T03:42:50.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.669+0000 7f480cfc0640 1 -- 172.21.15.38:0/240085233 <== osd.3 v2:172.21.15.38:6808/1286810294 9 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v122'286 uv286 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f480400d890 con 0x557e54794e50 2024-09-21T03:42:50.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.669+0000 7f4812d05f40 1 -- 172.21.15.38:0/240085233 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x557e547985e0 con 0x557e54794e50 2024-09-21T03:42:50.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.670+0000 7f480cfc0640 1 -- 172.21.15.38:0/240085233 <== osd.3 v2:172.21.15.38:6808/1286810294 10 ==== osd_op_reply(74 604.00000000 [omap-set-header] v122'287 uv287 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f480400d890 con 0x557e54794e50 2024-09-21T03:42:50.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.670+0000 7f4812d05f40 1 -- 172.21.15.38:0/240085233 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x557e547985e0 con 0x557e54794e50 2024-09-21T03:42:50.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.671+0000 7f480cfc0640 1 -- 172.21.15.38:0/240085233 <== osd.3 v2:172.21.15.38:6808/1286810294 11 ==== osd_op_reply(75 500.00000000 [create] v122'288 uv281 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f480400d890 con 0x557e54794e50 2024-09-21T03:42:50.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.671+0000 7f4812d05f40 1 -- 172.21.15.38:0/240085233 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e122) -- 0x557e547985e0 con 0x557e54794e50 2024-09-21T03:42:50.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.672+0000 7f480cfc0640 1 -- 172.21.15.38:0/240085233 <== osd.3 v2:172.21.15.38:6808/1286810294 12 ==== osd_op_reply(76 500.00000000 [delete] v122'289 uv289 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f480400d890 con 0x557e54794e50 2024-09-21T03:42:50.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.672+0000 7f4812d05f40 1 -- 172.21.15.38:0/240085233 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x557e547985e0 con 0x557e54794e50 2024-09-21T03:42:50.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.673+0000 7f480cfc0640 1 -- 172.21.15.38:0/240085233 <== osd.3 v2:172.21.15.38:6808/1286810294 13 ==== osd_op_reply(77 500.00000000 [create] v122'290 uv290 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f480400d890 con 0x557e54794e50 2024-09-21T03:42:50.675 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-09-21T03:42:50.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.673+0000 7f4812d05f40 1 -- 172.21.15.38:0/240085233 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x557e547985e0 con 0x557e54794e50 2024-09-21T03:42:50.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.674+0000 7f480cfc0640 1 -- 172.21.15.38:0/240085233 <== osd.3 v2:172.21.15.38:6808/1286810294 14 ==== osd_op_reply(78 500.00000000 [write 0~90] v122'291 uv291 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f480400d890 con 0x557e54794e50 2024-09-21T03:42:50.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.674+0000 7f4812d05f40 1 -- 172.21.15.38:0/240085233 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x557e547985e0 con 0x557e54794e50 2024-09-21T03:42:50.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.676+0000 7f480cfc0640 1 -- 172.21.15.38:0/240085233 <== osd.3 v2:172.21.15.38:6808/1286810294 15 ==== osd_op_reply(79 500.00000000 [omap-set-header] v122'292 uv292 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f480400d890 con 0x557e54794e50 2024-09-21T03:42:50.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.676+0000 7f4812d05f40 1 -- 172.21.15.38:0/240085233 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:80 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x557e547985e0 con 0x557e54790ad0 2024-09-21T03:42:50.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.677+0000 7f480bfbe640 1 -- 172.21.15.38:0/240085233 <== osd.0 v2:172.21.15.38:6800/2381639433 13 ==== osd_op_reply(80 mds_snaptable [create] v122'127 uv126 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f47f4063480 con 0x557e54790ad0 2024-09-21T03:42:50.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.677+0000 7f4812d05f40 1 -- 172.21.15.38:0/240085233 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:81 13.d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e122) -- 0x557e547985e0 con 0x557e54790ad0 2024-09-21T03:42:50.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.678+0000 7f480bfbe640 1 -- 172.21.15.38:0/240085233 <== osd.0 v2:172.21.15.38:6800/2381639433 14 ==== osd_op_reply(81 mds_snaptable [delete] v122'128 uv128 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f47f4063480 con 0x557e54790ad0 2024-09-21T03:42:50.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.678+0000 7f4812d05f40 1 -- 172.21.15.38:0/240085233 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:82 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x557e547985e0 con 0x557e54790ad0 2024-09-21T03:42:50.681 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-09-21T03:42:50.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.679+0000 7f480bfbe640 1 -- 172.21.15.38:0/240085233 <== osd.0 v2:172.21.15.38:6800/2381639433 15 ==== osd_op_reply(82 mds_snaptable [create] v122'129 uv129 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f47f4063480 con 0x557e54790ad0 2024-09-21T03:42:50.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.679+0000 7f4812d05f40 1 -- 172.21.15.38:0/240085233 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:83 13.d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x557e547985e0 con 0x557e54790ad0 2024-09-21T03:42:50.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.680+0000 7f480bfbe640 1 -- 172.21.15.38:0/240085233 <== osd.0 v2:172.21.15.38:6800/2381639433 16 ==== osd_op_reply(83 mds_snaptable [write 0~70] v122'130 uv130 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f47f4063480 con 0x557e54790ad0 2024-09-21T03:42:50.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.680+0000 7f4812d05f40 1 -- 172.21.15.38:0/240085233 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:84 13.d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x557e547985e0 con 0x557e54790ad0 2024-09-21T03:42:50.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.681+0000 7f480bfbe640 1 -- 172.21.15.38:0/240085233 <== osd.0 v2:172.21.15.38:6800/2381639433 17 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v122'131 uv131 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f47f4063480 con 0x557e54790ad0 2024-09-21T03:42:50.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.681+0000 7f4812d05f40 1 --2- 172.21.15.38:0/240085233 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x557e54798c60 0x557e5479b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:50.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.682+0000 7f4812d05f40 1 -- 172.21.15.38:0/240085233 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x557e5479b710 con 0x557e54798c60 2024-09-21T03:42:50.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.682+0000 7f480c7bf640 1 --2- 172.21.15.38:0/240085233 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x557e54798c60 0x557e5479b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:50.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.682+0000 7f480c7bf640 1 --2- 172.21.15.38:0/240085233 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x557e54798c60 0x557e5479b0e0 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:50.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.683+0000 7f480c7bf640 1 -- 172.21.15.38:0/240085233 <== osd.1 v2:172.21.15.38:6816/1491818732 1 ==== osd_op_reply(85 605.00000000 [create] v122'153 uv152 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f47fc021070 con 0x557e54798c60 2024-09-21T03:42:50.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.683+0000 7f4812d05f40 1 -- 172.21.15.38:0/240085233 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e122) -- 0x557e5479b7d0 con 0x557e54798c60 2024-09-21T03:42:50.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.684+0000 7f480c7bf640 1 -- 172.21.15.38:0/240085233 <== osd.1 v2:172.21.15.38:6816/1491818732 2 ==== osd_op_reply(86 605.00000000 [delete] v122'154 uv154 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f47fc021070 con 0x557e54798c60 2024-09-21T03:42:50.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.684+0000 7f4812d05f40 1 -- 172.21.15.38:0/240085233 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x557e5479bbc0 con 0x557e54798c60 2024-09-21T03:42:50.687 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-09-21T03:42:50.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.685+0000 7f480c7bf640 1 -- 172.21.15.38:0/240085233 <== osd.1 v2:172.21.15.38:6816/1491818732 3 ==== osd_op_reply(87 605.00000000 [create] v122'155 uv155 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f47fc021070 con 0x557e54798c60 2024-09-21T03:42:50.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.685+0000 7f4812d05f40 1 -- 172.21.15.38:0/240085233 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x557e5479cbd0 con 0x557e54798c60 2024-09-21T03:42:50.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.687+0000 7f480c7bf640 1 -- 172.21.15.38:0/240085233 <== osd.1 v2:172.21.15.38:6816/1491818732 4 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v122'156 uv156 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f47fc021070 con 0x557e54798c60 2024-09-21T03:42:50.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.687+0000 7f4812d05f40 1 -- 172.21.15.38:0/240085233 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x557e5479cbd0 con 0x557e54798c60 2024-09-21T03:42:50.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.688+0000 7f480c7bf640 1 -- 172.21.15.38:0/240085233 <== osd.1 v2:172.21.15.38:6816/1491818732 5 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v122'157 uv157 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f47fc021070 con 0x557e54798c60 2024-09-21T03:42:50.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.688+0000 7f4812d05f40 1 -- 172.21.15.38:0/240085233 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x557e5479bbc0 con 0x557e54798c60 2024-09-21T03:42:50.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.689+0000 7f480c7bf640 1 -- 172.21.15.38:0/240085233 <== osd.1 v2:172.21.15.38:6816/1491818732 6 ==== osd_op_reply(90 605.00000000 [omap-set-header] v122'158 uv158 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f47fc021070 con 0x557e54798c60 2024-09-21T03:42:50.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.689+0000 7f4812d05f40 1 -- 172.21.15.38:0/240085233 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x557e5479bbc0 con 0x557e54798c60 2024-09-21T03:42:50.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.690+0000 7f480c7bf640 1 -- 172.21.15.38:0/240085233 <== osd.1 v2:172.21.15.38:6816/1491818732 7 ==== osd_op_reply(91 mds0_inotable [create] v122'253 uv247 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f47fc021070 con 0x557e54798c60 2024-09-21T03:42:50.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.690+0000 7f4812d05f40 1 -- 172.21.15.38:0/240085233 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e122) -- 0x557e5479bbc0 con 0x557e54798c60 2024-09-21T03:42:50.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.691+0000 7f480c7bf640 1 -- 172.21.15.38:0/240085233 <== osd.1 v2:172.21.15.38:6816/1491818732 8 ==== osd_op_reply(92 mds0_inotable [delete] v122'254 uv254 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f47fc021070 con 0x557e54798c60 2024-09-21T03:42:50.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.691+0000 7f4812d05f40 1 -- 172.21.15.38:0/240085233 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x557e5479bbc0 con 0x557e54798c60 2024-09-21T03:42:50.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.692+0000 7f480c7bf640 1 -- 172.21.15.38:0/240085233 <== osd.1 v2:172.21.15.38:6816/1491818732 9 ==== osd_op_reply(93 mds0_inotable [create] v122'255 uv255 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f47fc021070 con 0x557e54798c60 2024-09-21T03:42:50.694 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-09-21T03:42:50.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.692+0000 7f4812d05f40 1 -- 172.21.15.38:0/240085233 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:94 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x557e5479bbc0 con 0x557e54798c60 2024-09-21T03:42:50.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.694+0000 7f480c7bf640 1 -- 172.21.15.38:0/240085233 <== osd.1 v2:172.21.15.38:6816/1491818732 10 ==== osd_op_reply(94 mds0_inotable [write 0~34] v122'256 uv256 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f47fc021070 con 0x557e54798c60 2024-09-21T03:42:50.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.694+0000 7f4812d05f40 1 -- 172.21.15.38:0/240085233 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:95 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x557e5479bbc0 con 0x557e54798c60 2024-09-21T03:42:50.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.695+0000 7f480c7bf640 1 -- 172.21.15.38:0/240085233 <== osd.1 v2:172.21.15.38:6816/1491818732 11 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v122'257 uv257 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f47fc021070 con 0x557e54798c60 2024-09-21T03:42:50.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.695+0000 7f4812d05f40 1 -- 172.21.15.38:0/240085233 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:96 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x557e5479bbc0 con 0x557e54798c60 2024-09-21T03:42:50.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.696+0000 7f480c7bf640 1 -- 172.21.15.38:0/240085233 <== osd.1 v2:172.21.15.38:6816/1491818732 12 ==== osd_op_reply(96 100.00000000 [create] v122'258 uv252 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f47fc021070 con 0x557e54798c60 2024-09-21T03:42:50.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.696+0000 7f4812d05f40 1 -- 172.21.15.38:0/240085233 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:97 13.13 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e122) -- 0x557e5479bbc0 con 0x557e54798c60 2024-09-21T03:42:50.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.697+0000 7f480c7bf640 1 -- 172.21.15.38:0/240085233 <== osd.1 v2:172.21.15.38:6816/1491818732 13 ==== osd_op_reply(97 100.00000000 [delete] v122'259 uv259 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f47fc021070 con 0x557e54798c60 2024-09-21T03:42:50.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.697+0000 7f4812d05f40 1 -- 172.21.15.38:0/240085233 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:98 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x557e5479bbc0 con 0x557e54798c60 2024-09-21T03:42:50.700 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-09-21T03:42:50.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.698+0000 7f480c7bf640 1 -- 172.21.15.38:0/240085233 <== osd.1 v2:172.21.15.38:6816/1491818732 14 ==== osd_op_reply(98 100.00000000 [create] v122'260 uv260 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f47fc021070 con 0x557e54798c60 2024-09-21T03:42:50.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.698+0000 7f4812d05f40 1 -- 172.21.15.38:0/240085233 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:99 13.13 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x557e5479bbc0 con 0x557e54798c60 2024-09-21T03:42:50.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.699+0000 7f480c7bf640 1 -- 172.21.15.38:0/240085233 <== osd.1 v2:172.21.15.38:6816/1491818732 15 ==== osd_op_reply(99 100.00000000 [omap-set-header] v122'261 uv261 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f47fc021070 con 0x557e54798c60 2024-09-21T03:42:50.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.699+0000 7f4812d05f40 1 -- 172.21.15.38:0/240085233 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:100 13.13 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x557e5479e9d0 con 0x557e54798c60 2024-09-21T03:42:50.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.701+0000 7f480c7bf640 1 -- 172.21.15.38:0/240085233 <== osd.1 v2:172.21.15.38:6816/1491818732 16 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v122'262 uv262 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f47fc021070 con 0x557e54798c60 2024-09-21T03:42:50.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.701+0000 7f4812d05f40 1 -- 172.21.15.38:0/240085233 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x557e5479c070 con 0x557e54794e50 2024-09-21T03:42:50.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.702+0000 7f480cfc0640 1 -- 172.21.15.38:0/240085233 <== osd.3 v2:172.21.15.38:6808/1286810294 16 ==== osd_op_reply(101 mds0_sessionmap [create] v122'111 uv110 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7f480400d890 con 0x557e54794e50 2024-09-21T03:42:50.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.702+0000 7f4812d05f40 1 -- 172.21.15.38:0/240085233 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e122) -- 0x557e5479c070 con 0x557e54794e50 2024-09-21T03:42:50.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.703+0000 7f480cfc0640 1 -- 172.21.15.38:0/240085233 <== osd.3 v2:172.21.15.38:6808/1286810294 17 ==== osd_op_reply(102 mds0_sessionmap [delete] v122'112 uv112 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f480400d890 con 0x557e54794e50 2024-09-21T03:42:50.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.703+0000 7f4812d05f40 1 -- 172.21.15.38:0/240085233 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:103 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x557e5479c070 con 0x557e54794e50 2024-09-21T03:42:50.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.704+0000 7f480cfc0640 1 -- 172.21.15.38:0/240085233 <== osd.3 v2:172.21.15.38:6808/1286810294 18 ==== osd_op_reply(103 mds0_sessionmap [create] v122'113 uv113 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f480400d890 con 0x557e54794e50 2024-09-21T03:42:50.706 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-09-21T03:42:50.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.704+0000 7f4812d05f40 1 -- 172.21.15.38:0/240085233 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:104 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x557e5479c070 con 0x557e54794e50 2024-09-21T03:42:50.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.705+0000 7f480cfc0640 1 -- 172.21.15.38:0/240085233 <== osd.3 v2:172.21.15.38:6808/1286810294 19 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v122'114 uv114 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f480400d890 con 0x557e54794e50 2024-09-21T03:42:50.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.705+0000 7f4812d05f40 1 -- 172.21.15.38:0/240085233 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x557e5479c070 con 0x557e54794e50 2024-09-21T03:42:50.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.706+0000 7f480cfc0640 1 -- 172.21.15.38:0/240085233 <== osd.3 v2:172.21.15.38:6808/1286810294 20 ==== osd_op_reply(105 609.00000000 [create] v122'153 uv152 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f480400d890 con 0x557e54794e50 2024-09-21T03:42:50.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.706+0000 7f4812d05f40 1 -- 172.21.15.38:0/240085233 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e122) -- 0x557e5479c070 con 0x557e54794e50 2024-09-21T03:42:50.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.707+0000 7f480cfc0640 1 -- 172.21.15.38:0/240085233 <== osd.3 v2:172.21.15.38:6808/1286810294 21 ==== osd_op_reply(106 609.00000000 [delete] v122'154 uv154 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f480400d890 con 0x557e54794e50 2024-09-21T03:42:50.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.707+0000 7f4812d05f40 1 -- 172.21.15.38:0/240085233 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x557e5479c070 con 0x557e54794e50 2024-09-21T03:42:50.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.709+0000 7f480cfc0640 1 -- 172.21.15.38:0/240085233 <== osd.3 v2:172.21.15.38:6808/1286810294 22 ==== osd_op_reply(107 609.00000000 [create] v122'155 uv155 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f480400d890 con 0x557e54794e50 2024-09-21T03:42:50.710 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-09-21T03:42:50.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.709+0000 7f4812d05f40 1 -- 172.21.15.38:0/240085233 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x557e5479d080 con 0x557e54794e50 2024-09-21T03:42:50.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.710+0000 7f480cfc0640 1 -- 172.21.15.38:0/240085233 <== osd.3 v2:172.21.15.38:6808/1286810294 23 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v122'156 uv156 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f480400d890 con 0x557e54794e50 2024-09-21T03:42:50.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.710+0000 7f4812d05f40 1 -- 172.21.15.38:0/240085233 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:109 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x557e5479d080 con 0x557e54794e50 2024-09-21T03:42:50.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.711+0000 7f480cfc0640 1 -- 172.21.15.38:0/240085233 <== osd.3 v2:172.21.15.38:6808/1286810294 24 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v122'157 uv157 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f480400d890 con 0x557e54794e50 2024-09-21T03:42:50.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.711+0000 7f4812d05f40 1 -- 172.21.15.38:0/240085233 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:110 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x557e5479c070 con 0x557e54794e50 2024-09-21T03:42:50.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.712+0000 7f480cfc0640 1 -- 172.21.15.38:0/240085233 <== osd.3 v2:172.21.15.38:6808/1286810294 25 ==== osd_op_reply(110 609.00000000 [omap-set-header] v122'158 uv158 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f480400d890 con 0x557e54794e50 2024-09-21T03:42:50.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.712+0000 7f4812d05f40 1 --2- 172.21.15.38:0/240085233 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x557e5479c6f0 0x557e5479cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:50.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.712+0000 7f4812d05f40 1 -- 172.21.15.38:0/240085233 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x557e5479d040 con 0x557e5479c6f0 2024-09-21T03:42:50.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.712+0000 7f480bfbe640 1 --2- 172.21.15.38:0/240085233 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x557e5479c6f0 0x557e5479cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:50.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.713+0000 7f480bfbe640 1 --2- 172.21.15.38:0/240085233 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x557e5479c6f0 0x557e5479cb00 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:50.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.714+0000 7f480bfbe640 1 -- 172.21.15.38:0/240085233 <== osd.2 v2:172.21.15.38:6824/2577858902 1 ==== osd_op_reply(111 400.00000000 [create] v122'130 uv129 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f47f4063480 con 0x557e5479c6f0 2024-09-21T03:42:50.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.714+0000 7f4812d05f40 1 -- 172.21.15.38:0/240085233 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e122) -- 0x557e5479d040 con 0x557e5479c6f0 2024-09-21T03:42:50.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.715+0000 7f480bfbe640 1 -- 172.21.15.38:0/240085233 <== osd.2 v2:172.21.15.38:6824/2577858902 2 ==== osd_op_reply(112 400.00000000 [delete] v122'131 uv131 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f47f4063480 con 0x557e5479c6f0 2024-09-21T03:42:50.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.715+0000 7f4812d05f40 1 -- 172.21.15.38:0/240085233 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x557e5479d040 con 0x557e5479c6f0 2024-09-21T03:42:50.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.716+0000 7f480bfbe640 1 -- 172.21.15.38:0/240085233 <== osd.2 v2:172.21.15.38:6824/2577858902 3 ==== osd_op_reply(113 400.00000000 [create] v122'132 uv132 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f47f4063480 con 0x557e5479c6f0 2024-09-21T03:42:50.718 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-09-21T03:42:50.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.716+0000 7f4812d05f40 1 -- 172.21.15.38:0/240085233 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:114 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x557e5479d040 con 0x557e5479c6f0 2024-09-21T03:42:50.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.717+0000 7f480bfbe640 1 -- 172.21.15.38:0/240085233 <== osd.2 v2:172.21.15.38:6824/2577858902 4 ==== osd_op_reply(114 400.00000000 [write 0~22] v122'133 uv133 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f47f4063480 con 0x557e5479c6f0 2024-09-21T03:42:50.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.717+0000 7f4812d05f40 1 -- 172.21.15.38:0/240085233 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:115 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x557e5479d040 con 0x557e5479c6f0 2024-09-21T03:42:50.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.719+0000 7f480bfbe640 1 -- 172.21.15.38:0/240085233 <== osd.2 v2:172.21.15.38:6824/2577858902 5 ==== osd_op_reply(115 400.00000000 [omap-set-header] v122'134 uv134 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f47f4063480 con 0x557e5479c6f0 2024-09-21T03:42:50.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.719+0000 7f4812d05f40 1 -- 172.21.15.38:0/240085233 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:116 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x557e5479d040 con 0x557e54798c60 2024-09-21T03:42:50.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.720+0000 7f480c7bf640 1 -- 172.21.15.38:0/240085233 <== osd.1 v2:172.21.15.38:6816/1491818732 17 ==== osd_op_reply(116 100.00000000.inode [create] v122'314 uv306 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7f47fc021070 con 0x557e54798c60 2024-09-21T03:42:50.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.720+0000 7f4812d05f40 1 -- 172.21.15.38:0/240085233 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:117 13.1f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e122) -- 0x557e5479d040 con 0x557e54798c60 2024-09-21T03:42:50.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.721+0000 7f480c7bf640 1 -- 172.21.15.38:0/240085233 <== osd.1 v2:172.21.15.38:6816/1491818732 18 ==== osd_op_reply(117 100.00000000.inode [delete] v122'315 uv315 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f47fc021070 con 0x557e54798c60 2024-09-21T03:42:50.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.721+0000 7f4812d05f40 1 -- 172.21.15.38:0/240085233 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:118 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x557e5479d040 con 0x557e54798c60 2024-09-21T03:42:50.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.722+0000 7f480c7bf640 1 -- 172.21.15.38:0/240085233 <== osd.1 v2:172.21.15.38:6816/1491818732 19 ==== osd_op_reply(118 100.00000000.inode [create] v122'316 uv316 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f47fc021070 con 0x557e54798c60 2024-09-21T03:42:50.724 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-09-21T03:42:50.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.722+0000 7f4812d05f40 1 -- 172.21.15.38:0/240085233 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:119 13.1f 13:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x557e5479d040 con 0x557e54798c60 2024-09-21T03:42:50.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.724+0000 7f480c7bf640 1 -- 172.21.15.38:0/240085233 <== osd.1 v2:172.21.15.38:6816/1491818732 20 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v122'317 uv317 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f47fc021070 con 0x557e54798c60 2024-09-21T03:42:50.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.724+0000 7f4812d05f40 1 -- 172.21.15.38:0/240085233 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:120 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x557e5479d040 con 0x557e54798c60 2024-09-21T03:42:50.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.725+0000 7f480c7bf640 1 -- 172.21.15.38:0/240085233 <== osd.1 v2:172.21.15.38:6816/1491818732 21 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v122'318 uv318 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f47fc021070 con 0x557e54798c60 2024-09-21T03:42:50.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.725+0000 7f4812d05f40 1 -- 172.21.15.38:0/240085233 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:121 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x557e5479d040 con 0x557e54798c60 2024-09-21T03:42:50.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.726+0000 7f480c7bf640 1 -- 172.21.15.38:0/240085233 <== osd.1 v2:172.21.15.38:6816/1491818732 22 ==== osd_op_reply(121 1.00000000 [create] v122'319 uv313 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7f47fc021070 con 0x557e54798c60 2024-09-21T03:42:50.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.726+0000 7f4812d05f40 1 -- 172.21.15.38:0/240085233 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:122 13.1f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e122) -- 0x557e5479d040 con 0x557e54798c60 2024-09-21T03:42:50.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.727+0000 7f480c7bf640 1 -- 172.21.15.38:0/240085233 <== osd.1 v2:172.21.15.38:6816/1491818732 23 ==== osd_op_reply(122 1.00000000 [delete] v122'320 uv320 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f47fc021070 con 0x557e54798c60 2024-09-21T03:42:50.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.727+0000 7f4812d05f40 1 -- 172.21.15.38:0/240085233 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:123 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x557e5479d040 con 0x557e54798c60 2024-09-21T03:42:50.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.728+0000 7f480c7bf640 1 -- 172.21.15.38:0/240085233 <== osd.1 v2:172.21.15.38:6816/1491818732 24 ==== osd_op_reply(123 1.00000000 [create] v122'321 uv321 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f47fc021070 con 0x557e54798c60 2024-09-21T03:42:50.730 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-09-21T03:42:50.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.728+0000 7f4812d05f40 1 -- 172.21.15.38:0/240085233 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:124 13.1f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x557e5479e4e0 con 0x557e54798c60 2024-09-21T03:42:50.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.729+0000 7f480c7bf640 1 -- 172.21.15.38:0/240085233 <== osd.1 v2:172.21.15.38:6816/1491818732 25 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v122'322 uv322 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f47fc021070 con 0x557e54798c60 2024-09-21T03:42:50.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.729+0000 7f4812d05f40 1 -- 172.21.15.38:0/240085233 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:125 13.1f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x557e5479d040 con 0x557e54798c60 2024-09-21T03:42:50.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.731+0000 7f480c7bf640 1 -- 172.21.15.38:0/240085233 <== osd.1 v2:172.21.15.38:6816/1491818732 26 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v122'323 uv323 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f47fc021070 con 0x557e54798c60 2024-09-21T03:42:50.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.731+0000 7f4812d05f40 1 -- 172.21.15.38:0/240085233 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:126 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x557e5479d040 con 0x557e54798c60 2024-09-21T03:42:50.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.732+0000 7f480c7bf640 1 -- 172.21.15.38:0/240085233 <== osd.1 v2:172.21.15.38:6816/1491818732 27 ==== osd_op_reply(126 1.00000000 [omap-set-header] v122'324 uv324 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f47fc021070 con 0x557e54798c60 2024-09-21T03:42:50.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.732+0000 7f4812d05f40 1 -- 172.21.15.38:0/240085233 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:127 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=584b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x557e5479e8e0 con 0x557e54798c60 2024-09-21T03:42:50.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.733+0000 7f480c7bf640 1 -- 172.21.15.38:0/240085233 <== osd.1 v2:172.21.15.38:6816/1491818732 28 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v122'325 uv325 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f47fc021070 con 0x557e54798c60 2024-09-21T03:42:50.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.733+0000 7f4812d05f40 1 -- 172.21.15.38:0/240085233 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x557e54790ad0 msgr2=0x557e54792ef0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:50.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.733+0000 7f4812d05f40 1 --2- 172.21.15.38:0/240085233 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x557e54790ad0 0x557e54792ef0 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:50.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.733+0000 7f4812d05f40 1 -- 172.21.15.38:0/240085233 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x557e54798c60 msgr2=0x557e5479b0e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:50.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.733+0000 7f4812d05f40 1 --2- 172.21.15.38:0/240085233 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x557e54798c60 0x557e5479b0e0 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:50.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.733+0000 7f4812d05f40 1 -- 172.21.15.38:0/240085233 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x557e5479c6f0 msgr2=0x557e5479cb00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:50.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.733+0000 7f4812d05f40 1 --2- 172.21.15.38:0/240085233 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x557e5479c6f0 0x557e5479cb00 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:50.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.733+0000 7f4812d05f40 1 -- 172.21.15.38:0/240085233 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x557e54794e50 msgr2=0x557e54797cb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:50.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.733+0000 7f4812d05f40 1 --2- 172.21.15.38:0/240085233 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x557e54794e50 0x557e54797cb0 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:50.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.733+0000 7f480cfc0640 1 -- 172.21.15.38:0/240085233 reap_dead start 2024-09-21T03:42:50.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.733+0000 7f4812d05f40 1 -- 172.21.15.38:0/240085233 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x557e5478b950 msgr2=0x557e5478dd70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:50.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.733+0000 7f4812d05f40 1 --2- 172.21.15.38:0/240085233 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x557e5478b950 0x557e5478dd70 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:50.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.733+0000 7f480cfc0640 1 -- 172.21.15.38:0/240085233 reap_dead start 2024-09-21T03:42:50.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.733+0000 7f4812d05f40 1 -- 172.21.15.38:0/240085233 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x557e5477ea90 msgr2=0x557e5477eea0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:50.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.733+0000 7f4812d05f40 1 --2- 172.21.15.38:0/240085233 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x557e5477ea90 0x557e5477eea0 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:50.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.733+0000 7f4812d05f40 1 -- 172.21.15.38:0/240085233 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f47ec0775a0 msgr2=0x7f47ec079ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:50.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.733+0000 7f4812d05f40 1 --2- 172.21.15.38:0/240085233 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f47ec0775a0 0x7f47ec079ab0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f47f4055000 tx=0x7f47f4058040 comp rx=0 tx=0).stop 2024-09-21T03:42:50.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.733+0000 7f4812d05f40 1 -- 172.21.15.38:0/240085233 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557e54610d00 msgr2=0x557e546553b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:50.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.733+0000 7f4812d05f40 1 --2- 172.21.15.38:0/240085233 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557e54610d00 0x557e546553b0 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7f47fc00c1d0 tx=0x7f47fc00c690 comp rx=0 tx=0).stop 2024-09-21T03:42:50.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.733+0000 7f4812d05f40 1 -- 172.21.15.38:0/240085233 shutdown_connections 2024-09-21T03:42:50.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.733+0000 7f4812d05f40 1 --2- 172.21.15.38:0/240085233 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x557e5477ea90 0x557e5477eea0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:50.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.733+0000 7f4812d05f40 1 --2- 172.21.15.38:0/240085233 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f47ec0775a0 0x7f47ec079ab0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:50.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.733+0000 7f4812d05f40 1 --2- 172.21.15.38:0/240085233 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557e54610d00 0x557e546553b0 unknown :-1 s=CLOSED pgs=1870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:50.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.733+0000 7f4812d05f40 1 -- 172.21.15.38:0/240085233 >> 172.21.15.38:0/240085233 conn(0x557e5452fac0 msgr2=0x557e546c5d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:50.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.733+0000 7f4812d05f40 1 -- 172.21.15.38:0/240085233 shutdown_connections 2024-09-21T03:42:50.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.733+0000 7f4812d05f40 1 -- 172.21.15.38:0/240085233 wait complete. 2024-09-21T03:42:50.738 DEBUG:teuthology.orchestra.run.smithi038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata truncate 500.00000000 0 2024-09-21T03:42:50.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.833+0000 7f5267ff8f40 1 Processor -- start 2024-09-21T03:42:50.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.833+0000 7f5267ff8f40 1 -- start start 2024-09-21T03:42:50.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.834+0000 7f5267ff8f40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5588cce91d00 0x5588ccf64470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:50.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.834+0000 7f5267ff8f40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5588ccf4c950 0x5588cced40c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:50.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.834+0000 7f5267ff8f40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5588ccf67670 0x5588ccf67c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:50.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.834+0000 7f5267ff8f40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5588ccddfd60 con 0x5588ccf67670 2024-09-21T03:42:50.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.834+0000 7f5267ff8f40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5588cce3e430 con 0x5588ccf4c950 2024-09-21T03:42:50.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.834+0000 7f5267ff8f40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x5588cced97b0 con 0x5588cce91d00 2024-09-21T03:42:50.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.834+0000 7f52622b5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5588ccf67670 0x5588ccf67c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:50.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.834+0000 7f52622b5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5588ccf67670 0x5588ccf67c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38494/0 (socket says 172.21.15.38:38494) 2024-09-21T03:42:50.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.834+0000 7f52622b5640 1 -- 172.21.15.38:0/2815962171 learned_addr learned my addr 172.21.15.38:0/2815962171 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:42:50.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.834+0000 7f52592b3640 1 --2- 172.21.15.38:0/2815962171 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5588ccf4c950 0x5588cced40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:50.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.834+0000 7f5261ab4640 1 --2- 172.21.15.38:0/2815962171 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5588cce91d00 0x5588ccf64470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:50.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.834+0000 7f52622b5640 1 -- 172.21.15.38:0/2815962171 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5588cce91d00 msgr2=0x5588ccf64470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:50.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.834+0000 7f52622b5640 1 --2- 172.21.15.38:0/2815962171 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5588cce91d00 0x5588ccf64470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:50.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.834+0000 7f52622b5640 1 -- 172.21.15.38:0/2815962171 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5588ccf4c950 msgr2=0x5588cced40c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:50.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.834+0000 7f52622b5640 1 --2- 172.21.15.38:0/2815962171 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5588ccf4c950 0x5588cced40c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:50.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.834+0000 7f52622b5640 1 -- 172.21.15.38:0/2815962171 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588ccec0070 con 0x5588ccf67670 2024-09-21T03:42:50.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.834+0000 7f52592b3640 1 --2- 172.21.15.38:0/2815962171 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5588ccf4c950 0x5588cced40c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:42:50.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.834+0000 7f52622b5640 1 --2- 172.21.15.38:0/2815962171 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5588ccf67670 0x5588ccf67c70 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7f525c00a4b0 tx=0x7f525c00a970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfe79bf23dc3a1c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:50.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.835+0000 7f5260ab2640 1 -- 172.21.15.38:0/2815962171 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f525c015070 con 0x5588ccf67670 2024-09-21T03:42:50.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.835+0000 7f5260ab2640 1 -- 172.21.15.38:0/2815962171 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f525c00e710 con 0x5588ccf67670 2024-09-21T03:42:50.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.835+0000 7f5260ab2640 1 -- 172.21.15.38:0/2815962171 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f525c00eb60 con 0x5588ccf67670 2024-09-21T03:42:50.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.835+0000 7f5267ff8f40 1 -- 172.21.15.38:0/2815962171 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5588ccf67670 msgr2=0x5588ccf67c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:50.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.835+0000 7f5267ff8f40 1 --2- 172.21.15.38:0/2815962171 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5588ccf67670 0x5588ccf67c70 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7f525c00a4b0 tx=0x7f525c00a970 comp rx=0 tx=0).stop 2024-09-21T03:42:50.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.835+0000 7f5267ff8f40 1 -- 172.21.15.38:0/2815962171 shutdown_connections 2024-09-21T03:42:50.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.835+0000 7f5267ff8f40 1 --2- 172.21.15.38:0/2815962171 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5588ccf67670 0x5588ccf67c70 unknown :-1 s=CLOSED pgs=1872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:50.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.835+0000 7f5267ff8f40 1 --2- 172.21.15.38:0/2815962171 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5588ccf4c950 0x5588cced40c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:50.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.835+0000 7f5267ff8f40 1 --2- 172.21.15.38:0/2815962171 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5588cce91d00 0x5588ccf64470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:50.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.835+0000 7f5267ff8f40 1 -- 172.21.15.38:0/2815962171 >> 172.21.15.38:0/2815962171 conn(0x5588ccdb0ac0 msgr2=0x5588ccf66640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:50.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.835+0000 7f5267ff8f40 1 -- 172.21.15.38:0/2815962171 shutdown_connections 2024-09-21T03:42:50.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.835+0000 7f5267ff8f40 1 -- 172.21.15.38:0/2815962171 wait complete. 2024-09-21T03:42:50.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.836+0000 7f5267ff8f40 1 Processor -- start 2024-09-21T03:42:50.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.836+0000 7f5267ff8f40 1 -- start start 2024-09-21T03:42:50.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.837+0000 7f5267ff8f40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5588cce91d00 0x5588ccf40b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:50.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.838+0000 7f5267ff8f40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5588ccf4c950 0x5588ccf66600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:50.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.838+0000 7f5267ff8f40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5588ccf67670 0x5588ccf422e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:50.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.838+0000 7f5267ff8f40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5588ccf4b410 con 0x5588cce91d00 2024-09-21T03:42:50.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.838+0000 7f5267ff8f40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5588ccf47f70 con 0x5588ccf4c950 2024-09-21T03:42:50.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.838+0000 7f5267ff8f40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x5588ccf48340 con 0x5588ccf67670 2024-09-21T03:42:50.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.838+0000 7f5261ab4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5588cce91d00 0x5588ccf40b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:50.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.838+0000 7f5261ab4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5588cce91d00 0x5588ccf40b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38496/0 (socket says 172.21.15.38:38496) 2024-09-21T03:42:50.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.838+0000 7f5261ab4640 1 -- 172.21.15.38:0/3044691793 learned_addr learned my addr 172.21.15.38:0/3044691793 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:42:50.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.838+0000 7f52622b5640 1 --2- 172.21.15.38:0/3044691793 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5588ccf67670 0x5588ccf422e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:50.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.838+0000 7f52592b3640 1 --2- 172.21.15.38:0/3044691793 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5588ccf4c950 0x5588ccf66600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:50.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.838+0000 7f5261ab4640 1 -- 172.21.15.38:0/3044691793 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5588ccf67670 msgr2=0x5588ccf422e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:50.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.838+0000 7f5261ab4640 1 --2- 172.21.15.38:0/3044691793 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5588ccf67670 0x5588ccf422e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:50.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.838+0000 7f5261ab4640 1 -- 172.21.15.38:0/3044691793 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5588ccf4c950 msgr2=0x5588ccf66600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:50.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.838+0000 7f5261ab4640 1 --2- 172.21.15.38:0/3044691793 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5588ccf4c950 0x5588ccf66600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:50.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.838+0000 7f5261ab4640 1 -- 172.21.15.38:0/3044691793 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588ccff7800 con 0x5588cce91d00 2024-09-21T03:42:50.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.838+0000 7f5261ab4640 1 --2- 172.21.15.38:0/3044691793 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5588cce91d00 0x5588ccf40b80 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7f5254025f50 tx=0x7f525404d9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:50.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.838+0000 7f525b7fe640 1 -- 172.21.15.38:0/3044691793 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52540566f0 con 0x5588cce91d00 2024-09-21T03:42:50.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.838+0000 7f525b7fe640 1 -- 172.21.15.38:0/3044691793 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5254056980 con 0x5588cce91d00 2024-09-21T03:42:50.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.838+0000 7f525b7fe640 1 -- 172.21.15.38:0/3044691793 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5254056dd0 con 0x5588cce91d00 2024-09-21T03:42:50.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.838+0000 7f5267ff8f40 1 -- 172.21.15.38:0/3044691793 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5588ccff7ae0 con 0x5588cce91d00 2024-09-21T03:42:50.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.838+0000 7f5267ff8f40 1 -- 172.21.15.38:0/3044691793 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5588ccff80a0 con 0x5588cce91d00 2024-09-21T03:42:50.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.840+0000 7f525b7fe640 1 -- 172.21.15.38:0/3044691793 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f5254062470 con 0x5588cce91d00 2024-09-21T03:42:50.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.841+0000 7f525b7fe640 1 --2- 172.21.15.38:0/3044691793 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5244077570 0x7f5244079a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:50.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.841+0000 7f525b7fe640 1 -- 172.21.15.38:0/3044691793 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 8540+0+0 (secure 0 0 0) 0x7f52540de360 con 0x5588cce91d00 2024-09-21T03:42:50.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.841+0000 7f52592b3640 1 --2- 172.21.15.38:0/3044691793 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5244077570 0x7f5244079a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:50.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.841+0000 7f5267ff8f40 1 --2- 172.21.15.38:0/3044691793 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x5588ccf66b40 0x5588cced57e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:50.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.841+0000 7f5267ff8f40 1 -- 172.21.15.38:0/3044691793 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:1 13.15 13:aa448500:::500.00000000:head [truncate 0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x5588ccff7120 con 0x5588ccf66b40 2024-09-21T03:42:50.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.841+0000 7f52622b5640 1 --2- 172.21.15.38:0/3044691793 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x5588ccf66b40 0x5588cced57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:50.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.841+0000 7f52592b3640 1 --2- 172.21.15.38:0/3044691793 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5244077570 0x7f5244079a80 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x5588ccf67d60 tx=0x7f524c0057d0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:50.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.842+0000 7f52622b5640 1 --2- 172.21.15.38:0/3044691793 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x5588ccf66b40 0x5588cced57e0 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:50.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.843+0000 7f52622b5640 1 -- 172.21.15.38:0/3044691793 <== osd.3 v2:172.21.15.38:6808/1286810294 1 ==== osd_op_reply(1 500.00000000 [truncate 0] v122'293 uv293 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f525c00e070 con 0x5588ccf66b40 2024-09-21T03:42:50.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.843+0000 7f5267ff8f40 1 -- 172.21.15.38:0/3044691793 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x5588ccf66b40 msgr2=0x5588cced57e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:50.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.843+0000 7f5267ff8f40 1 --2- 172.21.15.38:0/3044691793 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x5588ccf66b40 0x5588cced57e0 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:50.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.843+0000 7f5267ff8f40 1 -- 172.21.15.38:0/3044691793 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5244077570 msgr2=0x7f5244079a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:50.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.843+0000 7f5267ff8f40 1 --2- 172.21.15.38:0/3044691793 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5244077570 0x7f5244079a80 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x5588ccf67d60 tx=0x7f524c0057d0 comp rx=0 tx=0).stop 2024-09-21T03:42:50.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.843+0000 7f5267ff8f40 1 -- 172.21.15.38:0/3044691793 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5588cce91d00 msgr2=0x5588ccf40b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:50.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.843+0000 7f5267ff8f40 1 --2- 172.21.15.38:0/3044691793 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5588cce91d00 0x5588ccf40b80 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7f5254025f50 tx=0x7f525404d9c0 comp rx=0 tx=0).stop 2024-09-21T03:42:50.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.843+0000 7f52622b5640 1 -- 172.21.15.38:0/3044691793 reap_dead start 2024-09-21T03:42:50.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.843+0000 7f5267ff8f40 1 -- 172.21.15.38:0/3044691793 shutdown_connections 2024-09-21T03:42:50.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.843+0000 7f5267ff8f40 1 -- 172.21.15.38:0/3044691793 >> 172.21.15.38:0/3044691793 conn(0x5588ccdb0ac0 msgr2=0x5588ccf48690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:50.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.843+0000 7f5267ff8f40 1 -- 172.21.15.38:0/3044691793 shutdown_connections 2024-09-21T03:42:50.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.843+0000 7f5267ff8f40 1 -- 172.21.15.38:0/3044691793 wait complete. 2024-09-21T03:42:50.848 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-21T03:42:50.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.968+0000 7ffbc9990640 1 -- 172.21.15.38:0/3175150522 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffbb0004800 msgr2=0x7ffbb0004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:50.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.968+0000 7ffbc9990640 1 --2- 172.21.15.38:0/3175150522 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffbb0004800 0x7ffbb0004c10 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7ffbc4073ca0 tx=0x7ffbb0005830 comp rx=0 tx=0).stop 2024-09-21T03:42:50.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.968+0000 7ffbc37fe640 1 -- 172.21.15.38:0/3175150522 reap_dead start 2024-09-21T03:42:50.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.968+0000 7ffbc9990640 1 -- 172.21.15.38:0/3175150522 shutdown_connections 2024-09-21T03:42:50.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.968+0000 7ffbc9990640 1 -- 172.21.15.38:0/3175150522 >> 172.21.15.38:0/3175150522 conn(0x7ffbc40cdca0 msgr2=0x7ffbc40ce0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:50.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.968+0000 7ffbc9990640 1 -- 172.21.15.38:0/3175150522 shutdown_connections 2024-09-21T03:42:50.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.968+0000 7ffbc9990640 1 -- 172.21.15.38:0/3175150522 wait complete. 2024-09-21T03:42:50.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.970+0000 7ffbc9990640 1 Processor -- start 2024-09-21T03:42:50.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.970+0000 7ffbc9990640 1 -- start start 2024-09-21T03:42:50.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.971+0000 7ffbc9990640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffbb0004800 0x7ffbc40ca7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:50.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.971+0000 7ffbc9990640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffbc40c82d0 0x7ffbc415fa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:50.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.971+0000 7ffbc9990640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffbc41f6550 0x7ffbc41f6960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:50.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.971+0000 7ffbc9990640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ffbc40aed00 con 0x7ffbb0004800 2024-09-21T03:42:50.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.971+0000 7ffbc9990640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ffbc40aeb80 con 0x7ffbc40c82d0 2024-09-21T03:42:50.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.971+0000 7ffbc9990640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7ffbc4070610 con 0x7ffbc41f6550 2024-09-21T03:42:50.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.971+0000 7ffbc2ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffbb0004800 0x7ffbc40ca7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:50.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.971+0000 7ffbc2ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffbb0004800 0x7ffbc40ca7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38506/0 (socket says 172.21.15.38:38506) 2024-09-21T03:42:50.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.971+0000 7ffbc2ffd640 1 -- 172.21.15.38:0/2418655159 learned_addr learned my addr 172.21.15.38:0/2418655159 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:42:50.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.971+0000 7ffbc27fc640 1 --2- 172.21.15.38:0/2418655159 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffbc40c82d0 0x7ffbc415fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:50.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.971+0000 7ffbc37fe640 1 --2- 172.21.15.38:0/2418655159 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffbc41f6550 0x7ffbc41f6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:50.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.971+0000 7ffbc2ffd640 1 -- 172.21.15.38:0/2418655159 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffbc41f6550 msgr2=0x7ffbc41f6960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:50.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.971+0000 7ffbc2ffd640 1 --2- 172.21.15.38:0/2418655159 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffbc41f6550 0x7ffbc41f6960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:50.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.971+0000 7ffbc2ffd640 1 -- 172.21.15.38:0/2418655159 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffbc40c82d0 msgr2=0x7ffbc415fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:50.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.971+0000 7ffbc2ffd640 1 --2- 172.21.15.38:0/2418655159 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffbc40c82d0 0x7ffbc415fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:50.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.971+0000 7ffbc2ffd640 1 -- 172.21.15.38:0/2418655159 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbc41f6ea0 con 0x7ffbb0004800 2024-09-21T03:42:50.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.971+0000 7ffbc27fc640 1 --2- 172.21.15.38:0/2418655159 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffbc40c82d0 0x7ffbc415fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:42:50.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.972+0000 7ffbc37fe640 1 --2- 172.21.15.38:0/2418655159 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffbc41f6550 0x7ffbc41f6960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:42:50.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.972+0000 7ffbc2ffd640 1 --2- 172.21.15.38:0/2418655159 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffbb0004800 0x7ffbc40ca7e0 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7ffbbc011d90 tx=0x7ffbbc00f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:50.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.972+0000 7ffbbbfff640 1 -- 172.21.15.38:0/2418655159 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbbc003d80 con 0x7ffbb0004800 2024-09-21T03:42:50.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.972+0000 7ffbbbfff640 1 -- 172.21.15.38:0/2418655159 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffbbc004b20 con 0x7ffbb0004800 2024-09-21T03:42:50.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.972+0000 7ffbc9990640 1 -- 172.21.15.38:0/2418655159 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbc40cae30 con 0x7ffbb0004800 2024-09-21T03:42:50.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.972+0000 7ffbbbfff640 1 -- 172.21.15.38:0/2418655159 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbbc004050 con 0x7ffbb0004800 2024-09-21T03:42:50.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.972+0000 7ffbc9990640 1 -- 172.21.15.38:0/2418655159 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbc40cb3f0 con 0x7ffbb0004800 2024-09-21T03:42:50.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.973+0000 7ffbc9990640 1 -- 172.21.15.38:0/2418655159 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffbc4070b70 con 0x7ffbb0004800 2024-09-21T03:42:50.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.974+0000 7ffbbbfff640 1 -- 172.21.15.38:0/2418655159 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ffbbc0041f0 con 0x7ffbb0004800 2024-09-21T03:42:50.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.974+0000 7ffbbbfff640 1 --2- 172.21.15.38:0/2418655159 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ffba4078bc0 0x7ffba407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:50.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.974+0000 7ffbbbfff640 1 -- 172.21.15.38:0/2418655159 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 8540+0+0 (secure 0 0 0) 0x7ffbbc09c9e0 con 0x7ffbb0004800 2024-09-21T03:42:50.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.975+0000 7ffbc27fc640 1 --2- 172.21.15.38:0/2418655159 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ffba4078bc0 0x7ffba407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:50.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.975+0000 7ffbc27fc640 1 --2- 172.21.15.38:0/2418655159 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ffba4078bc0 0x7ffba407b0d0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7ffbac033000 tx=0x7ffbac02d000 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:50.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:50.979+0000 7ffbbbfff640 1 -- 172.21.15.38:0/2418655159 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7ffbc41f6ea0 con 0x7ffbb0004800 2024-09-21T03:42:51.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:51.205+0000 7ffbc9990640 1 -- 172.21.15.38:0/2418655159 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7ffbc40714d0 con 0x7ffbb0004800 2024-09-21T03:42:51.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:51.481+0000 7ffbbbfff640 1 -- 172.21.15.38:0/2418655159 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v242) ==== 162+0+0 (secure 0 0 0) 0x7ffbbc0642c0 con 0x7ffbb0004800 2024-09-21T03:42:51.483 INFO:teuthology.orchestra.run.smithi038.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-21T03:42:51.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:51.483+0000 7ffbc9990640 1 -- 172.21.15.38:0/2418655159 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ffba4078bc0 msgr2=0x7ffba407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:51.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:51.483+0000 7ffbc9990640 1 --2- 172.21.15.38:0/2418655159 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ffba4078bc0 0x7ffba407b0d0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7ffbac033000 tx=0x7ffbac02d000 comp rx=0 tx=0).stop 2024-09-21T03:42:51.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:51.483+0000 7ffbc9990640 1 -- 172.21.15.38:0/2418655159 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffbb0004800 msgr2=0x7ffbc40ca7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:51.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:51.483+0000 7ffbc9990640 1 --2- 172.21.15.38:0/2418655159 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffbb0004800 0x7ffbc40ca7e0 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7ffbbc011d90 tx=0x7ffbbc00f790 comp rx=0 tx=0).stop 2024-09-21T03:42:51.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:51.483+0000 7ffbc9990640 1 -- 172.21.15.38:0/2418655159 shutdown_connections 2024-09-21T03:42:51.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:51.483+0000 7ffbc9990640 1 --2- 172.21.15.38:0/2418655159 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ffba4078bc0 0x7ffba407b0d0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:51.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:51.483+0000 7ffbc9990640 1 --2- 172.21.15.38:0/2418655159 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffbc41f6550 0x7ffbc41f6960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:51.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:51.483+0000 7ffbc9990640 1 --2- 172.21.15.38:0/2418655159 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffbc40c82d0 0x7ffbc415fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:51.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:51.483+0000 7ffbc9990640 1 --2- 172.21.15.38:0/2418655159 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffbb0004800 0x7ffbc40ca7e0 unknown :-1 s=CLOSED pgs=1876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:51.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:51.483+0000 7ffbc9990640 1 -- 172.21.15.38:0/2418655159 >> 172.21.15.38:0/2418655159 conn(0x7ffbc40cdca0 msgr2=0x7ffbc406ff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:51.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:51.483+0000 7ffbc9990640 1 -- 172.21.15.38:0/2418655159 shutdown_connections 2024-09-21T03:42:51.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:51.483+0000 7ffbc9990640 1 -- 172.21.15.38:0/2418655159 wait complete. 2024-09-21T03:42:51.902 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:42:52.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:52.028+0000 7f8daa769640 1 -- 172.21.15.38:0/958127754 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8da40d0b40 msgr2=0x7f8da406e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:52.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:52.028+0000 7f8daa769640 1 --2- 172.21.15.38:0/958127754 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8da40d0b40 0x7f8da406e6f0 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7f8da40cd2c0 tx=0x7f8d9402a560 comp rx=0 tx=0).stop 2024-09-21T03:42:52.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:52.028+0000 7f8daa769640 1 -- 172.21.15.38:0/958127754 shutdown_connections 2024-09-21T03:42:52.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:52.028+0000 7f8daa769640 1 --2- 172.21.15.38:0/958127754 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8da40b4500 0x7f8da4073d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:52.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:52.028+0000 7f8daa769640 1 --2- 172.21.15.38:0/958127754 >> v2:172.21.15.195:3300/0 conn(0x7f8da406ec30 0x7f8da40b3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:52.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:52.028+0000 7f8daa769640 1 --2- 172.21.15.38:0/958127754 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8da40d0b40 0x7f8da406e6f0 unknown :-1 s=CLOSED pgs=1877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:52.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:52.028+0000 7f8daa769640 1 -- 172.21.15.38:0/958127754 >> 172.21.15.38:0/958127754 conn(0x7f8da40b2cc0 msgr2=0x7f8da40b30d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:52.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:52.028+0000 7f8daa769640 1 -- 172.21.15.38:0/958127754 shutdown_connections 2024-09-21T03:42:52.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:52.028+0000 7f8daa769640 1 -- 172.21.15.38:0/958127754 wait complete. 2024-09-21T03:42:52.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:52.030+0000 7f8daa769640 1 Processor -- start 2024-09-21T03:42:52.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:52.030+0000 7f8daa769640 1 -- start start 2024-09-21T03:42:52.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:52.032+0000 7f8daa769640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8da406ec30 0x7f8da40b2050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:52.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:52.032+0000 7f8daa769640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8da40b4500 0x7f8da415cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:52.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:52.032+0000 7f8daa769640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8da40d0b40 0x7f8da415d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:52.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:52.032+0000 7f8daa769640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f8da415b5d0 con 0x7f8da406ec30 2024-09-21T03:42:52.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:52.032+0000 7f8daa769640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f8da40745b0 con 0x7f8da40b4500 2024-09-21T03:42:52.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:52.032+0000 7f8daa769640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f8da415b750 con 0x7f8da40d0b40 2024-09-21T03:42:52.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:52.032+0000 7f8da3fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8da406ec30 0x7f8da40b2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:52.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:52.032+0000 7f8da3fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8da406ec30 0x7f8da40b2050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38526/0 (socket says 172.21.15.38:38526) 2024-09-21T03:42:52.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:52.032+0000 7f8da3fff640 1 -- 172.21.15.38:0/1522824280 learned_addr learned my addr 172.21.15.38:0/1522824280 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:42:52.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:52.032+0000 7f8da37fe640 1 --2- 172.21.15.38:0/1522824280 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8da40b4500 0x7f8da415cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:52.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:52.032+0000 7f8da8d06640 1 --2- 172.21.15.38:0/1522824280 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8da40d0b40 0x7f8da415d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:52.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:52.032+0000 7f8da3fff640 1 -- 172.21.15.38:0/1522824280 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8da40d0b40 msgr2=0x7f8da415d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:52.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:52.032+0000 7f8da3fff640 1 --2- 172.21.15.38:0/1522824280 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8da40d0b40 0x7f8da415d140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:52.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:52.032+0000 7f8da3fff640 1 -- 172.21.15.38:0/1522824280 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8da40b4500 msgr2=0x7f8da415cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:52.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:52.032+0000 7f8da3fff640 1 --2- 172.21.15.38:0/1522824280 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8da40b4500 0x7f8da415cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:52.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:52.032+0000 7f8da3fff640 1 -- 172.21.15.38:0/1522824280 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8da40a94e0 con 0x7f8da406ec30 2024-09-21T03:42:52.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:52.032+0000 7f8da37fe640 1 --2- 172.21.15.38:0/1522824280 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8da40b4500 0x7f8da415cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:42:52.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:52.032+0000 7f8da3fff640 1 --2- 172.21.15.38:0/1522824280 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8da406ec30 0x7f8da40b2050 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7f8d94034040 tx=0x7f8d940039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:52.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:52.033+0000 7f8da0ff9640 1 -- 172.21.15.38:0/1522824280 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d9403e030 con 0x7f8da406ec30 2024-09-21T03:42:52.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:52.033+0000 7f8da0ff9640 1 -- 172.21.15.38:0/1522824280 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8d940047a0 con 0x7f8da406ec30 2024-09-21T03:42:52.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:52.033+0000 7f8daa769640 1 -- 172.21.15.38:0/1522824280 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8da41499b0 con 0x7f8da406ec30 2024-09-21T03:42:52.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:52.033+0000 7f8da0ff9640 1 -- 172.21.15.38:0/1522824280 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d94035040 con 0x7f8da406ec30 2024-09-21T03:42:52.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:52.033+0000 7f8daa769640 1 -- 172.21.15.38:0/1522824280 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8da40a9a30 con 0x7f8da406ec30 2024-09-21T03:42:52.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:52.034+0000 7f8daa769640 1 -- 172.21.15.38:0/1522824280 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8da406d9b0 con 0x7f8da406ec30 2024-09-21T03:42:52.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:52.034+0000 7f8da0ff9640 1 -- 172.21.15.38:0/1522824280 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f8d94041030 con 0x7f8da406ec30 2024-09-21T03:42:52.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:52.035+0000 7f8da0ff9640 1 --2- 172.21.15.38:0/1522824280 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8d90078bc0 0x7f8d9007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:52.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:52.035+0000 7f8da0ff9640 1 -- 172.21.15.38:0/1522824280 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 8540+0+0 (secure 0 0 0) 0x7f8d940bc120 con 0x7f8da406ec30 2024-09-21T03:42:52.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:52.036+0000 7f8da37fe640 1 --2- 172.21.15.38:0/1522824280 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8d90078bc0 0x7f8d9007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:52.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:52.036+0000 7f8da37fe640 1 --2- 172.21.15.38:0/1522824280 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8d90078bc0 0x7f8d9007b0d0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f8d98005fd0 tx=0x7f8d98005ea0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:52.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:52.040+0000 7f8da0ff9640 1 -- 172.21.15.38:0/1522824280 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f8da406d9b0 con 0x7f8da406ec30 2024-09-21T03:42:52.188 INFO:tasks.ceph.mds.a.smithi038.stderr: -41> 2024-09-21T03:42:47.434+0000 7f24e1eef640 -1 log_channel(cluster) log [ERR] : missing journal object 2024-09-21T03:42:52.243 INFO:tasks.ceph.mds.a.smithi038.stdout:starting mds.a at 2024-09-21T03:42:52.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:52.269+0000 7f8daa769640 1 -- 172.21.15.38:0/1522824280 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8da4073d50 con 0x7f8da406ec30 2024-09-21T03:42:52.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:52.269+0000 7f8da0ff9640 1 -- 172.21.15.38:0/1522824280 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 243 v243) ==== 95+0+4401 (secure 0 0 0) 0x7f8d94083a90 con 0x7f8da406ec30 2024-09-21T03:42:52.271 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:42:52.272 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":243,"btime":"2024-09-21T03:42:51:484344+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6086,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/965972663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":965972663},{"type":"v1","addr":"172.21.15.195:6836","nonce":965972663}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6700,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/3176122334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":3176122334},{"type":"v1","addr":"172.21.15.195:6837","nonce":3176122334}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":243,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:40:02.319282+0000","modified":"2024-09-21T03:42:51.484325+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":122,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9438},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9438":{"gid":9438,"name":"c","rank":0,"incarnation":243,"state":"up:replay","state_seq":1,"addr":"172.21.15.38:6833/1995903506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":1995903506},{"type":"v1","addr":"172.21.15.38:6833","nonce":1995903506}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-21T03:42:52.272 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 243 2024-09-21T03:42:52.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:52.272+0000 7f8daa769640 1 -- 172.21.15.38:0/1522824280 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8d90078bc0 msgr2=0x7f8d9007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:52.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:52.272+0000 7f8daa769640 1 --2- 172.21.15.38:0/1522824280 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8d90078bc0 0x7f8d9007b0d0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f8d98005fd0 tx=0x7f8d98005ea0 comp rx=0 tx=0).stop 2024-09-21T03:42:52.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:52.272+0000 7f8daa769640 1 -- 172.21.15.38:0/1522824280 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8da406ec30 msgr2=0x7f8da40b2050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:52.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:52.272+0000 7f8daa769640 1 --2- 172.21.15.38:0/1522824280 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8da406ec30 0x7f8da40b2050 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7f8d94034040 tx=0x7f8d940039e0 comp rx=0 tx=0).stop 2024-09-21T03:42:52.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:52.272+0000 7f8daa769640 1 -- 172.21.15.38:0/1522824280 shutdown_connections 2024-09-21T03:42:52.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:52.272+0000 7f8daa769640 1 --2- 172.21.15.38:0/1522824280 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8d90078bc0 0x7f8d9007b0d0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:52.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:52.272+0000 7f8daa769640 1 --2- 172.21.15.38:0/1522824280 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8da40d0b40 0x7f8da415d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:52.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:52.272+0000 7f8daa769640 1 --2- 172.21.15.38:0/1522824280 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8da40b4500 0x7f8da415cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:52.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:52.272+0000 7f8daa769640 1 --2- 172.21.15.38:0/1522824280 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8da406ec30 0x7f8da40b2050 unknown :-1 s=CLOSED pgs=1878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:52.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:52.272+0000 7f8daa769640 1 -- 172.21.15.38:0/1522824280 >> 172.21.15.38:0/1522824280 conn(0x7f8da40b2cc0 msgr2=0x7f8da40cd110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:52.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:52.272+0000 7f8daa769640 1 -- 172.21.15.38:0/1522824280 shutdown_connections 2024-09-21T03:42:52.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:52.272+0000 7f8daa769640 1 -- 172.21.15.38:0/1522824280 wait complete. 2024-09-21T03:42:52.285 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 243, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T03:40:02.319282+0000', 'modified': '2024-09-21T03:42:51.484325+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 122, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9438}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9438': {'gid': 9438, 'name': 'c', 'rank': 0, 'incarnation': 243, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.38:6833/1995903506', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.38:6832', 'nonce': 1995903506}, {'type': 'v1', 'addr': '172.21.15.38:6833', 'nonce': 1995903506}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-09-21T03:42:52.285 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9438:up:replay 2024-09-21T03:42:52.285 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2024-09-21T03:42:57.689 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:42:57.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:57.819+0000 7f90cfe77640 1 -- 172.21.15.38:0/4272061138 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f90c8162280 msgr2=0x7f90c8162690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:57.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:57.819+0000 7f90cfe77640 1 --2- 172.21.15.38:0/4272061138 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f90c8162280 0x7f90c8162690 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f90bc009830 tx=0x7f90bc02a510 comp rx=0 tx=0).stop 2024-09-21T03:42:57.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:57.819+0000 7f90cfe77640 1 -- 172.21.15.38:0/4272061138 shutdown_connections 2024-09-21T03:42:57.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:57.819+0000 7f90cfe77640 1 --2- 172.21.15.38:0/4272061138 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f90c80b2e40 0x7f90c80b3250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:57.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:57.819+0000 7f90cfe77640 1 --2- 172.21.15.38:0/4272061138 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f90c8162280 0x7f90c8162690 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:57.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:57.819+0000 7f90cfe77640 1 -- 172.21.15.38:0/4272061138 >> 172.21.15.38:0/4272061138 conn(0x7f90c80b6b20 msgr2=0x7f90c80d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:57.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:57.820+0000 7f90cfe77640 1 -- 172.21.15.38:0/4272061138 shutdown_connections 2024-09-21T03:42:57.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:57.820+0000 7f90cfe77640 1 -- 172.21.15.38:0/4272061138 wait complete. 2024-09-21T03:42:57.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:57.821+0000 7f90cfe77640 1 Processor -- start 2024-09-21T03:42:57.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:57.821+0000 7f90cfe77640 1 -- start start 2024-09-21T03:42:57.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:57.823+0000 7f90cfe77640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f90c80b2e40 0x7f90c80ac070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:57.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:57.823+0000 7f90cfe77640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f90c8162280 0x7f90c81f5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:57.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:57.823+0000 7f90cfe77640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f90c814fa50 0x7f90c81f56c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:57.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:57.823+0000 7f90cfe77640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f90c815e4b0 con 0x7f90c80b2e40 2024-09-21T03:42:57.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:57.823+0000 7f90cfe77640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f90c80b39d0 con 0x7f90c814fa50 2024-09-21T03:42:57.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:57.823+0000 7f90cfe77640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f90c815e630 con 0x7f90c8162280 2024-09-21T03:42:57.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:57.823+0000 7f90cdc13640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f90c80b2e40 0x7f90c80ac070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:57.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:57.823+0000 7f90cdc13640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f90c80b2e40 0x7f90c80ac070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43950/0 (socket says 172.21.15.38:43950) 2024-09-21T03:42:57.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:57.823+0000 7f90cdc13640 1 -- 172.21.15.38:0/2847849051 learned_addr learned my addr 172.21.15.38:0/2847849051 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:42:57.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:57.823+0000 7f90ce414640 1 --2- 172.21.15.38:0/2847849051 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f90c814fa50 0x7f90c81f56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:57.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:57.823+0000 7f90cd412640 1 --2- 172.21.15.38:0/2847849051 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f90c8162280 0x7f90c81f5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:57.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:57.823+0000 7f90cdc13640 1 -- 172.21.15.38:0/2847849051 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f90c8162280 msgr2=0x7f90c81f5180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:57.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:57.823+0000 7f90cdc13640 1 --2- 172.21.15.38:0/2847849051 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f90c8162280 0x7f90c81f5180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:57.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:57.823+0000 7f90cdc13640 1 -- 172.21.15.38:0/2847849051 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f90c814fa50 msgr2=0x7f90c81f56c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:57.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:57.823+0000 7f90cdc13640 1 --2- 172.21.15.38:0/2847849051 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f90c814fa50 0x7f90c81f56c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:57.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:57.823+0000 7f90cdc13640 1 -- 172.21.15.38:0/2847849051 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90c80ac970 con 0x7f90c80b2e40 2024-09-21T03:42:57.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:57.823+0000 7f90ce414640 1 --2- 172.21.15.38:0/2847849051 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f90c814fa50 0x7f90c81f56c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:42:57.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:57.823+0000 7f90cd412640 1 --2- 172.21.15.38:0/2847849051 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f90c8162280 0x7f90c81f5180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:42:57.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:57.823+0000 7f90cdc13640 1 --2- 172.21.15.38:0/2847849051 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f90c80b2e40 0x7f90c80ac070 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7f90c400c3c0 tx=0x7f90c400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:57.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:57.824+0000 7f90ba7fc640 1 -- 172.21.15.38:0/2847849051 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90c4017070 con 0x7f90c80b2e40 2024-09-21T03:42:57.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:57.824+0000 7f90ba7fc640 1 -- 172.21.15.38:0/2847849051 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f90c40035f0 con 0x7f90c80b2e40 2024-09-21T03:42:57.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:57.824+0000 7f90cfe77640 1 -- 172.21.15.38:0/2847849051 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90c80acc50 con 0x7f90c80b2e40 2024-09-21T03:42:57.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:57.824+0000 7f90ba7fc640 1 -- 172.21.15.38:0/2847849051 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90c4003a80 con 0x7f90c80b2e40 2024-09-21T03:42:57.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:57.824+0000 7f90cfe77640 1 -- 172.21.15.38:0/2847849051 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90c80ad210 con 0x7f90c80b2e40 2024-09-21T03:42:57.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:57.825+0000 7f90cfe77640 1 -- 172.21.15.38:0/2847849051 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90c8162f00 con 0x7f90c80b2e40 2024-09-21T03:42:57.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:57.825+0000 7f90ba7fc640 1 -- 172.21.15.38:0/2847849051 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f90c401a070 con 0x7f90c80b2e40 2024-09-21T03:42:57.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:57.826+0000 7f90ba7fc640 1 --2- 172.21.15.38:0/2847849051 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f90ac078bc0 0x7f90ac07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:57.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:57.826+0000 7f90ba7fc640 1 -- 172.21.15.38:0/2847849051 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 8540+0+0 (secure 0 0 0) 0x7f90c409abb0 con 0x7f90c80b2e40 2024-09-21T03:42:57.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:57.826+0000 7f90cd412640 1 --2- 172.21.15.38:0/2847849051 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f90ac078bc0 0x7f90ac07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:57.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:57.827+0000 7f90cd412640 1 --2- 172.21.15.38:0/2847849051 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f90ac078bc0 0x7f90ac07b0d0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f90bc034000 tx=0x7f90bc004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:57.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:57.831+0000 7f90ba7fc640 1 -- 172.21.15.38:0/2847849051 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f90c8162bd0 con 0x7f90c80b2e40 2024-09-21T03:42:58.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.057+0000 7f90cfe77640 1 -- 172.21.15.38:0/2847849051 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f90c8068f30 con 0x7f90c80b2e40 2024-09-21T03:42:58.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.057+0000 7f90ba7fc640 1 -- 172.21.15.38:0/2847849051 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 246 v246) ==== 95+0+5234 (secure 0 0 0) 0x7f90c4062410 con 0x7f90c80b2e40 2024-09-21T03:42:58.059 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:42:58.060 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":246,"btime":"2024-09-21T03:42:54:500790+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6086,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/965972663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":965972663},{"type":"v1","addr":"172.21.15.195:6836","nonce":965972663}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6700,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/3176122334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":3176122334},{"type":"v1","addr":"172.21.15.195:6837","nonce":3176122334}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":9495,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/2097789246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":2097789246},{"type":"v1","addr":"172.21.15.38:6835","nonce":2097789246}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":244}],"filesystems":[{"mdsmap":{"epoch":246,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:40:02.319282+0000","modified":"2024-09-21T03:42:54.500782+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":122,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9438},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9438":{"gid":9438,"name":"c","rank":0,"incarnation":243,"state":"up:active","state_seq":4,"addr":"172.21.15.38:6833/1995903506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":1995903506},{"type":"v1","addr":"172.21.15.38:6833","nonce":1995903506}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9438,"qdb_cluster":[9438]},"id":6}]} 2024-09-21T03:42:58.060 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 246 2024-09-21T03:42:58.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.059+0000 7f90cfe77640 1 -- 172.21.15.38:0/2847849051 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f90ac078bc0 msgr2=0x7f90ac07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:58.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.059+0000 7f90cfe77640 1 --2- 172.21.15.38:0/2847849051 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f90ac078bc0 0x7f90ac07b0d0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f90bc034000 tx=0x7f90bc004730 comp rx=0 tx=0).stop 2024-09-21T03:42:58.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.059+0000 7f90cfe77640 1 -- 172.21.15.38:0/2847849051 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f90c80b2e40 msgr2=0x7f90c80ac070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:58.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.059+0000 7f90cfe77640 1 --2- 172.21.15.38:0/2847849051 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f90c80b2e40 0x7f90c80ac070 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7f90c400c3c0 tx=0x7f90c400c880 comp rx=0 tx=0).stop 2024-09-21T03:42:58.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.060+0000 7f90cfe77640 1 -- 172.21.15.38:0/2847849051 shutdown_connections 2024-09-21T03:42:58.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.060+0000 7f90cfe77640 1 --2- 172.21.15.38:0/2847849051 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f90c814fa50 0x7f90c81f56c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:58.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.060+0000 7f90cfe77640 1 --2- 172.21.15.38:0/2847849051 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f90ac078bc0 0x7f90ac07b0d0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:58.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.060+0000 7f90cfe77640 1 --2- 172.21.15.38:0/2847849051 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f90c8162280 0x7f90c81f5180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:58.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.060+0000 7f90cfe77640 1 --2- 172.21.15.38:0/2847849051 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f90c80b2e40 0x7f90c80ac070 unknown :-1 s=CLOSED pgs=1881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:58.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.060+0000 7f90cfe77640 1 -- 172.21.15.38:0/2847849051 >> 172.21.15.38:0/2847849051 conn(0x7f90c80b6b20 msgr2=0x7f90c815db10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:58.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.060+0000 7f90cfe77640 1 -- 172.21.15.38:0/2847849051 shutdown_connections 2024-09-21T03:42:58.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.060+0000 7f90cfe77640 1 -- 172.21.15.38:0/2847849051 wait complete. 2024-09-21T03:42:58.073 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 246, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T03:40:02.319282+0000', 'modified': '2024-09-21T03:42:54.500782+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 122, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9438}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9438': {'gid': 9438, 'name': 'c', 'rank': 0, 'incarnation': 243, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.38:6833/1995903506', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.38:6832', 'nonce': 1995903506}, {'type': 'v1', 'addr': '172.21.15.38:6833', 'nonce': 1995903506}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 9438, 'qdb_cluster': [9438]} 2024-09-21T03:42:58.073 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-21T03:42:58.073 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-21T03:42:58.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.188+0000 7f17e49ad640 1 -- 172.21.15.38:0/2796738734 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f17e015ee80 msgr2=0x7f17e00d0170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:58.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.188+0000 7f17e49ad640 1 --2- 172.21.15.38:0/2796738734 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f17e015ee80 0x7f17e00d0170 secure :-1 s=READY pgs=1883 cs=0 l=1 rev1=1 crypto rx=0x7f17d0009830 tx=0x7f17d002a3f0 comp rx=0 tx=0).stop 2024-09-21T03:42:58.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.188+0000 7f17e49ad640 1 -- 172.21.15.38:0/2796738734 shutdown_connections 2024-09-21T03:42:58.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.189+0000 7f17e49ad640 1 --2- 172.21.15.38:0/2796738734 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f17e015ee80 0x7f17e00d0170 unknown :-1 s=CLOSED pgs=1883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:58.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.189+0000 7f17e49ad640 1 -- 172.21.15.38:0/2796738734 >> 172.21.15.38:0/2796738734 conn(0x7f17e009d830 msgr2=0x7f17e00b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:58.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.189+0000 7f17e49ad640 1 -- 172.21.15.38:0/2796738734 shutdown_connections 2024-09-21T03:42:58.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.189+0000 7f17e49ad640 1 -- 172.21.15.38:0/2796738734 wait complete. 2024-09-21T03:42:58.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.190+0000 7f17e49ad640 1 Processor -- start 2024-09-21T03:42:58.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.191+0000 7f17e49ad640 1 -- start start 2024-09-21T03:42:58.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.192+0000 7f17e49ad640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f17e00b9ad0 0x7f17e01509a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:58.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.192+0000 7f17e49ad640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f17e00ceb90 0x7f17e006ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:58.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.192+0000 7f17e49ad640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f17e015ee80 0x7f17e006f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:58.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.192+0000 7f17e49ad640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f17e00b6910 con 0x7f17e00ceb90 2024-09-21T03:42:58.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.192+0000 7f17e49ad640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f17e00b6790 con 0x7f17e00b9ad0 2024-09-21T03:42:58.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.192+0000 7f17e49ad640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f17e00b6a90 con 0x7f17e015ee80 2024-09-21T03:42:58.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.192+0000 7f17ddd9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f17e00ceb90 0x7f17e006ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:58.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.192+0000 7f17ddd9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f17e00ceb90 0x7f17e006ee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43968/0 (socket says 172.21.15.38:43968) 2024-09-21T03:42:58.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.192+0000 7f17ddd9b640 1 -- 172.21.15.38:0/1906408833 learned_addr learned my addr 172.21.15.38:0/1906408833 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:42:58.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.192+0000 7f17de59c640 1 --2- 172.21.15.38:0/1906408833 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f17e00b9ad0 0x7f17e01509a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:58.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.192+0000 7f17ded9d640 1 --2- 172.21.15.38:0/1906408833 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f17e015ee80 0x7f17e006f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:58.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.192+0000 7f17ddd9b640 1 -- 172.21.15.38:0/1906408833 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f17e015ee80 msgr2=0x7f17e006f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:58.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.192+0000 7f17ddd9b640 1 --2- 172.21.15.38:0/1906408833 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f17e015ee80 0x7f17e006f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:58.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.192+0000 7f17ddd9b640 1 -- 172.21.15.38:0/1906408833 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f17e00b9ad0 msgr2=0x7f17e01509a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:58.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.192+0000 7f17ddd9b640 1 --2- 172.21.15.38:0/1906408833 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f17e00b9ad0 0x7f17e01509a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:58.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.192+0000 7f17ddd9b640 1 -- 172.21.15.38:0/1906408833 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17e0150ee0 con 0x7f17e00ceb90 2024-09-21T03:42:58.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.192+0000 7f17ddd9b640 1 --2- 172.21.15.38:0/1906408833 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f17e00ceb90 0x7f17e006ee10 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7f17d400b410 tx=0x7f17d4009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:58.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.193+0000 7f17ceffd640 1 -- 172.21.15.38:0/1906408833 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17d4015030 con 0x7f17e00ceb90 2024-09-21T03:42:58.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.193+0000 7f17ceffd640 1 -- 172.21.15.38:0/1906408833 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f17d4005a70 con 0x7f17e00ceb90 2024-09-21T03:42:58.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.193+0000 7f17e49ad640 1 -- 172.21.15.38:0/1906408833 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17e0159c90 con 0x7f17e00ceb90 2024-09-21T03:42:58.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.193+0000 7f17ceffd640 1 -- 172.21.15.38:0/1906408833 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17d4006020 con 0x7f17e00ceb90 2024-09-21T03:42:58.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.193+0000 7f17e49ad640 1 -- 172.21.15.38:0/1906408833 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17e015a250 con 0x7f17e00ceb90 2024-09-21T03:42:58.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.195+0000 7f17ceffd640 1 -- 172.21.15.38:0/1906408833 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f17d4018070 con 0x7f17e00ceb90 2024-09-21T03:42:58.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.195+0000 7f17ceffd640 1 --2- 172.21.15.38:0/1906408833 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f17c0078bc0 0x7f17c007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:58.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.195+0000 7f17ceffd640 1 -- 172.21.15.38:0/1906408833 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 8540+0+0 (secure 0 0 0) 0x7f17d409ae80 con 0x7f17e00ceb90 2024-09-21T03:42:58.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.196+0000 7f17de59c640 1 --2- 172.21.15.38:0/1906408833 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f17c0078bc0 0x7f17c007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:58.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.196+0000 7f17eb9fa740 1 Processor -- start 2024-09-21T03:42:58.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.196+0000 7f17eb9fa740 1 -- start start 2024-09-21T03:42:58.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.196+0000 7f17de59c640 1 --2- 172.21.15.38:0/1906408833 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f17c0078bc0 0x7f17c007b0d0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f17d0034000 tx=0x7f17d0004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:58.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.197+0000 7f17eb9fa740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562d01a9f270 0x562d01e70fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:58.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.197+0000 7f17eb9fa740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562d01a763b0 0x562d01e71500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:58.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.197+0000 7f17eb9fa740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562d01a7e590 0x562d01e8e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:58.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.197+0000 7f17eb9fa740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x562d01db9820 con 0x562d01a7e590 2024-09-21T03:42:58.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.197+0000 7f17eb9fa740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x562d01da8c50 con 0x562d01a763b0 2024-09-21T03:42:58.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.197+0000 7f17eb9fa740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x562d01daca60 con 0x562d01a9f270 2024-09-21T03:42:58.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.197+0000 7f17ded9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562d01a7e590 0x562d01e8e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:58.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.197+0000 7f17ded9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562d01a7e590 0x562d01e8e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43970/0 (socket says 172.21.15.38:43970) 2024-09-21T03:42:58.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.197+0000 7f17ded9d640 1 -- 172.21.15.38:0/3620295342 learned_addr learned my addr 172.21.15.38:0/3620295342 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:42:58.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.198+0000 7f17ddd9b640 1 --2- 172.21.15.38:0/3620295342 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562d01a763b0 0x562d01e71500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:58.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.198+0000 7f17ded9d640 1 -- 172.21.15.38:0/3620295342 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562d01a9f270 msgr2=0x562d01e70fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:58.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.198+0000 7f17ded9d640 1 --2- 172.21.15.38:0/3620295342 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562d01a9f270 0x562d01e70fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:58.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.198+0000 7f17ded9d640 1 -- 172.21.15.38:0/3620295342 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562d01a763b0 msgr2=0x562d01e71500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:58.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.198+0000 7f17ded9d640 1 --2- 172.21.15.38:0/3620295342 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562d01a763b0 0x562d01e71500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:58.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.198+0000 7f17ded9d640 1 -- 172.21.15.38:0/3620295342 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d01b0a850 con 0x562d01a7e590 2024-09-21T03:42:58.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.198+0000 7f17ded9d640 1 --2- 172.21.15.38:0/3620295342 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562d01a7e590 0x562d01e8e400 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7f17d8001e50 tx=0x7f17d80015b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fa76394ae591eba server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:58.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.198+0000 7f17bffff640 1 -- 172.21.15.38:0/3620295342 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17d800fe50 con 0x562d01a7e590 2024-09-21T03:42:58.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.198+0000 7f17bffff640 1 -- 172.21.15.38:0/3620295342 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f17d8018070 con 0x562d01a7e590 2024-09-21T03:42:58.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.198+0000 7f17bffff640 1 -- 172.21.15.38:0/3620295342 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17d8003bb0 con 0x562d01a7e590 2024-09-21T03:42:58.202 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.199+0000 7f17eb9fa740 1 -- 172.21.15.38:0/3620295342 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562d01a7e590 msgr2=0x562d01e8e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:58.202 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.199+0000 7f17eb9fa740 1 --2- 172.21.15.38:0/3620295342 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562d01a7e590 0x562d01e8e400 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7f17d8001e50 tx=0x7f17d80015b0 comp rx=0 tx=0).stop 2024-09-21T03:42:58.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.199+0000 7f17eb9fa740 1 -- 172.21.15.38:0/3620295342 shutdown_connections 2024-09-21T03:42:58.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.199+0000 7f17eb9fa740 1 --2- 172.21.15.38:0/3620295342 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562d01a7e590 0x562d01e8e400 unknown :-1 s=CLOSED pgs=1885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:58.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.199+0000 7f17eb9fa740 1 --2- 172.21.15.38:0/3620295342 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562d01a763b0 0x562d01e71500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:58.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.199+0000 7f17eb9fa740 1 --2- 172.21.15.38:0/3620295342 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562d01a9f270 0x562d01e70fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:58.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.199+0000 7f17eb9fa740 1 -- 172.21.15.38:0/3620295342 >> 172.21.15.38:0/3620295342 conn(0x562d01a933c0 msgr2=0x562d01e47280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:58.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.199+0000 7f17eb9fa740 1 -- 172.21.15.38:0/3620295342 shutdown_connections 2024-09-21T03:42:58.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.199+0000 7f17eb9fa740 1 -- 172.21.15.38:0/3620295342 wait complete. 2024-09-21T03:42:58.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.200+0000 7f17eb9fa740 1 -- start start 2024-09-21T03:42:58.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.200+0000 7f17eb9fa740 1 Processor -- start 2024-09-21T03:42:58.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.202+0000 7f17eb9fa740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562d01a763b0 0x562d01e83c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:58.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.202+0000 7f17eb9fa740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562d01a7e590 0x562d01e852b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:58.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.202+0000 7f17eb9fa740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562d01a9f270 0x562d01e6ff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:58.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.202+0000 7f17eb9fa740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x562d01b19620 con 0x562d01a7e590 2024-09-21T03:42:58.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.202+0000 7f17eb9fa740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x562d01d4e5f0 con 0x562d01a9f270 2024-09-21T03:42:58.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.202+0000 7f17eb9fa740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x562d01e71a40 con 0x562d01a763b0 2024-09-21T03:42:58.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.202+0000 7f17eb9fa740 10 client.0 _ll_register_callbacks cb 0x562d01d520a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:42:58.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.202+0000 7f17ddd9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562d01a7e590 0x562d01e852b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:58.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.202+0000 7f17ddd9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562d01a7e590 0x562d01e852b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43974/0 (socket says 172.21.15.38:43974) 2024-09-21T03:42:58.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.202+0000 7f17ddd9b640 1 -- 172.21.15.38:0/3768003804 learned_addr learned my addr 172.21.15.38:0/3768003804 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:42:58.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.202+0000 7f17de59c640 1 --2- 172.21.15.38:0/3768003804 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562d01a763b0 0x562d01e83c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:58.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.202+0000 7f17ded9d640 1 --2- 172.21.15.38:0/3768003804 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562d01a9f270 0x562d01e6ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:58.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.202+0000 7f17ddd9b640 1 -- 172.21.15.38:0/3768003804 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562d01a763b0 msgr2=0x562d01e83c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:58.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.202+0000 7f17ddd9b640 1 --2- 172.21.15.38:0/3768003804 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562d01a763b0 0x562d01e83c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:58.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.202+0000 7f17ddd9b640 1 -- 172.21.15.38:0/3768003804 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562d01a9f270 msgr2=0x562d01e6ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:58.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.202+0000 7f17ddd9b640 1 --2- 172.21.15.38:0/3768003804 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562d01a9f270 0x562d01e6ff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:58.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.202+0000 7f17ddd9b640 1 -- 172.21.15.38:0/3768003804 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d01b816b0 con 0x562d01a7e590 2024-09-21T03:42:58.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.202+0000 7f17ddd9b640 1 --2- 172.21.15.38:0/3768003804 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562d01a7e590 0x562d01e852b0 secure :-1 s=READY pgs=1886 cs=0 l=1 rev1=1 crypto rx=0x7f17d400b3e0 tx=0x7f17d4062000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f17e9c159884a0a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:58.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.202+0000 7f17bd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-21T03:42:58.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.202+0000 7f17bd7fa640 1 -- 172.21.15.38:0/3768003804 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17d406ba00 con 0x562d01a7e590 2024-09-21T03:42:58.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.202+0000 7f17bd7fa640 1 -- 172.21.15.38:0/3768003804 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(122..122 src has 1..122) ==== 8540+0+0 (secure 0 0 0) 0x7f17d4088170 con 0x562d01a7e590 2024-09-21T03:42:58.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.202+0000 7f17eb9fa740 1 -- 172.21.15.38:0/3768003804 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17d8018070 con 0x562d01a7e590 2024-09-21T03:42:58.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.202+0000 7f17bd7fa640 1 -- 172.21.15.38:0/3768003804 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=246) ==== 24+0+0 (secure 0 0 0) 0x7f17d4088370 con 0x562d01a7e590 2024-09-21T03:42:58.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.203+0000 7f17eb9fa740 10 client.9513 fetch_fsmap learned FSMap version 246 2024-09-21T03:42:58.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.203+0000 7f17eb9fa740 1 -- 172.21.15.38:0/3768003804 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=246,monmap=2+}) -- 0x562d01e9ecb0 con 0x562d01a7e590 2024-09-21T03:42:58.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.203+0000 7f17bd7fa640 1 -- 172.21.15.38:0/3768003804 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f17d8018070 con 0x562d01a7e590 2024-09-21T03:42:58.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.203+0000 7f17bd7fa640 1 -- 172.21.15.38:0/3768003804 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 246) ==== 3502+0+0 (secure 0 0 0) 0x7f17d4067ce0 con 0x562d01a7e590 2024-09-21T03:42:58.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.203+0000 7f17eb9fa740 10 client.9513 fetch_fsmap finished waiting for FSMap version 246 2024-09-21T03:42:58.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.203+0000 7f17eb9fa740 10 client.9513 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T03:42:58.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.203+0000 7f17eb9fa740 20 client.9513 populate_metadata read hostname 'smithi038' 2024-09-21T03:42:58.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.203+0000 7f17eb9fa740 1 --2- 172.21.15.38:0/3768003804 >> [v2:172.21.15.38:6832/1995903506,v1:172.21.15.38:6833/1995903506] conn(0x562d01a5de40 0x562d01e8f350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:58.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.203+0000 7f17eb9fa740 4 client.9513 mds_command: new command op to 9438 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:42:58.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.203+0000 7f17eb9fa740 1 -- 172.21.15.38:0/3768003804 --> [v2:172.21.15.38:6832/1995903506,v1:172.21.15.38:6833/1995903506] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562d01dbb770 con 0x562d01a5de40 2024-09-21T03:42:58.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.221+0000 7f17de59c640 1 --2- 172.21.15.38:0/3768003804 >> [v2:172.21.15.38:6832/1995903506,v1:172.21.15.38:6833/1995903506] conn(0x562d01a5de40 0x562d01e8f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:58.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.221+0000 7f17de59c640 1 --2- 172.21.15.38:0/3768003804 >> [v2:172.21.15.38:6832/1995903506,v1:172.21.15.38:6833/1995903506] conn(0x562d01a5de40 0x562d01e8f350 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f3f03419ee1c29df server_cookie=af78db3c9122162b in_seq=0 out_seq=0 2024-09-21T03:42:58.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.221+0000 7f17bd7fa640 10 client.9513 ms_handle_connect on v2:172.21.15.38:6832/1995903506 2024-09-21T03:42:58.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.222+0000 7f17bd7fa640 1 -- 172.21.15.38:0/3768003804 <== mds.0 v2:172.21.15.38:6832/1995903506 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562d01dbb770 con 0x562d01a5de40 2024-09-21T03:42:58.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.222+0000 7f17bd7fa640 10 client.9513 handle_command_reply: tid=0 2024-09-21T03:42:58.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.222+0000 7f17eb9fa740 1 client.9513 shutdown 2024-09-21T03:42:58.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.222+0000 7f17eb9fa740 1 -- 172.21.15.38:0/3768003804 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562d01a7e590 msgr2=0x562d01e852b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:58.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.222+0000 7f17eb9fa740 1 --2- 172.21.15.38:0/3768003804 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562d01a7e590 0x562d01e852b0 secure :-1 s=READY pgs=1886 cs=0 l=1 rev1=1 crypto rx=0x7f17d400b3e0 tx=0x7f17d4062000 comp rx=0 tx=0).stop 2024-09-21T03:42:58.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.222+0000 7f17eb9fa740 1 -- 172.21.15.38:0/3768003804 shutdown_connections 2024-09-21T03:42:58.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.222+0000 7f17eb9fa740 1 --2- 172.21.15.38:0/3768003804 >> [v2:172.21.15.38:6832/1995903506,v1:172.21.15.38:6833/1995903506] conn(0x562d01a5de40 0x562d01e8f350 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:58.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.222+0000 7f17eb9fa740 1 --2- 172.21.15.38:0/3768003804 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562d01a9f270 0x562d01e6ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:58.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.222+0000 7f17eb9fa740 1 --2- 172.21.15.38:0/3768003804 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562d01a7e590 0x562d01e852b0 unknown :-1 s=CLOSED pgs=1886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:58.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.222+0000 7f17bd7fa640 0 client.9513 ms_handle_reset on v2:172.21.15.38:6832/1995903506 2024-09-21T03:42:58.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.222+0000 7f17eb9fa740 1 --2- 172.21.15.38:0/3768003804 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562d01a763b0 0x562d01e83c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:58.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.222+0000 7f17eb9fa740 1 -- 172.21.15.38:0/3768003804 >> 172.21.15.38:0/3768003804 conn(0x562d01a933c0 msgr2=0x562d01e8d9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:58.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.222+0000 7f17eb9fa740 1 -- 172.21.15.38:0/3768003804 shutdown_connections 2024-09-21T03:42:58.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.222+0000 7f17eb9fa740 1 -- 172.21.15.38:0/3768003804 wait complete. 2024-09-21T03:42:58.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.222+0000 7f17eb9fa740 20 client.9513 trim_cache size 0 max 16384 2024-09-21T03:42:58.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.241+0000 7f17eb9fa740 1 Processor -- start 2024-09-21T03:42:58.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.241+0000 7f17eb9fa740 1 -- start start 2024-09-21T03:42:58.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.242+0000 7f17eb9fa740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562d01a763b0 0x562d01e41910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:58.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.242+0000 7f17eb9fa740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562d01a7e590 0x562d01e41e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:58.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.242+0000 7f17eb9fa740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562d01a9f270 0x562d01e84b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:58.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.242+0000 7f17eb9fa740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f17e0202e10 con 0x562d01a763b0 2024-09-21T03:42:58.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.242+0000 7f17eb9fa740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f17e0202110 con 0x562d01a7e590 2024-09-21T03:42:58.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.242+0000 7f17eb9fa740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f17e0202760 con 0x562d01a9f270 2024-09-21T03:42:58.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.242+0000 7f17de59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562d01a763b0 0x562d01e41910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:58.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.242+0000 7f17de59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562d01a763b0 0x562d01e41910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43978/0 (socket says 172.21.15.38:43978) 2024-09-21T03:42:58.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.242+0000 7f17de59c640 1 -- 172.21.15.38:0/614753625 learned_addr learned my addr 172.21.15.38:0/614753625 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:42:58.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.242+0000 7f17ded9d640 1 --2- 172.21.15.38:0/614753625 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562d01a9f270 0x562d01e84b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:58.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.242+0000 7f17ddd9b640 1 --2- 172.21.15.38:0/614753625 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562d01a7e590 0x562d01e41e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:58.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.242+0000 7f17de59c640 1 -- 172.21.15.38:0/614753625 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562d01a9f270 msgr2=0x562d01e84b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:58.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.242+0000 7f17de59c640 1 --2- 172.21.15.38:0/614753625 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562d01a9f270 0x562d01e84b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:58.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.242+0000 7f17de59c640 1 -- 172.21.15.38:0/614753625 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562d01a7e590 msgr2=0x562d01e41e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:58.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.242+0000 7f17de59c640 1 --2- 172.21.15.38:0/614753625 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562d01a7e590 0x562d01e41e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:58.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.242+0000 7f17de59c640 1 -- 172.21.15.38:0/614753625 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d01e88340 con 0x562d01a763b0 2024-09-21T03:42:58.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.243+0000 7f17de59c640 1 --2- 172.21.15.38:0/614753625 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562d01a763b0 0x562d01e41910 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7f17d0009920 tx=0x7f17d003e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cc00fe0f7a715f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:58.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.243+0000 7f17bf7fe640 1 -- 172.21.15.38:0/614753625 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17d0050640 con 0x562d01a763b0 2024-09-21T03:42:58.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.243+0000 7f17bf7fe640 1 -- 172.21.15.38:0/614753625 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f17d00508d0 con 0x562d01a763b0 2024-09-21T03:42:58.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.243+0000 7f17bf7fe640 1 -- 172.21.15.38:0/614753625 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17d0050cc0 con 0x562d01a763b0 2024-09-21T03:42:58.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.244+0000 7f17eb9fa740 1 -- 172.21.15.38:0/614753625 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562d01a763b0 msgr2=0x562d01e41910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:58.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.244+0000 7f17eb9fa740 1 --2- 172.21.15.38:0/614753625 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562d01a763b0 0x562d01e41910 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7f17d0009920 tx=0x7f17d003e070 comp rx=0 tx=0).stop 2024-09-21T03:42:58.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.244+0000 7f17eb9fa740 1 -- 172.21.15.38:0/614753625 shutdown_connections 2024-09-21T03:42:58.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.244+0000 7f17eb9fa740 1 --2- 172.21.15.38:0/614753625 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562d01a9f270 0x562d01e84b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:58.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.244+0000 7f17eb9fa740 1 --2- 172.21.15.38:0/614753625 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562d01a7e590 0x562d01e41e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:58.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.244+0000 7f17eb9fa740 1 --2- 172.21.15.38:0/614753625 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562d01a763b0 0x562d01e41910 unknown :-1 s=CLOSED pgs=1887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:58.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.244+0000 7f17eb9fa740 1 -- 172.21.15.38:0/614753625 >> 172.21.15.38:0/614753625 conn(0x562d01a933c0 msgr2=0x562d01e877c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:58.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.244+0000 7f17eb9fa740 1 -- 172.21.15.38:0/614753625 shutdown_connections 2024-09-21T03:42:58.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.244+0000 7f17eb9fa740 1 -- 172.21.15.38:0/614753625 wait complete. 2024-09-21T03:42:58.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.244+0000 7f17eb9fa740 1 -- start start 2024-09-21T03:42:58.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.245+0000 7f17eb9fa740 1 Processor -- start 2024-09-21T03:42:58.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.246+0000 7f17eb9fa740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562d01a763b0 0x562d01ea1b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:58.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.246+0000 7f17eb9fa740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562d01a7e590 0x562d01ef4c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:58.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.246+0000 7f17eb9fa740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562d01a9f270 0x562d01e41910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:58.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.246+0000 7f17eb9fa740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x562d01e84710 con 0x562d01a9f270 2024-09-21T03:42:58.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.246+0000 7f17eb9fa740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x562d01e9eff0 con 0x562d01a7e590 2024-09-21T03:42:58.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.246+0000 7f17eb9fa740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x562d01e9b730 con 0x562d01a763b0 2024-09-21T03:42:58.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.246+0000 7f17eb9fa740 10 client.0 _ll_register_callbacks cb 0x7f17d405e770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:42:58.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.246+0000 7f17ded9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562d01a9f270 0x562d01e41910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:58.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.246+0000 7f17ded9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562d01a9f270 0x562d01e41910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43986/0 (socket says 172.21.15.38:43986) 2024-09-21T03:42:58.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.246+0000 7f17ded9d640 1 -- 172.21.15.38:0/2958113614 learned_addr learned my addr 172.21.15.38:0/2958113614 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:42:58.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.246+0000 7f17de59c640 1 --2- 172.21.15.38:0/2958113614 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562d01a763b0 0x562d01ea1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:58.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.246+0000 7f17ddd9b640 1 --2- 172.21.15.38:0/2958113614 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562d01a7e590 0x562d01ef4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:58.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.247+0000 7f17ded9d640 1 -- 172.21.15.38:0/2958113614 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562d01a763b0 msgr2=0x562d01ea1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:58.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.247+0000 7f17ded9d640 1 --2- 172.21.15.38:0/2958113614 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562d01a763b0 0x562d01ea1b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:58.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.247+0000 7f17ded9d640 1 -- 172.21.15.38:0/2958113614 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562d01a7e590 msgr2=0x562d01ef4c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:58.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.247+0000 7f17ded9d640 1 --2- 172.21.15.38:0/2958113614 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562d01a7e590 0x562d01ef4c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:58.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.247+0000 7f17ded9d640 1 -- 172.21.15.38:0/2958113614 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d01ef5140 con 0x562d01a9f270 2024-09-21T03:42:58.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.247+0000 7f17ded9d640 1 --2- 172.21.15.38:0/2958113614 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562d01a9f270 0x562d01e41910 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7f17d8015040 tx=0x7f17d8002190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=628bd8580c233ec4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:58.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.247+0000 7f17bd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-21T03:42:58.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.247+0000 7f17bd7fa640 1 -- 172.21.15.38:0/2958113614 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17d8026020 con 0x562d01a9f270 2024-09-21T03:42:58.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.247+0000 7f17bd7fa640 1 -- 172.21.15.38:0/2958113614 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(122..122 src has 1..122) ==== 8540+0+0 (secure 0 0 0) 0x7f17d80054e0 con 0x562d01a9f270 2024-09-21T03:42:58.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.247+0000 7f17eb9fa740 1 -- 172.21.15.38:0/2958113614 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17d00508d0 con 0x562d01a9f270 2024-09-21T03:42:58.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.247+0000 7f17bd7fa640 1 -- 172.21.15.38:0/2958113614 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=246) ==== 24+0+0 (secure 0 0 0) 0x7f17d8031050 con 0x562d01a9f270 2024-09-21T03:42:58.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.247+0000 7f17eb9fa740 10 client.9519 fetch_fsmap learned FSMap version 246 2024-09-21T03:42:58.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.247+0000 7f17eb9fa740 1 -- 172.21.15.38:0/2958113614 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=246,monmap=2+}) -- 0x562d01eaf100 con 0x562d01a9f270 2024-09-21T03:42:58.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.247+0000 7f17bd7fa640 1 -- 172.21.15.38:0/2958113614 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f17d00508d0 con 0x562d01a9f270 2024-09-21T03:42:58.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.247+0000 7f17bd7fa640 1 -- 172.21.15.38:0/2958113614 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 246) ==== 3502+0+0 (secure 0 0 0) 0x7f17d801e410 con 0x562d01a9f270 2024-09-21T03:42:58.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.247+0000 7f17eb9fa740 10 client.9519 fetch_fsmap finished waiting for FSMap version 246 2024-09-21T03:42:58.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.247+0000 7f17eb9fa740 10 client.9519 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T03:42:58.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.247+0000 7f17eb9fa740 20 client.9519 populate_metadata read hostname 'smithi038' 2024-09-21T03:42:58.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.247+0000 7f17eb9fa740 1 --2- 172.21.15.38:0/2958113614 >> [v2:172.21.15.38:6832/1995903506,v1:172.21.15.38:6833/1995903506] conn(0x562d01f0fd30 0x562d01e99640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:58.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.247+0000 7f17eb9fa740 4 client.9519 mds_command: new command op to 9438 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T03:42:58.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.247+0000 7f17eb9fa740 1 -- 172.21.15.38:0/2958113614 --> [v2:172.21.15.38:6832/1995903506,v1:172.21.15.38:6833/1995903506] -- command(tid 0: {"prefix": "status"}) -- 0x562d01db7220 con 0x562d01f0fd30 2024-09-21T03:42:58.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.248+0000 7f17de59c640 1 --2- 172.21.15.38:0/2958113614 >> [v2:172.21.15.38:6832/1995903506,v1:172.21.15.38:6833/1995903506] conn(0x562d01f0fd30 0x562d01e99640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:58.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.248+0000 7f17bd7fa640 10 client.9519 ms_handle_connect on v2:172.21.15.38:6832/1995903506 2024-09-21T03:42:58.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.248+0000 7f17de59c640 1 --2- 172.21.15.38:0/2958113614 >> [v2:172.21.15.38:6832/1995903506,v1:172.21.15.38:6833/1995903506] conn(0x562d01f0fd30 0x562d01e99640 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3fc18c9078e157f7 server_cookie=b6ef4e8b5660dfa9 in_seq=0 out_seq=0 2024-09-21T03:42:58.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.248+0000 7f17bd7fa640 1 -- 172.21.15.38:0/2958113614 <== mds.0 v2:172.21.15.38:6832/1995903506 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x562d01db7220 con 0x562d01f0fd30 2024-09-21T03:42:58.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.248+0000 7f17bd7fa640 10 client.9519 handle_command_reply: tid=0 2024-09-21T03:42:58.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.248+0000 7f17eb9fa740 1 client.9519 shutdown 2024-09-21T03:42:58.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.248+0000 7f17eb9fa740 1 -- 172.21.15.38:0/2958113614 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562d01a9f270 msgr2=0x562d01e41910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:58.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.248+0000 7f17eb9fa740 1 --2- 172.21.15.38:0/2958113614 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562d01a9f270 0x562d01e41910 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7f17d8015040 tx=0x7f17d8002190 comp rx=0 tx=0).stop 2024-09-21T03:42:58.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.249+0000 7f17eb9fa740 1 -- 172.21.15.38:0/2958113614 shutdown_connections 2024-09-21T03:42:58.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.249+0000 7f17eb9fa740 1 --2- 172.21.15.38:0/2958113614 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562d01a9f270 0x562d01e41910 unknown :-1 s=CLOSED pgs=1888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:58.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.249+0000 7f17eb9fa740 1 --2- 172.21.15.38:0/2958113614 >> [v2:172.21.15.38:6832/1995903506,v1:172.21.15.38:6833/1995903506] conn(0x562d01f0fd30 0x562d01e99640 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:58.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.249+0000 7f17eb9fa740 1 --2- 172.21.15.38:0/2958113614 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562d01a7e590 0x562d01ef4c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:58.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.249+0000 7f17eb9fa740 1 --2- 172.21.15.38:0/2958113614 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562d01a763b0 0x562d01ea1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:58.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.249+0000 7f17eb9fa740 1 -- 172.21.15.38:0/2958113614 >> 172.21.15.38:0/2958113614 conn(0x562d01a933c0 msgr2=0x562d01e5ef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:58.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.249+0000 7f17bd7fa640 0 client.9519 ms_handle_reset on v2:172.21.15.38:6832/1995903506 2024-09-21T03:42:58.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.249+0000 7f17eb9fa740 1 -- 172.21.15.38:0/2958113614 shutdown_connections 2024-09-21T03:42:58.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.249+0000 7f17eb9fa740 1 -- 172.21.15.38:0/2958113614 wait complete. 2024-09-21T03:42:58.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.249+0000 7f17eb9fa740 20 client.9519 trim_cache size 0 max 16384 2024-09-21T03:42:58.256 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-09-21T03:42:58.256 INFO:teuthology.orchestra.run.smithi038.stdout: "cluster_fsid": "9d50d66f-656b-4fe2-8e3b-fe53b79c4d43", 2024-09-21T03:42:58.256 INFO:teuthology.orchestra.run.smithi038.stdout: "whoami": 0, 2024-09-21T03:42:58.256 INFO:teuthology.orchestra.run.smithi038.stdout: "id": 9438, 2024-09-21T03:42:58.256 INFO:teuthology.orchestra.run.smithi038.stdout: "want_state": "up:active", 2024-09-21T03:42:58.256 INFO:teuthology.orchestra.run.smithi038.stdout: "state": "up:active", 2024-09-21T03:42:58.256 INFO:teuthology.orchestra.run.smithi038.stdout: "fs_name": "cephfs", 2024-09-21T03:42:58.256 INFO:teuthology.orchestra.run.smithi038.stdout: "rank_uptime": 6.761643544, 2024-09-21T03:42:58.256 INFO:teuthology.orchestra.run.smithi038.stdout: "mdsmap_epoch": 246, 2024-09-21T03:42:58.256 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch": 122, 2024-09-21T03:42:58.256 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch_barrier": 122, 2024-09-21T03:42:58.256 INFO:teuthology.orchestra.run.smithi038.stdout: "uptime": 10.046179757999999 2024-09-21T03:42:58.257 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-09-21T03:42:58.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.249+0000 7f17e49ad640 1 -- 172.21.15.38:0/1906408833 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f17c0078bc0 msgr2=0x7f17c007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:58.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.249+0000 7f17e49ad640 1 --2- 172.21.15.38:0/1906408833 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f17c0078bc0 0x7f17c007b0d0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f17d0034000 tx=0x7f17d0004730 comp rx=0 tx=0).stop 2024-09-21T03:42:58.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.249+0000 7f17e49ad640 1 -- 172.21.15.38:0/1906408833 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f17e00ceb90 msgr2=0x7f17e006ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:58.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.249+0000 7f17e49ad640 1 --2- 172.21.15.38:0/1906408833 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f17e00ceb90 0x7f17e006ee10 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7f17d400b410 tx=0x7f17d4009f90 comp rx=0 tx=0).stop 2024-09-21T03:42:58.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.249+0000 7f17e49ad640 1 -- 172.21.15.38:0/1906408833 shutdown_connections 2024-09-21T03:42:58.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.249+0000 7f17e49ad640 1 --2- 172.21.15.38:0/1906408833 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f17c0078bc0 0x7f17c007b0d0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:58.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.249+0000 7f17e49ad640 1 --2- 172.21.15.38:0/1906408833 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f17e015ee80 0x7f17e006f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:58.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.249+0000 7f17e49ad640 1 --2- 172.21.15.38:0/1906408833 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f17e00ceb90 0x7f17e006ee10 unknown :-1 s=CLOSED pgs=1884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:58.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.249+0000 7f17e49ad640 1 --2- 172.21.15.38:0/1906408833 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f17e00b9ad0 0x7f17e01509a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:58.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.249+0000 7f17e49ad640 1 -- 172.21.15.38:0/1906408833 >> 172.21.15.38:0/1906408833 conn(0x7f17e009d830 msgr2=0x7f17e00b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:58.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.249+0000 7f17e49ad640 1 -- 172.21.15.38:0/1906408833 shutdown_connections 2024-09-21T03:42:58.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.249+0000 7f17e49ad640 1 -- 172.21.15.38:0/1906408833 wait complete. 2024-09-21T03:42:58.262 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-09-21T03:42:58.263 INFO:tasks.cephfs.test_damage:Daemons came up after mutation 'Truncate 500.00000000', proceeding to ls 2024-09-21T03:42:58.263 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T03:42:58.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.377+0000 7ff719c25640 1 -- 172.21.15.38:0/1081562416 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff7140b2e60 msgr2=0x7ff7140b3270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:58.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.377+0000 7ff719c25640 1 --2- 172.21.15.38:0/1081562416 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff7140b2e60 0x7ff7140b3270 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ff70c005fd0 tx=0x7ff70c02aa90 comp rx=0 tx=0).stop 2024-09-21T03:42:58.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.377+0000 7ff719c25640 1 -- 172.21.15.38:0/1081562416 shutdown_connections 2024-09-21T03:42:58.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.377+0000 7ff719c25640 1 --2- 172.21.15.38:0/1081562416 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff7140b2e60 0x7ff7140b3270 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:58.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.377+0000 7ff719c25640 1 --2- 172.21.15.38:0/1081562416 >> v2:172.21.15.195:3300/0 conn(0x7ff7140cfe00 0x7ff7140d2a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:58.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.377+0000 7ff719c25640 1 -- 172.21.15.38:0/1081562416 >> 172.21.15.38:0/1081562416 conn(0x7ff7140d4440 msgr2=0x7ff7140d4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:58.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.377+0000 7ff719c25640 1 -- 172.21.15.38:0/1081562416 shutdown_connections 2024-09-21T03:42:58.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.377+0000 7ff719c25640 1 -- 172.21.15.38:0/1081562416 wait complete. 2024-09-21T03:42:58.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.379+0000 7ff719c25640 1 Processor -- start 2024-09-21T03:42:58.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.379+0000 7ff719c25640 1 -- start start 2024-09-21T03:42:58.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.381+0000 7ff719c25640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff7140b2e60 0x7ff714155000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:58.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.381+0000 7ff719c25640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff7140cfe00 0x7ff7141f8ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:58.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.381+0000 7ff719c25640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff714162590 0x7ff714157520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:58.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.381+0000 7ff719c25640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ff7140b3930 con 0x7ff714162590 2024-09-21T03:42:58.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.381+0000 7ff719c25640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ff7140b37b0 con 0x7ff7140b2e60 2024-09-21T03:42:58.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.381+0000 7ff719c25640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7ff7140b3ab0 con 0x7ff7140cfe00 2024-09-21T03:42:58.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.381+0000 7ff713fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff714162590 0x7ff714157520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:58.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.381+0000 7ff713fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff714162590 0x7ff714157520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43992/0 (socket says 172.21.15.38:43992) 2024-09-21T03:42:58.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.381+0000 7ff713fff640 1 -- 172.21.15.38:0/3278093648 learned_addr learned my addr 172.21.15.38:0/3278093648 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:42:58.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.381+0000 7ff712ffd640 1 --2- 172.21.15.38:0/3278093648 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff7140cfe00 0x7ff7141f8ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:58.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.381+0000 7ff7137fe640 1 --2- 172.21.15.38:0/3278093648 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff7140b2e60 0x7ff714155000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:58.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.381+0000 7ff713fff640 1 -- 172.21.15.38:0/3278093648 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff7140cfe00 msgr2=0x7ff7141f8ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:58.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.381+0000 7ff713fff640 1 --2- 172.21.15.38:0/3278093648 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff7140cfe00 0x7ff7141f8ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:58.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.381+0000 7ff713fff640 1 -- 172.21.15.38:0/3278093648 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff7140b2e60 msgr2=0x7ff714155000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:58.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.381+0000 7ff713fff640 1 --2- 172.21.15.38:0/3278093648 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff7140b2e60 0x7ff714155000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:58.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.381+0000 7ff713fff640 1 -- 172.21.15.38:0/3278093648 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7141f9400 con 0x7ff714162590 2024-09-21T03:42:58.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.382+0000 7ff713fff640 1 --2- 172.21.15.38:0/3278093648 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff714162590 0x7ff714157520 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 crypto rx=0x7ff70c009d90 tx=0x7ff70c003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:58.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.382+0000 7ff6fbfff640 1 -- 172.21.15.38:0/3278093648 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff70c044070 con 0x7ff714162590 2024-09-21T03:42:58.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.382+0000 7ff719c25640 1 -- 172.21.15.38:0/3278093648 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7141f9910 con 0x7ff714162590 2024-09-21T03:42:58.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.382+0000 7ff6fbfff640 1 -- 172.21.15.38:0/3278093648 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff70c004040 con 0x7ff714162590 2024-09-21T03:42:58.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.382+0000 7ff6fbfff640 1 -- 172.21.15.38:0/3278093648 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff70c004430 con 0x7ff714162590 2024-09-21T03:42:58.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.382+0000 7ff719c25640 1 -- 172.21.15.38:0/3278093648 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7141f6c50 con 0x7ff714162590 2024-09-21T03:42:58.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.383+0000 7ff719c25640 1 -- 172.21.15.38:0/3278093648 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7141f9680 con 0x7ff714162590 2024-09-21T03:42:58.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.384+0000 7ff6fbfff640 1 -- 172.21.15.38:0/3278093648 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ff70c047070 con 0x7ff714162590 2024-09-21T03:42:58.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.384+0000 7ff6fbfff640 1 --2- 172.21.15.38:0/3278093648 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff700078bc0 0x7ff70007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:58.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.384+0000 7ff6fbfff640 1 -- 172.21.15.38:0/3278093648 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 8540+0+0 (secure 0 0 0) 0x7ff70c0bc6a0 con 0x7ff714162590 2024-09-21T03:42:58.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.384+0000 7ff7137fe640 1 --2- 172.21.15.38:0/3278093648 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff700078bc0 0x7ff70007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:58.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.385+0000 7ff7137fe640 1 --2- 172.21.15.38:0/3278093648 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff700078bc0 0x7ff70007b0d0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7ff704009b30 tx=0x7ff704005f90 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:58.390 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.389+0000 7ff6fbfff640 1 -- 172.21.15.38:0/3278093648 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7ff70c032030 con 0x7ff714162590 2024-09-21T03:42:58.448 INFO:tasks.ceph.mds.c.smithi038.stderr:2024-09-21T03:42:58.446+0000 7fa1ddd4f640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T03:42:58.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.615+0000 7ff719c25640 1 -- 172.21.15.38:0/3278093648 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7ff7140b3270 con 0x7ff714162590 2024-09-21T03:42:58.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.615+0000 7ff6fbfff640 1 -- 172.21.15.38:0/3278093648 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v246) ==== 53+0+73 (secure 0 0 0) 0x7ff70c083940 con 0x7ff714162590 2024-09-21T03:42:58.617 INFO:teuthology.orchestra.run.smithi038.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T03:42:58.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.618+0000 7ff719c25640 1 -- 172.21.15.38:0/3278093648 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff700078bc0 msgr2=0x7ff70007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:58.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.618+0000 7ff719c25640 1 --2- 172.21.15.38:0/3278093648 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff700078bc0 0x7ff70007b0d0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7ff704009b30 tx=0x7ff704005f90 comp rx=0 tx=0).stop 2024-09-21T03:42:58.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.618+0000 7ff719c25640 1 -- 172.21.15.38:0/3278093648 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff714162590 msgr2=0x7ff714157520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:58.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.618+0000 7ff719c25640 1 --2- 172.21.15.38:0/3278093648 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff714162590 0x7ff714157520 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 crypto rx=0x7ff70c009d90 tx=0x7ff70c003ce0 comp rx=0 tx=0).stop 2024-09-21T03:42:58.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.618+0000 7ff719c25640 1 -- 172.21.15.38:0/3278093648 shutdown_connections 2024-09-21T03:42:58.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.618+0000 7ff719c25640 1 --2- 172.21.15.38:0/3278093648 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff714162590 0x7ff714157520 unknown :-1 s=CLOSED pgs=1889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:58.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.618+0000 7ff719c25640 1 --2- 172.21.15.38:0/3278093648 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff700078bc0 0x7ff70007b0d0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:58.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.618+0000 7ff719c25640 1 --2- 172.21.15.38:0/3278093648 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff7140cfe00 0x7ff7141f8ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:58.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.618+0000 7ff719c25640 1 --2- 172.21.15.38:0/3278093648 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff7140b2e60 0x7ff714155000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:58.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.618+0000 7ff719c25640 1 -- 172.21.15.38:0/3278093648 >> 172.21.15.38:0/3278093648 conn(0x7ff7140d4440 msgr2=0x7ff71406cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:58.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.618+0000 7ff719c25640 1 -- 172.21.15.38:0/3278093648 shutdown_connections 2024-09-21T03:42:58.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58.618+0000 7ff719c25640 1 -- 172.21.15.38:0/3278093648 wait complete. 2024-09-21T03:42:58.631 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-21T03:42:58.631 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T03:42:58.631 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi038.front.sepia.ceph.com 2024-09-21T03:42:58.631 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-21T03:42:58.632 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T03:42:58.632 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-21T03:42:58.632 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-21T03:42:58.632 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T03:42:58.632 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T03:42:58.632 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:42:58.632 DEBUG:teuthology.orchestra.run.smithi038:> ip addr 2024-09-21T03:42:58.648 INFO:teuthology.orchestra.run.smithi038.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T03:42:58.649 INFO:teuthology.orchestra.run.smithi038.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T03:42:58.649 INFO:teuthology.orchestra.run.smithi038.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T03:42:58.649 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:42:58.649 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 ::1/128 scope host 2024-09-21T03:42:58.649 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:42:58.649 INFO:teuthology.orchestra.run.smithi038.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T03:42:58.649 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:6b:fd:a2 brd ff:ff:ff:ff:ff:ff 2024-09-21T03:42:58.649 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp7s0f0 2024-09-21T03:42:58.649 INFO:teuthology.orchestra.run.smithi038.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T03:42:58.649 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:6b:fd:a3 brd ff:ff:ff:ff:ff:ff 2024-09-21T03:42:58.649 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp7s0f1 2024-09-21T03:42:58.649 INFO:teuthology.orchestra.run.smithi038.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T03:42:58.650 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:bd:15:fc brd ff:ff:ff:ff:ff:ff 2024-09-21T03:42:58.650 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp1s0f0 2024-09-21T03:42:58.650 INFO:teuthology.orchestra.run.smithi038.stdout: inet 172.21.15.38/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-21T03:42:58.650 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft 42230sec preferred_lft 42230sec 2024-09-21T03:42:58.650 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 fe80::ec4:7aff:febd:15fc/64 scope link 2024-09-21T03:42:58.650 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:42:58.650 INFO:teuthology.orchestra.run.smithi038.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T03:42:58.650 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:bd:15:fd brd ff:ff:ff:ff:ff:ff 2024-09-21T03:42:58.650 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp1s0f1 2024-09-21T03:42:58.650 INFO:teuthology.orchestra.run.smithi038.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T03:42:58.650 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 92:55:c7:97:6f:b5 brd ff:ff:ff:ff:ff:ff 2024-09-21T03:42:58.650 INFO:teuthology.orchestra.run.smithi038.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-21T03:42:58.650 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:42:58.651 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 fe80::c49f:c1ff:fe21:b12a/64 scope link 2024-09-21T03:42:58.651 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:42:58.651 INFO:teuthology.orchestra.run.smithi038.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T03:42:58.651 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether a6:3a:83:ad:db:d5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T03:42:58.651 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 fe80::a43a:83ff:fead:dbd5/64 scope link 2024-09-21T03:42:58.651 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:42:58.651 INFO:teuthology.orchestra.run.smithi038.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T03:42:58.651 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 92:55:c7:97:6f:b5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T03:42:58.651 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 fe80::9055:c7ff:fe97:6fb5/64 scope link 2024-09-21T03:42:58.651 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:42:58.651 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:42:58.652 DEBUG:teuthology.orchestra.run.smithi038:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T03:42:58.713 INFO:teuthology.orchestra.run.smithi038.stdout:1 2024-09-21T03:42:58.714 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:42:58.714 DEBUG:teuthology.orchestra.run.smithi038:> route 2024-09-21T03:42:58.771 INFO:teuthology.orchestra.run.smithi038.stdout:Kernel IP routing table 2024-09-21T03:42:58.771 INFO:teuthology.orchestra.run.smithi038.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T03:42:58.771 INFO:teuthology.orchestra.run.smithi038.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-21T03:42:58.771 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-21T03:42:58.771 INFO:teuthology.orchestra.run.smithi038.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T03:42:58.772 DEBUG:teuthology.orchestra.run.smithi038:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T03:42:58.772 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-09-21T03:42:58.772 DEBUG:teuthology.orchestra.run.smithi038:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-21T03:42:58.772 DEBUG:teuthology.orchestra.run.smithi038:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-21T03:42:58.772 DEBUG:teuthology.orchestra.run.smithi038:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-21T03:42:58.772 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-09-21T03:42:58.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T03:42:58.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:42:58.900 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:42:58.901 DEBUG:teuthology.orchestra.run.smithi038:> ip netns list 2024-09-21T03:42:58.917 INFO:teuthology.orchestra.run.smithi038.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-21T03:42:58.917 INFO:teuthology.orchestra.run.smithi038.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-21T03:42:58.917 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T03:42:58.918 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:42:58.918 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip link set brx.0 up 2024-09-21T03:42:58.983 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T03:42:58.983 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-21T03:42:59.040 INFO:teuthology.orchestra.run.smithi038.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-21T03:42:59.040 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T03:42:59.040 DEBUG:teuthology.orchestra.run.smithi038:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-21T03:42:59.095 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-21T03:42:59.095 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:42:59.095 DEBUG:teuthology.orchestra.run.smithi038:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T03:42:59.191 INFO:teuthology.orchestra.run.smithi038.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T03:42:59.191 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T03:42:59.191 INFO:teuthology.orchestra.run.smithi038.stdout:invalid new device string format 2024-09-21T03:42:59.202 INFO:teuthology.orchestra.run.smithi038.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T03:42:59.202 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T03:42:59.202 INFO:teuthology.orchestra.run.smithi038.stdout:invalid new device string format 2024-09-21T03:42:59.202 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: resolved to: "172.21.15.38:3300,172.21.15.195:3300,172.21.15.195:3301" 2024-09-21T03:42:59.202 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T03:42:59.218 INFO:teuthology.orchestra.run.smithi038.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T03:42:59.218 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T03:42:59.218 INFO:teuthology.orchestra.run.smithi038.stdout:invalid new device string format 2024-09-21T03:42:59.218 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: resolved to: "172.21.15.38:3300,172.21.15.195:3300,172.21.15.195:3301" 2024-09-21T03:42:59.218 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T03:42:59.218 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: resolved to: "172.21.15.38:6789,172.21.15.195:6789,172.21.15.195:6790" 2024-09-21T03:42:59.218 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.38:6789,172.21.15.195:6789,172.21.15.195:6790:/ 2024-09-21T03:42:59.218 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=9d50d66f-656b-4fe2-8e3b-fe53b79c4d43" will pass to kernel 2024-09-21T03:42:59.219 INFO:teuthology.orchestra.run.smithi038.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-21T03:42:59.219 INFO:teuthology.orchestra.run.smithi038.stdout: You just mounted a file system that supports labels which does not 2024-09-21T03:42:59.219 INFO:teuthology.orchestra.run.smithi038.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-21T03:42:59.219 INFO:teuthology.orchestra.run.smithi038.stdout: applications will generate AVC messages and not be allowed access to 2024-09-21T03:42:59.219 INFO:teuthology.orchestra.run.smithi038.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-21T03:42:59.220 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T03:42:59.220 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:42:59.221 DEBUG:teuthology.orchestra.run.smithi038:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-21T03:42:59.246 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T03:42:59.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:59 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T03:42:59.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:42:59.344 INFO:teuthology.orchestra.run.smithi038.stdout:client8641 2024-09-21T03:42:59.344 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-21T03:42:59.344 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:42:59.360 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:42:59.360 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls -R .) 2024-09-21T03:42:59.403 INFO:teuthology.orchestra.run:waiting for 20 2024-09-21T03:42:59.418 INFO:teuthology.orchestra.run.smithi038.stdout:.: 2024-09-21T03:42:59.418 INFO:teuthology.orchestra.run.smithi038.stdout:subdir 2024-09-21T03:42:59.418 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:42:59.418 INFO:teuthology.orchestra.run.smithi038.stdout:./subdir: 2024-09-21T03:42:59.418 INFO:teuthology.orchestra.run.smithi038.stdout:sixmegs 2024-09-21T03:42:59.419 INFO:tasks.cephfs.test_damage:Result: Mutation 'Truncate 500.00000000' did not caused DAMAGED state 2024-09-21T03:42:59.419 INFO:tasks.cephfs.test_damage:Applying mutation 'Truncate mds_snaptable' 2024-09-21T03:42:59.419 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-21T03:42:59.419 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:42:59.475 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-21T03:42:59.475 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:42:59.532 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T03:42:59.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:59.687+0000 7f79244de640 1 -- 172.21.15.38:0/1615773914 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f791c0ceb90 msgr2=0x7f791c0cefa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:59.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:59.687+0000 7f79244de640 1 --2- 172.21.15.38:0/1615773914 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f791c0ceb90 0x7f791c0cefa0 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7f790c005fd0 tx=0x7f790c02a820 comp rx=0 tx=0).stop 2024-09-21T03:42:59.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:59.687+0000 7f79244de640 1 -- 172.21.15.38:0/1615773914 shutdown_connections 2024-09-21T03:42:59.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:59.687+0000 7f79244de640 1 --2- 172.21.15.38:0/1615773914 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f791c0ceb90 0x7f791c0cefa0 unknown :-1 s=CLOSED pgs=1891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:59.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:59.687+0000 7f79244de640 1 --2- 172.21.15.38:0/1615773914 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f791c15ee80 0x7f791c0d0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:59.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:59.687+0000 7f79244de640 1 -- 172.21.15.38:0/1615773914 >> 172.21.15.38:0/1615773914 conn(0x7f791c09d830 msgr2=0x7f791c0b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:59.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:59.687+0000 7f79244de640 1 -- 172.21.15.38:0/1615773914 shutdown_connections 2024-09-21T03:42:59.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:59.687+0000 7f79244de640 1 -- 172.21.15.38:0/1615773914 wait complete. 2024-09-21T03:42:59.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:59.689+0000 7f79244de640 1 Processor -- start 2024-09-21T03:42:59.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:59.689+0000 7f79244de640 1 -- start start 2024-09-21T03:42:59.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:59.690+0000 7f79244de640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f791c0b9ad0 0x7f791c0cb610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:59.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:59.690+0000 7f79244de640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f791c0ceb90 0x7f791c06e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:59.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:59.690+0000 7f79244de640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f791c15ee80 0x7f791c06ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:59.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:59.690+0000 7f79244de640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f791c0b6700 con 0x7f791c0ceb90 2024-09-21T03:42:59.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:59.690+0000 7f79244de640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f791c0b6580 con 0x7f791c15ee80 2024-09-21T03:42:59.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:59.690+0000 7f79244de640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f791c0b6880 con 0x7f791c0b9ad0 2024-09-21T03:42:59.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:59.690+0000 7f7921a79640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f791c0ceb90 0x7f791c06e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:59.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:59.690+0000 7f7921a79640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f791c0ceb90 0x7f791c06e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44014/0 (socket says 172.21.15.38:44014) 2024-09-21T03:42:59.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:59.690+0000 7f7921a79640 1 -- 172.21.15.38:0/1225443004 learned_addr learned my addr 172.21.15.38:0/1225443004 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:42:59.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:59.690+0000 7f792227a640 1 --2- 172.21.15.38:0/1225443004 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f791c0b9ad0 0x7f791c0cb610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:59.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:59.690+0000 7f7922a7b640 1 --2- 172.21.15.38:0/1225443004 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f791c15ee80 0x7f791c06ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:59.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:59.690+0000 7f7921a79640 1 -- 172.21.15.38:0/1225443004 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f791c0b9ad0 msgr2=0x7f791c0cb610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:59.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:59.690+0000 7f7921a79640 1 --2- 172.21.15.38:0/1225443004 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f791c0b9ad0 0x7f791c0cb610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:59.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:59.690+0000 7f7921a79640 1 -- 172.21.15.38:0/1225443004 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f791c15ee80 msgr2=0x7f791c06ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:59.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:59.690+0000 7f7921a79640 1 --2- 172.21.15.38:0/1225443004 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f791c15ee80 0x7f791c06ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:59.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:59.690+0000 7f7921a79640 1 -- 172.21.15.38:0/1225443004 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f791c0cc020 con 0x7f791c0ceb90 2024-09-21T03:42:59.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:59.690+0000 7f792227a640 1 --2- 172.21.15.38:0/1225443004 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f791c0b9ad0 0x7f791c0cb610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:42:59.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:59.690+0000 7f7922a7b640 1 --2- 172.21.15.38:0/1225443004 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f791c15ee80 0x7f791c06ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:42:59.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:59.691+0000 7f7921a79640 1 --2- 172.21.15.38:0/1225443004 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f791c0ceb90 0x7f791c06e6a0 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7f791000c3c0 tx=0x7f791000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:59.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:59.691+0000 7f791affd640 1 -- 172.21.15.38:0/1225443004 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7910017070 con 0x7f791c0ceb90 2024-09-21T03:42:59.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:59.691+0000 7f791affd640 1 -- 172.21.15.38:0/1225443004 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f79100035f0 con 0x7f791c0ceb90 2024-09-21T03:42:59.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:59.691+0000 7f79244de640 1 -- 172.21.15.38:0/1225443004 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f791c0cc300 con 0x7f791c0ceb90 2024-09-21T03:42:59.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:59.691+0000 7f791affd640 1 -- 172.21.15.38:0/1225443004 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7910003a80 con 0x7f791c0ceb90 2024-09-21T03:42:59.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:59.691+0000 7f79244de640 1 -- 172.21.15.38:0/1225443004 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f791c0ccec0 con 0x7f791c0ceb90 2024-09-21T03:42:59.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:59.692+0000 7f79244de640 1 -- 172.21.15.38:0/1225443004 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f791c0cc520 con 0x7f791c0ceb90 2024-09-21T03:42:59.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:59.693+0000 7f791affd640 1 -- 172.21.15.38:0/1225443004 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f791001a070 con 0x7f791c0ceb90 2024-09-21T03:42:59.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:59.693+0000 7f791affd640 1 --2- 172.21.15.38:0/1225443004 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7908078bc0 0x7f790807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:59.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:59.694+0000 7f791affd640 1 -- 172.21.15.38:0/1225443004 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 8540+0+0 (secure 0 0 0) 0x7f791009ab50 con 0x7f791c0ceb90 2024-09-21T03:42:59.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:59.694+0000 7f792227a640 1 --2- 172.21.15.38:0/1225443004 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7908078bc0 0x7f790807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:59.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:59.694+0000 7f792227a640 1 --2- 172.21.15.38:0/1225443004 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7908078bc0 0x7f790807b0d0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f7914009aa0 tx=0x7f7914005db0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:59.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:59.698+0000 7f791affd640 1 -- 172.21.15.38:0/1225443004 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f791c1592d0 con 0x7f791c0ceb90 2024-09-21T03:42:59.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:59.896+0000 7f79244de640 1 -- 172.21.15.38:0/1225443004 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f791c0b6d40 con 0x7f791c0ceb90 2024-09-21T03:42:59.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:59.897+0000 7f791affd640 1 -- 172.21.15.38:0/1225443004 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 52 entries v122) ==== 81+0+3164 (secure 0 0 0) 0x7f7910062430 con 0x7f791c0ceb90 2024-09-21T03:42:59.899 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6833/2459386631 2024-09-22T03:42:43.395609+0000 2024-09-21T03:42:59.899 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6833/245538755 2024-09-22T03:42:35.318756+0000 2024-09-21T03:42:59.899 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6832/245538755 2024-09-22T03:42:35.318756+0000 2024-09-21T03:42:59.899 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6832/4221150556 2024-09-22T03:42:27.240627+0000 2024-09-21T03:42:59.899 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6835/2930805359 2024-09-22T03:42:23.215190+0000 2024-09-21T03:42:59.899 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6834/2712360296 2024-09-22T03:42:20.931115+0000 2024-09-21T03:42:59.899 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6832/3537258034 2024-09-22T03:42:06.901189+0000 2024-09-21T03:42:59.899 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6833/3537258034 2024-09-22T03:42:06.901189+0000 2024-09-21T03:42:59.899 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6833/1272005650 2024-09-22T03:41:58.836862+0000 2024-09-21T03:42:59.900 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6835/2712360296 2024-09-22T03:42:20.931115+0000 2024-09-21T03:42:59.900 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6832/1272005650 2024-09-22T03:41:58.836862+0000 2024-09-21T03:42:59.900 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6835/3055025811 2024-09-22T03:42:39.361236+0000 2024-09-21T03:42:59.900 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6833/1717955365 2024-09-22T03:41:51.787804+0000 2024-09-21T03:42:59.900 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6832/3710736169 2024-09-22T03:40:59.454480+0000 2024-09-21T03:42:59.900 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6835/2751962827 2024-09-22T03:40:38.029580+0000 2024-09-21T03:42:59.900 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6835/425672974 2024-09-22T03:40:46.105521+0000 2024-09-21T03:42:59.900 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6834/3951934534 2024-09-22T03:42:31.276099+0000 2024-09-21T03:42:59.900 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6834/425672974 2024-09-22T03:40:46.105521+0000 2024-09-21T03:42:59.900 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6833/3402963947 2024-09-22T03:41:23.555005+0000 2024-09-21T03:42:59.900 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6835/2916238332 2024-09-22T03:41:42.553770+0000 2024-09-21T03:42:59.900 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6834/3572946698 2024-09-22T03:41:44.737273+0000 2024-09-21T03:42:59.900 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6834/2798021708 2024-09-22T03:42:47.436632+0000 2024-09-21T03:42:59.900 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6834/3864487171 2024-09-22T03:40:31.989837+0000 2024-09-21T03:42:59.901 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.195:6835/712370966 2024-09-22T03:40:11.633787+0000 2024-09-21T03:42:59.901 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6834/3055025811 2024-09-22T03:42:39.361236+0000 2024-09-21T03:42:59.901 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6833/3726830644 2024-09-22T03:41:12.532341+0000 2024-09-21T03:42:59.901 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6835/2798021708 2024-09-22T03:42:47.436632+0000 2024-09-21T03:42:59.901 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6832/3402963947 2024-09-22T03:41:23.555005+0000 2024-09-21T03:42:59.901 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6833/751618334 2024-09-22T03:40:42.053279+0000 2024-09-21T03:42:59.901 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6835/3951934534 2024-09-22T03:42:31.276099+0000 2024-09-21T03:42:59.901 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6832/751618334 2024-09-22T03:40:42.053279+0000 2024-09-21T03:42:59.901 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6835/2418225210 2024-09-22T03:42:02.869612+0000 2024-09-21T03:42:59.901 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6834/2418225210 2024-09-22T03:42:02.869612+0000 2024-09-21T03:42:59.901 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6832/1717955365 2024-09-22T03:41:51.787804+0000 2024-09-21T03:42:59.901 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6832/121492560 2024-09-22T03:41:10.371933+0000 2024-09-21T03:42:59.901 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6833/4221150556 2024-09-22T03:42:27.240627+0000 2024-09-21T03:42:59.901 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6835/3864487171 2024-09-22T03:40:31.989837+0000 2024-09-21T03:42:59.902 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6835/2934569248 2024-09-22T03:40:26.934591+0000 2024-09-21T03:42:59.902 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6833/121492560 2024-09-22T03:41:10.371933+0000 2024-09-21T03:42:59.902 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6832/3726830644 2024-09-22T03:41:12.532341+0000 2024-09-21T03:42:59.902 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6834/2934569248 2024-09-22T03:40:26.934591+0000 2024-09-21T03:42:59.902 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6834/2748959970 2024-09-22T03:41:16.565736+0000 2024-09-21T03:42:59.902 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.195:6837/712370966 2024-09-22T03:40:11.633787+0000 2024-09-21T03:42:59.902 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6833/3710736169 2024-09-22T03:40:59.454480+0000 2024-09-21T03:42:59.902 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6835/2748959970 2024-09-22T03:41:16.565736+0000 2024-09-21T03:42:59.902 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6834/2930805359 2024-09-22T03:42:23.215190+0000 2024-09-21T03:42:59.902 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6832/3553953266 2024-09-22T03:41:29.639383+0000 2024-09-21T03:42:59.902 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6832/2459386631 2024-09-22T03:42:43.395609+0000 2024-09-21T03:42:59.902 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6834/2751962827 2024-09-22T03:40:38.029580+0000 2024-09-21T03:42:59.902 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6833/3553953266 2024-09-22T03:41:29.639383+0000 2024-09-21T03:42:59.903 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6834/2916238332 2024-09-22T03:41:42.553770+0000 2024-09-21T03:42:59.903 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6835/3572946698 2024-09-22T03:41:44.737273+0000 2024-09-21T03:42:59.903 INFO:teuthology.orchestra.run.smithi038.stderr:listed 52 entries 2024-09-21T03:42:59.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:59.899+0000 7f79244de640 1 -- 172.21.15.38:0/1225443004 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7908078bc0 msgr2=0x7f790807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:59.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:59.899+0000 7f79244de640 1 --2- 172.21.15.38:0/1225443004 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7908078bc0 0x7f790807b0d0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f7914009aa0 tx=0x7f7914005db0 comp rx=0 tx=0).stop 2024-09-21T03:42:59.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:59.899+0000 7f79244de640 1 -- 172.21.15.38:0/1225443004 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f791c0ceb90 msgr2=0x7f791c06e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:59.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:59.899+0000 7f79244de640 1 --2- 172.21.15.38:0/1225443004 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f791c0ceb90 0x7f791c06e6a0 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7f791000c3c0 tx=0x7f791000c880 comp rx=0 tx=0).stop 2024-09-21T03:42:59.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:59.899+0000 7f79244de640 1 -- 172.21.15.38:0/1225443004 shutdown_connections 2024-09-21T03:42:59.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:59.899+0000 7f79244de640 1 --2- 172.21.15.38:0/1225443004 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f791c15ee80 0x7f791c06ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:59.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:59.899+0000 7f79244de640 1 --2- 172.21.15.38:0/1225443004 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f791c0ceb90 0x7f791c06e6a0 unknown :-1 s=CLOSED pgs=1892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:59.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:59.899+0000 7f79244de640 1 --2- 172.21.15.38:0/1225443004 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7908078bc0 0x7f790807b0d0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:59.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:59.899+0000 7f79244de640 1 --2- 172.21.15.38:0/1225443004 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f791c0b9ad0 0x7f791c0cb610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:59.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:59.899+0000 7f79244de640 1 -- 172.21.15.38:0/1225443004 >> 172.21.15.38:0/1225443004 conn(0x7f791c09d830 msgr2=0x7f791c0b31a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:59.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:59.899+0000 7f79244de640 1 -- 172.21.15.38:0/1225443004 shutdown_connections 2024-09-21T03:42:59.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:42:59.899+0000 7f79244de640 1 -- 172.21.15.38:0/1225443004 wait complete. 2024-09-21T03:42:59.914 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-21T03:42:59.914 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:42:59.914 DEBUG:teuthology.orchestra.run.smithi038:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-09-21T03:42:59.956 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi038.front.sepia.ceph.com 2024-09-21T03:42:59.956 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:42:59.956 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T03:43:00.012 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-21T03:43:00.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:00.167+0000 7f25d5aa9640 1 -- 172.21.15.38:0/4188513235 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f25d015ee80 msgr2=0x7f25d00d0170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:00.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:00.167+0000 7f25d5aa9640 1 --2- 172.21.15.38:0/4188513235 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f25d015ee80 0x7f25d00d0170 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f25c0005fd0 tx=0x7f25c002a540 comp rx=0 tx=0).stop 2024-09-21T03:43:00.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:00.167+0000 7f25d5aa9640 1 -- 172.21.15.38:0/4188513235 shutdown_connections 2024-09-21T03:43:00.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:00.167+0000 7f25d5aa9640 1 --2- 172.21.15.38:0/4188513235 >> v2:172.21.15.195:3300/0 conn(0x7f25d00b9ad0 0x7f25d00d0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:00.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:00.167+0000 7f25d5aa9640 1 --2- 172.21.15.38:0/4188513235 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f25d015ee80 0x7f25d00d0170 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:00.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:00.167+0000 7f25d5aa9640 1 -- 172.21.15.38:0/4188513235 >> 172.21.15.38:0/4188513235 conn(0x7f25d009d830 msgr2=0x7f25d00b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:00.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:00.167+0000 7f25d5aa9640 1 -- 172.21.15.38:0/4188513235 shutdown_connections 2024-09-21T03:43:00.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:00.167+0000 7f25d5aa9640 1 -- 172.21.15.38:0/4188513235 wait complete. 2024-09-21T03:43:00.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:00.169+0000 7f25d5aa9640 1 Processor -- start 2024-09-21T03:43:00.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:00.169+0000 7f25d5aa9640 1 -- start start 2024-09-21T03:43:00.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:00.171+0000 7f25d5aa9640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f25d00b9ad0 0x7f25d01f5e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:00.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:00.171+0000 7f25d5aa9640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f25d00d1120 0x7f25d01f28d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:00.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:00.171+0000 7f25d5aa9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f25d015ee80 0x7f25d006f240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:00.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:00.171+0000 7f25d5aa9640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f25d00b6700 con 0x7f25d015ee80 2024-09-21T03:43:00.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:00.171+0000 7f25d5aa9640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f25d00b6580 con 0x7f25d00d1120 2024-09-21T03:43:00.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:00.171+0000 7f25d5aa9640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f25d00b6880 con 0x7f25d00b9ad0 2024-09-21T03:43:00.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:00.171+0000 7f25cffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f25d015ee80 0x7f25d006f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:00.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:00.171+0000 7f25cffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f25d015ee80 0x7f25d006f240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44020/0 (socket says 172.21.15.38:44020) 2024-09-21T03:43:00.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:00.171+0000 7f25cffff640 1 -- 172.21.15.38:0/870790334 learned_addr learned my addr 172.21.15.38:0/870790334 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:43:00.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:00.171+0000 7f25cf7fe640 1 --2- 172.21.15.38:0/870790334 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f25d00b9ad0 0x7f25d01f5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:00.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:00.171+0000 7f25ceffd640 1 --2- 172.21.15.38:0/870790334 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f25d00d1120 0x7f25d01f28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:00.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:00.171+0000 7f25cf7fe640 1 -- 172.21.15.38:0/870790334 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f25d00d1120 msgr2=0x7f25d01f28d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:00.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:00.171+0000 7f25cf7fe640 1 --2- 172.21.15.38:0/870790334 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f25d00d1120 0x7f25d01f28d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:00.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:00.171+0000 7f25cf7fe640 1 -- 172.21.15.38:0/870790334 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f25d015ee80 msgr2=0x7f25d006f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:00.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:00.171+0000 7f25cf7fe640 1 --2- 172.21.15.38:0/870790334 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f25d015ee80 0x7f25d006f240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:00.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:00.171+0000 7f25cf7fe640 1 -- 172.21.15.38:0/870790334 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25d01f63d0 con 0x7f25d00b9ad0 2024-09-21T03:43:00.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:00.171+0000 7f25cffff640 1 --2- 172.21.15.38:0/870790334 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f25d015ee80 0x7f25d006f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:43:00.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:00.171+0000 7f25cf7fe640 1 --2- 172.21.15.38:0/870790334 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f25d00b9ad0 0x7f25d01f5e90 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f25c0031000 tx=0x7f25c00038a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:00.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:00.172+0000 7f25b7fff640 1 -- 172.21.15.38:0/870790334 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25c003e030 con 0x7f25d00b9ad0 2024-09-21T03:43:00.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:00.172+0000 7f25b7fff640 1 -- 172.21.15.38:0/870790334 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f25c0003ca0 con 0x7f25d00b9ad0 2024-09-21T03:43:00.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:00.172+0000 7f25d5aa9640 1 -- 172.21.15.38:0/870790334 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25d01f2e10 con 0x7f25d00b9ad0 2024-09-21T03:43:00.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:00.172+0000 7f25b7fff640 1 -- 172.21.15.38:0/870790334 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25c0035040 con 0x7f25d00b9ad0 2024-09-21T03:43:00.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:00.172+0000 7f25d5aa9640 1 -- 172.21.15.38:0/870790334 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f25d0159ac0 con 0x7f25d00b9ad0 2024-09-21T03:43:00.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:00.173+0000 7f25d5aa9640 1 -- 172.21.15.38:0/870790334 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25d00b6d30 con 0x7f25d00b9ad0 2024-09-21T03:43:00.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:00.174+0000 7f25b7fff640 1 -- 172.21.15.38:0/870790334 <== mon.2 v2:172.21.15.195:3301/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f25c0041030 con 0x7f25d00b9ad0 2024-09-21T03:43:00.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:00.174+0000 7f25b7fff640 1 --2- 172.21.15.38:0/870790334 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f25bc078bc0 0x7f25bc07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:00.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:00.174+0000 7f25b7fff640 1 -- 172.21.15.38:0/870790334 <== mon.2 v2:172.21.15.195:3301/0 5 ==== osd_map(122..122 src has 1..122) ==== 8540+0+0 (secure 0 0 0) 0x7f25c00bc1c0 con 0x7f25d00b9ad0 2024-09-21T03:43:00.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:00.175+0000 7f25ceffd640 1 --2- 172.21.15.38:0/870790334 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f25bc078bc0 0x7f25bc07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:00.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:00.175+0000 7f25ceffd640 1 --2- 172.21.15.38:0/870790334 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f25bc078bc0 0x7f25bc07b0d0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f25d00b9990 tx=0x7f25c4005f30 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:00.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:00.179+0000 7f25b7fff640 1 -- 172.21.15.38:0/870790334 <== mon.2 v2:172.21.15.195:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f25d01f63d0 con 0x7f25d00b9ad0 2024-09-21T03:43:00.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:00.404+0000 7f25d5aa9640 1 -- 172.21.15.38:0/870790334 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f25d01615d0 con 0x7f25d00b9ad0 2024-09-21T03:43:00.407 INFO:tasks.ceph.mon.a.smithi038.stderr:2024-09-21T03:43:00.405+0000 7f3f45713640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T03:43:00.412 INFO:tasks.ceph.mds.c.smithi038.stderr: -545> 2024-09-21T03:42:58.446+0000 7fa1ddd4f640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T03:43:00.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:00.411+0000 7f25b7fff640 1 -- 172.21.15.38:0/870790334 <== mon.2 v2:172.21.15.195:3301/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v247) ==== 161+0+0 (secure 0 0 0) 0x7f25c00834d0 con 0x7f25d00b9ad0 2024-09-21T03:43:00.414 INFO:teuthology.orchestra.run.smithi038.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-21T03:43:00.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:00.415+0000 7f25d5aa9640 1 -- 172.21.15.38:0/870790334 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f25bc078bc0 msgr2=0x7f25bc07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:00.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:00.415+0000 7f25d5aa9640 1 --2- 172.21.15.38:0/870790334 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f25bc078bc0 0x7f25bc07b0d0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f25d00b9990 tx=0x7f25c4005f30 comp rx=0 tx=0).stop 2024-09-21T03:43:00.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:00.415+0000 7f25d5aa9640 1 -- 172.21.15.38:0/870790334 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f25d00b9ad0 msgr2=0x7f25d01f5e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:00.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:00.415+0000 7f25d5aa9640 1 --2- 172.21.15.38:0/870790334 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f25d00b9ad0 0x7f25d01f5e90 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f25c0031000 tx=0x7f25c00038a0 comp rx=0 tx=0).stop 2024-09-21T03:43:00.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:00.415+0000 7f25d5aa9640 1 -- 172.21.15.38:0/870790334 shutdown_connections 2024-09-21T03:43:00.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:00.415+0000 7f25d5aa9640 1 --2- 172.21.15.38:0/870790334 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f25d015ee80 0x7f25d006f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:00.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:00.415+0000 7f25d5aa9640 1 --2- 172.21.15.38:0/870790334 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f25d00d1120 0x7f25d01f28d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:00.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:00.415+0000 7f25d5aa9640 1 --2- 172.21.15.38:0/870790334 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f25bc078bc0 0x7f25bc07b0d0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:00.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:00.415+0000 7f25d5aa9640 1 --2- 172.21.15.38:0/870790334 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f25d00b9ad0 0x7f25d01f5e90 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:00.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:00.415+0000 7f25d5aa9640 1 -- 172.21.15.38:0/870790334 >> 172.21.15.38:0/870790334 conn(0x7f25d009d830 msgr2=0x7f25d01618e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:00.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:00.415+0000 7f25d5aa9640 1 -- 172.21.15.38:0/870790334 shutdown_connections 2024-09-21T03:43:00.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:00.415+0000 7f25d5aa9640 1 -- 172.21.15.38:0/870790334 wait complete. 2024-09-21T03:43:00.429 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-09-21T03:43:00.470 INFO:tasks.ceph.mds.c.smithi038.stdout:starting mds.c at 2024-09-21T03:43:00.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:00.546+0000 7fe4ce1df640 1 -- 172.21.15.38:0/2422943655 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe4c80ceb90 msgr2=0x7fe4c80cefa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:00.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:00.546+0000 7fe4ce1df640 1 --2- 172.21.15.38:0/2422943655 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe4c80ceb90 0x7fe4c80cefa0 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7fe4c0005fd0 tx=0x7fe4c002aa10 comp rx=0 tx=0).stop 2024-09-21T03:43:00.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:00.546+0000 7fe4ce1df640 1 -- 172.21.15.38:0/2422943655 shutdown_connections 2024-09-21T03:43:00.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:00.546+0000 7fe4ce1df640 1 --2- 172.21.15.38:0/2422943655 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe4c80ceb90 0x7fe4c80cefa0 unknown :-1 s=CLOSED pgs=1897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:00.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:00.546+0000 7fe4ce1df640 1 -- 172.21.15.38:0/2422943655 >> 172.21.15.38:0/2422943655 conn(0x7fe4c809d830 msgr2=0x7fe4c80b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:00.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:00.546+0000 7fe4ce1df640 1 -- 172.21.15.38:0/2422943655 shutdown_connections 2024-09-21T03:43:00.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:00.546+0000 7fe4ce1df640 1 -- 172.21.15.38:0/2422943655 wait complete. 2024-09-21T03:43:00.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:00.548+0000 7fe4ce1df640 1 Processor -- start 2024-09-21T03:43:00.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:00.548+0000 7fe4ce1df640 1 -- start start 2024-09-21T03:43:00.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:00.550+0000 7fe4ce1df640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe4c80ceb90 0x7fe4c80cb610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:00.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:00.550+0000 7fe4ce1df640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe4c80d0170 0x7fe4c806f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:00.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:00.550+0000 7fe4ce1df640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe4c815ee80 0x7fe4c814ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:00.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:00.550+0000 7fe4ce1df640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fe4c80b6910 con 0x7fe4c80ceb90 2024-09-21T03:43:00.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:00.550+0000 7fe4ce1df640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fe4c80b6790 con 0x7fe4c80d0170 2024-09-21T03:43:00.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:00.550+0000 7fe4ce1df640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fe4c80b6a90 con 0x7fe4c815ee80 2024-09-21T03:43:00.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:00.550+0000 7fe4c77fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe4c80ceb90 0x7fe4c80cb610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:00.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:00.550+0000 7fe4c77fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe4c80ceb90 0x7fe4c80cb610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44062/0 (socket says 172.21.15.38:44062) 2024-09-21T03:43:00.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:00.550+0000 7fe4c77fe640 1 -- 172.21.15.38:0/1782133264 learned_addr learned my addr 172.21.15.38:0/1782133264 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:43:00.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:00.550+0000 7fe4c6ffd640 1 --2- 172.21.15.38:0/1782133264 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe4c80d0170 0x7fe4c806f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:00.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:00.550+0000 7fe4c7fff640 1 --2- 172.21.15.38:0/1782133264 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe4c815ee80 0x7fe4c814ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:00.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:00.550+0000 7fe4c77fe640 1 -- 172.21.15.38:0/1782133264 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe4c815ee80 msgr2=0x7fe4c814ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:00.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:00.550+0000 7fe4c77fe640 1 --2- 172.21.15.38:0/1782133264 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe4c815ee80 0x7fe4c814ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:00.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:00.550+0000 7fe4c77fe640 1 -- 172.21.15.38:0/1782133264 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe4c80d0170 msgr2=0x7fe4c806f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:00.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:00.550+0000 7fe4c77fe640 1 --2- 172.21.15.38:0/1782133264 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe4c80d0170 0x7fe4c806f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:00.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:00.550+0000 7fe4c77fe640 1 -- 172.21.15.38:0/1782133264 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4c80cbb50 con 0x7fe4c80ceb90 2024-09-21T03:43:00.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:00.550+0000 7fe4c77fe640 1 --2- 172.21.15.38:0/1782133264 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe4c80ceb90 0x7fe4c80cb610 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7fe4b800c3d0 tx=0x7fe4b800c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:00.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:00.551+0000 7fe4affff640 1 -- 172.21.15.38:0/1782133264 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4b8017070 con 0x7fe4c80ceb90 2024-09-21T03:43:00.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:00.551+0000 7fe4affff640 1 -- 172.21.15.38:0/1782133264 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe4b80053c0 con 0x7fe4c80ceb90 2024-09-21T03:43:00.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:00.551+0000 7fe4ce1df640 1 -- 172.21.15.38:0/1782133264 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4c80cde70 con 0x7fe4c80ceb90 2024-09-21T03:43:00.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:00.551+0000 7fe4affff640 1 -- 172.21.15.38:0/1782133264 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4b8003850 con 0x7fe4c80ceb90 2024-09-21T03:43:00.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:00.551+0000 7fe4ce1df640 1 -- 172.21.15.38:0/1782133264 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4c80ce430 con 0x7fe4c80ceb90 2024-09-21T03:43:00.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:00.552+0000 7fe4ce1df640 1 -- 172.21.15.38:0/1782133264 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4c80cefa0 con 0x7fe4c80ceb90 2024-09-21T03:43:00.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:00.553+0000 7fe4affff640 1 -- 172.21.15.38:0/1782133264 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fe4b801a070 con 0x7fe4c80ceb90 2024-09-21T03:43:00.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:00.553+0000 7fe4affff640 1 --2- 172.21.15.38:0/1782133264 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fe4b4078b90 0x7fe4b407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:00.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:00.553+0000 7fe4affff640 1 -- 172.21.15.38:0/1782133264 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 8626+0+0 (secure 0 0 0) 0x7fe4b809ac00 con 0x7fe4c80ceb90 2024-09-21T03:43:00.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:00.553+0000 7fe4c6ffd640 1 --2- 172.21.15.38:0/1782133264 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fe4b4078b90 0x7fe4b407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:00.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:00.554+0000 7fe4c6ffd640 1 --2- 172.21.15.38:0/1782133264 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fe4b4078b90 0x7fe4b407b0a0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fe4c806d0b0 tx=0x7fe4bc009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:00.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:00.558+0000 7fe4affff640 1 -- 172.21.15.38:0/1782133264 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fe4b80a1080 con 0x7fe4c80ceb90 2024-09-21T03:43:00.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:00.826+0000 7fe4ce1df640 1 -- 172.21.15.38:0/1782133264 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7fe4c8158440 con 0x7fe4c80ceb90 2024-09-21T03:43:01.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.413+0000 7fe4affff640 1 -- 172.21.15.38:0/1782133264 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 nothing to do: rank is not damaged v248) ==== 107+0+0 (secure 0 0 0) 0x7fe4c80cf4e0 con 0x7fe4c80ceb90 2024-09-21T03:43:01.415 INFO:teuthology.orchestra.run.smithi038.stderr:nothing to do: rank is not damaged 2024-09-21T03:43:01.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.416+0000 7fe4ce1df640 1 -- 172.21.15.38:0/1782133264 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fe4b4078b90 msgr2=0x7fe4b407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:01.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.416+0000 7fe4ce1df640 1 --2- 172.21.15.38:0/1782133264 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fe4b4078b90 0x7fe4b407b0a0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fe4c806d0b0 tx=0x7fe4bc009040 comp rx=0 tx=0).stop 2024-09-21T03:43:01.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.416+0000 7fe4ce1df640 1 -- 172.21.15.38:0/1782133264 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe4c80ceb90 msgr2=0x7fe4c80cb610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:01.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.416+0000 7fe4ce1df640 1 --2- 172.21.15.38:0/1782133264 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe4c80ceb90 0x7fe4c80cb610 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7fe4b800c3d0 tx=0x7fe4b800c890 comp rx=0 tx=0).stop 2024-09-21T03:43:01.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.416+0000 7fe4ce1df640 1 -- 172.21.15.38:0/1782133264 shutdown_connections 2024-09-21T03:43:01.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.416+0000 7fe4ce1df640 1 --2- 172.21.15.38:0/1782133264 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fe4b4078b90 0x7fe4b407b0a0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:01.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.416+0000 7fe4ce1df640 1 --2- 172.21.15.38:0/1782133264 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe4c815ee80 0x7fe4c814ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:01.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.416+0000 7fe4ce1df640 1 --2- 172.21.15.38:0/1782133264 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe4c80d0170 0x7fe4c806f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:01.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.416+0000 7fe4ce1df640 1 --2- 172.21.15.38:0/1782133264 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe4c80ceb90 0x7fe4c80cb610 unknown :-1 s=CLOSED pgs=1898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:01.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.416+0000 7fe4ce1df640 1 -- 172.21.15.38:0/1782133264 >> 172.21.15.38:0/1782133264 conn(0x7fe4c809d830 msgr2=0x7fe4c81618b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:01.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.416+0000 7fe4ce1df640 1 -- 172.21.15.38:0/1782133264 shutdown_connections 2024-09-21T03:43:01.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.416+0000 7fe4ce1df640 1 -- 172.21.15.38:0/1782133264 wait complete. 2024-09-21T03:43:01.433 DEBUG:teuthology.orchestra.run.smithi038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-09-21T03:43:01.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.535+0000 7efc2042df40 1 Processor -- start 2024-09-21T03:43:01.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.535+0000 7efc2042df40 1 -- start start 2024-09-21T03:43:01.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.537+0000 7efc2042df40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5644c3cf0d00 0x5644c3dc6540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:01.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.537+0000 7efc2042df40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5644c3d1ce30 0x5644c3d1d970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:01.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.537+0000 7efc2042df40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5644c3dab950 0x5644c3d1deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:01.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.537+0000 7efc2042df40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5644c3c9d430 con 0x5644c3dab950 2024-09-21T03:43:01.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.537+0000 7efc2042df40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5644c3caa1f0 con 0x5644c3d1ce30 2024-09-21T03:43:01.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.537+0000 7efc2042df40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x5644c3c3ed60 con 0x5644c3cf0d00 2024-09-21T03:43:01.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.537+0000 7efc1a6e8640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5644c3dab950 0x5644c3d1deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:01.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.537+0000 7efc1a6e8640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5644c3dab950 0x5644c3d1deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44074/0 (socket says 172.21.15.38:44074) 2024-09-21T03:43:01.541 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.537+0000 7efc1a6e8640 1 -- 172.21.15.38:0/86853541 learned_addr learned my addr 172.21.15.38:0/86853541 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:43:01.541 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.537+0000 7efc196e6640 1 --2- 172.21.15.38:0/86853541 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5644c3d1ce30 0x5644c3d1d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:01.541 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.537+0000 7efc19ee7640 1 --2- 172.21.15.38:0/86853541 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5644c3cf0d00 0x5644c3dc6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:01.541 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.537+0000 7efc1a6e8640 1 -- 172.21.15.38:0/86853541 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5644c3cf0d00 msgr2=0x5644c3dc6540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:01.541 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.537+0000 7efc1a6e8640 1 --2- 172.21.15.38:0/86853541 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5644c3cf0d00 0x5644c3dc6540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:01.541 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.537+0000 7efc1a6e8640 1 -- 172.21.15.38:0/86853541 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5644c3d1ce30 msgr2=0x5644c3d1d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:01.541 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.537+0000 7efc1a6e8640 1 --2- 172.21.15.38:0/86853541 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5644c3d1ce30 0x5644c3d1d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:01.541 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.537+0000 7efc1a6e8640 1 -- 172.21.15.38:0/86853541 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644c3d19e00 con 0x5644c3dab950 2024-09-21T03:43:01.541 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.538+0000 7efc1a6e8640 1 --2- 172.21.15.38:0/86853541 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5644c3dab950 0x5644c3d1deb0 secure :-1 s=READY pgs=1900 cs=0 l=1 rev1=1 crypto rx=0x7efc14001670 tx=0x7efc14001b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ae9b3539e53989b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:01.541 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.538+0000 7efc0bfff640 1 -- 172.21.15.38:0/86853541 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc14014070 con 0x5644c3dab950 2024-09-21T03:43:01.541 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.538+0000 7efc0bfff640 1 -- 172.21.15.38:0/86853541 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efc1400f9b0 con 0x5644c3dab950 2024-09-21T03:43:01.541 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.538+0000 7efc0bfff640 1 -- 172.21.15.38:0/86853541 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc1400fe00 con 0x5644c3dab950 2024-09-21T03:43:01.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.539+0000 7efc2042df40 1 -- 172.21.15.38:0/86853541 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5644c3dab950 msgr2=0x5644c3d1deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:01.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.539+0000 7efc2042df40 1 --2- 172.21.15.38:0/86853541 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5644c3dab950 0x5644c3d1deb0 secure :-1 s=READY pgs=1900 cs=0 l=1 rev1=1 crypto rx=0x7efc14001670 tx=0x7efc14001b30 comp rx=0 tx=0).stop 2024-09-21T03:43:01.543 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.539+0000 7efc2042df40 1 -- 172.21.15.38:0/86853541 shutdown_connections 2024-09-21T03:43:01.543 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.539+0000 7efc2042df40 1 --2- 172.21.15.38:0/86853541 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5644c3dab950 0x5644c3d1deb0 unknown :-1 s=CLOSED pgs=1900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:01.543 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.539+0000 7efc2042df40 1 --2- 172.21.15.38:0/86853541 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5644c3d1ce30 0x5644c3d1d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:01.543 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.539+0000 7efc2042df40 1 --2- 172.21.15.38:0/86853541 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5644c3cf0d00 0x5644c3dc6540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:01.543 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.539+0000 7efc2042df40 1 -- 172.21.15.38:0/86853541 >> 172.21.15.38:0/86853541 conn(0x5644c3c0fac0 msgr2=0x5644c3dc2770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:01.543 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.539+0000 7efc2042df40 1 -- 172.21.15.38:0/86853541 shutdown_connections 2024-09-21T03:43:01.543 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.539+0000 7efc2042df40 1 -- 172.21.15.38:0/86853541 wait complete. 2024-09-21T03:43:01.543 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.540+0000 7efc2042df40 1 Processor -- start 2024-09-21T03:43:01.543 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.540+0000 7efc2042df40 1 -- start start 2024-09-21T03:43:01.544 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.541+0000 7efc2042df40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5644c3cf0d00 0x5644c3e5ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:01.544 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.541+0000 7efc2042df40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5644c3d1ce30 0x5644c3dc17f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:01.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.541+0000 7efc2042df40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5644c3dab950 0x5644c3dc1d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:01.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.541+0000 7efc2042df40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5644c3d1d4e0 con 0x5644c3cf0d00 2024-09-21T03:43:01.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.541+0000 7efc2042df40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5644c3da7b30 con 0x5644c3d1ce30 2024-09-21T03:43:01.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.541+0000 7efc2042df40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x5644c3d1a0e0 con 0x5644c3dab950 2024-09-21T03:43:01.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.541+0000 7efc19ee7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5644c3cf0d00 0x5644c3e5ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:01.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.541+0000 7efc19ee7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5644c3cf0d00 0x5644c3e5ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44084/0 (socket says 172.21.15.38:44084) 2024-09-21T03:43:01.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.541+0000 7efc19ee7640 1 -- 172.21.15.38:0/1211757524 learned_addr learned my addr 172.21.15.38:0/1211757524 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:43:01.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.541+0000 7efc1a6e8640 1 --2- 172.21.15.38:0/1211757524 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5644c3dab950 0x5644c3dc1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:01.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.541+0000 7efc196e6640 1 --2- 172.21.15.38:0/1211757524 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5644c3d1ce30 0x5644c3dc17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:01.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.542+0000 7efc19ee7640 1 -- 172.21.15.38:0/1211757524 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5644c3dab950 msgr2=0x5644c3dc1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:01.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.542+0000 7efc19ee7640 1 --2- 172.21.15.38:0/1211757524 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5644c3dab950 0x5644c3dc1d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:01.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.542+0000 7efc19ee7640 1 -- 172.21.15.38:0/1211757524 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5644c3d1ce30 msgr2=0x5644c3dc17f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:01.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.542+0000 7efc19ee7640 1 --2- 172.21.15.38:0/1211757524 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5644c3d1ce30 0x5644c3dc17f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:01.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.542+0000 7efc19ee7640 1 -- 172.21.15.38:0/1211757524 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644c3dbf6f0 con 0x5644c3cf0d00 2024-09-21T03:43:01.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.542+0000 7efc19ee7640 1 --2- 172.21.15.38:0/1211757524 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5644c3cf0d00 0x5644c3e5ce70 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 crypto rx=0x7efc0c00ef60 tx=0x7efc0c00ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:01.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.543+0000 7efc0affd640 1 -- 172.21.15.38:0/1211757524 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc0c015020 con 0x5644c3cf0d00 2024-09-21T03:43:01.547 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.543+0000 7efc0affd640 1 -- 172.21.15.38:0/1211757524 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efc0c011450 con 0x5644c3cf0d00 2024-09-21T03:43:01.547 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.543+0000 7efc0affd640 1 -- 172.21.15.38:0/1211757524 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc0c011860 con 0x5644c3cf0d00 2024-09-21T03:43:01.547 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.543+0000 7efc2042df40 1 -- 172.21.15.38:0/1211757524 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5644c3dbf9d0 con 0x5644c3cf0d00 2024-09-21T03:43:01.547 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.543+0000 7efc2042df40 1 -- 172.21.15.38:0/1211757524 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644c3dc0300 con 0x5644c3cf0d00 2024-09-21T03:43:01.547 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.546+0000 7efc0affd640 1 -- 172.21.15.38:0/1211757524 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7efc0c018020 con 0x5644c3cf0d00 2024-09-21T03:43:01.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.546+0000 7efc0affd640 1 --2- 172.21.15.38:0/1211757524 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7efbfc0775a0 0x7efbfc079ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:01.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.546+0000 7efc0affd640 1 -- 172.21.15.38:0/1211757524 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 8626+0+0 (secure 0 0 0) 0x7efc0c09a5a0 con 0x5644c3cf0d00 2024-09-21T03:43:01.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.546+0000 7efc196e6640 1 --2- 172.21.15.38:0/1211757524 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7efbfc0775a0 0x7efbfc079ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:01.548 INFO:teuthology.orchestra.run.smithi038.stdout:Importing pool 2024-09-21T03:43:01.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.546+0000 7efc2042df40 1 --2- 172.21.15.38:0/1211757524 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x5644c3e61790 0x5644c3e69cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:01.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.546+0000 7efc2042df40 1 -- 172.21.15.38:0/1211757524 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5644c3e5d3b0 con 0x5644c3e61790 2024-09-21T03:43:01.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.547+0000 7efc1a6e8640 1 --2- 172.21.15.38:0/1211757524 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x5644c3e61790 0x5644c3e69cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:01.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.547+0000 7efc196e6640 1 --2- 172.21.15.38:0/1211757524 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7efbfc0775a0 0x7efbfc079ab0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7efc10055000 tx=0x7efc10003020 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:01.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.547+0000 7efc1a6e8640 1 --2- 172.21.15.38:0/1211757524 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x5644c3e61790 0x5644c3e69cc0 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:01.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.550+0000 7efc1a6e8640 1 -- 172.21.15.38:0/1211757524 <== osd.7 v2:172.21.15.195:6818/1606106342 1 ==== osd_op_reply(1 601.00000000 [create] v123'815 uv789 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7efc1400fa90 con 0x5644c3e61790 2024-09-21T03:43:01.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.550+0000 7efc2042df40 1 -- 172.21.15.38:0/1211757524 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e123) -- 0x5644c3dc6540 con 0x5644c3e61790 2024-09-21T03:43:01.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.552+0000 7efc1a6e8640 1 -- 172.21.15.38:0/1211757524 <== osd.7 v2:172.21.15.195:6818/1606106342 2 ==== osd_op_reply(2 601.00000000 [delete] v123'816 uv816 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efc1400fa90 con 0x5644c3e61790 2024-09-21T03:43:01.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.552+0000 7efc2042df40 1 -- 172.21.15.38:0/1211757524 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5644c3e6a200 con 0x5644c3e61790 2024-09-21T03:43:01.555 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-09-21T03:43:01.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.553+0000 7efc1a6e8640 1 -- 172.21.15.38:0/1211757524 <== osd.7 v2:172.21.15.195:6818/1606106342 3 ==== osd_op_reply(3 601.00000000 [create] v123'817 uv817 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efc1400fa90 con 0x5644c3e61790 2024-09-21T03:43:01.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.553+0000 7efc2042df40 1 -- 172.21.15.38:0/1211757524 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5644c3e6b5b0 con 0x5644c3e61790 2024-09-21T03:43:01.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.555+0000 7efc1a6e8640 1 -- 172.21.15.38:0/1211757524 <== osd.7 v2:172.21.15.195:6818/1606106342 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v123'818 uv818 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efc1400fa90 con 0x5644c3e61790 2024-09-21T03:43:01.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.555+0000 7efc2042df40 1 -- 172.21.15.38:0/1211757524 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5644c3e6b950 con 0x5644c3e61790 2024-09-21T03:43:01.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.556+0000 7efc1a6e8640 1 -- 172.21.15.38:0/1211757524 <== osd.7 v2:172.21.15.195:6818/1606106342 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v123'819 uv819 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efc1400fa90 con 0x5644c3e61790 2024-09-21T03:43:01.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.556+0000 7efc2042df40 1 -- 172.21.15.38:0/1211757524 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5644c3e6a5a0 con 0x5644c3e61790 2024-09-21T03:43:01.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.558+0000 7efc1a6e8640 1 -- 172.21.15.38:0/1211757524 <== osd.7 v2:172.21.15.195:6818/1606106342 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v123'820 uv820 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efc1400fa90 con 0x5644c3e61790 2024-09-21T03:43:01.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.558+0000 7efc2042df40 1 -- 172.21.15.38:0/1211757524 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5644c3e6a960 con 0x5644c3e61790 2024-09-21T03:43:01.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.560+0000 7efc1a6e8640 1 -- 172.21.15.38:0/1211757524 <== osd.7 v2:172.21.15.195:6818/1606106342 7 ==== osd_op_reply(7 602.00000000 [create] v123'821 uv795 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7efc1400fa90 con 0x5644c3e61790 2024-09-21T03:43:01.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.560+0000 7efc2042df40 1 -- 172.21.15.38:0/1211757524 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e123) -- 0x5644c3e6ad00 con 0x5644c3e61790 2024-09-21T03:43:01.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.562+0000 7efc1a6e8640 1 -- 172.21.15.38:0/1211757524 <== osd.7 v2:172.21.15.195:6818/1606106342 8 ==== osd_op_reply(8 602.00000000 [delete] v123'822 uv822 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efc1400fa90 con 0x5644c3e61790 2024-09-21T03:43:01.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.562+0000 7efc2042df40 1 -- 172.21.15.38:0/1211757524 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5644c3e6ad00 con 0x5644c3e61790 2024-09-21T03:43:01.565 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-09-21T03:43:01.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.563+0000 7efc1a6e8640 1 -- 172.21.15.38:0/1211757524 <== osd.7 v2:172.21.15.195:6818/1606106342 9 ==== osd_op_reply(9 602.00000000 [create] v123'823 uv823 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efc1400fa90 con 0x5644c3e61790 2024-09-21T03:43:01.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.563+0000 7efc2042df40 1 -- 172.21.15.38:0/1211757524 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5644c3e6ad00 con 0x5644c3e61790 2024-09-21T03:43:01.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.565+0000 7efc1a6e8640 1 -- 172.21.15.38:0/1211757524 <== osd.7 v2:172.21.15.195:6818/1606106342 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v123'824 uv824 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efc1400fa90 con 0x5644c3e61790 2024-09-21T03:43:01.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.565+0000 7efc2042df40 1 -- 172.21.15.38:0/1211757524 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5644c3e6ad00 con 0x5644c3e61790 2024-09-21T03:43:01.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.567+0000 7efc1a6e8640 1 -- 172.21.15.38:0/1211757524 <== osd.7 v2:172.21.15.195:6818/1606106342 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v123'825 uv825 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efc1400fa90 con 0x5644c3e61790 2024-09-21T03:43:01.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.567+0000 7efc2042df40 1 -- 172.21.15.38:0/1211757524 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5644c3e6ad00 con 0x5644c3e61790 2024-09-21T03:43:01.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.568+0000 7efc1a6e8640 1 -- 172.21.15.38:0/1211757524 <== osd.7 v2:172.21.15.195:6818/1606106342 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v123'826 uv826 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efc1400fa90 con 0x5644c3e61790 2024-09-21T03:43:01.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.568+0000 7efc2042df40 1 -- 172.21.15.38:0/1211757524 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:13 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5644c3e6ad00 con 0x5644c3e61790 2024-09-21T03:43:01.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.570+0000 7efc1a6e8640 1 -- 172.21.15.38:0/1211757524 <== osd.7 v2:172.21.15.195:6818/1606106342 13 ==== osd_op_reply(13 10000000000.00000000 [create] v123'827 uv802 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7efc1400fa90 con 0x5644c3e61790 2024-09-21T03:43:01.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.570+0000 7efc2042df40 1 -- 172.21.15.38:0/1211757524 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:14 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e123) -- 0x5644c3e6ad00 con 0x5644c3e61790 2024-09-21T03:43:01.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.572+0000 7efc1a6e8640 1 -- 172.21.15.38:0/1211757524 <== osd.7 v2:172.21.15.195:6818/1606106342 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v123'828 uv828 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7efc1400fa90 con 0x5644c3e61790 2024-09-21T03:43:01.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.572+0000 7efc2042df40 1 -- 172.21.15.38:0/1211757524 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:15 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5644c3e6ad00 con 0x5644c3e61790 2024-09-21T03:43:01.575 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-09-21T03:43:01.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.573+0000 7efc1a6e8640 1 -- 172.21.15.38:0/1211757524 <== osd.7 v2:172.21.15.195:6818/1606106342 15 ==== osd_op_reply(15 10000000000.00000000 [create] v123'829 uv829 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7efc1400fa90 con 0x5644c3e61790 2024-09-21T03:43:01.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.573+0000 7efc2042df40 1 -- 172.21.15.38:0/1211757524 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:16 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5644c3e6ad00 con 0x5644c3e61790 2024-09-21T03:43:01.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.575+0000 7efc1a6e8640 1 -- 172.21.15.38:0/1211757524 <== osd.7 v2:172.21.15.195:6818/1606106342 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v123'830 uv830 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7efc1400fa90 con 0x5644c3e61790 2024-09-21T03:43:01.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.575+0000 7efc2042df40 1 -- 172.21.15.38:0/1211757524 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:17 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5644c3e6ad00 con 0x5644c3e61790 2024-09-21T03:43:01.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.576+0000 7efc1a6e8640 1 -- 172.21.15.38:0/1211757524 <== osd.7 v2:172.21.15.195:6818/1606106342 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v123'831 uv831 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7efc1400fa90 con 0x5644c3e61790 2024-09-21T03:43:01.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.576+0000 7efc2042df40 1 -- 172.21.15.38:0/1211757524 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:18 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5644c3e6ad00 con 0x5644c3e61790 2024-09-21T03:43:01.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.578+0000 7efc1a6e8640 1 -- 172.21.15.38:0/1211757524 <== osd.7 v2:172.21.15.195:6818/1606106342 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v123'832 uv832 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7efc1400fa90 con 0x5644c3e61790 2024-09-21T03:43:01.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.578+0000 7efc2042df40 1 -- 172.21.15.38:0/1211757524 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:19 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=585b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5644c3e6ad00 con 0x5644c3e61790 2024-09-21T03:43:01.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.580+0000 7efc1a6e8640 1 -- 172.21.15.38:0/1211757524 <== osd.7 v2:172.21.15.195:6818/1606106342 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v123'833 uv833 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7efc1400fa90 con 0x5644c3e61790 2024-09-21T03:43:01.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.580+0000 7efc2042df40 1 -- 172.21.15.38:0/1211757524 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:20 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5644c3e6ad00 con 0x5644c3e61790 2024-09-21T03:43:01.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.581+0000 7efc1a6e8640 1 -- 172.21.15.38:0/1211757524 <== osd.7 v2:172.21.15.195:6818/1606106342 20 ==== osd_op_reply(20 600.00000000 [create] v123'834 uv808 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7efc1400fa90 con 0x5644c3e61790 2024-09-21T03:43:01.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.581+0000 7efc2042df40 1 -- 172.21.15.38:0/1211757524 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:21 13.10 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e123) -- 0x5644c3e6ad00 con 0x5644c3e61790 2024-09-21T03:43:01.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.583+0000 7efc1a6e8640 1 -- 172.21.15.38:0/1211757524 <== osd.7 v2:172.21.15.195:6818/1606106342 21 ==== osd_op_reply(21 600.00000000 [delete] v123'835 uv835 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efc1400fa90 con 0x5644c3e61790 2024-09-21T03:43:01.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.583+0000 7efc2042df40 1 -- 172.21.15.38:0/1211757524 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:22 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5644c3e6ad00 con 0x5644c3e61790 2024-09-21T03:43:01.586 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-09-21T03:43:01.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.584+0000 7efc1a6e8640 1 -- 172.21.15.38:0/1211757524 <== osd.7 v2:172.21.15.195:6818/1606106342 22 ==== osd_op_reply(22 600.00000000 [create] v123'836 uv836 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efc1400fa90 con 0x5644c3e61790 2024-09-21T03:43:01.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.585+0000 7efc2042df40 1 -- 172.21.15.38:0/1211757524 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:23 13.10 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5644c3e6ad00 con 0x5644c3e61790 2024-09-21T03:43:01.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.586+0000 7efc1a6e8640 1 -- 172.21.15.38:0/1211757524 <== osd.7 v2:172.21.15.195:6818/1606106342 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v123'837 uv837 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efc1400fa90 con 0x5644c3e61790 2024-09-21T03:43:01.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.586+0000 7efc2042df40 1 -- 172.21.15.38:0/1211757524 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:24 13.10 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5644c3e6ad00 con 0x5644c3e61790 2024-09-21T03:43:01.589 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.587+0000 7efc1a6e8640 1 -- 172.21.15.38:0/1211757524 <== osd.7 v2:172.21.15.195:6818/1606106342 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v123'838 uv838 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efc1400fa90 con 0x5644c3e61790 2024-09-21T03:43:01.589 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.588+0000 7efc2042df40 1 -- 172.21.15.38:0/1211757524 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:25 13.10 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5644c3e6ad00 con 0x5644c3e61790 2024-09-21T03:43:01.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.589+0000 7efc1a6e8640 1 -- 172.21.15.38:0/1211757524 <== osd.7 v2:172.21.15.195:6818/1606106342 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v123'839 uv839 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efc1400fa90 con 0x5644c3e61790 2024-09-21T03:43:01.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.589+0000 7efc2042df40 1 -- 172.21.15.38:0/1211757524 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:26 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5644c3e6ad00 con 0x5644c3e61790 2024-09-21T03:43:01.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.591+0000 7efc1a6e8640 1 -- 172.21.15.38:0/1211757524 <== osd.7 v2:172.21.15.195:6818/1606106342 26 ==== osd_op_reply(26 603.00000000 [create] v123'840 uv814 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7efc1400fa90 con 0x5644c3e61790 2024-09-21T03:43:01.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.591+0000 7efc2042df40 1 -- 172.21.15.38:0/1211757524 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:27 13.10 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e123) -- 0x5644c3e6ad00 con 0x5644c3e61790 2024-09-21T03:43:01.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.592+0000 7efc1a6e8640 1 -- 172.21.15.38:0/1211757524 <== osd.7 v2:172.21.15.195:6818/1606106342 27 ==== osd_op_reply(27 603.00000000 [delete] v123'841 uv841 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efc1400fa90 con 0x5644c3e61790 2024-09-21T03:43:01.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.592+0000 7efc2042df40 1 -- 172.21.15.38:0/1211757524 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:28 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5644c3e6ad00 con 0x5644c3e61790 2024-09-21T03:43:01.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.594+0000 7efc1a6e8640 1 -- 172.21.15.38:0/1211757524 <== osd.7 v2:172.21.15.195:6818/1606106342 28 ==== osd_op_reply(28 603.00000000 [create] v123'842 uv842 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efc1400fa90 con 0x5644c3e61790 2024-09-21T03:43:01.596 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-09-21T03:43:01.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.594+0000 7efc2042df40 1 -- 172.21.15.38:0/1211757524 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:29 13.10 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5644c3e6ad00 con 0x5644c3e61790 2024-09-21T03:43:01.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.595+0000 7efc1a6e8640 1 -- 172.21.15.38:0/1211757524 <== osd.7 v2:172.21.15.195:6818/1606106342 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v123'843 uv843 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efc1400fa90 con 0x5644c3e61790 2024-09-21T03:43:01.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.595+0000 7efc2042df40 1 -- 172.21.15.38:0/1211757524 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:30 13.10 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5644c3e6ad00 con 0x5644c3e61790 2024-09-21T03:43:01.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.597+0000 7efc1a6e8640 1 -- 172.21.15.38:0/1211757524 <== osd.7 v2:172.21.15.195:6818/1606106342 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v123'844 uv844 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efc1400fa90 con 0x5644c3e61790 2024-09-21T03:43:01.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.597+0000 7efc2042df40 1 -- 172.21.15.38:0/1211757524 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:31 13.10 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5644c3e6ad00 con 0x5644c3e61790 2024-09-21T03:43:01.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.599+0000 7efc1a6e8640 1 -- 172.21.15.38:0/1211757524 <== osd.7 v2:172.21.15.195:6818/1606106342 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v123'845 uv845 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efc1400fa90 con 0x5644c3e61790 2024-09-21T03:43:01.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.599+0000 7efc2042df40 1 --2- 172.21.15.38:0/1211757524 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x5644c3e6b150 0x5644c3e6e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:01.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.599+0000 7efc2042df40 1 -- 172.21.15.38:0/1211757524 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5644c3e6ad00 con 0x5644c3e6b150 2024-09-21T03:43:01.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.599+0000 7efc19ee7640 1 --2- 172.21.15.38:0/1211757524 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x5644c3e6b150 0x5644c3e6e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:01.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.600+0000 7efc19ee7640 1 --2- 172.21.15.38:0/1211757524 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x5644c3e6b150 0x5644c3e6e210 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:01.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.601+0000 7efc19ee7640 1 -- 172.21.15.38:0/1211757524 <== osd.6 v2:172.21.15.195:6826/4055825675 1 ==== osd_op_reply(32 1.00000000.inode [create] v123'595 uv574 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7efc0c021070 con 0x5644c3e6b150 2024-09-21T03:43:01.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.601+0000 7efc2042df40 1 -- 172.21.15.38:0/1211757524 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e123) -- 0x5644c3e6e750 con 0x5644c3e6b150 2024-09-21T03:43:01.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.603+0000 7efc19ee7640 1 -- 172.21.15.38:0/1211757524 <== osd.6 v2:172.21.15.195:6826/4055825675 2 ==== osd_op_reply(33 1.00000000.inode [delete] v123'596 uv596 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7efc0c021070 con 0x5644c3e6b150 2024-09-21T03:43:01.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.603+0000 7efc2042df40 1 -- 172.21.15.38:0/1211757524 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5644c3e6eaf0 con 0x5644c3e6b150 2024-09-21T03:43:01.606 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-09-21T03:43:01.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.604+0000 7efc19ee7640 1 -- 172.21.15.38:0/1211757524 <== osd.6 v2:172.21.15.195:6826/4055825675 3 ==== osd_op_reply(34 1.00000000.inode [create] v123'597 uv597 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7efc0c021070 con 0x5644c3e6b150 2024-09-21T03:43:01.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.604+0000 7efc2042df40 1 -- 172.21.15.38:0/1211757524 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5644c3e6ee90 con 0x5644c3e6b150 2024-09-21T03:43:01.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.606+0000 7efc19ee7640 1 -- 172.21.15.38:0/1211757524 <== osd.6 v2:172.21.15.195:6826/4055825675 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v123'598 uv598 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7efc0c021070 con 0x5644c3e6b150 2024-09-21T03:43:01.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.606+0000 7efc2042df40 1 -- 172.21.15.38:0/1211757524 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5644c3e6f230 con 0x5644c3e6b150 2024-09-21T03:43:01.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.607+0000 7efc19ee7640 1 -- 172.21.15.38:0/1211757524 <== osd.6 v2:172.21.15.195:6826/4055825675 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v123'599 uv599 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7efc0c021070 con 0x5644c3e6b150 2024-09-21T03:43:01.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.607+0000 7efc2042df40 1 -- 172.21.15.38:0/1211757524 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5644c3e6f5d0 con 0x5644c3e6b150 2024-09-21T03:43:01.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.609+0000 7efc19ee7640 1 -- 172.21.15.38:0/1211757524 <== osd.6 v2:172.21.15.195:6826/4055825675 6 ==== osd_op_reply(37 200.00000000 [create] v123'600 uv593 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7efc0c021070 con 0x5644c3e6b150 2024-09-21T03:43:01.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.609+0000 7efc2042df40 1 -- 172.21.15.38:0/1211757524 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e123) -- 0x5644c3e6f970 con 0x5644c3e6b150 2024-09-21T03:43:01.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.611+0000 7efc19ee7640 1 -- 172.21.15.38:0/1211757524 <== osd.6 v2:172.21.15.195:6826/4055825675 7 ==== osd_op_reply(38 200.00000000 [delete] v123'601 uv601 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efc0c021070 con 0x5644c3e6b150 2024-09-21T03:43:01.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.611+0000 7efc2042df40 1 -- 172.21.15.38:0/1211757524 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5644c3e6fd10 con 0x5644c3e6b150 2024-09-21T03:43:01.614 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-09-21T03:43:01.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.612+0000 7efc19ee7640 1 -- 172.21.15.38:0/1211757524 <== osd.6 v2:172.21.15.195:6826/4055825675 8 ==== osd_op_reply(39 200.00000000 [create] v123'602 uv602 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efc0c021070 con 0x5644c3e6b150 2024-09-21T03:43:01.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.612+0000 7efc2042df40 1 -- 172.21.15.38:0/1211757524 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5644c3e6fd10 con 0x5644c3e6b150 2024-09-21T03:43:01.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.614+0000 7efc19ee7640 1 -- 172.21.15.38:0/1211757524 <== osd.6 v2:172.21.15.195:6826/4055825675 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v123'603 uv603 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efc0c021070 con 0x5644c3e6b150 2024-09-21T03:43:01.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.614+0000 7efc2042df40 1 -- 172.21.15.38:0/1211757524 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5644c3e6fd10 con 0x5644c3e6b150 2024-09-21T03:43:01.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.615+0000 7efc19ee7640 1 -- 172.21.15.38:0/1211757524 <== osd.6 v2:172.21.15.195:6826/4055825675 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v123'604 uv604 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efc0c021070 con 0x5644c3e6b150 2024-09-21T03:43:01.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.615+0000 7efc2042df40 1 -- 172.21.15.38:0/1211757524 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:42 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5644c3e6fd10 con 0x5644c3e6b150 2024-09-21T03:43:01.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.617+0000 7efc19ee7640 1 -- 172.21.15.38:0/1211757524 <== osd.6 v2:172.21.15.195:6826/4055825675 11 ==== osd_op_reply(42 200.00000001 [create] v123'605 uv594 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7efc0c021070 con 0x5644c3e6b150 2024-09-21T03:43:01.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.617+0000 7efc2042df40 1 -- 172.21.15.38:0/1211757524 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:43 13.14 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e123) -- 0x5644c3e6fd10 con 0x5644c3e6b150 2024-09-21T03:43:01.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.618+0000 7efc19ee7640 1 -- 172.21.15.38:0/1211757524 <== osd.6 v2:172.21.15.195:6826/4055825675 12 ==== osd_op_reply(43 200.00000001 [delete] v123'606 uv606 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efc0c021070 con 0x5644c3e6b150 2024-09-21T03:43:01.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.618+0000 7efc2042df40 1 -- 172.21.15.38:0/1211757524 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:44 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5644c3e6fd10 con 0x5644c3e6b150 2024-09-21T03:43:01.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.619+0000 7efc19ee7640 1 -- 172.21.15.38:0/1211757524 <== osd.6 v2:172.21.15.195:6826/4055825675 13 ==== osd_op_reply(44 200.00000001 [create] v123'607 uv607 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efc0c021070 con 0x5644c3e6b150 2024-09-21T03:43:01.621 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-09-21T03:43:01.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.619+0000 7efc2042df40 1 -- 172.21.15.38:0/1211757524 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:45 13.14 13:2e2fa760:::200.00000001:head [write 0~17490 in=17490b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5644c3e74190 con 0x5644c3e6b150 2024-09-21T03:43:01.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.621+0000 7efc19ee7640 1 -- 172.21.15.38:0/1211757524 <== osd.6 v2:172.21.15.195:6826/4055825675 14 ==== osd_op_reply(45 200.00000001 [write 0~17490] v123'608 uv608 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efc0c021070 con 0x5644c3e6b150 2024-09-21T03:43:01.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.621+0000 7efc2042df40 1 -- 172.21.15.38:0/1211757524 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:46 13.14 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5644c3e74190 con 0x5644c3e6b150 2024-09-21T03:43:01.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.622+0000 7efc19ee7640 1 -- 172.21.15.38:0/1211757524 <== osd.6 v2:172.21.15.195:6826/4055825675 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v123'609 uv609 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efc0c021070 con 0x5644c3e6b150 2024-09-21T03:43:01.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.622+0000 7efc2042df40 1 -- 172.21.15.38:0/1211757524 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:47 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5644c3e6fd10 con 0x5644c3e6b150 2024-09-21T03:43:01.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.624+0000 7efc19ee7640 1 -- 172.21.15.38:0/1211757524 <== osd.6 v2:172.21.15.195:6826/4055825675 16 ==== osd_op_reply(47 606.00000000 [create] v123'610 uv590 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7efc0c021070 con 0x5644c3e6b150 2024-09-21T03:43:01.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.624+0000 7efc2042df40 1 -- 172.21.15.38:0/1211757524 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:48 13.14 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e123) -- 0x5644c3e6fd10 con 0x5644c3e6b150 2024-09-21T03:43:01.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.625+0000 7efc19ee7640 1 -- 172.21.15.38:0/1211757524 <== osd.6 v2:172.21.15.195:6826/4055825675 17 ==== osd_op_reply(48 606.00000000 [delete] v123'611 uv611 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efc0c021070 con 0x5644c3e6b150 2024-09-21T03:43:01.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.625+0000 7efc2042df40 1 -- 172.21.15.38:0/1211757524 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:49 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5644c3e6fd10 con 0x5644c3e6b150 2024-09-21T03:43:01.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.627+0000 7efc19ee7640 1 -- 172.21.15.38:0/1211757524 <== osd.6 v2:172.21.15.195:6826/4055825675 18 ==== osd_op_reply(49 606.00000000 [create] v123'612 uv612 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efc0c021070 con 0x5644c3e6b150 2024-09-21T03:43:01.629 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-09-21T03:43:01.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.627+0000 7efc2042df40 1 -- 172.21.15.38:0/1211757524 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:50 13.14 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5644c3e70d20 con 0x5644c3e6b150 2024-09-21T03:43:01.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.628+0000 7efc19ee7640 1 -- 172.21.15.38:0/1211757524 <== osd.6 v2:172.21.15.195:6826/4055825675 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v123'613 uv613 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efc0c021070 con 0x5644c3e6b150 2024-09-21T03:43:01.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.628+0000 7efc2042df40 1 -- 172.21.15.38:0/1211757524 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:51 13.14 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5644c3e70d20 con 0x5644c3e6b150 2024-09-21T03:43:01.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.629+0000 7efc19ee7640 1 -- 172.21.15.38:0/1211757524 <== osd.6 v2:172.21.15.195:6826/4055825675 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v123'614 uv614 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efc0c021070 con 0x5644c3e6b150 2024-09-21T03:43:01.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.629+0000 7efc2042df40 1 -- 172.21.15.38:0/1211757524 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:52 13.14 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5644c3e701c0 con 0x5644c3e6b150 2024-09-21T03:43:01.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.631+0000 7efc19ee7640 1 -- 172.21.15.38:0/1211757524 <== osd.6 v2:172.21.15.195:6826/4055825675 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v123'615 uv615 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efc0c021070 con 0x5644c3e6b150 2024-09-21T03:43:01.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.631+0000 7efc2042df40 1 --2- 172.21.15.38:0/1211757524 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x5644c3e70a90 0x5644c3e72eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:01.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.631+0000 7efc2042df40 1 -- 172.21.15.38:0/1211757524 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:53 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5644c3e701c0 con 0x5644c3e70a90 2024-09-21T03:43:01.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.631+0000 7efc196e6640 1 --2- 172.21.15.38:0/1211757524 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x5644c3e70a90 0x5644c3e72eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:01.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.631+0000 7efc196e6640 1 --2- 172.21.15.38:0/1211757524 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x5644c3e70a90 0x5644c3e72eb0 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:01.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.633+0000 7efc196e6640 1 -- 172.21.15.38:0/1211757524 <== osd.0 v2:172.21.15.38:6800/2381639433 1 ==== osd_op_reply(53 607.00000000 [create] v123'159 uv158 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7efc10056470 con 0x5644c3e70a90 2024-09-21T03:43:01.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.633+0000 7efc2042df40 1 -- 172.21.15.38:0/1211757524 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:54 13.1c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e123) -- 0x5644c3e734d0 con 0x5644c3e70a90 2024-09-21T03:43:01.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.634+0000 7efc196e6640 1 -- 172.21.15.38:0/1211757524 <== osd.0 v2:172.21.15.38:6800/2381639433 2 ==== osd_op_reply(54 607.00000000 [delete] v123'160 uv160 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efc10056470 con 0x5644c3e70a90 2024-09-21T03:43:01.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.634+0000 7efc2042df40 1 -- 172.21.15.38:0/1211757524 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:55 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5644c3e73940 con 0x5644c3e70a90 2024-09-21T03:43:01.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.635+0000 7efc196e6640 1 -- 172.21.15.38:0/1211757524 <== osd.0 v2:172.21.15.38:6800/2381639433 3 ==== osd_op_reply(55 607.00000000 [create] v123'161 uv161 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efc10056470 con 0x5644c3e70a90 2024-09-21T03:43:01.637 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-09-21T03:43:01.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.635+0000 7efc2042df40 1 -- 172.21.15.38:0/1211757524 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:56 13.1c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5644c3e751f0 con 0x5644c3e70a90 2024-09-21T03:43:01.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.636+0000 7efc196e6640 1 -- 172.21.15.38:0/1211757524 <== osd.0 v2:172.21.15.38:6800/2381639433 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v123'162 uv162 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efc10056470 con 0x5644c3e70a90 2024-09-21T03:43:01.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.636+0000 7efc2042df40 1 -- 172.21.15.38:0/1211757524 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:57 13.1c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5644c3e75590 con 0x5644c3e70a90 2024-09-21T03:43:01.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.637+0000 7efc196e6640 1 -- 172.21.15.38:0/1211757524 <== osd.0 v2:172.21.15.38:6800/2381639433 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v123'163 uv163 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efc10056470 con 0x5644c3e70a90 2024-09-21T03:43:01.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.637+0000 7efc2042df40 1 -- 172.21.15.38:0/1211757524 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:58 13.1c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5644c3e741e0 con 0x5644c3e70a90 2024-09-21T03:43:01.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.638+0000 7efc196e6640 1 -- 172.21.15.38:0/1211757524 <== osd.0 v2:172.21.15.38:6800/2381639433 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v123'164 uv164 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efc10056470 con 0x5644c3e70a90 2024-09-21T03:43:01.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.638+0000 7efc2042df40 1 --2- 172.21.15.38:0/1211757524 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x5644c3e74b00 0x5644c3e77940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:01.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.638+0000 7efc2042df40 1 -- 172.21.15.38:0/1211757524 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:59 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5644c3e77e80 con 0x5644c3e74b00 2024-09-21T03:43:01.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.638+0000 7efc1a6e8640 1 --2- 172.21.15.38:0/1211757524 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x5644c3e74b00 0x5644c3e77940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:01.641 DEBUG:teuthology.orchestra.run.smithi038:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T03:43:01.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.639+0000 7efc1a6e8640 1 --2- 172.21.15.38:0/1211757524 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x5644c3e74b00 0x5644c3e77940 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:01.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.640+0000 7efc1a6e8640 1 -- 172.21.15.38:0/1211757524 <== osd.3 v2:172.21.15.38:6808/1286810294 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v123'116 uv115 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7efc1400fa90 con 0x5644c3e74b00 2024-09-21T03:43:01.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.640+0000 7efc2042df40 1 -- 172.21.15.38:0/1211757524 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:60 13.6 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e123) -- 0x5644c3e77e80 con 0x5644c3e74b00 2024-09-21T03:43:01.644 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T03:43:01.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.641+0000 7efc1a6e8640 1 -- 172.21.15.38:0/1211757524 <== osd.3 v2:172.21.15.38:6808/1286810294 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v123'117 uv117 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7efc1400fa90 con 0x5644c3e74b00 2024-09-21T03:43:01.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.641+0000 7efc2042df40 1 -- 172.21.15.38:0/1211757524 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:61 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5644c3e782a0 con 0x5644c3e74b00 2024-09-21T03:43:01.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.642+0000 7efc1a6e8640 1 -- 172.21.15.38:0/1211757524 <== osd.3 v2:172.21.15.38:6808/1286810294 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v123'118 uv118 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7efc1400fa90 con 0x5644c3e74b00 2024-09-21T03:43:01.645 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-09-21T03:43:01.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.642+0000 7efc2042df40 1 -- 172.21.15.38:0/1211757524 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:62 13.6 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5644c3e782a0 con 0x5644c3e74b00 2024-09-21T03:43:01.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.643+0000 7efc1a6e8640 1 -- 172.21.15.38:0/1211757524 <== osd.3 v2:172.21.15.38:6808/1286810294 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v123'119 uv119 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7efc1400fa90 con 0x5644c3e74b00 2024-09-21T03:43:01.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.643+0000 7efc2042df40 1 -- 172.21.15.38:0/1211757524 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5644c3e782a0 con 0x5644c3e70a90 2024-09-21T03:43:01.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.645+0000 7efc196e6640 1 -- 172.21.15.38:0/1211757524 <== osd.0 v2:172.21.15.38:6800/2381639433 7 ==== osd_op_reply(63 608.00000000 [create] v123'159 uv158 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7efc10056470 con 0x5644c3e70a90 2024-09-21T03:43:01.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.645+0000 7efc2042df40 1 -- 172.21.15.38:0/1211757524 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e123) -- 0x5644c3e782a0 con 0x5644c3e70a90 2024-09-21T03:43:01.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.646+0000 7efc196e6640 1 -- 172.21.15.38:0/1211757524 <== osd.0 v2:172.21.15.38:6800/2381639433 8 ==== osd_op_reply(64 608.00000000 [delete] v123'160 uv160 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efc10056470 con 0x5644c3e70a90 2024-09-21T03:43:01.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.646+0000 7efc2042df40 1 -- 172.21.15.38:0/1211757524 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5644c3e782a0 con 0x5644c3e70a90 2024-09-21T03:43:01.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.647+0000 7efc196e6640 1 -- 172.21.15.38:0/1211757524 <== osd.0 v2:172.21.15.38:6800/2381639433 9 ==== osd_op_reply(65 608.00000000 [create] v123'161 uv161 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efc10056470 con 0x5644c3e70a90 2024-09-21T03:43:01.649 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-09-21T03:43:01.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.647+0000 7efc2042df40 1 -- 172.21.15.38:0/1211757524 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5644c3e792b0 con 0x5644c3e70a90 2024-09-21T03:43:01.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.649+0000 7efc196e6640 1 -- 172.21.15.38:0/1211757524 <== osd.0 v2:172.21.15.38:6800/2381639433 10 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v123'162 uv162 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efc10056470 con 0x5644c3e70a90 2024-09-21T03:43:01.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.649+0000 7efc2042df40 1 -- 172.21.15.38:0/1211757524 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5644c3e792b0 con 0x5644c3e70a90 2024-09-21T03:43:01.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.650+0000 7efc196e6640 1 -- 172.21.15.38:0/1211757524 <== osd.0 v2:172.21.15.38:6800/2381639433 11 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v123'163 uv163 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efc10056470 con 0x5644c3e70a90 2024-09-21T03:43:01.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.650+0000 7efc2042df40 1 -- 172.21.15.38:0/1211757524 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5644c3e782a0 con 0x5644c3e70a90 2024-09-21T03:43:01.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.651+0000 7efc196e6640 1 -- 172.21.15.38:0/1211757524 <== osd.0 v2:172.21.15.38:6800/2381639433 12 ==== osd_op_reply(68 608.00000000 [omap-set-header] v123'164 uv164 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efc10056470 con 0x5644c3e70a90 2024-09-21T03:43:01.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.651+0000 7efc2042df40 1 -- 172.21.15.38:0/1211757524 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5644c3e782a0 con 0x5644c3e74b00 2024-09-21T03:43:01.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.653+0000 7efc1a6e8640 1 -- 172.21.15.38:0/1211757524 <== osd.3 v2:172.21.15.38:6808/1286810294 5 ==== osd_op_reply(69 604.00000000 [create] v123'294 uv287 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7efc1400fa90 con 0x5644c3e74b00 2024-09-21T03:43:01.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.653+0000 7efc2042df40 1 -- 172.21.15.38:0/1211757524 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e123) -- 0x5644c3e782a0 con 0x5644c3e74b00 2024-09-21T03:43:01.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.654+0000 7efc1a6e8640 1 -- 172.21.15.38:0/1211757524 <== osd.3 v2:172.21.15.38:6808/1286810294 6 ==== osd_op_reply(70 604.00000000 [delete] v123'295 uv295 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efc1400fa90 con 0x5644c3e74b00 2024-09-21T03:43:01.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.654+0000 7efc2042df40 1 -- 172.21.15.38:0/1211757524 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5644c3e782a0 con 0x5644c3e74b00 2024-09-21T03:43:01.657 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-09-21T03:43:01.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.655+0000 7efc1a6e8640 1 -- 172.21.15.38:0/1211757524 <== osd.3 v2:172.21.15.38:6808/1286810294 7 ==== osd_op_reply(71 604.00000000 [create] v123'296 uv296 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efc1400fa90 con 0x5644c3e74b00 2024-09-21T03:43:01.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.655+0000 7efc2042df40 1 -- 172.21.15.38:0/1211757524 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5644c3e792b0 con 0x5644c3e74b00 2024-09-21T03:43:01.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.657+0000 7efc1a6e8640 1 -- 172.21.15.38:0/1211757524 <== osd.3 v2:172.21.15.38:6808/1286810294 8 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v123'297 uv297 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efc1400fa90 con 0x5644c3e74b00 2024-09-21T03:43:01.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.657+0000 7efc2042df40 1 -- 172.21.15.38:0/1211757524 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5644c3e792b0 con 0x5644c3e74b00 2024-09-21T03:43:01.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.658+0000 7efc1a6e8640 1 -- 172.21.15.38:0/1211757524 <== osd.3 v2:172.21.15.38:6808/1286810294 9 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v123'298 uv298 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efc1400fa90 con 0x5644c3e74b00 2024-09-21T03:43:01.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.658+0000 7efc2042df40 1 -- 172.21.15.38:0/1211757524 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5644c3e782a0 con 0x5644c3e74b00 2024-09-21T03:43:01.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.659+0000 7efc1a6e8640 1 -- 172.21.15.38:0/1211757524 <== osd.3 v2:172.21.15.38:6808/1286810294 10 ==== osd_op_reply(74 604.00000000 [omap-set-header] v123'299 uv299 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efc1400fa90 con 0x5644c3e74b00 2024-09-21T03:43:01.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.659+0000 7efc2042df40 1 -- 172.21.15.38:0/1211757524 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5644c3e782a0 con 0x5644c3e74b00 2024-09-21T03:43:01.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.661+0000 7efc1a6e8640 1 -- 172.21.15.38:0/1211757524 <== osd.3 v2:172.21.15.38:6808/1286810294 11 ==== osd_op_reply(75 500.00000000 [create] v123'300 uv293 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7efc1400fa90 con 0x5644c3e74b00 2024-09-21T03:43:01.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.661+0000 7efc2042df40 1 -- 172.21.15.38:0/1211757524 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e123) -- 0x5644c3e782a0 con 0x5644c3e74b00 2024-09-21T03:43:01.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.662+0000 7efc1a6e8640 1 -- 172.21.15.38:0/1211757524 <== osd.3 v2:172.21.15.38:6808/1286810294 12 ==== osd_op_reply(76 500.00000000 [delete] v123'301 uv301 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efc1400fa90 con 0x5644c3e74b00 2024-09-21T03:43:01.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.662+0000 7efc2042df40 1 -- 172.21.15.38:0/1211757524 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5644c3e782a0 con 0x5644c3e74b00 2024-09-21T03:43:01.665 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-09-21T03:43:01.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.663+0000 7efc1a6e8640 1 -- 172.21.15.38:0/1211757524 <== osd.3 v2:172.21.15.38:6808/1286810294 13 ==== osd_op_reply(77 500.00000000 [create] v123'302 uv302 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efc1400fa90 con 0x5644c3e74b00 2024-09-21T03:43:01.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.663+0000 7efc2042df40 1 -- 172.21.15.38:0/1211757524 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5644c3e782a0 con 0x5644c3e74b00 2024-09-21T03:43:01.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.665+0000 7efc1a6e8640 1 -- 172.21.15.38:0/1211757524 <== osd.3 v2:172.21.15.38:6808/1286810294 14 ==== osd_op_reply(78 500.00000000 [write 0~90] v123'303 uv303 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efc1400fa90 con 0x5644c3e74b00 2024-09-21T03:43:01.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.665+0000 7efc2042df40 1 -- 172.21.15.38:0/1211757524 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5644c3e782a0 con 0x5644c3e74b00 2024-09-21T03:43:01.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.666+0000 7efc1a6e8640 1 -- 172.21.15.38:0/1211757524 <== osd.3 v2:172.21.15.38:6808/1286810294 15 ==== osd_op_reply(79 500.00000000 [omap-set-header] v123'304 uv304 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efc1400fa90 con 0x5644c3e74b00 2024-09-21T03:43:01.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.666+0000 7efc2042df40 1 -- 172.21.15.38:0/1211757524 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:80 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5644c3e782a0 con 0x5644c3e70a90 2024-09-21T03:43:01.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.667+0000 7efc196e6640 1 -- 172.21.15.38:0/1211757524 <== osd.0 v2:172.21.15.38:6800/2381639433 13 ==== osd_op_reply(80 mds_snaptable [create] v123'132 uv131 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7efc10056470 con 0x5644c3e70a90 2024-09-21T03:43:01.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.668+0000 7efc2042df40 1 -- 172.21.15.38:0/1211757524 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:81 13.d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e123) -- 0x5644c3e782a0 con 0x5644c3e70a90 2024-09-21T03:43:01.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.669+0000 7efc196e6640 1 -- 172.21.15.38:0/1211757524 <== osd.0 v2:172.21.15.38:6800/2381639433 14 ==== osd_op_reply(81 mds_snaptable [delete] v123'133 uv133 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7efc10056470 con 0x5644c3e70a90 2024-09-21T03:43:01.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.669+0000 7efc2042df40 1 -- 172.21.15.38:0/1211757524 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:82 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5644c3e782a0 con 0x5644c3e70a90 2024-09-21T03:43:01.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.670+0000 7efc196e6640 1 -- 172.21.15.38:0/1211757524 <== osd.0 v2:172.21.15.38:6800/2381639433 15 ==== osd_op_reply(82 mds_snaptable [create] v123'134 uv134 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7efc10056470 con 0x5644c3e70a90 2024-09-21T03:43:01.672 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-09-21T03:43:01.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.670+0000 7efc2042df40 1 -- 172.21.15.38:0/1211757524 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:83 13.d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5644c3e782a0 con 0x5644c3e70a90 2024-09-21T03:43:01.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.671+0000 7efc196e6640 1 -- 172.21.15.38:0/1211757524 <== osd.0 v2:172.21.15.38:6800/2381639433 16 ==== osd_op_reply(83 mds_snaptable [write 0~70] v123'135 uv135 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7efc10056470 con 0x5644c3e70a90 2024-09-21T03:43:01.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.671+0000 7efc2042df40 1 -- 172.21.15.38:0/1211757524 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:84 13.d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5644c3e782a0 con 0x5644c3e70a90 2024-09-21T03:43:01.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.672+0000 7efc196e6640 1 -- 172.21.15.38:0/1211757524 <== osd.0 v2:172.21.15.38:6800/2381639433 17 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v123'136 uv136 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7efc10056470 con 0x5644c3e70a90 2024-09-21T03:43:01.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.672+0000 7efc2042df40 1 --2- 172.21.15.38:0/1211757524 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x5644c3e78b70 0x5644c3e7af90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:01.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.672+0000 7efc2042df40 1 -- 172.21.15.38:0/1211757524 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5644c3e7b4d0 con 0x5644c3e78b70 2024-09-21T03:43:01.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.672+0000 7efc19ee7640 1 --2- 172.21.15.38:0/1211757524 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x5644c3e78b70 0x5644c3e7af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:01.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.673+0000 7efc19ee7640 1 --2- 172.21.15.38:0/1211757524 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x5644c3e78b70 0x5644c3e7af90 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:01.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.674+0000 7efc19ee7640 1 -- 172.21.15.38:0/1211757524 <== osd.1 v2:172.21.15.38:6816/1491818732 1 ==== osd_op_reply(85 605.00000000 [create] v123'159 uv158 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7efc0c021070 con 0x5644c3e78b70 2024-09-21T03:43:01.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.674+0000 7efc2042df40 1 -- 172.21.15.38:0/1211757524 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e123) -- 0x5644c3e7b4d0 con 0x5644c3e78b70 2024-09-21T03:43:01.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.675+0000 7efc19ee7640 1 -- 172.21.15.38:0/1211757524 <== osd.1 v2:172.21.15.38:6816/1491818732 2 ==== osd_op_reply(86 605.00000000 [delete] v123'160 uv160 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efc0c021070 con 0x5644c3e78b70 2024-09-21T03:43:01.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.675+0000 7efc2042df40 1 -- 172.21.15.38:0/1211757524 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5644c3e7b8f0 con 0x5644c3e78b70 2024-09-21T03:43:01.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.676+0000 7efc19ee7640 1 -- 172.21.15.38:0/1211757524 <== osd.1 v2:172.21.15.38:6816/1491818732 3 ==== osd_op_reply(87 605.00000000 [create] v123'161 uv161 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efc0c021070 con 0x5644c3e78b70 2024-09-21T03:43:01.679 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-09-21T03:43:01.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.676+0000 7efc2042df40 1 -- 172.21.15.38:0/1211757524 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5644c3e7c900 con 0x5644c3e78b70 2024-09-21T03:43:01.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.678+0000 7efc19ee7640 1 -- 172.21.15.38:0/1211757524 <== osd.1 v2:172.21.15.38:6816/1491818732 4 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v123'162 uv162 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efc0c021070 con 0x5644c3e78b70 2024-09-21T03:43:01.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.678+0000 7efc2042df40 1 -- 172.21.15.38:0/1211757524 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5644c3e7c900 con 0x5644c3e78b70 2024-09-21T03:43:01.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.679+0000 7efc19ee7640 1 -- 172.21.15.38:0/1211757524 <== osd.1 v2:172.21.15.38:6816/1491818732 5 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v123'163 uv163 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efc0c021070 con 0x5644c3e78b70 2024-09-21T03:43:01.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.679+0000 7efc2042df40 1 -- 172.21.15.38:0/1211757524 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5644c3e7b8f0 con 0x5644c3e78b70 2024-09-21T03:43:01.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.680+0000 7efc19ee7640 1 -- 172.21.15.38:0/1211757524 <== osd.1 v2:172.21.15.38:6816/1491818732 6 ==== osd_op_reply(90 605.00000000 [omap-set-header] v123'164 uv164 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efc0c021070 con 0x5644c3e78b70 2024-09-21T03:43:01.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.680+0000 7efc2042df40 1 -- 172.21.15.38:0/1211757524 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5644c3e7b8f0 con 0x5644c3e78b70 2024-09-21T03:43:01.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.681+0000 7efc19ee7640 1 -- 172.21.15.38:0/1211757524 <== osd.1 v2:172.21.15.38:6816/1491818732 7 ==== osd_op_reply(91 mds0_inotable [create] v123'263 uv257 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7efc0c021070 con 0x5644c3e78b70 2024-09-21T03:43:01.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.681+0000 7efc2042df40 1 -- 172.21.15.38:0/1211757524 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e123) -- 0x5644c3e7b8f0 con 0x5644c3e78b70 2024-09-21T03:43:01.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.682+0000 7efc19ee7640 1 -- 172.21.15.38:0/1211757524 <== osd.1 v2:172.21.15.38:6816/1491818732 8 ==== osd_op_reply(92 mds0_inotable [delete] v123'264 uv264 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7efc0c021070 con 0x5644c3e78b70 2024-09-21T03:43:01.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.682+0000 7efc2042df40 1 -- 172.21.15.38:0/1211757524 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5644c3e7b8f0 con 0x5644c3e78b70 2024-09-21T03:43:01.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.683+0000 7efc19ee7640 1 -- 172.21.15.38:0/1211757524 <== osd.1 v2:172.21.15.38:6816/1491818732 9 ==== osd_op_reply(93 mds0_inotable [create] v123'265 uv265 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7efc0c021070 con 0x5644c3e78b70 2024-09-21T03:43:01.685 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-09-21T03:43:01.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.683+0000 7efc2042df40 1 -- 172.21.15.38:0/1211757524 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:94 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5644c3e7b8f0 con 0x5644c3e78b70 2024-09-21T03:43:01.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.684+0000 7efc19ee7640 1 -- 172.21.15.38:0/1211757524 <== osd.1 v2:172.21.15.38:6816/1491818732 10 ==== osd_op_reply(94 mds0_inotable [write 0~34] v123'266 uv266 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7efc0c021070 con 0x5644c3e78b70 2024-09-21T03:43:01.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.684+0000 7efc2042df40 1 -- 172.21.15.38:0/1211757524 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:95 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5644c3e7b8f0 con 0x5644c3e78b70 2024-09-21T03:43:01.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.685+0000 7efc19ee7640 1 -- 172.21.15.38:0/1211757524 <== osd.1 v2:172.21.15.38:6816/1491818732 11 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v123'267 uv267 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7efc0c021070 con 0x5644c3e78b70 2024-09-21T03:43:01.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.686+0000 7efc2042df40 1 -- 172.21.15.38:0/1211757524 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:96 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5644c3e7b8f0 con 0x5644c3e78b70 2024-09-21T03:43:01.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.687+0000 7efc19ee7640 1 -- 172.21.15.38:0/1211757524 <== osd.1 v2:172.21.15.38:6816/1491818732 12 ==== osd_op_reply(96 100.00000000 [create] v123'268 uv262 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7efc0c021070 con 0x5644c3e78b70 2024-09-21T03:43:01.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.687+0000 7efc2042df40 1 -- 172.21.15.38:0/1211757524 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:97 13.13 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e123) -- 0x5644c3e7b8f0 con 0x5644c3e78b70 2024-09-21T03:43:01.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.688+0000 7efc19ee7640 1 -- 172.21.15.38:0/1211757524 <== osd.1 v2:172.21.15.38:6816/1491818732 13 ==== osd_op_reply(97 100.00000000 [delete] v123'269 uv269 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efc0c021070 con 0x5644c3e78b70 2024-09-21T03:43:01.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.688+0000 7efc2042df40 1 -- 172.21.15.38:0/1211757524 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:98 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5644c3e7b8f0 con 0x5644c3e78b70 2024-09-21T03:43:01.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.689+0000 7efc19ee7640 1 -- 172.21.15.38:0/1211757524 <== osd.1 v2:172.21.15.38:6816/1491818732 14 ==== osd_op_reply(98 100.00000000 [create] v123'270 uv270 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efc0c021070 con 0x5644c3e78b70 2024-09-21T03:43:01.691 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-09-21T03:43:01.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.689+0000 7efc2042df40 1 -- 172.21.15.38:0/1211757524 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:99 13.13 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5644c3e7b8f0 con 0x5644c3e78b70 2024-09-21T03:43:01.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.690+0000 7efc19ee7640 1 -- 172.21.15.38:0/1211757524 <== osd.1 v2:172.21.15.38:6816/1491818732 15 ==== osd_op_reply(99 100.00000000 [omap-set-header] v123'271 uv271 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efc0c021070 con 0x5644c3e78b70 2024-09-21T03:43:01.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.690+0000 7efc2042df40 1 -- 172.21.15.38:0/1211757524 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:100 13.13 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5644c3e7e700 con 0x5644c3e78b70 2024-09-21T03:43:01.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.691+0000 7efc19ee7640 1 -- 172.21.15.38:0/1211757524 <== osd.1 v2:172.21.15.38:6816/1491818732 16 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v123'272 uv272 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efc0c021070 con 0x5644c3e78b70 2024-09-21T03:43:01.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.692+0000 7efc2042df40 1 -- 172.21.15.38:0/1211757524 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5644c3e7bda0 con 0x5644c3e74b00 2024-09-21T03:43:01.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.693+0000 7efc1a6e8640 1 -- 172.21.15.38:0/1211757524 <== osd.3 v2:172.21.15.38:6808/1286810294 16 ==== osd_op_reply(101 mds0_sessionmap [create] v123'116 uv114 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7efc1400fa90 con 0x5644c3e74b00 2024-09-21T03:43:01.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.693+0000 7efc2042df40 1 -- 172.21.15.38:0/1211757524 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e123) -- 0x5644c3e7bda0 con 0x5644c3e74b00 2024-09-21T03:43:01.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.694+0000 7efc1a6e8640 1 -- 172.21.15.38:0/1211757524 <== osd.3 v2:172.21.15.38:6808/1286810294 17 ==== osd_op_reply(102 mds0_sessionmap [delete] v123'117 uv117 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7efc1400fa90 con 0x5644c3e74b00 2024-09-21T03:43:01.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.694+0000 7efc2042df40 1 -- 172.21.15.38:0/1211757524 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:103 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5644c3e7bda0 con 0x5644c3e74b00 2024-09-21T03:43:01.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.695+0000 7efc1a6e8640 1 -- 172.21.15.38:0/1211757524 <== osd.3 v2:172.21.15.38:6808/1286810294 18 ==== osd_op_reply(103 mds0_sessionmap [create] v123'118 uv118 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7efc1400fa90 con 0x5644c3e74b00 2024-09-21T03:43:01.697 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-09-21T03:43:01.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.695+0000 7efc2042df40 1 -- 172.21.15.38:0/1211757524 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:104 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5644c3e7bda0 con 0x5644c3e74b00 2024-09-21T03:43:01.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.696+0000 7efc1a6e8640 1 -- 172.21.15.38:0/1211757524 <== osd.3 v2:172.21.15.38:6808/1286810294 19 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v123'119 uv119 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7efc1400fa90 con 0x5644c3e74b00 2024-09-21T03:43:01.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.696+0000 7efc2042df40 1 -- 172.21.15.38:0/1211757524 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5644c3e7bda0 con 0x5644c3e74b00 2024-09-21T03:43:01.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.697+0000 7efc1a6e8640 1 -- 172.21.15.38:0/1211757524 <== osd.3 v2:172.21.15.38:6808/1286810294 20 ==== osd_op_reply(105 609.00000000 [create] v123'159 uv158 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7efc1400fa90 con 0x5644c3e74b00 2024-09-21T03:43:01.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.697+0000 7efc2042df40 1 -- 172.21.15.38:0/1211757524 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e123) -- 0x5644c3e7bda0 con 0x5644c3e74b00 2024-09-21T03:43:01.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.698+0000 7efc1a6e8640 1 -- 172.21.15.38:0/1211757524 <== osd.3 v2:172.21.15.38:6808/1286810294 21 ==== osd_op_reply(106 609.00000000 [delete] v123'160 uv160 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efc1400fa90 con 0x5644c3e74b00 2024-09-21T03:43:01.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.698+0000 7efc2042df40 1 -- 172.21.15.38:0/1211757524 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5644c3e7bda0 con 0x5644c3e74b00 2024-09-21T03:43:01.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.699+0000 7efc1a6e8640 1 -- 172.21.15.38:0/1211757524 <== osd.3 v2:172.21.15.38:6808/1286810294 22 ==== osd_op_reply(107 609.00000000 [create] v123'161 uv161 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efc1400fa90 con 0x5644c3e74b00 2024-09-21T03:43:01.701 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-09-21T03:43:01.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.699+0000 7efc2042df40 1 -- 172.21.15.38:0/1211757524 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5644c3e7cdb0 con 0x5644c3e74b00 2024-09-21T03:43:01.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.700+0000 7efc1a6e8640 1 -- 172.21.15.38:0/1211757524 <== osd.3 v2:172.21.15.38:6808/1286810294 23 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v123'162 uv162 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efc1400fa90 con 0x5644c3e74b00 2024-09-21T03:43:01.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.700+0000 7efc2042df40 1 -- 172.21.15.38:0/1211757524 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:109 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5644c3e7cdb0 con 0x5644c3e74b00 2024-09-21T03:43:01.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.702+0000 7efc1a6e8640 1 -- 172.21.15.38:0/1211757524 <== osd.3 v2:172.21.15.38:6808/1286810294 24 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v123'163 uv163 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efc1400fa90 con 0x5644c3e74b00 2024-09-21T03:43:01.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.702+0000 7efc2042df40 1 -- 172.21.15.38:0/1211757524 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:110 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5644c3e7bda0 con 0x5644c3e74b00 2024-09-21T03:43:01.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.703+0000 7efc1a6e8640 1 -- 172.21.15.38:0/1211757524 <== osd.3 v2:172.21.15.38:6808/1286810294 25 ==== osd_op_reply(110 609.00000000 [omap-set-header] v123'164 uv164 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efc1400fa90 con 0x5644c3e74b00 2024-09-21T03:43:01.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.703+0000 7efc2042df40 1 --2- 172.21.15.38:0/1211757524 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x5644c3e7c2b0 0x5644c3e7c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:01.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.703+0000 7efc2042df40 1 -- 172.21.15.38:0/1211757524 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5644c3e7cc00 con 0x5644c3e7c2b0 2024-09-21T03:43:01.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.703+0000 7efc196e6640 1 --2- 172.21.15.38:0/1211757524 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x5644c3e7c2b0 0x5644c3e7c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:01.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.703+0000 7efc196e6640 1 --2- 172.21.15.38:0/1211757524 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x5644c3e7c2b0 0x5644c3e7c6c0 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:01.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.704+0000 7efc196e6640 1 -- 172.21.15.38:0/1211757524 <== osd.2 v2:172.21.15.38:6824/2577858902 1 ==== osd_op_reply(111 400.00000000 [create] v123'135 uv134 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7efc10056470 con 0x5644c3e7c2b0 2024-09-21T03:43:01.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.704+0000 7efc2042df40 1 -- 172.21.15.38:0/1211757524 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e123) -- 0x5644c3e7cc00 con 0x5644c3e7c2b0 2024-09-21T03:43:01.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.706+0000 7efc196e6640 1 -- 172.21.15.38:0/1211757524 <== osd.2 v2:172.21.15.38:6824/2577858902 2 ==== osd_op_reply(112 400.00000000 [delete] v123'136 uv136 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efc10056470 con 0x5644c3e7c2b0 2024-09-21T03:43:01.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.706+0000 7efc2042df40 1 -- 172.21.15.38:0/1211757524 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5644c3e7cc00 con 0x5644c3e7c2b0 2024-09-21T03:43:01.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.707+0000 7efc196e6640 1 -- 172.21.15.38:0/1211757524 <== osd.2 v2:172.21.15.38:6824/2577858902 3 ==== osd_op_reply(113 400.00000000 [create] v123'137 uv137 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efc10056470 con 0x5644c3e7c2b0 2024-09-21T03:43:01.709 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-09-21T03:43:01.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.707+0000 7efc2042df40 1 -- 172.21.15.38:0/1211757524 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:114 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5644c3e7cc00 con 0x5644c3e7c2b0 2024-09-21T03:43:01.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.708+0000 7efc196e6640 1 -- 172.21.15.38:0/1211757524 <== osd.2 v2:172.21.15.38:6824/2577858902 4 ==== osd_op_reply(114 400.00000000 [write 0~22] v123'138 uv138 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efc10056470 con 0x5644c3e7c2b0 2024-09-21T03:43:01.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.708+0000 7efc2042df40 1 -- 172.21.15.38:0/1211757524 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:115 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5644c3e7cc00 con 0x5644c3e7c2b0 2024-09-21T03:43:01.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.709+0000 7efc196e6640 1 -- 172.21.15.38:0/1211757524 <== osd.2 v2:172.21.15.38:6824/2577858902 5 ==== osd_op_reply(115 400.00000000 [omap-set-header] v123'139 uv139 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efc10056470 con 0x5644c3e7c2b0 2024-09-21T03:43:01.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.709+0000 7efc2042df40 1 -- 172.21.15.38:0/1211757524 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:116 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5644c3e7cc00 con 0x5644c3e78b70 2024-09-21T03:43:01.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.711+0000 7efc19ee7640 1 -- 172.21.15.38:0/1211757524 <== osd.1 v2:172.21.15.38:6816/1491818732 17 ==== osd_op_reply(116 100.00000000.inode [create] v123'327 uv318 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7efc0c021070 con 0x5644c3e78b70 2024-09-21T03:43:01.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.711+0000 7efc2042df40 1 -- 172.21.15.38:0/1211757524 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:117 13.1f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e123) -- 0x5644c3e7cc00 con 0x5644c3e78b70 2024-09-21T03:43:01.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.712+0000 7efc19ee7640 1 -- 172.21.15.38:0/1211757524 <== osd.1 v2:172.21.15.38:6816/1491818732 18 ==== osd_op_reply(117 100.00000000.inode [delete] v123'328 uv328 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7efc0c021070 con 0x5644c3e78b70 2024-09-21T03:43:01.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.712+0000 7efc2042df40 1 -- 172.21.15.38:0/1211757524 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:118 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5644c3e7cc00 con 0x5644c3e78b70 2024-09-21T03:43:01.715 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-09-21T03:43:01.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.713+0000 7efc19ee7640 1 -- 172.21.15.38:0/1211757524 <== osd.1 v2:172.21.15.38:6816/1491818732 19 ==== osd_op_reply(118 100.00000000.inode [create] v123'329 uv329 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7efc0c021070 con 0x5644c3e78b70 2024-09-21T03:43:01.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.713+0000 7efc2042df40 1 -- 172.21.15.38:0/1211757524 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:119 13.1f 13:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5644c3e7d2d0 con 0x5644c3e78b70 2024-09-21T03:43:01.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.714+0000 7efc19ee7640 1 -- 172.21.15.38:0/1211757524 <== osd.1 v2:172.21.15.38:6816/1491818732 20 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v123'330 uv330 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7efc0c021070 con 0x5644c3e78b70 2024-09-21T03:43:01.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.714+0000 7efc2042df40 1 -- 172.21.15.38:0/1211757524 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:120 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5644c3e7cc20 con 0x5644c3e78b70 2024-09-21T03:43:01.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.715+0000 7efc19ee7640 1 -- 172.21.15.38:0/1211757524 <== osd.1 v2:172.21.15.38:6816/1491818732 21 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v123'331 uv331 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7efc0c021070 con 0x5644c3e78b70 2024-09-21T03:43:01.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.715+0000 7efc2042df40 1 -- 172.21.15.38:0/1211757524 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:121 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5644c3e7d270 con 0x5644c3e78b70 2024-09-21T03:43:01.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.717+0000 7efc19ee7640 1 -- 172.21.15.38:0/1211757524 <== osd.1 v2:172.21.15.38:6816/1491818732 22 ==== osd_op_reply(121 1.00000000 [create] v123'332 uv325 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7efc0c021070 con 0x5644c3e78b70 2024-09-21T03:43:01.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.717+0000 7efc2042df40 1 -- 172.21.15.38:0/1211757524 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:122 13.1f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e123) -- 0x5644c3e7d230 con 0x5644c3e78b70 2024-09-21T03:43:01.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.718+0000 7efc19ee7640 1 -- 172.21.15.38:0/1211757524 <== osd.1 v2:172.21.15.38:6816/1491818732 23 ==== osd_op_reply(122 1.00000000 [delete] v123'333 uv333 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7efc0c021070 con 0x5644c3e78b70 2024-09-21T03:43:01.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.718+0000 7efc2042df40 1 -- 172.21.15.38:0/1211757524 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:123 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5644c3e7d1e0 con 0x5644c3e78b70 2024-09-21T03:43:01.720 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-09-21T03:43:01.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.719+0000 7efc19ee7640 1 -- 172.21.15.38:0/1211757524 <== osd.1 v2:172.21.15.38:6816/1491818732 24 ==== osd_op_reply(123 1.00000000 [create] v123'334 uv334 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7efc0c021070 con 0x5644c3e78b70 2024-09-21T03:43:01.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.719+0000 7efc2042df40 1 -- 172.21.15.38:0/1211757524 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:124 13.1f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5644c3e7d220 con 0x5644c3e78b70 2024-09-21T03:43:01.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.720+0000 7efc19ee7640 1 -- 172.21.15.38:0/1211757524 <== osd.1 v2:172.21.15.38:6816/1491818732 25 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v123'335 uv335 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7efc0c021070 con 0x5644c3e78b70 2024-09-21T03:43:01.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.720+0000 7efc2042df40 1 -- 172.21.15.38:0/1211757524 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:125 13.1f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5644c3e7e6c0 con 0x5644c3e78b70 2024-09-21T03:43:01.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.721+0000 7efc19ee7640 1 -- 172.21.15.38:0/1211757524 <== osd.1 v2:172.21.15.38:6816/1491818732 26 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v123'336 uv336 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7efc0c021070 con 0x5644c3e78b70 2024-09-21T03:43:01.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.721+0000 7efc2042df40 1 -- 172.21.15.38:0/1211757524 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:126 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5644c3e7e6c0 con 0x5644c3e78b70 2024-09-21T03:43:01.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.722+0000 7efc19ee7640 1 -- 172.21.15.38:0/1211757524 <== osd.1 v2:172.21.15.38:6816/1491818732 27 ==== osd_op_reply(126 1.00000000 [omap-set-header] v123'337 uv337 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7efc0c021070 con 0x5644c3e78b70 2024-09-21T03:43:01.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.722+0000 7efc2042df40 1 -- 172.21.15.38:0/1211757524 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:127 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=584b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5644c3e7d5d0 con 0x5644c3e78b70 2024-09-21T03:43:01.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.723+0000 7efc19ee7640 1 -- 172.21.15.38:0/1211757524 <== osd.1 v2:172.21.15.38:6816/1491818732 28 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v123'338 uv338 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7efc0c021070 con 0x5644c3e78b70 2024-09-21T03:43:01.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.723+0000 7efc2042df40 1 -- 172.21.15.38:0/1211757524 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x5644c3e70a90 msgr2=0x5644c3e72eb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:01.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.723+0000 7efc2042df40 1 --2- 172.21.15.38:0/1211757524 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x5644c3e70a90 0x5644c3e72eb0 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:01.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.723+0000 7efc2042df40 1 -- 172.21.15.38:0/1211757524 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x5644c3e78b70 msgr2=0x5644c3e7af90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:01.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.723+0000 7efc2042df40 1 --2- 172.21.15.38:0/1211757524 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x5644c3e78b70 0x5644c3e7af90 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:01.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.723+0000 7efc2042df40 1 -- 172.21.15.38:0/1211757524 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x5644c3e7c2b0 msgr2=0x5644c3e7c6c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:01.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.723+0000 7efc2042df40 1 --2- 172.21.15.38:0/1211757524 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x5644c3e7c2b0 0x5644c3e7c6c0 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:01.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.723+0000 7efc2042df40 1 -- 172.21.15.38:0/1211757524 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x5644c3e74b00 msgr2=0x5644c3e77940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:01.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.723+0000 7efc2042df40 1 --2- 172.21.15.38:0/1211757524 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x5644c3e74b00 0x5644c3e77940 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:01.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.723+0000 7efc2042df40 1 -- 172.21.15.38:0/1211757524 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x5644c3e6b150 msgr2=0x5644c3e6e210 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:01.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.723+0000 7efc2042df40 1 --2- 172.21.15.38:0/1211757524 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x5644c3e6b150 0x5644c3e6e210 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:01.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.723+0000 7efc1a6e8640 1 -- 172.21.15.38:0/1211757524 reap_dead start 2024-09-21T03:43:01.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.723+0000 7efc2042df40 1 -- 172.21.15.38:0/1211757524 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x5644c3e61790 msgr2=0x5644c3e69cc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:01.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.723+0000 7efc2042df40 1 --2- 172.21.15.38:0/1211757524 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x5644c3e61790 0x5644c3e69cc0 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:01.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.723+0000 7efc1a6e8640 1 -- 172.21.15.38:0/1211757524 reap_dead start 2024-09-21T03:43:01.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.723+0000 7efc1a6e8640 1 -- 172.21.15.38:0/1211757524 reap_dead start 2024-09-21T03:43:01.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.723+0000 7efc2042df40 1 -- 172.21.15.38:0/1211757524 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7efbfc0775a0 msgr2=0x7efbfc079ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:01.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.723+0000 7efc2042df40 1 --2- 172.21.15.38:0/1211757524 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7efbfc0775a0 0x7efbfc079ab0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7efc10055000 tx=0x7efc10003020 comp rx=0 tx=0).stop 2024-09-21T03:43:01.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.723+0000 7efc2042df40 1 -- 172.21.15.38:0/1211757524 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5644c3cf0d00 msgr2=0x5644c3e5ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:01.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.723+0000 7efc2042df40 1 --2- 172.21.15.38:0/1211757524 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5644c3cf0d00 0x5644c3e5ce70 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 crypto rx=0x7efc0c00ef60 tx=0x7efc0c00ef90 comp rx=0 tx=0).stop 2024-09-21T03:43:01.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.723+0000 7efc2042df40 1 -- 172.21.15.38:0/1211757524 shutdown_connections 2024-09-21T03:43:01.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.723+0000 7efc2042df40 1 --2- 172.21.15.38:0/1211757524 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x5644c3e61790 0x5644c3e69cc0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:01.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.724+0000 7efc2042df40 1 --2- 172.21.15.38:0/1211757524 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7efbfc0775a0 0x7efbfc079ab0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:01.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.724+0000 7efc2042df40 1 --2- 172.21.15.38:0/1211757524 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5644c3cf0d00 0x5644c3e5ce70 unknown :-1 s=CLOSED pgs=1901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:01.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.724+0000 7efc2042df40 1 -- 172.21.15.38:0/1211757524 >> 172.21.15.38:0/1211757524 conn(0x5644c3c0fac0 msgr2=0x5644c3dc2830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:01.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.724+0000 7efc2042df40 1 -- 172.21.15.38:0/1211757524 shutdown_connections 2024-09-21T03:43:01.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.724+0000 7efc2042df40 1 -- 172.21.15.38:0/1211757524 wait complete. 2024-09-21T03:43:01.729 DEBUG:teuthology.orchestra.run.smithi038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata truncate mds_snaptable 0 2024-09-21T03:43:01.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.823+0000 7ffa253eef40 1 Processor -- start 2024-09-21T03:43:01.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.823+0000 7ffa253eef40 1 -- start start 2024-09-21T03:43:01.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.824+0000 7ffa253eef40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f200cf4090 0x55f200d81e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:01.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.824+0000 7ffa253eef40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f200d69520 0x55f200cd6520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:01.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.824+0000 7ffa253eef40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f200d61ce0 0x55f200cf2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:01.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.824+0000 7ffa253eef40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55f200bfad60 con 0x55f200d61ce0 2024-09-21T03:43:01.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.824+0000 7ffa253eef40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55f200c59430 con 0x55f200cf4090 2024-09-21T03:43:01.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.824+0000 7ffa253eef40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55f200d82970 con 0x55f200d69520 2024-09-21T03:43:01.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.824+0000 7ffa1f6a9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f200d61ce0 0x55f200cf2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:01.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.824+0000 7ffa1f6a9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f200d61ce0 0x55f200cf2950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44102/0 (socket says 172.21.15.38:44102) 2024-09-21T03:43:01.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.824+0000 7ffa1f6a9640 1 -- 172.21.15.38:0/3663458268 learned_addr learned my addr 172.21.15.38:0/3663458268 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:43:01.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.824+0000 7ffa1eea8640 1 --2- 172.21.15.38:0/3663458268 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f200cf4090 0x55f200d81e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:01.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.824+0000 7ffa1e6a7640 1 --2- 172.21.15.38:0/3663458268 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f200d69520 0x55f200cd6520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:01.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.824+0000 7ffa1f6a9640 1 -- 172.21.15.38:0/3663458268 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f200d69520 msgr2=0x55f200cd6520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:01.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.824+0000 7ffa1f6a9640 1 --2- 172.21.15.38:0/3663458268 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f200d69520 0x55f200cd6520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:01.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.825+0000 7ffa1f6a9640 1 -- 172.21.15.38:0/3663458268 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f200cf4090 msgr2=0x55f200d81e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:01.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.825+0000 7ffa1f6a9640 1 --2- 172.21.15.38:0/3663458268 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f200cf4090 0x55f200d81e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:01.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.825+0000 7ffa1f6a9640 1 -- 172.21.15.38:0/3663458268 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f200cd6de0 con 0x55f200d61ce0 2024-09-21T03:43:01.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.825+0000 7ffa1f6a9640 1 --2- 172.21.15.38:0/3663458268 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f200d61ce0 0x55f200cf2950 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7ffa1800b7e0 tx=0x7ffa1800bca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b956020995ea3e8b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:01.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.825+0000 7ffa1d6a5640 1 -- 172.21.15.38:0/3663458268 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa1800db30 con 0x55f200d61ce0 2024-09-21T03:43:01.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.825+0000 7ffa1d6a5640 1 -- 172.21.15.38:0/3663458268 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffa1800ddc0 con 0x55f200d61ce0 2024-09-21T03:43:01.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.825+0000 7ffa1d6a5640 1 -- 172.21.15.38:0/3663458268 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa1800f690 con 0x55f200d61ce0 2024-09-21T03:43:01.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.825+0000 7ffa253eef40 1 -- 172.21.15.38:0/3663458268 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f200d61ce0 msgr2=0x55f200cf2950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:01.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.825+0000 7ffa253eef40 1 --2- 172.21.15.38:0/3663458268 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f200d61ce0 0x55f200cf2950 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7ffa1800b7e0 tx=0x7ffa1800bca0 comp rx=0 tx=0).stop 2024-09-21T03:43:01.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.825+0000 7ffa253eef40 1 -- 172.21.15.38:0/3663458268 shutdown_connections 2024-09-21T03:43:01.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.825+0000 7ffa253eef40 1 --2- 172.21.15.38:0/3663458268 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f200d61ce0 0x55f200cf2950 unknown :-1 s=CLOSED pgs=1904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:01.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.825+0000 7ffa253eef40 1 --2- 172.21.15.38:0/3663458268 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f200d69520 0x55f200cd6520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:01.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.825+0000 7ffa253eef40 1 --2- 172.21.15.38:0/3663458268 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f200cf4090 0x55f200d81e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:01.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.825+0000 7ffa253eef40 1 -- 172.21.15.38:0/3663458268 >> 172.21.15.38:0/3663458268 conn(0x55f200bcbac0 msgr2=0x55f200cad260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:01.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.826+0000 7ffa253eef40 1 -- 172.21.15.38:0/3663458268 shutdown_connections 2024-09-21T03:43:01.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.826+0000 7ffa253eef40 1 -- 172.21.15.38:0/3663458268 wait complete. 2024-09-21T03:43:01.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.826+0000 7ffa253eef40 1 Processor -- start 2024-09-21T03:43:01.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.826+0000 7ffa253eef40 1 -- start start 2024-09-21T03:43:01.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.828+0000 7ffa253eef40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f200cf4090 0x55f200d7a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:01.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.828+0000 7ffa253eef40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f200d69520 0x55f200d64530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:01.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.828+0000 7ffa253eef40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f200e0cc90 0x55f200d61180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:01.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.828+0000 7ffa253eef40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55f200cd7060 con 0x55f200d69520 2024-09-21T03:43:01.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.828+0000 7ffa253eef40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55f200d7d820 con 0x55f200e0cc90 2024-09-21T03:43:01.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.828+0000 7ffa253eef40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55f200d64a70 con 0x55f200cf4090 2024-09-21T03:43:01.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.828+0000 7ffa1e6a7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f200d69520 0x55f200d64530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:01.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.828+0000 7ffa1e6a7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f200d69520 0x55f200d64530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44118/0 (socket says 172.21.15.38:44118) 2024-09-21T03:43:01.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.828+0000 7ffa1e6a7640 1 -- 172.21.15.38:0/3020578464 learned_addr learned my addr 172.21.15.38:0/3020578464 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:43:01.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.828+0000 7ffa1eea8640 1 --2- 172.21.15.38:0/3020578464 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f200cf4090 0x55f200d7a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:01.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.828+0000 7ffa1f6a9640 1 --2- 172.21.15.38:0/3020578464 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f200e0cc90 0x55f200d61180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:01.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.828+0000 7ffa1e6a7640 1 -- 172.21.15.38:0/3020578464 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f200cf4090 msgr2=0x55f200d7a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:01.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.828+0000 7ffa1e6a7640 1 --2- 172.21.15.38:0/3020578464 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f200cf4090 0x55f200d7a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:01.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.828+0000 7ffa1e6a7640 1 -- 172.21.15.38:0/3020578464 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f200e0cc90 msgr2=0x55f200d61180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:01.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.828+0000 7ffa1e6a7640 1 --2- 172.21.15.38:0/3020578464 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f200e0cc90 0x55f200d61180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:01.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.828+0000 7ffa1e6a7640 1 -- 172.21.15.38:0/3020578464 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f200d7acb0 con 0x55f200d69520 2024-09-21T03:43:01.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.828+0000 7ffa1e6a7640 1 --2- 172.21.15.38:0/3020578464 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f200d69520 0x55f200d64530 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 crypto rx=0x7ffa14002e80 tx=0x7ffa1400b730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:01.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.828+0000 7ffa07fff640 1 -- 172.21.15.38:0/3020578464 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa14013070 con 0x55f200d69520 2024-09-21T03:43:01.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.829+0000 7ffa07fff640 1 -- 172.21.15.38:0/3020578464 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffa14006070 con 0x55f200d69520 2024-09-21T03:43:01.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.829+0000 7ffa253eef40 1 -- 172.21.15.38:0/3020578464 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55f200d7af90 con 0x55f200d69520 2024-09-21T03:43:01.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.829+0000 7ffa07fff640 1 -- 172.21.15.38:0/3020578464 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa1400fb80 con 0x55f200d69520 2024-09-21T03:43:01.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.829+0000 7ffa253eef40 1 -- 172.21.15.38:0/3020578464 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f200d7b270 con 0x55f200d69520 2024-09-21T03:43:01.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.830+0000 7ffa07fff640 1 -- 172.21.15.38:0/3020578464 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ffa14018460 con 0x55f200d69520 2024-09-21T03:43:01.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.831+0000 7ffa07fff640 1 --2- 172.21.15.38:0/3020578464 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ffa0c077570 0x7ffa0c079a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:01.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.831+0000 7ffa07fff640 1 -- 172.21.15.38:0/3020578464 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 8626+0+0 (secure 0 0 0) 0x7ffa1409a9e0 con 0x55f200d69520 2024-09-21T03:43:01.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.831+0000 7ffa1eea8640 1 --2- 172.21.15.38:0/3020578464 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ffa0c077570 0x7ffa0c079a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:01.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.831+0000 7ffa253eef40 1 --2- 172.21.15.38:0/3020578464 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x55f200cd8f50 0x55f200d83010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:01.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.831+0000 7ffa253eef40 1 -- 172.21.15.38:0/3020578464 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:1 13.d 13:b50e409b:::mds_snaptable:head [truncate 0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55f200e1c9b0 con 0x55f200cd8f50 2024-09-21T03:43:01.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.831+0000 7ffa1f6a9640 1 --2- 172.21.15.38:0/3020578464 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x55f200cd8f50 0x55f200d83010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:01.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.832+0000 7ffa1eea8640 1 --2- 172.21.15.38:0/3020578464 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ffa0c077570 0x7ffa0c079a80 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7ffa10055000 tx=0x7ffa10003020 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:01.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.832+0000 7ffa1f6a9640 1 --2- 172.21.15.38:0/3020578464 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x55f200cd8f50 0x55f200d83010 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:01.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.833+0000 7ffa1f6a9640 1 -- 172.21.15.38:0/3020578464 <== osd.0 v2:172.21.15.38:6800/2381639433 1 ==== osd_op_reply(1 mds_snaptable [truncate 0] v123'137 uv137 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7ffa1800b040 con 0x55f200cd8f50 2024-09-21T03:43:01.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.833+0000 7ffa253eef40 1 -- 172.21.15.38:0/3020578464 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x55f200cd8f50 msgr2=0x55f200d83010 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:01.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.833+0000 7ffa253eef40 1 --2- 172.21.15.38:0/3020578464 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x55f200cd8f50 0x55f200d83010 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:01.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.833+0000 7ffa253eef40 1 -- 172.21.15.38:0/3020578464 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ffa0c077570 msgr2=0x7ffa0c079a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:01.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.833+0000 7ffa253eef40 1 --2- 172.21.15.38:0/3020578464 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ffa0c077570 0x7ffa0c079a80 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7ffa10055000 tx=0x7ffa10003020 comp rx=0 tx=0).stop 2024-09-21T03:43:01.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.833+0000 7ffa253eef40 1 -- 172.21.15.38:0/3020578464 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f200d69520 msgr2=0x55f200d64530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:01.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.833+0000 7ffa253eef40 1 --2- 172.21.15.38:0/3020578464 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f200d69520 0x55f200d64530 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 crypto rx=0x7ffa14002e80 tx=0x7ffa1400b730 comp rx=0 tx=0).stop 2024-09-21T03:43:01.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.833+0000 7ffa1f6a9640 1 -- 172.21.15.38:0/3020578464 reap_dead start 2024-09-21T03:43:01.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.833+0000 7ffa253eef40 1 -- 172.21.15.38:0/3020578464 shutdown_connections 2024-09-21T03:43:01.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.833+0000 7ffa253eef40 1 -- 172.21.15.38:0/3020578464 >> 172.21.15.38:0/3020578464 conn(0x55f200bcbac0 msgr2=0x55f200cf2ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:01.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.833+0000 7ffa253eef40 1 -- 172.21.15.38:0/3020578464 shutdown_connections 2024-09-21T03:43:01.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.833+0000 7ffa253eef40 1 -- 172.21.15.38:0/3020578464 wait complete. 2024-09-21T03:43:01.838 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-21T03:43:01.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.955+0000 7f15907a1640 1 -- 172.21.15.38:0/3561744653 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f15880d1120 msgr2=0x7f15880ceb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:01.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.955+0000 7f15907a1640 1 --2- 172.21.15.38:0/3561744653 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f15880d1120 0x7f15880ceb90 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7f15880d2360 tx=0x7f158002a9a0 comp rx=0 tx=0).stop 2024-09-21T03:43:01.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.955+0000 7f15907a1640 1 -- 172.21.15.38:0/3561744653 shutdown_connections 2024-09-21T03:43:01.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.955+0000 7f15907a1640 1 --2- 172.21.15.38:0/3561744653 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f15880d1120 0x7f15880ceb90 unknown :-1 s=CLOSED pgs=1906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:01.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.955+0000 7f15907a1640 1 --2- 172.21.15.38:0/3561744653 >> v2:172.21.15.195:3300/0 conn(0x7f15880d0170 0x7f15880d0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:01.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.955+0000 7f15907a1640 1 -- 172.21.15.38:0/3561744653 >> 172.21.15.38:0/3561744653 conn(0x7f158809d830 msgr2=0x7f15880b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:01.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.955+0000 7f15907a1640 1 -- 172.21.15.38:0/3561744653 shutdown_connections 2024-09-21T03:43:01.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.955+0000 7f15907a1640 1 -- 172.21.15.38:0/3561744653 wait complete. 2024-09-21T03:43:01.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.957+0000 7f15907a1640 1 Processor -- start 2024-09-21T03:43:01.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.957+0000 7f15907a1640 1 -- start start 2024-09-21T03:43:01.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.959+0000 7f15907a1640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f15880d0170 0x7f15881570e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:01.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.959+0000 7f15907a1640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f15880d1120 0x7f15880b21e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:01.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.959+0000 7f15907a1640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f158815ee80 0x7f15880b2720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:01.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.959+0000 7f15907a1640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f15880b6700 con 0x7f15880d0170 2024-09-21T03:43:01.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.959+0000 7f15907a1640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f15880b6580 con 0x7f158815ee80 2024-09-21T03:43:01.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.959+0000 7f15907a1640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f15880b6880 con 0x7f15880d1120 2024-09-21T03:43:01.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.959+0000 7f158e53d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f15880d0170 0x7f15881570e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:01.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.959+0000 7f158e53d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f15880d0170 0x7f15881570e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44124/0 (socket says 172.21.15.38:44124) 2024-09-21T03:43:01.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.959+0000 7f158e53d640 1 -- 172.21.15.38:0/3425861723 learned_addr learned my addr 172.21.15.38:0/3425861723 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:43:01.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.959+0000 7f158dd3c640 1 --2- 172.21.15.38:0/3425861723 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f15880d1120 0x7f15880b21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:01.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.959+0000 7f158ed3e640 1 --2- 172.21.15.38:0/3425861723 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f158815ee80 0x7f15880b2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:01.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.959+0000 7f158e53d640 1 -- 172.21.15.38:0/3425861723 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f15880d1120 msgr2=0x7f15880b21e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:01.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.959+0000 7f158e53d640 1 --2- 172.21.15.38:0/3425861723 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f15880d1120 0x7f15880b21e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:01.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.959+0000 7f158e53d640 1 -- 172.21.15.38:0/3425861723 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f158815ee80 msgr2=0x7f15880b2720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:01.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.959+0000 7f158e53d640 1 --2- 172.21.15.38:0/3425861723 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f158815ee80 0x7f15880b2720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:01.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.959+0000 7f158e53d640 1 -- 172.21.15.38:0/3425861723 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f158806e4f0 con 0x7f15880d0170 2024-09-21T03:43:01.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.959+0000 7f158e53d640 1 --2- 172.21.15.38:0/3425861723 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f15880d0170 0x7f15881570e0 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x7f157c00c3a0 tx=0x7f157c00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:01.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.959+0000 7f157affd640 1 -- 172.21.15.38:0/3425861723 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f157c017070 con 0x7f15880d0170 2024-09-21T03:43:01.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.960+0000 7f157affd640 1 -- 172.21.15.38:0/3425861723 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f157c0053c0 con 0x7f15880d0170 2024-09-21T03:43:01.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.960+0000 7f15907a1640 1 -- 172.21.15.38:0/3425861723 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f158815a1f0 con 0x7f15880d0170 2024-09-21T03:43:01.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.960+0000 7f157affd640 1 -- 172.21.15.38:0/3425861723 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f157c003850 con 0x7f15880d0170 2024-09-21T03:43:01.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.960+0000 7f15907a1640 1 -- 172.21.15.38:0/3425861723 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f158806e790 con 0x7f15880d0170 2024-09-21T03:43:01.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.961+0000 7f15907a1640 1 -- 172.21.15.38:0/3425861723 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15880b6da0 con 0x7f15880d0170 2024-09-21T03:43:01.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.962+0000 7f157affd640 1 -- 172.21.15.38:0/3425861723 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f157c01a070 con 0x7f15880d0170 2024-09-21T03:43:01.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.962+0000 7f157affd640 1 --2- 172.21.15.38:0/3425861723 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f156c078bc0 0x7f156c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:01.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.962+0000 7f157affd640 1 -- 172.21.15.38:0/3425861723 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 8626+0+0 (secure 0 0 0) 0x7f157c09abc0 con 0x7f15880d0170 2024-09-21T03:43:01.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.962+0000 7f158dd3c640 1 --2- 172.21.15.38:0/3425861723 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f156c078bc0 0x7f156c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:01.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.963+0000 7f158dd3c640 1 --2- 172.21.15.38:0/3425861723 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f156c078bc0 0x7f156c07b0d0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f15880cee40 tx=0x7f1584005f30 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:01.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:01.967+0000 7f157affd640 1 -- 172.21.15.38:0/3425861723 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f15881592d0 con 0x7f15880d0170 2024-09-21T03:43:02.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:02.195+0000 7f15907a1640 1 -- 172.21.15.38:0/3425861723 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f15881583a0 con 0x7f15880d0170 2024-09-21T03:43:02.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:02.547+0000 7f157affd640 1 -- 172.21.15.38:0/3425861723 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v249) ==== 162+0+0 (secure 0 0 0) 0x7f157c061d70 con 0x7f15880d0170 2024-09-21T03:43:02.549 INFO:teuthology.orchestra.run.smithi038.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-21T03:43:02.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:02.550+0000 7f15907a1640 1 -- 172.21.15.38:0/3425861723 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f156c078bc0 msgr2=0x7f156c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:02.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:02.550+0000 7f15907a1640 1 --2- 172.21.15.38:0/3425861723 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f156c078bc0 0x7f156c07b0d0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f15880cee40 tx=0x7f1584005f30 comp rx=0 tx=0).stop 2024-09-21T03:43:02.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:02.550+0000 7f15907a1640 1 -- 172.21.15.38:0/3425861723 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f15880d0170 msgr2=0x7f15881570e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:02.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:02.550+0000 7f15907a1640 1 --2- 172.21.15.38:0/3425861723 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f15880d0170 0x7f15881570e0 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x7f157c00c3a0 tx=0x7f157c00c860 comp rx=0 tx=0).stop 2024-09-21T03:43:02.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:02.550+0000 7f15907a1640 1 -- 172.21.15.38:0/3425861723 shutdown_connections 2024-09-21T03:43:02.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:02.550+0000 7f15907a1640 1 --2- 172.21.15.38:0/3425861723 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f158815ee80 0x7f15880b2720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:02.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:02.550+0000 7f15907a1640 1 --2- 172.21.15.38:0/3425861723 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f156c078bc0 0x7f156c07b0d0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:02.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:02.550+0000 7f15907a1640 1 --2- 172.21.15.38:0/3425861723 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f15880d1120 0x7f15880b21e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:02.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:02.550+0000 7f15907a1640 1 --2- 172.21.15.38:0/3425861723 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f15880d0170 0x7f15881570e0 unknown :-1 s=CLOSED pgs=1907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:02.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:02.550+0000 7f15907a1640 1 -- 172.21.15.38:0/3425861723 >> 172.21.15.38:0/3425861723 conn(0x7f158809d830 msgr2=0x7f15880b5740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:02.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:02.550+0000 7f15907a1640 1 -- 172.21.15.38:0/3425861723 shutdown_connections 2024-09-21T03:43:02.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:02.550+0000 7f15907a1640 1 -- 172.21.15.38:0/3425861723 wait complete. 2024-09-21T03:43:02.560 INFO:tasks.ceph.mds.c.smithi038.stderr:2024-09-21T03:43:02.558+0000 7f130f33a640 -1 log_channel(cluster) log [ERR] : error decoding table object 'mds_snaptable': End of buffer [buffer:2] 2024-09-21T03:43:02.561 INFO:tasks.ceph.mon.a.smithi038.stderr:2024-09-21T03:43:02.559+0000 7f3f45713640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T03:43:02.561 INFO:tasks.ceph.mon.a.smithi038.stderr:2024-09-21T03:43:02.559+0000 7f3f45713640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-09-21T03:43:02.566 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:43:02.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:02.681+0000 7f2ab2532640 1 -- 172.21.15.38:0/2916011012 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2aac0b2e40 msgr2=0x7f2aac0b3250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:02.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:02.681+0000 7f2ab2532640 1 --2- 172.21.15.38:0/2916011012 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2aac0b2e40 0x7f2aac0b3250 secure :-1 s=READY pgs=1908 cs=0 l=1 rev1=1 crypto rx=0x7f2aa4005fd0 tx=0x7f2aa402aa90 comp rx=0 tx=0).stop 2024-09-21T03:43:02.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:02.681+0000 7f2ab2532640 1 -- 172.21.15.38:0/2916011012 shutdown_connections 2024-09-21T03:43:02.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:02.681+0000 7f2ab2532640 1 --2- 172.21.15.38:0/2916011012 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2aac0b2e40 0x7f2aac0b3250 unknown :-1 s=CLOSED pgs=1908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:02.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:02.681+0000 7f2ab2532640 1 --2- 172.21.15.38:0/2916011012 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2aac162280 0x7f2aac162690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:02.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:02.681+0000 7f2ab2532640 1 -- 172.21.15.38:0/2916011012 >> 172.21.15.38:0/2916011012 conn(0x7f2aac0b6b20 msgr2=0x7f2aac0d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:02.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:02.681+0000 7f2ab2532640 1 -- 172.21.15.38:0/2916011012 shutdown_connections 2024-09-21T03:43:02.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:02.681+0000 7f2ab2532640 1 -- 172.21.15.38:0/2916011012 wait complete. 2024-09-21T03:43:02.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:02.683+0000 7f2ab2532640 1 Processor -- start 2024-09-21T03:43:02.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:02.683+0000 7f2ab2532640 1 -- start start 2024-09-21T03:43:02.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:02.685+0000 7f2ab2532640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2aac0b2e40 0x7f2aac0ceee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:02.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:02.685+0000 7f2ab2532640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2aac162280 0x7f2aac0aba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:02.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:02.685+0000 7f2ab2532640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2aac0abf80 0x7f2aac0ac390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:02.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:02.685+0000 7f2ab2532640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f2aac15e4b0 con 0x7f2aac162280 2024-09-21T03:43:02.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:02.685+0000 7f2ab2532640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f2aac0b39a0 con 0x7f2aac0abf80 2024-09-21T03:43:02.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:02.685+0000 7f2ab2532640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f2aac15e630 con 0x7f2aac0b2e40 2024-09-21T03:43:02.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:02.685+0000 7f2aab7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2aac162280 0x7f2aac0aba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:02.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:02.685+0000 7f2aab7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2aac162280 0x7f2aac0aba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44832/0 (socket says 172.21.15.38:44832) 2024-09-21T03:43:02.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:02.685+0000 7f2aab7fe640 1 -- 172.21.15.38:0/1482776652 learned_addr learned my addr 172.21.15.38:0/1482776652 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:43:02.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:02.685+0000 7f2aabfff640 1 --2- 172.21.15.38:0/1482776652 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2aac0b2e40 0x7f2aac0ceee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:02.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:02.685+0000 7f2ab0acf640 1 --2- 172.21.15.38:0/1482776652 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2aac0abf80 0x7f2aac0ac390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:02.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:02.685+0000 7f2aab7fe640 1 -- 172.21.15.38:0/1482776652 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2aac0b2e40 msgr2=0x7f2aac0ceee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:02.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:02.685+0000 7f2aab7fe640 1 --2- 172.21.15.38:0/1482776652 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2aac0b2e40 0x7f2aac0ceee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:02.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:02.685+0000 7f2aab7fe640 1 -- 172.21.15.38:0/1482776652 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2aac0abf80 msgr2=0x7f2aac0ac390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:02.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:02.685+0000 7f2aab7fe640 1 --2- 172.21.15.38:0/1482776652 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2aac0abf80 0x7f2aac0ac390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:02.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:02.685+0000 7f2aab7fe640 1 -- 172.21.15.38:0/1482776652 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2aac0b29a0 con 0x7f2aac162280 2024-09-21T03:43:02.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:02.685+0000 7f2aab7fe640 1 --2- 172.21.15.38:0/1482776652 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2aac162280 0x7f2aac0aba40 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 crypto rx=0x7f2aa0009fd0 tx=0x7f2aa000ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:02.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:02.685+0000 7f2aa8ff9640 1 -- 172.21.15.38:0/1482776652 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aa000ccc0 con 0x7f2aac162280 2024-09-21T03:43:02.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:02.685+0000 7f2aa8ff9640 1 -- 172.21.15.38:0/1482776652 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2aa000ee70 con 0x7f2aac162280 2024-09-21T03:43:02.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:02.685+0000 7f2ab2532640 1 -- 172.21.15.38:0/1482776652 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2aac0aaf60 con 0x7f2aac162280 2024-09-21T03:43:02.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:02.685+0000 7f2aa8ff9640 1 -- 172.21.15.38:0/1482776652 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aa000ce60 con 0x7f2aac162280 2024-09-21T03:43:02.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:02.686+0000 7f2ab2532640 1 -- 172.21.15.38:0/1482776652 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2aac0ab520 con 0x7f2aac162280 2024-09-21T03:43:02.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:02.687+0000 7f2ab2532640 1 -- 172.21.15.38:0/1482776652 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2aac162f00 con 0x7f2aac162280 2024-09-21T03:43:02.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:02.687+0000 7f2aa8ff9640 1 -- 172.21.15.38:0/1482776652 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f2aa0004280 con 0x7f2aac162280 2024-09-21T03:43:02.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:02.688+0000 7f2aa8ff9640 1 --2- 172.21.15.38:0/1482776652 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2a98078bc0 0x7f2a9807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:02.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:02.688+0000 7f2aa8ff9640 1 -- 172.21.15.38:0/1482776652 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 8712+0+0 (secure 0 0 0) 0x7f2aa009c160 con 0x7f2aac162280 2024-09-21T03:43:02.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:02.688+0000 7f2aabfff640 1 --2- 172.21.15.38:0/1482776652 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2a98078bc0 0x7f2a9807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:02.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:02.689+0000 7f2aabfff640 1 --2- 172.21.15.38:0/1482776652 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2a98078bc0 0x7f2a9807b0d0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f2a9c001720 tx=0x7f2a9c009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:02.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:02.692+0000 7f2aa8ff9640 1 -- 172.21.15.38:0/1482776652 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f2aa00632f0 con 0x7f2aac162280 2024-09-21T03:43:02.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:02.915+0000 7f2ab2532640 1 -- 172.21.15.38:0/1482776652 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2aac06ebe0 con 0x7f2aac162280 2024-09-21T03:43:02.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:02.916+0000 7f2aa8ff9640 1 -- 172.21.15.38:0/1482776652 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 251 v251) ==== 95+0+4392 (secure 0 0 0) 0x7f2aa0067d20 con 0x7f2aac162280 2024-09-21T03:43:02.918 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:43:02.918 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":251,"btime":"2024-09-21T03:43:02:560471+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6086,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/965972663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":965972663},{"type":"v1","addr":"172.21.15.195:6836","nonce":965972663}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6700,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/3176122334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":3176122334},{"type":"v1","addr":"172.21.15.195:6837","nonce":3176122334}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":9495,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/2097789246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":2097789246},{"type":"v1","addr":"172.21.15.38:6835","nonce":2097789246}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":244}],"filesystems":[{"mdsmap":{"epoch":251,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:40:02.319282+0000","modified":"2024-09-21T03:43:02.560379+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":124,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-21T03:43:02.918 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 251 2024-09-21T03:43:02.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:02.918+0000 7f2ab2532640 1 -- 172.21.15.38:0/1482776652 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2a98078bc0 msgr2=0x7f2a9807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:02.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:02.918+0000 7f2ab2532640 1 --2- 172.21.15.38:0/1482776652 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2a98078bc0 0x7f2a9807b0d0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f2a9c001720 tx=0x7f2a9c009040 comp rx=0 tx=0).stop 2024-09-21T03:43:02.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:02.918+0000 7f2ab2532640 1 -- 172.21.15.38:0/1482776652 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2aac162280 msgr2=0x7f2aac0aba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:02.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:02.918+0000 7f2ab2532640 1 --2- 172.21.15.38:0/1482776652 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2aac162280 0x7f2aac0aba40 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 crypto rx=0x7f2aa0009fd0 tx=0x7f2aa000ec20 comp rx=0 tx=0).stop 2024-09-21T03:43:02.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:02.918+0000 7f2ab2532640 1 -- 172.21.15.38:0/1482776652 shutdown_connections 2024-09-21T03:43:02.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:02.918+0000 7f2ab2532640 1 --2- 172.21.15.38:0/1482776652 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2aac0abf80 0x7f2aac0ac390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:02.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:02.918+0000 7f2ab2532640 1 --2- 172.21.15.38:0/1482776652 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2aac162280 0x7f2aac0aba40 unknown :-1 s=CLOSED pgs=1909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:02.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:02.918+0000 7f2ab2532640 1 --2- 172.21.15.38:0/1482776652 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2a98078bc0 0x7f2a9807b0d0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:02.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:02.918+0000 7f2ab2532640 1 --2- 172.21.15.38:0/1482776652 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2aac0b2e40 0x7f2aac0ceee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:02.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:02.918+0000 7f2ab2532640 1 -- 172.21.15.38:0/1482776652 >> 172.21.15.38:0/1482776652 conn(0x7f2aac0b6b20 msgr2=0x7f2aac15d9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:02.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:02.918+0000 7f2ab2532640 1 -- 172.21.15.38:0/1482776652 shutdown_connections 2024-09-21T03:43:02.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:02.918+0000 7f2ab2532640 1 -- 172.21.15.38:0/1482776652 wait complete. 2024-09-21T03:43:02.931 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-21T03:43:02.931 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:43:03.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:03.046+0000 7fc4aed52640 1 -- 172.21.15.38:0/230375942 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc4a8162590 msgr2=0x7fc4a81629a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:03.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:03.046+0000 7fc4aed52640 1 --2- 172.21.15.38:0/230375942 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc4a8162590 0x7fc4a81629a0 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7fc494009830 tx=0x7fc49402a510 comp rx=0 tx=0).stop 2024-09-21T03:43:03.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:03.046+0000 7fc4aed52640 1 -- 172.21.15.38:0/230375942 shutdown_connections 2024-09-21T03:43:03.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:03.046+0000 7fc4aed52640 1 --2- 172.21.15.38:0/230375942 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc4a8162590 0x7fc4a81629a0 unknown :-1 s=CLOSED pgs=1911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:03.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:03.046+0000 7fc4aed52640 1 --2- 172.21.15.38:0/230375942 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc4a80cfe00 0x7fc4a80d2a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:03.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:03.046+0000 7fc4aed52640 1 -- 172.21.15.38:0/230375942 >> 172.21.15.38:0/230375942 conn(0x7fc4a80d4440 msgr2=0x7fc4a80d4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:03.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:03.046+0000 7fc4aed52640 1 -- 172.21.15.38:0/230375942 shutdown_connections 2024-09-21T03:43:03.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:03.046+0000 7fc4aed52640 1 -- 172.21.15.38:0/230375942 wait complete. 2024-09-21T03:43:03.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:03.048+0000 7fc4aed52640 1 Processor -- start 2024-09-21T03:43:03.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:03.048+0000 7fc4aed52640 1 -- start start 2024-09-21T03:43:03.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:03.050+0000 7fc4aed52640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc4a80b2e60 0x7fc4a81ea730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:03.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:03.050+0000 7fc4aed52640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc4a80cfe00 0x7fc4a80c7080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:03.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:03.050+0000 7fc4aed52640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc4a8162590 0x7fc4a81595d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:03.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:03.050+0000 7fc4aed52640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fc4a80b3b00 con 0x7fc4a80b2e60 2024-09-21T03:43:03.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:03.050+0000 7fc4aed52640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fc4a80b3980 con 0x7fc4a8162590 2024-09-21T03:43:03.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:03.050+0000 7fc4aed52640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fc4a80b3f60 con 0x7fc4a80cfe00 2024-09-21T03:43:03.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:03.050+0000 7fc4acaee640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc4a80b2e60 0x7fc4a81ea730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:03.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:03.050+0000 7fc4acaee640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc4a80b2e60 0x7fc4a81ea730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44846/0 (socket says 172.21.15.38:44846) 2024-09-21T03:43:03.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:03.050+0000 7fc4acaee640 1 -- 172.21.15.38:0/2416539662 learned_addr learned my addr 172.21.15.38:0/2416539662 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:43:03.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:03.050+0000 7fc4a7fff640 1 --2- 172.21.15.38:0/2416539662 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc4a80cfe00 0x7fc4a80c7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:03.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:03.050+0000 7fc4ad2ef640 1 --2- 172.21.15.38:0/2416539662 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc4a8162590 0x7fc4a81595d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:03.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:03.050+0000 7fc4acaee640 1 -- 172.21.15.38:0/2416539662 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc4a80cfe00 msgr2=0x7fc4a80c7080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:03.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:03.050+0000 7fc4acaee640 1 --2- 172.21.15.38:0/2416539662 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc4a80cfe00 0x7fc4a80c7080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:03.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:03.050+0000 7fc4acaee640 1 -- 172.21.15.38:0/2416539662 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc4a8162590 msgr2=0x7fc4a81595d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:03.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:03.050+0000 7fc4acaee640 1 --2- 172.21.15.38:0/2416539662 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc4a8162590 0x7fc4a81595d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:03.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:03.050+0000 7fc4acaee640 1 -- 172.21.15.38:0/2416539662 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4a81e9690 con 0x7fc4a80b2e60 2024-09-21T03:43:03.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:03.050+0000 7fc4ad2ef640 1 --2- 172.21.15.38:0/2416539662 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc4a8162590 0x7fc4a81595d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:43:03.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:03.050+0000 7fc4acaee640 1 --2- 172.21.15.38:0/2416539662 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc4a80b2e60 0x7fc4a81ea730 secure :-1 s=READY pgs=1912 cs=0 l=1 rev1=1 crypto rx=0x7fc49c00d770 tx=0x7fc49c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:03.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:03.050+0000 7fc4a57fa640 1 -- 172.21.15.38:0/2416539662 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc49c017070 con 0x7fc4a80b2e60 2024-09-21T03:43:03.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:03.050+0000 7fc4a57fa640 1 -- 172.21.15.38:0/2416539662 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc49c009d20 con 0x7fc4a80b2e60 2024-09-21T03:43:03.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:03.050+0000 7fc4aed52640 1 -- 172.21.15.38:0/2416539662 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4a81e9970 con 0x7fc4a80b2e60 2024-09-21T03:43:03.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:03.050+0000 7fc4a57fa640 1 -- 172.21.15.38:0/2416539662 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc49c0106c0 con 0x7fc4a80b2e60 2024-09-21T03:43:03.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:03.050+0000 7fc4aed52640 1 -- 172.21.15.38:0/2416539662 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4a81e9f30 con 0x7fc4a80b2e60 2024-09-21T03:43:03.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:03.051+0000 7fc4aed52640 1 -- 172.21.15.38:0/2416539662 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4a80d2a20 con 0x7fc4a80b2e60 2024-09-21T03:43:03.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:03.052+0000 7fc4a57fa640 1 -- 172.21.15.38:0/2416539662 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc49c01a070 con 0x7fc4a80b2e60 2024-09-21T03:43:03.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:03.053+0000 7fc4a57fa640 1 --2- 172.21.15.38:0/2416539662 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc48c078b90 0x7fc48c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:03.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:03.053+0000 7fc4a57fa640 1 -- 172.21.15.38:0/2416539662 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 8712+0+0 (secure 0 0 0) 0x7fc49c09bab0 con 0x7fc4a80b2e60 2024-09-21T03:43:03.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:03.053+0000 7fc4a7fff640 1 --2- 172.21.15.38:0/2416539662 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc48c078b90 0x7fc48c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:03.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:03.054+0000 7fc4a7fff640 1 --2- 172.21.15.38:0/2416539662 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc48c078b90 0x7fc48c07b0a0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fc494034000 tx=0x7fc494004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:03.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:03.057+0000 7fc4a57fa640 1 -- 172.21.15.38:0/2416539662 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fc49c0632f0 con 0x7fc4a80b2e60 2024-09-21T03:43:03.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:03.281+0000 7fc4aed52640 1 -- 172.21.15.38:0/2416539662 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc4a81eac70 con 0x7fc4a80b2e60 2024-09-21T03:43:03.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:03.282+0000 7fc4a57fa640 1 -- 172.21.15.38:0/2416539662 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 251 v251) ==== 95+0+4392 (secure 0 0 0) 0x7fc49c0850e0 con 0x7fc4a80b2e60 2024-09-21T03:43:03.284 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:43:03.284 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":251,"btime":"2024-09-21T03:43:02:560471+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6086,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/965972663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":965972663},{"type":"v1","addr":"172.21.15.195:6836","nonce":965972663}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6700,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/3176122334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":3176122334},{"type":"v1","addr":"172.21.15.195:6837","nonce":3176122334}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":9495,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/2097789246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":2097789246},{"type":"v1","addr":"172.21.15.38:6835","nonce":2097789246}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":244}],"filesystems":[{"mdsmap":{"epoch":251,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:40:02.319282+0000","modified":"2024-09-21T03:43:02.560379+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":124,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-21T03:43:03.284 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 251 2024-09-21T03:43:03.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:03.284+0000 7fc4aed52640 1 -- 172.21.15.38:0/2416539662 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc48c078b90 msgr2=0x7fc48c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:03.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:03.284+0000 7fc4aed52640 1 --2- 172.21.15.38:0/2416539662 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc48c078b90 0x7fc48c07b0a0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fc494034000 tx=0x7fc494004730 comp rx=0 tx=0).stop 2024-09-21T03:43:03.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:03.284+0000 7fc4aed52640 1 -- 172.21.15.38:0/2416539662 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc4a80b2e60 msgr2=0x7fc4a81ea730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:03.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:03.284+0000 7fc4aed52640 1 --2- 172.21.15.38:0/2416539662 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc4a80b2e60 0x7fc4a81ea730 secure :-1 s=READY pgs=1912 cs=0 l=1 rev1=1 crypto rx=0x7fc49c00d770 tx=0x7fc49c00dc30 comp rx=0 tx=0).stop 2024-09-21T03:43:03.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:03.284+0000 7fc4aed52640 1 -- 172.21.15.38:0/2416539662 shutdown_connections 2024-09-21T03:43:03.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:03.284+0000 7fc4aed52640 1 --2- 172.21.15.38:0/2416539662 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc4a8162590 0x7fc4a81595d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:03.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:03.284+0000 7fc4aed52640 1 --2- 172.21.15.38:0/2416539662 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc48c078b90 0x7fc48c07b0a0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:03.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:03.284+0000 7fc4aed52640 1 --2- 172.21.15.38:0/2416539662 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc4a80cfe00 0x7fc4a80c7080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:03.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:03.284+0000 7fc4aed52640 1 --2- 172.21.15.38:0/2416539662 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc4a80b2e60 0x7fc4a81ea730 unknown :-1 s=CLOSED pgs=1912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:03.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:03.284+0000 7fc4aed52640 1 -- 172.21.15.38:0/2416539662 >> 172.21.15.38:0/2416539662 conn(0x7fc4a80d4440 msgr2=0x7fc4a8160100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:03.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:03.284+0000 7fc4aed52640 1 -- 172.21.15.38:0/2416539662 shutdown_connections 2024-09-21T03:43:03.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:03.284+0000 7fc4aed52640 1 -- 172.21.15.38:0/2416539662 wait complete. 2024-09-21T03:43:03.297 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-21T03:43:03.298 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:43:03.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:03.412+0000 7efe41cbd640 1 -- 172.21.15.38:0/340430104 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efe3c162590 msgr2=0x7efe3c1629a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:03.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:03.412+0000 7efe41cbd640 1 --2- 172.21.15.38:0/340430104 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efe3c162590 0x7efe3c1629a0 secure :-1 s=READY pgs=1914 cs=0 l=1 rev1=1 crypto rx=0x7efe30009830 tx=0x7efe3002a530 comp rx=0 tx=0).stop 2024-09-21T03:43:03.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:03.412+0000 7efe41cbd640 1 -- 172.21.15.38:0/340430104 shutdown_connections 2024-09-21T03:43:03.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:03.412+0000 7efe41cbd640 1 --2- 172.21.15.38:0/340430104 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efe3c162590 0x7efe3c1629a0 unknown :-1 s=CLOSED pgs=1914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:03.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:03.412+0000 7efe41cbd640 1 --2- 172.21.15.38:0/340430104 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efe3c0cfe00 0x7efe3c0d2a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:03.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:03.412+0000 7efe41cbd640 1 -- 172.21.15.38:0/340430104 >> 172.21.15.38:0/340430104 conn(0x7efe3c0d4440 msgr2=0x7efe3c0d4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:03.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:03.412+0000 7efe41cbd640 1 -- 172.21.15.38:0/340430104 shutdown_connections 2024-09-21T03:43:03.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:03.412+0000 7efe41cbd640 1 -- 172.21.15.38:0/340430104 wait complete. 2024-09-21T03:43:03.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:03.414+0000 7efe41cbd640 1 Processor -- start 2024-09-21T03:43:03.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:03.414+0000 7efe41cbd640 1 -- start start 2024-09-21T03:43:03.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:03.415+0000 7efe41cbd640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efe3c0b2e60 0x7efe3c155680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:03.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:03.415+0000 7efe41cbd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efe3c0cfe00 0x7efe3c150810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:03.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:03.415+0000 7efe41cbd640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efe3c162590 0x7efe3c151ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:03.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:03.415+0000 7efe41cbd640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7efe3c0b3b00 con 0x7efe3c0cfe00 2024-09-21T03:43:03.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:03.415+0000 7efe41cbd640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7efe3c0b3980 con 0x7efe3c0b2e60 2024-09-21T03:43:03.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:03.415+0000 7efe41cbd640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7efe3c0b3f60 con 0x7efe3c162590 2024-09-21T03:43:03.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:03.415+0000 7efe3affd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efe3c0cfe00 0x7efe3c150810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:03.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:03.415+0000 7efe3affd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efe3c0cfe00 0x7efe3c150810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44852/0 (socket says 172.21.15.38:44852) 2024-09-21T03:43:03.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:03.415+0000 7efe3affd640 1 -- 172.21.15.38:0/3425106462 learned_addr learned my addr 172.21.15.38:0/3425106462 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:43:03.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:03.416+0000 7efe3b7fe640 1 --2- 172.21.15.38:0/3425106462 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efe3c0b2e60 0x7efe3c155680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:03.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:03.416+0000 7efe3bfff640 1 --2- 172.21.15.38:0/3425106462 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efe3c162590 0x7efe3c151ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:03.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:03.416+0000 7efe3affd640 1 -- 172.21.15.38:0/3425106462 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efe3c162590 msgr2=0x7efe3c151ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:03.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:03.416+0000 7efe3affd640 1 --2- 172.21.15.38:0/3425106462 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efe3c162590 0x7efe3c151ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:03.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:03.416+0000 7efe3affd640 1 -- 172.21.15.38:0/3425106462 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efe3c0b2e60 msgr2=0x7efe3c155680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:03.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:03.416+0000 7efe3affd640 1 --2- 172.21.15.38:0/3425106462 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efe3c0b2e60 0x7efe3c155680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:03.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:03.416+0000 7efe3affd640 1 -- 172.21.15.38:0/3425106462 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe3c156140 con 0x7efe3c0cfe00 2024-09-21T03:43:03.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:03.416+0000 7efe3affd640 1 --2- 172.21.15.38:0/3425106462 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efe3c0cfe00 0x7efe3c150810 secure :-1 s=READY pgs=1915 cs=0 l=1 rev1=1 crypto rx=0x7efe30009950 tx=0x7efe30033ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:03.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:03.416+0000 7efe23fff640 1 -- 172.21.15.38:0/3425106462 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe30031d60 con 0x7efe3c0cfe00 2024-09-21T03:43:03.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:03.416+0000 7efe23fff640 1 -- 172.21.15.38:0/3425106462 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efe30005020 con 0x7efe3c0cfe00 2024-09-21T03:43:03.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:03.416+0000 7efe41cbd640 1 -- 172.21.15.38:0/3425106462 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe3c156420 con 0x7efe3c0cfe00 2024-09-21T03:43:03.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:03.416+0000 7efe23fff640 1 -- 172.21.15.38:0/3425106462 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe30005450 con 0x7efe3c0cfe00 2024-09-21T03:43:03.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:03.416+0000 7efe41cbd640 1 -- 172.21.15.38:0/3425106462 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe3c1513a0 con 0x7efe3c0cfe00 2024-09-21T03:43:03.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:03.417+0000 7efe41cbd640 1 -- 172.21.15.38:0/3425106462 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe3c0d0390 con 0x7efe3c0cfe00 2024-09-21T03:43:03.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:03.418+0000 7efe23fff640 1 -- 172.21.15.38:0/3425106462 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7efe300047a0 con 0x7efe3c0cfe00 2024-09-21T03:43:03.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:03.419+0000 7efe23fff640 1 --2- 172.21.15.38:0/3425106462 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7efe28078b90 0x7efe2807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:03.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:03.419+0000 7efe23fff640 1 -- 172.21.15.38:0/3425106462 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 8712+0+0 (secure 0 0 0) 0x7efe300bd050 con 0x7efe3c0cfe00 2024-09-21T03:43:03.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:03.419+0000 7efe3b7fe640 1 --2- 172.21.15.38:0/3425106462 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7efe28078b90 0x7efe2807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:03.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:03.420+0000 7efe3b7fe640 1 --2- 172.21.15.38:0/3425106462 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7efe28078b90 0x7efe2807b0a0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7efe2c006fd0 tx=0x7efe2c008040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:03.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:03.422+0000 7efe23fff640 1 -- 172.21.15.38:0/3425106462 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7efe3c0d0390 con 0x7efe3c0cfe00 2024-09-21T03:43:03.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:03.654+0000 7efe41cbd640 1 -- 172.21.15.38:0/3425106462 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efe3c162380 con 0x7efe3c0cfe00 2024-09-21T03:43:03.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:03.655+0000 7efe23fff640 1 -- 172.21.15.38:0/3425106462 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 251 v251) ==== 95+0+4392 (secure 0 0 0) 0x7efe3c162380 con 0x7efe3c0cfe00 2024-09-21T03:43:03.657 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:43:03.657 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":251,"btime":"2024-09-21T03:43:02:560471+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6086,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/965972663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":965972663},{"type":"v1","addr":"172.21.15.195:6836","nonce":965972663}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6700,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/3176122334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":3176122334},{"type":"v1","addr":"172.21.15.195:6837","nonce":3176122334}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":9495,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/2097789246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":2097789246},{"type":"v1","addr":"172.21.15.38:6835","nonce":2097789246}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":244}],"filesystems":[{"mdsmap":{"epoch":251,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:40:02.319282+0000","modified":"2024-09-21T03:43:02.560379+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":124,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-21T03:43:03.657 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 251 2024-09-21T03:43:03.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:03.657+0000 7efe41cbd640 1 -- 172.21.15.38:0/3425106462 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7efe28078b90 msgr2=0x7efe2807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:03.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:03.657+0000 7efe41cbd640 1 --2- 172.21.15.38:0/3425106462 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7efe28078b90 0x7efe2807b0a0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7efe2c006fd0 tx=0x7efe2c008040 comp rx=0 tx=0).stop 2024-09-21T03:43:03.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:03.657+0000 7efe41cbd640 1 -- 172.21.15.38:0/3425106462 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efe3c0cfe00 msgr2=0x7efe3c150810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:03.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:03.657+0000 7efe41cbd640 1 --2- 172.21.15.38:0/3425106462 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efe3c0cfe00 0x7efe3c150810 secure :-1 s=READY pgs=1915 cs=0 l=1 rev1=1 crypto rx=0x7efe30009950 tx=0x7efe30033ce0 comp rx=0 tx=0).stop 2024-09-21T03:43:03.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:03.657+0000 7efe41cbd640 1 -- 172.21.15.38:0/3425106462 shutdown_connections 2024-09-21T03:43:03.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:03.657+0000 7efe41cbd640 1 --2- 172.21.15.38:0/3425106462 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7efe28078b90 0x7efe2807b0a0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:03.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:03.657+0000 7efe41cbd640 1 --2- 172.21.15.38:0/3425106462 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efe3c162590 0x7efe3c151ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:03.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:03.657+0000 7efe41cbd640 1 --2- 172.21.15.38:0/3425106462 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efe3c0cfe00 0x7efe3c150810 unknown :-1 s=CLOSED pgs=1915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:03.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:03.657+0000 7efe41cbd640 1 --2- 172.21.15.38:0/3425106462 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efe3c0b2e60 0x7efe3c155680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:03.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:03.657+0000 7efe41cbd640 1 -- 172.21.15.38:0/3425106462 >> 172.21.15.38:0/3425106462 conn(0x7efe3c0d4440 msgr2=0x7efe3c0d4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:03.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:03.657+0000 7efe41cbd640 1 -- 172.21.15.38:0/3425106462 shutdown_connections 2024-09-21T03:43:03.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:03.657+0000 7efe41cbd640 1 -- 172.21.15.38:0/3425106462 wait complete. 2024-09-21T03:43:03.670 INFO:tasks.cephfs.test_damage:Result: Mutation 'Truncate mds_snaptable' led to DAMAGED state 2024-09-21T03:43:03.670 INFO:tasks.cephfs.test_damage:Applying mutation 'Truncate mds0_inotable' 2024-09-21T03:43:03.671 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-21T03:43:03.671 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:43:03.686 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056795,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2354 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T03:43:03.686 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi038.front.sepia.ceph.com 2024-09-21T03:43:03.686 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:43:03.687 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T03:43:03.742 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T03:43:03.742 INFO:teuthology.orchestra.run.smithi038.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-09-21T03:43:03.742 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-21T03:43:03.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:03.898+0000 7f6d7fd52640 1 -- 172.21.15.38:0/3879599018 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6d780ceb90 msgr2=0x7f6d780cefa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:03.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:03.898+0000 7f6d7fd52640 1 --2- 172.21.15.38:0/3879599018 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6d780ceb90 0x7f6d780cefa0 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x7f6d70005fd0 tx=0x7f6d7002a960 comp rx=0 tx=0).stop 2024-09-21T03:43:03.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:03.898+0000 7f6d7fd52640 1 -- 172.21.15.38:0/3879599018 shutdown_connections 2024-09-21T03:43:03.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:03.898+0000 7f6d7fd52640 1 --2- 172.21.15.38:0/3879599018 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6d780ceb90 0x7f6d780cefa0 unknown :-1 s=CLOSED pgs=1916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:03.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:03.898+0000 7f6d7fd52640 1 --2- 172.21.15.38:0/3879599018 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6d7815ee80 0x7f6d780d0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:03.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:03.898+0000 7f6d7fd52640 1 -- 172.21.15.38:0/3879599018 >> 172.21.15.38:0/3879599018 conn(0x7f6d7809d830 msgr2=0x7f6d780b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:03.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:03.898+0000 7f6d7fd52640 1 -- 172.21.15.38:0/3879599018 shutdown_connections 2024-09-21T03:43:03.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:03.898+0000 7f6d7fd52640 1 -- 172.21.15.38:0/3879599018 wait complete. 2024-09-21T03:43:03.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:03.900+0000 7f6d7fd52640 1 Processor -- start 2024-09-21T03:43:03.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:03.900+0000 7f6d7fd52640 1 -- start start 2024-09-21T03:43:03.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:03.902+0000 7f6d7fd52640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6d780b9ad0 0x7f6d781f9c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:03.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:03.902+0000 7f6d7fd52640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6d780ceb90 0x7f6d780a1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:03.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:03.902+0000 7f6d7fd52640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6d7815ee80 0x7f6d781579c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:03.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:03.902+0000 7f6d7fd52640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f6d780b6700 con 0x7f6d7815ee80 2024-09-21T03:43:03.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:03.902+0000 7f6d7fd52640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f6d780b6580 con 0x7f6d780b9ad0 2024-09-21T03:43:03.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:03.902+0000 7f6d7fd52640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f6d780b6880 con 0x7f6d780ceb90 2024-09-21T03:43:03.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:03.902+0000 7f6d7e2ef640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6d7815ee80 0x7f6d781579c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:03.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:03.902+0000 7f6d7e2ef640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6d7815ee80 0x7f6d781579c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44884/0 (socket says 172.21.15.38:44884) 2024-09-21T03:43:03.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:03.902+0000 7f6d7e2ef640 1 -- 172.21.15.38:0/1538905149 learned_addr learned my addr 172.21.15.38:0/1538905149 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:43:03.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:03.902+0000 7f6d7daee640 1 --2- 172.21.15.38:0/1538905149 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6d780b9ad0 0x7f6d781f9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:03.906 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:03.902+0000 7f6d7d2ed640 1 --2- 172.21.15.38:0/1538905149 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6d780ceb90 0x7f6d780a1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:03.906 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:03.902+0000 7f6d7e2ef640 1 -- 172.21.15.38:0/1538905149 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6d780ceb90 msgr2=0x7f6d780a1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:03.906 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:03.902+0000 7f6d7e2ef640 1 --2- 172.21.15.38:0/1538905149 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6d780ceb90 0x7f6d780a1bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:03.906 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:03.902+0000 7f6d7e2ef640 1 -- 172.21.15.38:0/1538905149 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6d780b9ad0 msgr2=0x7f6d781f9c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:03.906 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:03.902+0000 7f6d7e2ef640 1 --2- 172.21.15.38:0/1538905149 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6d780b9ad0 0x7f6d781f9c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:03.906 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:03.902+0000 7f6d7e2ef640 1 -- 172.21.15.38:0/1538905149 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d78155420 con 0x7f6d7815ee80 2024-09-21T03:43:03.906 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:03.902+0000 7f6d7e2ef640 1 --2- 172.21.15.38:0/1538905149 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6d7815ee80 0x7f6d781579c0 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7f6d70009d90 tx=0x7f6d70003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:03.906 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:03.903+0000 7f6d767fc640 1 -- 172.21.15.38:0/1538905149 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d70034040 con 0x7f6d7815ee80 2024-09-21T03:43:03.906 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:03.903+0000 7f6d767fc640 1 -- 172.21.15.38:0/1538905149 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6d70004560 con 0x7f6d7815ee80 2024-09-21T03:43:03.906 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:03.903+0000 7f6d7fd52640 1 -- 172.21.15.38:0/1538905149 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d78155920 con 0x7f6d7815ee80 2024-09-21T03:43:03.906 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:03.903+0000 7f6d767fc640 1 -- 172.21.15.38:0/1538905149 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d70045480 con 0x7f6d7815ee80 2024-09-21T03:43:03.906 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:03.903+0000 7f6d7fd52640 1 -- 172.21.15.38:0/1538905149 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d78155ee0 con 0x7f6d7815ee80 2024-09-21T03:43:03.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:03.904+0000 7f6d7fd52640 1 -- 172.21.15.38:0/1538905149 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d780b6de0 con 0x7f6d7815ee80 2024-09-21T03:43:03.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:03.905+0000 7f6d767fc640 1 -- 172.21.15.38:0/1538905149 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f6d70037020 con 0x7f6d7815ee80 2024-09-21T03:43:03.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:03.905+0000 7f6d767fc640 1 --2- 172.21.15.38:0/1538905149 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6d64078bc0 0x7f6d6407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:03.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:03.905+0000 7f6d767fc640 1 -- 172.21.15.38:0/1538905149 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 8712+0+0 (secure 0 0 0) 0x7f6d700bce60 con 0x7f6d7815ee80 2024-09-21T03:43:03.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:03.906+0000 7f6d7daee640 1 --2- 172.21.15.38:0/1538905149 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6d64078bc0 0x7f6d6407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:03.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:03.906+0000 7f6d7daee640 1 --2- 172.21.15.38:0/1538905149 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6d64078bc0 0x7f6d6407b0d0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f6d680058c0 tx=0x7f6d68005f60 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:03.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:03.910+0000 7f6d767fc640 1 -- 172.21.15.38:0/1538905149 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f6d781592d0 con 0x7f6d7815ee80 2024-09-21T03:43:04.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:04.136+0000 7f6d7fd52640 1 -- 172.21.15.38:0/1538905149 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f6d7806e040 con 0x7f6d7815ee80 2024-09-21T03:43:04.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:04.569+0000 7f6d767fc640 1 -- 172.21.15.38:0/1538905149 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v252) ==== 161+0+0 (secure 0 0 0) 0x7f6d700840d0 con 0x7f6d7815ee80 2024-09-21T03:43:04.572 INFO:teuthology.orchestra.run.smithi038.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-21T03:43:04.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:04.572+0000 7f6d7fd52640 1 -- 172.21.15.38:0/1538905149 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6d64078bc0 msgr2=0x7f6d6407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:04.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:04.572+0000 7f6d7fd52640 1 --2- 172.21.15.38:0/1538905149 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6d64078bc0 0x7f6d6407b0d0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f6d680058c0 tx=0x7f6d68005f60 comp rx=0 tx=0).stop 2024-09-21T03:43:04.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:04.572+0000 7f6d7fd52640 1 -- 172.21.15.38:0/1538905149 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6d7815ee80 msgr2=0x7f6d781579c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:04.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:04.572+0000 7f6d7fd52640 1 --2- 172.21.15.38:0/1538905149 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6d7815ee80 0x7f6d781579c0 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7f6d70009d90 tx=0x7f6d70003f90 comp rx=0 tx=0).stop 2024-09-21T03:43:04.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:04.572+0000 7f6d7fd52640 1 -- 172.21.15.38:0/1538905149 shutdown_connections 2024-09-21T03:43:04.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:04.572+0000 7f6d7fd52640 1 --2- 172.21.15.38:0/1538905149 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6d7815ee80 0x7f6d781579c0 unknown :-1 s=CLOSED pgs=1917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:04.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:04.572+0000 7f6d7fd52640 1 --2- 172.21.15.38:0/1538905149 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6d64078bc0 0x7f6d6407b0d0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:04.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:04.572+0000 7f6d7fd52640 1 --2- 172.21.15.38:0/1538905149 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6d780ceb90 0x7f6d780a1bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:04.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:04.572+0000 7f6d7fd52640 1 --2- 172.21.15.38:0/1538905149 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6d780b9ad0 0x7f6d781f9c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:04.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:04.572+0000 7f6d7fd52640 1 -- 172.21.15.38:0/1538905149 >> 172.21.15.38:0/1538905149 conn(0x7f6d7809d830 msgr2=0x7f6d7815ffb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:04.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:04.572+0000 7f6d7fd52640 1 -- 172.21.15.38:0/1538905149 shutdown_connections 2024-09-21T03:43:04.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:04.572+0000 7f6d7fd52640 1 -- 172.21.15.38:0/1538905149 wait complete. 2024-09-21T03:43:04.586 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-09-21T03:43:04.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:04.704+0000 7f0b476d5640 1 -- 172.21.15.38:0/2997229201 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0b2c004800 msgr2=0x7f0b2c004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:04.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:04.704+0000 7f0b476d5640 1 --2- 172.21.15.38:0/2997229201 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0b2c004800 0x7f0b2c004c10 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 crypto rx=0x7f0b3000c3c0 tx=0x7f0b2c005830 comp rx=0 tx=0).stop 2024-09-21T03:43:04.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:04.704+0000 7f0b45c72640 1 -- 172.21.15.38:0/2997229201 reap_dead start 2024-09-21T03:43:04.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:04.704+0000 7f0b476d5640 1 -- 172.21.15.38:0/2997229201 shutdown_connections 2024-09-21T03:43:04.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:04.704+0000 7f0b476d5640 1 -- 172.21.15.38:0/2997229201 >> 172.21.15.38:0/2997229201 conn(0x7f0b4009d830 msgr2=0x7f0b400b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:04.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:04.704+0000 7f0b476d5640 1 -- 172.21.15.38:0/2997229201 shutdown_connections 2024-09-21T03:43:04.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:04.704+0000 7f0b476d5640 1 -- 172.21.15.38:0/2997229201 wait complete. 2024-09-21T03:43:04.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:04.706+0000 7f0b476d5640 1 Processor -- start 2024-09-21T03:43:04.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:04.706+0000 7f0b476d5640 1 -- start start 2024-09-21T03:43:04.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:04.707+0000 7f0b476d5640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0b2c004800 0x7f0b401573a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:04.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:04.707+0000 7f0b476d5640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0b4006e5e0 0x7f0b4006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:04.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:04.707+0000 7f0b476d5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0b400b2db0 0x7f0b400b31c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:04.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:04.707+0000 7f0b476d5640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f0b400d0500 con 0x7f0b400b2db0 2024-09-21T03:43:04.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:04.707+0000 7f0b476d5640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f0b400d0380 con 0x7f0b2c004800 2024-09-21T03:43:04.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:04.707+0000 7f0b476d5640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f0b400ba0d0 con 0x7f0b4006e5e0 2024-09-21T03:43:04.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:04.708+0000 7f0b45c72640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0b400b2db0 0x7f0b400b31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:04.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:04.708+0000 7f0b45c72640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0b400b2db0 0x7f0b400b31c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44908/0 (socket says 172.21.15.38:44908) 2024-09-21T03:43:04.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:04.708+0000 7f0b45c72640 1 -- 172.21.15.38:0/52144879 learned_addr learned my addr 172.21.15.38:0/52144879 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:43:04.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:04.708+0000 7f0b44c70640 1 --2- 172.21.15.38:0/52144879 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0b4006e5e0 0x7f0b4006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:04.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:04.708+0000 7f0b45471640 1 --2- 172.21.15.38:0/52144879 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0b2c004800 0x7f0b401573a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:04.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:04.708+0000 7f0b45c72640 1 -- 172.21.15.38:0/52144879 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0b4006e5e0 msgr2=0x7f0b4006e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:04.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:04.708+0000 7f0b45c72640 1 --2- 172.21.15.38:0/52144879 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0b4006e5e0 0x7f0b4006e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:04.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:04.708+0000 7f0b45c72640 1 -- 172.21.15.38:0/52144879 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0b2c004800 msgr2=0x7f0b401573a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:04.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:04.708+0000 7f0b45c72640 1 --2- 172.21.15.38:0/52144879 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0b2c004800 0x7f0b401573a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:04.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:04.708+0000 7f0b45c72640 1 -- 172.21.15.38:0/52144879 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b400b3700 con 0x7f0b400b2db0 2024-09-21T03:43:04.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:04.708+0000 7f0b45c72640 1 --2- 172.21.15.38:0/52144879 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0b400b2db0 0x7f0b400b31c0 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7f0b400d06d0 tx=0x7f0b38006430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:04.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:04.708+0000 7f0b35ffb640 1 -- 172.21.15.38:0/52144879 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b38011900 con 0x7f0b400b2db0 2024-09-21T03:43:04.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:04.708+0000 7f0b35ffb640 1 -- 172.21.15.38:0/52144879 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0b38011aa0 con 0x7f0b400b2db0 2024-09-21T03:43:04.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:04.708+0000 7f0b35ffb640 1 -- 172.21.15.38:0/52144879 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b38010420 con 0x7f0b400b2db0 2024-09-21T03:43:04.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:04.708+0000 7f0b476d5640 1 -- 172.21.15.38:0/52144879 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b400cb6b0 con 0x7f0b400b2db0 2024-09-21T03:43:04.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:04.709+0000 7f0b476d5640 1 -- 172.21.15.38:0/52144879 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b400cb950 con 0x7f0b400b2db0 2024-09-21T03:43:04.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:04.710+0000 7f0b476d5640 1 -- 172.21.15.38:0/52144879 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b400d0be0 con 0x7f0b400b2db0 2024-09-21T03:43:04.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:04.710+0000 7f0b35ffb640 1 -- 172.21.15.38:0/52144879 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f0b380105c0 con 0x7f0b400b2db0 2024-09-21T03:43:04.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:04.711+0000 7f0b35ffb640 1 --2- 172.21.15.38:0/52144879 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0b24078b90 0x7f0b2407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:04.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:04.711+0000 7f0b35ffb640 1 -- 172.21.15.38:0/52144879 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 8712+0+0 (secure 0 0 0) 0x7f0b3809c080 con 0x7f0b400b2db0 2024-09-21T03:43:04.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:04.711+0000 7f0b45471640 1 --2- 172.21.15.38:0/52144879 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0b24078b90 0x7f0b2407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:04.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:04.712+0000 7f0b45471640 1 --2- 172.21.15.38:0/52144879 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0b24078b90 0x7f0b2407b0a0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f0b400cfba0 tx=0x7f0b3c005f90 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:04.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:04.715+0000 7f0b35ffb640 1 -- 172.21.15.38:0/52144879 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f0b400b3700 con 0x7f0b400b2db0 2024-09-21T03:43:04.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:04.982+0000 7f0b476d5640 1 -- 172.21.15.38:0/52144879 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7f0b4006eff0 con 0x7f0b400b2db0 2024-09-21T03:43:05.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.576+0000 7f0b35ffb640 1 -- 172.21.15.38:0/52144879 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v253) ==== 101+0+0 (secure 0 0 0) 0x7f0b38063270 con 0x7f0b400b2db0 2024-09-21T03:43:05.579 INFO:teuthology.orchestra.run.smithi038.stderr:repaired: restoring rank 6:0 2024-09-21T03:43:05.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.580+0000 7f0b476d5640 1 -- 172.21.15.38:0/52144879 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0b24078b90 msgr2=0x7f0b2407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:05.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.580+0000 7f0b476d5640 1 --2- 172.21.15.38:0/52144879 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0b24078b90 0x7f0b2407b0a0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f0b400cfba0 tx=0x7f0b3c005f90 comp rx=0 tx=0).stop 2024-09-21T03:43:05.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.580+0000 7f0b476d5640 1 -- 172.21.15.38:0/52144879 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0b400b2db0 msgr2=0x7f0b400b31c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:05.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.580+0000 7f0b476d5640 1 --2- 172.21.15.38:0/52144879 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0b400b2db0 0x7f0b400b31c0 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7f0b400d06d0 tx=0x7f0b38006430 comp rx=0 tx=0).stop 2024-09-21T03:43:05.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.580+0000 7f0b476d5640 1 -- 172.21.15.38:0/52144879 shutdown_connections 2024-09-21T03:43:05.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.580+0000 7f0b476d5640 1 --2- 172.21.15.38:0/52144879 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0b400b2db0 0x7f0b400b31c0 unknown :-1 s=CLOSED pgs=1920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:05.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.580+0000 7f0b476d5640 1 --2- 172.21.15.38:0/52144879 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0b24078b90 0x7f0b2407b0a0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:05.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.580+0000 7f0b476d5640 1 --2- 172.21.15.38:0/52144879 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0b4006e5e0 0x7f0b4006e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:05.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.580+0000 7f0b476d5640 1 --2- 172.21.15.38:0/52144879 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0b2c004800 0x7f0b401573a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:05.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.580+0000 7f0b476d5640 1 -- 172.21.15.38:0/52144879 >> 172.21.15.38:0/52144879 conn(0x7f0b4009d830 msgr2=0x7f0b40158960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:05.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.580+0000 7f0b476d5640 1 -- 172.21.15.38:0/52144879 shutdown_connections 2024-09-21T03:43:05.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.580+0000 7f0b476d5640 1 -- 172.21.15.38:0/52144879 wait complete. 2024-09-21T03:43:05.596 DEBUG:teuthology.orchestra.run.smithi038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-09-21T03:43:05.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.654+0000 7fd35b7dff40 1 Processor -- start 2024-09-21T03:43:05.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.654+0000 7fd35b7dff40 1 -- start start 2024-09-21T03:43:05.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.655+0000 7fd35b7dff40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55afc32a5090 0x55afc3286670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:05.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.655+0000 7fd35b7dff40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55afc331a520 0x55afc325dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:05.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.655+0000 7fd35b7dff40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55afc3287080 0x55afc3287490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:05.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.655+0000 7fd35b7dff40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55afc320a430 con 0x55afc331a520 2024-09-21T03:43:05.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.655+0000 7fd35b7dff40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55afc32171f0 con 0x55afc3287080 2024-09-21T03:43:05.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.655+0000 7fd35b7dff40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55afc31abd60 con 0x55afc32a5090 2024-09-21T03:43:05.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.655+0000 7fd354a98640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55afc331a520 0x55afc325dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:05.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.655+0000 7fd355a9a640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55afc3287080 0x55afc3287490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:05.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.655+0000 7fd354a98640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55afc331a520 0x55afc325dcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44934/0 (socket says 172.21.15.38:44934) 2024-09-21T03:43:05.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.655+0000 7fd354a98640 1 -- 172.21.15.38:0/3042040906 learned_addr learned my addr 172.21.15.38:0/3042040906 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:43:05.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.655+0000 7fd355299640 1 --2- 172.21.15.38:0/3042040906 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55afc32a5090 0x55afc3286670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:05.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.656+0000 7fd354a98640 1 -- 172.21.15.38:0/3042040906 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55afc32a5090 msgr2=0x55afc3286670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:05.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.656+0000 7fd354a98640 1 --2- 172.21.15.38:0/3042040906 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55afc32a5090 0x55afc3286670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:05.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.656+0000 7fd354a98640 1 -- 172.21.15.38:0/3042040906 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55afc3287080 msgr2=0x55afc3287490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:05.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.656+0000 7fd354a98640 1 --2- 172.21.15.38:0/3042040906 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55afc3287080 0x55afc3287490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:05.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.656+0000 7fd354a98640 1 -- 172.21.15.38:0/3042040906 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55afc332f060 con 0x55afc331a520 2024-09-21T03:43:05.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.656+0000 7fd354a98640 1 --2- 172.21.15.38:0/3042040906 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55afc331a520 0x55afc325dcc0 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 crypto rx=0x7fd34c001db0 tx=0x7fd34c00b5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67804f1913925aa3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:05.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.656+0000 7fd3477fe640 1 -- 172.21.15.38:0/3042040906 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd34c00bde0 con 0x55afc331a520 2024-09-21T03:43:05.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.656+0000 7fd3477fe640 1 -- 172.21.15.38:0/3042040906 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd34c00f400 con 0x55afc331a520 2024-09-21T03:43:05.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.656+0000 7fd3477fe640 1 -- 172.21.15.38:0/3042040906 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd34c00f850 con 0x55afc331a520 2024-09-21T03:43:05.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.657+0000 7fd35b7dff40 1 -- 172.21.15.38:0/3042040906 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55afc331a520 msgr2=0x55afc325dcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:05.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.657+0000 7fd35b7dff40 1 --2- 172.21.15.38:0/3042040906 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55afc331a520 0x55afc325dcc0 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 crypto rx=0x7fd34c001db0 tx=0x7fd34c00b5b0 comp rx=0 tx=0).stop 2024-09-21T03:43:05.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.657+0000 7fd35b7dff40 1 -- 172.21.15.38:0/3042040906 shutdown_connections 2024-09-21T03:43:05.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.657+0000 7fd35b7dff40 1 --2- 172.21.15.38:0/3042040906 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55afc3287080 0x55afc3287490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:05.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.657+0000 7fd35b7dff40 1 --2- 172.21.15.38:0/3042040906 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55afc331a520 0x55afc325dcc0 unknown :-1 s=CLOSED pgs=1923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:05.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.657+0000 7fd35b7dff40 1 --2- 172.21.15.38:0/3042040906 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55afc32a5090 0x55afc3286670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:05.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.657+0000 7fd35b7dff40 1 -- 172.21.15.38:0/3042040906 >> 172.21.15.38:0/3042040906 conn(0x55afc317cac0 msgr2=0x55afc332adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:05.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.657+0000 7fd35b7dff40 1 -- 172.21.15.38:0/3042040906 shutdown_connections 2024-09-21T03:43:05.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.657+0000 7fd35b7dff40 1 -- 172.21.15.38:0/3042040906 wait complete. 2024-09-21T03:43:05.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.658+0000 7fd35b7dff40 1 Processor -- start 2024-09-21T03:43:05.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.658+0000 7fd35b7dff40 1 -- start start 2024-09-21T03:43:05.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.659+0000 7fd35b7dff40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55afc3287080 0x55afc33295f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:05.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.659+0000 7fd35b7dff40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55afc32a5090 0x55afc3317b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:05.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.659+0000 7fd35b7dff40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55afc331a520 0x55afc33180b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:05.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.659+0000 7fd35b7dff40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55afc333bb60 con 0x55afc32a5090 2024-09-21T03:43:05.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.659+0000 7fd35b7dff40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55afc332afd0 con 0x55afc331a520 2024-09-21T03:43:05.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.659+0000 7fd35b7dff40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55afc33c9230 con 0x55afc3287080 2024-09-21T03:43:05.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.659+0000 7fd354a98640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55afc32a5090 0x55afc3317b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:05.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.659+0000 7fd354a98640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55afc32a5090 0x55afc3317b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44944/0 (socket says 172.21.15.38:44944) 2024-09-21T03:43:05.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.659+0000 7fd354a98640 1 -- 172.21.15.38:0/4050539643 learned_addr learned my addr 172.21.15.38:0/4050539643 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:43:05.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.659+0000 7fd355a9a640 1 --2- 172.21.15.38:0/4050539643 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55afc331a520 0x55afc33180b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:05.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.659+0000 7fd355299640 1 --2- 172.21.15.38:0/4050539643 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55afc3287080 0x55afc33295f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:05.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.659+0000 7fd354a98640 1 -- 172.21.15.38:0/4050539643 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55afc3287080 msgr2=0x55afc33295f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:05.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.659+0000 7fd354a98640 1 --2- 172.21.15.38:0/4050539643 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55afc3287080 0x55afc33295f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:05.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.659+0000 7fd354a98640 1 -- 172.21.15.38:0/4050539643 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55afc331a520 msgr2=0x55afc33180b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:05.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.659+0000 7fd354a98640 1 --2- 172.21.15.38:0/4050539643 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55afc331a520 0x55afc33180b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:05.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.660+0000 7fd354a98640 1 -- 172.21.15.38:0/4050539643 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55afc33c8eb0 con 0x55afc32a5090 2024-09-21T03:43:05.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.660+0000 7fd355a9a640 1 --2- 172.21.15.38:0/4050539643 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55afc331a520 0x55afc33180b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:43:05.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.660+0000 7fd354a98640 1 --2- 172.21.15.38:0/4050539643 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55afc32a5090 0x55afc3317b70 secure :-1 s=READY pgs=1924 cs=0 l=1 rev1=1 crypto rx=0x7fd34c000f60 tx=0x7fd34c00fdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:05.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.660+0000 7fd3467fc640 1 -- 172.21.15.38:0/4050539643 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd34c01b430 con 0x55afc32a5090 2024-09-21T03:43:05.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.660+0000 7fd3467fc640 1 -- 172.21.15.38:0/4050539643 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd34c020020 con 0x55afc32a5090 2024-09-21T03:43:05.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.660+0000 7fd35b7dff40 1 -- 172.21.15.38:0/4050539643 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55afc3287b90 con 0x55afc32a5090 2024-09-21T03:43:05.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.660+0000 7fd3467fc640 1 -- 172.21.15.38:0/4050539643 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd34c01b740 con 0x55afc32a5090 2024-09-21T03:43:05.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.660+0000 7fd35b7dff40 1 -- 172.21.15.38:0/4050539643 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55afc332dc30 con 0x55afc32a5090 2024-09-21T03:43:05.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.662+0000 7fd3467fc640 1 -- 172.21.15.38:0/4050539643 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fd34c017070 con 0x55afc32a5090 2024-09-21T03:43:05.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.662+0000 7fd3467fc640 1 --2- 172.21.15.38:0/4050539643 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd338077570 0x7fd338079a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:05.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.662+0000 7fd3467fc640 1 -- 172.21.15.38:0/4050539643 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 8712+0+0 (secure 0 0 0) 0x7fd34c09b750 con 0x55afc32a5090 2024-09-21T03:43:05.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.663+0000 7fd355299640 1 --2- 172.21.15.38:0/4050539643 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd338077570 0x7fd338079a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:05.667 INFO:teuthology.orchestra.run.smithi038.stdout:Importing pool 2024-09-21T03:43:05.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.663+0000 7fd35b7dff40 1 --2- 172.21.15.38:0/4050539643 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x55afc332e170 0x55afc332a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:05.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.663+0000 7fd35b7dff40 1 -- 172.21.15.38:0/4050539643 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55afc33d46c0 con 0x55afc332e170 2024-09-21T03:43:05.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.663+0000 7fd355a9a640 1 --2- 172.21.15.38:0/4050539643 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x55afc332e170 0x55afc332a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:05.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.663+0000 7fd355299640 1 --2- 172.21.15.38:0/4050539643 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd338077570 0x7fd338079a80 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fd3480012a0 tx=0x7fd348001210 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:05.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.663+0000 7fd355a9a640 1 --2- 172.21.15.38:0/4050539643 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x55afc332e170 0x55afc332a910 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:05.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.666+0000 7fd355a9a640 1 -- 172.21.15.38:0/4050539643 <== osd.7 v2:172.21.15.195:6818/1606106342 1 ==== osd_op_reply(1 601.00000000 [create] v124'846 uv820 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fd35000b040 con 0x55afc332e170 2024-09-21T03:43:05.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.666+0000 7fd35b7dff40 1 -- 172.21.15.38:0/4050539643 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e124) -- 0x55afc33d4a60 con 0x55afc332e170 2024-09-21T03:43:05.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.667+0000 7fd355a9a640 1 -- 172.21.15.38:0/4050539643 <== osd.7 v2:172.21.15.195:6818/1606106342 2 ==== osd_op_reply(2 601.00000000 [delete] v124'847 uv847 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd35000b040 con 0x55afc332e170 2024-09-21T03:43:05.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.667+0000 7fd35b7dff40 1 -- 172.21.15.38:0/4050539643 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55afc33d4e00 con 0x55afc332e170 2024-09-21T03:43:05.671 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-09-21T03:43:05.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.669+0000 7fd355a9a640 1 -- 172.21.15.38:0/4050539643 <== osd.7 v2:172.21.15.195:6818/1606106342 3 ==== osd_op_reply(3 601.00000000 [create] v124'848 uv848 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd35000b040 con 0x55afc332e170 2024-09-21T03:43:05.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.669+0000 7fd35b7dff40 1 -- 172.21.15.38:0/4050539643 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55afc33d61b0 con 0x55afc332e170 2024-09-21T03:43:05.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.671+0000 7fd355a9a640 1 -- 172.21.15.38:0/4050539643 <== osd.7 v2:172.21.15.195:6818/1606106342 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v124'849 uv849 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd35000b040 con 0x55afc332e170 2024-09-21T03:43:05.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.671+0000 7fd35b7dff40 1 -- 172.21.15.38:0/4050539643 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55afc33d6550 con 0x55afc332e170 2024-09-21T03:43:05.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.672+0000 7fd355a9a640 1 -- 172.21.15.38:0/4050539643 <== osd.7 v2:172.21.15.195:6818/1606106342 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v124'850 uv850 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd35000b040 con 0x55afc332e170 2024-09-21T03:43:05.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.672+0000 7fd35b7dff40 1 -- 172.21.15.38:0/4050539643 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55afc33d51a0 con 0x55afc332e170 2024-09-21T03:43:05.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.674+0000 7fd355a9a640 1 -- 172.21.15.38:0/4050539643 <== osd.7 v2:172.21.15.195:6818/1606106342 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v124'851 uv851 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd35000b040 con 0x55afc332e170 2024-09-21T03:43:05.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.674+0000 7fd35b7dff40 1 -- 172.21.15.38:0/4050539643 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55afc33d5590 con 0x55afc332e170 2024-09-21T03:43:05.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.676+0000 7fd355a9a640 1 -- 172.21.15.38:0/4050539643 <== osd.7 v2:172.21.15.195:6818/1606106342 7 ==== osd_op_reply(7 602.00000000 [create] v124'852 uv826 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fd35000b040 con 0x55afc332e170 2024-09-21T03:43:05.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.676+0000 7fd35b7dff40 1 -- 172.21.15.38:0/4050539643 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e124) -- 0x55afc33d5980 con 0x55afc332e170 2024-09-21T03:43:05.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.677+0000 7fd355a9a640 1 -- 172.21.15.38:0/4050539643 <== osd.7 v2:172.21.15.195:6818/1606106342 8 ==== osd_op_reply(8 602.00000000 [delete] v124'853 uv853 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd35000b040 con 0x55afc332e170 2024-09-21T03:43:05.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.677+0000 7fd35b7dff40 1 -- 172.21.15.38:0/4050539643 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55afc33d5980 con 0x55afc332e170 2024-09-21T03:43:05.681 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-09-21T03:43:05.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.679+0000 7fd355a9a640 1 -- 172.21.15.38:0/4050539643 <== osd.7 v2:172.21.15.195:6818/1606106342 9 ==== osd_op_reply(9 602.00000000 [create] v124'854 uv854 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd35000b040 con 0x55afc332e170 2024-09-21T03:43:05.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.679+0000 7fd35b7dff40 1 -- 172.21.15.38:0/4050539643 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55afc33d5930 con 0x55afc332e170 2024-09-21T03:43:05.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.681+0000 7fd355a9a640 1 -- 172.21.15.38:0/4050539643 <== osd.7 v2:172.21.15.195:6818/1606106342 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v124'855 uv855 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd35000b040 con 0x55afc332e170 2024-09-21T03:43:05.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.681+0000 7fd35b7dff40 1 -- 172.21.15.38:0/4050539643 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55afc33d5930 con 0x55afc332e170 2024-09-21T03:43:05.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.682+0000 7fd355a9a640 1 -- 172.21.15.38:0/4050539643 <== osd.7 v2:172.21.15.195:6818/1606106342 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v124'856 uv856 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd35000b040 con 0x55afc332e170 2024-09-21T03:43:05.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.682+0000 7fd35b7dff40 1 -- 172.21.15.38:0/4050539643 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55afc33d5930 con 0x55afc332e170 2024-09-21T03:43:05.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.684+0000 7fd355a9a640 1 -- 172.21.15.38:0/4050539643 <== osd.7 v2:172.21.15.195:6818/1606106342 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v124'857 uv857 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd35000b040 con 0x55afc332e170 2024-09-21T03:43:05.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.684+0000 7fd35b7dff40 1 -- 172.21.15.38:0/4050539643 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:13 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55afc33d5930 con 0x55afc332e170 2024-09-21T03:43:05.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.686+0000 7fd355a9a640 1 -- 172.21.15.38:0/4050539643 <== osd.7 v2:172.21.15.195:6818/1606106342 13 ==== osd_op_reply(13 10000000000.00000000 [create] v124'858 uv833 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7fd35000b040 con 0x55afc332e170 2024-09-21T03:43:05.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.686+0000 7fd35b7dff40 1 -- 172.21.15.38:0/4050539643 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:14 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e124) -- 0x55afc33d5930 con 0x55afc332e170 2024-09-21T03:43:05.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.687+0000 7fd355a9a640 1 -- 172.21.15.38:0/4050539643 <== osd.7 v2:172.21.15.195:6818/1606106342 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v124'859 uv859 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fd35000b040 con 0x55afc332e170 2024-09-21T03:43:05.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.687+0000 7fd35b7dff40 1 -- 172.21.15.38:0/4050539643 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:15 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55afc33d5930 con 0x55afc332e170 2024-09-21T03:43:05.691 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-09-21T03:43:05.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.689+0000 7fd355a9a640 1 -- 172.21.15.38:0/4050539643 <== osd.7 v2:172.21.15.195:6818/1606106342 15 ==== osd_op_reply(15 10000000000.00000000 [create] v124'860 uv860 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fd35000b040 con 0x55afc332e170 2024-09-21T03:43:05.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.689+0000 7fd35b7dff40 1 -- 172.21.15.38:0/4050539643 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:16 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55afc33d5930 con 0x55afc332e170 2024-09-21T03:43:05.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.690+0000 7fd355a9a640 1 -- 172.21.15.38:0/4050539643 <== osd.7 v2:172.21.15.195:6818/1606106342 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v124'861 uv861 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fd35000b040 con 0x55afc332e170 2024-09-21T03:43:05.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.691+0000 7fd35b7dff40 1 -- 172.21.15.38:0/4050539643 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:17 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55afc33d5930 con 0x55afc332e170 2024-09-21T03:43:05.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.692+0000 7fd355a9a640 1 -- 172.21.15.38:0/4050539643 <== osd.7 v2:172.21.15.195:6818/1606106342 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v124'862 uv862 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fd35000b040 con 0x55afc332e170 2024-09-21T03:43:05.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.692+0000 7fd35b7dff40 1 -- 172.21.15.38:0/4050539643 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:18 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55afc33d5930 con 0x55afc332e170 2024-09-21T03:43:05.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.694+0000 7fd355a9a640 1 -- 172.21.15.38:0/4050539643 <== osd.7 v2:172.21.15.195:6818/1606106342 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v124'863 uv863 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fd35000b040 con 0x55afc332e170 2024-09-21T03:43:05.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.694+0000 7fd35b7dff40 1 -- 172.21.15.38:0/4050539643 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:19 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=585b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55afc33d68f0 con 0x55afc332e170 2024-09-21T03:43:05.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.696+0000 7fd355a9a640 1 -- 172.21.15.38:0/4050539643 <== osd.7 v2:172.21.15.195:6818/1606106342 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v124'864 uv864 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fd35000b040 con 0x55afc332e170 2024-09-21T03:43:05.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.696+0000 7fd35b7dff40 1 -- 172.21.15.38:0/4050539643 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:20 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55afc33d68f0 con 0x55afc332e170 2024-09-21T03:43:05.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.697+0000 7fd355a9a640 1 -- 172.21.15.38:0/4050539643 <== osd.7 v2:172.21.15.195:6818/1606106342 20 ==== osd_op_reply(20 600.00000000 [create] v124'865 uv839 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fd35000b040 con 0x55afc332e170 2024-09-21T03:43:05.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.697+0000 7fd35b7dff40 1 -- 172.21.15.38:0/4050539643 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:21 13.10 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e124) -- 0x55afc33d68f0 con 0x55afc332e170 2024-09-21T03:43:05.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.699+0000 7fd355a9a640 1 -- 172.21.15.38:0/4050539643 <== osd.7 v2:172.21.15.195:6818/1606106342 21 ==== osd_op_reply(21 600.00000000 [delete] v124'866 uv866 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd35000b040 con 0x55afc332e170 2024-09-21T03:43:05.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.699+0000 7fd35b7dff40 1 -- 172.21.15.38:0/4050539643 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:22 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55afc33d68f0 con 0x55afc332e170 2024-09-21T03:43:05.702 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-09-21T03:43:05.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.700+0000 7fd355a9a640 1 -- 172.21.15.38:0/4050539643 <== osd.7 v2:172.21.15.195:6818/1606106342 22 ==== osd_op_reply(22 600.00000000 [create] v124'867 uv867 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd35000b040 con 0x55afc332e170 2024-09-21T03:43:05.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.700+0000 7fd35b7dff40 1 -- 172.21.15.38:0/4050539643 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:23 13.10 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55afc33d7900 con 0x55afc332e170 2024-09-21T03:43:05.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.702+0000 7fd355a9a640 1 -- 172.21.15.38:0/4050539643 <== osd.7 v2:172.21.15.195:6818/1606106342 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v124'868 uv868 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd35000b040 con 0x55afc332e170 2024-09-21T03:43:05.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.702+0000 7fd35b7dff40 1 -- 172.21.15.38:0/4050539643 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:24 13.10 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55afc33d7900 con 0x55afc332e170 2024-09-21T03:43:05.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.703+0000 7fd355a9a640 1 -- 172.21.15.38:0/4050539643 <== osd.7 v2:172.21.15.195:6818/1606106342 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v124'869 uv869 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd35000b040 con 0x55afc332e170 2024-09-21T03:43:05.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.704+0000 7fd35b7dff40 1 -- 172.21.15.38:0/4050539643 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:25 13.10 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55afc33d68f0 con 0x55afc332e170 2024-09-21T03:43:05.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.705+0000 7fd355a9a640 1 -- 172.21.15.38:0/4050539643 <== osd.7 v2:172.21.15.195:6818/1606106342 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v124'870 uv870 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd35000b040 con 0x55afc332e170 2024-09-21T03:43:05.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.705+0000 7fd35b7dff40 1 -- 172.21.15.38:0/4050539643 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:26 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55afc33d68f0 con 0x55afc332e170 2024-09-21T03:43:05.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.707+0000 7fd355a9a640 1 -- 172.21.15.38:0/4050539643 <== osd.7 v2:172.21.15.195:6818/1606106342 26 ==== osd_op_reply(26 603.00000000 [create] v124'871 uv845 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fd35000b040 con 0x55afc332e170 2024-09-21T03:43:05.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.707+0000 7fd35b7dff40 1 -- 172.21.15.38:0/4050539643 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:27 13.10 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e124) -- 0x55afc33d68f0 con 0x55afc332e170 2024-09-21T03:43:05.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.708+0000 7fd355a9a640 1 -- 172.21.15.38:0/4050539643 <== osd.7 v2:172.21.15.195:6818/1606106342 27 ==== osd_op_reply(27 603.00000000 [delete] v124'872 uv872 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd35000b040 con 0x55afc332e170 2024-09-21T03:43:05.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.708+0000 7fd35b7dff40 1 -- 172.21.15.38:0/4050539643 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:28 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55afc33d68f0 con 0x55afc332e170 2024-09-21T03:43:05.712 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-09-21T03:43:05.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.710+0000 7fd355a9a640 1 -- 172.21.15.38:0/4050539643 <== osd.7 v2:172.21.15.195:6818/1606106342 28 ==== osd_op_reply(28 603.00000000 [create] v124'873 uv873 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd35000b040 con 0x55afc332e170 2024-09-21T03:43:05.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.710+0000 7fd35b7dff40 1 -- 172.21.15.38:0/4050539643 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:29 13.10 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55afc33d7900 con 0x55afc332e170 2024-09-21T03:43:05.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.711+0000 7fd355a9a640 1 -- 172.21.15.38:0/4050539643 <== osd.7 v2:172.21.15.195:6818/1606106342 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v124'874 uv874 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd35000b040 con 0x55afc332e170 2024-09-21T03:43:05.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.711+0000 7fd35b7dff40 1 -- 172.21.15.38:0/4050539643 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:30 13.10 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55afc33d7900 con 0x55afc332e170 2024-09-21T03:43:05.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.713+0000 7fd355a9a640 1 -- 172.21.15.38:0/4050539643 <== osd.7 v2:172.21.15.195:6818/1606106342 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v124'875 uv875 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd35000b040 con 0x55afc332e170 2024-09-21T03:43:05.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.713+0000 7fd35b7dff40 1 -- 172.21.15.38:0/4050539643 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:31 13.10 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55afc33d68f0 con 0x55afc332e170 2024-09-21T03:43:05.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.715+0000 7fd355a9a640 1 -- 172.21.15.38:0/4050539643 <== osd.7 v2:172.21.15.195:6818/1606106342 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v124'876 uv876 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd35000b040 con 0x55afc332e170 2024-09-21T03:43:05.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.715+0000 7fd35b7dff40 1 --2- 172.21.15.38:0/4050539643 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x55afc33d6e00 0x55afc33d9220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:05.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.715+0000 7fd35b7dff40 1 -- 172.21.15.38:0/4050539643 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55afc33d9760 con 0x55afc33d6e00 2024-09-21T03:43:05.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.715+0000 7fd355299640 1 --2- 172.21.15.38:0/4050539643 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x55afc33d6e00 0x55afc33d9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:05.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.716+0000 7fd355299640 1 --2- 172.21.15.38:0/4050539643 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x55afc33d6e00 0x55afc33d9220 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:05.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.718+0000 7fd355299640 1 -- 172.21.15.38:0/4050539643 <== osd.6 v2:172.21.15.195:6826/4055825675 1 ==== osd_op_reply(32 1.00000000.inode [create] v124'616 uv599 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7fd348012600 con 0x55afc33d6e00 2024-09-21T03:43:05.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.718+0000 7fd35b7dff40 1 -- 172.21.15.38:0/4050539643 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e124) -- 0x55afc33d9b00 con 0x55afc33d6e00 2024-09-21T03:43:05.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.720+0000 7fd355299640 1 -- 172.21.15.38:0/4050539643 <== osd.6 v2:172.21.15.195:6826/4055825675 2 ==== osd_op_reply(33 1.00000000.inode [delete] v124'617 uv617 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fd348012600 con 0x55afc33d6e00 2024-09-21T03:43:05.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.720+0000 7fd35b7dff40 1 -- 172.21.15.38:0/4050539643 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55afc33d9ea0 con 0x55afc33d6e00 2024-09-21T03:43:05.723 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-09-21T03:43:05.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.721+0000 7fd355299640 1 -- 172.21.15.38:0/4050539643 <== osd.6 v2:172.21.15.195:6826/4055825675 3 ==== osd_op_reply(34 1.00000000.inode [create] v124'618 uv618 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fd348012600 con 0x55afc33d6e00 2024-09-21T03:43:05.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.721+0000 7fd35b7dff40 1 -- 172.21.15.38:0/4050539643 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55afc33da6f0 con 0x55afc33d6e00 2024-09-21T03:43:05.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.723+0000 7fd355299640 1 -- 172.21.15.38:0/4050539643 <== osd.6 v2:172.21.15.195:6826/4055825675 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v124'619 uv619 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fd348012600 con 0x55afc33d6e00 2024-09-21T03:43:05.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.723+0000 7fd35b7dff40 1 -- 172.21.15.38:0/4050539643 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55afc33daa90 con 0x55afc33d6e00 2024-09-21T03:43:05.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.724+0000 7fd355299640 1 -- 172.21.15.38:0/4050539643 <== osd.6 v2:172.21.15.195:6826/4055825675 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v124'620 uv620 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fd348012600 con 0x55afc33d6e00 2024-09-21T03:43:05.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.724+0000 7fd35b7dff40 1 -- 172.21.15.38:0/4050539643 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55afc33dae30 con 0x55afc33d6e00 2024-09-21T03:43:05.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.725+0000 7fd355299640 1 -- 172.21.15.38:0/4050539643 <== osd.6 v2:172.21.15.195:6826/4055825675 6 ==== osd_op_reply(37 200.00000000 [create] v124'621 uv604 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fd348012600 con 0x55afc33d6e00 2024-09-21T03:43:05.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.725+0000 7fd35b7dff40 1 -- 172.21.15.38:0/4050539643 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e124) -- 0x55afc33db1d0 con 0x55afc33d6e00 2024-09-21T03:43:05.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.727+0000 7fd355299640 1 -- 172.21.15.38:0/4050539643 <== osd.6 v2:172.21.15.195:6826/4055825675 7 ==== osd_op_reply(38 200.00000000 [delete] v124'622 uv622 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd348012600 con 0x55afc33d6e00 2024-09-21T03:43:05.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.727+0000 7fd35b7dff40 1 -- 172.21.15.38:0/4050539643 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55afc33db570 con 0x55afc33d6e00 2024-09-21T03:43:05.730 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-09-21T03:43:05.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.728+0000 7fd355299640 1 -- 172.21.15.38:0/4050539643 <== osd.6 v2:172.21.15.195:6826/4055825675 8 ==== osd_op_reply(39 200.00000000 [create] v124'623 uv623 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd348012600 con 0x55afc33d6e00 2024-09-21T03:43:05.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.728+0000 7fd35b7dff40 1 -- 172.21.15.38:0/4050539643 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55afc33db570 con 0x55afc33d6e00 2024-09-21T03:43:05.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.729+0000 7fd355299640 1 -- 172.21.15.38:0/4050539643 <== osd.6 v2:172.21.15.195:6826/4055825675 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v124'624 uv624 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd348012600 con 0x55afc33d6e00 2024-09-21T03:43:05.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.729+0000 7fd35b7dff40 1 -- 172.21.15.38:0/4050539643 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55afc33db570 con 0x55afc33d6e00 2024-09-21T03:43:05.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.731+0000 7fd355299640 1 -- 172.21.15.38:0/4050539643 <== osd.6 v2:172.21.15.195:6826/4055825675 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v124'625 uv625 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd348012600 con 0x55afc33d6e00 2024-09-21T03:43:05.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.731+0000 7fd35b7dff40 1 -- 172.21.15.38:0/4050539643 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:42 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55afc33db570 con 0x55afc33d6e00 2024-09-21T03:43:05.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.732+0000 7fd355299640 1 -- 172.21.15.38:0/4050539643 <== osd.6 v2:172.21.15.195:6826/4055825675 11 ==== osd_op_reply(42 200.00000001 [create] v124'626 uv609 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fd348012600 con 0x55afc33d6e00 2024-09-21T03:43:05.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.732+0000 7fd35b7dff40 1 -- 172.21.15.38:0/4050539643 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:43 13.14 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e124) -- 0x55afc33db570 con 0x55afc33d6e00 2024-09-21T03:43:05.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.734+0000 7fd355299640 1 -- 172.21.15.38:0/4050539643 <== osd.6 v2:172.21.15.195:6826/4055825675 12 ==== osd_op_reply(43 200.00000001 [delete] v124'627 uv627 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd348012600 con 0x55afc33d6e00 2024-09-21T03:43:05.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.734+0000 7fd35b7dff40 1 -- 172.21.15.38:0/4050539643 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:44 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55afc33db570 con 0x55afc33d6e00 2024-09-21T03:43:05.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.735+0000 7fd355299640 1 -- 172.21.15.38:0/4050539643 <== osd.6 v2:172.21.15.195:6826/4055825675 13 ==== osd_op_reply(44 200.00000001 [create] v124'628 uv628 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd348012600 con 0x55afc33d6e00 2024-09-21T03:43:05.737 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-09-21T03:43:05.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.735+0000 7fd35b7dff40 1 -- 172.21.15.38:0/4050539643 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:45 13.14 13:2e2fa760:::200.00000001:head [write 0~17490 in=17490b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55afc33df9f0 con 0x55afc33d6e00 2024-09-21T03:43:05.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.737+0000 7fd355299640 1 -- 172.21.15.38:0/4050539643 <== osd.6 v2:172.21.15.195:6826/4055825675 14 ==== osd_op_reply(45 200.00000001 [write 0~17490] v124'629 uv629 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd348012600 con 0x55afc33d6e00 2024-09-21T03:43:05.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.737+0000 7fd35b7dff40 1 -- 172.21.15.38:0/4050539643 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:46 13.14 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55afc33df9f0 con 0x55afc33d6e00 2024-09-21T03:43:05.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.739+0000 7fd355299640 1 -- 172.21.15.38:0/4050539643 <== osd.6 v2:172.21.15.195:6826/4055825675 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v124'630 uv630 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd348012600 con 0x55afc33d6e00 2024-09-21T03:43:05.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.739+0000 7fd35b7dff40 1 -- 172.21.15.38:0/4050539643 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:47 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55afc33db570 con 0x55afc33d6e00 2024-09-21T03:43:05.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.740+0000 7fd355299640 1 -- 172.21.15.38:0/4050539643 <== osd.6 v2:172.21.15.195:6826/4055825675 16 ==== osd_op_reply(47 606.00000000 [create] v124'631 uv615 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fd348012600 con 0x55afc33d6e00 2024-09-21T03:43:05.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.740+0000 7fd35b7dff40 1 -- 172.21.15.38:0/4050539643 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:48 13.14 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e124) -- 0x55afc33db570 con 0x55afc33d6e00 2024-09-21T03:43:05.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.741+0000 7fd355299640 1 -- 172.21.15.38:0/4050539643 <== osd.6 v2:172.21.15.195:6826/4055825675 17 ==== osd_op_reply(48 606.00000000 [delete] v124'632 uv632 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd348012600 con 0x55afc33d6e00 2024-09-21T03:43:05.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.741+0000 7fd35b7dff40 1 -- 172.21.15.38:0/4050539643 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:49 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55afc33db570 con 0x55afc33d6e00 2024-09-21T03:43:05.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.743+0000 7fd355299640 1 -- 172.21.15.38:0/4050539643 <== osd.6 v2:172.21.15.195:6826/4055825675 18 ==== osd_op_reply(49 606.00000000 [create] v124'633 uv633 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd348012600 con 0x55afc33d6e00 2024-09-21T03:43:05.744 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-09-21T03:43:05.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.743+0000 7fd35b7dff40 1 -- 172.21.15.38:0/4050539643 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:50 13.14 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55afc33dc580 con 0x55afc33d6e00 2024-09-21T03:43:05.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.744+0000 7fd355299640 1 -- 172.21.15.38:0/4050539643 <== osd.6 v2:172.21.15.195:6826/4055825675 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v124'634 uv634 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd348012600 con 0x55afc33d6e00 2024-09-21T03:43:05.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.744+0000 7fd35b7dff40 1 -- 172.21.15.38:0/4050539643 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:51 13.14 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55afc33dc580 con 0x55afc33d6e00 2024-09-21T03:43:05.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.745+0000 7fd355299640 1 -- 172.21.15.38:0/4050539643 <== osd.6 v2:172.21.15.195:6826/4055825675 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v124'635 uv635 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd348012600 con 0x55afc33d6e00 2024-09-21T03:43:05.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.745+0000 7fd35b7dff40 1 -- 172.21.15.38:0/4050539643 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:52 13.14 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55afc33db570 con 0x55afc33d6e00 2024-09-21T03:43:05.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.746+0000 7fd355299640 1 -- 172.21.15.38:0/4050539643 <== osd.6 v2:172.21.15.195:6826/4055825675 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v124'636 uv636 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd348012600 con 0x55afc33d6e00 2024-09-21T03:43:05.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.746+0000 7fd35b7dff40 1 --2- 172.21.15.38:0/4050539643 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x55afc33dba80 0x55afc33ddea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:05.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.746+0000 7fd35b7dff40 1 -- 172.21.15.38:0/4050539643 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:53 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55afc33de3e0 con 0x55afc33dba80 2024-09-21T03:43:05.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.746+0000 7fd354a98640 1 --2- 172.21.15.38:0/4050539643 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x55afc33dba80 0x55afc33ddea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:05.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.747+0000 7fd354a98640 1 --2- 172.21.15.38:0/4050539643 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x55afc33dba80 0x55afc33ddea0 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:05.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.748+0000 7fd354a98640 1 -- 172.21.15.38:0/4050539643 <== osd.0 v2:172.21.15.38:6800/2381639433 1 ==== osd_op_reply(53 607.00000000 [create] v124'165 uv164 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fd34c05ec10 con 0x55afc33dba80 2024-09-21T03:43:05.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.748+0000 7fd35b7dff40 1 -- 172.21.15.38:0/4050539643 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:54 13.1c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e124) -- 0x55afc33de780 con 0x55afc33dba80 2024-09-21T03:43:05.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.749+0000 7fd354a98640 1 -- 172.21.15.38:0/4050539643 <== osd.0 v2:172.21.15.38:6800/2381639433 2 ==== osd_op_reply(54 607.00000000 [delete] v124'166 uv166 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd34c05ec10 con 0x55afc33dba80 2024-09-21T03:43:05.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.749+0000 7fd35b7dff40 1 -- 172.21.15.38:0/4050539643 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:55 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55afc33deb20 con 0x55afc33dba80 2024-09-21T03:43:05.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.750+0000 7fd354a98640 1 -- 172.21.15.38:0/4050539643 <== osd.0 v2:172.21.15.38:6800/2381639433 3 ==== osd_op_reply(55 607.00000000 [create] v124'167 uv167 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd34c05ec10 con 0x55afc33dba80 2024-09-21T03:43:05.752 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-09-21T03:43:05.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.750+0000 7fd35b7dff40 1 -- 172.21.15.38:0/4050539643 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:56 13.1c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55afc33dfed0 con 0x55afc33dba80 2024-09-21T03:43:05.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.751+0000 7fd354a98640 1 -- 172.21.15.38:0/4050539643 <== osd.0 v2:172.21.15.38:6800/2381639433 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v124'168 uv168 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd34c05ec10 con 0x55afc33dba80 2024-09-21T03:43:05.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.751+0000 7fd35b7dff40 1 -- 172.21.15.38:0/4050539643 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:57 13.1c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55afc33e0350 con 0x55afc33dba80 2024-09-21T03:43:05.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.752+0000 7fd354a98640 1 -- 172.21.15.38:0/4050539643 <== osd.0 v2:172.21.15.38:6800/2381639433 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v124'169 uv169 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd34c05ec10 con 0x55afc33dba80 2024-09-21T03:43:05.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.752+0000 7fd35b7dff40 1 -- 172.21.15.38:0/4050539643 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:58 13.1c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55afc33deec0 con 0x55afc33dba80 2024-09-21T03:43:05.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.753+0000 7fd354a98640 1 -- 172.21.15.38:0/4050539643 <== osd.0 v2:172.21.15.38:6800/2381639433 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v124'170 uv170 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd34c05ec10 con 0x55afc33dba80 2024-09-21T03:43:05.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.753+0000 7fd35b7dff40 1 --2- 172.21.15.38:0/4050539643 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x55afc33df930 0x55afc33e27b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:05.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.753+0000 7fd35b7dff40 1 -- 172.21.15.38:0/4050539643 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:59 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55afc33e2d80 con 0x55afc33df930 2024-09-21T03:43:05.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.753+0000 7fd355a9a640 1 --2- 172.21.15.38:0/4050539643 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x55afc33df930 0x55afc33e27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:05.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.754+0000 7fd355a9a640 1 --2- 172.21.15.38:0/4050539643 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x55afc33df930 0x55afc33e27b0 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:05.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.755+0000 7fd355a9a640 1 -- 172.21.15.38:0/4050539643 <== osd.3 v2:172.21.15.38:6808/1286810294 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v124'120 uv119 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7fd35000b040 con 0x55afc33df930 2024-09-21T03:43:05.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.755+0000 7fd35b7dff40 1 -- 172.21.15.38:0/4050539643 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:60 13.6 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e124) -- 0x55afc33e3190 con 0x55afc33df930 2024-09-21T03:43:05.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.756+0000 7fd355a9a640 1 -- 172.21.15.38:0/4050539643 <== osd.3 v2:172.21.15.38:6808/1286810294 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v124'121 uv121 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fd35000b040 con 0x55afc33df930 2024-09-21T03:43:05.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.756+0000 7fd35b7dff40 1 -- 172.21.15.38:0/4050539643 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:61 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55afc33e3190 con 0x55afc33df930 2024-09-21T03:43:05.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.757+0000 7fd355a9a640 1 -- 172.21.15.38:0/4050539643 <== osd.3 v2:172.21.15.38:6808/1286810294 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v124'122 uv122 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fd35000b040 con 0x55afc33df930 2024-09-21T03:43:05.759 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-09-21T03:43:05.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.757+0000 7fd35b7dff40 1 -- 172.21.15.38:0/4050539643 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:62 13.6 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55afc33e3190 con 0x55afc33df930 2024-09-21T03:43:05.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.758+0000 7fd355a9a640 1 -- 172.21.15.38:0/4050539643 <== osd.3 v2:172.21.15.38:6808/1286810294 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v124'123 uv123 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fd35000b040 con 0x55afc33df930 2024-09-21T03:43:05.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.758+0000 7fd35b7dff40 1 -- 172.21.15.38:0/4050539643 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55afc33e3190 con 0x55afc33dba80 2024-09-21T03:43:05.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.759+0000 7fd354a98640 1 -- 172.21.15.38:0/4050539643 <== osd.0 v2:172.21.15.38:6800/2381639433 7 ==== osd_op_reply(63 608.00000000 [create] v124'165 uv164 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fd34c05ec10 con 0x55afc33dba80 2024-09-21T03:43:05.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.759+0000 7fd35b7dff40 1 -- 172.21.15.38:0/4050539643 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e124) -- 0x55afc33e3580 con 0x55afc33dba80 2024-09-21T03:43:05.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.760+0000 7fd354a98640 1 -- 172.21.15.38:0/4050539643 <== osd.0 v2:172.21.15.38:6800/2381639433 8 ==== osd_op_reply(64 608.00000000 [delete] v124'166 uv166 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd34c05ec10 con 0x55afc33dba80 2024-09-21T03:43:05.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.760+0000 7fd35b7dff40 1 -- 172.21.15.38:0/4050539643 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55afc33e3580 con 0x55afc33dba80 2024-09-21T03:43:05.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.762+0000 7fd354a98640 1 -- 172.21.15.38:0/4050539643 <== osd.0 v2:172.21.15.38:6800/2381639433 9 ==== osd_op_reply(65 608.00000000 [create] v124'167 uv167 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd34c05ec10 con 0x55afc33dba80 2024-09-21T03:43:05.764 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-09-21T03:43:05.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.762+0000 7fd35b7dff40 1 -- 172.21.15.38:0/4050539643 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55afc33e4590 con 0x55afc33dba80 2024-09-21T03:43:05.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.763+0000 7fd354a98640 1 -- 172.21.15.38:0/4050539643 <== osd.0 v2:172.21.15.38:6800/2381639433 10 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v124'168 uv168 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd34c05ec10 con 0x55afc33dba80 2024-09-21T03:43:05.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.763+0000 7fd35b7dff40 1 -- 172.21.15.38:0/4050539643 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55afc33e4590 con 0x55afc33dba80 2024-09-21T03:43:05.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.764+0000 7fd354a98640 1 -- 172.21.15.38:0/4050539643 <== osd.0 v2:172.21.15.38:6800/2381639433 11 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v124'169 uv169 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd34c05ec10 con 0x55afc33dba80 2024-09-21T03:43:05.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.764+0000 7fd35b7dff40 1 -- 172.21.15.38:0/4050539643 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55afc33e3580 con 0x55afc33dba80 2024-09-21T03:43:05.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.765+0000 7fd354a98640 1 -- 172.21.15.38:0/4050539643 <== osd.0 v2:172.21.15.38:6800/2381639433 12 ==== osd_op_reply(68 608.00000000 [omap-set-header] v124'170 uv170 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd34c05ec10 con 0x55afc33dba80 2024-09-21T03:43:05.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.765+0000 7fd35b7dff40 1 -- 172.21.15.38:0/4050539643 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55afc33e3580 con 0x55afc33df930 2024-09-21T03:43:05.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.767+0000 7fd355a9a640 1 -- 172.21.15.38:0/4050539643 <== osd.3 v2:172.21.15.38:6808/1286810294 5 ==== osd_op_reply(69 604.00000000 [create] v124'305 uv299 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fd35000b040 con 0x55afc33df930 2024-09-21T03:43:05.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.767+0000 7fd35b7dff40 1 -- 172.21.15.38:0/4050539643 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e124) -- 0x55afc33e3580 con 0x55afc33df930 2024-09-21T03:43:05.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.768+0000 7fd355a9a640 1 -- 172.21.15.38:0/4050539643 <== osd.3 v2:172.21.15.38:6808/1286810294 6 ==== osd_op_reply(70 604.00000000 [delete] v124'306 uv306 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd35000b040 con 0x55afc33df930 2024-09-21T03:43:05.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.768+0000 7fd35b7dff40 1 -- 172.21.15.38:0/4050539643 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55afc33e3530 con 0x55afc33df930 2024-09-21T03:43:05.771 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-09-21T03:43:05.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.769+0000 7fd355a9a640 1 -- 172.21.15.38:0/4050539643 <== osd.3 v2:172.21.15.38:6808/1286810294 7 ==== osd_op_reply(71 604.00000000 [create] v124'307 uv307 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd35000b040 con 0x55afc33df930 2024-09-21T03:43:05.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.769+0000 7fd35b7dff40 1 -- 172.21.15.38:0/4050539643 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55afc33e4540 con 0x55afc33df930 2024-09-21T03:43:05.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.770+0000 7fd355a9a640 1 -- 172.21.15.38:0/4050539643 <== osd.3 v2:172.21.15.38:6808/1286810294 8 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v124'308 uv308 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd35000b040 con 0x55afc33df930 2024-09-21T03:43:05.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.770+0000 7fd35b7dff40 1 -- 172.21.15.38:0/4050539643 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55afc33e4540 con 0x55afc33df930 2024-09-21T03:43:05.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.771+0000 7fd355a9a640 1 -- 172.21.15.38:0/4050539643 <== osd.3 v2:172.21.15.38:6808/1286810294 9 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v124'309 uv309 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd35000b040 con 0x55afc33df930 2024-09-21T03:43:05.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.771+0000 7fd35b7dff40 1 -- 172.21.15.38:0/4050539643 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55afc33e3530 con 0x55afc33df930 2024-09-21T03:43:05.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.772+0000 7fd355a9a640 1 -- 172.21.15.38:0/4050539643 <== osd.3 v2:172.21.15.38:6808/1286810294 10 ==== osd_op_reply(74 604.00000000 [omap-set-header] v124'310 uv310 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd35000b040 con 0x55afc33df930 2024-09-21T03:43:05.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.773+0000 7fd35b7dff40 1 -- 172.21.15.38:0/4050539643 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55afc33e3530 con 0x55afc33df930 2024-09-21T03:43:05.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.774+0000 7fd355a9a640 1 -- 172.21.15.38:0/4050539643 <== osd.3 v2:172.21.15.38:6808/1286810294 11 ==== osd_op_reply(75 500.00000000 [create] v124'311 uv304 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fd35000b040 con 0x55afc33df930 2024-09-21T03:43:05.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.774+0000 7fd35b7dff40 1 -- 172.21.15.38:0/4050539643 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e124) -- 0x55afc33e3530 con 0x55afc33df930 2024-09-21T03:43:05.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.775+0000 7fd355a9a640 1 -- 172.21.15.38:0/4050539643 <== osd.3 v2:172.21.15.38:6808/1286810294 12 ==== osd_op_reply(76 500.00000000 [delete] v124'312 uv312 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd35000b040 con 0x55afc33df930 2024-09-21T03:43:05.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.775+0000 7fd35b7dff40 1 -- 172.21.15.38:0/4050539643 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55afc33e3530 con 0x55afc33df930 2024-09-21T03:43:05.778 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-09-21T03:43:05.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.776+0000 7fd355a9a640 1 -- 172.21.15.38:0/4050539643 <== osd.3 v2:172.21.15.38:6808/1286810294 13 ==== osd_op_reply(77 500.00000000 [create] v124'313 uv313 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd35000b040 con 0x55afc33df930 2024-09-21T03:43:05.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.776+0000 7fd35b7dff40 1 -- 172.21.15.38:0/4050539643 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55afc33e3530 con 0x55afc33df930 2024-09-21T03:43:05.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.777+0000 7fd355a9a640 1 -- 172.21.15.38:0/4050539643 <== osd.3 v2:172.21.15.38:6808/1286810294 14 ==== osd_op_reply(78 500.00000000 [write 0~90] v124'314 uv314 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd35000b040 con 0x55afc33df930 2024-09-21T03:43:05.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.777+0000 7fd35b7dff40 1 -- 172.21.15.38:0/4050539643 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55afc33e3530 con 0x55afc33df930 2024-09-21T03:43:05.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.779+0000 7fd355a9a640 1 -- 172.21.15.38:0/4050539643 <== osd.3 v2:172.21.15.38:6808/1286810294 15 ==== osd_op_reply(79 500.00000000 [omap-set-header] v124'315 uv315 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd35000b040 con 0x55afc33df930 2024-09-21T03:43:05.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.779+0000 7fd35b7dff40 1 -- 172.21.15.38:0/4050539643 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:80 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55afc33e3530 con 0x55afc33dba80 2024-09-21T03:43:05.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.780+0000 7fd354a98640 1 -- 172.21.15.38:0/4050539643 <== osd.0 v2:172.21.15.38:6800/2381639433 13 ==== osd_op_reply(80 mds_snaptable [create] v124'138 uv137 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7fd34c05ec10 con 0x55afc33dba80 2024-09-21T03:43:05.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.780+0000 7fd35b7dff40 1 -- 172.21.15.38:0/4050539643 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:81 13.d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e124) -- 0x55afc33e3530 con 0x55afc33dba80 2024-09-21T03:43:05.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.781+0000 7fd354a98640 1 -- 172.21.15.38:0/4050539643 <== osd.0 v2:172.21.15.38:6800/2381639433 14 ==== osd_op_reply(81 mds_snaptable [delete] v124'139 uv139 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fd34c05ec10 con 0x55afc33dba80 2024-09-21T03:43:05.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.781+0000 7fd35b7dff40 1 -- 172.21.15.38:0/4050539643 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:82 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55afc33e3530 con 0x55afc33dba80 2024-09-21T03:43:05.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.782+0000 7fd354a98640 1 -- 172.21.15.38:0/4050539643 <== osd.0 v2:172.21.15.38:6800/2381639433 15 ==== osd_op_reply(82 mds_snaptable [create] v124'140 uv140 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fd34c05ec10 con 0x55afc33dba80 2024-09-21T03:43:05.784 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-09-21T03:43:05.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.782+0000 7fd35b7dff40 1 -- 172.21.15.38:0/4050539643 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:83 13.d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55afc33e3530 con 0x55afc33dba80 2024-09-21T03:43:05.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.783+0000 7fd354a98640 1 -- 172.21.15.38:0/4050539643 <== osd.0 v2:172.21.15.38:6800/2381639433 16 ==== osd_op_reply(83 mds_snaptable [write 0~70] v124'141 uv141 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fd34c05ec10 con 0x55afc33dba80 2024-09-21T03:43:05.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.783+0000 7fd35b7dff40 1 -- 172.21.15.38:0/4050539643 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:84 13.d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55afc33e3530 con 0x55afc33dba80 2024-09-21T03:43:05.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.784+0000 7fd354a98640 1 -- 172.21.15.38:0/4050539643 <== osd.0 v2:172.21.15.38:6800/2381639433 17 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v124'142 uv142 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fd34c05ec10 con 0x55afc33dba80 2024-09-21T03:43:05.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.785+0000 7fd35b7dff40 1 --2- 172.21.15.38:0/4050539643 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x55afc33e3e50 0x55afc33e6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:05.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.785+0000 7fd35b7dff40 1 -- 172.21.15.38:0/4050539643 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55afc33e67b0 con 0x55afc33e3e50 2024-09-21T03:43:05.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.785+0000 7fd355299640 1 --2- 172.21.15.38:0/4050539643 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x55afc33e3e50 0x55afc33e6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:05.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.785+0000 7fd355299640 1 --2- 172.21.15.38:0/4050539643 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x55afc33e3e50 0x55afc33e6270 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:05.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.786+0000 7fd355299640 1 -- 172.21.15.38:0/4050539643 <== osd.1 v2:172.21.15.38:6816/1491818732 1 ==== osd_op_reply(85 605.00000000 [create] v124'165 uv164 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fd348012600 con 0x55afc33e3e50 2024-09-21T03:43:05.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.786+0000 7fd35b7dff40 1 -- 172.21.15.38:0/4050539643 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e124) -- 0x55afc33e67f0 con 0x55afc33e3e50 2024-09-21T03:43:05.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.788+0000 7fd355299640 1 -- 172.21.15.38:0/4050539643 <== osd.1 v2:172.21.15.38:6816/1491818732 2 ==== osd_op_reply(86 605.00000000 [delete] v124'166 uv166 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd348012600 con 0x55afc33e3e50 2024-09-21T03:43:05.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.788+0000 7fd35b7dff40 1 -- 172.21.15.38:0/4050539643 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55afc33e6c10 con 0x55afc33e3e50 2024-09-21T03:43:05.791 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-09-21T03:43:05.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.789+0000 7fd355299640 1 -- 172.21.15.38:0/4050539643 <== osd.1 v2:172.21.15.38:6816/1491818732 3 ==== osd_op_reply(87 605.00000000 [create] v124'167 uv167 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd348012600 con 0x55afc33e3e50 2024-09-21T03:43:05.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.789+0000 7fd35b7dff40 1 -- 172.21.15.38:0/4050539643 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55afc33e7c20 con 0x55afc33e3e50 2024-09-21T03:43:05.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.790+0000 7fd355299640 1 -- 172.21.15.38:0/4050539643 <== osd.1 v2:172.21.15.38:6816/1491818732 4 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v124'168 uv168 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd348012600 con 0x55afc33e3e50 2024-09-21T03:43:05.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.790+0000 7fd35b7dff40 1 -- 172.21.15.38:0/4050539643 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55afc33e7c20 con 0x55afc33e3e50 2024-09-21T03:43:05.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.791+0000 7fd355299640 1 -- 172.21.15.38:0/4050539643 <== osd.1 v2:172.21.15.38:6816/1491818732 5 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v124'169 uv169 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd348012600 con 0x55afc33e3e50 2024-09-21T03:43:05.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.791+0000 7fd35b7dff40 1 -- 172.21.15.38:0/4050539643 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55afc33e6c10 con 0x55afc33e3e50 2024-09-21T03:43:05.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.793+0000 7fd355299640 1 -- 172.21.15.38:0/4050539643 <== osd.1 v2:172.21.15.38:6816/1491818732 6 ==== osd_op_reply(90 605.00000000 [omap-set-header] v124'170 uv170 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd348012600 con 0x55afc33e3e50 2024-09-21T03:43:05.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.793+0000 7fd35b7dff40 1 -- 172.21.15.38:0/4050539643 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55afc33e6c10 con 0x55afc33e3e50 2024-09-21T03:43:05.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.794+0000 7fd355299640 1 -- 172.21.15.38:0/4050539643 <== osd.1 v2:172.21.15.38:6816/1491818732 7 ==== osd_op_reply(91 mds0_inotable [create] v124'273 uv267 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7fd348012600 con 0x55afc33e3e50 2024-09-21T03:43:05.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.794+0000 7fd35b7dff40 1 -- 172.21.15.38:0/4050539643 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e124) -- 0x55afc33e6c10 con 0x55afc33e3e50 2024-09-21T03:43:05.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.795+0000 7fd355299640 1 -- 172.21.15.38:0/4050539643 <== osd.1 v2:172.21.15.38:6816/1491818732 8 ==== osd_op_reply(92 mds0_inotable [delete] v124'274 uv274 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fd348012600 con 0x55afc33e3e50 2024-09-21T03:43:05.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.795+0000 7fd35b7dff40 1 -- 172.21.15.38:0/4050539643 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55afc33e6c10 con 0x55afc33e3e50 2024-09-21T03:43:05.798 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-09-21T03:43:05.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.796+0000 7fd355299640 1 -- 172.21.15.38:0/4050539643 <== osd.1 v2:172.21.15.38:6816/1491818732 9 ==== osd_op_reply(93 mds0_inotable [create] v124'275 uv275 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fd348012600 con 0x55afc33e3e50 2024-09-21T03:43:05.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.796+0000 7fd35b7dff40 1 -- 172.21.15.38:0/4050539643 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:94 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55afc33e6c10 con 0x55afc33e3e50 2024-09-21T03:43:05.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.797+0000 7fd355299640 1 -- 172.21.15.38:0/4050539643 <== osd.1 v2:172.21.15.38:6816/1491818732 10 ==== osd_op_reply(94 mds0_inotable [write 0~34] v124'276 uv276 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fd348012600 con 0x55afc33e3e50 2024-09-21T03:43:05.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.797+0000 7fd35b7dff40 1 -- 172.21.15.38:0/4050539643 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:95 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55afc33e6c10 con 0x55afc33e3e50 2024-09-21T03:43:05.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.798+0000 7fd355299640 1 -- 172.21.15.38:0/4050539643 <== osd.1 v2:172.21.15.38:6816/1491818732 11 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v124'277 uv277 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fd348012600 con 0x55afc33e3e50 2024-09-21T03:43:05.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.798+0000 7fd35b7dff40 1 -- 172.21.15.38:0/4050539643 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:96 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55afc33e6c10 con 0x55afc33e3e50 2024-09-21T03:43:05.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.799+0000 7fd355299640 1 -- 172.21.15.38:0/4050539643 <== osd.1 v2:172.21.15.38:6816/1491818732 12 ==== osd_op_reply(96 100.00000000 [create] v124'278 uv272 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fd348012600 con 0x55afc33e3e50 2024-09-21T03:43:05.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.800+0000 7fd35b7dff40 1 -- 172.21.15.38:0/4050539643 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:97 13.13 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e124) -- 0x55afc33e6c10 con 0x55afc33e3e50 2024-09-21T03:43:05.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.801+0000 7fd355299640 1 -- 172.21.15.38:0/4050539643 <== osd.1 v2:172.21.15.38:6816/1491818732 13 ==== osd_op_reply(97 100.00000000 [delete] v124'279 uv279 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd348012600 con 0x55afc33e3e50 2024-09-21T03:43:05.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.801+0000 7fd35b7dff40 1 -- 172.21.15.38:0/4050539643 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:98 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55afc33e6c10 con 0x55afc33e3e50 2024-09-21T03:43:05.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.802+0000 7fd355299640 1 -- 172.21.15.38:0/4050539643 <== osd.1 v2:172.21.15.38:6816/1491818732 14 ==== osd_op_reply(98 100.00000000 [create] v124'280 uv280 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd348012600 con 0x55afc33e3e50 2024-09-21T03:43:05.804 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-09-21T03:43:05.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.802+0000 7fd35b7dff40 1 -- 172.21.15.38:0/4050539643 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:99 13.13 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55afc33e6c10 con 0x55afc33e3e50 2024-09-21T03:43:05.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.803+0000 7fd355299640 1 -- 172.21.15.38:0/4050539643 <== osd.1 v2:172.21.15.38:6816/1491818732 15 ==== osd_op_reply(99 100.00000000 [omap-set-header] v124'281 uv281 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd348012600 con 0x55afc33e3e50 2024-09-21T03:43:05.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.803+0000 7fd35b7dff40 1 -- 172.21.15.38:0/4050539643 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:100 13.13 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55afc33e9a20 con 0x55afc33e3e50 2024-09-21T03:43:05.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.804+0000 7fd355299640 1 -- 172.21.15.38:0/4050539643 <== osd.1 v2:172.21.15.38:6816/1491818732 16 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v124'282 uv282 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd348012600 con 0x55afc33e3e50 2024-09-21T03:43:05.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.804+0000 7fd35b7dff40 1 -- 172.21.15.38:0/4050539643 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55afc33e70c0 con 0x55afc33df930 2024-09-21T03:43:05.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.805+0000 7fd355a9a640 1 -- 172.21.15.38:0/4050539643 <== osd.3 v2:172.21.15.38:6808/1286810294 16 ==== osd_op_reply(101 mds0_sessionmap [create] v124'120 uv119 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7fd35000b040 con 0x55afc33df930 2024-09-21T03:43:05.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.805+0000 7fd35b7dff40 1 -- 172.21.15.38:0/4050539643 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e124) -- 0x55afc33e70c0 con 0x55afc33df930 2024-09-21T03:43:05.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.806+0000 7fd355a9a640 1 -- 172.21.15.38:0/4050539643 <== osd.3 v2:172.21.15.38:6808/1286810294 17 ==== osd_op_reply(102 mds0_sessionmap [delete] v124'121 uv121 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fd35000b040 con 0x55afc33df930 2024-09-21T03:43:05.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.806+0000 7fd35b7dff40 1 -- 172.21.15.38:0/4050539643 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:103 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55afc33e70c0 con 0x55afc33df930 2024-09-21T03:43:05.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.807+0000 7fd355a9a640 1 -- 172.21.15.38:0/4050539643 <== osd.3 v2:172.21.15.38:6808/1286810294 18 ==== osd_op_reply(103 mds0_sessionmap [create] v124'122 uv122 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fd35000b040 con 0x55afc33df930 2024-09-21T03:43:05.809 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-09-21T03:43:05.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.807+0000 7fd35b7dff40 1 -- 172.21.15.38:0/4050539643 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:104 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55afc33e70c0 con 0x55afc33df930 2024-09-21T03:43:05.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.808+0000 7fd355a9a640 1 -- 172.21.15.38:0/4050539643 <== osd.3 v2:172.21.15.38:6808/1286810294 19 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v124'123 uv123 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fd35000b040 con 0x55afc33df930 2024-09-21T03:43:05.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.808+0000 7fd35b7dff40 1 -- 172.21.15.38:0/4050539643 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55afc33e70c0 con 0x55afc33df930 2024-09-21T03:43:05.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.810+0000 7fd355a9a640 1 -- 172.21.15.38:0/4050539643 <== osd.3 v2:172.21.15.38:6808/1286810294 20 ==== osd_op_reply(105 609.00000000 [create] v124'165 uv164 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fd35000b040 con 0x55afc33df930 2024-09-21T03:43:05.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.810+0000 7fd35b7dff40 1 -- 172.21.15.38:0/4050539643 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e124) -- 0x55afc33e70c0 con 0x55afc33df930 2024-09-21T03:43:05.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.811+0000 7fd355a9a640 1 -- 172.21.15.38:0/4050539643 <== osd.3 v2:172.21.15.38:6808/1286810294 21 ==== osd_op_reply(106 609.00000000 [delete] v124'166 uv166 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd35000b040 con 0x55afc33df930 2024-09-21T03:43:05.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.811+0000 7fd35b7dff40 1 -- 172.21.15.38:0/4050539643 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55afc33e70c0 con 0x55afc33df930 2024-09-21T03:43:05.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.812+0000 7fd355a9a640 1 -- 172.21.15.38:0/4050539643 <== osd.3 v2:172.21.15.38:6808/1286810294 22 ==== osd_op_reply(107 609.00000000 [create] v124'167 uv167 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd35000b040 con 0x55afc33df930 2024-09-21T03:43:05.814 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-09-21T03:43:05.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.812+0000 7fd35b7dff40 1 -- 172.21.15.38:0/4050539643 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55afc33e80d0 con 0x55afc33df930 2024-09-21T03:43:05.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.813+0000 7fd355a9a640 1 -- 172.21.15.38:0/4050539643 <== osd.3 v2:172.21.15.38:6808/1286810294 23 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v124'168 uv168 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd35000b040 con 0x55afc33df930 2024-09-21T03:43:05.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.813+0000 7fd35b7dff40 1 -- 172.21.15.38:0/4050539643 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:109 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55afc33e80d0 con 0x55afc33df930 2024-09-21T03:43:05.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.814+0000 7fd355a9a640 1 -- 172.21.15.38:0/4050539643 <== osd.3 v2:172.21.15.38:6808/1286810294 24 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v124'169 uv169 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd35000b040 con 0x55afc33df930 2024-09-21T03:43:05.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.814+0000 7fd35b7dff40 1 -- 172.21.15.38:0/4050539643 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:110 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55afc33e70c0 con 0x55afc33df930 2024-09-21T03:43:05.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.815+0000 7fd355a9a640 1 -- 172.21.15.38:0/4050539643 <== osd.3 v2:172.21.15.38:6808/1286810294 25 ==== osd_op_reply(110 609.00000000 [omap-set-header] v124'170 uv170 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd35000b040 con 0x55afc33df930 2024-09-21T03:43:05.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.815+0000 7fd35b7dff40 1 --2- 172.21.15.38:0/4050539643 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x55afc33e7990 0x55afc33e7da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:05.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.815+0000 7fd35b7dff40 1 -- 172.21.15.38:0/4050539643 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55afc33e70c0 con 0x55afc33e7990 2024-09-21T03:43:05.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.815+0000 7fd354a98640 1 --2- 172.21.15.38:0/4050539643 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x55afc33e7990 0x55afc33e7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:05.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.816+0000 7fd354a98640 1 --2- 172.21.15.38:0/4050539643 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x55afc33e7990 0x55afc33e7da0 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:05.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.817+0000 7fd354a98640 1 -- 172.21.15.38:0/4050539643 <== osd.2 v2:172.21.15.38:6824/2577858902 1 ==== osd_op_reply(111 400.00000000 [create] v124'140 uv139 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fd34c05ec10 con 0x55afc33e7990 2024-09-21T03:43:05.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.817+0000 7fd35b7dff40 1 -- 172.21.15.38:0/4050539643 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e124) -- 0x55afc33e70c0 con 0x55afc33e7990 2024-09-21T03:43:05.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.818+0000 7fd354a98640 1 -- 172.21.15.38:0/4050539643 <== osd.2 v2:172.21.15.38:6824/2577858902 2 ==== osd_op_reply(112 400.00000000 [delete] v124'141 uv141 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd34c05ec10 con 0x55afc33e7990 2024-09-21T03:43:05.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.818+0000 7fd35b7dff40 1 -- 172.21.15.38:0/4050539643 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55afc33e70c0 con 0x55afc33e7990 2024-09-21T03:43:05.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.819+0000 7fd354a98640 1 -- 172.21.15.38:0/4050539643 <== osd.2 v2:172.21.15.38:6824/2577858902 3 ==== osd_op_reply(113 400.00000000 [create] v124'142 uv142 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd34c05ec10 con 0x55afc33e7990 2024-09-21T03:43:05.821 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-09-21T03:43:05.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.819+0000 7fd35b7dff40 1 -- 172.21.15.38:0/4050539643 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:114 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55afc33e70c0 con 0x55afc33e7990 2024-09-21T03:43:05.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.821+0000 7fd354a98640 1 -- 172.21.15.38:0/4050539643 <== osd.2 v2:172.21.15.38:6824/2577858902 4 ==== osd_op_reply(114 400.00000000 [write 0~22] v124'143 uv143 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd34c05ec10 con 0x55afc33e7990 2024-09-21T03:43:05.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.821+0000 7fd35b7dff40 1 -- 172.21.15.38:0/4050539643 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:115 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55afc33e70c0 con 0x55afc33e7990 2024-09-21T03:43:05.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.822+0000 7fd354a98640 1 -- 172.21.15.38:0/4050539643 <== osd.2 v2:172.21.15.38:6824/2577858902 5 ==== osd_op_reply(115 400.00000000 [omap-set-header] v124'144 uv144 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd34c05ec10 con 0x55afc33e7990 2024-09-21T03:43:05.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.822+0000 7fd35b7dff40 1 -- 172.21.15.38:0/4050539643 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:116 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55afc33e70c0 con 0x55afc33e3e50 2024-09-21T03:43:05.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.823+0000 7fd355299640 1 -- 172.21.15.38:0/4050539643 <== osd.1 v2:172.21.15.38:6816/1491818732 17 ==== osd_op_reply(116 100.00000000.inode [create] v124'339 uv331 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7fd348012600 con 0x55afc33e3e50 2024-09-21T03:43:05.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.823+0000 7fd35b7dff40 1 -- 172.21.15.38:0/4050539643 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:117 13.1f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e124) -- 0x55afc33e70c0 con 0x55afc33e3e50 2024-09-21T03:43:05.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.824+0000 7fd355299640 1 -- 172.21.15.38:0/4050539643 <== osd.1 v2:172.21.15.38:6816/1491818732 18 ==== osd_op_reply(117 100.00000000.inode [delete] v124'340 uv340 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fd348012600 con 0x55afc33e3e50 2024-09-21T03:43:05.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.824+0000 7fd35b7dff40 1 -- 172.21.15.38:0/4050539643 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:118 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55afc33e70c0 con 0x55afc33e3e50 2024-09-21T03:43:05.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.825+0000 7fd355299640 1 -- 172.21.15.38:0/4050539643 <== osd.1 v2:172.21.15.38:6816/1491818732 19 ==== osd_op_reply(118 100.00000000.inode [create] v124'341 uv341 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fd348012600 con 0x55afc33e3e50 2024-09-21T03:43:05.827 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-09-21T03:43:05.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.825+0000 7fd35b7dff40 1 -- 172.21.15.38:0/4050539643 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:119 13.1f 13:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55afc33e70c0 con 0x55afc33e3e50 2024-09-21T03:43:05.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.827+0000 7fd355299640 1 -- 172.21.15.38:0/4050539643 <== osd.1 v2:172.21.15.38:6816/1491818732 20 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v124'342 uv342 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fd348012600 con 0x55afc33e3e50 2024-09-21T03:43:05.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.827+0000 7fd35b7dff40 1 -- 172.21.15.38:0/4050539643 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:120 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55afc33e70c0 con 0x55afc33e3e50 2024-09-21T03:43:05.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.828+0000 7fd355299640 1 -- 172.21.15.38:0/4050539643 <== osd.1 v2:172.21.15.38:6816/1491818732 21 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v124'343 uv343 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fd348012600 con 0x55afc33e3e50 2024-09-21T03:43:05.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.828+0000 7fd35b7dff40 1 -- 172.21.15.38:0/4050539643 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:121 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55afc33e70c0 con 0x55afc33e3e50 2024-09-21T03:43:05.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.829+0000 7fd355299640 1 -- 172.21.15.38:0/4050539643 <== osd.1 v2:172.21.15.38:6816/1491818732 22 ==== osd_op_reply(121 1.00000000 [create] v124'344 uv338 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7fd348012600 con 0x55afc33e3e50 2024-09-21T03:43:05.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.829+0000 7fd35b7dff40 1 -- 172.21.15.38:0/4050539643 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:122 13.1f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e124) -- 0x55afc33e70c0 con 0x55afc33e3e50 2024-09-21T03:43:05.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.830+0000 7fd355299640 1 -- 172.21.15.38:0/4050539643 <== osd.1 v2:172.21.15.38:6816/1491818732 23 ==== osd_op_reply(122 1.00000000 [delete] v124'345 uv345 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fd348012600 con 0x55afc33e3e50 2024-09-21T03:43:05.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.830+0000 7fd35b7dff40 1 -- 172.21.15.38:0/4050539643 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:123 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55afc33e70c0 con 0x55afc33e3e50 2024-09-21T03:43:05.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.831+0000 7fd355299640 1 -- 172.21.15.38:0/4050539643 <== osd.1 v2:172.21.15.38:6816/1491818732 24 ==== osd_op_reply(123 1.00000000 [create] v124'346 uv346 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fd348012600 con 0x55afc33e3e50 2024-09-21T03:43:05.833 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-09-21T03:43:05.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.831+0000 7fd35b7dff40 1 -- 172.21.15.38:0/4050539643 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:124 13.1f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55afc33e70c0 con 0x55afc33e3e50 2024-09-21T03:43:05.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.832+0000 7fd355299640 1 -- 172.21.15.38:0/4050539643 <== osd.1 v2:172.21.15.38:6816/1491818732 25 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v124'347 uv347 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fd348012600 con 0x55afc33e3e50 2024-09-21T03:43:05.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.832+0000 7fd35b7dff40 1 -- 172.21.15.38:0/4050539643 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:125 13.1f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55afc33e70c0 con 0x55afc33e3e50 2024-09-21T03:43:05.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.834+0000 7fd355299640 1 -- 172.21.15.38:0/4050539643 <== osd.1 v2:172.21.15.38:6816/1491818732 26 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v124'348 uv348 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fd348012600 con 0x55afc33e3e50 2024-09-21T03:43:05.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.834+0000 7fd35b7dff40 1 -- 172.21.15.38:0/4050539643 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:126 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55afc33e70c0 con 0x55afc33e3e50 2024-09-21T03:43:05.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.835+0000 7fd355299640 1 -- 172.21.15.38:0/4050539643 <== osd.1 v2:172.21.15.38:6816/1491818732 27 ==== osd_op_reply(126 1.00000000 [omap-set-header] v124'349 uv349 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fd348012600 con 0x55afc33e3e50 2024-09-21T03:43:05.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.835+0000 7fd35b7dff40 1 -- 172.21.15.38:0/4050539643 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:127 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=584b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55afc33e85a0 con 0x55afc33e3e50 2024-09-21T03:43:05.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.836+0000 7fd355299640 1 -- 172.21.15.38:0/4050539643 <== osd.1 v2:172.21.15.38:6816/1491818732 28 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v124'350 uv350 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fd348012600 con 0x55afc33e3e50 2024-09-21T03:43:05.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.836+0000 7fd35b7dff40 1 -- 172.21.15.38:0/4050539643 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x55afc33dba80 msgr2=0x55afc33ddea0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:05.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.836+0000 7fd35b7dff40 1 --2- 172.21.15.38:0/4050539643 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x55afc33dba80 0x55afc33ddea0 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:05.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.836+0000 7fd35b7dff40 1 -- 172.21.15.38:0/4050539643 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x55afc33e3e50 msgr2=0x55afc33e6270 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:05.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.836+0000 7fd35b7dff40 1 --2- 172.21.15.38:0/4050539643 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x55afc33e3e50 0x55afc33e6270 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:05.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.836+0000 7fd35b7dff40 1 -- 172.21.15.38:0/4050539643 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x55afc33e7990 msgr2=0x55afc33e7da0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:05.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.836+0000 7fd35b7dff40 1 --2- 172.21.15.38:0/4050539643 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x55afc33e7990 0x55afc33e7da0 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:05.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.836+0000 7fd35b7dff40 1 -- 172.21.15.38:0/4050539643 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x55afc33df930 msgr2=0x55afc33e27b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:05.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.836+0000 7fd35b7dff40 1 --2- 172.21.15.38:0/4050539643 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x55afc33df930 0x55afc33e27b0 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:05.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.836+0000 7fd35b7dff40 1 -- 172.21.15.38:0/4050539643 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x55afc33d6e00 msgr2=0x55afc33d9220 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:05.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.836+0000 7fd35b7dff40 1 --2- 172.21.15.38:0/4050539643 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x55afc33d6e00 0x55afc33d9220 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:05.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.836+0000 7fd355a9a640 1 -- 172.21.15.38:0/4050539643 reap_dead start 2024-09-21T03:43:05.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.836+0000 7fd35b7dff40 1 -- 172.21.15.38:0/4050539643 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x55afc332e170 msgr2=0x55afc332a910 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:05.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.836+0000 7fd35b7dff40 1 --2- 172.21.15.38:0/4050539643 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x55afc332e170 0x55afc332a910 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:05.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.836+0000 7fd355a9a640 1 -- 172.21.15.38:0/4050539643 reap_dead start 2024-09-21T03:43:05.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.836+0000 7fd355a9a640 1 -- 172.21.15.38:0/4050539643 reap_dead start 2024-09-21T03:43:05.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.836+0000 7fd35b7dff40 1 -- 172.21.15.38:0/4050539643 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd338077570 msgr2=0x7fd338079a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:05.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.836+0000 7fd35b7dff40 1 --2- 172.21.15.38:0/4050539643 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd338077570 0x7fd338079a80 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fd3480012a0 tx=0x7fd348001210 comp rx=0 tx=0).stop 2024-09-21T03:43:05.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.836+0000 7fd35b7dff40 1 -- 172.21.15.38:0/4050539643 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55afc32a5090 msgr2=0x55afc3317b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:05.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.836+0000 7fd35b7dff40 1 --2- 172.21.15.38:0/4050539643 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55afc32a5090 0x55afc3317b70 secure :-1 s=READY pgs=1924 cs=0 l=1 rev1=1 crypto rx=0x7fd34c000f60 tx=0x7fd34c00fdd0 comp rx=0 tx=0).stop 2024-09-21T03:43:05.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.836+0000 7fd35b7dff40 1 -- 172.21.15.38:0/4050539643 shutdown_connections 2024-09-21T03:43:05.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.836+0000 7fd35b7dff40 1 --2- 172.21.15.38:0/4050539643 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55afc32a5090 0x55afc3317b70 unknown :-1 s=CLOSED pgs=1924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:05.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.836+0000 7fd35b7dff40 1 --2- 172.21.15.38:0/4050539643 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd338077570 0x7fd338079a80 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:05.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.836+0000 7fd35b7dff40 1 -- 172.21.15.38:0/4050539643 >> 172.21.15.38:0/4050539643 conn(0x55afc317cac0 msgr2=0x55afc3318cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:05.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.836+0000 7fd35b7dff40 1 -- 172.21.15.38:0/4050539643 shutdown_connections 2024-09-21T03:43:05.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.836+0000 7fd35b7dff40 1 -- 172.21.15.38:0/4050539643 wait complete. 2024-09-21T03:43:05.841 DEBUG:teuthology.orchestra.run.smithi038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata truncate mds0_inotable 0 2024-09-21T03:43:05.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.937+0000 7f0e20676f40 1 Processor -- start 2024-09-21T03:43:05.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.937+0000 7f0e20676f40 1 -- start start 2024-09-21T03:43:05.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.938+0000 7f0e20676f40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a68d20dd00 0x55a68d2df420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:05.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.938+0000 7f0e20676f40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a68d2c8950 0x55a68d23b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:05.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.938+0000 7f0e20676f40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a68d2e3670 0x55a68d237740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:05.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.938+0000 7f0e20676f40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55a68d15bd60 con 0x55a68d20dd00 2024-09-21T03:43:05.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.938+0000 7f0e20676f40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55a68d1ba430 con 0x55a68d2c8950 2024-09-21T03:43:05.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.938+0000 7f0e20676f40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55a68d2df960 con 0x55a68d2e3670 2024-09-21T03:43:05.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.938+0000 7f0e1a130640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a68d20dd00 0x55a68d2df420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:05.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.939+0000 7f0e1a130640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a68d20dd00 0x55a68d2df420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44954/0 (socket says 172.21.15.38:44954) 2024-09-21T03:43:05.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.939+0000 7f0e1a130640 1 -- 172.21.15.38:0/3529121087 learned_addr learned my addr 172.21.15.38:0/3529121087 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:43:05.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.939+0000 7f0e1a931640 1 --2- 172.21.15.38:0/3529121087 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a68d2e3670 0x55a68d237740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:05.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.939+0000 7f0e1992f640 1 --2- 172.21.15.38:0/3529121087 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a68d2c8950 0x55a68d23b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:05.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.939+0000 7f0e1a130640 1 -- 172.21.15.38:0/3529121087 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a68d2e3670 msgr2=0x55a68d237740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:05.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.939+0000 7f0e1a130640 1 --2- 172.21.15.38:0/3529121087 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a68d2e3670 0x55a68d237740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:05.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.939+0000 7f0e1a130640 1 -- 172.21.15.38:0/3529121087 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a68d2c8950 msgr2=0x55a68d23b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:05.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.939+0000 7f0e1a130640 1 --2- 172.21.15.38:0/3529121087 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a68d2c8950 0x55a68d23b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:05.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.939+0000 7f0e1a130640 1 -- 172.21.15.38:0/3529121087 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a68d2e06c0 con 0x55a68d20dd00 2024-09-21T03:43:05.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.939+0000 7f0e1a931640 1 --2- 172.21.15.38:0/3529121087 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a68d2e3670 0x55a68d237740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:43:05.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.939+0000 7f0e1992f640 1 --2- 172.21.15.38:0/3529121087 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a68d2c8950 0x55a68d23b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:43:05.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.939+0000 7f0e1a130640 1 --2- 172.21.15.38:0/3529121087 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a68d20dd00 0x55a68d2df420 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 crypto rx=0x7f0e0c00a550 tx=0x7f0e0c003330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6cd3ade3c755675 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:05.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.939+0000 7f0e1892d640 1 -- 172.21.15.38:0/3529121087 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e0c064030 con 0x55a68d20dd00 2024-09-21T03:43:05.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.939+0000 7f0e1892d640 1 -- 172.21.15.38:0/3529121087 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0e0c004020 con 0x55a68d20dd00 2024-09-21T03:43:05.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.939+0000 7f0e1892d640 1 -- 172.21.15.38:0/3529121087 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e0c004290 con 0x55a68d20dd00 2024-09-21T03:43:05.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.940+0000 7f0e20676f40 1 -- 172.21.15.38:0/3529121087 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a68d20dd00 msgr2=0x55a68d2df420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:05.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.940+0000 7f0e20676f40 1 --2- 172.21.15.38:0/3529121087 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a68d20dd00 0x55a68d2df420 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 crypto rx=0x7f0e0c00a550 tx=0x7f0e0c003330 comp rx=0 tx=0).stop 2024-09-21T03:43:05.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.940+0000 7f0e20676f40 1 -- 172.21.15.38:0/3529121087 shutdown_connections 2024-09-21T03:43:05.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.940+0000 7f0e20676f40 1 --2- 172.21.15.38:0/3529121087 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a68d2e3670 0x55a68d237740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:05.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.940+0000 7f0e20676f40 1 --2- 172.21.15.38:0/3529121087 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a68d2c8950 0x55a68d23b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:05.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.940+0000 7f0e20676f40 1 --2- 172.21.15.38:0/3529121087 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a68d20dd00 0x55a68d2df420 unknown :-1 s=CLOSED pgs=1925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:05.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.940+0000 7f0e20676f40 1 -- 172.21.15.38:0/3529121087 >> 172.21.15.38:0/3529121087 conn(0x55a68d12cac0 msgr2=0x55a68d2e3a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:05.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.940+0000 7f0e20676f40 1 -- 172.21.15.38:0/3529121087 shutdown_connections 2024-09-21T03:43:05.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.940+0000 7f0e20676f40 1 -- 172.21.15.38:0/3529121087 wait complete. 2024-09-21T03:43:05.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.941+0000 7f0e20676f40 1 Processor -- start 2024-09-21T03:43:05.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.941+0000 7f0e20676f40 1 -- start start 2024-09-21T03:43:05.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.942+0000 7f0e20676f40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a68d20dd00 0x55a68d2c0180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:05.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.942+0000 7f0e20676f40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a68d2c8950 0x55a68d23aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:05.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.942+0000 7f0e20676f40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a68d2e3670 0x55a68d237650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:05.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.942+0000 7f0e20676f40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55a68d2c43e0 con 0x55a68d2e3670 2024-09-21T03:43:05.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.942+0000 7f0e20676f40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55a68d2c3cf0 con 0x55a68d20dd00 2024-09-21T03:43:05.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.942+0000 7f0e20676f40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55a68d2dfbd0 con 0x55a68d2c8950 2024-09-21T03:43:05.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.942+0000 7f0e1a931640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a68d2e3670 0x55a68d237650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:05.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.942+0000 7f0e1a931640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a68d2e3670 0x55a68d237650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44958/0 (socket says 172.21.15.38:44958) 2024-09-21T03:43:05.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.942+0000 7f0e1a931640 1 -- 172.21.15.38:0/1543361364 learned_addr learned my addr 172.21.15.38:0/1543361364 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:43:05.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.942+0000 7f0e1a130640 1 --2- 172.21.15.38:0/1543361364 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a68d20dd00 0x55a68d2c0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:05.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.942+0000 7f0e1992f640 1 --2- 172.21.15.38:0/1543361364 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a68d2c8950 0x55a68d23aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:05.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.943+0000 7f0e1a931640 1 -- 172.21.15.38:0/1543361364 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a68d2c8950 msgr2=0x55a68d23aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:05.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.943+0000 7f0e1a931640 1 --2- 172.21.15.38:0/1543361364 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a68d2c8950 0x55a68d23aee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:05.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.943+0000 7f0e1a931640 1 -- 172.21.15.38:0/1543361364 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a68d20dd00 msgr2=0x55a68d2c0180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:05.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.943+0000 7f0e1a931640 1 --2- 172.21.15.38:0/1543361364 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a68d20dd00 0x55a68d2c0180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:05.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.943+0000 7f0e1a931640 1 -- 172.21.15.38:0/1543361364 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a68d37a650 con 0x55a68d2e3670 2024-09-21T03:43:05.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.943+0000 7f0e1a931640 1 --2- 172.21.15.38:0/1543361364 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a68d2e3670 0x55a68d237650 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x7f0e1400ef50 tx=0x7f0e1400c620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:05.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.943+0000 7f0e137fe640 1 -- 172.21.15.38:0/1543361364 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e14015020 con 0x55a68d2e3670 2024-09-21T03:43:05.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.943+0000 7f0e137fe640 1 -- 172.21.15.38:0/1543361364 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0e14002e50 con 0x55a68d2e3670 2024-09-21T03:43:05.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.943+0000 7f0e20676f40 1 -- 172.21.15.38:0/1543361364 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55a68d2c36e0 con 0x55a68d2e3670 2024-09-21T03:43:05.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.943+0000 7f0e137fe640 1 -- 172.21.15.38:0/1543361364 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e140118f0 con 0x55a68d2e3670 2024-09-21T03:43:05.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.943+0000 7f0e20676f40 1 -- 172.21.15.38:0/1543361364 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a68d2c18a0 con 0x55a68d2e3670 2024-09-21T03:43:05.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.945+0000 7f0e137fe640 1 -- 172.21.15.38:0/1543361364 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f0e14018020 con 0x55a68d2e3670 2024-09-21T03:43:05.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.945+0000 7f0e137fe640 1 --2- 172.21.15.38:0/1543361364 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0dfc0775a0 0x7f0dfc079ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:05.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.945+0000 7f0e137fe640 1 -- 172.21.15.38:0/1543361364 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 8712+0+0 (secure 0 0 0) 0x7f0e1409a360 con 0x55a68d2e3670 2024-09-21T03:43:05.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.946+0000 7f0e20676f40 1 --2- 172.21.15.38:0/1543361364 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x55a68d2c1dc0 0x55a68d2c21d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:05.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.946+0000 7f0e20676f40 1 -- 172.21.15.38:0/1543361364 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:1 13.13 13:c91d4a1d:::mds0_inotable:head [truncate 0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55a68d2c2710 con 0x55a68d2c1dc0 2024-09-21T03:43:05.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.946+0000 7f0e1a130640 1 --2- 172.21.15.38:0/1543361364 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0dfc0775a0 0x7f0dfc079ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:05.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.946+0000 7f0e1992f640 1 --2- 172.21.15.38:0/1543361364 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x55a68d2c1dc0 0x55a68d2c21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:05.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.946+0000 7f0e1992f640 1 --2- 172.21.15.38:0/1543361364 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x55a68d2c1dc0 0x55a68d2c21d0 crc :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:05.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.946+0000 7f0e1a130640 1 --2- 172.21.15.38:0/1543361364 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0dfc0775a0 0x7f0dfc079ab0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f0e0c057040 tx=0x7f0e0c004c00 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:05.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.947+0000 7f0e1992f640 1 -- 172.21.15.38:0/1543361364 <== osd.1 v2:172.21.15.38:6816/1491818732 1 ==== osd_op_reply(1 mds0_inotable [truncate 0] v124'283 uv283 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f0e0400eb40 con 0x55a68d2c1dc0 2024-09-21T03:43:05.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.947+0000 7f0e20676f40 1 -- 172.21.15.38:0/1543361364 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x55a68d2c1dc0 msgr2=0x55a68d2c21d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:05.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.947+0000 7f0e20676f40 1 --2- 172.21.15.38:0/1543361364 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x55a68d2c1dc0 0x55a68d2c21d0 crc :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:05.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.947+0000 7f0e20676f40 1 -- 172.21.15.38:0/1543361364 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0dfc0775a0 msgr2=0x7f0dfc079ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:05.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.947+0000 7f0e20676f40 1 --2- 172.21.15.38:0/1543361364 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0dfc0775a0 0x7f0dfc079ab0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f0e0c057040 tx=0x7f0e0c004c00 comp rx=0 tx=0).stop 2024-09-21T03:43:05.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.947+0000 7f0e20676f40 1 -- 172.21.15.38:0/1543361364 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a68d2e3670 msgr2=0x55a68d237650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:05.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.947+0000 7f0e20676f40 1 --2- 172.21.15.38:0/1543361364 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a68d2e3670 0x55a68d237650 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x7f0e1400ef50 tx=0x7f0e1400c620 comp rx=0 tx=0).stop 2024-09-21T03:43:05.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.947+0000 7f0e1a931640 1 -- 172.21.15.38:0/1543361364 reap_dead start 2024-09-21T03:43:05.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.947+0000 7f0e20676f40 1 -- 172.21.15.38:0/1543361364 shutdown_connections 2024-09-21T03:43:05.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.947+0000 7f0e20676f40 1 -- 172.21.15.38:0/1543361364 >> 172.21.15.38:0/1543361364 conn(0x55a68d12cac0 msgr2=0x55a68d23b8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:05.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.948+0000 7f0e20676f40 1 -- 172.21.15.38:0/1543361364 shutdown_connections 2024-09-21T03:43:05.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:05.948+0000 7f0e20676f40 1 -- 172.21.15.38:0/1543361364 wait complete. 2024-09-21T03:43:05.952 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-21T03:43:06.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:06.069+0000 7fc6c7107640 1 -- 172.21.15.38:0/2366455085 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc6c015ee80 msgr2=0x7fc6c00d0170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:06.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:06.069+0000 7fc6c7107640 1 --2- 172.21.15.38:0/2366455085 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc6c015ee80 0x7fc6c00d0170 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fc6a8009830 tx=0x7fc6a802a480 comp rx=0 tx=0).stop 2024-09-21T03:43:06.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:06.070+0000 7fc6c7107640 1 -- 172.21.15.38:0/2366455085 shutdown_connections 2024-09-21T03:43:06.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:06.070+0000 7fc6c7107640 1 --2- 172.21.15.38:0/2366455085 >> v2:172.21.15.195:3300/0 conn(0x7fc6c00b9ad0 0x7fc6c00d0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:06.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:06.070+0000 7fc6c7107640 1 --2- 172.21.15.38:0/2366455085 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc6c015ee80 0x7fc6c00d0170 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:06.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:06.070+0000 7fc6c7107640 1 -- 172.21.15.38:0/2366455085 >> 172.21.15.38:0/2366455085 conn(0x7fc6c009d830 msgr2=0x7fc6c00b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:06.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:06.070+0000 7fc6c7107640 1 -- 172.21.15.38:0/2366455085 shutdown_connections 2024-09-21T03:43:06.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:06.070+0000 7fc6c7107640 1 -- 172.21.15.38:0/2366455085 wait complete. 2024-09-21T03:43:06.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:06.071+0000 7fc6c7107640 1 Processor -- start 2024-09-21T03:43:06.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:06.072+0000 7fc6c7107640 1 -- start start 2024-09-21T03:43:06.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:06.073+0000 7fc6c7107640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc6c00b9ad0 0x7fc6c006e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:06.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:06.073+0000 7fc6c7107640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc6c00d1120 0x7fc6c015b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:06.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:06.073+0000 7fc6c7107640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc6c015ee80 0x7fc6c00cbe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:06.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:06.073+0000 7fc6c7107640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fc6c00b6700 con 0x7fc6c00b9ad0 2024-09-21T03:43:06.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:06.073+0000 7fc6c7107640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fc6c00b6580 con 0x7fc6c00d1120 2024-09-21T03:43:06.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:06.073+0000 7fc6c7107640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fc6c00b6880 con 0x7fc6c015ee80 2024-09-21T03:43:06.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:06.073+0000 7fc6c4ea3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc6c00b9ad0 0x7fc6c006e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:06.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:06.073+0000 7fc6c4ea3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc6c00b9ad0 0x7fc6c006e540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44970/0 (socket says 172.21.15.38:44970) 2024-09-21T03:43:06.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:06.073+0000 7fc6c4ea3640 1 -- 172.21.15.38:0/918839133 learned_addr learned my addr 172.21.15.38:0/918839133 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:43:06.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:06.073+0000 7fc6bffff640 1 --2- 172.21.15.38:0/918839133 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc6c00d1120 0x7fc6c015b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:06.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:06.073+0000 7fc6c56a4640 1 --2- 172.21.15.38:0/918839133 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc6c015ee80 0x7fc6c00cbe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:06.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:06.073+0000 7fc6c4ea3640 1 -- 172.21.15.38:0/918839133 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc6c015ee80 msgr2=0x7fc6c00cbe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:06.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:06.074+0000 7fc6c4ea3640 1 --2- 172.21.15.38:0/918839133 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc6c015ee80 0x7fc6c00cbe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:06.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:06.074+0000 7fc6c4ea3640 1 -- 172.21.15.38:0/918839133 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc6c00d1120 msgr2=0x7fc6c015b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:06.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:06.074+0000 7fc6c4ea3640 1 --2- 172.21.15.38:0/918839133 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc6c00d1120 0x7fc6c015b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:06.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:06.074+0000 7fc6c4ea3640 1 -- 172.21.15.38:0/918839133 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6c006d4c0 con 0x7fc6c00b9ad0 2024-09-21T03:43:06.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:06.074+0000 7fc6c4ea3640 1 --2- 172.21.15.38:0/918839133 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc6c00b9ad0 0x7fc6c006e540 secure :-1 s=READY pgs=1927 cs=0 l=1 rev1=1 crypto rx=0x7fc6a8009950 tx=0x7fc6a8033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:06.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:06.074+0000 7fc6bd7fa640 1 -- 172.21.15.38:0/918839133 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6a8005c90 con 0x7fc6c00b9ad0 2024-09-21T03:43:06.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:06.074+0000 7fc6bd7fa640 1 -- 172.21.15.38:0/918839133 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc6a8031e50 con 0x7fc6c00b9ad0 2024-09-21T03:43:06.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:06.074+0000 7fc6bd7fa640 1 -- 172.21.15.38:0/918839133 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6a8005260 con 0x7fc6c00b9ad0 2024-09-21T03:43:06.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:06.074+0000 7fc6c7107640 1 -- 172.21.15.38:0/918839133 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6c006d7a0 con 0x7fc6c00b9ad0 2024-09-21T03:43:06.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:06.074+0000 7fc6c7107640 1 -- 172.21.15.38:0/918839133 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6c006dd60 con 0x7fc6c00b9ad0 2024-09-21T03:43:06.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:06.075+0000 7fc6c7107640 1 -- 172.21.15.38:0/918839133 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6c00b6d30 con 0x7fc6c00b9ad0 2024-09-21T03:43:06.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:06.076+0000 7fc6bd7fa640 1 -- 172.21.15.38:0/918839133 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc6a8005400 con 0x7fc6c00b9ad0 2024-09-21T03:43:06.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:06.076+0000 7fc6bd7fa640 1 --2- 172.21.15.38:0/918839133 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc6a0078b90 0x7fc6a007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:06.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:06.077+0000 7fc6bd7fa640 1 -- 172.21.15.38:0/918839133 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 8712+0+0 (secure 0 0 0) 0x7fc6a80bc8a0 con 0x7fc6c00b9ad0 2024-09-21T03:43:06.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:06.077+0000 7fc6bffff640 1 --2- 172.21.15.38:0/918839133 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc6a0078b90 0x7fc6a007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:06.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:06.077+0000 7fc6bffff640 1 --2- 172.21.15.38:0/918839133 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc6a0078b90 0x7fc6a007b0a0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fc6b0009590 tx=0x7fc6b0005f30 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:06.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:06.081+0000 7fc6bd7fa640 1 -- 172.21.15.38:0/918839133 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fc6a803f030 con 0x7fc6c00b9ad0 2024-09-21T03:43:06.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:06.306+0000 7fc6c7107640 1 -- 172.21.15.38:0/918839133 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7fc6c00cb0e0 con 0x7fc6c00b9ad0 2024-09-21T03:43:06.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:06.588+0000 7fc6bd7fa640 1 -- 172.21.15.38:0/918839133 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v254) ==== 162+0+0 (secure 0 0 0) 0x7fc6a8083ba0 con 0x7fc6c00b9ad0 2024-09-21T03:43:06.591 INFO:teuthology.orchestra.run.smithi038.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-21T03:43:06.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:06.591+0000 7fc6c7107640 1 -- 172.21.15.38:0/918839133 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc6a0078b90 msgr2=0x7fc6a007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:06.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:06.591+0000 7fc6c7107640 1 --2- 172.21.15.38:0/918839133 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc6a0078b90 0x7fc6a007b0a0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fc6b0009590 tx=0x7fc6b0005f30 comp rx=0 tx=0).stop 2024-09-21T03:43:06.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:06.591+0000 7fc6c7107640 1 -- 172.21.15.38:0/918839133 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc6c00b9ad0 msgr2=0x7fc6c006e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:06.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:06.591+0000 7fc6c7107640 1 --2- 172.21.15.38:0/918839133 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc6c00b9ad0 0x7fc6c006e540 secure :-1 s=READY pgs=1927 cs=0 l=1 rev1=1 crypto rx=0x7fc6a8009950 tx=0x7fc6a8033c20 comp rx=0 tx=0).stop 2024-09-21T03:43:06.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:06.591+0000 7fc6c7107640 1 -- 172.21.15.38:0/918839133 shutdown_connections 2024-09-21T03:43:06.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:06.591+0000 7fc6c7107640 1 --2- 172.21.15.38:0/918839133 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc6a0078b90 0x7fc6a007b0a0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:06.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:06.591+0000 7fc6c7107640 1 --2- 172.21.15.38:0/918839133 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc6c015ee80 0x7fc6c00cbe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:06.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:06.591+0000 7fc6c7107640 1 --2- 172.21.15.38:0/918839133 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc6c00d1120 0x7fc6c015b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:06.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:06.591+0000 7fc6c7107640 1 --2- 172.21.15.38:0/918839133 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc6c00b9ad0 0x7fc6c006e540 unknown :-1 s=CLOSED pgs=1927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:06.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:06.591+0000 7fc6c7107640 1 -- 172.21.15.38:0/918839133 >> 172.21.15.38:0/918839133 conn(0x7fc6c009d830 msgr2=0x7fc6c00ceb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:06.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:06.591+0000 7fc6c7107640 1 -- 172.21.15.38:0/918839133 shutdown_connections 2024-09-21T03:43:06.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:06.591+0000 7fc6c7107640 1 -- 172.21.15.38:0/918839133 wait complete. 2024-09-21T03:43:06.605 INFO:tasks.ceph.mds.a.smithi038.stderr:2024-09-21T03:43:06.603+0000 7f5302a41640 -1 log_channel(cluster) log [ERR] : error decoding table object 'mds0_inotable': End of buffer [buffer:2] 2024-09-21T03:43:06.606 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:43:06.606 INFO:tasks.ceph.mon.a.smithi038.stderr:2024-09-21T03:43:06.604+0000 7f3f45713640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T03:43:06.606 INFO:tasks.ceph.mon.a.smithi038.stderr:2024-09-21T03:43:06.604+0000 7f3f45713640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-09-21T03:43:06.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:06.722+0000 7f6de8007640 1 -- 172.21.15.38:0/1417998519 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6de00d1d00 msgr2=0x7f6de00cc930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:06.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:06.722+0000 7f6de8007640 1 --2- 172.21.15.38:0/1417998519 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6de00d1d00 0x7f6de00cc930 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 crypto rx=0x7f6de00b4ab0 tx=0x7f6dd002a530 comp rx=0 tx=0).stop 2024-09-21T03:43:06.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:06.723+0000 7f6de8007640 1 -- 172.21.15.38:0/1417998519 shutdown_connections 2024-09-21T03:43:06.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:06.723+0000 7f6de8007640 1 --2- 172.21.15.38:0/1417998519 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6de00d1d00 0x7f6de00cc930 unknown :-1 s=CLOSED pgs=1929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:06.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:06.723+0000 7f6de8007640 1 --2- 172.21.15.38:0/1417998519 >> v2:172.21.15.195:3300/0 conn(0x7f6de01536e0 0x7f6de00d17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:06.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:06.723+0000 7f6de8007640 1 -- 172.21.15.38:0/1417998519 >> 172.21.15.38:0/1417998519 conn(0x7f6de00cef40 msgr2=0x7f6de00cf350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:06.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:06.723+0000 7f6de8007640 1 -- 172.21.15.38:0/1417998519 shutdown_connections 2024-09-21T03:43:06.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:06.723+0000 7f6de8007640 1 -- 172.21.15.38:0/1417998519 wait complete. 2024-09-21T03:43:06.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:06.724+0000 7f6de8007640 1 Processor -- start 2024-09-21T03:43:06.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:06.724+0000 7f6de8007640 1 -- start start 2024-09-21T03:43:06.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:06.726+0000 7f6de8007640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6de00cce70 0x7f6de01f9f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:06.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:06.726+0000 7f6de8007640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6de00d1d00 0x7f6de00cc1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:06.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:06.726+0000 7f6de8007640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6de01536e0 0x7f6de015b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:06.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:06.726+0000 7f6de8007640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f6de01608a0 con 0x7f6de00cce70 2024-09-21T03:43:06.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:06.726+0000 7f6de8007640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f6de0160a20 con 0x7f6de01536e0 2024-09-21T03:43:06.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:06.726+0000 7f6de8007640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f6de0160ba0 con 0x7f6de00d1d00 2024-09-21T03:43:06.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:06.726+0000 7f6de5da3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6de00cce70 0x7f6de01f9f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:06.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:06.726+0000 7f6de5da3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6de00cce70 0x7f6de01f9f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44986/0 (socket says 172.21.15.38:44986) 2024-09-21T03:43:06.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:06.726+0000 7f6de5da3640 1 -- 172.21.15.38:0/2812912182 learned_addr learned my addr 172.21.15.38:0/2812912182 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:43:06.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:06.726+0000 7f6de5da3640 1 -- 172.21.15.38:0/2812912182 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6de00d1d00 msgr2=0x7f6de00cc1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:06.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:06.726+0000 7f6de65a4640 1 --2- 172.21.15.38:0/2812912182 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6de01536e0 0x7f6de015b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:06.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:06.726+0000 7f6de55a2640 1 --2- 172.21.15.38:0/2812912182 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6de00d1d00 0x7f6de00cc1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:06.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:06.726+0000 7f6de5da3640 1 --2- 172.21.15.38:0/2812912182 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6de00d1d00 0x7f6de00cc1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:06.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:06.726+0000 7f6de5da3640 1 -- 172.21.15.38:0/2812912182 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6de01536e0 msgr2=0x7f6de015b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:06.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:06.726+0000 7f6de5da3640 1 --2- 172.21.15.38:0/2812912182 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6de01536e0 0x7f6de015b5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:06.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:06.726+0000 7f6de5da3640 1 -- 172.21.15.38:0/2812912182 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6de00cc6f0 con 0x7f6de00cce70 2024-09-21T03:43:06.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:06.726+0000 7f6de55a2640 1 --2- 172.21.15.38:0/2812912182 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6de00d1d00 0x7f6de00cc1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:43:06.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:06.726+0000 7f6de65a4640 1 --2- 172.21.15.38:0/2812912182 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6de01536e0 0x7f6de015b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:43:06.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:06.726+0000 7f6de5da3640 1 --2- 172.21.15.38:0/2812912182 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6de00cce70 0x7f6de01f9f20 secure :-1 s=READY pgs=1930 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc00d780 tx=0x7f6ddc00dc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:06.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:06.726+0000 7f6dd67fc640 1 -- 172.21.15.38:0/2812912182 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ddc017070 con 0x7f6de00cce70 2024-09-21T03:43:06.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:06.726+0000 7f6dd67fc640 1 -- 172.21.15.38:0/2812912182 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6ddc00de90 con 0x7f6de00cce70 2024-09-21T03:43:06.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:06.726+0000 7f6de8007640 1 -- 172.21.15.38:0/2812912182 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6de0154f80 con 0x7f6de00cce70 2024-09-21T03:43:06.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:06.727+0000 7f6dd67fc640 1 -- 172.21.15.38:0/2812912182 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ddc0099d0 con 0x7f6de00cce70 2024-09-21T03:43:06.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:06.727+0000 7f6de8007640 1 -- 172.21.15.38:0/2812912182 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6de006e880 con 0x7f6de00cce70 2024-09-21T03:43:06.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:06.727+0000 7f6de8007640 1 -- 172.21.15.38:0/2812912182 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6de00cc930 con 0x7f6de00cce70 2024-09-21T03:43:06.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:06.728+0000 7f6dd67fc640 1 -- 172.21.15.38:0/2812912182 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f6ddc009b70 con 0x7f6de00cce70 2024-09-21T03:43:06.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:06.729+0000 7f6dd67fc640 1 --2- 172.21.15.38:0/2812912182 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6dc4078b90 0x7f6dc407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:06.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:06.729+0000 7f6dd67fc640 1 -- 172.21.15.38:0/2812912182 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 8798+0+0 (secure 0 0 0) 0x7f6ddc09bdd0 con 0x7f6de00cce70 2024-09-21T03:43:06.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:06.729+0000 7f6de55a2640 1 --2- 172.21.15.38:0/2812912182 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6dc4078b90 0x7f6dc407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:06.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:06.730+0000 7f6de55a2640 1 --2- 172.21.15.38:0/2812912182 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6dc4078b90 0x7f6dc407b0a0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f6dd0034000 tx=0x7f6dd0004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:06.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:06.733+0000 7f6dd67fc640 1 -- 172.21.15.38:0/2812912182 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f6ddc020020 con 0x7f6de00cce70 2024-09-21T03:43:06.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:06.961+0000 7f6de8007640 1 -- 172.21.15.38:0/2812912182 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6de0072000 con 0x7f6de00cce70 2024-09-21T03:43:06.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:06.962+0000 7f6dd67fc640 1 -- 172.21.15.38:0/2812912182 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 256 v256) ==== 95+0+3566 (secure 0 0 0) 0x7f6ddc09de00 con 0x7f6de00cce70 2024-09-21T03:43:06.964 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:43:06.964 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":256,"btime":"2024-09-21T03:43:06:604763+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6086,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/965972663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":965972663},{"type":"v1","addr":"172.21.15.195:6836","nonce":965972663}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6700,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/3176122334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":3176122334},{"type":"v1","addr":"172.21.15.195:6837","nonce":3176122334}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":256,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:40:02.319282+0000","modified":"2024-09-21T03:43:06.604692+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":125,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-21T03:43:06.964 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 256 2024-09-21T03:43:06.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:06.964+0000 7f6de8007640 1 -- 172.21.15.38:0/2812912182 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6dc4078b90 msgr2=0x7f6dc407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:06.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:06.964+0000 7f6de8007640 1 --2- 172.21.15.38:0/2812912182 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6dc4078b90 0x7f6dc407b0a0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f6dd0034000 tx=0x7f6dd0004730 comp rx=0 tx=0).stop 2024-09-21T03:43:06.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:06.964+0000 7f6de8007640 1 -- 172.21.15.38:0/2812912182 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6de00cce70 msgr2=0x7f6de01f9f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:06.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:06.964+0000 7f6de8007640 1 --2- 172.21.15.38:0/2812912182 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6de00cce70 0x7f6de01f9f20 secure :-1 s=READY pgs=1930 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc00d780 tx=0x7f6ddc00dc40 comp rx=0 tx=0).stop 2024-09-21T03:43:06.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:06.964+0000 7f6de8007640 1 -- 172.21.15.38:0/2812912182 shutdown_connections 2024-09-21T03:43:06.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:06.964+0000 7f6de8007640 1 --2- 172.21.15.38:0/2812912182 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6de01536e0 0x7f6de015b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:06.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:06.964+0000 7f6de8007640 1 --2- 172.21.15.38:0/2812912182 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6dc4078b90 0x7f6dc407b0a0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:06.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:06.964+0000 7f6de8007640 1 --2- 172.21.15.38:0/2812912182 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6de00d1d00 0x7f6de00cc1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:06.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:06.964+0000 7f6de8007640 1 --2- 172.21.15.38:0/2812912182 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6de00cce70 0x7f6de01f9f20 unknown :-1 s=CLOSED pgs=1930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:06.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:06.964+0000 7f6de8007640 1 -- 172.21.15.38:0/2812912182 >> 172.21.15.38:0/2812912182 conn(0x7f6de00cef40 msgr2=0x7f6de00cf350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:06.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:06.964+0000 7f6de8007640 1 -- 172.21.15.38:0/2812912182 shutdown_connections 2024-09-21T03:43:06.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:06.964+0000 7f6de8007640 1 -- 172.21.15.38:0/2812912182 wait complete. 2024-09-21T03:43:06.977 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-21T03:43:06.977 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:43:07.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:07.092+0000 7f666d48b640 1 -- 172.21.15.38:0/2153779827 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6668162280 msgr2=0x7f6668162690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:07.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:07.092+0000 7f666d48b640 1 --2- 172.21.15.38:0/2153779827 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6668162280 0x7f6668162690 secure :-1 s=READY pgs=1931 cs=0 l=1 rev1=1 crypto rx=0x7f66680d5040 tx=0x7f665c02a560 comp rx=0 tx=0).stop 2024-09-21T03:43:07.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:07.092+0000 7f666d48b640 1 -- 172.21.15.38:0/2153779827 shutdown_connections 2024-09-21T03:43:07.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:07.092+0000 7f666d48b640 1 --2- 172.21.15.38:0/2153779827 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f66680b2e40 0x7f66680b3250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:07.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:07.092+0000 7f666d48b640 1 --2- 172.21.15.38:0/2153779827 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6668162280 0x7f6668162690 unknown :-1 s=CLOSED pgs=1931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:07.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:07.092+0000 7f666d48b640 1 --2- 172.21.15.38:0/2153779827 >> v2:172.21.15.195:3300/0 conn(0x7f6668162f00 0x7f66680d24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:07.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:07.092+0000 7f666d48b640 1 -- 172.21.15.38:0/2153779827 >> 172.21.15.38:0/2153779827 conn(0x7f66680b6b20 msgr2=0x7f66680d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:07.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:07.092+0000 7f666d48b640 1 -- 172.21.15.38:0/2153779827 shutdown_connections 2024-09-21T03:43:07.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:07.092+0000 7f666d48b640 1 -- 172.21.15.38:0/2153779827 wait complete. 2024-09-21T03:43:07.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:07.094+0000 7f666d48b640 1 Processor -- start 2024-09-21T03:43:07.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:07.094+0000 7f666d48b640 1 -- start start 2024-09-21T03:43:07.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:07.095+0000 7f666d48b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f66680b2e40 0x7f666814ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:07.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:07.095+0000 7f666d48b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6668162280 0x7f666814af90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:07.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:07.095+0000 7f666d48b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f666814b4d0 0x7f6668159d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:07.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:07.095+0000 7f666d48b640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f66680b3dd0 con 0x7f666814b4d0 2024-09-21T03:43:07.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:07.095+0000 7f666d48b640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f66680b3f50 con 0x7f6668162280 2024-09-21T03:43:07.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:07.095+0000 7f666d48b640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f66680b40d0 con 0x7f66680b2e40 2024-09-21T03:43:07.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:07.095+0000 7f66677fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f666814b4d0 0x7f6668159d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:07.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:07.095+0000 7f66677fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f666814b4d0 0x7f6668159d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:45004/0 (socket says 172.21.15.38:45004) 2024-09-21T03:43:07.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:07.095+0000 7f66677fe640 1 -- 172.21.15.38:0/1157543969 learned_addr learned my addr 172.21.15.38:0/1157543969 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:43:07.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:07.095+0000 7f6666ffd640 1 --2- 172.21.15.38:0/1157543969 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f66680b2e40 0x7f666814ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:07.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:07.095+0000 7f66667fc640 1 --2- 172.21.15.38:0/1157543969 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6668162280 0x7f666814af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:07.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:07.096+0000 7f66677fe640 1 -- 172.21.15.38:0/1157543969 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f66680b2e40 msgr2=0x7f666814ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:07.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:07.096+0000 7f66677fe640 1 --2- 172.21.15.38:0/1157543969 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f66680b2e40 0x7f666814ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:07.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:07.096+0000 7f66677fe640 1 -- 172.21.15.38:0/1157543969 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6668162280 msgr2=0x7f666814af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:07.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:07.096+0000 7f66677fe640 1 --2- 172.21.15.38:0/1157543969 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6668162280 0x7f666814af90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:07.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:07.096+0000 7f66677fe640 1 -- 172.21.15.38:0/1157543969 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6668149ef0 con 0x7f666814b4d0 2024-09-21T03:43:07.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:07.096+0000 7f66677fe640 1 --2- 172.21.15.38:0/1157543969 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f666814b4d0 0x7f6668159d60 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7f666000ca80 tx=0x7f666000cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:07.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:07.096+0000 7f664f7fe640 1 -- 172.21.15.38:0/1157543969 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6660007df0 con 0x7f666814b4d0 2024-09-21T03:43:07.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:07.096+0000 7f664f7fe640 1 -- 172.21.15.38:0/1157543969 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f666000f560 con 0x7f666814b4d0 2024-09-21T03:43:07.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:07.096+0000 7f664f7fe640 1 -- 172.21.15.38:0/1157543969 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f666000f9b0 con 0x7f666814b4d0 2024-09-21T03:43:07.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:07.096+0000 7f666d48b640 1 -- 172.21.15.38:0/1157543969 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f666814a1d0 con 0x7f666814b4d0 2024-09-21T03:43:07.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:07.096+0000 7f666d48b640 1 -- 172.21.15.38:0/1157543969 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f666814a790 con 0x7f666814b4d0 2024-09-21T03:43:07.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:07.097+0000 7f666d48b640 1 -- 172.21.15.38:0/1157543969 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6668149c60 con 0x7f666814b4d0 2024-09-21T03:43:07.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:07.098+0000 7f664f7fe640 1 -- 172.21.15.38:0/1157543969 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f666000fb50 con 0x7f666814b4d0 2024-09-21T03:43:07.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:07.098+0000 7f664f7fe640 1 --2- 172.21.15.38:0/1157543969 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6654078b90 0x7f665407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:07.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:07.098+0000 7f664f7fe640 1 -- 172.21.15.38:0/1157543969 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 8798+0+0 (secure 0 0 0) 0x7f666009c370 con 0x7f666814b4d0 2024-09-21T03:43:07.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:07.099+0000 7f6666ffd640 1 --2- 172.21.15.38:0/1157543969 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6654078b90 0x7f665407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:07.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:07.099+0000 7f6666ffd640 1 --2- 172.21.15.38:0/1157543969 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6654078b90 0x7f665407b0a0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f665800bb50 tx=0x7f665800b2e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:07.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:07.103+0000 7f664f7fe640 1 -- 172.21.15.38:0/1157543969 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f6660014070 con 0x7f666814b4d0 2024-09-21T03:43:07.312 INFO:tasks.ceph.mds.c.smithi038.stderr: -42> 2024-09-21T03:43:02.558+0000 7f130f33a640 -1 log_channel(cluster) log [ERR] : error decoding table object 'mds_snaptable': End of buffer [buffer:2] 2024-09-21T03:43:07.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:07.333+0000 7f666d48b640 1 -- 172.21.15.38:0/1157543969 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6668162f00 con 0x7f666814b4d0 2024-09-21T03:43:07.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:07.333+0000 7f664f7fe640 1 -- 172.21.15.38:0/1157543969 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 256 v256) ==== 95+0+3566 (secure 0 0 0) 0x7f6660063500 con 0x7f666814b4d0 2024-09-21T03:43:07.335 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:43:07.335 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":256,"btime":"2024-09-21T03:43:06:604763+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6086,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/965972663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":965972663},{"type":"v1","addr":"172.21.15.195:6836","nonce":965972663}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6700,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/3176122334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":3176122334},{"type":"v1","addr":"172.21.15.195:6837","nonce":3176122334}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":256,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:40:02.319282+0000","modified":"2024-09-21T03:43:06.604692+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":125,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-21T03:43:07.336 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 256 2024-09-21T03:43:07.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:07.336+0000 7f666d48b640 1 -- 172.21.15.38:0/1157543969 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6654078b90 msgr2=0x7f665407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:07.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:07.336+0000 7f666d48b640 1 --2- 172.21.15.38:0/1157543969 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6654078b90 0x7f665407b0a0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f665800bb50 tx=0x7f665800b2e0 comp rx=0 tx=0).stop 2024-09-21T03:43:07.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:07.336+0000 7f666d48b640 1 -- 172.21.15.38:0/1157543969 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f666814b4d0 msgr2=0x7f6668159d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:07.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:07.336+0000 7f666d48b640 1 --2- 172.21.15.38:0/1157543969 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f666814b4d0 0x7f6668159d60 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7f666000ca80 tx=0x7f666000cf40 comp rx=0 tx=0).stop 2024-09-21T03:43:07.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:07.336+0000 7f666d48b640 1 -- 172.21.15.38:0/1157543969 shutdown_connections 2024-09-21T03:43:07.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:07.336+0000 7f666d48b640 1 --2- 172.21.15.38:0/1157543969 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f666814b4d0 0x7f6668159d60 unknown :-1 s=CLOSED pgs=1932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:07.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:07.336+0000 7f666d48b640 1 --2- 172.21.15.38:0/1157543969 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6668162280 0x7f666814af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:07.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:07.336+0000 7f666d48b640 1 --2- 172.21.15.38:0/1157543969 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6654078b90 0x7f665407b0a0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:07.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:07.336+0000 7f666d48b640 1 --2- 172.21.15.38:0/1157543969 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f66680b2e40 0x7f666814ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:07.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:07.336+0000 7f666d48b640 1 -- 172.21.15.38:0/1157543969 >> 172.21.15.38:0/1157543969 conn(0x7f66680b6b20 msgr2=0x7f66680b45a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:07.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:07.336+0000 7f666d48b640 1 -- 172.21.15.38:0/1157543969 shutdown_connections 2024-09-21T03:43:07.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:07.336+0000 7f666d48b640 1 -- 172.21.15.38:0/1157543969 wait complete. 2024-09-21T03:43:07.351 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-21T03:43:07.351 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:43:07.368 INFO:tasks.ceph.mds.c.smithi038.stdout:starting mds.c at 2024-09-21T03:43:07.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:07.466+0000 7fd81222e640 1 -- 172.21.15.38:0/2990288235 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd80c0d1d00 msgr2=0x7fd80c0cc930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:07.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:07.466+0000 7fd81222e640 1 --2- 172.21.15.38:0/2990288235 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd80c0d1d00 0x7fd80c0cc930 secure :-1 s=READY pgs=1935 cs=0 l=1 rev1=1 crypto rx=0x7fd80c0b29d0 tx=0x7fd80002a560 comp rx=0 tx=0).stop 2024-09-21T03:43:07.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:07.466+0000 7fd81222e640 1 -- 172.21.15.38:0/2990288235 shutdown_connections 2024-09-21T03:43:07.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:07.466+0000 7fd81222e640 1 --2- 172.21.15.38:0/2990288235 >> v2:172.21.15.195:3300/0 conn(0x7fd80c0cce70 0x7fd80c160190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:07.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:07.466+0000 7fd81222e640 1 --2- 172.21.15.38:0/2990288235 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd80c0d1d00 0x7fd80c0cc930 unknown :-1 s=CLOSED pgs=1935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:07.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:07.466+0000 7fd81222e640 1 --2- 172.21.15.38:0/2990288235 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd80c1536e0 0x7fd80c0d17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:07.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:07.466+0000 7fd81222e640 1 -- 172.21.15.38:0/2990288235 >> 172.21.15.38:0/2990288235 conn(0x7fd80c0cef40 msgr2=0x7fd80c0cf350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:07.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:07.466+0000 7fd81222e640 1 -- 172.21.15.38:0/2990288235 shutdown_connections 2024-09-21T03:43:07.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:07.466+0000 7fd81222e640 1 -- 172.21.15.38:0/2990288235 wait complete. 2024-09-21T03:43:07.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:07.468+0000 7fd81222e640 1 Processor -- start 2024-09-21T03:43:07.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:07.468+0000 7fd81222e640 1 -- start start 2024-09-21T03:43:07.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:07.469+0000 7fd81222e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd80c0cce70 0x7fd80c15bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:07.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:07.469+0000 7fd81222e640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd80c0d1d00 0x7fd80c06f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:07.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:07.469+0000 7fd81222e640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd80c1536e0 0x7fd80c153c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:07.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:07.469+0000 7fd81222e640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fd80c1606d0 con 0x7fd80c0cce70 2024-09-21T03:43:07.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:07.469+0000 7fd81222e640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd80c160850 con 0x7fd80c0d1d00 2024-09-21T03:43:07.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:07.469+0000 7fd81222e640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fd80c1609d0 con 0x7fd80c1536e0 2024-09-21T03:43:07.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:07.469+0000 7fd80b7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd80c0cce70 0x7fd80c15bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:07.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:07.469+0000 7fd80b7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd80c0cce70 0x7fd80c15bc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:45050/0 (socket says 172.21.15.38:45050) 2024-09-21T03:43:07.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:07.469+0000 7fd80b7fe640 1 -- 172.21.15.38:0/68485030 learned_addr learned my addr 172.21.15.38:0/68485030 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:43:07.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:07.469+0000 7fd80affd640 1 --2- 172.21.15.38:0/68485030 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd80c0d1d00 0x7fd80c06f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:07.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:07.469+0000 7fd80bfff640 1 --2- 172.21.15.38:0/68485030 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd80c1536e0 0x7fd80c153c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:07.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:07.470+0000 7fd80b7fe640 1 -- 172.21.15.38:0/68485030 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd80c1536e0 msgr2=0x7fd80c153c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:07.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:07.470+0000 7fd80b7fe640 1 --2- 172.21.15.38:0/68485030 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd80c1536e0 0x7fd80c153c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:07.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:07.470+0000 7fd80b7fe640 1 -- 172.21.15.38:0/68485030 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd80c0d1d00 msgr2=0x7fd80c06f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:07.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:07.470+0000 7fd80b7fe640 1 --2- 172.21.15.38:0/68485030 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd80c0d1d00 0x7fd80c06f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:07.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:07.470+0000 7fd80b7fe640 1 -- 172.21.15.38:0/68485030 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd80c155fd0 con 0x7fd80c0cce70 2024-09-21T03:43:07.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:07.470+0000 7fd80b7fe640 1 --2- 172.21.15.38:0/68485030 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd80c0cce70 0x7fd80c15bc20 secure :-1 s=READY pgs=1936 cs=0 l=1 rev1=1 crypto rx=0x7fd7fc00e7c0 tx=0x7fd7fc00ebc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:07.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:07.470+0000 7fd7f3fff640 1 -- 172.21.15.38:0/68485030 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7fc005220 con 0x7fd80c0cce70 2024-09-21T03:43:07.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:07.470+0000 7fd7f3fff640 1 -- 172.21.15.38:0/68485030 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd7fc0053e0 con 0x7fd80c0cce70 2024-09-21T03:43:07.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:07.470+0000 7fd81222e640 1 -- 172.21.15.38:0/68485030 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd80c1562b0 con 0x7fd80c0cce70 2024-09-21T03:43:07.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:07.470+0000 7fd7f3fff640 1 -- 172.21.15.38:0/68485030 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7fc00c850 con 0x7fd80c0cce70 2024-09-21T03:43:07.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:07.470+0000 7fd81222e640 1 -- 172.21.15.38:0/68485030 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd80c156870 con 0x7fd80c0cce70 2024-09-21T03:43:07.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:07.471+0000 7fd81222e640 1 -- 172.21.15.38:0/68485030 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd80c160ef0 con 0x7fd80c0cce70 2024-09-21T03:43:07.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:07.472+0000 7fd7f3fff640 1 -- 172.21.15.38:0/68485030 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fd7fc00c9f0 con 0x7fd80c0cce70 2024-09-21T03:43:07.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:07.472+0000 7fd7f3fff640 1 --2- 172.21.15.38:0/68485030 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd7f8078b90 0x7fd7f807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:07.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:07.472+0000 7fd7f3fff640 1 -- 172.21.15.38:0/68485030 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 8798+0+0 (secure 0 0 0) 0x7fd7fc09c1a0 con 0x7fd80c0cce70 2024-09-21T03:43:07.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:07.473+0000 7fd80affd640 1 --2- 172.21.15.38:0/68485030 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd7f8078b90 0x7fd7f807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:07.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:07.473+0000 7fd80affd640 1 --2- 172.21.15.38:0/68485030 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd7f8078b90 0x7fd7f807b0a0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fd800033000 tx=0x7fd800004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:07.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:07.477+0000 7fd7f3fff640 1 -- 172.21.15.38:0/68485030 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fd7fc0639b0 con 0x7fd80c0cce70 2024-09-21T03:43:07.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:07.703+0000 7fd81222e640 1 -- 172.21.15.38:0/68485030 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd80c1495b0 con 0x7fd80c0cce70 2024-09-21T03:43:07.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:07.703+0000 7fd7f3fff640 1 -- 172.21.15.38:0/68485030 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 257 v257) ==== 95+0+4389 (secure 0 0 0) 0x7fd7fc0632b0 con 0x7fd80c0cce70 2024-09-21T03:43:07.706 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:43:07.706 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":257,"btime":"2024-09-21T03:43:07:606935+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6086,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/965972663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":965972663},{"type":"v1","addr":"172.21.15.195:6836","nonce":965972663}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6700,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/3176122334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":3176122334},{"type":"v1","addr":"172.21.15.195:6837","nonce":3176122334}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":9654,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/775682959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":775682959},{"type":"v1","addr":"172.21.15.38:6833","nonce":775682959}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":257}],"filesystems":[{"mdsmap":{"epoch":256,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:40:02.319282+0000","modified":"2024-09-21T03:43:06.604692+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":125,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-21T03:43:07.707 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 257 2024-09-21T03:43:07.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:07.706+0000 7fd81222e640 1 -- 172.21.15.38:0/68485030 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd7f8078b90 msgr2=0x7fd7f807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:07.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:07.706+0000 7fd81222e640 1 --2- 172.21.15.38:0/68485030 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd7f8078b90 0x7fd7f807b0a0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fd800033000 tx=0x7fd800004730 comp rx=0 tx=0).stop 2024-09-21T03:43:07.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:07.706+0000 7fd81222e640 1 -- 172.21.15.38:0/68485030 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd80c0cce70 msgr2=0x7fd80c15bc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:07.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:07.706+0000 7fd81222e640 1 --2- 172.21.15.38:0/68485030 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd80c0cce70 0x7fd80c15bc20 secure :-1 s=READY pgs=1936 cs=0 l=1 rev1=1 crypto rx=0x7fd7fc00e7c0 tx=0x7fd7fc00ebc0 comp rx=0 tx=0).stop 2024-09-21T03:43:07.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:07.706+0000 7fd81222e640 1 -- 172.21.15.38:0/68485030 shutdown_connections 2024-09-21T03:43:07.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:07.706+0000 7fd81222e640 1 --2- 172.21.15.38:0/68485030 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd7f8078b90 0x7fd7f807b0a0 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:07.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:07.706+0000 7fd81222e640 1 --2- 172.21.15.38:0/68485030 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd80c1536e0 0x7fd80c153c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:07.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:07.706+0000 7fd81222e640 1 --2- 172.21.15.38:0/68485030 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd80c0d1d00 0x7fd80c06f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:07.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:07.706+0000 7fd81222e640 1 --2- 172.21.15.38:0/68485030 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd80c0cce70 0x7fd80c15bc20 unknown :-1 s=CLOSED pgs=1936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:07.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:07.706+0000 7fd81222e640 1 -- 172.21.15.38:0/68485030 >> 172.21.15.38:0/68485030 conn(0x7fd80c0cef40 msgr2=0x7fd80c0d0430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:07.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:07.706+0000 7fd81222e640 1 -- 172.21.15.38:0/68485030 shutdown_connections 2024-09-21T03:43:07.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:07.706+0000 7fd81222e640 1 -- 172.21.15.38:0/68485030 wait complete. 2024-09-21T03:43:07.719 INFO:tasks.cephfs.test_damage:Result: Mutation 'Truncate mds0_inotable' led to DAMAGED state 2024-09-21T03:43:07.719 INFO:tasks.cephfs.test_damage:Applying mutation 'Truncate 400.00000000' 2024-09-21T03:43:07.720 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-21T03:43:07.720 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:43:07.776 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056795,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2354 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T03:43:07.776 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi038.front.sepia.ceph.com 2024-09-21T03:43:07.776 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:43:07.776 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T03:43:07.832 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T03:43:07.832 INFO:teuthology.orchestra.run.smithi038.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-09-21T03:43:07.833 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-21T03:43:07.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:07.988+0000 7f6ab2b82640 1 -- 172.21.15.38:0/3773089827 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6aac0b9ad0 msgr2=0x7f6aac0d0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:07.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:07.988+0000 7f6ab2b82640 1 --2- 172.21.15.38:0/3773089827 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6aac0b9ad0 0x7f6aac0d0be0 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 crypto rx=0x7f6aac0cfba0 tx=0x7f6aa002a4d0 comp rx=0 tx=0).stop 2024-09-21T03:43:07.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:07.988+0000 7f6ab2b82640 1 -- 172.21.15.38:0/3773089827 shutdown_connections 2024-09-21T03:43:07.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:07.988+0000 7f6ab2b82640 1 --2- 172.21.15.38:0/3773089827 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6aac0b9ad0 0x7f6aac0d0be0 unknown :-1 s=CLOSED pgs=1937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:07.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:07.988+0000 7f6ab2b82640 1 --2- 172.21.15.38:0/3773089827 >> v2:172.21.15.195:3300/0 conn(0x7f6aac15ee80 0x7f6aac0d0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:07.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:07.988+0000 7f6ab2b82640 1 -- 172.21.15.38:0/3773089827 >> 172.21.15.38:0/3773089827 conn(0x7f6aac09d830 msgr2=0x7f6aac0b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:07.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:07.988+0000 7f6ab2b82640 1 -- 172.21.15.38:0/3773089827 shutdown_connections 2024-09-21T03:43:07.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:07.988+0000 7f6ab2b82640 1 -- 172.21.15.38:0/3773089827 wait complete. 2024-09-21T03:43:07.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:07.990+0000 7f6ab2b82640 1 Processor -- start 2024-09-21T03:43:07.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:07.990+0000 7f6ab2b82640 1 -- start start 2024-09-21T03:43:07.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:07.991+0000 7f6ab2b82640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6aac0b9ad0 0x7f6aac1f8920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:07.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:07.991+0000 7f6ab2b82640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6aac0d1120 0x7f6aac156400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:07.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:07.991+0000 7f6ab2b82640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6aac15ee80 0x7f6aac0b21e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:07.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:07.991+0000 7f6ab2b82640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f6aac0b6700 con 0x7f6aac15ee80 2024-09-21T03:43:07.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:07.991+0000 7f6ab2b82640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f6aac0b6580 con 0x7f6aac0d1120 2024-09-21T03:43:07.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:07.991+0000 7f6ab2b82640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f6aac0b6880 con 0x7f6aac0b9ad0 2024-09-21T03:43:07.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:07.991+0000 7f6ab111f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6aac15ee80 0x7f6aac0b21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:07.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:07.991+0000 7f6ab111f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6aac15ee80 0x7f6aac0b21e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:45070/0 (socket says 172.21.15.38:45070) 2024-09-21T03:43:07.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:07.991+0000 7f6ab111f640 1 -- 172.21.15.38:0/1911102691 learned_addr learned my addr 172.21.15.38:0/1911102691 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:43:07.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:07.991+0000 7f6ab091e640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6aac0b9ad0 0x7f6aac1f8920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:07.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:07.991+0000 7f6aabfff640 1 --2- 172.21.15.38:0/1911102691 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6aac0d1120 0x7f6aac156400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:07.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:07.992+0000 7f6ab111f640 1 -- 172.21.15.38:0/1911102691 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6aac0b9ad0 msgr2=0x7f6aac1f8920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:07.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:07.992+0000 7f6ab111f640 1 --2- 172.21.15.38:0/1911102691 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6aac0b9ad0 0x7f6aac1f8920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:07.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:07.992+0000 7f6ab111f640 1 -- 172.21.15.38:0/1911102691 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6aac0d1120 msgr2=0x7f6aac156400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:07.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:07.992+0000 7f6ab111f640 1 --2- 172.21.15.38:0/1911102691 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6aac0d1120 0x7f6aac156400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:07.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:07.992+0000 7f6ab111f640 1 -- 172.21.15.38:0/1911102691 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6aac1f9290 con 0x7f6aac15ee80 2024-09-21T03:43:07.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:07.992+0000 7f6ab111f640 1 --2- 172.21.15.38:0/1911102691 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6aac15ee80 0x7f6aac0b21e0 secure :-1 s=READY pgs=1938 cs=0 l=1 rev1=1 crypto rx=0x7f6aa4002230 tx=0x7f6aa400cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:07.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:07.992+0000 7f6aa97fa640 1 -- 172.21.15.38:0/1911102691 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aa4003b70 con 0x7f6aac15ee80 2024-09-21T03:43:07.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:07.992+0000 7f6aa97fa640 1 -- 172.21.15.38:0/1911102691 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6aa4003d10 con 0x7f6aac15ee80 2024-09-21T03:43:07.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:07.992+0000 7f6ab2b82640 1 -- 172.21.15.38:0/1911102691 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6aac1f9570 con 0x7f6aac15ee80 2024-09-21T03:43:07.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:07.992+0000 7f6aa97fa640 1 -- 172.21.15.38:0/1911102691 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aa4005330 con 0x7f6aac15ee80 2024-09-21T03:43:07.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:07.992+0000 7f6ab2b82640 1 -- 172.21.15.38:0/1911102691 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6aac157030 con 0x7f6aac15ee80 2024-09-21T03:43:07.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:07.993+0000 7f6ab2b82640 1 -- 172.21.15.38:0/1911102691 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6aac1f9000 con 0x7f6aac15ee80 2024-09-21T03:43:07.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:07.994+0000 7f6aa97fa640 1 -- 172.21.15.38:0/1911102691 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f6aa40054d0 con 0x7f6aac15ee80 2024-09-21T03:43:07.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:07.994+0000 7f6aa97fa640 1 --2- 172.21.15.38:0/1911102691 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6a98078b90 0x7f6a9807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:07.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:07.995+0000 7f6aa97fa640 1 -- 172.21.15.38:0/1911102691 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 8798+0+0 (secure 0 0 0) 0x7f6aa409b4c0 con 0x7f6aac15ee80 2024-09-21T03:43:07.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:07.995+0000 7f6ab091e640 1 --2- 172.21.15.38:0/1911102691 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6a98078b90 0x7f6a9807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:07.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:07.995+0000 7f6ab091e640 1 --2- 172.21.15.38:0/1911102691 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6a98078b90 0x7f6a9807b0a0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f6a9c00bb50 tx=0x7f6a9c00b2e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:08.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:07.999+0000 7f6aa97fa640 1 -- 172.21.15.38:0/1911102691 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f6aac1592d0 con 0x7f6aac15ee80 2024-09-21T03:43:08.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:08.224+0000 7f6ab2b82640 1 -- 172.21.15.38:0/1911102691 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f6aac0b6d30 con 0x7f6aac15ee80 2024-09-21T03:43:08.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:08.619+0000 7f6aa97fa640 1 -- 172.21.15.38:0/1911102691 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v258) ==== 161+0+0 (secure 0 0 0) 0x7f6aa4062770 con 0x7f6aac15ee80 2024-09-21T03:43:08.622 INFO:teuthology.orchestra.run.smithi038.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-21T03:43:08.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:08.622+0000 7f6ab2b82640 1 -- 172.21.15.38:0/1911102691 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6a98078b90 msgr2=0x7f6a9807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:08.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:08.622+0000 7f6ab2b82640 1 --2- 172.21.15.38:0/1911102691 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6a98078b90 0x7f6a9807b0a0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f6a9c00bb50 tx=0x7f6a9c00b2e0 comp rx=0 tx=0).stop 2024-09-21T03:43:08.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:08.622+0000 7f6ab2b82640 1 -- 172.21.15.38:0/1911102691 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6aac15ee80 msgr2=0x7f6aac0b21e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:08.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:08.622+0000 7f6ab2b82640 1 --2- 172.21.15.38:0/1911102691 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6aac15ee80 0x7f6aac0b21e0 secure :-1 s=READY pgs=1938 cs=0 l=1 rev1=1 crypto rx=0x7f6aa4002230 tx=0x7f6aa400cb90 comp rx=0 tx=0).stop 2024-09-21T03:43:08.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:08.622+0000 7f6ab2b82640 1 -- 172.21.15.38:0/1911102691 shutdown_connections 2024-09-21T03:43:08.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:08.622+0000 7f6ab2b82640 1 --2- 172.21.15.38:0/1911102691 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6aac15ee80 0x7f6aac0b21e0 unknown :-1 s=CLOSED pgs=1938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:08.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:08.622+0000 7f6ab2b82640 1 --2- 172.21.15.38:0/1911102691 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6aac0d1120 0x7f6aac156400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:08.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:08.622+0000 7f6ab2b82640 1 --2- 172.21.15.38:0/1911102691 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6a98078b90 0x7f6a9807b0a0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:08.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:08.622+0000 7f6ab2b82640 1 --2- 172.21.15.38:0/1911102691 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6aac0b9ad0 0x7f6aac1f8920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:08.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:08.622+0000 7f6ab2b82640 1 -- 172.21.15.38:0/1911102691 >> 172.21.15.38:0/1911102691 conn(0x7f6aac09d830 msgr2=0x7f6aac0b5270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:08.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:08.622+0000 7f6ab2b82640 1 -- 172.21.15.38:0/1911102691 shutdown_connections 2024-09-21T03:43:08.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:08.622+0000 7f6ab2b82640 1 -- 172.21.15.38:0/1911102691 wait complete. 2024-09-21T03:43:08.636 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-09-21T03:43:08.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:08.754+0000 7f21cd339640 1 -- 172.21.15.38:0/3060615239 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f21c80d1120 msgr2=0x7f21c80ceb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:08.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:08.754+0000 7f21cd339640 1 --2- 172.21.15.38:0/3060615239 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f21c80d1120 0x7f21c80ceb90 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f21c0009830 tx=0x7f21c002a470 comp rx=0 tx=0).stop 2024-09-21T03:43:08.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:08.754+0000 7f21cd339640 1 -- 172.21.15.38:0/3060615239 shutdown_connections 2024-09-21T03:43:08.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:08.754+0000 7f21cd339640 1 --2- 172.21.15.38:0/3060615239 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f21c80d1120 0x7f21c80ceb90 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:08.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:08.754+0000 7f21cd339640 1 --2- 172.21.15.38:0/3060615239 >> v2:172.21.15.195:3300/0 conn(0x7f21c80d0170 0x7f21c80d0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:08.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:08.754+0000 7f21cd339640 1 -- 172.21.15.38:0/3060615239 >> 172.21.15.38:0/3060615239 conn(0x7f21c809d830 msgr2=0x7f21c80b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:08.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:08.754+0000 7f21cd339640 1 -- 172.21.15.38:0/3060615239 shutdown_connections 2024-09-21T03:43:08.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:08.754+0000 7f21cd339640 1 -- 172.21.15.38:0/3060615239 wait complete. 2024-09-21T03:43:08.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:08.756+0000 7f21cd339640 1 Processor -- start 2024-09-21T03:43:08.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:08.756+0000 7f21cd339640 1 -- start start 2024-09-21T03:43:08.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:08.758+0000 7f21cd339640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f21c80d0170 0x7f21c806cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:08.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:08.758+0000 7f21cd339640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f21c80d1120 0x7f21c814b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:08.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:08.758+0000 7f21cd339640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f21c815ee80 0x7f21c80ccd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:08.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:08.758+0000 7f21cd339640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f21c80b6700 con 0x7f21c80d0170 2024-09-21T03:43:08.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:08.758+0000 7f21cd339640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f21c80b6580 con 0x7f21c815ee80 2024-09-21T03:43:08.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:08.758+0000 7f21cd339640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f21c80b6880 con 0x7f21c80d1120 2024-09-21T03:43:08.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:08.758+0000 7f21c6ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f21c80d0170 0x7f21c806cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:08.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:08.758+0000 7f21c6ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f21c80d0170 0x7f21c806cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:45080/0 (socket says 172.21.15.38:45080) 2024-09-21T03:43:08.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:08.758+0000 7f21c6ffd640 1 -- 172.21.15.38:0/3248729722 learned_addr learned my addr 172.21.15.38:0/3248729722 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:43:08.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:08.758+0000 7f21c77fe640 1 --2- 172.21.15.38:0/3248729722 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f21c815ee80 0x7f21c80ccd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:08.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:08.758+0000 7f21c67fc640 1 --2- 172.21.15.38:0/3248729722 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f21c80d1120 0x7f21c814b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:08.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:08.758+0000 7f21c6ffd640 1 -- 172.21.15.38:0/3248729722 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f21c80d1120 msgr2=0x7f21c814b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:08.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:08.758+0000 7f21c6ffd640 1 --2- 172.21.15.38:0/3248729722 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f21c80d1120 0x7f21c814b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:08.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:08.758+0000 7f21c6ffd640 1 -- 172.21.15.38:0/3248729722 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f21c815ee80 msgr2=0x7f21c80ccd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:08.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:08.758+0000 7f21c6ffd640 1 --2- 172.21.15.38:0/3248729722 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f21c815ee80 0x7f21c80ccd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:08.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:08.758+0000 7f21c6ffd640 1 -- 172.21.15.38:0/3248729722 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21c814a250 con 0x7f21c80d0170 2024-09-21T03:43:08.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:08.758+0000 7f21c6ffd640 1 --2- 172.21.15.38:0/3248729722 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f21c80d0170 0x7f21c806cde0 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x7f21b800cbf0 tx=0x7f21b800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:08.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:08.759+0000 7f21af7fe640 1 -- 172.21.15.38:0/3248729722 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21b8016020 con 0x7f21c80d0170 2024-09-21T03:43:08.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:08.759+0000 7f21af7fe640 1 -- 172.21.15.38:0/3248729722 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f21b800ad90 con 0x7f21c80d0170 2024-09-21T03:43:08.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:08.759+0000 7f21af7fe640 1 -- 172.21.15.38:0/3248729722 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21b8012680 con 0x7f21c80d0170 2024-09-21T03:43:08.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:08.759+0000 7f21cd339640 1 -- 172.21.15.38:0/3248729722 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21c814a530 con 0x7f21c80d0170 2024-09-21T03:43:08.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:08.759+0000 7f21cd339640 1 -- 172.21.15.38:0/3248729722 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21c814aaf0 con 0x7f21c80d0170 2024-09-21T03:43:08.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:08.760+0000 7f21cd339640 1 -- 172.21.15.38:0/3248729722 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21c80b6da0 con 0x7f21c80d0170 2024-09-21T03:43:08.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:08.761+0000 7f21af7fe640 1 -- 172.21.15.38:0/3248729722 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f21b8019020 con 0x7f21c80d0170 2024-09-21T03:43:08.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:08.761+0000 7f21af7fe640 1 --2- 172.21.15.38:0/3248729722 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f21b4078b90 0x7f21b407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:08.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:08.761+0000 7f21af7fe640 1 -- 172.21.15.38:0/3248729722 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 8798+0+0 (secure 0 0 0) 0x7f21b809be10 con 0x7f21c80d0170 2024-09-21T03:43:08.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:08.761+0000 7f21c67fc640 1 --2- 172.21.15.38:0/3248729722 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f21b4078b90 0x7f21b407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:08.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:08.762+0000 7f21c67fc640 1 --2- 172.21.15.38:0/3248729722 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f21b4078b90 0x7f21b407b0a0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f21c815e110 tx=0x7f21bc005f30 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:08.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:08.766+0000 7f21af7fe640 1 -- 172.21.15.38:0/3248729722 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f21b8020020 con 0x7f21c80d0170 2024-09-21T03:43:09.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.032+0000 7f21cd339640 1 -- 172.21.15.38:0/3248729722 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7f21c8158dc0 con 0x7f21c80d0170 2024-09-21T03:43:09.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.631+0000 7f21af7fe640 1 -- 172.21.15.38:0/3248729722 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v259) ==== 101+0+0 (secure 0 0 0) 0x7f21c81592d0 con 0x7f21c80d0170 2024-09-21T03:43:09.634 INFO:teuthology.orchestra.run.smithi038.stderr:repaired: restoring rank 6:0 2024-09-21T03:43:09.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.634+0000 7f21cd339640 1 -- 172.21.15.38:0/3248729722 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f21b4078b90 msgr2=0x7f21b407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:09.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.634+0000 7f21cd339640 1 --2- 172.21.15.38:0/3248729722 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f21b4078b90 0x7f21b407b0a0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f21c815e110 tx=0x7f21bc005f30 comp rx=0 tx=0).stop 2024-09-21T03:43:09.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.634+0000 7f21cd339640 1 -- 172.21.15.38:0/3248729722 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f21c80d0170 msgr2=0x7f21c806cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:09.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.634+0000 7f21cd339640 1 --2- 172.21.15.38:0/3248729722 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f21c80d0170 0x7f21c806cde0 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x7f21b800cbf0 tx=0x7f21b800a060 comp rx=0 tx=0).stop 2024-09-21T03:43:09.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.634+0000 7f21cd339640 1 -- 172.21.15.38:0/3248729722 shutdown_connections 2024-09-21T03:43:09.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.634+0000 7f21cd339640 1 --2- 172.21.15.38:0/3248729722 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f21c815ee80 0x7f21c80ccd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:09.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.634+0000 7f21cd339640 1 --2- 172.21.15.38:0/3248729722 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f21b4078b90 0x7f21b407b0a0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:09.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.634+0000 7f21cd339640 1 --2- 172.21.15.38:0/3248729722 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f21c80d1120 0x7f21c814b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:09.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.634+0000 7f21cd339640 1 --2- 172.21.15.38:0/3248729722 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f21c80d0170 0x7f21c806cde0 unknown :-1 s=CLOSED pgs=1939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:09.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.634+0000 7f21cd339640 1 -- 172.21.15.38:0/3248729722 >> 172.21.15.38:0/3248729722 conn(0x7f21c809d830 msgr2=0x7f21c80b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:09.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.635+0000 7f21cd339640 1 -- 172.21.15.38:0/3248729722 shutdown_connections 2024-09-21T03:43:09.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.635+0000 7f21cd339640 1 -- 172.21.15.38:0/3248729722 wait complete. 2024-09-21T03:43:09.651 DEBUG:teuthology.orchestra.run.smithi038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-09-21T03:43:09.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.709+0000 7f9bd216df40 1 Processor -- start 2024-09-21T03:43:09.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.709+0000 7f9bd216df40 1 -- start start 2024-09-21T03:43:09.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.710+0000 7f9bd216df40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56393232e240 0x5639323e62c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:09.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.710+0000 7f9bd216df40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563932375090 0x563932356770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:09.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.710+0000 7f9bd216df40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5639323ea520 0x5639324025e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:09.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.710+0000 7f9bd216df40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5639322da430 con 0x5639323ea520 2024-09-21T03:43:09.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.710+0000 7f9bd216df40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5639322e71f0 con 0x563932375090 2024-09-21T03:43:09.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.710+0000 7f9bd216df40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x56393227bd60 con 0x56393232e240 2024-09-21T03:43:09.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.710+0000 7f9bcc428640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5639323ea520 0x5639324025e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:09.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.710+0000 7f9bcc428640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5639323ea520 0x5639324025e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:45104/0 (socket says 172.21.15.38:45104) 2024-09-21T03:43:09.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.710+0000 7f9bcc428640 1 -- 172.21.15.38:0/424570669 learned_addr learned my addr 172.21.15.38:0/424570669 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:43:09.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.710+0000 7f9bcb426640 1 --2- 172.21.15.38:0/424570669 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563932375090 0x563932356770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:09.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.710+0000 7f9bcbc27640 1 --2- 172.21.15.38:0/424570669 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56393232e240 0x5639323e62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:09.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.711+0000 7f9bcc428640 1 -- 172.21.15.38:0/424570669 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56393232e240 msgr2=0x5639323e62c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:09.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.711+0000 7f9bcc428640 1 --2- 172.21.15.38:0/424570669 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56393232e240 0x5639323e62c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:09.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.711+0000 7f9bcc428640 1 -- 172.21.15.38:0/424570669 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563932375090 msgr2=0x563932356770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:09.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.711+0000 7f9bcc428640 1 --2- 172.21.15.38:0/424570669 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563932375090 0x563932356770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:09.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.711+0000 7f9bcc428640 1 -- 172.21.15.38:0/424570669 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563932494360 con 0x5639323ea520 2024-09-21T03:43:09.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.711+0000 7f9bcb426640 1 --2- 172.21.15.38:0/424570669 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563932375090 0x563932356770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:43:09.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.711+0000 7f9bcc428640 1 --2- 172.21.15.38:0/424570669 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5639323ea520 0x5639324025e0 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 crypto rx=0x7f9bc4001d00 tx=0x7f9bc400b5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8256ae8e089f306 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:09.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.711+0000 7f9bca424640 1 -- 172.21.15.38:0/424570669 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bc400bd80 con 0x5639323ea520 2024-09-21T03:43:09.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.711+0000 7f9bca424640 1 -- 172.21.15.38:0/424570669 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9bc4001e50 con 0x5639323ea520 2024-09-21T03:43:09.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.711+0000 7f9bca424640 1 -- 172.21.15.38:0/424570669 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bc400f660 con 0x5639323ea520 2024-09-21T03:43:09.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.712+0000 7f9bd216df40 1 -- 172.21.15.38:0/424570669 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5639323ea520 msgr2=0x5639324025e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:09.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.712+0000 7f9bd216df40 1 --2- 172.21.15.38:0/424570669 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5639323ea520 0x5639324025e0 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 crypto rx=0x7f9bc4001d00 tx=0x7f9bc400b5b0 comp rx=0 tx=0).stop 2024-09-21T03:43:09.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.712+0000 7f9bd216df40 1 -- 172.21.15.38:0/424570669 shutdown_connections 2024-09-21T03:43:09.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.712+0000 7f9bd216df40 1 --2- 172.21.15.38:0/424570669 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5639323ea520 0x5639324025e0 unknown :-1 s=CLOSED pgs=1942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:09.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.712+0000 7f9bd216df40 1 --2- 172.21.15.38:0/424570669 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563932375090 0x563932356770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:09.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.712+0000 7f9bd216df40 1 --2- 172.21.15.38:0/424570669 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56393232e240 0x5639323e62c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:09.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.712+0000 7f9bd216df40 1 -- 172.21.15.38:0/424570669 >> 172.21.15.38:0/424570669 conn(0x56393224cac0 msgr2=0x5639323fad20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:09.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.712+0000 7f9bd216df40 1 -- 172.21.15.38:0/424570669 shutdown_connections 2024-09-21T03:43:09.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.712+0000 7f9bd216df40 1 -- 172.21.15.38:0/424570669 wait complete. 2024-09-21T03:43:09.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.713+0000 7f9bd216df40 1 Processor -- start 2024-09-21T03:43:09.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.713+0000 7f9bd216df40 1 -- start start 2024-09-21T03:43:09.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.714+0000 7f9bd216df40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56393232e240 0x5639323fb260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:09.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.714+0000 7f9bd216df40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563932375090 0x5639323fe5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:09.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.714+0000 7f9bd216df40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5639323ea520 0x563932492800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:09.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.714+0000 7f9bd216df40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5639323e6800 con 0x5639323ea520 2024-09-21T03:43:09.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.714+0000 7f9bd216df40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x563932494640 con 0x563932375090 2024-09-21T03:43:09.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.714+0000 7f9bd216df40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x5639323e46d0 con 0x56393232e240 2024-09-21T03:43:09.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.714+0000 7f9bcc428640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5639323ea520 0x563932492800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:09.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.714+0000 7f9bcc428640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5639323ea520 0x563932492800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:45110/0 (socket says 172.21.15.38:45110) 2024-09-21T03:43:09.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.714+0000 7f9bcc428640 1 -- 172.21.15.38:0/2604388918 learned_addr learned my addr 172.21.15.38:0/2604388918 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:43:09.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.714+0000 7f9bcbc27640 1 --2- 172.21.15.38:0/2604388918 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56393232e240 0x5639323fb260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:09.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.714+0000 7f9bcb426640 1 --2- 172.21.15.38:0/2604388918 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563932375090 0x5639323fe5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:09.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.714+0000 7f9bcc428640 1 -- 172.21.15.38:0/2604388918 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56393232e240 msgr2=0x5639323fb260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:09.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.714+0000 7f9bcc428640 1 --2- 172.21.15.38:0/2604388918 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56393232e240 0x5639323fb260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:09.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.714+0000 7f9bcc428640 1 -- 172.21.15.38:0/2604388918 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563932375090 msgr2=0x5639323fe5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:09.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.714+0000 7f9bcc428640 1 --2- 172.21.15.38:0/2604388918 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563932375090 0x5639323fe5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:09.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.714+0000 7f9bcc428640 1 -- 172.21.15.38:0/2604388918 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563932493980 con 0x5639323ea520 2024-09-21T03:43:09.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.715+0000 7f9bcc428640 1 --2- 172.21.15.38:0/2604388918 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5639323ea520 0x563932492800 secure :-1 s=READY pgs=1943 cs=0 l=1 rev1=1 crypto rx=0x7f9bc4000f60 tx=0x7f9bc400feb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:09.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.715+0000 7f9bc9422640 1 -- 172.21.15.38:0/2604388918 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bc401b430 con 0x5639323ea520 2024-09-21T03:43:09.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.715+0000 7f9bc9422640 1 -- 172.21.15.38:0/2604388918 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9bc4020020 con 0x5639323ea520 2024-09-21T03:43:09.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.715+0000 7f9bd216df40 1 -- 172.21.15.38:0/2604388918 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x563932493c60 con 0x5639323ea520 2024-09-21T03:43:09.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.715+0000 7f9bc9422640 1 -- 172.21.15.38:0/2604388918 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bc401b780 con 0x5639323ea520 2024-09-21T03:43:09.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.715+0000 7f9bd216df40 1 -- 172.21.15.38:0/2604388918 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5639323e0ea0 con 0x5639323ea520 2024-09-21T03:43:09.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.717+0000 7f9bc9422640 1 -- 172.21.15.38:0/2604388918 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f9bc4017070 con 0x5639323ea520 2024-09-21T03:43:09.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.717+0000 7f9bc9422640 1 --2- 172.21.15.38:0/2604388918 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9bb8077570 0x7f9bb8079a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:09.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.717+0000 7f9bc9422640 1 -- 172.21.15.38:0/2604388918 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 8798+0+0 (secure 0 0 0) 0x7f9bc409ba30 con 0x5639323ea520 2024-09-21T03:43:09.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.717+0000 7f9bcbc27640 1 --2- 172.21.15.38:0/2604388918 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9bb8077570 0x7f9bb8079a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:09.721 INFO:teuthology.orchestra.run.smithi038.stdout:Importing pool 2024-09-21T03:43:09.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.718+0000 7f9bd216df40 1 --2- 172.21.15.38:0/2604388918 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x5639323e3ef0 0x5639323e16a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:09.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.718+0000 7f9bd216df40 1 -- 172.21.15.38:0/2604388918 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5639323e1be0 con 0x5639323e3ef0 2024-09-21T03:43:09.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.718+0000 7f9bcb426640 1 --2- 172.21.15.38:0/2604388918 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x5639323e3ef0 0x5639323e16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:09.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.718+0000 7f9bcbc27640 1 --2- 172.21.15.38:0/2604388918 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9bb8077570 0x7f9bb8079a80 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f9bbc006bf0 tx=0x7f9bbc006b80 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:09.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.718+0000 7f9bcb426640 1 --2- 172.21.15.38:0/2604388918 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x5639323e3ef0 0x5639323e16a0 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:09.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.721+0000 7f9bcb426640 1 -- 172.21.15.38:0/2604388918 <== osd.7 v2:172.21.15.195:6818/1606106342 1 ==== osd_op_reply(1 601.00000000 [create] v125'877 uv851 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9bc0056550 con 0x5639323e3ef0 2024-09-21T03:43:09.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.721+0000 7f9bd216df40 1 -- 172.21.15.38:0/2604388918 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e125) -- 0x5639324a4d40 con 0x5639323e3ef0 2024-09-21T03:43:09.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.722+0000 7f9bcb426640 1 -- 172.21.15.38:0/2604388918 <== osd.7 v2:172.21.15.195:6818/1606106342 2 ==== osd_op_reply(2 601.00000000 [delete] v125'878 uv878 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9bc0056550 con 0x5639323e3ef0 2024-09-21T03:43:09.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.722+0000 7f9bd216df40 1 -- 172.21.15.38:0/2604388918 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5639324a50e0 con 0x5639323e3ef0 2024-09-21T03:43:09.726 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-09-21T03:43:09.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.724+0000 7f9bcb426640 1 -- 172.21.15.38:0/2604388918 <== osd.7 v2:172.21.15.195:6818/1606106342 3 ==== osd_op_reply(3 601.00000000 [create] v125'879 uv879 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9bc0056550 con 0x5639323e3ef0 2024-09-21T03:43:09.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.724+0000 7f9bd216df40 1 -- 172.21.15.38:0/2604388918 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5639324a6490 con 0x5639323e3ef0 2024-09-21T03:43:09.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.725+0000 7f9bcb426640 1 -- 172.21.15.38:0/2604388918 <== osd.7 v2:172.21.15.195:6818/1606106342 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v125'880 uv880 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9bc0056550 con 0x5639323e3ef0 2024-09-21T03:43:09.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.726+0000 7f9bd216df40 1 -- 172.21.15.38:0/2604388918 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5639323e4300 con 0x5639323e3ef0 2024-09-21T03:43:09.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.727+0000 7f9bcb426640 1 -- 172.21.15.38:0/2604388918 <== osd.7 v2:172.21.15.195:6818/1606106342 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v125'881 uv881 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9bc0056550 con 0x5639323e3ef0 2024-09-21T03:43:09.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.728+0000 7f9bd216df40 1 -- 172.21.15.38:0/2604388918 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5639324a5480 con 0x5639323e3ef0 2024-09-21T03:43:09.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.729+0000 7f9bcb426640 1 -- 172.21.15.38:0/2604388918 <== osd.7 v2:172.21.15.195:6818/1606106342 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v125'882 uv882 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9bc0056550 con 0x5639323e3ef0 2024-09-21T03:43:09.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.729+0000 7f9bd216df40 1 -- 172.21.15.38:0/2604388918 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5639324a5890 con 0x5639323e3ef0 2024-09-21T03:43:09.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.731+0000 7f9bcb426640 1 -- 172.21.15.38:0/2604388918 <== osd.7 v2:172.21.15.195:6818/1606106342 7 ==== osd_op_reply(7 602.00000000 [create] v125'883 uv857 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9bc0056550 con 0x5639323e3ef0 2024-09-21T03:43:09.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.731+0000 7f9bd216df40 1 -- 172.21.15.38:0/2604388918 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e125) -- 0x5639324a5890 con 0x5639323e3ef0 2024-09-21T03:43:09.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.733+0000 7f9bcb426640 1 -- 172.21.15.38:0/2604388918 <== osd.7 v2:172.21.15.195:6818/1606106342 8 ==== osd_op_reply(8 602.00000000 [delete] v125'884 uv884 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9bc0056550 con 0x5639323e3ef0 2024-09-21T03:43:09.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.733+0000 7f9bd216df40 1 -- 172.21.15.38:0/2604388918 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5639324a5820 con 0x5639323e3ef0 2024-09-21T03:43:09.736 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-09-21T03:43:09.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.735+0000 7f9bcb426640 1 -- 172.21.15.38:0/2604388918 <== osd.7 v2:172.21.15.195:6818/1606106342 9 ==== osd_op_reply(9 602.00000000 [create] v125'885 uv885 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9bc0056550 con 0x5639323e3ef0 2024-09-21T03:43:09.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.735+0000 7f9bd216df40 1 -- 172.21.15.38:0/2604388918 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5639324a5820 con 0x5639323e3ef0 2024-09-21T03:43:09.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.736+0000 7f9bcb426640 1 -- 172.21.15.38:0/2604388918 <== osd.7 v2:172.21.15.195:6818/1606106342 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v125'886 uv886 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9bc0056550 con 0x5639323e3ef0 2024-09-21T03:43:09.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.736+0000 7f9bd216df40 1 -- 172.21.15.38:0/2604388918 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5639324a5820 con 0x5639323e3ef0 2024-09-21T03:43:09.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.738+0000 7f9bcb426640 1 -- 172.21.15.38:0/2604388918 <== osd.7 v2:172.21.15.195:6818/1606106342 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v125'887 uv887 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9bc0056550 con 0x5639323e3ef0 2024-09-21T03:43:09.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.738+0000 7f9bd216df40 1 -- 172.21.15.38:0/2604388918 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5639324a5820 con 0x5639323e3ef0 2024-09-21T03:43:09.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.740+0000 7f9bcb426640 1 -- 172.21.15.38:0/2604388918 <== osd.7 v2:172.21.15.195:6818/1606106342 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v125'888 uv888 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9bc0056550 con 0x5639323e3ef0 2024-09-21T03:43:09.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.740+0000 7f9bd216df40 1 -- 172.21.15.38:0/2604388918 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:13 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5639324a5820 con 0x5639323e3ef0 2024-09-21T03:43:09.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.741+0000 7f9bcb426640 1 -- 172.21.15.38:0/2604388918 <== osd.7 v2:172.21.15.195:6818/1606106342 13 ==== osd_op_reply(13 10000000000.00000000 [create] v125'889 uv864 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7f9bc0056550 con 0x5639323e3ef0 2024-09-21T03:43:09.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.741+0000 7f9bd216df40 1 -- 172.21.15.38:0/2604388918 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:14 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e125) -- 0x5639324a5820 con 0x5639323e3ef0 2024-09-21T03:43:09.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.743+0000 7f9bcb426640 1 -- 172.21.15.38:0/2604388918 <== osd.7 v2:172.21.15.195:6818/1606106342 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v125'890 uv890 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f9bc0056550 con 0x5639323e3ef0 2024-09-21T03:43:09.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.743+0000 7f9bd216df40 1 -- 172.21.15.38:0/2604388918 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:15 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5639324a5820 con 0x5639323e3ef0 2024-09-21T03:43:09.746 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-09-21T03:43:09.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.744+0000 7f9bcb426640 1 -- 172.21.15.38:0/2604388918 <== osd.7 v2:172.21.15.195:6818/1606106342 15 ==== osd_op_reply(15 10000000000.00000000 [create] v125'891 uv891 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f9bc0056550 con 0x5639323e3ef0 2024-09-21T03:43:09.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.745+0000 7f9bd216df40 1 -- 172.21.15.38:0/2604388918 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:16 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5639324a5820 con 0x5639323e3ef0 2024-09-21T03:43:09.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.746+0000 7f9bcb426640 1 -- 172.21.15.38:0/2604388918 <== osd.7 v2:172.21.15.195:6818/1606106342 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v125'892 uv892 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f9bc0056550 con 0x5639323e3ef0 2024-09-21T03:43:09.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.746+0000 7f9bd216df40 1 -- 172.21.15.38:0/2604388918 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:17 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5639324a5820 con 0x5639323e3ef0 2024-09-21T03:43:09.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.748+0000 7f9bcb426640 1 -- 172.21.15.38:0/2604388918 <== osd.7 v2:172.21.15.195:6818/1606106342 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v125'893 uv893 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f9bc0056550 con 0x5639323e3ef0 2024-09-21T03:43:09.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.748+0000 7f9bd216df40 1 -- 172.21.15.38:0/2604388918 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:18 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5639324a5820 con 0x5639323e3ef0 2024-09-21T03:43:09.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.750+0000 7f9bcb426640 1 -- 172.21.15.38:0/2604388918 <== osd.7 v2:172.21.15.195:6818/1606106342 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v125'894 uv894 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f9bc0056550 con 0x5639323e3ef0 2024-09-21T03:43:09.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.750+0000 7f9bd216df40 1 -- 172.21.15.38:0/2604388918 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:19 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=585b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5639324a5be0 con 0x5639323e3ef0 2024-09-21T03:43:09.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.751+0000 7f9bcb426640 1 -- 172.21.15.38:0/2604388918 <== osd.7 v2:172.21.15.195:6818/1606106342 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v125'895 uv895 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f9bc0056550 con 0x5639323e3ef0 2024-09-21T03:43:09.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.751+0000 7f9bd216df40 1 -- 172.21.15.38:0/2604388918 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:20 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5639324a6050 con 0x5639323e3ef0 2024-09-21T03:43:09.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.753+0000 7f9bcb426640 1 -- 172.21.15.38:0/2604388918 <== osd.7 v2:172.21.15.195:6818/1606106342 20 ==== osd_op_reply(20 600.00000000 [create] v125'896 uv870 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9bc0056550 con 0x5639323e3ef0 2024-09-21T03:43:09.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.753+0000 7f9bd216df40 1 -- 172.21.15.38:0/2604388918 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:21 13.10 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e125) -- 0x5639324a6050 con 0x5639323e3ef0 2024-09-21T03:43:09.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.755+0000 7f9bcb426640 1 -- 172.21.15.38:0/2604388918 <== osd.7 v2:172.21.15.195:6818/1606106342 21 ==== osd_op_reply(21 600.00000000 [delete] v125'897 uv897 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9bc0056550 con 0x5639323e3ef0 2024-09-21T03:43:09.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.755+0000 7f9bd216df40 1 -- 172.21.15.38:0/2604388918 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:22 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5639324a5ad0 con 0x5639323e3ef0 2024-09-21T03:43:09.758 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-09-21T03:43:09.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.756+0000 7f9bcb426640 1 -- 172.21.15.38:0/2604388918 <== osd.7 v2:172.21.15.195:6818/1606106342 22 ==== osd_op_reply(22 600.00000000 [create] v125'898 uv898 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9bc0056550 con 0x5639323e3ef0 2024-09-21T03:43:09.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.756+0000 7f9bd216df40 1 -- 172.21.15.38:0/2604388918 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:23 13.10 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5639324a5ad0 con 0x5639323e3ef0 2024-09-21T03:43:09.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.758+0000 7f9bcb426640 1 -- 172.21.15.38:0/2604388918 <== osd.7 v2:172.21.15.195:6818/1606106342 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v125'899 uv899 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9bc0056550 con 0x5639323e3ef0 2024-09-21T03:43:09.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.758+0000 7f9bd216df40 1 -- 172.21.15.38:0/2604388918 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:24 13.10 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5639324a5ad0 con 0x5639323e3ef0 2024-09-21T03:43:09.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.759+0000 7f9bcb426640 1 -- 172.21.15.38:0/2604388918 <== osd.7 v2:172.21.15.195:6818/1606106342 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v125'900 uv900 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9bc0056550 con 0x5639323e3ef0 2024-09-21T03:43:09.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.759+0000 7f9bd216df40 1 -- 172.21.15.38:0/2604388918 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:25 13.10 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5639324a5ad0 con 0x5639323e3ef0 2024-09-21T03:43:09.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.761+0000 7f9bcb426640 1 -- 172.21.15.38:0/2604388918 <== osd.7 v2:172.21.15.195:6818/1606106342 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v125'901 uv901 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9bc0056550 con 0x5639323e3ef0 2024-09-21T03:43:09.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.761+0000 7f9bd216df40 1 -- 172.21.15.38:0/2604388918 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:26 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5639324a5ad0 con 0x5639323e3ef0 2024-09-21T03:43:09.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.762+0000 7f9bcb426640 1 -- 172.21.15.38:0/2604388918 <== osd.7 v2:172.21.15.195:6818/1606106342 26 ==== osd_op_reply(26 603.00000000 [create] v125'902 uv876 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9bc0056550 con 0x5639323e3ef0 2024-09-21T03:43:09.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.763+0000 7f9bd216df40 1 -- 172.21.15.38:0/2604388918 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:27 13.10 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e125) -- 0x5639324a5ad0 con 0x5639323e3ef0 2024-09-21T03:43:09.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.764+0000 7f9bcb426640 1 -- 172.21.15.38:0/2604388918 <== osd.7 v2:172.21.15.195:6818/1606106342 27 ==== osd_op_reply(27 603.00000000 [delete] v125'903 uv903 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9bc0056550 con 0x5639323e3ef0 2024-09-21T03:43:09.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.764+0000 7f9bd216df40 1 -- 172.21.15.38:0/2604388918 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:28 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5639324a5ad0 con 0x5639323e3ef0 2024-09-21T03:43:09.767 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-09-21T03:43:09.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.766+0000 7f9bcb426640 1 -- 172.21.15.38:0/2604388918 <== osd.7 v2:172.21.15.195:6818/1606106342 28 ==== osd_op_reply(28 603.00000000 [create] v125'904 uv904 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9bc0056550 con 0x5639323e3ef0 2024-09-21T03:43:09.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.766+0000 7f9bd216df40 1 -- 172.21.15.38:0/2604388918 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:29 13.10 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5639324a5ad0 con 0x5639323e3ef0 2024-09-21T03:43:09.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.767+0000 7f9bcb426640 1 -- 172.21.15.38:0/2604388918 <== osd.7 v2:172.21.15.195:6818/1606106342 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v125'905 uv905 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9bc0056550 con 0x5639323e3ef0 2024-09-21T03:43:09.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.767+0000 7f9bd216df40 1 -- 172.21.15.38:0/2604388918 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:30 13.10 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5639324a5ad0 con 0x5639323e3ef0 2024-09-21T03:43:09.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.769+0000 7f9bcb426640 1 -- 172.21.15.38:0/2604388918 <== osd.7 v2:172.21.15.195:6818/1606106342 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v125'906 uv906 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9bc0056550 con 0x5639323e3ef0 2024-09-21T03:43:09.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.769+0000 7f9bd216df40 1 -- 172.21.15.38:0/2604388918 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:31 13.10 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5639324a5ad0 con 0x5639323e3ef0 2024-09-21T03:43:09.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.770+0000 7f9bcb426640 1 -- 172.21.15.38:0/2604388918 <== osd.7 v2:172.21.15.195:6818/1606106342 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v125'907 uv907 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9bc0056550 con 0x5639323e3ef0 2024-09-21T03:43:09.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.771+0000 7f9bd216df40 1 --2- 172.21.15.38:0/2604388918 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x5639324a6830 0x5639324a8c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:09.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.771+0000 7f9bd216df40 1 -- 172.21.15.38:0/2604388918 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5639324a9190 con 0x5639324a6830 2024-09-21T03:43:09.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.771+0000 7f9bcbc27640 1 --2- 172.21.15.38:0/2604388918 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x5639324a6830 0x5639324a8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:09.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.771+0000 7f9bcbc27640 1 --2- 172.21.15.38:0/2604388918 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x5639324a6830 0x5639324a8c50 crc :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:09.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.773+0000 7f9bcbc27640 1 -- 172.21.15.38:0/2604388918 <== osd.6 v2:172.21.15.195:6826/4055825675 1 ==== osd_op_reply(32 1.00000000.inode [create] v125'637 uv620 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f9bbc0015d0 con 0x5639324a6830 2024-09-21T03:43:09.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.773+0000 7f9bd216df40 1 -- 172.21.15.38:0/2604388918 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e125) -- 0x5639324a9530 con 0x5639324a6830 2024-09-21T03:43:09.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.775+0000 7f9bcbc27640 1 -- 172.21.15.38:0/2604388918 <== osd.6 v2:172.21.15.195:6826/4055825675 2 ==== osd_op_reply(33 1.00000000.inode [delete] v125'638 uv638 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f9bbc0015d0 con 0x5639324a6830 2024-09-21T03:43:09.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.775+0000 7f9bd216df40 1 -- 172.21.15.38:0/2604388918 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5639324a98d0 con 0x5639324a6830 2024-09-21T03:43:09.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.776+0000 7f9bcbc27640 1 -- 172.21.15.38:0/2604388918 <== osd.6 v2:172.21.15.195:6826/4055825675 3 ==== osd_op_reply(34 1.00000000.inode [create] v125'639 uv639 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f9bbc0015d0 con 0x5639324a6830 2024-09-21T03:43:09.778 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-09-21T03:43:09.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.776+0000 7f9bd216df40 1 -- 172.21.15.38:0/2604388918 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5639324aa120 con 0x5639324a6830 2024-09-21T03:43:09.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.778+0000 7f9bcbc27640 1 -- 172.21.15.38:0/2604388918 <== osd.6 v2:172.21.15.195:6826/4055825675 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v125'640 uv640 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f9bbc0015d0 con 0x5639324a6830 2024-09-21T03:43:09.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.778+0000 7f9bd216df40 1 -- 172.21.15.38:0/2604388918 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5639324aa4c0 con 0x5639324a6830 2024-09-21T03:43:09.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.779+0000 7f9bcbc27640 1 -- 172.21.15.38:0/2604388918 <== osd.6 v2:172.21.15.195:6826/4055825675 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v125'641 uv641 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f9bbc0015d0 con 0x5639324a6830 2024-09-21T03:43:09.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.779+0000 7f9bd216df40 1 -- 172.21.15.38:0/2604388918 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5639324aa860 con 0x5639324a6830 2024-09-21T03:43:09.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.780+0000 7f9bcbc27640 1 -- 172.21.15.38:0/2604388918 <== osd.6 v2:172.21.15.195:6826/4055825675 6 ==== osd_op_reply(37 200.00000000 [create] v125'642 uv625 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9bbc0015d0 con 0x5639324a6830 2024-09-21T03:43:09.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.781+0000 7f9bd216df40 1 -- 172.21.15.38:0/2604388918 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e125) -- 0x5639324aac00 con 0x5639324a6830 2024-09-21T03:43:09.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.782+0000 7f9bcbc27640 1 -- 172.21.15.38:0/2604388918 <== osd.6 v2:172.21.15.195:6826/4055825675 7 ==== osd_op_reply(38 200.00000000 [delete] v125'643 uv643 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9bbc0015d0 con 0x5639324a6830 2024-09-21T03:43:09.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.782+0000 7f9bd216df40 1 -- 172.21.15.38:0/2604388918 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5639324aafa0 con 0x5639324a6830 2024-09-21T03:43:09.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.783+0000 7f9bcbc27640 1 -- 172.21.15.38:0/2604388918 <== osd.6 v2:172.21.15.195:6826/4055825675 8 ==== osd_op_reply(39 200.00000000 [create] v125'644 uv644 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9bbc0015d0 con 0x5639324a6830 2024-09-21T03:43:09.785 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-09-21T03:43:09.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.783+0000 7f9bd216df40 1 -- 172.21.15.38:0/2604388918 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5639324a9d50 con 0x5639324a6830 2024-09-21T03:43:09.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.785+0000 7f9bcbc27640 1 -- 172.21.15.38:0/2604388918 <== osd.6 v2:172.21.15.195:6826/4055825675 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v125'645 uv645 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9bbc0015d0 con 0x5639324a6830 2024-09-21T03:43:09.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.785+0000 7f9bd216df40 1 -- 172.21.15.38:0/2604388918 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5639324a9d50 con 0x5639324a6830 2024-09-21T03:43:09.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.786+0000 7f9bcbc27640 1 -- 172.21.15.38:0/2604388918 <== osd.6 v2:172.21.15.195:6826/4055825675 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v125'646 uv646 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9bbc0015d0 con 0x5639324a6830 2024-09-21T03:43:09.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.786+0000 7f9bd216df40 1 -- 172.21.15.38:0/2604388918 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:42 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5639324a9d50 con 0x5639324a6830 2024-09-21T03:43:09.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.788+0000 7f9bcbc27640 1 -- 172.21.15.38:0/2604388918 <== osd.6 v2:172.21.15.195:6826/4055825675 11 ==== osd_op_reply(42 200.00000001 [create] v125'647 uv630 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9bbc0015d0 con 0x5639324a6830 2024-09-21T03:43:09.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.788+0000 7f9bd216df40 1 -- 172.21.15.38:0/2604388918 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:43 13.14 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e125) -- 0x5639324a9d50 con 0x5639324a6830 2024-09-21T03:43:09.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.789+0000 7f9bcbc27640 1 -- 172.21.15.38:0/2604388918 <== osd.6 v2:172.21.15.195:6826/4055825675 12 ==== osd_op_reply(43 200.00000001 [delete] v125'648 uv648 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9bbc0015d0 con 0x5639324a6830 2024-09-21T03:43:09.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.789+0000 7f9bd216df40 1 -- 172.21.15.38:0/2604388918 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:44 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5639324a9d50 con 0x5639324a6830 2024-09-21T03:43:09.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.790+0000 7f9bcbc27640 1 -- 172.21.15.38:0/2604388918 <== osd.6 v2:172.21.15.195:6826/4055825675 13 ==== osd_op_reply(44 200.00000001 [create] v125'649 uv649 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9bbc0015d0 con 0x5639324a6830 2024-09-21T03:43:09.792 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-09-21T03:43:09.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.790+0000 7f9bd216df40 1 -- 172.21.15.38:0/2604388918 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:45 13.14 13:2e2fa760:::200.00000001:head [write 0~17490 in=17490b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5639324a9d50 con 0x5639324a6830 2024-09-21T03:43:09.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.792+0000 7f9bcbc27640 1 -- 172.21.15.38:0/2604388918 <== osd.6 v2:172.21.15.195:6826/4055825675 14 ==== osd_op_reply(45 200.00000001 [write 0~17490] v125'650 uv650 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9bbc0015d0 con 0x5639324a6830 2024-09-21T03:43:09.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.792+0000 7f9bd216df40 1 -- 172.21.15.38:0/2604388918 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:46 13.14 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5639324a9d50 con 0x5639324a6830 2024-09-21T03:43:09.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.793+0000 7f9bcbc27640 1 -- 172.21.15.38:0/2604388918 <== osd.6 v2:172.21.15.195:6826/4055825675 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v125'651 uv651 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9bbc0015d0 con 0x5639324a6830 2024-09-21T03:43:09.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.793+0000 7f9bd216df40 1 -- 172.21.15.38:0/2604388918 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:47 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5639324a9d50 con 0x5639324a6830 2024-09-21T03:43:09.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.794+0000 7f9bcbc27640 1 -- 172.21.15.38:0/2604388918 <== osd.6 v2:172.21.15.195:6826/4055825675 16 ==== osd_op_reply(47 606.00000000 [create] v125'652 uv636 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9bbc0015d0 con 0x5639324a6830 2024-09-21T03:43:09.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.795+0000 7f9bd216df40 1 -- 172.21.15.38:0/2604388918 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:48 13.14 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e125) -- 0x5639324a9d50 con 0x5639324a6830 2024-09-21T03:43:09.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.796+0000 7f9bcbc27640 1 -- 172.21.15.38:0/2604388918 <== osd.6 v2:172.21.15.195:6826/4055825675 17 ==== osd_op_reply(48 606.00000000 [delete] v125'653 uv653 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9bbc0015d0 con 0x5639324a6830 2024-09-21T03:43:09.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.796+0000 7f9bd216df40 1 -- 172.21.15.38:0/2604388918 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:49 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5639324a9d50 con 0x5639324a6830 2024-09-21T03:43:09.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.797+0000 7f9bcbc27640 1 -- 172.21.15.38:0/2604388918 <== osd.6 v2:172.21.15.195:6826/4055825675 18 ==== osd_op_reply(49 606.00000000 [create] v125'654 uv654 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9bbc0015d0 con 0x5639324a6830 2024-09-21T03:43:09.799 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-09-21T03:43:09.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.797+0000 7f9bd216df40 1 -- 172.21.15.38:0/2604388918 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:50 13.14 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5639324a9d50 con 0x5639324a6830 2024-09-21T03:43:09.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.799+0000 7f9bcbc27640 1 -- 172.21.15.38:0/2604388918 <== osd.6 v2:172.21.15.195:6826/4055825675 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v125'655 uv655 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9bbc0015d0 con 0x5639324a6830 2024-09-21T03:43:09.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.799+0000 7f9bd216df40 1 -- 172.21.15.38:0/2604388918 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:51 13.14 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5639324a9d50 con 0x5639324a6830 2024-09-21T03:43:09.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.800+0000 7f9bcbc27640 1 -- 172.21.15.38:0/2604388918 <== osd.6 v2:172.21.15.195:6826/4055825675 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v125'656 uv656 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9bbc0015d0 con 0x5639324a6830 2024-09-21T03:43:09.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.800+0000 7f9bd216df40 1 -- 172.21.15.38:0/2604388918 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:52 13.14 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5639324a9d50 con 0x5639324a6830 2024-09-21T03:43:09.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.801+0000 7f9bcbc27640 1 -- 172.21.15.38:0/2604388918 <== osd.6 v2:172.21.15.195:6826/4055825675 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v125'657 uv657 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9bbc0015d0 con 0x5639324a6830 2024-09-21T03:43:09.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.801+0000 7f9bd216df40 1 --2- 172.21.15.38:0/2604388918 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x5639324ab960 0x5639324add80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:09.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.801+0000 7f9bd216df40 1 -- 172.21.15.38:0/2604388918 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:53 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5639324ae2c0 con 0x5639324ab960 2024-09-21T03:43:09.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.801+0000 7f9bcb426640 1 --2- 172.21.15.38:0/2604388918 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x5639324ab960 0x5639324add80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:09.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.802+0000 7f9bcb426640 1 --2- 172.21.15.38:0/2604388918 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x5639324ab960 0x5639324add80 crc :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:09.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.803+0000 7f9bcb426640 1 -- 172.21.15.38:0/2604388918 <== osd.0 v2:172.21.15.38:6800/2381639433 1 ==== osd_op_reply(53 607.00000000 [create] v125'171 uv170 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9bc0056550 con 0x5639324ab960 2024-09-21T03:43:09.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.803+0000 7f9bd216df40 1 -- 172.21.15.38:0/2604388918 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:54 13.1c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e125) -- 0x5639324ae2c0 con 0x5639324ab960 2024-09-21T03:43:09.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.804+0000 7f9bcb426640 1 -- 172.21.15.38:0/2604388918 <== osd.0 v2:172.21.15.38:6800/2381639433 2 ==== osd_op_reply(54 607.00000000 [delete] v125'172 uv172 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9bc0056550 con 0x5639324ab960 2024-09-21T03:43:09.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.804+0000 7f9bd216df40 1 -- 172.21.15.38:0/2604388918 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:55 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5639324ae6e0 con 0x5639324ab960 2024-09-21T03:43:09.807 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-09-21T03:43:09.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.805+0000 7f9bcb426640 1 -- 172.21.15.38:0/2604388918 <== osd.0 v2:172.21.15.38:6800/2381639433 3 ==== osd_op_reply(55 607.00000000 [create] v125'173 uv173 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9bc0056550 con 0x5639324ab960 2024-09-21T03:43:09.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.805+0000 7f9bd216df40 1 -- 172.21.15.38:0/2604388918 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:56 13.1c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5639324af6f0 con 0x5639324ab960 2024-09-21T03:43:09.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.806+0000 7f9bcb426640 1 -- 172.21.15.38:0/2604388918 <== osd.0 v2:172.21.15.38:6800/2381639433 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v125'174 uv174 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9bc0056550 con 0x5639324ab960 2024-09-21T03:43:09.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.806+0000 7f9bd216df40 1 -- 172.21.15.38:0/2604388918 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:57 13.1c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5639324af6f0 con 0x5639324ab960 2024-09-21T03:43:09.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.807+0000 7f9bcb426640 1 -- 172.21.15.38:0/2604388918 <== osd.0 v2:172.21.15.38:6800/2381639433 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v125'175 uv175 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9bc0056550 con 0x5639324ab960 2024-09-21T03:43:09.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.807+0000 7f9bd216df40 1 -- 172.21.15.38:0/2604388918 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:58 13.1c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5639324ae6e0 con 0x5639324ab960 2024-09-21T03:43:09.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.808+0000 7f9bcb426640 1 -- 172.21.15.38:0/2604388918 <== osd.0 v2:172.21.15.38:6800/2381639433 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v125'176 uv176 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9bc0056550 con 0x5639324ab960 2024-09-21T03:43:09.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.808+0000 7f9bd216df40 1 --2- 172.21.15.38:0/2604388918 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x5639324aed60 0x5639324b1180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:09.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.808+0000 7f9bd216df40 1 -- 172.21.15.38:0/2604388918 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:59 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5639324b1750 con 0x5639324aed60 2024-09-21T03:43:09.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.808+0000 7f9bcc428640 1 --2- 172.21.15.38:0/2604388918 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x5639324aed60 0x5639324b1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:09.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.808+0000 7f9bcc428640 1 --2- 172.21.15.38:0/2604388918 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x5639324aed60 0x5639324b1180 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:09.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.810+0000 7f9bcc428640 1 -- 172.21.15.38:0/2604388918 <== osd.3 v2:172.21.15.38:6808/1286810294 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v125'124 uv123 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f9bc402c070 con 0x5639324aed60 2024-09-21T03:43:09.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.810+0000 7f9bd216df40 1 -- 172.21.15.38:0/2604388918 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:60 13.6 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e125) -- 0x5639324b1c50 con 0x5639324aed60 2024-09-21T03:43:09.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.811+0000 7f9bcc428640 1 -- 172.21.15.38:0/2604388918 <== osd.3 v2:172.21.15.38:6808/1286810294 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v125'125 uv125 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f9bc402c070 con 0x5639324aed60 2024-09-21T03:43:09.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.811+0000 7f9bd216df40 1 -- 172.21.15.38:0/2604388918 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:61 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5639324b2150 con 0x5639324aed60 2024-09-21T03:43:09.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.812+0000 7f9bcc428640 1 -- 172.21.15.38:0/2604388918 <== osd.3 v2:172.21.15.38:6808/1286810294 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v125'126 uv126 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f9bc402c070 con 0x5639324aed60 2024-09-21T03:43:09.814 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-09-21T03:43:09.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.812+0000 7f9bd216df40 1 -- 172.21.15.38:0/2604388918 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:62 13.6 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5639324b2a10 con 0x5639324aed60 2024-09-21T03:43:09.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.813+0000 7f9bcc428640 1 -- 172.21.15.38:0/2604388918 <== osd.3 v2:172.21.15.38:6808/1286810294 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v125'127 uv127 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f9bc402c070 con 0x5639324aed60 2024-09-21T03:43:09.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.813+0000 7f9bd216df40 1 -- 172.21.15.38:0/2604388918 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5639324b2db0 con 0x5639324ab960 2024-09-21T03:43:09.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.814+0000 7f9bcb426640 1 -- 172.21.15.38:0/2604388918 <== osd.0 v2:172.21.15.38:6800/2381639433 7 ==== osd_op_reply(63 608.00000000 [create] v125'171 uv170 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9bc0056550 con 0x5639324ab960 2024-09-21T03:43:09.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.814+0000 7f9bd216df40 1 -- 172.21.15.38:0/2604388918 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e125) -- 0x5639324b2db0 con 0x5639324ab960 2024-09-21T03:43:09.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.816+0000 7f9bcb426640 1 -- 172.21.15.38:0/2604388918 <== osd.0 v2:172.21.15.38:6800/2381639433 8 ==== osd_op_reply(64 608.00000000 [delete] v125'172 uv172 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9bc0056550 con 0x5639324ab960 2024-09-21T03:43:09.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.816+0000 7f9bd216df40 1 -- 172.21.15.38:0/2604388918 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5639324b2db0 con 0x5639324ab960 2024-09-21T03:43:09.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.817+0000 7f9bcb426640 1 -- 172.21.15.38:0/2604388918 <== osd.0 v2:172.21.15.38:6800/2381639433 9 ==== osd_op_reply(65 608.00000000 [create] v125'173 uv173 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9bc0056550 con 0x5639324ab960 2024-09-21T03:43:09.819 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-09-21T03:43:09.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.817+0000 7f9bd216df40 1 -- 172.21.15.38:0/2604388918 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5639324b3dc0 con 0x5639324ab960 2024-09-21T03:43:09.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.818+0000 7f9bcb426640 1 -- 172.21.15.38:0/2604388918 <== osd.0 v2:172.21.15.38:6800/2381639433 10 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v125'174 uv174 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9bc0056550 con 0x5639324ab960 2024-09-21T03:43:09.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.818+0000 7f9bd216df40 1 -- 172.21.15.38:0/2604388918 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5639324b3dc0 con 0x5639324ab960 2024-09-21T03:43:09.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.819+0000 7f9bcb426640 1 -- 172.21.15.38:0/2604388918 <== osd.0 v2:172.21.15.38:6800/2381639433 11 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v125'175 uv175 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9bc0056550 con 0x5639324ab960 2024-09-21T03:43:09.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.819+0000 7f9bd216df40 1 -- 172.21.15.38:0/2604388918 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5639324b2db0 con 0x5639324ab960 2024-09-21T03:43:09.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.820+0000 7f9bcb426640 1 -- 172.21.15.38:0/2604388918 <== osd.0 v2:172.21.15.38:6800/2381639433 12 ==== osd_op_reply(68 608.00000000 [omap-set-header] v125'176 uv176 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9bc0056550 con 0x5639324ab960 2024-09-21T03:43:09.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.820+0000 7f9bd216df40 1 -- 172.21.15.38:0/2604388918 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5639324b2db0 con 0x5639324aed60 2024-09-21T03:43:09.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.822+0000 7f9bcc428640 1 -- 172.21.15.38:0/2604388918 <== osd.3 v2:172.21.15.38:6808/1286810294 5 ==== osd_op_reply(69 604.00000000 [create] v125'316 uv310 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9bc402c070 con 0x5639324aed60 2024-09-21T03:43:09.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.822+0000 7f9bd216df40 1 -- 172.21.15.38:0/2604388918 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e125) -- 0x5639324b3150 con 0x5639324aed60 2024-09-21T03:43:09.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.823+0000 7f9bcc428640 1 -- 172.21.15.38:0/2604388918 <== osd.3 v2:172.21.15.38:6808/1286810294 6 ==== osd_op_reply(70 604.00000000 [delete] v125'317 uv317 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9bc402c070 con 0x5639324aed60 2024-09-21T03:43:09.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.823+0000 7f9bd216df40 1 -- 172.21.15.38:0/2604388918 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5639324b34f0 con 0x5639324aed60 2024-09-21T03:43:09.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.824+0000 7f9bcc428640 1 -- 172.21.15.38:0/2604388918 <== osd.3 v2:172.21.15.38:6808/1286810294 7 ==== osd_op_reply(71 604.00000000 [create] v125'318 uv318 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9bc402c070 con 0x5639324aed60 2024-09-21T03:43:09.826 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-09-21T03:43:09.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.824+0000 7f9bd216df40 1 -- 172.21.15.38:0/2604388918 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5639324b48a0 con 0x5639324aed60 2024-09-21T03:43:09.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.825+0000 7f9bcc428640 1 -- 172.21.15.38:0/2604388918 <== osd.3 v2:172.21.15.38:6808/1286810294 8 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v125'319 uv319 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9bc402c070 con 0x5639324aed60 2024-09-21T03:43:09.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.825+0000 7f9bd216df40 1 -- 172.21.15.38:0/2604388918 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5639324b48a0 con 0x5639324aed60 2024-09-21T03:43:09.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.826+0000 7f9bcc428640 1 -- 172.21.15.38:0/2604388918 <== osd.3 v2:172.21.15.38:6808/1286810294 9 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v125'320 uv320 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9bc402c070 con 0x5639324aed60 2024-09-21T03:43:09.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.826+0000 7f9bd216df40 1 -- 172.21.15.38:0/2604388918 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5639324b3890 con 0x5639324aed60 2024-09-21T03:43:09.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.827+0000 7f9bcc428640 1 -- 172.21.15.38:0/2604388918 <== osd.3 v2:172.21.15.38:6808/1286810294 10 ==== osd_op_reply(74 604.00000000 [omap-set-header] v125'321 uv321 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9bc402c070 con 0x5639324aed60 2024-09-21T03:43:09.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.827+0000 7f9bd216df40 1 -- 172.21.15.38:0/2604388918 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5639324b3890 con 0x5639324aed60 2024-09-21T03:43:09.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.828+0000 7f9bcc428640 1 -- 172.21.15.38:0/2604388918 <== osd.3 v2:172.21.15.38:6808/1286810294 11 ==== osd_op_reply(75 500.00000000 [create] v125'322 uv315 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9bc402c070 con 0x5639324aed60 2024-09-21T03:43:09.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.828+0000 7f9bd216df40 1 -- 172.21.15.38:0/2604388918 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e125) -- 0x5639324b3890 con 0x5639324aed60 2024-09-21T03:43:09.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.830+0000 7f9bcc428640 1 -- 172.21.15.38:0/2604388918 <== osd.3 v2:172.21.15.38:6808/1286810294 12 ==== osd_op_reply(76 500.00000000 [delete] v125'323 uv323 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9bc402c070 con 0x5639324aed60 2024-09-21T03:43:09.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.830+0000 7f9bd216df40 1 -- 172.21.15.38:0/2604388918 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5639324b3890 con 0x5639324aed60 2024-09-21T03:43:09.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.831+0000 7f9bcc428640 1 -- 172.21.15.38:0/2604388918 <== osd.3 v2:172.21.15.38:6808/1286810294 13 ==== osd_op_reply(77 500.00000000 [create] v125'324 uv324 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9bc402c070 con 0x5639324aed60 2024-09-21T03:43:09.833 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-09-21T03:43:09.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.831+0000 7f9bd216df40 1 -- 172.21.15.38:0/2604388918 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5639324b3890 con 0x5639324aed60 2024-09-21T03:43:09.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.832+0000 7f9bcc428640 1 -- 172.21.15.38:0/2604388918 <== osd.3 v2:172.21.15.38:6808/1286810294 14 ==== osd_op_reply(78 500.00000000 [write 0~90] v125'325 uv325 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9bc402c070 con 0x5639324aed60 2024-09-21T03:43:09.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.832+0000 7f9bd216df40 1 -- 172.21.15.38:0/2604388918 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5639324b3890 con 0x5639324aed60 2024-09-21T03:43:09.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.833+0000 7f9bcc428640 1 -- 172.21.15.38:0/2604388918 <== osd.3 v2:172.21.15.38:6808/1286810294 15 ==== osd_op_reply(79 500.00000000 [omap-set-header] v125'326 uv326 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9bc402c070 con 0x5639324aed60 2024-09-21T03:43:09.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.833+0000 7f9bd216df40 1 -- 172.21.15.38:0/2604388918 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:80 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5639324b3890 con 0x5639324ab960 2024-09-21T03:43:09.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.834+0000 7f9bcb426640 1 -- 172.21.15.38:0/2604388918 <== osd.0 v2:172.21.15.38:6800/2381639433 13 ==== osd_op_reply(80 mds_snaptable [create] v125'143 uv142 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f9bc0056550 con 0x5639324ab960 2024-09-21T03:43:09.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.834+0000 7f9bd216df40 1 -- 172.21.15.38:0/2604388918 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:81 13.d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e125) -- 0x5639324b3890 con 0x5639324ab960 2024-09-21T03:43:09.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.835+0000 7f9bcb426640 1 -- 172.21.15.38:0/2604388918 <== osd.0 v2:172.21.15.38:6800/2381639433 14 ==== osd_op_reply(81 mds_snaptable [delete] v125'144 uv144 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f9bc0056550 con 0x5639324ab960 2024-09-21T03:43:09.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.835+0000 7f9bd216df40 1 -- 172.21.15.38:0/2604388918 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:82 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5639324b3890 con 0x5639324ab960 2024-09-21T03:43:09.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.836+0000 7f9bcb426640 1 -- 172.21.15.38:0/2604388918 <== osd.0 v2:172.21.15.38:6800/2381639433 15 ==== osd_op_reply(82 mds_snaptable [create] v125'145 uv145 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f9bc0056550 con 0x5639324ab960 2024-09-21T03:43:09.838 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-09-21T03:43:09.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.836+0000 7f9bd216df40 1 -- 172.21.15.38:0/2604388918 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:83 13.d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5639324b3890 con 0x5639324ab960 2024-09-21T03:43:09.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.837+0000 7f9bcb426640 1 -- 172.21.15.38:0/2604388918 <== osd.0 v2:172.21.15.38:6800/2381639433 16 ==== osd_op_reply(83 mds_snaptable [write 0~70] v125'146 uv146 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f9bc0056550 con 0x5639324ab960 2024-09-21T03:43:09.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.837+0000 7f9bd216df40 1 -- 172.21.15.38:0/2604388918 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:84 13.d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5639324b3890 con 0x5639324ab960 2024-09-21T03:43:09.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.838+0000 7f9bcb426640 1 -- 172.21.15.38:0/2604388918 <== osd.0 v2:172.21.15.38:6800/2381639433 17 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v125'147 uv147 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f9bc0056550 con 0x5639324ab960 2024-09-21T03:43:09.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.838+0000 7f9bd216df40 1 --2- 172.21.15.38:0/2604388918 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x5639324b3f10 0x5639324b6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:09.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.838+0000 7f9bd216df40 1 -- 172.21.15.38:0/2604388918 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5639324b68b0 con 0x5639324b3f10 2024-09-21T03:43:09.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.838+0000 7f9bcbc27640 1 --2- 172.21.15.38:0/2604388918 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x5639324b3f10 0x5639324b6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:09.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.839+0000 7f9bcbc27640 1 --2- 172.21.15.38:0/2604388918 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x5639324b3f10 0x5639324b6330 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:09.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.840+0000 7f9bcbc27640 1 -- 172.21.15.38:0/2604388918 <== osd.1 v2:172.21.15.38:6816/1491818732 1 ==== osd_op_reply(85 605.00000000 [create] v125'171 uv170 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9bbc0015d0 con 0x5639324b3f10 2024-09-21T03:43:09.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.840+0000 7f9bd216df40 1 -- 172.21.15.38:0/2604388918 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e125) -- 0x5639324b6970 con 0x5639324b3f10 2024-09-21T03:43:09.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.841+0000 7f9bcbc27640 1 -- 172.21.15.38:0/2604388918 <== osd.1 v2:172.21.15.38:6816/1491818732 2 ==== osd_op_reply(86 605.00000000 [delete] v125'172 uv172 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9bbc0015d0 con 0x5639324b3f10 2024-09-21T03:43:09.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.841+0000 7f9bd216df40 1 -- 172.21.15.38:0/2604388918 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5639324b6d90 con 0x5639324b3f10 2024-09-21T03:43:09.844 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-09-21T03:43:09.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.842+0000 7f9bcbc27640 1 -- 172.21.15.38:0/2604388918 <== osd.1 v2:172.21.15.38:6816/1491818732 3 ==== osd_op_reply(87 605.00000000 [create] v125'173 uv173 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9bbc0015d0 con 0x5639324b3f10 2024-09-21T03:43:09.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.842+0000 7f9bd216df40 1 -- 172.21.15.38:0/2604388918 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5639324b7da0 con 0x5639324b3f10 2024-09-21T03:43:09.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.844+0000 7f9bcbc27640 1 -- 172.21.15.38:0/2604388918 <== osd.1 v2:172.21.15.38:6816/1491818732 4 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v125'174 uv174 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9bbc0015d0 con 0x5639324b3f10 2024-09-21T03:43:09.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.844+0000 7f9bd216df40 1 -- 172.21.15.38:0/2604388918 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5639324b7da0 con 0x5639324b3f10 2024-09-21T03:43:09.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.845+0000 7f9bcbc27640 1 -- 172.21.15.38:0/2604388918 <== osd.1 v2:172.21.15.38:6816/1491818732 5 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v125'175 uv175 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9bbc0015d0 con 0x5639324b3f10 2024-09-21T03:43:09.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.845+0000 7f9bd216df40 1 -- 172.21.15.38:0/2604388918 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5639324b6d90 con 0x5639324b3f10 2024-09-21T03:43:09.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.846+0000 7f9bcbc27640 1 -- 172.21.15.38:0/2604388918 <== osd.1 v2:172.21.15.38:6816/1491818732 6 ==== osd_op_reply(90 605.00000000 [omap-set-header] v125'176 uv176 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9bbc0015d0 con 0x5639324b3f10 2024-09-21T03:43:09.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.846+0000 7f9bd216df40 1 -- 172.21.15.38:0/2604388918 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5639324b6d90 con 0x5639324b3f10 2024-09-21T03:43:09.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.847+0000 7f9bcbc27640 1 -- 172.21.15.38:0/2604388918 <== osd.1 v2:172.21.15.38:6816/1491818732 7 ==== osd_op_reply(91 mds0_inotable [create] v125'284 uv283 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f9bbc0015d0 con 0x5639324b3f10 2024-09-21T03:43:09.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.847+0000 7f9bd216df40 1 -- 172.21.15.38:0/2604388918 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e125) -- 0x5639324b6d90 con 0x5639324b3f10 2024-09-21T03:43:09.850 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.848+0000 7f9bcbc27640 1 -- 172.21.15.38:0/2604388918 <== osd.1 v2:172.21.15.38:6816/1491818732 8 ==== osd_op_reply(92 mds0_inotable [delete] v125'285 uv285 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f9bbc0015d0 con 0x5639324b3f10 2024-09-21T03:43:09.850 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.848+0000 7f9bd216df40 1 -- 172.21.15.38:0/2604388918 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5639324b6d90 con 0x5639324b3f10 2024-09-21T03:43:09.851 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.849+0000 7f9bcbc27640 1 -- 172.21.15.38:0/2604388918 <== osd.1 v2:172.21.15.38:6816/1491818732 9 ==== osd_op_reply(93 mds0_inotable [create] v125'286 uv286 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f9bbc0015d0 con 0x5639324b3f10 2024-09-21T03:43:09.851 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-09-21T03:43:09.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.850+0000 7f9bd216df40 1 -- 172.21.15.38:0/2604388918 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:94 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5639324b6d90 con 0x5639324b3f10 2024-09-21T03:43:09.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.851+0000 7f9bcbc27640 1 -- 172.21.15.38:0/2604388918 <== osd.1 v2:172.21.15.38:6816/1491818732 10 ==== osd_op_reply(94 mds0_inotable [write 0~34] v125'287 uv287 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f9bbc0015d0 con 0x5639324b3f10 2024-09-21T03:43:09.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.851+0000 7f9bd216df40 1 -- 172.21.15.38:0/2604388918 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:95 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5639324b25e0 con 0x5639324b3f10 2024-09-21T03:43:09.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.852+0000 7f9bcbc27640 1 -- 172.21.15.38:0/2604388918 <== osd.1 v2:172.21.15.38:6816/1491818732 11 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v125'288 uv288 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f9bbc0015d0 con 0x5639324b3f10 2024-09-21T03:43:09.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.852+0000 7f9bd216df40 1 -- 172.21.15.38:0/2604388918 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:96 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5639324b25e0 con 0x5639324b3f10 2024-09-21T03:43:09.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.853+0000 7f9bcbc27640 1 -- 172.21.15.38:0/2604388918 <== osd.1 v2:172.21.15.38:6816/1491818732 12 ==== osd_op_reply(96 100.00000000 [create] v125'289 uv282 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9bbc0015d0 con 0x5639324b3f10 2024-09-21T03:43:09.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.853+0000 7f9bd216df40 1 -- 172.21.15.38:0/2604388918 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:97 13.13 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e125) -- 0x5639324b2560 con 0x5639324b3f10 2024-09-21T03:43:09.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.854+0000 7f9bcbc27640 1 -- 172.21.15.38:0/2604388918 <== osd.1 v2:172.21.15.38:6816/1491818732 13 ==== osd_op_reply(97 100.00000000 [delete] v125'290 uv290 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9bbc0015d0 con 0x5639324b3f10 2024-09-21T03:43:09.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.854+0000 7f9bd216df40 1 -- 172.21.15.38:0/2604388918 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:98 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5639324b2560 con 0x5639324b3f10 2024-09-21T03:43:09.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.855+0000 7f9bcbc27640 1 -- 172.21.15.38:0/2604388918 <== osd.1 v2:172.21.15.38:6816/1491818732 14 ==== osd_op_reply(98 100.00000000 [create] v125'291 uv291 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9bbc0015d0 con 0x5639324b3f10 2024-09-21T03:43:09.857 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-09-21T03:43:09.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.855+0000 7f9bd216df40 1 -- 172.21.15.38:0/2604388918 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:99 13.13 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5639324b2560 con 0x5639324b3f10 2024-09-21T03:43:09.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.856+0000 7f9bcbc27640 1 -- 172.21.15.38:0/2604388918 <== osd.1 v2:172.21.15.38:6816/1491818732 15 ==== osd_op_reply(99 100.00000000 [omap-set-header] v125'292 uv292 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9bbc0015d0 con 0x5639324b3f10 2024-09-21T03:43:09.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.856+0000 7f9bd216df40 1 -- 172.21.15.38:0/2604388918 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:100 13.13 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5639324b9ba0 con 0x5639324b3f10 2024-09-21T03:43:09.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.857+0000 7f9bcbc27640 1 -- 172.21.15.38:0/2604388918 <== osd.1 v2:172.21.15.38:6816/1491818732 16 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v125'293 uv293 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9bbc0015d0 con 0x5639324b3f10 2024-09-21T03:43:09.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.857+0000 7f9bd216df40 1 -- 172.21.15.38:0/2604388918 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5639324b7240 con 0x5639324aed60 2024-09-21T03:43:09.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.859+0000 7f9bcc428640 1 -- 172.21.15.38:0/2604388918 <== osd.3 v2:172.21.15.38:6808/1286810294 16 ==== osd_op_reply(101 mds0_sessionmap [create] v125'124 uv123 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7f9bc402c070 con 0x5639324aed60 2024-09-21T03:43:09.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.859+0000 7f9bd216df40 1 -- 172.21.15.38:0/2604388918 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e125) -- 0x5639324b7240 con 0x5639324aed60 2024-09-21T03:43:09.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.860+0000 7f9bcc428640 1 -- 172.21.15.38:0/2604388918 <== osd.3 v2:172.21.15.38:6808/1286810294 17 ==== osd_op_reply(102 mds0_sessionmap [delete] v125'125 uv125 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f9bc402c070 con 0x5639324aed60 2024-09-21T03:43:09.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.860+0000 7f9bd216df40 1 -- 172.21.15.38:0/2604388918 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:103 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5639324b7240 con 0x5639324aed60 2024-09-21T03:43:09.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.861+0000 7f9bcc428640 1 -- 172.21.15.38:0/2604388918 <== osd.3 v2:172.21.15.38:6808/1286810294 18 ==== osd_op_reply(103 mds0_sessionmap [create] v125'126 uv126 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f9bc402c070 con 0x5639324aed60 2024-09-21T03:43:09.863 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-09-21T03:43:09.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.861+0000 7f9bd216df40 1 -- 172.21.15.38:0/2604388918 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:104 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5639324b7240 con 0x5639324aed60 2024-09-21T03:43:09.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.862+0000 7f9bcc428640 1 -- 172.21.15.38:0/2604388918 <== osd.3 v2:172.21.15.38:6808/1286810294 19 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v125'127 uv127 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f9bc402c070 con 0x5639324aed60 2024-09-21T03:43:09.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.862+0000 7f9bd216df40 1 -- 172.21.15.38:0/2604388918 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5639324b7240 con 0x5639324aed60 2024-09-21T03:43:09.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.863+0000 7f9bcc428640 1 -- 172.21.15.38:0/2604388918 <== osd.3 v2:172.21.15.38:6808/1286810294 20 ==== osd_op_reply(105 609.00000000 [create] v125'171 uv170 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9bc402c070 con 0x5639324aed60 2024-09-21T03:43:09.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.863+0000 7f9bd216df40 1 -- 172.21.15.38:0/2604388918 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e125) -- 0x5639324b7240 con 0x5639324aed60 2024-09-21T03:43:09.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.864+0000 7f9bcc428640 1 -- 172.21.15.38:0/2604388918 <== osd.3 v2:172.21.15.38:6808/1286810294 21 ==== osd_op_reply(106 609.00000000 [delete] v125'172 uv172 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9bc402c070 con 0x5639324aed60 2024-09-21T03:43:09.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.864+0000 7f9bd216df40 1 -- 172.21.15.38:0/2604388918 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5639324b7240 con 0x5639324aed60 2024-09-21T03:43:09.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.865+0000 7f9bcc428640 1 -- 172.21.15.38:0/2604388918 <== osd.3 v2:172.21.15.38:6808/1286810294 22 ==== osd_op_reply(107 609.00000000 [create] v125'173 uv173 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9bc402c070 con 0x5639324aed60 2024-09-21T03:43:09.867 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-09-21T03:43:09.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.865+0000 7f9bd216df40 1 -- 172.21.15.38:0/2604388918 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5639324b7240 con 0x5639324aed60 2024-09-21T03:43:09.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.866+0000 7f9bcc428640 1 -- 172.21.15.38:0/2604388918 <== osd.3 v2:172.21.15.38:6808/1286810294 23 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v125'174 uv174 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9bc402c070 con 0x5639324aed60 2024-09-21T03:43:09.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.866+0000 7f9bd216df40 1 -- 172.21.15.38:0/2604388918 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:109 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5639324b7240 con 0x5639324aed60 2024-09-21T03:43:09.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.867+0000 7f9bcc428640 1 -- 172.21.15.38:0/2604388918 <== osd.3 v2:172.21.15.38:6808/1286810294 24 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v125'175 uv175 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9bc402c070 con 0x5639324aed60 2024-09-21T03:43:09.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.867+0000 7f9bd216df40 1 -- 172.21.15.38:0/2604388918 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:110 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5639324b7240 con 0x5639324aed60 2024-09-21T03:43:09.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.868+0000 7f9bcc428640 1 -- 172.21.15.38:0/2604388918 <== osd.3 v2:172.21.15.38:6808/1286810294 25 ==== osd_op_reply(110 609.00000000 [omap-set-header] v125'176 uv176 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9bc402c070 con 0x5639324aed60 2024-09-21T03:43:09.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.869+0000 7f9bd216df40 1 --2- 172.21.15.38:0/2604388918 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x5639324b7750 0x5639324b9b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:09.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.869+0000 7f9bd216df40 1 -- 172.21.15.38:0/2604388918 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5639324ba1c0 con 0x5639324b7750 2024-09-21T03:43:09.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.869+0000 7f9bcb426640 1 --2- 172.21.15.38:0/2604388918 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x5639324b7750 0x5639324b9b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:09.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.869+0000 7f9bcb426640 1 --2- 172.21.15.38:0/2604388918 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x5639324b7750 0x5639324b9b70 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:09.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.870+0000 7f9bcb426640 1 -- 172.21.15.38:0/2604388918 <== osd.2 v2:172.21.15.38:6824/2577858902 1 ==== osd_op_reply(111 400.00000000 [create] v125'145 uv144 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9bc0056550 con 0x5639324b7750 2024-09-21T03:43:09.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.870+0000 7f9bd216df40 1 -- 172.21.15.38:0/2604388918 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e125) -- 0x5639324ba1c0 con 0x5639324b7750 2024-09-21T03:43:09.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.871+0000 7f9bcb426640 1 -- 172.21.15.38:0/2604388918 <== osd.2 v2:172.21.15.38:6824/2577858902 2 ==== osd_op_reply(112 400.00000000 [delete] v125'146 uv146 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9bc0056550 con 0x5639324b7750 2024-09-21T03:43:09.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.871+0000 7f9bd216df40 1 -- 172.21.15.38:0/2604388918 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5639324ba1c0 con 0x5639324b7750 2024-09-21T03:43:09.874 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-09-21T03:43:09.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.872+0000 7f9bcb426640 1 -- 172.21.15.38:0/2604388918 <== osd.2 v2:172.21.15.38:6824/2577858902 3 ==== osd_op_reply(113 400.00000000 [create] v125'147 uv147 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9bc0056550 con 0x5639324b7750 2024-09-21T03:43:09.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.872+0000 7f9bd216df40 1 -- 172.21.15.38:0/2604388918 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:114 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5639324ba5b0 con 0x5639324b7750 2024-09-21T03:43:09.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.874+0000 7f9bcb426640 1 -- 172.21.15.38:0/2604388918 <== osd.2 v2:172.21.15.38:6824/2577858902 4 ==== osd_op_reply(114 400.00000000 [write 0~22] v125'148 uv148 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9bc0056550 con 0x5639324b7750 2024-09-21T03:43:09.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.874+0000 7f9bd216df40 1 -- 172.21.15.38:0/2604388918 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:115 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5639324ba5b0 con 0x5639324b7750 2024-09-21T03:43:09.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.875+0000 7f9bcb426640 1 -- 172.21.15.38:0/2604388918 <== osd.2 v2:172.21.15.38:6824/2577858902 5 ==== osd_op_reply(115 400.00000000 [omap-set-header] v125'149 uv149 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9bc0056550 con 0x5639324b7750 2024-09-21T03:43:09.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.875+0000 7f9bd216df40 1 -- 172.21.15.38:0/2604388918 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:116 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5639324ba3b0 con 0x5639324b3f10 2024-09-21T03:43:09.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.876+0000 7f9bcbc27640 1 -- 172.21.15.38:0/2604388918 <== osd.1 v2:172.21.15.38:6816/1491818732 17 ==== osd_op_reply(116 100.00000000.inode [create] v125'351 uv343 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7f9bbc0015d0 con 0x5639324b3f10 2024-09-21T03:43:09.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.876+0000 7f9bd216df40 1 -- 172.21.15.38:0/2604388918 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:117 13.1f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e125) -- 0x5639324ba3b0 con 0x5639324b3f10 2024-09-21T03:43:09.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.877+0000 7f9bcbc27640 1 -- 172.21.15.38:0/2604388918 <== osd.1 v2:172.21.15.38:6816/1491818732 18 ==== osd_op_reply(117 100.00000000.inode [delete] v125'352 uv352 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f9bbc0015d0 con 0x5639324b3f10 2024-09-21T03:43:09.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.877+0000 7f9bd216df40 1 -- 172.21.15.38:0/2604388918 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:118 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5639324ba3d0 con 0x5639324b3f10 2024-09-21T03:43:09.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.878+0000 7f9bcbc27640 1 -- 172.21.15.38:0/2604388918 <== osd.1 v2:172.21.15.38:6816/1491818732 19 ==== osd_op_reply(118 100.00000000.inode [create] v125'353 uv353 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f9bbc0015d0 con 0x5639324b3f10 2024-09-21T03:43:09.880 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-09-21T03:43:09.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.878+0000 7f9bd216df40 1 -- 172.21.15.38:0/2604388918 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:119 13.1f 13:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5639324ba810 con 0x5639324b3f10 2024-09-21T03:43:09.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.879+0000 7f9bcbc27640 1 -- 172.21.15.38:0/2604388918 <== osd.1 v2:172.21.15.38:6816/1491818732 20 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v125'354 uv354 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f9bbc0015d0 con 0x5639324b3f10 2024-09-21T03:43:09.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.879+0000 7f9bd216df40 1 -- 172.21.15.38:0/2604388918 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:120 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5639324ba810 con 0x5639324b3f10 2024-09-21T03:43:09.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.881+0000 7f9bcbc27640 1 -- 172.21.15.38:0/2604388918 <== osd.1 v2:172.21.15.38:6816/1491818732 21 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v125'355 uv355 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f9bbc0015d0 con 0x5639324b3f10 2024-09-21T03:43:09.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.881+0000 7f9bd216df40 1 -- 172.21.15.38:0/2604388918 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:121 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5639324ba480 con 0x5639324b3f10 2024-09-21T03:43:09.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.882+0000 7f9bcbc27640 1 -- 172.21.15.38:0/2604388918 <== osd.1 v2:172.21.15.38:6816/1491818732 22 ==== osd_op_reply(121 1.00000000 [create] v125'356 uv350 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7f9bbc0015d0 con 0x5639324b3f10 2024-09-21T03:43:09.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.882+0000 7f9bd216df40 1 -- 172.21.15.38:0/2604388918 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:122 13.1f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e125) -- 0x5639324ba410 con 0x5639324b3f10 2024-09-21T03:43:09.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.883+0000 7f9bcbc27640 1 -- 172.21.15.38:0/2604388918 <== osd.1 v2:172.21.15.38:6816/1491818732 23 ==== osd_op_reply(122 1.00000000 [delete] v125'357 uv357 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f9bbc0015d0 con 0x5639324b3f10 2024-09-21T03:43:09.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.883+0000 7f9bd216df40 1 -- 172.21.15.38:0/2604388918 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:123 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5639324ba3d0 con 0x5639324b3f10 2024-09-21T03:43:09.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.884+0000 7f9bcbc27640 1 -- 172.21.15.38:0/2604388918 <== osd.1 v2:172.21.15.38:6816/1491818732 24 ==== osd_op_reply(123 1.00000000 [create] v125'358 uv358 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f9bbc0015d0 con 0x5639324b3f10 2024-09-21T03:43:09.886 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-09-21T03:43:09.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.884+0000 7f9bd216df40 1 -- 172.21.15.38:0/2604388918 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:124 13.1f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5639324ba3d0 con 0x5639324b3f10 2024-09-21T03:43:09.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.885+0000 7f9bcbc27640 1 -- 172.21.15.38:0/2604388918 <== osd.1 v2:172.21.15.38:6816/1491818732 25 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v125'359 uv359 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f9bbc0015d0 con 0x5639324b3f10 2024-09-21T03:43:09.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.885+0000 7f9bd216df40 1 -- 172.21.15.38:0/2604388918 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:125 13.1f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5639324ba3b0 con 0x5639324b3f10 2024-09-21T03:43:09.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.886+0000 7f9bcbc27640 1 -- 172.21.15.38:0/2604388918 <== osd.1 v2:172.21.15.38:6816/1491818732 26 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v125'360 uv360 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f9bbc0015d0 con 0x5639324b3f10 2024-09-21T03:43:09.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.886+0000 7f9bd216df40 1 -- 172.21.15.38:0/2604388918 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:126 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5639324ba390 con 0x5639324b3f10 2024-09-21T03:43:09.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.887+0000 7f9bcbc27640 1 -- 172.21.15.38:0/2604388918 <== osd.1 v2:172.21.15.38:6816/1491818732 27 ==== osd_op_reply(126 1.00000000 [omap-set-header] v125'361 uv361 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f9bbc0015d0 con 0x5639324b3f10 2024-09-21T03:43:09.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.887+0000 7f9bd216df40 1 -- 172.21.15.38:0/2604388918 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:127 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=584b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5639324ba790 con 0x5639324b3f10 2024-09-21T03:43:09.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.888+0000 7f9bcbc27640 1 -- 172.21.15.38:0/2604388918 <== osd.1 v2:172.21.15.38:6816/1491818732 28 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v125'362 uv362 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f9bbc0015d0 con 0x5639324b3f10 2024-09-21T03:43:09.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.888+0000 7f9bd216df40 1 -- 172.21.15.38:0/2604388918 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x5639324ab960 msgr2=0x5639324add80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:09.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.888+0000 7f9bd216df40 1 --2- 172.21.15.38:0/2604388918 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x5639324ab960 0x5639324add80 crc :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:09.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.888+0000 7f9bd216df40 1 -- 172.21.15.38:0/2604388918 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x5639324b3f10 msgr2=0x5639324b6330 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:09.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.888+0000 7f9bd216df40 1 --2- 172.21.15.38:0/2604388918 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x5639324b3f10 0x5639324b6330 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:09.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.888+0000 7f9bd216df40 1 -- 172.21.15.38:0/2604388918 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x5639324b7750 msgr2=0x5639324b9b70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:09.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.888+0000 7f9bd216df40 1 --2- 172.21.15.38:0/2604388918 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x5639324b7750 0x5639324b9b70 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:09.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.888+0000 7f9bd216df40 1 -- 172.21.15.38:0/2604388918 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x5639324aed60 msgr2=0x5639324b1180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:09.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.888+0000 7f9bd216df40 1 --2- 172.21.15.38:0/2604388918 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x5639324aed60 0x5639324b1180 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:09.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.888+0000 7f9bcc428640 1 -- 172.21.15.38:0/2604388918 reap_dead start 2024-09-21T03:43:09.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.888+0000 7f9bd216df40 1 -- 172.21.15.38:0/2604388918 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x5639324a6830 msgr2=0x5639324a8c50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:09.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.888+0000 7f9bd216df40 1 --2- 172.21.15.38:0/2604388918 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x5639324a6830 0x5639324a8c50 crc :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:09.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.888+0000 7f9bcc428640 1 -- 172.21.15.38:0/2604388918 reap_dead start 2024-09-21T03:43:09.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.888+0000 7f9bd216df40 1 -- 172.21.15.38:0/2604388918 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x5639323e3ef0 msgr2=0x5639323e16a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:09.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.888+0000 7f9bd216df40 1 --2- 172.21.15.38:0/2604388918 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x5639323e3ef0 0x5639323e16a0 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:09.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.888+0000 7f9bd216df40 1 -- 172.21.15.38:0/2604388918 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9bb8077570 msgr2=0x7f9bb8079a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:09.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.888+0000 7f9bd216df40 1 --2- 172.21.15.38:0/2604388918 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9bb8077570 0x7f9bb8079a80 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f9bbc006bf0 tx=0x7f9bbc006b80 comp rx=0 tx=0).stop 2024-09-21T03:43:09.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.888+0000 7f9bd216df40 1 -- 172.21.15.38:0/2604388918 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5639323ea520 msgr2=0x563932492800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:09.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.888+0000 7f9bd216df40 1 --2- 172.21.15.38:0/2604388918 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5639323ea520 0x563932492800 secure :-1 s=READY pgs=1943 cs=0 l=1 rev1=1 crypto rx=0x7f9bc4000f60 tx=0x7f9bc400feb0 comp rx=0 tx=0).stop 2024-09-21T03:43:09.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.889+0000 7f9bd216df40 1 -- 172.21.15.38:0/2604388918 shutdown_connections 2024-09-21T03:43:09.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.889+0000 7f9bd216df40 1 --2- 172.21.15.38:0/2604388918 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x5639323e3ef0 0x5639323e16a0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:09.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.889+0000 7f9bd216df40 1 --2- 172.21.15.38:0/2604388918 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5639323ea520 0x563932492800 unknown :-1 s=CLOSED pgs=1943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:09.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.889+0000 7f9bd216df40 1 --2- 172.21.15.38:0/2604388918 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9bb8077570 0x7f9bb8079a80 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:09.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.889+0000 7f9bd216df40 1 -- 172.21.15.38:0/2604388918 >> 172.21.15.38:0/2604388918 conn(0x56393224cac0 msgr2=0x5639323e9be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:09.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.889+0000 7f9bd216df40 1 -- 172.21.15.38:0/2604388918 shutdown_connections 2024-09-21T03:43:09.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.889+0000 7f9bd216df40 1 -- 172.21.15.38:0/2604388918 wait complete. 2024-09-21T03:43:09.894 DEBUG:teuthology.orchestra.run.smithi038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata truncate 400.00000000 0 2024-09-21T03:43:09.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.989+0000 7f7efd80bf40 1 Processor -- start 2024-09-21T03:43:09.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.989+0000 7f7efd80bf40 1 -- start start 2024-09-21T03:43:09.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.990+0000 7f7efd80bf40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e7e5157d00 0x55e7e52296f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:09.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.990+0000 7f7efd80bf40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e7e5212950 0x55e7e5211750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:09.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.990+0000 7f7efd80bf40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e7e522d670 0x55e7e5211c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:09.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.990+0000 7f7efd80bf40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55e7e50a5d60 con 0x55e7e5157d00 2024-09-21T03:43:09.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.990+0000 7f7efd80bf40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55e7e5104430 con 0x55e7e522d670 2024-09-21T03:43:09.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.990+0000 7f7efd80bf40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55e7e519f7b0 con 0x55e7e5212950 2024-09-21T03:43:09.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.991+0000 7f7ef65e8640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e7e5157d00 0x55e7e52296f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:09.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.991+0000 7f7ef65e8640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e7e5157d00 0x55e7e52296f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:45130/0 (socket says 172.21.15.38:45130) 2024-09-21T03:43:09.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.991+0000 7f7ef65e8640 1 -- 172.21.15.38:0/2374191956 learned_addr learned my addr 172.21.15.38:0/2374191956 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:43:09.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.991+0000 7f7ef5de7640 1 --2- 172.21.15.38:0/2374191956 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e7e5212950 0x55e7e5211750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:09.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.991+0000 7f7ef6de9640 1 --2- 172.21.15.38:0/2374191956 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e7e522d670 0x55e7e5211c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:09.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.991+0000 7f7ef65e8640 1 -- 172.21.15.38:0/2374191956 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e7e5212950 msgr2=0x55e7e5211750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:09.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.991+0000 7f7ef65e8640 1 --2- 172.21.15.38:0/2374191956 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e7e5212950 0x55e7e5211750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:09.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.991+0000 7f7ef65e8640 1 -- 172.21.15.38:0/2374191956 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e7e522d670 msgr2=0x55e7e5211c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:09.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.991+0000 7f7ef65e8640 1 --2- 172.21.15.38:0/2374191956 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e7e522d670 0x55e7e5211c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:09.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.991+0000 7f7ef65e8640 1 -- 172.21.15.38:0/2374191956 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e7e520eac0 con 0x55e7e5157d00 2024-09-21T03:43:09.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.991+0000 7f7ef65e8640 1 --2- 172.21.15.38:0/2374191956 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e7e5157d00 0x55e7e52296f0 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7f7ee8025f00 tx=0x7f7ee8002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3300f78e3d32f90 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:09.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.991+0000 7f7ef4de5640 1 -- 172.21.15.38:0/2374191956 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ee8056470 con 0x55e7e5157d00 2024-09-21T03:43:09.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.991+0000 7f7ef4de5640 1 -- 172.21.15.38:0/2374191956 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7ee8056610 con 0x55e7e5157d00 2024-09-21T03:43:09.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.991+0000 7f7ef4de5640 1 -- 172.21.15.38:0/2374191956 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ee8056a60 con 0x55e7e5157d00 2024-09-21T03:43:09.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.992+0000 7f7efd80bf40 1 -- 172.21.15.38:0/2374191956 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e7e5157d00 msgr2=0x55e7e52296f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:09.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.992+0000 7f7efd80bf40 1 --2- 172.21.15.38:0/2374191956 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e7e5157d00 0x55e7e52296f0 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7f7ee8025f00 tx=0x7f7ee8002fb0 comp rx=0 tx=0).stop 2024-09-21T03:43:09.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.992+0000 7f7efd80bf40 1 -- 172.21.15.38:0/2374191956 shutdown_connections 2024-09-21T03:43:09.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.992+0000 7f7efd80bf40 1 --2- 172.21.15.38:0/2374191956 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e7e522d670 0x55e7e5211c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:09.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.992+0000 7f7efd80bf40 1 --2- 172.21.15.38:0/2374191956 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e7e5212950 0x55e7e5211750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:09.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.992+0000 7f7efd80bf40 1 --2- 172.21.15.38:0/2374191956 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e7e5157d00 0x55e7e52296f0 unknown :-1 s=CLOSED pgs=1945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:09.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.992+0000 7f7efd80bf40 1 -- 172.21.15.38:0/2374191956 >> 172.21.15.38:0/2374191956 conn(0x55e7e5076ac0 msgr2=0x55e7e522da80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:09.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.992+0000 7f7efd80bf40 1 -- 172.21.15.38:0/2374191956 shutdown_connections 2024-09-21T03:43:09.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.992+0000 7f7efd80bf40 1 -- 172.21.15.38:0/2374191956 wait complete. 2024-09-21T03:43:09.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.993+0000 7f7efd80bf40 1 Processor -- start 2024-09-21T03:43:09.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.993+0000 7f7efd80bf40 1 -- start start 2024-09-21T03:43:09.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.994+0000 7f7efd80bf40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e7e5157d00 0x55e7e522da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:09.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.994+0000 7f7efd80bf40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e7e5212950 0x55e7e522dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:09.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.994+0000 7f7efd80bf40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e7e522d670 0x55e7e52b9680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:09.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.994+0000 7f7efd80bf40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55e7e52289f0 con 0x55e7e5212950 2024-09-21T03:43:09.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.994+0000 7f7efd80bf40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55e7e5184a30 con 0x55e7e522d670 2024-09-21T03:43:09.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.994+0000 7f7efd80bf40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55e7e5224600 con 0x55e7e5157d00 2024-09-21T03:43:09.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.994+0000 7f7ef5de7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e7e5212950 0x55e7e522dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:09.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.994+0000 7f7ef5de7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e7e5212950 0x55e7e522dfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:45138/0 (socket says 172.21.15.38:45138) 2024-09-21T03:43:09.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.994+0000 7f7ef5de7640 1 -- 172.21.15.38:0/3386579344 learned_addr learned my addr 172.21.15.38:0/3386579344 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:43:09.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.994+0000 7f7ef65e8640 1 --2- 172.21.15.38:0/3386579344 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e7e5157d00 0x55e7e522da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:09.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.994+0000 7f7ef6de9640 1 --2- 172.21.15.38:0/3386579344 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e7e522d670 0x55e7e52b9680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:09.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.995+0000 7f7ef5de7640 1 -- 172.21.15.38:0/3386579344 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e7e5157d00 msgr2=0x55e7e522da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:09.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.995+0000 7f7ef5de7640 1 --2- 172.21.15.38:0/3386579344 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e7e5157d00 0x55e7e522da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:09.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.995+0000 7f7ef5de7640 1 -- 172.21.15.38:0/3386579344 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e7e522d670 msgr2=0x55e7e52b9680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:10.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.995+0000 7f7ef5de7640 1 --2- 172.21.15.38:0/3386579344 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e7e522d670 0x55e7e52b9680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:10.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.995+0000 7f7ef5de7640 1 -- 172.21.15.38:0/3386579344 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e7e52bc490 con 0x55e7e5212950 2024-09-21T03:43:10.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.995+0000 7f7ef5de7640 1 --2- 172.21.15.38:0/3386579344 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e7e5212950 0x55e7e522dfc0 secure :-1 s=READY pgs=1946 cs=0 l=1 rev1=1 crypto rx=0x7f7eec00a720 tx=0x7f7eec00abe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:10.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.995+0000 7f7ee77fe640 1 -- 172.21.15.38:0/3386579344 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eec009d50 con 0x55e7e5212950 2024-09-21T03:43:10.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.995+0000 7f7ee77fe640 1 -- 172.21.15.38:0/3386579344 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7eec00ae30 con 0x55e7e5212950 2024-09-21T03:43:10.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.995+0000 7f7efd80bf40 1 -- 172.21.15.38:0/3386579344 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55e7e52bc770 con 0x55e7e5212950 2024-09-21T03:43:10.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.995+0000 7f7ee77fe640 1 -- 172.21.15.38:0/3386579344 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eec017690 con 0x55e7e5212950 2024-09-21T03:43:10.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.995+0000 7f7efd80bf40 1 -- 172.21.15.38:0/3386579344 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e7e5226b70 con 0x55e7e5212950 2024-09-21T03:43:10.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.997+0000 7f7ee77fe640 1 -- 172.21.15.38:0/3386579344 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f7eec017830 con 0x55e7e5212950 2024-09-21T03:43:10.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.997+0000 7f7ee77fe640 1 --2- 172.21.15.38:0/3386579344 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7ed8077570 0x7f7ed8079a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:10.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.997+0000 7f7ee77fe640 1 -- 172.21.15.38:0/3386579344 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 8798+0+0 (secure 0 0 0) 0x7f7eec09b140 con 0x55e7e5212950 2024-09-21T03:43:10.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.998+0000 7f7ef65e8640 1 --2- 172.21.15.38:0/3386579344 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7ed8077570 0x7f7ed8079a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:10.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.998+0000 7f7efd80bf40 1 --2- 172.21.15.38:0/3386579344 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x55e7e52ba790 0x55e7e52baba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:10.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.998+0000 7f7efd80bf40 1 -- 172.21.15.38:0/3386579344 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:1 13.f 13:f1f69726:::400.00000000:head [truncate 0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55e7e5224220 con 0x55e7e52ba790 2024-09-21T03:43:10.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.998+0000 7f7ef6de9640 1 --2- 172.21.15.38:0/3386579344 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x55e7e52ba790 0x55e7e52baba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:10.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.998+0000 7f7ef65e8640 1 --2- 172.21.15.38:0/3386579344 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7ed8077570 0x7f7ed8079a80 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f7ee8025f00 tx=0x7f7ee8064040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:10.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.998+0000 7f7ef6de9640 1 --2- 172.21.15.38:0/3386579344 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x55e7e52ba790 0x55e7e52baba0 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:10.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.999+0000 7f7ef6de9640 1 -- 172.21.15.38:0/3386579344 <== osd.2 v2:172.21.15.38:6824/2577858902 1 ==== osd_op_reply(1 400.00000000 [truncate 0] v125'150 uv150 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7ef00025f0 con 0x55e7e52ba790 2024-09-21T03:43:10.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.999+0000 7f7efd80bf40 1 -- 172.21.15.38:0/3386579344 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x55e7e52ba790 msgr2=0x55e7e52baba0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:10.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:09.999+0000 7f7efd80bf40 1 --2- 172.21.15.38:0/3386579344 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x55e7e52ba790 0x55e7e52baba0 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:10.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:10.000+0000 7f7efd80bf40 1 -- 172.21.15.38:0/3386579344 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7ed8077570 msgr2=0x7f7ed8079a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:10.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:10.000+0000 7f7efd80bf40 1 --2- 172.21.15.38:0/3386579344 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7ed8077570 0x7f7ed8079a80 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f7ee8025f00 tx=0x7f7ee8064040 comp rx=0 tx=0).stop 2024-09-21T03:43:10.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:10.000+0000 7f7efd80bf40 1 -- 172.21.15.38:0/3386579344 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e7e5212950 msgr2=0x55e7e522dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:10.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:10.000+0000 7f7efd80bf40 1 --2- 172.21.15.38:0/3386579344 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e7e5212950 0x55e7e522dfc0 secure :-1 s=READY pgs=1946 cs=0 l=1 rev1=1 crypto rx=0x7f7eec00a720 tx=0x7f7eec00abe0 comp rx=0 tx=0).stop 2024-09-21T03:43:10.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:10.000+0000 7f7ef6de9640 1 -- 172.21.15.38:0/3386579344 reap_dead start 2024-09-21T03:43:10.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:10.000+0000 7f7efd80bf40 1 -- 172.21.15.38:0/3386579344 shutdown_connections 2024-09-21T03:43:10.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:10.000+0000 7f7efd80bf40 1 -- 172.21.15.38:0/3386579344 >> 172.21.15.38:0/3386579344 conn(0x55e7e5076ac0 msgr2=0x55e7e52119f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:10.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:10.000+0000 7f7efd80bf40 1 -- 172.21.15.38:0/3386579344 shutdown_connections 2024-09-21T03:43:10.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:10.000+0000 7f7efd80bf40 1 -- 172.21.15.38:0/3386579344 wait complete. 2024-09-21T03:43:10.005 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-21T03:43:10.123 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:10.121+0000 7f306872b640 1 -- 172.21.15.38:0/755991727 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f30600d0170 msgr2=0x7f30600d0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:10.123 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:10.121+0000 7f306872b640 1 --2- 172.21.15.38:0/755991727 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f30600d0170 0x7f30600d0be0 secure :-1 s=READY pgs=1947 cs=0 l=1 rev1=1 crypto rx=0x7f30600d2080 tx=0x7f305002a420 comp rx=0 tx=0).stop 2024-09-21T03:43:10.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:10.121+0000 7f306872b640 1 -- 172.21.15.38:0/755991727 shutdown_connections 2024-09-21T03:43:10.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:10.121+0000 7f306872b640 1 --2- 172.21.15.38:0/755991727 >> v2:172.21.15.195:3300/0 conn(0x7f30600d1120 0x7f30600ceb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:10.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:10.121+0000 7f306872b640 1 --2- 172.21.15.38:0/755991727 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f30600d0170 0x7f30600d0be0 unknown :-1 s=CLOSED pgs=1947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:10.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:10.121+0000 7f306872b640 1 -- 172.21.15.38:0/755991727 >> 172.21.15.38:0/755991727 conn(0x7f306009d830 msgr2=0x7f30600b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:10.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:10.121+0000 7f306872b640 1 -- 172.21.15.38:0/755991727 shutdown_connections 2024-09-21T03:43:10.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:10.121+0000 7f306872b640 1 -- 172.21.15.38:0/755991727 wait complete. 2024-09-21T03:43:10.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:10.123+0000 7f306872b640 1 Processor -- start 2024-09-21T03:43:10.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:10.123+0000 7f306872b640 1 -- start start 2024-09-21T03:43:10.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:10.124+0000 7f306872b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f30600d0170 0x7f30600cb610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:10.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:10.124+0000 7f306872b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f30600d1120 0x7f306014f970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:10.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:10.124+0000 7f306872b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f306015ee80 0x7f30601f5df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:10.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:10.124+0000 7f306872b640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f30600b6700 con 0x7f30600d0170 2024-09-21T03:43:10.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:10.124+0000 7f306872b640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f30600b6580 con 0x7f30600d1120 2024-09-21T03:43:10.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:10.124+0000 7f306872b640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f30600b6880 con 0x7f306015ee80 2024-09-21T03:43:10.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:10.124+0000 7f30664c7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f30600d0170 0x7f30600cb610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:10.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:10.125+0000 7f30664c7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f30600d0170 0x7f30600cb610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:45154/0 (socket says 172.21.15.38:45154) 2024-09-21T03:43:10.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:10.125+0000 7f30664c7640 1 -- 172.21.15.38:0/3095605192 learned_addr learned my addr 172.21.15.38:0/3095605192 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:43:10.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:10.125+0000 7f3065cc6640 1 --2- 172.21.15.38:0/3095605192 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f30600d1120 0x7f306014f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:10.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:10.125+0000 7f3066cc8640 1 --2- 172.21.15.38:0/3095605192 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f306015ee80 0x7f30601f5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:10.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:10.125+0000 7f30664c7640 1 -- 172.21.15.38:0/3095605192 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f306015ee80 msgr2=0x7f30601f5df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:10.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:10.125+0000 7f30664c7640 1 --2- 172.21.15.38:0/3095605192 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f306015ee80 0x7f30601f5df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:10.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:10.125+0000 7f30664c7640 1 -- 172.21.15.38:0/3095605192 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f30600d1120 msgr2=0x7f306014f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:10.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:10.125+0000 7f30664c7640 1 --2- 172.21.15.38:0/3095605192 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f30600d1120 0x7f306014f970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:10.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:10.125+0000 7f30664c7640 1 -- 172.21.15.38:0/3095605192 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30601f93b0 con 0x7f30600d0170 2024-09-21T03:43:10.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:10.125+0000 7f30664c7640 1 --2- 172.21.15.38:0/3095605192 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f30600d0170 0x7f30600cb610 secure :-1 s=READY pgs=1948 cs=0 l=1 rev1=1 crypto rx=0x7f305c00b3f0 tx=0x7f305c00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:10.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:10.125+0000 7f3056ffd640 1 -- 172.21.15.38:0/3095605192 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f305c015020 con 0x7f30600d0170 2024-09-21T03:43:10.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:10.125+0000 7f3056ffd640 1 -- 172.21.15.38:0/3095605192 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f305c011920 con 0x7f30600d0170 2024-09-21T03:43:10.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:10.125+0000 7f3056ffd640 1 -- 172.21.15.38:0/3095605192 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f305c011d70 con 0x7f30600d0170 2024-09-21T03:43:10.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:10.125+0000 7f306872b640 1 -- 172.21.15.38:0/3095605192 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30601f9690 con 0x7f30600d0170 2024-09-21T03:43:10.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:10.125+0000 7f306872b640 1 -- 172.21.15.38:0/3095605192 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30600cbc50 con 0x7f30600d0170 2024-09-21T03:43:10.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:10.126+0000 7f306872b640 1 -- 172.21.15.38:0/3095605192 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30601f9bb0 con 0x7f30600d0170 2024-09-21T03:43:10.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:10.127+0000 7f3056ffd640 1 -- 172.21.15.38:0/3095605192 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f305c018020 con 0x7f30600d0170 2024-09-21T03:43:10.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:10.128+0000 7f3056ffd640 1 --2- 172.21.15.38:0/3095605192 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3044078b90 0x7f304407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:10.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:10.128+0000 7f3056ffd640 1 -- 172.21.15.38:0/3095605192 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 8798+0+0 (secure 0 0 0) 0x7f305c09b560 con 0x7f30600d0170 2024-09-21T03:43:10.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:10.128+0000 7f3065cc6640 1 --2- 172.21.15.38:0/3095605192 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3044078b90 0x7f304407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:10.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:10.128+0000 7f3065cc6640 1 --2- 172.21.15.38:0/3095605192 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3044078b90 0x7f304407b0a0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f3050034000 tx=0x7f3050004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:10.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:10.132+0000 7f3056ffd640 1 -- 172.21.15.38:0/3095605192 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f30601f9bb0 con 0x7f30600d0170 2024-09-21T03:43:10.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:10.358+0000 7f306872b640 1 -- 172.21.15.38:0/3095605192 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f30600b9d80 con 0x7f30600d0170 2024-09-21T03:43:10.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:10.564+0000 7f3056ffd640 1 -- 172.21.15.38:0/3095605192 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v260) ==== 162+0+0 (secure 0 0 0) 0x7f30600b9d80 con 0x7f30600d0170 2024-09-21T03:43:10.566 INFO:teuthology.orchestra.run.smithi038.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-21T03:43:10.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:10.567+0000 7f306872b640 1 -- 172.21.15.38:0/3095605192 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3044078b90 msgr2=0x7f304407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:10.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:10.567+0000 7f306872b640 1 --2- 172.21.15.38:0/3095605192 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3044078b90 0x7f304407b0a0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f3050034000 tx=0x7f3050004730 comp rx=0 tx=0).stop 2024-09-21T03:43:10.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:10.567+0000 7f306872b640 1 -- 172.21.15.38:0/3095605192 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f30600d0170 msgr2=0x7f30600cb610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:10.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:10.567+0000 7f306872b640 1 --2- 172.21.15.38:0/3095605192 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f30600d0170 0x7f30600cb610 secure :-1 s=READY pgs=1948 cs=0 l=1 rev1=1 crypto rx=0x7f305c00b3f0 tx=0x7f305c00b8b0 comp rx=0 tx=0).stop 2024-09-21T03:43:10.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:10.567+0000 7f306872b640 1 -- 172.21.15.38:0/3095605192 shutdown_connections 2024-09-21T03:43:10.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:10.567+0000 7f306872b640 1 --2- 172.21.15.38:0/3095605192 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3044078b90 0x7f304407b0a0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:10.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:10.567+0000 7f306872b640 1 --2- 172.21.15.38:0/3095605192 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f306015ee80 0x7f30601f5df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:10.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:10.567+0000 7f306872b640 1 --2- 172.21.15.38:0/3095605192 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f30600d1120 0x7f306014f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:10.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:10.567+0000 7f306872b640 1 --2- 172.21.15.38:0/3095605192 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f30600d0170 0x7f30600cb610 unknown :-1 s=CLOSED pgs=1948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:10.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:10.567+0000 7f306872b640 1 -- 172.21.15.38:0/3095605192 >> 172.21.15.38:0/3095605192 conn(0x7f306009d830 msgr2=0x7f306015df50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:10.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:10.567+0000 7f306872b640 1 -- 172.21.15.38:0/3095605192 shutdown_connections 2024-09-21T03:43:10.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:10.567+0000 7f306872b640 1 -- 172.21.15.38:0/3095605192 wait complete. 2024-09-21T03:43:10.571 INFO:tasks.ceph.mds.c.smithi038.stderr:2024-09-21T03:43:10.569+0000 7fde662bd640 -1 log_channel(cluster) log [ERR] : failed to read JournalPointer: -22 ((22) Invalid argument) 2024-09-21T03:43:10.571 INFO:tasks.ceph.mon.a.smithi038.stderr:2024-09-21T03:43:10.570+0000 7f3f45713640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T03:43:10.572 INFO:tasks.ceph.mon.a.smithi038.stderr:2024-09-21T03:43:10.570+0000 7f3f45713640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-09-21T03:43:10.582 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:43:10.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:10.701+0000 7fb90b6c5640 1 -- 172.21.15.38:0/1254913320 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb9040b2e60 msgr2=0x7fb9040b3270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:10.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:10.701+0000 7fb90b6c5640 1 --2- 172.21.15.38:0/1254913320 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb9040b2e60 0x7fb9040b3270 secure :-1 s=READY pgs=1950 cs=0 l=1 rev1=1 crypto rx=0x7fb8fc005fd0 tx=0x7fb8fc02a8b0 comp rx=0 tx=0).stop 2024-09-21T03:43:10.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:10.701+0000 7fb90b6c5640 1 -- 172.21.15.38:0/1254913320 shutdown_connections 2024-09-21T03:43:10.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:10.701+0000 7fb90b6c5640 1 --2- 172.21.15.38:0/1254913320 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb9040b2e60 0x7fb9040b3270 unknown :-1 s=CLOSED pgs=1950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:10.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:10.701+0000 7fb90b6c5640 1 --2- 172.21.15.38:0/1254913320 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb9040cfe00 0x7fb9040d2a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:10.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:10.701+0000 7fb90b6c5640 1 -- 172.21.15.38:0/1254913320 >> 172.21.15.38:0/1254913320 conn(0x7fb9040d4440 msgr2=0x7fb9040d4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:10.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:10.701+0000 7fb90b6c5640 1 -- 172.21.15.38:0/1254913320 shutdown_connections 2024-09-21T03:43:10.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:10.701+0000 7fb90b6c5640 1 -- 172.21.15.38:0/1254913320 wait complete. 2024-09-21T03:43:10.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:10.703+0000 7fb90b6c5640 1 Processor -- start 2024-09-21T03:43:10.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:10.703+0000 7fb90b6c5640 1 -- start start 2024-09-21T03:43:10.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:10.704+0000 7fb90b6c5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb9040b2e60 0x7fb90414cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:10.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:10.704+0000 7fb90b6c5640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb9040cfe00 0x7fb904151940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:10.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:10.704+0000 7fb90b6c5640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb904162590 0x7fb904151e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:10.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:10.704+0000 7fb90b6c5640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fb9040b3930 con 0x7fb9040b2e60 2024-09-21T03:43:10.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:10.704+0000 7fb90b6c5640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fb9040b37b0 con 0x7fb904162590 2024-09-21T03:43:10.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:10.704+0000 7fb90b6c5640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fb9040b3ab0 con 0x7fb9040cfe00 2024-09-21T03:43:10.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:10.704+0000 7fb909461640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb9040b2e60 0x7fb90414cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:10.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:10.704+0000 7fb909461640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb9040b2e60 0x7fb90414cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:45170/0 (socket says 172.21.15.38:45170) 2024-09-21T03:43:10.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:10.704+0000 7fb909461640 1 -- 172.21.15.38:0/4143324068 learned_addr learned my addr 172.21.15.38:0/4143324068 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:43:10.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:10.704+0000 7fb908c60640 1 --2- 172.21.15.38:0/4143324068 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb9040cfe00 0x7fb904151940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:10.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:10.704+0000 7fb909c62640 1 --2- 172.21.15.38:0/4143324068 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb904162590 0x7fb904151e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:10.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:10.704+0000 7fb909461640 1 -- 172.21.15.38:0/4143324068 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb9040cfe00 msgr2=0x7fb904151940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:10.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:10.704+0000 7fb909461640 1 --2- 172.21.15.38:0/4143324068 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb9040cfe00 0x7fb904151940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:10.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:10.704+0000 7fb909461640 1 -- 172.21.15.38:0/4143324068 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb904162590 msgr2=0x7fb904151e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:10.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:10.704+0000 7fb909461640 1 --2- 172.21.15.38:0/4143324068 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb904162590 0x7fb904151e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:10.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:10.704+0000 7fb909461640 1 -- 172.21.15.38:0/4143324068 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb90414de50 con 0x7fb9040b2e60 2024-09-21T03:43:10.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:10.704+0000 7fb908c60640 1 --2- 172.21.15.38:0/4143324068 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb9040cfe00 0x7fb904151940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:43:10.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:10.705+0000 7fb909461640 1 --2- 172.21.15.38:0/4143324068 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb9040b2e60 0x7fb90414cc20 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7fb90000d740 tx=0x7fb90000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:10.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:10.705+0000 7fb8f9ffb640 1 -- 172.21.15.38:0/4143324068 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb900017070 con 0x7fb9040b2e60 2024-09-21T03:43:10.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:10.705+0000 7fb8f9ffb640 1 -- 172.21.15.38:0/4143324068 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb900009d20 con 0x7fb9040b2e60 2024-09-21T03:43:10.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:10.705+0000 7fb90b6c5640 1 -- 172.21.15.38:0/4143324068 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb90414e130 con 0x7fb9040b2e60 2024-09-21T03:43:10.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:10.705+0000 7fb8f9ffb640 1 -- 172.21.15.38:0/4143324068 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9000106c0 con 0x7fb9040b2e60 2024-09-21T03:43:10.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:10.705+0000 7fb90b6c5640 1 -- 172.21.15.38:0/4143324068 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb904150d20 con 0x7fb9040b2e60 2024-09-21T03:43:10.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:10.706+0000 7fb90b6c5640 1 -- 172.21.15.38:0/4143324068 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb904150f80 con 0x7fb9040b2e60 2024-09-21T03:43:10.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:10.707+0000 7fb8f9ffb640 1 -- 172.21.15.38:0/4143324068 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb90001a070 con 0x7fb9040b2e60 2024-09-21T03:43:10.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:10.707+0000 7fb8f9ffb640 1 --2- 172.21.15.38:0/4143324068 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb8e8078b90 0x7fb8e807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:10.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:10.707+0000 7fb8f9ffb640 1 -- 172.21.15.38:0/4143324068 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 8884+0+0 (secure 0 0 0) 0x7fb90009bb80 con 0x7fb9040b2e60 2024-09-21T03:43:10.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:10.708+0000 7fb908c60640 1 --2- 172.21.15.38:0/4143324068 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb8e8078b90 0x7fb8e807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:10.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:10.708+0000 7fb908c60640 1 --2- 172.21.15.38:0/4143324068 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb8e8078b90 0x7fb8e807b0a0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fb904159290 tx=0x7fb8f4009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:10.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:10.712+0000 7fb8f9ffb640 1 -- 172.21.15.38:0/4143324068 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fb900063340 con 0x7fb9040b2e60 2024-09-21T03:43:10.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:10.941+0000 7fb90b6c5640 1 -- 172.21.15.38:0/4143324068 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb9040b3270 con 0x7fb9040b2e60 2024-09-21T03:43:10.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:10.942+0000 7fb8f9ffb640 1 -- 172.21.15.38:0/4143324068 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 261 v261) ==== 95+0+3566 (secure 0 0 0) 0x7fb900062c40 con 0x7fb9040b2e60 2024-09-21T03:43:10.944 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:43:10.944 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":261,"btime":"2024-09-21T03:43:10:571036+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6086,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/965972663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":965972663},{"type":"v1","addr":"172.21.15.195:6836","nonce":965972663}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6700,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/3176122334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":3176122334},{"type":"v1","addr":"172.21.15.195:6837","nonce":3176122334}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":261,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:40:02.319282+0000","modified":"2024-09-21T03:43:10.570947+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":126,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-21T03:43:10.945 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 261 2024-09-21T03:43:10.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:10.944+0000 7fb90b6c5640 1 -- 172.21.15.38:0/4143324068 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb8e8078b90 msgr2=0x7fb8e807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:10.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:10.944+0000 7fb90b6c5640 1 --2- 172.21.15.38:0/4143324068 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb8e8078b90 0x7fb8e807b0a0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fb904159290 tx=0x7fb8f4009040 comp rx=0 tx=0).stop 2024-09-21T03:43:10.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:10.944+0000 7fb90b6c5640 1 -- 172.21.15.38:0/4143324068 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb9040b2e60 msgr2=0x7fb90414cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:10.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:10.944+0000 7fb90b6c5640 1 --2- 172.21.15.38:0/4143324068 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb9040b2e60 0x7fb90414cc20 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7fb90000d740 tx=0x7fb90000dc00 comp rx=0 tx=0).stop 2024-09-21T03:43:10.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:10.944+0000 7fb90b6c5640 1 -- 172.21.15.38:0/4143324068 shutdown_connections 2024-09-21T03:43:10.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:10.944+0000 7fb90b6c5640 1 --2- 172.21.15.38:0/4143324068 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb904162590 0x7fb904151e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:10.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:10.944+0000 7fb90b6c5640 1 --2- 172.21.15.38:0/4143324068 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb8e8078b90 0x7fb8e807b0a0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:10.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:10.944+0000 7fb90b6c5640 1 --2- 172.21.15.38:0/4143324068 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb9040cfe00 0x7fb904151940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:10.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:10.944+0000 7fb90b6c5640 1 --2- 172.21.15.38:0/4143324068 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb9040b2e60 0x7fb90414cc20 unknown :-1 s=CLOSED pgs=1951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:10.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:10.944+0000 7fb90b6c5640 1 -- 172.21.15.38:0/4143324068 >> 172.21.15.38:0/4143324068 conn(0x7fb9040d4440 msgr2=0x7fb904160450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:10.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:10.944+0000 7fb90b6c5640 1 -- 172.21.15.38:0/4143324068 shutdown_connections 2024-09-21T03:43:10.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:10.944+0000 7fb90b6c5640 1 -- 172.21.15.38:0/4143324068 wait complete. 2024-09-21T03:43:10.958 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-21T03:43:10.958 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:43:11.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:11.073+0000 7f06042dd640 1 -- 172.21.15.38:0/1975232250 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f05e4003c10 msgr2=0x7f05e4004060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:11.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:11.073+0000 7f06042dd640 1 --2- 172.21.15.38:0/1975232250 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f05e4003c10 0x7f05e4004060 secure :-1 s=READY pgs=1953 cs=0 l=1 rev1=1 crypto rx=0x7f05ec00c590 tx=0x7f05e4005830 comp rx=0 tx=0).stop 2024-09-21T03:43:11.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:11.073+0000 7f060287a640 1 -- 172.21.15.38:0/1975232250 reap_dead start 2024-09-21T03:43:11.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:11.073+0000 7f06042dd640 1 -- 172.21.15.38:0/1975232250 shutdown_connections 2024-09-21T03:43:11.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:11.073+0000 7f06042dd640 1 -- 172.21.15.38:0/1975232250 >> 172.21.15.38:0/1975232250 conn(0x7f05fc0b6b20 msgr2=0x7f05fc0d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:11.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:11.073+0000 7f06042dd640 1 -- 172.21.15.38:0/1975232250 shutdown_connections 2024-09-21T03:43:11.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:11.073+0000 7f06042dd640 1 -- 172.21.15.38:0/1975232250 wait complete. 2024-09-21T03:43:11.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:11.075+0000 7f06042dd640 1 Processor -- start 2024-09-21T03:43:11.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:11.075+0000 7f06042dd640 1 -- start start 2024-09-21T03:43:11.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:11.076+0000 7f06042dd640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f05e4003c10 0x7f05fc1eabc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:11.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:11.076+0000 7f06042dd640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f05fc160530 0x7f05fc160940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:11.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:11.076+0000 7f06042dd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f05fc1f86f0 0x7f05fc1f8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:11.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:11.076+0000 7f06042dd640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f05fc0b31d0 con 0x7f05fc1f86f0 2024-09-21T03:43:11.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:11.076+0000 7f06042dd640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f05fc0b3050 con 0x7f05fc160530 2024-09-21T03:43:11.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:11.076+0000 7f06042dd640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f05fc0b3350 con 0x7f05e4003c10 2024-09-21T03:43:11.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:11.076+0000 7f060287a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f05fc1f86f0 0x7f05fc1f8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:11.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:11.076+0000 7f060287a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f05fc1f86f0 0x7f05fc1f8b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:45184/0 (socket says 172.21.15.38:45184) 2024-09-21T03:43:11.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:11.076+0000 7f060287a640 1 -- 172.21.15.38:0/2491209409 learned_addr learned my addr 172.21.15.38:0/2491209409 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:43:11.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:11.076+0000 7f0602079640 1 --2- 172.21.15.38:0/2491209409 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f05e4003c10 0x7f05fc1eabc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:11.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:11.076+0000 7f0601878640 1 --2- 172.21.15.38:0/2491209409 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f05fc160530 0x7f05fc160940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:11.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:11.076+0000 7f060287a640 1 -- 172.21.15.38:0/2491209409 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f05e4003c10 msgr2=0x7f05fc1eabc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:11.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:11.076+0000 7f060287a640 1 --2- 172.21.15.38:0/2491209409 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f05e4003c10 0x7f05fc1eabc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:11.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:11.076+0000 7f060287a640 1 -- 172.21.15.38:0/2491209409 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f05fc160530 msgr2=0x7f05fc160940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:11.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:11.076+0000 7f060287a640 1 --2- 172.21.15.38:0/2491209409 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f05fc160530 0x7f05fc160940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:11.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:11.077+0000 7f060287a640 1 -- 172.21.15.38:0/2491209409 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05fc1f9040 con 0x7f05fc1f86f0 2024-09-21T03:43:11.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:11.077+0000 7f060287a640 1 --2- 172.21.15.38:0/2491209409 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f05fc1f86f0 0x7f05fc1f8b00 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7f05fc0d0c40 tx=0x7f05f4011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:11.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:11.077+0000 7f05faffd640 1 -- 172.21.15.38:0/2491209409 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05f4004440 con 0x7f05fc1f86f0 2024-09-21T03:43:11.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:11.077+0000 7f05faffd640 1 -- 172.21.15.38:0/2491209409 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f05f400f920 con 0x7f05fc1f86f0 2024-09-21T03:43:11.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:11.077+0000 7f06042dd640 1 -- 172.21.15.38:0/2491209409 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05fc1eb100 con 0x7f05fc1f86f0 2024-09-21T03:43:11.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:11.077+0000 7f05faffd640 1 -- 172.21.15.38:0/2491209409 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05f400fd70 con 0x7f05fc1f86f0 2024-09-21T03:43:11.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:11.077+0000 7f06042dd640 1 -- 172.21.15.38:0/2491209409 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05fc1f6440 con 0x7f05fc1f86f0 2024-09-21T03:43:11.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:11.078+0000 7f06042dd640 1 -- 172.21.15.38:0/2491209409 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05fc0b38e0 con 0x7f05fc1f86f0 2024-09-21T03:43:11.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:11.079+0000 7f05faffd640 1 -- 172.21.15.38:0/2491209409 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f05f40035f0 con 0x7f05fc1f86f0 2024-09-21T03:43:11.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:11.080+0000 7f05faffd640 1 --2- 172.21.15.38:0/2491209409 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f05e8078b90 0x7f05e807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:11.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:11.080+0000 7f05faffd640 1 -- 172.21.15.38:0/2491209409 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 8884+0+0 (secure 0 0 0) 0x7f05f409c5f0 con 0x7f05fc1f86f0 2024-09-21T03:43:11.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:11.080+0000 7f0602079640 1 --2- 172.21.15.38:0/2491209409 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f05e8078b90 0x7f05e807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:11.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:11.080+0000 7f0602079640 1 --2- 172.21.15.38:0/2491209409 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f05e8078b90 0x7f05e807b0a0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f05ec033000 tx=0x7f05ec02d070 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:11.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:11.084+0000 7f05faffd640 1 -- 172.21.15.38:0/2491209409 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f05fc1f9040 con 0x7f05fc1f86f0 2024-09-21T03:43:11.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:11.310+0000 7f06042dd640 1 -- 172.21.15.38:0/2491209409 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f05fc160fb0 con 0x7f05fc1f86f0 2024-09-21T03:43:11.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:11.311+0000 7f05faffd640 1 -- 172.21.15.38:0/2491209409 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 261 v261) ==== 95+0+3566 (secure 0 0 0) 0x7f05f4063720 con 0x7f05fc1f86f0 2024-09-21T03:43:11.313 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:43:11.313 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":261,"btime":"2024-09-21T03:43:10:571036+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6086,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/965972663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":965972663},{"type":"v1","addr":"172.21.15.195:6836","nonce":965972663}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6700,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/3176122334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":3176122334},{"type":"v1","addr":"172.21.15.195:6837","nonce":3176122334}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":261,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:40:02.319282+0000","modified":"2024-09-21T03:43:10.570947+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":126,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-21T03:43:11.313 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 261 2024-09-21T03:43:11.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:11.313+0000 7f06042dd640 1 -- 172.21.15.38:0/2491209409 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f05e8078b90 msgr2=0x7f05e807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:11.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:11.313+0000 7f06042dd640 1 --2- 172.21.15.38:0/2491209409 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f05e8078b90 0x7f05e807b0a0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f05ec033000 tx=0x7f05ec02d070 comp rx=0 tx=0).stop 2024-09-21T03:43:11.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:11.313+0000 7f06042dd640 1 -- 172.21.15.38:0/2491209409 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f05fc1f86f0 msgr2=0x7f05fc1f8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:11.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:11.313+0000 7f06042dd640 1 --2- 172.21.15.38:0/2491209409 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f05fc1f86f0 0x7f05fc1f8b00 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7f05fc0d0c40 tx=0x7f05f4011b50 comp rx=0 tx=0).stop 2024-09-21T03:43:11.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:11.313+0000 7f06042dd640 1 -- 172.21.15.38:0/2491209409 shutdown_connections 2024-09-21T03:43:11.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:11.313+0000 7f06042dd640 1 --2- 172.21.15.38:0/2491209409 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f05fc1f86f0 0x7f05fc1f8b00 unknown :-1 s=CLOSED pgs=1954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:11.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:11.313+0000 7f06042dd640 1 --2- 172.21.15.38:0/2491209409 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f05fc160530 0x7f05fc160940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:11.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:11.313+0000 7f06042dd640 1 --2- 172.21.15.38:0/2491209409 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f05e8078b90 0x7f05e807b0a0 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:11.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:11.313+0000 7f06042dd640 1 --2- 172.21.15.38:0/2491209409 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f05e4003c10 0x7f05fc1eabc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:11.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:11.313+0000 7f06042dd640 1 -- 172.21.15.38:0/2491209409 >> 172.21.15.38:0/2491209409 conn(0x7f05fc0b6b20 msgr2=0x7f05fc0b42d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:11.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:11.314+0000 7f06042dd640 1 -- 172.21.15.38:0/2491209409 shutdown_connections 2024-09-21T03:43:11.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:11.314+0000 7f06042dd640 1 -- 172.21.15.38:0/2491209409 wait complete. 2024-09-21T03:43:11.327 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-21T03:43:11.327 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:43:11.356 INFO:tasks.ceph.mds.a.smithi038.stderr: -52> 2024-09-21T03:43:06.603+0000 7f5302a41640 -1 log_channel(cluster) log [ERR] : error decoding table object 'mds0_inotable': End of buffer [buffer:2] 2024-09-21T03:43:11.413 INFO:tasks.ceph.mds.a.smithi038.stdout:starting mds.a at 2024-09-21T03:43:11.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:11.445+0000 7faea9bad640 1 -- 172.21.15.38:0/2695994647 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faea40b45d0 msgr2=0x7faea40cdfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:11.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:11.445+0000 7faea9bad640 1 --2- 172.21.15.38:0/2695994647 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faea40b45d0 0x7faea40cdfe0 secure :-1 s=READY pgs=1959 cs=0 l=1 rev1=1 crypto rx=0x7fae9c005fd0 tx=0x7fae9c02a8b0 comp rx=0 tx=0).stop 2024-09-21T03:43:11.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:11.445+0000 7faea9bad640 1 -- 172.21.15.38:0/2695994647 shutdown_connections 2024-09-21T03:43:11.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:11.445+0000 7faea9bad640 1 --2- 172.21.15.38:0/2695994647 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faea40b45d0 0x7faea40cdfe0 unknown :-1 s=CLOSED pgs=1959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:11.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:11.445+0000 7faea9bad640 1 -- 172.21.15.38:0/2695994647 >> 172.21.15.38:0/2695994647 conn(0x7faea40cef40 msgr2=0x7faea40cf350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:11.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:11.445+0000 7faea9bad640 1 -- 172.21.15.38:0/2695994647 shutdown_connections 2024-09-21T03:43:11.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:11.445+0000 7faea9bad640 1 -- 172.21.15.38:0/2695994647 wait complete. 2024-09-21T03:43:11.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:11.447+0000 7faea9bad640 1 Processor -- start 2024-09-21T03:43:11.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:11.447+0000 7faea9bad640 1 -- start start 2024-09-21T03:43:11.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:11.449+0000 7faea9bad640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faea406f7b0 0x7faea4158b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:11.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:11.449+0000 7faea9bad640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faea40cc930 0x7faea40c6f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:11.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:11.449+0000 7faea9bad640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faea40c74d0 0x7faea40c78e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:11.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:11.449+0000 7faea9bad640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7faea40ce6a0 con 0x7faea40cc930 2024-09-21T03:43:11.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:11.449+0000 7faea9bad640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7faea40ce520 con 0x7faea406f7b0 2024-09-21T03:43:11.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:11.449+0000 7faea9bad640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7faea40ce820 con 0x7faea40c74d0 2024-09-21T03:43:11.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:11.449+0000 7faea2ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faea40cc930 0x7faea40c6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:11.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:11.449+0000 7faea2ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faea40cc930 0x7faea40c6f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:45214/0 (socket says 172.21.15.38:45214) 2024-09-21T03:43:11.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:11.449+0000 7faea2ffd640 1 -- 172.21.15.38:0/1177766023 learned_addr learned my addr 172.21.15.38:0/1177766023 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:43:11.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:11.449+0000 7faea3fff640 1 --2- 172.21.15.38:0/1177766023 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faea40c74d0 0x7faea40c78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:11.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:11.449+0000 7faea37fe640 1 --2- 172.21.15.38:0/1177766023 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faea406f7b0 0x7faea4158b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:11.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:11.449+0000 7faea2ffd640 1 -- 172.21.15.38:0/1177766023 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faea40c74d0 msgr2=0x7faea40c78e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:11.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:11.449+0000 7faea2ffd640 1 --2- 172.21.15.38:0/1177766023 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faea40c74d0 0x7faea40c78e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:11.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:11.449+0000 7faea2ffd640 1 -- 172.21.15.38:0/1177766023 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faea406f7b0 msgr2=0x7faea4158b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:11.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:11.449+0000 7faea2ffd640 1 --2- 172.21.15.38:0/1177766023 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faea406f7b0 0x7faea4158b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:11.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:11.449+0000 7faea2ffd640 1 -- 172.21.15.38:0/1177766023 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faea4159080 con 0x7faea40cc930 2024-09-21T03:43:11.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:11.449+0000 7faea2ffd640 1 --2- 172.21.15.38:0/1177766023 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faea40cc930 0x7faea40c6f90 secure :-1 s=READY pgs=1960 cs=0 l=1 rev1=1 crypto rx=0x7fae9800c3f0 tx=0x7fae9800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:11.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:11.450+0000 7fae8bfff640 1 -- 172.21.15.38:0/1177766023 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae98017070 con 0x7faea40cc930 2024-09-21T03:43:11.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:11.450+0000 7fae8bfff640 1 -- 172.21.15.38:0/1177766023 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fae980035f0 con 0x7faea40cc930 2024-09-21T03:43:11.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:11.450+0000 7faea9bad640 1 -- 172.21.15.38:0/1177766023 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faea414fe10 con 0x7faea40cc930 2024-09-21T03:43:11.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:11.450+0000 7fae8bfff640 1 -- 172.21.15.38:0/1177766023 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae98003a40 con 0x7faea40cc930 2024-09-21T03:43:11.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:11.450+0000 7faea9bad640 1 -- 172.21.15.38:0/1177766023 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faea41503d0 con 0x7faea40cc930 2024-09-21T03:43:11.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:11.451+0000 7faea9bad640 1 -- 172.21.15.38:0/1177766023 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faea40cdfe0 con 0x7faea40cc930 2024-09-21T03:43:11.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:11.451+0000 7fae8bfff640 1 -- 172.21.15.38:0/1177766023 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fae9801a070 con 0x7faea40cc930 2024-09-21T03:43:11.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:11.452+0000 7fae8bfff640 1 --2- 172.21.15.38:0/1177766023 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fae90078b90 0x7fae9007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:11.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:11.452+0000 7fae8bfff640 1 -- 172.21.15.38:0/1177766023 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 8884+0+0 (secure 0 0 0) 0x7fae9809ac60 con 0x7faea40cc930 2024-09-21T03:43:11.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:11.452+0000 7faea37fe640 1 --2- 172.21.15.38:0/1177766023 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fae90078b90 0x7fae9007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:11.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:11.453+0000 7faea37fe640 1 --2- 172.21.15.38:0/1177766023 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fae90078b90 0x7fae9007b0a0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fae94000f60 tx=0x7fae94009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:11.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:11.457+0000 7fae8bfff640 1 -- 172.21.15.38:0/1177766023 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fae98061cb0 con 0x7faea40cc930 2024-09-21T03:43:11.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:11.684+0000 7faea9bad640 1 -- 172.21.15.38:0/1177766023 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faea40b8260 con 0x7faea40cc930 2024-09-21T03:43:11.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:11.685+0000 7fae8bfff640 1 -- 172.21.15.38:0/1177766023 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 262 v262) ==== 95+0+4389 (secure 0 0 0) 0x7fae980666e0 con 0x7faea40cc930 2024-09-21T03:43:11.687 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:43:11.687 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":262,"btime":"2024-09-21T03:43:11:633603+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6086,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/965972663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":965972663},{"type":"v1","addr":"172.21.15.195:6836","nonce":965972663}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6700,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/3176122334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":3176122334},{"type":"v1","addr":"172.21.15.195:6837","nonce":3176122334}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":9720,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/141447142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":141447142},{"type":"v1","addr":"172.21.15.38:6835","nonce":141447142}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":262}],"filesystems":[{"mdsmap":{"epoch":261,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:40:02.319282+0000","modified":"2024-09-21T03:43:10.570947+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":126,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-21T03:43:11.687 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 262 2024-09-21T03:43:11.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:11.687+0000 7faea9bad640 1 -- 172.21.15.38:0/1177766023 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fae90078b90 msgr2=0x7fae9007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:11.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:11.687+0000 7faea9bad640 1 --2- 172.21.15.38:0/1177766023 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fae90078b90 0x7fae9007b0a0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fae94000f60 tx=0x7fae94009040 comp rx=0 tx=0).stop 2024-09-21T03:43:11.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:11.687+0000 7faea9bad640 1 -- 172.21.15.38:0/1177766023 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faea40cc930 msgr2=0x7faea40c6f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:11.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:11.687+0000 7faea9bad640 1 --2- 172.21.15.38:0/1177766023 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faea40cc930 0x7faea40c6f90 secure :-1 s=READY pgs=1960 cs=0 l=1 rev1=1 crypto rx=0x7fae9800c3f0 tx=0x7fae9800c8b0 comp rx=0 tx=0).stop 2024-09-21T03:43:11.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:11.687+0000 7faea9bad640 1 -- 172.21.15.38:0/1177766023 shutdown_connections 2024-09-21T03:43:11.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:11.687+0000 7faea9bad640 1 --2- 172.21.15.38:0/1177766023 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fae90078b90 0x7fae9007b0a0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:11.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:11.687+0000 7faea9bad640 1 --2- 172.21.15.38:0/1177766023 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faea40c74d0 0x7faea40c78e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:11.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:11.687+0000 7faea9bad640 1 --2- 172.21.15.38:0/1177766023 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faea40cc930 0x7faea40c6f90 unknown :-1 s=CLOSED pgs=1960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:11.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:11.687+0000 7faea9bad640 1 --2- 172.21.15.38:0/1177766023 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faea406f7b0 0x7faea4158b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:11.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:11.687+0000 7faea9bad640 1 -- 172.21.15.38:0/1177766023 >> 172.21.15.38:0/1177766023 conn(0x7faea40cef40 msgr2=0x7faea4161810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:11.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:11.687+0000 7faea9bad640 1 -- 172.21.15.38:0/1177766023 shutdown_connections 2024-09-21T03:43:11.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:11.687+0000 7faea9bad640 1 -- 172.21.15.38:0/1177766023 wait complete. 2024-09-21T03:43:11.701 INFO:tasks.cephfs.test_damage:Result: Mutation 'Truncate 400.00000000' led to DAMAGED state 2024-09-21T03:43:11.701 INFO:tasks.cephfs.test_damage:Applying mutation 'Truncate 100.00000000.inode' 2024-09-21T03:43:11.701 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-21T03:43:11.701 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:43:11.757 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056795,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2354 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T03:43:11.757 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi038.front.sepia.ceph.com 2024-09-21T03:43:11.757 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:43:11.757 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T03:43:11.812 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T03:43:11.812 INFO:teuthology.orchestra.run.smithi038.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-09-21T03:43:11.812 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-21T03:43:11.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:11.968+0000 7f6424040640 1 -- 172.21.15.38:0/3570059715 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6408003c30 msgr2=0x7f6408004080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:11.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:11.968+0000 7f6424040640 1 --2- 172.21.15.38:0/3570059715 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6408003c30 0x7f6408004080 secure :-1 s=READY pgs=1962 cs=0 l=1 rev1=1 crypto rx=0x7f6418009d00 tx=0x7f6408005830 comp rx=0 tx=0).stop 2024-09-21T03:43:11.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:11.968+0000 7f6424040640 1 -- 172.21.15.38:0/3570059715 shutdown_connections 2024-09-21T03:43:11.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:11.968+0000 7f6424040640 1 --2- 172.21.15.38:0/3570059715 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6408003c30 0x7f6408004080 unknown :-1 s=CLOSED pgs=1962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:11.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:11.968+0000 7f6424040640 1 -- 172.21.15.38:0/3570059715 >> 172.21.15.38:0/3570059715 conn(0x7f641c0cd460 msgr2=0x7f641c0cd870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:11.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:11.968+0000 7f6424040640 1 -- 172.21.15.38:0/3570059715 shutdown_connections 2024-09-21T03:43:11.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:11.969+0000 7f6424040640 1 -- 172.21.15.38:0/3570059715 wait complete. 2024-09-21T03:43:11.972 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:11.970+0000 7f6424040640 1 Processor -- start 2024-09-21T03:43:11.972 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:11.970+0000 7f6424040640 1 -- start start 2024-09-21T03:43:11.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:11.972+0000 7f6424040640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6408003c30 0x7f641c0af420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:11.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:11.972+0000 7f6424040640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f641c149a70 0x7f641c149480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:11.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:11.972+0000 7f6424040640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f641c14bb90 0x7f641c14a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:11.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:11.972+0000 7f6424040640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f641c156e60 con 0x7f641c14bb90 2024-09-21T03:43:11.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:11.972+0000 7f6424040640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f641c157190 con 0x7f6408003c30 2024-09-21T03:43:11.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:11.972+0000 7f6424040640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f641c156fe0 con 0x7f641c149a70 2024-09-21T03:43:11.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:11.972+0000 7f64225dd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f641c14bb90 0x7f641c14a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:11.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:11.972+0000 7f64225dd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f641c14bb90 0x7f641c14a810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:45222/0 (socket says 172.21.15.38:45222) 2024-09-21T03:43:11.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:11.972+0000 7f64225dd640 1 -- 172.21.15.38:0/499827655 learned_addr learned my addr 172.21.15.38:0/499827655 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:43:11.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:11.972+0000 7f6421ddc640 1 --2- 172.21.15.38:0/499827655 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6408003c30 0x7f641c0af420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:11.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:11.972+0000 7f64215db640 1 --2- 172.21.15.38:0/499827655 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f641c149a70 0x7f641c149480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:11.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:11.972+0000 7f64225dd640 1 -- 172.21.15.38:0/499827655 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f641c149a70 msgr2=0x7f641c149480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:11.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:11.972+0000 7f64225dd640 1 --2- 172.21.15.38:0/499827655 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f641c149a70 0x7f641c149480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:11.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:11.972+0000 7f64225dd640 1 -- 172.21.15.38:0/499827655 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6408003c30 msgr2=0x7f641c0af420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:11.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:11.972+0000 7f64225dd640 1 --2- 172.21.15.38:0/499827655 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6408003c30 0x7f641c0af420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:11.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:11.972+0000 7f64225dd640 1 -- 172.21.15.38:0/499827655 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f641c14d4a0 con 0x7f641c14bb90 2024-09-21T03:43:11.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:11.972+0000 7f64225dd640 1 --2- 172.21.15.38:0/499827655 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f641c14bb90 0x7f641c14a810 secure :-1 s=READY pgs=1963 cs=0 l=1 rev1=1 crypto rx=0x7f641c073180 tx=0x7f64140124a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:11.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:11.972+0000 7f64127fc640 1 -- 172.21.15.38:0/499827655 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f641401d070 con 0x7f641c14bb90 2024-09-21T03:43:11.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:11.972+0000 7f64127fc640 1 -- 172.21.15.38:0/499827655 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f641400e3c0 con 0x7f641c14bb90 2024-09-21T03:43:11.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:11.972+0000 7f6424040640 1 -- 172.21.15.38:0/499827655 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f641c14d940 con 0x7f641c14bb90 2024-09-21T03:43:11.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:11.973+0000 7f64127fc640 1 -- 172.21.15.38:0/499827655 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f641400ee60 con 0x7f641c14bb90 2024-09-21T03:43:11.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:11.973+0000 7f6424040640 1 -- 172.21.15.38:0/499827655 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f641c14d1c0 con 0x7f641c14bb90 2024-09-21T03:43:11.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:11.973+0000 7f6424040640 1 -- 172.21.15.38:0/499827655 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f641c073a10 con 0x7f641c14bb90 2024-09-21T03:43:11.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:11.974+0000 7f64127fc640 1 -- 172.21.15.38:0/499827655 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f6414020070 con 0x7f641c14bb90 2024-09-21T03:43:11.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:11.975+0000 7f64127fc640 1 --2- 172.21.15.38:0/499827655 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6400078b90 0x7f640007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:11.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:11.975+0000 7f64127fc640 1 -- 172.21.15.38:0/499827655 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 8884+0+0 (secure 0 0 0) 0x7f641409f230 con 0x7f641c14bb90 2024-09-21T03:43:11.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:11.975+0000 7f6421ddc640 1 --2- 172.21.15.38:0/499827655 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6400078b90 0x7f640007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:11.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:11.976+0000 7f6421ddc640 1 --2- 172.21.15.38:0/499827655 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6400078b90 0x7f640007b0a0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f6418035000 tx=0x7f64180046c0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:11.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:11.980+0000 7f64127fc640 1 -- 172.21.15.38:0/499827655 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f641c14d4a0 con 0x7f641c14bb90 2024-09-21T03:43:12.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:12.203+0000 7f6424040640 1 -- 172.21.15.38:0/499827655 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f641c14d6a0 con 0x7f641c14bb90 2024-09-21T03:43:12.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:12.646+0000 7f64127fc640 1 -- 172.21.15.38:0/499827655 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v263) ==== 161+0+0 (secure 0 0 0) 0x7f6414015070 con 0x7f641c14bb90 2024-09-21T03:43:12.648 INFO:teuthology.orchestra.run.smithi038.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-21T03:43:12.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:12.649+0000 7f6424040640 1 -- 172.21.15.38:0/499827655 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6400078b90 msgr2=0x7f640007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:12.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:12.649+0000 7f6424040640 1 --2- 172.21.15.38:0/499827655 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6400078b90 0x7f640007b0a0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f6418035000 tx=0x7f64180046c0 comp rx=0 tx=0).stop 2024-09-21T03:43:12.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:12.649+0000 7f6424040640 1 -- 172.21.15.38:0/499827655 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f641c14bb90 msgr2=0x7f641c14a810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:12.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:12.649+0000 7f6424040640 1 --2- 172.21.15.38:0/499827655 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f641c14bb90 0x7f641c14a810 secure :-1 s=READY pgs=1963 cs=0 l=1 rev1=1 crypto rx=0x7f641c073180 tx=0x7f64140124a0 comp rx=0 tx=0).stop 2024-09-21T03:43:12.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:12.649+0000 7f6424040640 1 -- 172.21.15.38:0/499827655 shutdown_connections 2024-09-21T03:43:12.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:12.649+0000 7f6424040640 1 --2- 172.21.15.38:0/499827655 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f641c14bb90 0x7f641c14a810 unknown :-1 s=CLOSED pgs=1963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:12.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:12.649+0000 7f6424040640 1 --2- 172.21.15.38:0/499827655 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6400078b90 0x7f640007b0a0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:12.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:12.649+0000 7f6424040640 1 --2- 172.21.15.38:0/499827655 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f641c149a70 0x7f641c149480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:12.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:12.649+0000 7f6424040640 1 --2- 172.21.15.38:0/499827655 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6408003c30 0x7f641c0af420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:12.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:12.649+0000 7f6424040640 1 -- 172.21.15.38:0/499827655 >> 172.21.15.38:0/499827655 conn(0x7f641c0cd460 msgr2=0x7f641c0b1490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:12.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:12.649+0000 7f6424040640 1 -- 172.21.15.38:0/499827655 shutdown_connections 2024-09-21T03:43:12.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:12.649+0000 7f6424040640 1 -- 172.21.15.38:0/499827655 wait complete. 2024-09-21T03:43:12.663 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-09-21T03:43:12.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:12.782+0000 7fef3a615640 1 -- 172.21.15.38:0/2160768944 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fef340d1120 msgr2=0x7fef340ceb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:12.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:12.782+0000 7fef3a615640 1 --2- 172.21.15.38:0/2160768944 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fef340d1120 0x7fef340ceb90 secure :-1 s=READY pgs=1964 cs=0 l=1 rev1=1 crypto rx=0x7fef340757e0 tx=0x7fef2c02a980 comp rx=0 tx=0).stop 2024-09-21T03:43:12.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:12.782+0000 7fef3a615640 1 -- 172.21.15.38:0/2160768944 shutdown_connections 2024-09-21T03:43:12.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:12.782+0000 7fef3a615640 1 --2- 172.21.15.38:0/2160768944 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fef340d1120 0x7fef340ceb90 unknown :-1 s=CLOSED pgs=1964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:12.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:12.782+0000 7fef3a615640 1 --2- 172.21.15.38:0/2160768944 >> v2:172.21.15.195:3300/0 conn(0x7fef340b9ad0 0x7fef340d0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:12.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:12.782+0000 7fef3a615640 1 --2- 172.21.15.38:0/2160768944 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fef3415ee80 0x7fef340d0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:12.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:12.782+0000 7fef3a615640 1 -- 172.21.15.38:0/2160768944 >> 172.21.15.38:0/2160768944 conn(0x7fef3409d830 msgr2=0x7fef340b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:12.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:12.782+0000 7fef3a615640 1 -- 172.21.15.38:0/2160768944 shutdown_connections 2024-09-21T03:43:12.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:12.782+0000 7fef3a615640 1 -- 172.21.15.38:0/2160768944 wait complete. 2024-09-21T03:43:12.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:12.784+0000 7fef3a615640 1 Processor -- start 2024-09-21T03:43:12.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:12.784+0000 7fef3a615640 1 -- start start 2024-09-21T03:43:12.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:12.785+0000 7fef3a615640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fef340b9ad0 0x7fef340cdf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:12.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:12.785+0000 7fef3a615640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fef340d1120 0x7fef3406e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:12.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:12.785+0000 7fef3a615640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fef3415ee80 0x7fef3406e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:12.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:12.785+0000 7fef3a615640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fef340cf460 con 0x7fef3415ee80 2024-09-21T03:43:12.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:12.785+0000 7fef3a615640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fef340cf2e0 con 0x7fef340d1120 2024-09-21T03:43:12.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:12.785+0000 7fef3a615640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fef340b6580 con 0x7fef340b9ad0 2024-09-21T03:43:12.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:12.785+0000 7fef38bb2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fef3415ee80 0x7fef3406e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:12.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:12.785+0000 7fef38bb2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fef3415ee80 0x7fef3406e7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35938/0 (socket says 172.21.15.38:35938) 2024-09-21T03:43:12.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:12.785+0000 7fef38bb2640 1 -- 172.21.15.38:0/4113195065 learned_addr learned my addr 172.21.15.38:0/4113195065 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:43:12.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:12.785+0000 7fef33fff640 1 --2- 172.21.15.38:0/4113195065 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fef340b9ad0 0x7fef340cdf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:12.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:12.785+0000 7fef337fe640 1 --2- 172.21.15.38:0/4113195065 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fef340d1120 0x7fef3406e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:12.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:12.786+0000 7fef38bb2640 1 -- 172.21.15.38:0/4113195065 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fef340b9ad0 msgr2=0x7fef340cdf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:12.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:12.786+0000 7fef38bb2640 1 --2- 172.21.15.38:0/4113195065 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fef340b9ad0 0x7fef340cdf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:12.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:12.786+0000 7fef38bb2640 1 -- 172.21.15.38:0/4113195065 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fef340d1120 msgr2=0x7fef3406e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:12.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:12.786+0000 7fef38bb2640 1 --2- 172.21.15.38:0/4113195065 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fef340d1120 0x7fef3406e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:12.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:12.786+0000 7fef38bb2640 1 -- 172.21.15.38:0/4113195065 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef340c88d0 con 0x7fef3415ee80 2024-09-21T03:43:12.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:12.786+0000 7fef38bb2640 1 --2- 172.21.15.38:0/4113195065 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fef3415ee80 0x7fef3406e7f0 secure :-1 s=READY pgs=1965 cs=0 l=1 rev1=1 crypto rx=0x7fef2c02af70 tx=0x7fef2c003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:12.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:12.786+0000 7fef30ff9640 1 -- 172.21.15.38:0/4113195065 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef2c034040 con 0x7fef3415ee80 2024-09-21T03:43:12.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:12.786+0000 7fef30ff9640 1 -- 172.21.15.38:0/4113195065 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fef2c004540 con 0x7fef3415ee80 2024-09-21T03:43:12.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:12.786+0000 7fef3a615640 1 -- 172.21.15.38:0/4113195065 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef340cad00 con 0x7fef3415ee80 2024-09-21T03:43:12.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:12.786+0000 7fef30ff9640 1 -- 172.21.15.38:0/4113195065 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef2c045480 con 0x7fef3415ee80 2024-09-21T03:43:12.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:12.786+0000 7fef3a615640 1 -- 172.21.15.38:0/4113195065 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef340c8560 con 0x7fef3415ee80 2024-09-21T03:43:12.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:12.787+0000 7fef3a615640 1 -- 172.21.15.38:0/4113195065 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef340caa70 con 0x7fef3415ee80 2024-09-21T03:43:12.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:12.788+0000 7fef30ff9640 1 -- 172.21.15.38:0/4113195065 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fef2c037020 con 0x7fef3415ee80 2024-09-21T03:43:12.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:12.788+0000 7fef30ff9640 1 --2- 172.21.15.38:0/4113195065 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fef20078b90 0x7fef2007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:12.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:12.789+0000 7fef30ff9640 1 -- 172.21.15.38:0/4113195065 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 8884+0+0 (secure 0 0 0) 0x7fef2c0bd350 con 0x7fef3415ee80 2024-09-21T03:43:12.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:12.789+0000 7fef33fff640 1 --2- 172.21.15.38:0/4113195065 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fef20078b90 0x7fef2007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:12.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:12.789+0000 7fef33fff640 1 --2- 172.21.15.38:0/4113195065 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fef20078b90 0x7fef2007b0a0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fef24005fd0 tx=0x7fef24005850 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:12.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:12.793+0000 7fef30ff9640 1 -- 172.21.15.38:0/4113195065 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fef340c88d0 con 0x7fef3415ee80 2024-09-21T03:43:13.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.056+0000 7fef3a615640 1 -- 172.21.15.38:0/4113195065 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7fef3406df90 con 0x7fef3415ee80 2024-09-21T03:43:13.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.654+0000 7fef30ff9640 1 -- 172.21.15.38:0/4113195065 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v264) ==== 101+0+0 (secure 0 0 0) 0x7fef3406df90 con 0x7fef3415ee80 2024-09-21T03:43:13.657 INFO:teuthology.orchestra.run.smithi038.stderr:repaired: restoring rank 6:0 2024-09-21T03:43:13.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.657+0000 7fef3a615640 1 -- 172.21.15.38:0/4113195065 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fef20078b90 msgr2=0x7fef2007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:13.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.657+0000 7fef3a615640 1 --2- 172.21.15.38:0/4113195065 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fef20078b90 0x7fef2007b0a0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fef24005fd0 tx=0x7fef24005850 comp rx=0 tx=0).stop 2024-09-21T03:43:13.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.657+0000 7fef3a615640 1 -- 172.21.15.38:0/4113195065 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fef3415ee80 msgr2=0x7fef3406e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:13.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.657+0000 7fef3a615640 1 --2- 172.21.15.38:0/4113195065 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fef3415ee80 0x7fef3406e7f0 secure :-1 s=READY pgs=1965 cs=0 l=1 rev1=1 crypto rx=0x7fef2c02af70 tx=0x7fef2c003f90 comp rx=0 tx=0).stop 2024-09-21T03:43:13.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.657+0000 7fef3a615640 1 -- 172.21.15.38:0/4113195065 shutdown_connections 2024-09-21T03:43:13.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.657+0000 7fef3a615640 1 --2- 172.21.15.38:0/4113195065 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fef3415ee80 0x7fef3406e7f0 unknown :-1 s=CLOSED pgs=1965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:13.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.657+0000 7fef3a615640 1 --2- 172.21.15.38:0/4113195065 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fef340d1120 0x7fef3406e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:13.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.657+0000 7fef3a615640 1 --2- 172.21.15.38:0/4113195065 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fef20078b90 0x7fef2007b0a0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:13.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.657+0000 7fef3a615640 1 --2- 172.21.15.38:0/4113195065 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fef340b9ad0 0x7fef340cdf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:13.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.657+0000 7fef3a615640 1 -- 172.21.15.38:0/4113195065 >> 172.21.15.38:0/4113195065 conn(0x7fef3409d830 msgr2=0x7fef341614d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:13.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.657+0000 7fef3a615640 1 -- 172.21.15.38:0/4113195065 shutdown_connections 2024-09-21T03:43:13.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.657+0000 7fef3a615640 1 -- 172.21.15.38:0/4113195065 wait complete. 2024-09-21T03:43:13.672 DEBUG:teuthology.orchestra.run.smithi038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-09-21T03:43:13.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.729+0000 7f144a7e5f40 1 Processor -- start 2024-09-21T03:43:13.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.729+0000 7f144a7e5f40 1 -- start start 2024-09-21T03:43:13.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.731+0000 7f144a7e5f40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e989943d00 0x55e989971c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:13.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.731+0000 7f144a7e5f40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e98996fe30 0x55e9899fc5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:13.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.731+0000 7f144a7e5f40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e9899fe950 0x55e989a19540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:13.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.731+0000 7f144a7e5f40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55e9898f0430 con 0x55e98996fe30 2024-09-21T03:43:13.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.731+0000 7f144a7e5f40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55e9898fd1f0 con 0x55e989943d00 2024-09-21T03:43:13.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.731+0000 7f144a7e5f40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55e989891d60 con 0x55e9899fe950 2024-09-21T03:43:13.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.731+0000 7f1443a9e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e98996fe30 0x55e9899fc5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:13.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.731+0000 7f1443a9e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e98996fe30 0x55e9899fc5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35968/0 (socket says 172.21.15.38:35968) 2024-09-21T03:43:13.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.731+0000 7f1443a9e640 1 -- 172.21.15.38:0/3189248731 learned_addr learned my addr 172.21.15.38:0/3189248731 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:43:13.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.731+0000 7f144429f640 1 --2- 172.21.15.38:0/3189248731 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e989943d00 0x55e989971c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:13.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.731+0000 7f1444aa0640 1 --2- 172.21.15.38:0/3189248731 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e9899fe950 0x55e989a19540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:13.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.731+0000 7f1443a9e640 1 -- 172.21.15.38:0/3189248731 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e9899fe950 msgr2=0x55e989a19540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:13.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.731+0000 7f1443a9e640 1 --2- 172.21.15.38:0/3189248731 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e9899fe950 0x55e989a19540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:13.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.731+0000 7f1443a9e640 1 -- 172.21.15.38:0/3189248731 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e989943d00 msgr2=0x55e989971c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:13.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.731+0000 7f1443a9e640 1 --2- 172.21.15.38:0/3189248731 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e989943d00 0x55e989971c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:13.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.731+0000 7f1443a9e640 1 -- 172.21.15.38:0/3189248731 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e989971a60 con 0x55e98996fe30 2024-09-21T03:43:13.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.731+0000 7f144429f640 1 --2- 172.21.15.38:0/3189248731 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e989943d00 0x55e989971c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:43:13.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.731+0000 7f1443a9e640 1 --2- 172.21.15.38:0/3189248731 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e98996fe30 0x55e9899fc5a0 secure :-1 s=READY pgs=1968 cs=0 l=1 rev1=1 crypto rx=0x7f1438026000 tx=0x7f1438002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce2a8321d967b1cb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:13.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.732+0000 7f1442a9c640 1 -- 172.21.15.38:0/3189248731 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f143804da60 con 0x55e98996fe30 2024-09-21T03:43:13.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.732+0000 7f1442a9c640 1 -- 172.21.15.38:0/3189248731 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1438056580 con 0x55e98996fe30 2024-09-21T03:43:13.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.732+0000 7f1442a9c640 1 -- 172.21.15.38:0/3189248731 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14380569d0 con 0x55e98996fe30 2024-09-21T03:43:13.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.732+0000 7f144a7e5f40 1 -- 172.21.15.38:0/3189248731 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e98996fe30 msgr2=0x55e9899fc5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:13.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.732+0000 7f144a7e5f40 1 --2- 172.21.15.38:0/3189248731 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e98996fe30 0x55e9899fc5a0 secure :-1 s=READY pgs=1968 cs=0 l=1 rev1=1 crypto rx=0x7f1438026000 tx=0x7f1438002fb0 comp rx=0 tx=0).stop 2024-09-21T03:43:13.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.732+0000 7f144a7e5f40 1 -- 172.21.15.38:0/3189248731 shutdown_connections 2024-09-21T03:43:13.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.732+0000 7f144a7e5f40 1 --2- 172.21.15.38:0/3189248731 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e9899fe950 0x55e989a19540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:13.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.732+0000 7f144a7e5f40 1 --2- 172.21.15.38:0/3189248731 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e98996fe30 0x55e9899fc5a0 unknown :-1 s=CLOSED pgs=1968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:13.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.732+0000 7f144a7e5f40 1 --2- 172.21.15.38:0/3189248731 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e989943d00 0x55e989971c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:13.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.732+0000 7f144a7e5f40 1 -- 172.21.15.38:0/3189248731 >> 172.21.15.38:0/3189248731 conn(0x55e989862ac0 msgr2=0x55e989a184b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:13.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.732+0000 7f144a7e5f40 1 -- 172.21.15.38:0/3189248731 shutdown_connections 2024-09-21T03:43:13.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.732+0000 7f144a7e5f40 1 -- 172.21.15.38:0/3189248731 wait complete. 2024-09-21T03:43:13.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.733+0000 7f144a7e5f40 1 Processor -- start 2024-09-21T03:43:13.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.733+0000 7f144a7e5f40 1 -- start start 2024-09-21T03:43:13.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.734+0000 7f144a7e5f40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e989943d00 0x55e989ab0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:13.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.735+0000 7f144a7e5f40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e98996fe30 0x55e9899f80a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:13.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.735+0000 7f144a7e5f40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e9899fe950 0x55e989987460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:13.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.735+0000 7f144a7e5f40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55e989988d60 con 0x55e9899fe950 2024-09-21T03:43:13.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.735+0000 7f144a7e5f40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55e9899fe070 con 0x55e98996fe30 2024-09-21T03:43:13.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.735+0000 7f144a7e5f40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55e9899fa020 con 0x55e989943d00 2024-09-21T03:43:13.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.735+0000 7f1444aa0640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e9899fe950 0x55e989987460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:13.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.735+0000 7f1444aa0640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e9899fe950 0x55e989987460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35974/0 (socket says 172.21.15.38:35974) 2024-09-21T03:43:13.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.735+0000 7f1444aa0640 1 -- 172.21.15.38:0/2656082291 learned_addr learned my addr 172.21.15.38:0/2656082291 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:43:13.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.735+0000 7f1443a9e640 1 --2- 172.21.15.38:0/2656082291 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e98996fe30 0x55e9899f80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:13.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.735+0000 7f144429f640 1 --2- 172.21.15.38:0/2656082291 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e989943d00 0x55e989ab0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:13.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.735+0000 7f1444aa0640 1 -- 172.21.15.38:0/2656082291 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e989943d00 msgr2=0x55e989ab0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:13.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.735+0000 7f1444aa0640 1 --2- 172.21.15.38:0/2656082291 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e989943d00 0x55e989ab0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:13.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.735+0000 7f1444aa0640 1 -- 172.21.15.38:0/2656082291 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e98996fe30 msgr2=0x55e9899f80a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:13.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.735+0000 7f1444aa0640 1 --2- 172.21.15.38:0/2656082291 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e98996fe30 0x55e9899f80a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:13.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.735+0000 7f1444aa0640 1 -- 172.21.15.38:0/2656082291 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9899fcae0 con 0x55e9899fe950 2024-09-21T03:43:13.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.735+0000 7f1444aa0640 1 --2- 172.21.15.38:0/2656082291 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e9899fe950 0x55e989987460 secure :-1 s=READY pgs=1969 cs=0 l=1 rev1=1 crypto rx=0x7f143c003130 tx=0x7f143c003220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:13.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.735+0000 7f1441a9a640 1 -- 172.21.15.38:0/2656082291 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f143c013480 con 0x55e9899fe950 2024-09-21T03:43:13.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.736+0000 7f1441a9a640 1 -- 172.21.15.38:0/2656082291 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f143c018070 con 0x55e9899fe950 2024-09-21T03:43:13.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.736+0000 7f1441a9a640 1 -- 172.21.15.38:0/2656082291 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f143c0139b0 con 0x55e9899fe950 2024-09-21T03:43:13.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.736+0000 7f144a7e5f40 1 -- 172.21.15.38:0/2656082291 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55e98996d930 con 0x55e9899fe950 2024-09-21T03:43:13.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.736+0000 7f144a7e5f40 1 -- 172.21.15.38:0/2656082291 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e989986f40 con 0x55e9899fe950 2024-09-21T03:43:13.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.737+0000 7f1441a9a640 1 -- 172.21.15.38:0/2656082291 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f143c013b50 con 0x55e9899fe950 2024-09-21T03:43:13.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.738+0000 7f1441a9a640 1 --2- 172.21.15.38:0/2656082291 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f14300775a0 0x7f1430079ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:13.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.738+0000 7f1441a9a640 1 -- 172.21.15.38:0/2656082291 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 8884+0+0 (secure 0 0 0) 0x7f143c09a450 con 0x55e9899fe950 2024-09-21T03:43:13.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.738+0000 7f144429f640 1 --2- 172.21.15.38:0/2656082291 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f14300775a0 0x7f1430079ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:13.741 INFO:teuthology.orchestra.run.smithi038.stdout:Importing pool 2024-09-21T03:43:13.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.738+0000 7f144a7e5f40 1 --2- 172.21.15.38:0/2656082291 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x55e9899884c0 0x55e989abd180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:13.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.738+0000 7f144a7e5f40 1 -- 172.21.15.38:0/2656082291 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55e9899879a0 con 0x55e9899884c0 2024-09-21T03:43:13.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.738+0000 7f1443a9e640 1 --2- 172.21.15.38:0/2656082291 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x55e9899884c0 0x55e989abd180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:13.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.739+0000 7f144429f640 1 --2- 172.21.15.38:0/2656082291 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f14300775a0 0x7f1430079ab0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f14340104f0 tx=0x7f1434003810 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:13.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.739+0000 7f1443a9e640 1 --2- 172.21.15.38:0/2656082291 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x55e9899884c0 0x55e989abd180 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:13.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.741+0000 7f1443a9e640 1 -- 172.21.15.38:0/2656082291 <== osd.7 v2:172.21.15.195:6818/1606106342 1 ==== osd_op_reply(1 601.00000000 [create] v126'908 uv882 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f143806a040 con 0x55e9899884c0 2024-09-21T03:43:13.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.741+0000 7f144a7e5f40 1 -- 172.21.15.38:0/2656082291 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e126) -- 0x55e989a15a50 con 0x55e9899884c0 2024-09-21T03:43:13.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.743+0000 7f1443a9e640 1 -- 172.21.15.38:0/2656082291 <== osd.7 v2:172.21.15.195:6818/1606106342 2 ==== osd_op_reply(2 601.00000000 [delete] v126'909 uv909 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f143806a040 con 0x55e9899884c0 2024-09-21T03:43:13.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.743+0000 7f144a7e5f40 1 -- 172.21.15.38:0/2656082291 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55e989abd6c0 con 0x55e9899884c0 2024-09-21T03:43:13.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.745+0000 7f1443a9e640 1 -- 172.21.15.38:0/2656082291 <== osd.7 v2:172.21.15.195:6818/1606106342 3 ==== osd_op_reply(3 601.00000000 [create] v126'910 uv910 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f143806a040 con 0x55e9899884c0 2024-09-21T03:43:13.747 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-09-21T03:43:13.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.745+0000 7f144a7e5f40 1 -- 172.21.15.38:0/2656082291 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55e989abea70 con 0x55e9899884c0 2024-09-21T03:43:13.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.746+0000 7f1443a9e640 1 -- 172.21.15.38:0/2656082291 <== osd.7 v2:172.21.15.195:6818/1606106342 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v126'911 uv911 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f143806a040 con 0x55e9899884c0 2024-09-21T03:43:13.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.746+0000 7f144a7e5f40 1 -- 172.21.15.38:0/2656082291 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55e989ab12e0 con 0x55e9899884c0 2024-09-21T03:43:13.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.748+0000 7f1443a9e640 1 -- 172.21.15.38:0/2656082291 <== osd.7 v2:172.21.15.195:6818/1606106342 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v126'912 uv912 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f143806a040 con 0x55e9899884c0 2024-09-21T03:43:13.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.748+0000 7f144a7e5f40 1 -- 172.21.15.38:0/2656082291 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55e989abda60 con 0x55e9899884c0 2024-09-21T03:43:13.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.750+0000 7f1443a9e640 1 -- 172.21.15.38:0/2656082291 <== osd.7 v2:172.21.15.195:6818/1606106342 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v126'913 uv913 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f143806a040 con 0x55e9899884c0 2024-09-21T03:43:13.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.750+0000 7f144a7e5f40 1 -- 172.21.15.38:0/2656082291 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55e989abde00 con 0x55e9899884c0 2024-09-21T03:43:13.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.752+0000 7f1443a9e640 1 -- 172.21.15.38:0/2656082291 <== osd.7 v2:172.21.15.195:6818/1606106342 7 ==== osd_op_reply(7 602.00000000 [create] v126'914 uv888 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f143806a040 con 0x55e9899884c0 2024-09-21T03:43:13.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.752+0000 7f144a7e5f40 1 -- 172.21.15.38:0/2656082291 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e126) -- 0x55e989abde00 con 0x55e9899884c0 2024-09-21T03:43:13.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.754+0000 7f1443a9e640 1 -- 172.21.15.38:0/2656082291 <== osd.7 v2:172.21.15.195:6818/1606106342 8 ==== osd_op_reply(8 602.00000000 [delete] v126'915 uv915 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f143806a040 con 0x55e9899884c0 2024-09-21T03:43:13.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.754+0000 7f144a7e5f40 1 -- 172.21.15.38:0/2656082291 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55e989abde00 con 0x55e9899884c0 2024-09-21T03:43:13.757 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-09-21T03:43:13.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.756+0000 7f1443a9e640 1 -- 172.21.15.38:0/2656082291 <== osd.7 v2:172.21.15.195:6818/1606106342 9 ==== osd_op_reply(9 602.00000000 [create] v126'916 uv916 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f143806a040 con 0x55e9899884c0 2024-09-21T03:43:13.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.756+0000 7f144a7e5f40 1 -- 172.21.15.38:0/2656082291 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55e989abde00 con 0x55e9899884c0 2024-09-21T03:43:13.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.757+0000 7f1443a9e640 1 -- 172.21.15.38:0/2656082291 <== osd.7 v2:172.21.15.195:6818/1606106342 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v126'917 uv917 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f143806a040 con 0x55e9899884c0 2024-09-21T03:43:13.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.757+0000 7f144a7e5f40 1 -- 172.21.15.38:0/2656082291 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55e989abde00 con 0x55e9899884c0 2024-09-21T03:43:13.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.759+0000 7f1443a9e640 1 -- 172.21.15.38:0/2656082291 <== osd.7 v2:172.21.15.195:6818/1606106342 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v126'918 uv918 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f143806a040 con 0x55e9899884c0 2024-09-21T03:43:13.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.759+0000 7f144a7e5f40 1 -- 172.21.15.38:0/2656082291 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55e989abde00 con 0x55e9899884c0 2024-09-21T03:43:13.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.760+0000 7f1443a9e640 1 -- 172.21.15.38:0/2656082291 <== osd.7 v2:172.21.15.195:6818/1606106342 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v126'919 uv919 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f143806a040 con 0x55e9899884c0 2024-09-21T03:43:13.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.760+0000 7f144a7e5f40 1 -- 172.21.15.38:0/2656082291 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:13 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55e989abde00 con 0x55e9899884c0 2024-09-21T03:43:13.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.762+0000 7f1443a9e640 1 -- 172.21.15.38:0/2656082291 <== osd.7 v2:172.21.15.195:6818/1606106342 13 ==== osd_op_reply(13 10000000000.00000000 [create] v126'920 uv895 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7f143806a040 con 0x55e9899884c0 2024-09-21T03:43:13.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.762+0000 7f144a7e5f40 1 -- 172.21.15.38:0/2656082291 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:14 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e126) -- 0x55e989abde00 con 0x55e9899884c0 2024-09-21T03:43:13.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.764+0000 7f1443a9e640 1 -- 172.21.15.38:0/2656082291 <== osd.7 v2:172.21.15.195:6818/1606106342 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v126'921 uv921 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f143806a040 con 0x55e9899884c0 2024-09-21T03:43:13.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.764+0000 7f144a7e5f40 1 -- 172.21.15.38:0/2656082291 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:15 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55e989abde00 con 0x55e9899884c0 2024-09-21T03:43:13.767 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-09-21T03:43:13.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.765+0000 7f1443a9e640 1 -- 172.21.15.38:0/2656082291 <== osd.7 v2:172.21.15.195:6818/1606106342 15 ==== osd_op_reply(15 10000000000.00000000 [create] v126'922 uv922 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f143806a040 con 0x55e9899884c0 2024-09-21T03:43:13.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.765+0000 7f144a7e5f40 1 -- 172.21.15.38:0/2656082291 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:16 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55e989abde00 con 0x55e9899884c0 2024-09-21T03:43:13.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.767+0000 7f1443a9e640 1 -- 172.21.15.38:0/2656082291 <== osd.7 v2:172.21.15.195:6818/1606106342 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v126'923 uv923 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f143806a040 con 0x55e9899884c0 2024-09-21T03:43:13.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.767+0000 7f144a7e5f40 1 -- 172.21.15.38:0/2656082291 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:17 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55e989abde00 con 0x55e9899884c0 2024-09-21T03:43:13.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.768+0000 7f1443a9e640 1 -- 172.21.15.38:0/2656082291 <== osd.7 v2:172.21.15.195:6818/1606106342 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v126'924 uv924 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f143806a040 con 0x55e9899884c0 2024-09-21T03:43:13.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.768+0000 7f144a7e5f40 1 -- 172.21.15.38:0/2656082291 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:18 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55e989abde00 con 0x55e9899884c0 2024-09-21T03:43:13.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.770+0000 7f1443a9e640 1 -- 172.21.15.38:0/2656082291 <== osd.7 v2:172.21.15.195:6818/1606106342 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v126'925 uv925 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f143806a040 con 0x55e9899884c0 2024-09-21T03:43:13.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.770+0000 7f144a7e5f40 1 -- 172.21.15.38:0/2656082291 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:19 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=585b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55e989abe1c0 con 0x55e9899884c0 2024-09-21T03:43:13.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.771+0000 7f1443a9e640 1 -- 172.21.15.38:0/2656082291 <== osd.7 v2:172.21.15.195:6818/1606106342 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v126'926 uv926 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f143806a040 con 0x55e9899884c0 2024-09-21T03:43:13.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.771+0000 7f144a7e5f40 1 -- 172.21.15.38:0/2656082291 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:20 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55e989abe630 con 0x55e9899884c0 2024-09-21T03:43:13.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.773+0000 7f1443a9e640 1 -- 172.21.15.38:0/2656082291 <== osd.7 v2:172.21.15.195:6818/1606106342 20 ==== osd_op_reply(20 600.00000000 [create] v126'927 uv901 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f143806a040 con 0x55e9899884c0 2024-09-21T03:43:13.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.773+0000 7f144a7e5f40 1 -- 172.21.15.38:0/2656082291 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:21 13.10 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e126) -- 0x55e989abe630 con 0x55e9899884c0 2024-09-21T03:43:13.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.774+0000 7f1443a9e640 1 -- 172.21.15.38:0/2656082291 <== osd.7 v2:172.21.15.195:6818/1606106342 21 ==== osd_op_reply(21 600.00000000 [delete] v126'928 uv928 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f143806a040 con 0x55e9899884c0 2024-09-21T03:43:13.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.774+0000 7f144a7e5f40 1 -- 172.21.15.38:0/2656082291 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:22 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55e989abe0b0 con 0x55e9899884c0 2024-09-21T03:43:13.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.776+0000 7f1443a9e640 1 -- 172.21.15.38:0/2656082291 <== osd.7 v2:172.21.15.195:6818/1606106342 22 ==== osd_op_reply(22 600.00000000 [create] v126'929 uv929 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f143806a040 con 0x55e9899884c0 2024-09-21T03:43:13.778 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-09-21T03:43:13.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.776+0000 7f144a7e5f40 1 -- 172.21.15.38:0/2656082291 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:23 13.10 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55e989abe0b0 con 0x55e9899884c0 2024-09-21T03:43:13.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.778+0000 7f1443a9e640 1 -- 172.21.15.38:0/2656082291 <== osd.7 v2:172.21.15.195:6818/1606106342 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v126'930 uv930 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f143806a040 con 0x55e9899884c0 2024-09-21T03:43:13.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.778+0000 7f144a7e5f40 1 -- 172.21.15.38:0/2656082291 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:24 13.10 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55e989abe0b0 con 0x55e9899884c0 2024-09-21T03:43:13.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.779+0000 7f1443a9e640 1 -- 172.21.15.38:0/2656082291 <== osd.7 v2:172.21.15.195:6818/1606106342 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v126'931 uv931 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f143806a040 con 0x55e9899884c0 2024-09-21T03:43:13.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.779+0000 7f144a7e5f40 1 -- 172.21.15.38:0/2656082291 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:25 13.10 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55e989abe0b0 con 0x55e9899884c0 2024-09-21T03:43:13.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.781+0000 7f1443a9e640 1 -- 172.21.15.38:0/2656082291 <== osd.7 v2:172.21.15.195:6818/1606106342 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v126'932 uv932 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f143806a040 con 0x55e9899884c0 2024-09-21T03:43:13.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.781+0000 7f144a7e5f40 1 -- 172.21.15.38:0/2656082291 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:26 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55e989abe0b0 con 0x55e9899884c0 2024-09-21T03:43:13.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.783+0000 7f1443a9e640 1 -- 172.21.15.38:0/2656082291 <== osd.7 v2:172.21.15.195:6818/1606106342 26 ==== osd_op_reply(26 603.00000000 [create] v126'933 uv907 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f143806a040 con 0x55e9899884c0 2024-09-21T03:43:13.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.783+0000 7f144a7e5f40 1 -- 172.21.15.38:0/2656082291 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:27 13.10 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e126) -- 0x55e989abe0b0 con 0x55e9899884c0 2024-09-21T03:43:13.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.784+0000 7f1443a9e640 1 -- 172.21.15.38:0/2656082291 <== osd.7 v2:172.21.15.195:6818/1606106342 27 ==== osd_op_reply(27 603.00000000 [delete] v126'934 uv934 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f143806a040 con 0x55e9899884c0 2024-09-21T03:43:13.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.784+0000 7f144a7e5f40 1 -- 172.21.15.38:0/2656082291 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:28 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55e989abe0b0 con 0x55e9899884c0 2024-09-21T03:43:13.788 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-09-21T03:43:13.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.786+0000 7f1443a9e640 1 -- 172.21.15.38:0/2656082291 <== osd.7 v2:172.21.15.195:6818/1606106342 28 ==== osd_op_reply(28 603.00000000 [create] v126'935 uv935 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f143806a040 con 0x55e9899884c0 2024-09-21T03:43:13.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.786+0000 7f144a7e5f40 1 -- 172.21.15.38:0/2656082291 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:29 13.10 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55e989abe0b0 con 0x55e9899884c0 2024-09-21T03:43:13.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.787+0000 7f1443a9e640 1 -- 172.21.15.38:0/2656082291 <== osd.7 v2:172.21.15.195:6818/1606106342 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v126'936 uv936 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f143806a040 con 0x55e9899884c0 2024-09-21T03:43:13.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.787+0000 7f144a7e5f40 1 -- 172.21.15.38:0/2656082291 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:30 13.10 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55e989abe0b0 con 0x55e9899884c0 2024-09-21T03:43:13.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.789+0000 7f1443a9e640 1 -- 172.21.15.38:0/2656082291 <== osd.7 v2:172.21.15.195:6818/1606106342 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v126'937 uv937 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f143806a040 con 0x55e9899884c0 2024-09-21T03:43:13.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.789+0000 7f144a7e5f40 1 -- 172.21.15.38:0/2656082291 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:31 13.10 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55e989abe0b0 con 0x55e9899884c0 2024-09-21T03:43:13.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.790+0000 7f1443a9e640 1 -- 172.21.15.38:0/2656082291 <== osd.7 v2:172.21.15.195:6818/1606106342 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v126'938 uv938 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f143806a040 con 0x55e9899884c0 2024-09-21T03:43:13.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.790+0000 7f144a7e5f40 1 --2- 172.21.15.38:0/2656082291 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x55e989abee10 0x55e989ac1230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:13.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.790+0000 7f144a7e5f40 1 -- 172.21.15.38:0/2656082291 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55e989abe0b0 con 0x55e989abee10 2024-09-21T03:43:13.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.791+0000 7f144429f640 1 --2- 172.21.15.38:0/2656082291 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x55e989abee10 0x55e989ac1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:13.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.791+0000 7f144429f640 1 --2- 172.21.15.38:0/2656082291 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x55e989abee10 0x55e989ac1230 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:13.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.793+0000 7f144429f640 1 -- 172.21.15.38:0/2656082291 <== osd.6 v2:172.21.15.195:6826/4055825675 1 ==== osd_op_reply(32 1.00000000.inode [create] v126'658 uv641 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f143401b040 con 0x55e989abee10 2024-09-21T03:43:13.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.793+0000 7f144a7e5f40 1 -- 172.21.15.38:0/2656082291 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e126) -- 0x55e989ac1770 con 0x55e989abee10 2024-09-21T03:43:13.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.795+0000 7f144429f640 1 -- 172.21.15.38:0/2656082291 <== osd.6 v2:172.21.15.195:6826/4055825675 2 ==== osd_op_reply(33 1.00000000.inode [delete] v126'659 uv659 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f143401b040 con 0x55e989abee10 2024-09-21T03:43:13.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.795+0000 7f144a7e5f40 1 -- 172.21.15.38:0/2656082291 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55e989ac1b10 con 0x55e989abee10 2024-09-21T03:43:13.798 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-09-21T03:43:13.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.796+0000 7f144429f640 1 -- 172.21.15.38:0/2656082291 <== osd.6 v2:172.21.15.195:6826/4055825675 3 ==== osd_op_reply(34 1.00000000.inode [create] v126'660 uv660 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f143401b040 con 0x55e989abee10 2024-09-21T03:43:13.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.796+0000 7f144a7e5f40 1 -- 172.21.15.38:0/2656082291 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55e989ac2360 con 0x55e989abee10 2024-09-21T03:43:13.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.798+0000 7f144429f640 1 -- 172.21.15.38:0/2656082291 <== osd.6 v2:172.21.15.195:6826/4055825675 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v126'661 uv661 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f143401b040 con 0x55e989abee10 2024-09-21T03:43:13.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.798+0000 7f144a7e5f40 1 -- 172.21.15.38:0/2656082291 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55e989ac2700 con 0x55e989abee10 2024-09-21T03:43:13.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.799+0000 7f144429f640 1 -- 172.21.15.38:0/2656082291 <== osd.6 v2:172.21.15.195:6826/4055825675 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v126'662 uv662 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f143401b040 con 0x55e989abee10 2024-09-21T03:43:13.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.799+0000 7f144a7e5f40 1 -- 172.21.15.38:0/2656082291 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55e989ac2aa0 con 0x55e989abee10 2024-09-21T03:43:13.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.800+0000 7f144429f640 1 -- 172.21.15.38:0/2656082291 <== osd.6 v2:172.21.15.195:6826/4055825675 6 ==== osd_op_reply(37 200.00000000 [create] v126'663 uv646 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f143401b040 con 0x55e989abee10 2024-09-21T03:43:13.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.800+0000 7f144a7e5f40 1 -- 172.21.15.38:0/2656082291 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e126) -- 0x55e989ac2e40 con 0x55e989abee10 2024-09-21T03:43:13.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.802+0000 7f144429f640 1 -- 172.21.15.38:0/2656082291 <== osd.6 v2:172.21.15.195:6826/4055825675 7 ==== osd_op_reply(38 200.00000000 [delete] v126'664 uv664 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f143401b040 con 0x55e989abee10 2024-09-21T03:43:13.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.802+0000 7f144a7e5f40 1 -- 172.21.15.38:0/2656082291 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55e989ac31e0 con 0x55e989abee10 2024-09-21T03:43:13.805 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-09-21T03:43:13.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.803+0000 7f144429f640 1 -- 172.21.15.38:0/2656082291 <== osd.6 v2:172.21.15.195:6826/4055825675 8 ==== osd_op_reply(39 200.00000000 [create] v126'665 uv665 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f143401b040 con 0x55e989abee10 2024-09-21T03:43:13.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.803+0000 7f144a7e5f40 1 -- 172.21.15.38:0/2656082291 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55e989ac31e0 con 0x55e989abee10 2024-09-21T03:43:13.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.804+0000 7f144429f640 1 -- 172.21.15.38:0/2656082291 <== osd.6 v2:172.21.15.195:6826/4055825675 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v126'666 uv666 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f143401b040 con 0x55e989abee10 2024-09-21T03:43:13.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.804+0000 7f144a7e5f40 1 -- 172.21.15.38:0/2656082291 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55e989ac31e0 con 0x55e989abee10 2024-09-21T03:43:13.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.805+0000 7f144429f640 1 -- 172.21.15.38:0/2656082291 <== osd.6 v2:172.21.15.195:6826/4055825675 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v126'667 uv667 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f143401b040 con 0x55e989abee10 2024-09-21T03:43:13.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.805+0000 7f144a7e5f40 1 -- 172.21.15.38:0/2656082291 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:42 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55e989ac31e0 con 0x55e989abee10 2024-09-21T03:43:13.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.807+0000 7f144429f640 1 -- 172.21.15.38:0/2656082291 <== osd.6 v2:172.21.15.195:6826/4055825675 11 ==== osd_op_reply(42 200.00000001 [create] v126'668 uv651 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f143401b040 con 0x55e989abee10 2024-09-21T03:43:13.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.807+0000 7f144a7e5f40 1 -- 172.21.15.38:0/2656082291 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:43 13.14 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e126) -- 0x55e989ac31e0 con 0x55e989abee10 2024-09-21T03:43:13.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.808+0000 7f144429f640 1 -- 172.21.15.38:0/2656082291 <== osd.6 v2:172.21.15.195:6826/4055825675 12 ==== osd_op_reply(43 200.00000001 [delete] v126'669 uv669 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f143401b040 con 0x55e989abee10 2024-09-21T03:43:13.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.808+0000 7f144a7e5f40 1 -- 172.21.15.38:0/2656082291 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:44 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55e989ac31e0 con 0x55e989abee10 2024-09-21T03:43:13.811 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-09-21T03:43:13.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.809+0000 7f144429f640 1 -- 172.21.15.38:0/2656082291 <== osd.6 v2:172.21.15.195:6826/4055825675 13 ==== osd_op_reply(44 200.00000001 [create] v126'670 uv670 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f143401b040 con 0x55e989abee10 2024-09-21T03:43:13.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.809+0000 7f144a7e5f40 1 -- 172.21.15.38:0/2656082291 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:45 13.14 13:2e2fa760:::200.00000001:head [write 0~17490 in=17490b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55e989ac7660 con 0x55e989abee10 2024-09-21T03:43:13.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.811+0000 7f144429f640 1 -- 172.21.15.38:0/2656082291 <== osd.6 v2:172.21.15.195:6826/4055825675 14 ==== osd_op_reply(45 200.00000001 [write 0~17490] v126'671 uv671 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f143401b040 con 0x55e989abee10 2024-09-21T03:43:13.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.811+0000 7f144a7e5f40 1 -- 172.21.15.38:0/2656082291 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:46 13.14 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55e989ac7660 con 0x55e989abee10 2024-09-21T03:43:13.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.812+0000 7f144429f640 1 -- 172.21.15.38:0/2656082291 <== osd.6 v2:172.21.15.195:6826/4055825675 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v126'672 uv672 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f143401b040 con 0x55e989abee10 2024-09-21T03:43:13.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.812+0000 7f144a7e5f40 1 -- 172.21.15.38:0/2656082291 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:47 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55e989ac31e0 con 0x55e989abee10 2024-09-21T03:43:13.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.813+0000 7f144429f640 1 -- 172.21.15.38:0/2656082291 <== osd.6 v2:172.21.15.195:6826/4055825675 16 ==== osd_op_reply(47 606.00000000 [create] v126'673 uv657 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f143401b040 con 0x55e989abee10 2024-09-21T03:43:13.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.813+0000 7f144a7e5f40 1 -- 172.21.15.38:0/2656082291 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:48 13.14 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e126) -- 0x55e989ac31e0 con 0x55e989abee10 2024-09-21T03:43:13.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.814+0000 7f144429f640 1 -- 172.21.15.38:0/2656082291 <== osd.6 v2:172.21.15.195:6826/4055825675 17 ==== osd_op_reply(48 606.00000000 [delete] v126'674 uv674 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f143401b040 con 0x55e989abee10 2024-09-21T03:43:13.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.814+0000 7f144a7e5f40 1 -- 172.21.15.38:0/2656082291 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:49 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55e989ac31e0 con 0x55e989abee10 2024-09-21T03:43:13.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.816+0000 7f144429f640 1 -- 172.21.15.38:0/2656082291 <== osd.6 v2:172.21.15.195:6826/4055825675 18 ==== osd_op_reply(49 606.00000000 [create] v126'675 uv675 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f143401b040 con 0x55e989abee10 2024-09-21T03:43:13.818 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-09-21T03:43:13.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.816+0000 7f144a7e5f40 1 -- 172.21.15.38:0/2656082291 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:50 13.14 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55e989ac41f0 con 0x55e989abee10 2024-09-21T03:43:13.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.817+0000 7f144429f640 1 -- 172.21.15.38:0/2656082291 <== osd.6 v2:172.21.15.195:6826/4055825675 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v126'676 uv676 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f143401b040 con 0x55e989abee10 2024-09-21T03:43:13.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.817+0000 7f144a7e5f40 1 -- 172.21.15.38:0/2656082291 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:51 13.14 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55e989ac41f0 con 0x55e989abee10 2024-09-21T03:43:13.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.818+0000 7f144429f640 1 -- 172.21.15.38:0/2656082291 <== osd.6 v2:172.21.15.195:6826/4055825675 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v126'677 uv677 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f143401b040 con 0x55e989abee10 2024-09-21T03:43:13.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.819+0000 7f144a7e5f40 1 -- 172.21.15.38:0/2656082291 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:52 13.14 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55e989ac31e0 con 0x55e989abee10 2024-09-21T03:43:13.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.820+0000 7f144429f640 1 -- 172.21.15.38:0/2656082291 <== osd.6 v2:172.21.15.195:6826/4055825675 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v126'678 uv678 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f143401b040 con 0x55e989abee10 2024-09-21T03:43:13.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.820+0000 7f144a7e5f40 1 --2- 172.21.15.38:0/2656082291 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x55e989ac3860 0x55e989ac5c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:13.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.820+0000 7f144a7e5f40 1 -- 172.21.15.38:0/2656082291 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:53 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55e989ac6200 con 0x55e989ac3860 2024-09-21T03:43:13.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.820+0000 7f1443a9e640 1 --2- 172.21.15.38:0/2656082291 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x55e989ac3860 0x55e989ac5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:13.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.820+0000 7f1443a9e640 1 --2- 172.21.15.38:0/2656082291 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x55e989ac3860 0x55e989ac5c80 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:13.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.822+0000 7f1443a9e640 1 -- 172.21.15.38:0/2656082291 <== osd.0 v2:172.21.15.38:6800/2381639433 1 ==== osd_op_reply(53 607.00000000 [create] v126'177 uv176 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f143806a040 con 0x55e989ac3860 2024-09-21T03:43:13.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.822+0000 7f144a7e5f40 1 -- 172.21.15.38:0/2656082291 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:54 13.1c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e126) -- 0x55e989ac65d0 con 0x55e989ac3860 2024-09-21T03:43:13.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.823+0000 7f1443a9e640 1 -- 172.21.15.38:0/2656082291 <== osd.0 v2:172.21.15.38:6800/2381639433 2 ==== osd_op_reply(54 607.00000000 [delete] v126'178 uv178 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f143806a040 con 0x55e989ac3860 2024-09-21T03:43:13.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.823+0000 7f144a7e5f40 1 -- 172.21.15.38:0/2656082291 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:55 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55e989ac65d0 con 0x55e989ac3860 2024-09-21T03:43:13.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.824+0000 7f1443a9e640 1 -- 172.21.15.38:0/2656082291 <== osd.0 v2:172.21.15.38:6800/2381639433 3 ==== osd_op_reply(55 607.00000000 [create] v126'179 uv179 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f143806a040 con 0x55e989ac3860 2024-09-21T03:43:13.826 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-09-21T03:43:13.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.824+0000 7f144a7e5f40 1 -- 172.21.15.38:0/2656082291 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:56 13.1c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55e989ac75e0 con 0x55e989ac3860 2024-09-21T03:43:13.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.825+0000 7f1443a9e640 1 -- 172.21.15.38:0/2656082291 <== osd.0 v2:172.21.15.38:6800/2381639433 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v126'180 uv180 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f143806a040 con 0x55e989ac3860 2024-09-21T03:43:13.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.825+0000 7f144a7e5f40 1 -- 172.21.15.38:0/2656082291 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:57 13.1c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55e989ac75e0 con 0x55e989ac3860 2024-09-21T03:43:13.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.826+0000 7f1443a9e640 1 -- 172.21.15.38:0/2656082291 <== osd.0 v2:172.21.15.38:6800/2381639433 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v126'181 uv181 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f143806a040 con 0x55e989ac3860 2024-09-21T03:43:13.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.826+0000 7f144a7e5f40 1 -- 172.21.15.38:0/2656082291 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:58 13.1c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55e989ac65d0 con 0x55e989ac3860 2024-09-21T03:43:13.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.827+0000 7f1443a9e640 1 -- 172.21.15.38:0/2656082291 <== osd.0 v2:172.21.15.38:6800/2381639433 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v126'182 uv182 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f143806a040 con 0x55e989ac3860 2024-09-21T03:43:13.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.827+0000 7f144a7e5f40 1 --2- 172.21.15.38:0/2656082291 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x55e989ac6c50 0x55e989ac9120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:13.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.827+0000 7f144a7e5f40 1 -- 172.21.15.38:0/2656082291 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:59 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55e989ac97b0 con 0x55e989ac6c50 2024-09-21T03:43:13.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.827+0000 7f1444aa0640 1 --2- 172.21.15.38:0/2656082291 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x55e989ac6c50 0x55e989ac9120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:13.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.827+0000 7f1444aa0640 1 --2- 172.21.15.38:0/2656082291 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x55e989ac6c50 0x55e989ac9120 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:13.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.828+0000 7f1444aa0640 1 -- 172.21.15.38:0/2656082291 <== osd.3 v2:172.21.15.38:6808/1286810294 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v126'128 uv127 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f143c068040 con 0x55e989ac6c50 2024-09-21T03:43:13.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.828+0000 7f144a7e5f40 1 -- 172.21.15.38:0/2656082291 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:60 13.6 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e126) -- 0x55e989ac9cc0 con 0x55e989ac6c50 2024-09-21T03:43:13.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.830+0000 7f1444aa0640 1 -- 172.21.15.38:0/2656082291 <== osd.3 v2:172.21.15.38:6808/1286810294 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v126'129 uv129 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f143c068040 con 0x55e989ac6c50 2024-09-21T03:43:13.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.830+0000 7f144a7e5f40 1 -- 172.21.15.38:0/2656082291 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:61 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55e989aca1c0 con 0x55e989ac6c50 2024-09-21T03:43:13.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.830+0000 7f1444aa0640 1 -- 172.21.15.38:0/2656082291 <== osd.3 v2:172.21.15.38:6808/1286810294 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v126'130 uv130 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f143c068040 con 0x55e989ac6c50 2024-09-21T03:43:13.832 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-09-21T03:43:13.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.831+0000 7f144a7e5f40 1 -- 172.21.15.38:0/2656082291 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:62 13.6 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55e989acaa80 con 0x55e989ac6c50 2024-09-21T03:43:13.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.832+0000 7f1444aa0640 1 -- 172.21.15.38:0/2656082291 <== osd.3 v2:172.21.15.38:6808/1286810294 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v126'131 uv131 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f143c068040 con 0x55e989ac6c50 2024-09-21T03:43:13.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.832+0000 7f144a7e5f40 1 -- 172.21.15.38:0/2656082291 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55e989acae20 con 0x55e989ac3860 2024-09-21T03:43:13.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.833+0000 7f1443a9e640 1 -- 172.21.15.38:0/2656082291 <== osd.0 v2:172.21.15.38:6800/2381639433 7 ==== osd_op_reply(63 608.00000000 [create] v126'177 uv176 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f143806a040 con 0x55e989ac3860 2024-09-21T03:43:13.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.833+0000 7f144a7e5f40 1 -- 172.21.15.38:0/2656082291 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e126) -- 0x55e989acae20 con 0x55e989ac3860 2024-09-21T03:43:13.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.834+0000 7f1443a9e640 1 -- 172.21.15.38:0/2656082291 <== osd.0 v2:172.21.15.38:6800/2381639433 8 ==== osd_op_reply(64 608.00000000 [delete] v126'178 uv178 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f143806a040 con 0x55e989ac3860 2024-09-21T03:43:13.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.834+0000 7f144a7e5f40 1 -- 172.21.15.38:0/2656082291 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55e989acae20 con 0x55e989ac3860 2024-09-21T03:43:13.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.835+0000 7f1443a9e640 1 -- 172.21.15.38:0/2656082291 <== osd.0 v2:172.21.15.38:6800/2381639433 9 ==== osd_op_reply(65 608.00000000 [create] v126'179 uv179 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f143806a040 con 0x55e989ac3860 2024-09-21T03:43:13.837 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-09-21T03:43:13.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.835+0000 7f144a7e5f40 1 -- 172.21.15.38:0/2656082291 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55e989acbe30 con 0x55e989ac3860 2024-09-21T03:43:13.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.836+0000 7f1443a9e640 1 -- 172.21.15.38:0/2656082291 <== osd.0 v2:172.21.15.38:6800/2381639433 10 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v126'180 uv180 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f143806a040 con 0x55e989ac3860 2024-09-21T03:43:13.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.836+0000 7f144a7e5f40 1 -- 172.21.15.38:0/2656082291 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55e989acbe30 con 0x55e989ac3860 2024-09-21T03:43:13.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.837+0000 7f1443a9e640 1 -- 172.21.15.38:0/2656082291 <== osd.0 v2:172.21.15.38:6800/2381639433 11 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v126'181 uv181 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f143806a040 con 0x55e989ac3860 2024-09-21T03:43:13.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.837+0000 7f144a7e5f40 1 -- 172.21.15.38:0/2656082291 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55e989acae20 con 0x55e989ac3860 2024-09-21T03:43:13.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.838+0000 7f1443a9e640 1 -- 172.21.15.38:0/2656082291 <== osd.0 v2:172.21.15.38:6800/2381639433 12 ==== osd_op_reply(68 608.00000000 [omap-set-header] v126'182 uv182 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f143806a040 con 0x55e989ac3860 2024-09-21T03:43:13.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.838+0000 7f144a7e5f40 1 -- 172.21.15.38:0/2656082291 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55e989acae20 con 0x55e989ac6c50 2024-09-21T03:43:13.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.840+0000 7f1444aa0640 1 -- 172.21.15.38:0/2656082291 <== osd.3 v2:172.21.15.38:6808/1286810294 5 ==== osd_op_reply(69 604.00000000 [create] v126'327 uv321 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f143c068040 con 0x55e989ac6c50 2024-09-21T03:43:13.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.840+0000 7f144a7e5f40 1 -- 172.21.15.38:0/2656082291 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e126) -- 0x55e989acb1c0 con 0x55e989ac6c50 2024-09-21T03:43:13.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.841+0000 7f1444aa0640 1 -- 172.21.15.38:0/2656082291 <== osd.3 v2:172.21.15.38:6808/1286810294 6 ==== osd_op_reply(70 604.00000000 [delete] v126'328 uv328 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f143c068040 con 0x55e989ac6c50 2024-09-21T03:43:13.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.841+0000 7f144a7e5f40 1 -- 172.21.15.38:0/2656082291 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55e989acb560 con 0x55e989ac6c50 2024-09-21T03:43:13.844 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-09-21T03:43:13.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.842+0000 7f1444aa0640 1 -- 172.21.15.38:0/2656082291 <== osd.3 v2:172.21.15.38:6808/1286810294 7 ==== osd_op_reply(71 604.00000000 [create] v126'329 uv329 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f143c068040 con 0x55e989ac6c50 2024-09-21T03:43:13.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.842+0000 7f144a7e5f40 1 -- 172.21.15.38:0/2656082291 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55e989acc910 con 0x55e989ac6c50 2024-09-21T03:43:13.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.843+0000 7f1444aa0640 1 -- 172.21.15.38:0/2656082291 <== osd.3 v2:172.21.15.38:6808/1286810294 8 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v126'330 uv330 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f143c068040 con 0x55e989ac6c50 2024-09-21T03:43:13.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.843+0000 7f144a7e5f40 1 -- 172.21.15.38:0/2656082291 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55e989acc910 con 0x55e989ac6c50 2024-09-21T03:43:13.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.845+0000 7f1444aa0640 1 -- 172.21.15.38:0/2656082291 <== osd.3 v2:172.21.15.38:6808/1286810294 9 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v126'331 uv331 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f143c068040 con 0x55e989ac6c50 2024-09-21T03:43:13.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.845+0000 7f144a7e5f40 1 -- 172.21.15.38:0/2656082291 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55e989acb900 con 0x55e989ac6c50 2024-09-21T03:43:13.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.846+0000 7f1444aa0640 1 -- 172.21.15.38:0/2656082291 <== osd.3 v2:172.21.15.38:6808/1286810294 10 ==== osd_op_reply(74 604.00000000 [omap-set-header] v126'332 uv332 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f143c068040 con 0x55e989ac6c50 2024-09-21T03:43:13.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.846+0000 7f144a7e5f40 1 -- 172.21.15.38:0/2656082291 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55e989acb900 con 0x55e989ac6c50 2024-09-21T03:43:13.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.847+0000 7f1444aa0640 1 -- 172.21.15.38:0/2656082291 <== osd.3 v2:172.21.15.38:6808/1286810294 11 ==== osd_op_reply(75 500.00000000 [create] v126'333 uv326 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f143c068040 con 0x55e989ac6c50 2024-09-21T03:43:13.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.847+0000 7f144a7e5f40 1 -- 172.21.15.38:0/2656082291 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e126) -- 0x55e989acb900 con 0x55e989ac6c50 2024-09-21T03:43:13.850 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.848+0000 7f1444aa0640 1 -- 172.21.15.38:0/2656082291 <== osd.3 v2:172.21.15.38:6808/1286810294 12 ==== osd_op_reply(76 500.00000000 [delete] v126'334 uv334 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f143c068040 con 0x55e989ac6c50 2024-09-21T03:43:13.850 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.848+0000 7f144a7e5f40 1 -- 172.21.15.38:0/2656082291 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55e989acb900 con 0x55e989ac6c50 2024-09-21T03:43:13.851 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-09-21T03:43:13.851 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.849+0000 7f1444aa0640 1 -- 172.21.15.38:0/2656082291 <== osd.3 v2:172.21.15.38:6808/1286810294 13 ==== osd_op_reply(77 500.00000000 [create] v126'335 uv335 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f143c068040 con 0x55e989ac6c50 2024-09-21T03:43:13.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.849+0000 7f144a7e5f40 1 -- 172.21.15.38:0/2656082291 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55e989acb900 con 0x55e989ac6c50 2024-09-21T03:43:13.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.851+0000 7f1444aa0640 1 -- 172.21.15.38:0/2656082291 <== osd.3 v2:172.21.15.38:6808/1286810294 14 ==== osd_op_reply(78 500.00000000 [write 0~90] v126'336 uv336 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f143c068040 con 0x55e989ac6c50 2024-09-21T03:43:13.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.851+0000 7f144a7e5f40 1 -- 172.21.15.38:0/2656082291 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55e989acb900 con 0x55e989ac6c50 2024-09-21T03:43:13.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.852+0000 7f1444aa0640 1 -- 172.21.15.38:0/2656082291 <== osd.3 v2:172.21.15.38:6808/1286810294 15 ==== osd_op_reply(79 500.00000000 [omap-set-header] v126'337 uv337 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f143c068040 con 0x55e989ac6c50 2024-09-21T03:43:13.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.852+0000 7f144a7e5f40 1 -- 172.21.15.38:0/2656082291 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:80 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55e989acb900 con 0x55e989ac3860 2024-09-21T03:43:13.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.853+0000 7f1443a9e640 1 -- 172.21.15.38:0/2656082291 <== osd.0 v2:172.21.15.38:6800/2381639433 13 ==== osd_op_reply(80 mds_snaptable [create] v126'148 uv147 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f143806a040 con 0x55e989ac3860 2024-09-21T03:43:13.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.853+0000 7f144a7e5f40 1 -- 172.21.15.38:0/2656082291 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:81 13.d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e126) -- 0x55e989acb900 con 0x55e989ac3860 2024-09-21T03:43:13.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.854+0000 7f1443a9e640 1 -- 172.21.15.38:0/2656082291 <== osd.0 v2:172.21.15.38:6800/2381639433 14 ==== osd_op_reply(81 mds_snaptable [delete] v126'149 uv149 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f143806a040 con 0x55e989ac3860 2024-09-21T03:43:13.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.854+0000 7f144a7e5f40 1 -- 172.21.15.38:0/2656082291 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:82 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55e989acb900 con 0x55e989ac3860 2024-09-21T03:43:13.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.855+0000 7f1443a9e640 1 -- 172.21.15.38:0/2656082291 <== osd.0 v2:172.21.15.38:6800/2381639433 15 ==== osd_op_reply(82 mds_snaptable [create] v126'150 uv150 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f143806a040 con 0x55e989ac3860 2024-09-21T03:43:13.857 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-09-21T03:43:13.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.855+0000 7f144a7e5f40 1 -- 172.21.15.38:0/2656082291 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:83 13.d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55e989acb900 con 0x55e989ac3860 2024-09-21T03:43:13.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.856+0000 7f1443a9e640 1 -- 172.21.15.38:0/2656082291 <== osd.0 v2:172.21.15.38:6800/2381639433 16 ==== osd_op_reply(83 mds_snaptable [write 0~70] v126'151 uv151 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f143806a040 con 0x55e989ac3860 2024-09-21T03:43:13.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.856+0000 7f144a7e5f40 1 -- 172.21.15.38:0/2656082291 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:84 13.d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55e989acb900 con 0x55e989ac3860 2024-09-21T03:43:13.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.857+0000 7f1443a9e640 1 -- 172.21.15.38:0/2656082291 <== osd.0 v2:172.21.15.38:6800/2381639433 17 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v126'152 uv152 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f143806a040 con 0x55e989ac3860 2024-09-21T03:43:13.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.857+0000 7f144a7e5f40 1 --2- 172.21.15.38:0/2656082291 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x55e989acbf80 0x55e989ace3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:13.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.857+0000 7f144a7e5f40 1 -- 172.21.15.38:0/2656082291 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55e989acea10 con 0x55e989acbf80 2024-09-21T03:43:13.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.858+0000 7f144429f640 1 --2- 172.21.15.38:0/2656082291 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x55e989acbf80 0x55e989ace3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:13.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.858+0000 7f144429f640 1 --2- 172.21.15.38:0/2656082291 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x55e989acbf80 0x55e989ace3a0 crc :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:13.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.859+0000 7f144429f640 1 -- 172.21.15.38:0/2656082291 <== osd.1 v2:172.21.15.38:6816/1491818732 1 ==== osd_op_reply(85 605.00000000 [create] v126'177 uv176 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f143401b040 con 0x55e989acbf80 2024-09-21T03:43:13.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.859+0000 7f144a7e5f40 1 -- 172.21.15.38:0/2656082291 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e126) -- 0x55e989acede0 con 0x55e989acbf80 2024-09-21T03:43:13.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.861+0000 7f144429f640 1 -- 172.21.15.38:0/2656082291 <== osd.1 v2:172.21.15.38:6816/1491818732 2 ==== osd_op_reply(86 605.00000000 [delete] v126'178 uv178 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f143401b040 con 0x55e989acbf80 2024-09-21T03:43:13.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.861+0000 7f144a7e5f40 1 -- 172.21.15.38:0/2656082291 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55e989acede0 con 0x55e989acbf80 2024-09-21T03:43:13.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.862+0000 7f144429f640 1 -- 172.21.15.38:0/2656082291 <== osd.1 v2:172.21.15.38:6816/1491818732 3 ==== osd_op_reply(87 605.00000000 [create] v126'179 uv179 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f143401b040 con 0x55e989acbf80 2024-09-21T03:43:13.864 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-09-21T03:43:13.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.862+0000 7f144a7e5f40 1 -- 172.21.15.38:0/2656082291 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55e989acfdf0 con 0x55e989acbf80 2024-09-21T03:43:13.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.863+0000 7f144429f640 1 -- 172.21.15.38:0/2656082291 <== osd.1 v2:172.21.15.38:6816/1491818732 4 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v126'180 uv180 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f143401b040 con 0x55e989acbf80 2024-09-21T03:43:13.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.863+0000 7f144a7e5f40 1 -- 172.21.15.38:0/2656082291 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55e989acfdf0 con 0x55e989acbf80 2024-09-21T03:43:13.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.864+0000 7f144429f640 1 -- 172.21.15.38:0/2656082291 <== osd.1 v2:172.21.15.38:6816/1491818732 5 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v126'181 uv181 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f143401b040 con 0x55e989acbf80 2024-09-21T03:43:13.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.864+0000 7f144a7e5f40 1 -- 172.21.15.38:0/2656082291 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55e989acede0 con 0x55e989acbf80 2024-09-21T03:43:13.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.865+0000 7f144429f640 1 -- 172.21.15.38:0/2656082291 <== osd.1 v2:172.21.15.38:6816/1491818732 6 ==== osd_op_reply(90 605.00000000 [omap-set-header] v126'182 uv182 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f143401b040 con 0x55e989acbf80 2024-09-21T03:43:13.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.865+0000 7f144a7e5f40 1 -- 172.21.15.38:0/2656082291 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55e989acede0 con 0x55e989acbf80 2024-09-21T03:43:13.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.866+0000 7f144429f640 1 -- 172.21.15.38:0/2656082291 <== osd.1 v2:172.21.15.38:6816/1491818732 7 ==== osd_op_reply(91 mds0_inotable [create] v126'294 uv288 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f143401b040 con 0x55e989acbf80 2024-09-21T03:43:13.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.867+0000 7f144a7e5f40 1 -- 172.21.15.38:0/2656082291 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e126) -- 0x55e989acede0 con 0x55e989acbf80 2024-09-21T03:43:13.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.868+0000 7f144429f640 1 -- 172.21.15.38:0/2656082291 <== osd.1 v2:172.21.15.38:6816/1491818732 8 ==== osd_op_reply(92 mds0_inotable [delete] v126'295 uv295 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f143401b040 con 0x55e989acbf80 2024-09-21T03:43:13.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.868+0000 7f144a7e5f40 1 -- 172.21.15.38:0/2656082291 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55e989acede0 con 0x55e989acbf80 2024-09-21T03:43:13.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.869+0000 7f144429f640 1 -- 172.21.15.38:0/2656082291 <== osd.1 v2:172.21.15.38:6816/1491818732 9 ==== osd_op_reply(93 mds0_inotable [create] v126'296 uv296 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f143401b040 con 0x55e989acbf80 2024-09-21T03:43:13.871 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-09-21T03:43:13.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.869+0000 7f144a7e5f40 1 -- 172.21.15.38:0/2656082291 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:94 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55e989aca670 con 0x55e989acbf80 2024-09-21T03:43:13.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.870+0000 7f144429f640 1 -- 172.21.15.38:0/2656082291 <== osd.1 v2:172.21.15.38:6816/1491818732 10 ==== osd_op_reply(94 mds0_inotable [write 0~34] v126'297 uv297 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f143401b040 con 0x55e989acbf80 2024-09-21T03:43:13.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.870+0000 7f144a7e5f40 1 -- 172.21.15.38:0/2656082291 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:95 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55e989aca670 con 0x55e989acbf80 2024-09-21T03:43:13.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.871+0000 7f144429f640 1 -- 172.21.15.38:0/2656082291 <== osd.1 v2:172.21.15.38:6816/1491818732 11 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v126'298 uv298 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f143401b040 con 0x55e989acbf80 2024-09-21T03:43:13.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.871+0000 7f144a7e5f40 1 -- 172.21.15.38:0/2656082291 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:96 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55e989aca670 con 0x55e989acbf80 2024-09-21T03:43:13.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.873+0000 7f144429f640 1 -- 172.21.15.38:0/2656082291 <== osd.1 v2:172.21.15.38:6816/1491818732 12 ==== osd_op_reply(96 100.00000000 [create] v126'299 uv293 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f143401b040 con 0x55e989acbf80 2024-09-21T03:43:13.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.873+0000 7f144a7e5f40 1 -- 172.21.15.38:0/2656082291 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:97 13.13 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e126) -- 0x55e989aca670 con 0x55e989acbf80 2024-09-21T03:43:13.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.874+0000 7f144429f640 1 -- 172.21.15.38:0/2656082291 <== osd.1 v2:172.21.15.38:6816/1491818732 13 ==== osd_op_reply(97 100.00000000 [delete] v126'300 uv300 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f143401b040 con 0x55e989acbf80 2024-09-21T03:43:13.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.874+0000 7f144a7e5f40 1 -- 172.21.15.38:0/2656082291 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:98 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55e989aca670 con 0x55e989acbf80 2024-09-21T03:43:13.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.875+0000 7f144429f640 1 -- 172.21.15.38:0/2656082291 <== osd.1 v2:172.21.15.38:6816/1491818732 14 ==== osd_op_reply(98 100.00000000 [create] v126'301 uv301 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f143401b040 con 0x55e989acbf80 2024-09-21T03:43:13.877 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-09-21T03:43:13.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.875+0000 7f144a7e5f40 1 -- 172.21.15.38:0/2656082291 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:99 13.13 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55e989aca670 con 0x55e989acbf80 2024-09-21T03:43:13.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.876+0000 7f144429f640 1 -- 172.21.15.38:0/2656082291 <== osd.1 v2:172.21.15.38:6816/1491818732 15 ==== osd_op_reply(99 100.00000000 [omap-set-header] v126'302 uv302 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f143401b040 con 0x55e989acbf80 2024-09-21T03:43:13.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.876+0000 7f144a7e5f40 1 -- 172.21.15.38:0/2656082291 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:100 13.13 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55e989ad1c90 con 0x55e989acbf80 2024-09-21T03:43:13.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.878+0000 7f144429f640 1 -- 172.21.15.38:0/2656082291 <== osd.1 v2:172.21.15.38:6816/1491818732 16 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v126'303 uv303 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f143401b040 con 0x55e989acbf80 2024-09-21T03:43:13.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.878+0000 7f144a7e5f40 1 -- 172.21.15.38:0/2656082291 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55e989acede0 con 0x55e989ac6c50 2024-09-21T03:43:13.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.879+0000 7f1444aa0640 1 -- 172.21.15.38:0/2656082291 <== osd.3 v2:172.21.15.38:6808/1286810294 16 ==== osd_op_reply(101 mds0_sessionmap [create] v126'128 uv127 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7f143c068040 con 0x55e989ac6c50 2024-09-21T03:43:13.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.879+0000 7f144a7e5f40 1 -- 172.21.15.38:0/2656082291 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e126) -- 0x55e989acede0 con 0x55e989ac6c50 2024-09-21T03:43:13.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.880+0000 7f1444aa0640 1 -- 172.21.15.38:0/2656082291 <== osd.3 v2:172.21.15.38:6808/1286810294 17 ==== osd_op_reply(102 mds0_sessionmap [delete] v126'129 uv129 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f143c068040 con 0x55e989ac6c50 2024-09-21T03:43:13.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.880+0000 7f144a7e5f40 1 -- 172.21.15.38:0/2656082291 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:103 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55e989acede0 con 0x55e989ac6c50 2024-09-21T03:43:13.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.881+0000 7f1444aa0640 1 -- 172.21.15.38:0/2656082291 <== osd.3 v2:172.21.15.38:6808/1286810294 18 ==== osd_op_reply(103 mds0_sessionmap [create] v126'130 uv130 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f143c068040 con 0x55e989ac6c50 2024-09-21T03:43:13.883 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-09-21T03:43:13.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.881+0000 7f144a7e5f40 1 -- 172.21.15.38:0/2656082291 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:104 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55e989acede0 con 0x55e989ac6c50 2024-09-21T03:43:13.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.882+0000 7f1444aa0640 1 -- 172.21.15.38:0/2656082291 <== osd.3 v2:172.21.15.38:6808/1286810294 19 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v126'131 uv131 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f143c068040 con 0x55e989ac6c50 2024-09-21T03:43:13.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.883+0000 7f144a7e5f40 1 -- 172.21.15.38:0/2656082291 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55e989acede0 con 0x55e989ac6c50 2024-09-21T03:43:13.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.884+0000 7f1444aa0640 1 -- 172.21.15.38:0/2656082291 <== osd.3 v2:172.21.15.38:6808/1286810294 20 ==== osd_op_reply(105 609.00000000 [create] v126'177 uv176 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f143c068040 con 0x55e989ac6c50 2024-09-21T03:43:13.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.884+0000 7f144a7e5f40 1 -- 172.21.15.38:0/2656082291 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e126) -- 0x55e989acede0 con 0x55e989ac6c50 2024-09-21T03:43:13.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.885+0000 7f1444aa0640 1 -- 172.21.15.38:0/2656082291 <== osd.3 v2:172.21.15.38:6808/1286810294 21 ==== osd_op_reply(106 609.00000000 [delete] v126'178 uv178 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f143c068040 con 0x55e989ac6c50 2024-09-21T03:43:13.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.885+0000 7f144a7e5f40 1 -- 172.21.15.38:0/2656082291 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55e989acede0 con 0x55e989ac6c50 2024-09-21T03:43:13.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.886+0000 7f1444aa0640 1 -- 172.21.15.38:0/2656082291 <== osd.3 v2:172.21.15.38:6808/1286810294 22 ==== osd_op_reply(107 609.00000000 [create] v126'179 uv179 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f143c068040 con 0x55e989ac6c50 2024-09-21T03:43:13.888 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-09-21T03:43:13.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.886+0000 7f144a7e5f40 1 -- 172.21.15.38:0/2656082291 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55e989acede0 con 0x55e989ac6c50 2024-09-21T03:43:13.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.887+0000 7f1444aa0640 1 -- 172.21.15.38:0/2656082291 <== osd.3 v2:172.21.15.38:6808/1286810294 23 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v126'180 uv180 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f143c068040 con 0x55e989ac6c50 2024-09-21T03:43:13.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.887+0000 7f144a7e5f40 1 -- 172.21.15.38:0/2656082291 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:109 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55e989acfdf0 con 0x55e989ac6c50 2024-09-21T03:43:13.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.888+0000 7f1444aa0640 1 -- 172.21.15.38:0/2656082291 <== osd.3 v2:172.21.15.38:6808/1286810294 24 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v126'181 uv181 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f143c068040 con 0x55e989ac6c50 2024-09-21T03:43:13.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.888+0000 7f144a7e5f40 1 -- 172.21.15.38:0/2656082291 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:110 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55e989acede0 con 0x55e989ac6c50 2024-09-21T03:43:13.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.889+0000 7f1444aa0640 1 -- 172.21.15.38:0/2656082291 <== osd.3 v2:172.21.15.38:6808/1286810294 25 ==== osd_op_reply(110 609.00000000 [omap-set-header] v126'182 uv182 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f143c068040 con 0x55e989ac6c50 2024-09-21T03:43:13.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.889+0000 7f144a7e5f40 1 --2- 172.21.15.38:0/2656082291 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x55e989acf2f0 0x55e989ad2350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:13.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.889+0000 7f144a7e5f40 1 -- 172.21.15.38:0/2656082291 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55e989ad1710 con 0x55e989acf2f0 2024-09-21T03:43:13.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.889+0000 7f1443a9e640 1 --2- 172.21.15.38:0/2656082291 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x55e989acf2f0 0x55e989ad2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:13.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.890+0000 7f1443a9e640 1 --2- 172.21.15.38:0/2656082291 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x55e989acf2f0 0x55e989ad2350 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:13.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.891+0000 7f1443a9e640 1 -- 172.21.15.38:0/2656082291 <== osd.2 v2:172.21.15.38:6824/2577858902 1 ==== osd_op_reply(111 400.00000000 [create] v126'151 uv150 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f143806a040 con 0x55e989acf2f0 2024-09-21T03:43:13.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.891+0000 7f144a7e5f40 1 -- 172.21.15.38:0/2656082291 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e126) -- 0x55e989ad1710 con 0x55e989acf2f0 2024-09-21T03:43:13.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.892+0000 7f1443a9e640 1 -- 172.21.15.38:0/2656082291 <== osd.2 v2:172.21.15.38:6824/2577858902 2 ==== osd_op_reply(112 400.00000000 [delete] v126'152 uv152 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f143806a040 con 0x55e989acf2f0 2024-09-21T03:43:13.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.892+0000 7f144a7e5f40 1 -- 172.21.15.38:0/2656082291 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55e989ad1710 con 0x55e989acf2f0 2024-09-21T03:43:13.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.893+0000 7f1443a9e640 1 -- 172.21.15.38:0/2656082291 <== osd.2 v2:172.21.15.38:6824/2577858902 3 ==== osd_op_reply(113 400.00000000 [create] v126'153 uv153 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f143806a040 con 0x55e989acf2f0 2024-09-21T03:43:13.895 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-09-21T03:43:13.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.893+0000 7f144a7e5f40 1 -- 172.21.15.38:0/2656082291 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:114 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55e989ad1710 con 0x55e989acf2f0 2024-09-21T03:43:13.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.894+0000 7f1443a9e640 1 -- 172.21.15.38:0/2656082291 <== osd.2 v2:172.21.15.38:6824/2577858902 4 ==== osd_op_reply(114 400.00000000 [write 0~22] v126'154 uv154 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f143806a040 con 0x55e989acf2f0 2024-09-21T03:43:13.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.894+0000 7f144a7e5f40 1 -- 172.21.15.38:0/2656082291 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:115 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55e989ad1710 con 0x55e989acf2f0 2024-09-21T03:43:13.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.895+0000 7f1443a9e640 1 -- 172.21.15.38:0/2656082291 <== osd.2 v2:172.21.15.38:6824/2577858902 5 ==== osd_op_reply(115 400.00000000 [omap-set-header] v126'155 uv155 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f143806a040 con 0x55e989acf2f0 2024-09-21T03:43:13.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.895+0000 7f144a7e5f40 1 -- 172.21.15.38:0/2656082291 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:116 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55e989ad1710 con 0x55e989acbf80 2024-09-21T03:43:13.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.896+0000 7f144429f640 1 -- 172.21.15.38:0/2656082291 <== osd.1 v2:172.21.15.38:6816/1491818732 17 ==== osd_op_reply(116 100.00000000.inode [create] v126'363 uv355 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7f143401b040 con 0x55e989acbf80 2024-09-21T03:43:13.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.896+0000 7f144a7e5f40 1 -- 172.21.15.38:0/2656082291 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:117 13.1f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e126) -- 0x55e989ad1710 con 0x55e989acbf80 2024-09-21T03:43:13.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.898+0000 7f144429f640 1 -- 172.21.15.38:0/2656082291 <== osd.1 v2:172.21.15.38:6816/1491818732 18 ==== osd_op_reply(117 100.00000000.inode [delete] v126'364 uv364 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f143401b040 con 0x55e989acbf80 2024-09-21T03:43:13.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.898+0000 7f144a7e5f40 1 -- 172.21.15.38:0/2656082291 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:118 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55e989ad1710 con 0x55e989acbf80 2024-09-21T03:43:13.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.899+0000 7f144429f640 1 -- 172.21.15.38:0/2656082291 <== osd.1 v2:172.21.15.38:6816/1491818732 19 ==== osd_op_reply(118 100.00000000.inode [create] v126'365 uv365 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f143401b040 con 0x55e989acbf80 2024-09-21T03:43:13.901 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-09-21T03:43:13.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.899+0000 7f144a7e5f40 1 -- 172.21.15.38:0/2656082291 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:119 13.1f 13:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55e989ad1710 con 0x55e989acbf80 2024-09-21T03:43:13.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.900+0000 7f144429f640 1 -- 172.21.15.38:0/2656082291 <== osd.1 v2:172.21.15.38:6816/1491818732 20 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v126'366 uv366 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f143401b040 con 0x55e989acbf80 2024-09-21T03:43:13.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.900+0000 7f144a7e5f40 1 -- 172.21.15.38:0/2656082291 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:120 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55e989ad1710 con 0x55e989acbf80 2024-09-21T03:43:13.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.901+0000 7f144429f640 1 -- 172.21.15.38:0/2656082291 <== osd.1 v2:172.21.15.38:6816/1491818732 21 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v126'367 uv367 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f143401b040 con 0x55e989acbf80 2024-09-21T03:43:13.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.901+0000 7f144a7e5f40 1 -- 172.21.15.38:0/2656082291 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:121 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55e989ad1710 con 0x55e989acbf80 2024-09-21T03:43:13.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.902+0000 7f144429f640 1 -- 172.21.15.38:0/2656082291 <== osd.1 v2:172.21.15.38:6816/1491818732 22 ==== osd_op_reply(121 1.00000000 [create] v126'368 uv362 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7f143401b040 con 0x55e989acbf80 2024-09-21T03:43:13.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.902+0000 7f144a7e5f40 1 -- 172.21.15.38:0/2656082291 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:122 13.1f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e126) -- 0x55e989ad1710 con 0x55e989acbf80 2024-09-21T03:43:13.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.903+0000 7f144429f640 1 -- 172.21.15.38:0/2656082291 <== osd.1 v2:172.21.15.38:6816/1491818732 23 ==== osd_op_reply(122 1.00000000 [delete] v126'369 uv369 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f143401b040 con 0x55e989acbf80 2024-09-21T03:43:13.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.903+0000 7f144a7e5f40 1 -- 172.21.15.38:0/2656082291 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:123 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55e989ad1710 con 0x55e989acbf80 2024-09-21T03:43:13.906 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.904+0000 7f144429f640 1 -- 172.21.15.38:0/2656082291 <== osd.1 v2:172.21.15.38:6816/1491818732 24 ==== osd_op_reply(123 1.00000000 [create] v126'370 uv370 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f143401b040 con 0x55e989acbf80 2024-09-21T03:43:13.906 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-09-21T03:43:13.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.904+0000 7f144a7e5f40 1 -- 172.21.15.38:0/2656082291 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:124 13.1f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55e989ad1710 con 0x55e989acbf80 2024-09-21T03:43:13.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.905+0000 7f144429f640 1 -- 172.21.15.38:0/2656082291 <== osd.1 v2:172.21.15.38:6816/1491818732 25 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v126'371 uv371 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f143401b040 con 0x55e989acbf80 2024-09-21T03:43:13.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.905+0000 7f144a7e5f40 1 -- 172.21.15.38:0/2656082291 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:125 13.1f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55e989ad1710 con 0x55e989acbf80 2024-09-21T03:43:13.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.907+0000 7f144429f640 1 -- 172.21.15.38:0/2656082291 <== osd.1 v2:172.21.15.38:6816/1491818732 26 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v126'372 uv372 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f143401b040 con 0x55e989acbf80 2024-09-21T03:43:13.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.907+0000 7f144a7e5f40 1 -- 172.21.15.38:0/2656082291 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:126 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55e989ad1710 con 0x55e989acbf80 2024-09-21T03:43:13.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.908+0000 7f144429f640 1 -- 172.21.15.38:0/2656082291 <== osd.1 v2:172.21.15.38:6816/1491818732 27 ==== osd_op_reply(126 1.00000000 [omap-set-header] v126'373 uv373 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f143401b040 con 0x55e989acbf80 2024-09-21T03:43:13.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.908+0000 7f144a7e5f40 1 -- 172.21.15.38:0/2656082291 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:127 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=584b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55e989ad2890 con 0x55e989acbf80 2024-09-21T03:43:13.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.909+0000 7f144429f640 1 -- 172.21.15.38:0/2656082291 <== osd.1 v2:172.21.15.38:6816/1491818732 28 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v126'374 uv374 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f143401b040 con 0x55e989acbf80 2024-09-21T03:43:13.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.909+0000 7f144a7e5f40 1 -- 172.21.15.38:0/2656082291 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x55e989ac3860 msgr2=0x55e989ac5c80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:13.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.909+0000 7f144a7e5f40 1 --2- 172.21.15.38:0/2656082291 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x55e989ac3860 0x55e989ac5c80 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:13.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.909+0000 7f144a7e5f40 1 -- 172.21.15.38:0/2656082291 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x55e989acbf80 msgr2=0x55e989ace3a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:13.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.909+0000 7f144a7e5f40 1 --2- 172.21.15.38:0/2656082291 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x55e989acbf80 0x55e989ace3a0 crc :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:13.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.909+0000 7f144a7e5f40 1 -- 172.21.15.38:0/2656082291 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x55e989acf2f0 msgr2=0x55e989ad2350 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:13.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.909+0000 7f144a7e5f40 1 --2- 172.21.15.38:0/2656082291 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x55e989acf2f0 0x55e989ad2350 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:13.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.909+0000 7f144a7e5f40 1 -- 172.21.15.38:0/2656082291 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x55e989ac6c50 msgr2=0x55e989ac9120 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:13.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.909+0000 7f144a7e5f40 1 --2- 172.21.15.38:0/2656082291 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x55e989ac6c50 0x55e989ac9120 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:13.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.909+0000 7f1444aa0640 1 -- 172.21.15.38:0/2656082291 reap_dead start 2024-09-21T03:43:13.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.909+0000 7f144a7e5f40 1 -- 172.21.15.38:0/2656082291 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x55e989abee10 msgr2=0x55e989ac1230 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:13.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.909+0000 7f144a7e5f40 1 --2- 172.21.15.38:0/2656082291 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x55e989abee10 0x55e989ac1230 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:13.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.909+0000 7f1444aa0640 1 -- 172.21.15.38:0/2656082291 reap_dead start 2024-09-21T03:43:13.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.909+0000 7f144a7e5f40 1 -- 172.21.15.38:0/2656082291 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x55e9899884c0 msgr2=0x55e989abd180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:13.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.909+0000 7f144a7e5f40 1 --2- 172.21.15.38:0/2656082291 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x55e9899884c0 0x55e989abd180 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:13.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.909+0000 7f144a7e5f40 1 -- 172.21.15.38:0/2656082291 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f14300775a0 msgr2=0x7f1430079ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:13.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.909+0000 7f144a7e5f40 1 --2- 172.21.15.38:0/2656082291 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f14300775a0 0x7f1430079ab0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f14340104f0 tx=0x7f1434003810 comp rx=0 tx=0).stop 2024-09-21T03:43:13.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.909+0000 7f144a7e5f40 1 -- 172.21.15.38:0/2656082291 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e9899fe950 msgr2=0x55e989987460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:13.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.909+0000 7f144a7e5f40 1 --2- 172.21.15.38:0/2656082291 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e9899fe950 0x55e989987460 secure :-1 s=READY pgs=1969 cs=0 l=1 rev1=1 crypto rx=0x7f143c003130 tx=0x7f143c003220 comp rx=0 tx=0).stop 2024-09-21T03:43:13.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.909+0000 7f144a7e5f40 1 -- 172.21.15.38:0/2656082291 shutdown_connections 2024-09-21T03:43:13.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.909+0000 7f144a7e5f40 1 --2- 172.21.15.38:0/2656082291 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x55e989abee10 0x55e989ac1230 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:13.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.909+0000 7f144a7e5f40 1 --2- 172.21.15.38:0/2656082291 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x55e9899884c0 0x55e989abd180 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:13.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.909+0000 7f144a7e5f40 1 --2- 172.21.15.38:0/2656082291 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e9899fe950 0x55e989987460 unknown :-1 s=CLOSED pgs=1969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:13.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.909+0000 7f144a7e5f40 1 --2- 172.21.15.38:0/2656082291 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f14300775a0 0x7f1430079ab0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:13.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.909+0000 7f144a7e5f40 1 -- 172.21.15.38:0/2656082291 >> 172.21.15.38:0/2656082291 conn(0x55e989862ac0 msgr2=0x55e98993f660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:13.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.910+0000 7f144a7e5f40 1 -- 172.21.15.38:0/2656082291 shutdown_connections 2024-09-21T03:43:13.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.910+0000 7f144a7e5f40 1 -- 172.21.15.38:0/2656082291 wait complete. 2024-09-21T03:43:13.915 DEBUG:teuthology.orchestra.run.smithi038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata truncate 100.00000000.inode 0 2024-09-21T03:43:13.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.969+0000 7f30e4b21f40 1 Processor -- start 2024-09-21T03:43:13.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.969+0000 7f30e4b21f40 1 -- start start 2024-09-21T03:43:13.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.970+0000 7f30e4b21f40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5623fbb01d00 0x5623fbbd7d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:13.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.970+0000 7f30e4b21f40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5623fbb2de30 0x5623fbbb80a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:13.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.970+0000 7f30e4b21f40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5623fbbbc950 0x5623fbafd900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:13.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.970+0000 7f30e4b21f40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5623fba4fd60 con 0x5623fbb01d00 2024-09-21T03:43:13.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.970+0000 7f30e4b21f40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5623fbaae430 con 0x5623fbb2de30 2024-09-21T03:43:13.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.970+0000 7f30e4b21f40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x5623fbb481b0 con 0x5623fbbbc950 2024-09-21T03:43:13.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.970+0000 7f30ddb93640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5623fbb01d00 0x5623fbbd7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:13.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.970+0000 7f30ddb93640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5623fbb01d00 0x5623fbbd7d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35992/0 (socket says 172.21.15.38:35992) 2024-09-21T03:43:13.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.970+0000 7f30ddb93640 1 -- 172.21.15.38:0/1858860708 learned_addr learned my addr 172.21.15.38:0/1858860708 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:43:13.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.970+0000 7f30de394640 1 --2- 172.21.15.38:0/1858860708 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5623fbbbc950 0x5623fbafd900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:13.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.970+0000 7f30dd392640 1 --2- 172.21.15.38:0/1858860708 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5623fbb2de30 0x5623fbbb80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:13.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.971+0000 7f30ddb93640 1 -- 172.21.15.38:0/1858860708 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5623fbbbc950 msgr2=0x5623fbafd900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:13.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.971+0000 7f30ddb93640 1 --2- 172.21.15.38:0/1858860708 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5623fbbbc950 0x5623fbafd900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:13.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.971+0000 7f30ddb93640 1 -- 172.21.15.38:0/1858860708 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5623fbb2de30 msgr2=0x5623fbbb80a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:13.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.971+0000 7f30ddb93640 1 --2- 172.21.15.38:0/1858860708 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5623fbb2de30 0x5623fbbb80a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:13.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.971+0000 7f30ddb93640 1 -- 172.21.15.38:0/1858860708 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623fbb2ea70 con 0x5623fbb01d00 2024-09-21T03:43:13.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.971+0000 7f30ddb93640 1 --2- 172.21.15.38:0/1858860708 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5623fbb01d00 0x5623fbbd7d90 secure :-1 s=READY pgs=1971 cs=0 l=1 rev1=1 crypto rx=0x7f30d000b630 tx=0x7f30d000baf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5820993c27d7a1fc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:13.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.971+0000 7f30cffff640 1 -- 172.21.15.38:0/1858860708 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30d000a940 con 0x5623fbb01d00 2024-09-21T03:43:13.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.971+0000 7f30cffff640 1 -- 172.21.15.38:0/1858860708 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f30d000aae0 con 0x5623fbb01d00 2024-09-21T03:43:13.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.971+0000 7f30cffff640 1 -- 172.21.15.38:0/1858860708 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30d0008450 con 0x5623fbb01d00 2024-09-21T03:43:13.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.971+0000 7f30e4b21f40 1 -- 172.21.15.38:0/1858860708 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5623fbb01d00 msgr2=0x5623fbbd7d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:13.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.971+0000 7f30e4b21f40 1 --2- 172.21.15.38:0/1858860708 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5623fbb01d00 0x5623fbbd7d90 secure :-1 s=READY pgs=1971 cs=0 l=1 rev1=1 crypto rx=0x7f30d000b630 tx=0x7f30d000baf0 comp rx=0 tx=0).stop 2024-09-21T03:43:13.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.971+0000 7f30e4b21f40 1 -- 172.21.15.38:0/1858860708 shutdown_connections 2024-09-21T03:43:13.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.971+0000 7f30e4b21f40 1 --2- 172.21.15.38:0/1858860708 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5623fbbbc950 0x5623fbafd900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:13.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.971+0000 7f30e4b21f40 1 --2- 172.21.15.38:0/1858860708 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5623fbb2de30 0x5623fbbb80a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:13.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.972+0000 7f30e4b21f40 1 --2- 172.21.15.38:0/1858860708 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5623fbb01d00 0x5623fbbd7d90 unknown :-1 s=CLOSED pgs=1971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:13.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.972+0000 7f30e4b21f40 1 -- 172.21.15.38:0/1858860708 >> 172.21.15.38:0/1858860708 conn(0x5623fba20ac0 msgr2=0x5623fbbd3750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:13.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.972+0000 7f30e4b21f40 1 -- 172.21.15.38:0/1858860708 shutdown_connections 2024-09-21T03:43:13.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.972+0000 7f30e4b21f40 1 -- 172.21.15.38:0/1858860708 wait complete. 2024-09-21T03:43:13.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.972+0000 7f30e4b21f40 1 Processor -- start 2024-09-21T03:43:13.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.972+0000 7f30e4b21f40 1 -- start start 2024-09-21T03:43:13.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.974+0000 7f30e4b21f40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5623fbb01d00 0x5623fbbb5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:13.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.974+0000 7f30e4b21f40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5623fbb2de30 0x5623fbb2f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:13.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.974+0000 7f30e4b21f40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5623fbbbc950 0x5623fbb2fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:13.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.974+0000 7f30e4b21f40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5623fbb2ed10 con 0x5623fbbbc950 2024-09-21T03:43:13.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.974+0000 7f30e4b21f40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5623fbbd7350 con 0x5623fbb01d00 2024-09-21T03:43:13.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.974+0000 7f30e4b21f40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x5623fbbd75f0 con 0x5623fbb2de30 2024-09-21T03:43:13.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.974+0000 7f30de394640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5623fbbbc950 0x5623fbb2fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:13.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.974+0000 7f30de394640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5623fbbbc950 0x5623fbb2fb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:36002/0 (socket says 172.21.15.38:36002) 2024-09-21T03:43:13.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.974+0000 7f30de394640 1 -- 172.21.15.38:0/3496048316 learned_addr learned my addr 172.21.15.38:0/3496048316 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:43:13.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.974+0000 7f30dd392640 1 --2- 172.21.15.38:0/3496048316 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5623fbb2de30 0x5623fbb2f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:13.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.974+0000 7f30de394640 1 -- 172.21.15.38:0/3496048316 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5623fbb2de30 msgr2=0x5623fbb2f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:13.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.974+0000 7f30de394640 1 --2- 172.21.15.38:0/3496048316 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5623fbb2de30 0x5623fbb2f5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:13.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.974+0000 7f30de394640 1 -- 172.21.15.38:0/3496048316 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5623fbb01d00 msgr2=0x5623fbbb5400 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T03:43:13.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.974+0000 7f30de394640 1 --2- 172.21.15.38:0/3496048316 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5623fbb01d00 0x5623fbbb5400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:13.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.974+0000 7f30de394640 1 -- 172.21.15.38:0/3496048316 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623fbbb5940 con 0x5623fbbbc950 2024-09-21T03:43:13.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.974+0000 7f30de394640 1 --2- 172.21.15.38:0/3496048316 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5623fbbbc950 0x5623fbb2fb00 secure :-1 s=READY pgs=1972 cs=0 l=1 rev1=1 crypto rx=0x7f30d8025e50 tx=0x7f30d804d810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:13.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.974+0000 7f30ceffd640 1 -- 172.21.15.38:0/3496048316 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30d8064070 con 0x5623fbbbc950 2024-09-21T03:43:13.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.974+0000 7f30ceffd640 1 -- 172.21.15.38:0/3496048316 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f30d8004020 con 0x5623fbbbc950 2024-09-21T03:43:13.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.974+0000 7f30e4b21f40 1 -- 172.21.15.38:0/3496048316 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5623fbbb5cf0 con 0x5623fbbbc950 2024-09-21T03:43:13.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.975+0000 7f30ceffd640 1 -- 172.21.15.38:0/3496048316 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30d8060430 con 0x5623fbbbc950 2024-09-21T03:43:13.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.975+0000 7f30e4b21f40 1 -- 172.21.15.38:0/3496048316 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5623fbbd2f10 con 0x5623fbbbc950 2024-09-21T03:43:13.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.976+0000 7f30ceffd640 1 -- 172.21.15.38:0/3496048316 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f30d805e040 con 0x5623fbbbc950 2024-09-21T03:43:13.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.977+0000 7f30ceffd640 1 --2- 172.21.15.38:0/3496048316 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f30c0077570 0x7f30c0079a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:13.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.977+0000 7f30ceffd640 1 -- 172.21.15.38:0/3496048316 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 8884+0+0 (secure 0 0 0) 0x7f30d80de850 con 0x5623fbbbc950 2024-09-21T03:43:13.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.977+0000 7f30ddb93640 1 --2- 172.21.15.38:0/3496048316 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f30c0077570 0x7f30c0079a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:13.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.977+0000 7f30e4b21f40 1 --2- 172.21.15.38:0/3496048316 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x5623fbbb81c0 0x5623fbc726b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:13.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.977+0000 7f30e4b21f40 1 -- 172.21.15.38:0/3496048316 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:1 13.1f 13:fe07bba1:::100.00000000.inode:head [truncate 0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x5623fbbb60a0 con 0x5623fbbb81c0 2024-09-21T03:43:13.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.977+0000 7f30dd392640 1 --2- 172.21.15.38:0/3496048316 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x5623fbbb81c0 0x5623fbc726b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:13.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.978+0000 7f30ddb93640 1 --2- 172.21.15.38:0/3496048316 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f30c0077570 0x7f30c0079a80 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f30d0005010 tx=0x7f30d0002fb0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:13.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.978+0000 7f30dd392640 1 --2- 172.21.15.38:0/3496048316 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x5623fbbb81c0 0x5623fbc726b0 crc :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:13.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.979+0000 7f30dd392640 1 -- 172.21.15.38:0/3496048316 <== osd.1 v2:172.21.15.38:6816/1491818732 1 ==== osd_op_reply(1 100.00000000.inode [truncate 0] v126'375 uv375 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f30d4010350 con 0x5623fbbb81c0 2024-09-21T03:43:13.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.979+0000 7f30e4b21f40 1 -- 172.21.15.38:0/3496048316 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x5623fbbb81c0 msgr2=0x5623fbc726b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:13.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.979+0000 7f30e4b21f40 1 --2- 172.21.15.38:0/3496048316 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x5623fbbb81c0 0x5623fbc726b0 crc :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:13.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.979+0000 7f30e4b21f40 1 -- 172.21.15.38:0/3496048316 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f30c0077570 msgr2=0x7f30c0079a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:13.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.979+0000 7f30e4b21f40 1 --2- 172.21.15.38:0/3496048316 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f30c0077570 0x7f30c0079a80 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f30d0005010 tx=0x7f30d0002fb0 comp rx=0 tx=0).stop 2024-09-21T03:43:13.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.979+0000 7f30e4b21f40 1 -- 172.21.15.38:0/3496048316 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5623fbbbc950 msgr2=0x5623fbb2fb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:13.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.979+0000 7f30e4b21f40 1 --2- 172.21.15.38:0/3496048316 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5623fbbbc950 0x5623fbb2fb00 secure :-1 s=READY pgs=1972 cs=0 l=1 rev1=1 crypto rx=0x7f30d8025e50 tx=0x7f30d804d810 comp rx=0 tx=0).stop 2024-09-21T03:43:13.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.979+0000 7f30de394640 1 -- 172.21.15.38:0/3496048316 reap_dead start 2024-09-21T03:43:13.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.979+0000 7f30e4b21f40 1 -- 172.21.15.38:0/3496048316 shutdown_connections 2024-09-21T03:43:13.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.979+0000 7f30e4b21f40 1 -- 172.21.15.38:0/3496048316 >> 172.21.15.38:0/3496048316 conn(0x5623fba20ac0 msgr2=0x5623fbb2e240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:13.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.979+0000 7f30e4b21f40 1 -- 172.21.15.38:0/3496048316 shutdown_connections 2024-09-21T03:43:13.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:13.979+0000 7f30e4b21f40 1 -- 172.21.15.38:0/3496048316 wait complete. 2024-09-21T03:43:13.986 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-21T03:43:14.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:14.103+0000 7f464503f640 1 -- 172.21.15.38:0/3773427581 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f46400d0bb0 msgr2=0x7f46400b5be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:14.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:14.103+0000 7f464503f640 1 --2- 172.21.15.38:0/3773427581 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f46400d0bb0 0x7f46400b5be0 secure :-1 s=READY pgs=1974 cs=0 l=1 rev1=1 crypto rx=0x7f4638005fd0 tx=0x7f463802a820 comp rx=0 tx=0).stop 2024-09-21T03:43:14.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:14.103+0000 7f464503f640 1 -- 172.21.15.38:0/3773427581 shutdown_connections 2024-09-21T03:43:14.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:14.103+0000 7f464503f640 1 --2- 172.21.15.38:0/3773427581 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f46400d0bb0 0x7f46400b5be0 unknown :-1 s=CLOSED pgs=1974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:14.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:14.103+0000 7f464503f640 1 --2- 172.21.15.38:0/3773427581 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f464015ca90 0x7f46400d0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:14.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:14.103+0000 7f464503f640 1 -- 172.21.15.38:0/3773427581 >> 172.21.15.38:0/3773427581 conn(0x7f46400b46e0 msgr2=0x7f46400d23e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:14.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:14.103+0000 7f464503f640 1 -- 172.21.15.38:0/3773427581 shutdown_connections 2024-09-21T03:43:14.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:14.103+0000 7f464503f640 1 -- 172.21.15.38:0/3773427581 wait complete. 2024-09-21T03:43:14.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:14.105+0000 7f464503f640 1 Processor -- start 2024-09-21T03:43:14.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:14.105+0000 7f464503f640 1 -- start start 2024-09-21T03:43:14.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:14.107+0000 7f464503f640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f46400ccc40 0x7f46401f2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:14.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:14.107+0000 7f464503f640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f46400d0bb0 0x7f4640148a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:14.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:14.107+0000 7f464503f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f464015ca90 0x7f4640148fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:14.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:14.107+0000 7f464503f640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f46400b65c0 con 0x7f464015ca90 2024-09-21T03:43:14.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:14.107+0000 7f464503f640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f46400b6440 con 0x7f46400d0bb0 2024-09-21T03:43:14.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:14.107+0000 7f464503f640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f46400b6740 con 0x7f46400ccc40 2024-09-21T03:43:14.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:14.107+0000 7f463f59e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f464015ca90 0x7f4640148fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:14.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:14.107+0000 7f463f59e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f464015ca90 0x7f4640148fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:36016/0 (socket says 172.21.15.38:36016) 2024-09-21T03:43:14.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:14.107+0000 7f463f59e640 1 -- 172.21.15.38:0/2359458650 learned_addr learned my addr 172.21.15.38:0/2359458650 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:43:14.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:14.107+0000 7f463e59c640 1 --2- 172.21.15.38:0/2359458650 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f46400d0bb0 0x7f4640148a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:14.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:14.107+0000 7f463ed9d640 1 --2- 172.21.15.38:0/2359458650 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f46400ccc40 0x7f46401f2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:14.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:14.107+0000 7f463f59e640 1 -- 172.21.15.38:0/2359458650 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f46400ccc40 msgr2=0x7f46401f2a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:14.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:14.107+0000 7f463f59e640 1 --2- 172.21.15.38:0/2359458650 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f46400ccc40 0x7f46401f2a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:14.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:14.107+0000 7f463f59e640 1 -- 172.21.15.38:0/2359458650 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f46400d0bb0 msgr2=0x7f4640148a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:14.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:14.107+0000 7f463f59e640 1 --2- 172.21.15.38:0/2359458650 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f46400d0bb0 0x7f4640148a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:14.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:14.107+0000 7f463f59e640 1 -- 172.21.15.38:0/2359458650 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4640146b80 con 0x7f464015ca90 2024-09-21T03:43:14.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:14.107+0000 7f463ed9d640 1 --2- 172.21.15.38:0/2359458650 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f46400ccc40 0x7f46401f2a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:43:14.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:14.108+0000 7f463f59e640 1 --2- 172.21.15.38:0/2359458650 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f464015ca90 0x7f4640148fd0 secure :-1 s=READY pgs=1975 cs=0 l=1 rev1=1 crypto rx=0x7f4638035040 tx=0x7f4638032d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:14.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:14.108+0000 7f46277fe640 1 -- 172.21.15.38:0/2359458650 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4638037040 con 0x7f464015ca90 2024-09-21T03:43:14.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:14.108+0000 7f46277fe640 1 -- 172.21.15.38:0/2359458650 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4638042070 con 0x7f464015ca90 2024-09-21T03:43:14.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:14.108+0000 7f464503f640 1 -- 172.21.15.38:0/2359458650 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4640146e60 con 0x7f464015ca90 2024-09-21T03:43:14.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:14.108+0000 7f46277fe640 1 -- 172.21.15.38:0/2359458650 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4638003e60 con 0x7f464015ca90 2024-09-21T03:43:14.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:14.108+0000 7f464503f640 1 -- 172.21.15.38:0/2359458650 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4640147420 con 0x7f464015ca90 2024-09-21T03:43:14.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:14.109+0000 7f464503f640 1 -- 172.21.15.38:0/2359458650 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46400b6cd0 con 0x7f464015ca90 2024-09-21T03:43:14.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:14.110+0000 7f46277fe640 1 -- 172.21.15.38:0/2359458650 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f4638004280 con 0x7f464015ca90 2024-09-21T03:43:14.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:14.110+0000 7f46277fe640 1 --2- 172.21.15.38:0/2359458650 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f462c078bc0 0x7f462c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:14.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:14.110+0000 7f46277fe640 1 -- 172.21.15.38:0/2359458650 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 8884+0+0 (secure 0 0 0) 0x7f46380bc710 con 0x7f464015ca90 2024-09-21T03:43:14.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:14.110+0000 7f463ed9d640 1 --2- 172.21.15.38:0/2359458650 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f462c078bc0 0x7f462c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:14.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:14.111+0000 7f463ed9d640 1 --2- 172.21.15.38:0/2359458650 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f462c078bc0 0x7f462c07b0d0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f4630000f60 tx=0x7f4630009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:14.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:14.115+0000 7f46277fe640 1 -- 172.21.15.38:0/2359458650 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f4638083e90 con 0x7f464015ca90 2024-09-21T03:43:14.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:14.342+0000 7f464503f640 1 -- 172.21.15.38:0/2359458650 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f46400762a0 con 0x7f464015ca90 2024-09-21T03:43:14.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:14.662+0000 7f46277fe640 1 -- 172.21.15.38:0/2359458650 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v265) ==== 162+0+0 (secure 0 0 0) 0x7f4638083790 con 0x7f464015ca90 2024-09-21T03:43:14.664 INFO:teuthology.orchestra.run.smithi038.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-21T03:43:14.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:14.664+0000 7f464503f640 1 -- 172.21.15.38:0/2359458650 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f462c078bc0 msgr2=0x7f462c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:14.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:14.664+0000 7f464503f640 1 --2- 172.21.15.38:0/2359458650 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f462c078bc0 0x7f462c07b0d0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f4630000f60 tx=0x7f4630009040 comp rx=0 tx=0).stop 2024-09-21T03:43:14.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:14.664+0000 7f464503f640 1 -- 172.21.15.38:0/2359458650 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f464015ca90 msgr2=0x7f4640148fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:14.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:14.664+0000 7f464503f640 1 --2- 172.21.15.38:0/2359458650 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f464015ca90 0x7f4640148fd0 secure :-1 s=READY pgs=1975 cs=0 l=1 rev1=1 crypto rx=0x7f4638035040 tx=0x7f4638032d10 comp rx=0 tx=0).stop 2024-09-21T03:43:14.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:14.664+0000 7f464503f640 1 -- 172.21.15.38:0/2359458650 shutdown_connections 2024-09-21T03:43:14.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:14.664+0000 7f464503f640 1 --2- 172.21.15.38:0/2359458650 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f464015ca90 0x7f4640148fd0 unknown :-1 s=CLOSED pgs=1975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:14.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:14.664+0000 7f464503f640 1 --2- 172.21.15.38:0/2359458650 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f46400d0bb0 0x7f4640148a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:14.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:14.664+0000 7f464503f640 1 --2- 172.21.15.38:0/2359458650 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f462c078bc0 0x7f462c07b0d0 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:14.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:14.664+0000 7f464503f640 1 --2- 172.21.15.38:0/2359458650 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f46400ccc40 0x7f46401f2a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:14.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:14.664+0000 7f464503f640 1 -- 172.21.15.38:0/2359458650 >> 172.21.15.38:0/2359458650 conn(0x7f46400b46e0 msgr2=0x7f46400cf7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:14.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:14.665+0000 7f464503f640 1 -- 172.21.15.38:0/2359458650 shutdown_connections 2024-09-21T03:43:14.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:14.665+0000 7f464503f640 1 -- 172.21.15.38:0/2359458650 wait complete. 2024-09-21T03:43:14.680 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:43:14.685 INFO:tasks.ceph.mds.a.smithi038.stderr:2024-09-21T03:43:14.683+0000 7fae65520640 -1 mds.0.cache.ino(0x100) No data while reading inode 0x100 2024-09-21T03:43:14.685 INFO:tasks.ceph.mds.a.smithi038.stderr:2024-09-21T03:43:14.683+0000 7fae65520640 -1 log_channel(cluster) log [ERR] : Error loading MDS rank 0: (2) No such file or directory 2024-09-21T03:43:14.686 INFO:tasks.ceph.mon.a.smithi038.stderr:2024-09-21T03:43:14.684+0000 7f3f45713640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T03:43:14.686 INFO:tasks.ceph.mon.a.smithi038.stderr:2024-09-21T03:43:14.684+0000 7f3f45713640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-09-21T03:43:14.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:14.797+0000 7f34931c6640 1 -- 172.21.15.38:0/2663321387 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3474004800 msgr2=0x7f3474004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:14.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:14.797+0000 7f34931c6640 1 --2- 172.21.15.38:0/2663321387 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3474004800 0x7f3474004c10 secure :-1 s=READY pgs=1977 cs=0 l=1 rev1=1 crypto rx=0x7f3480009b50 tx=0x7f3474005830 comp rx=0 tx=0).stop 2024-09-21T03:43:14.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:14.797+0000 7f3491763640 1 -- 172.21.15.38:0/2663321387 reap_dead start 2024-09-21T03:43:14.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:14.798+0000 7f34931c6640 1 -- 172.21.15.38:0/2663321387 shutdown_connections 2024-09-21T03:43:14.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:14.798+0000 7f34931c6640 1 -- 172.21.15.38:0/2663321387 >> 172.21.15.38:0/2663321387 conn(0x7f348c0d4440 msgr2=0x7f348c0d4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:14.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:14.798+0000 7f34931c6640 1 -- 172.21.15.38:0/2663321387 shutdown_connections 2024-09-21T03:43:14.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:14.798+0000 7f34931c6640 1 -- 172.21.15.38:0/2663321387 wait complete. 2024-09-21T03:43:14.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:14.799+0000 7f34931c6640 1 Processor -- start 2024-09-21T03:43:14.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:14.799+0000 7f34931c6640 1 -- start start 2024-09-21T03:43:14.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:14.801+0000 7f34931c6640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3474004800 0x7f348c0c7f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:14.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:14.801+0000 7f34931c6640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f348c14c450 0x7f348c14c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:14.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:14.801+0000 7f34931c6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f348c14cda0 0x7f348c1f89f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:14.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:14.801+0000 7f34931c6640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f348c162b20 con 0x7f348c14cda0 2024-09-21T03:43:14.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:14.801+0000 7f34931c6640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f348c1629a0 con 0x7f348c14c450 2024-09-21T03:43:14.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:14.801+0000 7f34931c6640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f348c162ca0 con 0x7f3474004800 2024-09-21T03:43:14.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:14.801+0000 7f3491763640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f348c14cda0 0x7f348c1f89f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:14.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:14.801+0000 7f3491763640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f348c14cda0 0x7f348c1f89f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:36048/0 (socket says 172.21.15.38:36048) 2024-09-21T03:43:14.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:14.801+0000 7f3491763640 1 -- 172.21.15.38:0/1001246674 learned_addr learned my addr 172.21.15.38:0/1001246674 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:43:14.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:14.801+0000 7f348bfff640 1 --2- 172.21.15.38:0/1001246674 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f348c14c450 0x7f348c14c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:14.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:14.801+0000 7f3490f62640 1 --2- 172.21.15.38:0/1001246674 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3474004800 0x7f348c0c7f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:14.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:14.801+0000 7f3491763640 1 -- 172.21.15.38:0/1001246674 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3474004800 msgr2=0x7f348c0c7f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:14.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:14.801+0000 7f3491763640 1 --2- 172.21.15.38:0/1001246674 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3474004800 0x7f348c0c7f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:14.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:14.801+0000 7f3491763640 1 -- 172.21.15.38:0/1001246674 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f348c14c450 msgr2=0x7f348c14c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:14.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:14.801+0000 7f3491763640 1 --2- 172.21.15.38:0/1001246674 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f348c14c450 0x7f348c14c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:14.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:14.801+0000 7f3491763640 1 -- 172.21.15.38:0/1001246674 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f348c0c88c0 con 0x7f348c14cda0 2024-09-21T03:43:14.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:14.801+0000 7f3490f62640 1 --2- 172.21.15.38:0/1001246674 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3474004800 0x7f348c0c7f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:43:14.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:14.802+0000 7f3491763640 1 --2- 172.21.15.38:0/1001246674 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f348c14cda0 0x7f348c1f89f0 secure :-1 s=READY pgs=1978 cs=0 l=1 rev1=1 crypto rx=0x7f348c0cea40 tx=0x7f348400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:14.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:14.802+0000 7f34897fa640 1 -- 172.21.15.38:0/1001246674 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3484003d80 con 0x7f348c14cda0 2024-09-21T03:43:14.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:14.802+0000 7f34897fa640 1 -- 172.21.15.38:0/1001246674 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3484004b20 con 0x7f348c14cda0 2024-09-21T03:43:14.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:14.802+0000 7f34897fa640 1 -- 172.21.15.38:0/1001246674 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3484004030 con 0x7f348c14cda0 2024-09-21T03:43:14.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:14.802+0000 7f34931c6640 1 -- 172.21.15.38:0/1001246674 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f348c0c8ba0 con 0x7f348c14cda0 2024-09-21T03:43:14.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:14.802+0000 7f34931c6640 1 -- 172.21.15.38:0/1001246674 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f348c0c9160 con 0x7f348c14cda0 2024-09-21T03:43:14.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:14.803+0000 7f34931c6640 1 -- 172.21.15.38:0/1001246674 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f348c0b3a70 con 0x7f348c14cda0 2024-09-21T03:43:14.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:14.804+0000 7f34897fa640 1 -- 172.21.15.38:0/1001246674 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f34840041d0 con 0x7f348c14cda0 2024-09-21T03:43:14.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:14.804+0000 7f34897fa640 1 --2- 172.21.15.38:0/1001246674 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3478078b90 0x7f347807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:14.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:14.804+0000 7f34897fa640 1 -- 172.21.15.38:0/1001246674 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 8970+0+0 (secure 0 0 0) 0x7f348409cd00 con 0x7f348c14cda0 2024-09-21T03:43:14.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:14.805+0000 7f3490f62640 1 --2- 172.21.15.38:0/1001246674 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3478078b90 0x7f347807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:14.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:14.805+0000 7f3490f62640 1 --2- 172.21.15.38:0/1001246674 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3478078b90 0x7f347807b0a0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f347c001330 tx=0x7f347c006930 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:14.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:14.808+0000 7f34897fa640 1 -- 172.21.15.38:0/1001246674 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f3484028080 con 0x7f348c14cda0 2024-09-21T03:43:15.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:15.035+0000 7f34931c6640 1 -- 172.21.15.38:0/1001246674 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f348c0c9960 con 0x7f348c14cda0 2024-09-21T03:43:15.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:15.036+0000 7f34897fa640 1 -- 172.21.15.38:0/1001246674 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 267 v267) ==== 95+0+3566 (secure 0 0 0) 0x7f34840644e0 con 0x7f348c14cda0 2024-09-21T03:43:15.038 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:43:15.038 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":267,"btime":"2024-09-21T03:43:14:685216+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6086,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/965972663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":965972663},{"type":"v1","addr":"172.21.15.195:6836","nonce":965972663}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6700,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/3176122334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":3176122334},{"type":"v1","addr":"172.21.15.195:6837","nonce":3176122334}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":267,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:40:02.319282+0000","modified":"2024-09-21T03:43:14.685150+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":127,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-21T03:43:15.038 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 267 2024-09-21T03:43:15.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:15.038+0000 7f34931c6640 1 -- 172.21.15.38:0/1001246674 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3478078b90 msgr2=0x7f347807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:15.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:15.038+0000 7f34931c6640 1 --2- 172.21.15.38:0/1001246674 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3478078b90 0x7f347807b0a0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f347c001330 tx=0x7f347c006930 comp rx=0 tx=0).stop 2024-09-21T03:43:15.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:15.038+0000 7f34931c6640 1 -- 172.21.15.38:0/1001246674 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f348c14cda0 msgr2=0x7f348c1f89f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:15.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:15.038+0000 7f34931c6640 1 --2- 172.21.15.38:0/1001246674 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f348c14cda0 0x7f348c1f89f0 secure :-1 s=READY pgs=1978 cs=0 l=1 rev1=1 crypto rx=0x7f348c0cea40 tx=0x7f348400f790 comp rx=0 tx=0).stop 2024-09-21T03:43:15.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:15.038+0000 7f34931c6640 1 -- 172.21.15.38:0/1001246674 shutdown_connections 2024-09-21T03:43:15.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:15.038+0000 7f34931c6640 1 --2- 172.21.15.38:0/1001246674 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f348c14cda0 0x7f348c1f89f0 unknown :-1 s=CLOSED pgs=1978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:15.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:15.038+0000 7f34931c6640 1 --2- 172.21.15.38:0/1001246674 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f348c14c450 0x7f348c14c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:15.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:15.038+0000 7f34931c6640 1 --2- 172.21.15.38:0/1001246674 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3478078b90 0x7f347807b0a0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:15.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:15.038+0000 7f34931c6640 1 --2- 172.21.15.38:0/1001246674 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3474004800 0x7f348c0c7f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:15.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:15.038+0000 7f34931c6640 1 -- 172.21.15.38:0/1001246674 >> 172.21.15.38:0/1001246674 conn(0x7f348c0d4440 msgr2=0x7f348c0d4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:15.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:15.038+0000 7f34931c6640 1 -- 172.21.15.38:0/1001246674 shutdown_connections 2024-09-21T03:43:15.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:15.038+0000 7f34931c6640 1 -- 172.21.15.38:0/1001246674 wait complete. 2024-09-21T03:43:15.052 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-21T03:43:15.052 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:43:15.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:15.207+0000 7fd0d73f1640 1 -- 172.21.15.38:0/1402428754 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd0d00d01f0 msgr2=0x7fd0d015daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:15.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:15.207+0000 7fd0d73f1640 1 --2- 172.21.15.38:0/1402428754 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd0d00d01f0 0x7fd0d015daf0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fd0c0005fd0 tx=0x7fd0c002aa90 comp rx=0 tx=0).stop 2024-09-21T03:43:15.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:15.207+0000 7fd0d73f1640 1 -- 172.21.15.38:0/1402428754 shutdown_connections 2024-09-21T03:43:15.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:15.207+0000 7fd0d73f1640 1 --2- 172.21.15.38:0/1402428754 >> v2:172.21.15.195:3300/0 conn(0x7fd0d00d0980 0x7fd0d00d0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:15.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:15.207+0000 7fd0d73f1640 1 --2- 172.21.15.38:0/1402428754 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd0d00d01f0 0x7fd0d015daf0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:15.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:15.207+0000 7fd0d73f1640 1 -- 172.21.15.38:0/1402428754 >> 172.21.15.38:0/1402428754 conn(0x7fd0d0072990 msgr2=0x7fd0d0072da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:15.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:15.207+0000 7fd0d73f1640 1 -- 172.21.15.38:0/1402428754 shutdown_connections 2024-09-21T03:43:15.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:15.207+0000 7fd0d73f1640 1 -- 172.21.15.38:0/1402428754 wait complete. 2024-09-21T03:43:15.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:15.209+0000 7fd0d73f1640 1 Processor -- start 2024-09-21T03:43:15.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:15.209+0000 7fd0d73f1640 1 -- start start 2024-09-21T03:43:15.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:15.210+0000 7fd0d73f1640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd0d00d01f0 0x7fd0d01464b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:15.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:15.210+0000 7fd0d73f1640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd0d00d0980 0x7fd0d00cbb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:15.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:15.210+0000 7fd0d73f1640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd0d015a6f0 0x7fd0d015eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:15.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:15.211+0000 7fd0d73f1640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fd0d00cd340 con 0x7fd0d00d0980 2024-09-21T03:43:15.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:15.211+0000 7fd0d73f1640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd0d00cd1c0 con 0x7fd0d00d01f0 2024-09-21T03:43:15.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:15.211+0000 7fd0d73f1640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fd0d00cd4c0 con 0x7fd0d015a6f0 2024-09-21T03:43:15.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:15.211+0000 7fd0d498c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd0d00d0980 0x7fd0d00cbb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:15.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:15.211+0000 7fd0d518d640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd0d00d01f0 0x7fd0d01464b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:15.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:15.211+0000 7fd0d498c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd0d00d0980 0x7fd0d00cbb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:36054/0 (socket says 172.21.15.38:36054) 2024-09-21T03:43:15.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:15.211+0000 7fd0d498c640 1 -- 172.21.15.38:0/1597131666 learned_addr learned my addr 172.21.15.38:0/1597131666 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:43:15.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:15.211+0000 7fd0d598e640 1 --2- 172.21.15.38:0/1597131666 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd0d015a6f0 0x7fd0d015eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:15.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:15.211+0000 7fd0d498c640 1 -- 172.21.15.38:0/1597131666 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd0d015a6f0 msgr2=0x7fd0d015eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:15.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:15.211+0000 7fd0d498c640 1 --2- 172.21.15.38:0/1597131666 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd0d015a6f0 0x7fd0d015eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:15.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:15.211+0000 7fd0d498c640 1 -- 172.21.15.38:0/1597131666 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd0d00d01f0 msgr2=0x7fd0d01464b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:15.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:15.211+0000 7fd0d498c640 1 --2- 172.21.15.38:0/1597131666 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd0d00d01f0 0x7fd0d01464b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:15.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:15.211+0000 7fd0d498c640 1 -- 172.21.15.38:0/1597131666 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0d01f6e30 con 0x7fd0d00d0980 2024-09-21T03:43:15.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:15.211+0000 7fd0d498c640 1 --2- 172.21.15.38:0/1597131666 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd0d00d0980 0x7fd0d00cbb10 secure :-1 s=READY pgs=1979 cs=0 l=1 rev1=1 crypto rx=0x7fd0c400c370 tx=0x7fd0c400c770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:15.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:15.211+0000 7fd0cdffb640 1 -- 172.21.15.38:0/1597131666 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0c4017070 con 0x7fd0d00d0980 2024-09-21T03:43:15.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:15.211+0000 7fd0cdffb640 1 -- 172.21.15.38:0/1597131666 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd0c40053c0 con 0x7fd0d00d0980 2024-09-21T03:43:15.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:15.211+0000 7fd0d73f1640 1 -- 172.21.15.38:0/1597131666 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0d01f7110 con 0x7fd0d00d0980 2024-09-21T03:43:15.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:15.211+0000 7fd0cdffb640 1 -- 172.21.15.38:0/1597131666 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0c4003850 con 0x7fd0d00d0980 2024-09-21T03:43:15.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:15.212+0000 7fd0d73f1640 1 -- 172.21.15.38:0/1597131666 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0d01f76d0 con 0x7fd0d00d0980 2024-09-21T03:43:15.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:15.213+0000 7fd0d73f1640 1 -- 172.21.15.38:0/1597131666 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0d00cd9e0 con 0x7fd0d00d0980 2024-09-21T03:43:15.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:15.214+0000 7fd0cdffb640 1 -- 172.21.15.38:0/1597131666 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fd0c401a070 con 0x7fd0d00d0980 2024-09-21T03:43:15.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:15.214+0000 7fd0cdffb640 1 --2- 172.21.15.38:0/1597131666 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd0bc078b90 0x7fd0bc07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:15.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:15.214+0000 7fd0cdffb640 1 -- 172.21.15.38:0/1597131666 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 8970+0+0 (secure 0 0 0) 0x7fd0c409ac50 con 0x7fd0d00d0980 2024-09-21T03:43:15.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:15.215+0000 7fd0d518d640 1 --2- 172.21.15.38:0/1597131666 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd0bc078b90 0x7fd0bc07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:15.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:15.215+0000 7fd0d518d640 1 --2- 172.21.15.38:0/1597131666 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd0bc078b90 0x7fd0bc07b0a0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fd0c0035000 tx=0x7fd0c0038040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:15.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:15.219+0000 7fd0cdffb640 1 -- 172.21.15.38:0/1597131666 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fd0c40a0020 con 0x7fd0d00d0980 2024-09-21T03:43:15.322 INFO:tasks.ceph.mds.c.smithi038.stderr: -51> 2024-09-21T03:43:10.569+0000 7fde662bd640 -1 log_channel(cluster) log [ERR] : failed to read JournalPointer: -22 ((22) Invalid argument) 2024-09-21T03:43:15.377 INFO:tasks.ceph.mds.c.smithi038.stdout:starting mds.c at 2024-09-21T03:43:15.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:15.445+0000 7fd0d73f1640 1 -- 172.21.15.38:0/1597131666 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd0d0159e10 con 0x7fd0d00d0980 2024-09-21T03:43:15.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:15.445+0000 7fd0cdffb640 1 -- 172.21.15.38:0/1597131666 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 267 v267) ==== 95+0+3566 (secure 0 0 0) 0x7fd0d00d12d0 con 0x7fd0d00d0980 2024-09-21T03:43:15.448 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:43:15.448 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":267,"btime":"2024-09-21T03:43:14:685216+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6086,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/965972663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":965972663},{"type":"v1","addr":"172.21.15.195:6836","nonce":965972663}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6700,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/3176122334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":3176122334},{"type":"v1","addr":"172.21.15.195:6837","nonce":3176122334}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":267,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:40:02.319282+0000","modified":"2024-09-21T03:43:14.685150+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":127,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-21T03:43:15.448 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 267 2024-09-21T03:43:15.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:15.447+0000 7fd0d73f1640 1 -- 172.21.15.38:0/1597131666 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd0bc078b90 msgr2=0x7fd0bc07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:15.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:15.447+0000 7fd0d73f1640 1 --2- 172.21.15.38:0/1597131666 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd0bc078b90 0x7fd0bc07b0a0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fd0c0035000 tx=0x7fd0c0038040 comp rx=0 tx=0).stop 2024-09-21T03:43:15.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:15.447+0000 7fd0d73f1640 1 -- 172.21.15.38:0/1597131666 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd0d00d0980 msgr2=0x7fd0d00cbb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:15.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:15.447+0000 7fd0d73f1640 1 --2- 172.21.15.38:0/1597131666 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd0d00d0980 0x7fd0d00cbb10 secure :-1 s=READY pgs=1979 cs=0 l=1 rev1=1 crypto rx=0x7fd0c400c370 tx=0x7fd0c400c770 comp rx=0 tx=0).stop 2024-09-21T03:43:15.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:15.448+0000 7fd0d73f1640 1 -- 172.21.15.38:0/1597131666 shutdown_connections 2024-09-21T03:43:15.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:15.448+0000 7fd0d73f1640 1 --2- 172.21.15.38:0/1597131666 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd0bc078b90 0x7fd0bc07b0a0 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:15.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:15.448+0000 7fd0d73f1640 1 --2- 172.21.15.38:0/1597131666 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd0d015a6f0 0x7fd0d015eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:15.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:15.448+0000 7fd0d73f1640 1 --2- 172.21.15.38:0/1597131666 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd0d00d0980 0x7fd0d00cbb10 unknown :-1 s=CLOSED pgs=1979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:15.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:15.448+0000 7fd0d73f1640 1 --2- 172.21.15.38:0/1597131666 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd0d00d01f0 0x7fd0d01464b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:15.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:15.448+0000 7fd0d73f1640 1 -- 172.21.15.38:0/1597131666 >> 172.21.15.38:0/1597131666 conn(0x7fd0d0072990 msgr2=0x7fd0d015b8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:15.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:15.448+0000 7fd0d73f1640 1 -- 172.21.15.38:0/1597131666 shutdown_connections 2024-09-21T03:43:15.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:15.448+0000 7fd0d73f1640 1 -- 172.21.15.38:0/1597131666 wait complete. 2024-09-21T03:43:15.461 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-21T03:43:15.461 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:43:15.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:15.575+0000 7f1b63fe4640 1 -- 172.21.15.38:0/1330540368 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1b5c162590 msgr2=0x7f1b5c1629a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:15.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:15.575+0000 7f1b63fe4640 1 --2- 172.21.15.38:0/1330540368 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1b5c162590 0x7f1b5c1629a0 secure :-1 s=READY pgs=1984 cs=0 l=1 rev1=1 crypto rx=0x7f1b5c08d3b0 tx=0x7f1b4c02a530 comp rx=0 tx=0).stop 2024-09-21T03:43:15.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:15.575+0000 7f1b63fe4640 1 -- 172.21.15.38:0/1330540368 shutdown_connections 2024-09-21T03:43:15.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:15.575+0000 7f1b63fe4640 1 --2- 172.21.15.38:0/1330540368 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1b5c162590 0x7f1b5c1629a0 unknown :-1 s=CLOSED pgs=1984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:15.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:15.575+0000 7f1b63fe4640 1 --2- 172.21.15.38:0/1330540368 >> v2:172.21.15.195:3300/0 conn(0x7f1b5c0cfe00 0x7f1b5c0d2a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:15.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:15.575+0000 7f1b63fe4640 1 -- 172.21.15.38:0/1330540368 >> 172.21.15.38:0/1330540368 conn(0x7f1b5c0d4440 msgr2=0x7f1b5c0d4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:15.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:15.575+0000 7f1b63fe4640 1 -- 172.21.15.38:0/1330540368 shutdown_connections 2024-09-21T03:43:15.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:15.575+0000 7f1b63fe4640 1 -- 172.21.15.38:0/1330540368 wait complete. 2024-09-21T03:43:15.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:15.577+0000 7f1b63fe4640 1 Processor -- start 2024-09-21T03:43:15.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:15.577+0000 7f1b63fe4640 1 -- start start 2024-09-21T03:43:15.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:15.578+0000 7f1b63fe4640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1b5c0b2e60 0x7f1b5c153840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:15.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:15.578+0000 7f1b63fe4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1b5c0cfe00 0x7f1b5c15ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:15.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:15.578+0000 7f1b63fe4640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1b5c162590 0x7f1b5c156500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:15.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:15.578+0000 7f1b63fe4640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f1b5c0b3980 con 0x7f1b5c0cfe00 2024-09-21T03:43:15.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:15.578+0000 7f1b63fe4640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f1b5c0cf5a0 con 0x7f1b5c0b2e60 2024-09-21T03:43:15.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:15.578+0000 7f1b63fe4640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f1b5c0b3b00 con 0x7f1b5c162590 2024-09-21T03:43:15.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:15.578+0000 7f1b6157f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1b5c0cfe00 0x7f1b5c15ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:15.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:15.578+0000 7f1b6157f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1b5c0cfe00 0x7f1b5c15ac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:36080/0 (socket says 172.21.15.38:36080) 2024-09-21T03:43:15.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:15.578+0000 7f1b6157f640 1 -- 172.21.15.38:0/406566148 learned_addr learned my addr 172.21.15.38:0/406566148 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:43:15.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:15.579+0000 7f1b61d80640 1 --2- 172.21.15.38:0/406566148 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1b5c0b2e60 0x7f1b5c153840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:15.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:15.579+0000 7f1b62581640 1 --2- 172.21.15.38:0/406566148 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1b5c162590 0x7f1b5c156500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:15.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:15.579+0000 7f1b6157f640 1 -- 172.21.15.38:0/406566148 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1b5c162590 msgr2=0x7f1b5c156500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:15.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:15.579+0000 7f1b6157f640 1 --2- 172.21.15.38:0/406566148 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1b5c162590 0x7f1b5c156500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:15.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:15.579+0000 7f1b6157f640 1 -- 172.21.15.38:0/406566148 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1b5c0b2e60 msgr2=0x7f1b5c153840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:15.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:15.579+0000 7f1b6157f640 1 --2- 172.21.15.38:0/406566148 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1b5c0b2e60 0x7f1b5c153840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:15.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:15.579+0000 7f1b6157f640 1 -- 172.21.15.38:0/406566148 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b5c1f54f0 con 0x7f1b5c0cfe00 2024-09-21T03:43:15.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:15.579+0000 7f1b62581640 1 --2- 172.21.15.38:0/406566148 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1b5c162590 0x7f1b5c156500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:43:15.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:15.579+0000 7f1b6157f640 1 --2- 172.21.15.38:0/406566148 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1b5c0cfe00 0x7f1b5c15ac00 secure :-1 s=READY pgs=1985 cs=0 l=1 rev1=1 crypto rx=0x7f1b4c009920 tx=0x7f1b4c033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:15.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:15.579+0000 7f1b527fc640 1 -- 172.21.15.38:0/406566148 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b4c031d60 con 0x7f1b5c0cfe00 2024-09-21T03:43:15.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:15.579+0000 7f1b527fc640 1 -- 172.21.15.38:0/406566148 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1b4c033e10 con 0x7f1b5c0cfe00 2024-09-21T03:43:15.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:15.579+0000 7f1b63fe4640 1 -- 172.21.15.38:0/406566148 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b5c1f57b0 con 0x7f1b5c0cfe00 2024-09-21T03:43:15.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:15.579+0000 7f1b527fc640 1 -- 172.21.15.38:0/406566148 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b4c005260 con 0x7f1b5c0cfe00 2024-09-21T03:43:15.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:15.579+0000 7f1b63fe4640 1 -- 172.21.15.38:0/406566148 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b5c1f5d70 con 0x7f1b5c0cfe00 2024-09-21T03:43:15.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:15.580+0000 7f1b63fe4640 1 -- 172.21.15.38:0/406566148 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b5c15b6a0 con 0x7f1b5c0cfe00 2024-09-21T03:43:15.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:15.581+0000 7f1b527fc640 1 -- 172.21.15.38:0/406566148 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f1b4c005400 con 0x7f1b5c0cfe00 2024-09-21T03:43:15.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:15.582+0000 7f1b527fc640 1 --2- 172.21.15.38:0/406566148 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1b40078bc0 0x7f1b4007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:15.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:15.582+0000 7f1b527fc640 1 -- 172.21.15.38:0/406566148 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 8970+0+0 (secure 0 0 0) 0x7f1b4c0bce00 con 0x7f1b5c0cfe00 2024-09-21T03:43:15.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:15.582+0000 7f1b61d80640 1 --2- 172.21.15.38:0/406566148 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1b40078bc0 0x7f1b4007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:15.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:15.583+0000 7f1b61d80640 1 --2- 172.21.15.38:0/406566148 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1b40078bc0 0x7f1b4007b0d0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f1b58009b30 tx=0x7f1b58005f90 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:15.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:15.586+0000 7f1b527fc640 1 -- 172.21.15.38:0/406566148 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f1b4c03f030 con 0x7f1b5c0cfe00 2024-09-21T03:43:15.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:15.811+0000 7f1b63fe4640 1 -- 172.21.15.38:0/406566148 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1b5c0d2a20 con 0x7f1b5c0cfe00 2024-09-21T03:43:15.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:15.812+0000 7f1b527fc640 1 -- 172.21.15.38:0/406566148 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 268 v268) ==== 95+0+4392 (secure 0 0 0) 0x7f1b4c083ff0 con 0x7f1b5c0cfe00 2024-09-21T03:43:15.814 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:43:15.814 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":268,"btime":"2024-09-21T03:43:15:686929+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6086,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/965972663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":965972663},{"type":"v1","addr":"172.21.15.195:6836","nonce":965972663}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6700,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/3176122334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":3176122334},{"type":"v1","addr":"172.21.15.195:6837","nonce":3176122334}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":9786,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/2309772457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":2309772457},{"type":"v1","addr":"172.21.15.38:6833","nonce":2309772457}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":268}],"filesystems":[{"mdsmap":{"epoch":267,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:40:02.319282+0000","modified":"2024-09-21T03:43:14.685150+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":127,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-21T03:43:15.814 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 268 2024-09-21T03:43:15.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:15.814+0000 7f1b63fe4640 1 -- 172.21.15.38:0/406566148 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1b40078bc0 msgr2=0x7f1b4007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:15.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:15.814+0000 7f1b63fe4640 1 --2- 172.21.15.38:0/406566148 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1b40078bc0 0x7f1b4007b0d0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f1b58009b30 tx=0x7f1b58005f90 comp rx=0 tx=0).stop 2024-09-21T03:43:15.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:15.814+0000 7f1b63fe4640 1 -- 172.21.15.38:0/406566148 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1b5c0cfe00 msgr2=0x7f1b5c15ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:15.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:15.814+0000 7f1b63fe4640 1 --2- 172.21.15.38:0/406566148 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1b5c0cfe00 0x7f1b5c15ac00 secure :-1 s=READY pgs=1985 cs=0 l=1 rev1=1 crypto rx=0x7f1b4c009920 tx=0x7f1b4c033c20 comp rx=0 tx=0).stop 2024-09-21T03:43:15.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:15.814+0000 7f1b63fe4640 1 -- 172.21.15.38:0/406566148 shutdown_connections 2024-09-21T03:43:15.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:15.814+0000 7f1b63fe4640 1 --2- 172.21.15.38:0/406566148 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1b40078bc0 0x7f1b4007b0d0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:15.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:15.814+0000 7f1b63fe4640 1 --2- 172.21.15.38:0/406566148 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1b5c162590 0x7f1b5c156500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:15.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:15.814+0000 7f1b63fe4640 1 --2- 172.21.15.38:0/406566148 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1b5c0cfe00 0x7f1b5c15ac00 unknown :-1 s=CLOSED pgs=1985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:15.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:15.814+0000 7f1b63fe4640 1 --2- 172.21.15.38:0/406566148 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1b5c0b2e60 0x7f1b5c153840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:15.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:15.814+0000 7f1b63fe4640 1 -- 172.21.15.38:0/406566148 >> 172.21.15.38:0/406566148 conn(0x7f1b5c0d4440 msgr2=0x7f1b5c0b3270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:15.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:15.814+0000 7f1b63fe4640 1 -- 172.21.15.38:0/406566148 shutdown_connections 2024-09-21T03:43:15.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:15.814+0000 7f1b63fe4640 1 -- 172.21.15.38:0/406566148 wait complete. 2024-09-21T03:43:15.827 INFO:tasks.cephfs.test_damage:Result: Mutation 'Truncate 100.00000000.inode' led to DAMAGED state 2024-09-21T03:43:15.827 INFO:tasks.cephfs.test_damage:Applying mutation 'Corrupt omap key 10000000000.00000000:sixmegs_head' 2024-09-21T03:43:15.828 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-21T03:43:15.828 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:43:15.844 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056795,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2354 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T03:43:15.844 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi038.front.sepia.ceph.com 2024-09-21T03:43:15.844 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:43:15.844 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T03:43:15.899 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T03:43:15.899 INFO:teuthology.orchestra.run.smithi038.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-09-21T03:43:15.900 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-21T03:43:16.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:16.054+0000 7f946ad20640 1 -- 172.21.15.38:0/2637783214 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f94640d1120 msgr2=0x7f94640ceb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:16.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:16.054+0000 7f946ad20640 1 --2- 172.21.15.38:0/2637783214 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f94640d1120 0x7f94640ceb90 secure :-1 s=READY pgs=1986 cs=0 l=1 rev1=1 crypto rx=0x7f945c005fd0 tx=0x7f945c02a970 comp rx=0 tx=0).stop 2024-09-21T03:43:16.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:16.054+0000 7f946ad20640 1 -- 172.21.15.38:0/2637783214 shutdown_connections 2024-09-21T03:43:16.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:16.054+0000 7f946ad20640 1 --2- 172.21.15.38:0/2637783214 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f94640d1120 0x7f94640ceb90 unknown :-1 s=CLOSED pgs=1986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:16.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:16.054+0000 7f946ad20640 1 --2- 172.21.15.38:0/2637783214 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f94640d0170 0x7f94640d0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:16.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:16.054+0000 7f946ad20640 1 -- 172.21.15.38:0/2637783214 >> 172.21.15.38:0/2637783214 conn(0x7f946409d830 msgr2=0x7f94640b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:16.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:16.054+0000 7f946ad20640 1 -- 172.21.15.38:0/2637783214 shutdown_connections 2024-09-21T03:43:16.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:16.054+0000 7f946ad20640 1 -- 172.21.15.38:0/2637783214 wait complete. 2024-09-21T03:43:16.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:16.056+0000 7f946ad20640 1 Processor -- start 2024-09-21T03:43:16.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:16.056+0000 7f946ad20640 1 -- start start 2024-09-21T03:43:16.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:16.057+0000 7f946ad20640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f94640d0170 0x7f946409e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:16.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:16.057+0000 7f946ad20640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f94640d1120 0x7f94641ef730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:16.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:16.057+0000 7f946ad20640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f946415ee80 0x7f94641f58e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:16.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:16.057+0000 7f946ad20640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f94640b6700 con 0x7f946415ee80 2024-09-21T03:43:16.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:16.057+0000 7f946ad20640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f94640b6580 con 0x7f94640d1120 2024-09-21T03:43:16.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:16.057+0000 7f946ad20640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f94640b6880 con 0x7f94640d0170 2024-09-21T03:43:16.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:16.057+0000 7f94692bd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f946415ee80 0x7f94641f58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:16.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:16.057+0000 7f94692bd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f946415ee80 0x7f94641f58e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:36090/0 (socket says 172.21.15.38:36090) 2024-09-21T03:43:16.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:16.057+0000 7f94692bd640 1 -- 172.21.15.38:0/949516239 learned_addr learned my addr 172.21.15.38:0/949516239 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:43:16.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:16.058+0000 7f9468abc640 1 --2- 172.21.15.38:0/949516239 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f94640d0170 0x7f946409e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:16.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:16.058+0000 7f9463fff640 1 --2- 172.21.15.38:0/949516239 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f94640d1120 0x7f94641ef730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:16.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:16.058+0000 7f94692bd640 1 -- 172.21.15.38:0/949516239 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f94640d0170 msgr2=0x7f946409e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:16.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:16.058+0000 7f94692bd640 1 --2- 172.21.15.38:0/949516239 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f94640d0170 0x7f946409e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:16.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:16.058+0000 7f94692bd640 1 -- 172.21.15.38:0/949516239 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f94640d1120 msgr2=0x7f94641ef730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:16.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:16.058+0000 7f94692bd640 1 --2- 172.21.15.38:0/949516239 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f94640d1120 0x7f94641ef730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:16.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:16.058+0000 7f94692bd640 1 -- 172.21.15.38:0/949516239 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94641f6380 con 0x7f946415ee80 2024-09-21T03:43:16.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:16.058+0000 7f94692bd640 1 --2- 172.21.15.38:0/949516239 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f946415ee80 0x7f94641f58e0 secure :-1 s=READY pgs=1987 cs=0 l=1 rev1=1 crypto rx=0x7f945c006010 tx=0x7f945c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:16.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:16.058+0000 7f94617fa640 1 -- 172.21.15.38:0/949516239 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f945c034070 con 0x7f946415ee80 2024-09-21T03:43:16.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:16.058+0000 7f94617fa640 1 -- 172.21.15.38:0/949516239 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f945c004640 con 0x7f946415ee80 2024-09-21T03:43:16.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:16.058+0000 7f946ad20640 1 -- 172.21.15.38:0/949516239 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94641f6660 con 0x7f946415ee80 2024-09-21T03:43:16.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:16.058+0000 7f94617fa640 1 -- 172.21.15.38:0/949516239 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f945c0436a0 con 0x7f946415ee80 2024-09-21T03:43:16.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:16.058+0000 7f946ad20640 1 -- 172.21.15.38:0/949516239 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94641efc70 con 0x7f946415ee80 2024-09-21T03:43:16.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:16.059+0000 7f946ad20640 1 -- 172.21.15.38:0/949516239 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94640cc660 con 0x7f946415ee80 2024-09-21T03:43:16.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:16.060+0000 7f94617fa640 1 -- 172.21.15.38:0/949516239 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f945c037020 con 0x7f946415ee80 2024-09-21T03:43:16.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:16.061+0000 7f94617fa640 1 --2- 172.21.15.38:0/949516239 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9450078bc0 0x7f945007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:16.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:16.061+0000 7f94617fa640 1 -- 172.21.15.38:0/949516239 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 8970+0+0 (secure 0 0 0) 0x7f945c0bce20 con 0x7f946415ee80 2024-09-21T03:43:16.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:16.061+0000 7f9468abc640 1 --2- 172.21.15.38:0/949516239 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9450078bc0 0x7f945007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:16.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:16.062+0000 7f9468abc640 1 --2- 172.21.15.38:0/949516239 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9450078bc0 0x7f945007b0d0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f9454006fd0 tx=0x7f9454009060 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:16.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:16.066+0000 7f94617fa640 1 -- 172.21.15.38:0/949516239 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f94641592d0 con 0x7f946415ee80 2024-09-21T03:43:16.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:16.292+0000 7f946ad20640 1 -- 172.21.15.38:0/949516239 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f94640d0a80 con 0x7f946415ee80 2024-09-21T03:43:16.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:16.696+0000 7f94617fa640 1 -- 172.21.15.38:0/949516239 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v269) ==== 161+0+0 (secure 0 0 0) 0x7f945c083f00 con 0x7f946415ee80 2024-09-21T03:43:16.698 INFO:teuthology.orchestra.run.smithi038.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-21T03:43:16.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:16.698+0000 7f946ad20640 1 -- 172.21.15.38:0/949516239 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9450078bc0 msgr2=0x7f945007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:16.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:16.698+0000 7f946ad20640 1 --2- 172.21.15.38:0/949516239 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9450078bc0 0x7f945007b0d0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f9454006fd0 tx=0x7f9454009060 comp rx=0 tx=0).stop 2024-09-21T03:43:16.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:16.698+0000 7f946ad20640 1 -- 172.21.15.38:0/949516239 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f946415ee80 msgr2=0x7f94641f58e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:16.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:16.698+0000 7f946ad20640 1 --2- 172.21.15.38:0/949516239 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f946415ee80 0x7f94641f58e0 secure :-1 s=READY pgs=1987 cs=0 l=1 rev1=1 crypto rx=0x7f945c006010 tx=0x7f945c004040 comp rx=0 tx=0).stop 2024-09-21T03:43:16.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:16.698+0000 7f946ad20640 1 -- 172.21.15.38:0/949516239 shutdown_connections 2024-09-21T03:43:16.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:16.698+0000 7f946ad20640 1 --2- 172.21.15.38:0/949516239 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f946415ee80 0x7f94641f58e0 unknown :-1 s=CLOSED pgs=1987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:16.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:16.699+0000 7f946ad20640 1 --2- 172.21.15.38:0/949516239 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f94640d1120 0x7f94641ef730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:16.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:16.699+0000 7f946ad20640 1 --2- 172.21.15.38:0/949516239 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9450078bc0 0x7f945007b0d0 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:16.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:16.699+0000 7f946ad20640 1 --2- 172.21.15.38:0/949516239 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f94640d0170 0x7f946409e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:16.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:16.699+0000 7f946ad20640 1 -- 172.21.15.38:0/949516239 >> 172.21.15.38:0/949516239 conn(0x7f946409d830 msgr2=0x7f94640b2920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:16.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:16.699+0000 7f946ad20640 1 -- 172.21.15.38:0/949516239 shutdown_connections 2024-09-21T03:43:16.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:16.699+0000 7f946ad20640 1 -- 172.21.15.38:0/949516239 wait complete. 2024-09-21T03:43:16.712 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-09-21T03:43:16.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:16.829+0000 7f655b719640 1 -- 172.21.15.38:0/3434646019 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f65540cd440 msgr2=0x7f65540d0670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:16.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:16.829+0000 7f655b719640 1 --2- 172.21.15.38:0/3434646019 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f65540cd440 0x7f65540d0670 secure :-1 s=READY pgs=1988 cs=0 l=1 rev1=1 crypto rx=0x7f65540d1940 tx=0x7f654402a560 comp rx=0 tx=0).stop 2024-09-21T03:43:16.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:16.829+0000 7f655b719640 1 -- 172.21.15.38:0/3434646019 shutdown_connections 2024-09-21T03:43:16.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:16.829+0000 7f655b719640 1 --2- 172.21.15.38:0/3434646019 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f65540cd440 0x7f65540d0670 unknown :-1 s=CLOSED pgs=1988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:16.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:16.829+0000 7f655b719640 1 --2- 172.21.15.38:0/3434646019 >> v2:172.21.15.195:3300/0 conn(0x7f65540ccc40 0x7f655415ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:16.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:16.829+0000 7f655b719640 1 -- 172.21.15.38:0/3434646019 >> 172.21.15.38:0/3434646019 conn(0x7f65540b46e0 msgr2=0x7f65540d23e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:16.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:16.829+0000 7f655b719640 1 -- 172.21.15.38:0/3434646019 shutdown_connections 2024-09-21T03:43:16.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:16.829+0000 7f655b719640 1 -- 172.21.15.38:0/3434646019 wait complete. 2024-09-21T03:43:16.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:16.831+0000 7f655b719640 1 Processor -- start 2024-09-21T03:43:16.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:16.831+0000 7f655b719640 1 -- start start 2024-09-21T03:43:16.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:16.832+0000 7f655b719640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f65540ccc40 0x7f65541f9140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:16.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:16.832+0000 7f655b719640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f65540cd440 0x7f655414e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:16.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:16.832+0000 7f655b719640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f65540d0bb0 0x7f65541e95c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:16.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:16.832+0000 7f655b719640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f6554070810 con 0x7f65540ccc40 2024-09-21T03:43:16.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:16.832+0000 7f655b719640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f65540b6e00 con 0x7f65540cd440 2024-09-21T03:43:16.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:16.832+0000 7f655b719640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f6554070990 con 0x7f65540d0bb0 2024-09-21T03:43:16.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:16.832+0000 7f65594b5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f65540ccc40 0x7f65541f9140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:16.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:16.833+0000 7f65594b5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f65540ccc40 0x7f65541f9140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:36114/0 (socket says 172.21.15.38:36114) 2024-09-21T03:43:16.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:16.833+0000 7f65594b5640 1 -- 172.21.15.38:0/1171596754 learned_addr learned my addr 172.21.15.38:0/1171596754 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:43:16.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:16.833+0000 7f6558cb4640 1 --2- 172.21.15.38:0/1171596754 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f65540cd440 0x7f655414e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:16.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:16.833+0000 7f65594b5640 1 -- 172.21.15.38:0/1171596754 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f65540d0bb0 msgr2=0x7f65541e95c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:16.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:16.833+0000 7f65594b5640 1 --2- 172.21.15.38:0/1171596754 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f65540d0bb0 0x7f65541e95c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:16.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:16.833+0000 7f65594b5640 1 -- 172.21.15.38:0/1171596754 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f65540cd440 msgr2=0x7f655414e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:16.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:16.833+0000 7f65594b5640 1 --2- 172.21.15.38:0/1171596754 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f65540cd440 0x7f655414e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:16.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:16.833+0000 7f65594b5640 1 -- 172.21.15.38:0/1171596754 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65541f9870 con 0x7f65540ccc40 2024-09-21T03:43:16.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:16.833+0000 7f65594b5640 1 --2- 172.21.15.38:0/1171596754 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f65540ccc40 0x7f65541f9140 secure :-1 s=READY pgs=1989 cs=0 l=1 rev1=1 crypto rx=0x7f655000d740 tx=0x7f655000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:16.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:16.833+0000 7f6549ffb640 1 -- 172.21.15.38:0/1171596754 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6550017070 con 0x7f65540ccc40 2024-09-21T03:43:16.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:16.833+0000 7f6549ffb640 1 -- 172.21.15.38:0/1171596754 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6550009d20 con 0x7f65540ccc40 2024-09-21T03:43:16.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:16.833+0000 7f655b719640 1 -- 172.21.15.38:0/1171596754 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65541f9b50 con 0x7f65540ccc40 2024-09-21T03:43:16.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:16.833+0000 7f6549ffb640 1 -- 172.21.15.38:0/1171596754 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65500106c0 con 0x7f65540ccc40 2024-09-21T03:43:16.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:16.833+0000 7f655b719640 1 -- 172.21.15.38:0/1171596754 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f655415e1b0 con 0x7f65540ccc40 2024-09-21T03:43:16.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:16.835+0000 7f655b719640 1 -- 172.21.15.38:0/1171596754 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65541fa070 con 0x7f65540ccc40 2024-09-21T03:43:16.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:16.835+0000 7f6549ffb640 1 -- 172.21.15.38:0/1171596754 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f655001a070 con 0x7f65540ccc40 2024-09-21T03:43:16.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:16.836+0000 7f6549ffb640 1 --2- 172.21.15.38:0/1171596754 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6538078b90 0x7f653807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:16.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:16.836+0000 7f6549ffb640 1 -- 172.21.15.38:0/1171596754 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 8970+0+0 (secure 0 0 0) 0x7f655009bbc0 con 0x7f65540ccc40 2024-09-21T03:43:16.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:16.836+0000 7f6558cb4640 1 --2- 172.21.15.38:0/1171596754 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6538078b90 0x7f653807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:16.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:16.837+0000 7f6558cb4640 1 --2- 172.21.15.38:0/1171596754 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6538078b90 0x7f653807b0a0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f6544033000 tx=0x7f6544004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:16.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:16.840+0000 7f6549ffb640 1 -- 172.21.15.38:0/1171596754 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f65500632f0 con 0x7f65540ccc40 2024-09-21T03:43:17.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.103+0000 7f655b719640 1 -- 172.21.15.38:0/1171596754 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7f65540716c0 con 0x7f65540ccc40 2024-09-21T03:43:17.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.707+0000 7f6549ffb640 1 -- 172.21.15.38:0/1171596754 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v270) ==== 101+0+0 (secure 0 0 0) 0x7f6550062bf0 con 0x7f65540ccc40 2024-09-21T03:43:17.709 INFO:teuthology.orchestra.run.smithi038.stderr:repaired: restoring rank 6:0 2024-09-21T03:43:17.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.710+0000 7f655b719640 1 -- 172.21.15.38:0/1171596754 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6538078b90 msgr2=0x7f653807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:17.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.710+0000 7f655b719640 1 --2- 172.21.15.38:0/1171596754 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6538078b90 0x7f653807b0a0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f6544033000 tx=0x7f6544004730 comp rx=0 tx=0).stop 2024-09-21T03:43:17.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.710+0000 7f655b719640 1 -- 172.21.15.38:0/1171596754 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f65540ccc40 msgr2=0x7f65541f9140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:17.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.710+0000 7f655b719640 1 --2- 172.21.15.38:0/1171596754 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f65540ccc40 0x7f65541f9140 secure :-1 s=READY pgs=1989 cs=0 l=1 rev1=1 crypto rx=0x7f655000d740 tx=0x7f655000dc00 comp rx=0 tx=0).stop 2024-09-21T03:43:17.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.710+0000 7f655b719640 1 -- 172.21.15.38:0/1171596754 shutdown_connections 2024-09-21T03:43:17.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.710+0000 7f655b719640 1 --2- 172.21.15.38:0/1171596754 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6538078b90 0x7f653807b0a0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:17.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.710+0000 7f655b719640 1 --2- 172.21.15.38:0/1171596754 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f65540d0bb0 0x7f65541e95c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:17.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.710+0000 7f655b719640 1 --2- 172.21.15.38:0/1171596754 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f65540cd440 0x7f655414e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:17.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.710+0000 7f655b719640 1 --2- 172.21.15.38:0/1171596754 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f65540ccc40 0x7f65541f9140 unknown :-1 s=CLOSED pgs=1989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:17.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.710+0000 7f655b719640 1 -- 172.21.15.38:0/1171596754 >> 172.21.15.38:0/1171596754 conn(0x7f65540b46e0 msgr2=0x7f65540cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:17.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.710+0000 7f655b719640 1 -- 172.21.15.38:0/1171596754 shutdown_connections 2024-09-21T03:43:17.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.710+0000 7f655b719640 1 -- 172.21.15.38:0/1171596754 wait complete. 2024-09-21T03:43:17.725 DEBUG:teuthology.orchestra.run.smithi038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-09-21T03:43:17.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.783+0000 7fb8ca03cf40 1 Processor -- start 2024-09-21T03:43:17.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.783+0000 7fb8ca03cf40 1 -- start start 2024-09-21T03:43:17.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.784+0000 7fb8ca03cf40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564829df8d00 0x564829df4900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:17.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.784+0000 7fb8ca03cf40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x564829e24e30 0x564829df4e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:17.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.784+0000 7fb8ca03cf40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564829eb3950 0x564829ecd4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:17.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.784+0000 7fb8ca03cf40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x564829d46d60 con 0x564829eb3950 2024-09-21T03:43:17.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.784+0000 7fb8ca03cf40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x564829da5430 con 0x564829df8d00 2024-09-21T03:43:17.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.784+0000 7fb8ca03cf40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x564829e41460 con 0x564829e24e30 2024-09-21T03:43:17.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.784+0000 7fb8c42f7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564829eb3950 0x564829ecd4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:17.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.784+0000 7fb8c42f7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564829eb3950 0x564829ecd4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:36126/0 (socket says 172.21.15.38:36126) 2024-09-21T03:43:17.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.784+0000 7fb8c42f7640 1 -- 172.21.15.38:0/1216257790 learned_addr learned my addr 172.21.15.38:0/1216257790 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:43:17.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.784+0000 7fb8c3af6640 1 --2- 172.21.15.38:0/1216257790 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564829df8d00 0x564829df4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:17.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.784+0000 7fb8c32f5640 1 --2- 172.21.15.38:0/1216257790 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x564829e24e30 0x564829df4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:17.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.785+0000 7fb8c42f7640 1 -- 172.21.15.38:0/1216257790 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x564829e24e30 msgr2=0x564829df4e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:17.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.785+0000 7fb8c42f7640 1 --2- 172.21.15.38:0/1216257790 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x564829e24e30 0x564829df4e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:17.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.785+0000 7fb8c42f7640 1 -- 172.21.15.38:0/1216257790 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564829df8d00 msgr2=0x564829df4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:17.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.785+0000 7fb8c42f7640 1 --2- 172.21.15.38:0/1216257790 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564829df8d00 0x564829df4900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:17.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.785+0000 7fb8c42f7640 1 -- 172.21.15.38:0/1216257790 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564829ea8270 con 0x564829eb3950 2024-09-21T03:43:17.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.785+0000 7fb8c32f5640 1 --2- 172.21.15.38:0/1216257790 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x564829e24e30 0x564829df4e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:43:17.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.785+0000 7fb8c42f7640 1 --2- 172.21.15.38:0/1216257790 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564829eb3950 0x564829ecd4d0 secure :-1 s=READY pgs=1991 cs=0 l=1 rev1=1 crypto rx=0x7fb8bc00b570 tx=0x7fb8bc00ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fea180de3fa4f324 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:17.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.785+0000 7fb8c22f3640 1 -- 172.21.15.38:0/1216257790 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8bc00a7e0 con 0x564829eb3950 2024-09-21T03:43:17.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.785+0000 7fb8c22f3640 1 -- 172.21.15.38:0/1216257790 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb8bc00a980 con 0x564829eb3950 2024-09-21T03:43:17.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.785+0000 7fb8c22f3640 1 -- 172.21.15.38:0/1216257790 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8bc00ad90 con 0x564829eb3950 2024-09-21T03:43:17.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.786+0000 7fb8ca03cf40 1 -- 172.21.15.38:0/1216257790 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564829eb3950 msgr2=0x564829ecd4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:17.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.786+0000 7fb8ca03cf40 1 --2- 172.21.15.38:0/1216257790 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564829eb3950 0x564829ecd4d0 secure :-1 s=READY pgs=1991 cs=0 l=1 rev1=1 crypto rx=0x7fb8bc00b570 tx=0x7fb8bc00ba30 comp rx=0 tx=0).stop 2024-09-21T03:43:17.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.786+0000 7fb8ca03cf40 1 -- 172.21.15.38:0/1216257790 shutdown_connections 2024-09-21T03:43:17.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.786+0000 7fb8ca03cf40 1 --2- 172.21.15.38:0/1216257790 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564829eb3950 0x564829ecd4d0 unknown :-1 s=CLOSED pgs=1991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:17.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.786+0000 7fb8ca03cf40 1 --2- 172.21.15.38:0/1216257790 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x564829e24e30 0x564829df4e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:17.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.786+0000 7fb8ca03cf40 1 --2- 172.21.15.38:0/1216257790 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564829df8d00 0x564829df4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:17.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.786+0000 7fb8ca03cf40 1 -- 172.21.15.38:0/1216257790 >> 172.21.15.38:0/1216257790 conn(0x564829d17ac0 msgr2=0x564829ece350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:17.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.786+0000 7fb8ca03cf40 1 -- 172.21.15.38:0/1216257790 shutdown_connections 2024-09-21T03:43:17.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.786+0000 7fb8ca03cf40 1 -- 172.21.15.38:0/1216257790 wait complete. 2024-09-21T03:43:17.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.787+0000 7fb8ca03cf40 1 Processor -- start 2024-09-21T03:43:17.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.787+0000 7fb8ca03cf40 1 -- start start 2024-09-21T03:43:17.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.788+0000 7fb8ca03cf40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564829df8d00 0x564829e3ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:17.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.788+0000 7fb8ca03cf40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x564829e24e30 0x564829e3d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:17.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.788+0000 7fb8ca03cf40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564829eb3950 0x564829ea9ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:17.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.788+0000 7fb8ca03cf40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x564829eb2930 con 0x564829df8d00 2024-09-21T03:43:17.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.788+0000 7fb8ca03cf40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x564829eb2170 con 0x564829eb3950 2024-09-21T03:43:17.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.788+0000 7fb8ca03cf40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x564829eaf9b0 con 0x564829e24e30 2024-09-21T03:43:17.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.788+0000 7fb8c3af6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564829df8d00 0x564829e3ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:17.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.788+0000 7fb8c3af6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564829df8d00 0x564829e3ce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:36136/0 (socket says 172.21.15.38:36136) 2024-09-21T03:43:17.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.788+0000 7fb8c3af6640 1 -- 172.21.15.38:0/1662063052 learned_addr learned my addr 172.21.15.38:0/1662063052 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:43:17.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.788+0000 7fb8c3af6640 1 -- 172.21.15.38:0/1662063052 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x564829e24e30 msgr2=0x564829e3d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:17.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.788+0000 7fb8c32f5640 1 --2- 172.21.15.38:0/1662063052 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x564829e24e30 0x564829e3d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:17.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.788+0000 7fb8c42f7640 1 --2- 172.21.15.38:0/1662063052 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564829eb3950 0x564829ea9ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:17.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.788+0000 7fb8c3af6640 1 --2- 172.21.15.38:0/1662063052 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x564829e24e30 0x564829e3d370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:17.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.788+0000 7fb8c3af6640 1 -- 172.21.15.38:0/1662063052 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564829eb3950 msgr2=0x564829ea9ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:17.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.788+0000 7fb8c3af6640 1 --2- 172.21.15.38:0/1662063052 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564829eb3950 0x564829ea9ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:17.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.788+0000 7fb8c3af6640 1 -- 172.21.15.38:0/1662063052 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564829ec9650 con 0x564829df8d00 2024-09-21T03:43:17.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.789+0000 7fb8c3af6640 1 --2- 172.21.15.38:0/1662063052 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564829df8d00 0x564829e3ce30 secure :-1 s=READY pgs=1992 cs=0 l=1 rev1=1 crypto rx=0x7fb8b400d3c0 tx=0x7fb8b400d880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:17.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.789+0000 7fb8c12f1640 1 -- 172.21.15.38:0/1662063052 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8b40125f0 con 0x564829df8d00 2024-09-21T03:43:17.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.789+0000 7fb8c12f1640 1 -- 172.21.15.38:0/1662063052 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb8b4018070 con 0x564829df8d00 2024-09-21T03:43:17.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.789+0000 7fb8c12f1640 1 -- 172.21.15.38:0/1662063052 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8b4012b00 con 0x564829df8d00 2024-09-21T03:43:17.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.789+0000 7fb8ca03cf40 1 -- 172.21.15.38:0/1662063052 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x564829df4be0 con 0x564829df8d00 2024-09-21T03:43:17.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.789+0000 7fb8ca03cf40 1 -- 172.21.15.38:0/1662063052 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564829ece6d0 con 0x564829df8d00 2024-09-21T03:43:17.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.791+0000 7fb8c12f1640 1 -- 172.21.15.38:0/1662063052 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb8b4012ca0 con 0x564829df8d00 2024-09-21T03:43:17.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.791+0000 7fb8c12f1640 1 --2- 172.21.15.38:0/1662063052 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb8b0077570 0x7fb8b0079a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:17.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.791+0000 7fb8c12f1640 1 -- 172.21.15.38:0/1662063052 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 8970+0+0 (secure 0 0 0) 0x7fb8b409a750 con 0x564829df8d00 2024-09-21T03:43:17.795 INFO:teuthology.orchestra.run.smithi038.stdout:Importing pool 2024-09-21T03:43:17.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.791+0000 7fb8c32f5640 1 --2- 172.21.15.38:0/1662063052 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb8b0077570 0x7fb8b0079a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:17.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.792+0000 7fb8ca03cf40 1 --2- 172.21.15.38:0/1662063052 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x564829ec8500 0x564829eaa020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:17.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.792+0000 7fb8ca03cf40 1 -- 172.21.15.38:0/1662063052 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x564829eaa560 con 0x564829ec8500 2024-09-21T03:43:17.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.792+0000 7fb8c42f7640 1 --2- 172.21.15.38:0/1662063052 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x564829ec8500 0x564829eaa020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:17.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.792+0000 7fb8c32f5640 1 --2- 172.21.15.38:0/1662063052 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb8b0077570 0x7fb8b0079a80 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fb8b8055000 tx=0x7fb8b8058040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:17.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.792+0000 7fb8c42f7640 1 --2- 172.21.15.38:0/1662063052 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x564829ec8500 0x564829eaa020 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:17.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.795+0000 7fb8c42f7640 1 -- 172.21.15.38:0/1662063052 <== osd.7 v2:172.21.15.195:6818/1606106342 1 ==== osd_op_reply(1 601.00000000 [create] v127'939 uv913 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb8bc00aa80 con 0x564829ec8500 2024-09-21T03:43:17.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.795+0000 7fb8ca03cf40 1 -- 172.21.15.38:0/1662063052 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e127) -- 0x564829ec8ed0 con 0x564829ec8500 2024-09-21T03:43:17.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.796+0000 7fb8c42f7640 1 -- 172.21.15.38:0/1662063052 <== osd.7 v2:172.21.15.195:6818/1606106342 2 ==== osd_op_reply(2 601.00000000 [delete] v127'940 uv940 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb8bc00aa80 con 0x564829ec8500 2024-09-21T03:43:17.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.796+0000 7fb8ca03cf40 1 -- 172.21.15.38:0/1662063052 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x564829f71f90 con 0x564829ec8500 2024-09-21T03:43:17.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.798+0000 7fb8c42f7640 1 -- 172.21.15.38:0/1662063052 <== osd.7 v2:172.21.15.195:6818/1606106342 3 ==== osd_op_reply(3 601.00000000 [create] v127'941 uv941 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb8bc00aa80 con 0x564829ec8500 2024-09-21T03:43:17.800 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-09-21T03:43:17.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.798+0000 7fb8ca03cf40 1 -- 172.21.15.38:0/1662063052 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x564829f73340 con 0x564829ec8500 2024-09-21T03:43:17.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.800+0000 7fb8c42f7640 1 -- 172.21.15.38:0/1662063052 <== osd.7 v2:172.21.15.195:6818/1606106342 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v127'942 uv942 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb8bc00aa80 con 0x564829ec8500 2024-09-21T03:43:17.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.800+0000 7fb8ca03cf40 1 -- 172.21.15.38:0/1662063052 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x564829f736e0 con 0x564829ec8500 2024-09-21T03:43:17.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.802+0000 7fb8c42f7640 1 -- 172.21.15.38:0/1662063052 <== osd.7 v2:172.21.15.195:6818/1606106342 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v127'943 uv943 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb8bc00aa80 con 0x564829ec8500 2024-09-21T03:43:17.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.802+0000 7fb8ca03cf40 1 -- 172.21.15.38:0/1662063052 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x564829f72330 con 0x564829ec8500 2024-09-21T03:43:17.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.803+0000 7fb8c42f7640 1 -- 172.21.15.38:0/1662063052 <== osd.7 v2:172.21.15.195:6818/1606106342 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v127'944 uv944 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb8bc00aa80 con 0x564829ec8500 2024-09-21T03:43:17.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.804+0000 7fb8ca03cf40 1 -- 172.21.15.38:0/1662063052 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x564829f726d0 con 0x564829ec8500 2024-09-21T03:43:17.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.805+0000 7fb8c42f7640 1 -- 172.21.15.38:0/1662063052 <== osd.7 v2:172.21.15.195:6818/1606106342 7 ==== osd_op_reply(7 602.00000000 [create] v127'945 uv919 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb8bc00aa80 con 0x564829ec8500 2024-09-21T03:43:17.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.805+0000 7fb8ca03cf40 1 -- 172.21.15.38:0/1662063052 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e127) -- 0x564829f726d0 con 0x564829ec8500 2024-09-21T03:43:17.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.807+0000 7fb8c42f7640 1 -- 172.21.15.38:0/1662063052 <== osd.7 v2:172.21.15.195:6818/1606106342 8 ==== osd_op_reply(8 602.00000000 [delete] v127'946 uv946 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb8bc00aa80 con 0x564829ec8500 2024-09-21T03:43:17.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.807+0000 7fb8ca03cf40 1 -- 172.21.15.38:0/1662063052 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x564829f726d0 con 0x564829ec8500 2024-09-21T03:43:17.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.808+0000 7fb8c42f7640 1 -- 172.21.15.38:0/1662063052 <== osd.7 v2:172.21.15.195:6818/1606106342 9 ==== osd_op_reply(9 602.00000000 [create] v127'947 uv947 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb8bc00aa80 con 0x564829ec8500 2024-09-21T03:43:17.811 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-09-21T03:43:17.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.809+0000 7fb8ca03cf40 1 -- 172.21.15.38:0/1662063052 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x564829f726d0 con 0x564829ec8500 2024-09-21T03:43:17.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.810+0000 7fb8c42f7640 1 -- 172.21.15.38:0/1662063052 <== osd.7 v2:172.21.15.195:6818/1606106342 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v127'948 uv948 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb8bc00aa80 con 0x564829ec8500 2024-09-21T03:43:17.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.810+0000 7fb8ca03cf40 1 -- 172.21.15.38:0/1662063052 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x564829f726d0 con 0x564829ec8500 2024-09-21T03:43:17.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.812+0000 7fb8c42f7640 1 -- 172.21.15.38:0/1662063052 <== osd.7 v2:172.21.15.195:6818/1606106342 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v127'949 uv949 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb8bc00aa80 con 0x564829ec8500 2024-09-21T03:43:17.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.812+0000 7fb8ca03cf40 1 -- 172.21.15.38:0/1662063052 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x564829f726d0 con 0x564829ec8500 2024-09-21T03:43:17.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.813+0000 7fb8c42f7640 1 -- 172.21.15.38:0/1662063052 <== osd.7 v2:172.21.15.195:6818/1606106342 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v127'950 uv950 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb8bc00aa80 con 0x564829ec8500 2024-09-21T03:43:17.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.813+0000 7fb8ca03cf40 1 -- 172.21.15.38:0/1662063052 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:13 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x564829f726d0 con 0x564829ec8500 2024-09-21T03:43:17.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.815+0000 7fb8c42f7640 1 -- 172.21.15.38:0/1662063052 <== osd.7 v2:172.21.15.195:6818/1606106342 13 ==== osd_op_reply(13 10000000000.00000000 [create] v127'951 uv926 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7fb8bc00aa80 con 0x564829ec8500 2024-09-21T03:43:17.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.815+0000 7fb8ca03cf40 1 -- 172.21.15.38:0/1662063052 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:14 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e127) -- 0x564829f726d0 con 0x564829ec8500 2024-09-21T03:43:17.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.817+0000 7fb8c42f7640 1 -- 172.21.15.38:0/1662063052 <== osd.7 v2:172.21.15.195:6818/1606106342 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v127'952 uv952 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fb8bc00aa80 con 0x564829ec8500 2024-09-21T03:43:17.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.817+0000 7fb8ca03cf40 1 -- 172.21.15.38:0/1662063052 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:15 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x564829f726d0 con 0x564829ec8500 2024-09-21T03:43:17.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.818+0000 7fb8c42f7640 1 -- 172.21.15.38:0/1662063052 <== osd.7 v2:172.21.15.195:6818/1606106342 15 ==== osd_op_reply(15 10000000000.00000000 [create] v127'953 uv953 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fb8bc00aa80 con 0x564829ec8500 2024-09-21T03:43:17.820 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-09-21T03:43:17.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.818+0000 7fb8ca03cf40 1 -- 172.21.15.38:0/1662063052 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:16 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x564829f726d0 con 0x564829ec8500 2024-09-21T03:43:17.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.820+0000 7fb8c42f7640 1 -- 172.21.15.38:0/1662063052 <== osd.7 v2:172.21.15.195:6818/1606106342 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v127'954 uv954 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fb8bc00aa80 con 0x564829ec8500 2024-09-21T03:43:17.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.820+0000 7fb8ca03cf40 1 -- 172.21.15.38:0/1662063052 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:17 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x564829f726d0 con 0x564829ec8500 2024-09-21T03:43:17.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.822+0000 7fb8c42f7640 1 -- 172.21.15.38:0/1662063052 <== osd.7 v2:172.21.15.195:6818/1606106342 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v127'955 uv955 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fb8bc00aa80 con 0x564829ec8500 2024-09-21T03:43:17.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.822+0000 7fb8ca03cf40 1 -- 172.21.15.38:0/1662063052 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:18 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x564829f726d0 con 0x564829ec8500 2024-09-21T03:43:17.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.825+0000 7fb8c42f7640 1 -- 172.21.15.38:0/1662063052 <== osd.7 v2:172.21.15.195:6818/1606106342 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v127'956 uv956 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fb8bc00aa80 con 0x564829ec8500 2024-09-21T03:43:17.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.825+0000 7fb8ca03cf40 1 -- 172.21.15.38:0/1662063052 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:19 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=585b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x564829f72f80 con 0x564829ec8500 2024-09-21T03:43:17.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.826+0000 7fb8c42f7640 1 -- 172.21.15.38:0/1662063052 <== osd.7 v2:172.21.15.195:6818/1606106342 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v127'957 uv957 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fb8bc00aa80 con 0x564829ec8500 2024-09-21T03:43:17.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.826+0000 7fb8ca03cf40 1 -- 172.21.15.38:0/1662063052 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:20 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x564829f72f60 con 0x564829ec8500 2024-09-21T03:43:17.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.828+0000 7fb8c42f7640 1 -- 172.21.15.38:0/1662063052 <== osd.7 v2:172.21.15.195:6818/1606106342 20 ==== osd_op_reply(20 600.00000000 [create] v127'958 uv932 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb8bc00aa80 con 0x564829ec8500 2024-09-21T03:43:17.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.828+0000 7fb8ca03cf40 1 -- 172.21.15.38:0/1662063052 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:21 13.10 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e127) -- 0x564829f72f60 con 0x564829ec8500 2024-09-21T03:43:17.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.829+0000 7fb8c42f7640 1 -- 172.21.15.38:0/1662063052 <== osd.7 v2:172.21.15.195:6818/1606106342 21 ==== osd_op_reply(21 600.00000000 [delete] v127'959 uv959 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb8bc00aa80 con 0x564829ec8500 2024-09-21T03:43:17.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.829+0000 7fb8ca03cf40 1 -- 172.21.15.38:0/1662063052 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:22 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x564829f72f60 con 0x564829ec8500 2024-09-21T03:43:17.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.831+0000 7fb8c42f7640 1 -- 172.21.15.38:0/1662063052 <== osd.7 v2:172.21.15.195:6818/1606106342 22 ==== osd_op_reply(22 600.00000000 [create] v127'960 uv960 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb8bc00aa80 con 0x564829ec8500 2024-09-21T03:43:17.833 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-09-21T03:43:17.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.831+0000 7fb8ca03cf40 1 -- 172.21.15.38:0/1662063052 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:23 13.10 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x564829f72f60 con 0x564829ec8500 2024-09-21T03:43:17.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.832+0000 7fb8c42f7640 1 -- 172.21.15.38:0/1662063052 <== osd.7 v2:172.21.15.195:6818/1606106342 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v127'961 uv961 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb8bc00aa80 con 0x564829ec8500 2024-09-21T03:43:17.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.833+0000 7fb8ca03cf40 1 -- 172.21.15.38:0/1662063052 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:24 13.10 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x564829f72f60 con 0x564829ec8500 2024-09-21T03:43:17.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.835+0000 7fb8c42f7640 1 -- 172.21.15.38:0/1662063052 <== osd.7 v2:172.21.15.195:6818/1606106342 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v127'962 uv962 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb8bc00aa80 con 0x564829ec8500 2024-09-21T03:43:17.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.835+0000 7fb8ca03cf40 1 -- 172.21.15.38:0/1662063052 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:25 13.10 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x564829f72f60 con 0x564829ec8500 2024-09-21T03:43:17.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.836+0000 7fb8c42f7640 1 -- 172.21.15.38:0/1662063052 <== osd.7 v2:172.21.15.195:6818/1606106342 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v127'963 uv963 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb8bc00aa80 con 0x564829ec8500 2024-09-21T03:43:17.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.836+0000 7fb8ca03cf40 1 -- 172.21.15.38:0/1662063052 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:26 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x564829f72f60 con 0x564829ec8500 2024-09-21T03:43:17.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.837+0000 7fb8c42f7640 1 -- 172.21.15.38:0/1662063052 <== osd.7 v2:172.21.15.195:6818/1606106342 26 ==== osd_op_reply(26 603.00000000 [create] v127'964 uv938 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb8bc00aa80 con 0x564829ec8500 2024-09-21T03:43:17.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.837+0000 7fb8ca03cf40 1 -- 172.21.15.38:0/1662063052 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:27 13.10 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e127) -- 0x564829f72f60 con 0x564829ec8500 2024-09-21T03:43:17.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.839+0000 7fb8c42f7640 1 -- 172.21.15.38:0/1662063052 <== osd.7 v2:172.21.15.195:6818/1606106342 27 ==== osd_op_reply(27 603.00000000 [delete] v127'965 uv965 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb8bc00aa80 con 0x564829ec8500 2024-09-21T03:43:17.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.839+0000 7fb8ca03cf40 1 -- 172.21.15.38:0/1662063052 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:28 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x564829f72f60 con 0x564829ec8500 2024-09-21T03:43:17.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.841+0000 7fb8c42f7640 1 -- 172.21.15.38:0/1662063052 <== osd.7 v2:172.21.15.195:6818/1606106342 28 ==== osd_op_reply(28 603.00000000 [create] v127'966 uv966 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb8bc00aa80 con 0x564829ec8500 2024-09-21T03:43:17.843 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-09-21T03:43:17.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.841+0000 7fb8ca03cf40 1 -- 172.21.15.38:0/1662063052 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:29 13.10 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x564829f72f60 con 0x564829ec8500 2024-09-21T03:43:17.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.843+0000 7fb8c42f7640 1 -- 172.21.15.38:0/1662063052 <== osd.7 v2:172.21.15.195:6818/1606106342 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v127'967 uv967 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb8bc00aa80 con 0x564829ec8500 2024-09-21T03:43:17.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.843+0000 7fb8ca03cf40 1 -- 172.21.15.38:0/1662063052 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:30 13.10 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x564829f72e30 con 0x564829ec8500 2024-09-21T03:43:17.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.844+0000 7fb8c42f7640 1 -- 172.21.15.38:0/1662063052 <== osd.7 v2:172.21.15.195:6818/1606106342 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v127'968 uv968 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb8bc00aa80 con 0x564829ec8500 2024-09-21T03:43:17.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.844+0000 7fb8ca03cf40 1 -- 172.21.15.38:0/1662063052 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:31 13.10 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x564829f72e30 con 0x564829ec8500 2024-09-21T03:43:17.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.846+0000 7fb8c42f7640 1 -- 172.21.15.38:0/1662063052 <== osd.7 v2:172.21.15.195:6818/1606106342 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v127'969 uv969 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb8bc00aa80 con 0x564829ec8500 2024-09-21T03:43:17.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.846+0000 7fb8ca03cf40 1 --2- 172.21.15.38:0/1662063052 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x564829f73f90 0x564829f763b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:17.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.846+0000 7fb8ca03cf40 1 -- 172.21.15.38:0/1662063052 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x564829f72e30 con 0x564829f73f90 2024-09-21T03:43:17.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.846+0000 7fb8c3af6640 1 --2- 172.21.15.38:0/1662063052 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x564829f73f90 0x564829f763b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:17.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.847+0000 7fb8c3af6640 1 --2- 172.21.15.38:0/1662063052 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x564829f73f90 0x564829f763b0 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:17.850 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.849+0000 7fb8c3af6640 1 -- 172.21.15.38:0/1662063052 <== osd.6 v2:172.21.15.195:6826/4055825675 1 ==== osd_op_reply(32 1.00000000.inode [create] v127'679 uv662 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7fb8b405da90 con 0x564829f73f90 2024-09-21T03:43:17.850 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.849+0000 7fb8ca03cf40 1 -- 172.21.15.38:0/1662063052 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e127) -- 0x564829f768f0 con 0x564829f73f90 2024-09-21T03:43:17.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.850+0000 7fb8c3af6640 1 -- 172.21.15.38:0/1662063052 <== osd.6 v2:172.21.15.195:6826/4055825675 2 ==== osd_op_reply(33 1.00000000.inode [delete] v127'680 uv680 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fb8b405da90 con 0x564829f73f90 2024-09-21T03:43:17.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.850+0000 7fb8ca03cf40 1 -- 172.21.15.38:0/1662063052 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x564829f76c90 con 0x564829f73f90 2024-09-21T03:43:17.853 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-09-21T03:43:17.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.851+0000 7fb8c3af6640 1 -- 172.21.15.38:0/1662063052 <== osd.6 v2:172.21.15.195:6826/4055825675 3 ==== osd_op_reply(34 1.00000000.inode [create] v127'681 uv681 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fb8b405da90 con 0x564829f73f90 2024-09-21T03:43:17.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.851+0000 7fb8ca03cf40 1 -- 172.21.15.38:0/1662063052 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x564829f774e0 con 0x564829f73f90 2024-09-21T03:43:17.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.853+0000 7fb8c3af6640 1 -- 172.21.15.38:0/1662063052 <== osd.6 v2:172.21.15.195:6826/4055825675 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v127'682 uv682 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fb8b405da90 con 0x564829f73f90 2024-09-21T03:43:17.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.853+0000 7fb8ca03cf40 1 -- 172.21.15.38:0/1662063052 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x564829f77880 con 0x564829f73f90 2024-09-21T03:43:17.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.854+0000 7fb8c3af6640 1 -- 172.21.15.38:0/1662063052 <== osd.6 v2:172.21.15.195:6826/4055825675 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v127'683 uv683 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fb8b405da90 con 0x564829f73f90 2024-09-21T03:43:17.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.854+0000 7fb8ca03cf40 1 -- 172.21.15.38:0/1662063052 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x564829f77c20 con 0x564829f73f90 2024-09-21T03:43:17.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.856+0000 7fb8c3af6640 1 -- 172.21.15.38:0/1662063052 <== osd.6 v2:172.21.15.195:6826/4055825675 6 ==== osd_op_reply(37 200.00000000 [create] v127'684 uv667 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb8b405da90 con 0x564829f73f90 2024-09-21T03:43:17.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.856+0000 7fb8ca03cf40 1 -- 172.21.15.38:0/1662063052 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e127) -- 0x564829f77fc0 con 0x564829f73f90 2024-09-21T03:43:17.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.857+0000 7fb8c3af6640 1 -- 172.21.15.38:0/1662063052 <== osd.6 v2:172.21.15.195:6826/4055825675 7 ==== osd_op_reply(38 200.00000000 [delete] v127'685 uv685 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb8b405da90 con 0x564829f73f90 2024-09-21T03:43:17.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.857+0000 7fb8ca03cf40 1 -- 172.21.15.38:0/1662063052 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x564829f78360 con 0x564829f73f90 2024-09-21T03:43:17.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.858+0000 7fb8c3af6640 1 -- 172.21.15.38:0/1662063052 <== osd.6 v2:172.21.15.195:6826/4055825675 8 ==== osd_op_reply(39 200.00000000 [create] v127'686 uv686 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb8b405da90 con 0x564829f73f90 2024-09-21T03:43:17.860 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-09-21T03:43:17.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.858+0000 7fb8ca03cf40 1 -- 172.21.15.38:0/1662063052 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x564829f78360 con 0x564829f73f90 2024-09-21T03:43:17.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.859+0000 7fb8c3af6640 1 -- 172.21.15.38:0/1662063052 <== osd.6 v2:172.21.15.195:6826/4055825675 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v127'687 uv687 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb8b405da90 con 0x564829f73f90 2024-09-21T03:43:17.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.860+0000 7fb8ca03cf40 1 -- 172.21.15.38:0/1662063052 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x564829f78360 con 0x564829f73f90 2024-09-21T03:43:17.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.861+0000 7fb8c3af6640 1 -- 172.21.15.38:0/1662063052 <== osd.6 v2:172.21.15.195:6826/4055825675 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v127'688 uv688 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb8b405da90 con 0x564829f73f90 2024-09-21T03:43:17.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.861+0000 7fb8ca03cf40 1 -- 172.21.15.38:0/1662063052 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:42 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x564829f78360 con 0x564829f73f90 2024-09-21T03:43:17.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.862+0000 7fb8c3af6640 1 -- 172.21.15.38:0/1662063052 <== osd.6 v2:172.21.15.195:6826/4055825675 11 ==== osd_op_reply(42 200.00000001 [create] v127'689 uv672 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb8b405da90 con 0x564829f73f90 2024-09-21T03:43:17.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.862+0000 7fb8ca03cf40 1 -- 172.21.15.38:0/1662063052 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:43 13.14 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e127) -- 0x564829f78360 con 0x564829f73f90 2024-09-21T03:43:17.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.864+0000 7fb8c3af6640 1 -- 172.21.15.38:0/1662063052 <== osd.6 v2:172.21.15.195:6826/4055825675 12 ==== osd_op_reply(43 200.00000001 [delete] v127'690 uv690 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb8b405da90 con 0x564829f73f90 2024-09-21T03:43:17.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.864+0000 7fb8ca03cf40 1 -- 172.21.15.38:0/1662063052 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:44 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x564829f78360 con 0x564829f73f90 2024-09-21T03:43:17.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.865+0000 7fb8c3af6640 1 -- 172.21.15.38:0/1662063052 <== osd.6 v2:172.21.15.195:6826/4055825675 13 ==== osd_op_reply(44 200.00000001 [create] v127'691 uv691 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb8b405da90 con 0x564829f73f90 2024-09-21T03:43:17.867 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-09-21T03:43:17.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.865+0000 7fb8ca03cf40 1 -- 172.21.15.38:0/1662063052 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:45 13.14 13:2e2fa760:::200.00000001:head [write 0~17490 in=17490b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x564829f7c7e0 con 0x564829f73f90 2024-09-21T03:43:17.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.866+0000 7fb8c3af6640 1 -- 172.21.15.38:0/1662063052 <== osd.6 v2:172.21.15.195:6826/4055825675 14 ==== osd_op_reply(45 200.00000001 [write 0~17490] v127'692 uv692 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb8b405da90 con 0x564829f73f90 2024-09-21T03:43:17.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.866+0000 7fb8ca03cf40 1 -- 172.21.15.38:0/1662063052 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:46 13.14 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x564829f7c7e0 con 0x564829f73f90 2024-09-21T03:43:17.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.867+0000 7fb8c3af6640 1 -- 172.21.15.38:0/1662063052 <== osd.6 v2:172.21.15.195:6826/4055825675 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v127'693 uv693 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb8b405da90 con 0x564829f73f90 2024-09-21T03:43:17.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.867+0000 7fb8ca03cf40 1 -- 172.21.15.38:0/1662063052 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:47 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x564829f78360 con 0x564829f73f90 2024-09-21T03:43:17.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.869+0000 7fb8c3af6640 1 -- 172.21.15.38:0/1662063052 <== osd.6 v2:172.21.15.195:6826/4055825675 16 ==== osd_op_reply(47 606.00000000 [create] v127'694 uv678 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb8b405da90 con 0x564829f73f90 2024-09-21T03:43:17.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.869+0000 7fb8ca03cf40 1 -- 172.21.15.38:0/1662063052 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:48 13.14 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e127) -- 0x564829f78360 con 0x564829f73f90 2024-09-21T03:43:17.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.870+0000 7fb8c3af6640 1 -- 172.21.15.38:0/1662063052 <== osd.6 v2:172.21.15.195:6826/4055825675 17 ==== osd_op_reply(48 606.00000000 [delete] v127'695 uv695 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb8b405da90 con 0x564829f73f90 2024-09-21T03:43:17.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.870+0000 7fb8ca03cf40 1 -- 172.21.15.38:0/1662063052 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:49 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x564829f78360 con 0x564829f73f90 2024-09-21T03:43:17.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.871+0000 7fb8c3af6640 1 -- 172.21.15.38:0/1662063052 <== osd.6 v2:172.21.15.195:6826/4055825675 18 ==== osd_op_reply(49 606.00000000 [create] v127'696 uv696 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb8b405da90 con 0x564829f73f90 2024-09-21T03:43:17.873 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-09-21T03:43:17.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.871+0000 7fb8ca03cf40 1 -- 172.21.15.38:0/1662063052 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:50 13.14 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x564829f79370 con 0x564829f73f90 2024-09-21T03:43:17.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.873+0000 7fb8c3af6640 1 -- 172.21.15.38:0/1662063052 <== osd.6 v2:172.21.15.195:6826/4055825675 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v127'697 uv697 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb8b405da90 con 0x564829f73f90 2024-09-21T03:43:17.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.873+0000 7fb8ca03cf40 1 -- 172.21.15.38:0/1662063052 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:51 13.14 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x564829f79370 con 0x564829f73f90 2024-09-21T03:43:17.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.874+0000 7fb8c3af6640 1 -- 172.21.15.38:0/1662063052 <== osd.6 v2:172.21.15.195:6826/4055825675 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v127'698 uv698 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb8b405da90 con 0x564829f73f90 2024-09-21T03:43:17.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.874+0000 7fb8ca03cf40 1 -- 172.21.15.38:0/1662063052 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:52 13.14 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x564829f78360 con 0x564829f73f90 2024-09-21T03:43:17.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.875+0000 7fb8c3af6640 1 -- 172.21.15.38:0/1662063052 <== osd.6 v2:172.21.15.195:6826/4055825675 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v127'699 uv699 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb8b405da90 con 0x564829f73f90 2024-09-21T03:43:17.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.875+0000 7fb8ca03cf40 1 --2- 172.21.15.38:0/1662063052 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x564829f789e0 0x564829f7ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:17.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.875+0000 7fb8ca03cf40 1 -- 172.21.15.38:0/1662063052 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:53 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x564829f7b380 con 0x564829f789e0 2024-09-21T03:43:17.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.875+0000 7fb8c32f5640 1 --2- 172.21.15.38:0/1662063052 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x564829f789e0 0x564829f7ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:17.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.876+0000 7fb8c32f5640 1 --2- 172.21.15.38:0/1662063052 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x564829f789e0 0x564829f7ae00 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:17.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.877+0000 7fb8c32f5640 1 -- 172.21.15.38:0/1662063052 <== osd.0 v2:172.21.15.38:6800/2381639433 1 ==== osd_op_reply(53 607.00000000 [create] v127'183 uv182 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb8b8065040 con 0x564829f789e0 2024-09-21T03:43:17.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.877+0000 7fb8ca03cf40 1 -- 172.21.15.38:0/1662063052 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:54 13.1c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e127) -- 0x564829f7b800 con 0x564829f789e0 2024-09-21T03:43:17.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.878+0000 7fb8c32f5640 1 -- 172.21.15.38:0/1662063052 <== osd.0 v2:172.21.15.38:6800/2381639433 2 ==== osd_op_reply(54 607.00000000 [delete] v127'184 uv184 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb8b8065040 con 0x564829f789e0 2024-09-21T03:43:17.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.878+0000 7fb8ca03cf40 1 -- 172.21.15.38:0/1662063052 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:55 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x564829f7bce0 con 0x564829f789e0 2024-09-21T03:43:17.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.879+0000 7fb8c32f5640 1 -- 172.21.15.38:0/1662063052 <== osd.0 v2:172.21.15.38:6800/2381639433 3 ==== osd_op_reply(55 607.00000000 [create] v127'185 uv185 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb8b8065040 con 0x564829f789e0 2024-09-21T03:43:17.881 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-09-21T03:43:17.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.879+0000 7fb8ca03cf40 1 -- 172.21.15.38:0/1662063052 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:56 13.1c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x564829f7d590 con 0x564829f789e0 2024-09-21T03:43:17.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.880+0000 7fb8c32f5640 1 -- 172.21.15.38:0/1662063052 <== osd.0 v2:172.21.15.38:6800/2381639433 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v127'186 uv186 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb8b8065040 con 0x564829f789e0 2024-09-21T03:43:17.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.880+0000 7fb8ca03cf40 1 -- 172.21.15.38:0/1662063052 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:57 13.1c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x564829f7d930 con 0x564829f789e0 2024-09-21T03:43:17.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.881+0000 7fb8c32f5640 1 -- 172.21.15.38:0/1662063052 <== osd.0 v2:172.21.15.38:6800/2381639433 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v127'187 uv187 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb8b8065040 con 0x564829f789e0 2024-09-21T03:43:17.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.881+0000 7fb8ca03cf40 1 -- 172.21.15.38:0/1662063052 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:58 13.1c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x564829f7c580 con 0x564829f789e0 2024-09-21T03:43:17.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.882+0000 7fb8c32f5640 1 -- 172.21.15.38:0/1662063052 <== osd.0 v2:172.21.15.38:6800/2381639433 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v127'188 uv188 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb8b8065040 con 0x564829f789e0 2024-09-21T03:43:17.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.882+0000 7fb8ca03cf40 1 --2- 172.21.15.38:0/1662063052 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x564829f7ce80 0x564829f7fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:17.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.882+0000 7fb8ca03cf40 1 -- 172.21.15.38:0/1662063052 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:59 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x564829f80220 con 0x564829f7ce80 2024-09-21T03:43:17.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.883+0000 7fb8c42f7640 1 --2- 172.21.15.38:0/1662063052 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x564829f7ce80 0x564829f7fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:17.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.883+0000 7fb8c42f7640 1 --2- 172.21.15.38:0/1662063052 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x564829f7ce80 0x564829f7fce0 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:17.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.884+0000 7fb8c42f7640 1 -- 172.21.15.38:0/1662063052 <== osd.3 v2:172.21.15.38:6808/1286810294 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v127'132 uv131 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7fb8bc00aa80 con 0x564829f7ce80 2024-09-21T03:43:17.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.884+0000 7fb8ca03cf40 1 -- 172.21.15.38:0/1662063052 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:60 13.6 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e127) -- 0x564829f80220 con 0x564829f7ce80 2024-09-21T03:43:17.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.885+0000 7fb8c42f7640 1 -- 172.21.15.38:0/1662063052 <== osd.3 v2:172.21.15.38:6808/1286810294 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v127'133 uv133 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fb8bc00aa80 con 0x564829f7ce80 2024-09-21T03:43:17.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.885+0000 7fb8ca03cf40 1 -- 172.21.15.38:0/1662063052 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:61 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x564829f80610 con 0x564829f7ce80 2024-09-21T03:43:17.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.887+0000 7fb8c42f7640 1 -- 172.21.15.38:0/1662063052 <== osd.3 v2:172.21.15.38:6808/1286810294 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v127'134 uv134 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fb8bc00aa80 con 0x564829f7ce80 2024-09-21T03:43:17.888 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-09-21T03:43:17.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.887+0000 7fb8ca03cf40 1 -- 172.21.15.38:0/1662063052 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:62 13.6 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x564829f80610 con 0x564829f7ce80 2024-09-21T03:43:17.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.888+0000 7fb8c42f7640 1 -- 172.21.15.38:0/1662063052 <== osd.3 v2:172.21.15.38:6808/1286810294 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v127'135 uv135 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fb8bc00aa80 con 0x564829f7ce80 2024-09-21T03:43:17.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.888+0000 7fb8ca03cf40 1 -- 172.21.15.38:0/1662063052 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x564829f80610 con 0x564829f789e0 2024-09-21T03:43:17.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.889+0000 7fb8c32f5640 1 -- 172.21.15.38:0/1662063052 <== osd.0 v2:172.21.15.38:6800/2381639433 7 ==== osd_op_reply(63 608.00000000 [create] v127'183 uv182 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb8b8065040 con 0x564829f789e0 2024-09-21T03:43:17.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.889+0000 7fb8ca03cf40 1 -- 172.21.15.38:0/1662063052 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e127) -- 0x564829f80610 con 0x564829f789e0 2024-09-21T03:43:17.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.891+0000 7fb8c32f5640 1 -- 172.21.15.38:0/1662063052 <== osd.0 v2:172.21.15.38:6800/2381639433 8 ==== osd_op_reply(64 608.00000000 [delete] v127'184 uv184 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb8b8065040 con 0x564829f789e0 2024-09-21T03:43:17.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.891+0000 7fb8ca03cf40 1 -- 172.21.15.38:0/1662063052 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x564829f80610 con 0x564829f789e0 2024-09-21T03:43:17.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.892+0000 7fb8c32f5640 1 -- 172.21.15.38:0/1662063052 <== osd.0 v2:172.21.15.38:6800/2381639433 9 ==== osd_op_reply(65 608.00000000 [create] v127'185 uv185 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb8b8065040 con 0x564829f789e0 2024-09-21T03:43:17.894 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-09-21T03:43:17.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.892+0000 7fb8ca03cf40 1 -- 172.21.15.38:0/1662063052 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x564829f81620 con 0x564829f789e0 2024-09-21T03:43:17.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.893+0000 7fb8c32f5640 1 -- 172.21.15.38:0/1662063052 <== osd.0 v2:172.21.15.38:6800/2381639433 10 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v127'186 uv186 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb8b8065040 con 0x564829f789e0 2024-09-21T03:43:17.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.893+0000 7fb8ca03cf40 1 -- 172.21.15.38:0/1662063052 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x564829f81620 con 0x564829f789e0 2024-09-21T03:43:17.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.894+0000 7fb8c32f5640 1 -- 172.21.15.38:0/1662063052 <== osd.0 v2:172.21.15.38:6800/2381639433 11 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v127'187 uv187 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb8b8065040 con 0x564829f789e0 2024-09-21T03:43:17.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.894+0000 7fb8ca03cf40 1 -- 172.21.15.38:0/1662063052 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x564829f80610 con 0x564829f789e0 2024-09-21T03:43:17.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.896+0000 7fb8c32f5640 1 -- 172.21.15.38:0/1662063052 <== osd.0 v2:172.21.15.38:6800/2381639433 12 ==== osd_op_reply(68 608.00000000 [omap-set-header] v127'188 uv188 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb8b8065040 con 0x564829f789e0 2024-09-21T03:43:17.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.896+0000 7fb8ca03cf40 1 -- 172.21.15.38:0/1662063052 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x564829f80610 con 0x564829f7ce80 2024-09-21T03:43:17.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.897+0000 7fb8c42f7640 1 -- 172.21.15.38:0/1662063052 <== osd.3 v2:172.21.15.38:6808/1286810294 5 ==== osd_op_reply(69 604.00000000 [create] v127'338 uv332 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb8bc00aa80 con 0x564829f7ce80 2024-09-21T03:43:17.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.897+0000 7fb8ca03cf40 1 -- 172.21.15.38:0/1662063052 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e127) -- 0x564829f80610 con 0x564829f7ce80 2024-09-21T03:43:17.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.898+0000 7fb8c42f7640 1 -- 172.21.15.38:0/1662063052 <== osd.3 v2:172.21.15.38:6808/1286810294 6 ==== osd_op_reply(70 604.00000000 [delete] v127'339 uv339 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb8bc00aa80 con 0x564829f7ce80 2024-09-21T03:43:17.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.898+0000 7fb8ca03cf40 1 -- 172.21.15.38:0/1662063052 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x564829f80610 con 0x564829f7ce80 2024-09-21T03:43:17.901 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-09-21T03:43:17.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.900+0000 7fb8c42f7640 1 -- 172.21.15.38:0/1662063052 <== osd.3 v2:172.21.15.38:6808/1286810294 7 ==== osd_op_reply(71 604.00000000 [create] v127'340 uv340 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb8bc00aa80 con 0x564829f7ce80 2024-09-21T03:43:17.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.900+0000 7fb8ca03cf40 1 -- 172.21.15.38:0/1662063052 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x564829f81620 con 0x564829f7ce80 2024-09-21T03:43:17.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.901+0000 7fb8c42f7640 1 -- 172.21.15.38:0/1662063052 <== osd.3 v2:172.21.15.38:6808/1286810294 8 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v127'341 uv341 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb8bc00aa80 con 0x564829f7ce80 2024-09-21T03:43:17.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.901+0000 7fb8ca03cf40 1 -- 172.21.15.38:0/1662063052 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x564829f81620 con 0x564829f7ce80 2024-09-21T03:43:17.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.902+0000 7fb8c42f7640 1 -- 172.21.15.38:0/1662063052 <== osd.3 v2:172.21.15.38:6808/1286810294 9 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v127'342 uv342 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb8bc00aa80 con 0x564829f7ce80 2024-09-21T03:43:17.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.902+0000 7fb8ca03cf40 1 -- 172.21.15.38:0/1662063052 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x564829f80610 con 0x564829f7ce80 2024-09-21T03:43:17.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.903+0000 7fb8c42f7640 1 -- 172.21.15.38:0/1662063052 <== osd.3 v2:172.21.15.38:6808/1286810294 10 ==== osd_op_reply(74 604.00000000 [omap-set-header] v127'343 uv343 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb8bc00aa80 con 0x564829f7ce80 2024-09-21T03:43:17.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.903+0000 7fb8ca03cf40 1 -- 172.21.15.38:0/1662063052 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x564829f80610 con 0x564829f7ce80 2024-09-21T03:43:17.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.904+0000 7fb8c42f7640 1 -- 172.21.15.38:0/1662063052 <== osd.3 v2:172.21.15.38:6808/1286810294 11 ==== osd_op_reply(75 500.00000000 [create] v127'344 uv337 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb8bc00aa80 con 0x564829f7ce80 2024-09-21T03:43:17.906 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.904+0000 7fb8ca03cf40 1 -- 172.21.15.38:0/1662063052 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e127) -- 0x564829f80610 con 0x564829f7ce80 2024-09-21T03:43:17.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.905+0000 7fb8c42f7640 1 -- 172.21.15.38:0/1662063052 <== osd.3 v2:172.21.15.38:6808/1286810294 12 ==== osd_op_reply(76 500.00000000 [delete] v127'345 uv345 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb8bc00aa80 con 0x564829f7ce80 2024-09-21T03:43:17.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.905+0000 7fb8ca03cf40 1 -- 172.21.15.38:0/1662063052 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x564829f80610 con 0x564829f7ce80 2024-09-21T03:43:17.908 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-09-21T03:43:17.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.906+0000 7fb8c42f7640 1 -- 172.21.15.38:0/1662063052 <== osd.3 v2:172.21.15.38:6808/1286810294 13 ==== osd_op_reply(77 500.00000000 [create] v127'346 uv346 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb8bc00aa80 con 0x564829f7ce80 2024-09-21T03:43:17.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.906+0000 7fb8ca03cf40 1 -- 172.21.15.38:0/1662063052 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x564829f80610 con 0x564829f7ce80 2024-09-21T03:43:17.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.907+0000 7fb8c42f7640 1 -- 172.21.15.38:0/1662063052 <== osd.3 v2:172.21.15.38:6808/1286810294 14 ==== osd_op_reply(78 500.00000000 [write 0~90] v127'347 uv347 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb8bc00aa80 con 0x564829f7ce80 2024-09-21T03:43:17.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.907+0000 7fb8ca03cf40 1 -- 172.21.15.38:0/1662063052 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x564829f80610 con 0x564829f7ce80 2024-09-21T03:43:17.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.908+0000 7fb8c42f7640 1 -- 172.21.15.38:0/1662063052 <== osd.3 v2:172.21.15.38:6808/1286810294 15 ==== osd_op_reply(79 500.00000000 [omap-set-header] v127'348 uv348 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb8bc00aa80 con 0x564829f7ce80 2024-09-21T03:43:17.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.908+0000 7fb8ca03cf40 1 -- 172.21.15.38:0/1662063052 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:80 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x564829f80610 con 0x564829f789e0 2024-09-21T03:43:17.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.909+0000 7fb8c32f5640 1 -- 172.21.15.38:0/1662063052 <== osd.0 v2:172.21.15.38:6800/2381639433 13 ==== osd_op_reply(80 mds_snaptable [create] v127'153 uv152 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7fb8b8065040 con 0x564829f789e0 2024-09-21T03:43:17.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.909+0000 7fb8ca03cf40 1 -- 172.21.15.38:0/1662063052 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:81 13.d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e127) -- 0x564829f80610 con 0x564829f789e0 2024-09-21T03:43:17.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.911+0000 7fb8c32f5640 1 -- 172.21.15.38:0/1662063052 <== osd.0 v2:172.21.15.38:6800/2381639433 14 ==== osd_op_reply(81 mds_snaptable [delete] v127'154 uv154 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fb8b8065040 con 0x564829f789e0 2024-09-21T03:43:17.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.911+0000 7fb8ca03cf40 1 -- 172.21.15.38:0/1662063052 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:82 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x564829f80610 con 0x564829f789e0 2024-09-21T03:43:17.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.912+0000 7fb8c32f5640 1 -- 172.21.15.38:0/1662063052 <== osd.0 v2:172.21.15.38:6800/2381639433 15 ==== osd_op_reply(82 mds_snaptable [create] v127'155 uv155 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fb8b8065040 con 0x564829f789e0 2024-09-21T03:43:17.914 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-09-21T03:43:17.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.912+0000 7fb8ca03cf40 1 -- 172.21.15.38:0/1662063052 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:83 13.d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x564829f80610 con 0x564829f789e0 2024-09-21T03:43:17.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.913+0000 7fb8c32f5640 1 -- 172.21.15.38:0/1662063052 <== osd.0 v2:172.21.15.38:6800/2381639433 16 ==== osd_op_reply(83 mds_snaptable [write 0~70] v127'156 uv156 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fb8b8065040 con 0x564829f789e0 2024-09-21T03:43:17.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.913+0000 7fb8ca03cf40 1 -- 172.21.15.38:0/1662063052 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:84 13.d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x564829f80610 con 0x564829f789e0 2024-09-21T03:43:17.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.914+0000 7fb8c32f5640 1 -- 172.21.15.38:0/1662063052 <== osd.0 v2:172.21.15.38:6800/2381639433 17 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v127'157 uv157 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fb8b8065040 con 0x564829f789e0 2024-09-21T03:43:17.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.914+0000 7fb8ca03cf40 1 --2- 172.21.15.38:0/1662063052 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x564829f80f10 0x564829f83330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:17.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.914+0000 7fb8ca03cf40 1 -- 172.21.15.38:0/1662063052 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x564829f83870 con 0x564829f80f10 2024-09-21T03:43:17.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.914+0000 7fb8c3af6640 1 --2- 172.21.15.38:0/1662063052 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x564829f80f10 0x564829f83330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:17.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.915+0000 7fb8c3af6640 1 --2- 172.21.15.38:0/1662063052 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x564829f80f10 0x564829f83330 crc :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:17.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.916+0000 7fb8c3af6640 1 -- 172.21.15.38:0/1662063052 <== osd.1 v2:172.21.15.38:6816/1491818732 1 ==== osd_op_reply(85 605.00000000 [create] v127'183 uv182 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb8b405da90 con 0x564829f80f10 2024-09-21T03:43:17.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.916+0000 7fb8ca03cf40 1 -- 172.21.15.38:0/1662063052 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e127) -- 0x564829f83870 con 0x564829f80f10 2024-09-21T03:43:17.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.918+0000 7fb8c3af6640 1 -- 172.21.15.38:0/1662063052 <== osd.1 v2:172.21.15.38:6816/1491818732 2 ==== osd_op_reply(86 605.00000000 [delete] v127'184 uv184 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb8b405da90 con 0x564829f80f10 2024-09-21T03:43:17.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.918+0000 7fb8ca03cf40 1 -- 172.21.15.38:0/1662063052 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x564829f83c60 con 0x564829f80f10 2024-09-21T03:43:17.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.919+0000 7fb8c3af6640 1 -- 172.21.15.38:0/1662063052 <== osd.1 v2:172.21.15.38:6816/1491818732 3 ==== osd_op_reply(87 605.00000000 [create] v127'185 uv185 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb8b405da90 con 0x564829f80f10 2024-09-21T03:43:17.921 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-09-21T03:43:17.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.919+0000 7fb8ca03cf40 1 -- 172.21.15.38:0/1662063052 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x564829f84c70 con 0x564829f80f10 2024-09-21T03:43:17.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.920+0000 7fb8c3af6640 1 -- 172.21.15.38:0/1662063052 <== osd.1 v2:172.21.15.38:6816/1491818732 4 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v127'186 uv186 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb8b405da90 con 0x564829f80f10 2024-09-21T03:43:17.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.920+0000 7fb8ca03cf40 1 -- 172.21.15.38:0/1662063052 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x564829f84c70 con 0x564829f80f10 2024-09-21T03:43:17.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.921+0000 7fb8c3af6640 1 -- 172.21.15.38:0/1662063052 <== osd.1 v2:172.21.15.38:6816/1491818732 5 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v127'187 uv187 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb8b405da90 con 0x564829f80f10 2024-09-21T03:43:17.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.921+0000 7fb8ca03cf40 1 -- 172.21.15.38:0/1662063052 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x564829f83c60 con 0x564829f80f10 2024-09-21T03:43:17.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.922+0000 7fb8c3af6640 1 -- 172.21.15.38:0/1662063052 <== osd.1 v2:172.21.15.38:6816/1491818732 6 ==== osd_op_reply(90 605.00000000 [omap-set-header] v127'188 uv188 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb8b405da90 con 0x564829f80f10 2024-09-21T03:43:17.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.922+0000 7fb8ca03cf40 1 -- 172.21.15.38:0/1662063052 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x564829f83c60 con 0x564829f80f10 2024-09-21T03:43:17.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.923+0000 7fb8c3af6640 1 -- 172.21.15.38:0/1662063052 <== osd.1 v2:172.21.15.38:6816/1491818732 7 ==== osd_op_reply(91 mds0_inotable [create] v127'304 uv298 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7fb8b405da90 con 0x564829f80f10 2024-09-21T03:43:17.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.923+0000 7fb8ca03cf40 1 -- 172.21.15.38:0/1662063052 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e127) -- 0x564829f83c60 con 0x564829f80f10 2024-09-21T03:43:17.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.924+0000 7fb8c3af6640 1 -- 172.21.15.38:0/1662063052 <== osd.1 v2:172.21.15.38:6816/1491818732 8 ==== osd_op_reply(92 mds0_inotable [delete] v127'305 uv305 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fb8b405da90 con 0x564829f80f10 2024-09-21T03:43:17.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.924+0000 7fb8ca03cf40 1 -- 172.21.15.38:0/1662063052 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x564829f83c60 con 0x564829f80f10 2024-09-21T03:43:17.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.926+0000 7fb8c3af6640 1 -- 172.21.15.38:0/1662063052 <== osd.1 v2:172.21.15.38:6816/1491818732 9 ==== osd_op_reply(93 mds0_inotable [create] v127'306 uv306 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fb8b405da90 con 0x564829f80f10 2024-09-21T03:43:17.928 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-09-21T03:43:17.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.926+0000 7fb8ca03cf40 1 -- 172.21.15.38:0/1662063052 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:94 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x564829f83c60 con 0x564829f80f10 2024-09-21T03:43:17.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.927+0000 7fb8c3af6640 1 -- 172.21.15.38:0/1662063052 <== osd.1 v2:172.21.15.38:6816/1491818732 10 ==== osd_op_reply(94 mds0_inotable [write 0~34] v127'307 uv307 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fb8b405da90 con 0x564829f80f10 2024-09-21T03:43:17.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.927+0000 7fb8ca03cf40 1 -- 172.21.15.38:0/1662063052 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:95 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x564829f83c60 con 0x564829f80f10 2024-09-21T03:43:17.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.928+0000 7fb8c3af6640 1 -- 172.21.15.38:0/1662063052 <== osd.1 v2:172.21.15.38:6816/1491818732 11 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v127'308 uv308 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fb8b405da90 con 0x564829f80f10 2024-09-21T03:43:17.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.928+0000 7fb8ca03cf40 1 -- 172.21.15.38:0/1662063052 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:96 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x564829f83c60 con 0x564829f80f10 2024-09-21T03:43:17.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.929+0000 7fb8c3af6640 1 -- 172.21.15.38:0/1662063052 <== osd.1 v2:172.21.15.38:6816/1491818732 12 ==== osd_op_reply(96 100.00000000 [create] v127'309 uv303 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb8b405da90 con 0x564829f80f10 2024-09-21T03:43:17.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.929+0000 7fb8ca03cf40 1 -- 172.21.15.38:0/1662063052 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:97 13.13 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e127) -- 0x564829f83c60 con 0x564829f80f10 2024-09-21T03:43:17.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.930+0000 7fb8c3af6640 1 -- 172.21.15.38:0/1662063052 <== osd.1 v2:172.21.15.38:6816/1491818732 13 ==== osd_op_reply(97 100.00000000 [delete] v127'310 uv310 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb8b405da90 con 0x564829f80f10 2024-09-21T03:43:17.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.930+0000 7fb8ca03cf40 1 -- 172.21.15.38:0/1662063052 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:98 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x564829f83c60 con 0x564829f80f10 2024-09-21T03:43:17.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.931+0000 7fb8c3af6640 1 -- 172.21.15.38:0/1662063052 <== osd.1 v2:172.21.15.38:6816/1491818732 14 ==== osd_op_reply(98 100.00000000 [create] v127'311 uv311 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb8b405da90 con 0x564829f80f10 2024-09-21T03:43:17.933 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-09-21T03:43:17.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.931+0000 7fb8ca03cf40 1 -- 172.21.15.38:0/1662063052 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:99 13.13 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x564829f83c60 con 0x564829f80f10 2024-09-21T03:43:17.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.932+0000 7fb8c3af6640 1 -- 172.21.15.38:0/1662063052 <== osd.1 v2:172.21.15.38:6816/1491818732 15 ==== osd_op_reply(99 100.00000000 [omap-set-header] v127'312 uv312 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb8b405da90 con 0x564829f80f10 2024-09-21T03:43:17.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.932+0000 7fb8ca03cf40 1 -- 172.21.15.38:0/1662063052 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:100 13.13 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x564829f86a70 con 0x564829f80f10 2024-09-21T03:43:17.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.933+0000 7fb8c3af6640 1 -- 172.21.15.38:0/1662063052 <== osd.1 v2:172.21.15.38:6816/1491818732 16 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v127'313 uv313 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb8b405da90 con 0x564829f80f10 2024-09-21T03:43:17.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.933+0000 7fb8ca03cf40 1 -- 172.21.15.38:0/1662063052 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x564829f84110 con 0x564829f7ce80 2024-09-21T03:43:17.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.934+0000 7fb8c42f7640 1 -- 172.21.15.38:0/1662063052 <== osd.3 v2:172.21.15.38:6808/1286810294 16 ==== osd_op_reply(101 mds0_sessionmap [create] v127'132 uv131 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7fb8bc00aa80 con 0x564829f7ce80 2024-09-21T03:43:17.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.934+0000 7fb8ca03cf40 1 -- 172.21.15.38:0/1662063052 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e127) -- 0x564829f84110 con 0x564829f7ce80 2024-09-21T03:43:17.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.935+0000 7fb8c42f7640 1 -- 172.21.15.38:0/1662063052 <== osd.3 v2:172.21.15.38:6808/1286810294 17 ==== osd_op_reply(102 mds0_sessionmap [delete] v127'133 uv133 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fb8bc00aa80 con 0x564829f7ce80 2024-09-21T03:43:17.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.936+0000 7fb8ca03cf40 1 -- 172.21.15.38:0/1662063052 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:103 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x564829f84110 con 0x564829f7ce80 2024-09-21T03:43:17.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.936+0000 7fb8c42f7640 1 -- 172.21.15.38:0/1662063052 <== osd.3 v2:172.21.15.38:6808/1286810294 18 ==== osd_op_reply(103 mds0_sessionmap [create] v127'134 uv134 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fb8bc00aa80 con 0x564829f7ce80 2024-09-21T03:43:17.938 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-09-21T03:43:17.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.937+0000 7fb8ca03cf40 1 -- 172.21.15.38:0/1662063052 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:104 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x564829f84110 con 0x564829f7ce80 2024-09-21T03:43:17.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.937+0000 7fb8c42f7640 1 -- 172.21.15.38:0/1662063052 <== osd.3 v2:172.21.15.38:6808/1286810294 19 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v127'135 uv135 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fb8bc00aa80 con 0x564829f7ce80 2024-09-21T03:43:17.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.937+0000 7fb8ca03cf40 1 -- 172.21.15.38:0/1662063052 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x564829f84110 con 0x564829f7ce80 2024-09-21T03:43:17.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.939+0000 7fb8c42f7640 1 -- 172.21.15.38:0/1662063052 <== osd.3 v2:172.21.15.38:6808/1286810294 20 ==== osd_op_reply(105 609.00000000 [create] v127'183 uv182 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb8bc00aa80 con 0x564829f7ce80 2024-09-21T03:43:17.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.939+0000 7fb8ca03cf40 1 -- 172.21.15.38:0/1662063052 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e127) -- 0x564829f84110 con 0x564829f7ce80 2024-09-21T03:43:17.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.940+0000 7fb8c42f7640 1 -- 172.21.15.38:0/1662063052 <== osd.3 v2:172.21.15.38:6808/1286810294 21 ==== osd_op_reply(106 609.00000000 [delete] v127'184 uv184 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb8bc00aa80 con 0x564829f7ce80 2024-09-21T03:43:17.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.940+0000 7fb8ca03cf40 1 -- 172.21.15.38:0/1662063052 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x564829f84110 con 0x564829f7ce80 2024-09-21T03:43:17.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.941+0000 7fb8c42f7640 1 -- 172.21.15.38:0/1662063052 <== osd.3 v2:172.21.15.38:6808/1286810294 22 ==== osd_op_reply(107 609.00000000 [create] v127'185 uv185 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb8bc00aa80 con 0x564829f7ce80 2024-09-21T03:43:17.943 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-09-21T03:43:17.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.941+0000 7fb8ca03cf40 1 -- 172.21.15.38:0/1662063052 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x564829f85120 con 0x564829f7ce80 2024-09-21T03:43:17.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.942+0000 7fb8c42f7640 1 -- 172.21.15.38:0/1662063052 <== osd.3 v2:172.21.15.38:6808/1286810294 23 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v127'186 uv186 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb8bc00aa80 con 0x564829f7ce80 2024-09-21T03:43:17.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.942+0000 7fb8ca03cf40 1 -- 172.21.15.38:0/1662063052 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:109 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x564829f85120 con 0x564829f7ce80 2024-09-21T03:43:17.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.943+0000 7fb8c42f7640 1 -- 172.21.15.38:0/1662063052 <== osd.3 v2:172.21.15.38:6808/1286810294 24 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v127'187 uv187 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb8bc00aa80 con 0x564829f7ce80 2024-09-21T03:43:17.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.943+0000 7fb8ca03cf40 1 -- 172.21.15.38:0/1662063052 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:110 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x564829f84110 con 0x564829f7ce80 2024-09-21T03:43:17.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.944+0000 7fb8c42f7640 1 -- 172.21.15.38:0/1662063052 <== osd.3 v2:172.21.15.38:6808/1286810294 25 ==== osd_op_reply(110 609.00000000 [omap-set-header] v127'188 uv188 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb8bc00aa80 con 0x564829f7ce80 2024-09-21T03:43:17.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.944+0000 7fb8ca03cf40 1 --2- 172.21.15.38:0/1662063052 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x564829f84620 0x564829f84a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:17.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.944+0000 7fb8ca03cf40 1 -- 172.21.15.38:0/1662063052 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x564829f84f70 con 0x564829f84620 2024-09-21T03:43:17.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.944+0000 7fb8c32f5640 1 --2- 172.21.15.38:0/1662063052 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x564829f84620 0x564829f84a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:17.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.945+0000 7fb8c32f5640 1 --2- 172.21.15.38:0/1662063052 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x564829f84620 0x564829f84a30 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:17.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.946+0000 7fb8c32f5640 1 -- 172.21.15.38:0/1662063052 <== osd.2 v2:172.21.15.38:6824/2577858902 1 ==== osd_op_reply(111 400.00000000 [create] v127'156 uv155 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb8b8065040 con 0x564829f84620 2024-09-21T03:43:17.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.946+0000 7fb8ca03cf40 1 -- 172.21.15.38:0/1662063052 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e127) -- 0x564829f84f70 con 0x564829f84620 2024-09-21T03:43:17.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.947+0000 7fb8c32f5640 1 -- 172.21.15.38:0/1662063052 <== osd.2 v2:172.21.15.38:6824/2577858902 2 ==== osd_op_reply(112 400.00000000 [delete] v127'157 uv157 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb8b8065040 con 0x564829f84620 2024-09-21T03:43:17.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.947+0000 7fb8ca03cf40 1 -- 172.21.15.38:0/1662063052 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x564829f84f70 con 0x564829f84620 2024-09-21T03:43:17.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.948+0000 7fb8c32f5640 1 -- 172.21.15.38:0/1662063052 <== osd.2 v2:172.21.15.38:6824/2577858902 3 ==== osd_op_reply(113 400.00000000 [create] v127'158 uv158 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb8b8065040 con 0x564829f84620 2024-09-21T03:43:17.950 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-09-21T03:43:17.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.948+0000 7fb8ca03cf40 1 -- 172.21.15.38:0/1662063052 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:114 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x564829f84f70 con 0x564829f84620 2024-09-21T03:43:17.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.950+0000 7fb8c32f5640 1 -- 172.21.15.38:0/1662063052 <== osd.2 v2:172.21.15.38:6824/2577858902 4 ==== osd_op_reply(114 400.00000000 [write 0~22] v127'159 uv159 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb8b8065040 con 0x564829f84620 2024-09-21T03:43:17.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.950+0000 7fb8ca03cf40 1 -- 172.21.15.38:0/1662063052 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:115 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x564829f84f70 con 0x564829f84620 2024-09-21T03:43:17.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.951+0000 7fb8c32f5640 1 -- 172.21.15.38:0/1662063052 <== osd.2 v2:172.21.15.38:6824/2577858902 5 ==== osd_op_reply(115 400.00000000 [omap-set-header] v127'160 uv160 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb8b8065040 con 0x564829f84620 2024-09-21T03:43:17.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.951+0000 7fb8ca03cf40 1 -- 172.21.15.38:0/1662063052 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:116 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x564829f84f70 con 0x564829f80f10 2024-09-21T03:43:17.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.952+0000 7fb8c3af6640 1 -- 172.21.15.38:0/1662063052 <== osd.1 v2:172.21.15.38:6816/1491818732 17 ==== osd_op_reply(116 100.00000000.inode [create] v127'376 uv375 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7fb8b405da90 con 0x564829f80f10 2024-09-21T03:43:17.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.952+0000 7fb8ca03cf40 1 -- 172.21.15.38:0/1662063052 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:117 13.1f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e127) -- 0x564829f84f70 con 0x564829f80f10 2024-09-21T03:43:17.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.953+0000 7fb8c3af6640 1 -- 172.21.15.38:0/1662063052 <== osd.1 v2:172.21.15.38:6816/1491818732 18 ==== osd_op_reply(117 100.00000000.inode [delete] v127'377 uv377 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fb8b405da90 con 0x564829f80f10 2024-09-21T03:43:17.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.953+0000 7fb8ca03cf40 1 -- 172.21.15.38:0/1662063052 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:118 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x564829f84f70 con 0x564829f80f10 2024-09-21T03:43:17.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.954+0000 7fb8c3af6640 1 -- 172.21.15.38:0/1662063052 <== osd.1 v2:172.21.15.38:6816/1491818732 19 ==== osd_op_reply(118 100.00000000.inode [create] v127'378 uv378 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fb8b405da90 con 0x564829f80f10 2024-09-21T03:43:17.956 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-09-21T03:43:17.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.954+0000 7fb8ca03cf40 1 -- 172.21.15.38:0/1662063052 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:119 13.1f 13:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x564829f84f70 con 0x564829f80f10 2024-09-21T03:43:17.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.955+0000 7fb8c3af6640 1 -- 172.21.15.38:0/1662063052 <== osd.1 v2:172.21.15.38:6816/1491818732 20 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v127'379 uv379 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fb8b405da90 con 0x564829f80f10 2024-09-21T03:43:17.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.955+0000 7fb8ca03cf40 1 -- 172.21.15.38:0/1662063052 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:120 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x564829f84f70 con 0x564829f80f10 2024-09-21T03:43:17.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.957+0000 7fb8c3af6640 1 -- 172.21.15.38:0/1662063052 <== osd.1 v2:172.21.15.38:6816/1491818732 21 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v127'380 uv380 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fb8b405da90 con 0x564829f80f10 2024-09-21T03:43:17.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.957+0000 7fb8ca03cf40 1 -- 172.21.15.38:0/1662063052 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:121 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x564829f84f70 con 0x564829f80f10 2024-09-21T03:43:17.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.958+0000 7fb8c3af6640 1 -- 172.21.15.38:0/1662063052 <== osd.1 v2:172.21.15.38:6816/1491818732 22 ==== osd_op_reply(121 1.00000000 [create] v127'381 uv374 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7fb8b405da90 con 0x564829f80f10 2024-09-21T03:43:17.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.958+0000 7fb8ca03cf40 1 -- 172.21.15.38:0/1662063052 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:122 13.1f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e127) -- 0x564829f84f70 con 0x564829f80f10 2024-09-21T03:43:17.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.959+0000 7fb8c3af6640 1 -- 172.21.15.38:0/1662063052 <== osd.1 v2:172.21.15.38:6816/1491818732 23 ==== osd_op_reply(122 1.00000000 [delete] v127'382 uv382 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fb8b405da90 con 0x564829f80f10 2024-09-21T03:43:17.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.959+0000 7fb8ca03cf40 1 -- 172.21.15.38:0/1662063052 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:123 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x564829f84f70 con 0x564829f80f10 2024-09-21T03:43:17.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.960+0000 7fb8c3af6640 1 -- 172.21.15.38:0/1662063052 <== osd.1 v2:172.21.15.38:6816/1491818732 24 ==== osd_op_reply(123 1.00000000 [create] v127'383 uv383 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fb8b405da90 con 0x564829f80f10 2024-09-21T03:43:17.962 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-09-21T03:43:17.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.960+0000 7fb8ca03cf40 1 -- 172.21.15.38:0/1662063052 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:124 13.1f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x564829f84f70 con 0x564829f80f10 2024-09-21T03:43:17.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.961+0000 7fb8c3af6640 1 -- 172.21.15.38:0/1662063052 <== osd.1 v2:172.21.15.38:6816/1491818732 25 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v127'384 uv384 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fb8b405da90 con 0x564829f80f10 2024-09-21T03:43:17.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.961+0000 7fb8ca03cf40 1 -- 172.21.15.38:0/1662063052 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:125 13.1f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x564829f84f70 con 0x564829f80f10 2024-09-21T03:43:17.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.962+0000 7fb8c3af6640 1 -- 172.21.15.38:0/1662063052 <== osd.1 v2:172.21.15.38:6816/1491818732 26 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v127'385 uv385 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fb8b405da90 con 0x564829f80f10 2024-09-21T03:43:17.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.962+0000 7fb8ca03cf40 1 -- 172.21.15.38:0/1662063052 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:126 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x564829f84f70 con 0x564829f80f10 2024-09-21T03:43:17.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.963+0000 7fb8c3af6640 1 -- 172.21.15.38:0/1662063052 <== osd.1 v2:172.21.15.38:6816/1491818732 27 ==== osd_op_reply(126 1.00000000 [omap-set-header] v127'386 uv386 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fb8b405da90 con 0x564829f80f10 2024-09-21T03:43:17.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.963+0000 7fb8ca03cf40 1 -- 172.21.15.38:0/1662063052 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:127 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=584b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x564829f865e0 con 0x564829f80f10 2024-09-21T03:43:17.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.965+0000 7fb8c3af6640 1 -- 172.21.15.38:0/1662063052 <== osd.1 v2:172.21.15.38:6816/1491818732 28 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v127'387 uv387 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fb8b405da90 con 0x564829f80f10 2024-09-21T03:43:17.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.965+0000 7fb8ca03cf40 1 -- 172.21.15.38:0/1662063052 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x564829f789e0 msgr2=0x564829f7ae00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:17.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.965+0000 7fb8ca03cf40 1 --2- 172.21.15.38:0/1662063052 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x564829f789e0 0x564829f7ae00 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:17.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.965+0000 7fb8ca03cf40 1 -- 172.21.15.38:0/1662063052 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x564829f80f10 msgr2=0x564829f83330 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:17.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.965+0000 7fb8ca03cf40 1 --2- 172.21.15.38:0/1662063052 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x564829f80f10 0x564829f83330 crc :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:17.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.965+0000 7fb8ca03cf40 1 -- 172.21.15.38:0/1662063052 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x564829f84620 msgr2=0x564829f84a30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:17.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.965+0000 7fb8ca03cf40 1 --2- 172.21.15.38:0/1662063052 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x564829f84620 0x564829f84a30 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:17.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.965+0000 7fb8ca03cf40 1 -- 172.21.15.38:0/1662063052 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x564829f7ce80 msgr2=0x564829f7fce0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:17.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.965+0000 7fb8ca03cf40 1 --2- 172.21.15.38:0/1662063052 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x564829f7ce80 0x564829f7fce0 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:17.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.965+0000 7fb8c42f7640 1 -- 172.21.15.38:0/1662063052 reap_dead start 2024-09-21T03:43:17.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.965+0000 7fb8ca03cf40 1 -- 172.21.15.38:0/1662063052 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x564829f73f90 msgr2=0x564829f763b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:17.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.965+0000 7fb8ca03cf40 1 --2- 172.21.15.38:0/1662063052 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x564829f73f90 0x564829f763b0 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:17.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.965+0000 7fb8c42f7640 1 -- 172.21.15.38:0/1662063052 reap_dead start 2024-09-21T03:43:17.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.965+0000 7fb8ca03cf40 1 -- 172.21.15.38:0/1662063052 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x564829ec8500 msgr2=0x564829eaa020 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:17.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.965+0000 7fb8ca03cf40 1 --2- 172.21.15.38:0/1662063052 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x564829ec8500 0x564829eaa020 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:17.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.965+0000 7fb8ca03cf40 1 -- 172.21.15.38:0/1662063052 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb8b0077570 msgr2=0x7fb8b0079a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:17.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.965+0000 7fb8ca03cf40 1 --2- 172.21.15.38:0/1662063052 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb8b0077570 0x7fb8b0079a80 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fb8b8055000 tx=0x7fb8b8058040 comp rx=0 tx=0).stop 2024-09-21T03:43:17.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.965+0000 7fb8ca03cf40 1 -- 172.21.15.38:0/1662063052 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564829df8d00 msgr2=0x564829e3ce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:17.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.965+0000 7fb8ca03cf40 1 --2- 172.21.15.38:0/1662063052 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564829df8d00 0x564829e3ce30 secure :-1 s=READY pgs=1992 cs=0 l=1 rev1=1 crypto rx=0x7fb8b400d3c0 tx=0x7fb8b400d880 comp rx=0 tx=0).stop 2024-09-21T03:43:17.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.965+0000 7fb8ca03cf40 1 -- 172.21.15.38:0/1662063052 shutdown_connections 2024-09-21T03:43:17.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.965+0000 7fb8ca03cf40 1 --2- 172.21.15.38:0/1662063052 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x564829ec8500 0x564829eaa020 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:17.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.965+0000 7fb8ca03cf40 1 --2- 172.21.15.38:0/1662063052 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb8b0077570 0x7fb8b0079a80 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:17.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.965+0000 7fb8ca03cf40 1 --2- 172.21.15.38:0/1662063052 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564829df8d00 0x564829e3ce30 unknown :-1 s=CLOSED pgs=1992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:17.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.965+0000 7fb8ca03cf40 1 -- 172.21.15.38:0/1662063052 >> 172.21.15.38:0/1662063052 conn(0x564829d17ac0 msgr2=0x564829e3e990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:17.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.965+0000 7fb8ca03cf40 1 -- 172.21.15.38:0/1662063052 shutdown_connections 2024-09-21T03:43:17.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:17.965+0000 7fb8ca03cf40 1 -- 172.21.15.38:0/1662063052 wait complete. 2024-09-21T03:43:17.970 DEBUG:teuthology.orchestra.run.smithi038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata setomapval 10000000000.00000000 sixmegs_head deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef 2024-09-21T03:43:18.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:18.067+0000 7f3c04f07f40 1 Processor -- start 2024-09-21T03:43:18.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:18.067+0000 7f3c04f07f40 1 -- start start 2024-09-21T03:43:18.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:18.068+0000 7f3c04f07f40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3be80028f0 0x55d72e5ba4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:18.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:18.068+0000 7f3c04f07f40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d72e6f16b0 0x55d72e6f1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:18.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:18.068+0000 7f3c04f07f40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55d72e6f2000 0x55d72e5b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:18.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:18.068+0000 7f3c04f07f40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55d72e539430 con 0x7f3be80028f0 2024-09-21T03:43:18.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:18.068+0000 7f3c04f07f40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55d72e5461f0 con 0x55d72e6f16b0 2024-09-21T03:43:18.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:18.068+0000 7f3c04f07f40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55d72e4dad60 con 0x55d72e6f2000 2024-09-21T03:43:18.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:18.068+0000 7f3bfe9c1640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3be80028f0 0x55d72e5ba4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:18.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:18.068+0000 7f3bfe9c1640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3be80028f0 0x55d72e5ba4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:36162/0 (socket says 172.21.15.38:36162) 2024-09-21T03:43:18.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:18.068+0000 7f3bfe9c1640 1 -- 172.21.15.38:0/887214201 learned_addr learned my addr 172.21.15.38:0/887214201 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:43:18.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:18.068+0000 7f3bfe1c0640 1 --2- 172.21.15.38:0/887214201 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d72e6f16b0 0x55d72e6f1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:18.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:18.069+0000 7f3bff1c2640 1 --2- 172.21.15.38:0/887214201 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55d72e6f2000 0x55d72e5b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:18.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:18.069+0000 7f3bfe9c1640 1 -- 172.21.15.38:0/887214201 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55d72e6f2000 msgr2=0x55d72e5b5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:18.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:18.069+0000 7f3bfe9c1640 1 --2- 172.21.15.38:0/887214201 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55d72e6f2000 0x55d72e5b5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:18.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:18.069+0000 7f3bfe9c1640 1 -- 172.21.15.38:0/887214201 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d72e6f16b0 msgr2=0x55d72e6f1ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:18.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:18.069+0000 7f3bfe9c1640 1 --2- 172.21.15.38:0/887214201 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d72e6f16b0 0x55d72e6f1ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:18.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:18.069+0000 7f3bfe9c1640 1 -- 172.21.15.38:0/887214201 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d72e661ab0 con 0x7f3be80028f0 2024-09-21T03:43:18.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:18.069+0000 7f3bff1c2640 1 --2- 172.21.15.38:0/887214201 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55d72e6f2000 0x55d72e5b5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:43:18.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:18.069+0000 7f3bfe9c1640 1 --2- 172.21.15.38:0/887214201 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3be80028f0 0x55d72e5ba4f0 secure :-1 s=READY pgs=1995 cs=0 l=1 rev1=1 crypto rx=0x7f3bf0009a20 tx=0x7f3bf0009ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4aed72552a579ee server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:18.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:18.069+0000 7f3bfd1be640 1 -- 172.21.15.38:0/887214201 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bf0009240 con 0x7f3be80028f0 2024-09-21T03:43:18.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:18.069+0000 7f3bfd1be640 1 -- 172.21.15.38:0/887214201 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3bf00093e0 con 0x7f3be80028f0 2024-09-21T03:43:18.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:18.069+0000 7f3bfd1be640 1 -- 172.21.15.38:0/887214201 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bf00059b0 con 0x7f3be80028f0 2024-09-21T03:43:18.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:18.070+0000 7f3c04f07f40 1 -- 172.21.15.38:0/887214201 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3be80028f0 msgr2=0x55d72e5ba4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:18.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:18.070+0000 7f3c04f07f40 1 --2- 172.21.15.38:0/887214201 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3be80028f0 0x55d72e5ba4f0 secure :-1 s=READY pgs=1995 cs=0 l=1 rev1=1 crypto rx=0x7f3bf0009a20 tx=0x7f3bf0009ee0 comp rx=0 tx=0).stop 2024-09-21T03:43:18.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:18.070+0000 7f3c04f07f40 1 -- 172.21.15.38:0/887214201 shutdown_connections 2024-09-21T03:43:18.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:18.070+0000 7f3c04f07f40 1 --2- 172.21.15.38:0/887214201 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55d72e6f2000 0x55d72e5b5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:18.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:18.070+0000 7f3c04f07f40 1 --2- 172.21.15.38:0/887214201 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d72e6f16b0 0x55d72e6f1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:18.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:18.070+0000 7f3c04f07f40 1 --2- 172.21.15.38:0/887214201 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3be80028f0 0x55d72e5ba4f0 unknown :-1 s=CLOSED pgs=1995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:18.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:18.070+0000 7f3c04f07f40 1 -- 172.21.15.38:0/887214201 >> 172.21.15.38:0/887214201 conn(0x55d72e4abac0 msgr2=0x55d72e643660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:18.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:18.070+0000 7f3c04f07f40 1 -- 172.21.15.38:0/887214201 shutdown_connections 2024-09-21T03:43:18.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:18.070+0000 7f3c04f07f40 1 -- 172.21.15.38:0/887214201 wait complete. 2024-09-21T03:43:18.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:18.071+0000 7f3c04f07f40 1 Processor -- start 2024-09-21T03:43:18.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:18.071+0000 7f3c04f07f40 1 -- start start 2024-09-21T03:43:18.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:18.072+0000 7f3c04f07f40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d72e6f16b0 0x55d72e65e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:18.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:18.072+0000 7f3c04f07f40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55d72e6f2000 0x55d72e5ba4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:18.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:18.072+0000 7f3c04f07f40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3be80028f0 0x55d72e5baa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:18.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:18.072+0000 7f3c04f07f40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55d72e661d50 con 0x7f3be80028f0 2024-09-21T03:43:18.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:18.072+0000 7f3c04f07f40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55d72e58ac30 con 0x55d72e6f16b0 2024-09-21T03:43:18.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:18.072+0000 7f3c04f07f40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55d72e58af10 con 0x55d72e6f2000 2024-09-21T03:43:18.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:18.072+0000 7f3bff1c2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3be80028f0 0x55d72e5baa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:18.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:18.072+0000 7f3bff1c2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3be80028f0 0x55d72e5baa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:36176/0 (socket says 172.21.15.38:36176) 2024-09-21T03:43:18.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:18.072+0000 7f3bff1c2640 1 -- 172.21.15.38:0/1436627989 learned_addr learned my addr 172.21.15.38:0/1436627989 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:43:18.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:18.072+0000 7f3bfe1c0640 1 --2- 172.21.15.38:0/1436627989 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55d72e6f2000 0x55d72e5ba4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:18.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:18.072+0000 7f3bfe9c1640 1 --2- 172.21.15.38:0/1436627989 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d72e6f16b0 0x55d72e65e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:18.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:18.072+0000 7f3bff1c2640 1 -- 172.21.15.38:0/1436627989 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55d72e6f2000 msgr2=0x55d72e5ba4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:18.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:18.072+0000 7f3bff1c2640 1 --2- 172.21.15.38:0/1436627989 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55d72e6f2000 0x55d72e5ba4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:18.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:18.072+0000 7f3bff1c2640 1 -- 172.21.15.38:0/1436627989 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d72e6f16b0 msgr2=0x55d72e65e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:18.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:18.072+0000 7f3bff1c2640 1 --2- 172.21.15.38:0/1436627989 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d72e6f16b0 0x55d72e65e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:18.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:18.072+0000 7f3bff1c2640 1 -- 172.21.15.38:0/1436627989 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d72e588b50 con 0x7f3be80028f0 2024-09-21T03:43:18.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:18.073+0000 7f3bff1c2640 1 --2- 172.21.15.38:0/1436627989 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3be80028f0 0x55d72e5baa30 secure :-1 s=READY pgs=1996 cs=0 l=1 rev1=1 crypto rx=0x55d72e5d1f60 tx=0x7f3bf800aaf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:18.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:18.073+0000 7f3be7fff640 1 -- 172.21.15.38:0/1436627989 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bf80099f0 con 0x7f3be80028f0 2024-09-21T03:43:18.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:18.073+0000 7f3be7fff640 1 -- 172.21.15.38:0/1436627989 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3bf801b070 con 0x7f3be80028f0 2024-09-21T03:43:18.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:18.073+0000 7f3c04f07f40 1 -- 172.21.15.38:0/1436627989 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55d72e588e30 con 0x7f3be80028f0 2024-09-21T03:43:18.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:18.073+0000 7f3be7fff640 1 -- 172.21.15.38:0/1436627989 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bf8009dd0 con 0x7f3be80028f0 2024-09-21T03:43:18.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:18.073+0000 7f3c04f07f40 1 -- 172.21.15.38:0/1436627989 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d72e6f9ea0 con 0x7f3be80028f0 2024-09-21T03:43:18.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:18.075+0000 7f3be7fff640 1 -- 172.21.15.38:0/1436627989 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f3bf8016470 con 0x7f3be80028f0 2024-09-21T03:43:18.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:18.075+0000 7f3be7fff640 1 --2- 172.21.15.38:0/1436627989 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3bec0775a0 0x7f3bec079ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:18.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:18.075+0000 7f3be7fff640 1 -- 172.21.15.38:0/1436627989 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 8970+0+0 (secure 0 0 0) 0x7f3bf809b800 con 0x7f3be80028f0 2024-09-21T03:43:18.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:18.075+0000 7f3bfe9c1640 1 --2- 172.21.15.38:0/1436627989 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3bec0775a0 0x7f3bec079ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:18.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:18.076+0000 7f3c04f07f40 1 --2- 172.21.15.38:0/1436627989 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x55d72e6fd9a0 0x55d72e6fddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:18.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:18.076+0000 7f3c04f07f40 1 -- 172.21.15.38:0/1436627989 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:1 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=104b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x55d72e5b6670 con 0x55d72e6fd9a0 2024-09-21T03:43:18.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:18.076+0000 7f3bfe1c0640 1 --2- 172.21.15.38:0/1436627989 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x55d72e6fd9a0 0x55d72e6fddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:18.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:18.076+0000 7f3bfe9c1640 1 --2- 172.21.15.38:0/1436627989 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3bec0775a0 0x7f3bec079ab0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f3bf0000f80 tx=0x7f3bf0018040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:18.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:18.076+0000 7f3bfe1c0640 1 --2- 172.21.15.38:0/1436627989 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x55d72e6fd9a0 0x55d72e6fddb0 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:18.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:18.078+0000 7f3bfe1c0640 1 -- 172.21.15.38:0/1436627989 <== osd.7 v2:172.21.15.195:6818/1606106342 1 ==== osd_op_reply(1 10000000000.00000000 [omap-set-vals] v127'970 uv970 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f3bf40014b0 con 0x55d72e6fd9a0 2024-09-21T03:43:18.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:18.078+0000 7f3c04f07f40 1 -- 172.21.15.38:0/1436627989 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x55d72e6fd9a0 msgr2=0x55d72e6fddb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:18.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:18.078+0000 7f3c04f07f40 1 --2- 172.21.15.38:0/1436627989 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x55d72e6fd9a0 0x55d72e6fddb0 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:18.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:18.078+0000 7f3c04f07f40 1 -- 172.21.15.38:0/1436627989 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3bec0775a0 msgr2=0x7f3bec079ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:18.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:18.078+0000 7f3c04f07f40 1 --2- 172.21.15.38:0/1436627989 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3bec0775a0 0x7f3bec079ab0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f3bf0000f80 tx=0x7f3bf0018040 comp rx=0 tx=0).stop 2024-09-21T03:43:18.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:18.078+0000 7f3c04f07f40 1 -- 172.21.15.38:0/1436627989 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3be80028f0 msgr2=0x55d72e5baa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:18.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:18.078+0000 7f3c04f07f40 1 --2- 172.21.15.38:0/1436627989 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3be80028f0 0x55d72e5baa30 secure :-1 s=READY pgs=1996 cs=0 l=1 rev1=1 crypto rx=0x55d72e5d1f60 tx=0x7f3bf800aaf0 comp rx=0 tx=0).stop 2024-09-21T03:43:18.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:18.078+0000 7f3bff1c2640 1 -- 172.21.15.38:0/1436627989 reap_dead start 2024-09-21T03:43:18.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:18.078+0000 7f3c04f07f40 1 -- 172.21.15.38:0/1436627989 shutdown_connections 2024-09-21T03:43:18.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:18.078+0000 7f3c04f07f40 1 -- 172.21.15.38:0/1436627989 >> 172.21.15.38:0/1436627989 conn(0x55d72e4abac0 msgr2=0x55d72e642ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:18.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:18.078+0000 7f3c04f07f40 1 -- 172.21.15.38:0/1436627989 shutdown_connections 2024-09-21T03:43:18.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:18.078+0000 7f3c04f07f40 1 -- 172.21.15.38:0/1436627989 wait complete. 2024-09-21T03:43:18.083 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-21T03:43:18.202 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:18.200+0000 7fd01060a640 1 -- 172.21.15.38:0/3178713243 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd0080ceb90 msgr2=0x7fd0080cefa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:18.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:18.200+0000 7fd01060a640 1 --2- 172.21.15.38:0/3178713243 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd0080ceb90 0x7fd0080cefa0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fd000009be0 tx=0x7fd00002a820 comp rx=0 tx=0).stop 2024-09-21T03:43:18.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:18.200+0000 7fd01060a640 1 -- 172.21.15.38:0/3178713243 shutdown_connections 2024-09-21T03:43:18.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:18.200+0000 7fd01060a640 1 --2- 172.21.15.38:0/3178713243 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd0080ceb90 0x7fd0080cefa0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:18.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:18.200+0000 7fd01060a640 1 -- 172.21.15.38:0/3178713243 >> 172.21.15.38:0/3178713243 conn(0x7fd00809d830 msgr2=0x7fd0080b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:18.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:18.200+0000 7fd01060a640 1 -- 172.21.15.38:0/3178713243 shutdown_connections 2024-09-21T03:43:18.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:18.200+0000 7fd01060a640 1 -- 172.21.15.38:0/3178713243 wait complete. 2024-09-21T03:43:18.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:18.202+0000 7fd01060a640 1 Processor -- start 2024-09-21T03:43:18.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:18.202+0000 7fd01060a640 1 -- start start 2024-09-21T03:43:18.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:18.203+0000 7fd01060a640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd0080ceb90 0x7fd008158a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:18.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:18.203+0000 7fd01060a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd0080d0170 0x7fd0081f88d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:18.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:18.203+0000 7fd01060a640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd00815ee80 0x7fd0080d3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:18.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:18.203+0000 7fd01060a640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fd0080b6910 con 0x7fd0080d0170 2024-09-21T03:43:18.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:18.203+0000 7fd01060a640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd0080b6790 con 0x7fd0080ceb90 2024-09-21T03:43:18.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:18.203+0000 7fd01060a640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fd0080b6a90 con 0x7fd00815ee80 2024-09-21T03:43:18.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:18.203+0000 7fd00dba5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd0080d0170 0x7fd0081f88d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:18.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:18.204+0000 7fd00dba5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd0080d0170 0x7fd0081f88d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:36192/0 (socket says 172.21.15.38:36192) 2024-09-21T03:43:18.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:18.204+0000 7fd00dba5640 1 -- 172.21.15.38:0/1211523181 learned_addr learned my addr 172.21.15.38:0/1211523181 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:43:18.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:18.204+0000 7fd00eba7640 1 --2- 172.21.15.38:0/1211523181 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd00815ee80 0x7fd0080d3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:18.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:18.204+0000 7fd00e3a6640 1 --2- 172.21.15.38:0/1211523181 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd0080ceb90 0x7fd008158a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:18.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:18.204+0000 7fd00dba5640 1 -- 172.21.15.38:0/1211523181 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd00815ee80 msgr2=0x7fd0080d3bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:18.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:18.204+0000 7fd00dba5640 1 --2- 172.21.15.38:0/1211523181 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd00815ee80 0x7fd0080d3bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:18.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:18.204+0000 7fd00dba5640 1 -- 172.21.15.38:0/1211523181 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd0080ceb90 msgr2=0x7fd008158a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:18.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:18.204+0000 7fd00dba5640 1 --2- 172.21.15.38:0/1211523181 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd0080ceb90 0x7fd008158a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:18.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:18.204+0000 7fd00dba5640 1 -- 172.21.15.38:0/1211523181 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd008158f60 con 0x7fd0080d0170 2024-09-21T03:43:18.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:18.204+0000 7fd00dba5640 1 --2- 172.21.15.38:0/1211523181 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd0080d0170 0x7fd0081f88d0 secure :-1 s=READY pgs=1998 cs=0 l=1 rev1=1 crypto rx=0x7fcff800c310 tx=0x7fcff800c7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:18.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:18.204+0000 7fcffeffd640 1 -- 172.21.15.38:0/1211523181 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcff8017070 con 0x7fd0080d0170 2024-09-21T03:43:18.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:18.204+0000 7fd01060a640 1 -- 172.21.15.38:0/1211523181 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0081f9570 con 0x7fd0080d0170 2024-09-21T03:43:18.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:18.204+0000 7fcffeffd640 1 -- 172.21.15.38:0/1211523181 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcff80053e0 con 0x7fd0080d0170 2024-09-21T03:43:18.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:18.204+0000 7fcffeffd640 1 -- 172.21.15.38:0/1211523181 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcff80038a0 con 0x7fd0080d0170 2024-09-21T03:43:18.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:18.204+0000 7fd01060a640 1 -- 172.21.15.38:0/1211523181 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0081f5760 con 0x7fd0080d0170 2024-09-21T03:43:18.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:18.205+0000 7fd01060a640 1 -- 172.21.15.38:0/1211523181 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0080cefa0 con 0x7fd0080d0170 2024-09-21T03:43:18.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:18.206+0000 7fcffeffd640 1 -- 172.21.15.38:0/1211523181 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fcff801a070 con 0x7fd0080d0170 2024-09-21T03:43:18.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:18.207+0000 7fcffeffd640 1 --2- 172.21.15.38:0/1211523181 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fcfec078b90 0x7fcfec07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:18.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:18.207+0000 7fcffeffd640 1 -- 172.21.15.38:0/1211523181 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 8970+0+0 (secure 0 0 0) 0x7fcff809adf0 con 0x7fd0080d0170 2024-09-21T03:43:18.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:18.207+0000 7fd00e3a6640 1 --2- 172.21.15.38:0/1211523181 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fcfec078b90 0x7fcfec07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:18.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:18.207+0000 7fd00e3a6640 1 --2- 172.21.15.38:0/1211523181 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fcfec078b90 0x7fcfec07b0a0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fd004000c00 tx=0x7fd004004600 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:18.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:18.211+0000 7fcffeffd640 1 -- 172.21.15.38:0/1211523181 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fd0080cf4e0 con 0x7fd0080d0170 2024-09-21T03:43:18.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:18.437+0000 7fd01060a640 1 -- 172.21.15.38:0/1211523181 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7fd0080762a0 con 0x7fd0080d0170 2024-09-21T03:43:18.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:18.714+0000 7fcffeffd640 1 -- 172.21.15.38:0/1211523181 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v271) ==== 162+0+0 (secure 0 0 0) 0x7fcff8061e80 con 0x7fd0080d0170 2024-09-21T03:43:18.716 INFO:teuthology.orchestra.run.smithi038.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-21T03:43:18.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:18.716+0000 7fd01060a640 1 -- 172.21.15.38:0/1211523181 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fcfec078b90 msgr2=0x7fcfec07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:18.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:18.716+0000 7fd01060a640 1 --2- 172.21.15.38:0/1211523181 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fcfec078b90 0x7fcfec07b0a0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fd004000c00 tx=0x7fd004004600 comp rx=0 tx=0).stop 2024-09-21T03:43:18.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:18.716+0000 7fd01060a640 1 -- 172.21.15.38:0/1211523181 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd0080d0170 msgr2=0x7fd0081f88d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:18.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:18.716+0000 7fd01060a640 1 --2- 172.21.15.38:0/1211523181 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd0080d0170 0x7fd0081f88d0 secure :-1 s=READY pgs=1998 cs=0 l=1 rev1=1 crypto rx=0x7fcff800c310 tx=0x7fcff800c7d0 comp rx=0 tx=0).stop 2024-09-21T03:43:18.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:18.716+0000 7fd01060a640 1 -- 172.21.15.38:0/1211523181 shutdown_connections 2024-09-21T03:43:18.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:18.716+0000 7fd01060a640 1 --2- 172.21.15.38:0/1211523181 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fcfec078b90 0x7fcfec07b0a0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:18.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:18.716+0000 7fd01060a640 1 --2- 172.21.15.38:0/1211523181 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd00815ee80 0x7fd0080d3bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:18.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:18.716+0000 7fd01060a640 1 --2- 172.21.15.38:0/1211523181 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd0080d0170 0x7fd0081f88d0 unknown :-1 s=CLOSED pgs=1998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:18.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:18.716+0000 7fd01060a640 1 --2- 172.21.15.38:0/1211523181 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd0080ceb90 0x7fd008158a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:18.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:18.716+0000 7fd01060a640 1 -- 172.21.15.38:0/1211523181 >> 172.21.15.38:0/1211523181 conn(0x7fd00809d830 msgr2=0x7fd0081619a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:18.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:18.717+0000 7fd01060a640 1 -- 172.21.15.38:0/1211523181 shutdown_connections 2024-09-21T03:43:18.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:18.717+0000 7fd01060a640 1 -- 172.21.15.38:0/1211523181 wait complete. 2024-09-21T03:43:19.136 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:43:19.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:19.264+0000 7f869c18b640 1 -- 172.21.15.38:0/1215045160 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f86940d1d00 msgr2=0x7f86940cc930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:19.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:19.264+0000 7f869c18b640 1 --2- 172.21.15.38:0/1215045160 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f86940d1d00 0x7f86940cc930 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f8684009830 tx=0x7f868402a500 comp rx=0 tx=0).stop 2024-09-21T03:43:19.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:19.264+0000 7f869c18b640 1 -- 172.21.15.38:0/1215045160 shutdown_connections 2024-09-21T03:43:19.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:19.264+0000 7f869c18b640 1 --2- 172.21.15.38:0/1215045160 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f86940d1d00 0x7f86940cc930 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:19.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:19.264+0000 7f869c18b640 1 --2- 172.21.15.38:0/1215045160 >> v2:172.21.15.195:3300/0 conn(0x7f86941536e0 0x7f86940d17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:19.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:19.264+0000 7f869c18b640 1 -- 172.21.15.38:0/1215045160 >> 172.21.15.38:0/1215045160 conn(0x7f86940cef40 msgr2=0x7f86940cf350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:19.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:19.264+0000 7f869c18b640 1 -- 172.21.15.38:0/1215045160 shutdown_connections 2024-09-21T03:43:19.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:19.264+0000 7f869c18b640 1 -- 172.21.15.38:0/1215045160 wait complete. 2024-09-21T03:43:19.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:19.266+0000 7f869c18b640 1 Processor -- start 2024-09-21T03:43:19.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:19.266+0000 7f869c18b640 1 -- start start 2024-09-21T03:43:19.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:19.267+0000 7f869c18b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f86940cce70 0x7f8694156570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:19.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:19.267+0000 7f869c18b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f86940d1d00 0x7f8694154d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:19.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:19.267+0000 7f869c18b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f86941536e0 0x7f86941552b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:19.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:19.267+0000 7f869c18b640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f86941608a0 con 0x7f86940d1d00 2024-09-21T03:43:19.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:19.267+0000 7f869c18b640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f8694160a20 con 0x7f86941536e0 2024-09-21T03:43:19.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:19.267+0000 7f869c18b640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f8694160ba0 con 0x7f86940cce70 2024-09-21T03:43:19.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:19.267+0000 7f8699726640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f86940d1d00 0x7f8694154d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:19.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:19.267+0000 7f8699726640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f86940d1d00 0x7f8694154d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:36208/0 (socket says 172.21.15.38:36208) 2024-09-21T03:43:19.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:19.267+0000 7f8699726640 1 -- 172.21.15.38:0/2345277759 learned_addr learned my addr 172.21.15.38:0/2345277759 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:43:19.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:19.268+0000 7f8699f27640 1 --2- 172.21.15.38:0/2345277759 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f86940cce70 0x7f8694156570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:19.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:19.268+0000 7f869a728640 1 --2- 172.21.15.38:0/2345277759 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f86941536e0 0x7f86941552b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:19.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:19.268+0000 7f8699726640 1 -- 172.21.15.38:0/2345277759 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f86940cce70 msgr2=0x7f8694156570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:19.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:19.268+0000 7f8699726640 1 --2- 172.21.15.38:0/2345277759 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f86940cce70 0x7f8694156570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:19.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:19.268+0000 7f8699726640 1 -- 172.21.15.38:0/2345277759 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f86941536e0 msgr2=0x7f86941552b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:19.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:19.268+0000 7f8699726640 1 --2- 172.21.15.38:0/2345277759 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f86941536e0 0x7f86941552b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:19.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:19.268+0000 7f8699726640 1 -- 172.21.15.38:0/2345277759 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8694153cd0 con 0x7f86940d1d00 2024-09-21T03:43:19.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:19.268+0000 7f8699726640 1 --2- 172.21.15.38:0/2345277759 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f86940d1d00 0x7f8694154d70 secure :-1 s=READY pgs=1999 cs=0 l=1 rev1=1 crypto rx=0x7f868402aaf0 tx=0x7f86840040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:19.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:19.268+0000 7f868a7fc640 1 -- 172.21.15.38:0/2345277759 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8684005220 con 0x7f86940d1d00 2024-09-21T03:43:19.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:19.268+0000 7f868a7fc640 1 -- 172.21.15.38:0/2345277759 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f86840054b0 con 0x7f86940d1d00 2024-09-21T03:43:19.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:19.268+0000 7f869c18b640 1 -- 172.21.15.38:0/2345277759 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8694153fb0 con 0x7f86940d1d00 2024-09-21T03:43:19.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:19.268+0000 7f868a7fc640 1 -- 172.21.15.38:0/2345277759 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8684004a00 con 0x7f86940d1d00 2024-09-21T03:43:19.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:19.268+0000 7f869c18b640 1 -- 172.21.15.38:0/2345277759 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8694154570 con 0x7f86940d1d00 2024-09-21T03:43:19.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:19.269+0000 7f869c18b640 1 -- 172.21.15.38:0/2345277759 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86940cc930 con 0x7f86940d1d00 2024-09-21T03:43:19.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:19.270+0000 7f868a7fc640 1 -- 172.21.15.38:0/2345277759 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f8684004ba0 con 0x7f86940d1d00 2024-09-21T03:43:19.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:19.271+0000 7f868a7fc640 1 --2- 172.21.15.38:0/2345277759 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8678078b90 0x7f867807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:19.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:19.271+0000 7f868a7fc640 1 -- 172.21.15.38:0/2345277759 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 8970+0+0 (secure 0 0 0) 0x7f86840bcce0 con 0x7f86940d1d00 2024-09-21T03:43:19.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:19.271+0000 7f8699f27640 1 --2- 172.21.15.38:0/2345277759 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8678078b90 0x7f867807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:19.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:19.272+0000 7f8699f27640 1 --2- 172.21.15.38:0/2345277759 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8678078b90 0x7f867807b0a0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f8690009b30 tx=0x7f8690005f90 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:19.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:19.276+0000 7f868a7fc640 1 -- 172.21.15.38:0/2345277759 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f868403f030 con 0x7f86940d1d00 2024-09-21T03:43:19.436 INFO:tasks.ceph.mds.a.smithi038.stderr: -51> 2024-09-21T03:43:14.683+0000 7fae65520640 -1 mds.0.cache.ino(0x100) No data while reading inode 0x100 2024-09-21T03:43:19.437 INFO:tasks.ceph.mds.a.smithi038.stderr: -41> 2024-09-21T03:43:14.683+0000 7fae65520640 -1 log_channel(cluster) log [ERR] : Error loading MDS rank 0: (2) No such file or directory 2024-09-21T03:43:19.491 INFO:tasks.ceph.mds.a.smithi038.stdout:starting mds.a at 2024-09-21T03:43:19.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:19.503+0000 7f869c18b640 1 -- 172.21.15.38:0/2345277759 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8694156d50 con 0x7f86940d1d00 2024-09-21T03:43:19.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:19.504+0000 7f868a7fc640 1 -- 172.21.15.38:0/2345277759 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 272 v272) ==== 95+0+4401 (secure 0 0 0) 0x7f8684031770 con 0x7f86940d1d00 2024-09-21T03:43:19.506 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:43:19.506 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":272,"btime":"2024-09-21T03:43:18:717827+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6086,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/965972663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":965972663},{"type":"v1","addr":"172.21.15.195:6836","nonce":965972663}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6700,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/3176122334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":3176122334},{"type":"v1","addr":"172.21.15.195:6837","nonce":3176122334}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":272,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:40:02.319282+0000","modified":"2024-09-21T03:43:18.717809+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":127,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9786},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9786":{"gid":9786,"name":"c","rank":0,"incarnation":272,"state":"up:replay","state_seq":1,"addr":"172.21.15.38:6833/2309772457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":2309772457},{"type":"v1","addr":"172.21.15.38:6833","nonce":2309772457}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-21T03:43:19.506 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 272 2024-09-21T03:43:19.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:19.506+0000 7f869c18b640 1 -- 172.21.15.38:0/2345277759 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8678078b90 msgr2=0x7f867807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:19.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:19.506+0000 7f869c18b640 1 --2- 172.21.15.38:0/2345277759 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8678078b90 0x7f867807b0a0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f8690009b30 tx=0x7f8690005f90 comp rx=0 tx=0).stop 2024-09-21T03:43:19.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:19.506+0000 7f869c18b640 1 -- 172.21.15.38:0/2345277759 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f86940d1d00 msgr2=0x7f8694154d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:19.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:19.506+0000 7f869c18b640 1 --2- 172.21.15.38:0/2345277759 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f86940d1d00 0x7f8694154d70 secure :-1 s=READY pgs=1999 cs=0 l=1 rev1=1 crypto rx=0x7f868402aaf0 tx=0x7f86840040c0 comp rx=0 tx=0).stop 2024-09-21T03:43:19.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:19.506+0000 7f869c18b640 1 -- 172.21.15.38:0/2345277759 shutdown_connections 2024-09-21T03:43:19.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:19.506+0000 7f869c18b640 1 --2- 172.21.15.38:0/2345277759 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f86941536e0 0x7f86941552b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:19.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:19.506+0000 7f869c18b640 1 --2- 172.21.15.38:0/2345277759 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f86940d1d00 0x7f8694154d70 unknown :-1 s=CLOSED pgs=1999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:19.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:19.506+0000 7f869c18b640 1 --2- 172.21.15.38:0/2345277759 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8678078b90 0x7f867807b0a0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:19.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:19.506+0000 7f869c18b640 1 --2- 172.21.15.38:0/2345277759 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f86940cce70 0x7f8694156570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:19.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:19.506+0000 7f869c18b640 1 -- 172.21.15.38:0/2345277759 >> 172.21.15.38:0/2345277759 conn(0x7f86940cef40 msgr2=0x7f86940d2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:19.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:19.506+0000 7f869c18b640 1 -- 172.21.15.38:0/2345277759 shutdown_connections 2024-09-21T03:43:19.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:19.506+0000 7f869c18b640 1 -- 172.21.15.38:0/2345277759 wait complete. 2024-09-21T03:43:19.519 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 272, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T03:40:02.319282+0000', 'modified': '2024-09-21T03:43:18.717809+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 127, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9786}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9786': {'gid': 9786, 'name': 'c', 'rank': 0, 'incarnation': 272, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.38:6833/2309772457', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.38:6832', 'nonce': 2309772457}, {'type': 'v1', 'addr': '172.21.15.38:6833', 'nonce': 2309772457}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-09-21T03:43:19.519 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9786:up:replay 2024-09-21T03:43:19.519 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2024-09-21T03:43:24.923 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:43:25.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.052+0000 7f3aaa60a640 1 -- 172.21.15.38:0/1512359602 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3aa40cfe00 msgr2=0x7f3aa40d2a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:25.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.052+0000 7f3aaa60a640 1 --2- 172.21.15.38:0/1512359602 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3aa40cfe00 0x7f3aa40d2a20 secure :-1 s=READY pgs=2003 cs=0 l=1 rev1=1 crypto rx=0x7f3a94009830 tx=0x7f3a9402a510 comp rx=0 tx=0).stop 2024-09-21T03:43:25.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.052+0000 7f3aaa60a640 1 -- 172.21.15.38:0/1512359602 shutdown_connections 2024-09-21T03:43:25.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.052+0000 7f3aaa60a640 1 --2- 172.21.15.38:0/1512359602 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3aa4162590 0x7f3aa41629a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:25.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.052+0000 7f3aaa60a640 1 --2- 172.21.15.38:0/1512359602 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3aa40cfe00 0x7f3aa40d2a20 unknown :-1 s=CLOSED pgs=2003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:25.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.052+0000 7f3aaa60a640 1 -- 172.21.15.38:0/1512359602 >> 172.21.15.38:0/1512359602 conn(0x7f3aa40d4440 msgr2=0x7f3aa40d4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:25.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.053+0000 7f3aaa60a640 1 -- 172.21.15.38:0/1512359602 shutdown_connections 2024-09-21T03:43:25.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.053+0000 7f3aaa60a640 1 -- 172.21.15.38:0/1512359602 wait complete. 2024-09-21T03:43:25.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.054+0000 7f3aaa60a640 1 Processor -- start 2024-09-21T03:43:25.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.055+0000 7f3aaa60a640 1 -- start start 2024-09-21T03:43:25.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.056+0000 7f3aaa60a640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3aa40b2e60 0x7f3aa4159a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:25.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.056+0000 7f3aaa60a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3aa40cfe00 0x7f3aa4151a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:25.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.056+0000 7f3aaa60a640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3aa4162590 0x7f3aa4155df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:25.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.056+0000 7f3aaa60a640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f3aa40b3980 con 0x7f3aa40cfe00 2024-09-21T03:43:25.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.056+0000 7f3aaa60a640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f3aa40cf5a0 con 0x7f3aa4162590 2024-09-21T03:43:25.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.056+0000 7f3aaa60a640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f3aa40b3b00 con 0x7f3aa40b2e60 2024-09-21T03:43:25.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.056+0000 7f3aa37fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3aa40cfe00 0x7f3aa4151a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:25.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.056+0000 7f3aa37fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3aa40cfe00 0x7f3aa4151a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:58362/0 (socket says 172.21.15.38:58362) 2024-09-21T03:43:25.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.056+0000 7f3aa37fe640 1 -- 172.21.15.38:0/1692021687 learned_addr learned my addr 172.21.15.38:0/1692021687 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:43:25.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.056+0000 7f3aa3fff640 1 --2- 172.21.15.38:0/1692021687 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3aa40b2e60 0x7f3aa4159a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:25.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.056+0000 7f3aa8ba7640 1 --2- 172.21.15.38:0/1692021687 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3aa4162590 0x7f3aa4155df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:25.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.056+0000 7f3aa37fe640 1 -- 172.21.15.38:0/1692021687 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3aa40b2e60 msgr2=0x7f3aa4159a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:25.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.056+0000 7f3aa37fe640 1 --2- 172.21.15.38:0/1692021687 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3aa40b2e60 0x7f3aa4159a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:25.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.056+0000 7f3aa37fe640 1 -- 172.21.15.38:0/1692021687 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3aa4162590 msgr2=0x7f3aa4155df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:25.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.056+0000 7f3aa37fe640 1 --2- 172.21.15.38:0/1692021687 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3aa4162590 0x7f3aa4155df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:25.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.056+0000 7f3aa37fe640 1 -- 172.21.15.38:0/1692021687 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3aa4154d50 con 0x7f3aa40cfe00 2024-09-21T03:43:25.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.056+0000 7f3aa3fff640 1 --2- 172.21.15.38:0/1692021687 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3aa40b2e60 0x7f3aa4159a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:43:25.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.057+0000 7f3aa37fe640 1 --2- 172.21.15.38:0/1692021687 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3aa40cfe00 0x7f3aa4151a40 secure :-1 s=READY pgs=2004 cs=0 l=1 rev1=1 crypto rx=0x7f3a9800e7c0 tx=0x7f3a9800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:25.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.057+0000 7f3aa0ff9640 1 -- 172.21.15.38:0/1692021687 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a9800ccc0 con 0x7f3aa40cfe00 2024-09-21T03:43:25.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.057+0000 7f3aa0ff9640 1 -- 172.21.15.38:0/1692021687 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3a98010430 con 0x7f3aa40cfe00 2024-09-21T03:43:25.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.057+0000 7f3aaa60a640 1 -- 172.21.15.38:0/1692021687 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3aa4155030 con 0x7f3aa40cfe00 2024-09-21T03:43:25.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.057+0000 7f3aa0ff9640 1 -- 172.21.15.38:0/1692021687 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a98010860 con 0x7f3aa40cfe00 2024-09-21T03:43:25.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.057+0000 7f3aaa60a640 1 -- 172.21.15.38:0/1692021687 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3aa41555f0 con 0x7f3aa40cfe00 2024-09-21T03:43:25.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.058+0000 7f3aaa60a640 1 -- 172.21.15.38:0/1692021687 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3aa4154ac0 con 0x7f3aa40cfe00 2024-09-21T03:43:25.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.059+0000 7f3aa0ff9640 1 -- 172.21.15.38:0/1692021687 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f3a98010a00 con 0x7f3aa40cfe00 2024-09-21T03:43:25.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.059+0000 7f3aa0ff9640 1 --2- 172.21.15.38:0/1692021687 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3a90078b90 0x7f3a9007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:25.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.060+0000 7f3aa0ff9640 1 -- 172.21.15.38:0/1692021687 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 8970+0+0 (secure 0 0 0) 0x7f3a9809c460 con 0x7f3aa40cfe00 2024-09-21T03:43:25.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.060+0000 7f3aa3fff640 1 --2- 172.21.15.38:0/1692021687 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3a90078b90 0x7f3a9007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:25.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.060+0000 7f3aa3fff640 1 --2- 172.21.15.38:0/1692021687 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3a90078b90 0x7f3a9007b0a0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f3a94034000 tx=0x7f3a94004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:25.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.064+0000 7f3aa0ff9640 1 -- 172.21.15.38:0/1692021687 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f3a98063c20 con 0x7f3aa40cfe00 2024-09-21T03:43:25.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.288+0000 7f3aaa60a640 1 -- 172.21.15.38:0/1692021687 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3aa40d2a20 con 0x7f3aa40cfe00 2024-09-21T03:43:25.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.289+0000 7f3aa0ff9640 1 -- 172.21.15.38:0/1692021687 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 275 v275) ==== 95+0+5234 (secure 0 0 0) 0x7f3a98063520 con 0x7f3aa40cfe00 2024-09-21T03:43:25.291 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:43:25.291 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":275,"btime":"2024-09-21T03:43:21:731479+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6086,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/965972663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":965972663},{"type":"v1","addr":"172.21.15.195:6836","nonce":965972663}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6700,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/3176122334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":3176122334},{"type":"v1","addr":"172.21.15.195:6837","nonce":3176122334}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":9840,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/3801772046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":3801772046},{"type":"v1","addr":"172.21.15.38:6835","nonce":3801772046}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":273}],"filesystems":[{"mdsmap":{"epoch":275,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:40:02.319282+0000","modified":"2024-09-21T03:43:21.731469+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":127,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9786},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9786":{"gid":9786,"name":"c","rank":0,"incarnation":272,"state":"up:active","state_seq":4,"addr":"172.21.15.38:6833/2309772457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":2309772457},{"type":"v1","addr":"172.21.15.38:6833","nonce":2309772457}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9786,"qdb_cluster":[9786]},"id":6}]} 2024-09-21T03:43:25.292 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 275 2024-09-21T03:43:25.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.291+0000 7f3aaa60a640 1 -- 172.21.15.38:0/1692021687 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3a90078b90 msgr2=0x7f3a9007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:25.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.291+0000 7f3aaa60a640 1 --2- 172.21.15.38:0/1692021687 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3a90078b90 0x7f3a9007b0a0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f3a94034000 tx=0x7f3a94004730 comp rx=0 tx=0).stop 2024-09-21T03:43:25.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.291+0000 7f3aaa60a640 1 -- 172.21.15.38:0/1692021687 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3aa40cfe00 msgr2=0x7f3aa4151a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:25.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.291+0000 7f3aaa60a640 1 --2- 172.21.15.38:0/1692021687 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3aa40cfe00 0x7f3aa4151a40 secure :-1 s=READY pgs=2004 cs=0 l=1 rev1=1 crypto rx=0x7f3a9800e7c0 tx=0x7f3a9800ec80 comp rx=0 tx=0).stop 2024-09-21T03:43:25.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.291+0000 7f3aaa60a640 1 -- 172.21.15.38:0/1692021687 shutdown_connections 2024-09-21T03:43:25.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.291+0000 7f3aaa60a640 1 --2- 172.21.15.38:0/1692021687 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3aa4162590 0x7f3aa4155df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:25.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.291+0000 7f3aaa60a640 1 --2- 172.21.15.38:0/1692021687 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3aa40cfe00 0x7f3aa4151a40 unknown :-1 s=CLOSED pgs=2004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:25.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.291+0000 7f3aaa60a640 1 --2- 172.21.15.38:0/1692021687 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3a90078b90 0x7f3a9007b0a0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:25.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.291+0000 7f3aaa60a640 1 --2- 172.21.15.38:0/1692021687 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3aa40b2e60 0x7f3aa4159a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:25.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.291+0000 7f3aaa60a640 1 -- 172.21.15.38:0/1692021687 >> 172.21.15.38:0/1692021687 conn(0x7f3aa40d4440 msgr2=0x7f3aa40d4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:25.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.291+0000 7f3aaa60a640 1 -- 172.21.15.38:0/1692021687 shutdown_connections 2024-09-21T03:43:25.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.291+0000 7f3aaa60a640 1 -- 172.21.15.38:0/1692021687 wait complete. 2024-09-21T03:43:25.305 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 275, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T03:40:02.319282+0000', 'modified': '2024-09-21T03:43:21.731469+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 127, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9786}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9786': {'gid': 9786, 'name': 'c', 'rank': 0, 'incarnation': 272, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.38:6833/2309772457', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.38:6832', 'nonce': 2309772457}, {'type': 'v1', 'addr': '172.21.15.38:6833', 'nonce': 2309772457}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 9786, 'qdb_cluster': [9786]} 2024-09-21T03:43:25.305 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-21T03:43:25.305 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-21T03:43:25.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.420+0000 7f3076ebd640 1 -- 172.21.15.38:0/4110891790 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f30700d1120 msgr2=0x7f30700ceb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:25.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.420+0000 7f3076ebd640 1 --2- 172.21.15.38:0/4110891790 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f30700d1120 0x7f30700ceb90 secure :-1 s=READY pgs=2005 cs=0 l=1 rev1=1 crypto rx=0x7f30700757e0 tx=0x7f306802a980 comp rx=0 tx=0).stop 2024-09-21T03:43:25.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.420+0000 7f3076ebd640 1 -- 172.21.15.38:0/4110891790 shutdown_connections 2024-09-21T03:43:25.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.420+0000 7f3076ebd640 1 --2- 172.21.15.38:0/4110891790 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f30700d1120 0x7f30700ceb90 unknown :-1 s=CLOSED pgs=2005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:25.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.420+0000 7f3076ebd640 1 --2- 172.21.15.38:0/4110891790 >> v2:172.21.15.195:3300/0 conn(0x7f30700b9ad0 0x7f30700d0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:25.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.420+0000 7f3076ebd640 1 --2- 172.21.15.38:0/4110891790 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f307015ee80 0x7f30700d0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:25.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.420+0000 7f3076ebd640 1 -- 172.21.15.38:0/4110891790 >> 172.21.15.38:0/4110891790 conn(0x7f307009d830 msgr2=0x7f30700b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:25.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.420+0000 7f3076ebd640 1 -- 172.21.15.38:0/4110891790 shutdown_connections 2024-09-21T03:43:25.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.420+0000 7f3076ebd640 1 -- 172.21.15.38:0/4110891790 wait complete. 2024-09-21T03:43:25.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.422+0000 7f3076ebd640 1 Processor -- start 2024-09-21T03:43:25.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.422+0000 7f3076ebd640 1 -- start start 2024-09-21T03:43:25.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.423+0000 7f3076ebd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f30700b9ad0 0x7f307009e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:25.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.423+0000 7f3076ebd640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f30700d1120 0x7f307014ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:25.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.423+0000 7f3076ebd640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f307015ee80 0x7f307014f230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:25.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.423+0000 7f3076ebd640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f30700cf460 con 0x7f30700b9ad0 2024-09-21T03:43:25.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.423+0000 7f3076ebd640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f30700cf2e0 con 0x7f30700d1120 2024-09-21T03:43:25.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.423+0000 7f3076ebd640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f30700b6580 con 0x7f307015ee80 2024-09-21T03:43:25.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.424+0000 7f3074c59640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f30700b9ad0 0x7f307009e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:25.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.424+0000 7f3074c59640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f30700b9ad0 0x7f307009e2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:58384/0 (socket says 172.21.15.38:58384) 2024-09-21T03:43:25.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.424+0000 7f3074c59640 1 -- 172.21.15.38:0/2421499009 learned_addr learned my addr 172.21.15.38:0/2421499009 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:43:25.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.424+0000 7f307545a640 1 --2- 172.21.15.38:0/2421499009 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f307015ee80 0x7f307014f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:25.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.424+0000 7f306ffff640 1 --2- 172.21.15.38:0/2421499009 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f30700d1120 0x7f307014ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:25.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.424+0000 7f3074c59640 1 -- 172.21.15.38:0/2421499009 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f307015ee80 msgr2=0x7f307014f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:25.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.424+0000 7f3074c59640 1 --2- 172.21.15.38:0/2421499009 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f307015ee80 0x7f307014f230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:25.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.424+0000 7f3074c59640 1 -- 172.21.15.38:0/2421499009 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f30700d1120 msgr2=0x7f307014ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:25.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.424+0000 7f3074c59640 1 --2- 172.21.15.38:0/2421499009 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f30700d1120 0x7f307014ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:25.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.424+0000 7f3074c59640 1 -- 172.21.15.38:0/2421499009 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30701f8610 con 0x7f30700b9ad0 2024-09-21T03:43:25.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.424+0000 7f306ffff640 1 --2- 172.21.15.38:0/2421499009 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f30700d1120 0x7f307014ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:43:25.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.424+0000 7f3074c59640 1 --2- 172.21.15.38:0/2421499009 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f30700b9ad0 0x7f307009e2a0 secure :-1 s=READY pgs=2006 cs=0 l=1 rev1=1 crypto rx=0x7f306000c9a0 tx=0x7f306000ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:25.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.424+0000 7f306d7fa640 1 -- 172.21.15.38:0/2421499009 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3060007b50 con 0x7f30700b9ad0 2024-09-21T03:43:25.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.424+0000 7f306d7fa640 1 -- 172.21.15.38:0/2421499009 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3060007de0 con 0x7f30700b9ad0 2024-09-21T03:43:25.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.424+0000 7f3076ebd640 1 -- 172.21.15.38:0/2421499009 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f307009e7e0 con 0x7f30700b9ad0 2024-09-21T03:43:25.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.424+0000 7f306d7fa640 1 -- 172.21.15.38:0/2421499009 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f306000f6b0 con 0x7f30700b9ad0 2024-09-21T03:43:25.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.424+0000 7f3076ebd640 1 -- 172.21.15.38:0/2421499009 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f307006ee80 con 0x7f30700b9ad0 2024-09-21T03:43:25.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.426+0000 7f306d7fa640 1 -- 172.21.15.38:0/2421499009 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f306000f850 con 0x7f30700b9ad0 2024-09-21T03:43:25.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.427+0000 7f306d7fa640 1 --2- 172.21.15.38:0/2421499009 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f305c078bc0 0x7f305c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:25.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.427+0000 7f306d7fa640 1 -- 172.21.15.38:0/2421499009 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 8970+0+0 (secure 0 0 0) 0x7f306009c120 con 0x7f30700b9ad0 2024-09-21T03:43:25.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.427+0000 7f306ffff640 1 --2- 172.21.15.38:0/2421499009 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f305c078bc0 0x7f305c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:25.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.427+0000 7f307de29740 1 Processor -- start 2024-09-21T03:43:25.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.427+0000 7f306ffff640 1 --2- 172.21.15.38:0/2421499009 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f305c078bc0 0x7f305c07b0d0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f307006fde0 tx=0x7f3064005d90 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:25.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.428+0000 7f307de29740 1 -- start start 2024-09-21T03:43:25.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.429+0000 7f307de29740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555afbd65270 0x555afc136fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:25.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.429+0000 7f307de29740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555afbd3c3b0 0x555afc1374e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:25.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.429+0000 7f307de29740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555afbd44590 0x555afc1543e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:25.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.429+0000 7f307de29740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x555afc07f820 con 0x555afbd44590 2024-09-21T03:43:25.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.429+0000 7f307de29740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x555afc06ec50 con 0x555afbd3c3b0 2024-09-21T03:43:25.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.429+0000 7f307de29740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x555afc072a60 con 0x555afbd65270 2024-09-21T03:43:25.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.429+0000 7f307545a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555afbd44590 0x555afc1543e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:25.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.429+0000 7f307545a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555afbd44590 0x555afc1543e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:58390/0 (socket says 172.21.15.38:58390) 2024-09-21T03:43:25.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.429+0000 7f307545a640 1 -- 172.21.15.38:0/670715538 learned_addr learned my addr 172.21.15.38:0/670715538 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:43:25.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.429+0000 7f306ffff640 1 --2- 172.21.15.38:0/670715538 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555afbd3c3b0 0x555afc1374e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:25.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.429+0000 7f3074c59640 1 --2- 172.21.15.38:0/670715538 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555afbd65270 0x555afc136fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:25.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.429+0000 7f307545a640 1 -- 172.21.15.38:0/670715538 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555afbd65270 msgr2=0x555afc136fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:25.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.429+0000 7f307545a640 1 --2- 172.21.15.38:0/670715538 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555afbd65270 0x555afc136fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:25.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.429+0000 7f307545a640 1 -- 172.21.15.38:0/670715538 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555afbd3c3b0 msgr2=0x555afc1374e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:25.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.429+0000 7f307545a640 1 --2- 172.21.15.38:0/670715538 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555afbd3c3b0 0x555afc1374e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:25.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.429+0000 7f307545a640 1 -- 172.21.15.38:0/670715538 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555afbdd0850 con 0x555afbd44590 2024-09-21T03:43:25.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.430+0000 7f307545a640 1 --2- 172.21.15.38:0/670715538 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555afbd44590 0x555afc1543e0 secure :-1 s=READY pgs=2007 cs=0 l=1 rev1=1 crypto rx=0x7f3068009d60 tx=0x7f3068004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e3004502c1785ec server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:25.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.430+0000 7f30567fc640 1 -- 172.21.15.38:0/670715538 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3068048070 con 0x555afbd44590 2024-09-21T03:43:25.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.430+0000 7f30567fc640 1 -- 172.21.15.38:0/670715538 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3068030b30 con 0x555afbd44590 2024-09-21T03:43:25.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.430+0000 7f30567fc640 1 -- 172.21.15.38:0/670715538 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3068034040 con 0x555afbd44590 2024-09-21T03:43:25.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.430+0000 7f307de29740 1 -- 172.21.15.38:0/670715538 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555afbd44590 msgr2=0x555afc1543e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:25.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.430+0000 7f307de29740 1 --2- 172.21.15.38:0/670715538 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555afbd44590 0x555afc1543e0 secure :-1 s=READY pgs=2007 cs=0 l=1 rev1=1 crypto rx=0x7f3068009d60 tx=0x7f3068004a40 comp rx=0 tx=0).stop 2024-09-21T03:43:25.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.431+0000 7f307de29740 1 -- 172.21.15.38:0/670715538 shutdown_connections 2024-09-21T03:43:25.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.431+0000 7f307de29740 1 --2- 172.21.15.38:0/670715538 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555afbd44590 0x555afc1543e0 unknown :-1 s=CLOSED pgs=2007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:25.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.431+0000 7f307de29740 1 --2- 172.21.15.38:0/670715538 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555afbd3c3b0 0x555afc1374e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:25.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.431+0000 7f307de29740 1 --2- 172.21.15.38:0/670715538 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555afbd65270 0x555afc136fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:25.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.431+0000 7f307de29740 1 -- 172.21.15.38:0/670715538 >> 172.21.15.38:0/670715538 conn(0x555afbd593c0 msgr2=0x555afc10d260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:25.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.431+0000 7f307de29740 1 -- 172.21.15.38:0/670715538 shutdown_connections 2024-09-21T03:43:25.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.431+0000 7f307de29740 1 -- 172.21.15.38:0/670715538 wait complete. 2024-09-21T03:43:25.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.431+0000 7f307de29740 1 -- start start 2024-09-21T03:43:25.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.432+0000 7f307de29740 1 Processor -- start 2024-09-21T03:43:25.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.433+0000 7f307de29740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555afbd3c3b0 0x555afc14d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:25.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.433+0000 7f307de29740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555afbd44590 0x555afc14e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:25.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.433+0000 7f307de29740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555afbd65270 0x555afc14a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:25.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.433+0000 7f307de29740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x555afbddf620 con 0x555afbd44590 2024-09-21T03:43:25.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.433+0000 7f307de29740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x555afc0145f0 con 0x555afbd3c3b0 2024-09-21T03:43:25.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.433+0000 7f307de29740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x555afc137a20 con 0x555afbd65270 2024-09-21T03:43:25.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.433+0000 7f307de29740 10 client.0 _ll_register_callbacks cb 0x555afc0180a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:43:25.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.433+0000 7f306ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555afbd44590 0x555afc14e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:25.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.433+0000 7f306ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555afbd44590 0x555afc14e7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:58396/0 (socket says 172.21.15.38:58396) 2024-09-21T03:43:25.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.433+0000 7f306ffff640 1 -- 172.21.15.38:0/3512535567 learned_addr learned my addr 172.21.15.38:0/3512535567 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:43:25.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.433+0000 7f307545a640 1 --2- 172.21.15.38:0/3512535567 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555afbd65270 0x555afc14a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:25.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.433+0000 7f306ffff640 1 -- 172.21.15.38:0/3512535567 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555afbd65270 msgr2=0x555afc14a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:25.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.433+0000 7f306ffff640 1 --2- 172.21.15.38:0/3512535567 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555afbd65270 0x555afc14a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:25.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.433+0000 7f306ffff640 1 -- 172.21.15.38:0/3512535567 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555afbd3c3b0 msgr2=0x555afc14d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:25.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.433+0000 7f306ffff640 1 --2- 172.21.15.38:0/3512535567 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555afbd3c3b0 0x555afc14d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:25.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.433+0000 7f306ffff640 1 -- 172.21.15.38:0/3512535567 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555afbe476b0 con 0x555afbd44590 2024-09-21T03:43:25.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.434+0000 7f306ffff640 1 --2- 172.21.15.38:0/3512535567 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555afbd44590 0x555afc14e7b0 secure :-1 s=READY pgs=2008 cs=0 l=1 rev1=1 crypto rx=0x7f3064010a10 tx=0x7f30640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e19126de7adcd154 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:25.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.434+0000 7f304f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-21T03:43:25.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.434+0000 7f304f7fe640 1 -- 172.21.15.38:0/3512535567 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3064005500 con 0x555afbd44590 2024-09-21T03:43:25.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.434+0000 7f304f7fe640 1 -- 172.21.15.38:0/3512535567 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(127..127 src has 1..127) ==== 8970+0+0 (secure 0 0 0) 0x7f30640047a0 con 0x555afbd44590 2024-09-21T03:43:25.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.434+0000 7f307de29740 1 -- 172.21.15.38:0/3512535567 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3068030b30 con 0x555afbd44590 2024-09-21T03:43:25.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.434+0000 7f304f7fe640 1 -- 172.21.15.38:0/3512535567 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=275) ==== 24+0+0 (secure 0 0 0) 0x7f30640049a0 con 0x555afbd44590 2024-09-21T03:43:25.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.434+0000 7f307de29740 10 client.9861 fetch_fsmap learned FSMap version 275 2024-09-21T03:43:25.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.434+0000 7f307de29740 1 -- 172.21.15.38:0/3512535567 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=275,monmap=2+}) -- 0x555afc14ecf0 con 0x555afbd44590 2024-09-21T03:43:25.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.434+0000 7f304f7fe640 1 -- 172.21.15.38:0/3512535567 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3068030b30 con 0x555afbd44590 2024-09-21T03:43:25.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.434+0000 7f304f7fe640 1 -- 172.21.15.38:0/3512535567 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 275) ==== 3502+0+0 (secure 0 0 0) 0x7f3064004b40 con 0x555afbd44590 2024-09-21T03:43:25.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.434+0000 7f307de29740 10 client.9861 fetch_fsmap finished waiting for FSMap version 275 2024-09-21T03:43:25.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.434+0000 7f307de29740 10 client.9861 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T03:43:25.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.434+0000 7f307de29740 20 client.9861 populate_metadata read hostname 'smithi038' 2024-09-21T03:43:25.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.434+0000 7f307de29740 1 --2- 172.21.15.38:0/3512535567 >> [v2:172.21.15.38:6832/2309772457,v1:172.21.15.38:6833/2309772457] conn(0x555afbd23e40 0x555afc124fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:25.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.434+0000 7f307de29740 4 client.9861 mds_command: new command op to 9786 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:43:25.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.434+0000 7f307de29740 1 -- 172.21.15.38:0/3512535567 --> [v2:172.21.15.38:6832/2309772457,v1:172.21.15.38:6833/2309772457] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555afc081770 con 0x555afbd23e40 2024-09-21T03:43:25.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.435+0000 7f3074c59640 1 --2- 172.21.15.38:0/3512535567 >> [v2:172.21.15.38:6832/2309772457,v1:172.21.15.38:6833/2309772457] conn(0x555afbd23e40 0x555afc124fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:25.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.435+0000 7f304f7fe640 10 client.9861 ms_handle_connect on v2:172.21.15.38:6832/2309772457 2024-09-21T03:43:25.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.435+0000 7f3074c59640 1 --2- 172.21.15.38:0/3512535567 >> [v2:172.21.15.38:6832/2309772457,v1:172.21.15.38:6833/2309772457] conn(0x555afbd23e40 0x555afc124fc0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f4352e34ce0846f4 server_cookie=6053f5f5cf236484 in_seq=0 out_seq=0 2024-09-21T03:43:25.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.436+0000 7f304f7fe640 1 -- 172.21.15.38:0/3512535567 <== mds.0 v2:172.21.15.38:6832/2309772457 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x555afc081770 con 0x555afbd23e40 2024-09-21T03:43:25.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.436+0000 7f304f7fe640 10 client.9861 handle_command_reply: tid=0 2024-09-21T03:43:25.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.436+0000 7f307de29740 1 client.9861 shutdown 2024-09-21T03:43:25.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.436+0000 7f307de29740 1 -- 172.21.15.38:0/3512535567 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555afbd44590 msgr2=0x555afc14e7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:25.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.436+0000 7f307de29740 1 --2- 172.21.15.38:0/3512535567 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555afbd44590 0x555afc14e7b0 secure :-1 s=READY pgs=2008 cs=0 l=1 rev1=1 crypto rx=0x7f3064010a10 tx=0x7f30640046c0 comp rx=0 tx=0).stop 2024-09-21T03:43:25.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.436+0000 7f307de29740 1 -- 172.21.15.38:0/3512535567 shutdown_connections 2024-09-21T03:43:25.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.436+0000 7f307de29740 1 --2- 172.21.15.38:0/3512535567 >> [v2:172.21.15.38:6832/2309772457,v1:172.21.15.38:6833/2309772457] conn(0x555afbd23e40 0x555afc124fc0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:25.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.436+0000 7f307de29740 1 --2- 172.21.15.38:0/3512535567 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555afbd65270 0x555afc14a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:25.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.436+0000 7f307de29740 1 --2- 172.21.15.38:0/3512535567 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555afbd44590 0x555afc14e7b0 unknown :-1 s=CLOSED pgs=2008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:25.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.436+0000 7f304f7fe640 0 client.9861 ms_handle_reset on v2:172.21.15.38:6832/2309772457 2024-09-21T03:43:25.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.436+0000 7f307de29740 1 --2- 172.21.15.38:0/3512535567 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555afbd3c3b0 0x555afc14d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:25.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.436+0000 7f307de29740 1 -- 172.21.15.38:0/3512535567 >> 172.21.15.38:0/3512535567 conn(0x555afbd593c0 msgr2=0x555afc1078f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:25.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.436+0000 7f307de29740 1 -- 172.21.15.38:0/3512535567 shutdown_connections 2024-09-21T03:43:25.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.436+0000 7f307de29740 1 -- 172.21.15.38:0/3512535567 wait complete. 2024-09-21T03:43:25.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.436+0000 7f307de29740 20 client.9861 trim_cache size 0 max 16384 2024-09-21T03:43:25.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.455+0000 7f307de29740 1 Processor -- start 2024-09-21T03:43:25.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.455+0000 7f307de29740 1 -- start start 2024-09-21T03:43:25.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.456+0000 7f307de29740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555afbd593c0 0x555afc176010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:25.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.456+0000 7f307de29740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555afbd3c3b0 0x555afc14ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:25.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.456+0000 7f307de29740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555afbd44590 0x555afc14b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:25.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.456+0000 7f307de29740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f3070203310 con 0x555afbd593c0 2024-09-21T03:43:25.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.456+0000 7f307de29740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f3070202610 con 0x555afbd44590 2024-09-21T03:43:25.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.456+0000 7f307de29740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f3070202c60 con 0x555afbd3c3b0 2024-09-21T03:43:25.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.456+0000 7f3074c59640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555afbd593c0 0x555afc176010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:25.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.456+0000 7f3074c59640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555afbd593c0 0x555afc176010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:58410/0 (socket says 172.21.15.38:58410) 2024-09-21T03:43:25.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.456+0000 7f3074c59640 1 -- 172.21.15.38:0/576589708 learned_addr learned my addr 172.21.15.38:0/576589708 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:43:25.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.456+0000 7f306ffff640 1 --2- 172.21.15.38:0/576589708 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555afbd3c3b0 0x555afc14ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:25.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.456+0000 7f307545a640 1 --2- 172.21.15.38:0/576589708 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555afbd44590 0x555afc14b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:25.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.457+0000 7f3074c59640 1 -- 172.21.15.38:0/576589708 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555afbd3c3b0 msgr2=0x555afc14ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:25.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.457+0000 7f3074c59640 1 --2- 172.21.15.38:0/576589708 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555afbd3c3b0 0x555afc14ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:25.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.457+0000 7f3074c59640 1 -- 172.21.15.38:0/576589708 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555afbd44590 msgr2=0x555afc14b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:25.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.457+0000 7f3074c59640 1 --2- 172.21.15.38:0/576589708 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555afbd44590 0x555afc14b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:25.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.457+0000 7f3074c59640 1 -- 172.21.15.38:0/576589708 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555afc137740 con 0x555afbd593c0 2024-09-21T03:43:25.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.457+0000 7f3074c59640 1 --2- 172.21.15.38:0/576589708 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555afbd593c0 0x555afc176010 secure :-1 s=READY pgs=2009 cs=0 l=1 rev1=1 crypto rx=0x7f3060062e50 tx=0x7f3060085040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3c1a4b12d438773 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:25.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.457+0000 7f3055ffb640 1 -- 172.21.15.38:0/576589708 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f306006cb70 con 0x555afbd593c0 2024-09-21T03:43:25.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.457+0000 7f3055ffb640 1 -- 172.21.15.38:0/576589708 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f306005f3c0 con 0x555afbd593c0 2024-09-21T03:43:25.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.457+0000 7f3055ffb640 1 -- 172.21.15.38:0/576589708 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f306000fda0 con 0x555afbd593c0 2024-09-21T03:43:25.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.458+0000 7f307de29740 1 -- 172.21.15.38:0/576589708 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555afbd593c0 msgr2=0x555afc176010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:25.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.458+0000 7f307de29740 1 --2- 172.21.15.38:0/576589708 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555afbd593c0 0x555afc176010 secure :-1 s=READY pgs=2009 cs=0 l=1 rev1=1 crypto rx=0x7f3060062e50 tx=0x7f3060085040 comp rx=0 tx=0).stop 2024-09-21T03:43:25.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.458+0000 7f307de29740 1 -- 172.21.15.38:0/576589708 shutdown_connections 2024-09-21T03:43:25.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.458+0000 7f307de29740 1 --2- 172.21.15.38:0/576589708 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555afbd44590 0x555afc14b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:25.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.458+0000 7f307de29740 1 --2- 172.21.15.38:0/576589708 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555afbd3c3b0 0x555afc14ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:25.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.458+0000 7f307de29740 1 --2- 172.21.15.38:0/576589708 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555afbd593c0 0x555afc176010 unknown :-1 s=CLOSED pgs=2009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:25.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.458+0000 7f307de29740 1 -- 172.21.15.38:0/576589708 >> 172.21.15.38:0/576589708 conn(0x555afc126ad0 msgr2=0x555afc14e480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:25.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.458+0000 7f307de29740 1 -- 172.21.15.38:0/576589708 shutdown_connections 2024-09-21T03:43:25.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.458+0000 7f307de29740 1 -- 172.21.15.38:0/576589708 wait complete. 2024-09-21T03:43:25.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.459+0000 7f307de29740 1 -- start start 2024-09-21T03:43:25.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.459+0000 7f307de29740 1 Processor -- start 2024-09-21T03:43:25.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.460+0000 7f307de29740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555afbd3c3b0 0x555afc106e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:25.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.460+0000 7f307de29740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555afbd44590 0x555afc106200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:25.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.460+0000 7f307de29740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555afbd593c0 0x555afc1d1740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:25.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.460+0000 7f307de29740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x555afc14a800 con 0x555afbd3c3b0 2024-09-21T03:43:25.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.460+0000 7f307de29740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x555afc149830 con 0x555afbd44590 2024-09-21T03:43:25.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.460+0000 7f307de29740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x555afc164af0 con 0x555afbd593c0 2024-09-21T03:43:25.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.460+0000 7f307de29740 10 client.0 _ll_register_callbacks cb 0x7f3064010660 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:43:25.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.460+0000 7f3074c59640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555afbd3c3b0 0x555afc106e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:25.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.460+0000 7f3074c59640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555afbd3c3b0 0x555afc106e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:58420/0 (socket says 172.21.15.38:58420) 2024-09-21T03:43:25.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.460+0000 7f3074c59640 1 -- 172.21.15.38:0/1862851858 learned_addr learned my addr 172.21.15.38:0/1862851858 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:43:25.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.461+0000 7f306ffff640 1 --2- 172.21.15.38:0/1862851858 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555afbd44590 0x555afc106200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:25.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.461+0000 7f307545a640 1 --2- 172.21.15.38:0/1862851858 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555afbd593c0 0x555afc1d1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:25.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.461+0000 7f3074c59640 1 -- 172.21.15.38:0/1862851858 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555afbd593c0 msgr2=0x555afc1d1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:25.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.461+0000 7f3074c59640 1 --2- 172.21.15.38:0/1862851858 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555afbd593c0 0x555afc1d1740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:25.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.461+0000 7f3074c59640 1 -- 172.21.15.38:0/1862851858 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555afbd44590 msgr2=0x555afc106200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:25.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.461+0000 7f3074c59640 1 --2- 172.21.15.38:0/1862851858 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555afbd44590 0x555afc106200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:25.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.461+0000 7f3074c59640 1 -- 172.21.15.38:0/1862851858 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555afc107990 con 0x555afbd3c3b0 2024-09-21T03:43:25.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.461+0000 7f307545a640 1 --2- 172.21.15.38:0/1862851858 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555afbd593c0 0x555afc1d1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:43:25.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.461+0000 7f306ffff640 1 --2- 172.21.15.38:0/1862851858 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555afbd44590 0x555afc106200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:43:25.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.461+0000 7f3074c59640 1 --2- 172.21.15.38:0/1862851858 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555afbd3c3b0 0x555afc106e50 secure :-1 s=READY pgs=2010 cs=0 l=1 rev1=1 crypto rx=0x7f30600630e0 tx=0x7f306005f8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84ac81df2054fb18 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:25.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.461+0000 7f304f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-21T03:43:25.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.461+0000 7f304f7fe640 1 -- 172.21.15.38:0/1862851858 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3060062c60 con 0x555afbd3c3b0 2024-09-21T03:43:25.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.461+0000 7f304f7fe640 1 -- 172.21.15.38:0/1862851858 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(127..127 src has 1..127) ==== 8970+0+0 (secure 0 0 0) 0x7f30600623b0 con 0x555afbd3c3b0 2024-09-21T03:43:25.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.461+0000 7f307de29740 1 -- 172.21.15.38:0/1862851858 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f306005f3c0 con 0x555afbd3c3b0 2024-09-21T03:43:25.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.461+0000 7f304f7fe640 1 -- 172.21.15.38:0/1862851858 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=275) ==== 24+0+0 (secure 0 0 0) 0x7f306006cb70 con 0x555afbd3c3b0 2024-09-21T03:43:25.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.461+0000 7f307de29740 10 client.9867 fetch_fsmap learned FSMap version 275 2024-09-21T03:43:25.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.461+0000 7f307de29740 1 -- 172.21.15.38:0/1862851858 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=275,monmap=2+}) -- 0x555afc125f50 con 0x555afbd3c3b0 2024-09-21T03:43:25.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.461+0000 7f304f7fe640 1 -- 172.21.15.38:0/1862851858 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f306005f3c0 con 0x555afbd3c3b0 2024-09-21T03:43:25.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.461+0000 7f304f7fe640 1 -- 172.21.15.38:0/1862851858 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 275) ==== 3502+0+0 (secure 0 0 0) 0x7f3060020030 con 0x555afbd3c3b0 2024-09-21T03:43:25.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.461+0000 7f307de29740 10 client.9867 fetch_fsmap finished waiting for FSMap version 275 2024-09-21T03:43:25.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.461+0000 7f307de29740 10 client.9867 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T03:43:25.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.461+0000 7f307de29740 20 client.9867 populate_metadata read hostname 'smithi038' 2024-09-21T03:43:25.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.462+0000 7f307de29740 1 --2- 172.21.15.38:0/1862851858 >> [v2:172.21.15.38:6832/2309772457,v1:172.21.15.38:6833/2309772457] conn(0x555afbcf9280 0x555afc14b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:25.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.462+0000 7f307de29740 4 client.9867 mds_command: new command op to 9786 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T03:43:25.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.462+0000 7f307de29740 1 -- 172.21.15.38:0/1862851858 --> [v2:172.21.15.38:6832/2309772457,v1:172.21.15.38:6833/2309772457] -- command(tid 0: {"prefix": "status"}) -- 0x555afc07d220 con 0x555afbcf9280 2024-09-21T03:43:25.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.462+0000 7f306ffff640 1 --2- 172.21.15.38:0/1862851858 >> [v2:172.21.15.38:6832/2309772457,v1:172.21.15.38:6833/2309772457] conn(0x555afbcf9280 0x555afc14b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:25.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.462+0000 7f304f7fe640 10 client.9867 ms_handle_connect on v2:172.21.15.38:6832/2309772457 2024-09-21T03:43:25.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.462+0000 7f306ffff640 1 --2- 172.21.15.38:0/1862851858 >> [v2:172.21.15.38:6832/2309772457,v1:172.21.15.38:6833/2309772457] conn(0x555afbcf9280 0x555afc14b230 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b197b289f638fd3f server_cookie=ad7f51cacdab44d6 in_seq=0 out_seq=0 2024-09-21T03:43:25.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.462+0000 7f304f7fe640 1 -- 172.21.15.38:0/1862851858 <== mds.0 v2:172.21.15.38:6832/2309772457 1 ==== command_reply(tid 0: 0 ) ==== 8+0+334 (crc 0 0 0) 0x555afc07d220 con 0x555afbcf9280 2024-09-21T03:43:25.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.462+0000 7f304f7fe640 10 client.9867 handle_command_reply: tid=0 2024-09-21T03:43:25.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.462+0000 7f307de29740 1 client.9867 shutdown 2024-09-21T03:43:25.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.462+0000 7f307de29740 1 -- 172.21.15.38:0/1862851858 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555afbd3c3b0 msgr2=0x555afc106e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:25.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.462+0000 7f307de29740 1 --2- 172.21.15.38:0/1862851858 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555afbd3c3b0 0x555afc106e50 secure :-1 s=READY pgs=2010 cs=0 l=1 rev1=1 crypto rx=0x7f30600630e0 tx=0x7f306005f8d0 comp rx=0 tx=0).stop 2024-09-21T03:43:25.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.463+0000 7f307de29740 1 -- 172.21.15.38:0/1862851858 shutdown_connections 2024-09-21T03:43:25.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.463+0000 7f307de29740 1 --2- 172.21.15.38:0/1862851858 >> [v2:172.21.15.38:6832/2309772457,v1:172.21.15.38:6833/2309772457] conn(0x555afbcf9280 0x555afc14b230 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:25.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.463+0000 7f307de29740 1 --2- 172.21.15.38:0/1862851858 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555afbd593c0 0x555afc1d1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:25.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.463+0000 7f307de29740 1 --2- 172.21.15.38:0/1862851858 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555afbd44590 0x555afc106200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:25.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.463+0000 7f304f7fe640 0 client.9867 ms_handle_reset on v2:172.21.15.38:6832/2309772457 2024-09-21T03:43:25.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.463+0000 7f307de29740 1 --2- 172.21.15.38:0/1862851858 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555afbd3c3b0 0x555afc106e50 unknown :-1 s=CLOSED pgs=2010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:25.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.463+0000 7f307de29740 1 -- 172.21.15.38:0/1862851858 >> 172.21.15.38:0/1862851858 conn(0x555afc126ad0 msgr2=0x555afc160810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:25.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.463+0000 7f307de29740 1 -- 172.21.15.38:0/1862851858 shutdown_connections 2024-09-21T03:43:25.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.463+0000 7f307de29740 1 -- 172.21.15.38:0/1862851858 wait complete. 2024-09-21T03:43:25.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.463+0000 7f307de29740 20 client.9867 trim_cache size 0 max 16384 2024-09-21T03:43:25.471 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-09-21T03:43:25.471 INFO:teuthology.orchestra.run.smithi038.stdout: "cluster_fsid": "9d50d66f-656b-4fe2-8e3b-fe53b79c4d43", 2024-09-21T03:43:25.471 INFO:teuthology.orchestra.run.smithi038.stdout: "whoami": 0, 2024-09-21T03:43:25.471 INFO:teuthology.orchestra.run.smithi038.stdout: "id": 9786, 2024-09-21T03:43:25.471 INFO:teuthology.orchestra.run.smithi038.stdout: "want_state": "up:active", 2024-09-21T03:43:25.471 INFO:teuthology.orchestra.run.smithi038.stdout: "state": "up:active", 2024-09-21T03:43:25.471 INFO:teuthology.orchestra.run.smithi038.stdout: "fs_name": "cephfs", 2024-09-21T03:43:25.471 INFO:teuthology.orchestra.run.smithi038.stdout: "rank_uptime": 6.7425895840000001, 2024-09-21T03:43:25.471 INFO:teuthology.orchestra.run.smithi038.stdout: "mdsmap_epoch": 275, 2024-09-21T03:43:25.472 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch": 127, 2024-09-21T03:43:25.472 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch_barrier": 127, 2024-09-21T03:43:25.472 INFO:teuthology.orchestra.run.smithi038.stdout: "uptime": 10.085793044000001 2024-09-21T03:43:25.472 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-09-21T03:43:25.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.463+0000 7f3076ebd640 1 -- 172.21.15.38:0/2421499009 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f305c078bc0 msgr2=0x7f305c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:25.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.463+0000 7f3076ebd640 1 --2- 172.21.15.38:0/2421499009 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f305c078bc0 0x7f305c07b0d0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f307006fde0 tx=0x7f3064005d90 comp rx=0 tx=0).stop 2024-09-21T03:43:25.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.463+0000 7f3076ebd640 1 -- 172.21.15.38:0/2421499009 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f30700b9ad0 msgr2=0x7f307009e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:25.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.463+0000 7f3076ebd640 1 --2- 172.21.15.38:0/2421499009 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f30700b9ad0 0x7f307009e2a0 secure :-1 s=READY pgs=2006 cs=0 l=1 rev1=1 crypto rx=0x7f306000c9a0 tx=0x7f306000ce60 comp rx=0 tx=0).stop 2024-09-21T03:43:25.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.463+0000 7f3076ebd640 1 -- 172.21.15.38:0/2421499009 shutdown_connections 2024-09-21T03:43:25.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.463+0000 7f3076ebd640 1 --2- 172.21.15.38:0/2421499009 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f305c078bc0 0x7f305c07b0d0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:25.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.463+0000 7f3076ebd640 1 --2- 172.21.15.38:0/2421499009 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f307015ee80 0x7f307014f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:25.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.463+0000 7f3076ebd640 1 --2- 172.21.15.38:0/2421499009 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f30700d1120 0x7f307014ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:25.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.463+0000 7f3076ebd640 1 --2- 172.21.15.38:0/2421499009 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f30700b9ad0 0x7f307009e2a0 unknown :-1 s=CLOSED pgs=2006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:25.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.463+0000 7f3076ebd640 1 -- 172.21.15.38:0/2421499009 >> 172.21.15.38:0/2421499009 conn(0x7f307009d830 msgr2=0x7f30700cac80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:25.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.463+0000 7f3076ebd640 1 -- 172.21.15.38:0/2421499009 shutdown_connections 2024-09-21T03:43:25.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.463+0000 7f3076ebd640 1 -- 172.21.15.38:0/2421499009 wait complete. 2024-09-21T03:43:25.476 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-09-21T03:43:25.476 INFO:tasks.cephfs.test_damage:Daemons came up after mutation 'Corrupt omap key 10000000000.00000000:sixmegs_head', proceeding to ls 2024-09-21T03:43:25.477 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T03:43:25.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.591+0000 7fc9435ba640 1 -- 172.21.15.38:0/3944842982 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc93c0cfe00 msgr2=0x7fc93c0d2a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:25.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.591+0000 7fc9435ba640 1 --2- 172.21.15.38:0/3944842982 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc93c0cfe00 0x7fc93c0d2a20 secure :-1 s=READY pgs=2012 cs=0 l=1 rev1=1 crypto rx=0x7fc93c0b5a40 tx=0x7fc93802a530 comp rx=0 tx=0).stop 2024-09-21T03:43:25.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.591+0000 7fc9435ba640 1 -- 172.21.15.38:0/3944842982 shutdown_connections 2024-09-21T03:43:25.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.591+0000 7fc9435ba640 1 --2- 172.21.15.38:0/3944842982 >> v2:172.21.15.195:3300/0 conn(0x7fc93c0b2e60 0x7fc93c0b3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:25.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.591+0000 7fc9435ba640 1 --2- 172.21.15.38:0/3944842982 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc93c0cfe00 0x7fc93c0d2a20 unknown :-1 s=CLOSED pgs=2012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:25.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.591+0000 7fc9435ba640 1 -- 172.21.15.38:0/3944842982 >> 172.21.15.38:0/3944842982 conn(0x7fc93c0d4440 msgr2=0x7fc93c0d4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:25.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.591+0000 7fc9435ba640 1 -- 172.21.15.38:0/3944842982 shutdown_connections 2024-09-21T03:43:25.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.591+0000 7fc9435ba640 1 -- 172.21.15.38:0/3944842982 wait complete. 2024-09-21T03:43:25.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.593+0000 7fc9435ba640 1 Processor -- start 2024-09-21T03:43:25.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.593+0000 7fc9435ba640 1 -- start start 2024-09-21T03:43:25.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.594+0000 7fc9435ba640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc93c0b2e60 0x7fc93c06db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:25.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.594+0000 7fc9435ba640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc93c0cfe00 0x7fc93c1f52a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:25.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.594+0000 7fc9435ba640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc93c162590 0x7fc93c1f57e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:25.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.594+0000 7fc9435ba640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fc93c0b3930 con 0x7fc93c0cfe00 2024-09-21T03:43:25.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.594+0000 7fc9435ba640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fc93c0b37b0 con 0x7fc93c162590 2024-09-21T03:43:25.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.594+0000 7fc9435ba640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fc93c0b3ab0 con 0x7fc93c0b2e60 2024-09-21T03:43:25.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.595+0000 7fc940b55640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc93c0cfe00 0x7fc93c1f52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:25.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.595+0000 7fc940b55640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc93c0cfe00 0x7fc93c1f52a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:58436/0 (socket says 172.21.15.38:58436) 2024-09-21T03:43:25.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.595+0000 7fc940b55640 1 -- 172.21.15.38:0/190794971 learned_addr learned my addr 172.21.15.38:0/190794971 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:43:25.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.595+0000 7fc941356640 1 --2- 172.21.15.38:0/190794971 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc93c0b2e60 0x7fc93c06db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:25.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.595+0000 7fc941b57640 1 --2- 172.21.15.38:0/190794971 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc93c162590 0x7fc93c1f57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:25.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.595+0000 7fc940b55640 1 -- 172.21.15.38:0/190794971 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc93c0b2e60 msgr2=0x7fc93c06db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:25.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.595+0000 7fc940b55640 1 --2- 172.21.15.38:0/190794971 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc93c0b2e60 0x7fc93c06db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:25.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.595+0000 7fc940b55640 1 -- 172.21.15.38:0/190794971 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc93c162590 msgr2=0x7fc93c1f57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:25.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.595+0000 7fc940b55640 1 --2- 172.21.15.38:0/190794971 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc93c162590 0x7fc93c1f57e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:25.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.595+0000 7fc940b55640 1 -- 172.21.15.38:0/190794971 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc93c1f60c0 con 0x7fc93c0cfe00 2024-09-21T03:43:25.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.595+0000 7fc940b55640 1 --2- 172.21.15.38:0/190794971 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc93c0cfe00 0x7fc93c1f52a0 secure :-1 s=READY pgs=2013 cs=0 l=1 rev1=1 crypto rx=0x7fc92c00c370 tx=0x7fc92c00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:25.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.595+0000 7fc931ffb640 1 -- 172.21.15.38:0/190794971 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc92c017070 con 0x7fc93c0cfe00 2024-09-21T03:43:25.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.595+0000 7fc931ffb640 1 -- 172.21.15.38:0/190794971 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc92c0035f0 con 0x7fc93c0cfe00 2024-09-21T03:43:25.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.595+0000 7fc931ffb640 1 -- 172.21.15.38:0/190794971 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc92c003a40 con 0x7fc93c0cfe00 2024-09-21T03:43:25.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.595+0000 7fc9435ba640 1 -- 172.21.15.38:0/190794971 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc93c147d30 con 0x7fc93c0cfe00 2024-09-21T03:43:25.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.596+0000 7fc9435ba640 1 -- 172.21.15.38:0/190794971 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc93c146720 con 0x7fc93c0cfe00 2024-09-21T03:43:25.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.596+0000 7fc9435ba640 1 -- 172.21.15.38:0/190794971 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc93c0b3270 con 0x7fc93c0cfe00 2024-09-21T03:43:25.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.597+0000 7fc931ffb640 1 -- 172.21.15.38:0/190794971 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc92c01a070 con 0x7fc93c0cfe00 2024-09-21T03:43:25.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.598+0000 7fc931ffb640 1 --2- 172.21.15.38:0/190794971 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc920078bc0 0x7fc92007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:25.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.598+0000 7fc931ffb640 1 -- 172.21.15.38:0/190794971 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 8970+0+0 (secure 0 0 0) 0x7fc92c09aba0 con 0x7fc93c0cfe00 2024-09-21T03:43:25.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.598+0000 7fc941356640 1 --2- 172.21.15.38:0/190794971 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc920078bc0 0x7fc92007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:25.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.598+0000 7fc941356640 1 --2- 172.21.15.38:0/190794971 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc920078bc0 0x7fc92007b0d0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fc938034000 tx=0x7fc938004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:25.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.602+0000 7fc931ffb640 1 -- 172.21.15.38:0/190794971 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fc92c061d10 con 0x7fc93c0cfe00 2024-09-21T03:43:25.698 INFO:tasks.ceph.mds.c.smithi038.stderr:2024-09-21T03:43:25.696+0000 7fe53d395640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T03:43:25.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.829+0000 7fc9435ba640 1 -- 172.21.15.38:0/190794971 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fc93c1f5e50 con 0x7fc93c0cfe00 2024-09-21T03:43:25.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.830+0000 7fc931ffb640 1 -- 172.21.15.38:0/190794971 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v275) ==== 53+0+73 (secure 0 0 0) 0x7fc92c066740 con 0x7fc93c0cfe00 2024-09-21T03:43:25.832 INFO:teuthology.orchestra.run.smithi038.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T03:43:25.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.832+0000 7fc9435ba640 1 -- 172.21.15.38:0/190794971 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc920078bc0 msgr2=0x7fc92007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:25.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.832+0000 7fc9435ba640 1 --2- 172.21.15.38:0/190794971 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc920078bc0 0x7fc92007b0d0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fc938034000 tx=0x7fc938004730 comp rx=0 tx=0).stop 2024-09-21T03:43:25.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.832+0000 7fc9435ba640 1 -- 172.21.15.38:0/190794971 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc93c0cfe00 msgr2=0x7fc93c1f52a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:25.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.832+0000 7fc9435ba640 1 --2- 172.21.15.38:0/190794971 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc93c0cfe00 0x7fc93c1f52a0 secure :-1 s=READY pgs=2013 cs=0 l=1 rev1=1 crypto rx=0x7fc92c00c370 tx=0x7fc92c00c830 comp rx=0 tx=0).stop 2024-09-21T03:43:25.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.832+0000 7fc9435ba640 1 -- 172.21.15.38:0/190794971 shutdown_connections 2024-09-21T03:43:25.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.832+0000 7fc9435ba640 1 --2- 172.21.15.38:0/190794971 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc93c162590 0x7fc93c1f57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:25.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.832+0000 7fc9435ba640 1 --2- 172.21.15.38:0/190794971 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc93c0cfe00 0x7fc93c1f52a0 unknown :-1 s=CLOSED pgs=2013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:25.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.832+0000 7fc9435ba640 1 --2- 172.21.15.38:0/190794971 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc920078bc0 0x7fc92007b0d0 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:25.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.832+0000 7fc9435ba640 1 --2- 172.21.15.38:0/190794971 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc93c0b2e60 0x7fc93c06db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:25.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.832+0000 7fc9435ba640 1 -- 172.21.15.38:0/190794971 >> 172.21.15.38:0/190794971 conn(0x7fc93c0d4440 msgr2=0x7fc93c06e610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:25.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.832+0000 7fc9435ba640 1 -- 172.21.15.38:0/190794971 shutdown_connections 2024-09-21T03:43:25.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:25.832+0000 7fc9435ba640 1 -- 172.21.15.38:0/190794971 wait complete. 2024-09-21T03:43:25.845 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-21T03:43:25.845 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T03:43:25.845 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi038.front.sepia.ceph.com 2024-09-21T03:43:25.845 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-21T03:43:25.845 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T03:43:25.846 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-21T03:43:25.846 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-21T03:43:25.846 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T03:43:25.846 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T03:43:25.846 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:43:25.846 DEBUG:teuthology.orchestra.run.smithi038:> ip addr 2024-09-21T03:43:25.862 INFO:teuthology.orchestra.run.smithi038.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T03:43:25.863 INFO:teuthology.orchestra.run.smithi038.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T03:43:25.863 INFO:teuthology.orchestra.run.smithi038.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T03:43:25.863 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:43:25.863 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 ::1/128 scope host 2024-09-21T03:43:25.863 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:43:25.863 INFO:teuthology.orchestra.run.smithi038.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T03:43:25.863 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:6b:fd:a2 brd ff:ff:ff:ff:ff:ff 2024-09-21T03:43:25.863 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp7s0f0 2024-09-21T03:43:25.863 INFO:teuthology.orchestra.run.smithi038.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T03:43:25.863 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:6b:fd:a3 brd ff:ff:ff:ff:ff:ff 2024-09-21T03:43:25.863 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp7s0f1 2024-09-21T03:43:25.864 INFO:teuthology.orchestra.run.smithi038.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T03:43:25.864 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:bd:15:fc brd ff:ff:ff:ff:ff:ff 2024-09-21T03:43:25.864 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp1s0f0 2024-09-21T03:43:25.864 INFO:teuthology.orchestra.run.smithi038.stdout: inet 172.21.15.38/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-21T03:43:25.864 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft 42203sec preferred_lft 42203sec 2024-09-21T03:43:25.864 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 fe80::ec4:7aff:febd:15fc/64 scope link 2024-09-21T03:43:25.864 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:43:25.864 INFO:teuthology.orchestra.run.smithi038.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T03:43:25.864 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:bd:15:fd brd ff:ff:ff:ff:ff:ff 2024-09-21T03:43:25.864 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp1s0f1 2024-09-21T03:43:25.864 INFO:teuthology.orchestra.run.smithi038.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T03:43:25.864 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 92:55:c7:97:6f:b5 brd ff:ff:ff:ff:ff:ff 2024-09-21T03:43:25.864 INFO:teuthology.orchestra.run.smithi038.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-21T03:43:25.864 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:43:25.865 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 fe80::c49f:c1ff:fe21:b12a/64 scope link 2024-09-21T03:43:25.865 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:43:25.865 INFO:teuthology.orchestra.run.smithi038.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T03:43:25.865 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether a6:3a:83:ad:db:d5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T03:43:25.865 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 fe80::a43a:83ff:fead:dbd5/64 scope link 2024-09-21T03:43:25.865 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:43:25.865 INFO:teuthology.orchestra.run.smithi038.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T03:43:25.865 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 92:55:c7:97:6f:b5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T03:43:25.865 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 fe80::9055:c7ff:fe97:6fb5/64 scope link 2024-09-21T03:43:25.865 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:43:25.865 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:43:25.866 DEBUG:teuthology.orchestra.run.smithi038:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T03:43:25.927 INFO:teuthology.orchestra.run.smithi038.stdout:1 2024-09-21T03:43:25.928 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:43:25.928 DEBUG:teuthology.orchestra.run.smithi038:> route 2024-09-21T03:43:25.985 INFO:teuthology.orchestra.run.smithi038.stdout:Kernel IP routing table 2024-09-21T03:43:25.985 INFO:teuthology.orchestra.run.smithi038.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T03:43:25.985 INFO:teuthology.orchestra.run.smithi038.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-21T03:43:25.985 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-21T03:43:25.985 INFO:teuthology.orchestra.run.smithi038.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T03:43:25.986 DEBUG:teuthology.orchestra.run.smithi038:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T03:43:25.986 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-09-21T03:43:25.986 DEBUG:teuthology.orchestra.run.smithi038:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-21T03:43:25.986 DEBUG:teuthology.orchestra.run.smithi038:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-21T03:43:25.986 DEBUG:teuthology.orchestra.run.smithi038:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-21T03:43:25.986 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-09-21T03:43:26.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:26 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T03:43:26.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:43:26.114 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:43:26.114 DEBUG:teuthology.orchestra.run.smithi038:> ip netns list 2024-09-21T03:43:26.130 INFO:teuthology.orchestra.run.smithi038.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-21T03:43:26.130 INFO:teuthology.orchestra.run.smithi038.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-21T03:43:26.130 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T03:43:26.131 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:43:26.131 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip link set brx.0 up 2024-09-21T03:43:26.196 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T03:43:26.197 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-21T03:43:26.252 INFO:teuthology.orchestra.run.smithi038.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-21T03:43:26.252 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T03:43:26.252 DEBUG:teuthology.orchestra.run.smithi038:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-21T03:43:26.308 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-21T03:43:26.308 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:43:26.308 DEBUG:teuthology.orchestra.run.smithi038:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T03:43:26.403 INFO:teuthology.orchestra.run.smithi038.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T03:43:26.404 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T03:43:26.404 INFO:teuthology.orchestra.run.smithi038.stdout:invalid new device string format 2024-09-21T03:43:26.414 INFO:teuthology.orchestra.run.smithi038.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T03:43:26.414 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T03:43:26.414 INFO:teuthology.orchestra.run.smithi038.stdout:invalid new device string format 2024-09-21T03:43:26.415 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: resolved to: "172.21.15.38:3300,172.21.15.195:3300,172.21.15.195:3301" 2024-09-21T03:43:26.415 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T03:43:26.425 INFO:teuthology.orchestra.run.smithi038.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T03:43:26.426 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T03:43:26.426 INFO:teuthology.orchestra.run.smithi038.stdout:invalid new device string format 2024-09-21T03:43:26.426 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: resolved to: "172.21.15.38:3300,172.21.15.195:3300,172.21.15.195:3301" 2024-09-21T03:43:26.426 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T03:43:26.426 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: resolved to: "172.21.15.38:6789,172.21.15.195:6789,172.21.15.195:6790" 2024-09-21T03:43:26.426 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.38:6789,172.21.15.195:6789,172.21.15.195:6790:/ 2024-09-21T03:43:26.426 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=9d50d66f-656b-4fe2-8e3b-fe53b79c4d43" will pass to kernel 2024-09-21T03:43:26.426 INFO:teuthology.orchestra.run.smithi038.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-21T03:43:26.427 INFO:teuthology.orchestra.run.smithi038.stdout: You just mounted a file system that supports labels which does not 2024-09-21T03:43:26.427 INFO:teuthology.orchestra.run.smithi038.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-21T03:43:26.427 INFO:teuthology.orchestra.run.smithi038.stdout: applications will generate AVC messages and not be allowed access to 2024-09-21T03:43:26.427 INFO:teuthology.orchestra.run.smithi038.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-21T03:43:26.428 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T03:43:26.428 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:43:26.428 DEBUG:teuthology.orchestra.run.smithi038:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-21T03:43:26.454 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T03:43:26.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:26 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T03:43:26.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:43:26.552 INFO:teuthology.orchestra.run.smithi038.stdout:client8938 2024-09-21T03:43:26.552 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-21T03:43:26.552 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:43:26.569 DEBUG:teuthology.orchestra.run.smithi038:> stdin-killer -- python3 -c ' 2024-09-21T03:43:26.569 DEBUG:teuthology.orchestra.run.smithi038:> import os 2024-09-21T03:43:26.569 DEBUG:teuthology.orchestra.run.smithi038:> import stat 2024-09-21T03:43:26.569 DEBUG:teuthology.orchestra.run.smithi038:> import json 2024-09-21T03:43:26.569 DEBUG:teuthology.orchestra.run.smithi038:> import sys 2024-09-21T03:43:26.569 DEBUG:teuthology.orchestra.run.smithi038:> 2024-09-21T03:43:26.569 DEBUG:teuthology.orchestra.run.smithi038:> try: 2024-09-21T03:43:26.569 DEBUG:teuthology.orchestra.run.smithi038:> s = os.stat('"'"'/home/ubuntu/cephtest/mnt.0/./subdir/sixmegs'"'"') 2024-09-21T03:43:26.569 DEBUG:teuthology.orchestra.run.smithi038:> except OSError as e: 2024-09-21T03:43:26.569 DEBUG:teuthology.orchestra.run.smithi038:> sys.exit(e.errno) 2024-09-21T03:43:26.569 DEBUG:teuthology.orchestra.run.smithi038:> 2024-09-21T03:43:26.570 DEBUG:teuthology.orchestra.run.smithi038:> attrs = ["st_mode", "st_ino", "st_dev", "st_nlink", "st_uid", "st_gid", "st_size", "st_atime", "st_mtime", "st_ctime"] 2024-09-21T03:43:26.570 DEBUG:teuthology.orchestra.run.smithi038:> print(json.dumps( 2024-09-21T03:43:26.570 DEBUG:teuthology.orchestra.run.smithi038:> dict([(a, getattr(s, a)) for a in attrs]), 2024-09-21T03:43:26.570 DEBUG:teuthology.orchestra.run.smithi038:> indent=2)) 2024-09-21T03:43:26.570 DEBUG:teuthology.orchestra.run.smithi038:> ' 2024-09-21T03:43:26.611 INFO:teuthology.orchestra.run:waiting for 20 2024-09-21T03:43:26.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:26 stdin-killer INFO: command exited with status 5: exiting normally with same code! 2024-09-21T03:43:26.688 DEBUG:teuthology.orchestra.run:got remote process result: 5 2024-09-21T03:43:26.688 INFO:tasks.cephfs.test_damage:Result: EIO on client 2024-09-21T03:43:26.688 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:43:26.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:26.844+0000 7f9735f82640 1 -- 172.21.15.38:0/1518127331 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f97300cea90 msgr2=0x7f97300d00d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:26.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:26.844+0000 7f9735f82640 1 --2- 172.21.15.38:0/1518127331 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f97300cea90 0x7f97300d00d0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f97200098e0 tx=0x7f972002a530 comp rx=0 tx=0).stop 2024-09-21T03:43:26.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:26.844+0000 7f9735f82640 1 -- 172.21.15.38:0/1518127331 shutdown_connections 2024-09-21T03:43:26.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:26.844+0000 7f9735f82640 1 --2- 172.21.15.38:0/1518127331 >> v2:172.21.15.195:3300/0 conn(0x7f9730162790 0x7f9730162ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:26.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:26.844+0000 7f9735f82640 1 --2- 172.21.15.38:0/1518127331 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f97300cea90 0x7f97300d00d0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:26.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:26.844+0000 7f9735f82640 1 -- 172.21.15.38:0/1518127331 >> 172.21.15.38:0/1518127331 conn(0x7f97300d4620 msgr2=0x7f97300d4a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:26.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:26.844+0000 7f9735f82640 1 -- 172.21.15.38:0/1518127331 shutdown_connections 2024-09-21T03:43:26.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:26.844+0000 7f9735f82640 1 -- 172.21.15.38:0/1518127331 wait complete. 2024-09-21T03:43:26.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:26.846+0000 7f9735f82640 1 Processor -- start 2024-09-21T03:43:26.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:26.846+0000 7f9735f82640 1 -- start start 2024-09-21T03:43:26.851 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:26.848+0000 7f9735f82640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f97300cea90 0x7f9730157080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:26.851 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:26.848+0000 7f9735f82640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f97300d3360 0x7f97301f78e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:26.851 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:26.848+0000 7f9735f82640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9730162790 0x7f97301f7e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:26.851 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:26.848+0000 7f9735f82640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f97300b4180 con 0x7f9730162790 2024-09-21T03:43:26.851 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:26.848+0000 7f9735f82640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f97300b4000 con 0x7f97300d3360 2024-09-21T03:43:26.851 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:26.848+0000 7f9735f82640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f97300b4300 con 0x7f97300cea90 2024-09-21T03:43:26.851 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:26.848+0000 7f972ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9730162790 0x7f97301f7e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:26.851 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:26.848+0000 7f972ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9730162790 0x7f97301f7e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:58452/0 (socket says 172.21.15.38:58452) 2024-09-21T03:43:26.851 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:26.848+0000 7f972ffff640 1 -- 172.21.15.38:0/3735034608 learned_addr learned my addr 172.21.15.38:0/3735034608 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:43:26.851 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:26.848+0000 7f972f7fe640 1 --2- 172.21.15.38:0/3735034608 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f97300cea90 0x7f9730157080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:26.851 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:26.848+0000 7f972effd640 1 --2- 172.21.15.38:0/3735034608 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f97300d3360 0x7f97301f78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:26.851 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:26.848+0000 7f972ffff640 1 -- 172.21.15.38:0/3735034608 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f97300cea90 msgr2=0x7f9730157080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:26.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:26.848+0000 7f972ffff640 1 --2- 172.21.15.38:0/3735034608 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f97300cea90 0x7f9730157080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:26.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:26.848+0000 7f972ffff640 1 -- 172.21.15.38:0/3735034608 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f97300d3360 msgr2=0x7f97301f78e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:26.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:26.848+0000 7f972ffff640 1 --2- 172.21.15.38:0/3735034608 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f97300d3360 0x7f97301f78e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:26.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:26.848+0000 7f972ffff640 1 -- 172.21.15.38:0/3735034608 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97301f6e70 con 0x7f9730162790 2024-09-21T03:43:26.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:26.848+0000 7f972ffff640 1 --2- 172.21.15.38:0/3735034608 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9730162790 0x7f97301f7e20 secure :-1 s=READY pgs=2014 cs=0 l=1 rev1=1 crypto rx=0x7f972800c730 tx=0x7f972800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:26.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:26.849+0000 7f9717fff640 1 -- 172.21.15.38:0/3735034608 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9728003b70 con 0x7f9730162790 2024-09-21T03:43:26.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:26.849+0000 7f9717fff640 1 -- 172.21.15.38:0/3735034608 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9728003d10 con 0x7f9730162790 2024-09-21T03:43:26.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:26.849+0000 7f9735f82640 1 -- 172.21.15.38:0/3735034608 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9730157cd0 con 0x7f9730162790 2024-09-21T03:43:26.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:26.849+0000 7f9717fff640 1 -- 172.21.15.38:0/3735034608 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f972800aab0 con 0x7f9730162790 2024-09-21T03:43:26.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:26.849+0000 7f9735f82640 1 -- 172.21.15.38:0/3735034608 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97301f8960 con 0x7f9730162790 2024-09-21T03:43:26.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:26.850+0000 7f9735f82640 1 -- 172.21.15.38:0/3735034608 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97301f8bc0 con 0x7f9730162790 2024-09-21T03:43:26.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:26.851+0000 7f9717fff640 1 -- 172.21.15.38:0/3735034608 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f972800ac50 con 0x7f9730162790 2024-09-21T03:43:26.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:26.851+0000 7f9717fff640 1 --2- 172.21.15.38:0/3735034608 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f971c078b90 0x7f971c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:26.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:26.851+0000 7f9717fff640 1 -- 172.21.15.38:0/3735034608 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 8970+0+0 (secure 0 0 0) 0x7f972809b770 con 0x7f9730162790 2024-09-21T03:43:26.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:26.851+0000 7f972f7fe640 1 --2- 172.21.15.38:0/3735034608 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f971c078b90 0x7f971c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:26.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:26.852+0000 7f972f7fe640 1 --2- 172.21.15.38:0/3735034608 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f971c078b90 0x7f971c07b0a0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f9720034000 tx=0x7f9720004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:26.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:26.856+0000 7f9717fff640 1 -- 172.21.15.38:0/3735034608 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f97301630e0 con 0x7f9730162790 2024-09-21T03:43:27.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.079+0000 7f9735f82640 1 -- 172.21.15.38:0/3735034608 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f97300b47b0 con 0x7f9730162790 2024-09-21T03:43:27.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.080+0000 7f9717fff640 1 -- 172.21.15.38:0/3735034608 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 275 v275) ==== 95+0+5234 (secure 0 0 0) 0x7f97280627d0 con 0x7f9730162790 2024-09-21T03:43:27.082 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:43:27.082 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":275,"btime":"2024-09-21T03:43:21:731479+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6086,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/965972663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":965972663},{"type":"v1","addr":"172.21.15.195:6836","nonce":965972663}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6700,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/3176122334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":3176122334},{"type":"v1","addr":"172.21.15.195:6837","nonce":3176122334}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":9840,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/3801772046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":3801772046},{"type":"v1","addr":"172.21.15.38:6835","nonce":3801772046}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":273}],"filesystems":[{"mdsmap":{"epoch":275,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:40:02.319282+0000","modified":"2024-09-21T03:43:21.731469+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":127,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9786},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9786":{"gid":9786,"name":"c","rank":0,"incarnation":272,"state":"up:active","state_seq":4,"addr":"172.21.15.38:6833/2309772457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":2309772457},{"type":"v1","addr":"172.21.15.38:6833","nonce":2309772457}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9786,"qdb_cluster":[9786]},"id":6}]} 2024-09-21T03:43:27.082 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 275 2024-09-21T03:43:27.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.082+0000 7f9735f82640 1 -- 172.21.15.38:0/3735034608 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f971c078b90 msgr2=0x7f971c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:27.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.082+0000 7f9735f82640 1 --2- 172.21.15.38:0/3735034608 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f971c078b90 0x7f971c07b0a0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f9720034000 tx=0x7f9720004730 comp rx=0 tx=0).stop 2024-09-21T03:43:27.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.082+0000 7f9735f82640 1 -- 172.21.15.38:0/3735034608 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9730162790 msgr2=0x7f97301f7e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:27.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.082+0000 7f9735f82640 1 --2- 172.21.15.38:0/3735034608 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9730162790 0x7f97301f7e20 secure :-1 s=READY pgs=2014 cs=0 l=1 rev1=1 crypto rx=0x7f972800c730 tx=0x7f972800cbf0 comp rx=0 tx=0).stop 2024-09-21T03:43:27.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.082+0000 7f9735f82640 1 -- 172.21.15.38:0/3735034608 shutdown_connections 2024-09-21T03:43:27.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.082+0000 7f9735f82640 1 --2- 172.21.15.38:0/3735034608 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9730162790 0x7f97301f7e20 unknown :-1 s=CLOSED pgs=2014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:27.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.082+0000 7f9735f82640 1 --2- 172.21.15.38:0/3735034608 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f97300d3360 0x7f97301f78e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:27.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.082+0000 7f9735f82640 1 --2- 172.21.15.38:0/3735034608 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f971c078b90 0x7f971c07b0a0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:27.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.082+0000 7f9735f82640 1 --2- 172.21.15.38:0/3735034608 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f97300cea90 0x7f9730157080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:27.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.082+0000 7f9735f82640 1 -- 172.21.15.38:0/3735034608 >> 172.21.15.38:0/3735034608 conn(0x7f97300d4620 msgr2=0x7f97300d15f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:27.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.082+0000 7f9735f82640 1 -- 172.21.15.38:0/3735034608 shutdown_connections 2024-09-21T03:43:27.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.082+0000 7f9735f82640 1 -- 172.21.15.38:0/3735034608 wait complete. 2024-09-21T03:43:27.095 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c damage ls --format=json-pretty 2024-09-21T03:43:27.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.213+0000 7f10ae924640 1 -- 172.21.15.38:0/3428629812 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1090004090 msgr2=0x7f1090004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:27.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.213+0000 7f10ae924640 1 --2- 172.21.15.38:0/3428629812 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1090004090 0x7f1090004800 secure :-1 s=READY pgs=2016 cs=0 l=1 rev1=1 crypto rx=0x7f109c00bbe0 tx=0x7f1090005830 comp rx=0 tx=0).stop 2024-09-21T03:43:27.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.213+0000 7f10acec1640 1 -- 172.21.15.38:0/3428629812 reap_dead start 2024-09-21T03:43:27.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.213+0000 7f10ae924640 1 -- 172.21.15.38:0/3428629812 shutdown_connections 2024-09-21T03:43:27.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.213+0000 7f10ae924640 1 -- 172.21.15.38:0/3428629812 >> 172.21.15.38:0/3428629812 conn(0x7f10a80b1b90 msgr2=0x7f10a80b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:27.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.213+0000 7f10ae924640 1 -- 172.21.15.38:0/3428629812 shutdown_connections 2024-09-21T03:43:27.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.213+0000 7f10ae924640 1 -- 172.21.15.38:0/3428629812 wait complete. 2024-09-21T03:43:27.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.214+0000 7f10ae924640 1 Processor -- start 2024-09-21T03:43:27.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.215+0000 7f10ae924640 1 -- start start 2024-09-21T03:43:27.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.216+0000 7f10ae924640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1090004090 0x7f10a80b2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:27.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.216+0000 7f10ae924640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f10a80cdf00 0x7f10a81f8400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:27.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.216+0000 7f10ae924640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f10a809d010 0x7f10a809d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:27.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.216+0000 7f10ae924640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f10a80701f0 con 0x7f10a80cdf00 2024-09-21T03:43:27.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.216+0000 7f10ae924640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f10a8070070 con 0x7f10a809d010 2024-09-21T03:43:27.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.216+0000 7f10ae924640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f10a8070370 con 0x7f1090004090 2024-09-21T03:43:27.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.216+0000 7f10a77fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f10a80cdf00 0x7f10a81f8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:27.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.216+0000 7f10a77fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f10a80cdf00 0x7f10a81f8400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:58470/0 (socket says 172.21.15.38:58470) 2024-09-21T03:43:27.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.216+0000 7f10a77fe640 1 -- 172.21.15.38:0/2636472082 learned_addr learned my addr 172.21.15.38:0/2636472082 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:43:27.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.216+0000 7f10a7fff640 1 --2- 172.21.15.38:0/2636472082 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1090004090 0x7f10a80b2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:27.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.216+0000 7f10acec1640 1 --2- 172.21.15.38:0/2636472082 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f10a809d010 0x7f10a809d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:27.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.216+0000 7f10a77fe640 1 -- 172.21.15.38:0/2636472082 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1090004090 msgr2=0x7f10a80b2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:27.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.216+0000 7f10a77fe640 1 --2- 172.21.15.38:0/2636472082 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1090004090 0x7f10a80b2ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:27.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.216+0000 7f10a77fe640 1 -- 172.21.15.38:0/2636472082 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f10a809d010 msgr2=0x7f10a809d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:27.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.217+0000 7f10a77fe640 1 --2- 172.21.15.38:0/2636472082 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f10a809d010 0x7f10a809d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:27.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.217+0000 7f10a77fe640 1 -- 172.21.15.38:0/2636472082 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10a809a9e0 con 0x7f10a80cdf00 2024-09-21T03:43:27.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.217+0000 7f10acec1640 1 --2- 172.21.15.38:0/2636472082 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f10a809d010 0x7f10a809d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:43:27.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.217+0000 7f10a77fe640 1 --2- 172.21.15.38:0/2636472082 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f10a80cdf00 0x7f10a81f8400 secure :-1 s=READY pgs=2017 cs=0 l=1 rev1=1 crypto rx=0x7f109c00bbe0 tx=0x7f109c02f4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:27.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.217+0000 7f10a4ff9640 1 -- 172.21.15.38:0/2636472082 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f109c045070 con 0x7f10a80cdf00 2024-09-21T03:43:27.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.217+0000 7f10a4ff9640 1 -- 172.21.15.38:0/2636472082 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f109c02fcc0 con 0x7f10a80cdf00 2024-09-21T03:43:27.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.217+0000 7f10ae924640 1 -- 172.21.15.38:0/2636472082 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10a80ca490 con 0x7f10a80cdf00 2024-09-21T03:43:27.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.217+0000 7f10a4ff9640 1 -- 172.21.15.38:0/2636472082 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f109c034050 con 0x7f10a80cdf00 2024-09-21T03:43:27.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.217+0000 7f10ae924640 1 -- 172.21.15.38:0/2636472082 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10a809ac80 con 0x7f10a80cdf00 2024-09-21T03:43:27.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.219+0000 7f10a4ff9640 1 -- 172.21.15.38:0/2636472082 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f109c048070 con 0x7f10a80cdf00 2024-09-21T03:43:27.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.219+0000 7f10a4ff9640 1 --2- 172.21.15.38:0/2636472082 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1094078b90 0x7f109407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:27.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.220+0000 7f10a4ff9640 1 -- 172.21.15.38:0/2636472082 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 8970+0+0 (secure 0 0 0) 0x7f109c0bc470 con 0x7f10a80cdf00 2024-09-21T03:43:27.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.220+0000 7f10a7fff640 1 --2- 172.21.15.38:0/2636472082 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1094078b90 0x7f109407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:27.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.220+0000 7f10b588e740 1 Processor -- start 2024-09-21T03:43:27.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.220+0000 7f10b588e740 1 -- start start 2024-09-21T03:43:27.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.222+0000 7f10b588e740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555d75e57350 0x555d76228d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:27.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.222+0000 7f10b588e740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555d75e2e490 0x555d762292c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:27.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.222+0000 7f10b588e740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555d75e36670 0x555d762461c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:27.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.222+0000 7f10b588e740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x555d761719a0 con 0x555d75e57350 2024-09-21T03:43:27.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.222+0000 7f10b588e740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x555d76160dd0 con 0x555d75e36670 2024-09-21T03:43:27.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.222+0000 7f10b588e740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x555d76164be0 con 0x555d75e2e490 2024-09-21T03:43:27.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.222+0000 7f10a7fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555d75e57350 0x555d76228d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:27.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.222+0000 7f10a7fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555d75e57350 0x555d76228d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:58476/0 (socket says 172.21.15.38:58476) 2024-09-21T03:43:27.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.222+0000 7f10a7fff640 1 -- 172.21.15.38:0/4234703369 learned_addr learned my addr 172.21.15.38:0/4234703369 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:43:27.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.222+0000 7f10a77fe640 1 --2- 172.21.15.38:0/4234703369 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555d75e2e490 0x555d762292c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:27.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.222+0000 7f10acec1640 1 --2- 172.21.15.38:0/4234703369 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555d75e36670 0x555d762461c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:27.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.222+0000 7f10a7fff640 1 -- 172.21.15.38:0/4234703369 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555d75e2e490 msgr2=0x555d762292c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:27.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.222+0000 7f10a7fff640 1 --2- 172.21.15.38:0/4234703369 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555d75e2e490 0x555d762292c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:27.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.222+0000 7f10a7fff640 1 -- 172.21.15.38:0/4234703369 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555d75e36670 msgr2=0x555d762461c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:27.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.222+0000 7f10a7fff640 1 --2- 172.21.15.38:0/4234703369 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555d75e36670 0x555d762461c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:27.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.222+0000 7f10a7fff640 1 -- 172.21.15.38:0/4234703369 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d760f9700 con 0x555d75e57350 2024-09-21T03:43:27.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.222+0000 7f10a7fff640 1 --2- 172.21.15.38:0/4234703369 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555d75e57350 0x555d76228d80 secure :-1 s=READY pgs=2018 cs=0 l=1 rev1=1 crypto rx=0x7f109800f600 tx=0x7f109801e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4c6a258a681a1d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:27.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.222+0000 7f108dffb640 1 -- 172.21.15.38:0/4234703369 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1098026070 con 0x555d75e57350 2024-09-21T03:43:27.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.222+0000 7f108dffb640 1 -- 172.21.15.38:0/4234703369 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1098016030 con 0x555d75e57350 2024-09-21T03:43:27.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.222+0000 7f108dffb640 1 -- 172.21.15.38:0/4234703369 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f109801b050 con 0x555d75e57350 2024-09-21T03:43:27.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.223+0000 7f10b588e740 1 -- 172.21.15.38:0/4234703369 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555d75e57350 msgr2=0x555d76228d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:27.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.223+0000 7f10b588e740 1 --2- 172.21.15.38:0/4234703369 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555d75e57350 0x555d76228d80 secure :-1 s=READY pgs=2018 cs=0 l=1 rev1=1 crypto rx=0x7f109800f600 tx=0x7f109801e000 comp rx=0 tx=0).stop 2024-09-21T03:43:27.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.223+0000 7f10b588e740 1 -- 172.21.15.38:0/4234703369 shutdown_connections 2024-09-21T03:43:27.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.223+0000 7f10b588e740 1 --2- 172.21.15.38:0/4234703369 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555d75e36670 0x555d762461c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:27.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.223+0000 7f10b588e740 1 --2- 172.21.15.38:0/4234703369 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555d75e2e490 0x555d762292c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:27.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.223+0000 7f10b588e740 1 --2- 172.21.15.38:0/4234703369 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555d75e57350 0x555d76228d80 unknown :-1 s=CLOSED pgs=2018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:27.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.223+0000 7f10b588e740 1 -- 172.21.15.38:0/4234703369 >> 172.21.15.38:0/4234703369 conn(0x555d75e4b4a0 msgr2=0x555d761ff040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:27.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.223+0000 7f10b588e740 1 -- 172.21.15.38:0/4234703369 shutdown_connections 2024-09-21T03:43:27.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.223+0000 7f10b588e740 1 -- 172.21.15.38:0/4234703369 wait complete. 2024-09-21T03:43:27.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.224+0000 7f10b588e740 1 -- start start 2024-09-21T03:43:27.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.224+0000 7f10a7fff640 1 --2- 172.21.15.38:0/2636472082 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1094078b90 0x7f109407b0a0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f1098003fb0 tx=0x7f1098003af0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:27.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.224+0000 7f10b588e740 1 Processor -- start 2024-09-21T03:43:27.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.226+0000 7f10b588e740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555d75e2e490 0x555d7623c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:27.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.226+0000 7f10b588e740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555d75e36670 0x555d762567b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:27.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.226+0000 7f10b588e740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555d75e57350 0x555d7623ba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:27.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.226+0000 7f10b588e740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x555d75ed1700 con 0x555d75e57350 2024-09-21T03:43:27.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.226+0000 7f10b588e740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x555d76106340 con 0x555d75e2e490 2024-09-21T03:43:27.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.226+0000 7f10b588e740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x555d76229800 con 0x555d75e36670 2024-09-21T03:43:27.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.226+0000 7f10b588e740 10 client.0 _ll_register_callbacks cb 0x555d76104ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:43:27.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.226+0000 7f10acec1640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555d75e57350 0x555d7623ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:27.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.226+0000 7f10acec1640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555d75e57350 0x555d7623ba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:58482/0 (socket says 172.21.15.38:58482) 2024-09-21T03:43:27.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.226+0000 7f10acec1640 1 -- 172.21.15.38:0/1369794014 learned_addr learned my addr 172.21.15.38:0/1369794014 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:43:27.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.226+0000 7f10a77fe640 1 --2- 172.21.15.38:0/1369794014 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555d75e36670 0x555d762567b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:27.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.226+0000 7f10a7fff640 1 --2- 172.21.15.38:0/1369794014 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555d75e2e490 0x555d7623c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:27.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.226+0000 7f10acec1640 1 -- 172.21.15.38:0/1369794014 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555d75e36670 msgr2=0x555d762567b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:27.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.226+0000 7f10acec1640 1 --2- 172.21.15.38:0/1369794014 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555d75e36670 0x555d762567b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:27.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.226+0000 7f10acec1640 1 -- 172.21.15.38:0/1369794014 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555d75e2e490 msgr2=0x555d7623c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:27.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.226+0000 7f10acec1640 1 --2- 172.21.15.38:0/1369794014 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555d75e2e490 0x555d7623c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:27.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.226+0000 7f10acec1640 1 -- 172.21.15.38:0/1369794014 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d75f3b7c0 con 0x555d75e57350 2024-09-21T03:43:27.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.226+0000 7f10a7fff640 1 --2- 172.21.15.38:0/1369794014 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555d75e2e490 0x555d7623c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:43:27.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.226+0000 7f10acec1640 1 --2- 172.21.15.38:0/1369794014 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555d75e57350 0x555d7623ba80 secure :-1 s=READY pgs=2019 cs=0 l=1 rev1=1 crypto rx=0x7f10a0003e20 tx=0x7f10a0006430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8a0042ca066b485 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:27.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.226+0000 7f1086ffd640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-21T03:43:27.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.226+0000 7f1086ffd640 1 -- 172.21.15.38:0/1369794014 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10a0012910 con 0x555d75e57350 2024-09-21T03:43:27.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.226+0000 7f1086ffd640 1 -- 172.21.15.38:0/1369794014 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(127..127 src has 1..127) ==== 8970+0+0 (secure 0 0 0) 0x7f10a0012ab0 con 0x555d75e57350 2024-09-21T03:43:27.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.226+0000 7f10b588e740 1 -- 172.21.15.38:0/1369794014 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1098016030 con 0x555d75e57350 2024-09-21T03:43:27.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.227+0000 7f1086ffd640 1 -- 172.21.15.38:0/1369794014 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=275) ==== 24+0+0 (secure 0 0 0) 0x7f10a0021050 con 0x555d75e57350 2024-09-21T03:43:27.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.227+0000 7f10b588e740 10 client.9891 fetch_fsmap learned FSMap version 275 2024-09-21T03:43:27.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.227+0000 7f10b588e740 1 -- 172.21.15.38:0/1369794014 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=275,monmap=2+}) -- 0x555d75f9bf50 con 0x555d75e57350 2024-09-21T03:43:27.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.227+0000 7f1086ffd640 1 -- 172.21.15.38:0/1369794014 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1098016030 con 0x555d75e57350 2024-09-21T03:43:27.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.227+0000 7f1086ffd640 1 -- 172.21.15.38:0/1369794014 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 275) ==== 3502+0+0 (secure 0 0 0) 0x7f10a001a060 con 0x555d75e57350 2024-09-21T03:43:27.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.227+0000 7f10b588e740 10 client.9891 fetch_fsmap finished waiting for FSMap version 275 2024-09-21T03:43:27.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.227+0000 7f10b588e740 10 client.9891 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T03:43:27.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.227+0000 7f10b588e740 20 client.9891 populate_metadata read hostname 'smithi038' 2024-09-21T03:43:27.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.227+0000 7f10b588e740 1 --2- 172.21.15.38:0/1369794014 >> [v2:172.21.15.38:6832/2309772457,v1:172.21.15.38:6833/2309772457] conn(0x555d75e16130 0x555d76221260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:27.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.227+0000 7f10b588e740 4 client.9891 mds_command: new command op to 9786 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:43:27.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.227+0000 7f10b588e740 1 -- 172.21.15.38:0/1369794014 --> [v2:172.21.15.38:6832/2309772457,v1:172.21.15.38:6833/2309772457] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555d761738f0 con 0x555d75e16130 2024-09-21T03:43:27.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.227+0000 7f10a7fff640 1 --2- 172.21.15.38:0/1369794014 >> [v2:172.21.15.38:6832/2309772457,v1:172.21.15.38:6833/2309772457] conn(0x555d75e16130 0x555d76221260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:27.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.227+0000 7f1086ffd640 10 client.9891 ms_handle_connect on v2:172.21.15.38:6832/2309772457 2024-09-21T03:43:27.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.228+0000 7f10a7fff640 1 --2- 172.21.15.38:0/1369794014 >> [v2:172.21.15.38:6832/2309772457,v1:172.21.15.38:6833/2309772457] conn(0x555d75e16130 0x555d76221260 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6630f55097b0cbd6 server_cookie=1b19a202735079ac in_seq=0 out_seq=0 2024-09-21T03:43:27.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.228+0000 7f1086ffd640 1 -- 172.21.15.38:0/1369794014 <== mds.0 v2:172.21.15.38:6832/2309772457 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x555d761738f0 con 0x555d75e16130 2024-09-21T03:43:27.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.228+0000 7f1086ffd640 10 client.9891 handle_command_reply: tid=0 2024-09-21T03:43:27.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.228+0000 7f10b588e740 1 client.9891 shutdown 2024-09-21T03:43:27.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.228+0000 7f10b588e740 1 -- 172.21.15.38:0/1369794014 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555d75e57350 msgr2=0x555d7623ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:27.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.228+0000 7f10b588e740 1 --2- 172.21.15.38:0/1369794014 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555d75e57350 0x555d7623ba80 secure :-1 s=READY pgs=2019 cs=0 l=1 rev1=1 crypto rx=0x7f10a0003e20 tx=0x7f10a0006430 comp rx=0 tx=0).stop 2024-09-21T03:43:27.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.229+0000 7f10b588e740 1 -- 172.21.15.38:0/1369794014 shutdown_connections 2024-09-21T03:43:27.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.229+0000 7f10b588e740 1 --2- 172.21.15.38:0/1369794014 >> [v2:172.21.15.38:6832/2309772457,v1:172.21.15.38:6833/2309772457] conn(0x555d75e16130 0x555d76221260 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:27.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.229+0000 7f10b588e740 1 --2- 172.21.15.38:0/1369794014 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555d75e57350 0x555d7623ba80 unknown :-1 s=CLOSED pgs=2019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:27.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.229+0000 7f10b588e740 1 --2- 172.21.15.38:0/1369794014 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555d75e36670 0x555d762567b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:27.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.229+0000 7f10b588e740 1 --2- 172.21.15.38:0/1369794014 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555d75e2e490 0x555d7623c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:27.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.229+0000 7f1086ffd640 0 client.9891 ms_handle_reset on v2:172.21.15.38:6832/2309772457 2024-09-21T03:43:27.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.229+0000 7f10b588e740 1 -- 172.21.15.38:0/1369794014 >> 172.21.15.38:0/1369794014 conn(0x555d75e4b4a0 msgr2=0x555d761f96d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:27.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.229+0000 7f10b588e740 1 -- 172.21.15.38:0/1369794014 shutdown_connections 2024-09-21T03:43:27.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.229+0000 7f10b588e740 1 -- 172.21.15.38:0/1369794014 wait complete. 2024-09-21T03:43:27.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.229+0000 7f10b588e740 20 client.9891 trim_cache size 0 max 16384 2024-09-21T03:43:27.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.242+0000 7f10b588e740 1 Processor -- start 2024-09-21T03:43:27.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.242+0000 7f10b588e740 1 -- start start 2024-09-21T03:43:27.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.244+0000 7f10b588e740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555d75e2e490 0x555d761f84f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:27.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.244+0000 7f10b588e740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555d75e36670 0x555d761f8a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:27.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.244+0000 7f10b588e740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555d75e57350 0x555d76266910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:27.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.244+0000 7f10b588e740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f10a8203b00 con 0x555d75e57350 2024-09-21T03:43:27.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.244+0000 7f10b588e740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f10a8202e00 con 0x555d75e2e490 2024-09-21T03:43:27.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.244+0000 7f10b588e740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f10a8203450 con 0x555d75e36670 2024-09-21T03:43:27.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.244+0000 7f10acec1640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555d75e57350 0x555d76266910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:27.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.244+0000 7f10acec1640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555d75e57350 0x555d76266910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:58492/0 (socket says 172.21.15.38:58492) 2024-09-21T03:43:27.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.244+0000 7f10acec1640 1 -- 172.21.15.38:0/1000518702 learned_addr learned my addr 172.21.15.38:0/1000518702 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:43:27.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.244+0000 7f10a77fe640 1 --2- 172.21.15.38:0/1000518702 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555d75e36670 0x555d761f8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:27.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.244+0000 7f10acec1640 1 -- 172.21.15.38:0/1000518702 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555d75e36670 msgr2=0x555d761f8a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:27.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.244+0000 7f10acec1640 1 --2- 172.21.15.38:0/1000518702 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555d75e36670 0x555d761f8a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:27.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.244+0000 7f10a7fff640 1 --2- 172.21.15.38:0/1000518702 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555d75e2e490 0x555d761f84f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:27.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.244+0000 7f10acec1640 1 -- 172.21.15.38:0/1000518702 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555d75e2e490 msgr2=0x555d761f84f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:27.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.244+0000 7f10acec1640 1 --2- 172.21.15.38:0/1000518702 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555d75e2e490 0x555d761f84f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:27.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.244+0000 7f10acec1640 1 -- 172.21.15.38:0/1000518702 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d76245040 con 0x555d75e57350 2024-09-21T03:43:27.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.244+0000 7f10acec1640 1 --2- 172.21.15.38:0/1000518702 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555d75e57350 0x555d76266910 secure :-1 s=READY pgs=2020 cs=0 l=1 rev1=1 crypto rx=0x7f10a0012e00 tx=0x7f10a0003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ca39b845fe11703 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:27.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.245+0000 7f108d7fa640 1 -- 172.21.15.38:0/1000518702 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10a0006c10 con 0x555d75e57350 2024-09-21T03:43:27.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.245+0000 7f108d7fa640 1 -- 172.21.15.38:0/1000518702 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f10a0012a00 con 0x555d75e57350 2024-09-21T03:43:27.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.245+0000 7f108d7fa640 1 -- 172.21.15.38:0/1000518702 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10a0012bb0 con 0x555d75e57350 2024-09-21T03:43:27.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.245+0000 7f10b588e740 1 -- 172.21.15.38:0/1000518702 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555d75e57350 msgr2=0x555d76266910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:27.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.245+0000 7f10b588e740 1 --2- 172.21.15.38:0/1000518702 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555d75e57350 0x555d76266910 secure :-1 s=READY pgs=2020 cs=0 l=1 rev1=1 crypto rx=0x7f10a0012e00 tx=0x7f10a0003970 comp rx=0 tx=0).stop 2024-09-21T03:43:27.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.245+0000 7f10b588e740 1 -- 172.21.15.38:0/1000518702 shutdown_connections 2024-09-21T03:43:27.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.245+0000 7f10b588e740 1 --2- 172.21.15.38:0/1000518702 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555d75e57350 0x555d76266910 unknown :-1 s=CLOSED pgs=2020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:27.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.245+0000 7f10b588e740 1 --2- 172.21.15.38:0/1000518702 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555d75e36670 0x555d761f8a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:27.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.245+0000 7f10b588e740 1 --2- 172.21.15.38:0/1000518702 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555d75e2e490 0x555d761f84f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:27.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.245+0000 7f10b588e740 1 -- 172.21.15.38:0/1000518702 >> 172.21.15.38:0/1000518702 conn(0x555d75e4b4a0 msgr2=0x555d761f4c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:27.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.245+0000 7f10b588e740 1 -- 172.21.15.38:0/1000518702 shutdown_connections 2024-09-21T03:43:27.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.245+0000 7f10b588e740 1 -- 172.21.15.38:0/1000518702 wait complete. 2024-09-21T03:43:27.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.246+0000 7f10b588e740 1 -- start start 2024-09-21T03:43:27.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.246+0000 7f10b588e740 1 Processor -- start 2024-09-21T03:43:27.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.248+0000 7f10b588e740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555d75e2e490 0x555d762c5d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:27.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.248+0000 7f10b588e740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555d75e36670 0x555d761f84f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:27.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.248+0000 7f10b588e740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555d75e57350 0x555d761f8a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:27.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.248+0000 7f10b588e740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x555d76256de0 con 0x555d75e57350 2024-09-21T03:43:27.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.248+0000 7f10b588e740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x555d762222e0 con 0x555d75e36670 2024-09-21T03:43:27.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.248+0000 7f10b588e740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x555d76208750 con 0x555d75e2e490 2024-09-21T03:43:27.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.248+0000 7f10b588e740 10 client.0 _ll_register_callbacks cb 0x7f10a0001250 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:43:27.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.248+0000 7f10acec1640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555d75e57350 0x555d761f8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:27.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.248+0000 7f10acec1640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555d75e57350 0x555d761f8a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:58496/0 (socket says 172.21.15.38:58496) 2024-09-21T03:43:27.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.248+0000 7f10acec1640 1 -- 172.21.15.38:0/386360658 learned_addr learned my addr 172.21.15.38:0/386360658 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:43:27.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.248+0000 7f10a7fff640 1 --2- 172.21.15.38:0/386360658 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555d75e2e490 0x555d762c5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:27.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.248+0000 7f10a77fe640 1 --2- 172.21.15.38:0/386360658 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555d75e36670 0x555d761f84f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:27.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.248+0000 7f10acec1640 1 -- 172.21.15.38:0/386360658 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555d75e2e490 msgr2=0x555d762c5d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:27.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.248+0000 7f10acec1640 1 --2- 172.21.15.38:0/386360658 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555d75e2e490 0x555d762c5d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:27.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.248+0000 7f10acec1640 1 -- 172.21.15.38:0/386360658 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555d75e36670 msgr2=0x555d761f84f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:27.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.248+0000 7f10acec1640 1 --2- 172.21.15.38:0/386360658 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555d75e36670 0x555d761f84f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:27.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.248+0000 7f10acec1640 1 -- 172.21.15.38:0/386360658 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d75ea0660 con 0x555d75e57350 2024-09-21T03:43:27.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.249+0000 7f10acec1640 1 --2- 172.21.15.38:0/386360658 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555d75e57350 0x555d761f8a30 secure :-1 s=READY pgs=2021 cs=0 l=1 rev1=1 crypto rx=0x7f10a00015c0 tx=0x7f10a001c6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e30eec24452ec58 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:27.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.249+0000 7f1086ffd640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-21T03:43:27.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.249+0000 7f1086ffd640 1 -- 172.21.15.38:0/386360658 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10a0037060 con 0x555d75e57350 2024-09-21T03:43:27.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.249+0000 7f10b588e740 1 -- 172.21.15.38:0/386360658 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f10a0012a00 con 0x555d75e57350 2024-09-21T03:43:27.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.249+0000 7f1086ffd640 1 -- 172.21.15.38:0/386360658 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(127..127 src has 1..127) ==== 8970+0+0 (secure 0 0 0) 0x7f10a001ca60 con 0x555d75e57350 2024-09-21T03:43:27.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.249+0000 7f1086ffd640 1 -- 172.21.15.38:0/386360658 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=275) ==== 24+0+0 (secure 0 0 0) 0x7f10a003e050 con 0x555d75e57350 2024-09-21T03:43:27.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.249+0000 7f10b588e740 10 client.9897 fetch_fsmap learned FSMap version 275 2024-09-21T03:43:27.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.249+0000 7f10b588e740 1 -- 172.21.15.38:0/386360658 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=275,monmap=2+}) -- 0x555d7623b640 con 0x555d75e57350 2024-09-21T03:43:27.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.249+0000 7f1086ffd640 1 -- 172.21.15.38:0/386360658 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f10a0012a00 con 0x555d75e57350 2024-09-21T03:43:27.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.249+0000 7f1086ffd640 1 -- 172.21.15.38:0/386360658 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 275) ==== 3502+0+0 (secure 0 0 0) 0x7f10a001cc60 con 0x555d75e57350 2024-09-21T03:43:27.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.249+0000 7f10b588e740 10 client.9897 fetch_fsmap finished waiting for FSMap version 275 2024-09-21T03:43:27.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.249+0000 7f10b588e740 10 client.9897 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T03:43:27.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.249+0000 7f10b588e740 20 client.9897 populate_metadata read hostname 'smithi038' 2024-09-21T03:43:27.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.249+0000 7f10b588e740 1 --2- 172.21.15.38:0/386360658 >> [v2:172.21.15.38:6832/2309772457,v1:172.21.15.38:6833/2309772457] conn(0x555d76207170 0x555d76207580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:27.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.249+0000 7f10b588e740 4 client.9897 mds_command: new command op to 9786 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-09-21T03:43:27.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.249+0000 7f10b588e740 1 -- 172.21.15.38:0/386360658 --> [v2:172.21.15.38:6832/2309772457,v1:172.21.15.38:6833/2309772457] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) -- 0x555d7616f3a0 con 0x555d76207170 2024-09-21T03:43:27.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.249+0000 7f10a7fff640 1 --2- 172.21.15.38:0/386360658 >> [v2:172.21.15.38:6832/2309772457,v1:172.21.15.38:6833/2309772457] conn(0x555d76207170 0x555d76207580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:27.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.250+0000 7f1086ffd640 10 client.9897 ms_handle_connect on v2:172.21.15.38:6832/2309772457 2024-09-21T03:43:27.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.250+0000 7f10a7fff640 1 --2- 172.21.15.38:0/386360658 >> [v2:172.21.15.38:6832/2309772457,v1:172.21.15.38:6833/2309772457] conn(0x555d76207170 0x555d76207580 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b17a85737fb77120 server_cookie=f2d78f3a809e21a4 in_seq=0 out_seq=0 2024-09-21T03:43:27.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.250+0000 7f1086ffd640 1 -- 172.21.15.38:0/386360658 <== mds.0 v2:172.21.15.38:6832/2309772457 1 ==== command_reply(tid 0: 0 ) ==== 8+0+215 (crc 0 0 0) 0x555d7616f3a0 con 0x555d76207170 2024-09-21T03:43:27.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.250+0000 7f1086ffd640 10 client.9897 handle_command_reply: tid=0 2024-09-21T03:43:27.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.250+0000 7f10b588e740 1 client.9897 shutdown 2024-09-21T03:43:27.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.250+0000 7f10b588e740 1 -- 172.21.15.38:0/386360658 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555d75e57350 msgr2=0x555d761f8a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:27.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.250+0000 7f10b588e740 1 --2- 172.21.15.38:0/386360658 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555d75e57350 0x555d761f8a30 secure :-1 s=READY pgs=2021 cs=0 l=1 rev1=1 crypto rx=0x7f10a00015c0 tx=0x7f10a001c6e0 comp rx=0 tx=0).stop 2024-09-21T03:43:27.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.250+0000 7f10b588e740 1 -- 172.21.15.38:0/386360658 shutdown_connections 2024-09-21T03:43:27.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.250+0000 7f10b588e740 1 --2- 172.21.15.38:0/386360658 >> [v2:172.21.15.38:6832/2309772457,v1:172.21.15.38:6833/2309772457] conn(0x555d76207170 0x555d76207580 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:27.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.250+0000 7f10b588e740 1 --2- 172.21.15.38:0/386360658 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555d75e57350 0x555d761f8a30 unknown :-1 s=CLOSED pgs=2021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:27.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.250+0000 7f10b588e740 1 --2- 172.21.15.38:0/386360658 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555d75e36670 0x555d761f84f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:27.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.250+0000 7f10b588e740 1 --2- 172.21.15.38:0/386360658 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555d75e2e490 0x555d762c5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:27.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.250+0000 7f1086ffd640 0 client.9897 ms_handle_reset on v2:172.21.15.38:6832/2309772457 2024-09-21T03:43:27.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.250+0000 7f10b588e740 1 -- 172.21.15.38:0/386360658 >> 172.21.15.38:0/386360658 conn(0x555d75e4b4a0 msgr2=0x555d762276e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:27.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.250+0000 7f10b588e740 1 -- 172.21.15.38:0/386360658 shutdown_connections 2024-09-21T03:43:27.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.250+0000 7f10b588e740 1 -- 172.21.15.38:0/386360658 wait complete. 2024-09-21T03:43:27.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.251+0000 7f10b588e740 20 client.9897 trim_cache size 0 max 16384 2024-09-21T03:43:27.257 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:43:27.258 INFO:teuthology.orchestra.run.smithi038.stdout:[ 2024-09-21T03:43:27.258 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-21T03:43:27.258 INFO:teuthology.orchestra.run.smithi038.stdout: "damage_type": "dentry", 2024-09-21T03:43:27.258 INFO:teuthology.orchestra.run.smithi038.stdout: "id": 1280492150, 2024-09-21T03:43:27.258 INFO:teuthology.orchestra.run.smithi038.stdout: "ino": 1099511627776, 2024-09-21T03:43:27.258 INFO:teuthology.orchestra.run.smithi038.stdout: "frag": "*", 2024-09-21T03:43:27.258 INFO:teuthology.orchestra.run.smithi038.stdout: "dname": "sixmegs", 2024-09-21T03:43:27.258 INFO:teuthology.orchestra.run.smithi038.stdout: "snap_id": "head", 2024-09-21T03:43:27.258 INFO:teuthology.orchestra.run.smithi038.stdout: "path": "/subdir/sixmegs" 2024-09-21T03:43:27.258 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-21T03:43:27.258 INFO:teuthology.orchestra.run.smithi038.stdout:] 2024-09-21T03:43:27.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.251+0000 7f10ae924640 1 -- 172.21.15.38:0/2636472082 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1094078b90 msgr2=0x7f109407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:27.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.251+0000 7f10ae924640 1 --2- 172.21.15.38:0/2636472082 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1094078b90 0x7f109407b0a0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f1098003fb0 tx=0x7f1098003af0 comp rx=0 tx=0).stop 2024-09-21T03:43:27.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.251+0000 7f10ae924640 1 -- 172.21.15.38:0/2636472082 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f10a80cdf00 msgr2=0x7f10a81f8400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:27.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.251+0000 7f10ae924640 1 --2- 172.21.15.38:0/2636472082 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f10a80cdf00 0x7f10a81f8400 secure :-1 s=READY pgs=2017 cs=0 l=1 rev1=1 crypto rx=0x7f109c00bbe0 tx=0x7f109c02f4f0 comp rx=0 tx=0).stop 2024-09-21T03:43:27.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.251+0000 7f10ae924640 1 -- 172.21.15.38:0/2636472082 shutdown_connections 2024-09-21T03:43:27.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.251+0000 7f10ae924640 1 --2- 172.21.15.38:0/2636472082 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f10a809d010 0x7f10a809d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:27.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.251+0000 7f10ae924640 1 --2- 172.21.15.38:0/2636472082 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f10a80cdf00 0x7f10a81f8400 unknown :-1 s=CLOSED pgs=2017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:27.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.251+0000 7f10ae924640 1 --2- 172.21.15.38:0/2636472082 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1094078b90 0x7f109407b0a0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:27.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.251+0000 7f10ae924640 1 --2- 172.21.15.38:0/2636472082 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1090004090 0x7f10a80b2ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:27.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.251+0000 7f10ae924640 1 -- 172.21.15.38:0/2636472082 >> 172.21.15.38:0/2636472082 conn(0x7f10a80b1b90 msgr2=0x7f10a8158410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:27.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.251+0000 7f10ae924640 1 -- 172.21.15.38:0/2636472082 shutdown_connections 2024-09-21T03:43:27.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.251+0000 7f10ae924640 1 -- 172.21.15.38:0/2636472082 wait complete. 2024-09-21T03:43:27.264 INFO:tasks.cephfs.test_damage:Applying mutation 'Corrupt omap key 100.00000000:stray0_head' 2024-09-21T03:43:27.264 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-21T03:43:27.264 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:43:27.280 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-21T03:43:27.280 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:43:27.337 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T03:43:27.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.493+0000 7f05a0b58640 1 -- 172.21.15.38:0/1701850507 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f059c15ca90 msgr2=0x7f059c0d0670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:27.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.493+0000 7f05a0b58640 1 --2- 172.21.15.38:0/1701850507 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f059c15ca90 0x7f059c0d0670 secure :-1 s=READY pgs=2022 cs=0 l=1 rev1=1 crypto rx=0x7f059c0cd050 tx=0x7f059002a4d0 comp rx=0 tx=0).stop 2024-09-21T03:43:27.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.493+0000 7f05a0b58640 1 -- 172.21.15.38:0/1701850507 shutdown_connections 2024-09-21T03:43:27.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.493+0000 7f05a0b58640 1 --2- 172.21.15.38:0/1701850507 >> v2:172.21.15.195:3300/0 conn(0x7f059c0d0bb0 0x7f059c0b5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:27.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.493+0000 7f05a0b58640 1 --2- 172.21.15.38:0/1701850507 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f059c15ca90 0x7f059c0d0670 unknown :-1 s=CLOSED pgs=2022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:27.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.493+0000 7f05a0b58640 1 -- 172.21.15.38:0/1701850507 >> 172.21.15.38:0/1701850507 conn(0x7f059c0b46e0 msgr2=0x7f059c0d23e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:27.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.493+0000 7f05a0b58640 1 -- 172.21.15.38:0/1701850507 shutdown_connections 2024-09-21T03:43:27.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.493+0000 7f05a0b58640 1 -- 172.21.15.38:0/1701850507 wait complete. 2024-09-21T03:43:27.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.495+0000 7f05a0b58640 1 Processor -- start 2024-09-21T03:43:27.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.495+0000 7f05a0b58640 1 -- start start 2024-09-21T03:43:27.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.497+0000 7f05a0b58640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f059c0ccc40 0x7f059c15a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:27.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.497+0000 7f05a0b58640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f059c0d0bb0 0x7f059c0cbb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:27.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.497+0000 7f05a0b58640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f059c15ca90 0x7f059c0cc0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:27.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.497+0000 7f05a0b58640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f059c0b6670 con 0x7f059c0ccc40 2024-09-21T03:43:27.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.497+0000 7f05a0b58640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f059c0b64f0 con 0x7f059c0d0bb0 2024-09-21T03:43:27.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.497+0000 7f05a0b58640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f059c0b67f0 con 0x7f059c15ca90 2024-09-21T03:43:27.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.497+0000 7f059a59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f059c0ccc40 0x7f059c15a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:27.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.497+0000 7f059a59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f059c0ccc40 0x7f059c15a420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:58520/0 (socket says 172.21.15.38:58520) 2024-09-21T03:43:27.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.497+0000 7f059a59c640 1 -- 172.21.15.38:0/768492290 learned_addr learned my addr 172.21.15.38:0/768492290 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:43:27.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.497+0000 7f0599d9b640 1 --2- 172.21.15.38:0/768492290 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f059c0d0bb0 0x7f059c0cbb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:27.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.497+0000 7f059ad9d640 1 --2- 172.21.15.38:0/768492290 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f059c15ca90 0x7f059c0cc0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:27.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.497+0000 7f059a59c640 1 -- 172.21.15.38:0/768492290 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f059c15ca90 msgr2=0x7f059c0cc0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:27.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.497+0000 7f059a59c640 1 --2- 172.21.15.38:0/768492290 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f059c15ca90 0x7f059c0cc0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:27.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.497+0000 7f059a59c640 1 -- 172.21.15.38:0/768492290 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f059c0d0bb0 msgr2=0x7f059c0cbb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:27.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.497+0000 7f059a59c640 1 --2- 172.21.15.38:0/768492290 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f059c0d0bb0 0x7f059c0cbb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:27.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.497+0000 7f059a59c640 1 -- 172.21.15.38:0/768492290 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f059c0a9ae0 con 0x7f059c0ccc40 2024-09-21T03:43:27.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.497+0000 7f059a59c640 1 --2- 172.21.15.38:0/768492290 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f059c0ccc40 0x7f059c15a420 secure :-1 s=READY pgs=2023 cs=0 l=1 rev1=1 crypto rx=0x7f058c00ba80 tx=0x7f058c00bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:27.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.498+0000 7f058affd640 1 -- 172.21.15.38:0/768492290 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f058c019070 con 0x7f059c0ccc40 2024-09-21T03:43:27.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.498+0000 7f058affd640 1 -- 172.21.15.38:0/768492290 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f058c00cd80 con 0x7f059c0ccc40 2024-09-21T03:43:27.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.498+0000 7f05a0b58640 1 -- 172.21.15.38:0/768492290 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f059c0a9dc0 con 0x7f059c0ccc40 2024-09-21T03:43:27.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.498+0000 7f058affd640 1 -- 172.21.15.38:0/768492290 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f058c0146a0 con 0x7f059c0ccc40 2024-09-21T03:43:27.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.498+0000 7f05a0b58640 1 -- 172.21.15.38:0/768492290 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f059c0aa380 con 0x7f059c0ccc40 2024-09-21T03:43:27.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.499+0000 7f05a0b58640 1 -- 172.21.15.38:0/768492290 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f059c0b6de0 con 0x7f059c0ccc40 2024-09-21T03:43:27.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.499+0000 7f058affd640 1 -- 172.21.15.38:0/768492290 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f058c01d3f0 con 0x7f059c0ccc40 2024-09-21T03:43:27.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.500+0000 7f058affd640 1 --2- 172.21.15.38:0/768492290 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f057c078bc0 0x7f057c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:27.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.500+0000 7f058affd640 1 -- 172.21.15.38:0/768492290 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 8970+0+0 (secure 0 0 0) 0x7f058c09b650 con 0x7f059c0ccc40 2024-09-21T03:43:27.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.500+0000 7f0599d9b640 1 --2- 172.21.15.38:0/768492290 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f057c078bc0 0x7f057c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:27.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.501+0000 7f0599d9b640 1 --2- 172.21.15.38:0/768492290 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f057c078bc0 0x7f057c07b0d0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f0590033000 tx=0x7f0590004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:27.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.505+0000 7f058affd640 1 -- 172.21.15.38:0/768492290 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f058c062730 con 0x7f059c0ccc40 2024-09-21T03:43:27.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.705+0000 7f05a0b58640 1 -- 172.21.15.38:0/768492290 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f059c0cc800 con 0x7f059c0ccc40 2024-09-21T03:43:27.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.706+0000 7f058affd640 1 -- 172.21.15.38:0/768492290 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 62 entries v127) ==== 81+0+3768 (secure 0 0 0) 0x7f058c067160 con 0x7f059c0ccc40 2024-09-21T03:43:27.708 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6835/141447142 2024-09-22T03:43:14.684929+0000 2024-09-21T03:43:27.708 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6834/141447142 2024-09-22T03:43:14.684929+0000 2024-09-21T03:43:27.708 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6835/3572946698 2024-09-22T03:41:44.737273+0000 2024-09-21T03:43:27.708 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6833/3553953266 2024-09-22T03:41:29.639383+0000 2024-09-21T03:43:27.708 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6834/2751962827 2024-09-22T03:40:38.029580+0000 2024-09-21T03:43:27.708 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6832/3553953266 2024-09-22T03:41:29.639383+0000 2024-09-21T03:43:27.708 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6834/2930805359 2024-09-22T03:42:23.215190+0000 2024-09-21T03:43:27.708 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6835/2748959970 2024-09-22T03:41:16.565736+0000 2024-09-21T03:43:27.708 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6833/3710736169 2024-09-22T03:40:59.454480+0000 2024-09-21T03:43:27.709 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6833/1995903506 2024-09-22T03:43:00.406120+0000 2024-09-21T03:43:27.709 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6833/775682959 2024-09-22T03:43:10.570678+0000 2024-09-21T03:43:27.709 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6834/2748959970 2024-09-22T03:41:16.565736+0000 2024-09-21T03:43:27.709 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6834/2934569248 2024-09-22T03:40:26.934591+0000 2024-09-21T03:43:27.709 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6832/3726830644 2024-09-22T03:41:12.532341+0000 2024-09-21T03:43:27.709 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6835/2097789246 2024-09-22T03:43:06.604471+0000 2024-09-21T03:43:27.709 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6835/2934569248 2024-09-22T03:40:26.934591+0000 2024-09-21T03:43:27.709 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6835/3864487171 2024-09-22T03:40:31.989837+0000 2024-09-21T03:43:27.709 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6833/4221150556 2024-09-22T03:42:27.240627+0000 2024-09-21T03:43:27.709 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6832/121492560 2024-09-22T03:41:10.371933+0000 2024-09-21T03:43:27.709 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6832/1717955365 2024-09-22T03:41:51.787804+0000 2024-09-21T03:43:27.709 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6834/2418225210 2024-09-22T03:42:02.869612+0000 2024-09-21T03:43:27.709 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6835/2418225210 2024-09-22T03:42:02.869612+0000 2024-09-21T03:43:27.710 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6833/121492560 2024-09-22T03:41:10.371933+0000 2024-09-21T03:43:27.710 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6835/3951934534 2024-09-22T03:42:31.276099+0000 2024-09-21T03:43:27.710 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6833/751618334 2024-09-22T03:40:42.053279+0000 2024-09-21T03:43:27.710 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.195:6837/712370966 2024-09-22T03:40:11.633787+0000 2024-09-21T03:43:27.710 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6832/3402963947 2024-09-22T03:41:23.555005+0000 2024-09-21T03:43:27.710 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6835/2798021708 2024-09-22T03:42:47.436632+0000 2024-09-21T03:43:27.710 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6833/3726830644 2024-09-22T03:41:12.532341+0000 2024-09-21T03:43:27.710 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6834/2916238332 2024-09-22T03:41:42.553770+0000 2024-09-21T03:43:27.710 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6834/3055025811 2024-09-22T03:42:39.361236+0000 2024-09-21T03:43:27.710 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.195:6835/712370966 2024-09-22T03:40:11.633787+0000 2024-09-21T03:43:27.710 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6834/3864487171 2024-09-22T03:40:31.989837+0000 2024-09-21T03:43:27.710 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6834/3572946698 2024-09-22T03:41:44.737273+0000 2024-09-21T03:43:27.711 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6832/1995903506 2024-09-22T03:43:00.406120+0000 2024-09-21T03:43:27.711 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6832/775682959 2024-09-22T03:43:10.570678+0000 2024-09-21T03:43:27.711 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6835/2916238332 2024-09-22T03:41:42.553770+0000 2024-09-21T03:43:27.711 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6833/3402963947 2024-09-22T03:41:23.555005+0000 2024-09-21T03:43:27.711 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6834/425672974 2024-09-22T03:40:46.105521+0000 2024-09-21T03:43:27.711 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6834/3951934534 2024-09-22T03:42:31.276099+0000 2024-09-21T03:43:27.711 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6835/425672974 2024-09-22T03:40:46.105521+0000 2024-09-21T03:43:27.711 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6832/3710736169 2024-09-22T03:40:59.454480+0000 2024-09-21T03:43:27.711 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6833/1717955365 2024-09-22T03:41:51.787804+0000 2024-09-21T03:43:27.711 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6835/3055025811 2024-09-22T03:42:39.361236+0000 2024-09-21T03:43:27.711 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6835/2751962827 2024-09-22T03:40:38.029580+0000 2024-09-21T03:43:27.711 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6835/2712360296 2024-09-22T03:42:20.931115+0000 2024-09-21T03:43:27.711 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6833/3537258034 2024-09-22T03:42:06.901189+0000 2024-09-21T03:43:27.711 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6832/3537258034 2024-09-22T03:42:06.901189+0000 2024-09-21T03:43:27.712 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6832/1272005650 2024-09-22T03:41:58.836862+0000 2024-09-21T03:43:27.712 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6834/2712360296 2024-09-22T03:42:20.931115+0000 2024-09-21T03:43:27.712 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6832/751618334 2024-09-22T03:40:42.053279+0000 2024-09-21T03:43:27.712 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6835/2930805359 2024-09-22T03:42:23.215190+0000 2024-09-21T03:43:27.712 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6832/4221150556 2024-09-22T03:42:27.240627+0000 2024-09-21T03:43:27.712 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6833/1272005650 2024-09-22T03:41:58.836862+0000 2024-09-21T03:43:27.712 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6832/245538755 2024-09-22T03:42:35.318756+0000 2024-09-21T03:43:27.712 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6833/245538755 2024-09-22T03:42:35.318756+0000 2024-09-21T03:43:27.712 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6833/2459386631 2024-09-22T03:42:43.395609+0000 2024-09-21T03:43:27.712 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6832/2459386631 2024-09-22T03:42:43.395609+0000 2024-09-21T03:43:27.712 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6834/2097789246 2024-09-22T03:43:06.604471+0000 2024-09-21T03:43:27.712 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6834/2798021708 2024-09-22T03:42:47.436632+0000 2024-09-21T03:43:27.712 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6832/111206731 2024-09-22T03:43:02.560071+0000 2024-09-21T03:43:27.713 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6833/111206731 2024-09-22T03:43:02.560071+0000 2024-09-21T03:43:27.713 INFO:teuthology.orchestra.run.smithi038.stderr:listed 62 entries 2024-09-21T03:43:27.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.708+0000 7f05a0b58640 1 -- 172.21.15.38:0/768492290 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f057c078bc0 msgr2=0x7f057c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:27.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.708+0000 7f05a0b58640 1 --2- 172.21.15.38:0/768492290 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f057c078bc0 0x7f057c07b0d0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f0590033000 tx=0x7f0590004730 comp rx=0 tx=0).stop 2024-09-21T03:43:27.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.708+0000 7f05a0b58640 1 -- 172.21.15.38:0/768492290 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f059c0ccc40 msgr2=0x7f059c15a420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:27.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.708+0000 7f05a0b58640 1 --2- 172.21.15.38:0/768492290 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f059c0ccc40 0x7f059c15a420 secure :-1 s=READY pgs=2023 cs=0 l=1 rev1=1 crypto rx=0x7f058c00ba80 tx=0x7f058c00bf40 comp rx=0 tx=0).stop 2024-09-21T03:43:27.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.708+0000 7f05a0b58640 1 -- 172.21.15.38:0/768492290 shutdown_connections 2024-09-21T03:43:27.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.708+0000 7f05a0b58640 1 --2- 172.21.15.38:0/768492290 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f057c078bc0 0x7f057c07b0d0 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:27.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.708+0000 7f05a0b58640 1 --2- 172.21.15.38:0/768492290 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f059c15ca90 0x7f059c0cc0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:27.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.708+0000 7f05a0b58640 1 --2- 172.21.15.38:0/768492290 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f059c0d0bb0 0x7f059c0cbb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:27.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.708+0000 7f05a0b58640 1 --2- 172.21.15.38:0/768492290 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f059c0ccc40 0x7f059c15a420 unknown :-1 s=CLOSED pgs=2023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:27.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.708+0000 7f05a0b58640 1 -- 172.21.15.38:0/768492290 >> 172.21.15.38:0/768492290 conn(0x7f059c0b46e0 msgr2=0x7f059c0b3280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:27.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.708+0000 7f05a0b58640 1 -- 172.21.15.38:0/768492290 shutdown_connections 2024-09-21T03:43:27.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.708+0000 7f05a0b58640 1 -- 172.21.15.38:0/768492290 wait complete. 2024-09-21T03:43:27.722 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-21T03:43:27.722 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:43:27.722 DEBUG:teuthology.orchestra.run.smithi038:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-09-21T03:43:27.749 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi038.front.sepia.ceph.com 2024-09-21T03:43:27.749 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:43:27.749 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T03:43:27.805 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-21T03:43:27.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.960+0000 7f33ca19d640 1 -- 172.21.15.38:0/2556956223 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f33ac004d40 msgr2=0x7f33ac005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:27.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.960+0000 7f33ca19d640 1 --2- 172.21.15.38:0/2556956223 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f33ac004d40 0x7f33ac005150 secure :-1 s=READY pgs=2025 cs=0 l=1 rev1=1 crypto rx=0x7f33bc00c4a0 tx=0x7f33ac005830 comp rx=0 tx=0).stop 2024-09-21T03:43:27.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.960+0000 7f33c999c640 1 -- 172.21.15.38:0/2556956223 reap_dead start 2024-09-21T03:43:27.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.960+0000 7f33ca19d640 1 -- 172.21.15.38:0/2556956223 shutdown_connections 2024-09-21T03:43:27.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.960+0000 7f33ca19d640 1 -- 172.21.15.38:0/2556956223 >> 172.21.15.38:0/2556956223 conn(0x7f33c40d41e0 msgr2=0x7f33c40d45f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:27.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.960+0000 7f33ca19d640 1 -- 172.21.15.38:0/2556956223 shutdown_connections 2024-09-21T03:43:27.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.960+0000 7f33ca19d640 1 -- 172.21.15.38:0/2556956223 wait complete. 2024-09-21T03:43:27.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.962+0000 7f33ca19d640 1 Processor -- start 2024-09-21T03:43:27.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.962+0000 7f33ca19d640 1 -- start start 2024-09-21T03:43:27.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.963+0000 7f33ca19d640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f33c41f85f0 0x7f33c41f8a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:27.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.963+0000 7f33ca19d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f33c40ccd30 0x7f33c40cd140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:27.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.963+0000 7f33ca19d640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f33c40cd680 0x7f33c414dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:27.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.963+0000 7f33ca19d640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f33c40d3420 con 0x7f33c40ccd30 2024-09-21T03:43:27.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.963+0000 7f33ca19d640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f33c40d32a0 con 0x7f33c40cd680 2024-09-21T03:43:27.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.963+0000 7f33ca19d640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f33c40d35a0 con 0x7f33c41f85f0 2024-09-21T03:43:27.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.964+0000 7f33c899a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f33c40ccd30 0x7f33c40cd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:27.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.964+0000 7f33c899a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f33c40ccd30 0x7f33c40cd140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:58534/0 (socket says 172.21.15.38:58534) 2024-09-21T03:43:27.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.964+0000 7f33c899a640 1 -- 172.21.15.38:0/348873771 learned_addr learned my addr 172.21.15.38:0/348873771 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:43:27.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.964+0000 7f33c999c640 1 --2- 172.21.15.38:0/348873771 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f33c40cd680 0x7f33c414dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:27.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.964+0000 7f33c919b640 1 --2- 172.21.15.38:0/348873771 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f33c41f85f0 0x7f33c41f8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:27.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.964+0000 7f33c899a640 1 -- 172.21.15.38:0/348873771 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f33c41f85f0 msgr2=0x7f33c41f8a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:27.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.964+0000 7f33c899a640 1 --2- 172.21.15.38:0/348873771 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f33c41f85f0 0x7f33c41f8a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:27.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.964+0000 7f33c899a640 1 -- 172.21.15.38:0/348873771 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f33c40cd680 msgr2=0x7f33c414dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:27.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.964+0000 7f33c899a640 1 --2- 172.21.15.38:0/348873771 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f33c40cd680 0x7f33c414dcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:27.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.964+0000 7f33c899a640 1 -- 172.21.15.38:0/348873771 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33c41f90f0 con 0x7f33c40ccd30 2024-09-21T03:43:27.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.964+0000 7f33c899a640 1 --2- 172.21.15.38:0/348873771 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f33c40ccd30 0x7f33c40cd140 secure :-1 s=READY pgs=2026 cs=0 l=1 rev1=1 crypto rx=0x7f33b8011a50 tx=0x7f33b8011f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:27.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.964+0000 7f33c1ffb640 1 -- 172.21.15.38:0/348873771 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33b800fcd0 con 0x7f33c40ccd30 2024-09-21T03:43:27.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.964+0000 7f33c1ffb640 1 -- 172.21.15.38:0/348873771 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f33b8004020 con 0x7f33c40ccd30 2024-09-21T03:43:27.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.964+0000 7f33c1ffb640 1 -- 172.21.15.38:0/348873771 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33b8004430 con 0x7f33c40ccd30 2024-09-21T03:43:27.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.964+0000 7f33ca19d640 1 -- 172.21.15.38:0/348873771 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33c41f93d0 con 0x7f33c40ccd30 2024-09-21T03:43:27.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.965+0000 7f33ca19d640 1 -- 172.21.15.38:0/348873771 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33c41f5e70 con 0x7f33c40ccd30 2024-09-21T03:43:27.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.965+0000 7f33ca19d640 1 -- 172.21.15.38:0/348873771 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33c40cf310 con 0x7f33c40ccd30 2024-09-21T03:43:27.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.966+0000 7f33c1ffb640 1 -- 172.21.15.38:0/348873771 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f33b80035f0 con 0x7f33c40ccd30 2024-09-21T03:43:27.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.967+0000 7f33c1ffb640 1 --2- 172.21.15.38:0/348873771 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f33b0078b90 0x7f33b007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:27.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.967+0000 7f33c1ffb640 1 -- 172.21.15.38:0/348873771 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 8970+0+0 (secure 0 0 0) 0x7f33b809c740 con 0x7f33c40ccd30 2024-09-21T03:43:27.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.967+0000 7f33c919b640 1 --2- 172.21.15.38:0/348873771 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f33b0078b90 0x7f33b007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:27.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.968+0000 7f33c919b640 1 --2- 172.21.15.38:0/348873771 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f33b0078b90 0x7f33b007b0a0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f33c415c4a0 tx=0x7f33b4009500 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:27.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:27.971+0000 7f33c1ffb640 1 -- 172.21.15.38:0/348873771 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f33b8063ea0 con 0x7f33c40ccd30 2024-09-21T03:43:28.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:28.197+0000 7f33ca19d640 1 -- 172.21.15.38:0/348873771 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f33c40b7ab0 con 0x7f33c40ccd30 2024-09-21T03:43:28.199 INFO:tasks.ceph.mon.a.smithi038.stderr:2024-09-21T03:43:28.197+0000 7f3f45713640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T03:43:28.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:28.199+0000 7f33c1ffb640 1 -- 172.21.15.38:0/348873771 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v276) ==== 161+0+0 (secure 0 0 0) 0x7f33b80637a0 con 0x7f33c40ccd30 2024-09-21T03:43:28.201 INFO:teuthology.orchestra.run.smithi038.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-21T03:43:28.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:28.202+0000 7f33ca19d640 1 -- 172.21.15.38:0/348873771 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f33b0078b90 msgr2=0x7f33b007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:28.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:28.202+0000 7f33ca19d640 1 --2- 172.21.15.38:0/348873771 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f33b0078b90 0x7f33b007b0a0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f33c415c4a0 tx=0x7f33b4009500 comp rx=0 tx=0).stop 2024-09-21T03:43:28.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:28.202+0000 7f33ca19d640 1 -- 172.21.15.38:0/348873771 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f33c40ccd30 msgr2=0x7f33c40cd140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:28.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:28.202+0000 7f33ca19d640 1 --2- 172.21.15.38:0/348873771 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f33c40ccd30 0x7f33c40cd140 secure :-1 s=READY pgs=2026 cs=0 l=1 rev1=1 crypto rx=0x7f33b8011a50 tx=0x7f33b8011f10 comp rx=0 tx=0).stop 2024-09-21T03:43:28.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:28.203+0000 7f33ca19d640 1 -- 172.21.15.38:0/348873771 shutdown_connections 2024-09-21T03:43:28.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:28.203+0000 7f33ca19d640 1 --2- 172.21.15.38:0/348873771 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f33c40cd680 0x7f33c414dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:28.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:28.203+0000 7f33ca19d640 1 --2- 172.21.15.38:0/348873771 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f33c40ccd30 0x7f33c40cd140 unknown :-1 s=CLOSED pgs=2026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:28.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:28.203+0000 7f33ca19d640 1 --2- 172.21.15.38:0/348873771 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f33b0078b90 0x7f33b007b0a0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:28.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:28.203+0000 7f33ca19d640 1 --2- 172.21.15.38:0/348873771 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f33c41f85f0 0x7f33c41f8a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:28.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:28.203+0000 7f33ca19d640 1 -- 172.21.15.38:0/348873771 >> 172.21.15.38:0/348873771 conn(0x7f33c40d41e0 msgr2=0x7f33c4161710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:28.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:28.203+0000 7f33ca19d640 1 -- 172.21.15.38:0/348873771 shutdown_connections 2024-09-21T03:43:28.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:28.203+0000 7f33ca19d640 1 -- 172.21.15.38:0/348873771 wait complete. 2024-09-21T03:43:28.206 INFO:tasks.ceph.mds.c.smithi038.stderr: -574> 2024-09-21T03:43:25.696+0000 7fe53d395640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T03:43:28.217 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-09-21T03:43:28.264 INFO:tasks.ceph.mds.c.smithi038.stdout:starting mds.c at 2024-09-21T03:43:28.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:28.336+0000 7f3de59da640 1 -- 172.21.15.38:0/2744112964 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3dcc0052d0 msgr2=0x7f3dcc00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:28.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:28.336+0000 7f3de59da640 1 --2- 172.21.15.38:0/2744112964 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3dcc0052d0 0x7f3dcc00b9c0 secure :-1 s=READY pgs=2031 cs=0 l=1 rev1=1 crypto rx=0x7f3dd8001f00 tx=0x7f3dcc005830 comp rx=0 tx=0).stop 2024-09-21T03:43:28.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:28.336+0000 7f3ddf7fe640 1 -- 172.21.15.38:0/2744112964 reap_dead start 2024-09-21T03:43:28.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:28.336+0000 7f3de59da640 1 -- 172.21.15.38:0/2744112964 shutdown_connections 2024-09-21T03:43:28.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:28.336+0000 7f3de59da640 1 -- 172.21.15.38:0/2744112964 >> 172.21.15.38:0/2744112964 conn(0x7f3de00b46e0 msgr2=0x7f3de00d23e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:28.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:28.336+0000 7f3de59da640 1 -- 172.21.15.38:0/2744112964 shutdown_connections 2024-09-21T03:43:28.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:28.336+0000 7f3de59da640 1 -- 172.21.15.38:0/2744112964 wait complete. 2024-09-21T03:43:28.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:28.338+0000 7f3de59da640 1 Processor -- start 2024-09-21T03:43:28.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:28.338+0000 7f3de59da640 1 -- start start 2024-09-21T03:43:28.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:28.339+0000 7f3de59da640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3dcc0052d0 0x7f3de01f3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:28.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:28.339+0000 7f3de59da640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3de00a9e40 0x7f3de00caab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:28.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:28.339+0000 7f3de59da640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3de015a8e0 0x7f3de00c9d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:28.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:28.339+0000 7f3de59da640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f3de00b62f0 con 0x7f3de00a9e40 2024-09-21T03:43:28.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:28.339+0000 7f3de59da640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f3de00d0990 con 0x7f3de015a8e0 2024-09-21T03:43:28.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:28.339+0000 7f3de59da640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f3de00b6470 con 0x7f3dcc0052d0 2024-09-21T03:43:28.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:28.339+0000 7f3dd7fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3de00a9e40 0x7f3de00caab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:28.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:28.340+0000 7f3dd7fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3de00a9e40 0x7f3de00caab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:58572/0 (socket says 172.21.15.38:58572) 2024-09-21T03:43:28.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:28.340+0000 7f3dd7fff640 1 -- 172.21.15.38:0/1651749260 learned_addr learned my addr 172.21.15.38:0/1651749260 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:43:28.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:28.340+0000 7f3ddeffd640 1 --2- 172.21.15.38:0/1651749260 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3dcc0052d0 0x7f3de01f3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:28.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:28.340+0000 7f3ddf7fe640 1 --2- 172.21.15.38:0/1651749260 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3de015a8e0 0x7f3de00c9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:28.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:28.340+0000 7f3dd7fff640 1 -- 172.21.15.38:0/1651749260 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3dcc0052d0 msgr2=0x7f3de01f3bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:28.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:28.340+0000 7f3dd7fff640 1 --2- 172.21.15.38:0/1651749260 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3dcc0052d0 0x7f3de01f3bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:28.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:28.340+0000 7f3dd7fff640 1 -- 172.21.15.38:0/1651749260 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3de015a8e0 msgr2=0x7f3de00c9d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:28.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:28.340+0000 7f3dd7fff640 1 --2- 172.21.15.38:0/1651749260 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3de015a8e0 0x7f3de00c9d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:28.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:28.340+0000 7f3dd7fff640 1 -- 172.21.15.38:0/1651749260 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3de00a94f0 con 0x7f3de00a9e40 2024-09-21T03:43:28.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:28.340+0000 7f3dd7fff640 1 --2- 172.21.15.38:0/1651749260 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3de00a9e40 0x7f3de00caab0 secure :-1 s=READY pgs=2032 cs=0 l=1 rev1=1 crypto rx=0x7f3dc80119b0 tx=0x7f3dc8011e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:28.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:28.340+0000 7f3ddcff9640 1 -- 172.21.15.38:0/1651749260 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dc800fd30 con 0x7f3de00a9e40 2024-09-21T03:43:28.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:28.340+0000 7f3ddcff9640 1 -- 172.21.15.38:0/1651749260 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3dc8005020 con 0x7f3de00a9e40 2024-09-21T03:43:28.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:28.340+0000 7f3de59da640 1 -- 172.21.15.38:0/1651749260 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3de00a9ab0 con 0x7f3de00a9e40 2024-09-21T03:43:28.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:28.340+0000 7f3ddcff9640 1 -- 172.21.15.38:0/1651749260 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dc8005450 con 0x7f3de00a9e40 2024-09-21T03:43:28.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:28.340+0000 7f3de59da640 1 -- 172.21.15.38:0/1651749260 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3de00aa360 con 0x7f3de00a9e40 2024-09-21T03:43:28.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:28.341+0000 7f3de59da640 1 -- 172.21.15.38:0/1651749260 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3de00b6a00 con 0x7f3de00a9e40 2024-09-21T03:43:28.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:28.342+0000 7f3ddcff9640 1 -- 172.21.15.38:0/1651749260 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f3dc80047a0 con 0x7f3de00a9e40 2024-09-21T03:43:28.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:28.343+0000 7f3ddcff9640 1 --2- 172.21.15.38:0/1651749260 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3dc0078b90 0x7f3dc007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:28.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:28.343+0000 7f3ddcff9640 1 -- 172.21.15.38:0/1651749260 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 9056+0+0 (secure 0 0 0) 0x7f3dc809c760 con 0x7f3de00a9e40 2024-09-21T03:43:28.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:28.343+0000 7f3ddeffd640 1 --2- 172.21.15.38:0/1651749260 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3dc0078b90 0x7f3dc007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:28.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:28.344+0000 7f3ddeffd640 1 --2- 172.21.15.38:0/1651749260 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3dc0078b90 0x7f3dc007b0a0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f3dd0001410 tx=0x7f3dd00016c0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:28.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:28.347+0000 7f3ddcff9640 1 -- 172.21.15.38:0/1651749260 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f3dc80637f0 con 0x7f3de00a9e40 2024-09-21T03:43:28.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:28.610+0000 7f3de59da640 1 -- 172.21.15.38:0/1651749260 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7f3de00b7f20 con 0x7f3de00a9e40 2024-09-21T03:43:29.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.202+0000 7f3ddcff9640 1 -- 172.21.15.38:0/1651749260 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 nothing to do: rank is not damaged v277) ==== 107+0+0 (secure 0 0 0) 0x7f3dc8068220 con 0x7f3de00a9e40 2024-09-21T03:43:29.204 INFO:teuthology.orchestra.run.smithi038.stderr:nothing to do: rank is not damaged 2024-09-21T03:43:29.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.205+0000 7f3de59da640 1 -- 172.21.15.38:0/1651749260 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3dc0078b90 msgr2=0x7f3dc007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:29.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.205+0000 7f3de59da640 1 --2- 172.21.15.38:0/1651749260 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3dc0078b90 0x7f3dc007b0a0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f3dd0001410 tx=0x7f3dd00016c0 comp rx=0 tx=0).stop 2024-09-21T03:43:29.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.206+0000 7f3de59da640 1 -- 172.21.15.38:0/1651749260 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3de00a9e40 msgr2=0x7f3de00caab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:29.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.206+0000 7f3de59da640 1 --2- 172.21.15.38:0/1651749260 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3de00a9e40 0x7f3de00caab0 secure :-1 s=READY pgs=2032 cs=0 l=1 rev1=1 crypto rx=0x7f3dc80119b0 tx=0x7f3dc8011e70 comp rx=0 tx=0).stop 2024-09-21T03:43:29.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.206+0000 7f3de59da640 1 -- 172.21.15.38:0/1651749260 shutdown_connections 2024-09-21T03:43:29.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.206+0000 7f3de59da640 1 --2- 172.21.15.38:0/1651749260 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3de015a8e0 0x7f3de00c9d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:29.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.206+0000 7f3de59da640 1 --2- 172.21.15.38:0/1651749260 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3de00a9e40 0x7f3de00caab0 unknown :-1 s=CLOSED pgs=2032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:29.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.206+0000 7f3de59da640 1 --2- 172.21.15.38:0/1651749260 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3dc0078b90 0x7f3dc007b0a0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:29.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.206+0000 7f3de59da640 1 --2- 172.21.15.38:0/1651749260 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3dcc0052d0 0x7f3de01f3bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:29.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.206+0000 7f3de59da640 1 -- 172.21.15.38:0/1651749260 >> 172.21.15.38:0/1651749260 conn(0x7f3de00b46e0 msgr2=0x7f3de00c8840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:29.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.206+0000 7f3de59da640 1 -- 172.21.15.38:0/1651749260 shutdown_connections 2024-09-21T03:43:29.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.206+0000 7f3de59da640 1 -- 172.21.15.38:0/1651749260 wait complete. 2024-09-21T03:43:29.223 DEBUG:teuthology.orchestra.run.smithi038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-09-21T03:43:29.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.281+0000 7f90bf3b6f40 1 Processor -- start 2024-09-21T03:43:29.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.281+0000 7f90bf3b6f40 1 -- start start 2024-09-21T03:43:29.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.283+0000 7f90bf3b6f40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x557ba788d240 0x557ba7888850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:29.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.283+0000 7f90bf3b6f40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557ba78d4090 0x557ba79f4f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:29.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.283+0000 7f90bf3b6f40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557ba7949520 0x557ba79f74a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:29.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.283+0000 7f90bf3b6f40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x557ba7960aa0 con 0x557ba7949520 2024-09-21T03:43:29.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.283+0000 7f90bf3b6f40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x557ba77dad60 con 0x557ba78d4090 2024-09-21T03:43:29.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.283+0000 7f90bf3b6f40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x557ba78bb9b0 con 0x557ba788d240 2024-09-21T03:43:29.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.283+0000 7f90b9671640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557ba7949520 0x557ba79f74a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:29.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.283+0000 7f90b9671640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557ba7949520 0x557ba79f74a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:58580/0 (socket says 172.21.15.38:58580) 2024-09-21T03:43:29.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.283+0000 7f90b9671640 1 -- 172.21.15.38:0/2223220108 learned_addr learned my addr 172.21.15.38:0/2223220108 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:43:29.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.283+0000 7f90b8e70640 1 --2- 172.21.15.38:0/2223220108 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x557ba788d240 0x557ba7888850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:29.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.283+0000 7f90b866f640 1 --2- 172.21.15.38:0/2223220108 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557ba78d4090 0x557ba79f4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:29.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.283+0000 7f90b9671640 1 -- 172.21.15.38:0/2223220108 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x557ba788d240 msgr2=0x557ba7888850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:29.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.283+0000 7f90b9671640 1 --2- 172.21.15.38:0/2223220108 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x557ba788d240 0x557ba7888850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:29.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.283+0000 7f90b9671640 1 -- 172.21.15.38:0/2223220108 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557ba78d4090 msgr2=0x557ba79f4f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:29.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.283+0000 7f90b9671640 1 --2- 172.21.15.38:0/2223220108 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557ba78d4090 0x557ba79f4f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:29.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.283+0000 7f90b9671640 1 -- 172.21.15.38:0/2223220108 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ba78b6020 con 0x557ba7949520 2024-09-21T03:43:29.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.283+0000 7f90b866f640 1 --2- 172.21.15.38:0/2223220108 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557ba78d4090 0x557ba79f4f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:43:29.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.283+0000 7f90b9671640 1 --2- 172.21.15.38:0/2223220108 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557ba7949520 0x557ba79f74a0 secure :-1 s=READY pgs=2034 cs=0 l=1 rev1=1 crypto rx=0x7f90b000b510 tx=0x7f90b000b9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0e9fde50092e991 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:29.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.283+0000 7f90b766d640 1 -- 172.21.15.38:0/2223220108 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90b000d880 con 0x557ba7949520 2024-09-21T03:43:29.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.283+0000 7f90b766d640 1 -- 172.21.15.38:0/2223220108 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f90b000db10 con 0x557ba7949520 2024-09-21T03:43:29.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.283+0000 7f90b766d640 1 -- 172.21.15.38:0/2223220108 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90b000e450 con 0x557ba7949520 2024-09-21T03:43:29.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.284+0000 7f90bf3b6f40 1 -- 172.21.15.38:0/2223220108 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557ba7949520 msgr2=0x557ba79f74a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:29.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.284+0000 7f90bf3b6f40 1 --2- 172.21.15.38:0/2223220108 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557ba7949520 0x557ba79f74a0 secure :-1 s=READY pgs=2034 cs=0 l=1 rev1=1 crypto rx=0x7f90b000b510 tx=0x7f90b000b9d0 comp rx=0 tx=0).stop 2024-09-21T03:43:29.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.284+0000 7f90bf3b6f40 1 -- 172.21.15.38:0/2223220108 shutdown_connections 2024-09-21T03:43:29.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.284+0000 7f90bf3b6f40 1 --2- 172.21.15.38:0/2223220108 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557ba7949520 0x557ba79f74a0 unknown :-1 s=CLOSED pgs=2034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:29.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.284+0000 7f90bf3b6f40 1 --2- 172.21.15.38:0/2223220108 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557ba78d4090 0x557ba79f4f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:29.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.284+0000 7f90bf3b6f40 1 --2- 172.21.15.38:0/2223220108 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x557ba788d240 0x557ba7888850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:29.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.284+0000 7f90bf3b6f40 1 -- 172.21.15.38:0/2223220108 >> 172.21.15.38:0/2223220108 conn(0x557ba77abac0 msgr2=0x557ba78b6cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:29.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.284+0000 7f90bf3b6f40 1 -- 172.21.15.38:0/2223220108 shutdown_connections 2024-09-21T03:43:29.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.284+0000 7f90bf3b6f40 1 -- 172.21.15.38:0/2223220108 wait complete. 2024-09-21T03:43:29.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.285+0000 7f90bf3b6f40 1 Processor -- start 2024-09-21T03:43:29.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.285+0000 7f90bf3b6f40 1 -- start start 2024-09-21T03:43:29.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.286+0000 7f90bf3b6f40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x557ba788d240 0x557ba795a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:29.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.286+0000 7f90bf3b6f40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557ba78d4090 0x557ba7a014d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:29.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.286+0000 7f90bf3b6f40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557ba7949520 0x557ba79ea760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:29.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.286+0000 7f90bf3b6f40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x557ba78b6290 con 0x557ba7949520 2024-09-21T03:43:29.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.286+0000 7f90bf3b6f40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x557ba79fe7c0 con 0x557ba78d4090 2024-09-21T03:43:29.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.286+0000 7f90bf3b6f40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x557ba79595b0 con 0x557ba788d240 2024-09-21T03:43:29.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.287+0000 7f90b9671640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557ba7949520 0x557ba79ea760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:29.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.287+0000 7f90b9671640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557ba7949520 0x557ba79ea760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:58594/0 (socket says 172.21.15.38:58594) 2024-09-21T03:43:29.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.287+0000 7f90b9671640 1 -- 172.21.15.38:0/939184927 learned_addr learned my addr 172.21.15.38:0/939184927 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:43:29.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.287+0000 7f90b8e70640 1 --2- 172.21.15.38:0/939184927 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x557ba788d240 0x557ba795a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:29.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.287+0000 7f90b866f640 1 --2- 172.21.15.38:0/939184927 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557ba78d4090 0x557ba7a014d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:29.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.287+0000 7f90b9671640 1 -- 172.21.15.38:0/939184927 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x557ba788d240 msgr2=0x557ba795a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:29.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.287+0000 7f90b9671640 1 --2- 172.21.15.38:0/939184927 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x557ba788d240 0x557ba795a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:29.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.287+0000 7f90b9671640 1 -- 172.21.15.38:0/939184927 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557ba78d4090 msgr2=0x557ba7a014d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:29.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.287+0000 7f90b9671640 1 --2- 172.21.15.38:0/939184927 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557ba78d4090 0x557ba7a014d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:29.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.287+0000 7f90b9671640 1 -- 172.21.15.38:0/939184927 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ba7a01af0 con 0x557ba7949520 2024-09-21T03:43:29.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.287+0000 7f90b9671640 1 --2- 172.21.15.38:0/939184927 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557ba7949520 0x557ba79ea760 secure :-1 s=READY pgs=2035 cs=0 l=1 rev1=1 crypto rx=0x7f90b000e020 tx=0x7f90b000e450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:29.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.287+0000 7f90b666b640 1 -- 172.21.15.38:0/939184927 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90b000e840 con 0x557ba7949520 2024-09-21T03:43:29.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.287+0000 7f90b666b640 1 -- 172.21.15.38:0/939184927 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f90b000ea00 con 0x557ba7949520 2024-09-21T03:43:29.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.287+0000 7f90b666b640 1 -- 172.21.15.38:0/939184927 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90b000ee50 con 0x557ba7949520 2024-09-21T03:43:29.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.287+0000 7f90bf3b6f40 1 -- 172.21.15.38:0/939184927 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x557ba79fe500 con 0x557ba7949520 2024-09-21T03:43:29.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.288+0000 7f90bf3b6f40 1 -- 172.21.15.38:0/939184927 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ba7961a10 con 0x557ba7949520 2024-09-21T03:43:29.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.289+0000 7f90b666b640 1 -- 172.21.15.38:0/939184927 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f90b00173f0 con 0x557ba7949520 2024-09-21T03:43:29.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.290+0000 7f90b666b640 1 --2- 172.21.15.38:0/939184927 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f90a4077740 0x7f90a4079c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:29.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.290+0000 7f90b666b640 1 -- 172.21.15.38:0/939184927 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 9056+0+0 (secure 0 0 0) 0x7f90b009c140 con 0x557ba7949520 2024-09-21T03:43:29.294 INFO:teuthology.orchestra.run.smithi038.stdout:Importing pool 2024-09-21T03:43:29.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.290+0000 7f90b8e70640 1 --2- 172.21.15.38:0/939184927 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f90a4077740 0x7f90a4079c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:29.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.290+0000 7f90bf3b6f40 1 --2- 172.21.15.38:0/939184927 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x557ba78ba810 0x557ba79fddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:29.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.290+0000 7f90bf3b6f40 1 -- 172.21.15.38:0/939184927 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557ba7a0a020 con 0x557ba78ba810 2024-09-21T03:43:29.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.290+0000 7f90b866f640 1 --2- 172.21.15.38:0/939184927 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x557ba78ba810 0x557ba79fddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:29.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.291+0000 7f90b8e70640 1 --2- 172.21.15.38:0/939184927 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f90a4077740 0x7f90a4079c50 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f90a8006fd0 tx=0x7f90a800d040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:29.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.291+0000 7f90b866f640 1 --2- 172.21.15.38:0/939184927 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x557ba78ba810 0x557ba79fddb0 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:29.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.293+0000 7f90b866f640 1 -- 172.21.15.38:0/939184927 <== osd.7 v2:172.21.15.195:6818/1606106342 1 ==== osd_op_reply(1 601.00000000 [create] v128'971 uv944 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f90ac056470 con 0x557ba78ba810 2024-09-21T03:43:29.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.293+0000 7f90bf3b6f40 1 -- 172.21.15.38:0/939184927 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e128) -- 0x557ba7a0a3c0 con 0x557ba78ba810 2024-09-21T03:43:29.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.295+0000 7f90b866f640 1 -- 172.21.15.38:0/939184927 <== osd.7 v2:172.21.15.195:6818/1606106342 2 ==== osd_op_reply(2 601.00000000 [delete] v128'972 uv972 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f90ac056470 con 0x557ba78ba810 2024-09-21T03:43:29.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.295+0000 7f90bf3b6f40 1 -- 172.21.15.38:0/939184927 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557ba7a0a760 con 0x557ba78ba810 2024-09-21T03:43:29.298 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-09-21T03:43:29.299 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.297+0000 7f90b866f640 1 -- 172.21.15.38:0/939184927 <== osd.7 v2:172.21.15.195:6818/1606106342 3 ==== osd_op_reply(3 601.00000000 [create] v128'973 uv973 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f90ac056470 con 0x557ba78ba810 2024-09-21T03:43:29.299 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.297+0000 7f90bf3b6f40 1 -- 172.21.15.38:0/939184927 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557ba7a0bb10 con 0x557ba78ba810 2024-09-21T03:43:29.300 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.298+0000 7f90b866f640 1 -- 172.21.15.38:0/939184927 <== osd.7 v2:172.21.15.195:6818/1606106342 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v128'974 uv974 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f90ac056470 con 0x557ba78ba810 2024-09-21T03:43:29.300 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.298+0000 7f90bf3b6f40 1 -- 172.21.15.38:0/939184927 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557ba7a0beb0 con 0x557ba78ba810 2024-09-21T03:43:29.302 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.300+0000 7f90b866f640 1 -- 172.21.15.38:0/939184927 <== osd.7 v2:172.21.15.195:6818/1606106342 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v128'975 uv975 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f90ac056470 con 0x557ba78ba810 2024-09-21T03:43:29.302 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.300+0000 7f90bf3b6f40 1 -- 172.21.15.38:0/939184927 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557ba7a0ab00 con 0x557ba78ba810 2024-09-21T03:43:29.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.302+0000 7f90b866f640 1 -- 172.21.15.38:0/939184927 <== osd.7 v2:172.21.15.195:6818/1606106342 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v128'976 uv976 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f90ac056470 con 0x557ba78ba810 2024-09-21T03:43:29.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.302+0000 7f90bf3b6f40 1 -- 172.21.15.38:0/939184927 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557ba7a0aef0 con 0x557ba78ba810 2024-09-21T03:43:29.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.304+0000 7f90b866f640 1 -- 172.21.15.38:0/939184927 <== osd.7 v2:172.21.15.195:6818/1606106342 7 ==== osd_op_reply(7 602.00000000 [create] v128'977 uv950 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f90ac056470 con 0x557ba78ba810 2024-09-21T03:43:29.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.304+0000 7f90bf3b6f40 1 -- 172.21.15.38:0/939184927 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e128) -- 0x557ba7a0aef0 con 0x557ba78ba810 2024-09-21T03:43:29.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.306+0000 7f90b866f640 1 -- 172.21.15.38:0/939184927 <== osd.7 v2:172.21.15.195:6818/1606106342 8 ==== osd_op_reply(8 602.00000000 [delete] v128'978 uv978 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f90ac056470 con 0x557ba78ba810 2024-09-21T03:43:29.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.306+0000 7f90bf3b6f40 1 -- 172.21.15.38:0/939184927 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557ba7a0aea0 con 0x557ba78ba810 2024-09-21T03:43:29.308 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-09-21T03:43:29.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.307+0000 7f90b866f640 1 -- 172.21.15.38:0/939184927 <== osd.7 v2:172.21.15.195:6818/1606106342 9 ==== osd_op_reply(9 602.00000000 [create] v128'979 uv979 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f90ac056470 con 0x557ba78ba810 2024-09-21T03:43:29.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.307+0000 7f90bf3b6f40 1 -- 172.21.15.38:0/939184927 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557ba7a0aea0 con 0x557ba78ba810 2024-09-21T03:43:29.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.308+0000 7f90b866f640 1 -- 172.21.15.38:0/939184927 <== osd.7 v2:172.21.15.195:6818/1606106342 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v128'980 uv980 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f90ac056470 con 0x557ba78ba810 2024-09-21T03:43:29.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.308+0000 7f90bf3b6f40 1 -- 172.21.15.38:0/939184927 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557ba7a0aea0 con 0x557ba78ba810 2024-09-21T03:43:29.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.311+0000 7f90b866f640 1 -- 172.21.15.38:0/939184927 <== osd.7 v2:172.21.15.195:6818/1606106342 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v128'981 uv981 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f90ac056470 con 0x557ba78ba810 2024-09-21T03:43:29.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.311+0000 7f90bf3b6f40 1 -- 172.21.15.38:0/939184927 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557ba7a0aea0 con 0x557ba78ba810 2024-09-21T03:43:29.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.312+0000 7f90b866f640 1 -- 172.21.15.38:0/939184927 <== osd.7 v2:172.21.15.195:6818/1606106342 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v128'982 uv982 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f90ac056470 con 0x557ba78ba810 2024-09-21T03:43:29.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.312+0000 7f90bf3b6f40 1 -- 172.21.15.38:0/939184927 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:13 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557ba7a0aea0 con 0x557ba78ba810 2024-09-21T03:43:29.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.314+0000 7f90b866f640 1 -- 172.21.15.38:0/939184927 <== osd.7 v2:172.21.15.195:6818/1606106342 13 ==== osd_op_reply(13 10000000000.00000000 [create] v128'983 uv970 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7f90ac056470 con 0x557ba78ba810 2024-09-21T03:43:29.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.314+0000 7f90bf3b6f40 1 -- 172.21.15.38:0/939184927 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:14 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e128) -- 0x557ba7a0aea0 con 0x557ba78ba810 2024-09-21T03:43:29.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.316+0000 7f90b866f640 1 -- 172.21.15.38:0/939184927 <== osd.7 v2:172.21.15.195:6818/1606106342 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v128'984 uv984 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f90ac056470 con 0x557ba78ba810 2024-09-21T03:43:29.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.316+0000 7f90bf3b6f40 1 -- 172.21.15.38:0/939184927 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:15 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557ba7a0aea0 con 0x557ba78ba810 2024-09-21T03:43:29.319 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-09-21T03:43:29.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.317+0000 7f90b866f640 1 -- 172.21.15.38:0/939184927 <== osd.7 v2:172.21.15.195:6818/1606106342 15 ==== osd_op_reply(15 10000000000.00000000 [create] v128'985 uv985 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f90ac056470 con 0x557ba78ba810 2024-09-21T03:43:29.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.317+0000 7f90bf3b6f40 1 -- 172.21.15.38:0/939184927 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:16 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557ba7a0aea0 con 0x557ba78ba810 2024-09-21T03:43:29.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.318+0000 7f90b866f640 1 -- 172.21.15.38:0/939184927 <== osd.7 v2:172.21.15.195:6818/1606106342 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v128'986 uv986 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f90ac056470 con 0x557ba78ba810 2024-09-21T03:43:29.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.318+0000 7f90bf3b6f40 1 -- 172.21.15.38:0/939184927 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:17 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557ba7a0aea0 con 0x557ba78ba810 2024-09-21T03:43:29.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.320+0000 7f90b866f640 1 -- 172.21.15.38:0/939184927 <== osd.7 v2:172.21.15.195:6818/1606106342 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v128'987 uv987 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f90ac056470 con 0x557ba78ba810 2024-09-21T03:43:29.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.320+0000 7f90bf3b6f40 1 -- 172.21.15.38:0/939184927 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:18 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557ba7a0aea0 con 0x557ba78ba810 2024-09-21T03:43:29.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.322+0000 7f90b866f640 1 -- 172.21.15.38:0/939184927 <== osd.7 v2:172.21.15.195:6818/1606106342 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v128'988 uv988 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f90ac056470 con 0x557ba78ba810 2024-09-21T03:43:29.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.322+0000 7f90bf3b6f40 1 -- 172.21.15.38:0/939184927 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:19 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=585b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557ba7a0b710 con 0x557ba78ba810 2024-09-21T03:43:29.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.323+0000 7f90b866f640 1 -- 172.21.15.38:0/939184927 <== osd.7 v2:172.21.15.195:6818/1606106342 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v128'989 uv989 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f90ac056470 con 0x557ba78ba810 2024-09-21T03:43:29.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.324+0000 7f90bf3b6f40 1 -- 172.21.15.38:0/939184927 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:20 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557ba7a0b710 con 0x557ba78ba810 2024-09-21T03:43:29.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.325+0000 7f90b866f640 1 -- 172.21.15.38:0/939184927 <== osd.7 v2:172.21.15.195:6818/1606106342 20 ==== osd_op_reply(20 600.00000000 [create] v128'990 uv963 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f90ac056470 con 0x557ba78ba810 2024-09-21T03:43:29.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.326+0000 7f90bf3b6f40 1 -- 172.21.15.38:0/939184927 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:21 13.10 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e128) -- 0x557ba7a0b710 con 0x557ba78ba810 2024-09-21T03:43:29.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.327+0000 7f90b866f640 1 -- 172.21.15.38:0/939184927 <== osd.7 v2:172.21.15.195:6818/1606106342 21 ==== osd_op_reply(21 600.00000000 [delete] v128'991 uv991 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f90ac056470 con 0x557ba78ba810 2024-09-21T03:43:29.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.327+0000 7f90bf3b6f40 1 -- 172.21.15.38:0/939184927 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:22 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557ba7a0b710 con 0x557ba78ba810 2024-09-21T03:43:29.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.329+0000 7f90b866f640 1 -- 172.21.15.38:0/939184927 <== osd.7 v2:172.21.15.195:6818/1606106342 22 ==== osd_op_reply(22 600.00000000 [create] v128'992 uv992 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f90ac056470 con 0x557ba78ba810 2024-09-21T03:43:29.331 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-09-21T03:43:29.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.329+0000 7f90bf3b6f40 1 -- 172.21.15.38:0/939184927 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:23 13.10 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557ba7a0b710 con 0x557ba78ba810 2024-09-21T03:43:29.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.330+0000 7f90b866f640 1 -- 172.21.15.38:0/939184927 <== osd.7 v2:172.21.15.195:6818/1606106342 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v128'993 uv993 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f90ac056470 con 0x557ba78ba810 2024-09-21T03:43:29.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.330+0000 7f90bf3b6f40 1 -- 172.21.15.38:0/939184927 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:24 13.10 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557ba7a0b600 con 0x557ba78ba810 2024-09-21T03:43:29.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.332+0000 7f90b866f640 1 -- 172.21.15.38:0/939184927 <== osd.7 v2:172.21.15.195:6818/1606106342 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v128'994 uv994 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f90ac056470 con 0x557ba78ba810 2024-09-21T03:43:29.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.332+0000 7f90bf3b6f40 1 -- 172.21.15.38:0/939184927 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:25 13.10 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557ba7a0b600 con 0x557ba78ba810 2024-09-21T03:43:29.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.333+0000 7f90b866f640 1 -- 172.21.15.38:0/939184927 <== osd.7 v2:172.21.15.195:6818/1606106342 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v128'995 uv995 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f90ac056470 con 0x557ba78ba810 2024-09-21T03:43:29.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.334+0000 7f90bf3b6f40 1 -- 172.21.15.38:0/939184927 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:26 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557ba7a0b600 con 0x557ba78ba810 2024-09-21T03:43:29.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.335+0000 7f90b866f640 1 -- 172.21.15.38:0/939184927 <== osd.7 v2:172.21.15.195:6818/1606106342 26 ==== osd_op_reply(26 603.00000000 [create] v128'996 uv969 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f90ac056470 con 0x557ba78ba810 2024-09-21T03:43:29.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.335+0000 7f90bf3b6f40 1 -- 172.21.15.38:0/939184927 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:27 13.10 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e128) -- 0x557ba7a0b600 con 0x557ba78ba810 2024-09-21T03:43:29.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.337+0000 7f90b866f640 1 -- 172.21.15.38:0/939184927 <== osd.7 v2:172.21.15.195:6818/1606106342 27 ==== osd_op_reply(27 603.00000000 [delete] v128'997 uv997 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f90ac056470 con 0x557ba78ba810 2024-09-21T03:43:29.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.337+0000 7f90bf3b6f40 1 -- 172.21.15.38:0/939184927 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:28 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557ba7a0b600 con 0x557ba78ba810 2024-09-21T03:43:29.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.338+0000 7f90b866f640 1 -- 172.21.15.38:0/939184927 <== osd.7 v2:172.21.15.195:6818/1606106342 28 ==== osd_op_reply(28 603.00000000 [create] v128'998 uv998 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f90ac056470 con 0x557ba78ba810 2024-09-21T03:43:29.340 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-09-21T03:43:29.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.339+0000 7f90bf3b6f40 1 -- 172.21.15.38:0/939184927 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:29 13.10 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557ba7a0b600 con 0x557ba78ba810 2024-09-21T03:43:29.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.341+0000 7f90b866f640 1 -- 172.21.15.38:0/939184927 <== osd.7 v2:172.21.15.195:6818/1606106342 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v128'999 uv999 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f90ac056470 con 0x557ba78ba810 2024-09-21T03:43:29.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.341+0000 7f90bf3b6f40 1 -- 172.21.15.38:0/939184927 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:30 13.10 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557ba7a0b600 con 0x557ba78ba810 2024-09-21T03:43:29.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.342+0000 7f90b866f640 1 -- 172.21.15.38:0/939184927 <== osd.7 v2:172.21.15.195:6818/1606106342 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v128'1000 uv1000 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f90ac056470 con 0x557ba78ba810 2024-09-21T03:43:29.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.343+0000 7f90bf3b6f40 1 -- 172.21.15.38:0/939184927 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:31 13.10 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557ba7a0b600 con 0x557ba78ba810 2024-09-21T03:43:29.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.344+0000 7f90b866f640 1 -- 172.21.15.38:0/939184927 <== osd.7 v2:172.21.15.195:6818/1606106342 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v128'1001 uv1001 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f90ac056470 con 0x557ba78ba810 2024-09-21T03:43:29.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.344+0000 7f90bf3b6f40 1 --2- 172.21.15.38:0/939184927 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x557ba7a0b600 0x557ba7a0e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:29.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.344+0000 7f90bf3b6f40 1 -- 172.21.15.38:0/939184927 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557ba7a0ecb0 con 0x557ba7a0b600 2024-09-21T03:43:29.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.344+0000 7f90b8e70640 1 --2- 172.21.15.38:0/939184927 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x557ba7a0b600 0x557ba7a0e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:29.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.345+0000 7f90b8e70640 1 --2- 172.21.15.38:0/939184927 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x557ba7a0b600 0x557ba7a0e770 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:29.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.347+0000 7f90b8e70640 1 -- 172.21.15.38:0/939184927 <== osd.6 v2:172.21.15.195:6826/4055825675 1 ==== osd_op_reply(32 1.00000000.inode [create] v128'704 uv683 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f90a8016430 con 0x557ba7a0b600 2024-09-21T03:43:29.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.347+0000 7f90bf3b6f40 1 -- 172.21.15.38:0/939184927 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e128) -- 0x557ba7a0f050 con 0x557ba7a0b600 2024-09-21T03:43:29.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.348+0000 7f90b8e70640 1 -- 172.21.15.38:0/939184927 <== osd.6 v2:172.21.15.195:6826/4055825675 2 ==== osd_op_reply(33 1.00000000.inode [delete] v128'705 uv705 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f90a8016430 con 0x557ba7a0b600 2024-09-21T03:43:29.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.348+0000 7f90bf3b6f40 1 -- 172.21.15.38:0/939184927 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557ba7a0f3f0 con 0x557ba7a0b600 2024-09-21T03:43:29.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.350+0000 7f90b8e70640 1 -- 172.21.15.38:0/939184927 <== osd.6 v2:172.21.15.195:6826/4055825675 3 ==== osd_op_reply(34 1.00000000.inode [create] v128'706 uv706 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f90a8016430 con 0x557ba7a0b600 2024-09-21T03:43:29.352 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-09-21T03:43:29.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.350+0000 7f90bf3b6f40 1 -- 172.21.15.38:0/939184927 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557ba7a0fc40 con 0x557ba7a0b600 2024-09-21T03:43:29.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.351+0000 7f90b8e70640 1 -- 172.21.15.38:0/939184927 <== osd.6 v2:172.21.15.195:6826/4055825675 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v128'707 uv707 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f90a8016430 con 0x557ba7a0b600 2024-09-21T03:43:29.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.351+0000 7f90bf3b6f40 1 -- 172.21.15.38:0/939184927 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557ba7a0ffe0 con 0x557ba7a0b600 2024-09-21T03:43:29.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.352+0000 7f90b8e70640 1 -- 172.21.15.38:0/939184927 <== osd.6 v2:172.21.15.195:6826/4055825675 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v128'708 uv708 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f90a8016430 con 0x557ba7a0b600 2024-09-21T03:43:29.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.352+0000 7f90bf3b6f40 1 -- 172.21.15.38:0/939184927 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557ba7a10380 con 0x557ba7a0b600 2024-09-21T03:43:29.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.354+0000 7f90b8e70640 1 -- 172.21.15.38:0/939184927 <== osd.6 v2:172.21.15.195:6826/4055825675 6 ==== osd_op_reply(37 200.00000000 [create] v128'709 uv702 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f90a8016430 con 0x557ba7a0b600 2024-09-21T03:43:29.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.354+0000 7f90bf3b6f40 1 -- 172.21.15.38:0/939184927 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e128) -- 0x557ba7a10720 con 0x557ba7a0b600 2024-09-21T03:43:29.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.355+0000 7f90b8e70640 1 -- 172.21.15.38:0/939184927 <== osd.6 v2:172.21.15.195:6826/4055825675 7 ==== osd_op_reply(38 200.00000000 [delete] v128'710 uv710 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f90a8016430 con 0x557ba7a0b600 2024-09-21T03:43:29.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.355+0000 7f90bf3b6f40 1 -- 172.21.15.38:0/939184927 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557ba7a10ac0 con 0x557ba7a0b600 2024-09-21T03:43:29.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.357+0000 7f90b8e70640 1 -- 172.21.15.38:0/939184927 <== osd.6 v2:172.21.15.195:6826/4055825675 8 ==== osd_op_reply(39 200.00000000 [create] v128'711 uv711 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f90a8016430 con 0x557ba7a0b600 2024-09-21T03:43:29.359 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-09-21T03:43:29.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.357+0000 7f90bf3b6f40 1 -- 172.21.15.38:0/939184927 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557ba7a10ac0 con 0x557ba7a0b600 2024-09-21T03:43:29.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.358+0000 7f90b8e70640 1 -- 172.21.15.38:0/939184927 <== osd.6 v2:172.21.15.195:6826/4055825675 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v128'712 uv712 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f90a8016430 con 0x557ba7a0b600 2024-09-21T03:43:29.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.358+0000 7f90bf3b6f40 1 -- 172.21.15.38:0/939184927 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557ba7a10ac0 con 0x557ba7a0b600 2024-09-21T03:43:29.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.360+0000 7f90b8e70640 1 -- 172.21.15.38:0/939184927 <== osd.6 v2:172.21.15.195:6826/4055825675 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v128'713 uv713 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f90a8016430 con 0x557ba7a0b600 2024-09-21T03:43:29.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.360+0000 7f90bf3b6f40 1 -- 172.21.15.38:0/939184927 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:42 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557ba7a10ac0 con 0x557ba7a0b600 2024-09-21T03:43:29.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.361+0000 7f90b8e70640 1 -- 172.21.15.38:0/939184927 <== osd.6 v2:172.21.15.195:6826/4055825675 11 ==== osd_op_reply(42 200.00000001 [create] v128'714 uv703 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f90a8016430 con 0x557ba7a0b600 2024-09-21T03:43:29.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.361+0000 7f90bf3b6f40 1 -- 172.21.15.38:0/939184927 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:43 13.14 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e128) -- 0x557ba7a10ac0 con 0x557ba7a0b600 2024-09-21T03:43:29.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.363+0000 7f90b8e70640 1 -- 172.21.15.38:0/939184927 <== osd.6 v2:172.21.15.195:6826/4055825675 12 ==== osd_op_reply(43 200.00000001 [delete] v128'715 uv715 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f90a8016430 con 0x557ba7a0b600 2024-09-21T03:43:29.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.363+0000 7f90bf3b6f40 1 -- 172.21.15.38:0/939184927 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:44 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557ba7a10ac0 con 0x557ba7a0b600 2024-09-21T03:43:29.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.364+0000 7f90b8e70640 1 -- 172.21.15.38:0/939184927 <== osd.6 v2:172.21.15.195:6826/4055825675 13 ==== osd_op_reply(44 200.00000001 [create] v128'716 uv716 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f90a8016430 con 0x557ba7a0b600 2024-09-21T03:43:29.366 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-09-21T03:43:29.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.364+0000 7f90bf3b6f40 1 -- 172.21.15.38:0/939184927 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:45 13.14 13:2e2fa760:::200.00000001:head [write 0~17490 in=17490b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557ba7a14f40 con 0x557ba7a0b600 2024-09-21T03:43:29.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.366+0000 7f90b8e70640 1 -- 172.21.15.38:0/939184927 <== osd.6 v2:172.21.15.195:6826/4055825675 14 ==== osd_op_reply(45 200.00000001 [write 0~17490] v128'717 uv717 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f90a8016430 con 0x557ba7a0b600 2024-09-21T03:43:29.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.366+0000 7f90bf3b6f40 1 -- 172.21.15.38:0/939184927 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:46 13.14 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557ba7a14f40 con 0x557ba7a0b600 2024-09-21T03:43:29.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.367+0000 7f90b8e70640 1 -- 172.21.15.38:0/939184927 <== osd.6 v2:172.21.15.195:6826/4055825675 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v128'718 uv718 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f90a8016430 con 0x557ba7a0b600 2024-09-21T03:43:29.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.367+0000 7f90bf3b6f40 1 -- 172.21.15.38:0/939184927 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:47 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557ba7a10ac0 con 0x557ba7a0b600 2024-09-21T03:43:29.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.368+0000 7f90b8e70640 1 -- 172.21.15.38:0/939184927 <== osd.6 v2:172.21.15.195:6826/4055825675 16 ==== osd_op_reply(47 606.00000000 [create] v128'719 uv699 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f90a8016430 con 0x557ba7a0b600 2024-09-21T03:43:29.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.368+0000 7f90bf3b6f40 1 -- 172.21.15.38:0/939184927 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:48 13.14 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e128) -- 0x557ba7a10ac0 con 0x557ba7a0b600 2024-09-21T03:43:29.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.370+0000 7f90b8e70640 1 -- 172.21.15.38:0/939184927 <== osd.6 v2:172.21.15.195:6826/4055825675 17 ==== osd_op_reply(48 606.00000000 [delete] v128'720 uv720 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f90a8016430 con 0x557ba7a0b600 2024-09-21T03:43:29.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.370+0000 7f90bf3b6f40 1 -- 172.21.15.38:0/939184927 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:49 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557ba7a10ac0 con 0x557ba7a0b600 2024-09-21T03:43:29.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.371+0000 7f90b8e70640 1 -- 172.21.15.38:0/939184927 <== osd.6 v2:172.21.15.195:6826/4055825675 18 ==== osd_op_reply(49 606.00000000 [create] v128'721 uv721 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f90a8016430 con 0x557ba7a0b600 2024-09-21T03:43:29.373 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-09-21T03:43:29.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.371+0000 7f90bf3b6f40 1 -- 172.21.15.38:0/939184927 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:50 13.14 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557ba7a11ad0 con 0x557ba7a0b600 2024-09-21T03:43:29.374 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.372+0000 7f90b8e70640 1 -- 172.21.15.38:0/939184927 <== osd.6 v2:172.21.15.195:6826/4055825675 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v128'722 uv722 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f90a8016430 con 0x557ba7a0b600 2024-09-21T03:43:29.374 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.372+0000 7f90bf3b6f40 1 -- 172.21.15.38:0/939184927 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:51 13.14 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557ba7a11ad0 con 0x557ba7a0b600 2024-09-21T03:43:29.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.374+0000 7f90b8e70640 1 -- 172.21.15.38:0/939184927 <== osd.6 v2:172.21.15.195:6826/4055825675 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v128'723 uv723 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f90a8016430 con 0x557ba7a0b600 2024-09-21T03:43:29.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.374+0000 7f90bf3b6f40 1 -- 172.21.15.38:0/939184927 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:52 13.14 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557ba7a10ac0 con 0x557ba7a0b600 2024-09-21T03:43:29.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.375+0000 7f90b8e70640 1 -- 172.21.15.38:0/939184927 <== osd.6 v2:172.21.15.195:6826/4055825675 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v128'724 uv724 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f90a8016430 con 0x557ba7a0b600 2024-09-21T03:43:29.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.375+0000 7f90bf3b6f40 1 --2- 172.21.15.38:0/939184927 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x557ba7a10fd0 0x557ba7a133f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:29.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.375+0000 7f90bf3b6f40 1 -- 172.21.15.38:0/939184927 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:53 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557ba7a13930 con 0x557ba7a10fd0 2024-09-21T03:43:29.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.375+0000 7f90b866f640 1 --2- 172.21.15.38:0/939184927 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x557ba7a10fd0 0x557ba7a133f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:29.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.375+0000 7f90b866f640 1 --2- 172.21.15.38:0/939184927 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x557ba7a10fd0 0x557ba7a133f0 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:29.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.377+0000 7f90b866f640 1 -- 172.21.15.38:0/939184927 <== osd.0 v2:172.21.15.38:6800/2381639433 1 ==== osd_op_reply(53 607.00000000 [create] v128'189 uv188 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f90ac056470 con 0x557ba7a10fd0 2024-09-21T03:43:29.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.377+0000 7f90bf3b6f40 1 -- 172.21.15.38:0/939184927 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:54 13.1c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e128) -- 0x557ba7a13930 con 0x557ba7a10fd0 2024-09-21T03:43:29.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.378+0000 7f90b866f640 1 -- 172.21.15.38:0/939184927 <== osd.0 v2:172.21.15.38:6800/2381639433 2 ==== osd_op_reply(54 607.00000000 [delete] v128'190 uv190 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f90ac056470 con 0x557ba7a10fd0 2024-09-21T03:43:29.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.378+0000 7f90bf3b6f40 1 -- 172.21.15.38:0/939184927 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:55 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557ba7a139b0 con 0x557ba7a10fd0 2024-09-21T03:43:29.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.379+0000 7f90b866f640 1 -- 172.21.15.38:0/939184927 <== osd.0 v2:172.21.15.38:6800/2381639433 3 ==== osd_op_reply(55 607.00000000 [create] v128'191 uv191 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f90ac056470 con 0x557ba7a10fd0 2024-09-21T03:43:29.381 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-09-21T03:43:29.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.379+0000 7f90bf3b6f40 1 -- 172.21.15.38:0/939184927 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:56 13.1c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557ba7a14db0 con 0x557ba7a10fd0 2024-09-21T03:43:29.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.380+0000 7f90b866f640 1 -- 172.21.15.38:0/939184927 <== osd.0 v2:172.21.15.38:6800/2381639433 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v128'192 uv192 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f90ac056470 con 0x557ba7a10fd0 2024-09-21T03:43:29.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.380+0000 7f90bf3b6f40 1 -- 172.21.15.38:0/939184927 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:57 13.1c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557ba7a14db0 con 0x557ba7a10fd0 2024-09-21T03:43:29.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.381+0000 7f90b866f640 1 -- 172.21.15.38:0/939184927 <== osd.0 v2:172.21.15.38:6800/2381639433 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v128'193 uv193 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f90ac056470 con 0x557ba7a10fd0 2024-09-21T03:43:29.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.381+0000 7f90bf3b6f40 1 -- 172.21.15.38:0/939184927 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:58 13.1c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557ba7a13da0 con 0x557ba7a10fd0 2024-09-21T03:43:29.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.382+0000 7f90b866f640 1 -- 172.21.15.38:0/939184927 <== osd.0 v2:172.21.15.38:6800/2381639433 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v128'194 uv194 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f90ac056470 con 0x557ba7a10fd0 2024-09-21T03:43:29.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.382+0000 7f90bf3b6f40 1 --2- 172.21.15.38:0/939184927 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x557ba7a14420 0x557ba7a16840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:29.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.382+0000 7f90bf3b6f40 1 -- 172.21.15.38:0/939184927 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:59 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557ba7a16e30 con 0x557ba7a14420 2024-09-21T03:43:29.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.382+0000 7f90b9671640 1 --2- 172.21.15.38:0/939184927 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x557ba7a14420 0x557ba7a16840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:29.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.382+0000 7f90b9671640 1 --2- 172.21.15.38:0/939184927 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x557ba7a14420 0x557ba7a16840 crc :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:29.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.384+0000 7f90b9671640 1 -- 172.21.15.38:0/939184927 <== osd.3 v2:172.21.15.38:6808/1286810294 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v128'138 uv137 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f90b006a0b0 con 0x557ba7a14420 2024-09-21T03:43:29.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.384+0000 7f90bf3b6f40 1 -- 172.21.15.38:0/939184927 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:60 13.6 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e128) -- 0x557ba7a17340 con 0x557ba7a14420 2024-09-21T03:43:29.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.385+0000 7f90b9671640 1 -- 172.21.15.38:0/939184927 <== osd.3 v2:172.21.15.38:6808/1286810294 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v128'139 uv139 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f90b006a0b0 con 0x557ba7a14420 2024-09-21T03:43:29.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.385+0000 7f90bf3b6f40 1 -- 172.21.15.38:0/939184927 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:61 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557ba7a17840 con 0x557ba7a14420 2024-09-21T03:43:29.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.386+0000 7f90b9671640 1 -- 172.21.15.38:0/939184927 <== osd.3 v2:172.21.15.38:6808/1286810294 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v128'140 uv140 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f90b006a0b0 con 0x557ba7a14420 2024-09-21T03:43:29.388 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-09-21T03:43:29.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.386+0000 7f90bf3b6f40 1 -- 172.21.15.38:0/939184927 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:62 13.6 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557ba7a18100 con 0x557ba7a14420 2024-09-21T03:43:29.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.387+0000 7f90b9671640 1 -- 172.21.15.38:0/939184927 <== osd.3 v2:172.21.15.38:6808/1286810294 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v128'141 uv141 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f90b006a0b0 con 0x557ba7a14420 2024-09-21T03:43:29.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.387+0000 7f90bf3b6f40 1 -- 172.21.15.38:0/939184927 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557ba7a184a0 con 0x557ba7a10fd0 2024-09-21T03:43:29.390 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.388+0000 7f90b866f640 1 -- 172.21.15.38:0/939184927 <== osd.0 v2:172.21.15.38:6800/2381639433 7 ==== osd_op_reply(63 608.00000000 [create] v128'189 uv188 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f90ac056470 con 0x557ba7a10fd0 2024-09-21T03:43:29.390 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.388+0000 7f90bf3b6f40 1 -- 172.21.15.38:0/939184927 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e128) -- 0x557ba7a184a0 con 0x557ba7a10fd0 2024-09-21T03:43:29.391 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.390+0000 7f90b866f640 1 -- 172.21.15.38:0/939184927 <== osd.0 v2:172.21.15.38:6800/2381639433 8 ==== osd_op_reply(64 608.00000000 [delete] v128'190 uv190 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f90ac056470 con 0x557ba7a10fd0 2024-09-21T03:43:29.391 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.390+0000 7f90bf3b6f40 1 -- 172.21.15.38:0/939184927 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557ba7a184a0 con 0x557ba7a10fd0 2024-09-21T03:43:29.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.391+0000 7f90b866f640 1 -- 172.21.15.38:0/939184927 <== osd.0 v2:172.21.15.38:6800/2381639433 9 ==== osd_op_reply(65 608.00000000 [create] v128'191 uv191 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f90ac056470 con 0x557ba7a10fd0 2024-09-21T03:43:29.393 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-09-21T03:43:29.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.391+0000 7f90bf3b6f40 1 -- 172.21.15.38:0/939184927 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557ba7a194b0 con 0x557ba7a10fd0 2024-09-21T03:43:29.394 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.392+0000 7f90b866f640 1 -- 172.21.15.38:0/939184927 <== osd.0 v2:172.21.15.38:6800/2381639433 10 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v128'192 uv192 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f90ac056470 con 0x557ba7a10fd0 2024-09-21T03:43:29.394 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.392+0000 7f90bf3b6f40 1 -- 172.21.15.38:0/939184927 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557ba7a194b0 con 0x557ba7a10fd0 2024-09-21T03:43:29.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.393+0000 7f90b866f640 1 -- 172.21.15.38:0/939184927 <== osd.0 v2:172.21.15.38:6800/2381639433 11 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v128'193 uv193 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f90ac056470 con 0x557ba7a10fd0 2024-09-21T03:43:29.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.393+0000 7f90bf3b6f40 1 -- 172.21.15.38:0/939184927 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557ba7a184a0 con 0x557ba7a10fd0 2024-09-21T03:43:29.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.394+0000 7f90b866f640 1 -- 172.21.15.38:0/939184927 <== osd.0 v2:172.21.15.38:6800/2381639433 12 ==== osd_op_reply(68 608.00000000 [omap-set-header] v128'194 uv194 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f90ac056470 con 0x557ba7a10fd0 2024-09-21T03:43:29.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.394+0000 7f90bf3b6f40 1 -- 172.21.15.38:0/939184927 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557ba7a184a0 con 0x557ba7a14420 2024-09-21T03:43:29.398 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.396+0000 7f90b9671640 1 -- 172.21.15.38:0/939184927 <== osd.3 v2:172.21.15.38:6808/1286810294 5 ==== osd_op_reply(69 604.00000000 [create] v128'349 uv343 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f90b006a0b0 con 0x557ba7a14420 2024-09-21T03:43:29.398 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.396+0000 7f90bf3b6f40 1 -- 172.21.15.38:0/939184927 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e128) -- 0x557ba7a18840 con 0x557ba7a14420 2024-09-21T03:43:29.399 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.397+0000 7f90b9671640 1 -- 172.21.15.38:0/939184927 <== osd.3 v2:172.21.15.38:6808/1286810294 6 ==== osd_op_reply(70 604.00000000 [delete] v128'350 uv350 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f90b006a0b0 con 0x557ba7a14420 2024-09-21T03:43:29.399 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.397+0000 7f90bf3b6f40 1 -- 172.21.15.38:0/939184927 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557ba7a18be0 con 0x557ba7a14420 2024-09-21T03:43:29.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.398+0000 7f90b9671640 1 -- 172.21.15.38:0/939184927 <== osd.3 v2:172.21.15.38:6808/1286810294 7 ==== osd_op_reply(71 604.00000000 [create] v128'351 uv351 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f90b006a0b0 con 0x557ba7a14420 2024-09-21T03:43:29.400 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-09-21T03:43:29.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.398+0000 7f90bf3b6f40 1 -- 172.21.15.38:0/939184927 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557ba7a19f90 con 0x557ba7a14420 2024-09-21T03:43:29.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.400+0000 7f90b9671640 1 -- 172.21.15.38:0/939184927 <== osd.3 v2:172.21.15.38:6808/1286810294 8 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v128'352 uv352 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f90b006a0b0 con 0x557ba7a14420 2024-09-21T03:43:29.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.400+0000 7f90bf3b6f40 1 -- 172.21.15.38:0/939184927 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557ba7a19f90 con 0x557ba7a14420 2024-09-21T03:43:29.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.401+0000 7f90b9671640 1 -- 172.21.15.38:0/939184927 <== osd.3 v2:172.21.15.38:6808/1286810294 9 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v128'353 uv353 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f90b006a0b0 con 0x557ba7a14420 2024-09-21T03:43:29.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.401+0000 7f90bf3b6f40 1 -- 172.21.15.38:0/939184927 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557ba7a18f80 con 0x557ba7a14420 2024-09-21T03:43:29.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.402+0000 7f90b9671640 1 -- 172.21.15.38:0/939184927 <== osd.3 v2:172.21.15.38:6808/1286810294 10 ==== osd_op_reply(74 604.00000000 [omap-set-header] v128'354 uv354 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f90b006a0b0 con 0x557ba7a14420 2024-09-21T03:43:29.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.402+0000 7f90bf3b6f40 1 -- 172.21.15.38:0/939184927 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557ba7a18f80 con 0x557ba7a14420 2024-09-21T03:43:29.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.403+0000 7f90b9671640 1 -- 172.21.15.38:0/939184927 <== osd.3 v2:172.21.15.38:6808/1286810294 11 ==== osd_op_reply(75 500.00000000 [create] v128'355 uv348 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f90b006a0b0 con 0x557ba7a14420 2024-09-21T03:43:29.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.403+0000 7f90bf3b6f40 1 -- 172.21.15.38:0/939184927 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e128) -- 0x557ba7a18f80 con 0x557ba7a14420 2024-09-21T03:43:29.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.404+0000 7f90b9671640 1 -- 172.21.15.38:0/939184927 <== osd.3 v2:172.21.15.38:6808/1286810294 12 ==== osd_op_reply(76 500.00000000 [delete] v128'356 uv356 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f90b006a0b0 con 0x557ba7a14420 2024-09-21T03:43:29.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.404+0000 7f90bf3b6f40 1 -- 172.21.15.38:0/939184927 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557ba7a18f80 con 0x557ba7a14420 2024-09-21T03:43:29.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.406+0000 7f90b9671640 1 -- 172.21.15.38:0/939184927 <== osd.3 v2:172.21.15.38:6808/1286810294 13 ==== osd_op_reply(77 500.00000000 [create] v128'357 uv357 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f90b006a0b0 con 0x557ba7a14420 2024-09-21T03:43:29.408 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-09-21T03:43:29.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.406+0000 7f90bf3b6f40 1 -- 172.21.15.38:0/939184927 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557ba7a18f80 con 0x557ba7a14420 2024-09-21T03:43:29.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.407+0000 7f90b9671640 1 -- 172.21.15.38:0/939184927 <== osd.3 v2:172.21.15.38:6808/1286810294 14 ==== osd_op_reply(78 500.00000000 [write 0~90] v128'358 uv358 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f90b006a0b0 con 0x557ba7a14420 2024-09-21T03:43:29.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.407+0000 7f90bf3b6f40 1 -- 172.21.15.38:0/939184927 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557ba7a18f80 con 0x557ba7a14420 2024-09-21T03:43:29.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.408+0000 7f90b9671640 1 -- 172.21.15.38:0/939184927 <== osd.3 v2:172.21.15.38:6808/1286810294 15 ==== osd_op_reply(79 500.00000000 [omap-set-header] v128'359 uv359 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f90b006a0b0 con 0x557ba7a14420 2024-09-21T03:43:29.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.408+0000 7f90bf3b6f40 1 -- 172.21.15.38:0/939184927 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:80 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557ba7a18f80 con 0x557ba7a10fd0 2024-09-21T03:43:29.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.409+0000 7f90b866f640 1 -- 172.21.15.38:0/939184927 <== osd.0 v2:172.21.15.38:6800/2381639433 13 ==== osd_op_reply(80 mds_snaptable [create] v128'158 uv157 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f90ac056470 con 0x557ba7a10fd0 2024-09-21T03:43:29.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.409+0000 7f90bf3b6f40 1 -- 172.21.15.38:0/939184927 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:81 13.d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e128) -- 0x557ba7a18f80 con 0x557ba7a10fd0 2024-09-21T03:43:29.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.411+0000 7f90b866f640 1 -- 172.21.15.38:0/939184927 <== osd.0 v2:172.21.15.38:6800/2381639433 14 ==== osd_op_reply(81 mds_snaptable [delete] v128'159 uv159 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f90ac056470 con 0x557ba7a10fd0 2024-09-21T03:43:29.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.411+0000 7f90bf3b6f40 1 -- 172.21.15.38:0/939184927 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:82 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557ba7a18f80 con 0x557ba7a10fd0 2024-09-21T03:43:29.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.412+0000 7f90b866f640 1 -- 172.21.15.38:0/939184927 <== osd.0 v2:172.21.15.38:6800/2381639433 15 ==== osd_op_reply(82 mds_snaptable [create] v128'160 uv160 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f90ac056470 con 0x557ba7a10fd0 2024-09-21T03:43:29.414 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-09-21T03:43:29.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.412+0000 7f90bf3b6f40 1 -- 172.21.15.38:0/939184927 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:83 13.d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557ba7a18f80 con 0x557ba7a10fd0 2024-09-21T03:43:29.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.413+0000 7f90b866f640 1 -- 172.21.15.38:0/939184927 <== osd.0 v2:172.21.15.38:6800/2381639433 16 ==== osd_op_reply(83 mds_snaptable [write 0~70] v128'161 uv161 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f90ac056470 con 0x557ba7a10fd0 2024-09-21T03:43:29.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.413+0000 7f90bf3b6f40 1 -- 172.21.15.38:0/939184927 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:84 13.d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557ba7a18f80 con 0x557ba7a10fd0 2024-09-21T03:43:29.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.414+0000 7f90b866f640 1 -- 172.21.15.38:0/939184927 <== osd.0 v2:172.21.15.38:6800/2381639433 17 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v128'162 uv162 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f90ac056470 con 0x557ba7a10fd0 2024-09-21T03:43:29.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.414+0000 7f90bf3b6f40 1 --2- 172.21.15.38:0/939184927 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x557ba7a19600 0x557ba7a1ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:29.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.414+0000 7f90bf3b6f40 1 -- 172.21.15.38:0/939184927 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557ba7a1c080 con 0x557ba7a19600 2024-09-21T03:43:29.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.414+0000 7f90b8e70640 1 --2- 172.21.15.38:0/939184927 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x557ba7a19600 0x557ba7a1ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:29.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.414+0000 7f90b8e70640 1 --2- 172.21.15.38:0/939184927 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x557ba7a19600 0x557ba7a1ba20 crc :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:29.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.416+0000 7f90b8e70640 1 -- 172.21.15.38:0/939184927 <== osd.1 v2:172.21.15.38:6816/1491818732 1 ==== osd_op_reply(85 605.00000000 [create] v128'189 uv188 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f90a8016430 con 0x557ba7a19600 2024-09-21T03:43:29.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.416+0000 7f90bf3b6f40 1 -- 172.21.15.38:0/939184927 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e128) -- 0x557ba7a1c450 con 0x557ba7a19600 2024-09-21T03:43:29.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.417+0000 7f90b8e70640 1 -- 172.21.15.38:0/939184927 <== osd.1 v2:172.21.15.38:6816/1491818732 2 ==== osd_op_reply(86 605.00000000 [delete] v128'190 uv190 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f90a8016430 con 0x557ba7a19600 2024-09-21T03:43:29.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.417+0000 7f90bf3b6f40 1 -- 172.21.15.38:0/939184927 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557ba7a1c450 con 0x557ba7a19600 2024-09-21T03:43:29.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.418+0000 7f90b8e70640 1 -- 172.21.15.38:0/939184927 <== osd.1 v2:172.21.15.38:6816/1491818732 3 ==== osd_op_reply(87 605.00000000 [create] v128'191 uv191 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f90a8016430 con 0x557ba7a19600 2024-09-21T03:43:29.420 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-09-21T03:43:29.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.418+0000 7f90bf3b6f40 1 -- 172.21.15.38:0/939184927 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557ba7a1d460 con 0x557ba7a19600 2024-09-21T03:43:29.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.420+0000 7f90b8e70640 1 -- 172.21.15.38:0/939184927 <== osd.1 v2:172.21.15.38:6816/1491818732 4 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v128'192 uv192 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f90a8016430 con 0x557ba7a19600 2024-09-21T03:43:29.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.420+0000 7f90bf3b6f40 1 -- 172.21.15.38:0/939184927 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557ba7a1d460 con 0x557ba7a19600 2024-09-21T03:43:29.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.421+0000 7f90b8e70640 1 -- 172.21.15.38:0/939184927 <== osd.1 v2:172.21.15.38:6816/1491818732 5 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v128'193 uv193 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f90a8016430 con 0x557ba7a19600 2024-09-21T03:43:29.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.421+0000 7f90bf3b6f40 1 -- 172.21.15.38:0/939184927 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557ba7a1c450 con 0x557ba7a19600 2024-09-21T03:43:29.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.422+0000 7f90b8e70640 1 -- 172.21.15.38:0/939184927 <== osd.1 v2:172.21.15.38:6816/1491818732 6 ==== osd_op_reply(90 605.00000000 [omap-set-header] v128'194 uv194 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f90a8016430 con 0x557ba7a19600 2024-09-21T03:43:29.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.422+0000 7f90bf3b6f40 1 -- 172.21.15.38:0/939184927 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557ba7a1c450 con 0x557ba7a19600 2024-09-21T03:43:29.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.424+0000 7f90b8e70640 1 -- 172.21.15.38:0/939184927 <== osd.1 v2:172.21.15.38:6816/1491818732 7 ==== osd_op_reply(91 mds0_inotable [create] v128'314 uv308 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f90a8016430 con 0x557ba7a19600 2024-09-21T03:43:29.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.424+0000 7f90bf3b6f40 1 -- 172.21.15.38:0/939184927 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e128) -- 0x557ba7a1c450 con 0x557ba7a19600 2024-09-21T03:43:29.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.425+0000 7f90b8e70640 1 -- 172.21.15.38:0/939184927 <== osd.1 v2:172.21.15.38:6816/1491818732 8 ==== osd_op_reply(92 mds0_inotable [delete] v128'315 uv315 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f90a8016430 con 0x557ba7a19600 2024-09-21T03:43:29.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.425+0000 7f90bf3b6f40 1 -- 172.21.15.38:0/939184927 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557ba7a1c450 con 0x557ba7a19600 2024-09-21T03:43:29.428 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-09-21T03:43:29.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.426+0000 7f90b8e70640 1 -- 172.21.15.38:0/939184927 <== osd.1 v2:172.21.15.38:6816/1491818732 9 ==== osd_op_reply(93 mds0_inotable [create] v128'316 uv316 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f90a8016430 con 0x557ba7a19600 2024-09-21T03:43:29.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.426+0000 7f90bf3b6f40 1 -- 172.21.15.38:0/939184927 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:94 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557ba7a17cf0 con 0x557ba7a19600 2024-09-21T03:43:29.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.427+0000 7f90b8e70640 1 -- 172.21.15.38:0/939184927 <== osd.1 v2:172.21.15.38:6816/1491818732 10 ==== osd_op_reply(94 mds0_inotable [write 0~34] v128'317 uv317 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f90a8016430 con 0x557ba7a19600 2024-09-21T03:43:29.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.427+0000 7f90bf3b6f40 1 -- 172.21.15.38:0/939184927 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:95 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557ba7a17cf0 con 0x557ba7a19600 2024-09-21T03:43:29.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.428+0000 7f90b8e70640 1 -- 172.21.15.38:0/939184927 <== osd.1 v2:172.21.15.38:6816/1491818732 11 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v128'318 uv318 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f90a8016430 con 0x557ba7a19600 2024-09-21T03:43:29.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.428+0000 7f90bf3b6f40 1 -- 172.21.15.38:0/939184927 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:96 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557ba7a17cf0 con 0x557ba7a19600 2024-09-21T03:43:29.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.430+0000 7f90b8e70640 1 -- 172.21.15.38:0/939184927 <== osd.1 v2:172.21.15.38:6816/1491818732 12 ==== osd_op_reply(96 100.00000000 [create] v128'319 uv313 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f90a8016430 con 0x557ba7a19600 2024-09-21T03:43:29.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.430+0000 7f90bf3b6f40 1 -- 172.21.15.38:0/939184927 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:97 13.13 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e128) -- 0x557ba7a17cf0 con 0x557ba7a19600 2024-09-21T03:43:29.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.431+0000 7f90b8e70640 1 -- 172.21.15.38:0/939184927 <== osd.1 v2:172.21.15.38:6816/1491818732 13 ==== osd_op_reply(97 100.00000000 [delete] v128'320 uv320 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f90a8016430 con 0x557ba7a19600 2024-09-21T03:43:29.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.431+0000 7f90bf3b6f40 1 -- 172.21.15.38:0/939184927 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:98 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557ba7a17cf0 con 0x557ba7a19600 2024-09-21T03:43:29.434 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-09-21T03:43:29.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.432+0000 7f90b8e70640 1 -- 172.21.15.38:0/939184927 <== osd.1 v2:172.21.15.38:6816/1491818732 14 ==== osd_op_reply(98 100.00000000 [create] v128'321 uv321 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f90a8016430 con 0x557ba7a19600 2024-09-21T03:43:29.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.432+0000 7f90bf3b6f40 1 -- 172.21.15.38:0/939184927 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:99 13.13 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557ba7a17cf0 con 0x557ba7a19600 2024-09-21T03:43:29.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.433+0000 7f90b8e70640 1 -- 172.21.15.38:0/939184927 <== osd.1 v2:172.21.15.38:6816/1491818732 15 ==== osd_op_reply(99 100.00000000 [omap-set-header] v128'322 uv322 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f90a8016430 con 0x557ba7a19600 2024-09-21T03:43:29.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.433+0000 7f90bf3b6f40 1 -- 172.21.15.38:0/939184927 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:100 13.13 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557ba7a1f2a0 con 0x557ba7a19600 2024-09-21T03:43:29.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.434+0000 7f90b8e70640 1 -- 172.21.15.38:0/939184927 <== osd.1 v2:172.21.15.38:6816/1491818732 16 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v128'323 uv323 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f90a8016430 con 0x557ba7a19600 2024-09-21T03:43:29.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.434+0000 7f90bf3b6f40 1 -- 172.21.15.38:0/939184927 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557ba7a1c900 con 0x557ba7a14420 2024-09-21T03:43:29.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.435+0000 7f90b9671640 1 -- 172.21.15.38:0/939184927 <== osd.3 v2:172.21.15.38:6808/1286810294 16 ==== osd_op_reply(101 mds0_sessionmap [create] v128'137 uv135 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7f90b006a0b0 con 0x557ba7a14420 2024-09-21T03:43:29.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.435+0000 7f90bf3b6f40 1 -- 172.21.15.38:0/939184927 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e128) -- 0x557ba7a1c900 con 0x557ba7a14420 2024-09-21T03:43:29.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.437+0000 7f90b9671640 1 -- 172.21.15.38:0/939184927 <== osd.3 v2:172.21.15.38:6808/1286810294 17 ==== osd_op_reply(102 mds0_sessionmap [delete] v128'138 uv138 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f90b006a0b0 con 0x557ba7a14420 2024-09-21T03:43:29.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.437+0000 7f90bf3b6f40 1 -- 172.21.15.38:0/939184927 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:103 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557ba7a1c900 con 0x557ba7a14420 2024-09-21T03:43:29.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.438+0000 7f90b9671640 1 -- 172.21.15.38:0/939184927 <== osd.3 v2:172.21.15.38:6808/1286810294 18 ==== osd_op_reply(103 mds0_sessionmap [create] v128'139 uv139 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f90b006a0b0 con 0x557ba7a14420 2024-09-21T03:43:29.440 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-09-21T03:43:29.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.438+0000 7f90bf3b6f40 1 -- 172.21.15.38:0/939184927 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:104 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557ba7a1c900 con 0x557ba7a14420 2024-09-21T03:43:29.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.439+0000 7f90b9671640 1 -- 172.21.15.38:0/939184927 <== osd.3 v2:172.21.15.38:6808/1286810294 19 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v128'140 uv140 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f90b006a0b0 con 0x557ba7a14420 2024-09-21T03:43:29.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.439+0000 7f90bf3b6f40 1 -- 172.21.15.38:0/939184927 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557ba7a1c900 con 0x557ba7a14420 2024-09-21T03:43:29.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.440+0000 7f90b9671640 1 -- 172.21.15.38:0/939184927 <== osd.3 v2:172.21.15.38:6808/1286810294 20 ==== osd_op_reply(105 609.00000000 [create] v128'189 uv188 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f90b006a0b0 con 0x557ba7a14420 2024-09-21T03:43:29.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.440+0000 7f90bf3b6f40 1 -- 172.21.15.38:0/939184927 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e128) -- 0x557ba7a1c900 con 0x557ba7a14420 2024-09-21T03:43:29.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.441+0000 7f90b9671640 1 -- 172.21.15.38:0/939184927 <== osd.3 v2:172.21.15.38:6808/1286810294 21 ==== osd_op_reply(106 609.00000000 [delete] v128'190 uv190 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f90b006a0b0 con 0x557ba7a14420 2024-09-21T03:43:29.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.441+0000 7f90bf3b6f40 1 -- 172.21.15.38:0/939184927 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557ba7a1c900 con 0x557ba7a14420 2024-09-21T03:43:29.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.442+0000 7f90b9671640 1 -- 172.21.15.38:0/939184927 <== osd.3 v2:172.21.15.38:6808/1286810294 22 ==== osd_op_reply(107 609.00000000 [create] v128'191 uv191 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f90b006a0b0 con 0x557ba7a14420 2024-09-21T03:43:29.444 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-09-21T03:43:29.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.442+0000 7f90bf3b6f40 1 -- 172.21.15.38:0/939184927 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557ba7a1d910 con 0x557ba7a14420 2024-09-21T03:43:29.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.443+0000 7f90b9671640 1 -- 172.21.15.38:0/939184927 <== osd.3 v2:172.21.15.38:6808/1286810294 23 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v128'192 uv192 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f90b006a0b0 con 0x557ba7a14420 2024-09-21T03:43:29.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.443+0000 7f90bf3b6f40 1 -- 172.21.15.38:0/939184927 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:109 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557ba7a1d910 con 0x557ba7a14420 2024-09-21T03:43:29.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.444+0000 7f90b9671640 1 -- 172.21.15.38:0/939184927 <== osd.3 v2:172.21.15.38:6808/1286810294 24 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v128'193 uv193 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f90b006a0b0 con 0x557ba7a14420 2024-09-21T03:43:29.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.445+0000 7f90bf3b6f40 1 -- 172.21.15.38:0/939184927 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:110 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557ba7a1c900 con 0x557ba7a14420 2024-09-21T03:43:29.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.446+0000 7f90b9671640 1 -- 172.21.15.38:0/939184927 <== osd.3 v2:172.21.15.38:6808/1286810294 25 ==== osd_op_reply(110 609.00000000 [omap-set-header] v128'194 uv194 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f90b006a0b0 con 0x557ba7a14420 2024-09-21T03:43:29.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.446+0000 7f90bf3b6f40 1 --2- 172.21.15.38:0/939184927 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x557ba7a1cf80 0x557ba7a1d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:29.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.446+0000 7f90bf3b6f40 1 -- 172.21.15.38:0/939184927 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557ba7a1d8d0 con 0x557ba7a1cf80 2024-09-21T03:43:29.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.446+0000 7f90b866f640 1 --2- 172.21.15.38:0/939184927 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x557ba7a1cf80 0x557ba7a1d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:29.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.446+0000 7f90b866f640 1 --2- 172.21.15.38:0/939184927 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x557ba7a1cf80 0x557ba7a1d390 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:29.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.447+0000 7f90b866f640 1 -- 172.21.15.38:0/939184927 <== osd.2 v2:172.21.15.38:6824/2577858902 1 ==== osd_op_reply(111 400.00000000 [create] v128'161 uv160 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f90ac056470 con 0x557ba7a1cf80 2024-09-21T03:43:29.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.447+0000 7f90bf3b6f40 1 -- 172.21.15.38:0/939184927 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e128) -- 0x557ba7a1d8d0 con 0x557ba7a1cf80 2024-09-21T03:43:29.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.449+0000 7f90b866f640 1 -- 172.21.15.38:0/939184927 <== osd.2 v2:172.21.15.38:6824/2577858902 2 ==== osd_op_reply(112 400.00000000 [delete] v128'162 uv162 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f90ac056470 con 0x557ba7a1cf80 2024-09-21T03:43:29.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.449+0000 7f90bf3b6f40 1 -- 172.21.15.38:0/939184927 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557ba7a1d8d0 con 0x557ba7a1cf80 2024-09-21T03:43:29.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.450+0000 7f90b866f640 1 -- 172.21.15.38:0/939184927 <== osd.2 v2:172.21.15.38:6824/2577858902 3 ==== osd_op_reply(113 400.00000000 [create] v128'163 uv163 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f90ac056470 con 0x557ba7a1cf80 2024-09-21T03:43:29.452 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-09-21T03:43:29.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.450+0000 7f90bf3b6f40 1 -- 172.21.15.38:0/939184927 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:114 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557ba7a1dd60 con 0x557ba7a1cf80 2024-09-21T03:43:29.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.451+0000 7f90b866f640 1 -- 172.21.15.38:0/939184927 <== osd.2 v2:172.21.15.38:6824/2577858902 4 ==== osd_op_reply(114 400.00000000 [write 0~22] v128'164 uv164 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f90ac056470 con 0x557ba7a1cf80 2024-09-21T03:43:29.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.451+0000 7f90bf3b6f40 1 -- 172.21.15.38:0/939184927 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:115 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557ba7a1dd60 con 0x557ba7a1cf80 2024-09-21T03:43:29.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.452+0000 7f90b866f640 1 -- 172.21.15.38:0/939184927 <== osd.2 v2:172.21.15.38:6824/2577858902 5 ==== osd_op_reply(115 400.00000000 [omap-set-header] v128'165 uv165 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f90ac056470 con 0x557ba7a1cf80 2024-09-21T03:43:29.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.452+0000 7f90bf3b6f40 1 -- 172.21.15.38:0/939184927 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:116 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557ba7a1dd10 con 0x557ba7a19600 2024-09-21T03:43:29.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.453+0000 7f90b8e70640 1 -- 172.21.15.38:0/939184927 <== osd.1 v2:172.21.15.38:6816/1491818732 17 ==== osd_op_reply(116 100.00000000.inode [create] v128'389 uv380 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7f90a8016430 con 0x557ba7a19600 2024-09-21T03:43:29.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.453+0000 7f90bf3b6f40 1 -- 172.21.15.38:0/939184927 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:117 13.1f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e128) -- 0x557ba7a1dd10 con 0x557ba7a19600 2024-09-21T03:43:29.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.454+0000 7f90b8e70640 1 -- 172.21.15.38:0/939184927 <== osd.1 v2:172.21.15.38:6816/1491818732 18 ==== osd_op_reply(117 100.00000000.inode [delete] v128'390 uv390 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f90a8016430 con 0x557ba7a19600 2024-09-21T03:43:29.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.455+0000 7f90bf3b6f40 1 -- 172.21.15.38:0/939184927 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:118 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557ba7a1dd10 con 0x557ba7a19600 2024-09-21T03:43:29.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.456+0000 7f90b8e70640 1 -- 172.21.15.38:0/939184927 <== osd.1 v2:172.21.15.38:6816/1491818732 19 ==== osd_op_reply(118 100.00000000.inode [create] v128'391 uv391 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f90a8016430 con 0x557ba7a19600 2024-09-21T03:43:29.458 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-09-21T03:43:29.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.456+0000 7f90bf3b6f40 1 -- 172.21.15.38:0/939184927 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:119 13.1f 13:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557ba7a1dd10 con 0x557ba7a19600 2024-09-21T03:43:29.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.457+0000 7f90b8e70640 1 -- 172.21.15.38:0/939184927 <== osd.1 v2:172.21.15.38:6816/1491818732 20 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v128'392 uv392 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f90a8016430 con 0x557ba7a19600 2024-09-21T03:43:29.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.457+0000 7f90bf3b6f40 1 -- 172.21.15.38:0/939184927 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:120 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557ba7a1dd10 con 0x557ba7a19600 2024-09-21T03:43:29.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.458+0000 7f90b8e70640 1 -- 172.21.15.38:0/939184927 <== osd.1 v2:172.21.15.38:6816/1491818732 21 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v128'393 uv393 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f90a8016430 con 0x557ba7a19600 2024-09-21T03:43:29.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.458+0000 7f90bf3b6f40 1 -- 172.21.15.38:0/939184927 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:121 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557ba7a1dd10 con 0x557ba7a19600 2024-09-21T03:43:29.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.459+0000 7f90b8e70640 1 -- 172.21.15.38:0/939184927 <== osd.1 v2:172.21.15.38:6816/1491818732 22 ==== osd_op_reply(121 1.00000000 [create] v128'394 uv387 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7f90a8016430 con 0x557ba7a19600 2024-09-21T03:43:29.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.459+0000 7f90bf3b6f40 1 -- 172.21.15.38:0/939184927 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:122 13.1f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e128) -- 0x557ba7a1dd10 con 0x557ba7a19600 2024-09-21T03:43:29.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.460+0000 7f90b8e70640 1 -- 172.21.15.38:0/939184927 <== osd.1 v2:172.21.15.38:6816/1491818732 23 ==== osd_op_reply(122 1.00000000 [delete] v128'395 uv395 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f90a8016430 con 0x557ba7a19600 2024-09-21T03:43:29.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.460+0000 7f90bf3b6f40 1 -- 172.21.15.38:0/939184927 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:123 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557ba7a1dd10 con 0x557ba7a19600 2024-09-21T03:43:29.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.461+0000 7f90b8e70640 1 -- 172.21.15.38:0/939184927 <== osd.1 v2:172.21.15.38:6816/1491818732 24 ==== osd_op_reply(123 1.00000000 [create] v128'396 uv396 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f90a8016430 con 0x557ba7a19600 2024-09-21T03:43:29.463 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-09-21T03:43:29.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.461+0000 7f90bf3b6f40 1 -- 172.21.15.38:0/939184927 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:124 13.1f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557ba7a1dd10 con 0x557ba7a19600 2024-09-21T03:43:29.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.462+0000 7f90b8e70640 1 -- 172.21.15.38:0/939184927 <== osd.1 v2:172.21.15.38:6816/1491818732 25 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v128'397 uv397 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f90a8016430 con 0x557ba7a19600 2024-09-21T03:43:29.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.462+0000 7f90bf3b6f40 1 -- 172.21.15.38:0/939184927 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:125 13.1f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557ba7a1ed20 con 0x557ba7a19600 2024-09-21T03:43:29.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.464+0000 7f90b8e70640 1 -- 172.21.15.38:0/939184927 <== osd.1 v2:172.21.15.38:6816/1491818732 26 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v128'398 uv398 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f90a8016430 con 0x557ba7a19600 2024-09-21T03:43:29.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.464+0000 7f90bf3b6f40 1 -- 172.21.15.38:0/939184927 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:126 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557ba7a1f130 con 0x557ba7a19600 2024-09-21T03:43:29.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.465+0000 7f90b8e70640 1 -- 172.21.15.38:0/939184927 <== osd.1 v2:172.21.15.38:6816/1491818732 27 ==== osd_op_reply(126 1.00000000 [omap-set-header] v128'399 uv399 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f90a8016430 con 0x557ba7a19600 2024-09-21T03:43:29.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.465+0000 7f90bf3b6f40 1 -- 172.21.15.38:0/939184927 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:127 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=584b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557ba7a1f520 con 0x557ba7a19600 2024-09-21T03:43:29.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.466+0000 7f90b8e70640 1 -- 172.21.15.38:0/939184927 <== osd.1 v2:172.21.15.38:6816/1491818732 28 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v128'400 uv400 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f90a8016430 con 0x557ba7a19600 2024-09-21T03:43:29.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.466+0000 7f90bf3b6f40 1 -- 172.21.15.38:0/939184927 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x557ba7a10fd0 msgr2=0x557ba7a133f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:29.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.466+0000 7f90bf3b6f40 1 --2- 172.21.15.38:0/939184927 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x557ba7a10fd0 0x557ba7a133f0 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:29.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.466+0000 7f90bf3b6f40 1 -- 172.21.15.38:0/939184927 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x557ba7a19600 msgr2=0x557ba7a1ba20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:29.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.466+0000 7f90bf3b6f40 1 --2- 172.21.15.38:0/939184927 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x557ba7a19600 0x557ba7a1ba20 crc :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:29.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.466+0000 7f90bf3b6f40 1 -- 172.21.15.38:0/939184927 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x557ba7a1cf80 msgr2=0x557ba7a1d390 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:29.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.466+0000 7f90bf3b6f40 1 --2- 172.21.15.38:0/939184927 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x557ba7a1cf80 0x557ba7a1d390 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:29.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.466+0000 7f90bf3b6f40 1 -- 172.21.15.38:0/939184927 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x557ba7a14420 msgr2=0x557ba7a16840 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:29.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.466+0000 7f90bf3b6f40 1 --2- 172.21.15.38:0/939184927 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x557ba7a14420 0x557ba7a16840 crc :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:29.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.466+0000 7f90b9671640 1 -- 172.21.15.38:0/939184927 reap_dead start 2024-09-21T03:43:29.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.466+0000 7f90bf3b6f40 1 -- 172.21.15.38:0/939184927 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x557ba7a0b600 msgr2=0x557ba7a0e770 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:29.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.466+0000 7f90bf3b6f40 1 --2- 172.21.15.38:0/939184927 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x557ba7a0b600 0x557ba7a0e770 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:29.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.466+0000 7f90b9671640 1 -- 172.21.15.38:0/939184927 reap_dead start 2024-09-21T03:43:29.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.466+0000 7f90bf3b6f40 1 -- 172.21.15.38:0/939184927 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x557ba78ba810 msgr2=0x557ba79fddb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:29.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.466+0000 7f90bf3b6f40 1 --2- 172.21.15.38:0/939184927 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x557ba78ba810 0x557ba79fddb0 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:29.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.466+0000 7f90bf3b6f40 1 -- 172.21.15.38:0/939184927 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f90a4077740 msgr2=0x7f90a4079c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:29.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.466+0000 7f90bf3b6f40 1 --2- 172.21.15.38:0/939184927 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f90a4077740 0x7f90a4079c50 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f90a8006fd0 tx=0x7f90a800d040 comp rx=0 tx=0).stop 2024-09-21T03:43:29.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.466+0000 7f90bf3b6f40 1 -- 172.21.15.38:0/939184927 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557ba7949520 msgr2=0x557ba79ea760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:29.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.466+0000 7f90bf3b6f40 1 --2- 172.21.15.38:0/939184927 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557ba7949520 0x557ba79ea760 secure :-1 s=READY pgs=2035 cs=0 l=1 rev1=1 crypto rx=0x7f90b000e020 tx=0x7f90b000e450 comp rx=0 tx=0).stop 2024-09-21T03:43:29.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.466+0000 7f90bf3b6f40 1 -- 172.21.15.38:0/939184927 shutdown_connections 2024-09-21T03:43:29.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.466+0000 7f90bf3b6f40 1 --2- 172.21.15.38:0/939184927 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x557ba78ba810 0x557ba79fddb0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:29.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.466+0000 7f90bf3b6f40 1 --2- 172.21.15.38:0/939184927 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557ba7949520 0x557ba79ea760 unknown :-1 s=CLOSED pgs=2035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:29.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.466+0000 7f90bf3b6f40 1 --2- 172.21.15.38:0/939184927 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f90a4077740 0x7f90a4079c50 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:29.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.466+0000 7f90bf3b6f40 1 -- 172.21.15.38:0/939184927 >> 172.21.15.38:0/939184927 conn(0x557ba77abac0 msgr2=0x557ba7888bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:29.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.466+0000 7f90bf3b6f40 1 -- 172.21.15.38:0/939184927 shutdown_connections 2024-09-21T03:43:29.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.466+0000 7f90bf3b6f40 1 -- 172.21.15.38:0/939184927 wait complete. 2024-09-21T03:43:29.472 DEBUG:teuthology.orchestra.run.smithi038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata setomapval 100.00000000 stray0_head deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef 2024-09-21T03:43:29.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.569+0000 7f428f23cf40 1 Processor -- start 2024-09-21T03:43:29.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.569+0000 7f428f23cf40 1 -- start start 2024-09-21T03:43:29.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.570+0000 7f428f23cf40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f42740030f0 0x560362bc52e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:29.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.570+0000 7f428f23cf40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560362bc3430 0x560362b1f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:29.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.570+0000 7f428f23cf40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560362bc6450 0x560362bc6860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:29.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.570+0000 7f428f23cf40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x560362a9e430 con 0x7f42740030f0 2024-09-21T03:43:29.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.570+0000 7f428f23cf40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x560362aab1f0 con 0x560362bc3430 2024-09-21T03:43:29.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.570+0000 7f428f23cf40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x560362a3fd60 con 0x560362bc6450 2024-09-21T03:43:29.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.570+0000 7f4288cf8640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f42740030f0 0x560362bc52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:29.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.570+0000 7f4288cf8640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f42740030f0 0x560362bc52e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:58620/0 (socket says 172.21.15.38:58620) 2024-09-21T03:43:29.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.570+0000 7f4288cf8640 1 -- 172.21.15.38:0/241540645 learned_addr learned my addr 172.21.15.38:0/241540645 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:43:29.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.571+0000 7f42894f9640 1 --2- 172.21.15.38:0/241540645 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560362bc6450 0x560362bc6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:29.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.571+0000 7f42884f7640 1 --2- 172.21.15.38:0/241540645 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560362bc3430 0x560362b1f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:29.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.571+0000 7f4288cf8640 1 -- 172.21.15.38:0/241540645 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560362bc6450 msgr2=0x560362bc6860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:29.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.571+0000 7f4288cf8640 1 --2- 172.21.15.38:0/241540645 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560362bc6450 0x560362bc6860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:29.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.571+0000 7f4288cf8640 1 -- 172.21.15.38:0/241540645 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560362bc3430 msgr2=0x560362b1f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:29.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.571+0000 7f4288cf8640 1 --2- 172.21.15.38:0/241540645 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560362bc3430 0x560362b1f4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:29.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.571+0000 7f4288cf8640 1 -- 172.21.15.38:0/241540645 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560362c51a70 con 0x7f42740030f0 2024-09-21T03:43:29.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.571+0000 7f42894f9640 1 --2- 172.21.15.38:0/241540645 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560362bc6450 0x560362bc6860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:43:29.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.571+0000 7f4288cf8640 1 --2- 172.21.15.38:0/241540645 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f42740030f0 0x560362bc52e0 secure :-1 s=READY pgs=2038 cs=0 l=1 rev1=1 crypto rx=0x7f4278009a50 tx=0x7f4278009f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11543d8d9dd986d4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:29.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.571+0000 7f42874f5640 1 -- 172.21.15.38:0/241540645 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f427800d810 con 0x7f42740030f0 2024-09-21T03:43:29.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.571+0000 7f42874f5640 1 -- 172.21.15.38:0/241540645 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f427800d9b0 con 0x7f42740030f0 2024-09-21T03:43:29.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.571+0000 7f42874f5640 1 -- 172.21.15.38:0/241540645 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f427800dde0 con 0x7f42740030f0 2024-09-21T03:43:29.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.572+0000 7f428f23cf40 1 -- 172.21.15.38:0/241540645 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f42740030f0 msgr2=0x560362bc52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:29.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.572+0000 7f428f23cf40 1 --2- 172.21.15.38:0/241540645 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f42740030f0 0x560362bc52e0 secure :-1 s=READY pgs=2038 cs=0 l=1 rev1=1 crypto rx=0x7f4278009a50 tx=0x7f4278009f10 comp rx=0 tx=0).stop 2024-09-21T03:43:29.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.572+0000 7f428f23cf40 1 -- 172.21.15.38:0/241540645 shutdown_connections 2024-09-21T03:43:29.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.572+0000 7f428f23cf40 1 --2- 172.21.15.38:0/241540645 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560362bc6450 0x560362bc6860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:29.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.572+0000 7f428f23cf40 1 --2- 172.21.15.38:0/241540645 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560362bc3430 0x560362b1f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:29.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.572+0000 7f428f23cf40 1 --2- 172.21.15.38:0/241540645 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f42740030f0 0x560362bc52e0 unknown :-1 s=CLOSED pgs=2038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:29.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.572+0000 7f428f23cf40 1 -- 172.21.15.38:0/241540645 >> 172.21.15.38:0/241540645 conn(0x560362a10ac0 msgr2=0x560362aefae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:29.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.572+0000 7f428f23cf40 1 -- 172.21.15.38:0/241540645 shutdown_connections 2024-09-21T03:43:29.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.572+0000 7f428f23cf40 1 -- 172.21.15.38:0/241540645 wait complete. 2024-09-21T03:43:29.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.573+0000 7f428f23cf40 1 Processor -- start 2024-09-21T03:43:29.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.573+0000 7f428f23cf40 1 -- start start 2024-09-21T03:43:29.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.574+0000 7f428f23cf40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560362bc3430 0x560362aef950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:29.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.574+0000 7f428f23cf40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560362bc6450 0x560362bc1250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:29.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.574+0000 7f428f23cf40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f42740030f0 0x560362ba7c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:29.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.574+0000 7f428f23cf40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x560362bc5b30 con 0x7f42740030f0 2024-09-21T03:43:29.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.574+0000 7f428f23cf40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x560362c51d10 con 0x560362bc3430 2024-09-21T03:43:29.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.574+0000 7f428f23cf40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x560362c51ff0 con 0x560362bc6450 2024-09-21T03:43:29.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.574+0000 7f42894f9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f42740030f0 0x560362ba7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:29.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.574+0000 7f42894f9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f42740030f0 0x560362ba7c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:58622/0 (socket says 172.21.15.38:58622) 2024-09-21T03:43:29.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.574+0000 7f42894f9640 1 -- 172.21.15.38:0/2861238331 learned_addr learned my addr 172.21.15.38:0/2861238331 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:43:29.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.574+0000 7f4288cf8640 1 --2- 172.21.15.38:0/2861238331 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560362bc3430 0x560362aef950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:29.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.574+0000 7f42884f7640 1 --2- 172.21.15.38:0/2861238331 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560362bc6450 0x560362bc1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:29.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.575+0000 7f42894f9640 1 -- 172.21.15.38:0/2861238331 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560362bc6450 msgr2=0x560362bc1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:29.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.575+0000 7f42894f9640 1 --2- 172.21.15.38:0/2861238331 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560362bc6450 0x560362bc1250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:29.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.575+0000 7f42894f9640 1 -- 172.21.15.38:0/2861238331 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560362bc3430 msgr2=0x560362aef950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:29.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.575+0000 7f42894f9640 1 --2- 172.21.15.38:0/2861238331 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560362bc3430 0x560362aef950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:29.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.575+0000 7f42894f9640 1 -- 172.21.15.38:0/2861238331 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560362bab380 con 0x7f42740030f0 2024-09-21T03:43:29.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.575+0000 7f42894f9640 1 --2- 172.21.15.38:0/2861238331 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f42740030f0 0x560362ba7c00 secure :-1 s=READY pgs=2039 cs=0 l=1 rev1=1 crypto rx=0x560362af35f0 tx=0x7f428000aaf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:29.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.575+0000 7f42864f3640 1 -- 172.21.15.38:0/2861238331 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42800097f0 con 0x7f42740030f0 2024-09-21T03:43:29.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.575+0000 7f42864f3640 1 -- 172.21.15.38:0/2861238331 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4280009990 con 0x7f42740030f0 2024-09-21T03:43:29.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.575+0000 7f428f23cf40 1 -- 172.21.15.38:0/2861238331 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x560362bab660 con 0x7f42740030f0 2024-09-21T03:43:29.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.575+0000 7f42864f3640 1 -- 172.21.15.38:0/2861238331 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4280009d80 con 0x7f42740030f0 2024-09-21T03:43:29.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.575+0000 7f428f23cf40 1 -- 172.21.15.38:0/2861238331 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560362babc20 con 0x7f42740030f0 2024-09-21T03:43:29.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.577+0000 7f42864f3640 1 -- 172.21.15.38:0/2861238331 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f428000b440 con 0x7f42740030f0 2024-09-21T03:43:29.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.577+0000 7f42864f3640 1 --2- 172.21.15.38:0/2861238331 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f42680775a0 0x7f4268079ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:29.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.577+0000 7f42864f3640 1 -- 172.21.15.38:0/2861238331 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 9056+0+0 (secure 0 0 0) 0x7f428009aee0 con 0x7f42740030f0 2024-09-21T03:43:29.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.577+0000 7f4288cf8640 1 --2- 172.21.15.38:0/2861238331 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f42680775a0 0x7f4268079ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:29.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.578+0000 7f428f23cf40 1 --2- 172.21.15.38:0/2861238331 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x560362b36020 0x560362b36430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:29.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.578+0000 7f428f23cf40 1 -- 172.21.15.38:0/2861238331 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:1 13.13 13:cd5a64a3:::100.00000000:head [omap-set-vals in=103b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x560362bc3070 con 0x560362b36020 2024-09-21T03:43:29.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.578+0000 7f42884f7640 1 --2- 172.21.15.38:0/2861238331 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x560362b36020 0x560362b36430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:29.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.578+0000 7f4288cf8640 1 --2- 172.21.15.38:0/2861238331 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f42680775a0 0x7f4268079ab0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f42780099f0 tx=0x7f42780014a0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:29.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.578+0000 7f42884f7640 1 --2- 172.21.15.38:0/2861238331 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x560362b36020 0x560362b36430 crc :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:29.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.579+0000 7f42884f7640 1 -- 172.21.15.38:0/2861238331 <== osd.1 v2:172.21.15.38:6816/1491818732 1 ==== osd_op_reply(1 100.00000000 [omap-set-vals] v128'324 uv324 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f42700014b0 con 0x560362b36020 2024-09-21T03:43:29.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.579+0000 7f428f23cf40 1 -- 172.21.15.38:0/2861238331 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x560362b36020 msgr2=0x560362b36430 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:29.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.579+0000 7f428f23cf40 1 --2- 172.21.15.38:0/2861238331 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x560362b36020 0x560362b36430 crc :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:29.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.580+0000 7f428f23cf40 1 -- 172.21.15.38:0/2861238331 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f42680775a0 msgr2=0x7f4268079ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:29.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.580+0000 7f428f23cf40 1 --2- 172.21.15.38:0/2861238331 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f42680775a0 0x7f4268079ab0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f42780099f0 tx=0x7f42780014a0 comp rx=0 tx=0).stop 2024-09-21T03:43:29.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.580+0000 7f428f23cf40 1 -- 172.21.15.38:0/2861238331 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f42740030f0 msgr2=0x560362ba7c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:29.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.580+0000 7f428f23cf40 1 --2- 172.21.15.38:0/2861238331 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f42740030f0 0x560362ba7c00 secure :-1 s=READY pgs=2039 cs=0 l=1 rev1=1 crypto rx=0x560362af35f0 tx=0x7f428000aaf0 comp rx=0 tx=0).stop 2024-09-21T03:43:29.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.580+0000 7f42894f9640 1 -- 172.21.15.38:0/2861238331 reap_dead start 2024-09-21T03:43:29.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.580+0000 7f428f23cf40 1 -- 172.21.15.38:0/2861238331 shutdown_connections 2024-09-21T03:43:29.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.580+0000 7f428f23cf40 1 -- 172.21.15.38:0/2861238331 >> 172.21.15.38:0/2861238331 conn(0x560362a10ac0 msgr2=0x560362bc6860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:29.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.580+0000 7f428f23cf40 1 -- 172.21.15.38:0/2861238331 shutdown_connections 2024-09-21T03:43:29.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.580+0000 7f428f23cf40 1 -- 172.21.15.38:0/2861238331 wait complete. 2024-09-21T03:43:29.584 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-21T03:43:29.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.702+0000 7faca4a94640 1 -- 172.21.15.38:0/645083055 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fac88004800 msgr2=0x7fac88004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:29.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.702+0000 7faca4a94640 1 --2- 172.21.15.38:0/645083055 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fac88004800 0x7fac88004c10 secure :-1 s=READY pgs=2041 cs=0 l=1 rev1=1 crypto rx=0x7faca00d28f0 tx=0x7fac88005830 comp rx=0 tx=0).stop 2024-09-21T03:43:29.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.703+0000 7faca4a94640 1 -- 172.21.15.38:0/645083055 shutdown_connections 2024-09-21T03:43:29.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.703+0000 7faca4a94640 1 --2- 172.21.15.38:0/645083055 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fac88004800 0x7fac88004c10 unknown :-1 s=CLOSED pgs=2041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:29.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.703+0000 7faca4a94640 1 -- 172.21.15.38:0/645083055 >> 172.21.15.38:0/645083055 conn(0x7faca009d830 msgr2=0x7faca00b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:29.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.703+0000 7faca4a94640 1 -- 172.21.15.38:0/645083055 shutdown_connections 2024-09-21T03:43:29.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.703+0000 7faca4a94640 1 -- 172.21.15.38:0/645083055 wait complete. 2024-09-21T03:43:29.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.705+0000 7faca4a94640 1 Processor -- start 2024-09-21T03:43:29.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.705+0000 7faca4a94640 1 -- start start 2024-09-21T03:43:29.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.706+0000 7faca4a94640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fac88004800 0x7faca014f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:29.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.706+0000 7faca4a94640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faca01577f0 0x7faca00b21e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:29.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.706+0000 7faca4a94640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faca00b2720 0x7faca006e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:29.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.706+0000 7faca4a94640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7faca00b9c50 con 0x7faca00b2720 2024-09-21T03:43:29.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.706+0000 7faca4a94640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7faca00b9ad0 con 0x7fac88004800 2024-09-21T03:43:29.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.706+0000 7faca4a94640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7faca00b9dd0 con 0x7faca01577f0 2024-09-21T03:43:29.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.706+0000 7fac9ed9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faca00b2720 0x7faca006e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:29.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.706+0000 7fac9ed9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faca00b2720 0x7faca006e1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:58644/0 (socket says 172.21.15.38:58644) 2024-09-21T03:43:29.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.706+0000 7fac9ed9d640 1 -- 172.21.15.38:0/2275856184 learned_addr learned my addr 172.21.15.38:0/2275856184 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:43:29.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.706+0000 7fac9e59c640 1 --2- 172.21.15.38:0/2275856184 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fac88004800 0x7faca014f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:29.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.706+0000 7fac9dd9b640 1 --2- 172.21.15.38:0/2275856184 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faca01577f0 0x7faca00b21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:29.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.706+0000 7fac9ed9d640 1 -- 172.21.15.38:0/2275856184 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faca01577f0 msgr2=0x7faca00b21e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:29.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.706+0000 7fac9ed9d640 1 --2- 172.21.15.38:0/2275856184 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faca01577f0 0x7faca00b21e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:29.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.706+0000 7fac9ed9d640 1 -- 172.21.15.38:0/2275856184 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fac88004800 msgr2=0x7faca014f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:29.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.706+0000 7fac9ed9d640 1 --2- 172.21.15.38:0/2275856184 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fac88004800 0x7faca014f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:29.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.706+0000 7fac9ed9d640 1 -- 172.21.15.38:0/2275856184 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faca00b2b30 con 0x7faca00b2720 2024-09-21T03:43:29.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.706+0000 7fac9e59c640 1 --2- 172.21.15.38:0/2275856184 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fac88004800 0x7faca014f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:43:29.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.707+0000 7fac9ed9d640 1 --2- 172.21.15.38:0/2275856184 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faca00b2720 0x7faca006e1e0 secure :-1 s=READY pgs=2042 cs=0 l=1 rev1=1 crypto rx=0x7faca00b3c20 tx=0x7fac980064f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:29.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.707+0000 7fac8effd640 1 -- 172.21.15.38:0/2275856184 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac98011900 con 0x7faca00b2720 2024-09-21T03:43:29.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.707+0000 7faca4a94640 1 -- 172.21.15.38:0/2275856184 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faca006d030 con 0x7faca00b2720 2024-09-21T03:43:29.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.707+0000 7fac8effd640 1 -- 172.21.15.38:0/2275856184 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fac98019070 con 0x7faca00b2720 2024-09-21T03:43:29.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.707+0000 7fac8effd640 1 -- 172.21.15.38:0/2275856184 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac98011d40 con 0x7faca00b2720 2024-09-21T03:43:29.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.707+0000 7faca4a94640 1 -- 172.21.15.38:0/2275856184 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faca006d5f0 con 0x7faca00b2720 2024-09-21T03:43:29.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.708+0000 7faca4a94640 1 -- 172.21.15.38:0/2275856184 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faca00ce730 con 0x7faca00b2720 2024-09-21T03:43:29.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.709+0000 7fac8effd640 1 -- 172.21.15.38:0/2275856184 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fac9801e460 con 0x7faca00b2720 2024-09-21T03:43:29.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.709+0000 7fac8effd640 1 --2- 172.21.15.38:0/2275856184 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fac80078b90 0x7fac8007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:29.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.709+0000 7fac8effd640 1 -- 172.21.15.38:0/2275856184 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 9056+0+0 (secure 0 0 0) 0x7fac9809c2f0 con 0x7faca00b2720 2024-09-21T03:43:29.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.709+0000 7fac9e59c640 1 --2- 172.21.15.38:0/2275856184 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fac80078b90 0x7fac8007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:29.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.710+0000 7fac9e59c640 1 --2- 172.21.15.38:0/2275856184 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fac80078b90 0x7fac8007b0a0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7faca006bb20 tx=0x7fac90006b70 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:29.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.714+0000 7fac8effd640 1 -- 172.21.15.38:0/2275856184 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7faca00b2b30 con 0x7faca00b2720 2024-09-21T03:43:29.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:29.946+0000 7faca4a94640 1 -- 172.21.15.38:0/2275856184 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7faca00ceb90 con 0x7faca00b2720 2024-09-21T03:43:30.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:30.569+0000 7fac8effd640 1 -- 172.21.15.38:0/2275856184 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v278) ==== 162+0+0 (secure 0 0 0) 0x7faca01592d0 con 0x7faca00b2720 2024-09-21T03:43:30.571 INFO:teuthology.orchestra.run.smithi038.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-21T03:43:30.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:30.573+0000 7faca4a94640 1 -- 172.21.15.38:0/2275856184 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fac80078b90 msgr2=0x7fac8007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:30.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:30.573+0000 7faca4a94640 1 --2- 172.21.15.38:0/2275856184 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fac80078b90 0x7fac8007b0a0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7faca006bb20 tx=0x7fac90006b70 comp rx=0 tx=0).stop 2024-09-21T03:43:30.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:30.573+0000 7faca4a94640 1 -- 172.21.15.38:0/2275856184 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faca00b2720 msgr2=0x7faca006e1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:30.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:30.573+0000 7faca4a94640 1 --2- 172.21.15.38:0/2275856184 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faca00b2720 0x7faca006e1e0 secure :-1 s=READY pgs=2042 cs=0 l=1 rev1=1 crypto rx=0x7faca00b3c20 tx=0x7fac980064f0 comp rx=0 tx=0).stop 2024-09-21T03:43:30.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:30.573+0000 7faca4a94640 1 -- 172.21.15.38:0/2275856184 shutdown_connections 2024-09-21T03:43:30.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:30.573+0000 7faca4a94640 1 --2- 172.21.15.38:0/2275856184 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faca00b2720 0x7faca006e1e0 unknown :-1 s=CLOSED pgs=2042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:30.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:30.573+0000 7faca4a94640 1 --2- 172.21.15.38:0/2275856184 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fac80078b90 0x7fac8007b0a0 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:30.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:30.573+0000 7faca4a94640 1 --2- 172.21.15.38:0/2275856184 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faca01577f0 0x7faca00b21e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:30.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:30.573+0000 7faca4a94640 1 --2- 172.21.15.38:0/2275856184 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fac88004800 0x7faca014f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:30.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:30.573+0000 7faca4a94640 1 -- 172.21.15.38:0/2275856184 >> 172.21.15.38:0/2275856184 conn(0x7faca009d830 msgr2=0x7faca00b33b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:30.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:30.573+0000 7faca4a94640 1 -- 172.21.15.38:0/2275856184 shutdown_connections 2024-09-21T03:43:30.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:30.573+0000 7faca4a94640 1 -- 172.21.15.38:0/2275856184 wait complete. 2024-09-21T03:43:30.592 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:43:30.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:30.713+0000 7fb04417b640 1 -- 172.21.15.38:0/1275684707 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb03c15b9f0 msgr2=0x7fb03c0cf930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:30.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:30.713+0000 7fb04417b640 1 --2- 172.21.15.38:0/1275684707 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb03c15b9f0 0x7fb03c0cf930 secure :-1 s=READY pgs=2044 cs=0 l=1 rev1=1 crypto rx=0x7fb038009830 tx=0x7fb03802a410 comp rx=0 tx=0).stop 2024-09-21T03:43:30.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:30.713+0000 7fb04417b640 1 -- 172.21.15.38:0/1275684707 shutdown_connections 2024-09-21T03:43:30.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:30.713+0000 7fb04417b640 1 --2- 172.21.15.38:0/1275684707 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb03c0cefa0 0x7fb03c0ce320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:30.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:30.713+0000 7fb04417b640 1 --2- 172.21.15.38:0/1275684707 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb03c15b9f0 0x7fb03c0cf930 unknown :-1 s=CLOSED pgs=2044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:30.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:30.713+0000 7fb04417b640 1 -- 172.21.15.38:0/1275684707 >> 172.21.15.38:0/1275684707 conn(0x7fb03c0cca40 msgr2=0x7fb03c0cce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:30.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:30.713+0000 7fb04417b640 1 -- 172.21.15.38:0/1275684707 shutdown_connections 2024-09-21T03:43:30.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:30.713+0000 7fb04417b640 1 -- 172.21.15.38:0/1275684707 wait complete. 2024-09-21T03:43:30.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:30.715+0000 7fb04417b640 1 Processor -- start 2024-09-21T03:43:30.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:30.715+0000 7fb04417b640 1 -- start start 2024-09-21T03:43:30.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:30.716+0000 7fb04417b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb03c0ce860 0x7fb03c15e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:30.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:30.716+0000 7fb04417b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb03c0cefa0 0x7fb03c1f5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:30.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:30.716+0000 7fb04417b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb03c15b9f0 0x7fb03c09e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:30.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:30.716+0000 7fb04417b640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fb03c06df70 con 0x7fb03c0cefa0 2024-09-21T03:43:30.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:30.716+0000 7fb04417b640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fb03c06ddf0 con 0x7fb03c15b9f0 2024-09-21T03:43:30.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:30.716+0000 7fb04417b640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fb03c06e0f0 con 0x7fb03c0ce860 2024-09-21T03:43:30.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:30.716+0000 7fb041716640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb03c0cefa0 0x7fb03c1f5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:30.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:30.716+0000 7fb041716640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb03c0cefa0 0x7fb03c1f5120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:58658/0 (socket says 172.21.15.38:58658) 2024-09-21T03:43:30.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:30.716+0000 7fb041716640 1 -- 172.21.15.38:0/496859964 learned_addr learned my addr 172.21.15.38:0/496859964 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:43:30.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:30.717+0000 7fb041f17640 1 --2- 172.21.15.38:0/496859964 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb03c0ce860 0x7fb03c15e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:30.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:30.717+0000 7fb041716640 1 -- 172.21.15.38:0/496859964 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb03c0ce860 msgr2=0x7fb03c15e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:30.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:30.717+0000 7fb041716640 1 --2- 172.21.15.38:0/496859964 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb03c0ce860 0x7fb03c15e710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:30.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:30.717+0000 7fb041716640 1 -- 172.21.15.38:0/496859964 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb03c15b9f0 msgr2=0x7fb03c09e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:30.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:30.717+0000 7fb041716640 1 --2- 172.21.15.38:0/496859964 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb03c15b9f0 0x7fb03c09e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:30.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:30.717+0000 7fb041716640 1 -- 172.21.15.38:0/496859964 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb03c09fc60 con 0x7fb03c0cefa0 2024-09-21T03:43:30.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:30.717+0000 7fb041716640 1 --2- 172.21.15.38:0/496859964 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb03c0cefa0 0x7fb03c1f5120 secure :-1 s=READY pgs=2045 cs=0 l=1 rev1=1 crypto rx=0x7fb02c00d6d0 tx=0x7fb02c00db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:30.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:30.717+0000 7fb0327fc640 1 -- 172.21.15.38:0/496859964 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb02c017070 con 0x7fb03c0cefa0 2024-09-21T03:43:30.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:30.717+0000 7fb0327fc640 1 -- 172.21.15.38:0/496859964 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb02c009b10 con 0x7fb03c0cefa0 2024-09-21T03:43:30.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:30.717+0000 7fb04417b640 1 -- 172.21.15.38:0/496859964 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb03c09ff40 con 0x7fb03c0cefa0 2024-09-21T03:43:30.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:30.717+0000 7fb0327fc640 1 -- 172.21.15.38:0/496859964 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb02c010460 con 0x7fb03c0cefa0 2024-09-21T03:43:30.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:30.717+0000 7fb04417b640 1 -- 172.21.15.38:0/496859964 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb03c15cf80 con 0x7fb03c0cefa0 2024-09-21T03:43:30.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:30.718+0000 7fb04417b640 1 -- 172.21.15.38:0/496859964 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb03c06e610 con 0x7fb03c0cefa0 2024-09-21T03:43:30.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:30.719+0000 7fb0327fc640 1 -- 172.21.15.38:0/496859964 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb02c01a070 con 0x7fb03c0cefa0 2024-09-21T03:43:30.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:30.720+0000 7fb0327fc640 1 --2- 172.21.15.38:0/496859964 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb020078b90 0x7fb02007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:30.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:30.720+0000 7fb0327fc640 1 -- 172.21.15.38:0/496859964 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 9056+0+0 (secure 0 0 0) 0x7fb02c09bc30 con 0x7fb03c0cefa0 2024-09-21T03:43:30.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:30.720+0000 7fb041f17640 1 --2- 172.21.15.38:0/496859964 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb020078b90 0x7fb02007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:30.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:30.721+0000 7fb041f17640 1 --2- 172.21.15.38:0/496859964 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb020078b90 0x7fb02007b0a0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fb038034000 tx=0x7fb03802bf90 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:30.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:30.724+0000 7fb0327fc640 1 -- 172.21.15.38:0/496859964 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fb02c063310 con 0x7fb03c0cefa0 2024-09-21T03:43:30.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:30.950+0000 7fb04417b640 1 -- 172.21.15.38:0/496859964 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb03c0a0110 con 0x7fb03c0cefa0 2024-09-21T03:43:30.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:30.951+0000 7fb0327fc640 1 -- 172.21.15.38:0/496859964 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 278 v278) ==== 95+0+5227 (secure 0 0 0) 0x7fb02c062c10 con 0x7fb03c0cefa0 2024-09-21T03:43:30.953 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:43:30.953 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":278,"btime":"2024-09-21T03:43:30:567634+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6086,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/965972663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":965972663},{"type":"v1","addr":"172.21.15.195:6836","nonce":965972663}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6700,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/3176122334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":3176122334},{"type":"v1","addr":"172.21.15.195:6837","nonce":3176122334}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":9840,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/3801772046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":3801772046},{"type":"v1","addr":"172.21.15.38:6835","nonce":3801772046}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":273}],"filesystems":[{"mdsmap":{"epoch":278,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:40:02.319282+0000","modified":"2024-09-21T03:43:30.567610+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":128,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9918},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9918":{"gid":9918,"name":"c","rank":0,"incarnation":278,"state":"up:replay","state_seq":1,"addr":"172.21.15.38:6833/1643781179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":1643781179},{"type":"v1","addr":"172.21.15.38:6833","nonce":1643781179}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-21T03:43:30.954 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 278 2024-09-21T03:43:30.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:30.953+0000 7fb04417b640 1 -- 172.21.15.38:0/496859964 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb020078b90 msgr2=0x7fb02007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:30.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:30.953+0000 7fb04417b640 1 --2- 172.21.15.38:0/496859964 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb020078b90 0x7fb02007b0a0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fb038034000 tx=0x7fb03802bf90 comp rx=0 tx=0).stop 2024-09-21T03:43:30.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:30.953+0000 7fb04417b640 1 -- 172.21.15.38:0/496859964 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb03c0cefa0 msgr2=0x7fb03c1f5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:30.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:30.953+0000 7fb04417b640 1 --2- 172.21.15.38:0/496859964 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb03c0cefa0 0x7fb03c1f5120 secure :-1 s=READY pgs=2045 cs=0 l=1 rev1=1 crypto rx=0x7fb02c00d6d0 tx=0x7fb02c00db90 comp rx=0 tx=0).stop 2024-09-21T03:43:30.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:30.953+0000 7fb04417b640 1 -- 172.21.15.38:0/496859964 shutdown_connections 2024-09-21T03:43:30.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:30.953+0000 7fb04417b640 1 --2- 172.21.15.38:0/496859964 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb03c15b9f0 0x7fb03c09e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:30.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:30.953+0000 7fb04417b640 1 --2- 172.21.15.38:0/496859964 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb03c0cefa0 0x7fb03c1f5120 unknown :-1 s=CLOSED pgs=2045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:30.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:30.953+0000 7fb04417b640 1 --2- 172.21.15.38:0/496859964 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb020078b90 0x7fb02007b0a0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:30.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:30.953+0000 7fb04417b640 1 --2- 172.21.15.38:0/496859964 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb03c0ce860 0x7fb03c15e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:30.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:30.953+0000 7fb04417b640 1 -- 172.21.15.38:0/496859964 >> 172.21.15.38:0/496859964 conn(0x7fb03c0cca40 msgr2=0x7fb03c0b4680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:30.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:30.953+0000 7fb04417b640 1 -- 172.21.15.38:0/496859964 shutdown_connections 2024-09-21T03:43:30.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:30.953+0000 7fb04417b640 1 -- 172.21.15.38:0/496859964 wait complete. 2024-09-21T03:43:30.967 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2024-09-21T03:43:31.796 DEBUG:teuthology.orchestra.run.smithi038:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T03:43:31.799 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T03:43:33.594 INFO:tasks.ceph.mds.c.smithi038.stderr:2024-09-21T03:43:33.592+0000 7fc65302b640 -1 mds.0.damage notify_dentry Damage to dentries in fragment * of ino 0x100is fatal because it is a system directory for this rank 2024-09-21T03:43:33.597 INFO:tasks.ceph.mon.a.smithi038.stderr:2024-09-21T03:43:33.596+0000 7f3f45713640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T03:43:33.598 INFO:tasks.ceph.mon.a.smithi038.stderr:2024-09-21T03:43:33.596+0000 7f3f45713640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-09-21T03:43:34.213 INFO:tasks.ceph.mds.c.smithi038.stderr:2024-09-21T03:43:34.211+0000 7fc657033640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T03:43:35.967 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:43:36.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:36.096+0000 7fa476663640 1 -- 172.21.15.38:0/443759982 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa470162280 msgr2=0x7fa470162690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:36.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:36.096+0000 7fa476663640 1 --2- 172.21.15.38:0/443759982 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa470162280 0x7fa470162690 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fa464009830 tx=0x7fa46402a4b0 comp rx=0 tx=0).stop 2024-09-21T03:43:36.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:36.097+0000 7fa476663640 1 -- 172.21.15.38:0/443759982 shutdown_connections 2024-09-21T03:43:36.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:36.097+0000 7fa476663640 1 --2- 172.21.15.38:0/443759982 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa470162280 0x7fa470162690 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:36.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:36.097+0000 7fa476663640 1 -- 172.21.15.38:0/443759982 >> 172.21.15.38:0/443759982 conn(0x7fa4700b6b20 msgr2=0x7fa4700d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:36.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:36.097+0000 7fa476663640 1 -- 172.21.15.38:0/443759982 shutdown_connections 2024-09-21T03:43:36.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:36.097+0000 7fa476663640 1 -- 172.21.15.38:0/443759982 wait complete. 2024-09-21T03:43:36.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:36.098+0000 7fa476663640 1 Processor -- start 2024-09-21T03:43:36.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:36.099+0000 7fa476663640 1 -- start start 2024-09-21T03:43:36.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:36.100+0000 7fa476663640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa4700b2e40 0x7fa47014e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:36.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:36.100+0000 7fa476663640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa470162280 0x7fa4700ce7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:36.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:36.100+0000 7fa476663640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa4700b2920 0x7fa4700b7ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:36.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:36.100+0000 7fa476663640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa47015f640 con 0x7fa4700b2e40 2024-09-21T03:43:36.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:36.100+0000 7fa476663640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fa47015f4c0 con 0x7fa470162280 2024-09-21T03:43:36.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:36.100+0000 7fa476663640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fa47015f7c0 con 0x7fa4700b2920 2024-09-21T03:43:36.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:36.100+0000 7fa46ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa4700b2e40 0x7fa47014e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:36.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:36.100+0000 7fa46ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa4700b2e40 0x7fa47014e1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49716/0 (socket says 172.21.15.38:49716) 2024-09-21T03:43:36.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:36.100+0000 7fa46ffff640 1 -- 172.21.15.38:0/1380274098 learned_addr learned my addr 172.21.15.38:0/1380274098 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:43:36.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:36.100+0000 7fa46f7fe640 1 --2- 172.21.15.38:0/1380274098 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa470162280 0x7fa4700ce7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:36.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:36.100+0000 7fa474c00640 1 --2- 172.21.15.38:0/1380274098 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa4700b2920 0x7fa4700b7ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:36.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:36.101+0000 7fa46ffff640 1 -- 172.21.15.38:0/1380274098 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa4700b2920 msgr2=0x7fa4700b7ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:36.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:36.101+0000 7fa46ffff640 1 --2- 172.21.15.38:0/1380274098 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa4700b2920 0x7fa4700b7ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:36.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:36.101+0000 7fa46ffff640 1 -- 172.21.15.38:0/1380274098 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa470162280 msgr2=0x7fa4700ce7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:36.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:36.101+0000 7fa46ffff640 1 --2- 172.21.15.38:0/1380274098 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa470162280 0x7fa4700ce7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:36.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:36.101+0000 7fa46ffff640 1 -- 172.21.15.38:0/1380274098 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4700ced20 con 0x7fa4700b2e40 2024-09-21T03:43:36.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:36.101+0000 7fa474c00640 1 --2- 172.21.15.38:0/1380274098 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa4700b2920 0x7fa4700b7ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:43:36.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:36.101+0000 7fa46ffff640 1 --2- 172.21.15.38:0/1380274098 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa4700b2e40 0x7fa47014e1b0 secure :-1 s=READY pgs=2047 cs=0 l=1 rev1=1 crypto rx=0x7fa46000c460 tx=0x7fa46000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:36.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:36.101+0000 7fa46cff9640 1 -- 172.21.15.38:0/1380274098 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4600052d0 con 0x7fa4700b2e40 2024-09-21T03:43:36.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:36.101+0000 7fa46cff9640 1 -- 172.21.15.38:0/1380274098 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa460005490 con 0x7fa4700b2e40 2024-09-21T03:43:36.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:36.101+0000 7fa476663640 1 -- 172.21.15.38:0/1380274098 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4700b1880 con 0x7fa4700b2e40 2024-09-21T03:43:36.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:36.101+0000 7fa46cff9640 1 -- 172.21.15.38:0/1380274098 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa460012870 con 0x7fa4700b2e40 2024-09-21T03:43:36.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:36.101+0000 7fa476663640 1 -- 172.21.15.38:0/1380274098 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4700b1e40 con 0x7fa4700b2e40 2024-09-21T03:43:36.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:36.102+0000 7fa476663640 1 -- 172.21.15.38:0/1380274098 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa470162f00 con 0x7fa4700b2e40 2024-09-21T03:43:36.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:36.103+0000 7fa46cff9640 1 -- 172.21.15.38:0/1380274098 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa460012a10 con 0x7fa4700b2e40 2024-09-21T03:43:36.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:36.103+0000 7fa46cff9640 1 --2- 172.21.15.38:0/1380274098 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa45c078b90 0x7fa45c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:36.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:36.104+0000 7fa46cff9640 1 -- 172.21.15.38:0/1380274098 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 9142+0+0 (secure 0 0 0) 0x7fa46009b670 con 0x7fa4700b2e40 2024-09-21T03:43:36.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:36.104+0000 7fa46f7fe640 1 --2- 172.21.15.38:0/1380274098 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa45c078b90 0x7fa45c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:36.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:36.104+0000 7fa46f7fe640 1 --2- 172.21.15.38:0/1380274098 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa45c078b90 0x7fa45c07b0a0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fa464034000 tx=0x7fa464004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:36.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:36.108+0000 7fa46cff9640 1 -- 172.21.15.38:0/1380274098 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fa4700ced20 con 0x7fa4700b2e40 2024-09-21T03:43:36.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:36.334+0000 7fa476663640 1 -- 172.21.15.38:0/1380274098 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa4700b48d0 con 0x7fa4700b2e40 2024-09-21T03:43:36.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:36.335+0000 7fa46cff9640 1 -- 172.21.15.38:0/1380274098 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 282 v282) ==== 95+0+4392 (secure 0 0 0) 0x7fa460062c10 con 0x7fa4700b2e40 2024-09-21T03:43:36.337 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:43:36.338 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":282,"btime":"2024-09-21T03:43:33:596835+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6086,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/965972663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":965972663},{"type":"v1","addr":"172.21.15.195:6836","nonce":965972663}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6700,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/3176122334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":3176122334},{"type":"v1","addr":"172.21.15.195:6837","nonce":3176122334}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":9840,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/3801772046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":3801772046},{"type":"v1","addr":"172.21.15.38:6835","nonce":3801772046}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":273}],"filesystems":[{"mdsmap":{"epoch":282,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:40:02.319282+0000","modified":"2024-09-21T03:43:33.596831+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":129,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-21T03:43:36.338 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 282 2024-09-21T03:43:36.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:36.337+0000 7fa476663640 1 -- 172.21.15.38:0/1380274098 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa45c078b90 msgr2=0x7fa45c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:36.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:36.337+0000 7fa476663640 1 --2- 172.21.15.38:0/1380274098 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa45c078b90 0x7fa45c07b0a0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fa464034000 tx=0x7fa464004730 comp rx=0 tx=0).stop 2024-09-21T03:43:36.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:36.337+0000 7fa476663640 1 -- 172.21.15.38:0/1380274098 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa4700b2e40 msgr2=0x7fa47014e1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:36.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:36.337+0000 7fa476663640 1 --2- 172.21.15.38:0/1380274098 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa4700b2e40 0x7fa47014e1b0 secure :-1 s=READY pgs=2047 cs=0 l=1 rev1=1 crypto rx=0x7fa46000c460 tx=0x7fa46000c920 comp rx=0 tx=0).stop 2024-09-21T03:43:36.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:36.337+0000 7fa476663640 1 -- 172.21.15.38:0/1380274098 shutdown_connections 2024-09-21T03:43:36.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:36.337+0000 7fa476663640 1 --2- 172.21.15.38:0/1380274098 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa45c078b90 0x7fa45c07b0a0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:36.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:36.337+0000 7fa476663640 1 --2- 172.21.15.38:0/1380274098 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa4700b2920 0x7fa4700b7ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:36.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:36.337+0000 7fa476663640 1 --2- 172.21.15.38:0/1380274098 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa470162280 0x7fa4700ce7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:36.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:36.337+0000 7fa476663640 1 --2- 172.21.15.38:0/1380274098 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa4700b2e40 0x7fa47014e1b0 unknown :-1 s=CLOSED pgs=2047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:36.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:36.337+0000 7fa476663640 1 -- 172.21.15.38:0/1380274098 >> 172.21.15.38:0/1380274098 conn(0x7fa4700b6b20 msgr2=0x7fa4700b3be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:36.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:36.337+0000 7fa476663640 1 -- 172.21.15.38:0/1380274098 shutdown_connections 2024-09-21T03:43:36.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:36.337+0000 7fa476663640 1 -- 172.21.15.38:0/1380274098 wait complete. 2024-09-21T03:43:36.351 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-09-21T03:43:36.352 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:43:36.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:36.464+0000 7f9df6671640 1 -- 172.21.15.38:0/3263903240 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9df00b2e40 msgr2=0x7f9df00b3250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:36.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:36.465+0000 7f9df6671640 1 --2- 172.21.15.38:0/3263903240 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9df00b2e40 0x7f9df00b3250 secure :-1 s=READY pgs=2048 cs=0 l=1 rev1=1 crypto rx=0x7f9df00d5040 tx=0x7f9de802aa30 comp rx=0 tx=0).stop 2024-09-21T03:43:36.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:36.465+0000 7f9df6671640 1 -- 172.21.15.38:0/3263903240 shutdown_connections 2024-09-21T03:43:36.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:36.465+0000 7f9df6671640 1 --2- 172.21.15.38:0/3263903240 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9df00b2e40 0x7f9df00b3250 unknown :-1 s=CLOSED pgs=2048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:36.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:36.465+0000 7f9df6671640 1 --2- 172.21.15.38:0/3263903240 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9df0162280 0x7f9df0162690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:36.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:36.465+0000 7f9df6671640 1 --2- 172.21.15.38:0/3263903240 >> v2:172.21.15.195:3300/0 conn(0x7f9df0162f00 0x7f9df00d24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:36.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:36.465+0000 7f9df6671640 1 -- 172.21.15.38:0/3263903240 >> 172.21.15.38:0/3263903240 conn(0x7f9df00b6b20 msgr2=0x7f9df00d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:36.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:36.465+0000 7f9df6671640 1 -- 172.21.15.38:0/3263903240 shutdown_connections 2024-09-21T03:43:36.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:36.465+0000 7f9df6671640 1 -- 172.21.15.38:0/3263903240 wait complete. 2024-09-21T03:43:36.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:36.466+0000 7f9df6671640 1 Processor -- start 2024-09-21T03:43:36.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:36.467+0000 7f9df6671640 1 -- start start 2024-09-21T03:43:36.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:36.468+0000 7f9df6671640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9df00b2e40 0x7f9df0157940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:36.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:36.468+0000 7f9df6671640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9df0162280 0x7f9df015a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:36.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:36.468+0000 7f9df6671640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9df0161120 0x7f9df0161530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:36.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:36.468+0000 7f9df6671640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f9df00b3dd0 con 0x7f9df0162280 2024-09-21T03:43:36.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:36.468+0000 7f9df6671640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f9df00b39a0 con 0x7f9df00b2e40 2024-09-21T03:43:36.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:36.468+0000 7f9df6671640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f9df00b3f50 con 0x7f9df0161120 2024-09-21T03:43:36.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:36.468+0000 7f9def7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9df0162280 0x7f9df015a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:36.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:36.468+0000 7f9deffff640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9df00b2e40 0x7f9df0157940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:36.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:36.468+0000 7f9deffff640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9df00b2e40 0x7f9df0157940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.38:38394/0 (socket says 172.21.15.38:38394) 2024-09-21T03:43:36.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:36.468+0000 7f9deffff640 1 -- 172.21.15.38:0/3784729946 learned_addr learned my addr 172.21.15.38:0/3784729946 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:43:36.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:36.468+0000 7f9def7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9df0162280 0x7f9df015a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49736/0 (socket says 172.21.15.38:49736) 2024-09-21T03:43:36.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:36.468+0000 7f9df4c0e640 1 --2- 172.21.15.38:0/3784729946 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9df0161120 0x7f9df0161530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:36.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:36.468+0000 7f9def7fe640 1 -- 172.21.15.38:0/3784729946 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9df0161120 msgr2=0x7f9df0161530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:36.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:36.468+0000 7f9def7fe640 1 --2- 172.21.15.38:0/3784729946 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9df0161120 0x7f9df0161530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:36.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:36.468+0000 7f9def7fe640 1 -- 172.21.15.38:0/3784729946 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9df00b2e40 msgr2=0x7f9df0157940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:36.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:36.468+0000 7f9def7fe640 1 --2- 172.21.15.38:0/3784729946 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9df00b2e40 0x7f9df0157940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:36.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:36.468+0000 7f9def7fe640 1 -- 172.21.15.38:0/3784729946 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9df0154610 con 0x7f9df0162280 2024-09-21T03:43:36.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:36.468+0000 7f9deffff640 1 --2- 172.21.15.38:0/3784729946 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9df00b2e40 0x7f9df0157940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:43:36.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:36.469+0000 7f9def7fe640 1 --2- 172.21.15.38:0/3784729946 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9df0162280 0x7f9df015a560 secure :-1 s=READY pgs=2049 cs=0 l=1 rev1=1 crypto rx=0x7f9de400d5e0 tx=0x7f9de400bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:36.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:36.469+0000 7f9decff9640 1 -- 172.21.15.38:0/3784729946 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9de4017070 con 0x7f9df0162280 2024-09-21T03:43:36.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:36.469+0000 7f9decff9640 1 -- 172.21.15.38:0/3784729946 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9de4005310 con 0x7f9df0162280 2024-09-21T03:43:36.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:36.469+0000 7f9df6671640 1 -- 172.21.15.38:0/3784729946 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9df01548f0 con 0x7f9df0162280 2024-09-21T03:43:36.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:36.469+0000 7f9decff9640 1 -- 172.21.15.38:0/3784729946 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9de40047a0 con 0x7f9df0162280 2024-09-21T03:43:36.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:36.469+0000 7f9df6671640 1 -- 172.21.15.38:0/3784729946 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9df0154eb0 con 0x7f9df0162280 2024-09-21T03:43:36.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:36.470+0000 7f9df6671640 1 -- 172.21.15.38:0/3784729946 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9df0162f00 con 0x7f9df0162280 2024-09-21T03:43:36.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:36.471+0000 7f9decff9640 1 -- 172.21.15.38:0/3784729946 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f9de4004940 con 0x7f9df0162280 2024-09-21T03:43:36.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:36.471+0000 7f9decff9640 1 --2- 172.21.15.38:0/3784729946 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9ddc078bc0 0x7f9ddc07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:36.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:36.471+0000 7f9decff9640 1 -- 172.21.15.38:0/3784729946 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 9142+0+0 (secure 0 0 0) 0x7f9de409ba80 con 0x7f9df0162280 2024-09-21T03:43:36.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:36.471+0000 7f9deffff640 1 --2- 172.21.15.38:0/3784729946 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9ddc078bc0 0x7f9ddc07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:36.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:36.472+0000 7f9deffff640 1 --2- 172.21.15.38:0/3784729946 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9ddc078bc0 0x7f9ddc07b0d0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f9de0011440 tx=0x7f9de0005f90 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:36.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:36.476+0000 7f9decff9640 1 -- 172.21.15.38:0/3784729946 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f9de4062ab0 con 0x7f9df0162280 2024-09-21T03:43:36.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:36.701+0000 7f9df6671640 1 -- 172.21.15.38:0/3784729946 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9df015deb0 con 0x7f9df0162280 2024-09-21T03:43:36.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:36.702+0000 7f9decff9640 1 -- 172.21.15.38:0/3784729946 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 282 v282) ==== 95+0+4392 (secure 0 0 0) 0x7f9de40674e0 con 0x7f9df0162280 2024-09-21T03:43:36.705 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:43:36.705 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":282,"btime":"2024-09-21T03:43:33:596835+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6086,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/965972663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":965972663},{"type":"v1","addr":"172.21.15.195:6836","nonce":965972663}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6700,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/3176122334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":3176122334},{"type":"v1","addr":"172.21.15.195:6837","nonce":3176122334}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":9840,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/3801772046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":3801772046},{"type":"v1","addr":"172.21.15.38:6835","nonce":3801772046}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":273}],"filesystems":[{"mdsmap":{"epoch":282,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:40:02.319282+0000","modified":"2024-09-21T03:43:33.596831+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":129,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-21T03:43:36.705 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 282 2024-09-21T03:43:36.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:36.704+0000 7f9df6671640 1 -- 172.21.15.38:0/3784729946 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9ddc078bc0 msgr2=0x7f9ddc07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:36.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:36.704+0000 7f9df6671640 1 --2- 172.21.15.38:0/3784729946 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9ddc078bc0 0x7f9ddc07b0d0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f9de0011440 tx=0x7f9de0005f90 comp rx=0 tx=0).stop 2024-09-21T03:43:36.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:36.704+0000 7f9df6671640 1 -- 172.21.15.38:0/3784729946 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9df0162280 msgr2=0x7f9df015a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:36.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:36.704+0000 7f9df6671640 1 --2- 172.21.15.38:0/3784729946 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9df0162280 0x7f9df015a560 secure :-1 s=READY pgs=2049 cs=0 l=1 rev1=1 crypto rx=0x7f9de400d5e0 tx=0x7f9de400bf90 comp rx=0 tx=0).stop 2024-09-21T03:43:36.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:36.704+0000 7f9df6671640 1 -- 172.21.15.38:0/3784729946 shutdown_connections 2024-09-21T03:43:36.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:36.704+0000 7f9df6671640 1 --2- 172.21.15.38:0/3784729946 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9ddc078bc0 0x7f9ddc07b0d0 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:36.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:36.704+0000 7f9df6671640 1 --2- 172.21.15.38:0/3784729946 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9df0161120 0x7f9df0161530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:36.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:36.704+0000 7f9df6671640 1 --2- 172.21.15.38:0/3784729946 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9df0162280 0x7f9df015a560 unknown :-1 s=CLOSED pgs=2049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:36.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:36.704+0000 7f9df6671640 1 --2- 172.21.15.38:0/3784729946 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9df00b2e40 0x7f9df0157940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:36.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:36.704+0000 7f9df6671640 1 -- 172.21.15.38:0/3784729946 >> 172.21.15.38:0/3784729946 conn(0x7f9df00b6b20 msgr2=0x7f9df006cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:36.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:36.704+0000 7f9df6671640 1 -- 172.21.15.38:0/3784729946 shutdown_connections 2024-09-21T03:43:36.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:36.704+0000 7f9df6671640 1 -- 172.21.15.38:0/3784729946 wait complete. 2024-09-21T03:43:36.718 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-21T03:43:36.719 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:43:36.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:36.832+0000 7f9f34c63640 1 -- 172.21.15.38:0/2022183845 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9f300b2e40 msgr2=0x7f9f300b3250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:36.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:36.832+0000 7f9f34c63640 1 --2- 172.21.15.38:0/2022183845 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9f300b2e40 0x7f9f300b3250 secure :-1 s=READY pgs=2050 cs=0 l=1 rev1=1 crypto rx=0x7f9f300d5250 tx=0x7f9f2802aa30 comp rx=0 tx=0).stop 2024-09-21T03:43:36.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:36.832+0000 7f9f34c63640 1 -- 172.21.15.38:0/2022183845 shutdown_connections 2024-09-21T03:43:36.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:36.832+0000 7f9f34c63640 1 --2- 172.21.15.38:0/2022183845 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9f300b2e40 0x7f9f300b3250 unknown :-1 s=CLOSED pgs=2050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:36.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:36.832+0000 7f9f34c63640 1 --2- 172.21.15.38:0/2022183845 >> v2:172.21.15.195:3300/0 conn(0x7f9f30162280 0x7f9f30162690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:36.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:36.832+0000 7f9f34c63640 1 --2- 172.21.15.38:0/2022183845 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9f30162f00 0x7f9f300d24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:36.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:36.832+0000 7f9f34c63640 1 -- 172.21.15.38:0/2022183845 >> 172.21.15.38:0/2022183845 conn(0x7f9f300b6b20 msgr2=0x7f9f300d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:36.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:36.832+0000 7f9f34c63640 1 -- 172.21.15.38:0/2022183845 shutdown_connections 2024-09-21T03:43:36.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:36.832+0000 7f9f34c63640 1 -- 172.21.15.38:0/2022183845 wait complete. 2024-09-21T03:43:36.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:36.834+0000 7f9f34c63640 1 Processor -- start 2024-09-21T03:43:36.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:36.834+0000 7f9f34c63640 1 -- start start 2024-09-21T03:43:36.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:36.835+0000 7f9f34c63640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9f300b2e40 0x7f9f301f5e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:36.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:36.835+0000 7f9f34c63640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9f30162280 0x7f9f300ce370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:36.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:36.835+0000 7f9f34c63640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9f30159840 0x7f9f3015e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:36.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:36.835+0000 7f9f34c63640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f9f300b3dd0 con 0x7f9f30162280 2024-09-21T03:43:36.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:36.835+0000 7f9f34c63640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f9f300b39a0 con 0x7f9f30159840 2024-09-21T03:43:36.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:36.835+0000 7f9f34c63640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f9f300b3f50 con 0x7f9f300b2e40 2024-09-21T03:43:36.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:36.836+0000 7f9f2dd9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9f30162280 0x7f9f300ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:36.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:36.836+0000 7f9f2dd9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9f30162280 0x7f9f300ce370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49750/0 (socket says 172.21.15.38:49750) 2024-09-21T03:43:36.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:36.836+0000 7f9f2dd9b640 1 -- 172.21.15.38:0/2818563343 learned_addr learned my addr 172.21.15.38:0/2818563343 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:43:36.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:36.836+0000 7f9f2ed9d640 1 --2- 172.21.15.38:0/2818563343 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9f30159840 0x7f9f3015e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:36.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:36.836+0000 7f9f2e59c640 1 --2- 172.21.15.38:0/2818563343 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9f300b2e40 0x7f9f301f5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:36.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:36.836+0000 7f9f2dd9b640 1 -- 172.21.15.38:0/2818563343 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9f300b2e40 msgr2=0x7f9f301f5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:36.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:36.836+0000 7f9f2dd9b640 1 --2- 172.21.15.38:0/2818563343 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9f300b2e40 0x7f9f301f5e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:36.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:36.836+0000 7f9f2dd9b640 1 -- 172.21.15.38:0/2818563343 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9f30159840 msgr2=0x7f9f3015e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:36.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:36.836+0000 7f9f2dd9b640 1 --2- 172.21.15.38:0/2818563343 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9f30159840 0x7f9f3015e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:36.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:36.836+0000 7f9f2dd9b640 1 -- 172.21.15.38:0/2818563343 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f300cb150 con 0x7f9f30162280 2024-09-21T03:43:36.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:36.836+0000 7f9f2dd9b640 1 --2- 172.21.15.38:0/2818563343 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9f30162280 0x7f9f300ce370 secure :-1 s=READY pgs=2051 cs=0 l=1 rev1=1 crypto rx=0x7f9f2400d720 tx=0x7f9f2400dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:36.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:36.836+0000 7f9f1effd640 1 -- 172.21.15.38:0/2818563343 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f24017070 con 0x7f9f30162280 2024-09-21T03:43:36.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:36.836+0000 7f9f1effd640 1 -- 172.21.15.38:0/2818563343 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9f24009ab0 con 0x7f9f30162280 2024-09-21T03:43:36.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:36.836+0000 7f9f34c63640 1 -- 172.21.15.38:0/2818563343 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f300cd5b0 con 0x7f9f30162280 2024-09-21T03:43:36.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:36.836+0000 7f9f1effd640 1 -- 172.21.15.38:0/2818563343 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f24010460 con 0x7f9f30162280 2024-09-21T03:43:36.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:36.836+0000 7f9f34c63640 1 -- 172.21.15.38:0/2818563343 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f300cdb70 con 0x7f9f30162280 2024-09-21T03:43:36.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:36.837+0000 7f9f34c63640 1 -- 172.21.15.38:0/2818563343 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f30162f00 con 0x7f9f30162280 2024-09-21T03:43:36.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:36.838+0000 7f9f1effd640 1 -- 172.21.15.38:0/2818563343 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f9f2401a070 con 0x7f9f30162280 2024-09-21T03:43:36.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:36.839+0000 7f9f1effd640 1 --2- 172.21.15.38:0/2818563343 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9f10078b90 0x7f9f1007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:36.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:36.839+0000 7f9f1effd640 1 -- 172.21.15.38:0/2818563343 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 9142+0+0 (secure 0 0 0) 0x7f9f2409bbf0 con 0x7f9f30162280 2024-09-21T03:43:36.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:36.839+0000 7f9f2e59c640 1 --2- 172.21.15.38:0/2818563343 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9f10078b90 0x7f9f1007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:36.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:36.840+0000 7f9f2e59c640 1 --2- 172.21.15.38:0/2818563343 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9f10078b90 0x7f9f1007b0a0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f9f20004630 tx=0x7f9f2000b2e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:36.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:36.844+0000 7f9f1effd640 1 -- 172.21.15.38:0/2818563343 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f9f24062ba0 con 0x7f9f30162280 2024-09-21T03:43:37.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:37.068+0000 7f9f34c63640 1 -- 172.21.15.38:0/2818563343 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9f300b3570 con 0x7f9f30162280 2024-09-21T03:43:37.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:37.069+0000 7f9f1effd640 1 -- 172.21.15.38:0/2818563343 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 282 v282) ==== 95+0+4392 (secure 0 0 0) 0x7f9f240675d0 con 0x7f9f30162280 2024-09-21T03:43:37.072 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:43:37.072 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":282,"btime":"2024-09-21T03:43:33:596835+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6086,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/965972663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":965972663},{"type":"v1","addr":"172.21.15.195:6836","nonce":965972663}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6700,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/3176122334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":3176122334},{"type":"v1","addr":"172.21.15.195:6837","nonce":3176122334}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":9840,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/3801772046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":3801772046},{"type":"v1","addr":"172.21.15.38:6835","nonce":3801772046}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":273}],"filesystems":[{"mdsmap":{"epoch":282,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:40:02.319282+0000","modified":"2024-09-21T03:43:33.596831+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":129,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-21T03:43:37.072 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 282 2024-09-21T03:43:37.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:37.071+0000 7f9f34c63640 1 -- 172.21.15.38:0/2818563343 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9f10078b90 msgr2=0x7f9f1007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:37.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:37.071+0000 7f9f34c63640 1 --2- 172.21.15.38:0/2818563343 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9f10078b90 0x7f9f1007b0a0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f9f20004630 tx=0x7f9f2000b2e0 comp rx=0 tx=0).stop 2024-09-21T03:43:37.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:37.071+0000 7f9f34c63640 1 -- 172.21.15.38:0/2818563343 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9f30162280 msgr2=0x7f9f300ce370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:37.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:37.071+0000 7f9f34c63640 1 --2- 172.21.15.38:0/2818563343 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9f30162280 0x7f9f300ce370 secure :-1 s=READY pgs=2051 cs=0 l=1 rev1=1 crypto rx=0x7f9f2400d720 tx=0x7f9f2400dbe0 comp rx=0 tx=0).stop 2024-09-21T03:43:37.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:37.071+0000 7f9f34c63640 1 -- 172.21.15.38:0/2818563343 shutdown_connections 2024-09-21T03:43:37.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:37.071+0000 7f9f34c63640 1 --2- 172.21.15.38:0/2818563343 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9f30159840 0x7f9f3015e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:37.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:37.071+0000 7f9f34c63640 1 --2- 172.21.15.38:0/2818563343 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9f30162280 0x7f9f300ce370 unknown :-1 s=CLOSED pgs=2051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:37.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:37.071+0000 7f9f34c63640 1 --2- 172.21.15.38:0/2818563343 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9f10078b90 0x7f9f1007b0a0 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:37.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:37.071+0000 7f9f34c63640 1 --2- 172.21.15.38:0/2818563343 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9f300b2e40 0x7f9f301f5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:37.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:37.071+0000 7f9f34c63640 1 -- 172.21.15.38:0/2818563343 >> 172.21.15.38:0/2818563343 conn(0x7f9f300b6b20 msgr2=0x7f9f3015c880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:37.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:37.071+0000 7f9f34c63640 1 -- 172.21.15.38:0/2818563343 shutdown_connections 2024-09-21T03:43:37.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:37.071+0000 7f9f34c63640 1 -- 172.21.15.38:0/2818563343 wait complete. 2024-09-21T03:43:37.086 INFO:tasks.cephfs.test_damage:Result: Mutation 'Corrupt omap key 100.00000000:stray0_head' led to DAMAGED state 2024-09-21T03:43:37.086 INFO:tasks.cephfs.test_damage:Applying mutation 'Corrupt omap key 1.00000000:subdir_head' 2024-09-21T03:43:37.086 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-21T03:43:37.086 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:43:37.102 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056795,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2354 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T03:43:37.102 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi038.front.sepia.ceph.com 2024-09-21T03:43:37.102 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:43:37.102 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T03:43:37.158 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T03:43:37.158 INFO:teuthology.orchestra.run.smithi038.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-09-21T03:43:37.159 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-21T03:43:37.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:37.312+0000 7f1897c1f640 1 -- 172.21.15.38:0/1476436363 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f189006def0 msgr2=0x7f189006f410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:37.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:37.312+0000 7f1897c1f640 1 --2- 172.21.15.38:0/1476436363 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f189006def0 0x7f189006f410 secure :-1 s=READY pgs=2053 cs=0 l=1 rev1=1 crypto rx=0x7f188c009830 tx=0x7f188c02a510 comp rx=0 tx=0).stop 2024-09-21T03:43:37.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:37.313+0000 7f1897c1f640 1 -- 172.21.15.38:0/1476436363 shutdown_connections 2024-09-21T03:43:37.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:37.313+0000 7f1897c1f640 1 --2- 172.21.15.38:0/1476436363 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f189006def0 0x7f189006f410 unknown :-1 s=CLOSED pgs=2053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:37.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:37.313+0000 7f1897c1f640 1 -- 172.21.15.38:0/1476436363 >> 172.21.15.38:0/1476436363 conn(0x7f18900cdca0 msgr2=0x7f18900ce0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:37.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:37.313+0000 7f1897c1f640 1 -- 172.21.15.38:0/1476436363 shutdown_connections 2024-09-21T03:43:37.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:37.313+0000 7f1897c1f640 1 -- 172.21.15.38:0/1476436363 wait complete. 2024-09-21T03:43:37.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:37.314+0000 7f1897c1f640 1 Processor -- start 2024-09-21T03:43:37.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:37.314+0000 7f1897c1f640 1 -- start start 2024-09-21T03:43:37.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:37.316+0000 7f1897c1f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f189006def0 0x7f18901f2500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:37.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:37.316+0000 7f1897c1f640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f18900b5260 0x7f18901532a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:37.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:37.316+0000 7f1897c1f640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f18900b5670 0x7f18900ccd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:37.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:37.316+0000 7f1897c1f640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f1890071480 con 0x7f189006def0 2024-09-21T03:43:37.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:37.316+0000 7f1897c1f640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f1890071300 con 0x7f18900b5670 2024-09-21T03:43:37.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:37.316+0000 7f1897c1f640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f1890071600 con 0x7f18900b5260 2024-09-21T03:43:37.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:37.316+0000 7f18959bb640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f189006def0 0x7f18901f2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:37.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:37.316+0000 7f18959bb640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f189006def0 0x7f18901f2500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49776/0 (socket says 172.21.15.38:49776) 2024-09-21T03:43:37.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:37.316+0000 7f18959bb640 1 -- 172.21.15.38:0/1871831227 learned_addr learned my addr 172.21.15.38:0/1871831227 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:43:37.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:37.316+0000 7f18961bc640 1 --2- 172.21.15.38:0/1871831227 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f18900b5670 0x7f18900ccd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:37.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:37.316+0000 7f18951ba640 1 --2- 172.21.15.38:0/1871831227 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f18900b5260 0x7f18901532a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:37.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:37.316+0000 7f18959bb640 1 -- 172.21.15.38:0/1871831227 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f18900b5260 msgr2=0x7f18901532a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:37.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:37.316+0000 7f18959bb640 1 --2- 172.21.15.38:0/1871831227 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f18900b5260 0x7f18901532a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:37.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:37.316+0000 7f18959bb640 1 -- 172.21.15.38:0/1871831227 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f18900b5670 msgr2=0x7f18900ccd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:37.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:37.316+0000 7f18959bb640 1 --2- 172.21.15.38:0/1871831227 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f18900b5670 0x7f18900ccd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:37.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:37.316+0000 7f18959bb640 1 -- 172.21.15.38:0/1871831227 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18901f5af0 con 0x7f189006def0 2024-09-21T03:43:37.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:37.316+0000 7f18951ba640 1 --2- 172.21.15.38:0/1871831227 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f18900b5260 0x7f18901532a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:43:37.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:37.316+0000 7f18959bb640 1 --2- 172.21.15.38:0/1871831227 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f189006def0 0x7f18901f2500 secure :-1 s=READY pgs=2054 cs=0 l=1 rev1=1 crypto rx=0x7f188c009950 tx=0x7f188c031690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:37.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:37.317+0000 7f18867fc640 1 -- 172.21.15.38:0/1871831227 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f188c003b00 con 0x7f189006def0 2024-09-21T03:43:37.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:37.317+0000 7f18867fc640 1 -- 172.21.15.38:0/1871831227 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f188c003d90 con 0x7f189006def0 2024-09-21T03:43:37.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:37.317+0000 7f1897c1f640 1 -- 172.21.15.38:0/1871831227 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1890074090 con 0x7f189006def0 2024-09-21T03:43:37.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:37.317+0000 7f18867fc640 1 -- 172.21.15.38:0/1871831227 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f188c005280 con 0x7f189006def0 2024-09-21T03:43:37.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:37.317+0000 7f1897c1f640 1 -- 172.21.15.38:0/1871831227 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1890074650 con 0x7f189006def0 2024-09-21T03:43:37.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:37.318+0000 7f1897c1f640 1 -- 172.21.15.38:0/1871831227 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1890071af0 con 0x7f189006def0 2024-09-21T03:43:37.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:37.319+0000 7f18867fc640 1 -- 172.21.15.38:0/1871831227 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f188c005420 con 0x7f189006def0 2024-09-21T03:43:37.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:37.319+0000 7f18867fc640 1 --2- 172.21.15.38:0/1871831227 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1874078b90 0x7f187407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:37.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:37.319+0000 7f18867fc640 1 -- 172.21.15.38:0/1871831227 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 9142+0+0 (secure 0 0 0) 0x7f188c0bf6d0 con 0x7f189006def0 2024-09-21T03:43:37.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:37.320+0000 7f18951ba640 1 --2- 172.21.15.38:0/1871831227 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1874078b90 0x7f187407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:37.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:37.320+0000 7f18951ba640 1 --2- 172.21.15.38:0/1871831227 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1874078b90 0x7f187407b0a0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f18901f2a40 tx=0x7f1880009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:37.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:37.324+0000 7f18867fc640 1 -- 172.21.15.38:0/1871831227 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f188c041030 con 0x7f189006def0 2024-09-21T03:43:37.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:37.548+0000 7f1897c1f640 1 -- 172.21.15.38:0/1871831227 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f189006a750 con 0x7f189006def0 2024-09-21T03:43:37.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:37.613+0000 7f18867fc640 1 -- 172.21.15.38:0/1871831227 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v283) ==== 161+0+0 (secure 0 0 0) 0x7f188c086cd0 con 0x7f189006def0 2024-09-21T03:43:37.615 INFO:teuthology.orchestra.run.smithi038.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-21T03:43:37.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:37.614+0000 7f1897c1f640 1 -- 172.21.15.38:0/1871831227 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1874078b90 msgr2=0x7f187407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:37.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:37.614+0000 7f1897c1f640 1 --2- 172.21.15.38:0/1871831227 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1874078b90 0x7f187407b0a0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f18901f2a40 tx=0x7f1880009040 comp rx=0 tx=0).stop 2024-09-21T03:43:37.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:37.614+0000 7f1897c1f640 1 -- 172.21.15.38:0/1871831227 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f189006def0 msgr2=0x7f18901f2500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:37.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:37.614+0000 7f1897c1f640 1 --2- 172.21.15.38:0/1871831227 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f189006def0 0x7f18901f2500 secure :-1 s=READY pgs=2054 cs=0 l=1 rev1=1 crypto rx=0x7f188c009950 tx=0x7f188c031690 comp rx=0 tx=0).stop 2024-09-21T03:43:37.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:37.614+0000 7f1897c1f640 1 -- 172.21.15.38:0/1871831227 shutdown_connections 2024-09-21T03:43:37.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:37.614+0000 7f1897c1f640 1 --2- 172.21.15.38:0/1871831227 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f18900b5670 0x7f18900ccd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:37.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:37.614+0000 7f1897c1f640 1 --2- 172.21.15.38:0/1871831227 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1874078b90 0x7f187407b0a0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:37.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:37.614+0000 7f1897c1f640 1 --2- 172.21.15.38:0/1871831227 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f18900b5260 0x7f18901532a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:37.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:37.614+0000 7f1897c1f640 1 --2- 172.21.15.38:0/1871831227 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f189006def0 0x7f18901f2500 unknown :-1 s=CLOSED pgs=2054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:37.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:37.614+0000 7f1897c1f640 1 -- 172.21.15.38:0/1871831227 >> 172.21.15.38:0/1871831227 conn(0x7f18900cdca0 msgr2=0x7f189015d450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:37.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:37.615+0000 7f1897c1f640 1 -- 172.21.15.38:0/1871831227 shutdown_connections 2024-09-21T03:43:37.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:37.615+0000 7f1897c1f640 1 -- 172.21.15.38:0/1871831227 wait complete. 2024-09-21T03:43:37.629 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-09-21T03:43:37.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:37.743+0000 7fbf33238640 1 -- 172.21.15.38:0/308923797 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbf2c0d0170 msgr2=0x7fbf2c0d0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:37.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:37.743+0000 7fbf33238640 1 --2- 172.21.15.38:0/308923797 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbf2c0d0170 0x7fbf2c0d0be0 secure :-1 s=READY pgs=2056 cs=0 l=1 rev1=1 crypto rx=0x7fbf1c009820 tx=0x7fbf1c02a480 comp rx=0 tx=0).stop 2024-09-21T03:43:37.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:37.743+0000 7fbf33238640 1 -- 172.21.15.38:0/308923797 shutdown_connections 2024-09-21T03:43:37.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:37.743+0000 7fbf33238640 1 --2- 172.21.15.38:0/308923797 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbf2c0d1120 0x7fbf2c0ceb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:37.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:37.743+0000 7fbf33238640 1 --2- 172.21.15.38:0/308923797 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbf2c0d0170 0x7fbf2c0d0be0 unknown :-1 s=CLOSED pgs=2056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:37.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:37.743+0000 7fbf33238640 1 -- 172.21.15.38:0/308923797 >> 172.21.15.38:0/308923797 conn(0x7fbf2c09d830 msgr2=0x7fbf2c0b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:37.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:37.743+0000 7fbf33238640 1 -- 172.21.15.38:0/308923797 shutdown_connections 2024-09-21T03:43:37.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:37.743+0000 7fbf33238640 1 -- 172.21.15.38:0/308923797 wait complete. 2024-09-21T03:43:37.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:37.745+0000 7fbf33238640 1 Processor -- start 2024-09-21T03:43:37.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:37.745+0000 7fbf33238640 1 -- start start 2024-09-21T03:43:37.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:37.746+0000 7fbf33238640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbf2c0d0170 0x7fbf2c06f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:37.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:37.746+0000 7fbf33238640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbf2c0d1120 0x7fbf2c158a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:37.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:37.746+0000 7fbf33238640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbf2c15ee80 0x7fbf2c0cd380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:37.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:37.746+0000 7fbf33238640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fbf2c0b6700 con 0x7fbf2c0d0170 2024-09-21T03:43:37.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:37.746+0000 7fbf33238640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fbf2c0b6580 con 0x7fbf2c0d1120 2024-09-21T03:43:37.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:37.746+0000 7fbf33238640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fbf2c0b6880 con 0x7fbf2c15ee80 2024-09-21T03:43:37.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:37.747+0000 7fbf30fd4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbf2c0d0170 0x7fbf2c06f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:37.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:37.747+0000 7fbf30fd4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbf2c0d0170 0x7fbf2c06f570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49798/0 (socket says 172.21.15.38:49798) 2024-09-21T03:43:37.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:37.747+0000 7fbf30fd4640 1 -- 172.21.15.38:0/4236112685 learned_addr learned my addr 172.21.15.38:0/4236112685 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:43:37.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:37.747+0000 7fbf2bfff640 1 --2- 172.21.15.38:0/4236112685 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbf2c0d1120 0x7fbf2c158a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:37.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:37.747+0000 7fbf317d5640 1 --2- 172.21.15.38:0/4236112685 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbf2c15ee80 0x7fbf2c0cd380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:37.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:37.747+0000 7fbf30fd4640 1 -- 172.21.15.38:0/4236112685 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbf2c15ee80 msgr2=0x7fbf2c0cd380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:37.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:37.747+0000 7fbf30fd4640 1 --2- 172.21.15.38:0/4236112685 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbf2c15ee80 0x7fbf2c0cd380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:37.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:37.747+0000 7fbf30fd4640 1 -- 172.21.15.38:0/4236112685 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbf2c0d1120 msgr2=0x7fbf2c158a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:37.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:37.747+0000 7fbf30fd4640 1 --2- 172.21.15.38:0/4236112685 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbf2c0d1120 0x7fbf2c158a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:37.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:37.747+0000 7fbf30fd4640 1 -- 172.21.15.38:0/4236112685 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf2c0cb610 con 0x7fbf2c0d0170 2024-09-21T03:43:37.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:37.747+0000 7fbf2bfff640 1 --2- 172.21.15.38:0/4236112685 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbf2c0d1120 0x7fbf2c158a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:43:37.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:37.747+0000 7fbf30fd4640 1 --2- 172.21.15.38:0/4236112685 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbf2c0d0170 0x7fbf2c06f570 secure :-1 s=READY pgs=2057 cs=0 l=1 rev1=1 crypto rx=0x7fbf1400c400 tx=0x7fbf1400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:37.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:37.747+0000 7fbf297fa640 1 -- 172.21.15.38:0/4236112685 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf14011790 con 0x7fbf2c0d0170 2024-09-21T03:43:37.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:37.747+0000 7fbf297fa640 1 -- 172.21.15.38:0/4236112685 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbf14017070 con 0x7fbf2c0d0170 2024-09-21T03:43:37.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:37.747+0000 7fbf33238640 1 -- 172.21.15.38:0/4236112685 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf2c0cb8d0 con 0x7fbf2c0d0170 2024-09-21T03:43:37.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:37.747+0000 7fbf297fa640 1 -- 172.21.15.38:0/4236112685 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf14011cc0 con 0x7fbf2c0d0170 2024-09-21T03:43:37.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:37.747+0000 7fbf33238640 1 -- 172.21.15.38:0/4236112685 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf2c0cbe90 con 0x7fbf2c0d0170 2024-09-21T03:43:37.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:37.748+0000 7fbf33238640 1 -- 172.21.15.38:0/4236112685 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf2c0b9d80 con 0x7fbf2c0d0170 2024-09-21T03:43:37.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:37.749+0000 7fbf297fa640 1 -- 172.21.15.38:0/4236112685 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fbf1401c460 con 0x7fbf2c0d0170 2024-09-21T03:43:37.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:37.750+0000 7fbf297fa640 1 --2- 172.21.15.38:0/4236112685 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fbf0c078b90 0x7fbf0c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:37.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:37.750+0000 7fbf297fa640 1 -- 172.21.15.38:0/4236112685 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 9142+0+0 (secure 0 0 0) 0x7fbf1409b990 con 0x7fbf2c0d0170 2024-09-21T03:43:37.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:37.750+0000 7fbf2bfff640 1 --2- 172.21.15.38:0/4236112685 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fbf0c078b90 0x7fbf0c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:37.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:37.751+0000 7fbf2bfff640 1 --2- 172.21.15.38:0/4236112685 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fbf0c078b90 0x7fbf0c07b0a0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fbf1c034000 tx=0x7fbf1c004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:37.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:37.754+0000 7fbf297fa640 1 -- 172.21.15.38:0/4236112685 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fbf2c0b9d80 con 0x7fbf2c0d0170 2024-09-21T03:43:38.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.017+0000 7fbf33238640 1 -- 172.21.15.38:0/4236112685 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7fbf2c0b6da0 con 0x7fbf2c0d0170 2024-09-21T03:43:38.346 INFO:tasks.ceph.mds.c.smithi038.stderr: -53> 2024-09-21T03:43:33.592+0000 7fc65302b640 -1 mds.0.damage notify_dentry Damage to dentries in fragment * of ino 0x100is fatal because it is a system directory for this rank 2024-09-21T03:43:38.346 INFO:tasks.ceph.mds.c.smithi038.stderr: -36> 2024-09-21T03:43:34.211+0000 7fc657033640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T03:43:38.403 INFO:tasks.ceph.mds.c.smithi038.stdout:starting mds.c at 2024-09-21T03:43:38.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.614+0000 7fbf297fa640 1 -- 172.21.15.38:0/4236112685 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v284) ==== 101+0+0 (secure 0 0 0) 0x7fbf2c1592d0 con 0x7fbf2c0d0170 2024-09-21T03:43:38.618 INFO:teuthology.orchestra.run.smithi038.stderr:repaired: restoring rank 6:0 2024-09-21T03:43:38.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.617+0000 7fbf33238640 1 -- 172.21.15.38:0/4236112685 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fbf0c078b90 msgr2=0x7fbf0c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:38.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.617+0000 7fbf33238640 1 --2- 172.21.15.38:0/4236112685 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fbf0c078b90 0x7fbf0c07b0a0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fbf1c034000 tx=0x7fbf1c004730 comp rx=0 tx=0).stop 2024-09-21T03:43:38.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.617+0000 7fbf33238640 1 -- 172.21.15.38:0/4236112685 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbf2c0d0170 msgr2=0x7fbf2c06f570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:38.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.617+0000 7fbf33238640 1 --2- 172.21.15.38:0/4236112685 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbf2c0d0170 0x7fbf2c06f570 secure :-1 s=READY pgs=2057 cs=0 l=1 rev1=1 crypto rx=0x7fbf1400c400 tx=0x7fbf1400c8c0 comp rx=0 tx=0).stop 2024-09-21T03:43:38.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.617+0000 7fbf33238640 1 -- 172.21.15.38:0/4236112685 shutdown_connections 2024-09-21T03:43:38.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.617+0000 7fbf33238640 1 --2- 172.21.15.38:0/4236112685 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fbf0c078b90 0x7fbf0c07b0a0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:38.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.617+0000 7fbf33238640 1 --2- 172.21.15.38:0/4236112685 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbf2c15ee80 0x7fbf2c0cd380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:38.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.617+0000 7fbf33238640 1 --2- 172.21.15.38:0/4236112685 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbf2c0d1120 0x7fbf2c158a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:38.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.617+0000 7fbf33238640 1 --2- 172.21.15.38:0/4236112685 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbf2c0d0170 0x7fbf2c06f570 unknown :-1 s=CLOSED pgs=2057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:38.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.617+0000 7fbf33238640 1 -- 172.21.15.38:0/4236112685 >> 172.21.15.38:0/4236112685 conn(0x7fbf2c09d830 msgr2=0x7fbf2c0b31a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:38.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.617+0000 7fbf33238640 1 -- 172.21.15.38:0/4236112685 shutdown_connections 2024-09-21T03:43:38.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.617+0000 7fbf33238640 1 -- 172.21.15.38:0/4236112685 wait complete. 2024-09-21T03:43:38.633 DEBUG:teuthology.orchestra.run.smithi038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-09-21T03:43:38.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.686+0000 7f43f961af40 1 Processor -- start 2024-09-21T03:43:38.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.686+0000 7f43f961af40 1 -- start start 2024-09-21T03:43:38.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.688+0000 7f43f961af40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55822d74bd00 0x55822d8024a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:38.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.688+0000 7f43f961af40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55822d777e30 0x55822d821350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:38.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.688+0000 7f43f961af40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55822d806950 0x55822d821890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:38.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.688+0000 7f43f961af40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55822d699d60 con 0x55822d74bd00 2024-09-21T03:43:38.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.688+0000 7f43f961af40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55822d6f8430 con 0x55822d777e30 2024-09-21T03:43:38.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.688+0000 7f43f961af40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55822d793ef0 con 0x55822d806950 2024-09-21T03:43:38.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.688+0000 7f43f2467640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55822d74bd00 0x55822d8024a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:38.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.688+0000 7f43f2467640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55822d74bd00 0x55822d8024a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49830/0 (socket says 172.21.15.38:49830) 2024-09-21T03:43:38.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.688+0000 7f43f2467640 1 -- 172.21.15.38:0/3582529264 learned_addr learned my addr 172.21.15.38:0/3582529264 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:43:38.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.688+0000 7f43f1c66640 1 --2- 172.21.15.38:0/3582529264 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55822d777e30 0x55822d821350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:38.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.688+0000 7f43f2c68640 1 --2- 172.21.15.38:0/3582529264 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55822d806950 0x55822d821890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:38.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.688+0000 7f43f2467640 1 -- 172.21.15.38:0/3582529264 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55822d806950 msgr2=0x55822d821890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:38.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.688+0000 7f43f2467640 1 --2- 172.21.15.38:0/3582529264 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55822d806950 0x55822d821890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:38.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.688+0000 7f43f2467640 1 -- 172.21.15.38:0/3582529264 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55822d777e30 msgr2=0x55822d821350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:38.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.688+0000 7f43f2467640 1 --2- 172.21.15.38:0/3582529264 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55822d777e30 0x55822d821350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:38.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.688+0000 7f43f2467640 1 -- 172.21.15.38:0/3582529264 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55822d805d40 con 0x55822d74bd00 2024-09-21T03:43:38.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.688+0000 7f43f1c66640 1 --2- 172.21.15.38:0/3582529264 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55822d777e30 0x55822d821350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:43:38.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.688+0000 7f43f2467640 1 --2- 172.21.15.38:0/3582529264 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55822d74bd00 0x55822d8024a0 secure :-1 s=READY pgs=2063 cs=0 l=1 rev1=1 crypto rx=0x7f43e4001e50 tx=0x7f43e400b730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ead419f86a9d6e58 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:38.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.688+0000 7f43f0c64640 1 -- 172.21.15.38:0/3582529264 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43e4012070 con 0x55822d74bd00 2024-09-21T03:43:38.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.688+0000 7f43f0c64640 1 -- 172.21.15.38:0/3582529264 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f43e4006050 con 0x55822d74bd00 2024-09-21T03:43:38.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.688+0000 7f43f0c64640 1 -- 172.21.15.38:0/3582529264 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43e400ebc0 con 0x55822d74bd00 2024-09-21T03:43:38.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.689+0000 7f43f961af40 1 -- 172.21.15.38:0/3582529264 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55822d74bd00 msgr2=0x55822d8024a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:38.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.689+0000 7f43f961af40 1 --2- 172.21.15.38:0/3582529264 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55822d74bd00 0x55822d8024a0 secure :-1 s=READY pgs=2063 cs=0 l=1 rev1=1 crypto rx=0x7f43e4001e50 tx=0x7f43e400b730 comp rx=0 tx=0).stop 2024-09-21T03:43:38.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.689+0000 7f43f961af40 1 -- 172.21.15.38:0/3582529264 shutdown_connections 2024-09-21T03:43:38.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.689+0000 7f43f961af40 1 --2- 172.21.15.38:0/3582529264 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55822d806950 0x55822d821890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:38.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.689+0000 7f43f961af40 1 --2- 172.21.15.38:0/3582529264 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55822d777e30 0x55822d821350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:38.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.689+0000 7f43f961af40 1 --2- 172.21.15.38:0/3582529264 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55822d74bd00 0x55822d8024a0 unknown :-1 s=CLOSED pgs=2063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:38.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.689+0000 7f43f961af40 1 -- 172.21.15.38:0/3582529264 >> 172.21.15.38:0/3582529264 conn(0x55822d66aac0 msgr2=0x55822d81d7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:38.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.689+0000 7f43f961af40 1 -- 172.21.15.38:0/3582529264 shutdown_connections 2024-09-21T03:43:38.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.689+0000 7f43f961af40 1 -- 172.21.15.38:0/3582529264 wait complete. 2024-09-21T03:43:38.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.690+0000 7f43f961af40 1 Processor -- start 2024-09-21T03:43:38.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.690+0000 7f43f961af40 1 -- start start 2024-09-21T03:43:38.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.691+0000 7f43f961af40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55822d74bd00 0x55822d7fe960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:38.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.691+0000 7f43f961af40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55822d777e30 0x55822d7ffda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:38.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.691+0000 7f43f961af40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55822d806950 0x55822d81e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:38.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.691+0000 7f43f961af40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55822d7787a0 con 0x55822d74bd00 2024-09-21T03:43:38.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.691+0000 7f43f961af40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55822d805fe0 con 0x55822d777e30 2024-09-21T03:43:38.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.691+0000 7f43f961af40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55822d778a00 con 0x55822d806950 2024-09-21T03:43:38.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.691+0000 7f43f2467640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55822d74bd00 0x55822d7fe960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:38.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.691+0000 7f43f2467640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55822d74bd00 0x55822d7fe960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49834/0 (socket says 172.21.15.38:49834) 2024-09-21T03:43:38.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.691+0000 7f43f2467640 1 -- 172.21.15.38:0/995950130 learned_addr learned my addr 172.21.15.38:0/995950130 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:43:38.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.692+0000 7f43f1c66640 1 --2- 172.21.15.38:0/995950130 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55822d777e30 0x55822d7ffda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:38.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.692+0000 7f43f2c68640 1 --2- 172.21.15.38:0/995950130 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55822d806950 0x55822d81e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:38.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.692+0000 7f43f2467640 1 -- 172.21.15.38:0/995950130 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55822d806950 msgr2=0x55822d81e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:38.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.692+0000 7f43f2467640 1 --2- 172.21.15.38:0/995950130 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55822d806950 0x55822d81e0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:38.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.692+0000 7f43f2467640 1 -- 172.21.15.38:0/995950130 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55822d777e30 msgr2=0x55822d7ffda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:38.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.692+0000 7f43f2467640 1 --2- 172.21.15.38:0/995950130 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55822d777e30 0x55822d7ffda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:38.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.692+0000 7f43f2467640 1 -- 172.21.15.38:0/995950130 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55822d81c070 con 0x55822d74bd00 2024-09-21T03:43:38.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.692+0000 7f43f2467640 1 --2- 172.21.15.38:0/995950130 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55822d74bd00 0x55822d7fe960 secure :-1 s=READY pgs=2064 cs=0 l=1 rev1=1 crypto rx=0x7f43e4010b70 tx=0x7f43e4010ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:38.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.692+0000 7f43e37fe640 1 -- 172.21.15.38:0/995950130 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43e4025070 con 0x55822d74bd00 2024-09-21T03:43:38.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.692+0000 7f43e37fe640 1 -- 172.21.15.38:0/995950130 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f43e400e920 con 0x55822d74bd00 2024-09-21T03:43:38.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.692+0000 7f43f961af40 1 -- 172.21.15.38:0/995950130 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55822d81c4e0 con 0x55822d74bd00 2024-09-21T03:43:38.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.692+0000 7f43e37fe640 1 -- 172.21.15.38:0/995950130 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43e4012040 con 0x55822d74bd00 2024-09-21T03:43:38.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.692+0000 7f43f961af40 1 -- 172.21.15.38:0/995950130 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55822d7ff5a0 con 0x55822d74bd00 2024-09-21T03:43:38.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.694+0000 7f43e37fe640 1 -- 172.21.15.38:0/995950130 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f43e4015070 con 0x55822d74bd00 2024-09-21T03:43:38.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.694+0000 7f43e37fe640 1 --2- 172.21.15.38:0/995950130 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f43d4077570 0x7f43d4079a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:38.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.695+0000 7f43e37fe640 1 -- 172.21.15.38:0/995950130 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 9142+0+0 (secure 0 0 0) 0x7f43e409b430 con 0x55822d74bd00 2024-09-21T03:43:38.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.695+0000 7f43f1c66640 1 --2- 172.21.15.38:0/995950130 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f43d4077570 0x7f43d4079a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:38.699 INFO:teuthology.orchestra.run.smithi038.stdout:Importing pool 2024-09-21T03:43:38.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.695+0000 7f43f961af40 1 --2- 172.21.15.38:0/995950130 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x55822d8014f0 0x55822d801900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:38.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.695+0000 7f43f961af40 1 -- 172.21.15.38:0/995950130 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55822d81d6a0 con 0x55822d8014f0 2024-09-21T03:43:38.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.695+0000 7f43f2c68640 1 --2- 172.21.15.38:0/995950130 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x55822d8014f0 0x55822d801900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:38.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.695+0000 7f43f1c66640 1 --2- 172.21.15.38:0/995950130 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f43d4077570 0x7f43d4079a80 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f43e8055000 tx=0x7f43e8058040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:38.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.696+0000 7f43f2c68640 1 --2- 172.21.15.38:0/995950130 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x55822d8014f0 0x55822d801900 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:38.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.698+0000 7f43f2c68640 1 -- 172.21.15.38:0/995950130 <== osd.7 v2:172.21.15.195:6818/1606106342 1 ==== osd_op_reply(1 601.00000000 [create] v129'1002 uv976 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f43ec00b040 con 0x55822d8014f0 2024-09-21T03:43:38.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.698+0000 7f43f961af40 1 -- 172.21.15.38:0/995950130 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e129) -- 0x55822d8c5550 con 0x55822d8014f0 2024-09-21T03:43:38.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.700+0000 7f43f2c68640 1 -- 172.21.15.38:0/995950130 <== osd.7 v2:172.21.15.195:6818/1606106342 2 ==== osd_op_reply(2 601.00000000 [delete] v129'1003 uv1003 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f43ec00b040 con 0x55822d8014f0 2024-09-21T03:43:38.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.700+0000 7f43f961af40 1 -- 172.21.15.38:0/995950130 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55822d8c58f0 con 0x55822d8014f0 2024-09-21T03:43:38.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.701+0000 7f43f2c68640 1 -- 172.21.15.38:0/995950130 <== osd.7 v2:172.21.15.195:6818/1606106342 3 ==== osd_op_reply(3 601.00000000 [create] v129'1004 uv1004 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f43ec00b040 con 0x55822d8014f0 2024-09-21T03:43:38.704 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-09-21T03:43:38.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.701+0000 7f43f961af40 1 -- 172.21.15.38:0/995950130 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55822d8c6ca0 con 0x55822d8014f0 2024-09-21T03:43:38.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.703+0000 7f43f2c68640 1 -- 172.21.15.38:0/995950130 <== osd.7 v2:172.21.15.195:6818/1606106342 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v129'1005 uv1005 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f43ec00b040 con 0x55822d8014f0 2024-09-21T03:43:38.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.703+0000 7f43f961af40 1 -- 172.21.15.38:0/995950130 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55822d8c7040 con 0x55822d8014f0 2024-09-21T03:43:38.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.705+0000 7f43f2c68640 1 -- 172.21.15.38:0/995950130 <== osd.7 v2:172.21.15.195:6818/1606106342 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v129'1006 uv1006 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f43ec00b040 con 0x55822d8014f0 2024-09-21T03:43:38.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.705+0000 7f43f961af40 1 -- 172.21.15.38:0/995950130 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55822d8c5c90 con 0x55822d8014f0 2024-09-21T03:43:38.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.706+0000 7f43f2c68640 1 -- 172.21.15.38:0/995950130 <== osd.7 v2:172.21.15.195:6818/1606106342 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v129'1007 uv1007 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f43ec00b040 con 0x55822d8014f0 2024-09-21T03:43:38.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.706+0000 7f43f961af40 1 -- 172.21.15.38:0/995950130 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55822d8c6080 con 0x55822d8014f0 2024-09-21T03:43:38.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.709+0000 7f43f2c68640 1 -- 172.21.15.38:0/995950130 <== osd.7 v2:172.21.15.195:6818/1606106342 7 ==== osd_op_reply(7 602.00000000 [create] v129'1008 uv982 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f43ec00b040 con 0x55822d8014f0 2024-09-21T03:43:38.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.709+0000 7f43f961af40 1 -- 172.21.15.38:0/995950130 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e129) -- 0x55822d8c6470 con 0x55822d8014f0 2024-09-21T03:43:38.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.711+0000 7f43f2c68640 1 -- 172.21.15.38:0/995950130 <== osd.7 v2:172.21.15.195:6818/1606106342 8 ==== osd_op_reply(8 602.00000000 [delete] v129'1009 uv1009 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f43ec00b040 con 0x55822d8014f0 2024-09-21T03:43:38.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.711+0000 7f43f961af40 1 -- 172.21.15.38:0/995950130 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55822d8c6470 con 0x55822d8014f0 2024-09-21T03:43:38.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.713+0000 7f43f2c68640 1 -- 172.21.15.38:0/995950130 <== osd.7 v2:172.21.15.195:6818/1606106342 9 ==== osd_op_reply(9 602.00000000 [create] v129'1010 uv1010 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f43ec00b040 con 0x55822d8014f0 2024-09-21T03:43:38.715 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-09-21T03:43:38.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.713+0000 7f43f961af40 1 -- 172.21.15.38:0/995950130 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55822d8c6420 con 0x55822d8014f0 2024-09-21T03:43:38.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.714+0000 7f43f2c68640 1 -- 172.21.15.38:0/995950130 <== osd.7 v2:172.21.15.195:6818/1606106342 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v129'1011 uv1011 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f43ec00b040 con 0x55822d8014f0 2024-09-21T03:43:38.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.714+0000 7f43f961af40 1 -- 172.21.15.38:0/995950130 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55822d8c6420 con 0x55822d8014f0 2024-09-21T03:43:38.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.715+0000 7f43f2c68640 1 -- 172.21.15.38:0/995950130 <== osd.7 v2:172.21.15.195:6818/1606106342 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v129'1012 uv1012 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f43ec00b040 con 0x55822d8014f0 2024-09-21T03:43:38.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.716+0000 7f43f961af40 1 -- 172.21.15.38:0/995950130 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55822d8c6420 con 0x55822d8014f0 2024-09-21T03:43:38.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.717+0000 7f43f2c68640 1 -- 172.21.15.38:0/995950130 <== osd.7 v2:172.21.15.195:6818/1606106342 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v129'1013 uv1013 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f43ec00b040 con 0x55822d8014f0 2024-09-21T03:43:38.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.717+0000 7f43f961af40 1 -- 172.21.15.38:0/995950130 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:13 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55822d8c6420 con 0x55822d8014f0 2024-09-21T03:43:38.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.720+0000 7f43f2c68640 1 -- 172.21.15.38:0/995950130 <== osd.7 v2:172.21.15.195:6818/1606106342 13 ==== osd_op_reply(13 10000000000.00000000 [create] v129'1014 uv989 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7f43ec00b040 con 0x55822d8014f0 2024-09-21T03:43:38.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.720+0000 7f43f961af40 1 -- 172.21.15.38:0/995950130 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:14 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e129) -- 0x55822d8c6420 con 0x55822d8014f0 2024-09-21T03:43:38.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.722+0000 7f43f2c68640 1 -- 172.21.15.38:0/995950130 <== osd.7 v2:172.21.15.195:6818/1606106342 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v129'1015 uv1015 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f43ec00b040 con 0x55822d8014f0 2024-09-21T03:43:38.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.722+0000 7f43f961af40 1 -- 172.21.15.38:0/995950130 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:15 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55822d8c6420 con 0x55822d8014f0 2024-09-21T03:43:38.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.722+0000 7f43f2c68640 1 -- 172.21.15.38:0/995950130 <== osd.7 v2:172.21.15.195:6818/1606106342 15 ==== osd_op_reply(15 10000000000.00000000 [create] v129'1016 uv1016 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f43ec00b040 con 0x55822d8014f0 2024-09-21T03:43:38.726 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-09-21T03:43:38.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.723+0000 7f43f961af40 1 -- 172.21.15.38:0/995950130 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:16 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55822d8c6420 con 0x55822d8014f0 2024-09-21T03:43:38.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.724+0000 7f43f2c68640 1 -- 172.21.15.38:0/995950130 <== osd.7 v2:172.21.15.195:6818/1606106342 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v129'1017 uv1017 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f43ec00b040 con 0x55822d8014f0 2024-09-21T03:43:38.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.724+0000 7f43f961af40 1 -- 172.21.15.38:0/995950130 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:17 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55822d8c6420 con 0x55822d8014f0 2024-09-21T03:43:38.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.726+0000 7f43f2c68640 1 -- 172.21.15.38:0/995950130 <== osd.7 v2:172.21.15.195:6818/1606106342 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v129'1018 uv1018 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f43ec00b040 con 0x55822d8014f0 2024-09-21T03:43:38.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.726+0000 7f43f961af40 1 -- 172.21.15.38:0/995950130 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:18 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55822d8c6420 con 0x55822d8014f0 2024-09-21T03:43:38.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.727+0000 7f43f2c68640 1 -- 172.21.15.38:0/995950130 <== osd.7 v2:172.21.15.195:6818/1606106342 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v129'1019 uv1019 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f43ec00b040 con 0x55822d8014f0 2024-09-21T03:43:38.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.727+0000 7f43f961af40 1 -- 172.21.15.38:0/995950130 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:19 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=585b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55822d8c6870 con 0x55822d8014f0 2024-09-21T03:43:38.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.729+0000 7f43f2c68640 1 -- 172.21.15.38:0/995950130 <== osd.7 v2:172.21.15.195:6818/1606106342 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v129'1020 uv1020 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f43ec00b040 con 0x55822d8014f0 2024-09-21T03:43:38.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.729+0000 7f43f961af40 1 -- 172.21.15.38:0/995950130 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:20 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55822d8c6820 con 0x55822d8014f0 2024-09-21T03:43:38.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.732+0000 7f43f2c68640 1 -- 172.21.15.38:0/995950130 <== osd.7 v2:172.21.15.195:6818/1606106342 20 ==== osd_op_reply(20 600.00000000 [create] v129'1021 uv995 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f43ec00b040 con 0x55822d8014f0 2024-09-21T03:43:38.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.732+0000 7f43f961af40 1 -- 172.21.15.38:0/995950130 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:21 13.10 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e129) -- 0x55822d8c67e0 con 0x55822d8014f0 2024-09-21T03:43:38.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.733+0000 7f43f2c68640 1 -- 172.21.15.38:0/995950130 <== osd.7 v2:172.21.15.195:6818/1606106342 21 ==== osd_op_reply(21 600.00000000 [delete] v129'1022 uv1022 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f43ec00b040 con 0x55822d8014f0 2024-09-21T03:43:38.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.733+0000 7f43f961af40 1 -- 172.21.15.38:0/995950130 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:22 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55822d8c67e0 con 0x55822d8014f0 2024-09-21T03:43:38.737 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-09-21T03:43:38.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.734+0000 7f43f2c68640 1 -- 172.21.15.38:0/995950130 <== osd.7 v2:172.21.15.195:6818/1606106342 22 ==== osd_op_reply(22 600.00000000 [create] v129'1023 uv1023 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f43ec00b040 con 0x55822d8014f0 2024-09-21T03:43:38.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.734+0000 7f43f961af40 1 -- 172.21.15.38:0/995950130 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:23 13.10 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55822d8c67e0 con 0x55822d8014f0 2024-09-21T03:43:38.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.736+0000 7f43f2c68640 1 -- 172.21.15.38:0/995950130 <== osd.7 v2:172.21.15.195:6818/1606106342 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v129'1024 uv1024 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f43ec00b040 con 0x55822d8014f0 2024-09-21T03:43:38.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.736+0000 7f43f961af40 1 -- 172.21.15.38:0/995950130 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:24 13.10 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55822d8c66d0 con 0x55822d8014f0 2024-09-21T03:43:38.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.738+0000 7f43f2c68640 1 -- 172.21.15.38:0/995950130 <== osd.7 v2:172.21.15.195:6818/1606106342 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v129'1025 uv1025 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f43ec00b040 con 0x55822d8014f0 2024-09-21T03:43:38.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.738+0000 7f43f961af40 1 -- 172.21.15.38:0/995950130 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:25 13.10 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55822d8c66d0 con 0x55822d8014f0 2024-09-21T03:43:38.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.740+0000 7f43f2c68640 1 -- 172.21.15.38:0/995950130 <== osd.7 v2:172.21.15.195:6818/1606106342 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v129'1026 uv1026 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f43ec00b040 con 0x55822d8014f0 2024-09-21T03:43:38.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.740+0000 7f43f961af40 1 -- 172.21.15.38:0/995950130 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:26 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55822d8c66d0 con 0x55822d8014f0 2024-09-21T03:43:38.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.740+0000 7f43f2c68640 1 -- 172.21.15.38:0/995950130 <== osd.7 v2:172.21.15.195:6818/1606106342 26 ==== osd_op_reply(26 603.00000000 [create] v129'1027 uv1001 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f43ec00b040 con 0x55822d8014f0 2024-09-21T03:43:38.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.740+0000 7f43f961af40 1 -- 172.21.15.38:0/995950130 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:27 13.10 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e129) -- 0x55822d8c66d0 con 0x55822d8014f0 2024-09-21T03:43:38.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.742+0000 7f43f2c68640 1 -- 172.21.15.38:0/995950130 <== osd.7 v2:172.21.15.195:6818/1606106342 27 ==== osd_op_reply(27 603.00000000 [delete] v129'1028 uv1028 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f43ec00b040 con 0x55822d8014f0 2024-09-21T03:43:38.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.742+0000 7f43f961af40 1 -- 172.21.15.38:0/995950130 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:28 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55822d8c66d0 con 0x55822d8014f0 2024-09-21T03:43:38.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.743+0000 7f43f2c68640 1 -- 172.21.15.38:0/995950130 <== osd.7 v2:172.21.15.195:6818/1606106342 28 ==== osd_op_reply(28 603.00000000 [create] v129'1029 uv1029 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f43ec00b040 con 0x55822d8014f0 2024-09-21T03:43:38.746 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-09-21T03:43:38.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.743+0000 7f43f961af40 1 -- 172.21.15.38:0/995950130 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:29 13.10 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55822d8c66d0 con 0x55822d8014f0 2024-09-21T03:43:38.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.745+0000 7f43f2c68640 1 -- 172.21.15.38:0/995950130 <== osd.7 v2:172.21.15.195:6818/1606106342 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v129'1030 uv1030 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f43ec00b040 con 0x55822d8014f0 2024-09-21T03:43:38.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.745+0000 7f43f961af40 1 -- 172.21.15.38:0/995950130 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:30 13.10 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55822d8c66d0 con 0x55822d8014f0 2024-09-21T03:43:38.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.746+0000 7f43f2c68640 1 -- 172.21.15.38:0/995950130 <== osd.7 v2:172.21.15.195:6818/1606106342 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v129'1031 uv1031 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f43ec00b040 con 0x55822d8014f0 2024-09-21T03:43:38.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.746+0000 7f43f961af40 1 -- 172.21.15.38:0/995950130 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:31 13.10 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55822d8c66d0 con 0x55822d8014f0 2024-09-21T03:43:38.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.748+0000 7f43f2c68640 1 -- 172.21.15.38:0/995950130 <== osd.7 v2:172.21.15.195:6818/1606106342 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v129'1032 uv1032 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f43ec00b040 con 0x55822d8014f0 2024-09-21T03:43:38.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.748+0000 7f43f961af40 1 --2- 172.21.15.38:0/995950130 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x55822d8c73e0 0x55822d8c9800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:38.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.748+0000 7f43f961af40 1 -- 172.21.15.38:0/995950130 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55822d8c9d40 con 0x55822d8c73e0 2024-09-21T03:43:38.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.748+0000 7f43f2467640 1 --2- 172.21.15.38:0/995950130 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x55822d8c73e0 0x55822d8c9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:38.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.749+0000 7f43f2467640 1 --2- 172.21.15.38:0/995950130 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x55822d8c73e0 0x55822d8c9800 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:38.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.751+0000 7f43f2467640 1 -- 172.21.15.38:0/995950130 <== osd.6 v2:172.21.15.195:6826/4055825675 1 ==== osd_op_reply(32 1.00000000.inode [create] v129'726 uv708 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f43e4014030 con 0x55822d8c73e0 2024-09-21T03:43:38.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.751+0000 7f43f961af40 1 -- 172.21.15.38:0/995950130 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e129) -- 0x55822d8ca0e0 con 0x55822d8c73e0 2024-09-21T03:43:38.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.752+0000 7f43f2467640 1 -- 172.21.15.38:0/995950130 <== osd.6 v2:172.21.15.195:6826/4055825675 2 ==== osd_op_reply(33 1.00000000.inode [delete] v129'727 uv727 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f43e4014030 con 0x55822d8c73e0 2024-09-21T03:43:38.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.752+0000 7f43f961af40 1 -- 172.21.15.38:0/995950130 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55822d8ca480 con 0x55822d8c73e0 2024-09-21T03:43:38.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.753+0000 7f43f2467640 1 -- 172.21.15.38:0/995950130 <== osd.6 v2:172.21.15.195:6826/4055825675 3 ==== osd_op_reply(34 1.00000000.inode [create] v129'728 uv728 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f43e4014030 con 0x55822d8c73e0 2024-09-21T03:43:38.756 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-09-21T03:43:38.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.753+0000 7f43f961af40 1 -- 172.21.15.38:0/995950130 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55822d8cacd0 con 0x55822d8c73e0 2024-09-21T03:43:38.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.755+0000 7f43f2467640 1 -- 172.21.15.38:0/995950130 <== osd.6 v2:172.21.15.195:6826/4055825675 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v129'729 uv729 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f43e4014030 con 0x55822d8c73e0 2024-09-21T03:43:38.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.755+0000 7f43f961af40 1 -- 172.21.15.38:0/995950130 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55822d8cb070 con 0x55822d8c73e0 2024-09-21T03:43:38.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.756+0000 7f43f2467640 1 -- 172.21.15.38:0/995950130 <== osd.6 v2:172.21.15.195:6826/4055825675 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v129'730 uv730 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f43e4014030 con 0x55822d8c73e0 2024-09-21T03:43:38.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.756+0000 7f43f961af40 1 -- 172.21.15.38:0/995950130 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55822d8cb410 con 0x55822d8c73e0 2024-09-21T03:43:38.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.757+0000 7f43f2467640 1 -- 172.21.15.38:0/995950130 <== osd.6 v2:172.21.15.195:6826/4055825675 6 ==== osd_op_reply(37 200.00000000 [create] v129'731 uv713 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f43e4014030 con 0x55822d8c73e0 2024-09-21T03:43:38.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.757+0000 7f43f961af40 1 -- 172.21.15.38:0/995950130 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e129) -- 0x55822d8cb7b0 con 0x55822d8c73e0 2024-09-21T03:43:38.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.759+0000 7f43f2467640 1 -- 172.21.15.38:0/995950130 <== osd.6 v2:172.21.15.195:6826/4055825675 7 ==== osd_op_reply(38 200.00000000 [delete] v129'732 uv732 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f43e4014030 con 0x55822d8c73e0 2024-09-21T03:43:38.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.759+0000 7f43f961af40 1 -- 172.21.15.38:0/995950130 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55822d8cb7b0 con 0x55822d8c73e0 2024-09-21T03:43:38.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.760+0000 7f43f2467640 1 -- 172.21.15.38:0/995950130 <== osd.6 v2:172.21.15.195:6826/4055825675 8 ==== osd_op_reply(39 200.00000000 [create] v129'733 uv733 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f43e4014030 con 0x55822d8c73e0 2024-09-21T03:43:38.763 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-09-21T03:43:38.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.760+0000 7f43f961af40 1 -- 172.21.15.38:0/995950130 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55822d8cb7b0 con 0x55822d8c73e0 2024-09-21T03:43:38.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.761+0000 7f43f2467640 1 -- 172.21.15.38:0/995950130 <== osd.6 v2:172.21.15.195:6826/4055825675 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v129'734 uv734 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f43e4014030 con 0x55822d8c73e0 2024-09-21T03:43:38.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.761+0000 7f43f961af40 1 -- 172.21.15.38:0/995950130 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55822d8cb7b0 con 0x55822d8c73e0 2024-09-21T03:43:38.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.762+0000 7f43f2467640 1 -- 172.21.15.38:0/995950130 <== osd.6 v2:172.21.15.195:6826/4055825675 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v129'735 uv735 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f43e4014030 con 0x55822d8c73e0 2024-09-21T03:43:38.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.762+0000 7f43f961af40 1 -- 172.21.15.38:0/995950130 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:42 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55822d8cb7b0 con 0x55822d8c73e0 2024-09-21T03:43:38.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.764+0000 7f43f2467640 1 -- 172.21.15.38:0/995950130 <== osd.6 v2:172.21.15.195:6826/4055825675 11 ==== osd_op_reply(42 200.00000001 [create] v129'736 uv718 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f43e4014030 con 0x55822d8c73e0 2024-09-21T03:43:38.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.764+0000 7f43f961af40 1 -- 172.21.15.38:0/995950130 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:43 13.14 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e129) -- 0x55822d8cb7b0 con 0x55822d8c73e0 2024-09-21T03:43:38.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.765+0000 7f43f2467640 1 -- 172.21.15.38:0/995950130 <== osd.6 v2:172.21.15.195:6826/4055825675 12 ==== osd_op_reply(43 200.00000001 [delete] v129'737 uv737 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f43e4014030 con 0x55822d8c73e0 2024-09-21T03:43:38.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.765+0000 7f43f961af40 1 -- 172.21.15.38:0/995950130 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:44 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55822d8cb7b0 con 0x55822d8c73e0 2024-09-21T03:43:38.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.766+0000 7f43f2467640 1 -- 172.21.15.38:0/995950130 <== osd.6 v2:172.21.15.195:6826/4055825675 13 ==== osd_op_reply(44 200.00000001 [create] v129'738 uv738 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f43e4014030 con 0x55822d8c73e0 2024-09-21T03:43:38.769 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-09-21T03:43:38.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.766+0000 7f43f961af40 1 -- 172.21.15.38:0/995950130 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:45 13.14 13:2e2fa760:::200.00000001:head [write 0~17490 in=17490b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55822d8cfc30 con 0x55822d8c73e0 2024-09-21T03:43:38.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.768+0000 7f43f2467640 1 -- 172.21.15.38:0/995950130 <== osd.6 v2:172.21.15.195:6826/4055825675 14 ==== osd_op_reply(45 200.00000001 [write 0~17490] v129'739 uv739 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f43e4014030 con 0x55822d8c73e0 2024-09-21T03:43:38.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.768+0000 7f43f961af40 1 -- 172.21.15.38:0/995950130 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:46 13.14 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55822d8cfc30 con 0x55822d8c73e0 2024-09-21T03:43:38.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.770+0000 7f43f2467640 1 -- 172.21.15.38:0/995950130 <== osd.6 v2:172.21.15.195:6826/4055825675 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v129'740 uv740 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f43e4014030 con 0x55822d8c73e0 2024-09-21T03:43:38.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.770+0000 7f43f961af40 1 -- 172.21.15.38:0/995950130 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:47 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55822d8cb7b0 con 0x55822d8c73e0 2024-09-21T03:43:38.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.771+0000 7f43f2467640 1 -- 172.21.15.38:0/995950130 <== osd.6 v2:172.21.15.195:6826/4055825675 16 ==== osd_op_reply(47 606.00000000 [create] v129'741 uv724 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f43e4014030 con 0x55822d8c73e0 2024-09-21T03:43:38.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.771+0000 7f43f961af40 1 -- 172.21.15.38:0/995950130 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:48 13.14 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e129) -- 0x55822d8cb7b0 con 0x55822d8c73e0 2024-09-21T03:43:38.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.773+0000 7f43f2467640 1 -- 172.21.15.38:0/995950130 <== osd.6 v2:172.21.15.195:6826/4055825675 17 ==== osd_op_reply(48 606.00000000 [delete] v129'742 uv742 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f43e4014030 con 0x55822d8c73e0 2024-09-21T03:43:38.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.773+0000 7f43f961af40 1 -- 172.21.15.38:0/995950130 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:49 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55822d8cb7b0 con 0x55822d8c73e0 2024-09-21T03:43:38.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.774+0000 7f43f2467640 1 -- 172.21.15.38:0/995950130 <== osd.6 v2:172.21.15.195:6826/4055825675 18 ==== osd_op_reply(49 606.00000000 [create] v129'743 uv743 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f43e4014030 con 0x55822d8c73e0 2024-09-21T03:43:38.777 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-09-21T03:43:38.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.775+0000 7f43f961af40 1 -- 172.21.15.38:0/995950130 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:50 13.14 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55822d8cc7c0 con 0x55822d8c73e0 2024-09-21T03:43:38.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.776+0000 7f43f2467640 1 -- 172.21.15.38:0/995950130 <== osd.6 v2:172.21.15.195:6826/4055825675 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v129'744 uv744 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f43e4014030 con 0x55822d8c73e0 2024-09-21T03:43:38.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.776+0000 7f43f961af40 1 -- 172.21.15.38:0/995950130 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:51 13.14 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55822d8cc7c0 con 0x55822d8c73e0 2024-09-21T03:43:38.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.777+0000 7f43f2467640 1 -- 172.21.15.38:0/995950130 <== osd.6 v2:172.21.15.195:6826/4055825675 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v129'745 uv745 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f43e4014030 con 0x55822d8c73e0 2024-09-21T03:43:38.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.777+0000 7f43f961af40 1 -- 172.21.15.38:0/995950130 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:52 13.14 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55822d8cb7b0 con 0x55822d8c73e0 2024-09-21T03:43:38.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.779+0000 7f43f2467640 1 -- 172.21.15.38:0/995950130 <== osd.6 v2:172.21.15.195:6826/4055825675 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v129'746 uv746 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f43e4014030 con 0x55822d8c73e0 2024-09-21T03:43:38.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.779+0000 7f43f961af40 1 --2- 172.21.15.38:0/995950130 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x55822d8cbcc0 0x55822d8ce0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:38.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.779+0000 7f43f961af40 1 -- 172.21.15.38:0/995950130 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:53 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55822d8ce620 con 0x55822d8cbcc0 2024-09-21T03:43:38.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.779+0000 7f43f1c66640 1 --2- 172.21.15.38:0/995950130 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x55822d8cbcc0 0x55822d8ce0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:38.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.779+0000 7f43f1c66640 1 --2- 172.21.15.38:0/995950130 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x55822d8cbcc0 0x55822d8ce0e0 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:38.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.781+0000 7f43f1c66640 1 -- 172.21.15.38:0/995950130 <== osd.0 v2:172.21.15.38:6800/2381639433 1 ==== osd_op_reply(53 607.00000000 [create] v129'195 uv194 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f43e804ec60 con 0x55822d8cbcc0 2024-09-21T03:43:38.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.781+0000 7f43f961af40 1 -- 172.21.15.38:0/995950130 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:54 13.1c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e129) -- 0x55822d8ceaa0 con 0x55822d8cbcc0 2024-09-21T03:43:38.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.782+0000 7f43f1c66640 1 -- 172.21.15.38:0/995950130 <== osd.0 v2:172.21.15.38:6800/2381639433 2 ==== osd_op_reply(54 607.00000000 [delete] v129'196 uv196 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f43e804ec60 con 0x55822d8cbcc0 2024-09-21T03:43:38.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.782+0000 7f43f961af40 1 -- 172.21.15.38:0/995950130 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:55 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55822d8cef80 con 0x55822d8cbcc0 2024-09-21T03:43:38.786 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-09-21T03:43:38.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.783+0000 7f43f1c66640 1 -- 172.21.15.38:0/995950130 <== osd.0 v2:172.21.15.38:6800/2381639433 3 ==== osd_op_reply(55 607.00000000 [create] v129'197 uv197 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f43e804ec60 con 0x55822d8cbcc0 2024-09-21T03:43:38.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.783+0000 7f43f961af40 1 -- 172.21.15.38:0/995950130 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:56 13.1c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55822d8d0830 con 0x55822d8cbcc0 2024-09-21T03:43:38.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.784+0000 7f43f1c66640 1 -- 172.21.15.38:0/995950130 <== osd.0 v2:172.21.15.38:6800/2381639433 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v129'198 uv198 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f43e804ec60 con 0x55822d8cbcc0 2024-09-21T03:43:38.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.784+0000 7f43f961af40 1 -- 172.21.15.38:0/995950130 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:57 13.1c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55822d8d0bd0 con 0x55822d8cbcc0 2024-09-21T03:43:38.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.785+0000 7f43f1c66640 1 -- 172.21.15.38:0/995950130 <== osd.0 v2:172.21.15.38:6800/2381639433 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v129'199 uv199 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f43e804ec60 con 0x55822d8cbcc0 2024-09-21T03:43:38.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.785+0000 7f43f961af40 1 -- 172.21.15.38:0/995950130 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:58 13.1c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55822d8cf820 con 0x55822d8cbcc0 2024-09-21T03:43:38.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.786+0000 7f43f1c66640 1 -- 172.21.15.38:0/995950130 <== osd.0 v2:172.21.15.38:6800/2381639433 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v129'200 uv200 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f43e804ec60 con 0x55822d8cbcc0 2024-09-21T03:43:38.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.786+0000 7f43f961af40 1 --2- 172.21.15.38:0/995950130 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x55822d8d0120 0x55822d8d2f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:38.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.786+0000 7f43f961af40 1 -- 172.21.15.38:0/995950130 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:59 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55822d8d34c0 con 0x55822d8d0120 2024-09-21T03:43:38.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.786+0000 7f43f2c68640 1 --2- 172.21.15.38:0/995950130 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x55822d8d0120 0x55822d8d2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:38.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.787+0000 7f43f2c68640 1 --2- 172.21.15.38:0/995950130 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x55822d8d0120 0x55822d8d2f80 crc :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:38.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.788+0000 7f43f2c68640 1 -- 172.21.15.38:0/995950130 <== osd.3 v2:172.21.15.38:6808/1286810294 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v129'142 uv141 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f43ec00b040 con 0x55822d8d0120 2024-09-21T03:43:38.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.788+0000 7f43f961af40 1 -- 172.21.15.38:0/995950130 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:60 13.6 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e129) -- 0x55822d8d34c0 con 0x55822d8d0120 2024-09-21T03:43:38.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.789+0000 7f43f2c68640 1 -- 172.21.15.38:0/995950130 <== osd.3 v2:172.21.15.38:6808/1286810294 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v129'143 uv143 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f43ec00b040 con 0x55822d8d0120 2024-09-21T03:43:38.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.789+0000 7f43f961af40 1 -- 172.21.15.38:0/995950130 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:61 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55822d8d38b0 con 0x55822d8d0120 2024-09-21T03:43:38.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.790+0000 7f43f2c68640 1 -- 172.21.15.38:0/995950130 <== osd.3 v2:172.21.15.38:6808/1286810294 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v129'144 uv144 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f43ec00b040 con 0x55822d8d0120 2024-09-21T03:43:38.793 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-09-21T03:43:38.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.790+0000 7f43f961af40 1 -- 172.21.15.38:0/995950130 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:62 13.6 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55822d8d38b0 con 0x55822d8d0120 2024-09-21T03:43:38.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.791+0000 7f43f2c68640 1 -- 172.21.15.38:0/995950130 <== osd.3 v2:172.21.15.38:6808/1286810294 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v129'145 uv145 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f43ec00b040 con 0x55822d8d0120 2024-09-21T03:43:38.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.791+0000 7f43f961af40 1 -- 172.21.15.38:0/995950130 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55822d8d38b0 con 0x55822d8cbcc0 2024-09-21T03:43:38.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.793+0000 7f43f1c66640 1 -- 172.21.15.38:0/995950130 <== osd.0 v2:172.21.15.38:6800/2381639433 7 ==== osd_op_reply(63 608.00000000 [create] v129'195 uv194 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f43e804ec60 con 0x55822d8cbcc0 2024-09-21T03:43:38.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.793+0000 7f43f961af40 1 -- 172.21.15.38:0/995950130 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e129) -- 0x55822d8d38b0 con 0x55822d8cbcc0 2024-09-21T03:43:38.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.794+0000 7f43f1c66640 1 -- 172.21.15.38:0/995950130 <== osd.0 v2:172.21.15.38:6800/2381639433 8 ==== osd_op_reply(64 608.00000000 [delete] v129'196 uv196 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f43e804ec60 con 0x55822d8cbcc0 2024-09-21T03:43:38.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.794+0000 7f43f961af40 1 -- 172.21.15.38:0/995950130 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55822d8d38b0 con 0x55822d8cbcc0 2024-09-21T03:43:38.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.795+0000 7f43f1c66640 1 -- 172.21.15.38:0/995950130 <== osd.0 v2:172.21.15.38:6800/2381639433 9 ==== osd_op_reply(65 608.00000000 [create] v129'197 uv197 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f43e804ec60 con 0x55822d8cbcc0 2024-09-21T03:43:38.798 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-09-21T03:43:38.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.795+0000 7f43f961af40 1 -- 172.21.15.38:0/995950130 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55822d8d48c0 con 0x55822d8cbcc0 2024-09-21T03:43:38.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.796+0000 7f43f1c66640 1 -- 172.21.15.38:0/995950130 <== osd.0 v2:172.21.15.38:6800/2381639433 10 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v129'198 uv198 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f43e804ec60 con 0x55822d8cbcc0 2024-09-21T03:43:38.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.796+0000 7f43f961af40 1 -- 172.21.15.38:0/995950130 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55822d8d48c0 con 0x55822d8cbcc0 2024-09-21T03:43:38.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.797+0000 7f43f1c66640 1 -- 172.21.15.38:0/995950130 <== osd.0 v2:172.21.15.38:6800/2381639433 11 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v129'199 uv199 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f43e804ec60 con 0x55822d8cbcc0 2024-09-21T03:43:38.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.797+0000 7f43f961af40 1 -- 172.21.15.38:0/995950130 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55822d8d38b0 con 0x55822d8cbcc0 2024-09-21T03:43:38.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.798+0000 7f43f1c66640 1 -- 172.21.15.38:0/995950130 <== osd.0 v2:172.21.15.38:6800/2381639433 12 ==== osd_op_reply(68 608.00000000 [omap-set-header] v129'200 uv200 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f43e804ec60 con 0x55822d8cbcc0 2024-09-21T03:43:38.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.798+0000 7f43f961af40 1 -- 172.21.15.38:0/995950130 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55822d8cf3f0 con 0x55822d8d0120 2024-09-21T03:43:38.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.800+0000 7f43f2c68640 1 -- 172.21.15.38:0/995950130 <== osd.3 v2:172.21.15.38:6808/1286810294 5 ==== osd_op_reply(69 604.00000000 [create] v129'360 uv354 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f43ec00b040 con 0x55822d8d0120 2024-09-21T03:43:38.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.800+0000 7f43f961af40 1 -- 172.21.15.38:0/995950130 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e129) -- 0x55822d8cf3f0 con 0x55822d8d0120 2024-09-21T03:43:38.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.801+0000 7f43f2c68640 1 -- 172.21.15.38:0/995950130 <== osd.3 v2:172.21.15.38:6808/1286810294 6 ==== osd_op_reply(70 604.00000000 [delete] v129'361 uv361 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f43ec00b040 con 0x55822d8d0120 2024-09-21T03:43:38.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.801+0000 7f43f961af40 1 -- 172.21.15.38:0/995950130 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55822d8cf370 con 0x55822d8d0120 2024-09-21T03:43:38.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.802+0000 7f43f2c68640 1 -- 172.21.15.38:0/995950130 <== osd.3 v2:172.21.15.38:6808/1286810294 7 ==== osd_op_reply(71 604.00000000 [create] v129'362 uv362 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f43ec00b040 con 0x55822d8d0120 2024-09-21T03:43:38.805 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-09-21T03:43:38.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.802+0000 7f43f961af40 1 -- 172.21.15.38:0/995950130 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55822d8d48c0 con 0x55822d8d0120 2024-09-21T03:43:38.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.803+0000 7f43f2c68640 1 -- 172.21.15.38:0/995950130 <== osd.3 v2:172.21.15.38:6808/1286810294 8 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v129'363 uv363 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f43ec00b040 con 0x55822d8d0120 2024-09-21T03:43:38.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.803+0000 7f43f961af40 1 -- 172.21.15.38:0/995950130 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55822d8d48c0 con 0x55822d8d0120 2024-09-21T03:43:38.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.804+0000 7f43f2c68640 1 -- 172.21.15.38:0/995950130 <== osd.3 v2:172.21.15.38:6808/1286810294 9 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v129'364 uv364 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f43ec00b040 con 0x55822d8d0120 2024-09-21T03:43:38.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.804+0000 7f43f961af40 1 -- 172.21.15.38:0/995950130 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55822d8d38b0 con 0x55822d8d0120 2024-09-21T03:43:38.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.806+0000 7f43f2c68640 1 -- 172.21.15.38:0/995950130 <== osd.3 v2:172.21.15.38:6808/1286810294 10 ==== osd_op_reply(74 604.00000000 [omap-set-header] v129'365 uv365 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f43ec00b040 con 0x55822d8d0120 2024-09-21T03:43:38.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.806+0000 7f43f961af40 1 -- 172.21.15.38:0/995950130 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55822d8d38b0 con 0x55822d8d0120 2024-09-21T03:43:38.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.807+0000 7f43f2c68640 1 -- 172.21.15.38:0/995950130 <== osd.3 v2:172.21.15.38:6808/1286810294 11 ==== osd_op_reply(75 500.00000000 [create] v129'366 uv359 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f43ec00b040 con 0x55822d8d0120 2024-09-21T03:43:38.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.807+0000 7f43f961af40 1 -- 172.21.15.38:0/995950130 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e129) -- 0x55822d8d38b0 con 0x55822d8d0120 2024-09-21T03:43:38.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.808+0000 7f43f2c68640 1 -- 172.21.15.38:0/995950130 <== osd.3 v2:172.21.15.38:6808/1286810294 12 ==== osd_op_reply(76 500.00000000 [delete] v129'367 uv367 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f43ec00b040 con 0x55822d8d0120 2024-09-21T03:43:38.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.808+0000 7f43f961af40 1 -- 172.21.15.38:0/995950130 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55822d8d38b0 con 0x55822d8d0120 2024-09-21T03:43:38.812 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-09-21T03:43:38.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.810+0000 7f43f2c68640 1 -- 172.21.15.38:0/995950130 <== osd.3 v2:172.21.15.38:6808/1286810294 13 ==== osd_op_reply(77 500.00000000 [create] v129'368 uv368 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f43ec00b040 con 0x55822d8d0120 2024-09-21T03:43:38.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.810+0000 7f43f961af40 1 -- 172.21.15.38:0/995950130 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55822d8d38b0 con 0x55822d8d0120 2024-09-21T03:43:38.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.811+0000 7f43f2c68640 1 -- 172.21.15.38:0/995950130 <== osd.3 v2:172.21.15.38:6808/1286810294 14 ==== osd_op_reply(78 500.00000000 [write 0~90] v129'369 uv369 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f43ec00b040 con 0x55822d8d0120 2024-09-21T03:43:38.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.811+0000 7f43f961af40 1 -- 172.21.15.38:0/995950130 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55822d8d38b0 con 0x55822d8d0120 2024-09-21T03:43:38.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.812+0000 7f43f2c68640 1 -- 172.21.15.38:0/995950130 <== osd.3 v2:172.21.15.38:6808/1286810294 15 ==== osd_op_reply(79 500.00000000 [omap-set-header] v129'370 uv370 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f43ec00b040 con 0x55822d8d0120 2024-09-21T03:43:38.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.812+0000 7f43f961af40 1 -- 172.21.15.38:0/995950130 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:80 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55822d8d38b0 con 0x55822d8cbcc0 2024-09-21T03:43:38.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.813+0000 7f43f1c66640 1 -- 172.21.15.38:0/995950130 <== osd.0 v2:172.21.15.38:6800/2381639433 13 ==== osd_op_reply(80 mds_snaptable [create] v129'163 uv162 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f43e804ec60 con 0x55822d8cbcc0 2024-09-21T03:43:38.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.813+0000 7f43f961af40 1 -- 172.21.15.38:0/995950130 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:81 13.d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e129) -- 0x55822d8d38b0 con 0x55822d8cbcc0 2024-09-21T03:43:38.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.815+0000 7f43f1c66640 1 -- 172.21.15.38:0/995950130 <== osd.0 v2:172.21.15.38:6800/2381639433 14 ==== osd_op_reply(81 mds_snaptable [delete] v129'164 uv164 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f43e804ec60 con 0x55822d8cbcc0 2024-09-21T03:43:38.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.815+0000 7f43f961af40 1 -- 172.21.15.38:0/995950130 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:82 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55822d8d38b0 con 0x55822d8cbcc0 2024-09-21T03:43:38.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.816+0000 7f43f1c66640 1 -- 172.21.15.38:0/995950130 <== osd.0 v2:172.21.15.38:6800/2381639433 15 ==== osd_op_reply(82 mds_snaptable [create] v129'165 uv165 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f43e804ec60 con 0x55822d8cbcc0 2024-09-21T03:43:38.819 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-09-21T03:43:38.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.816+0000 7f43f961af40 1 -- 172.21.15.38:0/995950130 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:83 13.d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55822d8d38b0 con 0x55822d8cbcc0 2024-09-21T03:43:38.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.817+0000 7f43f1c66640 1 -- 172.21.15.38:0/995950130 <== osd.0 v2:172.21.15.38:6800/2381639433 16 ==== osd_op_reply(83 mds_snaptable [write 0~70] v129'166 uv166 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f43e804ec60 con 0x55822d8cbcc0 2024-09-21T03:43:38.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.817+0000 7f43f961af40 1 -- 172.21.15.38:0/995950130 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:84 13.d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55822d8d38b0 con 0x55822d8cbcc0 2024-09-21T03:43:38.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.818+0000 7f43f1c66640 1 -- 172.21.15.38:0/995950130 <== osd.0 v2:172.21.15.38:6800/2381639433 17 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v129'167 uv167 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f43e804ec60 con 0x55822d8cbcc0 2024-09-21T03:43:38.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.818+0000 7f43f961af40 1 --2- 172.21.15.38:0/995950130 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x55822d8d41f0 0x55822d8d4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:38.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.818+0000 7f43f961af40 1 -- 172.21.15.38:0/995950130 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55822d8d6cd0 con 0x55822d8d41f0 2024-09-21T03:43:38.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.818+0000 7f43f2467640 1 --2- 172.21.15.38:0/995950130 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x55822d8d41f0 0x55822d8d4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:38.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.818+0000 7f43f2467640 1 --2- 172.21.15.38:0/995950130 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x55822d8d41f0 0x55822d8d4600 crc :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:38.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.820+0000 7f43f2467640 1 -- 172.21.15.38:0/995950130 <== osd.1 v2:172.21.15.38:6816/1491818732 1 ==== osd_op_reply(85 605.00000000 [create] v129'195 uv194 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f43e4014030 con 0x55822d8d41f0 2024-09-21T03:43:38.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.820+0000 7f43f961af40 1 -- 172.21.15.38:0/995950130 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e129) -- 0x55822d8d6cd0 con 0x55822d8d41f0 2024-09-21T03:43:38.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.821+0000 7f43f2467640 1 -- 172.21.15.38:0/995950130 <== osd.1 v2:172.21.15.38:6816/1491818732 2 ==== osd_op_reply(86 605.00000000 [delete] v129'196 uv196 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f43e4014030 con 0x55822d8d41f0 2024-09-21T03:43:38.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.821+0000 7f43f961af40 1 -- 172.21.15.38:0/995950130 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55822d8d6cd0 con 0x55822d8d41f0 2024-09-21T03:43:38.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.822+0000 7f43f2467640 1 -- 172.21.15.38:0/995950130 <== osd.1 v2:172.21.15.38:6816/1491818732 3 ==== osd_op_reply(87 605.00000000 [create] v129'197 uv197 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f43e4014030 con 0x55822d8d41f0 2024-09-21T03:43:38.825 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-09-21T03:43:38.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.822+0000 7f43f961af40 1 -- 172.21.15.38:0/995950130 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55822d8d8100 con 0x55822d8d41f0 2024-09-21T03:43:38.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.823+0000 7f43f2467640 1 -- 172.21.15.38:0/995950130 <== osd.1 v2:172.21.15.38:6816/1491818732 4 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v129'198 uv198 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f43e4014030 con 0x55822d8d41f0 2024-09-21T03:43:38.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.823+0000 7f43f961af40 1 -- 172.21.15.38:0/995950130 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55822d8d8100 con 0x55822d8d41f0 2024-09-21T03:43:38.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.824+0000 7f43f2467640 1 -- 172.21.15.38:0/995950130 <== osd.1 v2:172.21.15.38:6816/1491818732 5 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v129'199 uv199 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f43e4014030 con 0x55822d8d41f0 2024-09-21T03:43:38.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.825+0000 7f43f961af40 1 -- 172.21.15.38:0/995950130 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55822d8d70f0 con 0x55822d8d41f0 2024-09-21T03:43:38.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.826+0000 7f43f2467640 1 -- 172.21.15.38:0/995950130 <== osd.1 v2:172.21.15.38:6816/1491818732 6 ==== osd_op_reply(90 605.00000000 [omap-set-header] v129'200 uv200 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f43e4014030 con 0x55822d8d41f0 2024-09-21T03:43:38.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.826+0000 7f43f961af40 1 -- 172.21.15.38:0/995950130 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55822d8d70f0 con 0x55822d8d41f0 2024-09-21T03:43:38.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.827+0000 7f43f2467640 1 -- 172.21.15.38:0/995950130 <== osd.1 v2:172.21.15.38:6816/1491818732 7 ==== osd_op_reply(91 mds0_inotable [create] v129'325 uv318 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f43e4014030 con 0x55822d8d41f0 2024-09-21T03:43:38.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.827+0000 7f43f961af40 1 -- 172.21.15.38:0/995950130 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e129) -- 0x55822d8d70f0 con 0x55822d8d41f0 2024-09-21T03:43:38.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.828+0000 7f43f2467640 1 -- 172.21.15.38:0/995950130 <== osd.1 v2:172.21.15.38:6816/1491818732 8 ==== osd_op_reply(92 mds0_inotable [delete] v129'326 uv326 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f43e4014030 con 0x55822d8d41f0 2024-09-21T03:43:38.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.828+0000 7f43f961af40 1 -- 172.21.15.38:0/995950130 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55822d8d70f0 con 0x55822d8d41f0 2024-09-21T03:43:38.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.830+0000 7f43f2467640 1 -- 172.21.15.38:0/995950130 <== osd.1 v2:172.21.15.38:6816/1491818732 9 ==== osd_op_reply(93 mds0_inotable [create] v129'327 uv327 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f43e4014030 con 0x55822d8d41f0 2024-09-21T03:43:38.833 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-09-21T03:43:38.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.830+0000 7f43f961af40 1 -- 172.21.15.38:0/995950130 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:94 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55822d8d70f0 con 0x55822d8d41f0 2024-09-21T03:43:38.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.831+0000 7f43f2467640 1 -- 172.21.15.38:0/995950130 <== osd.1 v2:172.21.15.38:6816/1491818732 10 ==== osd_op_reply(94 mds0_inotable [write 0~34] v129'328 uv328 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f43e4014030 con 0x55822d8d41f0 2024-09-21T03:43:38.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.831+0000 7f43f961af40 1 -- 172.21.15.38:0/995950130 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:95 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55822d8d70f0 con 0x55822d8d41f0 2024-09-21T03:43:38.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.832+0000 7f43f2467640 1 -- 172.21.15.38:0/995950130 <== osd.1 v2:172.21.15.38:6816/1491818732 11 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v129'329 uv329 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f43e4014030 con 0x55822d8d41f0 2024-09-21T03:43:38.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.832+0000 7f43f961af40 1 -- 172.21.15.38:0/995950130 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:96 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55822d8d70f0 con 0x55822d8d41f0 2024-09-21T03:43:38.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.833+0000 7f43f2467640 1 -- 172.21.15.38:0/995950130 <== osd.1 v2:172.21.15.38:6816/1491818732 12 ==== osd_op_reply(96 100.00000000 [create] v129'330 uv324 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f43e4014030 con 0x55822d8d41f0 2024-09-21T03:43:38.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.833+0000 7f43f961af40 1 -- 172.21.15.38:0/995950130 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:97 13.13 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e129) -- 0x55822d8d70f0 con 0x55822d8d41f0 2024-09-21T03:43:38.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.834+0000 7f43f2467640 1 -- 172.21.15.38:0/995950130 <== osd.1 v2:172.21.15.38:6816/1491818732 13 ==== osd_op_reply(97 100.00000000 [delete] v129'331 uv331 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f43e4014030 con 0x55822d8d41f0 2024-09-21T03:43:38.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.834+0000 7f43f961af40 1 -- 172.21.15.38:0/995950130 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:98 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55822d8d70f0 con 0x55822d8d41f0 2024-09-21T03:43:38.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.836+0000 7f43f2467640 1 -- 172.21.15.38:0/995950130 <== osd.1 v2:172.21.15.38:6816/1491818732 14 ==== osd_op_reply(98 100.00000000 [create] v129'332 uv332 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f43e4014030 con 0x55822d8d41f0 2024-09-21T03:43:38.839 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-09-21T03:43:38.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.836+0000 7f43f961af40 1 -- 172.21.15.38:0/995950130 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:99 13.13 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55822d8d70f0 con 0x55822d8d41f0 2024-09-21T03:43:38.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.837+0000 7f43f2467640 1 -- 172.21.15.38:0/995950130 <== osd.1 v2:172.21.15.38:6816/1491818732 15 ==== osd_op_reply(99 100.00000000 [omap-set-header] v129'333 uv333 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f43e4014030 con 0x55822d8d41f0 2024-09-21T03:43:38.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.837+0000 7f43f961af40 1 -- 172.21.15.38:0/995950130 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:100 13.13 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55822d8d9f00 con 0x55822d8d41f0 2024-09-21T03:43:38.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.838+0000 7f43f2467640 1 -- 172.21.15.38:0/995950130 <== osd.1 v2:172.21.15.38:6816/1491818732 16 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v129'334 uv334 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f43e4014030 con 0x55822d8d41f0 2024-09-21T03:43:38.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.838+0000 7f43f961af40 1 -- 172.21.15.38:0/995950130 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55822d8d75a0 con 0x55822d8d0120 2024-09-21T03:43:38.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.840+0000 7f43f2c68640 1 -- 172.21.15.38:0/995950130 <== osd.3 v2:172.21.15.38:6808/1286810294 16 ==== osd_op_reply(101 mds0_sessionmap [create] v129'142 uv140 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7f43ec00b040 con 0x55822d8d0120 2024-09-21T03:43:38.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.840+0000 7f43f961af40 1 -- 172.21.15.38:0/995950130 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e129) -- 0x55822d8d75a0 con 0x55822d8d0120 2024-09-21T03:43:38.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.841+0000 7f43f2c68640 1 -- 172.21.15.38:0/995950130 <== osd.3 v2:172.21.15.38:6808/1286810294 17 ==== osd_op_reply(102 mds0_sessionmap [delete] v129'143 uv143 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f43ec00b040 con 0x55822d8d0120 2024-09-21T03:43:38.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.841+0000 7f43f961af40 1 -- 172.21.15.38:0/995950130 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:103 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55822d8d75a0 con 0x55822d8d0120 2024-09-21T03:43:38.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.842+0000 7f43f2c68640 1 -- 172.21.15.38:0/995950130 <== osd.3 v2:172.21.15.38:6808/1286810294 18 ==== osd_op_reply(103 mds0_sessionmap [create] v129'144 uv144 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f43ec00b040 con 0x55822d8d0120 2024-09-21T03:43:38.845 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-09-21T03:43:38.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.842+0000 7f43f961af40 1 -- 172.21.15.38:0/995950130 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:104 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55822d8d75a0 con 0x55822d8d0120 2024-09-21T03:43:38.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.843+0000 7f43f2c68640 1 -- 172.21.15.38:0/995950130 <== osd.3 v2:172.21.15.38:6808/1286810294 19 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v129'145 uv145 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f43ec00b040 con 0x55822d8d0120 2024-09-21T03:43:38.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.843+0000 7f43f961af40 1 -- 172.21.15.38:0/995950130 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55822d8d75a0 con 0x55822d8d0120 2024-09-21T03:43:38.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.844+0000 7f43f2c68640 1 -- 172.21.15.38:0/995950130 <== osd.3 v2:172.21.15.38:6808/1286810294 20 ==== osd_op_reply(105 609.00000000 [create] v129'195 uv194 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f43ec00b040 con 0x55822d8d0120 2024-09-21T03:43:38.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.844+0000 7f43f961af40 1 -- 172.21.15.38:0/995950130 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e129) -- 0x55822d8d75a0 con 0x55822d8d0120 2024-09-21T03:43:38.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.845+0000 7f43f2c68640 1 -- 172.21.15.38:0/995950130 <== osd.3 v2:172.21.15.38:6808/1286810294 21 ==== osd_op_reply(106 609.00000000 [delete] v129'196 uv196 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f43ec00b040 con 0x55822d8d0120 2024-09-21T03:43:38.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.845+0000 7f43f961af40 1 -- 172.21.15.38:0/995950130 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55822d8d75a0 con 0x55822d8d0120 2024-09-21T03:43:38.849 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-09-21T03:43:38.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.846+0000 7f43f2c68640 1 -- 172.21.15.38:0/995950130 <== osd.3 v2:172.21.15.38:6808/1286810294 22 ==== osd_op_reply(107 609.00000000 [create] v129'197 uv197 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f43ec00b040 con 0x55822d8d0120 2024-09-21T03:43:38.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.846+0000 7f43f961af40 1 -- 172.21.15.38:0/995950130 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55822d8d85b0 con 0x55822d8d0120 2024-09-21T03:43:38.850 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.848+0000 7f43f2c68640 1 -- 172.21.15.38:0/995950130 <== osd.3 v2:172.21.15.38:6808/1286810294 23 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v129'198 uv198 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f43ec00b040 con 0x55822d8d0120 2024-09-21T03:43:38.850 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.848+0000 7f43f961af40 1 -- 172.21.15.38:0/995950130 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:109 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55822d8d85b0 con 0x55822d8d0120 2024-09-21T03:43:38.851 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.849+0000 7f43f2c68640 1 -- 172.21.15.38:0/995950130 <== osd.3 v2:172.21.15.38:6808/1286810294 24 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v129'199 uv199 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f43ec00b040 con 0x55822d8d0120 2024-09-21T03:43:38.851 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.849+0000 7f43f961af40 1 -- 172.21.15.38:0/995950130 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:110 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55822d8d75a0 con 0x55822d8d0120 2024-09-21T03:43:38.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.850+0000 7f43f2c68640 1 -- 172.21.15.38:0/995950130 <== osd.3 v2:172.21.15.38:6808/1286810294 25 ==== osd_op_reply(110 609.00000000 [omap-set-header] v129'200 uv200 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f43ec00b040 con 0x55822d8d0120 2024-09-21T03:43:38.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.850+0000 7f43f961af40 1 --2- 172.21.15.38:0/995950130 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x55822d8d7c20 0x55822d8d8030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:38.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.850+0000 7f43f961af40 1 -- 172.21.15.38:0/995950130 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55822d8d8570 con 0x55822d8d7c20 2024-09-21T03:43:38.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.850+0000 7f43f1c66640 1 --2- 172.21.15.38:0/995950130 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x55822d8d7c20 0x55822d8d8030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:38.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.850+0000 7f43f1c66640 1 --2- 172.21.15.38:0/995950130 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x55822d8d7c20 0x55822d8d8030 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:38.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.852+0000 7f43f1c66640 1 -- 172.21.15.38:0/995950130 <== osd.2 v2:172.21.15.38:6824/2577858902 1 ==== osd_op_reply(111 400.00000000 [create] v129'166 uv165 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f43e804ec60 con 0x55822d8d7c20 2024-09-21T03:43:38.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.852+0000 7f43f961af40 1 -- 172.21.15.38:0/995950130 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e129) -- 0x55822d8d8570 con 0x55822d8d7c20 2024-09-21T03:43:38.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.853+0000 7f43f1c66640 1 -- 172.21.15.38:0/995950130 <== osd.2 v2:172.21.15.38:6824/2577858902 2 ==== osd_op_reply(112 400.00000000 [delete] v129'167 uv167 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f43e804ec60 con 0x55822d8d7c20 2024-09-21T03:43:38.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.853+0000 7f43f961af40 1 -- 172.21.15.38:0/995950130 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55822d8d8570 con 0x55822d8d7c20 2024-09-21T03:43:38.857 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-09-21T03:43:38.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.854+0000 7f43f1c66640 1 -- 172.21.15.38:0/995950130 <== osd.2 v2:172.21.15.38:6824/2577858902 3 ==== osd_op_reply(113 400.00000000 [create] v129'168 uv168 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f43e804ec60 con 0x55822d8d7c20 2024-09-21T03:43:38.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.854+0000 7f43f961af40 1 -- 172.21.15.38:0/995950130 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:114 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55822d8d8570 con 0x55822d8d7c20 2024-09-21T03:43:38.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.855+0000 7f43f1c66640 1 -- 172.21.15.38:0/995950130 <== osd.2 v2:172.21.15.38:6824/2577858902 4 ==== osd_op_reply(114 400.00000000 [write 0~22] v129'169 uv169 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f43e804ec60 con 0x55822d8d7c20 2024-09-21T03:43:38.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.855+0000 7f43f961af40 1 -- 172.21.15.38:0/995950130 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:115 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55822d8d8570 con 0x55822d8d7c20 2024-09-21T03:43:38.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.856+0000 7f43f1c66640 1 -- 172.21.15.38:0/995950130 <== osd.2 v2:172.21.15.38:6824/2577858902 5 ==== osd_op_reply(115 400.00000000 [omap-set-header] v129'170 uv170 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f43e804ec60 con 0x55822d8d7c20 2024-09-21T03:43:38.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.856+0000 7f43f961af40 1 -- 172.21.15.38:0/995950130 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:116 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55822d8d8570 con 0x55822d8d41f0 2024-09-21T03:43:38.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.858+0000 7f43f2467640 1 -- 172.21.15.38:0/995950130 <== osd.1 v2:172.21.15.38:6816/1491818732 17 ==== osd_op_reply(116 100.00000000.inode [create] v129'402 uv393 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7f43e4014030 con 0x55822d8d41f0 2024-09-21T03:43:38.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.858+0000 7f43f961af40 1 -- 172.21.15.38:0/995950130 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:117 13.1f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e129) -- 0x55822d8d8570 con 0x55822d8d41f0 2024-09-21T03:43:38.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.859+0000 7f43f2467640 1 -- 172.21.15.38:0/995950130 <== osd.1 v2:172.21.15.38:6816/1491818732 18 ==== osd_op_reply(117 100.00000000.inode [delete] v129'403 uv403 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f43e4014030 con 0x55822d8d41f0 2024-09-21T03:43:38.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.859+0000 7f43f961af40 1 -- 172.21.15.38:0/995950130 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:118 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55822d8d89b0 con 0x55822d8d41f0 2024-09-21T03:43:38.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.860+0000 7f43f2467640 1 -- 172.21.15.38:0/995950130 <== osd.1 v2:172.21.15.38:6816/1491818732 19 ==== osd_op_reply(118 100.00000000.inode [create] v129'404 uv404 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f43e4014030 con 0x55822d8d41f0 2024-09-21T03:43:38.863 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-09-21T03:43:38.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.860+0000 7f43f961af40 1 -- 172.21.15.38:0/995950130 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:119 13.1f 13:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55822d8d89b0 con 0x55822d8d41f0 2024-09-21T03:43:38.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.861+0000 7f43f2467640 1 -- 172.21.15.38:0/995950130 <== osd.1 v2:172.21.15.38:6816/1491818732 20 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v129'405 uv405 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f43e4014030 con 0x55822d8d41f0 2024-09-21T03:43:38.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.861+0000 7f43f961af40 1 -- 172.21.15.38:0/995950130 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:120 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55822d8d89b0 con 0x55822d8d41f0 2024-09-21T03:43:38.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.862+0000 7f43f2467640 1 -- 172.21.15.38:0/995950130 <== osd.1 v2:172.21.15.38:6816/1491818732 21 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v129'406 uv406 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f43e4014030 con 0x55822d8d41f0 2024-09-21T03:43:38.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.862+0000 7f43f961af40 1 -- 172.21.15.38:0/995950130 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:121 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55822d8d89b0 con 0x55822d8d41f0 2024-09-21T03:43:38.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.863+0000 7f43f2467640 1 -- 172.21.15.38:0/995950130 <== osd.1 v2:172.21.15.38:6816/1491818732 22 ==== osd_op_reply(121 1.00000000 [create] v129'407 uv400 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7f43e4014030 con 0x55822d8d41f0 2024-09-21T03:43:38.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.863+0000 7f43f961af40 1 -- 172.21.15.38:0/995950130 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:122 13.1f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e129) -- 0x55822d8d89b0 con 0x55822d8d41f0 2024-09-21T03:43:38.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.864+0000 7f43f2467640 1 -- 172.21.15.38:0/995950130 <== osd.1 v2:172.21.15.38:6816/1491818732 23 ==== osd_op_reply(122 1.00000000 [delete] v129'408 uv408 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f43e4014030 con 0x55822d8d41f0 2024-09-21T03:43:38.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.864+0000 7f43f961af40 1 -- 172.21.15.38:0/995950130 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:123 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55822d8d89b0 con 0x55822d8d41f0 2024-09-21T03:43:38.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.865+0000 7f43f2467640 1 -- 172.21.15.38:0/995950130 <== osd.1 v2:172.21.15.38:6816/1491818732 24 ==== osd_op_reply(123 1.00000000 [create] v129'409 uv409 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f43e4014030 con 0x55822d8d41f0 2024-09-21T03:43:38.868 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-09-21T03:43:38.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.866+0000 7f43f961af40 1 -- 172.21.15.38:0/995950130 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:124 13.1f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55822d8d99c0 con 0x55822d8d41f0 2024-09-21T03:43:38.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.867+0000 7f43f2467640 1 -- 172.21.15.38:0/995950130 <== osd.1 v2:172.21.15.38:6816/1491818732 25 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v129'410 uv410 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f43e4014030 con 0x55822d8d41f0 2024-09-21T03:43:38.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.867+0000 7f43f961af40 1 -- 172.21.15.38:0/995950130 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:125 13.1f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55822d8d9970 con 0x55822d8d41f0 2024-09-21T03:43:38.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.868+0000 7f43f2467640 1 -- 172.21.15.38:0/995950130 <== osd.1 v2:172.21.15.38:6816/1491818732 26 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v129'411 uv411 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f43e4014030 con 0x55822d8d41f0 2024-09-21T03:43:38.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.868+0000 7f43f961af40 1 -- 172.21.15.38:0/995950130 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:126 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55822d8d9da0 con 0x55822d8d41f0 2024-09-21T03:43:38.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.869+0000 7f43f2467640 1 -- 172.21.15.38:0/995950130 <== osd.1 v2:172.21.15.38:6816/1491818732 27 ==== osd_op_reply(126 1.00000000 [omap-set-header] v129'412 uv412 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f43e4014030 con 0x55822d8d41f0 2024-09-21T03:43:38.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.869+0000 7f43f961af40 1 -- 172.21.15.38:0/995950130 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:127 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=584b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55822d8d9da0 con 0x55822d8d41f0 2024-09-21T03:43:38.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.870+0000 7f43f2467640 1 -- 172.21.15.38:0/995950130 <== osd.1 v2:172.21.15.38:6816/1491818732 28 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v129'413 uv413 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f43e4014030 con 0x55822d8d41f0 2024-09-21T03:43:38.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.870+0000 7f43f961af40 1 -- 172.21.15.38:0/995950130 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x55822d8cbcc0 msgr2=0x55822d8ce0e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:38.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.870+0000 7f43f961af40 1 --2- 172.21.15.38:0/995950130 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x55822d8cbcc0 0x55822d8ce0e0 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:38.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.870+0000 7f43f961af40 1 -- 172.21.15.38:0/995950130 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x55822d8d41f0 msgr2=0x55822d8d4600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:38.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.870+0000 7f43f961af40 1 --2- 172.21.15.38:0/995950130 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x55822d8d41f0 0x55822d8d4600 crc :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:38.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.870+0000 7f43f961af40 1 -- 172.21.15.38:0/995950130 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x55822d8d7c20 msgr2=0x55822d8d8030 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:38.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.870+0000 7f43f961af40 1 --2- 172.21.15.38:0/995950130 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x55822d8d7c20 0x55822d8d8030 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:38.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.870+0000 7f43f961af40 1 -- 172.21.15.38:0/995950130 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x55822d8d0120 msgr2=0x55822d8d2f80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:38.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.870+0000 7f43f961af40 1 --2- 172.21.15.38:0/995950130 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x55822d8d0120 0x55822d8d2f80 crc :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:38.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.870+0000 7f43f2c68640 1 -- 172.21.15.38:0/995950130 reap_dead start 2024-09-21T03:43:38.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.870+0000 7f43f961af40 1 -- 172.21.15.38:0/995950130 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x55822d8c73e0 msgr2=0x55822d8c9800 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:38.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.870+0000 7f43f961af40 1 --2- 172.21.15.38:0/995950130 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x55822d8c73e0 0x55822d8c9800 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:38.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.870+0000 7f43f2c68640 1 -- 172.21.15.38:0/995950130 reap_dead start 2024-09-21T03:43:38.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.870+0000 7f43f961af40 1 -- 172.21.15.38:0/995950130 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x55822d8014f0 msgr2=0x55822d801900 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:38.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.870+0000 7f43f961af40 1 --2- 172.21.15.38:0/995950130 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x55822d8014f0 0x55822d801900 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:38.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.870+0000 7f43f961af40 1 -- 172.21.15.38:0/995950130 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f43d4077570 msgr2=0x7f43d4079a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:38.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.870+0000 7f43f961af40 1 --2- 172.21.15.38:0/995950130 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f43d4077570 0x7f43d4079a80 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f43e8055000 tx=0x7f43e8058040 comp rx=0 tx=0).stop 2024-09-21T03:43:38.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.870+0000 7f43f961af40 1 -- 172.21.15.38:0/995950130 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55822d74bd00 msgr2=0x55822d7fe960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:38.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.870+0000 7f43f961af40 1 --2- 172.21.15.38:0/995950130 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55822d74bd00 0x55822d7fe960 secure :-1 s=READY pgs=2064 cs=0 l=1 rev1=1 crypto rx=0x7f43e4010b70 tx=0x7f43e4010ba0 comp rx=0 tx=0).stop 2024-09-21T03:43:38.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.870+0000 7f43f961af40 1 -- 172.21.15.38:0/995950130 shutdown_connections 2024-09-21T03:43:38.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.870+0000 7f43f961af40 1 --2- 172.21.15.38:0/995950130 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x55822d8c73e0 0x55822d8c9800 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:38.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.870+0000 7f43f961af40 1 --2- 172.21.15.38:0/995950130 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x55822d8014f0 0x55822d801900 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:38.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.870+0000 7f43f961af40 1 --2- 172.21.15.38:0/995950130 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f43d4077570 0x7f43d4079a80 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:38.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.870+0000 7f43f961af40 1 --2- 172.21.15.38:0/995950130 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55822d74bd00 0x55822d7fe960 unknown :-1 s=CLOSED pgs=2064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:38.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.870+0000 7f43f961af40 1 -- 172.21.15.38:0/995950130 >> 172.21.15.38:0/995950130 conn(0x55822d66aac0 msgr2=0x55822d77a690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:38.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.870+0000 7f43f961af40 1 -- 172.21.15.38:0/995950130 shutdown_connections 2024-09-21T03:43:38.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.870+0000 7f43f961af40 1 -- 172.21.15.38:0/995950130 wait complete. 2024-09-21T03:43:38.877 DEBUG:teuthology.orchestra.run.smithi038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata setomapval 1.00000000 subdir_head deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef 2024-09-21T03:43:38.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.971+0000 7fbb2f71cf40 1 Processor -- start 2024-09-21T03:43:38.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.971+0000 7fbb2f71cf40 1 -- start start 2024-09-21T03:43:38.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.972+0000 7fbb2f71cf40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f882084240 0x55f8820c99e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:38.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.972+0000 7fbb2f71cf40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f8820cb090 0x55f8820b1030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:38.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.972+0000 7fbb2f71cf40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f882140520 0x55f8820b1570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:38.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.972+0000 7fbb2f71cf40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55f881fd1d60 con 0x55f882084240 2024-09-21T03:43:38.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.972+0000 7fbb2f71cf40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55f882030430 con 0x55f882140520 2024-09-21T03:43:38.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.972+0000 7fbb2f71cf40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55f882158350 con 0x55f8820cb090 2024-09-21T03:43:38.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.972+0000 7fbb288b7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f882084240 0x55f8820c99e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:38.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.972+0000 7fbb288b7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f882084240 0x55f8820c99e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49852/0 (socket says 172.21.15.38:49852) 2024-09-21T03:43:38.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.972+0000 7fbb288b7640 1 -- 172.21.15.38:0/31146360 learned_addr learned my addr 172.21.15.38:0/31146360 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:43:38.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.973+0000 7fbb280b6640 1 --2- 172.21.15.38:0/31146360 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f8820cb090 0x55f8820b1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:38.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.973+0000 7fbb290b8640 1 --2- 172.21.15.38:0/31146360 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f882140520 0x55f8820b1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:38.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.973+0000 7fbb288b7640 1 -- 172.21.15.38:0/31146360 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f8820cb090 msgr2=0x55f8820b1030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:38.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.973+0000 7fbb288b7640 1 --2- 172.21.15.38:0/31146360 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f8820cb090 0x55f8820b1030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:38.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.973+0000 7fbb288b7640 1 -- 172.21.15.38:0/31146360 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f882140520 msgr2=0x55f8820b1570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:38.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.973+0000 7fbb288b7640 1 --2- 172.21.15.38:0/31146360 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f882140520 0x55f8820b1570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:38.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.973+0000 7fbb288b7640 1 -- 172.21.15.38:0/31146360 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f882154870 con 0x55f882084240 2024-09-21T03:43:38.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.973+0000 7fbb288b7640 1 --2- 172.21.15.38:0/31146360 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f882084240 0x55f8820c99e0 secure :-1 s=READY pgs=2067 cs=0 l=1 rev1=1 crypto rx=0x7fbb18001e20 tx=0x7fbb1800b730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a70b01436ad6a08c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:38.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.973+0000 7fbb270b4640 1 -- 172.21.15.38:0/31146360 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb1800e830 con 0x55f882084240 2024-09-21T03:43:38.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.973+0000 7fbb270b4640 1 -- 172.21.15.38:0/31146360 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbb18006050 con 0x55f882084240 2024-09-21T03:43:38.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.973+0000 7fbb270b4640 1 -- 172.21.15.38:0/31146360 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb1800ed60 con 0x55f882084240 2024-09-21T03:43:38.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.974+0000 7fbb2f71cf40 1 -- 172.21.15.38:0/31146360 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f882084240 msgr2=0x55f8820c99e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:38.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.974+0000 7fbb2f71cf40 1 --2- 172.21.15.38:0/31146360 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f882084240 0x55f8820c99e0 secure :-1 s=READY pgs=2067 cs=0 l=1 rev1=1 crypto rx=0x7fbb18001e20 tx=0x7fbb1800b730 comp rx=0 tx=0).stop 2024-09-21T03:43:38.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.974+0000 7fbb2f71cf40 1 -- 172.21.15.38:0/31146360 shutdown_connections 2024-09-21T03:43:38.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.974+0000 7fbb2f71cf40 1 --2- 172.21.15.38:0/31146360 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f882140520 0x55f8820b1570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:38.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.974+0000 7fbb2f71cf40 1 --2- 172.21.15.38:0/31146360 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f8820cb090 0x55f8820b1030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:38.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.974+0000 7fbb2f71cf40 1 --2- 172.21.15.38:0/31146360 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f882084240 0x55f8820c99e0 unknown :-1 s=CLOSED pgs=2067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:38.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.974+0000 7fbb2f71cf40 1 -- 172.21.15.38:0/31146360 >> 172.21.15.38:0/31146360 conn(0x55f881fa2ac0 msgr2=0x55f8820adb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:38.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.974+0000 7fbb2f71cf40 1 -- 172.21.15.38:0/31146360 shutdown_connections 2024-09-21T03:43:38.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.974+0000 7fbb2f71cf40 1 -- 172.21.15.38:0/31146360 wait complete. 2024-09-21T03:43:38.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.975+0000 7fbb2f71cf40 1 Processor -- start 2024-09-21T03:43:38.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.975+0000 7fbb2f71cf40 1 -- start start 2024-09-21T03:43:38.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.976+0000 7fbb2f71cf40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f882084240 0x55f882124770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:38.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.976+0000 7fbb2f71cf40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f8820cb090 0x55f88213dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:38.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.976+0000 7fbb2f71cf40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f882140520 0x55f8821529b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:38.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.976+0000 7fbb2f71cf40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55f8820b1ab0 con 0x55f882084240 2024-09-21T03:43:38.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.976+0000 7fbb2f71cf40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55f882152710 con 0x55f8820cb090 2024-09-21T03:43:38.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.976+0000 7fbb2f71cf40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55f8821f1850 con 0x55f882140520 2024-09-21T03:43:38.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.976+0000 7fbb288b7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f882084240 0x55f882124770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:38.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.976+0000 7fbb288b7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f882084240 0x55f882124770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49854/0 (socket says 172.21.15.38:49854) 2024-09-21T03:43:38.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.976+0000 7fbb288b7640 1 -- 172.21.15.38:0/1202877121 learned_addr learned my addr 172.21.15.38:0/1202877121 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:43:38.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.976+0000 7fbb280b6640 1 --2- 172.21.15.38:0/1202877121 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f8820cb090 0x55f88213dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:38.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.976+0000 7fbb290b8640 1 --2- 172.21.15.38:0/1202877121 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f882140520 0x55f8821529b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:38.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.976+0000 7fbb288b7640 1 -- 172.21.15.38:0/1202877121 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f882140520 msgr2=0x55f8821529b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:38.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.976+0000 7fbb288b7640 1 --2- 172.21.15.38:0/1202877121 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f882140520 0x55f8821529b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:38.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.976+0000 7fbb288b7640 1 -- 172.21.15.38:0/1202877121 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f8820cb090 msgr2=0x55f88213dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:38.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.976+0000 7fbb288b7640 1 --2- 172.21.15.38:0/1202877121 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f8820cb090 0x55f88213dea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:38.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.976+0000 7fbb288b7640 1 -- 172.21.15.38:0/1202877121 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f882125070 con 0x55f882084240 2024-09-21T03:43:38.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.977+0000 7fbb288b7640 1 --2- 172.21.15.38:0/1202877121 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f882084240 0x55f882124770 secure :-1 s=READY pgs=2068 cs=0 l=1 rev1=1 crypto rx=0x7fbb1800e890 tx=0x7fbb1800e8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:38.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.977+0000 7fbb260b2640 1 -- 172.21.15.38:0/1202877121 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb1800fde0 con 0x55f882084240 2024-09-21T03:43:38.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.977+0000 7fbb260b2640 1 -- 172.21.15.38:0/1202877121 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbb18021420 con 0x55f882084240 2024-09-21T03:43:38.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.977+0000 7fbb260b2640 1 -- 172.21.15.38:0/1202877121 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb18021870 con 0x55f882084240 2024-09-21T03:43:38.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.977+0000 7fbb2f71cf40 1 -- 172.21.15.38:0/1202877121 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55f882125350 con 0x55f882084240 2024-09-21T03:43:38.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.977+0000 7fbb2f71cf40 1 -- 172.21.15.38:0/1202877121 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f882125910 con 0x55f882084240 2024-09-21T03:43:38.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.979+0000 7fbb260b2640 1 -- 172.21.15.38:0/1202877121 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fbb1802a430 con 0x55f882084240 2024-09-21T03:43:38.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.979+0000 7fbb260b2640 1 --2- 172.21.15.38:0/1202877121 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fbb14077570 0x7fbb14079a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:38.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.979+0000 7fbb260b2640 1 -- 172.21.15.38:0/1202877121 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 9142+0+0 (secure 0 0 0) 0x7fbb1809b9f0 con 0x55f882084240 2024-09-21T03:43:38.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.979+0000 7fbb280b6640 1 --2- 172.21.15.38:0/1202877121 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fbb14077570 0x7fbb14079a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:38.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.979+0000 7fbb2f71cf40 1 --2- 172.21.15.38:0/1202877121 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x55f882151ee0 0x55f8821f2f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:38.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.979+0000 7fbb2f71cf40 1 -- 172.21.15.38:0/1202877121 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:1 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=103b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55f88213e3e0 con 0x55f882151ee0 2024-09-21T03:43:38.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.980+0000 7fbb290b8640 1 --2- 172.21.15.38:0/1202877121 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x55f882151ee0 0x55f8821f2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:38.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.980+0000 7fbb280b6640 1 --2- 172.21.15.38:0/1202877121 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fbb14077570 0x7fbb14079a80 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x55f88213eb50 tx=0x7fbb1c00d040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:38.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.980+0000 7fbb290b8640 1 --2- 172.21.15.38:0/1202877121 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x55f882151ee0 0x55f8821f2f00 crc :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:38.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.981+0000 7fbb290b8640 1 -- 172.21.15.38:0/1202877121 <== osd.1 v2:172.21.15.38:6816/1491818732 1 ==== osd_op_reply(1 1.00000000 [omap-set-vals] v129'414 uv414 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fbb2000b040 con 0x55f882151ee0 2024-09-21T03:43:38.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.981+0000 7fbb2f71cf40 1 -- 172.21.15.38:0/1202877121 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x55f882151ee0 msgr2=0x55f8821f2f00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:38.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.981+0000 7fbb2f71cf40 1 --2- 172.21.15.38:0/1202877121 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x55f882151ee0 0x55f8821f2f00 crc :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:38.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.981+0000 7fbb2f71cf40 1 -- 172.21.15.38:0/1202877121 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fbb14077570 msgr2=0x7fbb14079a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:38.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.981+0000 7fbb2f71cf40 1 --2- 172.21.15.38:0/1202877121 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fbb14077570 0x7fbb14079a80 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x55f88213eb50 tx=0x7fbb1c00d040 comp rx=0 tx=0).stop 2024-09-21T03:43:38.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.981+0000 7fbb2f71cf40 1 -- 172.21.15.38:0/1202877121 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f882084240 msgr2=0x55f882124770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:38.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.981+0000 7fbb2f71cf40 1 --2- 172.21.15.38:0/1202877121 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f882084240 0x55f882124770 secure :-1 s=READY pgs=2068 cs=0 l=1 rev1=1 crypto rx=0x7fbb1800e890 tx=0x7fbb1800e8c0 comp rx=0 tx=0).stop 2024-09-21T03:43:38.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.981+0000 7fbb290b8640 1 -- 172.21.15.38:0/1202877121 reap_dead start 2024-09-21T03:43:38.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.981+0000 7fbb2f71cf40 1 -- 172.21.15.38:0/1202877121 shutdown_connections 2024-09-21T03:43:38.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.981+0000 7fbb2f71cf40 1 -- 172.21.15.38:0/1202877121 >> 172.21.15.38:0/1202877121 conn(0x55f881fa2ac0 msgr2=0x55f8820c99e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:38.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.981+0000 7fbb2f71cf40 1 -- 172.21.15.38:0/1202877121 shutdown_connections 2024-09-21T03:43:38.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:38.981+0000 7fbb2f71cf40 1 -- 172.21.15.38:0/1202877121 wait complete. 2024-09-21T03:43:38.987 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-21T03:43:39.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:39.104+0000 7f409d4e5640 1 -- 172.21.15.38:0/1687848999 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f409815c740 msgr2=0x7f40980b3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:39.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:39.104+0000 7f409d4e5640 1 --2- 172.21.15.38:0/1687848999 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f409815c740 0x7f40980b3740 secure :-1 s=READY pgs=2069 cs=0 l=1 rev1=1 crypto rx=0x7f40980733a0 tx=0x7f408802a5f0 comp rx=0 tx=0).stop 2024-09-21T03:43:39.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:39.104+0000 7f409d4e5640 1 -- 172.21.15.38:0/1687848999 shutdown_connections 2024-09-21T03:43:39.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:39.104+0000 7f409d4e5640 1 --2- 172.21.15.38:0/1687848999 >> v2:172.21.15.195:3300/0 conn(0x7f40980740f0 0x7f4098074500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:39.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:39.104+0000 7f409d4e5640 1 --2- 172.21.15.38:0/1687848999 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f409815c740 0x7f40980b3740 unknown :-1 s=CLOSED pgs=2069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:39.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:39.104+0000 7f409d4e5640 1 -- 172.21.15.38:0/1687848999 >> 172.21.15.38:0/1687848999 conn(0x7f40980735d0 msgr2=0x7f40980739e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:39.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:39.104+0000 7f409d4e5640 1 -- 172.21.15.38:0/1687848999 shutdown_connections 2024-09-21T03:43:39.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:39.104+0000 7f409d4e5640 1 -- 172.21.15.38:0/1687848999 wait complete. 2024-09-21T03:43:39.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:39.106+0000 7f409d4e5640 1 Processor -- start 2024-09-21T03:43:39.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:39.106+0000 7f409d4e5640 1 -- start start 2024-09-21T03:43:39.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:39.108+0000 7f409d4e5640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f409806ddf0 0x7f40980ae9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:39.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:39.108+0000 7f409d4e5640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f40980740f0 0x7f40981e9260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:39.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:39.108+0000 7f409d4e5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f409815c740 0x7f40981e97a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:39.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:39.108+0000 7f409d4e5640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f40980cfbf0 con 0x7f409815c740 2024-09-21T03:43:39.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:39.108+0000 7f409d4e5640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f40980cfa70 con 0x7f40980740f0 2024-09-21T03:43:39.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:39.108+0000 7f409d4e5640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f40980cfd70 con 0x7f409806ddf0 2024-09-21T03:43:39.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:39.108+0000 7f40977fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f409815c740 0x7f40981e97a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:39.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:39.108+0000 7f40977fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f409815c740 0x7f40981e97a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49862/0 (socket says 172.21.15.38:49862) 2024-09-21T03:43:39.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:39.108+0000 7f4096ffd640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f409806ddf0 0x7f40980ae9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:39.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:39.108+0000 7f40977fe640 1 -- 172.21.15.38:0/846754460 learned_addr learned my addr 172.21.15.38:0/846754460 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:43:39.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:39.108+0000 7f40967fc640 1 --2- 172.21.15.38:0/846754460 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f40980740f0 0x7f40981e9260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:39.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:39.108+0000 7f40977fe640 1 -- 172.21.15.38:0/846754460 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f409806ddf0 msgr2=0x7f40980ae9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:39.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:39.108+0000 7f40977fe640 1 --2- 172.21.15.38:0/846754460 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f409806ddf0 0x7f40980ae9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:39.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:39.108+0000 7f40977fe640 1 -- 172.21.15.38:0/846754460 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f40980740f0 msgr2=0x7f40981e9260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:39.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:39.108+0000 7f40977fe640 1 --2- 172.21.15.38:0/846754460 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f40980740f0 0x7f40981e9260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:39.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:39.108+0000 7f40977fe640 1 -- 172.21.15.38:0/846754460 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40980aef10 con 0x7f409815c740 2024-09-21T03:43:39.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:39.108+0000 7f40977fe640 1 --2- 172.21.15.38:0/846754460 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f409815c740 0x7f40981e97a0 secure :-1 s=READY pgs=2070 cs=0 l=1 rev1=1 crypto rx=0x7f409000c730 tx=0x7f409000cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:39.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:39.109+0000 7f407f7fe640 1 -- 172.21.15.38:0/846754460 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4090003b70 con 0x7f409815c740 2024-09-21T03:43:39.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:39.109+0000 7f407f7fe640 1 -- 172.21.15.38:0/846754460 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4090003d10 con 0x7f409815c740 2024-09-21T03:43:39.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:39.109+0000 7f409d4e5640 1 -- 172.21.15.38:0/846754460 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40981f5c90 con 0x7f409815c740 2024-09-21T03:43:39.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:39.109+0000 7f407f7fe640 1 -- 172.21.15.38:0/846754460 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4090005330 con 0x7f409815c740 2024-09-21T03:43:39.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:39.109+0000 7f409d4e5640 1 -- 172.21.15.38:0/846754460 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40981f6250 con 0x7f409815c740 2024-09-21T03:43:39.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:39.110+0000 7f409d4e5640 1 -- 172.21.15.38:0/846754460 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40980d0220 con 0x7f409815c740 2024-09-21T03:43:39.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:39.111+0000 7f407f7fe640 1 -- 172.21.15.38:0/846754460 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f40900054d0 con 0x7f409815c740 2024-09-21T03:43:39.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:39.111+0000 7f407f7fe640 1 --2- 172.21.15.38:0/846754460 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4084078b90 0x7f408407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:39.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:39.111+0000 7f407f7fe640 1 -- 172.21.15.38:0/846754460 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 9142+0+0 (secure 0 0 0) 0x7f409009b6b0 con 0x7f409815c740 2024-09-21T03:43:39.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:39.111+0000 7f4096ffd640 1 --2- 172.21.15.38:0/846754460 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4084078b90 0x7f408407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:39.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:39.112+0000 7f4096ffd640 1 --2- 172.21.15.38:0/846754460 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4084078b90 0x7f408407b0a0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f4088033000 tx=0x7f4088004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:39.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:39.116+0000 7f407f7fe640 1 -- 172.21.15.38:0/846754460 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f40980aef10 con 0x7f409815c740 2024-09-21T03:43:39.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:39.344+0000 7f409d4e5640 1 -- 172.21.15.38:0/846754460 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f40981f5a20 con 0x7f409815c740 2024-09-21T03:43:39.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:39.634+0000 7f407f7fe640 1 -- 172.21.15.38:0/846754460 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v285) ==== 162+0+0 (secure 0 0 0) 0x7f4090062770 con 0x7f409815c740 2024-09-21T03:43:39.637 INFO:teuthology.orchestra.run.smithi038.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-21T03:43:39.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:39.636+0000 7f409d4e5640 1 -- 172.21.15.38:0/846754460 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4084078b90 msgr2=0x7f408407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:39.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:39.636+0000 7f409d4e5640 1 --2- 172.21.15.38:0/846754460 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4084078b90 0x7f408407b0a0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f4088033000 tx=0x7f4088004730 comp rx=0 tx=0).stop 2024-09-21T03:43:39.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:39.636+0000 7f409d4e5640 1 -- 172.21.15.38:0/846754460 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f409815c740 msgr2=0x7f40981e97a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:39.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:39.636+0000 7f409d4e5640 1 --2- 172.21.15.38:0/846754460 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f409815c740 0x7f40981e97a0 secure :-1 s=READY pgs=2070 cs=0 l=1 rev1=1 crypto rx=0x7f409000c730 tx=0x7f409000cb30 comp rx=0 tx=0).stop 2024-09-21T03:43:39.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:39.636+0000 7f409d4e5640 1 -- 172.21.15.38:0/846754460 shutdown_connections 2024-09-21T03:43:39.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:39.636+0000 7f409d4e5640 1 --2- 172.21.15.38:0/846754460 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f409815c740 0x7f40981e97a0 unknown :-1 s=CLOSED pgs=2070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:39.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:39.636+0000 7f409d4e5640 1 --2- 172.21.15.38:0/846754460 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f40980740f0 0x7f40981e9260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:39.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:39.636+0000 7f409d4e5640 1 --2- 172.21.15.38:0/846754460 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4084078b90 0x7f408407b0a0 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:39.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:39.636+0000 7f409d4e5640 1 --2- 172.21.15.38:0/846754460 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f409806ddf0 0x7f40980ae9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:39.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:39.636+0000 7f409d4e5640 1 -- 172.21.15.38:0/846754460 >> 172.21.15.38:0/846754460 conn(0x7f40980735d0 msgr2=0x7f40980b4210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:39.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:39.636+0000 7f409d4e5640 1 -- 172.21.15.38:0/846754460 shutdown_connections 2024-09-21T03:43:39.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:39.636+0000 7f409d4e5640 1 -- 172.21.15.38:0/846754460 wait complete. 2024-09-21T03:43:40.055 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:43:40.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:40.180+0000 7fd57fbe6640 1 -- 172.21.15.38:0/2113422606 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd5781536e0 msgr2=0x7fd5780d17c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:40.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:40.180+0000 7fd57fbe6640 1 --2- 172.21.15.38:0/2113422606 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd5781536e0 0x7fd5780d17c0 secure :-1 s=READY pgs=2071 cs=0 l=1 rev1=1 crypto rx=0x7fd5780b2db0 tx=0x7fd57402a5f0 comp rx=0 tx=0).stop 2024-09-21T03:43:40.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:40.181+0000 7fd57fbe6640 1 -- 172.21.15.38:0/2113422606 shutdown_connections 2024-09-21T03:43:40.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:40.181+0000 7fd57fbe6640 1 --2- 172.21.15.38:0/2113422606 >> v2:172.21.15.195:3300/0 conn(0x7fd5780cd040 0x7fd5780d04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:40.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:40.181+0000 7fd57fbe6640 1 --2- 172.21.15.38:0/2113422606 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd5781536e0 0x7fd5780d17c0 unknown :-1 s=CLOSED pgs=2071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:40.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:40.181+0000 7fd57fbe6640 1 -- 172.21.15.38:0/2113422606 >> 172.21.15.38:0/2113422606 conn(0x7fd5780cef40 msgr2=0x7fd5780cf350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:40.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:40.181+0000 7fd57fbe6640 1 -- 172.21.15.38:0/2113422606 shutdown_connections 2024-09-21T03:43:40.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:40.181+0000 7fd57fbe6640 1 -- 172.21.15.38:0/2113422606 wait complete. 2024-09-21T03:43:40.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:40.182+0000 7fd57fbe6640 1 Processor -- start 2024-09-21T03:43:40.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:40.182+0000 7fd57fbe6640 1 -- start start 2024-09-21T03:43:40.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:40.184+0000 7fd57fbe6640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd5780d1d00 0x7fd57814d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:40.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:40.184+0000 7fd57fbe6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd5781536e0 0x7fd57814d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:40.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:40.184+0000 7fd57fbe6640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd5781f6520 0x7fd5781f6930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:40.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:40.184+0000 7fd57fbe6640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fd5780d0d90 con 0x7fd5781536e0 2024-09-21T03:43:40.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:40.184+0000 7fd57fbe6640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd5780d0c10 con 0x7fd5781f6520 2024-09-21T03:43:40.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:40.184+0000 7fd57fbe6640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fd5780d0f10 con 0x7fd5780d1d00 2024-09-21T03:43:40.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:40.184+0000 7fd57d181640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd5781536e0 0x7fd57814d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:40.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:40.184+0000 7fd57d181640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd5781536e0 0x7fd57814d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49874/0 (socket says 172.21.15.38:49874) 2024-09-21T03:43:40.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:40.184+0000 7fd57d181640 1 -- 172.21.15.38:0/1770183770 learned_addr learned my addr 172.21.15.38:0/1770183770 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:43:40.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:40.184+0000 7fd57e183640 1 --2- 172.21.15.38:0/1770183770 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd5781f6520 0x7fd5781f6930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:40.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:40.184+0000 7fd57d982640 1 --2- 172.21.15.38:0/1770183770 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd5780d1d00 0x7fd57814d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:40.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:40.184+0000 7fd57d181640 1 -- 172.21.15.38:0/1770183770 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd5780d1d00 msgr2=0x7fd57814d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:40.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:40.184+0000 7fd57d181640 1 --2- 172.21.15.38:0/1770183770 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd5780d1d00 0x7fd57814d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:40.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:40.184+0000 7fd57d181640 1 -- 172.21.15.38:0/1770183770 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd5781f6520 msgr2=0x7fd5781f6930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:40.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:40.184+0000 7fd57d181640 1 --2- 172.21.15.38:0/1770183770 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd5781f6520 0x7fd5781f6930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:40.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:40.184+0000 7fd57d181640 1 -- 172.21.15.38:0/1770183770 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5781f6e70 con 0x7fd5781536e0 2024-09-21T03:43:40.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:40.184+0000 7fd57d181640 1 --2- 172.21.15.38:0/1770183770 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd5781536e0 0x7fd57814d970 secure :-1 s=READY pgs=2072 cs=0 l=1 rev1=1 crypto rx=0x7fd56800c400 tx=0x7fd56800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:40.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:40.185+0000 7fd56e7fc640 1 -- 172.21.15.38:0/1770183770 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd568017070 con 0x7fd5781536e0 2024-09-21T03:43:40.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:40.185+0000 7fd56e7fc640 1 -- 172.21.15.38:0/1770183770 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd5680053c0 con 0x7fd5781536e0 2024-09-21T03:43:40.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:40.185+0000 7fd57fbe6640 1 -- 172.21.15.38:0/1770183770 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd57814e1d0 con 0x7fd5781536e0 2024-09-21T03:43:40.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:40.185+0000 7fd56e7fc640 1 -- 172.21.15.38:0/1770183770 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd568003850 con 0x7fd5781536e0 2024-09-21T03:43:40.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:40.185+0000 7fd57fbe6640 1 -- 172.21.15.38:0/1770183770 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5780b8a10 con 0x7fd5781536e0 2024-09-21T03:43:40.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:40.186+0000 7fd57fbe6640 1 -- 172.21.15.38:0/1770183770 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd57814df30 con 0x7fd5781536e0 2024-09-21T03:43:40.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:40.186+0000 7fd56e7fc640 1 -- 172.21.15.38:0/1770183770 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fd56801a070 con 0x7fd5781536e0 2024-09-21T03:43:40.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:40.187+0000 7fd56e7fc640 1 --2- 172.21.15.38:0/1770183770 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd55c078bc0 0x7fd55c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:40.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:40.187+0000 7fd56e7fc640 1 -- 172.21.15.38:0/1770183770 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 9142+0+0 (secure 0 0 0) 0x7fd56809ad30 con 0x7fd5781536e0 2024-09-21T03:43:40.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:40.187+0000 7fd57d982640 1 --2- 172.21.15.38:0/1770183770 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd55c078bc0 0x7fd55c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:40.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:40.188+0000 7fd57d982640 1 --2- 172.21.15.38:0/1770183770 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd55c078bc0 0x7fd55c07b0d0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fd574033000 tx=0x7fd574004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:40.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:40.192+0000 7fd56e7fc640 1 -- 172.21.15.38:0/1770183770 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fd57814df30 con 0x7fd5781536e0 2024-09-21T03:43:40.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:40.419+0000 7fd57fbe6640 1 -- 172.21.15.38:0/1770183770 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd5780d04c0 con 0x7fd5781536e0 2024-09-21T03:43:40.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:40.420+0000 7fd56e7fc640 1 -- 172.21.15.38:0/1770183770 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 286 v286) ==== 95+0+5224 (secure 0 0 0) 0x7fd5781f6e70 con 0x7fd5781536e0 2024-09-21T03:43:40.423 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:43:40.423 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":286,"btime":"2024-09-21T03:43:39:642693+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6086,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/965972663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":965972663},{"type":"v1","addr":"172.21.15.195:6836","nonce":965972663}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6700,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/3176122334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":3176122334},{"type":"v1","addr":"172.21.15.195:6837","nonce":3176122334}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":9840,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/3801772046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":3801772046},{"type":"v1","addr":"172.21.15.38:6835","nonce":3801772046}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":273}],"filesystems":[{"mdsmap":{"epoch":286,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:40:02.319282+0000","modified":"2024-09-21T03:43:39.642680+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":129,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9999},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9999":{"gid":9999,"name":"c","rank":0,"incarnation":286,"state":"up:replay","state_seq":1,"addr":"172.21.15.38:6833/626794495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":626794495},{"type":"v1","addr":"172.21.15.38:6833","nonce":626794495}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-21T03:43:40.423 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 286 2024-09-21T03:43:40.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:40.422+0000 7fd57fbe6640 1 -- 172.21.15.38:0/1770183770 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd55c078bc0 msgr2=0x7fd55c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:40.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:40.422+0000 7fd57fbe6640 1 --2- 172.21.15.38:0/1770183770 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd55c078bc0 0x7fd55c07b0d0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fd574033000 tx=0x7fd574004730 comp rx=0 tx=0).stop 2024-09-21T03:43:40.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:40.422+0000 7fd57fbe6640 1 -- 172.21.15.38:0/1770183770 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd5781536e0 msgr2=0x7fd57814d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:40.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:40.422+0000 7fd57fbe6640 1 --2- 172.21.15.38:0/1770183770 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd5781536e0 0x7fd57814d970 secure :-1 s=READY pgs=2072 cs=0 l=1 rev1=1 crypto rx=0x7fd56800c400 tx=0x7fd56800c8c0 comp rx=0 tx=0).stop 2024-09-21T03:43:40.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:40.422+0000 7fd57fbe6640 1 -- 172.21.15.38:0/1770183770 shutdown_connections 2024-09-21T03:43:40.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:40.422+0000 7fd57fbe6640 1 --2- 172.21.15.38:0/1770183770 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd5781f6520 0x7fd5781f6930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:40.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:40.422+0000 7fd57fbe6640 1 --2- 172.21.15.38:0/1770183770 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd5781536e0 0x7fd57814d970 unknown :-1 s=CLOSED pgs=2072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:40.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:40.422+0000 7fd57fbe6640 1 --2- 172.21.15.38:0/1770183770 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd55c078bc0 0x7fd55c07b0d0 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:40.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:40.422+0000 7fd57fbe6640 1 --2- 172.21.15.38:0/1770183770 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd5780d1d00 0x7fd57814d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:40.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:40.422+0000 7fd57fbe6640 1 -- 172.21.15.38:0/1770183770 >> 172.21.15.38:0/1770183770 conn(0x7fd5780cef40 msgr2=0x7fd5780cca80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:40.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:40.422+0000 7fd57fbe6640 1 -- 172.21.15.38:0/1770183770 shutdown_connections 2024-09-21T03:43:40.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:40.422+0000 7fd57fbe6640 1 -- 172.21.15.38:0/1770183770 wait complete. 2024-09-21T03:43:40.437 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 286, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T03:40:02.319282+0000', 'modified': '2024-09-21T03:43:39.642680+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 129, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9999}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9999': {'gid': 9999, 'name': 'c', 'rank': 0, 'incarnation': 286, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.38:6833/626794495', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.38:6832', 'nonce': 626794495}, {'type': 'v1', 'addr': '172.21.15.38:6833', 'nonce': 626794495}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-09-21T03:43:40.437 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9999:up:replay 2024-09-21T03:43:40.437 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2024-09-21T03:43:43.627 INFO:tasks.ceph.mds.c.smithi038.stderr:2024-09-21T03:43:43.625+0000 7ff2466da640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T03:43:45.673 INFO:tasks.ceph.mon.a.smithi038.stderr:2024-09-21T03:43:45.671+0000 7f3f47f18640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-09-21T03:43:45.911 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:43:46.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.040+0000 7f0edbf5f640 1 -- 172.21.15.38:0/2556796430 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0ed4162590 msgr2=0x7f0ed41629a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:46.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.040+0000 7f0edbf5f640 1 --2- 172.21.15.38:0/2556796430 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0ed4162590 0x7f0ed41629a0 secure :-1 s=READY pgs=2074 cs=0 l=1 rev1=1 crypto rx=0x7f0ec4009860 tx=0x7f0ec402a540 comp rx=0 tx=0).stop 2024-09-21T03:43:46.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.040+0000 7f0edbf5f640 1 -- 172.21.15.38:0/2556796430 shutdown_connections 2024-09-21T03:43:46.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.040+0000 7f0edbf5f640 1 --2- 172.21.15.38:0/2556796430 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0ed40b2e60 0x7f0ed40b3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:46.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.040+0000 7f0edbf5f640 1 --2- 172.21.15.38:0/2556796430 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0ed4162590 0x7f0ed41629a0 unknown :-1 s=CLOSED pgs=2074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:46.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.040+0000 7f0edbf5f640 1 -- 172.21.15.38:0/2556796430 >> 172.21.15.38:0/2556796430 conn(0x7f0ed40d4440 msgr2=0x7f0ed40d4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:46.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.040+0000 7f0edbf5f640 1 -- 172.21.15.38:0/2556796430 shutdown_connections 2024-09-21T03:43:46.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.040+0000 7f0edbf5f640 1 -- 172.21.15.38:0/2556796430 wait complete. 2024-09-21T03:43:46.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.041+0000 7f0edbf5f640 1 Processor -- start 2024-09-21T03:43:46.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.042+0000 7f0edbf5f640 1 -- start start 2024-09-21T03:43:46.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.043+0000 7f0edbf5f640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0ed40b2e60 0x7f0ed415c320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:46.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.043+0000 7f0edbf5f640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0ed40cfe00 0x7f0ed415e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:46.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.043+0000 7f0edbf5f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0ed4162590 0x7f0ed41f5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:46.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.043+0000 7f0edbf5f640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f0ed40b3b40 con 0x7f0ed4162590 2024-09-21T03:43:46.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.043+0000 7f0edbf5f640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f0ed40b39c0 con 0x7f0ed40cfe00 2024-09-21T03:43:46.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.043+0000 7f0edbf5f640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f0ed40cf590 con 0x7f0ed40b2e60 2024-09-21T03:43:46.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.043+0000 7f0eda4fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0ed4162590 0x7f0ed41f5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:46.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.043+0000 7f0eda4fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0ed4162590 0x7f0ed41f5300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54470/0 (socket says 172.21.15.38:54470) 2024-09-21T03:43:46.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.043+0000 7f0eda4fc640 1 -- 172.21.15.38:0/1645080015 learned_addr learned my addr 172.21.15.38:0/1645080015 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:43:46.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.043+0000 7f0ed9cfb640 1 --2- 172.21.15.38:0/1645080015 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0ed40b2e60 0x7f0ed415c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:46.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.043+0000 7f0ed94fa640 1 --2- 172.21.15.38:0/1645080015 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0ed40cfe00 0x7f0ed415e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:46.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.043+0000 7f0eda4fc640 1 -- 172.21.15.38:0/1645080015 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0ed40b2e60 msgr2=0x7f0ed415c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:46.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.043+0000 7f0eda4fc640 1 --2- 172.21.15.38:0/1645080015 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0ed40b2e60 0x7f0ed415c320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:46.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.043+0000 7f0eda4fc640 1 -- 172.21.15.38:0/1645080015 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0ed40cfe00 msgr2=0x7f0ed415e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:46.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.043+0000 7f0eda4fc640 1 --2- 172.21.15.38:0/1645080015 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0ed40cfe00 0x7f0ed415e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:46.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.043+0000 7f0eda4fc640 1 -- 172.21.15.38:0/1645080015 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ed415eaa0 con 0x7f0ed4162590 2024-09-21T03:43:46.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.044+0000 7f0eda4fc640 1 --2- 172.21.15.38:0/1645080015 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0ed4162590 0x7f0ed41f5300 secure :-1 s=READY pgs=2075 cs=0 l=1 rev1=1 crypto rx=0x7f0ecc00e960 tx=0x7f0ecc00ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:46.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.044+0000 7f0eca7fc640 1 -- 172.21.15.38:0/1645080015 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ecc004d20 con 0x7f0ed4162590 2024-09-21T03:43:46.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.044+0000 7f0eca7fc640 1 -- 172.21.15.38:0/1645080015 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0ecc00cb20 con 0x7f0ed4162590 2024-09-21T03:43:46.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.044+0000 7f0edbf5f640 1 -- 172.21.15.38:0/1645080015 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ed41f5f30 con 0x7f0ed4162590 2024-09-21T03:43:46.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.044+0000 7f0eca7fc640 1 -- 172.21.15.38:0/1645080015 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ecc005020 con 0x7f0ed4162590 2024-09-21T03:43:46.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.044+0000 7f0edbf5f640 1 -- 172.21.15.38:0/1645080015 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ed415b840 con 0x7f0ed4162590 2024-09-21T03:43:46.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.045+0000 7f0edbf5f640 1 -- 172.21.15.38:0/1645080015 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ed415bf60 con 0x7f0ed4162590 2024-09-21T03:43:46.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.046+0000 7f0eca7fc640 1 -- 172.21.15.38:0/1645080015 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f0ecc0051c0 con 0x7f0ed4162590 2024-09-21T03:43:46.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.046+0000 7f0eca7fc640 1 --2- 172.21.15.38:0/1645080015 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0eb8078b90 0x7f0eb807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:46.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.046+0000 7f0eca7fc640 1 -- 172.21.15.38:0/1645080015 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 9142+0+0 (secure 0 0 0) 0x7f0ecc09c270 con 0x7f0ed4162590 2024-09-21T03:43:46.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.046+0000 7f0ed9cfb640 1 --2- 172.21.15.38:0/1645080015 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0eb8078b90 0x7f0eb807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:46.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.047+0000 7f0ed9cfb640 1 --2- 172.21.15.38:0/1645080015 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0eb8078b90 0x7f0eb807b0a0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f0ed0001720 tx=0x7f0ed0007e90 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:46.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.051+0000 7f0eca7fc640 1 -- 172.21.15.38:0/1645080015 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f0ecc016030 con 0x7f0ed4162590 2024-09-21T03:43:46.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.277+0000 7f0edbf5f640 1 -- 172.21.15.38:0/1645080015 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0ed40b3270 con 0x7f0ed4162590 2024-09-21T03:43:46.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.278+0000 7f0eca7fc640 1 -- 172.21.15.38:0/1645080015 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 290 v290) ==== 95+0+5231 (secure 0 0 0) 0x7f0ecc0633c0 con 0x7f0ed4162590 2024-09-21T03:43:46.279 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:43:46.280 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":290,"btime":"2024-09-21T03:43:45:672563+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6086,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/965972663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":965972663},{"type":"v1","addr":"172.21.15.195:6836","nonce":965972663}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6700,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/3176122334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":3176122334},{"type":"v1","addr":"172.21.15.195:6837","nonce":3176122334}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":9840,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/3801772046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":3801772046},{"type":"v1","addr":"172.21.15.38:6835","nonce":3801772046}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":273}],"filesystems":[{"mdsmap":{"epoch":290,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:40:02.319282+0000","modified":"2024-09-21T03:43:45.657079+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":129,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9999},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9999":{"gid":9999,"name":"c","rank":0,"incarnation":286,"state":"up:active","state_seq":5,"addr":"172.21.15.38:6833/626794495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":626794495},{"type":"v1","addr":"172.21.15.38:6833","nonce":626794495}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9999,"qdb_cluster":[9999]},"id":6}]} 2024-09-21T03:43:46.280 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 290 2024-09-21T03:43:46.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.280+0000 7f0edbf5f640 1 -- 172.21.15.38:0/1645080015 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0eb8078b90 msgr2=0x7f0eb807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:46.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.280+0000 7f0edbf5f640 1 --2- 172.21.15.38:0/1645080015 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0eb8078b90 0x7f0eb807b0a0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f0ed0001720 tx=0x7f0ed0007e90 comp rx=0 tx=0).stop 2024-09-21T03:43:46.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.280+0000 7f0edbf5f640 1 -- 172.21.15.38:0/1645080015 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0ed4162590 msgr2=0x7f0ed41f5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:46.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.280+0000 7f0edbf5f640 1 --2- 172.21.15.38:0/1645080015 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0ed4162590 0x7f0ed41f5300 secure :-1 s=READY pgs=2075 cs=0 l=1 rev1=1 crypto rx=0x7f0ecc00e960 tx=0x7f0ecc00ee20 comp rx=0 tx=0).stop 2024-09-21T03:43:46.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.280+0000 7f0edbf5f640 1 -- 172.21.15.38:0/1645080015 shutdown_connections 2024-09-21T03:43:46.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.280+0000 7f0edbf5f640 1 --2- 172.21.15.38:0/1645080015 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0ed4162590 0x7f0ed41f5300 unknown :-1 s=CLOSED pgs=2075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:46.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.280+0000 7f0edbf5f640 1 --2- 172.21.15.38:0/1645080015 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0ed40cfe00 0x7f0ed415e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:46.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.280+0000 7f0edbf5f640 1 --2- 172.21.15.38:0/1645080015 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0eb8078b90 0x7f0eb807b0a0 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:46.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.280+0000 7f0edbf5f640 1 --2- 172.21.15.38:0/1645080015 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0ed40b2e60 0x7f0ed415c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:46.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.280+0000 7f0edbf5f640 1 -- 172.21.15.38:0/1645080015 >> 172.21.15.38:0/1645080015 conn(0x7f0ed40d4440 msgr2=0x7f0ed40d4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:46.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.280+0000 7f0edbf5f640 1 -- 172.21.15.38:0/1645080015 shutdown_connections 2024-09-21T03:43:46.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.280+0000 7f0edbf5f640 1 -- 172.21.15.38:0/1645080015 wait complete. 2024-09-21T03:43:46.294 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 290, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T03:40:02.319282+0000', 'modified': '2024-09-21T03:43:45.657079+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 129, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9999}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9999': {'gid': 9999, 'name': 'c', 'rank': 0, 'incarnation': 286, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.38:6833/626794495', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.38:6832', 'nonce': 626794495}, {'type': 'v1', 'addr': '172.21.15.38:6833', 'nonce': 626794495}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 9999, 'qdb_cluster': [9999]} 2024-09-21T03:43:46.294 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-21T03:43:46.294 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-21T03:43:46.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.409+0000 7f502076e640 1 -- 172.21.15.38:0/3178278007 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f50180b5260 msgr2=0x7f501815ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:46.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.409+0000 7f502076e640 1 --2- 172.21.15.38:0/3178278007 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f50180b5260 0x7f501815ba80 secure :-1 s=READY pgs=2076 cs=0 l=1 rev1=1 crypto rx=0x7f5008005fd0 tx=0x7f500802a5d0 comp rx=0 tx=0).stop 2024-09-21T03:43:46.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.409+0000 7f502076e640 1 -- 172.21.15.38:0/3178278007 shutdown_connections 2024-09-21T03:43:46.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.409+0000 7f502076e640 1 --2- 172.21.15.38:0/3178278007 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f501815bfc0 0x7f501815f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:46.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.409+0000 7f502076e640 1 --2- 172.21.15.38:0/3178278007 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f50180b5260 0x7f501815ba80 unknown :-1 s=CLOSED pgs=2076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:46.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.409+0000 7f502076e640 1 -- 172.21.15.38:0/3178278007 >> 172.21.15.38:0/3178278007 conn(0x7f50180cdca0 msgr2=0x7f50180ce0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:46.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.409+0000 7f502076e640 1 -- 172.21.15.38:0/3178278007 shutdown_connections 2024-09-21T03:43:46.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.409+0000 7f502076e640 1 -- 172.21.15.38:0/3178278007 wait complete. 2024-09-21T03:43:46.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.411+0000 7f502076e640 1 Processor -- start 2024-09-21T03:43:46.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.411+0000 7f502076e640 1 -- start start 2024-09-21T03:43:46.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.412+0000 7f502076e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f501806def0 0x7f501809a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:46.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.412+0000 7f502076e640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f50180b5260 0x7f5018155700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:46.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.412+0000 7f502076e640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f501815bfc0 0x7f501809d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:46.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.412+0000 7f502076e640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f501815ff80 con 0x7f501806def0 2024-09-21T03:43:46.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.412+0000 7f502076e640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f501815fe00 con 0x7f501815bfc0 2024-09-21T03:43:46.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.412+0000 7f502076e640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f5018071300 con 0x7f50180b5260 2024-09-21T03:43:46.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.412+0000 7f501e50a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f501806def0 0x7f501809a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:46.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.412+0000 7f501e50a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f501806def0 0x7f501809a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54480/0 (socket says 172.21.15.38:54480) 2024-09-21T03:43:46.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.412+0000 7f501e50a640 1 -- 172.21.15.38:0/2513722468 learned_addr learned my addr 172.21.15.38:0/2513722468 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:43:46.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.412+0000 7f501ed0b640 1 --2- 172.21.15.38:0/2513722468 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f501815bfc0 0x7f501809d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:46.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.412+0000 7f501dd09640 1 --2- 172.21.15.38:0/2513722468 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f50180b5260 0x7f5018155700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:46.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.413+0000 7f501e50a640 1 -- 172.21.15.38:0/2513722468 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f50180b5260 msgr2=0x7f5018155700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:46.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.413+0000 7f501e50a640 1 --2- 172.21.15.38:0/2513722468 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f50180b5260 0x7f5018155700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:46.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.413+0000 7f501e50a640 1 -- 172.21.15.38:0/2513722468 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f501815bfc0 msgr2=0x7f501809d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:46.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.413+0000 7f501e50a640 1 --2- 172.21.15.38:0/2513722468 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f501815bfc0 0x7f501809d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:46.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.413+0000 7f501e50a640 1 -- 172.21.15.38:0/2513722468 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5018155c40 con 0x7f501806def0 2024-09-21T03:43:46.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.413+0000 7f501e50a640 1 --2- 172.21.15.38:0/2513722468 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f501806def0 0x7f501809a9e0 secure :-1 s=READY pgs=2077 cs=0 l=1 rev1=1 crypto rx=0x7f501400c370 tx=0x7f501400c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:46.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.413+0000 7f500effd640 1 -- 172.21.15.38:0/2513722468 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5014017070 con 0x7f501806def0 2024-09-21T03:43:46.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.413+0000 7f500effd640 1 -- 172.21.15.38:0/2513722468 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f50140053c0 con 0x7f501806def0 2024-09-21T03:43:46.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.413+0000 7f502076e640 1 -- 172.21.15.38:0/2513722468 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f501809b400 con 0x7f501806def0 2024-09-21T03:43:46.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.413+0000 7f500effd640 1 -- 172.21.15.38:0/2513722468 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5014003850 con 0x7f501806def0 2024-09-21T03:43:46.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.413+0000 7f502076e640 1 -- 172.21.15.38:0/2513722468 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f501809b9c0 con 0x7f501806def0 2024-09-21T03:43:46.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.415+0000 7f500effd640 1 -- 172.21.15.38:0/2513722468 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f501401a070 con 0x7f501806def0 2024-09-21T03:43:46.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.415+0000 7f500effd640 1 --2- 172.21.15.38:0/2513722468 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4ffc078b90 0x7f4ffc07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:46.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.416+0000 7f500effd640 1 -- 172.21.15.38:0/2513722468 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 9142+0+0 (secure 0 0 0) 0x7f501409aed0 con 0x7f501806def0 2024-09-21T03:43:46.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.416+0000 7f501dd09640 1 --2- 172.21.15.38:0/2513722468 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4ffc078b90 0x7f4ffc07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:46.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.416+0000 7f501dd09640 1 --2- 172.21.15.38:0/2513722468 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4ffc078b90 0x7f4ffc07b0a0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f5008033000 tx=0x7f5008004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:46.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.416+0000 7f50277bd740 1 Processor -- start 2024-09-21T03:43:46.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.417+0000 7f50277bd740 1 -- start start 2024-09-21T03:43:46.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.418+0000 7f50277bd740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x563e86018270 0x563e863e9fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:46.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.418+0000 7f50277bd740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563e85fef3b0 0x563e863ea500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:46.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.418+0000 7f50277bd740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563e85ff7590 0x563e86407400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:46.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.418+0000 7f50277bd740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x563e86332820 con 0x563e85fef3b0 2024-09-21T03:43:46.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.418+0000 7f50277bd740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x563e86321c50 con 0x563e85ff7590 2024-09-21T03:43:46.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.418+0000 7f50277bd740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x563e86325a60 con 0x563e86018270 2024-09-21T03:43:46.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.418+0000 7f501dd09640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563e85fef3b0 0x563e863ea500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:46.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.418+0000 7f501dd09640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563e85fef3b0 0x563e863ea500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54486/0 (socket says 172.21.15.38:54486) 2024-09-21T03:43:46.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.418+0000 7f501dd09640 1 -- 172.21.15.38:0/3092938901 learned_addr learned my addr 172.21.15.38:0/3092938901 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:43:46.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.418+0000 7f501ed0b640 1 --2- 172.21.15.38:0/3092938901 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563e85ff7590 0x563e86407400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:46.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.418+0000 7f501e50a640 1 --2- 172.21.15.38:0/3092938901 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x563e86018270 0x563e863e9fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:46.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.418+0000 7f501dd09640 1 -- 172.21.15.38:0/3092938901 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x563e86018270 msgr2=0x563e863e9fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:46.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.418+0000 7f501dd09640 1 --2- 172.21.15.38:0/3092938901 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x563e86018270 0x563e863e9fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:46.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.418+0000 7f501dd09640 1 -- 172.21.15.38:0/3092938901 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563e85ff7590 msgr2=0x563e86407400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:46.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.418+0000 7f501dd09640 1 --2- 172.21.15.38:0/3092938901 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563e85ff7590 0x563e86407400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:46.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.418+0000 7f501dd09640 1 -- 172.21.15.38:0/3092938901 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e86083850 con 0x563e85fef3b0 2024-09-21T03:43:46.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.419+0000 7f501dd09640 1 --2- 172.21.15.38:0/3092938901 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563e85fef3b0 0x563e863ea500 secure :-1 s=READY pgs=2078 cs=0 l=1 rev1=1 crypto rx=0x7f500802abc0 tx=0x7f500803d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a47588a1aefdabd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:46.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.419+0000 7f4ffbfff640 1 -- 172.21.15.38:0/3092938901 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5008035040 con 0x563e85fef3b0 2024-09-21T03:43:46.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.419+0000 7f4ffbfff640 1 -- 172.21.15.38:0/3092938901 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f500804d070 con 0x563e85fef3b0 2024-09-21T03:43:46.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.419+0000 7f4ffbfff640 1 -- 172.21.15.38:0/3092938901 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5008003ac0 con 0x563e85fef3b0 2024-09-21T03:43:46.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.420+0000 7f50277bd740 1 -- 172.21.15.38:0/3092938901 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563e85fef3b0 msgr2=0x563e863ea500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:46.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.420+0000 7f50277bd740 1 --2- 172.21.15.38:0/3092938901 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563e85fef3b0 0x563e863ea500 secure :-1 s=READY pgs=2078 cs=0 l=1 rev1=1 crypto rx=0x7f500802abc0 tx=0x7f500803d000 comp rx=0 tx=0).stop 2024-09-21T03:43:46.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.420+0000 7f50277bd740 1 -- 172.21.15.38:0/3092938901 shutdown_connections 2024-09-21T03:43:46.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.420+0000 7f50277bd740 1 --2- 172.21.15.38:0/3092938901 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563e85ff7590 0x563e86407400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:46.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.420+0000 7f50277bd740 1 --2- 172.21.15.38:0/3092938901 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563e85fef3b0 0x563e863ea500 unknown :-1 s=CLOSED pgs=2078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:46.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.420+0000 7f50277bd740 1 --2- 172.21.15.38:0/3092938901 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x563e86018270 0x563e863e9fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:46.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.420+0000 7f50277bd740 1 -- 172.21.15.38:0/3092938901 >> 172.21.15.38:0/3092938901 conn(0x563e8600c3c0 msgr2=0x563e863c0280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:46.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.420+0000 7f50277bd740 1 -- 172.21.15.38:0/3092938901 shutdown_connections 2024-09-21T03:43:46.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.420+0000 7f50277bd740 1 -- 172.21.15.38:0/3092938901 wait complete. 2024-09-21T03:43:46.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.421+0000 7f50277bd740 1 -- start start 2024-09-21T03:43:46.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.421+0000 7f50277bd740 1 Processor -- start 2024-09-21T03:43:46.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.422+0000 7f50277bd740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x563e85fef3b0 0x563e863e22f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:46.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.422+0000 7f50277bd740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563e85ff7590 0x563e863e2830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:46.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.422+0000 7f50277bd740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563e86018270 0x563e863fc730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:46.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.422+0000 7f50277bd740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x563e86092620 con 0x563e85ff7590 2024-09-21T03:43:46.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.422+0000 7f50277bd740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x563e862c75f0 con 0x563e86018270 2024-09-21T03:43:46.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.422+0000 7f50277bd740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x563e863eaa40 con 0x563e85fef3b0 2024-09-21T03:43:46.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.422+0000 7f50277bd740 10 client.0 _ll_register_callbacks cb 0x563e862cb0a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:43:46.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.422+0000 7f501dd09640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563e85ff7590 0x563e863e2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:46.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.422+0000 7f501dd09640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563e85ff7590 0x563e863e2830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54502/0 (socket says 172.21.15.38:54502) 2024-09-21T03:43:46.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.422+0000 7f501dd09640 1 -- 172.21.15.38:0/1042768990 learned_addr learned my addr 172.21.15.38:0/1042768990 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:43:46.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.422+0000 7f501e50a640 1 --2- 172.21.15.38:0/1042768990 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x563e85fef3b0 0x563e863e22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:46.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.422+0000 7f501ed0b640 1 --2- 172.21.15.38:0/1042768990 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563e86018270 0x563e863fc730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:46.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.423+0000 7f501dd09640 1 -- 172.21.15.38:0/1042768990 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x563e85fef3b0 msgr2=0x563e863e22f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:46.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.423+0000 7f501dd09640 1 --2- 172.21.15.38:0/1042768990 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x563e85fef3b0 0x563e863e22f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:46.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.423+0000 7f501dd09640 1 -- 172.21.15.38:0/1042768990 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563e86018270 msgr2=0x563e863fc730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:46.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.423+0000 7f501dd09640 1 --2- 172.21.15.38:0/1042768990 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563e86018270 0x563e863fc730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:46.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.423+0000 7f501dd09640 1 -- 172.21.15.38:0/1042768990 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e860fa6b0 con 0x563e85ff7590 2024-09-21T03:43:46.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.423+0000 7f501dd09640 1 --2- 172.21.15.38:0/1042768990 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563e85ff7590 0x563e863e2830 secure :-1 s=READY pgs=2079 cs=0 l=1 rev1=1 crypto rx=0x7f500802abc0 tx=0x7f5008005670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e3443c6d9cbc357 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:46.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.423+0000 7f4ff97fa640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-21T03:43:46.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.423+0000 7f4ff97fa640 1 -- 172.21.15.38:0/1042768990 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50080047a0 con 0x563e85ff7590 2024-09-21T03:43:46.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.423+0000 7f50277bd740 1 -- 172.21.15.38:0/1042768990 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f500804d070 con 0x563e85ff7590 2024-09-21T03:43:46.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.423+0000 7f4ff97fa640 1 -- 172.21.15.38:0/1042768990 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(129..129 src has 1..129) ==== 9142+0+0 (secure 0 0 0) 0x7f5008004940 con 0x563e85ff7590 2024-09-21T03:43:46.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.423+0000 7f4ff97fa640 1 -- 172.21.15.38:0/1042768990 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=290) ==== 24+0+0 (secure 0 0 0) 0x7f5008004b40 con 0x563e85ff7590 2024-09-21T03:43:46.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.423+0000 7f50277bd740 10 client.10056 fetch_fsmap learned FSMap version 290 2024-09-21T03:43:46.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.423+0000 7f50277bd740 1 -- 172.21.15.38:0/1042768990 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=290,monmap=2+}) -- 0x563e863fd7b0 con 0x563e85ff7590 2024-09-21T03:43:46.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.423+0000 7f4ff97fa640 1 -- 172.21.15.38:0/1042768990 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f500804d070 con 0x563e85ff7590 2024-09-21T03:43:46.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.424+0000 7f4ff97fa640 1 -- 172.21.15.38:0/1042768990 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 290) ==== 3502+0+0 (secure 0 0 0) 0x7f5008004ce0 con 0x563e85ff7590 2024-09-21T03:43:46.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.424+0000 7f50277bd740 10 client.10056 fetch_fsmap finished waiting for FSMap version 290 2024-09-21T03:43:46.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.424+0000 7f50277bd740 10 client.10056 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T03:43:46.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.424+0000 7f50277bd740 20 client.10056 populate_metadata read hostname 'smithi038' 2024-09-21T03:43:46.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.424+0000 7f50277bd740 1 --2- 172.21.15.38:0/1042768990 >> [v2:172.21.15.38:6832/626794495,v1:172.21.15.38:6833/626794495] conn(0x563e85fd6e40 0x563e86421690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:46.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.424+0000 7f50277bd740 4 client.10056 mds_command: new command op to 9999 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:43:46.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.424+0000 7f50277bd740 1 -- 172.21.15.38:0/1042768990 --> [v2:172.21.15.38:6832/626794495,v1:172.21.15.38:6833/626794495] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563e86334770 con 0x563e85fd6e40 2024-09-21T03:43:46.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.424+0000 7f501e50a640 1 --2- 172.21.15.38:0/1042768990 >> [v2:172.21.15.38:6832/626794495,v1:172.21.15.38:6833/626794495] conn(0x563e85fd6e40 0x563e86421690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:46.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.424+0000 7f4ff97fa640 10 client.10056 ms_handle_connect on v2:172.21.15.38:6832/626794495 2024-09-21T03:43:46.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.424+0000 7f501e50a640 1 --2- 172.21.15.38:0/1042768990 >> [v2:172.21.15.38:6832/626794495,v1:172.21.15.38:6833/626794495] conn(0x563e85fd6e40 0x563e86421690 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=833b76c55cf9348 server_cookie=4485c30ac17a0d6b in_seq=0 out_seq=0 2024-09-21T03:43:46.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.425+0000 7f4ff97fa640 1 -- 172.21.15.38:0/1042768990 <== mds.0 v2:172.21.15.38:6832/626794495 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x563e86334770 con 0x563e85fd6e40 2024-09-21T03:43:46.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.425+0000 7f4ff97fa640 10 client.10056 handle_command_reply: tid=0 2024-09-21T03:43:46.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.425+0000 7f50277bd740 1 client.10056 shutdown 2024-09-21T03:43:46.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.425+0000 7f50277bd740 1 -- 172.21.15.38:0/1042768990 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563e85ff7590 msgr2=0x563e863e2830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:46.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.425+0000 7f50277bd740 1 --2- 172.21.15.38:0/1042768990 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563e85ff7590 0x563e863e2830 secure :-1 s=READY pgs=2079 cs=0 l=1 rev1=1 crypto rx=0x7f500802abc0 tx=0x7f5008005670 comp rx=0 tx=0).stop 2024-09-21T03:43:46.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.425+0000 7f50277bd740 1 -- 172.21.15.38:0/1042768990 shutdown_connections 2024-09-21T03:43:46.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.425+0000 7f50277bd740 1 --2- 172.21.15.38:0/1042768990 >> [v2:172.21.15.38:6832/626794495,v1:172.21.15.38:6833/626794495] conn(0x563e85fd6e40 0x563e86421690 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:46.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.425+0000 7f50277bd740 1 --2- 172.21.15.38:0/1042768990 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563e86018270 0x563e863fc730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:46.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.425+0000 7f50277bd740 1 --2- 172.21.15.38:0/1042768990 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563e85ff7590 0x563e863e2830 unknown :-1 s=CLOSED pgs=2079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:46.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.425+0000 7f50277bd740 1 --2- 172.21.15.38:0/1042768990 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x563e85fef3b0 0x563e863e22f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:46.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.425+0000 7f4ff97fa640 0 client.10056 ms_handle_reset on v2:172.21.15.38:6832/626794495 2024-09-21T03:43:46.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.425+0000 7f50277bd740 1 -- 172.21.15.38:0/1042768990 >> 172.21.15.38:0/1042768990 conn(0x563e8600c3c0 msgr2=0x563e86405600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:46.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.425+0000 7f50277bd740 1 -- 172.21.15.38:0/1042768990 shutdown_connections 2024-09-21T03:43:46.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.425+0000 7f50277bd740 1 -- 172.21.15.38:0/1042768990 wait complete. 2024-09-21T03:43:46.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.425+0000 7f50277bd740 20 client.10056 trim_cache size 0 max 16384 2024-09-21T03:43:46.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.444+0000 7f50277bd740 1 Processor -- start 2024-09-21T03:43:46.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.444+0000 7f50277bd740 1 -- start start 2024-09-21T03:43:46.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.445+0000 7f50277bd740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563e85fef3b0 0x563e863e2550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:46.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.445+0000 7f50277bd740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x563e85ff7590 0x563e863e5150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:46.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.445+0000 7f50277bd740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563e86018270 0x563e863e5690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:46.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.445+0000 7f50277bd740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f5018201310 con 0x563e85fef3b0 2024-09-21T03:43:46.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.445+0000 7f50277bd740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f50182005f0 con 0x563e86018270 2024-09-21T03:43:46.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.445+0000 7f50277bd740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f5018200c40 con 0x563e85ff7590 2024-09-21T03:43:46.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.445+0000 7f501e50a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563e85fef3b0 0x563e863e2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:46.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.445+0000 7f501e50a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563e85fef3b0 0x563e863e2550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54504/0 (socket says 172.21.15.38:54504) 2024-09-21T03:43:46.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.445+0000 7f501e50a640 1 -- 172.21.15.38:0/3953668147 learned_addr learned my addr 172.21.15.38:0/3953668147 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:43:46.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.446+0000 7f501e50a640 1 -- 172.21.15.38:0/3953668147 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x563e85ff7590 msgr2=0x563e863e5150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:46.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.446+0000 7f501e50a640 1 --2- 172.21.15.38:0/3953668147 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x563e85ff7590 0x563e863e5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:46.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.446+0000 7f501e50a640 1 -- 172.21.15.38:0/3953668147 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563e86018270 msgr2=0x563e863e5690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:46.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.446+0000 7f501ed0b640 1 --2- 172.21.15.38:0/3953668147 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563e86018270 0x563e863e5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:46.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.446+0000 7f501e50a640 1 --2- 172.21.15.38:0/3953668147 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563e86018270 0x563e863e5690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:46.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.446+0000 7f501e50a640 1 -- 172.21.15.38:0/3953668147 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e86405f10 con 0x563e85fef3b0 2024-09-21T03:43:46.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.446+0000 7f501ed0b640 1 --2- 172.21.15.38:0/3953668147 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563e86018270 0x563e863e5690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:43:46.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.446+0000 7f501e50a640 1 --2- 172.21.15.38:0/3953668147 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563e85fef3b0 0x563e863e2550 secure :-1 s=READY pgs=2080 cs=0 l=1 rev1=1 crypto rx=0x7f501405f090 tx=0x7f501401e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ef072527c02ab54 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:46.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.446+0000 7f4ffb7fe640 1 -- 172.21.15.38:0/3953668147 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50140986f0 con 0x563e85fef3b0 2024-09-21T03:43:46.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.446+0000 7f4ffb7fe640 1 -- 172.21.15.38:0/3953668147 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5014066040 con 0x563e85fef3b0 2024-09-21T03:43:46.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.446+0000 7f4ffb7fe640 1 -- 172.21.15.38:0/3953668147 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50140840d0 con 0x563e85fef3b0 2024-09-21T03:43:46.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.447+0000 7f50277bd740 1 -- 172.21.15.38:0/3953668147 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563e85fef3b0 msgr2=0x563e863e2550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:46.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.447+0000 7f50277bd740 1 --2- 172.21.15.38:0/3953668147 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563e85fef3b0 0x563e863e2550 secure :-1 s=READY pgs=2080 cs=0 l=1 rev1=1 crypto rx=0x7f501405f090 tx=0x7f501401e040 comp rx=0 tx=0).stop 2024-09-21T03:43:46.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.447+0000 7f50277bd740 1 -- 172.21.15.38:0/3953668147 shutdown_connections 2024-09-21T03:43:46.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.447+0000 7f50277bd740 1 --2- 172.21.15.38:0/3953668147 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563e86018270 0x563e863e5690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:46.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.447+0000 7f50277bd740 1 --2- 172.21.15.38:0/3953668147 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x563e85ff7590 0x563e863e5150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:46.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.447+0000 7f50277bd740 1 --2- 172.21.15.38:0/3953668147 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563e85fef3b0 0x563e863e2550 unknown :-1 s=CLOSED pgs=2080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:46.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.447+0000 7f50277bd740 1 -- 172.21.15.38:0/3953668147 >> 172.21.15.38:0/3953668147 conn(0x563e8600c3c0 msgr2=0x563e863fb180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:46.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.447+0000 7f50277bd740 1 -- 172.21.15.38:0/3953668147 shutdown_connections 2024-09-21T03:43:46.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.447+0000 7f50277bd740 1 -- 172.21.15.38:0/3953668147 wait complete. 2024-09-21T03:43:46.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.448+0000 7f50277bd740 1 -- start start 2024-09-21T03:43:46.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.448+0000 7f50277bd740 1 Processor -- start 2024-09-21T03:43:46.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.449+0000 7f50277bd740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563e85fef3b0 0x563e863fe530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:46.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.449+0000 7f50277bd740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563e85ff7590 0x563e86488d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:46.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.449+0000 7f50277bd740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x563e86018270 0x563e86400af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:46.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.449+0000 7f50277bd740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x563e863e76f0 con 0x563e85fef3b0 2024-09-21T03:43:46.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.449+0000 7f50277bd740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x563e863d8b70 con 0x563e85ff7590 2024-09-21T03:43:46.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.449+0000 7f50277bd740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x563e863e41a0 con 0x563e86018270 2024-09-21T03:43:46.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.449+0000 7f50277bd740 10 client.0 _ll_register_callbacks cb 0x7f5010009a90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:43:46.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.449+0000 7f501e50a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563e85fef3b0 0x563e863fe530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:46.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.449+0000 7f501e50a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563e85fef3b0 0x563e863fe530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54506/0 (socket says 172.21.15.38:54506) 2024-09-21T03:43:46.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.449+0000 7f501e50a640 1 -- 172.21.15.38:0/1572162396 learned_addr learned my addr 172.21.15.38:0/1572162396 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:43:46.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.449+0000 7f501dd09640 1 --2- 172.21.15.38:0/1572162396 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563e85ff7590 0x563e86488d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:46.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.450+0000 7f501ed0b640 1 --2- 172.21.15.38:0/1572162396 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x563e86018270 0x563e86400af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:46.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.450+0000 7f501e50a640 1 -- 172.21.15.38:0/1572162396 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x563e86018270 msgr2=0x563e86400af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:46.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.450+0000 7f501e50a640 1 --2- 172.21.15.38:0/1572162396 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x563e86018270 0x563e86400af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:46.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.450+0000 7f501e50a640 1 -- 172.21.15.38:0/1572162396 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563e85ff7590 msgr2=0x563e86488d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:46.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.450+0000 7f501e50a640 1 --2- 172.21.15.38:0/1572162396 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563e85ff7590 0x563e86488d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:46.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.450+0000 7f501e50a640 1 -- 172.21.15.38:0/1572162396 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e863b96d0 con 0x563e85fef3b0 2024-09-21T03:43:46.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.450+0000 7f501dd09640 1 --2- 172.21.15.38:0/1572162396 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563e85ff7590 0x563e86488d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:43:46.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.450+0000 7f501ed0b640 1 --2- 172.21.15.38:0/1572162396 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x563e86018270 0x563e86400af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:43:46.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.450+0000 7f501e50a640 1 --2- 172.21.15.38:0/1572162396 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563e85fef3b0 0x563e863fe530 secure :-1 s=READY pgs=2081 cs=0 l=1 rev1=1 crypto rx=0x7f5014061db0 tx=0x7f501406b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adf9f58cb2ea9fc2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:46.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.450+0000 7f4ff97fa640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-21T03:43:46.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.450+0000 7f4ff97fa640 1 -- 172.21.15.38:0/1572162396 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f501407b030 con 0x563e85fef3b0 2024-09-21T03:43:46.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.450+0000 7f4ff97fa640 1 -- 172.21.15.38:0/1572162396 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(129..129 src has 1..129) ==== 9142+0+0 (secure 0 0 0) 0x7f50140840c0 con 0x563e85fef3b0 2024-09-21T03:43:46.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.450+0000 7f50277bd740 1 -- 172.21.15.38:0/1572162396 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5014066040 con 0x563e85fef3b0 2024-09-21T03:43:46.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.450+0000 7f4ff97fa640 1 -- 172.21.15.38:0/1572162396 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=290) ==== 24+0+0 (secure 0 0 0) 0x7f50140842c0 con 0x563e85fef3b0 2024-09-21T03:43:46.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.450+0000 7f50277bd740 10 client.10062 fetch_fsmap learned FSMap version 290 2024-09-21T03:43:46.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.450+0000 7f50277bd740 1 -- 172.21.15.38:0/1572162396 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=290,monmap=2+}) -- 0x563e86484d50 con 0x563e85fef3b0 2024-09-21T03:43:46.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.450+0000 7f4ff97fa640 1 -- 172.21.15.38:0/1572162396 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5014066040 con 0x563e85fef3b0 2024-09-21T03:43:46.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.450+0000 7f4ff97fa640 1 -- 172.21.15.38:0/1572162396 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 290) ==== 3502+0+0 (secure 0 0 0) 0x7f5014061080 con 0x563e85fef3b0 2024-09-21T03:43:46.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.450+0000 7f50277bd740 10 client.10062 fetch_fsmap finished waiting for FSMap version 290 2024-09-21T03:43:46.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.450+0000 7f50277bd740 10 client.10062 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T03:43:46.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.450+0000 7f50277bd740 20 client.10062 populate_metadata read hostname 'smithi038' 2024-09-21T03:43:46.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.450+0000 7f50277bd740 1 --2- 172.21.15.38:0/1572162396 >> [v2:172.21.15.38:6832/626794495,v1:172.21.15.38:6833/626794495] conn(0x563e863e9aa0 0x563e863e9eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:46.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.451+0000 7f50277bd740 4 client.10062 mds_command: new command op to 9999 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T03:43:46.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.451+0000 7f50277bd740 1 -- 172.21.15.38:0/1572162396 --> [v2:172.21.15.38:6832/626794495,v1:172.21.15.38:6833/626794495] -- command(tid 0: {"prefix": "status"}) -- 0x563e86330220 con 0x563e863e9aa0 2024-09-21T03:43:46.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.451+0000 7f501dd09640 1 --2- 172.21.15.38:0/1572162396 >> [v2:172.21.15.38:6832/626794495,v1:172.21.15.38:6833/626794495] conn(0x563e863e9aa0 0x563e863e9eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:46.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.451+0000 7f4ff97fa640 10 client.10062 ms_handle_connect on v2:172.21.15.38:6832/626794495 2024-09-21T03:43:46.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.451+0000 7f501dd09640 1 --2- 172.21.15.38:0/1572162396 >> [v2:172.21.15.38:6832/626794495,v1:172.21.15.38:6833/626794495] conn(0x563e863e9aa0 0x563e863e9eb0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a223f30943e7f2a server_cookie=c624e1a0a621b5a1 in_seq=0 out_seq=0 2024-09-21T03:43:46.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.451+0000 7f4ff97fa640 1 -- 172.21.15.38:0/1572162396 <== mds.0 v2:172.21.15.38:6832/626794495 1 ==== command_reply(tid 0: 0 ) ==== 8+0+334 (crc 0 0 0) 0x563e86330220 con 0x563e863e9aa0 2024-09-21T03:43:46.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.451+0000 7f4ff97fa640 10 client.10062 handle_command_reply: tid=0 2024-09-21T03:43:46.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.451+0000 7f50277bd740 1 client.10062 shutdown 2024-09-21T03:43:46.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.451+0000 7f50277bd740 1 -- 172.21.15.38:0/1572162396 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563e85fef3b0 msgr2=0x563e863fe530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:46.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.451+0000 7f50277bd740 1 --2- 172.21.15.38:0/1572162396 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563e85fef3b0 0x563e863fe530 secure :-1 s=READY pgs=2081 cs=0 l=1 rev1=1 crypto rx=0x7f5014061db0 tx=0x7f501406b8d0 comp rx=0 tx=0).stop 2024-09-21T03:43:46.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.451+0000 7f50277bd740 1 -- 172.21.15.38:0/1572162396 shutdown_connections 2024-09-21T03:43:46.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.451+0000 7f50277bd740 1 --2- 172.21.15.38:0/1572162396 >> [v2:172.21.15.38:6832/626794495,v1:172.21.15.38:6833/626794495] conn(0x563e863e9aa0 0x563e863e9eb0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:46.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.451+0000 7f50277bd740 1 --2- 172.21.15.38:0/1572162396 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x563e86018270 0x563e86400af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:46.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.451+0000 7f50277bd740 1 --2- 172.21.15.38:0/1572162396 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563e85ff7590 0x563e86488d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:46.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.451+0000 7f4ff97fa640 0 client.10062 ms_handle_reset on v2:172.21.15.38:6832/626794495 2024-09-21T03:43:46.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.451+0000 7f50277bd740 1 --2- 172.21.15.38:0/1572162396 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563e85fef3b0 0x563e863fe530 unknown :-1 s=CLOSED pgs=2081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:46.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.451+0000 7f50277bd740 1 -- 172.21.15.38:0/1572162396 >> 172.21.15.38:0/1572162396 conn(0x563e8600c3c0 msgr2=0x563e863d7c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:46.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.452+0000 7f50277bd740 1 -- 172.21.15.38:0/1572162396 shutdown_connections 2024-09-21T03:43:46.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.452+0000 7f50277bd740 1 -- 172.21.15.38:0/1572162396 wait complete. 2024-09-21T03:43:46.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.452+0000 7f50277bd740 20 client.10062 trim_cache size 0 max 16384 2024-09-21T03:43:46.459 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-09-21T03:43:46.459 INFO:teuthology.orchestra.run.smithi038.stdout: "cluster_fsid": "9d50d66f-656b-4fe2-8e3b-fe53b79c4d43", 2024-09-21T03:43:46.459 INFO:teuthology.orchestra.run.smithi038.stdout: "whoami": 0, 2024-09-21T03:43:46.459 INFO:teuthology.orchestra.run.smithi038.stdout: "id": 9999, 2024-09-21T03:43:46.459 INFO:teuthology.orchestra.run.smithi038.stdout: "want_state": "up:active", 2024-09-21T03:43:46.460 INFO:teuthology.orchestra.run.smithi038.stdout: "state": "up:active", 2024-09-21T03:43:46.460 INFO:teuthology.orchestra.run.smithi038.stdout: "fs_name": "cephfs", 2024-09-21T03:43:46.460 INFO:teuthology.orchestra.run.smithi038.stdout: "rank_uptime": 6.8066473739999998, 2024-09-21T03:43:46.460 INFO:teuthology.orchestra.run.smithi038.stdout: "mdsmap_epoch": 290, 2024-09-21T03:43:46.460 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch": 129, 2024-09-21T03:43:46.460 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch_barrier": 129, 2024-09-21T03:43:46.460 INFO:teuthology.orchestra.run.smithi038.stdout: "uptime": 8.0487238489999999 2024-09-21T03:43:46.460 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-09-21T03:43:46.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.452+0000 7f502076e640 1 -- 172.21.15.38:0/2513722468 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4ffc078b90 msgr2=0x7f4ffc07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:46.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.452+0000 7f502076e640 1 --2- 172.21.15.38:0/2513722468 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4ffc078b90 0x7f4ffc07b0a0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f5008033000 tx=0x7f5008004730 comp rx=0 tx=0).stop 2024-09-21T03:43:46.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.452+0000 7f502076e640 1 -- 172.21.15.38:0/2513722468 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f501806def0 msgr2=0x7f501809a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:46.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.452+0000 7f502076e640 1 --2- 172.21.15.38:0/2513722468 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f501806def0 0x7f501809a9e0 secure :-1 s=READY pgs=2077 cs=0 l=1 rev1=1 crypto rx=0x7f501400c370 tx=0x7f501400c830 comp rx=0 tx=0).stop 2024-09-21T03:43:46.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.452+0000 7f502076e640 1 -- 172.21.15.38:0/2513722468 shutdown_connections 2024-09-21T03:43:46.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.452+0000 7f502076e640 1 --2- 172.21.15.38:0/2513722468 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f501815bfc0 0x7f501809d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:46.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.452+0000 7f502076e640 1 --2- 172.21.15.38:0/2513722468 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4ffc078b90 0x7f4ffc07b0a0 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:46.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.452+0000 7f502076e640 1 --2- 172.21.15.38:0/2513722468 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f50180b5260 0x7f5018155700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:46.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.452+0000 7f502076e640 1 --2- 172.21.15.38:0/2513722468 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f501806def0 0x7f501809a9e0 unknown :-1 s=CLOSED pgs=2077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:46.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.452+0000 7f502076e640 1 -- 172.21.15.38:0/2513722468 >> 172.21.15.38:0/2513722468 conn(0x7f50180cdca0 msgr2=0x7f50180aea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:46.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.452+0000 7f502076e640 1 -- 172.21.15.38:0/2513722468 shutdown_connections 2024-09-21T03:43:46.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.452+0000 7f502076e640 1 -- 172.21.15.38:0/2513722468 wait complete. 2024-09-21T03:43:46.465 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-09-21T03:43:46.465 INFO:tasks.cephfs.test_damage:Daemons came up after mutation 'Corrupt omap key 1.00000000:subdir_head', proceeding to ls 2024-09-21T03:43:46.465 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T03:43:46.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.579+0000 7fdbd9e63640 1 -- 172.21.15.38:0/2752700253 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdbd4162f00 msgr2=0x7fdbd40d24d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:46.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.579+0000 7fdbd9e63640 1 --2- 172.21.15.38:0/2752700253 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdbd4162f00 0x7fdbd40d24d0 secure :-1 s=READY pgs=2083 cs=0 l=1 rev1=1 crypto rx=0x7fdbc4009830 tx=0x7fdbc402a510 comp rx=0 tx=0).stop 2024-09-21T03:43:46.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.580+0000 7fdbd9e63640 1 -- 172.21.15.38:0/2752700253 shutdown_connections 2024-09-21T03:43:46.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.580+0000 7fdbd9e63640 1 --2- 172.21.15.38:0/2752700253 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdbd40b2e40 0x7fdbd40b3250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:46.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.580+0000 7fdbd9e63640 1 --2- 172.21.15.38:0/2752700253 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdbd4162f00 0x7fdbd40d24d0 unknown :-1 s=CLOSED pgs=2083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:46.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.580+0000 7fdbd9e63640 1 -- 172.21.15.38:0/2752700253 >> 172.21.15.38:0/2752700253 conn(0x7fdbd40b6b20 msgr2=0x7fdbd40d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:46.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.580+0000 7fdbd9e63640 1 -- 172.21.15.38:0/2752700253 shutdown_connections 2024-09-21T03:43:46.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.580+0000 7fdbd9e63640 1 -- 172.21.15.38:0/2752700253 wait complete. 2024-09-21T03:43:46.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.581+0000 7fdbd9e63640 1 Processor -- start 2024-09-21T03:43:46.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.582+0000 7fdbd9e63640 1 -- start start 2024-09-21T03:43:46.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.583+0000 7fdbd9e63640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdbd40b2e40 0x7fdbd414bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:46.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.583+0000 7fdbd9e63640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdbd4162280 0x7fdbd414e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:46.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.583+0000 7fdbd9e63640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdbd40ceb20 0x7fdbd40ac0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:46.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.583+0000 7fdbd9e63640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fdbd415e4b0 con 0x7fdbd4162280 2024-09-21T03:43:46.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.583+0000 7fdbd9e63640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fdbd40b3a20 con 0x7fdbd40ceb20 2024-09-21T03:43:46.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.583+0000 7fdbd9e63640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fdbd415e630 con 0x7fdbd40b2e40 2024-09-21T03:43:46.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.583+0000 7fdbd2ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdbd4162280 0x7fdbd414e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:46.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.583+0000 7fdbd2ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdbd4162280 0x7fdbd414e650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54536/0 (socket says 172.21.15.38:54536) 2024-09-21T03:43:46.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.583+0000 7fdbd2ffd640 1 -- 172.21.15.38:0/612413432 learned_addr learned my addr 172.21.15.38:0/612413432 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:43:46.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.583+0000 7fdbd3fff640 1 --2- 172.21.15.38:0/612413432 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdbd40ceb20 0x7fdbd40ac0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:46.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.583+0000 7fdbd37fe640 1 --2- 172.21.15.38:0/612413432 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdbd40b2e40 0x7fdbd414bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:46.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.583+0000 7fdbd2ffd640 1 -- 172.21.15.38:0/612413432 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdbd40b2e40 msgr2=0x7fdbd414bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:46.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.583+0000 7fdbd2ffd640 1 --2- 172.21.15.38:0/612413432 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdbd40b2e40 0x7fdbd414bd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:46.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.583+0000 7fdbd2ffd640 1 -- 172.21.15.38:0/612413432 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdbd40ceb20 msgr2=0x7fdbd40ac0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:46.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.583+0000 7fdbd2ffd640 1 --2- 172.21.15.38:0/612413432 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdbd40ceb20 0x7fdbd40ac0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:46.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.583+0000 7fdbd2ffd640 1 -- 172.21.15.38:0/612413432 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbd40ab010 con 0x7fdbd4162280 2024-09-21T03:43:46.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.583+0000 7fdbd37fe640 1 --2- 172.21.15.38:0/612413432 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdbd40b2e40 0x7fdbd414bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:43:46.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.584+0000 7fdbd2ffd640 1 --2- 172.21.15.38:0/612413432 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdbd4162280 0x7fdbd414e650 secure :-1 s=READY pgs=2084 cs=0 l=1 rev1=1 crypto rx=0x7fdbc800c400 tx=0x7fdbc800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:46.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.584+0000 7fdbbbfff640 1 -- 172.21.15.38:0/612413432 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbc8011790 con 0x7fdbd4162280 2024-09-21T03:43:46.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.584+0000 7fdbbbfff640 1 -- 172.21.15.38:0/612413432 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdbc8017070 con 0x7fdbd4162280 2024-09-21T03:43:46.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.584+0000 7fdbd9e63640 1 -- 172.21.15.38:0/612413432 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbd40ab2f0 con 0x7fdbd4162280 2024-09-21T03:43:46.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.584+0000 7fdbbbfff640 1 -- 172.21.15.38:0/612413432 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbc8011ce0 con 0x7fdbd4162280 2024-09-21T03:43:46.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.584+0000 7fdbd9e63640 1 -- 172.21.15.38:0/612413432 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbd40ab8b0 con 0x7fdbd4162280 2024-09-21T03:43:46.589 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.585+0000 7fdbd9e63640 1 -- 172.21.15.38:0/612413432 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbd4162f00 con 0x7fdbd4162280 2024-09-21T03:43:46.589 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.586+0000 7fdbbbfff640 1 -- 172.21.15.38:0/612413432 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fdbc801c460 con 0x7fdbd4162280 2024-09-21T03:43:46.589 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.587+0000 7fdbbbfff640 1 --2- 172.21.15.38:0/612413432 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fdbc0078b90 0x7fdbc007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:46.589 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.587+0000 7fdbbbfff640 1 -- 172.21.15.38:0/612413432 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 9142+0+0 (secure 0 0 0) 0x7fdbc809b790 con 0x7fdbd4162280 2024-09-21T03:43:46.589 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.587+0000 7fdbd37fe640 1 --2- 172.21.15.38:0/612413432 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fdbc0078b90 0x7fdbc007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:46.589 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.587+0000 7fdbd37fe640 1 --2- 172.21.15.38:0/612413432 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fdbc0078b90 0x7fdbc007b0a0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fdbc4034000 tx=0x7fdbc4004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:46.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.591+0000 7fdbbbfff640 1 -- 172.21.15.38:0/612413432 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fdbc8062eb0 con 0x7fdbd4162280 2024-09-21T03:43:46.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.817+0000 7fdbd9e63640 1 -- 172.21.15.38:0/612413432 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fdbd4069240 con 0x7fdbd4162280 2024-09-21T03:43:46.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.817+0000 7fdbbbfff640 1 -- 172.21.15.38:0/612413432 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v290) ==== 53+0+73 (secure 0 0 0) 0x7fdbc80627b0 con 0x7fdbd4162280 2024-09-21T03:43:46.820 INFO:teuthology.orchestra.run.smithi038.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T03:43:46.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.820+0000 7fdbd9e63640 1 -- 172.21.15.38:0/612413432 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fdbc0078b90 msgr2=0x7fdbc007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:46.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.820+0000 7fdbd9e63640 1 --2- 172.21.15.38:0/612413432 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fdbc0078b90 0x7fdbc007b0a0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fdbc4034000 tx=0x7fdbc4004730 comp rx=0 tx=0).stop 2024-09-21T03:43:46.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.820+0000 7fdbd9e63640 1 -- 172.21.15.38:0/612413432 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdbd4162280 msgr2=0x7fdbd414e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:46.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.820+0000 7fdbd9e63640 1 --2- 172.21.15.38:0/612413432 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdbd4162280 0x7fdbd414e650 secure :-1 s=READY pgs=2084 cs=0 l=1 rev1=1 crypto rx=0x7fdbc800c400 tx=0x7fdbc800c8c0 comp rx=0 tx=0).stop 2024-09-21T03:43:46.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.820+0000 7fdbd9e63640 1 -- 172.21.15.38:0/612413432 shutdown_connections 2024-09-21T03:43:46.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.820+0000 7fdbd9e63640 1 --2- 172.21.15.38:0/612413432 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdbd40ceb20 0x7fdbd40ac0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:46.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.820+0000 7fdbd9e63640 1 --2- 172.21.15.38:0/612413432 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdbd4162280 0x7fdbd414e650 unknown :-1 s=CLOSED pgs=2084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:46.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.820+0000 7fdbd9e63640 1 --2- 172.21.15.38:0/612413432 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fdbc0078b90 0x7fdbc007b0a0 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:46.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.820+0000 7fdbd9e63640 1 --2- 172.21.15.38:0/612413432 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdbd40b2e40 0x7fdbd414bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:46.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.820+0000 7fdbd9e63640 1 -- 172.21.15.38:0/612413432 >> 172.21.15.38:0/612413432 conn(0x7fdbd40b6b20 msgr2=0x7fdbd415dac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:46.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.820+0000 7fdbd9e63640 1 -- 172.21.15.38:0/612413432 shutdown_connections 2024-09-21T03:43:46.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:46.820+0000 7fdbd9e63640 1 -- 172.21.15.38:0/612413432 wait complete. 2024-09-21T03:43:46.833 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-21T03:43:46.833 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T03:43:46.833 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi038.front.sepia.ceph.com 2024-09-21T03:43:46.833 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-21T03:43:46.833 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T03:43:46.833 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-21T03:43:46.834 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-21T03:43:46.834 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T03:43:46.834 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T03:43:46.834 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:43:46.834 DEBUG:teuthology.orchestra.run.smithi038:> ip addr 2024-09-21T03:43:46.850 INFO:teuthology.orchestra.run.smithi038.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T03:43:46.850 INFO:teuthology.orchestra.run.smithi038.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T03:43:46.850 INFO:teuthology.orchestra.run.smithi038.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T03:43:46.851 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:43:46.851 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 ::1/128 scope host 2024-09-21T03:43:46.851 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:43:46.851 INFO:teuthology.orchestra.run.smithi038.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T03:43:46.851 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:6b:fd:a2 brd ff:ff:ff:ff:ff:ff 2024-09-21T03:43:46.851 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp7s0f0 2024-09-21T03:43:46.851 INFO:teuthology.orchestra.run.smithi038.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T03:43:46.851 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:6b:fd:a3 brd ff:ff:ff:ff:ff:ff 2024-09-21T03:43:46.851 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp7s0f1 2024-09-21T03:43:46.851 INFO:teuthology.orchestra.run.smithi038.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T03:43:46.851 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:bd:15:fc brd ff:ff:ff:ff:ff:ff 2024-09-21T03:43:46.851 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp1s0f0 2024-09-21T03:43:46.851 INFO:teuthology.orchestra.run.smithi038.stdout: inet 172.21.15.38/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-21T03:43:46.852 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft 42182sec preferred_lft 42182sec 2024-09-21T03:43:46.852 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 fe80::ec4:7aff:febd:15fc/64 scope link 2024-09-21T03:43:46.852 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:43:46.852 INFO:teuthology.orchestra.run.smithi038.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T03:43:46.852 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:bd:15:fd brd ff:ff:ff:ff:ff:ff 2024-09-21T03:43:46.852 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp1s0f1 2024-09-21T03:43:46.852 INFO:teuthology.orchestra.run.smithi038.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T03:43:46.853 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 92:55:c7:97:6f:b5 brd ff:ff:ff:ff:ff:ff 2024-09-21T03:43:46.853 INFO:teuthology.orchestra.run.smithi038.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-21T03:43:46.853 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:43:46.853 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 fe80::c49f:c1ff:fe21:b12a/64 scope link 2024-09-21T03:43:46.853 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:43:46.853 INFO:teuthology.orchestra.run.smithi038.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T03:43:46.853 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether a6:3a:83:ad:db:d5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T03:43:46.853 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 fe80::a43a:83ff:fead:dbd5/64 scope link 2024-09-21T03:43:46.853 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:43:46.853 INFO:teuthology.orchestra.run.smithi038.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T03:43:46.853 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 92:55:c7:97:6f:b5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T03:43:46.853 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 fe80::9055:c7ff:fe97:6fb5/64 scope link 2024-09-21T03:43:46.853 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:43:46.854 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:43:46.854 DEBUG:teuthology.orchestra.run.smithi038:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T03:43:46.915 INFO:teuthology.orchestra.run.smithi038.stdout:1 2024-09-21T03:43:46.917 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:43:46.918 DEBUG:teuthology.orchestra.run.smithi038:> route 2024-09-21T03:43:46.974 INFO:teuthology.orchestra.run.smithi038.stdout:Kernel IP routing table 2024-09-21T03:43:46.974 INFO:teuthology.orchestra.run.smithi038.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T03:43:46.974 INFO:teuthology.orchestra.run.smithi038.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-21T03:43:46.974 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-21T03:43:46.974 INFO:teuthology.orchestra.run.smithi038.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T03:43:46.974 DEBUG:teuthology.orchestra.run.smithi038:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T03:43:46.975 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-09-21T03:43:46.975 DEBUG:teuthology.orchestra.run.smithi038:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-21T03:43:46.975 DEBUG:teuthology.orchestra.run.smithi038:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-21T03:43:46.975 DEBUG:teuthology.orchestra.run.smithi038:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-21T03:43:46.975 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-09-21T03:43:47.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:47 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T03:43:47.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:43:47.103 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:43:47.103 DEBUG:teuthology.orchestra.run.smithi038:> ip netns list 2024-09-21T03:43:47.120 INFO:teuthology.orchestra.run.smithi038.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-21T03:43:47.120 INFO:teuthology.orchestra.run.smithi038.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-21T03:43:47.120 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T03:43:47.120 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:43:47.120 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip link set brx.0 up 2024-09-21T03:43:47.185 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T03:43:47.185 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-21T03:43:47.242 INFO:teuthology.orchestra.run.smithi038.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-21T03:43:47.242 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T03:43:47.242 DEBUG:teuthology.orchestra.run.smithi038:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-21T03:43:47.297 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-21T03:43:47.297 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:43:47.298 DEBUG:teuthology.orchestra.run.smithi038:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T03:43:47.393 INFO:teuthology.orchestra.run.smithi038.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T03:43:47.394 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T03:43:47.394 INFO:teuthology.orchestra.run.smithi038.stdout:invalid new device string format 2024-09-21T03:43:47.405 INFO:teuthology.orchestra.run.smithi038.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T03:43:47.405 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T03:43:47.406 INFO:teuthology.orchestra.run.smithi038.stdout:invalid new device string format 2024-09-21T03:43:47.406 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: resolved to: "172.21.15.38:3300,172.21.15.195:3300,172.21.15.195:3301" 2024-09-21T03:43:47.406 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T03:43:47.411 INFO:teuthology.orchestra.run.smithi038.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T03:43:47.411 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T03:43:47.411 INFO:teuthology.orchestra.run.smithi038.stdout:invalid new device string format 2024-09-21T03:43:47.412 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: resolved to: "172.21.15.38:3300,172.21.15.195:3300,172.21.15.195:3301" 2024-09-21T03:43:47.412 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T03:43:47.412 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: resolved to: "172.21.15.38:6789,172.21.15.195:6789,172.21.15.195:6790" 2024-09-21T03:43:47.412 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.38:6789,172.21.15.195:6789,172.21.15.195:6790:/ 2024-09-21T03:43:47.412 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=9d50d66f-656b-4fe2-8e3b-fe53b79c4d43" will pass to kernel 2024-09-21T03:43:47.412 INFO:teuthology.orchestra.run.smithi038.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-21T03:43:47.412 INFO:teuthology.orchestra.run.smithi038.stdout: You just mounted a file system that supports labels which does not 2024-09-21T03:43:47.412 INFO:teuthology.orchestra.run.smithi038.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-21T03:43:47.413 INFO:teuthology.orchestra.run.smithi038.stdout: applications will generate AVC messages and not be allowed access to 2024-09-21T03:43:47.413 INFO:teuthology.orchestra.run.smithi038.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-21T03:43:47.414 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T03:43:47.414 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:43:47.414 DEBUG:teuthology.orchestra.run.smithi038:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-21T03:43:47.439 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T03:43:47.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:47 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T03:43:47.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:43:47.537 INFO:teuthology.orchestra.run.smithi038.stdout:client10074 2024-09-21T03:43:47.537 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-21T03:43:47.537 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:43:47.553 DEBUG:teuthology.orchestra.run.smithi038:> stdin-killer -- python3 -c ' 2024-09-21T03:43:47.553 DEBUG:teuthology.orchestra.run.smithi038:> import os 2024-09-21T03:43:47.553 DEBUG:teuthology.orchestra.run.smithi038:> import stat 2024-09-21T03:43:47.554 DEBUG:teuthology.orchestra.run.smithi038:> import json 2024-09-21T03:43:47.554 DEBUG:teuthology.orchestra.run.smithi038:> import sys 2024-09-21T03:43:47.554 DEBUG:teuthology.orchestra.run.smithi038:> 2024-09-21T03:43:47.554 DEBUG:teuthology.orchestra.run.smithi038:> try: 2024-09-21T03:43:47.554 DEBUG:teuthology.orchestra.run.smithi038:> s = os.stat('"'"'/home/ubuntu/cephtest/mnt.0/./subdir'"'"') 2024-09-21T03:43:47.554 DEBUG:teuthology.orchestra.run.smithi038:> except OSError as e: 2024-09-21T03:43:47.554 DEBUG:teuthology.orchestra.run.smithi038:> sys.exit(e.errno) 2024-09-21T03:43:47.554 DEBUG:teuthology.orchestra.run.smithi038:> 2024-09-21T03:43:47.554 DEBUG:teuthology.orchestra.run.smithi038:> attrs = ["st_mode", "st_ino", "st_dev", "st_nlink", "st_uid", "st_gid", "st_size", "st_atime", "st_mtime", "st_ctime"] 2024-09-21T03:43:47.554 DEBUG:teuthology.orchestra.run.smithi038:> print(json.dumps( 2024-09-21T03:43:47.554 DEBUG:teuthology.orchestra.run.smithi038:> dict([(a, getattr(s, a)) for a in attrs]), 2024-09-21T03:43:47.554 DEBUG:teuthology.orchestra.run.smithi038:> indent=2)) 2024-09-21T03:43:47.554 DEBUG:teuthology.orchestra.run.smithi038:> ' 2024-09-21T03:43:47.596 INFO:teuthology.orchestra.run:waiting for 20 2024-09-21T03:43:47.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:47 stdin-killer INFO: command exited with status 5: exiting normally with same code! 2024-09-21T03:43:47.671 DEBUG:teuthology.orchestra.run:got remote process result: 5 2024-09-21T03:43:47.671 INFO:tasks.cephfs.test_damage:Result: EIO on client 2024-09-21T03:43:47.671 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:43:47.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:47.826+0000 7f2a85494640 1 -- 172.21.15.38:0/2262827166 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2a800d12e0 msgr2=0x7f2a800d16f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:47.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:47.826+0000 7f2a85494640 1 --2- 172.21.15.38:0/2262827166 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2a800d12e0 0x7f2a800d16f0 secure :-1 s=READY pgs=2086 cs=0 l=1 rev1=1 crypto rx=0x7f2a80074a40 tx=0x7f2a7402a5f0 comp rx=0 tx=0).stop 2024-09-21T03:43:47.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:47.826+0000 7f2a85494640 1 -- 172.21.15.38:0/2262827166 shutdown_connections 2024-09-21T03:43:47.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:47.826+0000 7f2a85494640 1 --2- 172.21.15.38:0/2262827166 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2a800d12e0 0x7f2a800d16f0 unknown :-1 s=CLOSED pgs=2086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:47.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:47.826+0000 7f2a85494640 1 --2- 172.21.15.38:0/2262827166 >> v2:172.21.15.195:3300/0 conn(0x7f2a8015cb10 0x7f2a8015cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:47.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:47.826+0000 7f2a85494640 1 -- 172.21.15.38:0/2262827166 >> 172.21.15.38:0/2262827166 conn(0x7f2a800d2350 msgr2=0x7f2a800d2760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:47.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:47.826+0000 7f2a85494640 1 -- 172.21.15.38:0/2262827166 shutdown_connections 2024-09-21T03:43:47.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:47.826+0000 7f2a85494640 1 -- 172.21.15.38:0/2262827166 wait complete. 2024-09-21T03:43:47.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:47.828+0000 7f2a85494640 1 Processor -- start 2024-09-21T03:43:47.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:47.828+0000 7f2a85494640 1 -- start start 2024-09-21T03:43:47.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:47.830+0000 7f2a85494640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2a800d12e0 0x7f2a8015c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:47.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:47.830+0000 7f2a85494640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2a8015cb10 0x7f2a8014f970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:47.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:47.830+0000 7f2a85494640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2a80160100 0x7f2a801e93f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:47.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:47.830+0000 7f2a85494640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f2a800b73e0 con 0x7f2a800d12e0 2024-09-21T03:43:47.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:47.830+0000 7f2a85494640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f2a800b7260 con 0x7f2a80160100 2024-09-21T03:43:47.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:47.830+0000 7f2a85494640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f2a80160c20 con 0x7f2a8015cb10 2024-09-21T03:43:47.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:47.830+0000 7f2a7effd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2a800d12e0 0x7f2a8015c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:47.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:47.830+0000 7f2a7effd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2a800d12e0 0x7f2a8015c0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54544/0 (socket says 172.21.15.38:54544) 2024-09-21T03:43:47.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:47.830+0000 7f2a7effd640 1 -- 172.21.15.38:0/603551815 learned_addr learned my addr 172.21.15.38:0/603551815 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:43:47.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:47.830+0000 7f2a7f7fe640 1 --2- 172.21.15.38:0/603551815 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2a80160100 0x7f2a801e93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:47.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:47.830+0000 7f2a7e7fc640 1 --2- 172.21.15.38:0/603551815 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2a8015cb10 0x7f2a8014f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:47.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:47.830+0000 7f2a7effd640 1 -- 172.21.15.38:0/603551815 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2a8015cb10 msgr2=0x7f2a8014f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:47.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:47.830+0000 7f2a7effd640 1 --2- 172.21.15.38:0/603551815 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2a8015cb10 0x7f2a8014f970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:47.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:47.830+0000 7f2a7effd640 1 -- 172.21.15.38:0/603551815 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2a80160100 msgr2=0x7f2a801e93f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:47.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:47.830+0000 7f2a7effd640 1 --2- 172.21.15.38:0/603551815 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2a80160100 0x7f2a801e93f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:47.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:47.830+0000 7f2a7effd640 1 -- 172.21.15.38:0/603551815 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a801571d0 con 0x7f2a800d12e0 2024-09-21T03:43:47.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:47.830+0000 7f2a7effd640 1 --2- 172.21.15.38:0/603551815 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2a800d12e0 0x7f2a8015c0f0 secure :-1 s=READY pgs=2087 cs=0 l=1 rev1=1 crypto rx=0x7f2a7000d700 tx=0x7f2a7000dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:47.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:47.830+0000 7f2a677fe640 1 -- 172.21.15.38:0/603551815 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a70017070 con 0x7f2a800d12e0 2024-09-21T03:43:47.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:47.830+0000 7f2a677fe640 1 -- 172.21.15.38:0/603551815 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2a70005310 con 0x7f2a800d12e0 2024-09-21T03:43:47.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:47.830+0000 7f2a85494640 1 -- 172.21.15.38:0/603551815 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a80157470 con 0x7f2a800d12e0 2024-09-21T03:43:47.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:47.830+0000 7f2a677fe640 1 -- 172.21.15.38:0/603551815 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a70009820 con 0x7f2a800d12e0 2024-09-21T03:43:47.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:47.830+0000 7f2a85494640 1 -- 172.21.15.38:0/603551815 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a80157a30 con 0x7f2a800d12e0 2024-09-21T03:43:47.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:47.832+0000 7f2a85494640 1 -- 172.21.15.38:0/603551815 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a800d16f0 con 0x7f2a800d12e0 2024-09-21T03:43:47.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:47.832+0000 7f2a677fe640 1 -- 172.21.15.38:0/603551815 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f2a700099c0 con 0x7f2a800d12e0 2024-09-21T03:43:47.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:47.833+0000 7f2a677fe640 1 --2- 172.21.15.38:0/603551815 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2a6c078bc0 0x7f2a6c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:47.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:47.833+0000 7f2a677fe640 1 -- 172.21.15.38:0/603551815 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 9142+0+0 (secure 0 0 0) 0x7f2a7009bc40 con 0x7f2a800d12e0 2024-09-21T03:43:47.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:47.833+0000 7f2a7e7fc640 1 --2- 172.21.15.38:0/603551815 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2a6c078bc0 0x7f2a6c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:47.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:47.834+0000 7f2a7e7fc640 1 --2- 172.21.15.38:0/603551815 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2a6c078bc0 0x7f2a6c07b0d0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f2a74033000 tx=0x7f2a74004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:47.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:47.838+0000 7f2a677fe640 1 -- 172.21.15.38:0/603551815 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f2a70062b90 con 0x7f2a800d12e0 2024-09-21T03:43:48.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.063+0000 7f2a85494640 1 -- 172.21.15.38:0/603551815 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2a800ce410 con 0x7f2a800d12e0 2024-09-21T03:43:48.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.064+0000 7f2a677fe640 1 -- 172.21.15.38:0/603551815 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 290 v290) ==== 95+0+5231 (secure 0 0 0) 0x7f2a700675c0 con 0x7f2a800d12e0 2024-09-21T03:43:48.066 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:43:48.066 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":290,"btime":"2024-09-21T03:43:45:672563+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6086,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/965972663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":965972663},{"type":"v1","addr":"172.21.15.195:6836","nonce":965972663}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6700,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/3176122334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":3176122334},{"type":"v1","addr":"172.21.15.195:6837","nonce":3176122334}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":9840,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/3801772046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":3801772046},{"type":"v1","addr":"172.21.15.38:6835","nonce":3801772046}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":273}],"filesystems":[{"mdsmap":{"epoch":290,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:40:02.319282+0000","modified":"2024-09-21T03:43:45.657079+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":129,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9999},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9999":{"gid":9999,"name":"c","rank":0,"incarnation":286,"state":"up:active","state_seq":5,"addr":"172.21.15.38:6833/626794495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":626794495},{"type":"v1","addr":"172.21.15.38:6833","nonce":626794495}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9999,"qdb_cluster":[9999]},"id":6}]} 2024-09-21T03:43:48.066 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 290 2024-09-21T03:43:48.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.066+0000 7f2a85494640 1 -- 172.21.15.38:0/603551815 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2a6c078bc0 msgr2=0x7f2a6c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:48.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.066+0000 7f2a85494640 1 --2- 172.21.15.38:0/603551815 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2a6c078bc0 0x7f2a6c07b0d0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f2a74033000 tx=0x7f2a74004730 comp rx=0 tx=0).stop 2024-09-21T03:43:48.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.066+0000 7f2a85494640 1 -- 172.21.15.38:0/603551815 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2a800d12e0 msgr2=0x7f2a8015c0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:48.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.066+0000 7f2a85494640 1 --2- 172.21.15.38:0/603551815 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2a800d12e0 0x7f2a8015c0f0 secure :-1 s=READY pgs=2087 cs=0 l=1 rev1=1 crypto rx=0x7f2a7000d700 tx=0x7f2a7000dbc0 comp rx=0 tx=0).stop 2024-09-21T03:43:48.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.066+0000 7f2a85494640 1 -- 172.21.15.38:0/603551815 shutdown_connections 2024-09-21T03:43:48.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.066+0000 7f2a85494640 1 --2- 172.21.15.38:0/603551815 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2a80160100 0x7f2a801e93f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:48.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.066+0000 7f2a85494640 1 --2- 172.21.15.38:0/603551815 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2a6c078bc0 0x7f2a6c07b0d0 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:48.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.066+0000 7f2a85494640 1 --2- 172.21.15.38:0/603551815 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2a8015cb10 0x7f2a8014f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:48.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.066+0000 7f2a85494640 1 --2- 172.21.15.38:0/603551815 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2a800d12e0 0x7f2a8015c0f0 unknown :-1 s=CLOSED pgs=2087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:48.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.066+0000 7f2a85494640 1 -- 172.21.15.38:0/603551815 >> 172.21.15.38:0/603551815 conn(0x7f2a800d2350 msgr2=0x7f2a80070970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:48.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.066+0000 7f2a85494640 1 -- 172.21.15.38:0/603551815 shutdown_connections 2024-09-21T03:43:48.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.066+0000 7f2a85494640 1 -- 172.21.15.38:0/603551815 wait complete. 2024-09-21T03:43:48.079 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c damage ls --format=json-pretty 2024-09-21T03:43:48.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.196+0000 7f1ff8f62640 1 -- 172.21.15.38:0/2994866456 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1fe0003a50 msgr2=0x7f1fe0003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:48.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.196+0000 7f1ff8f62640 1 --2- 172.21.15.38:0/2994866456 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1fe0003a50 0x7f1fe0003ec0 secure :-1 s=READY pgs=2089 cs=0 l=1 rev1=1 crypto rx=0x7f1fe4009b50 tx=0x7f1fe0005830 comp rx=0 tx=0).stop 2024-09-21T03:43:48.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.196+0000 7f1ff8f62640 1 -- 172.21.15.38:0/2994866456 shutdown_connections 2024-09-21T03:43:48.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.196+0000 7f1ff8f62640 1 --2- 172.21.15.38:0/2994866456 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1fe0003a50 0x7f1fe0003ec0 unknown :-1 s=CLOSED pgs=2089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:48.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.196+0000 7f1ff8f62640 1 -- 172.21.15.38:0/2994866456 >> 172.21.15.38:0/2994866456 conn(0x7f1ff409d830 msgr2=0x7f1ff40b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:48.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.196+0000 7f1ff8f62640 1 -- 172.21.15.38:0/2994866456 shutdown_connections 2024-09-21T03:43:48.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.196+0000 7f1ff8f62640 1 -- 172.21.15.38:0/2994866456 wait complete. 2024-09-21T03:43:48.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.198+0000 7f1ff8f62640 1 Processor -- start 2024-09-21T03:43:48.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.198+0000 7f1ff8f62640 1 -- start start 2024-09-21T03:43:48.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.199+0000 7f1ff8f62640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1fe0003a50 0x7f1ff40cb610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:48.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.199+0000 7f1ff8f62640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1ff41f94b0 0x7f1ff40ca4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:48.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.199+0000 7f1ff8f62640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1ff40d0e40 0x7f1ff40d1250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:48.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.199+0000 7f1ff8f62640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f1ff40cf120 con 0x7f1ff41f94b0 2024-09-21T03:43:48.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.199+0000 7f1ff8f62640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f1ff40cefa0 con 0x7f1fe0003a50 2024-09-21T03:43:48.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.199+0000 7f1ff8f62640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f1ff40cf2a0 con 0x7f1ff40d0e40 2024-09-21T03:43:48.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.200+0000 7f1febfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1ff41f94b0 0x7f1ff40ca4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:48.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.200+0000 7f1febfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1ff41f94b0 0x7f1ff40ca4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54562/0 (socket says 172.21.15.38:54562) 2024-09-21T03:43:48.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.200+0000 7f1febfff640 1 -- 172.21.15.38:0/322469018 learned_addr learned my addr 172.21.15.38:0/322469018 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:43:48.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.200+0000 7f1ff259c640 1 --2- 172.21.15.38:0/322469018 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1fe0003a50 0x7f1ff40cb610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:48.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.200+0000 7f1ff2d9d640 1 --2- 172.21.15.38:0/322469018 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1ff40d0e40 0x7f1ff40d1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:48.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.200+0000 7f1febfff640 1 -- 172.21.15.38:0/322469018 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1ff40d0e40 msgr2=0x7f1ff40d1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:48.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.200+0000 7f1febfff640 1 --2- 172.21.15.38:0/322469018 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1ff40d0e40 0x7f1ff40d1250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:48.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.200+0000 7f1febfff640 1 -- 172.21.15.38:0/322469018 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1fe0003a50 msgr2=0x7f1ff40cb610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:48.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.200+0000 7f1febfff640 1 --2- 172.21.15.38:0/322469018 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1fe0003a50 0x7f1ff40cb610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:48.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.200+0000 7f1febfff640 1 -- 172.21.15.38:0/322469018 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ff40d1790 con 0x7f1ff41f94b0 2024-09-21T03:43:48.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.200+0000 7f1febfff640 1 --2- 172.21.15.38:0/322469018 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1ff41f94b0 0x7f1ff40ca4f0 secure :-1 s=READY pgs=2090 cs=0 l=1 rev1=1 crypto rx=0x7f1fdc006df0 tx=0x7f1fdc0049d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:48.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.200+0000 7f1feaffd640 1 -- 172.21.15.38:0/322469018 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fdc015020 con 0x7f1ff41f94b0 2024-09-21T03:43:48.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.200+0000 7f1feaffd640 1 -- 172.21.15.38:0/322469018 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1fdc011980 con 0x7f1ff41f94b0 2024-09-21T03:43:48.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.200+0000 7f1ff8f62640 1 -- 172.21.15.38:0/322469018 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ff40cd230 con 0x7f1ff41f94b0 2024-09-21T03:43:48.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.200+0000 7f1feaffd640 1 -- 172.21.15.38:0/322469018 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fdc011dd0 con 0x7f1ff41f94b0 2024-09-21T03:43:48.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.200+0000 7f1ff8f62640 1 -- 172.21.15.38:0/322469018 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ff40d3e40 con 0x7f1ff41f94b0 2024-09-21T03:43:48.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.203+0000 7f1feaffd640 1 -- 172.21.15.38:0/322469018 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f1fdc018020 con 0x7f1ff41f94b0 2024-09-21T03:43:48.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.203+0000 7f1feaffd640 1 --2- 172.21.15.38:0/322469018 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1fd4078bc0 0x7f1fd407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:48.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.203+0000 7f1feaffd640 1 -- 172.21.15.38:0/322469018 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 9142+0+0 (secure 0 0 0) 0x7f1fdc09c350 con 0x7f1ff41f94b0 2024-09-21T03:43:48.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.204+0000 7f1ff259c640 1 --2- 172.21.15.38:0/322469018 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1fd4078bc0 0x7f1fd407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:48.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.204+0000 7f1fffed2740 1 Processor -- start 2024-09-21T03:43:48.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.204+0000 7f1fffed2740 1 -- start start 2024-09-21T03:43:48.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.204+0000 7f1ff259c640 1 --2- 172.21.15.38:0/322469018 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1fd4078bc0 0x7f1fd407b0d0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f1fe4035000 tx=0x7f1fe40046c0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:48.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.205+0000 7f1fffed2740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56521cce4350 0x56521d0b5d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:48.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.205+0000 7f1fffed2740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56521ccbb490 0x56521d0b62c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:48.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.205+0000 7f1fffed2740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56521ccc3670 0x56521d0d31c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:48.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.205+0000 7f1fffed2740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x56521cffe9a0 con 0x56521ccc3670 2024-09-21T03:43:48.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.205+0000 7f1fffed2740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x56521cfeddd0 con 0x56521cce4350 2024-09-21T03:43:48.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.205+0000 7f1fffed2740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x56521cff1be0 con 0x56521ccbb490 2024-09-21T03:43:48.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.206+0000 7f1ff2d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56521ccc3670 0x56521d0d31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:48.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.206+0000 7f1ff2d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56521ccc3670 0x56521d0d31c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54568/0 (socket says 172.21.15.38:54568) 2024-09-21T03:43:48.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.206+0000 7f1febfff640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56521ccbb490 0x56521d0b62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:48.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.206+0000 7f1ff2d9d640 1 -- 172.21.15.38:0/4247019410 learned_addr learned my addr 172.21.15.38:0/4247019410 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:43:48.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.206+0000 7f1ff259c640 1 --2- 172.21.15.38:0/4247019410 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56521cce4350 0x56521d0b5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:48.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.206+0000 7f1ff2d9d640 1 -- 172.21.15.38:0/4247019410 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56521ccbb490 msgr2=0x56521d0b62c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:48.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.206+0000 7f1ff2d9d640 1 --2- 172.21.15.38:0/4247019410 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56521ccbb490 0x56521d0b62c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:48.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.206+0000 7f1ff2d9d640 1 -- 172.21.15.38:0/4247019410 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56521cce4350 msgr2=0x56521d0b5d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:48.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.206+0000 7f1ff2d9d640 1 --2- 172.21.15.38:0/4247019410 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56521cce4350 0x56521d0b5d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:48.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.206+0000 7f1ff2d9d640 1 -- 172.21.15.38:0/4247019410 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56521cf86700 con 0x56521ccc3670 2024-09-21T03:43:48.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.206+0000 7f1ff2d9d640 1 --2- 172.21.15.38:0/4247019410 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56521ccc3670 0x56521d0d31c0 secure :-1 s=READY pgs=2091 cs=0 l=1 rev1=1 crypto rx=0x7f1ff408cbe0 tx=0x7f1fec004f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b33189bca5d00afc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:48.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.206+0000 7f1fd3fff640 1 -- 172.21.15.38:0/4247019410 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fec011990 con 0x56521ccc3670 2024-09-21T03:43:48.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.206+0000 7f1fd3fff640 1 -- 172.21.15.38:0/4247019410 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1fec016070 con 0x56521ccc3670 2024-09-21T03:43:48.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.206+0000 7f1fd3fff640 1 -- 172.21.15.38:0/4247019410 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fec011ea0 con 0x56521ccc3670 2024-09-21T03:43:48.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.207+0000 7f1fffed2740 1 -- 172.21.15.38:0/4247019410 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56521ccc3670 msgr2=0x56521d0d31c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:48.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.207+0000 7f1fffed2740 1 --2- 172.21.15.38:0/4247019410 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56521ccc3670 0x56521d0d31c0 secure :-1 s=READY pgs=2091 cs=0 l=1 rev1=1 crypto rx=0x7f1ff408cbe0 tx=0x7f1fec004f90 comp rx=0 tx=0).stop 2024-09-21T03:43:48.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.207+0000 7f1fffed2740 1 -- 172.21.15.38:0/4247019410 shutdown_connections 2024-09-21T03:43:48.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.207+0000 7f1fffed2740 1 --2- 172.21.15.38:0/4247019410 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56521ccc3670 0x56521d0d31c0 unknown :-1 s=CLOSED pgs=2091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:48.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.207+0000 7f1fffed2740 1 --2- 172.21.15.38:0/4247019410 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56521ccbb490 0x56521d0b62c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:48.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.207+0000 7f1fffed2740 1 --2- 172.21.15.38:0/4247019410 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56521cce4350 0x56521d0b5d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:48.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.207+0000 7f1fffed2740 1 -- 172.21.15.38:0/4247019410 >> 172.21.15.38:0/4247019410 conn(0x56521ccd84a0 msgr2=0x56521d08c040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:48.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.207+0000 7f1fffed2740 1 -- 172.21.15.38:0/4247019410 shutdown_connections 2024-09-21T03:43:48.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.207+0000 7f1fffed2740 1 -- 172.21.15.38:0/4247019410 wait complete. 2024-09-21T03:43:48.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.208+0000 7f1fffed2740 1 -- start start 2024-09-21T03:43:48.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.208+0000 7f1fffed2740 1 Processor -- start 2024-09-21T03:43:48.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.210+0000 7f1fffed2740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56521ccbb490 0x56521d0c95d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:48.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.210+0000 7f1fffed2740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56521ccc3670 0x56521d0ca820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:48.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.210+0000 7f1fffed2740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56521cce4350 0x56521d0e0370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:48.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.210+0000 7f1fffed2740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x56521cd5e700 con 0x56521cce4350 2024-09-21T03:43:48.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.210+0000 7f1fffed2740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x56521cf93340 con 0x56521ccbb490 2024-09-21T03:43:48.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.210+0000 7f1fffed2740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x56521d0ca050 con 0x56521ccc3670 2024-09-21T03:43:48.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.210+0000 7f1fffed2740 10 client.0 _ll_register_callbacks cb 0x56521cf91ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:43:48.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.210+0000 7f1ff2d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56521cce4350 0x56521d0e0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:48.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.210+0000 7f1ff2d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56521cce4350 0x56521d0e0370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54572/0 (socket says 172.21.15.38:54572) 2024-09-21T03:43:48.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.210+0000 7f1ff2d9d640 1 -- 172.21.15.38:0/3423832096 learned_addr learned my addr 172.21.15.38:0/3423832096 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:43:48.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.210+0000 7f1ff259c640 1 --2- 172.21.15.38:0/3423832096 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56521ccbb490 0x56521d0c95d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:48.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.210+0000 7f1febfff640 1 --2- 172.21.15.38:0/3423832096 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56521ccc3670 0x56521d0ca820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:48.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.210+0000 7f1ff2d9d640 1 -- 172.21.15.38:0/3423832096 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56521ccc3670 msgr2=0x56521d0ca820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:48.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.210+0000 7f1ff2d9d640 1 --2- 172.21.15.38:0/3423832096 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56521ccc3670 0x56521d0ca820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:48.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.210+0000 7f1ff2d9d640 1 -- 172.21.15.38:0/3423832096 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56521ccbb490 msgr2=0x56521d0c95d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:48.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.210+0000 7f1ff2d9d640 1 --2- 172.21.15.38:0/3423832096 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56521ccbb490 0x56521d0c95d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:48.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.210+0000 7f1ff2d9d640 1 -- 172.21.15.38:0/3423832096 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56521cdc87c0 con 0x56521cce4350 2024-09-21T03:43:48.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.210+0000 7f1febfff640 1 --2- 172.21.15.38:0/3423832096 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56521ccc3670 0x56521d0ca820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:43:48.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.210+0000 7f1ff259c640 1 --2- 172.21.15.38:0/3423832096 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56521ccbb490 0x56521d0c95d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:43:48.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.210+0000 7f1ff2d9d640 1 --2- 172.21.15.38:0/3423832096 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56521cce4350 0x56521d0e0370 secure :-1 s=READY pgs=2092 cs=0 l=1 rev1=1 crypto rx=0x7f1ff40d20f0 tx=0x7f1fec0119c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dc0fa2059c74f11 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:48.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.210+0000 7f1fd17fa640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-21T03:43:48.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.211+0000 7f1fd17fa640 1 -- 172.21.15.38:0/3423832096 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fec02a060 con 0x56521cce4350 2024-09-21T03:43:48.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.211+0000 7f1fd17fa640 1 -- 172.21.15.38:0/3423832096 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(129..129 src has 1..129) ==== 9142+0+0 (secure 0 0 0) 0x7f1fec011e30 con 0x56521cce4350 2024-09-21T03:43:48.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.211+0000 7f1fffed2740 1 -- 172.21.15.38:0/3423832096 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1fec016070 con 0x56521cce4350 2024-09-21T03:43:48.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.211+0000 7f1fd17fa640 1 -- 172.21.15.38:0/3423832096 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=290) ==== 24+0+0 (secure 0 0 0) 0x7f1fec031050 con 0x56521cce4350 2024-09-21T03:43:48.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.211+0000 7f1fffed2740 10 client.10092 fetch_fsmap learned FSMap version 290 2024-09-21T03:43:48.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.211+0000 7f1fffed2740 1 -- 172.21.15.38:0/3423832096 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=290,monmap=2+}) -- 0x56521ce28f50 con 0x56521cce4350 2024-09-21T03:43:48.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.211+0000 7f1fd17fa640 1 -- 172.21.15.38:0/3423832096 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1fec016070 con 0x56521cce4350 2024-09-21T03:43:48.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.211+0000 7f1fd17fa640 1 -- 172.21.15.38:0/3423832096 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 290) ==== 3502+0+0 (secure 0 0 0) 0x7f1fec012440 con 0x56521cce4350 2024-09-21T03:43:48.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.211+0000 7f1fffed2740 10 client.10092 fetch_fsmap finished waiting for FSMap version 290 2024-09-21T03:43:48.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.211+0000 7f1fffed2740 10 client.10092 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T03:43:48.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.211+0000 7f1fffed2740 20 client.10092 populate_metadata read hostname 'smithi038' 2024-09-21T03:43:48.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.211+0000 7f1fffed2740 1 --2- 172.21.15.38:0/3423832096 >> [v2:172.21.15.38:6832/626794495,v1:172.21.15.38:6833/626794495] conn(0x56521cca3130 0x56521d0d40d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:48.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.211+0000 7f1fffed2740 4 client.10092 mds_command: new command op to 9999 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:43:48.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.211+0000 7f1fffed2740 1 -- 172.21.15.38:0/3423832096 --> [v2:172.21.15.38:6832/626794495,v1:172.21.15.38:6833/626794495] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56521d0008f0 con 0x56521cca3130 2024-09-21T03:43:48.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.211+0000 7f1ff259c640 1 --2- 172.21.15.38:0/3423832096 >> [v2:172.21.15.38:6832/626794495,v1:172.21.15.38:6833/626794495] conn(0x56521cca3130 0x56521d0d40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:48.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.212+0000 7f1fd17fa640 10 client.10092 ms_handle_connect on v2:172.21.15.38:6832/626794495 2024-09-21T03:43:48.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.212+0000 7f1ff259c640 1 --2- 172.21.15.38:0/3423832096 >> [v2:172.21.15.38:6832/626794495,v1:172.21.15.38:6833/626794495] conn(0x56521cca3130 0x56521d0d40d0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb900c3cedea94cf server_cookie=c649d7c91ac796fc in_seq=0 out_seq=0 2024-09-21T03:43:48.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.212+0000 7f1fd17fa640 1 -- 172.21.15.38:0/3423832096 <== mds.0 v2:172.21.15.38:6832/626794495 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56521d0008f0 con 0x56521cca3130 2024-09-21T03:43:48.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.212+0000 7f1fd17fa640 10 client.10092 handle_command_reply: tid=0 2024-09-21T03:43:48.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.212+0000 7f1fffed2740 1 client.10092 shutdown 2024-09-21T03:43:48.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.213+0000 7f1fffed2740 1 -- 172.21.15.38:0/3423832096 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56521cce4350 msgr2=0x56521d0e0370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:48.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.213+0000 7f1fffed2740 1 --2- 172.21.15.38:0/3423832096 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56521cce4350 0x56521d0e0370 secure :-1 s=READY pgs=2092 cs=0 l=1 rev1=1 crypto rx=0x7f1ff40d20f0 tx=0x7f1fec0119c0 comp rx=0 tx=0).stop 2024-09-21T03:43:48.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.213+0000 7f1fffed2740 1 -- 172.21.15.38:0/3423832096 shutdown_connections 2024-09-21T03:43:48.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.213+0000 7f1fffed2740 1 --2- 172.21.15.38:0/3423832096 >> [v2:172.21.15.38:6832/626794495,v1:172.21.15.38:6833/626794495] conn(0x56521cca3130 0x56521d0d40d0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:48.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.213+0000 7f1fffed2740 1 --2- 172.21.15.38:0/3423832096 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56521cce4350 0x56521d0e0370 unknown :-1 s=CLOSED pgs=2092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:48.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.213+0000 7f1fffed2740 1 --2- 172.21.15.38:0/3423832096 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56521ccc3670 0x56521d0ca820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:48.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.213+0000 7f1fffed2740 1 --2- 172.21.15.38:0/3423832096 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56521ccbb490 0x56521d0c95d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:48.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.213+0000 7f1fd17fa640 0 client.10092 ms_handle_reset on v2:172.21.15.38:6832/626794495 2024-09-21T03:43:48.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.213+0000 7f1fffed2740 1 -- 172.21.15.38:0/3423832096 >> 172.21.15.38:0/3423832096 conn(0x56521ccd84a0 msgr2=0x56521d08bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:48.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.213+0000 7f1fffed2740 1 -- 172.21.15.38:0/3423832096 shutdown_connections 2024-09-21T03:43:48.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.213+0000 7f1fffed2740 1 -- 172.21.15.38:0/3423832096 wait complete. 2024-09-21T03:43:48.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.213+0000 7f1fffed2740 20 client.10092 trim_cache size 0 max 16384 2024-09-21T03:43:48.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.226+0000 7f1fffed2740 1 Processor -- start 2024-09-21T03:43:48.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.227+0000 7f1fffed2740 1 -- start start 2024-09-21T03:43:48.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.228+0000 7f1fffed2740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56521ccd84a0 0x56521d0cd370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:48.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.228+0000 7f1fffed2740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56521ccbb490 0x56521d0c8f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:48.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.228+0000 7f1fffed2740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56521ccc3670 0x56521d0c94a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:48.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.228+0000 7f1fffed2740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f1ff4203c60 con 0x56521ccbb490 2024-09-21T03:43:48.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.228+0000 7f1fffed2740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f1ff4202f40 con 0x56521ccd84a0 2024-09-21T03:43:48.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.228+0000 7f1fffed2740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f1ff4203590 con 0x56521ccc3670 2024-09-21T03:43:48.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.228+0000 7f1febfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56521ccbb490 0x56521d0c8f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:48.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.228+0000 7f1febfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56521ccbb490 0x56521d0c8f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54588/0 (socket says 172.21.15.38:54588) 2024-09-21T03:43:48.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.228+0000 7f1febfff640 1 -- 172.21.15.38:0/1515470842 learned_addr learned my addr 172.21.15.38:0/1515470842 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:43:48.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.228+0000 7f1ff259c640 1 --2- 172.21.15.38:0/1515470842 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56521ccd84a0 0x56521d0cd370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:48.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.228+0000 7f1ff2d9d640 1 --2- 172.21.15.38:0/1515470842 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56521ccc3670 0x56521d0c94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:48.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.228+0000 7f1febfff640 1 -- 172.21.15.38:0/1515470842 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56521ccc3670 msgr2=0x56521d0c94a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:48.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.228+0000 7f1febfff640 1 --2- 172.21.15.38:0/1515470842 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56521ccc3670 0x56521d0c94a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:48.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.228+0000 7f1febfff640 1 -- 172.21.15.38:0/1515470842 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56521ccd84a0 msgr2=0x56521d0cd370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:48.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.228+0000 7f1febfff640 1 --2- 172.21.15.38:0/1515470842 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56521ccd84a0 0x56521d0cd370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:48.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.228+0000 7f1febfff640 1 -- 172.21.15.38:0/1515470842 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56521cd2d660 con 0x56521ccbb490 2024-09-21T03:43:48.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.228+0000 7f1ff2d9d640 1 --2- 172.21.15.38:0/1515470842 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56521ccc3670 0x56521d0c94a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:43:48.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.228+0000 7f1febfff640 1 --2- 172.21.15.38:0/1515470842 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56521ccbb490 0x56521d0c8f60 secure :-1 s=READY pgs=2093 cs=0 l=1 rev1=1 crypto rx=0x7f1fdc001eb0 tx=0x7f1fdc09af90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b7542d211177694 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:48.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.229+0000 7f1fd37fe640 1 -- 172.21.15.38:0/1515470842 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fdc067f00 con 0x56521ccbb490 2024-09-21T03:43:48.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.229+0000 7f1fd37fe640 1 -- 172.21.15.38:0/1515470842 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1fdc05f740 con 0x56521ccbb490 2024-09-21T03:43:48.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.229+0000 7f1fd37fe640 1 -- 172.21.15.38:0/1515470842 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fdc062970 con 0x56521ccbb490 2024-09-21T03:43:48.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.229+0000 7f1fffed2740 1 -- 172.21.15.38:0/1515470842 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56521ccbb490 msgr2=0x56521d0c8f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:48.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.229+0000 7f1fffed2740 1 --2- 172.21.15.38:0/1515470842 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56521ccbb490 0x56521d0c8f60 secure :-1 s=READY pgs=2093 cs=0 l=1 rev1=1 crypto rx=0x7f1fdc001eb0 tx=0x7f1fdc09af90 comp rx=0 tx=0).stop 2024-09-21T03:43:48.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.229+0000 7f1fffed2740 1 -- 172.21.15.38:0/1515470842 shutdown_connections 2024-09-21T03:43:48.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.229+0000 7f1fffed2740 1 --2- 172.21.15.38:0/1515470842 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56521ccc3670 0x56521d0c94a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:48.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.229+0000 7f1fffed2740 1 --2- 172.21.15.38:0/1515470842 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56521ccbb490 0x56521d0c8f60 unknown :-1 s=CLOSED pgs=2093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:48.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.229+0000 7f1fffed2740 1 --2- 172.21.15.38:0/1515470842 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56521ccd84a0 0x56521d0cd370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:48.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.229+0000 7f1fffed2740 1 -- 172.21.15.38:0/1515470842 >> 172.21.15.38:0/1515470842 conn(0x56521d081f80 msgr2=0x56521d0e2a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:48.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.229+0000 7f1fffed2740 1 -- 172.21.15.38:0/1515470842 shutdown_connections 2024-09-21T03:43:48.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.229+0000 7f1fffed2740 1 -- 172.21.15.38:0/1515470842 wait complete. 2024-09-21T03:43:48.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.230+0000 7f1fffed2740 1 -- start start 2024-09-21T03:43:48.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.230+0000 7f1fffed2740 1 Processor -- start 2024-09-21T03:43:48.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.232+0000 7f1fffed2740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56521ccbb490 0x56521d14cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:48.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.232+0000 7f1fffed2740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56521ccc3670 0x56521d0cfc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:48.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.232+0000 7f1fffed2740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56521ccd84a0 0x56521d0c8f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:48.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.232+0000 7f1fffed2740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x56521d0e08b0 con 0x56521ccd84a0 2024-09-21T03:43:48.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.232+0000 7f1fffed2740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x56521d0c9df0 con 0x56521ccbb490 2024-09-21T03:43:48.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.232+0000 7f1fffed2740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x56521d0e32a0 con 0x56521ccc3670 2024-09-21T03:43:48.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.232+0000 7f1fffed2740 10 client.0 _ll_register_callbacks cb 0x7f1fe00036f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:43:48.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.232+0000 7f1ff2d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56521ccd84a0 0x56521d0c8f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:48.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.232+0000 7f1ff2d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56521ccd84a0 0x56521d0c8f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54600/0 (socket says 172.21.15.38:54600) 2024-09-21T03:43:48.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.232+0000 7f1ff2d9d640 1 -- 172.21.15.38:0/1245051604 learned_addr learned my addr 172.21.15.38:0/1245051604 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:43:48.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.232+0000 7f1ff259c640 1 --2- 172.21.15.38:0/1245051604 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56521ccbb490 0x56521d14cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:48.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.232+0000 7f1febfff640 1 --2- 172.21.15.38:0/1245051604 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56521ccc3670 0x56521d0cfc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:48.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.232+0000 7f1ff2d9d640 1 -- 172.21.15.38:0/1245051604 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56521ccc3670 msgr2=0x56521d0cfc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:48.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.232+0000 7f1ff2d9d640 1 --2- 172.21.15.38:0/1245051604 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56521ccc3670 0x56521d0cfc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:48.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.232+0000 7f1ff2d9d640 1 -- 172.21.15.38:0/1245051604 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56521ccbb490 msgr2=0x56521d14cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:48.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.232+0000 7f1ff2d9d640 1 --2- 172.21.15.38:0/1245051604 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56521ccbb490 0x56521d14cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:48.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.232+0000 7f1ff2d9d640 1 -- 172.21.15.38:0/1245051604 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56521d14d430 con 0x56521ccd84a0 2024-09-21T03:43:48.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.233+0000 7f1ff2d9d640 1 --2- 172.21.15.38:0/1245051604 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56521ccd84a0 0x56521d0c8f60 secure :-1 s=READY pgs=2094 cs=0 l=1 rev1=1 crypto rx=0x7f1ff40d20f0 tx=0x7f1fec011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=130d526553041709 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:48.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.233+0000 7f1fd17fa640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-21T03:43:48.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.233+0000 7f1fd17fa640 1 -- 172.21.15.38:0/1245051604 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fec02a040 con 0x56521ccd84a0 2024-09-21T03:43:48.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.233+0000 7f1fd17fa640 1 -- 172.21.15.38:0/1245051604 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(129..129 src has 1..129) ==== 9142+0+0 (secure 0 0 0) 0x7f1fec012440 con 0x56521ccd84a0 2024-09-21T03:43:48.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.233+0000 7f1fffed2740 1 -- 172.21.15.38:0/1245051604 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1fdc05f740 con 0x56521ccd84a0 2024-09-21T03:43:48.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.233+0000 7f1fd17fa640 1 -- 172.21.15.38:0/1245051604 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=290) ==== 24+0+0 (secure 0 0 0) 0x7f1fec03c050 con 0x56521ccd84a0 2024-09-21T03:43:48.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.233+0000 7f1fffed2740 10 client.10098 fetch_fsmap learned FSMap version 290 2024-09-21T03:43:48.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.233+0000 7f1fffed2740 1 -- 172.21.15.38:0/1245051604 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=290,monmap=2+}) -- 0x56521d14fd80 con 0x56521ccd84a0 2024-09-21T03:43:48.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.233+0000 7f1fd17fa640 1 -- 172.21.15.38:0/1245051604 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1fdc05f740 con 0x56521ccd84a0 2024-09-21T03:43:48.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.233+0000 7f1fd17fa640 1 -- 172.21.15.38:0/1245051604 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 290) ==== 3502+0+0 (secure 0 0 0) 0x7f1fec004ba0 con 0x56521ccd84a0 2024-09-21T03:43:48.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.233+0000 7f1fffed2740 10 client.10098 fetch_fsmap finished waiting for FSMap version 290 2024-09-21T03:43:48.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.233+0000 7f1fffed2740 10 client.10098 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T03:43:48.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.233+0000 7f1fffed2740 20 client.10098 populate_metadata read hostname 'smithi038' 2024-09-21T03:43:48.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.233+0000 7f1fffed2740 1 --2- 172.21.15.38:0/1245051604 >> [v2:172.21.15.38:6832/626794495,v1:172.21.15.38:6833/626794495] conn(0x56521cc78570 0x56521d0c94a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:48.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.233+0000 7f1fffed2740 4 client.10098 mds_command: new command op to 9999 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-09-21T03:43:48.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.233+0000 7f1fffed2740 1 -- 172.21.15.38:0/1245051604 --> [v2:172.21.15.38:6832/626794495,v1:172.21.15.38:6833/626794495] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) -- 0x56521cffc3a0 con 0x56521cc78570 2024-09-21T03:43:48.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.233+0000 7f1ff259c640 1 --2- 172.21.15.38:0/1245051604 >> [v2:172.21.15.38:6832/626794495,v1:172.21.15.38:6833/626794495] conn(0x56521cc78570 0x56521d0c94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:48.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.234+0000 7f1fd17fa640 10 client.10098 ms_handle_connect on v2:172.21.15.38:6832/626794495 2024-09-21T03:43:48.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.234+0000 7f1ff259c640 1 --2- 172.21.15.38:0/1245051604 >> [v2:172.21.15.38:6832/626794495,v1:172.21.15.38:6833/626794495] conn(0x56521cc78570 0x56521d0c94a0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61941882fd5bcb34 server_cookie=bac40cf04f4f676 in_seq=0 out_seq=0 2024-09-21T03:43:48.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.234+0000 7f1fd17fa640 1 -- 172.21.15.38:0/1245051604 <== mds.0 v2:172.21.15.38:6832/626794495 1 ==== command_reply(tid 0: 0 ) ==== 8+0+194 (crc 0 0 0) 0x56521cffc3a0 con 0x56521cc78570 2024-09-21T03:43:48.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.234+0000 7f1fd17fa640 10 client.10098 handle_command_reply: tid=0 2024-09-21T03:43:48.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.234+0000 7f1fffed2740 1 client.10098 shutdown 2024-09-21T03:43:48.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.234+0000 7f1fffed2740 1 -- 172.21.15.38:0/1245051604 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56521ccd84a0 msgr2=0x56521d0c8f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:48.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.234+0000 7f1fffed2740 1 --2- 172.21.15.38:0/1245051604 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56521ccd84a0 0x56521d0c8f60 secure :-1 s=READY pgs=2094 cs=0 l=1 rev1=1 crypto rx=0x7f1ff40d20f0 tx=0x7f1fec011ac0 comp rx=0 tx=0).stop 2024-09-21T03:43:48.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.234+0000 7f1fffed2740 1 -- 172.21.15.38:0/1245051604 shutdown_connections 2024-09-21T03:43:48.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.234+0000 7f1fffed2740 1 --2- 172.21.15.38:0/1245051604 >> [v2:172.21.15.38:6832/626794495,v1:172.21.15.38:6833/626794495] conn(0x56521cc78570 0x56521d0c94a0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:48.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.234+0000 7f1fffed2740 1 --2- 172.21.15.38:0/1245051604 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56521ccd84a0 0x56521d0c8f60 unknown :-1 s=CLOSED pgs=2094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:48.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.234+0000 7f1fffed2740 1 --2- 172.21.15.38:0/1245051604 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56521ccc3670 0x56521d0cfc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:48.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.234+0000 7f1fd17fa640 0 client.10098 ms_handle_reset on v2:172.21.15.38:6832/626794495 2024-09-21T03:43:48.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.234+0000 7f1fffed2740 1 --2- 172.21.15.38:0/1245051604 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56521ccbb490 0x56521d14cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:48.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.234+0000 7f1fffed2740 1 -- 172.21.15.38:0/1245051604 >> 172.21.15.38:0/1245051604 conn(0x56521d081f80 msgr2=0x56521d0e2a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:48.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.234+0000 7f1fffed2740 1 -- 172.21.15.38:0/1245051604 shutdown_connections 2024-09-21T03:43:48.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.234+0000 7f1fffed2740 1 -- 172.21.15.38:0/1245051604 wait complete. 2024-09-21T03:43:48.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.235+0000 7f1fffed2740 20 client.10098 trim_cache size 0 max 16384 2024-09-21T03:43:48.243 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:43:48.243 INFO:teuthology.orchestra.run.smithi038.stdout:[ 2024-09-21T03:43:48.244 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-21T03:43:48.244 INFO:teuthology.orchestra.run.smithi038.stdout: "damage_type": "dentry", 2024-09-21T03:43:48.244 INFO:teuthology.orchestra.run.smithi038.stdout: "id": 1389404921, 2024-09-21T03:43:48.244 INFO:teuthology.orchestra.run.smithi038.stdout: "ino": 1, 2024-09-21T03:43:48.244 INFO:teuthology.orchestra.run.smithi038.stdout: "frag": "*", 2024-09-21T03:43:48.244 INFO:teuthology.orchestra.run.smithi038.stdout: "dname": "subdir", 2024-09-21T03:43:48.244 INFO:teuthology.orchestra.run.smithi038.stdout: "snap_id": "head", 2024-09-21T03:43:48.244 INFO:teuthology.orchestra.run.smithi038.stdout: "path": "/subdir" 2024-09-21T03:43:48.244 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-21T03:43:48.244 INFO:teuthology.orchestra.run.smithi038.stdout:] 2024-09-21T03:43:48.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.235+0000 7f1ff8f62640 1 -- 172.21.15.38:0/322469018 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1fd4078bc0 msgr2=0x7f1fd407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:48.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.235+0000 7f1ff8f62640 1 --2- 172.21.15.38:0/322469018 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1fd4078bc0 0x7f1fd407b0d0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f1fe4035000 tx=0x7f1fe40046c0 comp rx=0 tx=0).stop 2024-09-21T03:43:48.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.235+0000 7f1ff8f62640 1 -- 172.21.15.38:0/322469018 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1ff41f94b0 msgr2=0x7f1ff40ca4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:48.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.235+0000 7f1ff8f62640 1 --2- 172.21.15.38:0/322469018 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1ff41f94b0 0x7f1ff40ca4f0 secure :-1 s=READY pgs=2090 cs=0 l=1 rev1=1 crypto rx=0x7f1fdc006df0 tx=0x7f1fdc0049d0 comp rx=0 tx=0).stop 2024-09-21T03:43:48.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.235+0000 7f1ff8f62640 1 -- 172.21.15.38:0/322469018 shutdown_connections 2024-09-21T03:43:48.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.235+0000 7f1ff8f62640 1 --2- 172.21.15.38:0/322469018 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1fd4078bc0 0x7f1fd407b0d0 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:48.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.235+0000 7f1ff8f62640 1 --2- 172.21.15.38:0/322469018 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1ff40d0e40 0x7f1ff40d1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:48.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.235+0000 7f1ff8f62640 1 --2- 172.21.15.38:0/322469018 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1ff41f94b0 0x7f1ff40ca4f0 unknown :-1 s=CLOSED pgs=2090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:48.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.235+0000 7f1ff8f62640 1 --2- 172.21.15.38:0/322469018 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1fe0003a50 0x7f1ff40cb610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:48.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.235+0000 7f1ff8f62640 1 -- 172.21.15.38:0/322469018 >> 172.21.15.38:0/322469018 conn(0x7f1ff409d830 msgr2=0x7f1ff4160130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:48.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.235+0000 7f1ff8f62640 1 -- 172.21.15.38:0/322469018 shutdown_connections 2024-09-21T03:43:48.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.235+0000 7f1ff8f62640 1 -- 172.21.15.38:0/322469018 wait complete. 2024-09-21T03:43:48.248 INFO:tasks.cephfs.test_damage:Applying mutation 'Corrupt omap header on 10000000000.00000000' 2024-09-21T03:43:48.248 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-21T03:43:48.248 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:43:48.265 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-21T03:43:48.265 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:43:48.321 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T03:43:48.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.476+0000 7fd5c2225640 1 -- 172.21.15.38:0/2593153641 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd5bc15ee80 msgr2=0x7fd5bc0d0170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:48.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.476+0000 7fd5c2225640 1 --2- 172.21.15.38:0/2593153641 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd5bc15ee80 0x7fd5bc0d0170 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fd5ac009830 tx=0x7fd5ac02a470 comp rx=0 tx=0).stop 2024-09-21T03:43:48.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.476+0000 7fd5c2225640 1 -- 172.21.15.38:0/2593153641 shutdown_connections 2024-09-21T03:43:48.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.476+0000 7fd5c2225640 1 --2- 172.21.15.38:0/2593153641 >> v2:172.21.15.195:3300/0 conn(0x7fd5bc0ceb90 0x7fd5bc0cefa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:48.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.476+0000 7fd5c2225640 1 --2- 172.21.15.38:0/2593153641 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd5bc15ee80 0x7fd5bc0d0170 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:48.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.476+0000 7fd5c2225640 1 -- 172.21.15.38:0/2593153641 >> 172.21.15.38:0/2593153641 conn(0x7fd5bc09d830 msgr2=0x7fd5bc0b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:48.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.476+0000 7fd5c2225640 1 -- 172.21.15.38:0/2593153641 shutdown_connections 2024-09-21T03:43:48.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.476+0000 7fd5c2225640 1 -- 172.21.15.38:0/2593153641 wait complete. 2024-09-21T03:43:48.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.478+0000 7fd5c2225640 1 Processor -- start 2024-09-21T03:43:48.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.478+0000 7fd5c2225640 1 -- start start 2024-09-21T03:43:48.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.479+0000 7fd5c2225640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd5bc0b9ad0 0x7fd5bc157c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:48.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.479+0000 7fd5c2225640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd5bc0ceb90 0x7fd5bc09e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:48.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.479+0000 7fd5c2225640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd5bc15ee80 0x7fd5bc09e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:48.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.479+0000 7fd5c2225640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fd5bc0b6700 con 0x7fd5bc0b9ad0 2024-09-21T03:43:48.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.479+0000 7fd5c2225640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd5bc0b6580 con 0x7fd5bc0ceb90 2024-09-21T03:43:48.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.479+0000 7fd5c2225640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fd5bc0b6880 con 0x7fd5bc15ee80 2024-09-21T03:43:48.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.479+0000 7fd5bb7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd5bc0b9ad0 0x7fd5bc157c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:48.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.479+0000 7fd5bb7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd5bc0b9ad0 0x7fd5bc157c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54602/0 (socket says 172.21.15.38:54602) 2024-09-21T03:43:48.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.479+0000 7fd5bb7fe640 1 -- 172.21.15.38:0/2519439972 learned_addr learned my addr 172.21.15.38:0/2519439972 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:43:48.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.480+0000 7fd5bbfff640 1 --2- 172.21.15.38:0/2519439972 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd5bc15ee80 0x7fd5bc09e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:48.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.480+0000 7fd5baffd640 1 --2- 172.21.15.38:0/2519439972 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd5bc0ceb90 0x7fd5bc09e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:48.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.480+0000 7fd5bb7fe640 1 -- 172.21.15.38:0/2519439972 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd5bc15ee80 msgr2=0x7fd5bc09e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:48.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.480+0000 7fd5bb7fe640 1 --2- 172.21.15.38:0/2519439972 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd5bc15ee80 0x7fd5bc09e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:48.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.480+0000 7fd5bb7fe640 1 -- 172.21.15.38:0/2519439972 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd5bc0ceb90 msgr2=0x7fd5bc09e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:48.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.480+0000 7fd5bb7fe640 1 --2- 172.21.15.38:0/2519439972 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd5bc0ceb90 0x7fd5bc09e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:48.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.480+0000 7fd5bb7fe640 1 -- 172.21.15.38:0/2519439972 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5bc0cdbd0 con 0x7fd5bc0b9ad0 2024-09-21T03:43:48.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.480+0000 7fd5baffd640 1 --2- 172.21.15.38:0/2519439972 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd5bc0ceb90 0x7fd5bc09e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:43:48.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.480+0000 7fd5bb7fe640 1 --2- 172.21.15.38:0/2519439972 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd5bc0b9ad0 0x7fd5bc157c30 secure :-1 s=READY pgs=2096 cs=0 l=1 rev1=1 crypto rx=0x7fd5ac009950 tx=0x7fd5ac033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:48.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.480+0000 7fd5a3fff640 1 -- 172.21.15.38:0/2519439972 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5ac005c90 con 0x7fd5bc0b9ad0 2024-09-21T03:43:48.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.480+0000 7fd5a3fff640 1 -- 172.21.15.38:0/2519439972 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd5ac031e50 con 0x7fd5bc0b9ad0 2024-09-21T03:43:48.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.480+0000 7fd5a3fff640 1 -- 172.21.15.38:0/2519439972 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5ac005260 con 0x7fd5bc0b9ad0 2024-09-21T03:43:48.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.480+0000 7fd5c2225640 1 -- 172.21.15.38:0/2519439972 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5bc06e200 con 0x7fd5bc0b9ad0 2024-09-21T03:43:48.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.480+0000 7fd5c2225640 1 -- 172.21.15.38:0/2519439972 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5bc06e4a0 con 0x7fd5bc0b9ad0 2024-09-21T03:43:48.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.481+0000 7fd5c2225640 1 -- 172.21.15.38:0/2519439972 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5bc0cf660 con 0x7fd5bc0b9ad0 2024-09-21T03:43:48.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.482+0000 7fd5a3fff640 1 -- 172.21.15.38:0/2519439972 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fd5ac005400 con 0x7fd5bc0b9ad0 2024-09-21T03:43:48.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.482+0000 7fd5a3fff640 1 --2- 172.21.15.38:0/2519439972 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd5a8078b90 0x7fd5a807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:48.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.483+0000 7fd5a3fff640 1 -- 172.21.15.38:0/2519439972 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 9142+0+0 (secure 0 0 0) 0x7fd5ac0bcae0 con 0x7fd5bc0b9ad0 2024-09-21T03:43:48.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.483+0000 7fd5baffd640 1 --2- 172.21.15.38:0/2519439972 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd5a8078b90 0x7fd5a807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:48.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.483+0000 7fd5baffd640 1 --2- 172.21.15.38:0/2519439972 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd5a8078b90 0x7fd5a807b0a0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fd5b0001470 tx=0x7fd5b0009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:48.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.487+0000 7fd5a3fff640 1 -- 172.21.15.38:0/2519439972 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fd5ac03f030 con 0x7fd5bc0b9ad0 2024-09-21T03:43:48.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.686+0000 7fd5c2225640 1 -- 172.21.15.38:0/2519439972 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fd5bc0b6da0 con 0x7fd5bc0b9ad0 2024-09-21T03:43:48.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.686+0000 7fd5a3fff640 1 -- 172.21.15.38:0/2519439972 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 66 entries v129) ==== 81+0+4012 (secure 0 0 0) 0x7fd5ac083c30 con 0x7fd5bc0b9ad0 2024-09-21T03:43:48.688 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6832/1643781179 2024-09-22T03:43:33.596532+0000 2024-09-21T03:43:48.689 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6833/2309772457 2024-09-22T03:43:28.198223+0000 2024-09-21T03:43:48.689 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6832/2309772457 2024-09-22T03:43:28.198223+0000 2024-09-21T03:43:48.689 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6835/141447142 2024-09-22T03:43:14.684929+0000 2024-09-21T03:43:48.689 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6834/141447142 2024-09-22T03:43:14.684929+0000 2024-09-21T03:43:48.689 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6835/3572946698 2024-09-22T03:41:44.737273+0000 2024-09-21T03:43:48.689 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6833/3553953266 2024-09-22T03:41:29.639383+0000 2024-09-21T03:43:48.689 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6834/2751962827 2024-09-22T03:40:38.029580+0000 2024-09-21T03:43:48.689 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6832/3553953266 2024-09-22T03:41:29.639383+0000 2024-09-21T03:43:48.689 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6834/2930805359 2024-09-22T03:42:23.215190+0000 2024-09-21T03:43:48.689 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6833/1643781179 2024-09-22T03:43:33.596532+0000 2024-09-21T03:43:48.689 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6835/2748959970 2024-09-22T03:41:16.565736+0000 2024-09-21T03:43:48.689 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6833/3710736169 2024-09-22T03:40:59.454480+0000 2024-09-21T03:43:48.690 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6833/1995903506 2024-09-22T03:43:00.406120+0000 2024-09-21T03:43:48.690 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6833/775682959 2024-09-22T03:43:10.570678+0000 2024-09-21T03:43:48.690 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6834/2748959970 2024-09-22T03:41:16.565736+0000 2024-09-21T03:43:48.690 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6834/2934569248 2024-09-22T03:40:26.934591+0000 2024-09-21T03:43:48.690 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6832/3726830644 2024-09-22T03:41:12.532341+0000 2024-09-21T03:43:48.690 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6835/2097789246 2024-09-22T03:43:06.604471+0000 2024-09-21T03:43:48.690 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6835/2934569248 2024-09-22T03:40:26.934591+0000 2024-09-21T03:43:48.690 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6835/3864487171 2024-09-22T03:40:31.989837+0000 2024-09-21T03:43:48.690 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6833/4221150556 2024-09-22T03:42:27.240627+0000 2024-09-21T03:43:48.690 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6832/121492560 2024-09-22T03:41:10.371933+0000 2024-09-21T03:43:48.690 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6832/1717955365 2024-09-22T03:41:51.787804+0000 2024-09-21T03:43:48.691 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6834/2418225210 2024-09-22T03:42:02.869612+0000 2024-09-21T03:43:48.691 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6835/2418225210 2024-09-22T03:42:02.869612+0000 2024-09-21T03:43:48.691 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6833/121492560 2024-09-22T03:41:10.371933+0000 2024-09-21T03:43:48.691 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6835/3951934534 2024-09-22T03:42:31.276099+0000 2024-09-21T03:43:48.691 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6833/751618334 2024-09-22T03:40:42.053279+0000 2024-09-21T03:43:48.691 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.195:6837/712370966 2024-09-22T03:40:11.633787+0000 2024-09-21T03:43:48.691 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6832/3402963947 2024-09-22T03:41:23.555005+0000 2024-09-21T03:43:48.691 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6835/2798021708 2024-09-22T03:42:47.436632+0000 2024-09-21T03:43:48.691 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6833/3726830644 2024-09-22T03:41:12.532341+0000 2024-09-21T03:43:48.691 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6834/2916238332 2024-09-22T03:41:42.553770+0000 2024-09-21T03:43:48.691 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6834/3055025811 2024-09-22T03:42:39.361236+0000 2024-09-21T03:43:48.691 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.195:6835/712370966 2024-09-22T03:40:11.633787+0000 2024-09-21T03:43:48.691 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6834/3864487171 2024-09-22T03:40:31.989837+0000 2024-09-21T03:43:48.691 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6834/3572946698 2024-09-22T03:41:44.737273+0000 2024-09-21T03:43:48.692 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6832/1995903506 2024-09-22T03:43:00.406120+0000 2024-09-21T03:43:48.692 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6832/775682959 2024-09-22T03:43:10.570678+0000 2024-09-21T03:43:48.692 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6835/2916238332 2024-09-22T03:41:42.553770+0000 2024-09-21T03:43:48.692 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6833/3402963947 2024-09-22T03:41:23.555005+0000 2024-09-21T03:43:48.692 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6834/425672974 2024-09-22T03:40:46.105521+0000 2024-09-21T03:43:48.692 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6834/3951934534 2024-09-22T03:42:31.276099+0000 2024-09-21T03:43:48.692 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6835/425672974 2024-09-22T03:40:46.105521+0000 2024-09-21T03:43:48.692 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6832/3710736169 2024-09-22T03:40:59.454480+0000 2024-09-21T03:43:48.692 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6833/1717955365 2024-09-22T03:41:51.787804+0000 2024-09-21T03:43:48.692 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6835/3055025811 2024-09-22T03:42:39.361236+0000 2024-09-21T03:43:48.692 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6835/2751962827 2024-09-22T03:40:38.029580+0000 2024-09-21T03:43:48.692 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6835/2712360296 2024-09-22T03:42:20.931115+0000 2024-09-21T03:43:48.692 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6833/3537258034 2024-09-22T03:42:06.901189+0000 2024-09-21T03:43:48.692 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6832/3537258034 2024-09-22T03:42:06.901189+0000 2024-09-21T03:43:48.692 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6832/1272005650 2024-09-22T03:41:58.836862+0000 2024-09-21T03:43:48.693 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6834/2712360296 2024-09-22T03:42:20.931115+0000 2024-09-21T03:43:48.693 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6832/751618334 2024-09-22T03:40:42.053279+0000 2024-09-21T03:43:48.693 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6835/2930805359 2024-09-22T03:42:23.215190+0000 2024-09-21T03:43:48.693 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6832/4221150556 2024-09-22T03:42:27.240627+0000 2024-09-21T03:43:48.693 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6833/1272005650 2024-09-22T03:41:58.836862+0000 2024-09-21T03:43:48.693 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6832/245538755 2024-09-22T03:42:35.318756+0000 2024-09-21T03:43:48.693 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6833/245538755 2024-09-22T03:42:35.318756+0000 2024-09-21T03:43:48.693 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6833/2459386631 2024-09-22T03:42:43.395609+0000 2024-09-21T03:43:48.693 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6832/2459386631 2024-09-22T03:42:43.395609+0000 2024-09-21T03:43:48.693 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6834/2097789246 2024-09-22T03:43:06.604471+0000 2024-09-21T03:43:48.693 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6834/2798021708 2024-09-22T03:42:47.436632+0000 2024-09-21T03:43:48.693 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6832/111206731 2024-09-22T03:43:02.560071+0000 2024-09-21T03:43:48.693 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6833/111206731 2024-09-22T03:43:02.560071+0000 2024-09-21T03:43:48.694 INFO:teuthology.orchestra.run.smithi038.stderr:listed 66 entries 2024-09-21T03:43:48.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.689+0000 7fd5c2225640 1 -- 172.21.15.38:0/2519439972 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd5a8078b90 msgr2=0x7fd5a807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:48.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.689+0000 7fd5c2225640 1 --2- 172.21.15.38:0/2519439972 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd5a8078b90 0x7fd5a807b0a0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fd5b0001470 tx=0x7fd5b0009040 comp rx=0 tx=0).stop 2024-09-21T03:43:48.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.689+0000 7fd5c2225640 1 -- 172.21.15.38:0/2519439972 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd5bc0b9ad0 msgr2=0x7fd5bc157c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:48.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.689+0000 7fd5c2225640 1 --2- 172.21.15.38:0/2519439972 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd5bc0b9ad0 0x7fd5bc157c30 secure :-1 s=READY pgs=2096 cs=0 l=1 rev1=1 crypto rx=0x7fd5ac009950 tx=0x7fd5ac033c20 comp rx=0 tx=0).stop 2024-09-21T03:43:48.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.689+0000 7fd5c2225640 1 -- 172.21.15.38:0/2519439972 shutdown_connections 2024-09-21T03:43:48.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.689+0000 7fd5c2225640 1 --2- 172.21.15.38:0/2519439972 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd5a8078b90 0x7fd5a807b0a0 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:48.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.689+0000 7fd5c2225640 1 --2- 172.21.15.38:0/2519439972 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd5bc15ee80 0x7fd5bc09e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:48.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.689+0000 7fd5c2225640 1 --2- 172.21.15.38:0/2519439972 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd5bc0ceb90 0x7fd5bc09e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:48.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.689+0000 7fd5c2225640 1 --2- 172.21.15.38:0/2519439972 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd5bc0b9ad0 0x7fd5bc157c30 unknown :-1 s=CLOSED pgs=2096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:48.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.689+0000 7fd5c2225640 1 -- 172.21.15.38:0/2519439972 >> 172.21.15.38:0/2519439972 conn(0x7fd5bc09d830 msgr2=0x7fd5bc0d32e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:48.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.689+0000 7fd5c2225640 1 -- 172.21.15.38:0/2519439972 shutdown_connections 2024-09-21T03:43:48.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.689+0000 7fd5c2225640 1 -- 172.21.15.38:0/2519439972 wait complete. 2024-09-21T03:43:48.702 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-21T03:43:48.702 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:43:48.702 DEBUG:teuthology.orchestra.run.smithi038:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-09-21T03:43:48.728 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi038.front.sepia.ceph.com 2024-09-21T03:43:48.728 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:43:48.728 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T03:43:48.786 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-21T03:43:48.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.943+0000 7fcd33943640 1 -- 172.21.15.38:0/2398979561 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcd2c15ca90 msgr2=0x7fcd2c0d0670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:48.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.943+0000 7fcd33943640 1 --2- 172.21.15.38:0/2398979561 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcd2c15ca90 0x7fcd2c0d0670 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fcd1c009830 tx=0x7fcd1c02a480 comp rx=0 tx=0).stop 2024-09-21T03:43:48.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.943+0000 7fcd33943640 1 -- 172.21.15.38:0/2398979561 shutdown_connections 2024-09-21T03:43:48.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.943+0000 7fcd33943640 1 --2- 172.21.15.38:0/2398979561 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcd2c0d0bb0 0x7fcd2c0b5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:48.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.943+0000 7fcd33943640 1 --2- 172.21.15.38:0/2398979561 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcd2c15ca90 0x7fcd2c0d0670 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:48.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.943+0000 7fcd33943640 1 -- 172.21.15.38:0/2398979561 >> 172.21.15.38:0/2398979561 conn(0x7fcd2c0b46e0 msgr2=0x7fcd2c0d23e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:48.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.943+0000 7fcd33943640 1 -- 172.21.15.38:0/2398979561 shutdown_connections 2024-09-21T03:43:48.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.943+0000 7fcd33943640 1 -- 172.21.15.38:0/2398979561 wait complete. 2024-09-21T03:43:48.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.945+0000 7fcd33943640 1 Processor -- start 2024-09-21T03:43:48.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.945+0000 7fcd33943640 1 -- start start 2024-09-21T03:43:48.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.946+0000 7fcd33943640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcd2c0ccc40 0x7fcd2c14d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:48.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.946+0000 7fcd33943640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcd2c0d0bb0 0x7fcd2c14fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:48.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.946+0000 7fcd33943640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcd2c15ca90 0x7fcd2c150070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:48.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.946+0000 7fcd33943640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fcd2c0b66a0 con 0x7fcd2c0d0bb0 2024-09-21T03:43:48.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.946+0000 7fcd33943640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fcd2c0b6520 con 0x7fcd2c15ca90 2024-09-21T03:43:48.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.946+0000 7fcd33943640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fcd2c0b6820 con 0x7fcd2c0ccc40 2024-09-21T03:43:48.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.946+0000 7fcd30ede640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcd2c0d0bb0 0x7fcd2c14fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:48.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.946+0000 7fcd30ede640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcd2c0d0bb0 0x7fcd2c14fb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54612/0 (socket says 172.21.15.38:54612) 2024-09-21T03:43:48.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.946+0000 7fcd30ede640 1 -- 172.21.15.38:0/1277908332 learned_addr learned my addr 172.21.15.38:0/1277908332 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:43:48.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.947+0000 7fcd31ee0640 1 --2- 172.21.15.38:0/1277908332 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcd2c15ca90 0x7fcd2c150070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:48.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.947+0000 7fcd316df640 1 --2- 172.21.15.38:0/1277908332 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcd2c0ccc40 0x7fcd2c14d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:48.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.947+0000 7fcd30ede640 1 -- 172.21.15.38:0/1277908332 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcd2c0ccc40 msgr2=0x7fcd2c14d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:48.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.947+0000 7fcd30ede640 1 --2- 172.21.15.38:0/1277908332 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcd2c0ccc40 0x7fcd2c14d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:48.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.947+0000 7fcd30ede640 1 -- 172.21.15.38:0/1277908332 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcd2c15ca90 msgr2=0x7fcd2c150070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:48.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.947+0000 7fcd30ede640 1 --2- 172.21.15.38:0/1277908332 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcd2c15ca90 0x7fcd2c150070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:48.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.947+0000 7fcd30ede640 1 -- 172.21.15.38:0/1277908332 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd2c14f630 con 0x7fcd2c0d0bb0 2024-09-21T03:43:48.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.947+0000 7fcd30ede640 1 --2- 172.21.15.38:0/1277908332 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcd2c0d0bb0 0x7fcd2c14fb30 secure :-1 s=READY pgs=2097 cs=0 l=1 rev1=1 crypto rx=0x7fcd1c000c00 tx=0x7fcd1c033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:48.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.947+0000 7fcd21ffb640 1 -- 172.21.15.38:0/1277908332 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd1c031d60 con 0x7fcd2c0d0bb0 2024-09-21T03:43:48.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.947+0000 7fcd21ffb640 1 -- 172.21.15.38:0/1277908332 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcd1c033e10 con 0x7fcd2c0d0bb0 2024-09-21T03:43:48.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.947+0000 7fcd33943640 1 -- 172.21.15.38:0/1277908332 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd2c1f3c00 con 0x7fcd2c0d0bb0 2024-09-21T03:43:48.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.947+0000 7fcd21ffb640 1 -- 172.21.15.38:0/1277908332 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd1c0332b0 con 0x7fcd2c0d0bb0 2024-09-21T03:43:48.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.948+0000 7fcd33943640 1 -- 172.21.15.38:0/1277908332 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd2c1f41c0 con 0x7fcd2c0d0bb0 2024-09-21T03:43:48.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.949+0000 7fcd33943640 1 -- 172.21.15.38:0/1277908332 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd2c0b6db0 con 0x7fcd2c0d0bb0 2024-09-21T03:43:48.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.949+0000 7fcd21ffb640 1 -- 172.21.15.38:0/1277908332 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fcd1c033450 con 0x7fcd2c0d0bb0 2024-09-21T03:43:48.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.950+0000 7fcd21ffb640 1 --2- 172.21.15.38:0/1277908332 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fcd10078b90 0x7fcd1007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:48.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.950+0000 7fcd21ffb640 1 -- 172.21.15.38:0/1277908332 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 9142+0+0 (secure 0 0 0) 0x7fcd1c0bcdb0 con 0x7fcd2c0d0bb0 2024-09-21T03:43:48.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.950+0000 7fcd316df640 1 --2- 172.21.15.38:0/1277908332 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fcd10078b90 0x7fcd1007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:48.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.951+0000 7fcd316df640 1 --2- 172.21.15.38:0/1277908332 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fcd10078b90 0x7fcd1007b0a0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fcd28001720 tx=0x7fcd28009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:48.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:48.955+0000 7fcd21ffb640 1 -- 172.21.15.38:0/1277908332 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fcd1c03f030 con 0x7fcd2c0d0bb0 2024-09-21T03:43:49.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:49.179+0000 7fcd33943640 1 -- 172.21.15.38:0/1277908332 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fcd2c1f4390 con 0x7fcd2c0d0bb0 2024-09-21T03:43:49.181 INFO:tasks.ceph.mon.a.smithi038.stderr:2024-09-21T03:43:49.179+0000 7f3f45713640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T03:43:49.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:49.182+0000 7fcd21ffb640 1 -- 172.21.15.38:0/1277908332 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v291) ==== 161+0+0 (secure 0 0 0) 0x7fcd1c083de0 con 0x7fcd2c0d0bb0 2024-09-21T03:43:49.184 INFO:teuthology.orchestra.run.smithi038.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-21T03:43:49.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:49.185+0000 7fcd33943640 1 -- 172.21.15.38:0/1277908332 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fcd10078b90 msgr2=0x7fcd1007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:49.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:49.185+0000 7fcd33943640 1 --2- 172.21.15.38:0/1277908332 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fcd10078b90 0x7fcd1007b0a0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fcd28001720 tx=0x7fcd28009040 comp rx=0 tx=0).stop 2024-09-21T03:43:49.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:49.185+0000 7fcd33943640 1 -- 172.21.15.38:0/1277908332 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcd2c0d0bb0 msgr2=0x7fcd2c14fb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:49.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:49.185+0000 7fcd33943640 1 --2- 172.21.15.38:0/1277908332 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcd2c0d0bb0 0x7fcd2c14fb30 secure :-1 s=READY pgs=2097 cs=0 l=1 rev1=1 crypto rx=0x7fcd1c000c00 tx=0x7fcd1c033c20 comp rx=0 tx=0).stop 2024-09-21T03:43:49.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:49.186+0000 7fcd33943640 1 -- 172.21.15.38:0/1277908332 shutdown_connections 2024-09-21T03:43:49.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:49.186+0000 7fcd33943640 1 --2- 172.21.15.38:0/1277908332 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcd2c15ca90 0x7fcd2c150070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:49.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:49.186+0000 7fcd33943640 1 --2- 172.21.15.38:0/1277908332 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcd2c0d0bb0 0x7fcd2c14fb30 unknown :-1 s=CLOSED pgs=2097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:49.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:49.186+0000 7fcd33943640 1 --2- 172.21.15.38:0/1277908332 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fcd10078b90 0x7fcd1007b0a0 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:49.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:49.186+0000 7fcd33943640 1 --2- 172.21.15.38:0/1277908332 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcd2c0ccc40 0x7fcd2c14d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:49.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:49.186+0000 7fcd33943640 1 -- 172.21.15.38:0/1277908332 >> 172.21.15.38:0/1277908332 conn(0x7fcd2c0b46e0 msgr2=0x7fcd2c0c6fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:49.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:49.186+0000 7fcd33943640 1 -- 172.21.15.38:0/1277908332 shutdown_connections 2024-09-21T03:43:49.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:49.186+0000 7fcd33943640 1 -- 172.21.15.38:0/1277908332 wait complete. 2024-09-21T03:43:49.189 INFO:tasks.ceph.mds.c.smithi038.stderr: -592> 2024-09-21T03:43:43.625+0000 7ff2466da640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T03:43:49.200 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-09-21T03:43:49.247 INFO:tasks.ceph.mds.c.smithi038.stdout:starting mds.c at 2024-09-21T03:43:49.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:49.319+0000 7f091a84d640 1 -- 172.21.15.38:0/742091787 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f091406def0 msgr2=0x7f091406f410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:49.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:49.319+0000 7f091a84d640 1 --2- 172.21.15.38:0/742091787 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f091406def0 0x7f091406f410 secure :-1 s=READY pgs=2100 cs=0 l=1 rev1=1 crypto rx=0x7f0914073190 tx=0x7f090402a510 comp rx=0 tx=0).stop 2024-09-21T03:43:49.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:49.319+0000 7f091a84d640 1 -- 172.21.15.38:0/742091787 shutdown_connections 2024-09-21T03:43:49.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:49.319+0000 7f091a84d640 1 --2- 172.21.15.38:0/742091787 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f09140b5260 0x7f091415ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:49.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:49.319+0000 7f091a84d640 1 --2- 172.21.15.38:0/742091787 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f091406def0 0x7f091406f410 unknown :-1 s=CLOSED pgs=2100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:49.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:49.319+0000 7f091a84d640 1 -- 172.21.15.38:0/742091787 >> 172.21.15.38:0/742091787 conn(0x7f09140cdca0 msgr2=0x7f09140ce0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:49.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:49.319+0000 7f091a84d640 1 -- 172.21.15.38:0/742091787 shutdown_connections 2024-09-21T03:43:49.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:49.319+0000 7f091a84d640 1 -- 172.21.15.38:0/742091787 wait complete. 2024-09-21T03:43:49.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:49.321+0000 7f091a84d640 1 Processor -- start 2024-09-21T03:43:49.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:49.321+0000 7f091a84d640 1 -- start start 2024-09-21T03:43:49.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:49.323+0000 7f091a84d640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f091406def0 0x7f09141f4190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:49.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:49.323+0000 7f091a84d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f09140b5260 0x7f09140b22f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:49.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:49.323+0000 7f091a84d640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f091415bfc0 0x7f09140b2830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:49.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:49.323+0000 7f091a84d640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f09140b5a00 con 0x7f09140b5260 2024-09-21T03:43:49.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:49.323+0000 7f091a84d640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f09140b5880 con 0x7f091406def0 2024-09-21T03:43:49.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:49.323+0000 7f091a84d640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f0914071300 con 0x7f091415bfc0 2024-09-21T03:43:49.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:49.323+0000 7f09137fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f09140b5260 0x7f09140b22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:49.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:49.323+0000 7f09137fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f09140b5260 0x7f09140b22f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54660/0 (socket says 172.21.15.38:54660) 2024-09-21T03:43:49.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:49.323+0000 7f09137fe640 1 -- 172.21.15.38:0/2129345546 learned_addr learned my addr 172.21.15.38:0/2129345546 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:43:49.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:49.323+0000 7f0913fff640 1 --2- 172.21.15.38:0/2129345546 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f091406def0 0x7f09141f4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:49.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:49.323+0000 7f0918dea640 1 --2- 172.21.15.38:0/2129345546 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f091415bfc0 0x7f09140b2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:49.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:49.323+0000 7f09137fe640 1 -- 172.21.15.38:0/2129345546 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f091415bfc0 msgr2=0x7f09140b2830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:49.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:49.323+0000 7f09137fe640 1 --2- 172.21.15.38:0/2129345546 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f091415bfc0 0x7f09140b2830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:49.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:49.323+0000 7f09137fe640 1 -- 172.21.15.38:0/2129345546 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f091406def0 msgr2=0x7f09141f4190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:49.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:49.323+0000 7f09137fe640 1 --2- 172.21.15.38:0/2129345546 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f091406def0 0x7f09141f4190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:49.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:49.323+0000 7f09137fe640 1 -- 172.21.15.38:0/2129345546 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f091414a010 con 0x7f09140b5260 2024-09-21T03:43:49.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:49.323+0000 7f09137fe640 1 --2- 172.21.15.38:0/2129345546 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f09140b5260 0x7f09140b22f0 secure :-1 s=READY pgs=2101 cs=0 l=1 rev1=1 crypto rx=0x7f090800b410 tx=0x7f090800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:49.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:49.324+0000 7f0910ff9640 1 -- 172.21.15.38:0/2129345546 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0908015020 con 0x7f09140b5260 2024-09-21T03:43:49.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:49.324+0000 7f0910ff9640 1 -- 172.21.15.38:0/2129345546 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0908011940 con 0x7f09140b5260 2024-09-21T03:43:49.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:49.324+0000 7f0910ff9640 1 -- 172.21.15.38:0/2129345546 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0908011d90 con 0x7f09140b5260 2024-09-21T03:43:49.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:49.324+0000 7f091a84d640 1 -- 172.21.15.38:0/2129345546 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f091414a2f0 con 0x7f09140b5260 2024-09-21T03:43:49.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:49.324+0000 7f091a84d640 1 -- 172.21.15.38:0/2129345546 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f091414a8b0 con 0x7f09140b5260 2024-09-21T03:43:49.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:49.325+0000 7f091a84d640 1 -- 172.21.15.38:0/2129345546 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0914149d80 con 0x7f09140b5260 2024-09-21T03:43:49.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:49.325+0000 7f0910ff9640 1 -- 172.21.15.38:0/2129345546 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f0908018020 con 0x7f09140b5260 2024-09-21T03:43:49.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:49.326+0000 7f0910ff9640 1 --2- 172.21.15.38:0/2129345546 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0900078b90 0x7f090007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:49.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:49.326+0000 7f0910ff9640 1 -- 172.21.15.38:0/2129345546 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 9228+0+0 (secure 0 0 0) 0x7f090809ba20 con 0x7f09140b5260 2024-09-21T03:43:49.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:49.326+0000 7f0913fff640 1 --2- 172.21.15.38:0/2129345546 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0900078b90 0x7f090007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:49.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:49.327+0000 7f0913fff640 1 --2- 172.21.15.38:0/2129345546 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0900078b90 0x7f090007b0a0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f0904033000 tx=0x7f0904004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:49.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:49.331+0000 7f0910ff9640 1 -- 172.21.15.38:0/2129345546 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f090801f020 con 0x7f09140b5260 2024-09-21T03:43:49.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:49.594+0000 7f091a84d640 1 -- 172.21.15.38:0/2129345546 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7f091406f410 con 0x7f09140b5260 2024-09-21T03:43:50.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.184+0000 7f0910ff9640 1 -- 172.21.15.38:0/2129345546 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 nothing to do: rank is not damaged v292) ==== 107+0+0 (secure 0 0 0) 0x7f0908062a00 con 0x7f09140b5260 2024-09-21T03:43:50.187 INFO:teuthology.orchestra.run.smithi038.stderr:nothing to do: rank is not damaged 2024-09-21T03:43:50.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.187+0000 7f091a84d640 1 -- 172.21.15.38:0/2129345546 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0900078b90 msgr2=0x7f090007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:50.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.187+0000 7f091a84d640 1 --2- 172.21.15.38:0/2129345546 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0900078b90 0x7f090007b0a0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f0904033000 tx=0x7f0904004730 comp rx=0 tx=0).stop 2024-09-21T03:43:50.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.187+0000 7f091a84d640 1 -- 172.21.15.38:0/2129345546 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f09140b5260 msgr2=0x7f09140b22f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:50.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.187+0000 7f091a84d640 1 --2- 172.21.15.38:0/2129345546 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f09140b5260 0x7f09140b22f0 secure :-1 s=READY pgs=2101 cs=0 l=1 rev1=1 crypto rx=0x7f090800b410 tx=0x7f090800b8d0 comp rx=0 tx=0).stop 2024-09-21T03:43:50.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.187+0000 7f091a84d640 1 -- 172.21.15.38:0/2129345546 shutdown_connections 2024-09-21T03:43:50.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.187+0000 7f091a84d640 1 --2- 172.21.15.38:0/2129345546 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0900078b90 0x7f090007b0a0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:50.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.187+0000 7f091a84d640 1 --2- 172.21.15.38:0/2129345546 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f091415bfc0 0x7f09140b2830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:50.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.187+0000 7f091a84d640 1 --2- 172.21.15.38:0/2129345546 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f09140b5260 0x7f09140b22f0 unknown :-1 s=CLOSED pgs=2101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:50.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.187+0000 7f091a84d640 1 --2- 172.21.15.38:0/2129345546 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f091406def0 0x7f09141f4190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:50.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.187+0000 7f091a84d640 1 -- 172.21.15.38:0/2129345546 >> 172.21.15.38:0/2129345546 conn(0x7f09140cdca0 msgr2=0x7f09140caa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:50.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.187+0000 7f091a84d640 1 -- 172.21.15.38:0/2129345546 shutdown_connections 2024-09-21T03:43:50.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.187+0000 7f091a84d640 1 -- 172.21.15.38:0/2129345546 wait complete. 2024-09-21T03:43:50.203 DEBUG:teuthology.orchestra.run.smithi038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-09-21T03:43:50.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.261+0000 7ff3f0661f40 1 Processor -- start 2024-09-21T03:43:50.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.261+0000 7ff3f0661f40 1 -- start start 2024-09-21T03:43:50.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.262+0000 7ff3f0661f40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff3d40030f0 0x55df6b561750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:50.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.262+0000 7ff3f0661f40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55df6b546660 0x55df6b548000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:50.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.262+0000 7ff3f0661f40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55df6b548540 0x55df6b4bd3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:50.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.262+0000 7ff3f0661f40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55df6b3ddd60 con 0x7ff3d40030f0 2024-09-21T03:43:50.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.262+0000 7ff3f0661f40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55df6b43c430 con 0x55df6b548540 2024-09-21T03:43:50.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.262+0000 7ff3f0661f40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55df6b4d7ef0 con 0x55df6b546660 2024-09-21T03:43:50.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.262+0000 7ff3ea11b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff3d40030f0 0x55df6b561750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:50.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.262+0000 7ff3ea11b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff3d40030f0 0x55df6b561750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54684/0 (socket says 172.21.15.38:54684) 2024-09-21T03:43:50.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.262+0000 7ff3ea11b640 1 -- 172.21.15.38:0/3947359320 learned_addr learned my addr 172.21.15.38:0/3947359320 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:43:50.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.262+0000 7ff3ea91c640 1 --2- 172.21.15.38:0/3947359320 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55df6b548540 0x55df6b4bd3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:50.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.262+0000 7ff3e991a640 1 --2- 172.21.15.38:0/3947359320 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55df6b546660 0x55df6b548000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:50.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.262+0000 7ff3ea11b640 1 -- 172.21.15.38:0/3947359320 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55df6b546660 msgr2=0x55df6b548000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:50.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.262+0000 7ff3ea11b640 1 --2- 172.21.15.38:0/3947359320 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55df6b546660 0x55df6b548000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:50.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.262+0000 7ff3ea11b640 1 -- 172.21.15.38:0/3947359320 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55df6b548540 msgr2=0x55df6b4bd3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:50.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.262+0000 7ff3ea11b640 1 --2- 172.21.15.38:0/3947359320 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55df6b548540 0x55df6b4bd3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:50.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.262+0000 7ff3ea11b640 1 -- 172.21.15.38:0/3947359320 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df6b546290 con 0x7ff3d40030f0 2024-09-21T03:43:50.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.262+0000 7ff3e991a640 1 --2- 172.21.15.38:0/3947359320 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55df6b546660 0x55df6b548000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:43:50.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.263+0000 7ff3ea11b640 1 --2- 172.21.15.38:0/3947359320 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff3d40030f0 0x55df6b561750 secure :-1 s=READY pgs=2104 cs=0 l=1 rev1=1 crypto rx=0x7ff3dc001400 tx=0x7ff3dc009ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0dd0e2f9e2b1a6d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:50.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.263+0000 7ff3e8918640 1 -- 172.21.15.38:0/3947359320 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3dc00d7e0 con 0x7ff3d40030f0 2024-09-21T03:43:50.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.263+0000 7ff3e8918640 1 -- 172.21.15.38:0/3947359320 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff3dc014020 con 0x7ff3d40030f0 2024-09-21T03:43:50.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.263+0000 7ff3e8918640 1 -- 172.21.15.38:0/3947359320 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3dc00dbc0 con 0x7ff3d40030f0 2024-09-21T03:43:50.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.263+0000 7ff3f0661f40 1 -- 172.21.15.38:0/3947359320 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff3d40030f0 msgr2=0x55df6b561750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:50.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.263+0000 7ff3f0661f40 1 --2- 172.21.15.38:0/3947359320 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff3d40030f0 0x55df6b561750 secure :-1 s=READY pgs=2104 cs=0 l=1 rev1=1 crypto rx=0x7ff3dc001400 tx=0x7ff3dc009ac0 comp rx=0 tx=0).stop 2024-09-21T03:43:50.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.263+0000 7ff3f0661f40 1 -- 172.21.15.38:0/3947359320 shutdown_connections 2024-09-21T03:43:50.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.263+0000 7ff3f0661f40 1 --2- 172.21.15.38:0/3947359320 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55df6b548540 0x55df6b4bd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:50.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.263+0000 7ff3f0661f40 1 --2- 172.21.15.38:0/3947359320 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55df6b546660 0x55df6b548000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:50.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.263+0000 7ff3f0661f40 1 --2- 172.21.15.38:0/3947359320 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff3d40030f0 0x55df6b561750 unknown :-1 s=CLOSED pgs=2104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:50.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.263+0000 7ff3f0661f40 1 -- 172.21.15.38:0/3947359320 >> 172.21.15.38:0/3947359320 conn(0x55df6b3aeac0 msgr2=0x55df6b5453b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:50.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.264+0000 7ff3f0661f40 1 -- 172.21.15.38:0/3947359320 shutdown_connections 2024-09-21T03:43:50.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.264+0000 7ff3f0661f40 1 -- 172.21.15.38:0/3947359320 wait complete. 2024-09-21T03:43:50.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.264+0000 7ff3f0661f40 1 Processor -- start 2024-09-21T03:43:50.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.264+0000 7ff3f0661f40 1 -- start start 2024-09-21T03:43:50.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.266+0000 7ff3f0661f40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55df6b546660 0x55df6b545430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:50.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.266+0000 7ff3f0661f40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55df6b548540 0x55df6b563140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:50.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.266+0000 7ff3f0661f40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55df6b4bc760 0x55df6b4bcb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:50.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.266+0000 7ff3f0661f40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55df6b545be0 con 0x55df6b4bc760 2024-09-21T03:43:50.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.266+0000 7ff3f0661f40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55df6b545e80 con 0x55df6b548540 2024-09-21T03:43:50.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.266+0000 7ff3f0661f40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55df6b4b7ef0 con 0x55df6b546660 2024-09-21T03:43:50.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.266+0000 7ff3ea91c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55df6b4bc760 0x55df6b4bcb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:50.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.266+0000 7ff3ea91c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55df6b4bc760 0x55df6b4bcb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54688/0 (socket says 172.21.15.38:54688) 2024-09-21T03:43:50.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.266+0000 7ff3ea91c640 1 -- 172.21.15.38:0/3834860302 learned_addr learned my addr 172.21.15.38:0/3834860302 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:43:50.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.266+0000 7ff3ea11b640 1 --2- 172.21.15.38:0/3834860302 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55df6b546660 0x55df6b545430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:50.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.266+0000 7ff3e991a640 1 --2- 172.21.15.38:0/3834860302 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55df6b548540 0x55df6b563140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:50.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.266+0000 7ff3ea91c640 1 -- 172.21.15.38:0/3834860302 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55df6b546660 msgr2=0x55df6b545430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:50.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.266+0000 7ff3ea91c640 1 --2- 172.21.15.38:0/3834860302 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55df6b546660 0x55df6b545430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:50.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.266+0000 7ff3ea91c640 1 -- 172.21.15.38:0/3834860302 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55df6b548540 msgr2=0x55df6b563140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:50.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.266+0000 7ff3ea91c640 1 --2- 172.21.15.38:0/3834860302 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55df6b548540 0x55df6b563140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:50.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.266+0000 7ff3ea91c640 1 -- 172.21.15.38:0/3834860302 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df6b4bd0b0 con 0x55df6b4bc760 2024-09-21T03:43:50.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.266+0000 7ff3ea91c640 1 --2- 172.21.15.38:0/3834860302 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55df6b4bc760 0x55df6b4bcb70 secure :-1 s=READY pgs=2105 cs=0 l=1 rev1=1 crypto rx=0x55df6b4d4f60 tx=0x7ff3e4010a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:50.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.267+0000 7ff3db7fe640 1 -- 172.21.15.38:0/3834860302 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3e401e070 con 0x55df6b4bc760 2024-09-21T03:43:50.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.267+0000 7ff3db7fe640 1 -- 172.21.15.38:0/3834860302 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff3e400e810 con 0x55df6b4bc760 2024-09-21T03:43:50.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.267+0000 7ff3f0661f40 1 -- 172.21.15.38:0/3834860302 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55df6b5fe320 con 0x55df6b4bc760 2024-09-21T03:43:50.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.267+0000 7ff3db7fe640 1 -- 172.21.15.38:0/3834860302 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3e400ec40 con 0x55df6b4bc760 2024-09-21T03:43:50.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.267+0000 7ff3f0661f40 1 -- 172.21.15.38:0/3834860302 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df6b5fe8e0 con 0x55df6b4bc760 2024-09-21T03:43:50.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.268+0000 7ff3db7fe640 1 -- 172.21.15.38:0/3834860302 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ff3e4021070 con 0x55df6b4bc760 2024-09-21T03:43:50.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.269+0000 7ff3db7fe640 1 --2- 172.21.15.38:0/3834860302 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff3cc077570 0x7ff3cc079a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:50.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.269+0000 7ff3db7fe640 1 -- 172.21.15.38:0/3834860302 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 9228+0+0 (secure 0 0 0) 0x7ff3e409b070 con 0x55df6b4bc760 2024-09-21T03:43:50.273 INFO:teuthology.orchestra.run.smithi038.stdout:Importing pool 2024-09-21T03:43:50.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.269+0000 7ff3ea11b640 1 --2- 172.21.15.38:0/3834860302 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff3cc077570 0x7ff3cc079a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:50.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.269+0000 7ff3f0661f40 1 --2- 172.21.15.38:0/3834860302 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x55df6b5fb550 0x55df6b607310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:50.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.269+0000 7ff3f0661f40 1 -- 172.21.15.38:0/3834860302 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55df6b607850 con 0x55df6b5fb550 2024-09-21T03:43:50.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.270+0000 7ff3e991a640 1 --2- 172.21.15.38:0/3834860302 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x55df6b5fb550 0x55df6b607310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:50.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.270+0000 7ff3ea11b640 1 --2- 172.21.15.38:0/3834860302 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff3cc077570 0x7ff3cc079a80 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x55df6b4d4ad0 tx=0x7ff3dc0092a0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:50.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.270+0000 7ff3e991a640 1 --2- 172.21.15.38:0/3834860302 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x55df6b5fb550 0x55df6b607310 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:50.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.275+0000 7ff3e991a640 1 -- 172.21.15.38:0/3834860302 <== osd.7 v2:172.21.15.195:6818/1606106342 1 ==== osd_op_reply(1 601.00000000 [create] v130'1033 uv1007 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ff3e00557b0 con 0x55df6b5fb550 2024-09-21T03:43:50.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.275+0000 7ff3f0661f40 1 -- 172.21.15.38:0/3834860302 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e130) -- 0x55df6b607bf0 con 0x55df6b5fb550 2024-09-21T03:43:50.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.277+0000 7ff3e991a640 1 -- 172.21.15.38:0/3834860302 <== osd.7 v2:172.21.15.195:6818/1606106342 2 ==== osd_op_reply(2 601.00000000 [delete] v130'1034 uv1034 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff3e00557b0 con 0x55df6b5fb550 2024-09-21T03:43:50.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.277+0000 7ff3f0661f40 1 -- 172.21.15.38:0/3834860302 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55df6b607f90 con 0x55df6b5fb550 2024-09-21T03:43:50.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.278+0000 7ff3e991a640 1 -- 172.21.15.38:0/3834860302 <== osd.7 v2:172.21.15.195:6818/1606106342 3 ==== osd_op_reply(3 601.00000000 [create] v130'1035 uv1035 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff3e00557b0 con 0x55df6b5fb550 2024-09-21T03:43:50.282 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-09-21T03:43:50.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.279+0000 7ff3f0661f40 1 -- 172.21.15.38:0/3834860302 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55df6b609340 con 0x55df6b5fb550 2024-09-21T03:43:50.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.280+0000 7ff3e991a640 1 -- 172.21.15.38:0/3834860302 <== osd.7 v2:172.21.15.195:6818/1606106342 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v130'1036 uv1036 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff3e00557b0 con 0x55df6b5fb550 2024-09-21T03:43:50.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.280+0000 7ff3f0661f40 1 -- 172.21.15.38:0/3834860302 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55df6b6096e0 con 0x55df6b5fb550 2024-09-21T03:43:50.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.282+0000 7ff3e991a640 1 -- 172.21.15.38:0/3834860302 <== osd.7 v2:172.21.15.195:6818/1606106342 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v130'1037 uv1037 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff3e00557b0 con 0x55df6b5fb550 2024-09-21T03:43:50.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.282+0000 7ff3f0661f40 1 -- 172.21.15.38:0/3834860302 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55df6b608330 con 0x55df6b5fb550 2024-09-21T03:43:50.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.284+0000 7ff3e991a640 1 -- 172.21.15.38:0/3834860302 <== osd.7 v2:172.21.15.195:6818/1606106342 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v130'1038 uv1038 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff3e00557b0 con 0x55df6b5fb550 2024-09-21T03:43:50.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.284+0000 7ff3f0661f40 1 -- 172.21.15.38:0/3834860302 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55df6b608720 con 0x55df6b5fb550 2024-09-21T03:43:50.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.286+0000 7ff3e991a640 1 -- 172.21.15.38:0/3834860302 <== osd.7 v2:172.21.15.195:6818/1606106342 7 ==== osd_op_reply(7 602.00000000 [create] v130'1039 uv1013 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ff3e00557b0 con 0x55df6b5fb550 2024-09-21T03:43:50.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.286+0000 7ff3f0661f40 1 -- 172.21.15.38:0/3834860302 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e130) -- 0x55df6b608b10 con 0x55df6b5fb550 2024-09-21T03:43:50.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.288+0000 7ff3e991a640 1 -- 172.21.15.38:0/3834860302 <== osd.7 v2:172.21.15.195:6818/1606106342 8 ==== osd_op_reply(8 602.00000000 [delete] v130'1040 uv1040 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff3e00557b0 con 0x55df6b5fb550 2024-09-21T03:43:50.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.288+0000 7ff3f0661f40 1 -- 172.21.15.38:0/3834860302 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55df6b608b10 con 0x55df6b5fb550 2024-09-21T03:43:50.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.289+0000 7ff3e991a640 1 -- 172.21.15.38:0/3834860302 <== osd.7 v2:172.21.15.195:6818/1606106342 9 ==== osd_op_reply(9 602.00000000 [create] v130'1041 uv1041 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff3e00557b0 con 0x55df6b5fb550 2024-09-21T03:43:50.291 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-09-21T03:43:50.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.289+0000 7ff3f0661f40 1 -- 172.21.15.38:0/3834860302 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55df6b608ac0 con 0x55df6b5fb550 2024-09-21T03:43:50.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.291+0000 7ff3e991a640 1 -- 172.21.15.38:0/3834860302 <== osd.7 v2:172.21.15.195:6818/1606106342 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v130'1042 uv1042 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff3e00557b0 con 0x55df6b5fb550 2024-09-21T03:43:50.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.291+0000 7ff3f0661f40 1 -- 172.21.15.38:0/3834860302 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55df6b608ac0 con 0x55df6b5fb550 2024-09-21T03:43:50.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.293+0000 7ff3e991a640 1 -- 172.21.15.38:0/3834860302 <== osd.7 v2:172.21.15.195:6818/1606106342 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v130'1043 uv1043 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff3e00557b0 con 0x55df6b5fb550 2024-09-21T03:43:50.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.293+0000 7ff3f0661f40 1 -- 172.21.15.38:0/3834860302 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55df6b608ac0 con 0x55df6b5fb550 2024-09-21T03:43:50.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.294+0000 7ff3e991a640 1 -- 172.21.15.38:0/3834860302 <== osd.7 v2:172.21.15.195:6818/1606106342 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v130'1044 uv1044 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff3e00557b0 con 0x55df6b5fb550 2024-09-21T03:43:50.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.294+0000 7ff3f0661f40 1 -- 172.21.15.38:0/3834860302 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:13 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55df6b608ac0 con 0x55df6b5fb550 2024-09-21T03:43:50.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.296+0000 7ff3e991a640 1 -- 172.21.15.38:0/3834860302 <== osd.7 v2:172.21.15.195:6818/1606106342 13 ==== osd_op_reply(13 10000000000.00000000 [create] v130'1045 uv1020 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7ff3e00557b0 con 0x55df6b5fb550 2024-09-21T03:43:50.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.296+0000 7ff3f0661f40 1 -- 172.21.15.38:0/3834860302 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:14 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e130) -- 0x55df6b608ac0 con 0x55df6b5fb550 2024-09-21T03:43:50.300 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.298+0000 7ff3e991a640 1 -- 172.21.15.38:0/3834860302 <== osd.7 v2:172.21.15.195:6818/1606106342 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v130'1046 uv1046 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7ff3e00557b0 con 0x55df6b5fb550 2024-09-21T03:43:50.300 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.298+0000 7ff3f0661f40 1 -- 172.21.15.38:0/3834860302 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:15 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55df6b608ac0 con 0x55df6b5fb550 2024-09-21T03:43:50.301 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-09-21T03:43:50.301 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.299+0000 7ff3e991a640 1 -- 172.21.15.38:0/3834860302 <== osd.7 v2:172.21.15.195:6818/1606106342 15 ==== osd_op_reply(15 10000000000.00000000 [create] v130'1047 uv1047 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7ff3e00557b0 con 0x55df6b5fb550 2024-09-21T03:43:50.301 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.299+0000 7ff3f0661f40 1 -- 172.21.15.38:0/3834860302 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:16 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55df6b608ac0 con 0x55df6b5fb550 2024-09-21T03:43:50.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.301+0000 7ff3e991a640 1 -- 172.21.15.38:0/3834860302 <== osd.7 v2:172.21.15.195:6818/1606106342 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v130'1048 uv1048 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7ff3e00557b0 con 0x55df6b5fb550 2024-09-21T03:43:50.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.301+0000 7ff3f0661f40 1 -- 172.21.15.38:0/3834860302 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:17 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55df6b608ac0 con 0x55df6b5fb550 2024-09-21T03:43:50.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.302+0000 7ff3e991a640 1 -- 172.21.15.38:0/3834860302 <== osd.7 v2:172.21.15.195:6818/1606106342 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v130'1049 uv1049 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7ff3e00557b0 con 0x55df6b5fb550 2024-09-21T03:43:50.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.302+0000 7ff3f0661f40 1 -- 172.21.15.38:0/3834860302 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:18 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55df6b608ac0 con 0x55df6b5fb550 2024-09-21T03:43:50.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.304+0000 7ff3e991a640 1 -- 172.21.15.38:0/3834860302 <== osd.7 v2:172.21.15.195:6818/1606106342 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v130'1050 uv1050 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7ff3e00557b0 con 0x55df6b5fb550 2024-09-21T03:43:50.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.304+0000 7ff3f0661f40 1 -- 172.21.15.38:0/3834860302 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:19 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=585b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55df6b608ac0 con 0x55df6b5fb550 2024-09-21T03:43:50.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.306+0000 7ff3e991a640 1 -- 172.21.15.38:0/3834860302 <== osd.7 v2:172.21.15.195:6818/1606106342 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v130'1051 uv1051 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7ff3e00557b0 con 0x55df6b5fb550 2024-09-21T03:43:50.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.306+0000 7ff3f0661f40 1 -- 172.21.15.38:0/3834860302 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:20 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55df6b608ac0 con 0x55df6b5fb550 2024-09-21T03:43:50.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.308+0000 7ff3e991a640 1 -- 172.21.15.38:0/3834860302 <== osd.7 v2:172.21.15.195:6818/1606106342 20 ==== osd_op_reply(20 600.00000000 [create] v130'1052 uv1026 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ff3e00557b0 con 0x55df6b5fb550 2024-09-21T03:43:50.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.308+0000 7ff3f0661f40 1 -- 172.21.15.38:0/3834860302 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:21 13.10 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e130) -- 0x55df6b608ac0 con 0x55df6b5fb550 2024-09-21T03:43:50.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.309+0000 7ff3e991a640 1 -- 172.21.15.38:0/3834860302 <== osd.7 v2:172.21.15.195:6818/1606106342 21 ==== osd_op_reply(21 600.00000000 [delete] v130'1053 uv1053 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff3e00557b0 con 0x55df6b5fb550 2024-09-21T03:43:50.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.310+0000 7ff3f0661f40 1 -- 172.21.15.38:0/3834860302 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:22 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55df6b608ac0 con 0x55df6b5fb550 2024-09-21T03:43:50.313 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-09-21T03:43:50.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.311+0000 7ff3e991a640 1 -- 172.21.15.38:0/3834860302 <== osd.7 v2:172.21.15.195:6818/1606106342 22 ==== osd_op_reply(22 600.00000000 [create] v130'1054 uv1054 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff3e00557b0 con 0x55df6b5fb550 2024-09-21T03:43:50.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.311+0000 7ff3f0661f40 1 -- 172.21.15.38:0/3834860302 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:23 13.10 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55df6b608ac0 con 0x55df6b5fb550 2024-09-21T03:43:50.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.312+0000 7ff3e991a640 1 -- 172.21.15.38:0/3834860302 <== osd.7 v2:172.21.15.195:6818/1606106342 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v130'1055 uv1055 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff3e00557b0 con 0x55df6b5fb550 2024-09-21T03:43:50.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.313+0000 7ff3f0661f40 1 -- 172.21.15.38:0/3834860302 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:24 13.10 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55df6b608ac0 con 0x55df6b5fb550 2024-09-21T03:43:50.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.314+0000 7ff3e991a640 1 -- 172.21.15.38:0/3834860302 <== osd.7 v2:172.21.15.195:6818/1606106342 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v130'1056 uv1056 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff3e00557b0 con 0x55df6b5fb550 2024-09-21T03:43:50.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.314+0000 7ff3f0661f40 1 -- 172.21.15.38:0/3834860302 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:25 13.10 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55df6b608ac0 con 0x55df6b5fb550 2024-09-21T03:43:50.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.316+0000 7ff3e991a640 1 -- 172.21.15.38:0/3834860302 <== osd.7 v2:172.21.15.195:6818/1606106342 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v130'1057 uv1057 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff3e00557b0 con 0x55df6b5fb550 2024-09-21T03:43:50.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.316+0000 7ff3f0661f40 1 -- 172.21.15.38:0/3834860302 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:26 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55df6b608ac0 con 0x55df6b5fb550 2024-09-21T03:43:50.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.317+0000 7ff3e991a640 1 -- 172.21.15.38:0/3834860302 <== osd.7 v2:172.21.15.195:6818/1606106342 26 ==== osd_op_reply(26 603.00000000 [create] v130'1058 uv1032 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ff3e00557b0 con 0x55df6b5fb550 2024-09-21T03:43:50.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.317+0000 7ff3f0661f40 1 -- 172.21.15.38:0/3834860302 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:27 13.10 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e130) -- 0x55df6b608ac0 con 0x55df6b5fb550 2024-09-21T03:43:50.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.319+0000 7ff3e991a640 1 -- 172.21.15.38:0/3834860302 <== osd.7 v2:172.21.15.195:6818/1606106342 27 ==== osd_op_reply(27 603.00000000 [delete] v130'1059 uv1059 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff3e00557b0 con 0x55df6b5fb550 2024-09-21T03:43:50.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.319+0000 7ff3f0661f40 1 -- 172.21.15.38:0/3834860302 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:28 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55df6b608ac0 con 0x55df6b5fb550 2024-09-21T03:43:50.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.320+0000 7ff3e991a640 1 -- 172.21.15.38:0/3834860302 <== osd.7 v2:172.21.15.195:6818/1606106342 28 ==== osd_op_reply(28 603.00000000 [create] v130'1060 uv1060 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff3e00557b0 con 0x55df6b5fb550 2024-09-21T03:43:50.323 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-09-21T03:43:50.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.321+0000 7ff3f0661f40 1 -- 172.21.15.38:0/3834860302 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:29 13.10 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55df6b608ac0 con 0x55df6b5fb550 2024-09-21T03:43:50.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.322+0000 7ff3e991a640 1 -- 172.21.15.38:0/3834860302 <== osd.7 v2:172.21.15.195:6818/1606106342 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v130'1061 uv1061 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff3e00557b0 con 0x55df6b5fb550 2024-09-21T03:43:50.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.322+0000 7ff3f0661f40 1 -- 172.21.15.38:0/3834860302 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:30 13.10 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55df6b608ac0 con 0x55df6b5fb550 2024-09-21T03:43:50.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.324+0000 7ff3e991a640 1 -- 172.21.15.38:0/3834860302 <== osd.7 v2:172.21.15.195:6818/1606106342 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v130'1062 uv1062 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff3e00557b0 con 0x55df6b5fb550 2024-09-21T03:43:50.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.324+0000 7ff3f0661f40 1 -- 172.21.15.38:0/3834860302 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:31 13.10 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55df6b608ac0 con 0x55df6b5fb550 2024-09-21T03:43:50.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.326+0000 7ff3e991a640 1 -- 172.21.15.38:0/3834860302 <== osd.7 v2:172.21.15.195:6818/1606106342 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v130'1063 uv1063 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff3e00557b0 con 0x55df6b5fb550 2024-09-21T03:43:50.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.326+0000 7ff3f0661f40 1 --2- 172.21.15.38:0/3834860302 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x55df6b608eb0 0x55df6b60bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:50.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.326+0000 7ff3f0661f40 1 -- 172.21.15.38:0/3834860302 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55df6b608ac0 con 0x55df6b608eb0 2024-09-21T03:43:50.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.326+0000 7ff3ea11b640 1 --2- 172.21.15.38:0/3834860302 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x55df6b608eb0 0x55df6b60bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:50.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.326+0000 7ff3ea11b640 1 --2- 172.21.15.38:0/3834860302 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x55df6b608eb0 0x55df6b60bfa0 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:50.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.328+0000 7ff3ea11b640 1 -- 172.21.15.38:0/3834860302 <== osd.6 v2:172.21.15.195:6826/4055825675 1 ==== osd_op_reply(32 1.00000000.inode [create] v130'751 uv730 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7ff3dc00db00 con 0x55df6b608eb0 2024-09-21T03:43:50.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.328+0000 7ff3f0661f40 1 -- 172.21.15.38:0/3834860302 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e130) -- 0x55df6b60c4e0 con 0x55df6b608eb0 2024-09-21T03:43:50.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.330+0000 7ff3ea11b640 1 -- 172.21.15.38:0/3834860302 <== osd.6 v2:172.21.15.195:6826/4055825675 2 ==== osd_op_reply(33 1.00000000.inode [delete] v130'752 uv752 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7ff3dc00db00 con 0x55df6b608eb0 2024-09-21T03:43:50.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.330+0000 7ff3f0661f40 1 -- 172.21.15.38:0/3834860302 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55df6b60c880 con 0x55df6b608eb0 2024-09-21T03:43:50.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.332+0000 7ff3ea11b640 1 -- 172.21.15.38:0/3834860302 <== osd.6 v2:172.21.15.195:6826/4055825675 3 ==== osd_op_reply(34 1.00000000.inode [create] v130'753 uv753 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7ff3dc00db00 con 0x55df6b608eb0 2024-09-21T03:43:50.334 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-09-21T03:43:50.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.332+0000 7ff3f0661f40 1 -- 172.21.15.38:0/3834860302 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55df6b60d0d0 con 0x55df6b608eb0 2024-09-21T03:43:50.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.333+0000 7ff3ea11b640 1 -- 172.21.15.38:0/3834860302 <== osd.6 v2:172.21.15.195:6826/4055825675 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v130'754 uv754 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7ff3dc00db00 con 0x55df6b608eb0 2024-09-21T03:43:50.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.333+0000 7ff3f0661f40 1 -- 172.21.15.38:0/3834860302 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55df6b60d470 con 0x55df6b608eb0 2024-09-21T03:43:50.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.335+0000 7ff3ea11b640 1 -- 172.21.15.38:0/3834860302 <== osd.6 v2:172.21.15.195:6826/4055825675 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v130'755 uv755 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7ff3dc00db00 con 0x55df6b608eb0 2024-09-21T03:43:50.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.335+0000 7ff3f0661f40 1 -- 172.21.15.38:0/3834860302 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55df6b60d810 con 0x55df6b608eb0 2024-09-21T03:43:50.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.337+0000 7ff3ea11b640 1 -- 172.21.15.38:0/3834860302 <== osd.6 v2:172.21.15.195:6826/4055825675 6 ==== osd_op_reply(37 200.00000000 [create] v130'756 uv749 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ff3dc00db00 con 0x55df6b608eb0 2024-09-21T03:43:50.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.337+0000 7ff3f0661f40 1 -- 172.21.15.38:0/3834860302 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e130) -- 0x55df6b60dbb0 con 0x55df6b608eb0 2024-09-21T03:43:50.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.338+0000 7ff3ea11b640 1 -- 172.21.15.38:0/3834860302 <== osd.6 v2:172.21.15.195:6826/4055825675 7 ==== osd_op_reply(38 200.00000000 [delete] v130'757 uv757 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff3dc00db00 con 0x55df6b608eb0 2024-09-21T03:43:50.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.338+0000 7ff3f0661f40 1 -- 172.21.15.38:0/3834860302 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55df6b60df50 con 0x55df6b608eb0 2024-09-21T03:43:50.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.340+0000 7ff3ea11b640 1 -- 172.21.15.38:0/3834860302 <== osd.6 v2:172.21.15.195:6826/4055825675 8 ==== osd_op_reply(39 200.00000000 [create] v130'758 uv758 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff3dc00db00 con 0x55df6b608eb0 2024-09-21T03:43:50.341 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-09-21T03:43:50.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.340+0000 7ff3f0661f40 1 -- 172.21.15.38:0/3834860302 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55df6b60df50 con 0x55df6b608eb0 2024-09-21T03:43:50.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.341+0000 7ff3ea11b640 1 -- 172.21.15.38:0/3834860302 <== osd.6 v2:172.21.15.195:6826/4055825675 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v130'759 uv759 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff3dc00db00 con 0x55df6b608eb0 2024-09-21T03:43:50.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.341+0000 7ff3f0661f40 1 -- 172.21.15.38:0/3834860302 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55df6b60df50 con 0x55df6b608eb0 2024-09-21T03:43:50.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.343+0000 7ff3ea11b640 1 -- 172.21.15.38:0/3834860302 <== osd.6 v2:172.21.15.195:6826/4055825675 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v130'760 uv760 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff3dc00db00 con 0x55df6b608eb0 2024-09-21T03:43:50.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.343+0000 7ff3f0661f40 1 -- 172.21.15.38:0/3834860302 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:42 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55df6b60df50 con 0x55df6b608eb0 2024-09-21T03:43:50.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.345+0000 7ff3ea11b640 1 -- 172.21.15.38:0/3834860302 <== osd.6 v2:172.21.15.195:6826/4055825675 11 ==== osd_op_reply(42 200.00000001 [create] v130'761 uv750 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ff3dc00db00 con 0x55df6b608eb0 2024-09-21T03:43:50.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.345+0000 7ff3f0661f40 1 -- 172.21.15.38:0/3834860302 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:43 13.14 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e130) -- 0x55df6b60df50 con 0x55df6b608eb0 2024-09-21T03:43:50.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.346+0000 7ff3ea11b640 1 -- 172.21.15.38:0/3834860302 <== osd.6 v2:172.21.15.195:6826/4055825675 12 ==== osd_op_reply(43 200.00000001 [delete] v130'762 uv762 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff3dc00db00 con 0x55df6b608eb0 2024-09-21T03:43:50.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.346+0000 7ff3f0661f40 1 -- 172.21.15.38:0/3834860302 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:44 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55df6b60df50 con 0x55df6b608eb0 2024-09-21T03:43:50.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.348+0000 7ff3ea11b640 1 -- 172.21.15.38:0/3834860302 <== osd.6 v2:172.21.15.195:6826/4055825675 13 ==== osd_op_reply(44 200.00000001 [create] v130'763 uv763 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff3dc00db00 con 0x55df6b608eb0 2024-09-21T03:43:50.350 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-09-21T03:43:50.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.348+0000 7ff3f0661f40 1 -- 172.21.15.38:0/3834860302 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:45 13.14 13:2e2fa760:::200.00000001:head [write 0~17490 in=17490b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55df6b6123d0 con 0x55df6b608eb0 2024-09-21T03:43:50.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.350+0000 7ff3ea11b640 1 -- 172.21.15.38:0/3834860302 <== osd.6 v2:172.21.15.195:6826/4055825675 14 ==== osd_op_reply(45 200.00000001 [write 0~17490] v130'764 uv764 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff3dc00db00 con 0x55df6b608eb0 2024-09-21T03:43:50.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.350+0000 7ff3f0661f40 1 -- 172.21.15.38:0/3834860302 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:46 13.14 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55df6b6123d0 con 0x55df6b608eb0 2024-09-21T03:43:50.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.351+0000 7ff3ea11b640 1 -- 172.21.15.38:0/3834860302 <== osd.6 v2:172.21.15.195:6826/4055825675 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v130'765 uv765 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff3dc00db00 con 0x55df6b608eb0 2024-09-21T03:43:50.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.352+0000 7ff3f0661f40 1 -- 172.21.15.38:0/3834860302 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:47 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55df6b60df50 con 0x55df6b608eb0 2024-09-21T03:43:50.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.353+0000 7ff3ea11b640 1 -- 172.21.15.38:0/3834860302 <== osd.6 v2:172.21.15.195:6826/4055825675 16 ==== osd_op_reply(47 606.00000000 [create] v130'766 uv746 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ff3dc00db00 con 0x55df6b608eb0 2024-09-21T03:43:50.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.353+0000 7ff3f0661f40 1 -- 172.21.15.38:0/3834860302 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:48 13.14 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e130) -- 0x55df6b60df50 con 0x55df6b608eb0 2024-09-21T03:43:50.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.355+0000 7ff3ea11b640 1 -- 172.21.15.38:0/3834860302 <== osd.6 v2:172.21.15.195:6826/4055825675 17 ==== osd_op_reply(48 606.00000000 [delete] v130'767 uv767 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff3dc00db00 con 0x55df6b608eb0 2024-09-21T03:43:50.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.355+0000 7ff3f0661f40 1 -- 172.21.15.38:0/3834860302 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:49 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55df6b60df50 con 0x55df6b608eb0 2024-09-21T03:43:50.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.356+0000 7ff3ea11b640 1 -- 172.21.15.38:0/3834860302 <== osd.6 v2:172.21.15.195:6826/4055825675 18 ==== osd_op_reply(49 606.00000000 [create] v130'768 uv768 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff3dc00db00 con 0x55df6b608eb0 2024-09-21T03:43:50.358 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-09-21T03:43:50.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.356+0000 7ff3f0661f40 1 -- 172.21.15.38:0/3834860302 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:50 13.14 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55df6b60ef60 con 0x55df6b608eb0 2024-09-21T03:43:50.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.357+0000 7ff3ea11b640 1 -- 172.21.15.38:0/3834860302 <== osd.6 v2:172.21.15.195:6826/4055825675 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v130'769 uv769 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff3dc00db00 con 0x55df6b608eb0 2024-09-21T03:43:50.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.357+0000 7ff3f0661f40 1 -- 172.21.15.38:0/3834860302 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:51 13.14 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55df6b60ef60 con 0x55df6b608eb0 2024-09-21T03:43:50.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.359+0000 7ff3ea11b640 1 -- 172.21.15.38:0/3834860302 <== osd.6 v2:172.21.15.195:6826/4055825675 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v130'770 uv770 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff3dc00db00 con 0x55df6b608eb0 2024-09-21T03:43:50.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.359+0000 7ff3f0661f40 1 -- 172.21.15.38:0/3834860302 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:52 13.14 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55df6b60df50 con 0x55df6b608eb0 2024-09-21T03:43:50.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.360+0000 7ff3ea11b640 1 -- 172.21.15.38:0/3834860302 <== osd.6 v2:172.21.15.195:6826/4055825675 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v130'771 uv771 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff3dc00db00 con 0x55df6b608eb0 2024-09-21T03:43:50.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.361+0000 7ff3f0661f40 1 --2- 172.21.15.38:0/3834860302 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x55df6b60e460 0x55df6b610880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:50.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.361+0000 7ff3f0661f40 1 -- 172.21.15.38:0/3834860302 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:53 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55df6b610dc0 con 0x55df6b60e460 2024-09-21T03:43:50.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.361+0000 7ff3e991a640 1 --2- 172.21.15.38:0/3834860302 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x55df6b60e460 0x55df6b610880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:50.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.361+0000 7ff3e991a640 1 --2- 172.21.15.38:0/3834860302 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x55df6b60e460 0x55df6b610880 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:50.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.364+0000 7ff3e991a640 1 -- 172.21.15.38:0/3834860302 <== osd.0 v2:172.21.15.38:6800/2381639433 1 ==== osd_op_reply(53 607.00000000 [create] v130'201 uv200 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ff3e00557b0 con 0x55df6b60e460 2024-09-21T03:43:50.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.364+0000 7ff3f0661f40 1 -- 172.21.15.38:0/3834860302 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:54 13.1c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e130) -- 0x55df6b610dc0 con 0x55df6b60e460 2024-09-21T03:43:50.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.366+0000 7ff3e991a640 1 -- 172.21.15.38:0/3834860302 <== osd.0 v2:172.21.15.38:6800/2381639433 2 ==== osd_op_reply(54 607.00000000 [delete] v130'202 uv202 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff3e00557b0 con 0x55df6b60e460 2024-09-21T03:43:50.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.366+0000 7ff3f0661f40 1 -- 172.21.15.38:0/3834860302 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:55 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55df6b610dc0 con 0x55df6b60e460 2024-09-21T03:43:50.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.367+0000 7ff3e991a640 1 -- 172.21.15.38:0/3834860302 <== osd.0 v2:172.21.15.38:6800/2381639433 3 ==== osd_op_reply(55 607.00000000 [create] v130'203 uv203 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff3e00557b0 con 0x55df6b60e460 2024-09-21T03:43:50.369 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-09-21T03:43:50.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.367+0000 7ff3f0661f40 1 -- 172.21.15.38:0/3834860302 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:56 13.1c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55df6b6121c0 con 0x55df6b60e460 2024-09-21T03:43:50.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.368+0000 7ff3e991a640 1 -- 172.21.15.38:0/3834860302 <== osd.0 v2:172.21.15.38:6800/2381639433 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v130'204 uv204 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff3e00557b0 con 0x55df6b60e460 2024-09-21T03:43:50.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.368+0000 7ff3f0661f40 1 -- 172.21.15.38:0/3834860302 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:57 13.1c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55df6b6121c0 con 0x55df6b60e460 2024-09-21T03:43:50.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.369+0000 7ff3e991a640 1 -- 172.21.15.38:0/3834860302 <== osd.0 v2:172.21.15.38:6800/2381639433 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v130'205 uv205 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff3e00557b0 con 0x55df6b60e460 2024-09-21T03:43:50.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.369+0000 7ff3f0661f40 1 -- 172.21.15.38:0/3834860302 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:58 13.1c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55df6b6111b0 con 0x55df6b60e460 2024-09-21T03:43:50.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.370+0000 7ff3e991a640 1 -- 172.21.15.38:0/3834860302 <== osd.0 v2:172.21.15.38:6800/2381639433 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v130'206 uv206 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff3e00557b0 con 0x55df6b60e460 2024-09-21T03:43:50.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.371+0000 7ff3f0661f40 1 --2- 172.21.15.38:0/3834860302 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x55df6b6116c0 0x55df6b613ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:50.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.371+0000 7ff3f0661f40 1 -- 172.21.15.38:0/3834860302 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:59 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55df6b6140f0 con 0x55df6b6116c0 2024-09-21T03:43:50.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.371+0000 7ff3ea91c640 1 --2- 172.21.15.38:0/3834860302 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x55df6b6116c0 0x55df6b613ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:50.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.371+0000 7ff3ea91c640 1 --2- 172.21.15.38:0/3834860302 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x55df6b6116c0 0x55df6b613ae0 crc :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:50.374 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.372+0000 7ff3ea91c640 1 -- 172.21.15.38:0/3834860302 <== osd.3 v2:172.21.15.38:6808/1286810294 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v130'147 uv146 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7ff3e4020030 con 0x55df6b6116c0 2024-09-21T03:43:50.374 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.372+0000 7ff3f0661f40 1 -- 172.21.15.38:0/3834860302 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:60 13.6 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e130) -- 0x55df6b614600 con 0x55df6b6116c0 2024-09-21T03:43:50.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.373+0000 7ff3ea91c640 1 -- 172.21.15.38:0/3834860302 <== osd.3 v2:172.21.15.38:6808/1286810294 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v130'148 uv148 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7ff3e4020030 con 0x55df6b6116c0 2024-09-21T03:43:50.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.373+0000 7ff3f0661f40 1 -- 172.21.15.38:0/3834860302 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:61 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55df6b614b00 con 0x55df6b6116c0 2024-09-21T03:43:50.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.374+0000 7ff3ea91c640 1 -- 172.21.15.38:0/3834860302 <== osd.3 v2:172.21.15.38:6808/1286810294 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v130'149 uv149 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7ff3e4020030 con 0x55df6b6116c0 2024-09-21T03:43:50.376 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-09-21T03:43:50.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.374+0000 7ff3f0661f40 1 -- 172.21.15.38:0/3834860302 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:62 13.6 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55df6b6153c0 con 0x55df6b6116c0 2024-09-21T03:43:50.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.375+0000 7ff3ea91c640 1 -- 172.21.15.38:0/3834860302 <== osd.3 v2:172.21.15.38:6808/1286810294 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v130'150 uv150 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7ff3e4020030 con 0x55df6b6116c0 2024-09-21T03:43:50.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.375+0000 7ff3f0661f40 1 -- 172.21.15.38:0/3834860302 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55df6b615760 con 0x55df6b60e460 2024-09-21T03:43:50.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.376+0000 7ff3e991a640 1 -- 172.21.15.38:0/3834860302 <== osd.0 v2:172.21.15.38:6800/2381639433 7 ==== osd_op_reply(63 608.00000000 [create] v130'201 uv200 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ff3e00557b0 con 0x55df6b60e460 2024-09-21T03:43:50.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.377+0000 7ff3f0661f40 1 -- 172.21.15.38:0/3834860302 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e130) -- 0x55df6b615760 con 0x55df6b60e460 2024-09-21T03:43:50.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.378+0000 7ff3e991a640 1 -- 172.21.15.38:0/3834860302 <== osd.0 v2:172.21.15.38:6800/2381639433 8 ==== osd_op_reply(64 608.00000000 [delete] v130'202 uv202 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff3e00557b0 con 0x55df6b60e460 2024-09-21T03:43:50.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.378+0000 7ff3f0661f40 1 -- 172.21.15.38:0/3834860302 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55df6b615760 con 0x55df6b60e460 2024-09-21T03:43:50.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.379+0000 7ff3e991a640 1 -- 172.21.15.38:0/3834860302 <== osd.0 v2:172.21.15.38:6800/2381639433 9 ==== osd_op_reply(65 608.00000000 [create] v130'203 uv203 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff3e00557b0 con 0x55df6b60e460 2024-09-21T03:43:50.381 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-09-21T03:43:50.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.379+0000 7ff3f0661f40 1 -- 172.21.15.38:0/3834860302 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55df6b616770 con 0x55df6b60e460 2024-09-21T03:43:50.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.380+0000 7ff3e991a640 1 -- 172.21.15.38:0/3834860302 <== osd.0 v2:172.21.15.38:6800/2381639433 10 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v130'204 uv204 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff3e00557b0 con 0x55df6b60e460 2024-09-21T03:43:50.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.380+0000 7ff3f0661f40 1 -- 172.21.15.38:0/3834860302 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55df6b616770 con 0x55df6b60e460 2024-09-21T03:43:50.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.381+0000 7ff3e991a640 1 -- 172.21.15.38:0/3834860302 <== osd.0 v2:172.21.15.38:6800/2381639433 11 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v130'205 uv205 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff3e00557b0 con 0x55df6b60e460 2024-09-21T03:43:50.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.381+0000 7ff3f0661f40 1 -- 172.21.15.38:0/3834860302 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55df6b615760 con 0x55df6b60e460 2024-09-21T03:43:50.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.382+0000 7ff3e991a640 1 -- 172.21.15.38:0/3834860302 <== osd.0 v2:172.21.15.38:6800/2381639433 12 ==== osd_op_reply(68 608.00000000 [omap-set-header] v130'206 uv206 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff3e00557b0 con 0x55df6b60e460 2024-09-21T03:43:50.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.382+0000 7ff3f0661f40 1 -- 172.21.15.38:0/3834860302 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55df6b615760 con 0x55df6b6116c0 2024-09-21T03:43:50.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.383+0000 7ff3ea91c640 1 -- 172.21.15.38:0/3834860302 <== osd.3 v2:172.21.15.38:6808/1286810294 5 ==== osd_op_reply(69 604.00000000 [create] v130'371 uv365 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ff3e4020030 con 0x55df6b6116c0 2024-09-21T03:43:50.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.383+0000 7ff3f0661f40 1 -- 172.21.15.38:0/3834860302 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e130) -- 0x55df6b615b00 con 0x55df6b6116c0 2024-09-21T03:43:50.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.384+0000 7ff3ea91c640 1 -- 172.21.15.38:0/3834860302 <== osd.3 v2:172.21.15.38:6808/1286810294 6 ==== osd_op_reply(70 604.00000000 [delete] v130'372 uv372 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff3e4020030 con 0x55df6b6116c0 2024-09-21T03:43:50.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.384+0000 7ff3f0661f40 1 -- 172.21.15.38:0/3834860302 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55df6b615ea0 con 0x55df6b6116c0 2024-09-21T03:43:50.387 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-09-21T03:43:50.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.385+0000 7ff3ea91c640 1 -- 172.21.15.38:0/3834860302 <== osd.3 v2:172.21.15.38:6808/1286810294 7 ==== osd_op_reply(71 604.00000000 [create] v130'373 uv373 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff3e4020030 con 0x55df6b6116c0 2024-09-21T03:43:50.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.386+0000 7ff3f0661f40 1 -- 172.21.15.38:0/3834860302 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55df6b616eb0 con 0x55df6b6116c0 2024-09-21T03:43:50.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.387+0000 7ff3ea91c640 1 -- 172.21.15.38:0/3834860302 <== osd.3 v2:172.21.15.38:6808/1286810294 8 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v130'374 uv374 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff3e4020030 con 0x55df6b6116c0 2024-09-21T03:43:50.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.387+0000 7ff3f0661f40 1 -- 172.21.15.38:0/3834860302 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55df6b616eb0 con 0x55df6b6116c0 2024-09-21T03:43:50.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.388+0000 7ff3ea91c640 1 -- 172.21.15.38:0/3834860302 <== osd.3 v2:172.21.15.38:6808/1286810294 9 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v130'375 uv375 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff3e4020030 con 0x55df6b6116c0 2024-09-21T03:43:50.390 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.388+0000 7ff3f0661f40 1 -- 172.21.15.38:0/3834860302 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55df6b615ea0 con 0x55df6b6116c0 2024-09-21T03:43:50.391 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.389+0000 7ff3ea91c640 1 -- 172.21.15.38:0/3834860302 <== osd.3 v2:172.21.15.38:6808/1286810294 10 ==== osd_op_reply(74 604.00000000 [omap-set-header] v130'376 uv376 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff3e4020030 con 0x55df6b6116c0 2024-09-21T03:43:50.391 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.389+0000 7ff3f0661f40 1 -- 172.21.15.38:0/3834860302 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55df6b615ea0 con 0x55df6b6116c0 2024-09-21T03:43:50.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.390+0000 7ff3ea91c640 1 -- 172.21.15.38:0/3834860302 <== osd.3 v2:172.21.15.38:6808/1286810294 11 ==== osd_op_reply(75 500.00000000 [create] v130'377 uv370 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ff3e4020030 con 0x55df6b6116c0 2024-09-21T03:43:50.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.390+0000 7ff3f0661f40 1 -- 172.21.15.38:0/3834860302 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e130) -- 0x55df6b615ea0 con 0x55df6b6116c0 2024-09-21T03:43:50.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.391+0000 7ff3ea91c640 1 -- 172.21.15.38:0/3834860302 <== osd.3 v2:172.21.15.38:6808/1286810294 12 ==== osd_op_reply(76 500.00000000 [delete] v130'378 uv378 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff3e4020030 con 0x55df6b6116c0 2024-09-21T03:43:50.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.391+0000 7ff3f0661f40 1 -- 172.21.15.38:0/3834860302 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55df6b615ea0 con 0x55df6b6116c0 2024-09-21T03:43:50.393 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-09-21T03:43:50.394 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.392+0000 7ff3ea91c640 1 -- 172.21.15.38:0/3834860302 <== osd.3 v2:172.21.15.38:6808/1286810294 13 ==== osd_op_reply(77 500.00000000 [create] v130'379 uv379 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff3e4020030 con 0x55df6b6116c0 2024-09-21T03:43:50.394 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.392+0000 7ff3f0661f40 1 -- 172.21.15.38:0/3834860302 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55df6b615ea0 con 0x55df6b6116c0 2024-09-21T03:43:50.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.393+0000 7ff3ea91c640 1 -- 172.21.15.38:0/3834860302 <== osd.3 v2:172.21.15.38:6808/1286810294 14 ==== osd_op_reply(78 500.00000000 [write 0~90] v130'380 uv380 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff3e4020030 con 0x55df6b6116c0 2024-09-21T03:43:50.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.393+0000 7ff3f0661f40 1 -- 172.21.15.38:0/3834860302 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55df6b615ea0 con 0x55df6b6116c0 2024-09-21T03:43:50.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.394+0000 7ff3ea91c640 1 -- 172.21.15.38:0/3834860302 <== osd.3 v2:172.21.15.38:6808/1286810294 15 ==== osd_op_reply(79 500.00000000 [omap-set-header] v130'381 uv381 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff3e4020030 con 0x55df6b6116c0 2024-09-21T03:43:50.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.394+0000 7ff3f0661f40 1 -- 172.21.15.38:0/3834860302 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:80 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55df6b615ea0 con 0x55df6b60e460 2024-09-21T03:43:50.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.395+0000 7ff3e991a640 1 -- 172.21.15.38:0/3834860302 <== osd.0 v2:172.21.15.38:6800/2381639433 13 ==== osd_op_reply(80 mds_snaptable [create] v130'168 uv167 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7ff3e00557b0 con 0x55df6b60e460 2024-09-21T03:43:50.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.395+0000 7ff3f0661f40 1 -- 172.21.15.38:0/3834860302 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:81 13.d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e130) -- 0x55df6b615ea0 con 0x55df6b60e460 2024-09-21T03:43:50.398 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.396+0000 7ff3e991a640 1 -- 172.21.15.38:0/3834860302 <== osd.0 v2:172.21.15.38:6800/2381639433 14 ==== osd_op_reply(81 mds_snaptable [delete] v130'169 uv169 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7ff3e00557b0 con 0x55df6b60e460 2024-09-21T03:43:50.398 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.396+0000 7ff3f0661f40 1 -- 172.21.15.38:0/3834860302 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:82 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55df6b615ea0 con 0x55df6b60e460 2024-09-21T03:43:50.399 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.397+0000 7ff3e991a640 1 -- 172.21.15.38:0/3834860302 <== osd.0 v2:172.21.15.38:6800/2381639433 15 ==== osd_op_reply(82 mds_snaptable [create] v130'170 uv170 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7ff3e00557b0 con 0x55df6b60e460 2024-09-21T03:43:50.399 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-09-21T03:43:50.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.397+0000 7ff3f0661f40 1 -- 172.21.15.38:0/3834860302 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:83 13.d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55df6b615ea0 con 0x55df6b60e460 2024-09-21T03:43:50.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.399+0000 7ff3e991a640 1 -- 172.21.15.38:0/3834860302 <== osd.0 v2:172.21.15.38:6800/2381639433 16 ==== osd_op_reply(83 mds_snaptable [write 0~70] v130'171 uv171 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7ff3e00557b0 con 0x55df6b60e460 2024-09-21T03:43:50.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.399+0000 7ff3f0661f40 1 -- 172.21.15.38:0/3834860302 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:84 13.d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55df6b615ea0 con 0x55df6b60e460 2024-09-21T03:43:50.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.400+0000 7ff3e991a640 1 -- 172.21.15.38:0/3834860302 <== osd.0 v2:172.21.15.38:6800/2381639433 17 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v130'172 uv172 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7ff3e00557b0 con 0x55df6b60e460 2024-09-21T03:43:50.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.400+0000 7ff3f0661f40 1 --2- 172.21.15.38:0/3834860302 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x55df6b616520 0x55df6b618960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:50.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.400+0000 7ff3f0661f40 1 -- 172.21.15.38:0/3834860302 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55df6b618fe0 con 0x55df6b616520 2024-09-21T03:43:50.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.400+0000 7ff3ea11b640 1 --2- 172.21.15.38:0/3834860302 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x55df6b616520 0x55df6b618960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:50.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.400+0000 7ff3ea11b640 1 --2- 172.21.15.38:0/3834860302 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x55df6b616520 0x55df6b618960 crc :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:50.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.402+0000 7ff3ea11b640 1 -- 172.21.15.38:0/3834860302 <== osd.1 v2:172.21.15.38:6816/1491818732 1 ==== osd_op_reply(85 605.00000000 [create] v130'201 uv200 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ff3dc00db00 con 0x55df6b616520 2024-09-21T03:43:50.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.402+0000 7ff3f0661f40 1 -- 172.21.15.38:0/3834860302 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e130) -- 0x55df6b6193b0 con 0x55df6b616520 2024-09-21T03:43:50.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.403+0000 7ff3ea11b640 1 -- 172.21.15.38:0/3834860302 <== osd.1 v2:172.21.15.38:6816/1491818732 2 ==== osd_op_reply(86 605.00000000 [delete] v130'202 uv202 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff3dc00db00 con 0x55df6b616520 2024-09-21T03:43:50.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.403+0000 7ff3f0661f40 1 -- 172.21.15.38:0/3834860302 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55df6b6193b0 con 0x55df6b616520 2024-09-21T03:43:50.406 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-09-21T03:43:50.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.404+0000 7ff3ea11b640 1 -- 172.21.15.38:0/3834860302 <== osd.1 v2:172.21.15.38:6816/1491818732 3 ==== osd_op_reply(87 605.00000000 [create] v130'203 uv203 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff3dc00db00 con 0x55df6b616520 2024-09-21T03:43:50.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.404+0000 7ff3f0661f40 1 -- 172.21.15.38:0/3834860302 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55df6b61a3c0 con 0x55df6b616520 2024-09-21T03:43:50.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.406+0000 7ff3ea11b640 1 -- 172.21.15.38:0/3834860302 <== osd.1 v2:172.21.15.38:6816/1491818732 4 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v130'204 uv204 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff3dc00db00 con 0x55df6b616520 2024-09-21T03:43:50.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.406+0000 7ff3f0661f40 1 -- 172.21.15.38:0/3834860302 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55df6b61a3c0 con 0x55df6b616520 2024-09-21T03:43:50.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.407+0000 7ff3ea11b640 1 -- 172.21.15.38:0/3834860302 <== osd.1 v2:172.21.15.38:6816/1491818732 5 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v130'205 uv205 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff3dc00db00 con 0x55df6b616520 2024-09-21T03:43:50.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.407+0000 7ff3f0661f40 1 -- 172.21.15.38:0/3834860302 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55df6b6193b0 con 0x55df6b616520 2024-09-21T03:43:50.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.408+0000 7ff3ea11b640 1 -- 172.21.15.38:0/3834860302 <== osd.1 v2:172.21.15.38:6816/1491818732 6 ==== osd_op_reply(90 605.00000000 [omap-set-header] v130'206 uv206 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff3dc00db00 con 0x55df6b616520 2024-09-21T03:43:50.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.408+0000 7ff3f0661f40 1 -- 172.21.15.38:0/3834860302 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55df6b6193b0 con 0x55df6b616520 2024-09-21T03:43:50.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.409+0000 7ff3ea11b640 1 -- 172.21.15.38:0/3834860302 <== osd.1 v2:172.21.15.38:6816/1491818732 7 ==== osd_op_reply(91 mds0_inotable [create] v130'335 uv329 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7ff3dc00db00 con 0x55df6b616520 2024-09-21T03:43:50.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.409+0000 7ff3f0661f40 1 -- 172.21.15.38:0/3834860302 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e130) -- 0x55df6b6193b0 con 0x55df6b616520 2024-09-21T03:43:50.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.411+0000 7ff3ea11b640 1 -- 172.21.15.38:0/3834860302 <== osd.1 v2:172.21.15.38:6816/1491818732 8 ==== osd_op_reply(92 mds0_inotable [delete] v130'336 uv336 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7ff3dc00db00 con 0x55df6b616520 2024-09-21T03:43:50.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.411+0000 7ff3f0661f40 1 -- 172.21.15.38:0/3834860302 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55df6b6193b0 con 0x55df6b616520 2024-09-21T03:43:50.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.412+0000 7ff3ea11b640 1 -- 172.21.15.38:0/3834860302 <== osd.1 v2:172.21.15.38:6816/1491818732 9 ==== osd_op_reply(93 mds0_inotable [create] v130'337 uv337 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7ff3dc00db00 con 0x55df6b616520 2024-09-21T03:43:50.414 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-09-21T03:43:50.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.412+0000 7ff3f0661f40 1 -- 172.21.15.38:0/3834860302 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:94 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55df6b614fd0 con 0x55df6b616520 2024-09-21T03:43:50.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.413+0000 7ff3ea11b640 1 -- 172.21.15.38:0/3834860302 <== osd.1 v2:172.21.15.38:6816/1491818732 10 ==== osd_op_reply(94 mds0_inotable [write 0~34] v130'338 uv338 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7ff3dc00db00 con 0x55df6b616520 2024-09-21T03:43:50.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.413+0000 7ff3f0661f40 1 -- 172.21.15.38:0/3834860302 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:95 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55df6b614fd0 con 0x55df6b616520 2024-09-21T03:43:50.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.414+0000 7ff3ea11b640 1 -- 172.21.15.38:0/3834860302 <== osd.1 v2:172.21.15.38:6816/1491818732 11 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v130'339 uv339 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7ff3dc00db00 con 0x55df6b616520 2024-09-21T03:43:50.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.414+0000 7ff3f0661f40 1 -- 172.21.15.38:0/3834860302 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:96 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55df6b614fb0 con 0x55df6b616520 2024-09-21T03:43:50.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.415+0000 7ff3ea11b640 1 -- 172.21.15.38:0/3834860302 <== osd.1 v2:172.21.15.38:6816/1491818732 12 ==== osd_op_reply(96 100.00000000 [create] v130'340 uv334 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ff3dc00db00 con 0x55df6b616520 2024-09-21T03:43:50.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.416+0000 7ff3f0661f40 1 -- 172.21.15.38:0/3834860302 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:97 13.13 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e130) -- 0x55df6b614fb0 con 0x55df6b616520 2024-09-21T03:43:50.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.417+0000 7ff3ea11b640 1 -- 172.21.15.38:0/3834860302 <== osd.1 v2:172.21.15.38:6816/1491818732 13 ==== osd_op_reply(97 100.00000000 [delete] v130'341 uv341 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff3dc00db00 con 0x55df6b616520 2024-09-21T03:43:50.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.417+0000 7ff3f0661f40 1 -- 172.21.15.38:0/3834860302 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:98 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55df6b614fb0 con 0x55df6b616520 2024-09-21T03:43:50.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.418+0000 7ff3ea11b640 1 -- 172.21.15.38:0/3834860302 <== osd.1 v2:172.21.15.38:6816/1491818732 14 ==== osd_op_reply(98 100.00000000 [create] v130'342 uv342 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff3dc00db00 con 0x55df6b616520 2024-09-21T03:43:50.420 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-09-21T03:43:50.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.418+0000 7ff3f0661f40 1 -- 172.21.15.38:0/3834860302 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:99 13.13 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55df6b614fb0 con 0x55df6b616520 2024-09-21T03:43:50.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.419+0000 7ff3ea11b640 1 -- 172.21.15.38:0/3834860302 <== osd.1 v2:172.21.15.38:6816/1491818732 15 ==== osd_op_reply(99 100.00000000 [omap-set-header] v130'343 uv343 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff3dc00db00 con 0x55df6b616520 2024-09-21T03:43:50.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.419+0000 7ff3f0661f40 1 -- 172.21.15.38:0/3834860302 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:100 13.13 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55df6b61c290 con 0x55df6b616520 2024-09-21T03:43:50.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.420+0000 7ff3ea11b640 1 -- 172.21.15.38:0/3834860302 <== osd.1 v2:172.21.15.38:6816/1491818732 16 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v130'344 uv344 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff3dc00db00 con 0x55df6b616520 2024-09-21T03:43:50.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.420+0000 7ff3f0661f40 1 -- 172.21.15.38:0/3834860302 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55df6b61c250 con 0x55df6b6116c0 2024-09-21T03:43:50.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.421+0000 7ff3ea91c640 1 -- 172.21.15.38:0/3834860302 <== osd.3 v2:172.21.15.38:6808/1286810294 16 ==== osd_op_reply(101 mds0_sessionmap [create] v130'147 uv145 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7ff3e4020030 con 0x55df6b6116c0 2024-09-21T03:43:50.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.421+0000 7ff3f0661f40 1 -- 172.21.15.38:0/3834860302 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e130) -- 0x55df6b61c250 con 0x55df6b6116c0 2024-09-21T03:43:50.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.423+0000 7ff3ea91c640 1 -- 172.21.15.38:0/3834860302 <== osd.3 v2:172.21.15.38:6808/1286810294 17 ==== osd_op_reply(102 mds0_sessionmap [delete] v130'148 uv148 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7ff3e4020030 con 0x55df6b6116c0 2024-09-21T03:43:50.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.423+0000 7ff3f0661f40 1 -- 172.21.15.38:0/3834860302 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:103 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55df6b61c250 con 0x55df6b6116c0 2024-09-21T03:43:50.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.424+0000 7ff3ea91c640 1 -- 172.21.15.38:0/3834860302 <== osd.3 v2:172.21.15.38:6808/1286810294 18 ==== osd_op_reply(103 mds0_sessionmap [create] v130'149 uv149 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7ff3e4020030 con 0x55df6b6116c0 2024-09-21T03:43:50.426 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-09-21T03:43:50.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.424+0000 7ff3f0661f40 1 -- 172.21.15.38:0/3834860302 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:104 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55df6b61c250 con 0x55df6b6116c0 2024-09-21T03:43:50.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.425+0000 7ff3ea91c640 1 -- 172.21.15.38:0/3834860302 <== osd.3 v2:172.21.15.38:6808/1286810294 19 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v130'150 uv150 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7ff3e4020030 con 0x55df6b6116c0 2024-09-21T03:43:50.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.425+0000 7ff3f0661f40 1 -- 172.21.15.38:0/3834860302 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55df6b61c250 con 0x55df6b6116c0 2024-09-21T03:43:50.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.426+0000 7ff3ea91c640 1 -- 172.21.15.38:0/3834860302 <== osd.3 v2:172.21.15.38:6808/1286810294 20 ==== osd_op_reply(105 609.00000000 [create] v130'201 uv200 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ff3e4020030 con 0x55df6b6116c0 2024-09-21T03:43:50.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.426+0000 7ff3f0661f40 1 -- 172.21.15.38:0/3834860302 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e130) -- 0x55df6b61c250 con 0x55df6b6116c0 2024-09-21T03:43:50.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.427+0000 7ff3ea91c640 1 -- 172.21.15.38:0/3834860302 <== osd.3 v2:172.21.15.38:6808/1286810294 21 ==== osd_op_reply(106 609.00000000 [delete] v130'202 uv202 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff3e4020030 con 0x55df6b6116c0 2024-09-21T03:43:50.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.427+0000 7ff3f0661f40 1 -- 172.21.15.38:0/3834860302 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55df6b61c250 con 0x55df6b6116c0 2024-09-21T03:43:50.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.428+0000 7ff3ea91c640 1 -- 172.21.15.38:0/3834860302 <== osd.3 v2:172.21.15.38:6808/1286810294 22 ==== osd_op_reply(107 609.00000000 [create] v130'203 uv203 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff3e4020030 con 0x55df6b6116c0 2024-09-21T03:43:50.430 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-09-21T03:43:50.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.428+0000 7ff3f0661f40 1 -- 172.21.15.38:0/3834860302 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55df6b61c250 con 0x55df6b6116c0 2024-09-21T03:43:50.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.429+0000 7ff3ea91c640 1 -- 172.21.15.38:0/3834860302 <== osd.3 v2:172.21.15.38:6808/1286810294 23 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v130'204 uv204 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff3e4020030 con 0x55df6b6116c0 2024-09-21T03:43:50.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.429+0000 7ff3f0661f40 1 -- 172.21.15.38:0/3834860302 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:109 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55df6b61c250 con 0x55df6b6116c0 2024-09-21T03:43:50.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.430+0000 7ff3ea91c640 1 -- 172.21.15.38:0/3834860302 <== osd.3 v2:172.21.15.38:6808/1286810294 24 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v130'205 uv205 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff3e4020030 con 0x55df6b6116c0 2024-09-21T03:43:50.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.430+0000 7ff3f0661f40 1 -- 172.21.15.38:0/3834860302 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:110 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55df6b61c250 con 0x55df6b6116c0 2024-09-21T03:43:50.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.431+0000 7ff3ea91c640 1 -- 172.21.15.38:0/3834860302 <== osd.3 v2:172.21.15.38:6808/1286810294 25 ==== osd_op_reply(110 609.00000000 [omap-set-header] v130'206 uv206 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff3e4020030 con 0x55df6b6116c0 2024-09-21T03:43:50.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.431+0000 7ff3f0661f40 1 --2- 172.21.15.38:0/3834860302 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x55df6b619dc0 0x55df6b61c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:50.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.431+0000 7ff3f0661f40 1 -- 172.21.15.38:0/3834860302 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55df6b61c250 con 0x55df6b619dc0 2024-09-21T03:43:50.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.431+0000 7ff3e991a640 1 --2- 172.21.15.38:0/3834860302 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x55df6b619dc0 0x55df6b61c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:50.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.432+0000 7ff3e991a640 1 --2- 172.21.15.38:0/3834860302 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x55df6b619dc0 0x55df6b61c950 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:50.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.433+0000 7ff3e991a640 1 -- 172.21.15.38:0/3834860302 <== osd.2 v2:172.21.15.38:6824/2577858902 1 ==== osd_op_reply(111 400.00000000 [create] v130'171 uv170 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ff3e00557b0 con 0x55df6b619dc0 2024-09-21T03:43:50.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.433+0000 7ff3f0661f40 1 -- 172.21.15.38:0/3834860302 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e130) -- 0x55df6b61c250 con 0x55df6b619dc0 2024-09-21T03:43:50.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.434+0000 7ff3e991a640 1 -- 172.21.15.38:0/3834860302 <== osd.2 v2:172.21.15.38:6824/2577858902 2 ==== osd_op_reply(112 400.00000000 [delete] v130'172 uv172 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff3e00557b0 con 0x55df6b619dc0 2024-09-21T03:43:50.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.434+0000 7ff3f0661f40 1 -- 172.21.15.38:0/3834860302 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55df6b61c250 con 0x55df6b619dc0 2024-09-21T03:43:50.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.435+0000 7ff3e991a640 1 -- 172.21.15.38:0/3834860302 <== osd.2 v2:172.21.15.38:6824/2577858902 3 ==== osd_op_reply(113 400.00000000 [create] v130'173 uv173 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff3e00557b0 con 0x55df6b619dc0 2024-09-21T03:43:50.437 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-09-21T03:43:50.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.435+0000 7ff3f0661f40 1 -- 172.21.15.38:0/3834860302 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:114 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55df6b61c250 con 0x55df6b619dc0 2024-09-21T03:43:50.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.436+0000 7ff3e991a640 1 -- 172.21.15.38:0/3834860302 <== osd.2 v2:172.21.15.38:6824/2577858902 4 ==== osd_op_reply(114 400.00000000 [write 0~22] v130'174 uv174 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff3e00557b0 con 0x55df6b619dc0 2024-09-21T03:43:50.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.436+0000 7ff3f0661f40 1 -- 172.21.15.38:0/3834860302 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:115 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55df6b61c250 con 0x55df6b619dc0 2024-09-21T03:43:50.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.437+0000 7ff3e991a640 1 -- 172.21.15.38:0/3834860302 <== osd.2 v2:172.21.15.38:6824/2577858902 5 ==== osd_op_reply(115 400.00000000 [omap-set-header] v130'175 uv175 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff3e00557b0 con 0x55df6b619dc0 2024-09-21T03:43:50.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.437+0000 7ff3f0661f40 1 -- 172.21.15.38:0/3834860302 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:116 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55df6b61c250 con 0x55df6b616520 2024-09-21T03:43:50.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.438+0000 7ff3ea11b640 1 -- 172.21.15.38:0/3834860302 <== osd.1 v2:172.21.15.38:6816/1491818732 17 ==== osd_op_reply(116 100.00000000.inode [create] v130'416 uv406 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7ff3dc00db00 con 0x55df6b616520 2024-09-21T03:43:50.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.438+0000 7ff3f0661f40 1 -- 172.21.15.38:0/3834860302 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:117 13.1f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e130) -- 0x55df6b61c250 con 0x55df6b616520 2024-09-21T03:43:50.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.439+0000 7ff3ea11b640 1 -- 172.21.15.38:0/3834860302 <== osd.1 v2:172.21.15.38:6816/1491818732 18 ==== osd_op_reply(117 100.00000000.inode [delete] v130'417 uv417 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7ff3dc00db00 con 0x55df6b616520 2024-09-21T03:43:50.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.439+0000 7ff3f0661f40 1 -- 172.21.15.38:0/3834860302 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:118 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55df6b61c250 con 0x55df6b616520 2024-09-21T03:43:50.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.440+0000 7ff3ea11b640 1 -- 172.21.15.38:0/3834860302 <== osd.1 v2:172.21.15.38:6816/1491818732 19 ==== osd_op_reply(118 100.00000000.inode [create] v130'418 uv418 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7ff3dc00db00 con 0x55df6b616520 2024-09-21T03:43:50.442 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-09-21T03:43:50.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.440+0000 7ff3f0661f40 1 -- 172.21.15.38:0/3834860302 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:119 13.1f 13:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55df6b61c250 con 0x55df6b616520 2024-09-21T03:43:50.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.442+0000 7ff3ea11b640 1 -- 172.21.15.38:0/3834860302 <== osd.1 v2:172.21.15.38:6816/1491818732 20 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v130'419 uv419 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7ff3dc00db00 con 0x55df6b616520 2024-09-21T03:43:50.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.442+0000 7ff3f0661f40 1 -- 172.21.15.38:0/3834860302 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:120 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55df6b61c250 con 0x55df6b616520 2024-09-21T03:43:50.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.443+0000 7ff3ea11b640 1 -- 172.21.15.38:0/3834860302 <== osd.1 v2:172.21.15.38:6816/1491818732 21 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v130'420 uv420 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7ff3dc00db00 con 0x55df6b616520 2024-09-21T03:43:50.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.443+0000 7ff3f0661f40 1 -- 172.21.15.38:0/3834860302 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:121 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55df6b61c250 con 0x55df6b616520 2024-09-21T03:43:50.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.444+0000 7ff3ea11b640 1 -- 172.21.15.38:0/3834860302 <== osd.1 v2:172.21.15.38:6816/1491818732 22 ==== osd_op_reply(121 1.00000000 [create] v130'421 uv414 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7ff3dc00db00 con 0x55df6b616520 2024-09-21T03:43:50.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.444+0000 7ff3f0661f40 1 -- 172.21.15.38:0/3834860302 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:122 13.1f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e130) -- 0x55df6b61c250 con 0x55df6b616520 2024-09-21T03:43:50.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.445+0000 7ff3ea11b640 1 -- 172.21.15.38:0/3834860302 <== osd.1 v2:172.21.15.38:6816/1491818732 23 ==== osd_op_reply(122 1.00000000 [delete] v130'422 uv422 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7ff3dc00db00 con 0x55df6b616520 2024-09-21T03:43:50.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.445+0000 7ff3f0661f40 1 -- 172.21.15.38:0/3834860302 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:123 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55df6b61c250 con 0x55df6b616520 2024-09-21T03:43:50.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.446+0000 7ff3ea11b640 1 -- 172.21.15.38:0/3834860302 <== osd.1 v2:172.21.15.38:6816/1491818732 24 ==== osd_op_reply(123 1.00000000 [create] v130'423 uv423 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7ff3dc00db00 con 0x55df6b616520 2024-09-21T03:43:50.448 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-09-21T03:43:50.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.446+0000 7ff3f0661f40 1 -- 172.21.15.38:0/3834860302 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:124 13.1f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55df6b61c250 con 0x55df6b616520 2024-09-21T03:43:50.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.447+0000 7ff3ea11b640 1 -- 172.21.15.38:0/3834860302 <== osd.1 v2:172.21.15.38:6816/1491818732 25 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v130'424 uv424 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7ff3dc00db00 con 0x55df6b616520 2024-09-21T03:43:50.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.447+0000 7ff3f0661f40 1 -- 172.21.15.38:0/3834860302 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:125 13.1f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55df6b61c250 con 0x55df6b616520 2024-09-21T03:43:50.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.448+0000 7ff3ea11b640 1 -- 172.21.15.38:0/3834860302 <== osd.1 v2:172.21.15.38:6816/1491818732 26 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v130'425 uv425 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7ff3dc00db00 con 0x55df6b616520 2024-09-21T03:43:50.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.448+0000 7ff3f0661f40 1 -- 172.21.15.38:0/3834860302 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:126 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55df6b61c250 con 0x55df6b616520 2024-09-21T03:43:50.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.449+0000 7ff3ea11b640 1 -- 172.21.15.38:0/3834860302 <== osd.1 v2:172.21.15.38:6816/1491818732 27 ==== osd_op_reply(126 1.00000000 [omap-set-header] v130'426 uv426 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7ff3dc00db00 con 0x55df6b616520 2024-09-21T03:43:50.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.449+0000 7ff3f0661f40 1 -- 172.21.15.38:0/3834860302 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:127 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=584b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55df6b61ce90 con 0x55df6b616520 2024-09-21T03:43:50.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.450+0000 7ff3ea11b640 1 -- 172.21.15.38:0/3834860302 <== osd.1 v2:172.21.15.38:6816/1491818732 28 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v130'427 uv427 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7ff3dc00db00 con 0x55df6b616520 2024-09-21T03:43:50.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.450+0000 7ff3f0661f40 1 -- 172.21.15.38:0/3834860302 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x55df6b60e460 msgr2=0x55df6b610880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:50.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.450+0000 7ff3f0661f40 1 --2- 172.21.15.38:0/3834860302 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x55df6b60e460 0x55df6b610880 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:50.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.450+0000 7ff3f0661f40 1 -- 172.21.15.38:0/3834860302 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x55df6b616520 msgr2=0x55df6b618960 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:50.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.450+0000 7ff3f0661f40 1 --2- 172.21.15.38:0/3834860302 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x55df6b616520 0x55df6b618960 crc :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:50.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.450+0000 7ff3f0661f40 1 -- 172.21.15.38:0/3834860302 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x55df6b619dc0 msgr2=0x55df6b61c950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:50.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.450+0000 7ff3f0661f40 1 --2- 172.21.15.38:0/3834860302 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x55df6b619dc0 0x55df6b61c950 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:50.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.450+0000 7ff3f0661f40 1 -- 172.21.15.38:0/3834860302 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x55df6b6116c0 msgr2=0x55df6b613ae0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:50.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.450+0000 7ff3f0661f40 1 --2- 172.21.15.38:0/3834860302 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x55df6b6116c0 0x55df6b613ae0 crc :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:50.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.451+0000 7ff3ea91c640 1 -- 172.21.15.38:0/3834860302 reap_dead start 2024-09-21T03:43:50.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.451+0000 7ff3f0661f40 1 -- 172.21.15.38:0/3834860302 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x55df6b608eb0 msgr2=0x55df6b60bfa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:50.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.451+0000 7ff3f0661f40 1 --2- 172.21.15.38:0/3834860302 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x55df6b608eb0 0x55df6b60bfa0 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:50.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.451+0000 7ff3f0661f40 1 -- 172.21.15.38:0/3834860302 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x55df6b5fb550 msgr2=0x55df6b607310 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:50.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.451+0000 7ff3f0661f40 1 --2- 172.21.15.38:0/3834860302 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x55df6b5fb550 0x55df6b607310 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:50.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.451+0000 7ff3ea91c640 1 -- 172.21.15.38:0/3834860302 reap_dead start 2024-09-21T03:43:50.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.451+0000 7ff3f0661f40 1 -- 172.21.15.38:0/3834860302 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff3cc077570 msgr2=0x7ff3cc079a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:50.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.451+0000 7ff3f0661f40 1 --2- 172.21.15.38:0/3834860302 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff3cc077570 0x7ff3cc079a80 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x55df6b4d4ad0 tx=0x7ff3dc0092a0 comp rx=0 tx=0).stop 2024-09-21T03:43:50.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.451+0000 7ff3f0661f40 1 -- 172.21.15.38:0/3834860302 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55df6b4bc760 msgr2=0x55df6b4bcb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:50.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.451+0000 7ff3f0661f40 1 --2- 172.21.15.38:0/3834860302 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55df6b4bc760 0x55df6b4bcb70 secure :-1 s=READY pgs=2105 cs=0 l=1 rev1=1 crypto rx=0x55df6b4d4f60 tx=0x7ff3e4010a40 comp rx=0 tx=0).stop 2024-09-21T03:43:50.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.451+0000 7ff3f0661f40 1 -- 172.21.15.38:0/3834860302 shutdown_connections 2024-09-21T03:43:50.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.451+0000 7ff3f0661f40 1 --2- 172.21.15.38:0/3834860302 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55df6b4bc760 0x55df6b4bcb70 unknown :-1 s=CLOSED pgs=2105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:50.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.451+0000 7ff3f0661f40 1 --2- 172.21.15.38:0/3834860302 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff3cc077570 0x7ff3cc079a80 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:50.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.451+0000 7ff3f0661f40 1 -- 172.21.15.38:0/3834860302 >> 172.21.15.38:0/3834860302 conn(0x55df6b3aeac0 msgr2=0x55df6b561750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:50.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.451+0000 7ff3f0661f40 1 -- 172.21.15.38:0/3834860302 shutdown_connections 2024-09-21T03:43:50.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.451+0000 7ff3f0661f40 1 -- 172.21.15.38:0/3834860302 wait complete. 2024-09-21T03:43:50.456 DEBUG:teuthology.orchestra.run.smithi038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata setomapheader 10000000000.00000000 deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef 2024-09-21T03:43:50.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.510+0000 7f2303d25f40 1 Processor -- start 2024-09-21T03:43:50.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.510+0000 7f2303d25f40 1 -- start start 2024-09-21T03:43:50.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.512+0000 7f2303d25f40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cc5e35cd00 0x55cc5e416b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:50.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.512+0000 7f2303d25f40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cc5e388e30 0x55cc5e4c1d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:50.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.512+0000 7f2303d25f40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55cc5e417950 0x55cc5e432350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:50.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.512+0000 7f2303d25f40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55cc5e2aad60 con 0x55cc5e35cd00 2024-09-21T03:43:50.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.512+0000 7f2303d25f40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55cc5e309430 con 0x55cc5e388e30 2024-09-21T03:43:50.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.512+0000 7f2303d25f40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55cc5e415940 con 0x55cc5e417950 2024-09-21T03:43:50.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.512+0000 7f22fcd12640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cc5e35cd00 0x55cc5e416b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:50.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.512+0000 7f22fcd12640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cc5e35cd00 0x55cc5e416b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54714/0 (socket says 172.21.15.38:54714) 2024-09-21T03:43:50.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.512+0000 7f22fcd12640 1 -- 172.21.15.38:0/2053001405 learned_addr learned my addr 172.21.15.38:0/2053001405 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:43:50.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.512+0000 7f22f7fff640 1 --2- 172.21.15.38:0/2053001405 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cc5e388e30 0x55cc5e4c1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:50.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.512+0000 7f22fd513640 1 --2- 172.21.15.38:0/2053001405 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55cc5e417950 0x55cc5e432350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:50.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.512+0000 7f22fcd12640 1 -- 172.21.15.38:0/2053001405 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55cc5e417950 msgr2=0x55cc5e432350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:50.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.512+0000 7f22fcd12640 1 --2- 172.21.15.38:0/2053001405 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55cc5e417950 0x55cc5e432350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:50.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.512+0000 7f22fcd12640 1 -- 172.21.15.38:0/2053001405 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cc5e388e30 msgr2=0x55cc5e4c1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:50.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.512+0000 7f22fcd12640 1 --2- 172.21.15.38:0/2053001405 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cc5e388e30 0x55cc5e4c1d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:50.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.512+0000 7f22fcd12640 1 -- 172.21.15.38:0/2053001405 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cc5e3a3150 con 0x55cc5e35cd00 2024-09-21T03:43:50.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.512+0000 7f22fd513640 1 --2- 172.21.15.38:0/2053001405 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55cc5e417950 0x55cc5e432350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:43:50.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.512+0000 7f22f7fff640 1 --2- 172.21.15.38:0/2053001405 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cc5e388e30 0x55cc5e4c1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:43:50.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.512+0000 7f22fcd12640 1 --2- 172.21.15.38:0/2053001405 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cc5e35cd00 0x55cc5e416b30 secure :-1 s=READY pgs=2108 cs=0 l=1 rev1=1 crypto rx=0x7f22f0001e10 tx=0x7f22f000b650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ae1173593f1c53f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:50.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.513+0000 7f22f6ffd640 1 -- 172.21.15.38:0/2053001405 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22f0016070 con 0x55cc5e35cd00 2024-09-21T03:43:50.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.513+0000 7f22f6ffd640 1 -- 172.21.15.38:0/2053001405 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f22f000d7c0 con 0x55cc5e35cd00 2024-09-21T03:43:50.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.513+0000 7f22f6ffd640 1 -- 172.21.15.38:0/2053001405 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22f000dc50 con 0x55cc5e35cd00 2024-09-21T03:43:50.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.513+0000 7f2303d25f40 1 -- 172.21.15.38:0/2053001405 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cc5e35cd00 msgr2=0x55cc5e416b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:50.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.513+0000 7f2303d25f40 1 --2- 172.21.15.38:0/2053001405 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cc5e35cd00 0x55cc5e416b30 secure :-1 s=READY pgs=2108 cs=0 l=1 rev1=1 crypto rx=0x7f22f0001e10 tx=0x7f22f000b650 comp rx=0 tx=0).stop 2024-09-21T03:43:50.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.513+0000 7f2303d25f40 1 -- 172.21.15.38:0/2053001405 shutdown_connections 2024-09-21T03:43:50.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.513+0000 7f2303d25f40 1 --2- 172.21.15.38:0/2053001405 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55cc5e417950 0x55cc5e432350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:50.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.513+0000 7f2303d25f40 1 --2- 172.21.15.38:0/2053001405 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cc5e388e30 0x55cc5e4c1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:50.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.513+0000 7f2303d25f40 1 --2- 172.21.15.38:0/2053001405 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cc5e35cd00 0x55cc5e416b30 unknown :-1 s=CLOSED pgs=2108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:50.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.513+0000 7f2303d25f40 1 -- 172.21.15.38:0/2053001405 >> 172.21.15.38:0/2053001405 conn(0x55cc5e27bac0 msgr2=0x55cc5e42e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:50.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.513+0000 7f2303d25f40 1 -- 172.21.15.38:0/2053001405 shutdown_connections 2024-09-21T03:43:50.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.513+0000 7f2303d25f40 1 -- 172.21.15.38:0/2053001405 wait complete. 2024-09-21T03:43:50.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.514+0000 7f2303d25f40 1 Processor -- start 2024-09-21T03:43:50.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.514+0000 7f2303d25f40 1 -- start start 2024-09-21T03:43:50.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.515+0000 7f2303d25f40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cc5e35cd00 0x55cc5e42c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:50.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.515+0000 7f2303d25f40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cc5e388e30 0x55cc5e42a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:50.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.515+0000 7f2303d25f40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55cc5e417950 0x55cc5e410060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:50.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.515+0000 7f2303d25f40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55cc5e412700 con 0x55cc5e35cd00 2024-09-21T03:43:50.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.515+0000 7f2303d25f40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55cc5e4c2270 con 0x55cc5e388e30 2024-09-21T03:43:50.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.515+0000 7f2303d25f40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55cc5e3871c0 con 0x55cc5e417950 2024-09-21T03:43:50.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.516+0000 7f22fcd12640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cc5e35cd00 0x55cc5e42c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:50.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.516+0000 7f22fcd12640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cc5e35cd00 0x55cc5e42c730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54728/0 (socket says 172.21.15.38:54728) 2024-09-21T03:43:50.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.516+0000 7f22fcd12640 1 -- 172.21.15.38:0/3754979420 learned_addr learned my addr 172.21.15.38:0/3754979420 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:43:50.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.516+0000 7f22fd513640 1 --2- 172.21.15.38:0/3754979420 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55cc5e417950 0x55cc5e410060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:50.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.516+0000 7f22f7fff640 1 --2- 172.21.15.38:0/3754979420 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cc5e388e30 0x55cc5e42a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:50.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.516+0000 7f22fcd12640 1 -- 172.21.15.38:0/3754979420 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55cc5e417950 msgr2=0x55cc5e410060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:50.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.516+0000 7f22fcd12640 1 --2- 172.21.15.38:0/3754979420 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55cc5e417950 0x55cc5e410060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:50.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.516+0000 7f22fcd12640 1 -- 172.21.15.38:0/3754979420 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cc5e388e30 msgr2=0x55cc5e42a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:50.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.516+0000 7f22fcd12640 1 --2- 172.21.15.38:0/3754979420 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cc5e388e30 0x55cc5e42a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:50.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.516+0000 7f22fcd12640 1 -- 172.21.15.38:0/3754979420 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cc5e3a0c10 con 0x55cc5e35cd00 2024-09-21T03:43:50.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.516+0000 7f22fd513640 1 --2- 172.21.15.38:0/3754979420 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55cc5e417950 0x55cc5e410060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:43:50.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.516+0000 7f22fcd12640 1 --2- 172.21.15.38:0/3754979420 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cc5e35cd00 0x55cc5e42c730 secure :-1 s=READY pgs=2109 cs=0 l=1 rev1=1 crypto rx=0x55cc5e415100 tx=0x7f22f0001330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:50.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.516+0000 7f22f5ffb640 1 -- 172.21.15.38:0/3754979420 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22f0022070 con 0x55cc5e35cd00 2024-09-21T03:43:50.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.516+0000 7f22f5ffb640 1 -- 172.21.15.38:0/3754979420 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f22f0017040 con 0x55cc5e35cd00 2024-09-21T03:43:50.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.516+0000 7f2303d25f40 1 -- 172.21.15.38:0/3754979420 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55cc5e42aad0 con 0x55cc5e35cd00 2024-09-21T03:43:50.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.516+0000 7f22f5ffb640 1 -- 172.21.15.38:0/3754979420 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22f0016070 con 0x55cc5e35cd00 2024-09-21T03:43:50.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.516+0000 7f2303d25f40 1 -- 172.21.15.38:0/3754979420 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cc5e3a0e80 con 0x55cc5e35cd00 2024-09-21T03:43:50.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.518+0000 7f22f5ffb640 1 -- 172.21.15.38:0/3754979420 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f22f0025070 con 0x55cc5e35cd00 2024-09-21T03:43:50.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.519+0000 7f22f5ffb640 1 --2- 172.21.15.38:0/3754979420 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f22e0077570 0x7f22e0079a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:50.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.519+0000 7f22f5ffb640 1 -- 172.21.15.38:0/3754979420 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 9228+0+0 (secure 0 0 0) 0x7f22f009b3a0 con 0x55cc5e35cd00 2024-09-21T03:43:50.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.519+0000 7f22f7fff640 1 --2- 172.21.15.38:0/3754979420 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f22e0077570 0x7f22e0079a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:50.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.519+0000 7f2303d25f40 1 --2- 172.21.15.38:0/3754979420 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x55cc5e4ba170 0x55cc5e4ba580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:50.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.519+0000 7f2303d25f40 1 -- 172.21.15.38:0/3754979420 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:1 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=80b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55cc5e4baac0 con 0x55cc5e4ba170 2024-09-21T03:43:50.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.519+0000 7f22fd513640 1 --2- 172.21.15.38:0/3754979420 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x55cc5e4ba170 0x55cc5e4ba580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:50.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.519+0000 7f22f7fff640 1 --2- 172.21.15.38:0/3754979420 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f22e0077570 0x7f22e0079a80 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f22e8055000 tx=0x7f22e8058040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:50.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.520+0000 7f22fd513640 1 --2- 172.21.15.38:0/3754979420 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x55cc5e4ba170 0x55cc5e4ba580 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:50.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.521+0000 7f22fd513640 1 -- 172.21.15.38:0/3754979420 <== osd.7 v2:172.21.15.195:6818/1606106342 1 ==== osd_op_reply(1 10000000000.00000000 [omap-set-header] v130'1064 uv1064 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f22f800b040 con 0x55cc5e4ba170 2024-09-21T03:43:50.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.521+0000 7f2303d25f40 1 -- 172.21.15.38:0/3754979420 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x55cc5e4ba170 msgr2=0x55cc5e4ba580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:50.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.521+0000 7f2303d25f40 1 --2- 172.21.15.38:0/3754979420 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x55cc5e4ba170 0x55cc5e4ba580 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:50.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.521+0000 7f2303d25f40 1 -- 172.21.15.38:0/3754979420 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f22e0077570 msgr2=0x7f22e0079a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:50.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.521+0000 7f2303d25f40 1 --2- 172.21.15.38:0/3754979420 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f22e0077570 0x7f22e0079a80 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f22e8055000 tx=0x7f22e8058040 comp rx=0 tx=0).stop 2024-09-21T03:43:50.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.521+0000 7f2303d25f40 1 -- 172.21.15.38:0/3754979420 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cc5e35cd00 msgr2=0x55cc5e42c730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:50.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.521+0000 7f2303d25f40 1 --2- 172.21.15.38:0/3754979420 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cc5e35cd00 0x55cc5e42c730 secure :-1 s=READY pgs=2109 cs=0 l=1 rev1=1 crypto rx=0x55cc5e415100 tx=0x7f22f0001330 comp rx=0 tx=0).stop 2024-09-21T03:43:50.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.521+0000 7f22fd513640 1 -- 172.21.15.38:0/3754979420 reap_dead start 2024-09-21T03:43:50.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.521+0000 7f2303d25f40 1 -- 172.21.15.38:0/3754979420 shutdown_connections 2024-09-21T03:43:50.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.521+0000 7f2303d25f40 1 -- 172.21.15.38:0/3754979420 >> 172.21.15.38:0/3754979420 conn(0x55cc5e27bac0 msgr2=0x55cc5e432350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:50.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.521+0000 7f2303d25f40 1 -- 172.21.15.38:0/3754979420 shutdown_connections 2024-09-21T03:43:50.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.521+0000 7f2303d25f40 1 -- 172.21.15.38:0/3754979420 wait complete. 2024-09-21T03:43:50.527 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-21T03:43:50.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.646+0000 7fabc44bd640 1 -- 172.21.15.38:0/397760378 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faba8003a30 msgr2=0x7faba8003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:50.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.646+0000 7fabc44bd640 1 --2- 172.21.15.38:0/397760378 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faba8003a30 0x7faba8003e60 secure :-1 s=READY pgs=2111 cs=0 l=1 rev1=1 crypto rx=0x7fabb8001740 tx=0x7faba8005830 comp rx=0 tx=0).stop 2024-09-21T03:43:50.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.646+0000 7fabc2a5a640 1 -- 172.21.15.38:0/397760378 reap_dead start 2024-09-21T03:43:50.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.646+0000 7fabc44bd640 1 -- 172.21.15.38:0/397760378 shutdown_connections 2024-09-21T03:43:50.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.646+0000 7fabc44bd640 1 -- 172.21.15.38:0/397760378 >> 172.21.15.38:0/397760378 conn(0x7fabbc09d830 msgr2=0x7fabbc0b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:50.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.646+0000 7fabc44bd640 1 -- 172.21.15.38:0/397760378 shutdown_connections 2024-09-21T03:43:50.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.646+0000 7fabc44bd640 1 -- 172.21.15.38:0/397760378 wait complete. 2024-09-21T03:43:50.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.648+0000 7fabc44bd640 1 Processor -- start 2024-09-21T03:43:50.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.648+0000 7fabc44bd640 1 -- start start 2024-09-21T03:43:50.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.649+0000 7fabc44bd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faba8003a30 0x7fabbc1f5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:50.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.649+0000 7fabc44bd640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fabbc0cdf40 0x7fabbc156f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:50.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.649+0000 7fabc44bd640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fabbc157660 0x7fabbc1f9140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:50.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.649+0000 7fabc44bd640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fabbc0d0df0 con 0x7faba8003a30 2024-09-21T03:43:50.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.649+0000 7fabc44bd640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fabbc0cf540 con 0x7fabbc157660 2024-09-21T03:43:50.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.649+0000 7fabc44bd640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fabbc0d0f70 con 0x7fabbc0cdf40 2024-09-21T03:43:50.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.649+0000 7fabc2259640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faba8003a30 0x7fabbc1f5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:50.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.649+0000 7fabc2259640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faba8003a30 0x7fabbc1f5460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54758/0 (socket says 172.21.15.38:54758) 2024-09-21T03:43:50.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.649+0000 7fabc2259640 1 -- 172.21.15.38:0/1663570342 learned_addr learned my addr 172.21.15.38:0/1663570342 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:43:50.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.649+0000 7fabc2a5a640 1 --2- 172.21.15.38:0/1663570342 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fabbc157660 0x7fabbc1f9140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:50.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.649+0000 7fabc1a58640 1 --2- 172.21.15.38:0/1663570342 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fabbc0cdf40 0x7fabbc156f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:50.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.649+0000 7fabc2259640 1 -- 172.21.15.38:0/1663570342 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fabbc0cdf40 msgr2=0x7fabbc156f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:50.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.649+0000 7fabc2259640 1 --2- 172.21.15.38:0/1663570342 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fabbc0cdf40 0x7fabbc156f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:50.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.649+0000 7fabc2259640 1 -- 172.21.15.38:0/1663570342 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fabbc157660 msgr2=0x7fabbc1f9140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:50.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.649+0000 7fabc2259640 1 --2- 172.21.15.38:0/1663570342 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fabbc157660 0x7fabbc1f9140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:50.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.649+0000 7fabc2259640 1 -- 172.21.15.38:0/1663570342 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabbc0cc130 con 0x7faba8003a30 2024-09-21T03:43:50.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.650+0000 7fabc2259640 1 --2- 172.21.15.38:0/1663570342 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faba8003a30 0x7fabbc1f5460 secure :-1 s=READY pgs=2112 cs=0 l=1 rev1=1 crypto rx=0x7fabb802a9f0 tx=0x7fabb8006150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:50.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.650+0000 7fabb2ffd640 1 -- 172.21.15.38:0/1663570342 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabb8037040 con 0x7faba8003a30 2024-09-21T03:43:50.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.650+0000 7fabb2ffd640 1 -- 172.21.15.38:0/1663570342 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fabb80035f0 con 0x7faba8003a30 2024-09-21T03:43:50.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.650+0000 7fabc44bd640 1 -- 172.21.15.38:0/1663570342 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabbc0cc410 con 0x7faba8003a30 2024-09-21T03:43:50.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.650+0000 7fabb2ffd640 1 -- 172.21.15.38:0/1663570342 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabb8003a20 con 0x7faba8003a30 2024-09-21T03:43:50.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.650+0000 7fabc44bd640 1 -- 172.21.15.38:0/1663570342 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabbc1f5b40 con 0x7faba8003a30 2024-09-21T03:43:50.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.651+0000 7fabc44bd640 1 -- 172.21.15.38:0/1663570342 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fabbc1f5da0 con 0x7faba8003a30 2024-09-21T03:43:50.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.652+0000 7fabb2ffd640 1 -- 172.21.15.38:0/1663570342 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fabb8043020 con 0x7faba8003a30 2024-09-21T03:43:50.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.652+0000 7fabb2ffd640 1 --2- 172.21.15.38:0/1663570342 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7faba0078b90 0x7faba007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:50.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.652+0000 7fabb2ffd640 1 -- 172.21.15.38:0/1663570342 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 9228+0+0 (secure 0 0 0) 0x7fabb80bb5e0 con 0x7faba8003a30 2024-09-21T03:43:50.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.652+0000 7fabc1a58640 1 --2- 172.21.15.38:0/1663570342 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7faba0078b90 0x7faba007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:50.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.653+0000 7fabc1a58640 1 --2- 172.21.15.38:0/1663570342 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7faba0078b90 0x7faba007b0a0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fabbc0b3480 tx=0x7fabac005f30 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:50.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.657+0000 7fabb2ffd640 1 -- 172.21.15.38:0/1663570342 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fabb804a020 con 0x7faba8003a30 2024-09-21T03:43:50.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:50.889+0000 7fabc44bd640 1 -- 172.21.15.38:0/1663570342 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7fabbc1f8a80 con 0x7faba8003a30 2024-09-21T03:43:51.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:51.764+0000 7fabb2ffd640 1 -- 172.21.15.38:0/1663570342 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v293) ==== 162+0+0 (secure 0 0 0) 0x7fabb8082640 con 0x7faba8003a30 2024-09-21T03:43:51.766 INFO:teuthology.orchestra.run.smithi038.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-21T03:43:51.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:51.768+0000 7fabc44bd640 1 -- 172.21.15.38:0/1663570342 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7faba0078b90 msgr2=0x7faba007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:51.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:51.768+0000 7fabc44bd640 1 --2- 172.21.15.38:0/1663570342 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7faba0078b90 0x7faba007b0a0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fabbc0b3480 tx=0x7fabac005f30 comp rx=0 tx=0).stop 2024-09-21T03:43:51.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:51.768+0000 7fabc44bd640 1 -- 172.21.15.38:0/1663570342 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faba8003a30 msgr2=0x7fabbc1f5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:51.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:51.768+0000 7fabc44bd640 1 --2- 172.21.15.38:0/1663570342 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faba8003a30 0x7fabbc1f5460 secure :-1 s=READY pgs=2112 cs=0 l=1 rev1=1 crypto rx=0x7fabb802a9f0 tx=0x7fabb8006150 comp rx=0 tx=0).stop 2024-09-21T03:43:51.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:51.768+0000 7fabc44bd640 1 -- 172.21.15.38:0/1663570342 shutdown_connections 2024-09-21T03:43:51.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:51.768+0000 7fabc44bd640 1 --2- 172.21.15.38:0/1663570342 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fabbc157660 0x7fabbc1f9140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:51.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:51.768+0000 7fabc44bd640 1 --2- 172.21.15.38:0/1663570342 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7faba0078b90 0x7faba007b0a0 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:51.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:51.768+0000 7fabc44bd640 1 --2- 172.21.15.38:0/1663570342 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fabbc0cdf40 0x7fabbc156f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:51.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:51.768+0000 7fabc44bd640 1 --2- 172.21.15.38:0/1663570342 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faba8003a30 0x7fabbc1f5460 unknown :-1 s=CLOSED pgs=2112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:51.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:51.768+0000 7fabc44bd640 1 -- 172.21.15.38:0/1663570342 >> 172.21.15.38:0/1663570342 conn(0x7fabbc09d830 msgr2=0x7fabbc15e2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:51.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:51.768+0000 7fabc44bd640 1 -- 172.21.15.38:0/1663570342 shutdown_connections 2024-09-21T03:43:51.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:51.768+0000 7fabc44bd640 1 -- 172.21.15.38:0/1663570342 wait complete. 2024-09-21T03:43:52.189 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:43:52.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:52.320+0000 7ff3120ce640 1 -- 172.21.15.38:0/141913627 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff2f4004800 msgr2=0x7ff2f4004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:52.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:52.320+0000 7ff3120ce640 1 --2- 172.21.15.38:0/141913627 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff2f4004800 0x7ff2f4004c10 secure :-1 s=READY pgs=2114 cs=0 l=1 rev1=1 crypto rx=0x7ff300006f80 tx=0x7ff2f4005830 comp rx=0 tx=0).stop 2024-09-21T03:43:52.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:52.320+0000 7ff30bfff640 1 -- 172.21.15.38:0/141913627 reap_dead start 2024-09-21T03:43:52.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:52.320+0000 7ff3120ce640 1 -- 172.21.15.38:0/141913627 shutdown_connections 2024-09-21T03:43:52.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:52.320+0000 7ff3120ce640 1 -- 172.21.15.38:0/141913627 >> 172.21.15.38:0/141913627 conn(0x7ff30c0b6b20 msgr2=0x7ff30c0d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:52.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:52.320+0000 7ff3120ce640 1 -- 172.21.15.38:0/141913627 shutdown_connections 2024-09-21T03:43:52.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:52.320+0000 7ff3120ce640 1 -- 172.21.15.38:0/141913627 wait complete. 2024-09-21T03:43:52.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:52.322+0000 7ff3120ce640 1 Processor -- start 2024-09-21T03:43:52.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:52.322+0000 7ff3120ce640 1 -- start start 2024-09-21T03:43:52.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:52.323+0000 7ff3120ce640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff2f4004800 0x7ff30c0ce7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:52.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:52.323+0000 7ff3120ce640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff30c15ab50 0x7ff30c15af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:52.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:52.323+0000 7ff3120ce640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff30c0c9f50 0x7ff30c0ca360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:52.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:52.323+0000 7ff3120ce640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ff30c0b31d0 con 0x7ff2f4004800 2024-09-21T03:43:52.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:52.323+0000 7ff3120ce640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ff30c0b3050 con 0x7ff30c0c9f50 2024-09-21T03:43:52.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:52.323+0000 7ff3120ce640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7ff30c0b3350 con 0x7ff30c15ab50 2024-09-21T03:43:52.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:52.323+0000 7ff30b7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff2f4004800 0x7ff30c0ce7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:52.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:52.323+0000 7ff30b7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff2f4004800 0x7ff30c0ce7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50662/0 (socket says 172.21.15.38:50662) 2024-09-21T03:43:52.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:52.323+0000 7ff30b7fe640 1 -- 172.21.15.38:0/209584626 learned_addr learned my addr 172.21.15.38:0/209584626 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:43:52.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:52.324+0000 7ff30bfff640 1 --2- 172.21.15.38:0/209584626 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff30c0c9f50 0x7ff30c0ca360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:52.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:52.324+0000 7ff30affd640 1 --2- 172.21.15.38:0/209584626 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff30c15ab50 0x7ff30c15af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:52.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:52.324+0000 7ff30b7fe640 1 -- 172.21.15.38:0/209584626 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff30c15ab50 msgr2=0x7ff30c15af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:52.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:52.324+0000 7ff30b7fe640 1 --2- 172.21.15.38:0/209584626 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff30c15ab50 0x7ff30c15af60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:52.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:52.324+0000 7ff30b7fe640 1 -- 172.21.15.38:0/209584626 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff30c0c9f50 msgr2=0x7ff30c0ca360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:52.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:52.324+0000 7ff30b7fe640 1 --2- 172.21.15.38:0/209584626 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff30c0c9f50 0x7ff30c0ca360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:52.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:52.324+0000 7ff30b7fe640 1 -- 172.21.15.38:0/209584626 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff30c0cc3b0 con 0x7ff2f4004800 2024-09-21T03:43:52.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:52.324+0000 7ff30b7fe640 1 --2- 172.21.15.38:0/209584626 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff2f4004800 0x7ff30c0ce7c0 secure :-1 s=READY pgs=2115 cs=0 l=1 rev1=1 crypto rx=0x7ff2fc004d90 tx=0x7ff2fc001730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:52.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:52.324+0000 7ff2f3fff640 1 -- 172.21.15.38:0/209584626 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2fc0107d0 con 0x7ff2f4004800 2024-09-21T03:43:52.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:52.324+0000 7ff2f3fff640 1 -- 172.21.15.38:0/209584626 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff2fc010a60 con 0x7ff2f4004800 2024-09-21T03:43:52.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:52.324+0000 7ff3120ce640 1 -- 172.21.15.38:0/209584626 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff30c0cae10 con 0x7ff2f4004800 2024-09-21T03:43:52.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:52.324+0000 7ff2f3fff640 1 -- 172.21.15.38:0/209584626 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2fc011460 con 0x7ff2f4004800 2024-09-21T03:43:52.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:52.324+0000 7ff3120ce640 1 -- 172.21.15.38:0/209584626 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff30c0cb310 con 0x7ff2f4004800 2024-09-21T03:43:52.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:52.325+0000 7ff3120ce640 1 -- 172.21.15.38:0/209584626 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff30c0b38e0 con 0x7ff2f4004800 2024-09-21T03:43:52.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:52.326+0000 7ff2f3fff640 1 -- 172.21.15.38:0/209584626 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ff2fc011600 con 0x7ff2f4004800 2024-09-21T03:43:52.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:52.327+0000 7ff2f3fff640 1 --2- 172.21.15.38:0/209584626 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff2f8078bc0 0x7ff2f807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:52.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:52.327+0000 7ff2f3fff640 1 -- 172.21.15.38:0/209584626 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 9228+0+0 (secure 0 0 0) 0x7ff2fc09c690 con 0x7ff2f4004800 2024-09-21T03:43:52.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:52.327+0000 7ff30affd640 1 --2- 172.21.15.38:0/209584626 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff2f8078bc0 0x7ff2f807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:52.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:52.328+0000 7ff30affd640 1 --2- 172.21.15.38:0/209584626 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff2f8078bc0 0x7ff2f807b0d0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7ff300033000 tx=0x7ff300001930 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:52.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:52.331+0000 7ff2f3fff640 1 -- 172.21.15.38:0/209584626 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7ff2fc063d00 con 0x7ff2f4004800 2024-09-21T03:43:52.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:52.558+0000 7ff3120ce640 1 -- 172.21.15.38:0/209584626 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff30c076230 con 0x7ff2f4004800 2024-09-21T03:43:52.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:52.558+0000 7ff2f3fff640 1 -- 172.21.15.38:0/209584626 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 294 v294) ==== 95+0+5230 (secure 0 0 0) 0x7ff2fc063600 con 0x7ff2f4004800 2024-09-21T03:43:52.561 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:43:52.561 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":294,"btime":"2024-09-21T03:43:51:769267+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6086,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/965972663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":965972663},{"type":"v1","addr":"172.21.15.195:6836","nonce":965972663}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6700,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/3176122334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":3176122334},{"type":"v1","addr":"172.21.15.195:6837","nonce":3176122334}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":9840,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/3801772046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":3801772046},{"type":"v1","addr":"172.21.15.38:6835","nonce":3801772046}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":273}],"filesystems":[{"mdsmap":{"epoch":294,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:40:02.319282+0000","modified":"2024-09-21T03:43:51.769247+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":130,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10113},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10113":{"gid":10113,"name":"c","rank":0,"incarnation":294,"state":"up:replay","state_seq":1,"addr":"172.21.15.38:6833/2444649344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":2444649344},{"type":"v1","addr":"172.21.15.38:6833","nonce":2444649344}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-21T03:43:52.561 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 294 2024-09-21T03:43:52.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:52.561+0000 7ff3120ce640 1 -- 172.21.15.38:0/209584626 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff2f8078bc0 msgr2=0x7ff2f807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:52.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:52.561+0000 7ff3120ce640 1 --2- 172.21.15.38:0/209584626 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff2f8078bc0 0x7ff2f807b0d0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7ff300033000 tx=0x7ff300001930 comp rx=0 tx=0).stop 2024-09-21T03:43:52.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:52.561+0000 7ff3120ce640 1 -- 172.21.15.38:0/209584626 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff2f4004800 msgr2=0x7ff30c0ce7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:52.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:52.561+0000 7ff3120ce640 1 --2- 172.21.15.38:0/209584626 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff2f4004800 0x7ff30c0ce7c0 secure :-1 s=READY pgs=2115 cs=0 l=1 rev1=1 crypto rx=0x7ff2fc004d90 tx=0x7ff2fc001730 comp rx=0 tx=0).stop 2024-09-21T03:43:52.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:52.561+0000 7ff3120ce640 1 -- 172.21.15.38:0/209584626 shutdown_connections 2024-09-21T03:43:52.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:52.561+0000 7ff3120ce640 1 --2- 172.21.15.38:0/209584626 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff30c0c9f50 0x7ff30c0ca360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:52.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:52.561+0000 7ff3120ce640 1 --2- 172.21.15.38:0/209584626 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff2f8078bc0 0x7ff2f807b0d0 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:52.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:52.561+0000 7ff3120ce640 1 --2- 172.21.15.38:0/209584626 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff30c15ab50 0x7ff30c15af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:52.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:52.561+0000 7ff3120ce640 1 --2- 172.21.15.38:0/209584626 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff2f4004800 0x7ff30c0ce7c0 unknown :-1 s=CLOSED pgs=2115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:52.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:52.561+0000 7ff3120ce640 1 -- 172.21.15.38:0/209584626 >> 172.21.15.38:0/209584626 conn(0x7ff30c0b6b20 msgr2=0x7ff30c0b8590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:52.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:52.561+0000 7ff3120ce640 1 -- 172.21.15.38:0/209584626 shutdown_connections 2024-09-21T03:43:52.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:52.561+0000 7ff3120ce640 1 -- 172.21.15.38:0/209584626 wait complete. 2024-09-21T03:43:52.574 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 294, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T03:40:02.319282+0000', 'modified': '2024-09-21T03:43:51.769247+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 130, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 10113}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_10113': {'gid': 10113, 'name': 'c', 'rank': 0, 'incarnation': 294, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.38:6833/2444649344', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.38:6832', 'nonce': 2444649344}, {'type': 'v1', 'addr': '172.21.15.38:6833', 'nonce': 2444649344}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-09-21T03:43:52.574 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_10113:up:replay 2024-09-21T03:43:52.574 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2024-09-21T03:43:55.194 INFO:tasks.ceph.mds.c.smithi038.stderr:2024-09-21T03:43:55.192+0000 7f1b94a94640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T03:43:57.979 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:43:58.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.109+0000 7f58dca70640 1 -- 172.21.15.38:0/2070423190 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f58d806e500 msgr2=0x7f58d80cc020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:58.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.109+0000 7f58dca70640 1 --2- 172.21.15.38:0/2070423190 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f58d806e500 0x7f58d80cc020 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f58d0005fd0 tx=0x7f58d002a8b0 comp rx=0 tx=0).stop 2024-09-21T03:43:58.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.109+0000 7f58dca70640 1 -- 172.21.15.38:0/2070423190 shutdown_connections 2024-09-21T03:43:58.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.109+0000 7f58dca70640 1 --2- 172.21.15.38:0/2070423190 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f58d806e500 0x7f58d80cc020 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:58.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.109+0000 7f58dca70640 1 --2- 172.21.15.38:0/2070423190 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f58d8074cb0 0x7f58d8073dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:58.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.109+0000 7f58dca70640 1 -- 172.21.15.38:0/2070423190 >> 172.21.15.38:0/2070423190 conn(0x7f58d80cca40 msgr2=0x7f58d80cce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:58.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.109+0000 7f58dca70640 1 -- 172.21.15.38:0/2070423190 shutdown_connections 2024-09-21T03:43:58.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.109+0000 7f58dca70640 1 -- 172.21.15.38:0/2070423190 wait complete. 2024-09-21T03:43:58.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.111+0000 7f58dca70640 1 Processor -- start 2024-09-21T03:43:58.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.111+0000 7f58dca70640 1 -- start start 2024-09-21T03:43:58.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.113+0000 7f58dca70640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f58d8074cb0 0x7f58d80b4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:58.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.113+0000 7f58dca70640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f58d80cbc10 0x7f58d8155900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:58.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.113+0000 7f58dca70640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f58d80b0c00 0x7f58d8155e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:58.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.113+0000 7f58dca70640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f58d806f360 con 0x7f58d80b0c00 2024-09-21T03:43:58.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.113+0000 7f58dca70640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f58d806f1e0 con 0x7f58d80cbc10 2024-09-21T03:43:58.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.113+0000 7f58dca70640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f58d806f4e0 con 0x7f58d8074cb0 2024-09-21T03:43:58.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.113+0000 7f58d6d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f58d80b0c00 0x7f58d8155e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:58.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.113+0000 7f58d6d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f58d80b0c00 0x7f58d8155e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50670/0 (socket says 172.21.15.38:50670) 2024-09-21T03:43:58.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.113+0000 7f58d6d9d640 1 -- 172.21.15.38:0/4043106912 learned_addr learned my addr 172.21.15.38:0/4043106912 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:43:58.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.113+0000 7f58d659c640 1 --2- 172.21.15.38:0/4043106912 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f58d8074cb0 0x7f58d80b4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:58.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.113+0000 7f58d5d9b640 1 --2- 172.21.15.38:0/4043106912 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f58d80cbc10 0x7f58d8155900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:58.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.113+0000 7f58d6d9d640 1 -- 172.21.15.38:0/4043106912 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f58d8074cb0 msgr2=0x7f58d80b4aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:58.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.113+0000 7f58d6d9d640 1 --2- 172.21.15.38:0/4043106912 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f58d8074cb0 0x7f58d80b4aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:58.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.113+0000 7f58d6d9d640 1 -- 172.21.15.38:0/4043106912 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f58d80cbc10 msgr2=0x7f58d8155900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:58.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.113+0000 7f58d6d9d640 1 --2- 172.21.15.38:0/4043106912 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f58d80cbc10 0x7f58d8155900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:58.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.113+0000 7f58d6d9d640 1 -- 172.21.15.38:0/4043106912 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58d80b3fe0 con 0x7f58d80b0c00 2024-09-21T03:43:58.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.114+0000 7f58d6d9d640 1 --2- 172.21.15.38:0/4043106912 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f58d80b0c00 0x7f58d8155e40 secure :-1 s=READY pgs=2116 cs=0 l=1 rev1=1 crypto rx=0x7f58d002aea0 tx=0x7f58d0033f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:58.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.114+0000 7f58ceffd640 1 -- 172.21.15.38:0/4043106912 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58d0031990 con 0x7f58d80b0c00 2024-09-21T03:43:58.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.114+0000 7f58ceffd640 1 -- 172.21.15.38:0/4043106912 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f58d0031c20 con 0x7f58d80b0c00 2024-09-21T03:43:58.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.114+0000 7f58dca70640 1 -- 172.21.15.38:0/4043106912 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58d80b1230 con 0x7f58d80b0c00 2024-09-21T03:43:58.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.114+0000 7f58ceffd640 1 -- 172.21.15.38:0/4043106912 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58d00330d0 con 0x7f58d80b0c00 2024-09-21T03:43:58.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.114+0000 7f58dca70640 1 -- 172.21.15.38:0/4043106912 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58d80b4580 con 0x7f58d80b0c00 2024-09-21T03:43:58.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.115+0000 7f58dca70640 1 -- 172.21.15.38:0/4043106912 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58d806fa00 con 0x7f58d80b0c00 2024-09-21T03:43:58.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.116+0000 7f58ceffd640 1 -- 172.21.15.38:0/4043106912 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f58d0033270 con 0x7f58d80b0c00 2024-09-21T03:43:58.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.116+0000 7f58ceffd640 1 --2- 172.21.15.38:0/4043106912 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f58b8078b90 0x7f58b807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:58.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.116+0000 7f58ceffd640 1 -- 172.21.15.38:0/4043106912 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 9228+0+0 (secure 0 0 0) 0x7f58d00bce00 con 0x7f58d80b0c00 2024-09-21T03:43:58.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.117+0000 7f58d659c640 1 --2- 172.21.15.38:0/4043106912 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f58b8078b90 0x7f58b807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:58.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.117+0000 7f58d659c640 1 --2- 172.21.15.38:0/4043106912 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f58b8078b90 0x7f58b807b0a0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f58c80095f0 tx=0x7f58c8005f90 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:58.123 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.121+0000 7f58ceffd640 1 -- 172.21.15.38:0/4043106912 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f58d003f030 con 0x7f58d80b0c00 2024-09-21T03:43:58.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.347+0000 7f58dca70640 1 -- 172.21.15.38:0/4043106912 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f58d809b0c0 con 0x7f58d80b0c00 2024-09-21T03:43:58.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.348+0000 7f58ceffd640 1 -- 172.21.15.38:0/4043106912 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 297 v297) ==== 95+0+5239 (secure 0 0 0) 0x7f58d00317b0 con 0x7f58d80b0c00 2024-09-21T03:43:58.350 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:43:58.350 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":297,"btime":"2024-09-21T03:43:54:787124+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6086,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/965972663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":965972663},{"type":"v1","addr":"172.21.15.195:6836","nonce":965972663}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6700,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/3176122334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":3176122334},{"type":"v1","addr":"172.21.15.195:6837","nonce":3176122334}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":9840,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/3801772046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":3801772046},{"type":"v1","addr":"172.21.15.38:6835","nonce":3801772046}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":273}],"filesystems":[{"mdsmap":{"epoch":297,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:40:02.319282+0000","modified":"2024-09-21T03:43:54.787114+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":130,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10113},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10113":{"gid":10113,"name":"c","rank":0,"incarnation":294,"state":"up:active","state_seq":4,"addr":"172.21.15.38:6833/2444649344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":2444649344},{"type":"v1","addr":"172.21.15.38:6833","nonce":2444649344}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10113,"qdb_cluster":[10113]},"id":6}]} 2024-09-21T03:43:58.350 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 297 2024-09-21T03:43:58.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.350+0000 7f58dca70640 1 -- 172.21.15.38:0/4043106912 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f58b8078b90 msgr2=0x7f58b807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:58.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.350+0000 7f58dca70640 1 --2- 172.21.15.38:0/4043106912 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f58b8078b90 0x7f58b807b0a0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f58c80095f0 tx=0x7f58c8005f90 comp rx=0 tx=0).stop 2024-09-21T03:43:58.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.350+0000 7f58dca70640 1 -- 172.21.15.38:0/4043106912 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f58d80b0c00 msgr2=0x7f58d8155e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:58.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.350+0000 7f58dca70640 1 --2- 172.21.15.38:0/4043106912 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f58d80b0c00 0x7f58d8155e40 secure :-1 s=READY pgs=2116 cs=0 l=1 rev1=1 crypto rx=0x7f58d002aea0 tx=0x7f58d0033f70 comp rx=0 tx=0).stop 2024-09-21T03:43:58.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.350+0000 7f58dca70640 1 -- 172.21.15.38:0/4043106912 shutdown_connections 2024-09-21T03:43:58.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.350+0000 7f58dca70640 1 --2- 172.21.15.38:0/4043106912 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f58d80b0c00 0x7f58d8155e40 unknown :-1 s=CLOSED pgs=2116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:58.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.350+0000 7f58dca70640 1 --2- 172.21.15.38:0/4043106912 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f58d80cbc10 0x7f58d8155900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:58.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.350+0000 7f58dca70640 1 --2- 172.21.15.38:0/4043106912 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f58b8078b90 0x7f58b807b0a0 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:58.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.350+0000 7f58dca70640 1 --2- 172.21.15.38:0/4043106912 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f58d8074cb0 0x7f58d80b4aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:58.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.350+0000 7f58dca70640 1 -- 172.21.15.38:0/4043106912 >> 172.21.15.38:0/4043106912 conn(0x7f58d80cca40 msgr2=0x7f58d81e4d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:58.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.350+0000 7f58dca70640 1 -- 172.21.15.38:0/4043106912 shutdown_connections 2024-09-21T03:43:58.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.350+0000 7f58dca70640 1 -- 172.21.15.38:0/4043106912 wait complete. 2024-09-21T03:43:58.363 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 297, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T03:40:02.319282+0000', 'modified': '2024-09-21T03:43:54.787114+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 130, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 10113}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_10113': {'gid': 10113, 'name': 'c', 'rank': 0, 'incarnation': 294, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.38:6833/2444649344', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.38:6832', 'nonce': 2444649344}, {'type': 'v1', 'addr': '172.21.15.38:6833', 'nonce': 2444649344}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10113, 'qdb_cluster': [10113]} 2024-09-21T03:43:58.364 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-21T03:43:58.364 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-21T03:43:58.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.479+0000 7ff7de27e640 1 -- 172.21.15.38:0/3548988084 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff7d80d1120 msgr2=0x7ff7d80ceb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:58.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.479+0000 7ff7de27e640 1 --2- 172.21.15.38:0/3548988084 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff7d80d1120 0x7ff7d80ceb90 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7ff7d0005fd0 tx=0x7ff7d002aa10 comp rx=0 tx=0).stop 2024-09-21T03:43:58.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.479+0000 7ff7de27e640 1 -- 172.21.15.38:0/3548988084 shutdown_connections 2024-09-21T03:43:58.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.479+0000 7ff7de27e640 1 --2- 172.21.15.38:0/3548988084 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff7d80d1120 0x7ff7d80ceb90 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:58.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.479+0000 7ff7de27e640 1 --2- 172.21.15.38:0/3548988084 >> v2:172.21.15.195:3300/0 conn(0x7ff7d80d0170 0x7ff7d80d0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:58.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.479+0000 7ff7de27e640 1 -- 172.21.15.38:0/3548988084 >> 172.21.15.38:0/3548988084 conn(0x7ff7d809d830 msgr2=0x7ff7d80b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:58.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.479+0000 7ff7de27e640 1 -- 172.21.15.38:0/3548988084 shutdown_connections 2024-09-21T03:43:58.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.479+0000 7ff7de27e640 1 -- 172.21.15.38:0/3548988084 wait complete. 2024-09-21T03:43:58.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.481+0000 7ff7de27e640 1 Processor -- start 2024-09-21T03:43:58.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.481+0000 7ff7de27e640 1 -- start start 2024-09-21T03:43:58.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.482+0000 7ff7de27e640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff7d80d0170 0x7ff7d80cd240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:58.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.482+0000 7ff7de27e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff7d80d1120 0x7ff7d806eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:58.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.483+0000 7ff7de27e640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff7d815ee80 0x7ff7d81f8b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:58.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.483+0000 7ff7de27e640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ff7d80b6700 con 0x7ff7d80d1120 2024-09-21T03:43:58.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.483+0000 7ff7de27e640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ff7d80b6580 con 0x7ff7d815ee80 2024-09-21T03:43:58.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.483+0000 7ff7de27e640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7ff7d80b6880 con 0x7ff7d80d0170 2024-09-21T03:43:58.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.483+0000 7ff7d77fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff7d80d1120 0x7ff7d806eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:58.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.483+0000 7ff7d77fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff7d80d1120 0x7ff7d806eb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50676/0 (socket says 172.21.15.38:50676) 2024-09-21T03:43:58.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.483+0000 7ff7d77fe640 1 -- 172.21.15.38:0/881193469 learned_addr learned my addr 172.21.15.38:0/881193469 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:43:58.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.483+0000 7ff7d7fff640 1 --2- 172.21.15.38:0/881193469 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff7d80d0170 0x7ff7d80cd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:58.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.483+0000 7ff7dc81b640 1 --2- 172.21.15.38:0/881193469 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff7d815ee80 0x7ff7d81f8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:58.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.483+0000 7ff7d77fe640 1 -- 172.21.15.38:0/881193469 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff7d80d0170 msgr2=0x7ff7d80cd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:58.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.483+0000 7ff7d77fe640 1 --2- 172.21.15.38:0/881193469 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff7d80d0170 0x7ff7d80cd240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:58.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.483+0000 7ff7d77fe640 1 -- 172.21.15.38:0/881193469 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff7d815ee80 msgr2=0x7ff7d81f8b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:58.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.483+0000 7ff7d77fe640 1 --2- 172.21.15.38:0/881193469 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff7d815ee80 0x7ff7d81f8b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:58.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.483+0000 7ff7d77fe640 1 -- 172.21.15.38:0/881193469 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7d81f99b0 con 0x7ff7d80d1120 2024-09-21T03:43:58.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.483+0000 7ff7d77fe640 1 --2- 172.21.15.38:0/881193469 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff7d80d1120 0x7ff7d806eb30 secure :-1 s=READY pgs=2117 cs=0 l=1 rev1=1 crypto rx=0x7ff7cc00ea20 tx=0x7ff7cc00eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:58.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.483+0000 7ff7d4ff9640 1 -- 172.21.15.38:0/881193469 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7cc00cd20 con 0x7ff7d80d1120 2024-09-21T03:43:58.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.483+0000 7ff7d4ff9640 1 -- 172.21.15.38:0/881193469 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff7cc005030 con 0x7ff7d80d1120 2024-09-21T03:43:58.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.483+0000 7ff7de27e640 1 -- 172.21.15.38:0/881193469 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7d80cb820 con 0x7ff7d80d1120 2024-09-21T03:43:58.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.483+0000 7ff7d4ff9640 1 -- 172.21.15.38:0/881193469 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7cc005460 con 0x7ff7d80d1120 2024-09-21T03:43:58.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.484+0000 7ff7de27e640 1 -- 172.21.15.38:0/881193469 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7d80cbde0 con 0x7ff7d80d1120 2024-09-21T03:43:58.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.485+0000 7ff7d4ff9640 1 -- 172.21.15.38:0/881193469 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ff7cc0047c0 con 0x7ff7d80d1120 2024-09-21T03:43:58.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.486+0000 7ff7d4ff9640 1 --2- 172.21.15.38:0/881193469 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff7c4078b90 0x7ff7c407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:58.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.486+0000 7ff7d4ff9640 1 -- 172.21.15.38:0/881193469 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 9228+0+0 (secure 0 0 0) 0x7ff7cc09c390 con 0x7ff7d80d1120 2024-09-21T03:43:58.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.486+0000 7ff7d7fff640 1 --2- 172.21.15.38:0/881193469 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff7c4078b90 0x7ff7c407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:58.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.486+0000 7ff7e51e8740 1 Processor -- start 2024-09-21T03:43:58.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.487+0000 7ff7e51e8740 1 -- start start 2024-09-21T03:43:58.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.487+0000 7ff7d7fff640 1 --2- 172.21.15.38:0/881193469 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff7c4078b90 0x7ff7c407b0a0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7ff7c8001720 tx=0x7ff7c8009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:58.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.488+0000 7ff7e51e8740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bd82f77270 0x55bd83348fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:58.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.488+0000 7ff7e51e8740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bd82f4e3b0 0x55bd83349500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:58.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.488+0000 7ff7e51e8740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55bd82f56590 0x55bd83366400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:58.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.488+0000 7ff7e51e8740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55bd83291820 con 0x55bd82f77270 2024-09-21T03:43:58.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.488+0000 7ff7e51e8740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55bd83280c50 con 0x55bd82f4e3b0 2024-09-21T03:43:58.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.488+0000 7ff7e51e8740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55bd83284a60 con 0x55bd82f56590 2024-09-21T03:43:58.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.488+0000 7ff7d7fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bd82f77270 0x55bd83348fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:58.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.488+0000 7ff7d7fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bd82f77270 0x55bd83348fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50686/0 (socket says 172.21.15.38:50686) 2024-09-21T03:43:58.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.488+0000 7ff7d7fff640 1 -- 172.21.15.38:0/1259843111 learned_addr learned my addr 172.21.15.38:0/1259843111 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:43:58.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.488+0000 7ff7dc81b640 1 --2- 172.21.15.38:0/1259843111 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55bd82f56590 0x55bd83366400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:58.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.488+0000 7ff7d77fe640 1 --2- 172.21.15.38:0/1259843111 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bd82f4e3b0 0x55bd83349500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:58.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.488+0000 7ff7d7fff640 1 -- 172.21.15.38:0/1259843111 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55bd82f56590 msgr2=0x55bd83366400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:58.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.488+0000 7ff7d7fff640 1 --2- 172.21.15.38:0/1259843111 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55bd82f56590 0x55bd83366400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:58.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.488+0000 7ff7d7fff640 1 -- 172.21.15.38:0/1259843111 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bd82f4e3b0 msgr2=0x55bd83349500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:58.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.488+0000 7ff7d7fff640 1 --2- 172.21.15.38:0/1259843111 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bd82f4e3b0 0x55bd83349500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:58.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.488+0000 7ff7d7fff640 1 -- 172.21.15.38:0/1259843111 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd82fe2850 con 0x55bd82f77270 2024-09-21T03:43:58.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.488+0000 7ff7dc81b640 1 --2- 172.21.15.38:0/1259843111 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55bd82f56590 0x55bd83366400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:43:58.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.488+0000 7ff7d7fff640 1 --2- 172.21.15.38:0/1259843111 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bd82f77270 0x55bd83348fc0 secure :-1 s=READY pgs=2118 cs=0 l=1 rev1=1 crypto rx=0x7ff7c800f940 tx=0x7ff7c8012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f02eb6d55526dcc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:58.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.489+0000 7ff7bdffb640 1 -- 172.21.15.38:0/1259843111 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7c8025070 con 0x55bd82f77270 2024-09-21T03:43:58.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.489+0000 7ff7bdffb640 1 -- 172.21.15.38:0/1259843111 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff7c801b050 con 0x55bd82f77270 2024-09-21T03:43:58.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.489+0000 7ff7bdffb640 1 -- 172.21.15.38:0/1259843111 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7c8014e40 con 0x55bd82f77270 2024-09-21T03:43:58.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.489+0000 7ff7e51e8740 1 -- 172.21.15.38:0/1259843111 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bd82f77270 msgr2=0x55bd83348fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:58.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.489+0000 7ff7e51e8740 1 --2- 172.21.15.38:0/1259843111 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bd82f77270 0x55bd83348fc0 secure :-1 s=READY pgs=2118 cs=0 l=1 rev1=1 crypto rx=0x7ff7c800f940 tx=0x7ff7c8012000 comp rx=0 tx=0).stop 2024-09-21T03:43:58.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.489+0000 7ff7e51e8740 1 -- 172.21.15.38:0/1259843111 shutdown_connections 2024-09-21T03:43:58.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.489+0000 7ff7e51e8740 1 --2- 172.21.15.38:0/1259843111 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55bd82f56590 0x55bd83366400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:58.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.489+0000 7ff7e51e8740 1 --2- 172.21.15.38:0/1259843111 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bd82f4e3b0 0x55bd83349500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:58.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.489+0000 7ff7e51e8740 1 --2- 172.21.15.38:0/1259843111 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bd82f77270 0x55bd83348fc0 unknown :-1 s=CLOSED pgs=2118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:58.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.489+0000 7ff7e51e8740 1 -- 172.21.15.38:0/1259843111 >> 172.21.15.38:0/1259843111 conn(0x55bd82f6b3c0 msgr2=0x55bd8331f280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:58.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.489+0000 7ff7e51e8740 1 -- 172.21.15.38:0/1259843111 shutdown_connections 2024-09-21T03:43:58.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.490+0000 7ff7e51e8740 1 -- 172.21.15.38:0/1259843111 wait complete. 2024-09-21T03:43:58.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.490+0000 7ff7e51e8740 1 -- start start 2024-09-21T03:43:58.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.491+0000 7ff7e51e8740 1 Processor -- start 2024-09-21T03:43:58.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.492+0000 7ff7e51e8740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bd82f4e3b0 0x55bd833412f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:58.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.492+0000 7ff7e51e8740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bd82f56590 0x55bd83341fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:58.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.492+0000 7ff7e51e8740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55bd82f77270 0x55bd833478e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:58.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.492+0000 7ff7e51e8740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55bd82ff1620 con 0x55bd82f4e3b0 2024-09-21T03:43:58.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.492+0000 7ff7e51e8740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55bd832265f0 con 0x55bd82f56590 2024-09-21T03:43:58.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.492+0000 7ff7e51e8740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55bd83349a40 con 0x55bd82f77270 2024-09-21T03:43:58.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.492+0000 7ff7e51e8740 10 client.0 _ll_register_callbacks cb 0x55bd8322a0a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:43:58.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.492+0000 7ff7d7fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bd82f4e3b0 0x55bd833412f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:58.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.492+0000 7ff7d7fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bd82f4e3b0 0x55bd833412f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50698/0 (socket says 172.21.15.38:50698) 2024-09-21T03:43:58.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.492+0000 7ff7d7fff640 1 -- 172.21.15.38:0/2772696385 learned_addr learned my addr 172.21.15.38:0/2772696385 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:43:58.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.492+0000 7ff7d77fe640 1 --2- 172.21.15.38:0/2772696385 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bd82f56590 0x55bd83341fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:58.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.492+0000 7ff7dc81b640 1 --2- 172.21.15.38:0/2772696385 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55bd82f77270 0x55bd833478e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:58.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.492+0000 7ff7d7fff640 1 -- 172.21.15.38:0/2772696385 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55bd82f77270 msgr2=0x55bd833478e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:58.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.492+0000 7ff7d7fff640 1 --2- 172.21.15.38:0/2772696385 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55bd82f77270 0x55bd833478e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:58.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.492+0000 7ff7d7fff640 1 -- 172.21.15.38:0/2772696385 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bd82f56590 msgr2=0x55bd83341fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:58.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.492+0000 7ff7d7fff640 1 --2- 172.21.15.38:0/2772696385 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bd82f56590 0x55bd83341fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:58.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.492+0000 7ff7d7fff640 1 -- 172.21.15.38:0/2772696385 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd830596b0 con 0x55bd82f4e3b0 2024-09-21T03:43:58.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.493+0000 7ff7d7fff640 1 --2- 172.21.15.38:0/2772696385 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bd82f4e3b0 0x55bd833412f0 secure :-1 s=READY pgs=2119 cs=0 l=1 rev1=1 crypto rx=0x55bd83320e50 tx=0x7ff7c8021650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7eaec0c80b2c1da6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:58.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.493+0000 7ff7b6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-21T03:43:58.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.493+0000 7ff7b6ffd640 1 -- 172.21.15.38:0/2772696385 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7c8025040 con 0x55bd82f4e3b0 2024-09-21T03:43:58.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.493+0000 7ff7b6ffd640 1 -- 172.21.15.38:0/2772696385 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(130..130 src has 1..130) ==== 9228+0+0 (secure 0 0 0) 0x7ff7c8021a40 con 0x55bd82f4e3b0 2024-09-21T03:43:58.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.493+0000 7ff7e51e8740 1 -- 172.21.15.38:0/2772696385 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff7c801b050 con 0x55bd82f4e3b0 2024-09-21T03:43:58.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.493+0000 7ff7b6ffd640 1 -- 172.21.15.38:0/2772696385 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=297) ==== 24+0+0 (secure 0 0 0) 0x7ff7c8039050 con 0x55bd82f4e3b0 2024-09-21T03:43:58.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.493+0000 7ff7e51e8740 10 client.10164 fetch_fsmap learned FSMap version 297 2024-09-21T03:43:58.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.493+0000 7ff7e51e8740 1 -- 172.21.15.38:0/2772696385 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=297,monmap=2+}) -- 0x55bd83336260 con 0x55bd82f4e3b0 2024-09-21T03:43:58.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.493+0000 7ff7b6ffd640 1 -- 172.21.15.38:0/2772696385 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff7c801b050 con 0x55bd82f4e3b0 2024-09-21T03:43:58.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.493+0000 7ff7b6ffd640 1 -- 172.21.15.38:0/2772696385 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 297) ==== 3502+0+0 (secure 0 0 0) 0x7ff7c8028030 con 0x55bd82f4e3b0 2024-09-21T03:43:58.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.493+0000 7ff7e51e8740 10 client.10164 fetch_fsmap finished waiting for FSMap version 297 2024-09-21T03:43:58.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.493+0000 7ff7e51e8740 10 client.10164 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T03:43:58.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.493+0000 7ff7e51e8740 20 client.10164 populate_metadata read hostname 'smithi038' 2024-09-21T03:43:58.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.493+0000 7ff7e51e8740 1 --2- 172.21.15.38:0/2772696385 >> [v2:172.21.15.38:6832/2444649344,v1:172.21.15.38:6833/2444649344] conn(0x55bd82f35e40 0x55bd83376530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:58.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.493+0000 7ff7e51e8740 4 client.10164 mds_command: new command op to 10113 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:43:58.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.493+0000 7ff7e51e8740 1 -- 172.21.15.38:0/2772696385 --> [v2:172.21.15.38:6832/2444649344,v1:172.21.15.38:6833/2444649344] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bd83293770 con 0x55bd82f35e40 2024-09-21T03:43:58.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.494+0000 7ff7d77fe640 1 --2- 172.21.15.38:0/2772696385 >> [v2:172.21.15.38:6832/2444649344,v1:172.21.15.38:6833/2444649344] conn(0x55bd82f35e40 0x55bd83376530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:58.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.494+0000 7ff7b6ffd640 10 client.10164 ms_handle_connect on v2:172.21.15.38:6832/2444649344 2024-09-21T03:43:58.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.494+0000 7ff7d77fe640 1 --2- 172.21.15.38:0/2772696385 >> [v2:172.21.15.38:6832/2444649344,v1:172.21.15.38:6833/2444649344] conn(0x55bd82f35e40 0x55bd83376530 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=abfd8458720448b1 server_cookie=a21e97ac84778f4 in_seq=0 out_seq=0 2024-09-21T03:43:58.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.495+0000 7ff7b6ffd640 1 -- 172.21.15.38:0/2772696385 <== mds.0 v2:172.21.15.38:6832/2444649344 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55bd83293770 con 0x55bd82f35e40 2024-09-21T03:43:58.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.495+0000 7ff7b6ffd640 10 client.10164 handle_command_reply: tid=0 2024-09-21T03:43:58.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.495+0000 7ff7e51e8740 1 client.10164 shutdown 2024-09-21T03:43:58.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.495+0000 7ff7e51e8740 1 -- 172.21.15.38:0/2772696385 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bd82f4e3b0 msgr2=0x55bd833412f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:58.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.495+0000 7ff7e51e8740 1 --2- 172.21.15.38:0/2772696385 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bd82f4e3b0 0x55bd833412f0 secure :-1 s=READY pgs=2119 cs=0 l=1 rev1=1 crypto rx=0x55bd83320e50 tx=0x7ff7c8021650 comp rx=0 tx=0).stop 2024-09-21T03:43:58.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.495+0000 7ff7e51e8740 1 -- 172.21.15.38:0/2772696385 shutdown_connections 2024-09-21T03:43:58.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.495+0000 7ff7e51e8740 1 --2- 172.21.15.38:0/2772696385 >> [v2:172.21.15.38:6832/2444649344,v1:172.21.15.38:6833/2444649344] conn(0x55bd82f35e40 0x55bd83376530 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:58.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.495+0000 7ff7e51e8740 1 --2- 172.21.15.38:0/2772696385 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55bd82f77270 0x55bd833478e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:58.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.495+0000 7ff7e51e8740 1 --2- 172.21.15.38:0/2772696385 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bd82f56590 0x55bd83341fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:58.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.495+0000 7ff7b6ffd640 0 client.10164 ms_handle_reset on v2:172.21.15.38:6832/2444649344 2024-09-21T03:43:58.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.495+0000 7ff7e51e8740 1 --2- 172.21.15.38:0/2772696385 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bd82f4e3b0 0x55bd833412f0 unknown :-1 s=CLOSED pgs=2119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:58.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.495+0000 7ff7e51e8740 1 -- 172.21.15.38:0/2772696385 >> 172.21.15.38:0/2772696385 conn(0x55bd82f6b3c0 msgr2=0x55bd83319b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:58.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.495+0000 7ff7e51e8740 1 -- 172.21.15.38:0/2772696385 shutdown_connections 2024-09-21T03:43:58.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.495+0000 7ff7e51e8740 1 -- 172.21.15.38:0/2772696385 wait complete. 2024-09-21T03:43:58.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.495+0000 7ff7e51e8740 20 client.10164 trim_cache size 0 max 16384 2024-09-21T03:43:58.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.514+0000 7ff7e51e8740 1 Processor -- start 2024-09-21T03:43:58.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.514+0000 7ff7e51e8740 1 -- start start 2024-09-21T03:43:58.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.515+0000 7ff7e51e8740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bd82f4e3b0 0x55bd83318550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:58.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.515+0000 7ff7e51e8740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bd82f56590 0x55bd83318a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:58.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.515+0000 7ff7e51e8740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55bd82f77270 0x55bd8335c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:58.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.515+0000 7ff7e51e8740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ff7d8203770 con 0x55bd82f4e3b0 2024-09-21T03:43:58.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.515+0000 7ff7e51e8740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ff7d8202a50 con 0x55bd82f56590 2024-09-21T03:43:58.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.515+0000 7ff7e51e8740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7ff7d82030a0 con 0x55bd82f77270 2024-09-21T03:43:58.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.515+0000 7ff7d7fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bd82f4e3b0 0x55bd83318550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:58.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.515+0000 7ff7d7fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bd82f4e3b0 0x55bd83318550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50700/0 (socket says 172.21.15.38:50700) 2024-09-21T03:43:58.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.515+0000 7ff7d7fff640 1 -- 172.21.15.38:0/2249900672 learned_addr learned my addr 172.21.15.38:0/2249900672 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:43:58.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.515+0000 7ff7d77fe640 1 --2- 172.21.15.38:0/2249900672 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bd82f56590 0x55bd83318a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:58.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.515+0000 7ff7dc81b640 1 --2- 172.21.15.38:0/2249900672 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55bd82f77270 0x55bd8335c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:58.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.515+0000 7ff7d7fff640 1 -- 172.21.15.38:0/2249900672 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55bd82f77270 msgr2=0x55bd8335c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:58.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.515+0000 7ff7d7fff640 1 --2- 172.21.15.38:0/2249900672 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55bd82f77270 0x55bd8335c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:58.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.515+0000 7ff7d7fff640 1 -- 172.21.15.38:0/2249900672 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bd82f56590 msgr2=0x55bd83318a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:58.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.515+0000 7ff7d7fff640 1 --2- 172.21.15.38:0/2249900672 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bd82f56590 0x55bd83318a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:58.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.515+0000 7ff7d7fff640 1 -- 172.21.15.38:0/2249900672 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd8335bd10 con 0x55bd82f4e3b0 2024-09-21T03:43:58.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.515+0000 7ff7d77fe640 1 --2- 172.21.15.38:0/2249900672 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bd82f56590 0x55bd83318a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:43:58.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.515+0000 7ff7dc81b640 1 --2- 172.21.15.38:0/2249900672 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55bd82f77270 0x55bd8335c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:43:58.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.516+0000 7ff7d7fff640 1 --2- 172.21.15.38:0/2249900672 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bd82f4e3b0 0x55bd83318550 secure :-1 s=READY pgs=2120 cs=0 l=1 rev1=1 crypto rx=0x7ff7c8021980 tx=0x7ff7c80219b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d327f7a47d5ef1b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:58.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.516+0000 7ff7bd7fa640 1 -- 172.21.15.38:0/2249900672 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7c8025070 con 0x55bd82f4e3b0 2024-09-21T03:43:58.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.516+0000 7ff7bd7fa640 1 -- 172.21.15.38:0/2249900672 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff7c801b070 con 0x55bd82f4e3b0 2024-09-21T03:43:58.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.516+0000 7ff7bd7fa640 1 -- 172.21.15.38:0/2249900672 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7c80358f0 con 0x55bd82f4e3b0 2024-09-21T03:43:58.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.516+0000 7ff7e51e8740 1 -- 172.21.15.38:0/2249900672 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bd82f4e3b0 msgr2=0x55bd83318550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:58.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.516+0000 7ff7e51e8740 1 --2- 172.21.15.38:0/2249900672 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bd82f4e3b0 0x55bd83318550 secure :-1 s=READY pgs=2120 cs=0 l=1 rev1=1 crypto rx=0x7ff7c8021980 tx=0x7ff7c80219b0 comp rx=0 tx=0).stop 2024-09-21T03:43:58.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.517+0000 7ff7e51e8740 1 -- 172.21.15.38:0/2249900672 shutdown_connections 2024-09-21T03:43:58.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.517+0000 7ff7e51e8740 1 --2- 172.21.15.38:0/2249900672 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55bd82f77270 0x55bd8335c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:58.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.517+0000 7ff7e51e8740 1 --2- 172.21.15.38:0/2249900672 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bd82f56590 0x55bd83318a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:58.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.517+0000 7ff7e51e8740 1 --2- 172.21.15.38:0/2249900672 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bd82f4e3b0 0x55bd83318550 unknown :-1 s=CLOSED pgs=2120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:58.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.517+0000 7ff7e51e8740 1 -- 172.21.15.38:0/2249900672 >> 172.21.15.38:0/2249900672 conn(0x55bd82f6b3c0 msgr2=0x55bd833427f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:58.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.517+0000 7ff7e51e8740 1 -- 172.21.15.38:0/2249900672 shutdown_connections 2024-09-21T03:43:58.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.517+0000 7ff7e51e8740 1 -- 172.21.15.38:0/2249900672 wait complete. 2024-09-21T03:43:58.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.517+0000 7ff7e51e8740 1 -- start start 2024-09-21T03:43:58.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.518+0000 7ff7e51e8740 1 Processor -- start 2024-09-21T03:43:58.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.519+0000 7ff7e51e8740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bd82f4e3b0 0x55bd833859e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:58.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.519+0000 7ff7e51e8740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bd82f56590 0x55bd833e6350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:58.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.519+0000 7ff7e51e8740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55bd82f77270 0x55bd833e5a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:58.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.519+0000 7ff7e51e8740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55bd83342530 con 0x55bd82f56590 2024-09-21T03:43:58.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.519+0000 7ff7e51e8740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55bd82ff1620 con 0x55bd82f4e3b0 2024-09-21T03:43:58.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.519+0000 7ff7e51e8740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55bd833367d0 con 0x55bd82f77270 2024-09-21T03:43:58.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.519+0000 7ff7e51e8740 10 client.0 _ll_register_callbacks cb 0x7ff7c8023070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:43:58.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.519+0000 7ff7d77fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bd82f56590 0x55bd833e6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:58.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.519+0000 7ff7d77fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bd82f56590 0x55bd833e6350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50702/0 (socket says 172.21.15.38:50702) 2024-09-21T03:43:58.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.519+0000 7ff7d77fe640 1 -- 172.21.15.38:0/1850536552 learned_addr learned my addr 172.21.15.38:0/1850536552 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:43:58.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.519+0000 7ff7d7fff640 1 --2- 172.21.15.38:0/1850536552 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bd82f4e3b0 0x55bd833859e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:58.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.519+0000 7ff7d77fe640 1 -- 172.21.15.38:0/1850536552 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55bd82f77270 msgr2=0x55bd833e5a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:58.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.519+0000 7ff7dc81b640 1 --2- 172.21.15.38:0/1850536552 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55bd82f77270 0x55bd833e5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:58.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.519+0000 7ff7d77fe640 1 --2- 172.21.15.38:0/1850536552 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55bd82f77270 0x55bd833e5a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:58.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.519+0000 7ff7d77fe640 1 -- 172.21.15.38:0/1850536552 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bd82f4e3b0 msgr2=0x55bd833859e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:58.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.519+0000 7ff7d77fe640 1 --2- 172.21.15.38:0/1850536552 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bd82f4e3b0 0x55bd833859e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:58.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.519+0000 7ff7d77fe640 1 -- 172.21.15.38:0/1850536552 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd8331a3a0 con 0x55bd82f56590 2024-09-21T03:43:58.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.519+0000 7ff7d7fff640 1 --2- 172.21.15.38:0/1850536552 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bd82f4e3b0 0x55bd833859e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:43:58.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.520+0000 7ff7d77fe640 1 --2- 172.21.15.38:0/1850536552 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bd82f56590 0x55bd833e6350 secure :-1 s=READY pgs=2121 cs=0 l=1 rev1=1 crypto rx=0x7ff7cc060670 tx=0x7ff7cc09c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e99d810120a4de8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:58.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.520+0000 7ff7b6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-21T03:43:58.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.520+0000 7ff7b6ffd640 1 -- 172.21.15.38:0/1850536552 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7cc07d030 con 0x55bd82f56590 2024-09-21T03:43:58.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.520+0000 7ff7b6ffd640 1 -- 172.21.15.38:0/1850536552 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(130..130 src has 1..130) ==== 9228+0+0 (secure 0 0 0) 0x7ff7cc09c0b0 con 0x55bd82f56590 2024-09-21T03:43:58.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.520+0000 7ff7e51e8740 1 -- 172.21.15.38:0/1850536552 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff7c801b070 con 0x55bd82f56590 2024-09-21T03:43:58.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.520+0000 7ff7b6ffd640 1 -- 172.21.15.38:0/1850536552 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=297) ==== 24+0+0 (secure 0 0 0) 0x7ff7cc063400 con 0x55bd82f56590 2024-09-21T03:43:58.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.520+0000 7ff7e51e8740 10 client.10170 fetch_fsmap learned FSMap version 297 2024-09-21T03:43:58.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.520+0000 7ff7e51e8740 1 -- 172.21.15.38:0/1850536552 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=297,monmap=2+}) -- 0x55bd833e6990 con 0x55bd82f56590 2024-09-21T03:43:58.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.520+0000 7ff7b6ffd640 1 -- 172.21.15.38:0/1850536552 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff7c801b070 con 0x55bd82f56590 2024-09-21T03:43:58.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.520+0000 7ff7b6ffd640 1 -- 172.21.15.38:0/1850536552 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 297) ==== 3502+0+0 (secure 0 0 0) 0x7ff7cc05f670 con 0x55bd82f56590 2024-09-21T03:43:58.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.520+0000 7ff7e51e8740 10 client.10170 fetch_fsmap finished waiting for FSMap version 297 2024-09-21T03:43:58.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.520+0000 7ff7e51e8740 10 client.10170 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T03:43:58.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.520+0000 7ff7e51e8740 20 client.10170 populate_metadata read hostname 'smithi038' 2024-09-21T03:43:58.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.520+0000 7ff7e51e8740 1 --2- 172.21.15.38:0/1850536552 >> [v2:172.21.15.38:6832/2444649344,v1:172.21.15.38:6833/2444649344] conn(0x55bd83318870 0x55bd8337c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:58.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.520+0000 7ff7e51e8740 4 client.10170 mds_command: new command op to 10113 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T03:43:58.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.520+0000 7ff7e51e8740 1 -- 172.21.15.38:0/1850536552 --> [v2:172.21.15.38:6832/2444649344,v1:172.21.15.38:6833/2444649344] -- command(tid 0: {"prefix": "status"}) -- 0x55bd8328f220 con 0x55bd83318870 2024-09-21T03:43:58.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.520+0000 7ff7d7fff640 1 --2- 172.21.15.38:0/1850536552 >> [v2:172.21.15.38:6832/2444649344,v1:172.21.15.38:6833/2444649344] conn(0x55bd83318870 0x55bd8337c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:58.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.521+0000 7ff7b6ffd640 10 client.10170 ms_handle_connect on v2:172.21.15.38:6832/2444649344 2024-09-21T03:43:58.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.521+0000 7ff7d7fff640 1 --2- 172.21.15.38:0/1850536552 >> [v2:172.21.15.38:6832/2444649344,v1:172.21.15.38:6833/2444649344] conn(0x55bd83318870 0x55bd8337c3d0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45e0d2303e8ca026 server_cookie=6e328686b5736fa8 in_seq=0 out_seq=0 2024-09-21T03:43:58.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.521+0000 7ff7b6ffd640 1 -- 172.21.15.38:0/1850536552 <== mds.0 v2:172.21.15.38:6832/2444649344 1 ==== command_reply(tid 0: 0 ) ==== 8+0+328 (crc 0 0 0) 0x55bd8328f220 con 0x55bd83318870 2024-09-21T03:43:58.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.521+0000 7ff7b6ffd640 10 client.10170 handle_command_reply: tid=0 2024-09-21T03:43:58.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.521+0000 7ff7e51e8740 1 client.10170 shutdown 2024-09-21T03:43:58.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.521+0000 7ff7e51e8740 1 -- 172.21.15.38:0/1850536552 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bd82f56590 msgr2=0x55bd833e6350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:58.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.521+0000 7ff7e51e8740 1 --2- 172.21.15.38:0/1850536552 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bd82f56590 0x55bd833e6350 secure :-1 s=READY pgs=2121 cs=0 l=1 rev1=1 crypto rx=0x7ff7cc060670 tx=0x7ff7cc09c040 comp rx=0 tx=0).stop 2024-09-21T03:43:58.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.521+0000 7ff7e51e8740 1 -- 172.21.15.38:0/1850536552 shutdown_connections 2024-09-21T03:43:58.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.521+0000 7ff7e51e8740 1 --2- 172.21.15.38:0/1850536552 >> [v2:172.21.15.38:6832/2444649344,v1:172.21.15.38:6833/2444649344] conn(0x55bd83318870 0x55bd8337c3d0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:58.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.521+0000 7ff7e51e8740 1 --2- 172.21.15.38:0/1850536552 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55bd82f77270 0x55bd833e5a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:58.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.521+0000 7ff7e51e8740 1 --2- 172.21.15.38:0/1850536552 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bd82f56590 0x55bd833e6350 unknown :-1 s=CLOSED pgs=2121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:58.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.521+0000 7ff7b6ffd640 0 client.10170 ms_handle_reset on v2:172.21.15.38:6832/2444649344 2024-09-21T03:43:58.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.521+0000 7ff7e51e8740 1 --2- 172.21.15.38:0/1850536552 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bd82f4e3b0 0x55bd833859e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:58.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.521+0000 7ff7e51e8740 1 -- 172.21.15.38:0/1850536552 >> 172.21.15.38:0/1850536552 conn(0x55bd82f6b3c0 msgr2=0x55bd833426b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:58.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.521+0000 7ff7e51e8740 1 -- 172.21.15.38:0/1850536552 shutdown_connections 2024-09-21T03:43:58.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.521+0000 7ff7e51e8740 1 -- 172.21.15.38:0/1850536552 wait complete. 2024-09-21T03:43:58.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.522+0000 7ff7e51e8740 20 client.10170 trim_cache size 0 max 16384 2024-09-21T03:43:58.531 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-09-21T03:43:58.531 INFO:teuthology.orchestra.run.smithi038.stdout: "cluster_fsid": "9d50d66f-656b-4fe2-8e3b-fe53b79c4d43", 2024-09-21T03:43:58.531 INFO:teuthology.orchestra.run.smithi038.stdout: "whoami": 0, 2024-09-21T03:43:58.531 INFO:teuthology.orchestra.run.smithi038.stdout: "id": 10113, 2024-09-21T03:43:58.531 INFO:teuthology.orchestra.run.smithi038.stdout: "want_state": "up:active", 2024-09-21T03:43:58.531 INFO:teuthology.orchestra.run.smithi038.stdout: "state": "up:active", 2024-09-21T03:43:58.531 INFO:teuthology.orchestra.run.smithi038.stdout: "fs_name": "cephfs", 2024-09-21T03:43:58.531 INFO:teuthology.orchestra.run.smithi038.stdout: "rank_uptime": 6.7511314870000003, 2024-09-21T03:43:58.531 INFO:teuthology.orchestra.run.smithi038.stdout: "mdsmap_epoch": 297, 2024-09-21T03:43:58.531 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch": 130, 2024-09-21T03:43:58.531 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch_barrier": 130, 2024-09-21T03:43:58.532 INFO:teuthology.orchestra.run.smithi038.stdout: "uptime": 9.274190591 2024-09-21T03:43:58.532 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-09-21T03:43:58.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.522+0000 7ff7de27e640 1 -- 172.21.15.38:0/881193469 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff7c4078b90 msgr2=0x7ff7c407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:58.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.522+0000 7ff7de27e640 1 --2- 172.21.15.38:0/881193469 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff7c4078b90 0x7ff7c407b0a0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7ff7c8001720 tx=0x7ff7c8009040 comp rx=0 tx=0).stop 2024-09-21T03:43:58.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.522+0000 7ff7de27e640 1 -- 172.21.15.38:0/881193469 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff7d80d1120 msgr2=0x7ff7d806eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:58.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.522+0000 7ff7de27e640 1 --2- 172.21.15.38:0/881193469 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff7d80d1120 0x7ff7d806eb30 secure :-1 s=READY pgs=2117 cs=0 l=1 rev1=1 crypto rx=0x7ff7cc00ea20 tx=0x7ff7cc00eee0 comp rx=0 tx=0).stop 2024-09-21T03:43:58.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.522+0000 7ff7de27e640 1 -- 172.21.15.38:0/881193469 shutdown_connections 2024-09-21T03:43:58.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.522+0000 7ff7de27e640 1 --2- 172.21.15.38:0/881193469 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff7d815ee80 0x7ff7d81f8b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:58.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.522+0000 7ff7de27e640 1 --2- 172.21.15.38:0/881193469 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff7d80d1120 0x7ff7d806eb30 unknown :-1 s=CLOSED pgs=2117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:58.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.522+0000 7ff7de27e640 1 --2- 172.21.15.38:0/881193469 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff7c4078b90 0x7ff7c407b0a0 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:58.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.522+0000 7ff7de27e640 1 --2- 172.21.15.38:0/881193469 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff7d80d0170 0x7ff7d80cd240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:58.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.522+0000 7ff7de27e640 1 -- 172.21.15.38:0/881193469 >> 172.21.15.38:0/881193469 conn(0x7ff7d809d830 msgr2=0x7ff7d815c9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:58.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.522+0000 7ff7de27e640 1 -- 172.21.15.38:0/881193469 shutdown_connections 2024-09-21T03:43:58.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.522+0000 7ff7de27e640 1 -- 172.21.15.38:0/881193469 wait complete. 2024-09-21T03:43:58.535 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-09-21T03:43:58.535 INFO:tasks.cephfs.test_damage:Daemons came up after mutation 'Corrupt omap header on 10000000000.00000000', proceeding to ls 2024-09-21T03:43:58.536 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T03:43:58.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.649+0000 7f8eaad62640 1 -- 172.21.15.38:0/2719340993 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8ea40cfe00 msgr2=0x7f8ea40d2a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:58.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.649+0000 7f8eaad62640 1 --2- 172.21.15.38:0/2719340993 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8ea40cfe00 0x7f8ea40d2a20 secure :-1 s=READY pgs=2123 cs=0 l=1 rev1=1 crypto rx=0x7f8e98009830 tx=0x7f8e9802a510 comp rx=0 tx=0).stop 2024-09-21T03:43:58.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.649+0000 7f8eaad62640 1 -- 172.21.15.38:0/2719340993 shutdown_connections 2024-09-21T03:43:58.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.649+0000 7f8eaad62640 1 --2- 172.21.15.38:0/2719340993 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8ea40b2e60 0x7f8ea40b3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:58.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.649+0000 7f8eaad62640 1 --2- 172.21.15.38:0/2719340993 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8ea40cfe00 0x7f8ea40d2a20 unknown :-1 s=CLOSED pgs=2123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:58.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.649+0000 7f8eaad62640 1 -- 172.21.15.38:0/2719340993 >> 172.21.15.38:0/2719340993 conn(0x7f8ea40d4440 msgr2=0x7f8ea40d4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:58.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.649+0000 7f8eaad62640 1 -- 172.21.15.38:0/2719340993 shutdown_connections 2024-09-21T03:43:58.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.649+0000 7f8eaad62640 1 -- 172.21.15.38:0/2719340993 wait complete. 2024-09-21T03:43:58.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.651+0000 7f8eaad62640 1 Processor -- start 2024-09-21T03:43:58.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.651+0000 7f8eaad62640 1 -- start start 2024-09-21T03:43:58.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.653+0000 7f8eaad62640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8ea40b2e60 0x7f8ea41ea6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:58.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.653+0000 7f8eaad62640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8ea40cfe00 0x7f8ea41eac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:58.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.653+0000 7f8eaad62640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8ea4162590 0x7f8ea41f8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:58.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.653+0000 7f8eaad62640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f8ea40b3930 con 0x7f8ea4162590 2024-09-21T03:43:58.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.653+0000 7f8eaad62640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f8ea40b37b0 con 0x7f8ea40b2e60 2024-09-21T03:43:58.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.653+0000 7f8eaad62640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f8ea40b3ab0 con 0x7f8ea40cfe00 2024-09-21T03:43:58.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.653+0000 7f8ea92ff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8ea4162590 0x7f8ea41f8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:58.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.653+0000 7f8ea92ff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8ea4162590 0x7f8ea41f8a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50718/0 (socket says 172.21.15.38:50718) 2024-09-21T03:43:58.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.653+0000 7f8ea92ff640 1 -- 172.21.15.38:0/3999190202 learned_addr learned my addr 172.21.15.38:0/3999190202 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:43:58.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.653+0000 7f8ea8afe640 1 --2- 172.21.15.38:0/3999190202 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8ea40b2e60 0x7f8ea41ea6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:58.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.653+0000 7f8ea3fff640 1 --2- 172.21.15.38:0/3999190202 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8ea40cfe00 0x7f8ea41eac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:58.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.653+0000 7f8ea92ff640 1 -- 172.21.15.38:0/3999190202 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8ea40cfe00 msgr2=0x7f8ea41eac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:58.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.653+0000 7f8ea92ff640 1 --2- 172.21.15.38:0/3999190202 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8ea40cfe00 0x7f8ea41eac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:58.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.653+0000 7f8ea92ff640 1 -- 172.21.15.38:0/3999190202 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8ea40b2e60 msgr2=0x7f8ea41ea6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:58.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.653+0000 7f8ea92ff640 1 --2- 172.21.15.38:0/3999190202 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8ea40b2e60 0x7f8ea41ea6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:58.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.653+0000 7f8ea92ff640 1 -- 172.21.15.38:0/3999190202 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ea415def0 con 0x7f8ea4162590 2024-09-21T03:43:58.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.653+0000 7f8ea8afe640 1 --2- 172.21.15.38:0/3999190202 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8ea40b2e60 0x7f8ea41ea6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:43:58.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.653+0000 7f8ea92ff640 1 --2- 172.21.15.38:0/3999190202 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8ea4162590 0x7f8ea41f8a60 secure :-1 s=READY pgs=2124 cs=0 l=1 rev1=1 crypto rx=0x7f8e9c00cad0 tx=0x7f8e9c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:58.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.653+0000 7f8ea17fa640 1 -- 172.21.15.38:0/3999190202 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e9c00ae20 con 0x7f8ea4162590 2024-09-21T03:43:58.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.654+0000 7f8ea17fa640 1 -- 172.21.15.38:0/3999190202 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8e9c00f450 con 0x7f8ea4162590 2024-09-21T03:43:58.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.654+0000 7f8eaad62640 1 -- 172.21.15.38:0/3999190202 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ea415e360 con 0x7f8ea4162590 2024-09-21T03:43:58.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.654+0000 7f8ea17fa640 1 -- 172.21.15.38:0/3999190202 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e9c00f8c0 con 0x7f8ea4162590 2024-09-21T03:43:58.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.654+0000 7f8eaad62640 1 -- 172.21.15.38:0/3999190202 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ea41e9ed0 con 0x7f8ea4162590 2024-09-21T03:43:58.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.655+0000 7f8eaad62640 1 -- 172.21.15.38:0/3999190202 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ea40b3270 con 0x7f8ea4162590 2024-09-21T03:43:58.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.655+0000 7f8ea17fa640 1 -- 172.21.15.38:0/3999190202 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f8e9c00fa60 con 0x7f8ea4162590 2024-09-21T03:43:58.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.656+0000 7f8ea17fa640 1 --2- 172.21.15.38:0/3999190202 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8e88078b90 0x7f8e8807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:58.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.656+0000 7f8ea17fa640 1 -- 172.21.15.38:0/3999190202 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 9228+0+0 (secure 0 0 0) 0x7f8e9c09c590 con 0x7f8ea4162590 2024-09-21T03:43:58.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.656+0000 7f8ea8afe640 1 --2- 172.21.15.38:0/3999190202 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8e88078b90 0x7f8e8807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:58.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.657+0000 7f8ea8afe640 1 --2- 172.21.15.38:0/3999190202 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8e88078b90 0x7f8e8807b0a0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f8e98034000 tx=0x7f8e98004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:58.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.661+0000 7f8ea17fa640 1 -- 172.21.15.38:0/3999190202 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f8e9c014030 con 0x7f8ea4162590 2024-09-21T03:43:58.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.887+0000 7f8eaad62640 1 -- 172.21.15.38:0/3999190202 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f8ea415e0c0 con 0x7f8ea4162590 2024-09-21T03:43:58.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.887+0000 7f8ea17fa640 1 -- 172.21.15.38:0/3999190202 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v297) ==== 53+0+73 (secure 0 0 0) 0x7f8e9c063690 con 0x7f8ea4162590 2024-09-21T03:43:58.889 INFO:teuthology.orchestra.run.smithi038.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T03:43:58.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.889+0000 7f8eaad62640 1 -- 172.21.15.38:0/3999190202 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8e88078b90 msgr2=0x7f8e8807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:58.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.889+0000 7f8eaad62640 1 --2- 172.21.15.38:0/3999190202 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8e88078b90 0x7f8e8807b0a0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f8e98034000 tx=0x7f8e98004730 comp rx=0 tx=0).stop 2024-09-21T03:43:58.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.889+0000 7f8eaad62640 1 -- 172.21.15.38:0/3999190202 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8ea4162590 msgr2=0x7f8ea41f8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:58.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.889+0000 7f8eaad62640 1 --2- 172.21.15.38:0/3999190202 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8ea4162590 0x7f8ea41f8a60 secure :-1 s=READY pgs=2124 cs=0 l=1 rev1=1 crypto rx=0x7f8e9c00cad0 tx=0x7f8e9c00cf90 comp rx=0 tx=0).stop 2024-09-21T03:43:58.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.890+0000 7f8eaad62640 1 -- 172.21.15.38:0/3999190202 shutdown_connections 2024-09-21T03:43:58.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.890+0000 7f8eaad62640 1 --2- 172.21.15.38:0/3999190202 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8ea4162590 0x7f8ea41f8a60 unknown :-1 s=CLOSED pgs=2124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:58.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.890+0000 7f8eaad62640 1 --2- 172.21.15.38:0/3999190202 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8e88078b90 0x7f8e8807b0a0 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:58.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.890+0000 7f8eaad62640 1 --2- 172.21.15.38:0/3999190202 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8ea40cfe00 0x7f8ea41eac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:58.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.890+0000 7f8eaad62640 1 --2- 172.21.15.38:0/3999190202 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8ea40b2e60 0x7f8ea41ea6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:58.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.890+0000 7f8eaad62640 1 -- 172.21.15.38:0/3999190202 >> 172.21.15.38:0/3999190202 conn(0x7f8ea40d4440 msgr2=0x7f8ea406e440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:58.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.890+0000 7f8eaad62640 1 -- 172.21.15.38:0/3999190202 shutdown_connections 2024-09-21T03:43:58.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:58.890+0000 7f8eaad62640 1 -- 172.21.15.38:0/3999190202 wait complete. 2024-09-21T03:43:58.903 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-21T03:43:58.903 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T03:43:58.903 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi038.front.sepia.ceph.com 2024-09-21T03:43:58.903 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-21T03:43:58.903 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T03:43:58.903 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-21T03:43:58.904 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-21T03:43:58.904 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T03:43:58.904 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T03:43:58.904 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:43:58.904 DEBUG:teuthology.orchestra.run.smithi038:> ip addr 2024-09-21T03:43:58.920 INFO:teuthology.orchestra.run.smithi038.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T03:43:58.920 INFO:teuthology.orchestra.run.smithi038.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T03:43:58.920 INFO:teuthology.orchestra.run.smithi038.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T03:43:58.921 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:43:58.921 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 ::1/128 scope host 2024-09-21T03:43:58.921 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:43:58.921 INFO:teuthology.orchestra.run.smithi038.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T03:43:58.921 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:6b:fd:a2 brd ff:ff:ff:ff:ff:ff 2024-09-21T03:43:58.921 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp7s0f0 2024-09-21T03:43:58.921 INFO:teuthology.orchestra.run.smithi038.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T03:43:58.921 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:6b:fd:a3 brd ff:ff:ff:ff:ff:ff 2024-09-21T03:43:58.921 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp7s0f1 2024-09-21T03:43:58.921 INFO:teuthology.orchestra.run.smithi038.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T03:43:58.921 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:bd:15:fc brd ff:ff:ff:ff:ff:ff 2024-09-21T03:43:58.922 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp1s0f0 2024-09-21T03:43:58.922 INFO:teuthology.orchestra.run.smithi038.stdout: inet 172.21.15.38/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-21T03:43:58.922 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft 42170sec preferred_lft 42170sec 2024-09-21T03:43:58.922 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 fe80::ec4:7aff:febd:15fc/64 scope link 2024-09-21T03:43:58.922 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:43:58.922 INFO:teuthology.orchestra.run.smithi038.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T03:43:58.922 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:bd:15:fd brd ff:ff:ff:ff:ff:ff 2024-09-21T03:43:58.922 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp1s0f1 2024-09-21T03:43:58.922 INFO:teuthology.orchestra.run.smithi038.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T03:43:58.922 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 92:55:c7:97:6f:b5 brd ff:ff:ff:ff:ff:ff 2024-09-21T03:43:58.922 INFO:teuthology.orchestra.run.smithi038.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-21T03:43:58.922 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:43:58.923 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 fe80::c49f:c1ff:fe21:b12a/64 scope link 2024-09-21T03:43:58.923 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:43:58.923 INFO:teuthology.orchestra.run.smithi038.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T03:43:58.923 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether a6:3a:83:ad:db:d5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T03:43:58.923 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 fe80::a43a:83ff:fead:dbd5/64 scope link 2024-09-21T03:43:58.923 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:43:58.923 INFO:teuthology.orchestra.run.smithi038.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T03:43:58.923 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 92:55:c7:97:6f:b5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T03:43:58.923 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 fe80::9055:c7ff:fe97:6fb5/64 scope link 2024-09-21T03:43:58.923 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:43:58.923 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:43:58.924 DEBUG:teuthology.orchestra.run.smithi038:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T03:43:58.984 INFO:teuthology.orchestra.run.smithi038.stdout:1 2024-09-21T03:43:58.986 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:43:58.986 DEBUG:teuthology.orchestra.run.smithi038:> route 2024-09-21T03:43:59.043 INFO:teuthology.orchestra.run.smithi038.stdout:Kernel IP routing table 2024-09-21T03:43:59.043 INFO:teuthology.orchestra.run.smithi038.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T03:43:59.044 INFO:teuthology.orchestra.run.smithi038.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-21T03:43:59.044 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-21T03:43:59.044 INFO:teuthology.orchestra.run.smithi038.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T03:43:59.044 DEBUG:teuthology.orchestra.run.smithi038:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T03:43:59.044 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-09-21T03:43:59.044 DEBUG:teuthology.orchestra.run.smithi038:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-21T03:43:59.044 DEBUG:teuthology.orchestra.run.smithi038:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-21T03:43:59.044 DEBUG:teuthology.orchestra.run.smithi038:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-21T03:43:59.044 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-09-21T03:43:59.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:59 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T03:43:59.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:43:59.174 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:43:59.174 DEBUG:teuthology.orchestra.run.smithi038:> ip netns list 2024-09-21T03:43:59.190 INFO:teuthology.orchestra.run.smithi038.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-21T03:43:59.190 INFO:teuthology.orchestra.run.smithi038.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-21T03:43:59.190 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T03:43:59.190 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:43:59.190 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip link set brx.0 up 2024-09-21T03:43:59.255 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T03:43:59.255 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-21T03:43:59.311 INFO:teuthology.orchestra.run.smithi038.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-21T03:43:59.312 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T03:43:59.312 DEBUG:teuthology.orchestra.run.smithi038:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-21T03:43:59.367 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-21T03:43:59.367 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:43:59.367 DEBUG:teuthology.orchestra.run.smithi038:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T03:43:59.462 INFO:teuthology.orchestra.run.smithi038.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T03:43:59.462 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T03:43:59.462 INFO:teuthology.orchestra.run.smithi038.stdout:invalid new device string format 2024-09-21T03:43:59.473 INFO:teuthology.orchestra.run.smithi038.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T03:43:59.473 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T03:43:59.473 INFO:teuthology.orchestra.run.smithi038.stdout:invalid new device string format 2024-09-21T03:43:59.473 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: resolved to: "172.21.15.38:3300,172.21.15.195:3300,172.21.15.195:3301" 2024-09-21T03:43:59.473 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T03:43:59.484 INFO:teuthology.orchestra.run.smithi038.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T03:43:59.484 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T03:43:59.484 INFO:teuthology.orchestra.run.smithi038.stdout:invalid new device string format 2024-09-21T03:43:59.484 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: resolved to: "172.21.15.38:3300,172.21.15.195:3300,172.21.15.195:3301" 2024-09-21T03:43:59.484 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T03:43:59.484 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: resolved to: "172.21.15.38:6789,172.21.15.195:6789,172.21.15.195:6790" 2024-09-21T03:43:59.485 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.38:6789,172.21.15.195:6789,172.21.15.195:6790:/ 2024-09-21T03:43:59.485 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=9d50d66f-656b-4fe2-8e3b-fe53b79c4d43" will pass to kernel 2024-09-21T03:43:59.485 INFO:teuthology.orchestra.run.smithi038.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-21T03:43:59.485 INFO:teuthology.orchestra.run.smithi038.stdout: You just mounted a file system that supports labels which does not 2024-09-21T03:43:59.485 INFO:teuthology.orchestra.run.smithi038.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-21T03:43:59.485 INFO:teuthology.orchestra.run.smithi038.stdout: applications will generate AVC messages and not be allowed access to 2024-09-21T03:43:59.485 INFO:teuthology.orchestra.run.smithi038.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-21T03:43:59.487 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T03:43:59.487 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:43:59.487 DEBUG:teuthology.orchestra.run.smithi038:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-21T03:43:59.512 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T03:43:59.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:59 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T03:43:59.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:43:59.610 INFO:teuthology.orchestra.run.smithi038.stdout:client9181 2024-09-21T03:43:59.610 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-21T03:43:59.610 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:43:59.626 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:43:59.626 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls -R .) 2024-09-21T03:43:59.669 INFO:teuthology.orchestra.run:waiting for 20 2024-09-21T03:43:59.684 INFO:tasks.ceph.mds.c.smithi038.stderr:2024-09-21T03:43:59.682+0000 7f1b90a8c640 -1 mds.0.cache.dir(0x10000000000) Corrupt fnode in dirfrag 0x10000000000: Decoder at 'void fnode_t::decode(ceph::buffer::v15_2_0::list::const_iterator&)' v=3 cannot decode v=100 minimal_decoder=101: Malformed input [buffer:3] 2024-09-21T03:43:59.685 INFO:teuthology.orchestra.run.smithi038.stdout:.: 2024-09-21T03:43:59.685 INFO:teuthology.orchestra.run.smithi038.stdout:subdir 2024-09-21T03:43:59.685 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:43:59.685 INFO:teuthology.orchestra.run.smithi038.stdout:./subdir: 2024-09-21T03:43:59.685 INFO:tasks.cephfs.test_damage:Result: Mutation 'Corrupt omap header on 10000000000.00000000' did not caused DAMAGED state 2024-09-21T03:43:59.685 INFO:tasks.cephfs.test_damage:Applying mutation 'Corrupt omap header on 600.00000000' 2024-09-21T03:43:59.685 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-21T03:43:59.685 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:43:59.741 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-21T03:43:59.741 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:43:59.797 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T03:43:59.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:59.952+0000 7f651c758640 1 -- 172.21.15.38:0/1240807836 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6500004d40 msgr2=0x7f65000051f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:59.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:59.952+0000 7f651c758640 1 --2- 172.21.15.38:0/1240807836 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6500004d40 0x7f65000051f0 secure :-1 s=READY pgs=2126 cs=0 l=1 rev1=1 crypto rx=0x7f650c009ed0 tx=0x7f6500005830 comp rx=0 tx=0).stop 2024-09-21T03:43:59.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:59.952+0000 7f651acf5640 1 -- 172.21.15.38:0/1240807836 reap_dead start 2024-09-21T03:43:59.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:59.953+0000 7f651c758640 1 -- 172.21.15.38:0/1240807836 shutdown_connections 2024-09-21T03:43:59.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:59.953+0000 7f651c758640 1 -- 172.21.15.38:0/1240807836 >> 172.21.15.38:0/1240807836 conn(0x7f651409d830 msgr2=0x7f65140b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:59.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:59.953+0000 7f651c758640 1 -- 172.21.15.38:0/1240807836 shutdown_connections 2024-09-21T03:43:59.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:59.953+0000 7f651c758640 1 -- 172.21.15.38:0/1240807836 wait complete. 2024-09-21T03:43:59.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:59.954+0000 7f651c758640 1 Processor -- start 2024-09-21T03:43:59.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:59.954+0000 7f651c758640 1 -- start start 2024-09-21T03:43:59.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:59.956+0000 7f651c758640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6500004d40 0x7f65141f9c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:59.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:59.956+0000 7f651c758640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f65141f6580 0x7f6514157910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:59.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:59.956+0000 7f651c758640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f651406cde0 0x7f651406d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:59.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:59.956+0000 7f651c758640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f65140cf330 con 0x7f65141f6580 2024-09-21T03:43:59.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:59.956+0000 7f651c758640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f65140cf1b0 con 0x7f651406cde0 2024-09-21T03:43:59.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:59.956+0000 7f651c758640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f65140d1480 con 0x7f6500004d40 2024-09-21T03:43:59.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:59.956+0000 7f6519cf3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f65141f6580 0x7f6514157910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:59.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:59.956+0000 7f6519cf3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f65141f6580 0x7f6514157910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50734/0 (socket says 172.21.15.38:50734) 2024-09-21T03:43:59.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:59.956+0000 7f6519cf3640 1 -- 172.21.15.38:0/411653951 learned_addr learned my addr 172.21.15.38:0/411653951 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:43:59.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:59.956+0000 7f651a4f4640 1 --2- 172.21.15.38:0/411653951 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6500004d40 0x7f65141f9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:59.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:59.956+0000 7f651acf5640 1 --2- 172.21.15.38:0/411653951 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f651406cde0 0x7f651406d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:59.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:59.956+0000 7f6519cf3640 1 -- 172.21.15.38:0/411653951 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6500004d40 msgr2=0x7f65141f9c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:59.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:59.956+0000 7f6519cf3640 1 --2- 172.21.15.38:0/411653951 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6500004d40 0x7f65141f9c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:59.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:59.956+0000 7f6519cf3640 1 -- 172.21.15.38:0/411653951 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f651406cde0 msgr2=0x7f651406d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:59.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:59.956+0000 7f6519cf3640 1 --2- 172.21.15.38:0/411653951 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f651406cde0 0x7f651406d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:59.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:59.956+0000 7f6519cf3640 1 -- 172.21.15.38:0/411653951 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f651406fb70 con 0x7f65141f6580 2024-09-21T03:43:59.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:59.956+0000 7f651a4f4640 1 --2- 172.21.15.38:0/411653951 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6500004d40 0x7f65141f9c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:43:59.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:59.956+0000 7f651acf5640 1 --2- 172.21.15.38:0/411653951 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f651406cde0 0x7f651406d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:43:59.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:59.956+0000 7f6519cf3640 1 --2- 172.21.15.38:0/411653951 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f65141f6580 0x7f6514157910 secure :-1 s=READY pgs=2127 cs=0 l=1 rev1=1 crypto rx=0x7f6504003ec0 tx=0x7f650400f720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:59.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:59.957+0000 7f650affd640 1 -- 172.21.15.38:0/411653951 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6504015070 con 0x7f65141f6580 2024-09-21T03:43:59.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:59.957+0000 7f650affd640 1 -- 172.21.15.38:0/411653951 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6504011860 con 0x7f65141f6580 2024-09-21T03:43:59.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:59.957+0000 7f651c758640 1 -- 172.21.15.38:0/411653951 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f651406f890 con 0x7f65141f6580 2024-09-21T03:43:59.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:59.957+0000 7f650affd640 1 -- 172.21.15.38:0/411653951 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6504011cb0 con 0x7f65141f6580 2024-09-21T03:43:59.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:59.957+0000 7f651c758640 1 -- 172.21.15.38:0/411653951 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65141f9430 con 0x7f65141f6580 2024-09-21T03:43:59.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:59.958+0000 7f651c758640 1 -- 172.21.15.38:0/411653951 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f651406f600 con 0x7f65141f6580 2024-09-21T03:43:59.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:59.958+0000 7f650affd640 1 -- 172.21.15.38:0/411653951 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f65040103c0 con 0x7f65141f6580 2024-09-21T03:43:59.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:59.959+0000 7f650affd640 1 --2- 172.21.15.38:0/411653951 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f64f8078bc0 0x7f64f807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:59.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:59.959+0000 7f650affd640 1 -- 172.21.15.38:0/411653951 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 9228+0+0 (secure 0 0 0) 0x7f650409c680 con 0x7f65141f6580 2024-09-21T03:43:59.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:59.959+0000 7f651a4f4640 1 --2- 172.21.15.38:0/411653951 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f64f8078bc0 0x7f64f807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:59.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:59.960+0000 7f651a4f4640 1 --2- 172.21.15.38:0/411653951 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f64f8078bc0 0x7f64f807b0d0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f6510001050 tx=0x7f6510006930 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:59.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:43:59.963+0000 7f650affd640 1 -- 172.21.15.38:0/411653951 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f65140cf4e0 con 0x7f65141f6580 2024-09-21T03:44:00.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:00.162+0000 7f651c758640 1 -- 172.21.15.38:0/411653951 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f65140d1000 con 0x7f65141f6580 2024-09-21T03:44:00.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:00.163+0000 7f650affd640 1 -- 172.21.15.38:0/411653951 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 68 entries v130) ==== 81+0+4132 (secure 0 0 0) 0x7f65140d1000 con 0x7f65141f6580 2024-09-21T03:44:00.165 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6833/626794495 2024-09-22T03:43:49.180309+0000 2024-09-21T03:44:00.165 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6832/1643781179 2024-09-22T03:43:33.596532+0000 2024-09-21T03:44:00.165 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6833/2309772457 2024-09-22T03:43:28.198223+0000 2024-09-21T03:44:00.165 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6832/2309772457 2024-09-22T03:43:28.198223+0000 2024-09-21T03:44:00.165 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6835/141447142 2024-09-22T03:43:14.684929+0000 2024-09-21T03:44:00.165 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6834/141447142 2024-09-22T03:43:14.684929+0000 2024-09-21T03:44:00.166 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6835/3572946698 2024-09-22T03:41:44.737273+0000 2024-09-21T03:44:00.166 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6833/3553953266 2024-09-22T03:41:29.639383+0000 2024-09-21T03:44:00.166 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6834/2751962827 2024-09-22T03:40:38.029580+0000 2024-09-21T03:44:00.166 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6832/3553953266 2024-09-22T03:41:29.639383+0000 2024-09-21T03:44:00.166 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6834/2930805359 2024-09-22T03:42:23.215190+0000 2024-09-21T03:44:00.166 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6832/626794495 2024-09-22T03:43:49.180309+0000 2024-09-21T03:44:00.166 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6833/1643781179 2024-09-22T03:43:33.596532+0000 2024-09-21T03:44:00.166 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6835/2748959970 2024-09-22T03:41:16.565736+0000 2024-09-21T03:44:00.166 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6833/3710736169 2024-09-22T03:40:59.454480+0000 2024-09-21T03:44:00.166 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6833/1995903506 2024-09-22T03:43:00.406120+0000 2024-09-21T03:44:00.166 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6833/775682959 2024-09-22T03:43:10.570678+0000 2024-09-21T03:44:00.167 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6834/2748959970 2024-09-22T03:41:16.565736+0000 2024-09-21T03:44:00.167 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6834/2934569248 2024-09-22T03:40:26.934591+0000 2024-09-21T03:44:00.167 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6832/3726830644 2024-09-22T03:41:12.532341+0000 2024-09-21T03:44:00.167 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6835/2097789246 2024-09-22T03:43:06.604471+0000 2024-09-21T03:44:00.167 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6835/2934569248 2024-09-22T03:40:26.934591+0000 2024-09-21T03:44:00.167 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6835/3864487171 2024-09-22T03:40:31.989837+0000 2024-09-21T03:44:00.167 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6833/4221150556 2024-09-22T03:42:27.240627+0000 2024-09-21T03:44:00.167 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6832/121492560 2024-09-22T03:41:10.371933+0000 2024-09-21T03:44:00.167 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6832/1717955365 2024-09-22T03:41:51.787804+0000 2024-09-21T03:44:00.167 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6834/2418225210 2024-09-22T03:42:02.869612+0000 2024-09-21T03:44:00.167 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6835/2418225210 2024-09-22T03:42:02.869612+0000 2024-09-21T03:44:00.167 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6833/121492560 2024-09-22T03:41:10.371933+0000 2024-09-21T03:44:00.168 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6835/3951934534 2024-09-22T03:42:31.276099+0000 2024-09-21T03:44:00.168 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6833/751618334 2024-09-22T03:40:42.053279+0000 2024-09-21T03:44:00.168 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.195:6837/712370966 2024-09-22T03:40:11.633787+0000 2024-09-21T03:44:00.168 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6832/3402963947 2024-09-22T03:41:23.555005+0000 2024-09-21T03:44:00.168 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6835/2798021708 2024-09-22T03:42:47.436632+0000 2024-09-21T03:44:00.168 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6833/3726830644 2024-09-22T03:41:12.532341+0000 2024-09-21T03:44:00.168 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6834/2916238332 2024-09-22T03:41:42.553770+0000 2024-09-21T03:44:00.168 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6834/3055025811 2024-09-22T03:42:39.361236+0000 2024-09-21T03:44:00.168 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.195:6835/712370966 2024-09-22T03:40:11.633787+0000 2024-09-21T03:44:00.168 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6834/3864487171 2024-09-22T03:40:31.989837+0000 2024-09-21T03:44:00.168 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6834/3572946698 2024-09-22T03:41:44.737273+0000 2024-09-21T03:44:00.168 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6832/1995903506 2024-09-22T03:43:00.406120+0000 2024-09-21T03:44:00.169 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6832/775682959 2024-09-22T03:43:10.570678+0000 2024-09-21T03:44:00.169 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6835/2916238332 2024-09-22T03:41:42.553770+0000 2024-09-21T03:44:00.169 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6833/3402963947 2024-09-22T03:41:23.555005+0000 2024-09-21T03:44:00.169 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6834/425672974 2024-09-22T03:40:46.105521+0000 2024-09-21T03:44:00.169 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6834/3951934534 2024-09-22T03:42:31.276099+0000 2024-09-21T03:44:00.169 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6835/425672974 2024-09-22T03:40:46.105521+0000 2024-09-21T03:44:00.169 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6832/3710736169 2024-09-22T03:40:59.454480+0000 2024-09-21T03:44:00.169 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6833/1717955365 2024-09-22T03:41:51.787804+0000 2024-09-21T03:44:00.169 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6835/3055025811 2024-09-22T03:42:39.361236+0000 2024-09-21T03:44:00.169 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6835/2751962827 2024-09-22T03:40:38.029580+0000 2024-09-21T03:44:00.169 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6835/2712360296 2024-09-22T03:42:20.931115+0000 2024-09-21T03:44:00.170 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6833/3537258034 2024-09-22T03:42:06.901189+0000 2024-09-21T03:44:00.170 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6832/3537258034 2024-09-22T03:42:06.901189+0000 2024-09-21T03:44:00.170 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6832/1272005650 2024-09-22T03:41:58.836862+0000 2024-09-21T03:44:00.170 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6834/2712360296 2024-09-22T03:42:20.931115+0000 2024-09-21T03:44:00.170 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6832/751618334 2024-09-22T03:40:42.053279+0000 2024-09-21T03:44:00.170 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6835/2930805359 2024-09-22T03:42:23.215190+0000 2024-09-21T03:44:00.170 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6832/4221150556 2024-09-22T03:42:27.240627+0000 2024-09-21T03:44:00.170 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6833/1272005650 2024-09-22T03:41:58.836862+0000 2024-09-21T03:44:00.170 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6832/245538755 2024-09-22T03:42:35.318756+0000 2024-09-21T03:44:00.170 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6833/245538755 2024-09-22T03:42:35.318756+0000 2024-09-21T03:44:00.170 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6833/2459386631 2024-09-22T03:42:43.395609+0000 2024-09-21T03:44:00.171 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6832/2459386631 2024-09-22T03:42:43.395609+0000 2024-09-21T03:44:00.171 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6834/2097789246 2024-09-22T03:43:06.604471+0000 2024-09-21T03:44:00.171 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6834/2798021708 2024-09-22T03:42:47.436632+0000 2024-09-21T03:44:00.171 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6832/111206731 2024-09-22T03:43:02.560071+0000 2024-09-21T03:44:00.171 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6833/111206731 2024-09-22T03:43:02.560071+0000 2024-09-21T03:44:00.172 INFO:teuthology.orchestra.run.smithi038.stderr:listed 68 entries 2024-09-21T03:44:00.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:00.165+0000 7f651c758640 1 -- 172.21.15.38:0/411653951 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f64f8078bc0 msgr2=0x7f64f807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:00.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:00.165+0000 7f651c758640 1 --2- 172.21.15.38:0/411653951 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f64f8078bc0 0x7f64f807b0d0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f6510001050 tx=0x7f6510006930 comp rx=0 tx=0).stop 2024-09-21T03:44:00.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:00.165+0000 7f651c758640 1 -- 172.21.15.38:0/411653951 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f65141f6580 msgr2=0x7f6514157910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:00.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:00.165+0000 7f651c758640 1 --2- 172.21.15.38:0/411653951 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f65141f6580 0x7f6514157910 secure :-1 s=READY pgs=2127 cs=0 l=1 rev1=1 crypto rx=0x7f6504003ec0 tx=0x7f650400f720 comp rx=0 tx=0).stop 2024-09-21T03:44:00.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:00.165+0000 7f651c758640 1 -- 172.21.15.38:0/411653951 shutdown_connections 2024-09-21T03:44:00.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:00.165+0000 7f651c758640 1 --2- 172.21.15.38:0/411653951 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f651406cde0 0x7f651406d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:00.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:00.165+0000 7f651c758640 1 --2- 172.21.15.38:0/411653951 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f65141f6580 0x7f6514157910 unknown :-1 s=CLOSED pgs=2127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:00.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:00.165+0000 7f651c758640 1 --2- 172.21.15.38:0/411653951 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f64f8078bc0 0x7f64f807b0d0 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:00.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:00.165+0000 7f651c758640 1 --2- 172.21.15.38:0/411653951 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6500004d40 0x7f65141f9c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:00.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:00.165+0000 7f651c758640 1 -- 172.21.15.38:0/411653951 >> 172.21.15.38:0/411653951 conn(0x7f651409d830 msgr2=0x7f65140b4880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:00.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:00.165+0000 7f651c758640 1 -- 172.21.15.38:0/411653951 shutdown_connections 2024-09-21T03:44:00.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:00.165+0000 7f651c758640 1 -- 172.21.15.38:0/411653951 wait complete. 2024-09-21T03:44:00.178 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-21T03:44:00.178 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:44:00.179 DEBUG:teuthology.orchestra.run.smithi038:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-09-21T03:44:00.212 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi038.front.sepia.ceph.com 2024-09-21T03:44:00.213 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:44:00.213 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T03:44:00.268 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-21T03:44:00.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:00.424+0000 7fcf9010b640 1 -- 172.21.15.38:0/157813425 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcf74004d40 msgr2=0x7fcf74005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:00.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:00.424+0000 7fcf9010b640 1 --2- 172.21.15.38:0/157813425 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcf74004d40 0x7fcf74005190 secure :-1 s=READY pgs=2129 cs=0 l=1 rev1=1 crypto rx=0x7fcf880b1e90 tx=0x7fcf74005830 comp rx=0 tx=0).stop 2024-09-21T03:44:00.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:00.424+0000 7fcf8e6a8640 1 -- 172.21.15.38:0/157813425 reap_dead start 2024-09-21T03:44:00.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:00.424+0000 7fcf9010b640 1 -- 172.21.15.38:0/157813425 shutdown_connections 2024-09-21T03:44:00.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:00.424+0000 7fcf9010b640 1 -- 172.21.15.38:0/157813425 >> 172.21.15.38:0/157813425 conn(0x7fcf880b46e0 msgr2=0x7fcf880d23e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:00.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:00.424+0000 7fcf9010b640 1 -- 172.21.15.38:0/157813425 shutdown_connections 2024-09-21T03:44:00.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:00.424+0000 7fcf9010b640 1 -- 172.21.15.38:0/157813425 wait complete. 2024-09-21T03:44:00.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:00.426+0000 7fcf9010b640 1 Processor -- start 2024-09-21T03:44:00.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:00.426+0000 7fcf9010b640 1 -- start start 2024-09-21T03:44:00.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:00.427+0000 7fcf9010b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcf74004d40 0x7fcf881f8ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:00.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:00.427+0000 7fcf9010b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcf881e8680 0x7fcf881f90e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:00.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:00.427+0000 7fcf9010b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcf881e91f0 0x7fcf880c9490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:00.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:00.427+0000 7fcf9010b640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fcf880b62f0 con 0x7fcf881e8680 2024-09-21T03:44:00.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:00.427+0000 7fcf9010b640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fcf880d09e0 con 0x7fcf74004d40 2024-09-21T03:44:00.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:00.427+0000 7fcf9010b640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fcf880b6470 con 0x7fcf881e91f0 2024-09-21T03:44:00.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:00.427+0000 7fcf8d6a6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcf881e8680 0x7fcf881f90e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:00.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:00.427+0000 7fcf8d6a6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcf881e8680 0x7fcf881f90e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50744/0 (socket says 172.21.15.38:50744) 2024-09-21T03:44:00.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:00.427+0000 7fcf8d6a6640 1 -- 172.21.15.38:0/2813621875 learned_addr learned my addr 172.21.15.38:0/2813621875 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:44:00.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:00.427+0000 7fcf8e6a8640 1 --2- 172.21.15.38:0/2813621875 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcf881e91f0 0x7fcf880c9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:00.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:00.427+0000 7fcf8dea7640 1 --2- 172.21.15.38:0/2813621875 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcf74004d40 0x7fcf881f8ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:00.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:00.428+0000 7fcf8d6a6640 1 -- 172.21.15.38:0/2813621875 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcf881e91f0 msgr2=0x7fcf880c9490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:00.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:00.428+0000 7fcf8d6a6640 1 --2- 172.21.15.38:0/2813621875 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcf881e91f0 0x7fcf880c9490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:00.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:00.428+0000 7fcf8d6a6640 1 -- 172.21.15.38:0/2813621875 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcf74004d40 msgr2=0x7fcf881f8ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:00.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:00.428+0000 7fcf8d6a6640 1 --2- 172.21.15.38:0/2813621875 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcf74004d40 0x7fcf881f8ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:00.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:00.428+0000 7fcf8d6a6640 1 -- 172.21.15.38:0/2813621875 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf881e8430 con 0x7fcf881e8680 2024-09-21T03:44:00.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:00.428+0000 7fcf8d6a6640 1 --2- 172.21.15.38:0/2813621875 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcf881e8680 0x7fcf881f90e0 secure :-1 s=READY pgs=2130 cs=0 l=1 rev1=1 crypto rx=0x7fcf78011a50 tx=0x7fcf78011e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:00.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:00.428+0000 7fcf7e7fc640 1 -- 172.21.15.38:0/2813621875 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf78004460 con 0x7fcf881e8680 2024-09-21T03:44:00.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:00.428+0000 7fcf7e7fc640 1 -- 172.21.15.38:0/2813621875 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcf78016070 con 0x7fcf881e8680 2024-09-21T03:44:00.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:00.428+0000 7fcf9010b640 1 -- 172.21.15.38:0/2813621875 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf881f9920 con 0x7fcf881e8680 2024-09-21T03:44:00.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:00.428+0000 7fcf7e7fc640 1 -- 172.21.15.38:0/2813621875 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf7800fb20 con 0x7fcf881e8680 2024-09-21T03:44:00.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:00.428+0000 7fcf9010b640 1 -- 172.21.15.38:0/2813621875 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf881e70b0 con 0x7fcf881e8680 2024-09-21T03:44:00.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:00.429+0000 7fcf9010b640 1 -- 172.21.15.38:0/2813621875 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf880cd3c0 con 0x7fcf881e8680 2024-09-21T03:44:00.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:00.430+0000 7fcf7e7fc640 1 -- 172.21.15.38:0/2813621875 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fcf78018030 con 0x7fcf881e8680 2024-09-21T03:44:00.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:00.431+0000 7fcf7e7fc640 1 --2- 172.21.15.38:0/2813621875 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fcf6c078b90 0x7fcf6c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:00.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:00.431+0000 7fcf7e7fc640 1 -- 172.21.15.38:0/2813621875 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 9228+0+0 (secure 0 0 0) 0x7fcf7809c310 con 0x7fcf881e8680 2024-09-21T03:44:00.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:00.431+0000 7fcf8dea7640 1 --2- 172.21.15.38:0/2813621875 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fcf6c078b90 0x7fcf6c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:00.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:00.432+0000 7fcf8dea7640 1 --2- 172.21.15.38:0/2813621875 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fcf6c078b90 0x7fcf6c07b0a0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fcf84001010 tx=0x7fcf840068d0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:00.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:00.435+0000 7fcf7e7fc640 1 -- 172.21.15.38:0/2813621875 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fcf881e8430 con 0x7fcf881e8680 2024-09-21T03:44:00.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:00.662+0000 7fcf9010b640 1 -- 172.21.15.38:0/2813621875 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fcf880b69b0 con 0x7fcf881e8680 2024-09-21T03:44:00.665 INFO:tasks.ceph.mon.a.smithi038.stderr:2024-09-21T03:44:00.663+0000 7f3f45713640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T03:44:00.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:00.665+0000 7fcf7e7fc640 1 -- 172.21.15.38:0/2813621875 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v298) ==== 161+0+0 (secure 0 0 0) 0x7fcf780638e0 con 0x7fcf881e8680 2024-09-21T03:44:00.667 INFO:teuthology.orchestra.run.smithi038.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-21T03:44:00.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:00.668+0000 7fcf9010b640 1 -- 172.21.15.38:0/2813621875 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fcf6c078b90 msgr2=0x7fcf6c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:00.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:00.668+0000 7fcf9010b640 1 --2- 172.21.15.38:0/2813621875 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fcf6c078b90 0x7fcf6c07b0a0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fcf84001010 tx=0x7fcf840068d0 comp rx=0 tx=0).stop 2024-09-21T03:44:00.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:00.668+0000 7fcf9010b640 1 -- 172.21.15.38:0/2813621875 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcf881e8680 msgr2=0x7fcf881f90e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:00.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:00.668+0000 7fcf9010b640 1 --2- 172.21.15.38:0/2813621875 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcf881e8680 0x7fcf881f90e0 secure :-1 s=READY pgs=2130 cs=0 l=1 rev1=1 crypto rx=0x7fcf78011a50 tx=0x7fcf78011e50 comp rx=0 tx=0).stop 2024-09-21T03:44:00.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:00.669+0000 7fcf9010b640 1 -- 172.21.15.38:0/2813621875 shutdown_connections 2024-09-21T03:44:00.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:00.669+0000 7fcf9010b640 1 --2- 172.21.15.38:0/2813621875 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fcf6c078b90 0x7fcf6c07b0a0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:00.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:00.669+0000 7fcf9010b640 1 --2- 172.21.15.38:0/2813621875 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcf881e91f0 0x7fcf880c9490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:00.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:00.669+0000 7fcf9010b640 1 --2- 172.21.15.38:0/2813621875 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcf881e8680 0x7fcf881f90e0 unknown :-1 s=CLOSED pgs=2130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:00.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:00.669+0000 7fcf9010b640 1 --2- 172.21.15.38:0/2813621875 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcf74004d40 0x7fcf881f8ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:00.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:00.669+0000 7fcf9010b640 1 -- 172.21.15.38:0/2813621875 >> 172.21.15.38:0/2813621875 conn(0x7fcf880b46e0 msgr2=0x7fcf8815ba30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:00.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:00.670+0000 7fcf9010b640 1 -- 172.21.15.38:0/2813621875 shutdown_connections 2024-09-21T03:44:00.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:00.670+0000 7fcf9010b640 1 -- 172.21.15.38:0/2813621875 wait complete. 2024-09-21T03:44:00.673 INFO:tasks.ceph.mds.c.smithi038.stderr: -673> 2024-09-21T03:43:55.192+0000 7f1b94a94640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T03:44:00.673 INFO:tasks.ceph.mds.c.smithi038.stderr: -138> 2024-09-21T03:43:59.682+0000 7f1b90a8c640 -1 mds.0.cache.dir(0x10000000000) Corrupt fnode in dirfrag 0x10000000000: Decoder at 'void fnode_t::decode(ceph::buffer::v15_2_0::list::const_iterator&)' v=3 cannot decode v=100 minimal_decoder=101: Malformed input [buffer:3] 2024-09-21T03:44:00.684 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-09-21T03:44:00.730 INFO:tasks.ceph.mds.c.smithi038.stdout:starting mds.c at 2024-09-21T03:44:00.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:00.802+0000 7f7d5a030640 1 -- 172.21.15.38:0/3588474618 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7d540cd440 msgr2=0x7f7d540d0670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:00.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:00.802+0000 7f7d5a030640 1 --2- 172.21.15.38:0/3588474618 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7d540cd440 0x7f7d540d0670 secure :-1 s=READY pgs=2135 cs=0 l=1 rev1=1 crypto rx=0x7f7d48009830 tx=0x7f7d4802a480 comp rx=0 tx=0).stop 2024-09-21T03:44:00.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:00.802+0000 7f7d5a030640 1 -- 172.21.15.38:0/3588474618 shutdown_connections 2024-09-21T03:44:00.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:00.802+0000 7f7d5a030640 1 --2- 172.21.15.38:0/3588474618 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7d540cd440 0x7f7d540d0670 unknown :-1 s=CLOSED pgs=2135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:00.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:00.802+0000 7f7d5a030640 1 --2- 172.21.15.38:0/3588474618 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7d540ccc40 0x7f7d5415ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:00.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:00.802+0000 7f7d5a030640 1 -- 172.21.15.38:0/3588474618 >> 172.21.15.38:0/3588474618 conn(0x7f7d540b46e0 msgr2=0x7f7d540d23e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:00.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:00.802+0000 7f7d5a030640 1 -- 172.21.15.38:0/3588474618 shutdown_connections 2024-09-21T03:44:00.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:00.802+0000 7f7d5a030640 1 -- 172.21.15.38:0/3588474618 wait complete. 2024-09-21T03:44:00.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:00.804+0000 7f7d5a030640 1 Processor -- start 2024-09-21T03:44:00.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:00.804+0000 7f7d5a030640 1 -- start start 2024-09-21T03:44:00.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:00.806+0000 7f7d5a030640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7d540ccc40 0x7f7d54151a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:00.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:00.806+0000 7f7d5a030640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7d540cd440 0x7f7d540cab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:00.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:00.806+0000 7f7d5a030640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7d540d0bb0 0x7f7d5406f990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:00.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:00.806+0000 7f7d5a030640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f7d54070810 con 0x7f7d540cd440 2024-09-21T03:44:00.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:00.806+0000 7f7d5a030640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f7d540b6e00 con 0x7f7d540d0bb0 2024-09-21T03:44:00.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:00.806+0000 7f7d5a030640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f7d54070990 con 0x7f7d540ccc40 2024-09-21T03:44:00.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:00.806+0000 7f7d52ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7d540cd440 0x7f7d540cab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:00.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:00.806+0000 7f7d52ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7d540cd440 0x7f7d540cab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50782/0 (socket says 172.21.15.38:50782) 2024-09-21T03:44:00.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:00.806+0000 7f7d52ffd640 1 -- 172.21.15.38:0/1884945630 learned_addr learned my addr 172.21.15.38:0/1884945630 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:44:00.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:00.806+0000 7f7d53fff640 1 --2- 172.21.15.38:0/1884945630 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7d540d0bb0 0x7f7d5406f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:00.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:00.806+0000 7f7d537fe640 1 --2- 172.21.15.38:0/1884945630 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7d540ccc40 0x7f7d54151a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:00.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:00.806+0000 7f7d52ffd640 1 -- 172.21.15.38:0/1884945630 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7d540ccc40 msgr2=0x7f7d54151a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:00.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:00.806+0000 7f7d52ffd640 1 --2- 172.21.15.38:0/1884945630 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7d540ccc40 0x7f7d54151a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:00.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:00.806+0000 7f7d52ffd640 1 -- 172.21.15.38:0/1884945630 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7d540d0bb0 msgr2=0x7f7d5406f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:00.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:00.806+0000 7f7d52ffd640 1 --2- 172.21.15.38:0/1884945630 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7d540d0bb0 0x7f7d5406f990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:00.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:00.806+0000 7f7d52ffd640 1 -- 172.21.15.38:0/1884945630 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d5406fed0 con 0x7f7d540cd440 2024-09-21T03:44:00.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:00.806+0000 7f7d537fe640 1 --2- 172.21.15.38:0/1884945630 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7d540ccc40 0x7f7d54151a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:44:00.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:00.806+0000 7f7d52ffd640 1 --2- 172.21.15.38:0/1884945630 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7d540cd440 0x7f7d540cab10 secure :-1 s=READY pgs=2136 cs=0 l=1 rev1=1 crypto rx=0x7f7d48009950 tx=0x7f7d48033c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:00.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:00.806+0000 7f7d3bfff640 1 -- 172.21.15.38:0/1884945630 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d48005c90 con 0x7f7d540cd440 2024-09-21T03:44:00.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:00.807+0000 7f7d3bfff640 1 -- 172.21.15.38:0/1884945630 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7d48031e50 con 0x7f7d540cd440 2024-09-21T03:44:00.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:00.807+0000 7f7d5a030640 1 -- 172.21.15.38:0/1884945630 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d540c9a70 con 0x7f7d540cd440 2024-09-21T03:44:00.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:00.807+0000 7f7d3bfff640 1 -- 172.21.15.38:0/1884945630 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d48033290 con 0x7f7d540cd440 2024-09-21T03:44:00.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:00.807+0000 7f7d5a030640 1 -- 172.21.15.38:0/1884945630 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d540ca030 con 0x7f7d540cd440 2024-09-21T03:44:00.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:00.808+0000 7f7d5a030640 1 -- 172.21.15.38:0/1884945630 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d54070ec0 con 0x7f7d540cd440 2024-09-21T03:44:00.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:00.808+0000 7f7d3bfff640 1 -- 172.21.15.38:0/1884945630 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f7d48033430 con 0x7f7d540cd440 2024-09-21T03:44:00.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:00.809+0000 7f7d3bfff640 1 --2- 172.21.15.38:0/1884945630 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7d40078b90 0x7f7d4007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:00.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:00.809+0000 7f7d3bfff640 1 -- 172.21.15.38:0/1884945630 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 9314+0+0 (secure 0 0 0) 0x7f7d480bd070 con 0x7f7d540cd440 2024-09-21T03:44:00.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:00.809+0000 7f7d537fe640 1 --2- 172.21.15.38:0/1884945630 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7d40078b90 0x7f7d4007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:00.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:00.810+0000 7f7d537fe640 1 --2- 172.21.15.38:0/1884945630 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7d40078b90 0x7f7d4007b0a0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f7d44005fd0 tx=0x7f7d4400f360 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:00.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:00.814+0000 7f7d3bfff640 1 -- 172.21.15.38:0/1884945630 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f7d4803f030 con 0x7f7d540cd440 2024-09-21T03:44:01.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.074+0000 7f7d5a030640 1 -- 172.21.15.38:0/1884945630 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7f7d540d0610 con 0x7f7d540cd440 2024-09-21T03:44:01.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.668+0000 7f7d3bfff640 1 -- 172.21.15.38:0/1884945630 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 nothing to do: rank is not damaged v299) ==== 107+0+0 (secure 0 0 0) 0x7f7d48084650 con 0x7f7d540cd440 2024-09-21T03:44:01.671 INFO:teuthology.orchestra.run.smithi038.stderr:nothing to do: rank is not damaged 2024-09-21T03:44:01.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.672+0000 7f7d5a030640 1 -- 172.21.15.38:0/1884945630 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7d40078b90 msgr2=0x7f7d4007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:01.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.672+0000 7f7d5a030640 1 --2- 172.21.15.38:0/1884945630 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7d40078b90 0x7f7d4007b0a0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f7d44005fd0 tx=0x7f7d4400f360 comp rx=0 tx=0).stop 2024-09-21T03:44:01.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.672+0000 7f7d5a030640 1 -- 172.21.15.38:0/1884945630 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7d540cd440 msgr2=0x7f7d540cab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:01.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.672+0000 7f7d5a030640 1 --2- 172.21.15.38:0/1884945630 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7d540cd440 0x7f7d540cab10 secure :-1 s=READY pgs=2136 cs=0 l=1 rev1=1 crypto rx=0x7f7d48009950 tx=0x7f7d48033c90 comp rx=0 tx=0).stop 2024-09-21T03:44:01.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.672+0000 7f7d5a030640 1 -- 172.21.15.38:0/1884945630 shutdown_connections 2024-09-21T03:44:01.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.672+0000 7f7d5a030640 1 --2- 172.21.15.38:0/1884945630 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7d540d0bb0 0x7f7d5406f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:01.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.672+0000 7f7d5a030640 1 --2- 172.21.15.38:0/1884945630 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7d540cd440 0x7f7d540cab10 unknown :-1 s=CLOSED pgs=2136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:01.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.672+0000 7f7d5a030640 1 --2- 172.21.15.38:0/1884945630 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7d40078b90 0x7f7d4007b0a0 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:01.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.672+0000 7f7d5a030640 1 --2- 172.21.15.38:0/1884945630 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7d540ccc40 0x7f7d54151a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:01.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.672+0000 7f7d5a030640 1 -- 172.21.15.38:0/1884945630 >> 172.21.15.38:0/1884945630 conn(0x7f7d540b46e0 msgr2=0x7f7d540d23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:01.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.672+0000 7f7d5a030640 1 -- 172.21.15.38:0/1884945630 shutdown_connections 2024-09-21T03:44:01.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.672+0000 7f7d5a030640 1 -- 172.21.15.38:0/1884945630 wait complete. 2024-09-21T03:44:01.687 DEBUG:teuthology.orchestra.run.smithi038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-09-21T03:44:01.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.744+0000 7fcc12835f40 1 Processor -- start 2024-09-21T03:44:01.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.744+0000 7fcc12835f40 1 -- start start 2024-09-21T03:44:01.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.745+0000 7fcc12835f40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55d874c18090 0x55d874c16950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:01.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.745+0000 7fcc12835f40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d874c8d520 0x55d874d31780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:01.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.745+0000 7fcc12835f40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d874c858b0 0x55d874d31cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:01.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.745+0000 7fcc12835f40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55d874b1ed60 con 0x55d874c8d520 2024-09-21T03:44:01.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.745+0000 7fcc12835f40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55d874b7d430 con 0x55d874c858b0 2024-09-21T03:44:01.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.745+0000 7fcc12835f40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55d874ca72c0 con 0x55d874c18090 2024-09-21T03:44:01.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.745+0000 7fcc03fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d874c8d520 0x55d874d31780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:01.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.745+0000 7fcc03fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d874c8d520 0x55d874d31780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50798/0 (socket says 172.21.15.38:50798) 2024-09-21T03:44:01.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.745+0000 7fcc03fff640 1 -- 172.21.15.38:0/4131291260 learned_addr learned my addr 172.21.15.38:0/4131291260 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:44:01.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.745+0000 7fcc0caf2640 1 --2- 172.21.15.38:0/4131291260 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d874c858b0 0x55d874d31cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:01.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.745+0000 7fcc0c2f1640 1 --2- 172.21.15.38:0/4131291260 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55d874c18090 0x55d874c16950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:01.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.745+0000 7fcc03fff640 1 -- 172.21.15.38:0/4131291260 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55d874c18090 msgr2=0x55d874c16950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:44:01.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.745+0000 7fcc03fff640 1 --2- 172.21.15.38:0/4131291260 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55d874c18090 0x55d874c16950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:01.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.745+0000 7fcc03fff640 1 -- 172.21.15.38:0/4131291260 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d874c858b0 msgr2=0x55d874d31cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:44:01.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.745+0000 7fcc03fff640 1 --2- 172.21.15.38:0/4131291260 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d874c858b0 0x55d874d31cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:01.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.746+0000 7fcc03fff640 1 -- 172.21.15.38:0/4131291260 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d874bfb2f0 con 0x55d874c8d520 2024-09-21T03:44:01.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.746+0000 7fcc0caf2640 1 --2- 172.21.15.38:0/4131291260 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d874c858b0 0x55d874d31cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:44:01.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.746+0000 7fcc03fff640 1 --2- 172.21.15.38:0/4131291260 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d874c8d520 0x55d874d31780 secure :-1 s=READY pgs=2138 cs=0 l=1 rev1=1 crypto rx=0x7fcbf400a430 tx=0x7fcbf400a8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e9f536441bb9e26 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:01.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.746+0000 7fcc0b2ef640 1 -- 172.21.15.38:0/4131291260 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbf400b920 con 0x55d874c8d520 2024-09-21T03:44:01.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.746+0000 7fcc0b2ef640 1 -- 172.21.15.38:0/4131291260 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcbf400f030 con 0x55d874c8d520 2024-09-21T03:44:01.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.746+0000 7fcc0b2ef640 1 -- 172.21.15.38:0/4131291260 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbf4014420 con 0x55d874c8d520 2024-09-21T03:44:01.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.747+0000 7fcc12835f40 1 -- 172.21.15.38:0/4131291260 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d874c8d520 msgr2=0x55d874d31780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:01.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.747+0000 7fcc12835f40 1 --2- 172.21.15.38:0/4131291260 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d874c8d520 0x55d874d31780 secure :-1 s=READY pgs=2138 cs=0 l=1 rev1=1 crypto rx=0x7fcbf400a430 tx=0x7fcbf400a8f0 comp rx=0 tx=0).stop 2024-09-21T03:44:01.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.747+0000 7fcc12835f40 1 -- 172.21.15.38:0/4131291260 shutdown_connections 2024-09-21T03:44:01.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.747+0000 7fcc12835f40 1 --2- 172.21.15.38:0/4131291260 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d874c858b0 0x55d874d31cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:01.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.747+0000 7fcc12835f40 1 --2- 172.21.15.38:0/4131291260 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d874c8d520 0x55d874d31780 unknown :-1 s=CLOSED pgs=2138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:01.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.747+0000 7fcc12835f40 1 --2- 172.21.15.38:0/4131291260 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55d874c18090 0x55d874c16950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:01.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.747+0000 7fcc12835f40 1 -- 172.21.15.38:0/4131291260 >> 172.21.15.38:0/4131291260 conn(0x55d874aefac0 msgr2=0x55d874bd11d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:01.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.747+0000 7fcc12835f40 1 -- 172.21.15.38:0/4131291260 shutdown_connections 2024-09-21T03:44:01.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.747+0000 7fcc12835f40 1 -- 172.21.15.38:0/4131291260 wait complete. 2024-09-21T03:44:01.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.748+0000 7fcc12835f40 1 Processor -- start 2024-09-21T03:44:01.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.748+0000 7fcc12835f40 1 -- start start 2024-09-21T03:44:01.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.749+0000 7fcc12835f40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d874c18090 0x55d874d2ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:01.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.749+0000 7fcc12835f40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d874c858b0 0x55d874d31180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:01.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.749+0000 7fcc12835f40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55d874c8d520 0x55d874ca5f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:01.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.749+0000 7fcc12835f40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55d874ca3a90 con 0x55d874c858b0 2024-09-21T03:44:01.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.749+0000 7fcc12835f40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55d874ca57e0 con 0x55d874c18090 2024-09-21T03:44:01.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.749+0000 7fcc12835f40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55d874ca7000 con 0x55d874c8d520 2024-09-21T03:44:01.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.749+0000 7fcc03fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d874c858b0 0x55d874d31180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:01.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.749+0000 7fcc03fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d874c858b0 0x55d874d31180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50806/0 (socket says 172.21.15.38:50806) 2024-09-21T03:44:01.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.749+0000 7fcc03fff640 1 -- 172.21.15.38:0/1130780980 learned_addr learned my addr 172.21.15.38:0/1130780980 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:44:01.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.749+0000 7fcc0caf2640 1 --2- 172.21.15.38:0/1130780980 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55d874c8d520 0x55d874ca5f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:01.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.749+0000 7fcc0c2f1640 1 --2- 172.21.15.38:0/1130780980 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d874c18090 0x55d874d2ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:01.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.749+0000 7fcc03fff640 1 -- 172.21.15.38:0/1130780980 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55d874c8d520 msgr2=0x55d874ca5f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:01.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.749+0000 7fcc03fff640 1 --2- 172.21.15.38:0/1130780980 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55d874c8d520 0x55d874ca5f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:01.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.749+0000 7fcc03fff640 1 -- 172.21.15.38:0/1130780980 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d874c18090 msgr2=0x55d874d2ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:01.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.749+0000 7fcc03fff640 1 --2- 172.21.15.38:0/1130780980 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d874c18090 0x55d874d2ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:01.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.749+0000 7fcc03fff640 1 -- 172.21.15.38:0/1130780980 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d874c169f0 con 0x55d874c858b0 2024-09-21T03:44:01.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.749+0000 7fcc0c2f1640 1 --2- 172.21.15.38:0/1130780980 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d874c18090 0x55d874d2ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:44:01.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.749+0000 7fcc0caf2640 1 --2- 172.21.15.38:0/1130780980 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55d874c8d520 0x55d874ca5f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:44:01.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.749+0000 7fcc03fff640 1 --2- 172.21.15.38:0/1130780980 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d874c858b0 0x55d874d31180 secure :-1 s=READY pgs=2139 cs=0 l=1 rev1=1 crypto rx=0x7fcbf400f000 tx=0x7fcbf400b9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:01.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.750+0000 7fcc0a2ed640 1 -- 172.21.15.38:0/1130780980 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbf401c620 con 0x55d874c858b0 2024-09-21T03:44:01.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.750+0000 7fcc0a2ed640 1 -- 172.21.15.38:0/1130780980 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcbf4021070 con 0x55d874c858b0 2024-09-21T03:44:01.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.750+0000 7fcc12835f40 1 -- 172.21.15.38:0/1130780980 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55d874c16cd0 con 0x55d874c858b0 2024-09-21T03:44:01.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.750+0000 7fcc0a2ed640 1 -- 172.21.15.38:0/1130780980 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbf401cb50 con 0x55d874c858b0 2024-09-21T03:44:01.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.750+0000 7fcc12835f40 1 -- 172.21.15.38:0/1130780980 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d874d2c420 con 0x55d874c858b0 2024-09-21T03:44:01.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.751+0000 7fcc0a2ed640 1 -- 172.21.15.38:0/1130780980 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fcbf401ccf0 con 0x55d874c858b0 2024-09-21T03:44:01.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.752+0000 7fcc0a2ed640 1 --2- 172.21.15.38:0/1130780980 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fcbec077570 0x7fcbec079a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:01.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.752+0000 7fcc0a2ed640 1 -- 172.21.15.38:0/1130780980 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 9314+0+0 (secure 0 0 0) 0x7fcbf409bfc0 con 0x55d874c858b0 2024-09-21T03:44:01.756 INFO:teuthology.orchestra.run.smithi038.stdout:Importing pool 2024-09-21T03:44:01.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.752+0000 7fcc12835f40 1 --2- 172.21.15.38:0/1130780980 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x55d874d322a0 0x55d874d2fa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:01.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.752+0000 7fcc0c2f1640 1 --2- 172.21.15.38:0/1130780980 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fcbec077570 0x7fcbec079a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:01.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.752+0000 7fcc12835f40 1 -- 172.21.15.38:0/1130780980 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55d874d2ff80 con 0x55d874d322a0 2024-09-21T03:44:01.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.753+0000 7fcc0caf2640 1 --2- 172.21.15.38:0/1130780980 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x55d874d322a0 0x55d874d2fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:01.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.753+0000 7fcc0c2f1640 1 --2- 172.21.15.38:0/1130780980 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fcbec077570 0x7fcbec079a80 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fcbfc055000 tx=0x7fcbfc058040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:01.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.753+0000 7fcc0caf2640 1 --2- 172.21.15.38:0/1130780980 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x55d874d322a0 0x55d874d2fa40 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:01.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.756+0000 7fcc0caf2640 1 -- 172.21.15.38:0/1130780980 <== osd.7 v2:172.21.15.195:6818/1606106342 1 ==== osd_op_reply(1 601.00000000 [create] v131'1065 uv1038 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fcc0400b040 con 0x55d874d322a0 2024-09-21T03:44:01.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.756+0000 7fcc12835f40 1 -- 172.21.15.38:0/1130780980 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e131) -- 0x55d874d49f80 con 0x55d874d322a0 2024-09-21T03:44:01.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.758+0000 7fcc0caf2640 1 -- 172.21.15.38:0/1130780980 <== osd.7 v2:172.21.15.195:6818/1606106342 2 ==== osd_op_reply(2 601.00000000 [delete] v131'1066 uv1066 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcc0400b040 con 0x55d874d322a0 2024-09-21T03:44:01.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.758+0000 7fcc12835f40 1 -- 172.21.15.38:0/1130780980 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55d874d4a320 con 0x55d874d322a0 2024-09-21T03:44:01.761 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-09-21T03:44:01.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.760+0000 7fcc0caf2640 1 -- 172.21.15.38:0/1130780980 <== osd.7 v2:172.21.15.195:6818/1606106342 3 ==== osd_op_reply(3 601.00000000 [create] v131'1067 uv1067 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcc0400b040 con 0x55d874d322a0 2024-09-21T03:44:01.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.760+0000 7fcc12835f40 1 -- 172.21.15.38:0/1130780980 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55d874d4b6d0 con 0x55d874d322a0 2024-09-21T03:44:01.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.761+0000 7fcc0caf2640 1 -- 172.21.15.38:0/1130780980 <== osd.7 v2:172.21.15.195:6818/1606106342 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v131'1068 uv1068 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcc0400b040 con 0x55d874d322a0 2024-09-21T03:44:01.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.761+0000 7fcc12835f40 1 -- 172.21.15.38:0/1130780980 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55d874d4ba70 con 0x55d874d322a0 2024-09-21T03:44:01.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.763+0000 7fcc0caf2640 1 -- 172.21.15.38:0/1130780980 <== osd.7 v2:172.21.15.195:6818/1606106342 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v131'1069 uv1069 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcc0400b040 con 0x55d874d322a0 2024-09-21T03:44:01.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.763+0000 7fcc12835f40 1 -- 172.21.15.38:0/1130780980 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55d874d4a6c0 con 0x55d874d322a0 2024-09-21T03:44:01.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.765+0000 7fcc0caf2640 1 -- 172.21.15.38:0/1130780980 <== osd.7 v2:172.21.15.195:6818/1606106342 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v131'1070 uv1070 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcc0400b040 con 0x55d874d322a0 2024-09-21T03:44:01.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.765+0000 7fcc12835f40 1 -- 172.21.15.38:0/1130780980 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55d874d4aab0 con 0x55d874d322a0 2024-09-21T03:44:01.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.767+0000 7fcc0caf2640 1 -- 172.21.15.38:0/1130780980 <== osd.7 v2:172.21.15.195:6818/1606106342 7 ==== osd_op_reply(7 602.00000000 [create] v131'1071 uv1044 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fcc0400b040 con 0x55d874d322a0 2024-09-21T03:44:01.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.767+0000 7fcc12835f40 1 -- 172.21.15.38:0/1130780980 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e131) -- 0x55d874d4aab0 con 0x55d874d322a0 2024-09-21T03:44:01.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.769+0000 7fcc0caf2640 1 -- 172.21.15.38:0/1130780980 <== osd.7 v2:172.21.15.195:6818/1606106342 8 ==== osd_op_reply(8 602.00000000 [delete] v131'1072 uv1072 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcc0400b040 con 0x55d874d322a0 2024-09-21T03:44:01.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.769+0000 7fcc12835f40 1 -- 172.21.15.38:0/1130780980 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55d874d4aab0 con 0x55d874d322a0 2024-09-21T03:44:01.772 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-09-21T03:44:01.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.770+0000 7fcc0caf2640 1 -- 172.21.15.38:0/1130780980 <== osd.7 v2:172.21.15.195:6818/1606106342 9 ==== osd_op_reply(9 602.00000000 [create] v131'1073 uv1073 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcc0400b040 con 0x55d874d322a0 2024-09-21T03:44:01.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.770+0000 7fcc12835f40 1 -- 172.21.15.38:0/1130780980 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55d874d4aab0 con 0x55d874d322a0 2024-09-21T03:44:01.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.772+0000 7fcc0caf2640 1 -- 172.21.15.38:0/1130780980 <== osd.7 v2:172.21.15.195:6818/1606106342 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v131'1074 uv1074 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcc0400b040 con 0x55d874d322a0 2024-09-21T03:44:01.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.772+0000 7fcc12835f40 1 -- 172.21.15.38:0/1130780980 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55d874d4aab0 con 0x55d874d322a0 2024-09-21T03:44:01.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.773+0000 7fcc0caf2640 1 -- 172.21.15.38:0/1130780980 <== osd.7 v2:172.21.15.195:6818/1606106342 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v131'1075 uv1075 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcc0400b040 con 0x55d874d322a0 2024-09-21T03:44:01.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.774+0000 7fcc12835f40 1 -- 172.21.15.38:0/1130780980 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55d874d4aab0 con 0x55d874d322a0 2024-09-21T03:44:01.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.775+0000 7fcc0caf2640 1 -- 172.21.15.38:0/1130780980 <== osd.7 v2:172.21.15.195:6818/1606106342 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v131'1076 uv1076 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcc0400b040 con 0x55d874d322a0 2024-09-21T03:44:01.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.775+0000 7fcc12835f40 1 -- 172.21.15.38:0/1130780980 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:13 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55d874d4aab0 con 0x55d874d322a0 2024-09-21T03:44:01.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.777+0000 7fcc0caf2640 1 -- 172.21.15.38:0/1130780980 <== osd.7 v2:172.21.15.195:6818/1606106342 13 ==== osd_op_reply(13 10000000000.00000000 [create] v131'1077 uv1064 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7fcc0400b040 con 0x55d874d322a0 2024-09-21T03:44:01.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.777+0000 7fcc12835f40 1 -- 172.21.15.38:0/1130780980 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:14 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e131) -- 0x55d874d4aab0 con 0x55d874d322a0 2024-09-21T03:44:01.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.779+0000 7fcc0caf2640 1 -- 172.21.15.38:0/1130780980 <== osd.7 v2:172.21.15.195:6818/1606106342 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v131'1078 uv1078 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fcc0400b040 con 0x55d874d322a0 2024-09-21T03:44:01.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.779+0000 7fcc12835f40 1 -- 172.21.15.38:0/1130780980 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:15 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55d874d4aab0 con 0x55d874d322a0 2024-09-21T03:44:01.782 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-09-21T03:44:01.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.780+0000 7fcc0caf2640 1 -- 172.21.15.38:0/1130780980 <== osd.7 v2:172.21.15.195:6818/1606106342 15 ==== osd_op_reply(15 10000000000.00000000 [create] v131'1079 uv1079 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fcc0400b040 con 0x55d874d322a0 2024-09-21T03:44:01.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.780+0000 7fcc12835f40 1 -- 172.21.15.38:0/1130780980 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:16 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55d874d4aab0 con 0x55d874d322a0 2024-09-21T03:44:01.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.782+0000 7fcc0caf2640 1 -- 172.21.15.38:0/1130780980 <== osd.7 v2:172.21.15.195:6818/1606106342 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v131'1080 uv1080 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fcc0400b040 con 0x55d874d322a0 2024-09-21T03:44:01.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.782+0000 7fcc12835f40 1 -- 172.21.15.38:0/1130780980 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:17 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55d874d4aab0 con 0x55d874d322a0 2024-09-21T03:44:01.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.783+0000 7fcc0caf2640 1 -- 172.21.15.38:0/1130780980 <== osd.7 v2:172.21.15.195:6818/1606106342 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v131'1081 uv1081 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fcc0400b040 con 0x55d874d322a0 2024-09-21T03:44:01.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.783+0000 7fcc12835f40 1 -- 172.21.15.38:0/1130780980 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:18 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55d874d4aab0 con 0x55d874d322a0 2024-09-21T03:44:01.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.785+0000 7fcc0caf2640 1 -- 172.21.15.38:0/1130780980 <== osd.7 v2:172.21.15.195:6818/1606106342 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v131'1082 uv1082 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fcc0400b040 con 0x55d874d322a0 2024-09-21T03:44:01.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.785+0000 7fcc12835f40 1 -- 172.21.15.38:0/1130780980 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:19 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=585b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55d874ca6750 con 0x55d874d322a0 2024-09-21T03:44:01.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.787+0000 7fcc0caf2640 1 -- 172.21.15.38:0/1130780980 <== osd.7 v2:172.21.15.195:6818/1606106342 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v131'1083 uv1083 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fcc0400b040 con 0x55d874d322a0 2024-09-21T03:44:01.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.787+0000 7fcc12835f40 1 -- 172.21.15.38:0/1130780980 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:20 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55d874ca6750 con 0x55d874d322a0 2024-09-21T03:44:01.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.788+0000 7fcc0caf2640 1 -- 172.21.15.38:0/1130780980 <== osd.7 v2:172.21.15.195:6818/1606106342 20 ==== osd_op_reply(20 600.00000000 [create] v131'1084 uv1057 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fcc0400b040 con 0x55d874d322a0 2024-09-21T03:44:01.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.788+0000 7fcc12835f40 1 -- 172.21.15.38:0/1130780980 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:21 13.10 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e131) -- 0x55d874ca6750 con 0x55d874d322a0 2024-09-21T03:44:01.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.790+0000 7fcc0caf2640 1 -- 172.21.15.38:0/1130780980 <== osd.7 v2:172.21.15.195:6818/1606106342 21 ==== osd_op_reply(21 600.00000000 [delete] v131'1085 uv1085 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcc0400b040 con 0x55d874d322a0 2024-09-21T03:44:01.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.790+0000 7fcc12835f40 1 -- 172.21.15.38:0/1130780980 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:22 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55d874ca6750 con 0x55d874d322a0 2024-09-21T03:44:01.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.791+0000 7fcc0caf2640 1 -- 172.21.15.38:0/1130780980 <== osd.7 v2:172.21.15.195:6818/1606106342 22 ==== osd_op_reply(22 600.00000000 [create] v131'1086 uv1086 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcc0400b040 con 0x55d874d322a0 2024-09-21T03:44:01.793 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-09-21T03:44:01.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.791+0000 7fcc12835f40 1 -- 172.21.15.38:0/1130780980 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:23 13.10 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55d874ca6750 con 0x55d874d322a0 2024-09-21T03:44:01.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.792+0000 7fcc0caf2640 1 -- 172.21.15.38:0/1130780980 <== osd.7 v2:172.21.15.195:6818/1606106342 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v131'1087 uv1087 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcc0400b040 con 0x55d874d322a0 2024-09-21T03:44:01.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.792+0000 7fcc12835f40 1 -- 172.21.15.38:0/1130780980 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:24 13.10 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55d874ca6750 con 0x55d874d322a0 2024-09-21T03:44:01.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.794+0000 7fcc0caf2640 1 -- 172.21.15.38:0/1130780980 <== osd.7 v2:172.21.15.195:6818/1606106342 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v131'1088 uv1088 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcc0400b040 con 0x55d874d322a0 2024-09-21T03:44:01.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.794+0000 7fcc12835f40 1 -- 172.21.15.38:0/1130780980 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:25 13.10 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55d874ca6750 con 0x55d874d322a0 2024-09-21T03:44:01.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.795+0000 7fcc0caf2640 1 -- 172.21.15.38:0/1130780980 <== osd.7 v2:172.21.15.195:6818/1606106342 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v131'1089 uv1089 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcc0400b040 con 0x55d874d322a0 2024-09-21T03:44:01.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.795+0000 7fcc12835f40 1 -- 172.21.15.38:0/1130780980 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:26 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55d874ca6750 con 0x55d874d322a0 2024-09-21T03:44:01.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.797+0000 7fcc0caf2640 1 -- 172.21.15.38:0/1130780980 <== osd.7 v2:172.21.15.195:6818/1606106342 26 ==== osd_op_reply(26 603.00000000 [create] v131'1090 uv1063 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fcc0400b040 con 0x55d874d322a0 2024-09-21T03:44:01.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.797+0000 7fcc12835f40 1 -- 172.21.15.38:0/1130780980 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:27 13.10 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e131) -- 0x55d874ca6750 con 0x55d874d322a0 2024-09-21T03:44:01.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.798+0000 7fcc0caf2640 1 -- 172.21.15.38:0/1130780980 <== osd.7 v2:172.21.15.195:6818/1606106342 27 ==== osd_op_reply(27 603.00000000 [delete] v131'1091 uv1091 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcc0400b040 con 0x55d874d322a0 2024-09-21T03:44:01.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.798+0000 7fcc12835f40 1 -- 172.21.15.38:0/1130780980 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:28 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55d874ca6750 con 0x55d874d322a0 2024-09-21T03:44:01.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.800+0000 7fcc0caf2640 1 -- 172.21.15.38:0/1130780980 <== osd.7 v2:172.21.15.195:6818/1606106342 28 ==== osd_op_reply(28 603.00000000 [create] v131'1092 uv1092 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcc0400b040 con 0x55d874d322a0 2024-09-21T03:44:01.802 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-09-21T03:44:01.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.800+0000 7fcc12835f40 1 -- 172.21.15.38:0/1130780980 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:29 13.10 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55d874ca6750 con 0x55d874d322a0 2024-09-21T03:44:01.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.801+0000 7fcc0caf2640 1 -- 172.21.15.38:0/1130780980 <== osd.7 v2:172.21.15.195:6818/1606106342 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v131'1093 uv1093 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcc0400b040 con 0x55d874d322a0 2024-09-21T03:44:01.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.801+0000 7fcc12835f40 1 -- 172.21.15.38:0/1130780980 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:30 13.10 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55d874ca6750 con 0x55d874d322a0 2024-09-21T03:44:01.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.803+0000 7fcc0caf2640 1 -- 172.21.15.38:0/1130780980 <== osd.7 v2:172.21.15.195:6818/1606106342 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v131'1094 uv1094 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcc0400b040 con 0x55d874d322a0 2024-09-21T03:44:01.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.803+0000 7fcc12835f40 1 -- 172.21.15.38:0/1130780980 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:31 13.10 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55d874ca6750 con 0x55d874d322a0 2024-09-21T03:44:01.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.804+0000 7fcc0caf2640 1 -- 172.21.15.38:0/1130780980 <== osd.7 v2:172.21.15.195:6818/1606106342 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v131'1095 uv1095 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcc0400b040 con 0x55d874d322a0 2024-09-21T03:44:01.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.804+0000 7fcc12835f40 1 --2- 172.21.15.38:0/1130780980 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x55d874d4b210 0x55d874d4e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:01.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.804+0000 7fcc12835f40 1 -- 172.21.15.38:0/1130780980 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55d874ca6750 con 0x55d874d4b210 2024-09-21T03:44:01.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.804+0000 7fcc0c2f1640 1 --2- 172.21.15.38:0/1130780980 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x55d874d4b210 0x55d874d4e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:01.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.805+0000 7fcc0c2f1640 1 --2- 172.21.15.38:0/1130780980 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x55d874d4b210 0x55d874d4e330 crc :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:01.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.807+0000 7fcc0c2f1640 1 -- 172.21.15.38:0/1130780980 <== osd.6 v2:172.21.15.195:6826/4055825675 1 ==== osd_op_reply(32 1.00000000.inode [create] v131'776 uv755 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7fcbfc004b60 con 0x55d874d4b210 2024-09-21T03:44:01.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.807+0000 7fcc12835f40 1 -- 172.21.15.38:0/1130780980 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e131) -- 0x55d874d4e870 con 0x55d874d4b210 2024-09-21T03:44:01.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.808+0000 7fcc0c2f1640 1 -- 172.21.15.38:0/1130780980 <== osd.6 v2:172.21.15.195:6826/4055825675 2 ==== osd_op_reply(33 1.00000000.inode [delete] v131'777 uv777 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fcbfc004b60 con 0x55d874d4b210 2024-09-21T03:44:01.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.808+0000 7fcc12835f40 1 -- 172.21.15.38:0/1130780980 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55d874d4ec10 con 0x55d874d4b210 2024-09-21T03:44:01.811 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-09-21T03:44:01.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.810+0000 7fcc0c2f1640 1 -- 172.21.15.38:0/1130780980 <== osd.6 v2:172.21.15.195:6826/4055825675 3 ==== osd_op_reply(34 1.00000000.inode [create] v131'778 uv778 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fcbfc004b60 con 0x55d874d4b210 2024-09-21T03:44:01.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.810+0000 7fcc12835f40 1 -- 172.21.15.38:0/1130780980 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55d874d4f460 con 0x55d874d4b210 2024-09-21T03:44:01.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.811+0000 7fcc0c2f1640 1 -- 172.21.15.38:0/1130780980 <== osd.6 v2:172.21.15.195:6826/4055825675 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v131'779 uv779 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fcbfc004b60 con 0x55d874d4b210 2024-09-21T03:44:01.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.811+0000 7fcc12835f40 1 -- 172.21.15.38:0/1130780980 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55d874d4f800 con 0x55d874d4b210 2024-09-21T03:44:01.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.813+0000 7fcc0c2f1640 1 -- 172.21.15.38:0/1130780980 <== osd.6 v2:172.21.15.195:6826/4055825675 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v131'780 uv780 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fcbfc004b60 con 0x55d874d4b210 2024-09-21T03:44:01.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.813+0000 7fcc12835f40 1 -- 172.21.15.38:0/1130780980 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55d874d4fba0 con 0x55d874d4b210 2024-09-21T03:44:01.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.814+0000 7fcc0c2f1640 1 -- 172.21.15.38:0/1130780980 <== osd.6 v2:172.21.15.195:6826/4055825675 6 ==== osd_op_reply(37 200.00000000 [create] v131'781 uv774 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fcbfc004b60 con 0x55d874d4b210 2024-09-21T03:44:01.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.814+0000 7fcc12835f40 1 -- 172.21.15.38:0/1130780980 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e131) -- 0x55d874d4ff40 con 0x55d874d4b210 2024-09-21T03:44:01.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.816+0000 7fcc0c2f1640 1 -- 172.21.15.38:0/1130780980 <== osd.6 v2:172.21.15.195:6826/4055825675 7 ==== osd_op_reply(38 200.00000000 [delete] v131'782 uv782 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcbfc004b60 con 0x55d874d4b210 2024-09-21T03:44:01.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.816+0000 7fcc12835f40 1 -- 172.21.15.38:0/1130780980 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55d874d4ff40 con 0x55d874d4b210 2024-09-21T03:44:01.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.817+0000 7fcc0c2f1640 1 -- 172.21.15.38:0/1130780980 <== osd.6 v2:172.21.15.195:6826/4055825675 8 ==== osd_op_reply(39 200.00000000 [create] v131'783 uv783 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcbfc004b60 con 0x55d874d4b210 2024-09-21T03:44:01.819 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-09-21T03:44:01.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.817+0000 7fcc12835f40 1 -- 172.21.15.38:0/1130780980 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55d874d4ff40 con 0x55d874d4b210 2024-09-21T03:44:01.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.819+0000 7fcc0c2f1640 1 -- 172.21.15.38:0/1130780980 <== osd.6 v2:172.21.15.195:6826/4055825675 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v131'784 uv784 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcbfc004b60 con 0x55d874d4b210 2024-09-21T03:44:01.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.819+0000 7fcc12835f40 1 -- 172.21.15.38:0/1130780980 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55d874d4ff40 con 0x55d874d4b210 2024-09-21T03:44:01.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.821+0000 7fcc0c2f1640 1 -- 172.21.15.38:0/1130780980 <== osd.6 v2:172.21.15.195:6826/4055825675 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v131'785 uv785 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcbfc004b60 con 0x55d874d4b210 2024-09-21T03:44:01.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.821+0000 7fcc12835f40 1 -- 172.21.15.38:0/1130780980 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:42 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55d874d4ff40 con 0x55d874d4b210 2024-09-21T03:44:01.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.823+0000 7fcc0c2f1640 1 -- 172.21.15.38:0/1130780980 <== osd.6 v2:172.21.15.195:6826/4055825675 11 ==== osd_op_reply(42 200.00000001 [create] v131'786 uv775 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fcbfc004b60 con 0x55d874d4b210 2024-09-21T03:44:01.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.823+0000 7fcc12835f40 1 -- 172.21.15.38:0/1130780980 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:43 13.14 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e131) -- 0x55d874d4ff40 con 0x55d874d4b210 2024-09-21T03:44:01.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.825+0000 7fcc0c2f1640 1 -- 172.21.15.38:0/1130780980 <== osd.6 v2:172.21.15.195:6826/4055825675 12 ==== osd_op_reply(43 200.00000001 [delete] v131'787 uv787 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcbfc004b60 con 0x55d874d4b210 2024-09-21T03:44:01.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.825+0000 7fcc12835f40 1 -- 172.21.15.38:0/1130780980 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:44 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55d874d4ff40 con 0x55d874d4b210 2024-09-21T03:44:01.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.826+0000 7fcc0c2f1640 1 -- 172.21.15.38:0/1130780980 <== osd.6 v2:172.21.15.195:6826/4055825675 13 ==== osd_op_reply(44 200.00000001 [create] v131'788 uv788 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcbfc004b60 con 0x55d874d4b210 2024-09-21T03:44:01.828 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-09-21T03:44:01.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.826+0000 7fcc12835f40 1 -- 172.21.15.38:0/1130780980 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:45 13.14 13:2e2fa760:::200.00000001:head [write 0~17490 in=17490b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55d874d543c0 con 0x55d874d4b210 2024-09-21T03:44:01.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.828+0000 7fcc0c2f1640 1 -- 172.21.15.38:0/1130780980 <== osd.6 v2:172.21.15.195:6826/4055825675 14 ==== osd_op_reply(45 200.00000001 [write 0~17490] v131'789 uv789 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcbfc004b60 con 0x55d874d4b210 2024-09-21T03:44:01.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.828+0000 7fcc12835f40 1 -- 172.21.15.38:0/1130780980 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:46 13.14 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55d874d543c0 con 0x55d874d4b210 2024-09-21T03:44:01.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.829+0000 7fcc0c2f1640 1 -- 172.21.15.38:0/1130780980 <== osd.6 v2:172.21.15.195:6826/4055825675 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v131'790 uv790 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcbfc004b60 con 0x55d874d4b210 2024-09-21T03:44:01.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.829+0000 7fcc12835f40 1 -- 172.21.15.38:0/1130780980 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:47 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55d874d4ff40 con 0x55d874d4b210 2024-09-21T03:44:01.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.830+0000 7fcc0c2f1640 1 -- 172.21.15.38:0/1130780980 <== osd.6 v2:172.21.15.195:6826/4055825675 16 ==== osd_op_reply(47 606.00000000 [create] v131'791 uv771 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fcbfc004b60 con 0x55d874d4b210 2024-09-21T03:44:01.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.830+0000 7fcc12835f40 1 -- 172.21.15.38:0/1130780980 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:48 13.14 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e131) -- 0x55d874d4ff40 con 0x55d874d4b210 2024-09-21T03:44:01.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.832+0000 7fcc0c2f1640 1 -- 172.21.15.38:0/1130780980 <== osd.6 v2:172.21.15.195:6826/4055825675 17 ==== osd_op_reply(48 606.00000000 [delete] v131'792 uv792 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcbfc004b60 con 0x55d874d4b210 2024-09-21T03:44:01.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.832+0000 7fcc12835f40 1 -- 172.21.15.38:0/1130780980 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:49 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55d874d4ff40 con 0x55d874d4b210 2024-09-21T03:44:01.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.833+0000 7fcc0c2f1640 1 -- 172.21.15.38:0/1130780980 <== osd.6 v2:172.21.15.195:6826/4055825675 18 ==== osd_op_reply(49 606.00000000 [create] v131'793 uv793 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcbfc004b60 con 0x55d874d4b210 2024-09-21T03:44:01.835 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-09-21T03:44:01.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.833+0000 7fcc12835f40 1 -- 172.21.15.38:0/1130780980 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:50 13.14 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55d874d50f50 con 0x55d874d4b210 2024-09-21T03:44:01.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.834+0000 7fcc0c2f1640 1 -- 172.21.15.38:0/1130780980 <== osd.6 v2:172.21.15.195:6826/4055825675 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v131'794 uv794 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcbfc004b60 con 0x55d874d4b210 2024-09-21T03:44:01.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.834+0000 7fcc12835f40 1 -- 172.21.15.38:0/1130780980 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:51 13.14 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55d874d50f50 con 0x55d874d4b210 2024-09-21T03:44:01.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.836+0000 7fcc0c2f1640 1 -- 172.21.15.38:0/1130780980 <== osd.6 v2:172.21.15.195:6826/4055825675 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v131'795 uv795 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcbfc004b60 con 0x55d874d4b210 2024-09-21T03:44:01.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.836+0000 7fcc12835f40 1 -- 172.21.15.38:0/1130780980 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:52 13.14 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55d874d4ff40 con 0x55d874d4b210 2024-09-21T03:44:01.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.837+0000 7fcc0c2f1640 1 -- 172.21.15.38:0/1130780980 <== osd.6 v2:172.21.15.195:6826/4055825675 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v131'796 uv796 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcbfc004b60 con 0x55d874d4b210 2024-09-21T03:44:01.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.837+0000 7fcc12835f40 1 --2- 172.21.15.38:0/1130780980 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x55d874d50450 0x55d874d52870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:01.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.837+0000 7fcc12835f40 1 -- 172.21.15.38:0/1130780980 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:53 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55d874d52e80 con 0x55d874d50450 2024-09-21T03:44:01.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.838+0000 7fcc03fff640 1 --2- 172.21.15.38:0/1130780980 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x55d874d50450 0x55d874d52870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:01.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.838+0000 7fcc03fff640 1 --2- 172.21.15.38:0/1130780980 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x55d874d50450 0x55d874d52870 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:01.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.839+0000 7fcc03fff640 1 -- 172.21.15.38:0/1130780980 <== osd.0 v2:172.21.15.38:6800/2381639433 1 ==== osd_op_reply(53 607.00000000 [create] v131'207 uv206 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fcbf405f1c0 con 0x55d874d50450 2024-09-21T03:44:01.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.839+0000 7fcc12835f40 1 -- 172.21.15.38:0/1130780980 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:54 13.1c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e131) -- 0x55d874d53300 con 0x55d874d50450 2024-09-21T03:44:01.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.841+0000 7fcc03fff640 1 -- 172.21.15.38:0/1130780980 <== osd.0 v2:172.21.15.38:6800/2381639433 2 ==== osd_op_reply(54 607.00000000 [delete] v131'208 uv208 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcbf405f1c0 con 0x55d874d50450 2024-09-21T03:44:01.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.841+0000 7fcc12835f40 1 -- 172.21.15.38:0/1130780980 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:55 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55d874d537e0 con 0x55d874d50450 2024-09-21T03:44:01.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.842+0000 7fcc03fff640 1 -- 172.21.15.38:0/1130780980 <== osd.0 v2:172.21.15.38:6800/2381639433 3 ==== osd_op_reply(55 607.00000000 [create] v131'209 uv209 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcbf405f1c0 con 0x55d874d50450 2024-09-21T03:44:01.844 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-09-21T03:44:01.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.842+0000 7fcc12835f40 1 -- 172.21.15.38:0/1130780980 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:56 13.1c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55d874d55090 con 0x55d874d50450 2024-09-21T03:44:01.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.843+0000 7fcc03fff640 1 -- 172.21.15.38:0/1130780980 <== osd.0 v2:172.21.15.38:6800/2381639433 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v131'210 uv210 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcbf405f1c0 con 0x55d874d50450 2024-09-21T03:44:01.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.843+0000 7fcc12835f40 1 -- 172.21.15.38:0/1130780980 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:57 13.1c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55d874d55430 con 0x55d874d50450 2024-09-21T03:44:01.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.844+0000 7fcc03fff640 1 -- 172.21.15.38:0/1130780980 <== osd.0 v2:172.21.15.38:6800/2381639433 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v131'211 uv211 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcbf405f1c0 con 0x55d874d50450 2024-09-21T03:44:01.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.844+0000 7fcc12835f40 1 -- 172.21.15.38:0/1130780980 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:58 13.1c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55d874d54080 con 0x55d874d50450 2024-09-21T03:44:01.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.845+0000 7fcc03fff640 1 -- 172.21.15.38:0/1130780980 <== osd.0 v2:172.21.15.38:6800/2381639433 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v131'212 uv212 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcbf405f1c0 con 0x55d874d50450 2024-09-21T03:44:01.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.845+0000 7fcc12835f40 1 --2- 172.21.15.38:0/1130780980 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x55d874d549a0 0x55d874d577e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:01.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.845+0000 7fcc12835f40 1 -- 172.21.15.38:0/1130780980 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:59 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55d874d57d20 con 0x55d874d549a0 2024-09-21T03:44:01.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.845+0000 7fcc0caf2640 1 --2- 172.21.15.38:0/1130780980 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x55d874d549a0 0x55d874d577e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:01.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.846+0000 7fcc0caf2640 1 --2- 172.21.15.38:0/1130780980 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x55d874d549a0 0x55d874d577e0 crc :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:01.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.847+0000 7fcc0caf2640 1 -- 172.21.15.38:0/1130780980 <== osd.3 v2:172.21.15.38:6808/1286810294 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v131'153 uv152 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7fcc0400b040 con 0x55d874d549a0 2024-09-21T03:44:01.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.847+0000 7fcc12835f40 1 -- 172.21.15.38:0/1130780980 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:60 13.6 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e131) -- 0x55d874d57d20 con 0x55d874d549a0 2024-09-21T03:44:01.851 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.848+0000 7fcc0caf2640 1 -- 172.21.15.38:0/1130780980 <== osd.3 v2:172.21.15.38:6808/1286810294 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v131'154 uv154 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fcc0400b040 con 0x55d874d549a0 2024-09-21T03:44:01.851 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.849+0000 7fcc12835f40 1 -- 172.21.15.38:0/1130780980 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:61 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55d874d58140 con 0x55d874d549a0 2024-09-21T03:44:01.851 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.849+0000 7fcc0caf2640 1 -- 172.21.15.38:0/1130780980 <== osd.3 v2:172.21.15.38:6808/1286810294 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v131'155 uv155 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fcc0400b040 con 0x55d874d549a0 2024-09-21T03:44:01.851 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-09-21T03:44:01.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.850+0000 7fcc12835f40 1 -- 172.21.15.38:0/1130780980 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:62 13.6 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55d874d58140 con 0x55d874d549a0 2024-09-21T03:44:01.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.851+0000 7fcc0caf2640 1 -- 172.21.15.38:0/1130780980 <== osd.3 v2:172.21.15.38:6808/1286810294 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v131'156 uv156 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fcc0400b040 con 0x55d874d549a0 2024-09-21T03:44:01.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.851+0000 7fcc12835f40 1 -- 172.21.15.38:0/1130780980 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55d874d58140 con 0x55d874d50450 2024-09-21T03:44:01.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.852+0000 7fcc03fff640 1 -- 172.21.15.38:0/1130780980 <== osd.0 v2:172.21.15.38:6800/2381639433 7 ==== osd_op_reply(63 608.00000000 [create] v131'207 uv206 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fcbf405f1c0 con 0x55d874d50450 2024-09-21T03:44:01.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.852+0000 7fcc12835f40 1 -- 172.21.15.38:0/1130780980 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e131) -- 0x55d874d584e0 con 0x55d874d50450 2024-09-21T03:44:01.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.853+0000 7fcc03fff640 1 -- 172.21.15.38:0/1130780980 <== osd.0 v2:172.21.15.38:6800/2381639433 8 ==== osd_op_reply(64 608.00000000 [delete] v131'208 uv208 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcbf405f1c0 con 0x55d874d50450 2024-09-21T03:44:01.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.853+0000 7fcc12835f40 1 -- 172.21.15.38:0/1130780980 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55d874d584e0 con 0x55d874d50450 2024-09-21T03:44:01.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.854+0000 7fcc03fff640 1 -- 172.21.15.38:0/1130780980 <== osd.0 v2:172.21.15.38:6800/2381639433 9 ==== osd_op_reply(65 608.00000000 [create] v131'209 uv209 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcbf405f1c0 con 0x55d874d50450 2024-09-21T03:44:01.856 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-09-21T03:44:01.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.854+0000 7fcc12835f40 1 -- 172.21.15.38:0/1130780980 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55d874d594f0 con 0x55d874d50450 2024-09-21T03:44:01.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.855+0000 7fcc03fff640 1 -- 172.21.15.38:0/1130780980 <== osd.0 v2:172.21.15.38:6800/2381639433 10 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v131'210 uv210 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcbf405f1c0 con 0x55d874d50450 2024-09-21T03:44:01.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.855+0000 7fcc12835f40 1 -- 172.21.15.38:0/1130780980 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55d874d594f0 con 0x55d874d50450 2024-09-21T03:44:01.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.857+0000 7fcc03fff640 1 -- 172.21.15.38:0/1130780980 <== osd.0 v2:172.21.15.38:6800/2381639433 11 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v131'211 uv211 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcbf405f1c0 con 0x55d874d50450 2024-09-21T03:44:01.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.857+0000 7fcc12835f40 1 -- 172.21.15.38:0/1130780980 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55d874d584e0 con 0x55d874d50450 2024-09-21T03:44:01.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.858+0000 7fcc03fff640 1 -- 172.21.15.38:0/1130780980 <== osd.0 v2:172.21.15.38:6800/2381639433 12 ==== osd_op_reply(68 608.00000000 [omap-set-header] v131'212 uv212 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcbf405f1c0 con 0x55d874d50450 2024-09-21T03:44:01.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.858+0000 7fcc12835f40 1 -- 172.21.15.38:0/1130780980 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55d874d584e0 con 0x55d874d549a0 2024-09-21T03:44:01.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.859+0000 7fcc0caf2640 1 -- 172.21.15.38:0/1130780980 <== osd.3 v2:172.21.15.38:6808/1286810294 5 ==== osd_op_reply(69 604.00000000 [create] v131'382 uv376 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fcc0400b040 con 0x55d874d549a0 2024-09-21T03:44:01.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.859+0000 7fcc12835f40 1 -- 172.21.15.38:0/1130780980 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e131) -- 0x55d874d584e0 con 0x55d874d549a0 2024-09-21T03:44:01.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.860+0000 7fcc0caf2640 1 -- 172.21.15.38:0/1130780980 <== osd.3 v2:172.21.15.38:6808/1286810294 6 ==== osd_op_reply(70 604.00000000 [delete] v131'383 uv383 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcc0400b040 con 0x55d874d549a0 2024-09-21T03:44:01.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.860+0000 7fcc12835f40 1 -- 172.21.15.38:0/1130780980 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55d874d584e0 con 0x55d874d549a0 2024-09-21T03:44:01.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.862+0000 7fcc0caf2640 1 -- 172.21.15.38:0/1130780980 <== osd.3 v2:172.21.15.38:6808/1286810294 7 ==== osd_op_reply(71 604.00000000 [create] v131'384 uv384 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcc0400b040 con 0x55d874d549a0 2024-09-21T03:44:01.864 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-09-21T03:44:01.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.862+0000 7fcc12835f40 1 -- 172.21.15.38:0/1130780980 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55d874d594f0 con 0x55d874d549a0 2024-09-21T03:44:01.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.863+0000 7fcc0caf2640 1 -- 172.21.15.38:0/1130780980 <== osd.3 v2:172.21.15.38:6808/1286810294 8 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v131'385 uv385 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcc0400b040 con 0x55d874d549a0 2024-09-21T03:44:01.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.863+0000 7fcc12835f40 1 -- 172.21.15.38:0/1130780980 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55d874d594f0 con 0x55d874d549a0 2024-09-21T03:44:01.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.864+0000 7fcc0caf2640 1 -- 172.21.15.38:0/1130780980 <== osd.3 v2:172.21.15.38:6808/1286810294 9 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v131'386 uv386 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcc0400b040 con 0x55d874d549a0 2024-09-21T03:44:01.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.864+0000 7fcc12835f40 1 -- 172.21.15.38:0/1130780980 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55d874d584e0 con 0x55d874d549a0 2024-09-21T03:44:01.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.865+0000 7fcc0caf2640 1 -- 172.21.15.38:0/1130780980 <== osd.3 v2:172.21.15.38:6808/1286810294 10 ==== osd_op_reply(74 604.00000000 [omap-set-header] v131'387 uv387 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcc0400b040 con 0x55d874d549a0 2024-09-21T03:44:01.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.865+0000 7fcc12835f40 1 -- 172.21.15.38:0/1130780980 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55d874d584e0 con 0x55d874d549a0 2024-09-21T03:44:01.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.866+0000 7fcc0caf2640 1 -- 172.21.15.38:0/1130780980 <== osd.3 v2:172.21.15.38:6808/1286810294 11 ==== osd_op_reply(75 500.00000000 [create] v131'388 uv381 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fcc0400b040 con 0x55d874d549a0 2024-09-21T03:44:01.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.866+0000 7fcc12835f40 1 -- 172.21.15.38:0/1130780980 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e131) -- 0x55d874d584e0 con 0x55d874d549a0 2024-09-21T03:44:01.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.867+0000 7fcc0caf2640 1 -- 172.21.15.38:0/1130780980 <== osd.3 v2:172.21.15.38:6808/1286810294 12 ==== osd_op_reply(76 500.00000000 [delete] v131'389 uv389 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcc0400b040 con 0x55d874d549a0 2024-09-21T03:44:01.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.867+0000 7fcc12835f40 1 -- 172.21.15.38:0/1130780980 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55d874d584e0 con 0x55d874d549a0 2024-09-21T03:44:01.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.868+0000 7fcc0caf2640 1 -- 172.21.15.38:0/1130780980 <== osd.3 v2:172.21.15.38:6808/1286810294 13 ==== osd_op_reply(77 500.00000000 [create] v131'390 uv390 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcc0400b040 con 0x55d874d549a0 2024-09-21T03:44:01.870 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-09-21T03:44:01.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.868+0000 7fcc12835f40 1 -- 172.21.15.38:0/1130780980 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55d874d584e0 con 0x55d874d549a0 2024-09-21T03:44:01.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.870+0000 7fcc0caf2640 1 -- 172.21.15.38:0/1130780980 <== osd.3 v2:172.21.15.38:6808/1286810294 14 ==== osd_op_reply(78 500.00000000 [write 0~90] v131'391 uv391 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcc0400b040 con 0x55d874d549a0 2024-09-21T03:44:01.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.870+0000 7fcc12835f40 1 -- 172.21.15.38:0/1130780980 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55d874d584e0 con 0x55d874d549a0 2024-09-21T03:44:01.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.871+0000 7fcc0caf2640 1 -- 172.21.15.38:0/1130780980 <== osd.3 v2:172.21.15.38:6808/1286810294 15 ==== osd_op_reply(79 500.00000000 [omap-set-header] v131'392 uv392 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcc0400b040 con 0x55d874d549a0 2024-09-21T03:44:01.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.871+0000 7fcc12835f40 1 -- 172.21.15.38:0/1130780980 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:80 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55d874d584e0 con 0x55d874d50450 2024-09-21T03:44:01.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.872+0000 7fcc03fff640 1 -- 172.21.15.38:0/1130780980 <== osd.0 v2:172.21.15.38:6800/2381639433 13 ==== osd_op_reply(80 mds_snaptable [create] v131'173 uv172 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7fcbf405f1c0 con 0x55d874d50450 2024-09-21T03:44:01.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.872+0000 7fcc12835f40 1 -- 172.21.15.38:0/1130780980 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:81 13.d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e131) -- 0x55d874d584e0 con 0x55d874d50450 2024-09-21T03:44:01.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.873+0000 7fcc03fff640 1 -- 172.21.15.38:0/1130780980 <== osd.0 v2:172.21.15.38:6800/2381639433 14 ==== osd_op_reply(81 mds_snaptable [delete] v131'174 uv174 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fcbf405f1c0 con 0x55d874d50450 2024-09-21T03:44:01.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.873+0000 7fcc12835f40 1 -- 172.21.15.38:0/1130780980 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:82 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55d874d584e0 con 0x55d874d50450 2024-09-21T03:44:01.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.874+0000 7fcc03fff640 1 -- 172.21.15.38:0/1130780980 <== osd.0 v2:172.21.15.38:6800/2381639433 15 ==== osd_op_reply(82 mds_snaptable [create] v131'175 uv175 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fcbf405f1c0 con 0x55d874d50450 2024-09-21T03:44:01.876 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-09-21T03:44:01.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.874+0000 7fcc12835f40 1 -- 172.21.15.38:0/1130780980 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:83 13.d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55d874d584e0 con 0x55d874d50450 2024-09-21T03:44:01.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.875+0000 7fcc03fff640 1 -- 172.21.15.38:0/1130780980 <== osd.0 v2:172.21.15.38:6800/2381639433 16 ==== osd_op_reply(83 mds_snaptable [write 0~70] v131'176 uv176 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fcbf405f1c0 con 0x55d874d50450 2024-09-21T03:44:01.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.875+0000 7fcc12835f40 1 -- 172.21.15.38:0/1130780980 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:84 13.d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55d874d584e0 con 0x55d874d50450 2024-09-21T03:44:01.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.876+0000 7fcc03fff640 1 -- 172.21.15.38:0/1130780980 <== osd.0 v2:172.21.15.38:6800/2381639433 17 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v131'177 uv177 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fcbf405f1c0 con 0x55d874d50450 2024-09-21T03:44:01.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.876+0000 7fcc12835f40 1 --2- 172.21.15.38:0/1130780980 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x55d874d58b60 0x55d874d5afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:01.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.876+0000 7fcc12835f40 1 -- 172.21.15.38:0/1130780980 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55d874d5b600 con 0x55d874d58b60 2024-09-21T03:44:01.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.876+0000 7fcc0c2f1640 1 --2- 172.21.15.38:0/1130780980 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x55d874d58b60 0x55d874d5afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:01.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.877+0000 7fcc0c2f1640 1 --2- 172.21.15.38:0/1130780980 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x55d874d58b60 0x55d874d5afa0 crc :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:01.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.878+0000 7fcc0c2f1640 1 -- 172.21.15.38:0/1130780980 <== osd.1 v2:172.21.15.38:6816/1491818732 1 ==== osd_op_reply(85 605.00000000 [create] v131'207 uv206 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fcbfc004b60 con 0x55d874d58b60 2024-09-21T03:44:01.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.878+0000 7fcc12835f40 1 -- 172.21.15.38:0/1130780980 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e131) -- 0x55d874d5b9d0 con 0x55d874d58b60 2024-09-21T03:44:01.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.879+0000 7fcc0c2f1640 1 -- 172.21.15.38:0/1130780980 <== osd.1 v2:172.21.15.38:6816/1491818732 2 ==== osd_op_reply(86 605.00000000 [delete] v131'208 uv208 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcbfc004b60 con 0x55d874d58b60 2024-09-21T03:44:01.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.879+0000 7fcc12835f40 1 -- 172.21.15.38:0/1130780980 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55d874d5b9d0 con 0x55d874d58b60 2024-09-21T03:44:01.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.880+0000 7fcc0c2f1640 1 -- 172.21.15.38:0/1130780980 <== osd.1 v2:172.21.15.38:6816/1491818732 3 ==== osd_op_reply(87 605.00000000 [create] v131'209 uv209 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcbfc004b60 con 0x55d874d58b60 2024-09-21T03:44:01.882 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-09-21T03:44:01.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.880+0000 7fcc12835f40 1 -- 172.21.15.38:0/1130780980 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55d874d5c9e0 con 0x55d874d58b60 2024-09-21T03:44:01.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.881+0000 7fcc0c2f1640 1 -- 172.21.15.38:0/1130780980 <== osd.1 v2:172.21.15.38:6816/1491818732 4 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v131'210 uv210 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcbfc004b60 con 0x55d874d58b60 2024-09-21T03:44:01.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.881+0000 7fcc12835f40 1 -- 172.21.15.38:0/1130780980 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55d874d5c9e0 con 0x55d874d58b60 2024-09-21T03:44:01.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.883+0000 7fcc0c2f1640 1 -- 172.21.15.38:0/1130780980 <== osd.1 v2:172.21.15.38:6816/1491818732 5 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v131'211 uv211 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcbfc004b60 con 0x55d874d58b60 2024-09-21T03:44:01.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.883+0000 7fcc12835f40 1 -- 172.21.15.38:0/1130780980 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55d874d5b9d0 con 0x55d874d58b60 2024-09-21T03:44:01.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.884+0000 7fcc0c2f1640 1 -- 172.21.15.38:0/1130780980 <== osd.1 v2:172.21.15.38:6816/1491818732 6 ==== osd_op_reply(90 605.00000000 [omap-set-header] v131'212 uv212 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcbfc004b60 con 0x55d874d58b60 2024-09-21T03:44:01.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.884+0000 7fcc12835f40 1 -- 172.21.15.38:0/1130780980 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55d874d5b9d0 con 0x55d874d58b60 2024-09-21T03:44:01.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.885+0000 7fcc0c2f1640 1 -- 172.21.15.38:0/1130780980 <== osd.1 v2:172.21.15.38:6816/1491818732 7 ==== osd_op_reply(91 mds0_inotable [create] v131'345 uv339 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7fcbfc004b60 con 0x55d874d58b60 2024-09-21T03:44:01.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.885+0000 7fcc12835f40 1 -- 172.21.15.38:0/1130780980 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e131) -- 0x55d874d5b9d0 con 0x55d874d58b60 2024-09-21T03:44:01.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.886+0000 7fcc0c2f1640 1 -- 172.21.15.38:0/1130780980 <== osd.1 v2:172.21.15.38:6816/1491818732 8 ==== osd_op_reply(92 mds0_inotable [delete] v131'346 uv346 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fcbfc004b60 con 0x55d874d58b60 2024-09-21T03:44:01.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.886+0000 7fcc12835f40 1 -- 172.21.15.38:0/1130780980 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55d874d5b9d0 con 0x55d874d58b60 2024-09-21T03:44:01.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.887+0000 7fcc0c2f1640 1 -- 172.21.15.38:0/1130780980 <== osd.1 v2:172.21.15.38:6816/1491818732 9 ==== osd_op_reply(93 mds0_inotable [create] v131'347 uv347 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fcbfc004b60 con 0x55d874d58b60 2024-09-21T03:44:01.889 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-09-21T03:44:01.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.887+0000 7fcc12835f40 1 -- 172.21.15.38:0/1130780980 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:94 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55d874d5bdc0 con 0x55d874d58b60 2024-09-21T03:44:01.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.888+0000 7fcc0c2f1640 1 -- 172.21.15.38:0/1130780980 <== osd.1 v2:172.21.15.38:6816/1491818732 10 ==== osd_op_reply(94 mds0_inotable [write 0~34] v131'348 uv348 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fcbfc004b60 con 0x55d874d58b60 2024-09-21T03:44:01.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.888+0000 7fcc12835f40 1 -- 172.21.15.38:0/1130780980 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:95 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55d874d5ba70 con 0x55d874d58b60 2024-09-21T03:44:01.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.889+0000 7fcc0c2f1640 1 -- 172.21.15.38:0/1130780980 <== osd.1 v2:172.21.15.38:6816/1491818732 11 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v131'349 uv349 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fcbfc004b60 con 0x55d874d58b60 2024-09-21T03:44:01.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.889+0000 7fcc12835f40 1 -- 172.21.15.38:0/1130780980 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:96 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55d874d5ba70 con 0x55d874d58b60 2024-09-21T03:44:01.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.890+0000 7fcc0c2f1640 1 -- 172.21.15.38:0/1130780980 <== osd.1 v2:172.21.15.38:6816/1491818732 12 ==== osd_op_reply(96 100.00000000 [create] v131'350 uv344 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fcbfc004b60 con 0x55d874d58b60 2024-09-21T03:44:01.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.890+0000 7fcc12835f40 1 -- 172.21.15.38:0/1130780980 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:97 13.13 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e131) -- 0x55d874d5ba70 con 0x55d874d58b60 2024-09-21T03:44:01.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.891+0000 7fcc0c2f1640 1 -- 172.21.15.38:0/1130780980 <== osd.1 v2:172.21.15.38:6816/1491818732 13 ==== osd_op_reply(97 100.00000000 [delete] v131'351 uv351 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcbfc004b60 con 0x55d874d58b60 2024-09-21T03:44:01.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.891+0000 7fcc12835f40 1 -- 172.21.15.38:0/1130780980 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:98 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55d874d5ba70 con 0x55d874d58b60 2024-09-21T03:44:01.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.892+0000 7fcc0c2f1640 1 -- 172.21.15.38:0/1130780980 <== osd.1 v2:172.21.15.38:6816/1491818732 14 ==== osd_op_reply(98 100.00000000 [create] v131'352 uv352 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcbfc004b60 con 0x55d874d58b60 2024-09-21T03:44:01.895 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-09-21T03:44:01.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.892+0000 7fcc12835f40 1 -- 172.21.15.38:0/1130780980 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:99 13.13 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55d874d5ba70 con 0x55d874d58b60 2024-09-21T03:44:01.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.893+0000 7fcc0c2f1640 1 -- 172.21.15.38:0/1130780980 <== osd.1 v2:172.21.15.38:6816/1491818732 15 ==== osd_op_reply(99 100.00000000 [omap-set-header] v131'353 uv353 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcbfc004b60 con 0x55d874d58b60 2024-09-21T03:44:01.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.894+0000 7fcc12835f40 1 -- 172.21.15.38:0/1130780980 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:100 13.13 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55d874d53cc0 con 0x55d874d58b60 2024-09-21T03:44:01.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.895+0000 7fcc0c2f1640 1 -- 172.21.15.38:0/1130780980 <== osd.1 v2:172.21.15.38:6816/1491818732 16 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v131'354 uv354 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcbfc004b60 con 0x55d874d58b60 2024-09-21T03:44:01.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.895+0000 7fcc12835f40 1 -- 172.21.15.38:0/1130780980 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55d874d53c80 con 0x55d874d549a0 2024-09-21T03:44:01.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.896+0000 7fcc0caf2640 1 -- 172.21.15.38:0/1130780980 <== osd.3 v2:172.21.15.38:6808/1286810294 16 ==== osd_op_reply(101 mds0_sessionmap [create] v131'152 uv150 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7fcc0400b040 con 0x55d874d549a0 2024-09-21T03:44:01.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.896+0000 7fcc12835f40 1 -- 172.21.15.38:0/1130780980 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e131) -- 0x55d874d53c80 con 0x55d874d549a0 2024-09-21T03:44:01.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.897+0000 7fcc0caf2640 1 -- 172.21.15.38:0/1130780980 <== osd.3 v2:172.21.15.38:6808/1286810294 17 ==== osd_op_reply(102 mds0_sessionmap [delete] v131'153 uv153 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fcc0400b040 con 0x55d874d549a0 2024-09-21T03:44:01.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.897+0000 7fcc12835f40 1 -- 172.21.15.38:0/1130780980 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:103 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55d874d53c80 con 0x55d874d549a0 2024-09-21T03:44:01.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.898+0000 7fcc0caf2640 1 -- 172.21.15.38:0/1130780980 <== osd.3 v2:172.21.15.38:6808/1286810294 18 ==== osd_op_reply(103 mds0_sessionmap [create] v131'154 uv154 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fcc0400b040 con 0x55d874d549a0 2024-09-21T03:44:01.900 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-09-21T03:44:01.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.898+0000 7fcc12835f40 1 -- 172.21.15.38:0/1130780980 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:104 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55d874d53c80 con 0x55d874d549a0 2024-09-21T03:44:01.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.899+0000 7fcc0caf2640 1 -- 172.21.15.38:0/1130780980 <== osd.3 v2:172.21.15.38:6808/1286810294 19 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v131'155 uv155 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fcc0400b040 con 0x55d874d549a0 2024-09-21T03:44:01.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.899+0000 7fcc12835f40 1 -- 172.21.15.38:0/1130780980 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55d874d53c60 con 0x55d874d549a0 2024-09-21T03:44:01.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.900+0000 7fcc0caf2640 1 -- 172.21.15.38:0/1130780980 <== osd.3 v2:172.21.15.38:6808/1286810294 20 ==== osd_op_reply(105 609.00000000 [create] v131'207 uv206 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fcc0400b040 con 0x55d874d549a0 2024-09-21T03:44:01.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.900+0000 7fcc12835f40 1 -- 172.21.15.38:0/1130780980 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e131) -- 0x55d874d53c60 con 0x55d874d549a0 2024-09-21T03:44:01.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.901+0000 7fcc0caf2640 1 -- 172.21.15.38:0/1130780980 <== osd.3 v2:172.21.15.38:6808/1286810294 21 ==== osd_op_reply(106 609.00000000 [delete] v131'208 uv208 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcc0400b040 con 0x55d874d549a0 2024-09-21T03:44:01.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.901+0000 7fcc12835f40 1 -- 172.21.15.38:0/1130780980 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55d874d53c60 con 0x55d874d549a0 2024-09-21T03:44:01.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.902+0000 7fcc0caf2640 1 -- 172.21.15.38:0/1130780980 <== osd.3 v2:172.21.15.38:6808/1286810294 22 ==== osd_op_reply(107 609.00000000 [create] v131'209 uv209 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcc0400b040 con 0x55d874d549a0 2024-09-21T03:44:01.904 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-09-21T03:44:01.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.902+0000 7fcc12835f40 1 -- 172.21.15.38:0/1130780980 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55d874d53c60 con 0x55d874d549a0 2024-09-21T03:44:01.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.903+0000 7fcc0caf2640 1 -- 172.21.15.38:0/1130780980 <== osd.3 v2:172.21.15.38:6808/1286810294 23 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v131'210 uv210 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcc0400b040 con 0x55d874d549a0 2024-09-21T03:44:01.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.903+0000 7fcc12835f40 1 -- 172.21.15.38:0/1130780980 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:109 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55d874d53c60 con 0x55d874d549a0 2024-09-21T03:44:01.906 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.904+0000 7fcc0caf2640 1 -- 172.21.15.38:0/1130780980 <== osd.3 v2:172.21.15.38:6808/1286810294 24 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v131'211 uv211 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcc0400b040 con 0x55d874d549a0 2024-09-21T03:44:01.906 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.904+0000 7fcc12835f40 1 -- 172.21.15.38:0/1130780980 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:110 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55d874d53c60 con 0x55d874d549a0 2024-09-21T03:44:01.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.905+0000 7fcc0caf2640 1 -- 172.21.15.38:0/1130780980 <== osd.3 v2:172.21.15.38:6808/1286810294 25 ==== osd_op_reply(110 609.00000000 [omap-set-header] v131'212 uv212 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcc0400b040 con 0x55d874d549a0 2024-09-21T03:44:01.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.905+0000 7fcc12835f40 1 --2- 172.21.15.38:0/1130780980 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x55d874d5c150 0x55d874d5edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:01.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.905+0000 7fcc12835f40 1 -- 172.21.15.38:0/1130780980 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55d874d53c60 con 0x55d874d5c150 2024-09-21T03:44:01.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.905+0000 7fcc03fff640 1 --2- 172.21.15.38:0/1130780980 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x55d874d5c150 0x55d874d5edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:01.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.906+0000 7fcc03fff640 1 --2- 172.21.15.38:0/1130780980 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x55d874d5c150 0x55d874d5edd0 crc :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:01.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.907+0000 7fcc03fff640 1 -- 172.21.15.38:0/1130780980 <== osd.2 v2:172.21.15.38:6824/2577858902 1 ==== osd_op_reply(111 400.00000000 [create] v131'176 uv175 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fcbf405f1c0 con 0x55d874d5c150 2024-09-21T03:44:01.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.907+0000 7fcc12835f40 1 -- 172.21.15.38:0/1130780980 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e131) -- 0x55d874d53c60 con 0x55d874d5c150 2024-09-21T03:44:01.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.908+0000 7fcc03fff640 1 -- 172.21.15.38:0/1130780980 <== osd.2 v2:172.21.15.38:6824/2577858902 2 ==== osd_op_reply(112 400.00000000 [delete] v131'177 uv177 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcbf405f1c0 con 0x55d874d5c150 2024-09-21T03:44:01.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.908+0000 7fcc12835f40 1 -- 172.21.15.38:0/1130780980 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55d874d53c60 con 0x55d874d5c150 2024-09-21T03:44:01.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.909+0000 7fcc03fff640 1 -- 172.21.15.38:0/1130780980 <== osd.2 v2:172.21.15.38:6824/2577858902 3 ==== osd_op_reply(113 400.00000000 [create] v131'178 uv178 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcbf405f1c0 con 0x55d874d5c150 2024-09-21T03:44:01.911 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-09-21T03:44:01.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.909+0000 7fcc12835f40 1 -- 172.21.15.38:0/1130780980 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:114 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55d874d5f360 con 0x55d874d5c150 2024-09-21T03:44:01.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.910+0000 7fcc03fff640 1 -- 172.21.15.38:0/1130780980 <== osd.2 v2:172.21.15.38:6824/2577858902 4 ==== osd_op_reply(114 400.00000000 [write 0~22] v131'179 uv179 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcbf405f1c0 con 0x55d874d5c150 2024-09-21T03:44:01.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.911+0000 7fcc12835f40 1 -- 172.21.15.38:0/1130780980 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:115 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55d874d5f360 con 0x55d874d5c150 2024-09-21T03:44:01.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.912+0000 7fcc03fff640 1 -- 172.21.15.38:0/1130780980 <== osd.2 v2:172.21.15.38:6824/2577858902 5 ==== osd_op_reply(115 400.00000000 [omap-set-header] v131'180 uv180 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcbf405f1c0 con 0x55d874d5c150 2024-09-21T03:44:01.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.912+0000 7fcc12835f40 1 -- 172.21.15.38:0/1130780980 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:116 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55d874d5f310 con 0x55d874d58b60 2024-09-21T03:44:01.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.913+0000 7fcc0c2f1640 1 -- 172.21.15.38:0/1130780980 <== osd.1 v2:172.21.15.38:6816/1491818732 17 ==== osd_op_reply(116 100.00000000.inode [create] v131'429 uv420 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7fcbfc004b60 con 0x55d874d58b60 2024-09-21T03:44:01.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.913+0000 7fcc12835f40 1 -- 172.21.15.38:0/1130780980 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:117 13.1f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e131) -- 0x55d874d5f310 con 0x55d874d58b60 2024-09-21T03:44:01.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.914+0000 7fcc0c2f1640 1 -- 172.21.15.38:0/1130780980 <== osd.1 v2:172.21.15.38:6816/1491818732 18 ==== osd_op_reply(117 100.00000000.inode [delete] v131'430 uv430 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fcbfc004b60 con 0x55d874d58b60 2024-09-21T03:44:01.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.914+0000 7fcc12835f40 1 -- 172.21.15.38:0/1130780980 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:118 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55d874d5f310 con 0x55d874d58b60 2024-09-21T03:44:01.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.916+0000 7fcc0c2f1640 1 -- 172.21.15.38:0/1130780980 <== osd.1 v2:172.21.15.38:6816/1491818732 19 ==== osd_op_reply(118 100.00000000.inode [create] v131'431 uv431 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fcbfc004b60 con 0x55d874d58b60 2024-09-21T03:44:01.918 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-09-21T03:44:01.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.916+0000 7fcc12835f40 1 -- 172.21.15.38:0/1130780980 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:119 13.1f 13:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55d874d5f310 con 0x55d874d58b60 2024-09-21T03:44:01.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.917+0000 7fcc0c2f1640 1 -- 172.21.15.38:0/1130780980 <== osd.1 v2:172.21.15.38:6816/1491818732 20 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v131'432 uv432 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fcbfc004b60 con 0x55d874d58b60 2024-09-21T03:44:01.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.917+0000 7fcc12835f40 1 -- 172.21.15.38:0/1130780980 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:120 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55d874d5f310 con 0x55d874d58b60 2024-09-21T03:44:01.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.918+0000 7fcc0c2f1640 1 -- 172.21.15.38:0/1130780980 <== osd.1 v2:172.21.15.38:6816/1491818732 21 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v131'433 uv433 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fcbfc004b60 con 0x55d874d58b60 2024-09-21T03:44:01.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.918+0000 7fcc12835f40 1 -- 172.21.15.38:0/1130780980 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:121 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55d874d5f310 con 0x55d874d58b60 2024-09-21T03:44:01.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.919+0000 7fcc0c2f1640 1 -- 172.21.15.38:0/1130780980 <== osd.1 v2:172.21.15.38:6816/1491818732 22 ==== osd_op_reply(121 1.00000000 [create] v131'434 uv427 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7fcbfc004b60 con 0x55d874d58b60 2024-09-21T03:44:01.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.919+0000 7fcc12835f40 1 -- 172.21.15.38:0/1130780980 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:122 13.1f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e131) -- 0x55d874d5f310 con 0x55d874d58b60 2024-09-21T03:44:01.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.920+0000 7fcc0c2f1640 1 -- 172.21.15.38:0/1130780980 <== osd.1 v2:172.21.15.38:6816/1491818732 23 ==== osd_op_reply(122 1.00000000 [delete] v131'435 uv435 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fcbfc004b60 con 0x55d874d58b60 2024-09-21T03:44:01.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.920+0000 7fcc12835f40 1 -- 172.21.15.38:0/1130780980 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:123 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55d874d5f310 con 0x55d874d58b60 2024-09-21T03:44:01.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.921+0000 7fcc0c2f1640 1 -- 172.21.15.38:0/1130780980 <== osd.1 v2:172.21.15.38:6816/1491818732 24 ==== osd_op_reply(123 1.00000000 [create] v131'436 uv436 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fcbfc004b60 con 0x55d874d58b60 2024-09-21T03:44:01.923 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-09-21T03:44:01.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.921+0000 7fcc12835f40 1 -- 172.21.15.38:0/1130780980 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:124 13.1f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55d874d60320 con 0x55d874d58b60 2024-09-21T03:44:01.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.922+0000 7fcc0c2f1640 1 -- 172.21.15.38:0/1130780980 <== osd.1 v2:172.21.15.38:6816/1491818732 25 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v131'437 uv437 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fcbfc004b60 con 0x55d874d58b60 2024-09-21T03:44:01.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.922+0000 7fcc12835f40 1 -- 172.21.15.38:0/1130780980 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:125 13.1f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55d874d60320 con 0x55d874d58b60 2024-09-21T03:44:01.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.923+0000 7fcc0c2f1640 1 -- 172.21.15.38:0/1130780980 <== osd.1 v2:172.21.15.38:6816/1491818732 26 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v131'438 uv438 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fcbfc004b60 con 0x55d874d58b60 2024-09-21T03:44:01.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.923+0000 7fcc12835f40 1 -- 172.21.15.38:0/1130780980 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:126 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55d874d5f310 con 0x55d874d58b60 2024-09-21T03:44:01.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.924+0000 7fcc0c2f1640 1 -- 172.21.15.38:0/1130780980 <== osd.1 v2:172.21.15.38:6816/1491818732 27 ==== osd_op_reply(126 1.00000000 [omap-set-header] v131'439 uv439 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fcbfc004b60 con 0x55d874d58b60 2024-09-21T03:44:01.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.924+0000 7fcc12835f40 1 -- 172.21.15.38:0/1130780980 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:127 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=584b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55d874d5f7c0 con 0x55d874d58b60 2024-09-21T03:44:01.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.926+0000 7fcc0c2f1640 1 -- 172.21.15.38:0/1130780980 <== osd.1 v2:172.21.15.38:6816/1491818732 28 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v131'440 uv440 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fcbfc004b60 con 0x55d874d58b60 2024-09-21T03:44:01.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.926+0000 7fcc12835f40 1 -- 172.21.15.38:0/1130780980 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x55d874d50450 msgr2=0x55d874d52870 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:01.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.926+0000 7fcc12835f40 1 --2- 172.21.15.38:0/1130780980 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x55d874d50450 0x55d874d52870 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:01.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.926+0000 7fcc12835f40 1 -- 172.21.15.38:0/1130780980 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x55d874d58b60 msgr2=0x55d874d5afa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:01.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.926+0000 7fcc12835f40 1 --2- 172.21.15.38:0/1130780980 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x55d874d58b60 0x55d874d5afa0 crc :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:01.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.926+0000 7fcc12835f40 1 -- 172.21.15.38:0/1130780980 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x55d874d5c150 msgr2=0x55d874d5edd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:01.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.926+0000 7fcc12835f40 1 --2- 172.21.15.38:0/1130780980 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x55d874d5c150 0x55d874d5edd0 crc :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:01.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.926+0000 7fcc12835f40 1 -- 172.21.15.38:0/1130780980 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x55d874d549a0 msgr2=0x55d874d577e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:01.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.926+0000 7fcc12835f40 1 --2- 172.21.15.38:0/1130780980 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x55d874d549a0 0x55d874d577e0 crc :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:01.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.926+0000 7fcc0caf2640 1 -- 172.21.15.38:0/1130780980 reap_dead start 2024-09-21T03:44:01.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.926+0000 7fcc12835f40 1 -- 172.21.15.38:0/1130780980 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x55d874d4b210 msgr2=0x55d874d4e330 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:01.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.926+0000 7fcc12835f40 1 --2- 172.21.15.38:0/1130780980 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x55d874d4b210 0x55d874d4e330 crc :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:01.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.926+0000 7fcc0caf2640 1 -- 172.21.15.38:0/1130780980 reap_dead start 2024-09-21T03:44:01.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.926+0000 7fcc12835f40 1 -- 172.21.15.38:0/1130780980 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x55d874d322a0 msgr2=0x55d874d2fa40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:01.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.926+0000 7fcc12835f40 1 --2- 172.21.15.38:0/1130780980 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x55d874d322a0 0x55d874d2fa40 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:01.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.926+0000 7fcc12835f40 1 -- 172.21.15.38:0/1130780980 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fcbec077570 msgr2=0x7fcbec079a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:01.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.926+0000 7fcc12835f40 1 --2- 172.21.15.38:0/1130780980 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fcbec077570 0x7fcbec079a80 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fcbfc055000 tx=0x7fcbfc058040 comp rx=0 tx=0).stop 2024-09-21T03:44:01.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.926+0000 7fcc12835f40 1 -- 172.21.15.38:0/1130780980 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d874c858b0 msgr2=0x55d874d31180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:01.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.926+0000 7fcc12835f40 1 --2- 172.21.15.38:0/1130780980 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d874c858b0 0x55d874d31180 secure :-1 s=READY pgs=2139 cs=0 l=1 rev1=1 crypto rx=0x7fcbf400f000 tx=0x7fcbf400b9d0 comp rx=0 tx=0).stop 2024-09-21T03:44:01.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.926+0000 7fcc12835f40 1 -- 172.21.15.38:0/1130780980 shutdown_connections 2024-09-21T03:44:01.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.926+0000 7fcc12835f40 1 --2- 172.21.15.38:0/1130780980 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x55d874d4b210 0x55d874d4e330 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:01.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.926+0000 7fcc12835f40 1 --2- 172.21.15.38:0/1130780980 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x55d874d322a0 0x55d874d2fa40 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:01.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.926+0000 7fcc12835f40 1 --2- 172.21.15.38:0/1130780980 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fcbec077570 0x7fcbec079a80 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:01.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.926+0000 7fcc12835f40 1 --2- 172.21.15.38:0/1130780980 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d874c858b0 0x55d874d31180 unknown :-1 s=CLOSED pgs=2139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:01.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.926+0000 7fcc12835f40 1 -- 172.21.15.38:0/1130780980 >> 172.21.15.38:0/1130780980 conn(0x55d874aefac0 msgr2=0x55d874bd11b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:01.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.926+0000 7fcc12835f40 1 -- 172.21.15.38:0/1130780980 shutdown_connections 2024-09-21T03:44:01.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:01.926+0000 7fcc12835f40 1 -- 172.21.15.38:0/1130780980 wait complete. 2024-09-21T03:44:01.932 DEBUG:teuthology.orchestra.run.smithi038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata setomapheader 600.00000000 deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef 2024-09-21T03:44:02.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.027+0000 7fbddf71ef40 1 Processor -- start 2024-09-21T03:44:02.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.027+0000 7fbddf71ef40 1 -- start start 2024-09-21T03:44:02.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.028+0000 7fbddf71ef40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x565141339a90 0x565141295eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:02.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.028+0000 7fbddf71ef40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5651412949e0 0x56514126aec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:02.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.029+0000 7fbddf71ef40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5651413266c0 0x565141326ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:02.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.029+0000 7fbddf71ef40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5651411b9d60 con 0x5651413266c0 2024-09-21T03:44:02.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.029+0000 7fbddf71ef40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x565141218430 con 0x565141339a90 2024-09-21T03:44:02.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.029+0000 7fbddf71ef40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x56514133f810 con 0x5651412949e0 2024-09-21T03:44:02.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.029+0000 7fbdd8f0c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5651413266c0 0x565141326ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:02.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.029+0000 7fbdd870b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x565141339a90 0x565141295eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:02.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.029+0000 7fbdd870b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x565141339a90 0x565141295eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.38:49324/0 (socket says 172.21.15.38:49324) 2024-09-21T03:44:02.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.029+0000 7fbdd870b640 1 -- 172.21.15.38:0/1400231130 learned_addr learned my addr 172.21.15.38:0/1400231130 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:44:02.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.029+0000 7fbdd7f0a640 1 --2- 172.21.15.38:0/1400231130 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5651412949e0 0x56514126aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:02.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.029+0000 7fbdd8f0c640 1 -- 172.21.15.38:0/1400231130 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5651412949e0 msgr2=0x56514126aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:44:02.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.029+0000 7fbdd8f0c640 1 --2- 172.21.15.38:0/1400231130 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5651412949e0 0x56514126aec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:02.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.029+0000 7fbdd8f0c640 1 -- 172.21.15.38:0/1400231130 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x565141339a90 msgr2=0x565141295eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:44:02.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.029+0000 7fbdd8f0c640 1 --2- 172.21.15.38:0/1400231130 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x565141339a90 0x565141295eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:02.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.029+0000 7fbdd8f0c640 1 -- 172.21.15.38:0/1400231130 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565141339ea0 con 0x5651413266c0 2024-09-21T03:44:02.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.029+0000 7fbdd870b640 1 --2- 172.21.15.38:0/1400231130 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x565141339a90 0x565141295eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:44:02.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.029+0000 7fbdd8f0c640 1 --2- 172.21.15.38:0/1400231130 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5651413266c0 0x565141326ad0 secure :-1 s=READY pgs=2142 cs=0 l=1 rev1=1 crypto rx=0x7fbdd0002000 tx=0x7fbdd00033a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c9312da052c73d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:02.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.029+0000 7fbdd6f08640 1 -- 172.21.15.38:0/1400231130 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdd000b480 con 0x5651413266c0 2024-09-21T03:44:02.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.029+0000 7fbdd6f08640 1 -- 172.21.15.38:0/1400231130 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbdd0068070 con 0x5651413266c0 2024-09-21T03:44:02.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.029+0000 7fbdd6f08640 1 -- 172.21.15.38:0/1400231130 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdd0055a40 con 0x5651413266c0 2024-09-21T03:44:02.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.030+0000 7fbddf71ef40 1 -- 172.21.15.38:0/1400231130 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5651413266c0 msgr2=0x565141326ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:02.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.030+0000 7fbddf71ef40 1 --2- 172.21.15.38:0/1400231130 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5651413266c0 0x565141326ad0 secure :-1 s=READY pgs=2142 cs=0 l=1 rev1=1 crypto rx=0x7fbdd0002000 tx=0x7fbdd00033a0 comp rx=0 tx=0).stop 2024-09-21T03:44:02.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.030+0000 7fbddf71ef40 1 -- 172.21.15.38:0/1400231130 shutdown_connections 2024-09-21T03:44:02.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.030+0000 7fbddf71ef40 1 --2- 172.21.15.38:0/1400231130 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5651413266c0 0x565141326ad0 unknown :-1 s=CLOSED pgs=2142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:02.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.030+0000 7fbddf71ef40 1 --2- 172.21.15.38:0/1400231130 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5651412949e0 0x56514126aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:02.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.030+0000 7fbddf71ef40 1 --2- 172.21.15.38:0/1400231130 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x565141339a90 0x565141295eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:02.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.030+0000 7fbddf71ef40 1 -- 172.21.15.38:0/1400231130 >> 172.21.15.38:0/1400231130 conn(0x56514118aac0 msgr2=0x5651412951e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:02.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.030+0000 7fbddf71ef40 1 -- 172.21.15.38:0/1400231130 shutdown_connections 2024-09-21T03:44:02.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.030+0000 7fbddf71ef40 1 -- 172.21.15.38:0/1400231130 wait complete. 2024-09-21T03:44:02.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.031+0000 7fbddf71ef40 1 Processor -- start 2024-09-21T03:44:02.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.031+0000 7fbddf71ef40 1 -- start start 2024-09-21T03:44:02.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.032+0000 7fbddf71ef40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5651412949e0 0x56514126aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:02.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.032+0000 7fbddf71ef40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5651413266c0 0x56514126b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:02.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.032+0000 7fbddf71ef40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x565141339a90 0x56514133b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:02.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.032+0000 7fbddf71ef40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5651413395a0 con 0x5651413266c0 2024-09-21T03:44:02.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.032+0000 7fbddf71ef40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x565141339350 con 0x5651412949e0 2024-09-21T03:44:02.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.032+0000 7fbddf71ef40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x565141338d60 con 0x565141339a90 2024-09-21T03:44:02.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.033+0000 7fbdd7f0a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5651413266c0 0x56514126b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:02.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.033+0000 7fbdd7f0a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5651413266c0 0x56514126b400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50838/0 (socket says 172.21.15.38:50838) 2024-09-21T03:44:02.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.033+0000 7fbdd7f0a640 1 -- 172.21.15.38:0/758359904 learned_addr learned my addr 172.21.15.38:0/758359904 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:44:02.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.033+0000 7fbdd870b640 1 --2- 172.21.15.38:0/758359904 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5651412949e0 0x56514126aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:02.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.033+0000 7fbdd8f0c640 1 --2- 172.21.15.38:0/758359904 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x565141339a90 0x56514133b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:02.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.033+0000 7fbdd7f0a640 1 -- 172.21.15.38:0/758359904 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x565141339a90 msgr2=0x56514133b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:02.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.033+0000 7fbdd7f0a640 1 --2- 172.21.15.38:0/758359904 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x565141339a90 0x56514133b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:02.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.033+0000 7fbdd7f0a640 1 -- 172.21.15.38:0/758359904 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5651412949e0 msgr2=0x56514126aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:02.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.033+0000 7fbdd7f0a640 1 --2- 172.21.15.38:0/758359904 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5651412949e0 0x56514126aec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:02.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.033+0000 7fbdd7f0a640 1 -- 172.21.15.38:0/758359904 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565141322390 con 0x5651413266c0 2024-09-21T03:44:02.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.033+0000 7fbdd7f0a640 1 --2- 172.21.15.38:0/758359904 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5651413266c0 0x56514126b400 secure :-1 s=READY pgs=2143 cs=0 l=1 rev1=1 crypto rx=0x7fbdcc00a6e0 tx=0x7fbdcc00aba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:02.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.033+0000 7fbdd5f06640 1 -- 172.21.15.38:0/758359904 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdcc01a020 con 0x5651413266c0 2024-09-21T03:44:02.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.033+0000 7fbdd5f06640 1 -- 172.21.15.38:0/758359904 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbdcc009db0 con 0x5651413266c0 2024-09-21T03:44:02.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.033+0000 7fbddf71ef40 1 -- 172.21.15.38:0/758359904 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x565141321ef0 con 0x5651413266c0 2024-09-21T03:44:02.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.033+0000 7fbdd5f06640 1 -- 172.21.15.38:0/758359904 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdcc016690 con 0x5651413266c0 2024-09-21T03:44:02.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.033+0000 7fbddf71ef40 1 -- 172.21.15.38:0/758359904 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651412b2ca0 con 0x5651413266c0 2024-09-21T03:44:02.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.035+0000 7fbdd5f06640 1 -- 172.21.15.38:0/758359904 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fbdcc008420 con 0x5651413266c0 2024-09-21T03:44:02.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.035+0000 7fbdd5f06640 1 --2- 172.21.15.38:0/758359904 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fbdc40775a0 0x7fbdc4079ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:02.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.036+0000 7fbdd5f06640 1 -- 172.21.15.38:0/758359904 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 9314+0+0 (secure 0 0 0) 0x7fbdcc09b760 con 0x5651413266c0 2024-09-21T03:44:02.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.036+0000 7fbdd870b640 1 --2- 172.21.15.38:0/758359904 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fbdc40775a0 0x7fbdc4079ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:02.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.036+0000 7fbddf71ef40 1 --2- 172.21.15.38:0/758359904 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x56514133bc50 0x5651413e51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:02.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.036+0000 7fbddf71ef40 1 -- 172.21.15.38:0/758359904 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:1 13.10 13:0d82a743:::600.00000000:head [omap-set-header in=80b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x565141321890 con 0x56514133bc50 2024-09-21T03:44:02.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.036+0000 7fbdd8f0c640 1 --2- 172.21.15.38:0/758359904 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x56514133bc50 0x5651413e51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:02.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.036+0000 7fbdd870b640 1 --2- 172.21.15.38:0/758359904 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fbdc40775a0 0x7fbdc4079ab0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fbdc80015e0 tx=0x7fbdc8000f80 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:02.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.036+0000 7fbdd8f0c640 1 --2- 172.21.15.38:0/758359904 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x56514133bc50 0x5651413e51e0 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:02.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.038+0000 7fbdd8f0c640 1 -- 172.21.15.38:0/758359904 <== osd.7 v2:172.21.15.195:6818/1606106342 1 ==== osd_op_reply(1 600.00000000 [omap-set-header] v131'1096 uv1096 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fbdd000b040 con 0x56514133bc50 2024-09-21T03:44:02.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.038+0000 7fbddf71ef40 1 -- 172.21.15.38:0/758359904 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x56514133bc50 msgr2=0x5651413e51e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:02.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.038+0000 7fbddf71ef40 1 --2- 172.21.15.38:0/758359904 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x56514133bc50 0x5651413e51e0 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:02.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.038+0000 7fbddf71ef40 1 -- 172.21.15.38:0/758359904 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fbdc40775a0 msgr2=0x7fbdc4079ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:02.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.038+0000 7fbddf71ef40 1 --2- 172.21.15.38:0/758359904 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fbdc40775a0 0x7fbdc4079ab0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fbdc80015e0 tx=0x7fbdc8000f80 comp rx=0 tx=0).stop 2024-09-21T03:44:02.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.038+0000 7fbddf71ef40 1 -- 172.21.15.38:0/758359904 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5651413266c0 msgr2=0x56514126b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:02.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.038+0000 7fbddf71ef40 1 --2- 172.21.15.38:0/758359904 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5651413266c0 0x56514126b400 secure :-1 s=READY pgs=2143 cs=0 l=1 rev1=1 crypto rx=0x7fbdcc00a6e0 tx=0x7fbdcc00aba0 comp rx=0 tx=0).stop 2024-09-21T03:44:02.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.038+0000 7fbdd8f0c640 1 -- 172.21.15.38:0/758359904 reap_dead start 2024-09-21T03:44:02.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.038+0000 7fbddf71ef40 1 -- 172.21.15.38:0/758359904 shutdown_connections 2024-09-21T03:44:02.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.038+0000 7fbddf71ef40 1 -- 172.21.15.38:0/758359904 >> 172.21.15.38:0/758359904 conn(0x56514118aac0 msgr2=0x565141326ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:02.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.038+0000 7fbddf71ef40 1 -- 172.21.15.38:0/758359904 shutdown_connections 2024-09-21T03:44:02.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.038+0000 7fbddf71ef40 1 -- 172.21.15.38:0/758359904 wait complete. 2024-09-21T03:44:02.043 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-21T03:44:02.145 DEBUG:teuthology.orchestra.run.smithi038:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T03:44:02.148 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T03:44:02.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.162+0000 7fbd03b13640 1 -- 172.21.15.38:0/2602605189 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbcfc0b2ca0 msgr2=0x7fbcfc0b30b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:02.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.162+0000 7fbd03b13640 1 --2- 172.21.15.38:0/2602605189 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbcfc0b2ca0 0x7fbcfc0b30b0 secure :-1 s=READY pgs=2145 cs=0 l=1 rev1=1 crypto rx=0x7fbcf4005fd0 tx=0x7fbcf402a8b0 comp rx=0 tx=0).stop 2024-09-21T03:44:02.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.162+0000 7fbd03b13640 1 -- 172.21.15.38:0/2602605189 shutdown_connections 2024-09-21T03:44:02.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.162+0000 7fbd03b13640 1 --2- 172.21.15.38:0/2602605189 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbcfc0b2ca0 0x7fbcfc0b30b0 unknown :-1 s=CLOSED pgs=2145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:02.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.162+0000 7fbd03b13640 1 --2- 172.21.15.38:0/2602605189 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbcfc15a300 0x7fbcfc06f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:02.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.162+0000 7fbd03b13640 1 -- 172.21.15.38:0/2602605189 >> 172.21.15.38:0/2602605189 conn(0x7fbcfc0cdca0 msgr2=0x7fbcfc0ce0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:02.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.162+0000 7fbd03b13640 1 -- 172.21.15.38:0/2602605189 shutdown_connections 2024-09-21T03:44:02.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.162+0000 7fbd03b13640 1 -- 172.21.15.38:0/2602605189 wait complete. 2024-09-21T03:44:02.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.165+0000 7fbd03b13640 1 Processor -- start 2024-09-21T03:44:02.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.165+0000 7fbd03b13640 1 -- start start 2024-09-21T03:44:02.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.167+0000 7fbd03b13640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbcfc0ae970 0x7fbcfc0c7180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:02.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.167+0000 7fbd03b13640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbcfc0b2ca0 0x7fbcfc1f6b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:02.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.167+0000 7fbd03b13640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbcfc15a300 0x7fbcfc1f7040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:02.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.167+0000 7fbd03b13640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fbcfc160530 con 0x7fbcfc15a300 2024-09-21T03:44:02.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.167+0000 7fbd03b13640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fbcfc1603b0 con 0x7fbcfc0ae970 2024-09-21T03:44:02.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.167+0000 7fbd03b13640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fbcfc1606b0 con 0x7fbcfc0b2ca0 2024-09-21T03:44:02.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.167+0000 7fbd020b0640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbcfc15a300 0x7fbcfc1f7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:02.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.167+0000 7fbd020b0640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbcfc15a300 0x7fbcfc1f7040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50864/0 (socket says 172.21.15.38:50864) 2024-09-21T03:44:02.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.167+0000 7fbd020b0640 1 -- 172.21.15.38:0/100353021 learned_addr learned my addr 172.21.15.38:0/100353021 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:44:02.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.167+0000 7fbd010ae640 1 --2- 172.21.15.38:0/100353021 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbcfc0b2ca0 0x7fbcfc1f6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:02.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.167+0000 7fbd018af640 1 --2- 172.21.15.38:0/100353021 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbcfc0ae970 0x7fbcfc0c7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:02.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.168+0000 7fbd020b0640 1 -- 172.21.15.38:0/100353021 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbcfc0b2ca0 msgr2=0x7fbcfc1f6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:02.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.168+0000 7fbd020b0640 1 --2- 172.21.15.38:0/100353021 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbcfc0b2ca0 0x7fbcfc1f6b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:02.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.168+0000 7fbd020b0640 1 -- 172.21.15.38:0/100353021 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbcfc0ae970 msgr2=0x7fbcfc0c7180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:02.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.168+0000 7fbd020b0640 1 --2- 172.21.15.38:0/100353021 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbcfc0ae970 0x7fbcfc0c7180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:02.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.168+0000 7fbd020b0640 1 -- 172.21.15.38:0/100353021 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcfc156cd0 con 0x7fbcfc15a300 2024-09-21T03:44:02.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.168+0000 7fbd020b0640 1 --2- 172.21.15.38:0/100353021 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbcfc15a300 0x7fbcfc1f7040 secure :-1 s=READY pgs=2146 cs=0 l=1 rev1=1 crypto rx=0x7fbcf4009cd0 tx=0x7fbcf40316d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:02.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.168+0000 7fbcf27fc640 1 -- 172.21.15.38:0/100353021 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcf4031990 con 0x7fbcfc15a300 2024-09-21T03:44:02.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.168+0000 7fbcf27fc640 1 -- 172.21.15.38:0/100353021 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbcf4031c20 con 0x7fbcfc15a300 2024-09-21T03:44:02.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.168+0000 7fbd03b13640 1 -- 172.21.15.38:0/100353021 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcfc1f7580 con 0x7fbcfc15a300 2024-09-21T03:44:02.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.168+0000 7fbcf27fc640 1 -- 172.21.15.38:0/100353021 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcf4033070 con 0x7fbcfc15a300 2024-09-21T03:44:02.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.168+0000 7fbd03b13640 1 -- 172.21.15.38:0/100353021 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcfc14b010 con 0x7fbcfc15a300 2024-09-21T03:44:02.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.169+0000 7fbd03b13640 1 -- 172.21.15.38:0/100353021 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbcfc160c40 con 0x7fbcfc15a300 2024-09-21T03:44:02.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.170+0000 7fbcf27fc640 1 -- 172.21.15.38:0/100353021 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fbcf4033210 con 0x7fbcfc15a300 2024-09-21T03:44:02.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.171+0000 7fbcf27fc640 1 --2- 172.21.15.38:0/100353021 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fbce0078bb0 0x7fbce007b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:02.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.171+0000 7fbcf27fc640 1 -- 172.21.15.38:0/100353021 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 9314+0+0 (secure 0 0 0) 0x7fbcf40bcfe0 con 0x7fbcfc15a300 2024-09-21T03:44:02.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.171+0000 7fbd018af640 1 --2- 172.21.15.38:0/100353021 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fbce0078bb0 0x7fbce007b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:02.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.172+0000 7fbd018af640 1 --2- 172.21.15.38:0/100353021 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fbce0078bb0 0x7fbce007b0c0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fbcf8009b30 tx=0x7fbcf8005f90 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:02.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.175+0000 7fbcf27fc640 1 -- 172.21.15.38:0/100353021 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fbcf403f030 con 0x7fbcfc15a300 2024-09-21T03:44:02.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.405+0000 7fbd03b13640 1 -- 172.21.15.38:0/100353021 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7fbcfc0ce150 con 0x7fbcfc15a300 2024-09-21T03:44:02.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.823+0000 7fbcf27fc640 1 -- 172.21.15.38:0/100353021 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v300) ==== 162+0+0 (secure 0 0 0) 0x7fbcf4083f50 con 0x7fbcfc15a300 2024-09-21T03:44:02.825 INFO:teuthology.orchestra.run.smithi038.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-21T03:44:02.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.827+0000 7fbd03b13640 1 -- 172.21.15.38:0/100353021 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fbce0078bb0 msgr2=0x7fbce007b0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:02.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.827+0000 7fbd03b13640 1 --2- 172.21.15.38:0/100353021 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fbce0078bb0 0x7fbce007b0c0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fbcf8009b30 tx=0x7fbcf8005f90 comp rx=0 tx=0).stop 2024-09-21T03:44:02.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.827+0000 7fbd03b13640 1 -- 172.21.15.38:0/100353021 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbcfc15a300 msgr2=0x7fbcfc1f7040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:02.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.827+0000 7fbd03b13640 1 --2- 172.21.15.38:0/100353021 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbcfc15a300 0x7fbcfc1f7040 secure :-1 s=READY pgs=2146 cs=0 l=1 rev1=1 crypto rx=0x7fbcf4009cd0 tx=0x7fbcf40316d0 comp rx=0 tx=0).stop 2024-09-21T03:44:02.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.827+0000 7fbd03b13640 1 -- 172.21.15.38:0/100353021 shutdown_connections 2024-09-21T03:44:02.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.827+0000 7fbd03b13640 1 --2- 172.21.15.38:0/100353021 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbcfc15a300 0x7fbcfc1f7040 unknown :-1 s=CLOSED pgs=2146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:02.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.827+0000 7fbd03b13640 1 --2- 172.21.15.38:0/100353021 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fbce0078bb0 0x7fbce007b0c0 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:02.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.827+0000 7fbd03b13640 1 --2- 172.21.15.38:0/100353021 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbcfc0b2ca0 0x7fbcfc1f6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:02.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.827+0000 7fbd03b13640 1 --2- 172.21.15.38:0/100353021 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbcfc0ae970 0x7fbcfc0c7180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:02.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.827+0000 7fbd03b13640 1 -- 172.21.15.38:0/100353021 >> 172.21.15.38:0/100353021 conn(0x7fbcfc0cdca0 msgr2=0x7fbcfc070b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:02.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.827+0000 7fbd03b13640 1 -- 172.21.15.38:0/100353021 shutdown_connections 2024-09-21T03:44:02.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.827+0000 7fbd03b13640 1 -- 172.21.15.38:0/100353021 wait complete. 2024-09-21T03:44:02.843 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:44:02.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.961+0000 7f8d83382640 1 -- 172.21.15.38:0/323189454 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8d7c0cd040 msgr2=0x7f8d7c0d04c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:02.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.961+0000 7f8d83382640 1 --2- 172.21.15.38:0/323189454 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8d7c0cd040 0x7f8d7c0d04c0 secure :-1 s=READY pgs=2148 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c005fd0 tx=0x7f8d6c02aac0 comp rx=0 tx=0).stop 2024-09-21T03:44:02.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.961+0000 7f8d83382640 1 -- 172.21.15.38:0/323189454 shutdown_connections 2024-09-21T03:44:02.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.961+0000 7f8d83382640 1 --2- 172.21.15.38:0/323189454 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8d7c0cd040 0x7f8d7c0d04c0 unknown :-1 s=CLOSED pgs=2148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:02.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.961+0000 7f8d83382640 1 --2- 172.21.15.38:0/323189454 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8d7c1536e0 0x7f8d7c0d17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:02.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.961+0000 7f8d83382640 1 -- 172.21.15.38:0/323189454 >> 172.21.15.38:0/323189454 conn(0x7f8d7c0cef40 msgr2=0x7f8d7c0cf350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:02.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.961+0000 7f8d83382640 1 -- 172.21.15.38:0/323189454 shutdown_connections 2024-09-21T03:44:02.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.962+0000 7f8d83382640 1 -- 172.21.15.38:0/323189454 wait complete. 2024-09-21T03:44:02.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.963+0000 7f8d83382640 1 Processor -- start 2024-09-21T03:44:02.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.963+0000 7f8d83382640 1 -- start start 2024-09-21T03:44:02.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.965+0000 7f8d83382640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8d7c0d1d00 0x7f8d7c158e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:02.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.965+0000 7f8d83382640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8d7c1536e0 0x7f8d7c1f6af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:02.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.965+0000 7f8d83382640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8d7c14f480 0x7f8d7c15a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:02.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.965+0000 7f8d83382640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f8d7c0d0d90 con 0x7f8d7c1536e0 2024-09-21T03:44:02.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.965+0000 7f8d83382640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f8d7c0d0c10 con 0x7f8d7c14f480 2024-09-21T03:44:02.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.965+0000 7f8d83382640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f8d7c0d0f10 con 0x7f8d7c0d1d00 2024-09-21T03:44:02.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.965+0000 7f8d8091d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8d7c1536e0 0x7f8d7c1f6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:02.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.965+0000 7f8d8091d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8d7c1536e0 0x7f8d7c1f6af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:45480/0 (socket says 172.21.15.38:45480) 2024-09-21T03:44:02.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.965+0000 7f8d8091d640 1 -- 172.21.15.38:0/4168764755 learned_addr learned my addr 172.21.15.38:0/4168764755 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:44:02.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.965+0000 7f8d8111e640 1 --2- 172.21.15.38:0/4168764755 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8d7c0d1d00 0x7f8d7c158e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:02.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.965+0000 7f8d8191f640 1 --2- 172.21.15.38:0/4168764755 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8d7c14f480 0x7f8d7c15a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:02.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.965+0000 7f8d8091d640 1 -- 172.21.15.38:0/4168764755 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8d7c0d1d00 msgr2=0x7f8d7c158e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:02.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.965+0000 7f8d8091d640 1 --2- 172.21.15.38:0/4168764755 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8d7c0d1d00 0x7f8d7c158e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:02.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.965+0000 7f8d8091d640 1 -- 172.21.15.38:0/4168764755 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8d7c14f480 msgr2=0x7f8d7c15a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:02.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.965+0000 7f8d8091d640 1 --2- 172.21.15.38:0/4168764755 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8d7c14f480 0x7f8d7c15a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:02.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.965+0000 7f8d8091d640 1 -- 172.21.15.38:0/4168764755 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d7c1f7030 con 0x7f8d7c1536e0 2024-09-21T03:44:02.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.965+0000 7f8d8191f640 1 --2- 172.21.15.38:0/4168764755 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8d7c14f480 0x7f8d7c15a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:44:02.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.965+0000 7f8d8091d640 1 --2- 172.21.15.38:0/4168764755 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8d7c1536e0 0x7f8d7c1f6af0 secure :-1 s=READY pgs=2149 cs=0 l=1 rev1=1 crypto rx=0x7f8d7000c340 tx=0x7f8d7000c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:02.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.965+0000 7f8d79ffb640 1 -- 172.21.15.38:0/4168764755 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d70017070 con 0x7f8d7c1536e0 2024-09-21T03:44:02.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.965+0000 7f8d79ffb640 1 -- 172.21.15.38:0/4168764755 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8d700035f0 con 0x7f8d7c1536e0 2024-09-21T03:44:02.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.965+0000 7f8d83382640 1 -- 172.21.15.38:0/4168764755 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d7c0b71b0 con 0x7f8d7c1536e0 2024-09-21T03:44:02.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.965+0000 7f8d79ffb640 1 -- 172.21.15.38:0/4168764755 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d70003a40 con 0x7f8d7c1536e0 2024-09-21T03:44:02.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.966+0000 7f8d83382640 1 -- 172.21.15.38:0/4168764755 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d7c0b7770 con 0x7f8d7c1536e0 2024-09-21T03:44:02.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.967+0000 7f8d83382640 1 -- 172.21.15.38:0/4168764755 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d7c0d04c0 con 0x7f8d7c1536e0 2024-09-21T03:44:02.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.967+0000 7f8d79ffb640 1 -- 172.21.15.38:0/4168764755 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f8d7001a070 con 0x7f8d7c1536e0 2024-09-21T03:44:02.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.968+0000 7f8d79ffb640 1 --2- 172.21.15.38:0/4168764755 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8d68078bc0 0x7f8d6807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:02.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.968+0000 7f8d79ffb640 1 -- 172.21.15.38:0/4168764755 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 9314+0+0 (secure 0 0 0) 0x7f8d7009ad60 con 0x7f8d7c1536e0 2024-09-21T03:44:02.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.968+0000 7f8d8111e640 1 --2- 172.21.15.38:0/4168764755 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8d68078bc0 0x7f8d6807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:02.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.969+0000 7f8d8111e640 1 --2- 172.21.15.38:0/4168764755 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8d68078bc0 0x7f8d6807b0d0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f8d7400bb00 tx=0x7f8d74005e30 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:02.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:02.972+0000 7f8d79ffb640 1 -- 172.21.15.38:0/4168764755 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f8d700623f0 con 0x7f8d7c1536e0 2024-09-21T03:44:03.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:03.198+0000 7f8d83382640 1 -- 172.21.15.38:0/4168764755 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8d7c15b590 con 0x7f8d7c1536e0 2024-09-21T03:44:03.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:03.198+0000 7f8d79ffb640 1 -- 172.21.15.38:0/4168764755 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 301 v301) ==== 95+0+5230 (secure 0 0 0) 0x7f8d70061cf0 con 0x7f8d7c1536e0 2024-09-21T03:44:03.200 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:44:03.201 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":301,"btime":"2024-09-21T03:44:02:827832+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6086,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/965972663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":965972663},{"type":"v1","addr":"172.21.15.195:6836","nonce":965972663}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6700,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/3176122334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":3176122334},{"type":"v1","addr":"172.21.15.195:6837","nonce":3176122334}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":9840,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/3801772046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":3801772046},{"type":"v1","addr":"172.21.15.38:6835","nonce":3801772046}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":273}],"filesystems":[{"mdsmap":{"epoch":301,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:40:02.319282+0000","modified":"2024-09-21T03:44:02.827810+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":131,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10197},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10197":{"gid":10197,"name":"c","rank":0,"incarnation":301,"state":"up:replay","state_seq":1,"addr":"172.21.15.38:6833/3103607164","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":3103607164},{"type":"v1","addr":"172.21.15.38:6833","nonce":3103607164}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-21T03:44:03.201 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 301 2024-09-21T03:44:03.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:03.200+0000 7f8d83382640 1 -- 172.21.15.38:0/4168764755 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8d68078bc0 msgr2=0x7f8d6807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:03.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:03.200+0000 7f8d83382640 1 --2- 172.21.15.38:0/4168764755 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8d68078bc0 0x7f8d6807b0d0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f8d7400bb00 tx=0x7f8d74005e30 comp rx=0 tx=0).stop 2024-09-21T03:44:03.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:03.200+0000 7f8d83382640 1 -- 172.21.15.38:0/4168764755 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8d7c1536e0 msgr2=0x7f8d7c1f6af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:03.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:03.200+0000 7f8d83382640 1 --2- 172.21.15.38:0/4168764755 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8d7c1536e0 0x7f8d7c1f6af0 secure :-1 s=READY pgs=2149 cs=0 l=1 rev1=1 crypto rx=0x7f8d7000c340 tx=0x7f8d7000c800 comp rx=0 tx=0).stop 2024-09-21T03:44:03.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:03.200+0000 7f8d83382640 1 -- 172.21.15.38:0/4168764755 shutdown_connections 2024-09-21T03:44:03.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:03.200+0000 7f8d83382640 1 --2- 172.21.15.38:0/4168764755 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8d7c14f480 0x7f8d7c15a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:03.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:03.200+0000 7f8d83382640 1 --2- 172.21.15.38:0/4168764755 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8d7c1536e0 0x7f8d7c1f6af0 unknown :-1 s=CLOSED pgs=2149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:03.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:03.200+0000 7f8d83382640 1 --2- 172.21.15.38:0/4168764755 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8d68078bc0 0x7f8d6807b0d0 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:03.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:03.200+0000 7f8d83382640 1 --2- 172.21.15.38:0/4168764755 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8d7c0d1d00 0x7f8d7c158e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:03.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:03.200+0000 7f8d83382640 1 -- 172.21.15.38:0/4168764755 >> 172.21.15.38:0/4168764755 conn(0x7f8d7c0cef40 msgr2=0x7f8d7c0d2cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:03.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:03.200+0000 7f8d83382640 1 -- 172.21.15.38:0/4168764755 shutdown_connections 2024-09-21T03:44:03.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:03.200+0000 7f8d83382640 1 -- 172.21.15.38:0/4168764755 wait complete. 2024-09-21T03:44:03.213 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2024-09-21T03:44:05.867 INFO:tasks.ceph.mds.c.smithi038.stderr:2024-09-21T03:44:05.865+0000 7f400d67f640 -1 mds.0.cache.dir(0x600) Corrupt fnode in dirfrag 0x600: Decoder at 'void fnode_t::decode(ceph::buffer::v15_2_0::list::const_iterator&)' v=3 cannot decode v=100 minimal_decoder=101: Malformed input [buffer:3] 2024-09-21T03:44:05.868 INFO:tasks.ceph.mds.c.smithi038.stderr:2024-09-21T03:44:05.865+0000 7f400d67f640 -1 mds.0.damage notify_dirfrag Damage to fragment * of ino 0x600 is fatal because it is a system directory for this rank 2024-09-21T03:44:05.868 INFO:tasks.ceph.mon.a.smithi038.stderr:2024-09-21T03:44:05.866+0000 7f3f45713640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T03:44:05.868 INFO:tasks.ceph.mon.a.smithi038.stderr:2024-09-21T03:44:05.866+0000 7f3f45713640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-09-21T03:44:06.679 INFO:tasks.ceph.mds.c.smithi038.stderr:2024-09-21T03:44:06.677+0000 7f4011687640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T03:44:08.214 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:44:08.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:08.344+0000 7fb58d313640 1 -- 172.21.15.38:0/2020136091 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb570005350 msgr2=0x7fb57000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:08.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:08.344+0000 7fb58d313640 1 --2- 172.21.15.38:0/2020136091 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb570005350 0x7fb57000b9c0 secure :-1 s=READY pgs=2151 cs=0 l=1 rev1=1 crypto rx=0x7fb580001d90 tx=0x7fb570005830 comp rx=0 tx=0).stop 2024-09-21T03:44:08.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:08.345+0000 7fb58d313640 1 -- 172.21.15.38:0/2020136091 shutdown_connections 2024-09-21T03:44:08.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:08.345+0000 7fb58d313640 1 --2- 172.21.15.38:0/2020136091 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb570005350 0x7fb57000b9c0 unknown :-1 s=CLOSED pgs=2151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:08.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:08.345+0000 7fb58d313640 1 -- 172.21.15.38:0/2020136091 >> 172.21.15.38:0/2020136091 conn(0x7fb5880b6b20 msgr2=0x7fb5880d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:08.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:08.345+0000 7fb58d313640 1 -- 172.21.15.38:0/2020136091 shutdown_connections 2024-09-21T03:44:08.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:08.345+0000 7fb58d313640 1 -- 172.21.15.38:0/2020136091 wait complete. 2024-09-21T03:44:08.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:08.347+0000 7fb58d313640 1 Processor -- start 2024-09-21T03:44:08.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:08.347+0000 7fb58d313640 1 -- start start 2024-09-21T03:44:08.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:08.348+0000 7fb58d313640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb570005350 0x7fb5881f8df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:08.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:08.348+0000 7fb58d313640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb5880aae30 0x7fb5880b2470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:08.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:08.348+0000 7fb58d313640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb5880b29b0 0x7fb5880ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:08.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:08.348+0000 7fb58d313640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fb588162810 con 0x7fb5880aae30 2024-09-21T03:44:08.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:08.348+0000 7fb58d313640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fb588162690 con 0x7fb570005350 2024-09-21T03:44:08.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:08.348+0000 7fb58d313640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fb588162990 con 0x7fb5880b29b0 2024-09-21T03:44:08.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:08.348+0000 7fb5867fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb5880aae30 0x7fb5880b2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:08.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:08.348+0000 7fb5867fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb5880aae30 0x7fb5880b2470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:45496/0 (socket says 172.21.15.38:45496) 2024-09-21T03:44:08.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:08.348+0000 7fb5867fc640 1 -- 172.21.15.38:0/3497462118 learned_addr learned my addr 172.21.15.38:0/3497462118 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:44:08.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:08.348+0000 7fb586ffd640 1 --2- 172.21.15.38:0/3497462118 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb570005350 0x7fb5881f8df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:08.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:08.348+0000 7fb5877fe640 1 --2- 172.21.15.38:0/3497462118 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb5880b29b0 0x7fb5880ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:08.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:08.348+0000 7fb5867fc640 1 -- 172.21.15.38:0/3497462118 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb5880b29b0 msgr2=0x7fb5880ceb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:08.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:08.348+0000 7fb5867fc640 1 --2- 172.21.15.38:0/3497462118 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb5880b29b0 0x7fb5880ceb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:08.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:08.348+0000 7fb5867fc640 1 -- 172.21.15.38:0/3497462118 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb570005350 msgr2=0x7fb5881f8df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:08.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:08.348+0000 7fb5867fc640 1 --2- 172.21.15.38:0/3497462118 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb570005350 0x7fb5881f8df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:08.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:08.348+0000 7fb5867fc640 1 -- 172.21.15.38:0/3497462118 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5880cd340 con 0x7fb5880aae30 2024-09-21T03:44:08.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:08.348+0000 7fb5877fe640 1 --2- 172.21.15.38:0/3497462118 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb5880b29b0 0x7fb5880ceb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:44:08.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:08.349+0000 7fb5867fc640 1 --2- 172.21.15.38:0/3497462118 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb5880aae30 0x7fb5880b2470 secure :-1 s=READY pgs=2152 cs=0 l=1 rev1=1 crypto rx=0x7fb57c011ab0 tx=0x7fb57c011eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:08.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:08.349+0000 7fb56f7fe640 1 -- 172.21.15.38:0/3497462118 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb57c01e070 con 0x7fb5880aae30 2024-09-21T03:44:08.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:08.349+0000 7fb56f7fe640 1 -- 172.21.15.38:0/3497462118 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb57c004460 con 0x7fb5880aae30 2024-09-21T03:44:08.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:08.349+0000 7fb58d313640 1 -- 172.21.15.38:0/3497462118 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5880cd620 con 0x7fb5880aae30 2024-09-21T03:44:08.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:08.349+0000 7fb56f7fe640 1 -- 172.21.15.38:0/3497462118 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb57c00fc00 con 0x7fb5880aae30 2024-09-21T03:44:08.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:08.349+0000 7fb58d313640 1 -- 172.21.15.38:0/3497462118 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5880cdbe0 con 0x7fb5880aae30 2024-09-21T03:44:08.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:08.350+0000 7fb58d313640 1 -- 172.21.15.38:0/3497462118 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5880d2850 con 0x7fb5880aae30 2024-09-21T03:44:08.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:08.351+0000 7fb56f7fe640 1 -- 172.21.15.38:0/3497462118 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb57c021070 con 0x7fb5880aae30 2024-09-21T03:44:08.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:08.352+0000 7fb56f7fe640 1 --2- 172.21.15.38:0/3497462118 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb574078b90 0x7fb57407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:08.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:08.352+0000 7fb56f7fe640 1 -- 172.21.15.38:0/3497462118 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 9400+0+0 (secure 0 0 0) 0x7fb57c09cd60 con 0x7fb5880aae30 2024-09-21T03:44:08.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:08.352+0000 7fb586ffd640 1 --2- 172.21.15.38:0/3497462118 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb574078b90 0x7fb57407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:08.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:08.353+0000 7fb586ffd640 1 --2- 172.21.15.38:0/3497462118 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb574078b90 0x7fb57407b0a0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fb578001410 tx=0x7fb5780068d0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:08.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:08.355+0000 7fb56f7fe640 1 -- 172.21.15.38:0/3497462118 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fb57c063d40 con 0x7fb5880aae30 2024-09-21T03:44:08.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:08.579+0000 7fb58d313640 1 -- 172.21.15.38:0/3497462118 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb58815ef10 con 0x7fb5880aae30 2024-09-21T03:44:08.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:08.580+0000 7fb56f7fe640 1 -- 172.21.15.38:0/3497462118 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 305 v305) ==== 95+0+4392 (secure 0 0 0) 0x7fb57c068770 con 0x7fb5880aae30 2024-09-21T03:44:08.582 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:44:08.582 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":305,"btime":"2024-09-21T03:44:05:867394+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6086,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/965972663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":965972663},{"type":"v1","addr":"172.21.15.195:6836","nonce":965972663}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6700,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/3176122334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":3176122334},{"type":"v1","addr":"172.21.15.195:6837","nonce":3176122334}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":9840,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/3801772046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":3801772046},{"type":"v1","addr":"172.21.15.38:6835","nonce":3801772046}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":273}],"filesystems":[{"mdsmap":{"epoch":305,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:40:02.319282+0000","modified":"2024-09-21T03:44:05.867389+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":132,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-21T03:44:08.582 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 305 2024-09-21T03:44:08.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:08.582+0000 7fb58d313640 1 -- 172.21.15.38:0/3497462118 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb574078b90 msgr2=0x7fb57407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:08.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:08.582+0000 7fb58d313640 1 --2- 172.21.15.38:0/3497462118 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb574078b90 0x7fb57407b0a0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fb578001410 tx=0x7fb5780068d0 comp rx=0 tx=0).stop 2024-09-21T03:44:08.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:08.582+0000 7fb58d313640 1 -- 172.21.15.38:0/3497462118 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb5880aae30 msgr2=0x7fb5880b2470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:08.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:08.582+0000 7fb58d313640 1 --2- 172.21.15.38:0/3497462118 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb5880aae30 0x7fb5880b2470 secure :-1 s=READY pgs=2152 cs=0 l=1 rev1=1 crypto rx=0x7fb57c011ab0 tx=0x7fb57c011eb0 comp rx=0 tx=0).stop 2024-09-21T03:44:08.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:08.582+0000 7fb58d313640 1 -- 172.21.15.38:0/3497462118 shutdown_connections 2024-09-21T03:44:08.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:08.582+0000 7fb58d313640 1 --2- 172.21.15.38:0/3497462118 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb574078b90 0x7fb57407b0a0 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:08.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:08.582+0000 7fb58d313640 1 --2- 172.21.15.38:0/3497462118 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb5880b29b0 0x7fb5880ceb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:08.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:08.582+0000 7fb58d313640 1 --2- 172.21.15.38:0/3497462118 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb5880aae30 0x7fb5880b2470 unknown :-1 s=CLOSED pgs=2152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:08.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:08.582+0000 7fb58d313640 1 --2- 172.21.15.38:0/3497462118 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb570005350 0x7fb5881f8df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:08.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:08.582+0000 7fb58d313640 1 -- 172.21.15.38:0/3497462118 >> 172.21.15.38:0/3497462118 conn(0x7fb5880b6b20 msgr2=0x7fb58815d9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:08.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:08.582+0000 7fb58d313640 1 -- 172.21.15.38:0/3497462118 shutdown_connections 2024-09-21T03:44:08.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:08.582+0000 7fb58d313640 1 -- 172.21.15.38:0/3497462118 wait complete. 2024-09-21T03:44:08.596 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-09-21T03:44:08.596 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:44:08.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:08.711+0000 7fb4b72d5640 1 -- 172.21.15.38:0/2585988784 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb4b0162f00 msgr2=0x7fb4b00d24d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:08.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:08.711+0000 7fb4b72d5640 1 --2- 172.21.15.38:0/2585988784 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb4b0162f00 0x7fb4b00d24d0 secure :-1 s=READY pgs=2153 cs=0 l=1 rev1=1 crypto rx=0x7fb4b00b7460 tx=0x7fb4ac02a560 comp rx=0 tx=0).stop 2024-09-21T03:44:08.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:08.711+0000 7fb4b72d5640 1 -- 172.21.15.38:0/2585988784 shutdown_connections 2024-09-21T03:44:08.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:08.711+0000 7fb4b72d5640 1 --2- 172.21.15.38:0/2585988784 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb4b00b2e40 0x7fb4b00b3250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:08.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:08.711+0000 7fb4b72d5640 1 --2- 172.21.15.38:0/2585988784 >> v2:172.21.15.195:3300/0 conn(0x7fb4b0162280 0x7fb4b0162690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:08.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:08.711+0000 7fb4b72d5640 1 --2- 172.21.15.38:0/2585988784 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb4b0162f00 0x7fb4b00d24d0 unknown :-1 s=CLOSED pgs=2153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:08.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:08.711+0000 7fb4b72d5640 1 -- 172.21.15.38:0/2585988784 >> 172.21.15.38:0/2585988784 conn(0x7fb4b00b6b20 msgr2=0x7fb4b00d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:08.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:08.711+0000 7fb4b72d5640 1 -- 172.21.15.38:0/2585988784 shutdown_connections 2024-09-21T03:44:08.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:08.711+0000 7fb4b72d5640 1 -- 172.21.15.38:0/2585988784 wait complete. 2024-09-21T03:44:08.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:08.713+0000 7fb4b72d5640 1 Processor -- start 2024-09-21T03:44:08.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:08.713+0000 7fb4b72d5640 1 -- start start 2024-09-21T03:44:08.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:08.714+0000 7fb4b72d5640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb4b00b2e40 0x7fb4b00b2300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:08.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:08.714+0000 7fb4b72d5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb4b0162280 0x7fb4b00b2840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:08.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:08.714+0000 7fb4b72d5640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb4b01f8640 0x7fb4b01f8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:08.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:08.714+0000 7fb4b72d5640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fb4b00b3dd0 con 0x7fb4b0162280 2024-09-21T03:44:08.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:08.714+0000 7fb4b72d5640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fb4b00b3f50 con 0x7fb4b01f8640 2024-09-21T03:44:08.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:08.714+0000 7fb4b72d5640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fb4b00b40d0 con 0x7fb4b00b2e40 2024-09-21T03:44:08.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:08.714+0000 7fb4b4870640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb4b0162280 0x7fb4b00b2840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:08.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:08.714+0000 7fb4b4870640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb4b0162280 0x7fb4b00b2840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:45506/0 (socket says 172.21.15.38:45506) 2024-09-21T03:44:08.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:08.714+0000 7fb4b4870640 1 -- 172.21.15.38:0/2790581655 learned_addr learned my addr 172.21.15.38:0/2790581655 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:44:08.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:08.715+0000 7fb4b5071640 1 --2- 172.21.15.38:0/2790581655 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb4b00b2e40 0x7fb4b00b2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:08.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:08.715+0000 7fb4b5872640 1 --2- 172.21.15.38:0/2790581655 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb4b01f8640 0x7fb4b01f8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:08.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:08.715+0000 7fb4b4870640 1 -- 172.21.15.38:0/2790581655 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb4b00b2e40 msgr2=0x7fb4b00b2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:08.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:08.715+0000 7fb4b4870640 1 --2- 172.21.15.38:0/2790581655 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb4b00b2e40 0x7fb4b00b2300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:08.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:08.715+0000 7fb4b4870640 1 -- 172.21.15.38:0/2790581655 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb4b01f8640 msgr2=0x7fb4b01f8a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:08.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:08.715+0000 7fb4b4870640 1 --2- 172.21.15.38:0/2790581655 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb4b01f8640 0x7fb4b01f8a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:08.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:08.715+0000 7fb4b4870640 1 -- 172.21.15.38:0/2790581655 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4b014c430 con 0x7fb4b0162280 2024-09-21T03:44:08.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:08.715+0000 7fb4b5872640 1 --2- 172.21.15.38:0/2790581655 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb4b01f8640 0x7fb4b01f8a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:44:08.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:08.715+0000 7fb4b4870640 1 --2- 172.21.15.38:0/2790581655 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb4b0162280 0x7fb4b00b2840 secure :-1 s=READY pgs=2154 cs=0 l=1 rev1=1 crypto rx=0x7fb4a000d770 tx=0x7fb4a000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:08.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:08.715+0000 7fb4a5ffb640 1 -- 172.21.15.38:0/2790581655 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4a0017070 con 0x7fb4b0162280 2024-09-21T03:44:08.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:08.715+0000 7fb4a5ffb640 1 -- 172.21.15.38:0/2790581655 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb4a0009d20 con 0x7fb4b0162280 2024-09-21T03:44:08.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:08.715+0000 7fb4a5ffb640 1 -- 172.21.15.38:0/2790581655 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4a00106c0 con 0x7fb4b0162280 2024-09-21T03:44:08.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:08.715+0000 7fb4b72d5640 1 -- 172.21.15.38:0/2790581655 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4b015b190 con 0x7fb4b0162280 2024-09-21T03:44:08.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:08.715+0000 7fb4b72d5640 1 -- 172.21.15.38:0/2790581655 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4b015b750 con 0x7fb4b0162280 2024-09-21T03:44:08.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:08.716+0000 7fb4b72d5640 1 -- 172.21.15.38:0/2790581655 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4b0162f00 con 0x7fb4b0162280 2024-09-21T03:44:08.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:08.717+0000 7fb4a5ffb640 1 -- 172.21.15.38:0/2790581655 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb4a001a070 con 0x7fb4b0162280 2024-09-21T03:44:08.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:08.718+0000 7fb4a5ffb640 1 --2- 172.21.15.38:0/2790581655 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb494078bc0 0x7fb49407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:08.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:08.718+0000 7fb4a5ffb640 1 -- 172.21.15.38:0/2790581655 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 9400+0+0 (secure 0 0 0) 0x7fb4a009bd80 con 0x7fb4b0162280 2024-09-21T03:44:08.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:08.718+0000 7fb4b5071640 1 --2- 172.21.15.38:0/2790581655 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb494078bc0 0x7fb49407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:08.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:08.718+0000 7fb4b5071640 1 --2- 172.21.15.38:0/2790581655 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb494078bc0 0x7fb49407b0d0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fb4ac033000 tx=0x7fb4ac004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:08.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:08.722+0000 7fb4a5ffb640 1 -- 172.21.15.38:0/2790581655 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fb4a00632a0 con 0x7fb4b0162280 2024-09-21T03:44:08.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:08.947+0000 7fb4b72d5640 1 -- 172.21.15.38:0/2790581655 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb4b014c1c0 con 0x7fb4b0162280 2024-09-21T03:44:08.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:08.947+0000 7fb4a5ffb640 1 -- 172.21.15.38:0/2790581655 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 305 v305) ==== 95+0+4392 (secure 0 0 0) 0x7fb4a0062ba0 con 0x7fb4b0162280 2024-09-21T03:44:08.949 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:44:08.949 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":305,"btime":"2024-09-21T03:44:05:867394+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6086,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/965972663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":965972663},{"type":"v1","addr":"172.21.15.195:6836","nonce":965972663}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6700,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/3176122334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":3176122334},{"type":"v1","addr":"172.21.15.195:6837","nonce":3176122334}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":9840,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/3801772046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":3801772046},{"type":"v1","addr":"172.21.15.38:6835","nonce":3801772046}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":273}],"filesystems":[{"mdsmap":{"epoch":305,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:40:02.319282+0000","modified":"2024-09-21T03:44:05.867389+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":132,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-21T03:44:08.950 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 305 2024-09-21T03:44:08.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:08.949+0000 7fb4b72d5640 1 -- 172.21.15.38:0/2790581655 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb494078bc0 msgr2=0x7fb49407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:08.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:08.949+0000 7fb4b72d5640 1 --2- 172.21.15.38:0/2790581655 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb494078bc0 0x7fb49407b0d0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fb4ac033000 tx=0x7fb4ac004730 comp rx=0 tx=0).stop 2024-09-21T03:44:08.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:08.949+0000 7fb4b72d5640 1 -- 172.21.15.38:0/2790581655 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb4b0162280 msgr2=0x7fb4b00b2840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:08.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:08.949+0000 7fb4b72d5640 1 --2- 172.21.15.38:0/2790581655 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb4b0162280 0x7fb4b00b2840 secure :-1 s=READY pgs=2154 cs=0 l=1 rev1=1 crypto rx=0x7fb4a000d770 tx=0x7fb4a000dc30 comp rx=0 tx=0).stop 2024-09-21T03:44:08.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:08.949+0000 7fb4b72d5640 1 -- 172.21.15.38:0/2790581655 shutdown_connections 2024-09-21T03:44:08.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:08.949+0000 7fb4b72d5640 1 --2- 172.21.15.38:0/2790581655 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb4b01f8640 0x7fb4b01f8a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:08.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:08.949+0000 7fb4b72d5640 1 --2- 172.21.15.38:0/2790581655 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb4b0162280 0x7fb4b00b2840 unknown :-1 s=CLOSED pgs=2154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:08.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:08.949+0000 7fb4b72d5640 1 --2- 172.21.15.38:0/2790581655 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb494078bc0 0x7fb49407b0d0 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:08.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:08.949+0000 7fb4b72d5640 1 --2- 172.21.15.38:0/2790581655 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb4b00b2e40 0x7fb4b00b2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:08.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:08.949+0000 7fb4b72d5640 1 -- 172.21.15.38:0/2790581655 >> 172.21.15.38:0/2790581655 conn(0x7fb4b00b6b20 msgr2=0x7fb4b015e4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:08.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:08.949+0000 7fb4b72d5640 1 -- 172.21.15.38:0/2790581655 shutdown_connections 2024-09-21T03:44:08.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:08.949+0000 7fb4b72d5640 1 -- 172.21.15.38:0/2790581655 wait complete. 2024-09-21T03:44:08.963 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-21T03:44:08.963 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:44:09.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:09.077+0000 7f8694913640 1 -- 172.21.15.38:0/3745672913 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8690162f00 msgr2=0x7f86900d24d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:09.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:09.077+0000 7f8694913640 1 --2- 172.21.15.38:0/3745672913 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8690162f00 0x7f86900d24d0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f8680005fd0 tx=0x7f868002aa90 comp rx=0 tx=0).stop 2024-09-21T03:44:09.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:09.077+0000 7f8694913640 1 -- 172.21.15.38:0/3745672913 shutdown_connections 2024-09-21T03:44:09.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:09.077+0000 7f8694913640 1 --2- 172.21.15.38:0/3745672913 >> v2:172.21.15.195:3300/0 conn(0x7f86900b2e40 0x7f86900b3250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:09.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:09.077+0000 7f8694913640 1 --2- 172.21.15.38:0/3745672913 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8690162f00 0x7f86900d24d0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:09.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:09.077+0000 7f8694913640 1 -- 172.21.15.38:0/3745672913 >> 172.21.15.38:0/3745672913 conn(0x7f86900b6b20 msgr2=0x7f86900d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:09.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:09.077+0000 7f8694913640 1 -- 172.21.15.38:0/3745672913 shutdown_connections 2024-09-21T03:44:09.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:09.077+0000 7f8694913640 1 -- 172.21.15.38:0/3745672913 wait complete. 2024-09-21T03:44:09.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:09.079+0000 7f8694913640 1 Processor -- start 2024-09-21T03:44:09.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:09.079+0000 7f8694913640 1 -- start start 2024-09-21T03:44:09.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:09.081+0000 7f8694913640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f86900b2e40 0x7f86900ce940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:09.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:09.081+0000 7f8694913640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8690162280 0x7f86900cee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:09.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:09.081+0000 7f8694913640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f869014f400 0x7f869014f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:09.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:09.081+0000 7f8694913640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f869015e4b0 con 0x7f86900b2e40 2024-09-21T03:44:09.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:09.081+0000 7f8694913640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f86900b39a0 con 0x7f869014f400 2024-09-21T03:44:09.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:09.081+0000 7f8694913640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f869015e630 con 0x7f8690162280 2024-09-21T03:44:09.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:09.081+0000 7f868e59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f86900b2e40 0x7f86900ce940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:09.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:09.081+0000 7f868e59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f86900b2e40 0x7f86900ce940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:45510/0 (socket says 172.21.15.38:45510) 2024-09-21T03:44:09.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:09.081+0000 7f868e59c640 1 -- 172.21.15.38:0/814394113 learned_addr learned my addr 172.21.15.38:0/814394113 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:44:09.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:09.081+0000 7f868ed9d640 1 --2- 172.21.15.38:0/814394113 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f869014f400 0x7f869014f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:09.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:09.081+0000 7f868dd9b640 1 --2- 172.21.15.38:0/814394113 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8690162280 0x7f86900cee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:09.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:09.081+0000 7f868e59c640 1 -- 172.21.15.38:0/814394113 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8690162280 msgr2=0x7f86900cee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:09.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:09.081+0000 7f868e59c640 1 --2- 172.21.15.38:0/814394113 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8690162280 0x7f86900cee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:09.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:09.081+0000 7f868e59c640 1 -- 172.21.15.38:0/814394113 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f869014f400 msgr2=0x7f869014f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:09.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:09.081+0000 7f868e59c640 1 --2- 172.21.15.38:0/814394113 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f869014f400 0x7f869014f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:09.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:09.081+0000 7f868e59c640 1 -- 172.21.15.38:0/814394113 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f869014fd50 con 0x7f86900b2e40 2024-09-21T03:44:09.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:09.081+0000 7f868dd9b640 1 --2- 172.21.15.38:0/814394113 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8690162280 0x7f86900cee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:44:09.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:09.081+0000 7f868e59c640 1 --2- 172.21.15.38:0/814394113 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f86900b2e40 0x7f86900ce940 secure :-1 s=READY pgs=2155 cs=0 l=1 rev1=1 crypto rx=0x7f8680000f80 tx=0x7f8680003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:09.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:09.082+0000 7f8686ffd640 1 -- 172.21.15.38:0/814394113 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8680044070 con 0x7f86900b2e40 2024-09-21T03:44:09.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:09.082+0000 7f8686ffd640 1 -- 172.21.15.38:0/814394113 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8680004250 con 0x7f86900b2e40 2024-09-21T03:44:09.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:09.082+0000 7f8694913640 1 -- 172.21.15.38:0/814394113 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86900ac140 con 0x7f86900b2e40 2024-09-21T03:44:09.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:09.082+0000 7f8686ffd640 1 -- 172.21.15.38:0/814394113 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8680004680 con 0x7f86900b2e40 2024-09-21T03:44:09.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:09.082+0000 7f8694913640 1 -- 172.21.15.38:0/814394113 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8690161860 con 0x7f86900b2e40 2024-09-21T03:44:09.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:09.083+0000 7f8694913640 1 -- 172.21.15.38:0/814394113 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8690162f00 con 0x7f86900b2e40 2024-09-21T03:44:09.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:09.084+0000 7f8686ffd640 1 -- 172.21.15.38:0/814394113 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f8680047070 con 0x7f86900b2e40 2024-09-21T03:44:09.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:09.084+0000 7f8686ffd640 1 --2- 172.21.15.38:0/814394113 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8670078b90 0x7f867007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:09.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:09.084+0000 7f8686ffd640 1 -- 172.21.15.38:0/814394113 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 9400+0+0 (secure 0 0 0) 0x7f86800bcc30 con 0x7f86900b2e40 2024-09-21T03:44:09.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:09.084+0000 7f868dd9b640 1 --2- 172.21.15.38:0/814394113 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8670078b90 0x7f867007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:09.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:09.085+0000 7f868dd9b640 1 --2- 172.21.15.38:0/814394113 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8670078b90 0x7f867007b0a0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f869015d790 tx=0x7f8678009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:09.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:09.088+0000 7f8686ffd640 1 -- 172.21.15.38:0/814394113 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f868004e080 con 0x7f86900b2e40 2024-09-21T03:44:09.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:09.314+0000 7f8694913640 1 -- 172.21.15.38:0/814394113 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f86900d2750 con 0x7f86900b2e40 2024-09-21T03:44:09.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:09.314+0000 7f8686ffd640 1 -- 172.21.15.38:0/814394113 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 305 v305) ==== 95+0+4392 (secure 0 0 0) 0x7f8680032030 con 0x7f86900b2e40 2024-09-21T03:44:09.319 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:44:09.319 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":305,"btime":"2024-09-21T03:44:05:867394+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6086,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/965972663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":965972663},{"type":"v1","addr":"172.21.15.195:6836","nonce":965972663}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6700,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/3176122334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":3176122334},{"type":"v1","addr":"172.21.15.195:6837","nonce":3176122334}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":9840,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/3801772046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":3801772046},{"type":"v1","addr":"172.21.15.38:6835","nonce":3801772046}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":273}],"filesystems":[{"mdsmap":{"epoch":305,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:40:02.319282+0000","modified":"2024-09-21T03:44:05.867389+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":132,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-21T03:44:09.320 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 305 2024-09-21T03:44:09.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:09.316+0000 7f8694913640 1 -- 172.21.15.38:0/814394113 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8670078b90 msgr2=0x7f867007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:09.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:09.316+0000 7f8694913640 1 --2- 172.21.15.38:0/814394113 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8670078b90 0x7f867007b0a0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f869015d790 tx=0x7f8678009040 comp rx=0 tx=0).stop 2024-09-21T03:44:09.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:09.316+0000 7f8694913640 1 -- 172.21.15.38:0/814394113 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f86900b2e40 msgr2=0x7f86900ce940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:09.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:09.316+0000 7f8694913640 1 --2- 172.21.15.38:0/814394113 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f86900b2e40 0x7f86900ce940 secure :-1 s=READY pgs=2155 cs=0 l=1 rev1=1 crypto rx=0x7f8680000f80 tx=0x7f8680003ce0 comp rx=0 tx=0).stop 2024-09-21T03:44:09.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:09.316+0000 7f8694913640 1 -- 172.21.15.38:0/814394113 shutdown_connections 2024-09-21T03:44:09.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:09.316+0000 7f8694913640 1 --2- 172.21.15.38:0/814394113 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f869014f400 0x7f869014f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:09.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:09.316+0000 7f8694913640 1 --2- 172.21.15.38:0/814394113 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8670078b90 0x7f867007b0a0 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:09.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:09.316+0000 7f8694913640 1 --2- 172.21.15.38:0/814394113 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8690162280 0x7f86900cee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:09.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:09.316+0000 7f8694913640 1 --2- 172.21.15.38:0/814394113 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f86900b2e40 0x7f86900ce940 unknown :-1 s=CLOSED pgs=2155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:09.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:09.317+0000 7f8694913640 1 -- 172.21.15.38:0/814394113 >> 172.21.15.38:0/814394113 conn(0x7f86900b6b20 msgr2=0x7f869006cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:09.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:09.317+0000 7f8694913640 1 -- 172.21.15.38:0/814394113 shutdown_connections 2024-09-21T03:44:09.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:09.317+0000 7f8694913640 1 -- 172.21.15.38:0/814394113 wait complete. 2024-09-21T03:44:09.330 INFO:tasks.cephfs.test_damage:Result: Mutation 'Corrupt omap header on 600.00000000' led to DAMAGED state 2024-09-21T03:44:09.330 INFO:tasks.cephfs.test_damage:Applying mutation 'Corrupt omap header on mds0_openfiles.0' 2024-09-21T03:44:09.330 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-21T03:44:09.330 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:44:09.347 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056795,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2354 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T03:44:09.347 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi038.front.sepia.ceph.com 2024-09-21T03:44:09.347 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:44:09.347 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T03:44:09.402 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T03:44:09.402 INFO:teuthology.orchestra.run.smithi038.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-09-21T03:44:09.402 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-21T03:44:09.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:09.559+0000 7f1d12c07640 1 -- 172.21.15.38:0/3967095169 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1cfc003a00 msgr2=0x7f1cfc003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:09.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:09.559+0000 7f1d12c07640 1 --2- 172.21.15.38:0/3967095169 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1cfc003a00 0x7f1cfc003e70 secure :-1 s=READY pgs=2157 cs=0 l=1 rev1=1 crypto rx=0x7f1d00009b50 tx=0x7f1cfc005830 comp rx=0 tx=0).stop 2024-09-21T03:44:09.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:09.559+0000 7f1d111a4640 1 -- 172.21.15.38:0/3967095169 reap_dead start 2024-09-21T03:44:09.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:09.559+0000 7f1d12c07640 1 -- 172.21.15.38:0/3967095169 shutdown_connections 2024-09-21T03:44:09.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:09.559+0000 7f1d12c07640 1 -- 172.21.15.38:0/3967095169 >> 172.21.15.38:0/3967095169 conn(0x7f1d0c09d830 msgr2=0x7f1d0c0b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:09.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:09.559+0000 7f1d12c07640 1 -- 172.21.15.38:0/3967095169 shutdown_connections 2024-09-21T03:44:09.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:09.559+0000 7f1d12c07640 1 -- 172.21.15.38:0/3967095169 wait complete. 2024-09-21T03:44:09.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:09.561+0000 7f1d12c07640 1 Processor -- start 2024-09-21T03:44:09.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:09.561+0000 7f1d12c07640 1 -- start start 2024-09-21T03:44:09.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:09.562+0000 7f1d12c07640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1cfc003a00 0x7f1d0c09e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:09.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:09.562+0000 7f1d12c07640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1d0c0cbfc0 0x7f1d0c1f8770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:09.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:09.562+0000 7f1d12c07640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1d0c1f8cb0 0x7f1d0c1f5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:09.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:09.562+0000 7f1d12c07640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f1d0c0d0be0 con 0x7f1cfc003a00 2024-09-21T03:44:09.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:09.562+0000 7f1d12c07640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f1d0c0cf4b0 con 0x7f1d0c0cbfc0 2024-09-21T03:44:09.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:09.562+0000 7f1d12c07640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f1d0c0d0d60 con 0x7f1d0c1f8cb0 2024-09-21T03:44:09.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:09.562+0000 7f1d109a3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1cfc003a00 0x7f1d0c09e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:09.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:09.563+0000 7f1d109a3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1cfc003a00 0x7f1d0c09e2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:45532/0 (socket says 172.21.15.38:45532) 2024-09-21T03:44:09.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:09.563+0000 7f1d109a3640 1 -- 172.21.15.38:0/2231120091 learned_addr learned my addr 172.21.15.38:0/2231120091 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:44:09.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:09.563+0000 7f1d0bfff640 1 --2- 172.21.15.38:0/2231120091 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1d0c0cbfc0 0x7f1d0c1f8770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:09.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:09.563+0000 7f1d111a4640 1 --2- 172.21.15.38:0/2231120091 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1d0c1f8cb0 0x7f1d0c1f5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:09.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:09.563+0000 7f1d109a3640 1 -- 172.21.15.38:0/2231120091 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1d0c1f8cb0 msgr2=0x7f1d0c1f5100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:09.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:09.563+0000 7f1d109a3640 1 --2- 172.21.15.38:0/2231120091 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1d0c1f8cb0 0x7f1d0c1f5100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:09.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:09.563+0000 7f1d109a3640 1 -- 172.21.15.38:0/2231120091 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1d0c0cbfc0 msgr2=0x7f1d0c1f8770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:09.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:09.563+0000 7f1d109a3640 1 --2- 172.21.15.38:0/2231120091 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1d0c0cbfc0 0x7f1d0c1f8770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:09.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:09.563+0000 7f1d109a3640 1 -- 172.21.15.38:0/2231120091 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d0c1f90c0 con 0x7f1cfc003a00 2024-09-21T03:44:09.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:09.563+0000 7f1d109a3640 1 --2- 172.21.15.38:0/2231120091 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1cfc003a00 0x7f1d0c09e2a0 secure :-1 s=READY pgs=2158 cs=0 l=1 rev1=1 crypto rx=0x7f1d00009c70 tx=0x7f1d00033670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:09.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:09.563+0000 7f1d097fa640 1 -- 172.21.15.38:0/2231120091 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d00043070 con 0x7f1cfc003a00 2024-09-21T03:44:09.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:09.563+0000 7f1d097fa640 1 -- 172.21.15.38:0/2231120091 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1d00004120 con 0x7f1cfc003a00 2024-09-21T03:44:09.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:09.563+0000 7f1d12c07640 1 -- 172.21.15.38:0/2231120091 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d0c1f5d80 con 0x7f1cfc003a00 2024-09-21T03:44:09.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:09.563+0000 7f1d097fa640 1 -- 172.21.15.38:0/2231120091 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d00038040 con 0x7f1cfc003a00 2024-09-21T03:44:09.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:09.563+0000 7f1d12c07640 1 -- 172.21.15.38:0/2231120091 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d0c06ded0 con 0x7f1cfc003a00 2024-09-21T03:44:09.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:09.565+0000 7f1d12c07640 1 -- 172.21.15.38:0/2231120091 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d0c0d0170 con 0x7f1cfc003a00 2024-09-21T03:44:09.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:09.565+0000 7f1d097fa640 1 -- 172.21.15.38:0/2231120091 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f1d00046070 con 0x7f1cfc003a00 2024-09-21T03:44:09.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:09.566+0000 7f1d097fa640 1 --2- 172.21.15.38:0/2231120091 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1cf0078b90 0x7f1cf007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:09.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:09.566+0000 7f1d097fa640 1 -- 172.21.15.38:0/2231120091 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 9400+0+0 (secure 0 0 0) 0x7f1d000bc010 con 0x7f1cfc003a00 2024-09-21T03:44:09.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:09.566+0000 7f1d0bfff640 1 --2- 172.21.15.38:0/2231120091 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1cf0078b90 0x7f1cf007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:09.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:09.566+0000 7f1d0bfff640 1 --2- 172.21.15.38:0/2231120091 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1cf0078b90 0x7f1cf007b0a0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f1d0c0cbd90 tx=0x7f1cf8005f30 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:09.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:09.570+0000 7f1d097fa640 1 -- 172.21.15.38:0/2231120091 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f1d0c1f90c0 con 0x7f1cfc003a00 2024-09-21T03:44:09.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:09.800+0000 7f1d12c07640 1 -- 172.21.15.38:0/2231120091 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f1d0c0d1880 con 0x7f1cfc003a00 2024-09-21T03:44:09.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:09.874+0000 7f1d097fa640 1 -- 172.21.15.38:0/2231120091 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v306) ==== 161+0+0 (secure 0 0 0) 0x7f1d00083060 con 0x7f1cfc003a00 2024-09-21T03:44:09.876 INFO:teuthology.orchestra.run.smithi038.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-21T03:44:09.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:09.877+0000 7f1d12c07640 1 -- 172.21.15.38:0/2231120091 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1cf0078b90 msgr2=0x7f1cf007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:09.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:09.877+0000 7f1d12c07640 1 --2- 172.21.15.38:0/2231120091 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1cf0078b90 0x7f1cf007b0a0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f1d0c0cbd90 tx=0x7f1cf8005f30 comp rx=0 tx=0).stop 2024-09-21T03:44:09.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:09.877+0000 7f1d12c07640 1 -- 172.21.15.38:0/2231120091 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1cfc003a00 msgr2=0x7f1d0c09e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:09.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:09.877+0000 7f1d12c07640 1 --2- 172.21.15.38:0/2231120091 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1cfc003a00 0x7f1d0c09e2a0 secure :-1 s=READY pgs=2158 cs=0 l=1 rev1=1 crypto rx=0x7f1d00009c70 tx=0x7f1d00033670 comp rx=0 tx=0).stop 2024-09-21T03:44:09.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:09.877+0000 7f1d12c07640 1 -- 172.21.15.38:0/2231120091 shutdown_connections 2024-09-21T03:44:09.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:09.877+0000 7f1d12c07640 1 --2- 172.21.15.38:0/2231120091 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1cf0078b90 0x7f1cf007b0a0 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:09.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:09.877+0000 7f1d12c07640 1 --2- 172.21.15.38:0/2231120091 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1d0c1f8cb0 0x7f1d0c1f5100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:09.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:09.877+0000 7f1d12c07640 1 --2- 172.21.15.38:0/2231120091 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1d0c0cbfc0 0x7f1d0c1f8770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:09.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:09.877+0000 7f1d12c07640 1 --2- 172.21.15.38:0/2231120091 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1cfc003a00 0x7f1d0c09e2a0 unknown :-1 s=CLOSED pgs=2158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:09.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:09.877+0000 7f1d12c07640 1 -- 172.21.15.38:0/2231120091 >> 172.21.15.38:0/2231120091 conn(0x7f1d0c09d830 msgr2=0x7f1d0c0b31a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:09.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:09.877+0000 7f1d12c07640 1 -- 172.21.15.38:0/2231120091 shutdown_connections 2024-09-21T03:44:09.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:09.877+0000 7f1d12c07640 1 -- 172.21.15.38:0/2231120091 wait complete. 2024-09-21T03:44:09.891 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-09-21T03:44:10.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:10.008+0000 7fd7ff838640 1 -- 172.21.15.38:0/11505056 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7e4003a30 msgr2=0x7fd7e4003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:10.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:10.008+0000 7fd7ff838640 1 --2- 172.21.15.38:0/11505056 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7e4003a30 0x7fd7e4003ea0 secure :-1 s=READY pgs=2160 cs=0 l=1 rev1=1 crypto rx=0x7fd7f400c3f0 tx=0x7fd7e4005830 comp rx=0 tx=0).stop 2024-09-21T03:44:10.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:10.008+0000 7fd7fddd5640 1 -- 172.21.15.38:0/11505056 reap_dead start 2024-09-21T03:44:10.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:10.008+0000 7fd7ff838640 1 -- 172.21.15.38:0/11505056 shutdown_connections 2024-09-21T03:44:10.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:10.008+0000 7fd7ff838640 1 -- 172.21.15.38:0/11505056 >> 172.21.15.38:0/11505056 conn(0x7fd7f809d830 msgr2=0x7fd7f80b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:10.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:10.008+0000 7fd7ff838640 1 -- 172.21.15.38:0/11505056 shutdown_connections 2024-09-21T03:44:10.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:10.008+0000 7fd7ff838640 1 -- 172.21.15.38:0/11505056 wait complete. 2024-09-21T03:44:10.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:10.010+0000 7fd7ff838640 1 Processor -- start 2024-09-21T03:44:10.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:10.010+0000 7fd7ff838640 1 -- start start 2024-09-21T03:44:10.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:10.012+0000 7fd7ff838640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7e4003a30 0x7fd7f81f9900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:10.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:10.012+0000 7fd7ff838640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd7f8157990 0x7fd7f8157120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:10.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:10.012+0000 7fd7ff838640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd7f806f030 0x7fd7f806f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:10.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:10.012+0000 7fd7ff838640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fd7f80cf330 con 0x7fd7e4003a30 2024-09-21T03:44:10.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:10.012+0000 7fd7ff838640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd7f80cf1b0 con 0x7fd7f806f030 2024-09-21T03:44:10.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:10.012+0000 7fd7ff838640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fd7f80b6580 con 0x7fd7f8157990 2024-09-21T03:44:10.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:10.012+0000 7fd7fd5d4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7e4003a30 0x7fd7f81f9900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:10.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:10.012+0000 7fd7fd5d4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7e4003a30 0x7fd7f81f9900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:45538/0 (socket says 172.21.15.38:45538) 2024-09-21T03:44:10.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:10.012+0000 7fd7fd5d4640 1 -- 172.21.15.38:0/2173511089 learned_addr learned my addr 172.21.15.38:0/2173511089 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:44:10.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:10.012+0000 7fd7fddd5640 1 --2- 172.21.15.38:0/2173511089 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd7f806f030 0x7fd7f806f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:10.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:10.012+0000 7fd7fcdd3640 1 --2- 172.21.15.38:0/2173511089 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd7f8157990 0x7fd7f8157120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:10.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:10.012+0000 7fd7fd5d4640 1 -- 172.21.15.38:0/2173511089 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd7f8157990 msgr2=0x7fd7f8157120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:10.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:10.012+0000 7fd7fd5d4640 1 --2- 172.21.15.38:0/2173511089 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd7f8157990 0x7fd7f8157120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:10.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:10.012+0000 7fd7fd5d4640 1 -- 172.21.15.38:0/2173511089 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd7f806f030 msgr2=0x7fd7f806f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:10.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:10.012+0000 7fd7fd5d4640 1 --2- 172.21.15.38:0/2173511089 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd7f806f030 0x7fd7f806f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:10.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:10.012+0000 7fd7fd5d4640 1 -- 172.21.15.38:0/2173511089 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7f806f980 con 0x7fd7e4003a30 2024-09-21T03:44:10.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:10.012+0000 7fd7fcdd3640 1 --2- 172.21.15.38:0/2173511089 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd7f8157990 0x7fd7f8157120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:44:10.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:10.012+0000 7fd7fd5d4640 1 --2- 172.21.15.38:0/2173511089 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7e4003a30 0x7fd7f81f9900 secure :-1 s=READY pgs=2161 cs=0 l=1 rev1=1 crypto rx=0x7fd7f400c4e0 tx=0x7fd7f40013c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:10.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:10.012+0000 7fd7edffb640 1 -- 172.21.15.38:0/2173511089 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7f4042070 con 0x7fd7e4003a30 2024-09-21T03:44:10.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:10.012+0000 7fd7edffb640 1 -- 172.21.15.38:0/2173511089 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd7f40035f0 con 0x7fd7e4003a30 2024-09-21T03:44:10.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:10.012+0000 7fd7ff838640 1 -- 172.21.15.38:0/2173511089 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7f815b9c0 con 0x7fd7e4003a30 2024-09-21T03:44:10.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:10.012+0000 7fd7edffb640 1 -- 172.21.15.38:0/2173511089 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7f4039070 con 0x7fd7e4003a30 2024-09-21T03:44:10.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:10.013+0000 7fd7ff838640 1 -- 172.21.15.38:0/2173511089 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7f815ac00 con 0x7fd7e4003a30 2024-09-21T03:44:10.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:10.014+0000 7fd7ff838640 1 -- 172.21.15.38:0/2173511089 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7f80b6ae0 con 0x7fd7e4003a30 2024-09-21T03:44:10.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:10.014+0000 7fd7edffb640 1 -- 172.21.15.38:0/2173511089 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fd7f4045070 con 0x7fd7e4003a30 2024-09-21T03:44:10.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:10.015+0000 7fd7edffb640 1 --2- 172.21.15.38:0/2173511089 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd7dc078b90 0x7fd7dc07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:10.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:10.015+0000 7fd7edffb640 1 -- 172.21.15.38:0/2173511089 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 9400+0+0 (secure 0 0 0) 0x7fd7f40bb620 con 0x7fd7e4003a30 2024-09-21T03:44:10.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:10.015+0000 7fd7fcdd3640 1 --2- 172.21.15.38:0/2173511089 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd7dc078b90 0x7fd7dc07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:10.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:10.016+0000 7fd7fcdd3640 1 --2- 172.21.15.38:0/2173511089 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd7dc078b90 0x7fd7dc07b0a0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fd7f80caea0 tx=0x7fd7e8006c10 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:10.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:10.019+0000 7fd7edffb640 1 -- 172.21.15.38:0/2173511089 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fd7f806f980 con 0x7fd7e4003a30 2024-09-21T03:44:10.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:10.283+0000 7fd7ff838640 1 -- 172.21.15.38:0/2173511089 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7fd7f80b1ad0 con 0x7fd7e4003a30 2024-09-21T03:44:10.621 INFO:tasks.ceph.mds.c.smithi038.stderr: -51> 2024-09-21T03:44:05.865+0000 7f400d67f640 -1 mds.0.cache.dir(0x600) Corrupt fnode in dirfrag 0x600: Decoder at 'void fnode_t::decode(ceph::buffer::v15_2_0::list::const_iterator&)' v=3 cannot decode v=100 minimal_decoder=101: Malformed input [buffer:3] 2024-09-21T03:44:10.621 INFO:tasks.ceph.mds.c.smithi038.stderr: -48> 2024-09-21T03:44:05.865+0000 7f400d67f640 -1 mds.0.damage notify_dirfrag Damage to fragment * of ino 0x600 is fatal because it is a system directory for this rank 2024-09-21T03:44:10.621 INFO:tasks.ceph.mds.c.smithi038.stderr: -34> 2024-09-21T03:44:06.677+0000 7f4011687640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T03:44:10.677 INFO:tasks.ceph.mds.c.smithi038.stdout:starting mds.c at 2024-09-21T03:44:10.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:10.882+0000 7fd7edffb640 1 -- 172.21.15.38:0/2173511089 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v307) ==== 101+0+0 (secure 0 0 0) 0x7fd7f81592d0 con 0x7fd7e4003a30 2024-09-21T03:44:10.885 INFO:teuthology.orchestra.run.smithi038.stderr:repaired: restoring rank 6:0 2024-09-21T03:44:10.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:10.885+0000 7fd7ff838640 1 -- 172.21.15.38:0/2173511089 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd7dc078b90 msgr2=0x7fd7dc07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:10.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:10.885+0000 7fd7ff838640 1 --2- 172.21.15.38:0/2173511089 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd7dc078b90 0x7fd7dc07b0a0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fd7f80caea0 tx=0x7fd7e8006c10 comp rx=0 tx=0).stop 2024-09-21T03:44:10.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:10.885+0000 7fd7ff838640 1 -- 172.21.15.38:0/2173511089 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7e4003a30 msgr2=0x7fd7f81f9900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:10.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:10.885+0000 7fd7ff838640 1 --2- 172.21.15.38:0/2173511089 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7e4003a30 0x7fd7f81f9900 secure :-1 s=READY pgs=2161 cs=0 l=1 rev1=1 crypto rx=0x7fd7f400c4e0 tx=0x7fd7f40013c0 comp rx=0 tx=0).stop 2024-09-21T03:44:10.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:10.885+0000 7fd7ff838640 1 -- 172.21.15.38:0/2173511089 shutdown_connections 2024-09-21T03:44:10.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:10.885+0000 7fd7ff838640 1 --2- 172.21.15.38:0/2173511089 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd7f806f030 0x7fd7f806f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:10.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:10.885+0000 7fd7ff838640 1 --2- 172.21.15.38:0/2173511089 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd7dc078b90 0x7fd7dc07b0a0 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:10.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:10.885+0000 7fd7ff838640 1 --2- 172.21.15.38:0/2173511089 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd7f8157990 0x7fd7f8157120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:10.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:10.885+0000 7fd7ff838640 1 --2- 172.21.15.38:0/2173511089 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7e4003a30 0x7fd7f81f9900 unknown :-1 s=CLOSED pgs=2161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:10.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:10.885+0000 7fd7ff838640 1 -- 172.21.15.38:0/2173511089 >> 172.21.15.38:0/2173511089 conn(0x7fd7f809d830 msgr2=0x7fd7f80b3440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:10.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:10.885+0000 7fd7ff838640 1 -- 172.21.15.38:0/2173511089 shutdown_connections 2024-09-21T03:44:10.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:10.885+0000 7fd7ff838640 1 -- 172.21.15.38:0/2173511089 wait complete. 2024-09-21T03:44:10.899 DEBUG:teuthology.orchestra.run.smithi038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-09-21T03:44:10.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:10.955+0000 7ff237d29f40 1 Processor -- start 2024-09-21T03:44:10.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:10.955+0000 7ff237d29f40 1 -- start start 2024-09-21T03:44:10.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:10.956+0000 7ff237d29f40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555f4704ad00 0x555f47048d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:10.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:10.956+0000 7ff237d29f40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555f47076e30 0x555f471aa8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:10.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:10.956+0000 7ff237d29f40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555f47105950 0x555f47100c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:10.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:10.956+0000 7ff237d29f40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x555f46f98d60 con 0x555f4704ad00 2024-09-21T03:44:10.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:10.956+0000 7ff237d29f40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x555f46ff7430 con 0x555f47105950 2024-09-21T03:44:10.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:10.956+0000 7ff237d29f40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x555f470927b0 con 0x555f47076e30 2024-09-21T03:44:10.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:10.956+0000 7ff2317e3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555f4704ad00 0x555f47048d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:10.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:10.956+0000 7ff2317e3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555f4704ad00 0x555f47048d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:45574/0 (socket says 172.21.15.38:45574) 2024-09-21T03:44:10.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:10.956+0000 7ff2317e3640 1 -- 172.21.15.38:0/3956263059 learned_addr learned my addr 172.21.15.38:0/3956263059 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:44:10.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:10.956+0000 7ff231fe4640 1 --2- 172.21.15.38:0/3956263059 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555f47105950 0x555f47100c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:10.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:10.956+0000 7ff230fe2640 1 --2- 172.21.15.38:0/3956263059 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555f47076e30 0x555f471aa8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:10.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:10.956+0000 7ff2317e3640 1 -- 172.21.15.38:0/3956263059 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555f47076e30 msgr2=0x555f471aa8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:44:10.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:10.956+0000 7ff2317e3640 1 --2- 172.21.15.38:0/3956263059 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555f47076e30 0x555f471aa8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:10.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:10.957+0000 7ff2317e3640 1 -- 172.21.15.38:0/3956263059 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555f47105950 msgr2=0x555f47100c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:44:10.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:10.957+0000 7ff2317e3640 1 --2- 172.21.15.38:0/3956263059 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555f47105950 0x555f47100c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:10.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:10.957+0000 7ff2317e3640 1 -- 172.21.15.38:0/3956263059 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f471aae30 con 0x555f4704ad00 2024-09-21T03:44:10.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:10.957+0000 7ff2317e3640 1 --2- 172.21.15.38:0/3956263059 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555f4704ad00 0x555f47048d20 secure :-1 s=READY pgs=2167 cs=0 l=1 rev1=1 crypto rx=0x7ff22400c5d0 tx=0x7ff22400ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44dd94d64b37d518 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:10.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:10.957+0000 7ff2237fe640 1 -- 172.21.15.38:0/3956263059 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff22400aae0 con 0x555f4704ad00 2024-09-21T03:44:10.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:10.957+0000 7ff2237fe640 1 -- 172.21.15.38:0/3956263059 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff22400ac80 con 0x555f4704ad00 2024-09-21T03:44:10.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:10.957+0000 7ff2237fe640 1 -- 172.21.15.38:0/3956263059 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2240156a0 con 0x555f4704ad00 2024-09-21T03:44:10.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:10.957+0000 7ff237d29f40 1 -- 172.21.15.38:0/3956263059 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555f4704ad00 msgr2=0x555f47048d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:10.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:10.957+0000 7ff237d29f40 1 --2- 172.21.15.38:0/3956263059 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555f4704ad00 0x555f47048d20 secure :-1 s=READY pgs=2167 cs=0 l=1 rev1=1 crypto rx=0x7ff22400c5d0 tx=0x7ff22400ca90 comp rx=0 tx=0).stop 2024-09-21T03:44:10.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:10.957+0000 7ff237d29f40 1 -- 172.21.15.38:0/3956263059 shutdown_connections 2024-09-21T03:44:10.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:10.957+0000 7ff237d29f40 1 --2- 172.21.15.38:0/3956263059 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555f47105950 0x555f47100c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:10.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:10.957+0000 7ff237d29f40 1 --2- 172.21.15.38:0/3956263059 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555f47076e30 0x555f471aa8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:10.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:10.957+0000 7ff237d29f40 1 --2- 172.21.15.38:0/3956263059 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555f4704ad00 0x555f47048d20 unknown :-1 s=CLOSED pgs=2167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:10.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:10.957+0000 7ff237d29f40 1 -- 172.21.15.38:0/3956263059 >> 172.21.15.38:0/3956263059 conn(0x555f46f69ac0 msgr2=0x555f47104c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:10.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:10.958+0000 7ff237d29f40 1 -- 172.21.15.38:0/3956263059 shutdown_connections 2024-09-21T03:44:10.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:10.958+0000 7ff237d29f40 1 -- 172.21.15.38:0/3956263059 wait complete. 2024-09-21T03:44:10.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:10.958+0000 7ff237d29f40 1 Processor -- start 2024-09-21T03:44:10.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:10.958+0000 7ff237d29f40 1 -- start start 2024-09-21T03:44:10.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:10.960+0000 7ff237d29f40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555f4704ad00 0x555f47101130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:10.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:10.960+0000 7ff237d29f40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555f47076e30 0x555f47078400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:10.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:10.960+0000 7ff237d29f40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555f47105950 0x555f47089670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:10.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:10.960+0000 7ff237d29f40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x555f4711cad0 con 0x555f4704ad00 2024-09-21T03:44:10.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:10.960+0000 7ff237d29f40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x555f4711cd70 con 0x555f47105950 2024-09-21T03:44:10.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:10.960+0000 7ff237d29f40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x555f47102380 con 0x555f47076e30 2024-09-21T03:44:10.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:10.960+0000 7ff2317e3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555f4704ad00 0x555f47101130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:10.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:10.960+0000 7ff2317e3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555f4704ad00 0x555f47101130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:45584/0 (socket says 172.21.15.38:45584) 2024-09-21T03:44:10.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:10.960+0000 7ff2317e3640 1 -- 172.21.15.38:0/2779525883 learned_addr learned my addr 172.21.15.38:0/2779525883 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:44:10.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:10.960+0000 7ff231fe4640 1 --2- 172.21.15.38:0/2779525883 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555f47105950 0x555f47089670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:10.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:10.960+0000 7ff230fe2640 1 --2- 172.21.15.38:0/2779525883 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555f47076e30 0x555f47078400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:10.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:10.960+0000 7ff2317e3640 1 -- 172.21.15.38:0/2779525883 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555f47076e30 msgr2=0x555f47078400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:10.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:10.960+0000 7ff2317e3640 1 --2- 172.21.15.38:0/2779525883 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555f47076e30 0x555f47078400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:10.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:10.960+0000 7ff2317e3640 1 -- 172.21.15.38:0/2779525883 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555f47105950 msgr2=0x555f47089670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:10.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:10.960+0000 7ff2317e3640 1 --2- 172.21.15.38:0/2779525883 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555f47105950 0x555f47089670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:10.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:10.960+0000 7ff2317e3640 1 -- 172.21.15.38:0/2779525883 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f4711bb00 con 0x555f4704ad00 2024-09-21T03:44:10.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:10.960+0000 7ff2317e3640 1 --2- 172.21.15.38:0/2779525883 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555f4704ad00 0x555f47101130 secure :-1 s=READY pgs=2168 cs=0 l=1 rev1=1 crypto rx=0x7ff224004160 tx=0x7ff2240156f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:10.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:10.960+0000 7ff2227fc640 1 -- 172.21.15.38:0/2779525883 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff224015cf0 con 0x555f4704ad00 2024-09-21T03:44:10.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:10.961+0000 7ff2227fc640 1 -- 172.21.15.38:0/2779525883 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff22401f430 con 0x555f4704ad00 2024-09-21T03:44:10.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:10.961+0000 7ff237d29f40 1 -- 172.21.15.38:0/2779525883 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x555f471b5350 con 0x555f4704ad00 2024-09-21T03:44:10.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:10.961+0000 7ff2227fc640 1 -- 172.21.15.38:0/2779525883 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff22401f860 con 0x555f4704ad00 2024-09-21T03:44:10.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:10.961+0000 7ff237d29f40 1 -- 172.21.15.38:0/2779525883 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f471b5910 con 0x555f4704ad00 2024-09-21T03:44:10.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:10.962+0000 7ff2227fc640 1 -- 172.21.15.38:0/2779525883 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ff22401fa00 con 0x555f4704ad00 2024-09-21T03:44:10.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:10.963+0000 7ff2227fc640 1 --2- 172.21.15.38:0/2779525883 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff2140775a0 0x7ff214079ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:10.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:10.963+0000 7ff2227fc640 1 -- 172.21.15.38:0/2779525883 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 9400+0+0 (secure 0 0 0) 0x7ff22409c690 con 0x555f4704ad00 2024-09-21T03:44:10.967 INFO:teuthology.orchestra.run.smithi038.stdout:Importing pool 2024-09-21T03:44:10.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:10.963+0000 7ff230fe2640 1 --2- 172.21.15.38:0/2779525883 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff2140775a0 0x7ff214079ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:10.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:10.963+0000 7ff237d29f40 1 --2- 172.21.15.38:0/2779525883 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x555f471a8b90 0x555f471a8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:10.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:10.963+0000 7ff237d29f40 1 -- 172.21.15.38:0/2779525883 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x555f47078940 con 0x555f471a8b90 2024-09-21T03:44:10.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:10.963+0000 7ff231fe4640 1 --2- 172.21.15.38:0/2779525883 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x555f471a8b90 0x555f471a8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:10.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:10.964+0000 7ff230fe2640 1 --2- 172.21.15.38:0/2779525883 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff2140775a0 0x7ff214079ab0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x555f471a9fb0 tx=0x7ff228003610 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:10.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:10.964+0000 7ff231fe4640 1 --2- 172.21.15.38:0/2779525883 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x555f471a8b90 0x555f471a8fa0 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:10.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:10.966+0000 7ff231fe4640 1 -- 172.21.15.38:0/2779525883 <== osd.7 v2:172.21.15.195:6818/1606106342 1 ==== osd_op_reply(1 601.00000000 [create] v132'1097 uv1070 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ff22c00b040 con 0x555f471a8b90 2024-09-21T03:44:10.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:10.966+0000 7ff237d29f40 1 -- 172.21.15.38:0/2779525883 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e132) -- 0x555f471c19c0 con 0x555f471a8b90 2024-09-21T03:44:10.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:10.968+0000 7ff231fe4640 1 -- 172.21.15.38:0/2779525883 <== osd.7 v2:172.21.15.195:6818/1606106342 2 ==== osd_op_reply(2 601.00000000 [delete] v132'1098 uv1098 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff22c00b040 con 0x555f471a8b90 2024-09-21T03:44:10.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:10.968+0000 7ff237d29f40 1 -- 172.21.15.38:0/2779525883 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x555f471c1d60 con 0x555f471a8b90 2024-09-21T03:44:10.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:10.969+0000 7ff231fe4640 1 -- 172.21.15.38:0/2779525883 <== osd.7 v2:172.21.15.195:6818/1606106342 3 ==== osd_op_reply(3 601.00000000 [create] v132'1099 uv1099 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff22c00b040 con 0x555f471a8b90 2024-09-21T03:44:10.971 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-09-21T03:44:10.972 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:10.969+0000 7ff237d29f40 1 -- 172.21.15.38:0/2779525883 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x555f471c3110 con 0x555f471a8b90 2024-09-21T03:44:10.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:10.971+0000 7ff231fe4640 1 -- 172.21.15.38:0/2779525883 <== osd.7 v2:172.21.15.195:6818/1606106342 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v132'1100 uv1100 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff22c00b040 con 0x555f471a8b90 2024-09-21T03:44:10.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:10.971+0000 7ff237d29f40 1 -- 172.21.15.38:0/2779525883 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x555f471c34b0 con 0x555f471a8b90 2024-09-21T03:44:10.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:10.972+0000 7ff231fe4640 1 -- 172.21.15.38:0/2779525883 <== osd.7 v2:172.21.15.195:6818/1606106342 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v132'1101 uv1101 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff22c00b040 con 0x555f471a8b90 2024-09-21T03:44:10.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:10.972+0000 7ff237d29f40 1 -- 172.21.15.38:0/2779525883 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x555f471c2100 con 0x555f471a8b90 2024-09-21T03:44:10.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:10.974+0000 7ff231fe4640 1 -- 172.21.15.38:0/2779525883 <== osd.7 v2:172.21.15.195:6818/1606106342 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v132'1102 uv1102 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff22c00b040 con 0x555f471a8b90 2024-09-21T03:44:10.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:10.974+0000 7ff237d29f40 1 -- 172.21.15.38:0/2779525883 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x555f471c2510 con 0x555f471a8b90 2024-09-21T03:44:10.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:10.976+0000 7ff231fe4640 1 -- 172.21.15.38:0/2779525883 <== osd.7 v2:172.21.15.195:6818/1606106342 7 ==== osd_op_reply(7 602.00000000 [create] v132'1103 uv1076 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ff22c00b040 con 0x555f471a8b90 2024-09-21T03:44:10.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:10.976+0000 7ff237d29f40 1 -- 172.21.15.38:0/2779525883 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e132) -- 0x555f471c2900 con 0x555f471a8b90 2024-09-21T03:44:10.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:10.977+0000 7ff231fe4640 1 -- 172.21.15.38:0/2779525883 <== osd.7 v2:172.21.15.195:6818/1606106342 8 ==== osd_op_reply(8 602.00000000 [delete] v132'1104 uv1104 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff22c00b040 con 0x555f471a8b90 2024-09-21T03:44:10.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:10.977+0000 7ff237d29f40 1 -- 172.21.15.38:0/2779525883 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x555f471c2900 con 0x555f471a8b90 2024-09-21T03:44:10.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:10.979+0000 7ff231fe4640 1 -- 172.21.15.38:0/2779525883 <== osd.7 v2:172.21.15.195:6818/1606106342 9 ==== osd_op_reply(9 602.00000000 [create] v132'1105 uv1105 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff22c00b040 con 0x555f471a8b90 2024-09-21T03:44:10.981 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-09-21T03:44:10.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:10.979+0000 7ff237d29f40 1 -- 172.21.15.38:0/2779525883 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x555f471c28b0 con 0x555f471a8b90 2024-09-21T03:44:10.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:10.981+0000 7ff231fe4640 1 -- 172.21.15.38:0/2779525883 <== osd.7 v2:172.21.15.195:6818/1606106342 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v132'1106 uv1106 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff22c00b040 con 0x555f471a8b90 2024-09-21T03:44:10.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:10.981+0000 7ff237d29f40 1 -- 172.21.15.38:0/2779525883 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x555f471c28b0 con 0x555f471a8b90 2024-09-21T03:44:10.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:10.982+0000 7ff231fe4640 1 -- 172.21.15.38:0/2779525883 <== osd.7 v2:172.21.15.195:6818/1606106342 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v132'1107 uv1107 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff22c00b040 con 0x555f471a8b90 2024-09-21T03:44:10.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:10.982+0000 7ff237d29f40 1 -- 172.21.15.38:0/2779525883 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x555f471c28b0 con 0x555f471a8b90 2024-09-21T03:44:10.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:10.983+0000 7ff231fe4640 1 -- 172.21.15.38:0/2779525883 <== osd.7 v2:172.21.15.195:6818/1606106342 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v132'1108 uv1108 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff22c00b040 con 0x555f471a8b90 2024-09-21T03:44:10.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:10.984+0000 7ff237d29f40 1 -- 172.21.15.38:0/2779525883 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:13 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x555f471c28b0 con 0x555f471a8b90 2024-09-21T03:44:10.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:10.986+0000 7ff231fe4640 1 -- 172.21.15.38:0/2779525883 <== osd.7 v2:172.21.15.195:6818/1606106342 13 ==== osd_op_reply(13 10000000000.00000000 [create] v132'1109 uv1083 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7ff22c00b040 con 0x555f471a8b90 2024-09-21T03:44:10.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:10.986+0000 7ff237d29f40 1 -- 172.21.15.38:0/2779525883 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:14 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e132) -- 0x555f471c28b0 con 0x555f471a8b90 2024-09-21T03:44:10.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:10.987+0000 7ff231fe4640 1 -- 172.21.15.38:0/2779525883 <== osd.7 v2:172.21.15.195:6818/1606106342 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v132'1110 uv1110 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7ff22c00b040 con 0x555f471a8b90 2024-09-21T03:44:10.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:10.987+0000 7ff237d29f40 1 -- 172.21.15.38:0/2779525883 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:15 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x555f471c28b0 con 0x555f471a8b90 2024-09-21T03:44:10.990 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-09-21T03:44:10.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:10.988+0000 7ff231fe4640 1 -- 172.21.15.38:0/2779525883 <== osd.7 v2:172.21.15.195:6818/1606106342 15 ==== osd_op_reply(15 10000000000.00000000 [create] v132'1111 uv1111 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7ff22c00b040 con 0x555f471a8b90 2024-09-21T03:44:10.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:10.988+0000 7ff237d29f40 1 -- 172.21.15.38:0/2779525883 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:16 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x555f471c28b0 con 0x555f471a8b90 2024-09-21T03:44:10.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:10.990+0000 7ff231fe4640 1 -- 172.21.15.38:0/2779525883 <== osd.7 v2:172.21.15.195:6818/1606106342 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v132'1112 uv1112 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7ff22c00b040 con 0x555f471a8b90 2024-09-21T03:44:10.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:10.990+0000 7ff237d29f40 1 -- 172.21.15.38:0/2779525883 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:17 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x555f471c28b0 con 0x555f471a8b90 2024-09-21T03:44:10.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:10.992+0000 7ff231fe4640 1 -- 172.21.15.38:0/2779525883 <== osd.7 v2:172.21.15.195:6818/1606106342 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v132'1113 uv1113 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7ff22c00b040 con 0x555f471a8b90 2024-09-21T03:44:10.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:10.992+0000 7ff237d29f40 1 -- 172.21.15.38:0/2779525883 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:18 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x555f471c28b0 con 0x555f471a8b90 2024-09-21T03:44:10.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:10.993+0000 7ff231fe4640 1 -- 172.21.15.38:0/2779525883 <== osd.7 v2:172.21.15.195:6818/1606106342 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v132'1114 uv1114 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7ff22c00b040 con 0x555f471a8b90 2024-09-21T03:44:10.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:10.993+0000 7ff237d29f40 1 -- 172.21.15.38:0/2779525883 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:19 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=585b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x555f471c28b0 con 0x555f471a8b90 2024-09-21T03:44:10.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:10.995+0000 7ff231fe4640 1 -- 172.21.15.38:0/2779525883 <== osd.7 v2:172.21.15.195:6818/1606106342 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v132'1115 uv1115 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7ff22c00b040 con 0x555f471a8b90 2024-09-21T03:44:10.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:10.995+0000 7ff237d29f40 1 -- 172.21.15.38:0/2779525883 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:20 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x555f471c28b0 con 0x555f471a8b90 2024-09-21T03:44:10.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:10.996+0000 7ff231fe4640 1 -- 172.21.15.38:0/2779525883 <== osd.7 v2:172.21.15.195:6818/1606106342 20 ==== osd_op_reply(20 600.00000000 [create] v132'1116 uv1096 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ff22c00b040 con 0x555f471a8b90 2024-09-21T03:44:10.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:10.996+0000 7ff237d29f40 1 -- 172.21.15.38:0/2779525883 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:21 13.10 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e132) -- 0x555f471c28b0 con 0x555f471a8b90 2024-09-21T03:44:11.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:10.998+0000 7ff231fe4640 1 -- 172.21.15.38:0/2779525883 <== osd.7 v2:172.21.15.195:6818/1606106342 21 ==== osd_op_reply(21 600.00000000 [delete] v132'1117 uv1117 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff22c00b040 con 0x555f471a8b90 2024-09-21T03:44:11.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:10.998+0000 7ff237d29f40 1 -- 172.21.15.38:0/2779525883 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:22 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x555f471c28b0 con 0x555f471a8b90 2024-09-21T03:44:11.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:10.999+0000 7ff231fe4640 1 -- 172.21.15.38:0/2779525883 <== osd.7 v2:172.21.15.195:6818/1606106342 22 ==== osd_op_reply(22 600.00000000 [create] v132'1118 uv1118 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff22c00b040 con 0x555f471a8b90 2024-09-21T03:44:11.001 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-09-21T03:44:11.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:10.999+0000 7ff237d29f40 1 -- 172.21.15.38:0/2779525883 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:23 13.10 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x555f471c28b0 con 0x555f471a8b90 2024-09-21T03:44:11.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.001+0000 7ff231fe4640 1 -- 172.21.15.38:0/2779525883 <== osd.7 v2:172.21.15.195:6818/1606106342 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v132'1119 uv1119 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff22c00b040 con 0x555f471a8b90 2024-09-21T03:44:11.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.001+0000 7ff237d29f40 1 -- 172.21.15.38:0/2779525883 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:24 13.10 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x555f471c28b0 con 0x555f471a8b90 2024-09-21T03:44:11.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.002+0000 7ff231fe4640 1 -- 172.21.15.38:0/2779525883 <== osd.7 v2:172.21.15.195:6818/1606106342 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v132'1120 uv1120 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff22c00b040 con 0x555f471a8b90 2024-09-21T03:44:11.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.002+0000 7ff237d29f40 1 -- 172.21.15.38:0/2779525883 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:25 13.10 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x555f471c28b0 con 0x555f471a8b90 2024-09-21T03:44:11.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.004+0000 7ff231fe4640 1 -- 172.21.15.38:0/2779525883 <== osd.7 v2:172.21.15.195:6818/1606106342 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v132'1121 uv1121 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff22c00b040 con 0x555f471a8b90 2024-09-21T03:44:11.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.004+0000 7ff237d29f40 1 -- 172.21.15.38:0/2779525883 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:26 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x555f471c28b0 con 0x555f471a8b90 2024-09-21T03:44:11.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.005+0000 7ff231fe4640 1 -- 172.21.15.38:0/2779525883 <== osd.7 v2:172.21.15.195:6818/1606106342 26 ==== osd_op_reply(26 603.00000000 [create] v132'1122 uv1095 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ff22c00b040 con 0x555f471a8b90 2024-09-21T03:44:11.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.006+0000 7ff237d29f40 1 -- 172.21.15.38:0/2779525883 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:27 13.10 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e132) -- 0x555f471c28b0 con 0x555f471a8b90 2024-09-21T03:44:11.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.007+0000 7ff231fe4640 1 -- 172.21.15.38:0/2779525883 <== osd.7 v2:172.21.15.195:6818/1606106342 27 ==== osd_op_reply(27 603.00000000 [delete] v132'1123 uv1123 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff22c00b040 con 0x555f471a8b90 2024-09-21T03:44:11.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.007+0000 7ff237d29f40 1 -- 172.21.15.38:0/2779525883 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:28 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x555f471c28b0 con 0x555f471a8b90 2024-09-21T03:44:11.010 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-09-21T03:44:11.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.009+0000 7ff231fe4640 1 -- 172.21.15.38:0/2779525883 <== osd.7 v2:172.21.15.195:6818/1606106342 28 ==== osd_op_reply(28 603.00000000 [create] v132'1124 uv1124 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff22c00b040 con 0x555f471a8b90 2024-09-21T03:44:11.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.009+0000 7ff237d29f40 1 -- 172.21.15.38:0/2779525883 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:29 13.10 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x555f471c28b0 con 0x555f471a8b90 2024-09-21T03:44:11.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.010+0000 7ff231fe4640 1 -- 172.21.15.38:0/2779525883 <== osd.7 v2:172.21.15.195:6818/1606106342 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v132'1125 uv1125 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff22c00b040 con 0x555f471a8b90 2024-09-21T03:44:11.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.010+0000 7ff237d29f40 1 -- 172.21.15.38:0/2779525883 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:30 13.10 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x555f471c28b0 con 0x555f471a8b90 2024-09-21T03:44:11.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.012+0000 7ff231fe4640 1 -- 172.21.15.38:0/2779525883 <== osd.7 v2:172.21.15.195:6818/1606106342 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v132'1126 uv1126 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff22c00b040 con 0x555f471a8b90 2024-09-21T03:44:11.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.012+0000 7ff237d29f40 1 -- 172.21.15.38:0/2779525883 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:31 13.10 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x555f471c28b0 con 0x555f471a8b90 2024-09-21T03:44:11.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.013+0000 7ff231fe4640 1 -- 172.21.15.38:0/2779525883 <== osd.7 v2:172.21.15.195:6818/1606106342 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v132'1127 uv1127 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff22c00b040 con 0x555f471a8b90 2024-09-21T03:44:11.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.013+0000 7ff237d29f40 1 --2- 172.21.15.38:0/2779525883 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x555f471c2ce0 0x555f471c5d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:11.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.014+0000 7ff237d29f40 1 -- 172.21.15.38:0/2779525883 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x555f471c28b0 con 0x555f471c2ce0 2024-09-21T03:44:11.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.014+0000 7ff2317e3640 1 --2- 172.21.15.38:0/2779525883 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x555f471c2ce0 0x555f471c5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:11.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.014+0000 7ff2317e3640 1 --2- 172.21.15.38:0/2779525883 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x555f471c2ce0 0x555f471c5d70 crc :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:11.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.016+0000 7ff2317e3640 1 -- 172.21.15.38:0/2779525883 <== osd.6 v2:172.21.15.195:6826/4055825675 1 ==== osd_op_reply(32 1.00000000.inode [create] v132'798 uv780 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7ff22401b070 con 0x555f471c2ce0 2024-09-21T03:44:11.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.016+0000 7ff237d29f40 1 -- 172.21.15.38:0/2779525883 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e132) -- 0x555f471c62b0 con 0x555f471c2ce0 2024-09-21T03:44:11.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.017+0000 7ff2317e3640 1 -- 172.21.15.38:0/2779525883 <== osd.6 v2:172.21.15.195:6826/4055825675 2 ==== osd_op_reply(33 1.00000000.inode [delete] v132'799 uv799 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7ff22401b070 con 0x555f471c2ce0 2024-09-21T03:44:11.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.017+0000 7ff237d29f40 1 -- 172.21.15.38:0/2779525883 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x555f471c6650 con 0x555f471c2ce0 2024-09-21T03:44:11.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.019+0000 7ff2317e3640 1 -- 172.21.15.38:0/2779525883 <== osd.6 v2:172.21.15.195:6826/4055825675 3 ==== osd_op_reply(34 1.00000000.inode [create] v132'800 uv800 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7ff22401b070 con 0x555f471c2ce0 2024-09-21T03:44:11.021 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-09-21T03:44:11.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.019+0000 7ff237d29f40 1 -- 172.21.15.38:0/2779525883 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x555f471c6ea0 con 0x555f471c2ce0 2024-09-21T03:44:11.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.021+0000 7ff2317e3640 1 -- 172.21.15.38:0/2779525883 <== osd.6 v2:172.21.15.195:6826/4055825675 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v132'801 uv801 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7ff22401b070 con 0x555f471c2ce0 2024-09-21T03:44:11.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.021+0000 7ff237d29f40 1 -- 172.21.15.38:0/2779525883 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x555f471c7240 con 0x555f471c2ce0 2024-09-21T03:44:11.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.022+0000 7ff2317e3640 1 -- 172.21.15.38:0/2779525883 <== osd.6 v2:172.21.15.195:6826/4055825675 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v132'802 uv802 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7ff22401b070 con 0x555f471c2ce0 2024-09-21T03:44:11.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.022+0000 7ff237d29f40 1 -- 172.21.15.38:0/2779525883 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x555f471c75e0 con 0x555f471c2ce0 2024-09-21T03:44:11.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.024+0000 7ff2317e3640 1 -- 172.21.15.38:0/2779525883 <== osd.6 v2:172.21.15.195:6826/4055825675 6 ==== osd_op_reply(37 200.00000000 [create] v132'803 uv785 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ff22401b070 con 0x555f471c2ce0 2024-09-21T03:44:11.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.024+0000 7ff237d29f40 1 -- 172.21.15.38:0/2779525883 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e132) -- 0x555f471c7980 con 0x555f471c2ce0 2024-09-21T03:44:11.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.025+0000 7ff2317e3640 1 -- 172.21.15.38:0/2779525883 <== osd.6 v2:172.21.15.195:6826/4055825675 7 ==== osd_op_reply(38 200.00000000 [delete] v132'804 uv804 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff22401b070 con 0x555f471c2ce0 2024-09-21T03:44:11.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.025+0000 7ff237d29f40 1 -- 172.21.15.38:0/2779525883 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x555f471c7d20 con 0x555f471c2ce0 2024-09-21T03:44:11.028 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-09-21T03:44:11.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.027+0000 7ff2317e3640 1 -- 172.21.15.38:0/2779525883 <== osd.6 v2:172.21.15.195:6826/4055825675 8 ==== osd_op_reply(39 200.00000000 [create] v132'805 uv805 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff22401b070 con 0x555f471c2ce0 2024-09-21T03:44:11.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.027+0000 7ff237d29f40 1 -- 172.21.15.38:0/2779525883 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x555f471c6ad0 con 0x555f471c2ce0 2024-09-21T03:44:11.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.028+0000 7ff2317e3640 1 -- 172.21.15.38:0/2779525883 <== osd.6 v2:172.21.15.195:6826/4055825675 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v132'806 uv806 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff22401b070 con 0x555f471c2ce0 2024-09-21T03:44:11.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.028+0000 7ff237d29f40 1 -- 172.21.15.38:0/2779525883 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x555f471c6ad0 con 0x555f471c2ce0 2024-09-21T03:44:11.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.029+0000 7ff2317e3640 1 -- 172.21.15.38:0/2779525883 <== osd.6 v2:172.21.15.195:6826/4055825675 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v132'807 uv807 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff22401b070 con 0x555f471c2ce0 2024-09-21T03:44:11.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.029+0000 7ff237d29f40 1 -- 172.21.15.38:0/2779525883 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:42 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x555f471c6ad0 con 0x555f471c2ce0 2024-09-21T03:44:11.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.031+0000 7ff2317e3640 1 -- 172.21.15.38:0/2779525883 <== osd.6 v2:172.21.15.195:6826/4055825675 11 ==== osd_op_reply(42 200.00000001 [create] v132'808 uv790 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ff22401b070 con 0x555f471c2ce0 2024-09-21T03:44:11.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.031+0000 7ff237d29f40 1 -- 172.21.15.38:0/2779525883 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:43 13.14 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e132) -- 0x555f471c6ad0 con 0x555f471c2ce0 2024-09-21T03:44:11.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.032+0000 7ff2317e3640 1 -- 172.21.15.38:0/2779525883 <== osd.6 v2:172.21.15.195:6826/4055825675 12 ==== osd_op_reply(43 200.00000001 [delete] v132'809 uv809 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff22401b070 con 0x555f471c2ce0 2024-09-21T03:44:11.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.032+0000 7ff237d29f40 1 -- 172.21.15.38:0/2779525883 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:44 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x555f471c6ad0 con 0x555f471c2ce0 2024-09-21T03:44:11.035 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-09-21T03:44:11.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.033+0000 7ff2317e3640 1 -- 172.21.15.38:0/2779525883 <== osd.6 v2:172.21.15.195:6826/4055825675 13 ==== osd_op_reply(44 200.00000001 [create] v132'810 uv810 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff22401b070 con 0x555f471c2ce0 2024-09-21T03:44:11.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.033+0000 7ff237d29f40 1 -- 172.21.15.38:0/2779525883 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:45 13.14 13:2e2fa760:::200.00000001:head [write 0~17490 in=17490b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x555f471c6ad0 con 0x555f471c2ce0 2024-09-21T03:44:11.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.035+0000 7ff2317e3640 1 -- 172.21.15.38:0/2779525883 <== osd.6 v2:172.21.15.195:6826/4055825675 14 ==== osd_op_reply(45 200.00000001 [write 0~17490] v132'811 uv811 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff22401b070 con 0x555f471c2ce0 2024-09-21T03:44:11.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.035+0000 7ff237d29f40 1 -- 172.21.15.38:0/2779525883 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:46 13.14 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x555f471c6ad0 con 0x555f471c2ce0 2024-09-21T03:44:11.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.036+0000 7ff2317e3640 1 -- 172.21.15.38:0/2779525883 <== osd.6 v2:172.21.15.195:6826/4055825675 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v132'812 uv812 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff22401b070 con 0x555f471c2ce0 2024-09-21T03:44:11.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.036+0000 7ff237d29f40 1 -- 172.21.15.38:0/2779525883 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:47 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x555f471c6ad0 con 0x555f471c2ce0 2024-09-21T03:44:11.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.038+0000 7ff2317e3640 1 -- 172.21.15.38:0/2779525883 <== osd.6 v2:172.21.15.195:6826/4055825675 16 ==== osd_op_reply(47 606.00000000 [create] v132'813 uv796 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ff22401b070 con 0x555f471c2ce0 2024-09-21T03:44:11.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.038+0000 7ff237d29f40 1 -- 172.21.15.38:0/2779525883 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:48 13.14 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e132) -- 0x555f471c6ad0 con 0x555f471c2ce0 2024-09-21T03:44:11.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.039+0000 7ff2317e3640 1 -- 172.21.15.38:0/2779525883 <== osd.6 v2:172.21.15.195:6826/4055825675 17 ==== osd_op_reply(48 606.00000000 [delete] v132'814 uv814 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff22401b070 con 0x555f471c2ce0 2024-09-21T03:44:11.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.039+0000 7ff237d29f40 1 -- 172.21.15.38:0/2779525883 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:49 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x555f471c6ad0 con 0x555f471c2ce0 2024-09-21T03:44:11.042 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-09-21T03:44:11.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.040+0000 7ff2317e3640 1 -- 172.21.15.38:0/2779525883 <== osd.6 v2:172.21.15.195:6826/4055825675 18 ==== osd_op_reply(49 606.00000000 [create] v132'815 uv815 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff22401b070 con 0x555f471c2ce0 2024-09-21T03:44:11.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.041+0000 7ff237d29f40 1 -- 172.21.15.38:0/2779525883 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:50 13.14 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x555f471c6ad0 con 0x555f471c2ce0 2024-09-21T03:44:11.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.042+0000 7ff2317e3640 1 -- 172.21.15.38:0/2779525883 <== osd.6 v2:172.21.15.195:6826/4055825675 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v132'816 uv816 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff22401b070 con 0x555f471c2ce0 2024-09-21T03:44:11.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.042+0000 7ff237d29f40 1 -- 172.21.15.38:0/2779525883 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:51 13.14 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x555f471c6ad0 con 0x555f471c2ce0 2024-09-21T03:44:11.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.043+0000 7ff2317e3640 1 -- 172.21.15.38:0/2779525883 <== osd.6 v2:172.21.15.195:6826/4055825675 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v132'817 uv817 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff22401b070 con 0x555f471c2ce0 2024-09-21T03:44:11.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.043+0000 7ff237d29f40 1 -- 172.21.15.38:0/2779525883 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:52 13.14 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x555f471c6ad0 con 0x555f471c2ce0 2024-09-21T03:44:11.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.044+0000 7ff2317e3640 1 -- 172.21.15.38:0/2779525883 <== osd.6 v2:172.21.15.195:6826/4055825675 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v132'818 uv818 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff22401b070 con 0x555f471c2ce0 2024-09-21T03:44:11.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.044+0000 7ff237d29f40 1 --2- 172.21.15.38:0/2779525883 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x555f471c86e0 0x555f471cab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:11.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.044+0000 7ff237d29f40 1 -- 172.21.15.38:0/2779525883 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:53 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x555f471cb040 con 0x555f471c86e0 2024-09-21T03:44:11.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.044+0000 7ff230fe2640 1 --2- 172.21.15.38:0/2779525883 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x555f471c86e0 0x555f471cab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:11.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.045+0000 7ff230fe2640 1 --2- 172.21.15.38:0/2779525883 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x555f471c86e0 0x555f471cab00 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:11.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.046+0000 7ff230fe2640 1 -- 172.21.15.38:0/2779525883 <== osd.0 v2:172.21.15.38:6800/2381639433 1 ==== osd_op_reply(53 607.00000000 [create] v132'213 uv212 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ff228013790 con 0x555f471c86e0 2024-09-21T03:44:11.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.046+0000 7ff237d29f40 1 -- 172.21.15.38:0/2779525883 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:54 13.1c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e132) -- 0x555f471cb420 con 0x555f471c86e0 2024-09-21T03:44:11.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.048+0000 7ff230fe2640 1 -- 172.21.15.38:0/2779525883 <== osd.0 v2:172.21.15.38:6800/2381639433 2 ==== osd_op_reply(54 607.00000000 [delete] v132'214 uv214 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff228013790 con 0x555f471c86e0 2024-09-21T03:44:11.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.048+0000 7ff237d29f40 1 -- 172.21.15.38:0/2779525883 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:55 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x555f471cb8d0 con 0x555f471c86e0 2024-09-21T03:44:11.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.049+0000 7ff230fe2640 1 -- 172.21.15.38:0/2779525883 <== osd.0 v2:172.21.15.38:6800/2381639433 3 ==== osd_op_reply(55 607.00000000 [create] v132'215 uv215 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff228013790 con 0x555f471c86e0 2024-09-21T03:44:11.051 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-09-21T03:44:11.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.049+0000 7ff237d29f40 1 -- 172.21.15.38:0/2779525883 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:56 13.1c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x555f471cd1b0 con 0x555f471c86e0 2024-09-21T03:44:11.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.050+0000 7ff230fe2640 1 -- 172.21.15.38:0/2779525883 <== osd.0 v2:172.21.15.38:6800/2381639433 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v132'216 uv216 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff228013790 con 0x555f471c86e0 2024-09-21T03:44:11.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.050+0000 7ff237d29f40 1 -- 172.21.15.38:0/2779525883 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:57 13.1c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x555f471cd550 con 0x555f471c86e0 2024-09-21T03:44:11.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.051+0000 7ff230fe2640 1 -- 172.21.15.38:0/2779525883 <== osd.0 v2:172.21.15.38:6800/2381639433 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v132'217 uv217 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff228013790 con 0x555f471c86e0 2024-09-21T03:44:11.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.051+0000 7ff237d29f40 1 -- 172.21.15.38:0/2779525883 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:58 13.1c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x555f471cc1a0 con 0x555f471c86e0 2024-09-21T03:44:11.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.052+0000 7ff230fe2640 1 -- 172.21.15.38:0/2779525883 <== osd.0 v2:172.21.15.38:6800/2381639433 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v132'218 uv218 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff228013790 con 0x555f471c86e0 2024-09-21T03:44:11.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.052+0000 7ff237d29f40 1 --2- 172.21.15.38:0/2779525883 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x555f471ccaa0 0x555f471cf900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:11.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.052+0000 7ff237d29f40 1 -- 172.21.15.38:0/2779525883 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:59 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x555f471cfe40 con 0x555f471ccaa0 2024-09-21T03:44:11.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.052+0000 7ff231fe4640 1 --2- 172.21.15.38:0/2779525883 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x555f471ccaa0 0x555f471cf900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:11.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.052+0000 7ff231fe4640 1 --2- 172.21.15.38:0/2779525883 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x555f471ccaa0 0x555f471cf900 crc :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:11.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.053+0000 7ff231fe4640 1 -- 172.21.15.38:0/2779525883 <== osd.3 v2:172.21.15.38:6808/1286810294 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v132'157 uv156 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7ff22c00b040 con 0x555f471ccaa0 2024-09-21T03:44:11.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.053+0000 7ff237d29f40 1 -- 172.21.15.38:0/2779525883 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:60 13.6 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e132) -- 0x555f471cfe40 con 0x555f471ccaa0 2024-09-21T03:44:11.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.055+0000 7ff231fe4640 1 -- 172.21.15.38:0/2779525883 <== osd.3 v2:172.21.15.38:6808/1286810294 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v132'158 uv158 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7ff22c00b040 con 0x555f471ccaa0 2024-09-21T03:44:11.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.055+0000 7ff237d29f40 1 -- 172.21.15.38:0/2779525883 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:61 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x555f471cfec0 con 0x555f471ccaa0 2024-09-21T03:44:11.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.056+0000 7ff231fe4640 1 -- 172.21.15.38:0/2779525883 <== osd.3 v2:172.21.15.38:6808/1286810294 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v132'159 uv159 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7ff22c00b040 con 0x555f471ccaa0 2024-09-21T03:44:11.058 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-09-21T03:44:11.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.056+0000 7ff237d29f40 1 -- 172.21.15.38:0/2779525883 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:62 13.6 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x555f471d02d0 con 0x555f471ccaa0 2024-09-21T03:44:11.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.057+0000 7ff231fe4640 1 -- 172.21.15.38:0/2779525883 <== osd.3 v2:172.21.15.38:6808/1286810294 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v132'160 uv160 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7ff22c00b040 con 0x555f471ccaa0 2024-09-21T03:44:11.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.057+0000 7ff237d29f40 1 -- 172.21.15.38:0/2779525883 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x555f471d02d0 con 0x555f471c86e0 2024-09-21T03:44:11.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.058+0000 7ff230fe2640 1 -- 172.21.15.38:0/2779525883 <== osd.0 v2:172.21.15.38:6800/2381639433 7 ==== osd_op_reply(63 608.00000000 [create] v132'213 uv212 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ff228013790 con 0x555f471c86e0 2024-09-21T03:44:11.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.058+0000 7ff237d29f40 1 -- 172.21.15.38:0/2779525883 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e132) -- 0x555f471d02d0 con 0x555f471c86e0 2024-09-21T03:44:11.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.059+0000 7ff230fe2640 1 -- 172.21.15.38:0/2779525883 <== osd.0 v2:172.21.15.38:6800/2381639433 8 ==== osd_op_reply(64 608.00000000 [delete] v132'214 uv214 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff228013790 con 0x555f471c86e0 2024-09-21T03:44:11.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.059+0000 7ff237d29f40 1 -- 172.21.15.38:0/2779525883 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x555f471d02d0 con 0x555f471c86e0 2024-09-21T03:44:11.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.060+0000 7ff230fe2640 1 -- 172.21.15.38:0/2779525883 <== osd.0 v2:172.21.15.38:6800/2381639433 9 ==== osd_op_reply(65 608.00000000 [create] v132'215 uv215 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff228013790 con 0x555f471c86e0 2024-09-21T03:44:11.062 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-09-21T03:44:11.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.060+0000 7ff237d29f40 1 -- 172.21.15.38:0/2779525883 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x555f471d12e0 con 0x555f471c86e0 2024-09-21T03:44:11.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.062+0000 7ff230fe2640 1 -- 172.21.15.38:0/2779525883 <== osd.0 v2:172.21.15.38:6800/2381639433 10 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v132'216 uv216 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff228013790 con 0x555f471c86e0 2024-09-21T03:44:11.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.062+0000 7ff237d29f40 1 -- 172.21.15.38:0/2779525883 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x555f471d12e0 con 0x555f471c86e0 2024-09-21T03:44:11.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.063+0000 7ff230fe2640 1 -- 172.21.15.38:0/2779525883 <== osd.0 v2:172.21.15.38:6800/2381639433 11 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v132'217 uv217 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff228013790 con 0x555f471c86e0 2024-09-21T03:44:11.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.063+0000 7ff237d29f40 1 -- 172.21.15.38:0/2779525883 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x555f471d02d0 con 0x555f471c86e0 2024-09-21T03:44:11.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.064+0000 7ff230fe2640 1 -- 172.21.15.38:0/2779525883 <== osd.0 v2:172.21.15.38:6800/2381639433 12 ==== osd_op_reply(68 608.00000000 [omap-set-header] v132'218 uv218 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff228013790 con 0x555f471c86e0 2024-09-21T03:44:11.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.064+0000 7ff237d29f40 1 -- 172.21.15.38:0/2779525883 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x555f471d02d0 con 0x555f471ccaa0 2024-09-21T03:44:11.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.065+0000 7ff231fe4640 1 -- 172.21.15.38:0/2779525883 <== osd.3 v2:172.21.15.38:6808/1286810294 5 ==== osd_op_reply(69 604.00000000 [create] v132'393 uv387 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ff22c00b040 con 0x555f471ccaa0 2024-09-21T03:44:11.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.066+0000 7ff237d29f40 1 -- 172.21.15.38:0/2779525883 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e132) -- 0x555f471d02d0 con 0x555f471ccaa0 2024-09-21T03:44:11.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.067+0000 7ff231fe4640 1 -- 172.21.15.38:0/2779525883 <== osd.3 v2:172.21.15.38:6808/1286810294 6 ==== osd_op_reply(70 604.00000000 [delete] v132'394 uv394 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff22c00b040 con 0x555f471ccaa0 2024-09-21T03:44:11.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.067+0000 7ff237d29f40 1 -- 172.21.15.38:0/2779525883 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x555f471d02d0 con 0x555f471ccaa0 2024-09-21T03:44:11.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.068+0000 7ff231fe4640 1 -- 172.21.15.38:0/2779525883 <== osd.3 v2:172.21.15.38:6808/1286810294 7 ==== osd_op_reply(71 604.00000000 [create] v132'395 uv395 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff22c00b040 con 0x555f471ccaa0 2024-09-21T03:44:11.070 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-09-21T03:44:11.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.068+0000 7ff237d29f40 1 -- 172.21.15.38:0/2779525883 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x555f471d12e0 con 0x555f471ccaa0 2024-09-21T03:44:11.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.069+0000 7ff231fe4640 1 -- 172.21.15.38:0/2779525883 <== osd.3 v2:172.21.15.38:6808/1286810294 8 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v132'396 uv396 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff22c00b040 con 0x555f471ccaa0 2024-09-21T03:44:11.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.069+0000 7ff237d29f40 1 -- 172.21.15.38:0/2779525883 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x555f471d12e0 con 0x555f471ccaa0 2024-09-21T03:44:11.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.071+0000 7ff231fe4640 1 -- 172.21.15.38:0/2779525883 <== osd.3 v2:172.21.15.38:6808/1286810294 9 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v132'397 uv397 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff22c00b040 con 0x555f471ccaa0 2024-09-21T03:44:11.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.071+0000 7ff237d29f40 1 -- 172.21.15.38:0/2779525883 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x555f471d02d0 con 0x555f471ccaa0 2024-09-21T03:44:11.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.072+0000 7ff231fe4640 1 -- 172.21.15.38:0/2779525883 <== osd.3 v2:172.21.15.38:6808/1286810294 10 ==== osd_op_reply(74 604.00000000 [omap-set-header] v132'398 uv398 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff22c00b040 con 0x555f471ccaa0 2024-09-21T03:44:11.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.072+0000 7ff237d29f40 1 -- 172.21.15.38:0/2779525883 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x555f471d02d0 con 0x555f471ccaa0 2024-09-21T03:44:11.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.073+0000 7ff231fe4640 1 -- 172.21.15.38:0/2779525883 <== osd.3 v2:172.21.15.38:6808/1286810294 11 ==== osd_op_reply(75 500.00000000 [create] v132'399 uv392 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ff22c00b040 con 0x555f471ccaa0 2024-09-21T03:44:11.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.073+0000 7ff237d29f40 1 -- 172.21.15.38:0/2779525883 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e132) -- 0x555f471d02d0 con 0x555f471ccaa0 2024-09-21T03:44:11.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.074+0000 7ff231fe4640 1 -- 172.21.15.38:0/2779525883 <== osd.3 v2:172.21.15.38:6808/1286810294 12 ==== osd_op_reply(76 500.00000000 [delete] v132'400 uv400 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff22c00b040 con 0x555f471ccaa0 2024-09-21T03:44:11.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.074+0000 7ff237d29f40 1 -- 172.21.15.38:0/2779525883 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x555f471d02d0 con 0x555f471ccaa0 2024-09-21T03:44:11.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.076+0000 7ff231fe4640 1 -- 172.21.15.38:0/2779525883 <== osd.3 v2:172.21.15.38:6808/1286810294 13 ==== osd_op_reply(77 500.00000000 [create] v132'401 uv401 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff22c00b040 con 0x555f471ccaa0 2024-09-21T03:44:11.077 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-09-21T03:44:11.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.076+0000 7ff237d29f40 1 -- 172.21.15.38:0/2779525883 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x555f471d02d0 con 0x555f471ccaa0 2024-09-21T03:44:11.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.077+0000 7ff231fe4640 1 -- 172.21.15.38:0/2779525883 <== osd.3 v2:172.21.15.38:6808/1286810294 14 ==== osd_op_reply(78 500.00000000 [write 0~90] v132'402 uv402 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff22c00b040 con 0x555f471ccaa0 2024-09-21T03:44:11.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.077+0000 7ff237d29f40 1 -- 172.21.15.38:0/2779525883 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x555f471d02d0 con 0x555f471ccaa0 2024-09-21T03:44:11.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.078+0000 7ff231fe4640 1 -- 172.21.15.38:0/2779525883 <== osd.3 v2:172.21.15.38:6808/1286810294 15 ==== osd_op_reply(79 500.00000000 [omap-set-header] v132'403 uv403 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff22c00b040 con 0x555f471ccaa0 2024-09-21T03:44:11.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.078+0000 7ff237d29f40 1 -- 172.21.15.38:0/2779525883 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:80 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x555f471d02d0 con 0x555f471c86e0 2024-09-21T03:44:11.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.079+0000 7ff230fe2640 1 -- 172.21.15.38:0/2779525883 <== osd.0 v2:172.21.15.38:6800/2381639433 13 ==== osd_op_reply(80 mds_snaptable [create] v132'178 uv177 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7ff228013790 con 0x555f471c86e0 2024-09-21T03:44:11.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.080+0000 7ff237d29f40 1 -- 172.21.15.38:0/2779525883 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:81 13.d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e132) -- 0x555f471d02d0 con 0x555f471c86e0 2024-09-21T03:44:11.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.081+0000 7ff230fe2640 1 -- 172.21.15.38:0/2779525883 <== osd.0 v2:172.21.15.38:6800/2381639433 14 ==== osd_op_reply(81 mds_snaptable [delete] v132'179 uv179 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7ff228013790 con 0x555f471c86e0 2024-09-21T03:44:11.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.081+0000 7ff237d29f40 1 -- 172.21.15.38:0/2779525883 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:82 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x555f471d02d0 con 0x555f471c86e0 2024-09-21T03:44:11.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.082+0000 7ff230fe2640 1 -- 172.21.15.38:0/2779525883 <== osd.0 v2:172.21.15.38:6800/2381639433 15 ==== osd_op_reply(82 mds_snaptable [create] v132'180 uv180 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7ff228013790 con 0x555f471c86e0 2024-09-21T03:44:11.084 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-09-21T03:44:11.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.082+0000 7ff237d29f40 1 -- 172.21.15.38:0/2779525883 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:83 13.d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x555f471d02d0 con 0x555f471c86e0 2024-09-21T03:44:11.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.083+0000 7ff230fe2640 1 -- 172.21.15.38:0/2779525883 <== osd.0 v2:172.21.15.38:6800/2381639433 16 ==== osd_op_reply(83 mds_snaptable [write 0~70] v132'181 uv181 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7ff228013790 con 0x555f471c86e0 2024-09-21T03:44:11.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.083+0000 7ff237d29f40 1 -- 172.21.15.38:0/2779525883 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:84 13.d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x555f471d02d0 con 0x555f471c86e0 2024-09-21T03:44:11.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.084+0000 7ff230fe2640 1 -- 172.21.15.38:0/2779525883 <== osd.0 v2:172.21.15.38:6800/2381639433 17 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v132'182 uv182 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7ff228013790 con 0x555f471c86e0 2024-09-21T03:44:11.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.084+0000 7ff237d29f40 1 --2- 172.21.15.38:0/2779525883 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x555f471d0950 0x555f471d2d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:11.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.084+0000 7ff237d29f40 1 -- 172.21.15.38:0/2779525883 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x555f471d3340 con 0x555f471d0950 2024-09-21T03:44:11.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.084+0000 7ff2317e3640 1 --2- 172.21.15.38:0/2779525883 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x555f471d0950 0x555f471d2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:11.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.085+0000 7ff2317e3640 1 --2- 172.21.15.38:0/2779525883 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x555f471d0950 0x555f471d2d70 crc :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:11.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.086+0000 7ff2317e3640 1 -- 172.21.15.38:0/2779525883 <== osd.1 v2:172.21.15.38:6816/1491818732 1 ==== osd_op_reply(85 605.00000000 [create] v132'213 uv212 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ff22401b070 con 0x555f471d0950 2024-09-21T03:44:11.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.086+0000 7ff237d29f40 1 -- 172.21.15.38:0/2779525883 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e132) -- 0x555f471d3710 con 0x555f471d0950 2024-09-21T03:44:11.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.087+0000 7ff2317e3640 1 -- 172.21.15.38:0/2779525883 <== osd.1 v2:172.21.15.38:6816/1491818732 2 ==== osd_op_reply(86 605.00000000 [delete] v132'214 uv214 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff22401b070 con 0x555f471d0950 2024-09-21T03:44:11.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.087+0000 7ff237d29f40 1 -- 172.21.15.38:0/2779525883 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x555f471d3710 con 0x555f471d0950 2024-09-21T03:44:11.090 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-09-21T03:44:11.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.088+0000 7ff2317e3640 1 -- 172.21.15.38:0/2779525883 <== osd.1 v2:172.21.15.38:6816/1491818732 3 ==== osd_op_reply(87 605.00000000 [create] v132'215 uv215 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff22401b070 con 0x555f471d0950 2024-09-21T03:44:11.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.088+0000 7ff237d29f40 1 -- 172.21.15.38:0/2779525883 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x555f471d4720 con 0x555f471d0950 2024-09-21T03:44:11.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.089+0000 7ff2317e3640 1 -- 172.21.15.38:0/2779525883 <== osd.1 v2:172.21.15.38:6816/1491818732 4 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v132'216 uv216 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff22401b070 con 0x555f471d0950 2024-09-21T03:44:11.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.090+0000 7ff237d29f40 1 -- 172.21.15.38:0/2779525883 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x555f471d4720 con 0x555f471d0950 2024-09-21T03:44:11.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.091+0000 7ff2317e3640 1 -- 172.21.15.38:0/2779525883 <== osd.1 v2:172.21.15.38:6816/1491818732 5 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v132'217 uv217 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff22401b070 con 0x555f471d0950 2024-09-21T03:44:11.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.091+0000 7ff237d29f40 1 -- 172.21.15.38:0/2779525883 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x555f471d3710 con 0x555f471d0950 2024-09-21T03:44:11.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.092+0000 7ff2317e3640 1 -- 172.21.15.38:0/2779525883 <== osd.1 v2:172.21.15.38:6816/1491818732 6 ==== osd_op_reply(90 605.00000000 [omap-set-header] v132'218 uv218 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff22401b070 con 0x555f471d0950 2024-09-21T03:44:11.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.092+0000 7ff237d29f40 1 -- 172.21.15.38:0/2779525883 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x555f471d3710 con 0x555f471d0950 2024-09-21T03:44:11.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.093+0000 7ff2317e3640 1 -- 172.21.15.38:0/2779525883 <== osd.1 v2:172.21.15.38:6816/1491818732 7 ==== osd_op_reply(91 mds0_inotable [create] v132'355 uv349 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7ff22401b070 con 0x555f471d0950 2024-09-21T03:44:11.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.093+0000 7ff237d29f40 1 -- 172.21.15.38:0/2779525883 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e132) -- 0x555f471d3710 con 0x555f471d0950 2024-09-21T03:44:11.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.094+0000 7ff2317e3640 1 -- 172.21.15.38:0/2779525883 <== osd.1 v2:172.21.15.38:6816/1491818732 8 ==== osd_op_reply(92 mds0_inotable [delete] v132'356 uv356 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7ff22401b070 con 0x555f471d0950 2024-09-21T03:44:11.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.094+0000 7ff237d29f40 1 -- 172.21.15.38:0/2779525883 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x555f471d3710 con 0x555f471d0950 2024-09-21T03:44:11.097 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-09-21T03:44:11.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.095+0000 7ff2317e3640 1 -- 172.21.15.38:0/2779525883 <== osd.1 v2:172.21.15.38:6816/1491818732 9 ==== osd_op_reply(93 mds0_inotable [create] v132'357 uv357 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7ff22401b070 con 0x555f471d0950 2024-09-21T03:44:11.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.095+0000 7ff237d29f40 1 -- 172.21.15.38:0/2779525883 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:94 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x555f471cbd90 con 0x555f471d0950 2024-09-21T03:44:11.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.096+0000 7ff2317e3640 1 -- 172.21.15.38:0/2779525883 <== osd.1 v2:172.21.15.38:6816/1491818732 10 ==== osd_op_reply(94 mds0_inotable [write 0~34] v132'358 uv358 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7ff22401b070 con 0x555f471d0950 2024-09-21T03:44:11.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.096+0000 7ff237d29f40 1 -- 172.21.15.38:0/2779525883 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:95 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x555f471cbd90 con 0x555f471d0950 2024-09-21T03:44:11.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.097+0000 7ff2317e3640 1 -- 172.21.15.38:0/2779525883 <== osd.1 v2:172.21.15.38:6816/1491818732 11 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v132'359 uv359 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7ff22401b070 con 0x555f471d0950 2024-09-21T03:44:11.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.097+0000 7ff237d29f40 1 -- 172.21.15.38:0/2779525883 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:96 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x555f471cbd90 con 0x555f471d0950 2024-09-21T03:44:11.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.098+0000 7ff2317e3640 1 -- 172.21.15.38:0/2779525883 <== osd.1 v2:172.21.15.38:6816/1491818732 12 ==== osd_op_reply(96 100.00000000 [create] v132'360 uv354 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ff22401b070 con 0x555f471d0950 2024-09-21T03:44:11.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.098+0000 7ff237d29f40 1 -- 172.21.15.38:0/2779525883 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:97 13.13 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e132) -- 0x555f471cbd90 con 0x555f471d0950 2024-09-21T03:44:11.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.099+0000 7ff2317e3640 1 -- 172.21.15.38:0/2779525883 <== osd.1 v2:172.21.15.38:6816/1491818732 13 ==== osd_op_reply(97 100.00000000 [delete] v132'361 uv361 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff22401b070 con 0x555f471d0950 2024-09-21T03:44:11.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.099+0000 7ff237d29f40 1 -- 172.21.15.38:0/2779525883 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:98 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x555f471cbd90 con 0x555f471d0950 2024-09-21T03:44:11.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.100+0000 7ff2317e3640 1 -- 172.21.15.38:0/2779525883 <== osd.1 v2:172.21.15.38:6816/1491818732 14 ==== osd_op_reply(98 100.00000000 [create] v132'362 uv362 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff22401b070 con 0x555f471d0950 2024-09-21T03:44:11.102 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-09-21T03:44:11.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.100+0000 7ff237d29f40 1 -- 172.21.15.38:0/2779525883 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:99 13.13 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x555f471cbd90 con 0x555f471d0950 2024-09-21T03:44:11.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.101+0000 7ff2317e3640 1 -- 172.21.15.38:0/2779525883 <== osd.1 v2:172.21.15.38:6816/1491818732 15 ==== osd_op_reply(99 100.00000000 [omap-set-header] v132'363 uv363 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff22401b070 con 0x555f471d0950 2024-09-21T03:44:11.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.102+0000 7ff237d29f40 1 -- 172.21.15.38:0/2779525883 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:100 13.13 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x555f471d6560 con 0x555f471d0950 2024-09-21T03:44:11.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.103+0000 7ff2317e3640 1 -- 172.21.15.38:0/2779525883 <== osd.1 v2:172.21.15.38:6816/1491818732 16 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v132'364 uv364 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff22401b070 con 0x555f471d0950 2024-09-21T03:44:11.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.103+0000 7ff237d29f40 1 -- 172.21.15.38:0/2779525883 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x555f471d3bc0 con 0x555f471ccaa0 2024-09-21T03:44:11.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.104+0000 7ff231fe4640 1 -- 172.21.15.38:0/2779525883 <== osd.3 v2:172.21.15.38:6808/1286810294 16 ==== osd_op_reply(101 mds0_sessionmap [create] v132'157 uv155 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7ff22c00b040 con 0x555f471ccaa0 2024-09-21T03:44:11.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.104+0000 7ff237d29f40 1 -- 172.21.15.38:0/2779525883 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e132) -- 0x555f471d3bc0 con 0x555f471ccaa0 2024-09-21T03:44:11.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.105+0000 7ff231fe4640 1 -- 172.21.15.38:0/2779525883 <== osd.3 v2:172.21.15.38:6808/1286810294 17 ==== osd_op_reply(102 mds0_sessionmap [delete] v132'158 uv158 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7ff22c00b040 con 0x555f471ccaa0 2024-09-21T03:44:11.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.105+0000 7ff237d29f40 1 -- 172.21.15.38:0/2779525883 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:103 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x555f471d3bc0 con 0x555f471ccaa0 2024-09-21T03:44:11.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.106+0000 7ff231fe4640 1 -- 172.21.15.38:0/2779525883 <== osd.3 v2:172.21.15.38:6808/1286810294 18 ==== osd_op_reply(103 mds0_sessionmap [create] v132'159 uv159 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7ff22c00b040 con 0x555f471ccaa0 2024-09-21T03:44:11.108 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-09-21T03:44:11.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.106+0000 7ff237d29f40 1 -- 172.21.15.38:0/2779525883 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:104 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x555f471d3bc0 con 0x555f471ccaa0 2024-09-21T03:44:11.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.107+0000 7ff231fe4640 1 -- 172.21.15.38:0/2779525883 <== osd.3 v2:172.21.15.38:6808/1286810294 19 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v132'160 uv160 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7ff22c00b040 con 0x555f471ccaa0 2024-09-21T03:44:11.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.107+0000 7ff237d29f40 1 -- 172.21.15.38:0/2779525883 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x555f471d3bc0 con 0x555f471ccaa0 2024-09-21T03:44:11.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.108+0000 7ff231fe4640 1 -- 172.21.15.38:0/2779525883 <== osd.3 v2:172.21.15.38:6808/1286810294 20 ==== osd_op_reply(105 609.00000000 [create] v132'213 uv212 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ff22c00b040 con 0x555f471ccaa0 2024-09-21T03:44:11.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.108+0000 7ff237d29f40 1 -- 172.21.15.38:0/2779525883 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e132) -- 0x555f471d3bc0 con 0x555f471ccaa0 2024-09-21T03:44:11.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.109+0000 7ff231fe4640 1 -- 172.21.15.38:0/2779525883 <== osd.3 v2:172.21.15.38:6808/1286810294 21 ==== osd_op_reply(106 609.00000000 [delete] v132'214 uv214 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff22c00b040 con 0x555f471ccaa0 2024-09-21T03:44:11.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.109+0000 7ff237d29f40 1 -- 172.21.15.38:0/2779525883 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x555f471d3bc0 con 0x555f471ccaa0 2024-09-21T03:44:11.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.110+0000 7ff231fe4640 1 -- 172.21.15.38:0/2779525883 <== osd.3 v2:172.21.15.38:6808/1286810294 22 ==== osd_op_reply(107 609.00000000 [create] v132'215 uv215 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff22c00b040 con 0x555f471ccaa0 2024-09-21T03:44:11.112 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-09-21T03:44:11.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.110+0000 7ff237d29f40 1 -- 172.21.15.38:0/2779525883 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x555f471d3bc0 con 0x555f471ccaa0 2024-09-21T03:44:11.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.111+0000 7ff231fe4640 1 -- 172.21.15.38:0/2779525883 <== osd.3 v2:172.21.15.38:6808/1286810294 23 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v132'216 uv216 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff22c00b040 con 0x555f471ccaa0 2024-09-21T03:44:11.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.111+0000 7ff237d29f40 1 -- 172.21.15.38:0/2779525883 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:109 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x555f471d3bc0 con 0x555f471ccaa0 2024-09-21T03:44:11.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.112+0000 7ff231fe4640 1 -- 172.21.15.38:0/2779525883 <== osd.3 v2:172.21.15.38:6808/1286810294 24 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v132'217 uv217 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff22c00b040 con 0x555f471ccaa0 2024-09-21T03:44:11.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.112+0000 7ff237d29f40 1 -- 172.21.15.38:0/2779525883 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:110 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x555f471d3bc0 con 0x555f471ccaa0 2024-09-21T03:44:11.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.113+0000 7ff231fe4640 1 -- 172.21.15.38:0/2779525883 <== osd.3 v2:172.21.15.38:6808/1286810294 25 ==== osd_op_reply(110 609.00000000 [omap-set-header] v132'218 uv218 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff22c00b040 con 0x555f471ccaa0 2024-09-21T03:44:11.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.113+0000 7ff237d29f40 1 --2- 172.21.15.38:0/2779525883 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x555f471d44c0 0x555f471d6c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:11.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.113+0000 7ff237d29f40 1 -- 172.21.15.38:0/2779525883 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x555f471d7140 con 0x555f471d44c0 2024-09-21T03:44:11.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.114+0000 7ff230fe2640 1 --2- 172.21.15.38:0/2779525883 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x555f471d44c0 0x555f471d6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:11.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.114+0000 7ff230fe2640 1 --2- 172.21.15.38:0/2779525883 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x555f471d44c0 0x555f471d6c00 crc :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:11.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.115+0000 7ff230fe2640 1 -- 172.21.15.38:0/2779525883 <== osd.2 v2:172.21.15.38:6824/2577858902 1 ==== osd_op_reply(111 400.00000000 [create] v132'181 uv180 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ff228013790 con 0x555f471d44c0 2024-09-21T03:44:11.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.115+0000 7ff237d29f40 1 -- 172.21.15.38:0/2779525883 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e132) -- 0x555f471d7140 con 0x555f471d44c0 2024-09-21T03:44:11.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.116+0000 7ff230fe2640 1 -- 172.21.15.38:0/2779525883 <== osd.2 v2:172.21.15.38:6824/2577858902 2 ==== osd_op_reply(112 400.00000000 [delete] v132'182 uv182 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff228013790 con 0x555f471d44c0 2024-09-21T03:44:11.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.116+0000 7ff237d29f40 1 -- 172.21.15.38:0/2779525883 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x555f471d7140 con 0x555f471d44c0 2024-09-21T03:44:11.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.117+0000 7ff230fe2640 1 -- 172.21.15.38:0/2779525883 <== osd.2 v2:172.21.15.38:6824/2577858902 3 ==== osd_op_reply(113 400.00000000 [create] v132'183 uv183 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff228013790 con 0x555f471d44c0 2024-09-21T03:44:11.119 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-09-21T03:44:11.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.117+0000 7ff237d29f40 1 -- 172.21.15.38:0/2779525883 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:114 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x555f471d7140 con 0x555f471d44c0 2024-09-21T03:44:11.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.118+0000 7ff230fe2640 1 -- 172.21.15.38:0/2779525883 <== osd.2 v2:172.21.15.38:6824/2577858902 4 ==== osd_op_reply(114 400.00000000 [write 0~22] v132'184 uv184 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff228013790 con 0x555f471d44c0 2024-09-21T03:44:11.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.118+0000 7ff237d29f40 1 -- 172.21.15.38:0/2779525883 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:115 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x555f471d7140 con 0x555f471d44c0 2024-09-21T03:44:11.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.120+0000 7ff230fe2640 1 -- 172.21.15.38:0/2779525883 <== osd.2 v2:172.21.15.38:6824/2577858902 5 ==== osd_op_reply(115 400.00000000 [omap-set-header] v132'185 uv185 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff228013790 con 0x555f471d44c0 2024-09-21T03:44:11.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.120+0000 7ff237d29f40 1 -- 172.21.15.38:0/2779525883 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:116 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x555f471d7140 con 0x555f471d0950 2024-09-21T03:44:11.123 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.121+0000 7ff2317e3640 1 -- 172.21.15.38:0/2779525883 <== osd.1 v2:172.21.15.38:6816/1491818732 17 ==== osd_op_reply(116 100.00000000.inode [create] v132'442 uv433 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7ff22401b070 con 0x555f471d0950 2024-09-21T03:44:11.123 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.121+0000 7ff237d29f40 1 -- 172.21.15.38:0/2779525883 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:117 13.1f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e132) -- 0x555f471d7140 con 0x555f471d0950 2024-09-21T03:44:11.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.122+0000 7ff2317e3640 1 -- 172.21.15.38:0/2779525883 <== osd.1 v2:172.21.15.38:6816/1491818732 18 ==== osd_op_reply(117 100.00000000.inode [delete] v132'443 uv443 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7ff22401b070 con 0x555f471d0950 2024-09-21T03:44:11.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.122+0000 7ff237d29f40 1 -- 172.21.15.38:0/2779525883 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:118 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x555f471d7140 con 0x555f471d0950 2024-09-21T03:44:11.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.123+0000 7ff2317e3640 1 -- 172.21.15.38:0/2779525883 <== osd.1 v2:172.21.15.38:6816/1491818732 19 ==== osd_op_reply(118 100.00000000.inode [create] v132'444 uv444 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7ff22401b070 con 0x555f471d0950 2024-09-21T03:44:11.125 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-09-21T03:44:11.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.123+0000 7ff237d29f40 1 -- 172.21.15.38:0/2779525883 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:119 13.1f 13:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x555f471d7140 con 0x555f471d0950 2024-09-21T03:44:11.126 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.124+0000 7ff2317e3640 1 -- 172.21.15.38:0/2779525883 <== osd.1 v2:172.21.15.38:6816/1491818732 20 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v132'445 uv445 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7ff22401b070 con 0x555f471d0950 2024-09-21T03:44:11.126 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.124+0000 7ff237d29f40 1 -- 172.21.15.38:0/2779525883 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:120 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x555f471d7140 con 0x555f471d0950 2024-09-21T03:44:11.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.125+0000 7ff2317e3640 1 -- 172.21.15.38:0/2779525883 <== osd.1 v2:172.21.15.38:6816/1491818732 21 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v132'446 uv446 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7ff22401b070 con 0x555f471d0950 2024-09-21T03:44:11.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.125+0000 7ff237d29f40 1 -- 172.21.15.38:0/2779525883 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:121 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x555f471d7140 con 0x555f471d0950 2024-09-21T03:44:11.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.126+0000 7ff2317e3640 1 -- 172.21.15.38:0/2779525883 <== osd.1 v2:172.21.15.38:6816/1491818732 22 ==== osd_op_reply(121 1.00000000 [create] v132'447 uv440 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7ff22401b070 con 0x555f471d0950 2024-09-21T03:44:11.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.127+0000 7ff237d29f40 1 -- 172.21.15.38:0/2779525883 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:122 13.1f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e132) -- 0x555f471d7140 con 0x555f471d0950 2024-09-21T03:44:11.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.128+0000 7ff2317e3640 1 -- 172.21.15.38:0/2779525883 <== osd.1 v2:172.21.15.38:6816/1491818732 23 ==== osd_op_reply(122 1.00000000 [delete] v132'448 uv448 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7ff22401b070 con 0x555f471d0950 2024-09-21T03:44:11.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.128+0000 7ff237d29f40 1 -- 172.21.15.38:0/2779525883 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:123 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x555f471d7140 con 0x555f471d0950 2024-09-21T03:44:11.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.129+0000 7ff2317e3640 1 -- 172.21.15.38:0/2779525883 <== osd.1 v2:172.21.15.38:6816/1491818732 24 ==== osd_op_reply(123 1.00000000 [create] v132'449 uv449 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7ff22401b070 con 0x555f471d0950 2024-09-21T03:44:11.131 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-09-21T03:44:11.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.129+0000 7ff237d29f40 1 -- 172.21.15.38:0/2779525883 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:124 13.1f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x555f471d8150 con 0x555f471d0950 2024-09-21T03:44:11.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.130+0000 7ff2317e3640 1 -- 172.21.15.38:0/2779525883 <== osd.1 v2:172.21.15.38:6816/1491818732 25 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v132'450 uv450 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7ff22401b070 con 0x555f471d0950 2024-09-21T03:44:11.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.130+0000 7ff237d29f40 1 -- 172.21.15.38:0/2779525883 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:125 13.1f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x555f471d8150 con 0x555f471d0950 2024-09-21T03:44:11.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.131+0000 7ff2317e3640 1 -- 172.21.15.38:0/2779525883 <== osd.1 v2:172.21.15.38:6816/1491818732 26 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v132'451 uv451 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7ff22401b070 con 0x555f471d0950 2024-09-21T03:44:11.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.131+0000 7ff237d29f40 1 -- 172.21.15.38:0/2779525883 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:126 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x555f471d7140 con 0x555f471d0950 2024-09-21T03:44:11.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.132+0000 7ff2317e3640 1 -- 172.21.15.38:0/2779525883 <== osd.1 v2:172.21.15.38:6816/1491818732 27 ==== osd_op_reply(126 1.00000000 [omap-set-header] v132'452 uv452 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7ff22401b070 con 0x555f471d0950 2024-09-21T03:44:11.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.132+0000 7ff237d29f40 1 -- 172.21.15.38:0/2779525883 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:127 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=584b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x555f471d7140 con 0x555f471d0950 2024-09-21T03:44:11.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.133+0000 7ff2317e3640 1 -- 172.21.15.38:0/2779525883 <== osd.1 v2:172.21.15.38:6816/1491818732 28 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v132'453 uv453 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7ff22401b070 con 0x555f471d0950 2024-09-21T03:44:11.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.133+0000 7ff237d29f40 1 -- 172.21.15.38:0/2779525883 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x555f471c86e0 msgr2=0x555f471cab00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:11.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.133+0000 7ff237d29f40 1 --2- 172.21.15.38:0/2779525883 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x555f471c86e0 0x555f471cab00 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:11.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.133+0000 7ff237d29f40 1 -- 172.21.15.38:0/2779525883 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x555f471d0950 msgr2=0x555f471d2d70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:11.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.133+0000 7ff237d29f40 1 --2- 172.21.15.38:0/2779525883 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x555f471d0950 0x555f471d2d70 crc :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:11.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.133+0000 7ff237d29f40 1 -- 172.21.15.38:0/2779525883 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x555f471d44c0 msgr2=0x555f471d6c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:11.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.133+0000 7ff237d29f40 1 --2- 172.21.15.38:0/2779525883 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x555f471d44c0 0x555f471d6c00 crc :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:11.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.133+0000 7ff237d29f40 1 -- 172.21.15.38:0/2779525883 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x555f471ccaa0 msgr2=0x555f471cf900 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:11.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.133+0000 7ff237d29f40 1 --2- 172.21.15.38:0/2779525883 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x555f471ccaa0 0x555f471cf900 crc :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:11.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.133+0000 7ff231fe4640 1 -- 172.21.15.38:0/2779525883 reap_dead start 2024-09-21T03:44:11.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.133+0000 7ff237d29f40 1 -- 172.21.15.38:0/2779525883 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x555f471c2ce0 msgr2=0x555f471c5d70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:11.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.133+0000 7ff237d29f40 1 --2- 172.21.15.38:0/2779525883 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x555f471c2ce0 0x555f471c5d70 crc :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:11.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.133+0000 7ff231fe4640 1 -- 172.21.15.38:0/2779525883 reap_dead start 2024-09-21T03:44:11.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.133+0000 7ff237d29f40 1 -- 172.21.15.38:0/2779525883 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x555f471a8b90 msgr2=0x555f471a8fa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:11.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.133+0000 7ff237d29f40 1 --2- 172.21.15.38:0/2779525883 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x555f471a8b90 0x555f471a8fa0 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:11.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.133+0000 7ff237d29f40 1 -- 172.21.15.38:0/2779525883 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff2140775a0 msgr2=0x7ff214079ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:11.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.133+0000 7ff237d29f40 1 --2- 172.21.15.38:0/2779525883 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff2140775a0 0x7ff214079ab0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x555f471a9fb0 tx=0x7ff228003610 comp rx=0 tx=0).stop 2024-09-21T03:44:11.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.133+0000 7ff237d29f40 1 -- 172.21.15.38:0/2779525883 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555f4704ad00 msgr2=0x555f47101130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:11.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.133+0000 7ff237d29f40 1 --2- 172.21.15.38:0/2779525883 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555f4704ad00 0x555f47101130 secure :-1 s=READY pgs=2168 cs=0 l=1 rev1=1 crypto rx=0x7ff224004160 tx=0x7ff2240156f0 comp rx=0 tx=0).stop 2024-09-21T03:44:11.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.133+0000 7ff237d29f40 1 -- 172.21.15.38:0/2779525883 shutdown_connections 2024-09-21T03:44:11.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.133+0000 7ff237d29f40 1 --2- 172.21.15.38:0/2779525883 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x555f471a8b90 0x555f471a8fa0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:11.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.134+0000 7ff237d29f40 1 --2- 172.21.15.38:0/2779525883 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff2140775a0 0x7ff214079ab0 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:11.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.134+0000 7ff237d29f40 1 --2- 172.21.15.38:0/2779525883 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555f4704ad00 0x555f47101130 unknown :-1 s=CLOSED pgs=2168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:11.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.134+0000 7ff237d29f40 1 -- 172.21.15.38:0/2779525883 >> 172.21.15.38:0/2779525883 conn(0x555f46f69ac0 msgr2=0x555f47048950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:11.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.134+0000 7ff237d29f40 1 -- 172.21.15.38:0/2779525883 shutdown_connections 2024-09-21T03:44:11.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.134+0000 7ff237d29f40 1 -- 172.21.15.38:0/2779525883 wait complete. 2024-09-21T03:44:11.139 DEBUG:teuthology.orchestra.run.smithi038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata setomapheader mds0_openfiles.0 deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef 2024-09-21T03:44:11.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.193+0000 7ff337611f40 1 Processor -- start 2024-09-21T03:44:11.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.193+0000 7ff337611f40 1 -- start start 2024-09-21T03:44:11.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.194+0000 7ff337611f40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56128d3f4d00 0x56128d4c6750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:11.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.194+0000 7ff337611f40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56128d420e30 0x56128d4ca350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:11.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.194+0000 7ff337611f40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56128d4af950 0x56128d4ca890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:11.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.194+0000 7ff337611f40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x56128d342d60 con 0x56128d3f4d00 2024-09-21T03:44:11.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.194+0000 7ff337611f40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x56128d3a1430 con 0x56128d4af950 2024-09-21T03:44:11.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.194+0000 7ff337611f40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x56128d43c7b0 con 0x56128d420e30 2024-09-21T03:44:11.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.194+0000 7ff3310cd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56128d3f4d00 0x56128d4c6750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:11.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.194+0000 7ff3310cd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56128d3f4d00 0x56128d4c6750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:45604/0 (socket says 172.21.15.38:45604) 2024-09-21T03:44:11.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.194+0000 7ff3310cd640 1 -- 172.21.15.38:0/2085712621 learned_addr learned my addr 172.21.15.38:0/2085712621 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:44:11.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.194+0000 7ff3318ce640 1 --2- 172.21.15.38:0/2085712621 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56128d4af950 0x56128d4ca890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:11.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.195+0000 7ff3308cc640 1 --2- 172.21.15.38:0/2085712621 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56128d420e30 0x56128d4ca350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:11.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.195+0000 7ff3310cd640 1 -- 172.21.15.38:0/2085712621 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56128d420e30 msgr2=0x56128d4ca350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:44:11.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.195+0000 7ff3310cd640 1 --2- 172.21.15.38:0/2085712621 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56128d420e30 0x56128d4ca350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:11.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.195+0000 7ff3310cd640 1 -- 172.21.15.38:0/2085712621 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56128d4af950 msgr2=0x56128d4ca890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:44:11.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.195+0000 7ff3310cd640 1 --2- 172.21.15.38:0/2085712621 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56128d4af950 0x56128d4ca890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:11.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.195+0000 7ff3310cd640 1 -- 172.21.15.38:0/2085712621 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56128d4a9860 con 0x56128d3f4d00 2024-09-21T03:44:11.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.195+0000 7ff3310cd640 1 --2- 172.21.15.38:0/2085712621 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56128d3f4d00 0x56128d4c6750 secure :-1 s=READY pgs=2170 cs=0 l=1 rev1=1 crypto rx=0x7ff324025f40 tx=0x7ff324002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67686998523bd27 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:11.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.195+0000 7ff3237fe640 1 -- 172.21.15.38:0/2085712621 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff32404d9a0 con 0x56128d3f4d00 2024-09-21T03:44:11.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.195+0000 7ff3237fe640 1 -- 172.21.15.38:0/2085712621 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff324056670 con 0x56128d3f4d00 2024-09-21T03:44:11.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.195+0000 7ff3237fe640 1 -- 172.21.15.38:0/2085712621 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff324056ac0 con 0x56128d3f4d00 2024-09-21T03:44:11.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.196+0000 7ff337611f40 1 -- 172.21.15.38:0/2085712621 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56128d3f4d00 msgr2=0x56128d4c6750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:11.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.196+0000 7ff337611f40 1 --2- 172.21.15.38:0/2085712621 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56128d3f4d00 0x56128d4c6750 secure :-1 s=READY pgs=2170 cs=0 l=1 rev1=1 crypto rx=0x7ff324025f40 tx=0x7ff324002fb0 comp rx=0 tx=0).stop 2024-09-21T03:44:11.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.196+0000 7ff337611f40 1 -- 172.21.15.38:0/2085712621 shutdown_connections 2024-09-21T03:44:11.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.196+0000 7ff337611f40 1 --2- 172.21.15.38:0/2085712621 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56128d4af950 0x56128d4ca890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:11.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.196+0000 7ff337611f40 1 --2- 172.21.15.38:0/2085712621 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56128d420e30 0x56128d4ca350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:11.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.196+0000 7ff337611f40 1 --2- 172.21.15.38:0/2085712621 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56128d3f4d00 0x56128d4c6750 unknown :-1 s=CLOSED pgs=2170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:11.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.196+0000 7ff337611f40 1 -- 172.21.15.38:0/2085712621 >> 172.21.15.38:0/2085712621 conn(0x56128d313ac0 msgr2=0x56128d4ab700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:11.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.196+0000 7ff337611f40 1 -- 172.21.15.38:0/2085712621 shutdown_connections 2024-09-21T03:44:11.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.196+0000 7ff337611f40 1 -- 172.21.15.38:0/2085712621 wait complete. 2024-09-21T03:44:11.202 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.197+0000 7ff337611f40 1 Processor -- start 2024-09-21T03:44:11.202 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.197+0000 7ff337611f40 1 -- start start 2024-09-21T03:44:11.202 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.198+0000 7ff337611f40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56128d3f4d00 0x56128d4c3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:11.202 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.198+0000 7ff337611f40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56128d420e30 0x56128d5576e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:11.202 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.198+0000 7ff337611f40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56128d4af950 0x56128d557c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:11.202 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.198+0000 7ff337611f40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x56128d4ad3a0 con 0x56128d420e30 2024-09-21T03:44:11.202 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.198+0000 7ff337611f40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x56128d4c7430 con 0x56128d4af950 2024-09-21T03:44:11.202 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.198+0000 7ff337611f40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x56128d4c76d0 con 0x56128d3f4d00 2024-09-21T03:44:11.202 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.198+0000 7ff3308cc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56128d420e30 0x56128d5576e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:11.202 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.198+0000 7ff3308cc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56128d420e30 0x56128d5576e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:45616/0 (socket says 172.21.15.38:45616) 2024-09-21T03:44:11.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.198+0000 7ff3310cd640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56128d3f4d00 0x56128d4c3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:11.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.198+0000 7ff3308cc640 1 -- 172.21.15.38:0/288527672 learned_addr learned my addr 172.21.15.38:0/288527672 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:44:11.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.198+0000 7ff3318ce640 1 --2- 172.21.15.38:0/288527672 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56128d4af950 0x56128d557c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:11.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.199+0000 7ff3308cc640 1 -- 172.21.15.38:0/288527672 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56128d3f4d00 msgr2=0x56128d4c3460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:11.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.199+0000 7ff3308cc640 1 --2- 172.21.15.38:0/288527672 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56128d3f4d00 0x56128d4c3460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:11.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.199+0000 7ff3308cc640 1 -- 172.21.15.38:0/288527672 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56128d4af950 msgr2=0x56128d557c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:11.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.199+0000 7ff3308cc640 1 --2- 172.21.15.38:0/288527672 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56128d4af950 0x56128d557c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:11.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.199+0000 7ff3308cc640 1 -- 172.21.15.38:0/288527672 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56128d4a3e10 con 0x56128d420e30 2024-09-21T03:44:11.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.199+0000 7ff3310cd640 1 --2- 172.21.15.38:0/288527672 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56128d3f4d00 0x56128d4c3460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:44:11.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.199+0000 7ff3308cc640 1 --2- 172.21.15.38:0/288527672 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56128d420e30 0x56128d5576e0 secure :-1 s=READY pgs=2171 cs=0 l=1 rev1=1 crypto rx=0x7ff328002cf0 tx=0x7ff32800c620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:11.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.199+0000 7ff3227fc640 1 -- 172.21.15.38:0/288527672 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff328015020 con 0x56128d420e30 2024-09-21T03:44:11.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.199+0000 7ff3227fc640 1 -- 172.21.15.38:0/288527672 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff328011450 con 0x56128d420e30 2024-09-21T03:44:11.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.199+0000 7ff337611f40 1 -- 172.21.15.38:0/288527672 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56128d5584f0 con 0x56128d420e30 2024-09-21T03:44:11.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.199+0000 7ff3227fc640 1 -- 172.21.15.38:0/288527672 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3280118a0 con 0x56128d420e30 2024-09-21T03:44:11.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.199+0000 7ff337611f40 1 -- 172.21.15.38:0/288527672 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56128d4c5030 con 0x56128d420e30 2024-09-21T03:44:11.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.201+0000 7ff3227fc640 1 -- 172.21.15.38:0/288527672 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ff328018020 con 0x56128d420e30 2024-09-21T03:44:11.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.201+0000 7ff3227fc640 1 --2- 172.21.15.38:0/288527672 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff314077570 0x7ff314079a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:11.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.201+0000 7ff3227fc640 1 -- 172.21.15.38:0/288527672 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 9400+0+0 (secure 0 0 0) 0x7ff32809a840 con 0x56128d420e30 2024-09-21T03:44:11.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.201+0000 7ff3310cd640 1 --2- 172.21.15.38:0/288527672 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff314077570 0x7ff314079a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:11.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.201+0000 7ff337611f40 1 --2- 172.21.15.38:0/288527672 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x56128d4c5b10 0x56128d56c050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:11.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.201+0000 7ff337611f40 1 -- 172.21.15.38:0/288527672 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:1 13.6 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=80b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x56128d55a600 con 0x56128d4c5b10 2024-09-21T03:44:11.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.202+0000 7ff3318ce640 1 --2- 172.21.15.38:0/288527672 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x56128d4c5b10 0x56128d56c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:11.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.202+0000 7ff3310cd640 1 --2- 172.21.15.38:0/288527672 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff314077570 0x7ff314079a80 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7ff324062000 tx=0x7ff324004940 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:11.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.202+0000 7ff3318ce640 1 --2- 172.21.15.38:0/288527672 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x56128d4c5b10 0x56128d56c050 crc :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:11.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.203+0000 7ff3318ce640 1 -- 172.21.15.38:0/288527672 <== osd.3 v2:172.21.15.38:6808/1286810294 1 ==== osd_op_reply(1 mds0_openfiles.0 [omap-set-header] v132'161 uv161 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7ff32c003040 con 0x56128d4c5b10 2024-09-21T03:44:11.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.203+0000 7ff337611f40 1 -- 172.21.15.38:0/288527672 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x56128d4c5b10 msgr2=0x56128d56c050 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:11.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.203+0000 7ff337611f40 1 --2- 172.21.15.38:0/288527672 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x56128d4c5b10 0x56128d56c050 crc :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:11.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.203+0000 7ff337611f40 1 -- 172.21.15.38:0/288527672 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff314077570 msgr2=0x7ff314079a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:11.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.203+0000 7ff337611f40 1 --2- 172.21.15.38:0/288527672 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff314077570 0x7ff314079a80 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7ff324062000 tx=0x7ff324004940 comp rx=0 tx=0).stop 2024-09-21T03:44:11.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.203+0000 7ff337611f40 1 -- 172.21.15.38:0/288527672 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56128d420e30 msgr2=0x56128d5576e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:11.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.203+0000 7ff337611f40 1 --2- 172.21.15.38:0/288527672 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56128d420e30 0x56128d5576e0 secure :-1 s=READY pgs=2171 cs=0 l=1 rev1=1 crypto rx=0x7ff328002cf0 tx=0x7ff32800c620 comp rx=0 tx=0).stop 2024-09-21T03:44:11.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.203+0000 7ff3318ce640 1 -- 172.21.15.38:0/288527672 reap_dead start 2024-09-21T03:44:11.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.203+0000 7ff337611f40 1 -- 172.21.15.38:0/288527672 shutdown_connections 2024-09-21T03:44:11.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.203+0000 7ff337611f40 1 -- 172.21.15.38:0/288527672 >> 172.21.15.38:0/288527672 conn(0x56128d313ac0 msgr2=0x56128d4ab6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:11.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.203+0000 7ff337611f40 1 -- 172.21.15.38:0/288527672 shutdown_connections 2024-09-21T03:44:11.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.203+0000 7ff337611f40 1 -- 172.21.15.38:0/288527672 wait complete. 2024-09-21T03:44:11.208 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-21T03:44:11.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.326+0000 7fbb363a8640 1 -- 172.21.15.38:0/1091637383 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbb3015ee80 msgr2=0x7fbb300d0170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:11.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.326+0000 7fbb363a8640 1 --2- 172.21.15.38:0/1091637383 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbb3015ee80 0x7fbb300d0170 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fbb20009830 tx=0x7fbb2002a4a0 comp rx=0 tx=0).stop 2024-09-21T03:44:11.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.326+0000 7fbb363a8640 1 -- 172.21.15.38:0/1091637383 shutdown_connections 2024-09-21T03:44:11.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.326+0000 7fbb363a8640 1 --2- 172.21.15.38:0/1091637383 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbb3015ee80 0x7fbb300d0170 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:11.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.326+0000 7fbb363a8640 1 -- 172.21.15.38:0/1091637383 >> 172.21.15.38:0/1091637383 conn(0x7fbb3009d830 msgr2=0x7fbb300b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:11.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.326+0000 7fbb363a8640 1 -- 172.21.15.38:0/1091637383 shutdown_connections 2024-09-21T03:44:11.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.326+0000 7fbb363a8640 1 -- 172.21.15.38:0/1091637383 wait complete. 2024-09-21T03:44:11.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.328+0000 7fbb363a8640 1 Processor -- start 2024-09-21T03:44:11.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.328+0000 7fbb363a8640 1 -- start start 2024-09-21T03:44:11.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.329+0000 7fbb363a8640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbb300b9ad0 0x7fbb300b21b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:11.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.329+0000 7fbb363a8640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbb300ceb90 0x7fbb30157b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:11.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.329+0000 7fbb363a8640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbb3015ee80 0x7fbb301f9510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:11.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.329+0000 7fbb363a8640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fbb300b6910 con 0x7fbb3015ee80 2024-09-21T03:44:11.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.329+0000 7fbb363a8640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fbb300b6790 con 0x7fbb300b9ad0 2024-09-21T03:44:11.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.329+0000 7fbb363a8640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fbb300b6a90 con 0x7fbb300ceb90 2024-09-21T03:44:11.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.330+0000 7fbb34945640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbb3015ee80 0x7fbb301f9510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:11.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.330+0000 7fbb34945640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbb3015ee80 0x7fbb301f9510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:45628/0 (socket says 172.21.15.38:45628) 2024-09-21T03:44:11.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.330+0000 7fbb34945640 1 -- 172.21.15.38:0/566920363 learned_addr learned my addr 172.21.15.38:0/566920363 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:44:11.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.330+0000 7fbb2ffff640 1 --2- 172.21.15.38:0/566920363 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbb300b9ad0 0x7fbb300b21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:11.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.330+0000 7fbb2f7fe640 1 --2- 172.21.15.38:0/566920363 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbb300ceb90 0x7fbb30157b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:11.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.330+0000 7fbb34945640 1 -- 172.21.15.38:0/566920363 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbb300ceb90 msgr2=0x7fbb30157b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:11.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.330+0000 7fbb34945640 1 --2- 172.21.15.38:0/566920363 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbb300ceb90 0x7fbb30157b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:11.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.330+0000 7fbb34945640 1 -- 172.21.15.38:0/566920363 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbb300b9ad0 msgr2=0x7fbb300b21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:11.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.330+0000 7fbb34945640 1 --2- 172.21.15.38:0/566920363 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbb300b9ad0 0x7fbb300b21b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:11.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.330+0000 7fbb34945640 1 -- 172.21.15.38:0/566920363 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb300b7cb0 con 0x7fbb3015ee80 2024-09-21T03:44:11.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.330+0000 7fbb34945640 1 --2- 172.21.15.38:0/566920363 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbb3015ee80 0x7fbb301f9510 secure :-1 s=READY pgs=2173 cs=0 l=1 rev1=1 crypto rx=0x7fbb2800c700 tx=0x7fbb2800cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:11.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.330+0000 7fbb2cff9640 1 -- 172.21.15.38:0/566920363 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb28003b70 con 0x7fbb3015ee80 2024-09-21T03:44:11.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.330+0000 7fbb363a8640 1 -- 172.21.15.38:0/566920363 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb300b1910 con 0x7fbb3015ee80 2024-09-21T03:44:11.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.330+0000 7fbb2cff9640 1 -- 172.21.15.38:0/566920363 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbb28003d10 con 0x7fbb3015ee80 2024-09-21T03:44:11.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.330+0000 7fbb2cff9640 1 -- 172.21.15.38:0/566920363 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb2800aab0 con 0x7fbb3015ee80 2024-09-21T03:44:11.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.330+0000 7fbb363a8640 1 -- 172.21.15.38:0/566920363 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb301f8ff0 con 0x7fbb3015ee80 2024-09-21T03:44:11.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.332+0000 7fbb363a8640 1 -- 172.21.15.38:0/566920363 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb300d0170 con 0x7fbb3015ee80 2024-09-21T03:44:11.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.332+0000 7fbb2cff9640 1 -- 172.21.15.38:0/566920363 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fbb2800ac50 con 0x7fbb3015ee80 2024-09-21T03:44:11.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.333+0000 7fbb2cff9640 1 --2- 172.21.15.38:0/566920363 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fbb1c078bc0 0x7fbb1c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:11.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.333+0000 7fbb2cff9640 1 -- 172.21.15.38:0/566920363 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 9400+0+0 (secure 0 0 0) 0x7fbb2809b960 con 0x7fbb3015ee80 2024-09-21T03:44:11.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.333+0000 7fbb2ffff640 1 --2- 172.21.15.38:0/566920363 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fbb1c078bc0 0x7fbb1c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:11.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.334+0000 7fbb2ffff640 1 --2- 172.21.15.38:0/566920363 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fbb1c078bc0 0x7fbb1c07b0d0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fbb20034000 tx=0x7fbb20004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:11.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.338+0000 7fbb2cff9640 1 -- 172.21.15.38:0/566920363 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fbb28062f80 con 0x7fbb3015ee80 2024-09-21T03:44:11.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.565+0000 7fbb363a8640 1 -- 172.21.15.38:0/566920363 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7fbb300b4b50 con 0x7fbb3015ee80 2024-09-21T03:44:11.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.890+0000 7fbb2cff9640 1 -- 172.21.15.38:0/566920363 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v308) ==== 162+0+0 (secure 0 0 0) 0x7fbb28062880 con 0x7fbb3015ee80 2024-09-21T03:44:11.893 INFO:teuthology.orchestra.run.smithi038.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-21T03:44:11.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.893+0000 7fbb363a8640 1 -- 172.21.15.38:0/566920363 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fbb1c078bc0 msgr2=0x7fbb1c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:11.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.893+0000 7fbb363a8640 1 --2- 172.21.15.38:0/566920363 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fbb1c078bc0 0x7fbb1c07b0d0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fbb20034000 tx=0x7fbb20004730 comp rx=0 tx=0).stop 2024-09-21T03:44:11.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.893+0000 7fbb363a8640 1 -- 172.21.15.38:0/566920363 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbb3015ee80 msgr2=0x7fbb301f9510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:11.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.893+0000 7fbb363a8640 1 --2- 172.21.15.38:0/566920363 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbb3015ee80 0x7fbb301f9510 secure :-1 s=READY pgs=2173 cs=0 l=1 rev1=1 crypto rx=0x7fbb2800c700 tx=0x7fbb2800cbc0 comp rx=0 tx=0).stop 2024-09-21T03:44:11.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.893+0000 7fbb363a8640 1 -- 172.21.15.38:0/566920363 shutdown_connections 2024-09-21T03:44:11.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.893+0000 7fbb363a8640 1 --2- 172.21.15.38:0/566920363 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbb3015ee80 0x7fbb301f9510 unknown :-1 s=CLOSED pgs=2173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:11.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.893+0000 7fbb363a8640 1 --2- 172.21.15.38:0/566920363 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fbb1c078bc0 0x7fbb1c07b0d0 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:11.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.893+0000 7fbb363a8640 1 --2- 172.21.15.38:0/566920363 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbb300ceb90 0x7fbb30157b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:11.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.893+0000 7fbb363a8640 1 --2- 172.21.15.38:0/566920363 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbb300b9ad0 0x7fbb300b21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:11.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.893+0000 7fbb363a8640 1 -- 172.21.15.38:0/566920363 >> 172.21.15.38:0/566920363 conn(0x7fbb3009d830 msgr2=0x7fbb30161aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:11.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.893+0000 7fbb363a8640 1 -- 172.21.15.38:0/566920363 shutdown_connections 2024-09-21T03:44:11.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:11.893+0000 7fbb363a8640 1 -- 172.21.15.38:0/566920363 wait complete. 2024-09-21T03:44:11.902 INFO:tasks.ceph.mds.c.smithi038.stderr:2024-09-21T03:44:11.900+0000 7f468abb9640 -1 mds.0.openfiles _load_finish: corrupted header/values: End of buffer [buffer:2] 2024-09-21T03:44:12.312 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:44:12.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:12.439+0000 7f7ebf369640 1 -- 172.21.15.38:0/1699149668 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7eb80b6250 msgr2=0x7f7eb80b6660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:12.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:12.439+0000 7f7ebf369640 1 --2- 172.21.15.38:0/1699149668 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7eb80b6250 0x7f7eb80b6660 secure :-1 s=READY pgs=2175 cs=0 l=1 rev1=1 crypto rx=0x7f7eb80d2320 tx=0x7f7eb002a530 comp rx=0 tx=0).stop 2024-09-21T03:44:12.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:12.439+0000 7f7ebf369640 1 -- 172.21.15.38:0/1699149668 shutdown_connections 2024-09-21T03:44:12.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:12.439+0000 7f7ebf369640 1 --2- 172.21.15.38:0/1699149668 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7eb80b6250 0x7f7eb80b6660 unknown :-1 s=CLOSED pgs=2175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:12.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:12.439+0000 7f7ebf369640 1 --2- 172.21.15.38:0/1699149668 >> v2:172.21.15.195:3300/0 conn(0x7f7eb80d19f0 0x7f7eb815cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:12.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:12.439+0000 7f7ebf369640 1 -- 172.21.15.38:0/1699149668 >> 172.21.15.38:0/1699149668 conn(0x7f7eb80d2350 msgr2=0x7f7eb80d2760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:12.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:12.439+0000 7f7ebf369640 1 -- 172.21.15.38:0/1699149668 shutdown_connections 2024-09-21T03:44:12.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:12.439+0000 7f7ebf369640 1 -- 172.21.15.38:0/1699149668 wait complete. 2024-09-21T03:44:12.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:12.441+0000 7f7ebf369640 1 Processor -- start 2024-09-21T03:44:12.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:12.441+0000 7f7ebf369640 1 -- start start 2024-09-21T03:44:12.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:12.442+0000 7f7ebf369640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7eb80b6250 0x7f7eb813cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:12.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:12.442+0000 7f7ebf369640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7eb80d19f0 0x7f7eb81563a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:12.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:12.442+0000 7f7ebf369640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7eb815cb10 0x7f7eb81568e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:12.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:12.442+0000 7f7ebf369640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f7eb80b6d20 con 0x7f7eb80b6250 2024-09-21T03:44:12.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:12.442+0000 7f7ebf369640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f7eb80b6ba0 con 0x7f7eb815cb10 2024-09-21T03:44:12.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:12.442+0000 7f7ebf369640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f7eb80b6ea0 con 0x7f7eb80d19f0 2024-09-21T03:44:12.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:12.443+0000 7f7ebd105640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7eb80b6250 0x7f7eb813cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:12.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:12.443+0000 7f7ebd105640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7eb80b6250 0x7f7eb813cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38498/0 (socket says 172.21.15.38:38498) 2024-09-21T03:44:12.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:12.443+0000 7f7ebd105640 1 -- 172.21.15.38:0/881437905 learned_addr learned my addr 172.21.15.38:0/881437905 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:44:12.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:12.443+0000 7f7ebd906640 1 --2- 172.21.15.38:0/881437905 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7eb815cb10 0x7f7eb81568e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:12.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:12.443+0000 7f7ebc904640 1 --2- 172.21.15.38:0/881437905 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7eb80d19f0 0x7f7eb81563a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:12.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:12.443+0000 7f7ebd105640 1 -- 172.21.15.38:0/881437905 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7eb80d19f0 msgr2=0x7f7eb81563a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:12.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:12.443+0000 7f7ebd105640 1 --2- 172.21.15.38:0/881437905 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7eb80d19f0 0x7f7eb81563a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:12.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:12.443+0000 7f7ebd105640 1 -- 172.21.15.38:0/881437905 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7eb815cb10 msgr2=0x7f7eb81568e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:12.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:12.443+0000 7f7ebd105640 1 --2- 172.21.15.38:0/881437905 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7eb815cb10 0x7f7eb81568e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:12.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:12.443+0000 7f7ebd105640 1 -- 172.21.15.38:0/881437905 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7eb814d390 con 0x7f7eb80b6250 2024-09-21T03:44:12.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:12.443+0000 7f7ebd105640 1 --2- 172.21.15.38:0/881437905 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7eb80b6250 0x7f7eb813cc90 secure :-1 s=READY pgs=2176 cs=0 l=1 rev1=1 crypto rx=0x7f7ea800c7a0 tx=0x7f7ea800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:12.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:12.443+0000 7f7eb5ffb640 1 -- 172.21.15.38:0/881437905 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ea8017070 con 0x7f7eb80b6250 2024-09-21T03:44:12.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:12.443+0000 7f7eb5ffb640 1 -- 172.21.15.38:0/881437905 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7ea8005070 con 0x7f7eb80b6250 2024-09-21T03:44:12.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:12.443+0000 7f7ebf369640 1 -- 172.21.15.38:0/881437905 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7eb8150540 con 0x7f7eb80b6250 2024-09-21T03:44:12.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:12.443+0000 7f7eb5ffb640 1 -- 172.21.15.38:0/881437905 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ea80054a0 con 0x7f7eb80b6250 2024-09-21T03:44:12.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:12.444+0000 7f7ebf369640 1 -- 172.21.15.38:0/881437905 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7eb814d910 con 0x7f7eb80b6250 2024-09-21T03:44:12.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:12.445+0000 7f7ebf369640 1 -- 172.21.15.38:0/881437905 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7eb80b7350 con 0x7f7eb80b6250 2024-09-21T03:44:12.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:12.445+0000 7f7eb5ffb640 1 -- 172.21.15.38:0/881437905 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f7ea801a070 con 0x7f7eb80b6250 2024-09-21T03:44:12.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:12.446+0000 7f7eb5ffb640 1 --2- 172.21.15.38:0/881437905 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7ea4078b90 0x7f7ea407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:12.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:12.446+0000 7f7eb5ffb640 1 -- 172.21.15.38:0/881437905 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 9400+0+0 (secure 0 0 0) 0x7f7ea809b270 con 0x7f7eb80b6250 2024-09-21T03:44:12.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:12.447+0000 7f7ebc904640 1 --2- 172.21.15.38:0/881437905 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7ea4078b90 0x7f7ea407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:12.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:12.447+0000 7f7ebc904640 1 --2- 172.21.15.38:0/881437905 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7ea4078b90 0x7f7ea407b0a0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f7eac005d70 tx=0x7f7eac005d00 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:12.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:12.451+0000 7f7eb5ffb640 1 -- 172.21.15.38:0/881437905 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f7ea8062880 con 0x7f7eb80b6250 2024-09-21T03:44:12.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:12.676+0000 7f7ebf369640 1 -- 172.21.15.38:0/881437905 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7eb8153480 con 0x7f7eb80b6250 2024-09-21T03:44:12.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:12.677+0000 7f7eb5ffb640 1 -- 172.21.15.38:0/881437905 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 309 v309) ==== 95+0+5230 (secure 0 0 0) 0x7f7ea8062180 con 0x7f7eb80b6250 2024-09-21T03:44:12.679 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:44:12.679 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":309,"btime":"2024-09-21T03:44:11:894217+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6086,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/965972663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":965972663},{"type":"v1","addr":"172.21.15.195:6836","nonce":965972663}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6700,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/3176122334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":3176122334},{"type":"v1","addr":"172.21.15.195:6837","nonce":3176122334}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":9840,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/3801772046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":3801772046},{"type":"v1","addr":"172.21.15.38:6835","nonce":3801772046}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":273}],"filesystems":[{"mdsmap":{"epoch":309,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:40:02.319282+0000","modified":"2024-09-21T03:44:11.894197+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":132,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10275},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10275":{"gid":10275,"name":"c","rank":0,"incarnation":309,"state":"up:replay","state_seq":1,"addr":"172.21.15.38:6833/2178429646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":2178429646},{"type":"v1","addr":"172.21.15.38:6833","nonce":2178429646}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-21T03:44:12.679 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 309 2024-09-21T03:44:12.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:12.679+0000 7f7ebf369640 1 -- 172.21.15.38:0/881437905 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7ea4078b90 msgr2=0x7f7ea407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:12.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:12.679+0000 7f7ebf369640 1 --2- 172.21.15.38:0/881437905 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7ea4078b90 0x7f7ea407b0a0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f7eac005d70 tx=0x7f7eac005d00 comp rx=0 tx=0).stop 2024-09-21T03:44:12.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:12.679+0000 7f7ebf369640 1 -- 172.21.15.38:0/881437905 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7eb80b6250 msgr2=0x7f7eb813cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:12.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:12.679+0000 7f7ebf369640 1 --2- 172.21.15.38:0/881437905 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7eb80b6250 0x7f7eb813cc90 secure :-1 s=READY pgs=2176 cs=0 l=1 rev1=1 crypto rx=0x7f7ea800c7a0 tx=0x7f7ea800cc60 comp rx=0 tx=0).stop 2024-09-21T03:44:12.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:12.679+0000 7f7ebf369640 1 -- 172.21.15.38:0/881437905 shutdown_connections 2024-09-21T03:44:12.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:12.679+0000 7f7ebf369640 1 --2- 172.21.15.38:0/881437905 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7eb815cb10 0x7f7eb81568e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:12.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:12.679+0000 7f7ebf369640 1 --2- 172.21.15.38:0/881437905 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7ea4078b90 0x7f7ea407b0a0 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:12.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:12.679+0000 7f7ebf369640 1 --2- 172.21.15.38:0/881437905 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7eb80d19f0 0x7f7eb81563a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:12.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:12.679+0000 7f7ebf369640 1 --2- 172.21.15.38:0/881437905 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7eb80b6250 0x7f7eb813cc90 unknown :-1 s=CLOSED pgs=2176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:12.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:12.680+0000 7f7ebf369640 1 -- 172.21.15.38:0/881437905 >> 172.21.15.38:0/881437905 conn(0x7f7eb80d2350 msgr2=0x7f7eb81603c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:12.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:12.680+0000 7f7ebf369640 1 -- 172.21.15.38:0/881437905 shutdown_connections 2024-09-21T03:44:12.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:12.680+0000 7f7ebf369640 1 -- 172.21.15.38:0/881437905 wait complete. 2024-09-21T03:44:12.693 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 309, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T03:40:02.319282+0000', 'modified': '2024-09-21T03:44:11.894197+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 132, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 10275}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_10275': {'gid': 10275, 'name': 'c', 'rank': 0, 'incarnation': 309, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.38:6833/2178429646', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.38:6832', 'nonce': 2178429646}, {'type': 'v1', 'addr': '172.21.15.38:6833', 'nonce': 2178429646}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-09-21T03:44:12.693 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_10275:up:replay 2024-09-21T03:44:12.693 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2024-09-21T03:44:15.891 INFO:tasks.ceph.mds.c.smithi038.stderr:2024-09-21T03:44:15.889+0000 7f468ebc1640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T03:44:18.097 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:44:18.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.227+0000 7fb0b63c3640 1 -- 172.21.15.38:0/3884979596 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb0b00d0b40 msgr2=0x7fb0b006e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:18.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.227+0000 7fb0b63c3640 1 --2- 172.21.15.38:0/3884979596 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb0b00d0b40 0x7fb0b006e6f0 secure :-1 s=READY pgs=2178 cs=0 l=1 rev1=1 crypto rx=0x7fb0b006e3e0 tx=0x7fb0a002a510 comp rx=0 tx=0).stop 2024-09-21T03:44:18.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.228+0000 7fb0b63c3640 1 -- 172.21.15.38:0/3884979596 shutdown_connections 2024-09-21T03:44:18.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.228+0000 7fb0b63c3640 1 --2- 172.21.15.38:0/3884979596 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb0b00d0b40 0x7fb0b006e6f0 unknown :-1 s=CLOSED pgs=2178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:18.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.228+0000 7fb0b63c3640 1 -- 172.21.15.38:0/3884979596 >> 172.21.15.38:0/3884979596 conn(0x7fb0b00b2cc0 msgr2=0x7fb0b00b30d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:18.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.228+0000 7fb0b63c3640 1 -- 172.21.15.38:0/3884979596 shutdown_connections 2024-09-21T03:44:18.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.228+0000 7fb0b63c3640 1 -- 172.21.15.38:0/3884979596 wait complete. 2024-09-21T03:44:18.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.229+0000 7fb0b63c3640 1 Processor -- start 2024-09-21T03:44:18.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.230+0000 7fb0b63c3640 1 -- start start 2024-09-21T03:44:18.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.231+0000 7fb0b63c3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb0b006ec30 0x7fb0b006cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:18.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.231+0000 7fb0b63c3640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb0b0073d50 0x7fb0b015db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:18.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.231+0000 7fb0b63c3640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb0b00d0b40 0x7fb0b015e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:18.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.231+0000 7fb0b63c3640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b015b960 con 0x7fb0b006ec30 2024-09-21T03:44:18.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.231+0000 7fb0b63c3640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b015b7e0 con 0x7fb0b00d0b40 2024-09-21T03:44:18.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.231+0000 7fb0b63c3640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fb0b015bae0 con 0x7fb0b0073d50 2024-09-21T03:44:18.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.231+0000 7fb0affff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb0b006ec30 0x7fb0b006cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:18.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.231+0000 7fb0affff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb0b006ec30 0x7fb0b006cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38518/0 (socket says 172.21.15.38:38518) 2024-09-21T03:44:18.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.231+0000 7fb0affff640 1 -- 172.21.15.38:0/144896660 learned_addr learned my addr 172.21.15.38:0/144896660 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:44:18.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.231+0000 7fb0b4960640 1 --2- 172.21.15.38:0/144896660 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb0b00d0b40 0x7fb0b015e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:18.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.231+0000 7fb0af7fe640 1 --2- 172.21.15.38:0/144896660 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb0b0073d50 0x7fb0b015db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:18.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.231+0000 7fb0affff640 1 -- 172.21.15.38:0/144896660 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb0b0073d50 msgr2=0x7fb0b015db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:18.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.231+0000 7fb0affff640 1 --2- 172.21.15.38:0/144896660 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb0b0073d50 0x7fb0b015db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:18.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.231+0000 7fb0affff640 1 -- 172.21.15.38:0/144896660 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb0b00d0b40 msgr2=0x7fb0b015e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:18.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.232+0000 7fb0affff640 1 --2- 172.21.15.38:0/144896660 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb0b00d0b40 0x7fb0b015e0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:18.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.232+0000 7fb0affff640 1 -- 172.21.15.38:0/144896660 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0b015a2a0 con 0x7fb0b006ec30 2024-09-21T03:44:18.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.232+0000 7fb0af7fe640 1 --2- 172.21.15.38:0/144896660 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb0b0073d50 0x7fb0b015db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:44:18.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.232+0000 7fb0affff640 1 --2- 172.21.15.38:0/144896660 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb0b006ec30 0x7fb0b006cde0 secure :-1 s=READY pgs=2179 cs=0 l=1 rev1=1 crypto rx=0x7fb0a0009920 tx=0x7fb0a0033d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:18.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.232+0000 7fb0acff9640 1 -- 172.21.15.38:0/144896660 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a0031e20 con 0x7fb0b006ec30 2024-09-21T03:44:18.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.232+0000 7fb0acff9640 1 -- 172.21.15.38:0/144896660 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb0a0005110 con 0x7fb0b006ec30 2024-09-21T03:44:18.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.232+0000 7fb0acff9640 1 -- 172.21.15.38:0/144896660 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a0005560 con 0x7fb0b006ec30 2024-09-21T03:44:18.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.232+0000 7fb0b63c3640 1 -- 172.21.15.38:0/144896660 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0b015a580 con 0x7fb0b006ec30 2024-09-21T03:44:18.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.232+0000 7fb0b63c3640 1 -- 172.21.15.38:0/144896660 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0b015ab40 con 0x7fb0b006ec30 2024-09-21T03:44:18.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.233+0000 7fb0b63c3640 1 -- 172.21.15.38:0/144896660 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0b015bfb0 con 0x7fb0b006ec30 2024-09-21T03:44:18.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.234+0000 7fb0acff9640 1 -- 172.21.15.38:0/144896660 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb0a00047a0 con 0x7fb0b006ec30 2024-09-21T03:44:18.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.234+0000 7fb0acff9640 1 --2- 172.21.15.38:0/144896660 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb09c078b90 0x7fb09c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:18.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.235+0000 7fb0acff9640 1 -- 172.21.15.38:0/144896660 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 9400+0+0 (secure 0 0 0) 0x7fb0a00bcf20 con 0x7fb0b006ec30 2024-09-21T03:44:18.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.235+0000 7fb0af7fe640 1 --2- 172.21.15.38:0/144896660 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb09c078b90 0x7fb09c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:18.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.235+0000 7fb0af7fe640 1 --2- 172.21.15.38:0/144896660 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb09c078b90 0x7fb09c07b0a0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fb0b015db10 tx=0x7fb0a4008040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:18.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.239+0000 7fb0acff9640 1 -- 172.21.15.38:0/144896660 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fb0a003f030 con 0x7fb0b006ec30 2024-09-21T03:44:18.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.464+0000 7fb0b63c3640 1 -- 172.21.15.38:0/144896660 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb0b006e800 con 0x7fb0b006ec30 2024-09-21T03:44:18.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.465+0000 7fb0acff9640 1 -- 172.21.15.38:0/144896660 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 312 v312) ==== 95+0+5239 (secure 0 0 0) 0x7fb0a00844a0 con 0x7fb0b006ec30 2024-09-21T03:44:18.467 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:44:18.467 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":312,"btime":"2024-09-21T03:44:14:907965+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6086,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/965972663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":965972663},{"type":"v1","addr":"172.21.15.195:6836","nonce":965972663}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6700,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/3176122334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":3176122334},{"type":"v1","addr":"172.21.15.195:6837","nonce":3176122334}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":9840,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/3801772046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":3801772046},{"type":"v1","addr":"172.21.15.38:6835","nonce":3801772046}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":273}],"filesystems":[{"mdsmap":{"epoch":312,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:40:02.319282+0000","modified":"2024-09-21T03:44:14.907958+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":132,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10275},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10275":{"gid":10275,"name":"c","rank":0,"incarnation":309,"state":"up:active","state_seq":4,"addr":"172.21.15.38:6833/2178429646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":2178429646},{"type":"v1","addr":"172.21.15.38:6833","nonce":2178429646}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10275,"qdb_cluster":[10275]},"id":6}]} 2024-09-21T03:44:18.467 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 312 2024-09-21T03:44:18.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.467+0000 7fb0b63c3640 1 -- 172.21.15.38:0/144896660 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb09c078b90 msgr2=0x7fb09c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:18.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.467+0000 7fb0b63c3640 1 --2- 172.21.15.38:0/144896660 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb09c078b90 0x7fb09c07b0a0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fb0b015db10 tx=0x7fb0a4008040 comp rx=0 tx=0).stop 2024-09-21T03:44:18.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.467+0000 7fb0b63c3640 1 -- 172.21.15.38:0/144896660 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb0b006ec30 msgr2=0x7fb0b006cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:18.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.467+0000 7fb0b63c3640 1 --2- 172.21.15.38:0/144896660 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb0b006ec30 0x7fb0b006cde0 secure :-1 s=READY pgs=2179 cs=0 l=1 rev1=1 crypto rx=0x7fb0a0009920 tx=0x7fb0a0033d60 comp rx=0 tx=0).stop 2024-09-21T03:44:18.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.467+0000 7fb0b63c3640 1 -- 172.21.15.38:0/144896660 shutdown_connections 2024-09-21T03:44:18.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.467+0000 7fb0b63c3640 1 --2- 172.21.15.38:0/144896660 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb0b00d0b40 0x7fb0b015e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:18.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.467+0000 7fb0b63c3640 1 --2- 172.21.15.38:0/144896660 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb09c078b90 0x7fb09c07b0a0 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:18.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.467+0000 7fb0b63c3640 1 --2- 172.21.15.38:0/144896660 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb0b0073d50 0x7fb0b015db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:18.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.467+0000 7fb0b63c3640 1 --2- 172.21.15.38:0/144896660 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb0b006ec30 0x7fb0b006cde0 unknown :-1 s=CLOSED pgs=2179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:18.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.467+0000 7fb0b63c3640 1 -- 172.21.15.38:0/144896660 >> 172.21.15.38:0/144896660 conn(0x7fb0b00b2cc0 msgr2=0x7fb0b00b30d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:18.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.467+0000 7fb0b63c3640 1 -- 172.21.15.38:0/144896660 shutdown_connections 2024-09-21T03:44:18.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.467+0000 7fb0b63c3640 1 -- 172.21.15.38:0/144896660 wait complete. 2024-09-21T03:44:18.481 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 312, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T03:40:02.319282+0000', 'modified': '2024-09-21T03:44:14.907958+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 132, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 10275}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_10275': {'gid': 10275, 'name': 'c', 'rank': 0, 'incarnation': 309, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.38:6833/2178429646', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.38:6832', 'nonce': 2178429646}, {'type': 'v1', 'addr': '172.21.15.38:6833', 'nonce': 2178429646}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10275, 'qdb_cluster': [10275]} 2024-09-21T03:44:18.481 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-21T03:44:18.481 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-21T03:44:18.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.597+0000 7f0a08e9b640 1 -- 172.21.15.38:0/2435889612 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0a0415ee80 msgr2=0x7f0a040d0170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:18.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.597+0000 7f0a08e9b640 1 --2- 172.21.15.38:0/2435889612 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0a0415ee80 0x7f0a040d0170 secure :-1 s=READY pgs=2180 cs=0 l=1 rev1=1 crypto rx=0x7f09f4005fd0 tx=0x7f09f402aa10 comp rx=0 tx=0).stop 2024-09-21T03:44:18.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.597+0000 7f0a08e9b640 1 -- 172.21.15.38:0/2435889612 shutdown_connections 2024-09-21T03:44:18.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.597+0000 7f0a08e9b640 1 --2- 172.21.15.38:0/2435889612 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0a040ceb90 0x7f0a040cefa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:18.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.597+0000 7f0a08e9b640 1 --2- 172.21.15.38:0/2435889612 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0a0415ee80 0x7f0a040d0170 unknown :-1 s=CLOSED pgs=2180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:18.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.597+0000 7f0a08e9b640 1 -- 172.21.15.38:0/2435889612 >> 172.21.15.38:0/2435889612 conn(0x7f0a0409d830 msgr2=0x7f0a040b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:18.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.597+0000 7f0a08e9b640 1 -- 172.21.15.38:0/2435889612 shutdown_connections 2024-09-21T03:44:18.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.597+0000 7f0a08e9b640 1 -- 172.21.15.38:0/2435889612 wait complete. 2024-09-21T03:44:18.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.599+0000 7f0a08e9b640 1 Processor -- start 2024-09-21T03:44:18.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.599+0000 7f0a08e9b640 1 -- start start 2024-09-21T03:44:18.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.600+0000 7f0a08e9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0a040b9ad0 0x7f0a0406f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:18.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.600+0000 7f0a08e9b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0a040ceb90 0x7f0a041f9240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:18.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.600+0000 7f0a08e9b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0a0415ee80 0x7f0a041f9780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:18.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.600+0000 7f0a08e9b640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f0a040b6700 con 0x7f0a040b9ad0 2024-09-21T03:44:18.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.600+0000 7f0a08e9b640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f0a040b6580 con 0x7f0a040ceb90 2024-09-21T03:44:18.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.600+0000 7f0a08e9b640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f0a040b6880 con 0x7f0a0415ee80 2024-09-21T03:44:18.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.600+0000 7f0a0259c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0a040b9ad0 0x7f0a0406f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:18.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.600+0000 7f0a0259c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0a040b9ad0 0x7f0a0406f1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38530/0 (socket says 172.21.15.38:38530) 2024-09-21T03:44:18.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.600+0000 7f0a0259c640 1 -- 172.21.15.38:0/1279006549 learned_addr learned my addr 172.21.15.38:0/1279006549 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:44:18.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.600+0000 7f0a01d9b640 1 --2- 172.21.15.38:0/1279006549 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0a040ceb90 0x7f0a041f9240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:18.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.600+0000 7f0a02d9d640 1 --2- 172.21.15.38:0/1279006549 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0a0415ee80 0x7f0a041f9780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:18.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.600+0000 7f0a0259c640 1 -- 172.21.15.38:0/1279006549 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0a0415ee80 msgr2=0x7f0a041f9780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:18.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.600+0000 7f0a0259c640 1 --2- 172.21.15.38:0/1279006549 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0a0415ee80 0x7f0a041f9780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:18.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.600+0000 7f0a0259c640 1 -- 172.21.15.38:0/1279006549 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0a040ceb90 msgr2=0x7f0a041f9240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:18.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.600+0000 7f0a0259c640 1 --2- 172.21.15.38:0/1279006549 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0a040ceb90 0x7f0a041f9240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:18.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.600+0000 7f0a0259c640 1 -- 172.21.15.38:0/1279006549 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a040cb610 con 0x7f0a040b9ad0 2024-09-21T03:44:18.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.600+0000 7f0a01d9b640 1 --2- 172.21.15.38:0/1279006549 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0a040ceb90 0x7f0a041f9240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:44:18.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.601+0000 7f0a0259c640 1 --2- 172.21.15.38:0/1279006549 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0a040b9ad0 0x7f0a0406f1b0 secure :-1 s=READY pgs=2181 cs=0 l=1 rev1=1 crypto rx=0x7f09f4000f80 tx=0x7f09f4003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:18.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.601+0000 7f09faffd640 1 -- 172.21.15.38:0/1279006549 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09f4033050 con 0x7f0a040b9ad0 2024-09-21T03:44:18.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.601+0000 7f09faffd640 1 -- 172.21.15.38:0/1279006549 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f09f40042d0 con 0x7f0a040b9ad0 2024-09-21T03:44:18.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.601+0000 7f09faffd640 1 -- 172.21.15.38:0/1279006549 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09f4004700 con 0x7f0a040b9ad0 2024-09-21T03:44:18.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.601+0000 7f0a08e9b640 1 -- 172.21.15.38:0/1279006549 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a040cb8f0 con 0x7f0a040b9ad0 2024-09-21T03:44:18.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.601+0000 7f0a08e9b640 1 -- 172.21.15.38:0/1279006549 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a040cbeb0 con 0x7f0a040b9ad0 2024-09-21T03:44:18.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.603+0000 7f09faffd640 1 -- 172.21.15.38:0/1279006549 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f09f4036070 con 0x7f0a040b9ad0 2024-09-21T03:44:18.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.603+0000 7f09faffd640 1 --2- 172.21.15.38:0/1279006549 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f09e4078bc0 0x7f09e407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:18.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.603+0000 7f09faffd640 1 -- 172.21.15.38:0/1279006549 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 9400+0+0 (secure 0 0 0) 0x7f09f40bce80 con 0x7f0a040b9ad0 2024-09-21T03:44:18.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.603+0000 7f0a01d9b640 1 --2- 172.21.15.38:0/1279006549 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f09e4078bc0 0x7f09e407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:18.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.604+0000 7f0a01d9b640 1 --2- 172.21.15.38:0/1279006549 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f09e4078bc0 0x7f09e407b0d0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f0a040690e0 tx=0x7f09ec009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:18.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.604+0000 7f0a0fe0b740 1 Processor -- start 2024-09-21T03:44:18.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.604+0000 7f0a0fe0b740 1 -- start start 2024-09-21T03:44:18.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.605+0000 7f0a0fe0b740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e102227270 0x55e1025f8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:18.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.606+0000 7f0a0fe0b740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e1021fe3b0 0x55e1025f9500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:18.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.606+0000 7f0a0fe0b740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e102206590 0x55e102616400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:18.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.606+0000 7f0a0fe0b740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55e102541820 con 0x55e1021fe3b0 2024-09-21T03:44:18.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.606+0000 7f0a0fe0b740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55e102530c50 con 0x55e102206590 2024-09-21T03:44:18.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.606+0000 7f0a0fe0b740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55e102534a60 con 0x55e102227270 2024-09-21T03:44:18.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.606+0000 7f0a01d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e1021fe3b0 0x55e1025f9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:18.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.606+0000 7f0a01d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e1021fe3b0 0x55e1025f9500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38538/0 (socket says 172.21.15.38:38538) 2024-09-21T03:44:18.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.606+0000 7f0a01d9b640 1 -- 172.21.15.38:0/2607505638 learned_addr learned my addr 172.21.15.38:0/2607505638 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:44:18.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.606+0000 7f0a0259c640 1 --2- 172.21.15.38:0/2607505638 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e102227270 0x55e1025f8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:18.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.606+0000 7f0a02d9d640 1 --2- 172.21.15.38:0/2607505638 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e102206590 0x55e102616400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:18.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.606+0000 7f0a01d9b640 1 -- 172.21.15.38:0/2607505638 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e102227270 msgr2=0x55e1025f8fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:44:18.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.606+0000 7f0a01d9b640 1 --2- 172.21.15.38:0/2607505638 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e102227270 0x55e1025f8fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:18.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.606+0000 7f0a01d9b640 1 -- 172.21.15.38:0/2607505638 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e102206590 msgr2=0x55e102616400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:44:18.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.606+0000 7f0a01d9b640 1 --2- 172.21.15.38:0/2607505638 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e102206590 0x55e102616400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:18.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.606+0000 7f0a01d9b640 1 -- 172.21.15.38:0/2607505638 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e102292850 con 0x55e1021fe3b0 2024-09-21T03:44:18.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.606+0000 7f0a02d9d640 1 --2- 172.21.15.38:0/2607505638 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e102206590 0x55e102616400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:44:18.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.606+0000 7f0a01d9b640 1 --2- 172.21.15.38:0/2607505638 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e1021fe3b0 0x55e1025f9500 secure :-1 s=READY pgs=2182 cs=0 l=1 rev1=1 crypto rx=0x7f09ec001da0 tx=0x7f09ec011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e76cd495964b71e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:18.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.606+0000 7f09e3fff640 1 -- 172.21.15.38:0/2607505638 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09ec0035f0 con 0x55e1021fe3b0 2024-09-21T03:44:18.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.606+0000 7f09e3fff640 1 -- 172.21.15.38:0/2607505638 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f09ec0037b0 con 0x55e1021fe3b0 2024-09-21T03:44:18.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.606+0000 7f09e3fff640 1 -- 172.21.15.38:0/2607505638 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09ec003c00 con 0x55e1021fe3b0 2024-09-21T03:44:18.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.607+0000 7f0a0fe0b740 1 -- 172.21.15.38:0/2607505638 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e1021fe3b0 msgr2=0x55e1025f9500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:18.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.607+0000 7f0a0fe0b740 1 --2- 172.21.15.38:0/2607505638 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e1021fe3b0 0x55e1025f9500 secure :-1 s=READY pgs=2182 cs=0 l=1 rev1=1 crypto rx=0x7f09ec001da0 tx=0x7f09ec011000 comp rx=0 tx=0).stop 2024-09-21T03:44:18.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.607+0000 7f0a0fe0b740 1 -- 172.21.15.38:0/2607505638 shutdown_connections 2024-09-21T03:44:18.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.607+0000 7f0a0fe0b740 1 --2- 172.21.15.38:0/2607505638 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e102206590 0x55e102616400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:18.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.607+0000 7f0a0fe0b740 1 --2- 172.21.15.38:0/2607505638 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e1021fe3b0 0x55e1025f9500 unknown :-1 s=CLOSED pgs=2182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:18.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.607+0000 7f0a0fe0b740 1 --2- 172.21.15.38:0/2607505638 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e102227270 0x55e1025f8fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:18.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.607+0000 7f0a0fe0b740 1 -- 172.21.15.38:0/2607505638 >> 172.21.15.38:0/2607505638 conn(0x55e10221b3c0 msgr2=0x55e1025cf280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:18.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.607+0000 7f0a0fe0b740 1 -- 172.21.15.38:0/2607505638 shutdown_connections 2024-09-21T03:44:18.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.607+0000 7f0a0fe0b740 1 -- 172.21.15.38:0/2607505638 wait complete. 2024-09-21T03:44:18.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.608+0000 7f0a0fe0b740 1 -- start start 2024-09-21T03:44:18.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.608+0000 7f0a0fe0b740 1 Processor -- start 2024-09-21T03:44:18.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.610+0000 7f0a0fe0b740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e1021fe3b0 0x55e10260dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:18.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.610+0000 7f0a0fe0b740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e102206590 0x55e10260c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:18.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.610+0000 7f0a0fe0b740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e102227270 0x55e10260b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:18.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.610+0000 7f0a0fe0b740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55e1022a1620 con 0x55e102227270 2024-09-21T03:44:18.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.610+0000 7f0a0fe0b740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55e1024d65f0 con 0x55e1021fe3b0 2024-09-21T03:44:18.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.610+0000 7f0a0fe0b740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55e1025f9a40 con 0x55e102206590 2024-09-21T03:44:18.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.610+0000 7f0a0fe0b740 10 client.0 _ll_register_callbacks cb 0x55e1024da0a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:44:18.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.610+0000 7f0a02d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e102227270 0x55e10260b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:18.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.610+0000 7f0a02d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e102227270 0x55e10260b1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38546/0 (socket says 172.21.15.38:38546) 2024-09-21T03:44:18.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.610+0000 7f0a02d9d640 1 -- 172.21.15.38:0/938704668 learned_addr learned my addr 172.21.15.38:0/938704668 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:44:18.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.610+0000 7f0a0259c640 1 --2- 172.21.15.38:0/938704668 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e1021fe3b0 0x55e10260dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:18.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.610+0000 7f0a01d9b640 1 --2- 172.21.15.38:0/938704668 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e102206590 0x55e10260c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:18.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.610+0000 7f0a02d9d640 1 -- 172.21.15.38:0/938704668 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e102206590 msgr2=0x55e10260c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:44:18.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.610+0000 7f0a02d9d640 1 --2- 172.21.15.38:0/938704668 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e102206590 0x55e10260c5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:18.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.610+0000 7f0a02d9d640 1 -- 172.21.15.38:0/938704668 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e1021fe3b0 msgr2=0x55e10260dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:44:18.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.610+0000 7f0a02d9d640 1 --2- 172.21.15.38:0/938704668 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e1021fe3b0 0x55e10260dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:18.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.610+0000 7f0a02d9d640 1 -- 172.21.15.38:0/938704668 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e1023096b0 con 0x55e102227270 2024-09-21T03:44:18.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.610+0000 7f0a02d9d640 1 --2- 172.21.15.38:0/938704668 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e102227270 0x55e10260b1f0 secure :-1 s=READY pgs=2183 cs=0 l=1 rev1=1 crypto rx=0x7f09fc00fe10 tx=0x7f09fc0117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15b53e6c8555e8c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:18.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.610+0000 7f09e17fa640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-21T03:44:18.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.611+0000 7f09e17fa640 1 -- 172.21.15.38:0/938704668 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09fc012860 con 0x55e102227270 2024-09-21T03:44:18.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.611+0000 7f09e17fa640 1 -- 172.21.15.38:0/938704668 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(132..132 src has 1..132) ==== 9400+0+0 (secure 0 0 0) 0x7f09fc012a00 con 0x55e102227270 2024-09-21T03:44:18.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.611+0000 7f0a0fe0b740 1 -- 172.21.15.38:0/938704668 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f09ec0037b0 con 0x55e102227270 2024-09-21T03:44:18.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.611+0000 7f09e17fa640 1 -- 172.21.15.38:0/938704668 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=312) ==== 24+0+0 (secure 0 0 0) 0x7f09fc012c00 con 0x55e102227270 2024-09-21T03:44:18.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.611+0000 7f0a0fe0b740 10 client.10332 fetch_fsmap learned FSMap version 312 2024-09-21T03:44:18.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.611+0000 7f0a0fe0b740 1 -- 172.21.15.38:0/938704668 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=312,monmap=2+}) -- 0x55e10260bd30 con 0x55e102227270 2024-09-21T03:44:18.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.611+0000 7f09e17fa640 1 -- 172.21.15.38:0/938704668 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f09ec0037b0 con 0x55e102227270 2024-09-21T03:44:18.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.611+0000 7f09e17fa640 1 -- 172.21.15.38:0/938704668 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 312) ==== 3502+0+0 (secure 0 0 0) 0x7f09fc004020 con 0x55e102227270 2024-09-21T03:44:18.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.611+0000 7f0a0fe0b740 10 client.10332 fetch_fsmap finished waiting for FSMap version 312 2024-09-21T03:44:18.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.611+0000 7f0a0fe0b740 10 client.10332 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T03:44:18.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.611+0000 7f0a0fe0b740 20 client.10332 populate_metadata read hostname 'smithi038' 2024-09-21T03:44:18.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.611+0000 7f0a0fe0b740 1 --2- 172.21.15.38:0/938704668 >> [v2:172.21.15.38:6832/2178429646,v1:172.21.15.38:6833/2178429646] conn(0x55e1021e5e40 0x55e102622400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:18.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.611+0000 7f0a0fe0b740 4 client.10332 mds_command: new command op to 10275 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:44:18.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.611+0000 7f0a0fe0b740 1 -- 172.21.15.38:0/938704668 --> [v2:172.21.15.38:6832/2178429646,v1:172.21.15.38:6833/2178429646] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e102543770 con 0x55e1021e5e40 2024-09-21T03:44:18.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.611+0000 7f0a0259c640 1 --2- 172.21.15.38:0/938704668 >> [v2:172.21.15.38:6832/2178429646,v1:172.21.15.38:6833/2178429646] conn(0x55e1021e5e40 0x55e102622400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:18.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.612+0000 7f09e17fa640 10 client.10332 ms_handle_connect on v2:172.21.15.38:6832/2178429646 2024-09-21T03:44:18.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.612+0000 7f0a0259c640 1 --2- 172.21.15.38:0/938704668 >> [v2:172.21.15.38:6832/2178429646,v1:172.21.15.38:6833/2178429646] conn(0x55e1021e5e40 0x55e102622400 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a1c93690bb3b090 server_cookie=10feb513c049e6de in_seq=0 out_seq=0 2024-09-21T03:44:18.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.612+0000 7f09e17fa640 1 -- 172.21.15.38:0/938704668 <== mds.0 v2:172.21.15.38:6832/2178429646 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e102543770 con 0x55e1021e5e40 2024-09-21T03:44:18.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.612+0000 7f09e17fa640 10 client.10332 handle_command_reply: tid=0 2024-09-21T03:44:18.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.612+0000 7f0a0fe0b740 1 client.10332 shutdown 2024-09-21T03:44:18.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.613+0000 7f0a0fe0b740 1 -- 172.21.15.38:0/938704668 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e102227270 msgr2=0x55e10260b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:18.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.613+0000 7f0a0fe0b740 1 --2- 172.21.15.38:0/938704668 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e102227270 0x55e10260b1f0 secure :-1 s=READY pgs=2183 cs=0 l=1 rev1=1 crypto rx=0x7f09fc00fe10 tx=0x7f09fc0117b0 comp rx=0 tx=0).stop 2024-09-21T03:44:18.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.613+0000 7f0a0fe0b740 1 -- 172.21.15.38:0/938704668 shutdown_connections 2024-09-21T03:44:18.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.613+0000 7f0a0fe0b740 1 --2- 172.21.15.38:0/938704668 >> [v2:172.21.15.38:6832/2178429646,v1:172.21.15.38:6833/2178429646] conn(0x55e1021e5e40 0x55e102622400 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:18.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.613+0000 7f0a0fe0b740 1 --2- 172.21.15.38:0/938704668 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e102227270 0x55e10260b1f0 unknown :-1 s=CLOSED pgs=2183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:18.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.613+0000 7f0a0fe0b740 1 --2- 172.21.15.38:0/938704668 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e102206590 0x55e10260c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:18.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.613+0000 7f0a0fe0b740 1 --2- 172.21.15.38:0/938704668 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e1021fe3b0 0x55e10260dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:18.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.613+0000 7f09e17fa640 0 client.10332 ms_handle_reset on v2:172.21.15.38:6832/2178429646 2024-09-21T03:44:18.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.613+0000 7f0a0fe0b740 1 -- 172.21.15.38:0/938704668 >> 172.21.15.38:0/938704668 conn(0x55e10221b3c0 msgr2=0x55e1025cf3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:18.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.613+0000 7f0a0fe0b740 1 -- 172.21.15.38:0/938704668 shutdown_connections 2024-09-21T03:44:18.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.613+0000 7f0a0fe0b740 1 -- 172.21.15.38:0/938704668 wait complete. 2024-09-21T03:44:18.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.613+0000 7f0a0fe0b740 20 client.10332 trim_cache size 0 max 16384 2024-09-21T03:44:18.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.631+0000 7f0a0fe0b740 1 Processor -- start 2024-09-21T03:44:18.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.631+0000 7f0a0fe0b740 1 -- start start 2024-09-21T03:44:18.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.633+0000 7f0a0fe0b740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e1021fe3b0 0x55e1025c8740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:18.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.633+0000 7f0a0fe0b740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e102206590 0x55e10260abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:18.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.633+0000 7f0a0fe0b740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e102227270 0x55e10260b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:18.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.633+0000 7f0a0fe0b740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f0a04203880 con 0x55e102227270 2024-09-21T03:44:18.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.633+0000 7f0a0fe0b740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f0a04202bd0 con 0x55e1021fe3b0 2024-09-21T03:44:18.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.633+0000 7f0a0fe0b740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f0a042031f0 con 0x55e102206590 2024-09-21T03:44:18.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.633+0000 7f0a02d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e102227270 0x55e10260b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:18.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.633+0000 7f0a02d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e102227270 0x55e10260b0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38556/0 (socket says 172.21.15.38:38556) 2024-09-21T03:44:18.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.633+0000 7f0a02d9d640 1 -- 172.21.15.38:0/2812503068 learned_addr learned my addr 172.21.15.38:0/2812503068 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:44:18.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.633+0000 7f0a0259c640 1 --2- 172.21.15.38:0/2812503068 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e1021fe3b0 0x55e1025c8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:18.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.633+0000 7f0a01d9b640 1 --2- 172.21.15.38:0/2812503068 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e102206590 0x55e10260abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:18.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.633+0000 7f0a02d9d640 1 -- 172.21.15.38:0/2812503068 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e102206590 msgr2=0x55e10260abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:44:18.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.633+0000 7f0a02d9d640 1 --2- 172.21.15.38:0/2812503068 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e102206590 0x55e10260abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:18.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.633+0000 7f0a02d9d640 1 -- 172.21.15.38:0/2812503068 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e1021fe3b0 msgr2=0x55e1025c8740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:44:18.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.633+0000 7f0a02d9d640 1 --2- 172.21.15.38:0/2812503068 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e1021fe3b0 0x55e1025c8740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:18.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.633+0000 7f0a02d9d640 1 -- 172.21.15.38:0/2812503068 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e1025ca930 con 0x55e102227270 2024-09-21T03:44:18.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.633+0000 7f0a0259c640 1 --2- 172.21.15.38:0/2812503068 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e1021fe3b0 0x55e1025c8740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:44:18.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.633+0000 7f0a02d9d640 1 --2- 172.21.15.38:0/2812503068 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e102227270 0x55e10260b0f0 secure :-1 s=READY pgs=2184 cs=0 l=1 rev1=1 crypto rx=0x7f09fc00ff60 tx=0x7f09fc00ff90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34106667f1258ac4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:18.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.633+0000 7f09e37fe640 1 -- 172.21.15.38:0/2812503068 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09fc012860 con 0x55e102227270 2024-09-21T03:44:18.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.634+0000 7f09e37fe640 1 -- 172.21.15.38:0/2812503068 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f09fc02e070 con 0x55e102227270 2024-09-21T03:44:18.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.634+0000 7f09e37fe640 1 -- 172.21.15.38:0/2812503068 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09fc012a60 con 0x55e102227270 2024-09-21T03:44:18.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.634+0000 7f0a0fe0b740 1 -- 172.21.15.38:0/2812503068 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e102227270 msgr2=0x55e10260b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:18.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.634+0000 7f0a0fe0b740 1 --2- 172.21.15.38:0/2812503068 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e102227270 0x55e10260b0f0 secure :-1 s=READY pgs=2184 cs=0 l=1 rev1=1 crypto rx=0x7f09fc00ff60 tx=0x7f09fc00ff90 comp rx=0 tx=0).stop 2024-09-21T03:44:18.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.634+0000 7f0a0fe0b740 1 -- 172.21.15.38:0/2812503068 shutdown_connections 2024-09-21T03:44:18.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.634+0000 7f0a0fe0b740 1 --2- 172.21.15.38:0/2812503068 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e102227270 0x55e10260b0f0 unknown :-1 s=CLOSED pgs=2184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:18.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.634+0000 7f0a0fe0b740 1 --2- 172.21.15.38:0/2812503068 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e102206590 0x55e10260abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:18.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.634+0000 7f0a0fe0b740 1 --2- 172.21.15.38:0/2812503068 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e1021fe3b0 0x55e1025c8740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:18.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.634+0000 7f0a0fe0b740 1 -- 172.21.15.38:0/2812503068 >> 172.21.15.38:0/2812503068 conn(0x55e10221b3c0 msgr2=0x55e10260c210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:18.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.634+0000 7f0a0fe0b740 1 -- 172.21.15.38:0/2812503068 shutdown_connections 2024-09-21T03:44:18.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.634+0000 7f0a0fe0b740 1 -- 172.21.15.38:0/2812503068 wait complete. 2024-09-21T03:44:18.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.635+0000 7f0a0fe0b740 1 -- start start 2024-09-21T03:44:18.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.635+0000 7f0a0fe0b740 1 Processor -- start 2024-09-21T03:44:18.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.637+0000 7f0a0fe0b740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e1021fe3b0 0x55e102694200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:18.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.637+0000 7f0a0fe0b740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e102206590 0x55e102690bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:18.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.637+0000 7f0a0fe0b740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e102227270 0x55e1026158b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:18.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.637+0000 7f0a0fe0b740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55e1025f6090 con 0x55e102206590 2024-09-21T03:44:18.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.637+0000 7f0a0fe0b740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55e10260cae0 con 0x55e1021fe3b0 2024-09-21T03:44:18.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.637+0000 7f0a0fe0b740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55e10260b940 con 0x55e102227270 2024-09-21T03:44:18.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.637+0000 7f0a0fe0b740 10 client.0 _ll_register_callbacks cb 0x7f09fc00f360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:44:18.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.637+0000 7f0a01d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e102206590 0x55e102690bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:18.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.637+0000 7f0a01d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e102206590 0x55e102690bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38570/0 (socket says 172.21.15.38:38570) 2024-09-21T03:44:18.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.637+0000 7f0a01d9b640 1 -- 172.21.15.38:0/723761085 learned_addr learned my addr 172.21.15.38:0/723761085 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:44:18.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.637+0000 7f0a0259c640 1 --2- 172.21.15.38:0/723761085 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e1021fe3b0 0x55e102694200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:18.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.637+0000 7f0a02d9d640 1 --2- 172.21.15.38:0/723761085 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e102227270 0x55e1026158b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:18.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.637+0000 7f0a01d9b640 1 -- 172.21.15.38:0/723761085 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e102227270 msgr2=0x55e1026158b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:44:18.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.637+0000 7f0a01d9b640 1 --2- 172.21.15.38:0/723761085 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e102227270 0x55e1026158b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:18.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.637+0000 7f0a01d9b640 1 -- 172.21.15.38:0/723761085 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e1021fe3b0 msgr2=0x55e102694200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:44:18.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.637+0000 7f0a01d9b640 1 --2- 172.21.15.38:0/723761085 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e1021fe3b0 0x55e102694200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:18.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.637+0000 7f0a01d9b640 1 -- 172.21.15.38:0/723761085 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e102691100 con 0x55e102206590 2024-09-21T03:44:18.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.637+0000 7f0a02d9d640 1 --2- 172.21.15.38:0/723761085 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e102227270 0x55e1026158b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:44:18.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.637+0000 7f0a01d9b640 1 --2- 172.21.15.38:0/723761085 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e102206590 0x55e102690bc0 secure :-1 s=READY pgs=2185 cs=0 l=1 rev1=1 crypto rx=0x7f09ec005040 tx=0x7f09ec007500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f49a1b439087d6c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:18.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.637+0000 7f09e17fa640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-21T03:44:18.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.638+0000 7f09e17fa640 1 -- 172.21.15.38:0/723761085 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09ec031060 con 0x55e102206590 2024-09-21T03:44:18.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.638+0000 7f09e17fa640 1 -- 172.21.15.38:0/723761085 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(132..132 src has 1..132) ==== 9400+0+0 (secure 0 0 0) 0x7f09ec006be0 con 0x55e102206590 2024-09-21T03:44:18.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.638+0000 7f0a0fe0b740 1 -- 172.21.15.38:0/723761085 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f09fc02e070 con 0x55e102206590 2024-09-21T03:44:18.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.638+0000 7f09e17fa640 1 -- 172.21.15.38:0/723761085 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=312) ==== 24+0+0 (secure 0 0 0) 0x7f09ec038050 con 0x55e102206590 2024-09-21T03:44:18.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.638+0000 7f0a0fe0b740 10 client.10338 fetch_fsmap learned FSMap version 312 2024-09-21T03:44:18.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.638+0000 7f0a0fe0b740 1 -- 172.21.15.38:0/723761085 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=312,monmap=2+}) -- 0x55e102693a00 con 0x55e102206590 2024-09-21T03:44:18.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.638+0000 7f09e17fa640 1 -- 172.21.15.38:0/723761085 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f09fc02e070 con 0x55e102206590 2024-09-21T03:44:18.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.638+0000 7f09e17fa640 1 -- 172.21.15.38:0/723761085 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 312) ==== 3502+0+0 (secure 0 0 0) 0x7f09ec0035f0 con 0x55e102206590 2024-09-21T03:44:18.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.638+0000 7f0a0fe0b740 10 client.10338 fetch_fsmap finished waiting for FSMap version 312 2024-09-21T03:44:18.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.638+0000 7f0a0fe0b740 10 client.10338 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T03:44:18.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.638+0000 7f0a0fe0b740 20 client.10338 populate_metadata read hostname 'smithi038' 2024-09-21T03:44:18.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.638+0000 7f0a0fe0b740 1 --2- 172.21.15.38:0/723761085 >> [v2:172.21.15.38:6832/2178429646,v1:172.21.15.38:6833/2178429646] conn(0x55e1025f59a0 0x55e102636cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:18.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.638+0000 7f0a0fe0b740 4 client.10338 mds_command: new command op to 10275 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T03:44:18.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.638+0000 7f0a0fe0b740 1 -- 172.21.15.38:0/723761085 --> [v2:172.21.15.38:6832/2178429646,v1:172.21.15.38:6833/2178429646] -- command(tid 0: {"prefix": "status"}) -- 0x55e10253f220 con 0x55e1025f59a0 2024-09-21T03:44:18.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.638+0000 7f0a0259c640 1 --2- 172.21.15.38:0/723761085 >> [v2:172.21.15.38:6832/2178429646,v1:172.21.15.38:6833/2178429646] conn(0x55e1025f59a0 0x55e102636cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:18.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.639+0000 7f09e17fa640 10 client.10338 ms_handle_connect on v2:172.21.15.38:6832/2178429646 2024-09-21T03:44:18.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.639+0000 7f0a0259c640 1 --2- 172.21.15.38:0/723761085 >> [v2:172.21.15.38:6832/2178429646,v1:172.21.15.38:6833/2178429646] conn(0x55e1025f59a0 0x55e102636cc0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aca63765f667f3c5 server_cookie=b8102f447ce0814b in_seq=0 out_seq=0 2024-09-21T03:44:18.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.639+0000 7f09e17fa640 1 -- 172.21.15.38:0/723761085 <== mds.0 v2:172.21.15.38:6832/2178429646 1 ==== command_reply(tid 0: 0 ) ==== 8+0+335 (crc 0 0 0) 0x55e10253f220 con 0x55e1025f59a0 2024-09-21T03:44:18.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.639+0000 7f09e17fa640 10 client.10338 handle_command_reply: tid=0 2024-09-21T03:44:18.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.639+0000 7f0a0fe0b740 1 client.10338 shutdown 2024-09-21T03:44:18.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.639+0000 7f0a0fe0b740 1 -- 172.21.15.38:0/723761085 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e102206590 msgr2=0x55e102690bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:18.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.639+0000 7f0a0fe0b740 1 --2- 172.21.15.38:0/723761085 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e102206590 0x55e102690bc0 secure :-1 s=READY pgs=2185 cs=0 l=1 rev1=1 crypto rx=0x7f09ec005040 tx=0x7f09ec007500 comp rx=0 tx=0).stop 2024-09-21T03:44:18.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.639+0000 7f0a0fe0b740 1 -- 172.21.15.38:0/723761085 shutdown_connections 2024-09-21T03:44:18.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.639+0000 7f0a0fe0b740 1 --2- 172.21.15.38:0/723761085 >> [v2:172.21.15.38:6832/2178429646,v1:172.21.15.38:6833/2178429646] conn(0x55e1025f59a0 0x55e102636cc0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:18.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.639+0000 7f0a0fe0b740 1 --2- 172.21.15.38:0/723761085 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e102227270 0x55e1026158b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:18.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.639+0000 7f09e17fa640 0 client.10338 ms_handle_reset on v2:172.21.15.38:6832/2178429646 2024-09-21T03:44:18.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.639+0000 7f0a0fe0b740 1 --2- 172.21.15.38:0/723761085 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e102206590 0x55e102690bc0 unknown :-1 s=CLOSED pgs=2185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:18.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.639+0000 7f0a0fe0b740 1 --2- 172.21.15.38:0/723761085 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e1021fe3b0 0x55e102694200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:18.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.639+0000 7f0a0fe0b740 1 -- 172.21.15.38:0/723761085 >> 172.21.15.38:0/723761085 conn(0x55e10221b3c0 msgr2=0x55e10260c180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:18.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.639+0000 7f0a0fe0b740 1 -- 172.21.15.38:0/723761085 shutdown_connections 2024-09-21T03:44:18.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.639+0000 7f0a0fe0b740 1 -- 172.21.15.38:0/723761085 wait complete. 2024-09-21T03:44:18.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.639+0000 7f0a0fe0b740 20 client.10338 trim_cache size 0 max 16384 2024-09-21T03:44:18.648 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-09-21T03:44:18.648 INFO:teuthology.orchestra.run.smithi038.stdout: "cluster_fsid": "9d50d66f-656b-4fe2-8e3b-fe53b79c4d43", 2024-09-21T03:44:18.648 INFO:teuthology.orchestra.run.smithi038.stdout: "whoami": 0, 2024-09-21T03:44:18.649 INFO:teuthology.orchestra.run.smithi038.stdout: "id": 10275, 2024-09-21T03:44:18.649 INFO:teuthology.orchestra.run.smithi038.stdout: "want_state": "up:active", 2024-09-21T03:44:18.649 INFO:teuthology.orchestra.run.smithi038.stdout: "state": "up:active", 2024-09-21T03:44:18.649 INFO:teuthology.orchestra.run.smithi038.stdout: "fs_name": "cephfs", 2024-09-21T03:44:18.649 INFO:teuthology.orchestra.run.smithi038.stdout: "rank_uptime": 6.7429340609999997, 2024-09-21T03:44:18.649 INFO:teuthology.orchestra.run.smithi038.stdout: "mdsmap_epoch": 312, 2024-09-21T03:44:18.649 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch": 132, 2024-09-21T03:44:18.649 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch_barrier": 132, 2024-09-21T03:44:18.649 INFO:teuthology.orchestra.run.smithi038.stdout: "uptime": 7.9623382380000001 2024-09-21T03:44:18.649 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-09-21T03:44:18.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.640+0000 7f0a08e9b640 1 -- 172.21.15.38:0/1279006549 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f09e4078bc0 msgr2=0x7f09e407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:18.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.640+0000 7f0a08e9b640 1 --2- 172.21.15.38:0/1279006549 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f09e4078bc0 0x7f09e407b0d0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f0a040690e0 tx=0x7f09ec009040 comp rx=0 tx=0).stop 2024-09-21T03:44:18.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.640+0000 7f0a08e9b640 1 -- 172.21.15.38:0/1279006549 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0a040b9ad0 msgr2=0x7f0a0406f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:18.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.640+0000 7f0a08e9b640 1 --2- 172.21.15.38:0/1279006549 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0a040b9ad0 0x7f0a0406f1b0 secure :-1 s=READY pgs=2181 cs=0 l=1 rev1=1 crypto rx=0x7f09f4000f80 tx=0x7f09f4003f10 comp rx=0 tx=0).stop 2024-09-21T03:44:18.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.640+0000 7f0a08e9b640 1 -- 172.21.15.38:0/1279006549 shutdown_connections 2024-09-21T03:44:18.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.640+0000 7f0a08e9b640 1 --2- 172.21.15.38:0/1279006549 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f09e4078bc0 0x7f09e407b0d0 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:18.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.640+0000 7f0a08e9b640 1 --2- 172.21.15.38:0/1279006549 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0a0415ee80 0x7f0a041f9780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:18.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.640+0000 7f0a08e9b640 1 --2- 172.21.15.38:0/1279006549 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0a040ceb90 0x7f0a041f9240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:18.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.640+0000 7f0a08e9b640 1 --2- 172.21.15.38:0/1279006549 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0a040b9ad0 0x7f0a0406f1b0 unknown :-1 s=CLOSED pgs=2181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:18.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.640+0000 7f0a08e9b640 1 -- 172.21.15.38:0/1279006549 >> 172.21.15.38:0/1279006549 conn(0x7f0a0409d830 msgr2=0x7f0a040cadb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:18.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.640+0000 7f0a08e9b640 1 -- 172.21.15.38:0/1279006549 shutdown_connections 2024-09-21T03:44:18.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.640+0000 7f0a08e9b640 1 -- 172.21.15.38:0/1279006549 wait complete. 2024-09-21T03:44:18.653 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-09-21T03:44:18.653 INFO:tasks.cephfs.test_damage:Daemons came up after mutation 'Corrupt omap header on mds0_openfiles.0', proceeding to ls 2024-09-21T03:44:18.654 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T03:44:18.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.768+0000 7f851e939640 1 -- 172.21.15.38:0/133210802 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f85180d01f0 msgr2=0x7f851815daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:18.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.768+0000 7f851e939640 1 --2- 172.21.15.38:0/133210802 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f85180d01f0 0x7f851815daf0 secure :-1 s=READY pgs=2186 cs=0 l=1 rev1=1 crypto rx=0x7f85180cec40 tx=0x7f850802a560 comp rx=0 tx=0).stop 2024-09-21T03:44:18.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.768+0000 7f851e939640 1 -- 172.21.15.38:0/133210802 shutdown_connections 2024-09-21T03:44:18.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.768+0000 7f851e939640 1 --2- 172.21.15.38:0/133210802 >> v2:172.21.15.195:3300/0 conn(0x7f851815a6f0 0x7f851815e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:18.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.768+0000 7f851e939640 1 --2- 172.21.15.38:0/133210802 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f85180d01f0 0x7f851815daf0 unknown :-1 s=CLOSED pgs=2186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:18.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.768+0000 7f851e939640 1 -- 172.21.15.38:0/133210802 >> 172.21.15.38:0/133210802 conn(0x7f8518072990 msgr2=0x7f8518072da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:18.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.768+0000 7f851e939640 1 -- 172.21.15.38:0/133210802 shutdown_connections 2024-09-21T03:44:18.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.768+0000 7f851e939640 1 -- 172.21.15.38:0/133210802 wait complete. 2024-09-21T03:44:18.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.770+0000 7f851e939640 1 Processor -- start 2024-09-21T03:44:18.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.770+0000 7f851e939640 1 -- start start 2024-09-21T03:44:18.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.771+0000 7f851e939640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f85180d01f0 0x7f85180c8f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:18.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.771+0000 7f851e939640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f85180d0980 0x7f851814c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:18.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.771+0000 7f851e939640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f851815a6f0 0x7f85181e96f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:18.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.771+0000 7f851e939640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f85180cd130 con 0x7f85180d01f0 2024-09-21T03:44:18.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.771+0000 7f851e939640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f85180ccfb0 con 0x7f85180d0980 2024-09-21T03:44:18.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.771+0000 7f851e939640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f85180cd2b0 con 0x7f851815a6f0 2024-09-21T03:44:18.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.771+0000 7f8517fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f85180d01f0 0x7f85180c8f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:18.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.771+0000 7f8517fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f85180d01f0 0x7f85180c8f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38588/0 (socket says 172.21.15.38:38588) 2024-09-21T03:44:18.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.771+0000 7f8517fff640 1 -- 172.21.15.38:0/353520643 learned_addr learned my addr 172.21.15.38:0/353520643 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:44:18.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.772+0000 7f85177fe640 1 --2- 172.21.15.38:0/353520643 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f85180d0980 0x7f851814c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:18.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.772+0000 7f8517fff640 1 -- 172.21.15.38:0/353520643 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f851815a6f0 msgr2=0x7f85181e96f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:18.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.772+0000 7f851ced6640 1 --2- 172.21.15.38:0/353520643 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f851815a6f0 0x7f85181e96f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:18.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.772+0000 7f8517fff640 1 --2- 172.21.15.38:0/353520643 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f851815a6f0 0x7f85181e96f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:18.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.772+0000 7f8517fff640 1 -- 172.21.15.38:0/353520643 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f85180d0980 msgr2=0x7f851814c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:18.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.772+0000 7f8517fff640 1 --2- 172.21.15.38:0/353520643 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f85180d0980 0x7f851814c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:18.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.772+0000 7f8517fff640 1 -- 172.21.15.38:0/353520643 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f851806fdc0 con 0x7f85180d01f0 2024-09-21T03:44:18.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.772+0000 7f85177fe640 1 --2- 172.21.15.38:0/353520643 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f85180d0980 0x7f851814c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:44:18.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.772+0000 7f851ced6640 1 --2- 172.21.15.38:0/353520643 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f851815a6f0 0x7f85181e96f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:44:18.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.772+0000 7f8517fff640 1 --2- 172.21.15.38:0/353520643 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f85180d01f0 0x7f85180c8f80 secure :-1 s=READY pgs=2187 cs=0 l=1 rev1=1 crypto rx=0x7f8508034040 tx=0x7f8508003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:18.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.772+0000 7f8514ff9640 1 -- 172.21.15.38:0/353520643 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f850803e030 con 0x7f85180d01f0 2024-09-21T03:44:18.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.772+0000 7f8514ff9640 1 -- 172.21.15.38:0/353520643 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8508003e00 con 0x7f85180d01f0 2024-09-21T03:44:18.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.772+0000 7f851e939640 1 -- 172.21.15.38:0/353520643 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85181f5cf0 con 0x7f85180d01f0 2024-09-21T03:44:18.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.772+0000 7f8514ff9640 1 -- 172.21.15.38:0/353520643 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8508035040 con 0x7f85180d01f0 2024-09-21T03:44:18.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.772+0000 7f851e939640 1 -- 172.21.15.38:0/353520643 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85181f62b0 con 0x7f85180d01f0 2024-09-21T03:44:18.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.773+0000 7f851e939640 1 -- 172.21.15.38:0/353520643 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85180cd7e0 con 0x7f85180d01f0 2024-09-21T03:44:18.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.774+0000 7f8514ff9640 1 -- 172.21.15.38:0/353520643 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f8508041030 con 0x7f85180d01f0 2024-09-21T03:44:18.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.775+0000 7f8514ff9640 1 --2- 172.21.15.38:0/353520643 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8504078bc0 0x7f850407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:18.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.775+0000 7f8514ff9640 1 -- 172.21.15.38:0/353520643 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 9400+0+0 (secure 0 0 0) 0x7f85080bc820 con 0x7f85180d01f0 2024-09-21T03:44:18.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.775+0000 7f85177fe640 1 --2- 172.21.15.38:0/353520643 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8504078bc0 0x7f850407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:18.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.775+0000 7f85177fe640 1 --2- 172.21.15.38:0/353520643 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8504078bc0 0x7f850407b0d0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f85180cfc30 tx=0x7f850c005dd0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:18.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:18.778+0000 7f8514ff9640 1 -- 172.21.15.38:0/353520643 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f8508083eb0 con 0x7f85180d01f0 2024-09-21T03:44:19.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:19.008+0000 7f851e939640 1 -- 172.21.15.38:0/353520643 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f851815f8e0 con 0x7f85180d01f0 2024-09-21T03:44:19.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:19.008+0000 7f8514ff9640 1 -- 172.21.15.38:0/353520643 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v312) ==== 53+0+73 (secure 0 0 0) 0x7f85080837b0 con 0x7f85180d01f0 2024-09-21T03:44:19.011 INFO:teuthology.orchestra.run.smithi038.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T03:44:19.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:19.010+0000 7f851e939640 1 -- 172.21.15.38:0/353520643 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8504078bc0 msgr2=0x7f850407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:19.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:19.010+0000 7f851e939640 1 --2- 172.21.15.38:0/353520643 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8504078bc0 0x7f850407b0d0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f85180cfc30 tx=0x7f850c005dd0 comp rx=0 tx=0).stop 2024-09-21T03:44:19.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:19.010+0000 7f851e939640 1 -- 172.21.15.38:0/353520643 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f85180d01f0 msgr2=0x7f85180c8f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:19.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:19.010+0000 7f851e939640 1 --2- 172.21.15.38:0/353520643 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f85180d01f0 0x7f85180c8f80 secure :-1 s=READY pgs=2187 cs=0 l=1 rev1=1 crypto rx=0x7f8508034040 tx=0x7f8508003910 comp rx=0 tx=0).stop 2024-09-21T03:44:19.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:19.011+0000 7f851e939640 1 -- 172.21.15.38:0/353520643 shutdown_connections 2024-09-21T03:44:19.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:19.011+0000 7f851e939640 1 --2- 172.21.15.38:0/353520643 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8504078bc0 0x7f850407b0d0 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:19.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:19.011+0000 7f851e939640 1 --2- 172.21.15.38:0/353520643 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f851815a6f0 0x7f85181e96f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:19.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:19.011+0000 7f851e939640 1 --2- 172.21.15.38:0/353520643 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f85180d0980 0x7f851814c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:19.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:19.011+0000 7f851e939640 1 --2- 172.21.15.38:0/353520643 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f85180d01f0 0x7f85180c8f80 unknown :-1 s=CLOSED pgs=2187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:19.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:19.011+0000 7f851e939640 1 -- 172.21.15.38:0/353520643 >> 172.21.15.38:0/353520643 conn(0x7f8518072990 msgr2=0x7f851815c690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:19.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:19.011+0000 7f851e939640 1 -- 172.21.15.38:0/353520643 shutdown_connections 2024-09-21T03:44:19.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:19.011+0000 7f851e939640 1 -- 172.21.15.38:0/353520643 wait complete. 2024-09-21T03:44:19.024 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-21T03:44:19.024 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T03:44:19.024 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi038.front.sepia.ceph.com 2024-09-21T03:44:19.024 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-21T03:44:19.024 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T03:44:19.024 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-21T03:44:19.024 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-21T03:44:19.025 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T03:44:19.025 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T03:44:19.025 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:44:19.025 DEBUG:teuthology.orchestra.run.smithi038:> ip addr 2024-09-21T03:44:19.041 INFO:teuthology.orchestra.run.smithi038.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T03:44:19.041 INFO:teuthology.orchestra.run.smithi038.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T03:44:19.041 INFO:teuthology.orchestra.run.smithi038.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T03:44:19.041 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:44:19.041 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 ::1/128 scope host 2024-09-21T03:44:19.041 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:44:19.041 INFO:teuthology.orchestra.run.smithi038.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T03:44:19.042 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:6b:fd:a2 brd ff:ff:ff:ff:ff:ff 2024-09-21T03:44:19.042 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp7s0f0 2024-09-21T03:44:19.042 INFO:teuthology.orchestra.run.smithi038.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T03:44:19.042 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:6b:fd:a3 brd ff:ff:ff:ff:ff:ff 2024-09-21T03:44:19.042 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp7s0f1 2024-09-21T03:44:19.042 INFO:teuthology.orchestra.run.smithi038.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T03:44:19.042 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:bd:15:fc brd ff:ff:ff:ff:ff:ff 2024-09-21T03:44:19.042 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp1s0f0 2024-09-21T03:44:19.042 INFO:teuthology.orchestra.run.smithi038.stdout: inet 172.21.15.38/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-21T03:44:19.042 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft 42150sec preferred_lft 42150sec 2024-09-21T03:44:19.042 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 fe80::ec4:7aff:febd:15fc/64 scope link 2024-09-21T03:44:19.042 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:44:19.042 INFO:teuthology.orchestra.run.smithi038.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T03:44:19.043 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:bd:15:fd brd ff:ff:ff:ff:ff:ff 2024-09-21T03:44:19.043 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp1s0f1 2024-09-21T03:44:19.043 INFO:teuthology.orchestra.run.smithi038.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T03:44:19.043 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 92:55:c7:97:6f:b5 brd ff:ff:ff:ff:ff:ff 2024-09-21T03:44:19.043 INFO:teuthology.orchestra.run.smithi038.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-21T03:44:19.043 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:44:19.043 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 fe80::c49f:c1ff:fe21:b12a/64 scope link 2024-09-21T03:44:19.043 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:44:19.043 INFO:teuthology.orchestra.run.smithi038.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T03:44:19.043 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether a6:3a:83:ad:db:d5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T03:44:19.043 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 fe80::a43a:83ff:fead:dbd5/64 scope link 2024-09-21T03:44:19.043 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:44:19.043 INFO:teuthology.orchestra.run.smithi038.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T03:44:19.043 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 92:55:c7:97:6f:b5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T03:44:19.044 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 fe80::9055:c7ff:fe97:6fb5/64 scope link 2024-09-21T03:44:19.044 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:44:19.044 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:44:19.044 DEBUG:teuthology.orchestra.run.smithi038:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T03:44:19.106 INFO:teuthology.orchestra.run.smithi038.stdout:1 2024-09-21T03:44:19.107 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:44:19.107 DEBUG:teuthology.orchestra.run.smithi038:> route 2024-09-21T03:44:19.164 INFO:teuthology.orchestra.run.smithi038.stdout:Kernel IP routing table 2024-09-21T03:44:19.164 INFO:teuthology.orchestra.run.smithi038.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T03:44:19.164 INFO:teuthology.orchestra.run.smithi038.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-21T03:44:19.165 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-21T03:44:19.165 INFO:teuthology.orchestra.run.smithi038.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T03:44:19.165 DEBUG:teuthology.orchestra.run.smithi038:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T03:44:19.165 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-09-21T03:44:19.165 DEBUG:teuthology.orchestra.run.smithi038:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-21T03:44:19.165 DEBUG:teuthology.orchestra.run.smithi038:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-21T03:44:19.166 DEBUG:teuthology.orchestra.run.smithi038:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-21T03:44:19.166 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-09-21T03:44:19.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:19 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T03:44:19.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:44:19.294 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:44:19.294 DEBUG:teuthology.orchestra.run.smithi038:> ip netns list 2024-09-21T03:44:19.310 INFO:teuthology.orchestra.run.smithi038.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-21T03:44:19.310 INFO:teuthology.orchestra.run.smithi038.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-21T03:44:19.310 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T03:44:19.310 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:44:19.310 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip link set brx.0 up 2024-09-21T03:44:19.375 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T03:44:19.376 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-21T03:44:19.431 INFO:teuthology.orchestra.run.smithi038.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-21T03:44:19.431 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T03:44:19.431 DEBUG:teuthology.orchestra.run.smithi038:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-21T03:44:19.487 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-21T03:44:19.487 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:44:19.487 DEBUG:teuthology.orchestra.run.smithi038:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T03:44:19.582 INFO:teuthology.orchestra.run.smithi038.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T03:44:19.582 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T03:44:19.582 INFO:teuthology.orchestra.run.smithi038.stdout:invalid new device string format 2024-09-21T03:44:19.593 INFO:teuthology.orchestra.run.smithi038.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T03:44:19.593 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T03:44:19.593 INFO:teuthology.orchestra.run.smithi038.stdout:invalid new device string format 2024-09-21T03:44:19.593 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: resolved to: "172.21.15.38:3300,172.21.15.195:3300,172.21.15.195:3301" 2024-09-21T03:44:19.593 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T03:44:19.601 INFO:teuthology.orchestra.run.smithi038.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T03:44:19.601 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T03:44:19.601 INFO:teuthology.orchestra.run.smithi038.stdout:invalid new device string format 2024-09-21T03:44:19.601 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: resolved to: "172.21.15.38:3300,172.21.15.195:3300,172.21.15.195:3301" 2024-09-21T03:44:19.602 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T03:44:19.602 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: resolved to: "172.21.15.38:6789,172.21.15.195:6789,172.21.15.195:6790" 2024-09-21T03:44:19.602 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.38:6789,172.21.15.195:6789,172.21.15.195:6790:/ 2024-09-21T03:44:19.602 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=9d50d66f-656b-4fe2-8e3b-fe53b79c4d43" will pass to kernel 2024-09-21T03:44:19.602 INFO:teuthology.orchestra.run.smithi038.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-21T03:44:19.602 INFO:teuthology.orchestra.run.smithi038.stdout: You just mounted a file system that supports labels which does not 2024-09-21T03:44:19.602 INFO:teuthology.orchestra.run.smithi038.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-21T03:44:19.602 INFO:teuthology.orchestra.run.smithi038.stdout: applications will generate AVC messages and not be allowed access to 2024-09-21T03:44:19.603 INFO:teuthology.orchestra.run.smithi038.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-21T03:44:19.604 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T03:44:19.604 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:44:19.604 DEBUG:teuthology.orchestra.run.smithi038:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-21T03:44:19.629 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T03:44:19.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:19 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T03:44:19.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:44:19.726 INFO:teuthology.orchestra.run.smithi038.stdout:client8180 2024-09-21T03:44:19.727 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-21T03:44:19.727 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:44:19.743 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:44:19.743 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls -R .) 2024-09-21T03:44:19.786 INFO:teuthology.orchestra.run:waiting for 20 2024-09-21T03:44:19.801 INFO:teuthology.orchestra.run.smithi038.stdout:.: 2024-09-21T03:44:19.802 INFO:teuthology.orchestra.run.smithi038.stdout:subdir 2024-09-21T03:44:19.802 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:44:19.802 INFO:teuthology.orchestra.run.smithi038.stdout:./subdir: 2024-09-21T03:44:19.802 INFO:teuthology.orchestra.run.smithi038.stdout:sixmegs 2024-09-21T03:44:19.802 INFO:tasks.cephfs.test_damage:Result: Mutation 'Corrupt omap header on mds0_openfiles.0' did not caused DAMAGED state 2024-09-21T03:44:19.802 INFO:tasks.cephfs.test_damage:Applying mutation 'Corrupt omap header on 100.00000000' 2024-09-21T03:44:19.802 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-21T03:44:19.802 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:44:19.858 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-21T03:44:19.858 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:44:19.914 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T03:44:20.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:20.070+0000 7fd3c527f640 1 -- 172.21.15.38:0/18636023 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd3a8004800 msgr2=0x7fd3a8004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:20.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:20.070+0000 7fd3c527f640 1 --2- 172.21.15.38:0/18636023 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd3a8004800 0x7fd3a8004c10 secure :-1 s=READY pgs=2189 cs=0 l=1 rev1=1 crypto rx=0x7fd3b4009b20 tx=0x7fd3a8005830 comp rx=0 tx=0).stop 2024-09-21T03:44:20.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:20.070+0000 7fd3bf7fe640 1 -- 172.21.15.38:0/18636023 reap_dead start 2024-09-21T03:44:20.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:20.070+0000 7fd3c527f640 1 -- 172.21.15.38:0/18636023 shutdown_connections 2024-09-21T03:44:20.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:20.070+0000 7fd3c527f640 1 -- 172.21.15.38:0/18636023 >> 172.21.15.38:0/18636023 conn(0x7fd3c00b46e0 msgr2=0x7fd3c00d23e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:20.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:20.070+0000 7fd3c527f640 1 -- 172.21.15.38:0/18636023 shutdown_connections 2024-09-21T03:44:20.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:20.070+0000 7fd3c527f640 1 -- 172.21.15.38:0/18636023 wait complete. 2024-09-21T03:44:20.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:20.072+0000 7fd3c527f640 1 Processor -- start 2024-09-21T03:44:20.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:20.072+0000 7fd3c527f640 1 -- start start 2024-09-21T03:44:20.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:20.073+0000 7fd3c527f640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd3a8004800 0x7fd3c00af3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:20.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:20.073+0000 7fd3c527f640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd3c01f62e0 0x7fd3c006fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:20.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:20.073+0000 7fd3c527f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd3c015e090 0x7fd3c00b2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:20.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:20.073+0000 7fd3c527f640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fd3c00b6100 con 0x7fd3c015e090 2024-09-21T03:44:20.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:20.073+0000 7fd3c527f640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd3c00b5f80 con 0x7fd3a8004800 2024-09-21T03:44:20.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:20.073+0000 7fd3c527f640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fd3c00b6280 con 0x7fd3c01f62e0 2024-09-21T03:44:20.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:20.073+0000 7fd3bf7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd3c015e090 0x7fd3c00b2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:20.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:20.073+0000 7fd3bf7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd3c015e090 0x7fd3c00b2ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38612/0 (socket says 172.21.15.38:38612) 2024-09-21T03:44:20.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:20.073+0000 7fd3bf7fe640 1 -- 172.21.15.38:0/2974957294 learned_addr learned my addr 172.21.15.38:0/2974957294 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:44:20.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:20.073+0000 7fd3beffd640 1 --2- 172.21.15.38:0/2974957294 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd3a8004800 0x7fd3c00af3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:20.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:20.074+0000 7fd3be7fc640 1 --2- 172.21.15.38:0/2974957294 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd3c01f62e0 0x7fd3c006fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:20.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:20.074+0000 7fd3bf7fe640 1 -- 172.21.15.38:0/2974957294 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd3c01f62e0 msgr2=0x7fd3c006fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:20.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:20.074+0000 7fd3bf7fe640 1 --2- 172.21.15.38:0/2974957294 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd3c01f62e0 0x7fd3c006fb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:20.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:20.074+0000 7fd3bf7fe640 1 -- 172.21.15.38:0/2974957294 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd3a8004800 msgr2=0x7fd3c00af3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:20.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:20.074+0000 7fd3bf7fe640 1 --2- 172.21.15.38:0/2974957294 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd3a8004800 0x7fd3c00af3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:20.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:20.074+0000 7fd3bf7fe640 1 -- 172.21.15.38:0/2974957294 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3c01f5840 con 0x7fd3c015e090 2024-09-21T03:44:20.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:20.074+0000 7fd3bf7fe640 1 --2- 172.21.15.38:0/2974957294 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd3c015e090 0x7fd3c00b2ba0 secure :-1 s=READY pgs=2190 cs=0 l=1 rev1=1 crypto rx=0x7fd3c00cd070 tx=0x7fd3b8011ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:20.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:20.074+0000 7fd3a77fe640 1 -- 172.21.15.38:0/2974957294 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3b8004b20 con 0x7fd3c015e090 2024-09-21T03:44:20.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:20.074+0000 7fd3a77fe640 1 -- 172.21.15.38:0/2974957294 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd3b8004cc0 con 0x7fd3c015e090 2024-09-21T03:44:20.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:20.074+0000 7fd3c527f640 1 -- 172.21.15.38:0/2974957294 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3c00aff20 con 0x7fd3c015e090 2024-09-21T03:44:20.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:20.074+0000 7fd3a77fe640 1 -- 172.21.15.38:0/2974957294 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3b8003bd0 con 0x7fd3c015e090 2024-09-21T03:44:20.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:20.074+0000 7fd3c527f640 1 -- 172.21.15.38:0/2974957294 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3c01f5ae0 con 0x7fd3c015e090 2024-09-21T03:44:20.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:20.075+0000 7fd3c527f640 1 -- 172.21.15.38:0/2974957294 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3c00cd970 con 0x7fd3c015e090 2024-09-21T03:44:20.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:20.076+0000 7fd3a77fe640 1 -- 172.21.15.38:0/2974957294 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fd3b8004030 con 0x7fd3c015e090 2024-09-21T03:44:20.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:20.077+0000 7fd3a77fe640 1 --2- 172.21.15.38:0/2974957294 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd3ac078b90 0x7fd3ac07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:20.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:20.077+0000 7fd3a77fe640 1 -- 172.21.15.38:0/2974957294 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 9400+0+0 (secure 0 0 0) 0x7fd3b809c650 con 0x7fd3c015e090 2024-09-21T03:44:20.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:20.077+0000 7fd3beffd640 1 --2- 172.21.15.38:0/2974957294 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd3ac078b90 0x7fd3ac07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:20.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:20.077+0000 7fd3beffd640 1 --2- 172.21.15.38:0/2974957294 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd3ac078b90 0x7fd3ac07b0a0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fd3b0001eb0 tx=0x7fd3b0001110 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:20.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:20.081+0000 7fd3a77fe640 1 -- 172.21.15.38:0/2974957294 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fd3c00cd970 con 0x7fd3c015e090 2024-09-21T03:44:20.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:20.279+0000 7fd3c527f640 1 -- 172.21.15.38:0/2974957294 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fd3c00c8d50 con 0x7fd3c015e090 2024-09-21T03:44:20.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:20.280+0000 7fd3a77fe640 1 -- 172.21.15.38:0/2974957294 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 72 entries v132) ==== 81+0+4376 (secure 0 0 0) 0x7fd3b8063cf0 con 0x7fd3c015e090 2024-09-21T03:44:20.282 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6833/3103607164 2024-09-22T03:44:05.867078+0000 2024-09-21T03:44:20.282 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6832/3103607164 2024-09-22T03:44:05.867078+0000 2024-09-21T03:44:20.282 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6832/2444649344 2024-09-22T03:44:00.664196+0000 2024-09-21T03:44:20.282 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6833/626794495 2024-09-22T03:43:49.180309+0000 2024-09-21T03:44:20.283 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6832/1643781179 2024-09-22T03:43:33.596532+0000 2024-09-21T03:44:20.283 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6833/2309772457 2024-09-22T03:43:28.198223+0000 2024-09-21T03:44:20.283 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6832/2309772457 2024-09-22T03:43:28.198223+0000 2024-09-21T03:44:20.283 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6835/141447142 2024-09-22T03:43:14.684929+0000 2024-09-21T03:44:20.283 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6834/141447142 2024-09-22T03:43:14.684929+0000 2024-09-21T03:44:20.283 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6835/3572946698 2024-09-22T03:41:44.737273+0000 2024-09-21T03:44:20.283 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6833/3553953266 2024-09-22T03:41:29.639383+0000 2024-09-21T03:44:20.283 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6834/2751962827 2024-09-22T03:40:38.029580+0000 2024-09-21T03:44:20.283 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6832/3553953266 2024-09-22T03:41:29.639383+0000 2024-09-21T03:44:20.283 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6834/2930805359 2024-09-22T03:42:23.215190+0000 2024-09-21T03:44:20.283 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6832/626794495 2024-09-22T03:43:49.180309+0000 2024-09-21T03:44:20.283 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6833/1643781179 2024-09-22T03:43:33.596532+0000 2024-09-21T03:44:20.284 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6835/2748959970 2024-09-22T03:41:16.565736+0000 2024-09-21T03:44:20.284 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6833/3710736169 2024-09-22T03:40:59.454480+0000 2024-09-21T03:44:20.284 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6833/1995903506 2024-09-22T03:43:00.406120+0000 2024-09-21T03:44:20.284 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6833/775682959 2024-09-22T03:43:10.570678+0000 2024-09-21T03:44:20.284 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6834/2748959970 2024-09-22T03:41:16.565736+0000 2024-09-21T03:44:20.284 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6834/2934569248 2024-09-22T03:40:26.934591+0000 2024-09-21T03:44:20.284 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6832/3726830644 2024-09-22T03:41:12.532341+0000 2024-09-21T03:44:20.284 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6835/2097789246 2024-09-22T03:43:06.604471+0000 2024-09-21T03:44:20.284 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6835/2934569248 2024-09-22T03:40:26.934591+0000 2024-09-21T03:44:20.284 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6835/3864487171 2024-09-22T03:40:31.989837+0000 2024-09-21T03:44:20.284 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6833/4221150556 2024-09-22T03:42:27.240627+0000 2024-09-21T03:44:20.284 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6832/121492560 2024-09-22T03:41:10.371933+0000 2024-09-21T03:44:20.285 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6832/1717955365 2024-09-22T03:41:51.787804+0000 2024-09-21T03:44:20.285 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6834/2418225210 2024-09-22T03:42:02.869612+0000 2024-09-21T03:44:20.285 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6835/2418225210 2024-09-22T03:42:02.869612+0000 2024-09-21T03:44:20.285 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6833/121492560 2024-09-22T03:41:10.371933+0000 2024-09-21T03:44:20.285 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6835/3951934534 2024-09-22T03:42:31.276099+0000 2024-09-21T03:44:20.285 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6833/751618334 2024-09-22T03:40:42.053279+0000 2024-09-21T03:44:20.285 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.195:6837/712370966 2024-09-22T03:40:11.633787+0000 2024-09-21T03:44:20.285 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6832/3402963947 2024-09-22T03:41:23.555005+0000 2024-09-21T03:44:20.285 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6835/2798021708 2024-09-22T03:42:47.436632+0000 2024-09-21T03:44:20.285 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6833/3726830644 2024-09-22T03:41:12.532341+0000 2024-09-21T03:44:20.285 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6834/2916238332 2024-09-22T03:41:42.553770+0000 2024-09-21T03:44:20.285 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6834/3055025811 2024-09-22T03:42:39.361236+0000 2024-09-21T03:44:20.286 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.195:6835/712370966 2024-09-22T03:40:11.633787+0000 2024-09-21T03:44:20.286 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6834/3864487171 2024-09-22T03:40:31.989837+0000 2024-09-21T03:44:20.286 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6834/3572946698 2024-09-22T03:41:44.737273+0000 2024-09-21T03:44:20.286 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6832/1995903506 2024-09-22T03:43:00.406120+0000 2024-09-21T03:44:20.286 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6832/775682959 2024-09-22T03:43:10.570678+0000 2024-09-21T03:44:20.286 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6835/2916238332 2024-09-22T03:41:42.553770+0000 2024-09-21T03:44:20.286 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6833/3402963947 2024-09-22T03:41:23.555005+0000 2024-09-21T03:44:20.286 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6834/425672974 2024-09-22T03:40:46.105521+0000 2024-09-21T03:44:20.286 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6834/3951934534 2024-09-22T03:42:31.276099+0000 2024-09-21T03:44:20.286 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6835/425672974 2024-09-22T03:40:46.105521+0000 2024-09-21T03:44:20.286 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6832/3710736169 2024-09-22T03:40:59.454480+0000 2024-09-21T03:44:20.286 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6833/1717955365 2024-09-22T03:41:51.787804+0000 2024-09-21T03:44:20.286 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6835/3055025811 2024-09-22T03:42:39.361236+0000 2024-09-21T03:44:20.287 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6835/2751962827 2024-09-22T03:40:38.029580+0000 2024-09-21T03:44:20.287 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6835/2712360296 2024-09-22T03:42:20.931115+0000 2024-09-21T03:44:20.287 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6833/3537258034 2024-09-22T03:42:06.901189+0000 2024-09-21T03:44:20.287 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6832/3537258034 2024-09-22T03:42:06.901189+0000 2024-09-21T03:44:20.287 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6832/1272005650 2024-09-22T03:41:58.836862+0000 2024-09-21T03:44:20.287 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6834/2712360296 2024-09-22T03:42:20.931115+0000 2024-09-21T03:44:20.287 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6832/751618334 2024-09-22T03:40:42.053279+0000 2024-09-21T03:44:20.287 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6835/2930805359 2024-09-22T03:42:23.215190+0000 2024-09-21T03:44:20.287 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6832/4221150556 2024-09-22T03:42:27.240627+0000 2024-09-21T03:44:20.287 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6833/1272005650 2024-09-22T03:41:58.836862+0000 2024-09-21T03:44:20.287 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6832/245538755 2024-09-22T03:42:35.318756+0000 2024-09-21T03:44:20.287 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6833/245538755 2024-09-22T03:42:35.318756+0000 2024-09-21T03:44:20.287 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6833/2444649344 2024-09-22T03:44:00.664196+0000 2024-09-21T03:44:20.287 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6833/2459386631 2024-09-22T03:42:43.395609+0000 2024-09-21T03:44:20.288 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6832/2459386631 2024-09-22T03:42:43.395609+0000 2024-09-21T03:44:20.288 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6834/2097789246 2024-09-22T03:43:06.604471+0000 2024-09-21T03:44:20.288 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6834/2798021708 2024-09-22T03:42:47.436632+0000 2024-09-21T03:44:20.288 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6832/111206731 2024-09-22T03:43:02.560071+0000 2024-09-21T03:44:20.288 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6833/111206731 2024-09-22T03:43:02.560071+0000 2024-09-21T03:44:20.289 INFO:teuthology.orchestra.run.smithi038.stderr:listed 72 entries 2024-09-21T03:44:20.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:20.282+0000 7fd3c527f640 1 -- 172.21.15.38:0/2974957294 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd3ac078b90 msgr2=0x7fd3ac07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:20.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:20.282+0000 7fd3c527f640 1 --2- 172.21.15.38:0/2974957294 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd3ac078b90 0x7fd3ac07b0a0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fd3b0001eb0 tx=0x7fd3b0001110 comp rx=0 tx=0).stop 2024-09-21T03:44:20.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:20.282+0000 7fd3c527f640 1 -- 172.21.15.38:0/2974957294 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd3c015e090 msgr2=0x7fd3c00b2ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:20.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:20.282+0000 7fd3c527f640 1 --2- 172.21.15.38:0/2974957294 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd3c015e090 0x7fd3c00b2ba0 secure :-1 s=READY pgs=2190 cs=0 l=1 rev1=1 crypto rx=0x7fd3c00cd070 tx=0x7fd3b8011ef0 comp rx=0 tx=0).stop 2024-09-21T03:44:20.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:20.282+0000 7fd3c527f640 1 -- 172.21.15.38:0/2974957294 shutdown_connections 2024-09-21T03:44:20.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:20.282+0000 7fd3c527f640 1 --2- 172.21.15.38:0/2974957294 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd3c015e090 0x7fd3c00b2ba0 unknown :-1 s=CLOSED pgs=2190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:20.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:20.282+0000 7fd3c527f640 1 --2- 172.21.15.38:0/2974957294 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd3ac078b90 0x7fd3ac07b0a0 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:20.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:20.282+0000 7fd3c527f640 1 --2- 172.21.15.38:0/2974957294 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd3c01f62e0 0x7fd3c006fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:20.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:20.282+0000 7fd3c527f640 1 --2- 172.21.15.38:0/2974957294 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd3a8004800 0x7fd3c00af3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:20.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:20.282+0000 7fd3c527f640 1 -- 172.21.15.38:0/2974957294 >> 172.21.15.38:0/2974957294 conn(0x7fd3c00b46e0 msgr2=0x7fd3c00b37b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:20.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:20.283+0000 7fd3c527f640 1 -- 172.21.15.38:0/2974957294 shutdown_connections 2024-09-21T03:44:20.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:20.283+0000 7fd3c527f640 1 -- 172.21.15.38:0/2974957294 wait complete. 2024-09-21T03:44:20.295 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-21T03:44:20.296 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:44:20.296 DEBUG:teuthology.orchestra.run.smithi038:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-09-21T03:44:20.338 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi038.front.sepia.ceph.com 2024-09-21T03:44:20.338 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:44:20.338 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T03:44:20.395 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-21T03:44:20.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:20.549+0000 7f0ce02ec640 1 -- 172.21.15.38:0/4288624508 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0cd80d1120 msgr2=0x7f0cd80ceb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:20.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:20.550+0000 7f0ce02ec640 1 --2- 172.21.15.38:0/4288624508 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0cd80d1120 0x7f0cd80ceb90 secure :-1 s=READY pgs=2191 cs=0 l=1 rev1=1 crypto rx=0x7f0cd80cfba0 tx=0x7f0cd002a980 comp rx=0 tx=0).stop 2024-09-21T03:44:20.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:20.550+0000 7f0ce02ec640 1 -- 172.21.15.38:0/4288624508 shutdown_connections 2024-09-21T03:44:20.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:20.550+0000 7f0ce02ec640 1 --2- 172.21.15.38:0/4288624508 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0cd80d1120 0x7f0cd80ceb90 unknown :-1 s=CLOSED pgs=2191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:20.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:20.550+0000 7f0ce02ec640 1 --2- 172.21.15.38:0/4288624508 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0cd80b9ad0 0x7f0cd80d0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:20.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:20.550+0000 7f0ce02ec640 1 --2- 172.21.15.38:0/4288624508 >> v2:172.21.15.195:3300/0 conn(0x7f0cd815ee80 0x7f0cd80d0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:20.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:20.550+0000 7f0ce02ec640 1 -- 172.21.15.38:0/4288624508 >> 172.21.15.38:0/4288624508 conn(0x7f0cd809d830 msgr2=0x7f0cd80b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:20.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:20.550+0000 7f0ce02ec640 1 -- 172.21.15.38:0/4288624508 shutdown_connections 2024-09-21T03:44:20.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:20.550+0000 7f0ce02ec640 1 -- 172.21.15.38:0/4288624508 wait complete. 2024-09-21T03:44:20.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:20.551+0000 7f0ce02ec640 1 Processor -- start 2024-09-21T03:44:20.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:20.552+0000 7f0ce02ec640 1 -- start start 2024-09-21T03:44:20.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:20.553+0000 7f0ce02ec640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0cd80b9ad0 0x7f0cd809e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:20.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:20.553+0000 7f0ce02ec640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0cd80d1120 0x7f0cd806efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:20.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:20.553+0000 7f0ce02ec640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0cd815ee80 0x7f0cd806f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:20.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:20.553+0000 7f0ce02ec640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f0cd80cf510 con 0x7f0cd80d1120 2024-09-21T03:44:20.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:20.553+0000 7f0ce02ec640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f0cd80cf390 con 0x7f0cd815ee80 2024-09-21T03:44:20.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:20.553+0000 7f0ce02ec640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f0cd80b6580 con 0x7f0cd80b9ad0 2024-09-21T03:44:20.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:20.553+0000 7f0cdd887640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0cd80d1120 0x7f0cd806efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:20.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:20.553+0000 7f0cdd887640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0cd80d1120 0x7f0cd806efb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38630/0 (socket says 172.21.15.38:38630) 2024-09-21T03:44:20.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:20.553+0000 7f0cdd887640 1 -- 172.21.15.38:0/206186943 learned_addr learned my addr 172.21.15.38:0/206186943 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:44:20.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:20.553+0000 7f0cde889640 1 --2- 172.21.15.38:0/206186943 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0cd815ee80 0x7f0cd806f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:20.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:20.553+0000 7f0cde088640 1 --2- 172.21.15.38:0/206186943 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0cd80b9ad0 0x7f0cd809e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:20.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:20.553+0000 7f0cdd887640 1 -- 172.21.15.38:0/206186943 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0cd80b9ad0 msgr2=0x7f0cd809e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:20.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:20.553+0000 7f0cdd887640 1 --2- 172.21.15.38:0/206186943 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0cd80b9ad0 0x7f0cd809e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:20.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:20.553+0000 7f0cdd887640 1 -- 172.21.15.38:0/206186943 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0cd815ee80 msgr2=0x7f0cd806f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:20.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:20.553+0000 7f0cdd887640 1 --2- 172.21.15.38:0/206186943 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0cd815ee80 0x7f0cd806f4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:20.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:20.553+0000 7f0cdd887640 1 -- 172.21.15.38:0/206186943 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cd80a2280 con 0x7f0cd80d1120 2024-09-21T03:44:20.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:20.553+0000 7f0cde088640 1 --2- 172.21.15.38:0/206186943 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0cd80b9ad0 0x7f0cd809e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:44:20.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:20.554+0000 7f0cdd887640 1 --2- 172.21.15.38:0/206186943 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0cd80d1120 0x7f0cd806efb0 secure :-1 s=READY pgs=2192 cs=0 l=1 rev1=1 crypto rx=0x7f0cc800d680 tx=0x7f0cc800db40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:20.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:20.554+0000 7f0cceffd640 1 -- 172.21.15.38:0/206186943 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cc8017070 con 0x7f0cd80d1120 2024-09-21T03:44:20.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:20.554+0000 7f0cceffd640 1 -- 172.21.15.38:0/206186943 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0cc8004310 con 0x7f0cd80d1120 2024-09-21T03:44:20.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:20.554+0000 7f0cceffd640 1 -- 172.21.15.38:0/206186943 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cc8009a20 con 0x7f0cd80d1120 2024-09-21T03:44:20.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:20.554+0000 7f0ce02ec640 1 -- 172.21.15.38:0/206186943 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cd809e7e0 con 0x7f0cd80d1120 2024-09-21T03:44:20.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:20.554+0000 7f0ce02ec640 1 -- 172.21.15.38:0/206186943 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cd80d4160 con 0x7f0cd80d1120 2024-09-21T03:44:20.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:20.555+0000 7f0ce02ec640 1 -- 172.21.15.38:0/206186943 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0cd80d43a0 con 0x7f0cd80d1120 2024-09-21T03:44:20.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:20.556+0000 7f0cceffd640 1 -- 172.21.15.38:0/206186943 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f0cc8009bc0 con 0x7f0cd80d1120 2024-09-21T03:44:20.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:20.556+0000 7f0cceffd640 1 --2- 172.21.15.38:0/206186943 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0cbc078b90 0x7f0cbc07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:20.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:20.556+0000 7f0cceffd640 1 -- 172.21.15.38:0/206186943 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 9400+0+0 (secure 0 0 0) 0x7f0cc809bc60 con 0x7f0cd80d1120 2024-09-21T03:44:20.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:20.556+0000 7f0cde088640 1 --2- 172.21.15.38:0/206186943 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0cbc078b90 0x7f0cbc07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:20.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:20.557+0000 7f0cde088640 1 --2- 172.21.15.38:0/206186943 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0cbc078b90 0x7f0cbc07b0a0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f0cd4005fd0 tx=0x7f0cd400f360 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:20.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:20.561+0000 7f0cceffd640 1 -- 172.21.15.38:0/206186943 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f0cd80d43a0 con 0x7f0cd80d1120 2024-09-21T03:44:20.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:20.786+0000 7f0ce02ec640 1 -- 172.21.15.38:0/206186943 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f0cd80d45d0 con 0x7f0cd80d1120 2024-09-21T03:44:20.788 INFO:tasks.ceph.mon.a.smithi038.stderr:2024-09-21T03:44:20.787+0000 7f3f45713640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T03:44:20.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:20.788+0000 7f0cceffd640 1 -- 172.21.15.38:0/206186943 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v313) ==== 161+0+0 (secure 0 0 0) 0x7f0cc8062b90 con 0x7f0cd80d1120 2024-09-21T03:44:20.790 INFO:teuthology.orchestra.run.smithi038.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-21T03:44:20.793 INFO:tasks.ceph.mds.c.smithi038.stderr: -1872> 2024-09-21T03:44:11.900+0000 7f468abb9640 -1 mds.0.openfiles _load_finish: corrupted header/values: End of buffer [buffer:2] 2024-09-21T03:44:20.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:20.792+0000 7f0ce02ec640 1 -- 172.21.15.38:0/206186943 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0cbc078b90 msgr2=0x7f0cbc07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:20.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:20.792+0000 7f0ce02ec640 1 --2- 172.21.15.38:0/206186943 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0cbc078b90 0x7f0cbc07b0a0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f0cd4005fd0 tx=0x7f0cd400f360 comp rx=0 tx=0).stop 2024-09-21T03:44:20.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:20.792+0000 7f0ce02ec640 1 -- 172.21.15.38:0/206186943 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0cd80d1120 msgr2=0x7f0cd806efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:20.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:20.792+0000 7f0ce02ec640 1 --2- 172.21.15.38:0/206186943 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0cd80d1120 0x7f0cd806efb0 secure :-1 s=READY pgs=2192 cs=0 l=1 rev1=1 crypto rx=0x7f0cc800d680 tx=0x7f0cc800db40 comp rx=0 tx=0).stop 2024-09-21T03:44:20.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:20.792+0000 7f0ce02ec640 1 -- 172.21.15.38:0/206186943 shutdown_connections 2024-09-21T03:44:20.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:20.792+0000 7f0ce02ec640 1 --2- 172.21.15.38:0/206186943 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0cd815ee80 0x7f0cd806f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:20.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:20.792+0000 7f0ce02ec640 1 --2- 172.21.15.38:0/206186943 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0cd80d1120 0x7f0cd806efb0 secure :-1 s=CLOSED pgs=2192 cs=0 l=1 rev1=1 crypto rx=0x7f0cc800d680 tx=0x7f0cc800db40 comp rx=0 tx=0).stop 2024-09-21T03:44:20.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:20.792+0000 7f0ce02ec640 1 --2- 172.21.15.38:0/206186943 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0cbc078b90 0x7f0cbc07b0a0 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:20.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:20.792+0000 7f0ce02ec640 1 --2- 172.21.15.38:0/206186943 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0cd80b9ad0 0x7f0cd809e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:20.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:20.792+0000 7f0ce02ec640 1 -- 172.21.15.38:0/206186943 >> 172.21.15.38:0/206186943 conn(0x7f0cd809d830 msgr2=0x7f0cd8160950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:20.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:20.792+0000 7f0ce02ec640 1 -- 172.21.15.38:0/206186943 shutdown_connections 2024-09-21T03:44:20.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:20.792+0000 7f0ce02ec640 1 -- 172.21.15.38:0/206186943 wait complete. 2024-09-21T03:44:20.795 INFO:tasks.ceph.mds.c.smithi038.stderr: -684> 2024-09-21T03:44:15.889+0000 7f468ebc1640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T03:44:20.806 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-09-21T03:44:20.851 INFO:tasks.ceph.mds.c.smithi038.stdout:starting mds.c at 2024-09-21T03:44:20.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:20.923+0000 7f50fa212640 1 -- 172.21.15.38:0/3887113754 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f50f40ccc40 msgr2=0x7f50f415ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:20.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:20.923+0000 7f50fa212640 1 --2- 172.21.15.38:0/3887113754 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f50f40ccc40 0x7f50f415ca90 secure :-1 s=READY pgs=2195 cs=0 l=1 rev1=1 crypto rx=0x7f50f40cd050 tx=0x7f50e402a4a0 comp rx=0 tx=0).stop 2024-09-21T03:44:20.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:20.923+0000 7f50fa212640 1 -- 172.21.15.38:0/3887113754 shutdown_connections 2024-09-21T03:44:20.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:20.923+0000 7f50fa212640 1 --2- 172.21.15.38:0/3887113754 >> v2:172.21.15.195:3300/0 conn(0x7f50f40d0d80 0x7f50f40b5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:20.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:20.923+0000 7f50fa212640 1 --2- 172.21.15.38:0/3887113754 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f50f40ccc40 0x7f50f415ca90 unknown :-1 s=CLOSED pgs=2195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:20.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:20.923+0000 7f50fa212640 1 -- 172.21.15.38:0/3887113754 >> 172.21.15.38:0/3887113754 conn(0x7f50f40b46e0 msgr2=0x7f50f40d23e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:20.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:20.923+0000 7f50fa212640 1 -- 172.21.15.38:0/3887113754 shutdown_connections 2024-09-21T03:44:20.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:20.923+0000 7f50fa212640 1 -- 172.21.15.38:0/3887113754 wait complete. 2024-09-21T03:44:20.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:20.925+0000 7f50fa212640 1 Processor -- start 2024-09-21T03:44:20.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:20.925+0000 7f50fa212640 1 -- start start 2024-09-21T03:44:20.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:20.926+0000 7f50fa212640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f50f40ccc40 0x7f50f415aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:20.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:20.926+0000 7f50fa212640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f50f40cd440 0x7f50f4154810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:20.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:20.926+0000 7f50fa212640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f50f41f5670 0x7f50f41f5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:20.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:20.926+0000 7f50fa212640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f50f40b6650 con 0x7f50f40ccc40 2024-09-21T03:44:20.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:20.926+0000 7f50fa212640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f50f40b64d0 con 0x7f50f41f5670 2024-09-21T03:44:20.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:20.926+0000 7f50fa212640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f50f40b67d0 con 0x7f50f40cd440 2024-09-21T03:44:20.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:20.926+0000 7f50f37fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f50f40ccc40 0x7f50f415aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:20.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:20.926+0000 7f50f37fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f50f40ccc40 0x7f50f415aef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38644/0 (socket says 172.21.15.38:38644) 2024-09-21T03:44:20.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:20.926+0000 7f50f37fe640 1 -- 172.21.15.38:0/977260875 learned_addr learned my addr 172.21.15.38:0/977260875 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:44:20.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:20.927+0000 7f50eaffd640 1 --2- 172.21.15.38:0/977260875 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f50f40cd440 0x7f50f4154810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:20.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:20.927+0000 7f50f3fff640 1 --2- 172.21.15.38:0/977260875 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f50f41f5670 0x7f50f41f5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:20.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:20.927+0000 7f50f37fe640 1 -- 172.21.15.38:0/977260875 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f50f40cd440 msgr2=0x7f50f4154810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:20.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:20.927+0000 7f50f37fe640 1 --2- 172.21.15.38:0/977260875 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f50f40cd440 0x7f50f4154810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:20.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:20.927+0000 7f50f37fe640 1 -- 172.21.15.38:0/977260875 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f50f41f5670 msgr2=0x7f50f41f5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:20.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:20.927+0000 7f50f37fe640 1 --2- 172.21.15.38:0/977260875 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f50f41f5670 0x7f50f41f5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:20.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:20.927+0000 7f50f37fe640 1 -- 172.21.15.38:0/977260875 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50f40ca510 con 0x7f50f40ccc40 2024-09-21T03:44:20.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:20.927+0000 7f50eaffd640 1 --2- 172.21.15.38:0/977260875 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f50f40cd440 0x7f50f4154810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:44:20.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:20.927+0000 7f50f37fe640 1 --2- 172.21.15.38:0/977260875 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f50f40ccc40 0x7f50f415aef0 secure :-1 s=READY pgs=2196 cs=0 l=1 rev1=1 crypto rx=0x7f50e4009780 tx=0x7f50e4033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:20.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:20.927+0000 7f50f0ff9640 1 -- 172.21.15.38:0/977260875 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50e4031d60 con 0x7f50f40ccc40 2024-09-21T03:44:20.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:20.927+0000 7f50f0ff9640 1 -- 172.21.15.38:0/977260875 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f50e4033e10 con 0x7f50f40ccc40 2024-09-21T03:44:20.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:20.927+0000 7f50fa212640 1 -- 172.21.15.38:0/977260875 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50f40ca7f0 con 0x7f50f40ccc40 2024-09-21T03:44:20.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:20.927+0000 7f50f0ff9640 1 -- 172.21.15.38:0/977260875 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50e4005260 con 0x7f50f40ccc40 2024-09-21T03:44:20.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:20.927+0000 7f50fa212640 1 -- 172.21.15.38:0/977260875 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50f406f150 con 0x7f50f40ccc40 2024-09-21T03:44:20.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:20.928+0000 7f50fa212640 1 -- 172.21.15.38:0/977260875 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50f40b6cf0 con 0x7f50f40ccc40 2024-09-21T03:44:20.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:20.929+0000 7f50f0ff9640 1 -- 172.21.15.38:0/977260875 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f50e4005400 con 0x7f50f40ccc40 2024-09-21T03:44:20.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:20.929+0000 7f50f0ff9640 1 --2- 172.21.15.38:0/977260875 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f50d4078bc0 0x7f50d407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:20.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:20.930+0000 7f50f0ff9640 1 -- 172.21.15.38:0/977260875 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 9486+0+0 (secure 0 0 0) 0x7f50e40bce80 con 0x7f50f40ccc40 2024-09-21T03:44:20.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:20.930+0000 7f50eaffd640 1 --2- 172.21.15.38:0/977260875 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f50d4078bc0 0x7f50d407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:20.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:20.930+0000 7f50eaffd640 1 --2- 172.21.15.38:0/977260875 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f50d4078bc0 0x7f50d407b0d0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f50f41603f0 tx=0x7f50dc009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:20.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:20.934+0000 7f50f0ff9640 1 -- 172.21.15.38:0/977260875 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f50e403f030 con 0x7f50f40ccc40 2024-09-21T03:44:21.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.199+0000 7f50fa212640 1 -- 172.21.15.38:0/977260875 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7f50f40c8970 con 0x7f50f40ccc40 2024-09-21T03:44:21.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.795+0000 7f50f0ff9640 1 -- 172.21.15.38:0/977260875 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 nothing to do: rank is not damaged v314) ==== 107+0+0 (secure 0 0 0) 0x7f50e40843e0 con 0x7f50f40ccc40 2024-09-21T03:44:21.797 INFO:teuthology.orchestra.run.smithi038.stderr:nothing to do: rank is not damaged 2024-09-21T03:44:21.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.798+0000 7f50fa212640 1 -- 172.21.15.38:0/977260875 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f50d4078bc0 msgr2=0x7f50d407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:21.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.798+0000 7f50fa212640 1 --2- 172.21.15.38:0/977260875 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f50d4078bc0 0x7f50d407b0d0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f50f41603f0 tx=0x7f50dc009040 comp rx=0 tx=0).stop 2024-09-21T03:44:21.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.798+0000 7f50fa212640 1 -- 172.21.15.38:0/977260875 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f50f40ccc40 msgr2=0x7f50f415aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:21.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.798+0000 7f50fa212640 1 --2- 172.21.15.38:0/977260875 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f50f40ccc40 0x7f50f415aef0 secure :-1 s=READY pgs=2196 cs=0 l=1 rev1=1 crypto rx=0x7f50e4009780 tx=0x7f50e4033c20 comp rx=0 tx=0).stop 2024-09-21T03:44:21.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.798+0000 7f50fa212640 1 -- 172.21.15.38:0/977260875 shutdown_connections 2024-09-21T03:44:21.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.798+0000 7f50fa212640 1 --2- 172.21.15.38:0/977260875 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f50f41f5670 0x7f50f41f5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:21.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.798+0000 7f50fa212640 1 --2- 172.21.15.38:0/977260875 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f50d4078bc0 0x7f50d407b0d0 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:21.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.798+0000 7f50fa212640 1 --2- 172.21.15.38:0/977260875 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f50f40cd440 0x7f50f4154810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:21.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.798+0000 7f50fa212640 1 --2- 172.21.15.38:0/977260875 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f50f40ccc40 0x7f50f415aef0 unknown :-1 s=CLOSED pgs=2196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:21.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.798+0000 7f50fa212640 1 -- 172.21.15.38:0/977260875 >> 172.21.15.38:0/977260875 conn(0x7f50f40b46e0 msgr2=0x7f50f4070100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:21.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.798+0000 7f50fa212640 1 -- 172.21.15.38:0/977260875 shutdown_connections 2024-09-21T03:44:21.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.798+0000 7f50fa212640 1 -- 172.21.15.38:0/977260875 wait complete. 2024-09-21T03:44:21.816 DEBUG:teuthology.orchestra.run.smithi038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-09-21T03:44:21.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.876+0000 7f7c35e14f40 1 Processor -- start 2024-09-21T03:44:21.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.876+0000 7f7c35e14f40 1 -- start start 2024-09-21T03:44:21.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.877+0000 7f7c35e14f40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5601b6661d00 0x5601b6737c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:21.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.877+0000 7f7c35e14f40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5601b671c950 0x5601b668e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:21.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.877+0000 7f7c35e14f40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5601b6737670 0x5601b668eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:21.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.877+0000 7f7c35e14f40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5601b66aa460 con 0x5601b6737670 2024-09-21T03:44:21.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.877+0000 7f7c35e14f40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5601b65afd60 con 0x5601b671c950 2024-09-21T03:44:21.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.877+0000 7f7c35e14f40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x5601b66a97b0 con 0x5601b6661d00 2024-09-21T03:44:21.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.878+0000 7f7c2f461640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5601b6737670 0x5601b668eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:21.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.878+0000 7f7c2f461640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5601b6737670 0x5601b668eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38652/0 (socket says 172.21.15.38:38652) 2024-09-21T03:44:21.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.878+0000 7f7c2f461640 1 -- 172.21.15.38:0/2382296176 learned_addr learned my addr 172.21.15.38:0/2382296176 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:44:21.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.878+0000 7f7c2e45f640 1 --2- 172.21.15.38:0/2382296176 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5601b671c950 0x5601b668e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:21.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.878+0000 7f7c2ec60640 1 --2- 172.21.15.38:0/2382296176 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5601b6661d00 0x5601b6737c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:21.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.878+0000 7f7c2f461640 1 -- 172.21.15.38:0/2382296176 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5601b6661d00 msgr2=0x5601b6737c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:44:21.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.878+0000 7f7c2f461640 1 --2- 172.21.15.38:0/2382296176 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5601b6661d00 0x5601b6737c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:21.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.878+0000 7f7c2f461640 1 -- 172.21.15.38:0/2382296176 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5601b671c950 msgr2=0x5601b668e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:44:21.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.878+0000 7f7c2f461640 1 --2- 172.21.15.38:0/2382296176 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5601b671c950 0x5601b668e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:21.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.878+0000 7f7c2f461640 1 -- 172.21.15.38:0/2382296176 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5601b668f430 con 0x5601b6737670 2024-09-21T03:44:21.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.878+0000 7f7c2f461640 1 --2- 172.21.15.38:0/2382296176 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5601b6737670 0x5601b668eef0 secure :-1 s=READY pgs=2198 cs=0 l=1 rev1=1 crypto rx=0x7f7c28025e80 tx=0x7f7c28002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=327ce09a7f08b33f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:21.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.878+0000 7f7c2d45d640 1 -- 172.21.15.38:0/2382296176 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c2804d8e0 con 0x5601b6737670 2024-09-21T03:44:21.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.878+0000 7f7c2d45d640 1 -- 172.21.15.38:0/2382296176 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7c2804dd70 con 0x5601b6737670 2024-09-21T03:44:21.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.878+0000 7f7c2d45d640 1 -- 172.21.15.38:0/2382296176 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c280566d0 con 0x5601b6737670 2024-09-21T03:44:21.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.879+0000 7f7c35e14f40 1 -- 172.21.15.38:0/2382296176 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5601b6737670 msgr2=0x5601b668eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:21.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.879+0000 7f7c35e14f40 1 --2- 172.21.15.38:0/2382296176 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5601b6737670 0x5601b668eef0 secure :-1 s=READY pgs=2198 cs=0 l=1 rev1=1 crypto rx=0x7f7c28025e80 tx=0x7f7c28002fb0 comp rx=0 tx=0).stop 2024-09-21T03:44:21.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.879+0000 7f7c35e14f40 1 -- 172.21.15.38:0/2382296176 shutdown_connections 2024-09-21T03:44:21.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.879+0000 7f7c35e14f40 1 --2- 172.21.15.38:0/2382296176 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5601b6737670 0x5601b668eef0 unknown :-1 s=CLOSED pgs=2198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:21.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.879+0000 7f7c35e14f40 1 --2- 172.21.15.38:0/2382296176 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5601b671c950 0x5601b668e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:21.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.879+0000 7f7c35e14f40 1 --2- 172.21.15.38:0/2382296176 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5601b6661d00 0x5601b6737c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:21.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.879+0000 7f7c35e14f40 1 -- 172.21.15.38:0/2382296176 >> 172.21.15.38:0/2382296176 conn(0x5601b6580ac0 msgr2=0x5601b668b890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:21.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.879+0000 7f7c35e14f40 1 -- 172.21.15.38:0/2382296176 shutdown_connections 2024-09-21T03:44:21.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.879+0000 7f7c35e14f40 1 -- 172.21.15.38:0/2382296176 wait complete. 2024-09-21T03:44:21.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.880+0000 7f7c35e14f40 1 Processor -- start 2024-09-21T03:44:21.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.880+0000 7f7c35e14f40 1 -- start start 2024-09-21T03:44:21.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.881+0000 7f7c35e14f40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5601b6661d00 0x5601b67cddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:21.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.881+0000 7f7c35e14f40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5601b671c950 0x5601b66a3d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:21.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.881+0000 7f7c35e14f40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5601b6737670 0x5601b66a4260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:21.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.881+0000 7f7c35e14f40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5601b6716ae0 con 0x5601b6661d00 2024-09-21T03:44:21.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.881+0000 7f7c35e14f40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5601b6736b70 con 0x5601b671c950 2024-09-21T03:44:21.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.881+0000 7f7c35e14f40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x5601b67353a0 con 0x5601b6737670 2024-09-21T03:44:21.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.881+0000 7f7c2ec60640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5601b6661d00 0x5601b67cddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:21.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.881+0000 7f7c2ec60640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5601b6661d00 0x5601b67cddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38660/0 (socket says 172.21.15.38:38660) 2024-09-21T03:44:21.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.882+0000 7f7c2ec60640 1 -- 172.21.15.38:0/2403603163 learned_addr learned my addr 172.21.15.38:0/2403603163 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:44:21.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.882+0000 7f7c2e45f640 1 --2- 172.21.15.38:0/2403603163 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5601b671c950 0x5601b66a3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:21.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.882+0000 7f7c2f461640 1 --2- 172.21.15.38:0/2403603163 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5601b6737670 0x5601b66a4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:21.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.882+0000 7f7c2ec60640 1 -- 172.21.15.38:0/2403603163 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5601b6737670 msgr2=0x5601b66a4260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:21.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.882+0000 7f7c2ec60640 1 --2- 172.21.15.38:0/2403603163 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5601b6737670 0x5601b66a4260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:21.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.882+0000 7f7c2ec60640 1 -- 172.21.15.38:0/2403603163 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5601b671c950 msgr2=0x5601b66a3d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:21.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.882+0000 7f7c2ec60640 1 --2- 172.21.15.38:0/2403603163 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5601b671c950 0x5601b66a3d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:21.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.882+0000 7f7c2ec60640 1 -- 172.21.15.38:0/2403603163 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5601b66a4980 con 0x5601b6661d00 2024-09-21T03:44:21.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.882+0000 7f7c2ec60640 1 --2- 172.21.15.38:0/2403603163 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5601b6661d00 0x5601b67cddb0 secure :-1 s=READY pgs=2199 cs=0 l=1 rev1=1 crypto rx=0x7f7c2000d6c0 tx=0x7f7c2000db80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:21.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.882+0000 7f7c17fff640 1 -- 172.21.15.38:0/2403603163 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c2000ca60 con 0x5601b6661d00 2024-09-21T03:44:21.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.882+0000 7f7c17fff640 1 -- 172.21.15.38:0/2403603163 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7c20014070 con 0x5601b6661d00 2024-09-21T03:44:21.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.882+0000 7f7c35e14f40 1 -- 172.21.15.38:0/2403603163 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5601b6718390 con 0x5601b6661d00 2024-09-21T03:44:21.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.882+0000 7f7c17fff640 1 -- 172.21.15.38:0/2403603163 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c20019470 con 0x5601b6661d00 2024-09-21T03:44:21.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.882+0000 7f7c35e14f40 1 -- 172.21.15.38:0/2403603163 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5601b671c0d0 con 0x5601b6661d00 2024-09-21T03:44:21.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.884+0000 7f7c17fff640 1 -- 172.21.15.38:0/2403603163 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f7c20019610 con 0x5601b6661d00 2024-09-21T03:44:21.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.885+0000 7f7c17fff640 1 --2- 172.21.15.38:0/2403603163 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7c1c0775a0 0x7f7c1c079ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:21.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.885+0000 7f7c17fff640 1 -- 172.21.15.38:0/2403603163 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 9486+0+0 (secure 0 0 0) 0x7f7c2009b8b0 con 0x5601b6661d00 2024-09-21T03:44:21.888 INFO:teuthology.orchestra.run.smithi038.stdout:Importing pool 2024-09-21T03:44:21.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.885+0000 7f7c35e14f40 1 --2- 172.21.15.38:0/2403603163 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x5601b672e650 0x5601b67361f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:21.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.885+0000 7f7c35e14f40 1 -- 172.21.15.38:0/2403603163 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5601b67dabc0 con 0x5601b672e650 2024-09-21T03:44:21.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.885+0000 7f7c2e45f640 1 --2- 172.21.15.38:0/2403603163 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7c1c0775a0 0x7f7c1c079ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:21.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.885+0000 7f7c2f461640 1 --2- 172.21.15.38:0/2403603163 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x5601b672e650 0x5601b67361f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:21.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.886+0000 7f7c2e45f640 1 --2- 172.21.15.38:0/2403603163 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7c1c0775a0 0x7f7c1c079ab0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x5601b6717b50 tx=0x7f7c2400e040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:21.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.886+0000 7f7c2f461640 1 --2- 172.21.15.38:0/2403603163 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x5601b672e650 0x5601b67361f0 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:21.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.889+0000 7f7c2f461640 1 -- 172.21.15.38:0/2403603163 <== osd.7 v2:172.21.15.195:6818/1606106342 1 ==== osd_op_reply(1 601.00000000 [create] v133'1128 uv1102 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7c2805f030 con 0x5601b672e650 2024-09-21T03:44:21.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.889+0000 7f7c35e14f40 1 -- 172.21.15.38:0/2403603163 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e133) -- 0x5601b67daf60 con 0x5601b672e650 2024-09-21T03:44:21.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.890+0000 7f7c2f461640 1 -- 172.21.15.38:0/2403603163 <== osd.7 v2:172.21.15.195:6818/1606106342 2 ==== osd_op_reply(2 601.00000000 [delete] v133'1129 uv1129 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7c2805f030 con 0x5601b672e650 2024-09-21T03:44:21.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.890+0000 7f7c35e14f40 1 -- 172.21.15.38:0/2403603163 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5601b67db300 con 0x5601b672e650 2024-09-21T03:44:21.894 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-09-21T03:44:21.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.892+0000 7f7c2f461640 1 -- 172.21.15.38:0/2403603163 <== osd.7 v2:172.21.15.195:6818/1606106342 3 ==== osd_op_reply(3 601.00000000 [create] v133'1130 uv1130 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7c2805f030 con 0x5601b672e650 2024-09-21T03:44:21.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.892+0000 7f7c35e14f40 1 -- 172.21.15.38:0/2403603163 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5601b67dc6b0 con 0x5601b672e650 2024-09-21T03:44:21.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.894+0000 7f7c2f461640 1 -- 172.21.15.38:0/2403603163 <== osd.7 v2:172.21.15.195:6818/1606106342 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v133'1131 uv1131 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7c2805f030 con 0x5601b672e650 2024-09-21T03:44:21.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.894+0000 7f7c35e14f40 1 -- 172.21.15.38:0/2403603163 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5601b67dca50 con 0x5601b672e650 2024-09-21T03:44:21.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.895+0000 7f7c2f461640 1 -- 172.21.15.38:0/2403603163 <== osd.7 v2:172.21.15.195:6818/1606106342 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v133'1132 uv1132 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7c2805f030 con 0x5601b672e650 2024-09-21T03:44:21.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.896+0000 7f7c35e14f40 1 -- 172.21.15.38:0/2403603163 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5601b67db6a0 con 0x5601b672e650 2024-09-21T03:44:21.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.897+0000 7f7c2f461640 1 -- 172.21.15.38:0/2403603163 <== osd.7 v2:172.21.15.195:6818/1606106342 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v133'1133 uv1133 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7c2805f030 con 0x5601b672e650 2024-09-21T03:44:21.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.898+0000 7f7c35e14f40 1 -- 172.21.15.38:0/2403603163 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5601b67dba40 con 0x5601b672e650 2024-09-21T03:44:21.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.899+0000 7f7c2f461640 1 -- 172.21.15.38:0/2403603163 <== osd.7 v2:172.21.15.195:6818/1606106342 7 ==== osd_op_reply(7 602.00000000 [create] v133'1134 uv1108 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7c2805f030 con 0x5601b672e650 2024-09-21T03:44:21.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.899+0000 7f7c35e14f40 1 -- 172.21.15.38:0/2403603163 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e133) -- 0x5601b67dba40 con 0x5601b672e650 2024-09-21T03:44:21.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.901+0000 7f7c2f461640 1 -- 172.21.15.38:0/2403603163 <== osd.7 v2:172.21.15.195:6818/1606106342 8 ==== osd_op_reply(8 602.00000000 [delete] v133'1135 uv1135 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7c2805f030 con 0x5601b672e650 2024-09-21T03:44:21.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.901+0000 7f7c35e14f40 1 -- 172.21.15.38:0/2403603163 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5601b67dba40 con 0x5601b672e650 2024-09-21T03:44:21.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.903+0000 7f7c2f461640 1 -- 172.21.15.38:0/2403603163 <== osd.7 v2:172.21.15.195:6818/1606106342 9 ==== osd_op_reply(9 602.00000000 [create] v133'1136 uv1136 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7c2805f030 con 0x5601b672e650 2024-09-21T03:44:21.905 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-09-21T03:44:21.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.903+0000 7f7c35e14f40 1 -- 172.21.15.38:0/2403603163 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5601b67dba40 con 0x5601b672e650 2024-09-21T03:44:21.906 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.904+0000 7f7c2f461640 1 -- 172.21.15.38:0/2403603163 <== osd.7 v2:172.21.15.195:6818/1606106342 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v133'1137 uv1137 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7c2805f030 con 0x5601b672e650 2024-09-21T03:44:21.906 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.904+0000 7f7c35e14f40 1 -- 172.21.15.38:0/2403603163 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5601b67dba40 con 0x5601b672e650 2024-09-21T03:44:21.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.906+0000 7f7c2f461640 1 -- 172.21.15.38:0/2403603163 <== osd.7 v2:172.21.15.195:6818/1606106342 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v133'1138 uv1138 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7c2805f030 con 0x5601b672e650 2024-09-21T03:44:21.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.906+0000 7f7c35e14f40 1 -- 172.21.15.38:0/2403603163 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5601b67dba40 con 0x5601b672e650 2024-09-21T03:44:21.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.908+0000 7f7c2f461640 1 -- 172.21.15.38:0/2403603163 <== osd.7 v2:172.21.15.195:6818/1606106342 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v133'1139 uv1139 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7c2805f030 con 0x5601b672e650 2024-09-21T03:44:21.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.908+0000 7f7c35e14f40 1 -- 172.21.15.38:0/2403603163 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:13 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5601b67dba40 con 0x5601b672e650 2024-09-21T03:44:21.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.909+0000 7f7c2f461640 1 -- 172.21.15.38:0/2403603163 <== osd.7 v2:172.21.15.195:6818/1606106342 13 ==== osd_op_reply(13 10000000000.00000000 [create] v133'1140 uv1115 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7f7c2805f030 con 0x5601b672e650 2024-09-21T03:44:21.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.909+0000 7f7c35e14f40 1 -- 172.21.15.38:0/2403603163 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:14 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e133) -- 0x5601b67dba40 con 0x5601b672e650 2024-09-21T03:44:21.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.912+0000 7f7c2f461640 1 -- 172.21.15.38:0/2403603163 <== osd.7 v2:172.21.15.195:6818/1606106342 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v133'1141 uv1141 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f7c2805f030 con 0x5601b672e650 2024-09-21T03:44:21.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.912+0000 7f7c35e14f40 1 -- 172.21.15.38:0/2403603163 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:15 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5601b67dba40 con 0x5601b672e650 2024-09-21T03:44:21.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.913+0000 7f7c2f461640 1 -- 172.21.15.38:0/2403603163 <== osd.7 v2:172.21.15.195:6818/1606106342 15 ==== osd_op_reply(15 10000000000.00000000 [create] v133'1142 uv1142 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f7c2805f030 con 0x5601b672e650 2024-09-21T03:44:21.915 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-09-21T03:44:21.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.913+0000 7f7c35e14f40 1 -- 172.21.15.38:0/2403603163 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:16 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5601b67dba40 con 0x5601b672e650 2024-09-21T03:44:21.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.915+0000 7f7c2f461640 1 -- 172.21.15.38:0/2403603163 <== osd.7 v2:172.21.15.195:6818/1606106342 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v133'1143 uv1143 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f7c2805f030 con 0x5601b672e650 2024-09-21T03:44:21.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.915+0000 7f7c35e14f40 1 -- 172.21.15.38:0/2403603163 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:17 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5601b67dba40 con 0x5601b672e650 2024-09-21T03:44:21.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.916+0000 7f7c2f461640 1 -- 172.21.15.38:0/2403603163 <== osd.7 v2:172.21.15.195:6818/1606106342 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v133'1144 uv1144 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f7c2805f030 con 0x5601b672e650 2024-09-21T03:44:21.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.916+0000 7f7c35e14f40 1 -- 172.21.15.38:0/2403603163 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:18 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5601b67dba40 con 0x5601b672e650 2024-09-21T03:44:21.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.918+0000 7f7c2f461640 1 -- 172.21.15.38:0/2403603163 <== osd.7 v2:172.21.15.195:6818/1606106342 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v133'1145 uv1145 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f7c2805f030 con 0x5601b672e650 2024-09-21T03:44:21.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.918+0000 7f7c35e14f40 1 -- 172.21.15.38:0/2403603163 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:19 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=585b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5601b67dbe90 con 0x5601b672e650 2024-09-21T03:44:21.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.919+0000 7f7c2f461640 1 -- 172.21.15.38:0/2403603163 <== osd.7 v2:172.21.15.195:6818/1606106342 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v133'1146 uv1146 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f7c2805f030 con 0x5601b672e650 2024-09-21T03:44:21.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.919+0000 7f7c35e14f40 1 -- 172.21.15.38:0/2403603163 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:20 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5601b67dbe20 con 0x5601b672e650 2024-09-21T03:44:21.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.922+0000 7f7c2f461640 1 -- 172.21.15.38:0/2403603163 <== osd.7 v2:172.21.15.195:6818/1606106342 20 ==== osd_op_reply(20 600.00000000 [create] v133'1147 uv1121 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7c2805f030 con 0x5601b672e650 2024-09-21T03:44:21.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.922+0000 7f7c35e14f40 1 -- 172.21.15.38:0/2403603163 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:21 13.10 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e133) -- 0x5601b67dbcf0 con 0x5601b672e650 2024-09-21T03:44:21.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.923+0000 7f7c2f461640 1 -- 172.21.15.38:0/2403603163 <== osd.7 v2:172.21.15.195:6818/1606106342 21 ==== osd_op_reply(21 600.00000000 [delete] v133'1148 uv1148 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7c2805f030 con 0x5601b672e650 2024-09-21T03:44:21.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.923+0000 7f7c35e14f40 1 -- 172.21.15.38:0/2403603163 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:22 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5601b67dbcf0 con 0x5601b672e650 2024-09-21T03:44:21.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.924+0000 7f7c2f461640 1 -- 172.21.15.38:0/2403603163 <== osd.7 v2:172.21.15.195:6818/1606106342 22 ==== osd_op_reply(22 600.00000000 [create] v133'1149 uv1149 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7c2805f030 con 0x5601b672e650 2024-09-21T03:44:21.926 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-09-21T03:44:21.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.924+0000 7f7c35e14f40 1 -- 172.21.15.38:0/2403603163 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:23 13.10 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5601b67dbcf0 con 0x5601b672e650 2024-09-21T03:44:21.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.926+0000 7f7c2f461640 1 -- 172.21.15.38:0/2403603163 <== osd.7 v2:172.21.15.195:6818/1606106342 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v133'1150 uv1150 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7c2805f030 con 0x5601b672e650 2024-09-21T03:44:21.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.926+0000 7f7c35e14f40 1 -- 172.21.15.38:0/2403603163 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:24 13.10 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5601b67dbcf0 con 0x5601b672e650 2024-09-21T03:44:21.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.927+0000 7f7c2f461640 1 -- 172.21.15.38:0/2403603163 <== osd.7 v2:172.21.15.195:6818/1606106342 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v133'1151 uv1151 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7c2805f030 con 0x5601b672e650 2024-09-21T03:44:21.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.927+0000 7f7c35e14f40 1 -- 172.21.15.38:0/2403603163 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:25 13.10 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5601b67dbcf0 con 0x5601b672e650 2024-09-21T03:44:21.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.929+0000 7f7c2f461640 1 -- 172.21.15.38:0/2403603163 <== osd.7 v2:172.21.15.195:6818/1606106342 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v133'1152 uv1152 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7c2805f030 con 0x5601b672e650 2024-09-21T03:44:21.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.929+0000 7f7c35e14f40 1 -- 172.21.15.38:0/2403603163 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:26 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5601b67dbcf0 con 0x5601b672e650 2024-09-21T03:44:21.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.931+0000 7f7c2f461640 1 -- 172.21.15.38:0/2403603163 <== osd.7 v2:172.21.15.195:6818/1606106342 26 ==== osd_op_reply(26 603.00000000 [create] v133'1153 uv1127 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7c2805f030 con 0x5601b672e650 2024-09-21T03:44:21.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.931+0000 7f7c35e14f40 1 -- 172.21.15.38:0/2403603163 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:27 13.10 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e133) -- 0x5601b67dbcf0 con 0x5601b672e650 2024-09-21T03:44:21.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.932+0000 7f7c2f461640 1 -- 172.21.15.38:0/2403603163 <== osd.7 v2:172.21.15.195:6818/1606106342 27 ==== osd_op_reply(27 603.00000000 [delete] v133'1154 uv1154 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7c2805f030 con 0x5601b672e650 2024-09-21T03:44:21.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.932+0000 7f7c35e14f40 1 -- 172.21.15.38:0/2403603163 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:28 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5601b67dbcf0 con 0x5601b672e650 2024-09-21T03:44:21.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.934+0000 7f7c2f461640 1 -- 172.21.15.38:0/2403603163 <== osd.7 v2:172.21.15.195:6818/1606106342 28 ==== osd_op_reply(28 603.00000000 [create] v133'1155 uv1155 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7c2805f030 con 0x5601b672e650 2024-09-21T03:44:21.936 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-09-21T03:44:21.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.934+0000 7f7c35e14f40 1 -- 172.21.15.38:0/2403603163 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:29 13.10 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5601b67dbcf0 con 0x5601b672e650 2024-09-21T03:44:21.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.936+0000 7f7c2f461640 1 -- 172.21.15.38:0/2403603163 <== osd.7 v2:172.21.15.195:6818/1606106342 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v133'1156 uv1156 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7c2805f030 con 0x5601b672e650 2024-09-21T03:44:21.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.936+0000 7f7c35e14f40 1 -- 172.21.15.38:0/2403603163 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:30 13.10 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5601b67dbcf0 con 0x5601b672e650 2024-09-21T03:44:21.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.937+0000 7f7c2f461640 1 -- 172.21.15.38:0/2403603163 <== osd.7 v2:172.21.15.195:6818/1606106342 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v133'1157 uv1157 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7c2805f030 con 0x5601b672e650 2024-09-21T03:44:21.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.937+0000 7f7c35e14f40 1 -- 172.21.15.38:0/2403603163 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:31 13.10 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5601b67dbcf0 con 0x5601b672e650 2024-09-21T03:44:21.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.939+0000 7f7c2f461640 1 -- 172.21.15.38:0/2403603163 <== osd.7 v2:172.21.15.195:6818/1606106342 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v133'1158 uv1158 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7c2805f030 con 0x5601b672e650 2024-09-21T03:44:21.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.939+0000 7f7c35e14f40 1 --2- 172.21.15.38:0/2403603163 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x5601b67dcdf0 0x5601b67df210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:21.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.939+0000 7f7c35e14f40 1 -- 172.21.15.38:0/2403603163 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5601b67dbcf0 con 0x5601b67dcdf0 2024-09-21T03:44:21.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.939+0000 7f7c2ec60640 1 --2- 172.21.15.38:0/2403603163 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x5601b67dcdf0 0x5601b67df210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:21.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.940+0000 7f7c2ec60640 1 --2- 172.21.15.38:0/2403603163 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x5601b67dcdf0 0x5601b67df210 crc :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:21.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.942+0000 7f7c2ec60640 1 -- 172.21.15.38:0/2403603163 <== osd.6 v2:172.21.15.195:6826/4055825675 1 ==== osd_op_reply(32 1.00000000.inode [create] v133'823 uv802 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f7c2000b040 con 0x5601b67dcdf0 2024-09-21T03:44:21.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.942+0000 7f7c35e14f40 1 -- 172.21.15.38:0/2403603163 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e133) -- 0x5601b67df750 con 0x5601b67dcdf0 2024-09-21T03:44:21.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.944+0000 7f7c2ec60640 1 -- 172.21.15.38:0/2403603163 <== osd.6 v2:172.21.15.195:6826/4055825675 2 ==== osd_op_reply(33 1.00000000.inode [delete] v133'824 uv824 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f7c2000b040 con 0x5601b67dcdf0 2024-09-21T03:44:21.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.944+0000 7f7c35e14f40 1 -- 172.21.15.38:0/2403603163 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5601b67dfaf0 con 0x5601b67dcdf0 2024-09-21T03:44:21.947 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-09-21T03:44:21.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.945+0000 7f7c2ec60640 1 -- 172.21.15.38:0/2403603163 <== osd.6 v2:172.21.15.195:6826/4055825675 3 ==== osd_op_reply(34 1.00000000.inode [create] v133'825 uv825 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f7c2000b040 con 0x5601b67dcdf0 2024-09-21T03:44:21.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.945+0000 7f7c35e14f40 1 -- 172.21.15.38:0/2403603163 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5601b67e0340 con 0x5601b67dcdf0 2024-09-21T03:44:21.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.946+0000 7f7c2ec60640 1 -- 172.21.15.38:0/2403603163 <== osd.6 v2:172.21.15.195:6826/4055825675 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v133'826 uv826 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f7c2000b040 con 0x5601b67dcdf0 2024-09-21T03:44:21.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.946+0000 7f7c35e14f40 1 -- 172.21.15.38:0/2403603163 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5601b67e06e0 con 0x5601b67dcdf0 2024-09-21T03:44:21.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.948+0000 7f7c2ec60640 1 -- 172.21.15.38:0/2403603163 <== osd.6 v2:172.21.15.195:6826/4055825675 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v133'827 uv827 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f7c2000b040 con 0x5601b67dcdf0 2024-09-21T03:44:21.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.948+0000 7f7c35e14f40 1 -- 172.21.15.38:0/2403603163 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5601b67e0a80 con 0x5601b67dcdf0 2024-09-21T03:44:21.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.950+0000 7f7c2ec60640 1 -- 172.21.15.38:0/2403603163 <== osd.6 v2:172.21.15.195:6826/4055825675 6 ==== osd_op_reply(37 200.00000000 [create] v133'828 uv821 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7c2000b040 con 0x5601b67dcdf0 2024-09-21T03:44:21.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.950+0000 7f7c35e14f40 1 -- 172.21.15.38:0/2403603163 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e133) -- 0x5601b67e0e20 con 0x5601b67dcdf0 2024-09-21T03:44:21.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.951+0000 7f7c2ec60640 1 -- 172.21.15.38:0/2403603163 <== osd.6 v2:172.21.15.195:6826/4055825675 7 ==== osd_op_reply(38 200.00000000 [delete] v133'829 uv829 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7c2000b040 con 0x5601b67dcdf0 2024-09-21T03:44:21.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.951+0000 7f7c35e14f40 1 -- 172.21.15.38:0/2403603163 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5601b67e0e20 con 0x5601b67dcdf0 2024-09-21T03:44:21.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.953+0000 7f7c2ec60640 1 -- 172.21.15.38:0/2403603163 <== osd.6 v2:172.21.15.195:6826/4055825675 8 ==== osd_op_reply(39 200.00000000 [create] v133'830 uv830 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7c2000b040 con 0x5601b67dcdf0 2024-09-21T03:44:21.955 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-09-21T03:44:21.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.953+0000 7f7c35e14f40 1 -- 172.21.15.38:0/2403603163 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5601b67e0e20 con 0x5601b67dcdf0 2024-09-21T03:44:21.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.954+0000 7f7c2ec60640 1 -- 172.21.15.38:0/2403603163 <== osd.6 v2:172.21.15.195:6826/4055825675 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v133'831 uv831 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7c2000b040 con 0x5601b67dcdf0 2024-09-21T03:44:21.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.954+0000 7f7c35e14f40 1 -- 172.21.15.38:0/2403603163 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5601b67e0e20 con 0x5601b67dcdf0 2024-09-21T03:44:21.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.956+0000 7f7c2ec60640 1 -- 172.21.15.38:0/2403603163 <== osd.6 v2:172.21.15.195:6826/4055825675 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v133'832 uv832 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7c2000b040 con 0x5601b67dcdf0 2024-09-21T03:44:21.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.956+0000 7f7c35e14f40 1 -- 172.21.15.38:0/2403603163 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:42 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5601b67e0e20 con 0x5601b67dcdf0 2024-09-21T03:44:21.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.957+0000 7f7c2ec60640 1 -- 172.21.15.38:0/2403603163 <== osd.6 v2:172.21.15.195:6826/4055825675 11 ==== osd_op_reply(42 200.00000001 [create] v133'833 uv822 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7c2000b040 con 0x5601b67dcdf0 2024-09-21T03:44:21.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.957+0000 7f7c35e14f40 1 -- 172.21.15.38:0/2403603163 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:43 13.14 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e133) -- 0x5601b67e0e20 con 0x5601b67dcdf0 2024-09-21T03:44:21.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.959+0000 7f7c2ec60640 1 -- 172.21.15.38:0/2403603163 <== osd.6 v2:172.21.15.195:6826/4055825675 12 ==== osd_op_reply(43 200.00000001 [delete] v133'834 uv834 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7c2000b040 con 0x5601b67dcdf0 2024-09-21T03:44:21.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.959+0000 7f7c35e14f40 1 -- 172.21.15.38:0/2403603163 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:44 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5601b67e0e20 con 0x5601b67dcdf0 2024-09-21T03:44:21.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.960+0000 7f7c2ec60640 1 -- 172.21.15.38:0/2403603163 <== osd.6 v2:172.21.15.195:6826/4055825675 13 ==== osd_op_reply(44 200.00000001 [create] v133'835 uv835 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7c2000b040 con 0x5601b67dcdf0 2024-09-21T03:44:21.962 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-09-21T03:44:21.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.960+0000 7f7c35e14f40 1 -- 172.21.15.38:0/2403603163 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:45 13.14 13:2e2fa760:::200.00000001:head [write 0~17490 in=17490b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5601b67e52a0 con 0x5601b67dcdf0 2024-09-21T03:44:21.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.962+0000 7f7c2ec60640 1 -- 172.21.15.38:0/2403603163 <== osd.6 v2:172.21.15.195:6826/4055825675 14 ==== osd_op_reply(45 200.00000001 [write 0~17490] v133'836 uv836 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7c2000b040 con 0x5601b67dcdf0 2024-09-21T03:44:21.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.962+0000 7f7c35e14f40 1 -- 172.21.15.38:0/2403603163 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:46 13.14 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5601b67e52a0 con 0x5601b67dcdf0 2024-09-21T03:44:21.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.963+0000 7f7c2ec60640 1 -- 172.21.15.38:0/2403603163 <== osd.6 v2:172.21.15.195:6826/4055825675 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v133'837 uv837 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7c2000b040 con 0x5601b67dcdf0 2024-09-21T03:44:21.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.963+0000 7f7c35e14f40 1 -- 172.21.15.38:0/2403603163 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:47 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5601b67e0e20 con 0x5601b67dcdf0 2024-09-21T03:44:21.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.965+0000 7f7c2ec60640 1 -- 172.21.15.38:0/2403603163 <== osd.6 v2:172.21.15.195:6826/4055825675 16 ==== osd_op_reply(47 606.00000000 [create] v133'838 uv818 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7c2000b040 con 0x5601b67dcdf0 2024-09-21T03:44:21.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.965+0000 7f7c35e14f40 1 -- 172.21.15.38:0/2403603163 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:48 13.14 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e133) -- 0x5601b67e0e20 con 0x5601b67dcdf0 2024-09-21T03:44:21.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.966+0000 7f7c2ec60640 1 -- 172.21.15.38:0/2403603163 <== osd.6 v2:172.21.15.195:6826/4055825675 17 ==== osd_op_reply(48 606.00000000 [delete] v133'839 uv839 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7c2000b040 con 0x5601b67dcdf0 2024-09-21T03:44:21.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.966+0000 7f7c35e14f40 1 -- 172.21.15.38:0/2403603163 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:49 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5601b67e0e20 con 0x5601b67dcdf0 2024-09-21T03:44:21.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.967+0000 7f7c2ec60640 1 -- 172.21.15.38:0/2403603163 <== osd.6 v2:172.21.15.195:6826/4055825675 18 ==== osd_op_reply(49 606.00000000 [create] v133'840 uv840 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7c2000b040 con 0x5601b67dcdf0 2024-09-21T03:44:21.970 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-09-21T03:44:21.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.967+0000 7f7c35e14f40 1 -- 172.21.15.38:0/2403603163 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:50 13.14 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5601b67e1e30 con 0x5601b67dcdf0 2024-09-21T03:44:21.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.969+0000 7f7c2ec60640 1 -- 172.21.15.38:0/2403603163 <== osd.6 v2:172.21.15.195:6826/4055825675 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v133'841 uv841 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7c2000b040 con 0x5601b67dcdf0 2024-09-21T03:44:21.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.969+0000 7f7c35e14f40 1 -- 172.21.15.38:0/2403603163 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:51 13.14 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5601b67e1e30 con 0x5601b67dcdf0 2024-09-21T03:44:21.972 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.970+0000 7f7c2ec60640 1 -- 172.21.15.38:0/2403603163 <== osd.6 v2:172.21.15.195:6826/4055825675 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v133'842 uv842 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7c2000b040 con 0x5601b67dcdf0 2024-09-21T03:44:21.972 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.970+0000 7f7c35e14f40 1 -- 172.21.15.38:0/2403603163 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:52 13.14 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5601b67e0e20 con 0x5601b67dcdf0 2024-09-21T03:44:21.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.972+0000 7f7c2ec60640 1 -- 172.21.15.38:0/2403603163 <== osd.6 v2:172.21.15.195:6826/4055825675 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v133'843 uv843 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7c2000b040 con 0x5601b67dcdf0 2024-09-21T03:44:21.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.972+0000 7f7c35e14f40 1 --2- 172.21.15.38:0/2403603163 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x5601b67e1330 0x5601b67e3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:21.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.972+0000 7f7c35e14f40 1 -- 172.21.15.38:0/2403603163 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:53 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5601b67e3ce0 con 0x5601b67e1330 2024-09-21T03:44:21.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.972+0000 7f7c2e45f640 1 --2- 172.21.15.38:0/2403603163 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x5601b67e1330 0x5601b67e3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:21.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.972+0000 7f7c2e45f640 1 --2- 172.21.15.38:0/2403603163 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x5601b67e1330 0x5601b67e3750 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:21.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.974+0000 7f7c2e45f640 1 -- 172.21.15.38:0/2403603163 <== osd.0 v2:172.21.15.38:6800/2381639433 1 ==== osd_op_reply(53 607.00000000 [create] v133'219 uv218 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7c2400a4e0 con 0x5601b67e1330 2024-09-21T03:44:21.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.974+0000 7f7c35e14f40 1 -- 172.21.15.38:0/2403603163 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:54 13.1c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e133) -- 0x5601b67e4160 con 0x5601b67e1330 2024-09-21T03:44:21.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.975+0000 7f7c2e45f640 1 -- 172.21.15.38:0/2403603163 <== osd.0 v2:172.21.15.38:6800/2381639433 2 ==== osd_op_reply(54 607.00000000 [delete] v133'220 uv220 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7c2400a4e0 con 0x5601b67e1330 2024-09-21T03:44:21.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.975+0000 7f7c35e14f40 1 -- 172.21.15.38:0/2403603163 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:55 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5601b67e4640 con 0x5601b67e1330 2024-09-21T03:44:21.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.976+0000 7f7c2e45f640 1 -- 172.21.15.38:0/2403603163 <== osd.0 v2:172.21.15.38:6800/2381639433 3 ==== osd_op_reply(55 607.00000000 [create] v133'221 uv221 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7c2400a4e0 con 0x5601b67e1330 2024-09-21T03:44:21.978 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-09-21T03:44:21.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.976+0000 7f7c35e14f40 1 -- 172.21.15.38:0/2403603163 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:56 13.1c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5601b67e5ef0 con 0x5601b67e1330 2024-09-21T03:44:21.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.977+0000 7f7c2e45f640 1 -- 172.21.15.38:0/2403603163 <== osd.0 v2:172.21.15.38:6800/2381639433 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v133'222 uv222 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7c2400a4e0 con 0x5601b67e1330 2024-09-21T03:44:21.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.977+0000 7f7c35e14f40 1 -- 172.21.15.38:0/2403603163 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:57 13.1c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5601b67e6290 con 0x5601b67e1330 2024-09-21T03:44:21.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.978+0000 7f7c2e45f640 1 -- 172.21.15.38:0/2403603163 <== osd.0 v2:172.21.15.38:6800/2381639433 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v133'223 uv223 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7c2400a4e0 con 0x5601b67e1330 2024-09-21T03:44:21.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.978+0000 7f7c35e14f40 1 -- 172.21.15.38:0/2403603163 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:58 13.1c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5601b67e4ee0 con 0x5601b67e1330 2024-09-21T03:44:21.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.979+0000 7f7c2e45f640 1 -- 172.21.15.38:0/2403603163 <== osd.0 v2:172.21.15.38:6800/2381639433 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v133'224 uv224 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7c2400a4e0 con 0x5601b67e1330 2024-09-21T03:44:21.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.979+0000 7f7c35e14f40 1 --2- 172.21.15.38:0/2403603163 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x5601b67e57e0 0x5601b67e8640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:21.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.979+0000 7f7c35e14f40 1 -- 172.21.15.38:0/2403603163 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:59 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5601b67e8b80 con 0x5601b67e57e0 2024-09-21T03:44:21.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.979+0000 7f7c2f461640 1 --2- 172.21.15.38:0/2403603163 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x5601b67e57e0 0x5601b67e8640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:21.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.979+0000 7f7c2f461640 1 --2- 172.21.15.38:0/2403603163 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x5601b67e57e0 0x5601b67e8640 crc :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:21.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.981+0000 7f7c2f461640 1 -- 172.21.15.38:0/2403603163 <== osd.3 v2:172.21.15.38:6808/1286810294 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v133'164 uv163 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f7c2805f030 con 0x5601b67e57e0 2024-09-21T03:44:21.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.981+0000 7f7c35e14f40 1 -- 172.21.15.38:0/2403603163 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:60 13.6 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e133) -- 0x5601b67e8b80 con 0x5601b67e57e0 2024-09-21T03:44:21.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.982+0000 7f7c2f461640 1 -- 172.21.15.38:0/2403603163 <== osd.3 v2:172.21.15.38:6808/1286810294 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v133'165 uv165 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f7c2805f030 con 0x5601b67e57e0 2024-09-21T03:44:21.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.982+0000 7f7c35e14f40 1 -- 172.21.15.38:0/2403603163 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:61 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5601b67e8c00 con 0x5601b67e57e0 2024-09-21T03:44:21.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.983+0000 7f7c2f461640 1 -- 172.21.15.38:0/2403603163 <== osd.3 v2:172.21.15.38:6808/1286810294 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v133'166 uv166 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f7c2805f030 con 0x5601b67e57e0 2024-09-21T03:44:21.985 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-09-21T03:44:21.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.983+0000 7f7c35e14f40 1 -- 172.21.15.38:0/2403603163 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:62 13.6 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5601b67e9040 con 0x5601b67e57e0 2024-09-21T03:44:21.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.985+0000 7f7c2f461640 1 -- 172.21.15.38:0/2403603163 <== osd.3 v2:172.21.15.38:6808/1286810294 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v133'167 uv167 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f7c2805f030 con 0x5601b67e57e0 2024-09-21T03:44:21.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.985+0000 7f7c35e14f40 1 -- 172.21.15.38:0/2403603163 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5601b67e9040 con 0x5601b67e1330 2024-09-21T03:44:21.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.986+0000 7f7c2e45f640 1 -- 172.21.15.38:0/2403603163 <== osd.0 v2:172.21.15.38:6800/2381639433 7 ==== osd_op_reply(63 608.00000000 [create] v133'219 uv218 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7c2400a4e0 con 0x5601b67e1330 2024-09-21T03:44:21.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.986+0000 7f7c35e14f40 1 -- 172.21.15.38:0/2403603163 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e133) -- 0x5601b67e93e0 con 0x5601b67e1330 2024-09-21T03:44:21.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.987+0000 7f7c2e45f640 1 -- 172.21.15.38:0/2403603163 <== osd.0 v2:172.21.15.38:6800/2381639433 8 ==== osd_op_reply(64 608.00000000 [delete] v133'220 uv220 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7c2400a4e0 con 0x5601b67e1330 2024-09-21T03:44:21.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.987+0000 7f7c35e14f40 1 -- 172.21.15.38:0/2403603163 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5601b67e93e0 con 0x5601b67e1330 2024-09-21T03:44:21.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.988+0000 7f7c2e45f640 1 -- 172.21.15.38:0/2403603163 <== osd.0 v2:172.21.15.38:6800/2381639433 9 ==== osd_op_reply(65 608.00000000 [create] v133'221 uv221 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7c2400a4e0 con 0x5601b67e1330 2024-09-21T03:44:21.990 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-09-21T03:44:21.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.988+0000 7f7c35e14f40 1 -- 172.21.15.38:0/2403603163 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5601b67ea3f0 con 0x5601b67e1330 2024-09-21T03:44:21.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.989+0000 7f7c2e45f640 1 -- 172.21.15.38:0/2403603163 <== osd.0 v2:172.21.15.38:6800/2381639433 10 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v133'222 uv222 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7c2400a4e0 con 0x5601b67e1330 2024-09-21T03:44:21.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.989+0000 7f7c35e14f40 1 -- 172.21.15.38:0/2403603163 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5601b67ea3f0 con 0x5601b67e1330 2024-09-21T03:44:21.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.990+0000 7f7c2e45f640 1 -- 172.21.15.38:0/2403603163 <== osd.0 v2:172.21.15.38:6800/2381639433 11 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v133'223 uv223 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7c2400a4e0 con 0x5601b67e1330 2024-09-21T03:44:21.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.990+0000 7f7c35e14f40 1 -- 172.21.15.38:0/2403603163 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5601b67e93e0 con 0x5601b67e1330 2024-09-21T03:44:21.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.992+0000 7f7c2e45f640 1 -- 172.21.15.38:0/2403603163 <== osd.0 v2:172.21.15.38:6800/2381639433 12 ==== osd_op_reply(68 608.00000000 [omap-set-header] v133'224 uv224 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7c2400a4e0 con 0x5601b67e1330 2024-09-21T03:44:21.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.992+0000 7f7c35e14f40 1 -- 172.21.15.38:0/2403603163 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5601b67e93e0 con 0x5601b67e57e0 2024-09-21T03:44:21.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.993+0000 7f7c2f461640 1 -- 172.21.15.38:0/2403603163 <== osd.3 v2:172.21.15.38:6808/1286810294 5 ==== osd_op_reply(69 604.00000000 [create] v133'404 uv398 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7c2805f030 con 0x5601b67e57e0 2024-09-21T03:44:21.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.993+0000 7f7c35e14f40 1 -- 172.21.15.38:0/2403603163 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e133) -- 0x5601b67e93e0 con 0x5601b67e57e0 2024-09-21T03:44:21.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.994+0000 7f7c2f461640 1 -- 172.21.15.38:0/2403603163 <== osd.3 v2:172.21.15.38:6808/1286810294 6 ==== osd_op_reply(70 604.00000000 [delete] v133'405 uv405 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7c2805f030 con 0x5601b67e57e0 2024-09-21T03:44:21.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.994+0000 7f7c35e14f40 1 -- 172.21.15.38:0/2403603163 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5601b67e93e0 con 0x5601b67e57e0 2024-09-21T03:44:21.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.995+0000 7f7c2f461640 1 -- 172.21.15.38:0/2403603163 <== osd.3 v2:172.21.15.38:6808/1286810294 7 ==== osd_op_reply(71 604.00000000 [create] v133'406 uv406 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7c2805f030 con 0x5601b67e57e0 2024-09-21T03:44:21.997 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-09-21T03:44:21.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.996+0000 7f7c35e14f40 1 -- 172.21.15.38:0/2403603163 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5601b67ea3f0 con 0x5601b67e57e0 2024-09-21T03:44:21.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.997+0000 7f7c2f461640 1 -- 172.21.15.38:0/2403603163 <== osd.3 v2:172.21.15.38:6808/1286810294 8 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v133'407 uv407 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7c2805f030 con 0x5601b67e57e0 2024-09-21T03:44:21.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.997+0000 7f7c35e14f40 1 -- 172.21.15.38:0/2403603163 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5601b67ea3f0 con 0x5601b67e57e0 2024-09-21T03:44:21.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.998+0000 7f7c2f461640 1 -- 172.21.15.38:0/2403603163 <== osd.3 v2:172.21.15.38:6808/1286810294 9 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v133'408 uv408 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7c2805f030 con 0x5601b67e57e0 2024-09-21T03:44:22.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.998+0000 7f7c35e14f40 1 -- 172.21.15.38:0/2403603163 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5601b67e93e0 con 0x5601b67e57e0 2024-09-21T03:44:22.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.999+0000 7f7c2f461640 1 -- 172.21.15.38:0/2403603163 <== osd.3 v2:172.21.15.38:6808/1286810294 10 ==== osd_op_reply(74 604.00000000 [omap-set-header] v133'409 uv409 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7c2805f030 con 0x5601b67e57e0 2024-09-21T03:44:22.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:21.999+0000 7f7c35e14f40 1 -- 172.21.15.38:0/2403603163 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5601b67e93e0 con 0x5601b67e57e0 2024-09-21T03:44:22.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.000+0000 7f7c2f461640 1 -- 172.21.15.38:0/2403603163 <== osd.3 v2:172.21.15.38:6808/1286810294 11 ==== osd_op_reply(75 500.00000000 [create] v133'410 uv403 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7c2805f030 con 0x5601b67e57e0 2024-09-21T03:44:22.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.000+0000 7f7c35e14f40 1 -- 172.21.15.38:0/2403603163 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e133) -- 0x5601b67e93e0 con 0x5601b67e57e0 2024-09-21T03:44:22.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.001+0000 7f7c2f461640 1 -- 172.21.15.38:0/2403603163 <== osd.3 v2:172.21.15.38:6808/1286810294 12 ==== osd_op_reply(76 500.00000000 [delete] v133'411 uv411 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7c2805f030 con 0x5601b67e57e0 2024-09-21T03:44:22.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.001+0000 7f7c35e14f40 1 -- 172.21.15.38:0/2403603163 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5601b67e93e0 con 0x5601b67e57e0 2024-09-21T03:44:22.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.002+0000 7f7c2f461640 1 -- 172.21.15.38:0/2403603163 <== osd.3 v2:172.21.15.38:6808/1286810294 13 ==== osd_op_reply(77 500.00000000 [create] v133'412 uv412 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7c2805f030 con 0x5601b67e57e0 2024-09-21T03:44:22.004 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-09-21T03:44:22.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.002+0000 7f7c35e14f40 1 -- 172.21.15.38:0/2403603163 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5601b67e93e0 con 0x5601b67e57e0 2024-09-21T03:44:22.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.003+0000 7f7c2f461640 1 -- 172.21.15.38:0/2403603163 <== osd.3 v2:172.21.15.38:6808/1286810294 14 ==== osd_op_reply(78 500.00000000 [write 0~90] v133'413 uv413 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7c2805f030 con 0x5601b67e57e0 2024-09-21T03:44:22.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.003+0000 7f7c35e14f40 1 -- 172.21.15.38:0/2403603163 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5601b67e93e0 con 0x5601b67e57e0 2024-09-21T03:44:22.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.004+0000 7f7c2f461640 1 -- 172.21.15.38:0/2403603163 <== osd.3 v2:172.21.15.38:6808/1286810294 15 ==== osd_op_reply(79 500.00000000 [omap-set-header] v133'414 uv414 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7c2805f030 con 0x5601b67e57e0 2024-09-21T03:44:22.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.004+0000 7f7c35e14f40 1 -- 172.21.15.38:0/2403603163 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:80 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5601b67e93e0 con 0x5601b67e1330 2024-09-21T03:44:22.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.005+0000 7f7c2e45f640 1 -- 172.21.15.38:0/2403603163 <== osd.0 v2:172.21.15.38:6800/2381639433 13 ==== osd_op_reply(80 mds_snaptable [create] v133'183 uv182 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f7c2400a4e0 con 0x5601b67e1330 2024-09-21T03:44:22.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.005+0000 7f7c35e14f40 1 -- 172.21.15.38:0/2403603163 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:81 13.d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e133) -- 0x5601b67e93e0 con 0x5601b67e1330 2024-09-21T03:44:22.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.007+0000 7f7c2e45f640 1 -- 172.21.15.38:0/2403603163 <== osd.0 v2:172.21.15.38:6800/2381639433 14 ==== osd_op_reply(81 mds_snaptable [delete] v133'184 uv184 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f7c2400a4e0 con 0x5601b67e1330 2024-09-21T03:44:22.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.007+0000 7f7c35e14f40 1 -- 172.21.15.38:0/2403603163 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:82 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5601b67e93e0 con 0x5601b67e1330 2024-09-21T03:44:22.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.008+0000 7f7c2e45f640 1 -- 172.21.15.38:0/2403603163 <== osd.0 v2:172.21.15.38:6800/2381639433 15 ==== osd_op_reply(82 mds_snaptable [create] v133'185 uv185 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f7c2400a4e0 con 0x5601b67e1330 2024-09-21T03:44:22.009 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-09-21T03:44:22.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.008+0000 7f7c35e14f40 1 -- 172.21.15.38:0/2403603163 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:83 13.d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5601b67e93e0 con 0x5601b67e1330 2024-09-21T03:44:22.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.009+0000 7f7c2e45f640 1 -- 172.21.15.38:0/2403603163 <== osd.0 v2:172.21.15.38:6800/2381639433 16 ==== osd_op_reply(83 mds_snaptable [write 0~70] v133'186 uv186 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f7c2400a4e0 con 0x5601b67e1330 2024-09-21T03:44:22.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.009+0000 7f7c35e14f40 1 -- 172.21.15.38:0/2403603163 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:84 13.d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5601b67e93e0 con 0x5601b67e1330 2024-09-21T03:44:22.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.010+0000 7f7c2e45f640 1 -- 172.21.15.38:0/2403603163 <== osd.0 v2:172.21.15.38:6800/2381639433 17 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v133'187 uv187 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f7c2400a4e0 con 0x5601b67e1330 2024-09-21T03:44:22.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.010+0000 7f7c35e14f40 1 --2- 172.21.15.38:0/2403603163 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x5601b67e9a60 0x5601b67ebe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:22.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.010+0000 7f7c35e14f40 1 -- 172.21.15.38:0/2403603163 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5601b67ec450 con 0x5601b67e9a60 2024-09-21T03:44:22.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.010+0000 7f7c2ec60640 1 --2- 172.21.15.38:0/2403603163 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x5601b67e9a60 0x5601b67ebe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:22.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.010+0000 7f7c2ec60640 1 --2- 172.21.15.38:0/2403603163 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x5601b67e9a60 0x5601b67ebe80 crc :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:22.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.011+0000 7f7c2ec60640 1 -- 172.21.15.38:0/2403603163 <== osd.1 v2:172.21.15.38:6816/1491818732 1 ==== osd_op_reply(85 605.00000000 [create] v133'219 uv218 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7c2000b040 con 0x5601b67e9a60 2024-09-21T03:44:22.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.011+0000 7f7c35e14f40 1 -- 172.21.15.38:0/2403603163 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e133) -- 0x5601b67ec510 con 0x5601b67e9a60 2024-09-21T03:44:22.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.012+0000 7f7c2ec60640 1 -- 172.21.15.38:0/2403603163 <== osd.1 v2:172.21.15.38:6816/1491818732 2 ==== osd_op_reply(86 605.00000000 [delete] v133'220 uv220 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7c2000b040 con 0x5601b67e9a60 2024-09-21T03:44:22.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.012+0000 7f7c35e14f40 1 -- 172.21.15.38:0/2403603163 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5601b67ec900 con 0x5601b67e9a60 2024-09-21T03:44:22.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.013+0000 7f7c2ec60640 1 -- 172.21.15.38:0/2403603163 <== osd.1 v2:172.21.15.38:6816/1491818732 3 ==== osd_op_reply(87 605.00000000 [create] v133'221 uv221 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7c2000b040 con 0x5601b67e9a60 2024-09-21T03:44:22.015 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-09-21T03:44:22.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.013+0000 7f7c35e14f40 1 -- 172.21.15.38:0/2403603163 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5601b67ed910 con 0x5601b67e9a60 2024-09-21T03:44:22.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.015+0000 7f7c2ec60640 1 -- 172.21.15.38:0/2403603163 <== osd.1 v2:172.21.15.38:6816/1491818732 4 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v133'222 uv222 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7c2000b040 con 0x5601b67e9a60 2024-09-21T03:44:22.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.015+0000 7f7c35e14f40 1 -- 172.21.15.38:0/2403603163 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5601b67ed910 con 0x5601b67e9a60 2024-09-21T03:44:22.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.016+0000 7f7c2ec60640 1 -- 172.21.15.38:0/2403603163 <== osd.1 v2:172.21.15.38:6816/1491818732 5 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v133'223 uv223 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7c2000b040 con 0x5601b67e9a60 2024-09-21T03:44:22.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.016+0000 7f7c35e14f40 1 -- 172.21.15.38:0/2403603163 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5601b67ec900 con 0x5601b67e9a60 2024-09-21T03:44:22.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.017+0000 7f7c2ec60640 1 -- 172.21.15.38:0/2403603163 <== osd.1 v2:172.21.15.38:6816/1491818732 6 ==== osd_op_reply(90 605.00000000 [omap-set-header] v133'224 uv224 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7c2000b040 con 0x5601b67e9a60 2024-09-21T03:44:22.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.017+0000 7f7c35e14f40 1 -- 172.21.15.38:0/2403603163 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5601b67ec900 con 0x5601b67e9a60 2024-09-21T03:44:22.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.018+0000 7f7c2ec60640 1 -- 172.21.15.38:0/2403603163 <== osd.1 v2:172.21.15.38:6816/1491818732 7 ==== osd_op_reply(91 mds0_inotable [create] v133'365 uv359 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f7c2000b040 con 0x5601b67e9a60 2024-09-21T03:44:22.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.018+0000 7f7c35e14f40 1 -- 172.21.15.38:0/2403603163 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e133) -- 0x5601b67ec900 con 0x5601b67e9a60 2024-09-21T03:44:22.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.019+0000 7f7c2ec60640 1 -- 172.21.15.38:0/2403603163 <== osd.1 v2:172.21.15.38:6816/1491818732 8 ==== osd_op_reply(92 mds0_inotable [delete] v133'366 uv366 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f7c2000b040 con 0x5601b67e9a60 2024-09-21T03:44:22.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.019+0000 7f7c35e14f40 1 -- 172.21.15.38:0/2403603163 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5601b67ec900 con 0x5601b67e9a60 2024-09-21T03:44:22.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.020+0000 7f7c2ec60640 1 -- 172.21.15.38:0/2403603163 <== osd.1 v2:172.21.15.38:6816/1491818732 9 ==== osd_op_reply(93 mds0_inotable [create] v133'367 uv367 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f7c2000b040 con 0x5601b67e9a60 2024-09-21T03:44:22.022 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-09-21T03:44:22.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.020+0000 7f7c35e14f40 1 -- 172.21.15.38:0/2403603163 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:94 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5601b67ec900 con 0x5601b67e9a60 2024-09-21T03:44:22.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.021+0000 7f7c2ec60640 1 -- 172.21.15.38:0/2403603163 <== osd.1 v2:172.21.15.38:6816/1491818732 10 ==== osd_op_reply(94 mds0_inotable [write 0~34] v133'368 uv368 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f7c2000b040 con 0x5601b67e9a60 2024-09-21T03:44:22.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.021+0000 7f7c35e14f40 1 -- 172.21.15.38:0/2403603163 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:95 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5601b67ec900 con 0x5601b67e9a60 2024-09-21T03:44:22.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.022+0000 7f7c2ec60640 1 -- 172.21.15.38:0/2403603163 <== osd.1 v2:172.21.15.38:6816/1491818732 11 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v133'369 uv369 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f7c2000b040 con 0x5601b67e9a60 2024-09-21T03:44:22.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.022+0000 7f7c35e14f40 1 -- 172.21.15.38:0/2403603163 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:96 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5601b67ec900 con 0x5601b67e9a60 2024-09-21T03:44:22.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.023+0000 7f7c2ec60640 1 -- 172.21.15.38:0/2403603163 <== osd.1 v2:172.21.15.38:6816/1491818732 12 ==== osd_op_reply(96 100.00000000 [create] v133'370 uv364 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7c2000b040 con 0x5601b67e9a60 2024-09-21T03:44:22.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.023+0000 7f7c35e14f40 1 -- 172.21.15.38:0/2403603163 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:97 13.13 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e133) -- 0x5601b67ec900 con 0x5601b67e9a60 2024-09-21T03:44:22.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.024+0000 7f7c2ec60640 1 -- 172.21.15.38:0/2403603163 <== osd.1 v2:172.21.15.38:6816/1491818732 13 ==== osd_op_reply(97 100.00000000 [delete] v133'371 uv371 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7c2000b040 con 0x5601b67e9a60 2024-09-21T03:44:22.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.024+0000 7f7c35e14f40 1 -- 172.21.15.38:0/2403603163 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:98 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5601b67ec900 con 0x5601b67e9a60 2024-09-21T03:44:22.027 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-09-21T03:44:22.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.025+0000 7f7c2ec60640 1 -- 172.21.15.38:0/2403603163 <== osd.1 v2:172.21.15.38:6816/1491818732 14 ==== osd_op_reply(98 100.00000000 [create] v133'372 uv372 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7c2000b040 con 0x5601b67e9a60 2024-09-21T03:44:22.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.025+0000 7f7c35e14f40 1 -- 172.21.15.38:0/2403603163 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:99 13.13 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5601b67ec900 con 0x5601b67e9a60 2024-09-21T03:44:22.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.026+0000 7f7c2ec60640 1 -- 172.21.15.38:0/2403603163 <== osd.1 v2:172.21.15.38:6816/1491818732 15 ==== osd_op_reply(99 100.00000000 [omap-set-header] v133'373 uv373 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7c2000b040 con 0x5601b67e9a60 2024-09-21T03:44:22.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.027+0000 7f7c35e14f40 1 -- 172.21.15.38:0/2403603163 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:100 13.13 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5601b67ef710 con 0x5601b67e9a60 2024-09-21T03:44:22.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.028+0000 7f7c2ec60640 1 -- 172.21.15.38:0/2403603163 <== osd.1 v2:172.21.15.38:6816/1491818732 16 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v133'374 uv374 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7c2000b040 con 0x5601b67e9a60 2024-09-21T03:44:22.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.028+0000 7f7c35e14f40 1 -- 172.21.15.38:0/2403603163 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5601b67ecdb0 con 0x5601b67e57e0 2024-09-21T03:44:22.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.029+0000 7f7c2f461640 1 -- 172.21.15.38:0/2403603163 <== osd.3 v2:172.21.15.38:6808/1286810294 16 ==== osd_op_reply(101 mds0_sessionmap [create] v133'162 uv160 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7f7c2805f030 con 0x5601b67e57e0 2024-09-21T03:44:22.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.029+0000 7f7c35e14f40 1 -- 172.21.15.38:0/2403603163 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e133) -- 0x5601b67ecdb0 con 0x5601b67e57e0 2024-09-21T03:44:22.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.030+0000 7f7c2f461640 1 -- 172.21.15.38:0/2403603163 <== osd.3 v2:172.21.15.38:6808/1286810294 17 ==== osd_op_reply(102 mds0_sessionmap [delete] v133'163 uv163 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f7c2805f030 con 0x5601b67e57e0 2024-09-21T03:44:22.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.030+0000 7f7c35e14f40 1 -- 172.21.15.38:0/2403603163 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:103 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5601b67ecdb0 con 0x5601b67e57e0 2024-09-21T03:44:22.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.031+0000 7f7c2f461640 1 -- 172.21.15.38:0/2403603163 <== osd.3 v2:172.21.15.38:6808/1286810294 18 ==== osd_op_reply(103 mds0_sessionmap [create] v133'164 uv164 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f7c2805f030 con 0x5601b67e57e0 2024-09-21T03:44:22.033 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-09-21T03:44:22.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.031+0000 7f7c35e14f40 1 -- 172.21.15.38:0/2403603163 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:104 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5601b67ecdb0 con 0x5601b67e57e0 2024-09-21T03:44:22.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.032+0000 7f7c2f461640 1 -- 172.21.15.38:0/2403603163 <== osd.3 v2:172.21.15.38:6808/1286810294 19 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v133'165 uv165 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f7c2805f030 con 0x5601b67e57e0 2024-09-21T03:44:22.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.032+0000 7f7c35e14f40 1 -- 172.21.15.38:0/2403603163 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5601b67ecdb0 con 0x5601b67e57e0 2024-09-21T03:44:22.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.033+0000 7f7c2f461640 1 -- 172.21.15.38:0/2403603163 <== osd.3 v2:172.21.15.38:6808/1286810294 20 ==== osd_op_reply(105 609.00000000 [create] v133'219 uv218 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7c2805f030 con 0x5601b67e57e0 2024-09-21T03:44:22.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.033+0000 7f7c35e14f40 1 -- 172.21.15.38:0/2403603163 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e133) -- 0x5601b67ecdb0 con 0x5601b67e57e0 2024-09-21T03:44:22.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.034+0000 7f7c2f461640 1 -- 172.21.15.38:0/2403603163 <== osd.3 v2:172.21.15.38:6808/1286810294 21 ==== osd_op_reply(106 609.00000000 [delete] v133'220 uv220 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7c2805f030 con 0x5601b67e57e0 2024-09-21T03:44:22.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.034+0000 7f7c35e14f40 1 -- 172.21.15.38:0/2403603163 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5601b67ecdb0 con 0x5601b67e57e0 2024-09-21T03:44:22.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.036+0000 7f7c2f461640 1 -- 172.21.15.38:0/2403603163 <== osd.3 v2:172.21.15.38:6808/1286810294 22 ==== osd_op_reply(107 609.00000000 [create] v133'221 uv221 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7c2805f030 con 0x5601b67e57e0 2024-09-21T03:44:22.038 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-09-21T03:44:22.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.036+0000 7f7c35e14f40 1 -- 172.21.15.38:0/2403603163 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5601b67eddc0 con 0x5601b67e57e0 2024-09-21T03:44:22.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.037+0000 7f7c2f461640 1 -- 172.21.15.38:0/2403603163 <== osd.3 v2:172.21.15.38:6808/1286810294 23 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v133'222 uv222 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7c2805f030 con 0x5601b67e57e0 2024-09-21T03:44:22.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.037+0000 7f7c35e14f40 1 -- 172.21.15.38:0/2403603163 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:109 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5601b67eddc0 con 0x5601b67e57e0 2024-09-21T03:44:22.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.038+0000 7f7c2f461640 1 -- 172.21.15.38:0/2403603163 <== osd.3 v2:172.21.15.38:6808/1286810294 24 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v133'223 uv223 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7c2805f030 con 0x5601b67e57e0 2024-09-21T03:44:22.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.038+0000 7f7c35e14f40 1 -- 172.21.15.38:0/2403603163 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:110 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5601b67ecdb0 con 0x5601b67e57e0 2024-09-21T03:44:22.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.040+0000 7f7c2f461640 1 -- 172.21.15.38:0/2403603163 <== osd.3 v2:172.21.15.38:6808/1286810294 25 ==== osd_op_reply(110 609.00000000 [omap-set-header] v133'224 uv224 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7c2805f030 con 0x5601b67e57e0 2024-09-21T03:44:22.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.040+0000 7f7c35e14f40 1 --2- 172.21.15.38:0/2403603163 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x5601b67ed430 0x5601b67ed840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:22.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.040+0000 7f7c35e14f40 1 -- 172.21.15.38:0/2403603163 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5601b67edd80 con 0x5601b67ed430 2024-09-21T03:44:22.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.040+0000 7f7c2e45f640 1 --2- 172.21.15.38:0/2403603163 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x5601b67ed430 0x5601b67ed840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:22.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.040+0000 7f7c2e45f640 1 --2- 172.21.15.38:0/2403603163 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x5601b67ed430 0x5601b67ed840 crc :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:22.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.041+0000 7f7c2e45f640 1 -- 172.21.15.38:0/2403603163 <== osd.2 v2:172.21.15.38:6824/2577858902 1 ==== osd_op_reply(111 400.00000000 [create] v133'186 uv185 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7c2400a4e0 con 0x5601b67ed430 2024-09-21T03:44:22.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.041+0000 7f7c35e14f40 1 -- 172.21.15.38:0/2403603163 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e133) -- 0x5601b67edd80 con 0x5601b67ed430 2024-09-21T03:44:22.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.042+0000 7f7c2e45f640 1 -- 172.21.15.38:0/2403603163 <== osd.2 v2:172.21.15.38:6824/2577858902 2 ==== osd_op_reply(112 400.00000000 [delete] v133'187 uv187 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7c2400a4e0 con 0x5601b67ed430 2024-09-21T03:44:22.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.043+0000 7f7c35e14f40 1 -- 172.21.15.38:0/2403603163 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5601b67edd80 con 0x5601b67ed430 2024-09-21T03:44:22.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.044+0000 7f7c2e45f640 1 -- 172.21.15.38:0/2403603163 <== osd.2 v2:172.21.15.38:6824/2577858902 3 ==== osd_op_reply(113 400.00000000 [create] v133'188 uv188 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7c2400a4e0 con 0x5601b67ed430 2024-09-21T03:44:22.046 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-09-21T03:44:22.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.044+0000 7f7c35e14f40 1 -- 172.21.15.38:0/2403603163 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:114 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5601b67edd80 con 0x5601b67ed430 2024-09-21T03:44:22.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.045+0000 7f7c2e45f640 1 -- 172.21.15.38:0/2403603163 <== osd.2 v2:172.21.15.38:6824/2577858902 4 ==== osd_op_reply(114 400.00000000 [write 0~22] v133'189 uv189 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7c2400a4e0 con 0x5601b67ed430 2024-09-21T03:44:22.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.045+0000 7f7c35e14f40 1 -- 172.21.15.38:0/2403603163 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:115 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5601b67edd80 con 0x5601b67ed430 2024-09-21T03:44:22.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.046+0000 7f7c2e45f640 1 -- 172.21.15.38:0/2403603163 <== osd.2 v2:172.21.15.38:6824/2577858902 5 ==== osd_op_reply(115 400.00000000 [omap-set-header] v133'190 uv190 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7c2400a4e0 con 0x5601b67ed430 2024-09-21T03:44:22.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.046+0000 7f7c35e14f40 1 -- 172.21.15.38:0/2403603163 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:116 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5601b67edd80 con 0x5601b67e9a60 2024-09-21T03:44:22.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.048+0000 7f7c2ec60640 1 -- 172.21.15.38:0/2403603163 <== osd.1 v2:172.21.15.38:6816/1491818732 17 ==== osd_op_reply(116 100.00000000.inode [create] v133'455 uv446 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7f7c2000b040 con 0x5601b67e9a60 2024-09-21T03:44:22.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.048+0000 7f7c35e14f40 1 -- 172.21.15.38:0/2403603163 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:117 13.1f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e133) -- 0x5601b67edd80 con 0x5601b67e9a60 2024-09-21T03:44:22.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.049+0000 7f7c2ec60640 1 -- 172.21.15.38:0/2403603163 <== osd.1 v2:172.21.15.38:6816/1491818732 18 ==== osd_op_reply(117 100.00000000.inode [delete] v133'456 uv456 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f7c2000b040 con 0x5601b67e9a60 2024-09-21T03:44:22.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.049+0000 7f7c35e14f40 1 -- 172.21.15.38:0/2403603163 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:118 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5601b67ee240 con 0x5601b67e9a60 2024-09-21T03:44:22.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.050+0000 7f7c2ec60640 1 -- 172.21.15.38:0/2403603163 <== osd.1 v2:172.21.15.38:6816/1491818732 19 ==== osd_op_reply(118 100.00000000.inode [create] v133'457 uv457 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f7c2000b040 con 0x5601b67e9a60 2024-09-21T03:44:22.052 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-09-21T03:44:22.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.050+0000 7f7c35e14f40 1 -- 172.21.15.38:0/2403603163 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:119 13.1f 13:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5601b67ee240 con 0x5601b67e9a60 2024-09-21T03:44:22.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.052+0000 7f7c2ec60640 1 -- 172.21.15.38:0/2403603163 <== osd.1 v2:172.21.15.38:6816/1491818732 20 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v133'458 uv458 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f7c2000b040 con 0x5601b67e9a60 2024-09-21T03:44:22.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.052+0000 7f7c35e14f40 1 -- 172.21.15.38:0/2403603163 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:120 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5601b67ee240 con 0x5601b67e9a60 2024-09-21T03:44:22.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.053+0000 7f7c2ec60640 1 -- 172.21.15.38:0/2403603163 <== osd.1 v2:172.21.15.38:6816/1491818732 21 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v133'459 uv459 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f7c2000b040 con 0x5601b67e9a60 2024-09-21T03:44:22.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.053+0000 7f7c35e14f40 1 -- 172.21.15.38:0/2403603163 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:121 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5601b67ee210 con 0x5601b67e9a60 2024-09-21T03:44:22.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.054+0000 7f7c2ec60640 1 -- 172.21.15.38:0/2403603163 <== osd.1 v2:172.21.15.38:6816/1491818732 22 ==== osd_op_reply(121 1.00000000 [create] v133'460 uv453 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7f7c2000b040 con 0x5601b67e9a60 2024-09-21T03:44:22.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.054+0000 7f7c35e14f40 1 -- 172.21.15.38:0/2403603163 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:122 13.1f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e133) -- 0x5601b67ee210 con 0x5601b67e9a60 2024-09-21T03:44:22.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.055+0000 7f7c2ec60640 1 -- 172.21.15.38:0/2403603163 <== osd.1 v2:172.21.15.38:6816/1491818732 23 ==== osd_op_reply(122 1.00000000 [delete] v133'461 uv461 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f7c2000b040 con 0x5601b67e9a60 2024-09-21T03:44:22.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.055+0000 7f7c35e14f40 1 -- 172.21.15.38:0/2403603163 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:123 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5601b67ee210 con 0x5601b67e9a60 2024-09-21T03:44:22.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.056+0000 7f7c2ec60640 1 -- 172.21.15.38:0/2403603163 <== osd.1 v2:172.21.15.38:6816/1491818732 24 ==== osd_op_reply(123 1.00000000 [create] v133'462 uv462 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f7c2000b040 con 0x5601b67e9a60 2024-09-21T03:44:22.058 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-09-21T03:44:22.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.056+0000 7f7c35e14f40 1 -- 172.21.15.38:0/2403603163 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:124 13.1f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5601b67ef220 con 0x5601b67e9a60 2024-09-21T03:44:22.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.058+0000 7f7c2ec60640 1 -- 172.21.15.38:0/2403603163 <== osd.1 v2:172.21.15.38:6816/1491818732 25 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v133'463 uv463 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f7c2000b040 con 0x5601b67e9a60 2024-09-21T03:44:22.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.058+0000 7f7c35e14f40 1 -- 172.21.15.38:0/2403603163 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:125 13.1f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5601b67ef220 con 0x5601b67e9a60 2024-09-21T03:44:22.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.059+0000 7f7c2ec60640 1 -- 172.21.15.38:0/2403603163 <== osd.1 v2:172.21.15.38:6816/1491818732 26 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v133'464 uv464 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f7c2000b040 con 0x5601b67e9a60 2024-09-21T03:44:22.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.059+0000 7f7c35e14f40 1 -- 172.21.15.38:0/2403603163 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:126 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5601b67ef670 con 0x5601b67e9a60 2024-09-21T03:44:22.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.060+0000 7f7c2ec60640 1 -- 172.21.15.38:0/2403603163 <== osd.1 v2:172.21.15.38:6816/1491818732 27 ==== osd_op_reply(126 1.00000000 [omap-set-header] v133'465 uv465 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f7c2000b040 con 0x5601b67e9a60 2024-09-21T03:44:22.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.060+0000 7f7c35e14f40 1 -- 172.21.15.38:0/2403603163 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:127 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=584b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5601b67ef620 con 0x5601b67e9a60 2024-09-21T03:44:22.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.061+0000 7f7c2ec60640 1 -- 172.21.15.38:0/2403603163 <== osd.1 v2:172.21.15.38:6816/1491818732 28 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v133'466 uv466 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f7c2000b040 con 0x5601b67e9a60 2024-09-21T03:44:22.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.061+0000 7f7c35e14f40 1 -- 172.21.15.38:0/2403603163 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x5601b67e1330 msgr2=0x5601b67e3750 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:22.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.061+0000 7f7c35e14f40 1 --2- 172.21.15.38:0/2403603163 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x5601b67e1330 0x5601b67e3750 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:22.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.061+0000 7f7c35e14f40 1 -- 172.21.15.38:0/2403603163 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x5601b67e9a60 msgr2=0x5601b67ebe80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:22.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.061+0000 7f7c35e14f40 1 --2- 172.21.15.38:0/2403603163 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x5601b67e9a60 0x5601b67ebe80 crc :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:22.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.061+0000 7f7c35e14f40 1 -- 172.21.15.38:0/2403603163 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x5601b67ed430 msgr2=0x5601b67ed840 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:22.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.061+0000 7f7c35e14f40 1 --2- 172.21.15.38:0/2403603163 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x5601b67ed430 0x5601b67ed840 crc :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:22.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.061+0000 7f7c35e14f40 1 -- 172.21.15.38:0/2403603163 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x5601b67e57e0 msgr2=0x5601b67e8640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:22.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.061+0000 7f7c35e14f40 1 --2- 172.21.15.38:0/2403603163 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x5601b67e57e0 0x5601b67e8640 crc :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:22.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.061+0000 7f7c2f461640 1 -- 172.21.15.38:0/2403603163 reap_dead start 2024-09-21T03:44:22.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.062+0000 7f7c35e14f40 1 -- 172.21.15.38:0/2403603163 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x5601b67dcdf0 msgr2=0x5601b67df210 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:22.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.062+0000 7f7c35e14f40 1 --2- 172.21.15.38:0/2403603163 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x5601b67dcdf0 0x5601b67df210 crc :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:22.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.062+0000 7f7c35e14f40 1 -- 172.21.15.38:0/2403603163 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x5601b672e650 msgr2=0x5601b67361f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:22.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.062+0000 7f7c35e14f40 1 --2- 172.21.15.38:0/2403603163 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x5601b672e650 0x5601b67361f0 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:22.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.062+0000 7f7c35e14f40 1 -- 172.21.15.38:0/2403603163 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7c1c0775a0 msgr2=0x7f7c1c079ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:22.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.062+0000 7f7c35e14f40 1 --2- 172.21.15.38:0/2403603163 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7c1c0775a0 0x7f7c1c079ab0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x5601b6717b50 tx=0x7f7c2400e040 comp rx=0 tx=0).stop 2024-09-21T03:44:22.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.062+0000 7f7c35e14f40 1 -- 172.21.15.38:0/2403603163 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5601b6661d00 msgr2=0x5601b67cddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:22.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.062+0000 7f7c35e14f40 1 --2- 172.21.15.38:0/2403603163 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5601b6661d00 0x5601b67cddb0 secure :-1 s=READY pgs=2199 cs=0 l=1 rev1=1 crypto rx=0x7f7c2000d6c0 tx=0x7f7c2000db80 comp rx=0 tx=0).stop 2024-09-21T03:44:22.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.062+0000 7f7c2f461640 1 -- 172.21.15.38:0/2403603163 reap_dead start 2024-09-21T03:44:22.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.062+0000 7f7c35e14f40 1 -- 172.21.15.38:0/2403603163 shutdown_connections 2024-09-21T03:44:22.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.062+0000 7f7c35e14f40 1 -- 172.21.15.38:0/2403603163 >> 172.21.15.38:0/2403603163 conn(0x5601b6580ac0 msgr2=0x5601b6737c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:22.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.062+0000 7f7c35e14f40 1 -- 172.21.15.38:0/2403603163 shutdown_connections 2024-09-21T03:44:22.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.062+0000 7f7c35e14f40 1 -- 172.21.15.38:0/2403603163 wait complete. 2024-09-21T03:44:22.067 DEBUG:teuthology.orchestra.run.smithi038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata setomapheader 100.00000000 deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef 2024-09-21T03:44:22.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.122+0000 7fe44c668f40 1 Processor -- start 2024-09-21T03:44:22.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.122+0000 7fe44c668f40 1 -- start start 2024-09-21T03:44:22.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.123+0000 7fe44c668f40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55db2da1b090 0x55db2daa1a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:22.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.123+0000 7fe44c668f40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55db2d9fd700 0x55db2db335f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:22.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.123+0000 7fe44c668f40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55db2da8afb0 0x55db2db33b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:22.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.123+0000 7fe44c668f40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55db2d980430 con 0x55db2da1b090 2024-09-21T03:44:22.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.123+0000 7fe44c668f40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55db2d98d1f0 con 0x55db2da8afb0 2024-09-21T03:44:22.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.123+0000 7fe44c668f40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55db2d921d60 con 0x55db2d9fd700 2024-09-21T03:44:22.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.123+0000 7fe446122640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55db2da1b090 0x55db2daa1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:22.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.123+0000 7fe446122640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55db2da1b090 0x55db2daa1a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38678/0 (socket says 172.21.15.38:38678) 2024-09-21T03:44:22.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.123+0000 7fe446122640 1 -- 172.21.15.38:0/2388593198 learned_addr learned my addr 172.21.15.38:0/2388593198 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:44:22.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.124+0000 7fe445921640 1 --2- 172.21.15.38:0/2388593198 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55db2d9fd700 0x55db2db335f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:22.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.124+0000 7fe446923640 1 --2- 172.21.15.38:0/2388593198 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55db2da8afb0 0x55db2db33b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:22.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.124+0000 7fe446122640 1 -- 172.21.15.38:0/2388593198 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55db2d9fd700 msgr2=0x55db2db335f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:44:22.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.124+0000 7fe446122640 1 --2- 172.21.15.38:0/2388593198 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55db2d9fd700 0x55db2db335f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:22.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.124+0000 7fe446122640 1 -- 172.21.15.38:0/2388593198 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55db2da8afb0 msgr2=0x55db2db33b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:44:22.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.124+0000 7fe446122640 1 --2- 172.21.15.38:0/2388593198 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55db2da8afb0 0x55db2db33b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:22.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.124+0000 7fe446122640 1 -- 172.21.15.38:0/2388593198 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db2d9fc940 con 0x55db2da1b090 2024-09-21T03:44:22.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.124+0000 7fe446122640 1 --2- 172.21.15.38:0/2388593198 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55db2da1b090 0x55db2daa1a40 secure :-1 s=READY pgs=2201 cs=0 l=1 rev1=1 crypto rx=0x7fe438000f80 tx=0x7fe438001360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dc7447a73a6d7e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:22.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.124+0000 7fe44491f640 1 -- 172.21.15.38:0/2388593198 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe438001830 con 0x55db2da1b090 2024-09-21T03:44:22.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.124+0000 7fe44491f640 1 -- 172.21.15.38:0/2388593198 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe4380019d0 con 0x55db2da1b090 2024-09-21T03:44:22.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.124+0000 7fe44491f640 1 -- 172.21.15.38:0/2388593198 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe438009330 con 0x55db2da1b090 2024-09-21T03:44:22.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.125+0000 7fe44c668f40 1 -- 172.21.15.38:0/2388593198 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55db2da1b090 msgr2=0x55db2daa1a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:22.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.125+0000 7fe44c668f40 1 --2- 172.21.15.38:0/2388593198 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55db2da1b090 0x55db2daa1a40 secure :-1 s=READY pgs=2201 cs=0 l=1 rev1=1 crypto rx=0x7fe438000f80 tx=0x7fe438001360 comp rx=0 tx=0).stop 2024-09-21T03:44:22.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.125+0000 7fe44c668f40 1 -- 172.21.15.38:0/2388593198 shutdown_connections 2024-09-21T03:44:22.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.125+0000 7fe44c668f40 1 --2- 172.21.15.38:0/2388593198 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55db2da8afb0 0x55db2db33b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:22.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.125+0000 7fe44c668f40 1 --2- 172.21.15.38:0/2388593198 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55db2d9fd700 0x55db2db335f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:22.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.125+0000 7fe44c668f40 1 --2- 172.21.15.38:0/2388593198 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55db2da1b090 0x55db2daa1a40 unknown :-1 s=CLOSED pgs=2201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:22.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.125+0000 7fe44c668f40 1 -- 172.21.15.38:0/2388593198 >> 172.21.15.38:0/2388593198 conn(0x55db2d8f2ac0 msgr2=0x55db2d9cf660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:22.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.125+0000 7fe44c668f40 1 -- 172.21.15.38:0/2388593198 shutdown_connections 2024-09-21T03:44:22.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.125+0000 7fe44c668f40 1 -- 172.21.15.38:0/2388593198 wait complete. 2024-09-21T03:44:22.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.126+0000 7fe44c668f40 1 Processor -- start 2024-09-21T03:44:22.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.126+0000 7fe44c668f40 1 -- start start 2024-09-21T03:44:22.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.127+0000 7fe44c668f40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55db2da1b090 0x55db2da8a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:22.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.127+0000 7fe44c668f40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55db2da8afb0 0x55db2db31590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:22.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.127+0000 7fe44c668f40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55db2daa4620 0x55db2db31ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:22.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.127+0000 7fe44c668f40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55db2d9fcc20 con 0x55db2da1b090 2024-09-21T03:44:22.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.127+0000 7fe44c668f40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55db2d9fcf00 con 0x55db2da8afb0 2024-09-21T03:44:22.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.127+0000 7fe44c668f40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55db2db2ed20 con 0x55db2daa4620 2024-09-21T03:44:22.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.127+0000 7fe446122640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55db2da1b090 0x55db2da8a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:22.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.127+0000 7fe446122640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55db2da1b090 0x55db2da8a230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38694/0 (socket says 172.21.15.38:38694) 2024-09-21T03:44:22.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.127+0000 7fe446122640 1 -- 172.21.15.38:0/3031391303 learned_addr learned my addr 172.21.15.38:0/3031391303 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:44:22.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.127+0000 7fe446923640 1 --2- 172.21.15.38:0/3031391303 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55db2daa4620 0x55db2db31ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:22.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.127+0000 7fe445921640 1 --2- 172.21.15.38:0/3031391303 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55db2da8afb0 0x55db2db31590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:22.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.127+0000 7fe446122640 1 -- 172.21.15.38:0/3031391303 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55db2daa4620 msgr2=0x55db2db31ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:22.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.127+0000 7fe446122640 1 --2- 172.21.15.38:0/3031391303 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55db2daa4620 0x55db2db31ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:22.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.127+0000 7fe446122640 1 -- 172.21.15.38:0/3031391303 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55db2da8afb0 msgr2=0x55db2db31590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:22.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.127+0000 7fe446122640 1 --2- 172.21.15.38:0/3031391303 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55db2da8afb0 0x55db2db31590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:22.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.127+0000 7fe446122640 1 -- 172.21.15.38:0/3031391303 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db2da8a770 con 0x55db2da1b090 2024-09-21T03:44:22.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.128+0000 7fe446122640 1 --2- 172.21.15.38:0/3031391303 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55db2da1b090 0x55db2da8a230 secure :-1 s=READY pgs=2202 cs=0 l=1 rev1=1 crypto rx=0x7fe4380013d0 tx=0x7fe43804edd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:22.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.128+0000 7fe4377fe640 1 -- 172.21.15.38:0/3031391303 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4380092d0 con 0x55db2da1b090 2024-09-21T03:44:22.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.128+0000 7fe4377fe640 1 -- 172.21.15.38:0/3031391303 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe4380017b0 con 0x55db2da1b090 2024-09-21T03:44:22.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.128+0000 7fe44c668f40 1 -- 172.21.15.38:0/3031391303 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55db2db35ad0 con 0x55db2da1b090 2024-09-21T03:44:22.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.128+0000 7fe4377fe640 1 -- 172.21.15.38:0/3031391303 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe438007580 con 0x55db2da1b090 2024-09-21T03:44:22.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.128+0000 7fe44c668f40 1 -- 172.21.15.38:0/3031391303 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db2da8d160 con 0x55db2da1b090 2024-09-21T03:44:22.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.129+0000 7fe4377fe640 1 -- 172.21.15.38:0/3031391303 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fe438075020 con 0x55db2da1b090 2024-09-21T03:44:22.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.130+0000 7fe4377fe640 1 --2- 172.21.15.38:0/3031391303 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fe4280775a0 0x7fe428079ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:22.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.130+0000 7fe4377fe640 1 -- 172.21.15.38:0/3031391303 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 9486+0+0 (secure 0 0 0) 0x7fe4380df1a0 con 0x55db2da1b090 2024-09-21T03:44:22.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.130+0000 7fe445921640 1 --2- 172.21.15.38:0/3031391303 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fe4280775a0 0x7fe428079ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:22.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.130+0000 7fe44c668f40 1 --2- 172.21.15.38:0/3031391303 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x55db2db2e3a0 0x55db2db4ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:22.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.130+0000 7fe44c668f40 1 -- 172.21.15.38:0/3031391303 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:1 13.13 13:cd5a64a3:::100.00000000:head [omap-set-header in=80b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x55db2d9cf810 con 0x55db2db2e3a0 2024-09-21T03:44:22.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.130+0000 7fe446923640 1 --2- 172.21.15.38:0/3031391303 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x55db2db2e3a0 0x55db2db4ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:22.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.131+0000 7fe445921640 1 --2- 172.21.15.38:0/3031391303 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fe4280775a0 0x7fe428079ab0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x55db2d9fc0e0 tx=0x7fe43c004880 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:22.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.131+0000 7fe446923640 1 --2- 172.21.15.38:0/3031391303 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x55db2db2e3a0 0x55db2db4ad50 crc :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:22.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.132+0000 7fe446923640 1 -- 172.21.15.38:0/3031391303 <== osd.1 v2:172.21.15.38:6816/1491818732 1 ==== osd_op_reply(1 100.00000000 [omap-set-header] v133'375 uv375 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe44000b040 con 0x55db2db2e3a0 2024-09-21T03:44:22.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.132+0000 7fe44c668f40 1 -- 172.21.15.38:0/3031391303 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x55db2db2e3a0 msgr2=0x55db2db4ad50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:22.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.132+0000 7fe44c668f40 1 --2- 172.21.15.38:0/3031391303 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x55db2db2e3a0 0x55db2db4ad50 crc :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:22.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.132+0000 7fe44c668f40 1 -- 172.21.15.38:0/3031391303 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fe4280775a0 msgr2=0x7fe428079ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:22.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.132+0000 7fe44c668f40 1 --2- 172.21.15.38:0/3031391303 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fe4280775a0 0x7fe428079ab0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x55db2d9fc0e0 tx=0x7fe43c004880 comp rx=0 tx=0).stop 2024-09-21T03:44:22.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.132+0000 7fe44c668f40 1 -- 172.21.15.38:0/3031391303 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55db2da1b090 msgr2=0x55db2da8a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:22.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.132+0000 7fe44c668f40 1 --2- 172.21.15.38:0/3031391303 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55db2da1b090 0x55db2da8a230 secure :-1 s=READY pgs=2202 cs=0 l=1 rev1=1 crypto rx=0x7fe4380013d0 tx=0x7fe43804edd0 comp rx=0 tx=0).stop 2024-09-21T03:44:22.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.132+0000 7fe446923640 1 -- 172.21.15.38:0/3031391303 reap_dead start 2024-09-21T03:44:22.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.132+0000 7fe44c668f40 1 -- 172.21.15.38:0/3031391303 shutdown_connections 2024-09-21T03:44:22.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.132+0000 7fe44c668f40 1 -- 172.21.15.38:0/3031391303 >> 172.21.15.38:0/3031391303 conn(0x55db2d8f2ac0 msgr2=0x55db2da8f380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:22.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.132+0000 7fe44c668f40 1 -- 172.21.15.38:0/3031391303 shutdown_connections 2024-09-21T03:44:22.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.133+0000 7fe44c668f40 1 -- 172.21.15.38:0/3031391303 wait complete. 2024-09-21T03:44:22.137 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-21T03:44:22.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.255+0000 7fe8a2b13640 1 -- 172.21.15.38:0/2106217422 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe89c0ce460 msgr2=0x7fe89c169890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:22.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.255+0000 7fe8a2b13640 1 --2- 172.21.15.38:0/2106217422 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe89c0ce460 0x7fe89c169890 secure :-1 s=READY pgs=2204 cs=0 l=1 rev1=1 crypto rx=0x7fe894005fd0 tx=0x7fe89402a720 comp rx=0 tx=0).stop 2024-09-21T03:44:22.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.255+0000 7fe8a2b13640 1 -- 172.21.15.38:0/2106217422 shutdown_connections 2024-09-21T03:44:22.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.255+0000 7fe8a2b13640 1 --2- 172.21.15.38:0/2106217422 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe89c0ce460 0x7fe89c169890 unknown :-1 s=CLOSED pgs=2204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:22.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.255+0000 7fe8a2b13640 1 -- 172.21.15.38:0/2106217422 >> 172.21.15.38:0/2106217422 conn(0x7fe89c0b46e0 msgr2=0x7fe89c0d23e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:22.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.255+0000 7fe8a2b13640 1 -- 172.21.15.38:0/2106217422 shutdown_connections 2024-09-21T03:44:22.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.255+0000 7fe8a2b13640 1 -- 172.21.15.38:0/2106217422 wait complete. 2024-09-21T03:44:22.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.256+0000 7fe8a2b13640 1 Processor -- start 2024-09-21T03:44:22.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.256+0000 7fe8a2b13640 1 -- start start 2024-09-21T03:44:22.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.258+0000 7fe8a2b13640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe89c0b5bd0 0x7fe89c15a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:22.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.258+0000 7fe8a2b13640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe89c0ce460 0x7fe89c15aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:22.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.258+0000 7fe8a2b13640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe89c15ead0 0x7fe89c151660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:22.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.258+0000 7fe8a2b13640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fe89c169fe0 con 0x7fe89c0ce460 2024-09-21T03:44:22.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.258+0000 7fe8a2b13640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fe89c0d0b60 con 0x7fe89c0b5bd0 2024-09-21T03:44:22.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.258+0000 7fe8a2b13640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fe89c16a160 con 0x7fe89c15ead0 2024-09-21T03:44:22.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.258+0000 7fe893fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe89c0ce460 0x7fe89c15aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:22.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.258+0000 7fe893fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe89c0ce460 0x7fe89c15aea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38700/0 (socket says 172.21.15.38:38700) 2024-09-21T03:44:22.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.258+0000 7fe893fff640 1 -- 172.21.15.38:0/540933137 learned_addr learned my addr 172.21.15.38:0/540933137 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:44:22.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.258+0000 7fe8a08af640 1 --2- 172.21.15.38:0/540933137 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe89c0b5bd0 0x7fe89c15a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:22.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.258+0000 7fe8a10b0640 1 --2- 172.21.15.38:0/540933137 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe89c15ead0 0x7fe89c151660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:22.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.258+0000 7fe893fff640 1 -- 172.21.15.38:0/540933137 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe89c15ead0 msgr2=0x7fe89c151660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:22.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.258+0000 7fe893fff640 1 --2- 172.21.15.38:0/540933137 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe89c15ead0 0x7fe89c151660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:22.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.258+0000 7fe893fff640 1 -- 172.21.15.38:0/540933137 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe89c0b5bd0 msgr2=0x7fe89c15a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:22.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.258+0000 7fe893fff640 1 --2- 172.21.15.38:0/540933137 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe89c0b5bd0 0x7fe89c15a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:22.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.258+0000 7fe893fff640 1 -- 172.21.15.38:0/540933137 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe89c15b6b0 con 0x7fe89c0ce460 2024-09-21T03:44:22.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.258+0000 7fe8a08af640 1 --2- 172.21.15.38:0/540933137 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe89c0b5bd0 0x7fe89c15a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:44:22.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.259+0000 7fe893fff640 1 --2- 172.21.15.38:0/540933137 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe89c0ce460 0x7fe89c15aea0 secure :-1 s=READY pgs=2205 cs=0 l=1 rev1=1 crypto rx=0x7fe88c00c430 tx=0x7fe88c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:22.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.259+0000 7fe8917fa640 1 -- 172.21.15.38:0/540933137 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe88c017070 con 0x7fe89c0ce460 2024-09-21T03:44:22.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.259+0000 7fe8917fa640 1 -- 172.21.15.38:0/540933137 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe88c0035f0 con 0x7fe89c0ce460 2024-09-21T03:44:22.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.259+0000 7fe8a2b13640 1 -- 172.21.15.38:0/540933137 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe89c15b990 con 0x7fe89c0ce460 2024-09-21T03:44:22.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.259+0000 7fe8917fa640 1 -- 172.21.15.38:0/540933137 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe88c003a40 con 0x7fe89c0ce460 2024-09-21T03:44:22.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.259+0000 7fe8a2b13640 1 -- 172.21.15.38:0/540933137 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe89c14ee50 con 0x7fe89c0ce460 2024-09-21T03:44:22.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.260+0000 7fe8a2b13640 1 -- 172.21.15.38:0/540933137 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe89c162710 con 0x7fe89c0ce460 2024-09-21T03:44:22.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.260+0000 7fe8917fa640 1 -- 172.21.15.38:0/540933137 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fe88c01a070 con 0x7fe89c0ce460 2024-09-21T03:44:22.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.261+0000 7fe8917fa640 1 --2- 172.21.15.38:0/540933137 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fe888078b90 0x7fe88807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:22.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.261+0000 7fe8917fa640 1 -- 172.21.15.38:0/540933137 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 9486+0+0 (secure 0 0 0) 0x7fe88c09af60 con 0x7fe89c0ce460 2024-09-21T03:44:22.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.261+0000 7fe8a08af640 1 --2- 172.21.15.38:0/540933137 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fe888078b90 0x7fe88807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:22.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.262+0000 7fe8a08af640 1 --2- 172.21.15.38:0/540933137 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fe888078b90 0x7fe88807b0a0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fe898007fd0 tx=0x7fe898009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:22.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.266+0000 7fe8917fa640 1 -- 172.21.15.38:0/540933137 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fe88c062550 con 0x7fe89c0ce460 2024-09-21T03:44:22.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.495+0000 7fe8a2b13640 1 -- 172.21.15.38:0/540933137 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7fe89c0cca40 con 0x7fe89c0ce460 2024-09-21T03:44:22.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.796+0000 7fe8917fa640 1 -- 172.21.15.38:0/540933137 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v315) ==== 162+0+0 (secure 0 0 0) 0x7fe88c061e50 con 0x7fe89c0ce460 2024-09-21T03:44:22.799 INFO:teuthology.orchestra.run.smithi038.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-21T03:44:22.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.799+0000 7fe8a2b13640 1 -- 172.21.15.38:0/540933137 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fe888078b90 msgr2=0x7fe88807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:22.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.799+0000 7fe8a2b13640 1 --2- 172.21.15.38:0/540933137 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fe888078b90 0x7fe88807b0a0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fe898007fd0 tx=0x7fe898009040 comp rx=0 tx=0).stop 2024-09-21T03:44:22.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.799+0000 7fe8a2b13640 1 -- 172.21.15.38:0/540933137 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe89c0ce460 msgr2=0x7fe89c15aea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:22.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.799+0000 7fe8a2b13640 1 --2- 172.21.15.38:0/540933137 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe89c0ce460 0x7fe89c15aea0 secure :-1 s=READY pgs=2205 cs=0 l=1 rev1=1 crypto rx=0x7fe88c00c430 tx=0x7fe88c00c8f0 comp rx=0 tx=0).stop 2024-09-21T03:44:22.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.799+0000 7fe8a2b13640 1 -- 172.21.15.38:0/540933137 shutdown_connections 2024-09-21T03:44:22.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.799+0000 7fe8a2b13640 1 --2- 172.21.15.38:0/540933137 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fe888078b90 0x7fe88807b0a0 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:22.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.799+0000 7fe8a2b13640 1 --2- 172.21.15.38:0/540933137 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe89c15ead0 0x7fe89c151660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:22.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.799+0000 7fe8a2b13640 1 --2- 172.21.15.38:0/540933137 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe89c0ce460 0x7fe89c15aea0 unknown :-1 s=CLOSED pgs=2205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:22.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.799+0000 7fe8a2b13640 1 --2- 172.21.15.38:0/540933137 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe89c0b5bd0 0x7fe89c15a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:22.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.799+0000 7fe8a2b13640 1 -- 172.21.15.38:0/540933137 >> 172.21.15.38:0/540933137 conn(0x7fe89c0b46e0 msgr2=0x7fe89c1574b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:22.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.799+0000 7fe8a2b13640 1 -- 172.21.15.38:0/540933137 shutdown_connections 2024-09-21T03:44:22.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.799+0000 7fe8a2b13640 1 -- 172.21.15.38:0/540933137 wait complete. 2024-09-21T03:44:22.815 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:44:22.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.931+0000 7f3de3406640 1 -- 172.21.15.38:0/2386829673 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3dc4004800 msgr2=0x7f3dc4004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:22.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.931+0000 7f3de3406640 1 --2- 172.21.15.38:0/2386829673 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3dc4004800 0x7f3dc4004c10 secure :-1 s=READY pgs=2207 cs=0 l=1 rev1=1 crypto rx=0x7f3dd0006fd0 tx=0x7f3dc4005830 comp rx=0 tx=0).stop 2024-09-21T03:44:22.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.931+0000 7f3de3406640 1 -- 172.21.15.38:0/2386829673 shutdown_connections 2024-09-21T03:44:22.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.931+0000 7f3de3406640 1 --2- 172.21.15.38:0/2386829673 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3dc4004800 0x7f3dc4004c10 unknown :-1 s=CLOSED pgs=2207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:22.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.931+0000 7f3de3406640 1 -- 172.21.15.38:0/2386829673 >> 172.21.15.38:0/2386829673 conn(0x7f3ddc0b6b20 msgr2=0x7f3ddc0d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:22.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.931+0000 7f3de3406640 1 -- 172.21.15.38:0/2386829673 shutdown_connections 2024-09-21T03:44:22.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.931+0000 7f3de3406640 1 -- 172.21.15.38:0/2386829673 wait complete. 2024-09-21T03:44:22.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.933+0000 7f3de3406640 1 Processor -- start 2024-09-21T03:44:22.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.933+0000 7f3de3406640 1 -- start start 2024-09-21T03:44:22.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.934+0000 7f3de3406640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3dc4004800 0x7f3ddc1f56c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:22.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.934+0000 7f3de3406640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3ddc153940 0x7f3ddc152bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:22.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.934+0000 7f3de3406640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3ddc1e9250 0x7f3ddc1e9660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:22.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.934+0000 7f3de3406640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f3ddc162b80 con 0x7f3ddc1e9250 2024-09-21T03:44:22.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.934+0000 7f3de3406640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f3ddc162a00 con 0x7f3dc4004800 2024-09-21T03:44:22.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.934+0000 7f3de3406640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f3ddc0d24d0 con 0x7f3ddc153940 2024-09-21T03:44:22.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.934+0000 7f3de19a3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3ddc1e9250 0x7f3ddc1e9660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:22.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.934+0000 7f3de19a3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3ddc1e9250 0x7f3ddc1e9660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41732/0 (socket says 172.21.15.38:41732) 2024-09-21T03:44:22.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.934+0000 7f3de19a3640 1 -- 172.21.15.38:0/3871040162 learned_addr learned my addr 172.21.15.38:0/3871040162 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:44:22.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.935+0000 7f3de09a1640 1 --2- 172.21.15.38:0/3871040162 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3ddc153940 0x7f3ddc152bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:22.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.935+0000 7f3de11a2640 1 --2- 172.21.15.38:0/3871040162 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3dc4004800 0x7f3ddc1f56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:22.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.935+0000 7f3de19a3640 1 -- 172.21.15.38:0/3871040162 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3ddc153940 msgr2=0x7f3ddc152bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:22.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.935+0000 7f3de19a3640 1 --2- 172.21.15.38:0/3871040162 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3ddc153940 0x7f3ddc152bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:22.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.935+0000 7f3de19a3640 1 -- 172.21.15.38:0/3871040162 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3dc4004800 msgr2=0x7f3ddc1f56c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:22.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.935+0000 7f3de19a3640 1 --2- 172.21.15.38:0/3871040162 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3dc4004800 0x7f3ddc1f56c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:22.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.935+0000 7f3de19a3640 1 -- 172.21.15.38:0/3871040162 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ddc1f5c00 con 0x7f3ddc1e9250 2024-09-21T03:44:22.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.935+0000 7f3de19a3640 1 --2- 172.21.15.38:0/3871040162 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3ddc1e9250 0x7f3ddc1e9660 secure :-1 s=READY pgs=2208 cs=0 l=1 rev1=1 crypto rx=0x7f3ddc0d0330 tx=0x7f3dd4001630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:22.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.935+0000 7f3dd9ffb640 1 -- 172.21.15.38:0/3871040162 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dd4011630 con 0x7f3ddc1e9250 2024-09-21T03:44:22.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.935+0000 7f3dd9ffb640 1 -- 172.21.15.38:0/3871040162 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3dd4018070 con 0x7f3ddc1e9250 2024-09-21T03:44:22.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.935+0000 7f3dd9ffb640 1 -- 172.21.15.38:0/3871040162 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dd4011a70 con 0x7f3ddc1e9250 2024-09-21T03:44:22.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.935+0000 7f3de3406640 1 -- 172.21.15.38:0/3871040162 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ddc1e81b0 con 0x7f3ddc1e9250 2024-09-21T03:44:22.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.935+0000 7f3de3406640 1 -- 172.21.15.38:0/3871040162 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ddc1e8770 con 0x7f3ddc1e9250 2024-09-21T03:44:22.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.936+0000 7f3de3406640 1 -- 172.21.15.38:0/3871040162 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ddc0d2a30 con 0x7f3ddc1e9250 2024-09-21T03:44:22.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.937+0000 7f3dd9ffb640 1 -- 172.21.15.38:0/3871040162 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f3dd4011c10 con 0x7f3ddc1e9250 2024-09-21T03:44:22.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.938+0000 7f3dd9ffb640 1 --2- 172.21.15.38:0/3871040162 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3dc8078b90 0x7f3dc807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:22.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.938+0000 7f3dd9ffb640 1 -- 172.21.15.38:0/3871040162 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 9486+0+0 (secure 0 0 0) 0x7f3dd409c1f0 con 0x7f3ddc1e9250 2024-09-21T03:44:22.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.938+0000 7f3de11a2640 1 --2- 172.21.15.38:0/3871040162 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3dc8078b90 0x7f3dc807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:22.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.939+0000 7f3de11a2640 1 --2- 172.21.15.38:0/3871040162 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3dc8078b90 0x7f3dc807b0a0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f3ddc06bb20 tx=0x7f3dcc009500 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:22.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:22.942+0000 7f3dd9ffb640 1 -- 172.21.15.38:0/3871040162 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f3dd4063750 con 0x7f3ddc1e9250 2024-09-21T03:44:23.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:23.171+0000 7f3de3406640 1 -- 172.21.15.38:0/3871040162 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3ddc0b4da0 con 0x7f3ddc1e9250 2024-09-21T03:44:23.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:23.172+0000 7f3dd9ffb640 1 -- 172.21.15.38:0/3871040162 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 316 v316) ==== 95+0+5230 (secure 0 0 0) 0x7f3dd4063050 con 0x7f3ddc1e9250 2024-09-21T03:44:23.174 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:44:23.174 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":316,"btime":"2024-09-21T03:44:22:800521+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6086,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/965972663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":965972663},{"type":"v1","addr":"172.21.15.195:6836","nonce":965972663}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6700,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/3176122334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":3176122334},{"type":"v1","addr":"172.21.15.195:6837","nonce":3176122334}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":9840,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/3801772046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":3801772046},{"type":"v1","addr":"172.21.15.38:6835","nonce":3801772046}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":273}],"filesystems":[{"mdsmap":{"epoch":316,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:40:02.319282+0000","modified":"2024-09-21T03:44:22.800480+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":133,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10359},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10359":{"gid":10359,"name":"c","rank":0,"incarnation":316,"state":"up:replay","state_seq":1,"addr":"172.21.15.38:6833/2851755473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":2851755473},{"type":"v1","addr":"172.21.15.38:6833","nonce":2851755473}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-21T03:44:23.174 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 316 2024-09-21T03:44:23.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:23.174+0000 7f3de3406640 1 -- 172.21.15.38:0/3871040162 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3dc8078b90 msgr2=0x7f3dc807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:23.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:23.174+0000 7f3de3406640 1 --2- 172.21.15.38:0/3871040162 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3dc8078b90 0x7f3dc807b0a0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f3ddc06bb20 tx=0x7f3dcc009500 comp rx=0 tx=0).stop 2024-09-21T03:44:23.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:23.174+0000 7f3de3406640 1 -- 172.21.15.38:0/3871040162 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3ddc1e9250 msgr2=0x7f3ddc1e9660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:23.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:23.174+0000 7f3de3406640 1 --2- 172.21.15.38:0/3871040162 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3ddc1e9250 0x7f3ddc1e9660 secure :-1 s=READY pgs=2208 cs=0 l=1 rev1=1 crypto rx=0x7f3ddc0d0330 tx=0x7f3dd4001630 comp rx=0 tx=0).stop 2024-09-21T03:44:23.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:23.174+0000 7f3de3406640 1 -- 172.21.15.38:0/3871040162 shutdown_connections 2024-09-21T03:44:23.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:23.174+0000 7f3de3406640 1 --2- 172.21.15.38:0/3871040162 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3ddc1e9250 0x7f3ddc1e9660 unknown :-1 s=CLOSED pgs=2208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:23.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:23.174+0000 7f3de3406640 1 --2- 172.21.15.38:0/3871040162 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3dc8078b90 0x7f3dc807b0a0 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:23.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:23.174+0000 7f3de3406640 1 --2- 172.21.15.38:0/3871040162 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3ddc153940 0x7f3ddc152bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:23.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:23.174+0000 7f3de3406640 1 --2- 172.21.15.38:0/3871040162 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3dc4004800 0x7f3ddc1f56c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:23.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:23.174+0000 7f3de3406640 1 -- 172.21.15.38:0/3871040162 >> 172.21.15.38:0/3871040162 conn(0x7f3ddc0b6b20 msgr2=0x7f3ddc0cf830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:23.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:23.174+0000 7f3de3406640 1 -- 172.21.15.38:0/3871040162 shutdown_connections 2024-09-21T03:44:23.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:23.174+0000 7f3de3406640 1 -- 172.21.15.38:0/3871040162 wait complete. 2024-09-21T03:44:23.188 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2024-09-21T03:44:25.829 INFO:tasks.ceph.mds.c.smithi038.stderr:2024-09-21T03:44:25.827+0000 7f39c1b6f640 -1 mds.0.cache.dir(0x100) Corrupt fnode in dirfrag 0x100: Decoder at 'void fnode_t::decode(ceph::buffer::v15_2_0::list::const_iterator&)' v=3 cannot decode v=100 minimal_decoder=101: Malformed input [buffer:3] 2024-09-21T03:44:25.831 INFO:tasks.ceph.mon.a.smithi038.stderr:2024-09-21T03:44:25.830+0000 7f3f45713640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T03:44:25.831 INFO:tasks.ceph.mon.a.smithi038.stderr:2024-09-21T03:44:25.830+0000 7f3f45713640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-09-21T03:44:26.805 INFO:tasks.ceph.mds.c.smithi038.stderr:2024-09-21T03:44:26.803+0000 7f39c5b77640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T03:44:28.188 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:44:28.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:28.317+0000 7f91d6425640 1 -- 172.21.15.38:0/3069088435 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f91d006e160 msgr2=0x7f91d01673c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:28.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:28.317+0000 7f91d6425640 1 --2- 172.21.15.38:0/3069088435 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f91d006e160 0x7f91d01673c0 secure :-1 s=READY pgs=2210 cs=0 l=1 rev1=1 crypto rx=0x7f91c8005fd0 tx=0x7f91c802a820 comp rx=0 tx=0).stop 2024-09-21T03:44:28.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:28.317+0000 7f91d6425640 1 -- 172.21.15.38:0/3069088435 shutdown_connections 2024-09-21T03:44:28.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:28.317+0000 7f91d6425640 1 --2- 172.21.15.38:0/3069088435 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f91d006e160 0x7f91d01673c0 unknown :-1 s=CLOSED pgs=2210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:28.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:28.317+0000 7f91d6425640 1 -- 172.21.15.38:0/3069088435 >> 172.21.15.38:0/3069088435 conn(0x7f91d00d0a30 msgr2=0x7f91d00d0e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:28.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:28.317+0000 7f91d6425640 1 -- 172.21.15.38:0/3069088435 shutdown_connections 2024-09-21T03:44:28.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:28.317+0000 7f91d6425640 1 -- 172.21.15.38:0/3069088435 wait complete. 2024-09-21T03:44:28.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:28.319+0000 7f91d6425640 1 Processor -- start 2024-09-21T03:44:28.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:28.319+0000 7f91d6425640 1 -- start start 2024-09-21T03:44:28.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:28.320+0000 7f91d6425640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f91d006e160 0x7f91d015c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:28.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:28.320+0000 7f91d6425640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f91d00cf700 0x7f91d013e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:28.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:28.320+0000 7f91d6425640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f91d0153410 0x7f91d00ae7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:28.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:28.320+0000 7f91d6425640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f91d0167b10 con 0x7f91d006e160 2024-09-21T03:44:28.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:28.320+0000 7f91d6425640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f91d015cbc0 con 0x7f91d00cf700 2024-09-21T03:44:28.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:28.320+0000 7f91d6425640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f91d0167c90 con 0x7f91d0153410 2024-09-21T03:44:28.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:28.320+0000 7f91cffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f91d006e160 0x7f91d015c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:28.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:28.321+0000 7f91cffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f91d006e160 0x7f91d015c250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41744/0 (socket says 172.21.15.38:41744) 2024-09-21T03:44:28.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:28.321+0000 7f91cffff640 1 -- 172.21.15.38:0/3914977389 learned_addr learned my addr 172.21.15.38:0/3914977389 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:44:28.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:28.321+0000 7f91cf7fe640 1 --2- 172.21.15.38:0/3914977389 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f91d00cf700 0x7f91d013e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:28.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:28.321+0000 7f91d49c2640 1 --2- 172.21.15.38:0/3914977389 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f91d0153410 0x7f91d00ae7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:28.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:28.321+0000 7f91cffff640 1 -- 172.21.15.38:0/3914977389 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f91d0153410 msgr2=0x7f91d00ae7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:28.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:28.321+0000 7f91cffff640 1 --2- 172.21.15.38:0/3914977389 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f91d0153410 0x7f91d00ae7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:28.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:28.321+0000 7f91cffff640 1 -- 172.21.15.38:0/3914977389 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f91d00cf700 msgr2=0x7f91d013e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:28.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:28.321+0000 7f91cffff640 1 --2- 172.21.15.38:0/3914977389 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f91d00cf700 0x7f91d013e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:28.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:28.321+0000 7f91cffff640 1 -- 172.21.15.38:0/3914977389 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91d01fa0d0 con 0x7f91d006e160 2024-09-21T03:44:28.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:28.321+0000 7f91d49c2640 1 --2- 172.21.15.38:0/3914977389 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f91d0153410 0x7f91d00ae7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:44:28.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:28.321+0000 7f91cffff640 1 --2- 172.21.15.38:0/3914977389 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f91d006e160 0x7f91d015c250 secure :-1 s=READY pgs=2211 cs=0 l=1 rev1=1 crypto rx=0x7f91c000c3c0 tx=0x7f91c000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:28.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:28.321+0000 7f91ccff9640 1 -- 172.21.15.38:0/3914977389 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91c0017070 con 0x7f91d006e160 2024-09-21T03:44:28.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:28.321+0000 7f91ccff9640 1 -- 172.21.15.38:0/3914977389 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f91c00035f0 con 0x7f91d006e160 2024-09-21T03:44:28.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:28.321+0000 7f91d6425640 1 -- 172.21.15.38:0/3914977389 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91d014c910 con 0x7f91d006e160 2024-09-21T03:44:28.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:28.321+0000 7f91ccff9640 1 -- 172.21.15.38:0/3914977389 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91c0003a80 con 0x7f91d006e160 2024-09-21T03:44:28.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:28.321+0000 7f91d6425640 1 -- 172.21.15.38:0/3914977389 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91d014ce90 con 0x7f91d006e160 2024-09-21T03:44:28.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:28.322+0000 7f91d6425640 1 -- 172.21.15.38:0/3914977389 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91d00cfb10 con 0x7f91d006e160 2024-09-21T03:44:28.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:28.323+0000 7f91ccff9640 1 -- 172.21.15.38:0/3914977389 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f91c001a070 con 0x7f91d006e160 2024-09-21T03:44:28.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:28.324+0000 7f91ccff9640 1 --2- 172.21.15.38:0/3914977389 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f91bc078b90 0x7f91bc07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:28.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:28.324+0000 7f91ccff9640 1 -- 172.21.15.38:0/3914977389 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 9572+0+0 (secure 0 0 0) 0x7f91c009b070 con 0x7f91d006e160 2024-09-21T03:44:28.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:28.324+0000 7f91cf7fe640 1 --2- 172.21.15.38:0/3914977389 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f91bc078b90 0x7f91bc07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:28.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:28.325+0000 7f91cf7fe640 1 --2- 172.21.15.38:0/3914977389 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f91bc078b90 0x7f91bc07b0a0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f91c4006fd0 tx=0x7f91c4008040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:28.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:28.329+0000 7f91ccff9640 1 -- 172.21.15.38:0/3914977389 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f91c00624e0 con 0x7f91d006e160 2024-09-21T03:44:28.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:28.558+0000 7f91d6425640 1 -- 172.21.15.38:0/3914977389 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f91d0069d30 con 0x7f91d006e160 2024-09-21T03:44:28.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:28.558+0000 7f91ccff9640 1 -- 172.21.15.38:0/3914977389 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 320 v320) ==== 95+0+4392 (secure 0 0 0) 0x7f91c0061de0 con 0x7f91d006e160 2024-09-21T03:44:28.562 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:44:28.562 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":320,"btime":"2024-09-21T03:44:25:830803+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6086,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/965972663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":965972663},{"type":"v1","addr":"172.21.15.195:6836","nonce":965972663}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6700,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/3176122334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":3176122334},{"type":"v1","addr":"172.21.15.195:6837","nonce":3176122334}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":9840,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/3801772046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":3801772046},{"type":"v1","addr":"172.21.15.38:6835","nonce":3801772046}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":273}],"filesystems":[{"mdsmap":{"epoch":320,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:40:02.319282+0000","modified":"2024-09-21T03:44:25.830798+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":134,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-21T03:44:28.562 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 320 2024-09-21T03:44:28.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:28.561+0000 7f91d6425640 1 -- 172.21.15.38:0/3914977389 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f91bc078b90 msgr2=0x7f91bc07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:28.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:28.561+0000 7f91d6425640 1 --2- 172.21.15.38:0/3914977389 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f91bc078b90 0x7f91bc07b0a0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f91c4006fd0 tx=0x7f91c4008040 comp rx=0 tx=0).stop 2024-09-21T03:44:28.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:28.561+0000 7f91d6425640 1 -- 172.21.15.38:0/3914977389 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f91d006e160 msgr2=0x7f91d015c250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:28.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:28.561+0000 7f91d6425640 1 --2- 172.21.15.38:0/3914977389 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f91d006e160 0x7f91d015c250 secure :-1 s=READY pgs=2211 cs=0 l=1 rev1=1 crypto rx=0x7f91c000c3c0 tx=0x7f91c000c880 comp rx=0 tx=0).stop 2024-09-21T03:44:28.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:28.561+0000 7f91d6425640 1 -- 172.21.15.38:0/3914977389 shutdown_connections 2024-09-21T03:44:28.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:28.561+0000 7f91d6425640 1 --2- 172.21.15.38:0/3914977389 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f91bc078b90 0x7f91bc07b0a0 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:28.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:28.561+0000 7f91d6425640 1 --2- 172.21.15.38:0/3914977389 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f91d0153410 0x7f91d00ae7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:28.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:28.561+0000 7f91d6425640 1 --2- 172.21.15.38:0/3914977389 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f91d00cf700 0x7f91d013e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:28.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:28.561+0000 7f91d6425640 1 --2- 172.21.15.38:0/3914977389 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f91d006e160 0x7f91d015c250 unknown :-1 s=CLOSED pgs=2211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:28.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:28.561+0000 7f91d6425640 1 -- 172.21.15.38:0/3914977389 >> 172.21.15.38:0/3914977389 conn(0x7f91d00d0a30 msgr2=0x7f91d00afc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:28.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:28.561+0000 7f91d6425640 1 -- 172.21.15.38:0/3914977389 shutdown_connections 2024-09-21T03:44:28.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:28.561+0000 7f91d6425640 1 -- 172.21.15.38:0/3914977389 wait complete. 2024-09-21T03:44:28.575 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-09-21T03:44:28.576 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:44:28.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:28.689+0000 7fdcf1f5e640 1 -- 172.21.15.38:0/463731869 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdcec162f00 msgr2=0x7fdcec0d24d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:28.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:28.689+0000 7fdcf1f5e640 1 --2- 172.21.15.38:0/463731869 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdcec162f00 0x7fdcec0d24d0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc009830 tx=0x7fdcdc02a510 comp rx=0 tx=0).stop 2024-09-21T03:44:28.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:28.690+0000 7fdcf1f5e640 1 -- 172.21.15.38:0/463731869 shutdown_connections 2024-09-21T03:44:28.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:28.690+0000 7fdcf1f5e640 1 --2- 172.21.15.38:0/463731869 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdcec0b2e40 0x7fdcec0b3250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:28.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:28.690+0000 7fdcf1f5e640 1 --2- 172.21.15.38:0/463731869 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdcec162f00 0x7fdcec0d24d0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:28.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:28.690+0000 7fdcf1f5e640 1 -- 172.21.15.38:0/463731869 >> 172.21.15.38:0/463731869 conn(0x7fdcec0b6b20 msgr2=0x7fdcec0d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:28.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:28.690+0000 7fdcf1f5e640 1 -- 172.21.15.38:0/463731869 shutdown_connections 2024-09-21T03:44:28.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:28.690+0000 7fdcf1f5e640 1 -- 172.21.15.38:0/463731869 wait complete. 2024-09-21T03:44:28.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:28.691+0000 7fdcf1f5e640 1 Processor -- start 2024-09-21T03:44:28.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:28.692+0000 7fdcf1f5e640 1 -- start start 2024-09-21T03:44:28.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:28.693+0000 7fdcf1f5e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdcec0b2e40 0x7fdcec0ad390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:28.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:28.693+0000 7fdcf1f5e640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdcec162280 0x7fdcec0ced20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:28.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:28.693+0000 7fdcf1f5e640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdcec14ca60 0x7fdcec14ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:28.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:28.693+0000 7fdcf1f5e640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fdcec15e4b0 con 0x7fdcec0b2e40 2024-09-21T03:44:28.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:28.693+0000 7fdcf1f5e640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fdcec0b3a20 con 0x7fdcec14ca60 2024-09-21T03:44:28.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:28.693+0000 7fdcf1f5e640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fdcec15e630 con 0x7fdcec162280 2024-09-21T03:44:28.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:28.693+0000 7fdceb7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdcec0b2e40 0x7fdcec0ad390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:28.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:28.693+0000 7fdceb7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdcec0b2e40 0x7fdcec0ad390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41752/0 (socket says 172.21.15.38:41752) 2024-09-21T03:44:28.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:28.693+0000 7fdceb7fe640 1 -- 172.21.15.38:0/1624542908 learned_addr learned my addr 172.21.15.38:0/1624542908 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:44:28.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:28.693+0000 7fdcebfff640 1 --2- 172.21.15.38:0/1624542908 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdcec14ca60 0x7fdcec14ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:28.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:28.693+0000 7fdceaffd640 1 --2- 172.21.15.38:0/1624542908 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdcec162280 0x7fdcec0ced20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:28.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:28.693+0000 7fdceb7fe640 1 -- 172.21.15.38:0/1624542908 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdcec162280 msgr2=0x7fdcec0ced20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:28.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:28.693+0000 7fdceb7fe640 1 --2- 172.21.15.38:0/1624542908 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdcec162280 0x7fdcec0ced20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:28.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:28.693+0000 7fdceb7fe640 1 -- 172.21.15.38:0/1624542908 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdcec14ca60 msgr2=0x7fdcec14ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:28.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:28.693+0000 7fdceb7fe640 1 --2- 172.21.15.38:0/1624542908 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdcec14ca60 0x7fdcec14ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:28.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:28.693+0000 7fdceb7fe640 1 -- 172.21.15.38:0/1624542908 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcec0ac2f0 con 0x7fdcec0b2e40 2024-09-21T03:44:28.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:28.694+0000 7fdceb7fe640 1 --2- 172.21.15.38:0/1624542908 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdcec0b2e40 0x7fdcec0ad390 secure :-1 s=READY pgs=2212 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc009950 tx=0x7fdcdc033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:28.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:28.694+0000 7fdcd3fff640 1 -- 172.21.15.38:0/1624542908 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcdc005c90 con 0x7fdcec0b2e40 2024-09-21T03:44:28.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:28.694+0000 7fdcd3fff640 1 -- 172.21.15.38:0/1624542908 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdcdc031e50 con 0x7fdcec0b2e40 2024-09-21T03:44:28.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:28.694+0000 7fdcd3fff640 1 -- 172.21.15.38:0/1624542908 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcdc005260 con 0x7fdcec0b2e40 2024-09-21T03:44:28.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:28.694+0000 7fdcf1f5e640 1 -- 172.21.15.38:0/1624542908 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcec0ac5d0 con 0x7fdcec0b2e40 2024-09-21T03:44:28.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:28.694+0000 7fdcf1f5e640 1 -- 172.21.15.38:0/1624542908 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcec0acb90 con 0x7fdcec0b2e40 2024-09-21T03:44:28.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:28.695+0000 7fdcf1f5e640 1 -- 172.21.15.38:0/1624542908 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdcec162f00 con 0x7fdcec0b2e40 2024-09-21T03:44:28.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:28.696+0000 7fdcd3fff640 1 -- 172.21.15.38:0/1624542908 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fdcdc005400 con 0x7fdcec0b2e40 2024-09-21T03:44:28.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:28.696+0000 7fdcd3fff640 1 --2- 172.21.15.38:0/1624542908 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fdcd8078bc0 0x7fdcd807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:28.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:28.696+0000 7fdcd3fff640 1 -- 172.21.15.38:0/1624542908 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 9572+0+0 (secure 0 0 0) 0x7fdcdc0bcd10 con 0x7fdcec0b2e40 2024-09-21T03:44:28.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:28.697+0000 7fdceaffd640 1 --2- 172.21.15.38:0/1624542908 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fdcd8078bc0 0x7fdcd807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:28.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:28.697+0000 7fdceaffd640 1 --2- 172.21.15.38:0/1624542908 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fdcd8078bc0 0x7fdcd807b0d0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fdce0007fd0 tx=0x7fdce0009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:28.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:28.701+0000 7fdcd3fff640 1 -- 172.21.15.38:0/1624542908 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fdcdc0c2020 con 0x7fdcec0b2e40 2024-09-21T03:44:28.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:28.925+0000 7fdcf1f5e640 1 -- 172.21.15.38:0/1624542908 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdcec050ff0 con 0x7fdcec0b2e40 2024-09-21T03:44:28.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:28.925+0000 7fdcd3fff640 1 -- 172.21.15.38:0/1624542908 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 320 v320) ==== 95+0+4392 (secure 0 0 0) 0x7fdcdc03f030 con 0x7fdcec0b2e40 2024-09-21T03:44:28.929 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:44:28.929 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":320,"btime":"2024-09-21T03:44:25:830803+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6086,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/965972663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":965972663},{"type":"v1","addr":"172.21.15.195:6836","nonce":965972663}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6700,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/3176122334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":3176122334},{"type":"v1","addr":"172.21.15.195:6837","nonce":3176122334}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":9840,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/3801772046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":3801772046},{"type":"v1","addr":"172.21.15.38:6835","nonce":3801772046}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":273}],"filesystems":[{"mdsmap":{"epoch":320,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:40:02.319282+0000","modified":"2024-09-21T03:44:25.830798+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":134,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-21T03:44:28.929 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 320 2024-09-21T03:44:28.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:28.927+0000 7fdcf1f5e640 1 -- 172.21.15.38:0/1624542908 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fdcd8078bc0 msgr2=0x7fdcd807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:28.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:28.927+0000 7fdcf1f5e640 1 --2- 172.21.15.38:0/1624542908 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fdcd8078bc0 0x7fdcd807b0d0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fdce0007fd0 tx=0x7fdce0009040 comp rx=0 tx=0).stop 2024-09-21T03:44:28.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:28.927+0000 7fdcf1f5e640 1 -- 172.21.15.38:0/1624542908 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdcec0b2e40 msgr2=0x7fdcec0ad390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:28.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:28.927+0000 7fdcf1f5e640 1 --2- 172.21.15.38:0/1624542908 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdcec0b2e40 0x7fdcec0ad390 secure :-1 s=READY pgs=2212 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc009950 tx=0x7fdcdc033c20 comp rx=0 tx=0).stop 2024-09-21T03:44:28.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:28.927+0000 7fdcf1f5e640 1 -- 172.21.15.38:0/1624542908 shutdown_connections 2024-09-21T03:44:28.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:28.927+0000 7fdcf1f5e640 1 --2- 172.21.15.38:0/1624542908 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdcec14ca60 0x7fdcec14ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:28.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:28.927+0000 7fdcf1f5e640 1 --2- 172.21.15.38:0/1624542908 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fdcd8078bc0 0x7fdcd807b0d0 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:28.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:28.927+0000 7fdcf1f5e640 1 --2- 172.21.15.38:0/1624542908 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdcec162280 0x7fdcec0ced20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:28.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:28.927+0000 7fdcf1f5e640 1 --2- 172.21.15.38:0/1624542908 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdcec0b2e40 0x7fdcec0ad390 unknown :-1 s=CLOSED pgs=2212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:28.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:28.927+0000 7fdcf1f5e640 1 -- 172.21.15.38:0/1624542908 >> 172.21.15.38:0/1624542908 conn(0x7fdcec0b6b20 msgr2=0x7fdcec06dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:28.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:28.928+0000 7fdcf1f5e640 1 -- 172.21.15.38:0/1624542908 shutdown_connections 2024-09-21T03:44:28.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:28.928+0000 7fdcf1f5e640 1 -- 172.21.15.38:0/1624542908 wait complete. 2024-09-21T03:44:28.942 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-21T03:44:28.942 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:44:29.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:29.056+0000 7fdcf580a640 1 -- 172.21.15.38:0/554116276 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdcdc0039e0 msgr2=0x7fdcdc003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:29.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:29.056+0000 7fdcf580a640 1 --2- 172.21.15.38:0/554116276 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdcdc0039e0 0x7fdcdc003e10 secure :-1 s=READY pgs=2214 cs=0 l=1 rev1=1 crypto rx=0x7fdce0009ed0 tx=0x7fdcdc005830 comp rx=0 tx=0).stop 2024-09-21T03:44:29.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:29.056+0000 7fdcef7fe640 1 -- 172.21.15.38:0/554116276 reap_dead start 2024-09-21T03:44:29.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:29.056+0000 7fdcf580a640 1 -- 172.21.15.38:0/554116276 shutdown_connections 2024-09-21T03:44:29.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:29.056+0000 7fdcf580a640 1 -- 172.21.15.38:0/554116276 >> 172.21.15.38:0/554116276 conn(0x7fdcf00b6b20 msgr2=0x7fdcf00d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:29.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:29.056+0000 7fdcf580a640 1 -- 172.21.15.38:0/554116276 shutdown_connections 2024-09-21T03:44:29.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:29.056+0000 7fdcf580a640 1 -- 172.21.15.38:0/554116276 wait complete. 2024-09-21T03:44:29.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:29.058+0000 7fdcf580a640 1 Processor -- start 2024-09-21T03:44:29.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:29.058+0000 7fdcf580a640 1 -- start start 2024-09-21T03:44:29.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:29.060+0000 7fdcf580a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdcdc0039e0 0x7fdcf01e8fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:29.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:29.060+0000 7fdcf580a640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdcf014ca00 0x7fdcf0149130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:29.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:29.060+0000 7fdcf580a640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdcf0149670 0x7fdcf0149d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:29.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:29.060+0000 7fdcf580a640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fdcf015f640 con 0x7fdcdc0039e0 2024-09-21T03:44:29.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:29.060+0000 7fdcf580a640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fdcf015f4c0 con 0x7fdcf014ca00 2024-09-21T03:44:29.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:29.060+0000 7fdcf580a640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fdcf015f7c0 con 0x7fdcf0149670 2024-09-21T03:44:29.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:29.060+0000 7fdceeffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdcdc0039e0 0x7fdcf01e8fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:29.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:29.060+0000 7fdceeffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdcdc0039e0 0x7fdcf01e8fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41778/0 (socket says 172.21.15.38:41778) 2024-09-21T03:44:29.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:29.060+0000 7fdceeffd640 1 -- 172.21.15.38:0/1560920031 learned_addr learned my addr 172.21.15.38:0/1560920031 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:44:29.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:29.060+0000 7fdcee7fc640 1 --2- 172.21.15.38:0/1560920031 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdcf014ca00 0x7fdcf0149130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:29.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:29.060+0000 7fdcef7fe640 1 --2- 172.21.15.38:0/1560920031 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdcf0149670 0x7fdcf0149d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:29.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:29.060+0000 7fdceeffd640 1 -- 172.21.15.38:0/1560920031 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdcf0149670 msgr2=0x7fdcf0149d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:29.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:29.060+0000 7fdceeffd640 1 --2- 172.21.15.38:0/1560920031 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdcf0149670 0x7fdcf0149d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:29.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:29.060+0000 7fdceeffd640 1 -- 172.21.15.38:0/1560920031 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdcf014ca00 msgr2=0x7fdcf0149130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:29.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:29.060+0000 7fdceeffd640 1 --2- 172.21.15.38:0/1560920031 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdcf014ca00 0x7fdcf0149130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:29.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:29.060+0000 7fdceeffd640 1 -- 172.21.15.38:0/1560920031 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcf01e98a0 con 0x7fdcdc0039e0 2024-09-21T03:44:29.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:29.060+0000 7fdceeffd640 1 --2- 172.21.15.38:0/1560920031 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdcdc0039e0 0x7fdcf01e8fe0 secure :-1 s=READY pgs=2215 cs=0 l=1 rev1=1 crypto rx=0x7fdce0009ed0 tx=0x7fdce0033c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:29.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:29.061+0000 7fdce7fff640 1 -- 172.21.15.38:0/1560920031 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdce0043070 con 0x7fdcdc0039e0 2024-09-21T03:44:29.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:29.061+0000 7fdce7fff640 1 -- 172.21.15.38:0/1560920031 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdce0033e30 con 0x7fdcdc0039e0 2024-09-21T03:44:29.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:29.061+0000 7fdce7fff640 1 -- 172.21.15.38:0/1560920031 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdce0038040 con 0x7fdcdc0039e0 2024-09-21T03:44:29.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:29.061+0000 7fdcf580a640 1 -- 172.21.15.38:0/1560920031 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcf01e9b80 con 0x7fdcdc0039e0 2024-09-21T03:44:29.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:29.061+0000 7fdcf580a640 1 -- 172.21.15.38:0/1560920031 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcf01ea140 con 0x7fdcdc0039e0 2024-09-21T03:44:29.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:29.062+0000 7fdcf580a640 1 -- 172.21.15.38:0/1560920031 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdcf00b3250 con 0x7fdcdc0039e0 2024-09-21T03:44:29.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:29.063+0000 7fdce7fff640 1 -- 172.21.15.38:0/1560920031 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fdce0046070 con 0x7fdcdc0039e0 2024-09-21T03:44:29.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:29.063+0000 7fdce7fff640 1 --2- 172.21.15.38:0/1560920031 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fdcd0078b90 0x7fdcd007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:29.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:29.063+0000 7fdce7fff640 1 -- 172.21.15.38:0/1560920031 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 9572+0+0 (secure 0 0 0) 0x7fdce00bc610 con 0x7fdcdc0039e0 2024-09-21T03:44:29.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:29.063+0000 7fdcee7fc640 1 --2- 172.21.15.38:0/1560920031 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fdcd0078b90 0x7fdcd007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:29.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:29.064+0000 7fdcee7fc640 1 --2- 172.21.15.38:0/1560920031 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fdcd0078b90 0x7fdcd007b0a0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fdcd8000fd0 tx=0x7fdcd8001e70 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:29.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:29.068+0000 7fdce7fff640 1 -- 172.21.15.38:0/1560920031 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fdce0083490 con 0x7fdcdc0039e0 2024-09-21T03:44:29.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:29.291+0000 7fdcf580a640 1 -- 172.21.15.38:0/1560920031 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdcf00b4930 con 0x7fdcdc0039e0 2024-09-21T03:44:29.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:29.291+0000 7fdce7fff640 1 -- 172.21.15.38:0/1560920031 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 320 v320) ==== 95+0+4392 (secure 0 0 0) 0x7fdce0087ec0 con 0x7fdcdc0039e0 2024-09-21T03:44:29.294 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:44:29.294 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":320,"btime":"2024-09-21T03:44:25:830803+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6086,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/965972663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":965972663},{"type":"v1","addr":"172.21.15.195:6836","nonce":965972663}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6700,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/3176122334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":3176122334},{"type":"v1","addr":"172.21.15.195:6837","nonce":3176122334}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":9840,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/3801772046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":3801772046},{"type":"v1","addr":"172.21.15.38:6835","nonce":3801772046}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":273}],"filesystems":[{"mdsmap":{"epoch":320,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:40:02.319282+0000","modified":"2024-09-21T03:44:25.830798+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":134,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-21T03:44:29.295 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 320 2024-09-21T03:44:29.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:29.293+0000 7fdcf580a640 1 -- 172.21.15.38:0/1560920031 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fdcd0078b90 msgr2=0x7fdcd007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:29.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:29.293+0000 7fdcf580a640 1 --2- 172.21.15.38:0/1560920031 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fdcd0078b90 0x7fdcd007b0a0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fdcd8000fd0 tx=0x7fdcd8001e70 comp rx=0 tx=0).stop 2024-09-21T03:44:29.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:29.293+0000 7fdcf580a640 1 -- 172.21.15.38:0/1560920031 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdcdc0039e0 msgr2=0x7fdcf01e8fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:29.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:29.293+0000 7fdcf580a640 1 --2- 172.21.15.38:0/1560920031 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdcdc0039e0 0x7fdcf01e8fe0 secure :-1 s=READY pgs=2215 cs=0 l=1 rev1=1 crypto rx=0x7fdce0009ed0 tx=0x7fdce0033c40 comp rx=0 tx=0).stop 2024-09-21T03:44:29.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:29.293+0000 7fdcf580a640 1 -- 172.21.15.38:0/1560920031 shutdown_connections 2024-09-21T03:44:29.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:29.293+0000 7fdcf580a640 1 --2- 172.21.15.38:0/1560920031 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fdcd0078b90 0x7fdcd007b0a0 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:29.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:29.293+0000 7fdcf580a640 1 --2- 172.21.15.38:0/1560920031 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdcf0149670 0x7fdcf0149d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:29.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:29.293+0000 7fdcf580a640 1 --2- 172.21.15.38:0/1560920031 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdcf014ca00 0x7fdcf0149130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:29.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:29.293+0000 7fdcf580a640 1 --2- 172.21.15.38:0/1560920031 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdcdc0039e0 0x7fdcf01e8fe0 unknown :-1 s=CLOSED pgs=2215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:29.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:29.293+0000 7fdcf580a640 1 -- 172.21.15.38:0/1560920031 >> 172.21.15.38:0/1560920031 conn(0x7fdcf00b6b20 msgr2=0x7fdcf015d9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:29.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:29.293+0000 7fdcf580a640 1 -- 172.21.15.38:0/1560920031 shutdown_connections 2024-09-21T03:44:29.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:29.293+0000 7fdcf580a640 1 -- 172.21.15.38:0/1560920031 wait complete. 2024-09-21T03:44:29.308 INFO:tasks.cephfs.test_damage:Result: Mutation 'Corrupt omap header on 100.00000000' led to DAMAGED state 2024-09-21T03:44:29.308 INFO:tasks.cephfs.test_damage:Applying mutation 'Corrupt omap header on mds0_sessionmap' 2024-09-21T03:44:29.308 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-21T03:44:29.308 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:44:29.325 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056795,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2354 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T03:44:29.325 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi038.front.sepia.ceph.com 2024-09-21T03:44:29.325 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:44:29.325 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T03:44:29.380 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T03:44:29.381 INFO:teuthology.orchestra.run.smithi038.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-09-21T03:44:29.381 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-21T03:44:29.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:29.534+0000 7fd6260c5640 1 -- 172.21.15.38:0/2958832127 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd6200d0170 msgr2=0x7fd6200d0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:29.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:29.534+0000 7fd6260c5640 1 --2- 172.21.15.38:0/2958832127 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd6200d0170 0x7fd6200d0be0 secure :-1 s=READY pgs=2217 cs=0 l=1 rev1=1 crypto rx=0x7fd6200d2080 tx=0x7fd61402a410 comp rx=0 tx=0).stop 2024-09-21T03:44:29.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:29.534+0000 7fd6260c5640 1 -- 172.21.15.38:0/2958832127 shutdown_connections 2024-09-21T03:44:29.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:29.534+0000 7fd6260c5640 1 --2- 172.21.15.38:0/2958832127 >> v2:172.21.15.195:3300/0 conn(0x7fd6200d1120 0x7fd6200ceb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:29.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:29.534+0000 7fd6260c5640 1 --2- 172.21.15.38:0/2958832127 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd6200d0170 0x7fd6200d0be0 unknown :-1 s=CLOSED pgs=2217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:29.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:29.534+0000 7fd6260c5640 1 -- 172.21.15.38:0/2958832127 >> 172.21.15.38:0/2958832127 conn(0x7fd62009d830 msgr2=0x7fd6200b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:29.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:29.534+0000 7fd6260c5640 1 -- 172.21.15.38:0/2958832127 shutdown_connections 2024-09-21T03:44:29.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:29.534+0000 7fd6260c5640 1 -- 172.21.15.38:0/2958832127 wait complete. 2024-09-21T03:44:29.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:29.536+0000 7fd6260c5640 1 Processor -- start 2024-09-21T03:44:29.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:29.536+0000 7fd6260c5640 1 -- start start 2024-09-21T03:44:29.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:29.538+0000 7fd6260c5640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd6200d0170 0x7fd6201f5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:29.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:29.538+0000 7fd6260c5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd6200d1120 0x7fd6201587f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:29.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:29.538+0000 7fd6260c5640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd62015ee80 0x7fd62006d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:29.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:29.538+0000 7fd6260c5640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fd6200b6700 con 0x7fd6200d1120 2024-09-21T03:44:29.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:29.538+0000 7fd6260c5640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd6200b6580 con 0x7fd6200d0170 2024-09-21T03:44:29.543 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:29.538+0000 7fd6260c5640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fd6200b6880 con 0x7fd62015ee80 2024-09-21T03:44:29.543 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:29.538+0000 7fd61effd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd6200d1120 0x7fd6201587f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:29.543 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:29.538+0000 7fd61effd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd6200d1120 0x7fd6201587f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41784/0 (socket says 172.21.15.38:41784) 2024-09-21T03:44:29.543 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:29.538+0000 7fd61effd640 1 -- 172.21.15.38:0/2810563549 learned_addr learned my addr 172.21.15.38:0/2810563549 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:44:29.543 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:29.538+0000 7fd61f7fe640 1 --2- 172.21.15.38:0/2810563549 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd6200d0170 0x7fd6201f5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:29.543 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:29.538+0000 7fd61ffff640 1 --2- 172.21.15.38:0/2810563549 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd62015ee80 0x7fd62006d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:29.543 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:29.538+0000 7fd61effd640 1 -- 172.21.15.38:0/2810563549 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd62015ee80 msgr2=0x7fd62006d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:29.543 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:29.538+0000 7fd61effd640 1 --2- 172.21.15.38:0/2810563549 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd62015ee80 0x7fd62006d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:29.543 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:29.538+0000 7fd61effd640 1 -- 172.21.15.38:0/2810563549 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd6200d0170 msgr2=0x7fd6201f5470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:29.543 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:29.538+0000 7fd61effd640 1 --2- 172.21.15.38:0/2810563549 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd6200d0170 0x7fd6201f5470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:29.544 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:29.538+0000 7fd61effd640 1 -- 172.21.15.38:0/2810563549 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6201f5e50 con 0x7fd6200d1120 2024-09-21T03:44:29.544 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:29.538+0000 7fd61f7fe640 1 --2- 172.21.15.38:0/2810563549 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd6200d0170 0x7fd6201f5470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:44:29.544 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:29.538+0000 7fd61effd640 1 --2- 172.21.15.38:0/2810563549 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd6200d1120 0x7fd6201587f0 secure :-1 s=READY pgs=2218 cs=0 l=1 rev1=1 crypto rx=0x7fd614009910 tx=0x7fd614005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:29.544 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:29.539+0000 7fd607fff640 1 -- 172.21.15.38:0/2810563549 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd61403f070 con 0x7fd6200d1120 2024-09-21T03:44:29.544 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:29.539+0000 7fd607fff640 1 -- 172.21.15.38:0/2810563549 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd614004340 con 0x7fd6200d1120 2024-09-21T03:44:29.544 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:29.539+0000 7fd6260c5640 1 -- 172.21.15.38:0/2810563549 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6201f6130 con 0x7fd6200d1120 2024-09-21T03:44:29.544 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:29.539+0000 7fd607fff640 1 -- 172.21.15.38:0/2810563549 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd614031bb0 con 0x7fd6200d1120 2024-09-21T03:44:29.544 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:29.539+0000 7fd6260c5640 1 -- 172.21.15.38:0/2810563549 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6200b1f60 con 0x7fd6200d1120 2024-09-21T03:44:29.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:29.540+0000 7fd6260c5640 1 -- 172.21.15.38:0/2810563549 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6201f5bc0 con 0x7fd6200d1120 2024-09-21T03:44:29.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:29.541+0000 7fd607fff640 1 -- 172.21.15.38:0/2810563549 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fd614042070 con 0x7fd6200d1120 2024-09-21T03:44:29.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:29.541+0000 7fd607fff640 1 --2- 172.21.15.38:0/2810563549 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd60c078bc0 0x7fd60c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:29.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:29.541+0000 7fd607fff640 1 -- 172.21.15.38:0/2810563549 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 9572+0+0 (secure 0 0 0) 0x7fd6140bcc30 con 0x7fd6200d1120 2024-09-21T03:44:29.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:29.541+0000 7fd61f7fe640 1 --2- 172.21.15.38:0/2810563549 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd60c078bc0 0x7fd60c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:29.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:29.542+0000 7fd61f7fe640 1 --2- 172.21.15.38:0/2810563549 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd60c078bc0 0x7fd60c07b0d0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fd610000f60 tx=0x7fd610009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:29.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:29.546+0000 7fd607fff640 1 -- 172.21.15.38:0/2810563549 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fd614084190 con 0x7fd6200d1120 2024-09-21T03:44:29.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:29.773+0000 7fd6260c5640 1 -- 172.21.15.38:0/2810563549 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fd6200b6da0 con 0x7fd6200d1120 2024-09-21T03:44:29.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:29.890+0000 7fd607fff640 1 -- 172.21.15.38:0/2810563549 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v321) ==== 161+0+0 (secure 0 0 0) 0x7fd614031770 con 0x7fd6200d1120 2024-09-21T03:44:29.894 INFO:teuthology.orchestra.run.smithi038.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-21T03:44:29.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:29.893+0000 7fd6260c5640 1 -- 172.21.15.38:0/2810563549 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd60c078bc0 msgr2=0x7fd60c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:29.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:29.893+0000 7fd6260c5640 1 --2- 172.21.15.38:0/2810563549 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd60c078bc0 0x7fd60c07b0d0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fd610000f60 tx=0x7fd610009040 comp rx=0 tx=0).stop 2024-09-21T03:44:29.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:29.893+0000 7fd6260c5640 1 -- 172.21.15.38:0/2810563549 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd6200d1120 msgr2=0x7fd6201587f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:29.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:29.893+0000 7fd6260c5640 1 --2- 172.21.15.38:0/2810563549 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd6200d1120 0x7fd6201587f0 secure :-1 s=READY pgs=2218 cs=0 l=1 rev1=1 crypto rx=0x7fd614009910 tx=0x7fd614005270 comp rx=0 tx=0).stop 2024-09-21T03:44:29.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:29.893+0000 7fd6260c5640 1 -- 172.21.15.38:0/2810563549 shutdown_connections 2024-09-21T03:44:29.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:29.893+0000 7fd6260c5640 1 --2- 172.21.15.38:0/2810563549 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd60c078bc0 0x7fd60c07b0d0 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:29.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:29.893+0000 7fd6260c5640 1 --2- 172.21.15.38:0/2810563549 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd62015ee80 0x7fd62006d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:29.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:29.893+0000 7fd6260c5640 1 --2- 172.21.15.38:0/2810563549 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd6200d1120 0x7fd6201587f0 unknown :-1 s=CLOSED pgs=2218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:29.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:29.893+0000 7fd6260c5640 1 --2- 172.21.15.38:0/2810563549 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd6200d0170 0x7fd6201f5470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:29.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:29.893+0000 7fd6260c5640 1 -- 172.21.15.38:0/2810563549 >> 172.21.15.38:0/2810563549 conn(0x7fd62009d830 msgr2=0x7fd6200caa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:29.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:29.893+0000 7fd6260c5640 1 -- 172.21.15.38:0/2810563549 shutdown_connections 2024-09-21T03:44:29.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:29.893+0000 7fd6260c5640 1 -- 172.21.15.38:0/2810563549 wait complete. 2024-09-21T03:44:29.908 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-09-21T03:44:30.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:30.023+0000 7fc2513e4640 1 -- 172.21.15.38:0/3958329283 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc24c0ceb90 msgr2=0x7fc24c0cefa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:30.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:30.023+0000 7fc2513e4640 1 --2- 172.21.15.38:0/3958329283 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc24c0ceb90 0x7fc24c0cefa0 secure :-1 s=READY pgs=2220 cs=0 l=1 rev1=1 crypto rx=0x7fc244005fd0 tx=0x7fc24402a820 comp rx=0 tx=0).stop 2024-09-21T03:44:30.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:30.024+0000 7fc2513e4640 1 -- 172.21.15.38:0/3958329283 shutdown_connections 2024-09-21T03:44:30.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:30.024+0000 7fc2513e4640 1 --2- 172.21.15.38:0/3958329283 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc24c0ceb90 0x7fc24c0cefa0 unknown :-1 s=CLOSED pgs=2220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:30.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:30.024+0000 7fc2513e4640 1 --2- 172.21.15.38:0/3958329283 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc24c15ee80 0x7fc24c0d0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:30.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:30.024+0000 7fc2513e4640 1 -- 172.21.15.38:0/3958329283 >> 172.21.15.38:0/3958329283 conn(0x7fc24c09d830 msgr2=0x7fc24c0b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:30.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:30.024+0000 7fc2513e4640 1 -- 172.21.15.38:0/3958329283 shutdown_connections 2024-09-21T03:44:30.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:30.024+0000 7fc2513e4640 1 -- 172.21.15.38:0/3958329283 wait complete. 2024-09-21T03:44:30.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:30.025+0000 7fc2513e4640 1 Processor -- start 2024-09-21T03:44:30.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:30.025+0000 7fc2513e4640 1 -- start start 2024-09-21T03:44:30.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:30.027+0000 7fc2513e4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc24c0b9ad0 0x7fc24c0cb610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:30.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:30.027+0000 7fc2513e4640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc24c0ceb90 0x7fc24c06e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:30.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:30.027+0000 7fc2513e4640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc24c15ee80 0x7fc24c06ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:30.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:30.027+0000 7fc2513e4640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fc24c0b6700 con 0x7fc24c0b9ad0 2024-09-21T03:44:30.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:30.027+0000 7fc2513e4640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fc24c0b6580 con 0x7fc24c0ceb90 2024-09-21T03:44:30.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:30.027+0000 7fc2513e4640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fc24c0b6880 con 0x7fc24c15ee80 2024-09-21T03:44:30.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:30.027+0000 7fc24affd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc24c0b9ad0 0x7fc24c0cb610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:30.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:30.027+0000 7fc24affd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc24c0b9ad0 0x7fc24c0cb610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41790/0 (socket says 172.21.15.38:41790) 2024-09-21T03:44:30.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:30.027+0000 7fc24affd640 1 -- 172.21.15.38:0/1273329200 learned_addr learned my addr 172.21.15.38:0/1273329200 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:44:30.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:30.027+0000 7fc243fff640 1 --2- 172.21.15.38:0/1273329200 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc24c0ceb90 0x7fc24c06e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:30.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:30.027+0000 7fc24b7fe640 1 --2- 172.21.15.38:0/1273329200 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc24c15ee80 0x7fc24c06ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:30.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:30.027+0000 7fc24affd640 1 -- 172.21.15.38:0/1273329200 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc24c15ee80 msgr2=0x7fc24c06ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:30.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:30.027+0000 7fc24affd640 1 --2- 172.21.15.38:0/1273329200 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc24c15ee80 0x7fc24c06ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:30.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:30.027+0000 7fc24affd640 1 -- 172.21.15.38:0/1273329200 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc24c0ceb90 msgr2=0x7fc24c06e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:30.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:30.027+0000 7fc24affd640 1 --2- 172.21.15.38:0/1273329200 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc24c0ceb90 0x7fc24c06e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:30.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:30.027+0000 7fc24affd640 1 -- 172.21.15.38:0/1273329200 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc24c0cc020 con 0x7fc24c0b9ad0 2024-09-21T03:44:30.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:30.027+0000 7fc243fff640 1 --2- 172.21.15.38:0/1273329200 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc24c0ceb90 0x7fc24c06e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:44:30.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:30.027+0000 7fc24affd640 1 --2- 172.21.15.38:0/1273329200 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc24c0b9ad0 0x7fc24c0cb610 secure :-1 s=READY pgs=2221 cs=0 l=1 rev1=1 crypto rx=0x7fc23c00d740 tx=0x7fc23c00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:30.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:30.028+0000 7fc2437fe640 1 -- 172.21.15.38:0/1273329200 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc23c017070 con 0x7fc24c0b9ad0 2024-09-21T03:44:30.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:30.028+0000 7fc2437fe640 1 -- 172.21.15.38:0/1273329200 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc23c009ab0 con 0x7fc24c0b9ad0 2024-09-21T03:44:30.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:30.028+0000 7fc2437fe640 1 -- 172.21.15.38:0/1273329200 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc23c010460 con 0x7fc24c0b9ad0 2024-09-21T03:44:30.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:30.028+0000 7fc2513e4640 1 -- 172.21.15.38:0/1273329200 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc24c0cc300 con 0x7fc24c0b9ad0 2024-09-21T03:44:30.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:30.028+0000 7fc2513e4640 1 -- 172.21.15.38:0/1273329200 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc24c0ccec0 con 0x7fc24c0b9ad0 2024-09-21T03:44:30.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:30.029+0000 7fc2513e4640 1 -- 172.21.15.38:0/1273329200 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc24c0b6d40 con 0x7fc24c0b9ad0 2024-09-21T03:44:30.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:30.030+0000 7fc2437fe640 1 -- 172.21.15.38:0/1273329200 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc23c01a070 con 0x7fc24c0b9ad0 2024-09-21T03:44:30.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:30.030+0000 7fc2437fe640 1 --2- 172.21.15.38:0/1273329200 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc22c078bc0 0x7fc22c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:30.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:30.030+0000 7fc2437fe640 1 -- 172.21.15.38:0/1273329200 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 9572+0+0 (secure 0 0 0) 0x7fc23c09bd70 con 0x7fc24c0b9ad0 2024-09-21T03:44:30.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:30.030+0000 7fc243fff640 1 --2- 172.21.15.38:0/1273329200 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc22c078bc0 0x7fc22c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:30.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:30.031+0000 7fc243fff640 1 --2- 172.21.15.38:0/1273329200 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc22c078bc0 0x7fc22c07b0d0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fc234007fd0 tx=0x7fc234009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:30.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:30.035+0000 7fc2437fe640 1 -- 172.21.15.38:0/1273329200 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fc23c063240 con 0x7fc24c0b9ad0 2024-09-21T03:44:30.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:30.300+0000 7fc2513e4640 1 -- 172.21.15.38:0/1273329200 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7fc24c0b4060 con 0x7fc24c0b9ad0 2024-09-21T03:44:30.582 INFO:tasks.ceph.mds.c.smithi038.stderr: -58> 2024-09-21T03:44:25.827+0000 7f39c1b6f640 -1 mds.0.cache.dir(0x100) Corrupt fnode in dirfrag 0x100: Decoder at 'void fnode_t::decode(ceph::buffer::v15_2_0::list::const_iterator&)' v=3 cannot decode v=100 minimal_decoder=101: Malformed input [buffer:3] 2024-09-21T03:44:30.582 INFO:tasks.ceph.mds.c.smithi038.stderr: -33> 2024-09-21T03:44:26.803+0000 7f39c5b77640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T03:44:30.638 INFO:tasks.ceph.mds.c.smithi038.stdout:starting mds.c at 2024-09-21T03:44:30.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:30.898+0000 7fc2437fe640 1 -- 172.21.15.38:0/1273329200 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v322) ==== 101+0+0 (secure 0 0 0) 0x7fc23c062b40 con 0x7fc24c0b9ad0 2024-09-21T03:44:30.901 INFO:teuthology.orchestra.run.smithi038.stderr:repaired: restoring rank 6:0 2024-09-21T03:44:30.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:30.900+0000 7fc2513e4640 1 -- 172.21.15.38:0/1273329200 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc22c078bc0 msgr2=0x7fc22c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:30.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:30.900+0000 7fc2513e4640 1 --2- 172.21.15.38:0/1273329200 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc22c078bc0 0x7fc22c07b0d0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fc234007fd0 tx=0x7fc234009040 comp rx=0 tx=0).stop 2024-09-21T03:44:30.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:30.900+0000 7fc2513e4640 1 -- 172.21.15.38:0/1273329200 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc24c0b9ad0 msgr2=0x7fc24c0cb610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:30.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:30.900+0000 7fc2513e4640 1 --2- 172.21.15.38:0/1273329200 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc24c0b9ad0 0x7fc24c0cb610 secure :-1 s=READY pgs=2221 cs=0 l=1 rev1=1 crypto rx=0x7fc23c00d740 tx=0x7fc23c00dc00 comp rx=0 tx=0).stop 2024-09-21T03:44:30.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:30.900+0000 7fc2513e4640 1 -- 172.21.15.38:0/1273329200 shutdown_connections 2024-09-21T03:44:30.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:30.900+0000 7fc2513e4640 1 --2- 172.21.15.38:0/1273329200 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc22c078bc0 0x7fc22c07b0d0 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:30.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:30.900+0000 7fc2513e4640 1 --2- 172.21.15.38:0/1273329200 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc24c15ee80 0x7fc24c06ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:30.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:30.900+0000 7fc2513e4640 1 --2- 172.21.15.38:0/1273329200 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc24c0ceb90 0x7fc24c06e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:30.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:30.901+0000 7fc2513e4640 1 --2- 172.21.15.38:0/1273329200 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc24c0b9ad0 0x7fc24c0cb610 unknown :-1 s=CLOSED pgs=2221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:30.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:30.901+0000 7fc2513e4640 1 -- 172.21.15.38:0/1273329200 >> 172.21.15.38:0/1273329200 conn(0x7fc24c09d830 msgr2=0x7fc24c0b31a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:30.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:30.901+0000 7fc2513e4640 1 -- 172.21.15.38:0/1273329200 shutdown_connections 2024-09-21T03:44:30.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:30.901+0000 7fc2513e4640 1 -- 172.21.15.38:0/1273329200 wait complete. 2024-09-21T03:44:30.917 DEBUG:teuthology.orchestra.run.smithi038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-09-21T03:44:30.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:30.970+0000 7fb018d2cf40 1 Processor -- start 2024-09-21T03:44:30.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:30.970+0000 7fb018d2cf40 1 -- start start 2024-09-21T03:44:30.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:30.971+0000 7fb018d2cf40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560e2d757240 0x560e2d79c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:30.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:30.971+0000 7fb018d2cf40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560e2d79e090 0x560e2d80dc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:30.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:30.971+0000 7fb018d2cf40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560e2d813520 0x560e2d80e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:30.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:30.971+0000 7fb018d2cf40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x560e2d6a4d60 con 0x560e2d813520 2024-09-21T03:44:30.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:30.971+0000 7fb018d2cf40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x560e2d703430 con 0x560e2d79e090 2024-09-21T03:44:30.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:30.971+0000 7fb018d2cf40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x560e2d82a820 con 0x560e2d757240 2024-09-21T03:44:30.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:30.971+0000 7fb012582640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560e2d813520 0x560e2d80e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:30.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:30.972+0000 7fb012582640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560e2d813520 0x560e2d80e1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41832/0 (socket says 172.21.15.38:41832) 2024-09-21T03:44:30.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:30.972+0000 7fb012582640 1 -- 172.21.15.38:0/4057575008 learned_addr learned my addr 172.21.15.38:0/4057575008 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:44:30.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:30.972+0000 7fb011580640 1 --2- 172.21.15.38:0/4057575008 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560e2d79e090 0x560e2d80dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:30.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:30.972+0000 7fb011d81640 1 --2- 172.21.15.38:0/4057575008 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560e2d757240 0x560e2d79c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:30.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:30.972+0000 7fb012582640 1 -- 172.21.15.38:0/4057575008 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560e2d757240 msgr2=0x560e2d79c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:44:30.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:30.972+0000 7fb012582640 1 --2- 172.21.15.38:0/4057575008 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560e2d757240 0x560e2d79c950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:30.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:30.972+0000 7fb012582640 1 -- 172.21.15.38:0/4057575008 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560e2d79e090 msgr2=0x560e2d80dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:44:30.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:30.972+0000 7fb012582640 1 --2- 172.21.15.38:0/4057575008 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560e2d79e090 0x560e2d80dc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:30.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:30.972+0000 7fb012582640 1 -- 172.21.15.38:0/4057575008 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e2d8129c0 con 0x560e2d813520 2024-09-21T03:44:30.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:30.972+0000 7fb012582640 1 --2- 172.21.15.38:0/4057575008 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560e2d813520 0x560e2d80e1b0 secure :-1 s=READY pgs=2225 cs=0 l=1 rev1=1 crypto rx=0x7fb00c00a4e0 tx=0x7fb00c00a9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a193e706e3fbce4e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:30.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:30.972+0000 7fb003fff640 1 -- 172.21.15.38:0/4057575008 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb00c015070 con 0x560e2d813520 2024-09-21T03:44:30.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:30.972+0000 7fb003fff640 1 -- 172.21.15.38:0/4057575008 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb00c00e730 con 0x560e2d813520 2024-09-21T03:44:30.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:30.972+0000 7fb003fff640 1 -- 172.21.15.38:0/4057575008 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb00c00eb80 con 0x560e2d813520 2024-09-21T03:44:30.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:30.973+0000 7fb018d2cf40 1 -- 172.21.15.38:0/4057575008 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560e2d813520 msgr2=0x560e2d80e1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:30.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:30.973+0000 7fb018d2cf40 1 --2- 172.21.15.38:0/4057575008 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560e2d813520 0x560e2d80e1b0 secure :-1 s=READY pgs=2225 cs=0 l=1 rev1=1 crypto rx=0x7fb00c00a4e0 tx=0x7fb00c00a9a0 comp rx=0 tx=0).stop 2024-09-21T03:44:30.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:30.973+0000 7fb018d2cf40 1 -- 172.21.15.38:0/4057575008 shutdown_connections 2024-09-21T03:44:30.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:30.973+0000 7fb018d2cf40 1 --2- 172.21.15.38:0/4057575008 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560e2d813520 0x560e2d80e1b0 unknown :-1 s=CLOSED pgs=2225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:30.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:30.973+0000 7fb018d2cf40 1 --2- 172.21.15.38:0/4057575008 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560e2d79e090 0x560e2d80dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:30.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:30.973+0000 7fb018d2cf40 1 --2- 172.21.15.38:0/4057575008 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560e2d757240 0x560e2d79c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:30.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:30.973+0000 7fb018d2cf40 1 -- 172.21.15.38:0/4057575008 >> 172.21.15.38:0/4057575008 conn(0x560e2d675ac0 msgr2=0x560e2d823fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:30.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:30.973+0000 7fb018d2cf40 1 -- 172.21.15.38:0/4057575008 shutdown_connections 2024-09-21T03:44:30.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:30.973+0000 7fb018d2cf40 1 -- 172.21.15.38:0/4057575008 wait complete. 2024-09-21T03:44:30.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:30.974+0000 7fb018d2cf40 1 Processor -- start 2024-09-21T03:44:30.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:30.974+0000 7fb018d2cf40 1 -- start start 2024-09-21T03:44:30.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:30.975+0000 7fb018d2cf40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560e2d757240 0x560e2d823c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:30.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:30.975+0000 7fb018d2cf40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560e2d79e090 0x560e2d80b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:30.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:30.975+0000 7fb018d2cf40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560e2d813520 0x560e2d80aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:30.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:30.975+0000 7fb018d2cf40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x560e2d811b80 con 0x560e2d813520 2024-09-21T03:44:30.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:30.975+0000 7fb018d2cf40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x560e2d8118a0 con 0x560e2d757240 2024-09-21T03:44:30.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:30.975+0000 7fb018d2cf40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x560e2d80be30 con 0x560e2d79e090 2024-09-21T03:44:30.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:30.975+0000 7fb012582640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560e2d813520 0x560e2d80aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:30.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:30.975+0000 7fb012582640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560e2d813520 0x560e2d80aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41834/0 (socket says 172.21.15.38:41834) 2024-09-21T03:44:30.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:30.975+0000 7fb012582640 1 -- 172.21.15.38:0/4115168567 learned_addr learned my addr 172.21.15.38:0/4115168567 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:44:30.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:30.975+0000 7fb011d81640 1 --2- 172.21.15.38:0/4115168567 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560e2d757240 0x560e2d823c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:30.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:30.975+0000 7fb011580640 1 --2- 172.21.15.38:0/4115168567 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560e2d79e090 0x560e2d80b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:30.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:30.975+0000 7fb012582640 1 -- 172.21.15.38:0/4115168567 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560e2d79e090 msgr2=0x560e2d80b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:30.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:30.975+0000 7fb012582640 1 --2- 172.21.15.38:0/4115168567 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560e2d79e090 0x560e2d80b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:30.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:30.976+0000 7fb012582640 1 -- 172.21.15.38:0/4115168567 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560e2d757240 msgr2=0x560e2d823c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:30.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:30.976+0000 7fb012582640 1 --2- 172.21.15.38:0/4115168567 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560e2d757240 0x560e2d823c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:30.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:30.976+0000 7fb012582640 1 -- 172.21.15.38:0/4115168567 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e2d79c9f0 con 0x560e2d813520 2024-09-21T03:44:30.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:30.976+0000 7fb011580640 1 --2- 172.21.15.38:0/4115168567 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560e2d79e090 0x560e2d80b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:44:30.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:30.976+0000 7fb012582640 1 --2- 172.21.15.38:0/4115168567 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560e2d813520 0x560e2d80aa30 secure :-1 s=READY pgs=2226 cs=0 l=1 rev1=1 crypto rx=0x7fb00c00aea0 tx=0x7fb00c005750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:30.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:30.976+0000 7fb002ffd640 1 -- 172.21.15.38:0/4115168567 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb00c020020 con 0x560e2d813520 2024-09-21T03:44:30.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:30.976+0000 7fb002ffd640 1 -- 172.21.15.38:0/4115168567 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb00c016040 con 0x560e2d813520 2024-09-21T03:44:30.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:30.976+0000 7fb018d2cf40 1 -- 172.21.15.38:0/4115168567 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x560e2d79ccd0 con 0x560e2d813520 2024-09-21T03:44:30.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:30.976+0000 7fb002ffd640 1 -- 172.21.15.38:0/4115168567 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb00c015070 con 0x560e2d813520 2024-09-21T03:44:30.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:30.976+0000 7fb018d2cf40 1 -- 172.21.15.38:0/4115168567 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e2d80bb70 con 0x560e2d813520 2024-09-21T03:44:30.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:30.978+0000 7fb002ffd640 1 -- 172.21.15.38:0/4115168567 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb00c023020 con 0x560e2d813520 2024-09-21T03:44:30.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:30.978+0000 7fb002ffd640 1 --2- 172.21.15.38:0/4115168567 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7faff40775a0 0x7faff4079ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:30.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:30.978+0000 7fb002ffd640 1 -- 172.21.15.38:0/4115168567 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 9572+0+0 (secure 0 0 0) 0x7fb00c09bd90 con 0x560e2d813520 2024-09-21T03:44:30.985 INFO:teuthology.orchestra.run.smithi038.stdout:Importing pool 2024-09-21T03:44:30.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:30.978+0000 7fb011d81640 1 --2- 172.21.15.38:0/4115168567 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7faff40775a0 0x7faff4079ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:30.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:30.978+0000 7fb018d2cf40 1 --2- 172.21.15.38:0/4115168567 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x560e2d79b7e0 0x560e2d824e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:30.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:30.978+0000 7fb018d2cf40 1 -- 172.21.15.38:0/4115168567 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x560e2d8cd930 con 0x560e2d79b7e0 2024-09-21T03:44:30.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:30.979+0000 7fb011580640 1 --2- 172.21.15.38:0/4115168567 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x560e2d79b7e0 0x560e2d824e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:30.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:30.979+0000 7fb011d81640 1 --2- 172.21.15.38:0/4115168567 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7faff40775a0 0x7faff4079ab0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fb00400b680 tx=0x7fb00400d040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:30.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:30.979+0000 7fb011580640 1 --2- 172.21.15.38:0/4115168567 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x560e2d79b7e0 0x560e2d824e40 crc :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:30.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:30.982+0000 7fb011580640 1 -- 172.21.15.38:0/4115168567 <== osd.7 v2:172.21.15.195:6818/1606106342 1 ==== osd_op_reply(1 601.00000000 [create] v134'1159 uv1133 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb008054370 con 0x560e2d79b7e0 2024-09-21T03:44:30.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:30.982+0000 7fb018d2cf40 1 -- 172.21.15.38:0/4115168567 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e134) -- 0x560e2d8cdcd0 con 0x560e2d79b7e0 2024-09-21T03:44:30.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:30.984+0000 7fb011580640 1 -- 172.21.15.38:0/4115168567 <== osd.7 v2:172.21.15.195:6818/1606106342 2 ==== osd_op_reply(2 601.00000000 [delete] v134'1160 uv1160 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb008054370 con 0x560e2d79b7e0 2024-09-21T03:44:30.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:30.984+0000 7fb018d2cf40 1 -- 172.21.15.38:0/4115168567 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x560e2d8ce070 con 0x560e2d79b7e0 2024-09-21T03:44:30.988 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-09-21T03:44:30.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:30.985+0000 7fb011580640 1 -- 172.21.15.38:0/4115168567 <== osd.7 v2:172.21.15.195:6818/1606106342 3 ==== osd_op_reply(3 601.00000000 [create] v134'1161 uv1161 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb008054370 con 0x560e2d79b7e0 2024-09-21T03:44:30.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:30.985+0000 7fb018d2cf40 1 -- 172.21.15.38:0/4115168567 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x560e2d8cf420 con 0x560e2d79b7e0 2024-09-21T03:44:30.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:30.987+0000 7fb011580640 1 -- 172.21.15.38:0/4115168567 <== osd.7 v2:172.21.15.195:6818/1606106342 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v134'1162 uv1162 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb008054370 con 0x560e2d79b7e0 2024-09-21T03:44:30.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:30.987+0000 7fb018d2cf40 1 -- 172.21.15.38:0/4115168567 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x560e2d8cf7c0 con 0x560e2d79b7e0 2024-09-21T03:44:30.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:30.988+0000 7fb011580640 1 -- 172.21.15.38:0/4115168567 <== osd.7 v2:172.21.15.195:6818/1606106342 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v134'1163 uv1163 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb008054370 con 0x560e2d79b7e0 2024-09-21T03:44:30.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:30.988+0000 7fb018d2cf40 1 -- 172.21.15.38:0/4115168567 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x560e2d8ce410 con 0x560e2d79b7e0 2024-09-21T03:44:30.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:30.990+0000 7fb011580640 1 -- 172.21.15.38:0/4115168567 <== osd.7 v2:172.21.15.195:6818/1606106342 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v134'1164 uv1164 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb008054370 con 0x560e2d79b7e0 2024-09-21T03:44:30.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:30.990+0000 7fb018d2cf40 1 -- 172.21.15.38:0/4115168567 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x560e2d8ce800 con 0x560e2d79b7e0 2024-09-21T03:44:30.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:30.991+0000 7fb011580640 1 -- 172.21.15.38:0/4115168567 <== osd.7 v2:172.21.15.195:6818/1606106342 7 ==== osd_op_reply(7 602.00000000 [create] v134'1165 uv1139 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb008054370 con 0x560e2d79b7e0 2024-09-21T03:44:30.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:30.992+0000 7fb018d2cf40 1 -- 172.21.15.38:0/4115168567 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e134) -- 0x560e2d8ce800 con 0x560e2d79b7e0 2024-09-21T03:44:30.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:30.993+0000 7fb011580640 1 -- 172.21.15.38:0/4115168567 <== osd.7 v2:172.21.15.195:6818/1606106342 8 ==== osd_op_reply(8 602.00000000 [delete] v134'1166 uv1166 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb008054370 con 0x560e2d79b7e0 2024-09-21T03:44:30.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:30.993+0000 7fb018d2cf40 1 -- 172.21.15.38:0/4115168567 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x560e2d8ce7b0 con 0x560e2d79b7e0 2024-09-21T03:44:30.997 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-09-21T03:44:30.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:30.995+0000 7fb011580640 1 -- 172.21.15.38:0/4115168567 <== osd.7 v2:172.21.15.195:6818/1606106342 9 ==== osd_op_reply(9 602.00000000 [create] v134'1167 uv1167 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb008054370 con 0x560e2d79b7e0 2024-09-21T03:44:30.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:30.995+0000 7fb018d2cf40 1 -- 172.21.15.38:0/4115168567 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x560e2d8ce7b0 con 0x560e2d79b7e0 2024-09-21T03:44:30.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:30.996+0000 7fb011580640 1 -- 172.21.15.38:0/4115168567 <== osd.7 v2:172.21.15.195:6818/1606106342 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v134'1168 uv1168 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb008054370 con 0x560e2d79b7e0 2024-09-21T03:44:30.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:30.996+0000 7fb018d2cf40 1 -- 172.21.15.38:0/4115168567 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x560e2d8ce7b0 con 0x560e2d79b7e0 2024-09-21T03:44:31.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:30.998+0000 7fb011580640 1 -- 172.21.15.38:0/4115168567 <== osd.7 v2:172.21.15.195:6818/1606106342 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v134'1169 uv1169 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb008054370 con 0x560e2d79b7e0 2024-09-21T03:44:31.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:30.998+0000 7fb018d2cf40 1 -- 172.21.15.38:0/4115168567 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x560e2d8ce7b0 con 0x560e2d79b7e0 2024-09-21T03:44:31.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:30.999+0000 7fb011580640 1 -- 172.21.15.38:0/4115168567 <== osd.7 v2:172.21.15.195:6818/1606106342 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v134'1170 uv1170 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb008054370 con 0x560e2d79b7e0 2024-09-21T03:44:31.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.000+0000 7fb018d2cf40 1 -- 172.21.15.38:0/4115168567 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:13 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x560e2d8ce7b0 con 0x560e2d79b7e0 2024-09-21T03:44:31.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.002+0000 7fb011580640 1 -- 172.21.15.38:0/4115168567 <== osd.7 v2:172.21.15.195:6818/1606106342 13 ==== osd_op_reply(13 10000000000.00000000 [create] v134'1171 uv1146 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7fb008054370 con 0x560e2d79b7e0 2024-09-21T03:44:31.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.002+0000 7fb018d2cf40 1 -- 172.21.15.38:0/4115168567 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:14 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e134) -- 0x560e2d8ce7b0 con 0x560e2d79b7e0 2024-09-21T03:44:31.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.003+0000 7fb011580640 1 -- 172.21.15.38:0/4115168567 <== osd.7 v2:172.21.15.195:6818/1606106342 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v134'1172 uv1172 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fb008054370 con 0x560e2d79b7e0 2024-09-21T03:44:31.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.003+0000 7fb018d2cf40 1 -- 172.21.15.38:0/4115168567 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:15 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x560e2d8ce7b0 con 0x560e2d79b7e0 2024-09-21T03:44:31.007 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-09-21T03:44:31.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.004+0000 7fb011580640 1 -- 172.21.15.38:0/4115168567 <== osd.7 v2:172.21.15.195:6818/1606106342 15 ==== osd_op_reply(15 10000000000.00000000 [create] v134'1173 uv1173 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fb008054370 con 0x560e2d79b7e0 2024-09-21T03:44:31.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.004+0000 7fb018d2cf40 1 -- 172.21.15.38:0/4115168567 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:16 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x560e2d8ce7b0 con 0x560e2d79b7e0 2024-09-21T03:44:31.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.007+0000 7fb011580640 1 -- 172.21.15.38:0/4115168567 <== osd.7 v2:172.21.15.195:6818/1606106342 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v134'1174 uv1174 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fb008054370 con 0x560e2d79b7e0 2024-09-21T03:44:31.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.007+0000 7fb018d2cf40 1 -- 172.21.15.38:0/4115168567 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:17 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x560e2d8ce7b0 con 0x560e2d79b7e0 2024-09-21T03:44:31.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.008+0000 7fb011580640 1 -- 172.21.15.38:0/4115168567 <== osd.7 v2:172.21.15.195:6818/1606106342 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v134'1175 uv1175 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fb008054370 con 0x560e2d79b7e0 2024-09-21T03:44:31.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.008+0000 7fb018d2cf40 1 -- 172.21.15.38:0/4115168567 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:18 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x560e2d8ce7b0 con 0x560e2d79b7e0 2024-09-21T03:44:31.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.009+0000 7fb011580640 1 -- 172.21.15.38:0/4115168567 <== osd.7 v2:172.21.15.195:6818/1606106342 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v134'1176 uv1176 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fb008054370 con 0x560e2d79b7e0 2024-09-21T03:44:31.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.009+0000 7fb018d2cf40 1 -- 172.21.15.38:0/4115168567 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:19 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=585b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x560e2d8ce7b0 con 0x560e2d79b7e0 2024-09-21T03:44:31.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.011+0000 7fb011580640 1 -- 172.21.15.38:0/4115168567 <== osd.7 v2:172.21.15.195:6818/1606106342 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v134'1177 uv1177 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fb008054370 con 0x560e2d79b7e0 2024-09-21T03:44:31.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.011+0000 7fb018d2cf40 1 -- 172.21.15.38:0/4115168567 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:20 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x560e2d8ce7b0 con 0x560e2d79b7e0 2024-09-21T03:44:31.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.012+0000 7fb011580640 1 -- 172.21.15.38:0/4115168567 <== osd.7 v2:172.21.15.195:6818/1606106342 20 ==== osd_op_reply(20 600.00000000 [create] v134'1178 uv1152 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb008054370 con 0x560e2d79b7e0 2024-09-21T03:44:31.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.012+0000 7fb018d2cf40 1 -- 172.21.15.38:0/4115168567 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:21 13.10 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e134) -- 0x560e2d8ce7b0 con 0x560e2d79b7e0 2024-09-21T03:44:31.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.014+0000 7fb011580640 1 -- 172.21.15.38:0/4115168567 <== osd.7 v2:172.21.15.195:6818/1606106342 21 ==== osd_op_reply(21 600.00000000 [delete] v134'1179 uv1179 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb008054370 con 0x560e2d79b7e0 2024-09-21T03:44:31.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.014+0000 7fb018d2cf40 1 -- 172.21.15.38:0/4115168567 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:22 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x560e2d8ce7b0 con 0x560e2d79b7e0 2024-09-21T03:44:31.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.015+0000 7fb011580640 1 -- 172.21.15.38:0/4115168567 <== osd.7 v2:172.21.15.195:6818/1606106342 22 ==== osd_op_reply(22 600.00000000 [create] v134'1180 uv1180 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb008054370 con 0x560e2d79b7e0 2024-09-21T03:44:31.018 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-09-21T03:44:31.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.015+0000 7fb018d2cf40 1 -- 172.21.15.38:0/4115168567 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:23 13.10 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x560e2d8ce7b0 con 0x560e2d79b7e0 2024-09-21T03:44:31.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.017+0000 7fb011580640 1 -- 172.21.15.38:0/4115168567 <== osd.7 v2:172.21.15.195:6818/1606106342 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v134'1181 uv1181 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb008054370 con 0x560e2d79b7e0 2024-09-21T03:44:31.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.017+0000 7fb018d2cf40 1 -- 172.21.15.38:0/4115168567 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:24 13.10 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x560e2d8ce7b0 con 0x560e2d79b7e0 2024-09-21T03:44:31.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.018+0000 7fb011580640 1 -- 172.21.15.38:0/4115168567 <== osd.7 v2:172.21.15.195:6818/1606106342 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v134'1182 uv1182 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb008054370 con 0x560e2d79b7e0 2024-09-21T03:44:31.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.019+0000 7fb018d2cf40 1 -- 172.21.15.38:0/4115168567 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:25 13.10 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x560e2d8ce7b0 con 0x560e2d79b7e0 2024-09-21T03:44:31.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.020+0000 7fb011580640 1 -- 172.21.15.38:0/4115168567 <== osd.7 v2:172.21.15.195:6818/1606106342 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v134'1183 uv1183 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb008054370 con 0x560e2d79b7e0 2024-09-21T03:44:31.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.020+0000 7fb018d2cf40 1 -- 172.21.15.38:0/4115168567 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:26 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x560e2d8ce7b0 con 0x560e2d79b7e0 2024-09-21T03:44:31.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.023+0000 7fb011580640 1 -- 172.21.15.38:0/4115168567 <== osd.7 v2:172.21.15.195:6818/1606106342 26 ==== osd_op_reply(26 603.00000000 [create] v134'1184 uv1158 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb008054370 con 0x560e2d79b7e0 2024-09-21T03:44:31.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.023+0000 7fb018d2cf40 1 -- 172.21.15.38:0/4115168567 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:27 13.10 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e134) -- 0x560e2d8ce7b0 con 0x560e2d79b7e0 2024-09-21T03:44:31.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.023+0000 7fb011580640 1 -- 172.21.15.38:0/4115168567 <== osd.7 v2:172.21.15.195:6818/1606106342 27 ==== osd_op_reply(27 603.00000000 [delete] v134'1185 uv1185 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb008054370 con 0x560e2d79b7e0 2024-09-21T03:44:31.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.024+0000 7fb018d2cf40 1 -- 172.21.15.38:0/4115168567 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:28 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x560e2d8ce7b0 con 0x560e2d79b7e0 2024-09-21T03:44:31.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.025+0000 7fb011580640 1 -- 172.21.15.38:0/4115168567 <== osd.7 v2:172.21.15.195:6818/1606106342 28 ==== osd_op_reply(28 603.00000000 [create] v134'1186 uv1186 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb008054370 con 0x560e2d79b7e0 2024-09-21T03:44:31.028 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-09-21T03:44:31.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.025+0000 7fb018d2cf40 1 -- 172.21.15.38:0/4115168567 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:29 13.10 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x560e2d8ce7b0 con 0x560e2d79b7e0 2024-09-21T03:44:31.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.027+0000 7fb011580640 1 -- 172.21.15.38:0/4115168567 <== osd.7 v2:172.21.15.195:6818/1606106342 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v134'1187 uv1187 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb008054370 con 0x560e2d79b7e0 2024-09-21T03:44:31.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.027+0000 7fb018d2cf40 1 -- 172.21.15.38:0/4115168567 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:30 13.10 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x560e2d8ce7b0 con 0x560e2d79b7e0 2024-09-21T03:44:31.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.028+0000 7fb011580640 1 -- 172.21.15.38:0/4115168567 <== osd.7 v2:172.21.15.195:6818/1606106342 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v134'1188 uv1188 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb008054370 con 0x560e2d79b7e0 2024-09-21T03:44:31.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.028+0000 7fb018d2cf40 1 -- 172.21.15.38:0/4115168567 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:31 13.10 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x560e2d8ce7b0 con 0x560e2d79b7e0 2024-09-21T03:44:31.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.030+0000 7fb011580640 1 -- 172.21.15.38:0/4115168567 <== osd.7 v2:172.21.15.195:6818/1606106342 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v134'1189 uv1189 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb008054370 con 0x560e2d79b7e0 2024-09-21T03:44:31.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.030+0000 7fb018d2cf40 1 --2- 172.21.15.38:0/4115168567 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x560e2d8cfb60 0x560e2d8d1f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:31.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.030+0000 7fb018d2cf40 1 -- 172.21.15.38:0/4115168567 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x560e2d8ce7b0 con 0x560e2d8cfb60 2024-09-21T03:44:31.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.030+0000 7fb011d81640 1 --2- 172.21.15.38:0/4115168567 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x560e2d8cfb60 0x560e2d8d1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:31.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.031+0000 7fb011d81640 1 --2- 172.21.15.38:0/4115168567 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x560e2d8cfb60 0x560e2d8d1f80 crc :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:31.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.032+0000 7fb011d81640 1 -- 172.21.15.38:0/4115168567 <== osd.6 v2:172.21.15.195:6826/4055825675 1 ==== osd_op_reply(32 1.00000000.inode [create] v134'845 uv827 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7fb004009b10 con 0x560e2d8cfb60 2024-09-21T03:44:31.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.032+0000 7fb018d2cf40 1 -- 172.21.15.38:0/4115168567 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e134) -- 0x560e2d8d24c0 con 0x560e2d8cfb60 2024-09-21T03:44:31.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.034+0000 7fb011d81640 1 -- 172.21.15.38:0/4115168567 <== osd.6 v2:172.21.15.195:6826/4055825675 2 ==== osd_op_reply(33 1.00000000.inode [delete] v134'846 uv846 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fb004009b10 con 0x560e2d8cfb60 2024-09-21T03:44:31.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.034+0000 7fb018d2cf40 1 -- 172.21.15.38:0/4115168567 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x560e2d8d2860 con 0x560e2d8cfb60 2024-09-21T03:44:31.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.035+0000 7fb011d81640 1 -- 172.21.15.38:0/4115168567 <== osd.6 v2:172.21.15.195:6826/4055825675 3 ==== osd_op_reply(34 1.00000000.inode [create] v134'847 uv847 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fb004009b10 con 0x560e2d8cfb60 2024-09-21T03:44:31.038 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-09-21T03:44:31.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.035+0000 7fb018d2cf40 1 -- 172.21.15.38:0/4115168567 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x560e2d8d2c00 con 0x560e2d8cfb60 2024-09-21T03:44:31.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.037+0000 7fb011d81640 1 -- 172.21.15.38:0/4115168567 <== osd.6 v2:172.21.15.195:6826/4055825675 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v134'848 uv848 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fb004009b10 con 0x560e2d8cfb60 2024-09-21T03:44:31.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.037+0000 7fb018d2cf40 1 -- 172.21.15.38:0/4115168567 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x560e2d8d2fa0 con 0x560e2d8cfb60 2024-09-21T03:44:31.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.038+0000 7fb011d81640 1 -- 172.21.15.38:0/4115168567 <== osd.6 v2:172.21.15.195:6826/4055825675 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v134'849 uv849 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fb004009b10 con 0x560e2d8cfb60 2024-09-21T03:44:31.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.038+0000 7fb018d2cf40 1 -- 172.21.15.38:0/4115168567 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x560e2d8d3340 con 0x560e2d8cfb60 2024-09-21T03:44:31.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.039+0000 7fb011d81640 1 -- 172.21.15.38:0/4115168567 <== osd.6 v2:172.21.15.195:6826/4055825675 6 ==== osd_op_reply(37 200.00000000 [create] v134'850 uv832 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb004009b10 con 0x560e2d8cfb60 2024-09-21T03:44:31.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.039+0000 7fb018d2cf40 1 -- 172.21.15.38:0/4115168567 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e134) -- 0x560e2d8d36e0 con 0x560e2d8cfb60 2024-09-21T03:44:31.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.041+0000 7fb011d81640 1 -- 172.21.15.38:0/4115168567 <== osd.6 v2:172.21.15.195:6826/4055825675 7 ==== osd_op_reply(38 200.00000000 [delete] v134'851 uv851 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb004009b10 con 0x560e2d8cfb60 2024-09-21T03:44:31.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.041+0000 7fb018d2cf40 1 -- 172.21.15.38:0/4115168567 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x560e2d8d36e0 con 0x560e2d8cfb60 2024-09-21T03:44:31.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.042+0000 7fb011d81640 1 -- 172.21.15.38:0/4115168567 <== osd.6 v2:172.21.15.195:6826/4055825675 8 ==== osd_op_reply(39 200.00000000 [create] v134'852 uv852 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb004009b10 con 0x560e2d8cfb60 2024-09-21T03:44:31.045 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-09-21T03:44:31.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.042+0000 7fb018d2cf40 1 -- 172.21.15.38:0/4115168567 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x560e2d8d36e0 con 0x560e2d8cfb60 2024-09-21T03:44:31.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.044+0000 7fb011d81640 1 -- 172.21.15.38:0/4115168567 <== osd.6 v2:172.21.15.195:6826/4055825675 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v134'853 uv853 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb004009b10 con 0x560e2d8cfb60 2024-09-21T03:44:31.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.044+0000 7fb018d2cf40 1 -- 172.21.15.38:0/4115168567 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x560e2d8d36e0 con 0x560e2d8cfb60 2024-09-21T03:44:31.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.045+0000 7fb011d81640 1 -- 172.21.15.38:0/4115168567 <== osd.6 v2:172.21.15.195:6826/4055825675 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v134'854 uv854 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb004009b10 con 0x560e2d8cfb60 2024-09-21T03:44:31.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.045+0000 7fb018d2cf40 1 -- 172.21.15.38:0/4115168567 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:42 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x560e2d8d36e0 con 0x560e2d8cfb60 2024-09-21T03:44:31.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.046+0000 7fb011d81640 1 -- 172.21.15.38:0/4115168567 <== osd.6 v2:172.21.15.195:6826/4055825675 11 ==== osd_op_reply(42 200.00000001 [create] v134'855 uv837 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb004009b10 con 0x560e2d8cfb60 2024-09-21T03:44:31.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.046+0000 7fb018d2cf40 1 -- 172.21.15.38:0/4115168567 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:43 13.14 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e134) -- 0x560e2d8d36e0 con 0x560e2d8cfb60 2024-09-21T03:44:31.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.048+0000 7fb011d81640 1 -- 172.21.15.38:0/4115168567 <== osd.6 v2:172.21.15.195:6826/4055825675 12 ==== osd_op_reply(43 200.00000001 [delete] v134'856 uv856 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb004009b10 con 0x560e2d8cfb60 2024-09-21T03:44:31.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.048+0000 7fb018d2cf40 1 -- 172.21.15.38:0/4115168567 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:44 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x560e2d8d36e0 con 0x560e2d8cfb60 2024-09-21T03:44:31.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.049+0000 7fb011d81640 1 -- 172.21.15.38:0/4115168567 <== osd.6 v2:172.21.15.195:6826/4055825675 13 ==== osd_op_reply(44 200.00000001 [create] v134'857 uv857 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb004009b10 con 0x560e2d8cfb60 2024-09-21T03:44:31.052 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-09-21T03:44:31.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.049+0000 7fb018d2cf40 1 -- 172.21.15.38:0/4115168567 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:45 13.14 13:2e2fa760:::200.00000001:head [write 0~17490 in=17490b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x560e2d8d7b60 con 0x560e2d8cfb60 2024-09-21T03:44:31.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.050+0000 7fb011d81640 1 -- 172.21.15.38:0/4115168567 <== osd.6 v2:172.21.15.195:6826/4055825675 14 ==== osd_op_reply(45 200.00000001 [write 0~17490] v134'858 uv858 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb004009b10 con 0x560e2d8cfb60 2024-09-21T03:44:31.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.050+0000 7fb018d2cf40 1 -- 172.21.15.38:0/4115168567 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:46 13.14 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x560e2d8d7b60 con 0x560e2d8cfb60 2024-09-21T03:44:31.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.052+0000 7fb011d81640 1 -- 172.21.15.38:0/4115168567 <== osd.6 v2:172.21.15.195:6826/4055825675 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v134'859 uv859 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb004009b10 con 0x560e2d8cfb60 2024-09-21T03:44:31.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.052+0000 7fb018d2cf40 1 -- 172.21.15.38:0/4115168567 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:47 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x560e2d8d36e0 con 0x560e2d8cfb60 2024-09-21T03:44:31.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.054+0000 7fb011d81640 1 -- 172.21.15.38:0/4115168567 <== osd.6 v2:172.21.15.195:6826/4055825675 16 ==== osd_op_reply(47 606.00000000 [create] v134'860 uv843 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb004009b10 con 0x560e2d8cfb60 2024-09-21T03:44:31.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.054+0000 7fb018d2cf40 1 -- 172.21.15.38:0/4115168567 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:48 13.14 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e134) -- 0x560e2d8d36e0 con 0x560e2d8cfb60 2024-09-21T03:44:31.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.055+0000 7fb011d81640 1 -- 172.21.15.38:0/4115168567 <== osd.6 v2:172.21.15.195:6826/4055825675 17 ==== osd_op_reply(48 606.00000000 [delete] v134'861 uv861 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb004009b10 con 0x560e2d8cfb60 2024-09-21T03:44:31.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.055+0000 7fb018d2cf40 1 -- 172.21.15.38:0/4115168567 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:49 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x560e2d8d36e0 con 0x560e2d8cfb60 2024-09-21T03:44:31.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.056+0000 7fb011d81640 1 -- 172.21.15.38:0/4115168567 <== osd.6 v2:172.21.15.195:6826/4055825675 18 ==== osd_op_reply(49 606.00000000 [create] v134'862 uv862 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb004009b10 con 0x560e2d8cfb60 2024-09-21T03:44:31.059 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-09-21T03:44:31.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.056+0000 7fb018d2cf40 1 -- 172.21.15.38:0/4115168567 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:50 13.14 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x560e2d8d46f0 con 0x560e2d8cfb60 2024-09-21T03:44:31.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.057+0000 7fb011d81640 1 -- 172.21.15.38:0/4115168567 <== osd.6 v2:172.21.15.195:6826/4055825675 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v134'863 uv863 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb004009b10 con 0x560e2d8cfb60 2024-09-21T03:44:31.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.057+0000 7fb018d2cf40 1 -- 172.21.15.38:0/4115168567 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:51 13.14 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x560e2d8d46f0 con 0x560e2d8cfb60 2024-09-21T03:44:31.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.059+0000 7fb011d81640 1 -- 172.21.15.38:0/4115168567 <== osd.6 v2:172.21.15.195:6826/4055825675 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v134'864 uv864 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb004009b10 con 0x560e2d8cfb60 2024-09-21T03:44:31.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.059+0000 7fb018d2cf40 1 -- 172.21.15.38:0/4115168567 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:52 13.14 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x560e2d8d36e0 con 0x560e2d8cfb60 2024-09-21T03:44:31.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.060+0000 7fb011d81640 1 -- 172.21.15.38:0/4115168567 <== osd.6 v2:172.21.15.195:6826/4055825675 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v134'865 uv865 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb004009b10 con 0x560e2d8cfb60 2024-09-21T03:44:31.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.060+0000 7fb018d2cf40 1 --2- 172.21.15.38:0/4115168567 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x560e2d8d3bf0 0x560e2d8d6010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:31.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.060+0000 7fb018d2cf40 1 -- 172.21.15.38:0/4115168567 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:53 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x560e2d8d6590 con 0x560e2d8d3bf0 2024-09-21T03:44:31.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.060+0000 7fb011580640 1 --2- 172.21.15.38:0/4115168567 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x560e2d8d3bf0 0x560e2d8d6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:31.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.060+0000 7fb011580640 1 --2- 172.21.15.38:0/4115168567 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x560e2d8d3bf0 0x560e2d8d6010 crc :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:31.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.062+0000 7fb011580640 1 -- 172.21.15.38:0/4115168567 <== osd.0 v2:172.21.15.38:6800/2381639433 1 ==== osd_op_reply(53 607.00000000 [create] v134'225 uv224 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb008062040 con 0x560e2d8d3bf0 2024-09-21T03:44:31.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.062+0000 7fb018d2cf40 1 -- 172.21.15.38:0/4115168567 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:54 13.1c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e134) -- 0x560e2d8d6650 con 0x560e2d8d3bf0 2024-09-21T03:44:31.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.063+0000 7fb011580640 1 -- 172.21.15.38:0/4115168567 <== osd.0 v2:172.21.15.38:6800/2381639433 2 ==== osd_op_reply(54 607.00000000 [delete] v134'226 uv226 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb008062040 con 0x560e2d8d3bf0 2024-09-21T03:44:31.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.063+0000 7fb018d2cf40 1 -- 172.21.15.38:0/4115168567 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:55 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x560e2d8d6a40 con 0x560e2d8d3bf0 2024-09-21T03:44:31.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.064+0000 7fb011580640 1 -- 172.21.15.38:0/4115168567 <== osd.0 v2:172.21.15.38:6800/2381639433 3 ==== osd_op_reply(55 607.00000000 [create] v134'227 uv227 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb008062040 con 0x560e2d8d3bf0 2024-09-21T03:44:31.067 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-09-21T03:44:31.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.064+0000 7fb018d2cf40 1 -- 172.21.15.38:0/4115168567 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:56 13.1c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x560e2d8d7a50 con 0x560e2d8d3bf0 2024-09-21T03:44:31.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.065+0000 7fb011580640 1 -- 172.21.15.38:0/4115168567 <== osd.0 v2:172.21.15.38:6800/2381639433 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v134'228 uv228 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb008062040 con 0x560e2d8d3bf0 2024-09-21T03:44:31.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.065+0000 7fb018d2cf40 1 -- 172.21.15.38:0/4115168567 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:57 13.1c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x560e2d8d7a50 con 0x560e2d8d3bf0 2024-09-21T03:44:31.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.066+0000 7fb011580640 1 -- 172.21.15.38:0/4115168567 <== osd.0 v2:172.21.15.38:6800/2381639433 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v134'229 uv229 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb008062040 con 0x560e2d8d3bf0 2024-09-21T03:44:31.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.066+0000 7fb018d2cf40 1 -- 172.21.15.38:0/4115168567 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:58 13.1c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x560e2d8d6a40 con 0x560e2d8d3bf0 2024-09-21T03:44:31.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.067+0000 7fb011580640 1 -- 172.21.15.38:0/4115168567 <== osd.0 v2:172.21.15.38:6800/2381639433 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v134'230 uv230 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb008062040 con 0x560e2d8d3bf0 2024-09-21T03:44:31.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.067+0000 7fb018d2cf40 1 --2- 172.21.15.38:0/4115168567 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x560e2d8d70c0 0x560e2d8d94e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:31.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.067+0000 7fb018d2cf40 1 -- 172.21.15.38:0/4115168567 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:59 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x560e2d8d9a60 con 0x560e2d8d70c0 2024-09-21T03:44:31.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.067+0000 7fb012582640 1 --2- 172.21.15.38:0/4115168567 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x560e2d8d70c0 0x560e2d8d94e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:31.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.068+0000 7fb012582640 1 --2- 172.21.15.38:0/4115168567 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x560e2d8d70c0 0x560e2d8d94e0 crc :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:31.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.069+0000 7fb012582640 1 -- 172.21.15.38:0/4115168567 <== osd.3 v2:172.21.15.38:6808/1286810294 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v134'168 uv167 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7fb00c00e070 con 0x560e2d8d70c0 2024-09-21T03:44:31.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.069+0000 7fb018d2cf40 1 -- 172.21.15.38:0/4115168567 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:60 13.6 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e134) -- 0x560e2d8d9f70 con 0x560e2d8d70c0 2024-09-21T03:44:31.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.070+0000 7fb012582640 1 -- 172.21.15.38:0/4115168567 <== osd.3 v2:172.21.15.38:6808/1286810294 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v134'169 uv169 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fb00c00e070 con 0x560e2d8d70c0 2024-09-21T03:44:31.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.070+0000 7fb018d2cf40 1 -- 172.21.15.38:0/4115168567 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:61 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x560e2d8da470 con 0x560e2d8d70c0 2024-09-21T03:44:31.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.071+0000 7fb012582640 1 -- 172.21.15.38:0/4115168567 <== osd.3 v2:172.21.15.38:6808/1286810294 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v134'170 uv170 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fb00c00e070 con 0x560e2d8d70c0 2024-09-21T03:44:31.074 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-09-21T03:44:31.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.071+0000 7fb018d2cf40 1 -- 172.21.15.38:0/4115168567 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:62 13.6 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x560e2d8dad30 con 0x560e2d8d70c0 2024-09-21T03:44:31.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.072+0000 7fb012582640 1 -- 172.21.15.38:0/4115168567 <== osd.3 v2:172.21.15.38:6808/1286810294 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v134'171 uv171 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fb00c00e070 con 0x560e2d8d70c0 2024-09-21T03:44:31.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.072+0000 7fb018d2cf40 1 -- 172.21.15.38:0/4115168567 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x560e2d8db0d0 con 0x560e2d8d3bf0 2024-09-21T03:44:31.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.074+0000 7fb011580640 1 -- 172.21.15.38:0/4115168567 <== osd.0 v2:172.21.15.38:6800/2381639433 7 ==== osd_op_reply(63 608.00000000 [create] v134'225 uv224 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb008062040 con 0x560e2d8d3bf0 2024-09-21T03:44:31.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.074+0000 7fb018d2cf40 1 -- 172.21.15.38:0/4115168567 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e134) -- 0x560e2d8db0d0 con 0x560e2d8d3bf0 2024-09-21T03:44:31.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.075+0000 7fb011580640 1 -- 172.21.15.38:0/4115168567 <== osd.0 v2:172.21.15.38:6800/2381639433 8 ==== osd_op_reply(64 608.00000000 [delete] v134'226 uv226 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb008062040 con 0x560e2d8d3bf0 2024-09-21T03:44:31.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.075+0000 7fb018d2cf40 1 -- 172.21.15.38:0/4115168567 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x560e2d8db0d0 con 0x560e2d8d3bf0 2024-09-21T03:44:31.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.076+0000 7fb011580640 1 -- 172.21.15.38:0/4115168567 <== osd.0 v2:172.21.15.38:6800/2381639433 9 ==== osd_op_reply(65 608.00000000 [create] v134'227 uv227 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb008062040 con 0x560e2d8d3bf0 2024-09-21T03:44:31.079 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-09-21T03:44:31.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.076+0000 7fb018d2cf40 1 -- 172.21.15.38:0/4115168567 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x560e2d8dc0e0 con 0x560e2d8d3bf0 2024-09-21T03:44:31.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.077+0000 7fb011580640 1 -- 172.21.15.38:0/4115168567 <== osd.0 v2:172.21.15.38:6800/2381639433 10 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v134'228 uv228 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb008062040 con 0x560e2d8d3bf0 2024-09-21T03:44:31.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.077+0000 7fb018d2cf40 1 -- 172.21.15.38:0/4115168567 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x560e2d8dc0e0 con 0x560e2d8d3bf0 2024-09-21T03:44:31.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.078+0000 7fb011580640 1 -- 172.21.15.38:0/4115168567 <== osd.0 v2:172.21.15.38:6800/2381639433 11 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v134'229 uv229 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb008062040 con 0x560e2d8d3bf0 2024-09-21T03:44:31.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.078+0000 7fb018d2cf40 1 -- 172.21.15.38:0/4115168567 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x560e2d8db0d0 con 0x560e2d8d3bf0 2024-09-21T03:44:31.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.079+0000 7fb011580640 1 -- 172.21.15.38:0/4115168567 <== osd.0 v2:172.21.15.38:6800/2381639433 12 ==== osd_op_reply(68 608.00000000 [omap-set-header] v134'230 uv230 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb008062040 con 0x560e2d8d3bf0 2024-09-21T03:44:31.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.079+0000 7fb018d2cf40 1 -- 172.21.15.38:0/4115168567 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x560e2d8db0d0 con 0x560e2d8d70c0 2024-09-21T03:44:31.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.081+0000 7fb012582640 1 -- 172.21.15.38:0/4115168567 <== osd.3 v2:172.21.15.38:6808/1286810294 5 ==== osd_op_reply(69 604.00000000 [create] v134'415 uv409 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb00c00e070 con 0x560e2d8d70c0 2024-09-21T03:44:31.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.081+0000 7fb018d2cf40 1 -- 172.21.15.38:0/4115168567 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e134) -- 0x560e2d8db470 con 0x560e2d8d70c0 2024-09-21T03:44:31.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.082+0000 7fb012582640 1 -- 172.21.15.38:0/4115168567 <== osd.3 v2:172.21.15.38:6808/1286810294 6 ==== osd_op_reply(70 604.00000000 [delete] v134'416 uv416 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb00c00e070 con 0x560e2d8d70c0 2024-09-21T03:44:31.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.082+0000 7fb018d2cf40 1 -- 172.21.15.38:0/4115168567 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x560e2d8db810 con 0x560e2d8d70c0 2024-09-21T03:44:31.086 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-09-21T03:44:31.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.083+0000 7fb012582640 1 -- 172.21.15.38:0/4115168567 <== osd.3 v2:172.21.15.38:6808/1286810294 7 ==== osd_op_reply(71 604.00000000 [create] v134'417 uv417 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb00c00e070 con 0x560e2d8d70c0 2024-09-21T03:44:31.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.083+0000 7fb018d2cf40 1 -- 172.21.15.38:0/4115168567 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x560e2d8dcbc0 con 0x560e2d8d70c0 2024-09-21T03:44:31.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.084+0000 7fb012582640 1 -- 172.21.15.38:0/4115168567 <== osd.3 v2:172.21.15.38:6808/1286810294 8 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v134'418 uv418 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb00c00e070 con 0x560e2d8d70c0 2024-09-21T03:44:31.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.084+0000 7fb018d2cf40 1 -- 172.21.15.38:0/4115168567 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x560e2d8dcbc0 con 0x560e2d8d70c0 2024-09-21T03:44:31.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.085+0000 7fb012582640 1 -- 172.21.15.38:0/4115168567 <== osd.3 v2:172.21.15.38:6808/1286810294 9 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v134'419 uv419 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb00c00e070 con 0x560e2d8d70c0 2024-09-21T03:44:31.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.085+0000 7fb018d2cf40 1 -- 172.21.15.38:0/4115168567 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x560e2d8dbbb0 con 0x560e2d8d70c0 2024-09-21T03:44:31.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.086+0000 7fb012582640 1 -- 172.21.15.38:0/4115168567 <== osd.3 v2:172.21.15.38:6808/1286810294 10 ==== osd_op_reply(74 604.00000000 [omap-set-header] v134'420 uv420 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb00c00e070 con 0x560e2d8d70c0 2024-09-21T03:44:31.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.086+0000 7fb018d2cf40 1 -- 172.21.15.38:0/4115168567 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x560e2d8dbbb0 con 0x560e2d8d70c0 2024-09-21T03:44:31.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.087+0000 7fb012582640 1 -- 172.21.15.38:0/4115168567 <== osd.3 v2:172.21.15.38:6808/1286810294 11 ==== osd_op_reply(75 500.00000000 [create] v134'421 uv414 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb00c00e070 con 0x560e2d8d70c0 2024-09-21T03:44:31.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.087+0000 7fb018d2cf40 1 -- 172.21.15.38:0/4115168567 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e134) -- 0x560e2d8dbbb0 con 0x560e2d8d70c0 2024-09-21T03:44:31.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.088+0000 7fb012582640 1 -- 172.21.15.38:0/4115168567 <== osd.3 v2:172.21.15.38:6808/1286810294 12 ==== osd_op_reply(76 500.00000000 [delete] v134'422 uv422 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb00c00e070 con 0x560e2d8d70c0 2024-09-21T03:44:31.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.088+0000 7fb018d2cf40 1 -- 172.21.15.38:0/4115168567 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x560e2d8dbbb0 con 0x560e2d8d70c0 2024-09-21T03:44:31.092 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-09-21T03:44:31.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.089+0000 7fb012582640 1 -- 172.21.15.38:0/4115168567 <== osd.3 v2:172.21.15.38:6808/1286810294 13 ==== osd_op_reply(77 500.00000000 [create] v134'423 uv423 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb00c00e070 con 0x560e2d8d70c0 2024-09-21T03:44:31.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.089+0000 7fb018d2cf40 1 -- 172.21.15.38:0/4115168567 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x560e2d8dbbb0 con 0x560e2d8d70c0 2024-09-21T03:44:31.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.090+0000 7fb012582640 1 -- 172.21.15.38:0/4115168567 <== osd.3 v2:172.21.15.38:6808/1286810294 14 ==== osd_op_reply(78 500.00000000 [write 0~90] v134'424 uv424 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb00c00e070 con 0x560e2d8d70c0 2024-09-21T03:44:31.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.090+0000 7fb018d2cf40 1 -- 172.21.15.38:0/4115168567 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x560e2d8dbbb0 con 0x560e2d8d70c0 2024-09-21T03:44:31.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.091+0000 7fb012582640 1 -- 172.21.15.38:0/4115168567 <== osd.3 v2:172.21.15.38:6808/1286810294 15 ==== osd_op_reply(79 500.00000000 [omap-set-header] v134'425 uv425 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb00c00e070 con 0x560e2d8d70c0 2024-09-21T03:44:31.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.091+0000 7fb018d2cf40 1 -- 172.21.15.38:0/4115168567 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:80 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x560e2d8dbbb0 con 0x560e2d8d3bf0 2024-09-21T03:44:31.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.093+0000 7fb011580640 1 -- 172.21.15.38:0/4115168567 <== osd.0 v2:172.21.15.38:6800/2381639433 13 ==== osd_op_reply(80 mds_snaptable [create] v134'188 uv187 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7fb008062040 con 0x560e2d8d3bf0 2024-09-21T03:44:31.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.093+0000 7fb018d2cf40 1 -- 172.21.15.38:0/4115168567 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:81 13.d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e134) -- 0x560e2d8dbbb0 con 0x560e2d8d3bf0 2024-09-21T03:44:31.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.094+0000 7fb011580640 1 -- 172.21.15.38:0/4115168567 <== osd.0 v2:172.21.15.38:6800/2381639433 14 ==== osd_op_reply(81 mds_snaptable [delete] v134'189 uv189 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fb008062040 con 0x560e2d8d3bf0 2024-09-21T03:44:31.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.094+0000 7fb018d2cf40 1 -- 172.21.15.38:0/4115168567 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:82 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x560e2d8dbbb0 con 0x560e2d8d3bf0 2024-09-21T03:44:31.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.095+0000 7fb011580640 1 -- 172.21.15.38:0/4115168567 <== osd.0 v2:172.21.15.38:6800/2381639433 15 ==== osd_op_reply(82 mds_snaptable [create] v134'190 uv190 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fb008062040 con 0x560e2d8d3bf0 2024-09-21T03:44:31.098 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-09-21T03:44:31.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.095+0000 7fb018d2cf40 1 -- 172.21.15.38:0/4115168567 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:83 13.d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x560e2d8dbbb0 con 0x560e2d8d3bf0 2024-09-21T03:44:31.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.096+0000 7fb011580640 1 -- 172.21.15.38:0/4115168567 <== osd.0 v2:172.21.15.38:6800/2381639433 16 ==== osd_op_reply(83 mds_snaptable [write 0~70] v134'191 uv191 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fb008062040 con 0x560e2d8d3bf0 2024-09-21T03:44:31.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.096+0000 7fb018d2cf40 1 -- 172.21.15.38:0/4115168567 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:84 13.d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x560e2d8dbbb0 con 0x560e2d8d3bf0 2024-09-21T03:44:31.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.097+0000 7fb011580640 1 -- 172.21.15.38:0/4115168567 <== osd.0 v2:172.21.15.38:6800/2381639433 17 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v134'192 uv192 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fb008062040 con 0x560e2d8d3bf0 2024-09-21T03:44:31.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.097+0000 7fb018d2cf40 1 --2- 172.21.15.38:0/4115168567 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x560e2d8dc230 0x560e2d8de650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:31.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.097+0000 7fb018d2cf40 1 -- 172.21.15.38:0/4115168567 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x560e2d8dec20 con 0x560e2d8dc230 2024-09-21T03:44:31.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.097+0000 7fb011d81640 1 --2- 172.21.15.38:0/4115168567 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x560e2d8dc230 0x560e2d8de650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:31.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.098+0000 7fb011d81640 1 --2- 172.21.15.38:0/4115168567 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x560e2d8dc230 0x560e2d8de650 crc :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:31.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.099+0000 7fb011d81640 1 -- 172.21.15.38:0/4115168567 <== osd.1 v2:172.21.15.38:6816/1491818732 1 ==== osd_op_reply(85 605.00000000 [create] v134'225 uv224 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb004009b10 con 0x560e2d8dc230 2024-09-21T03:44:31.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.100+0000 7fb018d2cf40 1 -- 172.21.15.38:0/4115168567 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e134) -- 0x560e2d8dece0 con 0x560e2d8dc230 2024-09-21T03:44:31.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.101+0000 7fb011d81640 1 -- 172.21.15.38:0/4115168567 <== osd.1 v2:172.21.15.38:6816/1491818732 2 ==== osd_op_reply(86 605.00000000 [delete] v134'226 uv226 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb004009b10 con 0x560e2d8dc230 2024-09-21T03:44:31.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.101+0000 7fb018d2cf40 1 -- 172.21.15.38:0/4115168567 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x560e2d8df100 con 0x560e2d8dc230 2024-09-21T03:44:31.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.102+0000 7fb011d81640 1 -- 172.21.15.38:0/4115168567 <== osd.1 v2:172.21.15.38:6816/1491818732 3 ==== osd_op_reply(87 605.00000000 [create] v134'227 uv227 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb004009b10 con 0x560e2d8dc230 2024-09-21T03:44:31.105 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-09-21T03:44:31.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.102+0000 7fb018d2cf40 1 -- 172.21.15.38:0/4115168567 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x560e2d8e0110 con 0x560e2d8dc230 2024-09-21T03:44:31.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.103+0000 7fb011d81640 1 -- 172.21.15.38:0/4115168567 <== osd.1 v2:172.21.15.38:6816/1491818732 4 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v134'228 uv228 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb004009b10 con 0x560e2d8dc230 2024-09-21T03:44:31.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.103+0000 7fb018d2cf40 1 -- 172.21.15.38:0/4115168567 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x560e2d8e0110 con 0x560e2d8dc230 2024-09-21T03:44:31.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.104+0000 7fb011d81640 1 -- 172.21.15.38:0/4115168567 <== osd.1 v2:172.21.15.38:6816/1491818732 5 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v134'229 uv229 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb004009b10 con 0x560e2d8dc230 2024-09-21T03:44:31.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.104+0000 7fb018d2cf40 1 -- 172.21.15.38:0/4115168567 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x560e2d8df100 con 0x560e2d8dc230 2024-09-21T03:44:31.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.105+0000 7fb011d81640 1 -- 172.21.15.38:0/4115168567 <== osd.1 v2:172.21.15.38:6816/1491818732 6 ==== osd_op_reply(90 605.00000000 [omap-set-header] v134'230 uv230 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb004009b10 con 0x560e2d8dc230 2024-09-21T03:44:31.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.105+0000 7fb018d2cf40 1 -- 172.21.15.38:0/4115168567 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x560e2d8df100 con 0x560e2d8dc230 2024-09-21T03:44:31.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.106+0000 7fb011d81640 1 -- 172.21.15.38:0/4115168567 <== osd.1 v2:172.21.15.38:6816/1491818732 7 ==== osd_op_reply(91 mds0_inotable [create] v134'376 uv369 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7fb004009b10 con 0x560e2d8dc230 2024-09-21T03:44:31.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.106+0000 7fb018d2cf40 1 -- 172.21.15.38:0/4115168567 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e134) -- 0x560e2d8df100 con 0x560e2d8dc230 2024-09-21T03:44:31.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.107+0000 7fb011d81640 1 -- 172.21.15.38:0/4115168567 <== osd.1 v2:172.21.15.38:6816/1491818732 8 ==== osd_op_reply(92 mds0_inotable [delete] v134'377 uv377 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fb004009b10 con 0x560e2d8dc230 2024-09-21T03:44:31.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.107+0000 7fb018d2cf40 1 -- 172.21.15.38:0/4115168567 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x560e2d8df100 con 0x560e2d8dc230 2024-09-21T03:44:31.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.109+0000 7fb011d81640 1 -- 172.21.15.38:0/4115168567 <== osd.1 v2:172.21.15.38:6816/1491818732 9 ==== osd_op_reply(93 mds0_inotable [create] v134'378 uv378 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fb004009b10 con 0x560e2d8dc230 2024-09-21T03:44:31.112 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-09-21T03:44:31.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.109+0000 7fb018d2cf40 1 -- 172.21.15.38:0/4115168567 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:94 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x560e2d8df100 con 0x560e2d8dc230 2024-09-21T03:44:31.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.110+0000 7fb011d81640 1 -- 172.21.15.38:0/4115168567 <== osd.1 v2:172.21.15.38:6816/1491818732 10 ==== osd_op_reply(94 mds0_inotable [write 0~34] v134'379 uv379 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fb004009b10 con 0x560e2d8dc230 2024-09-21T03:44:31.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.110+0000 7fb018d2cf40 1 -- 172.21.15.38:0/4115168567 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:95 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x560e2d8df100 con 0x560e2d8dc230 2024-09-21T03:44:31.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.111+0000 7fb011d81640 1 -- 172.21.15.38:0/4115168567 <== osd.1 v2:172.21.15.38:6816/1491818732 11 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v134'380 uv380 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fb004009b10 con 0x560e2d8dc230 2024-09-21T03:44:31.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.111+0000 7fb018d2cf40 1 -- 172.21.15.38:0/4115168567 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:96 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x560e2d8df100 con 0x560e2d8dc230 2024-09-21T03:44:31.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.112+0000 7fb011d81640 1 -- 172.21.15.38:0/4115168567 <== osd.1 v2:172.21.15.38:6816/1491818732 12 ==== osd_op_reply(96 100.00000000 [create] v134'381 uv375 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb004009b10 con 0x560e2d8dc230 2024-09-21T03:44:31.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.112+0000 7fb018d2cf40 1 -- 172.21.15.38:0/4115168567 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:97 13.13 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e134) -- 0x560e2d8df100 con 0x560e2d8dc230 2024-09-21T03:44:31.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.113+0000 7fb011d81640 1 -- 172.21.15.38:0/4115168567 <== osd.1 v2:172.21.15.38:6816/1491818732 13 ==== osd_op_reply(97 100.00000000 [delete] v134'382 uv382 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb004009b10 con 0x560e2d8dc230 2024-09-21T03:44:31.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.113+0000 7fb018d2cf40 1 -- 172.21.15.38:0/4115168567 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:98 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x560e2d8df100 con 0x560e2d8dc230 2024-09-21T03:44:31.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.114+0000 7fb011d81640 1 -- 172.21.15.38:0/4115168567 <== osd.1 v2:172.21.15.38:6816/1491818732 14 ==== osd_op_reply(98 100.00000000 [create] v134'383 uv383 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb004009b10 con 0x560e2d8dc230 2024-09-21T03:44:31.117 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-09-21T03:44:31.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.114+0000 7fb018d2cf40 1 -- 172.21.15.38:0/4115168567 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:99 13.13 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x560e2d8df100 con 0x560e2d8dc230 2024-09-21T03:44:31.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.115+0000 7fb011d81640 1 -- 172.21.15.38:0/4115168567 <== osd.1 v2:172.21.15.38:6816/1491818732 15 ==== osd_op_reply(99 100.00000000 [omap-set-header] v134'384 uv384 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb004009b10 con 0x560e2d8dc230 2024-09-21T03:44:31.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.115+0000 7fb018d2cf40 1 -- 172.21.15.38:0/4115168567 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:100 13.13 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x560e2d8e1eb0 con 0x560e2d8dc230 2024-09-21T03:44:31.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.116+0000 7fb011d81640 1 -- 172.21.15.38:0/4115168567 <== osd.1 v2:172.21.15.38:6816/1491818732 16 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v134'385 uv385 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb004009b10 con 0x560e2d8dc230 2024-09-21T03:44:31.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.116+0000 7fb018d2cf40 1 -- 172.21.15.38:0/4115168567 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x560e2d8e1e70 con 0x560e2d8d70c0 2024-09-21T03:44:31.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.117+0000 7fb012582640 1 -- 172.21.15.38:0/4115168567 <== osd.3 v2:172.21.15.38:6808/1286810294 16 ==== osd_op_reply(101 mds0_sessionmap [create] v134'167 uv165 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7fb00c00e070 con 0x560e2d8d70c0 2024-09-21T03:44:31.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.117+0000 7fb018d2cf40 1 -- 172.21.15.38:0/4115168567 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e134) -- 0x560e2d8e1e70 con 0x560e2d8d70c0 2024-09-21T03:44:31.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.119+0000 7fb012582640 1 -- 172.21.15.38:0/4115168567 <== osd.3 v2:172.21.15.38:6808/1286810294 17 ==== osd_op_reply(102 mds0_sessionmap [delete] v134'168 uv168 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fb00c00e070 con 0x560e2d8d70c0 2024-09-21T03:44:31.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.119+0000 7fb018d2cf40 1 -- 172.21.15.38:0/4115168567 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:103 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x560e2d8e1e70 con 0x560e2d8d70c0 2024-09-21T03:44:31.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.120+0000 7fb012582640 1 -- 172.21.15.38:0/4115168567 <== osd.3 v2:172.21.15.38:6808/1286810294 18 ==== osd_op_reply(103 mds0_sessionmap [create] v134'169 uv169 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fb00c00e070 con 0x560e2d8d70c0 2024-09-21T03:44:31.123 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-09-21T03:44:31.123 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.120+0000 7fb018d2cf40 1 -- 172.21.15.38:0/4115168567 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:104 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x560e2d8e1e70 con 0x560e2d8d70c0 2024-09-21T03:44:31.123 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.121+0000 7fb012582640 1 -- 172.21.15.38:0/4115168567 <== osd.3 v2:172.21.15.38:6808/1286810294 19 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v134'170 uv170 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fb00c00e070 con 0x560e2d8d70c0 2024-09-21T03:44:31.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.121+0000 7fb018d2cf40 1 -- 172.21.15.38:0/4115168567 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x560e2d8df4f0 con 0x560e2d8d70c0 2024-09-21T03:44:31.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.122+0000 7fb012582640 1 -- 172.21.15.38:0/4115168567 <== osd.3 v2:172.21.15.38:6808/1286810294 20 ==== osd_op_reply(105 609.00000000 [create] v134'225 uv224 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb00c00e070 con 0x560e2d8d70c0 2024-09-21T03:44:31.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.122+0000 7fb018d2cf40 1 -- 172.21.15.38:0/4115168567 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e134) -- 0x560e2d8df4f0 con 0x560e2d8d70c0 2024-09-21T03:44:31.126 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.123+0000 7fb012582640 1 -- 172.21.15.38:0/4115168567 <== osd.3 v2:172.21.15.38:6808/1286810294 21 ==== osd_op_reply(106 609.00000000 [delete] v134'226 uv226 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb00c00e070 con 0x560e2d8d70c0 2024-09-21T03:44:31.126 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.123+0000 7fb018d2cf40 1 -- 172.21.15.38:0/4115168567 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x560e2d8df4f0 con 0x560e2d8d70c0 2024-09-21T03:44:31.126 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.124+0000 7fb012582640 1 -- 172.21.15.38:0/4115168567 <== osd.3 v2:172.21.15.38:6808/1286810294 22 ==== osd_op_reply(107 609.00000000 [create] v134'227 uv227 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb00c00e070 con 0x560e2d8d70c0 2024-09-21T03:44:31.127 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-09-21T03:44:31.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.124+0000 7fb018d2cf40 1 -- 172.21.15.38:0/4115168567 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x560e2d8df4f0 con 0x560e2d8d70c0 2024-09-21T03:44:31.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.125+0000 7fb012582640 1 -- 172.21.15.38:0/4115168567 <== osd.3 v2:172.21.15.38:6808/1286810294 23 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v134'228 uv228 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb00c00e070 con 0x560e2d8d70c0 2024-09-21T03:44:31.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.125+0000 7fb018d2cf40 1 -- 172.21.15.38:0/4115168567 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:109 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x560e2d8df4f0 con 0x560e2d8d70c0 2024-09-21T03:44:31.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.126+0000 7fb012582640 1 -- 172.21.15.38:0/4115168567 <== osd.3 v2:172.21.15.38:6808/1286810294 24 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v134'229 uv229 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb00c00e070 con 0x560e2d8d70c0 2024-09-21T03:44:31.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.126+0000 7fb018d2cf40 1 -- 172.21.15.38:0/4115168567 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:110 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x560e2d8df4f0 con 0x560e2d8d70c0 2024-09-21T03:44:31.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.127+0000 7fb012582640 1 -- 172.21.15.38:0/4115168567 <== osd.3 v2:172.21.15.38:6808/1286810294 25 ==== osd_op_reply(110 609.00000000 [omap-set-header] v134'230 uv230 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb00c00e070 con 0x560e2d8d70c0 2024-09-21T03:44:31.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.127+0000 7fb018d2cf40 1 --2- 172.21.15.38:0/4115168567 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x560e2d8dfe10 0x560e2d8e2570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:31.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.127+0000 7fb018d2cf40 1 -- 172.21.15.38:0/4115168567 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x560e2d8e2ab0 con 0x560e2d8dfe10 2024-09-21T03:44:31.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.127+0000 7fb011580640 1 --2- 172.21.15.38:0/4115168567 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x560e2d8dfe10 0x560e2d8e2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:31.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.128+0000 7fb011580640 1 --2- 172.21.15.38:0/4115168567 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x560e2d8dfe10 0x560e2d8e2570 crc :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:31.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.129+0000 7fb011580640 1 -- 172.21.15.38:0/4115168567 <== osd.2 v2:172.21.15.38:6824/2577858902 1 ==== osd_op_reply(111 400.00000000 [create] v134'191 uv190 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb008062040 con 0x560e2d8dfe10 2024-09-21T03:44:31.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.129+0000 7fb018d2cf40 1 -- 172.21.15.38:0/4115168567 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e134) -- 0x560e2d8e2ab0 con 0x560e2d8dfe10 2024-09-21T03:44:31.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.130+0000 7fb011580640 1 -- 172.21.15.38:0/4115168567 <== osd.2 v2:172.21.15.38:6824/2577858902 2 ==== osd_op_reply(112 400.00000000 [delete] v134'192 uv192 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb008062040 con 0x560e2d8dfe10 2024-09-21T03:44:31.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.130+0000 7fb018d2cf40 1 -- 172.21.15.38:0/4115168567 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x560e2d8e2ab0 con 0x560e2d8dfe10 2024-09-21T03:44:31.134 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-09-21T03:44:31.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.131+0000 7fb011580640 1 -- 172.21.15.38:0/4115168567 <== osd.2 v2:172.21.15.38:6824/2577858902 3 ==== osd_op_reply(113 400.00000000 [create] v134'193 uv193 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb008062040 con 0x560e2d8dfe10 2024-09-21T03:44:31.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.131+0000 7fb018d2cf40 1 -- 172.21.15.38:0/4115168567 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:114 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x560e2d8e2ab0 con 0x560e2d8dfe10 2024-09-21T03:44:31.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.132+0000 7fb011580640 1 -- 172.21.15.38:0/4115168567 <== osd.2 v2:172.21.15.38:6824/2577858902 4 ==== osd_op_reply(114 400.00000000 [write 0~22] v134'194 uv194 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb008062040 con 0x560e2d8dfe10 2024-09-21T03:44:31.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.132+0000 7fb018d2cf40 1 -- 172.21.15.38:0/4115168567 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:115 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x560e2d8e2ab0 con 0x560e2d8dfe10 2024-09-21T03:44:31.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.133+0000 7fb011580640 1 -- 172.21.15.38:0/4115168567 <== osd.2 v2:172.21.15.38:6824/2577858902 5 ==== osd_op_reply(115 400.00000000 [omap-set-header] v134'195 uv195 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb008062040 con 0x560e2d8dfe10 2024-09-21T03:44:31.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.133+0000 7fb018d2cf40 1 -- 172.21.15.38:0/4115168567 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:116 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x560e2d8e2ab0 con 0x560e2d8dc230 2024-09-21T03:44:31.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.135+0000 7fb011d81640 1 -- 172.21.15.38:0/4115168567 <== osd.1 v2:172.21.15.38:6816/1491818732 17 ==== osd_op_reply(116 100.00000000.inode [create] v134'468 uv459 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7fb004009b10 con 0x560e2d8dc230 2024-09-21T03:44:31.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.135+0000 7fb018d2cf40 1 -- 172.21.15.38:0/4115168567 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:117 13.1f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e134) -- 0x560e2d8e2ab0 con 0x560e2d8dc230 2024-09-21T03:44:31.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.136+0000 7fb011d81640 1 -- 172.21.15.38:0/4115168567 <== osd.1 v2:172.21.15.38:6816/1491818732 18 ==== osd_op_reply(117 100.00000000.inode [delete] v134'469 uv469 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fb004009b10 con 0x560e2d8dc230 2024-09-21T03:44:31.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.136+0000 7fb018d2cf40 1 -- 172.21.15.38:0/4115168567 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:118 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x560e2d8e2ab0 con 0x560e2d8dc230 2024-09-21T03:44:31.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.137+0000 7fb011d81640 1 -- 172.21.15.38:0/4115168567 <== osd.1 v2:172.21.15.38:6816/1491818732 19 ==== osd_op_reply(118 100.00000000.inode [create] v134'470 uv470 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fb004009b10 con 0x560e2d8dc230 2024-09-21T03:44:31.140 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-09-21T03:44:31.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.137+0000 7fb018d2cf40 1 -- 172.21.15.38:0/4115168567 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:119 13.1f 13:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x560e2d8e2ab0 con 0x560e2d8dc230 2024-09-21T03:44:31.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.138+0000 7fb011d81640 1 -- 172.21.15.38:0/4115168567 <== osd.1 v2:172.21.15.38:6816/1491818732 20 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v134'471 uv471 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fb004009b10 con 0x560e2d8dc230 2024-09-21T03:44:31.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.138+0000 7fb018d2cf40 1 -- 172.21.15.38:0/4115168567 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:120 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x560e2d8e2ab0 con 0x560e2d8dc230 2024-09-21T03:44:31.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.140+0000 7fb011d81640 1 -- 172.21.15.38:0/4115168567 <== osd.1 v2:172.21.15.38:6816/1491818732 21 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v134'472 uv472 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fb004009b10 con 0x560e2d8dc230 2024-09-21T03:44:31.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.140+0000 7fb018d2cf40 1 -- 172.21.15.38:0/4115168567 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:121 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x560e2d8e2ab0 con 0x560e2d8dc230 2024-09-21T03:44:31.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.141+0000 7fb011d81640 1 -- 172.21.15.38:0/4115168567 <== osd.1 v2:172.21.15.38:6816/1491818732 22 ==== osd_op_reply(121 1.00000000 [create] v134'473 uv466 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7fb004009b10 con 0x560e2d8dc230 2024-09-21T03:44:31.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.141+0000 7fb018d2cf40 1 -- 172.21.15.38:0/4115168567 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:122 13.1f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e134) -- 0x560e2d8e2ab0 con 0x560e2d8dc230 2024-09-21T03:44:31.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.142+0000 7fb011d81640 1 -- 172.21.15.38:0/4115168567 <== osd.1 v2:172.21.15.38:6816/1491818732 23 ==== osd_op_reply(122 1.00000000 [delete] v134'474 uv474 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fb004009b10 con 0x560e2d8dc230 2024-09-21T03:44:31.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.142+0000 7fb018d2cf40 1 -- 172.21.15.38:0/4115168567 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:123 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x560e2d8e2ab0 con 0x560e2d8dc230 2024-09-21T03:44:31.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.143+0000 7fb011d81640 1 -- 172.21.15.38:0/4115168567 <== osd.1 v2:172.21.15.38:6816/1491818732 24 ==== osd_op_reply(123 1.00000000 [create] v134'475 uv475 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fb004009b10 con 0x560e2d8dc230 2024-09-21T03:44:31.146 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-09-21T03:44:31.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.143+0000 7fb018d2cf40 1 -- 172.21.15.38:0/4115168567 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:124 13.1f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x560e2d8e3ac0 con 0x560e2d8dc230 2024-09-21T03:44:31.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.144+0000 7fb011d81640 1 -- 172.21.15.38:0/4115168567 <== osd.1 v2:172.21.15.38:6816/1491818732 25 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v134'476 uv476 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fb004009b10 con 0x560e2d8dc230 2024-09-21T03:44:31.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.145+0000 7fb018d2cf40 1 -- 172.21.15.38:0/4115168567 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:125 13.1f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x560e2d8e3ac0 con 0x560e2d8dc230 2024-09-21T03:44:31.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.146+0000 7fb011d81640 1 -- 172.21.15.38:0/4115168567 <== osd.1 v2:172.21.15.38:6816/1491818732 26 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v134'477 uv477 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fb004009b10 con 0x560e2d8dc230 2024-09-21T03:44:31.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.146+0000 7fb018d2cf40 1 -- 172.21.15.38:0/4115168567 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:126 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x560e2d8e2ab0 con 0x560e2d8dc230 2024-09-21T03:44:31.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.147+0000 7fb011d81640 1 -- 172.21.15.38:0/4115168567 <== osd.1 v2:172.21.15.38:6816/1491818732 27 ==== osd_op_reply(126 1.00000000 [omap-set-header] v134'478 uv478 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fb004009b10 con 0x560e2d8dc230 2024-09-21T03:44:31.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.147+0000 7fb018d2cf40 1 -- 172.21.15.38:0/4115168567 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:127 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=584b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x560e2d8e3210 con 0x560e2d8dc230 2024-09-21T03:44:31.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.148+0000 7fb011d81640 1 -- 172.21.15.38:0/4115168567 <== osd.1 v2:172.21.15.38:6816/1491818732 28 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v134'479 uv479 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fb004009b10 con 0x560e2d8dc230 2024-09-21T03:44:31.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.148+0000 7fb018d2cf40 1 -- 172.21.15.38:0/4115168567 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x560e2d8d3bf0 msgr2=0x560e2d8d6010 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:31.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.148+0000 7fb018d2cf40 1 --2- 172.21.15.38:0/4115168567 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x560e2d8d3bf0 0x560e2d8d6010 crc :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:31.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.148+0000 7fb018d2cf40 1 -- 172.21.15.38:0/4115168567 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x560e2d8dc230 msgr2=0x560e2d8de650 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:31.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.148+0000 7fb018d2cf40 1 --2- 172.21.15.38:0/4115168567 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x560e2d8dc230 0x560e2d8de650 crc :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:31.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.148+0000 7fb018d2cf40 1 -- 172.21.15.38:0/4115168567 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x560e2d8dfe10 msgr2=0x560e2d8e2570 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:31.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.148+0000 7fb018d2cf40 1 --2- 172.21.15.38:0/4115168567 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x560e2d8dfe10 0x560e2d8e2570 crc :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:31.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.148+0000 7fb018d2cf40 1 -- 172.21.15.38:0/4115168567 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x560e2d8d70c0 msgr2=0x560e2d8d94e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:31.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.148+0000 7fb012582640 1 -- 172.21.15.38:0/4115168567 reap_dead start 2024-09-21T03:44:31.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.148+0000 7fb018d2cf40 1 --2- 172.21.15.38:0/4115168567 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x560e2d8d70c0 0x560e2d8d94e0 crc :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:31.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.148+0000 7fb018d2cf40 1 -- 172.21.15.38:0/4115168567 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x560e2d8cfb60 msgr2=0x560e2d8d1f80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:31.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.148+0000 7fb018d2cf40 1 --2- 172.21.15.38:0/4115168567 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x560e2d8cfb60 0x560e2d8d1f80 crc :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:31.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.148+0000 7fb018d2cf40 1 -- 172.21.15.38:0/4115168567 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x560e2d79b7e0 msgr2=0x560e2d824e40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:31.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.148+0000 7fb018d2cf40 1 --2- 172.21.15.38:0/4115168567 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x560e2d79b7e0 0x560e2d824e40 crc :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:31.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.148+0000 7fb018d2cf40 1 -- 172.21.15.38:0/4115168567 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7faff40775a0 msgr2=0x7faff4079ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:31.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.148+0000 7fb018d2cf40 1 --2- 172.21.15.38:0/4115168567 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7faff40775a0 0x7faff4079ab0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fb00400b680 tx=0x7fb00400d040 comp rx=0 tx=0).stop 2024-09-21T03:44:31.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.148+0000 7fb018d2cf40 1 -- 172.21.15.38:0/4115168567 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560e2d813520 msgr2=0x560e2d80aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:31.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.148+0000 7fb018d2cf40 1 --2- 172.21.15.38:0/4115168567 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560e2d813520 0x560e2d80aa30 secure :-1 s=READY pgs=2226 cs=0 l=1 rev1=1 crypto rx=0x7fb00c00aea0 tx=0x7fb00c005750 comp rx=0 tx=0).stop 2024-09-21T03:44:31.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.148+0000 7fb012582640 1 -- 172.21.15.38:0/4115168567 reap_dead start 2024-09-21T03:44:31.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.148+0000 7fb018d2cf40 1 -- 172.21.15.38:0/4115168567 shutdown_connections 2024-09-21T03:44:31.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.148+0000 7fb018d2cf40 1 -- 172.21.15.38:0/4115168567 >> 172.21.15.38:0/4115168567 conn(0x560e2d675ac0 msgr2=0x560e2d80de70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:31.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.148+0000 7fb018d2cf40 1 -- 172.21.15.38:0/4115168567 shutdown_connections 2024-09-21T03:44:31.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.148+0000 7fb018d2cf40 1 -- 172.21.15.38:0/4115168567 wait complete. 2024-09-21T03:44:31.154 DEBUG:teuthology.orchestra.run.smithi038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata setomapheader mds0_sessionmap deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef 2024-09-21T03:44:31.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.249+0000 7f67beed2f40 1 Processor -- start 2024-09-21T03:44:31.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.249+0000 7f67beed2f40 1 -- start start 2024-09-21T03:44:31.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.250+0000 7f67beed2f40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fbde31b240 0x55fbde3d4e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:31.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.250+0000 7f67beed2f40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fbde362090 0x55fbde344620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:31.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.250+0000 7f67beed2f40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55fbde3d7520 0x55fbde344b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:31.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.250+0000 7f67beed2f40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55fbde3eeaa0 con 0x55fbde31b240 2024-09-21T03:44:31.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.250+0000 7f67beed2f40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55fbde268d60 con 0x55fbde362090 2024-09-21T03:44:31.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.250+0000 7f67beed2f40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55fbde3499b0 con 0x55fbde3d7520 2024-09-21T03:44:31.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.250+0000 7f67b898c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fbde31b240 0x55fbde3d4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:31.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.250+0000 7f67b898c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fbde31b240 0x55fbde3d4e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41860/0 (socket says 172.21.15.38:41860) 2024-09-21T03:44:31.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.250+0000 7f67b898c640 1 -- 172.21.15.38:0/2811391924 learned_addr learned my addr 172.21.15.38:0/2811391924 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:44:31.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.250+0000 7f67b818b640 1 --2- 172.21.15.38:0/2811391924 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fbde362090 0x55fbde344620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:31.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.250+0000 7f67b918d640 1 --2- 172.21.15.38:0/2811391924 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55fbde3d7520 0x55fbde344b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:31.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.250+0000 7f67b898c640 1 -- 172.21.15.38:0/2811391924 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55fbde3d7520 msgr2=0x55fbde344b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:44:31.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.250+0000 7f67b898c640 1 --2- 172.21.15.38:0/2811391924 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55fbde3d7520 0x55fbde344b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:31.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.250+0000 7f67b898c640 1 -- 172.21.15.38:0/2811391924 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fbde362090 msgr2=0x55fbde344620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:44:31.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.250+0000 7f67b898c640 1 --2- 172.21.15.38:0/2811391924 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fbde362090 0x55fbde344620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:31.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.250+0000 7f67b898c640 1 -- 172.21.15.38:0/2811391924 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fbde3d6350 con 0x55fbde31b240 2024-09-21T03:44:31.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.251+0000 7f67b898c640 1 --2- 172.21.15.38:0/2811391924 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fbde31b240 0x55fbde3d4e50 secure :-1 s=READY pgs=2228 cs=0 l=1 rev1=1 crypto rx=0x7f67a8057040 tx=0x7f67a8003330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10db88cf3677cbc7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:31.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.251+0000 7f67b7189640 1 -- 172.21.15.38:0/2811391924 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67a8064020 con 0x55fbde31b240 2024-09-21T03:44:31.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.251+0000 7f67b7189640 1 -- 172.21.15.38:0/2811391924 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f67a804daa0 con 0x55fbde31b240 2024-09-21T03:44:31.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.251+0000 7f67b7189640 1 -- 172.21.15.38:0/2811391924 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67a8060470 con 0x55fbde31b240 2024-09-21T03:44:31.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.251+0000 7f67beed2f40 1 -- 172.21.15.38:0/2811391924 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fbde31b240 msgr2=0x55fbde3d4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:31.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.251+0000 7f67beed2f40 1 --2- 172.21.15.38:0/2811391924 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fbde31b240 0x55fbde3d4e50 secure :-1 s=READY pgs=2228 cs=0 l=1 rev1=1 crypto rx=0x7f67a8057040 tx=0x7f67a8003330 comp rx=0 tx=0).stop 2024-09-21T03:44:31.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.251+0000 7f67beed2f40 1 -- 172.21.15.38:0/2811391924 shutdown_connections 2024-09-21T03:44:31.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.251+0000 7f67beed2f40 1 --2- 172.21.15.38:0/2811391924 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55fbde3d7520 0x55fbde344b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:31.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.251+0000 7f67beed2f40 1 --2- 172.21.15.38:0/2811391924 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fbde362090 0x55fbde344620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:31.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.251+0000 7f67beed2f40 1 --2- 172.21.15.38:0/2811391924 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fbde31b240 0x55fbde3d4e50 unknown :-1 s=CLOSED pgs=2228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:31.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.251+0000 7f67beed2f40 1 -- 172.21.15.38:0/2811391924 >> 172.21.15.38:0/2811391924 conn(0x55fbde239ac0 msgr2=0x55fbde316660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:31.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.252+0000 7f67beed2f40 1 -- 172.21.15.38:0/2811391924 shutdown_connections 2024-09-21T03:44:31.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.252+0000 7f67beed2f40 1 -- 172.21.15.38:0/2811391924 wait complete. 2024-09-21T03:44:31.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.252+0000 7f67beed2f40 1 Processor -- start 2024-09-21T03:44:31.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.252+0000 7f67beed2f40 1 -- start start 2024-09-21T03:44:31.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.254+0000 7f67beed2f40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55fbde31b240 0x55fbde3e4920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:31.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.254+0000 7f67beed2f40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fbde362090 0x55fbde3bfb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:31.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.254+0000 7f67beed2f40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fbde3d7520 0x55fbde3ec750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:31.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.254+0000 7f67beed2f40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55fbde3e88f0 con 0x55fbde362090 2024-09-21T03:44:31.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.254+0000 7f67beed2f40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55fbde3e8bb0 con 0x55fbde3d7520 2024-09-21T03:44:31.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.254+0000 7f67beed2f40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55fbde3e8e90 con 0x55fbde31b240 2024-09-21T03:44:31.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.254+0000 7f67b818b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fbde362090 0x55fbde3bfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:31.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.254+0000 7f67b818b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fbde362090 0x55fbde3bfb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41870/0 (socket says 172.21.15.38:41870) 2024-09-21T03:44:31.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.254+0000 7f67b818b640 1 -- 172.21.15.38:0/989666155 learned_addr learned my addr 172.21.15.38:0/989666155 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:44:31.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.254+0000 7f67b898c640 1 --2- 172.21.15.38:0/989666155 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55fbde31b240 0x55fbde3e4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:31.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.254+0000 7f67b918d640 1 --2- 172.21.15.38:0/989666155 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fbde3d7520 0x55fbde3ec750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:31.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.254+0000 7f67b818b640 1 -- 172.21.15.38:0/989666155 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55fbde31b240 msgr2=0x55fbde3e4920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:31.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.254+0000 7f67b818b640 1 --2- 172.21.15.38:0/989666155 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55fbde31b240 0x55fbde3e4920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:31.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.254+0000 7f67b818b640 1 -- 172.21.15.38:0/989666155 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fbde3d7520 msgr2=0x55fbde3ec750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:31.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.254+0000 7f67b818b640 1 --2- 172.21.15.38:0/989666155 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fbde3d7520 0x55fbde3ec750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:31.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.254+0000 7f67b818b640 1 -- 172.21.15.38:0/989666155 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fbde3168b0 con 0x55fbde362090 2024-09-21T03:44:31.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.254+0000 7f67b818b640 1 --2- 172.21.15.38:0/989666155 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fbde362090 0x55fbde3bfb80 secure :-1 s=READY pgs=2229 cs=0 l=1 rev1=1 crypto rx=0x7f67ac008720 tx=0x7f67ac008810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:31.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.254+0000 7f67b6187640 1 -- 172.21.15.38:0/989666155 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67ac00a850 con 0x55fbde362090 2024-09-21T03:44:31.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.254+0000 7f67b6187640 1 -- 172.21.15.38:0/989666155 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f67ac00a9f0 con 0x55fbde362090 2024-09-21T03:44:31.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.255+0000 7f67beed2f40 1 -- 172.21.15.38:0/989666155 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55fbde316b90 con 0x55fbde362090 2024-09-21T03:44:31.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.255+0000 7f67b6187640 1 -- 172.21.15.38:0/989666155 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67ac00ae00 con 0x55fbde362090 2024-09-21T03:44:31.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.255+0000 7f67beed2f40 1 -- 172.21.15.38:0/989666155 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fbde3c03b0 con 0x55fbde362090 2024-09-21T03:44:31.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.256+0000 7f67b6187640 1 -- 172.21.15.38:0/989666155 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f67ac016430 con 0x55fbde362090 2024-09-21T03:44:31.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.257+0000 7f67b6187640 1 --2- 172.21.15.38:0/989666155 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f67a40775a0 0x7f67a4079ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:31.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.257+0000 7f67b6187640 1 -- 172.21.15.38:0/989666155 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 9572+0+0 (secure 0 0 0) 0x7f67ac09b030 con 0x55fbde362090 2024-09-21T03:44:31.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.257+0000 7f67b898c640 1 --2- 172.21.15.38:0/989666155 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f67a40775a0 0x7f67a4079ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:31.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.257+0000 7f67beed2f40 1 --2- 172.21.15.38:0/989666155 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x55fbde3e4f60 0x55fbde3e7f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:31.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.257+0000 7f67beed2f40 1 -- 172.21.15.38:0/989666155 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:1 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=80b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x55fbde493c90 con 0x55fbde3e4f60 2024-09-21T03:44:31.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.257+0000 7f67b918d640 1 --2- 172.21.15.38:0/989666155 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x55fbde3e4f60 0x55fbde3e7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:31.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.258+0000 7f67b898c640 1 --2- 172.21.15.38:0/989666155 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f67a40775a0 0x7f67a4079ab0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f67a8057040 tx=0x7f67a8004cc0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:31.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.258+0000 7f67b918d640 1 --2- 172.21.15.38:0/989666155 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x55fbde3e4f60 0x55fbde3e7f10 crc :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:31.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.259+0000 7f67b918d640 1 -- 172.21.15.38:0/989666155 <== osd.3 v2:172.21.15.38:6808/1286810294 1 ==== osd_op_reply(1 mds0_sessionmap [omap-set-header] v134'171 uv171 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f67b000c790 con 0x55fbde3e4f60 2024-09-21T03:44:31.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.259+0000 7f67beed2f40 1 -- 172.21.15.38:0/989666155 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x55fbde3e4f60 msgr2=0x55fbde3e7f10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:31.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.259+0000 7f67beed2f40 1 --2- 172.21.15.38:0/989666155 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x55fbde3e4f60 0x55fbde3e7f10 crc :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:31.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.259+0000 7f67beed2f40 1 -- 172.21.15.38:0/989666155 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f67a40775a0 msgr2=0x7f67a4079ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:31.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.259+0000 7f67beed2f40 1 --2- 172.21.15.38:0/989666155 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f67a40775a0 0x7f67a4079ab0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f67a8057040 tx=0x7f67a8004cc0 comp rx=0 tx=0).stop 2024-09-21T03:44:31.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.259+0000 7f67beed2f40 1 -- 172.21.15.38:0/989666155 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fbde362090 msgr2=0x55fbde3bfb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:31.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.259+0000 7f67beed2f40 1 --2- 172.21.15.38:0/989666155 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fbde362090 0x55fbde3bfb80 secure :-1 s=READY pgs=2229 cs=0 l=1 rev1=1 crypto rx=0x7f67ac008720 tx=0x7f67ac008810 comp rx=0 tx=0).stop 2024-09-21T03:44:31.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.259+0000 7f67b918d640 1 -- 172.21.15.38:0/989666155 reap_dead start 2024-09-21T03:44:31.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.259+0000 7f67beed2f40 1 -- 172.21.15.38:0/989666155 shutdown_connections 2024-09-21T03:44:31.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.259+0000 7f67beed2f40 1 -- 172.21.15.38:0/989666155 >> 172.21.15.38:0/989666155 conn(0x55fbde239ac0 msgr2=0x55fbde3d6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:31.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.259+0000 7f67beed2f40 1 -- 172.21.15.38:0/989666155 shutdown_connections 2024-09-21T03:44:31.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.259+0000 7f67beed2f40 1 -- 172.21.15.38:0/989666155 wait complete. 2024-09-21T03:44:31.268 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-21T03:44:31.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.385+0000 7efdb0607640 1 -- 172.21.15.38:0/3691621684 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efda80ceb90 msgr2=0x7efda80cefa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:31.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.385+0000 7efdb0607640 1 --2- 172.21.15.38:0/3691621684 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efda80ceb90 0x7efda80cefa0 secure :-1 s=READY pgs=2231 cs=0 l=1 rev1=1 crypto rx=0x7efda0005fd0 tx=0x7efda002a820 comp rx=0 tx=0).stop 2024-09-21T03:44:31.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.385+0000 7efdb0607640 1 -- 172.21.15.38:0/3691621684 shutdown_connections 2024-09-21T03:44:31.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.385+0000 7efdb0607640 1 --2- 172.21.15.38:0/3691621684 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efda80ceb90 0x7efda80cefa0 unknown :-1 s=CLOSED pgs=2231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:31.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.385+0000 7efdb0607640 1 --2- 172.21.15.38:0/3691621684 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efda815ee80 0x7efda80d0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:31.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.385+0000 7efdb0607640 1 -- 172.21.15.38:0/3691621684 >> 172.21.15.38:0/3691621684 conn(0x7efda809d830 msgr2=0x7efda80b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:31.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.385+0000 7efdb0607640 1 -- 172.21.15.38:0/3691621684 shutdown_connections 2024-09-21T03:44:31.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.385+0000 7efdb0607640 1 -- 172.21.15.38:0/3691621684 wait complete. 2024-09-21T03:44:31.390 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.387+0000 7efdb0607640 1 Processor -- start 2024-09-21T03:44:31.390 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.387+0000 7efdb0607640 1 -- start start 2024-09-21T03:44:31.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.388+0000 7efdb0607640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efda80b9ad0 0x7efda80cdaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:31.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.388+0000 7efdb0607640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efda80ceb90 0x7efda806dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:31.394 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.388+0000 7efdb0607640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efda815ee80 0x7efda806e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:31.394 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.388+0000 7efdb0607640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7efda80b6700 con 0x7efda815ee80 2024-09-21T03:44:31.394 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.388+0000 7efdb0607640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7efda80b6580 con 0x7efda80b9ad0 2024-09-21T03:44:31.394 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.388+0000 7efdb0607640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7efda80b6880 con 0x7efda80ceb90 2024-09-21T03:44:31.394 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.388+0000 7efdaeba4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efda815ee80 0x7efda806e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:31.394 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.388+0000 7efdaeba4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efda815ee80 0x7efda806e3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41884/0 (socket says 172.21.15.38:41884) 2024-09-21T03:44:31.394 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.388+0000 7efdaeba4640 1 -- 172.21.15.38:0/1346281786 learned_addr learned my addr 172.21.15.38:0/1346281786 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:44:31.394 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.389+0000 7efdae3a3640 1 --2- 172.21.15.38:0/1346281786 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efda80b9ad0 0x7efda80cdaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:31.394 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.389+0000 7efdadba2640 1 --2- 172.21.15.38:0/1346281786 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efda80ceb90 0x7efda806dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:31.394 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.389+0000 7efdaeba4640 1 -- 172.21.15.38:0/1346281786 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efda80ceb90 msgr2=0x7efda806dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:31.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.389+0000 7efdaeba4640 1 --2- 172.21.15.38:0/1346281786 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efda80ceb90 0x7efda806dea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:31.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.389+0000 7efdaeba4640 1 -- 172.21.15.38:0/1346281786 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efda80b9ad0 msgr2=0x7efda80cdaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:31.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.389+0000 7efdaeba4640 1 --2- 172.21.15.38:0/1346281786 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efda80b9ad0 0x7efda80cdaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:31.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.389+0000 7efdaeba4640 1 -- 172.21.15.38:0/1346281786 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efda80a0d50 con 0x7efda815ee80 2024-09-21T03:44:31.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.389+0000 7efdadba2640 1 --2- 172.21.15.38:0/1346281786 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efda80ceb90 0x7efda806dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:44:31.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.389+0000 7efdaeba4640 1 --2- 172.21.15.38:0/1346281786 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efda815ee80 0x7efda806e3e0 secure :-1 s=READY pgs=2232 cs=0 l=1 rev1=1 crypto rx=0x7efda0009cd0 tx=0x7efda0032d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:31.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.389+0000 7efd9effd640 1 -- 172.21.15.38:0/1346281786 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efda0037040 con 0x7efda815ee80 2024-09-21T03:44:31.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.389+0000 7efd9effd640 1 -- 172.21.15.38:0/1346281786 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efda0004040 con 0x7efda815ee80 2024-09-21T03:44:31.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.389+0000 7efd9effd640 1 -- 172.21.15.38:0/1346281786 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efda00042b0 con 0x7efda815ee80 2024-09-21T03:44:31.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.389+0000 7efdb0607640 1 -- 172.21.15.38:0/1346281786 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efda80a1030 con 0x7efda815ee80 2024-09-21T03:44:31.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.389+0000 7efdb0607640 1 -- 172.21.15.38:0/1346281786 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efda80a15f0 con 0x7efda815ee80 2024-09-21T03:44:31.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.390+0000 7efdb0607640 1 -- 172.21.15.38:0/1346281786 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efda80b6da0 con 0x7efda815ee80 2024-09-21T03:44:31.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.391+0000 7efd9effd640 1 -- 172.21.15.38:0/1346281786 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7efda0044020 con 0x7efda815ee80 2024-09-21T03:44:31.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.391+0000 7efd9effd640 1 --2- 172.21.15.38:0/1346281786 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7efd8c078bc0 0x7efd8c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:31.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.392+0000 7efd9effd640 1 -- 172.21.15.38:0/1346281786 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 9572+0+0 (secure 0 0 0) 0x7efda00bc1f0 con 0x7efda815ee80 2024-09-21T03:44:31.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.392+0000 7efdae3a3640 1 --2- 172.21.15.38:0/1346281786 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7efd8c078bc0 0x7efd8c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:31.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.392+0000 7efdae3a3640 1 --2- 172.21.15.38:0/1346281786 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7efd8c078bc0 0x7efd8c07b0d0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7efda400bb50 tx=0x7efda400b2e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:31.399 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.396+0000 7efd9effd640 1 -- 172.21.15.38:0/1346281786 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7efda004b020 con 0x7efda815ee80 2024-09-21T03:44:31.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.622+0000 7efdb0607640 1 -- 172.21.15.38:0/1346281786 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7efda80d2c80 con 0x7efda815ee80 2024-09-21T03:44:31.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.907+0000 7efd9effd640 1 -- 172.21.15.38:0/1346281786 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v323) ==== 162+0+0 (secure 0 0 0) 0x7efda008ae20 con 0x7efda815ee80 2024-09-21T03:44:31.910 INFO:teuthology.orchestra.run.smithi038.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-21T03:44:31.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.909+0000 7efdb0607640 1 -- 172.21.15.38:0/1346281786 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7efd8c078bc0 msgr2=0x7efd8c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:31.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.909+0000 7efdb0607640 1 --2- 172.21.15.38:0/1346281786 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7efd8c078bc0 0x7efd8c07b0d0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7efda400bb50 tx=0x7efda400b2e0 comp rx=0 tx=0).stop 2024-09-21T03:44:31.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.909+0000 7efdb0607640 1 -- 172.21.15.38:0/1346281786 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efda815ee80 msgr2=0x7efda806e3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:31.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.909+0000 7efdb0607640 1 --2- 172.21.15.38:0/1346281786 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efda815ee80 0x7efda806e3e0 secure :-1 s=READY pgs=2232 cs=0 l=1 rev1=1 crypto rx=0x7efda0009cd0 tx=0x7efda0032d10 comp rx=0 tx=0).stop 2024-09-21T03:44:31.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.909+0000 7efdb0607640 1 -- 172.21.15.38:0/1346281786 shutdown_connections 2024-09-21T03:44:31.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.909+0000 7efdb0607640 1 --2- 172.21.15.38:0/1346281786 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efda815ee80 0x7efda806e3e0 unknown :-1 s=CLOSED pgs=2232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:31.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.909+0000 7efdb0607640 1 --2- 172.21.15.38:0/1346281786 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7efd8c078bc0 0x7efd8c07b0d0 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:31.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.909+0000 7efdb0607640 1 --2- 172.21.15.38:0/1346281786 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efda80ceb90 0x7efda806dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:31.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.909+0000 7efdb0607640 1 --2- 172.21.15.38:0/1346281786 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efda80b9ad0 0x7efda80cdaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:31.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.909+0000 7efdb0607640 1 -- 172.21.15.38:0/1346281786 >> 172.21.15.38:0/1346281786 conn(0x7efda809d830 msgr2=0x7efda80b5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:31.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.909+0000 7efdb0607640 1 -- 172.21.15.38:0/1346281786 shutdown_connections 2024-09-21T03:44:31.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:31.909+0000 7efdb0607640 1 -- 172.21.15.38:0/1346281786 wait complete. 2024-09-21T03:44:31.922 INFO:tasks.ceph.mds.c.smithi038.stderr:2024-09-21T03:44:31.919+0000 7fa1c2718640 -1 log_channel(cluster) log [ERR] : corrupt sessionmap header: Decoder at 'virtual void SessionMapStore::decode_header(ceph::bufferlist&)' v=1 cannot decode v=100 minimal_decoder=101: Malformed input [buffer:3] 2024-09-21T03:44:31.923 INFO:tasks.ceph.mon.a.smithi038.stderr:2024-09-21T03:44:31.920+0000 7f3f45713640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T03:44:31.923 INFO:tasks.ceph.mon.a.smithi038.stderr:2024-09-21T03:44:31.920+0000 7f3f45713640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-09-21T03:44:31.926 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:44:32.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:32.040+0000 7fac855c6640 1 -- 172.21.15.38:0/2759934821 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fac800d19d0 msgr2=0x7fac8015c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:32.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:32.040+0000 7fac855c6640 1 --2- 172.21.15.38:0/2759934821 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fac800d19d0 0x7fac8015c6a0 secure :-1 s=READY pgs=2234 cs=0 l=1 rev1=1 crypto rx=0x7fac70009830 tx=0x7fac7002a510 comp rx=0 tx=0).stop 2024-09-21T03:44:32.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:32.040+0000 7fac855c6640 1 -- 172.21.15.38:0/2759934821 shutdown_connections 2024-09-21T03:44:32.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:32.040+0000 7fac855c6640 1 --2- 172.21.15.38:0/2759934821 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fac800cd1d0 0x7fac800b5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:32.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:32.040+0000 7fac855c6640 1 --2- 172.21.15.38:0/2759934821 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fac800d19d0 0x7fac8015c6a0 unknown :-1 s=CLOSED pgs=2234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:32.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:32.040+0000 7fac855c6640 1 -- 172.21.15.38:0/2759934821 >> 172.21.15.38:0/2759934821 conn(0x7fac800b41d0 msgr2=0x7fac800b45e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:32.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:32.040+0000 7fac855c6640 1 -- 172.21.15.38:0/2759934821 shutdown_connections 2024-09-21T03:44:32.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:32.040+0000 7fac855c6640 1 -- 172.21.15.38:0/2759934821 wait complete. 2024-09-21T03:44:32.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:32.042+0000 7fac855c6640 1 Processor -- start 2024-09-21T03:44:32.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:32.042+0000 7fac855c6640 1 -- start start 2024-09-21T03:44:32.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:32.043+0000 7fac855c6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fac800b2500 0x7fac800cbf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:32.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:32.043+0000 7fac855c6640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fac800d19d0 0x7fac801e6ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:32.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:32.043+0000 7fac855c6640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fac800c9310 0x7fac801e7430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:32.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:32.043+0000 7fac855c6640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fac800b6020 con 0x7fac800b2500 2024-09-21T03:44:32.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:32.043+0000 7fac855c6640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fac800b5ea0 con 0x7fac800c9310 2024-09-21T03:44:32.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:32.043+0000 7fac855c6640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fac800b61a0 con 0x7fac800d19d0 2024-09-21T03:44:32.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:32.044+0000 7fac7effd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fac800b2500 0x7fac800cbf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:32.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:32.044+0000 7fac7effd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fac800b2500 0x7fac800cbf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41910/0 (socket says 172.21.15.38:41910) 2024-09-21T03:44:32.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:32.044+0000 7fac7effd640 1 -- 172.21.15.38:0/2941535285 learned_addr learned my addr 172.21.15.38:0/2941535285 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:44:32.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:32.044+0000 7fac7f7fe640 1 --2- 172.21.15.38:0/2941535285 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fac800c9310 0x7fac801e7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:32.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:32.044+0000 7fac7e7fc640 1 --2- 172.21.15.38:0/2941535285 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fac800d19d0 0x7fac801e6ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:32.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:32.044+0000 7fac7effd640 1 -- 172.21.15.38:0/2941535285 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fac800d19d0 msgr2=0x7fac801e6ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:32.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:32.044+0000 7fac7effd640 1 --2- 172.21.15.38:0/2941535285 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fac800d19d0 0x7fac801e6ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:32.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:32.044+0000 7fac7effd640 1 -- 172.21.15.38:0/2941535285 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fac800c9310 msgr2=0x7fac801e7430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:32.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:32.044+0000 7fac7effd640 1 --2- 172.21.15.38:0/2941535285 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fac800c9310 0x7fac801e7430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:32.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:32.044+0000 7fac7effd640 1 -- 172.21.15.38:0/2941535285 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac800c6270 con 0x7fac800b2500 2024-09-21T03:44:32.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:32.044+0000 7fac7f7fe640 1 --2- 172.21.15.38:0/2941535285 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fac800c9310 0x7fac801e7430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:44:32.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:32.044+0000 7fac7e7fc640 1 --2- 172.21.15.38:0/2941535285 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fac800d19d0 0x7fac801e6ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:44:32.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:32.044+0000 7fac7effd640 1 --2- 172.21.15.38:0/2941535285 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fac800b2500 0x7fac800cbf90 secure :-1 s=READY pgs=2235 cs=0 l=1 rev1=1 crypto rx=0x7fac70000c00 tx=0x7fac70033d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:32.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:32.044+0000 7fac777fe640 1 -- 172.21.15.38:0/2941535285 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac70031d60 con 0x7fac800b2500 2024-09-21T03:44:32.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:32.044+0000 7fac777fe640 1 -- 172.21.15.38:0/2941535285 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fac700330d0 con 0x7fac800b2500 2024-09-21T03:44:32.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:32.044+0000 7fac855c6640 1 -- 172.21.15.38:0/2941535285 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac800c66f0 con 0x7fac800b2500 2024-09-21T03:44:32.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:32.044+0000 7fac777fe640 1 -- 172.21.15.38:0/2941535285 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac70033520 con 0x7fac800b2500 2024-09-21T03:44:32.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:32.044+0000 7fac855c6640 1 -- 172.21.15.38:0/2941535285 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac800c6cb0 con 0x7fac800b2500 2024-09-21T03:44:32.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:32.045+0000 7fac855c6640 1 -- 172.21.15.38:0/2941535285 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac800b6650 con 0x7fac800b2500 2024-09-21T03:44:32.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:32.046+0000 7fac777fe640 1 -- 172.21.15.38:0/2941535285 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fac700035f0 con 0x7fac800b2500 2024-09-21T03:44:32.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:32.047+0000 7fac777fe640 1 --2- 172.21.15.38:0/2941535285 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fac60078bc0 0x7fac6007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:32.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:32.047+0000 7fac777fe640 1 -- 172.21.15.38:0/2941535285 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(135..135 src has 1..135) ==== 9658+0+0 (secure 0 0 0) 0x7fac700bd310 con 0x7fac800b2500 2024-09-21T03:44:32.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:32.047+0000 7fac7e7fc640 1 --2- 172.21.15.38:0/2941535285 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fac60078bc0 0x7fac6007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:32.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:32.048+0000 7fac7e7fc640 1 --2- 172.21.15.38:0/2941535285 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fac60078bc0 0x7fac6007b0d0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fac800ca560 tx=0x7fac68009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:32.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:32.051+0000 7fac777fe640 1 -- 172.21.15.38:0/2941535285 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fac7003f030 con 0x7fac800b2500 2024-09-21T03:44:32.214 DEBUG:teuthology.orchestra.run.smithi038:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T03:44:32.217 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T03:44:32.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:32.277+0000 7fac855c6640 1 -- 172.21.15.38:0/2941535285 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fac800b4780 con 0x7fac800b2500 2024-09-21T03:44:32.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:32.277+0000 7fac777fe640 1 -- 172.21.15.38:0/2941535285 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 325 v325) ==== 95+0+4392 (secure 0 0 0) 0x7fac700840e0 con 0x7fac800b2500 2024-09-21T03:44:32.281 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:44:32.281 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":325,"btime":"2024-09-21T03:44:31:922134+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6086,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/965972663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":965972663},{"type":"v1","addr":"172.21.15.195:6836","nonce":965972663}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6700,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/3176122334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":3176122334},{"type":"v1","addr":"172.21.15.195:6837","nonce":3176122334}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":9840,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/3801772046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":3801772046},{"type":"v1","addr":"172.21.15.38:6835","nonce":3801772046}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":273}],"filesystems":[{"mdsmap":{"epoch":325,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:40:02.319282+0000","modified":"2024-09-21T03:44:31.922072+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":135,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-21T03:44:32.281 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 325 2024-09-21T03:44:32.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:32.280+0000 7fac855c6640 1 -- 172.21.15.38:0/2941535285 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fac60078bc0 msgr2=0x7fac6007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:32.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:32.280+0000 7fac855c6640 1 --2- 172.21.15.38:0/2941535285 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fac60078bc0 0x7fac6007b0d0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fac800ca560 tx=0x7fac68009040 comp rx=0 tx=0).stop 2024-09-21T03:44:32.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:32.280+0000 7fac855c6640 1 -- 172.21.15.38:0/2941535285 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fac800b2500 msgr2=0x7fac800cbf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:32.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:32.280+0000 7fac855c6640 1 --2- 172.21.15.38:0/2941535285 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fac800b2500 0x7fac800cbf90 secure :-1 s=READY pgs=2235 cs=0 l=1 rev1=1 crypto rx=0x7fac70000c00 tx=0x7fac70033d60 comp rx=0 tx=0).stop 2024-09-21T03:44:32.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:32.280+0000 7fac855c6640 1 -- 172.21.15.38:0/2941535285 shutdown_connections 2024-09-21T03:44:32.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:32.280+0000 7fac855c6640 1 --2- 172.21.15.38:0/2941535285 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fac800c9310 0x7fac801e7430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:32.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:32.280+0000 7fac855c6640 1 --2- 172.21.15.38:0/2941535285 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fac60078bc0 0x7fac6007b0d0 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:32.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:32.280+0000 7fac855c6640 1 --2- 172.21.15.38:0/2941535285 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fac800d19d0 0x7fac801e6ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:32.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:32.280+0000 7fac855c6640 1 --2- 172.21.15.38:0/2941535285 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fac800b2500 0x7fac800cbf90 unknown :-1 s=CLOSED pgs=2235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:32.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:32.280+0000 7fac855c6640 1 -- 172.21.15.38:0/2941535285 >> 172.21.15.38:0/2941535285 conn(0x7fac800b41d0 msgr2=0x7fac80070200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:32.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:32.280+0000 7fac855c6640 1 -- 172.21.15.38:0/2941535285 shutdown_connections 2024-09-21T03:44:32.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:32.280+0000 7fac855c6640 1 -- 172.21.15.38:0/2941535285 wait complete. 2024-09-21T03:44:32.294 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-21T03:44:32.295 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:44:32.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:32.449+0000 7ff2ac762640 1 -- 172.21.15.38:0/1128842370 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff2a4162280 msgr2=0x7ff2a4162690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:32.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:32.449+0000 7ff2ac762640 1 --2- 172.21.15.38:0/1128842370 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff2a4162280 0x7ff2a4162690 secure :-1 s=READY pgs=2236 cs=0 l=1 rev1=1 crypto rx=0x7ff2a40b7620 tx=0x7ff29402a560 comp rx=0 tx=0).stop 2024-09-21T03:44:32.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:32.449+0000 7ff2ac762640 1 -- 172.21.15.38:0/1128842370 shutdown_connections 2024-09-21T03:44:32.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:32.449+0000 7ff2ac762640 1 --2- 172.21.15.38:0/1128842370 >> v2:172.21.15.195:3300/0 conn(0x7ff2a40b2e40 0x7ff2a40b3250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:32.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:32.449+0000 7ff2ac762640 1 --2- 172.21.15.38:0/1128842370 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff2a4162280 0x7ff2a4162690 unknown :-1 s=CLOSED pgs=2236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:32.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:32.449+0000 7ff2ac762640 1 --2- 172.21.15.38:0/1128842370 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff2a4162f00 0x7ff2a40d24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:32.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:32.449+0000 7ff2ac762640 1 -- 172.21.15.38:0/1128842370 >> 172.21.15.38:0/1128842370 conn(0x7ff2a40b6b20 msgr2=0x7ff2a40d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:32.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:32.449+0000 7ff2ac762640 1 -- 172.21.15.38:0/1128842370 shutdown_connections 2024-09-21T03:44:32.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:32.449+0000 7ff2ac762640 1 -- 172.21.15.38:0/1128842370 wait complete. 2024-09-21T03:44:32.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:32.451+0000 7ff2ac762640 1 Processor -- start 2024-09-21T03:44:32.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:32.451+0000 7ff2ac762640 1 -- start start 2024-09-21T03:44:32.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:32.452+0000 7ff2ac762640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff2a40b2e40 0x7ff2a41eaac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:32.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:32.452+0000 7ff2ac762640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff2a4162280 0x7ff2a41f8a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:32.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:32.452+0000 7ff2ac762640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff2a41f9640 0x7ff2a41f8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:32.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:32.452+0000 7ff2ac762640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ff2a40b3dd0 con 0x7ff2a4162280 2024-09-21T03:44:32.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:32.452+0000 7ff2ac762640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ff2a40b39a0 con 0x7ff2a41f9640 2024-09-21T03:44:32.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:32.452+0000 7ff2ac762640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7ff2a40b3f50 con 0x7ff2a40b2e40 2024-09-21T03:44:32.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:32.452+0000 7ff2a9cfd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff2a4162280 0x7ff2a41f8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:32.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:32.452+0000 7ff2a9cfd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff2a4162280 0x7ff2a41f8a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49932/0 (socket says 172.21.15.38:49932) 2024-09-21T03:44:32.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:32.452+0000 7ff2a9cfd640 1 -- 172.21.15.38:0/863326355 learned_addr learned my addr 172.21.15.38:0/863326355 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:44:32.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:32.453+0000 7ff2aa4fe640 1 --2- 172.21.15.38:0/863326355 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff2a40b2e40 0x7ff2a41eaac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:32.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:32.453+0000 7ff2a9cfd640 1 -- 172.21.15.38:0/863326355 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff2a40b2e40 msgr2=0x7ff2a41eaac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:32.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:32.453+0000 7ff2aacff640 1 --2- 172.21.15.38:0/863326355 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff2a41f9640 0x7ff2a41f8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:32.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:32.453+0000 7ff2a9cfd640 1 --2- 172.21.15.38:0/863326355 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff2a40b2e40 0x7ff2a41eaac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:32.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:32.453+0000 7ff2a9cfd640 1 -- 172.21.15.38:0/863326355 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff2a41f9640 msgr2=0x7ff2a41f8fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:32.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:32.453+0000 7ff2a9cfd640 1 --2- 172.21.15.38:0/863326355 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff2a41f9640 0x7ff2a41f8fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:32.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:32.453+0000 7ff2a9cfd640 1 -- 172.21.15.38:0/863326355 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2a414be80 con 0x7ff2a4162280 2024-09-21T03:44:32.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:32.453+0000 7ff2aa4fe640 1 --2- 172.21.15.38:0/863326355 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff2a40b2e40 0x7ff2a41eaac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:44:32.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:32.453+0000 7ff2aacff640 1 --2- 172.21.15.38:0/863326355 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff2a41f9640 0x7ff2a41f8fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:44:32.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:32.453+0000 7ff2a9cfd640 1 --2- 172.21.15.38:0/863326355 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff2a4162280 0x7ff2a41f8a80 secure :-1 s=READY pgs=2237 cs=0 l=1 rev1=1 crypto rx=0x7ff294034040 tx=0x7ff2940039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:32.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:32.453+0000 7ff29affd640 1 -- 172.21.15.38:0/863326355 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff29403e030 con 0x7ff2a4162280 2024-09-21T03:44:32.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:32.453+0000 7ff29affd640 1 -- 172.21.15.38:0/863326355 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff2940047a0 con 0x7ff2a4162280 2024-09-21T03:44:32.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:32.453+0000 7ff29affd640 1 -- 172.21.15.38:0/863326355 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff294035040 con 0x7ff2a4162280 2024-09-21T03:44:32.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:32.453+0000 7ff2ac762640 1 -- 172.21.15.38:0/863326355 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2a414c160 con 0x7ff2a4162280 2024-09-21T03:44:32.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:32.453+0000 7ff2ac762640 1 -- 172.21.15.38:0/863326355 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2a40cc640 con 0x7ff2a4162280 2024-09-21T03:44:32.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:32.454+0000 7ff2ac762640 1 -- 172.21.15.38:0/863326355 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2a40caec0 con 0x7ff2a4162280 2024-09-21T03:44:32.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:32.455+0000 7ff29affd640 1 -- 172.21.15.38:0/863326355 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ff294041030 con 0x7ff2a4162280 2024-09-21T03:44:32.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:32.455+0000 7ff29affd640 1 --2- 172.21.15.38:0/863326355 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff288078b90 0x7ff28807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:32.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:32.455+0000 7ff29affd640 1 -- 172.21.15.38:0/863326355 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(135..135 src has 1..135) ==== 9658+0+0 (secure 0 0 0) 0x7ff2940bc4f0 con 0x7ff2a4162280 2024-09-21T03:44:32.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:32.456+0000 7ff2aa4fe640 1 --2- 172.21.15.38:0/863326355 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff288078b90 0x7ff28807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:32.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:32.456+0000 7ff2aa4fe640 1 --2- 172.21.15.38:0/863326355 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff288078b90 0x7ff28807b0a0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7ff2a000a870 tx=0x7ff2a0008040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:32.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:32.460+0000 7ff29affd640 1 -- 172.21.15.38:0/863326355 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7ff294083a90 con 0x7ff2a4162280 2024-09-21T03:44:32.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:32.683+0000 7ff2ac762640 1 -- 172.21.15.38:0/863326355 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff2a4162f00 con 0x7ff2a4162280 2024-09-21T03:44:32.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:32.683+0000 7ff29affd640 1 -- 172.21.15.38:0/863326355 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 325 v325) ==== 95+0+4392 (secure 0 0 0) 0x7ff294083390 con 0x7ff2a4162280 2024-09-21T03:44:32.687 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:44:32.687 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":325,"btime":"2024-09-21T03:44:31:922134+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6086,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/965972663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":965972663},{"type":"v1","addr":"172.21.15.195:6836","nonce":965972663}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6700,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/3176122334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":3176122334},{"type":"v1","addr":"172.21.15.195:6837","nonce":3176122334}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":9840,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/3801772046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":3801772046},{"type":"v1","addr":"172.21.15.38:6835","nonce":3801772046}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":273}],"filesystems":[{"mdsmap":{"epoch":325,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:40:02.319282+0000","modified":"2024-09-21T03:44:31.922072+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":135,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-21T03:44:32.687 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 325 2024-09-21T03:44:32.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:32.686+0000 7ff2ac762640 1 -- 172.21.15.38:0/863326355 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff288078b90 msgr2=0x7ff28807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:32.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:32.686+0000 7ff2ac762640 1 --2- 172.21.15.38:0/863326355 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff288078b90 0x7ff28807b0a0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7ff2a000a870 tx=0x7ff2a0008040 comp rx=0 tx=0).stop 2024-09-21T03:44:32.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:32.686+0000 7ff2ac762640 1 -- 172.21.15.38:0/863326355 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff2a4162280 msgr2=0x7ff2a41f8a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:32.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:32.686+0000 7ff2ac762640 1 --2- 172.21.15.38:0/863326355 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff2a4162280 0x7ff2a41f8a80 secure :-1 s=READY pgs=2237 cs=0 l=1 rev1=1 crypto rx=0x7ff294034040 tx=0x7ff2940039e0 comp rx=0 tx=0).stop 2024-09-21T03:44:32.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:32.686+0000 7ff2ac762640 1 -- 172.21.15.38:0/863326355 shutdown_connections 2024-09-21T03:44:32.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:32.686+0000 7ff2ac762640 1 --2- 172.21.15.38:0/863326355 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff2a41f9640 0x7ff2a41f8fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:32.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:32.686+0000 7ff2ac762640 1 --2- 172.21.15.38:0/863326355 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff2a4162280 0x7ff2a41f8a80 unknown :-1 s=CLOSED pgs=2237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:32.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:32.686+0000 7ff2ac762640 1 --2- 172.21.15.38:0/863326355 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff288078b90 0x7ff28807b0a0 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:32.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:32.686+0000 7ff2ac762640 1 --2- 172.21.15.38:0/863326355 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff2a40b2e40 0x7ff2a41eaac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:32.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:32.686+0000 7ff2ac762640 1 -- 172.21.15.38:0/863326355 >> 172.21.15.38:0/863326355 conn(0x7ff2a40b6b20 msgr2=0x7ff2a406dde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:32.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:32.686+0000 7ff2ac762640 1 -- 172.21.15.38:0/863326355 shutdown_connections 2024-09-21T03:44:32.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:32.686+0000 7ff2ac762640 1 -- 172.21.15.38:0/863326355 wait complete. 2024-09-21T03:44:32.700 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-21T03:44:32.700 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:44:32.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:32.813+0000 7fc363b5e640 1 -- 172.21.15.38:0/2734650945 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc35c0ce0d0 msgr2=0x7fc35c073e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:32.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:32.813+0000 7fc363b5e640 1 --2- 172.21.15.38:0/2734650945 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc35c0ce0d0 0x7fc35c073e20 secure :-1 s=READY pgs=2239 cs=0 l=1 rev1=1 crypto rx=0x7fc35c072840 tx=0x7fc35002a530 comp rx=0 tx=0).stop 2024-09-21T03:44:32.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:32.813+0000 7fc363b5e640 1 -- 172.21.15.38:0/2734650945 shutdown_connections 2024-09-21T03:44:32.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:32.813+0000 7fc363b5e640 1 --2- 172.21.15.38:0/2734650945 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc35c0ce0d0 0x7fc35c073e20 unknown :-1 s=CLOSED pgs=2239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:32.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:32.813+0000 7fc363b5e640 1 --2- 172.21.15.38:0/2734650945 >> v2:172.21.15.195:3300/0 conn(0x7fc35c0ca610 0x7fc35c0cdb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:32.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:32.813+0000 7fc363b5e640 1 -- 172.21.15.38:0/2734650945 >> 172.21.15.38:0/2734650945 conn(0x7fc35c0cca40 msgr2=0x7fc35c0cce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:32.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:32.813+0000 7fc363b5e640 1 -- 172.21.15.38:0/2734650945 shutdown_connections 2024-09-21T03:44:32.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:32.813+0000 7fc363b5e640 1 -- 172.21.15.38:0/2734650945 wait complete. 2024-09-21T03:44:32.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:32.815+0000 7fc363b5e640 1 Processor -- start 2024-09-21T03:44:32.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:32.815+0000 7fc363b5e640 1 -- start start 2024-09-21T03:44:32.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:32.816+0000 7fc363b5e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc35c074360 0x7fc35c09d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:32.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:32.816+0000 7fc363b5e640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc35c0ca610 0x7fc35c0b3e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:32.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:32.816+0000 7fc363b5e640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc35c0ce0d0 0x7fc35c0a0390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:32.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:32.816+0000 7fc363b5e640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fc35c15baa0 con 0x7fc35c074360 2024-09-21T03:44:32.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:32.816+0000 7fc363b5e640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fc35c15b920 con 0x7fc35c0ce0d0 2024-09-21T03:44:32.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:32.816+0000 7fc363b5e640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fc35c15bc20 con 0x7fc35c0ca610 2024-09-21T03:44:32.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:32.816+0000 7fc3618fa640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc35c074360 0x7fc35c09d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:32.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:32.816+0000 7fc3618fa640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc35c074360 0x7fc35c09d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49954/0 (socket says 172.21.15.38:49954) 2024-09-21T03:44:32.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:32.816+0000 7fc3618fa640 1 -- 172.21.15.38:0/3981531660 learned_addr learned my addr 172.21.15.38:0/3981531660 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:44:32.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:32.817+0000 7fc3620fb640 1 --2- 172.21.15.38:0/3981531660 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc35c0ce0d0 0x7fc35c0a0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:32.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:32.817+0000 7fc3610f9640 1 --2- 172.21.15.38:0/3981531660 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc35c0ca610 0x7fc35c0b3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:32.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:32.817+0000 7fc3618fa640 1 -- 172.21.15.38:0/3981531660 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc35c0ca610 msgr2=0x7fc35c0b3e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:32.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:32.817+0000 7fc3618fa640 1 --2- 172.21.15.38:0/3981531660 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc35c0ca610 0x7fc35c0b3e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:32.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:32.817+0000 7fc3618fa640 1 -- 172.21.15.38:0/3981531660 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc35c0ce0d0 msgr2=0x7fc35c0a0390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:32.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:32.817+0000 7fc3618fa640 1 --2- 172.21.15.38:0/3981531660 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc35c0ce0d0 0x7fc35c0a0390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:32.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:32.817+0000 7fc3618fa640 1 -- 172.21.15.38:0/3981531660 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc35c15e9c0 con 0x7fc35c074360 2024-09-21T03:44:32.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:32.817+0000 7fc3610f9640 1 --2- 172.21.15.38:0/3981531660 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc35c0ca610 0x7fc35c0b3e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:44:32.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:32.817+0000 7fc3618fa640 1 --2- 172.21.15.38:0/3981531660 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc35c074360 0x7fc35c09d640 secure :-1 s=READY pgs=2240 cs=0 l=1 rev1=1 crypto rx=0x7fc34c00ea20 tx=0x7fc34c00eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:32.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:32.817+0000 7fc35a7fc640 1 -- 172.21.15.38:0/3981531660 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc34c00cde0 con 0x7fc35c074360 2024-09-21T03:44:32.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:32.817+0000 7fc35a7fc640 1 -- 172.21.15.38:0/3981531660 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc34c011470 con 0x7fc35c074360 2024-09-21T03:44:32.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:32.817+0000 7fc363b5e640 1 -- 172.21.15.38:0/3981531660 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc35c15eca0 con 0x7fc35c074360 2024-09-21T03:44:32.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:32.817+0000 7fc35a7fc640 1 -- 172.21.15.38:0/3981531660 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc34c0118a0 con 0x7fc35c074360 2024-09-21T03:44:32.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:32.817+0000 7fc363b5e640 1 -- 172.21.15.38:0/3981531660 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc35c09d120 con 0x7fc35c074360 2024-09-21T03:44:32.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:32.818+0000 7fc363b5e640 1 -- 172.21.15.38:0/3981531660 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc35c071760 con 0x7fc35c074360 2024-09-21T03:44:32.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:32.819+0000 7fc35a7fc640 1 -- 172.21.15.38:0/3981531660 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc34c01a400 con 0x7fc35c074360 2024-09-21T03:44:32.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:32.819+0000 7fc35a7fc640 1 --2- 172.21.15.38:0/3981531660 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc348078bc0 0x7fc34807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:32.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:32.819+0000 7fc35a7fc640 1 -- 172.21.15.38:0/3981531660 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(135..135 src has 1..135) ==== 9658+0+0 (secure 0 0 0) 0x7fc34c09c920 con 0x7fc35c074360 2024-09-21T03:44:32.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:32.820+0000 7fc3610f9640 1 --2- 172.21.15.38:0/3981531660 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc348078bc0 0x7fc34807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:32.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:32.820+0000 7fc3610f9640 1 --2- 172.21.15.38:0/3981531660 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc348078bc0 0x7fc34807b0d0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fc350034000 tx=0x7fc350004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:32.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:32.824+0000 7fc35a7fc640 1 -- 172.21.15.38:0/3981531660 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fc35c071760 con 0x7fc35c074360 2024-09-21T03:44:33.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:33.052+0000 7fc363b5e640 1 -- 172.21.15.38:0/3981531660 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc35c075050 con 0x7fc35c074360 2024-09-21T03:44:33.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:33.052+0000 7fc35a7fc640 1 -- 172.21.15.38:0/3981531660 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 325 v325) ==== 95+0+4392 (secure 0 0 0) 0x7fc34c016030 con 0x7fc35c074360 2024-09-21T03:44:33.055 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:44:33.055 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":325,"btime":"2024-09-21T03:44:31:922134+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6086,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/965972663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":965972663},{"type":"v1","addr":"172.21.15.195:6836","nonce":965972663}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6700,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/3176122334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":3176122334},{"type":"v1","addr":"172.21.15.195:6837","nonce":3176122334}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":9840,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/3801772046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":3801772046},{"type":"v1","addr":"172.21.15.38:6835","nonce":3801772046}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":273}],"filesystems":[{"mdsmap":{"epoch":325,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:40:02.319282+0000","modified":"2024-09-21T03:44:31.922072+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":135,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-21T03:44:33.056 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 325 2024-09-21T03:44:33.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:33.054+0000 7fc363b5e640 1 -- 172.21.15.38:0/3981531660 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc348078bc0 msgr2=0x7fc34807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:33.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:33.054+0000 7fc363b5e640 1 --2- 172.21.15.38:0/3981531660 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc348078bc0 0x7fc34807b0d0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fc350034000 tx=0x7fc350004730 comp rx=0 tx=0).stop 2024-09-21T03:44:33.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:33.054+0000 7fc363b5e640 1 -- 172.21.15.38:0/3981531660 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc35c074360 msgr2=0x7fc35c09d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:33.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:33.054+0000 7fc363b5e640 1 --2- 172.21.15.38:0/3981531660 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc35c074360 0x7fc35c09d640 secure :-1 s=READY pgs=2240 cs=0 l=1 rev1=1 crypto rx=0x7fc34c00ea20 tx=0x7fc34c00eee0 comp rx=0 tx=0).stop 2024-09-21T03:44:33.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:33.054+0000 7fc363b5e640 1 -- 172.21.15.38:0/3981531660 shutdown_connections 2024-09-21T03:44:33.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:33.054+0000 7fc363b5e640 1 --2- 172.21.15.38:0/3981531660 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc35c0ce0d0 0x7fc35c0a0390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:33.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:33.054+0000 7fc363b5e640 1 --2- 172.21.15.38:0/3981531660 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc348078bc0 0x7fc34807b0d0 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:33.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:33.054+0000 7fc363b5e640 1 --2- 172.21.15.38:0/3981531660 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc35c0ca610 0x7fc35c0b3e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:33.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:33.054+0000 7fc363b5e640 1 --2- 172.21.15.38:0/3981531660 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc35c074360 0x7fc35c09d640 unknown :-1 s=CLOSED pgs=2240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:33.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:33.054+0000 7fc363b5e640 1 -- 172.21.15.38:0/3981531660 >> 172.21.15.38:0/3981531660 conn(0x7fc35c0cca40 msgr2=0x7fc35c0afae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:33.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:33.054+0000 7fc363b5e640 1 -- 172.21.15.38:0/3981531660 shutdown_connections 2024-09-21T03:44:33.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:33.054+0000 7fc363b5e640 1 -- 172.21.15.38:0/3981531660 wait complete. 2024-09-21T03:44:33.069 INFO:tasks.cephfs.test_damage:Result: Mutation 'Corrupt omap header on mds0_sessionmap' led to DAMAGED state 2024-09-21T03:44:33.069 INFO:tasks.cephfs.test_damage:Applying mutation 'Corrupt omap header on 1.00000000' 2024-09-21T03:44:33.069 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-21T03:44:33.069 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:44:33.085 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056795,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2354 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T03:44:33.085 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi038.front.sepia.ceph.com 2024-09-21T03:44:33.085 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:44:33.085 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T03:44:33.141 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T03:44:33.141 INFO:teuthology.orchestra.run.smithi038.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-09-21T03:44:33.142 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-21T03:44:33.299 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:33.296+0000 7f5e85c55640 1 -- 172.21.15.38:0/2155021418 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5e8015c740 msgr2=0x7f5e800b3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:33.299 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:33.296+0000 7f5e85c55640 1 --2- 172.21.15.38:0/2155021418 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5e8015c740 0x7f5e800b3740 secure :-1 s=READY pgs=2242 cs=0 l=1 rev1=1 crypto rx=0x7f5e800733a0 tx=0x7f5e7002a530 comp rx=0 tx=0).stop 2024-09-21T03:44:33.299 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:33.296+0000 7f5e85c55640 1 -- 172.21.15.38:0/2155021418 shutdown_connections 2024-09-21T03:44:33.299 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:33.296+0000 7f5e85c55640 1 --2- 172.21.15.38:0/2155021418 >> v2:172.21.15.195:3300/0 conn(0x7f5e800740f0 0x7f5e80074500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:33.299 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:33.296+0000 7f5e85c55640 1 --2- 172.21.15.38:0/2155021418 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5e8015c740 0x7f5e800b3740 unknown :-1 s=CLOSED pgs=2242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:33.300 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:33.296+0000 7f5e85c55640 1 -- 172.21.15.38:0/2155021418 >> 172.21.15.38:0/2155021418 conn(0x7f5e800735d0 msgr2=0x7f5e800739e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:33.300 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:33.296+0000 7f5e85c55640 1 -- 172.21.15.38:0/2155021418 shutdown_connections 2024-09-21T03:44:33.300 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:33.296+0000 7f5e85c55640 1 -- 172.21.15.38:0/2155021418 wait complete. 2024-09-21T03:44:33.300 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:33.298+0000 7f5e85c55640 1 Processor -- start 2024-09-21T03:44:33.300 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:33.298+0000 7f5e85c55640 1 -- start start 2024-09-21T03:44:33.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:33.299+0000 7f5e85c55640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5e8006ddf0 0x7f5e800cbc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:33.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:33.299+0000 7f5e85c55640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5e800740f0 0x7f5e8015a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:33.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:33.299+0000 7f5e85c55640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5e8015c740 0x7f5e801e9260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:33.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:33.299+0000 7f5e85c55640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f5e800cfbf0 con 0x7f5e8015c740 2024-09-21T03:44:33.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:33.299+0000 7f5e85c55640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f5e800cfa70 con 0x7f5e800740f0 2024-09-21T03:44:33.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:33.299+0000 7f5e85c55640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f5e800cfd70 con 0x7f5e8006ddf0 2024-09-21T03:44:33.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:33.299+0000 7f5e7ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5e8015c740 0x7f5e801e9260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:33.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:33.299+0000 7f5e7ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5e8015c740 0x7f5e801e9260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49974/0 (socket says 172.21.15.38:49974) 2024-09-21T03:44:33.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:33.299+0000 7f5e7ffff640 1 -- 172.21.15.38:0/3093250488 learned_addr learned my addr 172.21.15.38:0/3093250488 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:44:33.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:33.299+0000 7f5e7effd640 1 --2- 172.21.15.38:0/3093250488 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5e800740f0 0x7f5e8015a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:33.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:33.299+0000 7f5e7f7fe640 1 --2- 172.21.15.38:0/3093250488 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5e8006ddf0 0x7f5e800cbc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:33.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:33.300+0000 7f5e7ffff640 1 -- 172.21.15.38:0/3093250488 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5e8006ddf0 msgr2=0x7f5e800cbc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:33.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:33.300+0000 7f5e7ffff640 1 --2- 172.21.15.38:0/3093250488 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5e8006ddf0 0x7f5e800cbc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:33.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:33.300+0000 7f5e7ffff640 1 -- 172.21.15.38:0/3093250488 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5e800740f0 msgr2=0x7f5e8015a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:33.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:33.300+0000 7f5e7ffff640 1 --2- 172.21.15.38:0/3093250488 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5e800740f0 0x7f5e8015a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:33.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:33.300+0000 7f5e7ffff640 1 -- 172.21.15.38:0/3093250488 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e8015b2e0 con 0x7f5e8015c740 2024-09-21T03:44:33.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:33.300+0000 7f5e7ffff640 1 --2- 172.21.15.38:0/3093250488 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5e8015c740 0x7f5e801e9260 secure :-1 s=READY pgs=2243 cs=0 l=1 rev1=1 crypto rx=0x7f5e7800cb10 tx=0x7f5e7800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:33.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:33.300+0000 7f5e67fff640 1 -- 172.21.15.38:0/3093250488 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e78016020 con 0x7f5e8015c740 2024-09-21T03:44:33.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:33.300+0000 7f5e67fff640 1 -- 172.21.15.38:0/3093250488 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5e7800ad00 con 0x7f5e8015c740 2024-09-21T03:44:33.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:33.300+0000 7f5e85c55640 1 -- 172.21.15.38:0/3093250488 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e800cc7a0 con 0x7f5e8015c740 2024-09-21T03:44:33.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:33.300+0000 7f5e67fff640 1 -- 172.21.15.38:0/3093250488 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e78012680 con 0x7f5e8015c740 2024-09-21T03:44:33.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:33.300+0000 7f5e85c55640 1 -- 172.21.15.38:0/3093250488 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e801f58b0 con 0x7f5e8015c740 2024-09-21T03:44:33.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:33.301+0000 7f5e85c55640 1 -- 172.21.15.38:0/3093250488 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e800b1d40 con 0x7f5e8015c740 2024-09-21T03:44:33.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:33.302+0000 7f5e67fff640 1 -- 172.21.15.38:0/3093250488 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f5e78019020 con 0x7f5e8015c740 2024-09-21T03:44:33.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:33.302+0000 7f5e67fff640 1 --2- 172.21.15.38:0/3093250488 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5e6c078b90 0x7f5e6c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:33.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:33.302+0000 7f5e67fff640 1 -- 172.21.15.38:0/3093250488 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(135..135 src has 1..135) ==== 9658+0+0 (secure 0 0 0) 0x7f5e7809c170 con 0x7f5e8015c740 2024-09-21T03:44:33.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:33.303+0000 7f5e7f7fe640 1 --2- 172.21.15.38:0/3093250488 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5e6c078b90 0x7f5e6c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:33.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:33.303+0000 7f5e7f7fe640 1 --2- 172.21.15.38:0/3093250488 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5e6c078b90 0x7f5e6c07b0a0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f5e70034000 tx=0x7f5e70004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:33.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:33.307+0000 7f5e67fff640 1 -- 172.21.15.38:0/3093250488 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f5e800b1d40 con 0x7f5e8015c740 2024-09-21T03:44:33.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:33.534+0000 7f5e85c55640 1 -- 172.21.15.38:0/3093250488 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f5e800d0220 con 0x7f5e8015c740 2024-09-21T03:44:33.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:33.914+0000 7f5e67fff640 1 -- 172.21.15.38:0/3093250488 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v326) ==== 161+0+0 (secure 0 0 0) 0x7f5e78020020 con 0x7f5e8015c740 2024-09-21T03:44:33.917 INFO:teuthology.orchestra.run.smithi038.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-21T03:44:33.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:33.917+0000 7f5e85c55640 1 -- 172.21.15.38:0/3093250488 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5e6c078b90 msgr2=0x7f5e6c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:33.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:33.917+0000 7f5e85c55640 1 --2- 172.21.15.38:0/3093250488 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5e6c078b90 0x7f5e6c07b0a0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f5e70034000 tx=0x7f5e70004730 comp rx=0 tx=0).stop 2024-09-21T03:44:33.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:33.917+0000 7f5e85c55640 1 -- 172.21.15.38:0/3093250488 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5e8015c740 msgr2=0x7f5e801e9260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:33.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:33.917+0000 7f5e85c55640 1 --2- 172.21.15.38:0/3093250488 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5e8015c740 0x7f5e801e9260 secure :-1 s=READY pgs=2243 cs=0 l=1 rev1=1 crypto rx=0x7f5e7800cb10 tx=0x7f5e7800a060 comp rx=0 tx=0).stop 2024-09-21T03:44:33.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:33.917+0000 7f5e85c55640 1 -- 172.21.15.38:0/3093250488 shutdown_connections 2024-09-21T03:44:33.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:33.917+0000 7f5e85c55640 1 --2- 172.21.15.38:0/3093250488 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5e8015c740 0x7f5e801e9260 unknown :-1 s=CLOSED pgs=2243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:33.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:33.917+0000 7f5e85c55640 1 --2- 172.21.15.38:0/3093250488 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5e800740f0 0x7f5e8015a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:33.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:33.917+0000 7f5e85c55640 1 --2- 172.21.15.38:0/3093250488 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5e6c078b90 0x7f5e6c07b0a0 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:33.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:33.917+0000 7f5e85c55640 1 --2- 172.21.15.38:0/3093250488 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5e8006ddf0 0x7f5e800cbc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:33.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:33.917+0000 7f5e85c55640 1 -- 172.21.15.38:0/3093250488 >> 172.21.15.38:0/3093250488 conn(0x7f5e800735d0 msgr2=0x7f5e800b5a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:33.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:33.917+0000 7f5e85c55640 1 -- 172.21.15.38:0/3093250488 shutdown_connections 2024-09-21T03:44:33.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:33.917+0000 7f5e85c55640 1 -- 172.21.15.38:0/3093250488 wait complete. 2024-09-21T03:44:33.933 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-09-21T03:44:34.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:34.050+0000 7fe2bb96e640 1 -- 172.21.15.38:0/449042991 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe2a0004800 msgr2=0x7fe2a0004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:34.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:34.050+0000 7fe2bb96e640 1 --2- 172.21.15.38:0/449042991 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe2a0004800 0x7fe2a0004c10 secure :-1 s=READY pgs=2245 cs=0 l=1 rev1=1 crypto rx=0x7fe2b40d28f0 tx=0x7fe2a0005830 comp rx=0 tx=0).stop 2024-09-21T03:44:34.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:34.050+0000 7fe2bb96e640 1 -- 172.21.15.38:0/449042991 shutdown_connections 2024-09-21T03:44:34.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:34.050+0000 7fe2bb96e640 1 --2- 172.21.15.38:0/449042991 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe2a0004800 0x7fe2a0004c10 unknown :-1 s=CLOSED pgs=2245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:34.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:34.050+0000 7fe2bb96e640 1 -- 172.21.15.38:0/449042991 >> 172.21.15.38:0/449042991 conn(0x7fe2b409d830 msgr2=0x7fe2b40b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:34.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:34.050+0000 7fe2bb96e640 1 -- 172.21.15.38:0/449042991 shutdown_connections 2024-09-21T03:44:34.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:34.050+0000 7fe2bb96e640 1 -- 172.21.15.38:0/449042991 wait complete. 2024-09-21T03:44:34.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:34.052+0000 7fe2bb96e640 1 Processor -- start 2024-09-21T03:44:34.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:34.052+0000 7fe2bb96e640 1 -- start start 2024-09-21T03:44:34.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:34.053+0000 7fe2bb96e640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe2a0004800 0x7fe2b40ce2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:34.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:34.053+0000 7fe2bb96e640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe2b406e730 0x7fe2b40b23c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:34.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:34.053+0000 7fe2bb96e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe2b40b2900 0x7fe2b406f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:34.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:34.053+0000 7fe2bb96e640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fe2b40b9c50 con 0x7fe2b40b2900 2024-09-21T03:44:34.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:34.053+0000 7fe2bb96e640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fe2b40b9ad0 con 0x7fe2b406e730 2024-09-21T03:44:34.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:34.053+0000 7fe2bb96e640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fe2b40b9dd0 con 0x7fe2a0004800 2024-09-21T03:44:34.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:34.054+0000 7fe2b9f0b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe2b40b2900 0x7fe2b406f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:34.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:34.054+0000 7fe2b9f0b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe2b40b2900 0x7fe2b406f250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49992/0 (socket says 172.21.15.38:49992) 2024-09-21T03:44:34.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:34.054+0000 7fe2b9f0b640 1 -- 172.21.15.38:0/2969466874 learned_addr learned my addr 172.21.15.38:0/2969466874 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:44:34.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:34.054+0000 7fe2b8f09640 1 --2- 172.21.15.38:0/2969466874 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe2b406e730 0x7fe2b40b23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:34.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:34.054+0000 7fe2b970a640 1 --2- 172.21.15.38:0/2969466874 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe2a0004800 0x7fe2b40ce2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:34.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:34.054+0000 7fe2b9f0b640 1 -- 172.21.15.38:0/2969466874 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe2a0004800 msgr2=0x7fe2b40ce2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:34.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:34.054+0000 7fe2b9f0b640 1 --2- 172.21.15.38:0/2969466874 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe2a0004800 0x7fe2b40ce2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:34.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:34.054+0000 7fe2b9f0b640 1 -- 172.21.15.38:0/2969466874 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe2b406e730 msgr2=0x7fe2b40b23c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:34.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:34.054+0000 7fe2b9f0b640 1 --2- 172.21.15.38:0/2969466874 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe2b406e730 0x7fe2b40b23c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:34.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:34.054+0000 7fe2b9f0b640 1 -- 172.21.15.38:0/2969466874 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2b41f5430 con 0x7fe2b40b2900 2024-09-21T03:44:34.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:34.054+0000 7fe2b9f0b640 1 --2- 172.21.15.38:0/2969466874 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe2b40b2900 0x7fe2b406f250 secure :-1 s=READY pgs=2246 cs=0 l=1 rev1=1 crypto rx=0x7fe2b40b3c20 tx=0x7fe2ac006660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:34.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:34.054+0000 7fe2a5ffb640 1 -- 172.21.15.38:0/2969466874 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2ac00fa00 con 0x7fe2b40b2900 2024-09-21T03:44:34.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:34.054+0000 7fe2a5ffb640 1 -- 172.21.15.38:0/2969466874 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe2ac019070 con 0x7fe2b40b2900 2024-09-21T03:44:34.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:34.054+0000 7fe2a5ffb640 1 -- 172.21.15.38:0/2969466874 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2ac00fe60 con 0x7fe2b40b2900 2024-09-21T03:44:34.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:34.054+0000 7fe2bb96e640 1 -- 172.21.15.38:0/2969466874 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2b41f8eb0 con 0x7fe2b40b2900 2024-09-21T03:44:34.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:34.055+0000 7fe2bb96e640 1 -- 172.21.15.38:0/2969466874 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2b41f9470 con 0x7fe2b40b2900 2024-09-21T03:44:34.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:34.055+0000 7fe2bb96e640 1 -- 172.21.15.38:0/2969466874 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2b40cec70 con 0x7fe2b40b2900 2024-09-21T03:44:34.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:34.056+0000 7fe2a5ffb640 1 -- 172.21.15.38:0/2969466874 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fe2ac01e460 con 0x7fe2b40b2900 2024-09-21T03:44:34.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:34.057+0000 7fe2a5ffb640 1 --2- 172.21.15.38:0/2969466874 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fe298078b90 0x7fe29807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:34.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:34.057+0000 7fe2a5ffb640 1 -- 172.21.15.38:0/2969466874 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(135..135 src has 1..135) ==== 9658+0+0 (secure 0 0 0) 0x7fe2ac09c050 con 0x7fe2b40b2900 2024-09-21T03:44:34.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:34.057+0000 7fe2b970a640 1 --2- 172.21.15.38:0/2969466874 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fe298078b90 0x7fe29807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:34.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:34.057+0000 7fe2b970a640 1 --2- 172.21.15.38:0/2969466874 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fe298078b90 0x7fe29807b0a0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fe2b0000c00 tx=0x7fe2b0006b70 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:34.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:34.061+0000 7fe2a5ffb640 1 -- 172.21.15.38:0/2969466874 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fe2b41592d0 con 0x7fe2b40b2900 2024-09-21T03:44:34.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:34.327+0000 7fe2bb96e640 1 -- 172.21.15.38:0/2969466874 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7fe2b41f8c40 con 0x7fe2b40b2900 2024-09-21T03:44:34.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:34.916+0000 7fe2a5ffb640 1 -- 172.21.15.38:0/2969466874 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v327) ==== 101+0+0 (secure 0 0 0) 0x7fe2ac063560 con 0x7fe2b40b2900 2024-09-21T03:44:34.919 INFO:teuthology.orchestra.run.smithi038.stderr:repaired: restoring rank 6:0 2024-09-21T03:44:34.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:34.920+0000 7fe2bb96e640 1 -- 172.21.15.38:0/2969466874 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fe298078b90 msgr2=0x7fe29807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:34.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:34.920+0000 7fe2bb96e640 1 --2- 172.21.15.38:0/2969466874 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fe298078b90 0x7fe29807b0a0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fe2b0000c00 tx=0x7fe2b0006b70 comp rx=0 tx=0).stop 2024-09-21T03:44:34.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:34.920+0000 7fe2bb96e640 1 -- 172.21.15.38:0/2969466874 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe2b40b2900 msgr2=0x7fe2b406f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:34.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:34.920+0000 7fe2bb96e640 1 --2- 172.21.15.38:0/2969466874 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe2b40b2900 0x7fe2b406f250 secure :-1 s=READY pgs=2246 cs=0 l=1 rev1=1 crypto rx=0x7fe2b40b3c20 tx=0x7fe2ac006660 comp rx=0 tx=0).stop 2024-09-21T03:44:34.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:34.920+0000 7fe2bb96e640 1 -- 172.21.15.38:0/2969466874 shutdown_connections 2024-09-21T03:44:34.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:34.920+0000 7fe2bb96e640 1 --2- 172.21.15.38:0/2969466874 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe2b40b2900 0x7fe2b406f250 unknown :-1 s=CLOSED pgs=2246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:34.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:34.920+0000 7fe2bb96e640 1 --2- 172.21.15.38:0/2969466874 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe2b406e730 0x7fe2b40b23c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:34.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:34.920+0000 7fe2bb96e640 1 --2- 172.21.15.38:0/2969466874 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fe298078b90 0x7fe29807b0a0 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:34.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:34.920+0000 7fe2bb96e640 1 --2- 172.21.15.38:0/2969466874 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe2a0004800 0x7fe2b40ce2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:34.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:34.920+0000 7fe2bb96e640 1 -- 172.21.15.38:0/2969466874 >> 172.21.15.38:0/2969466874 conn(0x7fe2b409d830 msgr2=0x7fe2b415df80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:34.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:34.920+0000 7fe2bb96e640 1 -- 172.21.15.38:0/2969466874 shutdown_connections 2024-09-21T03:44:34.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:34.920+0000 7fe2bb96e640 1 -- 172.21.15.38:0/2969466874 wait complete. 2024-09-21T03:44:34.937 DEBUG:teuthology.orchestra.run.smithi038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-09-21T03:44:34.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:34.995+0000 7f404b15cf40 1 Processor -- start 2024-09-21T03:44:34.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:34.995+0000 7f404b15cf40 1 -- start start 2024-09-21T03:44:35.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:34.996+0000 7f404b15cf40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55ea05928240 0x55ea05a84fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:35.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:34.996+0000 7f404b15cf40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ea0596f090 0x55ea059fc550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:35.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:34.996+0000 7f404b15cf40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ea059e4520 0x55ea05923850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:35.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:34.996+0000 7f404b15cf40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55ea059fbaa0 con 0x55ea0596f090 2024-09-21T03:44:35.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:34.996+0000 7f404b15cf40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55ea05875d60 con 0x55ea059e4520 2024-09-21T03:44:35.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:34.996+0000 7f404b15cf40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55ea059569b0 con 0x55ea05928240 2024-09-21T03:44:35.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:34.997+0000 7f4044415640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ea0596f090 0x55ea059fc550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:35.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:34.997+0000 7f4044415640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ea0596f090 0x55ea059fc550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50016/0 (socket says 172.21.15.38:50016) 2024-09-21T03:44:35.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:34.997+0000 7f4044415640 1 -- 172.21.15.38:0/4269998597 learned_addr learned my addr 172.21.15.38:0/4269998597 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:44:35.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:34.997+0000 7f4045417640 1 --2- 172.21.15.38:0/4269998597 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ea059e4520 0x55ea05923850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:35.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:34.997+0000 7f4044c16640 1 --2- 172.21.15.38:0/4269998597 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55ea05928240 0x55ea05a84fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:35.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:34.997+0000 7f4044415640 1 -- 172.21.15.38:0/4269998597 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55ea05928240 msgr2=0x55ea05a84fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:44:35.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:34.997+0000 7f4044415640 1 --2- 172.21.15.38:0/4269998597 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55ea05928240 0x55ea05a84fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:35.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:34.997+0000 7f4044415640 1 -- 172.21.15.38:0/4269998597 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ea059e4520 msgr2=0x55ea05923850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:44:35.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:34.997+0000 7f4044415640 1 --2- 172.21.15.38:0/4269998597 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ea059e4520 0x55ea05923850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:35.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:34.997+0000 7f4044415640 1 -- 172.21.15.38:0/4269998597 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea05a85660 con 0x55ea0596f090 2024-09-21T03:44:35.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:34.997+0000 7f4044415640 1 --2- 172.21.15.38:0/4269998597 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ea0596f090 0x55ea059fc550 secure :-1 s=READY pgs=2248 cs=0 l=1 rev1=1 crypto rx=0x7f403800c660 tx=0x7f403800cb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e12c7048545105eb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:35.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:34.997+0000 7f4043413640 1 -- 172.21.15.38:0/4269998597 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f403800adc0 con 0x55ea0596f090 2024-09-21T03:44:35.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:34.997+0000 7f4043413640 1 -- 172.21.15.38:0/4269998597 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4038015440 con 0x55ea0596f090 2024-09-21T03:44:35.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:34.997+0000 7f4043413640 1 -- 172.21.15.38:0/4269998597 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4038015870 con 0x55ea0596f090 2024-09-21T03:44:35.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:34.998+0000 7f404b15cf40 1 -- 172.21.15.38:0/4269998597 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ea0596f090 msgr2=0x55ea059fc550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:35.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:34.998+0000 7f404b15cf40 1 --2- 172.21.15.38:0/4269998597 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ea0596f090 0x55ea059fc550 secure :-1 s=READY pgs=2248 cs=0 l=1 rev1=1 crypto rx=0x7f403800c660 tx=0x7f403800cb20 comp rx=0 tx=0).stop 2024-09-21T03:44:35.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:34.998+0000 7f404b15cf40 1 -- 172.21.15.38:0/4269998597 shutdown_connections 2024-09-21T03:44:35.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:34.998+0000 7f404b15cf40 1 --2- 172.21.15.38:0/4269998597 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ea059e4520 0x55ea05923850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:35.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:34.998+0000 7f404b15cf40 1 --2- 172.21.15.38:0/4269998597 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ea0596f090 0x55ea059fc550 unknown :-1 s=CLOSED pgs=2248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:35.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:34.998+0000 7f404b15cf40 1 --2- 172.21.15.38:0/4269998597 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55ea05928240 0x55ea05a84fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:35.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:34.998+0000 7f404b15cf40 1 -- 172.21.15.38:0/4269998597 >> 172.21.15.38:0/4269998597 conn(0x55ea05846ac0 msgr2=0x55ea05951a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:35.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:34.998+0000 7f404b15cf40 1 -- 172.21.15.38:0/4269998597 shutdown_connections 2024-09-21T03:44:35.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:34.998+0000 7f404b15cf40 1 -- 172.21.15.38:0/4269998597 wait complete. 2024-09-21T03:44:35.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:34.999+0000 7f404b15cf40 1 Processor -- start 2024-09-21T03:44:35.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:34.999+0000 7f404b15cf40 1 -- start start 2024-09-21T03:44:35.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.001+0000 7f404b15cf40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ea05928240 0x55ea05923970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:35.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.001+0000 7f404b15cf40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ea0596f090 0x55ea059f4270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:35.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.001+0000 7f404b15cf40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55ea059e4520 0x55ea059e3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:35.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.001+0000 7f404b15cf40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55ea059f4f40 con 0x55ea05928240 2024-09-21T03:44:35.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.001+0000 7f404b15cf40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55ea05952390 con 0x55ea0596f090 2024-09-21T03:44:35.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.001+0000 7f404b15cf40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55ea059f5190 con 0x55ea059e4520 2024-09-21T03:44:35.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.001+0000 7f4044c16640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ea05928240 0x55ea05923970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:35.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.001+0000 7f4044c16640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ea05928240 0x55ea05923970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50020/0 (socket says 172.21.15.38:50020) 2024-09-21T03:44:35.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.001+0000 7f4044c16640 1 -- 172.21.15.38:0/3369057575 learned_addr learned my addr 172.21.15.38:0/3369057575 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:44:35.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.001+0000 7f4045417640 1 --2- 172.21.15.38:0/3369057575 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55ea059e4520 0x55ea059e3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:35.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.001+0000 7f4044415640 1 --2- 172.21.15.38:0/3369057575 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ea0596f090 0x55ea059f4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:35.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.001+0000 7f4044c16640 1 -- 172.21.15.38:0/3369057575 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55ea059e4520 msgr2=0x55ea059e3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:35.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.001+0000 7f4044c16640 1 --2- 172.21.15.38:0/3369057575 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55ea059e4520 0x55ea059e3630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:35.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.001+0000 7f4044c16640 1 -- 172.21.15.38:0/3369057575 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ea0596f090 msgr2=0x55ea059f4270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:35.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.001+0000 7f4044c16640 1 --2- 172.21.15.38:0/3369057575 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ea0596f090 0x55ea059f4270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:35.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.001+0000 7f4044c16640 1 -- 172.21.15.38:0/3369057575 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea059dda80 con 0x55ea05928240 2024-09-21T03:44:35.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.001+0000 7f4044c16640 1 --2- 172.21.15.38:0/3369057575 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ea05928240 0x55ea05923970 secure :-1 s=READY pgs=2249 cs=0 l=1 rev1=1 crypto rx=0x7f4034025e80 tx=0x7f403404d6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:35.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.001+0000 7f4042411640 1 -- 172.21.15.38:0/3369057575 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4034062670 con 0x55ea05928240 2024-09-21T03:44:35.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.001+0000 7f4042411640 1 -- 172.21.15.38:0/3369057575 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4034062810 con 0x55ea05928240 2024-09-21T03:44:35.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.001+0000 7f4042411640 1 -- 172.21.15.38:0/3369057575 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4034062c60 con 0x55ea05928240 2024-09-21T03:44:35.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.001+0000 7f404b15cf40 1 -- 172.21.15.38:0/3369057575 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55ea059ddd20 con 0x55ea05928240 2024-09-21T03:44:35.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.001+0000 7f404b15cf40 1 -- 172.21.15.38:0/3369057575 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea059f7600 con 0x55ea05928240 2024-09-21T03:44:35.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.003+0000 7f4042411640 1 -- 172.21.15.38:0/3369057575 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f4034056470 con 0x55ea05928240 2024-09-21T03:44:35.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.003+0000 7f4042411640 1 --2- 172.21.15.38:0/3369057575 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f40300775a0 0x7f4030079ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:35.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.004+0000 7f4042411640 1 -- 172.21.15.38:0/3369057575 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(135..135 src has 1..135) ==== 9658+0+0 (secure 0 0 0) 0x7f40340de7d0 con 0x55ea05928240 2024-09-21T03:44:35.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.004+0000 7f4044415640 1 --2- 172.21.15.38:0/3369057575 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f40300775a0 0x7f4030079ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:35.008 INFO:teuthology.orchestra.run.smithi038.stdout:Importing pool 2024-09-21T03:44:35.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.004+0000 7f404b15cf40 1 --2- 172.21.15.38:0/3369057575 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x55ea059de800 0x55ea059f6930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:35.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.004+0000 7f404b15cf40 1 -- 172.21.15.38:0/3369057575 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55ea059f6e70 con 0x55ea059de800 2024-09-21T03:44:35.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.004+0000 7f4045417640 1 --2- 172.21.15.38:0/3369057575 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x55ea059de800 0x55ea059f6930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:35.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.004+0000 7f4044415640 1 --2- 172.21.15.38:0/3369057575 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f40300775a0 0x7f4030079ab0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x55ea05951880 tx=0x7f4038002fb0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:35.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.005+0000 7f4045417640 1 --2- 172.21.15.38:0/3369057575 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x55ea059de800 0x55ea059f6930 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:35.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.008+0000 7f4045417640 1 -- 172.21.15.38:0/3369057575 <== osd.7 v2:172.21.15.195:6818/1606106342 1 ==== osd_op_reply(1 601.00000000 [create] v135'1190 uv1164 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f403c00b040 con 0x55ea059de800 2024-09-21T03:44:35.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.008+0000 7f404b15cf40 1 -- 172.21.15.38:0/3369057575 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e135) -- 0x55ea05a9ed70 con 0x55ea059de800 2024-09-21T03:44:35.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.010+0000 7f4045417640 1 -- 172.21.15.38:0/3369057575 <== osd.7 v2:172.21.15.195:6818/1606106342 2 ==== osd_op_reply(2 601.00000000 [delete] v135'1191 uv1191 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f403c00b040 con 0x55ea059de800 2024-09-21T03:44:35.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.010+0000 7f404b15cf40 1 -- 172.21.15.38:0/3369057575 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55ea05a9f110 con 0x55ea059de800 2024-09-21T03:44:35.014 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-09-21T03:44:35.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.012+0000 7f4045417640 1 -- 172.21.15.38:0/3369057575 <== osd.7 v2:172.21.15.195:6818/1606106342 3 ==== osd_op_reply(3 601.00000000 [create] v135'1192 uv1192 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f403c00b040 con 0x55ea059de800 2024-09-21T03:44:35.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.012+0000 7f404b15cf40 1 -- 172.21.15.38:0/3369057575 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55ea05aa04c0 con 0x55ea059de800 2024-09-21T03:44:35.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.014+0000 7f4045417640 1 -- 172.21.15.38:0/3369057575 <== osd.7 v2:172.21.15.195:6818/1606106342 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v135'1193 uv1193 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f403c00b040 con 0x55ea059de800 2024-09-21T03:44:35.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.014+0000 7f404b15cf40 1 -- 172.21.15.38:0/3369057575 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55ea05aa0860 con 0x55ea059de800 2024-09-21T03:44:35.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.015+0000 7f4045417640 1 -- 172.21.15.38:0/3369057575 <== osd.7 v2:172.21.15.195:6818/1606106342 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v135'1194 uv1194 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f403c00b040 con 0x55ea059de800 2024-09-21T03:44:35.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.015+0000 7f404b15cf40 1 -- 172.21.15.38:0/3369057575 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55ea05a9f4b0 con 0x55ea059de800 2024-09-21T03:44:35.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.016+0000 7f4045417640 1 -- 172.21.15.38:0/3369057575 <== osd.7 v2:172.21.15.195:6818/1606106342 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v135'1195 uv1195 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f403c00b040 con 0x55ea059de800 2024-09-21T03:44:35.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.016+0000 7f404b15cf40 1 -- 172.21.15.38:0/3369057575 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55ea05a9f850 con 0x55ea059de800 2024-09-21T03:44:35.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.019+0000 7f4045417640 1 -- 172.21.15.38:0/3369057575 <== osd.7 v2:172.21.15.195:6818/1606106342 7 ==== osd_op_reply(7 602.00000000 [create] v135'1196 uv1170 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f403c00b040 con 0x55ea059de800 2024-09-21T03:44:35.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.019+0000 7f404b15cf40 1 -- 172.21.15.38:0/3369057575 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e135) -- 0x55ea05a9fc40 con 0x55ea059de800 2024-09-21T03:44:35.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.021+0000 7f4045417640 1 -- 172.21.15.38:0/3369057575 <== osd.7 v2:172.21.15.195:6818/1606106342 8 ==== osd_op_reply(8 602.00000000 [delete] v135'1197 uv1197 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f403c00b040 con 0x55ea059de800 2024-09-21T03:44:35.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.021+0000 7f404b15cf40 1 -- 172.21.15.38:0/3369057575 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55ea05a9fc40 con 0x55ea059de800 2024-09-21T03:44:35.025 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-09-21T03:44:35.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.023+0000 7f4045417640 1 -- 172.21.15.38:0/3369057575 <== osd.7 v2:172.21.15.195:6818/1606106342 9 ==== osd_op_reply(9 602.00000000 [create] v135'1198 uv1198 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f403c00b040 con 0x55ea059de800 2024-09-21T03:44:35.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.023+0000 7f404b15cf40 1 -- 172.21.15.38:0/3369057575 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55ea05a9fbf0 con 0x55ea059de800 2024-09-21T03:44:35.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.025+0000 7f4045417640 1 -- 172.21.15.38:0/3369057575 <== osd.7 v2:172.21.15.195:6818/1606106342 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v135'1199 uv1199 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f403c00b040 con 0x55ea059de800 2024-09-21T03:44:35.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.025+0000 7f404b15cf40 1 -- 172.21.15.38:0/3369057575 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55ea05a9fbf0 con 0x55ea059de800 2024-09-21T03:44:35.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.026+0000 7f4045417640 1 -- 172.21.15.38:0/3369057575 <== osd.7 v2:172.21.15.195:6818/1606106342 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v135'1200 uv1200 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f403c00b040 con 0x55ea059de800 2024-09-21T03:44:35.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.026+0000 7f404b15cf40 1 -- 172.21.15.38:0/3369057575 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55ea05a9fbf0 con 0x55ea059de800 2024-09-21T03:44:35.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.028+0000 7f4045417640 1 -- 172.21.15.38:0/3369057575 <== osd.7 v2:172.21.15.195:6818/1606106342 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v135'1201 uv1201 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f403c00b040 con 0x55ea059de800 2024-09-21T03:44:35.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.028+0000 7f404b15cf40 1 -- 172.21.15.38:0/3369057575 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:13 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55ea05a9fbf0 con 0x55ea059de800 2024-09-21T03:44:35.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.030+0000 7f4045417640 1 -- 172.21.15.38:0/3369057575 <== osd.7 v2:172.21.15.195:6818/1606106342 13 ==== osd_op_reply(13 10000000000.00000000 [create] v135'1202 uv1177 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7f403c00b040 con 0x55ea059de800 2024-09-21T03:44:35.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.030+0000 7f404b15cf40 1 -- 172.21.15.38:0/3369057575 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:14 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e135) -- 0x55ea05a9fbf0 con 0x55ea059de800 2024-09-21T03:44:35.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.032+0000 7f4045417640 1 -- 172.21.15.38:0/3369057575 <== osd.7 v2:172.21.15.195:6818/1606106342 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v135'1203 uv1203 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f403c00b040 con 0x55ea059de800 2024-09-21T03:44:35.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.032+0000 7f404b15cf40 1 -- 172.21.15.38:0/3369057575 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:15 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55ea05a9fbf0 con 0x55ea059de800 2024-09-21T03:44:35.035 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-09-21T03:44:35.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.033+0000 7f4045417640 1 -- 172.21.15.38:0/3369057575 <== osd.7 v2:172.21.15.195:6818/1606106342 15 ==== osd_op_reply(15 10000000000.00000000 [create] v135'1204 uv1204 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f403c00b040 con 0x55ea059de800 2024-09-21T03:44:35.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.033+0000 7f404b15cf40 1 -- 172.21.15.38:0/3369057575 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:16 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55ea05a9fbf0 con 0x55ea059de800 2024-09-21T03:44:35.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.035+0000 7f4045417640 1 -- 172.21.15.38:0/3369057575 <== osd.7 v2:172.21.15.195:6818/1606106342 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v135'1205 uv1205 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f403c00b040 con 0x55ea059de800 2024-09-21T03:44:35.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.035+0000 7f404b15cf40 1 -- 172.21.15.38:0/3369057575 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:17 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55ea05a9fbf0 con 0x55ea059de800 2024-09-21T03:44:35.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.035+0000 7f4045417640 1 -- 172.21.15.38:0/3369057575 <== osd.7 v2:172.21.15.195:6818/1606106342 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v135'1206 uv1206 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f403c00b040 con 0x55ea059de800 2024-09-21T03:44:35.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.035+0000 7f404b15cf40 1 -- 172.21.15.38:0/3369057575 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:18 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55ea05a9fbf0 con 0x55ea059de800 2024-09-21T03:44:35.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.038+0000 7f4045417640 1 -- 172.21.15.38:0/3369057575 <== osd.7 v2:172.21.15.195:6818/1606106342 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v135'1207 uv1207 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f403c00b040 con 0x55ea059de800 2024-09-21T03:44:35.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.038+0000 7f404b15cf40 1 -- 172.21.15.38:0/3369057575 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:19 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=585b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55ea05a9fff0 con 0x55ea059de800 2024-09-21T03:44:35.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.039+0000 7f4045417640 1 -- 172.21.15.38:0/3369057575 <== osd.7 v2:172.21.15.195:6818/1606106342 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v135'1208 uv1208 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f403c00b040 con 0x55ea059de800 2024-09-21T03:44:35.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.040+0000 7f404b15cf40 1 -- 172.21.15.38:0/3369057575 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:20 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55ea05a9ffd0 con 0x55ea059de800 2024-09-21T03:44:35.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.041+0000 7f4045417640 1 -- 172.21.15.38:0/3369057575 <== osd.7 v2:172.21.15.195:6818/1606106342 20 ==== osd_op_reply(20 600.00000000 [create] v135'1209 uv1183 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f403c00b040 con 0x55ea059de800 2024-09-21T03:44:35.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.041+0000 7f404b15cf40 1 -- 172.21.15.38:0/3369057575 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:21 13.10 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e135) -- 0x55ea05a9ffd0 con 0x55ea059de800 2024-09-21T03:44:35.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.043+0000 7f4045417640 1 -- 172.21.15.38:0/3369057575 <== osd.7 v2:172.21.15.195:6818/1606106342 21 ==== osd_op_reply(21 600.00000000 [delete] v135'1210 uv1210 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f403c00b040 con 0x55ea059de800 2024-09-21T03:44:35.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.043+0000 7f404b15cf40 1 -- 172.21.15.38:0/3369057575 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:22 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55ea05a9ffd0 con 0x55ea059de800 2024-09-21T03:44:35.046 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-09-21T03:44:35.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.044+0000 7f4045417640 1 -- 172.21.15.38:0/3369057575 <== osd.7 v2:172.21.15.195:6818/1606106342 22 ==== osd_op_reply(22 600.00000000 [create] v135'1211 uv1211 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f403c00b040 con 0x55ea059de800 2024-09-21T03:44:35.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.044+0000 7f404b15cf40 1 -- 172.21.15.38:0/3369057575 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:23 13.10 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55ea05a9ffd0 con 0x55ea059de800 2024-09-21T03:44:35.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.045+0000 7f4045417640 1 -- 172.21.15.38:0/3369057575 <== osd.7 v2:172.21.15.195:6818/1606106342 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v135'1212 uv1212 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f403c00b040 con 0x55ea059de800 2024-09-21T03:44:35.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.045+0000 7f404b15cf40 1 -- 172.21.15.38:0/3369057575 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:24 13.10 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55ea05a9ffd0 con 0x55ea059de800 2024-09-21T03:44:35.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.048+0000 7f4045417640 1 -- 172.21.15.38:0/3369057575 <== osd.7 v2:172.21.15.195:6818/1606106342 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v135'1213 uv1213 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f403c00b040 con 0x55ea059de800 2024-09-21T03:44:35.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.048+0000 7f404b15cf40 1 -- 172.21.15.38:0/3369057575 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:25 13.10 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55ea05a9ffd0 con 0x55ea059de800 2024-09-21T03:44:35.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.050+0000 7f4045417640 1 -- 172.21.15.38:0/3369057575 <== osd.7 v2:172.21.15.195:6818/1606106342 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v135'1214 uv1214 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f403c00b040 con 0x55ea059de800 2024-09-21T03:44:35.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.050+0000 7f404b15cf40 1 -- 172.21.15.38:0/3369057575 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:26 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55ea05a9fea0 con 0x55ea059de800 2024-09-21T03:44:35.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.052+0000 7f4045417640 1 -- 172.21.15.38:0/3369057575 <== osd.7 v2:172.21.15.195:6818/1606106342 26 ==== osd_op_reply(26 603.00000000 [create] v135'1215 uv1189 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f403c00b040 con 0x55ea059de800 2024-09-21T03:44:35.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.052+0000 7f404b15cf40 1 -- 172.21.15.38:0/3369057575 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:27 13.10 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e135) -- 0x55ea05a9fea0 con 0x55ea059de800 2024-09-21T03:44:35.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.053+0000 7f4045417640 1 -- 172.21.15.38:0/3369057575 <== osd.7 v2:172.21.15.195:6818/1606106342 27 ==== osd_op_reply(27 603.00000000 [delete] v135'1216 uv1216 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f403c00b040 con 0x55ea059de800 2024-09-21T03:44:35.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.053+0000 7f404b15cf40 1 -- 172.21.15.38:0/3369057575 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:28 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55ea05a9fea0 con 0x55ea059de800 2024-09-21T03:44:35.056 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-09-21T03:44:35.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.055+0000 7f4045417640 1 -- 172.21.15.38:0/3369057575 <== osd.7 v2:172.21.15.195:6818/1606106342 28 ==== osd_op_reply(28 603.00000000 [create] v135'1217 uv1217 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f403c00b040 con 0x55ea059de800 2024-09-21T03:44:35.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.055+0000 7f404b15cf40 1 -- 172.21.15.38:0/3369057575 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:29 13.10 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55ea05a9fea0 con 0x55ea059de800 2024-09-21T03:44:35.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.055+0000 7f4045417640 1 -- 172.21.15.38:0/3369057575 <== osd.7 v2:172.21.15.195:6818/1606106342 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v135'1218 uv1218 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f403c00b040 con 0x55ea059de800 2024-09-21T03:44:35.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.057+0000 7f404b15cf40 1 -- 172.21.15.38:0/3369057575 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:30 13.10 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55ea05a9fea0 con 0x55ea059de800 2024-09-21T03:44:35.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.058+0000 7f4045417640 1 -- 172.21.15.38:0/3369057575 <== osd.7 v2:172.21.15.195:6818/1606106342 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v135'1219 uv1219 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f403c00b040 con 0x55ea059de800 2024-09-21T03:44:35.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.058+0000 7f404b15cf40 1 -- 172.21.15.38:0/3369057575 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:31 13.10 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55ea05a9fea0 con 0x55ea059de800 2024-09-21T03:44:35.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.060+0000 7f4045417640 1 -- 172.21.15.38:0/3369057575 <== osd.7 v2:172.21.15.195:6818/1606106342 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v135'1220 uv1220 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f403c00b040 con 0x55ea059de800 2024-09-21T03:44:35.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.060+0000 7f404b15cf40 1 --2- 172.21.15.38:0/3369057575 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x55ea05aa1110 0x55ea05aa3530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:35.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.060+0000 7f404b15cf40 1 -- 172.21.15.38:0/3369057575 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55ea05a9fea0 con 0x55ea05aa1110 2024-09-21T03:44:35.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.060+0000 7f4044c16640 1 --2- 172.21.15.38:0/3369057575 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x55ea05aa1110 0x55ea05aa3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:35.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.060+0000 7f4044c16640 1 --2- 172.21.15.38:0/3369057575 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x55ea05aa1110 0x55ea05aa3530 crc :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:35.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.062+0000 7f4044c16640 1 -- 172.21.15.38:0/3369057575 <== osd.6 v2:172.21.15.195:6826/4055825675 1 ==== osd_op_reply(32 1.00000000.inode [create] v135'866 uv849 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f40340a1a50 con 0x55ea05aa1110 2024-09-21T03:44:35.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.062+0000 7f404b15cf40 1 -- 172.21.15.38:0/3369057575 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e135) -- 0x55ea05aa3a70 con 0x55ea05aa1110 2024-09-21T03:44:35.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.063+0000 7f4044c16640 1 -- 172.21.15.38:0/3369057575 <== osd.6 v2:172.21.15.195:6826/4055825675 2 ==== osd_op_reply(33 1.00000000.inode [delete] v135'867 uv867 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f40340a1a50 con 0x55ea05aa1110 2024-09-21T03:44:35.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.064+0000 7f404b15cf40 1 -- 172.21.15.38:0/3369057575 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55ea05aa3e10 con 0x55ea05aa1110 2024-09-21T03:44:35.068 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-09-21T03:44:35.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.065+0000 7f4044c16640 1 -- 172.21.15.38:0/3369057575 <== osd.6 v2:172.21.15.195:6826/4055825675 3 ==== osd_op_reply(34 1.00000000.inode [create] v135'868 uv868 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f40340a1a50 con 0x55ea05aa1110 2024-09-21T03:44:35.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.065+0000 7f404b15cf40 1 -- 172.21.15.38:0/3369057575 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55ea05aa4660 con 0x55ea05aa1110 2024-09-21T03:44:35.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.067+0000 7f4044c16640 1 -- 172.21.15.38:0/3369057575 <== osd.6 v2:172.21.15.195:6826/4055825675 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v135'869 uv869 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f40340a1a50 con 0x55ea05aa1110 2024-09-21T03:44:35.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.067+0000 7f404b15cf40 1 -- 172.21.15.38:0/3369057575 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55ea05aa4a00 con 0x55ea05aa1110 2024-09-21T03:44:35.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.068+0000 7f4044c16640 1 -- 172.21.15.38:0/3369057575 <== osd.6 v2:172.21.15.195:6826/4055825675 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v135'870 uv870 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f40340a1a50 con 0x55ea05aa1110 2024-09-21T03:44:35.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.068+0000 7f404b15cf40 1 -- 172.21.15.38:0/3369057575 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55ea05aa4da0 con 0x55ea05aa1110 2024-09-21T03:44:35.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.070+0000 7f4044c16640 1 -- 172.21.15.38:0/3369057575 <== osd.6 v2:172.21.15.195:6826/4055825675 6 ==== osd_op_reply(37 200.00000000 [create] v135'871 uv854 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f40340a1a50 con 0x55ea05aa1110 2024-09-21T03:44:35.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.070+0000 7f404b15cf40 1 -- 172.21.15.38:0/3369057575 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e135) -- 0x55ea05aa4da0 con 0x55ea05aa1110 2024-09-21T03:44:35.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.071+0000 7f4044c16640 1 -- 172.21.15.38:0/3369057575 <== osd.6 v2:172.21.15.195:6826/4055825675 7 ==== osd_op_reply(38 200.00000000 [delete] v135'872 uv872 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f40340a1a50 con 0x55ea05aa1110 2024-09-21T03:44:35.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.071+0000 7f404b15cf40 1 -- 172.21.15.38:0/3369057575 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55ea05aa4da0 con 0x55ea05aa1110 2024-09-21T03:44:35.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.073+0000 7f4044c16640 1 -- 172.21.15.38:0/3369057575 <== osd.6 v2:172.21.15.195:6826/4055825675 8 ==== osd_op_reply(39 200.00000000 [create] v135'873 uv873 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f40340a1a50 con 0x55ea05aa1110 2024-09-21T03:44:35.076 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-09-21T03:44:35.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.073+0000 7f404b15cf40 1 -- 172.21.15.38:0/3369057575 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55ea05aa4da0 con 0x55ea05aa1110 2024-09-21T03:44:35.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.074+0000 7f4044c16640 1 -- 172.21.15.38:0/3369057575 <== osd.6 v2:172.21.15.195:6826/4055825675 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v135'874 uv874 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f40340a1a50 con 0x55ea05aa1110 2024-09-21T03:44:35.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.074+0000 7f404b15cf40 1 -- 172.21.15.38:0/3369057575 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55ea05aa4da0 con 0x55ea05aa1110 2024-09-21T03:44:35.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.076+0000 7f4044c16640 1 -- 172.21.15.38:0/3369057575 <== osd.6 v2:172.21.15.195:6826/4055825675 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v135'875 uv875 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f40340a1a50 con 0x55ea05aa1110 2024-09-21T03:44:35.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.076+0000 7f404b15cf40 1 -- 172.21.15.38:0/3369057575 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:42 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55ea05aa4da0 con 0x55ea05aa1110 2024-09-21T03:44:35.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.077+0000 7f4044c16640 1 -- 172.21.15.38:0/3369057575 <== osd.6 v2:172.21.15.195:6826/4055825675 11 ==== osd_op_reply(42 200.00000001 [create] v135'876 uv859 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f40340a1a50 con 0x55ea05aa1110 2024-09-21T03:44:35.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.077+0000 7f404b15cf40 1 -- 172.21.15.38:0/3369057575 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:43 13.14 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e135) -- 0x55ea05aa4da0 con 0x55ea05aa1110 2024-09-21T03:44:35.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.079+0000 7f4044c16640 1 -- 172.21.15.38:0/3369057575 <== osd.6 v2:172.21.15.195:6826/4055825675 12 ==== osd_op_reply(43 200.00000001 [delete] v135'877 uv877 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f40340a1a50 con 0x55ea05aa1110 2024-09-21T03:44:35.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.079+0000 7f404b15cf40 1 -- 172.21.15.38:0/3369057575 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:44 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55ea05aa4da0 con 0x55ea05aa1110 2024-09-21T03:44:35.083 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-09-21T03:44:35.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.081+0000 7f4044c16640 1 -- 172.21.15.38:0/3369057575 <== osd.6 v2:172.21.15.195:6826/4055825675 13 ==== osd_op_reply(44 200.00000001 [create] v135'878 uv878 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f40340a1a50 con 0x55ea05aa1110 2024-09-21T03:44:35.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.081+0000 7f404b15cf40 1 -- 172.21.15.38:0/3369057575 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:45 13.14 13:2e2fa760:::200.00000001:head [write 0~17490 in=17490b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55ea05aa9220 con 0x55ea05aa1110 2024-09-21T03:44:35.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.082+0000 7f4044c16640 1 -- 172.21.15.38:0/3369057575 <== osd.6 v2:172.21.15.195:6826/4055825675 14 ==== osd_op_reply(45 200.00000001 [write 0~17490] v135'879 uv879 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f40340a1a50 con 0x55ea05aa1110 2024-09-21T03:44:35.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.082+0000 7f404b15cf40 1 -- 172.21.15.38:0/3369057575 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:46 13.14 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55ea05aa9220 con 0x55ea05aa1110 2024-09-21T03:44:35.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.084+0000 7f4044c16640 1 -- 172.21.15.38:0/3369057575 <== osd.6 v2:172.21.15.195:6826/4055825675 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v135'880 uv880 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f40340a1a50 con 0x55ea05aa1110 2024-09-21T03:44:35.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.084+0000 7f404b15cf40 1 -- 172.21.15.38:0/3369057575 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:47 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55ea05aa4da0 con 0x55ea05aa1110 2024-09-21T03:44:35.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.085+0000 7f4044c16640 1 -- 172.21.15.38:0/3369057575 <== osd.6 v2:172.21.15.195:6826/4055825675 16 ==== osd_op_reply(47 606.00000000 [create] v135'881 uv865 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f40340a1a50 con 0x55ea05aa1110 2024-09-21T03:44:35.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.086+0000 7f404b15cf40 1 -- 172.21.15.38:0/3369057575 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:48 13.14 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e135) -- 0x55ea05aa4da0 con 0x55ea05aa1110 2024-09-21T03:44:35.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.087+0000 7f4044c16640 1 -- 172.21.15.38:0/3369057575 <== osd.6 v2:172.21.15.195:6826/4055825675 17 ==== osd_op_reply(48 606.00000000 [delete] v135'882 uv882 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f40340a1a50 con 0x55ea05aa1110 2024-09-21T03:44:35.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.087+0000 7f404b15cf40 1 -- 172.21.15.38:0/3369057575 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:49 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55ea05aa4da0 con 0x55ea05aa1110 2024-09-21T03:44:35.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.089+0000 7f4044c16640 1 -- 172.21.15.38:0/3369057575 <== osd.6 v2:172.21.15.195:6826/4055825675 18 ==== osd_op_reply(49 606.00000000 [create] v135'883 uv883 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f40340a1a50 con 0x55ea05aa1110 2024-09-21T03:44:35.092 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-09-21T03:44:35.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.089+0000 7f404b15cf40 1 -- 172.21.15.38:0/3369057575 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:50 13.14 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55ea05aa5db0 con 0x55ea05aa1110 2024-09-21T03:44:35.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.090+0000 7f4044c16640 1 -- 172.21.15.38:0/3369057575 <== osd.6 v2:172.21.15.195:6826/4055825675 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v135'884 uv884 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f40340a1a50 con 0x55ea05aa1110 2024-09-21T03:44:35.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.090+0000 7f404b15cf40 1 -- 172.21.15.38:0/3369057575 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:51 13.14 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55ea05aa5db0 con 0x55ea05aa1110 2024-09-21T03:44:35.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.092+0000 7f4044c16640 1 -- 172.21.15.38:0/3369057575 <== osd.6 v2:172.21.15.195:6826/4055825675 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v135'885 uv885 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f40340a1a50 con 0x55ea05aa1110 2024-09-21T03:44:35.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.092+0000 7f404b15cf40 1 -- 172.21.15.38:0/3369057575 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- osd_op(unknown.0.0:52 13.14 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55ea05aa4da0 con 0x55ea05aa1110 2024-09-21T03:44:35.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.093+0000 7f4044c16640 1 -- 172.21.15.38:0/3369057575 <== osd.6 v2:172.21.15.195:6826/4055825675 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v135'886 uv886 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f40340a1a50 con 0x55ea05aa1110 2024-09-21T03:44:35.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.093+0000 7f404b15cf40 1 --2- 172.21.15.38:0/3369057575 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x55ea05aa52b0 0x55ea05aa76d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:35.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.093+0000 7f404b15cf40 1 -- 172.21.15.38:0/3369057575 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:53 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55ea05aa7c10 con 0x55ea05aa52b0 2024-09-21T03:44:35.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.093+0000 7f4044415640 1 --2- 172.21.15.38:0/3369057575 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x55ea05aa52b0 0x55ea05aa76d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:35.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.094+0000 7f4044415640 1 --2- 172.21.15.38:0/3369057575 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x55ea05aa52b0 0x55ea05aa76d0 crc :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:35.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.095+0000 7f4044415640 1 -- 172.21.15.38:0/3369057575 <== osd.0 v2:172.21.15.38:6800/2381639433 1 ==== osd_op_reply(53 607.00000000 [create] v135'231 uv230 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f4038020070 con 0x55ea05aa52b0 2024-09-21T03:44:35.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.095+0000 7f404b15cf40 1 -- 172.21.15.38:0/3369057575 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:54 13.1c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e135) -- 0x55ea05aa7ff0 con 0x55ea05aa52b0 2024-09-21T03:44:35.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.096+0000 7f4044415640 1 -- 172.21.15.38:0/3369057575 <== osd.0 v2:172.21.15.38:6800/2381639433 2 ==== osd_op_reply(54 607.00000000 [delete] v135'232 uv232 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4038020070 con 0x55ea05aa52b0 2024-09-21T03:44:35.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.096+0000 7f404b15cf40 1 -- 172.21.15.38:0/3369057575 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:55 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55ea05aa84d0 con 0x55ea05aa52b0 2024-09-21T03:44:35.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.097+0000 7f4044415640 1 -- 172.21.15.38:0/3369057575 <== osd.0 v2:172.21.15.38:6800/2381639433 3 ==== osd_op_reply(55 607.00000000 [create] v135'233 uv233 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4038020070 con 0x55ea05aa52b0 2024-09-21T03:44:35.100 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-09-21T03:44:35.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.097+0000 7f404b15cf40 1 -- 172.21.15.38:0/3369057575 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:56 13.1c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55ea05aa42b0 con 0x55ea05aa52b0 2024-09-21T03:44:35.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.099+0000 7f4044415640 1 -- 172.21.15.38:0/3369057575 <== osd.0 v2:172.21.15.38:6800/2381639433 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v135'234 uv234 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4038020070 con 0x55ea05aa52b0 2024-09-21T03:44:35.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.099+0000 7f404b15cf40 1 -- 172.21.15.38:0/3369057575 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:57 13.1c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55ea05aa9e60 con 0x55ea05aa52b0 2024-09-21T03:44:35.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.100+0000 7f4044415640 1 -- 172.21.15.38:0/3369057575 <== osd.0 v2:172.21.15.38:6800/2381639433 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v135'235 uv235 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4038020070 con 0x55ea05aa52b0 2024-09-21T03:44:35.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.100+0000 7f404b15cf40 1 -- 172.21.15.38:0/3369057575 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:58 13.1c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55ea05aa8d70 con 0x55ea05aa52b0 2024-09-21T03:44:35.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.101+0000 7f4044415640 1 -- 172.21.15.38:0/3369057575 <== osd.0 v2:172.21.15.38:6800/2381639433 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v135'236 uv236 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4038020070 con 0x55ea05aa52b0 2024-09-21T03:44:35.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.101+0000 7f404b15cf40 1 --2- 172.21.15.38:0/3369057575 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x55ea05aa9850 0x55ea05aac2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:35.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.101+0000 7f404b15cf40 1 -- 172.21.15.38:0/3369057575 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:59 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55ea05aac870 con 0x55ea05aa9850 2024-09-21T03:44:35.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.101+0000 7f4045417640 1 --2- 172.21.15.38:0/3369057575 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x55ea05aa9850 0x55ea05aac2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:35.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.102+0000 7f4045417640 1 --2- 172.21.15.38:0/3369057575 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x55ea05aa9850 0x55ea05aac2c0 crc :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:35.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.103+0000 7f4045417640 1 -- 172.21.15.38:0/3369057575 <== osd.3 v2:172.21.15.38:6808/1286810294 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v135'172 uv171 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f403c00b040 con 0x55ea05aa9850 2024-09-21T03:44:35.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.103+0000 7f404b15cf40 1 -- 172.21.15.38:0/3369057575 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:60 13.6 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e135) -- 0x55ea05aacc60 con 0x55ea05aa9850 2024-09-21T03:44:35.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.104+0000 7f4045417640 1 -- 172.21.15.38:0/3369057575 <== osd.3 v2:172.21.15.38:6808/1286810294 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v135'173 uv173 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f403c00b040 con 0x55ea05aa9850 2024-09-21T03:44:35.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.104+0000 7f404b15cf40 1 -- 172.21.15.38:0/3369057575 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:61 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55ea05aacc60 con 0x55ea05aa9850 2024-09-21T03:44:35.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.105+0000 7f4045417640 1 -- 172.21.15.38:0/3369057575 <== osd.3 v2:172.21.15.38:6808/1286810294 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v135'174 uv174 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f403c00b040 con 0x55ea05aa9850 2024-09-21T03:44:35.108 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-09-21T03:44:35.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.105+0000 7f404b15cf40 1 -- 172.21.15.38:0/3369057575 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:62 13.6 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55ea05aacc60 con 0x55ea05aa9850 2024-09-21T03:44:35.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.106+0000 7f4045417640 1 -- 172.21.15.38:0/3369057575 <== osd.3 v2:172.21.15.38:6808/1286810294 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v135'175 uv175 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f403c00b040 con 0x55ea05aa9850 2024-09-21T03:44:35.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.106+0000 7f404b15cf40 1 -- 172.21.15.38:0/3369057575 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55ea05aacba0 con 0x55ea05aa52b0 2024-09-21T03:44:35.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.107+0000 7f4044415640 1 -- 172.21.15.38:0/3369057575 <== osd.0 v2:172.21.15.38:6800/2381639433 7 ==== osd_op_reply(63 608.00000000 [create] v135'231 uv230 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f4038020070 con 0x55ea05aa52b0 2024-09-21T03:44:35.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.108+0000 7f404b15cf40 1 -- 172.21.15.38:0/3369057575 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e135) -- 0x55ea05aad080 con 0x55ea05aa52b0 2024-09-21T03:44:35.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.109+0000 7f4044415640 1 -- 172.21.15.38:0/3369057575 <== osd.0 v2:172.21.15.38:6800/2381639433 8 ==== osd_op_reply(64 608.00000000 [delete] v135'232 uv232 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4038020070 con 0x55ea05aa52b0 2024-09-21T03:44:35.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.109+0000 7f404b15cf40 1 -- 172.21.15.38:0/3369057575 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55ea05aad420 con 0x55ea05aa52b0 2024-09-21T03:44:35.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.110+0000 7f4044415640 1 -- 172.21.15.38:0/3369057575 <== osd.0 v2:172.21.15.38:6800/2381639433 9 ==== osd_op_reply(65 608.00000000 [create] v135'233 uv233 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4038020070 con 0x55ea05aa52b0 2024-09-21T03:44:35.113 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-09-21T03:44:35.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.110+0000 7f404b15cf40 1 -- 172.21.15.38:0/3369057575 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55ea05aae430 con 0x55ea05aa52b0 2024-09-21T03:44:35.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.111+0000 7f4044415640 1 -- 172.21.15.38:0/3369057575 <== osd.0 v2:172.21.15.38:6800/2381639433 10 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v135'234 uv234 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4038020070 con 0x55ea05aa52b0 2024-09-21T03:44:35.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.111+0000 7f404b15cf40 1 -- 172.21.15.38:0/3369057575 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55ea05aae430 con 0x55ea05aa52b0 2024-09-21T03:44:35.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.112+0000 7f4044415640 1 -- 172.21.15.38:0/3369057575 <== osd.0 v2:172.21.15.38:6800/2381639433 11 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v135'235 uv235 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4038020070 con 0x55ea05aa52b0 2024-09-21T03:44:35.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.112+0000 7f404b15cf40 1 -- 172.21.15.38:0/3369057575 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55ea05aad420 con 0x55ea05aa52b0 2024-09-21T03:44:35.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.113+0000 7f4044415640 1 -- 172.21.15.38:0/3369057575 <== osd.0 v2:172.21.15.38:6800/2381639433 12 ==== osd_op_reply(68 608.00000000 [omap-set-header] v135'236 uv236 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4038020070 con 0x55ea05aa52b0 2024-09-21T03:44:35.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.113+0000 7f404b15cf40 1 -- 172.21.15.38:0/3369057575 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55ea05aad420 con 0x55ea05aa9850 2024-09-21T03:44:35.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.116+0000 7f4045417640 1 -- 172.21.15.38:0/3369057575 <== osd.3 v2:172.21.15.38:6808/1286810294 5 ==== osd_op_reply(69 604.00000000 [create] v135'426 uv420 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f403c00b040 con 0x55ea05aa9850 2024-09-21T03:44:35.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.116+0000 7f404b15cf40 1 -- 172.21.15.38:0/3369057575 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e135) -- 0x55ea05aad420 con 0x55ea05aa9850 2024-09-21T03:44:35.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.117+0000 7f4045417640 1 -- 172.21.15.38:0/3369057575 <== osd.3 v2:172.21.15.38:6808/1286810294 6 ==== osd_op_reply(70 604.00000000 [delete] v135'427 uv427 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f403c00b040 con 0x55ea05aa9850 2024-09-21T03:44:35.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.117+0000 7f404b15cf40 1 -- 172.21.15.38:0/3369057575 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55ea05aad420 con 0x55ea05aa9850 2024-09-21T03:44:35.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.119+0000 7f4045417640 1 -- 172.21.15.38:0/3369057575 <== osd.3 v2:172.21.15.38:6808/1286810294 7 ==== osd_op_reply(71 604.00000000 [create] v135'428 uv428 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f403c00b040 con 0x55ea05aa9850 2024-09-21T03:44:35.121 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-09-21T03:44:35.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.119+0000 7f404b15cf40 1 -- 172.21.15.38:0/3369057575 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55ea05aae430 con 0x55ea05aa9850 2024-09-21T03:44:35.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.120+0000 7f4045417640 1 -- 172.21.15.38:0/3369057575 <== osd.3 v2:172.21.15.38:6808/1286810294 8 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v135'429 uv429 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f403c00b040 con 0x55ea05aa9850 2024-09-21T03:44:35.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.120+0000 7f404b15cf40 1 -- 172.21.15.38:0/3369057575 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55ea05aae430 con 0x55ea05aa9850 2024-09-21T03:44:35.123 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.120+0000 7f4045417640 1 -- 172.21.15.38:0/3369057575 <== osd.3 v2:172.21.15.38:6808/1286810294 9 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v135'430 uv430 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f403c00b040 con 0x55ea05aa9850 2024-09-21T03:44:35.123 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.120+0000 7f404b15cf40 1 -- 172.21.15.38:0/3369057575 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55ea05aad420 con 0x55ea05aa9850 2024-09-21T03:44:35.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.121+0000 7f4045417640 1 -- 172.21.15.38:0/3369057575 <== osd.3 v2:172.21.15.38:6808/1286810294 10 ==== osd_op_reply(74 604.00000000 [omap-set-header] v135'431 uv431 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f403c00b040 con 0x55ea05aa9850 2024-09-21T03:44:35.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.122+0000 7f404b15cf40 1 -- 172.21.15.38:0/3369057575 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55ea05aad420 con 0x55ea05aa9850 2024-09-21T03:44:35.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.123+0000 7f4045417640 1 -- 172.21.15.38:0/3369057575 <== osd.3 v2:172.21.15.38:6808/1286810294 11 ==== osd_op_reply(75 500.00000000 [create] v135'432 uv425 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f403c00b040 con 0x55ea05aa9850 2024-09-21T03:44:35.126 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.123+0000 7f404b15cf40 1 -- 172.21.15.38:0/3369057575 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e135) -- 0x55ea05aad420 con 0x55ea05aa9850 2024-09-21T03:44:35.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.124+0000 7f4045417640 1 -- 172.21.15.38:0/3369057575 <== osd.3 v2:172.21.15.38:6808/1286810294 12 ==== osd_op_reply(76 500.00000000 [delete] v135'433 uv433 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f403c00b040 con 0x55ea05aa9850 2024-09-21T03:44:35.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.124+0000 7f404b15cf40 1 -- 172.21.15.38:0/3369057575 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55ea05aad420 con 0x55ea05aa9850 2024-09-21T03:44:35.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.125+0000 7f4045417640 1 -- 172.21.15.38:0/3369057575 <== osd.3 v2:172.21.15.38:6808/1286810294 13 ==== osd_op_reply(77 500.00000000 [create] v135'434 uv434 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f403c00b040 con 0x55ea05aa9850 2024-09-21T03:44:35.128 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-09-21T03:44:35.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.125+0000 7f404b15cf40 1 -- 172.21.15.38:0/3369057575 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55ea05aad420 con 0x55ea05aa9850 2024-09-21T03:44:35.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.126+0000 7f4045417640 1 -- 172.21.15.38:0/3369057575 <== osd.3 v2:172.21.15.38:6808/1286810294 14 ==== osd_op_reply(78 500.00000000 [write 0~90] v135'435 uv435 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f403c00b040 con 0x55ea05aa9850 2024-09-21T03:44:35.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.126+0000 7f404b15cf40 1 -- 172.21.15.38:0/3369057575 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55ea05aad420 con 0x55ea05aa9850 2024-09-21T03:44:35.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.127+0000 7f4045417640 1 -- 172.21.15.38:0/3369057575 <== osd.3 v2:172.21.15.38:6808/1286810294 15 ==== osd_op_reply(79 500.00000000 [omap-set-header] v135'436 uv436 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f403c00b040 con 0x55ea05aa9850 2024-09-21T03:44:35.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.127+0000 7f404b15cf40 1 -- 172.21.15.38:0/3369057575 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:80 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55ea05aad420 con 0x55ea05aa52b0 2024-09-21T03:44:35.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.129+0000 7f4044415640 1 -- 172.21.15.38:0/3369057575 <== osd.0 v2:172.21.15.38:6800/2381639433 13 ==== osd_op_reply(80 mds_snaptable [create] v135'193 uv192 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f4038020070 con 0x55ea05aa52b0 2024-09-21T03:44:35.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.129+0000 7f404b15cf40 1 -- 172.21.15.38:0/3369057575 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:81 13.d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e135) -- 0x55ea05aad420 con 0x55ea05aa52b0 2024-09-21T03:44:35.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.130+0000 7f4044415640 1 -- 172.21.15.38:0/3369057575 <== osd.0 v2:172.21.15.38:6800/2381639433 14 ==== osd_op_reply(81 mds_snaptable [delete] v135'194 uv194 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f4038020070 con 0x55ea05aa52b0 2024-09-21T03:44:35.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.130+0000 7f404b15cf40 1 -- 172.21.15.38:0/3369057575 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:82 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55ea05aad420 con 0x55ea05aa52b0 2024-09-21T03:44:35.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.131+0000 7f4044415640 1 -- 172.21.15.38:0/3369057575 <== osd.0 v2:172.21.15.38:6800/2381639433 15 ==== osd_op_reply(82 mds_snaptable [create] v135'195 uv195 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f4038020070 con 0x55ea05aa52b0 2024-09-21T03:44:35.134 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-09-21T03:44:35.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.131+0000 7f404b15cf40 1 -- 172.21.15.38:0/3369057575 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:83 13.d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55ea05aad420 con 0x55ea05aa52b0 2024-09-21T03:44:35.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.132+0000 7f4044415640 1 -- 172.21.15.38:0/3369057575 <== osd.0 v2:172.21.15.38:6800/2381639433 16 ==== osd_op_reply(83 mds_snaptable [write 0~70] v135'196 uv196 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f4038020070 con 0x55ea05aa52b0 2024-09-21T03:44:35.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.132+0000 7f404b15cf40 1 -- 172.21.15.38:0/3369057575 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:84 13.d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55ea05aad420 con 0x55ea05aa52b0 2024-09-21T03:44:35.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.133+0000 7f4044415640 1 -- 172.21.15.38:0/3369057575 <== osd.0 v2:172.21.15.38:6800/2381639433 17 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v135'197 uv197 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f4038020070 con 0x55ea05aa52b0 2024-09-21T03:44:35.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.133+0000 7f404b15cf40 1 --2- 172.21.15.38:0/3369057575 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x55ea05aadd20 0x55ea05aae130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:35.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.134+0000 7f404b15cf40 1 -- 172.21.15.38:0/3369057575 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55ea05ab0840 con 0x55ea05aadd20 2024-09-21T03:44:35.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.134+0000 7f4044c16640 1 --2- 172.21.15.38:0/3369057575 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x55ea05aadd20 0x55ea05aae130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:35.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.134+0000 7f4044c16640 1 --2- 172.21.15.38:0/3369057575 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x55ea05aadd20 0x55ea05aae130 crc :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:35.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.136+0000 7f4044c16640 1 -- 172.21.15.38:0/3369057575 <== osd.1 v2:172.21.15.38:6816/1491818732 1 ==== osd_op_reply(85 605.00000000 [create] v135'231 uv230 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f40340a1a50 con 0x55ea05aadd20 2024-09-21T03:44:35.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.136+0000 7f404b15cf40 1 -- 172.21.15.38:0/3369057575 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e135) -- 0x55ea05ab0840 con 0x55ea05aadd20 2024-09-21T03:44:35.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.137+0000 7f4044c16640 1 -- 172.21.15.38:0/3369057575 <== osd.1 v2:172.21.15.38:6816/1491818732 2 ==== osd_op_reply(86 605.00000000 [delete] v135'232 uv232 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f40340a1a50 con 0x55ea05aadd20 2024-09-21T03:44:35.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.137+0000 7f404b15cf40 1 -- 172.21.15.38:0/3369057575 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55ea05ab0840 con 0x55ea05aadd20 2024-09-21T03:44:35.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.138+0000 7f4044c16640 1 -- 172.21.15.38:0/3369057575 <== osd.1 v2:172.21.15.38:6816/1491818732 3 ==== osd_op_reply(87 605.00000000 [create] v135'233 uv233 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f40340a1a50 con 0x55ea05aadd20 2024-09-21T03:44:35.141 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-09-21T03:44:35.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.138+0000 7f404b15cf40 1 -- 172.21.15.38:0/3369057575 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55ea05ab1c70 con 0x55ea05aadd20 2024-09-21T03:44:35.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.139+0000 7f4044c16640 1 -- 172.21.15.38:0/3369057575 <== osd.1 v2:172.21.15.38:6816/1491818732 4 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v135'234 uv234 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f40340a1a50 con 0x55ea05aadd20 2024-09-21T03:44:35.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.139+0000 7f404b15cf40 1 -- 172.21.15.38:0/3369057575 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55ea05ab1c70 con 0x55ea05aadd20 2024-09-21T03:44:35.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.140+0000 7f4044c16640 1 -- 172.21.15.38:0/3369057575 <== osd.1 v2:172.21.15.38:6816/1491818732 5 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v135'235 uv235 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f40340a1a50 con 0x55ea05aadd20 2024-09-21T03:44:35.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.140+0000 7f404b15cf40 1 -- 172.21.15.38:0/3369057575 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55ea05ab0c60 con 0x55ea05aadd20 2024-09-21T03:44:35.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.141+0000 7f4044c16640 1 -- 172.21.15.38:0/3369057575 <== osd.1 v2:172.21.15.38:6816/1491818732 6 ==== osd_op_reply(90 605.00000000 [omap-set-header] v135'236 uv236 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f40340a1a50 con 0x55ea05aadd20 2024-09-21T03:44:35.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.141+0000 7f404b15cf40 1 -- 172.21.15.38:0/3369057575 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55ea05ab0c60 con 0x55ea05aadd20 2024-09-21T03:44:35.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.142+0000 7f4044c16640 1 -- 172.21.15.38:0/3369057575 <== osd.1 v2:172.21.15.38:6816/1491818732 7 ==== osd_op_reply(91 mds0_inotable [create] v135'386 uv380 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f40340a1a50 con 0x55ea05aadd20 2024-09-21T03:44:35.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.142+0000 7f404b15cf40 1 -- 172.21.15.38:0/3369057575 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e135) -- 0x55ea05ab0c60 con 0x55ea05aadd20 2024-09-21T03:44:35.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.143+0000 7f4044c16640 1 -- 172.21.15.38:0/3369057575 <== osd.1 v2:172.21.15.38:6816/1491818732 8 ==== osd_op_reply(92 mds0_inotable [delete] v135'387 uv387 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f40340a1a50 con 0x55ea05aadd20 2024-09-21T03:44:35.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.143+0000 7f404b15cf40 1 -- 172.21.15.38:0/3369057575 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55ea05ab0c60 con 0x55ea05aadd20 2024-09-21T03:44:35.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.144+0000 7f4044c16640 1 -- 172.21.15.38:0/3369057575 <== osd.1 v2:172.21.15.38:6816/1491818732 9 ==== osd_op_reply(93 mds0_inotable [create] v135'388 uv388 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f40340a1a50 con 0x55ea05aadd20 2024-09-21T03:44:35.147 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-09-21T03:44:35.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.144+0000 7f404b15cf40 1 -- 172.21.15.38:0/3369057575 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:94 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55ea05ab0c60 con 0x55ea05aadd20 2024-09-21T03:44:35.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.145+0000 7f4044c16640 1 -- 172.21.15.38:0/3369057575 <== osd.1 v2:172.21.15.38:6816/1491818732 10 ==== osd_op_reply(94 mds0_inotable [write 0~34] v135'389 uv389 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f40340a1a50 con 0x55ea05aadd20 2024-09-21T03:44:35.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.145+0000 7f404b15cf40 1 -- 172.21.15.38:0/3369057575 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:95 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55ea05ab0c60 con 0x55ea05aadd20 2024-09-21T03:44:35.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.146+0000 7f4044c16640 1 -- 172.21.15.38:0/3369057575 <== osd.1 v2:172.21.15.38:6816/1491818732 11 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v135'390 uv390 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f40340a1a50 con 0x55ea05aadd20 2024-09-21T03:44:35.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.146+0000 7f404b15cf40 1 -- 172.21.15.38:0/3369057575 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:96 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55ea05ab0c60 con 0x55ea05aadd20 2024-09-21T03:44:35.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.147+0000 7f4044c16640 1 -- 172.21.15.38:0/3369057575 <== osd.1 v2:172.21.15.38:6816/1491818732 12 ==== osd_op_reply(96 100.00000000 [create] v135'391 uv385 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f40340a1a50 con 0x55ea05aadd20 2024-09-21T03:44:35.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.147+0000 7f404b15cf40 1 -- 172.21.15.38:0/3369057575 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:97 13.13 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e135) -- 0x55ea05ab0c60 con 0x55ea05aadd20 2024-09-21T03:44:35.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.148+0000 7f4044c16640 1 -- 172.21.15.38:0/3369057575 <== osd.1 v2:172.21.15.38:6816/1491818732 13 ==== osd_op_reply(97 100.00000000 [delete] v135'392 uv392 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f40340a1a50 con 0x55ea05aadd20 2024-09-21T03:44:35.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.148+0000 7f404b15cf40 1 -- 172.21.15.38:0/3369057575 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:98 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55ea05ab0c60 con 0x55ea05aadd20 2024-09-21T03:44:35.153 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-09-21T03:44:35.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.149+0000 7f4044c16640 1 -- 172.21.15.38:0/3369057575 <== osd.1 v2:172.21.15.38:6816/1491818732 14 ==== osd_op_reply(98 100.00000000 [create] v135'393 uv393 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f40340a1a50 con 0x55ea05aadd20 2024-09-21T03:44:35.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.149+0000 7f404b15cf40 1 -- 172.21.15.38:0/3369057575 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:99 13.13 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55ea05ab0c60 con 0x55ea05aadd20 2024-09-21T03:44:35.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.150+0000 7f4044c16640 1 -- 172.21.15.38:0/3369057575 <== osd.1 v2:172.21.15.38:6816/1491818732 15 ==== osd_op_reply(99 100.00000000 [omap-set-header] v135'394 uv394 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f40340a1a50 con 0x55ea05aadd20 2024-09-21T03:44:35.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.150+0000 7f404b15cf40 1 -- 172.21.15.38:0/3369057575 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:100 13.13 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55ea05ab3a70 con 0x55ea05aadd20 2024-09-21T03:44:35.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.151+0000 7f4044c16640 1 -- 172.21.15.38:0/3369057575 <== osd.1 v2:172.21.15.38:6816/1491818732 16 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v135'395 uv395 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f40340a1a50 con 0x55ea05aadd20 2024-09-21T03:44:35.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.151+0000 7f404b15cf40 1 -- 172.21.15.38:0/3369057575 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55ea05ab1110 con 0x55ea05aa9850 2024-09-21T03:44:35.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.152+0000 7f4045417640 1 -- 172.21.15.38:0/3369057575 <== osd.3 v2:172.21.15.38:6808/1286810294 16 ==== osd_op_reply(101 mds0_sessionmap [create] v135'172 uv171 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7f403c00b040 con 0x55ea05aa9850 2024-09-21T03:44:35.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.152+0000 7f404b15cf40 1 -- 172.21.15.38:0/3369057575 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e135) -- 0x55ea05ab1110 con 0x55ea05aa9850 2024-09-21T03:44:35.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.154+0000 7f4045417640 1 -- 172.21.15.38:0/3369057575 <== osd.3 v2:172.21.15.38:6808/1286810294 17 ==== osd_op_reply(102 mds0_sessionmap [delete] v135'173 uv173 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f403c00b040 con 0x55ea05aa9850 2024-09-21T03:44:35.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.154+0000 7f404b15cf40 1 -- 172.21.15.38:0/3369057575 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:103 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55ea05ab1110 con 0x55ea05aa9850 2024-09-21T03:44:35.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.155+0000 7f4045417640 1 -- 172.21.15.38:0/3369057575 <== osd.3 v2:172.21.15.38:6808/1286810294 18 ==== osd_op_reply(103 mds0_sessionmap [create] v135'174 uv174 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f403c00b040 con 0x55ea05aa9850 2024-09-21T03:44:35.157 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-09-21T03:44:35.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.155+0000 7f404b15cf40 1 -- 172.21.15.38:0/3369057575 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:104 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55ea05ab1110 con 0x55ea05aa9850 2024-09-21T03:44:35.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.156+0000 7f4045417640 1 -- 172.21.15.38:0/3369057575 <== osd.3 v2:172.21.15.38:6808/1286810294 19 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v135'175 uv175 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f403c00b040 con 0x55ea05aa9850 2024-09-21T03:44:35.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.156+0000 7f404b15cf40 1 -- 172.21.15.38:0/3369057575 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55ea05ab1110 con 0x55ea05aa9850 2024-09-21T03:44:35.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.157+0000 7f4045417640 1 -- 172.21.15.38:0/3369057575 <== osd.3 v2:172.21.15.38:6808/1286810294 20 ==== osd_op_reply(105 609.00000000 [create] v135'231 uv230 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f403c00b040 con 0x55ea05aa9850 2024-09-21T03:44:35.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.157+0000 7f404b15cf40 1 -- 172.21.15.38:0/3369057575 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e135) -- 0x55ea05ab1110 con 0x55ea05aa9850 2024-09-21T03:44:35.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.158+0000 7f4045417640 1 -- 172.21.15.38:0/3369057575 <== osd.3 v2:172.21.15.38:6808/1286810294 21 ==== osd_op_reply(106 609.00000000 [delete] v135'232 uv232 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f403c00b040 con 0x55ea05aa9850 2024-09-21T03:44:35.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.158+0000 7f404b15cf40 1 -- 172.21.15.38:0/3369057575 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55ea05ab1110 con 0x55ea05aa9850 2024-09-21T03:44:35.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.159+0000 7f4045417640 1 -- 172.21.15.38:0/3369057575 <== osd.3 v2:172.21.15.38:6808/1286810294 22 ==== osd_op_reply(107 609.00000000 [create] v135'233 uv233 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f403c00b040 con 0x55ea05aa9850 2024-09-21T03:44:35.162 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-09-21T03:44:35.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.159+0000 7f404b15cf40 1 -- 172.21.15.38:0/3369057575 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55ea05ab2120 con 0x55ea05aa9850 2024-09-21T03:44:35.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.160+0000 7f4045417640 1 -- 172.21.15.38:0/3369057575 <== osd.3 v2:172.21.15.38:6808/1286810294 23 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v135'234 uv234 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f403c00b040 con 0x55ea05aa9850 2024-09-21T03:44:35.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.160+0000 7f404b15cf40 1 -- 172.21.15.38:0/3369057575 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:109 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55ea05ab2120 con 0x55ea05aa9850 2024-09-21T03:44:35.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.161+0000 7f4045417640 1 -- 172.21.15.38:0/3369057575 <== osd.3 v2:172.21.15.38:6808/1286810294 24 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v135'235 uv235 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f403c00b040 con 0x55ea05aa9850 2024-09-21T03:44:35.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.161+0000 7f404b15cf40 1 -- 172.21.15.38:0/3369057575 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:110 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55ea05ab1110 con 0x55ea05aa9850 2024-09-21T03:44:35.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.162+0000 7f4045417640 1 -- 172.21.15.38:0/3369057575 <== osd.3 v2:172.21.15.38:6808/1286810294 25 ==== osd_op_reply(110 609.00000000 [omap-set-header] v135'236 uv236 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f403c00b040 con 0x55ea05aa9850 2024-09-21T03:44:35.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.162+0000 7f404b15cf40 1 --2- 172.21.15.38:0/3369057575 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x55ea05ab1620 0x55ea05ab1a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:35.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.162+0000 7f404b15cf40 1 -- 172.21.15.38:0/3369057575 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55ea05ab1f70 con 0x55ea05ab1620 2024-09-21T03:44:35.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.163+0000 7f4044415640 1 --2- 172.21.15.38:0/3369057575 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x55ea05ab1620 0x55ea05ab1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:35.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.163+0000 7f4044415640 1 --2- 172.21.15.38:0/3369057575 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x55ea05ab1620 0x55ea05ab1a30 crc :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:35.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.164+0000 7f4044415640 1 -- 172.21.15.38:0/3369057575 <== osd.2 v2:172.21.15.38:6824/2577858902 1 ==== osd_op_reply(111 400.00000000 [create] v135'196 uv195 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f4038020070 con 0x55ea05ab1620 2024-09-21T03:44:35.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.164+0000 7f404b15cf40 1 -- 172.21.15.38:0/3369057575 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e135) -- 0x55ea05ab1f70 con 0x55ea05ab1620 2024-09-21T03:44:35.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.165+0000 7f4044415640 1 -- 172.21.15.38:0/3369057575 <== osd.2 v2:172.21.15.38:6824/2577858902 2 ==== osd_op_reply(112 400.00000000 [delete] v135'197 uv197 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4038020070 con 0x55ea05ab1620 2024-09-21T03:44:35.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.165+0000 7f404b15cf40 1 -- 172.21.15.38:0/3369057575 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55ea05ab1f70 con 0x55ea05ab1620 2024-09-21T03:44:35.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.167+0000 7f4044415640 1 -- 172.21.15.38:0/3369057575 <== osd.2 v2:172.21.15.38:6824/2577858902 3 ==== osd_op_reply(113 400.00000000 [create] v135'198 uv198 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4038020070 con 0x55ea05ab1620 2024-09-21T03:44:35.169 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-09-21T03:44:35.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.167+0000 7f404b15cf40 1 -- 172.21.15.38:0/3369057575 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:114 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55ea05ab1f70 con 0x55ea05ab1620 2024-09-21T03:44:35.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.168+0000 7f4044415640 1 -- 172.21.15.38:0/3369057575 <== osd.2 v2:172.21.15.38:6824/2577858902 4 ==== osd_op_reply(114 400.00000000 [write 0~22] v135'199 uv199 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4038020070 con 0x55ea05ab1620 2024-09-21T03:44:35.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.168+0000 7f404b15cf40 1 -- 172.21.15.38:0/3369057575 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:115 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55ea05ab1f70 con 0x55ea05ab1620 2024-09-21T03:44:35.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.169+0000 7f4044415640 1 -- 172.21.15.38:0/3369057575 <== osd.2 v2:172.21.15.38:6824/2577858902 5 ==== osd_op_reply(115 400.00000000 [omap-set-header] v135'200 uv200 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4038020070 con 0x55ea05ab1620 2024-09-21T03:44:35.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.169+0000 7f404b15cf40 1 -- 172.21.15.38:0/3369057575 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:116 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55ea05ab1f70 con 0x55ea05aadd20 2024-09-21T03:44:35.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.170+0000 7f4044c16640 1 -- 172.21.15.38:0/3369057575 <== osd.1 v2:172.21.15.38:6816/1491818732 17 ==== osd_op_reply(116 100.00000000.inode [create] v135'480 uv472 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7f40340a1a50 con 0x55ea05aadd20 2024-09-21T03:44:35.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.170+0000 7f404b15cf40 1 -- 172.21.15.38:0/3369057575 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:117 13.1f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e135) -- 0x55ea05ab1f70 con 0x55ea05aadd20 2024-09-21T03:44:35.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.171+0000 7f4044c16640 1 -- 172.21.15.38:0/3369057575 <== osd.1 v2:172.21.15.38:6816/1491818732 18 ==== osd_op_reply(117 100.00000000.inode [delete] v135'481 uv481 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f40340a1a50 con 0x55ea05aadd20 2024-09-21T03:44:35.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.172+0000 7f404b15cf40 1 -- 172.21.15.38:0/3369057575 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:118 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55ea05ab1f70 con 0x55ea05aadd20 2024-09-21T03:44:35.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.173+0000 7f4044c16640 1 -- 172.21.15.38:0/3369057575 <== osd.1 v2:172.21.15.38:6816/1491818732 19 ==== osd_op_reply(118 100.00000000.inode [create] v135'482 uv482 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f40340a1a50 con 0x55ea05aadd20 2024-09-21T03:44:35.175 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-09-21T03:44:35.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.173+0000 7f404b15cf40 1 -- 172.21.15.38:0/3369057575 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:119 13.1f 13:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55ea05ab1f70 con 0x55ea05aadd20 2024-09-21T03:44:35.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.174+0000 7f4044c16640 1 -- 172.21.15.38:0/3369057575 <== osd.1 v2:172.21.15.38:6816/1491818732 20 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v135'483 uv483 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f40340a1a50 con 0x55ea05aadd20 2024-09-21T03:44:35.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.174+0000 7f404b15cf40 1 -- 172.21.15.38:0/3369057575 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:120 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55ea05ab1f70 con 0x55ea05aadd20 2024-09-21T03:44:35.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.175+0000 7f4044c16640 1 -- 172.21.15.38:0/3369057575 <== osd.1 v2:172.21.15.38:6816/1491818732 21 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v135'484 uv484 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f40340a1a50 con 0x55ea05aadd20 2024-09-21T03:44:35.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.175+0000 7f404b15cf40 1 -- 172.21.15.38:0/3369057575 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:121 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55ea05ab1f70 con 0x55ea05aadd20 2024-09-21T03:44:35.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.176+0000 7f4044c16640 1 -- 172.21.15.38:0/3369057575 <== osd.1 v2:172.21.15.38:6816/1491818732 22 ==== osd_op_reply(121 1.00000000 [create] v135'485 uv479 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7f40340a1a50 con 0x55ea05aadd20 2024-09-21T03:44:35.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.176+0000 7f404b15cf40 1 -- 172.21.15.38:0/3369057575 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:122 13.1f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e135) -- 0x55ea05ab1f70 con 0x55ea05aadd20 2024-09-21T03:44:35.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.177+0000 7f4044c16640 1 -- 172.21.15.38:0/3369057575 <== osd.1 v2:172.21.15.38:6816/1491818732 23 ==== osd_op_reply(122 1.00000000 [delete] v135'486 uv486 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f40340a1a50 con 0x55ea05aadd20 2024-09-21T03:44:35.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.177+0000 7f404b15cf40 1 -- 172.21.15.38:0/3369057575 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:123 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55ea05ab1f70 con 0x55ea05aadd20 2024-09-21T03:44:35.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.178+0000 7f4044c16640 1 -- 172.21.15.38:0/3369057575 <== osd.1 v2:172.21.15.38:6816/1491818732 24 ==== osd_op_reply(123 1.00000000 [create] v135'487 uv487 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f40340a1a50 con 0x55ea05aadd20 2024-09-21T03:44:35.181 INFO:teuthology.orchestra.run.smithi038.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-09-21T03:44:35.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.178+0000 7f404b15cf40 1 -- 172.21.15.38:0/3369057575 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:124 13.1f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55ea05ab1f70 con 0x55ea05aadd20 2024-09-21T03:44:35.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.179+0000 7f4044c16640 1 -- 172.21.15.38:0/3369057575 <== osd.1 v2:172.21.15.38:6816/1491818732 25 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v135'488 uv488 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f40340a1a50 con 0x55ea05aadd20 2024-09-21T03:44:35.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.179+0000 7f404b15cf40 1 -- 172.21.15.38:0/3369057575 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:125 13.1f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55ea05ab1f70 con 0x55ea05aadd20 2024-09-21T03:44:35.183 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.180+0000 7f4044c16640 1 -- 172.21.15.38:0/3369057575 <== osd.1 v2:172.21.15.38:6816/1491818732 26 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v135'489 uv489 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f40340a1a50 con 0x55ea05aadd20 2024-09-21T03:44:35.183 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.180+0000 7f404b15cf40 1 -- 172.21.15.38:0/3369057575 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:126 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55ea05ab1f70 con 0x55ea05aadd20 2024-09-21T03:44:35.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.181+0000 7f4044c16640 1 -- 172.21.15.38:0/3369057575 <== osd.1 v2:172.21.15.38:6816/1491818732 27 ==== osd_op_reply(126 1.00000000 [omap-set-header] v135'490 uv490 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f40340a1a50 con 0x55ea05aadd20 2024-09-21T03:44:35.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.181+0000 7f404b15cf40 1 -- 172.21.15.38:0/3369057575 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:127 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=584b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55ea05ab2500 con 0x55ea05aadd20 2024-09-21T03:44:35.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.186+0000 7f4044c16640 1 -- 172.21.15.38:0/3369057575 <== osd.1 v2:172.21.15.38:6816/1491818732 28 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v135'491 uv491 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f40340a1a50 con 0x55ea05aadd20 2024-09-21T03:44:35.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.186+0000 7f404b15cf40 1 -- 172.21.15.38:0/3369057575 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x55ea05aa52b0 msgr2=0x55ea05aa76d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:35.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.186+0000 7f404b15cf40 1 --2- 172.21.15.38:0/3369057575 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x55ea05aa52b0 0x55ea05aa76d0 crc :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:35.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.186+0000 7f404b15cf40 1 -- 172.21.15.38:0/3369057575 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x55ea05aadd20 msgr2=0x55ea05aae130 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:35.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.186+0000 7f404b15cf40 1 --2- 172.21.15.38:0/3369057575 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x55ea05aadd20 0x55ea05aae130 crc :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:35.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.186+0000 7f404b15cf40 1 -- 172.21.15.38:0/3369057575 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x55ea05ab1620 msgr2=0x55ea05ab1a30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:35.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.186+0000 7f404b15cf40 1 --2- 172.21.15.38:0/3369057575 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x55ea05ab1620 0x55ea05ab1a30 crc :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:35.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.186+0000 7f404b15cf40 1 -- 172.21.15.38:0/3369057575 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x55ea05aa9850 msgr2=0x55ea05aac2c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:35.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.186+0000 7f404b15cf40 1 --2- 172.21.15.38:0/3369057575 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x55ea05aa9850 0x55ea05aac2c0 crc :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:35.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.186+0000 7f404b15cf40 1 -- 172.21.15.38:0/3369057575 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x55ea05aa1110 msgr2=0x55ea05aa3530 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:35.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.186+0000 7f404b15cf40 1 --2- 172.21.15.38:0/3369057575 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x55ea05aa1110 0x55ea05aa3530 crc :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:35.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.186+0000 7f4045417640 1 -- 172.21.15.38:0/3369057575 reap_dead start 2024-09-21T03:44:35.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.186+0000 7f404b15cf40 1 -- 172.21.15.38:0/3369057575 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x55ea059de800 msgr2=0x55ea059f6930 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:35.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.186+0000 7f404b15cf40 1 --2- 172.21.15.38:0/3369057575 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x55ea059de800 0x55ea059f6930 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:35.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.186+0000 7f4045417640 1 -- 172.21.15.38:0/3369057575 reap_dead start 2024-09-21T03:44:35.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.186+0000 7f4045417640 1 -- 172.21.15.38:0/3369057575 reap_dead start 2024-09-21T03:44:35.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.186+0000 7f404b15cf40 1 -- 172.21.15.38:0/3369057575 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f40300775a0 msgr2=0x7f4030079ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:35.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.186+0000 7f404b15cf40 1 --2- 172.21.15.38:0/3369057575 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f40300775a0 0x7f4030079ab0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x55ea05951880 tx=0x7f4038002fb0 comp rx=0 tx=0).stop 2024-09-21T03:44:35.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.186+0000 7f404b15cf40 1 -- 172.21.15.38:0/3369057575 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ea05928240 msgr2=0x55ea05923970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:35.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.186+0000 7f404b15cf40 1 --2- 172.21.15.38:0/3369057575 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ea05928240 0x55ea05923970 secure :-1 s=READY pgs=2249 cs=0 l=1 rev1=1 crypto rx=0x7f4034025e80 tx=0x7f403404d6e0 comp rx=0 tx=0).stop 2024-09-21T03:44:35.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.186+0000 7f404b15cf40 1 -- 172.21.15.38:0/3369057575 shutdown_connections 2024-09-21T03:44:35.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.186+0000 7f404b15cf40 1 --2- 172.21.15.38:0/3369057575 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f40300775a0 0x7f4030079ab0 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:35.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.186+0000 7f404b15cf40 1 --2- 172.21.15.38:0/3369057575 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ea05928240 0x55ea05923970 unknown :-1 s=CLOSED pgs=2249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:35.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.186+0000 7f404b15cf40 1 -- 172.21.15.38:0/3369057575 >> 172.21.15.38:0/3369057575 conn(0x55ea05846ac0 msgr2=0x55ea059fd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:35.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.186+0000 7f404b15cf40 1 -- 172.21.15.38:0/3369057575 shutdown_connections 2024-09-21T03:44:35.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.186+0000 7f404b15cf40 1 -- 172.21.15.38:0/3369057575 wait complete. 2024-09-21T03:44:35.192 DEBUG:teuthology.orchestra.run.smithi038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata setomapheader 1.00000000 deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef 2024-09-21T03:44:35.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.288+0000 7f36c2c4af40 1 Processor -- start 2024-09-21T03:44:35.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.288+0000 7f36c2c4af40 1 -- start start 2024-09-21T03:44:35.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.289+0000 7f36c2c4af40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b84aea2090 0x55b84ae849e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:35.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.289+0000 7f36c2c4af40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b84af17520 0x55b84af2c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:35.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.289+0000 7f36c2c4af40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b84af280f0 0x55b84af2c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:35.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.289+0000 7f36c2c4af40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55b84ada8d60 con 0x55b84af280f0 2024-09-21T03:44:35.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.289+0000 7f36c2c4af40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55b84ae07430 con 0x55b84aea2090 2024-09-21T03:44:35.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.289+0000 7f36c2c4af40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55b84ae899b0 con 0x55b84af17520 2024-09-21T03:44:35.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.289+0000 7f36bcf07640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b84af280f0 0x55b84af2c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:35.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.289+0000 7f36bcf07640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b84af280f0 0x55b84af2c810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50036/0 (socket says 172.21.15.38:50036) 2024-09-21T03:44:35.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.289+0000 7f36bcf07640 1 -- 172.21.15.38:0/3796517897 learned_addr learned my addr 172.21.15.38:0/3796517897 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:44:35.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.289+0000 7f36bc706640 1 --2- 172.21.15.38:0/3796517897 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b84aea2090 0x55b84ae849e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:35.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.289+0000 7f36bbf05640 1 --2- 172.21.15.38:0/3796517897 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b84af17520 0x55b84af2c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:35.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.290+0000 7f36bcf07640 1 -- 172.21.15.38:0/3796517897 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b84af17520 msgr2=0x55b84af2c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:44:35.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.290+0000 7f36bcf07640 1 --2- 172.21.15.38:0/3796517897 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b84af17520 0x55b84af2c2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:35.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.290+0000 7f36bcf07640 1 -- 172.21.15.38:0/3796517897 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b84aea2090 msgr2=0x55b84ae849e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:44:35.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.290+0000 7f36bcf07640 1 --2- 172.21.15.38:0/3796517897 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b84aea2090 0x55b84ae849e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:35.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.290+0000 7f36bcf07640 1 -- 172.21.15.38:0/3796517897 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b84ae83620 con 0x55b84af280f0 2024-09-21T03:44:35.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.290+0000 7f36bbf05640 1 --2- 172.21.15.38:0/3796517897 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b84af17520 0x55b84af2c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:44:35.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.290+0000 7f36bcf07640 1 --2- 172.21.15.38:0/3796517897 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b84af280f0 0x55b84af2c810 secure :-1 s=READY pgs=2252 cs=0 l=1 rev1=1 crypto rx=0x7f36b400b6a0 tx=0x7f36b400bb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea4cc959053f463 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:35.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.290+0000 7f36baf03640 1 -- 172.21.15.38:0/3796517897 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36b4009a30 con 0x55b84af280f0 2024-09-21T03:44:35.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.290+0000 7f36baf03640 1 -- 172.21.15.38:0/3796517897 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f36b4009ce0 con 0x55b84af280f0 2024-09-21T03:44:35.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.290+0000 7f36baf03640 1 -- 172.21.15.38:0/3796517897 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36b400e6b0 con 0x55b84af280f0 2024-09-21T03:44:35.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.290+0000 7f36c2c4af40 1 -- 172.21.15.38:0/3796517897 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b84af280f0 msgr2=0x55b84af2c810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:35.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.290+0000 7f36c2c4af40 1 --2- 172.21.15.38:0/3796517897 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b84af280f0 0x55b84af2c810 secure :-1 s=READY pgs=2252 cs=0 l=1 rev1=1 crypto rx=0x7f36b400b6a0 tx=0x7f36b400bb60 comp rx=0 tx=0).stop 2024-09-21T03:44:35.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.291+0000 7f36c2c4af40 1 -- 172.21.15.38:0/3796517897 shutdown_connections 2024-09-21T03:44:35.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.291+0000 7f36c2c4af40 1 --2- 172.21.15.38:0/3796517897 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b84af280f0 0x55b84af2c810 unknown :-1 s=CLOSED pgs=2252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:35.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.291+0000 7f36c2c4af40 1 --2- 172.21.15.38:0/3796517897 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b84af17520 0x55b84af2c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:35.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.291+0000 7f36c2c4af40 1 --2- 172.21.15.38:0/3796517897 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b84aea2090 0x55b84ae849e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:35.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.291+0000 7f36c2c4af40 1 -- 172.21.15.38:0/3796517897 >> 172.21.15.38:0/3796517897 conn(0x55b84ad79ac0 msgr2=0x55b84ae840e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:35.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.291+0000 7f36c2c4af40 1 -- 172.21.15.38:0/3796517897 shutdown_connections 2024-09-21T03:44:35.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.291+0000 7f36c2c4af40 1 -- 172.21.15.38:0/3796517897 wait complete. 2024-09-21T03:44:35.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.291+0000 7f36c2c4af40 1 Processor -- start 2024-09-21T03:44:35.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.292+0000 7f36c2c4af40 1 -- start start 2024-09-21T03:44:35.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.293+0000 7f36c2c4af40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b84aea2090 0x55b84af137f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:35.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.293+0000 7f36c2c4af40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b84af17520 0x55b84af13d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:35.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.293+0000 7f36c2c4af40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b84af280f0 0x55b84af14270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:35.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.293+0000 7f36c2c4af40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55b84af27ba0 con 0x55b84aea2090 2024-09-21T03:44:35.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.293+0000 7f36c2c4af40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55b84af2f6d0 con 0x55b84af17520 2024-09-21T03:44:35.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.293+0000 7f36c2c4af40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55b84af14b30 con 0x55b84af280f0 2024-09-21T03:44:35.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.293+0000 7f36bc706640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b84aea2090 0x55b84af137f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:35.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.293+0000 7f36bc706640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b84aea2090 0x55b84af137f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50038/0 (socket says 172.21.15.38:50038) 2024-09-21T03:44:35.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.293+0000 7f36bc706640 1 -- 172.21.15.38:0/2517380624 learned_addr learned my addr 172.21.15.38:0/2517380624 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:44:35.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.293+0000 7f36bbf05640 1 --2- 172.21.15.38:0/2517380624 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b84af17520 0x55b84af13d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:35.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.293+0000 7f36bcf07640 1 --2- 172.21.15.38:0/2517380624 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b84af280f0 0x55b84af14270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:35.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.293+0000 7f36bc706640 1 -- 172.21.15.38:0/2517380624 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b84af280f0 msgr2=0x55b84af14270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:35.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.293+0000 7f36bc706640 1 --2- 172.21.15.38:0/2517380624 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b84af280f0 0x55b84af14270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:35.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.293+0000 7f36bc706640 1 -- 172.21.15.38:0/2517380624 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b84af17520 msgr2=0x55b84af13d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:35.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.293+0000 7f36bc706640 1 --2- 172.21.15.38:0/2517380624 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b84af17520 0x55b84af13d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:35.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.293+0000 7f36bc706640 1 -- 172.21.15.38:0/2517380624 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b84af29840 con 0x55b84aea2090 2024-09-21T03:44:35.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.293+0000 7f36bcf07640 1 --2- 172.21.15.38:0/2517380624 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b84af280f0 0x55b84af14270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:44:35.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.293+0000 7f36bc706640 1 --2- 172.21.15.38:0/2517380624 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b84aea2090 0x55b84af137f0 secure :-1 s=READY pgs=2253 cs=0 l=1 rev1=1 crypto rx=0x7f36ac046660 tx=0x7f36ac058490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:35.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.294+0000 7f36b9f01640 1 -- 172.21.15.38:0/2517380624 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36ac068070 con 0x55b84aea2090 2024-09-21T03:44:35.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.294+0000 7f36b9f01640 1 -- 172.21.15.38:0/2517380624 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f36ac0581c0 con 0x55b84aea2090 2024-09-21T03:44:35.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.294+0000 7f36c2c4af40 1 -- 172.21.15.38:0/2517380624 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55b84af29b20 con 0x55b84aea2090 2024-09-21T03:44:35.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.294+0000 7f36b9f01640 1 -- 172.21.15.38:0/2517380624 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36ac0588e0 con 0x55b84aea2090 2024-09-21T03:44:35.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.294+0000 7f36c2c4af40 1 -- 172.21.15.38:0/2517380624 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b84af2a0e0 con 0x55b84aea2090 2024-09-21T03:44:35.300 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.295+0000 7f36b9f01640 1 -- 172.21.15.38:0/2517380624 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f36ac06b070 con 0x55b84aea2090 2024-09-21T03:44:35.300 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.296+0000 7f36b9f01640 1 --2- 172.21.15.38:0/2517380624 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f36a80775a0 0x7f36a8079ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:35.301 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.296+0000 7f36b9f01640 1 -- 172.21.15.38:0/2517380624 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(135..135 src has 1..135) ==== 9658+0+0 (secure 0 0 0) 0x7f36ac0de7a0 con 0x55b84aea2090 2024-09-21T03:44:35.301 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.296+0000 7f36c2c4af40 1 --2- 172.21.15.38:0/2517380624 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x55b84af0e6d0 0x55b84afd1c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:35.301 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.296+0000 7f36bbf05640 1 --2- 172.21.15.38:0/2517380624 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f36a80775a0 0x7f36a8079ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:35.301 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.296+0000 7f36c2c4af40 1 -- 172.21.15.38:0/2517380624 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:1 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-header in=80b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55b84af277a0 con 0x55b84af0e6d0 2024-09-21T03:44:35.301 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.296+0000 7f36bcf07640 1 --2- 172.21.15.38:0/2517380624 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x55b84af0e6d0 0x55b84afd1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:35.301 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.297+0000 7f36bcf07640 1 --2- 172.21.15.38:0/2517380624 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x55b84af0e6d0 0x55b84afd1c30 crc :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:35.301 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.297+0000 7f36bbf05640 1 --2- 172.21.15.38:0/2517380624 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f36a80775a0 0x7f36a8079ab0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x55b84af2bb60 tx=0x7f36b0002790 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:35.301 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.298+0000 7f36bcf07640 1 -- 172.21.15.38:0/2517380624 <== osd.1 v2:172.21.15.38:6816/1491818732 1 ==== osd_op_reply(1 1.00000000 [omap-set-header] v135'492 uv492 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f36b4009830 con 0x55b84af0e6d0 2024-09-21T03:44:35.301 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.298+0000 7f36c2c4af40 1 -- 172.21.15.38:0/2517380624 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x55b84af0e6d0 msgr2=0x55b84afd1c30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:35.301 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.298+0000 7f36c2c4af40 1 --2- 172.21.15.38:0/2517380624 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x55b84af0e6d0 0x55b84afd1c30 crc :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:35.301 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.298+0000 7f36c2c4af40 1 -- 172.21.15.38:0/2517380624 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f36a80775a0 msgr2=0x7f36a8079ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:35.301 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.298+0000 7f36c2c4af40 1 --2- 172.21.15.38:0/2517380624 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f36a80775a0 0x7f36a8079ab0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x55b84af2bb60 tx=0x7f36b0002790 comp rx=0 tx=0).stop 2024-09-21T03:44:35.301 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.298+0000 7f36c2c4af40 1 -- 172.21.15.38:0/2517380624 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b84aea2090 msgr2=0x55b84af137f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:35.302 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.298+0000 7f36c2c4af40 1 --2- 172.21.15.38:0/2517380624 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b84aea2090 0x55b84af137f0 secure :-1 s=READY pgs=2253 cs=0 l=1 rev1=1 crypto rx=0x7f36ac046660 tx=0x7f36ac058490 comp rx=0 tx=0).stop 2024-09-21T03:44:35.302 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.298+0000 7f36bcf07640 1 -- 172.21.15.38:0/2517380624 reap_dead start 2024-09-21T03:44:35.302 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.298+0000 7f36c2c4af40 1 -- 172.21.15.38:0/2517380624 shutdown_connections 2024-09-21T03:44:35.302 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.298+0000 7f36c2c4af40 1 -- 172.21.15.38:0/2517380624 >> 172.21.15.38:0/2517380624 conn(0x55b84ad79ac0 msgr2=0x55b84ae849e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:35.302 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.298+0000 7f36c2c4af40 1 -- 172.21.15.38:0/2517380624 shutdown_connections 2024-09-21T03:44:35.302 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.298+0000 7f36c2c4af40 1 -- 172.21.15.38:0/2517380624 wait complete. 2024-09-21T03:44:35.303 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-21T03:44:35.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.421+0000 7f94bf320640 1 -- 172.21.15.38:0/2226607980 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f94b80d0170 msgr2=0x7f94b80d0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:35.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.421+0000 7f94bf320640 1 --2- 172.21.15.38:0/2226607980 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f94b80d0170 0x7f94b80d0be0 secure :-1 s=READY pgs=2254 cs=0 l=1 rev1=1 crypto rx=0x7f94b80d2080 tx=0x7f94a802a4d0 comp rx=0 tx=0).stop 2024-09-21T03:44:35.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.421+0000 7f94bf320640 1 -- 172.21.15.38:0/2226607980 shutdown_connections 2024-09-21T03:44:35.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.421+0000 7f94bf320640 1 --2- 172.21.15.38:0/2226607980 >> v2:172.21.15.195:3300/0 conn(0x7f94b80d1120 0x7f94b80ceb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:35.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.421+0000 7f94bf320640 1 --2- 172.21.15.38:0/2226607980 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f94b80d0170 0x7f94b80d0be0 unknown :-1 s=CLOSED pgs=2254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:35.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.421+0000 7f94bf320640 1 -- 172.21.15.38:0/2226607980 >> 172.21.15.38:0/2226607980 conn(0x7f94b809d830 msgr2=0x7f94b80b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:35.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.421+0000 7f94bf320640 1 -- 172.21.15.38:0/2226607980 shutdown_connections 2024-09-21T03:44:35.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.421+0000 7f94bf320640 1 -- 172.21.15.38:0/2226607980 wait complete. 2024-09-21T03:44:35.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.423+0000 7f94bf320640 1 Processor -- start 2024-09-21T03:44:35.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.423+0000 7f94bf320640 1 -- start start 2024-09-21T03:44:35.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.424+0000 7f94bf320640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f94b80d0170 0x7f94b806f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:35.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.424+0000 7f94bf320640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f94b80d1120 0x7f94b806cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:35.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.424+0000 7f94bf320640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f94b815ee80 0x7f94b81f62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:35.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.424+0000 7f94bf320640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f94b80b6700 con 0x7f94b80d1120 2024-09-21T03:44:35.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.424+0000 7f94bf320640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f94b80b6580 con 0x7f94b80d0170 2024-09-21T03:44:35.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.424+0000 7f94bf320640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f94b80b6880 con 0x7f94b815ee80 2024-09-21T03:44:35.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.424+0000 7f94bc8bb640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f94b80d1120 0x7f94b806cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:35.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.424+0000 7f94bc8bb640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f94b80d1120 0x7f94b806cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50056/0 (socket says 172.21.15.38:50056) 2024-09-21T03:44:35.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.424+0000 7f94bc8bb640 1 -- 172.21.15.38:0/1709079742 learned_addr learned my addr 172.21.15.38:0/1709079742 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:44:35.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.424+0000 7f94bd0bc640 1 --2- 172.21.15.38:0/1709079742 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f94b80d0170 0x7f94b806f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:35.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.424+0000 7f94bd8bd640 1 --2- 172.21.15.38:0/1709079742 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f94b815ee80 0x7f94b81f62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:35.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.425+0000 7f94bc8bb640 1 -- 172.21.15.38:0/1709079742 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f94b815ee80 msgr2=0x7f94b81f62e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:35.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.425+0000 7f94bc8bb640 1 --2- 172.21.15.38:0/1709079742 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f94b815ee80 0x7f94b81f62e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:35.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.425+0000 7f94bc8bb640 1 -- 172.21.15.38:0/1709079742 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f94b80d0170 msgr2=0x7f94b806f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:35.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.425+0000 7f94bc8bb640 1 --2- 172.21.15.38:0/1709079742 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f94b80d0170 0x7f94b806f890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:35.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.425+0000 7f94bc8bb640 1 -- 172.21.15.38:0/1709079742 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94b806d320 con 0x7f94b80d1120 2024-09-21T03:44:35.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.425+0000 7f94bd0bc640 1 --2- 172.21.15.38:0/1709079742 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f94b80d0170 0x7f94b806f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:44:35.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.425+0000 7f94bc8bb640 1 --2- 172.21.15.38:0/1709079742 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f94b80d1120 0x7f94b806cde0 secure :-1 s=READY pgs=2255 cs=0 l=1 rev1=1 crypto rx=0x7f94a802aac0 tx=0x7f94a80049d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:35.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.425+0000 7f94adffb640 1 -- 172.21.15.38:0/1709079742 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94a8031950 con 0x7f94b80d1120 2024-09-21T03:44:35.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.425+0000 7f94adffb640 1 -- 172.21.15.38:0/1709079742 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f94a8031be0 con 0x7f94b80d1120 2024-09-21T03:44:35.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.425+0000 7f94bf320640 1 -- 172.21.15.38:0/1709079742 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94b806d7a0 con 0x7f94b80d1120 2024-09-21T03:44:35.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.425+0000 7f94adffb640 1 -- 172.21.15.38:0/1709079742 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94a8005040 con 0x7f94b80d1120 2024-09-21T03:44:35.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.425+0000 7f94bf320640 1 -- 172.21.15.38:0/1709079742 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94b81f5ae0 con 0x7f94b80d1120 2024-09-21T03:44:35.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.426+0000 7f94bf320640 1 -- 172.21.15.38:0/1709079742 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94b815e2c0 con 0x7f94b80d1120 2024-09-21T03:44:35.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.427+0000 7f94adffb640 1 -- 172.21.15.38:0/1709079742 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f94a8005200 con 0x7f94b80d1120 2024-09-21T03:44:35.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.428+0000 7f94adffb640 1 --2- 172.21.15.38:0/1709079742 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f949c078bc0 0x7f949c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:35.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.428+0000 7f94adffb640 1 -- 172.21.15.38:0/1709079742 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(135..135 src has 1..135) ==== 9658+0+0 (secure 0 0 0) 0x7f94a80bcf60 con 0x7f94b80d1120 2024-09-21T03:44:35.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.428+0000 7f94bd0bc640 1 --2- 172.21.15.38:0/1709079742 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f949c078bc0 0x7f949c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:35.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.428+0000 7f94bd0bc640 1 --2- 172.21.15.38:0/1709079742 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f949c078bc0 0x7f949c07b0d0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f94b4002210 tx=0x7f94b4009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:35.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.432+0000 7f94adffb640 1 -- 172.21.15.38:0/1709079742 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f94b815e2c0 con 0x7f94b80d1120 2024-09-21T03:44:35.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.664+0000 7f94bf320640 1 -- 172.21.15.38:0/1709079742 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f94b80b6da0 con 0x7f94b80d1120 2024-09-21T03:44:35.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.932+0000 7f94adffb640 1 -- 172.21.15.38:0/1709079742 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v328) ==== 162+0+0 (secure 0 0 0) 0x7f94a803f030 con 0x7f94b80d1120 2024-09-21T03:44:35.934 INFO:teuthology.orchestra.run.smithi038.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-21T03:44:35.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.935+0000 7f94bf320640 1 -- 172.21.15.38:0/1709079742 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f949c078bc0 msgr2=0x7f949c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:35.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.935+0000 7f94bf320640 1 --2- 172.21.15.38:0/1709079742 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f949c078bc0 0x7f949c07b0d0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f94b4002210 tx=0x7f94b4009040 comp rx=0 tx=0).stop 2024-09-21T03:44:35.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.935+0000 7f94bf320640 1 -- 172.21.15.38:0/1709079742 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f94b80d1120 msgr2=0x7f94b806cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:35.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.935+0000 7f94bf320640 1 --2- 172.21.15.38:0/1709079742 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f94b80d1120 0x7f94b806cde0 secure :-1 s=READY pgs=2255 cs=0 l=1 rev1=1 crypto rx=0x7f94a802aac0 tx=0x7f94a80049d0 comp rx=0 tx=0).stop 2024-09-21T03:44:35.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.935+0000 7f94bf320640 1 -- 172.21.15.38:0/1709079742 shutdown_connections 2024-09-21T03:44:35.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.935+0000 7f94bf320640 1 --2- 172.21.15.38:0/1709079742 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f949c078bc0 0x7f949c07b0d0 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:35.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.935+0000 7f94bf320640 1 --2- 172.21.15.38:0/1709079742 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f94b815ee80 0x7f94b81f62e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:35.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.935+0000 7f94bf320640 1 --2- 172.21.15.38:0/1709079742 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f94b80d1120 0x7f94b806cde0 unknown :-1 s=CLOSED pgs=2255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:35.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.935+0000 7f94bf320640 1 --2- 172.21.15.38:0/1709079742 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f94b80d0170 0x7f94b806f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:35.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.935+0000 7f94bf320640 1 -- 172.21.15.38:0/1709079742 >> 172.21.15.38:0/1709079742 conn(0x7f94b809d830 msgr2=0x7f94b80b3fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:35.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.935+0000 7f94bf320640 1 -- 172.21.15.38:0/1709079742 shutdown_connections 2024-09-21T03:44:35.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:35.935+0000 7f94bf320640 1 -- 172.21.15.38:0/1709079742 wait complete. 2024-09-21T03:44:35.950 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:44:36.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:36.066+0000 7f06db09b640 1 -- 172.21.15.38:0/2020079753 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f06c0005350 msgr2=0x7f06c000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:36.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:36.067+0000 7f06db09b640 1 --2- 172.21.15.38:0/2020079753 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f06c0005350 0x7f06c000b9c0 secure :-1 s=READY pgs=2257 cs=0 l=1 rev1=1 crypto rx=0x7f06cc00c470 tx=0x7f06c0005830 comp rx=0 tx=0).stop 2024-09-21T03:44:36.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:36.067+0000 7f06db09b640 1 -- 172.21.15.38:0/2020079753 shutdown_connections 2024-09-21T03:44:36.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:36.067+0000 7f06db09b640 1 --2- 172.21.15.38:0/2020079753 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f06c0005350 0x7f06c000b9c0 unknown :-1 s=CLOSED pgs=2257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:36.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:36.067+0000 7f06db09b640 1 -- 172.21.15.38:0/2020079753 >> 172.21.15.38:0/2020079753 conn(0x7f06d40b6b20 msgr2=0x7f06d40d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:36.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:36.067+0000 7f06db09b640 1 -- 172.21.15.38:0/2020079753 shutdown_connections 2024-09-21T03:44:36.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:36.067+0000 7f06db09b640 1 -- 172.21.15.38:0/2020079753 wait complete. 2024-09-21T03:44:36.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:36.068+0000 7f06db09b640 1 Processor -- start 2024-09-21T03:44:36.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:36.069+0000 7f06db09b640 1 -- start start 2024-09-21T03:44:36.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:36.070+0000 7f06db09b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f06c0005350 0x7f06d41555f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:36.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:36.070+0000 7f06db09b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f06d415e4b0 0x7f06d41529d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:36.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:36.070+0000 7f06db09b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f06d415bc70 0x7f06d415c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:36.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:36.070+0000 7f06db09b640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f06d4162810 con 0x7f06c0005350 2024-09-21T03:44:36.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:36.070+0000 7f06db09b640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f06d4162690 con 0x7f06d415bc70 2024-09-21T03:44:36.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:36.070+0000 7f06db09b640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f06d4162990 con 0x7f06d415e4b0 2024-09-21T03:44:36.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:36.070+0000 7f06d8e37640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f06c0005350 0x7f06d41555f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:36.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:36.070+0000 7f06d8e37640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f06c0005350 0x7f06d41555f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50078/0 (socket says 172.21.15.38:50078) 2024-09-21T03:44:36.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:36.070+0000 7f06d8e37640 1 -- 172.21.15.38:0/2772719888 learned_addr learned my addr 172.21.15.38:0/2772719888 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:44:36.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:36.070+0000 7f06d8e37640 1 -- 172.21.15.38:0/2772719888 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f06d415e4b0 msgr2=0x7f06d41529d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:36.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:36.070+0000 7f06d8e37640 1 --2- 172.21.15.38:0/2772719888 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f06d415e4b0 0x7f06d41529d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:36.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:36.070+0000 7f06d8e37640 1 -- 172.21.15.38:0/2772719888 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f06d415bc70 msgr2=0x7f06d415c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:36.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:36.070+0000 7f06d8e37640 1 --2- 172.21.15.38:0/2772719888 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f06d415bc70 0x7f06d415c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:36.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:36.070+0000 7f06d8e37640 1 -- 172.21.15.38:0/2772719888 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06d415c5c0 con 0x7f06c0005350 2024-09-21T03:44:36.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:36.071+0000 7f06d8e37640 1 --2- 172.21.15.38:0/2772719888 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f06c0005350 0x7f06d41555f0 secure :-1 s=READY pgs=2258 cs=0 l=1 rev1=1 crypto rx=0x7f06c4003ed0 tx=0x7f06c4001630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:36.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:36.071+0000 7f06d17fa640 1 -- 172.21.15.38:0/2772719888 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06c4015070 con 0x7f06c0005350 2024-09-21T03:44:36.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:36.071+0000 7f06d17fa640 1 -- 172.21.15.38:0/2772719888 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f06c4011840 con 0x7f06c0005350 2024-09-21T03:44:36.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:36.071+0000 7f06db09b640 1 -- 172.21.15.38:0/2772719888 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06d4157030 con 0x7f06c0005350 2024-09-21T03:44:36.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:36.071+0000 7f06d17fa640 1 -- 172.21.15.38:0/2772719888 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06c4011c90 con 0x7f06c0005350 2024-09-21T03:44:36.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:36.071+0000 7f06db09b640 1 -- 172.21.15.38:0/2772719888 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06d41572f0 con 0x7f06c0005350 2024-09-21T03:44:36.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:36.072+0000 7f06db09b640 1 -- 172.21.15.38:0/2772719888 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06d40d2850 con 0x7f06c0005350 2024-09-21T03:44:36.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:36.073+0000 7f06d17fa640 1 -- 172.21.15.38:0/2772719888 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f06c4018070 con 0x7f06c0005350 2024-09-21T03:44:36.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:36.073+0000 7f06d17fa640 1 --2- 172.21.15.38:0/2772719888 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f06b4078b90 0x7f06b407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:36.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:36.073+0000 7f06d17fa640 1 -- 172.21.15.38:0/2772719888 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(135..135 src has 1..135) ==== 9658+0+0 (secure 0 0 0) 0x7f06c409c900 con 0x7f06c0005350 2024-09-21T03:44:36.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:36.074+0000 7f06d3fff640 1 --2- 172.21.15.38:0/2772719888 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f06b4078b90 0x7f06b407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:36.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:36.074+0000 7f06d3fff640 1 --2- 172.21.15.38:0/2772719888 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f06b4078b90 0x7f06b407b0a0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f06d415aa70 tx=0x7f06bc009780 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:36.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:36.078+0000 7f06d17fa640 1 -- 172.21.15.38:0/2772719888 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f06d415c5c0 con 0x7f06c0005350 2024-09-21T03:44:36.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:36.303+0000 7f06db09b640 1 -- 172.21.15.38:0/2772719888 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f06d41ea190 con 0x7f06c0005350 2024-09-21T03:44:36.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:36.303+0000 7f06d17fa640 1 -- 172.21.15.38:0/2772719888 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 329 v329) ==== 95+0+4401 (secure 0 0 0) 0x7f06c4063e10 con 0x7f06c0005350 2024-09-21T03:44:36.305 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:44:36.306 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":329,"btime":"2024-09-21T03:44:35:940492+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6086,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/965972663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":965972663},{"type":"v1","addr":"172.21.15.195:6836","nonce":965972663}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6700,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/3176122334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":3176122334},{"type":"v1","addr":"172.21.15.195:6837","nonce":3176122334}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":329,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:40:02.319282+0000","modified":"2024-09-21T03:44:35.940471+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":135,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9840},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9840":{"gid":9840,"name":"a","rank":0,"incarnation":329,"state":"up:replay","state_seq":1,"addr":"172.21.15.38:6835/3801772046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":3801772046},{"type":"v1","addr":"172.21.15.38:6835","nonce":3801772046}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-21T03:44:36.306 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 329 2024-09-21T03:44:36.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:36.305+0000 7f06db09b640 1 -- 172.21.15.38:0/2772719888 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f06b4078b90 msgr2=0x7f06b407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:36.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:36.305+0000 7f06db09b640 1 --2- 172.21.15.38:0/2772719888 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f06b4078b90 0x7f06b407b0a0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f06d415aa70 tx=0x7f06bc009780 comp rx=0 tx=0).stop 2024-09-21T03:44:36.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:36.305+0000 7f06db09b640 1 -- 172.21.15.38:0/2772719888 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f06c0005350 msgr2=0x7f06d41555f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:36.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:36.305+0000 7f06db09b640 1 --2- 172.21.15.38:0/2772719888 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f06c0005350 0x7f06d41555f0 secure :-1 s=READY pgs=2258 cs=0 l=1 rev1=1 crypto rx=0x7f06c4003ed0 tx=0x7f06c4001630 comp rx=0 tx=0).stop 2024-09-21T03:44:36.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:36.306+0000 7f06db09b640 1 -- 172.21.15.38:0/2772719888 shutdown_connections 2024-09-21T03:44:36.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:36.306+0000 7f06db09b640 1 --2- 172.21.15.38:0/2772719888 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f06d415bc70 0x7f06d415c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:36.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:36.306+0000 7f06db09b640 1 --2- 172.21.15.38:0/2772719888 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f06b4078b90 0x7f06b407b0a0 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:36.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:36.306+0000 7f06db09b640 1 --2- 172.21.15.38:0/2772719888 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f06d415e4b0 0x7f06d41529d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:36.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:36.306+0000 7f06db09b640 1 --2- 172.21.15.38:0/2772719888 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f06c0005350 0x7f06d41555f0 unknown :-1 s=CLOSED pgs=2258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:36.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:36.306+0000 7f06db09b640 1 -- 172.21.15.38:0/2772719888 >> 172.21.15.38:0/2772719888 conn(0x7f06d40b6b20 msgr2=0x7f06d415d0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:36.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:36.306+0000 7f06db09b640 1 -- 172.21.15.38:0/2772719888 shutdown_connections 2024-09-21T03:44:36.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:36.306+0000 7f06db09b640 1 -- 172.21.15.38:0/2772719888 wait complete. 2024-09-21T03:44:36.319 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2024-09-21T03:44:36.673 INFO:tasks.ceph.mds.c.smithi038.stderr: -50> 2024-09-21T03:44:31.919+0000 7fa1c2718640 -1 log_channel(cluster) log [ERR] : corrupt sessionmap header: Decoder at 'virtual void SessionMapStore::decode_header(ceph::bufferlist&)' v=1 cannot decode v=100 minimal_decoder=101: Malformed input [buffer:3] 2024-09-21T03:44:36.730 INFO:tasks.ceph.mds.c.smithi038.stdout:starting mds.c at 2024-09-21T03:44:38.967 INFO:tasks.ceph.mds.a.smithi038.stderr:2024-09-21T03:44:38.964+0000 7f53dcf61640 -1 mds.0.cache.dir(0x1) Corrupt fnode in dirfrag 0x1: Decoder at 'void fnode_t::decode(ceph::buffer::v15_2_0::list::const_iterator&)' v=3 cannot decode v=100 minimal_decoder=101: Malformed input [buffer:3] 2024-09-21T03:44:38.967 INFO:tasks.ceph.mds.a.smithi038.stderr:2024-09-21T03:44:38.964+0000 7f53dcf61640 -1 mds.0.damage notify_dirfrag Damage to fragment * of ino 0x1 is fatal because it is a system directory for this rank 2024-09-21T03:44:38.967 INFO:tasks.ceph.mon.a.smithi038.stderr:2024-09-21T03:44:38.966+0000 7f3f45713640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T03:44:38.967 INFO:tasks.ceph.mon.a.smithi038.stderr:2024-09-21T03:44:38.966+0000 7f3f45713640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-09-21T03:44:39.734 INFO:tasks.ceph.mds.a.smithi038.stderr:2024-09-21T03:44:39.732+0000 7f53e0f69640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T03:44:41.320 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:44:41.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:41.449+0000 7ff1d5893640 1 -- 172.21.15.38:0/2045144812 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff1b8004d40 msgr2=0x7ff1b8005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:41.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:41.449+0000 7ff1d5893640 1 --2- 172.21.15.38:0/2045144812 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff1b8004d40 0x7ff1b8005150 secure :-1 s=READY pgs=2263 cs=0 l=1 rev1=1 crypto rx=0x7ff1c8005fd0 tx=0x7ff1b8005830 comp rx=0 tx=0).stop 2024-09-21T03:44:41.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:41.449+0000 7ff1cf7fe640 1 -- 172.21.15.38:0/2045144812 reap_dead start 2024-09-21T03:44:41.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:41.449+0000 7ff1d5893640 1 -- 172.21.15.38:0/2045144812 shutdown_connections 2024-09-21T03:44:41.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:41.449+0000 7ff1d5893640 1 -- 172.21.15.38:0/2045144812 >> 172.21.15.38:0/2045144812 conn(0x7ff1d00b6b20 msgr2=0x7ff1d00d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:41.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:41.449+0000 7ff1d5893640 1 -- 172.21.15.38:0/2045144812 shutdown_connections 2024-09-21T03:44:41.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:41.449+0000 7ff1d5893640 1 -- 172.21.15.38:0/2045144812 wait complete. 2024-09-21T03:44:41.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:41.451+0000 7ff1d5893640 1 Processor -- start 2024-09-21T03:44:41.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:41.452+0000 7ff1d5893640 1 -- start start 2024-09-21T03:44:41.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:41.453+0000 7ff1d5893640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff1d01612b0 0x7ff1d00cb5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:41.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:41.453+0000 7ff1d5893640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff1d01496f0 0x7ff1d0149b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:41.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:41.453+0000 7ff1d5893640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff1d014a040 0x7ff1d01f8690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:41.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:41.453+0000 7ff1d5893640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ff1d00b35e0 con 0x7ff1d01496f0 2024-09-21T03:44:41.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:41.453+0000 7ff1d5893640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ff1d00b3460 con 0x7ff1d014a040 2024-09-21T03:44:41.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:41.453+0000 7ff1d5893640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7ff1d00b3760 con 0x7ff1d01612b0 2024-09-21T03:44:41.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:41.453+0000 7ff1ce7fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff1d01496f0 0x7ff1d0149b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:41.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:41.453+0000 7ff1ce7fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff1d01496f0 0x7ff1d0149b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50126/0 (socket says 172.21.15.38:50126) 2024-09-21T03:44:41.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:41.453+0000 7ff1ce7fc640 1 -- 172.21.15.38:0/2256377037 learned_addr learned my addr 172.21.15.38:0/2256377037 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:44:41.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:41.453+0000 7ff1ceffd640 1 --2- 172.21.15.38:0/2256377037 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff1d01612b0 0x7ff1d00cb5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:41.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:41.453+0000 7ff1cf7fe640 1 --2- 172.21.15.38:0/2256377037 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff1d014a040 0x7ff1d01f8690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:41.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:41.453+0000 7ff1ce7fc640 1 -- 172.21.15.38:0/2256377037 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff1d01612b0 msgr2=0x7ff1d00cb5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:41.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:41.453+0000 7ff1ce7fc640 1 --2- 172.21.15.38:0/2256377037 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff1d01612b0 0x7ff1d00cb5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:41.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:41.453+0000 7ff1ce7fc640 1 -- 172.21.15.38:0/2256377037 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff1d014a040 msgr2=0x7ff1d01f8690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:41.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:41.453+0000 7ff1ce7fc640 1 --2- 172.21.15.38:0/2256377037 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff1d014a040 0x7ff1d01f8690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:41.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:41.453+0000 7ff1ce7fc640 1 -- 172.21.15.38:0/2256377037 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1d01616c0 con 0x7ff1d01496f0 2024-09-21T03:44:41.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:41.454+0000 7ff1ce7fc640 1 --2- 172.21.15.38:0/2256377037 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff1d01496f0 0x7ff1d0149b00 secure :-1 s=READY pgs=2264 cs=0 l=1 rev1=1 crypto rx=0x7ff1c4011ae0 tx=0x7ff1c4009f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:41.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:41.454+0000 7ff1b7fff640 1 -- 172.21.15.38:0/2256377037 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1c400fcb0 con 0x7ff1d01496f0 2024-09-21T03:44:41.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:41.454+0000 7ff1b7fff640 1 -- 172.21.15.38:0/2256377037 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff1c400fe50 con 0x7ff1d01496f0 2024-09-21T03:44:41.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:41.454+0000 7ff1b7fff640 1 -- 172.21.15.38:0/2256377037 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1c4004280 con 0x7ff1d01496f0 2024-09-21T03:44:41.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:41.454+0000 7ff1d5893640 1 -- 172.21.15.38:0/2256377037 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1d0160210 con 0x7ff1d01496f0 2024-09-21T03:44:41.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:41.454+0000 7ff1d5893640 1 -- 172.21.15.38:0/2256377037 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1d01607d0 con 0x7ff1d01496f0 2024-09-21T03:44:41.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:41.455+0000 7ff1d5893640 1 -- 172.21.15.38:0/2256377037 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1d0162f00 con 0x7ff1d01496f0 2024-09-21T03:44:41.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:41.456+0000 7ff1b7fff640 1 -- 172.21.15.38:0/2256377037 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ff1c40035f0 con 0x7ff1d01496f0 2024-09-21T03:44:41.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:41.456+0000 7ff1b7fff640 1 --2- 172.21.15.38:0/2256377037 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff1bc078b90 0x7ff1bc07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:41.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:41.457+0000 7ff1b7fff640 1 -- 172.21.15.38:0/2256377037 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 9744+0+0 (secure 0 0 0) 0x7ff1c409c710 con 0x7ff1d01496f0 2024-09-21T03:44:41.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:41.457+0000 7ff1ceffd640 1 --2- 172.21.15.38:0/2256377037 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff1bc078b90 0x7ff1bc07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:41.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:41.457+0000 7ff1ceffd640 1 --2- 172.21.15.38:0/2256377037 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff1bc078b90 0x7ff1bc07b0a0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7ff1d00c8830 tx=0x7ff1c00012f0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:41.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:41.461+0000 7ff1b7fff640 1 -- 172.21.15.38:0/2256377037 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7ff1c40635d0 con 0x7ff1d01496f0 2024-09-21T03:44:41.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:41.691+0000 7ff1d5893640 1 -- 172.21.15.38:0/2256377037 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff1d015dfd0 con 0x7ff1d01496f0 2024-09-21T03:44:41.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:41.691+0000 7ff1b7fff640 1 -- 172.21.15.38:0/2256377037 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 333 v333) ==== 95+0+4390 (secure 0 0 0) 0x7ff1c4068000 con 0x7ff1d01496f0 2024-09-21T03:44:41.693 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:44:41.694 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":333,"btime":"2024-09-21T03:44:38:966738+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6086,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/965972663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":965972663},{"type":"v1","addr":"172.21.15.195:6836","nonce":965972663}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6700,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/3176122334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":3176122334},{"type":"v1","addr":"172.21.15.195:6837","nonce":3176122334}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":10551,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/791475497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":791475497},{"type":"v1","addr":"172.21.15.38:6833","nonce":791475497}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":330}],"filesystems":[{"mdsmap":{"epoch":333,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:40:02.319282+0000","modified":"2024-09-21T03:44:38.966733+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":136,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-21T03:44:41.694 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 333 2024-09-21T03:44:41.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:41.694+0000 7ff1d5893640 1 -- 172.21.15.38:0/2256377037 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff1bc078b90 msgr2=0x7ff1bc07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:41.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:41.694+0000 7ff1d5893640 1 --2- 172.21.15.38:0/2256377037 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff1bc078b90 0x7ff1bc07b0a0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7ff1d00c8830 tx=0x7ff1c00012f0 comp rx=0 tx=0).stop 2024-09-21T03:44:41.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:41.694+0000 7ff1d5893640 1 -- 172.21.15.38:0/2256377037 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff1d01496f0 msgr2=0x7ff1d0149b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:41.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:41.694+0000 7ff1d5893640 1 --2- 172.21.15.38:0/2256377037 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff1d01496f0 0x7ff1d0149b00 secure :-1 s=READY pgs=2264 cs=0 l=1 rev1=1 crypto rx=0x7ff1c4011ae0 tx=0x7ff1c4009f10 comp rx=0 tx=0).stop 2024-09-21T03:44:41.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:41.694+0000 7ff1d5893640 1 -- 172.21.15.38:0/2256377037 shutdown_connections 2024-09-21T03:44:41.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:41.694+0000 7ff1d5893640 1 --2- 172.21.15.38:0/2256377037 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff1d014a040 0x7ff1d01f8690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:41.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:41.694+0000 7ff1d5893640 1 --2- 172.21.15.38:0/2256377037 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff1d01496f0 0x7ff1d0149b00 unknown :-1 s=CLOSED pgs=2264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:41.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:41.694+0000 7ff1d5893640 1 --2- 172.21.15.38:0/2256377037 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff1bc078b90 0x7ff1bc07b0a0 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:41.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:41.694+0000 7ff1d5893640 1 --2- 172.21.15.38:0/2256377037 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff1d01612b0 0x7ff1d00cb5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:41.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:41.694+0000 7ff1d5893640 1 -- 172.21.15.38:0/2256377037 >> 172.21.15.38:0/2256377037 conn(0x7ff1d00b6b20 msgr2=0x7ff1d00d0c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:41.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:41.694+0000 7ff1d5893640 1 -- 172.21.15.38:0/2256377037 shutdown_connections 2024-09-21T03:44:41.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:41.694+0000 7ff1d5893640 1 -- 172.21.15.38:0/2256377037 wait complete. 2024-09-21T03:44:41.708 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-09-21T03:44:41.708 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:44:41.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:41.823+0000 7f98aba4d640 1 -- 172.21.15.38:0/2208312056 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f988c004d40 msgr2=0x7f988c005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:41.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:41.823+0000 7f98aba4d640 1 --2- 172.21.15.38:0/2208312056 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f988c004d40 0x7f988c005150 secure :-1 s=READY pgs=2266 cs=0 l=1 rev1=1 crypto rx=0x7f989c009b50 tx=0x7f988c005830 comp rx=0 tx=0).stop 2024-09-21T03:44:41.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:41.823+0000 7f98a9fea640 1 -- 172.21.15.38:0/2208312056 reap_dead start 2024-09-21T03:44:41.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:41.823+0000 7f98aba4d640 1 -- 172.21.15.38:0/2208312056 shutdown_connections 2024-09-21T03:44:41.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:41.823+0000 7f98aba4d640 1 -- 172.21.15.38:0/2208312056 >> 172.21.15.38:0/2208312056 conn(0x7f98a40d0a30 msgr2=0x7f98a40d0e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:41.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:41.823+0000 7f98aba4d640 1 -- 172.21.15.38:0/2208312056 shutdown_connections 2024-09-21T03:44:41.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:41.823+0000 7f98aba4d640 1 -- 172.21.15.38:0/2208312056 wait complete. 2024-09-21T03:44:41.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:41.825+0000 7f98aba4d640 1 Processor -- start 2024-09-21T03:44:41.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:41.825+0000 7f98aba4d640 1 -- start start 2024-09-21T03:44:41.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:41.826+0000 7f98aba4d640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f988c004d40 0x7f98a40cc960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:41.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:41.826+0000 7f98aba4d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f98a41481a0 0x7f98a41f9090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:41.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:41.826+0000 7f98aba4d640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f98a41f95d0 0x7f98a40aa3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:41.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:41.826+0000 7f98aba4d640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f98a406d6e0 con 0x7f98a41481a0 2024-09-21T03:44:41.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:41.826+0000 7f98aba4d640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f98a406d560 con 0x7f988c004d40 2024-09-21T03:44:41.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:41.826+0000 7f98aba4d640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f98a406d860 con 0x7f98a41f95d0 2024-09-21T03:44:41.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:41.826+0000 7f98a8fe8640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f98a41481a0 0x7f98a41f9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:41.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:41.826+0000 7f98a8fe8640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f98a41481a0 0x7f98a41f9090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50150/0 (socket says 172.21.15.38:50150) 2024-09-21T03:44:41.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:41.826+0000 7f98a8fe8640 1 -- 172.21.15.38:0/674825902 learned_addr learned my addr 172.21.15.38:0/674825902 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:44:41.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:41.827+0000 7f98a97e9640 1 --2- 172.21.15.38:0/674825902 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f988c004d40 0x7f98a40cc960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:41.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:41.827+0000 7f98a9fea640 1 --2- 172.21.15.38:0/674825902 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f98a41f95d0 0x7f98a40aa3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:41.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:41.827+0000 7f98a8fe8640 1 -- 172.21.15.38:0/674825902 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f98a41f95d0 msgr2=0x7f98a40aa3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:41.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:41.827+0000 7f98a8fe8640 1 --2- 172.21.15.38:0/674825902 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f98a41f95d0 0x7f98a40aa3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:41.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:41.827+0000 7f98a8fe8640 1 -- 172.21.15.38:0/674825902 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f988c004d40 msgr2=0x7f98a40cc960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:41.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:41.827+0000 7f98a8fe8640 1 --2- 172.21.15.38:0/674825902 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f988c004d40 0x7f98a40cc960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:41.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:41.827+0000 7f98a8fe8640 1 -- 172.21.15.38:0/674825902 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98a40a95e0 con 0x7f98a41481a0 2024-09-21T03:44:41.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:41.827+0000 7f98a97e9640 1 --2- 172.21.15.38:0/674825902 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f988c004d40 0x7f98a40cc960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:44:41.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:41.827+0000 7f98a9fea640 1 --2- 172.21.15.38:0/674825902 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f98a41f95d0 0x7f98a40aa3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:44:41.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:41.827+0000 7f98a8fe8640 1 --2- 172.21.15.38:0/674825902 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f98a41481a0 0x7f98a41f9090 secure :-1 s=READY pgs=2267 cs=0 l=1 rev1=1 crypto rx=0x7f9898006680 tx=0x7f98980066b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:41.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:41.827+0000 7f98a1ffb640 1 -- 172.21.15.38:0/674825902 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9898015070 con 0x7f98a41481a0 2024-09-21T03:44:41.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:41.827+0000 7f98a1ffb640 1 -- 172.21.15.38:0/674825902 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9898011920 con 0x7f98a41481a0 2024-09-21T03:44:41.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:41.827+0000 7f98aba4d640 1 -- 172.21.15.38:0/674825902 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98a40a98c0 con 0x7f98a41481a0 2024-09-21T03:44:41.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:41.827+0000 7f98a1ffb640 1 -- 172.21.15.38:0/674825902 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9898011d70 con 0x7f98a41481a0 2024-09-21T03:44:41.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:41.827+0000 7f98aba4d640 1 -- 172.21.15.38:0/674825902 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98a41485e0 con 0x7f98a41481a0 2024-09-21T03:44:41.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:41.828+0000 7f98aba4d640 1 -- 172.21.15.38:0/674825902 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98a41f9bd0 con 0x7f98a41481a0 2024-09-21T03:44:41.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:41.829+0000 7f98a1ffb640 1 -- 172.21.15.38:0/674825902 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f9898010420 con 0x7f98a41481a0 2024-09-21T03:44:41.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:41.830+0000 7f98a1ffb640 1 --2- 172.21.15.38:0/674825902 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9890078bc0 0x7f989007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:41.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:41.830+0000 7f98a1ffb640 1 -- 172.21.15.38:0/674825902 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 9744+0+0 (secure 0 0 0) 0x7f989809cb60 con 0x7f98a41481a0 2024-09-21T03:44:41.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:41.830+0000 7f98a97e9640 1 --2- 172.21.15.38:0/674825902 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9890078bc0 0x7f989007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:41.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:41.831+0000 7f98a97e9640 1 --2- 172.21.15.38:0/674825902 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9890078bc0 0x7f989007b0d0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f9894005fd0 tx=0x7f9894009500 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:41.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:41.835+0000 7f98a1ffb640 1 -- 172.21.15.38:0/674825902 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f9898063f50 con 0x7f98a41481a0 2024-09-21T03:44:42.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:42.061+0000 7f98aba4d640 1 -- 172.21.15.38:0/674825902 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f98a40d0640 con 0x7f98a41481a0 2024-09-21T03:44:42.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:42.062+0000 7f98a1ffb640 1 -- 172.21.15.38:0/674825902 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 333 v333) ==== 95+0+4390 (secure 0 0 0) 0x7f9898063850 con 0x7f98a41481a0 2024-09-21T03:44:42.064 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:44:42.064 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":333,"btime":"2024-09-21T03:44:38:966738+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6086,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/965972663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":965972663},{"type":"v1","addr":"172.21.15.195:6836","nonce":965972663}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6700,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/3176122334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":3176122334},{"type":"v1","addr":"172.21.15.195:6837","nonce":3176122334}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":10551,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/791475497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":791475497},{"type":"v1","addr":"172.21.15.38:6833","nonce":791475497}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":330}],"filesystems":[{"mdsmap":{"epoch":333,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:40:02.319282+0000","modified":"2024-09-21T03:44:38.966733+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":136,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-21T03:44:42.064 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 333 2024-09-21T03:44:42.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:42.064+0000 7f98aba4d640 1 -- 172.21.15.38:0/674825902 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9890078bc0 msgr2=0x7f989007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:42.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:42.064+0000 7f98aba4d640 1 --2- 172.21.15.38:0/674825902 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9890078bc0 0x7f989007b0d0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f9894005fd0 tx=0x7f9894009500 comp rx=0 tx=0).stop 2024-09-21T03:44:42.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:42.064+0000 7f98aba4d640 1 -- 172.21.15.38:0/674825902 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f98a41481a0 msgr2=0x7f98a41f9090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:42.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:42.064+0000 7f98aba4d640 1 --2- 172.21.15.38:0/674825902 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f98a41481a0 0x7f98a41f9090 secure :-1 s=READY pgs=2267 cs=0 l=1 rev1=1 crypto rx=0x7f9898006680 tx=0x7f98980066b0 comp rx=0 tx=0).stop 2024-09-21T03:44:42.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:42.064+0000 7f98aba4d640 1 -- 172.21.15.38:0/674825902 shutdown_connections 2024-09-21T03:44:42.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:42.064+0000 7f98aba4d640 1 --2- 172.21.15.38:0/674825902 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9890078bc0 0x7f989007b0d0 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:42.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:42.064+0000 7f98aba4d640 1 --2- 172.21.15.38:0/674825902 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f98a41f95d0 0x7f98a40aa3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:42.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:42.064+0000 7f98aba4d640 1 --2- 172.21.15.38:0/674825902 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f98a41481a0 0x7f98a41f9090 unknown :-1 s=CLOSED pgs=2267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:42.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:42.064+0000 7f98aba4d640 1 --2- 172.21.15.38:0/674825902 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f988c004d40 0x7f98a40cc960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:42.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:42.064+0000 7f98aba4d640 1 -- 172.21.15.38:0/674825902 >> 172.21.15.38:0/674825902 conn(0x7f98a40d0a30 msgr2=0x7f98a406fca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:42.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:42.064+0000 7f98aba4d640 1 -- 172.21.15.38:0/674825902 shutdown_connections 2024-09-21T03:44:42.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:42.064+0000 7f98aba4d640 1 -- 172.21.15.38:0/674825902 wait complete. 2024-09-21T03:44:42.077 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-21T03:44:42.077 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:44:42.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:42.193+0000 7f9148cd5640 1 -- 172.21.15.38:0/1015951062 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9130004d40 msgr2=0x7f9130005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:42.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:42.193+0000 7f9148cd5640 1 --2- 172.21.15.38:0/1015951062 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9130004d40 0x7f9130005150 secure :-1 s=READY pgs=2269 cs=0 l=1 rev1=1 crypto rx=0x7f9134005fd0 tx=0x7f9130005830 comp rx=0 tx=0).stop 2024-09-21T03:44:42.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:42.193+0000 7f9142d9d640 1 -- 172.21.15.38:0/1015951062 reap_dead start 2024-09-21T03:44:42.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:42.193+0000 7f9148cd5640 1 -- 172.21.15.38:0/1015951062 shutdown_connections 2024-09-21T03:44:42.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:42.193+0000 7f9148cd5640 1 -- 172.21.15.38:0/1015951062 >> 172.21.15.38:0/1015951062 conn(0x7f91440b6b20 msgr2=0x7f91440d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:42.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:42.193+0000 7f9148cd5640 1 -- 172.21.15.38:0/1015951062 shutdown_connections 2024-09-21T03:44:42.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:42.193+0000 7f9148cd5640 1 -- 172.21.15.38:0/1015951062 wait complete. 2024-09-21T03:44:42.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:42.195+0000 7f9148cd5640 1 Processor -- start 2024-09-21T03:44:42.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:42.195+0000 7f9148cd5640 1 -- start start 2024-09-21T03:44:42.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:42.196+0000 7f9148cd5640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9130004d40 0x7f9144157a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:42.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:42.196+0000 7f9148cd5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f91440c9d40 0x7f914415aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:42.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:42.196+0000 7f9148cd5640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f91440ca150 0x7f914415afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:42.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:42.196+0000 7f9148cd5640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f91440b35e0 con 0x7f91440c9d40 2024-09-21T03:44:42.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:42.196+0000 7f9148cd5640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f91440b3460 con 0x7f9130004d40 2024-09-21T03:44:42.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:42.196+0000 7f9148cd5640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f91440b3760 con 0x7f91440ca150 2024-09-21T03:44:42.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:42.196+0000 7f9141d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f91440c9d40 0x7f914415aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:42.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:42.196+0000 7f9141d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f91440c9d40 0x7f914415aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50170/0 (socket says 172.21.15.38:50170) 2024-09-21T03:44:42.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:42.196+0000 7f9141d9b640 1 -- 172.21.15.38:0/3680844938 learned_addr learned my addr 172.21.15.38:0/3680844938 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:44:42.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:42.197+0000 7f914259c640 1 --2- 172.21.15.38:0/3680844938 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9130004d40 0x7f9144157a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:42.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:42.197+0000 7f9142d9d640 1 --2- 172.21.15.38:0/3680844938 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f91440ca150 0x7f914415afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:42.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:42.197+0000 7f9141d9b640 1 -- 172.21.15.38:0/3680844938 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f91440ca150 msgr2=0x7f914415afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:42.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:42.197+0000 7f9141d9b640 1 --2- 172.21.15.38:0/3680844938 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f91440ca150 0x7f914415afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:42.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:42.197+0000 7f9141d9b640 1 -- 172.21.15.38:0/3680844938 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9130004d40 msgr2=0x7f9144157a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:42.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:42.197+0000 7f9141d9b640 1 --2- 172.21.15.38:0/3680844938 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9130004d40 0x7f9144157a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:42.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:42.197+0000 7f9141d9b640 1 -- 172.21.15.38:0/3680844938 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91440c9130 con 0x7f91440c9d40 2024-09-21T03:44:42.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:42.197+0000 7f9142d9d640 1 --2- 172.21.15.38:0/3680844938 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f91440ca150 0x7f914415afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:44:42.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:42.197+0000 7f9141d9b640 1 --2- 172.21.15.38:0/3680844938 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f91440c9d40 0x7f914415aa80 secure :-1 s=READY pgs=2270 cs=0 l=1 rev1=1 crypto rx=0x7f912c011ae0 tx=0x7f912c009f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:42.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:42.197+0000 7f913affd640 1 -- 172.21.15.38:0/3680844938 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f912c00fcb0 con 0x7f91440c9d40 2024-09-21T03:44:42.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:42.197+0000 7f913affd640 1 -- 172.21.15.38:0/3680844938 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f912c00fe50 con 0x7f91440c9d40 2024-09-21T03:44:42.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:42.197+0000 7f913affd640 1 -- 172.21.15.38:0/3680844938 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f912c004280 con 0x7f91440c9d40 2024-09-21T03:44:42.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:42.197+0000 7f9148cd5640 1 -- 172.21.15.38:0/3680844938 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9144158b80 con 0x7f91440c9d40 2024-09-21T03:44:42.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:42.197+0000 7f9148cd5640 1 -- 172.21.15.38:0/3680844938 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9144159140 con 0x7f91440c9d40 2024-09-21T03:44:42.202 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:42.198+0000 7f9148cd5640 1 -- 172.21.15.38:0/3680844938 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9144162f00 con 0x7f91440c9d40 2024-09-21T03:44:42.202 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:42.199+0000 7f913affd640 1 -- 172.21.15.38:0/3680844938 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f912c0035f0 con 0x7f91440c9d40 2024-09-21T03:44:42.202 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:42.199+0000 7f913affd640 1 --2- 172.21.15.38:0/3680844938 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9124078b90 0x7f912407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:42.202 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:42.199+0000 7f913affd640 1 -- 172.21.15.38:0/3680844938 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 9744+0+0 (secure 0 0 0) 0x7f912c09c780 con 0x7f91440c9d40 2024-09-21T03:44:42.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:42.200+0000 7f914259c640 1 --2- 172.21.15.38:0/3680844938 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9124078b90 0x7f912407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:42.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:42.200+0000 7f914259c640 1 --2- 172.21.15.38:0/3680844938 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9124078b90 0x7f912407b0a0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f913c001010 tx=0x7f913c001d60 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:42.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:42.204+0000 7f913affd640 1 -- 172.21.15.38:0/3680844938 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f912c0635f0 con 0x7f91440c9d40 2024-09-21T03:44:42.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:42.430+0000 7f9148cd5640 1 -- 172.21.15.38:0/3680844938 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f91440ca5c0 con 0x7f91440c9d40 2024-09-21T03:44:42.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:42.430+0000 7f913affd640 1 -- 172.21.15.38:0/3680844938 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 333 v333) ==== 95+0+4390 (secure 0 0 0) 0x7f912c06a090 con 0x7f91440c9d40 2024-09-21T03:44:42.433 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:44:42.433 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":333,"btime":"2024-09-21T03:44:38:966738+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6086,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/965972663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":965972663},{"type":"v1","addr":"172.21.15.195:6836","nonce":965972663}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6700,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/3176122334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":3176122334},{"type":"v1","addr":"172.21.15.195:6837","nonce":3176122334}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":10551,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/791475497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":791475497},{"type":"v1","addr":"172.21.15.38:6833","nonce":791475497}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":330}],"filesystems":[{"mdsmap":{"epoch":333,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:40:02.319282+0000","modified":"2024-09-21T03:44:38.966733+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":136,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-21T03:44:42.433 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 333 2024-09-21T03:44:42.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:42.432+0000 7f9148cd5640 1 -- 172.21.15.38:0/3680844938 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9124078b90 msgr2=0x7f912407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:42.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:42.433+0000 7f9148cd5640 1 --2- 172.21.15.38:0/3680844938 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9124078b90 0x7f912407b0a0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f913c001010 tx=0x7f913c001d60 comp rx=0 tx=0).stop 2024-09-21T03:44:42.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:42.433+0000 7f9148cd5640 1 -- 172.21.15.38:0/3680844938 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f91440c9d40 msgr2=0x7f914415aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:42.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:42.433+0000 7f9148cd5640 1 --2- 172.21.15.38:0/3680844938 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f91440c9d40 0x7f914415aa80 secure :-1 s=READY pgs=2270 cs=0 l=1 rev1=1 crypto rx=0x7f912c011ae0 tx=0x7f912c009f10 comp rx=0 tx=0).stop 2024-09-21T03:44:42.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:42.433+0000 7f9148cd5640 1 -- 172.21.15.38:0/3680844938 shutdown_connections 2024-09-21T03:44:42.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:42.433+0000 7f9148cd5640 1 --2- 172.21.15.38:0/3680844938 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9124078b90 0x7f912407b0a0 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:42.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:42.433+0000 7f9148cd5640 1 --2- 172.21.15.38:0/3680844938 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f91440ca150 0x7f914415afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:42.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:42.433+0000 7f9148cd5640 1 --2- 172.21.15.38:0/3680844938 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f91440c9d40 0x7f914415aa80 unknown :-1 s=CLOSED pgs=2270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:42.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:42.433+0000 7f9148cd5640 1 --2- 172.21.15.38:0/3680844938 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9130004d40 0x7f9144157a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:42.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:42.433+0000 7f9148cd5640 1 -- 172.21.15.38:0/3680844938 >> 172.21.15.38:0/3680844938 conn(0x7f91440b6b20 msgr2=0x7f914415daa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:42.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:42.433+0000 7f9148cd5640 1 -- 172.21.15.38:0/3680844938 shutdown_connections 2024-09-21T03:44:42.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:42.433+0000 7f9148cd5640 1 -- 172.21.15.38:0/3680844938 wait complete. 2024-09-21T03:44:42.446 INFO:tasks.cephfs.test_damage:Result: Mutation 'Corrupt omap header on 1.00000000' led to DAMAGED state 2024-09-21T03:44:42.447 INFO:tasks.cephfs.test_damage:All 39 mutations had expected outcomes 2024-09-21T03:44:42.447 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-21T03:44:42.447 DEBUG:teuthology.orchestra.run.smithi038:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T03:44:42.486 DEBUG:teuthology.orchestra.run.smithi195:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T03:44:42.529 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-21T03:44:42.529 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T03:44:42.529 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-21T03:44:42.530 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:44:42.546 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056795,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2354 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T03:44:42.546 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T03:44:42.546 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-09-21T03:44:42.546 DEBUG:teuthology.orchestra.run.smithi195:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:44:42.587 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056014,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497444k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=8978 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248720k,nr_inodes=812180,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497444k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T03:44:42.587 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T03:44:42.588 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-21T03:44:42.588 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:44:42.604 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056795,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2354 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T03:44:42.604 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T03:44:42.604 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-09-21T03:44:42.604 DEBUG:teuthology.orchestra.run.smithi195:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:44:42.645 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056014,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497444k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=8978 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248720k,nr_inodes=812180,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497444k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T03:44:42.645 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:44:42.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:42.761+0000 7fd321b7b640 1 -- 172.21.15.38:0/2608409603 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd31c0b2e40 msgr2=0x7fd31c0b3250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:42.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:42.761+0000 7fd321b7b640 1 --2- 172.21.15.38:0/2608409603 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd31c0b2e40 0x7fd31c0b3250 secure :-1 s=READY pgs=2272 cs=0 l=1 rev1=1 crypto rx=0x7fd314005fd0 tx=0x7fd31402a8b0 comp rx=0 tx=0).stop 2024-09-21T03:44:42.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:42.761+0000 7fd321b7b640 1 -- 172.21.15.38:0/2608409603 shutdown_connections 2024-09-21T03:44:42.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:42.761+0000 7fd321b7b640 1 --2- 172.21.15.38:0/2608409603 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd31c0b2e40 0x7fd31c0b3250 unknown :-1 s=CLOSED pgs=2272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:42.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:42.761+0000 7fd321b7b640 1 --2- 172.21.15.38:0/2608409603 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd31c162280 0x7fd31c162690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:42.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:42.761+0000 7fd321b7b640 1 -- 172.21.15.38:0/2608409603 >> 172.21.15.38:0/2608409603 conn(0x7fd31c0b6b20 msgr2=0x7fd31c0d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:42.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:42.761+0000 7fd321b7b640 1 -- 172.21.15.38:0/2608409603 shutdown_connections 2024-09-21T03:44:42.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:42.761+0000 7fd321b7b640 1 -- 172.21.15.38:0/2608409603 wait complete. 2024-09-21T03:44:42.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:42.763+0000 7fd321b7b640 1 Processor -- start 2024-09-21T03:44:42.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:42.763+0000 7fd321b7b640 1 -- start start 2024-09-21T03:44:42.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:42.764+0000 7fd321b7b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd31c0b2e40 0x7fd31c14f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:42.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:42.764+0000 7fd321b7b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd31c162280 0x7fd31c0ac870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:42.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:42.764+0000 7fd321b7b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd31c0aadc0 0x7fd31c0acdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:42.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:42.764+0000 7fd321b7b640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fd31c15e4b0 con 0x7fd31c0aadc0 2024-09-21T03:44:42.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:42.764+0000 7fd321b7b640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd31c0b39a0 con 0x7fd31c0b2e40 2024-09-21T03:44:42.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:42.764+0000 7fd321b7b640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fd31c15e630 con 0x7fd31c162280 2024-09-21T03:44:42.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:42.765+0000 7fd31bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd31c0aadc0 0x7fd31c0acdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:42.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:42.765+0000 7fd31bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd31c0aadc0 0x7fd31c0acdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43950/0 (socket says 172.21.15.38:43950) 2024-09-21T03:44:42.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:42.765+0000 7fd31bfff640 1 -- 172.21.15.38:0/3221896352 learned_addr learned my addr 172.21.15.38:0/3221896352 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:44:42.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:42.765+0000 7fd31affd640 1 --2- 172.21.15.38:0/3221896352 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd31c162280 0x7fd31c0ac870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:42.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:42.765+0000 7fd31b7fe640 1 --2- 172.21.15.38:0/3221896352 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd31c0b2e40 0x7fd31c14f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:42.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:42.765+0000 7fd31bfff640 1 -- 172.21.15.38:0/3221896352 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd31c162280 msgr2=0x7fd31c0ac870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:42.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:42.765+0000 7fd31bfff640 1 --2- 172.21.15.38:0/3221896352 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd31c162280 0x7fd31c0ac870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:42.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:42.765+0000 7fd31bfff640 1 -- 172.21.15.38:0/3221896352 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd31c0b2e40 msgr2=0x7fd31c14f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:42.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:42.765+0000 7fd31bfff640 1 --2- 172.21.15.38:0/3221896352 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd31c0b2e40 0x7fd31c14f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:42.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:42.765+0000 7fd31bfff640 1 -- 172.21.15.38:0/3221896352 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd31c14fdc0 con 0x7fd31c0aadc0 2024-09-21T03:44:42.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:42.765+0000 7fd31b7fe640 1 --2- 172.21.15.38:0/3221896352 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd31c0b2e40 0x7fd31c14f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:44:42.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:42.765+0000 7fd31bfff640 1 --2- 172.21.15.38:0/3221896352 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd31c0aadc0 0x7fd31c0acdb0 secure :-1 s=READY pgs=2273 cs=0 l=1 rev1=1 crypto rx=0x7fd314005fd0 tx=0x7fd314031760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:42.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:42.765+0000 7fd303fff640 1 -- 172.21.15.38:0/3221896352 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd314031990 con 0x7fd31c0aadc0 2024-09-21T03:44:42.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:42.765+0000 7fd303fff640 1 -- 172.21.15.38:0/3221896352 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd314031c20 con 0x7fd31c0aadc0 2024-09-21T03:44:42.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:42.765+0000 7fd321b7b640 1 -- 172.21.15.38:0/3221896352 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd31c0ab530 con 0x7fd31c0aadc0 2024-09-21T03:44:42.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:42.765+0000 7fd303fff640 1 -- 172.21.15.38:0/3221896352 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd314033070 con 0x7fd31c0aadc0 2024-09-21T03:44:42.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:42.765+0000 7fd321b7b640 1 -- 172.21.15.38:0/3221896352 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd31c0abaf0 con 0x7fd31c0aadc0 2024-09-21T03:44:42.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:42.766+0000 7fd321b7b640 1 -- 172.21.15.38:0/3221896352 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd31c0ab280 con 0x7fd31c0aadc0 2024-09-21T03:44:42.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:42.767+0000 7fd303fff640 1 -- 172.21.15.38:0/3221896352 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fd314033210 con 0x7fd31c0aadc0 2024-09-21T03:44:42.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:42.768+0000 7fd303fff640 1 --2- 172.21.15.38:0/3221896352 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd308078b90 0x7fd30807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:42.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:42.768+0000 7fd303fff640 1 -- 172.21.15.38:0/3221896352 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 9744+0+0 (secure 0 0 0) 0x7fd3140bd390 con 0x7fd31c0aadc0 2024-09-21T03:44:42.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:42.768+0000 7fd31b7fe640 1 --2- 172.21.15.38:0/3221896352 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd308078b90 0x7fd30807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:42.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:42.769+0000 7fd31b7fe640 1 --2- 172.21.15.38:0/3221896352 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd308078b90 0x7fd30807b0a0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fd30c000f60 tx=0x7fd30c009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:42.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:42.772+0000 7fd303fff640 1 -- 172.21.15.38:0/3221896352 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fd31404a030 con 0x7fd31c0aadc0 2024-09-21T03:44:42.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:42.997+0000 7fd321b7b640 1 -- 172.21.15.38:0/3221896352 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd31c162f00 con 0x7fd31c0aadc0 2024-09-21T03:44:42.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:42.998+0000 7fd303fff640 1 -- 172.21.15.38:0/3221896352 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 333 v333) ==== 95+0+4390 (secure 0 0 0) 0x7fd31403f030 con 0x7fd31c0aadc0 2024-09-21T03:44:43.000 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:44:43.000 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":333,"btime":"2024-09-21T03:44:38:966738+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6086,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/965972663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":965972663},{"type":"v1","addr":"172.21.15.195:6836","nonce":965972663}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6700,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/3176122334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":3176122334},{"type":"v1","addr":"172.21.15.195:6837","nonce":3176122334}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":10551,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/791475497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":791475497},{"type":"v1","addr":"172.21.15.38:6833","nonce":791475497}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":330}],"filesystems":[{"mdsmap":{"epoch":333,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:40:02.319282+0000","modified":"2024-09-21T03:44:38.966733+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":136,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-21T03:44:43.000 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 333 2024-09-21T03:44:43.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:43.000+0000 7fd321b7b640 1 -- 172.21.15.38:0/3221896352 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd308078b90 msgr2=0x7fd30807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:43.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:43.000+0000 7fd321b7b640 1 --2- 172.21.15.38:0/3221896352 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd308078b90 0x7fd30807b0a0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fd30c000f60 tx=0x7fd30c009040 comp rx=0 tx=0).stop 2024-09-21T03:44:43.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:43.000+0000 7fd321b7b640 1 -- 172.21.15.38:0/3221896352 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd31c0aadc0 msgr2=0x7fd31c0acdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:43.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:43.000+0000 7fd321b7b640 1 --2- 172.21.15.38:0/3221896352 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd31c0aadc0 0x7fd31c0acdb0 secure :-1 s=READY pgs=2273 cs=0 l=1 rev1=1 crypto rx=0x7fd314005fd0 tx=0x7fd314031760 comp rx=0 tx=0).stop 2024-09-21T03:44:43.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:43.000+0000 7fd321b7b640 1 -- 172.21.15.38:0/3221896352 shutdown_connections 2024-09-21T03:44:43.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:43.000+0000 7fd321b7b640 1 --2- 172.21.15.38:0/3221896352 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd31c0aadc0 0x7fd31c0acdb0 unknown :-1 s=CLOSED pgs=2273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:43.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:43.000+0000 7fd321b7b640 1 --2- 172.21.15.38:0/3221896352 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd308078b90 0x7fd30807b0a0 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:43.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:43.000+0000 7fd321b7b640 1 --2- 172.21.15.38:0/3221896352 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd31c162280 0x7fd31c0ac870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:43.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:43.000+0000 7fd321b7b640 1 --2- 172.21.15.38:0/3221896352 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd31c0b2e40 0x7fd31c14f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:43.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:43.000+0000 7fd321b7b640 1 -- 172.21.15.38:0/3221896352 >> 172.21.15.38:0/3221896352 conn(0x7fd31c0b6b20 msgr2=0x7fd31c06d7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:43.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:43.000+0000 7fd321b7b640 1 -- 172.21.15.38:0/3221896352 shutdown_connections 2024-09-21T03:44:43.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:43.000+0000 7fd321b7b640 1 -- 172.21.15.38:0/3221896352 wait complete. 2024-09-21T03:44:43.013 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:44:43.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:43.127+0000 7f8e57150640 1 -- 172.21.15.38:0/3388031331 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8e50162280 msgr2=0x7f8e50162690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:43.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:43.127+0000 7f8e57150640 1 --2- 172.21.15.38:0/3388031331 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8e50162280 0x7f8e50162690 secure :-1 s=READY pgs=2275 cs=0 l=1 rev1=1 crypto rx=0x7f8e500d5040 tx=0x7f8e3802a4a0 comp rx=0 tx=0).stop 2024-09-21T03:44:43.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:43.127+0000 7f8e57150640 1 -- 172.21.15.38:0/3388031331 shutdown_connections 2024-09-21T03:44:43.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:43.127+0000 7f8e57150640 1 --2- 172.21.15.38:0/3388031331 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8e50162280 0x7f8e50162690 unknown :-1 s=CLOSED pgs=2275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:43.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:43.127+0000 7f8e57150640 1 --2- 172.21.15.38:0/3388031331 >> v2:172.21.15.195:3300/0 conn(0x7f8e50162f00 0x7f8e500d24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:43.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:43.127+0000 7f8e57150640 1 -- 172.21.15.38:0/3388031331 >> 172.21.15.38:0/3388031331 conn(0x7f8e500b6b20 msgr2=0x7f8e500d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:43.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:43.127+0000 7f8e57150640 1 -- 172.21.15.38:0/3388031331 shutdown_connections 2024-09-21T03:44:43.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:43.127+0000 7f8e57150640 1 -- 172.21.15.38:0/3388031331 wait complete. 2024-09-21T03:44:43.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:43.129+0000 7f8e57150640 1 Processor -- start 2024-09-21T03:44:43.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:43.129+0000 7f8e57150640 1 -- start start 2024-09-21T03:44:43.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:43.130+0000 7f8e57150640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8e500b2e40 0x7f8e5015a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:43.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:43.130+0000 7f8e57150640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8e50162280 0x7f8e50157890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:43.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:43.131+0000 7f8e57150640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8e500ac300 0x7f8e500ac710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:43.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:43.131+0000 7f8e57150640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f8e5015e630 con 0x7f8e500b2e40 2024-09-21T03:44:43.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:43.131+0000 7f8e57150640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f8e5015e4b0 con 0x7f8e500ac300 2024-09-21T03:44:43.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:43.131+0000 7f8e57150640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f8e5015e7b0 con 0x7f8e50162280 2024-09-21T03:44:43.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:43.131+0000 7f8e54eec640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8e500b2e40 0x7f8e5015a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:43.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:43.131+0000 7f8e54eec640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8e500b2e40 0x7f8e5015a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43970/0 (socket says 172.21.15.38:43970) 2024-09-21T03:44:43.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:43.131+0000 7f8e54eec640 1 -- 172.21.15.38:0/3006173029 learned_addr learned my addr 172.21.15.38:0/3006173029 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:44:43.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:43.131+0000 7f8e4ffff640 1 --2- 172.21.15.38:0/3006173029 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8e50162280 0x7f8e50157890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:43.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:43.131+0000 7f8e556ed640 1 --2- 172.21.15.38:0/3006173029 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8e500ac300 0x7f8e500ac710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:43.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:43.131+0000 7f8e54eec640 1 -- 172.21.15.38:0/3006173029 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8e50162280 msgr2=0x7f8e50157890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:43.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:43.131+0000 7f8e54eec640 1 --2- 172.21.15.38:0/3006173029 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8e50162280 0x7f8e50157890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:43.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:43.131+0000 7f8e54eec640 1 -- 172.21.15.38:0/3006173029 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8e500ac300 msgr2=0x7f8e500ac710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:43.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:43.131+0000 7f8e54eec640 1 --2- 172.21.15.38:0/3006173029 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8e500ac300 0x7f8e500ac710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:43.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:43.131+0000 7f8e54eec640 1 -- 172.21.15.38:0/3006173029 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e500acc50 con 0x7f8e500b2e40 2024-09-21T03:44:43.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:43.131+0000 7f8e54eec640 1 --2- 172.21.15.38:0/3006173029 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8e500b2e40 0x7f8e5015a690 secure :-1 s=READY pgs=2276 cs=0 l=1 rev1=1 crypto rx=0x7f8e4000ea50 tx=0x7f8e4000ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:43.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:43.131+0000 7f8e4d7fa640 1 -- 172.21.15.38:0/3006173029 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e4000cdc0 con 0x7f8e500b2e40 2024-09-21T03:44:43.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:43.131+0000 7f8e4d7fa640 1 -- 172.21.15.38:0/3006173029 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8e40011560 con 0x7f8e500b2e40 2024-09-21T03:44:43.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:43.131+0000 7f8e57150640 1 -- 172.21.15.38:0/3006173029 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e501f5040 con 0x7f8e500b2e40 2024-09-21T03:44:43.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:43.131+0000 7f8e4d7fa640 1 -- 172.21.15.38:0/3006173029 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e40011990 con 0x7f8e500b2e40 2024-09-21T03:44:43.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:43.131+0000 7f8e57150640 1 -- 172.21.15.38:0/3006173029 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e501f55e0 con 0x7f8e500b2e40 2024-09-21T03:44:43.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:43.133+0000 7f8e57150640 1 -- 172.21.15.38:0/3006173029 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e50162f00 con 0x7f8e500b2e40 2024-09-21T03:44:43.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:43.133+0000 7f8e4d7fa640 1 -- 172.21.15.38:0/3006173029 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f8e40011b30 con 0x7f8e500b2e40 2024-09-21T03:44:43.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:43.134+0000 7f8e4d7fa640 1 --2- 172.21.15.38:0/3006173029 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8e30078b90 0x7f8e3007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:43.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:43.134+0000 7f8e4d7fa640 1 -- 172.21.15.38:0/3006173029 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 9744+0+0 (secure 0 0 0) 0x7f8e4009c9a0 con 0x7f8e500b2e40 2024-09-21T03:44:43.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:43.134+0000 7f8e4ffff640 1 --2- 172.21.15.38:0/3006173029 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8e30078b90 0x7f8e3007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:43.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:43.135+0000 7f8e4ffff640 1 --2- 172.21.15.38:0/3006173029 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8e30078b90 0x7f8e3007b0a0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f8e38034000 tx=0x7f8e38004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:43.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:43.138+0000 7f8e4d7fa640 1 -- 172.21.15.38:0/3006173029 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f8e500acc50 con 0x7f8e500b2e40 2024-09-21T03:44:43.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:43.363+0000 7f8e57150640 1 -- 172.21.15.38:0/3006173029 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8e500cfc40 con 0x7f8e500b2e40 2024-09-21T03:44:43.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:43.363+0000 7f8e4d7fa640 1 -- 172.21.15.38:0/3006173029 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 333 v333) ==== 95+0+4390 (secure 0 0 0) 0x7f8e40016030 con 0x7f8e500b2e40 2024-09-21T03:44:43.366 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:44:43.366 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":333,"btime":"2024-09-21T03:44:38:966738+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6086,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/965972663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":965972663},{"type":"v1","addr":"172.21.15.195:6836","nonce":965972663}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6700,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/3176122334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":3176122334},{"type":"v1","addr":"172.21.15.195:6837","nonce":3176122334}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":10551,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/791475497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":791475497},{"type":"v1","addr":"172.21.15.38:6833","nonce":791475497}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":330}],"filesystems":[{"mdsmap":{"epoch":333,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:40:02.319282+0000","modified":"2024-09-21T03:44:38.966733+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":136,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-21T03:44:43.366 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 333 2024-09-21T03:44:43.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:43.365+0000 7f8e57150640 1 -- 172.21.15.38:0/3006173029 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8e30078b90 msgr2=0x7f8e3007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:43.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:43.365+0000 7f8e57150640 1 --2- 172.21.15.38:0/3006173029 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8e30078b90 0x7f8e3007b0a0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f8e38034000 tx=0x7f8e38004730 comp rx=0 tx=0).stop 2024-09-21T03:44:43.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:43.365+0000 7f8e57150640 1 -- 172.21.15.38:0/3006173029 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8e500b2e40 msgr2=0x7f8e5015a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:43.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:43.365+0000 7f8e57150640 1 --2- 172.21.15.38:0/3006173029 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8e500b2e40 0x7f8e5015a690 secure :-1 s=READY pgs=2276 cs=0 l=1 rev1=1 crypto rx=0x7f8e4000ea50 tx=0x7f8e4000ef10 comp rx=0 tx=0).stop 2024-09-21T03:44:43.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:43.366+0000 7f8e57150640 1 -- 172.21.15.38:0/3006173029 shutdown_connections 2024-09-21T03:44:43.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:43.366+0000 7f8e57150640 1 --2- 172.21.15.38:0/3006173029 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8e500ac300 0x7f8e500ac710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:43.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:43.366+0000 7f8e57150640 1 --2- 172.21.15.38:0/3006173029 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8e30078b90 0x7f8e3007b0a0 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:43.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:43.366+0000 7f8e57150640 1 --2- 172.21.15.38:0/3006173029 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8e50162280 0x7f8e50157890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:43.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:43.366+0000 7f8e57150640 1 --2- 172.21.15.38:0/3006173029 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8e500b2e40 0x7f8e5015a690 unknown :-1 s=CLOSED pgs=2276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:43.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:43.366+0000 7f8e57150640 1 -- 172.21.15.38:0/3006173029 >> 172.21.15.38:0/3006173029 conn(0x7f8e500b6b20 msgr2=0x7f8e5015f5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:43.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:43.366+0000 7f8e57150640 1 -- 172.21.15.38:0/3006173029 shutdown_connections 2024-09-21T03:44:43.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:43.366+0000 7f8e57150640 1 -- 172.21.15.38:0/3006173029 wait complete. 2024-09-21T03:44:43.379 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T03:44:43.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:43.494+0000 7f86b42d5640 1 -- 172.21.15.38:0/2286986463 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f86ac0b2e40 msgr2=0x7f86ac0b3250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:43.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:43.494+0000 7f86b42d5640 1 --2- 172.21.15.38:0/2286986463 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f86ac0b2e40 0x7f86ac0b3250 secure :-1 s=READY pgs=2277 cs=0 l=1 rev1=1 crypto rx=0x7f86ac0d5040 tx=0x7f86a402a560 comp rx=0 tx=0).stop 2024-09-21T03:44:43.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:43.494+0000 7f86b42d5640 1 -- 172.21.15.38:0/2286986463 shutdown_connections 2024-09-21T03:44:43.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:43.494+0000 7f86b42d5640 1 --2- 172.21.15.38:0/2286986463 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f86ac0b2e40 0x7f86ac0b3250 unknown :-1 s=CLOSED pgs=2277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:43.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:43.494+0000 7f86b42d5640 1 --2- 172.21.15.38:0/2286986463 >> v2:172.21.15.195:3300/0 conn(0x7f86ac162f00 0x7f86ac0d24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:43.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:43.494+0000 7f86b42d5640 1 -- 172.21.15.38:0/2286986463 >> 172.21.15.38:0/2286986463 conn(0x7f86ac0b6b20 msgr2=0x7f86ac0d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:43.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:43.494+0000 7f86b42d5640 1 -- 172.21.15.38:0/2286986463 shutdown_connections 2024-09-21T03:44:43.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:43.494+0000 7f86b42d5640 1 -- 172.21.15.38:0/2286986463 wait complete. 2024-09-21T03:44:43.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:43.496+0000 7f86b42d5640 1 Processor -- start 2024-09-21T03:44:43.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:43.496+0000 7f86b42d5640 1 -- start start 2024-09-21T03:44:43.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:43.497+0000 7f86b42d5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f86ac0b2e40 0x7f86ac0cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:43.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:43.497+0000 7f86b42d5640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f86ac162280 0x7f86ac14ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:43.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:43.497+0000 7f86b42d5640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f86ac14ca40 0x7f86ac1f5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:43.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:43.497+0000 7f86b42d5640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f86ac15e4b0 con 0x7f86ac0b2e40 2024-09-21T03:44:43.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:43.497+0000 7f86b42d5640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f86ac0b39a0 con 0x7f86ac14ca40 2024-09-21T03:44:43.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:43.497+0000 7f86b42d5640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f86ac15e630 con 0x7f86ac162280 2024-09-21T03:44:43.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:43.497+0000 7f86b2071640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f86ac0b2e40 0x7f86ac0cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:43.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:43.497+0000 7f86b2071640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f86ac0b2e40 0x7f86ac0cebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44002/0 (socket says 172.21.15.38:44002) 2024-09-21T03:44:43.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:43.497+0000 7f86b2071640 1 -- 172.21.15.38:0/1545294436 learned_addr learned my addr 172.21.15.38:0/1545294436 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:44:43.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:43.497+0000 7f86b2872640 1 --2- 172.21.15.38:0/1545294436 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f86ac14ca40 0x7f86ac1f5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:43.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:43.497+0000 7f86b1870640 1 --2- 172.21.15.38:0/1545294436 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f86ac162280 0x7f86ac14ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:43.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:43.498+0000 7f86b2071640 1 -- 172.21.15.38:0/1545294436 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f86ac162280 msgr2=0x7f86ac14ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:43.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:43.498+0000 7f86b2071640 1 --2- 172.21.15.38:0/1545294436 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f86ac162280 0x7f86ac14ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:43.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:43.498+0000 7f86b2071640 1 -- 172.21.15.38:0/1545294436 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f86ac14ca40 msgr2=0x7f86ac1f5080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:43.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:43.498+0000 7f86b2071640 1 --2- 172.21.15.38:0/1545294436 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f86ac14ca40 0x7f86ac1f5080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:43.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:43.498+0000 7f86b2071640 1 -- 172.21.15.38:0/1545294436 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86ac0abf40 con 0x7f86ac0b2e40 2024-09-21T03:44:43.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:43.498+0000 7f86b2071640 1 --2- 172.21.15.38:0/1545294436 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f86ac0b2e40 0x7f86ac0cebb0 secure :-1 s=READY pgs=2278 cs=0 l=1 rev1=1 crypto rx=0x7f86a800eb30 tx=0x7f86a800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:43.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:43.498+0000 7f86a2ffd640 1 -- 172.21.15.38:0/1545294436 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86a8010430 con 0x7f86ac0b2e40 2024-09-21T03:44:43.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:43.498+0000 7f86a2ffd640 1 -- 172.21.15.38:0/1545294436 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f86a80105d0 con 0x7f86ac0b2e40 2024-09-21T03:44:43.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:43.498+0000 7f86a2ffd640 1 -- 172.21.15.38:0/1545294436 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86a8010a20 con 0x7f86ac0b2e40 2024-09-21T03:44:43.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:43.498+0000 7f86b42d5640 1 -- 172.21.15.38:0/1545294436 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86ac0ac220 con 0x7f86ac0b2e40 2024-09-21T03:44:43.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:43.498+0000 7f86b42d5640 1 -- 172.21.15.38:0/1545294436 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86ac0ac7e0 con 0x7f86ac0b2e40 2024-09-21T03:44:43.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:43.499+0000 7f86b42d5640 1 -- 172.21.15.38:0/1545294436 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86ac162f00 con 0x7f86ac0b2e40 2024-09-21T03:44:43.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:43.500+0000 7f86a2ffd640 1 -- 172.21.15.38:0/1545294436 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f86a8010bc0 con 0x7f86ac0b2e40 2024-09-21T03:44:43.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:43.500+0000 7f86a2ffd640 1 --2- 172.21.15.38:0/1545294436 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8690078b90 0x7f869007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:43.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:43.500+0000 7f86a2ffd640 1 -- 172.21.15.38:0/1545294436 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 9744+0+0 (secure 0 0 0) 0x7f86a809ca60 con 0x7f86ac0b2e40 2024-09-21T03:44:43.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:43.501+0000 7f86b1870640 1 --2- 172.21.15.38:0/1545294436 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8690078b90 0x7f869007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:43.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:43.501+0000 7f86b1870640 1 --2- 172.21.15.38:0/1545294436 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8690078b90 0x7f869007b0a0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f86ac0ba320 tx=0x7f869c009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:43.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:43.505+0000 7f86a2ffd640 1 -- 172.21.15.38:0/1545294436 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f86a8063890 con 0x7f86ac0b2e40 2024-09-21T03:44:43.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:43.693+0000 7f86b42d5640 1 -- 172.21.15.38:0/1545294436 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f86ac0b7950 con 0x7f86ac0b2e40 2024-09-21T03:44:43.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:43.695+0000 7f86a2ffd640 1 -- 172.21.15.38:0/1545294436 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v136) ==== 74+0+22476 (secure 0 0 0) 0x7f86a80682c0 con 0x7f86ac0b2e40 2024-09-21T03:44:43.697 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:44:43.697 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":136,"fsid":"9d50d66f-656b-4fe2-8e3b-fe53b79c4d43","created":"2024-09-21T03:33:10.677865+0000","modified":"2024-09-21T03:44:38.966438+0000","last_up_change":"2024-09-21T03:33:24.023421+0000","last_in_change":"2024-09-21T03:33:14.296193+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T03:33:22.992573+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-09-21T03:39:59.945436+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-09-21T03:40:01.239789+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dbab183f-7e63-48b8-960a-ae55fd5bf565","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6801","nonce":2381639433}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6803","nonce":2381639433}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6807","nonce":2381639433}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6805","nonce":2381639433}]},"public_addr":"172.21.15.38:6801/2381639433","cluster_addr":"172.21.15.38:6803/2381639433","heartbeat_back_addr":"172.21.15.38:6807/2381639433","heartbeat_front_addr":"172.21.15.38:6805/2381639433","state":["exists","up"]},{"osd":1,"uuid":"fa1c0e8a-ebc1-4359-8146-bd5342abbc16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6817","nonce":1491818732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6819","nonce":1491818732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6823","nonce":1491818732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6821","nonce":1491818732}]},"public_addr":"172.21.15.38:6817/1491818732","cluster_addr":"172.21.15.38:6819/1491818732","heartbeat_back_addr":"172.21.15.38:6823/1491818732","heartbeat_front_addr":"172.21.15.38:6821/1491818732","state":["exists","up"]},{"osd":2,"uuid":"2ed3cd2d-385d-4fc6-9869-214552a84803","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6825","nonce":2577858902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6827","nonce":2577858902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6831","nonce":2577858902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6829","nonce":2577858902}]},"public_addr":"172.21.15.38:6825/2577858902","cluster_addr":"172.21.15.38:6827/2577858902","heartbeat_back_addr":"172.21.15.38:6831/2577858902","heartbeat_front_addr":"172.21.15.38:6829/2577858902","state":["exists","up"]},{"osd":3,"uuid":"62d1d95e-919d-4250-be71-1d34cfa70dbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6809","nonce":1286810294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6811","nonce":1286810294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6815","nonce":1286810294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6813","nonce":1286810294}]},"public_addr":"172.21.15.38:6809/1286810294","cluster_addr":"172.21.15.38:6811/1286810294","heartbeat_back_addr":"172.21.15.38:6815/1286810294","heartbeat_front_addr":"172.21.15.38:6813/1286810294","state":["exists","up"]},{"osd":4,"uuid":"7c9f9314-99d7-4d73-8392-37809a9f54b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6803","nonce":3401837571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6805","nonce":3401837571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6809","nonce":3401837571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6807","nonce":3401837571}]},"public_addr":"172.21.15.195:6803/3401837571","cluster_addr":"172.21.15.195:6805/3401837571","heartbeat_back_addr":"172.21.15.195:6809/3401837571","heartbeat_front_addr":"172.21.15.195:6807/3401837571","state":["exists","up"]},{"osd":5,"uuid":"bde4fe9f-8414-4bba-bf4a-43973d1976bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6811","nonce":2080960180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6813","nonce":2080960180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6817","nonce":2080960180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6815","nonce":2080960180}]},"public_addr":"172.21.15.195:6811/2080960180","cluster_addr":"172.21.15.195:6813/2080960180","heartbeat_back_addr":"172.21.15.195:6817/2080960180","heartbeat_front_addr":"172.21.15.195:6815/2080960180","state":["exists","up"]},{"osd":6,"uuid":"6ee6d7f4-1a5f-4a4c-ac16-48fe05f6a002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6827","nonce":4055825675}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6829","nonce":4055825675}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6833","nonce":4055825675}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6831","nonce":4055825675}]},"public_addr":"172.21.15.195:6827/4055825675","cluster_addr":"172.21.15.195:6829/4055825675","heartbeat_back_addr":"172.21.15.195:6833/4055825675","heartbeat_front_addr":"172.21.15.195:6831/4055825675","state":["exists","up"]},{"osd":7,"uuid":"efeb5f51-329a-4484-8290-92e7d3503d05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6819","nonce":1606106342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6821","nonce":1606106342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6825","nonce":1606106342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6823","nonce":1606106342}]},"public_addr":"172.21.15.195:6819/1606106342","cluster_addr":"172.21.15.195:6821/1606106342","heartbeat_back_addr":"172.21.15.195:6825/1606106342","heartbeat_front_addr":"172.21.15.195:6823/1606106342","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.901343+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:21.115003+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.901623+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.835238+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.132199+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:19.817560+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.829690+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:21.087300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.a","mappings":[{"from":6,"to":5}]},{"pgid":"14.1f","mappings":[{"from":7,"to":0}]},{"pgid":"14.25","mappings":[{"from":7,"to":3}]},{"pgid":"14.26","mappings":[{"from":1,"to":0}]},{"pgid":"14.2d","mappings":[{"from":7,"to":0}]},{"pgid":"14.33","mappings":[{"from":1,"to":3}]},{"pgid":"14.34","mappings":[{"from":7,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.38:6832/999357334":"2024-09-22T03:44:31.921782+0000","172.21.15.38:6832/2851755473":"2024-09-22T03:44:25.830423+0000","172.21.15.38:6833/2178429646":"2024-09-22T03:44:20.787490+0000","172.21.15.38:6832/2178429646":"2024-09-22T03:44:20.787490+0000","172.21.15.38:6833/3103607164":"2024-09-22T03:44:05.867078+0000","172.21.15.38:6832/3103607164":"2024-09-22T03:44:05.867078+0000","172.21.15.38:6832/2444649344":"2024-09-22T03:44:00.664196+0000","172.21.15.38:6833/999357334":"2024-09-22T03:44:31.921782+0000","172.21.15.38:6833/626794495":"2024-09-22T03:43:49.180309+0000","172.21.15.38:6832/1643781179":"2024-09-22T03:43:33.596532+0000","172.21.15.38:6833/2309772457":"2024-09-22T03:43:28.198223+0000","172.21.15.38:6832/2309772457":"2024-09-22T03:43:28.198223+0000","172.21.15.38:6835/141447142":"2024-09-22T03:43:14.684929+0000","172.21.15.38:6834/141447142":"2024-09-22T03:43:14.684929+0000","172.21.15.38:6835/3572946698":"2024-09-22T03:41:44.737273+0000","172.21.15.38:6833/3553953266":"2024-09-22T03:41:29.639383+0000","172.21.15.38:6834/2751962827":"2024-09-22T03:40:38.029580+0000","172.21.15.38:6832/3553953266":"2024-09-22T03:41:29.639383+0000","172.21.15.38:6834/2930805359":"2024-09-22T03:42:23.215190+0000","172.21.15.38:6832/626794495":"2024-09-22T03:43:49.180309+0000","172.21.15.38:6833/1643781179":"2024-09-22T03:43:33.596532+0000","172.21.15.38:6835/2748959970":"2024-09-22T03:41:16.565736+0000","172.21.15.38:6833/3710736169":"2024-09-22T03:40:59.454480+0000","172.21.15.38:6833/1995903506":"2024-09-22T03:43:00.406120+0000","172.21.15.38:6833/775682959":"2024-09-22T03:43:10.570678+0000","172.21.15.38:6834/2748959970":"2024-09-22T03:41:16.565736+0000","172.21.15.38:6834/2934569248":"2024-09-22T03:40:26.934591+0000","172.21.15.38:6832/3726830644":"2024-09-22T03:41:12.532341+0000","172.21.15.38:6835/2097789246":"2024-09-22T03:43:06.604471+0000","172.21.15.38:6835/2934569248":"2024-09-22T03:40:26.934591+0000","172.21.15.38:6835/3864487171":"2024-09-22T03:40:31.989837+0000","172.21.15.38:6833/4221150556":"2024-09-22T03:42:27.240627+0000","172.21.15.38:6832/121492560":"2024-09-22T03:41:10.371933+0000","172.21.15.38:6832/1717955365":"2024-09-22T03:41:51.787804+0000","172.21.15.38:6834/2418225210":"2024-09-22T03:42:02.869612+0000","172.21.15.38:6833/2851755473":"2024-09-22T03:44:25.830423+0000","172.21.15.38:6835/2418225210":"2024-09-22T03:42:02.869612+0000","172.21.15.38:6833/121492560":"2024-09-22T03:41:10.371933+0000","172.21.15.38:6835/3951934534":"2024-09-22T03:42:31.276099+0000","172.21.15.38:6833/751618334":"2024-09-22T03:40:42.053279+0000","172.21.15.195:6837/712370966":"2024-09-22T03:40:11.633787+0000","172.21.15.38:6832/3402963947":"2024-09-22T03:41:23.555005+0000","172.21.15.38:6835/2798021708":"2024-09-22T03:42:47.436632+0000","172.21.15.38:6835/3801772046":"2024-09-22T03:44:38.966412+0000","172.21.15.38:6833/3726830644":"2024-09-22T03:41:12.532341+0000","172.21.15.38:6834/2916238332":"2024-09-22T03:41:42.553770+0000","172.21.15.38:6834/3055025811":"2024-09-22T03:42:39.361236+0000","172.21.15.195:6835/712370966":"2024-09-22T03:40:11.633787+0000","172.21.15.38:6834/3864487171":"2024-09-22T03:40:31.989837+0000","172.21.15.38:6834/3572946698":"2024-09-22T03:41:44.737273+0000","172.21.15.38:6832/1995903506":"2024-09-22T03:43:00.406120+0000","172.21.15.38:6832/775682959":"2024-09-22T03:43:10.570678+0000","172.21.15.38:6835/2916238332":"2024-09-22T03:41:42.553770+0000","172.21.15.38:6833/3402963947":"2024-09-22T03:41:23.555005+0000","172.21.15.38:6834/425672974":"2024-09-22T03:40:46.105521+0000","172.21.15.38:6834/3951934534":"2024-09-22T03:42:31.276099+0000","172.21.15.38:6835/425672974":"2024-09-22T03:40:46.105521+0000","172.21.15.38:6832/3710736169":"2024-09-22T03:40:59.454480+0000","172.21.15.38:6833/1717955365":"2024-09-22T03:41:51.787804+0000","172.21.15.38:6835/3055025811":"2024-09-22T03:42:39.361236+0000","172.21.15.38:6835/2751962827":"2024-09-22T03:40:38.029580+0000","172.21.15.38:6835/2712360296":"2024-09-22T03:42:20.931115+0000","172.21.15.38:6833/3537258034":"2024-09-22T03:42:06.901189+0000","172.21.15.38:6834/3801772046":"2024-09-22T03:44:38.966412+0000","172.21.15.38:6832/3537258034":"2024-09-22T03:42:06.901189+0000","172.21.15.38:6832/1272005650":"2024-09-22T03:41:58.836862+0000","172.21.15.38:6834/2712360296":"2024-09-22T03:42:20.931115+0000","172.21.15.38:6832/751618334":"2024-09-22T03:40:42.053279+0000","172.21.15.38:6835/2930805359":"2024-09-22T03:42:23.215190+0000","172.21.15.38:6832/4221150556":"2024-09-22T03:42:27.240627+0000","172.21.15.38:6833/1272005650":"2024-09-22T03:41:58.836862+0000","172.21.15.38:6832/245538755":"2024-09-22T03:42:35.318756+0000","172.21.15.38:6833/245538755":"2024-09-22T03:42:35.318756+0000","172.21.15.38:6833/2444649344":"2024-09-22T03:44:00.664196+0000","172.21.15.38:6833/2459386631":"2024-09-22T03:42:43.395609+0000","172.21.15.38:6832/2459386631":"2024-09-22T03:42:43.395609+0000","172.21.15.38:6834/2097789246":"2024-09-22T03:43:06.604471+0000","172.21.15.38:6834/2798021708":"2024-09-22T03:42:47.436632+0000","172.21.15.38:6832/111206731":"2024-09-22T03:43:02.560071+0000","172.21.15.38:6833/111206731":"2024-09-22T03:43:02.560071+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T03:44:43.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:43.697+0000 7f86b42d5640 1 -- 172.21.15.38:0/1545294436 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8690078b90 msgr2=0x7f869007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:43.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:43.697+0000 7f86b42d5640 1 --2- 172.21.15.38:0/1545294436 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8690078b90 0x7f869007b0a0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f86ac0ba320 tx=0x7f869c009040 comp rx=0 tx=0).stop 2024-09-21T03:44:43.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:43.697+0000 7f86b42d5640 1 -- 172.21.15.38:0/1545294436 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f86ac0b2e40 msgr2=0x7f86ac0cebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:43.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:43.697+0000 7f86b42d5640 1 --2- 172.21.15.38:0/1545294436 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f86ac0b2e40 0x7f86ac0cebb0 secure :-1 s=READY pgs=2278 cs=0 l=1 rev1=1 crypto rx=0x7f86a800eb30 tx=0x7f86a800c2d0 comp rx=0 tx=0).stop 2024-09-21T03:44:43.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:43.697+0000 7f86b42d5640 1 -- 172.21.15.38:0/1545294436 shutdown_connections 2024-09-21T03:44:43.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:43.697+0000 7f86b42d5640 1 --2- 172.21.15.38:0/1545294436 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f86ac14ca40 0x7f86ac1f5080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:43.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:43.697+0000 7f86b42d5640 1 --2- 172.21.15.38:0/1545294436 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8690078b90 0x7f869007b0a0 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:43.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:43.697+0000 7f86b42d5640 1 --2- 172.21.15.38:0/1545294436 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f86ac162280 0x7f86ac14ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:43.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:43.697+0000 7f86b42d5640 1 --2- 172.21.15.38:0/1545294436 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f86ac0b2e40 0x7f86ac0cebb0 unknown :-1 s=CLOSED pgs=2278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:43.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:43.697+0000 7f86b42d5640 1 -- 172.21.15.38:0/1545294436 >> 172.21.15.38:0/1545294436 conn(0x7f86ac0b6b20 msgr2=0x7f86ac06ddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:43.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:43.697+0000 7f86b42d5640 1 -- 172.21.15.38:0/1545294436 shutdown_connections 2024-09-21T03:44:43.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:43.697+0000 7f86b42d5640 1 -- 172.21.15.38:0/1545294436 wait complete. 2024-09-21T03:44:43.711 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-21T03:44:43.711 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:44:43.719 INFO:tasks.ceph.mds.a.smithi038.stderr: -56> 2024-09-21T03:44:38.964+0000 7f53dcf61640 -1 mds.0.cache.dir(0x1) Corrupt fnode in dirfrag 0x1: Decoder at 'void fnode_t::decode(ceph::buffer::v15_2_0::list::const_iterator&)' v=3 cannot decode v=100 minimal_decoder=101: Malformed input [buffer:3] 2024-09-21T03:44:43.719 INFO:tasks.ceph.mds.a.smithi038.stderr: -53> 2024-09-21T03:44:38.964+0000 7f53dcf61640 -1 mds.0.damage notify_dirfrag Damage to fragment * of ino 0x1 is fatal because it is a system directory for this rank 2024-09-21T03:44:43.719 INFO:tasks.ceph.mds.a.smithi038.stderr: -33> 2024-09-21T03:44:39.732+0000 7f53e0f69640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T03:44:43.774 INFO:tasks.ceph.mds.a.smithi038.stdout:starting mds.a at 2024-09-21T03:44:43.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:43.828+0000 7f5c627c6640 1 -- 172.21.15.38:0/4269598624 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5c5c162280 msgr2=0x7f5c5c162690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:43.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:43.828+0000 7f5c627c6640 1 --2- 172.21.15.38:0/4269598624 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5c5c162280 0x7f5c5c162690 secure :-1 s=READY pgs=2283 cs=0 l=1 rev1=1 crypto rx=0x7f5c5c0d0c00 tx=0x7f5c5002a4a0 comp rx=0 tx=0).stop 2024-09-21T03:44:43.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:43.828+0000 7f5c627c6640 1 -- 172.21.15.38:0/4269598624 shutdown_connections 2024-09-21T03:44:43.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:43.828+0000 7f5c627c6640 1 --2- 172.21.15.38:0/4269598624 >> v2:172.21.15.195:3300/0 conn(0x7f5c5c0b2e40 0x7f5c5c0b3250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:43.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:43.828+0000 7f5c627c6640 1 --2- 172.21.15.38:0/4269598624 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5c5c162280 0x7f5c5c162690 unknown :-1 s=CLOSED pgs=2283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:43.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:43.828+0000 7f5c627c6640 1 -- 172.21.15.38:0/4269598624 >> 172.21.15.38:0/4269598624 conn(0x7f5c5c0b6b20 msgr2=0x7f5c5c0d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:43.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:43.828+0000 7f5c627c6640 1 -- 172.21.15.38:0/4269598624 shutdown_connections 2024-09-21T03:44:43.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:43.828+0000 7f5c627c6640 1 -- 172.21.15.38:0/4269598624 wait complete. 2024-09-21T03:44:43.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:43.830+0000 7f5c627c6640 1 Processor -- start 2024-09-21T03:44:43.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:43.830+0000 7f5c627c6640 1 -- start start 2024-09-21T03:44:43.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:43.831+0000 7f5c627c6640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5c5c0b2e40 0x7f5c5c0ce940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:43.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:43.831+0000 7f5c627c6640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5c5c162280 0x7f5c5c0aca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:43.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:43.832+0000 7f5c627c6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5c5c0acfd0 0x7f5c5c1f86e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:43.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:43.832+0000 7f5c627c6640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f5c5c15e4b0 con 0x7f5c5c0acfd0 2024-09-21T03:44:43.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:43.832+0000 7f5c627c6640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f5c5c0b39a0 con 0x7f5c5c162280 2024-09-21T03:44:43.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:43.832+0000 7f5c627c6640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f5c5c15e630 con 0x7f5c5c0b2e40 2024-09-21T03:44:43.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:43.832+0000 7f5c5b7fe640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5c5c162280 0x7f5c5c0aca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:43.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:43.832+0000 7f5c60d63640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5c5c0acfd0 0x7f5c5c1f86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:43.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:43.832+0000 7f5c5bfff640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5c5c0b2e40 0x7f5c5c0ce940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:43.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:43.832+0000 7f5c60d63640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5c5c0acfd0 0x7f5c5c1f86e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44028/0 (socket says 172.21.15.38:44028) 2024-09-21T03:44:43.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:43.832+0000 7f5c60d63640 1 -- 172.21.15.38:0/4050471884 learned_addr learned my addr 172.21.15.38:0/4050471884 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:44:43.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:43.832+0000 7f5c60d63640 1 -- 172.21.15.38:0/4050471884 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5c5c0b2e40 msgr2=0x7f5c5c0ce940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:43.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:43.832+0000 7f5c60d63640 1 --2- 172.21.15.38:0/4050471884 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5c5c0b2e40 0x7f5c5c0ce940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:43.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:43.832+0000 7f5c60d63640 1 -- 172.21.15.38:0/4050471884 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5c5c162280 msgr2=0x7f5c5c0aca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:43.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:43.832+0000 7f5c60d63640 1 --2- 172.21.15.38:0/4050471884 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5c5c162280 0x7f5c5c0aca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:43.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:43.832+0000 7f5c60d63640 1 -- 172.21.15.38:0/4050471884 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c5c0ad7f0 con 0x7f5c5c0acfd0 2024-09-21T03:44:43.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:43.832+0000 7f5c60d63640 1 --2- 172.21.15.38:0/4050471884 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5c5c0acfd0 0x7f5c5c1f86e0 secure :-1 s=READY pgs=2284 cs=0 l=1 rev1=1 crypto rx=0x7f5c5400d9d0 tx=0x7f5c5400de90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:43.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:43.833+0000 7f5c58ff9640 1 -- 172.21.15.38:0/4050471884 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c54017070 con 0x7f5c5c0acfd0 2024-09-21T03:44:43.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:43.833+0000 7f5c58ff9640 1 -- 172.21.15.38:0/4050471884 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5c54003970 con 0x7f5c5c0acfd0 2024-09-21T03:44:43.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:43.833+0000 7f5c627c6640 1 -- 172.21.15.38:0/4050471884 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c5c14cbe0 con 0x7f5c5c0acfd0 2024-09-21T03:44:43.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:43.833+0000 7f5c58ff9640 1 -- 172.21.15.38:0/4050471884 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c5400be60 con 0x7f5c5c0acfd0 2024-09-21T03:44:43.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:43.833+0000 7f5c627c6640 1 -- 172.21.15.38:0/4050471884 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c5c14d1a0 con 0x7f5c5c0acfd0 2024-09-21T03:44:43.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:43.834+0000 7f5c627c6640 1 -- 172.21.15.38:0/4050471884 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c5c0ad560 con 0x7f5c5c0acfd0 2024-09-21T03:44:43.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:43.834+0000 7f5c58ff9640 1 -- 172.21.15.38:0/4050471884 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f5c54010460 con 0x7f5c5c0acfd0 2024-09-21T03:44:43.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:43.835+0000 7f5c58ff9640 1 --2- 172.21.15.38:0/4050471884 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5c48078b90 0x7f5c4807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:43.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:43.835+0000 7f5c58ff9640 1 -- 172.21.15.38:0/4050471884 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 9744+0+0 (secure 0 0 0) 0x7f5c5409c0c0 con 0x7f5c5c0acfd0 2024-09-21T03:44:43.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:43.835+0000 7f5c5bfff640 1 --2- 172.21.15.38:0/4050471884 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5c48078b90 0x7f5c4807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:43.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:43.836+0000 7f5c5bfff640 1 --2- 172.21.15.38:0/4050471884 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5c48078b90 0x7f5c4807b0a0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f5c4c000f60 tx=0x7f5c4c009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:43.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:43.840+0000 7f5c58ff9640 1 -- 172.21.15.38:0/4050471884 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f5c54062ea0 con 0x7f5c5c0acfd0 2024-09-21T03:44:44.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:44.066+0000 7f5c627c6640 1 -- 172.21.15.38:0/4050471884 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5c5c162f00 con 0x7f5c5c0acfd0 2024-09-21T03:44:44.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:44.066+0000 7f5c58ff9640 1 -- 172.21.15.38:0/4050471884 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 333 v333) ==== 95+0+4390 (secure 0 0 0) 0x7f5c540678d0 con 0x7f5c5c0acfd0 2024-09-21T03:44:44.068 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:44:44.069 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":333,"btime":"2024-09-21T03:44:38:966738+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6086,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/965972663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":965972663},{"type":"v1","addr":"172.21.15.195:6836","nonce":965972663}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6700,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/3176122334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":3176122334},{"type":"v1","addr":"172.21.15.195:6837","nonce":3176122334}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":10551,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/791475497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":791475497},{"type":"v1","addr":"172.21.15.38:6833","nonce":791475497}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":330}],"filesystems":[{"mdsmap":{"epoch":333,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:40:02.319282+0000","modified":"2024-09-21T03:44:38.966733+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":136,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-21T03:44:44.069 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 333 2024-09-21T03:44:44.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:44.068+0000 7f5c627c6640 1 -- 172.21.15.38:0/4050471884 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5c48078b90 msgr2=0x7f5c4807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:44.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:44.069+0000 7f5c627c6640 1 --2- 172.21.15.38:0/4050471884 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5c48078b90 0x7f5c4807b0a0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f5c4c000f60 tx=0x7f5c4c009040 comp rx=0 tx=0).stop 2024-09-21T03:44:44.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:44.069+0000 7f5c627c6640 1 -- 172.21.15.38:0/4050471884 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5c5c0acfd0 msgr2=0x7f5c5c1f86e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:44.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:44.069+0000 7f5c627c6640 1 --2- 172.21.15.38:0/4050471884 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5c5c0acfd0 0x7f5c5c1f86e0 secure :-1 s=READY pgs=2284 cs=0 l=1 rev1=1 crypto rx=0x7f5c5400d9d0 tx=0x7f5c5400de90 comp rx=0 tx=0).stop 2024-09-21T03:44:44.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:44.069+0000 7f5c627c6640 1 -- 172.21.15.38:0/4050471884 shutdown_connections 2024-09-21T03:44:44.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:44.069+0000 7f5c627c6640 1 --2- 172.21.15.38:0/4050471884 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5c5c0acfd0 0x7f5c5c1f86e0 unknown :-1 s=CLOSED pgs=2284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:44.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:44.069+0000 7f5c627c6640 1 --2- 172.21.15.38:0/4050471884 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5c5c162280 0x7f5c5c0aca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:44.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:44.069+0000 7f5c627c6640 1 --2- 172.21.15.38:0/4050471884 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5c48078b90 0x7f5c4807b0a0 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:44.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:44.069+0000 7f5c627c6640 1 --2- 172.21.15.38:0/4050471884 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5c5c0b2e40 0x7f5c5c0ce940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:44.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:44.069+0000 7f5c627c6640 1 -- 172.21.15.38:0/4050471884 >> 172.21.15.38:0/4050471884 conn(0x7f5c5c0b6b20 msgr2=0x7f5c5c068de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:44.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:44.069+0000 7f5c627c6640 1 -- 172.21.15.38:0/4050471884 shutdown_connections 2024-09-21T03:44:44.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:44.069+0000 7f5c627c6640 1 -- 172.21.15.38:0/4050471884 wait complete. 2024-09-21T03:44:44.082 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:44:44.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:44.238+0000 7f7461a2e640 1 -- 172.21.15.38:0/3471650023 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f745c15bfa0 msgr2=0x7f745c15c3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:44.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:44.238+0000 7f7461a2e640 1 --2- 172.21.15.38:0/3471650023 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f745c15bfa0 0x7f745c15c3b0 secure :-1 s=READY pgs=2286 cs=0 l=1 rev1=1 crypto rx=0x7f7454005fd0 tx=0x7f745402a8b0 comp rx=0 tx=0).stop 2024-09-21T03:44:44.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:44.238+0000 7f7461a2e640 1 -- 172.21.15.38:0/3471650023 shutdown_connections 2024-09-21T03:44:44.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:44.238+0000 7f7461a2e640 1 --2- 172.21.15.38:0/3471650023 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f745c15bfa0 0x7f745c15c3b0 unknown :-1 s=CLOSED pgs=2286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:44.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:44.238+0000 7f7461a2e640 1 --2- 172.21.15.38:0/3471650023 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f745c074210 0x7f745c158f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:44.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:44.238+0000 7f7461a2e640 1 -- 172.21.15.38:0/3471650023 >> 172.21.15.38:0/3471650023 conn(0x7f745c073560 msgr2=0x7f745c073970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:44.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:44.238+0000 7f7461a2e640 1 -- 172.21.15.38:0/3471650023 shutdown_connections 2024-09-21T03:44:44.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:44.238+0000 7f7461a2e640 1 -- 172.21.15.38:0/3471650023 wait complete. 2024-09-21T03:44:44.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:44.240+0000 7f7461a2e640 1 Processor -- start 2024-09-21T03:44:44.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:44.240+0000 7f7461a2e640 1 -- start start 2024-09-21T03:44:44.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:44.241+0000 7f7461a2e640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f745c074210 0x7f745c071960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:44.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:44.241+0000 7f7461a2e640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f745c15bfa0 0x7f745c0b5de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:44.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:44.241+0000 7f7461a2e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f745c15f970 0x7f745c1f6c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:44.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:44.241+0000 7f7461a2e640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f745c06df70 con 0x7f745c15f970 2024-09-21T03:44:44.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:44.241+0000 7f7461a2e640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f745c06ddf0 con 0x7f745c15bfa0 2024-09-21T03:44:44.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:44.241+0000 7f7461a2e640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f745c06e0f0 con 0x7f745c074210 2024-09-21T03:44:44.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:44.241+0000 7f745b7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f745c15f970 0x7f745c1f6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:44.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:44.241+0000 7f745b7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f745c15f970 0x7f745c1f6c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44044/0 (socket says 172.21.15.38:44044) 2024-09-21T03:44:44.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:44.241+0000 7f745b7fe640 1 -- 172.21.15.38:0/4054604310 learned_addr learned my addr 172.21.15.38:0/4054604310 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:44:44.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:44.242+0000 7f745affd640 1 --2- 172.21.15.38:0/4054604310 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f745c074210 0x7f745c071960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:44.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:44.242+0000 7f745a7fc640 1 --2- 172.21.15.38:0/4054604310 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f745c15bfa0 0x7f745c0b5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:44.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:44.242+0000 7f745b7fe640 1 -- 172.21.15.38:0/4054604310 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f745c074210 msgr2=0x7f745c071960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:44.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:44.242+0000 7f745b7fe640 1 --2- 172.21.15.38:0/4054604310 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f745c074210 0x7f745c071960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:44.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:44.242+0000 7f745b7fe640 1 -- 172.21.15.38:0/4054604310 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f745c15bfa0 msgr2=0x7f745c0b5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:44.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:44.242+0000 7f745b7fe640 1 --2- 172.21.15.38:0/4054604310 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f745c15bfa0 0x7f745c0b5de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:44.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:44.242+0000 7f745b7fe640 1 -- 172.21.15.38:0/4054604310 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f745c0c9d00 con 0x7f745c15f970 2024-09-21T03:44:44.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:44.242+0000 7f745a7fc640 1 --2- 172.21.15.38:0/4054604310 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f745c15bfa0 0x7f745c0b5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:44:44.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:44.242+0000 7f745b7fe640 1 --2- 172.21.15.38:0/4054604310 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f745c15f970 0x7f745c1f6c80 secure :-1 s=READY pgs=2287 cs=0 l=1 rev1=1 crypto rx=0x7f7454006010 tx=0x7f7454031650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:44.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:44.242+0000 7f7443fff640 1 -- 172.21.15.38:0/4054604310 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7454003e00 con 0x7f745c15f970 2024-09-21T03:44:44.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:44.242+0000 7f7443fff640 1 -- 172.21.15.38:0/4054604310 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7454031a80 con 0x7f745c15f970 2024-09-21T03:44:44.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:44.242+0000 7f7461a2e640 1 -- 172.21.15.38:0/4054604310 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f745c0c9fe0 con 0x7f745c15f970 2024-09-21T03:44:44.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:44.242+0000 7f7443fff640 1 -- 172.21.15.38:0/4054604310 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7454033070 con 0x7f745c15f970 2024-09-21T03:44:44.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:44.242+0000 7f7461a2e640 1 -- 172.21.15.38:0/4054604310 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f745c070ad0 con 0x7f745c15f970 2024-09-21T03:44:44.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:44.243+0000 7f7461a2e640 1 -- 172.21.15.38:0/4054604310 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f745c0c9a70 con 0x7f745c15f970 2024-09-21T03:44:44.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:44.244+0000 7f7443fff640 1 -- 172.21.15.38:0/4054604310 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f7454033210 con 0x7f745c15f970 2024-09-21T03:44:44.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:44.244+0000 7f7443fff640 1 --2- 172.21.15.38:0/4054604310 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7448078b90 0x7f744807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:44.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:44.245+0000 7f7443fff640 1 -- 172.21.15.38:0/4054604310 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 9744+0+0 (secure 0 0 0) 0x7f74540bd090 con 0x7f745c15f970 2024-09-21T03:44:44.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:44.245+0000 7f745affd640 1 --2- 172.21.15.38:0/4054604310 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7448078b90 0x7f744807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:44.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:44.245+0000 7f745affd640 1 --2- 172.21.15.38:0/4054604310 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7448078b90 0x7f744807b0a0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f744c009b30 tx=0x7f744c005f90 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:44.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:44.249+0000 7f7443fff640 1 -- 172.21.15.38:0/4054604310 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f745403f030 con 0x7f745c15f970 2024-09-21T03:44:44.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:44.472+0000 7f7461a2e640 1 -- 172.21.15.38:0/4054604310 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f745c070d30 con 0x7f745c15f970 2024-09-21T03:44:44.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:44.473+0000 7f7443fff640 1 -- 172.21.15.38:0/4054604310 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 334 v334) ==== 95+0+5214 (secure 0 0 0) 0x7f74540845a0 con 0x7f745c15f970 2024-09-21T03:44:44.475 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:44:44.475 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":334,"btime":"2024-09-21T03:44:44:227980+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6086,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/965972663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":965972663},{"type":"v1","addr":"172.21.15.195:6836","nonce":965972663}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6700,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/3176122334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":3176122334},{"type":"v1","addr":"172.21.15.195:6837","nonce":3176122334}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":10551,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/791475497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":791475497},{"type":"v1","addr":"172.21.15.38:6833","nonce":791475497}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":330},{"gid":10602,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/434821886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":434821886},{"type":"v1","addr":"172.21.15.38:6835","nonce":434821886}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":334}],"filesystems":[{"mdsmap":{"epoch":333,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:40:02.319282+0000","modified":"2024-09-21T03:44:38.966733+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":136,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-21T03:44:44.475 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 334 2024-09-21T03:44:44.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:44.475+0000 7f7461a2e640 1 -- 172.21.15.38:0/4054604310 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7448078b90 msgr2=0x7f744807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:44.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:44.475+0000 7f7461a2e640 1 --2- 172.21.15.38:0/4054604310 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7448078b90 0x7f744807b0a0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f744c009b30 tx=0x7f744c005f90 comp rx=0 tx=0).stop 2024-09-21T03:44:44.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:44.475+0000 7f7461a2e640 1 -- 172.21.15.38:0/4054604310 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f745c15f970 msgr2=0x7f745c1f6c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:44.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:44.475+0000 7f7461a2e640 1 --2- 172.21.15.38:0/4054604310 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f745c15f970 0x7f745c1f6c80 secure :-1 s=READY pgs=2287 cs=0 l=1 rev1=1 crypto rx=0x7f7454006010 tx=0x7f7454031650 comp rx=0 tx=0).stop 2024-09-21T03:44:44.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:44.475+0000 7f7461a2e640 1 -- 172.21.15.38:0/4054604310 shutdown_connections 2024-09-21T03:44:44.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:44.475+0000 7f7461a2e640 1 --2- 172.21.15.38:0/4054604310 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f745c15f970 0x7f745c1f6c80 unknown :-1 s=CLOSED pgs=2287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:44.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:44.475+0000 7f7461a2e640 1 --2- 172.21.15.38:0/4054604310 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f745c15bfa0 0x7f745c0b5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:44.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:44.475+0000 7f7461a2e640 1 --2- 172.21.15.38:0/4054604310 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7448078b90 0x7f744807b0a0 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:44.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:44.475+0000 7f7461a2e640 1 --2- 172.21.15.38:0/4054604310 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f745c074210 0x7f745c071960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:44.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:44.475+0000 7f7461a2e640 1 -- 172.21.15.38:0/4054604310 >> 172.21.15.38:0/4054604310 conn(0x7f745c073560 msgr2=0x7f745c0c68c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:44.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:44.475+0000 7f7461a2e640 1 -- 172.21.15.38:0/4054604310 shutdown_connections 2024-09-21T03:44:44.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:44.475+0000 7f7461a2e640 1 -- 172.21.15.38:0/4054604310 wait complete. 2024-09-21T03:44:44.489 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T03:44:44.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:44.603+0000 7f2260427640 1 -- 172.21.15.38:0/156315446 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2258162280 msgr2=0x7f2258162690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:44.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:44.603+0000 7f2260427640 1 --2- 172.21.15.38:0/156315446 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2258162280 0x7f2258162690 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f224c009860 tx=0x7f224c02a530 comp rx=0 tx=0).stop 2024-09-21T03:44:44.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:44.603+0000 7f2260427640 1 -- 172.21.15.38:0/156315446 shutdown_connections 2024-09-21T03:44:44.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:44.603+0000 7f2260427640 1 --2- 172.21.15.38:0/156315446 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2258162280 0x7f2258162690 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:44.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:44.603+0000 7f2260427640 1 -- 172.21.15.38:0/156315446 >> 172.21.15.38:0/156315446 conn(0x7f22580b6b20 msgr2=0x7f22580d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:44.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:44.603+0000 7f2260427640 1 -- 172.21.15.38:0/156315446 shutdown_connections 2024-09-21T03:44:44.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:44.603+0000 7f2260427640 1 -- 172.21.15.38:0/156315446 wait complete. 2024-09-21T03:44:44.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:44.605+0000 7f2260427640 1 Processor -- start 2024-09-21T03:44:44.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:44.605+0000 7f2260427640 1 -- start start 2024-09-21T03:44:44.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:44.607+0000 7f2260427640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f22580b2e40 0x7f22580cee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:44.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:44.607+0000 7f2260427640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2258162280 0x7f2258158260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:44.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:44.607+0000 7f2260427640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f22581f8650 0x7f22581f8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:44.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:44.607+0000 7f2260427640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f225815f640 con 0x7f22580b2e40 2024-09-21T03:44:44.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:44.607+0000 7f2260427640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f225815f4c0 con 0x7f2258162280 2024-09-21T03:44:44.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:44.607+0000 7f2260427640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f225815f7c0 con 0x7f22581f8650 2024-09-21T03:44:44.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:44.607+0000 7f225e1c3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f22580b2e40 0x7f22580cee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:44.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:44.607+0000 7f225e1c3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f22580b2e40 0x7f22580cee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44054/0 (socket says 172.21.15.38:44054) 2024-09-21T03:44:44.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:44.607+0000 7f225e1c3640 1 -- 172.21.15.38:0/918435118 learned_addr learned my addr 172.21.15.38:0/918435118 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:44:44.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:44.607+0000 7f225d9c2640 1 --2- 172.21.15.38:0/918435118 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2258162280 0x7f2258158260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:44.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:44.607+0000 7f225e9c4640 1 --2- 172.21.15.38:0/918435118 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f22581f8650 0x7f22581f8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:44.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:44.607+0000 7f225e1c3640 1 -- 172.21.15.38:0/918435118 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f22581f8650 msgr2=0x7f22581f8a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:44.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:44.607+0000 7f225e1c3640 1 --2- 172.21.15.38:0/918435118 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f22581f8650 0x7f22581f8a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:44.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:44.607+0000 7f225e1c3640 1 -- 172.21.15.38:0/918435118 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2258162280 msgr2=0x7f2258158260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:44.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:44.607+0000 7f225e1c3640 1 --2- 172.21.15.38:0/918435118 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2258162280 0x7f2258158260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:44.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:44.607+0000 7f225e1c3640 1 -- 172.21.15.38:0/918435118 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22580cf390 con 0x7f22580b2e40 2024-09-21T03:44:44.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:44.607+0000 7f225e1c3640 1 --2- 172.21.15.38:0/918435118 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f22580b2e40 0x7f22580cee50 secure :-1 s=READY pgs=2289 cs=0 l=1 rev1=1 crypto rx=0x7f225000c490 tx=0x7f225000c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:44.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:44.607+0000 7f2256ffd640 1 -- 172.21.15.38:0/918435118 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2250017070 con 0x7f22580b2e40 2024-09-21T03:44:44.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:44.608+0000 7f2256ffd640 1 -- 172.21.15.38:0/918435118 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f22500053e0 con 0x7f22580b2e40 2024-09-21T03:44:44.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:44.608+0000 7f2260427640 1 -- 172.21.15.38:0/918435118 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22581f8fa0 con 0x7f22580b2e40 2024-09-21T03:44:44.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:44.608+0000 7f2256ffd640 1 -- 172.21.15.38:0/918435118 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2250003850 con 0x7f22580b2e40 2024-09-21T03:44:44.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:44.608+0000 7f2260427640 1 -- 172.21.15.38:0/918435118 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2258158cb0 con 0x7f22580b2e40 2024-09-21T03:44:44.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:44.608+0000 7f2260427640 1 -- 172.21.15.38:0/918435118 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2258162f00 con 0x7f22580b2e40 2024-09-21T03:44:44.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:44.609+0000 7f2256ffd640 1 -- 172.21.15.38:0/918435118 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f225001a070 con 0x7f22580b2e40 2024-09-21T03:44:44.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:44.610+0000 7f2256ffd640 1 --2- 172.21.15.38:0/918435118 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2244078bc0 0x7f224407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:44.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:44.610+0000 7f2256ffd640 1 -- 172.21.15.38:0/918435118 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 9744+0+0 (secure 0 0 0) 0x7f225009b0f0 con 0x7f22580b2e40 2024-09-21T03:44:44.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:44.610+0000 7f225d9c2640 1 --2- 172.21.15.38:0/918435118 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2244078bc0 0x7f224407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:44.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:44.611+0000 7f225d9c2640 1 --2- 172.21.15.38:0/918435118 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2244078bc0 0x7f224407b0d0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f224c034000 tx=0x7f224c004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:44.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:44.615+0000 7f2256ffd640 1 -- 172.21.15.38:0/918435118 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f22581f8fa0 con 0x7f22580b2e40 2024-09-21T03:44:44.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:44.805+0000 7f2260427640 1 -- 172.21.15.38:0/918435118 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f225815dcd0 con 0x7f22580b2e40 2024-09-21T03:44:44.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:44.806+0000 7f2256ffd640 1 -- 172.21.15.38:0/918435118 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v136) ==== 74+0+22476 (secure 0 0 0) 0x7f22580cf390 con 0x7f22580b2e40 2024-09-21T03:44:44.809 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:44:44.809 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":136,"fsid":"9d50d66f-656b-4fe2-8e3b-fe53b79c4d43","created":"2024-09-21T03:33:10.677865+0000","modified":"2024-09-21T03:44:38.966438+0000","last_up_change":"2024-09-21T03:33:24.023421+0000","last_in_change":"2024-09-21T03:33:14.296193+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T03:33:22.992573+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-09-21T03:39:59.945436+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-09-21T03:40:01.239789+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dbab183f-7e63-48b8-960a-ae55fd5bf565","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6801","nonce":2381639433}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6803","nonce":2381639433}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6807","nonce":2381639433}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6805","nonce":2381639433}]},"public_addr":"172.21.15.38:6801/2381639433","cluster_addr":"172.21.15.38:6803/2381639433","heartbeat_back_addr":"172.21.15.38:6807/2381639433","heartbeat_front_addr":"172.21.15.38:6805/2381639433","state":["exists","up"]},{"osd":1,"uuid":"fa1c0e8a-ebc1-4359-8146-bd5342abbc16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6817","nonce":1491818732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6819","nonce":1491818732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6823","nonce":1491818732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6821","nonce":1491818732}]},"public_addr":"172.21.15.38:6817/1491818732","cluster_addr":"172.21.15.38:6819/1491818732","heartbeat_back_addr":"172.21.15.38:6823/1491818732","heartbeat_front_addr":"172.21.15.38:6821/1491818732","state":["exists","up"]},{"osd":2,"uuid":"2ed3cd2d-385d-4fc6-9869-214552a84803","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6825","nonce":2577858902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6827","nonce":2577858902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6831","nonce":2577858902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6829","nonce":2577858902}]},"public_addr":"172.21.15.38:6825/2577858902","cluster_addr":"172.21.15.38:6827/2577858902","heartbeat_back_addr":"172.21.15.38:6831/2577858902","heartbeat_front_addr":"172.21.15.38:6829/2577858902","state":["exists","up"]},{"osd":3,"uuid":"62d1d95e-919d-4250-be71-1d34cfa70dbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6809","nonce":1286810294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6811","nonce":1286810294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6815","nonce":1286810294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6813","nonce":1286810294}]},"public_addr":"172.21.15.38:6809/1286810294","cluster_addr":"172.21.15.38:6811/1286810294","heartbeat_back_addr":"172.21.15.38:6815/1286810294","heartbeat_front_addr":"172.21.15.38:6813/1286810294","state":["exists","up"]},{"osd":4,"uuid":"7c9f9314-99d7-4d73-8392-37809a9f54b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6803","nonce":3401837571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6805","nonce":3401837571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6809","nonce":3401837571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6807","nonce":3401837571}]},"public_addr":"172.21.15.195:6803/3401837571","cluster_addr":"172.21.15.195:6805/3401837571","heartbeat_back_addr":"172.21.15.195:6809/3401837571","heartbeat_front_addr":"172.21.15.195:6807/3401837571","state":["exists","up"]},{"osd":5,"uuid":"bde4fe9f-8414-4bba-bf4a-43973d1976bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6811","nonce":2080960180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6813","nonce":2080960180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6817","nonce":2080960180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6815","nonce":2080960180}]},"public_addr":"172.21.15.195:6811/2080960180","cluster_addr":"172.21.15.195:6813/2080960180","heartbeat_back_addr":"172.21.15.195:6817/2080960180","heartbeat_front_addr":"172.21.15.195:6815/2080960180","state":["exists","up"]},{"osd":6,"uuid":"6ee6d7f4-1a5f-4a4c-ac16-48fe05f6a002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6827","nonce":4055825675}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6829","nonce":4055825675}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6833","nonce":4055825675}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6831","nonce":4055825675}]},"public_addr":"172.21.15.195:6827/4055825675","cluster_addr":"172.21.15.195:6829/4055825675","heartbeat_back_addr":"172.21.15.195:6833/4055825675","heartbeat_front_addr":"172.21.15.195:6831/4055825675","state":["exists","up"]},{"osd":7,"uuid":"efeb5f51-329a-4484-8290-92e7d3503d05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6819","nonce":1606106342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6821","nonce":1606106342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6825","nonce":1606106342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6823","nonce":1606106342}]},"public_addr":"172.21.15.195:6819/1606106342","cluster_addr":"172.21.15.195:6821/1606106342","heartbeat_back_addr":"172.21.15.195:6825/1606106342","heartbeat_front_addr":"172.21.15.195:6823/1606106342","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.901343+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:21.115003+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.901623+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.835238+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.132199+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:19.817560+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.829690+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:21.087300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.a","mappings":[{"from":6,"to":5}]},{"pgid":"14.1f","mappings":[{"from":7,"to":0}]},{"pgid":"14.25","mappings":[{"from":7,"to":3}]},{"pgid":"14.26","mappings":[{"from":1,"to":0}]},{"pgid":"14.2d","mappings":[{"from":7,"to":0}]},{"pgid":"14.33","mappings":[{"from":1,"to":3}]},{"pgid":"14.34","mappings":[{"from":7,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.38:6832/999357334":"2024-09-22T03:44:31.921782+0000","172.21.15.38:6832/2851755473":"2024-09-22T03:44:25.830423+0000","172.21.15.38:6833/2178429646":"2024-09-22T03:44:20.787490+0000","172.21.15.38:6832/2178429646":"2024-09-22T03:44:20.787490+0000","172.21.15.38:6833/3103607164":"2024-09-22T03:44:05.867078+0000","172.21.15.38:6832/3103607164":"2024-09-22T03:44:05.867078+0000","172.21.15.38:6832/2444649344":"2024-09-22T03:44:00.664196+0000","172.21.15.38:6833/999357334":"2024-09-22T03:44:31.921782+0000","172.21.15.38:6833/626794495":"2024-09-22T03:43:49.180309+0000","172.21.15.38:6832/1643781179":"2024-09-22T03:43:33.596532+0000","172.21.15.38:6833/2309772457":"2024-09-22T03:43:28.198223+0000","172.21.15.38:6832/2309772457":"2024-09-22T03:43:28.198223+0000","172.21.15.38:6835/141447142":"2024-09-22T03:43:14.684929+0000","172.21.15.38:6834/141447142":"2024-09-22T03:43:14.684929+0000","172.21.15.38:6835/3572946698":"2024-09-22T03:41:44.737273+0000","172.21.15.38:6833/3553953266":"2024-09-22T03:41:29.639383+0000","172.21.15.38:6834/2751962827":"2024-09-22T03:40:38.029580+0000","172.21.15.38:6832/3553953266":"2024-09-22T03:41:29.639383+0000","172.21.15.38:6834/2930805359":"2024-09-22T03:42:23.215190+0000","172.21.15.38:6832/626794495":"2024-09-22T03:43:49.180309+0000","172.21.15.38:6833/1643781179":"2024-09-22T03:43:33.596532+0000","172.21.15.38:6835/2748959970":"2024-09-22T03:41:16.565736+0000","172.21.15.38:6833/3710736169":"2024-09-22T03:40:59.454480+0000","172.21.15.38:6833/1995903506":"2024-09-22T03:43:00.406120+0000","172.21.15.38:6833/775682959":"2024-09-22T03:43:10.570678+0000","172.21.15.38:6834/2748959970":"2024-09-22T03:41:16.565736+0000","172.21.15.38:6834/2934569248":"2024-09-22T03:40:26.934591+0000","172.21.15.38:6832/3726830644":"2024-09-22T03:41:12.532341+0000","172.21.15.38:6835/2097789246":"2024-09-22T03:43:06.604471+0000","172.21.15.38:6835/2934569248":"2024-09-22T03:40:26.934591+0000","172.21.15.38:6835/3864487171":"2024-09-22T03:40:31.989837+0000","172.21.15.38:6833/4221150556":"2024-09-22T03:42:27.240627+0000","172.21.15.38:6832/121492560":"2024-09-22T03:41:10.371933+0000","172.21.15.38:6832/1717955365":"2024-09-22T03:41:51.787804+0000","172.21.15.38:6834/2418225210":"2024-09-22T03:42:02.869612+0000","172.21.15.38:6833/2851755473":"2024-09-22T03:44:25.830423+0000","172.21.15.38:6835/2418225210":"2024-09-22T03:42:02.869612+0000","172.21.15.38:6833/121492560":"2024-09-22T03:41:10.371933+0000","172.21.15.38:6835/3951934534":"2024-09-22T03:42:31.276099+0000","172.21.15.38:6833/751618334":"2024-09-22T03:40:42.053279+0000","172.21.15.195:6837/712370966":"2024-09-22T03:40:11.633787+0000","172.21.15.38:6832/3402963947":"2024-09-22T03:41:23.555005+0000","172.21.15.38:6835/2798021708":"2024-09-22T03:42:47.436632+0000","172.21.15.38:6835/3801772046":"2024-09-22T03:44:38.966412+0000","172.21.15.38:6833/3726830644":"2024-09-22T03:41:12.532341+0000","172.21.15.38:6834/2916238332":"2024-09-22T03:41:42.553770+0000","172.21.15.38:6834/3055025811":"2024-09-22T03:42:39.361236+0000","172.21.15.195:6835/712370966":"2024-09-22T03:40:11.633787+0000","172.21.15.38:6834/3864487171":"2024-09-22T03:40:31.989837+0000","172.21.15.38:6834/3572946698":"2024-09-22T03:41:44.737273+0000","172.21.15.38:6832/1995903506":"2024-09-22T03:43:00.406120+0000","172.21.15.38:6832/775682959":"2024-09-22T03:43:10.570678+0000","172.21.15.38:6835/2916238332":"2024-09-22T03:41:42.553770+0000","172.21.15.38:6833/3402963947":"2024-09-22T03:41:23.555005+0000","172.21.15.38:6834/425672974":"2024-09-22T03:40:46.105521+0000","172.21.15.38:6834/3951934534":"2024-09-22T03:42:31.276099+0000","172.21.15.38:6835/425672974":"2024-09-22T03:40:46.105521+0000","172.21.15.38:6832/3710736169":"2024-09-22T03:40:59.454480+0000","172.21.15.38:6833/1717955365":"2024-09-22T03:41:51.787804+0000","172.21.15.38:6835/3055025811":"2024-09-22T03:42:39.361236+0000","172.21.15.38:6835/2751962827":"2024-09-22T03:40:38.029580+0000","172.21.15.38:6835/2712360296":"2024-09-22T03:42:20.931115+0000","172.21.15.38:6833/3537258034":"2024-09-22T03:42:06.901189+0000","172.21.15.38:6834/3801772046":"2024-09-22T03:44:38.966412+0000","172.21.15.38:6832/3537258034":"2024-09-22T03:42:06.901189+0000","172.21.15.38:6832/1272005650":"2024-09-22T03:41:58.836862+0000","172.21.15.38:6834/2712360296":"2024-09-22T03:42:20.931115+0000","172.21.15.38:6832/751618334":"2024-09-22T03:40:42.053279+0000","172.21.15.38:6835/2930805359":"2024-09-22T03:42:23.215190+0000","172.21.15.38:6832/4221150556":"2024-09-22T03:42:27.240627+0000","172.21.15.38:6833/1272005650":"2024-09-22T03:41:58.836862+0000","172.21.15.38:6832/245538755":"2024-09-22T03:42:35.318756+0000","172.21.15.38:6833/245538755":"2024-09-22T03:42:35.318756+0000","172.21.15.38:6833/2444649344":"2024-09-22T03:44:00.664196+0000","172.21.15.38:6833/2459386631":"2024-09-22T03:42:43.395609+0000","172.21.15.38:6832/2459386631":"2024-09-22T03:42:43.395609+0000","172.21.15.38:6834/2097789246":"2024-09-22T03:43:06.604471+0000","172.21.15.38:6834/2798021708":"2024-09-22T03:42:47.436632+0000","172.21.15.38:6832/111206731":"2024-09-22T03:43:02.560071+0000","172.21.15.38:6833/111206731":"2024-09-22T03:43:02.560071+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T03:44:44.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:44.809+0000 7f2260427640 1 -- 172.21.15.38:0/918435118 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2244078bc0 msgr2=0x7f224407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:44.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:44.809+0000 7f2260427640 1 --2- 172.21.15.38:0/918435118 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2244078bc0 0x7f224407b0d0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f224c034000 tx=0x7f224c004730 comp rx=0 tx=0).stop 2024-09-21T03:44:44.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:44.809+0000 7f2260427640 1 -- 172.21.15.38:0/918435118 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f22580b2e40 msgr2=0x7f22580cee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:44.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:44.809+0000 7f2260427640 1 --2- 172.21.15.38:0/918435118 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f22580b2e40 0x7f22580cee50 secure :-1 s=READY pgs=2289 cs=0 l=1 rev1=1 crypto rx=0x7f225000c490 tx=0x7f225000c950 comp rx=0 tx=0).stop 2024-09-21T03:44:44.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:44.809+0000 7f2260427640 1 -- 172.21.15.38:0/918435118 shutdown_connections 2024-09-21T03:44:44.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:44.809+0000 7f2260427640 1 --2- 172.21.15.38:0/918435118 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2244078bc0 0x7f224407b0d0 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:44.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:44.809+0000 7f2260427640 1 --2- 172.21.15.38:0/918435118 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f22581f8650 0x7f22581f8a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:44.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:44.809+0000 7f2260427640 1 --2- 172.21.15.38:0/918435118 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2258162280 0x7f2258158260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:44.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:44.809+0000 7f2260427640 1 --2- 172.21.15.38:0/918435118 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f22580b2e40 0x7f22580cee50 unknown :-1 s=CLOSED pgs=2289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:44.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:44.809+0000 7f2260427640 1 -- 172.21.15.38:0/918435118 >> 172.21.15.38:0/918435118 conn(0x7f22580b6b20 msgr2=0x7f225806cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:44.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:44.809+0000 7f2260427640 1 -- 172.21.15.38:0/918435118 shutdown_connections 2024-09-21T03:44:44.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:44.809+0000 7f2260427640 1 -- 172.21.15.38:0/918435118 wait complete. 2024-09-21T03:44:44.823 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-21T03:44:44.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:44.938+0000 7f99b3f58640 1 -- 172.21.15.38:0/2563069046 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f99ac0b9ad0 msgr2=0x7f99ac0d0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:44.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:44.938+0000 7f99b3f58640 1 --2- 172.21.15.38:0/2563069046 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f99ac0b9ad0 0x7f99ac0d0be0 secure :-1 s=READY pgs=2291 cs=0 l=1 rev1=1 crypto rx=0x7f99ac0cfba0 tx=0x7f99a002a410 comp rx=0 tx=0).stop 2024-09-21T03:44:44.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:44.938+0000 7f99b3f58640 1 -- 172.21.15.38:0/2563069046 shutdown_connections 2024-09-21T03:44:44.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:44.938+0000 7f99b3f58640 1 --2- 172.21.15.38:0/2563069046 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f99ac0b9ad0 0x7f99ac0d0be0 unknown :-1 s=CLOSED pgs=2291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:44.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:44.938+0000 7f99b3f58640 1 --2- 172.21.15.38:0/2563069046 >> v2:172.21.15.195:3300/0 conn(0x7f99ac15ee80 0x7f99ac0d0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:44.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:44.938+0000 7f99b3f58640 1 -- 172.21.15.38:0/2563069046 >> 172.21.15.38:0/2563069046 conn(0x7f99ac09d830 msgr2=0x7f99ac0b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:44.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:44.938+0000 7f99b3f58640 1 -- 172.21.15.38:0/2563069046 shutdown_connections 2024-09-21T03:44:44.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:44.938+0000 7f99b3f58640 1 -- 172.21.15.38:0/2563069046 wait complete. 2024-09-21T03:44:44.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:44.940+0000 7f99b3f58640 1 Processor -- start 2024-09-21T03:44:44.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:44.940+0000 7f99b3f58640 1 -- start start 2024-09-21T03:44:44.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:44.941+0000 7f99b3f58640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f99ac0b9ad0 0x7f99ac157850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:44.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:44.941+0000 7f99b3f58640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f99ac0d1120 0x7f99ac06da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:44.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:44.941+0000 7f99b3f58640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f99ac15ee80 0x7f99ac1f6550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:44.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:44.941+0000 7f99b3f58640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f99ac0b6700 con 0x7f99ac0b9ad0 2024-09-21T03:44:44.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:44.941+0000 7f99b3f58640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f99ac0b6580 con 0x7f99ac0d1120 2024-09-21T03:44:44.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:44.941+0000 7f99b3f58640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f99ac0b6880 con 0x7f99ac15ee80 2024-09-21T03:44:44.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:44.941+0000 7f99b1cf4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f99ac0b9ad0 0x7f99ac157850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:44.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:44.941+0000 7f99b1cf4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f99ac0b9ad0 0x7f99ac157850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44076/0 (socket says 172.21.15.38:44076) 2024-09-21T03:44:44.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:44.942+0000 7f99b1cf4640 1 -- 172.21.15.38:0/2242253122 learned_addr learned my addr 172.21.15.38:0/2242253122 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:44:44.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:44.942+0000 7f99b14f3640 1 --2- 172.21.15.38:0/2242253122 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f99ac0d1120 0x7f99ac06da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:44.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:44.942+0000 7f99b24f5640 1 --2- 172.21.15.38:0/2242253122 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f99ac15ee80 0x7f99ac1f6550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:44.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:44.942+0000 7f99b1cf4640 1 -- 172.21.15.38:0/2242253122 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f99ac15ee80 msgr2=0x7f99ac1f6550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:44.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:44.942+0000 7f99b1cf4640 1 --2- 172.21.15.38:0/2242253122 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f99ac15ee80 0x7f99ac1f6550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:44.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:44.942+0000 7f99b1cf4640 1 -- 172.21.15.38:0/2242253122 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f99ac0d1120 msgr2=0x7f99ac06da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:44.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:44.942+0000 7f99b1cf4640 1 --2- 172.21.15.38:0/2242253122 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f99ac0d1120 0x7f99ac06da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:44.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:44.942+0000 7f99b1cf4640 1 -- 172.21.15.38:0/2242253122 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99ac1f6a90 con 0x7f99ac0b9ad0 2024-09-21T03:44:44.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:44.942+0000 7f99b1cf4640 1 --2- 172.21.15.38:0/2242253122 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f99ac0b9ad0 0x7f99ac157850 secure :-1 s=READY pgs=2292 cs=0 l=1 rev1=1 crypto rx=0x7f999c00daf0 tx=0x7f999c00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:44.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:44.942+0000 7f99aa7fc640 1 -- 172.21.15.38:0/2242253122 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f999c017070 con 0x7f99ac0b9ad0 2024-09-21T03:44:44.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:44.942+0000 7f99aa7fc640 1 -- 172.21.15.38:0/2242253122 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f999c003970 con 0x7f99ac0b9ad0 2024-09-21T03:44:44.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:44.942+0000 7f99b3f58640 1 -- 172.21.15.38:0/2242253122 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99ac06cf60 con 0x7f99ac0b9ad0 2024-09-21T03:44:44.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:44.942+0000 7f99aa7fc640 1 -- 172.21.15.38:0/2242253122 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f999c003dc0 con 0x7f99ac0b9ad0 2024-09-21T03:44:44.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:44.942+0000 7f99b3f58640 1 -- 172.21.15.38:0/2242253122 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99ac06d220 con 0x7f99ac0b9ad0 2024-09-21T03:44:44.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:44.943+0000 7f99b3f58640 1 -- 172.21.15.38:0/2242253122 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99ac0b7d30 con 0x7f99ac0b9ad0 2024-09-21T03:44:44.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:44.944+0000 7f99aa7fc640 1 -- 172.21.15.38:0/2242253122 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f999c004020 con 0x7f99ac0b9ad0 2024-09-21T03:44:44.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:44.945+0000 7f99aa7fc640 1 --2- 172.21.15.38:0/2242253122 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9998078b90 0x7f999807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:44.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:44.945+0000 7f99aa7fc640 1 -- 172.21.15.38:0/2242253122 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 9744+0+0 (secure 0 0 0) 0x7f999c09c1d0 con 0x7f99ac0b9ad0 2024-09-21T03:44:44.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:44.945+0000 7f99b14f3640 1 --2- 172.21.15.38:0/2242253122 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9998078b90 0x7f999807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:44.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:44.946+0000 7f99b14f3640 1 --2- 172.21.15.38:0/2242253122 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9998078b90 0x7f999807b0a0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f99a0034000 tx=0x7f99a0004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:44.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:44.950+0000 7f99aa7fc640 1 -- 172.21.15.38:0/2242253122 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f99ac0b7d30 con 0x7f99ac0b9ad0 2024-09-21T03:44:45.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:45.174+0000 7f99b3f58640 1 -- 172.21.15.38:0/2242253122 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f99ac0b6d30 con 0x7f99ac0b9ad0 2024-09-21T03:44:45.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:45.240+0000 7f99aa7fc640 1 -- 172.21.15.38:0/2242253122 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v335) ==== 161+0+0 (secure 0 0 0) 0x7f999c020030 con 0x7f99ac0b9ad0 2024-09-21T03:44:45.242 INFO:teuthology.orchestra.run.smithi038.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-21T03:44:45.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:45.243+0000 7f99b3f58640 1 -- 172.21.15.38:0/2242253122 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9998078b90 msgr2=0x7f999807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:45.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:45.243+0000 7f99b3f58640 1 --2- 172.21.15.38:0/2242253122 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9998078b90 0x7f999807b0a0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f99a0034000 tx=0x7f99a0004730 comp rx=0 tx=0).stop 2024-09-21T03:44:45.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:45.243+0000 7f99b3f58640 1 -- 172.21.15.38:0/2242253122 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f99ac0b9ad0 msgr2=0x7f99ac157850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:45.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:45.243+0000 7f99b3f58640 1 --2- 172.21.15.38:0/2242253122 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f99ac0b9ad0 0x7f99ac157850 secure :-1 s=READY pgs=2292 cs=0 l=1 rev1=1 crypto rx=0x7f999c00daf0 tx=0x7f999c00b260 comp rx=0 tx=0).stop 2024-09-21T03:44:45.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:45.243+0000 7f99b3f58640 1 -- 172.21.15.38:0/2242253122 shutdown_connections 2024-09-21T03:44:45.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:45.243+0000 7f99b3f58640 1 --2- 172.21.15.38:0/2242253122 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9998078b90 0x7f999807b0a0 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:45.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:45.243+0000 7f99b3f58640 1 --2- 172.21.15.38:0/2242253122 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f99ac15ee80 0x7f99ac1f6550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:45.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:45.243+0000 7f99b3f58640 1 --2- 172.21.15.38:0/2242253122 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f99ac0d1120 0x7f99ac06da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:45.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:45.243+0000 7f99b3f58640 1 --2- 172.21.15.38:0/2242253122 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f99ac0b9ad0 0x7f99ac157850 unknown :-1 s=CLOSED pgs=2292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:45.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:45.243+0000 7f99b3f58640 1 -- 172.21.15.38:0/2242253122 >> 172.21.15.38:0/2242253122 conn(0x7f99ac09d830 msgr2=0x7f99ac15d740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:45.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:45.243+0000 7f99b3f58640 1 -- 172.21.15.38:0/2242253122 shutdown_connections 2024-09-21T03:44:45.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:45.243+0000 7f99b3f58640 1 -- 172.21.15.38:0/2242253122 wait complete. 2024-09-21T03:44:45.257 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-21T03:44:45.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:45.373+0000 7fb08f7f9640 1 -- 172.21.15.38:0/1210448216 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb070004800 msgr2=0x7fb070004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:45.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:45.373+0000 7fb08f7f9640 1 --2- 172.21.15.38:0/1210448216 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb070004800 0x7fb070004c10 secure :-1 s=READY pgs=2294 cs=0 l=1 rev1=1 crypto rx=0x7fb07c001690 tx=0x7fb070005830 comp rx=0 tx=0).stop 2024-09-21T03:44:45.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:45.374+0000 7fb08f7f9640 1 -- 172.21.15.38:0/1210448216 shutdown_connections 2024-09-21T03:44:45.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:45.374+0000 7fb08f7f9640 1 --2- 172.21.15.38:0/1210448216 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb070004800 0x7fb070004c10 unknown :-1 s=CLOSED pgs=2294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:45.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:45.374+0000 7fb08f7f9640 1 -- 172.21.15.38:0/1210448216 >> 172.21.15.38:0/1210448216 conn(0x7fb0880b1b90 msgr2=0x7fb0880b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:45.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:45.374+0000 7fb08f7f9640 1 -- 172.21.15.38:0/1210448216 shutdown_connections 2024-09-21T03:44:45.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:45.374+0000 7fb08f7f9640 1 -- 172.21.15.38:0/1210448216 wait complete. 2024-09-21T03:44:45.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:45.375+0000 7fb08f7f9640 1 Processor -- start 2024-09-21T03:44:45.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:45.376+0000 7fb08f7f9640 1 -- start start 2024-09-21T03:44:45.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:45.377+0000 7fb08f7f9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb070004800 0x7fb088154200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:45.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:45.377+0000 7fb08f7f9640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb0880b1fa0 0x7fb088156ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:45.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:45.377+0000 7fb08f7f9640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb0880a06a0 0x7fb088157410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:45.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:45.377+0000 7fb08f7f9640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fb0880cf410 con 0x7fb070004800 2024-09-21T03:44:45.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:45.377+0000 7fb08f7f9640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fb0880cf290 con 0x7fb0880a06a0 2024-09-21T03:44:45.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:45.377+0000 7fb08f7f9640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fb0880cf590 con 0x7fb0880b1fa0 2024-09-21T03:44:45.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:45.377+0000 7fb08d595640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb070004800 0x7fb088154200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:45.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:45.377+0000 7fb08d595640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb070004800 0x7fb088154200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44086/0 (socket says 172.21.15.38:44086) 2024-09-21T03:44:45.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:45.377+0000 7fb08d595640 1 -- 172.21.15.38:0/2113697738 learned_addr learned my addr 172.21.15.38:0/2113697738 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:44:45.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:45.377+0000 7fb08cd94640 1 --2- 172.21.15.38:0/2113697738 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb0880b1fa0 0x7fb088156ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:45.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:45.377+0000 7fb08dd96640 1 --2- 172.21.15.38:0/2113697738 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb0880a06a0 0x7fb088157410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:45.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:45.377+0000 7fb08d595640 1 -- 172.21.15.38:0/2113697738 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb0880b1fa0 msgr2=0x7fb088156ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:45.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:45.377+0000 7fb08d595640 1 --2- 172.21.15.38:0/2113697738 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb0880b1fa0 0x7fb088156ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:45.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:45.377+0000 7fb08d595640 1 -- 172.21.15.38:0/2113697738 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb0880a06a0 msgr2=0x7fb088157410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:45.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:45.377+0000 7fb08d595640 1 --2- 172.21.15.38:0/2113697738 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb0880a06a0 0x7fb088157410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:45.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:45.377+0000 7fb08d595640 1 -- 172.21.15.38:0/2113697738 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0881f70e0 con 0x7fb070004800 2024-09-21T03:44:45.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:45.377+0000 7fb08d595640 1 --2- 172.21.15.38:0/2113697738 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb070004800 0x7fb088154200 secure :-1 s=READY pgs=2295 cs=0 l=1 rev1=1 crypto rx=0x7fb080006600 tx=0x7fb0800039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:45.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:45.378+0000 7fb085ffb640 1 -- 172.21.15.38:0/2113697738 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb080015070 con 0x7fb070004800 2024-09-21T03:44:45.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:45.378+0000 7fb085ffb640 1 -- 172.21.15.38:0/2113697738 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb080011920 con 0x7fb070004800 2024-09-21T03:44:45.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:45.378+0000 7fb085ffb640 1 -- 172.21.15.38:0/2113697738 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb080011d70 con 0x7fb070004800 2024-09-21T03:44:45.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:45.378+0000 7fb08f7f9640 1 -- 172.21.15.38:0/2113697738 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0881f6560 con 0x7fb070004800 2024-09-21T03:44:45.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:45.378+0000 7fb08f7f9640 1 -- 172.21.15.38:0/2113697738 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0881f6b20 con 0x7fb070004800 2024-09-21T03:44:45.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:45.379+0000 7fb08f7f9640 1 -- 172.21.15.38:0/2113697738 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb08806ddf0 con 0x7fb070004800 2024-09-21T03:44:45.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:45.380+0000 7fb085ffb640 1 -- 172.21.15.38:0/2113697738 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb080018070 con 0x7fb070004800 2024-09-21T03:44:45.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:45.380+0000 7fb085ffb640 1 --2- 172.21.15.38:0/2113697738 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb074078b90 0x7fb07407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:45.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:45.380+0000 7fb085ffb640 1 -- 172.21.15.38:0/2113697738 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 9744+0+0 (secure 0 0 0) 0x7fb08009c970 con 0x7fb070004800 2024-09-21T03:44:45.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:45.380+0000 7fb08cd94640 1 --2- 172.21.15.38:0/2113697738 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb074078b90 0x7fb07407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:45.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:45.381+0000 7fb08cd94640 1 --2- 172.21.15.38:0/2113697738 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb074078b90 0x7fb07407b0a0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fb07c032000 tx=0x7fb07c02c040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:45.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:45.385+0000 7fb085ffb640 1 -- 172.21.15.38:0/2113697738 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fb080063750 con 0x7fb070004800 2024-09-21T03:44:45.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:45.609+0000 7fb08f7f9640 1 -- 172.21.15.38:0/2113697738 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fb0880cc4c0 con 0x7fb070004800 2024-09-21T03:44:46.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:46.242+0000 7fb085ffb640 1 -- 172.21.15.38:0/2113697738 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v336) ==== 104+0+0 (secure 0 0 0) 0x7fb080068180 con 0x7fb070004800 2024-09-21T03:44:46.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:46.245+0000 7fb08f7f9640 1 -- 172.21.15.38:0/2113697738 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb074078b90 msgr2=0x7fb07407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:46.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:46.245+0000 7fb08f7f9640 1 --2- 172.21.15.38:0/2113697738 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb074078b90 0x7fb07407b0a0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fb07c032000 tx=0x7fb07c02c040 comp rx=0 tx=0).stop 2024-09-21T03:44:46.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:46.245+0000 7fb08f7f9640 1 -- 172.21.15.38:0/2113697738 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb070004800 msgr2=0x7fb088154200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:46.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:46.245+0000 7fb08f7f9640 1 --2- 172.21.15.38:0/2113697738 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb070004800 0x7fb088154200 secure :-1 s=READY pgs=2295 cs=0 l=1 rev1=1 crypto rx=0x7fb080006600 tx=0x7fb0800039c0 comp rx=0 tx=0).stop 2024-09-21T03:44:46.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:46.245+0000 7fb08f7f9640 1 -- 172.21.15.38:0/2113697738 shutdown_connections 2024-09-21T03:44:46.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:46.245+0000 7fb08f7f9640 1 --2- 172.21.15.38:0/2113697738 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb0880a06a0 0x7fb088157410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:46.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:46.245+0000 7fb08f7f9640 1 --2- 172.21.15.38:0/2113697738 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb074078b90 0x7fb07407b0a0 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:46.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:46.245+0000 7fb08f7f9640 1 --2- 172.21.15.38:0/2113697738 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb0880b1fa0 0x7fb088156ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:46.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:46.245+0000 7fb08f7f9640 1 --2- 172.21.15.38:0/2113697738 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb070004800 0x7fb088154200 unknown :-1 s=CLOSED pgs=2295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:46.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:46.245+0000 7fb08f7f9640 1 -- 172.21.15.38:0/2113697738 >> 172.21.15.38:0/2113697738 conn(0x7fb0880b1b90 msgr2=0x7fb08806f480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:46.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:46.245+0000 7fb08f7f9640 1 -- 172.21.15.38:0/2113697738 shutdown_connections 2024-09-21T03:44:46.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:46.245+0000 7fb08f7f9640 1 -- 172.21.15.38:0/2113697738 wait complete. 2024-09-21T03:44:46.261 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-21T03:44:46.391 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:46.389+0000 7f6021c25640 1 -- 172.21.15.38:0/353068947 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6004004d40 msgr2=0x7f6004005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:46.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:46.389+0000 7f6021c25640 1 --2- 172.21.15.38:0/353068947 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6004004d40 0x7f6004005150 secure :-1 s=READY pgs=2297 cs=0 l=1 rev1=1 crypto rx=0x7f601c072480 tx=0x7f6004005830 comp rx=0 tx=0).stop 2024-09-21T03:44:46.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:46.389+0000 7f601bfff640 1 -- 172.21.15.38:0/353068947 reap_dead start 2024-09-21T03:44:46.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:46.389+0000 7f6021c25640 1 -- 172.21.15.38:0/353068947 shutdown_connections 2024-09-21T03:44:46.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:46.389+0000 7f6021c25640 1 -- 172.21.15.38:0/353068947 >> 172.21.15.38:0/353068947 conn(0x7f601c0d2280 msgr2=0x7f601c0d2690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:46.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:46.389+0000 7f6021c25640 1 -- 172.21.15.38:0/353068947 shutdown_connections 2024-09-21T03:44:46.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:46.389+0000 7f6021c25640 1 -- 172.21.15.38:0/353068947 wait complete. 2024-09-21T03:44:46.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:46.391+0000 7f6021c25640 1 Processor -- start 2024-09-21T03:44:46.394 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:46.391+0000 7f6021c25640 1 -- start start 2024-09-21T03:44:46.398 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:46.392+0000 7f6021c25640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6004004d40 0x7f601c0b7500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:46.398 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:46.392+0000 7f6021c25640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f601c0b7a40 0x7f601c06f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:46.398 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:46.392+0000 7f6021c25640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f601c06f9e0 0x7f601c155390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:46.399 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:46.392+0000 7f6021c25640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f601c0d0390 con 0x7f6004004d40 2024-09-21T03:44:46.399 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:46.392+0000 7f6021c25640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f601c0d0210 con 0x7f601c0b7a40 2024-09-21T03:44:46.399 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:46.392+0000 7f6021c25640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f601c0ccae0 con 0x7f601c06f9e0 2024-09-21T03:44:46.399 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:46.392+0000 7f601b7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6004004d40 0x7f601c0b7500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:46.399 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:46.392+0000 7f601b7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6004004d40 0x7f601c0b7500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44102/0 (socket says 172.21.15.38:44102) 2024-09-21T03:44:46.399 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:46.392+0000 7f601b7fe640 1 -- 172.21.15.38:0/4029799464 learned_addr learned my addr 172.21.15.38:0/4029799464 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:44:46.399 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:46.393+0000 7f601affd640 1 --2- 172.21.15.38:0/4029799464 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f601c0b7a40 0x7f601c06f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:46.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:46.393+0000 7f601bfff640 1 --2- 172.21.15.38:0/4029799464 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f601c06f9e0 0x7f601c155390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:46.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:46.393+0000 7f601b7fe640 1 -- 172.21.15.38:0/4029799464 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f601c06f9e0 msgr2=0x7f601c155390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:46.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:46.393+0000 7f601b7fe640 1 --2- 172.21.15.38:0/4029799464 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f601c06f9e0 0x7f601c155390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:46.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:46.393+0000 7f601b7fe640 1 -- 172.21.15.38:0/4029799464 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f601c0b7a40 msgr2=0x7f601c06f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:46.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:46.393+0000 7f601b7fe640 1 --2- 172.21.15.38:0/4029799464 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f601c0b7a40 0x7f601c06f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:46.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:46.393+0000 7f601b7fe640 1 -- 172.21.15.38:0/4029799464 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f601c1f8d30 con 0x7f6004004d40 2024-09-21T03:44:46.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:46.393+0000 7f601affd640 1 --2- 172.21.15.38:0/4029799464 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f601c0b7a40 0x7f601c06f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:44:46.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:46.393+0000 7f601bfff640 1 --2- 172.21.15.38:0/4029799464 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f601c06f9e0 0x7f601c155390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:44:46.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:46.393+0000 7f601b7fe640 1 --2- 172.21.15.38:0/4029799464 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6004004d40 0x7f601c0b7500 secure :-1 s=READY pgs=2298 cs=0 l=1 rev1=1 crypto rx=0x7f600c011ae0 tx=0x7f600c009ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:46.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:46.393+0000 7f6003fff640 1 -- 172.21.15.38:0/4029799464 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f600c01e070 con 0x7f6004004d40 2024-09-21T03:44:46.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:46.393+0000 7f6003fff640 1 -- 172.21.15.38:0/4029799464 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f600c00fc40 con 0x7f6004004d40 2024-09-21T03:44:46.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:46.393+0000 7f6021c25640 1 -- 172.21.15.38:0/4029799464 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f601c1f9010 con 0x7f6004004d40 2024-09-21T03:44:46.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:46.393+0000 7f6003fff640 1 -- 172.21.15.38:0/4029799464 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f600c004030 con 0x7f6004004d40 2024-09-21T03:44:46.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:46.393+0000 7f6021c25640 1 -- 172.21.15.38:0/4029799464 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f601c1f95d0 con 0x7f6004004d40 2024-09-21T03:44:46.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:46.394+0000 7f6021c25640 1 -- 172.21.15.38:0/4029799464 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f601c15c500 con 0x7f6004004d40 2024-09-21T03:44:46.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:46.395+0000 7f6003fff640 1 -- 172.21.15.38:0/4029799464 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f600c021070 con 0x7f6004004d40 2024-09-21T03:44:46.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:46.395+0000 7f6003fff640 1 --2- 172.21.15.38:0/4029799464 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6008078bc0 0x7f600807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:46.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:46.396+0000 7f6003fff640 1 -- 172.21.15.38:0/4029799464 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 9744+0+0 (secure 0 0 0) 0x7f600c09c5b0 con 0x7f6004004d40 2024-09-21T03:44:46.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:46.396+0000 7f601affd640 1 --2- 172.21.15.38:0/4029799464 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6008078bc0 0x7f600807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:46.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:46.396+0000 7f601affd640 1 --2- 172.21.15.38:0/4029799464 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6008078bc0 0x7f600807b0d0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f6010000c00 tx=0x7f60100012b0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:46.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:46.400+0000 7f6003fff640 1 -- 172.21.15.38:0/4029799464 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f600c063350 con 0x7f6004004d40 2024-09-21T03:44:46.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:46.589+0000 7f6021c25640 1 -- 172.21.15.38:0/4029799464 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f601c0cdcc0 con 0x7f6004004d40 2024-09-21T03:44:47.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:47.262+0000 7f6003fff640 1 -- 172.21.15.38:0/4029799464 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v137) ==== 181+0+0 (secure 0 0 0) 0x7f600c067d80 con 0x7f6004004d40 2024-09-21T03:44:47.265 INFO:teuthology.orchestra.run.smithi038.stderr:pool 'cephfs_metadata' removed 2024-09-21T03:44:47.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:47.265+0000 7f6021c25640 1 -- 172.21.15.38:0/4029799464 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6008078bc0 msgr2=0x7f600807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:47.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:47.265+0000 7f6021c25640 1 --2- 172.21.15.38:0/4029799464 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6008078bc0 0x7f600807b0d0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f6010000c00 tx=0x7f60100012b0 comp rx=0 tx=0).stop 2024-09-21T03:44:47.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:47.265+0000 7f6021c25640 1 -- 172.21.15.38:0/4029799464 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6004004d40 msgr2=0x7f601c0b7500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:47.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:47.265+0000 7f6021c25640 1 --2- 172.21.15.38:0/4029799464 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6004004d40 0x7f601c0b7500 secure :-1 s=READY pgs=2298 cs=0 l=1 rev1=1 crypto rx=0x7f600c011ae0 tx=0x7f600c009ea0 comp rx=0 tx=0).stop 2024-09-21T03:44:47.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:47.266+0000 7f6021c25640 1 -- 172.21.15.38:0/4029799464 shutdown_connections 2024-09-21T03:44:47.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:47.266+0000 7f6021c25640 1 --2- 172.21.15.38:0/4029799464 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6008078bc0 0x7f600807b0d0 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:47.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:47.266+0000 7f6021c25640 1 --2- 172.21.15.38:0/4029799464 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f601c06f9e0 0x7f601c155390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:47.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:47.266+0000 7f6021c25640 1 --2- 172.21.15.38:0/4029799464 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f601c0b7a40 0x7f601c06f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:47.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:47.266+0000 7f6021c25640 1 --2- 172.21.15.38:0/4029799464 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6004004d40 0x7f601c0b7500 unknown :-1 s=CLOSED pgs=2298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:47.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:47.266+0000 7f6021c25640 1 -- 172.21.15.38:0/4029799464 >> 172.21.15.38:0/4029799464 conn(0x7f601c0d2280 msgr2=0x7f601c159480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:47.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:47.266+0000 7f6021c25640 1 -- 172.21.15.38:0/4029799464 shutdown_connections 2024-09-21T03:44:47.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:47.266+0000 7f6021c25640 1 -- 172.21.15.38:0/4029799464 wait complete. 2024-09-21T03:44:47.284 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-21T03:44:47.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:47.411+0000 7f8c478de640 1 -- 172.21.15.38:0/874656265 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8c28004d90 msgr2=0x7f8c28005220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:47.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:47.411+0000 7f8c478de640 1 --2- 172.21.15.38:0/874656265 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8c28004d90 0x7f8c28005220 secure :-1 s=READY pgs=2300 cs=0 l=1 rev1=1 crypto rx=0x7f8c38001830 tx=0x7f8c28005830 comp rx=0 tx=0).stop 2024-09-21T03:44:47.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:47.411+0000 7f8c45e7b640 1 -- 172.21.15.38:0/874656265 reap_dead start 2024-09-21T03:44:47.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:47.411+0000 7f8c478de640 1 -- 172.21.15.38:0/874656265 shutdown_connections 2024-09-21T03:44:47.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:47.411+0000 7f8c478de640 1 -- 172.21.15.38:0/874656265 >> 172.21.15.38:0/874656265 conn(0x7f8c400d44e0 msgr2=0x7f8c400d48f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:47.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:47.411+0000 7f8c478de640 1 -- 172.21.15.38:0/874656265 shutdown_connections 2024-09-21T03:44:47.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:47.411+0000 7f8c478de640 1 -- 172.21.15.38:0/874656265 wait complete. 2024-09-21T03:44:47.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:47.412+0000 7f8c478de640 1 Processor -- start 2024-09-21T03:44:47.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:47.413+0000 7f8c478de640 1 -- start start 2024-09-21T03:44:47.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:47.414+0000 7f8c478de640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8c400cc0b0 0x7f8c400ca490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:47.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:47.414+0000 7f8c478de640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8c40153850 0x7f8c401f8d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:47.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:47.414+0000 7f8c478de640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8c401f9290 0x7f8c40160c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:47.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:47.414+0000 7f8c478de640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f8c400b62f0 con 0x7f8c400cc0b0 2024-09-21T03:44:47.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:47.414+0000 7f8c478de640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f8c400b6170 con 0x7f8c40153850 2024-09-21T03:44:47.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:47.414+0000 7f8c478de640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f8c400d2ce0 con 0x7f8c401f9290 2024-09-21T03:44:47.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:47.414+0000 7f8c4567a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8c400cc0b0 0x7f8c400ca490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:47.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:47.414+0000 7f8c4567a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8c400cc0b0 0x7f8c400ca490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44122/0 (socket says 172.21.15.38:44122) 2024-09-21T03:44:47.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:47.414+0000 7f8c4567a640 1 -- 172.21.15.38:0/3465116959 learned_addr learned my addr 172.21.15.38:0/3465116959 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:44:47.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:47.414+0000 7f8c45e7b640 1 --2- 172.21.15.38:0/3465116959 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8c401f9290 0x7f8c40160c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:47.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:47.414+0000 7f8c44e79640 1 --2- 172.21.15.38:0/3465116959 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8c40153850 0x7f8c401f8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:47.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:47.414+0000 7f8c4567a640 1 -- 172.21.15.38:0/3465116959 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8c401f9290 msgr2=0x7f8c40160c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:47.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:47.414+0000 7f8c4567a640 1 --2- 172.21.15.38:0/3465116959 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8c401f9290 0x7f8c40160c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:47.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:47.414+0000 7f8c4567a640 1 -- 172.21.15.38:0/3465116959 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8c40153850 msgr2=0x7f8c401f8d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:47.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:47.414+0000 7f8c4567a640 1 --2- 172.21.15.38:0/3465116959 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8c40153850 0x7f8c401f8d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:47.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:47.414+0000 7f8c4567a640 1 -- 172.21.15.38:0/3465116959 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c40157900 con 0x7f8c400cc0b0 2024-09-21T03:44:47.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:47.414+0000 7f8c44e79640 1 --2- 172.21.15.38:0/3465116959 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8c40153850 0x7f8c401f8d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:44:47.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:47.414+0000 7f8c45e7b640 1 --2- 172.21.15.38:0/3465116959 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8c401f9290 0x7f8c40160c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:44:47.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:47.415+0000 7f8c4567a640 1 --2- 172.21.15.38:0/3465116959 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8c400cc0b0 0x7f8c400ca490 secure :-1 s=READY pgs=2301 cs=0 l=1 rev1=1 crypto rx=0x7f8c30011d60 tx=0x7f8c3000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:47.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:47.415+0000 7f8c3dffb640 1 -- 172.21.15.38:0/3465116959 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c30003d80 con 0x7f8c400cc0b0 2024-09-21T03:44:47.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:47.415+0000 7f8c3dffb640 1 -- 172.21.15.38:0/3465116959 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8c30011e50 con 0x7f8c400cc0b0 2024-09-21T03:44:47.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:47.415+0000 7f8c478de640 1 -- 172.21.15.38:0/3465116959 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c401f9c70 con 0x7f8c400cc0b0 2024-09-21T03:44:47.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:47.415+0000 7f8c3dffb640 1 -- 172.21.15.38:0/3465116959 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c30004d40 con 0x7f8c400cc0b0 2024-09-21T03:44:47.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:47.415+0000 7f8c478de640 1 -- 172.21.15.38:0/3465116959 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c40160490 con 0x7f8c400cc0b0 2024-09-21T03:44:47.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:47.416+0000 7f8c478de640 1 -- 172.21.15.38:0/3465116959 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c401626d0 con 0x7f8c400cc0b0 2024-09-21T03:44:47.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:47.417+0000 7f8c3dffb640 1 -- 172.21.15.38:0/3465116959 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f8c30004050 con 0x7f8c400cc0b0 2024-09-21T03:44:47.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:47.418+0000 7f8c3dffb640 1 --2- 172.21.15.38:0/3465116959 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8c2c078b90 0x7f8c2c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:47.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:47.418+0000 7f8c3dffb640 1 -- 172.21.15.38:0/3465116959 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 9277+0+0 (secure 0 0 0) 0x7f8c3009cbe0 con 0x7f8c400cc0b0 2024-09-21T03:44:47.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:47.418+0000 7f8c44e79640 1 --2- 172.21.15.38:0/3465116959 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8c2c078b90 0x7f8c2c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:47.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:47.418+0000 7f8c44e79640 1 --2- 172.21.15.38:0/3465116959 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8c2c078b90 0x7f8c2c07b0a0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f8c34003670 tx=0x7f8c340068a0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:47.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:47.422+0000 7f8c3dffb640 1 -- 172.21.15.38:0/3465116959 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f8c30064270 con 0x7f8c400cc0b0 2024-09-21T03:44:47.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:47.608+0000 7f8c478de640 1 -- 172.21.15.38:0/3465116959 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f8c4015b5a0 con 0x7f8c400cc0b0 2024-09-21T03:44:48.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:48.271+0000 7f8c3dffb640 1 -- 172.21.15.38:0/3465116959 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v138) ==== 169+0+0 (secure 0 0 0) 0x7f8c30063b70 con 0x7f8c400cc0b0 2024-09-21T03:44:48.273 INFO:teuthology.orchestra.run.smithi038.stderr:pool 'cephfs_data' removed 2024-09-21T03:44:48.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:48.275+0000 7f8c478de640 1 -- 172.21.15.38:0/3465116959 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8c2c078b90 msgr2=0x7f8c2c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:48.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:48.275+0000 7f8c478de640 1 --2- 172.21.15.38:0/3465116959 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8c2c078b90 0x7f8c2c07b0a0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f8c34003670 tx=0x7f8c340068a0 comp rx=0 tx=0).stop 2024-09-21T03:44:48.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:48.275+0000 7f8c478de640 1 -- 172.21.15.38:0/3465116959 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8c400cc0b0 msgr2=0x7f8c400ca490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:48.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:48.275+0000 7f8c478de640 1 --2- 172.21.15.38:0/3465116959 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8c400cc0b0 0x7f8c400ca490 secure :-1 s=READY pgs=2301 cs=0 l=1 rev1=1 crypto rx=0x7f8c30011d60 tx=0x7f8c3000f790 comp rx=0 tx=0).stop 2024-09-21T03:44:48.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:48.276+0000 7f8c478de640 1 -- 172.21.15.38:0/3465116959 shutdown_connections 2024-09-21T03:44:48.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:48.276+0000 7f8c478de640 1 --2- 172.21.15.38:0/3465116959 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8c2c078b90 0x7f8c2c07b0a0 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:48.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:48.276+0000 7f8c478de640 1 --2- 172.21.15.38:0/3465116959 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8c401f9290 0x7f8c40160c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:48.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:48.276+0000 7f8c478de640 1 --2- 172.21.15.38:0/3465116959 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8c40153850 0x7f8c401f8d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:48.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:48.276+0000 7f8c478de640 1 --2- 172.21.15.38:0/3465116959 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8c400cc0b0 0x7f8c400ca490 unknown :-1 s=CLOSED pgs=2301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:48.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:48.276+0000 7f8c478de640 1 -- 172.21.15.38:0/3465116959 >> 172.21.15.38:0/3465116959 conn(0x7f8c400d44e0 msgr2=0x7f8c400b2f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:48.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:48.279+0000 7f8c478de640 1 -- 172.21.15.38:0/3465116959 shutdown_connections 2024-09-21T03:44:48.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:48.279+0000 7f8c478de640 1 -- 172.21.15.38:0/3465116959 wait complete. 2024-09-21T03:44:48.297 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 31 2024-09-21T03:44:48.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:48.419+0000 7f0103e0e640 1 -- 172.21.15.38:0/2161718649 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f00fc15ee80 msgr2=0x7f00fc0d0170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:48.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:48.419+0000 7f0103e0e640 1 --2- 172.21.15.38:0/2161718649 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f00fc15ee80 0x7f00fc0d0170 secure :-1 s=READY pgs=2303 cs=0 l=1 rev1=1 crypto rx=0x7f00f8009830 tx=0x7f00f802a480 comp rx=0 tx=0).stop 2024-09-21T03:44:48.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:48.419+0000 7f0103e0e640 1 -- 172.21.15.38:0/2161718649 shutdown_connections 2024-09-21T03:44:48.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:48.419+0000 7f0103e0e640 1 --2- 172.21.15.38:0/2161718649 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f00fc0ceb90 0x7f00fc0cefa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:48.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:48.419+0000 7f0103e0e640 1 --2- 172.21.15.38:0/2161718649 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f00fc15ee80 0x7f00fc0d0170 unknown :-1 s=CLOSED pgs=2303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:48.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:48.419+0000 7f0103e0e640 1 -- 172.21.15.38:0/2161718649 >> 172.21.15.38:0/2161718649 conn(0x7f00fc09d830 msgr2=0x7f00fc0b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:48.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:48.419+0000 7f0103e0e640 1 -- 172.21.15.38:0/2161718649 shutdown_connections 2024-09-21T03:44:48.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:48.419+0000 7f0103e0e640 1 -- 172.21.15.38:0/2161718649 wait complete. 2024-09-21T03:44:48.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:48.421+0000 7f0103e0e640 1 Processor -- start 2024-09-21T03:44:48.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:48.421+0000 7f0103e0e640 1 -- start start 2024-09-21T03:44:48.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:48.422+0000 7f0103e0e640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f00fc0b9ad0 0x7f00fc0ce440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:48.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:48.422+0000 7f0103e0e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f00fc0ceb90 0x7f00fc158540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:48.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:48.422+0000 7f0103e0e640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f00fc15ee80 0x7f00fc1f8b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:48.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:48.422+0000 7f0103e0e640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f00fc0b6700 con 0x7f00fc0ceb90 2024-09-21T03:44:48.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:48.422+0000 7f0103e0e640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f00fc0b6580 con 0x7f00fc0b9ad0 2024-09-21T03:44:48.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:48.422+0000 7f0103e0e640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f00fc0b6880 con 0x7f00fc15ee80 2024-09-21T03:44:48.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:48.422+0000 7f01013a9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f00fc0ceb90 0x7f00fc158540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:48.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:48.422+0000 7f01013a9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f00fc0ceb90 0x7f00fc158540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44132/0 (socket says 172.21.15.38:44132) 2024-09-21T03:44:48.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:48.422+0000 7f01013a9640 1 -- 172.21.15.38:0/1538894781 learned_addr learned my addr 172.21.15.38:0/1538894781 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:44:48.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:48.423+0000 7f01023ab640 1 --2- 172.21.15.38:0/1538894781 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f00fc15ee80 0x7f00fc1f8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:48.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:48.423+0000 7f0101baa640 1 --2- 172.21.15.38:0/1538894781 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f00fc0b9ad0 0x7f00fc0ce440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:48.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:48.423+0000 7f01013a9640 1 -- 172.21.15.38:0/1538894781 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f00fc15ee80 msgr2=0x7f00fc1f8b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:48.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:48.423+0000 7f01013a9640 1 --2- 172.21.15.38:0/1538894781 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f00fc15ee80 0x7f00fc1f8b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:48.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:48.423+0000 7f01013a9640 1 -- 172.21.15.38:0/1538894781 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f00fc0b9ad0 msgr2=0x7f00fc0ce440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:48.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:48.423+0000 7f01013a9640 1 --2- 172.21.15.38:0/1538894781 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f00fc0b9ad0 0x7f00fc0ce440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:48.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:48.423+0000 7f01013a9640 1 -- 172.21.15.38:0/1538894781 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00fc06f0b0 con 0x7f00fc0ceb90 2024-09-21T03:44:48.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:48.423+0000 7f0101baa640 1 --2- 172.21.15.38:0/1538894781 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f00fc0b9ad0 0x7f00fc0ce440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:44:48.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:48.423+0000 7f01013a9640 1 --2- 172.21.15.38:0/1538894781 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f00fc0ceb90 0x7f00fc158540 secure :-1 s=READY pgs=2304 cs=0 l=1 rev1=1 crypto rx=0x7f00ec001eb0 tx=0x7f00ec00c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:48.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:48.423+0000 7f00f27fc640 1 -- 172.21.15.38:0/1538894781 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00ec017070 con 0x7f00fc0ceb90 2024-09-21T03:44:48.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:48.423+0000 7f00f27fc640 1 -- 172.21.15.38:0/1538894781 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f00ec0035f0 con 0x7f00fc0ceb90 2024-09-21T03:44:48.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:48.423+0000 7f0103e0e640 1 -- 172.21.15.38:0/1538894781 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00fc06f350 con 0x7f00fc0ceb90 2024-09-21T03:44:48.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:48.423+0000 7f00f27fc640 1 -- 172.21.15.38:0/1538894781 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00ec003a20 con 0x7f00fc0ceb90 2024-09-21T03:44:48.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:48.423+0000 7f0103e0e640 1 -- 172.21.15.38:0/1538894781 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00fc06d220 con 0x7f00fc0ceb90 2024-09-21T03:44:48.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:48.424+0000 7f0103e0e640 1 -- 172.21.15.38:0/1538894781 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00fc0b6d30 con 0x7f00fc0ceb90 2024-09-21T03:44:48.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:48.425+0000 7f00f27fc640 1 -- 172.21.15.38:0/1538894781 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f00ec01a070 con 0x7f00fc0ceb90 2024-09-21T03:44:48.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:48.426+0000 7f00f27fc640 1 --2- 172.21.15.38:0/1538894781 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f00e0078b90 0x7f00e007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:48.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:48.426+0000 7f00f27fc640 1 -- 172.21.15.38:0/1538894781 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 8689+0+0 (secure 0 0 0) 0x7f00ec09ab00 con 0x7f00fc0ceb90 2024-09-21T03:44:48.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:48.426+0000 7f0101baa640 1 --2- 172.21.15.38:0/1538894781 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f00e0078b90 0x7f00e007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:48.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:48.427+0000 7f0101baa640 1 --2- 172.21.15.38:0/1538894781 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f00e0078b90 0x7f00e007b0a0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f00f8034000 tx=0x7f00f8004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:48.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:48.430+0000 7f00f27fc640 1 -- 172.21.15.38:0/1538894781 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f00fc1592d0 con 0x7f00fc0ceb90 2024-09-21T03:44:48.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:48.619+0000 7f0103e0e640 1 -- 172.21.15.38:0/1538894781 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "config reset", "num": 31} v 0) -- 0x7f00fc0cafe0 con 0x7f00fc0ceb90 2024-09-21T03:44:48.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:48.623+0000 7f00f27fc640 1 -- 172.21.15.38:0/1538894781 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 31}]=0 v48) ==== 71+0+0 (secure 0 0 0) 0x7f00ec061cf0 con 0x7f00fc0ceb90 2024-09-21T03:44:48.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:48.626+0000 7f0103e0e640 1 -- 172.21.15.38:0/1538894781 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f00e0078b90 msgr2=0x7f00e007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:48.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:48.626+0000 7f0103e0e640 1 --2- 172.21.15.38:0/1538894781 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f00e0078b90 0x7f00e007b0a0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f00f8034000 tx=0x7f00f8004730 comp rx=0 tx=0).stop 2024-09-21T03:44:48.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:48.626+0000 7f0103e0e640 1 -- 172.21.15.38:0/1538894781 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f00fc0ceb90 msgr2=0x7f00fc158540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:48.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:48.626+0000 7f0103e0e640 1 --2- 172.21.15.38:0/1538894781 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f00fc0ceb90 0x7f00fc158540 secure :-1 s=READY pgs=2304 cs=0 l=1 rev1=1 crypto rx=0x7f00ec001eb0 tx=0x7f00ec00c820 comp rx=0 tx=0).stop 2024-09-21T03:44:48.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:48.626+0000 7f0103e0e640 1 -- 172.21.15.38:0/1538894781 shutdown_connections 2024-09-21T03:44:48.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:48.626+0000 7f0103e0e640 1 --2- 172.21.15.38:0/1538894781 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f00e0078b90 0x7f00e007b0a0 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:48.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:48.626+0000 7f0103e0e640 1 --2- 172.21.15.38:0/1538894781 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f00fc15ee80 0x7f00fc1f8b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:48.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:48.626+0000 7f0103e0e640 1 --2- 172.21.15.38:0/1538894781 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f00fc0ceb90 0x7f00fc158540 unknown :-1 s=CLOSED pgs=2304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:48.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:48.626+0000 7f0103e0e640 1 --2- 172.21.15.38:0/1538894781 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f00fc0b9ad0 0x7f00fc0ce440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:48.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:48.626+0000 7f0103e0e640 1 -- 172.21.15.38:0/1538894781 >> 172.21.15.38:0/1538894781 conn(0x7f00fc09d830 msgr2=0x7f00fc0b3fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:48.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:48.626+0000 7f0103e0e640 1 -- 172.21.15.38:0/1538894781 shutdown_connections 2024-09-21T03:44:48.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:48.626+0000 7f0103e0e640 1 -- 172.21.15.38:0/1538894781 wait complete. 2024-09-21T03:44:48.639 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_damage.TestDamage.test_object_deletion' 2024-09-21T03:44:48.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:48.754+0000 7fcbc52b4640 1 -- 172.21.15.38:0/2826106423 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcbc00d19f0 msgr2=0x7fcbc015cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:48.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:48.754+0000 7fcbc52b4640 1 --2- 172.21.15.38:0/2826106423 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcbc00d19f0 0x7fcbc015cf20 secure :-1 s=READY pgs=2305 cs=0 l=1 rev1=1 crypto rx=0x7fcbc00b3cf0 tx=0x7fcba802a5f0 comp rx=0 tx=0).stop 2024-09-21T03:44:48.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:48.754+0000 7fcbc52b4640 1 -- 172.21.15.38:0/2826106423 shutdown_connections 2024-09-21T03:44:48.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:48.754+0000 7fcbc52b4640 1 --2- 172.21.15.38:0/2826106423 >> v2:172.21.15.195:3300/0 conn(0x7fcbc00b6250 0x7fcbc00b6660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:48.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:48.754+0000 7fcbc52b4640 1 --2- 172.21.15.38:0/2826106423 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcbc00d19f0 0x7fcbc015cf20 unknown :-1 s=CLOSED pgs=2305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:48.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:48.754+0000 7fcbc52b4640 1 -- 172.21.15.38:0/2826106423 >> 172.21.15.38:0/2826106423 conn(0x7fcbc00d2350 msgr2=0x7fcbc00d2760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:48.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:48.754+0000 7fcbc52b4640 1 -- 172.21.15.38:0/2826106423 shutdown_connections 2024-09-21T03:44:48.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:48.754+0000 7fcbc52b4640 1 -- 172.21.15.38:0/2826106423 wait complete. 2024-09-21T03:44:48.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:48.756+0000 7fcbc52b4640 1 Processor -- start 2024-09-21T03:44:48.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:48.756+0000 7fcbc52b4640 1 -- start start 2024-09-21T03:44:48.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:48.757+0000 7fcbc52b4640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcbc00b6250 0x7fcbc013e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:48.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:48.757+0000 7fcbc52b4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcbc00d19f0 0x7fcbc01511f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:48.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:48.757+0000 7fcbc52b4640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcbc015cb10 0x7fcbc01e6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:48.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:48.757+0000 7fcbc52b4640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fcbc00b6d20 con 0x7fcbc00d19f0 2024-09-21T03:44:48.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:48.757+0000 7fcbc52b4640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fcbc00b6ba0 con 0x7fcbc015cb10 2024-09-21T03:44:48.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:48.757+0000 7fcbc52b4640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fcbc00b6ea0 con 0x7fcbc00b6250 2024-09-21T03:44:48.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:48.757+0000 7fcbbe7fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcbc00d19f0 0x7fcbc01511f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:48.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:48.757+0000 7fcbbe7fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcbc00d19f0 0x7fcbc01511f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44156/0 (socket says 172.21.15.38:44156) 2024-09-21T03:44:48.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:48.757+0000 7fcbbe7fc640 1 -- 172.21.15.38:0/2481967132 learned_addr learned my addr 172.21.15.38:0/2481967132 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:44:48.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:48.757+0000 7fcbbf7fe640 1 --2- 172.21.15.38:0/2481967132 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcbc015cb10 0x7fcbc01e6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:48.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:48.757+0000 7fcbbeffd640 1 --2- 172.21.15.38:0/2481967132 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcbc00b6250 0x7fcbc013e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:48.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:48.757+0000 7fcbbe7fc640 1 -- 172.21.15.38:0/2481967132 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcbc00b6250 msgr2=0x7fcbc013e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:48.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:48.757+0000 7fcbbe7fc640 1 --2- 172.21.15.38:0/2481967132 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcbc00b6250 0x7fcbc013e810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:48.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:48.757+0000 7fcbbe7fc640 1 -- 172.21.15.38:0/2481967132 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcbc015cb10 msgr2=0x7fcbc01e6cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:48.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:48.757+0000 7fcbbe7fc640 1 --2- 172.21.15.38:0/2481967132 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcbc015cb10 0x7fcbc01e6cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:48.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:48.757+0000 7fcbbe7fc640 1 -- 172.21.15.38:0/2481967132 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbc01f7e80 con 0x7fcbc00d19f0 2024-09-21T03:44:48.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:48.758+0000 7fcbbe7fc640 1 --2- 172.21.15.38:0/2481967132 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcbc00d19f0 0x7fcbc01511f0 secure :-1 s=READY pgs=2306 cs=0 l=1 rev1=1 crypto rx=0x7fcba8034040 tx=0x7fcba8003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:48.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:48.758+0000 7fcbb77fe640 1 -- 172.21.15.38:0/2481967132 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcba803e030 con 0x7fcbc00d19f0 2024-09-21T03:44:48.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:48.758+0000 7fcbb77fe640 1 -- 172.21.15.38:0/2481967132 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcba8003d10 con 0x7fcbc00d19f0 2024-09-21T03:44:48.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:48.758+0000 7fcbb77fe640 1 -- 172.21.15.38:0/2481967132 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcba8035040 con 0x7fcbc00d19f0 2024-09-21T03:44:48.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:48.758+0000 7fcbc52b4640 1 -- 172.21.15.38:0/2481967132 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbc01e78d0 con 0x7fcbc00d19f0 2024-09-21T03:44:48.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:48.758+0000 7fcbc52b4640 1 -- 172.21.15.38:0/2481967132 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbc01e7e90 con 0x7fcbc00d19f0 2024-09-21T03:44:48.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:48.759+0000 7fcbc52b4640 1 -- 172.21.15.38:0/2481967132 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcbc01e7640 con 0x7fcbc00d19f0 2024-09-21T03:44:48.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:48.760+0000 7fcbb77fe640 1 -- 172.21.15.38:0/2481967132 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fcba8041030 con 0x7fcbc00d19f0 2024-09-21T03:44:48.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:48.760+0000 7fcbb77fe640 1 --2- 172.21.15.38:0/2481967132 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fcba0078bc0 0x7fcba007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:48.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:48.760+0000 7fcbb77fe640 1 -- 172.21.15.38:0/2481967132 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 8689+0+0 (secure 0 0 0) 0x7fcba80bc010 con 0x7fcbc00d19f0 2024-09-21T03:44:48.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:48.760+0000 7fcbbeffd640 1 --2- 172.21.15.38:0/2481967132 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fcba0078bc0 0x7fcba007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:48.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:48.761+0000 7fcbbeffd640 1 --2- 172.21.15.38:0/2481967132 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fcba0078bc0 0x7fcba007b0d0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fcbb0007fd0 tx=0x7fcbb0009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:48.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:48.765+0000 7fcbb77fe640 1 -- 172.21.15.38:0/2481967132 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fcba8083850 con 0x7fcbc00d19f0 2024-09-21T03:44:49.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:49.025+0000 7fcbc52b4640 1 -- 172.21.15.38:0/2481967132 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_damage.TestDamage.test_object_deletion"]} v 0) -- 0x7fcbc00b7350 con 0x7fcbc00d19f0 2024-09-21T03:44:49.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:49.274+0000 7fcbb77fe640 1 -- 172.21.15.38:0/2481967132 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_damage.TestDamage.test_object_deletion"]}]=0 v584) ==== 135+0+0 (secure 0 0 0) 0x7fcba8083150 con 0x7fcbc00d19f0 2024-09-21T03:44:49.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:49.276+0000 7fcbc52b4640 1 -- 172.21.15.38:0/2481967132 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fcba0078bc0 msgr2=0x7fcba007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:49.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:49.276+0000 7fcbc52b4640 1 --2- 172.21.15.38:0/2481967132 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fcba0078bc0 0x7fcba007b0d0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fcbb0007fd0 tx=0x7fcbb0009040 comp rx=0 tx=0).stop 2024-09-21T03:44:49.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:49.276+0000 7fcbc52b4640 1 -- 172.21.15.38:0/2481967132 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcbc00d19f0 msgr2=0x7fcbc01511f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:49.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:49.276+0000 7fcbc52b4640 1 --2- 172.21.15.38:0/2481967132 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcbc00d19f0 0x7fcbc01511f0 secure :-1 s=READY pgs=2306 cs=0 l=1 rev1=1 crypto rx=0x7fcba8034040 tx=0x7fcba8003820 comp rx=0 tx=0).stop 2024-09-21T03:44:49.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:49.276+0000 7fcbc52b4640 1 -- 172.21.15.38:0/2481967132 shutdown_connections 2024-09-21T03:44:49.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:49.276+0000 7fcbc52b4640 1 --2- 172.21.15.38:0/2481967132 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcbc015cb10 0x7fcbc01e6cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:49.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:49.276+0000 7fcbc52b4640 1 --2- 172.21.15.38:0/2481967132 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcbc00d19f0 0x7fcbc01511f0 unknown :-1 s=CLOSED pgs=2306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:49.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:49.276+0000 7fcbc52b4640 1 --2- 172.21.15.38:0/2481967132 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fcba0078bc0 0x7fcba007b0d0 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:49.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:49.276+0000 7fcbc52b4640 1 --2- 172.21.15.38:0/2481967132 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcbc00b6250 0x7fcbc013e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:49.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:49.276+0000 7fcbc52b4640 1 -- 172.21.15.38:0/2481967132 >> 172.21.15.38:0/2481967132 conn(0x7fcbc00d2350 msgr2=0x7fcbc0071170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:49.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:49.276+0000 7fcbc52b4640 1 -- 172.21.15.38:0/2481967132 shutdown_connections 2024-09-21T03:44:49.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:49.276+0000 7fcbc52b4640 1 -- 172.21.15.38:0/2481967132 wait complete. 2024-09-21T03:44:49.290 INFO:tasks.cephfs_test_runner:That the MDS has a clean 'damaged' response to loss of any single metadata object ... ok 2024-09-21T03:44:49.290 INFO:tasks.cephfs_test_runner:Starting test: test_open_ino_errors (tasks.cephfs.test_damage.TestDamage) That errors encountered during opening inos are properly propagated 2024-09-21T03:44:49.290 INFO:tasks.cephfs_test_runner:test_open_ino_errors (tasks.cephfs.test_damage.TestDamage) 2024-09-21T03:44:49.291 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T03:44:49.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:49.405+0000 7fb774c4d640 1 -- 172.21.15.38:0/4135943041 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb770162280 msgr2=0x7fb770162690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:49.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:49.405+0000 7fb774c4d640 1 --2- 172.21.15.38:0/4135943041 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb770162280 0x7fb770162690 secure :-1 s=READY pgs=2307 cs=0 l=1 rev1=1 crypto rx=0x7fb7700d5040 tx=0x7fb76402a560 comp rx=0 tx=0).stop 2024-09-21T03:44:49.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:49.405+0000 7fb774c4d640 1 -- 172.21.15.38:0/4135943041 shutdown_connections 2024-09-21T03:44:49.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:49.405+0000 7fb774c4d640 1 --2- 172.21.15.38:0/4135943041 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb7700b2e40 0x7fb7700b3250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:49.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:49.405+0000 7fb774c4d640 1 --2- 172.21.15.38:0/4135943041 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb770162280 0x7fb770162690 unknown :-1 s=CLOSED pgs=2307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:49.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:49.405+0000 7fb774c4d640 1 --2- 172.21.15.38:0/4135943041 >> v2:172.21.15.195:3300/0 conn(0x7fb770162f00 0x7fb7700d24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:49.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:49.405+0000 7fb774c4d640 1 -- 172.21.15.38:0/4135943041 >> 172.21.15.38:0/4135943041 conn(0x7fb7700b6b20 msgr2=0x7fb7700d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:49.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:49.405+0000 7fb774c4d640 1 -- 172.21.15.38:0/4135943041 shutdown_connections 2024-09-21T03:44:49.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:49.405+0000 7fb774c4d640 1 -- 172.21.15.38:0/4135943041 wait complete. 2024-09-21T03:44:49.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:49.407+0000 7fb774c4d640 1 Processor -- start 2024-09-21T03:44:49.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:49.407+0000 7fb774c4d640 1 -- start start 2024-09-21T03:44:49.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:49.408+0000 7fb774c4d640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb7700b2e40 0x7fb770159ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:49.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:49.408+0000 7fb774c4d640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb770162280 0x7fb7700ca900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:49.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:49.408+0000 7fb774c4d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb7701f6700 0x7fb7701f6b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:49.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:49.408+0000 7fb774c4d640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fb7700b3dd0 con 0x7fb7701f6700 2024-09-21T03:44:49.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:49.408+0000 7fb774c4d640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fb7700b3f50 con 0x7fb7700b2e40 2024-09-21T03:44:49.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:49.408+0000 7fb774c4d640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fb7700b40d0 con 0x7fb770162280 2024-09-21T03:44:49.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:49.408+0000 7fb76ed9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb7701f6700 0x7fb7701f6b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:49.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:49.408+0000 7fb76ed9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb7701f6700 0x7fb7701f6b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44178/0 (socket says 172.21.15.38:44178) 2024-09-21T03:44:49.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:49.408+0000 7fb76ed9d640 1 -- 172.21.15.38:0/4115839648 learned_addr learned my addr 172.21.15.38:0/4115839648 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:44:49.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:49.409+0000 7fb76dd9b640 1 --2- 172.21.15.38:0/4115839648 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb770162280 0x7fb7700ca900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:49.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:49.409+0000 7fb76e59c640 1 --2- 172.21.15.38:0/4115839648 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb7700b2e40 0x7fb770159ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:49.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:49.409+0000 7fb76ed9d640 1 -- 172.21.15.38:0/4115839648 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb770162280 msgr2=0x7fb7700ca900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:49.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:49.409+0000 7fb76ed9d640 1 --2- 172.21.15.38:0/4115839648 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb770162280 0x7fb7700ca900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:49.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:49.409+0000 7fb76ed9d640 1 -- 172.21.15.38:0/4115839648 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb7700b2e40 msgr2=0x7fb770159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:49.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:49.409+0000 7fb76ed9d640 1 --2- 172.21.15.38:0/4115839648 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb7700b2e40 0x7fb770159ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:49.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:49.409+0000 7fb76ed9d640 1 -- 172.21.15.38:0/4115839648 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb770158e30 con 0x7fb7701f6700 2024-09-21T03:44:49.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:49.409+0000 7fb76ed9d640 1 --2- 172.21.15.38:0/4115839648 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb7701f6700 0x7fb7701f6b10 secure :-1 s=READY pgs=2308 cs=0 l=1 rev1=1 crypto rx=0x7fb7680058c0 tx=0x7fb76800da70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:49.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:49.409+0000 7fb75effd640 1 -- 172.21.15.38:0/4115839648 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb768017070 con 0x7fb7701f6700 2024-09-21T03:44:49.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:49.409+0000 7fb75effd640 1 -- 172.21.15.38:0/4115839648 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb768005310 con 0x7fb7701f6700 2024-09-21T03:44:49.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:49.409+0000 7fb774c4d640 1 -- 172.21.15.38:0/4115839648 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb770159110 con 0x7fb7701f6700 2024-09-21T03:44:49.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:49.409+0000 7fb75effd640 1 -- 172.21.15.38:0/4115839648 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7680047a0 con 0x7fb7701f6700 2024-09-21T03:44:49.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:49.409+0000 7fb774c4d640 1 -- 172.21.15.38:0/4115839648 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7701596d0 con 0x7fb7701f6700 2024-09-21T03:44:49.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:49.410+0000 7fb774c4d640 1 -- 172.21.15.38:0/4115839648 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb770162f00 con 0x7fb7701f6700 2024-09-21T03:44:49.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:49.411+0000 7fb75effd640 1 -- 172.21.15.38:0/4115839648 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb768004940 con 0x7fb7701f6700 2024-09-21T03:44:49.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:49.411+0000 7fb75effd640 1 --2- 172.21.15.38:0/4115839648 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb750078b90 0x7fb75007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:49.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:49.412+0000 7fb75effd640 1 -- 172.21.15.38:0/4115839648 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 8689+0+0 (secure 0 0 0) 0x7fb76809b890 con 0x7fb7701f6700 2024-09-21T03:44:49.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:49.412+0000 7fb76e59c640 1 --2- 172.21.15.38:0/4115839648 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb750078b90 0x7fb75007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:49.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:49.412+0000 7fb76e59c640 1 --2- 172.21.15.38:0/4115839648 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb750078b90 0x7fb75007b0a0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fb760005fd0 tx=0x7fb7600097d0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:49.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:49.416+0000 7fb75effd640 1 -- 172.21.15.38:0/4115839648 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fb768063160 con 0x7fb7701f6700 2024-09-21T03:44:49.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:49.610+0000 7fb774c4d640 1 -- 172.21.15.38:0/4115839648 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb77015d660 con 0x7fb7701f6700 2024-09-21T03:44:49.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:49.611+0000 7fb75effd640 1 -- 172.21.15.38:0/4115839648 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v138) ==== 74+0+18383 (secure 0 0 0) 0x7fb768062a60 con 0x7fb7701f6700 2024-09-21T03:44:49.614 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:44:49.614 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":138,"fsid":"9d50d66f-656b-4fe2-8e3b-fe53b79c4d43","created":"2024-09-21T03:33:10.677865+0000","modified":"2024-09-21T03:44:48.263913+0000","last_up_change":"2024-09-21T03:33:24.023421+0000","last_in_change":"2024-09-21T03:33:14.296193+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T03:33:22.992573+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dbab183f-7e63-48b8-960a-ae55fd5bf565","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6801","nonce":2381639433}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6803","nonce":2381639433}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6807","nonce":2381639433}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6805","nonce":2381639433}]},"public_addr":"172.21.15.38:6801/2381639433","cluster_addr":"172.21.15.38:6803/2381639433","heartbeat_back_addr":"172.21.15.38:6807/2381639433","heartbeat_front_addr":"172.21.15.38:6805/2381639433","state":["exists","up"]},{"osd":1,"uuid":"fa1c0e8a-ebc1-4359-8146-bd5342abbc16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6817","nonce":1491818732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6819","nonce":1491818732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6823","nonce":1491818732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6821","nonce":1491818732}]},"public_addr":"172.21.15.38:6817/1491818732","cluster_addr":"172.21.15.38:6819/1491818732","heartbeat_back_addr":"172.21.15.38:6823/1491818732","heartbeat_front_addr":"172.21.15.38:6821/1491818732","state":["exists","up"]},{"osd":2,"uuid":"2ed3cd2d-385d-4fc6-9869-214552a84803","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6825","nonce":2577858902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6827","nonce":2577858902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6831","nonce":2577858902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6829","nonce":2577858902}]},"public_addr":"172.21.15.38:6825/2577858902","cluster_addr":"172.21.15.38:6827/2577858902","heartbeat_back_addr":"172.21.15.38:6831/2577858902","heartbeat_front_addr":"172.21.15.38:6829/2577858902","state":["exists","up"]},{"osd":3,"uuid":"62d1d95e-919d-4250-be71-1d34cfa70dbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6809","nonce":1286810294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6811","nonce":1286810294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6815","nonce":1286810294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6813","nonce":1286810294}]},"public_addr":"172.21.15.38:6809/1286810294","cluster_addr":"172.21.15.38:6811/1286810294","heartbeat_back_addr":"172.21.15.38:6815/1286810294","heartbeat_front_addr":"172.21.15.38:6813/1286810294","state":["exists","up"]},{"osd":4,"uuid":"7c9f9314-99d7-4d73-8392-37809a9f54b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6803","nonce":3401837571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6805","nonce":3401837571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6809","nonce":3401837571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6807","nonce":3401837571}]},"public_addr":"172.21.15.195:6803/3401837571","cluster_addr":"172.21.15.195:6805/3401837571","heartbeat_back_addr":"172.21.15.195:6809/3401837571","heartbeat_front_addr":"172.21.15.195:6807/3401837571","state":["exists","up"]},{"osd":5,"uuid":"bde4fe9f-8414-4bba-bf4a-43973d1976bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6811","nonce":2080960180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6813","nonce":2080960180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6817","nonce":2080960180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6815","nonce":2080960180}]},"public_addr":"172.21.15.195:6811/2080960180","cluster_addr":"172.21.15.195:6813/2080960180","heartbeat_back_addr":"172.21.15.195:6817/2080960180","heartbeat_front_addr":"172.21.15.195:6815/2080960180","state":["exists","up"]},{"osd":6,"uuid":"6ee6d7f4-1a5f-4a4c-ac16-48fe05f6a002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6827","nonce":4055825675}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6829","nonce":4055825675}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6833","nonce":4055825675}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6831","nonce":4055825675}]},"public_addr":"172.21.15.195:6827/4055825675","cluster_addr":"172.21.15.195:6829/4055825675","heartbeat_back_addr":"172.21.15.195:6833/4055825675","heartbeat_front_addr":"172.21.15.195:6831/4055825675","state":["exists","up"]},{"osd":7,"uuid":"efeb5f51-329a-4484-8290-92e7d3503d05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6819","nonce":1606106342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6821","nonce":1606106342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6825","nonce":1606106342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6823","nonce":1606106342}]},"public_addr":"172.21.15.195:6819/1606106342","cluster_addr":"172.21.15.195:6821/1606106342","heartbeat_back_addr":"172.21.15.195:6825/1606106342","heartbeat_front_addr":"172.21.15.195:6823/1606106342","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.901343+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:21.115003+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.901623+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.835238+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.132199+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:19.817560+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.829690+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:21.087300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.38:6832/999357334":"2024-09-22T03:44:31.921782+0000","172.21.15.38:6832/2851755473":"2024-09-22T03:44:25.830423+0000","172.21.15.38:6833/2178429646":"2024-09-22T03:44:20.787490+0000","172.21.15.38:6832/2178429646":"2024-09-22T03:44:20.787490+0000","172.21.15.38:6833/3103607164":"2024-09-22T03:44:05.867078+0000","172.21.15.38:6832/3103607164":"2024-09-22T03:44:05.867078+0000","172.21.15.38:6832/2444649344":"2024-09-22T03:44:00.664196+0000","172.21.15.38:6833/999357334":"2024-09-22T03:44:31.921782+0000","172.21.15.38:6833/626794495":"2024-09-22T03:43:49.180309+0000","172.21.15.38:6832/1643781179":"2024-09-22T03:43:33.596532+0000","172.21.15.38:6833/2309772457":"2024-09-22T03:43:28.198223+0000","172.21.15.38:6832/2309772457":"2024-09-22T03:43:28.198223+0000","172.21.15.38:6835/141447142":"2024-09-22T03:43:14.684929+0000","172.21.15.38:6834/141447142":"2024-09-22T03:43:14.684929+0000","172.21.15.38:6835/3572946698":"2024-09-22T03:41:44.737273+0000","172.21.15.38:6833/3553953266":"2024-09-22T03:41:29.639383+0000","172.21.15.38:6834/2751962827":"2024-09-22T03:40:38.029580+0000","172.21.15.38:6832/3553953266":"2024-09-22T03:41:29.639383+0000","172.21.15.38:6834/2930805359":"2024-09-22T03:42:23.215190+0000","172.21.15.38:6832/626794495":"2024-09-22T03:43:49.180309+0000","172.21.15.38:6833/1643781179":"2024-09-22T03:43:33.596532+0000","172.21.15.38:6835/2748959970":"2024-09-22T03:41:16.565736+0000","172.21.15.38:6833/3710736169":"2024-09-22T03:40:59.454480+0000","172.21.15.38:6833/1995903506":"2024-09-22T03:43:00.406120+0000","172.21.15.38:6833/775682959":"2024-09-22T03:43:10.570678+0000","172.21.15.38:6834/2748959970":"2024-09-22T03:41:16.565736+0000","172.21.15.38:6834/2934569248":"2024-09-22T03:40:26.934591+0000","172.21.15.38:6832/3726830644":"2024-09-22T03:41:12.532341+0000","172.21.15.38:6835/2097789246":"2024-09-22T03:43:06.604471+0000","172.21.15.38:6835/2934569248":"2024-09-22T03:40:26.934591+0000","172.21.15.38:6835/3864487171":"2024-09-22T03:40:31.989837+0000","172.21.15.38:6833/4221150556":"2024-09-22T03:42:27.240627+0000","172.21.15.38:6832/121492560":"2024-09-22T03:41:10.371933+0000","172.21.15.38:6832/1717955365":"2024-09-22T03:41:51.787804+0000","172.21.15.38:6834/2418225210":"2024-09-22T03:42:02.869612+0000","172.21.15.38:6833/2851755473":"2024-09-22T03:44:25.830423+0000","172.21.15.38:6835/2418225210":"2024-09-22T03:42:02.869612+0000","172.21.15.38:6833/121492560":"2024-09-22T03:41:10.371933+0000","172.21.15.38:6835/3951934534":"2024-09-22T03:42:31.276099+0000","172.21.15.38:6833/751618334":"2024-09-22T03:40:42.053279+0000","172.21.15.195:6837/712370966":"2024-09-22T03:40:11.633787+0000","172.21.15.38:6832/3402963947":"2024-09-22T03:41:23.555005+0000","172.21.15.38:6835/2798021708":"2024-09-22T03:42:47.436632+0000","172.21.15.38:6835/3801772046":"2024-09-22T03:44:38.966412+0000","172.21.15.38:6833/3726830644":"2024-09-22T03:41:12.532341+0000","172.21.15.38:6834/2916238332":"2024-09-22T03:41:42.553770+0000","172.21.15.38:6834/3055025811":"2024-09-22T03:42:39.361236+0000","172.21.15.195:6835/712370966":"2024-09-22T03:40:11.633787+0000","172.21.15.38:6834/3864487171":"2024-09-22T03:40:31.989837+0000","172.21.15.38:6834/3572946698":"2024-09-22T03:41:44.737273+0000","172.21.15.38:6832/1995903506":"2024-09-22T03:43:00.406120+0000","172.21.15.38:6832/775682959":"2024-09-22T03:43:10.570678+0000","172.21.15.38:6835/2916238332":"2024-09-22T03:41:42.553770+0000","172.21.15.38:6833/3402963947":"2024-09-22T03:41:23.555005+0000","172.21.15.38:6834/425672974":"2024-09-22T03:40:46.105521+0000","172.21.15.38:6834/3951934534":"2024-09-22T03:42:31.276099+0000","172.21.15.38:6835/425672974":"2024-09-22T03:40:46.105521+0000","172.21.15.38:6832/3710736169":"2024-09-22T03:40:59.454480+0000","172.21.15.38:6833/1717955365":"2024-09-22T03:41:51.787804+0000","172.21.15.38:6835/3055025811":"2024-09-22T03:42:39.361236+0000","172.21.15.38:6835/2751962827":"2024-09-22T03:40:38.029580+0000","172.21.15.38:6835/2712360296":"2024-09-22T03:42:20.931115+0000","172.21.15.38:6833/3537258034":"2024-09-22T03:42:06.901189+0000","172.21.15.38:6834/3801772046":"2024-09-22T03:44:38.966412+0000","172.21.15.38:6832/3537258034":"2024-09-22T03:42:06.901189+0000","172.21.15.38:6832/1272005650":"2024-09-22T03:41:58.836862+0000","172.21.15.38:6834/2712360296":"2024-09-22T03:42:20.931115+0000","172.21.15.38:6832/751618334":"2024-09-22T03:40:42.053279+0000","172.21.15.38:6835/2930805359":"2024-09-22T03:42:23.215190+0000","172.21.15.38:6832/4221150556":"2024-09-22T03:42:27.240627+0000","172.21.15.38:6833/1272005650":"2024-09-22T03:41:58.836862+0000","172.21.15.38:6832/245538755":"2024-09-22T03:42:35.318756+0000","172.21.15.38:6833/245538755":"2024-09-22T03:42:35.318756+0000","172.21.15.38:6833/2444649344":"2024-09-22T03:44:00.664196+0000","172.21.15.38:6833/2459386631":"2024-09-22T03:42:43.395609+0000","172.21.15.38:6832/2459386631":"2024-09-22T03:42:43.395609+0000","172.21.15.38:6834/2097789246":"2024-09-22T03:43:06.604471+0000","172.21.15.38:6834/2798021708":"2024-09-22T03:42:47.436632+0000","172.21.15.38:6832/111206731":"2024-09-22T03:43:02.560071+0000","172.21.15.38:6833/111206731":"2024-09-22T03:43:02.560071+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T03:44:49.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:49.614+0000 7fb774c4d640 1 -- 172.21.15.38:0/4115839648 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb750078b90 msgr2=0x7fb75007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:49.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:49.614+0000 7fb774c4d640 1 --2- 172.21.15.38:0/4115839648 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb750078b90 0x7fb75007b0a0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fb760005fd0 tx=0x7fb7600097d0 comp rx=0 tx=0).stop 2024-09-21T03:44:49.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:49.614+0000 7fb774c4d640 1 -- 172.21.15.38:0/4115839648 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb7701f6700 msgr2=0x7fb7701f6b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:49.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:49.614+0000 7fb774c4d640 1 --2- 172.21.15.38:0/4115839648 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb7701f6700 0x7fb7701f6b10 secure :-1 s=READY pgs=2308 cs=0 l=1 rev1=1 crypto rx=0x7fb7680058c0 tx=0x7fb76800da70 comp rx=0 tx=0).stop 2024-09-21T03:44:49.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:49.614+0000 7fb774c4d640 1 -- 172.21.15.38:0/4115839648 shutdown_connections 2024-09-21T03:44:49.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:49.614+0000 7fb774c4d640 1 --2- 172.21.15.38:0/4115839648 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb7701f6700 0x7fb7701f6b10 unknown :-1 s=CLOSED pgs=2308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:49.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:49.614+0000 7fb774c4d640 1 --2- 172.21.15.38:0/4115839648 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb750078b90 0x7fb75007b0a0 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:49.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:49.614+0000 7fb774c4d640 1 --2- 172.21.15.38:0/4115839648 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb770162280 0x7fb7700ca900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:49.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:49.614+0000 7fb774c4d640 1 --2- 172.21.15.38:0/4115839648 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb7700b2e40 0x7fb770159ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:49.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:49.614+0000 7fb774c4d640 1 -- 172.21.15.38:0/4115839648 >> 172.21.15.38:0/4115839648 conn(0x7fb7700b6b20 msgr2=0x7fb7700b7b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:49.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:49.614+0000 7fb774c4d640 1 -- 172.21.15.38:0/4115839648 shutdown_connections 2024-09-21T03:44:49.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:49.614+0000 7fb774c4d640 1 -- 172.21.15.38:0/4115839648 wait complete. 2024-09-21T03:44:49.628 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T03:33:22.992573+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-21T03:44:49.628 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-21T03:44:49.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:49.745+0000 7f3268d90640 1 -- 172.21.15.38:0/3019653150 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3264156a50 msgr2=0x7f3264074620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:49.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:49.745+0000 7f3268d90640 1 --2- 172.21.15.38:0/3019653150 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3264156a50 0x7f3264074620 secure :-1 s=READY pgs=2310 cs=0 l=1 rev1=1 crypto rx=0x7f3254005fd0 tx=0x7f325402a8b0 comp rx=0 tx=0).stop 2024-09-21T03:44:49.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:49.745+0000 7f3268d90640 1 -- 172.21.15.38:0/3019653150 shutdown_connections 2024-09-21T03:44:49.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:49.745+0000 7f3268d90640 1 --2- 172.21.15.38:0/3019653150 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3264156a50 0x7f3264074620 unknown :-1 s=CLOSED pgs=2310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:49.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:49.745+0000 7f3268d90640 1 -- 172.21.15.38:0/3019653150 >> 172.21.15.38:0/3019653150 conn(0x7f32640b1b90 msgr2=0x7f32640b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:49.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:49.745+0000 7f3268d90640 1 -- 172.21.15.38:0/3019653150 shutdown_connections 2024-09-21T03:44:49.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:49.745+0000 7f3268d90640 1 -- 172.21.15.38:0/3019653150 wait complete. 2024-09-21T03:44:49.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:49.747+0000 7f3268d90640 1 Processor -- start 2024-09-21T03:44:49.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:49.747+0000 7f3268d90640 1 -- start start 2024-09-21T03:44:49.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:49.748+0000 7f3268d90640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3264074b60 0x7f32640a0460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:49.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:49.748+0000 7f3268d90640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3264156a50 0x7f32641f7e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:49.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:49.749+0000 7f3268d90640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f32640cb350 0x7f326415f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:49.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:49.749+0000 7f3268d90640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f32640cf790 con 0x7f32640cb350 2024-09-21T03:44:49.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:49.749+0000 7f3268d90640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f32640cf610 con 0x7f3264074b60 2024-09-21T03:44:49.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:49.749+0000 7f3268d90640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f32640cf910 con 0x7f3264156a50 2024-09-21T03:44:49.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:49.749+0000 7f3262d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f32640cb350 0x7f326415f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:49.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:49.749+0000 7f3262d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f32640cb350 0x7f326415f2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44194/0 (socket says 172.21.15.38:44194) 2024-09-21T03:44:49.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:49.749+0000 7f3262d9d640 1 -- 172.21.15.38:0/3746333526 learned_addr learned my addr 172.21.15.38:0/3746333526 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:44:49.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:49.749+0000 7f326259c640 1 --2- 172.21.15.38:0/3746333526 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3264074b60 0x7f32640a0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:49.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:49.749+0000 7f3261d9b640 1 --2- 172.21.15.38:0/3746333526 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3264156a50 0x7f32641f7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:49.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:49.749+0000 7f3262d9d640 1 -- 172.21.15.38:0/3746333526 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3264156a50 msgr2=0x7f32641f7e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:49.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:49.749+0000 7f3262d9d640 1 --2- 172.21.15.38:0/3746333526 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3264156a50 0x7f32641f7e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:49.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:49.749+0000 7f3262d9d640 1 -- 172.21.15.38:0/3746333526 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3264074b60 msgr2=0x7f32640a0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:49.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:49.749+0000 7f3262d9d640 1 --2- 172.21.15.38:0/3746333526 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3264074b60 0x7f32640a0460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:49.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:49.749+0000 7f3262d9d640 1 -- 172.21.15.38:0/3746333526 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32640a09a0 con 0x7f32640cb350 2024-09-21T03:44:49.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:49.749+0000 7f3262d9d640 1 --2- 172.21.15.38:0/3746333526 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f32640cb350 0x7f326415f2d0 secure :-1 s=READY pgs=2311 cs=0 l=1 rev1=1 crypto rx=0x7f325c00c3c0 tx=0x7f325c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:49.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:49.749+0000 7f325affd640 1 -- 172.21.15.38:0/3746333526 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f325c011790 con 0x7f32640cb350 2024-09-21T03:44:49.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:49.749+0000 7f325affd640 1 -- 172.21.15.38:0/3746333526 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f325c017070 con 0x7f32640cb350 2024-09-21T03:44:49.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:49.749+0000 7f3268d90640 1 -- 172.21.15.38:0/3746333526 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32640a0eb0 con 0x7f32640cb350 2024-09-21T03:44:49.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:49.749+0000 7f325affd640 1 -- 172.21.15.38:0/3746333526 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f325c011ca0 con 0x7f32640cb350 2024-09-21T03:44:49.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:49.750+0000 7f3268d90640 1 -- 172.21.15.38:0/3746333526 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32640cab50 con 0x7f32640cb350 2024-09-21T03:44:49.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:49.750+0000 7f3268d90640 1 -- 172.21.15.38:0/3746333526 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3264074620 con 0x7f32640cb350 2024-09-21T03:44:49.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:49.751+0000 7f325affd640 1 -- 172.21.15.38:0/3746333526 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f325c01c460 con 0x7f32640cb350 2024-09-21T03:44:49.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:49.752+0000 7f325affd640 1 --2- 172.21.15.38:0/3746333526 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3244078b90 0x7f324407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:49.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:49.752+0000 7f325affd640 1 -- 172.21.15.38:0/3746333526 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 8689+0+0 (secure 0 0 0) 0x7f325c09b530 con 0x7f32640cb350 2024-09-21T03:44:49.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:49.752+0000 7f326259c640 1 --2- 172.21.15.38:0/3746333526 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3244078b90 0x7f324407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:49.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:49.753+0000 7f326259c640 1 --2- 172.21.15.38:0/3746333526 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3244078b90 0x7f324407b0a0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f3254034000 tx=0x7f3254004ab0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:49.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:49.757+0000 7f325affd640 1 -- 172.21.15.38:0/3746333526 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f325c062720 con 0x7f32640cb350 2024-09-21T03:44:49.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:49.941+0000 7f3268d90640 1 -- 172.21.15.38:0/3746333526 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f3264070a20 con 0x7f32640cb350 2024-09-21T03:44:49.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:49.941+0000 7f325affd640 1 -- 172.21.15.38:0/3746333526 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v138) ==== 93+0+10 (secure 0 0 0) 0x7f325c067150 con 0x7f32640cb350 2024-09-21T03:44:49.943 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 1 2024-09-21T03:44:49.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:49.944+0000 7f3268d90640 1 -- 172.21.15.38:0/3746333526 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3244078b90 msgr2=0x7f324407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:49.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:49.944+0000 7f3268d90640 1 --2- 172.21.15.38:0/3746333526 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3244078b90 0x7f324407b0a0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f3254034000 tx=0x7f3254004ab0 comp rx=0 tx=0).stop 2024-09-21T03:44:49.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:49.944+0000 7f3268d90640 1 -- 172.21.15.38:0/3746333526 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f32640cb350 msgr2=0x7f326415f2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:49.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:49.944+0000 7f3268d90640 1 --2- 172.21.15.38:0/3746333526 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f32640cb350 0x7f326415f2d0 secure :-1 s=READY pgs=2311 cs=0 l=1 rev1=1 crypto rx=0x7f325c00c3c0 tx=0x7f325c00c880 comp rx=0 tx=0).stop 2024-09-21T03:44:49.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:49.944+0000 7f3268d90640 1 -- 172.21.15.38:0/3746333526 shutdown_connections 2024-09-21T03:44:49.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:49.944+0000 7f3268d90640 1 --2- 172.21.15.38:0/3746333526 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f32640cb350 0x7f326415f2d0 unknown :-1 s=CLOSED pgs=2311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:49.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:49.944+0000 7f3268d90640 1 --2- 172.21.15.38:0/3746333526 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3244078b90 0x7f324407b0a0 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:49.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:49.944+0000 7f3268d90640 1 --2- 172.21.15.38:0/3746333526 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3264156a50 0x7f32641f7e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:49.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:49.944+0000 7f3268d90640 1 --2- 172.21.15.38:0/3746333526 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3264074b60 0x7f32640a0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:49.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:49.944+0000 7f3268d90640 1 -- 172.21.15.38:0/3746333526 >> 172.21.15.38:0/3746333526 conn(0x7f32640b1b90 msgr2=0x7f326415be10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:49.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:49.944+0000 7f3268d90640 1 -- 172.21.15.38:0/3746333526 shutdown_connections 2024-09-21T03:44:49.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:49.944+0000 7f3268d90640 1 -- 172.21.15.38:0/3746333526 wait complete. 2024-09-21T03:44:49.957 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_damage.TestDamage.test_open_ino_errors' 2024-09-21T03:44:50.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:50.071+0000 7f15889ba640 1 -- 172.21.15.38:0/542920029 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f158415a6f0 msgr2=0x7f158415e230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:50.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:50.071+0000 7f15889ba640 1 --2- 172.21.15.38:0/542920029 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f158415a6f0 0x7f158415e230 secure :-1 s=READY pgs=2312 cs=0 l=1 rev1=1 crypto rx=0x7f1584073e10 tx=0x7f157802a570 comp rx=0 tx=0).stop 2024-09-21T03:44:50.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:50.071+0000 7f15889ba640 1 -- 172.21.15.38:0/542920029 shutdown_connections 2024-09-21T03:44:50.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:50.071+0000 7f15889ba640 1 --2- 172.21.15.38:0/542920029 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f15840d0980 0x7f15840d0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:50.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:50.071+0000 7f15889ba640 1 --2- 172.21.15.38:0/542920029 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f158415a6f0 0x7f158415e230 unknown :-1 s=CLOSED pgs=2312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:50.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:50.071+0000 7f15889ba640 1 --2- 172.21.15.38:0/542920029 >> v2:172.21.15.195:3300/0 conn(0x7f15840d01f0 0x7f158415daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:50.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:50.071+0000 7f15889ba640 1 -- 172.21.15.38:0/542920029 >> 172.21.15.38:0/542920029 conn(0x7f1584072990 msgr2=0x7f1584072da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:50.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:50.071+0000 7f15889ba640 1 -- 172.21.15.38:0/542920029 shutdown_connections 2024-09-21T03:44:50.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:50.071+0000 7f15889ba640 1 -- 172.21.15.38:0/542920029 wait complete. 2024-09-21T03:44:50.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:50.073+0000 7f15889ba640 1 Processor -- start 2024-09-21T03:44:50.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:50.073+0000 7f15889ba640 1 -- start start 2024-09-21T03:44:50.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:50.074+0000 7f15889ba640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f15840d01f0 0x7f1584145500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:50.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:50.074+0000 7f15889ba640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f15840d0980 0x7f15841f3810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:50.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:50.074+0000 7f15889ba640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f158415a6f0 0x7f15841532d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:50.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:50.074+0000 7f15889ba640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f15840ccfb0 con 0x7f15840d0980 2024-09-21T03:44:50.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:50.074+0000 7f15889ba640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f158415e980 con 0x7f158415a6f0 2024-09-21T03:44:50.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:50.074+0000 7f15889ba640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f15840cd130 con 0x7f15840d01f0 2024-09-21T03:44:50.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:50.074+0000 7f1581d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f15840d0980 0x7f15841f3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:50.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:50.075+0000 7f1581d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f15840d0980 0x7f15841f3810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44224/0 (socket says 172.21.15.38:44224) 2024-09-21T03:44:50.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:50.075+0000 7f1581d9b640 1 -- 172.21.15.38:0/3994764701 learned_addr learned my addr 172.21.15.38:0/3994764701 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:44:50.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:50.075+0000 7f158259c640 1 --2- 172.21.15.38:0/3994764701 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f15840d01f0 0x7f1584145500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:50.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:50.075+0000 7f1582d9d640 1 --2- 172.21.15.38:0/3994764701 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f158415a6f0 0x7f15841532d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:50.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:50.075+0000 7f1581d9b640 1 -- 172.21.15.38:0/3994764701 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f15840d01f0 msgr2=0x7f1584145500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:50.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:50.075+0000 7f1581d9b640 1 --2- 172.21.15.38:0/3994764701 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f15840d01f0 0x7f1584145500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:50.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:50.075+0000 7f1581d9b640 1 -- 172.21.15.38:0/3994764701 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f158415a6f0 msgr2=0x7f15841532d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:50.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:50.075+0000 7f1581d9b640 1 --2- 172.21.15.38:0/3994764701 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f158415a6f0 0x7f15841532d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:50.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:50.075+0000 7f1581d9b640 1 -- 172.21.15.38:0/3994764701 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1584147ce0 con 0x7f15840d0980 2024-09-21T03:44:50.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:50.075+0000 7f1581d9b640 1 --2- 172.21.15.38:0/3994764701 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f15840d0980 0x7f15841f3810 secure :-1 s=READY pgs=2313 cs=0 l=1 rev1=1 crypto rx=0x7f1578034040 tx=0x7f15780038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:50.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:50.075+0000 7f1572ffd640 1 -- 172.21.15.38:0/3994764701 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f157803e030 con 0x7f15840d0980 2024-09-21T03:44:50.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:50.075+0000 7f1572ffd640 1 -- 172.21.15.38:0/3994764701 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1578003ca0 con 0x7f15840d0980 2024-09-21T03:44:50.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:50.075+0000 7f15889ba640 1 -- 172.21.15.38:0/3994764701 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1584147fc0 con 0x7f15840d0980 2024-09-21T03:44:50.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:50.075+0000 7f1572ffd640 1 -- 172.21.15.38:0/3994764701 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1578035040 con 0x7f15840d0980 2024-09-21T03:44:50.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:50.075+0000 7f15889ba640 1 -- 172.21.15.38:0/3994764701 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15840c6ed0 con 0x7f15840d0980 2024-09-21T03:44:50.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:50.076+0000 7f15889ba640 1 -- 172.21.15.38:0/3994764701 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15840d0d90 con 0x7f15840d0980 2024-09-21T03:44:50.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:50.077+0000 7f1572ffd640 1 -- 172.21.15.38:0/3994764701 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f1578041030 con 0x7f15840d0980 2024-09-21T03:44:50.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:50.078+0000 7f1572ffd640 1 --2- 172.21.15.38:0/3994764701 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1564078b90 0x7f156407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:50.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:50.078+0000 7f1572ffd640 1 -- 172.21.15.38:0/3994764701 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 8689+0+0 (secure 0 0 0) 0x7f15780bbf30 con 0x7f15840d0980 2024-09-21T03:44:50.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:50.078+0000 7f158259c640 1 --2- 172.21.15.38:0/3994764701 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1564078b90 0x7f156407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:50.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:50.078+0000 7f158259c640 1 --2- 172.21.15.38:0/3994764701 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1564078b90 0x7f156407b0a0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f1574005fd0 tx=0x7f1574009500 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:50.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:50.082+0000 7f1572ffd640 1 -- 172.21.15.38:0/3994764701 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f1578083120 con 0x7f15840d0980 2024-09-21T03:44:50.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:50.345+0000 7f15889ba640 1 -- 172.21.15.38:0/3994764701 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_damage.TestDamage.test_open_ino_errors"]} v 0) -- 0x7f1584159e50 con 0x7f15840d0980 2024-09-21T03:44:51.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:51.282+0000 7f1572ffd640 1 -- 172.21.15.38:0/3994764701 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_damage.TestDamage.test_open_ino_errors"]}]=0 v586) ==== 138+0+0 (secure 0 0 0) 0x7f1578087b50 con 0x7f15840d0980 2024-09-21T03:44:51.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:51.285+0000 7f15889ba640 1 -- 172.21.15.38:0/3994764701 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1564078b90 msgr2=0x7f156407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:51.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:51.285+0000 7f15889ba640 1 --2- 172.21.15.38:0/3994764701 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1564078b90 0x7f156407b0a0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f1574005fd0 tx=0x7f1574009500 comp rx=0 tx=0).stop 2024-09-21T03:44:51.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:51.285+0000 7f15889ba640 1 -- 172.21.15.38:0/3994764701 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f15840d0980 msgr2=0x7f15841f3810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:51.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:51.285+0000 7f15889ba640 1 --2- 172.21.15.38:0/3994764701 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f15840d0980 0x7f15841f3810 secure :-1 s=READY pgs=2313 cs=0 l=1 rev1=1 crypto rx=0x7f1578034040 tx=0x7f15780038a0 comp rx=0 tx=0).stop 2024-09-21T03:44:51.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:51.285+0000 7f15889ba640 1 -- 172.21.15.38:0/3994764701 shutdown_connections 2024-09-21T03:44:51.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:51.285+0000 7f15889ba640 1 --2- 172.21.15.38:0/3994764701 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f158415a6f0 0x7f15841532d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:51.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:51.286+0000 7f15889ba640 1 --2- 172.21.15.38:0/3994764701 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f15840d0980 0x7f15841f3810 unknown :-1 s=CLOSED pgs=2313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:51.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:51.286+0000 7f15889ba640 1 --2- 172.21.15.38:0/3994764701 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1564078b90 0x7f156407b0a0 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:51.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:51.286+0000 7f15889ba640 1 --2- 172.21.15.38:0/3994764701 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f15840d01f0 0x7f1584145500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:51.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:51.286+0000 7f15889ba640 1 -- 172.21.15.38:0/3994764701 >> 172.21.15.38:0/3994764701 conn(0x7f1584072990 msgr2=0x7f1584068d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:51.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:51.286+0000 7f15889ba640 1 -- 172.21.15.38:0/3994764701 shutdown_connections 2024-09-21T03:44:51.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:51.286+0000 7f15889ba640 1 -- 172.21.15.38:0/3994764701 wait complete. 2024-09-21T03:44:51.301 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-21T03:44:51.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:51.423+0000 7f22fd99d640 1 -- 172.21.15.38:0/2291477375 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f22f806d9c0 msgr2=0x7f22f8154ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:51.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:51.423+0000 7f22fd99d640 1 --2- 172.21.15.38:0/2291477375 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f22f806d9c0 0x7f22f8154ea0 secure :-1 s=READY pgs=2314 cs=0 l=1 rev1=1 crypto rx=0x7f22f80b6de0 tx=0x7f22f002aa30 comp rx=0 tx=0).stop 2024-09-21T03:44:51.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:51.423+0000 7f22fd99d640 1 -- 172.21.15.38:0/2291477375 shutdown_connections 2024-09-21T03:44:51.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:51.423+0000 7f22fd99d640 1 --2- 172.21.15.38:0/2291477375 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f22f806d9c0 0x7f22f8154ea0 unknown :-1 s=CLOSED pgs=2314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:51.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:51.423+0000 7f22fd99d640 1 --2- 172.21.15.38:0/2291477375 >> v2:172.21.15.195:3300/0 conn(0x7f22f80b9e40 0x7f22f81622b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:51.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:51.423+0000 7f22fd99d640 1 -- 172.21.15.38:0/2291477375 >> 172.21.15.38:0/2291477375 conn(0x7f22f80d1930 msgr2=0x7f22f80d42e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:51.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:51.423+0000 7f22fd99d640 1 -- 172.21.15.38:0/2291477375 shutdown_connections 2024-09-21T03:44:51.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:51.423+0000 7f22fd99d640 1 -- 172.21.15.38:0/2291477375 wait complete. 2024-09-21T03:44:51.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:51.425+0000 7f22fd99d640 1 Processor -- start 2024-09-21T03:44:51.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:51.425+0000 7f22fd99d640 1 -- start start 2024-09-21T03:44:51.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:51.426+0000 7f22fd99d640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f22f806d9c0 0x7f22f8156ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:51.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:51.426+0000 7f22fd99d640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f22f80b9e40 0x7f22f8152f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:51.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:51.426+0000 7f22fd99d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f22f81627f0 0x7f22f80cca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:51.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:51.426+0000 7f22fd99d640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f22f8155770 con 0x7f22f81627f0 2024-09-21T03:44:51.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:51.426+0000 7f22fd99d640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f22f81555f0 con 0x7f22f80b9e40 2024-09-21T03:44:51.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:51.426+0000 7f22fd99d640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f22f81558f0 con 0x7f22f806d9c0 2024-09-21T03:44:51.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:51.427+0000 7f22fd19c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f22f81627f0 0x7f22f80cca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:51.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:51.427+0000 7f22fd19c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f22f81627f0 0x7f22f80cca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44230/0 (socket says 172.21.15.38:44230) 2024-09-21T03:44:51.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:51.427+0000 7f22fd19c640 1 -- 172.21.15.38:0/1515848682 learned_addr learned my addr 172.21.15.38:0/1515848682 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:44:51.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:51.427+0000 7f22f7fff640 1 --2- 172.21.15.38:0/1515848682 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f22f80b9e40 0x7f22f8152f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:51.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:51.427+0000 7f22fc99b640 1 --2- 172.21.15.38:0/1515848682 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f22f806d9c0 0x7f22f8156ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:51.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:51.427+0000 7f22fd19c640 1 -- 172.21.15.38:0/1515848682 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f22f806d9c0 msgr2=0x7f22f8156ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:51.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:51.427+0000 7f22fd19c640 1 --2- 172.21.15.38:0/1515848682 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f22f806d9c0 0x7f22f8156ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:51.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:51.427+0000 7f22fd19c640 1 -- 172.21.15.38:0/1515848682 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f22f80b9e40 msgr2=0x7f22f8152f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:51.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:51.427+0000 7f22fd19c640 1 --2- 172.21.15.38:0/1515848682 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f22f80b9e40 0x7f22f8152f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:51.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:51.427+0000 7f22fd19c640 1 -- 172.21.15.38:0/1515848682 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22f80b6b60 con 0x7f22f81627f0 2024-09-21T03:44:51.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:51.427+0000 7f22f7fff640 1 --2- 172.21.15.38:0/1515848682 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f22f80b9e40 0x7f22f8152f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:44:51.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:51.427+0000 7f22fd19c640 1 --2- 172.21.15.38:0/1515848682 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f22f81627f0 0x7f22f80cca10 secure :-1 s=READY pgs=2315 cs=0 l=1 rev1=1 crypto rx=0x7f22f002b020 tx=0x7f22f0003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:51.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:51.427+0000 7f22f57fa640 1 -- 172.21.15.38:0/1515848682 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22f0033050 con 0x7f22f81627f0 2024-09-21T03:44:51.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:51.427+0000 7f22fd99d640 1 -- 172.21.15.38:0/1515848682 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22f815bae0 con 0x7f22f81627f0 2024-09-21T03:44:51.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:51.428+0000 7f22fd99d640 1 -- 172.21.15.38:0/1515848682 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22f815c060 con 0x7f22f81627f0 2024-09-21T03:44:51.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:51.428+0000 7f22f57fa640 1 -- 172.21.15.38:0/1515848682 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f22f0004550 con 0x7f22f81627f0 2024-09-21T03:44:51.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:51.428+0000 7f22f57fa640 1 -- 172.21.15.38:0/1515848682 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22f0038450 con 0x7f22f81627f0 2024-09-21T03:44:51.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:51.429+0000 7f22fd99d640 1 -- 172.21.15.38:0/1515848682 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22dc005290 con 0x7f22f81627f0 2024-09-21T03:44:51.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:51.429+0000 7f22f57fa640 1 -- 172.21.15.38:0/1515848682 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f22f0036070 con 0x7f22f81627f0 2024-09-21T03:44:51.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:51.430+0000 7f22f57fa640 1 --2- 172.21.15.38:0/1515848682 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f22d8078bc0 0x7f22d807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:51.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:51.430+0000 7f22f57fa640 1 -- 172.21.15.38:0/1515848682 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 8689+0+0 (secure 0 0 0) 0x7f22f00bce40 con 0x7f22f81627f0 2024-09-21T03:44:51.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:51.430+0000 7f22fc99b640 1 --2- 172.21.15.38:0/1515848682 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f22d8078bc0 0x7f22d807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:51.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:51.431+0000 7f22fc99b640 1 --2- 172.21.15.38:0/1515848682 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f22d8078bc0 0x7f22d807b0d0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f22e0005d70 tx=0x7f22e0005d00 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:51.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:51.435+0000 7f22f57fa640 1 -- 172.21.15.38:0/1515848682 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f22f8163170 con 0x7f22f81627f0 2024-09-21T03:44:51.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:51.621+0000 7f22fd99d640 1 -- 172.21.15.38:0/1515848682 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f22dc005580 con 0x7f22f81627f0 2024-09-21T03:44:51.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:51.622+0000 7f22f57fa640 1 -- 172.21.15.38:0/1515848682 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v48) ==== 59+0+2294 (secure 0 0 0) 0x7f22f00846b0 con 0x7f22f81627f0 2024-09-21T03:44:51.624 INFO:teuthology.orchestra.run.smithi038.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-21T03:44:51.624 INFO:teuthology.orchestra.run.smithi038.stdout:mon advanced mon_op_complaint_time 120 2024-09-21T03:44:51.624 INFO:teuthology.orchestra.run.smithi038.stdout:mgr advanced client_mount_timeout 30 2024-09-21T03:44:51.624 INFO:teuthology.orchestra.run.smithi038.stdout:mgr advanced debug_client 20/20 2024-09-21T03:44:51.624 INFO:teuthology.orchestra.run.smithi038.stdout:mgr advanced debug_mgr 20/20 2024-09-21T03:44:51.624 INFO:teuthology.orchestra.run.smithi038.stdout:mgr advanced debug_ms 1/1 2024-09-21T03:44:51.624 INFO:teuthology.orchestra.run.smithi038.stdout:mgr dev mon_warn_on_pool_no_app false 2024-09-21T03:44:51.624 INFO:teuthology.orchestra.run.smithi038.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-21T03:44:51.625 INFO:teuthology.orchestra.run.smithi038.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 3844.962174 2024-09-21T03:44:51.625 INFO:teuthology.orchestra.run.smithi038.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 4916.015278 2024-09-21T03:44:51.625 INFO:teuthology.orchestra.run.smithi038.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 4165.407418 2024-09-21T03:44:51.625 INFO:teuthology.orchestra.run.smithi038.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 3760.481604 2024-09-21T03:44:51.625 INFO:teuthology.orchestra.run.smithi038.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 6052.882314 2024-09-21T03:44:51.625 INFO:teuthology.orchestra.run.smithi038.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 8967.576248 2024-09-21T03:44:51.625 INFO:teuthology.orchestra.run.smithi038.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 6129.316416 2024-09-21T03:44:51.625 INFO:teuthology.orchestra.run.smithi038.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 8751.950197 2024-09-21T03:44:51.625 INFO:teuthology.orchestra.run.smithi038.stdout:mds advanced debug_mds 20/20 2024-09-21T03:44:51.625 INFO:teuthology.orchestra.run.smithi038.stdout:mds advanced debug_mds_balancer 20/20 2024-09-21T03:44:51.625 INFO:teuthology.orchestra.run.smithi038.stdout:mds advanced debug_ms 2/2 2024-09-21T03:44:51.625 INFO:teuthology.orchestra.run.smithi038.stdout:mds dev mds_debug_frag true 2024-09-21T03:44:51.626 INFO:teuthology.orchestra.run.smithi038.stdout:mds dev mds_debug_scatterstat true 2024-09-21T03:44:51.626 INFO:teuthology.orchestra.run.smithi038.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-21T03:44:51.626 INFO:teuthology.orchestra.run.smithi038.stdout:mds dev mds_verify_scatter true 2024-09-21T03:44:51.626 INFO:teuthology.orchestra.run.smithi038.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-21T03:44:51.626 INFO:teuthology.orchestra.run.smithi038.stdout:mds advanced rados_mon_op_timeout 900 2024-09-21T03:44:51.626 INFO:teuthology.orchestra.run.smithi038.stdout:mds advanced rados_osd_op_timeout 900 2024-09-21T03:44:51.626 INFO:teuthology.orchestra.run.smithi038.stdout:client advanced client_mount_timeout 600 2024-09-21T03:44:51.626 INFO:teuthology.orchestra.run.smithi038.stdout:client advanced debug_client 20/20 2024-09-21T03:44:51.626 INFO:teuthology.orchestra.run.smithi038.stdout:client advanced debug_ms 1/1 2024-09-21T03:44:51.626 INFO:teuthology.orchestra.run.smithi038.stdout:client advanced rados_mon_op_timeout 900 2024-09-21T03:44:51.626 INFO:teuthology.orchestra.run.smithi038.stdout:client advanced rados_osd_op_timeout 900 2024-09-21T03:44:51.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:51.624+0000 7f22fd99d640 1 -- 172.21.15.38:0/1515848682 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f22d8078bc0 msgr2=0x7f22d807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:51.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:51.624+0000 7f22fd99d640 1 --2- 172.21.15.38:0/1515848682 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f22d8078bc0 0x7f22d807b0d0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f22e0005d70 tx=0x7f22e0005d00 comp rx=0 tx=0).stop 2024-09-21T03:44:51.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:51.624+0000 7f22fd99d640 1 -- 172.21.15.38:0/1515848682 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f22f81627f0 msgr2=0x7f22f80cca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:51.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:51.624+0000 7f22fd99d640 1 --2- 172.21.15.38:0/1515848682 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f22f81627f0 0x7f22f80cca10 secure :-1 s=READY pgs=2315 cs=0 l=1 rev1=1 crypto rx=0x7f22f002b020 tx=0x7f22f0003f10 comp rx=0 tx=0).stop 2024-09-21T03:44:51.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:51.624+0000 7f22fd99d640 1 -- 172.21.15.38:0/1515848682 shutdown_connections 2024-09-21T03:44:51.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:51.624+0000 7f22fd99d640 1 --2- 172.21.15.38:0/1515848682 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f22f81627f0 0x7f22f80cca10 unknown :-1 s=CLOSED pgs=2315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:51.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:51.624+0000 7f22fd99d640 1 --2- 172.21.15.38:0/1515848682 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f22f80b9e40 0x7f22f8152f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:51.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:51.624+0000 7f22fd99d640 1 --2- 172.21.15.38:0/1515848682 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f22d8078bc0 0x7f22d807b0d0 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:51.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:51.624+0000 7f22fd99d640 1 --2- 172.21.15.38:0/1515848682 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f22f806d9c0 0x7f22f8156ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:51.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:51.624+0000 7f22fd99d640 1 -- 172.21.15.38:0/1515848682 >> 172.21.15.38:0/1515848682 conn(0x7f22f80d1930 msgr2=0x7f22f815fd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:51.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:51.624+0000 7f22fd99d640 1 -- 172.21.15.38:0/1515848682 shutdown_connections 2024-09-21T03:44:51.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:51.624+0000 7f22fd99d640 1 -- 172.21.15.38:0/1515848682 wait complete. 2024-09-21T03:44:51.637 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-21T03:44:51.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:51.755+0000 7f8f398bd640 1 -- 172.21.15.38:0/1891800176 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8f340ceb90 msgr2=0x7f8f340cefa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:51.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:51.755+0000 7f8f398bd640 1 --2- 172.21.15.38:0/1891800176 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8f340ceb90 0x7f8f340cefa0 secure :-1 s=READY pgs=2317 cs=0 l=1 rev1=1 crypto rx=0x7f8f340cfba0 tx=0x7f8f2c02a790 comp rx=0 tx=0).stop 2024-09-21T03:44:51.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:51.756+0000 7f8f398bd640 1 -- 172.21.15.38:0/1891800176 shutdown_connections 2024-09-21T03:44:51.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:51.756+0000 7f8f398bd640 1 --2- 172.21.15.38:0/1891800176 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8f340ceb90 0x7f8f340cefa0 unknown :-1 s=CLOSED pgs=2317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:51.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:51.756+0000 7f8f398bd640 1 --2- 172.21.15.38:0/1891800176 >> v2:172.21.15.195:3300/0 conn(0x7f8f3415ee80 0x7f8f340d0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:51.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:51.756+0000 7f8f398bd640 1 -- 172.21.15.38:0/1891800176 >> 172.21.15.38:0/1891800176 conn(0x7f8f3409d830 msgr2=0x7f8f340b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:51.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:51.756+0000 7f8f398bd640 1 -- 172.21.15.38:0/1891800176 shutdown_connections 2024-09-21T03:44:51.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:51.756+0000 7f8f398bd640 1 -- 172.21.15.38:0/1891800176 wait complete. 2024-09-21T03:44:51.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:51.757+0000 7f8f398bd640 1 Processor -- start 2024-09-21T03:44:51.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:51.758+0000 7f8f398bd640 1 -- start start 2024-09-21T03:44:51.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:51.759+0000 7f8f398bd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8f340b9ad0 0x7f8f341f5930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:51.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:51.759+0000 7f8f398bd640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8f340ceb90 0x7f8f341562f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:51.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:51.759+0000 7f8f398bd640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8f3415ee80 0x7f8f34156aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:51.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:51.759+0000 7f8f398bd640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f8f340b6580 con 0x7f8f340b9ad0 2024-09-21T03:44:51.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:51.759+0000 7f8f398bd640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f8f340ba0f0 con 0x7f8f3415ee80 2024-09-21T03:44:51.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:51.759+0000 7f8f398bd640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f8f340b6700 con 0x7f8f340ceb90 2024-09-21T03:44:51.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:51.759+0000 7f8f32ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8f340b9ad0 0x7f8f341f5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:51.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:51.759+0000 7f8f32ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8f340b9ad0 0x7f8f341f5930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44238/0 (socket says 172.21.15.38:44238) 2024-09-21T03:44:51.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:51.759+0000 7f8f32ffd640 1 -- 172.21.15.38:0/2632521426 learned_addr learned my addr 172.21.15.38:0/2632521426 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:44:51.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:51.759+0000 7f8f337fe640 1 --2- 172.21.15.38:0/2632521426 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8f3415ee80 0x7f8f34156aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:51.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:51.759+0000 7f8f32ffd640 1 -- 172.21.15.38:0/2632521426 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8f340ceb90 msgr2=0x7f8f341562f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:51.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:51.759+0000 7f8f327fc640 1 --2- 172.21.15.38:0/2632521426 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8f340ceb90 0x7f8f341562f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:51.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:51.759+0000 7f8f32ffd640 1 --2- 172.21.15.38:0/2632521426 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8f340ceb90 0x7f8f341562f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:51.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:51.759+0000 7f8f32ffd640 1 -- 172.21.15.38:0/2632521426 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8f3415ee80 msgr2=0x7f8f34156aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:51.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:51.759+0000 7f8f32ffd640 1 --2- 172.21.15.38:0/2632521426 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8f3415ee80 0x7f8f34156aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:51.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:51.759+0000 7f8f32ffd640 1 -- 172.21.15.38:0/2632521426 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f341f5e70 con 0x7f8f340b9ad0 2024-09-21T03:44:51.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:51.759+0000 7f8f337fe640 1 --2- 172.21.15.38:0/2632521426 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8f3415ee80 0x7f8f34156aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:44:51.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:51.759+0000 7f8f327fc640 1 --2- 172.21.15.38:0/2632521426 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8f340ceb90 0x7f8f341562f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:44:51.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:51.760+0000 7f8f32ffd640 1 --2- 172.21.15.38:0/2632521426 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8f340b9ad0 0x7f8f341f5930 secure :-1 s=READY pgs=2318 cs=0 l=1 rev1=1 crypto rx=0x7f8f2400c9a0 tx=0x7f8f2400ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:51.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:51.760+0000 7f8f1bfff640 1 -- 172.21.15.38:0/2632521426 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f24007bb0 con 0x7f8f340b9ad0 2024-09-21T03:44:51.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:51.760+0000 7f8f1bfff640 1 -- 172.21.15.38:0/2632521426 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8f24007e40 con 0x7f8f340b9ad0 2024-09-21T03:44:51.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:51.760+0000 7f8f398bd640 1 -- 172.21.15.38:0/2632521426 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f340b27a0 con 0x7f8f340b9ad0 2024-09-21T03:44:51.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:51.760+0000 7f8f1bfff640 1 -- 172.21.15.38:0/2632521426 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f2400f690 con 0x7f8f340b9ad0 2024-09-21T03:44:51.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:51.760+0000 7f8f398bd640 1 -- 172.21.15.38:0/2632521426 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f341f5130 con 0x7f8f340b9ad0 2024-09-21T03:44:51.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:51.761+0000 7f8f398bd640 1 -- 172.21.15.38:0/2632521426 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f340b6bf0 con 0x7f8f340b9ad0 2024-09-21T03:44:51.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:51.762+0000 7f8f1bfff640 1 -- 172.21.15.38:0/2632521426 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f8f2400f830 con 0x7f8f340b9ad0 2024-09-21T03:44:51.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:51.762+0000 7f8f1bfff640 1 --2- 172.21.15.38:0/2632521426 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8f20078b90 0x7f8f2007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:51.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:51.762+0000 7f8f1bfff640 1 -- 172.21.15.38:0/2632521426 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 8689+0+0 (secure 0 0 0) 0x7f8f2409c110 con 0x7f8f340b9ad0 2024-09-21T03:44:51.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:51.762+0000 7f8f327fc640 1 --2- 172.21.15.38:0/2632521426 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8f20078b90 0x7f8f2007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:51.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:51.763+0000 7f8f327fc640 1 --2- 172.21.15.38:0/2632521426 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8f20078b90 0x7f8f2007b0a0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f8f34157fb0 tx=0x7f8f28009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:51.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:51.767+0000 7f8f1bfff640 1 -- 172.21.15.38:0/2632521426 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f8f24014030 con 0x7f8f340b9ad0 2024-09-21T03:44:51.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:51.957+0000 7f8f398bd640 1 -- 172.21.15.38:0/2632521426 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f8f34156830 con 0x7f8f340b9ad0 2024-09-21T03:44:51.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:51.961+0000 7f8f1bfff640 1 -- 172.21.15.38:0/2632521426 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v49)=0 v49) ==== 122+0+0 (secure 0 0 0) 0x7f8f240639e0 con 0x7f8f340b9ad0 2024-09-21T03:44:51.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:51.963+0000 7f8f398bd640 1 -- 172.21.15.38:0/2632521426 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8f20078b90 msgr2=0x7f8f2007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:51.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:51.963+0000 7f8f398bd640 1 --2- 172.21.15.38:0/2632521426 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8f20078b90 0x7f8f2007b0a0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f8f34157fb0 tx=0x7f8f28009040 comp rx=0 tx=0).stop 2024-09-21T03:44:51.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:51.963+0000 7f8f398bd640 1 -- 172.21.15.38:0/2632521426 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8f340b9ad0 msgr2=0x7f8f341f5930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:51.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:51.963+0000 7f8f398bd640 1 --2- 172.21.15.38:0/2632521426 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8f340b9ad0 0x7f8f341f5930 secure :-1 s=READY pgs=2318 cs=0 l=1 rev1=1 crypto rx=0x7f8f2400c9a0 tx=0x7f8f2400ce60 comp rx=0 tx=0).stop 2024-09-21T03:44:51.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:51.963+0000 7f8f398bd640 1 -- 172.21.15.38:0/2632521426 shutdown_connections 2024-09-21T03:44:51.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:51.963+0000 7f8f398bd640 1 --2- 172.21.15.38:0/2632521426 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8f3415ee80 0x7f8f34156aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:51.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:51.963+0000 7f8f398bd640 1 --2- 172.21.15.38:0/2632521426 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8f20078b90 0x7f8f2007b0a0 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:51.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:51.963+0000 7f8f398bd640 1 --2- 172.21.15.38:0/2632521426 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8f340ceb90 0x7f8f341562f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:51.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:51.963+0000 7f8f398bd640 1 --2- 172.21.15.38:0/2632521426 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8f340b9ad0 0x7f8f341f5930 unknown :-1 s=CLOSED pgs=2318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:51.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:51.963+0000 7f8f398bd640 1 -- 172.21.15.38:0/2632521426 >> 172.21.15.38:0/2632521426 conn(0x7f8f3409d830 msgr2=0x7f8f3415c9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:51.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:51.964+0000 7f8f398bd640 1 -- 172.21.15.38:0/2632521426 shutdown_connections 2024-09-21T03:44:51.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:51.964+0000 7f8f398bd640 1 -- 172.21.15.38:0/2632521426 wait complete. 2024-09-21T03:44:51.977 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-21T03:44:51.978 DEBUG:teuthology.orchestra.run.smithi038:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T03:44:52.015 DEBUG:teuthology.orchestra.run.smithi195:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T03:44:52.073 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-21T03:44:52.074 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-21T03:44:52.074 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:44:52.090 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056795,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2354 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T03:44:52.090 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-09-21T03:44:52.090 DEBUG:teuthology.orchestra.run.smithi195:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:44:52.107 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056014,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497444k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=8978 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248720k,nr_inodes=812180,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497444k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T03:44:52.108 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-21T03:44:52.108 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:44:52.146 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056795,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2354 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T03:44:52.146 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-09-21T03:44:52.146 DEBUG:teuthology.orchestra.run.smithi195:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:44:52.164 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056014,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497444k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=8978 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248720k,nr_inodes=812180,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497444k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T03:44:52.164 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-21T03:44:52.165 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:44:52.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:52.303+0000 7fc894ac5640 1 -- 172.21.15.38:0/3633317232 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc87c004210 msgr2=0x7fc87c004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:52.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:52.303+0000 7fc894ac5640 1 --2- 172.21.15.38:0/3633317232 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc87c004210 0x7fc87c004800 secure :-1 s=READY pgs=2320 cs=0 l=1 rev1=1 crypto rx=0x7fc878009b20 tx=0x7fc87c005830 comp rx=0 tx=0).stop 2024-09-21T03:44:52.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:52.303+0000 7fc88ed9d640 1 -- 172.21.15.38:0/3633317232 reap_dead start 2024-09-21T03:44:52.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:52.303+0000 7fc894ac5640 1 -- 172.21.15.38:0/3633317232 shutdown_connections 2024-09-21T03:44:52.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:52.303+0000 7fc894ac5640 1 -- 172.21.15.38:0/3633317232 >> 172.21.15.38:0/3633317232 conn(0x7fc8900b6b20 msgr2=0x7fc8900d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:52.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:52.303+0000 7fc894ac5640 1 -- 172.21.15.38:0/3633317232 shutdown_connections 2024-09-21T03:44:52.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:52.303+0000 7fc894ac5640 1 -- 172.21.15.38:0/3633317232 wait complete. 2024-09-21T03:44:52.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:52.305+0000 7fc894ac5640 1 Processor -- start 2024-09-21T03:44:52.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:52.305+0000 7fc894ac5640 1 -- start start 2024-09-21T03:44:52.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:52.306+0000 7fc894ac5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc87c004210 0x7fc8900ce940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:52.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:52.306+0000 7fc894ac5640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc8900cee80 0x7fc8901f4b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:52.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:52.306+0000 7fc894ac5640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc8901f5070 0x7fc8901f8170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:52.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:52.307+0000 7fc894ac5640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fc890162810 con 0x7fc87c004210 2024-09-21T03:44:52.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:52.307+0000 7fc894ac5640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fc890162690 con 0x7fc8900cee80 2024-09-21T03:44:52.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:52.307+0000 7fc894ac5640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fc890162990 con 0x7fc8901f5070 2024-09-21T03:44:52.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:52.307+0000 7fc88e59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc87c004210 0x7fc8900ce940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:52.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:52.307+0000 7fc88e59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc87c004210 0x7fc8900ce940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40334/0 (socket says 172.21.15.38:40334) 2024-09-21T03:44:52.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:52.307+0000 7fc88e59c640 1 -- 172.21.15.38:0/3601939358 learned_addr learned my addr 172.21.15.38:0/3601939358 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:44:52.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:52.307+0000 7fc887fff640 1 --2- 172.21.15.38:0/3601939358 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc8900cee80 0x7fc8901f4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:52.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:52.307+0000 7fc88ed9d640 1 --2- 172.21.15.38:0/3601939358 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc8901f5070 0x7fc8901f8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:52.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:52.307+0000 7fc88e59c640 1 -- 172.21.15.38:0/3601939358 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc8901f5070 msgr2=0x7fc8901f8170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:52.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:52.307+0000 7fc88e59c640 1 --2- 172.21.15.38:0/3601939358 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc8901f5070 0x7fc8901f8170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:52.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:52.307+0000 7fc88e59c640 1 -- 172.21.15.38:0/3601939358 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc8900cee80 msgr2=0x7fc8901f4b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:52.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:52.307+0000 7fc88e59c640 1 --2- 172.21.15.38:0/3601939358 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc8900cee80 0x7fc8901f4b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:52.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:52.307+0000 7fc88e59c640 1 -- 172.21.15.38:0/3601939358 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8901f5a60 con 0x7fc87c004210 2024-09-21T03:44:52.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:52.307+0000 7fc88e59c640 1 --2- 172.21.15.38:0/3601939358 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc87c004210 0x7fc8900ce940 secure :-1 s=READY pgs=2321 cs=0 l=1 rev1=1 crypto rx=0x7fc880003e90 tx=0x7fc880001800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:52.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:52.307+0000 7fc886ffd640 1 -- 172.21.15.38:0/3601939358 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc880015070 con 0x7fc87c004210 2024-09-21T03:44:52.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:52.307+0000 7fc886ffd640 1 -- 172.21.15.38:0/3601939358 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc880011900 con 0x7fc87c004210 2024-09-21T03:44:52.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:52.307+0000 7fc894ac5640 1 -- 172.21.15.38:0/3601939358 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc89014c5f0 con 0x7fc87c004210 2024-09-21T03:44:52.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:52.307+0000 7fc886ffd640 1 -- 172.21.15.38:0/3601939358 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc880011cd0 con 0x7fc87c004210 2024-09-21T03:44:52.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:52.307+0000 7fc894ac5640 1 -- 172.21.15.38:0/3601939358 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8900abeb0 con 0x7fc87c004210 2024-09-21T03:44:52.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:52.309+0000 7fc894ac5640 1 -- 172.21.15.38:0/3601939358 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc890162f00 con 0x7fc87c004210 2024-09-21T03:44:52.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:52.309+0000 7fc886ffd640 1 -- 172.21.15.38:0/3601939358 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc880018070 con 0x7fc87c004210 2024-09-21T03:44:52.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:52.310+0000 7fc886ffd640 1 --2- 172.21.15.38:0/3601939358 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc870078b90 0x7fc87007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:52.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:52.310+0000 7fc886ffd640 1 -- 172.21.15.38:0/3601939358 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 8689+0+0 (secure 0 0 0) 0x7fc88009c5b0 con 0x7fc87c004210 2024-09-21T03:44:52.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:52.310+0000 7fc887fff640 1 --2- 172.21.15.38:0/3601939358 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc870078b90 0x7fc87007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:52.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:52.311+0000 7fc887fff640 1 --2- 172.21.15.38:0/3601939358 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc870078b90 0x7fc87007b0a0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fc878035000 tx=0x7fc8780046c0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:52.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:52.315+0000 7fc886ffd640 1 -- 172.21.15.38:0/3601939358 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fc8800637b0 con 0x7fc87c004210 2024-09-21T03:44:52.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:52.539+0000 7fc894ac5640 1 -- 172.21.15.38:0/3601939358 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc89014c7c0 con 0x7fc87c004210 2024-09-21T03:44:52.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:52.540+0000 7fc886ffd640 1 -- 172.21.15.38:0/3601939358 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 336 v336) ==== 95+0+3913 (secure 0 0 0) 0x7fc8800681e0 con 0x7fc87c004210 2024-09-21T03:44:52.543 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:44:52.543 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":336,"btime":"2024-09-21T03:44:46:240929+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6086,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/965972663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":965972663},{"type":"v1","addr":"172.21.15.195:6836","nonce":965972663}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6700,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/3176122334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":3176122334},{"type":"v1","addr":"172.21.15.195:6837","nonce":3176122334}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":10551,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/791475497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":791475497},{"type":"v1","addr":"172.21.15.38:6833","nonce":791475497}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":330},{"gid":10602,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/434821886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":434821886},{"type":"v1","addr":"172.21.15.38:6835","nonce":434821886}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":334}],"filesystems":[]} 2024-09-21T03:44:52.543 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 336 2024-09-21T03:44:52.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:52.542+0000 7fc894ac5640 1 -- 172.21.15.38:0/3601939358 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc870078b90 msgr2=0x7fc87007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:52.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:52.542+0000 7fc894ac5640 1 --2- 172.21.15.38:0/3601939358 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc870078b90 0x7fc87007b0a0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fc878035000 tx=0x7fc8780046c0 comp rx=0 tx=0).stop 2024-09-21T03:44:52.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:52.542+0000 7fc894ac5640 1 -- 172.21.15.38:0/3601939358 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc87c004210 msgr2=0x7fc8900ce940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:52.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:52.542+0000 7fc894ac5640 1 --2- 172.21.15.38:0/3601939358 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc87c004210 0x7fc8900ce940 secure :-1 s=READY pgs=2321 cs=0 l=1 rev1=1 crypto rx=0x7fc880003e90 tx=0x7fc880001800 comp rx=0 tx=0).stop 2024-09-21T03:44:52.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:52.542+0000 7fc894ac5640 1 -- 172.21.15.38:0/3601939358 shutdown_connections 2024-09-21T03:44:52.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:52.542+0000 7fc894ac5640 1 --2- 172.21.15.38:0/3601939358 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc870078b90 0x7fc87007b0a0 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:52.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:52.542+0000 7fc894ac5640 1 --2- 172.21.15.38:0/3601939358 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc8901f5070 0x7fc8901f8170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:52.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:52.542+0000 7fc894ac5640 1 --2- 172.21.15.38:0/3601939358 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc8900cee80 0x7fc8901f4b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:52.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:52.542+0000 7fc894ac5640 1 --2- 172.21.15.38:0/3601939358 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc87c004210 0x7fc8900ce940 unknown :-1 s=CLOSED pgs=2321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:52.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:52.542+0000 7fc894ac5640 1 -- 172.21.15.38:0/3601939358 >> 172.21.15.38:0/3601939358 conn(0x7fc8900b6b20 msgr2=0x7fc89006e640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:52.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:52.542+0000 7fc894ac5640 1 -- 172.21.15.38:0/3601939358 shutdown_connections 2024-09-21T03:44:52.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:52.542+0000 7fc894ac5640 1 -- 172.21.15.38:0/3601939358 wait complete. 2024-09-21T03:44:52.556 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-21T03:44:52.556 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-21T03:44:52.557 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-21T03:44:52.557 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T03:44:52.557 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-21T03:44:52.557 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-21T03:44:52.557 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-21T03:44:52.557 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T03:44:52.557 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-21T03:44:52.558 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-21T03:44:52.558 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-21T03:44:52.558 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T03:44:52.558 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-21T03:44:52.558 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-21T03:44:52.558 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-21T03:44:52.559 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T03:44:52.571 INFO:tasks.ceph.mds.a:Stopped 2024-09-21T03:44:52.571 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-21T03:44:52.573 INFO:tasks.ceph.mds.d:Stopped 2024-09-21T03:44:52.573 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-21T03:44:52.573 INFO:tasks.ceph.mds.c:Stopped 2024-09-21T03:44:52.573 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-21T03:44:52.574 INFO:tasks.ceph.mds.b:Stopped 2024-09-21T03:44:52.574 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-21T03:44:52.615 INFO:tasks.ceph.mds.a:Started 2024-09-21T03:44:52.616 DEBUG:teuthology.parallel:result is None 2024-09-21T03:44:52.619 INFO:tasks.ceph.mds.d:Started 2024-09-21T03:44:52.619 INFO:tasks.ceph.mds.c:Started 2024-09-21T03:44:52.620 DEBUG:teuthology.parallel:result is None 2024-09-21T03:44:52.620 DEBUG:teuthology.parallel:result is None 2024-09-21T03:44:52.622 INFO:tasks.ceph.mds.b:Started 2024-09-21T03:44:52.622 DEBUG:teuthology.parallel:result is None 2024-09-21T03:44:52.623 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-21T03:44:52.704 INFO:tasks.ceph.mds.c.smithi038.stdout:starting mds.c at 2024-09-21T03:44:52.705 INFO:tasks.ceph.mds.a.smithi038.stdout:starting mds.a at 2024-09-21T03:44:52.715 INFO:tasks.ceph.mds.d.smithi195.stdout:starting mds.d at 2024-09-21T03:44:52.717 INFO:tasks.ceph.mds.b.smithi195.stdout:starting mds.b at 2024-09-21T03:44:52.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:52.747+0000 7f120bd52640 1 -- 172.21.15.38:0/410112536 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f11ec003c10 msgr2=0x7f11ec004080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:52.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:52.747+0000 7f120bd52640 1 --2- 172.21.15.38:0/410112536 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f11ec003c10 0x7f11ec004080 secure :-1 s=READY pgs=2329 cs=0 l=1 rev1=1 crypto rx=0x7f11f400c740 tx=0x7f11ec005830 comp rx=0 tx=0).stop 2024-09-21T03:44:52.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:52.747+0000 7f120a2ef640 1 -- 172.21.15.38:0/410112536 reap_dead start 2024-09-21T03:44:52.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:52.747+0000 7f120bd52640 1 -- 172.21.15.38:0/410112536 shutdown_connections 2024-09-21T03:44:52.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:52.747+0000 7f120bd52640 1 -- 172.21.15.38:0/410112536 >> 172.21.15.38:0/410112536 conn(0x7f12040b46e0 msgr2=0x7f12040d23e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:52.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:52.747+0000 7f120bd52640 1 -- 172.21.15.38:0/410112536 shutdown_connections 2024-09-21T03:44:52.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:52.747+0000 7f120bd52640 1 -- 172.21.15.38:0/410112536 wait complete. 2024-09-21T03:44:52.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:52.749+0000 7f120bd52640 1 Processor -- start 2024-09-21T03:44:52.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:52.749+0000 7f120bd52640 1 -- start start 2024-09-21T03:44:52.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:52.750+0000 7f120bd52640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f11ec003c10 0x7f12041f6f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:52.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:52.750+0000 7f120bd52640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f120414fa70 0x7f12040c68a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:52.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:52.750+0000 7f120bd52640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f120414fe80 0x7f12040c6de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:52.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:52.750+0000 7f120bd52640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f12040b62f0 con 0x7f120414fe80 2024-09-21T03:44:52.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:52.750+0000 7f120bd52640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f12040d09e0 con 0x7f120414fa70 2024-09-21T03:44:52.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:52.750+0000 7f120bd52640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f12040b6470 con 0x7f11ec003c10 2024-09-21T03:44:52.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:52.750+0000 7f120a2ef640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f120414fe80 0x7f12040c6de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:52.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:52.750+0000 7f120a2ef640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f120414fe80 0x7f12040c6de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40390/0 (socket says 172.21.15.38:40390) 2024-09-21T03:44:52.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:52.750+0000 7f120a2ef640 1 -- 172.21.15.38:0/739698474 learned_addr learned my addr 172.21.15.38:0/739698474 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:44:52.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:52.750+0000 7f1209aee640 1 --2- 172.21.15.38:0/739698474 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f11ec003c10 0x7f12041f6f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:52.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:52.750+0000 7f12092ed640 1 --2- 172.21.15.38:0/739698474 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f120414fa70 0x7f12040c68a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:52.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:52.750+0000 7f120a2ef640 1 -- 172.21.15.38:0/739698474 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f11ec003c10 msgr2=0x7f12041f6f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:52.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:52.750+0000 7f120a2ef640 1 --2- 172.21.15.38:0/739698474 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f11ec003c10 0x7f12041f6f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:52.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:52.750+0000 7f120a2ef640 1 -- 172.21.15.38:0/739698474 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f120414fa70 msgr2=0x7f12040c68a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:52.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:52.750+0000 7f120a2ef640 1 --2- 172.21.15.38:0/739698474 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f120414fa70 0x7f12040c68a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:52.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:52.750+0000 7f120a2ef640 1 -- 172.21.15.38:0/739698474 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12041567b0 con 0x7f120414fe80 2024-09-21T03:44:52.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:52.751+0000 7f120a2ef640 1 --2- 172.21.15.38:0/739698474 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f120414fe80 0x7f12040c6de0 secure :-1 s=READY pgs=2330 cs=0 l=1 rev1=1 crypto rx=0x7f1204072110 tx=0x7f11fc001800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:52.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:52.751+0000 7f12027fc640 1 -- 172.21.15.38:0/739698474 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11fc011630 con 0x7f120414fe80 2024-09-21T03:44:52.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:52.751+0000 7f12027fc640 1 -- 172.21.15.38:0/739698474 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f11fc018070 con 0x7f120414fe80 2024-09-21T03:44:52.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:52.751+0000 7f120bd52640 1 -- 172.21.15.38:0/739698474 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12041f7470 con 0x7f120414fe80 2024-09-21T03:44:52.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:52.751+0000 7f12027fc640 1 -- 172.21.15.38:0/739698474 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11fc011a50 con 0x7f120414fe80 2024-09-21T03:44:52.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:52.751+0000 7f120bd52640 1 -- 172.21.15.38:0/739698474 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12041f3460 con 0x7f120414fe80 2024-09-21T03:44:52.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:52.752+0000 7f120bd52640 1 -- 172.21.15.38:0/739698474 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12041f31d0 con 0x7f120414fe80 2024-09-21T03:44:52.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:52.753+0000 7f12027fc640 1 -- 172.21.15.38:0/739698474 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f11fc011bf0 con 0x7f120414fe80 2024-09-21T03:44:52.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:52.753+0000 7f12027fc640 1 --2- 172.21.15.38:0/739698474 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f11f0078b90 0x7f11f007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:52.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:52.753+0000 7f12027fc640 1 -- 172.21.15.38:0/739698474 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 8689+0+0 (secure 0 0 0) 0x7f11fc09be30 con 0x7f120414fe80 2024-09-21T03:44:52.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:52.753+0000 7f1209aee640 1 --2- 172.21.15.38:0/739698474 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f11f0078b90 0x7f11f007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:52.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:52.754+0000 7f1209aee640 1 --2- 172.21.15.38:0/739698474 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f11f0078b90 0x7f11f007b0a0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f11f4033000 tx=0x7f11f402d070 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:52.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:52.758+0000 7f12027fc640 1 -- 172.21.15.38:0/739698474 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f12041f7470 con 0x7f120414fe80 2024-09-21T03:44:52.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:52.958+0000 7f120bd52640 1 -- 172.21.15.38:0/739698474 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f12040d2af0 con 0x7f120414fe80 2024-09-21T03:44:53.301 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:53.299+0000 7f12027fc640 1 -- 172.21.15.38:0/739698474 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v140) ==== 97+0+0 (secure 0 0 0) 0x7f12041567b0 con 0x7f120414fe80 2024-09-21T03:44:53.301 INFO:teuthology.orchestra.run.smithi038.stderr: removed all blocklist entries 2024-09-21T03:44:53.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:53.302+0000 7f120bd52640 1 -- 172.21.15.38:0/739698474 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f11f0078b90 msgr2=0x7f11f007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:53.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:53.302+0000 7f120bd52640 1 --2- 172.21.15.38:0/739698474 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f11f0078b90 0x7f11f007b0a0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f11f4033000 tx=0x7f11f402d070 comp rx=0 tx=0).stop 2024-09-21T03:44:53.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:53.302+0000 7f120bd52640 1 -- 172.21.15.38:0/739698474 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f120414fe80 msgr2=0x7f12040c6de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:53.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:53.302+0000 7f120bd52640 1 --2- 172.21.15.38:0/739698474 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f120414fe80 0x7f12040c6de0 secure :-1 s=READY pgs=2330 cs=0 l=1 rev1=1 crypto rx=0x7f1204072110 tx=0x7f11fc001800 comp rx=0 tx=0).stop 2024-09-21T03:44:53.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:53.302+0000 7f120bd52640 1 -- 172.21.15.38:0/739698474 shutdown_connections 2024-09-21T03:44:53.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:53.302+0000 7f120bd52640 1 --2- 172.21.15.38:0/739698474 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f120414fe80 0x7f12040c6de0 unknown :-1 s=CLOSED pgs=2330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:53.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:53.302+0000 7f120bd52640 1 --2- 172.21.15.38:0/739698474 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f120414fa70 0x7f12040c68a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:53.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:53.302+0000 7f120bd52640 1 --2- 172.21.15.38:0/739698474 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f11f0078b90 0x7f11f007b0a0 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:53.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:53.302+0000 7f120bd52640 1 --2- 172.21.15.38:0/739698474 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f11ec003c10 0x7f12041f6f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:53.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:53.302+0000 7f120bd52640 1 -- 172.21.15.38:0/739698474 >> 172.21.15.38:0/739698474 conn(0x7f12040b46e0 msgr2=0x7f12040714f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:53.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:53.302+0000 7f120bd52640 1 -- 172.21.15.38:0/739698474 shutdown_connections 2024-09-21T03:44:53.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:53.302+0000 7f120bd52640 1 -- 172.21.15.38:0/739698474 wait complete. 2024-09-21T03:44:53.317 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-21T03:44:53.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:53.433+0000 7fc3b458b640 1 -- 172.21.15.38:0/732687800 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc3ac15f4c0 msgr2=0x7fc3ac0b2e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:53.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:53.433+0000 7fc3b458b640 1 --2- 172.21.15.38:0/732687800 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc3ac15f4c0 0x7fc3ac0b2e40 secure :-1 s=READY pgs=2332 cs=0 l=1 rev1=1 crypto rx=0x7fc3a4005fd0 tx=0x7fc3a402a8b0 comp rx=0 tx=0).stop 2024-09-21T03:44:53.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:53.433+0000 7fc3b458b640 1 -- 172.21.15.38:0/732687800 shutdown_connections 2024-09-21T03:44:53.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:53.433+0000 7fc3b458b640 1 --2- 172.21.15.38:0/732687800 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc3ac15f4c0 0x7fc3ac0b2e40 unknown :-1 s=CLOSED pgs=2332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:53.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:53.433+0000 7fc3b458b640 1 -- 172.21.15.38:0/732687800 >> 172.21.15.38:0/732687800 conn(0x7fc3ac0b6b20 msgr2=0x7fc3ac0d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:53.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:53.433+0000 7fc3b458b640 1 -- 172.21.15.38:0/732687800 shutdown_connections 2024-09-21T03:44:53.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:53.433+0000 7fc3b458b640 1 -- 172.21.15.38:0/732687800 wait complete. 2024-09-21T03:44:53.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:53.435+0000 7fc3b458b640 1 Processor -- start 2024-09-21T03:44:53.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:53.435+0000 7fc3b458b640 1 -- start start 2024-09-21T03:44:53.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:53.436+0000 7fc3b458b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc3ac15f4c0 0x7fc3ac1f9530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:53.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:53.436+0000 7fc3b458b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc3ac162280 0x7fc3ac0add80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:53.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:53.436+0000 7fc3b458b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc3ac161640 0x7fc3ac1f8990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:53.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:53.436+0000 7fc3b458b640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fc3ac0b3710 con 0x7fc3ac162280 2024-09-21T03:44:53.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:53.436+0000 7fc3b458b640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fc3ac0b3590 con 0x7fc3ac15f4c0 2024-09-21T03:44:53.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:53.436+0000 7fc3b458b640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fc3ac0b3890 con 0x7fc3ac161640 2024-09-21T03:44:53.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:53.436+0000 7fc3b1b26640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc3ac162280 0x7fc3ac0add80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:53.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:53.436+0000 7fc3b1b26640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc3ac162280 0x7fc3ac0add80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40402/0 (socket says 172.21.15.38:40402) 2024-09-21T03:44:53.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:53.436+0000 7fc3b1b26640 1 -- 172.21.15.38:0/3551375599 learned_addr learned my addr 172.21.15.38:0/3551375599 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:44:53.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:53.436+0000 7fc3b2327640 1 --2- 172.21.15.38:0/3551375599 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc3ac15f4c0 0x7fc3ac1f9530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:53.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:53.436+0000 7fc3b2b28640 1 --2- 172.21.15.38:0/3551375599 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc3ac161640 0x7fc3ac1f8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:53.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:53.436+0000 7fc3b1b26640 1 -- 172.21.15.38:0/3551375599 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc3ac161640 msgr2=0x7fc3ac1f8990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:53.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:53.436+0000 7fc3b1b26640 1 --2- 172.21.15.38:0/3551375599 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc3ac161640 0x7fc3ac1f8990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:53.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:53.436+0000 7fc3b1b26640 1 -- 172.21.15.38:0/3551375599 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc3ac15f4c0 msgr2=0x7fc3ac1f9530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:53.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:53.436+0000 7fc3b1b26640 1 --2- 172.21.15.38:0/3551375599 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc3ac15f4c0 0x7fc3ac1f9530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:53.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:53.436+0000 7fc3b1b26640 1 -- 172.21.15.38:0/3551375599 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3ac1605a0 con 0x7fc3ac162280 2024-09-21T03:44:53.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:53.436+0000 7fc3b2b28640 1 --2- 172.21.15.38:0/3551375599 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc3ac161640 0x7fc3ac1f8990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:44:53.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:53.437+0000 7fc3b1b26640 1 --2- 172.21.15.38:0/3551375599 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc3ac162280 0x7fc3ac0add80 secure :-1 s=READY pgs=2333 cs=0 l=1 rev1=1 crypto rx=0x7fc39c00c430 tx=0x7fc39c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:53.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:53.437+0000 7fc3a2ffd640 1 -- 172.21.15.38:0/3551375599 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc39c017070 con 0x7fc3ac162280 2024-09-21T03:44:53.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:53.437+0000 7fc3a2ffd640 1 -- 172.21.15.38:0/3551375599 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc39c0053c0 con 0x7fc3ac162280 2024-09-21T03:44:53.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:53.437+0000 7fc3b458b640 1 -- 172.21.15.38:0/3551375599 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3ac160880 con 0x7fc3ac162280 2024-09-21T03:44:53.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:53.437+0000 7fc3a2ffd640 1 -- 172.21.15.38:0/3551375599 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc39c003850 con 0x7fc3ac162280 2024-09-21T03:44:53.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:53.437+0000 7fc3b458b640 1 -- 172.21.15.38:0/3551375599 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3ac160e40 con 0x7fc3ac162280 2024-09-21T03:44:53.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:53.438+0000 7fc3b458b640 1 -- 172.21.15.38:0/3551375599 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3ac0b55a0 con 0x7fc3ac162280 2024-09-21T03:44:53.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:53.439+0000 7fc3a2ffd640 1 -- 172.21.15.38:0/3551375599 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc39c01a070 con 0x7fc3ac162280 2024-09-21T03:44:53.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:53.439+0000 7fc3a2ffd640 1 --2- 172.21.15.38:0/3551375599 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc390078b90 0x7fc39007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:53.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:53.439+0000 7fc3a2ffd640 1 -- 172.21.15.38:0/3551375599 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5249+0+0 (secure 0 0 0) 0x7fc39c099ef0 con 0x7fc3ac162280 2024-09-21T03:44:53.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:53.439+0000 7fc3b2327640 1 --2- 172.21.15.38:0/3551375599 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc390078b90 0x7fc39007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:53.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:53.440+0000 7fc3b2327640 1 --2- 172.21.15.38:0/3551375599 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc390078b90 0x7fc39007b0a0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fc3a8000f60 tx=0x7fc3a8009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:53.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:53.444+0000 7fc3a2ffd640 1 -- 172.21.15.38:0/3551375599 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fc3ac0b55a0 con 0x7fc3ac162280 2024-09-21T03:44:53.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:53.701+0000 7fc3b458b640 1 -- 172.21.15.38:0/3551375599 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fc3ac162f00 con 0x7fc3ac162280 2024-09-21T03:44:53.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:53.702+0000 7fc3a2ffd640 1 -- 172.21.15.38:0/3551375599 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v23) ==== 80+0+5304 (secure 0 0 0) 0x7fc39c062580 con 0x7fc3ac162280 2024-09-21T03:44:53.705 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:44:53.705 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-09-21T03:44:53.705 INFO:teuthology.orchestra.run.smithi038.stdout: "auth_dump": [ 2024-09-21T03:44:53.705 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-21T03:44:53.705 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "mds.a", 2024-09-21T03:44:53.706 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQBPPu5mfzGIMRAAmq0TaC3J1VU3GxzhTgJ/Zw==", 2024-09-21T03:44:53.706 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-09-21T03:44:53.706 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow", 2024-09-21T03:44:53.706 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow *", 2024-09-21T03:44:53.706 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow *", 2024-09-21T03:44:53.706 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-09-21T03:44:53.707 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-21T03:44:53.707 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-21T03:44:53.707 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-21T03:44:53.707 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "mds.b", 2024-09-21T03:44:53.707 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQBQPu5m/fzuAhAApyigRlQ8OrV/sf0ap6Fn6A==", 2024-09-21T03:44:53.707 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-09-21T03:44:53.707 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow", 2024-09-21T03:44:53.708 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow *", 2024-09-21T03:44:53.708 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow *", 2024-09-21T03:44:53.708 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-09-21T03:44:53.708 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-21T03:44:53.708 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-21T03:44:53.708 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-21T03:44:53.708 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "mds.c", 2024-09-21T03:44:53.709 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQBPPu5mQipIORAArRn8k8SvvpG8x03F9I2/gQ==", 2024-09-21T03:44:53.709 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-09-21T03:44:53.709 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow", 2024-09-21T03:44:53.709 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow *", 2024-09-21T03:44:53.709 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow *", 2024-09-21T03:44:53.709 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-09-21T03:44:53.709 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-21T03:44:53.710 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-21T03:44:53.710 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-21T03:44:53.710 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "mds.d", 2024-09-21T03:44:53.710 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQBQPu5mzlyDChAAqbPKXS3EtnOqqeHhA1ehIw==", 2024-09-21T03:44:53.710 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-09-21T03:44:53.710 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow", 2024-09-21T03:44:53.711 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow *", 2024-09-21T03:44:53.711 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow *", 2024-09-21T03:44:53.711 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-09-21T03:44:53.711 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-21T03:44:53.711 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-21T03:44:53.711 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-21T03:44:53.711 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "osd.0", 2024-09-21T03:44:53.712 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQBXPu5mMZggAhAAtMPdJipheizZhpby9APj4A==", 2024-09-21T03:44:53.712 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-09-21T03:44:53.712 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow profile osd", 2024-09-21T03:44:53.712 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile osd", 2024-09-21T03:44:53.712 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-09-21T03:44:53.712 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-21T03:44:53.713 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-21T03:44:53.713 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-21T03:44:53.713 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "osd.1", 2024-09-21T03:44:53.713 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQBaPu5mm9TFIhAAWPJCvBYMgMbKr7Z7Diq58A==", 2024-09-21T03:44:53.713 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-09-21T03:44:53.713 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow profile osd", 2024-09-21T03:44:53.713 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile osd", 2024-09-21T03:44:53.714 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-09-21T03:44:53.714 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-21T03:44:53.714 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-21T03:44:53.714 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-21T03:44:53.714 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "osd.2", 2024-09-21T03:44:53.714 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQBePu5mqjljCBAAcdEpzJq6D8e1L+q1s0GjWQ==", 2024-09-21T03:44:53.714 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-09-21T03:44:53.715 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow profile osd", 2024-09-21T03:44:53.715 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile osd", 2024-09-21T03:44:53.715 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-09-21T03:44:53.715 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-21T03:44:53.715 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-21T03:44:53.715 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-21T03:44:53.715 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "osd.3", 2024-09-21T03:44:53.716 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQBhPu5mRjfdKBAAJ6+PlEiRp/xvxny6SjHV8Q==", 2024-09-21T03:44:53.716 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-09-21T03:44:53.716 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow profile osd", 2024-09-21T03:44:53.716 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile osd", 2024-09-21T03:44:53.716 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-09-21T03:44:53.716 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-21T03:44:53.717 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-21T03:44:53.717 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-21T03:44:53.717 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "osd.4", 2024-09-21T03:44:53.717 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQBlPu5m8bNyNxAA2rERruGojEDKmcXN2K6uog==", 2024-09-21T03:44:53.717 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-09-21T03:44:53.717 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow profile osd", 2024-09-21T03:44:53.717 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile osd", 2024-09-21T03:44:53.718 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-09-21T03:44:53.718 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-21T03:44:53.718 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-21T03:44:53.718 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-21T03:44:53.718 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "osd.5", 2024-09-21T03:44:53.718 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQBpPu5mBB2iDBAAvsvhAPMvNWII2Zr9Je45YQ==", 2024-09-21T03:44:53.719 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-09-21T03:44:53.719 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow profile osd", 2024-09-21T03:44:53.719 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile osd", 2024-09-21T03:44:53.719 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-09-21T03:44:53.719 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-21T03:44:53.719 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-21T03:44:53.719 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-21T03:44:53.720 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "osd.6", 2024-09-21T03:44:53.720 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQBsPu5mMQMaLRAAznzYdoNGHWOI2p2mNN/rQw==", 2024-09-21T03:44:53.720 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-09-21T03:44:53.720 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow profile osd", 2024-09-21T03:44:53.720 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile osd", 2024-09-21T03:44:53.721 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-09-21T03:44:53.721 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-21T03:44:53.721 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-21T03:44:53.721 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-21T03:44:53.722 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "osd.7", 2024-09-21T03:44:53.722 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQBwPu5mHu0WERAAir8bmtuNOOj/2D4hLaoYOA==", 2024-09-21T03:44:53.722 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-09-21T03:44:53.722 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow profile osd", 2024-09-21T03:44:53.722 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile osd", 2024-09-21T03:44:53.722 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-09-21T03:44:53.722 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-21T03:44:53.722 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-21T03:44:53.722 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-21T03:44:53.722 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "client.0", 2024-09-21T03:44:53.723 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQBQPu5m6ydBDxAAlWZ2XzQTPBcYPriPGRJVeA==", 2024-09-21T03:44:53.723 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-09-21T03:44:53.723 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow", 2024-09-21T03:44:53.723 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow r", 2024-09-21T03:44:53.723 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-21T03:44:53.723 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-21T03:44:53.723 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-21T03:44:53.723 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-21T03:44:53.723 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "client.1", 2024-09-21T03:44:53.723 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQBQPu5mlWP0FhAAkNwfaJ6rPr3r4ZRVbPxX8g==", 2024-09-21T03:44:53.723 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-09-21T03:44:53.723 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow", 2024-09-21T03:44:53.723 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow r", 2024-09-21T03:44:53.724 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow rw", 2024-09-21T03:44:53.724 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow rwx" 2024-09-21T03:44:53.724 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-21T03:44:53.724 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-21T03:44:53.724 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-21T03:44:53.724 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "client.2", 2024-09-21T03:44:53.724 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQBQPu5m48oaExAAOzYrR/SsykpzH2TkWHs39Q==", 2024-09-21T03:44:53.724 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-09-21T03:44:53.724 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow", 2024-09-21T03:44:53.724 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow r", 2024-09-21T03:44:53.724 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow rw", 2024-09-21T03:44:53.725 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow rwx" 2024-09-21T03:44:53.725 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-21T03:44:53.725 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-21T03:44:53.725 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-21T03:44:53.725 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "client.3", 2024-09-21T03:44:53.725 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQBQPu5mVQSzGhAAAo6vizLc7GOnenQRsgtn1A==", 2024-09-21T03:44:53.725 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-09-21T03:44:53.725 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow", 2024-09-21T03:44:53.725 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow r", 2024-09-21T03:44:53.725 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow rw", 2024-09-21T03:44:53.725 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow rwx" 2024-09-21T03:44:53.725 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-21T03:44:53.726 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-21T03:44:53.726 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-21T03:44:53.726 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "client.admin", 2024-09-21T03:44:53.726 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQBPPu5mPHphExAA7D+/V7qhoNyoc6h0GnNMlQ==", 2024-09-21T03:44:53.726 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-09-21T03:44:53.726 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow *", 2024-09-21T03:44:53.726 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow *", 2024-09-21T03:44:53.726 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow *", 2024-09-21T03:44:53.726 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-09-21T03:44:53.726 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-21T03:44:53.726 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-21T03:44:53.726 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-21T03:44:53.726 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "mgr.x", 2024-09-21T03:44:53.727 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQBPPu5m6K2oLRAAzldoBXAMSnWhxVFwPZkHOw==", 2024-09-21T03:44:53.727 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-09-21T03:44:53.727 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow *", 2024-09-21T03:44:53.727 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile mgr", 2024-09-21T03:44:53.727 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-09-21T03:44:53.727 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-21T03:44:53.727 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-21T03:44:53.727 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-21T03:44:53.727 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "mgr.y", 2024-09-21T03:44:53.727 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQBPPu5mWEz+KBAASx2KzDril3lD9cEawy56jA==", 2024-09-21T03:44:53.727 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-09-21T03:44:53.727 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow *", 2024-09-21T03:44:53.727 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile mgr", 2024-09-21T03:44:53.727 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-09-21T03:44:53.728 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-21T03:44:53.728 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-21T03:44:53.728 INFO:teuthology.orchestra.run.smithi038.stdout: ] 2024-09-21T03:44:53.728 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-09-21T03:44:53.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:53.704+0000 7fc3b458b640 1 -- 172.21.15.38:0/3551375599 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc390078b90 msgr2=0x7fc39007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:53.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:53.704+0000 7fc3b458b640 1 --2- 172.21.15.38:0/3551375599 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc390078b90 0x7fc39007b0a0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fc3a8000f60 tx=0x7fc3a8009040 comp rx=0 tx=0).stop 2024-09-21T03:44:53.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:53.704+0000 7fc3b458b640 1 -- 172.21.15.38:0/3551375599 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc3ac162280 msgr2=0x7fc3ac0add80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:53.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:53.704+0000 7fc3b458b640 1 --2- 172.21.15.38:0/3551375599 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc3ac162280 0x7fc3ac0add80 secure :-1 s=READY pgs=2333 cs=0 l=1 rev1=1 crypto rx=0x7fc39c00c430 tx=0x7fc39c00c8f0 comp rx=0 tx=0).stop 2024-09-21T03:44:53.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:53.704+0000 7fc3b458b640 1 -- 172.21.15.38:0/3551375599 shutdown_connections 2024-09-21T03:44:53.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:53.704+0000 7fc3b458b640 1 --2- 172.21.15.38:0/3551375599 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc390078b90 0x7fc39007b0a0 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:53.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:53.704+0000 7fc3b458b640 1 --2- 172.21.15.38:0/3551375599 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc3ac161640 0x7fc3ac1f8990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:53.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:53.704+0000 7fc3b458b640 1 --2- 172.21.15.38:0/3551375599 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc3ac162280 0x7fc3ac0add80 unknown :-1 s=CLOSED pgs=2333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:53.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:53.704+0000 7fc3b458b640 1 --2- 172.21.15.38:0/3551375599 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc3ac15f4c0 0x7fc3ac1f9530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:53.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:53.704+0000 7fc3b458b640 1 -- 172.21.15.38:0/3551375599 >> 172.21.15.38:0/3551375599 conn(0x7fc3ac0b6b20 msgr2=0x7fc3ac15c990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:53.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:53.705+0000 7fc3b458b640 1 -- 172.21.15.38:0/3551375599 shutdown_connections 2024-09-21T03:44:53.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:53.705+0000 7fc3b458b640 1 -- 172.21.15.38:0/3551375599 wait complete. 2024-09-21T03:44:53.731 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-21T03:44:53.850 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:53.848+0000 7fae88069640 1 -- 172.21.15.38:0/2364523857 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fae800b5fc0 msgr2=0x7fae8015e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:53.850 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:53.848+0000 7fae88069640 1 --2- 172.21.15.38:0/2364523857 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fae800b5fc0 0x7fae8015e7a0 secure :-1 s=READY pgs=2335 cs=0 l=1 rev1=1 crypto rx=0x7fae74009830 tx=0x7fae7402a530 comp rx=0 tx=0).stop 2024-09-21T03:44:53.850 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:53.848+0000 7fae88069640 1 -- 172.21.15.38:0/2364523857 shutdown_connections 2024-09-21T03:44:53.851 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:53.848+0000 7fae88069640 1 --2- 172.21.15.38:0/2364523857 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fae800b5fc0 0x7fae8015e7a0 unknown :-1 s=CLOSED pgs=2335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:53.851 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:53.848+0000 7fae88069640 1 -- 172.21.15.38:0/2364523857 >> 172.21.15.38:0/2364523857 conn(0x7fae800d2350 msgr2=0x7fae800d2760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:53.851 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:53.848+0000 7fae88069640 1 -- 172.21.15.38:0/2364523857 shutdown_connections 2024-09-21T03:44:53.851 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:53.848+0000 7fae88069640 1 -- 172.21.15.38:0/2364523857 wait complete. 2024-09-21T03:44:53.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:53.850+0000 7fae88069640 1 Processor -- start 2024-09-21T03:44:53.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:53.850+0000 7fae88069640 1 -- start start 2024-09-21T03:44:53.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:53.851+0000 7fae88069640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fae8006e080 0x7fae80156e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:53.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:53.851+0000 7fae88069640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fae800d18c0 0x7fae800cb580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:53.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:53.851+0000 7fae88069640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fae8015ee90 0x7fae801f5650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:53.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:53.851+0000 7fae88069640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fae800b32c0 con 0x7fae8006e080 2024-09-21T03:44:53.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:53.851+0000 7fae88069640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fae800b3140 con 0x7fae8015ee90 2024-09-21T03:44:53.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:53.851+0000 7fae88069640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fae800b3440 con 0x7fae800d18c0 2024-09-21T03:44:53.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:53.851+0000 7fae85e05640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fae8006e080 0x7fae80156e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:53.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:53.851+0000 7fae85e05640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fae8006e080 0x7fae80156e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40416/0 (socket says 172.21.15.38:40416) 2024-09-21T03:44:53.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:53.851+0000 7fae85e05640 1 -- 172.21.15.38:0/3501076400 learned_addr learned my addr 172.21.15.38:0/3501076400 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:44:53.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:53.852+0000 7fae85604640 1 --2- 172.21.15.38:0/3501076400 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fae800d18c0 0x7fae800cb580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:53.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:53.852+0000 7fae86606640 1 --2- 172.21.15.38:0/3501076400 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fae8015ee90 0x7fae801f5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:53.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:53.852+0000 7fae85e05640 1 -- 172.21.15.38:0/3501076400 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fae800d18c0 msgr2=0x7fae800cb580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:53.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:53.852+0000 7fae85e05640 1 --2- 172.21.15.38:0/3501076400 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fae800d18c0 0x7fae800cb580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:53.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:53.852+0000 7fae85e05640 1 -- 172.21.15.38:0/3501076400 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fae8015ee90 msgr2=0x7fae801f5650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:53.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:53.852+0000 7fae85e05640 1 --2- 172.21.15.38:0/3501076400 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fae8015ee90 0x7fae801f5650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:53.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:53.852+0000 7fae85e05640 1 -- 172.21.15.38:0/3501076400 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae800ce110 con 0x7fae8006e080 2024-09-21T03:44:53.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:53.852+0000 7fae86606640 1 --2- 172.21.15.38:0/3501076400 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fae8015ee90 0x7fae801f5650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:44:53.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:53.852+0000 7fae85e05640 1 --2- 172.21.15.38:0/3501076400 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fae8006e080 0x7fae80156e10 secure :-1 s=READY pgs=2336 cs=0 l=1 rev1=1 crypto rx=0x7fae7800c3f0 tx=0x7fae7800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:53.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:53.852+0000 7fae7e7fc640 1 -- 172.21.15.38:0/3501076400 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae78017070 con 0x7fae8006e080 2024-09-21T03:44:53.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:53.852+0000 7fae7e7fc640 1 -- 172.21.15.38:0/3501076400 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fae780035f0 con 0x7fae8006e080 2024-09-21T03:44:53.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:53.852+0000 7fae88069640 1 -- 172.21.15.38:0/3501076400 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae800ce3f0 con 0x7fae8006e080 2024-09-21T03:44:53.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:53.852+0000 7fae7e7fc640 1 -- 172.21.15.38:0/3501076400 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae78003a40 con 0x7fae8006e080 2024-09-21T03:44:53.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:53.852+0000 7fae88069640 1 -- 172.21.15.38:0/3501076400 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae800ce9b0 con 0x7fae8006e080 2024-09-21T03:44:53.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:53.854+0000 7fae88069640 1 -- 172.21.15.38:0/3501076400 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae8015e7a0 con 0x7fae8006e080 2024-09-21T03:44:53.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:53.854+0000 7fae7e7fc640 1 -- 172.21.15.38:0/3501076400 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fae7801a070 con 0x7fae8006e080 2024-09-21T03:44:53.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:53.855+0000 7fae7e7fc640 1 --2- 172.21.15.38:0/3501076400 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fae6c078bc0 0x7fae6c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:53.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:53.855+0000 7fae7e7fc640 1 -- 172.21.15.38:0/3501076400 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5249+0+0 (secure 0 0 0) 0x7fae78099f90 con 0x7fae8006e080 2024-09-21T03:44:53.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:53.855+0000 7fae85604640 1 --2- 172.21.15.38:0/3501076400 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fae6c078bc0 0x7fae6c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:53.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:53.855+0000 7fae85604640 1 --2- 172.21.15.38:0/3501076400 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fae6c078bc0 0x7fae6c07b0d0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fae74034000 tx=0x7fae74004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:53.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:53.859+0000 7fae7e7fc640 1 -- 172.21.15.38:0/3501076400 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fae78062470 con 0x7fae8006e080 2024-09-21T03:44:54.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:54.048+0000 7fae88069640 1 -- 172.21.15.38:0/3501076400 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fae800734e0 con 0x7fae8006e080 2024-09-21T03:44:54.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:54.048+0000 7fae7e7fc640 1 -- 172.21.15.38:0/3501076400 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v142) ==== 84+0+65 (secure 0 0 0) 0x7fae78061d70 con 0x7fae8006e080 2024-09-21T03:44:54.050 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:44:54.050 INFO:teuthology.orchestra.run.smithi038.stdout:[ 2024-09-21T03:44:54.050 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-21T03:44:54.051 INFO:teuthology.orchestra.run.smithi038.stdout: "poolnum": 1, 2024-09-21T03:44:54.051 INFO:teuthology.orchestra.run.smithi038.stdout: "poolname": ".mgr" 2024-09-21T03:44:54.051 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-21T03:44:54.051 INFO:teuthology.orchestra.run.smithi038.stdout:] 2024-09-21T03:44:54.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:54.050+0000 7fae88069640 1 -- 172.21.15.38:0/3501076400 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fae6c078bc0 msgr2=0x7fae6c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:54.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:54.050+0000 7fae88069640 1 --2- 172.21.15.38:0/3501076400 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fae6c078bc0 0x7fae6c07b0d0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fae74034000 tx=0x7fae74004730 comp rx=0 tx=0).stop 2024-09-21T03:44:54.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:54.050+0000 7fae88069640 1 -- 172.21.15.38:0/3501076400 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fae8006e080 msgr2=0x7fae80156e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:54.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:54.050+0000 7fae88069640 1 --2- 172.21.15.38:0/3501076400 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fae8006e080 0x7fae80156e10 secure :-1 s=READY pgs=2336 cs=0 l=1 rev1=1 crypto rx=0x7fae7800c3f0 tx=0x7fae7800c8b0 comp rx=0 tx=0).stop 2024-09-21T03:44:54.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:54.050+0000 7fae88069640 1 -- 172.21.15.38:0/3501076400 shutdown_connections 2024-09-21T03:44:54.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:54.050+0000 7fae88069640 1 --2- 172.21.15.38:0/3501076400 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fae8015ee90 0x7fae801f5650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:54.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:54.050+0000 7fae88069640 1 --2- 172.21.15.38:0/3501076400 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fae6c078bc0 0x7fae6c07b0d0 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:54.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:54.050+0000 7fae88069640 1 --2- 172.21.15.38:0/3501076400 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fae800d18c0 0x7fae800cb580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:54.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:54.050+0000 7fae88069640 1 --2- 172.21.15.38:0/3501076400 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fae8006e080 0x7fae80156e10 unknown :-1 s=CLOSED pgs=2336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:54.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:54.050+0000 7fae88069640 1 -- 172.21.15.38:0/3501076400 >> 172.21.15.38:0/3501076400 conn(0x7fae800d2350 msgr2=0x7fae80070720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:54.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:54.050+0000 7fae88069640 1 -- 172.21.15.38:0/3501076400 shutdown_connections 2024-09-21T03:44:54.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:54.050+0000 7fae88069640 1 -- 172.21.15.38:0/3501076400 wait complete. 2024-09-21T03:44:54.064 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-21T03:44:54.064 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-21T03:44:54.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:54.182+0000 7fb74b64d640 1 -- 172.21.15.38:0/3066903062 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb7440d0460 msgr2=0x7fb744161ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:54.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:54.182+0000 7fb74b64d640 1 --2- 172.21.15.38:0/3066903062 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb7440d0460 0x7fb744161ed0 secure :-1 s=READY pgs=2337 cs=0 l=1 rev1=1 crypto rx=0x7fb7440ba580 tx=0x7fb74002a560 comp rx=0 tx=0).stop 2024-09-21T03:44:54.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:54.183+0000 7fb74b64d640 1 -- 172.21.15.38:0/3066903062 shutdown_connections 2024-09-21T03:44:54.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:54.183+0000 7fb74b64d640 1 --2- 172.21.15.38:0/3066903062 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb7440cf200 0x7fb7440cf610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:54.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:54.183+0000 7fb74b64d640 1 --2- 172.21.15.38:0/3066903062 >> v2:172.21.15.195:3300/0 conn(0x7fb7440b7b60 0x7fb7440b7f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:54.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:54.183+0000 7fb74b64d640 1 --2- 172.21.15.38:0/3066903062 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb7440d0460 0x7fb744161ed0 unknown :-1 s=CLOSED pgs=2337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:54.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:54.183+0000 7fb74b64d640 1 -- 172.21.15.38:0/3066903062 >> 172.21.15.38:0/3066903062 conn(0x7fb7440d4510 msgr2=0x7fb7440d4920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:54.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:54.183+0000 7fb74b64d640 1 -- 172.21.15.38:0/3066903062 shutdown_connections 2024-09-21T03:44:54.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:54.183+0000 7fb74b64d640 1 -- 172.21.15.38:0/3066903062 wait complete. 2024-09-21T03:44:54.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:54.184+0000 7fb74b64d640 1 Processor -- start 2024-09-21T03:44:54.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:54.184+0000 7fb74b64d640 1 -- start start 2024-09-21T03:44:54.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:54.186+0000 7fb74b64d640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb7440b7b60 0x7fb7440cc870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:54.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:54.186+0000 7fb74b64d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb7440cf200 0x7fb74415aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:54.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:54.186+0000 7fb74b64d640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb7440d0460 0x7fb7440b8560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:54.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:54.186+0000 7fb74b64d640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fb7440b6380 con 0x7fb7440cf200 2024-09-21T03:44:54.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:54.186+0000 7fb74b64d640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fb7440b6200 con 0x7fb7440b7b60 2024-09-21T03:44:54.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:54.186+0000 7fb74b64d640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fb7440b6500 con 0x7fb7440d0460 2024-09-21T03:44:54.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:54.186+0000 7fb748be8640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb7440cf200 0x7fb74415aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:54.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:54.186+0000 7fb748be8640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb7440cf200 0x7fb74415aca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40436/0 (socket says 172.21.15.38:40436) 2024-09-21T03:44:54.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:54.186+0000 7fb748be8640 1 -- 172.21.15.38:0/1653158762 learned_addr learned my addr 172.21.15.38:0/1653158762 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:44:54.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:54.186+0000 7fb7493e9640 1 --2- 172.21.15.38:0/1653158762 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb7440b7b60 0x7fb7440cc870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:54.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:54.186+0000 7fb749bea640 1 --2- 172.21.15.38:0/1653158762 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb7440d0460 0x7fb7440b8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:54.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:54.186+0000 7fb748be8640 1 -- 172.21.15.38:0/1653158762 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb7440d0460 msgr2=0x7fb7440b8560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:54.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:54.186+0000 7fb748be8640 1 --2- 172.21.15.38:0/1653158762 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb7440d0460 0x7fb7440b8560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:54.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:54.186+0000 7fb748be8640 1 -- 172.21.15.38:0/1653158762 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb7440b7b60 msgr2=0x7fb7440cc870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:54.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:54.186+0000 7fb748be8640 1 --2- 172.21.15.38:0/1653158762 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb7440b7b60 0x7fb7440cc870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:54.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:54.186+0000 7fb748be8640 1 -- 172.21.15.38:0/1653158762 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb74415b1e0 con 0x7fb7440cf200 2024-09-21T03:44:54.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:54.186+0000 7fb748be8640 1 --2- 172.21.15.38:0/1653158762 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb7440cf200 0x7fb74415aca0 secure :-1 s=READY pgs=2338 cs=0 l=1 rev1=1 crypto rx=0x7fb73400c970 tx=0x7fb73400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:54.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:54.187+0000 7fb739ffb640 1 -- 172.21.15.38:0/1653158762 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb734007bb0 con 0x7fb7440cf200 2024-09-21T03:44:54.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:54.187+0000 7fb739ffb640 1 -- 172.21.15.38:0/1653158762 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb734007e40 con 0x7fb7440cf200 2024-09-21T03:44:54.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:54.187+0000 7fb74b64d640 1 -- 172.21.15.38:0/1653158762 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb74415f6f0 con 0x7fb7440cf200 2024-09-21T03:44:54.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:54.187+0000 7fb739ffb640 1 -- 172.21.15.38:0/1653158762 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb73400f6b0 con 0x7fb7440cf200 2024-09-21T03:44:54.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:54.187+0000 7fb74b64d640 1 -- 172.21.15.38:0/1653158762 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb74415fcb0 con 0x7fb7440cf200 2024-09-21T03:44:54.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:54.188+0000 7fb74b64d640 1 -- 172.21.15.38:0/1653158762 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7440ce6c0 con 0x7fb7440cf200 2024-09-21T03:44:54.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:54.188+0000 7fb739ffb640 1 -- 172.21.15.38:0/1653158762 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb73400f850 con 0x7fb7440cf200 2024-09-21T03:44:54.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:54.189+0000 7fb739ffb640 1 --2- 172.21.15.38:0/1653158762 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb728078b90 0x7fb72807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:54.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:54.189+0000 7fb739ffb640 1 -- 172.21.15.38:0/1653158762 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5249+0+0 (secure 0 0 0) 0x7fb73409b190 con 0x7fb7440cf200 2024-09-21T03:44:54.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:54.189+0000 7fb7493e9640 1 --2- 172.21.15.38:0/1653158762 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb728078b90 0x7fb72807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:54.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:54.190+0000 7fb7493e9640 1 --2- 172.21.15.38:0/1653158762 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb728078b90 0x7fb72807b0a0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fb740033000 tx=0x7fb740004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:54.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:54.194+0000 7fb739ffb640 1 -- 172.21.15.38:0/1653158762 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fb734014030 con 0x7fb7440cf200 2024-09-21T03:44:54.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:54.381+0000 7fb74b64d640 1 -- 172.21.15.38:0/1653158762 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fb7440cf610 con 0x7fb7440cf200 2024-09-21T03:44:55.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:55.311+0000 7fb739ffb640 1 -- 172.21.15.38:0/1653158762 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v143) ==== 138+0+0 (secure 0 0 0) 0x7fb734063150 con 0x7fb7440cf200 2024-09-21T03:44:55.313 INFO:teuthology.orchestra.run.smithi038.stderr:pool 'cephfs_metadata' created 2024-09-21T03:44:55.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:55.315+0000 7fb74b64d640 1 -- 172.21.15.38:0/1653158762 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb728078b90 msgr2=0x7fb72807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:55.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:55.315+0000 7fb74b64d640 1 --2- 172.21.15.38:0/1653158762 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb728078b90 0x7fb72807b0a0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fb740033000 tx=0x7fb740004730 comp rx=0 tx=0).stop 2024-09-21T03:44:55.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:55.315+0000 7fb74b64d640 1 -- 172.21.15.38:0/1653158762 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb7440cf200 msgr2=0x7fb74415aca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:55.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:55.315+0000 7fb74b64d640 1 --2- 172.21.15.38:0/1653158762 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb7440cf200 0x7fb74415aca0 secure :-1 s=READY pgs=2338 cs=0 l=1 rev1=1 crypto rx=0x7fb73400c970 tx=0x7fb73400ce30 comp rx=0 tx=0).stop 2024-09-21T03:44:55.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:55.316+0000 7fb74b64d640 1 -- 172.21.15.38:0/1653158762 shutdown_connections 2024-09-21T03:44:55.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:55.316+0000 7fb74b64d640 1 --2- 172.21.15.38:0/1653158762 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb728078b90 0x7fb72807b0a0 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:55.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:55.316+0000 7fb74b64d640 1 --2- 172.21.15.38:0/1653158762 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb7440d0460 0x7fb7440b8560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:55.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:55.316+0000 7fb74b64d640 1 --2- 172.21.15.38:0/1653158762 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb7440cf200 0x7fb74415aca0 unknown :-1 s=CLOSED pgs=2338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:55.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:55.316+0000 7fb74b64d640 1 --2- 172.21.15.38:0/1653158762 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb7440b7b60 0x7fb7440cc870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:55.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:55.316+0000 7fb74b64d640 1 -- 172.21.15.38:0/1653158762 >> 172.21.15.38:0/1653158762 conn(0x7fb7440d4510 msgr2=0x7fb74415da80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:55.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:55.316+0000 7fb74b64d640 1 -- 172.21.15.38:0/1653158762 shutdown_connections 2024-09-21T03:44:55.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:55.316+0000 7fb74b64d640 1 -- 172.21.15.38:0/1653158762 wait complete. 2024-09-21T03:44:55.335 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-21T03:44:55.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:55.463+0000 7f351a8de640 1 -- 172.21.15.38:0/3272781068 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f351415f4c0 msgr2=0x7f35140b2e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:55.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:55.463+0000 7f351a8de640 1 --2- 172.21.15.38:0/3272781068 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f351415f4c0 0x7f35140b2e40 secure :-1 s=READY pgs=2340 cs=0 l=1 rev1=1 crypto rx=0x7f350c005fd0 tx=0x7f350c02a820 comp rx=0 tx=0).stop 2024-09-21T03:44:55.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:55.463+0000 7f351a8de640 1 -- 172.21.15.38:0/3272781068 shutdown_connections 2024-09-21T03:44:55.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:55.463+0000 7f351a8de640 1 --2- 172.21.15.38:0/3272781068 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f351415f4c0 0x7f35140b2e40 unknown :-1 s=CLOSED pgs=2340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:55.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:55.463+0000 7f351a8de640 1 -- 172.21.15.38:0/3272781068 >> 172.21.15.38:0/3272781068 conn(0x7f35140b6b20 msgr2=0x7f35140d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:55.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:55.464+0000 7f351a8de640 1 -- 172.21.15.38:0/3272781068 shutdown_connections 2024-09-21T03:44:55.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:55.464+0000 7f351a8de640 1 -- 172.21.15.38:0/3272781068 wait complete. 2024-09-21T03:44:55.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:55.465+0000 7f351a8de640 1 Processor -- start 2024-09-21T03:44:55.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:55.466+0000 7f351a8de640 1 -- start start 2024-09-21T03:44:55.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:55.467+0000 7f351a8de640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f351415f4c0 0x7f35140cec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:55.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:55.467+0000 7f351a8de640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3514162280 0x7f351414e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:55.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:55.467+0000 7f351a8de640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f351414e9c0 0x7f351414edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:55.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:55.467+0000 7f351a8de640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f35140b3710 con 0x7f3514162280 2024-09-21T03:44:55.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:55.467+0000 7f351a8de640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f35140b3590 con 0x7f351415f4c0 2024-09-21T03:44:55.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:55.467+0000 7f351a8de640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f35140b3890 con 0x7f351414e9c0 2024-09-21T03:44:55.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:55.467+0000 7f35137fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3514162280 0x7f351414e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:55.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:55.467+0000 7f35137fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3514162280 0x7f351414e480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40448/0 (socket says 172.21.15.38:40448) 2024-09-21T03:44:55.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:55.467+0000 7f35137fe640 1 -- 172.21.15.38:0/2872310204 learned_addr learned my addr 172.21.15.38:0/2872310204 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:44:55.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:55.467+0000 7f3513fff640 1 --2- 172.21.15.38:0/2872310204 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f351415f4c0 0x7f35140cec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:55.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:55.467+0000 7f3518e7b640 1 --2- 172.21.15.38:0/2872310204 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f351414e9c0 0x7f351414edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:55.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:55.467+0000 7f35137fe640 1 -- 172.21.15.38:0/2872310204 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f351414e9c0 msgr2=0x7f351414edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:55.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:55.467+0000 7f35137fe640 1 --2- 172.21.15.38:0/2872310204 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f351414e9c0 0x7f351414edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:55.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:55.467+0000 7f35137fe640 1 -- 172.21.15.38:0/2872310204 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f351415f4c0 msgr2=0x7f35140cec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:55.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:55.467+0000 7f35137fe640 1 --2- 172.21.15.38:0/2872310204 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f351415f4c0 0x7f35140cec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:55.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:55.467+0000 7f35137fe640 1 -- 172.21.15.38:0/2872310204 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f351414f310 con 0x7f3514162280 2024-09-21T03:44:55.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:55.467+0000 7f3518e7b640 1 --2- 172.21.15.38:0/2872310204 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f351414e9c0 0x7f351414edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:44:55.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:55.467+0000 7f35137fe640 1 --2- 172.21.15.38:0/2872310204 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3514162280 0x7f351414e480 secure :-1 s=READY pgs=2341 cs=0 l=1 rev1=1 crypto rx=0x7f350800b3f0 tx=0x7f350800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:55.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:55.468+0000 7f3510ff9640 1 -- 172.21.15.38:0/2872310204 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3508015020 con 0x7f3514162280 2024-09-21T03:44:55.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:55.468+0000 7f3510ff9640 1 -- 172.21.15.38:0/2872310204 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3508011920 con 0x7f3514162280 2024-09-21T03:44:55.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:55.468+0000 7f351a8de640 1 -- 172.21.15.38:0/2872310204 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f351414d9c0 con 0x7f3514162280 2024-09-21T03:44:55.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:55.468+0000 7f3510ff9640 1 -- 172.21.15.38:0/2872310204 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3508011d70 con 0x7f3514162280 2024-09-21T03:44:55.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:55.468+0000 7f351a8de640 1 -- 172.21.15.38:0/2872310204 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35140ae7d0 con 0x7f3514162280 2024-09-21T03:44:55.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:55.469+0000 7f351a8de640 1 -- 172.21.15.38:0/2872310204 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3514162f00 con 0x7f3514162280 2024-09-21T03:44:55.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:55.470+0000 7f3510ff9640 1 -- 172.21.15.38:0/2872310204 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f3508018020 con 0x7f3514162280 2024-09-21T03:44:55.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:55.470+0000 7f3510ff9640 1 --2- 172.21.15.38:0/2872310204 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3500078b90 0x7f350007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:55.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:55.470+0000 7f3510ff9640 1 -- 172.21.15.38:0/2872310204 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 5619+0+0 (secure 0 0 0) 0x7f350809b090 con 0x7f3514162280 2024-09-21T03:44:55.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:55.470+0000 7f3513fff640 1 --2- 172.21.15.38:0/2872310204 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3500078b90 0x7f350007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:55.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:55.471+0000 7f3513fff640 1 --2- 172.21.15.38:0/2872310204 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3500078b90 0x7f350007b0a0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f3504000f60 tx=0x7f3504009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:55.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:55.474+0000 7f3510ff9640 1 -- 172.21.15.38:0/2872310204 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f350801f020 con 0x7f3514162280 2024-09-21T03:44:55.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:55.662+0000 7f351a8de640 1 -- 172.21.15.38:0/2872310204 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f3514162a10 con 0x7f3514162280 2024-09-21T03:44:56.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:56.313+0000 7f3510ff9640 1 -- 172.21.15.38:0/2872310204 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v144) ==== 170+0+0 (secure 0 0 0) 0x7f3508063600 con 0x7f3514162280 2024-09-21T03:44:56.315 INFO:teuthology.orchestra.run.smithi038.stderr:pool 'cephfs_data' created 2024-09-21T03:44:56.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:56.317+0000 7f351a8de640 1 -- 172.21.15.38:0/2872310204 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3500078b90 msgr2=0x7f350007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:56.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:56.317+0000 7f351a8de640 1 --2- 172.21.15.38:0/2872310204 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3500078b90 0x7f350007b0a0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f3504000f60 tx=0x7f3504009040 comp rx=0 tx=0).stop 2024-09-21T03:44:56.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:56.317+0000 7f351a8de640 1 -- 172.21.15.38:0/2872310204 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3514162280 msgr2=0x7f351414e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:56.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:56.317+0000 7f351a8de640 1 --2- 172.21.15.38:0/2872310204 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3514162280 0x7f351414e480 secure :-1 s=READY pgs=2341 cs=0 l=1 rev1=1 crypto rx=0x7f350800b3f0 tx=0x7f350800b8b0 comp rx=0 tx=0).stop 2024-09-21T03:44:56.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:56.317+0000 7f351a8de640 1 -- 172.21.15.38:0/2872310204 shutdown_connections 2024-09-21T03:44:56.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:56.317+0000 7f351a8de640 1 --2- 172.21.15.38:0/2872310204 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3500078b90 0x7f350007b0a0 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:56.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:56.317+0000 7f351a8de640 1 --2- 172.21.15.38:0/2872310204 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f351414e9c0 0x7f351414edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:56.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:56.317+0000 7f351a8de640 1 --2- 172.21.15.38:0/2872310204 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3514162280 0x7f351414e480 secure :-1 s=CLOSED pgs=2341 cs=0 l=1 rev1=1 crypto rx=0x7f350800b3f0 tx=0x7f350800b8b0 comp rx=0 tx=0).stop 2024-09-21T03:44:56.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:56.317+0000 7f351a8de640 1 --2- 172.21.15.38:0/2872310204 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f351415f4c0 0x7f35140cec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:56.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:56.317+0000 7f351a8de640 1 -- 172.21.15.38:0/2872310204 >> 172.21.15.38:0/2872310204 conn(0x7f35140b6b20 msgr2=0x7f351406e470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:56.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:56.317+0000 7f351a8de640 1 -- 172.21.15.38:0/2872310204 shutdown_connections 2024-09-21T03:44:56.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:56.317+0000 7f351a8de640 1 -- 172.21.15.38:0/2872310204 wait complete. 2024-09-21T03:44:56.344 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-21T03:44:56.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:56.467+0000 7f5ee3094640 1 -- 172.21.15.38:0/667474281 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5edc0d1120 msgr2=0x7f5edc0ceb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:56.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:56.467+0000 7f5ee3094640 1 --2- 172.21.15.38:0/667474281 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5edc0d1120 0x7f5edc0ceb90 secure :-1 s=READY pgs=2343 cs=0 l=1 rev1=1 crypto rx=0x7f5edc0d2360 tx=0x7f5ed402a9a0 comp rx=0 tx=0).stop 2024-09-21T03:44:56.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:56.468+0000 7f5ee3094640 1 -- 172.21.15.38:0/667474281 shutdown_connections 2024-09-21T03:44:56.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:56.468+0000 7f5ee3094640 1 --2- 172.21.15.38:0/667474281 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5edc0d1120 0x7f5edc0ceb90 unknown :-1 s=CLOSED pgs=2343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:56.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:56.468+0000 7f5ee3094640 1 --2- 172.21.15.38:0/667474281 >> v2:172.21.15.195:3300/0 conn(0x7f5edc0d0170 0x7f5edc0d0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:56.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:56.468+0000 7f5ee3094640 1 -- 172.21.15.38:0/667474281 >> 172.21.15.38:0/667474281 conn(0x7f5edc09d830 msgr2=0x7f5edc0b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:56.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:56.468+0000 7f5ee3094640 1 -- 172.21.15.38:0/667474281 shutdown_connections 2024-09-21T03:44:56.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:56.468+0000 7f5ee3094640 1 -- 172.21.15.38:0/667474281 wait complete. 2024-09-21T03:44:56.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:56.469+0000 7f5ee3094640 1 Processor -- start 2024-09-21T03:44:56.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:56.469+0000 7f5ee3094640 1 -- start start 2024-09-21T03:44:56.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:56.471+0000 7f5ee3094640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5edc0d0170 0x7f5edc15c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:56.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:56.471+0000 7f5ee3094640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5edc0d1120 0x7f5edc1f9c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:56.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:56.471+0000 7f5ee3094640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5edc15ee80 0x7f5edc157910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:56.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:56.471+0000 7f5ee3094640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f5edc0b6700 con 0x7f5edc15ee80 2024-09-21T03:44:56.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:56.471+0000 7f5ee3094640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f5edc0b6580 con 0x7f5edc0d0170 2024-09-21T03:44:56.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:56.471+0000 7f5ee3094640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f5edc0b6880 con 0x7f5edc0d1120 2024-09-21T03:44:56.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:56.471+0000 7f5ee1631640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5edc15ee80 0x7f5edc157910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:56.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:56.471+0000 7f5ee1631640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5edc15ee80 0x7f5edc157910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40466/0 (socket says 172.21.15.38:40466) 2024-09-21T03:44:56.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:56.471+0000 7f5ee1631640 1 -- 172.21.15.38:0/1327727318 learned_addr learned my addr 172.21.15.38:0/1327727318 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:44:56.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:56.471+0000 7f5ee0e30640 1 --2- 172.21.15.38:0/1327727318 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5edc0d0170 0x7f5edc15c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:56.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:56.471+0000 7f5ed3fff640 1 --2- 172.21.15.38:0/1327727318 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5edc0d1120 0x7f5edc1f9c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:56.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:56.471+0000 7f5ee1631640 1 -- 172.21.15.38:0/1327727318 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5edc0d1120 msgr2=0x7f5edc1f9c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:56.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:56.471+0000 7f5ee1631640 1 --2- 172.21.15.38:0/1327727318 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5edc0d1120 0x7f5edc1f9c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:56.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:56.471+0000 7f5ee1631640 1 -- 172.21.15.38:0/1327727318 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5edc0d0170 msgr2=0x7f5edc15c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:56.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:56.471+0000 7f5ee1631640 1 --2- 172.21.15.38:0/1327727318 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5edc0d0170 0x7f5edc15c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:56.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:56.471+0000 7f5ee1631640 1 -- 172.21.15.38:0/1327727318 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5edc157e50 con 0x7f5edc15ee80 2024-09-21T03:44:56.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:56.472+0000 7f5ee1631640 1 --2- 172.21.15.38:0/1327727318 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5edc15ee80 0x7f5edc157910 secure :-1 s=READY pgs=2344 cs=0 l=1 rev1=1 crypto rx=0x7f5ed4009d60 tx=0x7f5ed4003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:56.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:56.472+0000 7f5ed17fa640 1 -- 172.21.15.38:0/1327727318 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ed4033050 con 0x7f5edc15ee80 2024-09-21T03:44:56.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:56.472+0000 7f5ed17fa640 1 -- 172.21.15.38:0/1327727318 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5ed40044d0 con 0x7f5edc15ee80 2024-09-21T03:44:56.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:56.472+0000 7f5ee3094640 1 -- 172.21.15.38:0/1327727318 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5edc06ed90 con 0x7f5edc15ee80 2024-09-21T03:44:56.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:56.472+0000 7f5ed17fa640 1 -- 172.21.15.38:0/1327727318 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ed4038450 con 0x7f5edc15ee80 2024-09-21T03:44:56.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:56.472+0000 7f5ee3094640 1 -- 172.21.15.38:0/1327727318 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5edc0cb790 con 0x7f5edc15ee80 2024-09-21T03:44:56.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:56.473+0000 7f5ee3094640 1 -- 172.21.15.38:0/1327727318 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5edc06acf0 con 0x7f5edc15ee80 2024-09-21T03:44:56.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:56.474+0000 7f5ed17fa640 1 -- 172.21.15.38:0/1327727318 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f5ed4036070 con 0x7f5edc15ee80 2024-09-21T03:44:56.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:56.474+0000 7f5ed17fa640 1 --2- 172.21.15.38:0/1327727318 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5ec8078bc0 0x7f5ec807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:56.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:56.474+0000 7f5ed17fa640 1 -- 172.21.15.38:0/1327727318 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 6001+0+0 (secure 0 0 0) 0x7f5ed40bc0b0 con 0x7f5edc15ee80 2024-09-21T03:44:56.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:56.474+0000 7f5ee0e30640 1 --2- 172.21.15.38:0/1327727318 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5ec8078bc0 0x7f5ec807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:56.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:56.475+0000 7f5ee0e30640 1 --2- 172.21.15.38:0/1327727318 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5ec8078bc0 0x7f5ec807b0d0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f5ed8000f60 tx=0x7f5ed8009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:56.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:56.479+0000 7f5ed17fa640 1 -- 172.21.15.38:0/1327727318 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f5edc06acf0 con 0x7f5edc15ee80 2024-09-21T03:44:56.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:56.709+0000 7f5ee3094640 1 -- 172.21.15.38:0/1327727318 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f5edc0b6da0 con 0x7f5edc15ee80 2024-09-21T03:44:57.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:57.340+0000 7f5ed17fa640 1 -- 172.21.15.38:0/1327727318 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 15 and data pool 16 v339) ==== 174+0+0 (secure 0 0 0) 0x7f5ed4083d80 con 0x7f5edc15ee80 2024-09-21T03:44:57.345 INFO:teuthology.orchestra.run.smithi038.stderr:new fs with metadata pool 15 and data pool 16 2024-09-21T03:44:57.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:57.352+0000 7f5ee3094640 1 -- 172.21.15.38:0/1327727318 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5ec8078bc0 msgr2=0x7f5ec807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:57.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:57.352+0000 7f5ee3094640 1 --2- 172.21.15.38:0/1327727318 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5ec8078bc0 0x7f5ec807b0d0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f5ed8000f60 tx=0x7f5ed8009040 comp rx=0 tx=0).stop 2024-09-21T03:44:57.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:57.352+0000 7f5ee3094640 1 -- 172.21.15.38:0/1327727318 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5edc15ee80 msgr2=0x7f5edc157910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:57.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:57.352+0000 7f5ee3094640 1 --2- 172.21.15.38:0/1327727318 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5edc15ee80 0x7f5edc157910 secure :-1 s=READY pgs=2344 cs=0 l=1 rev1=1 crypto rx=0x7f5ed4009d60 tx=0x7f5ed4003f10 comp rx=0 tx=0).stop 2024-09-21T03:44:57.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:57.353+0000 7f5ee3094640 1 -- 172.21.15.38:0/1327727318 shutdown_connections 2024-09-21T03:44:57.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:57.353+0000 7f5ee3094640 1 --2- 172.21.15.38:0/1327727318 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5edc15ee80 0x7f5edc157910 unknown :-1 s=CLOSED pgs=2344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:57.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:57.353+0000 7f5ee3094640 1 --2- 172.21.15.38:0/1327727318 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5ec8078bc0 0x7f5ec807b0d0 secure :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f5ed8000f60 tx=0x7f5ed8009040 comp rx=0 tx=0).stop 2024-09-21T03:44:57.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:57.353+0000 7f5ee3094640 1 --2- 172.21.15.38:0/1327727318 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5edc0d1120 0x7f5edc1f9c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:57.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:57.353+0000 7f5ee3094640 1 --2- 172.21.15.38:0/1327727318 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5edc0d0170 0x7f5edc15c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:57.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:57.353+0000 7f5ee3094640 1 -- 172.21.15.38:0/1327727318 >> 172.21.15.38:0/1327727318 conn(0x7f5edc09d830 msgr2=0x7f5edc0b5920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:57.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:57.357+0000 7f5ee3094640 1 -- 172.21.15.38:0/1327727318 shutdown_connections 2024-09-21T03:44:57.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:57.357+0000 7f5ee3094640 1 -- 172.21.15.38:0/1327727318 wait complete. 2024-09-21T03:44:57.383 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T03:44:57.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:57.502+0000 7f2923fff640 1 -- 172.21.15.38:0/1518985435 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f29240b9e40 msgr2=0x7f29241622b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:57.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:57.502+0000 7f2923fff640 1 --2- 172.21.15.38:0/1518985435 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f29240b9e40 0x7f29241622b0 secure :-1 s=READY pgs=2345 cs=0 l=1 rev1=1 crypto rx=0x7f29240c7db0 tx=0x7f291402aac0 comp rx=0 tx=0).stop 2024-09-21T03:44:57.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:57.502+0000 7f2923fff640 1 -- 172.21.15.38:0/1518985435 shutdown_connections 2024-09-21T03:44:57.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:57.502+0000 7f2923fff640 1 --2- 172.21.15.38:0/1518985435 >> v2:172.21.15.195:3300/0 conn(0x7f292406d9c0 0x7f2924154ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:57.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:57.502+0000 7f2923fff640 1 --2- 172.21.15.38:0/1518985435 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f29240b9e40 0x7f29241622b0 unknown :-1 s=CLOSED pgs=2345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:57.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:57.502+0000 7f2923fff640 1 -- 172.21.15.38:0/1518985435 >> 172.21.15.38:0/1518985435 conn(0x7f29240d1930 msgr2=0x7f29240d42e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:57.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:57.502+0000 7f2923fff640 1 -- 172.21.15.38:0/1518985435 shutdown_connections 2024-09-21T03:44:57.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:57.502+0000 7f2923fff640 1 -- 172.21.15.38:0/1518985435 wait complete. 2024-09-21T03:44:57.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:57.504+0000 7f2923fff640 1 Processor -- start 2024-09-21T03:44:57.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:57.504+0000 7f2923fff640 1 -- start start 2024-09-21T03:44:57.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:57.505+0000 7f2923fff640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f292406d9c0 0x7f2924151d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:57.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:57.505+0000 7f2923fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f29240b9e40 0x7f29241f99b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:57.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:57.505+0000 7f2923fff640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f29241627f0 0x7f29241573c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:57.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:57.505+0000 7f2923fff640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f2924155770 con 0x7f29240b9e40 2024-09-21T03:44:57.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:57.505+0000 7f2923fff640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f29241555f0 con 0x7f29241627f0 2024-09-21T03:44:57.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:57.505+0000 7f2923fff640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f29241558f0 con 0x7f292406d9c0 2024-09-21T03:44:57.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:57.505+0000 7f29227fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f29240b9e40 0x7f29241f99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:57.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:57.505+0000 7f29227fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f29240b9e40 0x7f29241f99b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40470/0 (socket says 172.21.15.38:40470) 2024-09-21T03:44:57.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:57.505+0000 7f29227fc640 1 -- 172.21.15.38:0/3211898840 learned_addr learned my addr 172.21.15.38:0/3211898840 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:44:57.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:57.505+0000 7f29237fe640 1 --2- 172.21.15.38:0/3211898840 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f29241627f0 0x7f29241573c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:57.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:57.505+0000 7f2922ffd640 1 --2- 172.21.15.38:0/3211898840 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f292406d9c0 0x7f2924151d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:57.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:57.505+0000 7f29227fc640 1 -- 172.21.15.38:0/3211898840 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f292406d9c0 msgr2=0x7f2924151d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:57.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:57.506+0000 7f29227fc640 1 --2- 172.21.15.38:0/3211898840 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f292406d9c0 0x7f2924151d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:57.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:57.506+0000 7f29227fc640 1 -- 172.21.15.38:0/3211898840 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f29241627f0 msgr2=0x7f29241573c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:57.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:57.506+0000 7f29227fc640 1 --2- 172.21.15.38:0/3211898840 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f29241627f0 0x7f29241573c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:57.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:57.506+0000 7f29227fc640 1 -- 172.21.15.38:0/3211898840 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f292415df00 con 0x7f29240b9e40 2024-09-21T03:44:57.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:57.506+0000 7f29227fc640 1 --2- 172.21.15.38:0/3211898840 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f29240b9e40 0x7f29241f99b0 secure :-1 s=READY pgs=2346 cs=0 l=1 rev1=1 crypto rx=0x7f291800c3a0 tx=0x7f291800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:57.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:57.506+0000 7f290b7fe640 1 -- 172.21.15.38:0/3211898840 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2918017070 con 0x7f29240b9e40 2024-09-21T03:44:57.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:57.506+0000 7f290b7fe640 1 -- 172.21.15.38:0/3211898840 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f29180053c0 con 0x7f29240b9e40 2024-09-21T03:44:57.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:57.506+0000 7f2923fff640 1 -- 172.21.15.38:0/3211898840 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f292415e1e0 con 0x7f29240b9e40 2024-09-21T03:44:57.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:57.506+0000 7f290b7fe640 1 -- 172.21.15.38:0/3211898840 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2918003850 con 0x7f29240b9e40 2024-09-21T03:44:57.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:57.506+0000 7f2923fff640 1 -- 172.21.15.38:0/3211898840 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2924151550 con 0x7f29240b9e40 2024-09-21T03:44:57.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:57.507+0000 7f2923fff640 1 -- 172.21.15.38:0/3211898840 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29240b7fd0 con 0x7f29240b9e40 2024-09-21T03:44:57.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:57.508+0000 7f290b7fe640 1 -- 172.21.15.38:0/3211898840 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f291801a070 con 0x7f29240b9e40 2024-09-21T03:44:57.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:57.508+0000 7f290b7fe640 1 --2- 172.21.15.38:0/3211898840 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2910078bc0 0x7f291007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:57.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:57.509+0000 7f290b7fe640 1 -- 172.21.15.38:0/3211898840 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 6101+0+0 (secure 0 0 0) 0x7f291809a1b0 con 0x7f29240b9e40 2024-09-21T03:44:57.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:57.509+0000 7f2922ffd640 1 --2- 172.21.15.38:0/3211898840 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2910078bc0 0x7f291007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:57.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:57.509+0000 7f2922ffd640 1 --2- 172.21.15.38:0/3211898840 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2910078bc0 0x7f291007b0d0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f2914035000 tx=0x7f2914038040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:57.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:57.513+0000 7f290b7fe640 1 -- 172.21.15.38:0/3211898840 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f2924163170 con 0x7f29240b9e40 2024-09-21T03:44:57.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:57.704+0000 7f2923fff640 1 -- 172.21.15.38:0/3211898840 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2924160000 con 0x7f29240b9e40 2024-09-21T03:44:57.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:57.705+0000 7f290b7fe640 1 -- 172.21.15.38:0/3211898840 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v145) ==== 74+0+16997 (secure 0 0 0) 0x7f2918061d90 con 0x7f29240b9e40 2024-09-21T03:44:57.707 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:44:57.708 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":145,"fsid":"9d50d66f-656b-4fe2-8e3b-fe53b79c4d43","created":"2024-09-21T03:33:10.677865+0000","modified":"2024-09-21T03:44:56.710499+0000","last_up_change":"2024-09-21T03:33:24.023421+0000","last_in_change":"2024-09-21T03:33:14.296193+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T03:33:22.992573+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-09-21T03:44:54.386654+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"145","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-09-21T03:44:55.667528+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"145","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dbab183f-7e63-48b8-960a-ae55fd5bf565","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6801","nonce":2381639433}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6803","nonce":2381639433}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6807","nonce":2381639433}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6805","nonce":2381639433}]},"public_addr":"172.21.15.38:6801/2381639433","cluster_addr":"172.21.15.38:6803/2381639433","heartbeat_back_addr":"172.21.15.38:6807/2381639433","heartbeat_front_addr":"172.21.15.38:6805/2381639433","state":["exists","up"]},{"osd":1,"uuid":"fa1c0e8a-ebc1-4359-8146-bd5342abbc16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6817","nonce":1491818732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6819","nonce":1491818732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6823","nonce":1491818732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6821","nonce":1491818732}]},"public_addr":"172.21.15.38:6817/1491818732","cluster_addr":"172.21.15.38:6819/1491818732","heartbeat_back_addr":"172.21.15.38:6823/1491818732","heartbeat_front_addr":"172.21.15.38:6821/1491818732","state":["exists","up"]},{"osd":2,"uuid":"2ed3cd2d-385d-4fc6-9869-214552a84803","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6825","nonce":2577858902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6827","nonce":2577858902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6831","nonce":2577858902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6829","nonce":2577858902}]},"public_addr":"172.21.15.38:6825/2577858902","cluster_addr":"172.21.15.38:6827/2577858902","heartbeat_back_addr":"172.21.15.38:6831/2577858902","heartbeat_front_addr":"172.21.15.38:6829/2577858902","state":["exists","up"]},{"osd":3,"uuid":"62d1d95e-919d-4250-be71-1d34cfa70dbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6809","nonce":1286810294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6811","nonce":1286810294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6815","nonce":1286810294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6813","nonce":1286810294}]},"public_addr":"172.21.15.38:6809/1286810294","cluster_addr":"172.21.15.38:6811/1286810294","heartbeat_back_addr":"172.21.15.38:6815/1286810294","heartbeat_front_addr":"172.21.15.38:6813/1286810294","state":["exists","up"]},{"osd":4,"uuid":"7c9f9314-99d7-4d73-8392-37809a9f54b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6803","nonce":3401837571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6805","nonce":3401837571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6809","nonce":3401837571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6807","nonce":3401837571}]},"public_addr":"172.21.15.195:6803/3401837571","cluster_addr":"172.21.15.195:6805/3401837571","heartbeat_back_addr":"172.21.15.195:6809/3401837571","heartbeat_front_addr":"172.21.15.195:6807/3401837571","state":["exists","up"]},{"osd":5,"uuid":"bde4fe9f-8414-4bba-bf4a-43973d1976bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6811","nonce":2080960180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6813","nonce":2080960180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6817","nonce":2080960180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6815","nonce":2080960180}]},"public_addr":"172.21.15.195:6811/2080960180","cluster_addr":"172.21.15.195:6813/2080960180","heartbeat_back_addr":"172.21.15.195:6817/2080960180","heartbeat_front_addr":"172.21.15.195:6815/2080960180","state":["exists","up"]},{"osd":6,"uuid":"6ee6d7f4-1a5f-4a4c-ac16-48fe05f6a002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6827","nonce":4055825675}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6829","nonce":4055825675}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6833","nonce":4055825675}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6831","nonce":4055825675}]},"public_addr":"172.21.15.195:6827/4055825675","cluster_addr":"172.21.15.195:6829/4055825675","heartbeat_back_addr":"172.21.15.195:6833/4055825675","heartbeat_front_addr":"172.21.15.195:6831/4055825675","state":["exists","up"]},{"osd":7,"uuid":"efeb5f51-329a-4484-8290-92e7d3503d05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6819","nonce":1606106342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6821","nonce":1606106342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6825","nonce":1606106342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6823","nonce":1606106342}]},"public_addr":"172.21.15.195:6819/1606106342","cluster_addr":"172.21.15.195:6821/1606106342","heartbeat_back_addr":"172.21.15.195:6825/1606106342","heartbeat_front_addr":"172.21.15.195:6823/1606106342","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.901343+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:21.115003+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.901623+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.835238+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.132199+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:19.817560+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.829690+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:21.087300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T03:44:57.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:57.707+0000 7f2923fff640 1 -- 172.21.15.38:0/3211898840 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2910078bc0 msgr2=0x7f291007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:57.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:57.707+0000 7f2923fff640 1 --2- 172.21.15.38:0/3211898840 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2910078bc0 0x7f291007b0d0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f2914035000 tx=0x7f2914038040 comp rx=0 tx=0).stop 2024-09-21T03:44:57.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:57.707+0000 7f2923fff640 1 -- 172.21.15.38:0/3211898840 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f29240b9e40 msgr2=0x7f29241f99b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:57.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:57.707+0000 7f2923fff640 1 --2- 172.21.15.38:0/3211898840 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f29240b9e40 0x7f29241f99b0 secure :-1 s=READY pgs=2346 cs=0 l=1 rev1=1 crypto rx=0x7f291800c3a0 tx=0x7f291800c860 comp rx=0 tx=0).stop 2024-09-21T03:44:57.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:57.707+0000 7f2923fff640 1 -- 172.21.15.38:0/3211898840 shutdown_connections 2024-09-21T03:44:57.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:57.707+0000 7f2923fff640 1 --2- 172.21.15.38:0/3211898840 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f29241627f0 0x7f29241573c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:57.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:57.707+0000 7f2923fff640 1 --2- 172.21.15.38:0/3211898840 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f29240b9e40 0x7f29241f99b0 unknown :-1 s=CLOSED pgs=2346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:57.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:57.707+0000 7f2923fff640 1 --2- 172.21.15.38:0/3211898840 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2910078bc0 0x7f291007b0d0 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:57.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:57.707+0000 7f2923fff640 1 --2- 172.21.15.38:0/3211898840 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f292406d9c0 0x7f2924151d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:57.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:57.707+0000 7f2923fff640 1 -- 172.21.15.38:0/3211898840 >> 172.21.15.38:0/3211898840 conn(0x7f29240d1930 msgr2=0x7f29241616e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:57.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:57.707+0000 7f2923fff640 1 -- 172.21.15.38:0/3211898840 shutdown_connections 2024-09-21T03:44:57.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:57.707+0000 7f2923fff640 1 -- 172.21.15.38:0/3211898840 wait complete. 2024-09-21T03:44:57.721 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T03:44:57.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:57.837+0000 7f568acd5640 1 -- 172.21.15.38:0/2337142506 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5670003c10 msgr2=0x7f5670004060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:57.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:57.837+0000 7f568acd5640 1 --2- 172.21.15.38:0/2337142506 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5670003c10 0x7f5670004060 secure :-1 s=READY pgs=2348 cs=0 l=1 rev1=1 crypto rx=0x7f5674001f30 tx=0x7f5670005830 comp rx=0 tx=0).stop 2024-09-21T03:44:57.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:57.837+0000 7f5689272640 1 -- 172.21.15.38:0/2337142506 reap_dead start 2024-09-21T03:44:57.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:57.837+0000 7f568acd5640 1 -- 172.21.15.38:0/2337142506 shutdown_connections 2024-09-21T03:44:57.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:57.837+0000 7f568acd5640 1 -- 172.21.15.38:0/2337142506 >> 172.21.15.38:0/2337142506 conn(0x7f56840b6b20 msgr2=0x7f56840d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:57.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:57.837+0000 7f568acd5640 1 -- 172.21.15.38:0/2337142506 shutdown_connections 2024-09-21T03:44:57.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:57.837+0000 7f568acd5640 1 -- 172.21.15.38:0/2337142506 wait complete. 2024-09-21T03:44:57.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:57.839+0000 7f568acd5640 1 Processor -- start 2024-09-21T03:44:57.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:57.839+0000 7f568acd5640 1 -- start start 2024-09-21T03:44:57.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:57.840+0000 7f568acd5640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5670003c10 0x7f56840cab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:57.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:57.840+0000 7f568acd5640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f56840cc370 0x7f5684151830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:57.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:57.840+0000 7f568acd5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f56840cd180 0x7f56840cd590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:57.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:57.840+0000 7f568acd5640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f5684162810 con 0x7f56840cd180 2024-09-21T03:44:57.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:57.840+0000 7f568acd5640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f5684162690 con 0x7f56840cc370 2024-09-21T03:44:57.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:57.840+0000 7f568acd5640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f5684162990 con 0x7f5670003c10 2024-09-21T03:44:57.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:57.841+0000 7f5689272640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f56840cd180 0x7f56840cd590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:57.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:57.841+0000 7f5689272640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f56840cd180 0x7f56840cd590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40486/0 (socket says 172.21.15.38:40486) 2024-09-21T03:44:57.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:57.841+0000 7f5689272640 1 -- 172.21.15.38:0/4187977169 learned_addr learned my addr 172.21.15.38:0/4187977169 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:44:57.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:57.841+0000 7f5688a71640 1 --2- 172.21.15.38:0/4187977169 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5670003c10 0x7f56840cab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:57.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:57.841+0000 7f5683fff640 1 --2- 172.21.15.38:0/4187977169 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f56840cc370 0x7f5684151830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:57.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:57.841+0000 7f5689272640 1 -- 172.21.15.38:0/4187977169 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5670003c10 msgr2=0x7f56840cab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:57.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:57.841+0000 7f5689272640 1 --2- 172.21.15.38:0/4187977169 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5670003c10 0x7f56840cab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:57.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:57.841+0000 7f5689272640 1 -- 172.21.15.38:0/4187977169 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f56840cc370 msgr2=0x7f5684151830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:57.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:57.841+0000 7f5689272640 1 --2- 172.21.15.38:0/4187977169 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f56840cc370 0x7f5684151830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:57.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:57.841+0000 7f5689272640 1 -- 172.21.15.38:0/4187977169 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56840cb2d0 con 0x7f56840cd180 2024-09-21T03:44:57.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:57.841+0000 7f5689272640 1 --2- 172.21.15.38:0/4187977169 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f56840cd180 0x7f56840cd590 secure :-1 s=READY pgs=2349 cs=0 l=1 rev1=1 crypto rx=0x7f56840d0300 tx=0x7f567c0049d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:57.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:57.841+0000 7f56817fa640 1 -- 172.21.15.38:0/4187977169 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f567c011870 con 0x7f56840cd180 2024-09-21T03:44:57.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:57.841+0000 7f56817fa640 1 -- 172.21.15.38:0/4187977169 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f567c017070 con 0x7f56840cd180 2024-09-21T03:44:57.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:57.841+0000 7f56817fa640 1 -- 172.21.15.38:0/4187977169 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f567c011d80 con 0x7f56840cd180 2024-09-21T03:44:57.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:57.841+0000 7f568acd5640 1 -- 172.21.15.38:0/4187977169 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56840cb5b0 con 0x7f56840cd180 2024-09-21T03:44:57.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:57.841+0000 7f568acd5640 1 -- 172.21.15.38:0/4187977169 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56840cbb70 con 0x7f56840cd180 2024-09-21T03:44:57.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:57.842+0000 7f568acd5640 1 -- 172.21.15.38:0/4187977169 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5684162f00 con 0x7f56840cd180 2024-09-21T03:44:57.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:57.843+0000 7f56817fa640 1 -- 172.21.15.38:0/4187977169 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f567c01c460 con 0x7f56840cd180 2024-09-21T03:44:57.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:57.843+0000 7f56817fa640 1 --2- 172.21.15.38:0/4187977169 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5664078b90 0x7f566407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:57.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:57.844+0000 7f56817fa640 1 -- 172.21.15.38:0/4187977169 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 6101+0+0 (secure 0 0 0) 0x7f567c09b7a0 con 0x7f56840cd180 2024-09-21T03:44:57.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:57.844+0000 7f5688a71640 1 --2- 172.21.15.38:0/4187977169 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5664078b90 0x7f566407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:57.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:57.844+0000 7f5688a71640 1 --2- 172.21.15.38:0/4187977169 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5664078b90 0x7f566407b0a0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f5674033000 tx=0x7f56740016e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:57.850 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:57.848+0000 7f56817fa640 1 -- 172.21.15.38:0/4187977169 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f5684162bd0 con 0x7f56840cd180 2024-09-21T03:44:58.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:58.037+0000 7f568acd5640 1 -- 172.21.15.38:0/4187977169 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5684151d70 con 0x7f56840cd180 2024-09-21T03:44:58.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:58.038+0000 7f56817fa640 1 -- 172.21.15.38:0/4187977169 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v145) ==== 74+0+16997 (secure 0 0 0) 0x7f567c063a40 con 0x7f56840cd180 2024-09-21T03:44:58.040 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:44:58.041 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":145,"fsid":"9d50d66f-656b-4fe2-8e3b-fe53b79c4d43","created":"2024-09-21T03:33:10.677865+0000","modified":"2024-09-21T03:44:56.710499+0000","last_up_change":"2024-09-21T03:33:24.023421+0000","last_in_change":"2024-09-21T03:33:14.296193+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T03:33:22.992573+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-09-21T03:44:54.386654+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"145","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-09-21T03:44:55.667528+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"145","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dbab183f-7e63-48b8-960a-ae55fd5bf565","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6801","nonce":2381639433}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6803","nonce":2381639433}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6807","nonce":2381639433}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6805","nonce":2381639433}]},"public_addr":"172.21.15.38:6801/2381639433","cluster_addr":"172.21.15.38:6803/2381639433","heartbeat_back_addr":"172.21.15.38:6807/2381639433","heartbeat_front_addr":"172.21.15.38:6805/2381639433","state":["exists","up"]},{"osd":1,"uuid":"fa1c0e8a-ebc1-4359-8146-bd5342abbc16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6817","nonce":1491818732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6819","nonce":1491818732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6823","nonce":1491818732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6821","nonce":1491818732}]},"public_addr":"172.21.15.38:6817/1491818732","cluster_addr":"172.21.15.38:6819/1491818732","heartbeat_back_addr":"172.21.15.38:6823/1491818732","heartbeat_front_addr":"172.21.15.38:6821/1491818732","state":["exists","up"]},{"osd":2,"uuid":"2ed3cd2d-385d-4fc6-9869-214552a84803","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6825","nonce":2577858902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6827","nonce":2577858902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6831","nonce":2577858902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6829","nonce":2577858902}]},"public_addr":"172.21.15.38:6825/2577858902","cluster_addr":"172.21.15.38:6827/2577858902","heartbeat_back_addr":"172.21.15.38:6831/2577858902","heartbeat_front_addr":"172.21.15.38:6829/2577858902","state":["exists","up"]},{"osd":3,"uuid":"62d1d95e-919d-4250-be71-1d34cfa70dbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6809","nonce":1286810294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6811","nonce":1286810294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6815","nonce":1286810294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6813","nonce":1286810294}]},"public_addr":"172.21.15.38:6809/1286810294","cluster_addr":"172.21.15.38:6811/1286810294","heartbeat_back_addr":"172.21.15.38:6815/1286810294","heartbeat_front_addr":"172.21.15.38:6813/1286810294","state":["exists","up"]},{"osd":4,"uuid":"7c9f9314-99d7-4d73-8392-37809a9f54b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6803","nonce":3401837571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6805","nonce":3401837571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6809","nonce":3401837571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6807","nonce":3401837571}]},"public_addr":"172.21.15.195:6803/3401837571","cluster_addr":"172.21.15.195:6805/3401837571","heartbeat_back_addr":"172.21.15.195:6809/3401837571","heartbeat_front_addr":"172.21.15.195:6807/3401837571","state":["exists","up"]},{"osd":5,"uuid":"bde4fe9f-8414-4bba-bf4a-43973d1976bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6811","nonce":2080960180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6813","nonce":2080960180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6817","nonce":2080960180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6815","nonce":2080960180}]},"public_addr":"172.21.15.195:6811/2080960180","cluster_addr":"172.21.15.195:6813/2080960180","heartbeat_back_addr":"172.21.15.195:6817/2080960180","heartbeat_front_addr":"172.21.15.195:6815/2080960180","state":["exists","up"]},{"osd":6,"uuid":"6ee6d7f4-1a5f-4a4c-ac16-48fe05f6a002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6827","nonce":4055825675}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6829","nonce":4055825675}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6833","nonce":4055825675}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6831","nonce":4055825675}]},"public_addr":"172.21.15.195:6827/4055825675","cluster_addr":"172.21.15.195:6829/4055825675","heartbeat_back_addr":"172.21.15.195:6833/4055825675","heartbeat_front_addr":"172.21.15.195:6831/4055825675","state":["exists","up"]},{"osd":7,"uuid":"efeb5f51-329a-4484-8290-92e7d3503d05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6819","nonce":1606106342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6821","nonce":1606106342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6825","nonce":1606106342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6823","nonce":1606106342}]},"public_addr":"172.21.15.195:6819/1606106342","cluster_addr":"172.21.15.195:6821/1606106342","heartbeat_back_addr":"172.21.15.195:6825/1606106342","heartbeat_front_addr":"172.21.15.195:6823/1606106342","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.901343+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:21.115003+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.901623+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.835238+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.132199+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:19.817560+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.829690+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:21.087300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T03:44:58.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:58.040+0000 7f568acd5640 1 -- 172.21.15.38:0/4187977169 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5664078b90 msgr2=0x7f566407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:58.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:58.040+0000 7f568acd5640 1 --2- 172.21.15.38:0/4187977169 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5664078b90 0x7f566407b0a0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f5674033000 tx=0x7f56740016e0 comp rx=0 tx=0).stop 2024-09-21T03:44:58.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:58.040+0000 7f568acd5640 1 -- 172.21.15.38:0/4187977169 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f56840cd180 msgr2=0x7f56840cd590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:58.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:58.040+0000 7f568acd5640 1 --2- 172.21.15.38:0/4187977169 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f56840cd180 0x7f56840cd590 secure :-1 s=READY pgs=2349 cs=0 l=1 rev1=1 crypto rx=0x7f56840d0300 tx=0x7f567c0049d0 comp rx=0 tx=0).stop 2024-09-21T03:44:58.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:58.040+0000 7f568acd5640 1 -- 172.21.15.38:0/4187977169 shutdown_connections 2024-09-21T03:44:58.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:58.040+0000 7f568acd5640 1 --2- 172.21.15.38:0/4187977169 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f56840cd180 0x7f56840cd590 unknown :-1 s=CLOSED pgs=2349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:58.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:58.040+0000 7f568acd5640 1 --2- 172.21.15.38:0/4187977169 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f56840cc370 0x7f5684151830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:58.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:58.040+0000 7f568acd5640 1 --2- 172.21.15.38:0/4187977169 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5664078b90 0x7f566407b0a0 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:58.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:58.040+0000 7f568acd5640 1 --2- 172.21.15.38:0/4187977169 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5670003c10 0x7f56840cab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:58.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:58.040+0000 7f568acd5640 1 -- 172.21.15.38:0/4187977169 >> 172.21.15.38:0/4187977169 conn(0x7f56840b6b20 msgr2=0x7f568406cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:58.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:58.040+0000 7f568acd5640 1 -- 172.21.15.38:0/4187977169 shutdown_connections 2024-09-21T03:44:58.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:58.040+0000 7f568acd5640 1 -- 172.21.15.38:0/4187977169 wait complete. 2024-09-21T03:44:58.055 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-21T03:44:58.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:58.173+0000 7fa6ae6cc640 1 -- 172.21.15.38:0/4199565666 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa6a80b9ad0 msgr2=0x7fa6a80d0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:58.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:58.173+0000 7fa6ae6cc640 1 --2- 172.21.15.38:0/4199565666 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa6a80b9ad0 0x7fa6a80d0be0 secure :-1 s=READY pgs=2350 cs=0 l=1 rev1=1 crypto rx=0x7fa6a80cfba0 tx=0x7fa69c02a4b0 comp rx=0 tx=0).stop 2024-09-21T03:44:58.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:58.173+0000 7fa6ae6cc640 1 -- 172.21.15.38:0/4199565666 shutdown_connections 2024-09-21T03:44:58.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:58.173+0000 7fa6ae6cc640 1 --2- 172.21.15.38:0/4199565666 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa6a80d1120 0x7fa6a80ceb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:58.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:58.173+0000 7fa6ae6cc640 1 --2- 172.21.15.38:0/4199565666 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa6a80b9ad0 0x7fa6a80d0be0 unknown :-1 s=CLOSED pgs=2350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:58.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:58.173+0000 7fa6ae6cc640 1 --2- 172.21.15.38:0/4199565666 >> v2:172.21.15.195:3300/0 conn(0x7fa6a815ee80 0x7fa6a80d0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:58.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:58.173+0000 7fa6ae6cc640 1 -- 172.21.15.38:0/4199565666 >> 172.21.15.38:0/4199565666 conn(0x7fa6a809d830 msgr2=0x7fa6a80b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:58.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:58.173+0000 7fa6ae6cc640 1 -- 172.21.15.38:0/4199565666 shutdown_connections 2024-09-21T03:44:58.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:58.173+0000 7fa6ae6cc640 1 -- 172.21.15.38:0/4199565666 wait complete. 2024-09-21T03:44:58.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:58.175+0000 7fa6ae6cc640 1 Processor -- start 2024-09-21T03:44:58.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:58.175+0000 7fa6ae6cc640 1 -- start start 2024-09-21T03:44:58.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:58.176+0000 7fa6ae6cc640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa6a80b9ad0 0x7fa6a8157f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:58.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:58.176+0000 7fa6ae6cc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa6a80d1120 0x7fa6a80d3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:58.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:58.176+0000 7fa6ae6cc640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa6a815ee80 0x7fa6a814f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:58.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:58.176+0000 7fa6ae6cc640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa6a80cf520 con 0x7fa6a80d1120 2024-09-21T03:44:58.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:58.176+0000 7fa6ae6cc640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fa6a80cf3a0 con 0x7fa6a80b9ad0 2024-09-21T03:44:58.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:58.176+0000 7fa6ae6cc640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fa6a80b6580 con 0x7fa6a815ee80 2024-09-21T03:44:58.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:58.176+0000 7fa6a77fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa6a80d1120 0x7fa6a80d3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:58.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:58.176+0000 7fa6a77fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa6a80d1120 0x7fa6a80d3dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40514/0 (socket says 172.21.15.38:40514) 2024-09-21T03:44:58.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:58.176+0000 7fa6a77fe640 1 -- 172.21.15.38:0/2297518522 learned_addr learned my addr 172.21.15.38:0/2297518522 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:44:58.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:58.176+0000 7fa6a7fff640 1 --2- 172.21.15.38:0/2297518522 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa6a80b9ad0 0x7fa6a8157f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:58.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:58.176+0000 7fa6acc69640 1 --2- 172.21.15.38:0/2297518522 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa6a815ee80 0x7fa6a814f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:58.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:58.176+0000 7fa6a77fe640 1 -- 172.21.15.38:0/2297518522 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa6a815ee80 msgr2=0x7fa6a814f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:58.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:58.176+0000 7fa6a77fe640 1 --2- 172.21.15.38:0/2297518522 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa6a815ee80 0x7fa6a814f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:58.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:58.177+0000 7fa6a77fe640 1 -- 172.21.15.38:0/2297518522 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa6a80b9ad0 msgr2=0x7fa6a8157f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:58.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:58.177+0000 7fa6a77fe640 1 --2- 172.21.15.38:0/2297518522 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa6a80b9ad0 0x7fa6a8157f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:58.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:58.177+0000 7fa6a77fe640 1 -- 172.21.15.38:0/2297518522 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6a806f0a0 con 0x7fa6a80d1120 2024-09-21T03:44:58.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:58.177+0000 7fa6acc69640 1 --2- 172.21.15.38:0/2297518522 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa6a815ee80 0x7fa6a814f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:44:58.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:58.177+0000 7fa6a77fe640 1 --2- 172.21.15.38:0/2297518522 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa6a80d1120 0x7fa6a80d3dd0 secure :-1 s=READY pgs=2351 cs=0 l=1 rev1=1 crypto rx=0x7fa69c0099d0 tx=0x7fa69c0052f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:58.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:58.177+0000 7fa6a4ff9640 1 -- 172.21.15.38:0/2297518522 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa69c031b50 con 0x7fa6a80d1120 2024-09-21T03:44:58.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:58.177+0000 7fa6a4ff9640 1 -- 172.21.15.38:0/2297518522 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa69c031de0 con 0x7fa6a80d1120 2024-09-21T03:44:58.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:58.177+0000 7fa6ae6cc640 1 -- 172.21.15.38:0/2297518522 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6a806f380 con 0x7fa6a80d1120 2024-09-21T03:44:58.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:58.177+0000 7fa6a4ff9640 1 -- 172.21.15.38:0/2297518522 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa69c033330 con 0x7fa6a80d1120 2024-09-21T03:44:58.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:58.177+0000 7fa6ae6cc640 1 -- 172.21.15.38:0/2297518522 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6a81f95c0 con 0x7fa6a80d1120 2024-09-21T03:44:58.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:58.178+0000 7fa6ae6cc640 1 -- 172.21.15.38:0/2297518522 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6a81f9a20 con 0x7fa6a80d1120 2024-09-21T03:44:58.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:58.179+0000 7fa6a4ff9640 1 -- 172.21.15.38:0/2297518522 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa69c0334d0 con 0x7fa6a80d1120 2024-09-21T03:44:58.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:58.180+0000 7fa6a4ff9640 1 --2- 172.21.15.38:0/2297518522 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa694078b90 0x7fa69407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:58.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:58.180+0000 7fa6a4ff9640 1 -- 172.21.15.38:0/2297518522 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 6101+0+0 (secure 0 0 0) 0x7fa69c0bc2e0 con 0x7fa6a80d1120 2024-09-21T03:44:58.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:58.180+0000 7fa6a7fff640 1 --2- 172.21.15.38:0/2297518522 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa694078b90 0x7fa69407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:58.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:58.180+0000 7fa6a7fff640 1 --2- 172.21.15.38:0/2297518522 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa694078b90 0x7fa69407b0a0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7fa69800a490 tx=0x7fa69800a420 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:58.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:58.184+0000 7fa6a4ff9640 1 -- 172.21.15.38:0/2297518522 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fa69c03f030 con 0x7fa6a80d1120 2024-09-21T03:44:58.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:58.416+0000 7fa6ae6cc640 1 -- 172.21.15.38:0/2297518522 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fa6a806a7c0 con 0x7fa6a80d1120 2024-09-21T03:44:59.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:59.365+0000 7fa6a4ff9640 1 -- 172.21.15.38:0/2297518522 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v342) ==== 118+0+0 (secure 0 0 0) 0x7fa69c084620 con 0x7fa6a80d1120 2024-09-21T03:44:59.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:59.368+0000 7fa6ae6cc640 1 -- 172.21.15.38:0/2297518522 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa694078b90 msgr2=0x7fa69407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:59.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:59.368+0000 7fa6ae6cc640 1 --2- 172.21.15.38:0/2297518522 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa694078b90 0x7fa69407b0a0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7fa69800a490 tx=0x7fa69800a420 comp rx=0 tx=0).stop 2024-09-21T03:44:59.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:59.368+0000 7fa6ae6cc640 1 -- 172.21.15.38:0/2297518522 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa6a80d1120 msgr2=0x7fa6a80d3dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:59.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:59.368+0000 7fa6ae6cc640 1 --2- 172.21.15.38:0/2297518522 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa6a80d1120 0x7fa6a80d3dd0 secure :-1 s=READY pgs=2351 cs=0 l=1 rev1=1 crypto rx=0x7fa69c0099d0 tx=0x7fa69c0052f0 comp rx=0 tx=0).stop 2024-09-21T03:44:59.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:59.369+0000 7fa6ae6cc640 1 -- 172.21.15.38:0/2297518522 shutdown_connections 2024-09-21T03:44:59.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:59.369+0000 7fa6ae6cc640 1 --2- 172.21.15.38:0/2297518522 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa694078b90 0x7fa69407b0a0 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:59.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:59.369+0000 7fa6ae6cc640 1 --2- 172.21.15.38:0/2297518522 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa6a815ee80 0x7fa6a814f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:59.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:59.369+0000 7fa6ae6cc640 1 --2- 172.21.15.38:0/2297518522 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa6a80d1120 0x7fa6a80d3dd0 unknown :-1 s=CLOSED pgs=2351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:59.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:59.369+0000 7fa6ae6cc640 1 --2- 172.21.15.38:0/2297518522 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa6a80b9ad0 0x7fa6a8157f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:59.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:59.369+0000 7fa6ae6cc640 1 -- 172.21.15.38:0/2297518522 >> 172.21.15.38:0/2297518522 conn(0x7fa6a809d830 msgr2=0x7fa6a81611f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:59.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:59.369+0000 7fa6ae6cc640 1 -- 172.21.15.38:0/2297518522 shutdown_connections 2024-09-21T03:44:59.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:59.369+0000 7fa6ae6cc640 1 -- 172.21.15.38:0/2297518522 wait complete. 2024-09-21T03:44:59.385 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-21T03:44:59.385 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-21T03:44:59.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:59.511+0000 7f3209be4640 1 -- 172.21.15.38:0/786665531 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f320415ee80 msgr2=0x7f32040d0170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:59.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:59.511+0000 7f3209be4640 1 --2- 172.21.15.38:0/786665531 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f320415ee80 0x7f32040d0170 secure :-1 s=READY pgs=2353 cs=0 l=1 rev1=1 crypto rx=0x7f31f4005fd0 tx=0x7f31f402a560 comp rx=0 tx=0).stop 2024-09-21T03:44:59.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:59.511+0000 7f3209be4640 1 -- 172.21.15.38:0/786665531 shutdown_connections 2024-09-21T03:44:59.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:59.511+0000 7f3209be4640 1 --2- 172.21.15.38:0/786665531 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f320415ee80 0x7f32040d0170 unknown :-1 s=CLOSED pgs=2353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:59.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:59.511+0000 7f3209be4640 1 -- 172.21.15.38:0/786665531 >> 172.21.15.38:0/786665531 conn(0x7f320409d830 msgr2=0x7f32040b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:59.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:59.511+0000 7f3209be4640 1 -- 172.21.15.38:0/786665531 shutdown_connections 2024-09-21T03:44:59.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:59.511+0000 7f3209be4640 1 -- 172.21.15.38:0/786665531 wait complete. 2024-09-21T03:44:59.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:59.513+0000 7f3209be4640 1 Processor -- start 2024-09-21T03:44:59.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:59.513+0000 7f3209be4640 1 -- start start 2024-09-21T03:44:59.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:59.514+0000 7f3209be4640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f32040b9ad0 0x7f32040cd630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:59.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:59.514+0000 7f3209be4640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f32040ceb90 0x7f320415ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:59.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:59.514+0000 7f3209be4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f320415ee80 0x7f320406cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:59.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:59.514+0000 7f3209be4640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f32040b6910 con 0x7f320415ee80 2024-09-21T03:44:59.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:59.514+0000 7f3209be4640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f32040b6790 con 0x7f32040ceb90 2024-09-21T03:44:59.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:59.514+0000 7f3209be4640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f32040b6a90 con 0x7f32040b9ad0 2024-09-21T03:44:59.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:59.514+0000 7f3203fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f320415ee80 0x7f320406cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:59.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:59.514+0000 7f3203fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f320415ee80 0x7f320406cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40538/0 (socket says 172.21.15.38:40538) 2024-09-21T03:44:59.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:59.514+0000 7f3203fff640 1 -- 172.21.15.38:0/1120221418 learned_addr learned my addr 172.21.15.38:0/1120221418 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:44:59.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:59.514+0000 7f3202ffd640 1 --2- 172.21.15.38:0/1120221418 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f32040ceb90 0x7f320415ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:59.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:59.514+0000 7f32037fe640 1 --2- 172.21.15.38:0/1120221418 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f32040b9ad0 0x7f32040cd630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:59.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:59.515+0000 7f3203fff640 1 -- 172.21.15.38:0/1120221418 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f32040b9ad0 msgr2=0x7f32040cd630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:59.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:59.515+0000 7f3203fff640 1 --2- 172.21.15.38:0/1120221418 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f32040b9ad0 0x7f32040cd630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:59.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:59.515+0000 7f3203fff640 1 -- 172.21.15.38:0/1120221418 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f32040ceb90 msgr2=0x7f320415ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:59.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:59.515+0000 7f3203fff640 1 --2- 172.21.15.38:0/1120221418 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f32040ceb90 0x7f320415ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:59.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:59.515+0000 7f3203fff640 1 -- 172.21.15.38:0/1120221418 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32040cc8b0 con 0x7f320415ee80 2024-09-21T03:44:59.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:59.515+0000 7f3203fff640 1 --2- 172.21.15.38:0/1120221418 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f320415ee80 0x7f320406cde0 secure :-1 s=READY pgs=2354 cs=0 l=1 rev1=1 crypto rx=0x7f31fc00c700 tx=0x7f31fc00cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:59.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:59.515+0000 7f31ebfff640 1 -- 172.21.15.38:0/1120221418 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31fc003b70 con 0x7f320415ee80 2024-09-21T03:44:59.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:59.515+0000 7f31ebfff640 1 -- 172.21.15.38:0/1120221418 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f31fc003d10 con 0x7f320415ee80 2024-09-21T03:44:59.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:59.515+0000 7f31ebfff640 1 -- 172.21.15.38:0/1120221418 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31fc00aab0 con 0x7f320415ee80 2024-09-21T03:44:59.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:59.515+0000 7f3209be4640 1 -- 172.21.15.38:0/1120221418 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32040ccb50 con 0x7f320415ee80 2024-09-21T03:44:59.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:59.515+0000 7f3209be4640 1 -- 172.21.15.38:0/1120221418 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f320415b0d0 con 0x7f320415ee80 2024-09-21T03:44:59.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:59.516+0000 7f3209be4640 1 -- 172.21.15.38:0/1120221418 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32040cc610 con 0x7f320415ee80 2024-09-21T03:44:59.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:59.517+0000 7f31ebfff640 1 -- 172.21.15.38:0/1120221418 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f31fc00ac50 con 0x7f320415ee80 2024-09-21T03:44:59.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:59.517+0000 7f31ebfff640 1 --2- 172.21.15.38:0/1120221418 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f31f0078bc0 0x7f31f007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:59.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:59.518+0000 7f31ebfff640 1 -- 172.21.15.38:0/1120221418 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 6101+0+0 (secure 0 0 0) 0x7f31fc09abb0 con 0x7f320415ee80 2024-09-21T03:44:59.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:59.518+0000 7f32037fe640 1 --2- 172.21.15.38:0/1120221418 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f31f0078bc0 0x7f31f007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:59.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:59.518+0000 7f32037fe640 1 --2- 172.21.15.38:0/1120221418 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f31f0078bc0 0x7f31f007b0d0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f31f4033000 tx=0x7f31f4004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:59.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:59.522+0000 7f31ebfff640 1 -- 172.21.15.38:0/1120221418 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f31fc062770 con 0x7f320415ee80 2024-09-21T03:44:59.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:44:59.751+0000 7f3209be4640 1 -- 172.21.15.38:0/1120221418 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f320406fa20 con 0x7f320415ee80 2024-09-21T03:45:00.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:00.367+0000 7f31ebfff640 1 -- 172.21.15.38:0/1120221418 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v343) ==== 122+0+0 (secure 0 0 0) 0x7f31fc0671a0 con 0x7f320415ee80 2024-09-21T03:45:00.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:00.370+0000 7f3209be4640 1 -- 172.21.15.38:0/1120221418 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f31f0078bc0 msgr2=0x7f31f007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:00.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:00.370+0000 7f3209be4640 1 --2- 172.21.15.38:0/1120221418 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f31f0078bc0 0x7f31f007b0d0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f31f4033000 tx=0x7f31f4004730 comp rx=0 tx=0).stop 2024-09-21T03:45:00.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:00.370+0000 7f3209be4640 1 -- 172.21.15.38:0/1120221418 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f320415ee80 msgr2=0x7f320406cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:00.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:00.370+0000 7f3209be4640 1 --2- 172.21.15.38:0/1120221418 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f320415ee80 0x7f320406cde0 secure :-1 s=READY pgs=2354 cs=0 l=1 rev1=1 crypto rx=0x7f31fc00c700 tx=0x7f31fc00cbc0 comp rx=0 tx=0).stop 2024-09-21T03:45:00.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:00.370+0000 7f3209be4640 1 -- 172.21.15.38:0/1120221418 shutdown_connections 2024-09-21T03:45:00.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:00.370+0000 7f3209be4640 1 --2- 172.21.15.38:0/1120221418 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f320415ee80 0x7f320406cde0 unknown :-1 s=CLOSED pgs=2354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:00.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:00.370+0000 7f3209be4640 1 --2- 172.21.15.38:0/1120221418 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f32040ceb90 0x7f320415ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:00.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:00.370+0000 7f3209be4640 1 --2- 172.21.15.38:0/1120221418 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f31f0078bc0 0x7f31f007b0d0 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:00.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:00.370+0000 7f3209be4640 1 --2- 172.21.15.38:0/1120221418 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f32040b9ad0 0x7f32040cd630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:00.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:00.370+0000 7f3209be4640 1 -- 172.21.15.38:0/1120221418 >> 172.21.15.38:0/1120221418 conn(0x7f320409d830 msgr2=0x7f320415ffb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:00.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:00.370+0000 7f3209be4640 1 -- 172.21.15.38:0/1120221418 shutdown_connections 2024-09-21T03:45:00.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:00.370+0000 7f3209be4640 1 -- 172.21.15.38:0/1120221418 wait complete. 2024-09-21T03:45:00.386 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-21T03:45:00.386 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:45:00.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:00.504+0000 7f813beb4640 1 -- 172.21.15.38:0/3168288616 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8120004800 msgr2=0x7f8120004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:00.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:00.504+0000 7f813beb4640 1 --2- 172.21.15.38:0/3168288616 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8120004800 0x7f8120004c10 secure :-1 s=READY pgs=2356 cs=0 l=1 rev1=1 crypto rx=0x7f8124006fc0 tx=0x7f8120005830 comp rx=0 tx=0).stop 2024-09-21T03:45:00.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:00.504+0000 7f813a451640 1 -- 172.21.15.38:0/3168288616 reap_dead start 2024-09-21T03:45:00.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:00.504+0000 7f813beb4640 1 -- 172.21.15.38:0/3168288616 shutdown_connections 2024-09-21T03:45:00.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:00.504+0000 7f813beb4640 1 -- 172.21.15.38:0/3168288616 >> 172.21.15.38:0/3168288616 conn(0x7f81340b6b20 msgr2=0x7f81340d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:00.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:00.504+0000 7f813beb4640 1 -- 172.21.15.38:0/3168288616 shutdown_connections 2024-09-21T03:45:00.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:00.504+0000 7f813beb4640 1 -- 172.21.15.38:0/3168288616 wait complete. 2024-09-21T03:45:00.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:00.506+0000 7f813beb4640 1 Processor -- start 2024-09-21T03:45:00.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:00.506+0000 7f813beb4640 1 -- start start 2024-09-21T03:45:00.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:00.507+0000 7f813beb4640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8120004800 0x7f8134158240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:00.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:00.508+0000 7f813beb4640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f813415a910 0x7f8134159eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:00.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:00.508+0000 7f813beb4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f813415b2a0 0x7f81340ce9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:00.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:00.508+0000 7f813beb4640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f81340b31d0 con 0x7f813415b2a0 2024-09-21T03:45:00.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:00.508+0000 7f813beb4640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f81340b3050 con 0x7f813415a910 2024-09-21T03:45:00.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:00.508+0000 7f813beb4640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f81340b3350 con 0x7f8120004800 2024-09-21T03:45:00.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:00.508+0000 7f813a451640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f813415b2a0 0x7f81340ce9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:00.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:00.508+0000 7f813a451640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f813415b2a0 0x7f81340ce9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40548/0 (socket says 172.21.15.38:40548) 2024-09-21T03:45:00.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:00.508+0000 7f813a451640 1 -- 172.21.15.38:0/18483703 learned_addr learned my addr 172.21.15.38:0/18483703 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:45:00.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:00.508+0000 7f8139c50640 1 --2- 172.21.15.38:0/18483703 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8120004800 0x7f8134158240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:00.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:00.508+0000 7f813944f640 1 --2- 172.21.15.38:0/18483703 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f813415a910 0x7f8134159eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:00.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:00.508+0000 7f813a451640 1 -- 172.21.15.38:0/18483703 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8120004800 msgr2=0x7f8134158240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:00.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:00.508+0000 7f813a451640 1 --2- 172.21.15.38:0/18483703 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8120004800 0x7f8134158240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:00.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:00.508+0000 7f813a451640 1 -- 172.21.15.38:0/18483703 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f813415a910 msgr2=0x7f8134159eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:00.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:00.508+0000 7f813a451640 1 --2- 172.21.15.38:0/18483703 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f813415a910 0x7f8134159eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:00.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:00.508+0000 7f813a451640 1 -- 172.21.15.38:0/18483703 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81341f8c10 con 0x7f813415b2a0 2024-09-21T03:45:00.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:00.508+0000 7f813944f640 1 --2- 172.21.15.38:0/18483703 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f813415a910 0x7f8134159eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:45:00.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:00.508+0000 7f813a451640 1 --2- 172.21.15.38:0/18483703 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f813415b2a0 0x7f81340ce9e0 secure :-1 s=READY pgs=2357 cs=0 l=1 rev1=1 crypto rx=0x7f81340d0c40 tx=0x7f812c011bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:00.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:00.508+0000 7f812a7fc640 1 -- 172.21.15.38:0/18483703 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f812c00f920 con 0x7f813415b2a0 2024-09-21T03:45:00.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:00.508+0000 7f812a7fc640 1 -- 172.21.15.38:0/18483703 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f812c00fae0 con 0x7f813415b2a0 2024-09-21T03:45:00.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:00.508+0000 7f812a7fc640 1 -- 172.21.15.38:0/18483703 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f812c005020 con 0x7f813415b2a0 2024-09-21T03:45:00.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:00.508+0000 7f813beb4640 1 -- 172.21.15.38:0/18483703 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81341f8ef0 con 0x7f813415b2a0 2024-09-21T03:45:00.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:00.509+0000 7f813beb4640 1 -- 172.21.15.38:0/18483703 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81341f94b0 con 0x7f813415b2a0 2024-09-21T03:45:00.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:00.509+0000 7f813beb4640 1 -- 172.21.15.38:0/18483703 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81341f8980 con 0x7f813415b2a0 2024-09-21T03:45:00.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:00.510+0000 7f812a7fc640 1 -- 172.21.15.38:0/18483703 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f812c0051c0 con 0x7f813415b2a0 2024-09-21T03:45:00.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:00.511+0000 7f812a7fc640 1 --2- 172.21.15.38:0/18483703 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8118078bc0 0x7f811807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:00.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:00.511+0000 7f812a7fc640 1 -- 172.21.15.38:0/18483703 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 6101+0+0 (secure 0 0 0) 0x7f812c09b7a0 con 0x7f813415b2a0 2024-09-21T03:45:00.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:00.511+0000 7f8139c50640 1 --2- 172.21.15.38:0/18483703 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8118078bc0 0x7f811807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:00.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:00.512+0000 7f8139c50640 1 --2- 172.21.15.38:0/18483703 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8118078bc0 0x7f811807b0d0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f8130000f80 tx=0x7f8130006ef0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:00.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:00.515+0000 7f812a7fc640 1 -- 172.21.15.38:0/18483703 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f812c011e20 con 0x7f813415b2a0 2024-09-21T03:45:00.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:00.747+0000 7f813beb4640 1 -- 172.21.15.38:0/18483703 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f81340b38e0 con 0x7f813415b2a0 2024-09-21T03:45:00.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:00.748+0000 7f812a7fc640 1 -- 172.21.15.38:0/18483703 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 343 v343) ==== 95+0+5239 (secure 0 0 0) 0x7f812c06c020 con 0x7f813415b2a0 2024-09-21T03:45:00.750 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:45:00.750 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":343,"btime":"2024-09-21T03:45:00:366373+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8399,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/3606957623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":3606957623},{"type":"v1","addr":"172.21.15.38:6835","nonce":3606957623}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":338},{"gid":8405,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/456651486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":456651486},{"type":"v1","addr":"172.21.15.195:6837","nonce":456651486}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":338},{"gid":9604,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/2649311840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":2649311840},{"type":"v1","addr":"172.21.15.195:6835","nonce":2649311840}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":338}],"filesystems":[{"mdsmap":{"epoch":343,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:44:56.710446+0000","modified":"2024-09-21T03:44:59.752336+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10719},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10719":{"gid":10719,"name":"c","rank":0,"incarnation":339,"state":"up:active","state_seq":3,"addr":"172.21.15.38:6833/3866292839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":3866292839},{"type":"v1","addr":"172.21.15.38:6833","nonce":3866292839}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10719,"qdb_cluster":[10719]},"id":7}]} 2024-09-21T03:45:00.750 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 343 2024-09-21T03:45:00.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:00.750+0000 7f813beb4640 1 -- 172.21.15.38:0/18483703 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8118078bc0 msgr2=0x7f811807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:00.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:00.750+0000 7f813beb4640 1 --2- 172.21.15.38:0/18483703 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8118078bc0 0x7f811807b0d0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f8130000f80 tx=0x7f8130006ef0 comp rx=0 tx=0).stop 2024-09-21T03:45:00.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:00.750+0000 7f813beb4640 1 -- 172.21.15.38:0/18483703 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f813415b2a0 msgr2=0x7f81340ce9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:00.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:00.750+0000 7f813beb4640 1 --2- 172.21.15.38:0/18483703 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f813415b2a0 0x7f81340ce9e0 secure :-1 s=READY pgs=2357 cs=0 l=1 rev1=1 crypto rx=0x7f81340d0c40 tx=0x7f812c011bd0 comp rx=0 tx=0).stop 2024-09-21T03:45:00.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:00.750+0000 7f813beb4640 1 -- 172.21.15.38:0/18483703 shutdown_connections 2024-09-21T03:45:00.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:00.750+0000 7f813beb4640 1 --2- 172.21.15.38:0/18483703 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f813415b2a0 0x7f81340ce9e0 unknown :-1 s=CLOSED pgs=2357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:00.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:00.750+0000 7f813beb4640 1 --2- 172.21.15.38:0/18483703 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f813415a910 0x7f8134159eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:00.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:00.750+0000 7f813beb4640 1 --2- 172.21.15.38:0/18483703 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f8118078bc0 0x7f811807b0d0 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:00.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:00.750+0000 7f813beb4640 1 --2- 172.21.15.38:0/18483703 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8120004800 0x7f8134158240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:00.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:00.750+0000 7f813beb4640 1 -- 172.21.15.38:0/18483703 >> 172.21.15.38:0/18483703 conn(0x7f81340b6b20 msgr2=0x7f8134157960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:00.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:00.750+0000 7f813beb4640 1 -- 172.21.15.38:0/18483703 shutdown_connections 2024-09-21T03:45:00.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:00.750+0000 7f813beb4640 1 -- 172.21.15.38:0/18483703 wait complete. 2024-09-21T03:45:00.763 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T03:45:00.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:00.879+0000 7f607128b640 1 -- 172.21.15.38:0/2443653223 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6058004800 msgr2=0x7f6058004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:00.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:00.879+0000 7f607128b640 1 --2- 172.21.15.38:0/2443653223 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6058004800 0x7f6058004c10 secure :-1 s=READY pgs=2359 cs=0 l=1 rev1=1 crypto rx=0x7f6054006fa0 tx=0x7f6058005830 comp rx=0 tx=0).stop 2024-09-21T03:45:00.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:00.879+0000 7f607128b640 1 -- 172.21.15.38:0/2443653223 shutdown_connections 2024-09-21T03:45:00.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:00.879+0000 7f607128b640 1 --2- 172.21.15.38:0/2443653223 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6058004800 0x7f6058004c10 unknown :-1 s=CLOSED pgs=2359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:00.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:00.879+0000 7f607128b640 1 -- 172.21.15.38:0/2443653223 >> 172.21.15.38:0/2443653223 conn(0x7f606c0d1140 msgr2=0x7f606c0d1550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:00.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:00.879+0000 7f607128b640 1 -- 172.21.15.38:0/2443653223 shutdown_connections 2024-09-21T03:45:00.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:00.879+0000 7f607128b640 1 -- 172.21.15.38:0/2443653223 wait complete. 2024-09-21T03:45:00.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:00.881+0000 7f607128b640 1 Processor -- start 2024-09-21T03:45:00.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:00.881+0000 7f607128b640 1 -- start start 2024-09-21T03:45:00.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:00.882+0000 7f607128b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6058004800 0x7f606c0b4860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:00.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:00.882+0000 7f607128b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f606c1e6000 0x7f606c1e6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:00.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:00.882+0000 7f607128b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f606c15dcf0 0x7f606c15e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:00.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:00.882+0000 7f607128b640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f606c070290 con 0x7f606c1e6000 2024-09-21T03:45:00.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:00.882+0000 7f607128b640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f606c070110 con 0x7f606c15dcf0 2024-09-21T03:45:00.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:00.882+0000 7f607128b640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f606c073ea0 con 0x7f6058004800 2024-09-21T03:45:00.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:00.882+0000 7f606a7fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f606c1e6000 0x7f606c1e6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:00.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:00.882+0000 7f606a7fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f606c1e6000 0x7f606c1e6410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40562/0 (socket says 172.21.15.38:40562) 2024-09-21T03:45:00.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:00.882+0000 7f606a7fc640 1 -- 172.21.15.38:0/3406312447 learned_addr learned my addr 172.21.15.38:0/3406312447 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:45:00.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:00.882+0000 7f606b7fe640 1 --2- 172.21.15.38:0/3406312447 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f606c15dcf0 0x7f606c15e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:00.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:00.882+0000 7f606a7fc640 1 -- 172.21.15.38:0/3406312447 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6058004800 msgr2=0x7f606c0b4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:00.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:00.882+0000 7f606affd640 1 --2- 172.21.15.38:0/3406312447 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6058004800 0x7f606c0b4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:00.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:00.883+0000 7f606a7fc640 1 --2- 172.21.15.38:0/3406312447 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6058004800 0x7f606c0b4860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:00.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:00.883+0000 7f606a7fc640 1 -- 172.21.15.38:0/3406312447 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f606c15dcf0 msgr2=0x7f606c15e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:00.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:00.883+0000 7f606a7fc640 1 --2- 172.21.15.38:0/3406312447 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f606c15dcf0 0x7f606c15e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:00.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:00.883+0000 7f606a7fc640 1 -- 172.21.15.38:0/3406312447 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f606c15e640 con 0x7f606c1e6000 2024-09-21T03:45:00.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:00.883+0000 7f606b7fe640 1 --2- 172.21.15.38:0/3406312447 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f606c15dcf0 0x7f606c15e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:45:00.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:00.883+0000 7f606a7fc640 1 --2- 172.21.15.38:0/3406312447 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f606c1e6000 0x7f606c1e6410 secure :-1 s=READY pgs=2360 cs=0 l=1 rev1=1 crypto rx=0x7f605402b0a0 tx=0x7f6054001cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:00.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:00.883+0000 7f60637fe640 1 -- 172.21.15.38:0/3406312447 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6054003a00 con 0x7f606c1e6000 2024-09-21T03:45:00.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:00.883+0000 7f60637fe640 1 -- 172.21.15.38:0/3406312447 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6054003c90 con 0x7f606c1e6000 2024-09-21T03:45:00.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:00.883+0000 7f607128b640 1 -- 172.21.15.38:0/3406312447 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f606c1f6660 con 0x7f606c1e6000 2024-09-21T03:45:00.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:00.883+0000 7f60637fe640 1 -- 172.21.15.38:0/3406312447 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f605402e6e0 con 0x7f606c1e6000 2024-09-21T03:45:00.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:00.883+0000 7f607128b640 1 -- 172.21.15.38:0/3406312447 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f606c1f6c20 con 0x7f606c1e6000 2024-09-21T03:45:00.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:00.884+0000 7f607128b640 1 -- 172.21.15.38:0/3406312447 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f606c0743a0 con 0x7f606c1e6000 2024-09-21T03:45:00.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:00.885+0000 7f60637fe640 1 -- 172.21.15.38:0/3406312447 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f6054035020 con 0x7f606c1e6000 2024-09-21T03:45:00.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:00.885+0000 7f60637fe640 1 --2- 172.21.15.38:0/3406312447 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f604c078b90 0x7f604c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:00.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:00.885+0000 7f60637fe640 1 -- 172.21.15.38:0/3406312447 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 6101+0+0 (secure 0 0 0) 0x7f60540bb390 con 0x7f606c1e6000 2024-09-21T03:45:00.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:00.886+0000 7f606affd640 1 --2- 172.21.15.38:0/3406312447 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f604c078b90 0x7f604c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:00.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:00.886+0000 7f606affd640 1 --2- 172.21.15.38:0/3406312447 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f604c078b90 0x7f604c07b0a0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f605c009a90 tx=0x7f605c005f90 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:00.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:00.890+0000 7f60637fe640 1 -- 172.21.15.38:0/3406312447 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f6054047030 con 0x7f606c1e6000 2024-09-21T03:45:01.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:01.077+0000 7f607128b640 1 -- 172.21.15.38:0/3406312447 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f606c074f20 con 0x7f606c1e6000 2024-09-21T03:45:01.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:01.079+0000 7f60637fe640 1 -- 172.21.15.38:0/3406312447 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v146) ==== 74+0+16984 (secure 0 0 0) 0x7f606c15e640 con 0x7f606c1e6000 2024-09-21T03:45:01.081 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:45:01.082 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":146,"fsid":"9d50d66f-656b-4fe2-8e3b-fe53b79c4d43","created":"2024-09-21T03:33:10.677865+0000","modified":"2024-09-21T03:44:58.330065+0000","last_up_change":"2024-09-21T03:33:24.023421+0000","last_in_change":"2024-09-21T03:33:14.296193+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T03:33:22.992573+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-09-21T03:44:54.386654+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"145","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-09-21T03:44:55.667528+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"146","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dbab183f-7e63-48b8-960a-ae55fd5bf565","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6801","nonce":2381639433}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6803","nonce":2381639433}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6807","nonce":2381639433}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6805","nonce":2381639433}]},"public_addr":"172.21.15.38:6801/2381639433","cluster_addr":"172.21.15.38:6803/2381639433","heartbeat_back_addr":"172.21.15.38:6807/2381639433","heartbeat_front_addr":"172.21.15.38:6805/2381639433","state":["exists","up"]},{"osd":1,"uuid":"fa1c0e8a-ebc1-4359-8146-bd5342abbc16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6817","nonce":1491818732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6819","nonce":1491818732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6823","nonce":1491818732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6821","nonce":1491818732}]},"public_addr":"172.21.15.38:6817/1491818732","cluster_addr":"172.21.15.38:6819/1491818732","heartbeat_back_addr":"172.21.15.38:6823/1491818732","heartbeat_front_addr":"172.21.15.38:6821/1491818732","state":["exists","up"]},{"osd":2,"uuid":"2ed3cd2d-385d-4fc6-9869-214552a84803","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6825","nonce":2577858902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6827","nonce":2577858902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6831","nonce":2577858902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6829","nonce":2577858902}]},"public_addr":"172.21.15.38:6825/2577858902","cluster_addr":"172.21.15.38:6827/2577858902","heartbeat_back_addr":"172.21.15.38:6831/2577858902","heartbeat_front_addr":"172.21.15.38:6829/2577858902","state":["exists","up"]},{"osd":3,"uuid":"62d1d95e-919d-4250-be71-1d34cfa70dbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6809","nonce":1286810294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6811","nonce":1286810294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6815","nonce":1286810294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6813","nonce":1286810294}]},"public_addr":"172.21.15.38:6809/1286810294","cluster_addr":"172.21.15.38:6811/1286810294","heartbeat_back_addr":"172.21.15.38:6815/1286810294","heartbeat_front_addr":"172.21.15.38:6813/1286810294","state":["exists","up"]},{"osd":4,"uuid":"7c9f9314-99d7-4d73-8392-37809a9f54b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6803","nonce":3401837571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6805","nonce":3401837571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6809","nonce":3401837571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6807","nonce":3401837571}]},"public_addr":"172.21.15.195:6803/3401837571","cluster_addr":"172.21.15.195:6805/3401837571","heartbeat_back_addr":"172.21.15.195:6809/3401837571","heartbeat_front_addr":"172.21.15.195:6807/3401837571","state":["exists","up"]},{"osd":5,"uuid":"bde4fe9f-8414-4bba-bf4a-43973d1976bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6811","nonce":2080960180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6813","nonce":2080960180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6817","nonce":2080960180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6815","nonce":2080960180}]},"public_addr":"172.21.15.195:6811/2080960180","cluster_addr":"172.21.15.195:6813/2080960180","heartbeat_back_addr":"172.21.15.195:6817/2080960180","heartbeat_front_addr":"172.21.15.195:6815/2080960180","state":["exists","up"]},{"osd":6,"uuid":"6ee6d7f4-1a5f-4a4c-ac16-48fe05f6a002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6827","nonce":4055825675}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6829","nonce":4055825675}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6833","nonce":4055825675}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6831","nonce":4055825675}]},"public_addr":"172.21.15.195:6827/4055825675","cluster_addr":"172.21.15.195:6829/4055825675","heartbeat_back_addr":"172.21.15.195:6833/4055825675","heartbeat_front_addr":"172.21.15.195:6831/4055825675","state":["exists","up"]},{"osd":7,"uuid":"efeb5f51-329a-4484-8290-92e7d3503d05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6819","nonce":1606106342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6821","nonce":1606106342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6825","nonce":1606106342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6823","nonce":1606106342}]},"public_addr":"172.21.15.195:6819/1606106342","cluster_addr":"172.21.15.195:6821/1606106342","heartbeat_back_addr":"172.21.15.195:6825/1606106342","heartbeat_front_addr":"172.21.15.195:6823/1606106342","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.901343+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:21.115003+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.901623+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.835238+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.132199+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:19.817560+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.829690+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:21.087300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T03:45:01.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:01.081+0000 7f607128b640 1 -- 172.21.15.38:0/3406312447 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f604c078b90 msgr2=0x7f604c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:01.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:01.081+0000 7f607128b640 1 --2- 172.21.15.38:0/3406312447 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f604c078b90 0x7f604c07b0a0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f605c009a90 tx=0x7f605c005f90 comp rx=0 tx=0).stop 2024-09-21T03:45:01.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:01.081+0000 7f607128b640 1 -- 172.21.15.38:0/3406312447 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f606c1e6000 msgr2=0x7f606c1e6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:01.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:01.081+0000 7f607128b640 1 --2- 172.21.15.38:0/3406312447 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f606c1e6000 0x7f606c1e6410 secure :-1 s=READY pgs=2360 cs=0 l=1 rev1=1 crypto rx=0x7f605402b0a0 tx=0x7f6054001cf0 comp rx=0 tx=0).stop 2024-09-21T03:45:01.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:01.081+0000 7f607128b640 1 -- 172.21.15.38:0/3406312447 shutdown_connections 2024-09-21T03:45:01.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:01.081+0000 7f607128b640 1 --2- 172.21.15.38:0/3406312447 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f606c15dcf0 0x7f606c15e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:01.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:01.081+0000 7f607128b640 1 --2- 172.21.15.38:0/3406312447 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f606c1e6000 0x7f606c1e6410 unknown :-1 s=CLOSED pgs=2360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:01.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:01.081+0000 7f607128b640 1 --2- 172.21.15.38:0/3406312447 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f604c078b90 0x7f604c07b0a0 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:01.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:01.081+0000 7f607128b640 1 --2- 172.21.15.38:0/3406312447 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6058004800 0x7f606c0b4860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:01.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:01.081+0000 7f607128b640 1 -- 172.21.15.38:0/3406312447 >> 172.21.15.38:0/3406312447 conn(0x7f606c0d1140 msgr2=0x7f606c0b1120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:01.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:01.081+0000 7f607128b640 1 -- 172.21.15.38:0/3406312447 shutdown_connections 2024-09-21T03:45:01.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:01.081+0000 7f607128b640 1 -- 172.21.15.38:0/3406312447 wait complete. 2024-09-21T03:45:01.095 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-21T03:45:01.096 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T03:45:01.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:01.209+0000 7f98d43e6640 1 -- 172.21.15.38:0/3015038075 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f98cc162f00 msgr2=0x7f98cc0d24d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:01.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:01.209+0000 7f98d43e6640 1 --2- 172.21.15.38:0/3015038075 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f98cc162f00 0x7f98cc0d24d0 secure :-1 s=READY pgs=2361 cs=0 l=1 rev1=1 crypto rx=0x7f98c8005fd0 tx=0x7f98c802a5d0 comp rx=0 tx=0).stop 2024-09-21T03:45:01.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:01.209+0000 7f98d43e6640 1 -- 172.21.15.38:0/3015038075 shutdown_connections 2024-09-21T03:45:01.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:01.209+0000 7f98d43e6640 1 --2- 172.21.15.38:0/3015038075 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f98cc162280 0x7f98cc162690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:01.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:01.209+0000 7f98d43e6640 1 --2- 172.21.15.38:0/3015038075 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f98cc162f00 0x7f98cc0d24d0 unknown :-1 s=CLOSED pgs=2361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:01.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:01.209+0000 7f98d43e6640 1 -- 172.21.15.38:0/3015038075 >> 172.21.15.38:0/3015038075 conn(0x7f98cc0b6b20 msgr2=0x7f98cc0d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:01.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:01.210+0000 7f98d43e6640 1 -- 172.21.15.38:0/3015038075 shutdown_connections 2024-09-21T03:45:01.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:01.210+0000 7f98d43e6640 1 -- 172.21.15.38:0/3015038075 wait complete. 2024-09-21T03:45:01.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:01.211+0000 7f98d43e6640 1 Processor -- start 2024-09-21T03:45:01.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:01.211+0000 7f98d43e6640 1 -- start start 2024-09-21T03:45:01.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:01.213+0000 7f98d43e6640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f98cc0b2e40 0x7f98cc0adb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:01.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:01.213+0000 7f98d43e6640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f98cc162280 0x7f98cc0cee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:01.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:01.213+0000 7f98d43e6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f98cc14d350 0x7f98cc14d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:01.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:01.213+0000 7f98d43e6640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f98cc15e630 con 0x7f98cc14d350 2024-09-21T03:45:01.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:01.213+0000 7f98d43e6640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f98cc15e4b0 con 0x7f98cc162280 2024-09-21T03:45:01.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:01.213+0000 7f98d43e6640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f98cc15e7b0 con 0x7f98cc0b2e40 2024-09-21T03:45:01.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:01.213+0000 7f98d2983640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f98cc14d350 0x7f98cc14d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:01.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:01.213+0000 7f98d2983640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f98cc14d350 0x7f98cc14d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40572/0 (socket says 172.21.15.38:40572) 2024-09-21T03:45:01.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:01.213+0000 7f98d2983640 1 -- 172.21.15.38:0/1769208252 learned_addr learned my addr 172.21.15.38:0/1769208252 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:45:01.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:01.213+0000 7f98d2182640 1 --2- 172.21.15.38:0/1769208252 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f98cc0b2e40 0x7f98cc0adb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:01.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:01.213+0000 7f98d1981640 1 --2- 172.21.15.38:0/1769208252 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f98cc162280 0x7f98cc0cee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:01.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:01.213+0000 7f98d2983640 1 -- 172.21.15.38:0/1769208252 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f98cc0b2e40 msgr2=0x7f98cc0adb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:01.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:01.213+0000 7f98d2983640 1 --2- 172.21.15.38:0/1769208252 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f98cc0b2e40 0x7f98cc0adb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:01.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:01.213+0000 7f98d2983640 1 -- 172.21.15.38:0/1769208252 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f98cc162280 msgr2=0x7f98cc0cee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:01.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:01.213+0000 7f98d2983640 1 --2- 172.21.15.38:0/1769208252 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f98cc162280 0x7f98cc0cee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:01.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:01.213+0000 7f98d2983640 1 -- 172.21.15.38:0/1769208252 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98cc0acad0 con 0x7f98cc14d350 2024-09-21T03:45:01.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:01.213+0000 7f98d2983640 1 --2- 172.21.15.38:0/1769208252 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f98cc14d350 0x7f98cc14d760 secure :-1 s=READY pgs=2362 cs=0 l=1 rev1=1 crypto rx=0x7f98c400c700 tx=0x7f98c400cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:01.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:01.214+0000 7f98c2ffd640 1 -- 172.21.15.38:0/1769208252 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98c4003b70 con 0x7f98cc14d350 2024-09-21T03:45:01.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:01.214+0000 7f98c2ffd640 1 -- 172.21.15.38:0/1769208252 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f98c4003d10 con 0x7f98cc14d350 2024-09-21T03:45:01.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:01.214+0000 7f98d43e6640 1 -- 172.21.15.38:0/1769208252 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98cc0acdb0 con 0x7f98cc14d350 2024-09-21T03:45:01.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:01.214+0000 7f98c2ffd640 1 -- 172.21.15.38:0/1769208252 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98c400aab0 con 0x7f98cc14d350 2024-09-21T03:45:01.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:01.214+0000 7f98d43e6640 1 -- 172.21.15.38:0/1769208252 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98cc0ad370 con 0x7f98cc14d350 2024-09-21T03:45:01.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:01.215+0000 7f98d43e6640 1 -- 172.21.15.38:0/1769208252 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98cc162f00 con 0x7f98cc14d350 2024-09-21T03:45:01.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:01.215+0000 7f98c2ffd640 1 -- 172.21.15.38:0/1769208252 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f98c400ac50 con 0x7f98cc14d350 2024-09-21T03:45:01.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:01.216+0000 7f98c2ffd640 1 --2- 172.21.15.38:0/1769208252 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f98b0078b90 0x7f98b007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:01.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:01.216+0000 7f98c2ffd640 1 -- 172.21.15.38:0/1769208252 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 6101+0+0 (secure 0 0 0) 0x7f98c409ac20 con 0x7f98cc14d350 2024-09-21T03:45:01.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:01.216+0000 7f98d2182640 1 --2- 172.21.15.38:0/1769208252 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f98b0078b90 0x7f98b007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:01.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:01.217+0000 7f98d2182640 1 --2- 172.21.15.38:0/1769208252 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f98b0078b90 0x7f98b007b0a0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f98c8033000 tx=0x7f98c8004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:01.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:01.220+0000 7f98c2ffd640 1 -- 172.21.15.38:0/1769208252 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f98cc162bd0 con 0x7f98cc14d350 2024-09-21T03:45:01.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:01.407+0000 7f98d43e6640 1 -- 172.21.15.38:0/1769208252 --> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f98cc0cf350 con 0x7f98b0078b90 2024-09-21T03:45:01.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:01.412+0000 7f98c2ffd640 1 -- 172.21.15.38:0/1769208252 <== mgr.4117 v2:172.21.15.195:6800/1631901344 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252055 (secure 0 0 0) 0x7f98cc0cf350 con 0x7f98b0078b90 2024-09-21T03:45:01.416 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:45:01.418 INFO:teuthology.orchestra.run.smithi038.stderr:dumped all 2024-09-21T03:45:01.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:01.415+0000 7f98d43e6640 1 -- 172.21.15.38:0/1769208252 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f98b0078b90 msgr2=0x7f98b007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:01.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:01.415+0000 7f98d43e6640 1 --2- 172.21.15.38:0/1769208252 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f98b0078b90 0x7f98b007b0a0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f98c8033000 tx=0x7f98c8004730 comp rx=0 tx=0).stop 2024-09-21T03:45:01.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:01.415+0000 7f98d43e6640 1 -- 172.21.15.38:0/1769208252 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f98cc14d350 msgr2=0x7f98cc14d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:01.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:01.415+0000 7f98d43e6640 1 --2- 172.21.15.38:0/1769208252 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f98cc14d350 0x7f98cc14d760 secure :-1 s=READY pgs=2362 cs=0 l=1 rev1=1 crypto rx=0x7f98c400c700 tx=0x7f98c400cbc0 comp rx=0 tx=0).stop 2024-09-21T03:45:01.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:01.415+0000 7f98d43e6640 1 -- 172.21.15.38:0/1769208252 shutdown_connections 2024-09-21T03:45:01.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:01.415+0000 7f98d43e6640 1 --2- 172.21.15.38:0/1769208252 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f98cc14d350 0x7f98cc14d760 unknown :-1 s=CLOSED pgs=2362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:01.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:01.415+0000 7f98d43e6640 1 --2- 172.21.15.38:0/1769208252 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f98cc162280 0x7f98cc0cee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:01.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:01.415+0000 7f98d43e6640 1 --2- 172.21.15.38:0/1769208252 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f98b0078b90 0x7f98b007b0a0 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:01.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:01.415+0000 7f98d43e6640 1 --2- 172.21.15.38:0/1769208252 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f98cc0b2e40 0x7f98cc0adb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:01.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:01.415+0000 7f98d43e6640 1 -- 172.21.15.38:0/1769208252 >> 172.21.15.38:0/1769208252 conn(0x7f98cc0b6b20 msgr2=0x7f98cc0b3be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:01.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:01.415+0000 7f98d43e6640 1 -- 172.21.15.38:0/1769208252 shutdown_connections 2024-09-21T03:45:01.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:01.415+0000 7f98d43e6640 1 -- 172.21.15.38:0/1769208252 wait complete. 2024-09-21T03:45:01.429 INFO:teuthology.orchestra.run.smithi038.stdout:{"pg_ready":true,"pg_map":{"version":488,"stamp":"2024-09-21T03:45:01.172110+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":141,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":105,"ondisk_log_size":105,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":118,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":306960,"kb_used_data":15848,"kb_used_omap":12,"kb_used_meta":290931,"kb_avail":754667760,"statfs":{"total":773094113280,"available":772779786240,"internally_reserved":0,"allocated":16228352,"data_stored":8782956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12748,"internal_metadata":297913908},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.859588"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":10,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T03:44:57.348031+0000","last_change":"2024-09-21T03:44:57.340688+0000","last_active":"2024-09-21T03:44:57.348031+0000","last_peered":"2024-09-21T03:44:57.348031+0000","last_clean":"2024-09-21T03:44:57.348031+0000","last_became_active":"2024-09-21T03:44:57.340356+0000","last_became_peered":"2024-09-21T03:44:57.340356+0000","last_unstale":"2024-09-21T03:44:57.348031+0000","last_undegraded":"2024-09-21T03:44:57.348031+0000","last_fullsized":"2024-09-21T03:44:57.348031+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:31:55.663249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.915466+0000","last_change":"2024-09-21T03:44:57.354834+0000","last_active":"2024-09-21T03:44:58.915466+0000","last_peered":"2024-09-21T03:44:58.915466+0000","last_clean":"2024-09-21T03:44:58.915466+0000","last_became_active":"2024-09-21T03:44:57.354518+0000","last_became_peered":"2024-09-21T03:44:57.354518+0000","last_unstale":"2024-09-21T03:44:58.915466+0000","last_undegraded":"2024-09-21T03:44:58.915466+0000","last_fullsized":"2024-09-21T03:44:58.915466+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:21:26.621000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":1,"reported_epoch":144,"state":"creating+peering","last_fresh":"2024-09-21T03:44:56.319458+0000","last_change":"2024-09-21T03:44:56.319458+0000","last_active":"2024-09-21T03:44:56.311610+0000","last_peered":"2024-09-21T03:44:56.311610+0000","last_clean":"2024-09-21T03:44:56.311610+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:44:56.319458+0000","last_undegraded":"2024-09-21T03:44:56.319458+0000","last_fullsized":"2024-09-21T03:44:56.319458+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":10,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T03:44:57.359756+0000","last_change":"2024-09-21T03:44:57.356466+0000","last_active":"2024-09-21T03:44:57.359756+0000","last_peered":"2024-09-21T03:44:57.359756+0000","last_clean":"2024-09-21T03:44:57.359756+0000","last_became_active":"2024-09-21T03:44:57.355938+0000","last_became_peered":"2024-09-21T03:44:57.355938+0000","last_unstale":"2024-09-21T03:44:57.359756+0000","last_undegraded":"2024-09-21T03:44:57.359756+0000","last_fullsized":"2024-09-21T03:44:57.359756+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:05:36.290619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":10,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T03:44:57.353943+0000","last_change":"2024-09-21T03:44:57.351972+0000","last_active":"2024-09-21T03:44:57.353943+0000","last_peered":"2024-09-21T03:44:57.353943+0000","last_clean":"2024-09-21T03:44:57.353943+0000","last_became_active":"2024-09-21T03:44:57.351561+0000","last_became_peered":"2024-09-21T03:44:57.351561+0000","last_unstale":"2024-09-21T03:44:57.353943+0000","last_undegraded":"2024-09-21T03:44:57.353943+0000","last_fullsized":"2024-09-21T03:44:57.353943+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:21:51.370818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.353120+0000","last_change":"2024-09-21T03:44:57.349848+0000","last_active":"2024-09-21T03:44:58.353120+0000","last_peered":"2024-09-21T03:44:58.353120+0000","last_clean":"2024-09-21T03:44:58.353120+0000","last_became_active":"2024-09-21T03:44:57.349387+0000","last_became_peered":"2024-09-21T03:44:57.349387+0000","last_unstale":"2024-09-21T03:44:58.353120+0000","last_undegraded":"2024-09-21T03:44:58.353120+0000","last_fullsized":"2024-09-21T03:44:58.353120+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:10:55.772054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.352606+0000","last_change":"2024-09-21T03:44:57.355533+0000","last_active":"2024-09-21T03:44:58.352606+0000","last_peered":"2024-09-21T03:44:58.352606+0000","last_clean":"2024-09-21T03:44:58.352606+0000","last_became_active":"2024-09-21T03:44:57.354892+0000","last_became_peered":"2024-09-21T03:44:57.354892+0000","last_unstale":"2024-09-21T03:44:58.352606+0000","last_undegraded":"2024-09-21T03:44:58.352606+0000","last_fullsized":"2024-09-21T03:44:58.352606+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:05:41.952462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":10,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T03:44:57.356024+0000","last_change":"2024-09-21T03:44:57.350864+0000","last_active":"2024-09-21T03:44:57.356024+0000","last_peered":"2024-09-21T03:44:57.356024+0000","last_clean":"2024-09-21T03:44:57.356024+0000","last_became_active":"2024-09-21T03:44:57.350215+0000","last_became_peered":"2024-09-21T03:44:57.350215+0000","last_unstale":"2024-09-21T03:44:57.356024+0000","last_undegraded":"2024-09-21T03:44:57.356024+0000","last_fullsized":"2024-09-21T03:44:57.356024+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:23:12.061449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.915427+0000","last_change":"2024-09-21T03:44:57.356640+0000","last_active":"2024-09-21T03:44:58.915427+0000","last_peered":"2024-09-21T03:44:58.915427+0000","last_clean":"2024-09-21T03:44:58.915427+0000","last_became_active":"2024-09-21T03:44:57.355966+0000","last_became_peered":"2024-09-21T03:44:57.355966+0000","last_unstale":"2024-09-21T03:44:58.915427+0000","last_undegraded":"2024-09-21T03:44:58.915427+0000","last_fullsized":"2024-09-21T03:44:58.915427+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:40:49.216619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":10,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T03:44:57.359889+0000","last_change":"2024-09-21T03:44:57.359320+0000","last_active":"2024-09-21T03:44:57.359889+0000","last_peered":"2024-09-21T03:44:57.359889+0000","last_clean":"2024-09-21T03:44:57.359889+0000","last_became_active":"2024-09-21T03:44:57.358988+0000","last_became_peered":"2024-09-21T03:44:57.358988+0000","last_unstale":"2024-09-21T03:44:57.359889+0000","last_undegraded":"2024-09-21T03:44:57.359889+0000","last_fullsized":"2024-09-21T03:44:57.359889+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:29:11.643073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.337467+0000","last_change":"2024-09-21T03:44:57.362260+0000","last_active":"2024-09-21T03:44:58.337467+0000","last_peered":"2024-09-21T03:44:58.337467+0000","last_clean":"2024-09-21T03:44:58.337467+0000","last_became_active":"2024-09-21T03:44:57.361624+0000","last_became_peered":"2024-09-21T03:44:57.361624+0000","last_unstale":"2024-09-21T03:44:58.337467+0000","last_undegraded":"2024-09-21T03:44:58.337467+0000","last_fullsized":"2024-09-21T03:44:58.337467+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:00:23.014227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.352884+0000","last_change":"2024-09-21T03:44:57.353828+0000","last_active":"2024-09-21T03:44:58.352884+0000","last_peered":"2024-09-21T03:44:58.352884+0000","last_clean":"2024-09-21T03:44:58.352884+0000","last_became_active":"2024-09-21T03:44:57.353516+0000","last_became_peered":"2024-09-21T03:44:57.353516+0000","last_unstale":"2024-09-21T03:44:58.352884+0000","last_undegraded":"2024-09-21T03:44:58.352884+0000","last_fullsized":"2024-09-21T03:44:58.352884+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:49:05.536169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":10,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T03:44:57.366752+0000","last_change":"2024-09-21T03:44:57.363696+0000","last_active":"2024-09-21T03:44:57.366752+0000","last_peered":"2024-09-21T03:44:57.366752+0000","last_clean":"2024-09-21T03:44:57.366752+0000","last_became_active":"2024-09-21T03:44:57.362737+0000","last_became_peered":"2024-09-21T03:44:57.362737+0000","last_unstale":"2024-09-21T03:44:57.366752+0000","last_undegraded":"2024-09-21T03:44:57.366752+0000","last_fullsized":"2024-09-21T03:44:57.366752+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:58:38.157904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.353349+0000","last_change":"2024-09-21T03:44:57.352690+0000","last_active":"2024-09-21T03:44:58.353349+0000","last_peered":"2024-09-21T03:44:58.353349+0000","last_clean":"2024-09-21T03:44:58.353349+0000","last_became_active":"2024-09-21T03:44:57.352358+0000","last_became_peered":"2024-09-21T03:44:57.352358+0000","last_unstale":"2024-09-21T03:44:58.353349+0000","last_undegraded":"2024-09-21T03:44:58.353349+0000","last_fullsized":"2024-09-21T03:44:58.353349+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:42:27.812988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.915095+0000","last_change":"2024-09-21T03:44:57.369797+0000","last_active":"2024-09-21T03:44:58.915095+0000","last_peered":"2024-09-21T03:44:58.915095+0000","last_clean":"2024-09-21T03:44:58.915095+0000","last_became_active":"2024-09-21T03:44:57.369463+0000","last_became_peered":"2024-09-21T03:44:57.369463+0000","last_unstale":"2024-09-21T03:44:58.915095+0000","last_undegraded":"2024-09-21T03:44:58.915095+0000","last_fullsized":"2024-09-21T03:44:58.915095+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:59:36.159782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.338697+0000","last_change":"2024-09-21T03:44:57.341545+0000","last_active":"2024-09-21T03:44:58.338697+0000","last_peered":"2024-09-21T03:44:58.338697+0000","last_clean":"2024-09-21T03:44:58.338697+0000","last_became_active":"2024-09-21T03:44:57.341220+0000","last_became_peered":"2024-09-21T03:44:57.341220+0000","last_unstale":"2024-09-21T03:44:58.338697+0000","last_undegraded":"2024-09-21T03:44:58.338697+0000","last_fullsized":"2024-09-21T03:44:58.338697+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:40:08.834183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":1,"reported_epoch":144,"state":"creating+peering","last_fresh":"2024-09-21T03:44:56.321953+0000","last_change":"2024-09-21T03:44:56.321953+0000","last_active":"2024-09-21T03:44:56.311610+0000","last_peered":"2024-09-21T03:44:56.311610+0000","last_clean":"2024-09-21T03:44:56.311610+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:44:56.321953+0000","last_undegraded":"2024-09-21T03:44:56.321953+0000","last_fullsized":"2024-09-21T03:44:56.321953+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.915438+0000","last_change":"2024-09-21T03:44:57.357272+0000","last_active":"2024-09-21T03:44:58.915438+0000","last_peered":"2024-09-21T03:44:58.915438+0000","last_clean":"2024-09-21T03:44:58.915438+0000","last_became_active":"2024-09-21T03:44:57.356952+0000","last_became_peered":"2024-09-21T03:44:57.356952+0000","last_unstale":"2024-09-21T03:44:58.915438+0000","last_undegraded":"2024-09-21T03:44:58.915438+0000","last_fullsized":"2024-09-21T03:44:58.915438+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:35:23.462964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":10,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T03:44:57.354219+0000","last_change":"2024-09-21T03:44:57.351765+0000","last_active":"2024-09-21T03:44:57.354219+0000","last_peered":"2024-09-21T03:44:57.354219+0000","last_clean":"2024-09-21T03:44:57.354219+0000","last_became_active":"2024-09-21T03:44:57.351478+0000","last_became_peered":"2024-09-21T03:44:57.351478+0000","last_unstale":"2024-09-21T03:44:57.354219+0000","last_undegraded":"2024-09-21T03:44:57.354219+0000","last_fullsized":"2024-09-21T03:44:57.354219+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:00:35.318345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.339047+0000","last_change":"2024-09-21T03:44:57.358796+0000","last_active":"2024-09-21T03:44:58.339047+0000","last_peered":"2024-09-21T03:44:58.339047+0000","last_clean":"2024-09-21T03:44:58.339047+0000","last_became_active":"2024-09-21T03:44:57.358439+0000","last_became_peered":"2024-09-21T03:44:57.358439+0000","last_unstale":"2024-09-21T03:44:58.339047+0000","last_undegraded":"2024-09-21T03:44:58.339047+0000","last_fullsized":"2024-09-21T03:44:58.339047+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:05:51.280587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":1,"reported_epoch":144,"state":"creating+peering","last_fresh":"2024-09-21T03:44:56.322208+0000","last_change":"2024-09-21T03:44:56.322208+0000","last_active":"2024-09-21T03:44:56.311610+0000","last_peered":"2024-09-21T03:44:56.311610+0000","last_clean":"2024-09-21T03:44:56.311610+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:44:56.322208+0000","last_undegraded":"2024-09-21T03:44:56.322208+0000","last_fullsized":"2024-09-21T03:44:56.322208+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.456326+0000","last_change":"2024-09-21T03:44:57.357854+0000","last_active":"2024-09-21T03:44:58.456326+0000","last_peered":"2024-09-21T03:44:58.456326+0000","last_clean":"2024-09-21T03:44:58.456326+0000","last_became_active":"2024-09-21T03:44:57.357541+0000","last_became_peered":"2024-09-21T03:44:57.357541+0000","last_unstale":"2024-09-21T03:44:58.456326+0000","last_undegraded":"2024-09-21T03:44:58.456326+0000","last_fullsized":"2024-09-21T03:44:58.456326+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:04:52.143962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":10,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T03:44:57.355917+0000","last_change":"2024-09-21T03:44:57.353700+0000","last_active":"2024-09-21T03:44:57.355917+0000","last_peered":"2024-09-21T03:44:57.355917+0000","last_clean":"2024-09-21T03:44:57.355917+0000","last_became_active":"2024-09-21T03:44:57.351501+0000","last_became_peered":"2024-09-21T03:44:57.351501+0000","last_unstale":"2024-09-21T03:44:57.355917+0000","last_undegraded":"2024-09-21T03:44:57.355917+0000","last_fullsized":"2024-09-21T03:44:57.355917+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:00:30.971449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":10,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T03:44:57.356048+0000","last_change":"2024-09-21T03:44:57.351638+0000","last_active":"2024-09-21T03:44:57.356048+0000","last_peered":"2024-09-21T03:44:57.356048+0000","last_clean":"2024-09-21T03:44:57.356048+0000","last_became_active":"2024-09-21T03:44:57.351325+0000","last_became_peered":"2024-09-21T03:44:57.351325+0000","last_unstale":"2024-09-21T03:44:57.356048+0000","last_undegraded":"2024-09-21T03:44:57.356048+0000","last_fullsized":"2024-09-21T03:44:57.356048+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:05:57.831029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.339252+0000","last_change":"2024-09-21T03:44:57.359418+0000","last_active":"2024-09-21T03:44:58.339252+0000","last_peered":"2024-09-21T03:44:58.339252+0000","last_clean":"2024-09-21T03:44:58.339252+0000","last_became_active":"2024-09-21T03:44:57.358564+0000","last_became_peered":"2024-09-21T03:44:57.358564+0000","last_unstale":"2024-09-21T03:44:58.339252+0000","last_undegraded":"2024-09-21T03:44:58.339252+0000","last_fullsized":"2024-09-21T03:44:58.339252+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:21:26.978675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.339318+0000","last_change":"2024-09-21T03:44:57.360155+0000","last_active":"2024-09-21T03:44:58.339318+0000","last_peered":"2024-09-21T03:44:58.339318+0000","last_clean":"2024-09-21T03:44:58.339318+0000","last_became_active":"2024-09-21T03:44:57.359645+0000","last_became_peered":"2024-09-21T03:44:57.359645+0000","last_unstale":"2024-09-21T03:44:58.339318+0000","last_undegraded":"2024-09-21T03:44:58.339318+0000","last_fullsized":"2024-09-21T03:44:58.339318+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:14:53.602909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.339127+0000","last_change":"2024-09-21T03:44:57.357568+0000","last_active":"2024-09-21T03:44:58.339127+0000","last_peered":"2024-09-21T03:44:58.339127+0000","last_clean":"2024-09-21T03:44:58.339127+0000","last_became_active":"2024-09-21T03:44:57.357185+0000","last_became_peered":"2024-09-21T03:44:57.357185+0000","last_unstale":"2024-09-21T03:44:58.339127+0000","last_undegraded":"2024-09-21T03:44:58.339127+0000","last_fullsized":"2024-09-21T03:44:58.339127+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:57:24.936155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.353110+0000","last_change":"2024-09-21T03:44:57.352231+0000","last_active":"2024-09-21T03:44:58.353110+0000","last_peered":"2024-09-21T03:44:58.353110+0000","last_clean":"2024-09-21T03:44:58.353110+0000","last_became_active":"2024-09-21T03:44:57.351887+0000","last_became_peered":"2024-09-21T03:44:57.351887+0000","last_unstale":"2024-09-21T03:44:58.353110+0000","last_undegraded":"2024-09-21T03:44:58.353110+0000","last_fullsized":"2024-09-21T03:44:58.353110+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:46:07.069681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":10,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T03:44:57.358271+0000","last_change":"2024-09-21T03:44:57.357470+0000","last_active":"2024-09-21T03:44:57.358271+0000","last_peered":"2024-09-21T03:44:57.358271+0000","last_clean":"2024-09-21T03:44:57.358271+0000","last_became_active":"2024-09-21T03:44:57.357091+0000","last_became_peered":"2024-09-21T03:44:57.357091+0000","last_unstale":"2024-09-21T03:44:57.358271+0000","last_undegraded":"2024-09-21T03:44:57.358271+0000","last_fullsized":"2024-09-21T03:44:57.358271+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:13:25.917453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.337478+0000","last_change":"2024-09-21T03:44:57.361022+0000","last_active":"2024-09-21T03:44:58.337478+0000","last_peered":"2024-09-21T03:44:58.337478+0000","last_clean":"2024-09-21T03:44:58.337478+0000","last_became_active":"2024-09-21T03:44:57.360697+0000","last_became_peered":"2024-09-21T03:44:57.360697+0000","last_unstale":"2024-09-21T03:44:58.337478+0000","last_undegraded":"2024-09-21T03:44:58.337478+0000","last_fullsized":"2024-09-21T03:44:58.337478+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:17:17.662702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.352611+0000","last_change":"2024-09-21T03:44:57.357841+0000","last_active":"2024-09-21T03:44:58.352611+0000","last_peered":"2024-09-21T03:44:58.352611+0000","last_clean":"2024-09-21T03:44:58.352611+0000","last_became_active":"2024-09-21T03:44:57.357299+0000","last_became_peered":"2024-09-21T03:44:57.357299+0000","last_unstale":"2024-09-21T03:44:58.352611+0000","last_undegraded":"2024-09-21T03:44:58.352611+0000","last_fullsized":"2024-09-21T03:44:58.352611+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:36:45.265239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.456162+0000","last_change":"2024-09-21T03:44:57.366423+0000","last_active":"2024-09-21T03:44:58.456162+0000","last_peered":"2024-09-21T03:44:58.456162+0000","last_clean":"2024-09-21T03:44:58.456162+0000","last_became_active":"2024-09-21T03:44:57.366083+0000","last_became_peered":"2024-09-21T03:44:57.366083+0000","last_unstale":"2024-09-21T03:44:58.456162+0000","last_undegraded":"2024-09-21T03:44:58.456162+0000","last_fullsized":"2024-09-21T03:44:58.456162+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:14:54.627243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":1,"reported_epoch":144,"state":"creating+peering","last_fresh":"2024-09-21T03:44:56.321750+0000","last_change":"2024-09-21T03:44:56.321750+0000","last_active":"2024-09-21T03:44:56.311610+0000","last_peered":"2024-09-21T03:44:56.311610+0000","last_clean":"2024-09-21T03:44:56.311610+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:44:56.321750+0000","last_undegraded":"2024-09-21T03:44:56.321750+0000","last_fullsized":"2024-09-21T03:44:56.321750+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1f","version":"145'4","reported_seq":19,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.456694+0000","last_change":"2024-09-21T03:44:56.332485+0000","last_active":"2024-09-21T03:44:58.456694+0000","last_peered":"2024-09-21T03:44:58.456694+0000","last_clean":"2024-09-21T03:44:58.456694+0000","last_became_active":"2024-09-21T03:44:56.332204+0000","last_became_peered":"2024-09-21T03:44:56.332204+0000","last_unstale":"2024-09-21T03:44:58.456694+0000","last_undegraded":"2024-09-21T03:44:58.456694+0000","last_fullsized":"2024-09-21T03:44:58.456694+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_clean_scrub_stamp":"2024-09-21T03:44:55.304777+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:15:02.688269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":10,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T03:44:57.356612+0000","last_change":"2024-09-21T03:44:57.352443+0000","last_active":"2024-09-21T03:44:57.356612+0000","last_peered":"2024-09-21T03:44:57.356612+0000","last_clean":"2024-09-21T03:44:57.356612+0000","last_became_active":"2024-09-21T03:44:57.351086+0000","last_became_peered":"2024-09-21T03:44:57.351086+0000","last_unstale":"2024-09-21T03:44:57.356612+0000","last_undegraded":"2024-09-21T03:44:57.356612+0000","last_fullsized":"2024-09-21T03:44:57.356612+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:27:41.239844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T03:44:56.342206+0000","last_change":"2024-09-21T03:44:56.341615+0000","last_active":"2024-09-21T03:44:56.342206+0000","last_peered":"2024-09-21T03:44:56.342206+0000","last_clean":"2024-09-21T03:44:56.342206+0000","last_became_active":"2024-09-21T03:44:56.341306+0000","last_became_peered":"2024-09-21T03:44:56.341306+0000","last_unstale":"2024-09-21T03:44:56.342206+0000","last_undegraded":"2024-09-21T03:44:56.342206+0000","last_fullsized":"2024-09-21T03:44:56.342206+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_clean_scrub_stamp":"2024-09-21T03:44:55.304777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:36:07.439425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.915135+0000","last_change":"2024-09-21T03:44:57.349398+0000","last_active":"2024-09-21T03:44:58.915135+0000","last_peered":"2024-09-21T03:44:58.915135+0000","last_clean":"2024-09-21T03:44:58.915135+0000","last_became_active":"2024-09-21T03:44:57.349067+0000","last_became_peered":"2024-09-21T03:44:57.349067+0000","last_unstale":"2024-09-21T03:44:58.915135+0000","last_undegraded":"2024-09-21T03:44:58.915135+0000","last_fullsized":"2024-09-21T03:44:58.915135+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:19:45.952024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"145'2","reported_seq":17,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.915396+0000","last_change":"2024-09-21T03:44:56.343325+0000","last_active":"2024-09-21T03:44:58.915396+0000","last_peered":"2024-09-21T03:44:58.915396+0000","last_clean":"2024-09-21T03:44:58.915396+0000","last_became_active":"2024-09-21T03:44:56.342964+0000","last_became_peered":"2024-09-21T03:44:56.342964+0000","last_unstale":"2024-09-21T03:44:58.915396+0000","last_undegraded":"2024-09-21T03:44:58.915396+0000","last_fullsized":"2024-09-21T03:44:58.915396+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_clean_scrub_stamp":"2024-09-21T03:44:55.304777+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:19:36.404105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.337488+0000","last_change":"2024-09-21T03:44:57.360138+0000","last_active":"2024-09-21T03:44:58.337488+0000","last_peered":"2024-09-21T03:44:58.337488+0000","last_clean":"2024-09-21T03:44:58.337488+0000","last_became_active":"2024-09-21T03:44:57.359800+0000","last_became_peered":"2024-09-21T03:44:57.359800+0000","last_unstale":"2024-09-21T03:44:58.337488+0000","last_undegraded":"2024-09-21T03:44:58.337488+0000","last_fullsized":"2024-09-21T03:44:58.337488+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:33:24.015377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1c","version":"145'2","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T03:44:57.373298+0000","last_change":"2024-09-21T03:44:56.334534+0000","last_active":"2024-09-21T03:44:57.373298+0000","last_peered":"2024-09-21T03:44:57.373298+0000","last_clean":"2024-09-21T03:44:57.373298+0000","last_became_active":"2024-09-21T03:44:56.333800+0000","last_became_peered":"2024-09-21T03:44:56.333800+0000","last_unstale":"2024-09-21T03:44:57.373298+0000","last_undegraded":"2024-09-21T03:44:57.373298+0000","last_fullsized":"2024-09-21T03:44:57.373298+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_clean_scrub_stamp":"2024-09-21T03:44:55.304777+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:44:41.072083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":10,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T03:44:57.350445+0000","last_change":"2024-09-21T03:44:57.342301+0000","last_active":"2024-09-21T03:44:57.350445+0000","last_peered":"2024-09-21T03:44:57.350445+0000","last_clean":"2024-09-21T03:44:57.350445+0000","last_became_active":"2024-09-21T03:44:57.341880+0000","last_became_peered":"2024-09-21T03:44:57.341880+0000","last_unstale":"2024-09-21T03:44:57.350445+0000","last_undegraded":"2024-09-21T03:44:57.350445+0000","last_fullsized":"2024-09-21T03:44:57.350445+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:10:55.772054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1b","version":"145'1","reported_seq":16,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.337514+0000","last_change":"2024-09-21T03:44:56.342732+0000","last_active":"2024-09-21T03:44:58.337514+0000","last_peered":"2024-09-21T03:44:58.337514+0000","last_clean":"2024-09-21T03:44:58.337514+0000","last_became_active":"2024-09-21T03:44:56.342319+0000","last_became_peered":"2024-09-21T03:44:56.342319+0000","last_unstale":"2024-09-21T03:44:58.337514+0000","last_undegraded":"2024-09-21T03:44:58.337514+0000","last_fullsized":"2024-09-21T03:44:58.337514+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_clean_scrub_stamp":"2024-09-21T03:44:55.304777+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:31:32.978494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":10,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T03:44:57.350445+0000","last_change":"2024-09-21T03:44:57.346978+0000","last_active":"2024-09-21T03:44:57.350445+0000","last_peered":"2024-09-21T03:44:57.350445+0000","last_clean":"2024-09-21T03:44:57.350445+0000","last_became_active":"2024-09-21T03:44:57.346633+0000","last_became_peered":"2024-09-21T03:44:57.346633+0000","last_unstale":"2024-09-21T03:44:57.350445+0000","last_undegraded":"2024-09-21T03:44:57.350445+0000","last_fullsized":"2024-09-21T03:44:57.350445+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:17:57.312760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1a","version":"145'1","reported_seq":16,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.338782+0000","last_change":"2024-09-21T03:44:56.342500+0000","last_active":"2024-09-21T03:44:58.338782+0000","last_peered":"2024-09-21T03:44:58.338782+0000","last_clean":"2024-09-21T03:44:58.338782+0000","last_became_active":"2024-09-21T03:44:56.342198+0000","last_became_peered":"2024-09-21T03:44:56.342198+0000","last_unstale":"2024-09-21T03:44:58.338782+0000","last_undegraded":"2024-09-21T03:44:58.338782+0000","last_fullsized":"2024-09-21T03:44:58.338782+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_clean_scrub_stamp":"2024-09-21T03:44:55.304777+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:24:33.377335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.337947+0000","last_change":"2024-09-21T03:44:57.360511+0000","last_active":"2024-09-21T03:44:58.337947+0000","last_peered":"2024-09-21T03:44:58.337947+0000","last_clean":"2024-09-21T03:44:58.337947+0000","last_became_active":"2024-09-21T03:44:57.360163+0000","last_became_peered":"2024-09-21T03:44:57.360163+0000","last_unstale":"2024-09-21T03:44:58.337947+0000","last_undegraded":"2024-09-21T03:44:58.337947+0000","last_fullsized":"2024-09-21T03:44:58.337947+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:05:31.071185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":15,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.338632+0000","last_change":"2024-09-21T03:44:56.343444+0000","last_active":"2024-09-21T03:44:58.338632+0000","last_peered":"2024-09-21T03:44:58.338632+0000","last_clean":"2024-09-21T03:44:58.338632+0000","last_became_active":"2024-09-21T03:44:56.343128+0000","last_became_peered":"2024-09-21T03:44:56.343128+0000","last_unstale":"2024-09-21T03:44:58.338632+0000","last_undegraded":"2024-09-21T03:44:58.338632+0000","last_fullsized":"2024-09-21T03:44:58.338632+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_clean_scrub_stamp":"2024-09-21T03:44:55.304777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:49:27.666317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":10,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T03:44:57.351262+0000","last_change":"2024-09-21T03:44:57.348168+0000","last_active":"2024-09-21T03:44:57.351262+0000","last_peered":"2024-09-21T03:44:57.351262+0000","last_clean":"2024-09-21T03:44:57.351262+0000","last_became_active":"2024-09-21T03:44:57.347823+0000","last_became_peered":"2024-09-21T03:44:57.347823+0000","last_unstale":"2024-09-21T03:44:57.351262+0000","last_undegraded":"2024-09-21T03:44:57.351262+0000","last_fullsized":"2024-09-21T03:44:57.351262+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:43:22.624675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":15,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.456487+0000","last_change":"2024-09-21T03:44:56.337896+0000","last_active":"2024-09-21T03:44:58.456487+0000","last_peered":"2024-09-21T03:44:58.456487+0000","last_clean":"2024-09-21T03:44:58.456487+0000","last_became_active":"2024-09-21T03:44:56.337298+0000","last_became_peered":"2024-09-21T03:44:56.337298+0000","last_unstale":"2024-09-21T03:44:58.456487+0000","last_undegraded":"2024-09-21T03:44:58.456487+0000","last_fullsized":"2024-09-21T03:44:58.456487+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_clean_scrub_stamp":"2024-09-21T03:44:55.304777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:58:50.496950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":10,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T03:44:57.351652+0000","last_change":"2024-09-21T03:44:57.350094+0000","last_active":"2024-09-21T03:44:57.351652+0000","last_peered":"2024-09-21T03:44:57.351652+0000","last_clean":"2024-09-21T03:44:57.351652+0000","last_became_active":"2024-09-21T03:44:57.349777+0000","last_became_peered":"2024-09-21T03:44:57.349777+0000","last_unstale":"2024-09-21T03:44:57.351652+0000","last_undegraded":"2024-09-21T03:44:57.351652+0000","last_fullsized":"2024-09-21T03:44:57.351652+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:36:06.178611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":15,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.353239+0000","last_change":"2024-09-21T03:44:56.329230+0000","last_active":"2024-09-21T03:44:58.353239+0000","last_peered":"2024-09-21T03:44:58.353239+0000","last_clean":"2024-09-21T03:44:58.353239+0000","last_became_active":"2024-09-21T03:44:56.328911+0000","last_became_peered":"2024-09-21T03:44:56.328911+0000","last_unstale":"2024-09-21T03:44:58.353239+0000","last_undegraded":"2024-09-21T03:44:58.353239+0000","last_fullsized":"2024-09-21T03:44:58.353239+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_clean_scrub_stamp":"2024-09-21T03:44:55.304777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:16:45.547857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":10,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T03:44:57.351808+0000","last_change":"2024-09-21T03:44:57.347015+0000","last_active":"2024-09-21T03:44:57.351808+0000","last_peered":"2024-09-21T03:44:57.351808+0000","last_clean":"2024-09-21T03:44:57.351808+0000","last_became_active":"2024-09-21T03:44:57.346668+0000","last_became_peered":"2024-09-21T03:44:57.346668+0000","last_unstale":"2024-09-21T03:44:57.351808+0000","last_undegraded":"2024-09-21T03:44:57.351808+0000","last_fullsized":"2024-09-21T03:44:57.351808+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:49:55.140933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T03:44:57.341754+0000","last_change":"2024-09-21T03:44:56.333386+0000","last_active":"2024-09-21T03:44:57.341754+0000","last_peered":"2024-09-21T03:44:57.341754+0000","last_clean":"2024-09-21T03:44:57.341754+0000","last_became_active":"2024-09-21T03:44:56.333082+0000","last_became_peered":"2024-09-21T03:44:56.333082+0000","last_unstale":"2024-09-21T03:44:57.341754+0000","last_undegraded":"2024-09-21T03:44:57.341754+0000","last_fullsized":"2024-09-21T03:44:57.341754+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_clean_scrub_stamp":"2024-09-21T03:44:55.304777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:10:24.715196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":10,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T03:44:57.355956+0000","last_change":"2024-09-21T03:44:57.350041+0000","last_active":"2024-09-21T03:44:57.355956+0000","last_peered":"2024-09-21T03:44:57.355956+0000","last_clean":"2024-09-21T03:44:57.355956+0000","last_became_active":"2024-09-21T03:44:57.349706+0000","last_became_peered":"2024-09-21T03:44:57.349706+0000","last_unstale":"2024-09-21T03:44:57.355956+0000","last_undegraded":"2024-09-21T03:44:57.355956+0000","last_fullsized":"2024-09-21T03:44:57.355956+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:35:23.899135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.15","version":"145'3","reported_seq":18,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.352972+0000","last_change":"2024-09-21T03:44:56.324998+0000","last_active":"2024-09-21T03:44:58.352972+0000","last_peered":"2024-09-21T03:44:58.352972+0000","last_clean":"2024-09-21T03:44:58.352972+0000","last_became_active":"2024-09-21T03:44:56.324464+0000","last_became_peered":"2024-09-21T03:44:56.324464+0000","last_unstale":"2024-09-21T03:44:58.352972+0000","last_undegraded":"2024-09-21T03:44:58.352972+0000","last_fullsized":"2024-09-21T03:44:58.352972+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_clean_scrub_stamp":"2024-09-21T03:44:55.304777+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:27:45.431858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.456464+0000","last_change":"2024-09-21T03:44:57.355113+0000","last_active":"2024-09-21T03:44:58.456464+0000","last_peered":"2024-09-21T03:44:58.456464+0000","last_clean":"2024-09-21T03:44:58.456464+0000","last_became_active":"2024-09-21T03:44:57.354744+0000","last_became_peered":"2024-09-21T03:44:57.354744+0000","last_unstale":"2024-09-21T03:44:58.456464+0000","last_undegraded":"2024-09-21T03:44:58.456464+0000","last_fullsized":"2024-09-21T03:44:58.456464+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:54:24.418739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.14","version":"145'6","reported_seq":21,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.337583+0000","last_change":"2024-09-21T03:44:56.341013+0000","last_active":"2024-09-21T03:44:58.337583+0000","last_peered":"2024-09-21T03:44:58.337583+0000","last_clean":"2024-09-21T03:44:58.337583+0000","last_became_active":"2024-09-21T03:44:56.337878+0000","last_became_peered":"2024-09-21T03:44:56.337878+0000","last_unstale":"2024-09-21T03:44:58.337583+0000","last_undegraded":"2024-09-21T03:44:58.337583+0000","last_fullsized":"2024-09-21T03:44:58.337583+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_clean_scrub_stamp":"2024-09-21T03:44:55.304777+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:00:34.311511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.915365+0000","last_change":"2024-09-21T03:44:57.356176+0000","last_active":"2024-09-21T03:44:58.915365+0000","last_peered":"2024-09-21T03:44:58.915365+0000","last_clean":"2024-09-21T03:44:58.915365+0000","last_became_active":"2024-09-21T03:44:57.355832+0000","last_became_peered":"2024-09-21T03:44:57.355832+0000","last_unstale":"2024-09-21T03:44:58.915365+0000","last_undegraded":"2024-09-21T03:44:58.915365+0000","last_fullsized":"2024-09-21T03:44:58.915365+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:47:30.592382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.13","version":"145'2","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T03:44:57.367038+0000","last_change":"2024-09-21T03:44:56.327633+0000","last_active":"2024-09-21T03:44:57.367038+0000","last_peered":"2024-09-21T03:44:57.367038+0000","last_clean":"2024-09-21T03:44:57.367038+0000","last_became_active":"2024-09-21T03:44:56.327313+0000","last_became_peered":"2024-09-21T03:44:56.327313+0000","last_unstale":"2024-09-21T03:44:57.367038+0000","last_undegraded":"2024-09-21T03:44:57.367038+0000","last_fullsized":"2024-09-21T03:44:57.367038+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_clean_scrub_stamp":"2024-09-21T03:44:55.304777+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:36:44.258406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.915449+0000","last_change":"2024-09-21T03:44:57.352611+0000","last_active":"2024-09-21T03:44:58.915449+0000","last_peered":"2024-09-21T03:44:58.915449+0000","last_clean":"2024-09-21T03:44:58.915449+0000","last_became_active":"2024-09-21T03:44:57.352136+0000","last_became_peered":"2024-09-21T03:44:57.352136+0000","last_unstale":"2024-09-21T03:44:58.915449+0000","last_undegraded":"2024-09-21T03:44:58.915449+0000","last_fullsized":"2024-09-21T03:44:58.915449+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:26:06.882445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T03:44:57.341635+0000","last_change":"2024-09-21T03:44:56.337958+0000","last_active":"2024-09-21T03:44:57.341635+0000","last_peered":"2024-09-21T03:44:57.341635+0000","last_clean":"2024-09-21T03:44:57.341635+0000","last_became_active":"2024-09-21T03:44:56.337623+0000","last_became_peered":"2024-09-21T03:44:56.337623+0000","last_unstale":"2024-09-21T03:44:57.341635+0000","last_undegraded":"2024-09-21T03:44:57.341635+0000","last_fullsized":"2024-09-21T03:44:57.341635+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_clean_scrub_stamp":"2024-09-21T03:44:55.304777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:45:31.964595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.456704+0000","last_change":"2024-09-21T03:44:57.358895+0000","last_active":"2024-09-21T03:44:58.456704+0000","last_peered":"2024-09-21T03:44:58.456704+0000","last_clean":"2024-09-21T03:44:58.456704+0000","last_became_active":"2024-09-21T03:44:57.358591+0000","last_became_peered":"2024-09-21T03:44:57.358591+0000","last_unstale":"2024-09-21T03:44:58.456704+0000","last_undegraded":"2024-09-21T03:44:58.456704+0000","last_fullsized":"2024-09-21T03:44:58.456704+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:38:18.158452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T03:44:57.337893+0000","last_change":"2024-09-21T03:44:56.327693+0000","last_active":"2024-09-21T03:44:57.337893+0000","last_peered":"2024-09-21T03:44:57.337893+0000","last_clean":"2024-09-21T03:44:57.337893+0000","last_became_active":"2024-09-21T03:44:56.327364+0000","last_became_peered":"2024-09-21T03:44:56.327364+0000","last_unstale":"2024-09-21T03:44:57.337893+0000","last_undegraded":"2024-09-21T03:44:57.337893+0000","last_fullsized":"2024-09-21T03:44:57.337893+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_clean_scrub_stamp":"2024-09-21T03:44:55.304777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:16:04.551826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":1,"reported_epoch":144,"state":"creating+peering","last_fresh":"2024-09-21T03:44:56.321369+0000","last_change":"2024-09-21T03:44:56.321369+0000","last_active":"2024-09-21T03:44:56.311610+0000","last_peered":"2024-09-21T03:44:56.311610+0000","last_clean":"2024-09-21T03:44:56.311610+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:44:56.321369+0000","last_undegraded":"2024-09-21T03:44:56.321369+0000","last_fullsized":"2024-09-21T03:44:56.321369+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.10","version":"145'8","reported_seq":23,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.337672+0000","last_change":"2024-09-21T03:44:56.342213+0000","last_active":"2024-09-21T03:44:58.337672+0000","last_peered":"2024-09-21T03:44:58.337672+0000","last_clean":"2024-09-21T03:44:58.337672+0000","last_became_active":"2024-09-21T03:44:56.341888+0000","last_became_peered":"2024-09-21T03:44:56.341888+0000","last_unstale":"2024-09-21T03:44:58.337672+0000","last_undegraded":"2024-09-21T03:44:58.337672+0000","last_fullsized":"2024-09-21T03:44:58.337672+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_clean_scrub_stamp":"2024-09-21T03:44:55.304777+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:36:05.171778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":10,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T03:44:57.355999+0000","last_change":"2024-09-21T03:44:57.350536+0000","last_active":"2024-09-21T03:44:57.355999+0000","last_peered":"2024-09-21T03:44:57.355999+0000","last_clean":"2024-09-21T03:44:57.355999+0000","last_became_active":"2024-09-21T03:44:57.349795+0000","last_became_peered":"2024-09-21T03:44:57.349795+0000","last_unstale":"2024-09-21T03:44:57.355999+0000","last_undegraded":"2024-09-21T03:44:57.355999+0000","last_fullsized":"2024-09-21T03:44:57.355999+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:09:26.811858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.f","version":"145'1","reported_seq":16,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.353274+0000","last_change":"2024-09-21T03:44:56.339856+0000","last_active":"2024-09-21T03:44:58.353274+0000","last_peered":"2024-09-21T03:44:58.353274+0000","last_clean":"2024-09-21T03:44:58.353274+0000","last_became_active":"2024-09-21T03:44:56.339305+0000","last_became_peered":"2024-09-21T03:44:56.339305+0000","last_unstale":"2024-09-21T03:44:58.353274+0000","last_undegraded":"2024-09-21T03:44:58.353274+0000","last_fullsized":"2024-09-21T03:44:58.353274+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_clean_scrub_stamp":"2024-09-21T03:44:55.304777+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:37:25.372042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.338656+0000","last_change":"2024-09-21T03:44:57.341004+0000","last_active":"2024-09-21T03:44:58.338656+0000","last_peered":"2024-09-21T03:44:58.338656+0000","last_clean":"2024-09-21T03:44:58.338656+0000","last_became_active":"2024-09-21T03:44:57.340620+0000","last_became_peered":"2024-09-21T03:44:57.340620+0000","last_unstale":"2024-09-21T03:44:58.338656+0000","last_undegraded":"2024-09-21T03:44:58.338656+0000","last_fullsized":"2024-09-21T03:44:58.338656+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:09:01.018690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.d","version":"145'1","reported_seq":16,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.339122+0000","last_change":"2024-09-21T03:44:56.340547+0000","last_active":"2024-09-21T03:44:58.339122+0000","last_peered":"2024-09-21T03:44:58.339122+0000","last_clean":"2024-09-21T03:44:58.339122+0000","last_became_active":"2024-09-21T03:44:56.340237+0000","last_became_peered":"2024-09-21T03:44:56.340237+0000","last_unstale":"2024-09-21T03:44:58.339122+0000","last_undegraded":"2024-09-21T03:44:58.339122+0000","last_fullsized":"2024-09-21T03:44:58.339122+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_clean_scrub_stamp":"2024-09-21T03:44:55.304777+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:19:41.082645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.456321+0000","last_change":"2024-09-21T03:44:57.358052+0000","last_active":"2024-09-21T03:44:58.456321+0000","last_peered":"2024-09-21T03:44:58.456321+0000","last_clean":"2024-09-21T03:44:58.456321+0000","last_became_active":"2024-09-21T03:44:57.357731+0000","last_became_peered":"2024-09-21T03:44:57.357731+0000","last_unstale":"2024-09-21T03:44:58.456321+0000","last_undegraded":"2024-09-21T03:44:58.456321+0000","last_fullsized":"2024-09-21T03:44:58.456321+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:02:38.958342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":15,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.353016+0000","last_change":"2024-09-21T03:44:56.339649+0000","last_active":"2024-09-21T03:44:58.353016+0000","last_peered":"2024-09-21T03:44:58.353016+0000","last_clean":"2024-09-21T03:44:58.353016+0000","last_became_active":"2024-09-21T03:44:56.339317+0000","last_became_peered":"2024-09-21T03:44:56.339317+0000","last_unstale":"2024-09-21T03:44:58.353016+0000","last_undegraded":"2024-09-21T03:44:58.353016+0000","last_fullsized":"2024-09-21T03:44:58.353016+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_clean_scrub_stamp":"2024-09-21T03:44:55.304777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:58:28.485819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":10,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T03:44:57.353966+0000","last_change":"2024-09-21T03:44:57.350478+0000","last_active":"2024-09-21T03:44:57.353966+0000","last_peered":"2024-09-21T03:44:57.353966+0000","last_clean":"2024-09-21T03:44:57.353966+0000","last_became_active":"2024-09-21T03:44:57.348990+0000","last_became_peered":"2024-09-21T03:44:57.348990+0000","last_unstale":"2024-09-21T03:44:57.353966+0000","last_undegraded":"2024-09-21T03:44:57.353966+0000","last_fullsized":"2024-09-21T03:44:57.353966+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:40:27.993685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T03:44:56.330403+0000","last_change":"2024-09-21T03:44:56.329417+0000","last_active":"2024-09-21T03:44:56.330403+0000","last_peered":"2024-09-21T03:44:56.330403+0000","last_clean":"2024-09-21T03:44:56.330403+0000","last_became_active":"2024-09-21T03:44:56.329098+0000","last_became_peered":"2024-09-21T03:44:56.329098+0000","last_unstale":"2024-09-21T03:44:56.330403+0000","last_undegraded":"2024-09-21T03:44:56.330403+0000","last_fullsized":"2024-09-21T03:44:56.330403+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_clean_scrub_stamp":"2024-09-21T03:44:55.304777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:53:45.362390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.456620+0000","last_change":"2024-09-21T03:44:57.354679+0000","last_active":"2024-09-21T03:44:58.456620+0000","last_peered":"2024-09-21T03:44:58.456620+0000","last_clean":"2024-09-21T03:44:58.456620+0000","last_became_active":"2024-09-21T03:44:57.354357+0000","last_became_peered":"2024-09-21T03:44:57.354357+0000","last_unstale":"2024-09-21T03:44:58.456620+0000","last_undegraded":"2024-09-21T03:44:58.456620+0000","last_fullsized":"2024-09-21T03:44:58.456620+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:29:19.509171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T03:44:56.331506+0000","last_change":"2024-09-21T03:44:56.330054+0000","last_active":"2024-09-21T03:44:56.331506+0000","last_peered":"2024-09-21T03:44:56.331506+0000","last_clean":"2024-09-21T03:44:56.331506+0000","last_became_active":"2024-09-21T03:44:56.329151+0000","last_became_peered":"2024-09-21T03:44:56.329151+0000","last_unstale":"2024-09-21T03:44:56.331506+0000","last_undegraded":"2024-09-21T03:44:56.331506+0000","last_fullsized":"2024-09-21T03:44:56.331506+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_clean_scrub_stamp":"2024-09-21T03:44:55.304777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:15:21.096050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.456652+0000","last_change":"2024-09-21T03:44:57.363634+0000","last_active":"2024-09-21T03:44:58.456652+0000","last_peered":"2024-09-21T03:44:58.456652+0000","last_clean":"2024-09-21T03:44:58.456652+0000","last_became_active":"2024-09-21T03:44:57.363259+0000","last_became_peered":"2024-09-21T03:44:57.363259+0000","last_unstale":"2024-09-21T03:44:58.456652+0000","last_undegraded":"2024-09-21T03:44:58.456652+0000","last_fullsized":"2024-09-21T03:44:58.456652+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:53:39.529389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":15,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.353119+0000","last_change":"2024-09-21T03:44:56.327686+0000","last_active":"2024-09-21T03:44:58.353119+0000","last_peered":"2024-09-21T03:44:58.353119+0000","last_clean":"2024-09-21T03:44:58.353119+0000","last_became_active":"2024-09-21T03:44:56.327355+0000","last_became_peered":"2024-09-21T03:44:56.327355+0000","last_unstale":"2024-09-21T03:44:58.353119+0000","last_undegraded":"2024-09-21T03:44:58.353119+0000","last_fullsized":"2024-09-21T03:44:58.353119+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_clean_scrub_stamp":"2024-09-21T03:44:55.304777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:26:58.029665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":10,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T03:44:57.352767+0000","last_change":"2024-09-21T03:44:57.351050+0000","last_active":"2024-09-21T03:44:57.352767+0000","last_peered":"2024-09-21T03:44:57.352767+0000","last_clean":"2024-09-21T03:44:57.352767+0000","last_became_active":"2024-09-21T03:44:57.350731+0000","last_became_peered":"2024-09-21T03:44:57.350731+0000","last_unstale":"2024-09-21T03:44:57.352767+0000","last_undegraded":"2024-09-21T03:44:57.352767+0000","last_fullsized":"2024-09-21T03:44:57.352767+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:31:33.985328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T03:44:57.340435+0000","last_change":"2024-09-21T03:44:56.334506+0000","last_active":"2024-09-21T03:44:57.340435+0000","last_peered":"2024-09-21T03:44:57.340435+0000","last_clean":"2024-09-21T03:44:57.340435+0000","last_became_active":"2024-09-21T03:44:56.334007+0000","last_became_peered":"2024-09-21T03:44:56.334007+0000","last_unstale":"2024-09-21T03:44:57.340435+0000","last_undegraded":"2024-09-21T03:44:57.340435+0000","last_fullsized":"2024-09-21T03:44:57.340435+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_clean_scrub_stamp":"2024-09-21T03:44:55.304777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:57:15.575727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.456155+0000","last_change":"2024-09-21T03:44:57.350289+0000","last_active":"2024-09-21T03:44:58.456155+0000","last_peered":"2024-09-21T03:44:58.456155+0000","last_clean":"2024-09-21T03:44:58.456155+0000","last_became_active":"2024-09-21T03:44:57.349932+0000","last_became_peered":"2024-09-21T03:44:57.349932+0000","last_unstale":"2024-09-21T03:44:58.456155+0000","last_undegraded":"2024-09-21T03:44:58.456155+0000","last_fullsized":"2024-09-21T03:44:58.456155+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:49:41.592146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T03:44:56.329848+0000","last_change":"2024-09-21T03:44:56.328517+0000","last_active":"2024-09-21T03:44:56.329848+0000","last_peered":"2024-09-21T03:44:56.329848+0000","last_clean":"2024-09-21T03:44:56.329848+0000","last_became_active":"2024-09-21T03:44:56.328048+0000","last_became_peered":"2024-09-21T03:44:56.328048+0000","last_unstale":"2024-09-21T03:44:56.329848+0000","last_undegraded":"2024-09-21T03:44:56.329848+0000","last_fullsized":"2024-09-21T03:44:56.329848+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_clean_scrub_stamp":"2024-09-21T03:44:55.304777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:24:58.009008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.337538+0000","last_change":"2024-09-21T03:44:57.359324+0000","last_active":"2024-09-21T03:44:58.337538+0000","last_peered":"2024-09-21T03:44:58.337538+0000","last_clean":"2024-09-21T03:44:58.337538+0000","last_became_active":"2024-09-21T03:44:57.358966+0000","last_became_peered":"2024-09-21T03:44:57.358966+0000","last_unstale":"2024-09-21T03:44:58.337538+0000","last_undegraded":"2024-09-21T03:44:58.337538+0000","last_fullsized":"2024-09-21T03:44:58.337538+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:30:55.516774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T03:44:56.338457+0000","last_change":"2024-09-21T03:44:56.336489+0000","last_active":"2024-09-21T03:44:56.338457+0000","last_peered":"2024-09-21T03:44:56.338457+0000","last_clean":"2024-09-21T03:44:56.338457+0000","last_became_active":"2024-09-21T03:44:56.336089+0000","last_became_peered":"2024-09-21T03:44:56.336089+0000","last_unstale":"2024-09-21T03:44:56.338457+0000","last_undegraded":"2024-09-21T03:44:56.338457+0000","last_fullsized":"2024-09-21T03:44:56.338457+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_clean_scrub_stamp":"2024-09-21T03:44:55.304777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:30:01.117090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":10,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T03:44:57.356840+0000","last_change":"2024-09-21T03:44:57.355765+0000","last_active":"2024-09-21T03:44:57.356840+0000","last_peered":"2024-09-21T03:44:57.356840+0000","last_clean":"2024-09-21T03:44:57.356840+0000","last_became_active":"2024-09-21T03:44:57.355383+0000","last_became_peered":"2024-09-21T03:44:57.355383+0000","last_unstale":"2024-09-21T03:44:57.356840+0000","last_undegraded":"2024-09-21T03:44:57.356840+0000","last_fullsized":"2024-09-21T03:44:57.356840+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:44:27.836492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":15,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.337802+0000","last_change":"2024-09-21T03:44:56.338420+0000","last_active":"2024-09-21T03:44:58.337802+0000","last_peered":"2024-09-21T03:44:58.337802+0000","last_clean":"2024-09-21T03:44:58.337802+0000","last_became_active":"2024-09-21T03:44:56.337865+0000","last_became_peered":"2024-09-21T03:44:56.337865+0000","last_unstale":"2024-09-21T03:44:58.337802+0000","last_undegraded":"2024-09-21T03:44:58.337802+0000","last_fullsized":"2024-09-21T03:44:58.337802+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_clean_scrub_stamp":"2024-09-21T03:44:55.304777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:40:26.986852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.339010+0000","last_change":"2024-09-21T03:44:57.337661+0000","last_active":"2024-09-21T03:44:58.339010+0000","last_peered":"2024-09-21T03:44:58.339010+0000","last_clean":"2024-09-21T03:44:58.339010+0000","last_became_active":"2024-09-21T03:44:57.337356+0000","last_became_peered":"2024-09-21T03:44:57.337356+0000","last_unstale":"2024-09-21T03:44:58.339010+0000","last_undegraded":"2024-09-21T03:44:58.339010+0000","last_fullsized":"2024-09-21T03:44:58.339010+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:19:49.944906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T03:44:57.338702+0000","last_change":"2024-09-21T03:44:56.337786+0000","last_active":"2024-09-21T03:44:57.338702+0000","last_peered":"2024-09-21T03:44:57.338702+0000","last_clean":"2024-09-21T03:44:57.338702+0000","last_became_active":"2024-09-21T03:44:56.337258+0000","last_became_peered":"2024-09-21T03:44:56.337258+0000","last_unstale":"2024-09-21T03:44:57.338702+0000","last_undegraded":"2024-09-21T03:44:57.338702+0000","last_fullsized":"2024-09-21T03:44:57.338702+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_clean_scrub_stamp":"2024-09-21T03:44:55.304777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:29:06.273027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":10,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T03:44:57.351689+0000","last_change":"2024-09-21T03:44:57.350221+0000","last_active":"2024-09-21T03:44:57.351689+0000","last_peered":"2024-09-21T03:44:57.351689+0000","last_clean":"2024-09-21T03:44:57.351689+0000","last_became_active":"2024-09-21T03:44:57.349707+0000","last_became_peered":"2024-09-21T03:44:57.349707+0000","last_unstale":"2024-09-21T03:44:57.351689+0000","last_undegraded":"2024-09-21T03:44:57.351689+0000","last_fullsized":"2024-09-21T03:44:57.351689+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:50:57.792589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":15,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.337614+0000","last_change":"2024-09-21T03:44:56.340358+0000","last_active":"2024-09-21T03:44:58.337614+0000","last_peered":"2024-09-21T03:44:58.337614+0000","last_clean":"2024-09-21T03:44:58.337614+0000","last_became_active":"2024-09-21T03:44:56.340044+0000","last_became_peered":"2024-09-21T03:44:56.340044+0000","last_unstale":"2024-09-21T03:44:58.337614+0000","last_undegraded":"2024-09-21T03:44:58.337614+0000","last_fullsized":"2024-09-21T03:44:58.337614+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_clean_scrub_stamp":"2024-09-21T03:44:55.304777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:50:56.785756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":10,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T03:44:57.350495+0000","last_change":"2024-09-21T03:44:57.348196+0000","last_active":"2024-09-21T03:44:57.350495+0000","last_peered":"2024-09-21T03:44:57.350495+0000","last_clean":"2024-09-21T03:44:57.350495+0000","last_became_active":"2024-09-21T03:44:57.347856+0000","last_became_peered":"2024-09-21T03:44:57.347856+0000","last_unstale":"2024-09-21T03:44:57.350495+0000","last_undegraded":"2024-09-21T03:44:57.350495+0000","last_fullsized":"2024-09-21T03:44:57.350495+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:32:59.029041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T03:44:56.338491+0000","last_change":"2024-09-21T03:44:56.337049+0000","last_active":"2024-09-21T03:44:56.338491+0000","last_peered":"2024-09-21T03:44:56.338491+0000","last_clean":"2024-09-21T03:44:56.338491+0000","last_became_active":"2024-09-21T03:44:56.336315+0000","last_became_peered":"2024-09-21T03:44:56.336315+0000","last_unstale":"2024-09-21T03:44:56.338491+0000","last_undegraded":"2024-09-21T03:44:56.338491+0000","last_fullsized":"2024-09-21T03:44:56.338491+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_clean_scrub_stamp":"2024-09-21T03:44:55.304777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:31:45.105527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.353229+0000","last_change":"2024-09-21T03:44:57.366623+0000","last_active":"2024-09-21T03:44:58.353229+0000","last_peered":"2024-09-21T03:44:58.353229+0000","last_clean":"2024-09-21T03:44:58.353229+0000","last_became_active":"2024-09-21T03:44:57.366298+0000","last_became_peered":"2024-09-21T03:44:57.366298+0000","last_unstale":"2024-09-21T03:44:58.353229+0000","last_undegraded":"2024-09-21T03:44:58.353229+0000","last_fullsized":"2024-09-21T03:44:58.353229+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:01:23.359578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":15,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.338658+0000","last_change":"2024-09-21T03:44:56.339692+0000","last_active":"2024-09-21T03:44:58.338658+0000","last_peered":"2024-09-21T03:44:58.338658+0000","last_clean":"2024-09-21T03:44:58.338658+0000","last_became_active":"2024-09-21T03:44:56.339376+0000","last_became_peered":"2024-09-21T03:44:56.339376+0000","last_unstale":"2024-09-21T03:44:58.338658+0000","last_undegraded":"2024-09-21T03:44:58.338658+0000","last_fullsized":"2024-09-21T03:44:58.338658+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_clean_scrub_stamp":"2024-09-21T03:44:55.304777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:17:10.658062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.915523+0000","last_change":"2024-09-21T03:44:57.356308+0000","last_active":"2024-09-21T03:44:58.915523+0000","last_peered":"2024-09-21T03:44:58.915523+0000","last_clean":"2024-09-21T03:44:58.915523+0000","last_became_active":"2024-09-21T03:44:57.355511+0000","last_became_peered":"2024-09-21T03:44:57.355511+0000","last_unstale":"2024-09-21T03:44:58.915523+0000","last_undegraded":"2024-09-21T03:44:58.915523+0000","last_fullsized":"2024-09-21T03:44:58.915523+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:08:21.639028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T03:44:56.342190+0000","last_change":"2024-09-21T03:44:56.341499+0000","last_active":"2024-09-21T03:44:56.342190+0000","last_peered":"2024-09-21T03:44:56.342190+0000","last_clean":"2024-09-21T03:44:56.342190+0000","last_became_active":"2024-09-21T03:44:56.341058+0000","last_became_peered":"2024-09-21T03:44:56.341058+0000","last_unstale":"2024-09-21T03:44:56.342190+0000","last_undegraded":"2024-09-21T03:44:56.342190+0000","last_fullsized":"2024-09-21T03:44:56.342190+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_clean_scrub_stamp":"2024-09-21T03:44:55.304777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:51:47.787796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.352897+0000","last_change":"2024-09-21T03:44:57.355188+0000","last_active":"2024-09-21T03:44:58.352897+0000","last_peered":"2024-09-21T03:44:58.352897+0000","last_clean":"2024-09-21T03:44:58.352897+0000","last_became_active":"2024-09-21T03:44:57.354477+0000","last_became_peered":"2024-09-21T03:44:57.354477+0000","last_unstale":"2024-09-21T03:44:58.352897+0000","last_undegraded":"2024-09-21T03:44:58.352897+0000","last_fullsized":"2024-09-21T03:44:58.352897+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:28:53.047166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"145'2","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T03:44:57.366695+0000","last_change":"2024-09-21T03:44:56.335259+0000","last_active":"2024-09-21T03:44:57.366695+0000","last_peered":"2024-09-21T03:44:57.366695+0000","last_clean":"2024-09-21T03:44:57.366695+0000","last_became_active":"2024-09-21T03:44:56.334868+0000","last_became_peered":"2024-09-21T03:44:56.334868+0000","last_unstale":"2024-09-21T03:44:57.366695+0000","last_undegraded":"2024-09-21T03:44:57.366695+0000","last_fullsized":"2024-09-21T03:44:57.366695+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_clean_scrub_stamp":"2024-09-21T03:44:55.304777+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:45:29.403030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":355,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.338537+0000","last_change":"2024-09-21T03:33:25.319776+0000","last_active":"2024-09-21T03:44:58.338537+0000","last_peered":"2024-09-21T03:44:58.338537+0000","last_clean":"2024-09-21T03:44:58.338537+0000","last_became_active":"2024-09-21T03:33:25.318399+0000","last_became_peered":"2024-09-21T03:33:25.318399+0000","last_unstale":"2024-09-21T03:44:58.338537+0000","last_undegraded":"2024-09-21T03:44:58.338537+0000","last_fullsized":"2024-09-21T03:44:58.338537+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:23.016672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:23.016672+0000","last_clean_scrub_stamp":"2024-09-21T03:33:23.016672+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:32:35.906114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":4696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":33,"ondisk_log_size":33,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":1,"up_from":16,"seq":68719476879,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41936,"kb_used_data":2428,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94329904,"statfs":{"total":96636764160,"available":96593821696,"internally_reserved":0,"allocated":2486272,"data_stored":1546129,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509583,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41356,"kb_used_data":1848,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330484,"statfs":{"total":96636764160,"available":96594415616,"internally_reserved":0,"allocated":1892352,"data_stored":956321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":40434123},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509582,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41364,"kb_used_data":1852,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330476,"statfs":{"total":96636764160,"available":96594407424,"internally_reserved":0,"allocated":1896448,"data_stored":957281,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509583,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37184,"kb_used_data":1836,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334656,"statfs":{"total":96636764160,"available":96598687744,"internally_reserved":0,"allocated":1880064,"data_stored":949089,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509582,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37180,"kb_used_data":1832,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334660,"statfs":{"total":96636764160,"available":96598691840,"internally_reserved":0,"allocated":1875968,"data_stored":949559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542286,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33028,"kb_used_data":1840,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338812,"statfs":{"total":96636764160,"available":96602943488,"internally_reserved":0,"allocated":1884160,"data_stored":950589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574990,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37744,"kb_used_data":2396,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334096,"statfs":{"total":96636764160,"available":96598114304,"internally_reserved":0,"allocated":2453504,"data_stored":1532178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542287,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37168,"kb_used_data":1816,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334672,"statfs":{"total":96636764160,"available":96598704128,"internally_reserved":0,"allocated":1859584,"data_stored":941810,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T03:45:01.433 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T03:45:01.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:01.583+0000 7ff10bbe6640 1 -- 172.21.15.38:0/3045018118 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff1040b2e40 msgr2=0x7ff1040b3250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:01.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:01.583+0000 7ff10bbe6640 1 --2- 172.21.15.38:0/3045018118 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff1040b2e40 0x7ff1040b3250 secure :-1 s=READY pgs=2363 cs=0 l=1 rev1=1 crypto rx=0x7ff1040d5250 tx=0x7ff0fc02aa30 comp rx=0 tx=0).stop 2024-09-21T03:45:01.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:01.584+0000 7ff10bbe6640 1 -- 172.21.15.38:0/3045018118 shutdown_connections 2024-09-21T03:45:01.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:01.584+0000 7ff10bbe6640 1 --2- 172.21.15.38:0/3045018118 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff1040b2e40 0x7ff1040b3250 unknown :-1 s=CLOSED pgs=2363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:01.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:01.584+0000 7ff10bbe6640 1 --2- 172.21.15.38:0/3045018118 >> v2:172.21.15.195:3300/0 conn(0x7ff104162280 0x7ff104162690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:01.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:01.584+0000 7ff10bbe6640 1 --2- 172.21.15.38:0/3045018118 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff104162f00 0x7ff1040d24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:01.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:01.584+0000 7ff10bbe6640 1 -- 172.21.15.38:0/3045018118 >> 172.21.15.38:0/3045018118 conn(0x7ff1040b6b20 msgr2=0x7ff1040d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:01.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:01.584+0000 7ff10bbe6640 1 -- 172.21.15.38:0/3045018118 shutdown_connections 2024-09-21T03:45:01.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:01.584+0000 7ff10bbe6640 1 -- 172.21.15.38:0/3045018118 wait complete. 2024-09-21T03:45:01.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:01.585+0000 7ff10bbe6640 1 Processor -- start 2024-09-21T03:45:01.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:01.586+0000 7ff10bbe6640 1 -- start start 2024-09-21T03:45:01.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:01.587+0000 7ff10bbe6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff1040b2e40 0x7ff1041f5e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:01.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:01.587+0000 7ff10bbe6640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff104162280 0x7ff1040ce370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:01.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:01.587+0000 7ff10bbe6640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff104159840 0x7ff10415e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:01.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:01.587+0000 7ff10bbe6640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ff1040b3dd0 con 0x7ff1040b2e40 2024-09-21T03:45:01.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:01.587+0000 7ff10bbe6640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ff1040b39a0 con 0x7ff104159840 2024-09-21T03:45:01.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:01.587+0000 7ff10bbe6640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7ff1040b3f50 con 0x7ff104162280 2024-09-21T03:45:01.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:01.587+0000 7ff109982640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff1040b2e40 0x7ff1041f5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:01.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:01.587+0000 7ff109982640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff1040b2e40 0x7ff1041f5e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40580/0 (socket says 172.21.15.38:40580) 2024-09-21T03:45:01.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:01.587+0000 7ff109982640 1 -- 172.21.15.38:0/3355720655 learned_addr learned my addr 172.21.15.38:0/3355720655 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:45:01.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:01.587+0000 7ff109181640 1 --2- 172.21.15.38:0/3355720655 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff104162280 0x7ff1040ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:01.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:01.587+0000 7ff109982640 1 -- 172.21.15.38:0/3355720655 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff104162280 msgr2=0x7ff1040ce370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:01.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:01.587+0000 7ff109982640 1 --2- 172.21.15.38:0/3355720655 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff104162280 0x7ff1040ce370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:01.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:01.587+0000 7ff109982640 1 -- 172.21.15.38:0/3355720655 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff104159840 msgr2=0x7ff10415e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:01.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:01.587+0000 7ff109982640 1 --2- 172.21.15.38:0/3355720655 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff104159840 0x7ff10415e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:01.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:01.587+0000 7ff109982640 1 -- 172.21.15.38:0/3355720655 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1040cb150 con 0x7ff1040b2e40 2024-09-21T03:45:01.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:01.587+0000 7ff109982640 1 --2- 172.21.15.38:0/3355720655 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff1040b2e40 0x7ff1041f5e10 secure :-1 s=READY pgs=2364 cs=0 l=1 rev1=1 crypto rx=0x7ff10000d6d0 tx=0x7ff10000db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:01.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:01.587+0000 7ff0fa7fc640 1 -- 172.21.15.38:0/3355720655 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff100017070 con 0x7ff1040b2e40 2024-09-21T03:45:01.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:01.588+0000 7ff0fa7fc640 1 -- 172.21.15.38:0/3355720655 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff100005330 con 0x7ff1040b2e40 2024-09-21T03:45:01.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:01.588+0000 7ff10bbe6640 1 -- 172.21.15.38:0/3355720655 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1040cd5b0 con 0x7ff1040b2e40 2024-09-21T03:45:01.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:01.588+0000 7ff0fa7fc640 1 -- 172.21.15.38:0/3355720655 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff100009820 con 0x7ff1040b2e40 2024-09-21T03:45:01.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:01.588+0000 7ff10bbe6640 1 -- 172.21.15.38:0/3355720655 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1040cdb70 con 0x7ff1040b2e40 2024-09-21T03:45:01.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:01.589+0000 7ff10bbe6640 1 -- 172.21.15.38:0/3355720655 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff104162f00 con 0x7ff1040b2e40 2024-09-21T03:45:01.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:01.589+0000 7ff0fa7fc640 1 -- 172.21.15.38:0/3355720655 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ff100009bd0 con 0x7ff1040b2e40 2024-09-21T03:45:01.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:01.590+0000 7ff0fa7fc640 1 --2- 172.21.15.38:0/3355720655 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff0e8078b90 0x7ff0e807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:01.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:01.590+0000 7ff0fa7fc640 1 -- 172.21.15.38:0/3355720655 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 6101+0+0 (secure 0 0 0) 0x7ff10009b210 con 0x7ff1040b2e40 2024-09-21T03:45:01.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:01.591+0000 7ff109181640 1 --2- 172.21.15.38:0/3355720655 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff0e8078b90 0x7ff0e807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:01.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:01.591+0000 7ff109181640 1 --2- 172.21.15.38:0/3355720655 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff0e8078b90 0x7ff0e807b0a0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7ff1040b5670 tx=0x7ff0f4005c90 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:01.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:01.595+0000 7ff0fa7fc640 1 -- 172.21.15.38:0/3355720655 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7ff1000634c0 con 0x7ff1040b2e40 2024-09-21T03:45:01.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:01.780+0000 7ff10bbe6640 1 -- 172.21.15.38:0/3355720655 --> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff1040b3570 con 0x7ff0e8078b90 2024-09-21T03:45:01.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:01.785+0000 7ff0fa7fc640 1 -- 172.21.15.38:0/3355720655 <== mgr.4117 v2:172.21.15.195:6800/1631901344 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252055 (secure 0 0 0) 0x7ff1040b3570 con 0x7ff0e8078b90 2024-09-21T03:45:01.789 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:45:01.790 INFO:teuthology.orchestra.run.smithi038.stderr:dumped all 2024-09-21T03:45:01.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:01.788+0000 7ff10bbe6640 1 -- 172.21.15.38:0/3355720655 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff0e8078b90 msgr2=0x7ff0e807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:01.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:01.788+0000 7ff10bbe6640 1 --2- 172.21.15.38:0/3355720655 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff0e8078b90 0x7ff0e807b0a0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7ff1040b5670 tx=0x7ff0f4005c90 comp rx=0 tx=0).stop 2024-09-21T03:45:01.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:01.788+0000 7ff10bbe6640 1 -- 172.21.15.38:0/3355720655 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff1040b2e40 msgr2=0x7ff1041f5e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:01.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:01.788+0000 7ff10bbe6640 1 --2- 172.21.15.38:0/3355720655 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff1040b2e40 0x7ff1041f5e10 secure :-1 s=READY pgs=2364 cs=0 l=1 rev1=1 crypto rx=0x7ff10000d6d0 tx=0x7ff10000db90 comp rx=0 tx=0).stop 2024-09-21T03:45:01.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:01.788+0000 7ff10bbe6640 1 -- 172.21.15.38:0/3355720655 shutdown_connections 2024-09-21T03:45:01.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:01.788+0000 7ff10bbe6640 1 --2- 172.21.15.38:0/3355720655 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff104159840 0x7ff10415e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:01.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:01.788+0000 7ff10bbe6640 1 --2- 172.21.15.38:0/3355720655 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff0e8078b90 0x7ff0e807b0a0 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:01.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:01.788+0000 7ff10bbe6640 1 --2- 172.21.15.38:0/3355720655 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff104162280 0x7ff1040ce370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:01.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:01.788+0000 7ff10bbe6640 1 --2- 172.21.15.38:0/3355720655 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff1040b2e40 0x7ff1041f5e10 unknown :-1 s=CLOSED pgs=2364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:01.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:01.788+0000 7ff10bbe6640 1 -- 172.21.15.38:0/3355720655 >> 172.21.15.38:0/3355720655 conn(0x7ff1040b6b20 msgr2=0x7ff10415c880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:01.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:01.788+0000 7ff10bbe6640 1 -- 172.21.15.38:0/3355720655 shutdown_connections 2024-09-21T03:45:01.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:01.788+0000 7ff10bbe6640 1 -- 172.21.15.38:0/3355720655 wait complete. 2024-09-21T03:45:01.801 INFO:teuthology.orchestra.run.smithi038.stdout:{"pg_ready":true,"pg_map":{"version":488,"stamp":"2024-09-21T03:45:01.172110+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":141,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":105,"ondisk_log_size":105,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":118,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":306960,"kb_used_data":15848,"kb_used_omap":12,"kb_used_meta":290931,"kb_avail":754667760,"statfs":{"total":773094113280,"available":772779786240,"internally_reserved":0,"allocated":16228352,"data_stored":8782956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12748,"internal_metadata":297913908},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.859588"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":10,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T03:44:57.348031+0000","last_change":"2024-09-21T03:44:57.340688+0000","last_active":"2024-09-21T03:44:57.348031+0000","last_peered":"2024-09-21T03:44:57.348031+0000","last_clean":"2024-09-21T03:44:57.348031+0000","last_became_active":"2024-09-21T03:44:57.340356+0000","last_became_peered":"2024-09-21T03:44:57.340356+0000","last_unstale":"2024-09-21T03:44:57.348031+0000","last_undegraded":"2024-09-21T03:44:57.348031+0000","last_fullsized":"2024-09-21T03:44:57.348031+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:31:55.663249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.915466+0000","last_change":"2024-09-21T03:44:57.354834+0000","last_active":"2024-09-21T03:44:58.915466+0000","last_peered":"2024-09-21T03:44:58.915466+0000","last_clean":"2024-09-21T03:44:58.915466+0000","last_became_active":"2024-09-21T03:44:57.354518+0000","last_became_peered":"2024-09-21T03:44:57.354518+0000","last_unstale":"2024-09-21T03:44:58.915466+0000","last_undegraded":"2024-09-21T03:44:58.915466+0000","last_fullsized":"2024-09-21T03:44:58.915466+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:21:26.621000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":1,"reported_epoch":144,"state":"creating+peering","last_fresh":"2024-09-21T03:44:56.319458+0000","last_change":"2024-09-21T03:44:56.319458+0000","last_active":"2024-09-21T03:44:56.311610+0000","last_peered":"2024-09-21T03:44:56.311610+0000","last_clean":"2024-09-21T03:44:56.311610+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:44:56.319458+0000","last_undegraded":"2024-09-21T03:44:56.319458+0000","last_fullsized":"2024-09-21T03:44:56.319458+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":10,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T03:44:57.359756+0000","last_change":"2024-09-21T03:44:57.356466+0000","last_active":"2024-09-21T03:44:57.359756+0000","last_peered":"2024-09-21T03:44:57.359756+0000","last_clean":"2024-09-21T03:44:57.359756+0000","last_became_active":"2024-09-21T03:44:57.355938+0000","last_became_peered":"2024-09-21T03:44:57.355938+0000","last_unstale":"2024-09-21T03:44:57.359756+0000","last_undegraded":"2024-09-21T03:44:57.359756+0000","last_fullsized":"2024-09-21T03:44:57.359756+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:05:36.290619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":10,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T03:44:57.353943+0000","last_change":"2024-09-21T03:44:57.351972+0000","last_active":"2024-09-21T03:44:57.353943+0000","last_peered":"2024-09-21T03:44:57.353943+0000","last_clean":"2024-09-21T03:44:57.353943+0000","last_became_active":"2024-09-21T03:44:57.351561+0000","last_became_peered":"2024-09-21T03:44:57.351561+0000","last_unstale":"2024-09-21T03:44:57.353943+0000","last_undegraded":"2024-09-21T03:44:57.353943+0000","last_fullsized":"2024-09-21T03:44:57.353943+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:21:51.370818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.353120+0000","last_change":"2024-09-21T03:44:57.349848+0000","last_active":"2024-09-21T03:44:58.353120+0000","last_peered":"2024-09-21T03:44:58.353120+0000","last_clean":"2024-09-21T03:44:58.353120+0000","last_became_active":"2024-09-21T03:44:57.349387+0000","last_became_peered":"2024-09-21T03:44:57.349387+0000","last_unstale":"2024-09-21T03:44:58.353120+0000","last_undegraded":"2024-09-21T03:44:58.353120+0000","last_fullsized":"2024-09-21T03:44:58.353120+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:10:55.772054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.352606+0000","last_change":"2024-09-21T03:44:57.355533+0000","last_active":"2024-09-21T03:44:58.352606+0000","last_peered":"2024-09-21T03:44:58.352606+0000","last_clean":"2024-09-21T03:44:58.352606+0000","last_became_active":"2024-09-21T03:44:57.354892+0000","last_became_peered":"2024-09-21T03:44:57.354892+0000","last_unstale":"2024-09-21T03:44:58.352606+0000","last_undegraded":"2024-09-21T03:44:58.352606+0000","last_fullsized":"2024-09-21T03:44:58.352606+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:05:41.952462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":10,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T03:44:57.356024+0000","last_change":"2024-09-21T03:44:57.350864+0000","last_active":"2024-09-21T03:44:57.356024+0000","last_peered":"2024-09-21T03:44:57.356024+0000","last_clean":"2024-09-21T03:44:57.356024+0000","last_became_active":"2024-09-21T03:44:57.350215+0000","last_became_peered":"2024-09-21T03:44:57.350215+0000","last_unstale":"2024-09-21T03:44:57.356024+0000","last_undegraded":"2024-09-21T03:44:57.356024+0000","last_fullsized":"2024-09-21T03:44:57.356024+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:23:12.061449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.915427+0000","last_change":"2024-09-21T03:44:57.356640+0000","last_active":"2024-09-21T03:44:58.915427+0000","last_peered":"2024-09-21T03:44:58.915427+0000","last_clean":"2024-09-21T03:44:58.915427+0000","last_became_active":"2024-09-21T03:44:57.355966+0000","last_became_peered":"2024-09-21T03:44:57.355966+0000","last_unstale":"2024-09-21T03:44:58.915427+0000","last_undegraded":"2024-09-21T03:44:58.915427+0000","last_fullsized":"2024-09-21T03:44:58.915427+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:40:49.216619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":10,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T03:44:57.359889+0000","last_change":"2024-09-21T03:44:57.359320+0000","last_active":"2024-09-21T03:44:57.359889+0000","last_peered":"2024-09-21T03:44:57.359889+0000","last_clean":"2024-09-21T03:44:57.359889+0000","last_became_active":"2024-09-21T03:44:57.358988+0000","last_became_peered":"2024-09-21T03:44:57.358988+0000","last_unstale":"2024-09-21T03:44:57.359889+0000","last_undegraded":"2024-09-21T03:44:57.359889+0000","last_fullsized":"2024-09-21T03:44:57.359889+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:29:11.643073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.337467+0000","last_change":"2024-09-21T03:44:57.362260+0000","last_active":"2024-09-21T03:44:58.337467+0000","last_peered":"2024-09-21T03:44:58.337467+0000","last_clean":"2024-09-21T03:44:58.337467+0000","last_became_active":"2024-09-21T03:44:57.361624+0000","last_became_peered":"2024-09-21T03:44:57.361624+0000","last_unstale":"2024-09-21T03:44:58.337467+0000","last_undegraded":"2024-09-21T03:44:58.337467+0000","last_fullsized":"2024-09-21T03:44:58.337467+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:00:23.014227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.352884+0000","last_change":"2024-09-21T03:44:57.353828+0000","last_active":"2024-09-21T03:44:58.352884+0000","last_peered":"2024-09-21T03:44:58.352884+0000","last_clean":"2024-09-21T03:44:58.352884+0000","last_became_active":"2024-09-21T03:44:57.353516+0000","last_became_peered":"2024-09-21T03:44:57.353516+0000","last_unstale":"2024-09-21T03:44:58.352884+0000","last_undegraded":"2024-09-21T03:44:58.352884+0000","last_fullsized":"2024-09-21T03:44:58.352884+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:49:05.536169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":10,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T03:44:57.366752+0000","last_change":"2024-09-21T03:44:57.363696+0000","last_active":"2024-09-21T03:44:57.366752+0000","last_peered":"2024-09-21T03:44:57.366752+0000","last_clean":"2024-09-21T03:44:57.366752+0000","last_became_active":"2024-09-21T03:44:57.362737+0000","last_became_peered":"2024-09-21T03:44:57.362737+0000","last_unstale":"2024-09-21T03:44:57.366752+0000","last_undegraded":"2024-09-21T03:44:57.366752+0000","last_fullsized":"2024-09-21T03:44:57.366752+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:58:38.157904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.353349+0000","last_change":"2024-09-21T03:44:57.352690+0000","last_active":"2024-09-21T03:44:58.353349+0000","last_peered":"2024-09-21T03:44:58.353349+0000","last_clean":"2024-09-21T03:44:58.353349+0000","last_became_active":"2024-09-21T03:44:57.352358+0000","last_became_peered":"2024-09-21T03:44:57.352358+0000","last_unstale":"2024-09-21T03:44:58.353349+0000","last_undegraded":"2024-09-21T03:44:58.353349+0000","last_fullsized":"2024-09-21T03:44:58.353349+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:42:27.812988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.915095+0000","last_change":"2024-09-21T03:44:57.369797+0000","last_active":"2024-09-21T03:44:58.915095+0000","last_peered":"2024-09-21T03:44:58.915095+0000","last_clean":"2024-09-21T03:44:58.915095+0000","last_became_active":"2024-09-21T03:44:57.369463+0000","last_became_peered":"2024-09-21T03:44:57.369463+0000","last_unstale":"2024-09-21T03:44:58.915095+0000","last_undegraded":"2024-09-21T03:44:58.915095+0000","last_fullsized":"2024-09-21T03:44:58.915095+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:59:36.159782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.338697+0000","last_change":"2024-09-21T03:44:57.341545+0000","last_active":"2024-09-21T03:44:58.338697+0000","last_peered":"2024-09-21T03:44:58.338697+0000","last_clean":"2024-09-21T03:44:58.338697+0000","last_became_active":"2024-09-21T03:44:57.341220+0000","last_became_peered":"2024-09-21T03:44:57.341220+0000","last_unstale":"2024-09-21T03:44:58.338697+0000","last_undegraded":"2024-09-21T03:44:58.338697+0000","last_fullsized":"2024-09-21T03:44:58.338697+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:40:08.834183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":1,"reported_epoch":144,"state":"creating+peering","last_fresh":"2024-09-21T03:44:56.321953+0000","last_change":"2024-09-21T03:44:56.321953+0000","last_active":"2024-09-21T03:44:56.311610+0000","last_peered":"2024-09-21T03:44:56.311610+0000","last_clean":"2024-09-21T03:44:56.311610+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:44:56.321953+0000","last_undegraded":"2024-09-21T03:44:56.321953+0000","last_fullsized":"2024-09-21T03:44:56.321953+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.915438+0000","last_change":"2024-09-21T03:44:57.357272+0000","last_active":"2024-09-21T03:44:58.915438+0000","last_peered":"2024-09-21T03:44:58.915438+0000","last_clean":"2024-09-21T03:44:58.915438+0000","last_became_active":"2024-09-21T03:44:57.356952+0000","last_became_peered":"2024-09-21T03:44:57.356952+0000","last_unstale":"2024-09-21T03:44:58.915438+0000","last_undegraded":"2024-09-21T03:44:58.915438+0000","last_fullsized":"2024-09-21T03:44:58.915438+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:35:23.462964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":10,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T03:44:57.354219+0000","last_change":"2024-09-21T03:44:57.351765+0000","last_active":"2024-09-21T03:44:57.354219+0000","last_peered":"2024-09-21T03:44:57.354219+0000","last_clean":"2024-09-21T03:44:57.354219+0000","last_became_active":"2024-09-21T03:44:57.351478+0000","last_became_peered":"2024-09-21T03:44:57.351478+0000","last_unstale":"2024-09-21T03:44:57.354219+0000","last_undegraded":"2024-09-21T03:44:57.354219+0000","last_fullsized":"2024-09-21T03:44:57.354219+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:00:35.318345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.339047+0000","last_change":"2024-09-21T03:44:57.358796+0000","last_active":"2024-09-21T03:44:58.339047+0000","last_peered":"2024-09-21T03:44:58.339047+0000","last_clean":"2024-09-21T03:44:58.339047+0000","last_became_active":"2024-09-21T03:44:57.358439+0000","last_became_peered":"2024-09-21T03:44:57.358439+0000","last_unstale":"2024-09-21T03:44:58.339047+0000","last_undegraded":"2024-09-21T03:44:58.339047+0000","last_fullsized":"2024-09-21T03:44:58.339047+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:05:51.280587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":1,"reported_epoch":144,"state":"creating+peering","last_fresh":"2024-09-21T03:44:56.322208+0000","last_change":"2024-09-21T03:44:56.322208+0000","last_active":"2024-09-21T03:44:56.311610+0000","last_peered":"2024-09-21T03:44:56.311610+0000","last_clean":"2024-09-21T03:44:56.311610+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:44:56.322208+0000","last_undegraded":"2024-09-21T03:44:56.322208+0000","last_fullsized":"2024-09-21T03:44:56.322208+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.456326+0000","last_change":"2024-09-21T03:44:57.357854+0000","last_active":"2024-09-21T03:44:58.456326+0000","last_peered":"2024-09-21T03:44:58.456326+0000","last_clean":"2024-09-21T03:44:58.456326+0000","last_became_active":"2024-09-21T03:44:57.357541+0000","last_became_peered":"2024-09-21T03:44:57.357541+0000","last_unstale":"2024-09-21T03:44:58.456326+0000","last_undegraded":"2024-09-21T03:44:58.456326+0000","last_fullsized":"2024-09-21T03:44:58.456326+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:04:52.143962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":10,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T03:44:57.355917+0000","last_change":"2024-09-21T03:44:57.353700+0000","last_active":"2024-09-21T03:44:57.355917+0000","last_peered":"2024-09-21T03:44:57.355917+0000","last_clean":"2024-09-21T03:44:57.355917+0000","last_became_active":"2024-09-21T03:44:57.351501+0000","last_became_peered":"2024-09-21T03:44:57.351501+0000","last_unstale":"2024-09-21T03:44:57.355917+0000","last_undegraded":"2024-09-21T03:44:57.355917+0000","last_fullsized":"2024-09-21T03:44:57.355917+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:00:30.971449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":10,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T03:44:57.356048+0000","last_change":"2024-09-21T03:44:57.351638+0000","last_active":"2024-09-21T03:44:57.356048+0000","last_peered":"2024-09-21T03:44:57.356048+0000","last_clean":"2024-09-21T03:44:57.356048+0000","last_became_active":"2024-09-21T03:44:57.351325+0000","last_became_peered":"2024-09-21T03:44:57.351325+0000","last_unstale":"2024-09-21T03:44:57.356048+0000","last_undegraded":"2024-09-21T03:44:57.356048+0000","last_fullsized":"2024-09-21T03:44:57.356048+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:05:57.831029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.339252+0000","last_change":"2024-09-21T03:44:57.359418+0000","last_active":"2024-09-21T03:44:58.339252+0000","last_peered":"2024-09-21T03:44:58.339252+0000","last_clean":"2024-09-21T03:44:58.339252+0000","last_became_active":"2024-09-21T03:44:57.358564+0000","last_became_peered":"2024-09-21T03:44:57.358564+0000","last_unstale":"2024-09-21T03:44:58.339252+0000","last_undegraded":"2024-09-21T03:44:58.339252+0000","last_fullsized":"2024-09-21T03:44:58.339252+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:21:26.978675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.339318+0000","last_change":"2024-09-21T03:44:57.360155+0000","last_active":"2024-09-21T03:44:58.339318+0000","last_peered":"2024-09-21T03:44:58.339318+0000","last_clean":"2024-09-21T03:44:58.339318+0000","last_became_active":"2024-09-21T03:44:57.359645+0000","last_became_peered":"2024-09-21T03:44:57.359645+0000","last_unstale":"2024-09-21T03:44:58.339318+0000","last_undegraded":"2024-09-21T03:44:58.339318+0000","last_fullsized":"2024-09-21T03:44:58.339318+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:14:53.602909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.339127+0000","last_change":"2024-09-21T03:44:57.357568+0000","last_active":"2024-09-21T03:44:58.339127+0000","last_peered":"2024-09-21T03:44:58.339127+0000","last_clean":"2024-09-21T03:44:58.339127+0000","last_became_active":"2024-09-21T03:44:57.357185+0000","last_became_peered":"2024-09-21T03:44:57.357185+0000","last_unstale":"2024-09-21T03:44:58.339127+0000","last_undegraded":"2024-09-21T03:44:58.339127+0000","last_fullsized":"2024-09-21T03:44:58.339127+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:57:24.936155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.353110+0000","last_change":"2024-09-21T03:44:57.352231+0000","last_active":"2024-09-21T03:44:58.353110+0000","last_peered":"2024-09-21T03:44:58.353110+0000","last_clean":"2024-09-21T03:44:58.353110+0000","last_became_active":"2024-09-21T03:44:57.351887+0000","last_became_peered":"2024-09-21T03:44:57.351887+0000","last_unstale":"2024-09-21T03:44:58.353110+0000","last_undegraded":"2024-09-21T03:44:58.353110+0000","last_fullsized":"2024-09-21T03:44:58.353110+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:46:07.069681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":10,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T03:44:57.358271+0000","last_change":"2024-09-21T03:44:57.357470+0000","last_active":"2024-09-21T03:44:57.358271+0000","last_peered":"2024-09-21T03:44:57.358271+0000","last_clean":"2024-09-21T03:44:57.358271+0000","last_became_active":"2024-09-21T03:44:57.357091+0000","last_became_peered":"2024-09-21T03:44:57.357091+0000","last_unstale":"2024-09-21T03:44:57.358271+0000","last_undegraded":"2024-09-21T03:44:57.358271+0000","last_fullsized":"2024-09-21T03:44:57.358271+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:13:25.917453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.337478+0000","last_change":"2024-09-21T03:44:57.361022+0000","last_active":"2024-09-21T03:44:58.337478+0000","last_peered":"2024-09-21T03:44:58.337478+0000","last_clean":"2024-09-21T03:44:58.337478+0000","last_became_active":"2024-09-21T03:44:57.360697+0000","last_became_peered":"2024-09-21T03:44:57.360697+0000","last_unstale":"2024-09-21T03:44:58.337478+0000","last_undegraded":"2024-09-21T03:44:58.337478+0000","last_fullsized":"2024-09-21T03:44:58.337478+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:17:17.662702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.352611+0000","last_change":"2024-09-21T03:44:57.357841+0000","last_active":"2024-09-21T03:44:58.352611+0000","last_peered":"2024-09-21T03:44:58.352611+0000","last_clean":"2024-09-21T03:44:58.352611+0000","last_became_active":"2024-09-21T03:44:57.357299+0000","last_became_peered":"2024-09-21T03:44:57.357299+0000","last_unstale":"2024-09-21T03:44:58.352611+0000","last_undegraded":"2024-09-21T03:44:58.352611+0000","last_fullsized":"2024-09-21T03:44:58.352611+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:36:45.265239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.456162+0000","last_change":"2024-09-21T03:44:57.366423+0000","last_active":"2024-09-21T03:44:58.456162+0000","last_peered":"2024-09-21T03:44:58.456162+0000","last_clean":"2024-09-21T03:44:58.456162+0000","last_became_active":"2024-09-21T03:44:57.366083+0000","last_became_peered":"2024-09-21T03:44:57.366083+0000","last_unstale":"2024-09-21T03:44:58.456162+0000","last_undegraded":"2024-09-21T03:44:58.456162+0000","last_fullsized":"2024-09-21T03:44:58.456162+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:14:54.627243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":1,"reported_epoch":144,"state":"creating+peering","last_fresh":"2024-09-21T03:44:56.321750+0000","last_change":"2024-09-21T03:44:56.321750+0000","last_active":"2024-09-21T03:44:56.311610+0000","last_peered":"2024-09-21T03:44:56.311610+0000","last_clean":"2024-09-21T03:44:56.311610+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:44:56.321750+0000","last_undegraded":"2024-09-21T03:44:56.321750+0000","last_fullsized":"2024-09-21T03:44:56.321750+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1f","version":"145'4","reported_seq":19,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.456694+0000","last_change":"2024-09-21T03:44:56.332485+0000","last_active":"2024-09-21T03:44:58.456694+0000","last_peered":"2024-09-21T03:44:58.456694+0000","last_clean":"2024-09-21T03:44:58.456694+0000","last_became_active":"2024-09-21T03:44:56.332204+0000","last_became_peered":"2024-09-21T03:44:56.332204+0000","last_unstale":"2024-09-21T03:44:58.456694+0000","last_undegraded":"2024-09-21T03:44:58.456694+0000","last_fullsized":"2024-09-21T03:44:58.456694+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_clean_scrub_stamp":"2024-09-21T03:44:55.304777+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:15:02.688269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":10,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T03:44:57.356612+0000","last_change":"2024-09-21T03:44:57.352443+0000","last_active":"2024-09-21T03:44:57.356612+0000","last_peered":"2024-09-21T03:44:57.356612+0000","last_clean":"2024-09-21T03:44:57.356612+0000","last_became_active":"2024-09-21T03:44:57.351086+0000","last_became_peered":"2024-09-21T03:44:57.351086+0000","last_unstale":"2024-09-21T03:44:57.356612+0000","last_undegraded":"2024-09-21T03:44:57.356612+0000","last_fullsized":"2024-09-21T03:44:57.356612+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:27:41.239844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T03:44:56.342206+0000","last_change":"2024-09-21T03:44:56.341615+0000","last_active":"2024-09-21T03:44:56.342206+0000","last_peered":"2024-09-21T03:44:56.342206+0000","last_clean":"2024-09-21T03:44:56.342206+0000","last_became_active":"2024-09-21T03:44:56.341306+0000","last_became_peered":"2024-09-21T03:44:56.341306+0000","last_unstale":"2024-09-21T03:44:56.342206+0000","last_undegraded":"2024-09-21T03:44:56.342206+0000","last_fullsized":"2024-09-21T03:44:56.342206+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_clean_scrub_stamp":"2024-09-21T03:44:55.304777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:36:07.439425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.915135+0000","last_change":"2024-09-21T03:44:57.349398+0000","last_active":"2024-09-21T03:44:58.915135+0000","last_peered":"2024-09-21T03:44:58.915135+0000","last_clean":"2024-09-21T03:44:58.915135+0000","last_became_active":"2024-09-21T03:44:57.349067+0000","last_became_peered":"2024-09-21T03:44:57.349067+0000","last_unstale":"2024-09-21T03:44:58.915135+0000","last_undegraded":"2024-09-21T03:44:58.915135+0000","last_fullsized":"2024-09-21T03:44:58.915135+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:19:45.952024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"145'2","reported_seq":17,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.915396+0000","last_change":"2024-09-21T03:44:56.343325+0000","last_active":"2024-09-21T03:44:58.915396+0000","last_peered":"2024-09-21T03:44:58.915396+0000","last_clean":"2024-09-21T03:44:58.915396+0000","last_became_active":"2024-09-21T03:44:56.342964+0000","last_became_peered":"2024-09-21T03:44:56.342964+0000","last_unstale":"2024-09-21T03:44:58.915396+0000","last_undegraded":"2024-09-21T03:44:58.915396+0000","last_fullsized":"2024-09-21T03:44:58.915396+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_clean_scrub_stamp":"2024-09-21T03:44:55.304777+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:19:36.404105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.337488+0000","last_change":"2024-09-21T03:44:57.360138+0000","last_active":"2024-09-21T03:44:58.337488+0000","last_peered":"2024-09-21T03:44:58.337488+0000","last_clean":"2024-09-21T03:44:58.337488+0000","last_became_active":"2024-09-21T03:44:57.359800+0000","last_became_peered":"2024-09-21T03:44:57.359800+0000","last_unstale":"2024-09-21T03:44:58.337488+0000","last_undegraded":"2024-09-21T03:44:58.337488+0000","last_fullsized":"2024-09-21T03:44:58.337488+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:33:24.015377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1c","version":"145'2","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T03:44:57.373298+0000","last_change":"2024-09-21T03:44:56.334534+0000","last_active":"2024-09-21T03:44:57.373298+0000","last_peered":"2024-09-21T03:44:57.373298+0000","last_clean":"2024-09-21T03:44:57.373298+0000","last_became_active":"2024-09-21T03:44:56.333800+0000","last_became_peered":"2024-09-21T03:44:56.333800+0000","last_unstale":"2024-09-21T03:44:57.373298+0000","last_undegraded":"2024-09-21T03:44:57.373298+0000","last_fullsized":"2024-09-21T03:44:57.373298+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_clean_scrub_stamp":"2024-09-21T03:44:55.304777+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:44:41.072083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":10,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T03:44:57.350445+0000","last_change":"2024-09-21T03:44:57.342301+0000","last_active":"2024-09-21T03:44:57.350445+0000","last_peered":"2024-09-21T03:44:57.350445+0000","last_clean":"2024-09-21T03:44:57.350445+0000","last_became_active":"2024-09-21T03:44:57.341880+0000","last_became_peered":"2024-09-21T03:44:57.341880+0000","last_unstale":"2024-09-21T03:44:57.350445+0000","last_undegraded":"2024-09-21T03:44:57.350445+0000","last_fullsized":"2024-09-21T03:44:57.350445+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:10:55.772054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1b","version":"145'1","reported_seq":16,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.337514+0000","last_change":"2024-09-21T03:44:56.342732+0000","last_active":"2024-09-21T03:44:58.337514+0000","last_peered":"2024-09-21T03:44:58.337514+0000","last_clean":"2024-09-21T03:44:58.337514+0000","last_became_active":"2024-09-21T03:44:56.342319+0000","last_became_peered":"2024-09-21T03:44:56.342319+0000","last_unstale":"2024-09-21T03:44:58.337514+0000","last_undegraded":"2024-09-21T03:44:58.337514+0000","last_fullsized":"2024-09-21T03:44:58.337514+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_clean_scrub_stamp":"2024-09-21T03:44:55.304777+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:31:32.978494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":10,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T03:44:57.350445+0000","last_change":"2024-09-21T03:44:57.346978+0000","last_active":"2024-09-21T03:44:57.350445+0000","last_peered":"2024-09-21T03:44:57.350445+0000","last_clean":"2024-09-21T03:44:57.350445+0000","last_became_active":"2024-09-21T03:44:57.346633+0000","last_became_peered":"2024-09-21T03:44:57.346633+0000","last_unstale":"2024-09-21T03:44:57.350445+0000","last_undegraded":"2024-09-21T03:44:57.350445+0000","last_fullsized":"2024-09-21T03:44:57.350445+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:17:57.312760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1a","version":"145'1","reported_seq":16,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.338782+0000","last_change":"2024-09-21T03:44:56.342500+0000","last_active":"2024-09-21T03:44:58.338782+0000","last_peered":"2024-09-21T03:44:58.338782+0000","last_clean":"2024-09-21T03:44:58.338782+0000","last_became_active":"2024-09-21T03:44:56.342198+0000","last_became_peered":"2024-09-21T03:44:56.342198+0000","last_unstale":"2024-09-21T03:44:58.338782+0000","last_undegraded":"2024-09-21T03:44:58.338782+0000","last_fullsized":"2024-09-21T03:44:58.338782+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_clean_scrub_stamp":"2024-09-21T03:44:55.304777+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:24:33.377335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.337947+0000","last_change":"2024-09-21T03:44:57.360511+0000","last_active":"2024-09-21T03:44:58.337947+0000","last_peered":"2024-09-21T03:44:58.337947+0000","last_clean":"2024-09-21T03:44:58.337947+0000","last_became_active":"2024-09-21T03:44:57.360163+0000","last_became_peered":"2024-09-21T03:44:57.360163+0000","last_unstale":"2024-09-21T03:44:58.337947+0000","last_undegraded":"2024-09-21T03:44:58.337947+0000","last_fullsized":"2024-09-21T03:44:58.337947+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:05:31.071185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":15,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.338632+0000","last_change":"2024-09-21T03:44:56.343444+0000","last_active":"2024-09-21T03:44:58.338632+0000","last_peered":"2024-09-21T03:44:58.338632+0000","last_clean":"2024-09-21T03:44:58.338632+0000","last_became_active":"2024-09-21T03:44:56.343128+0000","last_became_peered":"2024-09-21T03:44:56.343128+0000","last_unstale":"2024-09-21T03:44:58.338632+0000","last_undegraded":"2024-09-21T03:44:58.338632+0000","last_fullsized":"2024-09-21T03:44:58.338632+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_clean_scrub_stamp":"2024-09-21T03:44:55.304777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:49:27.666317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":10,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T03:44:57.351262+0000","last_change":"2024-09-21T03:44:57.348168+0000","last_active":"2024-09-21T03:44:57.351262+0000","last_peered":"2024-09-21T03:44:57.351262+0000","last_clean":"2024-09-21T03:44:57.351262+0000","last_became_active":"2024-09-21T03:44:57.347823+0000","last_became_peered":"2024-09-21T03:44:57.347823+0000","last_unstale":"2024-09-21T03:44:57.351262+0000","last_undegraded":"2024-09-21T03:44:57.351262+0000","last_fullsized":"2024-09-21T03:44:57.351262+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:43:22.624675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":15,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.456487+0000","last_change":"2024-09-21T03:44:56.337896+0000","last_active":"2024-09-21T03:44:58.456487+0000","last_peered":"2024-09-21T03:44:58.456487+0000","last_clean":"2024-09-21T03:44:58.456487+0000","last_became_active":"2024-09-21T03:44:56.337298+0000","last_became_peered":"2024-09-21T03:44:56.337298+0000","last_unstale":"2024-09-21T03:44:58.456487+0000","last_undegraded":"2024-09-21T03:44:58.456487+0000","last_fullsized":"2024-09-21T03:44:58.456487+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_clean_scrub_stamp":"2024-09-21T03:44:55.304777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:58:50.496950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":10,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T03:44:57.351652+0000","last_change":"2024-09-21T03:44:57.350094+0000","last_active":"2024-09-21T03:44:57.351652+0000","last_peered":"2024-09-21T03:44:57.351652+0000","last_clean":"2024-09-21T03:44:57.351652+0000","last_became_active":"2024-09-21T03:44:57.349777+0000","last_became_peered":"2024-09-21T03:44:57.349777+0000","last_unstale":"2024-09-21T03:44:57.351652+0000","last_undegraded":"2024-09-21T03:44:57.351652+0000","last_fullsized":"2024-09-21T03:44:57.351652+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:36:06.178611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":15,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.353239+0000","last_change":"2024-09-21T03:44:56.329230+0000","last_active":"2024-09-21T03:44:58.353239+0000","last_peered":"2024-09-21T03:44:58.353239+0000","last_clean":"2024-09-21T03:44:58.353239+0000","last_became_active":"2024-09-21T03:44:56.328911+0000","last_became_peered":"2024-09-21T03:44:56.328911+0000","last_unstale":"2024-09-21T03:44:58.353239+0000","last_undegraded":"2024-09-21T03:44:58.353239+0000","last_fullsized":"2024-09-21T03:44:58.353239+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_clean_scrub_stamp":"2024-09-21T03:44:55.304777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:16:45.547857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":10,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T03:44:57.351808+0000","last_change":"2024-09-21T03:44:57.347015+0000","last_active":"2024-09-21T03:44:57.351808+0000","last_peered":"2024-09-21T03:44:57.351808+0000","last_clean":"2024-09-21T03:44:57.351808+0000","last_became_active":"2024-09-21T03:44:57.346668+0000","last_became_peered":"2024-09-21T03:44:57.346668+0000","last_unstale":"2024-09-21T03:44:57.351808+0000","last_undegraded":"2024-09-21T03:44:57.351808+0000","last_fullsized":"2024-09-21T03:44:57.351808+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:49:55.140933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T03:44:57.341754+0000","last_change":"2024-09-21T03:44:56.333386+0000","last_active":"2024-09-21T03:44:57.341754+0000","last_peered":"2024-09-21T03:44:57.341754+0000","last_clean":"2024-09-21T03:44:57.341754+0000","last_became_active":"2024-09-21T03:44:56.333082+0000","last_became_peered":"2024-09-21T03:44:56.333082+0000","last_unstale":"2024-09-21T03:44:57.341754+0000","last_undegraded":"2024-09-21T03:44:57.341754+0000","last_fullsized":"2024-09-21T03:44:57.341754+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_clean_scrub_stamp":"2024-09-21T03:44:55.304777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:10:24.715196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":10,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T03:44:57.355956+0000","last_change":"2024-09-21T03:44:57.350041+0000","last_active":"2024-09-21T03:44:57.355956+0000","last_peered":"2024-09-21T03:44:57.355956+0000","last_clean":"2024-09-21T03:44:57.355956+0000","last_became_active":"2024-09-21T03:44:57.349706+0000","last_became_peered":"2024-09-21T03:44:57.349706+0000","last_unstale":"2024-09-21T03:44:57.355956+0000","last_undegraded":"2024-09-21T03:44:57.355956+0000","last_fullsized":"2024-09-21T03:44:57.355956+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:35:23.899135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.15","version":"145'3","reported_seq":18,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.352972+0000","last_change":"2024-09-21T03:44:56.324998+0000","last_active":"2024-09-21T03:44:58.352972+0000","last_peered":"2024-09-21T03:44:58.352972+0000","last_clean":"2024-09-21T03:44:58.352972+0000","last_became_active":"2024-09-21T03:44:56.324464+0000","last_became_peered":"2024-09-21T03:44:56.324464+0000","last_unstale":"2024-09-21T03:44:58.352972+0000","last_undegraded":"2024-09-21T03:44:58.352972+0000","last_fullsized":"2024-09-21T03:44:58.352972+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_clean_scrub_stamp":"2024-09-21T03:44:55.304777+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:27:45.431858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.456464+0000","last_change":"2024-09-21T03:44:57.355113+0000","last_active":"2024-09-21T03:44:58.456464+0000","last_peered":"2024-09-21T03:44:58.456464+0000","last_clean":"2024-09-21T03:44:58.456464+0000","last_became_active":"2024-09-21T03:44:57.354744+0000","last_became_peered":"2024-09-21T03:44:57.354744+0000","last_unstale":"2024-09-21T03:44:58.456464+0000","last_undegraded":"2024-09-21T03:44:58.456464+0000","last_fullsized":"2024-09-21T03:44:58.456464+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:54:24.418739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.14","version":"145'6","reported_seq":21,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.337583+0000","last_change":"2024-09-21T03:44:56.341013+0000","last_active":"2024-09-21T03:44:58.337583+0000","last_peered":"2024-09-21T03:44:58.337583+0000","last_clean":"2024-09-21T03:44:58.337583+0000","last_became_active":"2024-09-21T03:44:56.337878+0000","last_became_peered":"2024-09-21T03:44:56.337878+0000","last_unstale":"2024-09-21T03:44:58.337583+0000","last_undegraded":"2024-09-21T03:44:58.337583+0000","last_fullsized":"2024-09-21T03:44:58.337583+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_clean_scrub_stamp":"2024-09-21T03:44:55.304777+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:00:34.311511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.915365+0000","last_change":"2024-09-21T03:44:57.356176+0000","last_active":"2024-09-21T03:44:58.915365+0000","last_peered":"2024-09-21T03:44:58.915365+0000","last_clean":"2024-09-21T03:44:58.915365+0000","last_became_active":"2024-09-21T03:44:57.355832+0000","last_became_peered":"2024-09-21T03:44:57.355832+0000","last_unstale":"2024-09-21T03:44:58.915365+0000","last_undegraded":"2024-09-21T03:44:58.915365+0000","last_fullsized":"2024-09-21T03:44:58.915365+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:47:30.592382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.13","version":"145'2","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T03:44:57.367038+0000","last_change":"2024-09-21T03:44:56.327633+0000","last_active":"2024-09-21T03:44:57.367038+0000","last_peered":"2024-09-21T03:44:57.367038+0000","last_clean":"2024-09-21T03:44:57.367038+0000","last_became_active":"2024-09-21T03:44:56.327313+0000","last_became_peered":"2024-09-21T03:44:56.327313+0000","last_unstale":"2024-09-21T03:44:57.367038+0000","last_undegraded":"2024-09-21T03:44:57.367038+0000","last_fullsized":"2024-09-21T03:44:57.367038+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_clean_scrub_stamp":"2024-09-21T03:44:55.304777+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:36:44.258406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.915449+0000","last_change":"2024-09-21T03:44:57.352611+0000","last_active":"2024-09-21T03:44:58.915449+0000","last_peered":"2024-09-21T03:44:58.915449+0000","last_clean":"2024-09-21T03:44:58.915449+0000","last_became_active":"2024-09-21T03:44:57.352136+0000","last_became_peered":"2024-09-21T03:44:57.352136+0000","last_unstale":"2024-09-21T03:44:58.915449+0000","last_undegraded":"2024-09-21T03:44:58.915449+0000","last_fullsized":"2024-09-21T03:44:58.915449+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:26:06.882445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T03:44:57.341635+0000","last_change":"2024-09-21T03:44:56.337958+0000","last_active":"2024-09-21T03:44:57.341635+0000","last_peered":"2024-09-21T03:44:57.341635+0000","last_clean":"2024-09-21T03:44:57.341635+0000","last_became_active":"2024-09-21T03:44:56.337623+0000","last_became_peered":"2024-09-21T03:44:56.337623+0000","last_unstale":"2024-09-21T03:44:57.341635+0000","last_undegraded":"2024-09-21T03:44:57.341635+0000","last_fullsized":"2024-09-21T03:44:57.341635+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_clean_scrub_stamp":"2024-09-21T03:44:55.304777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:45:31.964595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.456704+0000","last_change":"2024-09-21T03:44:57.358895+0000","last_active":"2024-09-21T03:44:58.456704+0000","last_peered":"2024-09-21T03:44:58.456704+0000","last_clean":"2024-09-21T03:44:58.456704+0000","last_became_active":"2024-09-21T03:44:57.358591+0000","last_became_peered":"2024-09-21T03:44:57.358591+0000","last_unstale":"2024-09-21T03:44:58.456704+0000","last_undegraded":"2024-09-21T03:44:58.456704+0000","last_fullsized":"2024-09-21T03:44:58.456704+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:38:18.158452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T03:44:57.337893+0000","last_change":"2024-09-21T03:44:56.327693+0000","last_active":"2024-09-21T03:44:57.337893+0000","last_peered":"2024-09-21T03:44:57.337893+0000","last_clean":"2024-09-21T03:44:57.337893+0000","last_became_active":"2024-09-21T03:44:56.327364+0000","last_became_peered":"2024-09-21T03:44:56.327364+0000","last_unstale":"2024-09-21T03:44:57.337893+0000","last_undegraded":"2024-09-21T03:44:57.337893+0000","last_fullsized":"2024-09-21T03:44:57.337893+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_clean_scrub_stamp":"2024-09-21T03:44:55.304777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:16:04.551826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":1,"reported_epoch":144,"state":"creating+peering","last_fresh":"2024-09-21T03:44:56.321369+0000","last_change":"2024-09-21T03:44:56.321369+0000","last_active":"2024-09-21T03:44:56.311610+0000","last_peered":"2024-09-21T03:44:56.311610+0000","last_clean":"2024-09-21T03:44:56.311610+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:44:56.321369+0000","last_undegraded":"2024-09-21T03:44:56.321369+0000","last_fullsized":"2024-09-21T03:44:56.321369+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.10","version":"145'8","reported_seq":23,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.337672+0000","last_change":"2024-09-21T03:44:56.342213+0000","last_active":"2024-09-21T03:44:58.337672+0000","last_peered":"2024-09-21T03:44:58.337672+0000","last_clean":"2024-09-21T03:44:58.337672+0000","last_became_active":"2024-09-21T03:44:56.341888+0000","last_became_peered":"2024-09-21T03:44:56.341888+0000","last_unstale":"2024-09-21T03:44:58.337672+0000","last_undegraded":"2024-09-21T03:44:58.337672+0000","last_fullsized":"2024-09-21T03:44:58.337672+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_clean_scrub_stamp":"2024-09-21T03:44:55.304777+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:36:05.171778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":10,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T03:44:57.355999+0000","last_change":"2024-09-21T03:44:57.350536+0000","last_active":"2024-09-21T03:44:57.355999+0000","last_peered":"2024-09-21T03:44:57.355999+0000","last_clean":"2024-09-21T03:44:57.355999+0000","last_became_active":"2024-09-21T03:44:57.349795+0000","last_became_peered":"2024-09-21T03:44:57.349795+0000","last_unstale":"2024-09-21T03:44:57.355999+0000","last_undegraded":"2024-09-21T03:44:57.355999+0000","last_fullsized":"2024-09-21T03:44:57.355999+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:09:26.811858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.f","version":"145'1","reported_seq":16,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.353274+0000","last_change":"2024-09-21T03:44:56.339856+0000","last_active":"2024-09-21T03:44:58.353274+0000","last_peered":"2024-09-21T03:44:58.353274+0000","last_clean":"2024-09-21T03:44:58.353274+0000","last_became_active":"2024-09-21T03:44:56.339305+0000","last_became_peered":"2024-09-21T03:44:56.339305+0000","last_unstale":"2024-09-21T03:44:58.353274+0000","last_undegraded":"2024-09-21T03:44:58.353274+0000","last_fullsized":"2024-09-21T03:44:58.353274+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_clean_scrub_stamp":"2024-09-21T03:44:55.304777+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:37:25.372042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.338656+0000","last_change":"2024-09-21T03:44:57.341004+0000","last_active":"2024-09-21T03:44:58.338656+0000","last_peered":"2024-09-21T03:44:58.338656+0000","last_clean":"2024-09-21T03:44:58.338656+0000","last_became_active":"2024-09-21T03:44:57.340620+0000","last_became_peered":"2024-09-21T03:44:57.340620+0000","last_unstale":"2024-09-21T03:44:58.338656+0000","last_undegraded":"2024-09-21T03:44:58.338656+0000","last_fullsized":"2024-09-21T03:44:58.338656+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:09:01.018690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.d","version":"145'1","reported_seq":16,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.339122+0000","last_change":"2024-09-21T03:44:56.340547+0000","last_active":"2024-09-21T03:44:58.339122+0000","last_peered":"2024-09-21T03:44:58.339122+0000","last_clean":"2024-09-21T03:44:58.339122+0000","last_became_active":"2024-09-21T03:44:56.340237+0000","last_became_peered":"2024-09-21T03:44:56.340237+0000","last_unstale":"2024-09-21T03:44:58.339122+0000","last_undegraded":"2024-09-21T03:44:58.339122+0000","last_fullsized":"2024-09-21T03:44:58.339122+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_clean_scrub_stamp":"2024-09-21T03:44:55.304777+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:19:41.082645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.456321+0000","last_change":"2024-09-21T03:44:57.358052+0000","last_active":"2024-09-21T03:44:58.456321+0000","last_peered":"2024-09-21T03:44:58.456321+0000","last_clean":"2024-09-21T03:44:58.456321+0000","last_became_active":"2024-09-21T03:44:57.357731+0000","last_became_peered":"2024-09-21T03:44:57.357731+0000","last_unstale":"2024-09-21T03:44:58.456321+0000","last_undegraded":"2024-09-21T03:44:58.456321+0000","last_fullsized":"2024-09-21T03:44:58.456321+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:02:38.958342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":15,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.353016+0000","last_change":"2024-09-21T03:44:56.339649+0000","last_active":"2024-09-21T03:44:58.353016+0000","last_peered":"2024-09-21T03:44:58.353016+0000","last_clean":"2024-09-21T03:44:58.353016+0000","last_became_active":"2024-09-21T03:44:56.339317+0000","last_became_peered":"2024-09-21T03:44:56.339317+0000","last_unstale":"2024-09-21T03:44:58.353016+0000","last_undegraded":"2024-09-21T03:44:58.353016+0000","last_fullsized":"2024-09-21T03:44:58.353016+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_clean_scrub_stamp":"2024-09-21T03:44:55.304777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:58:28.485819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":10,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T03:44:57.353966+0000","last_change":"2024-09-21T03:44:57.350478+0000","last_active":"2024-09-21T03:44:57.353966+0000","last_peered":"2024-09-21T03:44:57.353966+0000","last_clean":"2024-09-21T03:44:57.353966+0000","last_became_active":"2024-09-21T03:44:57.348990+0000","last_became_peered":"2024-09-21T03:44:57.348990+0000","last_unstale":"2024-09-21T03:44:57.353966+0000","last_undegraded":"2024-09-21T03:44:57.353966+0000","last_fullsized":"2024-09-21T03:44:57.353966+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:40:27.993685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T03:44:56.330403+0000","last_change":"2024-09-21T03:44:56.329417+0000","last_active":"2024-09-21T03:44:56.330403+0000","last_peered":"2024-09-21T03:44:56.330403+0000","last_clean":"2024-09-21T03:44:56.330403+0000","last_became_active":"2024-09-21T03:44:56.329098+0000","last_became_peered":"2024-09-21T03:44:56.329098+0000","last_unstale":"2024-09-21T03:44:56.330403+0000","last_undegraded":"2024-09-21T03:44:56.330403+0000","last_fullsized":"2024-09-21T03:44:56.330403+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_clean_scrub_stamp":"2024-09-21T03:44:55.304777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:53:45.362390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.456620+0000","last_change":"2024-09-21T03:44:57.354679+0000","last_active":"2024-09-21T03:44:58.456620+0000","last_peered":"2024-09-21T03:44:58.456620+0000","last_clean":"2024-09-21T03:44:58.456620+0000","last_became_active":"2024-09-21T03:44:57.354357+0000","last_became_peered":"2024-09-21T03:44:57.354357+0000","last_unstale":"2024-09-21T03:44:58.456620+0000","last_undegraded":"2024-09-21T03:44:58.456620+0000","last_fullsized":"2024-09-21T03:44:58.456620+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:29:19.509171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T03:44:56.331506+0000","last_change":"2024-09-21T03:44:56.330054+0000","last_active":"2024-09-21T03:44:56.331506+0000","last_peered":"2024-09-21T03:44:56.331506+0000","last_clean":"2024-09-21T03:44:56.331506+0000","last_became_active":"2024-09-21T03:44:56.329151+0000","last_became_peered":"2024-09-21T03:44:56.329151+0000","last_unstale":"2024-09-21T03:44:56.331506+0000","last_undegraded":"2024-09-21T03:44:56.331506+0000","last_fullsized":"2024-09-21T03:44:56.331506+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_clean_scrub_stamp":"2024-09-21T03:44:55.304777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:15:21.096050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.456652+0000","last_change":"2024-09-21T03:44:57.363634+0000","last_active":"2024-09-21T03:44:58.456652+0000","last_peered":"2024-09-21T03:44:58.456652+0000","last_clean":"2024-09-21T03:44:58.456652+0000","last_became_active":"2024-09-21T03:44:57.363259+0000","last_became_peered":"2024-09-21T03:44:57.363259+0000","last_unstale":"2024-09-21T03:44:58.456652+0000","last_undegraded":"2024-09-21T03:44:58.456652+0000","last_fullsized":"2024-09-21T03:44:58.456652+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:53:39.529389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":15,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.353119+0000","last_change":"2024-09-21T03:44:56.327686+0000","last_active":"2024-09-21T03:44:58.353119+0000","last_peered":"2024-09-21T03:44:58.353119+0000","last_clean":"2024-09-21T03:44:58.353119+0000","last_became_active":"2024-09-21T03:44:56.327355+0000","last_became_peered":"2024-09-21T03:44:56.327355+0000","last_unstale":"2024-09-21T03:44:58.353119+0000","last_undegraded":"2024-09-21T03:44:58.353119+0000","last_fullsized":"2024-09-21T03:44:58.353119+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_clean_scrub_stamp":"2024-09-21T03:44:55.304777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:26:58.029665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":10,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T03:44:57.352767+0000","last_change":"2024-09-21T03:44:57.351050+0000","last_active":"2024-09-21T03:44:57.352767+0000","last_peered":"2024-09-21T03:44:57.352767+0000","last_clean":"2024-09-21T03:44:57.352767+0000","last_became_active":"2024-09-21T03:44:57.350731+0000","last_became_peered":"2024-09-21T03:44:57.350731+0000","last_unstale":"2024-09-21T03:44:57.352767+0000","last_undegraded":"2024-09-21T03:44:57.352767+0000","last_fullsized":"2024-09-21T03:44:57.352767+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:31:33.985328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T03:44:57.340435+0000","last_change":"2024-09-21T03:44:56.334506+0000","last_active":"2024-09-21T03:44:57.340435+0000","last_peered":"2024-09-21T03:44:57.340435+0000","last_clean":"2024-09-21T03:44:57.340435+0000","last_became_active":"2024-09-21T03:44:56.334007+0000","last_became_peered":"2024-09-21T03:44:56.334007+0000","last_unstale":"2024-09-21T03:44:57.340435+0000","last_undegraded":"2024-09-21T03:44:57.340435+0000","last_fullsized":"2024-09-21T03:44:57.340435+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_clean_scrub_stamp":"2024-09-21T03:44:55.304777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:57:15.575727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.456155+0000","last_change":"2024-09-21T03:44:57.350289+0000","last_active":"2024-09-21T03:44:58.456155+0000","last_peered":"2024-09-21T03:44:58.456155+0000","last_clean":"2024-09-21T03:44:58.456155+0000","last_became_active":"2024-09-21T03:44:57.349932+0000","last_became_peered":"2024-09-21T03:44:57.349932+0000","last_unstale":"2024-09-21T03:44:58.456155+0000","last_undegraded":"2024-09-21T03:44:58.456155+0000","last_fullsized":"2024-09-21T03:44:58.456155+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:49:41.592146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T03:44:56.329848+0000","last_change":"2024-09-21T03:44:56.328517+0000","last_active":"2024-09-21T03:44:56.329848+0000","last_peered":"2024-09-21T03:44:56.329848+0000","last_clean":"2024-09-21T03:44:56.329848+0000","last_became_active":"2024-09-21T03:44:56.328048+0000","last_became_peered":"2024-09-21T03:44:56.328048+0000","last_unstale":"2024-09-21T03:44:56.329848+0000","last_undegraded":"2024-09-21T03:44:56.329848+0000","last_fullsized":"2024-09-21T03:44:56.329848+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_clean_scrub_stamp":"2024-09-21T03:44:55.304777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:24:58.009008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.337538+0000","last_change":"2024-09-21T03:44:57.359324+0000","last_active":"2024-09-21T03:44:58.337538+0000","last_peered":"2024-09-21T03:44:58.337538+0000","last_clean":"2024-09-21T03:44:58.337538+0000","last_became_active":"2024-09-21T03:44:57.358966+0000","last_became_peered":"2024-09-21T03:44:57.358966+0000","last_unstale":"2024-09-21T03:44:58.337538+0000","last_undegraded":"2024-09-21T03:44:58.337538+0000","last_fullsized":"2024-09-21T03:44:58.337538+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:30:55.516774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T03:44:56.338457+0000","last_change":"2024-09-21T03:44:56.336489+0000","last_active":"2024-09-21T03:44:56.338457+0000","last_peered":"2024-09-21T03:44:56.338457+0000","last_clean":"2024-09-21T03:44:56.338457+0000","last_became_active":"2024-09-21T03:44:56.336089+0000","last_became_peered":"2024-09-21T03:44:56.336089+0000","last_unstale":"2024-09-21T03:44:56.338457+0000","last_undegraded":"2024-09-21T03:44:56.338457+0000","last_fullsized":"2024-09-21T03:44:56.338457+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_clean_scrub_stamp":"2024-09-21T03:44:55.304777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:30:01.117090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":10,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T03:44:57.356840+0000","last_change":"2024-09-21T03:44:57.355765+0000","last_active":"2024-09-21T03:44:57.356840+0000","last_peered":"2024-09-21T03:44:57.356840+0000","last_clean":"2024-09-21T03:44:57.356840+0000","last_became_active":"2024-09-21T03:44:57.355383+0000","last_became_peered":"2024-09-21T03:44:57.355383+0000","last_unstale":"2024-09-21T03:44:57.356840+0000","last_undegraded":"2024-09-21T03:44:57.356840+0000","last_fullsized":"2024-09-21T03:44:57.356840+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:44:27.836492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":15,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.337802+0000","last_change":"2024-09-21T03:44:56.338420+0000","last_active":"2024-09-21T03:44:58.337802+0000","last_peered":"2024-09-21T03:44:58.337802+0000","last_clean":"2024-09-21T03:44:58.337802+0000","last_became_active":"2024-09-21T03:44:56.337865+0000","last_became_peered":"2024-09-21T03:44:56.337865+0000","last_unstale":"2024-09-21T03:44:58.337802+0000","last_undegraded":"2024-09-21T03:44:58.337802+0000","last_fullsized":"2024-09-21T03:44:58.337802+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_clean_scrub_stamp":"2024-09-21T03:44:55.304777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:40:26.986852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.339010+0000","last_change":"2024-09-21T03:44:57.337661+0000","last_active":"2024-09-21T03:44:58.339010+0000","last_peered":"2024-09-21T03:44:58.339010+0000","last_clean":"2024-09-21T03:44:58.339010+0000","last_became_active":"2024-09-21T03:44:57.337356+0000","last_became_peered":"2024-09-21T03:44:57.337356+0000","last_unstale":"2024-09-21T03:44:58.339010+0000","last_undegraded":"2024-09-21T03:44:58.339010+0000","last_fullsized":"2024-09-21T03:44:58.339010+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:19:49.944906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T03:44:57.338702+0000","last_change":"2024-09-21T03:44:56.337786+0000","last_active":"2024-09-21T03:44:57.338702+0000","last_peered":"2024-09-21T03:44:57.338702+0000","last_clean":"2024-09-21T03:44:57.338702+0000","last_became_active":"2024-09-21T03:44:56.337258+0000","last_became_peered":"2024-09-21T03:44:56.337258+0000","last_unstale":"2024-09-21T03:44:57.338702+0000","last_undegraded":"2024-09-21T03:44:57.338702+0000","last_fullsized":"2024-09-21T03:44:57.338702+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_clean_scrub_stamp":"2024-09-21T03:44:55.304777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:29:06.273027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":10,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T03:44:57.351689+0000","last_change":"2024-09-21T03:44:57.350221+0000","last_active":"2024-09-21T03:44:57.351689+0000","last_peered":"2024-09-21T03:44:57.351689+0000","last_clean":"2024-09-21T03:44:57.351689+0000","last_became_active":"2024-09-21T03:44:57.349707+0000","last_became_peered":"2024-09-21T03:44:57.349707+0000","last_unstale":"2024-09-21T03:44:57.351689+0000","last_undegraded":"2024-09-21T03:44:57.351689+0000","last_fullsized":"2024-09-21T03:44:57.351689+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:50:57.792589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":15,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.337614+0000","last_change":"2024-09-21T03:44:56.340358+0000","last_active":"2024-09-21T03:44:58.337614+0000","last_peered":"2024-09-21T03:44:58.337614+0000","last_clean":"2024-09-21T03:44:58.337614+0000","last_became_active":"2024-09-21T03:44:56.340044+0000","last_became_peered":"2024-09-21T03:44:56.340044+0000","last_unstale":"2024-09-21T03:44:58.337614+0000","last_undegraded":"2024-09-21T03:44:58.337614+0000","last_fullsized":"2024-09-21T03:44:58.337614+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_clean_scrub_stamp":"2024-09-21T03:44:55.304777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:50:56.785756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":10,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T03:44:57.350495+0000","last_change":"2024-09-21T03:44:57.348196+0000","last_active":"2024-09-21T03:44:57.350495+0000","last_peered":"2024-09-21T03:44:57.350495+0000","last_clean":"2024-09-21T03:44:57.350495+0000","last_became_active":"2024-09-21T03:44:57.347856+0000","last_became_peered":"2024-09-21T03:44:57.347856+0000","last_unstale":"2024-09-21T03:44:57.350495+0000","last_undegraded":"2024-09-21T03:44:57.350495+0000","last_fullsized":"2024-09-21T03:44:57.350495+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:32:59.029041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T03:44:56.338491+0000","last_change":"2024-09-21T03:44:56.337049+0000","last_active":"2024-09-21T03:44:56.338491+0000","last_peered":"2024-09-21T03:44:56.338491+0000","last_clean":"2024-09-21T03:44:56.338491+0000","last_became_active":"2024-09-21T03:44:56.336315+0000","last_became_peered":"2024-09-21T03:44:56.336315+0000","last_unstale":"2024-09-21T03:44:56.338491+0000","last_undegraded":"2024-09-21T03:44:56.338491+0000","last_fullsized":"2024-09-21T03:44:56.338491+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_clean_scrub_stamp":"2024-09-21T03:44:55.304777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:31:45.105527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.353229+0000","last_change":"2024-09-21T03:44:57.366623+0000","last_active":"2024-09-21T03:44:58.353229+0000","last_peered":"2024-09-21T03:44:58.353229+0000","last_clean":"2024-09-21T03:44:58.353229+0000","last_became_active":"2024-09-21T03:44:57.366298+0000","last_became_peered":"2024-09-21T03:44:57.366298+0000","last_unstale":"2024-09-21T03:44:58.353229+0000","last_undegraded":"2024-09-21T03:44:58.353229+0000","last_fullsized":"2024-09-21T03:44:58.353229+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:01:23.359578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":15,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.338658+0000","last_change":"2024-09-21T03:44:56.339692+0000","last_active":"2024-09-21T03:44:58.338658+0000","last_peered":"2024-09-21T03:44:58.338658+0000","last_clean":"2024-09-21T03:44:58.338658+0000","last_became_active":"2024-09-21T03:44:56.339376+0000","last_became_peered":"2024-09-21T03:44:56.339376+0000","last_unstale":"2024-09-21T03:44:58.338658+0000","last_undegraded":"2024-09-21T03:44:58.338658+0000","last_fullsized":"2024-09-21T03:44:58.338658+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_clean_scrub_stamp":"2024-09-21T03:44:55.304777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:17:10.658062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.915523+0000","last_change":"2024-09-21T03:44:57.356308+0000","last_active":"2024-09-21T03:44:58.915523+0000","last_peered":"2024-09-21T03:44:58.915523+0000","last_clean":"2024-09-21T03:44:58.915523+0000","last_became_active":"2024-09-21T03:44:57.355511+0000","last_became_peered":"2024-09-21T03:44:57.355511+0000","last_unstale":"2024-09-21T03:44:58.915523+0000","last_undegraded":"2024-09-21T03:44:58.915523+0000","last_fullsized":"2024-09-21T03:44:58.915523+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:08:21.639028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T03:44:56.342190+0000","last_change":"2024-09-21T03:44:56.341499+0000","last_active":"2024-09-21T03:44:56.342190+0000","last_peered":"2024-09-21T03:44:56.342190+0000","last_clean":"2024-09-21T03:44:56.342190+0000","last_became_active":"2024-09-21T03:44:56.341058+0000","last_became_peered":"2024-09-21T03:44:56.341058+0000","last_unstale":"2024-09-21T03:44:56.342190+0000","last_undegraded":"2024-09-21T03:44:56.342190+0000","last_fullsized":"2024-09-21T03:44:56.342190+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_clean_scrub_stamp":"2024-09-21T03:44:55.304777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:51:47.787796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.352897+0000","last_change":"2024-09-21T03:44:57.355188+0000","last_active":"2024-09-21T03:44:58.352897+0000","last_peered":"2024-09-21T03:44:58.352897+0000","last_clean":"2024-09-21T03:44:58.352897+0000","last_became_active":"2024-09-21T03:44:57.354477+0000","last_became_peered":"2024-09-21T03:44:57.354477+0000","last_unstale":"2024-09-21T03:44:58.352897+0000","last_undegraded":"2024-09-21T03:44:58.352897+0000","last_fullsized":"2024-09-21T03:44:58.352897+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:28:53.047166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"145'2","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T03:44:57.366695+0000","last_change":"2024-09-21T03:44:56.335259+0000","last_active":"2024-09-21T03:44:57.366695+0000","last_peered":"2024-09-21T03:44:57.366695+0000","last_clean":"2024-09-21T03:44:57.366695+0000","last_became_active":"2024-09-21T03:44:56.334868+0000","last_became_peered":"2024-09-21T03:44:56.334868+0000","last_unstale":"2024-09-21T03:44:57.366695+0000","last_undegraded":"2024-09-21T03:44:57.366695+0000","last_fullsized":"2024-09-21T03:44:57.366695+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_clean_scrub_stamp":"2024-09-21T03:44:55.304777+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:45:29.403030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":355,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.338537+0000","last_change":"2024-09-21T03:33:25.319776+0000","last_active":"2024-09-21T03:44:58.338537+0000","last_peered":"2024-09-21T03:44:58.338537+0000","last_clean":"2024-09-21T03:44:58.338537+0000","last_became_active":"2024-09-21T03:33:25.318399+0000","last_became_peered":"2024-09-21T03:33:25.318399+0000","last_unstale":"2024-09-21T03:44:58.338537+0000","last_undegraded":"2024-09-21T03:44:58.338537+0000","last_fullsized":"2024-09-21T03:44:58.338537+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:23.016672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:23.016672+0000","last_clean_scrub_stamp":"2024-09-21T03:33:23.016672+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:32:35.906114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":4696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":33,"ondisk_log_size":33,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":1,"up_from":16,"seq":68719476879,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41936,"kb_used_data":2428,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94329904,"statfs":{"total":96636764160,"available":96593821696,"internally_reserved":0,"allocated":2486272,"data_stored":1546129,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509583,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41356,"kb_used_data":1848,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330484,"statfs":{"total":96636764160,"available":96594415616,"internally_reserved":0,"allocated":1892352,"data_stored":956321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":40434123},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509582,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41364,"kb_used_data":1852,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330476,"statfs":{"total":96636764160,"available":96594407424,"internally_reserved":0,"allocated":1896448,"data_stored":957281,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509583,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37184,"kb_used_data":1836,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334656,"statfs":{"total":96636764160,"available":96598687744,"internally_reserved":0,"allocated":1880064,"data_stored":949089,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509582,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37180,"kb_used_data":1832,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334660,"statfs":{"total":96636764160,"available":96598691840,"internally_reserved":0,"allocated":1875968,"data_stored":949559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542286,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33028,"kb_used_data":1840,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338812,"statfs":{"total":96636764160,"available":96602943488,"internally_reserved":0,"allocated":1884160,"data_stored":950589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574990,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37744,"kb_used_data":2396,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334096,"statfs":{"total":96636764160,"available":96598114304,"internally_reserved":0,"allocated":2453504,"data_stored":1532178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542287,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37168,"kb_used_data":1816,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334672,"statfs":{"total":96636764160,"available":96598704128,"internally_reserved":0,"allocated":1859584,"data_stored":941810,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T03:45:01.806 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T03:45:01.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:01.958+0000 7f4f36b2b640 1 -- 172.21.15.38:0/3759533122 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f1c004d40 msgr2=0x7f4f1c005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:01.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:01.958+0000 7f4f36b2b640 1 --2- 172.21.15.38:0/3759533122 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f1c004d40 0x7f4f1c005150 secure :-1 s=READY pgs=2366 cs=0 l=1 rev1=1 crypto rx=0x7f4f301611c0 tx=0x7f4f1c005830 comp rx=0 tx=0).stop 2024-09-21T03:45:01.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:01.958+0000 7f4f350c8640 1 -- 172.21.15.38:0/3759533122 reap_dead start 2024-09-21T03:45:01.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:01.958+0000 7f4f36b2b640 1 -- 172.21.15.38:0/3759533122 shutdown_connections 2024-09-21T03:45:01.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:01.958+0000 7f4f36b2b640 1 -- 172.21.15.38:0/3759533122 >> 172.21.15.38:0/3759533122 conn(0x7f4f300d2350 msgr2=0x7f4f300d2760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:01.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:01.958+0000 7f4f36b2b640 1 -- 172.21.15.38:0/3759533122 shutdown_connections 2024-09-21T03:45:01.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:01.958+0000 7f4f36b2b640 1 -- 172.21.15.38:0/3759533122 wait complete. 2024-09-21T03:45:01.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:01.960+0000 7f4f36b2b640 1 Processor -- start 2024-09-21T03:45:01.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:01.960+0000 7f4f36b2b640 1 -- start start 2024-09-21T03:45:01.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:01.961+0000 7f4f36b2b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4f1c004d40 0x7f4f30156940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:01.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:01.961+0000 7f4f36b2b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f301f64a0 0x7f4f3014cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:01.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:01.961+0000 7f4f36b2b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4f3014f800 0x7f4f3014fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:01.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:01.961+0000 7f4f36b2b640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f4f300b67e0 con 0x7f4f301f64a0 2024-09-21T03:45:01.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:01.961+0000 7f4f36b2b640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f4f300b6660 con 0x7f4f3014f800 2024-09-21T03:45:01.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:01.961+0000 7f4f36b2b640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f4f300b6960 con 0x7f4f1c004d40 2024-09-21T03:45:01.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:01.961+0000 7f4f2ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f301f64a0 0x7f4f3014cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:01.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:01.961+0000 7f4f2ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f301f64a0 0x7f4f3014cfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40588/0 (socket says 172.21.15.38:40588) 2024-09-21T03:45:01.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:01.961+0000 7f4f2ffff640 1 -- 172.21.15.38:0/1374498245 learned_addr learned my addr 172.21.15.38:0/1374498245 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:45:01.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:01.961+0000 7f4f348c7640 1 --2- 172.21.15.38:0/1374498245 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4f1c004d40 0x7f4f30156940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:01.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:01.962+0000 7f4f2ffff640 1 -- 172.21.15.38:0/1374498245 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4f1c004d40 msgr2=0x7f4f30156940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:01.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:01.962+0000 7f4f2ffff640 1 --2- 172.21.15.38:0/1374498245 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4f1c004d40 0x7f4f30156940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:01.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:01.962+0000 7f4f350c8640 1 --2- 172.21.15.38:0/1374498245 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4f3014f800 0x7f4f3014fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:01.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:01.962+0000 7f4f2ffff640 1 -- 172.21.15.38:0/1374498245 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4f3014f800 msgr2=0x7f4f3014fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:01.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:01.962+0000 7f4f2ffff640 1 --2- 172.21.15.38:0/1374498245 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4f3014f800 0x7f4f3014fc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:01.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:01.962+0000 7f4f2ffff640 1 -- 172.21.15.38:0/1374498245 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f301f2eb0 con 0x7f4f301f64a0 2024-09-21T03:45:01.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:01.962+0000 7f4f2ffff640 1 --2- 172.21.15.38:0/1374498245 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f301f64a0 0x7f4f3014cfd0 secure :-1 s=READY pgs=2367 cs=0 l=1 rev1=1 crypto rx=0x7f4f18006680 tx=0x7f4f180066b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:01.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:01.962+0000 7f4f2d7fa640 1 -- 172.21.15.38:0/1374498245 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f18015070 con 0x7f4f301f64a0 2024-09-21T03:45:01.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:01.962+0000 7f4f2d7fa640 1 -- 172.21.15.38:0/1374498245 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4f18011920 con 0x7f4f301f64a0 2024-09-21T03:45:01.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:01.962+0000 7f4f36b2b640 1 -- 172.21.15.38:0/1374498245 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f301f6bf0 con 0x7f4f301f64a0 2024-09-21T03:45:01.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:01.962+0000 7f4f2d7fa640 1 -- 172.21.15.38:0/1374498245 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f18011d70 con 0x7f4f301f64a0 2024-09-21T03:45:01.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:01.962+0000 7f4f36b2b640 1 -- 172.21.15.38:0/1374498245 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f301f71b0 con 0x7f4f301f64a0 2024-09-21T03:45:01.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:01.963+0000 7f4f36b2b640 1 -- 172.21.15.38:0/1374498245 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f3015cf20 con 0x7f4f301f64a0 2024-09-21T03:45:01.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:01.964+0000 7f4f2d7fa640 1 -- 172.21.15.38:0/1374498245 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f4f18018070 con 0x7f4f301f64a0 2024-09-21T03:45:01.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:01.965+0000 7f4f2d7fa640 1 --2- 172.21.15.38:0/1374498245 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4f10078b90 0x7f4f1007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:01.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:01.965+0000 7f4f2d7fa640 1 -- 172.21.15.38:0/1374498245 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 6101+0+0 (secure 0 0 0) 0x7f4f1809bdc0 con 0x7f4f301f64a0 2024-09-21T03:45:01.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:01.965+0000 7f4f348c7640 1 --2- 172.21.15.38:0/1374498245 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4f10078b90 0x7f4f1007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:01.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:01.965+0000 7f4f348c7640 1 --2- 172.21.15.38:0/1374498245 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4f10078b90 0x7f4f1007b0a0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f4f28009c10 tx=0x7f4f28005f90 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:01.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:01.969+0000 7f4f2d7fa640 1 -- 172.21.15.38:0/1374498245 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f4f301f2eb0 con 0x7f4f301f64a0 2024-09-21T03:45:02.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:02.153+0000 7f4f36b2b640 1 -- 172.21.15.38:0/1374498245 --> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4f3014d510 con 0x7f4f10078b90 2024-09-21T03:45:02.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:02.158+0000 7f4f2d7fa640 1 -- 172.21.15.38:0/1374498245 <== mgr.4117 v2:172.21.15.195:6800/1631901344 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252055 (secure 0 0 0) 0x7f4f2805e8b0 con 0x7f4f10078b90 2024-09-21T03:45:02.163 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:45:02.164 INFO:teuthology.orchestra.run.smithi038.stderr:dumped all 2024-09-21T03:45:02.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:02.161+0000 7f4f36b2b640 1 -- 172.21.15.38:0/1374498245 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4f10078b90 msgr2=0x7f4f1007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:02.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:02.161+0000 7f4f36b2b640 1 --2- 172.21.15.38:0/1374498245 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4f10078b90 0x7f4f1007b0a0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f4f28009c10 tx=0x7f4f28005f90 comp rx=0 tx=0).stop 2024-09-21T03:45:02.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:02.161+0000 7f4f36b2b640 1 -- 172.21.15.38:0/1374498245 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f301f64a0 msgr2=0x7f4f3014cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:02.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:02.161+0000 7f4f36b2b640 1 --2- 172.21.15.38:0/1374498245 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f301f64a0 0x7f4f3014cfd0 secure :-1 s=READY pgs=2367 cs=0 l=1 rev1=1 crypto rx=0x7f4f18006680 tx=0x7f4f180066b0 comp rx=0 tx=0).stop 2024-09-21T03:45:02.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:02.161+0000 7f4f36b2b640 1 -- 172.21.15.38:0/1374498245 shutdown_connections 2024-09-21T03:45:02.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:02.161+0000 7f4f36b2b640 1 --2- 172.21.15.38:0/1374498245 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4f3014f800 0x7f4f3014fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:02.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:02.161+0000 7f4f36b2b640 1 --2- 172.21.15.38:0/1374498245 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f301f64a0 0x7f4f3014cfd0 unknown :-1 s=CLOSED pgs=2367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:02.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:02.161+0000 7f4f36b2b640 1 --2- 172.21.15.38:0/1374498245 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4f10078b90 0x7f4f1007b0a0 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:02.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:02.161+0000 7f4f36b2b640 1 --2- 172.21.15.38:0/1374498245 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4f1c004d40 0x7f4f30156940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:02.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:02.161+0000 7f4f36b2b640 1 -- 172.21.15.38:0/1374498245 >> 172.21.15.38:0/1374498245 conn(0x7f4f300d2350 msgr2=0x7f4f300d16f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:02.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:02.161+0000 7f4f36b2b640 1 -- 172.21.15.38:0/1374498245 shutdown_connections 2024-09-21T03:45:02.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:02.161+0000 7f4f36b2b640 1 -- 172.21.15.38:0/1374498245 wait complete. 2024-09-21T03:45:02.174 INFO:teuthology.orchestra.run.smithi038.stdout:{"pg_ready":true,"pg_map":{"version":488,"stamp":"2024-09-21T03:45:01.172110+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":141,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":105,"ondisk_log_size":105,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":118,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":306960,"kb_used_data":15848,"kb_used_omap":12,"kb_used_meta":290931,"kb_avail":754667760,"statfs":{"total":773094113280,"available":772779786240,"internally_reserved":0,"allocated":16228352,"data_stored":8782956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12748,"internal_metadata":297913908},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.859588"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":10,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T03:44:57.348031+0000","last_change":"2024-09-21T03:44:57.340688+0000","last_active":"2024-09-21T03:44:57.348031+0000","last_peered":"2024-09-21T03:44:57.348031+0000","last_clean":"2024-09-21T03:44:57.348031+0000","last_became_active":"2024-09-21T03:44:57.340356+0000","last_became_peered":"2024-09-21T03:44:57.340356+0000","last_unstale":"2024-09-21T03:44:57.348031+0000","last_undegraded":"2024-09-21T03:44:57.348031+0000","last_fullsized":"2024-09-21T03:44:57.348031+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:31:55.663249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.915466+0000","last_change":"2024-09-21T03:44:57.354834+0000","last_active":"2024-09-21T03:44:58.915466+0000","last_peered":"2024-09-21T03:44:58.915466+0000","last_clean":"2024-09-21T03:44:58.915466+0000","last_became_active":"2024-09-21T03:44:57.354518+0000","last_became_peered":"2024-09-21T03:44:57.354518+0000","last_unstale":"2024-09-21T03:44:58.915466+0000","last_undegraded":"2024-09-21T03:44:58.915466+0000","last_fullsized":"2024-09-21T03:44:58.915466+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:21:26.621000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":1,"reported_epoch":144,"state":"creating+peering","last_fresh":"2024-09-21T03:44:56.319458+0000","last_change":"2024-09-21T03:44:56.319458+0000","last_active":"2024-09-21T03:44:56.311610+0000","last_peered":"2024-09-21T03:44:56.311610+0000","last_clean":"2024-09-21T03:44:56.311610+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:44:56.319458+0000","last_undegraded":"2024-09-21T03:44:56.319458+0000","last_fullsized":"2024-09-21T03:44:56.319458+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":10,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T03:44:57.359756+0000","last_change":"2024-09-21T03:44:57.356466+0000","last_active":"2024-09-21T03:44:57.359756+0000","last_peered":"2024-09-21T03:44:57.359756+0000","last_clean":"2024-09-21T03:44:57.359756+0000","last_became_active":"2024-09-21T03:44:57.355938+0000","last_became_peered":"2024-09-21T03:44:57.355938+0000","last_unstale":"2024-09-21T03:44:57.359756+0000","last_undegraded":"2024-09-21T03:44:57.359756+0000","last_fullsized":"2024-09-21T03:44:57.359756+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:05:36.290619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":10,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T03:44:57.353943+0000","last_change":"2024-09-21T03:44:57.351972+0000","last_active":"2024-09-21T03:44:57.353943+0000","last_peered":"2024-09-21T03:44:57.353943+0000","last_clean":"2024-09-21T03:44:57.353943+0000","last_became_active":"2024-09-21T03:44:57.351561+0000","last_became_peered":"2024-09-21T03:44:57.351561+0000","last_unstale":"2024-09-21T03:44:57.353943+0000","last_undegraded":"2024-09-21T03:44:57.353943+0000","last_fullsized":"2024-09-21T03:44:57.353943+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:21:51.370818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.353120+0000","last_change":"2024-09-21T03:44:57.349848+0000","last_active":"2024-09-21T03:44:58.353120+0000","last_peered":"2024-09-21T03:44:58.353120+0000","last_clean":"2024-09-21T03:44:58.353120+0000","last_became_active":"2024-09-21T03:44:57.349387+0000","last_became_peered":"2024-09-21T03:44:57.349387+0000","last_unstale":"2024-09-21T03:44:58.353120+0000","last_undegraded":"2024-09-21T03:44:58.353120+0000","last_fullsized":"2024-09-21T03:44:58.353120+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:10:55.772054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.352606+0000","last_change":"2024-09-21T03:44:57.355533+0000","last_active":"2024-09-21T03:44:58.352606+0000","last_peered":"2024-09-21T03:44:58.352606+0000","last_clean":"2024-09-21T03:44:58.352606+0000","last_became_active":"2024-09-21T03:44:57.354892+0000","last_became_peered":"2024-09-21T03:44:57.354892+0000","last_unstale":"2024-09-21T03:44:58.352606+0000","last_undegraded":"2024-09-21T03:44:58.352606+0000","last_fullsized":"2024-09-21T03:44:58.352606+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:05:41.952462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":10,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T03:44:57.356024+0000","last_change":"2024-09-21T03:44:57.350864+0000","last_active":"2024-09-21T03:44:57.356024+0000","last_peered":"2024-09-21T03:44:57.356024+0000","last_clean":"2024-09-21T03:44:57.356024+0000","last_became_active":"2024-09-21T03:44:57.350215+0000","last_became_peered":"2024-09-21T03:44:57.350215+0000","last_unstale":"2024-09-21T03:44:57.356024+0000","last_undegraded":"2024-09-21T03:44:57.356024+0000","last_fullsized":"2024-09-21T03:44:57.356024+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:23:12.061449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.915427+0000","last_change":"2024-09-21T03:44:57.356640+0000","last_active":"2024-09-21T03:44:58.915427+0000","last_peered":"2024-09-21T03:44:58.915427+0000","last_clean":"2024-09-21T03:44:58.915427+0000","last_became_active":"2024-09-21T03:44:57.355966+0000","last_became_peered":"2024-09-21T03:44:57.355966+0000","last_unstale":"2024-09-21T03:44:58.915427+0000","last_undegraded":"2024-09-21T03:44:58.915427+0000","last_fullsized":"2024-09-21T03:44:58.915427+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:40:49.216619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":10,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T03:44:57.359889+0000","last_change":"2024-09-21T03:44:57.359320+0000","last_active":"2024-09-21T03:44:57.359889+0000","last_peered":"2024-09-21T03:44:57.359889+0000","last_clean":"2024-09-21T03:44:57.359889+0000","last_became_active":"2024-09-21T03:44:57.358988+0000","last_became_peered":"2024-09-21T03:44:57.358988+0000","last_unstale":"2024-09-21T03:44:57.359889+0000","last_undegraded":"2024-09-21T03:44:57.359889+0000","last_fullsized":"2024-09-21T03:44:57.359889+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:29:11.643073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.337467+0000","last_change":"2024-09-21T03:44:57.362260+0000","last_active":"2024-09-21T03:44:58.337467+0000","last_peered":"2024-09-21T03:44:58.337467+0000","last_clean":"2024-09-21T03:44:58.337467+0000","last_became_active":"2024-09-21T03:44:57.361624+0000","last_became_peered":"2024-09-21T03:44:57.361624+0000","last_unstale":"2024-09-21T03:44:58.337467+0000","last_undegraded":"2024-09-21T03:44:58.337467+0000","last_fullsized":"2024-09-21T03:44:58.337467+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:00:23.014227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.352884+0000","last_change":"2024-09-21T03:44:57.353828+0000","last_active":"2024-09-21T03:44:58.352884+0000","last_peered":"2024-09-21T03:44:58.352884+0000","last_clean":"2024-09-21T03:44:58.352884+0000","last_became_active":"2024-09-21T03:44:57.353516+0000","last_became_peered":"2024-09-21T03:44:57.353516+0000","last_unstale":"2024-09-21T03:44:58.352884+0000","last_undegraded":"2024-09-21T03:44:58.352884+0000","last_fullsized":"2024-09-21T03:44:58.352884+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:49:05.536169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":10,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T03:44:57.366752+0000","last_change":"2024-09-21T03:44:57.363696+0000","last_active":"2024-09-21T03:44:57.366752+0000","last_peered":"2024-09-21T03:44:57.366752+0000","last_clean":"2024-09-21T03:44:57.366752+0000","last_became_active":"2024-09-21T03:44:57.362737+0000","last_became_peered":"2024-09-21T03:44:57.362737+0000","last_unstale":"2024-09-21T03:44:57.366752+0000","last_undegraded":"2024-09-21T03:44:57.366752+0000","last_fullsized":"2024-09-21T03:44:57.366752+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:58:38.157904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.353349+0000","last_change":"2024-09-21T03:44:57.352690+0000","last_active":"2024-09-21T03:44:58.353349+0000","last_peered":"2024-09-21T03:44:58.353349+0000","last_clean":"2024-09-21T03:44:58.353349+0000","last_became_active":"2024-09-21T03:44:57.352358+0000","last_became_peered":"2024-09-21T03:44:57.352358+0000","last_unstale":"2024-09-21T03:44:58.353349+0000","last_undegraded":"2024-09-21T03:44:58.353349+0000","last_fullsized":"2024-09-21T03:44:58.353349+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:42:27.812988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.915095+0000","last_change":"2024-09-21T03:44:57.369797+0000","last_active":"2024-09-21T03:44:58.915095+0000","last_peered":"2024-09-21T03:44:58.915095+0000","last_clean":"2024-09-21T03:44:58.915095+0000","last_became_active":"2024-09-21T03:44:57.369463+0000","last_became_peered":"2024-09-21T03:44:57.369463+0000","last_unstale":"2024-09-21T03:44:58.915095+0000","last_undegraded":"2024-09-21T03:44:58.915095+0000","last_fullsized":"2024-09-21T03:44:58.915095+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:59:36.159782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.338697+0000","last_change":"2024-09-21T03:44:57.341545+0000","last_active":"2024-09-21T03:44:58.338697+0000","last_peered":"2024-09-21T03:44:58.338697+0000","last_clean":"2024-09-21T03:44:58.338697+0000","last_became_active":"2024-09-21T03:44:57.341220+0000","last_became_peered":"2024-09-21T03:44:57.341220+0000","last_unstale":"2024-09-21T03:44:58.338697+0000","last_undegraded":"2024-09-21T03:44:58.338697+0000","last_fullsized":"2024-09-21T03:44:58.338697+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:40:08.834183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":1,"reported_epoch":144,"state":"creating+peering","last_fresh":"2024-09-21T03:44:56.321953+0000","last_change":"2024-09-21T03:44:56.321953+0000","last_active":"2024-09-21T03:44:56.311610+0000","last_peered":"2024-09-21T03:44:56.311610+0000","last_clean":"2024-09-21T03:44:56.311610+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:44:56.321953+0000","last_undegraded":"2024-09-21T03:44:56.321953+0000","last_fullsized":"2024-09-21T03:44:56.321953+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.915438+0000","last_change":"2024-09-21T03:44:57.357272+0000","last_active":"2024-09-21T03:44:58.915438+0000","last_peered":"2024-09-21T03:44:58.915438+0000","last_clean":"2024-09-21T03:44:58.915438+0000","last_became_active":"2024-09-21T03:44:57.356952+0000","last_became_peered":"2024-09-21T03:44:57.356952+0000","last_unstale":"2024-09-21T03:44:58.915438+0000","last_undegraded":"2024-09-21T03:44:58.915438+0000","last_fullsized":"2024-09-21T03:44:58.915438+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:35:23.462964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":10,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T03:44:57.354219+0000","last_change":"2024-09-21T03:44:57.351765+0000","last_active":"2024-09-21T03:44:57.354219+0000","last_peered":"2024-09-21T03:44:57.354219+0000","last_clean":"2024-09-21T03:44:57.354219+0000","last_became_active":"2024-09-21T03:44:57.351478+0000","last_became_peered":"2024-09-21T03:44:57.351478+0000","last_unstale":"2024-09-21T03:44:57.354219+0000","last_undegraded":"2024-09-21T03:44:57.354219+0000","last_fullsized":"2024-09-21T03:44:57.354219+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:00:35.318345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.339047+0000","last_change":"2024-09-21T03:44:57.358796+0000","last_active":"2024-09-21T03:44:58.339047+0000","last_peered":"2024-09-21T03:44:58.339047+0000","last_clean":"2024-09-21T03:44:58.339047+0000","last_became_active":"2024-09-21T03:44:57.358439+0000","last_became_peered":"2024-09-21T03:44:57.358439+0000","last_unstale":"2024-09-21T03:44:58.339047+0000","last_undegraded":"2024-09-21T03:44:58.339047+0000","last_fullsized":"2024-09-21T03:44:58.339047+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:05:51.280587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":1,"reported_epoch":144,"state":"creating+peering","last_fresh":"2024-09-21T03:44:56.322208+0000","last_change":"2024-09-21T03:44:56.322208+0000","last_active":"2024-09-21T03:44:56.311610+0000","last_peered":"2024-09-21T03:44:56.311610+0000","last_clean":"2024-09-21T03:44:56.311610+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:44:56.322208+0000","last_undegraded":"2024-09-21T03:44:56.322208+0000","last_fullsized":"2024-09-21T03:44:56.322208+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.456326+0000","last_change":"2024-09-21T03:44:57.357854+0000","last_active":"2024-09-21T03:44:58.456326+0000","last_peered":"2024-09-21T03:44:58.456326+0000","last_clean":"2024-09-21T03:44:58.456326+0000","last_became_active":"2024-09-21T03:44:57.357541+0000","last_became_peered":"2024-09-21T03:44:57.357541+0000","last_unstale":"2024-09-21T03:44:58.456326+0000","last_undegraded":"2024-09-21T03:44:58.456326+0000","last_fullsized":"2024-09-21T03:44:58.456326+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:04:52.143962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":10,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T03:44:57.355917+0000","last_change":"2024-09-21T03:44:57.353700+0000","last_active":"2024-09-21T03:44:57.355917+0000","last_peered":"2024-09-21T03:44:57.355917+0000","last_clean":"2024-09-21T03:44:57.355917+0000","last_became_active":"2024-09-21T03:44:57.351501+0000","last_became_peered":"2024-09-21T03:44:57.351501+0000","last_unstale":"2024-09-21T03:44:57.355917+0000","last_undegraded":"2024-09-21T03:44:57.355917+0000","last_fullsized":"2024-09-21T03:44:57.355917+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:00:30.971449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":10,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T03:44:57.356048+0000","last_change":"2024-09-21T03:44:57.351638+0000","last_active":"2024-09-21T03:44:57.356048+0000","last_peered":"2024-09-21T03:44:57.356048+0000","last_clean":"2024-09-21T03:44:57.356048+0000","last_became_active":"2024-09-21T03:44:57.351325+0000","last_became_peered":"2024-09-21T03:44:57.351325+0000","last_unstale":"2024-09-21T03:44:57.356048+0000","last_undegraded":"2024-09-21T03:44:57.356048+0000","last_fullsized":"2024-09-21T03:44:57.356048+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:05:57.831029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.339252+0000","last_change":"2024-09-21T03:44:57.359418+0000","last_active":"2024-09-21T03:44:58.339252+0000","last_peered":"2024-09-21T03:44:58.339252+0000","last_clean":"2024-09-21T03:44:58.339252+0000","last_became_active":"2024-09-21T03:44:57.358564+0000","last_became_peered":"2024-09-21T03:44:57.358564+0000","last_unstale":"2024-09-21T03:44:58.339252+0000","last_undegraded":"2024-09-21T03:44:58.339252+0000","last_fullsized":"2024-09-21T03:44:58.339252+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:21:26.978675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.339318+0000","last_change":"2024-09-21T03:44:57.360155+0000","last_active":"2024-09-21T03:44:58.339318+0000","last_peered":"2024-09-21T03:44:58.339318+0000","last_clean":"2024-09-21T03:44:58.339318+0000","last_became_active":"2024-09-21T03:44:57.359645+0000","last_became_peered":"2024-09-21T03:44:57.359645+0000","last_unstale":"2024-09-21T03:44:58.339318+0000","last_undegraded":"2024-09-21T03:44:58.339318+0000","last_fullsized":"2024-09-21T03:44:58.339318+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:14:53.602909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.339127+0000","last_change":"2024-09-21T03:44:57.357568+0000","last_active":"2024-09-21T03:44:58.339127+0000","last_peered":"2024-09-21T03:44:58.339127+0000","last_clean":"2024-09-21T03:44:58.339127+0000","last_became_active":"2024-09-21T03:44:57.357185+0000","last_became_peered":"2024-09-21T03:44:57.357185+0000","last_unstale":"2024-09-21T03:44:58.339127+0000","last_undegraded":"2024-09-21T03:44:58.339127+0000","last_fullsized":"2024-09-21T03:44:58.339127+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:57:24.936155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.353110+0000","last_change":"2024-09-21T03:44:57.352231+0000","last_active":"2024-09-21T03:44:58.353110+0000","last_peered":"2024-09-21T03:44:58.353110+0000","last_clean":"2024-09-21T03:44:58.353110+0000","last_became_active":"2024-09-21T03:44:57.351887+0000","last_became_peered":"2024-09-21T03:44:57.351887+0000","last_unstale":"2024-09-21T03:44:58.353110+0000","last_undegraded":"2024-09-21T03:44:58.353110+0000","last_fullsized":"2024-09-21T03:44:58.353110+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:46:07.069681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":10,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T03:44:57.358271+0000","last_change":"2024-09-21T03:44:57.357470+0000","last_active":"2024-09-21T03:44:57.358271+0000","last_peered":"2024-09-21T03:44:57.358271+0000","last_clean":"2024-09-21T03:44:57.358271+0000","last_became_active":"2024-09-21T03:44:57.357091+0000","last_became_peered":"2024-09-21T03:44:57.357091+0000","last_unstale":"2024-09-21T03:44:57.358271+0000","last_undegraded":"2024-09-21T03:44:57.358271+0000","last_fullsized":"2024-09-21T03:44:57.358271+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:13:25.917453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.337478+0000","last_change":"2024-09-21T03:44:57.361022+0000","last_active":"2024-09-21T03:44:58.337478+0000","last_peered":"2024-09-21T03:44:58.337478+0000","last_clean":"2024-09-21T03:44:58.337478+0000","last_became_active":"2024-09-21T03:44:57.360697+0000","last_became_peered":"2024-09-21T03:44:57.360697+0000","last_unstale":"2024-09-21T03:44:58.337478+0000","last_undegraded":"2024-09-21T03:44:58.337478+0000","last_fullsized":"2024-09-21T03:44:58.337478+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:17:17.662702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.352611+0000","last_change":"2024-09-21T03:44:57.357841+0000","last_active":"2024-09-21T03:44:58.352611+0000","last_peered":"2024-09-21T03:44:58.352611+0000","last_clean":"2024-09-21T03:44:58.352611+0000","last_became_active":"2024-09-21T03:44:57.357299+0000","last_became_peered":"2024-09-21T03:44:57.357299+0000","last_unstale":"2024-09-21T03:44:58.352611+0000","last_undegraded":"2024-09-21T03:44:58.352611+0000","last_fullsized":"2024-09-21T03:44:58.352611+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:36:45.265239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.456162+0000","last_change":"2024-09-21T03:44:57.366423+0000","last_active":"2024-09-21T03:44:58.456162+0000","last_peered":"2024-09-21T03:44:58.456162+0000","last_clean":"2024-09-21T03:44:58.456162+0000","last_became_active":"2024-09-21T03:44:57.366083+0000","last_became_peered":"2024-09-21T03:44:57.366083+0000","last_unstale":"2024-09-21T03:44:58.456162+0000","last_undegraded":"2024-09-21T03:44:58.456162+0000","last_fullsized":"2024-09-21T03:44:58.456162+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:14:54.627243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":1,"reported_epoch":144,"state":"creating+peering","last_fresh":"2024-09-21T03:44:56.321750+0000","last_change":"2024-09-21T03:44:56.321750+0000","last_active":"2024-09-21T03:44:56.311610+0000","last_peered":"2024-09-21T03:44:56.311610+0000","last_clean":"2024-09-21T03:44:56.311610+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:44:56.321750+0000","last_undegraded":"2024-09-21T03:44:56.321750+0000","last_fullsized":"2024-09-21T03:44:56.321750+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1f","version":"145'4","reported_seq":19,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.456694+0000","last_change":"2024-09-21T03:44:56.332485+0000","last_active":"2024-09-21T03:44:58.456694+0000","last_peered":"2024-09-21T03:44:58.456694+0000","last_clean":"2024-09-21T03:44:58.456694+0000","last_became_active":"2024-09-21T03:44:56.332204+0000","last_became_peered":"2024-09-21T03:44:56.332204+0000","last_unstale":"2024-09-21T03:44:58.456694+0000","last_undegraded":"2024-09-21T03:44:58.456694+0000","last_fullsized":"2024-09-21T03:44:58.456694+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_clean_scrub_stamp":"2024-09-21T03:44:55.304777+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:15:02.688269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":10,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T03:44:57.356612+0000","last_change":"2024-09-21T03:44:57.352443+0000","last_active":"2024-09-21T03:44:57.356612+0000","last_peered":"2024-09-21T03:44:57.356612+0000","last_clean":"2024-09-21T03:44:57.356612+0000","last_became_active":"2024-09-21T03:44:57.351086+0000","last_became_peered":"2024-09-21T03:44:57.351086+0000","last_unstale":"2024-09-21T03:44:57.356612+0000","last_undegraded":"2024-09-21T03:44:57.356612+0000","last_fullsized":"2024-09-21T03:44:57.356612+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:27:41.239844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T03:44:56.342206+0000","last_change":"2024-09-21T03:44:56.341615+0000","last_active":"2024-09-21T03:44:56.342206+0000","last_peered":"2024-09-21T03:44:56.342206+0000","last_clean":"2024-09-21T03:44:56.342206+0000","last_became_active":"2024-09-21T03:44:56.341306+0000","last_became_peered":"2024-09-21T03:44:56.341306+0000","last_unstale":"2024-09-21T03:44:56.342206+0000","last_undegraded":"2024-09-21T03:44:56.342206+0000","last_fullsized":"2024-09-21T03:44:56.342206+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_clean_scrub_stamp":"2024-09-21T03:44:55.304777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:36:07.439425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.915135+0000","last_change":"2024-09-21T03:44:57.349398+0000","last_active":"2024-09-21T03:44:58.915135+0000","last_peered":"2024-09-21T03:44:58.915135+0000","last_clean":"2024-09-21T03:44:58.915135+0000","last_became_active":"2024-09-21T03:44:57.349067+0000","last_became_peered":"2024-09-21T03:44:57.349067+0000","last_unstale":"2024-09-21T03:44:58.915135+0000","last_undegraded":"2024-09-21T03:44:58.915135+0000","last_fullsized":"2024-09-21T03:44:58.915135+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:19:45.952024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"145'2","reported_seq":17,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.915396+0000","last_change":"2024-09-21T03:44:56.343325+0000","last_active":"2024-09-21T03:44:58.915396+0000","last_peered":"2024-09-21T03:44:58.915396+0000","last_clean":"2024-09-21T03:44:58.915396+0000","last_became_active":"2024-09-21T03:44:56.342964+0000","last_became_peered":"2024-09-21T03:44:56.342964+0000","last_unstale":"2024-09-21T03:44:58.915396+0000","last_undegraded":"2024-09-21T03:44:58.915396+0000","last_fullsized":"2024-09-21T03:44:58.915396+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_clean_scrub_stamp":"2024-09-21T03:44:55.304777+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:19:36.404105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.337488+0000","last_change":"2024-09-21T03:44:57.360138+0000","last_active":"2024-09-21T03:44:58.337488+0000","last_peered":"2024-09-21T03:44:58.337488+0000","last_clean":"2024-09-21T03:44:58.337488+0000","last_became_active":"2024-09-21T03:44:57.359800+0000","last_became_peered":"2024-09-21T03:44:57.359800+0000","last_unstale":"2024-09-21T03:44:58.337488+0000","last_undegraded":"2024-09-21T03:44:58.337488+0000","last_fullsized":"2024-09-21T03:44:58.337488+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:33:24.015377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1c","version":"145'2","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T03:44:57.373298+0000","last_change":"2024-09-21T03:44:56.334534+0000","last_active":"2024-09-21T03:44:57.373298+0000","last_peered":"2024-09-21T03:44:57.373298+0000","last_clean":"2024-09-21T03:44:57.373298+0000","last_became_active":"2024-09-21T03:44:56.333800+0000","last_became_peered":"2024-09-21T03:44:56.333800+0000","last_unstale":"2024-09-21T03:44:57.373298+0000","last_undegraded":"2024-09-21T03:44:57.373298+0000","last_fullsized":"2024-09-21T03:44:57.373298+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_clean_scrub_stamp":"2024-09-21T03:44:55.304777+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:44:41.072083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":10,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T03:44:57.350445+0000","last_change":"2024-09-21T03:44:57.342301+0000","last_active":"2024-09-21T03:44:57.350445+0000","last_peered":"2024-09-21T03:44:57.350445+0000","last_clean":"2024-09-21T03:44:57.350445+0000","last_became_active":"2024-09-21T03:44:57.341880+0000","last_became_peered":"2024-09-21T03:44:57.341880+0000","last_unstale":"2024-09-21T03:44:57.350445+0000","last_undegraded":"2024-09-21T03:44:57.350445+0000","last_fullsized":"2024-09-21T03:44:57.350445+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:10:55.772054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1b","version":"145'1","reported_seq":16,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.337514+0000","last_change":"2024-09-21T03:44:56.342732+0000","last_active":"2024-09-21T03:44:58.337514+0000","last_peered":"2024-09-21T03:44:58.337514+0000","last_clean":"2024-09-21T03:44:58.337514+0000","last_became_active":"2024-09-21T03:44:56.342319+0000","last_became_peered":"2024-09-21T03:44:56.342319+0000","last_unstale":"2024-09-21T03:44:58.337514+0000","last_undegraded":"2024-09-21T03:44:58.337514+0000","last_fullsized":"2024-09-21T03:44:58.337514+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_clean_scrub_stamp":"2024-09-21T03:44:55.304777+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:31:32.978494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":10,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T03:44:57.350445+0000","last_change":"2024-09-21T03:44:57.346978+0000","last_active":"2024-09-21T03:44:57.350445+0000","last_peered":"2024-09-21T03:44:57.350445+0000","last_clean":"2024-09-21T03:44:57.350445+0000","last_became_active":"2024-09-21T03:44:57.346633+0000","last_became_peered":"2024-09-21T03:44:57.346633+0000","last_unstale":"2024-09-21T03:44:57.350445+0000","last_undegraded":"2024-09-21T03:44:57.350445+0000","last_fullsized":"2024-09-21T03:44:57.350445+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:17:57.312760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1a","version":"145'1","reported_seq":16,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.338782+0000","last_change":"2024-09-21T03:44:56.342500+0000","last_active":"2024-09-21T03:44:58.338782+0000","last_peered":"2024-09-21T03:44:58.338782+0000","last_clean":"2024-09-21T03:44:58.338782+0000","last_became_active":"2024-09-21T03:44:56.342198+0000","last_became_peered":"2024-09-21T03:44:56.342198+0000","last_unstale":"2024-09-21T03:44:58.338782+0000","last_undegraded":"2024-09-21T03:44:58.338782+0000","last_fullsized":"2024-09-21T03:44:58.338782+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_clean_scrub_stamp":"2024-09-21T03:44:55.304777+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:24:33.377335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.337947+0000","last_change":"2024-09-21T03:44:57.360511+0000","last_active":"2024-09-21T03:44:58.337947+0000","last_peered":"2024-09-21T03:44:58.337947+0000","last_clean":"2024-09-21T03:44:58.337947+0000","last_became_active":"2024-09-21T03:44:57.360163+0000","last_became_peered":"2024-09-21T03:44:57.360163+0000","last_unstale":"2024-09-21T03:44:58.337947+0000","last_undegraded":"2024-09-21T03:44:58.337947+0000","last_fullsized":"2024-09-21T03:44:58.337947+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:05:31.071185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":15,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.338632+0000","last_change":"2024-09-21T03:44:56.343444+0000","last_active":"2024-09-21T03:44:58.338632+0000","last_peered":"2024-09-21T03:44:58.338632+0000","last_clean":"2024-09-21T03:44:58.338632+0000","last_became_active":"2024-09-21T03:44:56.343128+0000","last_became_peered":"2024-09-21T03:44:56.343128+0000","last_unstale":"2024-09-21T03:44:58.338632+0000","last_undegraded":"2024-09-21T03:44:58.338632+0000","last_fullsized":"2024-09-21T03:44:58.338632+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_clean_scrub_stamp":"2024-09-21T03:44:55.304777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:49:27.666317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":10,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T03:44:57.351262+0000","last_change":"2024-09-21T03:44:57.348168+0000","last_active":"2024-09-21T03:44:57.351262+0000","last_peered":"2024-09-21T03:44:57.351262+0000","last_clean":"2024-09-21T03:44:57.351262+0000","last_became_active":"2024-09-21T03:44:57.347823+0000","last_became_peered":"2024-09-21T03:44:57.347823+0000","last_unstale":"2024-09-21T03:44:57.351262+0000","last_undegraded":"2024-09-21T03:44:57.351262+0000","last_fullsized":"2024-09-21T03:44:57.351262+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:43:22.624675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":15,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.456487+0000","last_change":"2024-09-21T03:44:56.337896+0000","last_active":"2024-09-21T03:44:58.456487+0000","last_peered":"2024-09-21T03:44:58.456487+0000","last_clean":"2024-09-21T03:44:58.456487+0000","last_became_active":"2024-09-21T03:44:56.337298+0000","last_became_peered":"2024-09-21T03:44:56.337298+0000","last_unstale":"2024-09-21T03:44:58.456487+0000","last_undegraded":"2024-09-21T03:44:58.456487+0000","last_fullsized":"2024-09-21T03:44:58.456487+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_clean_scrub_stamp":"2024-09-21T03:44:55.304777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:58:50.496950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":10,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T03:44:57.351652+0000","last_change":"2024-09-21T03:44:57.350094+0000","last_active":"2024-09-21T03:44:57.351652+0000","last_peered":"2024-09-21T03:44:57.351652+0000","last_clean":"2024-09-21T03:44:57.351652+0000","last_became_active":"2024-09-21T03:44:57.349777+0000","last_became_peered":"2024-09-21T03:44:57.349777+0000","last_unstale":"2024-09-21T03:44:57.351652+0000","last_undegraded":"2024-09-21T03:44:57.351652+0000","last_fullsized":"2024-09-21T03:44:57.351652+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:36:06.178611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":15,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.353239+0000","last_change":"2024-09-21T03:44:56.329230+0000","last_active":"2024-09-21T03:44:58.353239+0000","last_peered":"2024-09-21T03:44:58.353239+0000","last_clean":"2024-09-21T03:44:58.353239+0000","last_became_active":"2024-09-21T03:44:56.328911+0000","last_became_peered":"2024-09-21T03:44:56.328911+0000","last_unstale":"2024-09-21T03:44:58.353239+0000","last_undegraded":"2024-09-21T03:44:58.353239+0000","last_fullsized":"2024-09-21T03:44:58.353239+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_clean_scrub_stamp":"2024-09-21T03:44:55.304777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:16:45.547857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":10,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T03:44:57.351808+0000","last_change":"2024-09-21T03:44:57.347015+0000","last_active":"2024-09-21T03:44:57.351808+0000","last_peered":"2024-09-21T03:44:57.351808+0000","last_clean":"2024-09-21T03:44:57.351808+0000","last_became_active":"2024-09-21T03:44:57.346668+0000","last_became_peered":"2024-09-21T03:44:57.346668+0000","last_unstale":"2024-09-21T03:44:57.351808+0000","last_undegraded":"2024-09-21T03:44:57.351808+0000","last_fullsized":"2024-09-21T03:44:57.351808+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:49:55.140933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T03:44:57.341754+0000","last_change":"2024-09-21T03:44:56.333386+0000","last_active":"2024-09-21T03:44:57.341754+0000","last_peered":"2024-09-21T03:44:57.341754+0000","last_clean":"2024-09-21T03:44:57.341754+0000","last_became_active":"2024-09-21T03:44:56.333082+0000","last_became_peered":"2024-09-21T03:44:56.333082+0000","last_unstale":"2024-09-21T03:44:57.341754+0000","last_undegraded":"2024-09-21T03:44:57.341754+0000","last_fullsized":"2024-09-21T03:44:57.341754+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_clean_scrub_stamp":"2024-09-21T03:44:55.304777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:10:24.715196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":10,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T03:44:57.355956+0000","last_change":"2024-09-21T03:44:57.350041+0000","last_active":"2024-09-21T03:44:57.355956+0000","last_peered":"2024-09-21T03:44:57.355956+0000","last_clean":"2024-09-21T03:44:57.355956+0000","last_became_active":"2024-09-21T03:44:57.349706+0000","last_became_peered":"2024-09-21T03:44:57.349706+0000","last_unstale":"2024-09-21T03:44:57.355956+0000","last_undegraded":"2024-09-21T03:44:57.355956+0000","last_fullsized":"2024-09-21T03:44:57.355956+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:35:23.899135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.15","version":"145'3","reported_seq":18,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.352972+0000","last_change":"2024-09-21T03:44:56.324998+0000","last_active":"2024-09-21T03:44:58.352972+0000","last_peered":"2024-09-21T03:44:58.352972+0000","last_clean":"2024-09-21T03:44:58.352972+0000","last_became_active":"2024-09-21T03:44:56.324464+0000","last_became_peered":"2024-09-21T03:44:56.324464+0000","last_unstale":"2024-09-21T03:44:58.352972+0000","last_undegraded":"2024-09-21T03:44:58.352972+0000","last_fullsized":"2024-09-21T03:44:58.352972+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_clean_scrub_stamp":"2024-09-21T03:44:55.304777+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:27:45.431858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.456464+0000","last_change":"2024-09-21T03:44:57.355113+0000","last_active":"2024-09-21T03:44:58.456464+0000","last_peered":"2024-09-21T03:44:58.456464+0000","last_clean":"2024-09-21T03:44:58.456464+0000","last_became_active":"2024-09-21T03:44:57.354744+0000","last_became_peered":"2024-09-21T03:44:57.354744+0000","last_unstale":"2024-09-21T03:44:58.456464+0000","last_undegraded":"2024-09-21T03:44:58.456464+0000","last_fullsized":"2024-09-21T03:44:58.456464+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:54:24.418739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.14","version":"145'6","reported_seq":21,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.337583+0000","last_change":"2024-09-21T03:44:56.341013+0000","last_active":"2024-09-21T03:44:58.337583+0000","last_peered":"2024-09-21T03:44:58.337583+0000","last_clean":"2024-09-21T03:44:58.337583+0000","last_became_active":"2024-09-21T03:44:56.337878+0000","last_became_peered":"2024-09-21T03:44:56.337878+0000","last_unstale":"2024-09-21T03:44:58.337583+0000","last_undegraded":"2024-09-21T03:44:58.337583+0000","last_fullsized":"2024-09-21T03:44:58.337583+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_clean_scrub_stamp":"2024-09-21T03:44:55.304777+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:00:34.311511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.915365+0000","last_change":"2024-09-21T03:44:57.356176+0000","last_active":"2024-09-21T03:44:58.915365+0000","last_peered":"2024-09-21T03:44:58.915365+0000","last_clean":"2024-09-21T03:44:58.915365+0000","last_became_active":"2024-09-21T03:44:57.355832+0000","last_became_peered":"2024-09-21T03:44:57.355832+0000","last_unstale":"2024-09-21T03:44:58.915365+0000","last_undegraded":"2024-09-21T03:44:58.915365+0000","last_fullsized":"2024-09-21T03:44:58.915365+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:47:30.592382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.13","version":"145'2","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T03:44:57.367038+0000","last_change":"2024-09-21T03:44:56.327633+0000","last_active":"2024-09-21T03:44:57.367038+0000","last_peered":"2024-09-21T03:44:57.367038+0000","last_clean":"2024-09-21T03:44:57.367038+0000","last_became_active":"2024-09-21T03:44:56.327313+0000","last_became_peered":"2024-09-21T03:44:56.327313+0000","last_unstale":"2024-09-21T03:44:57.367038+0000","last_undegraded":"2024-09-21T03:44:57.367038+0000","last_fullsized":"2024-09-21T03:44:57.367038+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_clean_scrub_stamp":"2024-09-21T03:44:55.304777+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:36:44.258406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.915449+0000","last_change":"2024-09-21T03:44:57.352611+0000","last_active":"2024-09-21T03:44:58.915449+0000","last_peered":"2024-09-21T03:44:58.915449+0000","last_clean":"2024-09-21T03:44:58.915449+0000","last_became_active":"2024-09-21T03:44:57.352136+0000","last_became_peered":"2024-09-21T03:44:57.352136+0000","last_unstale":"2024-09-21T03:44:58.915449+0000","last_undegraded":"2024-09-21T03:44:58.915449+0000","last_fullsized":"2024-09-21T03:44:58.915449+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:26:06.882445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T03:44:57.341635+0000","last_change":"2024-09-21T03:44:56.337958+0000","last_active":"2024-09-21T03:44:57.341635+0000","last_peered":"2024-09-21T03:44:57.341635+0000","last_clean":"2024-09-21T03:44:57.341635+0000","last_became_active":"2024-09-21T03:44:56.337623+0000","last_became_peered":"2024-09-21T03:44:56.337623+0000","last_unstale":"2024-09-21T03:44:57.341635+0000","last_undegraded":"2024-09-21T03:44:57.341635+0000","last_fullsized":"2024-09-21T03:44:57.341635+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_clean_scrub_stamp":"2024-09-21T03:44:55.304777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:45:31.964595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.456704+0000","last_change":"2024-09-21T03:44:57.358895+0000","last_active":"2024-09-21T03:44:58.456704+0000","last_peered":"2024-09-21T03:44:58.456704+0000","last_clean":"2024-09-21T03:44:58.456704+0000","last_became_active":"2024-09-21T03:44:57.358591+0000","last_became_peered":"2024-09-21T03:44:57.358591+0000","last_unstale":"2024-09-21T03:44:58.456704+0000","last_undegraded":"2024-09-21T03:44:58.456704+0000","last_fullsized":"2024-09-21T03:44:58.456704+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:38:18.158452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T03:44:57.337893+0000","last_change":"2024-09-21T03:44:56.327693+0000","last_active":"2024-09-21T03:44:57.337893+0000","last_peered":"2024-09-21T03:44:57.337893+0000","last_clean":"2024-09-21T03:44:57.337893+0000","last_became_active":"2024-09-21T03:44:56.327364+0000","last_became_peered":"2024-09-21T03:44:56.327364+0000","last_unstale":"2024-09-21T03:44:57.337893+0000","last_undegraded":"2024-09-21T03:44:57.337893+0000","last_fullsized":"2024-09-21T03:44:57.337893+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_clean_scrub_stamp":"2024-09-21T03:44:55.304777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:16:04.551826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":1,"reported_epoch":144,"state":"creating+peering","last_fresh":"2024-09-21T03:44:56.321369+0000","last_change":"2024-09-21T03:44:56.321369+0000","last_active":"2024-09-21T03:44:56.311610+0000","last_peered":"2024-09-21T03:44:56.311610+0000","last_clean":"2024-09-21T03:44:56.311610+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:44:56.321369+0000","last_undegraded":"2024-09-21T03:44:56.321369+0000","last_fullsized":"2024-09-21T03:44:56.321369+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.10","version":"145'8","reported_seq":23,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.337672+0000","last_change":"2024-09-21T03:44:56.342213+0000","last_active":"2024-09-21T03:44:58.337672+0000","last_peered":"2024-09-21T03:44:58.337672+0000","last_clean":"2024-09-21T03:44:58.337672+0000","last_became_active":"2024-09-21T03:44:56.341888+0000","last_became_peered":"2024-09-21T03:44:56.341888+0000","last_unstale":"2024-09-21T03:44:58.337672+0000","last_undegraded":"2024-09-21T03:44:58.337672+0000","last_fullsized":"2024-09-21T03:44:58.337672+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_clean_scrub_stamp":"2024-09-21T03:44:55.304777+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:36:05.171778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":10,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T03:44:57.355999+0000","last_change":"2024-09-21T03:44:57.350536+0000","last_active":"2024-09-21T03:44:57.355999+0000","last_peered":"2024-09-21T03:44:57.355999+0000","last_clean":"2024-09-21T03:44:57.355999+0000","last_became_active":"2024-09-21T03:44:57.349795+0000","last_became_peered":"2024-09-21T03:44:57.349795+0000","last_unstale":"2024-09-21T03:44:57.355999+0000","last_undegraded":"2024-09-21T03:44:57.355999+0000","last_fullsized":"2024-09-21T03:44:57.355999+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:09:26.811858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.f","version":"145'1","reported_seq":16,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.353274+0000","last_change":"2024-09-21T03:44:56.339856+0000","last_active":"2024-09-21T03:44:58.353274+0000","last_peered":"2024-09-21T03:44:58.353274+0000","last_clean":"2024-09-21T03:44:58.353274+0000","last_became_active":"2024-09-21T03:44:56.339305+0000","last_became_peered":"2024-09-21T03:44:56.339305+0000","last_unstale":"2024-09-21T03:44:58.353274+0000","last_undegraded":"2024-09-21T03:44:58.353274+0000","last_fullsized":"2024-09-21T03:44:58.353274+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_clean_scrub_stamp":"2024-09-21T03:44:55.304777+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:37:25.372042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.338656+0000","last_change":"2024-09-21T03:44:57.341004+0000","last_active":"2024-09-21T03:44:58.338656+0000","last_peered":"2024-09-21T03:44:58.338656+0000","last_clean":"2024-09-21T03:44:58.338656+0000","last_became_active":"2024-09-21T03:44:57.340620+0000","last_became_peered":"2024-09-21T03:44:57.340620+0000","last_unstale":"2024-09-21T03:44:58.338656+0000","last_undegraded":"2024-09-21T03:44:58.338656+0000","last_fullsized":"2024-09-21T03:44:58.338656+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:09:01.018690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.d","version":"145'1","reported_seq":16,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.339122+0000","last_change":"2024-09-21T03:44:56.340547+0000","last_active":"2024-09-21T03:44:58.339122+0000","last_peered":"2024-09-21T03:44:58.339122+0000","last_clean":"2024-09-21T03:44:58.339122+0000","last_became_active":"2024-09-21T03:44:56.340237+0000","last_became_peered":"2024-09-21T03:44:56.340237+0000","last_unstale":"2024-09-21T03:44:58.339122+0000","last_undegraded":"2024-09-21T03:44:58.339122+0000","last_fullsized":"2024-09-21T03:44:58.339122+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_clean_scrub_stamp":"2024-09-21T03:44:55.304777+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:19:41.082645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.456321+0000","last_change":"2024-09-21T03:44:57.358052+0000","last_active":"2024-09-21T03:44:58.456321+0000","last_peered":"2024-09-21T03:44:58.456321+0000","last_clean":"2024-09-21T03:44:58.456321+0000","last_became_active":"2024-09-21T03:44:57.357731+0000","last_became_peered":"2024-09-21T03:44:57.357731+0000","last_unstale":"2024-09-21T03:44:58.456321+0000","last_undegraded":"2024-09-21T03:44:58.456321+0000","last_fullsized":"2024-09-21T03:44:58.456321+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:02:38.958342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":15,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.353016+0000","last_change":"2024-09-21T03:44:56.339649+0000","last_active":"2024-09-21T03:44:58.353016+0000","last_peered":"2024-09-21T03:44:58.353016+0000","last_clean":"2024-09-21T03:44:58.353016+0000","last_became_active":"2024-09-21T03:44:56.339317+0000","last_became_peered":"2024-09-21T03:44:56.339317+0000","last_unstale":"2024-09-21T03:44:58.353016+0000","last_undegraded":"2024-09-21T03:44:58.353016+0000","last_fullsized":"2024-09-21T03:44:58.353016+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_clean_scrub_stamp":"2024-09-21T03:44:55.304777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:58:28.485819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":10,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T03:44:57.353966+0000","last_change":"2024-09-21T03:44:57.350478+0000","last_active":"2024-09-21T03:44:57.353966+0000","last_peered":"2024-09-21T03:44:57.353966+0000","last_clean":"2024-09-21T03:44:57.353966+0000","last_became_active":"2024-09-21T03:44:57.348990+0000","last_became_peered":"2024-09-21T03:44:57.348990+0000","last_unstale":"2024-09-21T03:44:57.353966+0000","last_undegraded":"2024-09-21T03:44:57.353966+0000","last_fullsized":"2024-09-21T03:44:57.353966+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:40:27.993685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T03:44:56.330403+0000","last_change":"2024-09-21T03:44:56.329417+0000","last_active":"2024-09-21T03:44:56.330403+0000","last_peered":"2024-09-21T03:44:56.330403+0000","last_clean":"2024-09-21T03:44:56.330403+0000","last_became_active":"2024-09-21T03:44:56.329098+0000","last_became_peered":"2024-09-21T03:44:56.329098+0000","last_unstale":"2024-09-21T03:44:56.330403+0000","last_undegraded":"2024-09-21T03:44:56.330403+0000","last_fullsized":"2024-09-21T03:44:56.330403+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_clean_scrub_stamp":"2024-09-21T03:44:55.304777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:53:45.362390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.456620+0000","last_change":"2024-09-21T03:44:57.354679+0000","last_active":"2024-09-21T03:44:58.456620+0000","last_peered":"2024-09-21T03:44:58.456620+0000","last_clean":"2024-09-21T03:44:58.456620+0000","last_became_active":"2024-09-21T03:44:57.354357+0000","last_became_peered":"2024-09-21T03:44:57.354357+0000","last_unstale":"2024-09-21T03:44:58.456620+0000","last_undegraded":"2024-09-21T03:44:58.456620+0000","last_fullsized":"2024-09-21T03:44:58.456620+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:29:19.509171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T03:44:56.331506+0000","last_change":"2024-09-21T03:44:56.330054+0000","last_active":"2024-09-21T03:44:56.331506+0000","last_peered":"2024-09-21T03:44:56.331506+0000","last_clean":"2024-09-21T03:44:56.331506+0000","last_became_active":"2024-09-21T03:44:56.329151+0000","last_became_peered":"2024-09-21T03:44:56.329151+0000","last_unstale":"2024-09-21T03:44:56.331506+0000","last_undegraded":"2024-09-21T03:44:56.331506+0000","last_fullsized":"2024-09-21T03:44:56.331506+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_clean_scrub_stamp":"2024-09-21T03:44:55.304777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:15:21.096050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.456652+0000","last_change":"2024-09-21T03:44:57.363634+0000","last_active":"2024-09-21T03:44:58.456652+0000","last_peered":"2024-09-21T03:44:58.456652+0000","last_clean":"2024-09-21T03:44:58.456652+0000","last_became_active":"2024-09-21T03:44:57.363259+0000","last_became_peered":"2024-09-21T03:44:57.363259+0000","last_unstale":"2024-09-21T03:44:58.456652+0000","last_undegraded":"2024-09-21T03:44:58.456652+0000","last_fullsized":"2024-09-21T03:44:58.456652+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:53:39.529389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":15,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.353119+0000","last_change":"2024-09-21T03:44:56.327686+0000","last_active":"2024-09-21T03:44:58.353119+0000","last_peered":"2024-09-21T03:44:58.353119+0000","last_clean":"2024-09-21T03:44:58.353119+0000","last_became_active":"2024-09-21T03:44:56.327355+0000","last_became_peered":"2024-09-21T03:44:56.327355+0000","last_unstale":"2024-09-21T03:44:58.353119+0000","last_undegraded":"2024-09-21T03:44:58.353119+0000","last_fullsized":"2024-09-21T03:44:58.353119+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_clean_scrub_stamp":"2024-09-21T03:44:55.304777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:26:58.029665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":10,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T03:44:57.352767+0000","last_change":"2024-09-21T03:44:57.351050+0000","last_active":"2024-09-21T03:44:57.352767+0000","last_peered":"2024-09-21T03:44:57.352767+0000","last_clean":"2024-09-21T03:44:57.352767+0000","last_became_active":"2024-09-21T03:44:57.350731+0000","last_became_peered":"2024-09-21T03:44:57.350731+0000","last_unstale":"2024-09-21T03:44:57.352767+0000","last_undegraded":"2024-09-21T03:44:57.352767+0000","last_fullsized":"2024-09-21T03:44:57.352767+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:31:33.985328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T03:44:57.340435+0000","last_change":"2024-09-21T03:44:56.334506+0000","last_active":"2024-09-21T03:44:57.340435+0000","last_peered":"2024-09-21T03:44:57.340435+0000","last_clean":"2024-09-21T03:44:57.340435+0000","last_became_active":"2024-09-21T03:44:56.334007+0000","last_became_peered":"2024-09-21T03:44:56.334007+0000","last_unstale":"2024-09-21T03:44:57.340435+0000","last_undegraded":"2024-09-21T03:44:57.340435+0000","last_fullsized":"2024-09-21T03:44:57.340435+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_clean_scrub_stamp":"2024-09-21T03:44:55.304777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:57:15.575727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.456155+0000","last_change":"2024-09-21T03:44:57.350289+0000","last_active":"2024-09-21T03:44:58.456155+0000","last_peered":"2024-09-21T03:44:58.456155+0000","last_clean":"2024-09-21T03:44:58.456155+0000","last_became_active":"2024-09-21T03:44:57.349932+0000","last_became_peered":"2024-09-21T03:44:57.349932+0000","last_unstale":"2024-09-21T03:44:58.456155+0000","last_undegraded":"2024-09-21T03:44:58.456155+0000","last_fullsized":"2024-09-21T03:44:58.456155+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:49:41.592146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T03:44:56.329848+0000","last_change":"2024-09-21T03:44:56.328517+0000","last_active":"2024-09-21T03:44:56.329848+0000","last_peered":"2024-09-21T03:44:56.329848+0000","last_clean":"2024-09-21T03:44:56.329848+0000","last_became_active":"2024-09-21T03:44:56.328048+0000","last_became_peered":"2024-09-21T03:44:56.328048+0000","last_unstale":"2024-09-21T03:44:56.329848+0000","last_undegraded":"2024-09-21T03:44:56.329848+0000","last_fullsized":"2024-09-21T03:44:56.329848+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_clean_scrub_stamp":"2024-09-21T03:44:55.304777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:24:58.009008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.337538+0000","last_change":"2024-09-21T03:44:57.359324+0000","last_active":"2024-09-21T03:44:58.337538+0000","last_peered":"2024-09-21T03:44:58.337538+0000","last_clean":"2024-09-21T03:44:58.337538+0000","last_became_active":"2024-09-21T03:44:57.358966+0000","last_became_peered":"2024-09-21T03:44:57.358966+0000","last_unstale":"2024-09-21T03:44:58.337538+0000","last_undegraded":"2024-09-21T03:44:58.337538+0000","last_fullsized":"2024-09-21T03:44:58.337538+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:30:55.516774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T03:44:56.338457+0000","last_change":"2024-09-21T03:44:56.336489+0000","last_active":"2024-09-21T03:44:56.338457+0000","last_peered":"2024-09-21T03:44:56.338457+0000","last_clean":"2024-09-21T03:44:56.338457+0000","last_became_active":"2024-09-21T03:44:56.336089+0000","last_became_peered":"2024-09-21T03:44:56.336089+0000","last_unstale":"2024-09-21T03:44:56.338457+0000","last_undegraded":"2024-09-21T03:44:56.338457+0000","last_fullsized":"2024-09-21T03:44:56.338457+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_clean_scrub_stamp":"2024-09-21T03:44:55.304777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:30:01.117090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":10,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T03:44:57.356840+0000","last_change":"2024-09-21T03:44:57.355765+0000","last_active":"2024-09-21T03:44:57.356840+0000","last_peered":"2024-09-21T03:44:57.356840+0000","last_clean":"2024-09-21T03:44:57.356840+0000","last_became_active":"2024-09-21T03:44:57.355383+0000","last_became_peered":"2024-09-21T03:44:57.355383+0000","last_unstale":"2024-09-21T03:44:57.356840+0000","last_undegraded":"2024-09-21T03:44:57.356840+0000","last_fullsized":"2024-09-21T03:44:57.356840+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:44:27.836492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":15,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.337802+0000","last_change":"2024-09-21T03:44:56.338420+0000","last_active":"2024-09-21T03:44:58.337802+0000","last_peered":"2024-09-21T03:44:58.337802+0000","last_clean":"2024-09-21T03:44:58.337802+0000","last_became_active":"2024-09-21T03:44:56.337865+0000","last_became_peered":"2024-09-21T03:44:56.337865+0000","last_unstale":"2024-09-21T03:44:58.337802+0000","last_undegraded":"2024-09-21T03:44:58.337802+0000","last_fullsized":"2024-09-21T03:44:58.337802+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_clean_scrub_stamp":"2024-09-21T03:44:55.304777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:40:26.986852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.339010+0000","last_change":"2024-09-21T03:44:57.337661+0000","last_active":"2024-09-21T03:44:58.339010+0000","last_peered":"2024-09-21T03:44:58.339010+0000","last_clean":"2024-09-21T03:44:58.339010+0000","last_became_active":"2024-09-21T03:44:57.337356+0000","last_became_peered":"2024-09-21T03:44:57.337356+0000","last_unstale":"2024-09-21T03:44:58.339010+0000","last_undegraded":"2024-09-21T03:44:58.339010+0000","last_fullsized":"2024-09-21T03:44:58.339010+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:19:49.944906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T03:44:57.338702+0000","last_change":"2024-09-21T03:44:56.337786+0000","last_active":"2024-09-21T03:44:57.338702+0000","last_peered":"2024-09-21T03:44:57.338702+0000","last_clean":"2024-09-21T03:44:57.338702+0000","last_became_active":"2024-09-21T03:44:56.337258+0000","last_became_peered":"2024-09-21T03:44:56.337258+0000","last_unstale":"2024-09-21T03:44:57.338702+0000","last_undegraded":"2024-09-21T03:44:57.338702+0000","last_fullsized":"2024-09-21T03:44:57.338702+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_clean_scrub_stamp":"2024-09-21T03:44:55.304777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:29:06.273027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":10,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T03:44:57.351689+0000","last_change":"2024-09-21T03:44:57.350221+0000","last_active":"2024-09-21T03:44:57.351689+0000","last_peered":"2024-09-21T03:44:57.351689+0000","last_clean":"2024-09-21T03:44:57.351689+0000","last_became_active":"2024-09-21T03:44:57.349707+0000","last_became_peered":"2024-09-21T03:44:57.349707+0000","last_unstale":"2024-09-21T03:44:57.351689+0000","last_undegraded":"2024-09-21T03:44:57.351689+0000","last_fullsized":"2024-09-21T03:44:57.351689+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:50:57.792589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":15,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.337614+0000","last_change":"2024-09-21T03:44:56.340358+0000","last_active":"2024-09-21T03:44:58.337614+0000","last_peered":"2024-09-21T03:44:58.337614+0000","last_clean":"2024-09-21T03:44:58.337614+0000","last_became_active":"2024-09-21T03:44:56.340044+0000","last_became_peered":"2024-09-21T03:44:56.340044+0000","last_unstale":"2024-09-21T03:44:58.337614+0000","last_undegraded":"2024-09-21T03:44:58.337614+0000","last_fullsized":"2024-09-21T03:44:58.337614+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_clean_scrub_stamp":"2024-09-21T03:44:55.304777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:50:56.785756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":10,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T03:44:57.350495+0000","last_change":"2024-09-21T03:44:57.348196+0000","last_active":"2024-09-21T03:44:57.350495+0000","last_peered":"2024-09-21T03:44:57.350495+0000","last_clean":"2024-09-21T03:44:57.350495+0000","last_became_active":"2024-09-21T03:44:57.347856+0000","last_became_peered":"2024-09-21T03:44:57.347856+0000","last_unstale":"2024-09-21T03:44:57.350495+0000","last_undegraded":"2024-09-21T03:44:57.350495+0000","last_fullsized":"2024-09-21T03:44:57.350495+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:32:59.029041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T03:44:56.338491+0000","last_change":"2024-09-21T03:44:56.337049+0000","last_active":"2024-09-21T03:44:56.338491+0000","last_peered":"2024-09-21T03:44:56.338491+0000","last_clean":"2024-09-21T03:44:56.338491+0000","last_became_active":"2024-09-21T03:44:56.336315+0000","last_became_peered":"2024-09-21T03:44:56.336315+0000","last_unstale":"2024-09-21T03:44:56.338491+0000","last_undegraded":"2024-09-21T03:44:56.338491+0000","last_fullsized":"2024-09-21T03:44:56.338491+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_clean_scrub_stamp":"2024-09-21T03:44:55.304777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:31:45.105527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.353229+0000","last_change":"2024-09-21T03:44:57.366623+0000","last_active":"2024-09-21T03:44:58.353229+0000","last_peered":"2024-09-21T03:44:58.353229+0000","last_clean":"2024-09-21T03:44:58.353229+0000","last_became_active":"2024-09-21T03:44:57.366298+0000","last_became_peered":"2024-09-21T03:44:57.366298+0000","last_unstale":"2024-09-21T03:44:58.353229+0000","last_undegraded":"2024-09-21T03:44:58.353229+0000","last_fullsized":"2024-09-21T03:44:58.353229+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:01:23.359578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":15,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.338658+0000","last_change":"2024-09-21T03:44:56.339692+0000","last_active":"2024-09-21T03:44:58.338658+0000","last_peered":"2024-09-21T03:44:58.338658+0000","last_clean":"2024-09-21T03:44:58.338658+0000","last_became_active":"2024-09-21T03:44:56.339376+0000","last_became_peered":"2024-09-21T03:44:56.339376+0000","last_unstale":"2024-09-21T03:44:58.338658+0000","last_undegraded":"2024-09-21T03:44:58.338658+0000","last_fullsized":"2024-09-21T03:44:58.338658+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_clean_scrub_stamp":"2024-09-21T03:44:55.304777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:17:10.658062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.915523+0000","last_change":"2024-09-21T03:44:57.356308+0000","last_active":"2024-09-21T03:44:58.915523+0000","last_peered":"2024-09-21T03:44:58.915523+0000","last_clean":"2024-09-21T03:44:58.915523+0000","last_became_active":"2024-09-21T03:44:57.355511+0000","last_became_peered":"2024-09-21T03:44:57.355511+0000","last_unstale":"2024-09-21T03:44:58.915523+0000","last_undegraded":"2024-09-21T03:44:58.915523+0000","last_fullsized":"2024-09-21T03:44:58.915523+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:08:21.639028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T03:44:56.342190+0000","last_change":"2024-09-21T03:44:56.341499+0000","last_active":"2024-09-21T03:44:56.342190+0000","last_peered":"2024-09-21T03:44:56.342190+0000","last_clean":"2024-09-21T03:44:56.342190+0000","last_became_active":"2024-09-21T03:44:56.341058+0000","last_became_peered":"2024-09-21T03:44:56.341058+0000","last_unstale":"2024-09-21T03:44:56.342190+0000","last_undegraded":"2024-09-21T03:44:56.342190+0000","last_fullsized":"2024-09-21T03:44:56.342190+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_clean_scrub_stamp":"2024-09-21T03:44:55.304777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:51:47.787796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.352897+0000","last_change":"2024-09-21T03:44:57.355188+0000","last_active":"2024-09-21T03:44:58.352897+0000","last_peered":"2024-09-21T03:44:58.352897+0000","last_clean":"2024-09-21T03:44:58.352897+0000","last_became_active":"2024-09-21T03:44:57.354477+0000","last_became_peered":"2024-09-21T03:44:57.354477+0000","last_unstale":"2024-09-21T03:44:58.352897+0000","last_undegraded":"2024-09-21T03:44:58.352897+0000","last_fullsized":"2024-09-21T03:44:58.352897+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:28:53.047166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"145'2","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T03:44:57.366695+0000","last_change":"2024-09-21T03:44:56.335259+0000","last_active":"2024-09-21T03:44:57.366695+0000","last_peered":"2024-09-21T03:44:57.366695+0000","last_clean":"2024-09-21T03:44:57.366695+0000","last_became_active":"2024-09-21T03:44:56.334868+0000","last_became_peered":"2024-09-21T03:44:56.334868+0000","last_unstale":"2024-09-21T03:44:57.366695+0000","last_undegraded":"2024-09-21T03:44:57.366695+0000","last_fullsized":"2024-09-21T03:44:57.366695+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_clean_scrub_stamp":"2024-09-21T03:44:55.304777+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:45:29.403030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":355,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.338537+0000","last_change":"2024-09-21T03:33:25.319776+0000","last_active":"2024-09-21T03:44:58.338537+0000","last_peered":"2024-09-21T03:44:58.338537+0000","last_clean":"2024-09-21T03:44:58.338537+0000","last_became_active":"2024-09-21T03:33:25.318399+0000","last_became_peered":"2024-09-21T03:33:25.318399+0000","last_unstale":"2024-09-21T03:44:58.338537+0000","last_undegraded":"2024-09-21T03:44:58.338537+0000","last_fullsized":"2024-09-21T03:44:58.338537+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:23.016672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:23.016672+0000","last_clean_scrub_stamp":"2024-09-21T03:33:23.016672+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:32:35.906114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":4696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":33,"ondisk_log_size":33,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":1,"up_from":16,"seq":68719476879,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41936,"kb_used_data":2428,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94329904,"statfs":{"total":96636764160,"available":96593821696,"internally_reserved":0,"allocated":2486272,"data_stored":1546129,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509583,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41356,"kb_used_data":1848,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330484,"statfs":{"total":96636764160,"available":96594415616,"internally_reserved":0,"allocated":1892352,"data_stored":956321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":40434123},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509582,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41364,"kb_used_data":1852,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330476,"statfs":{"total":96636764160,"available":96594407424,"internally_reserved":0,"allocated":1896448,"data_stored":957281,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509583,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37184,"kb_used_data":1836,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334656,"statfs":{"total":96636764160,"available":96598687744,"internally_reserved":0,"allocated":1880064,"data_stored":949089,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509582,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37180,"kb_used_data":1832,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334660,"statfs":{"total":96636764160,"available":96598691840,"internally_reserved":0,"allocated":1875968,"data_stored":949559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542286,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33028,"kb_used_data":1840,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338812,"statfs":{"total":96636764160,"available":96602943488,"internally_reserved":0,"allocated":1884160,"data_stored":950589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574990,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37744,"kb_used_data":2396,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334096,"statfs":{"total":96636764160,"available":96598114304,"internally_reserved":0,"allocated":2453504,"data_stored":1532178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542287,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37168,"kb_used_data":1816,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334672,"statfs":{"total":96636764160,"available":96598704128,"internally_reserved":0,"allocated":1859584,"data_stored":941810,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T03:45:02.178 INFO:tasks.ceph.ceph_manager.ceph:PG 16.3d is not active+clean 2024-09-21T03:45:02.178 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '16.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 144, 'state': 'creating+peering', 'last_fresh': '2024-09-21T03:44:56.319458+0000', 'last_change': '2024-09-21T03:44:56.319458+0000', 'last_active': '2024-09-21T03:44:56.311610+0000', 'last_peered': '2024-09-21T03:44:56.311610+0000', 'last_clean': '2024-09-21T03:44:56.311610+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T03:44:56.319458+0000', 'last_undegraded': '2024-09-21T03:44:56.319458+0000', 'last_fullsized': '2024-09-21T03:44:56.319458+0000', 'mapping_epoch': 144, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 144, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T03:44:56.311610+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T03:44:56.311610+0000', 'last_clean_scrub_stamp': '2024-09-21T03:44:56.311610+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-21T03:45:02.179 INFO:tasks.ceph.ceph_manager.ceph:PG 16.2f is not active+clean 2024-09-21T03:45:02.179 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '16.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 144, 'state': 'creating+peering', 'last_fresh': '2024-09-21T03:44:56.321953+0000', 'last_change': '2024-09-21T03:44:56.321953+0000', 'last_active': '2024-09-21T03:44:56.311610+0000', 'last_peered': '2024-09-21T03:44:56.311610+0000', 'last_clean': '2024-09-21T03:44:56.311610+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T03:44:56.321953+0000', 'last_undegraded': '2024-09-21T03:44:56.321953+0000', 'last_fullsized': '2024-09-21T03:44:56.321953+0000', 'mapping_epoch': 144, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 144, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T03:44:56.311610+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T03:44:56.311610+0000', 'last_clean_scrub_stamp': '2024-09-21T03:44:56.311610+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-21T03:45:02.179 INFO:tasks.ceph.ceph_manager.ceph:PG 16.2b is not active+clean 2024-09-21T03:45:02.179 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '16.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 144, 'state': 'creating+peering', 'last_fresh': '2024-09-21T03:44:56.322208+0000', 'last_change': '2024-09-21T03:44:56.322208+0000', 'last_active': '2024-09-21T03:44:56.311610+0000', 'last_peered': '2024-09-21T03:44:56.311610+0000', 'last_clean': '2024-09-21T03:44:56.311610+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T03:44:56.322208+0000', 'last_undegraded': '2024-09-21T03:44:56.322208+0000', 'last_fullsized': '2024-09-21T03:44:56.322208+0000', 'mapping_epoch': 144, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 144, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T03:44:56.311610+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T03:44:56.311610+0000', 'last_clean_scrub_stamp': '2024-09-21T03:44:56.311610+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-21T03:45:02.179 INFO:tasks.ceph.ceph_manager.ceph:PG 16.0 is not active+clean 2024-09-21T03:45:02.179 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '16.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 144, 'state': 'creating+peering', 'last_fresh': '2024-09-21T03:44:56.321750+0000', 'last_change': '2024-09-21T03:44:56.321750+0000', 'last_active': '2024-09-21T03:44:56.311610+0000', 'last_peered': '2024-09-21T03:44:56.311610+0000', 'last_clean': '2024-09-21T03:44:56.311610+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T03:44:56.321750+0000', 'last_undegraded': '2024-09-21T03:44:56.321750+0000', 'last_fullsized': '2024-09-21T03:44:56.321750+0000', 'mapping_epoch': 144, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 144, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T03:44:56.311610+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T03:44:56.311610+0000', 'last_clean_scrub_stamp': '2024-09-21T03:44:56.311610+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-21T03:45:02.179 INFO:tasks.ceph.ceph_manager.ceph:PG 16.f is not active+clean 2024-09-21T03:45:02.179 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '16.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 144, 'state': 'creating+peering', 'last_fresh': '2024-09-21T03:44:56.321369+0000', 'last_change': '2024-09-21T03:44:56.321369+0000', 'last_active': '2024-09-21T03:44:56.311610+0000', 'last_peered': '2024-09-21T03:44:56.311610+0000', 'last_clean': '2024-09-21T03:44:56.311610+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T03:44:56.321369+0000', 'last_undegraded': '2024-09-21T03:44:56.321369+0000', 'last_fullsized': '2024-09-21T03:44:56.321369+0000', 'mapping_epoch': 144, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 144, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T03:44:56.311610+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T03:44:56.311610+0000', 'last_clean_scrub_stamp': '2024-09-21T03:44:56.311610+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-21T03:45:02.180 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-21T03:45:02.279 DEBUG:teuthology.orchestra.run.smithi038:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T03:45:02.281 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T03:45:02.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:02.330+0000 7f1286f7b640 1 -- 172.21.15.38:0/814217627 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f126c003a30 msgr2=0x7f126c003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:02.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:02.330+0000 7f1286f7b640 1 --2- 172.21.15.38:0/814217627 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f126c003a30 0x7f126c003ea0 secure :-1 s=READY pgs=2369 cs=0 l=1 rev1=1 crypto rx=0x7f1270009b50 tx=0x7f126c005830 comp rx=0 tx=0).stop 2024-09-21T03:45:02.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:02.330+0000 7f1285518640 1 -- 172.21.15.38:0/814217627 reap_dead start 2024-09-21T03:45:02.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:02.330+0000 7f1286f7b640 1 -- 172.21.15.38:0/814217627 shutdown_connections 2024-09-21T03:45:02.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:02.330+0000 7f1286f7b640 1 -- 172.21.15.38:0/814217627 >> 172.21.15.38:0/814217627 conn(0x7f12800d4440 msgr2=0x7f12800d4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:02.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:02.330+0000 7f1286f7b640 1 -- 172.21.15.38:0/814217627 shutdown_connections 2024-09-21T03:45:02.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:02.330+0000 7f1286f7b640 1 -- 172.21.15.38:0/814217627 wait complete. 2024-09-21T03:45:02.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:02.332+0000 7f1286f7b640 1 Processor -- start 2024-09-21T03:45:02.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:02.332+0000 7f1286f7b640 1 -- start start 2024-09-21T03:45:02.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:02.333+0000 7f1286f7b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f126c003a30 0x7f12800cdb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:02.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:02.333+0000 7f1286f7b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f12800c7f70 0x7f1280158210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:02.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:02.333+0000 7f1286f7b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f128015dfa0 0x7f12801f8c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:02.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:02.333+0000 7f1286f7b640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f12800b50f0 con 0x7f12800c7f70 2024-09-21T03:45:02.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:02.333+0000 7f1286f7b640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f12800b4f70 con 0x7f128015dfa0 2024-09-21T03:45:02.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:02.333+0000 7f1286f7b640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f12800b5270 con 0x7f126c003a30 2024-09-21T03:45:02.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:02.333+0000 7f127ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f12800c7f70 0x7f1280158210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:02.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:02.333+0000 7f127ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f12800c7f70 0x7f1280158210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43498/0 (socket says 172.21.15.38:43498) 2024-09-21T03:45:02.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:02.333+0000 7f127ffff640 1 -- 172.21.15.38:0/1306064724 learned_addr learned my addr 172.21.15.38:0/1306064724 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:45:02.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:02.333+0000 7f1284d17640 1 --2- 172.21.15.38:0/1306064724 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f126c003a30 0x7f12800cdb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:02.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:02.333+0000 7f1285518640 1 --2- 172.21.15.38:0/1306064724 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f128015dfa0 0x7f12801f8c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:02.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:02.334+0000 7f127ffff640 1 -- 172.21.15.38:0/1306064724 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f126c003a30 msgr2=0x7f12800cdb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:02.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:02.334+0000 7f127ffff640 1 --2- 172.21.15.38:0/1306064724 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f126c003a30 0x7f12800cdb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:02.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:02.334+0000 7f127ffff640 1 -- 172.21.15.38:0/1306064724 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f128015dfa0 msgr2=0x7f12801f8c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:02.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:02.334+0000 7f127ffff640 1 --2- 172.21.15.38:0/1306064724 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f128015dfa0 0x7f12801f8c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:02.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:02.334+0000 7f127ffff640 1 -- 172.21.15.38:0/1306064724 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1280158750 con 0x7f12800c7f70 2024-09-21T03:45:02.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:02.334+0000 7f1285518640 1 --2- 172.21.15.38:0/1306064724 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f128015dfa0 0x7f12801f8c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:45:02.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:02.334+0000 7f127ffff640 1 --2- 172.21.15.38:0/1306064724 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f12800c7f70 0x7f1280158210 secure :-1 s=READY pgs=2370 cs=0 l=1 rev1=1 crypto rx=0x7f1268001480 tx=0x7f12680039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:02.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:02.334+0000 7f127d7fa640 1 -- 172.21.15.38:0/1306064724 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1268015070 con 0x7f12800c7f70 2024-09-21T03:45:02.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:02.334+0000 7f127d7fa640 1 -- 172.21.15.38:0/1306064724 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1268011900 con 0x7f12800c7f70 2024-09-21T03:45:02.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:02.334+0000 7f1286f7b640 1 -- 172.21.15.38:0/1306064724 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12800ce1d0 con 0x7f12800c7f70 2024-09-21T03:45:02.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:02.334+0000 7f127d7fa640 1 -- 172.21.15.38:0/1306064724 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1268011d50 con 0x7f12800c7f70 2024-09-21T03:45:02.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:02.334+0000 7f1286f7b640 1 -- 172.21.15.38:0/1306064724 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f128015afd0 con 0x7f12800c7f70 2024-09-21T03:45:02.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:02.335+0000 7f1286f7b640 1 -- 172.21.15.38:0/1306064724 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1280162380 con 0x7f12800c7f70 2024-09-21T03:45:02.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:02.336+0000 7f127d7fa640 1 -- 172.21.15.38:0/1306064724 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f1268010420 con 0x7f12800c7f70 2024-09-21T03:45:02.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:02.336+0000 7f127d7fa640 1 --2- 172.21.15.38:0/1306064724 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1260078bc0 0x7f126007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:02.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:02.336+0000 7f127d7fa640 1 -- 172.21.15.38:0/1306064724 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 6101+0+0 (secure 0 0 0) 0x7f126809bab0 con 0x7f12800c7f70 2024-09-21T03:45:02.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:02.337+0000 7f1284d17640 1 --2- 172.21.15.38:0/1306064724 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1260078bc0 0x7f126007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:02.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:02.337+0000 7f1284d17640 1 --2- 172.21.15.38:0/1306064724 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1260078bc0 0x7f126007b0d0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f1270035000 tx=0x7f127002d040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:02.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:02.341+0000 7f127d7fa640 1 -- 172.21.15.38:0/1306064724 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f1280162380 con 0x7f12800c7f70 2024-09-21T03:45:02.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:02.610+0000 7f1286f7b640 1 -- 172.21.15.38:0/1306064724 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f12800d2da0 con 0x7f12800c7f70 2024-09-21T03:45:02.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:02.610+0000 7f127d7fa640 1 -- 172.21.15.38:0/1306064724 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1145 (secure 0 0 0) 0x7f12680637b0 con 0x7f12800c7f70 2024-09-21T03:45:02.613 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:45:02.613 INFO:teuthology.orchestra.run.smithi038.stdout:{"fsid":"9d50d66f-656b-4fe2-8e3b-fe53b79c4d43","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":711,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":146,"num_osds":8,"num_up_osds":8,"osd_up_since":1726889604,"num_in_osds":8,"osd_in_since":1726889594,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":92},{"state_name":"creating+peering","count":5}],"num_pgs":97,"num_pools":3,"num_objects":21,"data_bytes":592778,"bytes_used":314327040,"bytes_avail":772779786240,"bytes_total":773094113280,"inactive_pgs_ratio":0.051546391099691391,"write_bytes_sec":2446,"read_op_per_sec":0,"write_op_per_sec":6},"fsmap":{"epoch":343,"btime":"2024-09-21T03:45:00:366373+0000","id":7,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":7,"rank":0,"name":"c","status":"up:active","gid":10719}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":54,"modified":"2024-09-21T03:44:33.164709+0000","services":{}},"progress_events":{}} 2024-09-21T03:45:02.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:02.613+0000 7f1286f7b640 1 -- 172.21.15.38:0/1306064724 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1260078bc0 msgr2=0x7f126007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:02.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:02.613+0000 7f1286f7b640 1 --2- 172.21.15.38:0/1306064724 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1260078bc0 0x7f126007b0d0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f1270035000 tx=0x7f127002d040 comp rx=0 tx=0).stop 2024-09-21T03:45:02.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:02.613+0000 7f1286f7b640 1 -- 172.21.15.38:0/1306064724 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f12800c7f70 msgr2=0x7f1280158210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:02.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:02.613+0000 7f1286f7b640 1 --2- 172.21.15.38:0/1306064724 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f12800c7f70 0x7f1280158210 secure :-1 s=READY pgs=2370 cs=0 l=1 rev1=1 crypto rx=0x7f1268001480 tx=0x7f12680039c0 comp rx=0 tx=0).stop 2024-09-21T03:45:02.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:02.613+0000 7f1286f7b640 1 -- 172.21.15.38:0/1306064724 shutdown_connections 2024-09-21T03:45:02.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:02.613+0000 7f1286f7b640 1 --2- 172.21.15.38:0/1306064724 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f128015dfa0 0x7f12801f8c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:02.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:02.613+0000 7f1286f7b640 1 --2- 172.21.15.38:0/1306064724 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f12800c7f70 0x7f1280158210 unknown :-1 s=CLOSED pgs=2370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:02.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:02.613+0000 7f1286f7b640 1 --2- 172.21.15.38:0/1306064724 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1260078bc0 0x7f126007b0d0 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:02.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:02.613+0000 7f1286f7b640 1 --2- 172.21.15.38:0/1306064724 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f126c003a30 0x7f12800cdb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:02.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:02.613+0000 7f1286f7b640 1 -- 172.21.15.38:0/1306064724 >> 172.21.15.38:0/1306064724 conn(0x7f12800d4440 msgr2=0x7f12800d0c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:02.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:02.613+0000 7f1286f7b640 1 -- 172.21.15.38:0/1306064724 shutdown_connections 2024-09-21T03:45:02.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:02.613+0000 7f1286f7b640 1 -- 172.21.15.38:0/1306064724 wait complete. 2024-09-21T03:45:02.626 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-21T03:45:02.627 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T03:45:02.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:02.741+0000 7f12c7ddf640 1 -- 172.21.15.38:0/2574901550 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f12ac005250 msgr2=0x7f12ac00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:02.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:02.741+0000 7f12c7ddf640 1 --2- 172.21.15.38:0/2574901550 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f12ac005250 0x7f12ac00b9c0 secure :-1 s=READY pgs=2372 cs=0 l=1 rev1=1 crypto rx=0x7f12b800c780 tx=0x7f12ac005830 comp rx=0 tx=0).stop 2024-09-21T03:45:02.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:02.741+0000 7f12c7ddf640 1 -- 172.21.15.38:0/2574901550 shutdown_connections 2024-09-21T03:45:02.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:02.741+0000 7f12c7ddf640 1 --2- 172.21.15.38:0/2574901550 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f12ac005250 0x7f12ac00b9c0 unknown :-1 s=CLOSED pgs=2372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:02.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:02.741+0000 7f12c7ddf640 1 -- 172.21.15.38:0/2574901550 >> 172.21.15.38:0/2574901550 conn(0x7f12c00d4600 msgr2=0x7f12c00d4a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:02.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:02.741+0000 7f12c7ddf640 1 -- 172.21.15.38:0/2574901550 shutdown_connections 2024-09-21T03:45:02.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:02.741+0000 7f12c7ddf640 1 -- 172.21.15.38:0/2574901550 wait complete. 2024-09-21T03:45:02.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:02.743+0000 7f12c7ddf640 1 Processor -- start 2024-09-21T03:45:02.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:02.743+0000 7f12c7ddf640 1 -- start start 2024-09-21T03:45:02.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:02.744+0000 7f12c7ddf640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f12ac005250 0x7f12c0160410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:02.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:02.744+0000 7f12c7ddf640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f12c0159bd0 0x7f12c0161ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:02.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:02.744+0000 7f12c7ddf640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f12c00c9530 0x7f12c00c9940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:02.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:02.744+0000 7f12c7ddf640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f12c0162f50 con 0x7f12c0159bd0 2024-09-21T03:45:02.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:02.745+0000 7f12c7ddf640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f12c0162dd0 con 0x7f12ac005250 2024-09-21T03:45:02.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:02.745+0000 7f12c7ddf640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f12c01635d0 con 0x7f12c00c9530 2024-09-21T03:45:02.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:02.745+0000 7f12c537a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f12c0159bd0 0x7f12c0161ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:02.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:02.745+0000 7f12c537a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f12c0159bd0 0x7f12c0161ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43516/0 (socket says 172.21.15.38:43516) 2024-09-21T03:45:02.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:02.745+0000 7f12c537a640 1 -- 172.21.15.38:0/3310073108 learned_addr learned my addr 172.21.15.38:0/3310073108 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:45:02.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:02.745+0000 7f12c5b7b640 1 --2- 172.21.15.38:0/3310073108 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f12ac005250 0x7f12c0160410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:02.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:02.745+0000 7f12c637c640 1 --2- 172.21.15.38:0/3310073108 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f12c00c9530 0x7f12c00c9940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:02.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:02.745+0000 7f12c537a640 1 -- 172.21.15.38:0/3310073108 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f12c00c9530 msgr2=0x7f12c00c9940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:02.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:02.745+0000 7f12c537a640 1 --2- 172.21.15.38:0/3310073108 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f12c00c9530 0x7f12c00c9940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:02.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:02.745+0000 7f12c537a640 1 -- 172.21.15.38:0/3310073108 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f12ac005250 msgr2=0x7f12c0160410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:02.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:02.745+0000 7f12c537a640 1 --2- 172.21.15.38:0/3310073108 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f12ac005250 0x7f12c0160410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:02.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:02.745+0000 7f12c537a640 1 -- 172.21.15.38:0/3310073108 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12c00c9e80 con 0x7f12c0159bd0 2024-09-21T03:45:02.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:02.745+0000 7f12c537a640 1 --2- 172.21.15.38:0/3310073108 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f12c0159bd0 0x7f12c0161ea0 secure :-1 s=READY pgs=2373 cs=0 l=1 rev1=1 crypto rx=0x7f12b00118a0 tx=0x7f12b0011ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:02.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:02.745+0000 7f12b67fc640 1 -- 172.21.15.38:0/3310073108 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12b0004460 con 0x7f12c0159bd0 2024-09-21T03:45:02.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:02.745+0000 7f12b67fc640 1 -- 172.21.15.38:0/3310073108 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f12b000f9a0 con 0x7f12c0159bd0 2024-09-21T03:45:02.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:02.745+0000 7f12c7ddf640 1 -- 172.21.15.38:0/3310073108 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12c01623e0 con 0x7f12c0159bd0 2024-09-21T03:45:02.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:02.745+0000 7f12b67fc640 1 -- 172.21.15.38:0/3310073108 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12b000fdf0 con 0x7f12c0159bd0 2024-09-21T03:45:02.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:02.745+0000 7f12c7ddf640 1 -- 172.21.15.38:0/3310073108 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12c00cbbf0 con 0x7f12c0159bd0 2024-09-21T03:45:02.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:02.746+0000 7f12c7ddf640 1 -- 172.21.15.38:0/3310073108 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12c00cf790 con 0x7f12c0159bd0 2024-09-21T03:45:02.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:02.747+0000 7f12b67fc640 1 -- 172.21.15.38:0/3310073108 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f12b00035f0 con 0x7f12c0159bd0 2024-09-21T03:45:02.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:02.748+0000 7f12b67fc640 1 --2- 172.21.15.38:0/3310073108 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f12a4078bc0 0x7f12a407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:02.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:02.748+0000 7f12b67fc640 1 -- 172.21.15.38:0/3310073108 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 6101+0+0 (secure 0 0 0) 0x7f12b009c300 con 0x7f12c0159bd0 2024-09-21T03:45:02.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:02.748+0000 7f12c5b7b640 1 --2- 172.21.15.38:0/3310073108 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f12a4078bc0 0x7f12a407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:02.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:02.749+0000 7f12c5b7b640 1 --2- 172.21.15.38:0/3310073108 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f12a4078bc0 0x7f12a407b0d0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f12bc006680 tx=0x7f12bc006b70 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:02.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:02.752+0000 7f12b67fc640 1 -- 172.21.15.38:0/3310073108 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f12c00c9e80 con 0x7f12c0159bd0 2024-09-21T03:45:02.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:02.939+0000 7f12c7ddf640 1 -- 172.21.15.38:0/3310073108 --> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f12c006e090 con 0x7f12a4078bc0 2024-09-21T03:45:02.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:02.943+0000 7f12b67fc640 1 -- 172.21.15.38:0/3310073108 <== mgr.4117 v2:172.21.15.195:6800/1631901344 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252055 (secure 0 0 0) 0x7f12c006e090 con 0x7f12a4078bc0 2024-09-21T03:45:02.948 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:45:02.949 INFO:teuthology.orchestra.run.smithi038.stderr:dumped all 2024-09-21T03:45:02.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:02.947+0000 7f12c7ddf640 1 -- 172.21.15.38:0/3310073108 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f12a4078bc0 msgr2=0x7f12a407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:02.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:02.947+0000 7f12c7ddf640 1 --2- 172.21.15.38:0/3310073108 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f12a4078bc0 0x7f12a407b0d0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f12bc006680 tx=0x7f12bc006b70 comp rx=0 tx=0).stop 2024-09-21T03:45:02.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:02.947+0000 7f12c7ddf640 1 -- 172.21.15.38:0/3310073108 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f12c0159bd0 msgr2=0x7f12c0161ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:02.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:02.947+0000 7f12c7ddf640 1 --2- 172.21.15.38:0/3310073108 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f12c0159bd0 0x7f12c0161ea0 secure :-1 s=READY pgs=2373 cs=0 l=1 rev1=1 crypto rx=0x7f12b00118a0 tx=0x7f12b0011ca0 comp rx=0 tx=0).stop 2024-09-21T03:45:02.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:02.947+0000 7f12c7ddf640 1 -- 172.21.15.38:0/3310073108 shutdown_connections 2024-09-21T03:45:02.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:02.947+0000 7f12c7ddf640 1 --2- 172.21.15.38:0/3310073108 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f12a4078bc0 0x7f12a407b0d0 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:02.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:02.947+0000 7f12c7ddf640 1 --2- 172.21.15.38:0/3310073108 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f12c00c9530 0x7f12c00c9940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:02.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:02.947+0000 7f12c7ddf640 1 --2- 172.21.15.38:0/3310073108 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f12c0159bd0 0x7f12c0161ea0 unknown :-1 s=CLOSED pgs=2373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:02.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:02.947+0000 7f12c7ddf640 1 --2- 172.21.15.38:0/3310073108 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f12ac005250 0x7f12c0160410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:02.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:02.947+0000 7f12c7ddf640 1 -- 172.21.15.38:0/3310073108 >> 172.21.15.38:0/3310073108 conn(0x7f12c00d4600 msgr2=0x7f12c015cb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:02.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:02.947+0000 7f12c7ddf640 1 -- 172.21.15.38:0/3310073108 shutdown_connections 2024-09-21T03:45:02.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:02.947+0000 7f12c7ddf640 1 -- 172.21.15.38:0/3310073108 wait complete. 2024-09-21T03:45:02.960 INFO:teuthology.orchestra.run.smithi038.stdout:{"pg_ready":true,"pg_map":{"version":488,"stamp":"2024-09-21T03:45:01.172110+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":141,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":105,"ondisk_log_size":105,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":118,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":306960,"kb_used_data":15848,"kb_used_omap":12,"kb_used_meta":290931,"kb_avail":754667760,"statfs":{"total":773094113280,"available":772779786240,"internally_reserved":0,"allocated":16228352,"data_stored":8782956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12748,"internal_metadata":297913908},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.859588"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":10,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T03:44:57.348031+0000","last_change":"2024-09-21T03:44:57.340688+0000","last_active":"2024-09-21T03:44:57.348031+0000","last_peered":"2024-09-21T03:44:57.348031+0000","last_clean":"2024-09-21T03:44:57.348031+0000","last_became_active":"2024-09-21T03:44:57.340356+0000","last_became_peered":"2024-09-21T03:44:57.340356+0000","last_unstale":"2024-09-21T03:44:57.348031+0000","last_undegraded":"2024-09-21T03:44:57.348031+0000","last_fullsized":"2024-09-21T03:44:57.348031+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:31:55.663249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.915466+0000","last_change":"2024-09-21T03:44:57.354834+0000","last_active":"2024-09-21T03:44:58.915466+0000","last_peered":"2024-09-21T03:44:58.915466+0000","last_clean":"2024-09-21T03:44:58.915466+0000","last_became_active":"2024-09-21T03:44:57.354518+0000","last_became_peered":"2024-09-21T03:44:57.354518+0000","last_unstale":"2024-09-21T03:44:58.915466+0000","last_undegraded":"2024-09-21T03:44:58.915466+0000","last_fullsized":"2024-09-21T03:44:58.915466+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:21:26.621000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":1,"reported_epoch":144,"state":"creating+peering","last_fresh":"2024-09-21T03:44:56.319458+0000","last_change":"2024-09-21T03:44:56.319458+0000","last_active":"2024-09-21T03:44:56.311610+0000","last_peered":"2024-09-21T03:44:56.311610+0000","last_clean":"2024-09-21T03:44:56.311610+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:44:56.319458+0000","last_undegraded":"2024-09-21T03:44:56.319458+0000","last_fullsized":"2024-09-21T03:44:56.319458+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":10,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T03:44:57.359756+0000","last_change":"2024-09-21T03:44:57.356466+0000","last_active":"2024-09-21T03:44:57.359756+0000","last_peered":"2024-09-21T03:44:57.359756+0000","last_clean":"2024-09-21T03:44:57.359756+0000","last_became_active":"2024-09-21T03:44:57.355938+0000","last_became_peered":"2024-09-21T03:44:57.355938+0000","last_unstale":"2024-09-21T03:44:57.359756+0000","last_undegraded":"2024-09-21T03:44:57.359756+0000","last_fullsized":"2024-09-21T03:44:57.359756+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:05:36.290619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":10,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T03:44:57.353943+0000","last_change":"2024-09-21T03:44:57.351972+0000","last_active":"2024-09-21T03:44:57.353943+0000","last_peered":"2024-09-21T03:44:57.353943+0000","last_clean":"2024-09-21T03:44:57.353943+0000","last_became_active":"2024-09-21T03:44:57.351561+0000","last_became_peered":"2024-09-21T03:44:57.351561+0000","last_unstale":"2024-09-21T03:44:57.353943+0000","last_undegraded":"2024-09-21T03:44:57.353943+0000","last_fullsized":"2024-09-21T03:44:57.353943+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:21:51.370818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.353120+0000","last_change":"2024-09-21T03:44:57.349848+0000","last_active":"2024-09-21T03:44:58.353120+0000","last_peered":"2024-09-21T03:44:58.353120+0000","last_clean":"2024-09-21T03:44:58.353120+0000","last_became_active":"2024-09-21T03:44:57.349387+0000","last_became_peered":"2024-09-21T03:44:57.349387+0000","last_unstale":"2024-09-21T03:44:58.353120+0000","last_undegraded":"2024-09-21T03:44:58.353120+0000","last_fullsized":"2024-09-21T03:44:58.353120+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:10:55.772054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.352606+0000","last_change":"2024-09-21T03:44:57.355533+0000","last_active":"2024-09-21T03:44:58.352606+0000","last_peered":"2024-09-21T03:44:58.352606+0000","last_clean":"2024-09-21T03:44:58.352606+0000","last_became_active":"2024-09-21T03:44:57.354892+0000","last_became_peered":"2024-09-21T03:44:57.354892+0000","last_unstale":"2024-09-21T03:44:58.352606+0000","last_undegraded":"2024-09-21T03:44:58.352606+0000","last_fullsized":"2024-09-21T03:44:58.352606+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:05:41.952462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":10,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T03:44:57.356024+0000","last_change":"2024-09-21T03:44:57.350864+0000","last_active":"2024-09-21T03:44:57.356024+0000","last_peered":"2024-09-21T03:44:57.356024+0000","last_clean":"2024-09-21T03:44:57.356024+0000","last_became_active":"2024-09-21T03:44:57.350215+0000","last_became_peered":"2024-09-21T03:44:57.350215+0000","last_unstale":"2024-09-21T03:44:57.356024+0000","last_undegraded":"2024-09-21T03:44:57.356024+0000","last_fullsized":"2024-09-21T03:44:57.356024+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:23:12.061449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.915427+0000","last_change":"2024-09-21T03:44:57.356640+0000","last_active":"2024-09-21T03:44:58.915427+0000","last_peered":"2024-09-21T03:44:58.915427+0000","last_clean":"2024-09-21T03:44:58.915427+0000","last_became_active":"2024-09-21T03:44:57.355966+0000","last_became_peered":"2024-09-21T03:44:57.355966+0000","last_unstale":"2024-09-21T03:44:58.915427+0000","last_undegraded":"2024-09-21T03:44:58.915427+0000","last_fullsized":"2024-09-21T03:44:58.915427+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:40:49.216619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":10,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T03:44:57.359889+0000","last_change":"2024-09-21T03:44:57.359320+0000","last_active":"2024-09-21T03:44:57.359889+0000","last_peered":"2024-09-21T03:44:57.359889+0000","last_clean":"2024-09-21T03:44:57.359889+0000","last_became_active":"2024-09-21T03:44:57.358988+0000","last_became_peered":"2024-09-21T03:44:57.358988+0000","last_unstale":"2024-09-21T03:44:57.359889+0000","last_undegraded":"2024-09-21T03:44:57.359889+0000","last_fullsized":"2024-09-21T03:44:57.359889+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:29:11.643073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.337467+0000","last_change":"2024-09-21T03:44:57.362260+0000","last_active":"2024-09-21T03:44:58.337467+0000","last_peered":"2024-09-21T03:44:58.337467+0000","last_clean":"2024-09-21T03:44:58.337467+0000","last_became_active":"2024-09-21T03:44:57.361624+0000","last_became_peered":"2024-09-21T03:44:57.361624+0000","last_unstale":"2024-09-21T03:44:58.337467+0000","last_undegraded":"2024-09-21T03:44:58.337467+0000","last_fullsized":"2024-09-21T03:44:58.337467+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:00:23.014227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.352884+0000","last_change":"2024-09-21T03:44:57.353828+0000","last_active":"2024-09-21T03:44:58.352884+0000","last_peered":"2024-09-21T03:44:58.352884+0000","last_clean":"2024-09-21T03:44:58.352884+0000","last_became_active":"2024-09-21T03:44:57.353516+0000","last_became_peered":"2024-09-21T03:44:57.353516+0000","last_unstale":"2024-09-21T03:44:58.352884+0000","last_undegraded":"2024-09-21T03:44:58.352884+0000","last_fullsized":"2024-09-21T03:44:58.352884+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:49:05.536169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":10,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T03:44:57.366752+0000","last_change":"2024-09-21T03:44:57.363696+0000","last_active":"2024-09-21T03:44:57.366752+0000","last_peered":"2024-09-21T03:44:57.366752+0000","last_clean":"2024-09-21T03:44:57.366752+0000","last_became_active":"2024-09-21T03:44:57.362737+0000","last_became_peered":"2024-09-21T03:44:57.362737+0000","last_unstale":"2024-09-21T03:44:57.366752+0000","last_undegraded":"2024-09-21T03:44:57.366752+0000","last_fullsized":"2024-09-21T03:44:57.366752+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:58:38.157904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.353349+0000","last_change":"2024-09-21T03:44:57.352690+0000","last_active":"2024-09-21T03:44:58.353349+0000","last_peered":"2024-09-21T03:44:58.353349+0000","last_clean":"2024-09-21T03:44:58.353349+0000","last_became_active":"2024-09-21T03:44:57.352358+0000","last_became_peered":"2024-09-21T03:44:57.352358+0000","last_unstale":"2024-09-21T03:44:58.353349+0000","last_undegraded":"2024-09-21T03:44:58.353349+0000","last_fullsized":"2024-09-21T03:44:58.353349+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:42:27.812988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.915095+0000","last_change":"2024-09-21T03:44:57.369797+0000","last_active":"2024-09-21T03:44:58.915095+0000","last_peered":"2024-09-21T03:44:58.915095+0000","last_clean":"2024-09-21T03:44:58.915095+0000","last_became_active":"2024-09-21T03:44:57.369463+0000","last_became_peered":"2024-09-21T03:44:57.369463+0000","last_unstale":"2024-09-21T03:44:58.915095+0000","last_undegraded":"2024-09-21T03:44:58.915095+0000","last_fullsized":"2024-09-21T03:44:58.915095+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:59:36.159782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.338697+0000","last_change":"2024-09-21T03:44:57.341545+0000","last_active":"2024-09-21T03:44:58.338697+0000","last_peered":"2024-09-21T03:44:58.338697+0000","last_clean":"2024-09-21T03:44:58.338697+0000","last_became_active":"2024-09-21T03:44:57.341220+0000","last_became_peered":"2024-09-21T03:44:57.341220+0000","last_unstale":"2024-09-21T03:44:58.338697+0000","last_undegraded":"2024-09-21T03:44:58.338697+0000","last_fullsized":"2024-09-21T03:44:58.338697+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:40:08.834183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":1,"reported_epoch":144,"state":"creating+peering","last_fresh":"2024-09-21T03:44:56.321953+0000","last_change":"2024-09-21T03:44:56.321953+0000","last_active":"2024-09-21T03:44:56.311610+0000","last_peered":"2024-09-21T03:44:56.311610+0000","last_clean":"2024-09-21T03:44:56.311610+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:44:56.321953+0000","last_undegraded":"2024-09-21T03:44:56.321953+0000","last_fullsized":"2024-09-21T03:44:56.321953+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.915438+0000","last_change":"2024-09-21T03:44:57.357272+0000","last_active":"2024-09-21T03:44:58.915438+0000","last_peered":"2024-09-21T03:44:58.915438+0000","last_clean":"2024-09-21T03:44:58.915438+0000","last_became_active":"2024-09-21T03:44:57.356952+0000","last_became_peered":"2024-09-21T03:44:57.356952+0000","last_unstale":"2024-09-21T03:44:58.915438+0000","last_undegraded":"2024-09-21T03:44:58.915438+0000","last_fullsized":"2024-09-21T03:44:58.915438+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:35:23.462964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":10,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T03:44:57.354219+0000","last_change":"2024-09-21T03:44:57.351765+0000","last_active":"2024-09-21T03:44:57.354219+0000","last_peered":"2024-09-21T03:44:57.354219+0000","last_clean":"2024-09-21T03:44:57.354219+0000","last_became_active":"2024-09-21T03:44:57.351478+0000","last_became_peered":"2024-09-21T03:44:57.351478+0000","last_unstale":"2024-09-21T03:44:57.354219+0000","last_undegraded":"2024-09-21T03:44:57.354219+0000","last_fullsized":"2024-09-21T03:44:57.354219+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:00:35.318345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.339047+0000","last_change":"2024-09-21T03:44:57.358796+0000","last_active":"2024-09-21T03:44:58.339047+0000","last_peered":"2024-09-21T03:44:58.339047+0000","last_clean":"2024-09-21T03:44:58.339047+0000","last_became_active":"2024-09-21T03:44:57.358439+0000","last_became_peered":"2024-09-21T03:44:57.358439+0000","last_unstale":"2024-09-21T03:44:58.339047+0000","last_undegraded":"2024-09-21T03:44:58.339047+0000","last_fullsized":"2024-09-21T03:44:58.339047+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:05:51.280587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":1,"reported_epoch":144,"state":"creating+peering","last_fresh":"2024-09-21T03:44:56.322208+0000","last_change":"2024-09-21T03:44:56.322208+0000","last_active":"2024-09-21T03:44:56.311610+0000","last_peered":"2024-09-21T03:44:56.311610+0000","last_clean":"2024-09-21T03:44:56.311610+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:44:56.322208+0000","last_undegraded":"2024-09-21T03:44:56.322208+0000","last_fullsized":"2024-09-21T03:44:56.322208+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.456326+0000","last_change":"2024-09-21T03:44:57.357854+0000","last_active":"2024-09-21T03:44:58.456326+0000","last_peered":"2024-09-21T03:44:58.456326+0000","last_clean":"2024-09-21T03:44:58.456326+0000","last_became_active":"2024-09-21T03:44:57.357541+0000","last_became_peered":"2024-09-21T03:44:57.357541+0000","last_unstale":"2024-09-21T03:44:58.456326+0000","last_undegraded":"2024-09-21T03:44:58.456326+0000","last_fullsized":"2024-09-21T03:44:58.456326+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:04:52.143962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":10,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T03:44:57.355917+0000","last_change":"2024-09-21T03:44:57.353700+0000","last_active":"2024-09-21T03:44:57.355917+0000","last_peered":"2024-09-21T03:44:57.355917+0000","last_clean":"2024-09-21T03:44:57.355917+0000","last_became_active":"2024-09-21T03:44:57.351501+0000","last_became_peered":"2024-09-21T03:44:57.351501+0000","last_unstale":"2024-09-21T03:44:57.355917+0000","last_undegraded":"2024-09-21T03:44:57.355917+0000","last_fullsized":"2024-09-21T03:44:57.355917+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:00:30.971449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":10,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T03:44:57.356048+0000","last_change":"2024-09-21T03:44:57.351638+0000","last_active":"2024-09-21T03:44:57.356048+0000","last_peered":"2024-09-21T03:44:57.356048+0000","last_clean":"2024-09-21T03:44:57.356048+0000","last_became_active":"2024-09-21T03:44:57.351325+0000","last_became_peered":"2024-09-21T03:44:57.351325+0000","last_unstale":"2024-09-21T03:44:57.356048+0000","last_undegraded":"2024-09-21T03:44:57.356048+0000","last_fullsized":"2024-09-21T03:44:57.356048+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:05:57.831029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.339252+0000","last_change":"2024-09-21T03:44:57.359418+0000","last_active":"2024-09-21T03:44:58.339252+0000","last_peered":"2024-09-21T03:44:58.339252+0000","last_clean":"2024-09-21T03:44:58.339252+0000","last_became_active":"2024-09-21T03:44:57.358564+0000","last_became_peered":"2024-09-21T03:44:57.358564+0000","last_unstale":"2024-09-21T03:44:58.339252+0000","last_undegraded":"2024-09-21T03:44:58.339252+0000","last_fullsized":"2024-09-21T03:44:58.339252+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:21:26.978675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.339318+0000","last_change":"2024-09-21T03:44:57.360155+0000","last_active":"2024-09-21T03:44:58.339318+0000","last_peered":"2024-09-21T03:44:58.339318+0000","last_clean":"2024-09-21T03:44:58.339318+0000","last_became_active":"2024-09-21T03:44:57.359645+0000","last_became_peered":"2024-09-21T03:44:57.359645+0000","last_unstale":"2024-09-21T03:44:58.339318+0000","last_undegraded":"2024-09-21T03:44:58.339318+0000","last_fullsized":"2024-09-21T03:44:58.339318+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:14:53.602909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.339127+0000","last_change":"2024-09-21T03:44:57.357568+0000","last_active":"2024-09-21T03:44:58.339127+0000","last_peered":"2024-09-21T03:44:58.339127+0000","last_clean":"2024-09-21T03:44:58.339127+0000","last_became_active":"2024-09-21T03:44:57.357185+0000","last_became_peered":"2024-09-21T03:44:57.357185+0000","last_unstale":"2024-09-21T03:44:58.339127+0000","last_undegraded":"2024-09-21T03:44:58.339127+0000","last_fullsized":"2024-09-21T03:44:58.339127+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:57:24.936155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.353110+0000","last_change":"2024-09-21T03:44:57.352231+0000","last_active":"2024-09-21T03:44:58.353110+0000","last_peered":"2024-09-21T03:44:58.353110+0000","last_clean":"2024-09-21T03:44:58.353110+0000","last_became_active":"2024-09-21T03:44:57.351887+0000","last_became_peered":"2024-09-21T03:44:57.351887+0000","last_unstale":"2024-09-21T03:44:58.353110+0000","last_undegraded":"2024-09-21T03:44:58.353110+0000","last_fullsized":"2024-09-21T03:44:58.353110+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:46:07.069681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":10,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T03:44:57.358271+0000","last_change":"2024-09-21T03:44:57.357470+0000","last_active":"2024-09-21T03:44:57.358271+0000","last_peered":"2024-09-21T03:44:57.358271+0000","last_clean":"2024-09-21T03:44:57.358271+0000","last_became_active":"2024-09-21T03:44:57.357091+0000","last_became_peered":"2024-09-21T03:44:57.357091+0000","last_unstale":"2024-09-21T03:44:57.358271+0000","last_undegraded":"2024-09-21T03:44:57.358271+0000","last_fullsized":"2024-09-21T03:44:57.358271+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:13:25.917453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.337478+0000","last_change":"2024-09-21T03:44:57.361022+0000","last_active":"2024-09-21T03:44:58.337478+0000","last_peered":"2024-09-21T03:44:58.337478+0000","last_clean":"2024-09-21T03:44:58.337478+0000","last_became_active":"2024-09-21T03:44:57.360697+0000","last_became_peered":"2024-09-21T03:44:57.360697+0000","last_unstale":"2024-09-21T03:44:58.337478+0000","last_undegraded":"2024-09-21T03:44:58.337478+0000","last_fullsized":"2024-09-21T03:44:58.337478+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:17:17.662702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.352611+0000","last_change":"2024-09-21T03:44:57.357841+0000","last_active":"2024-09-21T03:44:58.352611+0000","last_peered":"2024-09-21T03:44:58.352611+0000","last_clean":"2024-09-21T03:44:58.352611+0000","last_became_active":"2024-09-21T03:44:57.357299+0000","last_became_peered":"2024-09-21T03:44:57.357299+0000","last_unstale":"2024-09-21T03:44:58.352611+0000","last_undegraded":"2024-09-21T03:44:58.352611+0000","last_fullsized":"2024-09-21T03:44:58.352611+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:36:45.265239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.456162+0000","last_change":"2024-09-21T03:44:57.366423+0000","last_active":"2024-09-21T03:44:58.456162+0000","last_peered":"2024-09-21T03:44:58.456162+0000","last_clean":"2024-09-21T03:44:58.456162+0000","last_became_active":"2024-09-21T03:44:57.366083+0000","last_became_peered":"2024-09-21T03:44:57.366083+0000","last_unstale":"2024-09-21T03:44:58.456162+0000","last_undegraded":"2024-09-21T03:44:58.456162+0000","last_fullsized":"2024-09-21T03:44:58.456162+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:14:54.627243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":1,"reported_epoch":144,"state":"creating+peering","last_fresh":"2024-09-21T03:44:56.321750+0000","last_change":"2024-09-21T03:44:56.321750+0000","last_active":"2024-09-21T03:44:56.311610+0000","last_peered":"2024-09-21T03:44:56.311610+0000","last_clean":"2024-09-21T03:44:56.311610+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:44:56.321750+0000","last_undegraded":"2024-09-21T03:44:56.321750+0000","last_fullsized":"2024-09-21T03:44:56.321750+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1f","version":"145'4","reported_seq":19,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.456694+0000","last_change":"2024-09-21T03:44:56.332485+0000","last_active":"2024-09-21T03:44:58.456694+0000","last_peered":"2024-09-21T03:44:58.456694+0000","last_clean":"2024-09-21T03:44:58.456694+0000","last_became_active":"2024-09-21T03:44:56.332204+0000","last_became_peered":"2024-09-21T03:44:56.332204+0000","last_unstale":"2024-09-21T03:44:58.456694+0000","last_undegraded":"2024-09-21T03:44:58.456694+0000","last_fullsized":"2024-09-21T03:44:58.456694+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_clean_scrub_stamp":"2024-09-21T03:44:55.304777+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:15:02.688269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":10,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T03:44:57.356612+0000","last_change":"2024-09-21T03:44:57.352443+0000","last_active":"2024-09-21T03:44:57.356612+0000","last_peered":"2024-09-21T03:44:57.356612+0000","last_clean":"2024-09-21T03:44:57.356612+0000","last_became_active":"2024-09-21T03:44:57.351086+0000","last_became_peered":"2024-09-21T03:44:57.351086+0000","last_unstale":"2024-09-21T03:44:57.356612+0000","last_undegraded":"2024-09-21T03:44:57.356612+0000","last_fullsized":"2024-09-21T03:44:57.356612+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:27:41.239844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T03:44:56.342206+0000","last_change":"2024-09-21T03:44:56.341615+0000","last_active":"2024-09-21T03:44:56.342206+0000","last_peered":"2024-09-21T03:44:56.342206+0000","last_clean":"2024-09-21T03:44:56.342206+0000","last_became_active":"2024-09-21T03:44:56.341306+0000","last_became_peered":"2024-09-21T03:44:56.341306+0000","last_unstale":"2024-09-21T03:44:56.342206+0000","last_undegraded":"2024-09-21T03:44:56.342206+0000","last_fullsized":"2024-09-21T03:44:56.342206+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_clean_scrub_stamp":"2024-09-21T03:44:55.304777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:36:07.439425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.915135+0000","last_change":"2024-09-21T03:44:57.349398+0000","last_active":"2024-09-21T03:44:58.915135+0000","last_peered":"2024-09-21T03:44:58.915135+0000","last_clean":"2024-09-21T03:44:58.915135+0000","last_became_active":"2024-09-21T03:44:57.349067+0000","last_became_peered":"2024-09-21T03:44:57.349067+0000","last_unstale":"2024-09-21T03:44:58.915135+0000","last_undegraded":"2024-09-21T03:44:58.915135+0000","last_fullsized":"2024-09-21T03:44:58.915135+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:19:45.952024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"145'2","reported_seq":17,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.915396+0000","last_change":"2024-09-21T03:44:56.343325+0000","last_active":"2024-09-21T03:44:58.915396+0000","last_peered":"2024-09-21T03:44:58.915396+0000","last_clean":"2024-09-21T03:44:58.915396+0000","last_became_active":"2024-09-21T03:44:56.342964+0000","last_became_peered":"2024-09-21T03:44:56.342964+0000","last_unstale":"2024-09-21T03:44:58.915396+0000","last_undegraded":"2024-09-21T03:44:58.915396+0000","last_fullsized":"2024-09-21T03:44:58.915396+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_clean_scrub_stamp":"2024-09-21T03:44:55.304777+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:19:36.404105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.337488+0000","last_change":"2024-09-21T03:44:57.360138+0000","last_active":"2024-09-21T03:44:58.337488+0000","last_peered":"2024-09-21T03:44:58.337488+0000","last_clean":"2024-09-21T03:44:58.337488+0000","last_became_active":"2024-09-21T03:44:57.359800+0000","last_became_peered":"2024-09-21T03:44:57.359800+0000","last_unstale":"2024-09-21T03:44:58.337488+0000","last_undegraded":"2024-09-21T03:44:58.337488+0000","last_fullsized":"2024-09-21T03:44:58.337488+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:33:24.015377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1c","version":"145'2","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T03:44:57.373298+0000","last_change":"2024-09-21T03:44:56.334534+0000","last_active":"2024-09-21T03:44:57.373298+0000","last_peered":"2024-09-21T03:44:57.373298+0000","last_clean":"2024-09-21T03:44:57.373298+0000","last_became_active":"2024-09-21T03:44:56.333800+0000","last_became_peered":"2024-09-21T03:44:56.333800+0000","last_unstale":"2024-09-21T03:44:57.373298+0000","last_undegraded":"2024-09-21T03:44:57.373298+0000","last_fullsized":"2024-09-21T03:44:57.373298+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_clean_scrub_stamp":"2024-09-21T03:44:55.304777+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:44:41.072083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":10,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T03:44:57.350445+0000","last_change":"2024-09-21T03:44:57.342301+0000","last_active":"2024-09-21T03:44:57.350445+0000","last_peered":"2024-09-21T03:44:57.350445+0000","last_clean":"2024-09-21T03:44:57.350445+0000","last_became_active":"2024-09-21T03:44:57.341880+0000","last_became_peered":"2024-09-21T03:44:57.341880+0000","last_unstale":"2024-09-21T03:44:57.350445+0000","last_undegraded":"2024-09-21T03:44:57.350445+0000","last_fullsized":"2024-09-21T03:44:57.350445+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:10:55.772054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1b","version":"145'1","reported_seq":16,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.337514+0000","last_change":"2024-09-21T03:44:56.342732+0000","last_active":"2024-09-21T03:44:58.337514+0000","last_peered":"2024-09-21T03:44:58.337514+0000","last_clean":"2024-09-21T03:44:58.337514+0000","last_became_active":"2024-09-21T03:44:56.342319+0000","last_became_peered":"2024-09-21T03:44:56.342319+0000","last_unstale":"2024-09-21T03:44:58.337514+0000","last_undegraded":"2024-09-21T03:44:58.337514+0000","last_fullsized":"2024-09-21T03:44:58.337514+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_clean_scrub_stamp":"2024-09-21T03:44:55.304777+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:31:32.978494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":10,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T03:44:57.350445+0000","last_change":"2024-09-21T03:44:57.346978+0000","last_active":"2024-09-21T03:44:57.350445+0000","last_peered":"2024-09-21T03:44:57.350445+0000","last_clean":"2024-09-21T03:44:57.350445+0000","last_became_active":"2024-09-21T03:44:57.346633+0000","last_became_peered":"2024-09-21T03:44:57.346633+0000","last_unstale":"2024-09-21T03:44:57.350445+0000","last_undegraded":"2024-09-21T03:44:57.350445+0000","last_fullsized":"2024-09-21T03:44:57.350445+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:17:57.312760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1a","version":"145'1","reported_seq":16,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.338782+0000","last_change":"2024-09-21T03:44:56.342500+0000","last_active":"2024-09-21T03:44:58.338782+0000","last_peered":"2024-09-21T03:44:58.338782+0000","last_clean":"2024-09-21T03:44:58.338782+0000","last_became_active":"2024-09-21T03:44:56.342198+0000","last_became_peered":"2024-09-21T03:44:56.342198+0000","last_unstale":"2024-09-21T03:44:58.338782+0000","last_undegraded":"2024-09-21T03:44:58.338782+0000","last_fullsized":"2024-09-21T03:44:58.338782+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_clean_scrub_stamp":"2024-09-21T03:44:55.304777+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:24:33.377335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.337947+0000","last_change":"2024-09-21T03:44:57.360511+0000","last_active":"2024-09-21T03:44:58.337947+0000","last_peered":"2024-09-21T03:44:58.337947+0000","last_clean":"2024-09-21T03:44:58.337947+0000","last_became_active":"2024-09-21T03:44:57.360163+0000","last_became_peered":"2024-09-21T03:44:57.360163+0000","last_unstale":"2024-09-21T03:44:58.337947+0000","last_undegraded":"2024-09-21T03:44:58.337947+0000","last_fullsized":"2024-09-21T03:44:58.337947+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:05:31.071185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":15,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.338632+0000","last_change":"2024-09-21T03:44:56.343444+0000","last_active":"2024-09-21T03:44:58.338632+0000","last_peered":"2024-09-21T03:44:58.338632+0000","last_clean":"2024-09-21T03:44:58.338632+0000","last_became_active":"2024-09-21T03:44:56.343128+0000","last_became_peered":"2024-09-21T03:44:56.343128+0000","last_unstale":"2024-09-21T03:44:58.338632+0000","last_undegraded":"2024-09-21T03:44:58.338632+0000","last_fullsized":"2024-09-21T03:44:58.338632+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_clean_scrub_stamp":"2024-09-21T03:44:55.304777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:49:27.666317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":10,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T03:44:57.351262+0000","last_change":"2024-09-21T03:44:57.348168+0000","last_active":"2024-09-21T03:44:57.351262+0000","last_peered":"2024-09-21T03:44:57.351262+0000","last_clean":"2024-09-21T03:44:57.351262+0000","last_became_active":"2024-09-21T03:44:57.347823+0000","last_became_peered":"2024-09-21T03:44:57.347823+0000","last_unstale":"2024-09-21T03:44:57.351262+0000","last_undegraded":"2024-09-21T03:44:57.351262+0000","last_fullsized":"2024-09-21T03:44:57.351262+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:43:22.624675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":15,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.456487+0000","last_change":"2024-09-21T03:44:56.337896+0000","last_active":"2024-09-21T03:44:58.456487+0000","last_peered":"2024-09-21T03:44:58.456487+0000","last_clean":"2024-09-21T03:44:58.456487+0000","last_became_active":"2024-09-21T03:44:56.337298+0000","last_became_peered":"2024-09-21T03:44:56.337298+0000","last_unstale":"2024-09-21T03:44:58.456487+0000","last_undegraded":"2024-09-21T03:44:58.456487+0000","last_fullsized":"2024-09-21T03:44:58.456487+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_clean_scrub_stamp":"2024-09-21T03:44:55.304777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:58:50.496950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":10,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T03:44:57.351652+0000","last_change":"2024-09-21T03:44:57.350094+0000","last_active":"2024-09-21T03:44:57.351652+0000","last_peered":"2024-09-21T03:44:57.351652+0000","last_clean":"2024-09-21T03:44:57.351652+0000","last_became_active":"2024-09-21T03:44:57.349777+0000","last_became_peered":"2024-09-21T03:44:57.349777+0000","last_unstale":"2024-09-21T03:44:57.351652+0000","last_undegraded":"2024-09-21T03:44:57.351652+0000","last_fullsized":"2024-09-21T03:44:57.351652+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:36:06.178611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":15,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.353239+0000","last_change":"2024-09-21T03:44:56.329230+0000","last_active":"2024-09-21T03:44:58.353239+0000","last_peered":"2024-09-21T03:44:58.353239+0000","last_clean":"2024-09-21T03:44:58.353239+0000","last_became_active":"2024-09-21T03:44:56.328911+0000","last_became_peered":"2024-09-21T03:44:56.328911+0000","last_unstale":"2024-09-21T03:44:58.353239+0000","last_undegraded":"2024-09-21T03:44:58.353239+0000","last_fullsized":"2024-09-21T03:44:58.353239+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_clean_scrub_stamp":"2024-09-21T03:44:55.304777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:16:45.547857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":10,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T03:44:57.351808+0000","last_change":"2024-09-21T03:44:57.347015+0000","last_active":"2024-09-21T03:44:57.351808+0000","last_peered":"2024-09-21T03:44:57.351808+0000","last_clean":"2024-09-21T03:44:57.351808+0000","last_became_active":"2024-09-21T03:44:57.346668+0000","last_became_peered":"2024-09-21T03:44:57.346668+0000","last_unstale":"2024-09-21T03:44:57.351808+0000","last_undegraded":"2024-09-21T03:44:57.351808+0000","last_fullsized":"2024-09-21T03:44:57.351808+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:49:55.140933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T03:44:57.341754+0000","last_change":"2024-09-21T03:44:56.333386+0000","last_active":"2024-09-21T03:44:57.341754+0000","last_peered":"2024-09-21T03:44:57.341754+0000","last_clean":"2024-09-21T03:44:57.341754+0000","last_became_active":"2024-09-21T03:44:56.333082+0000","last_became_peered":"2024-09-21T03:44:56.333082+0000","last_unstale":"2024-09-21T03:44:57.341754+0000","last_undegraded":"2024-09-21T03:44:57.341754+0000","last_fullsized":"2024-09-21T03:44:57.341754+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_clean_scrub_stamp":"2024-09-21T03:44:55.304777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:10:24.715196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":10,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T03:44:57.355956+0000","last_change":"2024-09-21T03:44:57.350041+0000","last_active":"2024-09-21T03:44:57.355956+0000","last_peered":"2024-09-21T03:44:57.355956+0000","last_clean":"2024-09-21T03:44:57.355956+0000","last_became_active":"2024-09-21T03:44:57.349706+0000","last_became_peered":"2024-09-21T03:44:57.349706+0000","last_unstale":"2024-09-21T03:44:57.355956+0000","last_undegraded":"2024-09-21T03:44:57.355956+0000","last_fullsized":"2024-09-21T03:44:57.355956+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:35:23.899135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.15","version":"145'3","reported_seq":18,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.352972+0000","last_change":"2024-09-21T03:44:56.324998+0000","last_active":"2024-09-21T03:44:58.352972+0000","last_peered":"2024-09-21T03:44:58.352972+0000","last_clean":"2024-09-21T03:44:58.352972+0000","last_became_active":"2024-09-21T03:44:56.324464+0000","last_became_peered":"2024-09-21T03:44:56.324464+0000","last_unstale":"2024-09-21T03:44:58.352972+0000","last_undegraded":"2024-09-21T03:44:58.352972+0000","last_fullsized":"2024-09-21T03:44:58.352972+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_clean_scrub_stamp":"2024-09-21T03:44:55.304777+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:27:45.431858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.456464+0000","last_change":"2024-09-21T03:44:57.355113+0000","last_active":"2024-09-21T03:44:58.456464+0000","last_peered":"2024-09-21T03:44:58.456464+0000","last_clean":"2024-09-21T03:44:58.456464+0000","last_became_active":"2024-09-21T03:44:57.354744+0000","last_became_peered":"2024-09-21T03:44:57.354744+0000","last_unstale":"2024-09-21T03:44:58.456464+0000","last_undegraded":"2024-09-21T03:44:58.456464+0000","last_fullsized":"2024-09-21T03:44:58.456464+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:54:24.418739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.14","version":"145'6","reported_seq":21,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.337583+0000","last_change":"2024-09-21T03:44:56.341013+0000","last_active":"2024-09-21T03:44:58.337583+0000","last_peered":"2024-09-21T03:44:58.337583+0000","last_clean":"2024-09-21T03:44:58.337583+0000","last_became_active":"2024-09-21T03:44:56.337878+0000","last_became_peered":"2024-09-21T03:44:56.337878+0000","last_unstale":"2024-09-21T03:44:58.337583+0000","last_undegraded":"2024-09-21T03:44:58.337583+0000","last_fullsized":"2024-09-21T03:44:58.337583+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_clean_scrub_stamp":"2024-09-21T03:44:55.304777+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:00:34.311511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.915365+0000","last_change":"2024-09-21T03:44:57.356176+0000","last_active":"2024-09-21T03:44:58.915365+0000","last_peered":"2024-09-21T03:44:58.915365+0000","last_clean":"2024-09-21T03:44:58.915365+0000","last_became_active":"2024-09-21T03:44:57.355832+0000","last_became_peered":"2024-09-21T03:44:57.355832+0000","last_unstale":"2024-09-21T03:44:58.915365+0000","last_undegraded":"2024-09-21T03:44:58.915365+0000","last_fullsized":"2024-09-21T03:44:58.915365+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:47:30.592382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.13","version":"145'2","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T03:44:57.367038+0000","last_change":"2024-09-21T03:44:56.327633+0000","last_active":"2024-09-21T03:44:57.367038+0000","last_peered":"2024-09-21T03:44:57.367038+0000","last_clean":"2024-09-21T03:44:57.367038+0000","last_became_active":"2024-09-21T03:44:56.327313+0000","last_became_peered":"2024-09-21T03:44:56.327313+0000","last_unstale":"2024-09-21T03:44:57.367038+0000","last_undegraded":"2024-09-21T03:44:57.367038+0000","last_fullsized":"2024-09-21T03:44:57.367038+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_clean_scrub_stamp":"2024-09-21T03:44:55.304777+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:36:44.258406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.915449+0000","last_change":"2024-09-21T03:44:57.352611+0000","last_active":"2024-09-21T03:44:58.915449+0000","last_peered":"2024-09-21T03:44:58.915449+0000","last_clean":"2024-09-21T03:44:58.915449+0000","last_became_active":"2024-09-21T03:44:57.352136+0000","last_became_peered":"2024-09-21T03:44:57.352136+0000","last_unstale":"2024-09-21T03:44:58.915449+0000","last_undegraded":"2024-09-21T03:44:58.915449+0000","last_fullsized":"2024-09-21T03:44:58.915449+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:26:06.882445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T03:44:57.341635+0000","last_change":"2024-09-21T03:44:56.337958+0000","last_active":"2024-09-21T03:44:57.341635+0000","last_peered":"2024-09-21T03:44:57.341635+0000","last_clean":"2024-09-21T03:44:57.341635+0000","last_became_active":"2024-09-21T03:44:56.337623+0000","last_became_peered":"2024-09-21T03:44:56.337623+0000","last_unstale":"2024-09-21T03:44:57.341635+0000","last_undegraded":"2024-09-21T03:44:57.341635+0000","last_fullsized":"2024-09-21T03:44:57.341635+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_clean_scrub_stamp":"2024-09-21T03:44:55.304777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:45:31.964595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.456704+0000","last_change":"2024-09-21T03:44:57.358895+0000","last_active":"2024-09-21T03:44:58.456704+0000","last_peered":"2024-09-21T03:44:58.456704+0000","last_clean":"2024-09-21T03:44:58.456704+0000","last_became_active":"2024-09-21T03:44:57.358591+0000","last_became_peered":"2024-09-21T03:44:57.358591+0000","last_unstale":"2024-09-21T03:44:58.456704+0000","last_undegraded":"2024-09-21T03:44:58.456704+0000","last_fullsized":"2024-09-21T03:44:58.456704+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:38:18.158452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T03:44:57.337893+0000","last_change":"2024-09-21T03:44:56.327693+0000","last_active":"2024-09-21T03:44:57.337893+0000","last_peered":"2024-09-21T03:44:57.337893+0000","last_clean":"2024-09-21T03:44:57.337893+0000","last_became_active":"2024-09-21T03:44:56.327364+0000","last_became_peered":"2024-09-21T03:44:56.327364+0000","last_unstale":"2024-09-21T03:44:57.337893+0000","last_undegraded":"2024-09-21T03:44:57.337893+0000","last_fullsized":"2024-09-21T03:44:57.337893+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_clean_scrub_stamp":"2024-09-21T03:44:55.304777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:16:04.551826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":1,"reported_epoch":144,"state":"creating+peering","last_fresh":"2024-09-21T03:44:56.321369+0000","last_change":"2024-09-21T03:44:56.321369+0000","last_active":"2024-09-21T03:44:56.311610+0000","last_peered":"2024-09-21T03:44:56.311610+0000","last_clean":"2024-09-21T03:44:56.311610+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T03:44:56.321369+0000","last_undegraded":"2024-09-21T03:44:56.321369+0000","last_fullsized":"2024-09-21T03:44:56.321369+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.10","version":"145'8","reported_seq":23,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.337672+0000","last_change":"2024-09-21T03:44:56.342213+0000","last_active":"2024-09-21T03:44:58.337672+0000","last_peered":"2024-09-21T03:44:58.337672+0000","last_clean":"2024-09-21T03:44:58.337672+0000","last_became_active":"2024-09-21T03:44:56.341888+0000","last_became_peered":"2024-09-21T03:44:56.341888+0000","last_unstale":"2024-09-21T03:44:58.337672+0000","last_undegraded":"2024-09-21T03:44:58.337672+0000","last_fullsized":"2024-09-21T03:44:58.337672+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_clean_scrub_stamp":"2024-09-21T03:44:55.304777+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:36:05.171778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":10,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T03:44:57.355999+0000","last_change":"2024-09-21T03:44:57.350536+0000","last_active":"2024-09-21T03:44:57.355999+0000","last_peered":"2024-09-21T03:44:57.355999+0000","last_clean":"2024-09-21T03:44:57.355999+0000","last_became_active":"2024-09-21T03:44:57.349795+0000","last_became_peered":"2024-09-21T03:44:57.349795+0000","last_unstale":"2024-09-21T03:44:57.355999+0000","last_undegraded":"2024-09-21T03:44:57.355999+0000","last_fullsized":"2024-09-21T03:44:57.355999+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:09:26.811858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.f","version":"145'1","reported_seq":16,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.353274+0000","last_change":"2024-09-21T03:44:56.339856+0000","last_active":"2024-09-21T03:44:58.353274+0000","last_peered":"2024-09-21T03:44:58.353274+0000","last_clean":"2024-09-21T03:44:58.353274+0000","last_became_active":"2024-09-21T03:44:56.339305+0000","last_became_peered":"2024-09-21T03:44:56.339305+0000","last_unstale":"2024-09-21T03:44:58.353274+0000","last_undegraded":"2024-09-21T03:44:58.353274+0000","last_fullsized":"2024-09-21T03:44:58.353274+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_clean_scrub_stamp":"2024-09-21T03:44:55.304777+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:37:25.372042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.338656+0000","last_change":"2024-09-21T03:44:57.341004+0000","last_active":"2024-09-21T03:44:58.338656+0000","last_peered":"2024-09-21T03:44:58.338656+0000","last_clean":"2024-09-21T03:44:58.338656+0000","last_became_active":"2024-09-21T03:44:57.340620+0000","last_became_peered":"2024-09-21T03:44:57.340620+0000","last_unstale":"2024-09-21T03:44:58.338656+0000","last_undegraded":"2024-09-21T03:44:58.338656+0000","last_fullsized":"2024-09-21T03:44:58.338656+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:09:01.018690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.d","version":"145'1","reported_seq":16,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.339122+0000","last_change":"2024-09-21T03:44:56.340547+0000","last_active":"2024-09-21T03:44:58.339122+0000","last_peered":"2024-09-21T03:44:58.339122+0000","last_clean":"2024-09-21T03:44:58.339122+0000","last_became_active":"2024-09-21T03:44:56.340237+0000","last_became_peered":"2024-09-21T03:44:56.340237+0000","last_unstale":"2024-09-21T03:44:58.339122+0000","last_undegraded":"2024-09-21T03:44:58.339122+0000","last_fullsized":"2024-09-21T03:44:58.339122+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_clean_scrub_stamp":"2024-09-21T03:44:55.304777+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:19:41.082645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.456321+0000","last_change":"2024-09-21T03:44:57.358052+0000","last_active":"2024-09-21T03:44:58.456321+0000","last_peered":"2024-09-21T03:44:58.456321+0000","last_clean":"2024-09-21T03:44:58.456321+0000","last_became_active":"2024-09-21T03:44:57.357731+0000","last_became_peered":"2024-09-21T03:44:57.357731+0000","last_unstale":"2024-09-21T03:44:58.456321+0000","last_undegraded":"2024-09-21T03:44:58.456321+0000","last_fullsized":"2024-09-21T03:44:58.456321+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:02:38.958342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":15,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.353016+0000","last_change":"2024-09-21T03:44:56.339649+0000","last_active":"2024-09-21T03:44:58.353016+0000","last_peered":"2024-09-21T03:44:58.353016+0000","last_clean":"2024-09-21T03:44:58.353016+0000","last_became_active":"2024-09-21T03:44:56.339317+0000","last_became_peered":"2024-09-21T03:44:56.339317+0000","last_unstale":"2024-09-21T03:44:58.353016+0000","last_undegraded":"2024-09-21T03:44:58.353016+0000","last_fullsized":"2024-09-21T03:44:58.353016+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_clean_scrub_stamp":"2024-09-21T03:44:55.304777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:58:28.485819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":10,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T03:44:57.353966+0000","last_change":"2024-09-21T03:44:57.350478+0000","last_active":"2024-09-21T03:44:57.353966+0000","last_peered":"2024-09-21T03:44:57.353966+0000","last_clean":"2024-09-21T03:44:57.353966+0000","last_became_active":"2024-09-21T03:44:57.348990+0000","last_became_peered":"2024-09-21T03:44:57.348990+0000","last_unstale":"2024-09-21T03:44:57.353966+0000","last_undegraded":"2024-09-21T03:44:57.353966+0000","last_fullsized":"2024-09-21T03:44:57.353966+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:40:27.993685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T03:44:56.330403+0000","last_change":"2024-09-21T03:44:56.329417+0000","last_active":"2024-09-21T03:44:56.330403+0000","last_peered":"2024-09-21T03:44:56.330403+0000","last_clean":"2024-09-21T03:44:56.330403+0000","last_became_active":"2024-09-21T03:44:56.329098+0000","last_became_peered":"2024-09-21T03:44:56.329098+0000","last_unstale":"2024-09-21T03:44:56.330403+0000","last_undegraded":"2024-09-21T03:44:56.330403+0000","last_fullsized":"2024-09-21T03:44:56.330403+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_clean_scrub_stamp":"2024-09-21T03:44:55.304777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:53:45.362390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.456620+0000","last_change":"2024-09-21T03:44:57.354679+0000","last_active":"2024-09-21T03:44:58.456620+0000","last_peered":"2024-09-21T03:44:58.456620+0000","last_clean":"2024-09-21T03:44:58.456620+0000","last_became_active":"2024-09-21T03:44:57.354357+0000","last_became_peered":"2024-09-21T03:44:57.354357+0000","last_unstale":"2024-09-21T03:44:58.456620+0000","last_undegraded":"2024-09-21T03:44:58.456620+0000","last_fullsized":"2024-09-21T03:44:58.456620+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:29:19.509171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T03:44:56.331506+0000","last_change":"2024-09-21T03:44:56.330054+0000","last_active":"2024-09-21T03:44:56.331506+0000","last_peered":"2024-09-21T03:44:56.331506+0000","last_clean":"2024-09-21T03:44:56.331506+0000","last_became_active":"2024-09-21T03:44:56.329151+0000","last_became_peered":"2024-09-21T03:44:56.329151+0000","last_unstale":"2024-09-21T03:44:56.331506+0000","last_undegraded":"2024-09-21T03:44:56.331506+0000","last_fullsized":"2024-09-21T03:44:56.331506+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_clean_scrub_stamp":"2024-09-21T03:44:55.304777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:15:21.096050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.456652+0000","last_change":"2024-09-21T03:44:57.363634+0000","last_active":"2024-09-21T03:44:58.456652+0000","last_peered":"2024-09-21T03:44:58.456652+0000","last_clean":"2024-09-21T03:44:58.456652+0000","last_became_active":"2024-09-21T03:44:57.363259+0000","last_became_peered":"2024-09-21T03:44:57.363259+0000","last_unstale":"2024-09-21T03:44:58.456652+0000","last_undegraded":"2024-09-21T03:44:58.456652+0000","last_fullsized":"2024-09-21T03:44:58.456652+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:53:39.529389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":15,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.353119+0000","last_change":"2024-09-21T03:44:56.327686+0000","last_active":"2024-09-21T03:44:58.353119+0000","last_peered":"2024-09-21T03:44:58.353119+0000","last_clean":"2024-09-21T03:44:58.353119+0000","last_became_active":"2024-09-21T03:44:56.327355+0000","last_became_peered":"2024-09-21T03:44:56.327355+0000","last_unstale":"2024-09-21T03:44:58.353119+0000","last_undegraded":"2024-09-21T03:44:58.353119+0000","last_fullsized":"2024-09-21T03:44:58.353119+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_clean_scrub_stamp":"2024-09-21T03:44:55.304777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:26:58.029665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":10,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T03:44:57.352767+0000","last_change":"2024-09-21T03:44:57.351050+0000","last_active":"2024-09-21T03:44:57.352767+0000","last_peered":"2024-09-21T03:44:57.352767+0000","last_clean":"2024-09-21T03:44:57.352767+0000","last_became_active":"2024-09-21T03:44:57.350731+0000","last_became_peered":"2024-09-21T03:44:57.350731+0000","last_unstale":"2024-09-21T03:44:57.352767+0000","last_undegraded":"2024-09-21T03:44:57.352767+0000","last_fullsized":"2024-09-21T03:44:57.352767+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:31:33.985328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T03:44:57.340435+0000","last_change":"2024-09-21T03:44:56.334506+0000","last_active":"2024-09-21T03:44:57.340435+0000","last_peered":"2024-09-21T03:44:57.340435+0000","last_clean":"2024-09-21T03:44:57.340435+0000","last_became_active":"2024-09-21T03:44:56.334007+0000","last_became_peered":"2024-09-21T03:44:56.334007+0000","last_unstale":"2024-09-21T03:44:57.340435+0000","last_undegraded":"2024-09-21T03:44:57.340435+0000","last_fullsized":"2024-09-21T03:44:57.340435+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_clean_scrub_stamp":"2024-09-21T03:44:55.304777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:57:15.575727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.456155+0000","last_change":"2024-09-21T03:44:57.350289+0000","last_active":"2024-09-21T03:44:58.456155+0000","last_peered":"2024-09-21T03:44:58.456155+0000","last_clean":"2024-09-21T03:44:58.456155+0000","last_became_active":"2024-09-21T03:44:57.349932+0000","last_became_peered":"2024-09-21T03:44:57.349932+0000","last_unstale":"2024-09-21T03:44:58.456155+0000","last_undegraded":"2024-09-21T03:44:58.456155+0000","last_fullsized":"2024-09-21T03:44:58.456155+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:49:41.592146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T03:44:56.329848+0000","last_change":"2024-09-21T03:44:56.328517+0000","last_active":"2024-09-21T03:44:56.329848+0000","last_peered":"2024-09-21T03:44:56.329848+0000","last_clean":"2024-09-21T03:44:56.329848+0000","last_became_active":"2024-09-21T03:44:56.328048+0000","last_became_peered":"2024-09-21T03:44:56.328048+0000","last_unstale":"2024-09-21T03:44:56.329848+0000","last_undegraded":"2024-09-21T03:44:56.329848+0000","last_fullsized":"2024-09-21T03:44:56.329848+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_clean_scrub_stamp":"2024-09-21T03:44:55.304777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:24:58.009008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.337538+0000","last_change":"2024-09-21T03:44:57.359324+0000","last_active":"2024-09-21T03:44:58.337538+0000","last_peered":"2024-09-21T03:44:58.337538+0000","last_clean":"2024-09-21T03:44:58.337538+0000","last_became_active":"2024-09-21T03:44:57.358966+0000","last_became_peered":"2024-09-21T03:44:57.358966+0000","last_unstale":"2024-09-21T03:44:58.337538+0000","last_undegraded":"2024-09-21T03:44:58.337538+0000","last_fullsized":"2024-09-21T03:44:58.337538+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:30:55.516774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T03:44:56.338457+0000","last_change":"2024-09-21T03:44:56.336489+0000","last_active":"2024-09-21T03:44:56.338457+0000","last_peered":"2024-09-21T03:44:56.338457+0000","last_clean":"2024-09-21T03:44:56.338457+0000","last_became_active":"2024-09-21T03:44:56.336089+0000","last_became_peered":"2024-09-21T03:44:56.336089+0000","last_unstale":"2024-09-21T03:44:56.338457+0000","last_undegraded":"2024-09-21T03:44:56.338457+0000","last_fullsized":"2024-09-21T03:44:56.338457+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_clean_scrub_stamp":"2024-09-21T03:44:55.304777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:30:01.117090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":10,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T03:44:57.356840+0000","last_change":"2024-09-21T03:44:57.355765+0000","last_active":"2024-09-21T03:44:57.356840+0000","last_peered":"2024-09-21T03:44:57.356840+0000","last_clean":"2024-09-21T03:44:57.356840+0000","last_became_active":"2024-09-21T03:44:57.355383+0000","last_became_peered":"2024-09-21T03:44:57.355383+0000","last_unstale":"2024-09-21T03:44:57.356840+0000","last_undegraded":"2024-09-21T03:44:57.356840+0000","last_fullsized":"2024-09-21T03:44:57.356840+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:44:27.836492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":15,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.337802+0000","last_change":"2024-09-21T03:44:56.338420+0000","last_active":"2024-09-21T03:44:58.337802+0000","last_peered":"2024-09-21T03:44:58.337802+0000","last_clean":"2024-09-21T03:44:58.337802+0000","last_became_active":"2024-09-21T03:44:56.337865+0000","last_became_peered":"2024-09-21T03:44:56.337865+0000","last_unstale":"2024-09-21T03:44:58.337802+0000","last_undegraded":"2024-09-21T03:44:58.337802+0000","last_fullsized":"2024-09-21T03:44:58.337802+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_clean_scrub_stamp":"2024-09-21T03:44:55.304777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:40:26.986852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.339010+0000","last_change":"2024-09-21T03:44:57.337661+0000","last_active":"2024-09-21T03:44:58.339010+0000","last_peered":"2024-09-21T03:44:58.339010+0000","last_clean":"2024-09-21T03:44:58.339010+0000","last_became_active":"2024-09-21T03:44:57.337356+0000","last_became_peered":"2024-09-21T03:44:57.337356+0000","last_unstale":"2024-09-21T03:44:58.339010+0000","last_undegraded":"2024-09-21T03:44:58.339010+0000","last_fullsized":"2024-09-21T03:44:58.339010+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:19:49.944906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T03:44:57.338702+0000","last_change":"2024-09-21T03:44:56.337786+0000","last_active":"2024-09-21T03:44:57.338702+0000","last_peered":"2024-09-21T03:44:57.338702+0000","last_clean":"2024-09-21T03:44:57.338702+0000","last_became_active":"2024-09-21T03:44:56.337258+0000","last_became_peered":"2024-09-21T03:44:56.337258+0000","last_unstale":"2024-09-21T03:44:57.338702+0000","last_undegraded":"2024-09-21T03:44:57.338702+0000","last_fullsized":"2024-09-21T03:44:57.338702+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_clean_scrub_stamp":"2024-09-21T03:44:55.304777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:29:06.273027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":10,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T03:44:57.351689+0000","last_change":"2024-09-21T03:44:57.350221+0000","last_active":"2024-09-21T03:44:57.351689+0000","last_peered":"2024-09-21T03:44:57.351689+0000","last_clean":"2024-09-21T03:44:57.351689+0000","last_became_active":"2024-09-21T03:44:57.349707+0000","last_became_peered":"2024-09-21T03:44:57.349707+0000","last_unstale":"2024-09-21T03:44:57.351689+0000","last_undegraded":"2024-09-21T03:44:57.351689+0000","last_fullsized":"2024-09-21T03:44:57.351689+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:50:57.792589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":15,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.337614+0000","last_change":"2024-09-21T03:44:56.340358+0000","last_active":"2024-09-21T03:44:58.337614+0000","last_peered":"2024-09-21T03:44:58.337614+0000","last_clean":"2024-09-21T03:44:58.337614+0000","last_became_active":"2024-09-21T03:44:56.340044+0000","last_became_peered":"2024-09-21T03:44:56.340044+0000","last_unstale":"2024-09-21T03:44:58.337614+0000","last_undegraded":"2024-09-21T03:44:58.337614+0000","last_fullsized":"2024-09-21T03:44:58.337614+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_clean_scrub_stamp":"2024-09-21T03:44:55.304777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:50:56.785756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":10,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T03:44:57.350495+0000","last_change":"2024-09-21T03:44:57.348196+0000","last_active":"2024-09-21T03:44:57.350495+0000","last_peered":"2024-09-21T03:44:57.350495+0000","last_clean":"2024-09-21T03:44:57.350495+0000","last_became_active":"2024-09-21T03:44:57.347856+0000","last_became_peered":"2024-09-21T03:44:57.347856+0000","last_unstale":"2024-09-21T03:44:57.350495+0000","last_undegraded":"2024-09-21T03:44:57.350495+0000","last_fullsized":"2024-09-21T03:44:57.350495+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:32:59.029041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T03:44:56.338491+0000","last_change":"2024-09-21T03:44:56.337049+0000","last_active":"2024-09-21T03:44:56.338491+0000","last_peered":"2024-09-21T03:44:56.338491+0000","last_clean":"2024-09-21T03:44:56.338491+0000","last_became_active":"2024-09-21T03:44:56.336315+0000","last_became_peered":"2024-09-21T03:44:56.336315+0000","last_unstale":"2024-09-21T03:44:56.338491+0000","last_undegraded":"2024-09-21T03:44:56.338491+0000","last_fullsized":"2024-09-21T03:44:56.338491+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_clean_scrub_stamp":"2024-09-21T03:44:55.304777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:31:45.105527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.353229+0000","last_change":"2024-09-21T03:44:57.366623+0000","last_active":"2024-09-21T03:44:58.353229+0000","last_peered":"2024-09-21T03:44:58.353229+0000","last_clean":"2024-09-21T03:44:58.353229+0000","last_became_active":"2024-09-21T03:44:57.366298+0000","last_became_peered":"2024-09-21T03:44:57.366298+0000","last_unstale":"2024-09-21T03:44:58.353229+0000","last_undegraded":"2024-09-21T03:44:58.353229+0000","last_fullsized":"2024-09-21T03:44:58.353229+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:01:23.359578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":15,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.338658+0000","last_change":"2024-09-21T03:44:56.339692+0000","last_active":"2024-09-21T03:44:58.338658+0000","last_peered":"2024-09-21T03:44:58.338658+0000","last_clean":"2024-09-21T03:44:58.338658+0000","last_became_active":"2024-09-21T03:44:56.339376+0000","last_became_peered":"2024-09-21T03:44:56.339376+0000","last_unstale":"2024-09-21T03:44:58.338658+0000","last_undegraded":"2024-09-21T03:44:58.338658+0000","last_fullsized":"2024-09-21T03:44:58.338658+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_clean_scrub_stamp":"2024-09-21T03:44:55.304777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:17:10.658062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.915523+0000","last_change":"2024-09-21T03:44:57.356308+0000","last_active":"2024-09-21T03:44:58.915523+0000","last_peered":"2024-09-21T03:44:58.915523+0000","last_clean":"2024-09-21T03:44:58.915523+0000","last_became_active":"2024-09-21T03:44:57.355511+0000","last_became_peered":"2024-09-21T03:44:57.355511+0000","last_unstale":"2024-09-21T03:44:58.915523+0000","last_undegraded":"2024-09-21T03:44:58.915523+0000","last_fullsized":"2024-09-21T03:44:58.915523+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:08:21.639028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T03:44:56.342190+0000","last_change":"2024-09-21T03:44:56.341499+0000","last_active":"2024-09-21T03:44:56.342190+0000","last_peered":"2024-09-21T03:44:56.342190+0000","last_clean":"2024-09-21T03:44:56.342190+0000","last_became_active":"2024-09-21T03:44:56.341058+0000","last_became_peered":"2024-09-21T03:44:56.341058+0000","last_unstale":"2024-09-21T03:44:56.342190+0000","last_undegraded":"2024-09-21T03:44:56.342190+0000","last_fullsized":"2024-09-21T03:44:56.342190+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_clean_scrub_stamp":"2024-09-21T03:44:55.304777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:51:47.787796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.352897+0000","last_change":"2024-09-21T03:44:57.355188+0000","last_active":"2024-09-21T03:44:58.352897+0000","last_peered":"2024-09-21T03:44:58.352897+0000","last_clean":"2024-09-21T03:44:58.352897+0000","last_became_active":"2024-09-21T03:44:57.354477+0000","last_became_peered":"2024-09-21T03:44:57.354477+0000","last_unstale":"2024-09-21T03:44:58.352897+0000","last_undegraded":"2024-09-21T03:44:58.352897+0000","last_fullsized":"2024-09-21T03:44:58.352897+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:28:53.047166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"145'2","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T03:44:57.366695+0000","last_change":"2024-09-21T03:44:56.335259+0000","last_active":"2024-09-21T03:44:57.366695+0000","last_peered":"2024-09-21T03:44:57.366695+0000","last_clean":"2024-09-21T03:44:57.366695+0000","last_became_active":"2024-09-21T03:44:56.334868+0000","last_became_peered":"2024-09-21T03:44:56.334868+0000","last_unstale":"2024-09-21T03:44:57.366695+0000","last_undegraded":"2024-09-21T03:44:57.366695+0000","last_fullsized":"2024-09-21T03:44:57.366695+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_clean_scrub_stamp":"2024-09-21T03:44:55.304777+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:45:29.403030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":355,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.338537+0000","last_change":"2024-09-21T03:33:25.319776+0000","last_active":"2024-09-21T03:44:58.338537+0000","last_peered":"2024-09-21T03:44:58.338537+0000","last_clean":"2024-09-21T03:44:58.338537+0000","last_became_active":"2024-09-21T03:33:25.318399+0000","last_became_peered":"2024-09-21T03:33:25.318399+0000","last_unstale":"2024-09-21T03:44:58.338537+0000","last_undegraded":"2024-09-21T03:44:58.338537+0000","last_fullsized":"2024-09-21T03:44:58.338537+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:23.016672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:23.016672+0000","last_clean_scrub_stamp":"2024-09-21T03:33:23.016672+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:32:35.906114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":4696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":33,"ondisk_log_size":33,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":1,"up_from":16,"seq":68719476879,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41936,"kb_used_data":2428,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94329904,"statfs":{"total":96636764160,"available":96593821696,"internally_reserved":0,"allocated":2486272,"data_stored":1546129,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509583,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41356,"kb_used_data":1848,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330484,"statfs":{"total":96636764160,"available":96594415616,"internally_reserved":0,"allocated":1892352,"data_stored":956321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":40434123},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509582,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41364,"kb_used_data":1852,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330476,"statfs":{"total":96636764160,"available":96594407424,"internally_reserved":0,"allocated":1896448,"data_stored":957281,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509583,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37184,"kb_used_data":1836,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334656,"statfs":{"total":96636764160,"available":96598687744,"internally_reserved":0,"allocated":1880064,"data_stored":949089,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509582,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37180,"kb_used_data":1832,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334660,"statfs":{"total":96636764160,"available":96598691840,"internally_reserved":0,"allocated":1875968,"data_stored":949559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542286,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33028,"kb_used_data":1840,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338812,"statfs":{"total":96636764160,"available":96602943488,"internally_reserved":0,"allocated":1884160,"data_stored":950589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574990,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37744,"kb_used_data":2396,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334096,"statfs":{"total":96636764160,"available":96598114304,"internally_reserved":0,"allocated":2453504,"data_stored":1532178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542287,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37168,"kb_used_data":1816,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334672,"statfs":{"total":96636764160,"available":96598704128,"internally_reserved":0,"allocated":1859584,"data_stored":941810,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T03:45:03.305 INFO:tasks.ceph.mds.c.smithi038.stderr:2024-09-21T03:45:03.303+0000 7f7ef4be4640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T03:45:05.965 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T03:45:06.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:06.095+0000 7fa416f69640 1 -- 172.21.15.38:0/1795015420 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa3f8003a00 msgr2=0x7fa3f8003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:06.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:06.095+0000 7fa416f69640 1 --2- 172.21.15.38:0/1795015420 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa3f8003a00 0x7fa3f8003e70 secure :-1 s=READY pgs=2375 cs=0 l=1 rev1=1 crypto rx=0x7fa40000c4a0 tx=0x7fa3f8005830 comp rx=0 tx=0).stop 2024-09-21T03:45:06.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:06.095+0000 7fa415506640 1 -- 172.21.15.38:0/1795015420 reap_dead start 2024-09-21T03:45:06.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:06.095+0000 7fa416f69640 1 -- 172.21.15.38:0/1795015420 shutdown_connections 2024-09-21T03:45:06.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:06.095+0000 7fa416f69640 1 -- 172.21.15.38:0/1795015420 >> 172.21.15.38:0/1795015420 conn(0x7fa4100b6b20 msgr2=0x7fa4100d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:06.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:06.095+0000 7fa416f69640 1 -- 172.21.15.38:0/1795015420 shutdown_connections 2024-09-21T03:45:06.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:06.095+0000 7fa416f69640 1 -- 172.21.15.38:0/1795015420 wait complete. 2024-09-21T03:45:06.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:06.097+0000 7fa416f69640 1 Processor -- start 2024-09-21T03:45:06.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:06.097+0000 7fa416f69640 1 -- start start 2024-09-21T03:45:06.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:06.098+0000 7fa416f69640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa3f8003a00 0x7fa410154ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:06.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:06.098+0000 7fa416f69640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa410154ff0 0x7fa4100cbac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:06.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:06.098+0000 7fa416f69640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa4100cc000 0x7fa4100ab520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:06.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:06.098+0000 7fa416f69640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa4100b35e0 con 0x7fa3f8003a00 2024-09-21T03:45:06.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:06.098+0000 7fa416f69640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fa4100b3460 con 0x7fa410154ff0 2024-09-21T03:45:06.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:06.098+0000 7fa416f69640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fa4100b3760 con 0x7fa4100cc000 2024-09-21T03:45:06.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:06.098+0000 7fa414d05640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa3f8003a00 0x7fa410154ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:06.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:06.098+0000 7fa414d05640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa3f8003a00 0x7fa410154ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43526/0 (socket says 172.21.15.38:43526) 2024-09-21T03:45:06.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:06.098+0000 7fa414d05640 1 -- 172.21.15.38:0/2615849129 learned_addr learned my addr 172.21.15.38:0/2615849129 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:45:06.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:06.099+0000 7fa40ffff640 1 --2- 172.21.15.38:0/2615849129 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa410154ff0 0x7fa4100cbac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:06.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:06.099+0000 7fa415506640 1 --2- 172.21.15.38:0/2615849129 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa4100cc000 0x7fa4100ab520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:06.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:06.099+0000 7fa414d05640 1 -- 172.21.15.38:0/2615849129 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa4100cc000 msgr2=0x7fa4100ab520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:06.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:06.099+0000 7fa414d05640 1 --2- 172.21.15.38:0/2615849129 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa4100cc000 0x7fa4100ab520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:06.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:06.099+0000 7fa414d05640 1 -- 172.21.15.38:0/2615849129 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa410154ff0 msgr2=0x7fa4100cbac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:06.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:06.099+0000 7fa414d05640 1 --2- 172.21.15.38:0/2615849129 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa410154ff0 0x7fa4100cbac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:06.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:06.099+0000 7fa414d05640 1 -- 172.21.15.38:0/2615849129 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4101589b0 con 0x7fa3f8003a00 2024-09-21T03:45:06.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:06.099+0000 7fa414d05640 1 --2- 172.21.15.38:0/2615849129 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa3f8003a00 0x7fa410154ab0 secure :-1 s=READY pgs=2376 cs=0 l=1 rev1=1 crypto rx=0x7fa40000c590 tx=0x7fa4000060e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:06.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:06.099+0000 7fa40d7fa640 1 -- 172.21.15.38:0/2615849129 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa40002b660 con 0x7fa3f8003a00 2024-09-21T03:45:06.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:06.099+0000 7fa40d7fa640 1 -- 172.21.15.38:0/2615849129 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa40002b800 con 0x7fa3f8003a00 2024-09-21T03:45:06.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:06.099+0000 7fa416f69640 1 -- 172.21.15.38:0/2615849129 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4100cebb0 con 0x7fa3f8003a00 2024-09-21T03:45:06.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:06.099+0000 7fa40d7fa640 1 -- 172.21.15.38:0/2615849129 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa40002bc50 con 0x7fa3f8003a00 2024-09-21T03:45:06.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:06.099+0000 7fa416f69640 1 -- 172.21.15.38:0/2615849129 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4100cd050 con 0x7fa3f8003a00 2024-09-21T03:45:06.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:06.100+0000 7fa416f69640 1 -- 172.21.15.38:0/2615849129 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa410162f00 con 0x7fa3f8003a00 2024-09-21T03:45:06.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:06.101+0000 7fa40d7fa640 1 -- 172.21.15.38:0/2615849129 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa400034470 con 0x7fa3f8003a00 2024-09-21T03:45:06.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:06.102+0000 7fa40d7fa640 1 --2- 172.21.15.38:0/2615849129 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa3fc078b90 0x7fa3fc07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:06.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:06.102+0000 7fa40d7fa640 1 -- 172.21.15.38:0/2615849129 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 6101+0+0 (secure 0 0 0) 0x7fa4000bae20 con 0x7fa3f8003a00 2024-09-21T03:45:06.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:06.102+0000 7fa40ffff640 1 --2- 172.21.15.38:0/2615849129 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa3fc078b90 0x7fa3fc07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:06.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:06.102+0000 7fa40ffff640 1 --2- 172.21.15.38:0/2615849129 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa3fc078b90 0x7fa3fc07b0a0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fa4100cf3b0 tx=0x7fa404005f90 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:06.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:06.106+0000 7fa40d7fa640 1 -- 172.21.15.38:0/2615849129 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fa410162bd0 con 0x7fa3f8003a00 2024-09-21T03:45:06.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:06.294+0000 7fa416f69640 1 -- 172.21.15.38:0/2615849129 --> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa4100d1b70 con 0x7fa3fc078b90 2024-09-21T03:45:06.302 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:06.299+0000 7fa40d7fa640 1 -- 172.21.15.38:0/2615849129 <== mgr.4117 v2:172.21.15.195:6800/1631901344 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252405 (secure 0 0 0) 0x7fa40405ea10 con 0x7fa3fc078b90 2024-09-21T03:45:06.307 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:45:06.308 INFO:teuthology.orchestra.run.smithi038.stderr:dumped all 2024-09-21T03:45:06.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:06.302+0000 7fa416f69640 1 -- 172.21.15.38:0/2615849129 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa3fc078b90 msgr2=0x7fa3fc07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:06.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:06.302+0000 7fa416f69640 1 --2- 172.21.15.38:0/2615849129 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa3fc078b90 0x7fa3fc07b0a0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fa4100cf3b0 tx=0x7fa404005f90 comp rx=0 tx=0).stop 2024-09-21T03:45:06.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:06.302+0000 7fa416f69640 1 -- 172.21.15.38:0/2615849129 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa3f8003a00 msgr2=0x7fa410154ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:06.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:06.302+0000 7fa416f69640 1 --2- 172.21.15.38:0/2615849129 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa3f8003a00 0x7fa410154ab0 secure :-1 s=READY pgs=2376 cs=0 l=1 rev1=1 crypto rx=0x7fa40000c590 tx=0x7fa4000060e0 comp rx=0 tx=0).stop 2024-09-21T03:45:06.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:06.303+0000 7fa416f69640 1 -- 172.21.15.38:0/2615849129 shutdown_connections 2024-09-21T03:45:06.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:06.303+0000 7fa416f69640 1 --2- 172.21.15.38:0/2615849129 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa3fc078b90 0x7fa3fc07b0a0 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:06.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:06.303+0000 7fa416f69640 1 --2- 172.21.15.38:0/2615849129 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa4100cc000 0x7fa4100ab520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:06.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:06.303+0000 7fa416f69640 1 --2- 172.21.15.38:0/2615849129 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa410154ff0 0x7fa4100cbac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:06.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:06.303+0000 7fa416f69640 1 --2- 172.21.15.38:0/2615849129 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa3f8003a00 0x7fa410154ab0 unknown :-1 s=CLOSED pgs=2376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:06.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:06.303+0000 7fa416f69640 1 -- 172.21.15.38:0/2615849129 >> 172.21.15.38:0/2615849129 conn(0x7fa4100b6b20 msgr2=0x7fa4100b90b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:06.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:06.303+0000 7fa416f69640 1 -- 172.21.15.38:0/2615849129 shutdown_connections 2024-09-21T03:45:06.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:06.303+0000 7fa416f69640 1 -- 172.21.15.38:0/2615849129 wait complete. 2024-09-21T03:45:06.317 INFO:teuthology.orchestra.run.smithi038.stdout:{"pg_ready":true,"pg_map":{"version":490,"stamp":"2024-09-21T03:45:05.173210+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":307076,"kb_used_data":15944,"kb_used_omap":12,"kb_used_meta":290931,"kb_avail":754667644,"statfs":{"total":773094113280,"available":772779667456,"internally_reserved":0,"allocated":16326656,"data_stored":8830908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12748,"internal_metadata":297913908},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002082"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.336322+0000","last_change":"2024-09-21T03:44:57.340688+0000","last_active":"2024-09-21T03:44:58.336322+0000","last_peered":"2024-09-21T03:44:58.336322+0000","last_clean":"2024-09-21T03:44:58.336322+0000","last_became_active":"2024-09-21T03:44:57.340356+0000","last_became_peered":"2024-09-21T03:44:57.340356+0000","last_unstale":"2024-09-21T03:44:58.336322+0000","last_undegraded":"2024-09-21T03:44:58.336322+0000","last_fullsized":"2024-09-21T03:44:58.336322+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:24:51.943894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.915466+0000","last_change":"2024-09-21T03:44:57.354834+0000","last_active":"2024-09-21T03:44:58.915466+0000","last_peered":"2024-09-21T03:44:58.915466+0000","last_clean":"2024-09-21T03:44:58.915466+0000","last_became_active":"2024-09-21T03:44:57.354518+0000","last_became_peered":"2024-09-21T03:44:57.354518+0000","last_unstale":"2024-09-21T03:44:58.915466+0000","last_undegraded":"2024-09-21T03:44:58.915466+0000","last_fullsized":"2024-09-21T03:44:58.915466+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:21:26.621000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.349900+0000","last_change":"2024-09-21T03:44:57.353805+0000","last_active":"2024-09-21T03:44:58.349900+0000","last_peered":"2024-09-21T03:44:58.349900+0000","last_clean":"2024-09-21T03:44:58.349900+0000","last_became_active":"2024-09-21T03:44:57.352464+0000","last_became_peered":"2024-09-21T03:44:57.352464+0000","last_unstale":"2024-09-21T03:44:58.349900+0000","last_undegraded":"2024-09-21T03:44:58.349900+0000","last_fullsized":"2024-09-21T03:44:58.349900+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:47:32.226379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.335376+0000","last_change":"2024-09-21T03:44:57.356466+0000","last_active":"2024-09-21T03:44:58.335376+0000","last_peered":"2024-09-21T03:44:58.335376+0000","last_clean":"2024-09-21T03:44:58.335376+0000","last_became_active":"2024-09-21T03:44:57.355938+0000","last_became_peered":"2024-09-21T03:44:57.355938+0000","last_unstale":"2024-09-21T03:44:58.335376+0000","last_undegraded":"2024-09-21T03:44:58.335376+0000","last_fullsized":"2024-09-21T03:44:58.335376+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:16:10.502815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.335203+0000","last_change":"2024-09-21T03:44:57.351972+0000","last_active":"2024-09-21T03:44:58.335203+0000","last_peered":"2024-09-21T03:44:58.335203+0000","last_clean":"2024-09-21T03:44:58.335203+0000","last_became_active":"2024-09-21T03:44:57.351561+0000","last_became_peered":"2024-09-21T03:44:57.351561+0000","last_unstale":"2024-09-21T03:44:58.335203+0000","last_undegraded":"2024-09-21T03:44:58.335203+0000","last_fullsized":"2024-09-21T03:44:58.335203+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:33:43.994424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.353120+0000","last_change":"2024-09-21T03:44:57.349848+0000","last_active":"2024-09-21T03:44:58.353120+0000","last_peered":"2024-09-21T03:44:58.353120+0000","last_clean":"2024-09-21T03:44:58.353120+0000","last_became_active":"2024-09-21T03:44:57.349387+0000","last_became_peered":"2024-09-21T03:44:57.349387+0000","last_unstale":"2024-09-21T03:44:58.353120+0000","last_undegraded":"2024-09-21T03:44:58.353120+0000","last_fullsized":"2024-09-21T03:44:58.353120+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:10:55.772054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.352606+0000","last_change":"2024-09-21T03:44:57.355533+0000","last_active":"2024-09-21T03:44:58.352606+0000","last_peered":"2024-09-21T03:44:58.352606+0000","last_clean":"2024-09-21T03:44:58.352606+0000","last_became_active":"2024-09-21T03:44:57.354892+0000","last_became_peered":"2024-09-21T03:44:57.354892+0000","last_unstale":"2024-09-21T03:44:58.352606+0000","last_undegraded":"2024-09-21T03:44:58.352606+0000","last_fullsized":"2024-09-21T03:44:58.352606+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:05:41.952462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.337763+0000","last_change":"2024-09-21T03:44:57.350864+0000","last_active":"2024-09-21T03:44:58.337763+0000","last_peered":"2024-09-21T03:44:58.337763+0000","last_clean":"2024-09-21T03:44:58.337763+0000","last_became_active":"2024-09-21T03:44:57.350215+0000","last_became_peered":"2024-09-21T03:44:57.350215+0000","last_unstale":"2024-09-21T03:44:58.337763+0000","last_undegraded":"2024-09-21T03:44:58.337763+0000","last_fullsized":"2024-09-21T03:44:58.337763+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:57:25.073668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.915427+0000","last_change":"2024-09-21T03:44:57.356640+0000","last_active":"2024-09-21T03:44:58.915427+0000","last_peered":"2024-09-21T03:44:58.915427+0000","last_clean":"2024-09-21T03:44:58.915427+0000","last_became_active":"2024-09-21T03:44:57.355966+0000","last_became_peered":"2024-09-21T03:44:57.355966+0000","last_unstale":"2024-09-21T03:44:58.915427+0000","last_undegraded":"2024-09-21T03:44:58.915427+0000","last_fullsized":"2024-09-21T03:44:58.915427+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:40:49.216619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.338187+0000","last_change":"2024-09-21T03:44:57.359320+0000","last_active":"2024-09-21T03:44:58.338187+0000","last_peered":"2024-09-21T03:44:58.338187+0000","last_clean":"2024-09-21T03:44:58.338187+0000","last_became_active":"2024-09-21T03:44:57.358988+0000","last_became_peered":"2024-09-21T03:44:57.358988+0000","last_unstale":"2024-09-21T03:44:58.338187+0000","last_undegraded":"2024-09-21T03:44:58.338187+0000","last_fullsized":"2024-09-21T03:44:58.338187+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:43:22.624675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.337467+0000","last_change":"2024-09-21T03:44:57.362260+0000","last_active":"2024-09-21T03:44:58.337467+0000","last_peered":"2024-09-21T03:44:58.337467+0000","last_clean":"2024-09-21T03:44:58.337467+0000","last_became_active":"2024-09-21T03:44:57.361624+0000","last_became_peered":"2024-09-21T03:44:57.361624+0000","last_unstale":"2024-09-21T03:44:58.337467+0000","last_undegraded":"2024-09-21T03:44:58.337467+0000","last_fullsized":"2024-09-21T03:44:58.337467+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:00:23.014227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.352884+0000","last_change":"2024-09-21T03:44:57.353828+0000","last_active":"2024-09-21T03:44:58.352884+0000","last_peered":"2024-09-21T03:44:58.352884+0000","last_clean":"2024-09-21T03:44:58.352884+0000","last_became_active":"2024-09-21T03:44:57.353516+0000","last_became_peered":"2024-09-21T03:44:57.353516+0000","last_unstale":"2024-09-21T03:44:58.352884+0000","last_undegraded":"2024-09-21T03:44:58.352884+0000","last_fullsized":"2024-09-21T03:44:58.352884+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:49:05.536169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.335365+0000","last_change":"2024-09-21T03:44:57.363696+0000","last_active":"2024-09-21T03:44:58.335365+0000","last_peered":"2024-09-21T03:44:58.335365+0000","last_clean":"2024-09-21T03:44:58.335365+0000","last_became_active":"2024-09-21T03:44:57.362737+0000","last_became_peered":"2024-09-21T03:44:57.362737+0000","last_unstale":"2024-09-21T03:44:58.335365+0000","last_undegraded":"2024-09-21T03:44:58.335365+0000","last_fullsized":"2024-09-21T03:44:58.335365+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:17:17.662702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.353349+0000","last_change":"2024-09-21T03:44:57.352690+0000","last_active":"2024-09-21T03:44:58.353349+0000","last_peered":"2024-09-21T03:44:58.353349+0000","last_clean":"2024-09-21T03:44:58.353349+0000","last_became_active":"2024-09-21T03:44:57.352358+0000","last_became_peered":"2024-09-21T03:44:57.352358+0000","last_unstale":"2024-09-21T03:44:58.353349+0000","last_undegraded":"2024-09-21T03:44:58.353349+0000","last_fullsized":"2024-09-21T03:44:58.353349+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:42:27.812988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.915095+0000","last_change":"2024-09-21T03:44:57.369797+0000","last_active":"2024-09-21T03:44:58.915095+0000","last_peered":"2024-09-21T03:44:58.915095+0000","last_clean":"2024-09-21T03:44:58.915095+0000","last_became_active":"2024-09-21T03:44:57.369463+0000","last_became_peered":"2024-09-21T03:44:57.369463+0000","last_unstale":"2024-09-21T03:44:58.915095+0000","last_undegraded":"2024-09-21T03:44:58.915095+0000","last_fullsized":"2024-09-21T03:44:58.915095+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:59:36.159782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.338697+0000","last_change":"2024-09-21T03:44:57.341545+0000","last_active":"2024-09-21T03:44:58.338697+0000","last_peered":"2024-09-21T03:44:58.338697+0000","last_clean":"2024-09-21T03:44:58.338697+0000","last_became_active":"2024-09-21T03:44:57.341220+0000","last_became_peered":"2024-09-21T03:44:57.341220+0000","last_unstale":"2024-09-21T03:44:58.338697+0000","last_undegraded":"2024-09-21T03:44:58.338697+0000","last_fullsized":"2024-09-21T03:44:58.338697+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:40:08.834183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.350171+0000","last_change":"2024-09-21T03:44:57.353372+0000","last_active":"2024-09-21T03:44:58.350171+0000","last_peered":"2024-09-21T03:44:58.350171+0000","last_clean":"2024-09-21T03:44:58.350171+0000","last_became_active":"2024-09-21T03:44:57.353042+0000","last_became_peered":"2024-09-21T03:44:57.353042+0000","last_unstale":"2024-09-21T03:44:58.350171+0000","last_undegraded":"2024-09-21T03:44:58.350171+0000","last_fullsized":"2024-09-21T03:44:58.350171+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:42:05.420636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.915438+0000","last_change":"2024-09-21T03:44:57.357272+0000","last_active":"2024-09-21T03:44:58.915438+0000","last_peered":"2024-09-21T03:44:58.915438+0000","last_clean":"2024-09-21T03:44:58.915438+0000","last_became_active":"2024-09-21T03:44:57.356952+0000","last_became_peered":"2024-09-21T03:44:57.356952+0000","last_unstale":"2024-09-21T03:44:58.915438+0000","last_undegraded":"2024-09-21T03:44:58.915438+0000","last_fullsized":"2024-09-21T03:44:58.915438+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:35:23.462964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.335878+0000","last_change":"2024-09-21T03:44:57.351765+0000","last_active":"2024-09-21T03:44:58.335878+0000","last_peered":"2024-09-21T03:44:58.335878+0000","last_clean":"2024-09-21T03:44:58.335878+0000","last_became_active":"2024-09-21T03:44:57.351478+0000","last_became_peered":"2024-09-21T03:44:57.351478+0000","last_unstale":"2024-09-21T03:44:58.335878+0000","last_undegraded":"2024-09-21T03:44:58.335878+0000","last_fullsized":"2024-09-21T03:44:58.335878+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:30:35.733399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.339047+0000","last_change":"2024-09-21T03:44:57.358796+0000","last_active":"2024-09-21T03:44:58.339047+0000","last_peered":"2024-09-21T03:44:58.339047+0000","last_clean":"2024-09-21T03:44:58.339047+0000","last_became_active":"2024-09-21T03:44:57.358439+0000","last_became_peered":"2024-09-21T03:44:57.358439+0000","last_unstale":"2024-09-21T03:44:58.339047+0000","last_undegraded":"2024-09-21T03:44:58.339047+0000","last_fullsized":"2024-09-21T03:44:58.339047+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:05:51.280587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.349898+0000","last_change":"2024-09-21T03:44:57.357827+0000","last_active":"2024-09-21T03:44:58.349898+0000","last_peered":"2024-09-21T03:44:58.349898+0000","last_clean":"2024-09-21T03:44:58.349898+0000","last_became_active":"2024-09-21T03:44:57.357509+0000","last_became_peered":"2024-09-21T03:44:57.357509+0000","last_unstale":"2024-09-21T03:44:58.349898+0000","last_undegraded":"2024-09-21T03:44:58.349898+0000","last_fullsized":"2024-09-21T03:44:58.349898+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:58:26.577333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.456326+0000","last_change":"2024-09-21T03:44:57.357854+0000","last_active":"2024-09-21T03:44:58.456326+0000","last_peered":"2024-09-21T03:44:58.456326+0000","last_clean":"2024-09-21T03:44:58.456326+0000","last_became_active":"2024-09-21T03:44:57.357541+0000","last_became_peered":"2024-09-21T03:44:57.357541+0000","last_unstale":"2024-09-21T03:44:58.456326+0000","last_undegraded":"2024-09-21T03:44:58.456326+0000","last_fullsized":"2024-09-21T03:44:58.456326+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:04:52.143962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.337862+0000","last_change":"2024-09-21T03:44:57.353700+0000","last_active":"2024-09-21T03:44:58.337862+0000","last_peered":"2024-09-21T03:44:58.337862+0000","last_clean":"2024-09-21T03:44:58.337862+0000","last_became_active":"2024-09-21T03:44:57.351501+0000","last_became_peered":"2024-09-21T03:44:57.351501+0000","last_unstale":"2024-09-21T03:44:58.337862+0000","last_undegraded":"2024-09-21T03:44:58.337862+0000","last_fullsized":"2024-09-21T03:44:58.337862+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:29:07.279860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.337748+0000","last_change":"2024-09-21T03:44:57.351638+0000","last_active":"2024-09-21T03:44:58.337748+0000","last_peered":"2024-09-21T03:44:58.337748+0000","last_clean":"2024-09-21T03:44:58.337748+0000","last_became_active":"2024-09-21T03:44:57.351325+0000","last_became_peered":"2024-09-21T03:44:57.351325+0000","last_unstale":"2024-09-21T03:44:58.337748+0000","last_undegraded":"2024-09-21T03:44:58.337748+0000","last_fullsized":"2024-09-21T03:44:58.337748+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:36:45.265239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.339252+0000","last_change":"2024-09-21T03:44:57.359418+0000","last_active":"2024-09-21T03:44:58.339252+0000","last_peered":"2024-09-21T03:44:58.339252+0000","last_clean":"2024-09-21T03:44:58.339252+0000","last_became_active":"2024-09-21T03:44:57.358564+0000","last_became_peered":"2024-09-21T03:44:57.358564+0000","last_unstale":"2024-09-21T03:44:58.339252+0000","last_undegraded":"2024-09-21T03:44:58.339252+0000","last_fullsized":"2024-09-21T03:44:58.339252+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:21:26.978675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.339318+0000","last_change":"2024-09-21T03:44:57.360155+0000","last_active":"2024-09-21T03:44:58.339318+0000","last_peered":"2024-09-21T03:44:58.339318+0000","last_clean":"2024-09-21T03:44:58.339318+0000","last_became_active":"2024-09-21T03:44:57.359645+0000","last_became_peered":"2024-09-21T03:44:57.359645+0000","last_unstale":"2024-09-21T03:44:58.339318+0000","last_undegraded":"2024-09-21T03:44:58.339318+0000","last_fullsized":"2024-09-21T03:44:58.339318+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:14:53.602909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.339127+0000","last_change":"2024-09-21T03:44:57.357568+0000","last_active":"2024-09-21T03:44:58.339127+0000","last_peered":"2024-09-21T03:44:58.339127+0000","last_clean":"2024-09-21T03:44:58.339127+0000","last_became_active":"2024-09-21T03:44:57.357185+0000","last_became_peered":"2024-09-21T03:44:57.357185+0000","last_unstale":"2024-09-21T03:44:58.339127+0000","last_undegraded":"2024-09-21T03:44:58.339127+0000","last_fullsized":"2024-09-21T03:44:58.339127+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:57:24.936155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.353110+0000","last_change":"2024-09-21T03:44:57.352231+0000","last_active":"2024-09-21T03:44:58.353110+0000","last_peered":"2024-09-21T03:44:58.353110+0000","last_clean":"2024-09-21T03:44:58.353110+0000","last_became_active":"2024-09-21T03:44:57.351887+0000","last_became_peered":"2024-09-21T03:44:57.351887+0000","last_unstale":"2024-09-21T03:44:58.353110+0000","last_undegraded":"2024-09-21T03:44:58.353110+0000","last_fullsized":"2024-09-21T03:44:58.353110+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:46:07.069681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.337919+0000","last_change":"2024-09-21T03:44:57.357470+0000","last_active":"2024-09-21T03:44:58.337919+0000","last_peered":"2024-09-21T03:44:58.337919+0000","last_clean":"2024-09-21T03:44:58.337919+0000","last_became_active":"2024-09-21T03:44:57.357091+0000","last_became_peered":"2024-09-21T03:44:57.357091+0000","last_unstale":"2024-09-21T03:44:58.337919+0000","last_undegraded":"2024-09-21T03:44:58.337919+0000","last_fullsized":"2024-09-21T03:44:58.337919+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:46:07.069681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.337478+0000","last_change":"2024-09-21T03:44:57.361022+0000","last_active":"2024-09-21T03:44:58.337478+0000","last_peered":"2024-09-21T03:44:58.337478+0000","last_clean":"2024-09-21T03:44:58.337478+0000","last_became_active":"2024-09-21T03:44:57.360697+0000","last_became_peered":"2024-09-21T03:44:57.360697+0000","last_unstale":"2024-09-21T03:44:58.337478+0000","last_undegraded":"2024-09-21T03:44:58.337478+0000","last_fullsized":"2024-09-21T03:44:58.337478+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:17:17.662702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.352611+0000","last_change":"2024-09-21T03:44:57.357841+0000","last_active":"2024-09-21T03:44:58.352611+0000","last_peered":"2024-09-21T03:44:58.352611+0000","last_clean":"2024-09-21T03:44:58.352611+0000","last_became_active":"2024-09-21T03:44:57.357299+0000","last_became_peered":"2024-09-21T03:44:57.357299+0000","last_unstale":"2024-09-21T03:44:58.352611+0000","last_undegraded":"2024-09-21T03:44:58.352611+0000","last_fullsized":"2024-09-21T03:44:58.352611+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:36:45.265239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.456162+0000","last_change":"2024-09-21T03:44:57.366423+0000","last_active":"2024-09-21T03:44:58.456162+0000","last_peered":"2024-09-21T03:44:58.456162+0000","last_clean":"2024-09-21T03:44:58.456162+0000","last_became_active":"2024-09-21T03:44:57.366083+0000","last_became_peered":"2024-09-21T03:44:57.366083+0000","last_unstale":"2024-09-21T03:44:58.456162+0000","last_undegraded":"2024-09-21T03:44:58.456162+0000","last_fullsized":"2024-09-21T03:44:58.456162+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:14:54.627243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.349692+0000","last_change":"2024-09-21T03:44:57.352618+0000","last_active":"2024-09-21T03:44:58.349692+0000","last_peered":"2024-09-21T03:44:58.349692+0000","last_clean":"2024-09-21T03:44:58.349692+0000","last_became_active":"2024-09-21T03:44:57.352189+0000","last_became_peered":"2024-09-21T03:44:57.352189+0000","last_unstale":"2024-09-21T03:44:58.349692+0000","last_undegraded":"2024-09-21T03:44:58.349692+0000","last_fullsized":"2024-09-21T03:44:58.349692+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:38:44.361006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1f","version":"145'4","reported_seq":19,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.456694+0000","last_change":"2024-09-21T03:44:56.332485+0000","last_active":"2024-09-21T03:44:58.456694+0000","last_peered":"2024-09-21T03:44:58.456694+0000","last_clean":"2024-09-21T03:44:58.456694+0000","last_became_active":"2024-09-21T03:44:56.332204+0000","last_became_peered":"2024-09-21T03:44:56.332204+0000","last_unstale":"2024-09-21T03:44:58.456694+0000","last_undegraded":"2024-09-21T03:44:58.456694+0000","last_fullsized":"2024-09-21T03:44:58.456694+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_clean_scrub_stamp":"2024-09-21T03:44:55.304777+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:15:02.688269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.338001+0000","last_change":"2024-09-21T03:44:57.352443+0000","last_active":"2024-09-21T03:44:58.338001+0000","last_peered":"2024-09-21T03:44:58.338001+0000","last_clean":"2024-09-21T03:44:58.338001+0000","last_became_active":"2024-09-21T03:44:57.351086+0000","last_became_peered":"2024-09-21T03:44:57.351086+0000","last_unstale":"2024-09-21T03:44:58.338001+0000","last_undegraded":"2024-09-21T03:44:58.338001+0000","last_fullsized":"2024-09-21T03:44:58.338001+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:31:55.663249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1e","version":"145'1","reported_seq":16,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.350034+0000","last_change":"2024-09-21T03:44:56.341615+0000","last_active":"2024-09-21T03:44:58.350034+0000","last_peered":"2024-09-21T03:44:58.350034+0000","last_clean":"2024-09-21T03:44:58.350034+0000","last_became_active":"2024-09-21T03:44:56.341306+0000","last_became_peered":"2024-09-21T03:44:56.341306+0000","last_unstale":"2024-09-21T03:44:58.350034+0000","last_undegraded":"2024-09-21T03:44:58.350034+0000","last_fullsized":"2024-09-21T03:44:58.350034+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_clean_scrub_stamp":"2024-09-21T03:44:55.304777+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:12:55.488245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.915135+0000","last_change":"2024-09-21T03:44:57.349398+0000","last_active":"2024-09-21T03:44:58.915135+0000","last_peered":"2024-09-21T03:44:58.915135+0000","last_clean":"2024-09-21T03:44:58.915135+0000","last_became_active":"2024-09-21T03:44:57.349067+0000","last_became_peered":"2024-09-21T03:44:57.349067+0000","last_unstale":"2024-09-21T03:44:58.915135+0000","last_undegraded":"2024-09-21T03:44:58.915135+0000","last_fullsized":"2024-09-21T03:44:58.915135+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:19:45.952024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"145'2","reported_seq":17,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.915396+0000","last_change":"2024-09-21T03:44:56.343325+0000","last_active":"2024-09-21T03:44:58.915396+0000","last_peered":"2024-09-21T03:44:58.915396+0000","last_clean":"2024-09-21T03:44:58.915396+0000","last_became_active":"2024-09-21T03:44:56.342964+0000","last_became_peered":"2024-09-21T03:44:56.342964+0000","last_unstale":"2024-09-21T03:44:58.915396+0000","last_undegraded":"2024-09-21T03:44:58.915396+0000","last_fullsized":"2024-09-21T03:44:58.915396+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_clean_scrub_stamp":"2024-09-21T03:44:55.304777+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:19:36.404105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.337488+0000","last_change":"2024-09-21T03:44:57.360138+0000","last_active":"2024-09-21T03:44:58.337488+0000","last_peered":"2024-09-21T03:44:58.337488+0000","last_clean":"2024-09-21T03:44:58.337488+0000","last_became_active":"2024-09-21T03:44:57.359800+0000","last_became_peered":"2024-09-21T03:44:57.359800+0000","last_unstale":"2024-09-21T03:44:58.337488+0000","last_undegraded":"2024-09-21T03:44:58.337488+0000","last_fullsized":"2024-09-21T03:44:58.337488+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:33:24.015377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1c","version":"145'2","reported_seq":17,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.338178+0000","last_change":"2024-09-21T03:44:56.334534+0000","last_active":"2024-09-21T03:44:58.338178+0000","last_peered":"2024-09-21T03:44:58.338178+0000","last_clean":"2024-09-21T03:44:58.338178+0000","last_became_active":"2024-09-21T03:44:56.333800+0000","last_became_peered":"2024-09-21T03:44:56.333800+0000","last_unstale":"2024-09-21T03:44:58.338178+0000","last_undegraded":"2024-09-21T03:44:58.338178+0000","last_fullsized":"2024-09-21T03:44:58.338178+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_clean_scrub_stamp":"2024-09-21T03:44:55.304777+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:44:41.072083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.336017+0000","last_change":"2024-09-21T03:44:57.342301+0000","last_active":"2024-09-21T03:44:58.336017+0000","last_peered":"2024-09-21T03:44:58.336017+0000","last_clean":"2024-09-21T03:44:58.336017+0000","last_became_active":"2024-09-21T03:44:57.341880+0000","last_became_peered":"2024-09-21T03:44:57.341880+0000","last_unstale":"2024-09-21T03:44:58.336017+0000","last_undegraded":"2024-09-21T03:44:58.336017+0000","last_fullsized":"2024-09-21T03:44:58.336017+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:13:17.457095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1b","version":"145'1","reported_seq":16,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.337514+0000","last_change":"2024-09-21T03:44:56.342732+0000","last_active":"2024-09-21T03:44:58.337514+0000","last_peered":"2024-09-21T03:44:58.337514+0000","last_clean":"2024-09-21T03:44:58.337514+0000","last_became_active":"2024-09-21T03:44:56.342319+0000","last_became_peered":"2024-09-21T03:44:56.342319+0000","last_unstale":"2024-09-21T03:44:58.337514+0000","last_undegraded":"2024-09-21T03:44:58.337514+0000","last_fullsized":"2024-09-21T03:44:58.337514+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_clean_scrub_stamp":"2024-09-21T03:44:55.304777+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:31:32.978494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.336575+0000","last_change":"2024-09-21T03:44:57.346978+0000","last_active":"2024-09-21T03:44:58.336575+0000","last_peered":"2024-09-21T03:44:58.336575+0000","last_clean":"2024-09-21T03:44:58.336575+0000","last_became_active":"2024-09-21T03:44:57.346633+0000","last_became_peered":"2024-09-21T03:44:57.346633+0000","last_unstale":"2024-09-21T03:44:58.336575+0000","last_undegraded":"2024-09-21T03:44:58.336575+0000","last_fullsized":"2024-09-21T03:44:58.336575+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:46:09.078290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1a","version":"145'1","reported_seq":16,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.338782+0000","last_change":"2024-09-21T03:44:56.342500+0000","last_active":"2024-09-21T03:44:58.338782+0000","last_peered":"2024-09-21T03:44:58.338782+0000","last_clean":"2024-09-21T03:44:58.338782+0000","last_became_active":"2024-09-21T03:44:56.342198+0000","last_became_peered":"2024-09-21T03:44:56.342198+0000","last_unstale":"2024-09-21T03:44:58.338782+0000","last_undegraded":"2024-09-21T03:44:58.338782+0000","last_fullsized":"2024-09-21T03:44:58.338782+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_clean_scrub_stamp":"2024-09-21T03:44:55.304777+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:24:33.377335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.337947+0000","last_change":"2024-09-21T03:44:57.360511+0000","last_active":"2024-09-21T03:44:58.337947+0000","last_peered":"2024-09-21T03:44:58.337947+0000","last_clean":"2024-09-21T03:44:58.337947+0000","last_became_active":"2024-09-21T03:44:57.360163+0000","last_became_peered":"2024-09-21T03:44:57.360163+0000","last_unstale":"2024-09-21T03:44:58.337947+0000","last_undegraded":"2024-09-21T03:44:58.337947+0000","last_fullsized":"2024-09-21T03:44:58.337947+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:05:31.071185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":15,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.338632+0000","last_change":"2024-09-21T03:44:56.343444+0000","last_active":"2024-09-21T03:44:58.338632+0000","last_peered":"2024-09-21T03:44:58.338632+0000","last_clean":"2024-09-21T03:44:58.338632+0000","last_became_active":"2024-09-21T03:44:56.343128+0000","last_became_peered":"2024-09-21T03:44:56.343128+0000","last_unstale":"2024-09-21T03:44:58.338632+0000","last_undegraded":"2024-09-21T03:44:58.338632+0000","last_fullsized":"2024-09-21T03:44:58.338632+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_clean_scrub_stamp":"2024-09-21T03:44:55.304777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:49:27.666317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.336579+0000","last_change":"2024-09-21T03:44:57.348168+0000","last_active":"2024-09-21T03:44:58.336579+0000","last_peered":"2024-09-21T03:44:58.336579+0000","last_clean":"2024-09-21T03:44:58.336579+0000","last_became_active":"2024-09-21T03:44:57.347823+0000","last_became_peered":"2024-09-21T03:44:57.347823+0000","last_unstale":"2024-09-21T03:44:58.336579+0000","last_undegraded":"2024-09-21T03:44:58.336579+0000","last_fullsized":"2024-09-21T03:44:58.336579+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:22:02.815105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":15,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.456487+0000","last_change":"2024-09-21T03:44:56.337896+0000","last_active":"2024-09-21T03:44:58.456487+0000","last_peered":"2024-09-21T03:44:58.456487+0000","last_clean":"2024-09-21T03:44:58.456487+0000","last_became_active":"2024-09-21T03:44:56.337298+0000","last_became_peered":"2024-09-21T03:44:56.337298+0000","last_unstale":"2024-09-21T03:44:58.456487+0000","last_undegraded":"2024-09-21T03:44:58.456487+0000","last_fullsized":"2024-09-21T03:44:58.456487+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_clean_scrub_stamp":"2024-09-21T03:44:55.304777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:58:50.496950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.335030+0000","last_change":"2024-09-21T03:44:57.350094+0000","last_active":"2024-09-21T03:44:58.335030+0000","last_peered":"2024-09-21T03:44:58.335030+0000","last_clean":"2024-09-21T03:44:58.335030+0000","last_became_active":"2024-09-21T03:44:57.349777+0000","last_became_peered":"2024-09-21T03:44:57.349777+0000","last_unstale":"2024-09-21T03:44:58.335030+0000","last_undegraded":"2024-09-21T03:44:58.335030+0000","last_fullsized":"2024-09-21T03:44:58.335030+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:00:23.014227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":15,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.353239+0000","last_change":"2024-09-21T03:44:56.329230+0000","last_active":"2024-09-21T03:44:58.353239+0000","last_peered":"2024-09-21T03:44:58.353239+0000","last_clean":"2024-09-21T03:44:58.353239+0000","last_became_active":"2024-09-21T03:44:56.328911+0000","last_became_peered":"2024-09-21T03:44:56.328911+0000","last_unstale":"2024-09-21T03:44:58.353239+0000","last_undegraded":"2024-09-21T03:44:58.353239+0000","last_fullsized":"2024-09-21T03:44:58.353239+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_clean_scrub_stamp":"2024-09-21T03:44:55.304777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:16:45.547857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.334744+0000","last_change":"2024-09-21T03:44:57.347015+0000","last_active":"2024-09-21T03:44:58.334744+0000","last_peered":"2024-09-21T03:44:58.334744+0000","last_clean":"2024-09-21T03:44:58.334744+0000","last_became_active":"2024-09-21T03:44:57.346668+0000","last_became_peered":"2024-09-21T03:44:57.346668+0000","last_unstale":"2024-09-21T03:44:58.334744+0000","last_undegraded":"2024-09-21T03:44:58.334744+0000","last_fullsized":"2024-09-21T03:44:58.334744+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:17:49.549006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":15,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.338116+0000","last_change":"2024-09-21T03:44:56.333386+0000","last_active":"2024-09-21T03:44:58.338116+0000","last_peered":"2024-09-21T03:44:58.338116+0000","last_clean":"2024-09-21T03:44:58.338116+0000","last_became_active":"2024-09-21T03:44:56.333082+0000","last_became_peered":"2024-09-21T03:44:56.333082+0000","last_unstale":"2024-09-21T03:44:58.338116+0000","last_undegraded":"2024-09-21T03:44:58.338116+0000","last_fullsized":"2024-09-21T03:44:58.338116+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_clean_scrub_stamp":"2024-09-21T03:44:55.304777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:10:24.715196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.338035+0000","last_change":"2024-09-21T03:44:57.350041+0000","last_active":"2024-09-21T03:44:58.338035+0000","last_peered":"2024-09-21T03:44:58.338035+0000","last_clean":"2024-09-21T03:44:58.338035+0000","last_became_active":"2024-09-21T03:44:57.349706+0000","last_became_peered":"2024-09-21T03:44:57.349706+0000","last_unstale":"2024-09-21T03:44:58.338035+0000","last_undegraded":"2024-09-21T03:44:58.338035+0000","last_fullsized":"2024-09-21T03:44:58.338035+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:01:23.359578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.15","version":"145'3","reported_seq":18,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.352972+0000","last_change":"2024-09-21T03:44:56.324998+0000","last_active":"2024-09-21T03:44:58.352972+0000","last_peered":"2024-09-21T03:44:58.352972+0000","last_clean":"2024-09-21T03:44:58.352972+0000","last_became_active":"2024-09-21T03:44:56.324464+0000","last_became_peered":"2024-09-21T03:44:56.324464+0000","last_unstale":"2024-09-21T03:44:58.352972+0000","last_undegraded":"2024-09-21T03:44:58.352972+0000","last_fullsized":"2024-09-21T03:44:58.352972+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_clean_scrub_stamp":"2024-09-21T03:44:55.304777+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:27:45.431858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.456464+0000","last_change":"2024-09-21T03:44:57.355113+0000","last_active":"2024-09-21T03:44:58.456464+0000","last_peered":"2024-09-21T03:44:58.456464+0000","last_clean":"2024-09-21T03:44:58.456464+0000","last_became_active":"2024-09-21T03:44:57.354744+0000","last_became_peered":"2024-09-21T03:44:57.354744+0000","last_unstale":"2024-09-21T03:44:58.456464+0000","last_undegraded":"2024-09-21T03:44:58.456464+0000","last_fullsized":"2024-09-21T03:44:58.456464+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:54:24.418739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.14","version":"145'6","reported_seq":21,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.337583+0000","last_change":"2024-09-21T03:44:56.341013+0000","last_active":"2024-09-21T03:44:58.337583+0000","last_peered":"2024-09-21T03:44:58.337583+0000","last_clean":"2024-09-21T03:44:58.337583+0000","last_became_active":"2024-09-21T03:44:56.337878+0000","last_became_peered":"2024-09-21T03:44:56.337878+0000","last_unstale":"2024-09-21T03:44:58.337583+0000","last_undegraded":"2024-09-21T03:44:58.337583+0000","last_fullsized":"2024-09-21T03:44:58.337583+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_clean_scrub_stamp":"2024-09-21T03:44:55.304777+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:00:34.311511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.915365+0000","last_change":"2024-09-21T03:44:57.356176+0000","last_active":"2024-09-21T03:44:58.915365+0000","last_peered":"2024-09-21T03:44:58.915365+0000","last_clean":"2024-09-21T03:44:58.915365+0000","last_became_active":"2024-09-21T03:44:57.355832+0000","last_became_peered":"2024-09-21T03:44:57.355832+0000","last_unstale":"2024-09-21T03:44:58.915365+0000","last_undegraded":"2024-09-21T03:44:58.915365+0000","last_fullsized":"2024-09-21T03:44:58.915365+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:47:30.592382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.13","version":"145'2","reported_seq":17,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.335390+0000","last_change":"2024-09-21T03:44:56.327633+0000","last_active":"2024-09-21T03:44:58.335390+0000","last_peered":"2024-09-21T03:44:58.335390+0000","last_clean":"2024-09-21T03:44:58.335390+0000","last_became_active":"2024-09-21T03:44:56.327313+0000","last_became_peered":"2024-09-21T03:44:56.327313+0000","last_unstale":"2024-09-21T03:44:58.335390+0000","last_undegraded":"2024-09-21T03:44:58.335390+0000","last_fullsized":"2024-09-21T03:44:58.335390+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_clean_scrub_stamp":"2024-09-21T03:44:55.304777+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:36:44.258406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.915449+0000","last_change":"2024-09-21T03:44:57.352611+0000","last_active":"2024-09-21T03:44:58.915449+0000","last_peered":"2024-09-21T03:44:58.915449+0000","last_clean":"2024-09-21T03:44:58.915449+0000","last_became_active":"2024-09-21T03:44:57.352136+0000","last_became_peered":"2024-09-21T03:44:57.352136+0000","last_unstale":"2024-09-21T03:44:58.915449+0000","last_undegraded":"2024-09-21T03:44:58.915449+0000","last_fullsized":"2024-09-21T03:44:58.915449+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:26:06.882445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":15,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.338141+0000","last_change":"2024-09-21T03:44:56.337958+0000","last_active":"2024-09-21T03:44:58.338141+0000","last_peered":"2024-09-21T03:44:58.338141+0000","last_clean":"2024-09-21T03:44:58.338141+0000","last_became_active":"2024-09-21T03:44:56.337623+0000","last_became_peered":"2024-09-21T03:44:56.337623+0000","last_unstale":"2024-09-21T03:44:58.338141+0000","last_undegraded":"2024-09-21T03:44:58.338141+0000","last_fullsized":"2024-09-21T03:44:58.338141+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_clean_scrub_stamp":"2024-09-21T03:44:55.304777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:45:31.964595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.456704+0000","last_change":"2024-09-21T03:44:57.358895+0000","last_active":"2024-09-21T03:44:58.456704+0000","last_peered":"2024-09-21T03:44:58.456704+0000","last_clean":"2024-09-21T03:44:58.456704+0000","last_became_active":"2024-09-21T03:44:57.358591+0000","last_became_peered":"2024-09-21T03:44:57.358591+0000","last_unstale":"2024-09-21T03:44:58.456704+0000","last_undegraded":"2024-09-21T03:44:58.456704+0000","last_fullsized":"2024-09-21T03:44:58.456704+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:38:18.158452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":15,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.334691+0000","last_change":"2024-09-21T03:44:56.327693+0000","last_active":"2024-09-21T03:44:58.334691+0000","last_peered":"2024-09-21T03:44:58.334691+0000","last_clean":"2024-09-21T03:44:58.334691+0000","last_became_active":"2024-09-21T03:44:56.327364+0000","last_became_peered":"2024-09-21T03:44:56.327364+0000","last_unstale":"2024-09-21T03:44:58.334691+0000","last_undegraded":"2024-09-21T03:44:58.334691+0000","last_fullsized":"2024-09-21T03:44:58.334691+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_clean_scrub_stamp":"2024-09-21T03:44:55.304777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:16:04.551826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.350325+0000","last_change":"2024-09-21T03:44:57.353000+0000","last_active":"2024-09-21T03:44:58.350325+0000","last_peered":"2024-09-21T03:44:58.350325+0000","last_clean":"2024-09-21T03:44:58.350325+0000","last_became_active":"2024-09-21T03:44:57.352632+0000","last_became_peered":"2024-09-21T03:44:57.352632+0000","last_unstale":"2024-09-21T03:44:58.350325+0000","last_undegraded":"2024-09-21T03:44:58.350325+0000","last_fullsized":"2024-09-21T03:44:58.350325+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:59:58.601180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.10","version":"145'8","reported_seq":23,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.337672+0000","last_change":"2024-09-21T03:44:56.342213+0000","last_active":"2024-09-21T03:44:58.337672+0000","last_peered":"2024-09-21T03:44:58.337672+0000","last_clean":"2024-09-21T03:44:58.337672+0000","last_became_active":"2024-09-21T03:44:56.341888+0000","last_became_peered":"2024-09-21T03:44:56.341888+0000","last_unstale":"2024-09-21T03:44:58.337672+0000","last_undegraded":"2024-09-21T03:44:58.337672+0000","last_fullsized":"2024-09-21T03:44:58.337672+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_clean_scrub_stamp":"2024-09-21T03:44:55.304777+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:36:05.171778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.338090+0000","last_change":"2024-09-21T03:44:57.350536+0000","last_active":"2024-09-21T03:44:58.338090+0000","last_peered":"2024-09-21T03:44:58.338090+0000","last_clean":"2024-09-21T03:44:58.338090+0000","last_became_active":"2024-09-21T03:44:57.349795+0000","last_became_peered":"2024-09-21T03:44:57.349795+0000","last_unstale":"2024-09-21T03:44:58.338090+0000","last_undegraded":"2024-09-21T03:44:58.338090+0000","last_fullsized":"2024-09-21T03:44:58.338090+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:32:59.029041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.f","version":"145'1","reported_seq":16,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.353274+0000","last_change":"2024-09-21T03:44:56.339856+0000","last_active":"2024-09-21T03:44:58.353274+0000","last_peered":"2024-09-21T03:44:58.353274+0000","last_clean":"2024-09-21T03:44:58.353274+0000","last_became_active":"2024-09-21T03:44:56.339305+0000","last_became_peered":"2024-09-21T03:44:56.339305+0000","last_unstale":"2024-09-21T03:44:58.353274+0000","last_undegraded":"2024-09-21T03:44:58.353274+0000","last_fullsized":"2024-09-21T03:44:58.353274+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_clean_scrub_stamp":"2024-09-21T03:44:55.304777+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:37:25.372042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.338656+0000","last_change":"2024-09-21T03:44:57.341004+0000","last_active":"2024-09-21T03:44:58.338656+0000","last_peered":"2024-09-21T03:44:58.338656+0000","last_clean":"2024-09-21T03:44:58.338656+0000","last_became_active":"2024-09-21T03:44:57.340620+0000","last_became_peered":"2024-09-21T03:44:57.340620+0000","last_unstale":"2024-09-21T03:44:58.338656+0000","last_undegraded":"2024-09-21T03:44:58.338656+0000","last_fullsized":"2024-09-21T03:44:58.338656+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:09:01.018690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.d","version":"145'1","reported_seq":16,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.339122+0000","last_change":"2024-09-21T03:44:56.340547+0000","last_active":"2024-09-21T03:44:58.339122+0000","last_peered":"2024-09-21T03:44:58.339122+0000","last_clean":"2024-09-21T03:44:58.339122+0000","last_became_active":"2024-09-21T03:44:56.340237+0000","last_became_peered":"2024-09-21T03:44:56.340237+0000","last_unstale":"2024-09-21T03:44:58.339122+0000","last_undegraded":"2024-09-21T03:44:58.339122+0000","last_fullsized":"2024-09-21T03:44:58.339122+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_clean_scrub_stamp":"2024-09-21T03:44:55.304777+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:19:41.082645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.456321+0000","last_change":"2024-09-21T03:44:57.358052+0000","last_active":"2024-09-21T03:44:58.456321+0000","last_peered":"2024-09-21T03:44:58.456321+0000","last_clean":"2024-09-21T03:44:58.456321+0000","last_became_active":"2024-09-21T03:44:57.357731+0000","last_became_peered":"2024-09-21T03:44:57.357731+0000","last_unstale":"2024-09-21T03:44:58.456321+0000","last_undegraded":"2024-09-21T03:44:58.456321+0000","last_fullsized":"2024-09-21T03:44:58.456321+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:02:38.958342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":15,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.353016+0000","last_change":"2024-09-21T03:44:56.339649+0000","last_active":"2024-09-21T03:44:58.353016+0000","last_peered":"2024-09-21T03:44:58.353016+0000","last_clean":"2024-09-21T03:44:58.353016+0000","last_became_active":"2024-09-21T03:44:56.339317+0000","last_became_peered":"2024-09-21T03:44:56.339317+0000","last_unstale":"2024-09-21T03:44:58.353016+0000","last_undegraded":"2024-09-21T03:44:58.353016+0000","last_fullsized":"2024-09-21T03:44:58.353016+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_clean_scrub_stamp":"2024-09-21T03:44:55.304777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:58:28.485819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.335686+0000","last_change":"2024-09-21T03:44:57.350478+0000","last_active":"2024-09-21T03:44:58.335686+0000","last_peered":"2024-09-21T03:44:58.335686+0000","last_clean":"2024-09-21T03:44:58.335686+0000","last_became_active":"2024-09-21T03:44:57.348990+0000","last_became_peered":"2024-09-21T03:44:57.348990+0000","last_unstale":"2024-09-21T03:44:58.335686+0000","last_undegraded":"2024-09-21T03:44:58.335686+0000","last_fullsized":"2024-09-21T03:44:58.335686+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:05:31.071185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.b","version":"145'2","reported_seq":17,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.349943+0000","last_change":"2024-09-21T03:44:56.329417+0000","last_active":"2024-09-21T03:44:58.349943+0000","last_peered":"2024-09-21T03:44:58.349943+0000","last_clean":"2024-09-21T03:44:58.349943+0000","last_became_active":"2024-09-21T03:44:56.329098+0000","last_became_peered":"2024-09-21T03:44:56.329098+0000","last_unstale":"2024-09-21T03:44:58.349943+0000","last_undegraded":"2024-09-21T03:44:58.349943+0000","last_fullsized":"2024-09-21T03:44:58.349943+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_clean_scrub_stamp":"2024-09-21T03:44:55.304777+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:47:42.275460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.456620+0000","last_change":"2024-09-21T03:44:57.354679+0000","last_active":"2024-09-21T03:44:58.456620+0000","last_peered":"2024-09-21T03:44:58.456620+0000","last_clean":"2024-09-21T03:44:58.456620+0000","last_became_active":"2024-09-21T03:44:57.354357+0000","last_became_peered":"2024-09-21T03:44:57.354357+0000","last_unstale":"2024-09-21T03:44:58.456620+0000","last_undegraded":"2024-09-21T03:44:58.456620+0000","last_fullsized":"2024-09-21T03:44:58.456620+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:29:19.509171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":15,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.349623+0000","last_change":"2024-09-21T03:44:56.330054+0000","last_active":"2024-09-21T03:44:58.349623+0000","last_peered":"2024-09-21T03:44:58.349623+0000","last_clean":"2024-09-21T03:44:58.349623+0000","last_became_active":"2024-09-21T03:44:56.329151+0000","last_became_peered":"2024-09-21T03:44:56.329151+0000","last_unstale":"2024-09-21T03:44:58.349623+0000","last_undegraded":"2024-09-21T03:44:58.349623+0000","last_fullsized":"2024-09-21T03:44:58.349623+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_clean_scrub_stamp":"2024-09-21T03:44:55.304777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:45:12.907491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.456652+0000","last_change":"2024-09-21T03:44:57.363634+0000","last_active":"2024-09-21T03:44:58.456652+0000","last_peered":"2024-09-21T03:44:58.456652+0000","last_clean":"2024-09-21T03:44:58.456652+0000","last_became_active":"2024-09-21T03:44:57.363259+0000","last_became_peered":"2024-09-21T03:44:57.363259+0000","last_unstale":"2024-09-21T03:44:58.456652+0000","last_undegraded":"2024-09-21T03:44:58.456652+0000","last_fullsized":"2024-09-21T03:44:58.456652+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:53:39.529389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":15,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.353119+0000","last_change":"2024-09-21T03:44:56.327686+0000","last_active":"2024-09-21T03:44:58.353119+0000","last_peered":"2024-09-21T03:44:58.353119+0000","last_clean":"2024-09-21T03:44:58.353119+0000","last_became_active":"2024-09-21T03:44:56.327355+0000","last_became_peered":"2024-09-21T03:44:56.327355+0000","last_unstale":"2024-09-21T03:44:58.353119+0000","last_undegraded":"2024-09-21T03:44:58.353119+0000","last_fullsized":"2024-09-21T03:44:58.353119+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_clean_scrub_stamp":"2024-09-21T03:44:55.304777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:26:58.029665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.336394+0000","last_change":"2024-09-21T03:44:57.351050+0000","last_active":"2024-09-21T03:44:58.336394+0000","last_peered":"2024-09-21T03:44:58.336394+0000","last_clean":"2024-09-21T03:44:58.336394+0000","last_became_active":"2024-09-21T03:44:57.350731+0000","last_became_peered":"2024-09-21T03:44:57.350731+0000","last_unstale":"2024-09-21T03:44:58.336394+0000","last_undegraded":"2024-09-21T03:44:58.336394+0000","last_fullsized":"2024-09-21T03:44:58.336394+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:45:07.755917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":15,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.338259+0000","last_change":"2024-09-21T03:44:56.334506+0000","last_active":"2024-09-21T03:44:58.338259+0000","last_peered":"2024-09-21T03:44:58.338259+0000","last_clean":"2024-09-21T03:44:58.338259+0000","last_became_active":"2024-09-21T03:44:56.334007+0000","last_became_peered":"2024-09-21T03:44:56.334007+0000","last_unstale":"2024-09-21T03:44:58.338259+0000","last_undegraded":"2024-09-21T03:44:58.338259+0000","last_fullsized":"2024-09-21T03:44:58.338259+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_clean_scrub_stamp":"2024-09-21T03:44:55.304777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:57:15.575727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.456155+0000","last_change":"2024-09-21T03:44:57.350289+0000","last_active":"2024-09-21T03:44:58.456155+0000","last_peered":"2024-09-21T03:44:58.456155+0000","last_clean":"2024-09-21T03:44:58.456155+0000","last_became_active":"2024-09-21T03:44:57.349932+0000","last_became_peered":"2024-09-21T03:44:57.349932+0000","last_unstale":"2024-09-21T03:44:58.456155+0000","last_undegraded":"2024-09-21T03:44:58.456155+0000","last_fullsized":"2024-09-21T03:44:58.456155+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:49:41.592146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.7","version":"145'2","reported_seq":17,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.350343+0000","last_change":"2024-09-21T03:44:56.328517+0000","last_active":"2024-09-21T03:44:58.350343+0000","last_peered":"2024-09-21T03:44:58.350343+0000","last_clean":"2024-09-21T03:44:58.350343+0000","last_became_active":"2024-09-21T03:44:56.328048+0000","last_became_peered":"2024-09-21T03:44:56.328048+0000","last_unstale":"2024-09-21T03:44:58.350343+0000","last_undegraded":"2024-09-21T03:44:58.350343+0000","last_fullsized":"2024-09-21T03:44:58.350343+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_clean_scrub_stamp":"2024-09-21T03:44:55.304777+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:09:01.750079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.337538+0000","last_change":"2024-09-21T03:44:57.359324+0000","last_active":"2024-09-21T03:44:58.337538+0000","last_peered":"2024-09-21T03:44:58.337538+0000","last_clean":"2024-09-21T03:44:58.337538+0000","last_became_active":"2024-09-21T03:44:57.358966+0000","last_became_peered":"2024-09-21T03:44:57.358966+0000","last_unstale":"2024-09-21T03:44:58.337538+0000","last_undegraded":"2024-09-21T03:44:58.337538+0000","last_fullsized":"2024-09-21T03:44:58.337538+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:30:55.516774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.6","version":"146'1","reported_seq":16,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.350153+0000","last_change":"2024-09-21T03:44:56.336489+0000","last_active":"2024-09-21T03:44:58.350153+0000","last_peered":"2024-09-21T03:44:58.350153+0000","last_clean":"2024-09-21T03:44:58.350153+0000","last_became_active":"2024-09-21T03:44:56.336089+0000","last_became_peered":"2024-09-21T03:44:56.336089+0000","last_unstale":"2024-09-21T03:44:58.350153+0000","last_undegraded":"2024-09-21T03:44:58.350153+0000","last_fullsized":"2024-09-21T03:44:58.350153+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_clean_scrub_stamp":"2024-09-21T03:44:55.304777+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:40:40.070208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.337864+0000","last_change":"2024-09-21T03:44:57.355765+0000","last_active":"2024-09-21T03:44:58.337864+0000","last_peered":"2024-09-21T03:44:58.337864+0000","last_clean":"2024-09-21T03:44:58.337864+0000","last_became_active":"2024-09-21T03:44:57.355383+0000","last_became_peered":"2024-09-21T03:44:57.355383+0000","last_unstale":"2024-09-21T03:44:58.337864+0000","last_undegraded":"2024-09-21T03:44:58.337864+0000","last_fullsized":"2024-09-21T03:44:58.337864+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:45:30.409863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":15,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.337802+0000","last_change":"2024-09-21T03:44:56.338420+0000","last_active":"2024-09-21T03:44:58.337802+0000","last_peered":"2024-09-21T03:44:58.337802+0000","last_clean":"2024-09-21T03:44:58.337802+0000","last_became_active":"2024-09-21T03:44:56.337865+0000","last_became_peered":"2024-09-21T03:44:56.337865+0000","last_unstale":"2024-09-21T03:44:58.337802+0000","last_undegraded":"2024-09-21T03:44:58.337802+0000","last_fullsized":"2024-09-21T03:44:58.337802+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_clean_scrub_stamp":"2024-09-21T03:44:55.304777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:40:26.986852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.339010+0000","last_change":"2024-09-21T03:44:57.337661+0000","last_active":"2024-09-21T03:44:58.339010+0000","last_peered":"2024-09-21T03:44:58.339010+0000","last_clean":"2024-09-21T03:44:58.339010+0000","last_became_active":"2024-09-21T03:44:57.337356+0000","last_became_peered":"2024-09-21T03:44:57.337356+0000","last_unstale":"2024-09-21T03:44:58.339010+0000","last_undegraded":"2024-09-21T03:44:58.339010+0000","last_fullsized":"2024-09-21T03:44:58.339010+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:19:49.944906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":15,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.336114+0000","last_change":"2024-09-21T03:44:56.337786+0000","last_active":"2024-09-21T03:44:58.336114+0000","last_peered":"2024-09-21T03:44:58.336114+0000","last_clean":"2024-09-21T03:44:58.336114+0000","last_became_active":"2024-09-21T03:44:56.337258+0000","last_became_peered":"2024-09-21T03:44:56.337258+0000","last_unstale":"2024-09-21T03:44:58.336114+0000","last_undegraded":"2024-09-21T03:44:58.336114+0000","last_fullsized":"2024-09-21T03:44:58.336114+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_clean_scrub_stamp":"2024-09-21T03:44:55.304777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:29:06.273027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.335568+0000","last_change":"2024-09-21T03:44:57.350221+0000","last_active":"2024-09-21T03:44:58.335568+0000","last_peered":"2024-09-21T03:44:58.335568+0000","last_clean":"2024-09-21T03:44:58.335568+0000","last_became_active":"2024-09-21T03:44:57.349707+0000","last_became_peered":"2024-09-21T03:44:57.349707+0000","last_unstale":"2024-09-21T03:44:58.335568+0000","last_undegraded":"2024-09-21T03:44:58.335568+0000","last_fullsized":"2024-09-21T03:44:58.335568+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:30:55.516774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":15,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.337614+0000","last_change":"2024-09-21T03:44:56.340358+0000","last_active":"2024-09-21T03:44:58.337614+0000","last_peered":"2024-09-21T03:44:58.337614+0000","last_clean":"2024-09-21T03:44:58.337614+0000","last_became_active":"2024-09-21T03:44:56.340044+0000","last_became_peered":"2024-09-21T03:44:56.340044+0000","last_unstale":"2024-09-21T03:44:58.337614+0000","last_undegraded":"2024-09-21T03:44:58.337614+0000","last_fullsized":"2024-09-21T03:44:58.337614+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_clean_scrub_stamp":"2024-09-21T03:44:55.304777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:50:56.785756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.336113+0000","last_change":"2024-09-21T03:44:57.348196+0000","last_active":"2024-09-21T03:44:58.336113+0000","last_peered":"2024-09-21T03:44:58.336113+0000","last_clean":"2024-09-21T03:44:58.336113+0000","last_became_active":"2024-09-21T03:44:57.347856+0000","last_became_peered":"2024-09-21T03:44:57.347856+0000","last_unstale":"2024-09-21T03:44:58.336113+0000","last_undegraded":"2024-09-21T03:44:58.336113+0000","last_fullsized":"2024-09-21T03:44:58.336113+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:43:22.802681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":15,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.349722+0000","last_change":"2024-09-21T03:44:56.337049+0000","last_active":"2024-09-21T03:44:58.349722+0000","last_peered":"2024-09-21T03:44:58.349722+0000","last_clean":"2024-09-21T03:44:58.349722+0000","last_became_active":"2024-09-21T03:44:56.336315+0000","last_became_peered":"2024-09-21T03:44:56.336315+0000","last_unstale":"2024-09-21T03:44:58.349722+0000","last_undegraded":"2024-09-21T03:44:58.349722+0000","last_fullsized":"2024-09-21T03:44:58.349722+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_clean_scrub_stamp":"2024-09-21T03:44:55.304777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:46:17.695474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.353229+0000","last_change":"2024-09-21T03:44:57.366623+0000","last_active":"2024-09-21T03:44:58.353229+0000","last_peered":"2024-09-21T03:44:58.353229+0000","last_clean":"2024-09-21T03:44:58.353229+0000","last_became_active":"2024-09-21T03:44:57.366298+0000","last_became_peered":"2024-09-21T03:44:57.366298+0000","last_unstale":"2024-09-21T03:44:58.353229+0000","last_undegraded":"2024-09-21T03:44:58.353229+0000","last_fullsized":"2024-09-21T03:44:58.353229+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:01:23.359578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":15,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.338658+0000","last_change":"2024-09-21T03:44:56.339692+0000","last_active":"2024-09-21T03:44:58.338658+0000","last_peered":"2024-09-21T03:44:58.338658+0000","last_clean":"2024-09-21T03:44:58.338658+0000","last_became_active":"2024-09-21T03:44:56.339376+0000","last_became_peered":"2024-09-21T03:44:56.339376+0000","last_unstale":"2024-09-21T03:44:58.338658+0000","last_undegraded":"2024-09-21T03:44:58.338658+0000","last_fullsized":"2024-09-21T03:44:58.338658+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_clean_scrub_stamp":"2024-09-21T03:44:55.304777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:17:10.658062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.915523+0000","last_change":"2024-09-21T03:44:57.356308+0000","last_active":"2024-09-21T03:44:58.915523+0000","last_peered":"2024-09-21T03:44:58.915523+0000","last_clean":"2024-09-21T03:44:58.915523+0000","last_became_active":"2024-09-21T03:44:57.355511+0000","last_became_peered":"2024-09-21T03:44:57.355511+0000","last_unstale":"2024-09-21T03:44:58.915523+0000","last_undegraded":"2024-09-21T03:44:58.915523+0000","last_fullsized":"2024-09-21T03:44:58.915523+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:08:21.639028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":15,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.349691+0000","last_change":"2024-09-21T03:44:56.341499+0000","last_active":"2024-09-21T03:44:58.349691+0000","last_peered":"2024-09-21T03:44:58.349691+0000","last_clean":"2024-09-21T03:44:58.349691+0000","last_became_active":"2024-09-21T03:44:56.341058+0000","last_became_peered":"2024-09-21T03:44:56.341058+0000","last_unstale":"2024-09-21T03:44:58.349691+0000","last_undegraded":"2024-09-21T03:44:58.349691+0000","last_fullsized":"2024-09-21T03:44:58.349691+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_clean_scrub_stamp":"2024-09-21T03:44:55.304777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:58:37.670658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.352897+0000","last_change":"2024-09-21T03:44:57.355188+0000","last_active":"2024-09-21T03:44:58.352897+0000","last_peered":"2024-09-21T03:44:58.352897+0000","last_clean":"2024-09-21T03:44:58.352897+0000","last_became_active":"2024-09-21T03:44:57.354477+0000","last_became_peered":"2024-09-21T03:44:57.354477+0000","last_unstale":"2024-09-21T03:44:58.352897+0000","last_undegraded":"2024-09-21T03:44:58.352897+0000","last_fullsized":"2024-09-21T03:44:58.352897+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:56.311610+0000","last_clean_scrub_stamp":"2024-09-21T03:44:56.311610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:28:53.047166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"145'2","reported_seq":17,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.335737+0000","last_change":"2024-09-21T03:44:56.335259+0000","last_active":"2024-09-21T03:44:58.335737+0000","last_peered":"2024-09-21T03:44:58.335737+0000","last_clean":"2024-09-21T03:44:58.335737+0000","last_became_active":"2024-09-21T03:44:56.334868+0000","last_became_peered":"2024-09-21T03:44:56.334868+0000","last_unstale":"2024-09-21T03:44:58.335737+0000","last_undegraded":"2024-09-21T03:44:58.335737+0000","last_fullsized":"2024-09-21T03:44:58.335737+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:55.304777+0000","last_clean_scrub_stamp":"2024-09-21T03:44:55.304777+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:45:29.403030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":355,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-21T03:44:58.338537+0000","last_change":"2024-09-21T03:33:25.319776+0000","last_active":"2024-09-21T03:44:58.338537+0000","last_peered":"2024-09-21T03:44:58.338537+0000","last_clean":"2024-09-21T03:44:58.338537+0000","last_became_active":"2024-09-21T03:33:25.318399+0000","last_became_peered":"2024-09-21T03:33:25.318399+0000","last_unstale":"2024-09-21T03:44:58.338537+0000","last_undegraded":"2024-09-21T03:44:58.338537+0000","last_fullsized":"2024-09-21T03:44:58.338537+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:23.016672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:23.016672+0000","last_clean_scrub_stamp":"2024-09-21T03:33:23.016672+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:32:35.906114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":1,"up_from":16,"seq":68719476880,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41936,"kb_used_data":2428,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94329904,"statfs":{"total":96636764160,"available":96593821696,"internally_reserved":0,"allocated":2486272,"data_stored":1546129,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509584,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41356,"kb_used_data":1848,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330484,"statfs":{"total":96636764160,"available":96594415616,"internally_reserved":0,"allocated":1892352,"data_stored":956321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":40434123},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509583,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41364,"kb_used_data":1852,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330476,"statfs":{"total":96636764160,"available":96594407424,"internally_reserved":0,"allocated":1896448,"data_stored":957281,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509584,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37200,"kb_used_data":1848,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334640,"statfs":{"total":96636764160,"available":96598671360,"internally_reserved":0,"allocated":1892352,"data_stored":955781,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509583,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37196,"kb_used_data":1844,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334644,"statfs":{"total":96636764160,"available":96598675456,"internally_reserved":0,"allocated":1888256,"data_stored":956251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542287,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33044,"kb_used_data":1852,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338796,"statfs":{"total":96636764160,"available":96602927104,"internally_reserved":0,"allocated":1896448,"data_stored":957281,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574991,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37784,"kb_used_data":2428,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334056,"statfs":{"total":96636764160,"available":96598073344,"internally_reserved":0,"allocated":2486272,"data_stored":1546161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542288,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37196,"kb_used_data":1844,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334644,"statfs":{"total":96636764160,"available":96598675456,"internally_reserved":0,"allocated":1888256,"data_stored":955703,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T03:45:06.322 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-21T03:45:06.322 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-21T03:45:06.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:06.480+0000 7f4e39cf9640 1 -- 172.21.15.38:0/490388739 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4e340cea90 msgr2=0x7f4e340d00d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:06.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:06.480+0000 7f4e39cf9640 1 --2- 172.21.15.38:0/490388739 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4e340cea90 0x7f4e340d00d0 secure :-1 s=READY pgs=2377 cs=0 l=1 rev1=1 crypto rx=0x7f4e340ba5b0 tx=0x7f4e2402a4b0 comp rx=0 tx=0).stop 2024-09-21T03:45:06.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:06.480+0000 7f4e39cf9640 1 -- 172.21.15.38:0/490388739 shutdown_connections 2024-09-21T03:45:06.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:06.480+0000 7f4e39cf9640 1 --2- 172.21.15.38:0/490388739 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4e340d3360 0x7f4e340d3770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:06.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:06.480+0000 7f4e39cf9640 1 --2- 172.21.15.38:0/490388739 >> v2:172.21.15.195:3300/0 conn(0x7f4e34162790 0x7f4e34162ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:06.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:06.480+0000 7f4e39cf9640 1 --2- 172.21.15.38:0/490388739 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4e340cea90 0x7f4e340d00d0 unknown :-1 s=CLOSED pgs=2377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:06.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:06.480+0000 7f4e39cf9640 1 -- 172.21.15.38:0/490388739 >> 172.21.15.38:0/490388739 conn(0x7f4e340d4620 msgr2=0x7f4e340d4a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:06.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:06.480+0000 7f4e39cf9640 1 -- 172.21.15.38:0/490388739 shutdown_connections 2024-09-21T03:45:06.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:06.480+0000 7f4e39cf9640 1 -- 172.21.15.38:0/490388739 wait complete. 2024-09-21T03:45:06.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:06.482+0000 7f4e39cf9640 1 Processor -- start 2024-09-21T03:45:06.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:06.482+0000 7f4e39cf9640 1 -- start start 2024-09-21T03:45:06.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:06.483+0000 7f4e39cf9640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4e340cea90 0x7f4e34154da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:06.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:06.483+0000 7f4e39cf9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4e340d3360 0x7f4e341f9c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:06.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:06.483+0000 7f4e39cf9640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4e34162790 0x7f4e3415bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:06.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:06.483+0000 7f4e39cf9640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f4e340b3f70 con 0x7f4e340d3360 2024-09-21T03:45:06.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:06.483+0000 7f4e39cf9640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f4e340b3df0 con 0x7f4e340cea90 2024-09-21T03:45:06.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:06.483+0000 7f4e39cf9640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f4e340b40f0 con 0x7f4e34162790 2024-09-21T03:45:06.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:06.484+0000 7f4e32ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4e340d3360 0x7f4e341f9c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:06.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:06.484+0000 7f4e32ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4e340d3360 0x7f4e341f9c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43534/0 (socket says 172.21.15.38:43534) 2024-09-21T03:45:06.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:06.484+0000 7f4e32ffd640 1 -- 172.21.15.38:0/3736392011 learned_addr learned my addr 172.21.15.38:0/3736392011 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:45:06.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:06.484+0000 7f4e337fe640 1 --2- 172.21.15.38:0/3736392011 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4e340cea90 0x7f4e34154da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:06.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:06.484+0000 7f4e33fff640 1 --2- 172.21.15.38:0/3736392011 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4e34162790 0x7f4e3415bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:06.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:06.484+0000 7f4e32ffd640 1 -- 172.21.15.38:0/3736392011 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4e34162790 msgr2=0x7f4e3415bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:06.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:06.484+0000 7f4e32ffd640 1 --2- 172.21.15.38:0/3736392011 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4e34162790 0x7f4e3415bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:06.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:06.484+0000 7f4e32ffd640 1 -- 172.21.15.38:0/3736392011 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4e340cea90 msgr2=0x7f4e34154da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:06.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:06.484+0000 7f4e32ffd640 1 --2- 172.21.15.38:0/3736392011 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4e340cea90 0x7f4e34154da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:06.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:06.484+0000 7f4e32ffd640 1 -- 172.21.15.38:0/3736392011 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e34159040 con 0x7f4e340d3360 2024-09-21T03:45:06.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:06.484+0000 7f4e33fff640 1 --2- 172.21.15.38:0/3736392011 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4e34162790 0x7f4e3415bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:45:06.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:06.484+0000 7f4e32ffd640 1 --2- 172.21.15.38:0/3736392011 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4e340d3360 0x7f4e341f9c80 secure :-1 s=READY pgs=2378 cs=0 l=1 rev1=1 crypto rx=0x7f4e2800c9a0 tx=0x7f4e2800ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:06.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:06.484+0000 7f4e1bfff640 1 -- 172.21.15.38:0/3736392011 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e28007bb0 con 0x7f4e340d3360 2024-09-21T03:45:06.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:06.484+0000 7f4e1bfff640 1 -- 172.21.15.38:0/3736392011 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4e28007e40 con 0x7f4e340d3360 2024-09-21T03:45:06.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:06.484+0000 7f4e39cf9640 1 -- 172.21.15.38:0/3736392011 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e34159970 con 0x7f4e340d3360 2024-09-21T03:45:06.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:06.484+0000 7f4e1bfff640 1 -- 172.21.15.38:0/3736392011 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e2800f6b0 con 0x7f4e340d3360 2024-09-21T03:45:06.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:06.484+0000 7f4e39cf9640 1 -- 172.21.15.38:0/3736392011 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e34159f30 con 0x7f4e340d3360 2024-09-21T03:45:06.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:06.485+0000 7f4e39cf9640 1 -- 172.21.15.38:0/3736392011 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e340b4610 con 0x7f4e340d3360 2024-09-21T03:45:06.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:06.486+0000 7f4e1bfff640 1 -- 172.21.15.38:0/3736392011 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f4e2800f850 con 0x7f4e340d3360 2024-09-21T03:45:06.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:06.487+0000 7f4e1bfff640 1 --2- 172.21.15.38:0/3736392011 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4e20078b90 0x7f4e2007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:06.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:06.487+0000 7f4e1bfff640 1 -- 172.21.15.38:0/3736392011 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 6101+0+0 (secure 0 0 0) 0x7f4e2809b6b0 con 0x7f4e340d3360 2024-09-21T03:45:06.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:06.487+0000 7f4e337fe640 1 --2- 172.21.15.38:0/3736392011 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4e20078b90 0x7f4e2007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:06.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:06.488+0000 7f4e337fe640 1 --2- 172.21.15.38:0/3736392011 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4e20078b90 0x7f4e2007b0a0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f4e24034000 tx=0x7f4e24004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:06.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:06.491+0000 7f4e1bfff640 1 -- 172.21.15.38:0/3736392011 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f4e28014030 con 0x7f4e340d3360 2024-09-21T03:45:06.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:06.750+0000 7f4e39cf9640 1 -- 172.21.15.38:0/3736392011 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f4e34158dd0 con 0x7f4e340d3360 2024-09-21T03:45:06.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:06.754+0000 7f4e1bfff640 1 -- 172.21.15.38:0/3736392011 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v24) ==== 190+0+147 (secure 0 0 0) 0x7f4e28063970 con 0x7f4e340d3360 2024-09-21T03:45:06.756 INFO:teuthology.orchestra.run.smithi038.stdout:[client.0] 2024-09-21T03:45:06.756 INFO:teuthology.orchestra.run.smithi038.stdout: key = AQBQPu5m6ydBDxAAlWZ2XzQTPBcYPriPGRJVeA== 2024-09-21T03:45:06.757 INFO:teuthology.orchestra.run.smithi038.stdout: caps mds = "allow" 2024-09-21T03:45:06.757 INFO:teuthology.orchestra.run.smithi038.stdout: caps mon = "allow r" 2024-09-21T03:45:06.757 INFO:teuthology.orchestra.run.smithi038.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-21T03:45:06.757 INFO:teuthology.orchestra.run.smithi038.stderr:updated caps for client.0 2024-09-21T03:45:06.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:06.756+0000 7f4e39cf9640 1 -- 172.21.15.38:0/3736392011 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4e20078b90 msgr2=0x7f4e2007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:06.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:06.756+0000 7f4e39cf9640 1 --2- 172.21.15.38:0/3736392011 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4e20078b90 0x7f4e2007b0a0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f4e24034000 tx=0x7f4e24004730 comp rx=0 tx=0).stop 2024-09-21T03:45:06.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:06.756+0000 7f4e39cf9640 1 -- 172.21.15.38:0/3736392011 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4e340d3360 msgr2=0x7f4e341f9c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:06.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:06.756+0000 7f4e39cf9640 1 --2- 172.21.15.38:0/3736392011 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4e340d3360 0x7f4e341f9c80 secure :-1 s=READY pgs=2378 cs=0 l=1 rev1=1 crypto rx=0x7f4e2800c9a0 tx=0x7f4e2800ce60 comp rx=0 tx=0).stop 2024-09-21T03:45:06.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:06.756+0000 7f4e39cf9640 1 -- 172.21.15.38:0/3736392011 shutdown_connections 2024-09-21T03:45:06.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:06.756+0000 7f4e39cf9640 1 --2- 172.21.15.38:0/3736392011 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4e20078b90 0x7f4e2007b0a0 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:06.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:06.756+0000 7f4e39cf9640 1 --2- 172.21.15.38:0/3736392011 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4e34162790 0x7f4e3415bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:06.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:06.756+0000 7f4e39cf9640 1 --2- 172.21.15.38:0/3736392011 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4e340d3360 0x7f4e341f9c80 unknown :-1 s=CLOSED pgs=2378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:06.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:06.756+0000 7f4e39cf9640 1 --2- 172.21.15.38:0/3736392011 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4e340cea90 0x7f4e34154da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:06.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:06.756+0000 7f4e39cf9640 1 -- 172.21.15.38:0/3736392011 >> 172.21.15.38:0/3736392011 conn(0x7f4e340d4620 msgr2=0x7f4e340b4820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:06.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:06.757+0000 7f4e39cf9640 1 -- 172.21.15.38:0/3736392011 shutdown_connections 2024-09-21T03:45:06.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:06.757+0000 7f4e39cf9640 1 -- 172.21.15.38:0/3736392011 wait complete. 2024-09-21T03:45:06.770 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:45:06.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:06.885+0000 7effc799b640 1 -- 172.21.15.38:0/2395043416 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7effac003a00 msgr2=0x7effac003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:06.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:06.885+0000 7effc799b640 1 --2- 172.21.15.38:0/2395043416 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7effac003a00 0x7effac003e70 secure :-1 s=READY pgs=2380 cs=0 l=1 rev1=1 crypto rx=0x7effbc009b30 tx=0x7effac005830 comp rx=0 tx=0).stop 2024-09-21T03:45:06.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:06.885+0000 7effc5f38640 1 -- 172.21.15.38:0/2395043416 reap_dead start 2024-09-21T03:45:06.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:06.885+0000 7effc799b640 1 -- 172.21.15.38:0/2395043416 shutdown_connections 2024-09-21T03:45:06.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:06.885+0000 7effc799b640 1 -- 172.21.15.38:0/2395043416 >> 172.21.15.38:0/2395043416 conn(0x7effc00d4440 msgr2=0x7effc00d4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:06.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:06.885+0000 7effc799b640 1 -- 172.21.15.38:0/2395043416 shutdown_connections 2024-09-21T03:45:06.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:06.885+0000 7effc799b640 1 -- 172.21.15.38:0/2395043416 wait complete. 2024-09-21T03:45:06.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:06.887+0000 7effc799b640 1 Processor -- start 2024-09-21T03:45:06.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:06.887+0000 7effc799b640 1 -- start start 2024-09-21T03:45:06.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:06.888+0000 7effc799b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7effac003a00 0x7effc0154e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:06.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:06.888+0000 7effc799b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7effc0155a40 0x7effc0160f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:06.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:06.888+0000 7effc799b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7effc0161440 0x7effc015a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:06.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:06.888+0000 7effc799b640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7effc00d2a20 con 0x7effc0161440 2024-09-21T03:45:06.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:06.888+0000 7effc799b640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7effc00b3b90 con 0x7effac003a00 2024-09-21T03:45:06.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:06.888+0000 7effc799b640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7effc00d2ba0 con 0x7effc0155a40 2024-09-21T03:45:06.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:06.889+0000 7effc5f38640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7effc0161440 0x7effc015a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:06.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:06.889+0000 7effc5f38640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7effc0161440 0x7effc015a870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43554/0 (socket says 172.21.15.38:43554) 2024-09-21T03:45:06.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:06.889+0000 7effc5f38640 1 -- 172.21.15.38:0/943386322 learned_addr learned my addr 172.21.15.38:0/943386322 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:45:06.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:06.889+0000 7effc5737640 1 --2- 172.21.15.38:0/943386322 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7effac003a00 0x7effc0154e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:06.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:06.889+0000 7effc4f36640 1 --2- 172.21.15.38:0/943386322 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7effc0155a40 0x7effc0160f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:06.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:06.889+0000 7effc5f38640 1 -- 172.21.15.38:0/943386322 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7effc0155a40 msgr2=0x7effc0160f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:06.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:06.889+0000 7effc5f38640 1 --2- 172.21.15.38:0/943386322 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7effc0155a40 0x7effc0160f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:06.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:06.889+0000 7effc5f38640 1 -- 172.21.15.38:0/943386322 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7effac003a00 msgr2=0x7effc0154e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:06.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:06.889+0000 7effc5f38640 1 --2- 172.21.15.38:0/943386322 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7effac003a00 0x7effc0154e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:06.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:06.889+0000 7effc5f38640 1 -- 172.21.15.38:0/943386322 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effc0161850 con 0x7effc0161440 2024-09-21T03:45:06.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:06.889+0000 7effc5f38640 1 --2- 172.21.15.38:0/943386322 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7effc0161440 0x7effc015a870 secure :-1 s=READY pgs=2381 cs=0 l=1 rev1=1 crypto rx=0x7effc00b5fc0 tx=0x7effb80064e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:06.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:06.889+0000 7effb5ffb640 1 -- 172.21.15.38:0/943386322 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effb8015070 con 0x7effc0161440 2024-09-21T03:45:06.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:06.889+0000 7effb5ffb640 1 -- 172.21.15.38:0/943386322 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7effb8010980 con 0x7effc0161440 2024-09-21T03:45:06.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:06.889+0000 7effc799b640 1 -- 172.21.15.38:0/943386322 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effc01f8cb0 con 0x7effc0161440 2024-09-21T03:45:06.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:06.889+0000 7effb5ffb640 1 -- 172.21.15.38:0/943386322 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effb8010dd0 con 0x7effc0161440 2024-09-21T03:45:06.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:06.890+0000 7effc799b640 1 -- 172.21.15.38:0/943386322 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effc01f9270 con 0x7effc0161440 2024-09-21T03:45:06.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:06.890+0000 7effc799b640 1 -- 172.21.15.38:0/943386322 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7effc00b37a0 con 0x7effc0161440 2024-09-21T03:45:06.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:06.891+0000 7effb5ffb640 1 -- 172.21.15.38:0/943386322 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7effb8018070 con 0x7effc0161440 2024-09-21T03:45:06.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:06.892+0000 7effb5ffb640 1 --2- 172.21.15.38:0/943386322 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7effa4078bc0 0x7effa407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:06.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:06.892+0000 7effb5ffb640 1 -- 172.21.15.38:0/943386322 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 6101+0+0 (secure 0 0 0) 0x7effb809b560 con 0x7effc0161440 2024-09-21T03:45:06.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:06.892+0000 7effc5737640 1 --2- 172.21.15.38:0/943386322 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7effa4078bc0 0x7effa407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:06.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:06.893+0000 7effc5737640 1 --2- 172.21.15.38:0/943386322 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7effa4078bc0 0x7effa407b0d0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7effbc009b30 tx=0x7effbc0046c0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:06.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:06.897+0000 7effb5ffb640 1 -- 172.21.15.38:0/943386322 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7effc00b37a0 con 0x7effc0161440 2024-09-21T03:45:07.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.130+0000 7effc799b640 1 -- 172.21.15.38:0/943386322 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7effc01629a0 con 0x7effc0161440 2024-09-21T03:45:07.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.130+0000 7effb5ffb640 1 -- 172.21.15.38:0/943386322 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 343 v343) ==== 95+0+5239 (secure 0 0 0) 0x7effc0161850 con 0x7effc0161440 2024-09-21T03:45:07.132 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:45:07.132 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":343,"btime":"2024-09-21T03:45:00:366373+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8399,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/3606957623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":3606957623},{"type":"v1","addr":"172.21.15.38:6835","nonce":3606957623}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":338},{"gid":8405,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/456651486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":456651486},{"type":"v1","addr":"172.21.15.195:6837","nonce":456651486}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":338},{"gid":9604,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/2649311840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":2649311840},{"type":"v1","addr":"172.21.15.195:6835","nonce":2649311840}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":338}],"filesystems":[{"mdsmap":{"epoch":343,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:44:56.710446+0000","modified":"2024-09-21T03:44:59.752336+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10719},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10719":{"gid":10719,"name":"c","rank":0,"incarnation":339,"state":"up:active","state_seq":3,"addr":"172.21.15.38:6833/3866292839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":3866292839},{"type":"v1","addr":"172.21.15.38:6833","nonce":3866292839}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10719,"qdb_cluster":[10719]},"id":7}]} 2024-09-21T03:45:07.133 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 343 2024-09-21T03:45:07.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.133+0000 7effc799b640 1 -- 172.21.15.38:0/943386322 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7effa4078bc0 msgr2=0x7effa407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:07.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.133+0000 7effc799b640 1 --2- 172.21.15.38:0/943386322 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7effa4078bc0 0x7effa407b0d0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7effbc009b30 tx=0x7effbc0046c0 comp rx=0 tx=0).stop 2024-09-21T03:45:07.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.133+0000 7effc799b640 1 -- 172.21.15.38:0/943386322 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7effc0161440 msgr2=0x7effc015a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:07.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.133+0000 7effc799b640 1 --2- 172.21.15.38:0/943386322 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7effc0161440 0x7effc015a870 secure :-1 s=READY pgs=2381 cs=0 l=1 rev1=1 crypto rx=0x7effc00b5fc0 tx=0x7effb80064e0 comp rx=0 tx=0).stop 2024-09-21T03:45:07.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.133+0000 7effc799b640 1 -- 172.21.15.38:0/943386322 shutdown_connections 2024-09-21T03:45:07.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.133+0000 7effc799b640 1 --2- 172.21.15.38:0/943386322 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7effc0161440 0x7effc015a870 unknown :-1 s=CLOSED pgs=2381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:07.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.133+0000 7effc799b640 1 --2- 172.21.15.38:0/943386322 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7effa4078bc0 0x7effa407b0d0 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:07.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.133+0000 7effc799b640 1 --2- 172.21.15.38:0/943386322 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7effc0155a40 0x7effc0160f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:07.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.133+0000 7effc799b640 1 --2- 172.21.15.38:0/943386322 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7effac003a00 0x7effc0154e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:07.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.133+0000 7effc799b640 1 -- 172.21.15.38:0/943386322 >> 172.21.15.38:0/943386322 conn(0x7effc00d4440 msgr2=0x7effc015f5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:07.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.133+0000 7effc799b640 1 -- 172.21.15.38:0/943386322 shutdown_connections 2024-09-21T03:45:07.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.133+0000 7effc799b640 1 -- 172.21.15.38:0/943386322 wait complete. 2024-09-21T03:45:07.549 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 343, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T03:44:56.710446+0000', 'modified': '2024-09-21T03:44:59.752336+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 10719}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_10719': {'gid': 10719, 'name': 'c', 'rank': 0, 'incarnation': 339, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.38:6833/3866292839', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.38:6832', 'nonce': 3866292839}, {'type': 'v1', 'addr': '172.21.15.38:6833', 'nonce': 3866292839}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10719, 'qdb_cluster': [10719]} 2024-09-21T03:45:07.549 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-21T03:45:07.550 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-21T03:45:07.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.671+0000 7fccabcce640 1 -- 172.21.15.38:0/2999844652 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcca40d0d80 msgr2=0x7fcca4167550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:07.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.671+0000 7fccabcce640 1 --2- 172.21.15.38:0/2999844652 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcca40d0d80 0x7fcca4167550 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fcc9c005fd0 tx=0x7fcc9c02a840 comp rx=0 tx=0).stop 2024-09-21T03:45:07.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.671+0000 7fccabcce640 1 -- 172.21.15.38:0/2999844652 shutdown_connections 2024-09-21T03:45:07.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.671+0000 7fccabcce640 1 --2- 172.21.15.38:0/2999844652 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcca40d0d80 0x7fcca4167550 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:07.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.671+0000 7fccabcce640 1 -- 172.21.15.38:0/2999844652 >> 172.21.15.38:0/2999844652 conn(0x7fcca40b46e0 msgr2=0x7fcca40d23e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:07.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.671+0000 7fccabcce640 1 -- 172.21.15.38:0/2999844652 shutdown_connections 2024-09-21T03:45:07.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.671+0000 7fccabcce640 1 -- 172.21.15.38:0/2999844652 wait complete. 2024-09-21T03:45:07.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.673+0000 7fccabcce640 1 Processor -- start 2024-09-21T03:45:07.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.673+0000 7fccabcce640 1 -- start start 2024-09-21T03:45:07.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.674+0000 7fccabcce640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcca40ccc40 0x7fcca4151830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:07.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.674+0000 7fccabcce640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcca415ca90 0x7fcca40c9980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:07.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.674+0000 7fccabcce640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcca40c9490 0x7fcca40c9ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:07.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.674+0000 7fccabcce640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fcca4167e20 con 0x7fcca40ccc40 2024-09-21T03:45:07.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.674+0000 7fccabcce640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fcca4167ca0 con 0x7fcca40c9490 2024-09-21T03:45:07.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.674+0000 7fccabcce640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fcca4167fa0 con 0x7fcca415ca90 2024-09-21T03:45:07.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.674+0000 7fcca9a6a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcca40ccc40 0x7fcca4151830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:07.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.674+0000 7fcca9a6a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcca40ccc40 0x7fcca4151830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43558/0 (socket says 172.21.15.38:43558) 2024-09-21T03:45:07.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.674+0000 7fcca9a6a640 1 -- 172.21.15.38:0/3168456424 learned_addr learned my addr 172.21.15.38:0/3168456424 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:45:07.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.675+0000 7fccaa26b640 1 --2- 172.21.15.38:0/3168456424 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcca40c9490 0x7fcca40c9ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:07.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.675+0000 7fcca9269640 1 --2- 172.21.15.38:0/3168456424 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcca415ca90 0x7fcca40c9980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:07.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.675+0000 7fcca9a6a640 1 -- 172.21.15.38:0/3168456424 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcca415ca90 msgr2=0x7fcca40c9980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:07.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.675+0000 7fcca9a6a640 1 --2- 172.21.15.38:0/3168456424 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcca415ca90 0x7fcca40c9980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:07.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.675+0000 7fcca9a6a640 1 -- 172.21.15.38:0/3168456424 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcca40c9490 msgr2=0x7fcca40c9ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:07.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.675+0000 7fcca9a6a640 1 --2- 172.21.15.38:0/3168456424 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcca40c9490 0x7fcca40c9ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:07.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.675+0000 7fcca9a6a640 1 -- 172.21.15.38:0/3168456424 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcca40cc230 con 0x7fcca40ccc40 2024-09-21T03:45:07.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.675+0000 7fcca9269640 1 --2- 172.21.15.38:0/3168456424 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcca415ca90 0x7fcca40c9980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:45:07.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.675+0000 7fcca9a6a640 1 --2- 172.21.15.38:0/3168456424 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcca40ccc40 0x7fcca4151830 secure :-1 s=READY pgs=2383 cs=0 l=1 rev1=1 crypto rx=0x7fcca000c430 tx=0x7fcca000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:07.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.675+0000 7fcc9a7fc640 1 -- 172.21.15.38:0/3168456424 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcca0017070 con 0x7fcca40ccc40 2024-09-21T03:45:07.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.675+0000 7fccabcce640 1 -- 172.21.15.38:0/3168456424 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcca415f290 con 0x7fcca40ccc40 2024-09-21T03:45:07.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.675+0000 7fcc9a7fc640 1 -- 172.21.15.38:0/3168456424 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcca00053c0 con 0x7fcca40ccc40 2024-09-21T03:45:07.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.675+0000 7fcc9a7fc640 1 -- 172.21.15.38:0/3168456424 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcca0003850 con 0x7fcca40ccc40 2024-09-21T03:45:07.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.675+0000 7fccabcce640 1 -- 172.21.15.38:0/3168456424 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcca4151d70 con 0x7fcca40ccc40 2024-09-21T03:45:07.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.677+0000 7fcc9a7fc640 1 -- 172.21.15.38:0/3168456424 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fcca001a070 con 0x7fcca40ccc40 2024-09-21T03:45:07.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.678+0000 7fcc9a7fc640 1 --2- 172.21.15.38:0/3168456424 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fcc88078b90 0x7fcc8807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:07.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.678+0000 7fcc9a7fc640 1 -- 172.21.15.38:0/3168456424 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 6101+0+0 (secure 0 0 0) 0x7fcca009a1f0 con 0x7fcca40ccc40 2024-09-21T03:45:07.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.678+0000 7fcca9269640 1 --2- 172.21.15.38:0/3168456424 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fcc88078b90 0x7fcc8807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:07.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.679+0000 7fccb2c36740 1 Processor -- start 2024-09-21T03:45:07.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.679+0000 7fcca9269640 1 --2- 172.21.15.38:0/3168456424 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fcc88078b90 0x7fcc8807b0a0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fcca40c8fb0 tx=0x7fcc94009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:07.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.679+0000 7fccb2c36740 1 -- start start 2024-09-21T03:45:07.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.680+0000 7fccb2c36740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56331bcb0270 0x56331c081fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:07.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.680+0000 7fccb2c36740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56331bc873b0 0x56331c0824e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:07.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.680+0000 7fccb2c36740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56331bc8f590 0x56331c09f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:07.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.680+0000 7fccb2c36740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x56331bfca820 con 0x56331bc8f590 2024-09-21T03:45:07.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.680+0000 7fccb2c36740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x56331bfb9c50 con 0x56331bc873b0 2024-09-21T03:45:07.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.680+0000 7fccb2c36740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x56331bfbda60 con 0x56331bcb0270 2024-09-21T03:45:07.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.680+0000 7fccaa26b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56331bc8f590 0x56331c09f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:07.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.680+0000 7fccaa26b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56331bc8f590 0x56331c09f3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43562/0 (socket says 172.21.15.38:43562) 2024-09-21T03:45:07.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.680+0000 7fccaa26b640 1 -- 172.21.15.38:0/3516645424 learned_addr learned my addr 172.21.15.38:0/3516645424 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:45:07.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.680+0000 7fcca9269640 1 --2- 172.21.15.38:0/3516645424 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56331bc873b0 0x56331c0824e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:07.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.680+0000 7fcca9a6a640 1 --2- 172.21.15.38:0/3516645424 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56331bcb0270 0x56331c081fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:07.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.681+0000 7fccaa26b640 1 -- 172.21.15.38:0/3516645424 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56331bcb0270 msgr2=0x56331c081fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:07.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.681+0000 7fccaa26b640 1 --2- 172.21.15.38:0/3516645424 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56331bcb0270 0x56331c081fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:07.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.681+0000 7fccaa26b640 1 -- 172.21.15.38:0/3516645424 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56331bc873b0 msgr2=0x56331c0824e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:07.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.681+0000 7fccaa26b640 1 --2- 172.21.15.38:0/3516645424 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56331bc873b0 0x56331c0824e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:07.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.681+0000 7fccaa26b640 1 -- 172.21.15.38:0/3516645424 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56331bd1b850 con 0x56331bc8f590 2024-09-21T03:45:07.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.681+0000 7fccaa26b640 1 --2- 172.21.15.38:0/3516645424 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56331bc8f590 0x56331c09f3e0 secure :-1 s=READY pgs=2384 cs=0 l=1 rev1=1 crypto rx=0x7fcc9c005fd0 tx=0x7fcc9c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b85ad9d7c7d7e90 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:07.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.681+0000 7fcc877fe640 1 -- 172.21.15.38:0/3516645424 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc9c041070 con 0x56331bc8f590 2024-09-21T03:45:07.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.681+0000 7fcc877fe640 1 -- 172.21.15.38:0/3516645424 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcc9c003d60 con 0x56331bc8f590 2024-09-21T03:45:07.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.681+0000 7fcc877fe640 1 -- 172.21.15.38:0/3516645424 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc9c004260 con 0x56331bc8f590 2024-09-21T03:45:07.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.682+0000 7fccb2c36740 1 -- 172.21.15.38:0/3516645424 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56331bc8f590 msgr2=0x56331c09f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:07.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.682+0000 7fccb2c36740 1 --2- 172.21.15.38:0/3516645424 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56331bc8f590 0x56331c09f3e0 secure :-1 s=READY pgs=2384 cs=0 l=1 rev1=1 crypto rx=0x7fcc9c005fd0 tx=0x7fcc9c004a40 comp rx=0 tx=0).stop 2024-09-21T03:45:07.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.682+0000 7fccb2c36740 1 -- 172.21.15.38:0/3516645424 shutdown_connections 2024-09-21T03:45:07.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.682+0000 7fccb2c36740 1 --2- 172.21.15.38:0/3516645424 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56331bc8f590 0x56331c09f3e0 unknown :-1 s=CLOSED pgs=2384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:07.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.682+0000 7fccb2c36740 1 --2- 172.21.15.38:0/3516645424 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56331bc873b0 0x56331c0824e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:07.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.682+0000 7fccb2c36740 1 --2- 172.21.15.38:0/3516645424 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56331bcb0270 0x56331c081fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:07.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.682+0000 7fccb2c36740 1 -- 172.21.15.38:0/3516645424 >> 172.21.15.38:0/3516645424 conn(0x56331bca43c0 msgr2=0x56331c058260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:07.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.682+0000 7fccb2c36740 1 -- 172.21.15.38:0/3516645424 shutdown_connections 2024-09-21T03:45:07.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.682+0000 7fccb2c36740 1 -- 172.21.15.38:0/3516645424 wait complete. 2024-09-21T03:45:07.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.683+0000 7fccb2c36740 1 -- start start 2024-09-21T03:45:07.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.683+0000 7fccb2c36740 1 Processor -- start 2024-09-21T03:45:07.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.684+0000 7fccb2c36740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56331bc873b0 0x56331c06ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:07.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.684+0000 7fccb2c36740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56331bc8f590 0x56331c097d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:07.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.684+0000 7fccb2c36740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56331bcb0270 0x56331c0965f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:07.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.684+0000 7fccb2c36740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x56331bd2a620 con 0x56331bc8f590 2024-09-21T03:45:07.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.684+0000 7fccb2c36740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x56331bf5f5f0 con 0x56331bc873b0 2024-09-21T03:45:07.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.684+0000 7fccb2c36740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x56331c082a20 con 0x56331bcb0270 2024-09-21T03:45:07.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.685+0000 7fccb2c36740 10 client.0 _ll_register_callbacks cb 0x56331bf630a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:45:07.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.685+0000 7fcca9269640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56331bc8f590 0x56331c097d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:07.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.685+0000 7fcca9269640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56331bc8f590 0x56331c097d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43574/0 (socket says 172.21.15.38:43574) 2024-09-21T03:45:07.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.685+0000 7fcca9269640 1 -- 172.21.15.38:0/1379085838 learned_addr learned my addr 172.21.15.38:0/1379085838 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:45:07.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.685+0000 7fcca9a6a640 1 --2- 172.21.15.38:0/1379085838 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56331bc873b0 0x56331c06ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:07.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.685+0000 7fccaa26b640 1 --2- 172.21.15.38:0/1379085838 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56331bcb0270 0x56331c0965f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:07.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.685+0000 7fcca9269640 1 -- 172.21.15.38:0/1379085838 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56331bcb0270 msgr2=0x56331c0965f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:07.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.685+0000 7fcca9269640 1 --2- 172.21.15.38:0/1379085838 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56331bcb0270 0x56331c0965f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:07.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.685+0000 7fcca9269640 1 -- 172.21.15.38:0/1379085838 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56331bc873b0 msgr2=0x56331c06ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:07.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.685+0000 7fcca9269640 1 --2- 172.21.15.38:0/1379085838 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56331bc873b0 0x56331c06ffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:07.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.685+0000 7fcca9269640 1 -- 172.21.15.38:0/1379085838 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56331bd926b0 con 0x56331bc8f590 2024-09-21T03:45:07.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.685+0000 7fcca9269640 1 --2- 172.21.15.38:0/1379085838 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56331bc8f590 0x56331c097d00 secure :-1 s=READY pgs=2385 cs=0 l=1 rev1=1 crypto rx=0x7fcc9400ee90 tx=0x7fcc94001270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c34390e084ae1b2d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:07.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.685+0000 7fcc84ff9640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-21T03:45:07.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.685+0000 7fcc84ff9640 1 -- 172.21.15.38:0/1379085838 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc94025020 con 0x56331bc8f590 2024-09-21T03:45:07.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.685+0000 7fcc84ff9640 1 -- 172.21.15.38:0/1379085838 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(146..146 src has 1..146) ==== 6101+0+0 (secure 0 0 0) 0x7fcc94010040 con 0x56331bc8f590 2024-09-21T03:45:07.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.685+0000 7fccb2c36740 1 -- 172.21.15.38:0/1379085838 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc9c003d60 con 0x56331bc8f590 2024-09-21T03:45:07.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.686+0000 7fcc84ff9640 1 -- 172.21.15.38:0/1379085838 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=343) ==== 24+0+0 (secure 0 0 0) 0x7fcc9402f050 con 0x56331bc8f590 2024-09-21T03:45:07.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.686+0000 7fccb2c36740 10 client.10884 fetch_fsmap learned FSMap version 343 2024-09-21T03:45:07.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.686+0000 7fccb2c36740 1 -- 172.21.15.38:0/1379085838 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=343,monmap=2+}) -- 0x56331c070590 con 0x56331bc8f590 2024-09-21T03:45:07.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.686+0000 7fcc84ff9640 1 -- 172.21.15.38:0/1379085838 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcc9c003d60 con 0x56331bc8f590 2024-09-21T03:45:07.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.686+0000 7fcc84ff9640 1 -- 172.21.15.38:0/1379085838 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 343) ==== 3502+0+0 (secure 0 0 0) 0x7fcc94032020 con 0x56331bc8f590 2024-09-21T03:45:07.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.686+0000 7fccb2c36740 10 client.10884 fetch_fsmap finished waiting for FSMap version 343 2024-09-21T03:45:07.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.686+0000 7fccb2c36740 10 client.10884 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T03:45:07.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.686+0000 7fccb2c36740 20 client.10884 populate_metadata read hostname 'smithi038' 2024-09-21T03:45:07.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.686+0000 7fccb2c36740 1 --2- 172.21.15.38:0/1379085838 >> [v2:172.21.15.38:6832/3866292839,v1:172.21.15.38:6833/3866292839] conn(0x56331bc6ee40 0x56331c0af3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:07.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.686+0000 7fccb2c36740 4 client.10884 mds_command: new command op to 10719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:45:07.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.686+0000 7fccb2c36740 1 -- 172.21.15.38:0/1379085838 --> [v2:172.21.15.38:6832/3866292839,v1:172.21.15.38:6833/3866292839] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56331bfcc770 con 0x56331bc6ee40 2024-09-21T03:45:07.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.686+0000 7fcca9a6a640 1 --2- 172.21.15.38:0/1379085838 >> [v2:172.21.15.38:6832/3866292839,v1:172.21.15.38:6833/3866292839] conn(0x56331bc6ee40 0x56331c0af3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:07.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.686+0000 7fcc84ff9640 10 client.10884 ms_handle_connect on v2:172.21.15.38:6832/3866292839 2024-09-21T03:45:07.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.687+0000 7fcca9a6a640 1 --2- 172.21.15.38:0/1379085838 >> [v2:172.21.15.38:6832/3866292839,v1:172.21.15.38:6833/3866292839] conn(0x56331bc6ee40 0x56331c0af3e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f1fbcda1cd07212e server_cookie=9a4daf875b497d7d in_seq=0 out_seq=0 2024-09-21T03:45:07.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.687+0000 7fcc84ff9640 1 -- 172.21.15.38:0/1379085838 <== mds.0 v2:172.21.15.38:6832/3866292839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56331bfcc770 con 0x56331bc6ee40 2024-09-21T03:45:07.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.687+0000 7fcc84ff9640 10 client.10884 handle_command_reply: tid=0 2024-09-21T03:45:07.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.687+0000 7fccb2c36740 1 client.10884 shutdown 2024-09-21T03:45:07.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.687+0000 7fccb2c36740 1 -- 172.21.15.38:0/1379085838 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56331bc8f590 msgr2=0x56331c097d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:07.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.687+0000 7fccb2c36740 1 --2- 172.21.15.38:0/1379085838 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56331bc8f590 0x56331c097d00 secure :-1 s=READY pgs=2385 cs=0 l=1 rev1=1 crypto rx=0x7fcc9400ee90 tx=0x7fcc94001270 comp rx=0 tx=0).stop 2024-09-21T03:45:07.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.687+0000 7fccb2c36740 1 -- 172.21.15.38:0/1379085838 shutdown_connections 2024-09-21T03:45:07.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.687+0000 7fccb2c36740 1 --2- 172.21.15.38:0/1379085838 >> [v2:172.21.15.38:6832/3866292839,v1:172.21.15.38:6833/3866292839] conn(0x56331bc6ee40 0x56331c0af3e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:07.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.687+0000 7fccb2c36740 1 --2- 172.21.15.38:0/1379085838 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56331bcb0270 0x56331c0965f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:07.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.687+0000 7fccb2c36740 1 --2- 172.21.15.38:0/1379085838 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56331bc8f590 0x56331c097d00 unknown :-1 s=CLOSED pgs=2385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:07.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.687+0000 7fccb2c36740 1 --2- 172.21.15.38:0/1379085838 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56331bc873b0 0x56331c06ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:07.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.687+0000 7fcc84ff9640 0 client.10884 ms_handle_reset on v2:172.21.15.38:6832/3866292839 2024-09-21T03:45:07.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.687+0000 7fccb2c36740 1 -- 172.21.15.38:0/1379085838 >> 172.21.15.38:0/1379085838 conn(0x56331bca43c0 msgr2=0x56331c09ef90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:07.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.688+0000 7fccb2c36740 1 -- 172.21.15.38:0/1379085838 shutdown_connections 2024-09-21T03:45:07.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.688+0000 7fccb2c36740 1 -- 172.21.15.38:0/1379085838 wait complete. 2024-09-21T03:45:07.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.688+0000 7fccb2c36740 20 client.10884 trim_cache size 0 max 16384 2024-09-21T03:45:07.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.706+0000 7fccb2c36740 1 Processor -- start 2024-09-21T03:45:07.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.706+0000 7fccb2c36740 1 -- start start 2024-09-21T03:45:07.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.707+0000 7fccb2c36740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56331bc873b0 0x56331c0ae360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:07.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.707+0000 7fccb2c36740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56331bc8f590 0x56331c0ae8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:07.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.707+0000 7fccb2c36740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56331bcb0270 0x56331c081c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:07.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.708+0000 7fccb2c36740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fcca41ff180 con 0x56331bc8f590 2024-09-21T03:45:07.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.708+0000 7fccb2c36740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fcca4157020 con 0x56331bc873b0 2024-09-21T03:45:07.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.708+0000 7fccb2c36740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fcca41feb60 con 0x56331bcb0270 2024-09-21T03:45:07.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.708+0000 7fcca9269640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56331bc8f590 0x56331c0ae8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:07.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.708+0000 7fcca9269640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56331bc8f590 0x56331c0ae8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43584/0 (socket says 172.21.15.38:43584) 2024-09-21T03:45:07.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.708+0000 7fcca9269640 1 -- 172.21.15.38:0/1336846300 learned_addr learned my addr 172.21.15.38:0/1336846300 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:45:07.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.708+0000 7fccaa26b640 1 --2- 172.21.15.38:0/1336846300 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56331bcb0270 0x56331c081c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:07.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.708+0000 7fcca9a6a640 1 --2- 172.21.15.38:0/1336846300 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56331bc873b0 0x56331c0ae360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:07.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.708+0000 7fcca9269640 1 -- 172.21.15.38:0/1336846300 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56331bcb0270 msgr2=0x56331c081c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:07.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.708+0000 7fcca9269640 1 --2- 172.21.15.38:0/1336846300 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56331bcb0270 0x56331c081c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:07.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.708+0000 7fcca9269640 1 -- 172.21.15.38:0/1336846300 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56331bc873b0 msgr2=0x56331c0ae360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:07.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.708+0000 7fcca9269640 1 --2- 172.21.15.38:0/1336846300 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56331bc873b0 0x56331c0ae360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:07.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.708+0000 7fcca9269640 1 -- 172.21.15.38:0/1336846300 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56331c09e580 con 0x56331bc8f590 2024-09-21T03:45:07.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.708+0000 7fcca9269640 1 --2- 172.21.15.38:0/1336846300 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56331bc8f590 0x56331c0ae8a0 secure :-1 s=READY pgs=2386 cs=0 l=1 rev1=1 crypto rx=0x7fcc9400e880 tx=0x7fcc94015c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=404edee772ddec55 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:07.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.708+0000 7fcc86ffd640 1 -- 172.21.15.38:0/1336846300 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc94025030 con 0x56331bc8f590 2024-09-21T03:45:07.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.708+0000 7fcc86ffd640 1 -- 172.21.15.38:0/1336846300 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcc94021680 con 0x56331bc8f590 2024-09-21T03:45:07.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.708+0000 7fcc86ffd640 1 -- 172.21.15.38:0/1336846300 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc940042a0 con 0x56331bc8f590 2024-09-21T03:45:07.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.709+0000 7fccb2c36740 1 -- 172.21.15.38:0/1336846300 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56331bc8f590 msgr2=0x56331c0ae8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:07.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.709+0000 7fccb2c36740 1 --2- 172.21.15.38:0/1336846300 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56331bc8f590 0x56331c0ae8a0 secure :-1 s=READY pgs=2386 cs=0 l=1 rev1=1 crypto rx=0x7fcc9400e880 tx=0x7fcc94015c90 comp rx=0 tx=0).stop 2024-09-21T03:45:07.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.709+0000 7fccb2c36740 1 -- 172.21.15.38:0/1336846300 shutdown_connections 2024-09-21T03:45:07.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.709+0000 7fccb2c36740 1 --2- 172.21.15.38:0/1336846300 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56331bcb0270 0x56331c081c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:07.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.709+0000 7fccb2c36740 1 --2- 172.21.15.38:0/1336846300 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56331bc8f590 0x56331c0ae8a0 unknown :-1 s=CLOSED pgs=2386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:07.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.709+0000 7fccb2c36740 1 --2- 172.21.15.38:0/1336846300 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56331bc873b0 0x56331c0ae360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:07.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.709+0000 7fccb2c36740 1 -- 172.21.15.38:0/1336846300 >> 172.21.15.38:0/1336846300 conn(0x56331bca43c0 msgr2=0x56331c094fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:07.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.709+0000 7fccb2c36740 1 -- 172.21.15.38:0/1336846300 shutdown_connections 2024-09-21T03:45:07.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.709+0000 7fccb2c36740 1 -- 172.21.15.38:0/1336846300 wait complete. 2024-09-21T03:45:07.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.710+0000 7fccb2c36740 1 -- start start 2024-09-21T03:45:07.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.710+0000 7fccb2c36740 1 Processor -- start 2024-09-21T03:45:07.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.711+0000 7fccb2c36740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56331bc873b0 0x56331c105410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:07.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.711+0000 7fccb2c36740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56331bc8f590 0x56331c0b1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:07.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.711+0000 7fccb2c36740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56331bcb0270 0x56331c0ae360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:07.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.711+0000 7fccb2c36740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x56331c070930 con 0x56331bcb0270 2024-09-21T03:45:07.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.711+0000 7fccb2c36740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x56331c094c90 con 0x56331bc8f590 2024-09-21T03:45:07.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.711+0000 7fccb2c36740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x56331c093ea0 con 0x56331bc873b0 2024-09-21T03:45:07.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.712+0000 7fccb2c36740 10 client.0 _ll_register_callbacks cb 0x7fcc94001cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:45:07.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.712+0000 7fccaa26b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56331bcb0270 0x56331c0ae360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:07.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.712+0000 7fccaa26b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56331bcb0270 0x56331c0ae360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43592/0 (socket says 172.21.15.38:43592) 2024-09-21T03:45:07.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.712+0000 7fccaa26b640 1 -- 172.21.15.38:0/3609243192 learned_addr learned my addr 172.21.15.38:0/3609243192 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:45:07.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.712+0000 7fcca9269640 1 --2- 172.21.15.38:0/3609243192 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56331bc8f590 0x56331c0b1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:07.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.712+0000 7fcca9a6a640 1 --2- 172.21.15.38:0/3609243192 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56331bc873b0 0x56331c105410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:07.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.712+0000 7fccaa26b640 1 -- 172.21.15.38:0/3609243192 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56331bc873b0 msgr2=0x56331c105410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:07.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.712+0000 7fccaa26b640 1 --2- 172.21.15.38:0/3609243192 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56331bc873b0 0x56331c105410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:07.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.712+0000 7fccaa26b640 1 -- 172.21.15.38:0/3609243192 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56331bc8f590 msgr2=0x56331c0b1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:07.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.712+0000 7fccaa26b640 1 --2- 172.21.15.38:0/3609243192 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56331bc8f590 0x56331c0b1cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:07.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.712+0000 7fccaa26b640 1 -- 172.21.15.38:0/3609243192 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56331c11c850 con 0x56331bcb0270 2024-09-21T03:45:07.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.712+0000 7fccaa26b640 1 --2- 172.21.15.38:0/3609243192 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56331bcb0270 0x56331c0ae360 secure :-1 s=READY pgs=2387 cs=0 l=1 rev1=1 crypto rx=0x7fcc9c009ca0 tx=0x7fcc9c032c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb723e988511e53a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:07.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.712+0000 7fcc84ff9640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-21T03:45:07.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.712+0000 7fcc84ff9640 1 -- 172.21.15.38:0/3609243192 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc9c041040 con 0x56331bcb0270 2024-09-21T03:45:07.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.712+0000 7fcc84ff9640 1 -- 172.21.15.38:0/3609243192 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(146..146 src has 1..146) ==== 6101+0+0 (secure 0 0 0) 0x7fcc9c046460 con 0x56331bcb0270 2024-09-21T03:45:07.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.712+0000 7fccb2c36740 1 -- 172.21.15.38:0/3609243192 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56331c053350 con 0x56331bcb0270 2024-09-21T03:45:07.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.712+0000 7fcc84ff9640 1 -- 172.21.15.38:0/3609243192 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=343) ==== 24+0+0 (secure 0 0 0) 0x7fcc9c046660 con 0x56331bcb0270 2024-09-21T03:45:07.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.713+0000 7fccb2c36740 10 client.10890 fetch_fsmap learned FSMap version 343 2024-09-21T03:45:07.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.713+0000 7fccb2c36740 1 -- 172.21.15.38:0/3609243192 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=343,monmap=2+}) -- 0x56331c11d5e0 con 0x56331bcb0270 2024-09-21T03:45:07.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.713+0000 7fcc84ff9640 1 -- 172.21.15.38:0/3609243192 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x56331c053350 con 0x56331bcb0270 2024-09-21T03:45:07.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.713+0000 7fcc84ff9640 1 -- 172.21.15.38:0/3609243192 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 343) ==== 3502+0+0 (secure 0 0 0) 0x7fcc9c060080 con 0x56331bcb0270 2024-09-21T03:45:07.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.713+0000 7fccb2c36740 10 client.10890 fetch_fsmap finished waiting for FSMap version 343 2024-09-21T03:45:07.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.713+0000 7fccb2c36740 10 client.10890 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T03:45:07.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.713+0000 7fccb2c36740 20 client.10890 populate_metadata read hostname 'smithi038' 2024-09-21T03:45:07.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.713+0000 7fccb2c36740 1 --2- 172.21.15.38:0/3609243192 >> [v2:172.21.15.38:6832/3866292839,v1:172.21.15.38:6833/3866292839] conn(0x56331c07e730 0x56331c097370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:07.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.713+0000 7fccb2c36740 4 client.10890 mds_command: new command op to 10719 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T03:45:07.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.713+0000 7fccb2c36740 1 -- 172.21.15.38:0/3609243192 --> [v2:172.21.15.38:6832/3866292839,v1:172.21.15.38:6833/3866292839] -- command(tid 0: {"prefix": "status"}) -- 0x56331c09dcf0 con 0x56331c07e730 2024-09-21T03:45:07.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.713+0000 7fcca9a6a640 1 --2- 172.21.15.38:0/3609243192 >> [v2:172.21.15.38:6832/3866292839,v1:172.21.15.38:6833/3866292839] conn(0x56331c07e730 0x56331c097370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:07.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.713+0000 7fcc84ff9640 10 client.10890 ms_handle_connect on v2:172.21.15.38:6832/3866292839 2024-09-21T03:45:07.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.713+0000 7fcca9a6a640 1 --2- 172.21.15.38:0/3609243192 >> [v2:172.21.15.38:6832/3866292839,v1:172.21.15.38:6833/3866292839] conn(0x56331c07e730 0x56331c097370 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=522c9d3e155d88cf server_cookie=8ebfdfe6730c0a19 in_seq=0 out_seq=0 2024-09-21T03:45:07.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.714+0000 7fcc84ff9640 1 -- 172.21.15.38:0/3609243192 <== mds.0 v2:172.21.15.38:6832/3866292839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+329 (crc 0 0 0) 0x56331c09dcf0 con 0x56331c07e730 2024-09-21T03:45:07.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.714+0000 7fcc84ff9640 10 client.10890 handle_command_reply: tid=0 2024-09-21T03:45:07.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.714+0000 7fccb2c36740 1 client.10890 shutdown 2024-09-21T03:45:07.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.714+0000 7fccb2c36740 1 -- 172.21.15.38:0/3609243192 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56331bcb0270 msgr2=0x56331c0ae360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:07.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.714+0000 7fccb2c36740 1 --2- 172.21.15.38:0/3609243192 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56331bcb0270 0x56331c0ae360 secure :-1 s=READY pgs=2387 cs=0 l=1 rev1=1 crypto rx=0x7fcc9c009ca0 tx=0x7fcc9c032c00 comp rx=0 tx=0).stop 2024-09-21T03:45:07.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.714+0000 7fccb2c36740 1 -- 172.21.15.38:0/3609243192 shutdown_connections 2024-09-21T03:45:07.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.714+0000 7fccb2c36740 1 --2- 172.21.15.38:0/3609243192 >> [v2:172.21.15.38:6832/3866292839,v1:172.21.15.38:6833/3866292839] conn(0x56331c07e730 0x56331c097370 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:07.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.714+0000 7fccb2c36740 1 --2- 172.21.15.38:0/3609243192 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56331bcb0270 0x56331c0ae360 unknown :-1 s=CLOSED pgs=2387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:07.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.714+0000 7fccb2c36740 1 --2- 172.21.15.38:0/3609243192 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56331bc8f590 0x56331c0b1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:07.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.714+0000 7fccb2c36740 1 --2- 172.21.15.38:0/3609243192 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56331bc873b0 0x56331c105410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:07.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.714+0000 7fcc84ff9640 0 client.10890 ms_handle_reset on v2:172.21.15.38:6832/3866292839 2024-09-21T03:45:07.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.714+0000 7fccb2c36740 1 -- 172.21.15.38:0/3609243192 >> 172.21.15.38:0/3609243192 conn(0x56331bca43c0 msgr2=0x56331c094fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:07.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.714+0000 7fccb2c36740 1 -- 172.21.15.38:0/3609243192 shutdown_connections 2024-09-21T03:45:07.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.714+0000 7fccb2c36740 1 -- 172.21.15.38:0/3609243192 wait complete. 2024-09-21T03:45:07.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.714+0000 7fccb2c36740 20 client.10890 trim_cache size 0 max 16384 2024-09-21T03:45:07.721 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-09-21T03:45:07.722 INFO:teuthology.orchestra.run.smithi038.stdout: "cluster_fsid": "9d50d66f-656b-4fe2-8e3b-fe53b79c4d43", 2024-09-21T03:45:07.722 INFO:teuthology.orchestra.run.smithi038.stdout: "whoami": 0, 2024-09-21T03:45:07.722 INFO:teuthology.orchestra.run.smithi038.stdout: "id": 10719, 2024-09-21T03:45:07.722 INFO:teuthology.orchestra.run.smithi038.stdout: "want_state": "up:active", 2024-09-21T03:45:07.722 INFO:teuthology.orchestra.run.smithi038.stdout: "state": "up:active", 2024-09-21T03:45:07.722 INFO:teuthology.orchestra.run.smithi038.stdout: "fs_name": "cephfs", 2024-09-21T03:45:07.722 INFO:teuthology.orchestra.run.smithi038.stdout: "rank_uptime": 10.372056289, 2024-09-21T03:45:07.722 INFO:teuthology.orchestra.run.smithi038.stdout: "mdsmap_epoch": 343, 2024-09-21T03:45:07.722 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch": 146, 2024-09-21T03:45:07.722 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch_barrier": 146, 2024-09-21T03:45:07.722 INFO:teuthology.orchestra.run.smithi038.stdout: "uptime": 15.009830365999999 2024-09-21T03:45:07.722 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-09-21T03:45:07.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.715+0000 7fccabcce640 1 -- 172.21.15.38:0/3168456424 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fcc88078b90 msgr2=0x7fcc8807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:07.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.715+0000 7fccabcce640 1 --2- 172.21.15.38:0/3168456424 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fcc88078b90 0x7fcc8807b0a0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fcca40c8fb0 tx=0x7fcc94009040 comp rx=0 tx=0).stop 2024-09-21T03:45:07.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.715+0000 7fccabcce640 1 -- 172.21.15.38:0/3168456424 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcca40ccc40 msgr2=0x7fcca4151830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:07.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.715+0000 7fccabcce640 1 --2- 172.21.15.38:0/3168456424 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcca40ccc40 0x7fcca4151830 secure :-1 s=READY pgs=2383 cs=0 l=1 rev1=1 crypto rx=0x7fcca000c430 tx=0x7fcca000c8f0 comp rx=0 tx=0).stop 2024-09-21T03:45:07.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.715+0000 7fccabcce640 1 -- 172.21.15.38:0/3168456424 shutdown_connections 2024-09-21T03:45:07.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.715+0000 7fccabcce640 1 --2- 172.21.15.38:0/3168456424 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcca40c9490 0x7fcca40c9ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:07.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.715+0000 7fccabcce640 1 --2- 172.21.15.38:0/3168456424 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fcc88078b90 0x7fcc8807b0a0 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:07.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.715+0000 7fccabcce640 1 --2- 172.21.15.38:0/3168456424 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcca415ca90 0x7fcca40c9980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:07.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.715+0000 7fccabcce640 1 --2- 172.21.15.38:0/3168456424 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcca40ccc40 0x7fcca4151830 unknown :-1 s=CLOSED pgs=2383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:07.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.715+0000 7fccabcce640 1 -- 172.21.15.38:0/3168456424 >> 172.21.15.38:0/3168456424 conn(0x7fcca40b46e0 msgr2=0x7fcca40ced90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:07.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.715+0000 7fccabcce640 1 -- 172.21.15.38:0/3168456424 shutdown_connections 2024-09-21T03:45:07.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.715+0000 7fccabcce640 1 -- 172.21.15.38:0/3168456424 wait complete. 2024-09-21T03:45:07.728 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T03:45:07.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.843+0000 7f2b37a71640 1 -- 172.21.15.38:0/2994783293 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2b300b2e40 msgr2=0x7f2b300b3250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:07.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.843+0000 7f2b37a71640 1 --2- 172.21.15.38:0/2994783293 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2b300b2e40 0x7f2b300b3250 secure :-1 s=READY pgs=2389 cs=0 l=1 rev1=1 crypto rx=0x7f2b28005fd0 tx=0x7f2b2802a8b0 comp rx=0 tx=0).stop 2024-09-21T03:45:07.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.843+0000 7f2b37a71640 1 -- 172.21.15.38:0/2994783293 shutdown_connections 2024-09-21T03:45:07.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.843+0000 7f2b37a71640 1 --2- 172.21.15.38:0/2994783293 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2b300b2e40 0x7f2b300b3250 unknown :-1 s=CLOSED pgs=2389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:07.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.843+0000 7f2b37a71640 1 --2- 172.21.15.38:0/2994783293 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2b30162280 0x7f2b30162690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:07.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.843+0000 7f2b37a71640 1 -- 172.21.15.38:0/2994783293 >> 172.21.15.38:0/2994783293 conn(0x7f2b300b6b20 msgr2=0x7f2b300d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:07.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.843+0000 7f2b37a71640 1 -- 172.21.15.38:0/2994783293 shutdown_connections 2024-09-21T03:45:07.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.843+0000 7f2b37a71640 1 -- 172.21.15.38:0/2994783293 wait complete. 2024-09-21T03:45:07.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.844+0000 7f2b37a71640 1 Processor -- start 2024-09-21T03:45:07.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.845+0000 7f2b37a71640 1 -- start start 2024-09-21T03:45:07.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.846+0000 7f2b37a71640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2b300b2e40 0x7f2b3015a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:07.850 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.846+0000 7f2b37a71640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2b30162280 0x7f2b30144e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:07.850 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.846+0000 7f2b37a71640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2b301453c0 0x7f2b3014a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:07.850 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.846+0000 7f2b37a71640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f2b3015e4b0 con 0x7f2b30162280 2024-09-21T03:45:07.850 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.846+0000 7f2b37a71640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f2b300b39a0 con 0x7f2b300b2e40 2024-09-21T03:45:07.850 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.846+0000 7f2b37a71640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f2b3015e630 con 0x7f2b301453c0 2024-09-21T03:45:07.850 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.846+0000 7f2b3500c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2b30162280 0x7f2b30144e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:07.850 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.846+0000 7f2b3500c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2b30162280 0x7f2b30144e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43620/0 (socket says 172.21.15.38:43620) 2024-09-21T03:45:07.850 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.846+0000 7f2b3500c640 1 -- 172.21.15.38:0/4175360477 learned_addr learned my addr 172.21.15.38:0/4175360477 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:45:07.850 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.846+0000 7f2b3580d640 1 --2- 172.21.15.38:0/4175360477 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2b300b2e40 0x7f2b3015a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:07.851 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.846+0000 7f2b3600e640 1 --2- 172.21.15.38:0/4175360477 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2b301453c0 0x7f2b3014a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:07.851 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.846+0000 7f2b3500c640 1 -- 172.21.15.38:0/4175360477 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2b301453c0 msgr2=0x7f2b3014a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:07.851 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.846+0000 7f2b3500c640 1 --2- 172.21.15.38:0/4175360477 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2b301453c0 0x7f2b3014a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:07.851 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.846+0000 7f2b3500c640 1 -- 172.21.15.38:0/4175360477 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2b300b2e40 msgr2=0x7f2b3015a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:07.851 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.846+0000 7f2b3500c640 1 --2- 172.21.15.38:0/4175360477 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2b300b2e40 0x7f2b3015a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:07.851 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.846+0000 7f2b3500c640 1 -- 172.21.15.38:0/4175360477 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b30158fb0 con 0x7f2b30162280 2024-09-21T03:45:07.851 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.847+0000 7f2b3500c640 1 --2- 172.21.15.38:0/4175360477 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2b30162280 0x7f2b30144e80 secure :-1 s=READY pgs=2390 cs=0 l=1 rev1=1 crypto rx=0x7f2b2000c970 tx=0x7f2b2000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:07.851 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.847+0000 7f2b267fc640 1 -- 172.21.15.38:0/4175360477 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b20004230 con 0x7f2b30162280 2024-09-21T03:45:07.851 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.847+0000 7f2b267fc640 1 -- 172.21.15.38:0/4175360477 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2b200079b0 con 0x7f2b30162280 2024-09-21T03:45:07.851 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.847+0000 7f2b37a71640 1 -- 172.21.15.38:0/4175360477 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b30159290 con 0x7f2b30162280 2024-09-21T03:45:07.851 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.847+0000 7f2b267fc640 1 -- 172.21.15.38:0/4175360477 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b200044c0 con 0x7f2b30162280 2024-09-21T03:45:07.851 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.847+0000 7f2b37a71640 1 -- 172.21.15.38:0/4175360477 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b30159850 con 0x7f2b30162280 2024-09-21T03:45:07.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.848+0000 7f2b37a71640 1 -- 172.21.15.38:0/4175360477 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b30162f00 con 0x7f2b30162280 2024-09-21T03:45:07.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.849+0000 7f2b267fc640 1 -- 172.21.15.38:0/4175360477 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f2b200035f0 con 0x7f2b30162280 2024-09-21T03:45:07.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.849+0000 7f2b267fc640 1 --2- 172.21.15.38:0/4175360477 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2b14078b90 0x7f2b1407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:07.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.849+0000 7f2b267fc640 1 -- 172.21.15.38:0/4175360477 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 6101+0+0 (secure 0 0 0) 0x7f2b2009b1c0 con 0x7f2b30162280 2024-09-21T03:45:07.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.849+0000 7f2b3580d640 1 --2- 172.21.15.38:0/4175360477 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2b14078b90 0x7f2b1407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:07.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.850+0000 7f2b3580d640 1 --2- 172.21.15.38:0/4175360477 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2b14078b90 0x7f2b1407b0a0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c000f60 tx=0x7f2b2c009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:07.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:07.854+0000 7f2b267fc640 1 -- 172.21.15.38:0/4175360477 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f2b20014070 con 0x7f2b30162280 2024-09-21T03:45:08.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:08.079+0000 7f2b37a71640 1 -- 172.21.15.38:0/4175360477 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f2b301629b0 con 0x7f2b30162280 2024-09-21T03:45:08.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:08.079+0000 7f2b267fc640 1 -- 172.21.15.38:0/4175360477 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v343) ==== 53+0+73 (secure 0 0 0) 0x7f2b20062f40 con 0x7f2b30162280 2024-09-21T03:45:08.081 INFO:teuthology.orchestra.run.smithi038.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T03:45:08.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:08.081+0000 7f2b37a71640 1 -- 172.21.15.38:0/4175360477 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2b14078b90 msgr2=0x7f2b1407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:08.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:08.081+0000 7f2b37a71640 1 --2- 172.21.15.38:0/4175360477 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2b14078b90 0x7f2b1407b0a0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c000f60 tx=0x7f2b2c009040 comp rx=0 tx=0).stop 2024-09-21T03:45:08.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:08.081+0000 7f2b37a71640 1 -- 172.21.15.38:0/4175360477 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2b30162280 msgr2=0x7f2b30144e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:08.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:08.081+0000 7f2b37a71640 1 --2- 172.21.15.38:0/4175360477 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2b30162280 0x7f2b30144e80 secure :-1 s=READY pgs=2390 cs=0 l=1 rev1=1 crypto rx=0x7f2b2000c970 tx=0x7f2b2000ce30 comp rx=0 tx=0).stop 2024-09-21T03:45:08.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:08.081+0000 7f2b37a71640 1 -- 172.21.15.38:0/4175360477 shutdown_connections 2024-09-21T03:45:08.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:08.081+0000 7f2b37a71640 1 --2- 172.21.15.38:0/4175360477 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2b14078b90 0x7f2b1407b0a0 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:08.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:08.081+0000 7f2b37a71640 1 --2- 172.21.15.38:0/4175360477 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2b301453c0 0x7f2b3014a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:08.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:08.081+0000 7f2b37a71640 1 --2- 172.21.15.38:0/4175360477 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2b30162280 0x7f2b30144e80 unknown :-1 s=CLOSED pgs=2390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:08.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:08.081+0000 7f2b37a71640 1 --2- 172.21.15.38:0/4175360477 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2b300b2e40 0x7f2b3015a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:08.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:08.081+0000 7f2b37a71640 1 -- 172.21.15.38:0/4175360477 >> 172.21.15.38:0/4175360477 conn(0x7f2b300b6b20 msgr2=0x7f2b3015dac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:08.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:08.081+0000 7f2b37a71640 1 -- 172.21.15.38:0/4175360477 shutdown_connections 2024-09-21T03:45:08.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:08.081+0000 7f2b37a71640 1 -- 172.21.15.38:0/4175360477 wait complete. 2024-09-21T03:45:08.095 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-21T03:45:08.095 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T03:45:08.095 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi038.front.sepia.ceph.com 2024-09-21T03:45:08.095 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-21T03:45:08.095 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T03:45:08.095 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-21T03:45:08.095 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-21T03:45:08.095 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T03:45:08.095 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T03:45:08.095 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:45:08.096 DEBUG:teuthology.orchestra.run.smithi038:> ip addr 2024-09-21T03:45:08.112 INFO:teuthology.orchestra.run.smithi038.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T03:45:08.112 INFO:teuthology.orchestra.run.smithi038.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T03:45:08.112 INFO:teuthology.orchestra.run.smithi038.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T03:45:08.112 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:45:08.112 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 ::1/128 scope host 2024-09-21T03:45:08.112 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:45:08.113 INFO:teuthology.orchestra.run.smithi038.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T03:45:08.113 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:6b:fd:a2 brd ff:ff:ff:ff:ff:ff 2024-09-21T03:45:08.113 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp7s0f0 2024-09-21T03:45:08.113 INFO:teuthology.orchestra.run.smithi038.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T03:45:08.113 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:6b:fd:a3 brd ff:ff:ff:ff:ff:ff 2024-09-21T03:45:08.113 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp7s0f1 2024-09-21T03:45:08.113 INFO:teuthology.orchestra.run.smithi038.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T03:45:08.113 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:bd:15:fc brd ff:ff:ff:ff:ff:ff 2024-09-21T03:45:08.113 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp1s0f0 2024-09-21T03:45:08.113 INFO:teuthology.orchestra.run.smithi038.stdout: inet 172.21.15.38/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-21T03:45:08.113 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft 42101sec preferred_lft 42101sec 2024-09-21T03:45:08.113 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 fe80::ec4:7aff:febd:15fc/64 scope link 2024-09-21T03:45:08.113 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:45:08.114 INFO:teuthology.orchestra.run.smithi038.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T03:45:08.114 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:bd:15:fd brd ff:ff:ff:ff:ff:ff 2024-09-21T03:45:08.114 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp1s0f1 2024-09-21T03:45:08.114 INFO:teuthology.orchestra.run.smithi038.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T03:45:08.114 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 92:55:c7:97:6f:b5 brd ff:ff:ff:ff:ff:ff 2024-09-21T03:45:08.114 INFO:teuthology.orchestra.run.smithi038.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-21T03:45:08.114 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:45:08.114 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 fe80::c49f:c1ff:fe21:b12a/64 scope link 2024-09-21T03:45:08.114 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:45:08.114 INFO:teuthology.orchestra.run.smithi038.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T03:45:08.114 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether a6:3a:83:ad:db:d5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T03:45:08.114 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 fe80::a43a:83ff:fead:dbd5/64 scope link 2024-09-21T03:45:08.114 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:45:08.114 INFO:teuthology.orchestra.run.smithi038.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T03:45:08.115 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 92:55:c7:97:6f:b5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T03:45:08.115 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 fe80::9055:c7ff:fe97:6fb5/64 scope link 2024-09-21T03:45:08.115 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:45:08.115 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:45:08.115 DEBUG:teuthology.orchestra.run.smithi038:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T03:45:08.177 INFO:teuthology.orchestra.run.smithi038.stdout:1 2024-09-21T03:45:08.178 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:45:08.179 DEBUG:teuthology.orchestra.run.smithi038:> route 2024-09-21T03:45:08.235 INFO:teuthology.orchestra.run.smithi038.stdout:Kernel IP routing table 2024-09-21T03:45:08.235 INFO:teuthology.orchestra.run.smithi038.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T03:45:08.235 INFO:teuthology.orchestra.run.smithi038.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-21T03:45:08.236 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-21T03:45:08.236 INFO:teuthology.orchestra.run.smithi038.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T03:45:08.236 DEBUG:teuthology.orchestra.run.smithi038:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T03:45:08.236 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-09-21T03:45:08.236 DEBUG:teuthology.orchestra.run.smithi038:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-21T03:45:08.236 DEBUG:teuthology.orchestra.run.smithi038:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-21T03:45:08.236 DEBUG:teuthology.orchestra.run.smithi038:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-21T03:45:08.236 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-09-21T03:45:08.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:08 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T03:45:08.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:45:08.365 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:45:08.365 DEBUG:teuthology.orchestra.run.smithi038:> ip netns list 2024-09-21T03:45:08.380 INFO:teuthology.orchestra.run.smithi038.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-21T03:45:08.381 INFO:teuthology.orchestra.run.smithi038.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-21T03:45:08.381 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T03:45:08.381 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:45:08.381 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip link set brx.0 up 2024-09-21T03:45:08.446 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T03:45:08.446 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-21T03:45:08.503 INFO:teuthology.orchestra.run.smithi038.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-21T03:45:08.503 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T03:45:08.503 DEBUG:teuthology.orchestra.run.smithi038:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-21T03:45:08.558 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-21T03:45:08.559 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:45:08.559 DEBUG:teuthology.orchestra.run.smithi038:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T03:45:08.657 INFO:teuthology.orchestra.run.smithi038.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T03:45:08.657 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T03:45:08.657 INFO:teuthology.orchestra.run.smithi038.stdout:invalid new device string format 2024-09-21T03:45:08.668 INFO:teuthology.orchestra.run.smithi038.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T03:45:08.668 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T03:45:08.668 INFO:teuthology.orchestra.run.smithi038.stdout:invalid new device string format 2024-09-21T03:45:08.668 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: resolved to: "172.21.15.38:3300,172.21.15.195:3300,172.21.15.195:3301" 2024-09-21T03:45:08.668 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T03:45:08.674 INFO:teuthology.orchestra.run.smithi038.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T03:45:08.674 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T03:45:08.674 INFO:teuthology.orchestra.run.smithi038.stdout:invalid new device string format 2024-09-21T03:45:08.674 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: resolved to: "172.21.15.38:3300,172.21.15.195:3300,172.21.15.195:3301" 2024-09-21T03:45:08.674 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T03:45:08.674 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: resolved to: "172.21.15.38:6789,172.21.15.195:6789,172.21.15.195:6790" 2024-09-21T03:45:08.674 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.38:6789,172.21.15.195:6789,172.21.15.195:6790:/ 2024-09-21T03:45:08.674 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=9d50d66f-656b-4fe2-8e3b-fe53b79c4d43" will pass to kernel 2024-09-21T03:45:08.675 INFO:teuthology.orchestra.run.smithi038.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-21T03:45:08.675 INFO:teuthology.orchestra.run.smithi038.stdout: You just mounted a file system that supports labels which does not 2024-09-21T03:45:08.675 INFO:teuthology.orchestra.run.smithi038.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-21T03:45:08.675 INFO:teuthology.orchestra.run.smithi038.stdout: applications will generate AVC messages and not be allowed access to 2024-09-21T03:45:08.675 INFO:teuthology.orchestra.run.smithi038.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-21T03:45:08.676 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T03:45:08.676 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:45:08.676 DEBUG:teuthology.orchestra.run.smithi038:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-21T03:45:08.702 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T03:45:08.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:08 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T03:45:08.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:45:08.805 INFO:teuthology.orchestra.run.smithi038.stdout:client10902 2024-09-21T03:45:08.806 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-21T03:45:08.806 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:45:08.821 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:45:08.822 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir dir1) 2024-09-21T03:45:08.884 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:45:08.885 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch dir1/file1) 2024-09-21T03:45:08.943 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:45:08.943 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir dir2) 2024-09-21T03:45:09.001 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:45:09.001 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch dir2/file2) 2024-09-21T03:45:09.061 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:45:09.062 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir testdir) 2024-09-21T03:45:09.120 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:45:09.120 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec ln dir1/file1 testdir/hardlink1) 2024-09-21T03:45:09.181 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:45:09.181 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec ln dir2/file2 testdir/hardlink2) 2024-09-21T03:45:09.239 DEBUG:teuthology.orchestra.run.smithi038:> stdin-killer -- python3 -c ' 2024-09-21T03:45:09.239 DEBUG:teuthology.orchestra.run.smithi038:> import os 2024-09-21T03:45:09.239 DEBUG:teuthology.orchestra.run.smithi038:> import stat 2024-09-21T03:45:09.239 DEBUG:teuthology.orchestra.run.smithi038:> 2024-09-21T03:45:09.239 DEBUG:teuthology.orchestra.run.smithi038:> print(os.stat("/home/ubuntu/cephtest/mnt.0/dir1/file1").st_ino) 2024-09-21T03:45:09.239 DEBUG:teuthology.orchestra.run.smithi038:> ' 2024-09-21T03:45:09.346 INFO:teuthology.orchestra.run.smithi038.stdout:1099511627777 2024-09-21T03:45:09.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:45:09.355 DEBUG:teuthology.orchestra.run.smithi038:> stdin-killer -- python3 -c ' 2024-09-21T03:45:09.355 DEBUG:teuthology.orchestra.run.smithi038:> import os 2024-09-21T03:45:09.355 DEBUG:teuthology.orchestra.run.smithi038:> import stat 2024-09-21T03:45:09.355 DEBUG:teuthology.orchestra.run.smithi038:> 2024-09-21T03:45:09.355 DEBUG:teuthology.orchestra.run.smithi038:> print(os.stat("/home/ubuntu/cephtest/mnt.0/dir2/file2").st_ino) 2024-09-21T03:45:09.355 DEBUG:teuthology.orchestra.run.smithi038:> ' 2024-09-21T03:45:09.461 INFO:teuthology.orchestra.run.smithi038.stdout:1099511628279 2024-09-21T03:45:09.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:45:09.470 DEBUG:teuthology.orchestra.run.smithi038:> stdin-killer -- python3 -c ' 2024-09-21T03:45:09.470 DEBUG:teuthology.orchestra.run.smithi038:> import os 2024-09-21T03:45:09.470 DEBUG:teuthology.orchestra.run.smithi038:> import stat 2024-09-21T03:45:09.470 DEBUG:teuthology.orchestra.run.smithi038:> 2024-09-21T03:45:09.471 DEBUG:teuthology.orchestra.run.smithi038:> print(os.stat("/home/ubuntu/cephtest/mnt.0/dir2").st_ino) 2024-09-21T03:45:09.471 DEBUG:teuthology.orchestra.run.smithi038:> ' 2024-09-21T03:45:09.578 INFO:teuthology.orchestra.run.smithi038.stdout:1099511628278 2024-09-21T03:45:09.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:45:09.587 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-21T03:45:09.587 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:45:09.643 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-21T03:45:09.643 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:45:09.700 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T03:45:09.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:09.856+0000 7f9bb18a4640 1 -- 172.21.15.38:0/66201814 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9b94004800 msgr2=0x7f9b94004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:09.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:09.856+0000 7f9bb18a4640 1 --2- 172.21.15.38:0/66201814 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9b94004800 0x7f9b94004c10 secure :-1 s=READY pgs=2393 cs=0 l=1 rev1=1 crypto rx=0x7f9bac0b5300 tx=0x7f9b94005830 comp rx=0 tx=0).stop 2024-09-21T03:45:09.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:09.856+0000 7f9bb18a4640 1 -- 172.21.15.38:0/66201814 shutdown_connections 2024-09-21T03:45:09.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:09.856+0000 7f9bb18a4640 1 --2- 172.21.15.38:0/66201814 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9b94004800 0x7f9b94004c10 unknown :-1 s=CLOSED pgs=2393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:09.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:09.856+0000 7f9bb18a4640 1 -- 172.21.15.38:0/66201814 >> 172.21.15.38:0/66201814 conn(0x7f9bac0b46e0 msgr2=0x7f9bac0d23e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:09.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:09.856+0000 7f9bb18a4640 1 -- 172.21.15.38:0/66201814 shutdown_connections 2024-09-21T03:45:09.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:09.856+0000 7f9bb18a4640 1 -- 172.21.15.38:0/66201814 wait complete. 2024-09-21T03:45:09.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:09.858+0000 7f9bb18a4640 1 Processor -- start 2024-09-21T03:45:09.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:09.858+0000 7f9bb18a4640 1 -- start start 2024-09-21T03:45:09.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:09.859+0000 7f9bb18a4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9b94004800 0x7f9bac155340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:09.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:09.859+0000 7f9bb18a4640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9bac1526c0 0x7f9bac1f2ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:09.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:09.859+0000 7f9bb18a4640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9bac1f3420 0x7f9bac0c9490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:09.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:09.859+0000 7f9bb18a4640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f9bac0d0670 con 0x7f9b94004800 2024-09-21T03:45:09.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:09.859+0000 7f9bb18a4640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f9bac0cda30 con 0x7f9bac1526c0 2024-09-21T03:45:09.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:09.859+0000 7f9bb18a4640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f9bac0d07f0 con 0x7f9bac1f3420 2024-09-21T03:45:09.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:09.860+0000 7f9baaffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9b94004800 0x7f9bac155340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:09.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:09.860+0000 7f9baaffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9b94004800 0x7f9bac155340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43644/0 (socket says 172.21.15.38:43644) 2024-09-21T03:45:09.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:09.860+0000 7f9baaffd640 1 -- 172.21.15.38:0/2920257451 learned_addr learned my addr 172.21.15.38:0/2920257451 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:45:09.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:09.860+0000 7f9bab7fe640 1 --2- 172.21.15.38:0/2920257451 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9bac1f3420 0x7f9bac0c9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:09.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:09.860+0000 7f9baa7fc640 1 --2- 172.21.15.38:0/2920257451 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9bac1526c0 0x7f9bac1f2ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:09.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:09.860+0000 7f9baaffd640 1 -- 172.21.15.38:0/2920257451 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9bac1f3420 msgr2=0x7f9bac0c9490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:09.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:09.860+0000 7f9baaffd640 1 --2- 172.21.15.38:0/2920257451 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9bac1f3420 0x7f9bac0c9490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:09.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:09.860+0000 7f9baaffd640 1 -- 172.21.15.38:0/2920257451 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9bac1526c0 msgr2=0x7f9bac1f2ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:09.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:09.860+0000 7f9baaffd640 1 --2- 172.21.15.38:0/2920257451 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9bac1526c0 0x7f9bac1f2ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:09.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:09.860+0000 7f9baaffd640 1 -- 172.21.15.38:0/2920257451 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bac1f3b60 con 0x7f9b94004800 2024-09-21T03:45:09.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:09.860+0000 7f9baa7fc640 1 --2- 172.21.15.38:0/2920257451 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9bac1526c0 0x7f9bac1f2ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:45:09.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:09.860+0000 7f9baaffd640 1 --2- 172.21.15.38:0/2920257451 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9b94004800 0x7f9bac155340 secure :-1 s=READY pgs=2394 cs=0 l=1 rev1=1 crypto rx=0x7f9b9c004d90 tx=0x7f9b9c001630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:09.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:09.860+0000 7f9b93fff640 1 -- 172.21.15.38:0/2920257451 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b9c001d40 con 0x7f9b94004800 2024-09-21T03:45:09.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:09.860+0000 7f9b93fff640 1 -- 172.21.15.38:0/2920257451 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9b9c016070 con 0x7f9b94004800 2024-09-21T03:45:09.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:09.860+0000 7f9b93fff640 1 -- 172.21.15.38:0/2920257451 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b9c010b60 con 0x7f9b94004800 2024-09-21T03:45:09.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:09.860+0000 7f9bb18a4640 1 -- 172.21.15.38:0/2920257451 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bac154860 con 0x7f9b94004800 2024-09-21T03:45:09.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:09.860+0000 7f9bb18a4640 1 -- 172.21.15.38:0/2920257451 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bac154e20 con 0x7f9b94004800 2024-09-21T03:45:09.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:09.861+0000 7f9bb18a4640 1 -- 172.21.15.38:0/2920257451 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9bac0d0d50 con 0x7f9b94004800 2024-09-21T03:45:09.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:09.862+0000 7f9b93fff640 1 -- 172.21.15.38:0/2920257451 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f9b9c010d00 con 0x7f9b94004800 2024-09-21T03:45:09.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:09.863+0000 7f9b93fff640 1 --2- 172.21.15.38:0/2920257451 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9b98078b90 0x7f9b9807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:09.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:09.863+0000 7f9b93fff640 1 -- 172.21.15.38:0/2920257451 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 6101+0+0 (secure 0 0 0) 0x7f9b9c09b540 con 0x7f9b94004800 2024-09-21T03:45:09.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:09.863+0000 7f9baa7fc640 1 --2- 172.21.15.38:0/2920257451 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9b98078b90 0x7f9b9807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:09.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:09.863+0000 7f9baa7fc640 1 --2- 172.21.15.38:0/2920257451 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9b98078b90 0x7f9b9807b0a0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f9ba0033000 tx=0x7f9ba0000fe0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:09.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:09.867+0000 7f9b93fff640 1 -- 172.21.15.38:0/2920257451 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f9b9c063880 con 0x7f9b94004800 2024-09-21T03:45:10.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:10.067+0000 7f9bb18a4640 1 -- 172.21.15.38:0/2920257451 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f9bac159fd0 con 0x7f9b94004800 2024-09-21T03:45:10.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:10.067+0000 7f9b93fff640 1 -- 172.21.15.38:0/2920257451 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v146) ==== 80+0+0 (secure 0 0 0) 0x7f9b9c063180 con 0x7f9b94004800 2024-09-21T03:45:10.069 INFO:teuthology.orchestra.run.smithi038.stderr:listed 0 entries 2024-09-21T03:45:10.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:10.069+0000 7f9bb18a4640 1 -- 172.21.15.38:0/2920257451 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9b98078b90 msgr2=0x7f9b9807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:10.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:10.069+0000 7f9bb18a4640 1 --2- 172.21.15.38:0/2920257451 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9b98078b90 0x7f9b9807b0a0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f9ba0033000 tx=0x7f9ba0000fe0 comp rx=0 tx=0).stop 2024-09-21T03:45:10.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:10.069+0000 7f9bb18a4640 1 -- 172.21.15.38:0/2920257451 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9b94004800 msgr2=0x7f9bac155340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:10.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:10.069+0000 7f9bb18a4640 1 --2- 172.21.15.38:0/2920257451 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9b94004800 0x7f9bac155340 secure :-1 s=READY pgs=2394 cs=0 l=1 rev1=1 crypto rx=0x7f9b9c004d90 tx=0x7f9b9c001630 comp rx=0 tx=0).stop 2024-09-21T03:45:10.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:10.069+0000 7f9bb18a4640 1 -- 172.21.15.38:0/2920257451 shutdown_connections 2024-09-21T03:45:10.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:10.069+0000 7f9bb18a4640 1 --2- 172.21.15.38:0/2920257451 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9b98078b90 0x7f9b9807b0a0 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:10.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:10.069+0000 7f9bb18a4640 1 --2- 172.21.15.38:0/2920257451 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9bac1f3420 0x7f9bac0c9490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:10.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:10.069+0000 7f9bb18a4640 1 --2- 172.21.15.38:0/2920257451 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9bac1526c0 0x7f9bac1f2ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:10.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:10.069+0000 7f9bb18a4640 1 --2- 172.21.15.38:0/2920257451 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9b94004800 0x7f9bac155340 unknown :-1 s=CLOSED pgs=2394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:10.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:10.069+0000 7f9bb18a4640 1 -- 172.21.15.38:0/2920257451 >> 172.21.15.38:0/2920257451 conn(0x7f9bac0b46e0 msgr2=0x7f9bac071230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:10.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:10.070+0000 7f9bb18a4640 1 -- 172.21.15.38:0/2920257451 shutdown_connections 2024-09-21T03:45:10.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:10.070+0000 7f9bb18a4640 1 -- 172.21.15.38:0/2920257451 wait complete. 2024-09-21T03:45:10.083 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-21T03:45:10.083 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:45:10.083 DEBUG:teuthology.orchestra.run.smithi038:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-21T03:45:10.140 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi038.front.sepia.ceph.com 2024-09-21T03:45:10.141 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:45:10.141 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T03:45:10.167 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:45:10.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:10.321+0000 7f7093113640 1 -- 172.21.15.38:0/729484849 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f708c0afd70 msgr2=0x7f708c06f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:10.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:10.321+0000 7f7093113640 1 --2- 172.21.15.38:0/729484849 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f708c0afd70 0x7f708c06f350 secure :-1 s=READY pgs=2396 cs=0 l=1 rev1=1 crypto rx=0x7f708c0cfb80 tx=0x7f708002a510 comp rx=0 tx=0).stop 2024-09-21T03:45:10.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:10.321+0000 7f7093113640 1 -- 172.21.15.38:0/729484849 shutdown_connections 2024-09-21T03:45:10.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:10.321+0000 7f7093113640 1 --2- 172.21.15.38:0/729484849 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f708c0afd70 0x7f708c06f350 unknown :-1 s=CLOSED pgs=2396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:10.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:10.321+0000 7f7093113640 1 -- 172.21.15.38:0/729484849 >> 172.21.15.38:0/729484849 conn(0x7f708c0d0a30 msgr2=0x7f708c0d0e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:10.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:10.321+0000 7f7093113640 1 -- 172.21.15.38:0/729484849 shutdown_connections 2024-09-21T03:45:10.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:10.321+0000 7f7093113640 1 -- 172.21.15.38:0/729484849 wait complete. 2024-09-21T03:45:10.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:10.323+0000 7f7093113640 1 Processor -- start 2024-09-21T03:45:10.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:10.323+0000 7f7093113640 1 -- start start 2024-09-21T03:45:10.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:10.324+0000 7f7093113640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f708c06e1d0 0x7f708c1f57d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:10.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:10.324+0000 7f7093113640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f708c06f890 0x7f708c1517e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:10.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:10.324+0000 7f7093113640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f708c0afd70 0x7f708c0aecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:10.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:10.324+0000 7f7093113640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f708c15b3c0 con 0x7f708c06e1d0 2024-09-21T03:45:10.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:10.324+0000 7f7093113640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f708c15b240 con 0x7f708c06f890 2024-09-21T03:45:10.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:10.324+0000 7f7093113640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f708c15b540 con 0x7f708c0afd70 2024-09-21T03:45:10.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:10.324+0000 7f7090eaf640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f708c06e1d0 0x7f708c1f57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:10.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:10.324+0000 7f7090eaf640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f708c06e1d0 0x7f708c1f57d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43662/0 (socket says 172.21.15.38:43662) 2024-09-21T03:45:10.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:10.324+0000 7f7090eaf640 1 -- 172.21.15.38:0/1260602013 learned_addr learned my addr 172.21.15.38:0/1260602013 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:45:10.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:10.325+0000 7f708bfff640 1 --2- 172.21.15.38:0/1260602013 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f708c06f890 0x7f708c1517e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:10.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:10.325+0000 7f70916b0640 1 --2- 172.21.15.38:0/1260602013 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f708c0afd70 0x7f708c0aecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:10.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:10.325+0000 7f7090eaf640 1 -- 172.21.15.38:0/1260602013 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f708c0afd70 msgr2=0x7f708c0aecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:10.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:10.325+0000 7f7090eaf640 1 --2- 172.21.15.38:0/1260602013 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f708c0afd70 0x7f708c0aecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:10.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:10.325+0000 7f7090eaf640 1 -- 172.21.15.38:0/1260602013 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f708c06f890 msgr2=0x7f708c1517e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:10.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:10.325+0000 7f7090eaf640 1 --2- 172.21.15.38:0/1260602013 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f708c06f890 0x7f708c1517e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:10.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:10.325+0000 7f7090eaf640 1 -- 172.21.15.38:0/1260602013 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f708c151d20 con 0x7f708c06e1d0 2024-09-21T03:45:10.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:10.325+0000 7f7090eaf640 1 --2- 172.21.15.38:0/1260602013 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f708c06e1d0 0x7f708c1f57d0 secure :-1 s=READY pgs=2397 cs=0 l=1 rev1=1 crypto rx=0x7f707c00c3d0 tx=0x7f707c00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:10.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:10.325+0000 7f70897fa640 1 -- 172.21.15.38:0/1260602013 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f707c017070 con 0x7f708c06e1d0 2024-09-21T03:45:10.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:10.325+0000 7f70897fa640 1 -- 172.21.15.38:0/1260602013 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f707c0053c0 con 0x7f708c06e1d0 2024-09-21T03:45:10.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:10.325+0000 7f7093113640 1 -- 172.21.15.38:0/1260602013 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f708c1f60f0 con 0x7f708c06e1d0 2024-09-21T03:45:10.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:10.325+0000 7f70897fa640 1 -- 172.21.15.38:0/1260602013 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f707c003850 con 0x7f708c06e1d0 2024-09-21T03:45:10.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:10.325+0000 7f7093113640 1 -- 172.21.15.38:0/1260602013 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f708c1f66b0 con 0x7f708c06e1d0 2024-09-21T03:45:10.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:10.326+0000 7f7093113640 1 -- 172.21.15.38:0/1260602013 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f708c06f350 con 0x7f708c06e1d0 2024-09-21T03:45:10.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:10.327+0000 7f70897fa640 1 -- 172.21.15.38:0/1260602013 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f707c01a070 con 0x7f708c06e1d0 2024-09-21T03:45:10.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:10.328+0000 7f70897fa640 1 --2- 172.21.15.38:0/1260602013 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7078078bc0 0x7f707807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:10.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:10.328+0000 7f70897fa640 1 -- 172.21.15.38:0/1260602013 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 6101+0+0 (secure 0 0 0) 0x7f707c09a130 con 0x7f708c06e1d0 2024-09-21T03:45:10.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:10.328+0000 7f708bfff640 1 --2- 172.21.15.38:0/1260602013 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7078078bc0 0x7f707807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:10.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:10.329+0000 7f708bfff640 1 --2- 172.21.15.38:0/1260602013 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7078078bc0 0x7f707807b0d0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f7080034000 tx=0x7f7080004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:10.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:10.333+0000 7f70897fa640 1 -- 172.21.15.38:0/1260602013 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f707c061e20 con 0x7f708c06e1d0 2024-09-21T03:45:10.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:10.556+0000 7f7093113640 1 -- 172.21.15.38:0/1260602013 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f708c0cca10 con 0x7f708c06e1d0 2024-09-21T03:45:10.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:10.557+0000 7f70897fa640 1 -- 172.21.15.38:0/1260602013 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 343 v343) ==== 95+0+5239 (secure 0 0 0) 0x7f707c066850 con 0x7f708c06e1d0 2024-09-21T03:45:10.559 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:45:10.559 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":343,"btime":"2024-09-21T03:45:00:366373+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8399,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/3606957623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":3606957623},{"type":"v1","addr":"172.21.15.38:6835","nonce":3606957623}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":338},{"gid":8405,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/456651486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":456651486},{"type":"v1","addr":"172.21.15.195:6837","nonce":456651486}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":338},{"gid":9604,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/2649311840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":2649311840},{"type":"v1","addr":"172.21.15.195:6835","nonce":2649311840}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":338}],"filesystems":[{"mdsmap":{"epoch":343,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:44:56.710446+0000","modified":"2024-09-21T03:44:59.752336+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10719},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10719":{"gid":10719,"name":"c","rank":0,"incarnation":339,"state":"up:active","state_seq":3,"addr":"172.21.15.38:6833/3866292839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":3866292839},{"type":"v1","addr":"172.21.15.38:6833","nonce":3866292839}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10719,"qdb_cluster":[10719]},"id":7}]} 2024-09-21T03:45:10.559 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 343 2024-09-21T03:45:10.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:10.559+0000 7f7093113640 1 -- 172.21.15.38:0/1260602013 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7078078bc0 msgr2=0x7f707807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:10.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:10.559+0000 7f7093113640 1 --2- 172.21.15.38:0/1260602013 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7078078bc0 0x7f707807b0d0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f7080034000 tx=0x7f7080004730 comp rx=0 tx=0).stop 2024-09-21T03:45:10.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:10.559+0000 7f7093113640 1 -- 172.21.15.38:0/1260602013 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f708c06e1d0 msgr2=0x7f708c1f57d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:10.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:10.559+0000 7f7093113640 1 --2- 172.21.15.38:0/1260602013 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f708c06e1d0 0x7f708c1f57d0 secure :-1 s=READY pgs=2397 cs=0 l=1 rev1=1 crypto rx=0x7f707c00c3d0 tx=0x7f707c00c890 comp rx=0 tx=0).stop 2024-09-21T03:45:10.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:10.559+0000 7f7093113640 1 -- 172.21.15.38:0/1260602013 shutdown_connections 2024-09-21T03:45:10.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:10.559+0000 7f7093113640 1 --2- 172.21.15.38:0/1260602013 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7078078bc0 0x7f707807b0d0 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:10.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:10.559+0000 7f7093113640 1 --2- 172.21.15.38:0/1260602013 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f708c0afd70 0x7f708c0aecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:10.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:10.559+0000 7f7093113640 1 --2- 172.21.15.38:0/1260602013 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f708c06f890 0x7f708c1517e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:10.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:10.559+0000 7f7093113640 1 --2- 172.21.15.38:0/1260602013 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f708c06e1d0 0x7f708c1f57d0 unknown :-1 s=CLOSED pgs=2397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:10.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:10.559+0000 7f7093113640 1 -- 172.21.15.38:0/1260602013 >> 172.21.15.38:0/1260602013 conn(0x7f708c0d0a30 msgr2=0x7f708c0b0aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:10.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:10.559+0000 7f7093113640 1 -- 172.21.15.38:0/1260602013 shutdown_connections 2024-09-21T03:45:10.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:10.559+0000 7f7093113640 1 -- 172.21.15.38:0/1260602013 wait complete. 2024-09-21T03:45:10.574 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2024-09-21T03:45:10.713 INFO:teuthology.orchestra.run.smithi038.stdout:{"message":"","return_code":0} 2024-09-21T03:45:10.714 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-09-21T03:45:10.714 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-21T03:45:10.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:10.830+0000 7fdd4657b640 1 -- 172.21.15.38:0/3302181336 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd4006ddf0 msgr2=0x7fdd4006e200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:10.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:10.830+0000 7fdd4657b640 1 --2- 172.21.15.38:0/3302181336 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd4006ddf0 0x7fdd4006e200 secure :-1 s=READY pgs=2399 cs=0 l=1 rev1=1 crypto rx=0x7fdd40072500 tx=0x7fdd3802a820 comp rx=0 tx=0).stop 2024-09-21T03:45:10.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:10.830+0000 7fdd4657b640 1 -- 172.21.15.38:0/3302181336 shutdown_connections 2024-09-21T03:45:10.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:10.830+0000 7fdd4657b640 1 --2- 172.21.15.38:0/3302181336 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd4006ddf0 0x7fdd4006e200 unknown :-1 s=CLOSED pgs=2399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:10.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:10.830+0000 7fdd4657b640 1 --2- 172.21.15.38:0/3302181336 >> v2:172.21.15.195:3300/0 conn(0x7fdd4015b730 0x7fdd400b5d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:10.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:10.830+0000 7fdd4657b640 1 -- 172.21.15.38:0/3302181336 >> 172.21.15.38:0/3302181336 conn(0x7fdd400735d0 msgr2=0x7fdd400739e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:10.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:10.830+0000 7fdd4657b640 1 -- 172.21.15.38:0/3302181336 shutdown_connections 2024-09-21T03:45:10.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:10.830+0000 7fdd4657b640 1 -- 172.21.15.38:0/3302181336 wait complete. 2024-09-21T03:45:10.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:10.832+0000 7fdd4657b640 1 Processor -- start 2024-09-21T03:45:10.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:10.832+0000 7fdd4657b640 1 -- start start 2024-09-21T03:45:10.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:10.834+0000 7fdd4657b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdd4006ddf0 0x7fdd4015d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:10.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:10.834+0000 7fdd4657b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdd400b6290 0x7fdd401579b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:10.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:10.834+0000 7fdd4657b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd4015b730 0x7fdd400ce4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:10.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:10.834+0000 7fdd4657b640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fdd40071820 con 0x7fdd4015b730 2024-09-21T03:45:10.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:10.834+0000 7fdd4657b640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fdd400716a0 con 0x7fdd400b6290 2024-09-21T03:45:10.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:10.834+0000 7fdd4657b640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fdd400719a0 con 0x7fdd4006ddf0 2024-09-21T03:45:10.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:10.834+0000 7fdd44b18640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd4015b730 0x7fdd400ce4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:10.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:10.834+0000 7fdd44b18640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd4015b730 0x7fdd400ce4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43672/0 (socket says 172.21.15.38:43672) 2024-09-21T03:45:10.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:10.834+0000 7fdd44b18640 1 -- 172.21.15.38:0/145791189 learned_addr learned my addr 172.21.15.38:0/145791189 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:45:10.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:10.834+0000 7fdd3f7fe640 1 --2- 172.21.15.38:0/145791189 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdd400b6290 0x7fdd401579b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:10.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:10.834+0000 7fdd3ffff640 1 --2- 172.21.15.38:0/145791189 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdd4006ddf0 0x7fdd4015d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:10.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:10.834+0000 7fdd44b18640 1 -- 172.21.15.38:0/145791189 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdd4006ddf0 msgr2=0x7fdd4015d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:10.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:10.834+0000 7fdd44b18640 1 --2- 172.21.15.38:0/145791189 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdd4006ddf0 0x7fdd4015d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:10.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:10.834+0000 7fdd44b18640 1 -- 172.21.15.38:0/145791189 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdd400b6290 msgr2=0x7fdd401579b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:10.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:10.834+0000 7fdd44b18640 1 --2- 172.21.15.38:0/145791189 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdd400b6290 0x7fdd401579b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:10.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:10.834+0000 7fdd44b18640 1 -- 172.21.15.38:0/145791189 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd40157ef0 con 0x7fdd4015b730 2024-09-21T03:45:10.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:10.834+0000 7fdd44b18640 1 --2- 172.21.15.38:0/145791189 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd4015b730 0x7fdd400ce4c0 secure :-1 s=READY pgs=2400 cs=0 l=1 rev1=1 crypto rx=0x7fdd38009ca0 tx=0x7fdd38032d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:10.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:10.835+0000 7fdd3cff9640 1 -- 172.21.15.38:0/145791189 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd3803f020 con 0x7fdd4015b730 2024-09-21T03:45:10.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:10.835+0000 7fdd3cff9640 1 -- 172.21.15.38:0/145791189 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdd38004040 con 0x7fdd4015b730 2024-09-21T03:45:10.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:10.835+0000 7fdd4657b640 1 -- 172.21.15.38:0/145791189 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd4015dbd0 con 0x7fdd4015b730 2024-09-21T03:45:10.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:10.835+0000 7fdd3cff9640 1 -- 172.21.15.38:0/145791189 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd38037040 con 0x7fdd4015b730 2024-09-21T03:45:10.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:10.835+0000 7fdd4657b640 1 -- 172.21.15.38:0/145791189 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd4015e190 con 0x7fdd4015b730 2024-09-21T03:45:10.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:10.836+0000 7fdd4657b640 1 -- 172.21.15.38:0/145791189 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd4015d940 con 0x7fdd4015b730 2024-09-21T03:45:10.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:10.836+0000 7fdd3cff9640 1 -- 172.21.15.38:0/145791189 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fdd38042020 con 0x7fdd4015b730 2024-09-21T03:45:10.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:10.837+0000 7fdd3cff9640 1 --2- 172.21.15.38:0/145791189 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fdd2c078bc0 0x7fdd2c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:10.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:10.837+0000 7fdd3cff9640 1 -- 172.21.15.38:0/145791189 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 6101+0+0 (secure 0 0 0) 0x7fdd380bb500 con 0x7fdd4015b730 2024-09-21T03:45:10.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:10.837+0000 7fdd3ffff640 1 --2- 172.21.15.38:0/145791189 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fdd2c078bc0 0x7fdd2c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:10.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:10.838+0000 7fdd3ffff640 1 --2- 172.21.15.38:0/145791189 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fdd2c078bc0 0x7fdd2c07b0d0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fdd3000bb50 tx=0x7fdd3000b2e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:10.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:10.842+0000 7fdd3cff9640 1 -- 172.21.15.38:0/145791189 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fdd38049030 con 0x7fdd4015b730 2024-09-21T03:45:11.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.067+0000 7fdd4657b640 1 -- 172.21.15.38:0/145791189 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fdd40071f30 con 0x7fdd4015b730 2024-09-21T03:45:11.069 INFO:tasks.ceph.mon.a.smithi038.stderr:2024-09-21T03:45:11.067+0000 7f3f45713640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T03:45:11.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.073+0000 7fdd3cff9640 1 -- 172.21.15.38:0/145791189 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v344) ==== 161+0+0 (secure 0 0 0) 0x7fdd380831f0 con 0x7fdd4015b730 2024-09-21T03:45:11.075 INFO:teuthology.orchestra.run.smithi038.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-21T03:45:11.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.077+0000 7fdd4657b640 1 -- 172.21.15.38:0/145791189 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fdd2c078bc0 msgr2=0x7fdd2c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:11.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.077+0000 7fdd4657b640 1 --2- 172.21.15.38:0/145791189 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fdd2c078bc0 0x7fdd2c07b0d0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fdd3000bb50 tx=0x7fdd3000b2e0 comp rx=0 tx=0).stop 2024-09-21T03:45:11.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.077+0000 7fdd4657b640 1 -- 172.21.15.38:0/145791189 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd4015b730 msgr2=0x7fdd400ce4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:11.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.077+0000 7fdd4657b640 1 --2- 172.21.15.38:0/145791189 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd4015b730 0x7fdd400ce4c0 secure :-1 s=READY pgs=2400 cs=0 l=1 rev1=1 crypto rx=0x7fdd38009ca0 tx=0x7fdd38032d10 comp rx=0 tx=0).stop 2024-09-21T03:45:11.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.077+0000 7fdd4657b640 1 -- 172.21.15.38:0/145791189 shutdown_connections 2024-09-21T03:45:11.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.077+0000 7fdd4657b640 1 --2- 172.21.15.38:0/145791189 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd4015b730 0x7fdd400ce4c0 unknown :-1 s=CLOSED pgs=2400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:11.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.077+0000 7fdd4657b640 1 --2- 172.21.15.38:0/145791189 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdd400b6290 0x7fdd401579b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:11.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.077+0000 7fdd4657b640 1 --2- 172.21.15.38:0/145791189 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fdd2c078bc0 0x7fdd2c07b0d0 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:11.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.077+0000 7fdd4657b640 1 --2- 172.21.15.38:0/145791189 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdd4006ddf0 0x7fdd4015d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:11.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.077+0000 7fdd4657b640 1 -- 172.21.15.38:0/145791189 >> 172.21.15.38:0/145791189 conn(0x7fdd400735d0 msgr2=0x7fdd400cb6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:11.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.077+0000 7fdd4657b640 1 -- 172.21.15.38:0/145791189 shutdown_connections 2024-09-21T03:45:11.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.077+0000 7fdd4657b640 1 -- 172.21.15.38:0/145791189 wait complete. 2024-09-21T03:45:11.082 INFO:tasks.ceph.mds.c.smithi038.stderr: -7017> 2024-09-21T03:45:03.303+0000 7f7ef4be4640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T03:45:11.091 DEBUG:teuthology.orchestra.run.smithi038:> cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank cephfs:0 journal reset --yes-i-really-really-mean-it 2024-09-21T03:45:11.143 INFO:tasks.ceph.mds.c.smithi038.stdout:starting mds.c at 2024-09-21T03:45:11.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.145+0000 7f7085b21280 1 -- start start 2024-09-21T03:45:11.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.145+0000 7f7085b21280 1 Processor -- start 2024-09-21T03:45:11.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.147+0000 7f7085b21280 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563831405680 0x56383212eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:11.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.147+0000 7f7085b21280 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563831405b00 0x56383212e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:11.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.147+0000 7f7085b21280 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x563831406400 0x56383212e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:11.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.147+0000 7f7085b21280 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x56383144e180 con 0x563831405680 2024-09-21T03:45:11.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.147+0000 7f7085b21280 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5638313b3800 con 0x563831405b00 2024-09-21T03:45:11.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.147+0000 7f7085b21280 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x5638313b3380 con 0x563831406400 2024-09-21T03:45:11.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.147+0000 7f708385f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563831405680 0x56383212eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:11.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.147+0000 7f708385f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563831405680 0x56383212eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43700/0 (socket says 172.21.15.38:43700) 2024-09-21T03:45:11.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.147+0000 7f708385f640 1 -- 172.21.15.38:0/2762339813 learned_addr learned my addr 172.21.15.38:0/2762339813 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:45:11.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.147+0000 7f708305e640 1 --2- 172.21.15.38:0/2762339813 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563831405b00 0x56383212e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:11.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.147+0000 7f7084060640 1 --2- 172.21.15.38:0/2762339813 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x563831406400 0x56383212e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:11.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.147+0000 7f708385f640 1 -- 172.21.15.38:0/2762339813 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x563831406400 msgr2=0x56383212e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:11.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.147+0000 7f708385f640 1 --2- 172.21.15.38:0/2762339813 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x563831406400 0x56383212e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:11.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.147+0000 7f708385f640 1 -- 172.21.15.38:0/2762339813 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563831405b00 msgr2=0x56383212e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:11.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.147+0000 7f708385f640 1 --2- 172.21.15.38:0/2762339813 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563831405b00 0x56383212e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:11.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.147+0000 7f708385f640 1 -- 172.21.15.38:0/2762339813 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56383138e780 con 0x563831405680 2024-09-21T03:45:11.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.147+0000 7f708385f640 1 --2- 172.21.15.38:0/2762339813 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563831405680 0x56383212eb00 secure :-1 s=READY pgs=2404 cs=0 l=1 rev1=1 crypto rx=0x563832185ce0 tx=0x563832142180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a2f74833a591027 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:11.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.147+0000 7f708105a640 1 -- 172.21.15.38:0/2762339813 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x56382d346340 con 0x563831405680 2024-09-21T03:45:11.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.147+0000 7f708105a640 1 -- 172.21.15.38:0/2762339813 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x56382d34c540 con 0x563831405680 2024-09-21T03:45:11.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.147+0000 7f708105a640 1 -- 172.21.15.38:0/2762339813 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x56382d346ea0 con 0x563831405680 2024-09-21T03:45:11.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.147+0000 7f7085b21280 1 -- 172.21.15.38:0/2762339813 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56383138f4a0 con 0x563831405680 2024-09-21T03:45:11.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.148+0000 7f7085b21280 1 -- 172.21.15.38:0/2762339813 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({fsmap=0}) -- 0x56383138e000 con 0x563831405680 2024-09-21T03:45:11.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.148+0000 7f7085b21280 4 waiting for MDS map... 2024-09-21T03:45:11.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.148+0000 7f708105a640 1 -- 172.21.15.38:0/2762339813 <== mon.0 v2:172.21.15.38:3300/0 4 ==== osd_map(147..147 src has 1..147) ==== 6187+0+0 (secure 0 0 0) 0x563832236000 con 0x563831405680 2024-09-21T03:45:11.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.148+0000 7f708105a640 1 -- 172.21.15.38:0/2762339813 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 344) ==== 2893+0+0 (secure 0 0 0) 0x5638313ab180 con 0x563831405680 2024-09-21T03:45:11.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.148+0000 7f7085b21280 4 Got MDS map 344 2024-09-21T03:45:11.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.148+0000 7f7085b21280 10 main: JournalTool::main 2024-09-21T03:45:11.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.148+0000 7f7085b21280 4 main: JournalTool: connecting to RADOS... 2024-09-21T03:45:11.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.149+0000 7f7085b21280 1 Processor -- start 2024-09-21T03:45:11.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.149+0000 7f7085b21280 1 -- start start 2024-09-21T03:45:11.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.150+0000 7f7085b21280 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x563831404d80 0x56383212f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:11.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.150+0000 7f7085b21280 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563831404900 0x56383212f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:11.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.150+0000 7f7085b21280 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563831404480 0x56383212fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:11.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.150+0000 7f7085b21280 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5638313b3980 con 0x563831404480 2024-09-21T03:45:11.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.150+0000 7f7085b21280 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x56383144ed80 con 0x563831404900 2024-09-21T03:45:11.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.150+0000 7f7085b21280 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x56383144ef00 con 0x563831404d80 2024-09-21T03:45:11.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.150+0000 7f708305e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563831404480 0x56383212fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:11.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.150+0000 7f708305e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563831404480 0x56383212fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43708/0 (socket says 172.21.15.38:43708) 2024-09-21T03:45:11.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.150+0000 7f708305e640 1 -- 172.21.15.38:0/2367330767 learned_addr learned my addr 172.21.15.38:0/2367330767 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:45:11.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.150+0000 7f708385f640 1 --2- 172.21.15.38:0/2367330767 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563831404900 0x56383212f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:11.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.151+0000 7f708305e640 1 -- 172.21.15.38:0/2367330767 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x563831404d80 msgr2=0x56383212f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:11.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.151+0000 7f7084060640 1 --2- 172.21.15.38:0/2367330767 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x563831404d80 0x56383212f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:11.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.151+0000 7f708305e640 1 --2- 172.21.15.38:0/2367330767 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x563831404d80 0x56383212f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:11.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.151+0000 7f708305e640 1 -- 172.21.15.38:0/2367330767 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563831404900 msgr2=0x56383212f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:11.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.151+0000 7f708305e640 1 --2- 172.21.15.38:0/2367330767 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563831404900 0x56383212f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:11.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.151+0000 7f708305e640 1 -- 172.21.15.38:0/2367330767 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56383138e3c0 con 0x563831404480 2024-09-21T03:45:11.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.151+0000 7f708385f640 1 --2- 172.21.15.38:0/2367330767 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563831404900 0x56383212f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:45:11.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.151+0000 7f7084060640 1 --2- 172.21.15.38:0/2367330767 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x563831404d80 0x56383212f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:45:11.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.151+0000 7f708305e640 1 --2- 172.21.15.38:0/2367330767 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563831404480 0x56383212fb80 secure :-1 s=READY pgs=2405 cs=0 l=1 rev1=1 crypto rx=0x563832238b40 tx=0x563832244c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c02182e2afc2deb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:11.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.151+0000 7f707f056640 1 -- 172.21.15.38:0/2367330767 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x56382d346340 con 0x563831404480 2024-09-21T03:45:11.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.151+0000 7f707f056640 1 -- 172.21.15.38:0/2367330767 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x56382d34cfc0 con 0x563831404480 2024-09-21T03:45:11.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.151+0000 7f707f056640 1 -- 172.21.15.38:0/2367330767 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x56382d346ea0 con 0x563831404480 2024-09-21T03:45:11.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.151+0000 7f7085b21280 1 -- 172.21.15.38:0/2367330767 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563831404480 msgr2=0x56383212fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:11.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.151+0000 7f7085b21280 1 --2- 172.21.15.38:0/2367330767 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563831404480 0x56383212fb80 secure :-1 s=READY pgs=2405 cs=0 l=1 rev1=1 crypto rx=0x563832238b40 tx=0x563832244c00 comp rx=0 tx=0).stop 2024-09-21T03:45:11.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.152+0000 7f7085b21280 1 -- 172.21.15.38:0/2367330767 shutdown_connections 2024-09-21T03:45:11.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.152+0000 7f7085b21280 1 --2- 172.21.15.38:0/2367330767 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563831404480 0x56383212fb80 unknown :-1 s=CLOSED pgs=2405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:11.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.152+0000 7f7085b21280 1 --2- 172.21.15.38:0/2367330767 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563831404900 0x56383212f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:11.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.152+0000 7f7085b21280 1 --2- 172.21.15.38:0/2367330767 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x563831404d80 0x56383212f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:11.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.152+0000 7f7085b21280 1 -- 172.21.15.38:0/2367330767 >> 172.21.15.38:0/2367330767 conn(0x563831405200 msgr2=0x563831479800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:11.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.152+0000 7f7085b21280 1 -- 172.21.15.38:0/2367330767 shutdown_connections 2024-09-21T03:45:11.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.152+0000 7f7085b21280 1 -- 172.21.15.38:0/2367330767 wait complete. 2024-09-21T03:45:11.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.152+0000 7f7085b21280 1 Processor -- start 2024-09-21T03:45:11.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.153+0000 7f7085b21280 1 -- start start 2024-09-21T03:45:11.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.154+0000 7f7085b21280 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563831404480 0x56383212fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:11.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.154+0000 7f7085b21280 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x563831404900 0x56383212f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:11.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.154+0000 7f7085b21280 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563831404d80 0x56383212f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:11.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.154+0000 7f7085b21280 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x56383144f200 con 0x563831404480 2024-09-21T03:45:11.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.154+0000 7f7085b21280 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x56383144f080 con 0x563831404d80 2024-09-21T03:45:11.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.154+0000 7f7085b21280 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x56383144f380 con 0x563831404900 2024-09-21T03:45:11.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.154+0000 7f7084060640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563831404480 0x56383212fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:11.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.154+0000 7f7084060640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563831404480 0x56383212fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43722/0 (socket says 172.21.15.38:43722) 2024-09-21T03:45:11.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.154+0000 7f7084060640 1 -- 172.21.15.38:0/3022837076 learned_addr learned my addr 172.21.15.38:0/3022837076 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:45:11.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.154+0000 7f708305e640 1 --2- 172.21.15.38:0/3022837076 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563831404d80 0x56383212f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:11.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.154+0000 7f708385f640 1 --2- 172.21.15.38:0/3022837076 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x563831404900 0x56383212f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:11.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.154+0000 7f7084060640 1 -- 172.21.15.38:0/3022837076 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x563831404900 msgr2=0x56383212f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:11.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.154+0000 7f7084060640 1 --2- 172.21.15.38:0/3022837076 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x563831404900 0x56383212f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:11.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.154+0000 7f7084060640 1 -- 172.21.15.38:0/3022837076 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563831404d80 msgr2=0x56383212f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:11.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.154+0000 7f7084060640 1 --2- 172.21.15.38:0/3022837076 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563831404d80 0x56383212f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:11.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.154+0000 7f7084060640 1 -- 172.21.15.38:0/3022837076 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56383138e960 con 0x563831404480 2024-09-21T03:45:11.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.154+0000 7f7084060640 1 --2- 172.21.15.38:0/3022837076 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563831404480 0x56383212fb80 secure :-1 s=READY pgs=2406 cs=0 l=1 rev1=1 crypto rx=0x563832239e30 tx=0x563832244f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:11.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.154+0000 7f707d853640 1 -- 172.21.15.38:0/3022837076 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x56382d346340 con 0x563831404480 2024-09-21T03:45:11.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.154+0000 7f707d853640 1 -- 172.21.15.38:0/3022837076 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x56382d34c380 con 0x563831404480 2024-09-21T03:45:11.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.155+0000 7f7085b21280 1 -- 172.21.15.38:0/3022837076 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56383138f680 con 0x563831404480 2024-09-21T03:45:11.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.155+0000 7f707d853640 1 -- 172.21.15.38:0/3022837076 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x56382d346ea0 con 0x563831404480 2024-09-21T03:45:11.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.155+0000 7f7085b21280 1 -- 172.21.15.38:0/3022837076 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56383138f860 con 0x563831404480 2024-09-21T03:45:11.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.155+0000 7f7085b21280 4 main: JournalTool: resolving pool 15 2024-09-21T03:45:11.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.156+0000 7f707d853640 1 -- 172.21.15.38:0/3022837076 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x5638322a8c00 con 0x563831404480 2024-09-21T03:45:11.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.156+0000 7f707d853640 1 --2- 172.21.15.38:0/3022837076 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x563831407180 0x563832130100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:11.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.156+0000 7f707d853640 1 -- 172.21.15.38:0/3022837076 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 6187+0+0 (secure 0 0 0) 0x5638322a3000 con 0x563831404480 2024-09-21T03:45:11.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.156+0000 7f7085b21280 4 main: JournalTool: creating IoCtx.. 2024-09-21T03:45:11.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.156+0000 7f7085b21280 4 main: Executing for rank 0 2024-09-21T03:45:11.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.156+0000 7f708385f640 1 --2- 172.21.15.38:0/3022837076 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x563831407180 0x563832130100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:11.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.157+0000 7f708385f640 1 --2- 172.21.15.38:0/3022837076 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x563831407180 0x563832130100 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x563832185d40 tx=0x5638321429c0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:11.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.157+0000 7f7085b21280 1 -- start start 2024-09-21T03:45:11.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.157+0000 7f7085b21280 1 Processor -- start 2024-09-21T03:45:11.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.158+0000 7f7085b21280 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563831406880 0x563832130680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:11.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.158+0000 7f7085b21280 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x563831406d00 0x563832130c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:11.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.158+0000 7f7085b21280 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563831407600 0x563832131180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:11.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.158+0000 7f7085b21280 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x56383144ef00 con 0x563831406880 2024-09-21T03:45:11.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.158+0000 7f7085b21280 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x56383144ed80 con 0x563831407600 2024-09-21T03:45:11.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.158+0000 7f7085b21280 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x56383144f800 con 0x563831406d00 2024-09-21T03:45:11.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.158+0000 7f7084060640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563831406880 0x563832130680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:11.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.158+0000 7f7084060640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563831406880 0x563832130680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43726/0 (socket says 172.21.15.38:43726) 2024-09-21T03:45:11.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.158+0000 7f7084060640 1 -- 172.21.15.38:0/1717830250 learned_addr learned my addr 172.21.15.38:0/1717830250 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:45:11.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.158+0000 7f708385f640 1 --2- 172.21.15.38:0/1717830250 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x563831406d00 0x563832130c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:11.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.158+0000 7f708305e640 1 --2- 172.21.15.38:0/1717830250 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563831407600 0x563832131180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:11.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.159+0000 7f7084060640 1 -- 172.21.15.38:0/1717830250 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x563831406d00 msgr2=0x563832130c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:11.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.159+0000 7f7084060640 1 --2- 172.21.15.38:0/1717830250 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x563831406d00 0x563832130c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:11.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.159+0000 7f7084060640 1 -- 172.21.15.38:0/1717830250 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563831407600 msgr2=0x563832131180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:11.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.159+0000 7f7084060640 1 --2- 172.21.15.38:0/1717830250 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563831407600 0x563832131180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:11.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.159+0000 7f7084060640 1 -- 172.21.15.38:0/1717830250 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56383138fa40 con 0x563831406880 2024-09-21T03:45:11.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.159+0000 7f708305e640 1 --2- 172.21.15.38:0/1717830250 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563831407600 0x563832131180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:45:11.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.159+0000 7f7084060640 1 --2- 172.21.15.38:0/1717830250 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563831406880 0x563832130680 secure :-1 s=READY pgs=2407 cs=0 l=1 rev1=1 crypto rx=0x5638317aa6f0 tx=0x5638322459e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=479e664742213595 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:11.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.159+0000 7f707a84d640 1 -- 172.21.15.38:0/1717830250 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5638316db6c0 con 0x563831406880 2024-09-21T03:45:11.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.159+0000 7f707a84d640 1 -- 172.21.15.38:0/1717830250 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x56382d34cfc0 con 0x563831406880 2024-09-21T03:45:11.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.159+0000 7f7085b21280 1 -- 172.21.15.38:0/1717830250 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56383138e960 con 0x563831406880 2024-09-21T03:45:11.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.159+0000 7f707a84d640 1 -- 172.21.15.38:0/1717830250 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5638316db860 con 0x563831406880 2024-09-21T03:45:11.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.159+0000 7f7085b21280 1 -- 172.21.15.38:0/1717830250 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({fsmap=0}) -- 0x56383138f680 con 0x563831406880 2024-09-21T03:45:11.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.159+0000 7f7085b21280 4 waiting for MDS map... 2024-09-21T03:45:11.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.159+0000 7f707a84d640 1 -- 172.21.15.38:0/1717830250 <== mon.0 v2:172.21.15.38:3300/0 4 ==== osd_map(147..147 src has 1..147) ==== 6187+0+0 (secure 0 0 0) 0x5638317be200 con 0x563831406880 2024-09-21T03:45:11.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.159+0000 7f707a84d640 1 -- 172.21.15.38:0/1717830250 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 344) ==== 2893+0+0 (secure 0 0 0) 0x5638313ab500 con 0x563831406880 2024-09-21T03:45:11.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.159+0000 7f7085b21280 4 Got MDS map 344 2024-09-21T03:45:11.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.159+0000 7f7085b21280 1 --2- 172.21.15.38:0/1717830250 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x563831407a80 0x563832131700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:11.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.159+0000 7f7085b21280 1 -- 172.21.15.38:0/1717830250 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- osd_op(unknown.0.0:1 15.f 15:f1f69726:::400.00000000:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e147) -- 0x5638317d2400 con 0x563831407a80 2024-09-21T03:45:11.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.159+0000 7f708385f640 1 --2- 172.21.15.38:0/1717830250 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x563831407a80 0x563832131700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:11.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.160+0000 7f708385f640 1 --2- 172.21.15.38:0/1717830250 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x563831407a80 0x563832131700 crc :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=e0d57587b8dc086d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:11.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.160+0000 7f708385f640 1 -- 172.21.15.38:0/1717830250 <== osd.2 v2:172.21.15.38:6824/2577858902 1 ==== osd_op_reply(1 400.00000000 [read 0~22 out=22b] v0'0 uv1 ondisk = 0) ==== 156+0+22 (crc 0 0 0) 0x563831494000 con 0x563831407a80 2024-09-21T03:45:11.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.160+0000 7f7085b21280 1 --2- 172.21.15.38:0/1717830250 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x5638317d8000 0x5638317e0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:11.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.160+0000 7f7085b21280 1 -- 172.21.15.38:0/1717830250 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:2 15.14 15:292cf221:::200.00000000:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e147) -- 0x5638317d2800 con 0x5638317d8000 2024-09-21T03:45:11.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.161+0000 7f708305e640 1 --2- 172.21.15.38:0/1717830250 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x5638317d8000 0x5638317e0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:11.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.161+0000 7f708305e640 1 --2- 172.21.15.38:0/1717830250 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x5638317d8000 0x5638317e0000 crc :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=dd84958460d97983 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:11.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.161+0000 7f708305e640 1 -- 172.21.15.38:0/1717830250 <== osd.3 v2:172.21.15.38:6808/1286810294 1 ==== osd_op_reply(2 200.00000000 [read 0~90 out=90b] v0'0 uv21 ondisk = 0) ==== 156+0+90 (crc 0 0 0) 0x563831494500 con 0x5638317d8000 2024-09-21T03:45:11.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.161+0000 7f707904a640 1 -- 172.21.15.38:0/1717830250 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:3 15.14 15:2e2fa760:::200.00000001:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e147) -- 0x563832147c00 con 0x5638317d8000 2024-09-21T03:45:11.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.161+0000 7f707904a640 1 -- 172.21.15.38:0/1717830250 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:4 15.1b 15:ddb4e4d7:::200.00000002:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e147) -- 0x563832147400 con 0x5638317d8000 2024-09-21T03:45:11.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.161+0000 7f708305e640 1 -- 172.21.15.38:0/1717830250 <== osd.3 v2:172.21.15.38:6808/1286810294 2 ==== osd_op_reply(3 200.00000001 [stat out=16b] v0'0 uv19 ondisk = 0) ==== 156+0+16 (crc 0 0 0) 0x563831494500 con 0x5638317d8000 2024-09-21T03:45:11.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.161+0000 7f708305e640 1 -- 172.21.15.38:0/1717830250 <== osd.3 v2:172.21.15.38:6808/1286810294 3 ==== osd_op_reply(4 200.00000002 [stat] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x563831494500 con 0x5638317d8000 2024-09-21T03:45:11.169 INFO:teuthology.orchestra.run.smithi038.stdout:old journal was 4226629~2030 2024-09-21T03:45:11.169 INFO:teuthology.orchestra.run.smithi038.stdout:new journal start will be 8388608 (4159949 bytes past old end) 2024-09-21T03:45:11.169 INFO:teuthology.orchestra.run.smithi038.stdout:writing journal head 2024-09-21T03:45:11.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.162+0000 7f7085b21280 1 -- 172.21.15.38:0/1717830250 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:5 15.14 15:292cf221:::200.00000000:head [writefull 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e147) -- 0x5638317d2800 con 0x5638317d8000 2024-09-21T03:45:11.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.163+0000 7f708305e640 1 -- 172.21.15.38:0/1717830250 <== osd.3 v2:172.21.15.38:6808/1286810294 4 ==== osd_op_reply(5 200.00000000 [writefull 0~90] v147'22 uv22 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x563831494500 con 0x5638317d8000 2024-09-21T03:45:11.170 INFO:teuthology.orchestra.run.smithi038.stdout:writing EResetJournal entry 2024-09-21T03:45:11.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.163+0000 7f7085b21280 1 -- 172.21.15.38:0/1717830250 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:6 15.1b 15:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e147) -- 0x5638317d2c00 con 0x5638317d8000 2024-09-21T03:45:11.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.163+0000 7f7085b21280 1 --2- 172.21.15.38:0/1717830250 >> [v2:172.21.15.195:6810/2080960180,v1:172.21.15.195:6811/2080960180] conn(0x5638317d8480 0x5638317e0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:11.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.163+0000 7f7085b21280 1 -- 172.21.15.38:0/1717830250 --> [v2:172.21.15.195:6810/2080960180,v1:172.21.15.195:6811/2080960180] -- osd_op(unknown.0.0:7 15.b 15:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e147) -- 0x5638317d3000 con 0x5638317d8480 2024-09-21T03:45:11.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.163+0000 7f7085b21280 1 --2- 172.21.15.38:0/1717830250 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x5638317d8900 0x5638317e0b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:11.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.163+0000 7f7085b21280 1 -- 172.21.15.38:0/1717830250 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- osd_op(unknown.0.0:8 15.1f 15:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e147) -- 0x5638317d3400 con 0x5638317d8900 2024-09-21T03:45:11.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.163+0000 7f7085b21280 1 --2- 172.21.15.38:0/1717830250 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x5638317d8d80 0x5638317e1080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:11.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.163+0000 7f7085b21280 1 -- 172.21.15.38:0/1717830250 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- osd_op(unknown.0.0:9 15.1a 15:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e147) -- 0x5638317d3800 con 0x5638317d8d80 2024-09-21T03:45:11.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.163+0000 7f7085b21280 1 -- 172.21.15.38:0/1717830250 --> [v2:172.21.15.195:6810/2080960180,v1:172.21.15.195:6811/2080960180] -- osd_op(unknown.0.0:10 15.1e 15:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e147) -- 0x5638317d3c00 con 0x5638317d8480 2024-09-21T03:45:11.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.163+0000 7f7085b21280 1 --2- 172.21.15.38:0/1717830250 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x5638317d9200 0x5638317e1600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:11.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.163+0000 7f7085b21280 1 -- 172.21.15.38:0/1717830250 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- osd_op(unknown.0.0:11 15.1d 15:bec1251c:::200.00000007:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e147) -- 0x5638317f2000 con 0x5638317d9200 2024-09-21T03:45:11.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.164+0000 7f7084060640 1 --2- 172.21.15.38:0/1717830250 >> [v2:172.21.15.195:6810/2080960180,v1:172.21.15.195:6811/2080960180] conn(0x5638317d8480 0x5638317e0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:11.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.164+0000 7f708385f640 1 --2- 172.21.15.38:0/1717830250 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x5638317d8900 0x5638317e0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:11.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.164+0000 7f7084060640 1 --2- 172.21.15.38:0/1717830250 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x5638317d9200 0x5638317e1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:11.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.164+0000 7f708305e640 1 --2- 172.21.15.38:0/1717830250 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x5638317d8d80 0x5638317e1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:11.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.164+0000 7f708305e640 1 --2- 172.21.15.38:0/1717830250 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x5638317d8d80 0x5638317e1080 crc :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=3686f263d10bf823 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:11.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.164+0000 7f7084060640 1 --2- 172.21.15.38:0/1717830250 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x5638317d9200 0x5638317e1600 crc :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=d64a51bdf9534738 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:11.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.164+0000 7f7084060640 1 --2- 172.21.15.38:0/1717830250 >> [v2:172.21.15.195:6810/2080960180,v1:172.21.15.195:6811/2080960180] conn(0x5638317d8480 0x5638317e0580 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=16ada120e7a4cf74 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:11.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.165+0000 7f708385f640 1 --2- 172.21.15.38:0/1717830250 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x5638317d8900 0x5638317e0b00 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=af333714da006bf9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:11.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.165+0000 7f708305e640 1 -- 172.21.15.38:0/1717830250 <== osd.3 v2:172.21.15.38:6808/1286810294 5 ==== osd_op_reply(6 200.00000002 [delete] v147'2 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x563831494500 con 0x5638317d8000 2024-09-21T03:45:11.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.167+0000 7f708385f640 1 -- 172.21.15.38:0/1717830250 <== osd.7 v2:172.21.15.195:6818/1606106342 1 ==== osd_op_reply(8 200.00000004 [delete] v147'7 uv6 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x563831494000 con 0x5638317d8900 2024-09-21T03:45:11.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.168+0000 7f708305e640 1 -- 172.21.15.38:0/1717830250 <== osd.1 v2:172.21.15.38:6816/1491818732 1 ==== osd_op_reply(9 200.00000005 [delete] v147'2 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x563831494500 con 0x5638317d8d80 2024-09-21T03:45:11.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.168+0000 7f7084060640 1 -- 172.21.15.38:0/1717830250 <== osd.0 v2:172.21.15.38:6800/2381639433 1 ==== osd_op_reply(11 200.00000007 [delete] v147'3 uv2 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x563831494780 con 0x5638317d9200 2024-09-21T03:45:11.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.168+0000 7f7084060640 1 -- 172.21.15.38:0/1717830250 <== osd.5 v2:172.21.15.195:6810/2080960180 1 ==== osd_op_reply(10 200.00000006 [delete] v147'2 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x563831494780 con 0x5638317d8480 2024-09-21T03:45:11.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.168+0000 7f7084060640 1 -- 172.21.15.38:0/1717830250 <== osd.5 v2:172.21.15.195:6810/2080960180 2 ==== osd_op_reply(7 200.00000003 [delete] v147'4 uv3 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x563831494780 con 0x5638317d8480 2024-09-21T03:45:11.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.168+0000 7f707904a640 1 -- 172.21.15.38:0/1717830250 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- osd_op(unknown.0.0:12 15.1b 15:ddb4e4d7:::200.00000002:head [write 0~48 in=48b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e147) -- 0x563832147800 con 0x5638317d8000 2024-09-21T03:45:11.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.169+0000 7f708305e640 1 -- 172.21.15.38:0/1717830250 <== osd.3 v2:172.21.15.38:6808/1286810294 6 ==== osd_op_reply(12 200.00000002 [write 0~48] v147'3 uv3 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x563831494500 con 0x5638317d8000 2024-09-21T03:45:11.175 INFO:teuthology.orchestra.run.smithi038.stdout:done 2024-09-21T03:45:11.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.169+0000 7f7085b21280 1 -- 172.21.15.38:0/1717830250 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x5638317d9200 msgr2=0x5638317e1600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:11.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.169+0000 7f7085b21280 1 --2- 172.21.15.38:0/1717830250 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x5638317d9200 0x5638317e1600 crc :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:11.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.169+0000 7f7085b21280 1 -- 172.21.15.38:0/1717830250 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x5638317d8d80 msgr2=0x5638317e1080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:11.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.169+0000 7f7085b21280 1 --2- 172.21.15.38:0/1717830250 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x5638317d8d80 0x5638317e1080 crc :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:11.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.169+0000 7f7085b21280 1 -- 172.21.15.38:0/1717830250 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x563831407a80 msgr2=0x563832131700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:11.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.169+0000 7f7085b21280 1 --2- 172.21.15.38:0/1717830250 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x563831407a80 0x563832131700 crc :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:11.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.169+0000 7f7085b21280 1 -- 172.21.15.38:0/1717830250 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x5638317d8000 msgr2=0x5638317e0000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:11.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.169+0000 7f7085b21280 1 --2- 172.21.15.38:0/1717830250 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x5638317d8000 0x5638317e0000 crc :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:11.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.169+0000 7f708305e640 1 -- 172.21.15.38:0/1717830250 reap_dead start 2024-09-21T03:45:11.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.169+0000 7f708305e640 1 -- 172.21.15.38:0/1717830250 reap_dead start 2024-09-21T03:45:11.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.169+0000 7f7085b21280 1 -- 172.21.15.38:0/1717830250 >> [v2:172.21.15.195:6810/2080960180,v1:172.21.15.195:6811/2080960180] conn(0x5638317d8480 msgr2=0x5638317e0580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:11.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.170+0000 7f7085b21280 1 --2- 172.21.15.38:0/1717830250 >> [v2:172.21.15.195:6810/2080960180,v1:172.21.15.195:6811/2080960180] conn(0x5638317d8480 0x5638317e0580 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:11.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.170+0000 7f7085b21280 1 -- 172.21.15.38:0/1717830250 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x5638317d8900 msgr2=0x5638317e0b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:11.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.170+0000 7f7085b21280 1 --2- 172.21.15.38:0/1717830250 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x5638317d8900 0x5638317e0b00 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:11.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.170+0000 7f7085b21280 1 -- 172.21.15.38:0/1717830250 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563831406880 msgr2=0x563832130680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:11.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.170+0000 7f7085b21280 1 --2- 172.21.15.38:0/1717830250 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563831406880 0x563832130680 secure :-1 s=READY pgs=2407 cs=0 l=1 rev1=1 crypto rx=0x5638317aa6f0 tx=0x5638322459e0 comp rx=0 tx=0).stop 2024-09-21T03:45:11.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.170+0000 7f7085b21280 1 -- 172.21.15.38:0/1717830250 shutdown_connections 2024-09-21T03:45:11.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.170+0000 7f7085b21280 1 --2- 172.21.15.38:0/1717830250 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x5638317d8900 0x5638317e0b00 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:11.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.170+0000 7f7085b21280 1 --2- 172.21.15.38:0/1717830250 >> [v2:172.21.15.195:6810/2080960180,v1:172.21.15.195:6811/2080960180] conn(0x5638317d8480 0x5638317e0580 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:11.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.170+0000 7f7085b21280 1 --2- 172.21.15.38:0/1717830250 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563831406880 0x563832130680 unknown :-1 s=CLOSED pgs=2407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:11.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.170+0000 7f7085b21280 1 -- 172.21.15.38:0/1717830250 >> 172.21.15.38:0/1717830250 conn(0x563831404000 msgr2=0x56383175a800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:11.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.170+0000 7f7085b21280 1 -- 172.21.15.38:0/1717830250 shutdown_connections 2024-09-21T03:45:11.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.170+0000 7f7085b21280 1 -- 172.21.15.38:0/1717830250 wait complete. 2024-09-21T03:45:11.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.170+0000 7f7085b21280 1 -- 172.21.15.38:0/3022837076 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x563831407180 msgr2=0x563832130100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:11.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.170+0000 7f7085b21280 1 --2- 172.21.15.38:0/3022837076 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x563831407180 0x563832130100 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x563832185d40 tx=0x5638321429c0 comp rx=0 tx=0).stop 2024-09-21T03:45:11.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.170+0000 7f7085b21280 1 -- 172.21.15.38:0/3022837076 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563831404480 msgr2=0x56383212fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:11.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.170+0000 7f7085b21280 1 --2- 172.21.15.38:0/3022837076 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563831404480 0x56383212fb80 secure :-1 s=READY pgs=2406 cs=0 l=1 rev1=1 crypto rx=0x563832239e30 tx=0x563832244f60 comp rx=0 tx=0).stop 2024-09-21T03:45:11.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.170+0000 7f7085b21280 1 -- 172.21.15.38:0/3022837076 shutdown_connections 2024-09-21T03:45:11.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.170+0000 7f7085b21280 1 --2- 172.21.15.38:0/3022837076 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563831404d80 0x56383212f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:11.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.170+0000 7f7085b21280 1 --2- 172.21.15.38:0/3022837076 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x563831407180 0x563832130100 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:11.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.170+0000 7f7085b21280 1 --2- 172.21.15.38:0/3022837076 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x563831404900 0x56383212f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:11.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.170+0000 7f7085b21280 1 --2- 172.21.15.38:0/3022837076 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563831404480 0x56383212fb80 unknown :-1 s=CLOSED pgs=2406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:11.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.170+0000 7f7085b21280 1 -- 172.21.15.38:0/3022837076 >> 172.21.15.38:0/3022837076 conn(0x563831405200 msgr2=0x563831479800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:11.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.170+0000 7f7085b21280 1 -- 172.21.15.38:0/3022837076 shutdown_connections 2024-09-21T03:45:11.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.170+0000 7f7085b21280 1 -- 172.21.15.38:0/3022837076 wait complete. 2024-09-21T03:45:11.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.171+0000 7f7085b21280 1 -- 172.21.15.38:0/2762339813 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563831405680 msgr2=0x56383212eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:11.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.171+0000 7f7085b21280 1 --2- 172.21.15.38:0/2762339813 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563831405680 0x56383212eb00 secure :-1 s=READY pgs=2404 cs=0 l=1 rev1=1 crypto rx=0x563832185ce0 tx=0x563832142180 comp rx=0 tx=0).stop 2024-09-21T03:45:11.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.171+0000 7f7085b21280 1 -- 172.21.15.38:0/2762339813 shutdown_connections 2024-09-21T03:45:11.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.171+0000 7f7085b21280 1 --2- 172.21.15.38:0/2762339813 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x563831406400 0x56383212e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:11.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.171+0000 7f7085b21280 1 --2- 172.21.15.38:0/2762339813 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563831405b00 0x56383212e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:11.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.171+0000 7f7085b21280 1 --2- 172.21.15.38:0/2762339813 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563831405680 0x56383212eb00 unknown :-1 s=CLOSED pgs=2404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:11.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.171+0000 7f7085b21280 1 -- 172.21.15.38:0/2762339813 >> 172.21.15.38:0/2762339813 conn(0x563831405f80 msgr2=0x563831479000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:11.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.171+0000 7f7085b21280 1 -- 172.21.15.38:0/2762339813 shutdown_connections 2024-09-21T03:45:11.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.171+0000 7f7085b21280 1 -- 172.21.15.38:0/2762339813 wait complete. 2024-09-21T03:45:11.182 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-journal-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '--rank', 'cephfs:0', 'journal', 'reset', '--yes-i-really-really-mean-it'] in time 0:00:00.091014, result: old journal was 4226629~2030 new journal start will be 8388608 (4159949 bytes past old end) writing journal head writing EResetJournal entry done 2024-09-21T03:45:11.183 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-21T03:45:11.302 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.300+0000 7f3400816640 1 -- 172.21.15.38:0/3076917669 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f33fc15ee80 msgr2=0x7f33fc0d0170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:11.302 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.300+0000 7f3400816640 1 --2- 172.21.15.38:0/3076917669 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f33fc15ee80 0x7f33fc0d0170 secure :-1 s=READY pgs=2408 cs=0 l=1 rev1=1 crypto rx=0x7f33ec005fd0 tx=0x7f33ec02a970 comp rx=0 tx=0).stop 2024-09-21T03:45:11.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.300+0000 7f3400816640 1 -- 172.21.15.38:0/3076917669 shutdown_connections 2024-09-21T03:45:11.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.300+0000 7f3400816640 1 --2- 172.21.15.38:0/3076917669 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f33fc0ceb90 0x7f33fc0cefa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:11.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.300+0000 7f3400816640 1 --2- 172.21.15.38:0/3076917669 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f33fc15ee80 0x7f33fc0d0170 unknown :-1 s=CLOSED pgs=2408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:11.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.300+0000 7f3400816640 1 -- 172.21.15.38:0/3076917669 >> 172.21.15.38:0/3076917669 conn(0x7f33fc09d830 msgr2=0x7f33fc0b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:11.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.300+0000 7f3400816640 1 -- 172.21.15.38:0/3076917669 shutdown_connections 2024-09-21T03:45:11.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.300+0000 7f3400816640 1 -- 172.21.15.38:0/3076917669 wait complete. 2024-09-21T03:45:11.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.302+0000 7f3400816640 1 Processor -- start 2024-09-21T03:45:11.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.302+0000 7f3400816640 1 -- start start 2024-09-21T03:45:11.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.303+0000 7f3400816640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f33fc0b9ad0 0x7f33fc06cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:11.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.303+0000 7f3400816640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f33fc0ceb90 0x7f33fc158950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:11.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.303+0000 7f3400816640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f33fc15ee80 0x7f33fc1f8f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:11.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.303+0000 7f3400816640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f33fc0b6700 con 0x7f33fc0ceb90 2024-09-21T03:45:11.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.303+0000 7f3400816640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f33fc0b6580 con 0x7f33fc0b9ad0 2024-09-21T03:45:11.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.303+0000 7f3400816640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f33fc0b6880 con 0x7f33fc15ee80 2024-09-21T03:45:11.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.304+0000 7f33f9d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f33fc0ceb90 0x7f33fc158950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:11.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.304+0000 7f33f9d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f33fc0ceb90 0x7f33fc158950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43736/0 (socket says 172.21.15.38:43736) 2024-09-21T03:45:11.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.304+0000 7f33f9d9b640 1 -- 172.21.15.38:0/4144963285 learned_addr learned my addr 172.21.15.38:0/4144963285 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:45:11.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.304+0000 7f33fa59c640 1 --2- 172.21.15.38:0/4144963285 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f33fc0b9ad0 0x7f33fc06cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:11.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.304+0000 7f33fad9d640 1 --2- 172.21.15.38:0/4144963285 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f33fc15ee80 0x7f33fc1f8f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:11.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.304+0000 7f33f9d9b640 1 -- 172.21.15.38:0/4144963285 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f33fc15ee80 msgr2=0x7f33fc1f8f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:11.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.304+0000 7f33f9d9b640 1 --2- 172.21.15.38:0/4144963285 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f33fc15ee80 0x7f33fc1f8f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:11.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.304+0000 7f33f9d9b640 1 -- 172.21.15.38:0/4144963285 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f33fc0b9ad0 msgr2=0x7f33fc06cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:11.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.304+0000 7f33f9d9b640 1 --2- 172.21.15.38:0/4144963285 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f33fc0b9ad0 0x7f33fc06cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:11.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.304+0000 7f33f9d9b640 1 -- 172.21.15.38:0/4144963285 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33fc06d6c0 con 0x7f33fc0ceb90 2024-09-21T03:45:11.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.304+0000 7f33fad9d640 1 --2- 172.21.15.38:0/4144963285 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f33fc15ee80 0x7f33fc1f8f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:45:11.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.304+0000 7f33f9d9b640 1 --2- 172.21.15.38:0/4144963285 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f33fc0ceb90 0x7f33fc158950 secure :-1 s=READY pgs=2409 cs=0 l=1 rev1=1 crypto rx=0x7f33f000b410 tx=0x7f33f0009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:11.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.304+0000 7f33eaffd640 1 -- 172.21.15.38:0/4144963285 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33f0015030 con 0x7f33fc0ceb90 2024-09-21T03:45:11.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.304+0000 7f33eaffd640 1 -- 172.21.15.38:0/4144963285 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f33f0005a70 con 0x7f33fc0ceb90 2024-09-21T03:45:11.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.304+0000 7f33eaffd640 1 -- 172.21.15.38:0/4144963285 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33f0006020 con 0x7f33fc0ceb90 2024-09-21T03:45:11.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.304+0000 7f3400816640 1 -- 172.21.15.38:0/4144963285 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33fc06d9a0 con 0x7f33fc0ceb90 2024-09-21T03:45:11.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.304+0000 7f3400816640 1 -- 172.21.15.38:0/4144963285 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33fc06df60 con 0x7f33fc0ceb90 2024-09-21T03:45:11.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.305+0000 7f3400816640 1 -- 172.21.15.38:0/4144963285 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33fc06e760 con 0x7f33fc0ceb90 2024-09-21T03:45:11.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.306+0000 7f33eaffd640 1 -- 172.21.15.38:0/4144963285 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f33f0018070 con 0x7f33fc0ceb90 2024-09-21T03:45:11.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.307+0000 7f33eaffd640 1 --2- 172.21.15.38:0/4144963285 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f33dc078b90 0x7f33dc07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:11.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.307+0000 7f33eaffd640 1 -- 172.21.15.38:0/4144963285 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 6187+0+0 (secure 0 0 0) 0x7f33f009a990 con 0x7f33fc0ceb90 2024-09-21T03:45:11.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.307+0000 7f33fa59c640 1 --2- 172.21.15.38:0/4144963285 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f33dc078b90 0x7f33dc07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:11.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.307+0000 7f33fa59c640 1 --2- 172.21.15.38:0/4144963285 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f33dc078b90 0x7f33dc07b0a0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f33ec035000 tx=0x7f33ec03c040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:11.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.311+0000 7f33eaffd640 1 -- 172.21.15.38:0/4144963285 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f33f0062be0 con 0x7f33fc0ceb90 2024-09-21T03:45:11.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:11.538+0000 7f3400816640 1 -- 172.21.15.38:0/4144963285 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f33fc0b6d20 con 0x7f33fc0ceb90 2024-09-21T03:45:12.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:12.085+0000 7f33eaffd640 1 -- 172.21.15.38:0/4144963285 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v345) ==== 162+0+0 (secure 0 0 0) 0x7f33f00624e0 con 0x7f33fc0ceb90 2024-09-21T03:45:12.087 INFO:teuthology.orchestra.run.smithi038.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-21T03:45:12.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:12.088+0000 7f3400816640 1 -- 172.21.15.38:0/4144963285 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f33dc078b90 msgr2=0x7f33dc07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:12.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:12.088+0000 7f3400816640 1 --2- 172.21.15.38:0/4144963285 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f33dc078b90 0x7f33dc07b0a0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f33ec035000 tx=0x7f33ec03c040 comp rx=0 tx=0).stop 2024-09-21T03:45:12.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:12.088+0000 7f3400816640 1 -- 172.21.15.38:0/4144963285 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f33fc0ceb90 msgr2=0x7f33fc158950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:12.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:12.088+0000 7f3400816640 1 --2- 172.21.15.38:0/4144963285 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f33fc0ceb90 0x7f33fc158950 secure :-1 s=READY pgs=2409 cs=0 l=1 rev1=1 crypto rx=0x7f33f000b410 tx=0x7f33f0009f90 comp rx=0 tx=0).stop 2024-09-21T03:45:12.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:12.088+0000 7f3400816640 1 -- 172.21.15.38:0/4144963285 shutdown_connections 2024-09-21T03:45:12.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:12.088+0000 7f3400816640 1 --2- 172.21.15.38:0/4144963285 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f33dc078b90 0x7f33dc07b0a0 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:12.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:12.088+0000 7f3400816640 1 --2- 172.21.15.38:0/4144963285 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f33fc15ee80 0x7f33fc1f8f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:12.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:12.088+0000 7f3400816640 1 --2- 172.21.15.38:0/4144963285 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f33fc0ceb90 0x7f33fc158950 unknown :-1 s=CLOSED pgs=2409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:12.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:12.088+0000 7f3400816640 1 --2- 172.21.15.38:0/4144963285 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f33fc0b9ad0 0x7f33fc06cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:12.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:12.088+0000 7f3400816640 1 -- 172.21.15.38:0/4144963285 >> 172.21.15.38:0/4144963285 conn(0x7f33fc09d830 msgr2=0x7f33fc1618e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:12.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:12.088+0000 7f3400816640 1 -- 172.21.15.38:0/4144963285 shutdown_connections 2024-09-21T03:45:12.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:12.088+0000 7f3400816640 1 -- 172.21.15.38:0/4144963285 wait complete. 2024-09-21T03:45:12.105 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:45:12.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:12.224+0000 7f2a7f8d3640 1 -- 172.21.15.38:0/1939531210 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2a780d0980 msgr2=0x7f2a780d0d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:12.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:12.224+0000 7f2a7f8d3640 1 --2- 172.21.15.38:0/1939531210 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2a780d0980 0x7f2a780d0d90 secure :-1 s=READY pgs=2410 cs=0 l=1 rev1=1 crypto rx=0x7f2a68005fd0 tx=0x7f2a6802aa90 comp rx=0 tx=0).stop 2024-09-21T03:45:12.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:12.224+0000 7f2a7f8d3640 1 -- 172.21.15.38:0/1939531210 shutdown_connections 2024-09-21T03:45:12.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:12.224+0000 7f2a7f8d3640 1 --2- 172.21.15.38:0/1939531210 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2a780d0980 0x7f2a780d0d90 unknown :-1 s=CLOSED pgs=2410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:12.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:12.224+0000 7f2a7f8d3640 1 --2- 172.21.15.38:0/1939531210 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2a780d01f0 0x7f2a7815daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:12.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:12.224+0000 7f2a7f8d3640 1 -- 172.21.15.38:0/1939531210 >> 172.21.15.38:0/1939531210 conn(0x7f2a78072990 msgr2=0x7f2a78072da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:12.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:12.224+0000 7f2a7f8d3640 1 -- 172.21.15.38:0/1939531210 shutdown_connections 2024-09-21T03:45:12.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:12.224+0000 7f2a7f8d3640 1 -- 172.21.15.38:0/1939531210 wait complete. 2024-09-21T03:45:12.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:12.226+0000 7f2a7f8d3640 1 Processor -- start 2024-09-21T03:45:12.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:12.226+0000 7f2a7f8d3640 1 -- start start 2024-09-21T03:45:12.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:12.227+0000 7f2a7f8d3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2a780d01f0 0x7f2a78146c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:12.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:12.227+0000 7f2a7f8d3640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2a780d0980 0x7f2a781471a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:12.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:12.227+0000 7f2a7f8d3640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2a7815a6f0 0x7f2a78145ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:12.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:12.227+0000 7f2a7f8d3640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f2a780cd340 con 0x7f2a780d01f0 2024-09-21T03:45:12.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:12.227+0000 7f2a7f8d3640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f2a780cd1c0 con 0x7f2a7815a6f0 2024-09-21T03:45:12.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:12.227+0000 7f2a7f8d3640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f2a780cd4c0 con 0x7f2a780d0980 2024-09-21T03:45:12.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:12.227+0000 7f2a7d66f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2a780d01f0 0x7f2a78146c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:12.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:12.228+0000 7f2a7d66f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2a780d01f0 0x7f2a78146c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43758/0 (socket says 172.21.15.38:43758) 2024-09-21T03:45:12.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:12.228+0000 7f2a7d66f640 1 -- 172.21.15.38:0/2852206197 learned_addr learned my addr 172.21.15.38:0/2852206197 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:45:12.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:12.228+0000 7f2a7de70640 1 --2- 172.21.15.38:0/2852206197 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2a7815a6f0 0x7f2a78145ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:12.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:12.228+0000 7f2a7ce6e640 1 --2- 172.21.15.38:0/2852206197 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2a780d0980 0x7f2a781471a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:12.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:12.228+0000 7f2a7d66f640 1 -- 172.21.15.38:0/2852206197 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2a780d0980 msgr2=0x7f2a781471a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:12.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:12.228+0000 7f2a7d66f640 1 --2- 172.21.15.38:0/2852206197 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2a780d0980 0x7f2a781471a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:12.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:12.228+0000 7f2a7d66f640 1 -- 172.21.15.38:0/2852206197 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2a7815a6f0 msgr2=0x7f2a78145ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:12.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:12.228+0000 7f2a7d66f640 1 --2- 172.21.15.38:0/2852206197 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2a7815a6f0 0x7f2a78145ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:12.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:12.228+0000 7f2a7d66f640 1 -- 172.21.15.38:0/2852206197 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a780c8630 con 0x7f2a780d01f0 2024-09-21T03:45:12.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:12.228+0000 7f2a7d66f640 1 --2- 172.21.15.38:0/2852206197 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2a780d01f0 0x7f2a78146c60 secure :-1 s=READY pgs=2411 cs=0 l=1 rev1=1 crypto rx=0x7f2a7000d740 tx=0x7f2a7000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:12.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:12.228+0000 7f2a75ffb640 1 -- 172.21.15.38:0/2852206197 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a70017070 con 0x7f2a780d01f0 2024-09-21T03:45:12.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:12.228+0000 7f2a75ffb640 1 -- 172.21.15.38:0/2852206197 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2a70009d20 con 0x7f2a780d01f0 2024-09-21T03:45:12.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:12.228+0000 7f2a7f8d3640 1 -- 172.21.15.38:0/2852206197 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a780c7870 con 0x7f2a780d01f0 2024-09-21T03:45:12.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:12.228+0000 7f2a75ffb640 1 -- 172.21.15.38:0/2852206197 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a700106c0 con 0x7f2a780d01f0 2024-09-21T03:45:12.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:12.229+0000 7f2a7f8d3640 1 -- 172.21.15.38:0/2852206197 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a780c7e30 con 0x7f2a780d01f0 2024-09-21T03:45:12.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:12.229+0000 7f2a7f8d3640 1 -- 172.21.15.38:0/2852206197 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a7814d040 con 0x7f2a780d01f0 2024-09-21T03:45:12.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:12.230+0000 7f2a75ffb640 1 -- 172.21.15.38:0/2852206197 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f2a7001a070 con 0x7f2a780d01f0 2024-09-21T03:45:12.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:12.231+0000 7f2a75ffb640 1 --2- 172.21.15.38:0/2852206197 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2a64078b90 0x7f2a6407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:12.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:12.231+0000 7f2a75ffb640 1 -- 172.21.15.38:0/2852206197 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 6187+0+0 (secure 0 0 0) 0x7f2a7009b000 con 0x7f2a780d01f0 2024-09-21T03:45:12.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:12.231+0000 7f2a7ce6e640 1 --2- 172.21.15.38:0/2852206197 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2a64078b90 0x7f2a6407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:12.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:12.232+0000 7f2a7ce6e640 1 --2- 172.21.15.38:0/2852206197 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2a64078b90 0x7f2a6407b0a0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f2a7814ccb0 tx=0x7f2a6c009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:12.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:12.235+0000 7f2a75ffb640 1 -- 172.21.15.38:0/2852206197 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f2a70062cb0 con 0x7f2a780d01f0 2024-09-21T03:45:12.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:12.463+0000 7f2a7f8d3640 1 -- 172.21.15.38:0/2852206197 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2a7815daf0 con 0x7f2a780d01f0 2024-09-21T03:45:12.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:12.464+0000 7f2a75ffb640 1 -- 172.21.15.38:0/2852206197 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 346 v346) ==== 95+0+5232 (secure 0 0 0) 0x7f2a700676e0 con 0x7f2a780d01f0 2024-09-21T03:45:12.466 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:45:12.466 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":346,"btime":"2024-09-21T03:45:12:088392+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8399,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/3606957623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":3606957623},{"type":"v1","addr":"172.21.15.38:6835","nonce":3606957623}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":338},{"gid":8405,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/456651486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":456651486},{"type":"v1","addr":"172.21.15.195:6837","nonce":456651486}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":338},{"gid":9604,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/2649311840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":2649311840},{"type":"v1","addr":"172.21.15.195:6835","nonce":2649311840}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":338}],"filesystems":[{"mdsmap":{"epoch":346,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:44:56.710446+0000","modified":"2024-09-21T03:45:12.088368+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":147,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10935},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10935":{"gid":10935,"name":"c","rank":0,"incarnation":346,"state":"up:replay","state_seq":1,"addr":"172.21.15.38:6833/3839260519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":3839260519},{"type":"v1","addr":"172.21.15.38:6833","nonce":3839260519}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":7}]} 2024-09-21T03:45:12.466 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 346 2024-09-21T03:45:12.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:12.467+0000 7f2a7f8d3640 1 -- 172.21.15.38:0/2852206197 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2a64078b90 msgr2=0x7f2a6407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:12.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:12.467+0000 7f2a7f8d3640 1 --2- 172.21.15.38:0/2852206197 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2a64078b90 0x7f2a6407b0a0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f2a7814ccb0 tx=0x7f2a6c009040 comp rx=0 tx=0).stop 2024-09-21T03:45:12.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:12.467+0000 7f2a7f8d3640 1 -- 172.21.15.38:0/2852206197 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2a780d01f0 msgr2=0x7f2a78146c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:12.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:12.467+0000 7f2a7f8d3640 1 --2- 172.21.15.38:0/2852206197 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2a780d01f0 0x7f2a78146c60 secure :-1 s=READY pgs=2411 cs=0 l=1 rev1=1 crypto rx=0x7f2a7000d740 tx=0x7f2a7000dc00 comp rx=0 tx=0).stop 2024-09-21T03:45:12.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:12.467+0000 7f2a7f8d3640 1 -- 172.21.15.38:0/2852206197 shutdown_connections 2024-09-21T03:45:12.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:12.467+0000 7f2a7f8d3640 1 --2- 172.21.15.38:0/2852206197 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2a7815a6f0 0x7f2a78145ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:12.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:12.467+0000 7f2a7f8d3640 1 --2- 172.21.15.38:0/2852206197 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2a64078b90 0x7f2a6407b0a0 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:12.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:12.467+0000 7f2a7f8d3640 1 --2- 172.21.15.38:0/2852206197 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2a780d0980 0x7f2a781471a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:12.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:12.467+0000 7f2a7f8d3640 1 --2- 172.21.15.38:0/2852206197 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2a780d01f0 0x7f2a78146c60 unknown :-1 s=CLOSED pgs=2411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:12.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:12.467+0000 7f2a7f8d3640 1 -- 172.21.15.38:0/2852206197 >> 172.21.15.38:0/2852206197 conn(0x7f2a78072990 msgr2=0x7f2a78068d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:12.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:12.467+0000 7f2a7f8d3640 1 -- 172.21.15.38:0/2852206197 shutdown_connections 2024-09-21T03:45:12.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:12.467+0000 7f2a7f8d3640 1 -- 172.21.15.38:0/2852206197 wait complete. 2024-09-21T03:45:12.883 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 346, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T03:44:56.710446+0000', 'modified': '2024-09-21T03:45:12.088368+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 147, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 10935}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_10935': {'gid': 10935, 'name': 'c', 'rank': 0, 'incarnation': 346, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.38:6833/3839260519', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.38:6832', 'nonce': 3839260519}, {'type': 'v1', 'addr': '172.21.15.38:6833', 'nonce': 3839260519}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-09-21T03:45:12.883 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_10935:up:replay 2024-09-21T03:45:13.884 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:45:14.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:14.014+0000 7fab0b827640 1 -- 172.21.15.38:0/964918407 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faaf0004800 msgr2=0x7faaf0004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:14.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:14.014+0000 7fab0b827640 1 --2- 172.21.15.38:0/964918407 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faaf0004800 0x7faaf0004c10 secure :-1 s=READY pgs=2413 cs=0 l=1 rev1=1 crypto rx=0x7faaf4009b20 tx=0x7faaf0005830 comp rx=0 tx=0).stop 2024-09-21T03:45:14.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:14.014+0000 7fab09dc4640 1 -- 172.21.15.38:0/964918407 reap_dead start 2024-09-21T03:45:14.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:14.014+0000 7fab0b827640 1 -- 172.21.15.38:0/964918407 shutdown_connections 2024-09-21T03:45:14.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:14.014+0000 7fab0b827640 1 -- 172.21.15.38:0/964918407 >> 172.21.15.38:0/964918407 conn(0x7fab040d4600 msgr2=0x7fab040d4a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:14.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:14.014+0000 7fab0b827640 1 -- 172.21.15.38:0/964918407 shutdown_connections 2024-09-21T03:45:14.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:14.014+0000 7fab0b827640 1 -- 172.21.15.38:0/964918407 wait complete. 2024-09-21T03:45:14.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:14.016+0000 7fab0b827640 1 Processor -- start 2024-09-21T03:45:14.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:14.016+0000 7fab0b827640 1 -- start start 2024-09-21T03:45:14.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:14.017+0000 7fab0b827640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faaf0004800 0x7fab040c7ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:14.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:14.017+0000 7fab0b827640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fab0414c820 0x7fab040cbe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:14.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:14.017+0000 7fab0b827640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fab040cc390 0x7fab040cceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:14.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:14.017+0000 7fab0b827640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fab040d3d00 con 0x7fab0414c820 2024-09-21T03:45:14.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:14.017+0000 7fab0b827640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fab040b5670 con 0x7faaf0004800 2024-09-21T03:45:14.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:14.017+0000 7fab0b827640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fab040d3e80 con 0x7fab040cc390 2024-09-21T03:45:14.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:14.017+0000 7fab08dc2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fab0414c820 0x7fab040cbe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:14.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:14.017+0000 7fab08dc2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fab0414c820 0x7fab040cbe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46298/0 (socket says 172.21.15.38:46298) 2024-09-21T03:45:14.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:14.017+0000 7fab08dc2640 1 -- 172.21.15.38:0/1076951593 learned_addr learned my addr 172.21.15.38:0/1076951593 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:45:14.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:14.017+0000 7fab095c3640 1 --2- 172.21.15.38:0/1076951593 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faaf0004800 0x7fab040c7ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:14.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:14.017+0000 7fab09dc4640 1 --2- 172.21.15.38:0/1076951593 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fab040cc390 0x7fab040cceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:14.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:14.018+0000 7fab08dc2640 1 -- 172.21.15.38:0/1076951593 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fab040cc390 msgr2=0x7fab040cceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:14.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:14.018+0000 7fab08dc2640 1 --2- 172.21.15.38:0/1076951593 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fab040cc390 0x7fab040cceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:14.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:14.018+0000 7fab08dc2640 1 -- 172.21.15.38:0/1076951593 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faaf0004800 msgr2=0x7fab040c7ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:14.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:14.018+0000 7fab08dc2640 1 --2- 172.21.15.38:0/1076951593 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faaf0004800 0x7fab040c7ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:14.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:14.018+0000 7fab08dc2640 1 -- 172.21.15.38:0/1076951593 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab041f9e80 con 0x7fab0414c820 2024-09-21T03:45:14.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:14.018+0000 7fab08dc2640 1 --2- 172.21.15.38:0/1076951593 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fab0414c820 0x7fab040cbe50 secure :-1 s=READY pgs=2414 cs=0 l=1 rev1=1 crypto rx=0x7faaf4009c40 tx=0x7faaf4033670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:14.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:14.018+0000 7faaf9ffb640 1 -- 172.21.15.38:0/1076951593 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaf4038040 con 0x7fab0414c820 2024-09-21T03:45:14.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:14.018+0000 7faaf9ffb640 1 -- 172.21.15.38:0/1076951593 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faaf402b7e0 con 0x7fab0414c820 2024-09-21T03:45:14.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:14.018+0000 7fab0b827640 1 -- 172.21.15.38:0/1076951593 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab040cb370 con 0x7fab0414c820 2024-09-21T03:45:14.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:14.018+0000 7faaf9ffb640 1 -- 172.21.15.38:0/1076951593 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaf4004280 con 0x7fab0414c820 2024-09-21T03:45:14.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:14.018+0000 7fab0b827640 1 -- 172.21.15.38:0/1076951593 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab040cb930 con 0x7fab0414c820 2024-09-21T03:45:14.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:14.019+0000 7fab0b827640 1 -- 172.21.15.38:0/1076951593 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab04162bc0 con 0x7fab0414c820 2024-09-21T03:45:14.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:14.020+0000 7faaf9ffb640 1 -- 172.21.15.38:0/1076951593 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7faaf4045020 con 0x7fab0414c820 2024-09-21T03:45:14.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:14.021+0000 7faaf9ffb640 1 --2- 172.21.15.38:0/1076951593 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7faae8078bc0 0x7faae807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:14.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:14.021+0000 7faaf9ffb640 1 -- 172.21.15.38:0/1076951593 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 6187+0+0 (secure 0 0 0) 0x7faaf40bb6b0 con 0x7fab0414c820 2024-09-21T03:45:14.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:14.021+0000 7fab095c3640 1 --2- 172.21.15.38:0/1076951593 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7faae8078bc0 0x7faae807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:14.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:14.021+0000 7fab095c3640 1 --2- 172.21.15.38:0/1076951593 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7faae8078bc0 0x7faae807b0d0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fab00009b30 tx=0x7fab00005f90 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:14.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:14.025+0000 7faaf9ffb640 1 -- 172.21.15.38:0/1076951593 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7faaf4083a60 con 0x7fab0414c820 2024-09-21T03:45:14.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:14.259+0000 7fab0b827640 1 -- 172.21.15.38:0/1076951593 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fab040b35d0 con 0x7fab0414c820 2024-09-21T03:45:14.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:14.260+0000 7faaf9ffb640 1 -- 172.21.15.38:0/1076951593 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 348 v348) ==== 95+0+5232 (secure 0 0 0) 0x7faaf40b9e20 con 0x7fab0414c820 2024-09-21T03:45:14.263 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:45:14.263 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":348,"btime":"2024-09-21T03:45:14:100045+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8399,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/3606957623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":3606957623},{"type":"v1","addr":"172.21.15.38:6835","nonce":3606957623}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":338},{"gid":8405,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/456651486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":456651486},{"type":"v1","addr":"172.21.15.195:6837","nonce":456651486}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":338},{"gid":9604,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/2649311840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":2649311840},{"type":"v1","addr":"172.21.15.195:6835","nonce":2649311840}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":338}],"filesystems":[{"mdsmap":{"epoch":348,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:44:56.710446+0000","modified":"2024-09-21T03:45:13.102302+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":147,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10935},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10935":{"gid":10935,"name":"c","rank":0,"incarnation":346,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.38:6833/3839260519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":3839260519},{"type":"v1","addr":"172.21.15.38:6833","nonce":3839260519}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":7}]} 2024-09-21T03:45:14.263 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 348 2024-09-21T03:45:14.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:14.262+0000 7fab0b827640 1 -- 172.21.15.38:0/1076951593 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7faae8078bc0 msgr2=0x7faae807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:14.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:14.262+0000 7fab0b827640 1 --2- 172.21.15.38:0/1076951593 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7faae8078bc0 0x7faae807b0d0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fab00009b30 tx=0x7fab00005f90 comp rx=0 tx=0).stop 2024-09-21T03:45:14.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:14.262+0000 7fab0b827640 1 -- 172.21.15.38:0/1076951593 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fab0414c820 msgr2=0x7fab040cbe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:14.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:14.262+0000 7fab0b827640 1 --2- 172.21.15.38:0/1076951593 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fab0414c820 0x7fab040cbe50 secure :-1 s=READY pgs=2414 cs=0 l=1 rev1=1 crypto rx=0x7faaf4009c40 tx=0x7faaf4033670 comp rx=0 tx=0).stop 2024-09-21T03:45:14.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:14.262+0000 7fab0b827640 1 -- 172.21.15.38:0/1076951593 shutdown_connections 2024-09-21T03:45:14.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:14.262+0000 7fab0b827640 1 --2- 172.21.15.38:0/1076951593 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7faae8078bc0 0x7faae807b0d0 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:14.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:14.262+0000 7fab0b827640 1 --2- 172.21.15.38:0/1076951593 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fab040cc390 0x7fab040cceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:14.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:14.262+0000 7fab0b827640 1 --2- 172.21.15.38:0/1076951593 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fab0414c820 0x7fab040cbe50 unknown :-1 s=CLOSED pgs=2414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:14.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:14.262+0000 7fab0b827640 1 --2- 172.21.15.38:0/1076951593 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faaf0004800 0x7fab040c7ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:14.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:14.262+0000 7fab0b827640 1 -- 172.21.15.38:0/1076951593 >> 172.21.15.38:0/1076951593 conn(0x7fab040d4600 msgr2=0x7fab040d4a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:14.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:14.263+0000 7fab0b827640 1 -- 172.21.15.38:0/1076951593 shutdown_connections 2024-09-21T03:45:14.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:14.263+0000 7fab0b827640 1 -- 172.21.15.38:0/1076951593 wait complete. 2024-09-21T03:45:14.678 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 348, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T03:44:56.710446+0000', 'modified': '2024-09-21T03:45:13.102302+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 147, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 10935}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_10935': {'gid': 10935, 'name': 'c', 'rank': 0, 'incarnation': 346, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.38:6833/3839260519', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.38:6832', 'nonce': 3839260519}, {'type': 'v1', 'addr': '172.21.15.38:6833', 'nonce': 3839260519}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-09-21T03:45:14.679 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_10935:up:rejoin 2024-09-21T03:45:15.679 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:45:15.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:15.810+0000 7f350d720640 1 -- 172.21.15.38:0/244613780 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f35080d3360 msgr2=0x7f35080d3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:15.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:15.810+0000 7f350d720640 1 --2- 172.21.15.38:0/244613780 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f35080d3360 0x7f35080d3770 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f3500005fd0 tx=0x7f350002a820 comp rx=0 tx=0).stop 2024-09-21T03:45:15.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:15.810+0000 7f350d720640 1 -- 172.21.15.38:0/244613780 shutdown_connections 2024-09-21T03:45:15.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:15.810+0000 7f350d720640 1 --2- 172.21.15.38:0/244613780 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f35080d3360 0x7f35080d3770 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:15.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:15.810+0000 7f350d720640 1 --2- 172.21.15.38:0/244613780 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f35080cea90 0x7f35080d00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:15.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:15.810+0000 7f350d720640 1 -- 172.21.15.38:0/244613780 >> 172.21.15.38:0/244613780 conn(0x7f35080d4620 msgr2=0x7f35080d4a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:15.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:15.810+0000 7f350d720640 1 -- 172.21.15.38:0/244613780 shutdown_connections 2024-09-21T03:45:15.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:15.810+0000 7f350d720640 1 -- 172.21.15.38:0/244613780 wait complete. 2024-09-21T03:45:15.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:15.812+0000 7f350d720640 1 Processor -- start 2024-09-21T03:45:15.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:15.812+0000 7f350d720640 1 -- start start 2024-09-21T03:45:15.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:15.813+0000 7f350d720640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f35080cea90 0x7f3508145410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:15.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:15.813+0000 7f350d720640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f35080d3360 0x7f35080ccf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:15.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:15.813+0000 7f350d720640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3508162790 0x7f35080cd4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:15.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:15.813+0000 7f350d720640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f35080b4120 con 0x7f35080cea90 2024-09-21T03:45:15.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:15.813+0000 7f350d720640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f35080b3fa0 con 0x7f3508162790 2024-09-21T03:45:15.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:15.813+0000 7f350d720640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f35080b42a0 con 0x7f35080d3360 2024-09-21T03:45:15.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:15.814+0000 7f3506ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f35080cea90 0x7f3508145410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:15.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:15.814+0000 7f3506ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f35080cea90 0x7f3508145410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46314/0 (socket says 172.21.15.38:46314) 2024-09-21T03:45:15.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:15.814+0000 7f3506ffd640 1 -- 172.21.15.38:0/2732068138 learned_addr learned my addr 172.21.15.38:0/2732068138 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:45:15.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:15.814+0000 7f35077fe640 1 --2- 172.21.15.38:0/2732068138 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3508162790 0x7f35080cd4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:15.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:15.814+0000 7f35067fc640 1 --2- 172.21.15.38:0/2732068138 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f35080d3360 0x7f35080ccf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:15.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:15.814+0000 7f3506ffd640 1 -- 172.21.15.38:0/2732068138 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f35080d3360 msgr2=0x7f35080ccf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:15.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:15.814+0000 7f3506ffd640 1 --2- 172.21.15.38:0/2732068138 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f35080d3360 0x7f35080ccf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:15.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:15.814+0000 7f3506ffd640 1 -- 172.21.15.38:0/2732068138 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3508162790 msgr2=0x7f35080cd4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:15.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:15.814+0000 7f3506ffd640 1 --2- 172.21.15.38:0/2732068138 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3508162790 0x7f35080cd4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:15.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:15.814+0000 7f3506ffd640 1 -- 172.21.15.38:0/2732068138 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35081550e0 con 0x7f35080cea90 2024-09-21T03:45:15.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:15.814+0000 7f3506ffd640 1 --2- 172.21.15.38:0/2732068138 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f35080cea90 0x7f3508145410 secure :-1 s=READY pgs=2415 cs=0 l=1 rev1=1 crypto rx=0x7f34f800ea20 tx=0x7f34f800eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:15.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:15.814+0000 7f34ef7fe640 1 -- 172.21.15.38:0/2732068138 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34f800ce40 con 0x7f35080cea90 2024-09-21T03:45:15.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:15.814+0000 7f350d720640 1 -- 172.21.15.38:0/2732068138 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3508147e90 con 0x7f35080cea90 2024-09-21T03:45:15.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:15.814+0000 7f34ef7fe640 1 -- 172.21.15.38:0/2732068138 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f34f8004020 con 0x7f35080cea90 2024-09-21T03:45:15.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:15.814+0000 7f34ef7fe640 1 -- 172.21.15.38:0/2732068138 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34f8004470 con 0x7f35080cea90 2024-09-21T03:45:15.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:15.814+0000 7f350d720640 1 -- 172.21.15.38:0/2732068138 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3508148450 con 0x7f35080cea90 2024-09-21T03:45:15.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:15.815+0000 7f350d720640 1 -- 172.21.15.38:0/2732068138 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35080b4750 con 0x7f35080cea90 2024-09-21T03:45:15.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:15.816+0000 7f34ef7fe640 1 -- 172.21.15.38:0/2732068138 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f34f8005030 con 0x7f35080cea90 2024-09-21T03:45:15.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:15.817+0000 7f34ef7fe640 1 --2- 172.21.15.38:0/2732068138 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f34f4078b90 0x7f34f407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:15.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:15.817+0000 7f34ef7fe640 1 -- 172.21.15.38:0/2732068138 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 6187+0+0 (secure 0 0 0) 0x7f34f809b750 con 0x7f35080cea90 2024-09-21T03:45:15.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:15.818+0000 7f35067fc640 1 --2- 172.21.15.38:0/2732068138 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f34f4078b90 0x7f34f407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:15.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:15.818+0000 7f35067fc640 1 --2- 172.21.15.38:0/2732068138 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f34f4078b90 0x7f34f407b0a0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f3508051030 tx=0x7f34fc009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:15.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:15.822+0000 7f34ef7fe640 1 -- 172.21.15.38:0/2732068138 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f35081550e0 con 0x7f35080cea90 2024-09-21T03:45:16.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.048+0000 7f350d720640 1 -- 172.21.15.38:0/2732068138 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f350806e9b0 con 0x7f35080cea90 2024-09-21T03:45:16.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.049+0000 7f34ef7fe640 1 -- 172.21.15.38:0/2732068138 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 349 v349) ==== 95+0+5241 (secure 0 0 0) 0x7f34f80639c0 con 0x7f35080cea90 2024-09-21T03:45:16.051 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:45:16.051 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":349,"btime":"2024-09-21T03:45:15:115320+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8399,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/3606957623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":3606957623},{"type":"v1","addr":"172.21.15.38:6835","nonce":3606957623}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":338},{"gid":8405,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/456651486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":456651486},{"type":"v1","addr":"172.21.15.195:6837","nonce":456651486}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":338},{"gid":9604,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/2649311840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":2649311840},{"type":"v1","addr":"172.21.15.195:6835","nonce":2649311840}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":338}],"filesystems":[{"mdsmap":{"epoch":349,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:44:56.710446+0000","modified":"2024-09-21T03:45:15.115308+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":147,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10935},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10935":{"gid":10935,"name":"c","rank":0,"incarnation":346,"state":"up:active","state_seq":4,"addr":"172.21.15.38:6833/3839260519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":3839260519},{"type":"v1","addr":"172.21.15.38:6833","nonce":3839260519}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10935,"qdb_cluster":[10935]},"id":7}]} 2024-09-21T03:45:16.051 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 349 2024-09-21T03:45:16.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.051+0000 7f350d720640 1 -- 172.21.15.38:0/2732068138 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f34f4078b90 msgr2=0x7f34f407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:16.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.051+0000 7f350d720640 1 --2- 172.21.15.38:0/2732068138 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f34f4078b90 0x7f34f407b0a0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f3508051030 tx=0x7f34fc009040 comp rx=0 tx=0).stop 2024-09-21T03:45:16.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.051+0000 7f350d720640 1 -- 172.21.15.38:0/2732068138 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f35080cea90 msgr2=0x7f3508145410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:16.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.051+0000 7f350d720640 1 --2- 172.21.15.38:0/2732068138 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f35080cea90 0x7f3508145410 secure :-1 s=READY pgs=2415 cs=0 l=1 rev1=1 crypto rx=0x7f34f800ea20 tx=0x7f34f800eee0 comp rx=0 tx=0).stop 2024-09-21T03:45:16.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.051+0000 7f350d720640 1 -- 172.21.15.38:0/2732068138 shutdown_connections 2024-09-21T03:45:16.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.051+0000 7f350d720640 1 --2- 172.21.15.38:0/2732068138 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3508162790 0x7f35080cd4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:16.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.051+0000 7f350d720640 1 --2- 172.21.15.38:0/2732068138 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f34f4078b90 0x7f34f407b0a0 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:16.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.051+0000 7f350d720640 1 --2- 172.21.15.38:0/2732068138 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f35080d3360 0x7f35080ccf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:16.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.051+0000 7f350d720640 1 --2- 172.21.15.38:0/2732068138 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f35080cea90 0x7f3508145410 unknown :-1 s=CLOSED pgs=2415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:16.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.051+0000 7f350d720640 1 -- 172.21.15.38:0/2732068138 >> 172.21.15.38:0/2732068138 conn(0x7f35080d4620 msgr2=0x7f35080d4a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:16.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.051+0000 7f350d720640 1 -- 172.21.15.38:0/2732068138 shutdown_connections 2024-09-21T03:45:16.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.051+0000 7f350d720640 1 -- 172.21.15.38:0/2732068138 wait complete. 2024-09-21T03:45:16.468 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 349, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T03:44:56.710446+0000', 'modified': '2024-09-21T03:45:15.115308+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 147, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 10935}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_10935': {'gid': 10935, 'name': 'c', 'rank': 0, 'incarnation': 346, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.38:6833/3839260519', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.38:6832', 'nonce': 3839260519}, {'type': 'v1', 'addr': '172.21.15.38:6833', 'nonce': 3839260519}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10935, 'qdb_cluster': [10935]} 2024-09-21T03:45:16.468 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-21T03:45:16.468 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-21T03:45:16.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.588+0000 7ffa1e96e640 1 -- 172.21.15.38:0/2720403988 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffa180ceb90 msgr2=0x7ffa180cefa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:16.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.588+0000 7ffa1e96e640 1 --2- 172.21.15.38:0/2720403988 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffa180ceb90 0x7ffa180cefa0 secure :-1 s=READY pgs=2417 cs=0 l=1 rev1=1 crypto rx=0x7ffa10005fd0 tx=0x7ffa1002a820 comp rx=0 tx=0).stop 2024-09-21T03:45:16.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.588+0000 7ffa1e96e640 1 -- 172.21.15.38:0/2720403988 shutdown_connections 2024-09-21T03:45:16.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.588+0000 7ffa1e96e640 1 --2- 172.21.15.38:0/2720403988 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffa180ceb90 0x7ffa180cefa0 unknown :-1 s=CLOSED pgs=2417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:16.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.588+0000 7ffa1e96e640 1 -- 172.21.15.38:0/2720403988 >> 172.21.15.38:0/2720403988 conn(0x7ffa1809d830 msgr2=0x7ffa180b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:16.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.588+0000 7ffa1e96e640 1 -- 172.21.15.38:0/2720403988 shutdown_connections 2024-09-21T03:45:16.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.588+0000 7ffa1e96e640 1 -- 172.21.15.38:0/2720403988 wait complete. 2024-09-21T03:45:16.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.590+0000 7ffa1e96e640 1 Processor -- start 2024-09-21T03:45:16.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.590+0000 7ffa1e96e640 1 -- start start 2024-09-21T03:45:16.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.592+0000 7ffa1e96e640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffa180ceb90 0x7ffa1806f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:16.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.592+0000 7ffa1e96e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffa180d0170 0x7ffa180cdf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:16.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.592+0000 7ffa1e96e640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffa1815ee80 0x7ffa1815b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:16.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.592+0000 7ffa1e96e640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ffa180b6790 con 0x7ffa180d0170 2024-09-21T03:45:16.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.592+0000 7ffa1e96e640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ffa180d14b0 con 0x7ffa180ceb90 2024-09-21T03:45:16.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.592+0000 7ffa1e96e640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7ffa180b6910 con 0x7ffa1815ee80 2024-09-21T03:45:16.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.592+0000 7ffa177fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffa180d0170 0x7ffa180cdf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:16.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.592+0000 7ffa177fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffa180d0170 0x7ffa180cdf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46340/0 (socket says 172.21.15.38:46340) 2024-09-21T03:45:16.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.592+0000 7ffa177fe640 1 -- 172.21.15.38:0/3358999554 learned_addr learned my addr 172.21.15.38:0/3358999554 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:45:16.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.592+0000 7ffa17fff640 1 --2- 172.21.15.38:0/3358999554 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffa180ceb90 0x7ffa1806f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:16.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.592+0000 7ffa1cf0b640 1 --2- 172.21.15.38:0/3358999554 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffa1815ee80 0x7ffa1815b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:16.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.592+0000 7ffa177fe640 1 -- 172.21.15.38:0/3358999554 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffa1815ee80 msgr2=0x7ffa1815b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:16.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.592+0000 7ffa177fe640 1 --2- 172.21.15.38:0/3358999554 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffa1815ee80 0x7ffa1815b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:16.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.592+0000 7ffa177fe640 1 -- 172.21.15.38:0/3358999554 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffa180ceb90 msgr2=0x7ffa1806f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:16.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.592+0000 7ffa177fe640 1 --2- 172.21.15.38:0/3358999554 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffa180ceb90 0x7ffa1806f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:16.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.592+0000 7ffa177fe640 1 -- 172.21.15.38:0/3358999554 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa180b19f0 con 0x7ffa180d0170 2024-09-21T03:45:16.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.592+0000 7ffa1cf0b640 1 --2- 172.21.15.38:0/3358999554 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffa1815ee80 0x7ffa1815b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:45:16.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.592+0000 7ffa177fe640 1 --2- 172.21.15.38:0/3358999554 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffa180d0170 0x7ffa180cdf00 secure :-1 s=READY pgs=2418 cs=0 l=1 rev1=1 crypto rx=0x7ffa0c00c440 tx=0x7ffa0c00c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:16.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.593+0000 7ffa14ff9640 1 -- 172.21.15.38:0/3358999554 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa0c017070 con 0x7ffa180d0170 2024-09-21T03:45:16.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.593+0000 7ffa14ff9640 1 -- 172.21.15.38:0/3358999554 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffa0c0053c0 con 0x7ffa180d0170 2024-09-21T03:45:16.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.593+0000 7ffa14ff9640 1 -- 172.21.15.38:0/3358999554 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa0c0038a0 con 0x7ffa180d0170 2024-09-21T03:45:16.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.593+0000 7ffa1e96e640 1 -- 172.21.15.38:0/3358999554 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa180b1c90 con 0x7ffa180d0170 2024-09-21T03:45:16.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.593+0000 7ffa1e96e640 1 -- 172.21.15.38:0/3358999554 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa180b2250 con 0x7ffa180d0170 2024-09-21T03:45:16.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.595+0000 7ffa14ff9640 1 -- 172.21.15.38:0/3358999554 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ffa0c01a070 con 0x7ffa180d0170 2024-09-21T03:45:16.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.595+0000 7ffa14ff9640 1 --2- 172.21.15.38:0/3358999554 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ffa04078b90 0x7ffa0407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:16.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.596+0000 7ffa14ff9640 1 -- 172.21.15.38:0/3358999554 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 6187+0+0 (secure 0 0 0) 0x7ffa0c09a1c0 con 0x7ffa180d0170 2024-09-21T03:45:16.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.596+0000 7ffa17fff640 1 --2- 172.21.15.38:0/3358999554 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ffa04078b90 0x7ffa0407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:16.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.596+0000 7ffa259ba740 1 Processor -- start 2024-09-21T03:45:16.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.596+0000 7ffa17fff640 1 --2- 172.21.15.38:0/3358999554 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ffa04078b90 0x7ffa0407b0a0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7ffa080013f0 tx=0x7ffa08009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:16.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.597+0000 7ffa259ba740 1 -- start start 2024-09-21T03:45:16.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.598+0000 7ffa259ba740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a915bb8270 0x55a915f89fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:16.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.598+0000 7ffa259ba740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a915b8f3b0 0x55a915f8a500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:16.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.598+0000 7ffa259ba740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a915b97590 0x55a915fa7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:16.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.598+0000 7ffa259ba740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55a915ed2820 con 0x55a915bb8270 2024-09-21T03:45:16.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.598+0000 7ffa259ba740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55a915ec1c50 con 0x55a915b97590 2024-09-21T03:45:16.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.598+0000 7ffa259ba740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55a915ec5a60 con 0x55a915b8f3b0 2024-09-21T03:45:16.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.598+0000 7ffa17fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a915bb8270 0x55a915f89fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:16.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.598+0000 7ffa17fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a915bb8270 0x55a915f89fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46342/0 (socket says 172.21.15.38:46342) 2024-09-21T03:45:16.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.598+0000 7ffa17fff640 1 -- 172.21.15.38:0/2597619885 learned_addr learned my addr 172.21.15.38:0/2597619885 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:45:16.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.598+0000 7ffa1cf0b640 1 --2- 172.21.15.38:0/2597619885 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a915b97590 0x55a915fa7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:16.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.598+0000 7ffa177fe640 1 --2- 172.21.15.38:0/2597619885 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a915b8f3b0 0x55a915f8a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:16.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.598+0000 7ffa17fff640 1 -- 172.21.15.38:0/2597619885 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a915b8f3b0 msgr2=0x55a915f8a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:16.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.598+0000 7ffa17fff640 1 --2- 172.21.15.38:0/2597619885 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a915b8f3b0 0x55a915f8a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:16.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.598+0000 7ffa17fff640 1 -- 172.21.15.38:0/2597619885 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a915b97590 msgr2=0x55a915fa7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:16.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.598+0000 7ffa17fff640 1 --2- 172.21.15.38:0/2597619885 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a915b97590 0x55a915fa7400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:16.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.598+0000 7ffa17fff640 1 -- 172.21.15.38:0/2597619885 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a915c23850 con 0x55a915bb8270 2024-09-21T03:45:16.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.598+0000 7ffa1cf0b640 1 --2- 172.21.15.38:0/2597619885 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a915b97590 0x55a915fa7400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:45:16.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.598+0000 7ffa17fff640 1 --2- 172.21.15.38:0/2597619885 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a915bb8270 0x55a915f89fc0 secure :-1 s=READY pgs=2419 cs=0 l=1 rev1=1 crypto rx=0x7ffa0800e670 tx=0x7ffa08011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e6c7526eb4e8bc9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:16.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.598+0000 7ff9fdffb640 1 -- 172.21.15.38:0/2597619885 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa08025070 con 0x55a915bb8270 2024-09-21T03:45:16.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.599+0000 7ff9fdffb640 1 -- 172.21.15.38:0/2597619885 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffa08015870 con 0x55a915bb8270 2024-09-21T03:45:16.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.599+0000 7ff9fdffb640 1 -- 172.21.15.38:0/2597619885 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa08015c60 con 0x55a915bb8270 2024-09-21T03:45:16.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.599+0000 7ffa259ba740 1 -- 172.21.15.38:0/2597619885 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a915bb8270 msgr2=0x55a915f89fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:16.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.599+0000 7ffa259ba740 1 --2- 172.21.15.38:0/2597619885 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a915bb8270 0x55a915f89fc0 secure :-1 s=READY pgs=2419 cs=0 l=1 rev1=1 crypto rx=0x7ffa0800e670 tx=0x7ffa08011000 comp rx=0 tx=0).stop 2024-09-21T03:45:16.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.599+0000 7ffa259ba740 1 -- 172.21.15.38:0/2597619885 shutdown_connections 2024-09-21T03:45:16.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.599+0000 7ffa259ba740 1 --2- 172.21.15.38:0/2597619885 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a915b97590 0x55a915fa7400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:16.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.599+0000 7ffa259ba740 1 --2- 172.21.15.38:0/2597619885 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a915b8f3b0 0x55a915f8a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:16.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.599+0000 7ffa259ba740 1 --2- 172.21.15.38:0/2597619885 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a915bb8270 0x55a915f89fc0 unknown :-1 s=CLOSED pgs=2419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:16.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.599+0000 7ffa259ba740 1 -- 172.21.15.38:0/2597619885 >> 172.21.15.38:0/2597619885 conn(0x55a915bac3c0 msgr2=0x55a915f60280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:16.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.599+0000 7ffa259ba740 1 -- 172.21.15.38:0/2597619885 shutdown_connections 2024-09-21T03:45:16.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.599+0000 7ffa259ba740 1 -- 172.21.15.38:0/2597619885 wait complete. 2024-09-21T03:45:16.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.600+0000 7ffa259ba740 1 -- start start 2024-09-21T03:45:16.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.601+0000 7ffa259ba740 1 Processor -- start 2024-09-21T03:45:16.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.602+0000 7ffa259ba740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a915b8f3b0 0x55a915fb3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:16.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.602+0000 7ffa259ba740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a915b97590 0x55a915f9bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:16.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.602+0000 7ffa259ba740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a915bb8270 0x55a915f83400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:16.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.602+0000 7ffa259ba740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55a915c32620 con 0x55a915b97590 2024-09-21T03:45:16.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.602+0000 7ffa259ba740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55a915e675f0 con 0x55a915bb8270 2024-09-21T03:45:16.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.602+0000 7ffa259ba740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55a915f8aa40 con 0x55a915b8f3b0 2024-09-21T03:45:16.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.602+0000 7ffa259ba740 10 client.0 _ll_register_callbacks cb 0x55a915e6b0a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:45:16.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.602+0000 7ffa177fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a915b97590 0x55a915f9bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:16.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.602+0000 7ffa177fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a915b97590 0x55a915f9bbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46354/0 (socket says 172.21.15.38:46354) 2024-09-21T03:45:16.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.602+0000 7ffa177fe640 1 -- 172.21.15.38:0/1218000437 learned_addr learned my addr 172.21.15.38:0/1218000437 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:45:16.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.602+0000 7ffa1cf0b640 1 --2- 172.21.15.38:0/1218000437 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a915bb8270 0x55a915f83400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:16.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.602+0000 7ffa17fff640 1 --2- 172.21.15.38:0/1218000437 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a915b8f3b0 0x55a915fb3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:16.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.602+0000 7ffa177fe640 1 -- 172.21.15.38:0/1218000437 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a915b8f3b0 msgr2=0x55a915fb3ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:16.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.602+0000 7ffa177fe640 1 --2- 172.21.15.38:0/1218000437 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a915b8f3b0 0x55a915fb3ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:16.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.602+0000 7ffa177fe640 1 -- 172.21.15.38:0/1218000437 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a915bb8270 msgr2=0x55a915f83400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:16.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.602+0000 7ffa177fe640 1 --2- 172.21.15.38:0/1218000437 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a915bb8270 0x55a915f83400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:16.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.602+0000 7ffa177fe640 1 -- 172.21.15.38:0/1218000437 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a915c9a6b0 con 0x55a915b97590 2024-09-21T03:45:16.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.602+0000 7ffa17fff640 1 --2- 172.21.15.38:0/1218000437 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a915b8f3b0 0x55a915fb3ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:45:16.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.603+0000 7ffa177fe640 1 --2- 172.21.15.38:0/1218000437 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a915b97590 0x55a915f9bbb0 secure :-1 s=READY pgs=2420 cs=0 l=1 rev1=1 crypto rx=0x7ffa0c098450 tx=0x7ffa0c05e2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab5e6b025a0ff332 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:16.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.603+0000 7ff9f6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-21T03:45:16.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.603+0000 7ff9f6ffd640 1 -- 172.21.15.38:0/1218000437 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa0c088170 con 0x55a915b97590 2024-09-21T03:45:16.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.603+0000 7ff9f6ffd640 1 -- 172.21.15.38:0/1218000437 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(147..147 src has 1..147) ==== 6187+0+0 (secure 0 0 0) 0x7ffa0c05d700 con 0x55a915b97590 2024-09-21T03:45:16.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.603+0000 7ffa259ba740 1 -- 172.21.15.38:0/1218000437 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa08015870 con 0x55a915b97590 2024-09-21T03:45:16.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.603+0000 7ff9f6ffd640 1 -- 172.21.15.38:0/1218000437 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=349) ==== 24+0+0 (secure 0 0 0) 0x7ffa0c0624b0 con 0x55a915b97590 2024-09-21T03:45:16.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.603+0000 7ffa259ba740 10 client.10983 fetch_fsmap learned FSMap version 349 2024-09-21T03:45:16.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.603+0000 7ffa259ba740 1 -- 172.21.15.38:0/1218000437 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=349,monmap=2+}) -- 0x55a915f9dfc0 con 0x55a915b97590 2024-09-21T03:45:16.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.603+0000 7ff9f6ffd640 1 -- 172.21.15.38:0/1218000437 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffa08015870 con 0x55a915b97590 2024-09-21T03:45:16.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.603+0000 7ff9f6ffd640 1 -- 172.21.15.38:0/1218000437 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 349) ==== 3502+0+0 (secure 0 0 0) 0x7ffa0c05d900 con 0x55a915b97590 2024-09-21T03:45:16.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.603+0000 7ffa259ba740 10 client.10983 fetch_fsmap finished waiting for FSMap version 349 2024-09-21T03:45:16.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.603+0000 7ffa259ba740 10 client.10983 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T03:45:16.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.603+0000 7ffa259ba740 20 client.10983 populate_metadata read hostname 'smithi038' 2024-09-21T03:45:16.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.603+0000 7ffa259ba740 1 --2- 172.21.15.38:0/1218000437 >> [v2:172.21.15.38:6832/3839260519,v1:172.21.15.38:6833/3839260519] conn(0x55a915b76e40 0x55a915fb76e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:16.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.603+0000 7ffa259ba740 4 client.10983 mds_command: new command op to 10935 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:45:16.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.603+0000 7ffa259ba740 1 -- 172.21.15.38:0/1218000437 --> [v2:172.21.15.38:6832/3839260519,v1:172.21.15.38:6833/3839260519] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a915ed4770 con 0x55a915b76e40 2024-09-21T03:45:16.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.604+0000 7ffa17fff640 1 --2- 172.21.15.38:0/1218000437 >> [v2:172.21.15.38:6832/3839260519,v1:172.21.15.38:6833/3839260519] conn(0x55a915b76e40 0x55a915fb76e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:16.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.604+0000 7ff9f6ffd640 10 client.10983 ms_handle_connect on v2:172.21.15.38:6832/3839260519 2024-09-21T03:45:16.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.604+0000 7ffa17fff640 1 --2- 172.21.15.38:0/1218000437 >> [v2:172.21.15.38:6832/3839260519,v1:172.21.15.38:6833/3839260519] conn(0x55a915b76e40 0x55a915fb76e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9cc17e34d38606a8 server_cookie=2568e2acfc50b6ae in_seq=0 out_seq=0 2024-09-21T03:45:16.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.605+0000 7ff9f6ffd640 1 -- 172.21.15.38:0/1218000437 <== mds.0 v2:172.21.15.38:6832/3839260519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a915ed4770 con 0x55a915b76e40 2024-09-21T03:45:16.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.605+0000 7ff9f6ffd640 10 client.10983 handle_command_reply: tid=0 2024-09-21T03:45:16.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.605+0000 7ffa259ba740 1 client.10983 shutdown 2024-09-21T03:45:16.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.605+0000 7ffa259ba740 1 -- 172.21.15.38:0/1218000437 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a915b97590 msgr2=0x55a915f9bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:16.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.605+0000 7ffa259ba740 1 --2- 172.21.15.38:0/1218000437 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a915b97590 0x55a915f9bbb0 secure :-1 s=READY pgs=2420 cs=0 l=1 rev1=1 crypto rx=0x7ffa0c098450 tx=0x7ffa0c05e2d0 comp rx=0 tx=0).stop 2024-09-21T03:45:16.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.605+0000 7ffa259ba740 1 -- 172.21.15.38:0/1218000437 shutdown_connections 2024-09-21T03:45:16.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.605+0000 7ffa259ba740 1 --2- 172.21.15.38:0/1218000437 >> [v2:172.21.15.38:6832/3839260519,v1:172.21.15.38:6833/3839260519] conn(0x55a915b76e40 0x55a915fb76e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:16.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.605+0000 7ffa259ba740 1 --2- 172.21.15.38:0/1218000437 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a915bb8270 0x55a915f83400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:16.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.605+0000 7ffa259ba740 1 --2- 172.21.15.38:0/1218000437 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a915b97590 0x55a915f9bbb0 unknown :-1 s=CLOSED pgs=2420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:16.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.605+0000 7ffa259ba740 1 --2- 172.21.15.38:0/1218000437 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a915b8f3b0 0x55a915fb3ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:16.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.605+0000 7ff9f6ffd640 0 client.10983 ms_handle_reset on v2:172.21.15.38:6832/3839260519 2024-09-21T03:45:16.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.605+0000 7ffa259ba740 1 -- 172.21.15.38:0/1218000437 >> 172.21.15.38:0/1218000437 conn(0x55a915bac3c0 msgr2=0x55a915fa69d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:16.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.605+0000 7ffa259ba740 1 -- 172.21.15.38:0/1218000437 shutdown_connections 2024-09-21T03:45:16.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.605+0000 7ffa259ba740 1 -- 172.21.15.38:0/1218000437 wait complete. 2024-09-21T03:45:16.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.605+0000 7ffa259ba740 20 client.10983 trim_cache size 0 max 16384 2024-09-21T03:45:16.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.624+0000 7ffa259ba740 1 Processor -- start 2024-09-21T03:45:16.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.624+0000 7ffa259ba740 1 -- start start 2024-09-21T03:45:16.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.625+0000 7ffa259ba740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a915b8f3b0 0x55a915f84440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:16.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.625+0000 7ffa259ba740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a915b97590 0x55a915fc7ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:16.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.625+0000 7ffa259ba740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a915bb8270 0x55a915fc80e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:16.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.625+0000 7ffa259ba740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ffa18201800 con 0x55a915b97590 2024-09-21T03:45:16.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.625+0000 7ffa259ba740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ffa18200b60 con 0x55a915b8f3b0 2024-09-21T03:45:16.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.625+0000 7ffa259ba740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7ffa182011b0 con 0x55a915bb8270 2024-09-21T03:45:16.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.625+0000 7ffa177fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a915b97590 0x55a915fc7ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:16.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.625+0000 7ffa177fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a915b97590 0x55a915fc7ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46366/0 (socket says 172.21.15.38:46366) 2024-09-21T03:45:16.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.625+0000 7ffa177fe640 1 -- 172.21.15.38:0/738388676 learned_addr learned my addr 172.21.15.38:0/738388676 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:45:16.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.626+0000 7ffa177fe640 1 -- 172.21.15.38:0/738388676 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a915bb8270 msgr2=0x55a915fc80e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:16.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.626+0000 7ffa177fe640 1 --2- 172.21.15.38:0/738388676 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a915bb8270 0x55a915fc80e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:16.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.626+0000 7ffa177fe640 1 -- 172.21.15.38:0/738388676 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a915b8f3b0 msgr2=0x55a915f84440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:16.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.626+0000 7ffa17fff640 1 --2- 172.21.15.38:0/738388676 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a915b8f3b0 0x55a915f84440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:16.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.626+0000 7ffa177fe640 1 --2- 172.21.15.38:0/738388676 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a915b8f3b0 0x55a915f84440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:16.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.626+0000 7ffa177fe640 1 -- 172.21.15.38:0/738388676 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a915f79190 con 0x55a915b97590 2024-09-21T03:45:16.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.626+0000 7ffa17fff640 1 --2- 172.21.15.38:0/738388676 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a915b8f3b0 0x55a915f84440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:45:16.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.626+0000 7ffa177fe640 1 --2- 172.21.15.38:0/738388676 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a915b97590 0x55a915fc7ba0 secure :-1 s=READY pgs=2421 cs=0 l=1 rev1=1 crypto rx=0x7ffa0c00ce00 tx=0x7ffa0c000f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6693fad06020a196 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:16.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.626+0000 7ff9fd7fa640 1 -- 172.21.15.38:0/738388676 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa0c084030 con 0x55a915b97590 2024-09-21T03:45:16.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.626+0000 7ff9fd7fa640 1 -- 172.21.15.38:0/738388676 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffa0c065070 con 0x55a915b97590 2024-09-21T03:45:16.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.626+0000 7ff9fd7fa640 1 -- 172.21.15.38:0/738388676 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa0c061520 con 0x55a915b97590 2024-09-21T03:45:16.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.627+0000 7ffa259ba740 1 -- 172.21.15.38:0/738388676 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a915b97590 msgr2=0x55a915fc7ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:16.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.627+0000 7ffa259ba740 1 --2- 172.21.15.38:0/738388676 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a915b97590 0x55a915fc7ba0 secure :-1 s=READY pgs=2421 cs=0 l=1 rev1=1 crypto rx=0x7ffa0c00ce00 tx=0x7ffa0c000f20 comp rx=0 tx=0).stop 2024-09-21T03:45:16.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.627+0000 7ffa259ba740 1 -- 172.21.15.38:0/738388676 shutdown_connections 2024-09-21T03:45:16.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.627+0000 7ffa259ba740 1 --2- 172.21.15.38:0/738388676 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a915bb8270 0x55a915fc80e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:16.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.627+0000 7ffa259ba740 1 --2- 172.21.15.38:0/738388676 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a915b97590 0x55a915fc7ba0 unknown :-1 s=CLOSED pgs=2421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:16.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.627+0000 7ffa259ba740 1 --2- 172.21.15.38:0/738388676 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a915b8f3b0 0x55a915f84440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:16.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.627+0000 7ffa259ba740 1 -- 172.21.15.38:0/738388676 >> 172.21.15.38:0/738388676 conn(0x55a915bac3c0 msgr2=0x55a915f77690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:16.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.627+0000 7ffa259ba740 1 -- 172.21.15.38:0/738388676 shutdown_connections 2024-09-21T03:45:16.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.627+0000 7ffa259ba740 1 -- 172.21.15.38:0/738388676 wait complete. 2024-09-21T03:45:16.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.628+0000 7ffa259ba740 1 -- start start 2024-09-21T03:45:16.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.628+0000 7ffa259ba740 1 Processor -- start 2024-09-21T03:45:16.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.629+0000 7ffa259ba740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a915b8f3b0 0x55a915f59c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:16.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.629+0000 7ffa259ba740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a915b97590 0x55a915f80f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:16.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.629+0000 7ffa259ba740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a915bb8270 0x55a9160281b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:16.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.629+0000 7ffa259ba740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55a915f9ecc0 con 0x55a915b8f3b0 2024-09-21T03:45:16.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.629+0000 7ffa259ba740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55a915f9c140 con 0x55a915bb8270 2024-09-21T03:45:16.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.629+0000 7ffa259ba740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55a915f77db0 con 0x55a915b97590 2024-09-21T03:45:16.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.629+0000 7ffa259ba740 10 client.0 _ll_register_callbacks cb 0x7ffa0c001e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:45:16.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.629+0000 7ffa17fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a915b8f3b0 0x55a915f59c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:16.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.629+0000 7ffa17fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a915b8f3b0 0x55a915f59c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46370/0 (socket says 172.21.15.38:46370) 2024-09-21T03:45:16.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.629+0000 7ffa17fff640 1 -- 172.21.15.38:0/214886164 learned_addr learned my addr 172.21.15.38:0/214886164 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:45:16.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.629+0000 7ffa1cf0b640 1 --2- 172.21.15.38:0/214886164 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a915bb8270 0x55a9160281b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:16.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.629+0000 7ffa177fe640 1 --2- 172.21.15.38:0/214886164 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a915b97590 0x55a915f80f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:16.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.630+0000 7ffa17fff640 1 -- 172.21.15.38:0/214886164 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a915b97590 msgr2=0x55a915f80f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:16.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.630+0000 7ffa17fff640 1 --2- 172.21.15.38:0/214886164 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a915b97590 0x55a915f80f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:16.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.630+0000 7ffa17fff640 1 -- 172.21.15.38:0/214886164 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a915bb8270 msgr2=0x55a9160281b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:16.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.630+0000 7ffa17fff640 1 --2- 172.21.15.38:0/214886164 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a915bb8270 0x55a9160281b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:16.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.630+0000 7ffa17fff640 1 -- 172.21.15.38:0/214886164 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a915f5a140 con 0x55a915b8f3b0 2024-09-21T03:45:16.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.630+0000 7ffa1cf0b640 1 --2- 172.21.15.38:0/214886164 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a915bb8270 0x55a9160281b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:45:16.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.630+0000 7ffa17fff640 1 --2- 172.21.15.38:0/214886164 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a915b8f3b0 0x55a915f59c00 secure :-1 s=READY pgs=2422 cs=0 l=1 rev1=1 crypto rx=0x7ffa08007a80 tx=0x7ffa0800ef30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c24c71bac84ea733 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:16.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.630+0000 7ff9f6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-21T03:45:16.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.630+0000 7ff9f6ffd640 1 -- 172.21.15.38:0/214886164 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa08039060 con 0x55a915b8f3b0 2024-09-21T03:45:16.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.630+0000 7ff9f6ffd640 1 -- 172.21.15.38:0/214886164 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(147..147 src has 1..147) ==== 6187+0+0 (secure 0 0 0) 0x7ffa0802d840 con 0x55a915b8f3b0 2024-09-21T03:45:16.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.630+0000 7ffa259ba740 1 -- 172.21.15.38:0/214886164 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa0c065070 con 0x55a915b8f3b0 2024-09-21T03:45:16.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.630+0000 7ff9f6ffd640 1 -- 172.21.15.38:0/214886164 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=349) ==== 24+0+0 (secure 0 0 0) 0x7ffa0803f050 con 0x55a915b8f3b0 2024-09-21T03:45:16.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.630+0000 7ffa259ba740 10 client.10989 fetch_fsmap learned FSMap version 349 2024-09-21T03:45:16.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.630+0000 7ffa259ba740 1 -- 172.21.15.38:0/214886164 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=349,monmap=2+}) -- 0x55a915f5a410 con 0x55a915b8f3b0 2024-09-21T03:45:16.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.630+0000 7ff9f6ffd640 1 -- 172.21.15.38:0/214886164 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffa0c065070 con 0x55a915b8f3b0 2024-09-21T03:45:16.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.630+0000 7ff9f6ffd640 1 -- 172.21.15.38:0/214886164 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 349) ==== 3502+0+0 (secure 0 0 0) 0x7ffa08042020 con 0x55a915b8f3b0 2024-09-21T03:45:16.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.630+0000 7ffa259ba740 10 client.10989 fetch_fsmap finished waiting for FSMap version 349 2024-09-21T03:45:16.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.631+0000 7ffa259ba740 10 client.10989 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T03:45:16.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.631+0000 7ffa259ba740 20 client.10989 populate_metadata read hostname 'smithi038' 2024-09-21T03:45:16.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.631+0000 7ffa259ba740 1 --2- 172.21.15.38:0/214886164 >> [v2:172.21.15.38:6832/3839260519,v1:172.21.15.38:6833/3839260519] conn(0x55a915f84b20 0x55a915f79710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:16.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.631+0000 7ffa259ba740 4 client.10989 mds_command: new command op to 10935 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T03:45:16.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.631+0000 7ffa259ba740 1 -- 172.21.15.38:0/214886164 --> [v2:172.21.15.38:6832/3839260519,v1:172.21.15.38:6833/3839260519] -- command(tid 0: {"prefix": "status"}) -- 0x55a915ed0220 con 0x55a915f84b20 2024-09-21T03:45:16.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.631+0000 7ffa177fe640 1 --2- 172.21.15.38:0/214886164 >> [v2:172.21.15.38:6832/3839260519,v1:172.21.15.38:6833/3839260519] conn(0x55a915f84b20 0x55a915f79710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:16.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.631+0000 7ff9f6ffd640 10 client.10989 ms_handle_connect on v2:172.21.15.38:6832/3839260519 2024-09-21T03:45:16.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.631+0000 7ffa177fe640 1 --2- 172.21.15.38:0/214886164 >> [v2:172.21.15.38:6832/3839260519,v1:172.21.15.38:6833/3839260519] conn(0x55a915f84b20 0x55a915f79710 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bbec48993b7656d3 server_cookie=94db65f44c99fa69 in_seq=0 out_seq=0 2024-09-21T03:45:16.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.631+0000 7ff9f6ffd640 1 -- 172.21.15.38:0/214886164 <== mds.0 v2:172.21.15.38:6832/3839260519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+335 (crc 0 0 0) 0x55a915ed0220 con 0x55a915f84b20 2024-09-21T03:45:16.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.631+0000 7ff9f6ffd640 10 client.10989 handle_command_reply: tid=0 2024-09-21T03:45:16.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.631+0000 7ffa259ba740 1 client.10989 shutdown 2024-09-21T03:45:16.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.632+0000 7ffa259ba740 1 -- 172.21.15.38:0/214886164 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a915b8f3b0 msgr2=0x55a915f59c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:16.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.632+0000 7ffa259ba740 1 --2- 172.21.15.38:0/214886164 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a915b8f3b0 0x55a915f59c00 secure :-1 s=READY pgs=2422 cs=0 l=1 rev1=1 crypto rx=0x7ffa08007a80 tx=0x7ffa0800ef30 comp rx=0 tx=0).stop 2024-09-21T03:45:16.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.632+0000 7ffa259ba740 1 -- 172.21.15.38:0/214886164 shutdown_connections 2024-09-21T03:45:16.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.632+0000 7ffa259ba740 1 --2- 172.21.15.38:0/214886164 >> [v2:172.21.15.38:6832/3839260519,v1:172.21.15.38:6833/3839260519] conn(0x55a915f84b20 0x55a915f79710 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:16.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.632+0000 7ffa259ba740 1 --2- 172.21.15.38:0/214886164 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a915bb8270 0x55a9160281b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:16.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.632+0000 7ffa259ba740 1 --2- 172.21.15.38:0/214886164 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a915b97590 0x55a915f80f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:16.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.632+0000 7ff9f6ffd640 0 client.10989 ms_handle_reset on v2:172.21.15.38:6832/3839260519 2024-09-21T03:45:16.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.632+0000 7ffa259ba740 1 --2- 172.21.15.38:0/214886164 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a915b8f3b0 0x55a915f59c00 unknown :-1 s=CLOSED pgs=2422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:16.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.632+0000 7ffa259ba740 1 -- 172.21.15.38:0/214886164 >> 172.21.15.38:0/214886164 conn(0x55a915bac3c0 msgr2=0x55a915f77630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:16.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.632+0000 7ffa259ba740 1 -- 172.21.15.38:0/214886164 shutdown_connections 2024-09-21T03:45:16.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.632+0000 7ffa259ba740 1 -- 172.21.15.38:0/214886164 wait complete. 2024-09-21T03:45:16.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.632+0000 7ffa259ba740 20 client.10989 trim_cache size 0 max 16384 2024-09-21T03:45:16.645 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-09-21T03:45:16.645 INFO:teuthology.orchestra.run.smithi038.stdout: "cluster_fsid": "9d50d66f-656b-4fe2-8e3b-fe53b79c4d43", 2024-09-21T03:45:16.645 INFO:teuthology.orchestra.run.smithi038.stdout: "whoami": 0, 2024-09-21T03:45:16.645 INFO:teuthology.orchestra.run.smithi038.stdout: "id": 10935, 2024-09-21T03:45:16.645 INFO:teuthology.orchestra.run.smithi038.stdout: "want_state": "up:active", 2024-09-21T03:45:16.645 INFO:teuthology.orchestra.run.smithi038.stdout: "state": "up:active", 2024-09-21T03:45:16.645 INFO:teuthology.orchestra.run.smithi038.stdout: "fs_name": "cephfs", 2024-09-21T03:45:16.646 INFO:teuthology.orchestra.run.smithi038.stdout: "rank_uptime": 4.5378955779999997, 2024-09-21T03:45:16.646 INFO:teuthology.orchestra.run.smithi038.stdout: "mdsmap_epoch": 349, 2024-09-21T03:45:16.646 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch": 147, 2024-09-21T03:45:16.646 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch_barrier": 147, 2024-09-21T03:45:16.646 INFO:teuthology.orchestra.run.smithi038.stdout: "uptime": 5.4891099590000003 2024-09-21T03:45:16.646 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-09-21T03:45:16.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.632+0000 7ffa1e96e640 1 -- 172.21.15.38:0/3358999554 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ffa04078b90 msgr2=0x7ffa0407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:16.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.632+0000 7ffa1e96e640 1 --2- 172.21.15.38:0/3358999554 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ffa04078b90 0x7ffa0407b0a0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7ffa080013f0 tx=0x7ffa08009040 comp rx=0 tx=0).stop 2024-09-21T03:45:16.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.632+0000 7ffa1e96e640 1 -- 172.21.15.38:0/3358999554 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffa180d0170 msgr2=0x7ffa180cdf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:16.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.632+0000 7ffa1e96e640 1 --2- 172.21.15.38:0/3358999554 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffa180d0170 0x7ffa180cdf00 secure :-1 s=READY pgs=2418 cs=0 l=1 rev1=1 crypto rx=0x7ffa0c00c440 tx=0x7ffa0c00c900 comp rx=0 tx=0).stop 2024-09-21T03:45:16.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.632+0000 7ffa1e96e640 1 -- 172.21.15.38:0/3358999554 shutdown_connections 2024-09-21T03:45:16.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.632+0000 7ffa1e96e640 1 --2- 172.21.15.38:0/3358999554 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ffa04078b90 0x7ffa0407b0a0 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:16.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.632+0000 7ffa1e96e640 1 --2- 172.21.15.38:0/3358999554 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffa1815ee80 0x7ffa1815b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:16.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.632+0000 7ffa1e96e640 1 --2- 172.21.15.38:0/3358999554 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffa180d0170 0x7ffa180cdf00 unknown :-1 s=CLOSED pgs=2418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:16.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.632+0000 7ffa1e96e640 1 --2- 172.21.15.38:0/3358999554 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffa180ceb90 0x7ffa1806f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:16.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.632+0000 7ffa1e96e640 1 -- 172.21.15.38:0/3358999554 >> 172.21.15.38:0/3358999554 conn(0x7ffa1809d830 msgr2=0x7ffa18161bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:16.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.633+0000 7ffa1e96e640 1 -- 172.21.15.38:0/3358999554 shutdown_connections 2024-09-21T03:45:16.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.633+0000 7ffa1e96e640 1 -- 172.21.15.38:0/3358999554 wait complete. 2024-09-21T03:45:16.648 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T03:45:16.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.763+0000 7f2ecd263640 1 -- 172.21.15.38:0/2718297675 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2ec80cfe00 msgr2=0x7f2ec80d2a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:16.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.763+0000 7f2ecd263640 1 --2- 172.21.15.38:0/2718297675 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2ec80cfe00 0x7f2ec80d2a20 secure :-1 s=READY pgs=2424 cs=0 l=1 rev1=1 crypto rx=0x7f2ec80b5a40 tx=0x7f2eb802a530 comp rx=0 tx=0).stop 2024-09-21T03:45:16.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.763+0000 7f2ecd263640 1 -- 172.21.15.38:0/2718297675 shutdown_connections 2024-09-21T03:45:16.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.763+0000 7f2ecd263640 1 --2- 172.21.15.38:0/2718297675 >> v2:172.21.15.195:3300/0 conn(0x7f2ec80b2e60 0x7f2ec80b3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:16.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.763+0000 7f2ecd263640 1 --2- 172.21.15.38:0/2718297675 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2ec80cfe00 0x7f2ec80d2a20 unknown :-1 s=CLOSED pgs=2424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:16.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.763+0000 7f2ecd263640 1 -- 172.21.15.38:0/2718297675 >> 172.21.15.38:0/2718297675 conn(0x7f2ec80d4440 msgr2=0x7f2ec80d4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:16.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.763+0000 7f2ecd263640 1 -- 172.21.15.38:0/2718297675 shutdown_connections 2024-09-21T03:45:16.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.763+0000 7f2ecd263640 1 -- 172.21.15.38:0/2718297675 wait complete. 2024-09-21T03:45:16.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.765+0000 7f2ecd263640 1 Processor -- start 2024-09-21T03:45:16.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.765+0000 7f2ecd263640 1 -- start start 2024-09-21T03:45:16.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.767+0000 7f2ecd263640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2ec80b2e60 0x7f2ec8157df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:16.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.767+0000 7f2ecd263640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2ec80cfe00 0x7f2ec8145810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:16.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.767+0000 7f2ecd263640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2ec8162590 0x7f2ec81f86b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:16.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.767+0000 7f2ecd263640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f2ec80b3930 con 0x7f2ec8162590 2024-09-21T03:45:16.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.767+0000 7f2ecd263640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f2ec80b37b0 con 0x7f2ec80cfe00 2024-09-21T03:45:16.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.767+0000 7f2ecd263640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f2ec80b3ab0 con 0x7f2ec80b2e60 2024-09-21T03:45:16.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.767+0000 7f2ec77fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2ec8162590 0x7f2ec81f86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:16.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.767+0000 7f2ec77fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2ec8162590 0x7f2ec81f86b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46382/0 (socket says 172.21.15.38:46382) 2024-09-21T03:45:16.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.767+0000 7f2ec77fe640 1 -- 172.21.15.38:0/491185720 learned_addr learned my addr 172.21.15.38:0/491185720 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:45:16.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.767+0000 7f2ec67fc640 1 --2- 172.21.15.38:0/491185720 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2ec80cfe00 0x7f2ec8145810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:16.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.767+0000 7f2ec6ffd640 1 --2- 172.21.15.38:0/491185720 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2ec80b2e60 0x7f2ec8157df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:16.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.767+0000 7f2ec77fe640 1 -- 172.21.15.38:0/491185720 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2ec80b2e60 msgr2=0x7f2ec8157df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:16.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.767+0000 7f2ec77fe640 1 --2- 172.21.15.38:0/491185720 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2ec80b2e60 0x7f2ec8157df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:16.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.767+0000 7f2ec77fe640 1 -- 172.21.15.38:0/491185720 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2ec80cfe00 msgr2=0x7f2ec8145810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:16.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.767+0000 7f2ec77fe640 1 --2- 172.21.15.38:0/491185720 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2ec80cfe00 0x7f2ec8145810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:16.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.767+0000 7f2ec77fe640 1 -- 172.21.15.38:0/491185720 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ec8156d50 con 0x7f2ec8162590 2024-09-21T03:45:16.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.767+0000 7f2ec77fe640 1 --2- 172.21.15.38:0/491185720 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2ec8162590 0x7f2ec81f86b0 secure :-1 s=READY pgs=2425 cs=0 l=1 rev1=1 crypto rx=0x7f2ec000cad0 tx=0x7f2ec000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:16.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.767+0000 7f2eaf7fe640 1 -- 172.21.15.38:0/491185720 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ec000ae20 con 0x7f2ec8162590 2024-09-21T03:45:16.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.767+0000 7f2eaf7fe640 1 -- 172.21.15.38:0/491185720 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2ec000f450 con 0x7f2ec8162590 2024-09-21T03:45:16.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.767+0000 7f2eaf7fe640 1 -- 172.21.15.38:0/491185720 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ec000f8c0 con 0x7f2ec8162590 2024-09-21T03:45:16.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.767+0000 7f2ecd263640 1 -- 172.21.15.38:0/491185720 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ec8157030 con 0x7f2ec8162590 2024-09-21T03:45:16.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.768+0000 7f2ecd263640 1 -- 172.21.15.38:0/491185720 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ec81575f0 con 0x7f2ec8162590 2024-09-21T03:45:16.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.769+0000 7f2ecd263640 1 -- 172.21.15.38:0/491185720 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ec80b3270 con 0x7f2ec8162590 2024-09-21T03:45:16.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.770+0000 7f2eaf7fe640 1 -- 172.21.15.38:0/491185720 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f2ec000fa60 con 0x7f2ec8162590 2024-09-21T03:45:16.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.770+0000 7f2eaf7fe640 1 --2- 172.21.15.38:0/491185720 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2eb4078bc0 0x7f2eb407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:16.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.770+0000 7f2eaf7fe640 1 -- 172.21.15.38:0/491185720 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 6187+0+0 (secure 0 0 0) 0x7f2ec009ba10 con 0x7f2ec8162590 2024-09-21T03:45:16.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.770+0000 7f2ec6ffd640 1 --2- 172.21.15.38:0/491185720 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2eb4078bc0 0x7f2eb407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:16.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.771+0000 7f2ec6ffd640 1 --2- 172.21.15.38:0/491185720 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2eb4078bc0 0x7f2eb407b0d0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f2eb8034000 tx=0x7f2eb8004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:16.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:16.775+0000 7f2eaf7fe640 1 -- 172.21.15.38:0/491185720 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f2ec0014030 con 0x7f2ec8162590 2024-09-21T03:45:17.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:17.007+0000 7f2ecd263640 1 -- 172.21.15.38:0/491185720 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f2ec806f4f0 con 0x7f2ec8162590 2024-09-21T03:45:17.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:17.007+0000 7f2eaf7fe640 1 -- 172.21.15.38:0/491185720 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v349) ==== 53+0+73 (secure 0 0 0) 0x7f2ec0063d00 con 0x7f2ec8162590 2024-09-21T03:45:17.010 INFO:teuthology.orchestra.run.smithi038.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T03:45:17.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:17.010+0000 7f2ecd263640 1 -- 172.21.15.38:0/491185720 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2eb4078bc0 msgr2=0x7f2eb407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:17.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:17.010+0000 7f2ecd263640 1 --2- 172.21.15.38:0/491185720 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2eb4078bc0 0x7f2eb407b0d0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f2eb8034000 tx=0x7f2eb8004730 comp rx=0 tx=0).stop 2024-09-21T03:45:17.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:17.010+0000 7f2ecd263640 1 -- 172.21.15.38:0/491185720 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2ec8162590 msgr2=0x7f2ec81f86b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:17.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:17.010+0000 7f2ecd263640 1 --2- 172.21.15.38:0/491185720 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2ec8162590 0x7f2ec81f86b0 secure :-1 s=READY pgs=2425 cs=0 l=1 rev1=1 crypto rx=0x7f2ec000cad0 tx=0x7f2ec000cf90 comp rx=0 tx=0).stop 2024-09-21T03:45:17.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:17.010+0000 7f2ecd263640 1 -- 172.21.15.38:0/491185720 shutdown_connections 2024-09-21T03:45:17.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:17.010+0000 7f2ecd263640 1 --2- 172.21.15.38:0/491185720 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2ec8162590 0x7f2ec81f86b0 unknown :-1 s=CLOSED pgs=2425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:17.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:17.010+0000 7f2ecd263640 1 --2- 172.21.15.38:0/491185720 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2ec80cfe00 0x7f2ec8145810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:17.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:17.010+0000 7f2ecd263640 1 --2- 172.21.15.38:0/491185720 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f2eb4078bc0 0x7f2eb407b0d0 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:17.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:17.010+0000 7f2ecd263640 1 --2- 172.21.15.38:0/491185720 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2ec80b2e60 0x7f2ec8157df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:17.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:17.010+0000 7f2ecd263640 1 -- 172.21.15.38:0/491185720 >> 172.21.15.38:0/491185720 conn(0x7f2ec80d4440 msgr2=0x7f2ec8158df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:17.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:17.010+0000 7f2ecd263640 1 -- 172.21.15.38:0/491185720 shutdown_connections 2024-09-21T03:45:17.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:17.010+0000 7f2ecd263640 1 -- 172.21.15.38:0/491185720 wait complete. 2024-09-21T03:45:17.023 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-21T03:45:17.023 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T03:45:17.023 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi038.front.sepia.ceph.com 2024-09-21T03:45:17.023 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-21T03:45:17.023 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T03:45:17.024 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-21T03:45:17.024 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-21T03:45:17.024 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T03:45:17.024 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T03:45:17.024 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:45:17.024 DEBUG:teuthology.orchestra.run.smithi038:> ip addr 2024-09-21T03:45:17.040 INFO:teuthology.orchestra.run.smithi038.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T03:45:17.040 INFO:teuthology.orchestra.run.smithi038.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T03:45:17.040 INFO:teuthology.orchestra.run.smithi038.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T03:45:17.041 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:45:17.041 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 ::1/128 scope host 2024-09-21T03:45:17.041 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:45:17.041 INFO:teuthology.orchestra.run.smithi038.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T03:45:17.041 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:6b:fd:a2 brd ff:ff:ff:ff:ff:ff 2024-09-21T03:45:17.041 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp7s0f0 2024-09-21T03:45:17.041 INFO:teuthology.orchestra.run.smithi038.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T03:45:17.041 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:6b:fd:a3 brd ff:ff:ff:ff:ff:ff 2024-09-21T03:45:17.041 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp7s0f1 2024-09-21T03:45:17.041 INFO:teuthology.orchestra.run.smithi038.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T03:45:17.041 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:bd:15:fc brd ff:ff:ff:ff:ff:ff 2024-09-21T03:45:17.041 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp1s0f0 2024-09-21T03:45:17.041 INFO:teuthology.orchestra.run.smithi038.stdout: inet 172.21.15.38/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-21T03:45:17.042 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft 42092sec preferred_lft 42092sec 2024-09-21T03:45:17.042 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 fe80::ec4:7aff:febd:15fc/64 scope link 2024-09-21T03:45:17.042 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:45:17.042 INFO:teuthology.orchestra.run.smithi038.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T03:45:17.042 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:bd:15:fd brd ff:ff:ff:ff:ff:ff 2024-09-21T03:45:17.042 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp1s0f1 2024-09-21T03:45:17.042 INFO:teuthology.orchestra.run.smithi038.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T03:45:17.042 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 92:55:c7:97:6f:b5 brd ff:ff:ff:ff:ff:ff 2024-09-21T03:45:17.042 INFO:teuthology.orchestra.run.smithi038.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-21T03:45:17.042 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:45:17.042 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 fe80::c49f:c1ff:fe21:b12a/64 scope link 2024-09-21T03:45:17.042 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:45:17.043 INFO:teuthology.orchestra.run.smithi038.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T03:45:17.043 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether a6:3a:83:ad:db:d5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T03:45:17.043 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 fe80::a43a:83ff:fead:dbd5/64 scope link 2024-09-21T03:45:17.043 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:45:17.043 INFO:teuthology.orchestra.run.smithi038.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T03:45:17.043 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 92:55:c7:97:6f:b5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T03:45:17.043 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 fe80::9055:c7ff:fe97:6fb5/64 scope link 2024-09-21T03:45:17.043 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:45:17.043 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:45:17.043 DEBUG:teuthology.orchestra.run.smithi038:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T03:45:17.094 INFO:tasks.ceph.mds.c.smithi038.stderr:2024-09-21T03:45:17.093+0000 7f6c69fd9640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T03:45:17.105 INFO:teuthology.orchestra.run.smithi038.stdout:1 2024-09-21T03:45:17.106 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:45:17.107 DEBUG:teuthology.orchestra.run.smithi038:> route 2024-09-21T03:45:17.166 INFO:teuthology.orchestra.run.smithi038.stdout:Kernel IP routing table 2024-09-21T03:45:17.166 INFO:teuthology.orchestra.run.smithi038.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T03:45:17.166 INFO:teuthology.orchestra.run.smithi038.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-21T03:45:17.166 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-21T03:45:17.166 INFO:teuthology.orchestra.run.smithi038.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T03:45:17.166 DEBUG:teuthology.orchestra.run.smithi038:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T03:45:17.166 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-09-21T03:45:17.167 DEBUG:teuthology.orchestra.run.smithi038:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-21T03:45:17.167 DEBUG:teuthology.orchestra.run.smithi038:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-21T03:45:17.167 DEBUG:teuthology.orchestra.run.smithi038:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-21T03:45:17.167 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-09-21T03:45:17.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:17 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T03:45:17.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:45:17.295 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:45:17.295 DEBUG:teuthology.orchestra.run.smithi038:> ip netns list 2024-09-21T03:45:17.311 INFO:teuthology.orchestra.run.smithi038.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-21T03:45:17.311 INFO:teuthology.orchestra.run.smithi038.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-21T03:45:17.311 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T03:45:17.311 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:45:17.311 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip link set brx.0 up 2024-09-21T03:45:17.376 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T03:45:17.376 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-21T03:45:17.433 INFO:teuthology.orchestra.run.smithi038.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-21T03:45:17.433 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T03:45:17.433 DEBUG:teuthology.orchestra.run.smithi038:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-21T03:45:17.488 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-21T03:45:17.488 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:45:17.488 DEBUG:teuthology.orchestra.run.smithi038:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T03:45:17.584 INFO:teuthology.orchestra.run.smithi038.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T03:45:17.584 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T03:45:17.584 INFO:teuthology.orchestra.run.smithi038.stdout:invalid new device string format 2024-09-21T03:45:17.595 INFO:teuthology.orchestra.run.smithi038.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T03:45:17.595 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T03:45:17.596 INFO:teuthology.orchestra.run.smithi038.stdout:invalid new device string format 2024-09-21T03:45:17.596 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: resolved to: "172.21.15.38:3300,172.21.15.195:3300,172.21.15.195:3301" 2024-09-21T03:45:17.596 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T03:45:17.603 INFO:teuthology.orchestra.run.smithi038.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T03:45:17.603 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T03:45:17.603 INFO:teuthology.orchestra.run.smithi038.stdout:invalid new device string format 2024-09-21T03:45:17.603 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: resolved to: "172.21.15.38:3300,172.21.15.195:3300,172.21.15.195:3301" 2024-09-21T03:45:17.603 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T03:45:17.603 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: resolved to: "172.21.15.38:6789,172.21.15.195:6789,172.21.15.195:6790" 2024-09-21T03:45:17.603 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.38:6789,172.21.15.195:6789,172.21.15.195:6790:/ 2024-09-21T03:45:17.603 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=9d50d66f-656b-4fe2-8e3b-fe53b79c4d43" will pass to kernel 2024-09-21T03:45:17.604 INFO:teuthology.orchestra.run.smithi038.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-21T03:45:17.604 INFO:teuthology.orchestra.run.smithi038.stdout: You just mounted a file system that supports labels which does not 2024-09-21T03:45:17.604 INFO:teuthology.orchestra.run.smithi038.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-21T03:45:17.604 INFO:teuthology.orchestra.run.smithi038.stdout: applications will generate AVC messages and not be allowed access to 2024-09-21T03:45:17.604 INFO:teuthology.orchestra.run.smithi038.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-21T03:45:17.606 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T03:45:17.606 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:45:17.606 DEBUG:teuthology.orchestra.run.smithi038:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-21T03:45:17.631 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T03:45:17.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:17 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T03:45:17.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:45:17.729 INFO:teuthology.orchestra.run.smithi038.stdout:client8546 2024-09-21T03:45:17.729 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-21T03:45:17.729 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:45:17.745 DEBUG:teuthology.orchestra.run.smithi038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000001.00000000 parent 2024-09-21T03:45:17.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:17.840+0000 7f06c8936f40 1 Processor -- start 2024-09-21T03:45:17.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:17.840+0000 7f06c8936f40 1 -- start start 2024-09-21T03:45:17.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:17.842+0000 7f06c8936f40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f06ac002320 0x5635f2150f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:17.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:17.842+0000 7f06c8936f40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5635f214b9b0 0x5635f21c48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:17.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:17.842+0000 7f06c8936f40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5635f2137f80 0x5635f214f110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:17.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:17.842+0000 7f06c8936f40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5635f20b9430 con 0x5635f214b9b0 2024-09-21T03:45:17.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:17.842+0000 7f06c8936f40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5635f20c61f0 con 0x7f06ac002320 2024-09-21T03:45:17.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:17.842+0000 7f06c8936f40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x5635f205ad60 con 0x5635f2137f80 2024-09-21T03:45:17.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:17.842+0000 7f06c1bef640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5635f214b9b0 0x5635f21c48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:17.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:17.842+0000 7f06c1bef640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5635f214b9b0 0x5635f21c48a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46402/0 (socket says 172.21.15.38:46402) 2024-09-21T03:45:17.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:17.842+0000 7f06c1bef640 1 -- 172.21.15.38:0/3165505970 learned_addr learned my addr 172.21.15.38:0/3165505970 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:45:17.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:17.842+0000 7f06c23f0640 1 --2- 172.21.15.38:0/3165505970 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f06ac002320 0x5635f2150f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:17.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:17.842+0000 7f06c2bf1640 1 --2- 172.21.15.38:0/3165505970 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5635f2137f80 0x5635f214f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:17.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:17.842+0000 7f06c1bef640 1 -- 172.21.15.38:0/3165505970 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5635f2137f80 msgr2=0x5635f214f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:17.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:17.842+0000 7f06c1bef640 1 --2- 172.21.15.38:0/3165505970 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5635f2137f80 0x5635f214f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:17.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:17.842+0000 7f06c1bef640 1 -- 172.21.15.38:0/3165505970 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f06ac002320 msgr2=0x5635f2150f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:17.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:17.842+0000 7f06c1bef640 1 --2- 172.21.15.38:0/3165505970 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f06ac002320 0x5635f2150f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:17.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:17.842+0000 7f06c1bef640 1 -- 172.21.15.38:0/3165505970 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5635f21c8950 con 0x5635f214b9b0 2024-09-21T03:45:17.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:17.842+0000 7f06c23f0640 1 --2- 172.21.15.38:0/3165505970 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f06ac002320 0x5635f2150f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:45:17.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:17.842+0000 7f06c1bef640 1 --2- 172.21.15.38:0/3165505970 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5635f214b9b0 0x5635f21c48a0 secure :-1 s=READY pgs=2428 cs=0 l=1 rev1=1 crypto rx=0x7f06b800b680 tx=0x7f06b800bb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d24cda0e1dd8525 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:17.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:17.842+0000 7f06c0bed640 1 -- 172.21.15.38:0/3165505970 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06b800ac10 con 0x5635f214b9b0 2024-09-21T03:45:17.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:17.843+0000 7f06c0bed640 1 -- 172.21.15.38:0/3165505970 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f06b800adb0 con 0x5635f214b9b0 2024-09-21T03:45:17.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:17.843+0000 7f06c0bed640 1 -- 172.21.15.38:0/3165505970 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06b80176b0 con 0x5635f214b9b0 2024-09-21T03:45:17.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:17.843+0000 7f06c8936f40 1 -- 172.21.15.38:0/3165505970 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5635f214b9b0 msgr2=0x5635f21c48a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:17.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:17.843+0000 7f06c8936f40 1 --2- 172.21.15.38:0/3165505970 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5635f214b9b0 0x5635f21c48a0 secure :-1 s=READY pgs=2428 cs=0 l=1 rev1=1 crypto rx=0x7f06b800b680 tx=0x7f06b800bb40 comp rx=0 tx=0).stop 2024-09-21T03:45:17.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:17.843+0000 7f06c8936f40 1 -- 172.21.15.38:0/3165505970 shutdown_connections 2024-09-21T03:45:17.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:17.843+0000 7f06c8936f40 1 --2- 172.21.15.38:0/3165505970 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5635f2137f80 0x5635f214f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:17.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:17.843+0000 7f06c8936f40 1 --2- 172.21.15.38:0/3165505970 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5635f214b9b0 0x5635f21c48a0 unknown :-1 s=CLOSED pgs=2428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:17.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:17.843+0000 7f06c8936f40 1 --2- 172.21.15.38:0/3165505970 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f06ac002320 0x5635f2150f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:17.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:17.843+0000 7f06c8936f40 1 -- 172.21.15.38:0/3165505970 >> 172.21.15.38:0/3165505970 conn(0x5635f202bac0 msgr2=0x5635f210e050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:17.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:17.843+0000 7f06c8936f40 1 -- 172.21.15.38:0/3165505970 shutdown_connections 2024-09-21T03:45:17.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:17.843+0000 7f06c8936f40 1 -- 172.21.15.38:0/3165505970 wait complete. 2024-09-21T03:45:17.851 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:17.844+0000 7f06c8936f40 1 Processor -- start 2024-09-21T03:45:17.851 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:17.844+0000 7f06c8936f40 1 -- start start 2024-09-21T03:45:17.851 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:17.845+0000 7f06c8936f40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5635f2137f80 0x5635f21c2540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:17.851 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:17.845+0000 7f06c8936f40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5635f214b9b0 0x5635f2155a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:17.851 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:17.845+0000 7f06c8936f40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f06ac002320 0x5635f2155f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:17.851 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:17.845+0000 7f06c8936f40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5635f21df3f0 con 0x5635f214b9b0 2024-09-21T03:45:17.851 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:17.845+0000 7f06c8936f40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5635f21dfb70 con 0x5635f2137f80 2024-09-21T03:45:17.851 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:17.845+0000 7f06c8936f40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x5635f21c85e0 con 0x7f06ac002320 2024-09-21T03:45:17.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:17.846+0000 7f06c1bef640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5635f214b9b0 0x5635f2155a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:17.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:17.846+0000 7f06c1bef640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5635f214b9b0 0x5635f2155a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46418/0 (socket says 172.21.15.38:46418) 2024-09-21T03:45:17.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:17.846+0000 7f06c1bef640 1 -- 172.21.15.38:0/2898522883 learned_addr learned my addr 172.21.15.38:0/2898522883 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:45:17.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:17.846+0000 7f06c23f0640 1 --2- 172.21.15.38:0/2898522883 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5635f2137f80 0x5635f21c2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:17.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:17.846+0000 7f06c2bf1640 1 --2- 172.21.15.38:0/2898522883 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f06ac002320 0x5635f2155f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:17.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:17.846+0000 7f06c1bef640 1 -- 172.21.15.38:0/2898522883 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f06ac002320 msgr2=0x5635f2155f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:17.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:17.846+0000 7f06c1bef640 1 --2- 172.21.15.38:0/2898522883 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f06ac002320 0x5635f2155f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:17.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:17.846+0000 7f06c1bef640 1 -- 172.21.15.38:0/2898522883 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5635f2137f80 msgr2=0x5635f21c2540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:17.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:17.846+0000 7f06c1bef640 1 --2- 172.21.15.38:0/2898522883 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5635f2137f80 0x5635f21c2540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:17.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:17.846+0000 7f06c1bef640 1 -- 172.21.15.38:0/2898522883 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5635f214dcf0 con 0x5635f214b9b0 2024-09-21T03:45:17.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:17.846+0000 7f06c1bef640 1 --2- 172.21.15.38:0/2898522883 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5635f214b9b0 0x5635f2155a30 secure :-1 s=READY pgs=2429 cs=0 l=1 rev1=1 crypto rx=0x7f06b8018990 tx=0x7f06b80189c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:17.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:17.846+0000 7f06b37fe640 1 -- 172.21.15.38:0/2898522883 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06b8018df0 con 0x5635f214b9b0 2024-09-21T03:45:17.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:17.846+0000 7f06b37fe640 1 -- 172.21.15.38:0/2898522883 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f06b8022410 con 0x5635f214b9b0 2024-09-21T03:45:17.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:17.846+0000 7f06b37fe640 1 -- 172.21.15.38:0/2898522883 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06b8022860 con 0x5635f214b9b0 2024-09-21T03:45:17.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:17.846+0000 7f06c8936f40 1 -- 172.21.15.38:0/2898522883 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5635f214dfd0 con 0x5635f214b9b0 2024-09-21T03:45:17.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:17.846+0000 7f06c8936f40 1 -- 172.21.15.38:0/2898522883 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5635f214e2b0 con 0x5635f214b9b0 2024-09-21T03:45:17.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:17.848+0000 7f06b37fe640 1 -- 172.21.15.38:0/2898522883 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f06b8020030 con 0x5635f214b9b0 2024-09-21T03:45:17.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:17.848+0000 7f06b37fe640 1 --2- 172.21.15.38:0/2898522883 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f06a40775a0 0x7f06a4079ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:17.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:17.849+0000 7f06b37fe640 1 -- 172.21.15.38:0/2898522883 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 6303+0+0 (secure 0 0 0) 0x7f06b809b780 con 0x5635f214b9b0 2024-09-21T03:45:17.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:17.849+0000 7f06c23f0640 1 --2- 172.21.15.38:0/2898522883 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f06a40775a0 0x7f06a4079ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:17.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:17.849+0000 7f06c8936f40 1 --2- 172.21.15.38:0/2898522883 >> [v2:172.21.15.195:6802/3401837571,v1:172.21.15.195:6803/3401837571] conn(0x5635f2151cb0 0x5635f21520c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:17.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:17.849+0000 7f06c8936f40 1 -- 172.21.15.38:0/2898522883 --> [v2:172.21.15.195:6802/3401837571,v1:172.21.15.195:6803/3401837571] -- osd_op(unknown.0.0:1 16.7 16:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e148) -- 0x5635f2152600 con 0x5635f2151cb0 2024-09-21T03:45:17.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:17.849+0000 7f06c2bf1640 1 --2- 172.21.15.38:0/2898522883 >> [v2:172.21.15.195:6802/3401837571,v1:172.21.15.195:6803/3401837571] conn(0x5635f2151cb0 0x5635f21520c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:17.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:17.849+0000 7f06c23f0640 1 --2- 172.21.15.38:0/2898522883 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f06a40775a0 0x7f06a4079ab0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f06b4055000 tx=0x7f06b405b040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:17.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:17.849+0000 7f06c2bf1640 1 --2- 172.21.15.38:0/2898522883 >> [v2:172.21.15.195:6802/3401837571,v1:172.21.15.195:6803/3401837571] conn(0x5635f2151cb0 0x5635f21520c0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:17.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:17.850+0000 7f06c2bf1640 1 -- 172.21.15.38:0/2898522883 <== osd.4 v2:172.21.15.195:6802/3401837571 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (91) out=91b] v0'0 uv6 ondisk = 0) ==== 164+0+91 (crc 0 0 0) 0x7f06bc00f4c0 con 0x5635f2151cb0 2024-09-21T03:45:17.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:17.850+0000 7f06c8936f40 1 -- 172.21.15.38:0/2898522883 >> [v2:172.21.15.195:6802/3401837571,v1:172.21.15.195:6803/3401837571] conn(0x5635f2151cb0 msgr2=0x5635f21520c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:17.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:17.850+0000 7f06c8936f40 1 --2- 172.21.15.38:0/2898522883 >> [v2:172.21.15.195:6802/3401837571,v1:172.21.15.195:6803/3401837571] conn(0x5635f2151cb0 0x5635f21520c0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:17.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:17.850+0000 7f06c8936f40 1 -- 172.21.15.38:0/2898522883 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f06a40775a0 msgr2=0x7f06a4079ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:17.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:17.850+0000 7f06c8936f40 1 --2- 172.21.15.38:0/2898522883 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f06a40775a0 0x7f06a4079ab0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f06b4055000 tx=0x7f06b405b040 comp rx=0 tx=0).stop 2024-09-21T03:45:17.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:17.850+0000 7f06c8936f40 1 -- 172.21.15.38:0/2898522883 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5635f214b9b0 msgr2=0x5635f2155a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:17.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:17.850+0000 7f06c8936f40 1 --2- 172.21.15.38:0/2898522883 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5635f214b9b0 0x5635f2155a30 secure :-1 s=READY pgs=2429 cs=0 l=1 rev1=1 crypto rx=0x7f06b8018990 tx=0x7f06b80189c0 comp rx=0 tx=0).stop 2024-09-21T03:45:17.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:17.850+0000 7f06c2bf1640 1 -- 172.21.15.38:0/2898522883 reap_dead start 2024-09-21T03:45:17.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:17.850+0000 7f06c8936f40 1 -- 172.21.15.38:0/2898522883 shutdown_connections 2024-09-21T03:45:17.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:17.850+0000 7f06c8936f40 1 -- 172.21.15.38:0/2898522883 >> 172.21.15.38:0/2898522883 conn(0x5635f202bac0 msgr2=0x5635f214fda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:17.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:17.850+0000 7f06c8936f40 1 -- 172.21.15.38:0/2898522883 shutdown_connections 2024-09-21T03:45:17.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:17.850+0000 7f06c8936f40 1 -- 172.21.15.38:0/2898522883 wait complete. 2024-09-21T03:45:17.856 INFO:teuthology.orchestra.run.smithi038.stdout:Ufile1dir1 2024-09-21T03:45:17.857 DEBUG:teuthology.orchestra.run.smithi038:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2024-09-21T03:45:17.959 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-09-21T03:45:17.959 INFO:teuthology.orchestra.run.smithi038.stdout: "ino": 1099511627777, 2024-09-21T03:45:17.959 INFO:teuthology.orchestra.run.smithi038.stdout: "ancestors": [ 2024-09-21T03:45:17.959 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-21T03:45:17.959 INFO:teuthology.orchestra.run.smithi038.stdout: "dirino": 1099511627776, 2024-09-21T03:45:17.959 INFO:teuthology.orchestra.run.smithi038.stdout: "dname": "file1", 2024-09-21T03:45:17.959 INFO:teuthology.orchestra.run.smithi038.stdout: "version": 8 2024-09-21T03:45:17.959 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-21T03:45:17.959 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-21T03:45:17.959 INFO:teuthology.orchestra.run.smithi038.stdout: "dirino": 1, 2024-09-21T03:45:17.959 INFO:teuthology.orchestra.run.smithi038.stdout: "dname": "dir1", 2024-09-21T03:45:17.960 INFO:teuthology.orchestra.run.smithi038.stdout: "version": 29 2024-09-21T03:45:17.960 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-21T03:45:17.960 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-21T03:45:17.960 INFO:teuthology.orchestra.run.smithi038.stdout: "pool": 16, 2024-09-21T03:45:17.960 INFO:teuthology.orchestra.run.smithi038.stdout: "old_pools": [] 2024-09-21T03:45:17.960 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-09-21T03:45:17.960 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:45:17.976 DEBUG:teuthology.orchestra.run.smithi038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data setxattr 10000000001.00000000 parent rhubarb 2024-09-21T03:45:18.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.033+0000 7ff04b321f40 1 Processor -- start 2024-09-21T03:45:18.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.033+0000 7ff04b321f40 1 -- start start 2024-09-21T03:45:18.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.034+0000 7ff04b321f40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f8f189cd00 0x55f8f19f9af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:18.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.034+0000 7ff04b321f40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f8f1957950 0x55f8f1898900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:18.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.034+0000 7ff04b321f40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f8f1972670 0x55f8f1898e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:18.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.034+0000 7ff04b321f40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55f8f17ead60 con 0x55f8f189cd00 2024-09-21T03:45:18.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.034+0000 7ff04b321f40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55f8f1849430 con 0x55f8f1972670 2024-09-21T03:45:18.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.034+0000 7ff04b321f40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55f8f18e47b0 con 0x55f8f1957950 2024-09-21T03:45:18.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.034+0000 7ff044362640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f8f189cd00 0x55f8f19f9af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:18.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.034+0000 7ff044362640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f8f189cd00 0x55f8f19f9af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46432/0 (socket says 172.21.15.38:46432) 2024-09-21T03:45:18.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.034+0000 7ff044362640 1 -- 172.21.15.38:0/2590992129 learned_addr learned my addr 172.21.15.38:0/2590992129 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:45:18.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.035+0000 7ff044362640 1 -- 172.21.15.38:0/2590992129 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f8f1957950 msgr2=0x55f8f1898900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:18.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.035+0000 7ff044362640 1 --2- 172.21.15.38:0/2590992129 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f8f1957950 0x55f8f1898900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:18.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.035+0000 7ff044b63640 1 --2- 172.21.15.38:0/2590992129 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f8f1972670 0x55f8f1898e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:18.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.035+0000 7ff044362640 1 -- 172.21.15.38:0/2590992129 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f8f1972670 msgr2=0x55f8f1898e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:18.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.035+0000 7ff044362640 1 --2- 172.21.15.38:0/2590992129 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f8f1972670 0x55f8f1898e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:18.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.035+0000 7ff044362640 1 -- 172.21.15.38:0/2590992129 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f8f18c94c0 con 0x55f8f189cd00 2024-09-21T03:45:18.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.035+0000 7ff044b63640 1 --2- 172.21.15.38:0/2590992129 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f8f1972670 0x55f8f1898e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:45:18.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.035+0000 7ff044362640 1 --2- 172.21.15.38:0/2590992129 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f8f189cd00 0x55f8f19f9af0 secure :-1 s=READY pgs=2431 cs=0 l=1 rev1=1 crypto rx=0x7ff034000f80 tx=0x7ff034003330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1010d4cc41c27abf server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:18.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.035+0000 7ff042b5f640 1 -- 172.21.15.38:0/2590992129 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff034064020 con 0x55f8f189cd00 2024-09-21T03:45:18.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.035+0000 7ff042b5f640 1 -- 172.21.15.38:0/2590992129 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff03404d850 con 0x55f8f189cd00 2024-09-21T03:45:18.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.035+0000 7ff042b5f640 1 -- 172.21.15.38:0/2590992129 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff03404dcc0 con 0x55f8f189cd00 2024-09-21T03:45:18.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.036+0000 7ff04b321f40 1 -- 172.21.15.38:0/2590992129 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f8f189cd00 msgr2=0x55f8f19f9af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:18.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.036+0000 7ff04b321f40 1 --2- 172.21.15.38:0/2590992129 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f8f189cd00 0x55f8f19f9af0 secure :-1 s=READY pgs=2431 cs=0 l=1 rev1=1 crypto rx=0x7ff034000f80 tx=0x7ff034003330 comp rx=0 tx=0).stop 2024-09-21T03:45:18.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.036+0000 7ff04b321f40 1 -- 172.21.15.38:0/2590992129 shutdown_connections 2024-09-21T03:45:18.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.036+0000 7ff04b321f40 1 --2- 172.21.15.38:0/2590992129 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f8f1972670 0x55f8f1898e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:18.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.036+0000 7ff04b321f40 1 --2- 172.21.15.38:0/2590992129 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f8f1957950 0x55f8f1898900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:18.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.036+0000 7ff04b321f40 1 --2- 172.21.15.38:0/2590992129 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f8f189cd00 0x55f8f19f9af0 unknown :-1 s=CLOSED pgs=2431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:18.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.036+0000 7ff04b321f40 1 -- 172.21.15.38:0/2590992129 >> 172.21.15.38:0/2590992129 conn(0x55f8f17bbac0 msgr2=0x55f8f1972a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:18.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.036+0000 7ff04b321f40 1 -- 172.21.15.38:0/2590992129 shutdown_connections 2024-09-21T03:45:18.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.036+0000 7ff04b321f40 1 -- 172.21.15.38:0/2590992129 wait complete. 2024-09-21T03:45:18.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.037+0000 7ff04b321f40 1 Processor -- start 2024-09-21T03:45:18.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.037+0000 7ff04b321f40 1 -- start start 2024-09-21T03:45:18.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.038+0000 7ff04b321f40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f8f189cd00 0x55f8f196e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:18.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.038+0000 7ff04b321f40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f8f1957950 0x55f8f19fb450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:18.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.038+0000 7ff04b321f40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f8f1972670 0x55f8f19fb990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:18.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.038+0000 7ff04b321f40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55f8f18c9c40 con 0x55f8f1957950 2024-09-21T03:45:18.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.038+0000 7ff04b321f40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55f8f1954fc0 con 0x55f8f1972670 2024-09-21T03:45:18.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.038+0000 7ff04b321f40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55f8f18ca240 con 0x55f8f189cd00 2024-09-21T03:45:18.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.038+0000 7ff043b61640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f8f1957950 0x55f8f19fb450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:18.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.038+0000 7ff043b61640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f8f1957950 0x55f8f19fb450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46438/0 (socket says 172.21.15.38:46438) 2024-09-21T03:45:18.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.038+0000 7ff043b61640 1 -- 172.21.15.38:0/2418456465 learned_addr learned my addr 172.21.15.38:0/2418456465 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:45:18.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.038+0000 7ff044b63640 1 --2- 172.21.15.38:0/2418456465 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f8f1972670 0x55f8f19fb990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:18.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.038+0000 7ff044362640 1 --2- 172.21.15.38:0/2418456465 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f8f189cd00 0x55f8f196e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:18.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.039+0000 7ff043b61640 1 -- 172.21.15.38:0/2418456465 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f8f189cd00 msgr2=0x55f8f196e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:18.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.039+0000 7ff043b61640 1 --2- 172.21.15.38:0/2418456465 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f8f189cd00 0x55f8f196e470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:18.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.039+0000 7ff043b61640 1 -- 172.21.15.38:0/2418456465 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f8f1972670 msgr2=0x55f8f19fb990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:18.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.039+0000 7ff043b61640 1 --2- 172.21.15.38:0/2418456465 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f8f1972670 0x55f8f19fb990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:18.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.039+0000 7ff043b61640 1 -- 172.21.15.38:0/2418456465 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f8f19fbed0 con 0x55f8f1957950 2024-09-21T03:45:18.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.039+0000 7ff043b61640 1 --2- 172.21.15.38:0/2418456465 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f8f1957950 0x55f8f19fb450 secure :-1 s=READY pgs=2432 cs=0 l=1 rev1=1 crypto rx=0x7ff03800a6e0 tx=0x7ff03800aba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:18.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.039+0000 7ff041b5d640 1 -- 172.21.15.38:0/2418456465 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff038009920 con 0x55f8f1957950 2024-09-21T03:45:18.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.039+0000 7ff041b5d640 1 -- 172.21.15.38:0/2418456465 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff038009ac0 con 0x55f8f1957950 2024-09-21T03:45:18.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.039+0000 7ff041b5d640 1 -- 172.21.15.38:0/2418456465 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff038008450 con 0x55f8f1957950 2024-09-21T03:45:18.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.039+0000 7ff04b321f40 1 -- 172.21.15.38:0/2418456465 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55f8f19fa690 con 0x55f8f1957950 2024-09-21T03:45:18.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.039+0000 7ff04b321f40 1 -- 172.21.15.38:0/2418456465 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f8f19fac50 con 0x55f8f1957950 2024-09-21T03:45:18.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.041+0000 7ff041b5d640 1 -- 172.21.15.38:0/2418456465 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ff0380085f0 con 0x55f8f1957950 2024-09-21T03:45:18.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.041+0000 7ff041b5d640 1 --2- 172.21.15.38:0/2418456465 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff030077570 0x7ff030079a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:18.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.041+0000 7ff041b5d640 1 -- 172.21.15.38:0/2418456465 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 6303+0+0 (secure 0 0 0) 0x7ff03809a240 con 0x55f8f1957950 2024-09-21T03:45:18.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.042+0000 7ff044362640 1 --2- 172.21.15.38:0/2418456465 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff030077570 0x7ff030079a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:18.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.042+0000 7ff04b321f40 1 --2- 172.21.15.38:0/2418456465 >> [v2:172.21.15.195:6802/3401837571,v1:172.21.15.195:6803/3401837571] conn(0x55f8f18df490 0x55f8f1a16830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:18.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.042+0000 7ff04b321f40 1 -- 172.21.15.38:0/2418456465 --> [v2:172.21.15.195:6802/3401837571,v1:172.21.15.195:6803/3401837571] -- osd_op(unknown.0.0:1 16.7 16:e2e07930:::10000000001.00000000:head [setxattr parent (7) in=13b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148) -- 0x55f8f1a16d70 con 0x55f8f18df490 2024-09-21T03:45:18.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.042+0000 7ff044b63640 1 --2- 172.21.15.38:0/2418456465 >> [v2:172.21.15.195:6802/3401837571,v1:172.21.15.195:6803/3401837571] conn(0x55f8f18df490 0x55f8f1a16830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:18.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.042+0000 7ff044362640 1 --2- 172.21.15.38:0/2418456465 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff030077570 0x7ff030079a80 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7ff034062000 tx=0x7ff034004c00 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:18.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.042+0000 7ff044b63640 1 --2- 172.21.15.38:0/2418456465 >> [v2:172.21.15.195:6802/3401837571,v1:172.21.15.195:6803/3401837571] conn(0x55f8f18df490 0x55f8f1a16830 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:18.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.044+0000 7ff044b63640 1 -- 172.21.15.38:0/2418456465 <== osd.4 v2:172.21.15.195:6802/3401837571 1 ==== osd_op_reply(1 10000000001.00000000 [setxattr (7)] v148'7 uv7 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7ff03c00f680 con 0x55f8f18df490 2024-09-21T03:45:18.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.044+0000 7ff04b321f40 1 -- 172.21.15.38:0/2418456465 >> [v2:172.21.15.195:6802/3401837571,v1:172.21.15.195:6803/3401837571] conn(0x55f8f18df490 msgr2=0x55f8f1a16830 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:18.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.044+0000 7ff04b321f40 1 --2- 172.21.15.38:0/2418456465 >> [v2:172.21.15.195:6802/3401837571,v1:172.21.15.195:6803/3401837571] conn(0x55f8f18df490 0x55f8f1a16830 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:18.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.044+0000 7ff04b321f40 1 -- 172.21.15.38:0/2418456465 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff030077570 msgr2=0x7ff030079a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:18.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.044+0000 7ff04b321f40 1 --2- 172.21.15.38:0/2418456465 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff030077570 0x7ff030079a80 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7ff034062000 tx=0x7ff034004c00 comp rx=0 tx=0).stop 2024-09-21T03:45:18.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.044+0000 7ff04b321f40 1 -- 172.21.15.38:0/2418456465 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f8f1957950 msgr2=0x55f8f19fb450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:18.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.044+0000 7ff04b321f40 1 --2- 172.21.15.38:0/2418456465 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f8f1957950 0x55f8f19fb450 secure :-1 s=READY pgs=2432 cs=0 l=1 rev1=1 crypto rx=0x7ff03800a6e0 tx=0x7ff03800aba0 comp rx=0 tx=0).stop 2024-09-21T03:45:18.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.044+0000 7ff044b63640 1 -- 172.21.15.38:0/2418456465 reap_dead start 2024-09-21T03:45:18.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.044+0000 7ff04b321f40 1 -- 172.21.15.38:0/2418456465 shutdown_connections 2024-09-21T03:45:18.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.044+0000 7ff04b321f40 1 -- 172.21.15.38:0/2418456465 >> 172.21.15.38:0/2418456465 conn(0x55f8f17bbac0 msgr2=0x55f8f19f9e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:18.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.044+0000 7ff04b321f40 1 -- 172.21.15.38:0/2418456465 shutdown_connections 2024-09-21T03:45:18.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.044+0000 7ff04b321f40 1 -- 172.21.15.38:0/2418456465 wait complete. 2024-09-21T03:45:18.049 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:45:18.049 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat testdir/hardlink1) 2024-09-21T03:45:18.066 INFO:tasks.ceph.mds.c.smithi038.stderr:2024-09-21T03:45:18.064+0000 7f6c65fd1640 -1 mds.0.cache corrupt backtrace on ino x00x10000000001: Decoder at 'void inode_backtrace_t::decode(ceph::buffer::v15_2_0::list::const_iterator&)' v=5 cannot decode v=114 minimal_decoder=104: Malformed input [buffer:3] 2024-09-21T03:45:18.066 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T03:45:18.067 INFO:teuthology.orchestra.run.smithi038.stderr:stat: cannot statx 'testdir/hardlink1': Input/output error 2024-09-21T03:45:18.067 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:45:18.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.225+0000 7fee06950640 1 -- 172.21.15.38:0/1159981624 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fede8004000 msgr2=0x7fede8004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:18.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.225+0000 7fee06950640 1 --2- 172.21.15.38:0/1159981624 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fede8004000 0x7fede8004800 secure :-1 s=READY pgs=2434 cs=0 l=1 rev1=1 crypto rx=0x7fedf4009b20 tx=0x7fede8005830 comp rx=0 tx=0).stop 2024-09-21T03:45:18.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.225+0000 7fee04eed640 1 -- 172.21.15.38:0/1159981624 reap_dead start 2024-09-21T03:45:18.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.225+0000 7fee06950640 1 -- 172.21.15.38:0/1159981624 shutdown_connections 2024-09-21T03:45:18.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.225+0000 7fee06950640 1 -- 172.21.15.38:0/1159981624 >> 172.21.15.38:0/1159981624 conn(0x7fee000b6b20 msgr2=0x7fee000d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:18.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.225+0000 7fee06950640 1 -- 172.21.15.38:0/1159981624 shutdown_connections 2024-09-21T03:45:18.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.225+0000 7fee06950640 1 -- 172.21.15.38:0/1159981624 wait complete. 2024-09-21T03:45:18.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.227+0000 7fee06950640 1 Processor -- start 2024-09-21T03:45:18.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.227+0000 7fee06950640 1 -- start start 2024-09-21T03:45:18.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.228+0000 7fee06950640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fede8004000 0x7fee000ab6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:18.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.228+0000 7fee06950640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fee000b2370 0x7fee001f8520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:18.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.228+0000 7fee06950640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fee000b2780 0x7fee001f8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:18.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.228+0000 7fee06950640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fee00162810 con 0x7fede8004000 2024-09-21T03:45:18.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.229+0000 7fee06950640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fee00162690 con 0x7fee000b2370 2024-09-21T03:45:18.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.229+0000 7fee06950640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fee00162990 con 0x7fee000b2780 2024-09-21T03:45:18.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.229+0000 7fedfffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fede8004000 0x7fee000ab6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:18.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.229+0000 7fedfffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fede8004000 0x7fee000ab6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46460/0 (socket says 172.21.15.38:46460) 2024-09-21T03:45:18.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.229+0000 7fedfffff640 1 -- 172.21.15.38:0/877801244 learned_addr learned my addr 172.21.15.38:0/877801244 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:45:18.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.229+0000 7fedff7fe640 1 --2- 172.21.15.38:0/877801244 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fee000b2370 0x7fee001f8520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:18.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.229+0000 7fee04eed640 1 --2- 172.21.15.38:0/877801244 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fee000b2780 0x7fee001f8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:18.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.229+0000 7fedfffff640 1 -- 172.21.15.38:0/877801244 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fee000b2780 msgr2=0x7fee001f8a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:18.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.229+0000 7fedfffff640 1 --2- 172.21.15.38:0/877801244 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fee000b2780 0x7fee001f8a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:18.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.229+0000 7fedfffff640 1 -- 172.21.15.38:0/877801244 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fee000b2370 msgr2=0x7fee001f8520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:18.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.229+0000 7fedfffff640 1 --2- 172.21.15.38:0/877801244 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fee000b2370 0x7fee001f8520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:18.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.229+0000 7fedfffff640 1 -- 172.21.15.38:0/877801244 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee000abc00 con 0x7fede8004000 2024-09-21T03:45:18.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.229+0000 7fedfffff640 1 --2- 172.21.15.38:0/877801244 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fede8004000 0x7fee000ab6c0 secure :-1 s=READY pgs=2435 cs=0 l=1 rev1=1 crypto rx=0x7fedf0006e00 tx=0x7fedf00049d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:18.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.229+0000 7fedfcff9640 1 -- 172.21.15.38:0/877801244 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedf0015020 con 0x7fede8004000 2024-09-21T03:45:18.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.229+0000 7fedfcff9640 1 -- 172.21.15.38:0/877801244 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fedf00119a0 con 0x7fede8004000 2024-09-21T03:45:18.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.229+0000 7fee06950640 1 -- 172.21.15.38:0/877801244 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee000cbf60 con 0x7fede8004000 2024-09-21T03:45:18.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.229+0000 7fedfcff9640 1 -- 172.21.15.38:0/877801244 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedf0011df0 con 0x7fede8004000 2024-09-21T03:45:18.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.229+0000 7fee06950640 1 -- 172.21.15.38:0/877801244 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee000cc520 con 0x7fede8004000 2024-09-21T03:45:18.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.230+0000 7fee06950640 1 -- 172.21.15.38:0/877801244 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee00162f00 con 0x7fede8004000 2024-09-21T03:45:18.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.231+0000 7fedfcff9640 1 -- 172.21.15.38:0/877801244 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fedf0018020 con 0x7fede8004000 2024-09-21T03:45:18.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.232+0000 7fedfcff9640 1 --2- 172.21.15.38:0/877801244 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fedec078b90 0x7fedec07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:18.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.232+0000 7fedfcff9640 1 -- 172.21.15.38:0/877801244 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 6303+0+0 (secure 0 0 0) 0x7fedf009b7a0 con 0x7fede8004000 2024-09-21T03:45:18.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.232+0000 7fedff7fe640 1 --2- 172.21.15.38:0/877801244 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fedec078b90 0x7fedec07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:18.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.233+0000 7fedff7fe640 1 --2- 172.21.15.38:0/877801244 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fedec078b90 0x7fedec07b0a0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7fedf4035000 tx=0x7fedf40046c0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:18.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.237+0000 7fedfcff9640 1 -- 172.21.15.38:0/877801244 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fedf001f030 con 0x7fede8004000 2024-09-21T03:45:18.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.461+0000 7fee06950640 1 -- 172.21.15.38:0/877801244 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fee0015f990 con 0x7fede8004000 2024-09-21T03:45:18.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.462+0000 7fedfcff9640 1 -- 172.21.15.38:0/877801244 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 349 v349) ==== 95+0+5241 (secure 0 0 0) 0x7fee000abc00 con 0x7fede8004000 2024-09-21T03:45:18.464 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:45:18.464 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":349,"btime":"2024-09-21T03:45:15:115320+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8399,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/3606957623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":3606957623},{"type":"v1","addr":"172.21.15.38:6835","nonce":3606957623}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":338},{"gid":8405,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/456651486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":456651486},{"type":"v1","addr":"172.21.15.195:6837","nonce":456651486}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":338},{"gid":9604,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/2649311840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":2649311840},{"type":"v1","addr":"172.21.15.195:6835","nonce":2649311840}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":338}],"filesystems":[{"mdsmap":{"epoch":349,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:44:56.710446+0000","modified":"2024-09-21T03:45:15.115308+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":147,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10935},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10935":{"gid":10935,"name":"c","rank":0,"incarnation":346,"state":"up:active","state_seq":4,"addr":"172.21.15.38:6833/3839260519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":3839260519},{"type":"v1","addr":"172.21.15.38:6833","nonce":3839260519}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10935,"qdb_cluster":[10935]},"id":7}]} 2024-09-21T03:45:18.464 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 349 2024-09-21T03:45:18.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.464+0000 7fee06950640 1 -- 172.21.15.38:0/877801244 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fedec078b90 msgr2=0x7fedec07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:18.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.464+0000 7fee06950640 1 --2- 172.21.15.38:0/877801244 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fedec078b90 0x7fedec07b0a0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7fedf4035000 tx=0x7fedf40046c0 comp rx=0 tx=0).stop 2024-09-21T03:45:18.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.464+0000 7fee06950640 1 -- 172.21.15.38:0/877801244 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fede8004000 msgr2=0x7fee000ab6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:18.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.464+0000 7fee06950640 1 --2- 172.21.15.38:0/877801244 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fede8004000 0x7fee000ab6c0 secure :-1 s=READY pgs=2435 cs=0 l=1 rev1=1 crypto rx=0x7fedf0006e00 tx=0x7fedf00049d0 comp rx=0 tx=0).stop 2024-09-21T03:45:18.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.464+0000 7fee06950640 1 -- 172.21.15.38:0/877801244 shutdown_connections 2024-09-21T03:45:18.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.464+0000 7fee06950640 1 --2- 172.21.15.38:0/877801244 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fedec078b90 0x7fedec07b0a0 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:18.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.464+0000 7fee06950640 1 --2- 172.21.15.38:0/877801244 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fee000b2780 0x7fee001f8a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:18.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.464+0000 7fee06950640 1 --2- 172.21.15.38:0/877801244 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fee000b2370 0x7fee001f8520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:18.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.464+0000 7fee06950640 1 --2- 172.21.15.38:0/877801244 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fede8004000 0x7fee000ab6c0 unknown :-1 s=CLOSED pgs=2435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:18.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.464+0000 7fee06950640 1 -- 172.21.15.38:0/877801244 >> 172.21.15.38:0/877801244 conn(0x7fee000b6b20 msgr2=0x7fee000b3be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:18.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.464+0000 7fee06950640 1 -- 172.21.15.38:0/877801244 shutdown_connections 2024-09-21T03:45:18.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.464+0000 7fee06950640 1 -- 172.21.15.38:0/877801244 wait complete. 2024-09-21T03:45:18.478 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c damage ls --format=json-pretty 2024-09-21T03:45:18.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.595+0000 7f66373eb640 1 -- 172.21.15.38:0/2697355622 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f661c003cb0 msgr2=0x7f661c004100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:18.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.595+0000 7f66373eb640 1 --2- 172.21.15.38:0/2697355622 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f661c003cb0 0x7f661c004100 secure :-1 s=READY pgs=2437 cs=0 l=1 rev1=1 crypto rx=0x7f662c00c310 tx=0x7f661c005830 comp rx=0 tx=0).stop 2024-09-21T03:45:18.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.595+0000 7f6635988640 1 -- 172.21.15.38:0/2697355622 reap_dead start 2024-09-21T03:45:18.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.595+0000 7f66373eb640 1 -- 172.21.15.38:0/2697355622 shutdown_connections 2024-09-21T03:45:18.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.595+0000 7f66373eb640 1 -- 172.21.15.38:0/2697355622 >> 172.21.15.38:0/2697355622 conn(0x7f663009d830 msgr2=0x7f66300b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:18.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.595+0000 7f66373eb640 1 -- 172.21.15.38:0/2697355622 shutdown_connections 2024-09-21T03:45:18.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.595+0000 7f66373eb640 1 -- 172.21.15.38:0/2697355622 wait complete. 2024-09-21T03:45:18.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.597+0000 7f66373eb640 1 Processor -- start 2024-09-21T03:45:18.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.597+0000 7f66373eb640 1 -- start start 2024-09-21T03:45:18.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.598+0000 7f66373eb640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f661c003cb0 0x7f663015a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:18.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.598+0000 7f66373eb640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f663014c2a0 0x7f66300b2660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:18.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.598+0000 7f66373eb640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f66300b2ba0 0x7f66301f8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:18.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.598+0000 7f66373eb640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f66300d0f70 con 0x7f663014c2a0 2024-09-21T03:45:18.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.598+0000 7f66373eb640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f66300d0df0 con 0x7f661c003cb0 2024-09-21T03:45:18.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.598+0000 7f66373eb640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f66300cf4f0 con 0x7f66300b2ba0 2024-09-21T03:45:18.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.598+0000 7f6634986640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f663014c2a0 0x7f66300b2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:18.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.598+0000 7f6634986640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f663014c2a0 0x7f66300b2660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46480/0 (socket says 172.21.15.38:46480) 2024-09-21T03:45:18.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.598+0000 7f6634986640 1 -- 172.21.15.38:0/552804698 learned_addr learned my addr 172.21.15.38:0/552804698 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:45:18.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.598+0000 7f6635187640 1 --2- 172.21.15.38:0/552804698 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f661c003cb0 0x7f663015a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:18.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.598+0000 7f6635988640 1 --2- 172.21.15.38:0/552804698 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f66300b2ba0 0x7f66301f8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:18.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.599+0000 7f6634986640 1 -- 172.21.15.38:0/552804698 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f66300b2ba0 msgr2=0x7f66301f8b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:18.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.599+0000 7f6634986640 1 --2- 172.21.15.38:0/552804698 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f66300b2ba0 0x7f66301f8b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:18.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.599+0000 7f6634986640 1 -- 172.21.15.38:0/552804698 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f661c003cb0 msgr2=0x7f663015a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:18.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.599+0000 7f6634986640 1 --2- 172.21.15.38:0/552804698 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f661c003cb0 0x7f663015a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:18.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.599+0000 7f6634986640 1 -- 172.21.15.38:0/552804698 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66300b2fb0 con 0x7f663014c2a0 2024-09-21T03:45:18.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.599+0000 7f6634986640 1 --2- 172.21.15.38:0/552804698 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f663014c2a0 0x7f66300b2660 secure :-1 s=READY pgs=2438 cs=0 l=1 rev1=1 crypto rx=0x7f6620011ab0 tx=0x7f6620011f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:18.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.599+0000 7f6625ffb640 1 -- 172.21.15.38:0/552804698 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f662001e070 con 0x7f663014c2a0 2024-09-21T03:45:18.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.599+0000 7f6625ffb640 1 -- 172.21.15.38:0/552804698 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f662000fc70 con 0x7f663014c2a0 2024-09-21T03:45:18.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.599+0000 7f66373eb640 1 -- 172.21.15.38:0/552804698 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f663006d2f0 con 0x7f663014c2a0 2024-09-21T03:45:18.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.599+0000 7f6625ffb640 1 -- 172.21.15.38:0/552804698 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6620019410 con 0x7f663014c2a0 2024-09-21T03:45:18.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.599+0000 7f66373eb640 1 -- 172.21.15.38:0/552804698 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f663006d8b0 con 0x7f663014c2a0 2024-09-21T03:45:18.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.601+0000 7f6625ffb640 1 -- 172.21.15.38:0/552804698 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f6620021070 con 0x7f663014c2a0 2024-09-21T03:45:18.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.601+0000 7f6625ffb640 1 --2- 172.21.15.38:0/552804698 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6614078bc0 0x7f661407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:18.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.602+0000 7f6625ffb640 1 -- 172.21.15.38:0/552804698 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 6303+0+0 (secure 0 0 0) 0x7f662009b740 con 0x7f663014c2a0 2024-09-21T03:45:18.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.602+0000 7f6635187640 1 --2- 172.21.15.38:0/552804698 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6614078bc0 0x7f661407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:18.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.602+0000 7f663e438740 1 Processor -- start 2024-09-21T03:45:18.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.602+0000 7f663e438740 1 -- start start 2024-09-21T03:45:18.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.602+0000 7f6635187640 1 --2- 172.21.15.38:0/552804698 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6614078bc0 0x7f661407b0d0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f662c033000 tx=0x7f662c001a20 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:18.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.604+0000 7f663e438740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5628a7b6c350 0x5628a7f3dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:18.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.604+0000 7f663e438740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5628a7b43490 0x5628a7f3e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:18.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.604+0000 7f663e438740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5628a7b4b670 0x5628a7f5b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:18.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.604+0000 7f663e438740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5628a7e869a0 con 0x5628a7b6c350 2024-09-21T03:45:18.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.604+0000 7f663e438740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5628a7e75dd0 con 0x5628a7b43490 2024-09-21T03:45:18.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.604+0000 7f663e438740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x5628a7e79be0 con 0x5628a7b4b670 2024-09-21T03:45:18.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.604+0000 7f6635187640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5628a7b6c350 0x5628a7f3dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:18.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.604+0000 7f6635187640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5628a7b6c350 0x5628a7f3dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46494/0 (socket says 172.21.15.38:46494) 2024-09-21T03:45:18.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.604+0000 7f6635187640 1 -- 172.21.15.38:0/1483107636 learned_addr learned my addr 172.21.15.38:0/1483107636 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:45:18.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.604+0000 7f6634986640 1 --2- 172.21.15.38:0/1483107636 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5628a7b43490 0x5628a7f3e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:18.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.604+0000 7f6635988640 1 --2- 172.21.15.38:0/1483107636 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5628a7b4b670 0x5628a7f5b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:18.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.604+0000 7f6635187640 1 -- 172.21.15.38:0/1483107636 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5628a7b4b670 msgr2=0x5628a7f5b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:18.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.604+0000 7f6635187640 1 --2- 172.21.15.38:0/1483107636 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5628a7b4b670 0x5628a7f5b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:18.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.604+0000 7f6635187640 1 -- 172.21.15.38:0/1483107636 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5628a7b43490 msgr2=0x5628a7f3e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:18.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.604+0000 7f6635187640 1 --2- 172.21.15.38:0/1483107636 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5628a7b43490 0x5628a7f3e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:18.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.604+0000 7f6635187640 1 -- 172.21.15.38:0/1483107636 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5628a7e0e700 con 0x5628a7b6c350 2024-09-21T03:45:18.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.604+0000 7f6635187640 1 --2- 172.21.15.38:0/1483107636 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5628a7b6c350 0x5628a7f3dd80 secure :-1 s=READY pgs=2439 cs=0 l=1 rev1=1 crypto rx=0x7f662c02b100 tx=0x7f662c03d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ad768a57cd14d86 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:18.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.604+0000 7f661affd640 1 -- 172.21.15.38:0/1483107636 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f662c031040 con 0x5628a7b6c350 2024-09-21T03:45:18.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.604+0000 7f661affd640 1 -- 172.21.15.38:0/1483107636 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f662c0035f0 con 0x5628a7b6c350 2024-09-21T03:45:18.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.604+0000 7f661affd640 1 -- 172.21.15.38:0/1483107636 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f662c003a20 con 0x5628a7b6c350 2024-09-21T03:45:18.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.605+0000 7f663e438740 1 -- 172.21.15.38:0/1483107636 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5628a7b6c350 msgr2=0x5628a7f3dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:18.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.605+0000 7f663e438740 1 --2- 172.21.15.38:0/1483107636 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5628a7b6c350 0x5628a7f3dd80 secure :-1 s=READY pgs=2439 cs=0 l=1 rev1=1 crypto rx=0x7f662c02b100 tx=0x7f662c03d000 comp rx=0 tx=0).stop 2024-09-21T03:45:18.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.605+0000 7f663e438740 1 -- 172.21.15.38:0/1483107636 shutdown_connections 2024-09-21T03:45:18.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.605+0000 7f663e438740 1 --2- 172.21.15.38:0/1483107636 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5628a7b4b670 0x5628a7f5b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:18.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.605+0000 7f663e438740 1 --2- 172.21.15.38:0/1483107636 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5628a7b43490 0x5628a7f3e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:18.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.605+0000 7f663e438740 1 --2- 172.21.15.38:0/1483107636 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5628a7b6c350 0x5628a7f3dd80 unknown :-1 s=CLOSED pgs=2439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:18.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.605+0000 7f663e438740 1 -- 172.21.15.38:0/1483107636 >> 172.21.15.38:0/1483107636 conn(0x5628a7b604a0 msgr2=0x5628a7f14040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:18.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.605+0000 7f663e438740 1 -- 172.21.15.38:0/1483107636 shutdown_connections 2024-09-21T03:45:18.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.605+0000 7f663e438740 1 -- 172.21.15.38:0/1483107636 wait complete. 2024-09-21T03:45:18.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.606+0000 7f663e438740 1 -- start start 2024-09-21T03:45:18.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.606+0000 7f663e438740 1 Processor -- start 2024-09-21T03:45:18.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.608+0000 7f663e438740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5628a7b43490 0x5628a7f688b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:18.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.608+0000 7f663e438740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5628a7b4b670 0x5628a7f51710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:18.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.608+0000 7f663e438740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5628a7b6c350 0x5628a7f591e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:18.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.608+0000 7f663e438740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5628a7be6700 con 0x5628a7b6c350 2024-09-21T03:45:18.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.608+0000 7f663e438740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5628a7e1b340 con 0x5628a7b4b670 2024-09-21T03:45:18.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.608+0000 7f663e438740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x5628a7f3e800 con 0x5628a7b43490 2024-09-21T03:45:18.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.608+0000 7f663e438740 10 client.0 _ll_register_callbacks cb 0x5628a7e19ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:45:18.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.608+0000 7f6635988640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5628a7b6c350 0x5628a7f591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:18.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.608+0000 7f6635988640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5628a7b6c350 0x5628a7f591e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46498/0 (socket says 172.21.15.38:46498) 2024-09-21T03:45:18.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.608+0000 7f6635988640 1 -- 172.21.15.38:0/3626972498 learned_addr learned my addr 172.21.15.38:0/3626972498 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:45:18.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.608+0000 7f6635187640 1 --2- 172.21.15.38:0/3626972498 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5628a7b43490 0x5628a7f688b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:18.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.608+0000 7f6634986640 1 --2- 172.21.15.38:0/3626972498 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5628a7b4b670 0x5628a7f51710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:18.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.608+0000 7f6635988640 1 -- 172.21.15.38:0/3626972498 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5628a7b43490 msgr2=0x5628a7f688b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:18.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.608+0000 7f6635988640 1 --2- 172.21.15.38:0/3626972498 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5628a7b43490 0x5628a7f688b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:18.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.608+0000 7f6635988640 1 -- 172.21.15.38:0/3626972498 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5628a7b4b670 msgr2=0x5628a7f51710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:18.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.608+0000 7f6635988640 1 --2- 172.21.15.38:0/3626972498 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5628a7b4b670 0x5628a7f51710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:18.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.608+0000 7f6635988640 1 -- 172.21.15.38:0/3626972498 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5628a7c507c0 con 0x5628a7b6c350 2024-09-21T03:45:18.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.608+0000 7f6635988640 1 --2- 172.21.15.38:0/3626972498 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5628a7b6c350 0x5628a7f591e0 secure :-1 s=READY pgs=2440 cs=0 l=1 rev1=1 crypto rx=0x7f663008cbe0 tx=0x7f6628013820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=503ad1da4a64d18e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:18.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.608+0000 7f660ffff640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-21T03:45:18.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.609+0000 7f660ffff640 1 -- 172.21.15.38:0/3626972498 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6628003e50 con 0x5628a7b6c350 2024-09-21T03:45:18.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.609+0000 7f660ffff640 1 -- 172.21.15.38:0/3626972498 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(149..149 src has 1..149) ==== 6303+0+0 (secure 0 0 0) 0x7f6628017a50 con 0x5628a7b6c350 2024-09-21T03:45:18.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.609+0000 7f663e438740 1 -- 172.21.15.38:0/3626972498 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f662c0035f0 con 0x5628a7b6c350 2024-09-21T03:45:18.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.609+0000 7f660ffff640 1 -- 172.21.15.38:0/3626972498 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=349) ==== 24+0+0 (secure 0 0 0) 0x7f6628017c50 con 0x5628a7b6c350 2024-09-21T03:45:18.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.609+0000 7f663e438740 10 client.11034 fetch_fsmap learned FSMap version 349 2024-09-21T03:45:18.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.609+0000 7f663e438740 1 -- 172.21.15.38:0/3626972498 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=349,monmap=2+}) -- 0x5628a7cb0f50 con 0x5628a7b6c350 2024-09-21T03:45:18.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.609+0000 7f660ffff640 1 -- 172.21.15.38:0/3626972498 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f662c0035f0 con 0x5628a7b6c350 2024-09-21T03:45:18.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.609+0000 7f660ffff640 1 -- 172.21.15.38:0/3626972498 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 349) ==== 3502+0+0 (secure 0 0 0) 0x7f6628006170 con 0x5628a7b6c350 2024-09-21T03:45:18.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.609+0000 7f663e438740 10 client.11034 fetch_fsmap finished waiting for FSMap version 349 2024-09-21T03:45:18.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.609+0000 7f663e438740 10 client.11034 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T03:45:18.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.609+0000 7f663e438740 20 client.11034 populate_metadata read hostname 'smithi038' 2024-09-21T03:45:18.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.609+0000 7f663e438740 1 --2- 172.21.15.38:0/3626972498 >> [v2:172.21.15.38:6832/3839260519,v1:172.21.15.38:6833/3839260519] conn(0x5628a7b2b130 0x5628a7f2bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:18.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.609+0000 7f663e438740 4 client.11034 mds_command: new command op to 10935 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:45:18.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.609+0000 7f663e438740 1 -- 172.21.15.38:0/3626972498 --> [v2:172.21.15.38:6832/3839260519,v1:172.21.15.38:6833/3839260519] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5628a7e888f0 con 0x5628a7b2b130 2024-09-21T03:45:18.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.609+0000 7f6635187640 1 --2- 172.21.15.38:0/3626972498 >> [v2:172.21.15.38:6832/3839260519,v1:172.21.15.38:6833/3839260519] conn(0x5628a7b2b130 0x5628a7f2bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:18.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.610+0000 7f660ffff640 10 client.11034 ms_handle_connect on v2:172.21.15.38:6832/3839260519 2024-09-21T03:45:18.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.610+0000 7f6635187640 1 --2- 172.21.15.38:0/3626972498 >> [v2:172.21.15.38:6832/3839260519,v1:172.21.15.38:6833/3839260519] conn(0x5628a7b2b130 0x5628a7f2bda0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ddd94f625473ce06 server_cookie=b4e2b481cbe97a50 in_seq=0 out_seq=0 2024-09-21T03:45:18.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.610+0000 7f660ffff640 1 -- 172.21.15.38:0/3626972498 <== mds.0 v2:172.21.15.38:6832/3839260519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5628a7e888f0 con 0x5628a7b2b130 2024-09-21T03:45:18.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.610+0000 7f660ffff640 10 client.11034 handle_command_reply: tid=0 2024-09-21T03:45:18.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.610+0000 7f663e438740 1 client.11034 shutdown 2024-09-21T03:45:18.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.611+0000 7f663e438740 1 -- 172.21.15.38:0/3626972498 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5628a7b6c350 msgr2=0x5628a7f591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:18.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.611+0000 7f663e438740 1 --2- 172.21.15.38:0/3626972498 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5628a7b6c350 0x5628a7f591e0 secure :-1 s=READY pgs=2440 cs=0 l=1 rev1=1 crypto rx=0x7f663008cbe0 tx=0x7f6628013820 comp rx=0 tx=0).stop 2024-09-21T03:45:18.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.611+0000 7f663e438740 1 -- 172.21.15.38:0/3626972498 shutdown_connections 2024-09-21T03:45:18.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.611+0000 7f663e438740 1 --2- 172.21.15.38:0/3626972498 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5628a7b6c350 0x5628a7f591e0 unknown :-1 s=CLOSED pgs=2440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:18.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.611+0000 7f663e438740 1 --2- 172.21.15.38:0/3626972498 >> [v2:172.21.15.38:6832/3839260519,v1:172.21.15.38:6833/3839260519] conn(0x5628a7b2b130 0x5628a7f2bda0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:18.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.611+0000 7f663e438740 1 --2- 172.21.15.38:0/3626972498 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5628a7b4b670 0x5628a7f51710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:18.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.611+0000 7f663e438740 1 --2- 172.21.15.38:0/3626972498 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5628a7b43490 0x5628a7f688b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:18.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.611+0000 7f660ffff640 0 client.11034 ms_handle_reset on v2:172.21.15.38:6832/3839260519 2024-09-21T03:45:18.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.611+0000 7f663e438740 1 -- 172.21.15.38:0/3626972498 >> 172.21.15.38:0/3626972498 conn(0x5628a7b604a0 msgr2=0x5628a7f5a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:18.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.611+0000 7f663e438740 1 -- 172.21.15.38:0/3626972498 shutdown_connections 2024-09-21T03:45:18.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.611+0000 7f663e438740 1 -- 172.21.15.38:0/3626972498 wait complete. 2024-09-21T03:45:18.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.611+0000 7f663e438740 20 client.11034 trim_cache size 0 max 16384 2024-09-21T03:45:18.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.624+0000 7f663e438740 1 Processor -- start 2024-09-21T03:45:18.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.624+0000 7f663e438740 1 -- start start 2024-09-21T03:45:18.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.626+0000 7f663e438740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5628a7b43490 0x5628a7f58dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:18.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.626+0000 7f663e438740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5628a7b4b670 0x5628a7f59300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:18.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.626+0000 7f663e438740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5628a7b6c350 0x5628a7f548f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:18.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.626+0000 7f663e438740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f6630201ff0 con 0x5628a7b43490 2024-09-21T03:45:18.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.626+0000 7f663e438740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f663006f3f0 con 0x5628a7b4b670 2024-09-21T03:45:18.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.626+0000 7f663e438740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f66302019a0 con 0x5628a7b6c350 2024-09-21T03:45:18.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.626+0000 7f6635187640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5628a7b43490 0x5628a7f58dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:18.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.626+0000 7f6635187640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5628a7b43490 0x5628a7f58dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46500/0 (socket says 172.21.15.38:46500) 2024-09-21T03:45:18.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.626+0000 7f6635187640 1 -- 172.21.15.38:0/4161535326 learned_addr learned my addr 172.21.15.38:0/4161535326 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:45:18.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.626+0000 7f6634986640 1 --2- 172.21.15.38:0/4161535326 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5628a7b4b670 0x5628a7f59300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:18.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.626+0000 7f6635988640 1 --2- 172.21.15.38:0/4161535326 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5628a7b6c350 0x5628a7f548f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:18.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.626+0000 7f6635187640 1 -- 172.21.15.38:0/4161535326 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5628a7b6c350 msgr2=0x5628a7f548f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:18.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.626+0000 7f6635187640 1 --2- 172.21.15.38:0/4161535326 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5628a7b6c350 0x5628a7f548f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:18.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.626+0000 7f6635187640 1 -- 172.21.15.38:0/4161535326 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5628a7b4b670 msgr2=0x5628a7f59300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:18.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.626+0000 7f6635187640 1 --2- 172.21.15.38:0/4161535326 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5628a7b4b670 0x5628a7f59300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:18.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.626+0000 7f6635187640 1 -- 172.21.15.38:0/4161535326 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5628a7bb5660 con 0x5628a7b43490 2024-09-21T03:45:18.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.626+0000 7f6635187640 1 --2- 172.21.15.38:0/4161535326 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5628a7b43490 0x5628a7f58dc0 secure :-1 s=READY pgs=2441 cs=0 l=1 rev1=1 crypto rx=0x7f662c02b100 tx=0x7f662c004000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe649694022519dc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:18.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.626+0000 7f661a7fc640 1 -- 172.21.15.38:0/4161535326 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f662c031040 con 0x5628a7b43490 2024-09-21T03:45:18.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.626+0000 7f661a7fc640 1 -- 172.21.15.38:0/4161535326 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f662c003880 con 0x5628a7b43490 2024-09-21T03:45:18.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.626+0000 7f661a7fc640 1 -- 172.21.15.38:0/4161535326 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f662c003cb0 con 0x5628a7b43490 2024-09-21T03:45:18.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.627+0000 7f663e438740 1 -- 172.21.15.38:0/4161535326 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5628a7b43490 msgr2=0x5628a7f58dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:18.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.627+0000 7f663e438740 1 --2- 172.21.15.38:0/4161535326 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5628a7b43490 0x5628a7f58dc0 secure :-1 s=READY pgs=2441 cs=0 l=1 rev1=1 crypto rx=0x7f662c02b100 tx=0x7f662c004000 comp rx=0 tx=0).stop 2024-09-21T03:45:18.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.627+0000 7f663e438740 1 -- 172.21.15.38:0/4161535326 shutdown_connections 2024-09-21T03:45:18.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.627+0000 7f663e438740 1 --2- 172.21.15.38:0/4161535326 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5628a7b6c350 0x5628a7f548f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:18.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.627+0000 7f663e438740 1 --2- 172.21.15.38:0/4161535326 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5628a7b4b670 0x5628a7f59300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:18.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.627+0000 7f663e438740 1 --2- 172.21.15.38:0/4161535326 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5628a7b43490 0x5628a7f58dc0 unknown :-1 s=CLOSED pgs=2441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:18.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.627+0000 7f663e438740 1 -- 172.21.15.38:0/4161535326 >> 172.21.15.38:0/4161535326 conn(0x5628a7b604a0 msgr2=0x5628a7f0d4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:18.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.627+0000 7f663e438740 1 -- 172.21.15.38:0/4161535326 shutdown_connections 2024-09-21T03:45:18.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.627+0000 7f663e438740 1 -- 172.21.15.38:0/4161535326 wait complete. 2024-09-21T03:45:18.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.628+0000 7f663e438740 1 -- start start 2024-09-21T03:45:18.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.628+0000 7f663e438740 1 Processor -- start 2024-09-21T03:45:18.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.630+0000 7f663e438740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5628a7b43490 0x5628a7f61d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:18.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.630+0000 7f663e438740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5628a7b4b670 0x5628a7fd15a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:18.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.630+0000 7f663e438740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5628a7b6c350 0x5628a7f2d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:18.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.630+0000 7f663e438740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5628a7f51cf0 con 0x5628a7b4b670 2024-09-21T03:45:18.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.630+0000 7f663e438740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5628a7f537d0 con 0x5628a7b6c350 2024-09-21T03:45:18.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.630+0000 7f663e438740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x5628a7f6b980 con 0x5628a7b43490 2024-09-21T03:45:18.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.630+0000 7f663e438740 10 client.0 _ll_register_callbacks cb 0x7f6628001ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:45:18.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.630+0000 7f6634986640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5628a7b4b670 0x5628a7fd15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:18.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.630+0000 7f6634986640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5628a7b4b670 0x5628a7fd15a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46508/0 (socket says 172.21.15.38:46508) 2024-09-21T03:45:18.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.630+0000 7f6634986640 1 -- 172.21.15.38:0/1634189470 learned_addr learned my addr 172.21.15.38:0/1634189470 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:45:18.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.630+0000 7f6635187640 1 --2- 172.21.15.38:0/1634189470 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5628a7b43490 0x5628a7f61d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:18.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.630+0000 7f6635988640 1 --2- 172.21.15.38:0/1634189470 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5628a7b6c350 0x5628a7f2d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:18.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.630+0000 7f6634986640 1 -- 172.21.15.38:0/1634189470 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5628a7b43490 msgr2=0x5628a7f61d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:18.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.630+0000 7f6634986640 1 --2- 172.21.15.38:0/1634189470 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5628a7b43490 0x5628a7f61d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:18.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.630+0000 7f6634986640 1 -- 172.21.15.38:0/1634189470 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5628a7b6c350 msgr2=0x5628a7f2d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:18.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.630+0000 7f6634986640 1 --2- 172.21.15.38:0/1634189470 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5628a7b6c350 0x5628a7f2d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:18.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.630+0000 7f6634986640 1 -- 172.21.15.38:0/1634189470 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5628a7f7b970 con 0x5628a7b4b670 2024-09-21T03:45:18.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.630+0000 7f6634986640 1 --2- 172.21.15.38:0/1634189470 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5628a7b4b670 0x5628a7fd15a0 secure :-1 s=READY pgs=2442 cs=0 l=1 rev1=1 crypto rx=0x7f662000f790 tx=0x7f6620009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d87054e50bed27b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:18.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.630+0000 7f660ffff640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-21T03:45:18.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.630+0000 7f660ffff640 1 -- 172.21.15.38:0/1634189470 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6620089030 con 0x5628a7b4b670 2024-09-21T03:45:18.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.630+0000 7f660ffff640 1 -- 172.21.15.38:0/1634189470 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(149..149 src has 1..149) ==== 6303+0+0 (secure 0 0 0) 0x7f662005ec20 con 0x5628a7b4b670 2024-09-21T03:45:18.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.630+0000 7f663e438740 1 -- 172.21.15.38:0/1634189470 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f662c003880 con 0x5628a7b4b670 2024-09-21T03:45:18.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.631+0000 7f660ffff640 1 -- 172.21.15.38:0/1634189470 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=349) ==== 24+0+0 (secure 0 0 0) 0x7f6620067d00 con 0x5628a7b4b670 2024-09-21T03:45:18.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.631+0000 7f663e438740 10 client.11040 fetch_fsmap learned FSMap version 349 2024-09-21T03:45:18.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.631+0000 7f663e438740 1 -- 172.21.15.38:0/1634189470 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=349,monmap=2+}) -- 0x5628a7fd1ae0 con 0x5628a7b4b670 2024-09-21T03:45:18.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.631+0000 7f660ffff640 1 -- 172.21.15.38:0/1634189470 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f662c003880 con 0x5628a7b4b670 2024-09-21T03:45:18.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.631+0000 7f660ffff640 1 -- 172.21.15.38:0/1634189470 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 349) ==== 3502+0+0 (secure 0 0 0) 0x7f662005ee20 con 0x5628a7b4b670 2024-09-21T03:45:18.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.631+0000 7f663e438740 10 client.11040 fetch_fsmap finished waiting for FSMap version 349 2024-09-21T03:45:18.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.631+0000 7f663e438740 10 client.11040 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T03:45:18.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.631+0000 7f663e438740 20 client.11040 populate_metadata read hostname 'smithi038' 2024-09-21T03:45:18.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.631+0000 7f663e438740 1 --2- 172.21.15.38:0/1634189470 >> [v2:172.21.15.38:6832/3839260519,v1:172.21.15.38:6833/3839260519] conn(0x5628a7f3e550 0x5628a7f2bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:18.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.631+0000 7f663e438740 4 client.11040 mds_command: new command op to 10935 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-09-21T03:45:18.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.631+0000 7f663e438740 1 -- 172.21.15.38:0/1634189470 --> [v2:172.21.15.38:6832/3839260519,v1:172.21.15.38:6833/3839260519] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) -- 0x5628a7e843a0 con 0x5628a7f3e550 2024-09-21T03:45:18.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.631+0000 7f6635187640 1 --2- 172.21.15.38:0/1634189470 >> [v2:172.21.15.38:6832/3839260519,v1:172.21.15.38:6833/3839260519] conn(0x5628a7f3e550 0x5628a7f2bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:18.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.631+0000 7f660ffff640 10 client.11040 ms_handle_connect on v2:172.21.15.38:6832/3839260519 2024-09-21T03:45:18.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.632+0000 7f6635187640 1 --2- 172.21.15.38:0/1634189470 >> [v2:172.21.15.38:6832/3839260519,v1:172.21.15.38:6833/3839260519] conn(0x5628a7f3e550 0x5628a7f2bcf0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=261b1acde28d9249 server_cookie=39eca55b57065daa in_seq=0 out_seq=0 2024-09-21T03:45:18.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.632+0000 7f660ffff640 1 -- 172.21.15.38:0/1634189470 <== mds.0 v2:172.21.15.38:6832/3839260519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+145 (crc 0 0 0) 0x5628a7e843a0 con 0x5628a7f3e550 2024-09-21T03:45:18.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.632+0000 7f660ffff640 10 client.11040 handle_command_reply: tid=0 2024-09-21T03:45:18.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.632+0000 7f663e438740 1 client.11040 shutdown 2024-09-21T03:45:18.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.632+0000 7f663e438740 1 -- 172.21.15.38:0/1634189470 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5628a7b4b670 msgr2=0x5628a7fd15a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:18.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.632+0000 7f663e438740 1 --2- 172.21.15.38:0/1634189470 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5628a7b4b670 0x5628a7fd15a0 secure :-1 s=READY pgs=2442 cs=0 l=1 rev1=1 crypto rx=0x7f662000f790 tx=0x7f6620009f90 comp rx=0 tx=0).stop 2024-09-21T03:45:18.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.632+0000 7f663e438740 1 -- 172.21.15.38:0/1634189470 shutdown_connections 2024-09-21T03:45:18.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.632+0000 7f663e438740 1 --2- 172.21.15.38:0/1634189470 >> [v2:172.21.15.38:6832/3839260519,v1:172.21.15.38:6833/3839260519] conn(0x5628a7f3e550 0x5628a7f2bcf0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:18.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.632+0000 7f663e438740 1 --2- 172.21.15.38:0/1634189470 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5628a7b6c350 0x5628a7f2d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:18.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.632+0000 7f660ffff640 0 client.11040 ms_handle_reset on v2:172.21.15.38:6832/3839260519 2024-09-21T03:45:18.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.632+0000 7f663e438740 1 --2- 172.21.15.38:0/1634189470 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5628a7b4b670 0x5628a7fd15a0 unknown :-1 s=CLOSED pgs=2442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:18.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.632+0000 7f663e438740 1 --2- 172.21.15.38:0/1634189470 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5628a7b43490 0x5628a7f61d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:18.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.632+0000 7f663e438740 1 -- 172.21.15.38:0/1634189470 >> 172.21.15.38:0/1634189470 conn(0x5628a7b604a0 msgr2=0x5628a7f0d4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:18.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.632+0000 7f663e438740 1 -- 172.21.15.38:0/1634189470 shutdown_connections 2024-09-21T03:45:18.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.632+0000 7f663e438740 1 -- 172.21.15.38:0/1634189470 wait complete. 2024-09-21T03:45:18.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.632+0000 7f663e438740 20 client.11040 trim_cache size 0 max 16384 2024-09-21T03:45:18.639 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:45:18.639 INFO:teuthology.orchestra.run.smithi038.stdout:[ 2024-09-21T03:45:18.639 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-21T03:45:18.639 INFO:teuthology.orchestra.run.smithi038.stdout: "damage_type": "backtrace", 2024-09-21T03:45:18.639 INFO:teuthology.orchestra.run.smithi038.stdout: "id": 1336616810, 2024-09-21T03:45:18.640 INFO:teuthology.orchestra.run.smithi038.stdout: "ino": 1099511627777, 2024-09-21T03:45:18.640 INFO:teuthology.orchestra.run.smithi038.stdout: "path": "/testdir/hardlink1" 2024-09-21T03:45:18.640 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-21T03:45:18.640 INFO:teuthology.orchestra.run.smithi038.stdout:] 2024-09-21T03:45:18.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.633+0000 7f66373eb640 1 -- 172.21.15.38:0/552804698 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6614078bc0 msgr2=0x7f661407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:18.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.633+0000 7f66373eb640 1 --2- 172.21.15.38:0/552804698 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6614078bc0 0x7f661407b0d0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f662c033000 tx=0x7f662c001a20 comp rx=0 tx=0).stop 2024-09-21T03:45:18.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.633+0000 7f66373eb640 1 -- 172.21.15.38:0/552804698 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f663014c2a0 msgr2=0x7f66300b2660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:18.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.633+0000 7f66373eb640 1 --2- 172.21.15.38:0/552804698 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f663014c2a0 0x7f66300b2660 secure :-1 s=READY pgs=2438 cs=0 l=1 rev1=1 crypto rx=0x7f6620011ab0 tx=0x7f6620011f70 comp rx=0 tx=0).stop 2024-09-21T03:45:18.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.633+0000 7f66373eb640 1 -- 172.21.15.38:0/552804698 shutdown_connections 2024-09-21T03:45:18.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.633+0000 7f66373eb640 1 --2- 172.21.15.38:0/552804698 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6614078bc0 0x7f661407b0d0 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:18.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.633+0000 7f66373eb640 1 --2- 172.21.15.38:0/552804698 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f66300b2ba0 0x7f66301f8b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:18.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.633+0000 7f66373eb640 1 --2- 172.21.15.38:0/552804698 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f663014c2a0 0x7f66300b2660 unknown :-1 s=CLOSED pgs=2438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:18.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.633+0000 7f66373eb640 1 --2- 172.21.15.38:0/552804698 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f661c003cb0 0x7f663015a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:18.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.633+0000 7f66373eb640 1 -- 172.21.15.38:0/552804698 >> 172.21.15.38:0/552804698 conn(0x7f663009d830 msgr2=0x7f6630160a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:18.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.633+0000 7f66373eb640 1 -- 172.21.15.38:0/552804698 shutdown_connections 2024-09-21T03:45:18.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.633+0000 7f66373eb640 1 -- 172.21.15.38:0/552804698 wait complete. 2024-09-21T03:45:18.646 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:45:18.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.761+0000 7f0ee5c4d640 1 -- 172.21.15.38:0/1874964298 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0ec8005250 msgr2=0x7f0ec800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:18.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.761+0000 7f0ee5c4d640 1 --2- 172.21.15.38:0/1874964298 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0ec8005250 0x7f0ec800b9c0 secure :-1 s=READY pgs=2444 cs=0 l=1 rev1=1 crypto rx=0x7f0ee01571a0 tx=0x7f0ec8005830 comp rx=0 tx=0).stop 2024-09-21T03:45:18.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.761+0000 7f0ee5c4d640 1 -- 172.21.15.38:0/1874964298 shutdown_connections 2024-09-21T03:45:18.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.761+0000 7f0ee5c4d640 1 --2- 172.21.15.38:0/1874964298 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0ec8005250 0x7f0ec800b9c0 unknown :-1 s=CLOSED pgs=2444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:18.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.761+0000 7f0ee5c4d640 1 -- 172.21.15.38:0/1874964298 >> 172.21.15.38:0/1874964298 conn(0x7f0ee00cef40 msgr2=0x7f0ee00cf350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:18.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.761+0000 7f0ee5c4d640 1 -- 172.21.15.38:0/1874964298 shutdown_connections 2024-09-21T03:45:18.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.761+0000 7f0ee5c4d640 1 -- 172.21.15.38:0/1874964298 wait complete. 2024-09-21T03:45:18.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.763+0000 7f0ee5c4d640 1 Processor -- start 2024-09-21T03:45:18.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.763+0000 7f0ee5c4d640 1 -- start start 2024-09-21T03:45:18.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.764+0000 7f0ee5c4d640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0ec8005250 0x7f0ee01560b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:18.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.764+0000 7f0ee5c4d640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0ee0156a00 0x7f0ee00b6f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:18.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.764+0000 7f0ee5c4d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0ee00b7460 0x7f0ee014f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:18.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.764+0000 7f0ee5c4d640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f0ee00d1b50 con 0x7f0ee00b7460 2024-09-21T03:45:18.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.764+0000 7f0ee5c4d640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f0ee00d19d0 con 0x7f0ee0156a00 2024-09-21T03:45:18.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.764+0000 7f0ee5c4d640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f0ee01566b0 con 0x7f0ec8005250 2024-09-21T03:45:18.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.765+0000 7f0edffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0ee00b7460 0x7f0ee014f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:18.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.765+0000 7f0edffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0ee00b7460 0x7f0ee014f4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46532/0 (socket says 172.21.15.38:46532) 2024-09-21T03:45:18.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.765+0000 7f0edffff640 1 -- 172.21.15.38:0/344444485 learned_addr learned my addr 172.21.15.38:0/344444485 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:45:18.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.765+0000 7f0edf7fe640 1 --2- 172.21.15.38:0/344444485 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0ec8005250 0x7f0ee01560b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:18.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.765+0000 7f0edeffd640 1 --2- 172.21.15.38:0/344444485 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0ee0156a00 0x7f0ee00b6f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:18.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.765+0000 7f0edffff640 1 -- 172.21.15.38:0/344444485 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0ec8005250 msgr2=0x7f0ee01560b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:18.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.765+0000 7f0edffff640 1 --2- 172.21.15.38:0/344444485 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0ec8005250 0x7f0ee01560b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:18.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.765+0000 7f0edffff640 1 -- 172.21.15.38:0/344444485 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0ee0156a00 msgr2=0x7f0ee00b6f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:18.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.765+0000 7f0edffff640 1 --2- 172.21.15.38:0/344444485 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0ee0156a00 0x7f0ee00b6f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:18.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.765+0000 7f0edffff640 1 -- 172.21.15.38:0/344444485 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ee00b7870 con 0x7f0ee00b7460 2024-09-21T03:45:18.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.765+0000 7f0edeffd640 1 --2- 172.21.15.38:0/344444485 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0ee0156a00 0x7f0ee00b6f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:45:18.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.765+0000 7f0edffff640 1 --2- 172.21.15.38:0/344444485 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0ee00b7460 0x7f0ee014f4c0 secure :-1 s=READY pgs=2445 cs=0 l=1 rev1=1 crypto rx=0x7f0ed800c6d0 tx=0x7f0ed800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:18.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.765+0000 7f0ec7fff640 1 -- 172.21.15.38:0/344444485 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ed8039070 con 0x7f0ee00b7460 2024-09-21T03:45:18.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.765+0000 7f0ec7fff640 1 -- 172.21.15.38:0/344444485 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0ed802bc50 con 0x7f0ee00b7460 2024-09-21T03:45:18.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.765+0000 7f0ee5c4d640 1 -- 172.21.15.38:0/344444485 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ee01f66f0 con 0x7f0ee00b7460 2024-09-21T03:45:18.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.765+0000 7f0ec7fff640 1 -- 172.21.15.38:0/344444485 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ed8006290 con 0x7f0ee00b7460 2024-09-21T03:45:18.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.765+0000 7f0ee5c4d640 1 -- 172.21.15.38:0/344444485 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ee01f6cb0 con 0x7f0ee00b7460 2024-09-21T03:45:18.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.766+0000 7f0ee5c4d640 1 -- 172.21.15.38:0/344444485 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ee00ccd10 con 0x7f0ee00b7460 2024-09-21T03:45:18.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.767+0000 7f0ec7fff640 1 -- 172.21.15.38:0/344444485 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f0ed8006430 con 0x7f0ee00b7460 2024-09-21T03:45:18.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.768+0000 7f0ec7fff640 1 --2- 172.21.15.38:0/344444485 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0ecc078bc0 0x7f0ecc07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:18.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.768+0000 7f0ec7fff640 1 -- 172.21.15.38:0/344444485 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 6303+0+0 (secure 0 0 0) 0x7f0ed80bad40 con 0x7f0ee00b7460 2024-09-21T03:45:18.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.768+0000 7f0edf7fe640 1 --2- 172.21.15.38:0/344444485 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0ecc078bc0 0x7f0ecc07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:18.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.769+0000 7f0edf7fe640 1 --2- 172.21.15.38:0/344444485 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0ecc078bc0 0x7f0ecc07b0d0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f0ed0009bd0 tx=0x7f0ed0005f90 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:18.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.773+0000 7f0ec7fff640 1 -- 172.21.15.38:0/344444485 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f0ee00b7870 con 0x7f0ee00b7460 2024-09-21T03:45:19.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.998+0000 7f0ee5c4d640 1 -- 172.21.15.38:0/344444485 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0ee00b6460 con 0x7f0ee00b7460 2024-09-21T03:45:19.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:18.998+0000 7f0ec7fff640 1 -- 172.21.15.38:0/344444485 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 349 v349) ==== 95+0+5241 (secure 0 0 0) 0x7f0ed8083080 con 0x7f0ee00b7460 2024-09-21T03:45:19.000 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:45:19.001 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":349,"btime":"2024-09-21T03:45:15:115320+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8399,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/3606957623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":3606957623},{"type":"v1","addr":"172.21.15.38:6835","nonce":3606957623}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":338},{"gid":8405,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/456651486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":456651486},{"type":"v1","addr":"172.21.15.195:6837","nonce":456651486}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":338},{"gid":9604,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/2649311840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":2649311840},{"type":"v1","addr":"172.21.15.195:6835","nonce":2649311840}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":338}],"filesystems":[{"mdsmap":{"epoch":349,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:44:56.710446+0000","modified":"2024-09-21T03:45:15.115308+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":147,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10935},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10935":{"gid":10935,"name":"c","rank":0,"incarnation":346,"state":"up:active","state_seq":4,"addr":"172.21.15.38:6833/3839260519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":3839260519},{"type":"v1","addr":"172.21.15.38:6833","nonce":3839260519}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10935,"qdb_cluster":[10935]},"id":7}]} 2024-09-21T03:45:19.001 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 349 2024-09-21T03:45:19.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.001+0000 7f0ee5c4d640 1 -- 172.21.15.38:0/344444485 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0ecc078bc0 msgr2=0x7f0ecc07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:19.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.001+0000 7f0ee5c4d640 1 --2- 172.21.15.38:0/344444485 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0ecc078bc0 0x7f0ecc07b0d0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f0ed0009bd0 tx=0x7f0ed0005f90 comp rx=0 tx=0).stop 2024-09-21T03:45:19.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.001+0000 7f0ee5c4d640 1 -- 172.21.15.38:0/344444485 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0ee00b7460 msgr2=0x7f0ee014f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:19.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.001+0000 7f0ee5c4d640 1 --2- 172.21.15.38:0/344444485 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0ee00b7460 0x7f0ee014f4c0 secure :-1 s=READY pgs=2445 cs=0 l=1 rev1=1 crypto rx=0x7f0ed800c6d0 tx=0x7f0ed800c2d0 comp rx=0 tx=0).stop 2024-09-21T03:45:19.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.001+0000 7f0ee5c4d640 1 -- 172.21.15.38:0/344444485 shutdown_connections 2024-09-21T03:45:19.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.001+0000 7f0ee5c4d640 1 --2- 172.21.15.38:0/344444485 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0ee00b7460 0x7f0ee014f4c0 unknown :-1 s=CLOSED pgs=2445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:19.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.001+0000 7f0ee5c4d640 1 --2- 172.21.15.38:0/344444485 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0ee0156a00 0x7f0ee00b6f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:19.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.001+0000 7f0ee5c4d640 1 --2- 172.21.15.38:0/344444485 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0ecc078bc0 0x7f0ecc07b0d0 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:19.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.001+0000 7f0ee5c4d640 1 --2- 172.21.15.38:0/344444485 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0ec8005250 0x7f0ee01560b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:19.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.001+0000 7f0ee5c4d640 1 -- 172.21.15.38:0/344444485 >> 172.21.15.38:0/344444485 conn(0x7f0ee00cef40 msgr2=0x7f0ee0070100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:19.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.001+0000 7f0ee5c4d640 1 -- 172.21.15.38:0/344444485 shutdown_connections 2024-09-21T03:45:19.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.001+0000 7f0ee5c4d640 1 -- 172.21.15.38:0/344444485 wait complete. 2024-09-21T03:45:19.015 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c damage rm 1336616810 2024-09-21T03:45:19.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.133+0000 7f98f4932640 1 -- 172.21.15.38:0/3925108442 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f98dc004800 msgr2=0x7f98dc004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:19.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.133+0000 7f98f4932640 1 --2- 172.21.15.38:0/3925108442 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f98dc004800 0x7f98dc004c10 secure :-1 s=READY pgs=2447 cs=0 l=1 rev1=1 crypto rx=0x7f98d8001800 tx=0x7f98dc005830 comp rx=0 tx=0).stop 2024-09-21T03:45:19.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.133+0000 7f98eed9d640 1 -- 172.21.15.38:0/3925108442 reap_dead start 2024-09-21T03:45:19.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.133+0000 7f98f4932640 1 -- 172.21.15.38:0/3925108442 shutdown_connections 2024-09-21T03:45:19.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.133+0000 7f98f4932640 1 -- 172.21.15.38:0/3925108442 >> 172.21.15.38:0/3925108442 conn(0x7f98f00b1b90 msgr2=0x7f98f00b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:19.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.133+0000 7f98f4932640 1 -- 172.21.15.38:0/3925108442 shutdown_connections 2024-09-21T03:45:19.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.133+0000 7f98f4932640 1 -- 172.21.15.38:0/3925108442 wait complete. 2024-09-21T03:45:19.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.135+0000 7f98f4932640 1 Processor -- start 2024-09-21T03:45:19.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.135+0000 7f98f4932640 1 -- start start 2024-09-21T03:45:19.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.136+0000 7f98f4932640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f98dc004800 0x7f98f00b0a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:19.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.136+0000 7f98f4932640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f98f00af1c0 0x7f98f00b50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:19.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.136+0000 7f98f4932640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f98f00b5620 0x7f98f00cdc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:19.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.136+0000 7f98f4932640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f98f0072f10 con 0x7f98f00af1c0 2024-09-21T03:45:19.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.136+0000 7f98f4932640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f98f0072d90 con 0x7f98f00b5620 2024-09-21T03:45:19.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.136+0000 7f98f4932640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f98f0073090 con 0x7f98dc004800 2024-09-21T03:45:19.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.136+0000 7f98edd9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f98f00af1c0 0x7f98f00b50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:19.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.136+0000 7f98edd9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f98f00af1c0 0x7f98f00b50e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46548/0 (socket says 172.21.15.38:46548) 2024-09-21T03:45:19.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.136+0000 7f98edd9b640 1 -- 172.21.15.38:0/983876041 learned_addr learned my addr 172.21.15.38:0/983876041 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:45:19.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.137+0000 7f98eed9d640 1 --2- 172.21.15.38:0/983876041 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f98f00b5620 0x7f98f00cdc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:19.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.137+0000 7f98ee59c640 1 --2- 172.21.15.38:0/983876041 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f98dc004800 0x7f98f00b0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:19.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.137+0000 7f98edd9b640 1 -- 172.21.15.38:0/983876041 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f98dc004800 msgr2=0x7f98f00b0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:19.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.137+0000 7f98edd9b640 1 --2- 172.21.15.38:0/983876041 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f98dc004800 0x7f98f00b0a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:19.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.137+0000 7f98edd9b640 1 -- 172.21.15.38:0/983876041 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f98f00b5620 msgr2=0x7f98f00cdc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:19.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.137+0000 7f98edd9b640 1 --2- 172.21.15.38:0/983876041 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f98f00b5620 0x7f98f00cdc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:19.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.137+0000 7f98edd9b640 1 -- 172.21.15.38:0/983876041 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98f00b5a30 con 0x7f98f00af1c0 2024-09-21T03:45:19.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.137+0000 7f98edd9b640 1 --2- 172.21.15.38:0/983876041 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f98f00af1c0 0x7f98f00b50e0 secure :-1 s=READY pgs=2448 cs=0 l=1 rev1=1 crypto rx=0x7f98d8000f60 tx=0x7f98d8006940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:19.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.137+0000 7f98e6ffd640 1 -- 172.21.15.38:0/983876041 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98d8043070 con 0x7f98f00af1c0 2024-09-21T03:45:19.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.137+0000 7f98e6ffd640 1 -- 172.21.15.38:0/983876041 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f98d80038e0 con 0x7f98f00af1c0 2024-09-21T03:45:19.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.137+0000 7f98e6ffd640 1 -- 172.21.15.38:0/983876041 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98d8003d30 con 0x7f98f00af1c0 2024-09-21T03:45:19.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.137+0000 7f98f4932640 1 -- 172.21.15.38:0/983876041 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98f00b0f90 con 0x7f98f00af1c0 2024-09-21T03:45:19.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.137+0000 7f98f4932640 1 -- 172.21.15.38:0/983876041 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98f01f6890 con 0x7f98f00af1c0 2024-09-21T03:45:19.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.139+0000 7f98e6ffd640 1 -- 172.21.15.38:0/983876041 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f98d8046070 con 0x7f98f00af1c0 2024-09-21T03:45:19.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.139+0000 7f98e6ffd640 1 --2- 172.21.15.38:0/983876041 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f98d0078b90 0x7f98d007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:19.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.140+0000 7f98e6ffd640 1 -- 172.21.15.38:0/983876041 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 6303+0+0 (secure 0 0 0) 0x7f98d80baeb0 con 0x7f98f00af1c0 2024-09-21T03:45:19.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.140+0000 7f98ee59c640 1 --2- 172.21.15.38:0/983876041 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f98d0078b90 0x7f98d007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:19.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.140+0000 7f98ee59c640 1 --2- 172.21.15.38:0/983876041 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f98d0078b90 0x7f98d007b0a0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f98e0001be0 tx=0x7f98e0001490 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:19.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.140+0000 7f98fb8a0740 1 Processor -- start 2024-09-21T03:45:19.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.141+0000 7f98fb8a0740 1 -- start start 2024-09-21T03:45:19.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.142+0000 7f98fb8a0740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55d77f83e2a0 0x55d77fc2d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:19.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.142+0000 7f98fb8a0740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d77f8153e0 0x55d77fc10280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:19.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.142+0000 7f98fb8a0740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d77f81d5c0 0x55d77fc107c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:19.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.142+0000 7f98fb8a0740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55d77fb58910 con 0x55d77f8153e0 2024-09-21T03:45:19.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.142+0000 7f98fb8a0740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55d77fb47d40 con 0x55d77f81d5c0 2024-09-21T03:45:19.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.142+0000 7f98fb8a0740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55d77fb4bb50 con 0x55d77f83e2a0 2024-09-21T03:45:19.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.142+0000 7f98edd9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d77f8153e0 0x55d77fc10280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:19.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.142+0000 7f98edd9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d77f8153e0 0x55d77fc10280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46552/0 (socket says 172.21.15.38:46552) 2024-09-21T03:45:19.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.142+0000 7f98edd9b640 1 -- 172.21.15.38:0/173064540 learned_addr learned my addr 172.21.15.38:0/173064540 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:45:19.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.142+0000 7f98eed9d640 1 --2- 172.21.15.38:0/173064540 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d77f81d5c0 0x55d77fc107c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:19.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.142+0000 7f98ee59c640 1 --2- 172.21.15.38:0/173064540 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55d77f83e2a0 0x55d77fc2d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:19.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.142+0000 7f98edd9b640 1 -- 172.21.15.38:0/173064540 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55d77f83e2a0 msgr2=0x55d77fc2d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:19.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.142+0000 7f98edd9b640 1 --2- 172.21.15.38:0/173064540 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55d77f83e2a0 0x55d77fc2d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:19.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.142+0000 7f98edd9b640 1 -- 172.21.15.38:0/173064540 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d77f81d5c0 msgr2=0x55d77fc107c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:19.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.142+0000 7f98edd9b640 1 --2- 172.21.15.38:0/173064540 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d77f81d5c0 0x55d77fc107c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:19.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.142+0000 7f98edd9b640 1 -- 172.21.15.38:0/173064540 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d77f922710 con 0x55d77f8153e0 2024-09-21T03:45:19.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.142+0000 7f98ee59c640 1 --2- 172.21.15.38:0/173064540 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55d77f83e2a0 0x55d77fc2d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:45:19.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.143+0000 7f98edd9b640 1 --2- 172.21.15.38:0/173064540 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d77f8153e0 0x55d77fc10280 secure :-1 s=READY pgs=2449 cs=0 l=1 rev1=1 crypto rx=0x7f98d80b9130 tx=0x7f98d807e970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb82517be361d5f7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:19.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.143+0000 7f98cffff640 1 -- 172.21.15.38:0/173064540 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98d80829a0 con 0x55d77f8153e0 2024-09-21T03:45:19.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.143+0000 7f98cffff640 1 -- 172.21.15.38:0/173064540 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f98d8087480 con 0x55d77f8153e0 2024-09-21T03:45:19.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.143+0000 7f98cffff640 1 -- 172.21.15.38:0/173064540 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98d808c3c0 con 0x55d77f8153e0 2024-09-21T03:45:19.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.143+0000 7f98fb8a0740 1 -- 172.21.15.38:0/173064540 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d77f8153e0 msgr2=0x55d77fc10280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:19.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.144+0000 7f98fb8a0740 1 --2- 172.21.15.38:0/173064540 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d77f8153e0 0x55d77fc10280 secure :-1 s=READY pgs=2449 cs=0 l=1 rev1=1 crypto rx=0x7f98d80b9130 tx=0x7f98d807e970 comp rx=0 tx=0).stop 2024-09-21T03:45:19.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.144+0000 7f98fb8a0740 1 -- 172.21.15.38:0/173064540 shutdown_connections 2024-09-21T03:45:19.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.144+0000 7f98fb8a0740 1 --2- 172.21.15.38:0/173064540 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d77f81d5c0 0x55d77fc107c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:19.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.144+0000 7f98fb8a0740 1 --2- 172.21.15.38:0/173064540 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d77f8153e0 0x55d77fc10280 unknown :-1 s=CLOSED pgs=2449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:19.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.144+0000 7f98fb8a0740 1 --2- 172.21.15.38:0/173064540 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55d77f83e2a0 0x55d77fc2d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:19.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.144+0000 7f98fb8a0740 1 -- 172.21.15.38:0/173064540 >> 172.21.15.38:0/173064540 conn(0x55d77f8323f0 msgr2=0x55d77fbe64a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:19.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.144+0000 7f98fb8a0740 1 -- 172.21.15.38:0/173064540 shutdown_connections 2024-09-21T03:45:19.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.144+0000 7f98fb8a0740 1 -- 172.21.15.38:0/173064540 wait complete. 2024-09-21T03:45:19.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.145+0000 7f98fb8a0740 1 -- start start 2024-09-21T03:45:19.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.145+0000 7f98fb8a0740 1 Processor -- start 2024-09-21T03:45:19.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.146+0000 7f98fb8a0740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55d77f8153e0 0x55d77fc085b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:19.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.146+0000 7f98fb8a0740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d77f81d5c0 0x55d77fbfe4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:19.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.146+0000 7f98fb8a0740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d77f83e2a0 0x55d77fbe54f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:19.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.146+0000 7f98fb8a0740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55d77f8b8650 con 0x55d77f83e2a0 2024-09-21T03:45:19.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.146+0000 7f98fb8a0740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55d77face5f0 con 0x55d77f81d5c0 2024-09-21T03:45:19.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.146+0000 7f98fb8a0740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55d77fb47d40 con 0x55d77f8153e0 2024-09-21T03:45:19.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.146+0000 7f98fb8a0740 10 client.0 _ll_register_callbacks cb 0x55d77fb42060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:45:19.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.146+0000 7f98eed9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d77f83e2a0 0x55d77fbe54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:19.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.146+0000 7f98eed9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d77f83e2a0 0x55d77fbe54f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46568/0 (socket says 172.21.15.38:46568) 2024-09-21T03:45:19.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.146+0000 7f98eed9d640 1 -- 172.21.15.38:0/371848366 learned_addr learned my addr 172.21.15.38:0/371848366 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:45:19.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.147+0000 7f98edd9b640 1 --2- 172.21.15.38:0/371848366 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d77f81d5c0 0x55d77fbfe4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:19.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.147+0000 7f98ee59c640 1 --2- 172.21.15.38:0/371848366 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55d77f8153e0 0x55d77fc085b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:19.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.147+0000 7f98eed9d640 1 -- 172.21.15.38:0/371848366 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55d77f8153e0 msgr2=0x55d77fc085b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:19.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.147+0000 7f98eed9d640 1 --2- 172.21.15.38:0/371848366 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55d77f8153e0 0x55d77fc085b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:19.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.147+0000 7f98eed9d640 1 -- 172.21.15.38:0/371848366 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d77f81d5c0 msgr2=0x55d77fbfe4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:19.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.147+0000 7f98eed9d640 1 --2- 172.21.15.38:0/371848366 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d77f81d5c0 0x55d77fbfe4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:19.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.147+0000 7f98eed9d640 1 -- 172.21.15.38:0/371848366 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d77f982ea0 con 0x55d77f83e2a0 2024-09-21T03:45:19.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.147+0000 7f98ee59c640 1 --2- 172.21.15.38:0/371848366 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55d77f8153e0 0x55d77fc085b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:45:19.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.147+0000 7f98edd9b640 1 --2- 172.21.15.38:0/371848366 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d77f81d5c0 0x55d77fbfe4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:45:19.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.147+0000 7f98eed9d640 1 --2- 172.21.15.38:0/371848366 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d77f83e2a0 0x55d77fbe54f0 secure :-1 s=READY pgs=2450 cs=0 l=1 rev1=1 crypto rx=0x7f98f0072280 tx=0x7f98e80138f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cbfbb928c69e7a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:19.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.147+0000 7f98cd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-21T03:45:19.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.147+0000 7f98cd7fa640 1 -- 172.21.15.38:0/371848366 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98e8004030 con 0x55d77f83e2a0 2024-09-21T03:45:19.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.147+0000 7f98fb8a0740 1 -- 172.21.15.38:0/371848366 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55d77fb25b60 con 0x55d77f83e2a0 2024-09-21T03:45:19.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.147+0000 7f98cd7fa640 1 -- 172.21.15.38:0/371848366 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(149..149 src has 1..149) ==== 6303+0+0 (secure 0 0 0) 0x7f98e80041d0 con 0x55d77f83e2a0 2024-09-21T03:45:19.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.147+0000 7f98cd7fa640 1 -- 172.21.15.38:0/371848366 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=349) ==== 24+0+0 (secure 0 0 0) 0x7f98e8026050 con 0x55d77f83e2a0 2024-09-21T03:45:19.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.147+0000 7f98fb8a0740 10 client.11058 fetch_fsmap learned FSMap version 349 2024-09-21T03:45:19.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.147+0000 7f98fb8a0740 1 -- 172.21.15.38:0/371848366 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=349,monmap=2+}) -- 0x55d77f8875b0 con 0x55d77f83e2a0 2024-09-21T03:45:19.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.148+0000 7f98cd7fa640 1 -- 172.21.15.38:0/371848366 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55d77fb25b60 con 0x55d77f83e2a0 2024-09-21T03:45:19.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.148+0000 7f98cd7fa640 1 -- 172.21.15.38:0/371848366 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 349) ==== 3502+0+0 (secure 0 0 0) 0x7f98e8029020 con 0x55d77f83e2a0 2024-09-21T03:45:19.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.148+0000 7f98fb8a0740 10 client.11058 fetch_fsmap finished waiting for FSMap version 349 2024-09-21T03:45:19.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.148+0000 7f98fb8a0740 10 client.11058 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T03:45:19.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.148+0000 7f98fb8a0740 20 client.11058 populate_metadata read hostname 'smithi038' 2024-09-21T03:45:19.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.148+0000 7f98fb8a0740 1 --2- 172.21.15.38:0/371848366 >> [v2:172.21.15.38:6832/3839260519,v1:172.21.15.38:6833/3839260519] conn(0x55d77f7fd080 0x55d77fc21c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:19.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.148+0000 7f98fb8a0740 4 client.11058 mds_command: new command op to 10935 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:45:19.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.148+0000 7f98fb8a0740 1 -- 172.21.15.38:0/371848366 --> [v2:172.21.15.38:6832/3839260519,v1:172.21.15.38:6833/3839260519] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d77fb5a860 con 0x55d77f7fd080 2024-09-21T03:45:19.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.148+0000 7f98ee59c640 1 --2- 172.21.15.38:0/371848366 >> [v2:172.21.15.38:6832/3839260519,v1:172.21.15.38:6833/3839260519] conn(0x55d77f7fd080 0x55d77fc21c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:19.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.148+0000 7f98cd7fa640 10 client.11058 ms_handle_connect on v2:172.21.15.38:6832/3839260519 2024-09-21T03:45:19.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.148+0000 7f98ee59c640 1 --2- 172.21.15.38:0/371848366 >> [v2:172.21.15.38:6832/3839260519,v1:172.21.15.38:6833/3839260519] conn(0x55d77f7fd080 0x55d77fc21c40 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ead81fef7ad48cc2 server_cookie=e9966bf37ebc90a0 in_seq=0 out_seq=0 2024-09-21T03:45:19.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.149+0000 7f98cd7fa640 1 -- 172.21.15.38:0/371848366 <== mds.0 v2:172.21.15.38:6832/3839260519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d77fb5a860 con 0x55d77f7fd080 2024-09-21T03:45:19.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.149+0000 7f98cd7fa640 10 client.11058 handle_command_reply: tid=0 2024-09-21T03:45:19.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.149+0000 7f98fb8a0740 1 client.11058 shutdown 2024-09-21T03:45:19.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.149+0000 7f98fb8a0740 1 -- 172.21.15.38:0/371848366 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d77f83e2a0 msgr2=0x55d77fbe54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:19.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.149+0000 7f98fb8a0740 1 --2- 172.21.15.38:0/371848366 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d77f83e2a0 0x55d77fbe54f0 secure :-1 s=READY pgs=2450 cs=0 l=1 rev1=1 crypto rx=0x7f98f0072280 tx=0x7f98e80138f0 comp rx=0 tx=0).stop 2024-09-21T03:45:19.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.149+0000 7f98fb8a0740 1 -- 172.21.15.38:0/371848366 shutdown_connections 2024-09-21T03:45:19.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.149+0000 7f98fb8a0740 1 --2- 172.21.15.38:0/371848366 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d77f83e2a0 0x55d77fbe54f0 unknown :-1 s=CLOSED pgs=2450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:19.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.149+0000 7f98fb8a0740 1 --2- 172.21.15.38:0/371848366 >> [v2:172.21.15.38:6832/3839260519,v1:172.21.15.38:6833/3839260519] conn(0x55d77f7fd080 0x55d77fc21c40 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:19.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.149+0000 7f98fb8a0740 1 --2- 172.21.15.38:0/371848366 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d77f81d5c0 0x55d77fbfe4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:19.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.149+0000 7f98fb8a0740 1 --2- 172.21.15.38:0/371848366 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55d77f8153e0 0x55d77fc085b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:19.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.149+0000 7f98cd7fa640 0 client.11058 ms_handle_reset on v2:172.21.15.38:6832/3839260519 2024-09-21T03:45:19.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.149+0000 7f98fb8a0740 1 -- 172.21.15.38:0/371848366 >> 172.21.15.38:0/371848366 conn(0x55d77f8323f0 msgr2=0x55d77fc10280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:19.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.149+0000 7f98fb8a0740 1 -- 172.21.15.38:0/371848366 shutdown_connections 2024-09-21T03:45:19.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.149+0000 7f98fb8a0740 1 -- 172.21.15.38:0/371848366 wait complete. 2024-09-21T03:45:19.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.150+0000 7f98fb8a0740 20 client.11058 trim_cache size 0 max 16384 2024-09-21T03:45:19.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.163+0000 7f98fb8a0740 1 Processor -- start 2024-09-21T03:45:19.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.163+0000 7f98fb8a0740 1 -- start start 2024-09-21T03:45:19.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.165+0000 7f98fb8a0740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d77f8153e0 0x55d77fbe75c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:19.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.165+0000 7f98fb8a0740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d77f81d5c0 0x55d77fc10280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:19.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.165+0000 7f98fb8a0740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55d77f83e2a0 0x55d77fc107c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:19.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.165+0000 7f98fb8a0740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f98f01ffa60 con 0x55d77f81d5c0 2024-09-21T03:45:19.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.165+0000 7f98fb8a0740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f98f009f620 con 0x55d77f8153e0 2024-09-21T03:45:19.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.165+0000 7f98fb8a0740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f98f01ff3f0 con 0x55d77f83e2a0 2024-09-21T03:45:19.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.165+0000 7f98edd9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d77f81d5c0 0x55d77fc10280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:19.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.165+0000 7f98edd9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d77f81d5c0 0x55d77fc10280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46570/0 (socket says 172.21.15.38:46570) 2024-09-21T03:45:19.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.165+0000 7f98edd9b640 1 -- 172.21.15.38:0/1237503465 learned_addr learned my addr 172.21.15.38:0/1237503465 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:45:19.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.165+0000 7f98ee59c640 1 --2- 172.21.15.38:0/1237503465 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d77f8153e0 0x55d77fbe75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:19.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.165+0000 7f98eed9d640 1 --2- 172.21.15.38:0/1237503465 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55d77f83e2a0 0x55d77fc107c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:19.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.165+0000 7f98edd9b640 1 -- 172.21.15.38:0/1237503465 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55d77f83e2a0 msgr2=0x55d77fc107c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:19.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.165+0000 7f98edd9b640 1 --2- 172.21.15.38:0/1237503465 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55d77f83e2a0 0x55d77fc107c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:19.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.165+0000 7f98edd9b640 1 -- 172.21.15.38:0/1237503465 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d77f8153e0 msgr2=0x55d77fbe75c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:19.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.165+0000 7f98edd9b640 1 --2- 172.21.15.38:0/1237503465 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d77f8153e0 0x55d77fbe75c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:19.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.165+0000 7f98edd9b640 1 -- 172.21.15.38:0/1237503465 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d77fc0a2f0 con 0x55d77f81d5c0 2024-09-21T03:45:19.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.165+0000 7f98eed9d640 1 --2- 172.21.15.38:0/1237503465 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55d77f83e2a0 0x55d77fc107c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:45:19.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.165+0000 7f98edd9b640 1 --2- 172.21.15.38:0/1237503465 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d77f81d5c0 0x55d77fc10280 secure :-1 s=READY pgs=2451 cs=0 l=1 rev1=1 crypto rx=0x7f98d807fc10 tx=0x7f98d8040040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8152edfb51c28bc0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:19.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.165+0000 7f98cf7fe640 1 -- 172.21.15.38:0/1237503465 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98d8087480 con 0x55d77f81d5c0 2024-09-21T03:45:19.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.166+0000 7f98cf7fe640 1 -- 172.21.15.38:0/1237503465 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f98d80a0020 con 0x55d77f81d5c0 2024-09-21T03:45:19.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.166+0000 7f98cf7fe640 1 -- 172.21.15.38:0/1237503465 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98d80829a0 con 0x55d77f81d5c0 2024-09-21T03:45:19.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.166+0000 7f98fb8a0740 1 -- 172.21.15.38:0/1237503465 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d77f81d5c0 msgr2=0x55d77fc10280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:19.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.166+0000 7f98fb8a0740 1 --2- 172.21.15.38:0/1237503465 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d77f81d5c0 0x55d77fc10280 secure :-1 s=READY pgs=2451 cs=0 l=1 rev1=1 crypto rx=0x7f98d807fc10 tx=0x7f98d8040040 comp rx=0 tx=0).stop 2024-09-21T03:45:19.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.166+0000 7f98fb8a0740 1 -- 172.21.15.38:0/1237503465 shutdown_connections 2024-09-21T03:45:19.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.166+0000 7f98fb8a0740 1 --2- 172.21.15.38:0/1237503465 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55d77f83e2a0 0x55d77fc107c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:19.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.166+0000 7f98fb8a0740 1 --2- 172.21.15.38:0/1237503465 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d77f81d5c0 0x55d77fc10280 unknown :-1 s=CLOSED pgs=2451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:19.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.166+0000 7f98fb8a0740 1 --2- 172.21.15.38:0/1237503465 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d77f8153e0 0x55d77fbe75c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:19.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.166+0000 7f98fb8a0740 1 -- 172.21.15.38:0/1237503465 >> 172.21.15.38:0/1237503465 conn(0x55d77f8323f0 msgr2=0x55d77fbe64e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:19.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.166+0000 7f98fb8a0740 1 -- 172.21.15.38:0/1237503465 shutdown_connections 2024-09-21T03:45:19.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.166+0000 7f98fb8a0740 1 -- 172.21.15.38:0/1237503465 wait complete. 2024-09-21T03:45:19.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.167+0000 7f98fb8a0740 1 -- start start 2024-09-21T03:45:19.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.167+0000 7f98fb8a0740 1 Processor -- start 2024-09-21T03:45:19.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.169+0000 7f98fb8a0740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d77f8153e0 0x55d77fc10840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:19.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.169+0000 7f98fb8a0740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d77f81d5c0 0x55d77fc2d540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:19.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.169+0000 7f98fb8a0740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55d77f83e2a0 0x55d77fca9ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:19.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.169+0000 7f98fb8a0740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55d77fc23390 con 0x55d77f81d5c0 2024-09-21T03:45:19.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.169+0000 7f98fb8a0740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55d77fb58910 con 0x55d77f8153e0 2024-09-21T03:45:19.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.169+0000 7f98fb8a0740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55d77fc3d680 con 0x55d77f83e2a0 2024-09-21T03:45:19.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.169+0000 7f98fb8a0740 10 client.0 _ll_register_callbacks cb 0x7f98e8009610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:45:19.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.169+0000 7f98edd9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d77f81d5c0 0x55d77fc2d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:19.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.169+0000 7f98edd9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d77f81d5c0 0x55d77fc2d540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46572/0 (socket says 172.21.15.38:46572) 2024-09-21T03:45:19.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.169+0000 7f98edd9b640 1 -- 172.21.15.38:0/4031461866 learned_addr learned my addr 172.21.15.38:0/4031461866 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:45:19.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.169+0000 7f98ee59c640 1 --2- 172.21.15.38:0/4031461866 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d77f8153e0 0x55d77fc10840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:19.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.169+0000 7f98eed9d640 1 --2- 172.21.15.38:0/4031461866 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55d77f83e2a0 0x55d77fca9ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:19.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.169+0000 7f98edd9b640 1 -- 172.21.15.38:0/4031461866 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55d77f83e2a0 msgr2=0x55d77fca9ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:19.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.169+0000 7f98edd9b640 1 --2- 172.21.15.38:0/4031461866 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55d77f83e2a0 0x55d77fca9ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:19.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.169+0000 7f98edd9b640 1 -- 172.21.15.38:0/4031461866 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d77f8153e0 msgr2=0x55d77fc10840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:19.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.169+0000 7f98edd9b640 1 --2- 172.21.15.38:0/4031461866 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d77f8153e0 0x55d77fc10840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:19.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.169+0000 7f98edd9b640 1 -- 172.21.15.38:0/4031461866 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d77fc4d090 con 0x55d77f81d5c0 2024-09-21T03:45:19.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.169+0000 7f98edd9b640 1 --2- 172.21.15.38:0/4031461866 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d77f81d5c0 0x55d77fc2d540 secure :-1 s=READY pgs=2452 cs=0 l=1 rev1=1 crypto rx=0x7f98d8006650 tx=0x7f98d8087390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3619b55f4b3f06e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:19.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.170+0000 7f98cd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-21T03:45:19.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.170+0000 7f98cd7fa640 1 -- 172.21.15.38:0/4031461866 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98d8087480 con 0x55d77f81d5c0 2024-09-21T03:45:19.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.170+0000 7f98cd7fa640 1 -- 172.21.15.38:0/4031461866 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(149..149 src has 1..149) ==== 6303+0+0 (secure 0 0 0) 0x7f98d80a1070 con 0x55d77f81d5c0 2024-09-21T03:45:19.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.170+0000 7f98fb8a0740 1 -- 172.21.15.38:0/4031461866 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f98d80a0020 con 0x55d77f81d5c0 2024-09-21T03:45:19.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.170+0000 7f98cd7fa640 1 -- 172.21.15.38:0/4031461866 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=349) ==== 24+0+0 (secure 0 0 0) 0x7f98d808bf30 con 0x55d77f81d5c0 2024-09-21T03:45:19.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.170+0000 7f98fb8a0740 10 client.11064 fetch_fsmap learned FSMap version 349 2024-09-21T03:45:19.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.170+0000 7f98fb8a0740 1 -- 172.21.15.38:0/4031461866 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=349,monmap=2+}) -- 0x55d77fc4d090 con 0x55d77f81d5c0 2024-09-21T03:45:19.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.170+0000 7f98cd7fa640 1 -- 172.21.15.38:0/4031461866 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f98d80a0020 con 0x55d77f81d5c0 2024-09-21T03:45:19.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.170+0000 7f98cd7fa640 1 -- 172.21.15.38:0/4031461866 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 349) ==== 3502+0+0 (secure 0 0 0) 0x7f98d8098b50 con 0x55d77f81d5c0 2024-09-21T03:45:19.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.170+0000 7f98fb8a0740 10 client.11064 fetch_fsmap finished waiting for FSMap version 349 2024-09-21T03:45:19.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.170+0000 7f98fb8a0740 10 client.11064 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T03:45:19.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.170+0000 7f98fb8a0740 20 client.11064 populate_metadata read hostname 'smithi038' 2024-09-21T03:45:19.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.170+0000 7f98fb8a0740 1 --2- 172.21.15.38:0/4031461866 >> [v2:172.21.15.38:6832/3839260519,v1:172.21.15.38:6833/3839260519] conn(0x55d77fcad6a0 0x55d77fcadab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:19.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.170+0000 7f98fb8a0740 4 client.11064 mds_command: new command op to 10935 tid=0 multi_id=0 [{"prefix": "damage rm", "damage_id": 1336616810}] 2024-09-21T03:45:19.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.170+0000 7f98fb8a0740 1 -- 172.21.15.38:0/4031461866 --> [v2:172.21.15.38:6832/3839260519,v1:172.21.15.38:6833/3839260519] -- command(tid 0: {"prefix": "damage rm", "damage_id": 1336616810}) -- 0x55d77fb56310 con 0x55d77fcad6a0 2024-09-21T03:45:19.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.170+0000 7f98ee59c640 1 --2- 172.21.15.38:0/4031461866 >> [v2:172.21.15.38:6832/3839260519,v1:172.21.15.38:6833/3839260519] conn(0x55d77fcad6a0 0x55d77fcadab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:19.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.171+0000 7f98cd7fa640 10 client.11064 ms_handle_connect on v2:172.21.15.38:6832/3839260519 2024-09-21T03:45:19.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.171+0000 7f98ee59c640 1 --2- 172.21.15.38:0/4031461866 >> [v2:172.21.15.38:6832/3839260519,v1:172.21.15.38:6833/3839260519] conn(0x55d77fcad6a0 0x55d77fcadab0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c33973e854ee91cd server_cookie=d08bd382d5aa9346 in_seq=0 out_seq=0 2024-09-21T03:45:19.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.171+0000 7f98cd7fa640 1 -- 172.21.15.38:0/4031461866 <== mds.0 v2:172.21.15.38:6832/3839260519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+0 (crc 0 0 0) 0x55d77fb56310 con 0x55d77fcad6a0 2024-09-21T03:45:19.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.171+0000 7f98cd7fa640 10 client.11064 handle_command_reply: tid=0 2024-09-21T03:45:19.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.171+0000 7f98fb8a0740 1 client.11064 shutdown 2024-09-21T03:45:19.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.171+0000 7f98fb8a0740 1 -- 172.21.15.38:0/4031461866 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d77f81d5c0 msgr2=0x55d77fc2d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:19.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.171+0000 7f98fb8a0740 1 --2- 172.21.15.38:0/4031461866 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d77f81d5c0 0x55d77fc2d540 secure :-1 s=READY pgs=2452 cs=0 l=1 rev1=1 crypto rx=0x7f98d8006650 tx=0x7f98d8087390 comp rx=0 tx=0).stop 2024-09-21T03:45:19.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.171+0000 7f98fb8a0740 1 -- 172.21.15.38:0/4031461866 shutdown_connections 2024-09-21T03:45:19.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.171+0000 7f98fb8a0740 1 --2- 172.21.15.38:0/4031461866 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55d77f83e2a0 0x55d77fca9ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:19.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.171+0000 7f98fb8a0740 1 --2- 172.21.15.38:0/4031461866 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d77f81d5c0 0x55d77fc2d540 unknown :-1 s=CLOSED pgs=2452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:19.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.171+0000 7f98fb8a0740 1 --2- 172.21.15.38:0/4031461866 >> [v2:172.21.15.38:6832/3839260519,v1:172.21.15.38:6833/3839260519] conn(0x55d77fcad6a0 0x55d77fcadab0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:19.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.171+0000 7f98fb8a0740 1 --2- 172.21.15.38:0/4031461866 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d77f8153e0 0x55d77fc10840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:19.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.171+0000 7f98fb8a0740 1 -- 172.21.15.38:0/4031461866 >> 172.21.15.38:0/4031461866 conn(0x55d77f8323f0 msgr2=0x55d77fc0cdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:19.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.171+0000 7f98cd7fa640 0 client.11064 ms_handle_reset on v2:172.21.15.38:6832/3839260519 2024-09-21T03:45:19.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.171+0000 7f98fb8a0740 1 -- 172.21.15.38:0/4031461866 shutdown_connections 2024-09-21T03:45:19.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.171+0000 7f98fb8a0740 1 -- 172.21.15.38:0/4031461866 wait complete. 2024-09-21T03:45:19.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.171+0000 7f98fb8a0740 20 client.11064 trim_cache size 0 max 16384 2024-09-21T03:45:19.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.172+0000 7f98f4932640 1 -- 172.21.15.38:0/983876041 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f98d0078b90 msgr2=0x7f98d007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:19.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.172+0000 7f98f4932640 1 --2- 172.21.15.38:0/983876041 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f98d0078b90 0x7f98d007b0a0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f98e0001be0 tx=0x7f98e0001490 comp rx=0 tx=0).stop 2024-09-21T03:45:19.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.172+0000 7f98f4932640 1 -- 172.21.15.38:0/983876041 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f98f00af1c0 msgr2=0x7f98f00b50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:19.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.172+0000 7f98f4932640 1 --2- 172.21.15.38:0/983876041 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f98f00af1c0 0x7f98f00b50e0 secure :-1 s=READY pgs=2448 cs=0 l=1 rev1=1 crypto rx=0x7f98d8000f60 tx=0x7f98d8006940 comp rx=0 tx=0).stop 2024-09-21T03:45:19.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.172+0000 7f98f4932640 1 -- 172.21.15.38:0/983876041 shutdown_connections 2024-09-21T03:45:19.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.172+0000 7f98f4932640 1 --2- 172.21.15.38:0/983876041 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f98f00b5620 0x7f98f00cdc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:19.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.172+0000 7f98f4932640 1 --2- 172.21.15.38:0/983876041 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f98f00af1c0 0x7f98f00b50e0 unknown :-1 s=CLOSED pgs=2448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:19.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.172+0000 7f98f4932640 1 --2- 172.21.15.38:0/983876041 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f98d0078b90 0x7f98d007b0a0 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:19.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.172+0000 7f98f4932640 1 --2- 172.21.15.38:0/983876041 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f98dc004800 0x7f98f00b0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:19.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.172+0000 7f98f4932640 1 -- 172.21.15.38:0/983876041 >> 172.21.15.38:0/983876041 conn(0x7f98f00b1b90 msgr2=0x7f98f015adf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:19.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.172+0000 7f98f4932640 1 -- 172.21.15.38:0/983876041 shutdown_connections 2024-09-21T03:45:19.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.172+0000 7f98f4932640 1 -- 172.21.15.38:0/983876041 wait complete. 2024-09-21T03:45:19.185 DEBUG:teuthology.orchestra.run.smithi038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rm 100000001f6.00000000 2024-09-21T03:45:19.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.238+0000 7ff08a668f40 1 Processor -- start 2024-09-21T03:45:19.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.238+0000 7ff08a668f40 1 -- start start 2024-09-21T03:45:19.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.240+0000 7ff08a668f40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f09e4e8240 0x55f09e5a0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:19.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.240+0000 7ff08a668f40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f09e52f090 0x55f09e513c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:19.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.240+0000 7ff08a668f40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f09e5a4520 0x55f09e5bc800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:19.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.240+0000 7ff08a668f40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55f09e435d60 con 0x55f09e4e8240 2024-09-21T03:45:19.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.240+0000 7ff08a668f40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55f09e494430 con 0x55f09e5a4520 2024-09-21T03:45:19.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.240+0000 7ff08a668f40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55f09e5bbaa0 con 0x55f09e52f090 2024-09-21T03:45:19.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.240+0000 7ff084124640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f09e4e8240 0x55f09e5a0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:19.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.240+0000 7ff084124640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f09e4e8240 0x55f09e5a0ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46590/0 (socket says 172.21.15.38:46590) 2024-09-21T03:45:19.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.240+0000 7ff084124640 1 -- 172.21.15.38:0/4210020780 learned_addr learned my addr 172.21.15.38:0/4210020780 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:45:19.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.240+0000 7ff083923640 1 --2- 172.21.15.38:0/4210020780 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f09e52f090 0x55f09e513c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:19.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.240+0000 7ff084925640 1 --2- 172.21.15.38:0/4210020780 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f09e5a4520 0x55f09e5bc800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:19.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.240+0000 7ff084124640 1 -- 172.21.15.38:0/4210020780 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f09e52f090 msgr2=0x55f09e513c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:19.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.240+0000 7ff084124640 1 --2- 172.21.15.38:0/4210020780 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f09e52f090 0x55f09e513c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:19.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.240+0000 7ff084124640 1 -- 172.21.15.38:0/4210020780 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f09e5a4520 msgr2=0x55f09e5bc800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:19.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.240+0000 7ff084124640 1 --2- 172.21.15.38:0/4210020780 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f09e5a4520 0x55f09e5bc800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:19.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.240+0000 7ff084124640 1 -- 172.21.15.38:0/4210020780 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f09e52caf0 con 0x55f09e4e8240 2024-09-21T03:45:19.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.240+0000 7ff084124640 1 --2- 172.21.15.38:0/4210020780 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f09e4e8240 0x55f09e5a0ca0 secure :-1 s=READY pgs=2454 cs=0 l=1 rev1=1 crypto rx=0x7ff07400c580 tx=0x7ff07400c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=438fdd75a5ea705d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:19.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.241+0000 7ff082921640 1 -- 172.21.15.38:0/4210020780 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff07400a9e0 con 0x55f09e4e8240 2024-09-21T03:45:19.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.241+0000 7ff082921640 1 -- 172.21.15.38:0/4210020780 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff07400ac70 con 0x55f09e4e8240 2024-09-21T03:45:19.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.241+0000 7ff082921640 1 -- 172.21.15.38:0/4210020780 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff074008680 con 0x55f09e4e8240 2024-09-21T03:45:19.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.241+0000 7ff08a668f40 1 -- 172.21.15.38:0/4210020780 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f09e4e8240 msgr2=0x55f09e5a0ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:19.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.241+0000 7ff08a668f40 1 --2- 172.21.15.38:0/4210020780 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f09e4e8240 0x55f09e5a0ca0 secure :-1 s=READY pgs=2454 cs=0 l=1 rev1=1 crypto rx=0x7ff07400c580 tx=0x7ff07400c980 comp rx=0 tx=0).stop 2024-09-21T03:45:19.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.241+0000 7ff08a668f40 1 -- 172.21.15.38:0/4210020780 shutdown_connections 2024-09-21T03:45:19.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.241+0000 7ff08a668f40 1 --2- 172.21.15.38:0/4210020780 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f09e5a4520 0x55f09e5bc800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:19.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.241+0000 7ff08a668f40 1 --2- 172.21.15.38:0/4210020780 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f09e52f090 0x55f09e513c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:19.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.241+0000 7ff08a668f40 1 --2- 172.21.15.38:0/4210020780 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f09e4e8240 0x55f09e5a0ca0 unknown :-1 s=CLOSED pgs=2454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:19.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.241+0000 7ff08a668f40 1 -- 172.21.15.38:0/4210020780 >> 172.21.15.38:0/4210020780 conn(0x55f09e406ac0 msgr2=0x55f09e4e6cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:19.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.241+0000 7ff08a668f40 1 -- 172.21.15.38:0/4210020780 shutdown_connections 2024-09-21T03:45:19.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.241+0000 7ff08a668f40 1 -- 172.21.15.38:0/4210020780 wait complete. 2024-09-21T03:45:19.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.242+0000 7ff08a668f40 1 Processor -- start 2024-09-21T03:45:19.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.242+0000 7ff08a668f40 1 -- start start 2024-09-21T03:45:19.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.243+0000 7ff08a668f40 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f09e4e8240 0x55f09e594070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:19.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.243+0000 7ff08a668f40 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f09e52f090 0x55f09e5bc640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:19.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.243+0000 7ff08a668f40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f09e5a4520 0x55f09e5bcb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:19.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.243+0000 7ff08a668f40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55f09e52c8a0 con 0x55f09e5a4520 2024-09-21T03:45:19.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.243+0000 7ff08a668f40 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55f09e4e75a0 con 0x55f09e4e8240 2024-09-21T03:45:19.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.243+0000 7ff08a668f40 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55f09e5a3c10 con 0x55f09e52f090 2024-09-21T03:45:19.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.244+0000 7ff084925640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f09e5a4520 0x55f09e5bcb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:19.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.244+0000 7ff084925640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f09e5a4520 0x55f09e5bcb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46600/0 (socket says 172.21.15.38:46600) 2024-09-21T03:45:19.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.244+0000 7ff084925640 1 -- 172.21.15.38:0/1257577748 learned_addr learned my addr 172.21.15.38:0/1257577748 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:45:19.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.244+0000 7ff084124640 1 --2- 172.21.15.38:0/1257577748 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f09e4e8240 0x55f09e594070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:19.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.244+0000 7ff083923640 1 --2- 172.21.15.38:0/1257577748 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f09e52f090 0x55f09e5bc640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:19.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.244+0000 7ff084925640 1 -- 172.21.15.38:0/1257577748 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f09e52f090 msgr2=0x55f09e5bc640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:19.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.244+0000 7ff084925640 1 --2- 172.21.15.38:0/1257577748 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f09e52f090 0x55f09e5bc640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:19.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.244+0000 7ff084925640 1 -- 172.21.15.38:0/1257577748 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f09e4e8240 msgr2=0x55f09e594070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:19.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.244+0000 7ff084925640 1 --2- 172.21.15.38:0/1257577748 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f09e4e8240 0x55f09e594070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:19.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.244+0000 7ff084925640 1 -- 172.21.15.38:0/1257577748 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f09e5b8a30 con 0x55f09e5a4520 2024-09-21T03:45:19.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.244+0000 7ff084925640 1 --2- 172.21.15.38:0/1257577748 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f09e5a4520 0x55f09e5bcb80 secure :-1 s=READY pgs=2455 cs=0 l=1 rev1=1 crypto rx=0x7ff07c00d190 tx=0x7ff07c00d590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:19.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.244+0000 7ff08191f640 1 -- 172.21.15.38:0/1257577748 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff07c006020 con 0x55f09e5a4520 2024-09-21T03:45:19.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.244+0000 7ff08191f640 1 -- 172.21.15.38:0/1257577748 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff07c0061c0 con 0x55f09e5a4520 2024-09-21T03:45:19.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.244+0000 7ff08a668f40 1 -- 172.21.15.38:0/1257577748 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55f09e5b58f0 con 0x55f09e5a4520 2024-09-21T03:45:19.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.244+0000 7ff08191f640 1 -- 172.21.15.38:0/1257577748 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff07c013480 con 0x55f09e5a4520 2024-09-21T03:45:19.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.244+0000 7ff08a668f40 1 -- 172.21.15.38:0/1257577748 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f09e5b5eb0 con 0x55f09e5a4520 2024-09-21T03:45:19.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.246+0000 7ff08191f640 1 -- 172.21.15.38:0/1257577748 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ff07c01a030 con 0x55f09e5a4520 2024-09-21T03:45:19.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.247+0000 7ff08191f640 1 --2- 172.21.15.38:0/1257577748 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff070077570 0x7ff070079a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:19.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.247+0000 7ff08191f640 1 -- 172.21.15.38:0/1257577748 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 6303+0+0 (secure 0 0 0) 0x7ff07c0990f0 con 0x55f09e5a4520 2024-09-21T03:45:19.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.247+0000 7ff084124640 1 --2- 172.21.15.38:0/1257577748 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff070077570 0x7ff070079a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:19.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.247+0000 7ff08a668f40 1 --2- 172.21.15.38:0/1257577748 >> [v2:172.21.15.195:6810/2080960180,v1:172.21.15.195:6811/2080960180] conn(0x55f09e5a0710 0x55f09e6583f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:19.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.247+0000 7ff08a668f40 1 -- 172.21.15.38:0/1257577748 --> [v2:172.21.15.195:6810/2080960180,v1:172.21.15.195:6811/2080960180] -- osd_op(unknown.0.0:1 15.a 15:57fe7535:::100000001f6.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e149) -- 0x55f09e5945b0 con 0x55f09e5a0710 2024-09-21T03:45:19.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.247+0000 7ff083923640 1 --2- 172.21.15.38:0/1257577748 >> [v2:172.21.15.195:6810/2080960180,v1:172.21.15.195:6811/2080960180] conn(0x55f09e5a0710 0x55f09e6583f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:19.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.247+0000 7ff084124640 1 --2- 172.21.15.38:0/1257577748 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff070077570 0x7ff070079a80 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7ff07400cfd0 tx=0x7ff074018040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:19.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.248+0000 7ff083923640 1 --2- 172.21.15.38:0/1257577748 >> [v2:172.21.15.195:6810/2080960180,v1:172.21.15.195:6811/2080960180] conn(0x55f09e5a0710 0x55f09e6583f0 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:19.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.249+0000 7ff083923640 1 -- 172.21.15.38:0/1257577748 <== osd.5 v2:172.21.15.195:6810/2080960180 1 ==== osd_op_reply(1 100000001f6.00000000 [delete] v149'3 uv3 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7ff07804ec80 con 0x55f09e5a0710 2024-09-21T03:45:19.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.249+0000 7ff08a668f40 1 -- 172.21.15.38:0/1257577748 >> [v2:172.21.15.195:6810/2080960180,v1:172.21.15.195:6811/2080960180] conn(0x55f09e5a0710 msgr2=0x55f09e6583f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:19.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.249+0000 7ff08a668f40 1 --2- 172.21.15.38:0/1257577748 >> [v2:172.21.15.195:6810/2080960180,v1:172.21.15.195:6811/2080960180] conn(0x55f09e5a0710 0x55f09e6583f0 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:19.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.249+0000 7ff08a668f40 1 -- 172.21.15.38:0/1257577748 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff070077570 msgr2=0x7ff070079a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:19.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.249+0000 7ff08a668f40 1 --2- 172.21.15.38:0/1257577748 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff070077570 0x7ff070079a80 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7ff07400cfd0 tx=0x7ff074018040 comp rx=0 tx=0).stop 2024-09-21T03:45:19.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.249+0000 7ff08a668f40 1 -- 172.21.15.38:0/1257577748 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f09e5a4520 msgr2=0x55f09e5bcb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:19.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.249+0000 7ff08a668f40 1 --2- 172.21.15.38:0/1257577748 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f09e5a4520 0x55f09e5bcb80 secure :-1 s=READY pgs=2455 cs=0 l=1 rev1=1 crypto rx=0x7ff07c00d190 tx=0x7ff07c00d590 comp rx=0 tx=0).stop 2024-09-21T03:45:19.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.249+0000 7ff084925640 1 -- 172.21.15.38:0/1257577748 reap_dead start 2024-09-21T03:45:19.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.249+0000 7ff08a668f40 1 -- 172.21.15.38:0/1257577748 shutdown_connections 2024-09-21T03:45:19.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.249+0000 7ff08a668f40 1 -- 172.21.15.38:0/1257577748 >> 172.21.15.38:0/1257577748 conn(0x55f09e406ac0 msgr2=0x55f09e5a0cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:19.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.250+0000 7ff08a668f40 1 -- 172.21.15.38:0/1257577748 shutdown_connections 2024-09-21T03:45:19.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.250+0000 7ff08a668f40 1 -- 172.21.15.38:0/1257577748 wait complete. 2024-09-21T03:45:19.255 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:45:19.255 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat testdir/hardlink2) 2024-09-21T03:45:19.272 INFO:tasks.ceph.mds.c.smithi038.stderr:2024-09-21T03:45:19.270+0000 7f6c65fd1640 -1 log_channel(cluster) log [ERR] : dir 0x100000001f6 object missing on disk; some files may be lost (/dir2) 2024-09-21T03:45:19.273 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T03:45:19.273 INFO:teuthology.orchestra.run.smithi038.stderr:stat: cannot statx 'testdir/hardlink2': Input/output error 2024-09-21T03:45:19.274 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:45:19.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.427+0000 7ffa084ec640 1 -- 172.21.15.38:0/1939843431 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffa000b2e40 msgr2=0x7ffa000b3250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:19.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.427+0000 7ffa084ec640 1 --2- 172.21.15.38:0/1939843431 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffa000b2e40 0x7ffa000b3250 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7ff9f8005fd0 tx=0x7ff9f802a8b0 comp rx=0 tx=0).stop 2024-09-21T03:45:19.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.428+0000 7ffa084ec640 1 -- 172.21.15.38:0/1939843431 shutdown_connections 2024-09-21T03:45:19.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.428+0000 7ffa084ec640 1 --2- 172.21.15.38:0/1939843431 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffa000b2e40 0x7ffa000b3250 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:19.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.428+0000 7ffa084ec640 1 --2- 172.21.15.38:0/1939843431 >> v2:172.21.15.195:3300/0 conn(0x7ffa00162280 0x7ffa00162690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:19.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.428+0000 7ffa084ec640 1 -- 172.21.15.38:0/1939843431 >> 172.21.15.38:0/1939843431 conn(0x7ffa000b6b20 msgr2=0x7ffa000d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:19.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.428+0000 7ffa084ec640 1 -- 172.21.15.38:0/1939843431 shutdown_connections 2024-09-21T03:45:19.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.428+0000 7ffa084ec640 1 -- 172.21.15.38:0/1939843431 wait complete. 2024-09-21T03:45:19.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.429+0000 7ffa084ec640 1 Processor -- start 2024-09-21T03:45:19.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.429+0000 7ffa084ec640 1 -- start start 2024-09-21T03:45:19.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.431+0000 7ffa084ec640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffa000b2e40 0x7ffa000acf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:19.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.431+0000 7ffa084ec640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffa00162280 0x7ffa001f8d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:19.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.431+0000 7ffa084ec640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffa001f92a0 0x7ffa0014c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:19.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.431+0000 7ffa084ec640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ffa0015e4b0 con 0x7ffa00162280 2024-09-21T03:45:19.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.431+0000 7ffa084ec640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ffa000b39d0 con 0x7ffa000b2e40 2024-09-21T03:45:19.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.431+0000 7ffa084ec640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7ffa0015e630 con 0x7ffa001f92a0 2024-09-21T03:45:19.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.431+0000 7ffa05a87640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffa00162280 0x7ffa001f8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:19.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.431+0000 7ffa05a87640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffa00162280 0x7ffa001f8d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46610/0 (socket says 172.21.15.38:46610) 2024-09-21T03:45:19.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.431+0000 7ffa05a87640 1 -- 172.21.15.38:0/809633907 learned_addr learned my addr 172.21.15.38:0/809633907 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:45:19.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.431+0000 7ffa06288640 1 --2- 172.21.15.38:0/809633907 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffa000b2e40 0x7ffa000acf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:19.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.431+0000 7ffa06a89640 1 --2- 172.21.15.38:0/809633907 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffa001f92a0 0x7ffa0014c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:19.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.431+0000 7ffa05a87640 1 -- 172.21.15.38:0/809633907 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffa001f92a0 msgr2=0x7ffa0014c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:19.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.431+0000 7ffa05a87640 1 --2- 172.21.15.38:0/809633907 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffa001f92a0 0x7ffa0014c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:19.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.431+0000 7ffa05a87640 1 -- 172.21.15.38:0/809633907 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffa000b2e40 msgr2=0x7ffa000acf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:19.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.431+0000 7ffa05a87640 1 --2- 172.21.15.38:0/809633907 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffa000b2e40 0x7ffa000acf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:19.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.431+0000 7ffa05a87640 1 -- 172.21.15.38:0/809633907 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa000ad870 con 0x7ffa00162280 2024-09-21T03:45:19.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.431+0000 7ffa06288640 1 --2- 172.21.15.38:0/809633907 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffa000b2e40 0x7ffa000acf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:45:19.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.431+0000 7ffa05a87640 1 --2- 172.21.15.38:0/809633907 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffa00162280 0x7ffa001f8d60 secure :-1 s=READY pgs=2456 cs=0 l=1 rev1=1 crypto rx=0x7ff9f000e9c0 tx=0x7ff9f000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:19.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.432+0000 7ff9f6ffd640 1 -- 172.21.15.38:0/809633907 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9f000cd80 con 0x7ffa00162280 2024-09-21T03:45:19.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.432+0000 7ff9f6ffd640 1 -- 172.21.15.38:0/809633907 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff9f0004020 con 0x7ffa00162280 2024-09-21T03:45:19.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.432+0000 7ffa084ec640 1 -- 172.21.15.38:0/809633907 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa000adb50 con 0x7ffa00162280 2024-09-21T03:45:19.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.432+0000 7ff9f6ffd640 1 -- 172.21.15.38:0/809633907 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9f0004450 con 0x7ffa00162280 2024-09-21T03:45:19.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.432+0000 7ffa084ec640 1 -- 172.21.15.38:0/809633907 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa000ae110 con 0x7ffa00162280 2024-09-21T03:45:19.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.433+0000 7ffa084ec640 1 -- 172.21.15.38:0/809633907 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa000d1d00 con 0x7ffa00162280 2024-09-21T03:45:19.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.433+0000 7ff9f6ffd640 1 -- 172.21.15.38:0/809633907 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ff9f00035f0 con 0x7ffa00162280 2024-09-21T03:45:19.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.434+0000 7ff9f6ffd640 1 --2- 172.21.15.38:0/809633907 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff9e4078b90 0x7ff9e407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:19.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.434+0000 7ff9f6ffd640 1 -- 172.21.15.38:0/809633907 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 6303+0+0 (secure 0 0 0) 0x7ff9f009b9c0 con 0x7ffa00162280 2024-09-21T03:45:19.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.434+0000 7ffa06288640 1 --2- 172.21.15.38:0/809633907 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff9e4078b90 0x7ff9e407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:19.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.435+0000 7ffa06288640 1 --2- 172.21.15.38:0/809633907 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff9e4078b90 0x7ff9e407b0a0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc000f60 tx=0x7ff9fc009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:19.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.439+0000 7ff9f6ffd640 1 -- 172.21.15.38:0/809633907 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7ffa000d1d00 con 0x7ffa00162280 2024-09-21T03:45:19.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.664+0000 7ffa084ec640 1 -- 172.21.15.38:0/809633907 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffa0006d910 con 0x7ffa00162280 2024-09-21T03:45:19.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.664+0000 7ff9f6ffd640 1 -- 172.21.15.38:0/809633907 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 349 v349) ==== 95+0+5241 (secure 0 0 0) 0x7ff9f009de20 con 0x7ffa00162280 2024-09-21T03:45:19.670 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:45:19.670 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":349,"btime":"2024-09-21T03:45:15:115320+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8399,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/3606957623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":3606957623},{"type":"v1","addr":"172.21.15.38:6835","nonce":3606957623}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":338},{"gid":8405,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/456651486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":456651486},{"type":"v1","addr":"172.21.15.195:6837","nonce":456651486}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":338},{"gid":9604,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/2649311840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":2649311840},{"type":"v1","addr":"172.21.15.195:6835","nonce":2649311840}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":338}],"filesystems":[{"mdsmap":{"epoch":349,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:44:56.710446+0000","modified":"2024-09-21T03:45:15.115308+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":147,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10935},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10935":{"gid":10935,"name":"c","rank":0,"incarnation":346,"state":"up:active","state_seq":4,"addr":"172.21.15.38:6833/3839260519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":3839260519},{"type":"v1","addr":"172.21.15.38:6833","nonce":3839260519}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10935,"qdb_cluster":[10935]},"id":7}]} 2024-09-21T03:45:19.671 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 349 2024-09-21T03:45:19.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.666+0000 7ffa084ec640 1 -- 172.21.15.38:0/809633907 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff9e4078b90 msgr2=0x7ff9e407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:19.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.666+0000 7ffa084ec640 1 --2- 172.21.15.38:0/809633907 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff9e4078b90 0x7ff9e407b0a0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc000f60 tx=0x7ff9fc009040 comp rx=0 tx=0).stop 2024-09-21T03:45:19.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.666+0000 7ffa084ec640 1 -- 172.21.15.38:0/809633907 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffa00162280 msgr2=0x7ffa001f8d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:19.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.666+0000 7ffa084ec640 1 --2- 172.21.15.38:0/809633907 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffa00162280 0x7ffa001f8d60 secure :-1 s=READY pgs=2456 cs=0 l=1 rev1=1 crypto rx=0x7ff9f000e9c0 tx=0x7ff9f000ee80 comp rx=0 tx=0).stop 2024-09-21T03:45:19.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.667+0000 7ffa084ec640 1 -- 172.21.15.38:0/809633907 shutdown_connections 2024-09-21T03:45:19.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.667+0000 7ffa084ec640 1 --2- 172.21.15.38:0/809633907 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff9e4078b90 0x7ff9e407b0a0 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:19.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.667+0000 7ffa084ec640 1 --2- 172.21.15.38:0/809633907 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffa001f92a0 0x7ffa0014c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:19.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.667+0000 7ffa084ec640 1 --2- 172.21.15.38:0/809633907 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffa00162280 0x7ffa001f8d60 unknown :-1 s=CLOSED pgs=2456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:19.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.667+0000 7ffa084ec640 1 --2- 172.21.15.38:0/809633907 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffa000b2e40 0x7ffa000acf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:19.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.667+0000 7ffa084ec640 1 -- 172.21.15.38:0/809633907 >> 172.21.15.38:0/809633907 conn(0x7ffa000b6b20 msgr2=0x7ffa0006cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:19.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.667+0000 7ffa084ec640 1 -- 172.21.15.38:0/809633907 shutdown_connections 2024-09-21T03:45:19.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.667+0000 7ffa084ec640 1 -- 172.21.15.38:0/809633907 wait complete. 2024-09-21T03:45:19.681 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c damage ls --format=json-pretty 2024-09-21T03:45:19.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.797+0000 7f7af8cb6640 1 -- 172.21.15.38:0/3268462339 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7af40ceb90 msgr2=0x7f7af40cefa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:19.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.797+0000 7f7af8cb6640 1 --2- 172.21.15.38:0/3268462339 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7af40ceb90 0x7f7af40cefa0 secure :-1 s=READY pgs=2458 cs=0 l=1 rev1=1 crypto rx=0x7f7aec009830 tx=0x7f7aec02a4a0 comp rx=0 tx=0).stop 2024-09-21T03:45:19.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.797+0000 7f7af8cb6640 1 -- 172.21.15.38:0/3268462339 shutdown_connections 2024-09-21T03:45:19.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.797+0000 7f7af8cb6640 1 --2- 172.21.15.38:0/3268462339 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7af40ceb90 0x7f7af40cefa0 unknown :-1 s=CLOSED pgs=2458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:19.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.797+0000 7f7af8cb6640 1 -- 172.21.15.38:0/3268462339 >> 172.21.15.38:0/3268462339 conn(0x7f7af409d830 msgr2=0x7f7af40b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:19.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.797+0000 7f7af8cb6640 1 -- 172.21.15.38:0/3268462339 shutdown_connections 2024-09-21T03:45:19.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.797+0000 7f7af8cb6640 1 -- 172.21.15.38:0/3268462339 wait complete. 2024-09-21T03:45:19.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.799+0000 7f7af8cb6640 1 Processor -- start 2024-09-21T03:45:19.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.799+0000 7f7af8cb6640 1 -- start start 2024-09-21T03:45:19.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.800+0000 7f7af8cb6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7af40ceb90 0x7f7af4158a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:19.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.800+0000 7f7af8cb6640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7af40d0170 0x7f7af4153e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:19.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.800+0000 7f7af8cb6640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7af415ee80 0x7f7af406cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:19.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.800+0000 7f7af8cb6640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f7af40b6910 con 0x7f7af40ceb90 2024-09-21T03:45:19.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.800+0000 7f7af8cb6640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f7af40b6790 con 0x7f7af40d0170 2024-09-21T03:45:19.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.800+0000 7f7af8cb6640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f7af40b6a90 con 0x7f7af415ee80 2024-09-21T03:45:19.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.800+0000 7f7af259c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7af40ceb90 0x7f7af4158a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:19.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.800+0000 7f7af259c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7af40ceb90 0x7f7af4158a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46626/0 (socket says 172.21.15.38:46626) 2024-09-21T03:45:19.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.800+0000 7f7af259c640 1 -- 172.21.15.38:0/3072551684 learned_addr learned my addr 172.21.15.38:0/3072551684 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:45:19.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.800+0000 7f7af1d9b640 1 --2- 172.21.15.38:0/3072551684 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7af40d0170 0x7f7af4153e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:19.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.801+0000 7f7af2d9d640 1 --2- 172.21.15.38:0/3072551684 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7af415ee80 0x7f7af406cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:19.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.801+0000 7f7af259c640 1 -- 172.21.15.38:0/3072551684 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7af415ee80 msgr2=0x7f7af406cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:19.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.801+0000 7f7af259c640 1 --2- 172.21.15.38:0/3072551684 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7af415ee80 0x7f7af406cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:19.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.801+0000 7f7af259c640 1 -- 172.21.15.38:0/3072551684 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7af40d0170 msgr2=0x7f7af4153e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:19.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.801+0000 7f7af259c640 1 --2- 172.21.15.38:0/3072551684 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7af40d0170 0x7f7af4153e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:19.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.801+0000 7f7af259c640 1 -- 172.21.15.38:0/3072551684 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7af41f9880 con 0x7f7af40ceb90 2024-09-21T03:45:19.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.801+0000 7f7af2d9d640 1 --2- 172.21.15.38:0/3072551684 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7af415ee80 0x7f7af406cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:45:19.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.801+0000 7f7af259c640 1 --2- 172.21.15.38:0/3072551684 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7af40ceb90 0x7f7af4158a90 secure :-1 s=READY pgs=2459 cs=0 l=1 rev1=1 crypto rx=0x7f7ae400c730 tx=0x7f7ae400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:19.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.801+0000 7f7aeaffd640 1 -- 172.21.15.38:0/3072551684 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ae4003b70 con 0x7f7af40ceb90 2024-09-21T03:45:19.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.801+0000 7f7aeaffd640 1 -- 172.21.15.38:0/3072551684 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7ae4003d30 con 0x7f7af40ceb90 2024-09-21T03:45:19.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.801+0000 7f7af8cb6640 1 -- 172.21.15.38:0/3072551684 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7af4157cd0 con 0x7f7af40ceb90 2024-09-21T03:45:19.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.801+0000 7f7aeaffd640 1 -- 172.21.15.38:0/3072551684 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ae400aab0 con 0x7f7af40ceb90 2024-09-21T03:45:19.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.801+0000 7f7af8cb6640 1 -- 172.21.15.38:0/3072551684 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7af4158290 con 0x7f7af40ceb90 2024-09-21T03:45:19.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.803+0000 7f7aeaffd640 1 -- 172.21.15.38:0/3072551684 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f7ae400ac50 con 0x7f7af40ceb90 2024-09-21T03:45:19.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.803+0000 7f7aeaffd640 1 --2- 172.21.15.38:0/3072551684 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7ad4078b90 0x7f7ad407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:19.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.804+0000 7f7aeaffd640 1 -- 172.21.15.38:0/3072551684 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 6303+0+0 (secure 0 0 0) 0x7f7ae409ad90 con 0x7f7af40ceb90 2024-09-21T03:45:19.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.804+0000 7f7af1d9b640 1 --2- 172.21.15.38:0/3072551684 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7ad4078b90 0x7f7ad407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:19.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.804+0000 7f7af1d9b640 1 --2- 172.21.15.38:0/3072551684 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7ad4078b90 0x7f7ad407b0a0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f7af406f360 tx=0x7f7adc009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:19.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.805+0000 7f7affc1f740 1 Processor -- start 2024-09-21T03:45:19.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.805+0000 7f7affc1f740 1 -- start start 2024-09-21T03:45:19.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.806+0000 7f7affc1f740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56272b96f350 0x56272bd40d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:19.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.806+0000 7f7affc1f740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56272b946490 0x56272bd412c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:19.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.806+0000 7f7affc1f740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56272b94e670 0x56272bd5e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:19.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.806+0000 7f7affc1f740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x56272bc899a0 con 0x56272b96f350 2024-09-21T03:45:19.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.806+0000 7f7affc1f740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x56272bc78dd0 con 0x56272b94e670 2024-09-21T03:45:19.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.806+0000 7f7affc1f740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x56272bc7cbe0 con 0x56272b946490 2024-09-21T03:45:19.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.806+0000 7f7af259c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56272b96f350 0x56272bd40d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:19.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.806+0000 7f7af259c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56272b96f350 0x56272bd40d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46638/0 (socket says 172.21.15.38:46638) 2024-09-21T03:45:19.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.806+0000 7f7af259c640 1 -- 172.21.15.38:0/1104375338 learned_addr learned my addr 172.21.15.38:0/1104375338 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:45:19.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.806+0000 7f7af1d9b640 1 --2- 172.21.15.38:0/1104375338 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56272b946490 0x56272bd412c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:19.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.806+0000 7f7af2d9d640 1 --2- 172.21.15.38:0/1104375338 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56272b94e670 0x56272bd5e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:19.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.806+0000 7f7af259c640 1 -- 172.21.15.38:0/1104375338 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56272b946490 msgr2=0x56272bd412c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:19.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.806+0000 7f7af259c640 1 --2- 172.21.15.38:0/1104375338 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56272b946490 0x56272bd412c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:19.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.806+0000 7f7af259c640 1 -- 172.21.15.38:0/1104375338 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56272b94e670 msgr2=0x56272bd5e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:19.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.806+0000 7f7af259c640 1 --2- 172.21.15.38:0/1104375338 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56272b94e670 0x56272bd5e1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:19.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.806+0000 7f7af259c640 1 -- 172.21.15.38:0/1104375338 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56272bc11700 con 0x56272b96f350 2024-09-21T03:45:19.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.806+0000 7f7af1d9b640 1 --2- 172.21.15.38:0/1104375338 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56272b946490 0x56272bd412c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:45:19.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.807+0000 7f7af259c640 1 --2- 172.21.15.38:0/1104375338 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56272b96f350 0x56272bd40d80 secure :-1 s=READY pgs=2460 cs=0 l=1 rev1=1 crypto rx=0x7f7ae4062570 tx=0x7f7ae4062770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbcfe6526cf83d4f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:19.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.807+0000 7f7ad3fff640 1 -- 172.21.15.38:0/1104375338 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ae4062f70 con 0x56272b96f350 2024-09-21T03:45:19.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.807+0000 7f7ad3fff640 1 -- 172.21.15.38:0/1104375338 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7ae4062870 con 0x56272b96f350 2024-09-21T03:45:19.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.807+0000 7f7ad3fff640 1 -- 172.21.15.38:0/1104375338 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ae4063da0 con 0x56272b96f350 2024-09-21T03:45:19.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.808+0000 7f7affc1f740 1 -- 172.21.15.38:0/1104375338 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56272b96f350 msgr2=0x56272bd40d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:19.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.808+0000 7f7affc1f740 1 --2- 172.21.15.38:0/1104375338 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56272b96f350 0x56272bd40d80 secure :-1 s=READY pgs=2460 cs=0 l=1 rev1=1 crypto rx=0x7f7ae4062570 tx=0x7f7ae4062770 comp rx=0 tx=0).stop 2024-09-21T03:45:19.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.808+0000 7f7affc1f740 1 -- 172.21.15.38:0/1104375338 shutdown_connections 2024-09-21T03:45:19.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.808+0000 7f7affc1f740 1 --2- 172.21.15.38:0/1104375338 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56272b94e670 0x56272bd5e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:19.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.808+0000 7f7affc1f740 1 --2- 172.21.15.38:0/1104375338 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56272b946490 0x56272bd412c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:19.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.808+0000 7f7affc1f740 1 --2- 172.21.15.38:0/1104375338 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56272b96f350 0x56272bd40d80 unknown :-1 s=CLOSED pgs=2460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:19.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.808+0000 7f7affc1f740 1 -- 172.21.15.38:0/1104375338 >> 172.21.15.38:0/1104375338 conn(0x56272b9634a0 msgr2=0x56272bd17040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:19.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.808+0000 7f7affc1f740 1 -- 172.21.15.38:0/1104375338 shutdown_connections 2024-09-21T03:45:19.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.808+0000 7f7affc1f740 1 -- 172.21.15.38:0/1104375338 wait complete. 2024-09-21T03:45:19.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.809+0000 7f7affc1f740 1 -- start start 2024-09-21T03:45:19.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.809+0000 7f7affc1f740 1 Processor -- start 2024-09-21T03:45:19.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.810+0000 7f7affc1f740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56272b946490 0x56272bd56aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:19.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.810+0000 7f7affc1f740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56272b94e670 0x56272bd53d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:19.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.810+0000 7f7affc1f740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56272b96f350 0x56272bd2f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:19.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.810+0000 7f7affc1f740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x56272b9e9700 con 0x56272b946490 2024-09-21T03:45:19.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.810+0000 7f7affc1f740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x56272bc1e340 con 0x56272b94e670 2024-09-21T03:45:19.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.810+0000 7f7affc1f740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x56272bd41800 con 0x56272b96f350 2024-09-21T03:45:19.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.810+0000 7f7affc1f740 10 client.0 _ll_register_callbacks cb 0x56272bc1cee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:45:19.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.810+0000 7f7af259c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56272b946490 0x56272bd56aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:19.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.810+0000 7f7af259c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56272b946490 0x56272bd56aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46640/0 (socket says 172.21.15.38:46640) 2024-09-21T03:45:19.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.810+0000 7f7af259c640 1 -- 172.21.15.38:0/3875137407 learned_addr learned my addr 172.21.15.38:0/3875137407 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:45:19.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.811+0000 7f7af259c640 1 -- 172.21.15.38:0/3875137407 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56272b96f350 msgr2=0x56272bd2f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:19.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.811+0000 7f7af259c640 1 --2- 172.21.15.38:0/3875137407 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56272b96f350 0x56272bd2f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:19.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.811+0000 7f7af259c640 1 -- 172.21.15.38:0/3875137407 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56272b94e670 msgr2=0x56272bd53d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:19.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.811+0000 7f7af1d9b640 1 --2- 172.21.15.38:0/3875137407 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56272b94e670 0x56272bd53d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:19.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.811+0000 7f7af259c640 1 --2- 172.21.15.38:0/3875137407 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56272b94e670 0x56272bd53d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:19.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.811+0000 7f7af259c640 1 -- 172.21.15.38:0/3875137407 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56272ba537c0 con 0x56272b946490 2024-09-21T03:45:19.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.811+0000 7f7af259c640 1 --2- 172.21.15.38:0/3875137407 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56272b946490 0x56272bd56aa0 secure :-1 s=READY pgs=2461 cs=0 l=1 rev1=1 crypto rx=0x7f7ae4062800 tx=0x7f7ae405ed90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cc0b7328d747d4d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:19.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.811+0000 7f7ad17fa640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-21T03:45:19.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.811+0000 7f7ad17fa640 1 -- 172.21.15.38:0/3875137407 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ae4067350 con 0x56272b946490 2024-09-21T03:45:19.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.811+0000 7f7ad17fa640 1 -- 172.21.15.38:0/3875137407 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(149..149 src has 1..149) ==== 6303+0+0 (secure 0 0 0) 0x7f7ae4061ad0 con 0x56272b946490 2024-09-21T03:45:19.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.811+0000 7f7affc1f740 1 -- 172.21.15.38:0/3875137407 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ae4062870 con 0x56272b946490 2024-09-21T03:45:19.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.811+0000 7f7ad17fa640 1 -- 172.21.15.38:0/3875137407 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=349) ==== 24+0+0 (secure 0 0 0) 0x7f7ae4083050 con 0x56272b946490 2024-09-21T03:45:19.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.811+0000 7f7affc1f740 10 client.11091 fetch_fsmap learned FSMap version 349 2024-09-21T03:45:19.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.811+0000 7f7affc1f740 1 -- 172.21.15.38:0/3875137407 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=349,monmap=2+}) -- 0x56272bab3f50 con 0x56272b946490 2024-09-21T03:45:19.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.811+0000 7f7ad17fa640 1 -- 172.21.15.38:0/3875137407 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7ae4062870 con 0x56272b946490 2024-09-21T03:45:19.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.811+0000 7f7ad17fa640 1 -- 172.21.15.38:0/3875137407 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 349) ==== 3502+0+0 (secure 0 0 0) 0x7f7ae405e1c0 con 0x56272b946490 2024-09-21T03:45:19.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.812+0000 7f7affc1f740 10 client.11091 fetch_fsmap finished waiting for FSMap version 349 2024-09-21T03:45:19.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.812+0000 7f7affc1f740 10 client.11091 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T03:45:19.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.812+0000 7f7affc1f740 20 client.11091 populate_metadata read hostname 'smithi038' 2024-09-21T03:45:19.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.812+0000 7f7affc1f740 1 --2- 172.21.15.38:0/3875137407 >> [v2:172.21.15.38:6832/3839260519,v1:172.21.15.38:6833/3839260519] conn(0x56272b92e130 0x56272bd5e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:19.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.812+0000 7f7affc1f740 4 client.11091 mds_command: new command op to 10935 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:45:19.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.812+0000 7f7affc1f740 1 -- 172.21.15.38:0/3875137407 --> [v2:172.21.15.38:6832/3839260519,v1:172.21.15.38:6833/3839260519] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56272bc8b8f0 con 0x56272b92e130 2024-09-21T03:45:19.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.812+0000 7f7af1d9b640 1 --2- 172.21.15.38:0/3875137407 >> [v2:172.21.15.38:6832/3839260519,v1:172.21.15.38:6833/3839260519] conn(0x56272b92e130 0x56272bd5e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:19.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.812+0000 7f7ad17fa640 10 client.11091 ms_handle_connect on v2:172.21.15.38:6832/3839260519 2024-09-21T03:45:19.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.812+0000 7f7af1d9b640 1 --2- 172.21.15.38:0/3875137407 >> [v2:172.21.15.38:6832/3839260519,v1:172.21.15.38:6833/3839260519] conn(0x56272b92e130 0x56272bd5e2e0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5cbd5b94349adc27 server_cookie=cc71ce6e558b99a4 in_seq=0 out_seq=0 2024-09-21T03:45:19.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.813+0000 7f7ad17fa640 1 -- 172.21.15.38:0/3875137407 <== mds.0 v2:172.21.15.38:6832/3839260519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56272bc8b8f0 con 0x56272b92e130 2024-09-21T03:45:19.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.813+0000 7f7ad17fa640 10 client.11091 handle_command_reply: tid=0 2024-09-21T03:45:19.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.813+0000 7f7affc1f740 1 client.11091 shutdown 2024-09-21T03:45:19.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.813+0000 7f7affc1f740 1 -- 172.21.15.38:0/3875137407 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56272b946490 msgr2=0x56272bd56aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:19.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.813+0000 7f7affc1f740 1 --2- 172.21.15.38:0/3875137407 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56272b946490 0x56272bd56aa0 secure :-1 s=READY pgs=2461 cs=0 l=1 rev1=1 crypto rx=0x7f7ae4062800 tx=0x7f7ae405ed90 comp rx=0 tx=0).stop 2024-09-21T03:45:19.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.813+0000 7f7affc1f740 1 -- 172.21.15.38:0/3875137407 shutdown_connections 2024-09-21T03:45:19.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.813+0000 7f7affc1f740 1 --2- 172.21.15.38:0/3875137407 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56272b96f350 0x56272bd2f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:19.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.813+0000 7f7affc1f740 1 --2- 172.21.15.38:0/3875137407 >> [v2:172.21.15.38:6832/3839260519,v1:172.21.15.38:6833/3839260519] conn(0x56272b92e130 0x56272bd5e2e0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:19.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.813+0000 7f7affc1f740 1 --2- 172.21.15.38:0/3875137407 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56272b94e670 0x56272bd53d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:19.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.813+0000 7f7ad17fa640 0 client.11091 ms_handle_reset on v2:172.21.15.38:6832/3839260519 2024-09-21T03:45:19.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.813+0000 7f7affc1f740 1 --2- 172.21.15.38:0/3875137407 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56272b946490 0x56272bd56aa0 unknown :-1 s=CLOSED pgs=2461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:19.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.813+0000 7f7affc1f740 1 -- 172.21.15.38:0/3875137407 >> 172.21.15.38:0/3875137407 conn(0x56272b9634a0 msgr2=0x56272bd5d790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:19.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.813+0000 7f7affc1f740 1 -- 172.21.15.38:0/3875137407 shutdown_connections 2024-09-21T03:45:19.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.813+0000 7f7affc1f740 1 -- 172.21.15.38:0/3875137407 wait complete. 2024-09-21T03:45:19.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.813+0000 7f7affc1f740 20 client.11091 trim_cache size 0 max 16384 2024-09-21T03:45:19.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.827+0000 7f7affc1f740 1 Processor -- start 2024-09-21T03:45:19.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.827+0000 7f7affc1f740 1 -- start start 2024-09-21T03:45:19.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.828+0000 7f7affc1f740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56272b946490 0x56272bd16230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:19.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.828+0000 7f7affc1f740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56272b94e670 0x56272bd5e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:19.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.828+0000 7f7affc1f740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56272b96f350 0x56272bd5e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:19.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.828+0000 7f7affc1f740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f7af4201bb0 con 0x56272b94e670 2024-09-21T03:45:19.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.828+0000 7f7affc1f740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f7af4200f10 con 0x56272b96f350 2024-09-21T03:45:19.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.828+0000 7f7affc1f740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f7af4201560 con 0x56272b946490 2024-09-21T03:45:19.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.828+0000 7f7af1d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56272b94e670 0x56272bd5e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:19.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.828+0000 7f7af1d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56272b94e670 0x56272bd5e2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46652/0 (socket says 172.21.15.38:46652) 2024-09-21T03:45:19.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.828+0000 7f7af1d9b640 1 -- 172.21.15.38:0/1882431453 learned_addr learned my addr 172.21.15.38:0/1882431453 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:45:19.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.828+0000 7f7af2d9d640 1 --2- 172.21.15.38:0/1882431453 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56272b96f350 0x56272bd5e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:19.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.828+0000 7f7af259c640 1 --2- 172.21.15.38:0/1882431453 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56272b946490 0x56272bd16230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:19.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.829+0000 7f7af1d9b640 1 -- 172.21.15.38:0/1882431453 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56272b946490 msgr2=0x56272bd16230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:19.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.829+0000 7f7af1d9b640 1 --2- 172.21.15.38:0/1882431453 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56272b946490 0x56272bd16230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:19.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.829+0000 7f7af1d9b640 1 -- 172.21.15.38:0/1882431453 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56272b96f350 msgr2=0x56272bd5e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:19.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.829+0000 7f7af1d9b640 1 --2- 172.21.15.38:0/1882431453 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56272b96f350 0x56272bd5e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:19.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.829+0000 7f7af1d9b640 1 -- 172.21.15.38:0/1882431453 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56272b9b8660 con 0x56272b94e670 2024-09-21T03:45:19.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.829+0000 7f7af2d9d640 1 --2- 172.21.15.38:0/1882431453 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56272b96f350 0x56272bd5e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:45:19.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.829+0000 7f7af1d9b640 1 --2- 172.21.15.38:0/1882431453 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56272b94e670 0x56272bd5e2e0 secure :-1 s=READY pgs=2462 cs=0 l=1 rev1=1 crypto rx=0x7f7af40ce780 tx=0x7f7adc012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7aed55a6fb9b4da server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:19.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.829+0000 7f7ad37fe640 1 -- 172.21.15.38:0/1882431453 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7adc02f070 con 0x56272b94e670 2024-09-21T03:45:19.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.829+0000 7f7ad37fe640 1 -- 172.21.15.38:0/1882431453 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7adc01b030 con 0x56272b94e670 2024-09-21T03:45:19.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.829+0000 7f7ad37fe640 1 -- 172.21.15.38:0/1882431453 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7adc015b10 con 0x56272b94e670 2024-09-21T03:45:19.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.830+0000 7f7affc1f740 1 -- 172.21.15.38:0/1882431453 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56272b94e670 msgr2=0x56272bd5e2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:19.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.830+0000 7f7affc1f740 1 --2- 172.21.15.38:0/1882431453 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56272b94e670 0x56272bd5e2e0 secure :-1 s=READY pgs=2462 cs=0 l=1 rev1=1 crypto rx=0x7f7af40ce780 tx=0x7f7adc012000 comp rx=0 tx=0).stop 2024-09-21T03:45:19.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.830+0000 7f7affc1f740 1 -- 172.21.15.38:0/1882431453 shutdown_connections 2024-09-21T03:45:19.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.830+0000 7f7affc1f740 1 --2- 172.21.15.38:0/1882431453 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56272b96f350 0x56272bd5e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:19.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.830+0000 7f7affc1f740 1 --2- 172.21.15.38:0/1882431453 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56272b94e670 0x56272bd5e2e0 unknown :-1 s=CLOSED pgs=2462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:19.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.830+0000 7f7affc1f740 1 --2- 172.21.15.38:0/1882431453 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56272b946490 0x56272bd16230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:19.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.830+0000 7f7affc1f740 1 -- 172.21.15.38:0/1882431453 >> 172.21.15.38:0/1882431453 conn(0x56272b9634a0 msgr2=0x56272bd12380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:19.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.830+0000 7f7affc1f740 1 -- 172.21.15.38:0/1882431453 shutdown_connections 2024-09-21T03:45:19.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.830+0000 7f7affc1f740 1 -- 172.21.15.38:0/1882431453 wait complete. 2024-09-21T03:45:19.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.831+0000 7f7affc1f740 1 -- start start 2024-09-21T03:45:19.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.831+0000 7f7affc1f740 1 Processor -- start 2024-09-21T03:45:19.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.832+0000 7f7affc1f740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56272b946490 0x56272bdda6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:19.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.832+0000 7f7affc1f740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56272b94e670 0x56272bd5e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:19.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.832+0000 7f7affc1f740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56272b96f350 0x56272bd530e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:19.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.832+0000 7f7affc1f740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x56272b9e9700 con 0x56272b94e670 2024-09-21T03:45:19.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.832+0000 7f7affc1f740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x56272bc1e340 con 0x56272b96f350 2024-09-21T03:45:19.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.832+0000 7f7affc1f740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x56272bd6bc60 con 0x56272b946490 2024-09-21T03:45:19.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.832+0000 7f7affc1f740 10 client.0 _ll_register_callbacks cb 0x7f7ae4087070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:45:19.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.832+0000 7f7af1d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56272b94e670 0x56272bd5e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:19.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.832+0000 7f7af1d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56272b94e670 0x56272bd5e0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46660/0 (socket says 172.21.15.38:46660) 2024-09-21T03:45:19.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.832+0000 7f7af1d9b640 1 -- 172.21.15.38:0/92167647 learned_addr learned my addr 172.21.15.38:0/92167647 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:45:19.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.833+0000 7f7af2d9d640 1 --2- 172.21.15.38:0/92167647 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56272b96f350 0x56272bd530e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:19.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.833+0000 7f7af259c640 1 --2- 172.21.15.38:0/92167647 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56272b946490 0x56272bdda6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:19.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.833+0000 7f7af1d9b640 1 -- 172.21.15.38:0/92167647 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56272b946490 msgr2=0x56272bdda6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:19.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.833+0000 7f7af1d9b640 1 --2- 172.21.15.38:0/92167647 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56272b946490 0x56272bdda6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:19.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.833+0000 7f7af1d9b640 1 -- 172.21.15.38:0/92167647 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56272b96f350 msgr2=0x56272bd530e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:19.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.833+0000 7f7af1d9b640 1 --2- 172.21.15.38:0/92167647 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56272b96f350 0x56272bd530e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:19.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.833+0000 7f7af1d9b640 1 -- 172.21.15.38:0/92167647 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56272bd55790 con 0x56272b94e670 2024-09-21T03:45:19.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.833+0000 7f7af2d9d640 1 --2- 172.21.15.38:0/92167647 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56272b96f350 0x56272bd530e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:45:19.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.833+0000 7f7af1d9b640 1 --2- 172.21.15.38:0/92167647 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56272b94e670 0x56272bd5e0a0 secure :-1 s=READY pgs=2463 cs=0 l=1 rev1=1 crypto rx=0x7f7af408d980 tx=0x7f7adc015ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd376447ca574837 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:19.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.833+0000 7f7ad17fa640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-21T03:45:19.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.833+0000 7f7ad17fa640 1 -- 172.21.15.38:0/92167647 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7adc02f040 con 0x56272b94e670 2024-09-21T03:45:19.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.833+0000 7f7ad17fa640 1 -- 172.21.15.38:0/92167647 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(149..149 src has 1..149) ==== 6303+0+0 (secure 0 0 0) 0x7f7adc007050 con 0x56272b94e670 2024-09-21T03:45:19.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.833+0000 7f7affc1f740 1 -- 172.21.15.38:0/92167647 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7adc01b030 con 0x56272b94e670 2024-09-21T03:45:19.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.833+0000 7f7ad17fa640 1 -- 172.21.15.38:0/92167647 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=349) ==== 24+0+0 (secure 0 0 0) 0x7f7adc040050 con 0x56272b94e670 2024-09-21T03:45:19.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.833+0000 7f7affc1f740 10 client.11097 fetch_fsmap learned FSMap version 349 2024-09-21T03:45:19.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.833+0000 7f7affc1f740 1 -- 172.21.15.38:0/92167647 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=349,monmap=2+}) -- 0x56272bddac10 con 0x56272b94e670 2024-09-21T03:45:19.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.833+0000 7f7ad17fa640 1 -- 172.21.15.38:0/92167647 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7adc01b030 con 0x56272b94e670 2024-09-21T03:45:19.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.834+0000 7f7ad17fa640 1 -- 172.21.15.38:0/92167647 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 349) ==== 3502+0+0 (secure 0 0 0) 0x7f7adc043020 con 0x56272b94e670 2024-09-21T03:45:19.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.834+0000 7f7affc1f740 10 client.11097 fetch_fsmap finished waiting for FSMap version 349 2024-09-21T03:45:19.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.834+0000 7f7affc1f740 10 client.11097 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T03:45:19.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.834+0000 7f7affc1f740 20 client.11097 populate_metadata read hostname 'smithi038' 2024-09-21T03:45:19.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.834+0000 7f7affc1f740 1 --2- 172.21.15.38:0/92167647 >> [v2:172.21.15.38:6832/3839260519,v1:172.21.15.38:6833/3839260519] conn(0x56272bd176b0 0x56272bd5a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:19.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.834+0000 7f7affc1f740 4 client.11097 mds_command: new command op to 10935 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-09-21T03:45:19.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.834+0000 7f7affc1f740 1 -- 172.21.15.38:0/92167647 --> [v2:172.21.15.38:6832/3839260519,v1:172.21.15.38:6833/3839260519] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) -- 0x56272bc873a0 con 0x56272bd176b0 2024-09-21T03:45:19.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.834+0000 7f7af259c640 1 --2- 172.21.15.38:0/92167647 >> [v2:172.21.15.38:6832/3839260519,v1:172.21.15.38:6833/3839260519] conn(0x56272bd176b0 0x56272bd5a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:19.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.834+0000 7f7ad17fa640 10 client.11097 ms_handle_connect on v2:172.21.15.38:6832/3839260519 2024-09-21T03:45:19.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.834+0000 7f7af259c640 1 --2- 172.21.15.38:0/92167647 >> [v2:172.21.15.38:6832/3839260519,v1:172.21.15.38:6833/3839260519] conn(0x56272bd176b0 0x56272bd5a730 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=499288d3c06119b3 server_cookie=c238d381b8abbfa5 in_seq=0 out_seq=0 2024-09-21T03:45:19.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.834+0000 7f7ad17fa640 1 -- 172.21.15.38:0/92167647 <== mds.0 v2:172.21.15.38:6832/3839260519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+293 (crc 0 0 0) 0x56272bc873a0 con 0x56272bd176b0 2024-09-21T03:45:19.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.834+0000 7f7ad17fa640 10 client.11097 handle_command_reply: tid=0 2024-09-21T03:45:19.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.834+0000 7f7affc1f740 1 client.11097 shutdown 2024-09-21T03:45:19.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.835+0000 7f7affc1f740 1 -- 172.21.15.38:0/92167647 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56272b94e670 msgr2=0x56272bd5e0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:19.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.835+0000 7f7affc1f740 1 --2- 172.21.15.38:0/92167647 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56272b94e670 0x56272bd5e0a0 secure :-1 s=READY pgs=2463 cs=0 l=1 rev1=1 crypto rx=0x7f7af408d980 tx=0x7f7adc015ae0 comp rx=0 tx=0).stop 2024-09-21T03:45:19.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.835+0000 7f7affc1f740 1 -- 172.21.15.38:0/92167647 shutdown_connections 2024-09-21T03:45:19.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.835+0000 7f7affc1f740 1 --2- 172.21.15.38:0/92167647 >> [v2:172.21.15.38:6832/3839260519,v1:172.21.15.38:6833/3839260519] conn(0x56272bd176b0 0x56272bd5a730 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:19.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.835+0000 7f7affc1f740 1 --2- 172.21.15.38:0/92167647 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56272b96f350 0x56272bd530e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:19.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.835+0000 7f7ad17fa640 0 client.11097 ms_handle_reset on v2:172.21.15.38:6832/3839260519 2024-09-21T03:45:19.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.835+0000 7f7affc1f740 1 --2- 172.21.15.38:0/92167647 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56272b94e670 0x56272bd5e0a0 unknown :-1 s=CLOSED pgs=2463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:19.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.835+0000 7f7affc1f740 1 --2- 172.21.15.38:0/92167647 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56272b946490 0x56272bdda6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:19.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.835+0000 7f7affc1f740 1 -- 172.21.15.38:0/92167647 >> 172.21.15.38:0/92167647 conn(0x56272b9634a0 msgr2=0x56272bd122e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:19.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.835+0000 7f7affc1f740 1 -- 172.21.15.38:0/92167647 shutdown_connections 2024-09-21T03:45:19.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.835+0000 7f7affc1f740 1 -- 172.21.15.38:0/92167647 wait complete. 2024-09-21T03:45:19.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.835+0000 7f7affc1f740 20 client.11097 trim_cache size 0 max 16384 2024-09-21T03:45:19.843 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:45:19.844 INFO:teuthology.orchestra.run.smithi038.stdout:[ 2024-09-21T03:45:19.844 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-21T03:45:19.844 INFO:teuthology.orchestra.run.smithi038.stdout: "damage_type": "backtrace", 2024-09-21T03:45:19.844 INFO:teuthology.orchestra.run.smithi038.stdout: "id": 433208531, 2024-09-21T03:45:19.844 INFO:teuthology.orchestra.run.smithi038.stdout: "ino": 1099511628279, 2024-09-21T03:45:19.844 INFO:teuthology.orchestra.run.smithi038.stdout: "path": "/testdir/hardlink2" 2024-09-21T03:45:19.844 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-21T03:45:19.844 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-21T03:45:19.844 INFO:teuthology.orchestra.run.smithi038.stdout: "damage_type": "dir_frag", 2024-09-21T03:45:19.844 INFO:teuthology.orchestra.run.smithi038.stdout: "id": 1495185665, 2024-09-21T03:45:19.844 INFO:teuthology.orchestra.run.smithi038.stdout: "ino": 1099511628278, 2024-09-21T03:45:19.844 INFO:teuthology.orchestra.run.smithi038.stdout: "frag": "*", 2024-09-21T03:45:19.845 INFO:teuthology.orchestra.run.smithi038.stdout: "path": "/dir2" 2024-09-21T03:45:19.845 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-21T03:45:19.845 INFO:teuthology.orchestra.run.smithi038.stdout:] 2024-09-21T03:45:19.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.835+0000 7f7af8cb6640 1 -- 172.21.15.38:0/3072551684 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7ad4078b90 msgr2=0x7f7ad407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:19.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.835+0000 7f7af8cb6640 1 --2- 172.21.15.38:0/3072551684 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7ad4078b90 0x7f7ad407b0a0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f7af406f360 tx=0x7f7adc009040 comp rx=0 tx=0).stop 2024-09-21T03:45:19.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.835+0000 7f7af8cb6640 1 -- 172.21.15.38:0/3072551684 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7af40ceb90 msgr2=0x7f7af4158a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:19.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.835+0000 7f7af8cb6640 1 --2- 172.21.15.38:0/3072551684 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7af40ceb90 0x7f7af4158a90 secure :-1 s=READY pgs=2459 cs=0 l=1 rev1=1 crypto rx=0x7f7ae400c730 tx=0x7f7ae400cbf0 comp rx=0 tx=0).stop 2024-09-21T03:45:19.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.835+0000 7f7af8cb6640 1 -- 172.21.15.38:0/3072551684 shutdown_connections 2024-09-21T03:45:19.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.835+0000 7f7af8cb6640 1 --2- 172.21.15.38:0/3072551684 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f7ad4078b90 0x7f7ad407b0a0 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:19.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.835+0000 7f7af8cb6640 1 --2- 172.21.15.38:0/3072551684 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7af415ee80 0x7f7af406cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:19.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.835+0000 7f7af8cb6640 1 --2- 172.21.15.38:0/3072551684 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7af40d0170 0x7f7af4153e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:19.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.835+0000 7f7af8cb6640 1 --2- 172.21.15.38:0/3072551684 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7af40ceb90 0x7f7af4158a90 unknown :-1 s=CLOSED pgs=2459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:19.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.835+0000 7f7af8cb6640 1 -- 172.21.15.38:0/3072551684 >> 172.21.15.38:0/3072551684 conn(0x7f7af409d830 msgr2=0x7f7af4161840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:19.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.836+0000 7f7af8cb6640 1 -- 172.21.15.38:0/3072551684 shutdown_connections 2024-09-21T03:45:19.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.836+0000 7f7af8cb6640 1 -- 172.21.15.38:0/3072551684 wait complete. 2024-09-21T03:45:19.850 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:45:19.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.963+0000 7f985c09b640 1 -- 172.21.15.38:0/3698556413 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9854162280 msgr2=0x7f9854162690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:19.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.963+0000 7f985c09b640 1 --2- 172.21.15.38:0/3698556413 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9854162280 0x7f9854162690 secure :-1 s=READY pgs=2464 cs=0 l=1 rev1=1 crypto rx=0x7f9844005fd0 tx=0x7f984402a5d0 comp rx=0 tx=0).stop 2024-09-21T03:45:19.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.963+0000 7f985c09b640 1 -- 172.21.15.38:0/3698556413 shutdown_connections 2024-09-21T03:45:19.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.963+0000 7f985c09b640 1 --2- 172.21.15.38:0/3698556413 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f98540b2e40 0x7f98540b3250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:19.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.963+0000 7f985c09b640 1 --2- 172.21.15.38:0/3698556413 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9854162280 0x7f9854162690 unknown :-1 s=CLOSED pgs=2464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:19.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.963+0000 7f985c09b640 1 -- 172.21.15.38:0/3698556413 >> 172.21.15.38:0/3698556413 conn(0x7f98540b6b20 msgr2=0x7f98540d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:19.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.963+0000 7f985c09b640 1 -- 172.21.15.38:0/3698556413 shutdown_connections 2024-09-21T03:45:19.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.963+0000 7f985c09b640 1 -- 172.21.15.38:0/3698556413 wait complete. 2024-09-21T03:45:19.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.965+0000 7f985c09b640 1 Processor -- start 2024-09-21T03:45:19.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.965+0000 7f985c09b640 1 -- start start 2024-09-21T03:45:19.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.966+0000 7f985c09b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f98540b2e40 0x7f98540b19a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:19.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.966+0000 7f985c09b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9854162280 0x7f98540ac5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:19.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.966+0000 7f985c09b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f98540acb30 0x7f98541f8730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:19.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.966+0000 7f985c09b640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f985415e4b0 con 0x7f9854162280 2024-09-21T03:45:19.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.966+0000 7f985c09b640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f98540b39d0 con 0x7f98540acb30 2024-09-21T03:45:19.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.966+0000 7f985c09b640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f985415e630 con 0x7f98540b2e40 2024-09-21T03:45:19.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.966+0000 7f9859636640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9854162280 0x7f98540ac5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:19.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.966+0000 7f9859636640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9854162280 0x7f98540ac5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46674/0 (socket says 172.21.15.38:46674) 2024-09-21T03:45:19.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.966+0000 7f9859636640 1 -- 172.21.15.38:0/3480758572 learned_addr learned my addr 172.21.15.38:0/3480758572 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:45:19.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.966+0000 7f985a638640 1 --2- 172.21.15.38:0/3480758572 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f98540acb30 0x7f98541f8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:19.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.966+0000 7f9859e37640 1 --2- 172.21.15.38:0/3480758572 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f98540b2e40 0x7f98540b19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:19.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.967+0000 7f9859636640 1 -- 172.21.15.38:0/3480758572 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f98540b2e40 msgr2=0x7f98540b19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:19.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.967+0000 7f9859636640 1 --2- 172.21.15.38:0/3480758572 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f98540b2e40 0x7f98540b19a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:19.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.967+0000 7f9859636640 1 -- 172.21.15.38:0/3480758572 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f98540acb30 msgr2=0x7f98541f8730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:19.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.967+0000 7f9859636640 1 --2- 172.21.15.38:0/3480758572 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f98540acb30 0x7f98541f8730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:19.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.967+0000 7f9859636640 1 -- 172.21.15.38:0/3480758572 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98540ad340 con 0x7f9854162280 2024-09-21T03:45:19.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.967+0000 7f9859636640 1 --2- 172.21.15.38:0/3480758572 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9854162280 0x7f98540ac5f0 secure :-1 s=READY pgs=2465 cs=0 l=1 rev1=1 crypto rx=0x7f9844034040 tx=0x7f98440038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:19.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.967+0000 7f984a7fc640 1 -- 172.21.15.38:0/3480758572 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f984403e030 con 0x7f9854162280 2024-09-21T03:45:19.972 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.967+0000 7f984a7fc640 1 -- 172.21.15.38:0/3480758572 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9844003ca0 con 0x7f9854162280 2024-09-21T03:45:19.972 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.967+0000 7f985c09b640 1 -- 172.21.15.38:0/3480758572 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98540ad620 con 0x7f9854162280 2024-09-21T03:45:19.972 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.967+0000 7f984a7fc640 1 -- 172.21.15.38:0/3480758572 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9844035040 con 0x7f9854162280 2024-09-21T03:45:19.972 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.967+0000 7f985c09b640 1 -- 172.21.15.38:0/3480758572 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98540adbe0 con 0x7f9854162280 2024-09-21T03:45:19.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.968+0000 7f985c09b640 1 -- 172.21.15.38:0/3480758572 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9854162f00 con 0x7f9854162280 2024-09-21T03:45:19.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.969+0000 7f984a7fc640 1 -- 172.21.15.38:0/3480758572 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f9844041030 con 0x7f9854162280 2024-09-21T03:45:19.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.969+0000 7f984a7fc640 1 --2- 172.21.15.38:0/3480758572 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9838078bc0 0x7f983807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:19.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.969+0000 7f984a7fc640 1 -- 172.21.15.38:0/3480758572 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 6303+0+0 (secure 0 0 0) 0x7f98440bb4b0 con 0x7f9854162280 2024-09-21T03:45:19.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.970+0000 7f9859e37640 1 --2- 172.21.15.38:0/3480758572 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9838078bc0 0x7f983807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:19.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.970+0000 7f9859e37640 1 --2- 172.21.15.38:0/3480758572 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9838078bc0 0x7f983807b0d0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f9850001720 tx=0x7f9850009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:19.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:19.974+0000 7f984a7fc640 1 -- 172.21.15.38:0/3480758572 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f9844082fd0 con 0x7f9854162280 2024-09-21T03:45:20.202 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.199+0000 7f985c09b640 1 -- 172.21.15.38:0/3480758572 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f98540ba130 con 0x7f9854162280 2024-09-21T03:45:20.202 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.200+0000 7f984a7fc640 1 -- 172.21.15.38:0/3480758572 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 349 v349) ==== 95+0+5241 (secure 0 0 0) 0x7f9844087a00 con 0x7f9854162280 2024-09-21T03:45:20.203 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:45:20.203 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":349,"btime":"2024-09-21T03:45:15:115320+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8399,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/3606957623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":3606957623},{"type":"v1","addr":"172.21.15.38:6835","nonce":3606957623}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":338},{"gid":8405,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/456651486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":456651486},{"type":"v1","addr":"172.21.15.195:6837","nonce":456651486}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":338},{"gid":9604,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/2649311840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":2649311840},{"type":"v1","addr":"172.21.15.195:6835","nonce":2649311840}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":338}],"filesystems":[{"mdsmap":{"epoch":349,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:44:56.710446+0000","modified":"2024-09-21T03:45:15.115308+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":147,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10935},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10935":{"gid":10935,"name":"c","rank":0,"incarnation":346,"state":"up:active","state_seq":4,"addr":"172.21.15.38:6833/3839260519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":3839260519},{"type":"v1","addr":"172.21.15.38:6833","nonce":3839260519}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10935,"qdb_cluster":[10935]},"id":7}]} 2024-09-21T03:45:20.203 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 349 2024-09-21T03:45:20.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.202+0000 7f985c09b640 1 -- 172.21.15.38:0/3480758572 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9838078bc0 msgr2=0x7f983807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:20.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.202+0000 7f985c09b640 1 --2- 172.21.15.38:0/3480758572 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9838078bc0 0x7f983807b0d0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f9850001720 tx=0x7f9850009040 comp rx=0 tx=0).stop 2024-09-21T03:45:20.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.202+0000 7f985c09b640 1 -- 172.21.15.38:0/3480758572 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9854162280 msgr2=0x7f98540ac5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:20.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.202+0000 7f985c09b640 1 --2- 172.21.15.38:0/3480758572 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9854162280 0x7f98540ac5f0 secure :-1 s=READY pgs=2465 cs=0 l=1 rev1=1 crypto rx=0x7f9844034040 tx=0x7f98440038a0 comp rx=0 tx=0).stop 2024-09-21T03:45:20.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.202+0000 7f985c09b640 1 -- 172.21.15.38:0/3480758572 shutdown_connections 2024-09-21T03:45:20.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.202+0000 7f985c09b640 1 --2- 172.21.15.38:0/3480758572 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f98540acb30 0x7f98541f8730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:20.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.202+0000 7f985c09b640 1 --2- 172.21.15.38:0/3480758572 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9854162280 0x7f98540ac5f0 unknown :-1 s=CLOSED pgs=2465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:20.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.202+0000 7f985c09b640 1 --2- 172.21.15.38:0/3480758572 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9838078bc0 0x7f983807b0d0 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:20.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.202+0000 7f985c09b640 1 --2- 172.21.15.38:0/3480758572 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f98540b2e40 0x7f98540b19a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:20.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.202+0000 7f985c09b640 1 -- 172.21.15.38:0/3480758572 >> 172.21.15.38:0/3480758572 conn(0x7f98540b6b20 msgr2=0x7f985415db30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:20.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.202+0000 7f985c09b640 1 -- 172.21.15.38:0/3480758572 shutdown_connections 2024-09-21T03:45:20.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.202+0000 7f985c09b640 1 -- 172.21.15.38:0/3480758572 wait complete. 2024-09-21T03:45:20.216 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c damage rm 433208531 2024-09-21T03:45:20.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.333+0000 7fb909fc3640 1 -- 172.21.15.38:0/2601269794 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb8f0003c30 msgr2=0x7fb8f0004080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:20.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.333+0000 7fb909fc3640 1 --2- 172.21.15.38:0/2601269794 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb8f0003c30 0x7fb8f0004080 secure :-1 s=READY pgs=2467 cs=0 l=1 rev1=1 crypto rx=0x7fb8f4009b50 tx=0x7fb8f0005830 comp rx=0 tx=0).stop 2024-09-21T03:45:20.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.333+0000 7fb909fc3640 1 -- 172.21.15.38:0/2601269794 shutdown_connections 2024-09-21T03:45:20.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.333+0000 7fb909fc3640 1 --2- 172.21.15.38:0/2601269794 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb8f0003c30 0x7fb8f0004080 unknown :-1 s=CLOSED pgs=2467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:20.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.333+0000 7fb909fc3640 1 -- 172.21.15.38:0/2601269794 >> 172.21.15.38:0/2601269794 conn(0x7fb9040b46e0 msgr2=0x7fb9040d23e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:20.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.333+0000 7fb909fc3640 1 -- 172.21.15.38:0/2601269794 shutdown_connections 2024-09-21T03:45:20.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.333+0000 7fb909fc3640 1 -- 172.21.15.38:0/2601269794 wait complete. 2024-09-21T03:45:20.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.335+0000 7fb909fc3640 1 Processor -- start 2024-09-21T03:45:20.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.336+0000 7fb909fc3640 1 -- start start 2024-09-21T03:45:20.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.337+0000 7fb909fc3640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb8f0003c30 0x7fb90415e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:20.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.337+0000 7fb909fc3640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb9040c8b60 0x7fb9040b31e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:20.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.337+0000 7fb909fc3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb9040cc3b0 0x7fb9040b3720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:20.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.337+0000 7fb909fc3640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fb9040d0670 con 0x7fb9040cc3b0 2024-09-21T03:45:20.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.337+0000 7fb909fc3640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fb90415cdb0 con 0x7fb9040c8b60 2024-09-21T03:45:20.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.337+0000 7fb909fc3640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fb9040d07f0 con 0x7fb8f0003c30 2024-09-21T03:45:20.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.337+0000 7fb903fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb9040cc3b0 0x7fb9040b3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:20.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.337+0000 7fb903fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb9040cc3b0 0x7fb9040b3720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46690/0 (socket says 172.21.15.38:46690) 2024-09-21T03:45:20.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.337+0000 7fb903fff640 1 -- 172.21.15.38:0/1121717600 learned_addr learned my addr 172.21.15.38:0/1121717600 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:45:20.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.337+0000 7fb9037fe640 1 --2- 172.21.15.38:0/1121717600 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb8f0003c30 0x7fb90415e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:20.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.337+0000 7fb8faffd640 1 --2- 172.21.15.38:0/1121717600 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb9040c8b60 0x7fb9040b31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:20.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.337+0000 7fb903fff640 1 -- 172.21.15.38:0/1121717600 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb8f0003c30 msgr2=0x7fb90415e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:20.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.337+0000 7fb903fff640 1 --2- 172.21.15.38:0/1121717600 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb8f0003c30 0x7fb90415e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:20.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.337+0000 7fb903fff640 1 -- 172.21.15.38:0/1121717600 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb9040c8b60 msgr2=0x7fb9040b31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:20.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.337+0000 7fb903fff640 1 --2- 172.21.15.38:0/1121717600 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb9040c8b60 0x7fb9040b31e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:20.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.338+0000 7fb903fff640 1 -- 172.21.15.38:0/1121717600 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9041454b0 con 0x7fb9040cc3b0 2024-09-21T03:45:20.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.338+0000 7fb903fff640 1 --2- 172.21.15.38:0/1121717600 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb9040cc3b0 0x7fb9040b3720 secure :-1 s=READY pgs=2468 cs=0 l=1 rev1=1 crypto rx=0x7fb904072110 tx=0x7fb8fc001b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:20.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.338+0000 7fb900ff9640 1 -- 172.21.15.38:0/1121717600 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8fc010890 con 0x7fb9040cc3b0 2024-09-21T03:45:20.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.338+0000 7fb900ff9640 1 -- 172.21.15.38:0/1121717600 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb8fc016070 con 0x7fb9040cc3b0 2024-09-21T03:45:20.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.338+0000 7fb909fc3640 1 -- 172.21.15.38:0/1121717600 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb904145790 con 0x7fb9040cc3b0 2024-09-21T03:45:20.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.338+0000 7fb900ff9640 1 -- 172.21.15.38:0/1121717600 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8fc010da0 con 0x7fb9040cc3b0 2024-09-21T03:45:20.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.338+0000 7fb909fc3640 1 -- 172.21.15.38:0/1121717600 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb904145d50 con 0x7fb9040cc3b0 2024-09-21T03:45:20.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.340+0000 7fb900ff9640 1 -- 172.21.15.38:0/1121717600 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb8fc01b460 con 0x7fb9040cc3b0 2024-09-21T03:45:20.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.341+0000 7fb900ff9640 1 --2- 172.21.15.38:0/1121717600 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb8e4078bc0 0x7fb8e407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:20.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.341+0000 7fb900ff9640 1 -- 172.21.15.38:0/1121717600 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 6303+0+0 (secure 0 0 0) 0x7fb8fc09b7d0 con 0x7fb9040cc3b0 2024-09-21T03:45:20.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.341+0000 7fb9037fe640 1 --2- 172.21.15.38:0/1121717600 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb8e4078bc0 0x7fb8e407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:20.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.342+0000 7fb911009740 1 Processor -- start 2024-09-21T03:45:20.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.342+0000 7fb911009740 1 -- start start 2024-09-21T03:45:20.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.342+0000 7fb9037fe640 1 --2- 172.21.15.38:0/1121717600 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb8e4078bc0 0x7fb8e407b0d0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7fb8f4035000 tx=0x7fb8f40046c0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:20.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.343+0000 7fb911009740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555581493490 0x555581880c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:20.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.343+0000 7fb911009740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55558146a5d0 0x555581836740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:20.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.343+0000 7fb911009740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5555814727b0 0x555581861a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:20.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.343+0000 7fb911009740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5555817ad9b0 con 0x5555814727b0 2024-09-21T03:45:20.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.343+0000 7fb911009740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55558179cde0 con 0x55558146a5d0 2024-09-21T03:45:20.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.343+0000 7fb911009740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x5555817a0bf0 con 0x555581493490 2024-09-21T03:45:20.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.343+0000 7fb903fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5555814727b0 0x555581861a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:20.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.343+0000 7fb903fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5555814727b0 0x555581861a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46700/0 (socket says 172.21.15.38:46700) 2024-09-21T03:45:20.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.343+0000 7fb903fff640 1 -- 172.21.15.38:0/336253293 learned_addr learned my addr 172.21.15.38:0/336253293 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:45:20.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.343+0000 7fb8faffd640 1 --2- 172.21.15.38:0/336253293 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55558146a5d0 0x555581836740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:20.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.343+0000 7fb9037fe640 1 --2- 172.21.15.38:0/336253293 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555581493490 0x555581880c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:20.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.343+0000 7fb903fff640 1 -- 172.21.15.38:0/336253293 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555581493490 msgr2=0x555581880c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:20.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.344+0000 7fb903fff640 1 --2- 172.21.15.38:0/336253293 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555581493490 0x555581880c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:20.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.344+0000 7fb903fff640 1 -- 172.21.15.38:0/336253293 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55558146a5d0 msgr2=0x555581836740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:20.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.344+0000 7fb903fff640 1 --2- 172.21.15.38:0/336253293 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55558146a5d0 0x555581836740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:20.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.344+0000 7fb903fff640 1 -- 172.21.15.38:0/336253293 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555581577900 con 0x5555814727b0 2024-09-21T03:45:20.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.344+0000 7fb903fff640 1 --2- 172.21.15.38:0/336253293 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5555814727b0 0x555581861a20 secure :-1 s=READY pgs=2469 cs=0 l=1 rev1=1 crypto rx=0x7fb8fc006020 tx=0x7fb8fc05f3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2135421d5a3a98be server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:20.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.344+0000 7fb8f97fa640 1 -- 172.21.15.38:0/336253293 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8fc062f10 con 0x5555814727b0 2024-09-21T03:45:20.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.344+0000 7fb8f97fa640 1 -- 172.21.15.38:0/336253293 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb8fc063400 con 0x5555814727b0 2024-09-21T03:45:20.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.344+0000 7fb8f97fa640 1 -- 172.21.15.38:0/336253293 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8fc0628c0 con 0x5555814727b0 2024-09-21T03:45:20.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.345+0000 7fb911009740 1 -- 172.21.15.38:0/336253293 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5555814727b0 msgr2=0x555581861a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:20.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.345+0000 7fb911009740 1 --2- 172.21.15.38:0/336253293 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5555814727b0 0x555581861a20 secure :-1 s=READY pgs=2469 cs=0 l=1 rev1=1 crypto rx=0x7fb8fc006020 tx=0x7fb8fc05f3a0 comp rx=0 tx=0).stop 2024-09-21T03:45:20.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.345+0000 7fb911009740 1 -- 172.21.15.38:0/336253293 shutdown_connections 2024-09-21T03:45:20.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.345+0000 7fb911009740 1 --2- 172.21.15.38:0/336253293 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5555814727b0 0x555581861a20 unknown :-1 s=CLOSED pgs=2469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:20.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.345+0000 7fb911009740 1 --2- 172.21.15.38:0/336253293 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55558146a5d0 0x555581836740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:20.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.345+0000 7fb911009740 1 --2- 172.21.15.38:0/336253293 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555581493490 0x555581880c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:20.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.345+0000 7fb911009740 1 -- 172.21.15.38:0/336253293 >> 172.21.15.38:0/336253293 conn(0x5555814875e0 msgr2=0x55558183af10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:20.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.345+0000 7fb911009740 1 -- 172.21.15.38:0/336253293 shutdown_connections 2024-09-21T03:45:20.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.345+0000 7fb911009740 1 -- 172.21.15.38:0/336253293 wait complete. 2024-09-21T03:45:20.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.346+0000 7fb911009740 1 -- start start 2024-09-21T03:45:20.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.346+0000 7fb911009740 1 Processor -- start 2024-09-21T03:45:20.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.347+0000 7fb911009740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55558146a5d0 0x555581897660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:20.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.347+0000 7fb911009740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5555814727b0 0x555581865500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:20.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.347+0000 7fb911009740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555581493490 0x555581896ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:20.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.347+0000 7fb911009740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55558150d840 con 0x55558146a5d0 2024-09-21T03:45:20.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.347+0000 7fb911009740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x555581723710 con 0x555581493490 2024-09-21T03:45:20.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.347+0000 7fb911009740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x5555818647c0 con 0x5555814727b0 2024-09-21T03:45:20.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.347+0000 7fb9037fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55558146a5d0 0x555581897660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:20.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.347+0000 7fb911009740 10 client.0 _ll_register_callbacks cb 0x5555817964e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:45:20.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.347+0000 7fb9037fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55558146a5d0 0x555581897660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46716/0 (socket says 172.21.15.38:46716) 2024-09-21T03:45:20.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.347+0000 7fb9037fe640 1 -- 172.21.15.38:0/3834871663 learned_addr learned my addr 172.21.15.38:0/3834871663 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:45:20.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.348+0000 7fb8faffd640 1 --2- 172.21.15.38:0/3834871663 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5555814727b0 0x555581865500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:20.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.348+0000 7fb903fff640 1 --2- 172.21.15.38:0/3834871663 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555581493490 0x555581896ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:20.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.348+0000 7fb9037fe640 1 -- 172.21.15.38:0/3834871663 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5555814727b0 msgr2=0x555581865500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:20.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.348+0000 7fb9037fe640 1 --2- 172.21.15.38:0/3834871663 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5555814727b0 0x555581865500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:20.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.348+0000 7fb9037fe640 1 -- 172.21.15.38:0/3834871663 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555581493490 msgr2=0x555581896ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:20.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.348+0000 7fb9037fe640 1 --2- 172.21.15.38:0/3834871663 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555581493490 0x555581896ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:20.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.348+0000 7fb9037fe640 1 -- 172.21.15.38:0/3834871663 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5555814dc7a0 con 0x55558146a5d0 2024-09-21T03:45:20.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.348+0000 7fb9037fe640 1 --2- 172.21.15.38:0/3834871663 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55558146a5d0 0x555581897660 secure :-1 s=READY pgs=2470 cs=0 l=1 rev1=1 crypto rx=0x7fb8f4009c70 tx=0x7fb8f4005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=787476fdea902339 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:20.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.348+0000 7fb8e27fc640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-21T03:45:20.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.348+0000 7fb8e27fc640 1 -- 172.21.15.38:0/3834871663 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8f404d030 con 0x55558146a5d0 2024-09-21T03:45:20.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.348+0000 7fb8e27fc640 1 -- 172.21.15.38:0/3834871663 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(149..149 src has 1..149) ==== 6303+0+0 (secure 0 0 0) 0x7fb8f403da30 con 0x55558146a5d0 2024-09-21T03:45:20.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.348+0000 7fb911009740 1 -- 172.21.15.38:0/3834871663 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb8fc063400 con 0x55558146a5d0 2024-09-21T03:45:20.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.348+0000 7fb8e27fc640 1 -- 172.21.15.38:0/3834871663 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=349) ==== 24+0+0 (secure 0 0 0) 0x7fb8f4059050 con 0x55558146a5d0 2024-09-21T03:45:20.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.349+0000 7fb911009740 10 client.11115 fetch_fsmap learned FSMap version 349 2024-09-21T03:45:20.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.349+0000 7fb911009740 1 -- 172.21.15.38:0/3834871663 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=349,monmap=2+}) -- 0x5555815d8090 con 0x55558146a5d0 2024-09-21T03:45:20.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.349+0000 7fb8e27fc640 1 -- 172.21.15.38:0/3834871663 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb8fc063400 con 0x55558146a5d0 2024-09-21T03:45:20.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.349+0000 7fb8e27fc640 1 -- 172.21.15.38:0/3834871663 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 349) ==== 3502+0+0 (secure 0 0 0) 0x7fb8f405c020 con 0x55558146a5d0 2024-09-21T03:45:20.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.349+0000 7fb911009740 10 client.11115 fetch_fsmap finished waiting for FSMap version 349 2024-09-21T03:45:20.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.349+0000 7fb911009740 10 client.11115 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T03:45:20.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.349+0000 7fb911009740 20 client.11115 populate_metadata read hostname 'smithi038' 2024-09-21T03:45:20.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.349+0000 7fb911009740 1 --2- 172.21.15.38:0/3834871663 >> [v2:172.21.15.38:6832/3839260519,v1:172.21.15.38:6833/3839260519] conn(0x555581452060 0x555581877f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:20.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.349+0000 7fb911009740 4 client.11115 mds_command: new command op to 10935 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:45:20.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.349+0000 7fb911009740 1 -- 172.21.15.38:0/3834871663 --> [v2:172.21.15.38:6832/3839260519,v1:172.21.15.38:6833/3839260519] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5555817af900 con 0x555581452060 2024-09-21T03:45:20.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.349+0000 7fb8faffd640 1 --2- 172.21.15.38:0/3834871663 >> [v2:172.21.15.38:6832/3839260519,v1:172.21.15.38:6833/3839260519] conn(0x555581452060 0x555581877f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:20.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.349+0000 7fb8e27fc640 10 client.11115 ms_handle_connect on v2:172.21.15.38:6832/3839260519 2024-09-21T03:45:20.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.349+0000 7fb8faffd640 1 --2- 172.21.15.38:0/3834871663 >> [v2:172.21.15.38:6832/3839260519,v1:172.21.15.38:6833/3839260519] conn(0x555581452060 0x555581877f40 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=714142dca215e5a7 server_cookie=8307d2c2e11fcc5 in_seq=0 out_seq=0 2024-09-21T03:45:20.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.350+0000 7fb8e27fc640 1 -- 172.21.15.38:0/3834871663 <== mds.0 v2:172.21.15.38:6832/3839260519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5555817af900 con 0x555581452060 2024-09-21T03:45:20.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.350+0000 7fb8e27fc640 10 client.11115 handle_command_reply: tid=0 2024-09-21T03:45:20.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.350+0000 7fb911009740 1 client.11115 shutdown 2024-09-21T03:45:20.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.350+0000 7fb911009740 1 -- 172.21.15.38:0/3834871663 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55558146a5d0 msgr2=0x555581897660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:20.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.350+0000 7fb911009740 1 --2- 172.21.15.38:0/3834871663 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55558146a5d0 0x555581897660 secure :-1 s=READY pgs=2470 cs=0 l=1 rev1=1 crypto rx=0x7fb8f4009c70 tx=0x7fb8f4005f90 comp rx=0 tx=0).stop 2024-09-21T03:45:20.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.350+0000 7fb911009740 1 -- 172.21.15.38:0/3834871663 shutdown_connections 2024-09-21T03:45:20.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.350+0000 7fb911009740 1 --2- 172.21.15.38:0/3834871663 >> [v2:172.21.15.38:6832/3839260519,v1:172.21.15.38:6833/3839260519] conn(0x555581452060 0x555581877f40 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:20.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.350+0000 7fb911009740 1 --2- 172.21.15.38:0/3834871663 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555581493490 0x555581896ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:20.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.350+0000 7fb8e27fc640 0 client.11115 ms_handle_reset on v2:172.21.15.38:6832/3839260519 2024-09-21T03:45:20.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.350+0000 7fb911009740 1 --2- 172.21.15.38:0/3834871663 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5555814727b0 0x555581865500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:20.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.350+0000 7fb911009740 1 --2- 172.21.15.38:0/3834871663 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55558146a5d0 0x555581897660 unknown :-1 s=CLOSED pgs=2470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:20.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.350+0000 7fb911009740 1 -- 172.21.15.38:0/3834871663 >> 172.21.15.38:0/3834871663 conn(0x5555814875e0 msgr2=0x555581853060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:20.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.350+0000 7fb911009740 1 -- 172.21.15.38:0/3834871663 shutdown_connections 2024-09-21T03:45:20.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.350+0000 7fb911009740 1 -- 172.21.15.38:0/3834871663 wait complete. 2024-09-21T03:45:20.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.351+0000 7fb911009740 20 client.11115 trim_cache size 0 max 16384 2024-09-21T03:45:20.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.364+0000 7fb911009740 1 Processor -- start 2024-09-21T03:45:20.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.364+0000 7fb911009740 1 -- start start 2024-09-21T03:45:20.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.366+0000 7fb911009740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55558146a5d0 0x555581852e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:20.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.366+0000 7fb911009740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5555814727b0 0x55558183aee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:20.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.366+0000 7fb911009740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555581493490 0x555581875fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:20.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.366+0000 7fb911009740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fb90406ff40 con 0x555581493490 2024-09-21T03:45:20.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.366+0000 7fb911009740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fb90406f2a0 con 0x55558146a5d0 2024-09-21T03:45:20.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.366+0000 7fb911009740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fb90406f8f0 con 0x5555814727b0 2024-09-21T03:45:20.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.366+0000 7fb903fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555581493490 0x555581875fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:20.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.366+0000 7fb903fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555581493490 0x555581875fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46732/0 (socket says 172.21.15.38:46732) 2024-09-21T03:45:20.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.366+0000 7fb903fff640 1 -- 172.21.15.38:0/1020691676 learned_addr learned my addr 172.21.15.38:0/1020691676 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:45:20.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.366+0000 7fb8faffd640 1 --2- 172.21.15.38:0/1020691676 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5555814727b0 0x55558183aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:20.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.366+0000 7fb9037fe640 1 --2- 172.21.15.38:0/1020691676 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55558146a5d0 0x555581852e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:20.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.366+0000 7fb903fff640 1 -- 172.21.15.38:0/1020691676 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5555814727b0 msgr2=0x55558183aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:20.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.366+0000 7fb903fff640 1 --2- 172.21.15.38:0/1020691676 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5555814727b0 0x55558183aee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:20.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.366+0000 7fb903fff640 1 -- 172.21.15.38:0/1020691676 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55558146a5d0 msgr2=0x555581852e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:20.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.366+0000 7fb903fff640 1 --2- 172.21.15.38:0/1020691676 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55558146a5d0 0x555581852e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:20.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.366+0000 7fb903fff640 1 -- 172.21.15.38:0/1020691676 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555581575c00 con 0x555581493490 2024-09-21T03:45:20.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.366+0000 7fb903fff640 1 --2- 172.21.15.38:0/1020691676 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555581493490 0x555581875fd0 secure :-1 s=READY pgs=2471 cs=0 l=1 rev1=1 crypto rx=0x7fb8fc06c180 tx=0x7fb8fc066040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=744484a7abb1ce6c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:20.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.367+0000 7fb8f8ff9640 1 -- 172.21.15.38:0/1020691676 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8fc063400 con 0x555581493490 2024-09-21T03:45:20.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.367+0000 7fb8f8ff9640 1 -- 172.21.15.38:0/1020691676 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb8fc062f10 con 0x555581493490 2024-09-21T03:45:20.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.367+0000 7fb8f8ff9640 1 -- 172.21.15.38:0/1020691676 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8fc0628c0 con 0x555581493490 2024-09-21T03:45:20.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.367+0000 7fb911009740 1 -- 172.21.15.38:0/1020691676 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555581493490 msgr2=0x555581875fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:20.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.367+0000 7fb911009740 1 --2- 172.21.15.38:0/1020691676 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555581493490 0x555581875fd0 secure :-1 s=READY pgs=2471 cs=0 l=1 rev1=1 crypto rx=0x7fb8fc06c180 tx=0x7fb8fc066040 comp rx=0 tx=0).stop 2024-09-21T03:45:20.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.368+0000 7fb911009740 1 -- 172.21.15.38:0/1020691676 shutdown_connections 2024-09-21T03:45:20.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.368+0000 7fb911009740 1 --2- 172.21.15.38:0/1020691676 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555581493490 0x555581875fd0 unknown :-1 s=CLOSED pgs=2471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:20.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.368+0000 7fb911009740 1 --2- 172.21.15.38:0/1020691676 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5555814727b0 0x55558183aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:20.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.368+0000 7fb911009740 1 --2- 172.21.15.38:0/1020691676 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55558146a5d0 0x555581852e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:20.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.368+0000 7fb911009740 1 -- 172.21.15.38:0/1020691676 >> 172.21.15.38:0/1020691676 conn(0x5555814875e0 msgr2=0x55558184ff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:20.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.368+0000 7fb911009740 1 -- 172.21.15.38:0/1020691676 shutdown_connections 2024-09-21T03:45:20.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.368+0000 7fb911009740 1 -- 172.21.15.38:0/1020691676 wait complete. 2024-09-21T03:45:20.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.368+0000 7fb911009740 1 -- start start 2024-09-21T03:45:20.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.369+0000 7fb911009740 1 Processor -- start 2024-09-21T03:45:20.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.370+0000 7fb911009740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55558146a5d0 0x5555818a7780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:20.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.370+0000 7fb911009740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5555814727b0 0x555581903640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:20.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.370+0000 7fb911009740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555581493490 0x55558184ff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:20.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.370+0000 7fb911009740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x555581863480 con 0x5555814727b0 2024-09-21T03:45:20.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.370+0000 7fb911009740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x555581897000 con 0x55558146a5d0 2024-09-21T03:45:20.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.370+0000 7fb911009740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x555581863a40 con 0x555581493490 2024-09-21T03:45:20.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.370+0000 7fb911009740 10 client.0 _ll_register_callbacks cb 0x7fb8f000c190 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:45:20.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.370+0000 7fb8faffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5555814727b0 0x555581903640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:20.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.370+0000 7fb8faffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5555814727b0 0x555581903640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46738/0 (socket says 172.21.15.38:46738) 2024-09-21T03:45:20.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.370+0000 7fb8faffd640 1 -- 172.21.15.38:0/2420845335 learned_addr learned my addr 172.21.15.38:0/2420845335 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:45:20.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.370+0000 7fb903fff640 1 --2- 172.21.15.38:0/2420845335 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555581493490 0x55558184ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:20.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.370+0000 7fb9037fe640 1 --2- 172.21.15.38:0/2420845335 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55558146a5d0 0x5555818a7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:20.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.370+0000 7fb8faffd640 1 -- 172.21.15.38:0/2420845335 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555581493490 msgr2=0x55558184ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:20.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.370+0000 7fb8faffd640 1 --2- 172.21.15.38:0/2420845335 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555581493490 0x55558184ff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:20.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.370+0000 7fb8faffd640 1 -- 172.21.15.38:0/2420845335 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55558146a5d0 msgr2=0x5555818a7780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:20.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.370+0000 7fb8faffd640 1 --2- 172.21.15.38:0/2420845335 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55558146a5d0 0x5555818a7780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:20.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.370+0000 7fb8faffd640 1 -- 172.21.15.38:0/2420845335 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5555818a7cc0 con 0x5555814727b0 2024-09-21T03:45:20.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.371+0000 7fb8faffd640 1 --2- 172.21.15.38:0/2420845335 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5555814727b0 0x555581903640 secure :-1 s=READY pgs=2472 cs=0 l=1 rev1=1 crypto rx=0x7fb8ec001880 tx=0x7fb8ec013820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cff775fb3eeaf15d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:20.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.371+0000 7fb8e27fc640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-21T03:45:20.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.371+0000 7fb8e27fc640 1 -- 172.21.15.38:0/2420845335 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8ec004d50 con 0x5555814727b0 2024-09-21T03:45:20.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.371+0000 7fb8e27fc640 1 -- 172.21.15.38:0/2420845335 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(149..149 src has 1..149) ==== 6303+0+0 (secure 0 0 0) 0x7fb8ec01f3b0 con 0x5555814727b0 2024-09-21T03:45:20.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.371+0000 7fb911009740 1 -- 172.21.15.38:0/2420845335 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb8fc062f10 con 0x5555814727b0 2024-09-21T03:45:20.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.371+0000 7fb8e27fc640 1 -- 172.21.15.38:0/2420845335 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=349) ==== 24+0+0 (secure 0 0 0) 0x7fb8ec031050 con 0x5555814727b0 2024-09-21T03:45:20.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.371+0000 7fb911009740 10 client.11121 fetch_fsmap learned FSMap version 349 2024-09-21T03:45:20.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.371+0000 7fb911009740 1 -- 172.21.15.38:0/2420845335 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=349,monmap=2+}) -- 0x555581876810 con 0x5555814727b0 2024-09-21T03:45:20.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.371+0000 7fb8e27fc640 1 -- 172.21.15.38:0/2420845335 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb8fc062f10 con 0x5555814727b0 2024-09-21T03:45:20.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.371+0000 7fb8e27fc640 1 -- 172.21.15.38:0/2420845335 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 349) ==== 3502+0+0 (secure 0 0 0) 0x7fb8ec034020 con 0x5555814727b0 2024-09-21T03:45:20.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.371+0000 7fb911009740 10 client.11121 fetch_fsmap finished waiting for FSMap version 349 2024-09-21T03:45:20.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.371+0000 7fb911009740 10 client.11121 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T03:45:20.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.371+0000 7fb911009740 20 client.11121 populate_metadata read hostname 'smithi038' 2024-09-21T03:45:20.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.371+0000 7fb911009740 1 --2- 172.21.15.38:0/2420845335 >> [v2:172.21.15.38:6832/3839260519,v1:172.21.15.38:6833/3839260519] conn(0x555581907930 0x555581864950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:20.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.371+0000 7fb911009740 4 client.11121 mds_command: new command op to 10935 tid=0 multi_id=0 [{"prefix": "damage rm", "damage_id": 433208531}] 2024-09-21T03:45:20.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.371+0000 7fb911009740 1 -- 172.21.15.38:0/2420845335 --> [v2:172.21.15.38:6832/3839260519,v1:172.21.15.38:6833/3839260519] -- command(tid 0: {"prefix": "damage rm", "damage_id": 433208531}) -- 0x5555817ab3b0 con 0x555581907930 2024-09-21T03:45:20.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.371+0000 7fb9037fe640 1 --2- 172.21.15.38:0/2420845335 >> [v2:172.21.15.38:6832/3839260519,v1:172.21.15.38:6833/3839260519] conn(0x555581907930 0x555581864950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:20.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.372+0000 7fb8e27fc640 10 client.11121 ms_handle_connect on v2:172.21.15.38:6832/3839260519 2024-09-21T03:45:20.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.372+0000 7fb9037fe640 1 --2- 172.21.15.38:0/2420845335 >> [v2:172.21.15.38:6832/3839260519,v1:172.21.15.38:6833/3839260519] conn(0x555581907930 0x555581864950 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1aa0d62980158176 server_cookie=b6339ef69546ded6 in_seq=0 out_seq=0 2024-09-21T03:45:20.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.372+0000 7fb8e27fc640 1 -- 172.21.15.38:0/2420845335 <== mds.0 v2:172.21.15.38:6832/3839260519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+0 (crc 0 0 0) 0x5555817ab3b0 con 0x555581907930 2024-09-21T03:45:20.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.372+0000 7fb8e27fc640 10 client.11121 handle_command_reply: tid=0 2024-09-21T03:45:20.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.372+0000 7fb911009740 1 client.11121 shutdown 2024-09-21T03:45:20.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.372+0000 7fb911009740 1 -- 172.21.15.38:0/2420845335 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5555814727b0 msgr2=0x555581903640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:20.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.372+0000 7fb911009740 1 --2- 172.21.15.38:0/2420845335 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5555814727b0 0x555581903640 secure :-1 s=READY pgs=2472 cs=0 l=1 rev1=1 crypto rx=0x7fb8ec001880 tx=0x7fb8ec013820 comp rx=0 tx=0).stop 2024-09-21T03:45:20.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.372+0000 7fb911009740 1 -- 172.21.15.38:0/2420845335 shutdown_connections 2024-09-21T03:45:20.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.372+0000 7fb911009740 1 --2- 172.21.15.38:0/2420845335 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555581493490 0x55558184ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:20.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.372+0000 7fb911009740 1 --2- 172.21.15.38:0/2420845335 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5555814727b0 0x555581903640 unknown :-1 s=CLOSED pgs=2472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:20.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.372+0000 7fb911009740 1 --2- 172.21.15.38:0/2420845335 >> [v2:172.21.15.38:6832/3839260519,v1:172.21.15.38:6833/3839260519] conn(0x555581907930 0x555581864950 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:20.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.372+0000 7fb911009740 1 --2- 172.21.15.38:0/2420845335 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55558146a5d0 0x5555818a7780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:20.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.372+0000 7fb911009740 1 -- 172.21.15.38:0/2420845335 >> 172.21.15.38:0/2420845335 conn(0x5555814875e0 msgr2=0x55558183aee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:20.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.372+0000 7fb8e27fc640 0 client.11121 ms_handle_reset on v2:172.21.15.38:6832/3839260519 2024-09-21T03:45:20.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.372+0000 7fb911009740 1 -- 172.21.15.38:0/2420845335 shutdown_connections 2024-09-21T03:45:20.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.372+0000 7fb911009740 1 -- 172.21.15.38:0/2420845335 wait complete. 2024-09-21T03:45:20.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.372+0000 7fb911009740 20 client.11121 trim_cache size 0 max 16384 2024-09-21T03:45:20.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.373+0000 7fb909fc3640 1 -- 172.21.15.38:0/1121717600 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb8e4078bc0 msgr2=0x7fb8e407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:20.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.373+0000 7fb909fc3640 1 --2- 172.21.15.38:0/1121717600 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb8e4078bc0 0x7fb8e407b0d0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7fb8f4035000 tx=0x7fb8f40046c0 comp rx=0 tx=0).stop 2024-09-21T03:45:20.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.373+0000 7fb909fc3640 1 -- 172.21.15.38:0/1121717600 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb9040cc3b0 msgr2=0x7fb9040b3720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:20.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.373+0000 7fb909fc3640 1 --2- 172.21.15.38:0/1121717600 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb9040cc3b0 0x7fb9040b3720 secure :-1 s=READY pgs=2468 cs=0 l=1 rev1=1 crypto rx=0x7fb904072110 tx=0x7fb8fc001b80 comp rx=0 tx=0).stop 2024-09-21T03:45:20.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.373+0000 7fb909fc3640 1 -- 172.21.15.38:0/1121717600 shutdown_connections 2024-09-21T03:45:20.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.373+0000 7fb909fc3640 1 --2- 172.21.15.38:0/1121717600 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb9040cc3b0 0x7fb9040b3720 unknown :-1 s=CLOSED pgs=2468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:20.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.373+0000 7fb909fc3640 1 --2- 172.21.15.38:0/1121717600 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb9040c8b60 0x7fb9040b31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:20.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.373+0000 7fb909fc3640 1 --2- 172.21.15.38:0/1121717600 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb8e4078bc0 0x7fb8e407b0d0 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:20.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.373+0000 7fb909fc3640 1 --2- 172.21.15.38:0/1121717600 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb8f0003c30 0x7fb90415e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:20.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.373+0000 7fb909fc3640 1 -- 172.21.15.38:0/1121717600 >> 172.21.15.38:0/1121717600 conn(0x7fb9040b46e0 msgr2=0x7fb9040b72e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:20.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.373+0000 7fb909fc3640 1 -- 172.21.15.38:0/1121717600 shutdown_connections 2024-09-21T03:45:20.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.373+0000 7fb909fc3640 1 -- 172.21.15.38:0/1121717600 wait complete. 2024-09-21T03:45:20.387 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:45:20.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.500+0000 7fb54b558640 1 -- 172.21.15.38:0/605866282 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb5440d0b40 msgr2=0x7fb54406e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:20.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.500+0000 7fb54b558640 1 --2- 172.21.15.38:0/605866282 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb5440d0b40 0x7fb54406e6f0 secure :-1 s=READY pgs=2473 cs=0 l=1 rev1=1 crypto rx=0x7fb53c005fd0 tx=0x7fb53c02a5d0 comp rx=0 tx=0).stop 2024-09-21T03:45:20.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.500+0000 7fb54b558640 1 -- 172.21.15.38:0/605866282 shutdown_connections 2024-09-21T03:45:20.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.500+0000 7fb54b558640 1 --2- 172.21.15.38:0/605866282 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb54406ec30 0x7fb5440b3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:20.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.500+0000 7fb54b558640 1 --2- 172.21.15.38:0/605866282 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb5440d0b40 0x7fb54406e6f0 unknown :-1 s=CLOSED pgs=2473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:20.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.500+0000 7fb54b558640 1 -- 172.21.15.38:0/605866282 >> 172.21.15.38:0/605866282 conn(0x7fb5440b2cc0 msgr2=0x7fb5440b30d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:20.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.501+0000 7fb54b558640 1 -- 172.21.15.38:0/605866282 shutdown_connections 2024-09-21T03:45:20.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.501+0000 7fb54b558640 1 -- 172.21.15.38:0/605866282 wait complete. 2024-09-21T03:45:20.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.502+0000 7fb54b558640 1 Processor -- start 2024-09-21T03:45:20.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.502+0000 7fb54b558640 1 -- start start 2024-09-21T03:45:20.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.504+0000 7fb54b558640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb54406ec30 0x7fb54414c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:20.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.504+0000 7fb54b558640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb5440b4500 0x7fb54406cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:20.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.504+0000 7fb54b558640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb5440d0b40 0x7fb54406d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:20.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.504+0000 7fb54b558640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fb54415b750 con 0x7fb5440b4500 2024-09-21T03:45:20.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.504+0000 7fb54b558640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fb54415b5d0 con 0x7fb5440d0b40 2024-09-21T03:45:20.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.504+0000 7fb54b558640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fb54415b8d0 con 0x7fb54406ec30 2024-09-21T03:45:20.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.504+0000 7fb548af3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb5440b4500 0x7fb54406cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:20.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.504+0000 7fb548af3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb5440b4500 0x7fb54406cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46768/0 (socket says 172.21.15.38:46768) 2024-09-21T03:45:20.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.504+0000 7fb548af3640 1 -- 172.21.15.38:0/3568395493 learned_addr learned my addr 172.21.15.38:0/3568395493 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:45:20.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.504+0000 7fb5492f4640 1 --2- 172.21.15.38:0/3568395493 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb54406ec30 0x7fb54414c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:20.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.504+0000 7fb549af5640 1 --2- 172.21.15.38:0/3568395493 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb5440d0b40 0x7fb54406d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:20.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.504+0000 7fb548af3640 1 -- 172.21.15.38:0/3568395493 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb54406ec30 msgr2=0x7fb54414c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:20.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.504+0000 7fb548af3640 1 --2- 172.21.15.38:0/3568395493 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb54406ec30 0x7fb54414c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:20.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.504+0000 7fb548af3640 1 -- 172.21.15.38:0/3568395493 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb5440d0b40 msgr2=0x7fb54406d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:20.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.504+0000 7fb548af3640 1 --2- 172.21.15.38:0/3568395493 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb5440d0b40 0x7fb54406d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:20.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.504+0000 7fb548af3640 1 -- 172.21.15.38:0/3568395493 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb54414cce0 con 0x7fb5440b4500 2024-09-21T03:45:20.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.504+0000 7fb548af3640 1 --2- 172.21.15.38:0/3568395493 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb5440b4500 0x7fb54406cde0 secure :-1 s=READY pgs=2474 cs=0 l=1 rev1=1 crypto rx=0x7fb53800e790 tx=0x7fb53800ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:20.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.505+0000 7fb541ffb640 1 -- 172.21.15.38:0/3568395493 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb53800eea0 con 0x7fb5440b4500 2024-09-21T03:45:20.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.505+0000 7fb541ffb640 1 -- 172.21.15.38:0/3568395493 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb53800cdb0 con 0x7fb5440b4500 2024-09-21T03:45:20.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.505+0000 7fb541ffb640 1 -- 172.21.15.38:0/3568395493 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb538010690 con 0x7fb5440b4500 2024-09-21T03:45:20.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.505+0000 7fb54b558640 1 -- 172.21.15.38:0/3568395493 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb54414a030 con 0x7fb5440b4500 2024-09-21T03:45:20.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.505+0000 7fb54b558640 1 -- 172.21.15.38:0/3568395493 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb54414a5f0 con 0x7fb5440b4500 2024-09-21T03:45:20.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.506+0000 7fb54b558640 1 -- 172.21.15.38:0/3568395493 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5440b3fc0 con 0x7fb5440b4500 2024-09-21T03:45:20.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.507+0000 7fb541ffb640 1 -- 172.21.15.38:0/3568395493 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb538010830 con 0x7fb5440b4500 2024-09-21T03:45:20.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.507+0000 7fb541ffb640 1 --2- 172.21.15.38:0/3568395493 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb530078b90 0x7fb53007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:20.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.507+0000 7fb541ffb640 1 -- 172.21.15.38:0/3568395493 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 6303+0+0 (secure 0 0 0) 0x7fb53809b960 con 0x7fb5440b4500 2024-09-21T03:45:20.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.507+0000 7fb5492f4640 1 --2- 172.21.15.38:0/3568395493 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb530078b90 0x7fb53007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:20.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.508+0000 7fb5492f4640 1 --2- 172.21.15.38:0/3568395493 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb530078b90 0x7fb53007b0a0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fb53c033000 tx=0x7fb53c004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:20.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.512+0000 7fb541ffb640 1 -- 172.21.15.38:0/3568395493 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fb54414cce0 con 0x7fb5440b4500 2024-09-21T03:45:20.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.742+0000 7fb54b558640 1 -- 172.21.15.38:0/3568395493 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb5440cc310 con 0x7fb5440b4500 2024-09-21T03:45:20.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.742+0000 7fb541ffb640 1 -- 172.21.15.38:0/3568395493 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 349 v349) ==== 95+0+5241 (secure 0 0 0) 0x7fb538063480 con 0x7fb5440b4500 2024-09-21T03:45:20.745 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:45:20.745 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":349,"btime":"2024-09-21T03:45:15:115320+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8399,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/3606957623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":3606957623},{"type":"v1","addr":"172.21.15.38:6835","nonce":3606957623}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":338},{"gid":8405,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/456651486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":456651486},{"type":"v1","addr":"172.21.15.195:6837","nonce":456651486}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":338},{"gid":9604,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/2649311840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":2649311840},{"type":"v1","addr":"172.21.15.195:6835","nonce":2649311840}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":338}],"filesystems":[{"mdsmap":{"epoch":349,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:44:56.710446+0000","modified":"2024-09-21T03:45:15.115308+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":147,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10935},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10935":{"gid":10935,"name":"c","rank":0,"incarnation":346,"state":"up:active","state_seq":4,"addr":"172.21.15.38:6833/3839260519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":3839260519},{"type":"v1","addr":"172.21.15.38:6833","nonce":3839260519}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10935,"qdb_cluster":[10935]},"id":7}]} 2024-09-21T03:45:20.746 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 349 2024-09-21T03:45:20.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.745+0000 7fb54b558640 1 -- 172.21.15.38:0/3568395493 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb530078b90 msgr2=0x7fb53007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:20.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.745+0000 7fb54b558640 1 --2- 172.21.15.38:0/3568395493 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb530078b90 0x7fb53007b0a0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fb53c033000 tx=0x7fb53c004730 comp rx=0 tx=0).stop 2024-09-21T03:45:20.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.745+0000 7fb54b558640 1 -- 172.21.15.38:0/3568395493 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb5440b4500 msgr2=0x7fb54406cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:20.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.745+0000 7fb54b558640 1 --2- 172.21.15.38:0/3568395493 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb5440b4500 0x7fb54406cde0 secure :-1 s=READY pgs=2474 cs=0 l=1 rev1=1 crypto rx=0x7fb53800e790 tx=0x7fb53800ec50 comp rx=0 tx=0).stop 2024-09-21T03:45:20.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.745+0000 7fb54b558640 1 -- 172.21.15.38:0/3568395493 shutdown_connections 2024-09-21T03:45:20.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.745+0000 7fb54b558640 1 --2- 172.21.15.38:0/3568395493 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb5440d0b40 0x7fb54406d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:20.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.745+0000 7fb54b558640 1 --2- 172.21.15.38:0/3568395493 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb5440b4500 0x7fb54406cde0 unknown :-1 s=CLOSED pgs=2474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:20.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.745+0000 7fb54b558640 1 --2- 172.21.15.38:0/3568395493 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb530078b90 0x7fb53007b0a0 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:20.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.745+0000 7fb54b558640 1 --2- 172.21.15.38:0/3568395493 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb54406ec30 0x7fb54414c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:20.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.745+0000 7fb54b558640 1 -- 172.21.15.38:0/3568395493 >> 172.21.15.38:0/3568395493 conn(0x7fb5440b2cc0 msgr2=0x7fb544073d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:20.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.745+0000 7fb54b558640 1 -- 172.21.15.38:0/3568395493 shutdown_connections 2024-09-21T03:45:20.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.745+0000 7fb54b558640 1 -- 172.21.15.38:0/3568395493 wait complete. 2024-09-21T03:45:20.760 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c damage rm 1495185665 2024-09-21T03:45:20.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.877+0000 7f01bac4d640 1 -- 172.21.15.38:0/3068689266 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f01a0004d40 msgr2=0x7f01a0005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:20.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.877+0000 7f01bac4d640 1 --2- 172.21.15.38:0/3068689266 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f01a0004d40 0x7f01a0005150 secure :-1 s=READY pgs=2476 cs=0 l=1 rev1=1 crypto rx=0x7f01b40c4d20 tx=0x7f01a0005830 comp rx=0 tx=0).stop 2024-09-21T03:45:20.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.877+0000 7f01b91ea640 1 -- 172.21.15.38:0/3068689266 reap_dead start 2024-09-21T03:45:20.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.877+0000 7f01bac4d640 1 -- 172.21.15.38:0/3068689266 shutdown_connections 2024-09-21T03:45:20.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.877+0000 7f01bac4d640 1 -- 172.21.15.38:0/3068689266 >> 172.21.15.38:0/3068689266 conn(0x7f01b40cdca0 msgr2=0x7f01b40ce0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:20.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.877+0000 7f01bac4d640 1 -- 172.21.15.38:0/3068689266 shutdown_connections 2024-09-21T03:45:20.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.877+0000 7f01bac4d640 1 -- 172.21.15.38:0/3068689266 wait complete. 2024-09-21T03:45:20.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.879+0000 7f01bac4d640 1 Processor -- start 2024-09-21T03:45:20.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.879+0000 7f01bac4d640 1 -- start start 2024-09-21T03:45:20.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.880+0000 7f01bac4d640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f01a0004d40 0x7f01b41f99c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:20.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.880+0000 7f01bac4d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f01b409dda0 0x7f01b40cd100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:20.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.880+0000 7f01bac4d640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f01b41e91c0 0x7f01b41e95d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:20.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.880+0000 7f01bac4d640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f01b40af200 con 0x7f01b409dda0 2024-09-21T03:45:20.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.880+0000 7f01bac4d640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f01b40af080 con 0x7f01b41e91c0 2024-09-21T03:45:20.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.880+0000 7f01bac4d640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f01b40b30b0 con 0x7f01a0004d40 2024-09-21T03:45:20.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.881+0000 7f01b3fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f01b409dda0 0x7f01b40cd100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:20.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.881+0000 7f01b3fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f01b409dda0 0x7f01b40cd100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46786/0 (socket says 172.21.15.38:46786) 2024-09-21T03:45:20.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.881+0000 7f01b3fff640 1 -- 172.21.15.38:0/3718773261 learned_addr learned my addr 172.21.15.38:0/3718773261 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:45:20.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.881+0000 7f01b89e9640 1 --2- 172.21.15.38:0/3718773261 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f01a0004d40 0x7f01b41f99c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:20.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.881+0000 7f01b91ea640 1 --2- 172.21.15.38:0/3718773261 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f01b41e91c0 0x7f01b41e95d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:20.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.881+0000 7f01b3fff640 1 -- 172.21.15.38:0/3718773261 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f01a0004d40 msgr2=0x7f01b41f99c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:20.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.881+0000 7f01b3fff640 1 --2- 172.21.15.38:0/3718773261 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f01a0004d40 0x7f01b41f99c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:20.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.881+0000 7f01b3fff640 1 -- 172.21.15.38:0/3718773261 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f01b41e91c0 msgr2=0x7f01b41e95d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:20.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.881+0000 7f01b3fff640 1 --2- 172.21.15.38:0/3718773261 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f01b41e91c0 0x7f01b41e95d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:20.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.881+0000 7f01b3fff640 1 -- 172.21.15.38:0/3718773261 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01b41e9b10 con 0x7f01b409dda0 2024-09-21T03:45:20.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.881+0000 7f01b91ea640 1 --2- 172.21.15.38:0/3718773261 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f01b41e91c0 0x7f01b41e95d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:45:20.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.881+0000 7f01b3fff640 1 --2- 172.21.15.38:0/3718773261 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f01b409dda0 0x7f01b40cd100 secure :-1 s=READY pgs=2477 cs=0 l=1 rev1=1 crypto rx=0x7f01a4011d60 tx=0x7f01a400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:20.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.881+0000 7f01b17fa640 1 -- 172.21.15.38:0/3718773261 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01a4003d80 con 0x7f01b409dda0 2024-09-21T03:45:20.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.881+0000 7f01b17fa640 1 -- 172.21.15.38:0/3718773261 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f01a4004b20 con 0x7f01b409dda0 2024-09-21T03:45:20.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.881+0000 7f01bac4d640 1 -- 172.21.15.38:0/3718773261 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01b409e500 con 0x7f01b409dda0 2024-09-21T03:45:20.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.881+0000 7f01b17fa640 1 -- 172.21.15.38:0/3718773261 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01a4004050 con 0x7f01b409dda0 2024-09-21T03:45:20.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.881+0000 7f01bac4d640 1 -- 172.21.15.38:0/3718773261 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01b414b0b0 con 0x7f01b409dda0 2024-09-21T03:45:20.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.883+0000 7f01b17fa640 1 -- 172.21.15.38:0/3718773261 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f01a40041f0 con 0x7f01b409dda0 2024-09-21T03:45:20.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.884+0000 7f01b17fa640 1 --2- 172.21.15.38:0/3718773261 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0194078b90 0x7f019407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:20.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.884+0000 7f01b17fa640 1 -- 172.21.15.38:0/3718773261 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 6303+0+0 (secure 0 0 0) 0x7f01a409bfc0 con 0x7f01b409dda0 2024-09-21T03:45:20.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.884+0000 7f01b89e9640 1 --2- 172.21.15.38:0/3718773261 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0194078b90 0x7f019407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:20.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.884+0000 7f01c1bc2740 1 Processor -- start 2024-09-21T03:45:20.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.884+0000 7f01c1bc2740 1 -- start start 2024-09-21T03:45:20.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.884+0000 7f01b89e9640 1 --2- 172.21.15.38:0/3718773261 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0194078b90 0x7f019407b0a0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f019c000ff0 tx=0x7f019c0011c0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:20.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.886+0000 7f01c1bc2740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55742f85e2a0 0x55742fc4d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:20.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.886+0000 7f01c1bc2740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55742f8353e0 0x55742fc30220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:20.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.886+0000 7f01c1bc2740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55742f83d5c0 0x55742fc30760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:20.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.886+0000 7f01c1bc2740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55742fb78910 con 0x55742f8353e0 2024-09-21T03:45:20.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.886+0000 7f01c1bc2740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55742fb67d40 con 0x55742f83d5c0 2024-09-21T03:45:20.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.886+0000 7f01c1bc2740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55742fb6bb50 con 0x55742f85e2a0 2024-09-21T03:45:20.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.886+0000 7f01b3fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55742f8353e0 0x55742fc30220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:20.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.886+0000 7f01b3fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55742f8353e0 0x55742fc30220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46792/0 (socket says 172.21.15.38:46792) 2024-09-21T03:45:20.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.886+0000 7f01b3fff640 1 -- 172.21.15.38:0/1116216644 learned_addr learned my addr 172.21.15.38:0/1116216644 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:45:20.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.886+0000 7f01b89e9640 1 --2- 172.21.15.38:0/1116216644 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55742f85e2a0 0x55742fc4d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:20.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.886+0000 7f01b91ea640 1 --2- 172.21.15.38:0/1116216644 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55742f83d5c0 0x55742fc30760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:20.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.886+0000 7f01b3fff640 1 -- 172.21.15.38:0/1116216644 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55742f85e2a0 msgr2=0x55742fc4d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:20.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.886+0000 7f01b3fff640 1 --2- 172.21.15.38:0/1116216644 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55742f85e2a0 0x55742fc4d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:20.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.886+0000 7f01b3fff640 1 -- 172.21.15.38:0/1116216644 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55742f83d5c0 msgr2=0x55742fc30760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:20.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.886+0000 7f01b3fff640 1 --2- 172.21.15.38:0/1116216644 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55742f83d5c0 0x55742fc30760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:20.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.886+0000 7f01b3fff640 1 -- 172.21.15.38:0/1116216644 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55742f942710 con 0x55742f8353e0 2024-09-21T03:45:20.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.886+0000 7f01b3fff640 1 --2- 172.21.15.38:0/1116216644 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55742f8353e0 0x55742fc30220 secure :-1 s=READY pgs=2478 cs=0 l=1 rev1=1 crypto rx=0x7f01a409a240 tx=0x7f01a4063520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9de80bf1633275d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:20.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.887+0000 7f01aa7fc640 1 -- 172.21.15.38:0/1116216644 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01a40642d0 con 0x55742f8353e0 2024-09-21T03:45:20.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.887+0000 7f01aa7fc640 1 -- 172.21.15.38:0/1116216644 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f01a4063bd0 con 0x55742f8353e0 2024-09-21T03:45:20.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.887+0000 7f01aa7fc640 1 -- 172.21.15.38:0/1116216644 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01a406d2a0 con 0x55742f8353e0 2024-09-21T03:45:20.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.887+0000 7f01c1bc2740 1 -- 172.21.15.38:0/1116216644 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55742f8353e0 msgr2=0x55742fc30220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:20.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.887+0000 7f01c1bc2740 1 --2- 172.21.15.38:0/1116216644 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55742f8353e0 0x55742fc30220 secure :-1 s=READY pgs=2478 cs=0 l=1 rev1=1 crypto rx=0x7f01a409a240 tx=0x7f01a4063520 comp rx=0 tx=0).stop 2024-09-21T03:45:20.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.887+0000 7f01c1bc2740 1 -- 172.21.15.38:0/1116216644 shutdown_connections 2024-09-21T03:45:20.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.887+0000 7f01c1bc2740 1 --2- 172.21.15.38:0/1116216644 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55742f83d5c0 0x55742fc30760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:20.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.887+0000 7f01c1bc2740 1 --2- 172.21.15.38:0/1116216644 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55742f8353e0 0x55742fc30220 unknown :-1 s=CLOSED pgs=2478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:20.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.887+0000 7f01c1bc2740 1 --2- 172.21.15.38:0/1116216644 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55742f85e2a0 0x55742fc4d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:20.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.887+0000 7f01c1bc2740 1 -- 172.21.15.38:0/1116216644 >> 172.21.15.38:0/1116216644 conn(0x55742f8523f0 msgr2=0x55742fc06440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:20.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.887+0000 7f01c1bc2740 1 -- 172.21.15.38:0/1116216644 shutdown_connections 2024-09-21T03:45:20.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.887+0000 7f01c1bc2740 1 -- 172.21.15.38:0/1116216644 wait complete. 2024-09-21T03:45:20.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.888+0000 7f01c1bc2740 1 -- start start 2024-09-21T03:45:20.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.888+0000 7f01c1bc2740 1 Processor -- start 2024-09-21T03:45:20.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.890+0000 7f01c1bc2740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55742f8353e0 0x55742fc299c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:20.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.890+0000 7f01c1bc2740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55742f83d5c0 0x55742fc29220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:20.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.890+0000 7f01c1bc2740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55742f85e2a0 0x55742fc44510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:20.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.890+0000 7f01c1bc2740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55742f8d8650 con 0x55742f83d5c0 2024-09-21T03:45:20.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.890+0000 7f01c1bc2740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55742faee5f0 con 0x55742f85e2a0 2024-09-21T03:45:20.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.890+0000 7f01c1bc2740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55742fc2c5f0 con 0x55742f8353e0 2024-09-21T03:45:20.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.890+0000 7f01c1bc2740 10 client.0 _ll_register_callbacks cb 0x55742fb62060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:45:20.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.890+0000 7f01b3fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55742f83d5c0 0x55742fc29220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:20.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.890+0000 7f01b3fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55742f83d5c0 0x55742fc29220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46794/0 (socket says 172.21.15.38:46794) 2024-09-21T03:45:20.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.890+0000 7f01b3fff640 1 -- 172.21.15.38:0/276232550 learned_addr learned my addr 172.21.15.38:0/276232550 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:45:20.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.890+0000 7f01b89e9640 1 --2- 172.21.15.38:0/276232550 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55742f8353e0 0x55742fc299c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:20.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.890+0000 7f01b91ea640 1 --2- 172.21.15.38:0/276232550 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55742f85e2a0 0x55742fc44510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:20.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.890+0000 7f01b3fff640 1 -- 172.21.15.38:0/276232550 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55742f8353e0 msgr2=0x55742fc299c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:20.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.890+0000 7f01b3fff640 1 --2- 172.21.15.38:0/276232550 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55742f8353e0 0x55742fc299c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:20.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.890+0000 7f01b3fff640 1 -- 172.21.15.38:0/276232550 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55742f85e2a0 msgr2=0x55742fc44510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:20.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.890+0000 7f01b3fff640 1 --2- 172.21.15.38:0/276232550 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55742f85e2a0 0x55742fc44510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:20.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.890+0000 7f01b3fff640 1 -- 172.21.15.38:0/276232550 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55742f9a2ea0 con 0x55742f83d5c0 2024-09-21T03:45:20.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.891+0000 7f01b3fff640 1 --2- 172.21.15.38:0/276232550 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55742f83d5c0 0x55742fc29220 secure :-1 s=READY pgs=2479 cs=0 l=1 rev1=1 crypto rx=0x7f01a409a2f0 tx=0x7f01a406d270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dc930cb3020adf1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:20.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.891+0000 7f01937fe640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-21T03:45:20.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.891+0000 7f01937fe640 1 -- 172.21.15.38:0/276232550 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01a406d4d0 con 0x55742f83d5c0 2024-09-21T03:45:20.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.891+0000 7f01937fe640 1 -- 172.21.15.38:0/276232550 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(149..149 src has 1..149) ==== 6303+0+0 (secure 0 0 0) 0x7f01a4068150 con 0x55742f83d5c0 2024-09-21T03:45:20.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.891+0000 7f01c1bc2740 1 -- 172.21.15.38:0/276232550 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55742fb45b60 con 0x55742f83d5c0 2024-09-21T03:45:20.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.891+0000 7f01937fe640 1 -- 172.21.15.38:0/276232550 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=349) ==== 24+0+0 (secure 0 0 0) 0x7f01a4072050 con 0x55742f83d5c0 2024-09-21T03:45:20.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.891+0000 7f01c1bc2740 10 client.11139 fetch_fsmap learned FSMap version 349 2024-09-21T03:45:20.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.891+0000 7f01c1bc2740 1 -- 172.21.15.38:0/276232550 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=349,monmap=2+}) -- 0x55742f8a75b0 con 0x55742f83d5c0 2024-09-21T03:45:20.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.891+0000 7f01937fe640 1 -- 172.21.15.38:0/276232550 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55742fb45b60 con 0x55742f83d5c0 2024-09-21T03:45:20.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.891+0000 7f01937fe640 1 -- 172.21.15.38:0/276232550 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 349) ==== 3502+0+0 (secure 0 0 0) 0x7f01a4075020 con 0x55742f83d5c0 2024-09-21T03:45:20.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.891+0000 7f01c1bc2740 10 client.11139 fetch_fsmap finished waiting for FSMap version 349 2024-09-21T03:45:20.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.891+0000 7f01c1bc2740 10 client.11139 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T03:45:20.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.891+0000 7f01c1bc2740 20 client.11139 populate_metadata read hostname 'smithi038' 2024-09-21T03:45:20.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.891+0000 7f01c1bc2740 1 --2- 172.21.15.38:0/276232550 >> [v2:172.21.15.38:6832/3839260519,v1:172.21.15.38:6833/3839260519] conn(0x55742f81d080 0x55742fc2b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:20.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.891+0000 7f01c1bc2740 4 client.11139 mds_command: new command op to 10935 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:45:20.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.891+0000 7f01c1bc2740 1 -- 172.21.15.38:0/276232550 --> [v2:172.21.15.38:6832/3839260519,v1:172.21.15.38:6833/3839260519] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55742fb7a860 con 0x55742f81d080 2024-09-21T03:45:20.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.891+0000 7f01b89e9640 1 --2- 172.21.15.38:0/276232550 >> [v2:172.21.15.38:6832/3839260519,v1:172.21.15.38:6833/3839260519] conn(0x55742f81d080 0x55742fc2b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:20.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.892+0000 7f01937fe640 10 client.11139 ms_handle_connect on v2:172.21.15.38:6832/3839260519 2024-09-21T03:45:20.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.892+0000 7f01b89e9640 1 --2- 172.21.15.38:0/276232550 >> [v2:172.21.15.38:6832/3839260519,v1:172.21.15.38:6833/3839260519] conn(0x55742f81d080 0x55742fc2b2b0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=255b9b662d00c71a server_cookie=bf4fbb6519f3ed45 in_seq=0 out_seq=0 2024-09-21T03:45:20.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.892+0000 7f01937fe640 1 -- 172.21.15.38:0/276232550 <== mds.0 v2:172.21.15.38:6832/3839260519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55742fb7a860 con 0x55742f81d080 2024-09-21T03:45:20.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.892+0000 7f01937fe640 10 client.11139 handle_command_reply: tid=0 2024-09-21T03:45:20.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.893+0000 7f01c1bc2740 1 client.11139 shutdown 2024-09-21T03:45:20.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.893+0000 7f01c1bc2740 1 -- 172.21.15.38:0/276232550 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55742f83d5c0 msgr2=0x55742fc29220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:20.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.893+0000 7f01c1bc2740 1 --2- 172.21.15.38:0/276232550 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55742f83d5c0 0x55742fc29220 secure :-1 s=READY pgs=2479 cs=0 l=1 rev1=1 crypto rx=0x7f01a409a2f0 tx=0x7f01a406d270 comp rx=0 tx=0).stop 2024-09-21T03:45:20.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.893+0000 7f01c1bc2740 1 -- 172.21.15.38:0/276232550 shutdown_connections 2024-09-21T03:45:20.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.893+0000 7f01c1bc2740 1 --2- 172.21.15.38:0/276232550 >> [v2:172.21.15.38:6832/3839260519,v1:172.21.15.38:6833/3839260519] conn(0x55742f81d080 0x55742fc2b2b0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:20.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.893+0000 7f01c1bc2740 1 --2- 172.21.15.38:0/276232550 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55742f85e2a0 0x55742fc44510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:20.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.893+0000 7f01c1bc2740 1 --2- 172.21.15.38:0/276232550 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55742f83d5c0 0x55742fc29220 unknown :-1 s=CLOSED pgs=2479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:20.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.893+0000 7f01937fe640 0 client.11139 ms_handle_reset on v2:172.21.15.38:6832/3839260519 2024-09-21T03:45:20.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.893+0000 7f01c1bc2740 1 --2- 172.21.15.38:0/276232550 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55742f8353e0 0x55742fc299c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:20.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.893+0000 7f01c1bc2740 1 -- 172.21.15.38:0/276232550 >> 172.21.15.38:0/276232550 conn(0x55742f8523f0 msgr2=0x55742fc4dbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:20.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.893+0000 7f01c1bc2740 1 -- 172.21.15.38:0/276232550 shutdown_connections 2024-09-21T03:45:20.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.893+0000 7f01c1bc2740 1 -- 172.21.15.38:0/276232550 wait complete. 2024-09-21T03:45:20.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.893+0000 7f01c1bc2740 20 client.11139 trim_cache size 0 max 16384 2024-09-21T03:45:20.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.907+0000 7f01c1bc2740 1 Processor -- start 2024-09-21T03:45:20.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.907+0000 7f01c1bc2740 1 -- start start 2024-09-21T03:45:20.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.908+0000 7f01c1bc2740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55742f8353e0 0x55742fc07190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:20.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.908+0000 7f01c1bc2740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55742f83d5c0 0x55742fc076d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:20.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.908+0000 7f01c1bc2740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55742f85e2a0 0x55742fc04420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:20.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.908+0000 7f01c1bc2740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f01b4201ee0 con 0x55742f8353e0 2024-09-21T03:45:20.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.908+0000 7f01c1bc2740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f01b4201220 con 0x55742f83d5c0 2024-09-21T03:45:20.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.908+0000 7f01c1bc2740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f01b4201870 con 0x55742f85e2a0 2024-09-21T03:45:20.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.908+0000 7f01b89e9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55742f8353e0 0x55742fc07190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:20.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.908+0000 7f01b89e9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55742f8353e0 0x55742fc07190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46804/0 (socket says 172.21.15.38:46804) 2024-09-21T03:45:20.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.908+0000 7f01b89e9640 1 -- 172.21.15.38:0/2502847132 learned_addr learned my addr 172.21.15.38:0/2502847132 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:45:20.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.908+0000 7f01b3fff640 1 --2- 172.21.15.38:0/2502847132 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55742f83d5c0 0x55742fc076d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:20.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.908+0000 7f01b89e9640 1 -- 172.21.15.38:0/2502847132 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55742f85e2a0 msgr2=0x55742fc04420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:20.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.908+0000 7f01b89e9640 1 --2- 172.21.15.38:0/2502847132 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55742f85e2a0 0x55742fc04420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:20.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.908+0000 7f01b89e9640 1 -- 172.21.15.38:0/2502847132 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55742f83d5c0 msgr2=0x55742fc076d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:20.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.908+0000 7f01b89e9640 1 --2- 172.21.15.38:0/2502847132 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55742f83d5c0 0x55742fc076d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:20.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.908+0000 7f01b89e9640 1 -- 172.21.15.38:0/2502847132 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55742fc02fa0 con 0x55742f8353e0 2024-09-21T03:45:20.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.908+0000 7f01b3fff640 1 --2- 172.21.15.38:0/2502847132 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55742f83d5c0 0x55742fc076d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:45:20.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.909+0000 7f01b89e9640 1 --2- 172.21.15.38:0/2502847132 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55742f8353e0 0x55742fc07190 secure :-1 s=READY pgs=2480 cs=0 l=1 rev1=1 crypto rx=0x7f019c004b50 tx=0x7f019c015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a27f28970a89b485 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:20.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.909+0000 7f01a9ffb640 1 -- 172.21.15.38:0/2502847132 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f019c02e070 con 0x55742f8353e0 2024-09-21T03:45:20.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.909+0000 7f01a9ffb640 1 -- 172.21.15.38:0/2502847132 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f019c0138e0 con 0x55742f8353e0 2024-09-21T03:45:20.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.909+0000 7f01a9ffb640 1 -- 172.21.15.38:0/2502847132 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f019c013af0 con 0x55742f8353e0 2024-09-21T03:45:20.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.909+0000 7f01c1bc2740 1 -- 172.21.15.38:0/2502847132 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55742f8353e0 msgr2=0x55742fc07190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:20.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.909+0000 7f01c1bc2740 1 --2- 172.21.15.38:0/2502847132 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55742f8353e0 0x55742fc07190 secure :-1 s=READY pgs=2480 cs=0 l=1 rev1=1 crypto rx=0x7f019c004b50 tx=0x7f019c015000 comp rx=0 tx=0).stop 2024-09-21T03:45:20.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.909+0000 7f01c1bc2740 1 -- 172.21.15.38:0/2502847132 shutdown_connections 2024-09-21T03:45:20.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.909+0000 7f01c1bc2740 1 --2- 172.21.15.38:0/2502847132 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55742f85e2a0 0x55742fc04420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:20.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.909+0000 7f01c1bc2740 1 --2- 172.21.15.38:0/2502847132 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55742f83d5c0 0x55742fc076d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:20.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.909+0000 7f01c1bc2740 1 --2- 172.21.15.38:0/2502847132 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55742f8353e0 0x55742fc07190 unknown :-1 s=CLOSED pgs=2480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:20.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.909+0000 7f01c1bc2740 1 -- 172.21.15.38:0/2502847132 >> 172.21.15.38:0/2502847132 conn(0x55742f8523f0 msgr2=0x55742fc29790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:20.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.910+0000 7f01c1bc2740 1 -- 172.21.15.38:0/2502847132 shutdown_connections 2024-09-21T03:45:20.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.910+0000 7f01c1bc2740 1 -- 172.21.15.38:0/2502847132 wait complete. 2024-09-21T03:45:20.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.910+0000 7f01c1bc2740 1 -- start start 2024-09-21T03:45:20.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.911+0000 7f01c1bc2740 1 Processor -- start 2024-09-21T03:45:20.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.912+0000 7f01c1bc2740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55742f8353e0 0x55742fccdd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:20.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.912+0000 7f01c1bc2740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55742f83d5c0 0x55742fc283b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:20.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.912+0000 7f01c1bc2740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55742f85e2a0 0x55742fc4d980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:20.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.912+0000 7f01c1bc2740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55742fc05f70 con 0x55742f85e2a0 2024-09-21T03:45:20.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.912+0000 7f01c1bc2740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55742f8d8650 con 0x55742f83d5c0 2024-09-21T03:45:20.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.912+0000 7f01c1bc2740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55742fc5d4c0 con 0x55742f8353e0 2024-09-21T03:45:20.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.912+0000 7f01c1bc2740 10 client.0 _ll_register_callbacks cb 0x7f01a405fb00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:45:20.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.912+0000 7f01b91ea640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55742f85e2a0 0x55742fc4d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:20.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.912+0000 7f01b91ea640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55742f85e2a0 0x55742fc4d980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46820/0 (socket says 172.21.15.38:46820) 2024-09-21T03:45:20.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.912+0000 7f01b91ea640 1 -- 172.21.15.38:0/2771408398 learned_addr learned my addr 172.21.15.38:0/2771408398 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:45:20.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.912+0000 7f01b3fff640 1 --2- 172.21.15.38:0/2771408398 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55742f83d5c0 0x55742fc283b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:20.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.912+0000 7f01b89e9640 1 --2- 172.21.15.38:0/2771408398 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55742f8353e0 0x55742fccdd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:20.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.912+0000 7f01b91ea640 1 -- 172.21.15.38:0/2771408398 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55742f8353e0 msgr2=0x55742fccdd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:20.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.912+0000 7f01b91ea640 1 --2- 172.21.15.38:0/2771408398 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55742f8353e0 0x55742fccdd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:20.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.912+0000 7f01b91ea640 1 -- 172.21.15.38:0/2771408398 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55742f83d5c0 msgr2=0x55742fc283b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:20.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.912+0000 7f01b91ea640 1 --2- 172.21.15.38:0/2771408398 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55742f83d5c0 0x55742fc283b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:20.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.912+0000 7f01b91ea640 1 -- 172.21.15.38:0/2771408398 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55742fc46030 con 0x55742f85e2a0 2024-09-21T03:45:20.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.912+0000 7f01b89e9640 1 --2- 172.21.15.38:0/2771408398 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55742f8353e0 0x55742fccdd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:45:20.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.913+0000 7f01b91ea640 1 --2- 172.21.15.38:0/2771408398 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55742f85e2a0 0x55742fc4d980 secure :-1 s=READY pgs=2481 cs=0 l=1 rev1=1 crypto rx=0x7f01ac0096b0 tx=0x7f01ac02bc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ffb37ea8c3c2fdc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:20.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.913+0000 7f01937fe640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-21T03:45:20.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.913+0000 7f01937fe640 1 -- 172.21.15.38:0/2771408398 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01ac038040 con 0x55742f85e2a0 2024-09-21T03:45:20.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.913+0000 7f01937fe640 1 -- 172.21.15.38:0/2771408398 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(149..149 src has 1..149) ==== 6303+0+0 (secure 0 0 0) 0x7f01ac03e450 con 0x55742f85e2a0 2024-09-21T03:45:20.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.913+0000 7f01c1bc2740 1 -- 172.21.15.38:0/2771408398 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f019c0138e0 con 0x55742f85e2a0 2024-09-21T03:45:20.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.913+0000 7f01937fe640 1 -- 172.21.15.38:0/2771408398 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=349) ==== 24+0+0 (secure 0 0 0) 0x7f01ac04e050 con 0x55742f85e2a0 2024-09-21T03:45:20.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.913+0000 7f01c1bc2740 10 client.11145 fetch_fsmap learned FSMap version 349 2024-09-21T03:45:20.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.913+0000 7f01c1bc2740 1 -- 172.21.15.38:0/2771408398 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=349,monmap=2+}) -- 0x55742fc2d490 con 0x55742f85e2a0 2024-09-21T03:45:20.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.913+0000 7f01937fe640 1 -- 172.21.15.38:0/2771408398 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f019c0138e0 con 0x55742f85e2a0 2024-09-21T03:45:20.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.913+0000 7f01937fe640 1 -- 172.21.15.38:0/2771408398 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 349) ==== 3502+0+0 (secure 0 0 0) 0x7f01ac051020 con 0x55742f85e2a0 2024-09-21T03:45:20.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.913+0000 7f01c1bc2740 10 client.11145 fetch_fsmap finished waiting for FSMap version 349 2024-09-21T03:45:20.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.913+0000 7f01c1bc2740 10 client.11145 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T03:45:20.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.913+0000 7f01c1bc2740 20 client.11145 populate_metadata read hostname 'smithi038' 2024-09-21T03:45:20.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.913+0000 7f01c1bc2740 1 --2- 172.21.15.38:0/2771408398 >> [v2:172.21.15.38:6832/3839260519,v1:172.21.15.38:6833/3839260519] conn(0x55742fc4dec0 0x55742fc4a240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:20.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.913+0000 7f01c1bc2740 4 client.11145 mds_command: new command op to 10935 tid=0 multi_id=0 [{"prefix": "damage rm", "damage_id": 1495185665}] 2024-09-21T03:45:20.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.913+0000 7f01c1bc2740 1 -- 172.21.15.38:0/2771408398 --> [v2:172.21.15.38:6832/3839260519,v1:172.21.15.38:6833/3839260519] -- command(tid 0: {"prefix": "damage rm", "damage_id": 1495185665}) -- 0x55742fc4c390 con 0x55742fc4dec0 2024-09-21T03:45:20.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.914+0000 7f01b89e9640 1 --2- 172.21.15.38:0/2771408398 >> [v2:172.21.15.38:6832/3839260519,v1:172.21.15.38:6833/3839260519] conn(0x55742fc4dec0 0x55742fc4a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:20.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.914+0000 7f01937fe640 10 client.11145 ms_handle_connect on v2:172.21.15.38:6832/3839260519 2024-09-21T03:45:20.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.914+0000 7f01b89e9640 1 --2- 172.21.15.38:0/2771408398 >> [v2:172.21.15.38:6832/3839260519,v1:172.21.15.38:6833/3839260519] conn(0x55742fc4dec0 0x55742fc4a240 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=187c40fd292cdc8c server_cookie=157991526a819107 in_seq=0 out_seq=0 2024-09-21T03:45:20.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.914+0000 7f01937fe640 1 -- 172.21.15.38:0/2771408398 <== mds.0 v2:172.21.15.38:6832/3839260519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+0 (crc 0 0 0) 0x55742fc4c390 con 0x55742fc4dec0 2024-09-21T03:45:20.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.914+0000 7f01937fe640 10 client.11145 handle_command_reply: tid=0 2024-09-21T03:45:20.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.914+0000 7f01c1bc2740 1 client.11145 shutdown 2024-09-21T03:45:20.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.914+0000 7f01c1bc2740 1 -- 172.21.15.38:0/2771408398 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55742f85e2a0 msgr2=0x55742fc4d980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:20.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.914+0000 7f01c1bc2740 1 --2- 172.21.15.38:0/2771408398 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55742f85e2a0 0x55742fc4d980 secure :-1 s=READY pgs=2481 cs=0 l=1 rev1=1 crypto rx=0x7f01ac0096b0 tx=0x7f01ac02bc70 comp rx=0 tx=0).stop 2024-09-21T03:45:20.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.914+0000 7f01c1bc2740 1 -- 172.21.15.38:0/2771408398 shutdown_connections 2024-09-21T03:45:20.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.914+0000 7f01c1bc2740 1 --2- 172.21.15.38:0/2771408398 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55742f85e2a0 0x55742fc4d980 unknown :-1 s=CLOSED pgs=2481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:20.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.914+0000 7f01c1bc2740 1 --2- 172.21.15.38:0/2771408398 >> [v2:172.21.15.38:6832/3839260519,v1:172.21.15.38:6833/3839260519] conn(0x55742fc4dec0 0x55742fc4a240 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:20.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.914+0000 7f01c1bc2740 1 --2- 172.21.15.38:0/2771408398 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55742f83d5c0 0x55742fc283b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:20.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.914+0000 7f01937fe640 0 client.11145 ms_handle_reset on v2:172.21.15.38:6832/3839260519 2024-09-21T03:45:20.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.914+0000 7f01c1bc2740 1 --2- 172.21.15.38:0/2771408398 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55742f8353e0 0x55742fccdd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:20.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.914+0000 7f01c1bc2740 1 -- 172.21.15.38:0/2771408398 >> 172.21.15.38:0/2771408398 conn(0x55742f8523f0 msgr2=0x55742fc1f1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:20.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.915+0000 7f01c1bc2740 1 -- 172.21.15.38:0/2771408398 shutdown_connections 2024-09-21T03:45:20.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.915+0000 7f01c1bc2740 1 -- 172.21.15.38:0/2771408398 wait complete. 2024-09-21T03:45:20.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.915+0000 7f01c1bc2740 20 client.11145 trim_cache size 0 max 16384 2024-09-21T03:45:20.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.915+0000 7f01bac4d640 1 -- 172.21.15.38:0/3718773261 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0194078b90 msgr2=0x7f019407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:20.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.915+0000 7f01bac4d640 1 --2- 172.21.15.38:0/3718773261 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0194078b90 0x7f019407b0a0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f019c000ff0 tx=0x7f019c0011c0 comp rx=0 tx=0).stop 2024-09-21T03:45:20.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.915+0000 7f01bac4d640 1 -- 172.21.15.38:0/3718773261 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f01b409dda0 msgr2=0x7f01b40cd100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:20.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.915+0000 7f01bac4d640 1 --2- 172.21.15.38:0/3718773261 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f01b409dda0 0x7f01b40cd100 secure :-1 s=READY pgs=2477 cs=0 l=1 rev1=1 crypto rx=0x7f01a4011d60 tx=0x7f01a400f790 comp rx=0 tx=0).stop 2024-09-21T03:45:20.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.915+0000 7f01bac4d640 1 -- 172.21.15.38:0/3718773261 shutdown_connections 2024-09-21T03:45:20.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.915+0000 7f01bac4d640 1 --2- 172.21.15.38:0/3718773261 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f01b41e91c0 0x7f01b41e95d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:20.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.915+0000 7f01bac4d640 1 --2- 172.21.15.38:0/3718773261 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f01b409dda0 0x7f01b40cd100 unknown :-1 s=CLOSED pgs=2477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:20.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.915+0000 7f01bac4d640 1 --2- 172.21.15.38:0/3718773261 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0194078b90 0x7f019407b0a0 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:20.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.915+0000 7f01bac4d640 1 --2- 172.21.15.38:0/3718773261 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f01a0004d40 0x7f01b41f99c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:20.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.915+0000 7f01bac4d640 1 -- 172.21.15.38:0/3718773261 >> 172.21.15.38:0/3718773261 conn(0x7f01b40cdca0 msgr2=0x7f01b415ee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:20.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.915+0000 7f01bac4d640 1 -- 172.21.15.38:0/3718773261 shutdown_connections 2024-09-21T03:45:20.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:20.915+0000 7f01bac4d640 1 -- 172.21.15.38:0/3718773261 wait complete. 2024-09-21T03:45:20.929 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-21T03:45:20.930 DEBUG:teuthology.orchestra.run.smithi038:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T03:45:20.977 DEBUG:teuthology.orchestra.run.smithi195:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T03:45:21.026 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-21T03:45:21.027 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T03:45:21.027 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-21T03:45:21.027 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:45:21.043 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-21T03:45:21.043 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:45:21.102 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T03:45:21.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:21.254+0000 7fba19b4a640 1 -- 172.21.15.38:0/2540902943 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fba140ccc40 msgr2=0x7fba1415ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:21.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:21.254+0000 7fba19b4a640 1 --2- 172.21.15.38:0/2540902943 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fba140ccc40 0x7fba1415ca90 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fba04009830 tx=0x7fba0402a480 comp rx=0 tx=0).stop 2024-09-21T03:45:21.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:21.254+0000 7fba19b4a640 1 -- 172.21.15.38:0/2540902943 shutdown_connections 2024-09-21T03:45:21.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:21.254+0000 7fba19b4a640 1 --2- 172.21.15.38:0/2540902943 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fba140cd440 0x7fba140d0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:21.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:21.254+0000 7fba19b4a640 1 --2- 172.21.15.38:0/2540902943 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fba140ccc40 0x7fba1415ca90 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:21.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:21.254+0000 7fba19b4a640 1 -- 172.21.15.38:0/2540902943 >> 172.21.15.38:0/2540902943 conn(0x7fba140b46e0 msgr2=0x7fba140d23e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:21.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:21.254+0000 7fba19b4a640 1 -- 172.21.15.38:0/2540902943 shutdown_connections 2024-09-21T03:45:21.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:21.254+0000 7fba19b4a640 1 -- 172.21.15.38:0/2540902943 wait complete. 2024-09-21T03:45:21.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:21.256+0000 7fba19b4a640 1 Processor -- start 2024-09-21T03:45:21.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:21.256+0000 7fba19b4a640 1 -- start start 2024-09-21T03:45:21.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:21.257+0000 7fba19b4a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fba140ccc40 0x7fba141f8e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:21.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:21.257+0000 7fba19b4a640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fba140cd440 0x7fba141f9390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:21.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:21.257+0000 7fba19b4a640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fba140d0bb0 0x7fba141f98d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:21.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:21.257+0000 7fba19b4a640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fba14070810 con 0x7fba140ccc40 2024-09-21T03:45:21.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:21.257+0000 7fba19b4a640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fba140b6e00 con 0x7fba140cd440 2024-09-21T03:45:21.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:21.257+0000 7fba19b4a640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fba14070990 con 0x7fba140d0bb0 2024-09-21T03:45:21.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:21.257+0000 7fba137fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fba140ccc40 0x7fba141f8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:21.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:21.257+0000 7fba137fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fba140ccc40 0x7fba141f8e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46826/0 (socket says 172.21.15.38:46826) 2024-09-21T03:45:21.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:21.257+0000 7fba137fe640 1 -- 172.21.15.38:0/1385142387 learned_addr learned my addr 172.21.15.38:0/1385142387 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:45:21.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:21.257+0000 7fba13fff640 1 --2- 172.21.15.38:0/1385142387 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fba140d0bb0 0x7fba141f98d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:21.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:21.258+0000 7fba12ffd640 1 --2- 172.21.15.38:0/1385142387 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fba140cd440 0x7fba141f9390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:21.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:21.258+0000 7fba137fe640 1 -- 172.21.15.38:0/1385142387 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fba140d0bb0 msgr2=0x7fba141f98d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:21.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:21.258+0000 7fba137fe640 1 --2- 172.21.15.38:0/1385142387 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fba140d0bb0 0x7fba141f98d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:21.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:21.258+0000 7fba137fe640 1 -- 172.21.15.38:0/1385142387 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fba140cd440 msgr2=0x7fba141f9390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:21.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:21.258+0000 7fba137fe640 1 --2- 172.21.15.38:0/1385142387 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fba140cd440 0x7fba141f9390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:21.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:21.258+0000 7fba137fe640 1 -- 172.21.15.38:0/1385142387 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba1415fbb0 con 0x7fba140ccc40 2024-09-21T03:45:21.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:21.258+0000 7fba13fff640 1 --2- 172.21.15.38:0/1385142387 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fba140d0bb0 0x7fba141f98d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:45:21.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:21.258+0000 7fba12ffd640 1 --2- 172.21.15.38:0/1385142387 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fba140cd440 0x7fba141f9390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:45:21.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:21.258+0000 7fba137fe640 1 --2- 172.21.15.38:0/1385142387 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fba140ccc40 0x7fba141f8e50 secure :-1 s=READY pgs=2482 cs=0 l=1 rev1=1 crypto rx=0x7fba04009950 tx=0x7fba04033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:21.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:21.258+0000 7fba0bfff640 1 -- 172.21.15.38:0/1385142387 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba04031d60 con 0x7fba140ccc40 2024-09-21T03:45:21.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:21.258+0000 7fba0bfff640 1 -- 172.21.15.38:0/1385142387 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fba04033e10 con 0x7fba140ccc40 2024-09-21T03:45:21.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:21.258+0000 7fba19b4a640 1 -- 172.21.15.38:0/1385142387 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba1415fe50 con 0x7fba140ccc40 2024-09-21T03:45:21.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:21.258+0000 7fba0bfff640 1 -- 172.21.15.38:0/1385142387 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba04005260 con 0x7fba140ccc40 2024-09-21T03:45:21.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:21.258+0000 7fba19b4a640 1 -- 172.21.15.38:0/1385142387 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba1406f670 con 0x7fba140ccc40 2024-09-21T03:45:21.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:21.259+0000 7fba19b4a640 1 -- 172.21.15.38:0/1385142387 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba14070ed0 con 0x7fba140ccc40 2024-09-21T03:45:21.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:21.260+0000 7fba0bfff640 1 -- 172.21.15.38:0/1385142387 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fba04005400 con 0x7fba140ccc40 2024-09-21T03:45:21.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:21.260+0000 7fba0bfff640 1 --2- 172.21.15.38:0/1385142387 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb9f4078bc0 0x7fb9f407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:21.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:21.261+0000 7fba0bfff640 1 -- 172.21.15.38:0/1385142387 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 6303+0+0 (secure 0 0 0) 0x7fba040bc170 con 0x7fba140ccc40 2024-09-21T03:45:21.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:21.261+0000 7fba12ffd640 1 --2- 172.21.15.38:0/1385142387 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb9f4078bc0 0x7fb9f407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:21.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:21.261+0000 7fba12ffd640 1 --2- 172.21.15.38:0/1385142387 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb9f4078bc0 0x7fb9f407b0d0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7fba1415f9b0 tx=0x7fb9fc005f90 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:21.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:21.265+0000 7fba0bfff640 1 -- 172.21.15.38:0/1385142387 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fba0403f030 con 0x7fba140ccc40 2024-09-21T03:45:21.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:21.465+0000 7fba19b4a640 1 -- 172.21.15.38:0/1385142387 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fba14160020 con 0x7fba140ccc40 2024-09-21T03:45:21.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:21.465+0000 7fba0bfff640 1 -- 172.21.15.38:0/1385142387 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v149) ==== 80+0+122 (secure 0 0 0) 0x7fba04084360 con 0x7fba140ccc40 2024-09-21T03:45:21.468 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6833/3866292839 2024-09-22T03:45:11.068318+0000 2024-09-21T03:45:21.468 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6832/3866292839 2024-09-22T03:45:11.068318+0000 2024-09-21T03:45:21.469 INFO:teuthology.orchestra.run.smithi038.stderr:listed 2 entries 2024-09-21T03:45:21.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:21.467+0000 7fba19b4a640 1 -- 172.21.15.38:0/1385142387 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb9f4078bc0 msgr2=0x7fb9f407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:21.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:21.467+0000 7fba19b4a640 1 --2- 172.21.15.38:0/1385142387 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb9f4078bc0 0x7fb9f407b0d0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7fba1415f9b0 tx=0x7fb9fc005f90 comp rx=0 tx=0).stop 2024-09-21T03:45:21.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:21.468+0000 7fba19b4a640 1 -- 172.21.15.38:0/1385142387 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fba140ccc40 msgr2=0x7fba141f8e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:21.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:21.468+0000 7fba19b4a640 1 --2- 172.21.15.38:0/1385142387 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fba140ccc40 0x7fba141f8e50 secure :-1 s=READY pgs=2482 cs=0 l=1 rev1=1 crypto rx=0x7fba04009950 tx=0x7fba04033c20 comp rx=0 tx=0).stop 2024-09-21T03:45:21.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:21.468+0000 7fba19b4a640 1 -- 172.21.15.38:0/1385142387 shutdown_connections 2024-09-21T03:45:21.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:21.468+0000 7fba19b4a640 1 --2- 172.21.15.38:0/1385142387 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb9f4078bc0 0x7fb9f407b0d0 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:21.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:21.468+0000 7fba19b4a640 1 --2- 172.21.15.38:0/1385142387 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fba140d0bb0 0x7fba141f98d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:21.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:21.468+0000 7fba19b4a640 1 --2- 172.21.15.38:0/1385142387 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fba140cd440 0x7fba141f9390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:21.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:21.468+0000 7fba19b4a640 1 --2- 172.21.15.38:0/1385142387 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fba140ccc40 0x7fba141f8e50 unknown :-1 s=CLOSED pgs=2482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:21.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:21.468+0000 7fba19b4a640 1 -- 172.21.15.38:0/1385142387 >> 172.21.15.38:0/1385142387 conn(0x7fba140b46e0 msgr2=0x7fba14070100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:21.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:21.468+0000 7fba19b4a640 1 -- 172.21.15.38:0/1385142387 shutdown_connections 2024-09-21T03:45:21.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:21.468+0000 7fba19b4a640 1 -- 172.21.15.38:0/1385142387 wait complete. 2024-09-21T03:45:21.482 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-21T03:45:21.482 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:45:21.482 DEBUG:teuthology.orchestra.run.smithi038:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-21T03:45:21.511 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi038.front.sepia.ceph.com 2024-09-21T03:45:21.511 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:45:21.511 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T03:45:21.567 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T03:45:21.567 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-09-21T03:45:21.568 DEBUG:teuthology.orchestra.run.smithi195:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:45:21.586 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056014,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497444k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=8978 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248720k,nr_inodes=812180,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497444k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T03:45:21.587 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T03:45:21.587 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-21T03:45:21.587 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:45:21.624 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056795,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2354 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T03:45:21.624 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T03:45:21.624 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-09-21T03:45:21.624 DEBUG:teuthology.orchestra.run.smithi195:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:45:21.645 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056014,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497444k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=8978 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248720k,nr_inodes=812180,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497444k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T03:45:21.645 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:45:21.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:21.777+0000 7f6c4a63f640 1 -- 172.21.15.38:0/3066112830 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6c4406e330 msgr2=0x7f6c4406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:21.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:21.777+0000 7f6c4a63f640 1 --2- 172.21.15.38:0/3066112830 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6c4406e330 0x7f6c4406ede0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c005fd0 tx=0x7f6c3c02a8b0 comp rx=0 tx=0).stop 2024-09-21T03:45:21.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:21.777+0000 7f6c4a63f640 1 -- 172.21.15.38:0/3066112830 shutdown_connections 2024-09-21T03:45:21.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:21.777+0000 7f6c4a63f640 1 --2- 172.21.15.38:0/3066112830 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6c4406e330 0x7f6c4406ede0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:21.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:21.777+0000 7f6c4a63f640 1 --2- 172.21.15.38:0/3066112830 >> v2:172.21.15.195:3300/0 conn(0x7f6c44073dd0 0x7f6c4406ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:21.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:21.777+0000 7f6c4a63f640 1 -- 172.21.15.38:0/3066112830 >> 172.21.15.38:0/3066112830 conn(0x7f6c440cca40 msgr2=0x7f6c440cce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:21.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:21.777+0000 7f6c4a63f640 1 -- 172.21.15.38:0/3066112830 shutdown_connections 2024-09-21T03:45:21.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:21.778+0000 7f6c4a63f640 1 -- 172.21.15.38:0/3066112830 wait complete. 2024-09-21T03:45:21.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:21.779+0000 7f6c4a63f640 1 Processor -- start 2024-09-21T03:45:21.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:21.779+0000 7f6c4a63f640 1 -- start start 2024-09-21T03:45:21.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:21.781+0000 7f6c4a63f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6c4406e330 0x7f6c441f8810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:21.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:21.781+0000 7f6c4a63f640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6c44073dd0 0x7f6c440ce2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:21.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:21.781+0000 7f6c4a63f640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6c44074cb0 0x7f6c44151bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:21.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:21.781+0000 7f6c4a63f640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f6c4406f4a0 con 0x7f6c4406e330 2024-09-21T03:45:21.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:21.781+0000 7f6c4a63f640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f6c4406f320 con 0x7f6c44074cb0 2024-09-21T03:45:21.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:21.781+0000 7f6c4a63f640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f6c4406f620 con 0x7f6c44073dd0 2024-09-21T03:45:21.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:21.781+0000 7f6c43fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6c4406e330 0x7f6c441f8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:21.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:21.781+0000 7f6c43fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6c4406e330 0x7f6c441f8810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46828/0 (socket says 172.21.15.38:46828) 2024-09-21T03:45:21.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:21.781+0000 7f6c43fff640 1 -- 172.21.15.38:0/3144764986 learned_addr learned my addr 172.21.15.38:0/3144764986 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:45:21.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:21.781+0000 7f6c48bdc640 1 --2- 172.21.15.38:0/3144764986 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6c44074cb0 0x7f6c44151bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:21.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:21.781+0000 7f6c437fe640 1 --2- 172.21.15.38:0/3144764986 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6c44073dd0 0x7f6c440ce2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:21.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:21.781+0000 7f6c43fff640 1 -- 172.21.15.38:0/3144764986 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6c44073dd0 msgr2=0x7f6c440ce2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:21.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:21.781+0000 7f6c43fff640 1 --2- 172.21.15.38:0/3144764986 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6c44073dd0 0x7f6c440ce2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:21.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:21.781+0000 7f6c43fff640 1 -- 172.21.15.38:0/3144764986 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6c44074cb0 msgr2=0x7f6c44151bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:21.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:21.781+0000 7f6c43fff640 1 --2- 172.21.15.38:0/3144764986 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6c44074cb0 0x7f6c44151bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:21.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:21.781+0000 7f6c43fff640 1 -- 172.21.15.38:0/3144764986 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c440a06f0 con 0x7f6c4406e330 2024-09-21T03:45:21.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:21.781+0000 7f6c43fff640 1 --2- 172.21.15.38:0/3144764986 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6c4406e330 0x7f6c441f8810 secure :-1 s=READY pgs=2483 cs=0 l=1 rev1=1 crypto rx=0x7f6c3400c3a0 tx=0x7f6c3400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:21.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:21.782+0000 7f6c40ff9640 1 -- 172.21.15.38:0/3144764986 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c34011790 con 0x7f6c4406e330 2024-09-21T03:45:21.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:21.782+0000 7f6c40ff9640 1 -- 172.21.15.38:0/3144764986 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6c34017070 con 0x7f6c4406e330 2024-09-21T03:45:21.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:21.782+0000 7f6c4a63f640 1 -- 172.21.15.38:0/3144764986 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c441543e0 con 0x7f6c4406e330 2024-09-21T03:45:21.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:21.782+0000 7f6c40ff9640 1 -- 172.21.15.38:0/3144764986 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c34011c80 con 0x7f6c4406e330 2024-09-21T03:45:21.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:21.782+0000 7f6c4a63f640 1 -- 172.21.15.38:0/3144764986 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c440ce810 con 0x7f6c4406e330 2024-09-21T03:45:21.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:21.783+0000 7f6c4a63f640 1 -- 172.21.15.38:0/3144764986 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c4406ede0 con 0x7f6c4406e330 2024-09-21T03:45:21.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:21.784+0000 7f6c40ff9640 1 -- 172.21.15.38:0/3144764986 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f6c3401c460 con 0x7f6c4406e330 2024-09-21T03:45:21.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:21.784+0000 7f6c40ff9640 1 --2- 172.21.15.38:0/3144764986 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6c30078b90 0x7f6c3007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:21.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:21.784+0000 7f6c40ff9640 1 -- 172.21.15.38:0/3144764986 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 6303+0+0 (secure 0 0 0) 0x7f6c3409b4d0 con 0x7f6c4406e330 2024-09-21T03:45:21.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:21.785+0000 7f6c437fe640 1 --2- 172.21.15.38:0/3144764986 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6c30078b90 0x7f6c3007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:21.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:21.785+0000 7f6c437fe640 1 --2- 172.21.15.38:0/3144764986 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6c30078b90 0x7f6c3007b0a0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f6c441594c0 tx=0x7f6c38005f90 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:21.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:21.788+0000 7f6c40ff9640 1 -- 172.21.15.38:0/3144764986 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f6c34063270 con 0x7f6c4406e330 2024-09-21T03:45:22.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:22.014+0000 7f6c4a63f640 1 -- 172.21.15.38:0/3144764986 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6c441545b0 con 0x7f6c4406e330 2024-09-21T03:45:22.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:22.014+0000 7f6c40ff9640 1 -- 172.21.15.38:0/3144764986 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 349 v349) ==== 95+0+5241 (secure 0 0 0) 0x7f6c34067ca0 con 0x7f6c4406e330 2024-09-21T03:45:22.018 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:45:22.018 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":349,"btime":"2024-09-21T03:45:15:115320+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8399,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/3606957623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":3606957623},{"type":"v1","addr":"172.21.15.38:6835","nonce":3606957623}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":338},{"gid":8405,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/456651486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":456651486},{"type":"v1","addr":"172.21.15.195:6837","nonce":456651486}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":338},{"gid":9604,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/2649311840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":2649311840},{"type":"v1","addr":"172.21.15.195:6835","nonce":2649311840}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":338}],"filesystems":[{"mdsmap":{"epoch":349,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:44:56.710446+0000","modified":"2024-09-21T03:45:15.115308+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":147,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10935},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10935":{"gid":10935,"name":"c","rank":0,"incarnation":346,"state":"up:active","state_seq":4,"addr":"172.21.15.38:6833/3839260519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":3839260519},{"type":"v1","addr":"172.21.15.38:6833","nonce":3839260519}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10935,"qdb_cluster":[10935]},"id":7}]} 2024-09-21T03:45:22.018 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 349 2024-09-21T03:45:22.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:22.017+0000 7f6c4a63f640 1 -- 172.21.15.38:0/3144764986 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6c30078b90 msgr2=0x7f6c3007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:22.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:22.017+0000 7f6c4a63f640 1 --2- 172.21.15.38:0/3144764986 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6c30078b90 0x7f6c3007b0a0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f6c441594c0 tx=0x7f6c38005f90 comp rx=0 tx=0).stop 2024-09-21T03:45:22.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:22.017+0000 7f6c4a63f640 1 -- 172.21.15.38:0/3144764986 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6c4406e330 msgr2=0x7f6c441f8810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:22.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:22.017+0000 7f6c4a63f640 1 --2- 172.21.15.38:0/3144764986 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6c4406e330 0x7f6c441f8810 secure :-1 s=READY pgs=2483 cs=0 l=1 rev1=1 crypto rx=0x7f6c3400c3a0 tx=0x7f6c3400c860 comp rx=0 tx=0).stop 2024-09-21T03:45:22.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:22.017+0000 7f6c4a63f640 1 -- 172.21.15.38:0/3144764986 shutdown_connections 2024-09-21T03:45:22.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:22.017+0000 7f6c4a63f640 1 --2- 172.21.15.38:0/3144764986 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6c44074cb0 0x7f6c44151bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:22.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:22.017+0000 7f6c4a63f640 1 --2- 172.21.15.38:0/3144764986 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f6c30078b90 0x7f6c3007b0a0 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:22.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:22.017+0000 7f6c4a63f640 1 --2- 172.21.15.38:0/3144764986 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6c44073dd0 0x7f6c440ce2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:22.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:22.017+0000 7f6c4a63f640 1 --2- 172.21.15.38:0/3144764986 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6c4406e330 0x7f6c441f8810 unknown :-1 s=CLOSED pgs=2483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:22.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:22.017+0000 7f6c4a63f640 1 -- 172.21.15.38:0/3144764986 >> 172.21.15.38:0/3144764986 conn(0x7f6c440cca40 msgr2=0x7f6c44070090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:22.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:22.017+0000 7f6c4a63f640 1 -- 172.21.15.38:0/3144764986 shutdown_connections 2024-09-21T03:45:22.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:22.017+0000 7f6c4a63f640 1 -- 172.21.15.38:0/3144764986 wait complete. 2024-09-21T03:45:22.031 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:45:22.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:22.145+0000 7fa806b52640 1 -- 172.21.15.38:0/970191344 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa8000b2e40 msgr2=0x7fa8000b3250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:22.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:22.145+0000 7fa806b52640 1 --2- 172.21.15.38:0/970191344 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa8000b2e40 0x7fa8000b3250 secure :-1 s=READY pgs=2484 cs=0 l=1 rev1=1 crypto rx=0x7fa8000d5250 tx=0x7fa7f802aa30 comp rx=0 tx=0).stop 2024-09-21T03:45:22.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:22.145+0000 7fa806b52640 1 -- 172.21.15.38:0/970191344 shutdown_connections 2024-09-21T03:45:22.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:22.145+0000 7fa806b52640 1 --2- 172.21.15.38:0/970191344 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa8000b2e40 0x7fa8000b3250 unknown :-1 s=CLOSED pgs=2484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:22.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:22.145+0000 7fa806b52640 1 --2- 172.21.15.38:0/970191344 >> v2:172.21.15.195:3300/0 conn(0x7fa800162280 0x7fa800162690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:22.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:22.145+0000 7fa806b52640 1 --2- 172.21.15.38:0/970191344 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa800162f00 0x7fa8000d24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:22.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:22.145+0000 7fa806b52640 1 -- 172.21.15.38:0/970191344 >> 172.21.15.38:0/970191344 conn(0x7fa8000b6b20 msgr2=0x7fa8000d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:22.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:22.145+0000 7fa806b52640 1 -- 172.21.15.38:0/970191344 shutdown_connections 2024-09-21T03:45:22.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:22.145+0000 7fa806b52640 1 -- 172.21.15.38:0/970191344 wait complete. 2024-09-21T03:45:22.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:22.147+0000 7fa806b52640 1 Processor -- start 2024-09-21T03:45:22.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:22.147+0000 7fa806b52640 1 -- start start 2024-09-21T03:45:22.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:22.148+0000 7fa806b52640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa8000b2e40 0x7fa8000b8880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:22.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:22.148+0000 7fa806b52640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa800162280 0x7fa800159920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:22.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:22.148+0000 7fa806b52640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa80015e4b0 0x7fa80015e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:22.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:22.148+0000 7fa806b52640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa8000b3dd0 con 0x7fa8000b2e40 2024-09-21T03:45:22.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:22.148+0000 7fa806b52640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fa8000b39a0 con 0x7fa80015e4b0 2024-09-21T03:45:22.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:22.148+0000 7fa806b52640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fa8000b3f50 con 0x7fa800162280 2024-09-21T03:45:22.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:22.148+0000 7fa8048ee640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa8000b2e40 0x7fa8000b8880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:22.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:22.148+0000 7fa8048ee640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa8000b2e40 0x7fa8000b8880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46844/0 (socket says 172.21.15.38:46844) 2024-09-21T03:45:22.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:22.148+0000 7fa8048ee640 1 -- 172.21.15.38:0/3307812842 learned_addr learned my addr 172.21.15.38:0/3307812842 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:45:22.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:22.149+0000 7fa8050ef640 1 --2- 172.21.15.38:0/3307812842 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa80015e4b0 0x7fa80015e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:22.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:22.149+0000 7fa7fffff640 1 --2- 172.21.15.38:0/3307812842 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa800162280 0x7fa800159920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:22.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:22.149+0000 7fa8048ee640 1 -- 172.21.15.38:0/3307812842 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa800162280 msgr2=0x7fa800159920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:22.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:22.149+0000 7fa8048ee640 1 --2- 172.21.15.38:0/3307812842 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa800162280 0x7fa800159920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:22.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:22.149+0000 7fa8048ee640 1 -- 172.21.15.38:0/3307812842 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa80015e4b0 msgr2=0x7fa80015e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:22.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:22.149+0000 7fa8048ee640 1 --2- 172.21.15.38:0/3307812842 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa80015e4b0 0x7fa80015e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:22.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:22.149+0000 7fa8048ee640 1 -- 172.21.15.38:0/3307812842 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa80015ee00 con 0x7fa8000b2e40 2024-09-21T03:45:22.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:22.149+0000 7fa8048ee640 1 --2- 172.21.15.38:0/3307812842 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa8000b2e40 0x7fa8000b8880 secure :-1 s=READY pgs=2485 cs=0 l=1 rev1=1 crypto rx=0x7fa7f000b410 tx=0x7fa7f000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:22.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:22.149+0000 7fa7fd7fa640 1 -- 172.21.15.38:0/3307812842 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7f0015020 con 0x7fa8000b2e40 2024-09-21T03:45:22.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:22.149+0000 7fa7fd7fa640 1 -- 172.21.15.38:0/3307812842 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa7f0011940 con 0x7fa8000b2e40 2024-09-21T03:45:22.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:22.149+0000 7fa806b52640 1 -- 172.21.15.38:0/3307812842 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8000b8dc0 con 0x7fa8000b2e40 2024-09-21T03:45:22.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:22.149+0000 7fa7fd7fa640 1 -- 172.21.15.38:0/3307812842 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7f0011d90 con 0x7fa8000b2e40 2024-09-21T03:45:22.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:22.149+0000 7fa806b52640 1 -- 172.21.15.38:0/3307812842 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8001f8b30 con 0x7fa8000b2e40 2024-09-21T03:45:22.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:22.150+0000 7fa806b52640 1 -- 172.21.15.38:0/3307812842 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa800162f00 con 0x7fa8000b2e40 2024-09-21T03:45:22.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:22.151+0000 7fa7fd7fa640 1 -- 172.21.15.38:0/3307812842 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa7f0018020 con 0x7fa8000b2e40 2024-09-21T03:45:22.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:22.152+0000 7fa7fd7fa640 1 --2- 172.21.15.38:0/3307812842 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa7ec078b90 0x7fa7ec07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:22.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:22.152+0000 7fa7fd7fa640 1 -- 172.21.15.38:0/3307812842 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 6303+0+0 (secure 0 0 0) 0x7fa7f009b290 con 0x7fa8000b2e40 2024-09-21T03:45:22.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:22.152+0000 7fa7fffff640 1 --2- 172.21.15.38:0/3307812842 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa7ec078b90 0x7fa7ec07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:22.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:22.153+0000 7fa7fffff640 1 --2- 172.21.15.38:0/3307812842 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa7ec078b90 0x7fa7ec07b0a0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fa7f400a490 tx=0x7fa7f400a420 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:22.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:22.155+0000 7fa7fd7fa640 1 -- 172.21.15.38:0/3307812842 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fa7f001f020 con 0x7fa8000b2e40 2024-09-21T03:45:22.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:22.382+0000 7fa806b52640 1 -- 172.21.15.38:0/3307812842 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa800068e80 con 0x7fa8000b2e40 2024-09-21T03:45:22.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:22.383+0000 7fa7fd7fa640 1 -- 172.21.15.38:0/3307812842 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 349 v349) ==== 95+0+5241 (secure 0 0 0) 0x7fa80015ee00 con 0x7fa8000b2e40 2024-09-21T03:45:22.386 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:45:22.386 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":349,"btime":"2024-09-21T03:45:15:115320+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8399,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/3606957623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":3606957623},{"type":"v1","addr":"172.21.15.38:6835","nonce":3606957623}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":338},{"gid":8405,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/456651486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":456651486},{"type":"v1","addr":"172.21.15.195:6837","nonce":456651486}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":338},{"gid":9604,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/2649311840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":2649311840},{"type":"v1","addr":"172.21.15.195:6835","nonce":2649311840}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":338}],"filesystems":[{"mdsmap":{"epoch":349,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:44:56.710446+0000","modified":"2024-09-21T03:45:15.115308+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":147,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10935},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10935":{"gid":10935,"name":"c","rank":0,"incarnation":346,"state":"up:active","state_seq":4,"addr":"172.21.15.38:6833/3839260519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":3839260519},{"type":"v1","addr":"172.21.15.38:6833","nonce":3839260519}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10935,"qdb_cluster":[10935]},"id":7}]} 2024-09-21T03:45:22.387 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 349 2024-09-21T03:45:22.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:22.385+0000 7fa806b52640 1 -- 172.21.15.38:0/3307812842 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa7ec078b90 msgr2=0x7fa7ec07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:22.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:22.385+0000 7fa806b52640 1 --2- 172.21.15.38:0/3307812842 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa7ec078b90 0x7fa7ec07b0a0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fa7f400a490 tx=0x7fa7f400a420 comp rx=0 tx=0).stop 2024-09-21T03:45:22.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:22.385+0000 7fa806b52640 1 -- 172.21.15.38:0/3307812842 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa8000b2e40 msgr2=0x7fa8000b8880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:22.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:22.385+0000 7fa806b52640 1 --2- 172.21.15.38:0/3307812842 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa8000b2e40 0x7fa8000b8880 secure :-1 s=READY pgs=2485 cs=0 l=1 rev1=1 crypto rx=0x7fa7f000b410 tx=0x7fa7f000b8d0 comp rx=0 tx=0).stop 2024-09-21T03:45:22.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:22.385+0000 7fa806b52640 1 -- 172.21.15.38:0/3307812842 shutdown_connections 2024-09-21T03:45:22.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:22.385+0000 7fa806b52640 1 --2- 172.21.15.38:0/3307812842 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa80015e4b0 0x7fa80015e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:22.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:22.385+0000 7fa806b52640 1 --2- 172.21.15.38:0/3307812842 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fa7ec078b90 0x7fa7ec07b0a0 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:22.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:22.385+0000 7fa806b52640 1 --2- 172.21.15.38:0/3307812842 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa800162280 0x7fa800159920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:22.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:22.385+0000 7fa806b52640 1 --2- 172.21.15.38:0/3307812842 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa8000b2e40 0x7fa8000b8880 unknown :-1 s=CLOSED pgs=2485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:22.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:22.385+0000 7fa806b52640 1 -- 172.21.15.38:0/3307812842 >> 172.21.15.38:0/3307812842 conn(0x7fa8000b6b20 msgr2=0x7fa80015c990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:22.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:22.386+0000 7fa806b52640 1 -- 172.21.15.38:0/3307812842 shutdown_connections 2024-09-21T03:45:22.390 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:22.386+0000 7fa806b52640 1 -- 172.21.15.38:0/3307812842 wait complete. 2024-09-21T03:45:22.400 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T03:45:22.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:22.513+0000 7feec1007640 1 -- 172.21.15.38:0/1364004856 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feebc162f00 msgr2=0x7feebc0d24d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:22.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:22.513+0000 7feec1007640 1 --2- 172.21.15.38:0/1364004856 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feebc162f00 0x7feebc0d24d0 secure :-1 s=READY pgs=2487 cs=0 l=1 rev1=1 crypto rx=0x7feeac009830 tx=0x7feeac02a510 comp rx=0 tx=0).stop 2024-09-21T03:45:22.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:22.513+0000 7feec1007640 1 -- 172.21.15.38:0/1364004856 shutdown_connections 2024-09-21T03:45:22.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:22.513+0000 7feec1007640 1 --2- 172.21.15.38:0/1364004856 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7feebc162280 0x7feebc162690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:22.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:22.513+0000 7feec1007640 1 --2- 172.21.15.38:0/1364004856 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feebc162f00 0x7feebc0d24d0 unknown :-1 s=CLOSED pgs=2487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:22.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:22.513+0000 7feec1007640 1 -- 172.21.15.38:0/1364004856 >> 172.21.15.38:0/1364004856 conn(0x7feebc0b6b20 msgr2=0x7feebc0d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:22.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:22.513+0000 7feec1007640 1 -- 172.21.15.38:0/1364004856 shutdown_connections 2024-09-21T03:45:22.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:22.513+0000 7feec1007640 1 -- 172.21.15.38:0/1364004856 wait complete. 2024-09-21T03:45:22.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:22.515+0000 7feec1007640 1 Processor -- start 2024-09-21T03:45:22.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:22.515+0000 7feec1007640 1 -- start start 2024-09-21T03:45:22.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:22.517+0000 7feec1007640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7feebc0b2e40 0x7feebc0cefb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:22.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:22.517+0000 7feec1007640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feebc162280 0x7feebc0b26f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:22.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:22.517+0000 7feec1007640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feebc1599e0 0x7feebc0cb0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:22.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:22.517+0000 7feec1007640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7feebc15e630 con 0x7feebc162280 2024-09-21T03:45:22.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:22.517+0000 7feec1007640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7feebc15e4b0 con 0x7feebc1599e0 2024-09-21T03:45:22.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:22.517+0000 7feec1007640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7feebc15e7b0 con 0x7feebc0b2e40 2024-09-21T03:45:22.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:22.517+0000 7feeb259c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feebc162280 0x7feebc0b26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:22.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:22.517+0000 7feeb259c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feebc162280 0x7feebc0b26f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:56378/0 (socket says 172.21.15.38:56378) 2024-09-21T03:45:22.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:22.517+0000 7feeb259c640 1 -- 172.21.15.38:0/759426246 learned_addr learned my addr 172.21.15.38:0/759426246 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:45:22.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:22.517+0000 7feebad9d640 1 --2- 172.21.15.38:0/759426246 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7feebc0b2e40 0x7feebc0cefb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:22.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:22.517+0000 7feebb59e640 1 --2- 172.21.15.38:0/759426246 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feebc1599e0 0x7feebc0cb0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:22.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:22.517+0000 7feeb259c640 1 -- 172.21.15.38:0/759426246 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7feebc0b2e40 msgr2=0x7feebc0cefb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:22.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:22.517+0000 7feeb259c640 1 --2- 172.21.15.38:0/759426246 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7feebc0b2e40 0x7feebc0cefb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:22.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:22.517+0000 7feeb259c640 1 -- 172.21.15.38:0/759426246 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feebc1599e0 msgr2=0x7feebc0cb0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:22.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:22.517+0000 7feeb259c640 1 --2- 172.21.15.38:0/759426246 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feebc1599e0 0x7feebc0cb0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:22.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:22.517+0000 7feeb259c640 1 -- 172.21.15.38:0/759426246 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feebc0b2c30 con 0x7feebc162280 2024-09-21T03:45:22.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:22.517+0000 7feeb259c640 1 --2- 172.21.15.38:0/759426246 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feebc162280 0x7feebc0b26f0 secure :-1 s=READY pgs=2488 cs=0 l=1 rev1=1 crypto rx=0x7feea400b410 tx=0x7feea400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:22.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:22.518+0000 7feeb3fff640 1 -- 172.21.15.38:0/759426246 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feea4015020 con 0x7feebc162280 2024-09-21T03:45:22.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:22.518+0000 7feeb3fff640 1 -- 172.21.15.38:0/759426246 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7feea4011920 con 0x7feebc162280 2024-09-21T03:45:22.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:22.518+0000 7feec1007640 1 -- 172.21.15.38:0/759426246 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feebc157b20 con 0x7feebc162280 2024-09-21T03:45:22.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:22.518+0000 7feeb3fff640 1 -- 172.21.15.38:0/759426246 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feea4011d70 con 0x7feebc162280 2024-09-21T03:45:22.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:22.518+0000 7feec1007640 1 -- 172.21.15.38:0/759426246 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feebc15b1d0 con 0x7feebc162280 2024-09-21T03:45:22.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:22.519+0000 7feec1007640 1 -- 172.21.15.38:0/759426246 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feebc162f00 con 0x7feebc162280 2024-09-21T03:45:22.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:22.520+0000 7feeb3fff640 1 -- 172.21.15.38:0/759426246 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7feea4018020 con 0x7feebc162280 2024-09-21T03:45:22.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:22.520+0000 7feeb3fff640 1 --2- 172.21.15.38:0/759426246 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fee9c078b90 0x7fee9c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:22.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:22.520+0000 7feeb3fff640 1 -- 172.21.15.38:0/759426246 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 6303+0+0 (secure 0 0 0) 0x7feea409b350 con 0x7feebc162280 2024-09-21T03:45:22.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:22.521+0000 7feebad9d640 1 --2- 172.21.15.38:0/759426246 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fee9c078b90 0x7fee9c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:22.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:22.521+0000 7feebad9d640 1 --2- 172.21.15.38:0/759426246 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fee9c078b90 0x7fee9c07b0a0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7feeac034000 tx=0x7feeac004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:22.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:22.524+0000 7feeb3fff640 1 -- 172.21.15.38:0/759426246 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7feea401f020 con 0x7feebc162280 2024-09-21T03:45:22.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:22.712+0000 7feec1007640 1 -- 172.21.15.38:0/759426246 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7feebc14b6a0 con 0x7feebc162280 2024-09-21T03:45:22.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:22.713+0000 7feeb3fff640 1 -- 172.21.15.38:0/759426246 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v149) ==== 74+0+17362 (secure 0 0 0) 0x7feebc157b20 con 0x7feebc162280 2024-09-21T03:45:22.717 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:45:22.717 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":149,"fsid":"9d50d66f-656b-4fe2-8e3b-fe53b79c4d43","created":"2024-09-21T03:33:10.677865+0000","modified":"2024-09-21T03:45:18.148053+0000","last_up_change":"2024-09-21T03:33:24.023421+0000","last_in_change":"2024-09-21T03:33:14.296193+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T03:33:22.992573+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-09-21T03:44:54.386654+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"145","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-09-21T03:44:55.667528+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"146","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dbab183f-7e63-48b8-960a-ae55fd5bf565","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6801","nonce":2381639433}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6803","nonce":2381639433}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6807","nonce":2381639433}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6805","nonce":2381639433}]},"public_addr":"172.21.15.38:6801/2381639433","cluster_addr":"172.21.15.38:6803/2381639433","heartbeat_back_addr":"172.21.15.38:6807/2381639433","heartbeat_front_addr":"172.21.15.38:6805/2381639433","state":["exists","up"]},{"osd":1,"uuid":"fa1c0e8a-ebc1-4359-8146-bd5342abbc16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6817","nonce":1491818732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6819","nonce":1491818732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6823","nonce":1491818732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6821","nonce":1491818732}]},"public_addr":"172.21.15.38:6817/1491818732","cluster_addr":"172.21.15.38:6819/1491818732","heartbeat_back_addr":"172.21.15.38:6823/1491818732","heartbeat_front_addr":"172.21.15.38:6821/1491818732","state":["exists","up"]},{"osd":2,"uuid":"2ed3cd2d-385d-4fc6-9869-214552a84803","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6825","nonce":2577858902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6827","nonce":2577858902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6831","nonce":2577858902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6829","nonce":2577858902}]},"public_addr":"172.21.15.38:6825/2577858902","cluster_addr":"172.21.15.38:6827/2577858902","heartbeat_back_addr":"172.21.15.38:6831/2577858902","heartbeat_front_addr":"172.21.15.38:6829/2577858902","state":["exists","up"]},{"osd":3,"uuid":"62d1d95e-919d-4250-be71-1d34cfa70dbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6809","nonce":1286810294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6811","nonce":1286810294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6815","nonce":1286810294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6813","nonce":1286810294}]},"public_addr":"172.21.15.38:6809/1286810294","cluster_addr":"172.21.15.38:6811/1286810294","heartbeat_back_addr":"172.21.15.38:6815/1286810294","heartbeat_front_addr":"172.21.15.38:6813/1286810294","state":["exists","up"]},{"osd":4,"uuid":"7c9f9314-99d7-4d73-8392-37809a9f54b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6803","nonce":3401837571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6805","nonce":3401837571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6809","nonce":3401837571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6807","nonce":3401837571}]},"public_addr":"172.21.15.195:6803/3401837571","cluster_addr":"172.21.15.195:6805/3401837571","heartbeat_back_addr":"172.21.15.195:6809/3401837571","heartbeat_front_addr":"172.21.15.195:6807/3401837571","state":["exists","up"]},{"osd":5,"uuid":"bde4fe9f-8414-4bba-bf4a-43973d1976bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6811","nonce":2080960180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6813","nonce":2080960180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6817","nonce":2080960180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6815","nonce":2080960180}]},"public_addr":"172.21.15.195:6811/2080960180","cluster_addr":"172.21.15.195:6813/2080960180","heartbeat_back_addr":"172.21.15.195:6817/2080960180","heartbeat_front_addr":"172.21.15.195:6815/2080960180","state":["exists","up"]},{"osd":6,"uuid":"6ee6d7f4-1a5f-4a4c-ac16-48fe05f6a002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6827","nonce":4055825675}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6829","nonce":4055825675}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6833","nonce":4055825675}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6831","nonce":4055825675}]},"public_addr":"172.21.15.195:6827/4055825675","cluster_addr":"172.21.15.195:6829/4055825675","heartbeat_back_addr":"172.21.15.195:6833/4055825675","heartbeat_front_addr":"172.21.15.195:6831/4055825675","state":["exists","up"]},{"osd":7,"uuid":"efeb5f51-329a-4484-8290-92e7d3503d05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6819","nonce":1606106342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6821","nonce":1606106342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6825","nonce":1606106342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6823","nonce":1606106342}]},"public_addr":"172.21.15.195:6819/1606106342","cluster_addr":"172.21.15.195:6821/1606106342","heartbeat_back_addr":"172.21.15.195:6825/1606106342","heartbeat_front_addr":"172.21.15.195:6823/1606106342","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.901343+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:21.115003+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.901623+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.835238+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.132199+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:19.817560+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.829690+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:21.087300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.5","mappings":[{"from":4,"to":3}]},{"pgid":"16.1c","mappings":[{"from":4,"to":3}]},{"pgid":"16.3b","mappings":[{"from":4,"to":6}]},{"pgid":"16.3e","mappings":[{"from":2,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.38:6833/3866292839":"2024-09-22T03:45:11.068318+0000","172.21.15.38:6832/3866292839":"2024-09-22T03:45:11.068318+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T03:45:22.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:22.716+0000 7feec1007640 1 -- 172.21.15.38:0/759426246 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fee9c078b90 msgr2=0x7fee9c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:22.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:22.716+0000 7feec1007640 1 --2- 172.21.15.38:0/759426246 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fee9c078b90 0x7fee9c07b0a0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7feeac034000 tx=0x7feeac004730 comp rx=0 tx=0).stop 2024-09-21T03:45:22.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:22.716+0000 7feec1007640 1 -- 172.21.15.38:0/759426246 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feebc162280 msgr2=0x7feebc0b26f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:22.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:22.716+0000 7feec1007640 1 --2- 172.21.15.38:0/759426246 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feebc162280 0x7feebc0b26f0 secure :-1 s=READY pgs=2488 cs=0 l=1 rev1=1 crypto rx=0x7feea400b410 tx=0x7feea400b8d0 comp rx=0 tx=0).stop 2024-09-21T03:45:22.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:22.716+0000 7feec1007640 1 -- 172.21.15.38:0/759426246 shutdown_connections 2024-09-21T03:45:22.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:22.716+0000 7feec1007640 1 --2- 172.21.15.38:0/759426246 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feebc1599e0 0x7feebc0cb0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:22.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:22.716+0000 7feec1007640 1 --2- 172.21.15.38:0/759426246 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feebc162280 0x7feebc0b26f0 unknown :-1 s=CLOSED pgs=2488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:22.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:22.716+0000 7feec1007640 1 --2- 172.21.15.38:0/759426246 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fee9c078b90 0x7fee9c07b0a0 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:22.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:22.716+0000 7feec1007640 1 --2- 172.21.15.38:0/759426246 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7feebc0b2e40 0x7feebc0cefb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:22.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:22.716+0000 7feec1007640 1 -- 172.21.15.38:0/759426246 >> 172.21.15.38:0/759426246 conn(0x7feebc0b6b20 msgr2=0x7feebc15f680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:22.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:22.716+0000 7feec1007640 1 -- 172.21.15.38:0/759426246 shutdown_connections 2024-09-21T03:45:22.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:22.716+0000 7feec1007640 1 -- 172.21.15.38:0/759426246 wait complete. 2024-09-21T03:45:22.730 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-21T03:45:22.730 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:45:22.850 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:22.847+0000 7ff9f25ad640 1 -- 172.21.15.38:0/532179268 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff9d4004800 msgr2=0x7ff9d4004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:22.850 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:22.847+0000 7ff9f25ad640 1 --2- 172.21.15.38:0/532179268 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff9d4004800 0x7ff9d4004c10 secure :-1 s=READY pgs=2490 cs=0 l=1 rev1=1 crypto rx=0x7ff9e0009b50 tx=0x7ff9d4005830 comp rx=0 tx=0).stop 2024-09-21T03:45:22.850 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:22.847+0000 7ff9f0b4a640 1 -- 172.21.15.38:0/532179268 reap_dead start 2024-09-21T03:45:22.850 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:22.847+0000 7ff9f25ad640 1 -- 172.21.15.38:0/532179268 shutdown_connections 2024-09-21T03:45:22.850 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:22.847+0000 7ff9f25ad640 1 -- 172.21.15.38:0/532179268 >> 172.21.15.38:0/532179268 conn(0x7ff9ec0d0a30 msgr2=0x7ff9ec0d0e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:22.850 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:22.847+0000 7ff9f25ad640 1 -- 172.21.15.38:0/532179268 shutdown_connections 2024-09-21T03:45:22.851 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:22.847+0000 7ff9f25ad640 1 -- 172.21.15.38:0/532179268 wait complete. 2024-09-21T03:45:22.851 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:22.849+0000 7ff9f25ad640 1 Processor -- start 2024-09-21T03:45:22.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:22.849+0000 7ff9f25ad640 1 -- start start 2024-09-21T03:45:22.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:22.850+0000 7ff9f25ad640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff9d4004800 0x7ff9ec142dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:22.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:22.850+0000 7ff9f25ad640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff9ec145eb0 0x7ff9ec1f6550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:22.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:22.850+0000 7ff9f25ad640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff9ec1f6a90 0x7ff9ec158550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:22.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:22.850+0000 7ff9f25ad640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ff9ec0cfea0 con 0x7ff9d4004800 2024-09-21T03:45:22.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:22.850+0000 7ff9f25ad640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ff9ec0cfd20 con 0x7ff9ec1f6a90 2024-09-21T03:45:22.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:22.850+0000 7ff9f25ad640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7ff9ec0d0020 con 0x7ff9ec145eb0 2024-09-21T03:45:22.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:22.850+0000 7ff9ebfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff9d4004800 0x7ff9ec142dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:22.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:22.850+0000 7ff9ebfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff9d4004800 0x7ff9ec142dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:56396/0 (socket says 172.21.15.38:56396) 2024-09-21T03:45:22.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:22.850+0000 7ff9ebfff640 1 -- 172.21.15.38:0/3664903572 learned_addr learned my addr 172.21.15.38:0/3664903572 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:45:22.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:22.851+0000 7ff9ebfff640 1 -- 172.21.15.38:0/3664903572 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff9ec145eb0 msgr2=0x7ff9ec1f6550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:22.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:22.851+0000 7ff9ebfff640 1 --2- 172.21.15.38:0/3664903572 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff9ec145eb0 0x7ff9ec1f6550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:22.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:22.851+0000 7ff9f0b4a640 1 --2- 172.21.15.38:0/3664903572 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff9ec1f6a90 0x7ff9ec158550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:22.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:22.851+0000 7ff9ebfff640 1 -- 172.21.15.38:0/3664903572 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff9ec1f6a90 msgr2=0x7ff9ec158550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:22.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:22.851+0000 7ff9ebfff640 1 --2- 172.21.15.38:0/3664903572 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff9ec1f6a90 0x7ff9ec158550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:22.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:22.851+0000 7ff9ebfff640 1 -- 172.21.15.38:0/3664903572 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9ec14f7a0 con 0x7ff9d4004800 2024-09-21T03:45:22.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:22.851+0000 7ff9f0b4a640 1 --2- 172.21.15.38:0/3664903572 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff9ec1f6a90 0x7ff9ec158550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:45:22.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:22.851+0000 7ff9ebfff640 1 --2- 172.21.15.38:0/3664903572 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff9d4004800 0x7ff9ec142dc0 secure :-1 s=READY pgs=2491 cs=0 l=1 rev1=1 crypto rx=0x7ff9dc003f50 tx=0x7ff9dc0016f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:22.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:22.851+0000 7ff9e8ff9640 1 -- 172.21.15.38:0/3664903572 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9dc011640 con 0x7ff9d4004800 2024-09-21T03:45:22.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:22.851+0000 7ff9e8ff9640 1 -- 172.21.15.38:0/3664903572 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff9dc016020 con 0x7ff9d4004800 2024-09-21T03:45:22.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:22.851+0000 7ff9f25ad640 1 -- 172.21.15.38:0/3664903572 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9ec1f6ea0 con 0x7ff9d4004800 2024-09-21T03:45:22.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:22.851+0000 7ff9e8ff9640 1 -- 172.21.15.38:0/3664903572 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9dc011a60 con 0x7ff9d4004800 2024-09-21T03:45:22.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:22.851+0000 7ff9f25ad640 1 -- 172.21.15.38:0/3664903572 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9ec0a9690 con 0x7ff9d4004800 2024-09-21T03:45:22.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:22.852+0000 7ff9f25ad640 1 -- 172.21.15.38:0/3664903572 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9ec06d560 con 0x7ff9d4004800 2024-09-21T03:45:22.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:22.853+0000 7ff9e8ff9640 1 -- 172.21.15.38:0/3664903572 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ff9dc011c00 con 0x7ff9d4004800 2024-09-21T03:45:22.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:22.853+0000 7ff9e8ff9640 1 --2- 172.21.15.38:0/3664903572 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff9d8078b90 0x7ff9d807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:22.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:22.854+0000 7ff9e8ff9640 1 -- 172.21.15.38:0/3664903572 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 6303+0+0 (secure 0 0 0) 0x7ff9dc09bb70 con 0x7ff9d4004800 2024-09-21T03:45:22.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:22.854+0000 7ff9eb7fe640 1 --2- 172.21.15.38:0/3664903572 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff9d8078b90 0x7ff9d807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:22.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:22.854+0000 7ff9eb7fe640 1 --2- 172.21.15.38:0/3664903572 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff9d8078b90 0x7ff9d807b0a0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7ff9e0035000 tx=0x7ff9e0004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:22.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:22.858+0000 7ff9e8ff9640 1 -- 172.21.15.38:0/3664903572 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7ff9dc020020 con 0x7ff9d4004800 2024-09-21T03:45:23.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:23.085+0000 7ff9f25ad640 1 -- 172.21.15.38:0/3664903572 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff9ec069d30 con 0x7ff9d4004800 2024-09-21T03:45:23.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:23.085+0000 7ff9e8ff9640 1 -- 172.21.15.38:0/3664903572 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 349 v349) ==== 95+0+5241 (secure 0 0 0) 0x7ff9dc063df0 con 0x7ff9d4004800 2024-09-21T03:45:23.088 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:45:23.088 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":349,"btime":"2024-09-21T03:45:15:115320+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8399,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/3606957623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":3606957623},{"type":"v1","addr":"172.21.15.38:6835","nonce":3606957623}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":338},{"gid":8405,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/456651486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":456651486},{"type":"v1","addr":"172.21.15.195:6837","nonce":456651486}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":338},{"gid":9604,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/2649311840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":2649311840},{"type":"v1","addr":"172.21.15.195:6835","nonce":2649311840}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":338}],"filesystems":[{"mdsmap":{"epoch":349,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:44:56.710446+0000","modified":"2024-09-21T03:45:15.115308+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":147,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10935},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10935":{"gid":10935,"name":"c","rank":0,"incarnation":346,"state":"up:active","state_seq":4,"addr":"172.21.15.38:6833/3839260519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":3839260519},{"type":"v1","addr":"172.21.15.38:6833","nonce":3839260519}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10935,"qdb_cluster":[10935]},"id":7}]} 2024-09-21T03:45:23.089 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 349 2024-09-21T03:45:23.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:23.088+0000 7ff9f25ad640 1 -- 172.21.15.38:0/3664903572 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff9d8078b90 msgr2=0x7ff9d807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:23.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:23.088+0000 7ff9f25ad640 1 --2- 172.21.15.38:0/3664903572 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff9d8078b90 0x7ff9d807b0a0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7ff9e0035000 tx=0x7ff9e0004730 comp rx=0 tx=0).stop 2024-09-21T03:45:23.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:23.088+0000 7ff9f25ad640 1 -- 172.21.15.38:0/3664903572 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff9d4004800 msgr2=0x7ff9ec142dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:23.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:23.088+0000 7ff9f25ad640 1 --2- 172.21.15.38:0/3664903572 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff9d4004800 0x7ff9ec142dc0 secure :-1 s=READY pgs=2491 cs=0 l=1 rev1=1 crypto rx=0x7ff9dc003f50 tx=0x7ff9dc0016f0 comp rx=0 tx=0).stop 2024-09-21T03:45:23.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:23.088+0000 7ff9f25ad640 1 -- 172.21.15.38:0/3664903572 shutdown_connections 2024-09-21T03:45:23.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:23.088+0000 7ff9f25ad640 1 --2- 172.21.15.38:0/3664903572 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff9ec1f6a90 0x7ff9ec158550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:23.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:23.088+0000 7ff9f25ad640 1 --2- 172.21.15.38:0/3664903572 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff9d8078b90 0x7ff9d807b0a0 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:23.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:23.088+0000 7ff9f25ad640 1 --2- 172.21.15.38:0/3664903572 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff9ec145eb0 0x7ff9ec1f6550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:23.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:23.088+0000 7ff9f25ad640 1 --2- 172.21.15.38:0/3664903572 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff9d4004800 0x7ff9ec142dc0 unknown :-1 s=CLOSED pgs=2491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:23.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:23.088+0000 7ff9f25ad640 1 -- 172.21.15.38:0/3664903572 >> 172.21.15.38:0/3664903572 conn(0x7ff9ec0d0a30 msgr2=0x7ff9ec0b0470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:23.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:23.088+0000 7ff9f25ad640 1 -- 172.21.15.38:0/3664903572 shutdown_connections 2024-09-21T03:45:23.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:23.088+0000 7ff9f25ad640 1 -- 172.21.15.38:0/3664903572 wait complete. 2024-09-21T03:45:23.102 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:45:23.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:23.216+0000 7f78ef6cc640 1 -- 172.21.15.38:0/1569900927 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f78e8162590 msgr2=0x7f78e81629a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:23.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:23.216+0000 7f78ef6cc640 1 --2- 172.21.15.38:0/1569900927 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f78e8162590 0x7f78e81629a0 secure :-1 s=READY pgs=2493 cs=0 l=1 rev1=1 crypto rx=0x7f78d8005fd0 tx=0x7f78d802a670 comp rx=0 tx=0).stop 2024-09-21T03:45:23.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:23.216+0000 7f78ef6cc640 1 -- 172.21.15.38:0/1569900927 shutdown_connections 2024-09-21T03:45:23.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:23.216+0000 7f78ef6cc640 1 --2- 172.21.15.38:0/1569900927 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f78e8162590 0x7f78e81629a0 unknown :-1 s=CLOSED pgs=2493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:23.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:23.216+0000 7f78ef6cc640 1 -- 172.21.15.38:0/1569900927 >> 172.21.15.38:0/1569900927 conn(0x7f78e80d4440 msgr2=0x7f78e80d4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:23.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:23.216+0000 7f78ef6cc640 1 -- 172.21.15.38:0/1569900927 shutdown_connections 2024-09-21T03:45:23.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:23.216+0000 7f78ef6cc640 1 -- 172.21.15.38:0/1569900927 wait complete. 2024-09-21T03:45:23.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:23.218+0000 7f78ef6cc640 1 Processor -- start 2024-09-21T03:45:23.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:23.218+0000 7f78ef6cc640 1 -- start start 2024-09-21T03:45:23.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:23.219+0000 7f78ef6cc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f78e80b2e60 0x7f78e81582b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:23.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:23.219+0000 7f78ef6cc640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f78e80cfe00 0x7f78e80c9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:23.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:23.219+0000 7f78ef6cc640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f78e8162590 0x7f78e80c9fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:23.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:23.219+0000 7f78ef6cc640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f78e80b3b00 con 0x7f78e80b2e60 2024-09-21T03:45:23.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:23.219+0000 7f78ef6cc640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f78e80b3980 con 0x7f78e8162590 2024-09-21T03:45:23.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:23.219+0000 7f78ef6cc640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f78e80b4f70 con 0x7f78e80cfe00 2024-09-21T03:45:23.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:23.219+0000 7f78ed468640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f78e80b2e60 0x7f78e81582b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:23.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:23.219+0000 7f78ed468640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f78e80b2e60 0x7f78e81582b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:56422/0 (socket says 172.21.15.38:56422) 2024-09-21T03:45:23.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:23.219+0000 7f78ed468640 1 -- 172.21.15.38:0/2682004565 learned_addr learned my addr 172.21.15.38:0/2682004565 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:45:23.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:23.220+0000 7f78ecc67640 1 --2- 172.21.15.38:0/2682004565 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f78e80cfe00 0x7f78e80c9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:23.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:23.220+0000 7f78edc69640 1 --2- 172.21.15.38:0/2682004565 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f78e8162590 0x7f78e80c9fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:23.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:23.220+0000 7f78ed468640 1 -- 172.21.15.38:0/2682004565 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f78e80cfe00 msgr2=0x7f78e80c9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:23.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:23.220+0000 7f78ed468640 1 --2- 172.21.15.38:0/2682004565 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f78e80cfe00 0x7f78e80c9aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:23.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:23.220+0000 7f78ed468640 1 -- 172.21.15.38:0/2682004565 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f78e8162590 msgr2=0x7f78e80c9fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:23.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:23.220+0000 7f78ed468640 1 --2- 172.21.15.38:0/2682004565 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f78e8162590 0x7f78e80c9fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:23.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:23.220+0000 7f78ed468640 1 -- 172.21.15.38:0/2682004565 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78e815b590 con 0x7f78e80b2e60 2024-09-21T03:45:23.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:23.220+0000 7f78ed468640 1 --2- 172.21.15.38:0/2682004565 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f78e80b2e60 0x7f78e81582b0 secure :-1 s=READY pgs=2494 cs=0 l=1 rev1=1 crypto rx=0x7f78e400c760 tx=0x7f78e400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:23.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:23.220+0000 7f78ddffb640 1 -- 172.21.15.38:0/2682004565 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78e4003b70 con 0x7f78e80b2e60 2024-09-21T03:45:23.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:23.220+0000 7f78ddffb640 1 -- 172.21.15.38:0/2682004565 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f78e4003d30 con 0x7f78e80b2e60 2024-09-21T03:45:23.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:23.220+0000 7f78ef6cc640 1 -- 172.21.15.38:0/2682004565 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78e815b870 con 0x7f78e80b2e60 2024-09-21T03:45:23.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:23.220+0000 7f78ddffb640 1 -- 172.21.15.38:0/2682004565 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78e400aab0 con 0x7f78e80b2e60 2024-09-21T03:45:23.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:23.220+0000 7f78ef6cc640 1 -- 172.21.15.38:0/2682004565 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78e81587f0 con 0x7f78e80b2e60 2024-09-21T03:45:23.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:23.222+0000 7f78ef6cc640 1 -- 172.21.15.38:0/2682004565 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78e81629a0 con 0x7f78e80b2e60 2024-09-21T03:45:23.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:23.222+0000 7f78ddffb640 1 -- 172.21.15.38:0/2682004565 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f78e400ac50 con 0x7f78e80b2e60 2024-09-21T03:45:23.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:23.223+0000 7f78ddffb640 1 --2- 172.21.15.38:0/2682004565 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f78cc078b90 0x7f78cc07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:23.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:23.223+0000 7f78ddffb640 1 -- 172.21.15.38:0/2682004565 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 6303+0+0 (secure 0 0 0) 0x7f78e409ac10 con 0x7f78e80b2e60 2024-09-21T03:45:23.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:23.223+0000 7f78ecc67640 1 --2- 172.21.15.38:0/2682004565 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f78cc078b90 0x7f78cc07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:23.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:23.224+0000 7f78ecc67640 1 --2- 172.21.15.38:0/2682004565 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f78cc078b90 0x7f78cc07b0a0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f78d8033000 tx=0x7f78d8004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:23.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:23.227+0000 7f78ddffb640 1 -- 172.21.15.38:0/2682004565 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f78e4062840 con 0x7f78e80b2e60 2024-09-21T03:45:23.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:23.453+0000 7f78ef6cc640 1 -- 172.21.15.38:0/2682004565 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f78e806ddf0 con 0x7f78e80b2e60 2024-09-21T03:45:23.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:23.453+0000 7f78ddffb640 1 -- 172.21.15.38:0/2682004565 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 349 v349) ==== 95+0+5241 (secure 0 0 0) 0x7f78e4067270 con 0x7f78e80b2e60 2024-09-21T03:45:23.495 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:45:23.495 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":349,"btime":"2024-09-21T03:45:15:115320+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8399,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/3606957623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":3606957623},{"type":"v1","addr":"172.21.15.38:6835","nonce":3606957623}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":338},{"gid":8405,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/456651486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":456651486},{"type":"v1","addr":"172.21.15.195:6837","nonce":456651486}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":338},{"gid":9604,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/2649311840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":2649311840},{"type":"v1","addr":"172.21.15.195:6835","nonce":2649311840}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":338}],"filesystems":[{"mdsmap":{"epoch":349,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:44:56.710446+0000","modified":"2024-09-21T03:45:15.115308+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":147,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10935},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10935":{"gid":10935,"name":"c","rank":0,"incarnation":346,"state":"up:active","state_seq":4,"addr":"172.21.15.38:6833/3839260519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":3839260519},{"type":"v1","addr":"172.21.15.38:6833","nonce":3839260519}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10935,"qdb_cluster":[10935]},"id":7}]} 2024-09-21T03:45:23.496 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 349 2024-09-21T03:45:23.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:23.456+0000 7f78ef6cc640 1 -- 172.21.15.38:0/2682004565 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f78cc078b90 msgr2=0x7f78cc07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:23.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:23.456+0000 7f78ef6cc640 1 --2- 172.21.15.38:0/2682004565 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f78cc078b90 0x7f78cc07b0a0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f78d8033000 tx=0x7f78d8004730 comp rx=0 tx=0).stop 2024-09-21T03:45:23.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:23.456+0000 7f78ef6cc640 1 -- 172.21.15.38:0/2682004565 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f78e80b2e60 msgr2=0x7f78e81582b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:23.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:23.456+0000 7f78ef6cc640 1 --2- 172.21.15.38:0/2682004565 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f78e80b2e60 0x7f78e81582b0 secure :-1 s=READY pgs=2494 cs=0 l=1 rev1=1 crypto rx=0x7f78e400c760 tx=0x7f78e400cc20 comp rx=0 tx=0).stop 2024-09-21T03:45:23.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:23.456+0000 7f78ef6cc640 1 -- 172.21.15.38:0/2682004565 shutdown_connections 2024-09-21T03:45:23.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:23.456+0000 7f78ef6cc640 1 --2- 172.21.15.38:0/2682004565 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f78e8162590 0x7f78e80c9fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:23.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:23.456+0000 7f78ef6cc640 1 --2- 172.21.15.38:0/2682004565 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f78cc078b90 0x7f78cc07b0a0 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:23.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:23.456+0000 7f78ef6cc640 1 --2- 172.21.15.38:0/2682004565 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f78e80cfe00 0x7f78e80c9aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:23.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:23.456+0000 7f78ef6cc640 1 --2- 172.21.15.38:0/2682004565 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f78e80b2e60 0x7f78e81582b0 unknown :-1 s=CLOSED pgs=2494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:23.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:23.456+0000 7f78ef6cc640 1 -- 172.21.15.38:0/2682004565 >> 172.21.15.38:0/2682004565 conn(0x7f78e80d4440 msgr2=0x7f78e80b3270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:23.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:23.456+0000 7f78ef6cc640 1 -- 172.21.15.38:0/2682004565 shutdown_connections 2024-09-21T03:45:23.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:23.456+0000 7f78ef6cc640 1 -- 172.21.15.38:0/2682004565 wait complete. 2024-09-21T03:45:23.498 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T03:45:23.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:23.611+0000 7f0b15b69640 1 -- 172.21.15.38:0/1542063312 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0b100d01f0 msgr2=0x7f0b1015daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:23.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:23.611+0000 7f0b15b69640 1 --2- 172.21.15.38:0/1542063312 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0b100d01f0 0x7f0b1015daf0 secure :-1 s=READY pgs=2495 cs=0 l=1 rev1=1 crypto rx=0x7f0b100cec40 tx=0x7f0b0002a570 comp rx=0 tx=0).stop 2024-09-21T03:45:23.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:23.611+0000 7f0b15b69640 1 -- 172.21.15.38:0/1542063312 shutdown_connections 2024-09-21T03:45:23.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:23.611+0000 7f0b15b69640 1 --2- 172.21.15.38:0/1542063312 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0b100d0980 0x7f0b100d0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:23.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:23.611+0000 7f0b15b69640 1 --2- 172.21.15.38:0/1542063312 >> v2:172.21.15.195:3300/0 conn(0x7f0b1015a6f0 0x7f0b1015e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:23.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:23.611+0000 7f0b15b69640 1 --2- 172.21.15.38:0/1542063312 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0b100d01f0 0x7f0b1015daf0 unknown :-1 s=CLOSED pgs=2495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:23.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:23.611+0000 7f0b15b69640 1 -- 172.21.15.38:0/1542063312 >> 172.21.15.38:0/1542063312 conn(0x7f0b10072990 msgr2=0x7f0b10072da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:23.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:23.611+0000 7f0b15b69640 1 -- 172.21.15.38:0/1542063312 shutdown_connections 2024-09-21T03:45:23.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:23.611+0000 7f0b15b69640 1 -- 172.21.15.38:0/1542063312 wait complete. 2024-09-21T03:45:23.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:23.613+0000 7f0b15b69640 1 Processor -- start 2024-09-21T03:45:23.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:23.613+0000 7f0b15b69640 1 -- start start 2024-09-21T03:45:23.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:23.614+0000 7f0b15b69640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0b100d01f0 0x7f0b1014b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:23.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:23.614+0000 7f0b15b69640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0b100d0980 0x7f0b100a9450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:23.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:23.614+0000 7f0b15b69640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0b1015a6f0 0x7f0b100aacc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:23.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:23.614+0000 7f0b15b69640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f0b100ccfb0 con 0x7f0b1015a6f0 2024-09-21T03:45:23.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:23.614+0000 7f0b15b69640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f0b1015e980 con 0x7f0b100d01f0 2024-09-21T03:45:23.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:23.614+0000 7f0b15b69640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f0b100cd130 con 0x7f0b100d0980 2024-09-21T03:45:23.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:23.614+0000 7f0b0ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0b1015a6f0 0x7f0b100aacc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:23.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:23.614+0000 7f0b0ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0b1015a6f0 0x7f0b100aacc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:56426/0 (socket says 172.21.15.38:56426) 2024-09-21T03:45:23.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:23.614+0000 7f0b0ffff640 1 -- 172.21.15.38:0/853069635 learned_addr learned my addr 172.21.15.38:0/853069635 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:45:23.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:23.615+0000 7f0b0f7fe640 1 --2- 172.21.15.38:0/853069635 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0b100d01f0 0x7f0b1014b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:23.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:23.615+0000 7f0b0effd640 1 --2- 172.21.15.38:0/853069635 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0b100d0980 0x7f0b100a9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:23.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:23.615+0000 7f0b0ffff640 1 -- 172.21.15.38:0/853069635 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0b100d0980 msgr2=0x7f0b100a9450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:23.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:23.615+0000 7f0b0ffff640 1 --2- 172.21.15.38:0/853069635 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0b100d0980 0x7f0b100a9450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:23.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:23.615+0000 7f0b0ffff640 1 -- 172.21.15.38:0/853069635 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0b100d01f0 msgr2=0x7f0b1014b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:23.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:23.615+0000 7f0b0ffff640 1 --2- 172.21.15.38:0/853069635 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0b100d01f0 0x7f0b1014b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:23.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:23.615+0000 7f0b0ffff640 1 -- 172.21.15.38:0/853069635 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b100aa990 con 0x7f0b1015a6f0 2024-09-21T03:45:23.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:23.615+0000 7f0b0effd640 1 --2- 172.21.15.38:0/853069635 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0b100d0980 0x7f0b100a9450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:45:23.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:23.615+0000 7f0b0ffff640 1 --2- 172.21.15.38:0/853069635 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0b1015a6f0 0x7f0b100aacc0 secure :-1 s=READY pgs=2496 cs=0 l=1 rev1=1 crypto rx=0x7f0b0800ea00 tx=0x7f0b0800eec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:23.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:23.615+0000 7f0af7fff640 1 -- 172.21.15.38:0/853069635 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b0800cde0 con 0x7f0b1015a6f0 2024-09-21T03:45:23.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:23.615+0000 7f0af7fff640 1 -- 172.21.15.38:0/853069635 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0b08004b20 con 0x7f0b1015a6f0 2024-09-21T03:45:23.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:23.615+0000 7f0b15b69640 1 -- 172.21.15.38:0/853069635 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b100aa0f0 con 0x7f0b1015a6f0 2024-09-21T03:45:23.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:23.615+0000 7f0af7fff640 1 -- 172.21.15.38:0/853069635 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b08005020 con 0x7f0b1015a6f0 2024-09-21T03:45:23.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:23.615+0000 7f0b15b69640 1 -- 172.21.15.38:0/853069635 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b100aa6b0 con 0x7f0b1015a6f0 2024-09-21T03:45:23.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:23.616+0000 7f0b15b69640 1 -- 172.21.15.38:0/853069635 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b100ce490 con 0x7f0b1015a6f0 2024-09-21T03:45:23.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:23.617+0000 7f0af7fff640 1 -- 172.21.15.38:0/853069635 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f0b080051c0 con 0x7f0b1015a6f0 2024-09-21T03:45:23.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:23.618+0000 7f0af7fff640 1 --2- 172.21.15.38:0/853069635 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0afc078bc0 0x7f0afc07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:23.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:23.618+0000 7f0af7fff640 1 -- 172.21.15.38:0/853069635 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 6303+0+0 (secure 0 0 0) 0x7f0b0809bfb0 con 0x7f0b1015a6f0 2024-09-21T03:45:23.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:23.618+0000 7f0b0f7fe640 1 --2- 172.21.15.38:0/853069635 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0afc078bc0 0x7f0afc07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:23.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:23.618+0000 7f0b0f7fe640 1 --2- 172.21.15.38:0/853069635 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0afc078bc0 0x7f0afc07b0d0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f0b00033000 tx=0x7f0b00004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:23.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:23.621+0000 7f0af7fff640 1 -- 172.21.15.38:0/853069635 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f0b100ce490 con 0x7f0b1015a6f0 2024-09-21T03:45:23.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:23.812+0000 7f0b15b69640 1 -- 172.21.15.38:0/853069635 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0b1014bb00 con 0x7f0b1015a6f0 2024-09-21T03:45:23.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:23.813+0000 7f0af7fff640 1 -- 172.21.15.38:0/853069635 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v149) ==== 74+0+17362 (secure 0 0 0) 0x7f0b08064180 con 0x7f0b1015a6f0 2024-09-21T03:45:23.816 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:45:23.816 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":149,"fsid":"9d50d66f-656b-4fe2-8e3b-fe53b79c4d43","created":"2024-09-21T03:33:10.677865+0000","modified":"2024-09-21T03:45:18.148053+0000","last_up_change":"2024-09-21T03:33:24.023421+0000","last_in_change":"2024-09-21T03:33:14.296193+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T03:33:22.992573+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-09-21T03:44:54.386654+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"145","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-09-21T03:44:55.667528+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"146","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dbab183f-7e63-48b8-960a-ae55fd5bf565","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6801","nonce":2381639433}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6803","nonce":2381639433}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6807","nonce":2381639433}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6805","nonce":2381639433}]},"public_addr":"172.21.15.38:6801/2381639433","cluster_addr":"172.21.15.38:6803/2381639433","heartbeat_back_addr":"172.21.15.38:6807/2381639433","heartbeat_front_addr":"172.21.15.38:6805/2381639433","state":["exists","up"]},{"osd":1,"uuid":"fa1c0e8a-ebc1-4359-8146-bd5342abbc16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6817","nonce":1491818732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6819","nonce":1491818732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6823","nonce":1491818732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6821","nonce":1491818732}]},"public_addr":"172.21.15.38:6817/1491818732","cluster_addr":"172.21.15.38:6819/1491818732","heartbeat_back_addr":"172.21.15.38:6823/1491818732","heartbeat_front_addr":"172.21.15.38:6821/1491818732","state":["exists","up"]},{"osd":2,"uuid":"2ed3cd2d-385d-4fc6-9869-214552a84803","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6825","nonce":2577858902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6827","nonce":2577858902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6831","nonce":2577858902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6829","nonce":2577858902}]},"public_addr":"172.21.15.38:6825/2577858902","cluster_addr":"172.21.15.38:6827/2577858902","heartbeat_back_addr":"172.21.15.38:6831/2577858902","heartbeat_front_addr":"172.21.15.38:6829/2577858902","state":["exists","up"]},{"osd":3,"uuid":"62d1d95e-919d-4250-be71-1d34cfa70dbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6809","nonce":1286810294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6811","nonce":1286810294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6815","nonce":1286810294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6813","nonce":1286810294}]},"public_addr":"172.21.15.38:6809/1286810294","cluster_addr":"172.21.15.38:6811/1286810294","heartbeat_back_addr":"172.21.15.38:6815/1286810294","heartbeat_front_addr":"172.21.15.38:6813/1286810294","state":["exists","up"]},{"osd":4,"uuid":"7c9f9314-99d7-4d73-8392-37809a9f54b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6803","nonce":3401837571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6805","nonce":3401837571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6809","nonce":3401837571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6807","nonce":3401837571}]},"public_addr":"172.21.15.195:6803/3401837571","cluster_addr":"172.21.15.195:6805/3401837571","heartbeat_back_addr":"172.21.15.195:6809/3401837571","heartbeat_front_addr":"172.21.15.195:6807/3401837571","state":["exists","up"]},{"osd":5,"uuid":"bde4fe9f-8414-4bba-bf4a-43973d1976bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6811","nonce":2080960180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6813","nonce":2080960180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6817","nonce":2080960180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6815","nonce":2080960180}]},"public_addr":"172.21.15.195:6811/2080960180","cluster_addr":"172.21.15.195:6813/2080960180","heartbeat_back_addr":"172.21.15.195:6817/2080960180","heartbeat_front_addr":"172.21.15.195:6815/2080960180","state":["exists","up"]},{"osd":6,"uuid":"6ee6d7f4-1a5f-4a4c-ac16-48fe05f6a002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6827","nonce":4055825675}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6829","nonce":4055825675}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6833","nonce":4055825675}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6831","nonce":4055825675}]},"public_addr":"172.21.15.195:6827/4055825675","cluster_addr":"172.21.15.195:6829/4055825675","heartbeat_back_addr":"172.21.15.195:6833/4055825675","heartbeat_front_addr":"172.21.15.195:6831/4055825675","state":["exists","up"]},{"osd":7,"uuid":"efeb5f51-329a-4484-8290-92e7d3503d05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6819","nonce":1606106342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6821","nonce":1606106342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6825","nonce":1606106342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6823","nonce":1606106342}]},"public_addr":"172.21.15.195:6819/1606106342","cluster_addr":"172.21.15.195:6821/1606106342","heartbeat_back_addr":"172.21.15.195:6825/1606106342","heartbeat_front_addr":"172.21.15.195:6823/1606106342","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.901343+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:21.115003+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.901623+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.835238+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.132199+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:19.817560+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.829690+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:21.087300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.5","mappings":[{"from":4,"to":3}]},{"pgid":"16.1c","mappings":[{"from":4,"to":3}]},{"pgid":"16.3b","mappings":[{"from":4,"to":6}]},{"pgid":"16.3e","mappings":[{"from":2,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.38:6833/3866292839":"2024-09-22T03:45:11.068318+0000","172.21.15.38:6832/3866292839":"2024-09-22T03:45:11.068318+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T03:45:23.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:23.815+0000 7f0b15b69640 1 -- 172.21.15.38:0/853069635 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0afc078bc0 msgr2=0x7f0afc07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:23.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:23.815+0000 7f0b15b69640 1 --2- 172.21.15.38:0/853069635 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0afc078bc0 0x7f0afc07b0d0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f0b00033000 tx=0x7f0b00004730 comp rx=0 tx=0).stop 2024-09-21T03:45:23.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:23.815+0000 7f0b15b69640 1 -- 172.21.15.38:0/853069635 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0b1015a6f0 msgr2=0x7f0b100aacc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:23.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:23.815+0000 7f0b15b69640 1 --2- 172.21.15.38:0/853069635 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0b1015a6f0 0x7f0b100aacc0 secure :-1 s=READY pgs=2496 cs=0 l=1 rev1=1 crypto rx=0x7f0b0800ea00 tx=0x7f0b0800eec0 comp rx=0 tx=0).stop 2024-09-21T03:45:23.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:23.815+0000 7f0b15b69640 1 -- 172.21.15.38:0/853069635 shutdown_connections 2024-09-21T03:45:23.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:23.815+0000 7f0b15b69640 1 --2- 172.21.15.38:0/853069635 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0b1015a6f0 0x7f0b100aacc0 unknown :-1 s=CLOSED pgs=2496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:23.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:23.815+0000 7f0b15b69640 1 --2- 172.21.15.38:0/853069635 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0afc078bc0 0x7f0afc07b0d0 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:23.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:23.815+0000 7f0b15b69640 1 --2- 172.21.15.38:0/853069635 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0b100d0980 0x7f0b100a9450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:23.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:23.815+0000 7f0b15b69640 1 --2- 172.21.15.38:0/853069635 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0b100d01f0 0x7f0b1014b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:23.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:23.815+0000 7f0b15b69640 1 -- 172.21.15.38:0/853069635 >> 172.21.15.38:0/853069635 conn(0x7f0b10072990 msgr2=0x7f0b1015d220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:23.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:23.816+0000 7f0b15b69640 1 -- 172.21.15.38:0/853069635 shutdown_connections 2024-09-21T03:45:23.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:23.816+0000 7f0b15b69640 1 -- 172.21.15.38:0/853069635 wait complete. 2024-09-21T03:45:23.830 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-21T03:45:23.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:23.946+0000 7fe525007640 1 -- 172.21.15.38:0/1236542526 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe5200ceb90 msgr2=0x7fe5200cefa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:23.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:23.946+0000 7fe525007640 1 --2- 172.21.15.38:0/1236542526 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe5200ceb90 0x7fe5200cefa0 secure :-1 s=READY pgs=2498 cs=0 l=1 rev1=1 crypto rx=0x7fe5200cfba0 tx=0x7fe51802a790 comp rx=0 tx=0).stop 2024-09-21T03:45:23.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:23.946+0000 7fe525007640 1 -- 172.21.15.38:0/1236542526 shutdown_connections 2024-09-21T03:45:23.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:23.946+0000 7fe525007640 1 --2- 172.21.15.38:0/1236542526 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe5200ceb90 0x7fe5200cefa0 unknown :-1 s=CLOSED pgs=2498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:23.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:23.946+0000 7fe525007640 1 --2- 172.21.15.38:0/1236542526 >> v2:172.21.15.195:3300/0 conn(0x7fe52015ee80 0x7fe5200d0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:23.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:23.946+0000 7fe525007640 1 -- 172.21.15.38:0/1236542526 >> 172.21.15.38:0/1236542526 conn(0x7fe52009d830 msgr2=0x7fe5200b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:23.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:23.946+0000 7fe525007640 1 -- 172.21.15.38:0/1236542526 shutdown_connections 2024-09-21T03:45:23.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:23.946+0000 7fe525007640 1 -- 172.21.15.38:0/1236542526 wait complete. 2024-09-21T03:45:23.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:23.948+0000 7fe525007640 1 Processor -- start 2024-09-21T03:45:23.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:23.948+0000 7fe525007640 1 -- start start 2024-09-21T03:45:23.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:23.949+0000 7fe525007640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe5200b9ad0 0x7fe5201f5f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:23.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:23.949+0000 7fe525007640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe5200ceb90 0x7fe5200b23f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:23.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:23.949+0000 7fe525007640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe52015ee80 0x7fe52006d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:23.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:23.949+0000 7fe525007640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fe5200b6580 con 0x7fe5200b9ad0 2024-09-21T03:45:23.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:23.949+0000 7fe525007640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fe5200ba0f0 con 0x7fe5200ceb90 2024-09-21T03:45:23.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:23.949+0000 7fe525007640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fe5200b6700 con 0x7fe52015ee80 2024-09-21T03:45:23.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:23.949+0000 7fe51ed9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe5200b9ad0 0x7fe5201f5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:23.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:23.949+0000 7fe51ed9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe5200b9ad0 0x7fe5201f5f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:56432/0 (socket says 172.21.15.38:56432) 2024-09-21T03:45:23.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:23.949+0000 7fe51ed9d640 1 -- 172.21.15.38:0/311728066 learned_addr learned my addr 172.21.15.38:0/311728066 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:45:23.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:23.949+0000 7fe51f59e640 1 --2- 172.21.15.38:0/311728066 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe52015ee80 0x7fe52006d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:23.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:23.949+0000 7fe517fff640 1 --2- 172.21.15.38:0/311728066 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe5200ceb90 0x7fe5200b23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:23.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:23.949+0000 7fe51ed9d640 1 -- 172.21.15.38:0/311728066 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe52015ee80 msgr2=0x7fe52006d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:23.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:23.949+0000 7fe51ed9d640 1 --2- 172.21.15.38:0/311728066 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe52015ee80 0x7fe52006d680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:23.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:23.949+0000 7fe51ed9d640 1 -- 172.21.15.38:0/311728066 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe5200ceb90 msgr2=0x7fe5200b23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:23.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:23.949+0000 7fe51ed9d640 1 --2- 172.21.15.38:0/311728066 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe5200ceb90 0x7fe5200b23f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:23.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:23.949+0000 7fe51ed9d640 1 -- 172.21.15.38:0/311728066 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5201f6460 con 0x7fe5200b9ad0 2024-09-21T03:45:23.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:23.949+0000 7fe517fff640 1 --2- 172.21.15.38:0/311728066 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe5200ceb90 0x7fe5200b23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:45:23.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:23.950+0000 7fe51ed9d640 1 --2- 172.21.15.38:0/311728066 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe5200b9ad0 0x7fe5201f5f20 secure :-1 s=READY pgs=2499 cs=0 l=1 rev1=1 crypto rx=0x7fe51000d750 tx=0x7fe51000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:23.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:23.950+0000 7fe5177fe640 1 -- 172.21.15.38:0/311728066 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe510017070 con 0x7fe5200b9ad0 2024-09-21T03:45:23.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:23.950+0000 7fe5177fe640 1 -- 172.21.15.38:0/311728066 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe510004310 con 0x7fe5200b9ad0 2024-09-21T03:45:23.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:23.950+0000 7fe525007640 1 -- 172.21.15.38:0/311728066 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5200cc650 con 0x7fe5200b9ad0 2024-09-21T03:45:23.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:23.950+0000 7fe5177fe640 1 -- 172.21.15.38:0/311728066 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe51000de60 con 0x7fe5200b9ad0 2024-09-21T03:45:23.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:23.950+0000 7fe525007640 1 -- 172.21.15.38:0/311728066 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe52006ce80 con 0x7fe5200b9ad0 2024-09-21T03:45:23.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:23.951+0000 7fe525007640 1 -- 172.21.15.38:0/311728066 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5200b6c20 con 0x7fe5200b9ad0 2024-09-21T03:45:23.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:23.952+0000 7fe5177fe640 1 -- 172.21.15.38:0/311728066 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fe510009c30 con 0x7fe5200b9ad0 2024-09-21T03:45:23.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:23.952+0000 7fe5177fe640 1 --2- 172.21.15.38:0/311728066 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fe500078b90 0x7fe50007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:23.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:23.953+0000 7fe5177fe640 1 -- 172.21.15.38:0/311728066 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 6303+0+0 (secure 0 0 0) 0x7fe51009b430 con 0x7fe5200b9ad0 2024-09-21T03:45:23.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:23.953+0000 7fe517fff640 1 --2- 172.21.15.38:0/311728066 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fe500078b90 0x7fe50007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:23.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:23.953+0000 7fe517fff640 1 --2- 172.21.15.38:0/311728066 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fe500078b90 0x7fe50007b0a0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7fe5200cad60 tx=0x7fe508009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:23.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:23.957+0000 7fe5177fe640 1 -- 172.21.15.38:0/311728066 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fe510020020 con 0x7fe5200b9ad0 2024-09-21T03:45:24.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:24.183+0000 7fe525007640 1 -- 172.21.15.38:0/311728066 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fe520158c80 con 0x7fe5200b9ad0 2024-09-21T03:45:24.190 INFO:tasks.ceph.mon.a.smithi038.stderr:2024-09-21T03:45:24.187+0000 7f3f45713640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T03:45:24.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:24.189+0000 7fe5177fe640 1 -- 172.21.15.38:0/311728066 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v350) ==== 161+0+0 (secure 0 0 0) 0x7fe5201f6460 con 0x7fe5200b9ad0 2024-09-21T03:45:24.192 INFO:teuthology.orchestra.run.smithi038.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-21T03:45:24.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:24.191+0000 7fe525007640 1 -- 172.21.15.38:0/311728066 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fe500078b90 msgr2=0x7fe50007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:24.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:24.192+0000 7fe525007640 1 --2- 172.21.15.38:0/311728066 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fe500078b90 0x7fe50007b0a0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7fe5200cad60 tx=0x7fe508009040 comp rx=0 tx=0).stop 2024-09-21T03:45:24.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:24.192+0000 7fe525007640 1 -- 172.21.15.38:0/311728066 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe5200b9ad0 msgr2=0x7fe5201f5f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:24.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:24.192+0000 7fe525007640 1 --2- 172.21.15.38:0/311728066 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe5200b9ad0 0x7fe5201f5f20 secure :-1 s=READY pgs=2499 cs=0 l=1 rev1=1 crypto rx=0x7fe51000d750 tx=0x7fe51000dc10 comp rx=0 tx=0).stop 2024-09-21T03:45:24.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:24.192+0000 7fe525007640 1 -- 172.21.15.38:0/311728066 shutdown_connections 2024-09-21T03:45:24.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:24.192+0000 7fe525007640 1 --2- 172.21.15.38:0/311728066 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fe500078b90 0x7fe50007b0a0 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:24.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:24.192+0000 7fe525007640 1 --2- 172.21.15.38:0/311728066 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe52015ee80 0x7fe52006d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:24.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:24.192+0000 7fe525007640 1 --2- 172.21.15.38:0/311728066 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe5200ceb90 0x7fe5200b23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:24.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:24.192+0000 7fe525007640 1 --2- 172.21.15.38:0/311728066 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe5200b9ad0 0x7fe5201f5f20 unknown :-1 s=CLOSED pgs=2499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:24.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:24.192+0000 7fe525007640 1 -- 172.21.15.38:0/311728066 >> 172.21.15.38:0/311728066 conn(0x7fe52009d830 msgr2=0x7fe52015e220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:24.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:24.192+0000 7fe525007640 1 -- 172.21.15.38:0/311728066 shutdown_connections 2024-09-21T03:45:24.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:24.192+0000 7fe525007640 1 -- 172.21.15.38:0/311728066 wait complete. 2024-09-21T03:45:24.197 INFO:tasks.ceph.mds.c.smithi038.stderr: -1114> 2024-09-21T03:45:17.093+0000 7f6c69fd9640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T03:45:24.197 INFO:tasks.ceph.mds.c.smithi038.stderr: -694> 2024-09-21T03:45:18.064+0000 7f6c65fd1640 -1 mds.0.cache corrupt backtrace on ino x00x10000000001: Decoder at 'void inode_backtrace_t::decode(ceph::buffer::v15_2_0::list::const_iterator&)' v=5 cannot decode v=114 minimal_decoder=104: Malformed input [buffer:3] 2024-09-21T03:45:24.198 INFO:tasks.ceph.mds.c.smithi038.stderr: -447> 2024-09-21T03:45:19.270+0000 7f6c65fd1640 -1 log_channel(cluster) log [ERR] : dir 0x100000001f6 object missing on disk; some files may be lost (/dir2) 2024-09-21T03:45:24.207 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-21T03:45:24.253 INFO:tasks.ceph.mds.c.smithi038.stdout:starting mds.c at 2024-09-21T03:45:24.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:24.325+0000 7f66ce412640 1 -- 172.21.15.38:0/3337274526 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f66c80b5670 msgr2=0x7f66c815f630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:24.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:24.325+0000 7f66ce412640 1 --2- 172.21.15.38:0/3337274526 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f66c80b5670 0x7f66c815f630 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f66c0005fd0 tx=0x7f66c002a8b0 comp rx=0 tx=0).stop 2024-09-21T03:45:24.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:24.325+0000 7f66ce412640 1 -- 172.21.15.38:0/3337274526 shutdown_connections 2024-09-21T03:45:24.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:24.325+0000 7f66ce412640 1 --2- 172.21.15.38:0/3337274526 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f66c80b5670 0x7f66c815f630 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:24.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:24.325+0000 7f66ce412640 1 --2- 172.21.15.38:0/3337274526 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f66c806def0 0x7f66c806f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:24.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:24.325+0000 7f66ce412640 1 -- 172.21.15.38:0/3337274526 >> 172.21.15.38:0/3337274526 conn(0x7f66c80cdca0 msgr2=0x7f66c80ce0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:24.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:24.325+0000 7f66ce412640 1 -- 172.21.15.38:0/3337274526 shutdown_connections 2024-09-21T03:45:24.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:24.325+0000 7f66ce412640 1 -- 172.21.15.38:0/3337274526 wait complete. 2024-09-21T03:45:24.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:24.327+0000 7f66ce412640 1 Processor -- start 2024-09-21T03:45:24.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:24.327+0000 7f66ce412640 1 -- start start 2024-09-21T03:45:24.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:24.328+0000 7f66ce412640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f66c806def0 0x7f66c8145b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:24.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:24.328+0000 7f66ce412640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f66c80b5260 0x7f66c81580d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:24.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:24.328+0000 7f66ce412640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f66c80b5670 0x7f66c8156a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:24.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:24.328+0000 7f66ce412640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f66c815ffb0 con 0x7f66c80b5670 2024-09-21T03:45:24.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:24.328+0000 7f66ce412640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f66c815fe30 con 0x7f66c80b5260 2024-09-21T03:45:24.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:24.328+0000 7f66ce412640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f66c8071300 con 0x7f66c806def0 2024-09-21T03:45:24.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:24.328+0000 7f66cc9af640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f66c80b5670 0x7f66c8156a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:24.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:24.328+0000 7f66cc9af640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f66c80b5670 0x7f66c8156a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:56452/0 (socket says 172.21.15.38:56452) 2024-09-21T03:45:24.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:24.328+0000 7f66cc9af640 1 -- 172.21.15.38:0/3094284727 learned_addr learned my addr 172.21.15.38:0/3094284727 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:45:24.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:24.328+0000 7f66c77fe640 1 --2- 172.21.15.38:0/3094284727 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f66c80b5260 0x7f66c81580d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:24.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:24.328+0000 7f66c7fff640 1 --2- 172.21.15.38:0/3094284727 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f66c806def0 0x7f66c8145b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:24.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:24.329+0000 7f66cc9af640 1 -- 172.21.15.38:0/3094284727 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f66c806def0 msgr2=0x7f66c8145b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:24.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:24.329+0000 7f66cc9af640 1 --2- 172.21.15.38:0/3094284727 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f66c806def0 0x7f66c8145b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:24.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:24.329+0000 7f66cc9af640 1 -- 172.21.15.38:0/3094284727 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f66c80b5260 msgr2=0x7f66c81580d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:24.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:24.329+0000 7f66cc9af640 1 --2- 172.21.15.38:0/3094284727 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f66c80b5260 0x7f66c81580d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:24.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:24.329+0000 7f66cc9af640 1 -- 172.21.15.38:0/3094284727 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66c81464a0 con 0x7f66c80b5670 2024-09-21T03:45:24.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:24.329+0000 7f66c77fe640 1 --2- 172.21.15.38:0/3094284727 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f66c80b5260 0x7f66c81580d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:45:24.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:24.329+0000 7f66cc9af640 1 --2- 172.21.15.38:0/3094284727 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f66c80b5670 0x7f66c8156a70 secure :-1 s=READY pgs=2502 cs=0 l=1 rev1=1 crypto rx=0x7f66c002aea0 tx=0x7f66c0033f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:24.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:24.329+0000 7f66c4ff9640 1 -- 172.21.15.38:0/3094284727 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66c0031990 con 0x7f66c80b5670 2024-09-21T03:45:24.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:24.329+0000 7f66c4ff9640 1 -- 172.21.15.38:0/3094284727 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f66c0031c20 con 0x7f66c80b5670 2024-09-21T03:45:24.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:24.329+0000 7f66ce412640 1 -- 172.21.15.38:0/3094284727 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66c8146780 con 0x7f66c80b5670 2024-09-21T03:45:24.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:24.329+0000 7f66c4ff9640 1 -- 172.21.15.38:0/3094284727 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66c00330d0 con 0x7f66c80b5670 2024-09-21T03:45:24.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:24.329+0000 7f66ce412640 1 -- 172.21.15.38:0/3094284727 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66c8158900 con 0x7f66c80b5670 2024-09-21T03:45:24.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:24.330+0000 7f66ce412640 1 -- 172.21.15.38:0/3094284727 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66c8146210 con 0x7f66c80b5670 2024-09-21T03:45:24.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:24.331+0000 7f66c4ff9640 1 -- 172.21.15.38:0/3094284727 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f66c0033270 con 0x7f66c80b5670 2024-09-21T03:45:24.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:24.331+0000 7f66c4ff9640 1 --2- 172.21.15.38:0/3094284727 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f66b4078b90 0x7f66b407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:24.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:24.332+0000 7f66c4ff9640 1 -- 172.21.15.38:0/3094284727 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(150..150 src has 1..150) ==== 6389+0+0 (secure 0 0 0) 0x7f66c00bc320 con 0x7f66c80b5670 2024-09-21T03:45:24.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:24.332+0000 7f66c7fff640 1 --2- 172.21.15.38:0/3094284727 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f66b4078b90 0x7f66b407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:24.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:24.332+0000 7f66c7fff640 1 --2- 172.21.15.38:0/3094284727 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f66b4078b90 0x7f66b407b0a0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f66c815efd0 tx=0x7f66b8005f90 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:24.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:24.336+0000 7f66c4ff9640 1 -- 172.21.15.38:0/3094284727 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f66c003f030 con 0x7f66c80b5670 2024-09-21T03:45:24.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:24.564+0000 7f66ce412640 1 -- 172.21.15.38:0/3094284727 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f66c8158b60 con 0x7f66c80b5670 2024-09-21T03:45:25.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:25.196+0000 7f66c4ff9640 1 -- 172.21.15.38:0/3094284727 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v351) ==== 104+0+0 (secure 0 0 0) 0x7f66c0083ef0 con 0x7f66c80b5670 2024-09-21T03:45:25.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:25.200+0000 7f66ce412640 1 -- 172.21.15.38:0/3094284727 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f66b4078b90 msgr2=0x7f66b407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:25.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:25.200+0000 7f66ce412640 1 --2- 172.21.15.38:0/3094284727 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f66b4078b90 0x7f66b407b0a0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f66c815efd0 tx=0x7f66b8005f90 comp rx=0 tx=0).stop 2024-09-21T03:45:25.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:25.200+0000 7f66ce412640 1 -- 172.21.15.38:0/3094284727 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f66c80b5670 msgr2=0x7f66c8156a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:25.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:25.200+0000 7f66ce412640 1 --2- 172.21.15.38:0/3094284727 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f66c80b5670 0x7f66c8156a70 secure :-1 s=READY pgs=2502 cs=0 l=1 rev1=1 crypto rx=0x7f66c002aea0 tx=0x7f66c0033f70 comp rx=0 tx=0).stop 2024-09-21T03:45:25.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:25.200+0000 7f66ce412640 1 -- 172.21.15.38:0/3094284727 shutdown_connections 2024-09-21T03:45:25.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:25.200+0000 7f66ce412640 1 --2- 172.21.15.38:0/3094284727 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f66c80b5670 0x7f66c8156a70 unknown :-1 s=CLOSED pgs=2502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:25.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:25.200+0000 7f66ce412640 1 --2- 172.21.15.38:0/3094284727 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f66c80b5260 0x7f66c81580d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:25.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:25.200+0000 7f66ce412640 1 --2- 172.21.15.38:0/3094284727 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f66b4078b90 0x7f66b407b0a0 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:25.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:25.200+0000 7f66ce412640 1 --2- 172.21.15.38:0/3094284727 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f66c806def0 0x7f66c8145b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:25.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:25.200+0000 7f66ce412640 1 -- 172.21.15.38:0/3094284727 >> 172.21.15.38:0/3094284727 conn(0x7f66c80cdca0 msgr2=0x7f66c8068d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:25.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:25.200+0000 7f66ce412640 1 -- 172.21.15.38:0/3094284727 shutdown_connections 2024-09-21T03:45:25.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:25.200+0000 7f66ce412640 1 -- 172.21.15.38:0/3094284727 wait complete. 2024-09-21T03:45:25.218 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-21T03:45:25.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:25.344+0000 7f48c2f13640 1 -- 172.21.15.38:0/1587698883 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f48bc15b840 msgr2=0x7f48bc0d0f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:25.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:25.344+0000 7f48c2f13640 1 --2- 172.21.15.38:0/1587698883 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f48bc15b840 0x7f48bc0d0f60 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f48b0009830 tx=0x7f48b002a510 comp rx=0 tx=0).stop 2024-09-21T03:45:25.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:25.344+0000 7f48c2f13640 1 -- 172.21.15.38:0/1587698883 shutdown_connections 2024-09-21T03:45:25.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:25.344+0000 7f48c2f13640 1 --2- 172.21.15.38:0/1587698883 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f48bc09a9e0 0x7f48bc0d14a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:25.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:25.344+0000 7f48c2f13640 1 --2- 172.21.15.38:0/1587698883 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f48bc15b840 0x7f48bc0d0f60 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:25.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:25.344+0000 7f48c2f13640 1 -- 172.21.15.38:0/1587698883 >> 172.21.15.38:0/1587698883 conn(0x7f48bc0741a0 msgr2=0x7f48bc0b60f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:25.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:25.344+0000 7f48c2f13640 1 -- 172.21.15.38:0/1587698883 shutdown_connections 2024-09-21T03:45:25.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:25.344+0000 7f48c2f13640 1 -- 172.21.15.38:0/1587698883 wait complete. 2024-09-21T03:45:25.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:25.346+0000 7f48c2f13640 1 Processor -- start 2024-09-21T03:45:25.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:25.346+0000 7f48c2f13640 1 -- start start 2024-09-21T03:45:25.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:25.347+0000 7f48c2f13640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f48bc06def0 0x7f48bc0b0020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:25.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:25.347+0000 7f48c2f13640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f48bc09a9e0 0x7f48bc151c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:25.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:25.347+0000 7f48c2f13640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f48bc15b840 0x7f48bc1e70d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:25.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:25.347+0000 7f48c2f13640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f48bc0d0ae0 con 0x7f48bc15b840 2024-09-21T03:45:25.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:25.347+0000 7f48c2f13640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f48bc06f7e0 con 0x7f48bc09a9e0 2024-09-21T03:45:25.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:25.347+0000 7f48c2f13640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f48bc0d0c60 con 0x7f48bc06def0 2024-09-21T03:45:25.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:25.347+0000 7f48c14b0640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f48bc15b840 0x7f48bc1e70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:25.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:25.347+0000 7f48c14b0640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f48bc15b840 0x7f48bc1e70d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:56466/0 (socket says 172.21.15.38:56466) 2024-09-21T03:45:25.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:25.347+0000 7f48c14b0640 1 -- 172.21.15.38:0/461795268 learned_addr learned my addr 172.21.15.38:0/461795268 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:45:25.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:25.347+0000 7f48c0caf640 1 --2- 172.21.15.38:0/461795268 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f48bc06def0 0x7f48bc0b0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:25.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:25.347+0000 7f48bbfff640 1 --2- 172.21.15.38:0/461795268 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f48bc09a9e0 0x7f48bc151c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:25.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:25.347+0000 7f48c14b0640 1 -- 172.21.15.38:0/461795268 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f48bc06def0 msgr2=0x7f48bc0b0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:25.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:25.347+0000 7f48c14b0640 1 --2- 172.21.15.38:0/461795268 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f48bc06def0 0x7f48bc0b0020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:25.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:25.347+0000 7f48c14b0640 1 -- 172.21.15.38:0/461795268 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f48bc09a9e0 msgr2=0x7f48bc151c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:25.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:25.347+0000 7f48c14b0640 1 --2- 172.21.15.38:0/461795268 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f48bc09a9e0 0x7f48bc151c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:25.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:25.347+0000 7f48c14b0640 1 -- 172.21.15.38:0/461795268 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48bc0b0560 con 0x7f48bc15b840 2024-09-21T03:45:25.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:25.348+0000 7f48c14b0640 1 --2- 172.21.15.38:0/461795268 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f48bc15b840 0x7f48bc1e70d0 secure :-1 s=READY pgs=2504 cs=0 l=1 rev1=1 crypto rx=0x7f48b400caa0 tx=0x7f48b400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:25.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:25.348+0000 7f48b97fa640 1 -- 172.21.15.38:0/461795268 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48b4016020 con 0x7f48bc15b840 2024-09-21T03:45:25.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:25.348+0000 7f48b97fa640 1 -- 172.21.15.38:0/461795268 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f48b400acf0 con 0x7f48bc15b840 2024-09-21T03:45:25.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:25.348+0000 7f48c2f13640 1 -- 172.21.15.38:0/461795268 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48bc152640 con 0x7f48bc15b840 2024-09-21T03:45:25.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:25.348+0000 7f48b97fa640 1 -- 172.21.15.38:0/461795268 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48b4012680 con 0x7f48bc15b840 2024-09-21T03:45:25.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:25.348+0000 7f48c2f13640 1 -- 172.21.15.38:0/461795268 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48bc1588f0 con 0x7f48bc15b840 2024-09-21T03:45:25.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:25.349+0000 7f48c2f13640 1 -- 172.21.15.38:0/461795268 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48bc0b0730 con 0x7f48bc15b840 2024-09-21T03:45:25.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:25.350+0000 7f48b97fa640 1 -- 172.21.15.38:0/461795268 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f48b4019020 con 0x7f48bc15b840 2024-09-21T03:45:25.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:25.350+0000 7f48b97fa640 1 --2- 172.21.15.38:0/461795268 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f48a0078b90 0x7f48a007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:25.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:25.351+0000 7f48b97fa640 1 -- 172.21.15.38:0/461795268 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(150..150 src has 1..150) ==== 6389+0+0 (secure 0 0 0) 0x7f48b409b440 con 0x7f48bc15b840 2024-09-21T03:45:25.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:25.351+0000 7f48c0caf640 1 --2- 172.21.15.38:0/461795268 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f48a0078b90 0x7f48a007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:25.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:25.351+0000 7f48c0caf640 1 --2- 172.21.15.38:0/461795268 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f48a0078b90 0x7f48a007b0a0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f48b0034000 tx=0x7f48b0004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:25.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:25.355+0000 7f48b97fa640 1 -- 172.21.15.38:0/461795268 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f48b4020020 con 0x7f48bc15b840 2024-09-21T03:45:25.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:25.548+0000 7f48c2f13640 1 -- 172.21.15.38:0/461795268 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f48bc06ad10 con 0x7f48bc15b840 2024-09-21T03:45:26.202 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:26.200+0000 7f48b97fa640 1 -- 172.21.15.38:0/461795268 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v151) ==== 181+0+0 (secure 0 0 0) 0x7f48bc06ad10 con 0x7f48bc15b840 2024-09-21T03:45:26.202 INFO:teuthology.orchestra.run.smithi038.stderr:pool 'cephfs_metadata' removed 2024-09-21T03:45:26.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:26.204+0000 7f48c2f13640 1 -- 172.21.15.38:0/461795268 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f48a0078b90 msgr2=0x7f48a007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:26.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:26.204+0000 7f48c2f13640 1 --2- 172.21.15.38:0/461795268 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f48a0078b90 0x7f48a007b0a0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f48b0034000 tx=0x7f48b0004730 comp rx=0 tx=0).stop 2024-09-21T03:45:26.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:26.204+0000 7f48c2f13640 1 -- 172.21.15.38:0/461795268 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f48bc15b840 msgr2=0x7f48bc1e70d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:26.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:26.204+0000 7f48c2f13640 1 --2- 172.21.15.38:0/461795268 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f48bc15b840 0x7f48bc1e70d0 secure :-1 s=READY pgs=2504 cs=0 l=1 rev1=1 crypto rx=0x7f48b400caa0 tx=0x7f48b400cf60 comp rx=0 tx=0).stop 2024-09-21T03:45:26.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:26.205+0000 7f48c2f13640 1 -- 172.21.15.38:0/461795268 shutdown_connections 2024-09-21T03:45:26.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:26.205+0000 7f48c2f13640 1 --2- 172.21.15.38:0/461795268 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f48bc15b840 0x7f48bc1e70d0 unknown :-1 s=CLOSED pgs=2504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:26.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:26.205+0000 7f48c2f13640 1 --2- 172.21.15.38:0/461795268 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f48bc09a9e0 0x7f48bc151c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:26.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:26.205+0000 7f48c2f13640 1 --2- 172.21.15.38:0/461795268 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f48a0078b90 0x7f48a007b0a0 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:26.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:26.205+0000 7f48c2f13640 1 --2- 172.21.15.38:0/461795268 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f48bc06def0 0x7f48bc0b0020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:26.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:26.205+0000 7f48c2f13640 1 -- 172.21.15.38:0/461795268 >> 172.21.15.38:0/461795268 conn(0x7f48bc0741a0 msgr2=0x7f48bc0b60f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:26.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:26.206+0000 7f48c2f13640 1 -- 172.21.15.38:0/461795268 shutdown_connections 2024-09-21T03:45:26.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:26.206+0000 7f48c2f13640 1 -- 172.21.15.38:0/461795268 wait complete. 2024-09-21T03:45:26.224 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-21T03:45:26.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:26.351+0000 7f89ddba1640 1 -- 172.21.15.38:0/4070996400 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f89d81624f0 msgr2=0x7f89d80b39a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:26.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:26.351+0000 7f89ddba1640 1 --2- 172.21.15.38:0/4070996400 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f89d81624f0 0x7f89d80b39a0 secure :-1 s=READY pgs=2506 cs=0 l=1 rev1=1 crypto rx=0x7f89d80ba2f0 tx=0x7f89c802a4a0 comp rx=0 tx=0).stop 2024-09-21T03:45:26.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:26.351+0000 7f89ddba1640 1 -- 172.21.15.38:0/4070996400 shutdown_connections 2024-09-21T03:45:26.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:26.351+0000 7f89ddba1640 1 --2- 172.21.15.38:0/4070996400 >> v2:172.21.15.195:3300/0 conn(0x7f89d80b6090 0x7f89d80b64a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:26.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:26.351+0000 7f89ddba1640 1 --2- 172.21.15.38:0/4070996400 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f89d81624f0 0x7f89d80b39a0 unknown :-1 s=CLOSED pgs=2506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:26.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:26.351+0000 7f89ddba1640 1 -- 172.21.15.38:0/4070996400 >> 172.21.15.38:0/4070996400 conn(0x7f89d809d800 msgr2=0x7f89d80b94c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:26.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:26.351+0000 7f89ddba1640 1 -- 172.21.15.38:0/4070996400 shutdown_connections 2024-09-21T03:45:26.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:26.351+0000 7f89ddba1640 1 -- 172.21.15.38:0/4070996400 wait complete. 2024-09-21T03:45:26.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:26.353+0000 7f89ddba1640 1 Processor -- start 2024-09-21T03:45:26.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:26.353+0000 7f89ddba1640 1 -- start start 2024-09-21T03:45:26.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:26.354+0000 7f89ddba1640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f89d80b6090 0x7f89d806f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:26.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:26.354+0000 7f89ddba1640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f89d80d34c0 0x7f89d80b1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:26.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:26.354+0000 7f89ddba1640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f89d81624f0 0x7f89d80b1c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:26.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:26.355+0000 7f89ddba1640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f89d80d44c0 con 0x7f89d80d34c0 2024-09-21T03:45:26.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:26.355+0000 7f89ddba1640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f89d80d3b20 con 0x7f89d81624f0 2024-09-21T03:45:26.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:26.355+0000 7f89ddba1640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f89d80d4640 con 0x7f89d80b6090 2024-09-21T03:45:26.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:26.355+0000 7f89d6ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f89d80d34c0 0x7f89d80b1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:26.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:26.355+0000 7f89d6ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f89d80d34c0 0x7f89d80b1700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:56486/0 (socket says 172.21.15.38:56486) 2024-09-21T03:45:26.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:26.355+0000 7f89d6ffd640 1 -- 172.21.15.38:0/2978850196 learned_addr learned my addr 172.21.15.38:0/2978850196 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:45:26.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:26.355+0000 7f89d7fff640 1 --2- 172.21.15.38:0/2978850196 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f89d81624f0 0x7f89d80b1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:26.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:26.355+0000 7f89d77fe640 1 --2- 172.21.15.38:0/2978850196 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f89d80b6090 0x7f89d806f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:26.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:26.355+0000 7f89d6ffd640 1 -- 172.21.15.38:0/2978850196 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f89d80b6090 msgr2=0x7f89d806f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:26.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:26.355+0000 7f89d6ffd640 1 --2- 172.21.15.38:0/2978850196 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f89d80b6090 0x7f89d806f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:26.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:26.355+0000 7f89d6ffd640 1 -- 172.21.15.38:0/2978850196 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f89d81624f0 msgr2=0x7f89d80b1c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:26.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:26.355+0000 7f89d6ffd640 1 --2- 172.21.15.38:0/2978850196 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f89d81624f0 0x7f89d80b1c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:26.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:26.355+0000 7f89d6ffd640 1 -- 172.21.15.38:0/2978850196 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89d81f9790 con 0x7f89d80d34c0 2024-09-21T03:45:26.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:26.355+0000 7f89d7fff640 1 --2- 172.21.15.38:0/2978850196 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f89d81624f0 0x7f89d80b1c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:45:26.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:26.355+0000 7f89d6ffd640 1 --2- 172.21.15.38:0/2978850196 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f89d80d34c0 0x7f89d80b1700 secure :-1 s=READY pgs=2507 cs=0 l=1 rev1=1 crypto rx=0x7f89cc00c370 tx=0x7f89cc00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:26.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:26.355+0000 7f89bffff640 1 -- 172.21.15.38:0/2978850196 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89cc017070 con 0x7f89d80d34c0 2024-09-21T03:45:26.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:26.355+0000 7f89bffff640 1 -- 172.21.15.38:0/2978850196 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f89cc0035f0 con 0x7f89d80d34c0 2024-09-21T03:45:26.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:26.355+0000 7f89ddba1640 1 -- 172.21.15.38:0/2978850196 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89d81f9a50 con 0x7f89d80d34c0 2024-09-21T03:45:26.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:26.355+0000 7f89bffff640 1 -- 172.21.15.38:0/2978850196 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89cc003a40 con 0x7f89d80d34c0 2024-09-21T03:45:26.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:26.355+0000 7f89ddba1640 1 -- 172.21.15.38:0/2978850196 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89d81fa010 con 0x7f89d80d34c0 2024-09-21T03:45:26.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:26.357+0000 7f89ddba1640 1 -- 172.21.15.38:0/2978850196 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89d80b64a0 con 0x7f89d80d34c0 2024-09-21T03:45:26.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:26.357+0000 7f89bffff640 1 -- 172.21.15.38:0/2978850196 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f89cc01a070 con 0x7f89d80d34c0 2024-09-21T03:45:26.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:26.358+0000 7f89bffff640 1 --2- 172.21.15.38:0/2978850196 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f89c4078b90 0x7f89c407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:26.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:26.358+0000 7f89bffff640 1 -- 172.21.15.38:0/2978850196 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 5951+0+0 (secure 0 0 0) 0x7f89cc09a0f0 con 0x7f89d80d34c0 2024-09-21T03:45:26.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:26.358+0000 7f89d77fe640 1 --2- 172.21.15.38:0/2978850196 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f89c4078b90 0x7f89c407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:26.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:26.359+0000 7f89d77fe640 1 --2- 172.21.15.38:0/2978850196 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f89c4078b90 0x7f89c407b0a0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f89c8034000 tx=0x7f89c8004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:26.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:26.363+0000 7f89bffff640 1 -- 172.21.15.38:0/2978850196 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f89cc062440 con 0x7f89d80d34c0 2024-09-21T03:45:26.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:26.550+0000 7f89ddba1640 1 -- 172.21.15.38:0/2978850196 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f89d8162270 con 0x7f89d80d34c0 2024-09-21T03:45:27.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:27.204+0000 7f89bffff640 1 -- 172.21.15.38:0/2978850196 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v152) ==== 169+0+0 (secure 0 0 0) 0x7f89cc061d40 con 0x7f89d80d34c0 2024-09-21T03:45:27.208 INFO:teuthology.orchestra.run.smithi038.stderr:pool 'cephfs_data' removed 2024-09-21T03:45:27.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:27.210+0000 7f89ddba1640 1 -- 172.21.15.38:0/2978850196 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f89c4078b90 msgr2=0x7f89c407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:27.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:27.210+0000 7f89ddba1640 1 --2- 172.21.15.38:0/2978850196 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f89c4078b90 0x7f89c407b0a0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f89c8034000 tx=0x7f89c8004730 comp rx=0 tx=0).stop 2024-09-21T03:45:27.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:27.210+0000 7f89ddba1640 1 -- 172.21.15.38:0/2978850196 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f89d80d34c0 msgr2=0x7f89d80b1700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:27.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:27.210+0000 7f89ddba1640 1 --2- 172.21.15.38:0/2978850196 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f89d80d34c0 0x7f89d80b1700 secure :-1 s=READY pgs=2507 cs=0 l=1 rev1=1 crypto rx=0x7f89cc00c370 tx=0x7f89cc00c830 comp rx=0 tx=0).stop 2024-09-21T03:45:27.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:27.214+0000 7f89ddba1640 1 -- 172.21.15.38:0/2978850196 shutdown_connections 2024-09-21T03:45:27.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:27.214+0000 7f89ddba1640 1 --2- 172.21.15.38:0/2978850196 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f89d81624f0 0x7f89d80b1c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:27.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:27.214+0000 7f89ddba1640 1 --2- 172.21.15.38:0/2978850196 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f89d80d34c0 0x7f89d80b1700 unknown :-1 s=CLOSED pgs=2507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:27.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:27.214+0000 7f89ddba1640 1 --2- 172.21.15.38:0/2978850196 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f89c4078b90 0x7f89c407b0a0 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:27.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:27.214+0000 7f89ddba1640 1 --2- 172.21.15.38:0/2978850196 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f89d80b6090 0x7f89d806f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:27.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:27.214+0000 7f89ddba1640 1 -- 172.21.15.38:0/2978850196 >> 172.21.15.38:0/2978850196 conn(0x7f89d809d800 msgr2=0x7f89d815cb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:27.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:27.217+0000 7f89ddba1640 1 -- 172.21.15.38:0/2978850196 shutdown_connections 2024-09-21T03:45:27.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:27.218+0000 7f89ddba1640 1 -- 172.21.15.38:0/2978850196 wait complete. 2024-09-21T03:45:27.237 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 31 2024-09-21T03:45:27.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:27.359+0000 7f0b2d32b640 1 -- 172.21.15.38:0/3872116760 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0b280ceb90 msgr2=0x7f0b280cefa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:27.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:27.359+0000 7f0b2d32b640 1 --2- 172.21.15.38:0/3872116760 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0b280ceb90 0x7f0b280cefa0 secure :-1 s=READY pgs=2508 cs=0 l=1 rev1=1 crypto rx=0x7f0b20005fd0 tx=0x7f0b2002aa10 comp rx=0 tx=0).stop 2024-09-21T03:45:27.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:27.359+0000 7f0b2d32b640 1 -- 172.21.15.38:0/3872116760 shutdown_connections 2024-09-21T03:45:27.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:27.359+0000 7f0b2d32b640 1 --2- 172.21.15.38:0/3872116760 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0b280ceb90 0x7f0b280cefa0 unknown :-1 s=CLOSED pgs=2508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:27.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:27.359+0000 7f0b2d32b640 1 --2- 172.21.15.38:0/3872116760 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0b2815ee80 0x7f0b280d0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:27.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:27.359+0000 7f0b2d32b640 1 -- 172.21.15.38:0/3872116760 >> 172.21.15.38:0/3872116760 conn(0x7f0b2809d830 msgr2=0x7f0b280b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:27.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:27.359+0000 7f0b2d32b640 1 -- 172.21.15.38:0/3872116760 shutdown_connections 2024-09-21T03:45:27.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:27.359+0000 7f0b2d32b640 1 -- 172.21.15.38:0/3872116760 wait complete. 2024-09-21T03:45:27.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:27.361+0000 7f0b2d32b640 1 Processor -- start 2024-09-21T03:45:27.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:27.361+0000 7f0b2d32b640 1 -- start start 2024-09-21T03:45:27.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:27.362+0000 7f0b2d32b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0b280b9ad0 0x7f0b28158130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:27.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:27.362+0000 7f0b2d32b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0b280ceb90 0x7f0b280cdbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:27.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:27.362+0000 7f0b2d32b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0b2815ee80 0x7f0b280ce130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:27.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:27.362+0000 7f0b2d32b640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f0b280b6700 con 0x7f0b2815ee80 2024-09-21T03:45:27.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:27.362+0000 7f0b2d32b640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f0b280b6580 con 0x7f0b280b9ad0 2024-09-21T03:45:27.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:27.362+0000 7f0b2d32b640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f0b280b6880 con 0x7f0b280ceb90 2024-09-21T03:45:27.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:27.363+0000 7f0b277fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0b2815ee80 0x7f0b280ce130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:27.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:27.363+0000 7f0b277fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0b2815ee80 0x7f0b280ce130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:56508/0 (socket says 172.21.15.38:56508) 2024-09-21T03:45:27.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:27.363+0000 7f0b277fe640 1 -- 172.21.15.38:0/1513824809 learned_addr learned my addr 172.21.15.38:0/1513824809 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:45:27.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:27.363+0000 7f0b26ffd640 1 --2- 172.21.15.38:0/1513824809 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0b280b9ad0 0x7f0b28158130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:27.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:27.363+0000 7f0b267fc640 1 --2- 172.21.15.38:0/1513824809 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0b280ceb90 0x7f0b280cdbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:27.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:27.363+0000 7f0b277fe640 1 -- 172.21.15.38:0/1513824809 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0b280ceb90 msgr2=0x7f0b280cdbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:27.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:27.363+0000 7f0b277fe640 1 --2- 172.21.15.38:0/1513824809 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0b280ceb90 0x7f0b280cdbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:27.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:27.363+0000 7f0b277fe640 1 -- 172.21.15.38:0/1513824809 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0b280b9ad0 msgr2=0x7f0b28158130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:27.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:27.363+0000 7f0b277fe640 1 --2- 172.21.15.38:0/1513824809 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0b280b9ad0 0x7f0b28158130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:27.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:27.363+0000 7f0b277fe640 1 -- 172.21.15.38:0/1513824809 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b281f5760 con 0x7f0b2815ee80 2024-09-21T03:45:27.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:27.363+0000 7f0b267fc640 1 --2- 172.21.15.38:0/1513824809 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0b280ceb90 0x7f0b280cdbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:45:27.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:27.363+0000 7f0b277fe640 1 --2- 172.21.15.38:0/1513824809 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0b2815ee80 0x7f0b280ce130 secure :-1 s=READY pgs=2509 cs=0 l=1 rev1=1 crypto rx=0x7f0b20009d90 tx=0x7f0b20003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:27.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:27.363+0000 7f0b0f7fe640 1 -- 172.21.15.38:0/1513824809 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b20033050 con 0x7f0b2815ee80 2024-09-21T03:45:27.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:27.363+0000 7f0b0f7fe640 1 -- 172.21.15.38:0/1513824809 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0b20004250 con 0x7f0b2815ee80 2024-09-21T03:45:27.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:27.363+0000 7f0b0f7fe640 1 -- 172.21.15.38:0/1513824809 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b20004680 con 0x7f0b2815ee80 2024-09-21T03:45:27.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:27.363+0000 7f0b2d32b640 1 -- 172.21.15.38:0/1513824809 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b281f5a40 con 0x7f0b2815ee80 2024-09-21T03:45:27.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:27.363+0000 7f0b2d32b640 1 -- 172.21.15.38:0/1513824809 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b281f6000 con 0x7f0b2815ee80 2024-09-21T03:45:27.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:27.364+0000 7f0b2d32b640 1 -- 172.21.15.38:0/1513824809 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b280b6da0 con 0x7f0b2815ee80 2024-09-21T03:45:27.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:27.365+0000 7f0b0f7fe640 1 -- 172.21.15.38:0/1513824809 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f0b20036070 con 0x7f0b2815ee80 2024-09-21T03:45:27.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:27.366+0000 7f0b0f7fe640 1 --2- 172.21.15.38:0/1513824809 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0b14078b90 0x7f0b1407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:27.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:27.366+0000 7f0b0f7fe640 1 -- 172.21.15.38:0/1513824809 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 5421+0+0 (secure 0 0 0) 0x7f0b200bc340 con 0x7f0b2815ee80 2024-09-21T03:45:27.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:27.366+0000 7f0b26ffd640 1 --2- 172.21.15.38:0/1513824809 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0b14078b90 0x7f0b1407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:27.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:27.367+0000 7f0b26ffd640 1 --2- 172.21.15.38:0/1513824809 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0b14078b90 0x7f0b1407b0a0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f0b18005fd0 tx=0x7f0b18005850 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:27.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:27.370+0000 7f0b0f7fe640 1 -- 172.21.15.38:0/1513824809 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f0b20084960 con 0x7f0b2815ee80 2024-09-21T03:45:27.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:27.557+0000 7f0b2d32b640 1 -- 172.21.15.38:0/1513824809 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "config reset", "num": 31} v 0) -- 0x7f0b280b3e30 con 0x7f0b2815ee80 2024-09-21T03:45:27.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:27.561+0000 7f0b0f7fe640 1 -- 172.21.15.38:0/1513824809 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 31}]=0 v50) ==== 71+0+0 (secure 0 0 0) 0x7f0b20004820 con 0x7f0b2815ee80 2024-09-21T03:45:27.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:27.563+0000 7f0b2d32b640 1 -- 172.21.15.38:0/1513824809 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0b14078b90 msgr2=0x7f0b1407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:27.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:27.563+0000 7f0b2d32b640 1 --2- 172.21.15.38:0/1513824809 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0b14078b90 0x7f0b1407b0a0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f0b18005fd0 tx=0x7f0b18005850 comp rx=0 tx=0).stop 2024-09-21T03:45:27.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:27.563+0000 7f0b2d32b640 1 -- 172.21.15.38:0/1513824809 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0b2815ee80 msgr2=0x7f0b280ce130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:27.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:27.563+0000 7f0b2d32b640 1 --2- 172.21.15.38:0/1513824809 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0b2815ee80 0x7f0b280ce130 secure :-1 s=READY pgs=2509 cs=0 l=1 rev1=1 crypto rx=0x7f0b20009d90 tx=0x7f0b20003f10 comp rx=0 tx=0).stop 2024-09-21T03:45:27.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:27.563+0000 7f0b2d32b640 1 -- 172.21.15.38:0/1513824809 shutdown_connections 2024-09-21T03:45:27.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:27.563+0000 7f0b2d32b640 1 --2- 172.21.15.38:0/1513824809 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0b2815ee80 0x7f0b280ce130 unknown :-1 s=CLOSED pgs=2509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:27.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:27.563+0000 7f0b2d32b640 1 --2- 172.21.15.38:0/1513824809 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0b14078b90 0x7f0b1407b0a0 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:27.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:27.563+0000 7f0b2d32b640 1 --2- 172.21.15.38:0/1513824809 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0b280ceb90 0x7f0b280cdbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:27.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:27.563+0000 7f0b2d32b640 1 --2- 172.21.15.38:0/1513824809 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0b280b9ad0 0x7f0b28158130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:27.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:27.563+0000 7f0b2d32b640 1 -- 172.21.15.38:0/1513824809 >> 172.21.15.38:0/1513824809 conn(0x7f0b2809d830 msgr2=0x7f0b280b44a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:27.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:27.563+0000 7f0b2d32b640 1 -- 172.21.15.38:0/1513824809 shutdown_connections 2024-09-21T03:45:27.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:27.563+0000 7f0b2d32b640 1 -- 172.21.15.38:0/1513824809 wait complete. 2024-09-21T03:45:27.577 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_damage.TestDamage.test_open_ino_errors' 2024-09-21T03:45:27.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:27.696+0000 7f9ef235f640 1 -- 172.21.15.38:0/319867126 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9eec0cf210 msgr2=0x7f9eec0cf620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:27.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:27.696+0000 7f9ef235f640 1 --2- 172.21.15.38:0/319867126 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9eec0cf210 0x7f9eec0cf620 secure :-1 s=READY pgs=2510 cs=0 l=1 rev1=1 crypto rx=0x7f9eec08c970 tx=0x7f9ee402aa30 comp rx=0 tx=0).stop 2024-09-21T03:45:27.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:27.696+0000 7f9ef235f640 1 -- 172.21.15.38:0/319867126 shutdown_connections 2024-09-21T03:45:27.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:27.696+0000 7f9ef235f640 1 --2- 172.21.15.38:0/319867126 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9eec0cf210 0x7f9eec0cf620 unknown :-1 s=CLOSED pgs=2510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:27.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:27.696+0000 7f9ef235f640 1 --2- 172.21.15.38:0/319867126 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9eec0d0d10 0x7f9eec0d1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:27.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:27.696+0000 7f9ef235f640 1 --2- 172.21.15.38:0/319867126 >> v2:172.21.15.195:3300/0 conn(0x7f9eec162460 0x7f9eec0b9880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:27.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:27.696+0000 7f9ef235f640 1 -- 172.21.15.38:0/319867126 >> 172.21.15.38:0/319867126 conn(0x7f9eec09d7e0 msgr2=0x7f9eec0b74a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:27.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:27.696+0000 7f9ef235f640 1 -- 172.21.15.38:0/319867126 shutdown_connections 2024-09-21T03:45:27.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:27.696+0000 7f9ef235f640 1 -- 172.21.15.38:0/319867126 wait complete. 2024-09-21T03:45:27.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:27.697+0000 7f9ef235f640 1 Processor -- start 2024-09-21T03:45:27.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:27.698+0000 7f9ef235f640 1 -- start start 2024-09-21T03:45:27.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:27.699+0000 7f9ef235f640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9eec0cf210 0x7f9eec15cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:27.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:27.699+0000 7f9ef235f640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9eec0d0d10 0x7f9eec0c8b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:27.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:27.699+0000 7f9ef235f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9eec162460 0x7f9eec0c90a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:27.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:27.699+0000 7f9ef235f640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f9eec162a10 con 0x7f9eec162460 2024-09-21T03:45:27.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:27.699+0000 7f9ef235f640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f9eec0b9fd0 con 0x7f9eec0d0d10 2024-09-21T03:45:27.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:27.699+0000 7f9ef235f640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f9eec15e7c0 con 0x7f9eec0cf210 2024-09-21T03:45:27.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:27.699+0000 7f9ef08fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9eec162460 0x7f9eec0c90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:27.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:27.699+0000 7f9ef08fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9eec162460 0x7f9eec0c90a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:56528/0 (socket says 172.21.15.38:56528) 2024-09-21T03:45:27.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:27.699+0000 7f9ef08fc640 1 -- 172.21.15.38:0/2673136264 learned_addr learned my addr 172.21.15.38:0/2673136264 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:45:27.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:27.699+0000 7f9ee3fff640 1 --2- 172.21.15.38:0/2673136264 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9eec0d0d10 0x7f9eec0c8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:27.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:27.699+0000 7f9eebfff640 1 --2- 172.21.15.38:0/2673136264 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9eec0cf210 0x7f9eec15cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:27.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:27.699+0000 7f9ef08fc640 1 -- 172.21.15.38:0/2673136264 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9eec0cf210 msgr2=0x7f9eec15cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:27.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:27.699+0000 7f9ef08fc640 1 --2- 172.21.15.38:0/2673136264 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9eec0cf210 0x7f9eec15cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:27.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:27.699+0000 7f9ef08fc640 1 -- 172.21.15.38:0/2673136264 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9eec0d0d10 msgr2=0x7f9eec0c8b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:27.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:27.699+0000 7f9ef08fc640 1 --2- 172.21.15.38:0/2673136264 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9eec0d0d10 0x7f9eec0c8b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:27.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:27.699+0000 7f9ef08fc640 1 -- 172.21.15.38:0/2673136264 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9eec0c7da0 con 0x7f9eec162460 2024-09-21T03:45:27.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:27.700+0000 7f9ef08fc640 1 --2- 172.21.15.38:0/2673136264 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9eec162460 0x7f9eec0c90a0 secure :-1 s=READY pgs=2511 cs=0 l=1 rev1=1 crypto rx=0x7f9ee4009d60 tx=0x7f9ee4003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:27.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:27.700+0000 7f9ee97fa640 1 -- 172.21.15.38:0/2673136264 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ee4033050 con 0x7f9eec162460 2024-09-21T03:45:27.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:27.700+0000 7f9ee97fa640 1 -- 172.21.15.38:0/2673136264 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9ee4004450 con 0x7f9eec162460 2024-09-21T03:45:27.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:27.700+0000 7f9ef235f640 1 -- 172.21.15.38:0/2673136264 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9eec0c8080 con 0x7f9eec162460 2024-09-21T03:45:27.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:27.700+0000 7f9ee97fa640 1 -- 172.21.15.38:0/2673136264 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ee4038450 con 0x7f9eec162460 2024-09-21T03:45:27.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:27.700+0000 7f9ef235f640 1 -- 172.21.15.38:0/2673136264 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9eec0c8640 con 0x7f9eec162460 2024-09-21T03:45:27.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:27.701+0000 7f9ef235f640 1 -- 172.21.15.38:0/2673136264 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9eec0cb180 con 0x7f9eec162460 2024-09-21T03:45:27.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:27.702+0000 7f9ee97fa640 1 -- 172.21.15.38:0/2673136264 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f9ee4036070 con 0x7f9eec162460 2024-09-21T03:45:27.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:27.702+0000 7f9ee97fa640 1 --2- 172.21.15.38:0/2673136264 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9ecc078bc0 0x7f9ecc07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:27.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:27.702+0000 7f9ee97fa640 1 -- 172.21.15.38:0/2673136264 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 5421+0+0 (secure 0 0 0) 0x7f9ee40bc170 con 0x7f9eec162460 2024-09-21T03:45:27.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:27.702+0000 7f9eebfff640 1 --2- 172.21.15.38:0/2673136264 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9ecc078bc0 0x7f9ecc07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:27.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:27.703+0000 7f9eebfff640 1 --2- 172.21.15.38:0/2673136264 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9ecc078bc0 0x7f9ecc07b0d0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f9edc0058c0 tx=0x7f9edc005850 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:27.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:27.707+0000 7f9ee97fa640 1 -- 172.21.15.38:0/2673136264 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f9ee40846d0 con 0x7f9eec162460 2024-09-21T03:45:27.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:27.967+0000 7f9ef235f640 1 -- 172.21.15.38:0/2673136264 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_damage.TestDamage.test_open_ino_errors"]} v 0) -- 0x7f9eec0b8010 con 0x7f9eec162460 2024-09-21T03:45:28.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:28.211+0000 7f9ee97fa640 1 -- 172.21.15.38:0/2673136264 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_damage.TestDamage.test_open_ino_errors"]}]=0 v620) ==== 135+0+0 (secure 0 0 0) 0x7f9eec0b8010 con 0x7f9eec162460 2024-09-21T03:45:28.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:28.214+0000 7f9ef235f640 1 -- 172.21.15.38:0/2673136264 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9ecc078bc0 msgr2=0x7f9ecc07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:28.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:28.214+0000 7f9ef235f640 1 --2- 172.21.15.38:0/2673136264 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9ecc078bc0 0x7f9ecc07b0d0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f9edc0058c0 tx=0x7f9edc005850 comp rx=0 tx=0).stop 2024-09-21T03:45:28.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:28.214+0000 7f9ef235f640 1 -- 172.21.15.38:0/2673136264 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9eec162460 msgr2=0x7f9eec0c90a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:28.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:28.214+0000 7f9ef235f640 1 --2- 172.21.15.38:0/2673136264 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9eec162460 0x7f9eec0c90a0 secure :-1 s=READY pgs=2511 cs=0 l=1 rev1=1 crypto rx=0x7f9ee4009d60 tx=0x7f9ee4003f10 comp rx=0 tx=0).stop 2024-09-21T03:45:28.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:28.214+0000 7f9ef235f640 1 -- 172.21.15.38:0/2673136264 shutdown_connections 2024-09-21T03:45:28.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:28.214+0000 7f9ef235f640 1 --2- 172.21.15.38:0/2673136264 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9eec162460 0x7f9eec0c90a0 unknown :-1 s=CLOSED pgs=2511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:28.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:28.214+0000 7f9ef235f640 1 --2- 172.21.15.38:0/2673136264 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9eec0d0d10 0x7f9eec0c8b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:28.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:28.214+0000 7f9ef235f640 1 --2- 172.21.15.38:0/2673136264 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f9ecc078bc0 0x7f9ecc07b0d0 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:28.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:28.214+0000 7f9ef235f640 1 --2- 172.21.15.38:0/2673136264 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9eec0cf210 0x7f9eec15cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:28.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:28.214+0000 7f9ef235f640 1 -- 172.21.15.38:0/2673136264 >> 172.21.15.38:0/2673136264 conn(0x7f9eec09d7e0 msgr2=0x7f9eec068d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:28.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:28.214+0000 7f9ef235f640 1 -- 172.21.15.38:0/2673136264 shutdown_connections 2024-09-21T03:45:28.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:28.214+0000 7f9ef235f640 1 -- 172.21.15.38:0/2673136264 wait complete. 2024-09-21T03:45:28.228 INFO:tasks.cephfs_test_runner:That errors encountered during opening inos are properly propagated ... ok 2024-09-21T03:45:28.228 INFO:tasks.cephfs_test_runner: 2024-09-21T03:45:28.229 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-09-21T03:45:28.229 INFO:tasks.cephfs_test_runner:Ran 6 tests in 673.353s 2024-09-21T03:45:28.229 INFO:tasks.cephfs_test_runner: 2024-09-21T03:45:28.229 INFO:tasks.cephfs_test_runner:OK 2024-09-21T03:45:28.229 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-09-21T03:45:28.238 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-09-21T03:45:28.259 INFO:tasks.kclient:Unmounting kernel clients... 2024-09-21T03:45:28.259 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-21T03:45:28.259 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:45:28.276 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056795,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2354 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T03:45:28.276 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-21T03:45:28.276 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:45:28.333 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056795,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2354 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T03:45:28.333 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-09-21T03:45:28.333 DEBUG:teuthology.orchestra.run.smithi195:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:45:28.353 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056014,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497444k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=8978 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248720k,nr_inodes=812180,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497444k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T03:45:28.353 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-09-21T03:45:28.353 DEBUG:teuthology.orchestra.run.smithi195:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:45:28.412 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056014,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497444k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=8978 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248720k,nr_inodes=812180,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497444k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T03:45:28.412 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:45:28.412 DEBUG:teuthology.orchestra.run.smithi038:> ip netns list 2024-09-21T03:45:28.428 INFO:teuthology.orchestra.run.smithi038.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-21T03:45:28.429 INFO:teuthology.orchestra.run.smithi038.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-21T03:45:28.429 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:45:28.429 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T03:45:28.495 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:45:28.495 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T03:45:28.563 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:45:28.563 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip link delete ceph-brx 2024-09-21T03:45:28.675 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:45:28.675 DEBUG:teuthology.orchestra.run.smithi038:> ip netns list 2024-09-21T03:45:28.692 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:45:28.692 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip link delete ceph-brx 2024-09-21T03:45:28.761 INFO:teuthology.orchestra.run.smithi038.stderr:Cannot find device "ceph-brx" 2024-09-21T03:45:28.762 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T03:45:28.763 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:45:28.763 DEBUG:teuthology.orchestra.run.smithi195:> ip netns list 2024-09-21T03:45:28.783 INFO:teuthology.orchestra.run.smithi195.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-21T03:45:28.784 INFO:teuthology.orchestra.run.smithi195.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-21T03:45:28.784 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:45:28.784 DEBUG:teuthology.orchestra.run.smithi195:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-21T03:45:28.853 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:45:28.854 DEBUG:teuthology.orchestra.run.smithi195:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-21T03:45:28.922 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:45:28.922 DEBUG:teuthology.orchestra.run.smithi195:> sudo ip link delete ceph-brx 2024-09-21T03:45:29.033 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:45:29.033 DEBUG:teuthology.orchestra.run.smithi195:> ip netns list 2024-09-21T03:45:29.050 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:45:29.050 DEBUG:teuthology.orchestra.run.smithi195:> sudo ip link delete ceph-brx 2024-09-21T03:45:29.120 INFO:teuthology.orchestra.run.smithi195.stderr:Cannot find device "ceph-brx" 2024-09-21T03:45:29.121 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T03:45:29.121 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-09-21T03:45:29.131 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-21T03:45:29.131 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T03:45:29.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:29.251+0000 7fb64222e640 1 -- 172.21.15.38:0/529638688 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb63c15b9f0 msgr2=0x7fb63c0cf930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:29.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:29.251+0000 7fb64222e640 1 --2- 172.21.15.38:0/529638688 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb63c15b9f0 0x7fb63c0cf930 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fb62c009830 tx=0x7fb62c02a510 comp rx=0 tx=0).stop 2024-09-21T03:45:29.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:29.252+0000 7fb64222e640 1 -- 172.21.15.38:0/529638688 shutdown_connections 2024-09-21T03:45:29.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:29.252+0000 7fb64222e640 1 --2- 172.21.15.38:0/529638688 >> v2:172.21.15.195:3300/0 conn(0x7fb63c0cefa0 0x7fb63c0ce320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:29.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:29.252+0000 7fb64222e640 1 --2- 172.21.15.38:0/529638688 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb63c15b9f0 0x7fb63c0cf930 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:29.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:29.252+0000 7fb64222e640 1 -- 172.21.15.38:0/529638688 >> 172.21.15.38:0/529638688 conn(0x7fb63c0cca40 msgr2=0x7fb63c0cce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:29.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:29.252+0000 7fb64222e640 1 -- 172.21.15.38:0/529638688 shutdown_connections 2024-09-21T03:45:29.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:29.252+0000 7fb64222e640 1 -- 172.21.15.38:0/529638688 wait complete. 2024-09-21T03:45:29.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:29.253+0000 7fb64222e640 1 Processor -- start 2024-09-21T03:45:29.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:29.254+0000 7fb64222e640 1 -- start start 2024-09-21T03:45:29.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:29.255+0000 7fb64222e640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb63c0ce860 0x7fb63c15abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:29.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:29.255+0000 7fb64222e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb63c0cefa0 0x7fb63c0c2e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:29.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:29.255+0000 7fb64222e640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb63c15b9f0 0x7fb63c1f7050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:29.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:29.255+0000 7fb64222e640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fb63c06df70 con 0x7fb63c0cefa0 2024-09-21T03:45:29.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:29.255+0000 7fb64222e640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fb63c06ddf0 con 0x7fb63c0ce860 2024-09-21T03:45:29.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:29.255+0000 7fb64222e640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fb63c06e0f0 con 0x7fb63c15b9f0 2024-09-21T03:45:29.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:29.255+0000 7fb63affd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb63c0cefa0 0x7fb63c0c2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:29.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:29.255+0000 7fb63affd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb63c0cefa0 0x7fb63c0c2e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:56534/0 (socket says 172.21.15.38:56534) 2024-09-21T03:45:29.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:29.255+0000 7fb63affd640 1 -- 172.21.15.38:0/4111830265 learned_addr learned my addr 172.21.15.38:0/4111830265 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:45:29.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:29.255+0000 7fb63b7fe640 1 --2- 172.21.15.38:0/4111830265 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb63c0ce860 0x7fb63c15abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:29.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:29.255+0000 7fb63bfff640 1 --2- 172.21.15.38:0/4111830265 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb63c15b9f0 0x7fb63c1f7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:29.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:29.255+0000 7fb63affd640 1 -- 172.21.15.38:0/4111830265 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb63c15b9f0 msgr2=0x7fb63c1f7050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:29.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:29.255+0000 7fb63affd640 1 --2- 172.21.15.38:0/4111830265 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb63c15b9f0 0x7fb63c1f7050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:29.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:29.255+0000 7fb63affd640 1 -- 172.21.15.38:0/4111830265 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb63c0ce860 msgr2=0x7fb63c15abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:29.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:29.255+0000 7fb63affd640 1 --2- 172.21.15.38:0/4111830265 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb63c0ce860 0x7fb63c15abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:29.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:29.255+0000 7fb63affd640 1 -- 172.21.15.38:0/4111830265 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb63c09aa80 con 0x7fb63c0cefa0 2024-09-21T03:45:29.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:29.256+0000 7fb63affd640 1 --2- 172.21.15.38:0/4111830265 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb63c0cefa0 0x7fb63c0c2e20 secure :-1 s=READY pgs=2512 cs=0 l=1 rev1=1 crypto rx=0x7fb63000ea50 tx=0x7fb63000ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:29.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:29.256+0000 7fb623fff640 1 -- 172.21.15.38:0/4111830265 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb63000cd30 con 0x7fb63c0cefa0 2024-09-21T03:45:29.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:29.256+0000 7fb623fff640 1 -- 172.21.15.38:0/4111830265 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb630004020 con 0x7fb63c0cefa0 2024-09-21T03:45:29.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:29.256+0000 7fb623fff640 1 -- 172.21.15.38:0/4111830265 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb630004470 con 0x7fb63c0cefa0 2024-09-21T03:45:29.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:29.256+0000 7fb64222e640 1 -- 172.21.15.38:0/4111830265 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb63c09ad60 con 0x7fb63c0cefa0 2024-09-21T03:45:29.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:29.256+0000 7fb64222e640 1 -- 172.21.15.38:0/4111830265 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb63c09b320 con 0x7fb63c0cefa0 2024-09-21T03:45:29.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:29.257+0000 7fb64222e640 1 -- 172.21.15.38:0/4111830265 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb63c0b5cf0 con 0x7fb63c0cefa0 2024-09-21T03:45:29.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:29.258+0000 7fb623fff640 1 -- 172.21.15.38:0/4111830265 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb6300035f0 con 0x7fb63c0cefa0 2024-09-21T03:45:29.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:29.258+0000 7fb623fff640 1 --2- 172.21.15.38:0/4111830265 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb628078bc0 0x7fb62807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:29.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:29.258+0000 7fb623fff640 1 -- 172.21.15.38:0/4111830265 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 5421+0+0 (secure 0 0 0) 0x7fb63009b3d0 con 0x7fb63c0cefa0 2024-09-21T03:45:29.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:29.258+0000 7fb63b7fe640 1 --2- 172.21.15.38:0/4111830265 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb628078bc0 0x7fb62807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:29.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:29.259+0000 7fb63b7fe640 1 --2- 172.21.15.38:0/4111830265 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb628078bc0 0x7fb62807b0d0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fb62c034000 tx=0x7fb62c004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:29.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:29.263+0000 7fb623fff640 1 -- 172.21.15.38:0/4111830265 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fb63c0b5cf0 con 0x7fb63c0cefa0 2024-09-21T03:45:29.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:29.447+0000 7fb64222e640 1 -- 172.21.15.38:0/4111830265 --> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb63c09cb60 con 0x7fb628078bc0 2024-09-21T03:45:29.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:29.448+0000 7fb623fff640 1 -- 172.21.15.38:0/4111830265 <== mgr.4117 v2:172.21.15.195:6800/1631901344 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13472 (secure 0 0 0) 0x7fb63c09cb60 con 0x7fb628078bc0 2024-09-21T03:45:29.450 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:45:29.450 INFO:teuthology.orchestra.run.smithi038.stderr:dumped all 2024-09-21T03:45:29.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:29.450+0000 7fb64222e640 1 -- 172.21.15.38:0/4111830265 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb628078bc0 msgr2=0x7fb62807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:29.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:29.450+0000 7fb64222e640 1 --2- 172.21.15.38:0/4111830265 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb628078bc0 0x7fb62807b0d0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fb62c034000 tx=0x7fb62c004730 comp rx=0 tx=0).stop 2024-09-21T03:45:29.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:29.450+0000 7fb64222e640 1 -- 172.21.15.38:0/4111830265 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb63c0cefa0 msgr2=0x7fb63c0c2e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:29.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:29.450+0000 7fb64222e640 1 --2- 172.21.15.38:0/4111830265 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb63c0cefa0 0x7fb63c0c2e20 secure :-1 s=READY pgs=2512 cs=0 l=1 rev1=1 crypto rx=0x7fb63000ea50 tx=0x7fb63000ef10 comp rx=0 tx=0).stop 2024-09-21T03:45:29.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:29.450+0000 7fb64222e640 1 -- 172.21.15.38:0/4111830265 shutdown_connections 2024-09-21T03:45:29.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:29.450+0000 7fb64222e640 1 --2- 172.21.15.38:0/4111830265 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb628078bc0 0x7fb62807b0d0 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:29.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:29.450+0000 7fb64222e640 1 --2- 172.21.15.38:0/4111830265 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb63c15b9f0 0x7fb63c1f7050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:29.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:29.450+0000 7fb64222e640 1 --2- 172.21.15.38:0/4111830265 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb63c0cefa0 0x7fb63c0c2e20 unknown :-1 s=CLOSED pgs=2512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:29.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:29.450+0000 7fb64222e640 1 --2- 172.21.15.38:0/4111830265 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb63c0ce860 0x7fb63c15abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:29.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:29.450+0000 7fb64222e640 1 -- 172.21.15.38:0/4111830265 >> 172.21.15.38:0/4111830265 conn(0x7fb63c0cca40 msgr2=0x7fb63c0b4e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:29.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:29.450+0000 7fb64222e640 1 -- 172.21.15.38:0/4111830265 shutdown_connections 2024-09-21T03:45:29.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:29.450+0000 7fb64222e640 1 -- 172.21.15.38:0/4111830265 wait complete. 2024-09-21T03:45:29.464 INFO:teuthology.orchestra.run.smithi038.stdout:{"pg_ready":true,"pg_map":{"version":508,"stamp":"2024-09-21T03:45:29.180061+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":116,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":307572,"kb_used_data":16456,"kb_used_omap":12,"kb_used_meta":290931,"kb_avail":754667148,"statfs":{"total":773094113280,"available":772779159552,"internally_reserved":0,"allocated":16850944,"data_stored":9107036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12748,"internal_metadata":297913908},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-36145,"num_objects":-27,"num_object_clones":0,"num_object_copies":-54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-27,"num_whiteouts":0,"num_read":-90,"num_read_kb":-92,"num_write":-97,"num_write_kb":-70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-16,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001766"},"pg_stats":[{"pgid":"1.0","version":"17'72","reported_seq":362,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-21T03:45:18.161661+0000","last_change":"2024-09-21T03:33:25.319776+0000","last_active":"2024-09-21T03:45:18.161661+0000","last_peered":"2024-09-21T03:45:18.161661+0000","last_clean":"2024-09-21T03:45:18.161661+0000","last_became_active":"2024-09-21T03:33:25.318399+0000","last_became_peered":"2024-09-21T03:33:25.318399+0000","last_unstale":"2024-09-21T03:45:18.161661+0000","last_undegraded":"2024-09-21T03:45:18.161661+0000","last_fullsized":"2024-09-21T03:45:18.161661+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:23.016672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:23.016672+0000","last_clean_scrub_stamp":"2024-09-21T03:33:23.016672+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:32:35.906114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":1,"up_from":16,"seq":68719476884,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41980,"kb_used_data":2468,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94329860,"statfs":{"total":96636764160,"available":96593776640,"internally_reserved":0,"allocated":2527232,"data_stored":1566193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509589,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":41436,"kb_used_data":1920,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330404,"statfs":{"total":96636764160,"available":96594333696,"internally_reserved":0,"allocated":1966080,"data_stored":994671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":40434123},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509588,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":41428,"kb_used_data":1920,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330412,"statfs":{"total":96636764160,"available":96594341888,"internally_reserved":0,"allocated":1966080,"data_stored":994671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509589,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37268,"kb_used_data":1920,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334572,"statfs":{"total":96636764160,"available":96598601728,"internally_reserved":0,"allocated":1966080,"data_stored":994671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509588,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37268,"kb_used_data":1920,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334572,"statfs":{"total":96636764160,"available":96598601728,"internally_reserved":0,"allocated":1966080,"data_stored":994671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542292,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33108,"kb_used_data":1920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338732,"statfs":{"total":96636764160,"available":96602861568,"internally_reserved":0,"allocated":1966080,"data_stored":994671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574996,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37832,"kb_used_data":2484,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334008,"statfs":{"total":96636764160,"available":96598024192,"internally_reserved":0,"allocated":2543616,"data_stored":1578928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542293,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37252,"kb_used_data":1904,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334588,"statfs":{"total":96636764160,"available":96598618112,"internally_reserved":0,"allocated":1949696,"data_stored":988560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T03:45:29.464 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T03:45:29.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:29.578+0000 7f18f97c3640 1 -- 172.21.15.38:0/1542613340 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f18f415e800 msgr2=0x7f18f415ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:29.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:29.578+0000 7f18f97c3640 1 --2- 172.21.15.38:0/1542613340 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f18f415e800 0x7f18f415ec10 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f18ec005fd0 tx=0x7f18ec02a8b0 comp rx=0 tx=0).stop 2024-09-21T03:45:29.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:29.579+0000 7f18f97c3640 1 -- 172.21.15.38:0/1542613340 shutdown_connections 2024-09-21T03:45:29.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:29.579+0000 7f18f97c3640 1 --2- 172.21.15.38:0/1542613340 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f18f415e800 0x7f18f415ec10 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:29.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:29.579+0000 7f18f97c3640 1 --2- 172.21.15.38:0/1542613340 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f18f4073e20 0x7f18f4074230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:29.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:29.579+0000 7f18f97c3640 1 -- 172.21.15.38:0/1542613340 >> 172.21.15.38:0/1542613340 conn(0x7f18f40cca40 msgr2=0x7f18f40cce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:29.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:29.579+0000 7f18f97c3640 1 -- 172.21.15.38:0/1542613340 shutdown_connections 2024-09-21T03:45:29.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:29.579+0000 7f18f97c3640 1 -- 172.21.15.38:0/1542613340 wait complete. 2024-09-21T03:45:29.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:29.580+0000 7f18f97c3640 1 Processor -- start 2024-09-21T03:45:29.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:29.581+0000 7f18f97c3640 1 -- start start 2024-09-21T03:45:29.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:29.582+0000 7f18f97c3640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f18f4073e20 0x7f18f409ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:29.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:29.582+0000 7f18f97c3640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f18f40ca610 0x7f18f41f5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:29.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:29.582+0000 7f18f97c3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f18f415e800 0x7f18f40b5310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:29.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:29.582+0000 7f18f97c3640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f18f415f2d0 con 0x7f18f415e800 2024-09-21T03:45:29.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:29.582+0000 7f18f97c3640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f18f415f150 con 0x7f18f40ca610 2024-09-21T03:45:29.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:29.582+0000 7f18f97c3640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f18f415f450 con 0x7f18f4073e20 2024-09-21T03:45:29.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:29.582+0000 7f18f37fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f18f415e800 0x7f18f40b5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:29.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:29.582+0000 7f18f37fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f18f415e800 0x7f18f40b5310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:56540/0 (socket says 172.21.15.38:56540) 2024-09-21T03:45:29.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:29.582+0000 7f18f37fe640 1 -- 172.21.15.38:0/2656443969 learned_addr learned my addr 172.21.15.38:0/2656443969 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:45:29.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:29.582+0000 7f18ea7fc640 1 --2- 172.21.15.38:0/2656443969 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f18f40ca610 0x7f18f41f5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:29.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:29.582+0000 7f18f2ffd640 1 --2- 172.21.15.38:0/2656443969 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f18f4073e20 0x7f18f409ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:29.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:29.583+0000 7f18f37fe640 1 -- 172.21.15.38:0/2656443969 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f18f4073e20 msgr2=0x7f18f409ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:29.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:29.583+0000 7f18f37fe640 1 --2- 172.21.15.38:0/2656443969 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f18f4073e20 0x7f18f409ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:29.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:29.583+0000 7f18f37fe640 1 -- 172.21.15.38:0/2656443969 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f18f40ca610 msgr2=0x7f18f41f5550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:29.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:29.583+0000 7f18f37fe640 1 --2- 172.21.15.38:0/2656443969 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f18f40ca610 0x7f18f41f5550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:29.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:29.583+0000 7f18f37fe640 1 -- 172.21.15.38:0/2656443969 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18f40b4f10 con 0x7f18f415e800 2024-09-21T03:45:29.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:29.583+0000 7f18ea7fc640 1 --2- 172.21.15.38:0/2656443969 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f18f40ca610 0x7f18f41f5550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:45:29.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:29.583+0000 7f18f2ffd640 1 --2- 172.21.15.38:0/2656443969 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f18f4073e20 0x7f18f409ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:45:29.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:29.583+0000 7f18f37fe640 1 --2- 172.21.15.38:0/2656443969 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f18f415e800 0x7f18f40b5310 secure :-1 s=READY pgs=2514 cs=0 l=1 rev1=1 crypto rx=0x7f18ec02aea0 tx=0x7f18ec0316f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:29.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:29.583+0000 7f18ebfff640 1 -- 172.21.15.38:0/2656443969 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18ec031da0 con 0x7f18f415e800 2024-09-21T03:45:29.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:29.583+0000 7f18ebfff640 1 -- 172.21.15.38:0/2656443969 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f18ec003a60 con 0x7f18f415e800 2024-09-21T03:45:29.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:29.583+0000 7f18f97c3640 1 -- 172.21.15.38:0/2656443969 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18f409e4b0 con 0x7f18f415e800 2024-09-21T03:45:29.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:29.583+0000 7f18ebfff640 1 -- 172.21.15.38:0/2656443969 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18ec033070 con 0x7f18f415e800 2024-09-21T03:45:29.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:29.583+0000 7f18f97c3640 1 -- 172.21.15.38:0/2656443969 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18f40b4c30 con 0x7f18f415e800 2024-09-21T03:45:29.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:29.584+0000 7f18f97c3640 1 -- 172.21.15.38:0/2656443969 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18f415ec10 con 0x7f18f415e800 2024-09-21T03:45:29.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:29.585+0000 7f18ebfff640 1 -- 172.21.15.38:0/2656443969 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f18ec033210 con 0x7f18f415e800 2024-09-21T03:45:29.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:29.586+0000 7f18ebfff640 1 --2- 172.21.15.38:0/2656443969 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f18d4078bc0 0x7f18d407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:29.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:29.586+0000 7f18ebfff640 1 -- 172.21.15.38:0/2656443969 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 5421+0+0 (secure 0 0 0) 0x7f18ec0bbff0 con 0x7f18f415e800 2024-09-21T03:45:29.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:29.586+0000 7f18f2ffd640 1 --2- 172.21.15.38:0/2656443969 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f18d4078bc0 0x7f18d407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:29.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:29.587+0000 7f18f2ffd640 1 --2- 172.21.15.38:0/2656443969 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f18d4078bc0 0x7f18d407b0d0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f18e4007a50 tx=0x7f18e4009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:29.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:29.590+0000 7f18ebfff640 1 -- 172.21.15.38:0/2656443969 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f18ec03f030 con 0x7f18f415e800 2024-09-21T03:45:29.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:29.773+0000 7f18f97c3640 1 -- 172.21.15.38:0/2656443969 --> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f18f40c7ff0 con 0x7f18d4078bc0 2024-09-21T03:45:29.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:29.774+0000 7f18ebfff640 1 -- 172.21.15.38:0/2656443969 <== mgr.4117 v2:172.21.15.195:6800/1631901344 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13472 (secure 0 0 0) 0x7f18f40c7ff0 con 0x7f18d4078bc0 2024-09-21T03:45:29.777 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:45:29.777 INFO:teuthology.orchestra.run.smithi038.stderr:dumped all 2024-09-21T03:45:29.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:29.777+0000 7f18f97c3640 1 -- 172.21.15.38:0/2656443969 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f18d4078bc0 msgr2=0x7f18d407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:29.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:29.777+0000 7f18f97c3640 1 --2- 172.21.15.38:0/2656443969 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f18d4078bc0 0x7f18d407b0d0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f18e4007a50 tx=0x7f18e4009040 comp rx=0 tx=0).stop 2024-09-21T03:45:29.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:29.777+0000 7f18f97c3640 1 -- 172.21.15.38:0/2656443969 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f18f415e800 msgr2=0x7f18f40b5310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:29.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:29.777+0000 7f18f97c3640 1 --2- 172.21.15.38:0/2656443969 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f18f415e800 0x7f18f40b5310 secure :-1 s=READY pgs=2514 cs=0 l=1 rev1=1 crypto rx=0x7f18ec02aea0 tx=0x7f18ec0316f0 comp rx=0 tx=0).stop 2024-09-21T03:45:29.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:29.777+0000 7f18f97c3640 1 -- 172.21.15.38:0/2656443969 shutdown_connections 2024-09-21T03:45:29.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:29.777+0000 7f18f97c3640 1 --2- 172.21.15.38:0/2656443969 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f18f415e800 0x7f18f40b5310 unknown :-1 s=CLOSED pgs=2514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:29.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:29.777+0000 7f18f97c3640 1 --2- 172.21.15.38:0/2656443969 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f18f40ca610 0x7f18f41f5550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:29.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:29.777+0000 7f18f97c3640 1 --2- 172.21.15.38:0/2656443969 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f18d4078bc0 0x7f18d407b0d0 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:29.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:29.777+0000 7f18f97c3640 1 --2- 172.21.15.38:0/2656443969 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f18f4073e20 0x7f18f409ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:29.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:29.777+0000 7f18f97c3640 1 -- 172.21.15.38:0/2656443969 >> 172.21.15.38:0/2656443969 conn(0x7f18f40cca40 msgr2=0x7f18f415c630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:29.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:29.777+0000 7f18f97c3640 1 -- 172.21.15.38:0/2656443969 shutdown_connections 2024-09-21T03:45:29.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:29.777+0000 7f18f97c3640 1 -- 172.21.15.38:0/2656443969 wait complete. 2024-09-21T03:45:29.791 INFO:teuthology.orchestra.run.smithi038.stdout:{"pg_ready":true,"pg_map":{"version":508,"stamp":"2024-09-21T03:45:29.180061+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":116,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":307572,"kb_used_data":16456,"kb_used_omap":12,"kb_used_meta":290931,"kb_avail":754667148,"statfs":{"total":773094113280,"available":772779159552,"internally_reserved":0,"allocated":16850944,"data_stored":9107036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12748,"internal_metadata":297913908},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-36145,"num_objects":-27,"num_object_clones":0,"num_object_copies":-54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-27,"num_whiteouts":0,"num_read":-90,"num_read_kb":-92,"num_write":-97,"num_write_kb":-70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-16,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001766"},"pg_stats":[{"pgid":"1.0","version":"17'72","reported_seq":362,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-21T03:45:18.161661+0000","last_change":"2024-09-21T03:33:25.319776+0000","last_active":"2024-09-21T03:45:18.161661+0000","last_peered":"2024-09-21T03:45:18.161661+0000","last_clean":"2024-09-21T03:45:18.161661+0000","last_became_active":"2024-09-21T03:33:25.318399+0000","last_became_peered":"2024-09-21T03:33:25.318399+0000","last_unstale":"2024-09-21T03:45:18.161661+0000","last_undegraded":"2024-09-21T03:45:18.161661+0000","last_fullsized":"2024-09-21T03:45:18.161661+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:23.016672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:23.016672+0000","last_clean_scrub_stamp":"2024-09-21T03:33:23.016672+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:32:35.906114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":1,"up_from":16,"seq":68719476884,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41980,"kb_used_data":2468,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94329860,"statfs":{"total":96636764160,"available":96593776640,"internally_reserved":0,"allocated":2527232,"data_stored":1566193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509589,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":41436,"kb_used_data":1920,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330404,"statfs":{"total":96636764160,"available":96594333696,"internally_reserved":0,"allocated":1966080,"data_stored":994671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":40434123},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509588,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":41428,"kb_used_data":1920,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330412,"statfs":{"total":96636764160,"available":96594341888,"internally_reserved":0,"allocated":1966080,"data_stored":994671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509589,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37268,"kb_used_data":1920,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334572,"statfs":{"total":96636764160,"available":96598601728,"internally_reserved":0,"allocated":1966080,"data_stored":994671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509588,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37268,"kb_used_data":1920,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334572,"statfs":{"total":96636764160,"available":96598601728,"internally_reserved":0,"allocated":1966080,"data_stored":994671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542292,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33108,"kb_used_data":1920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338732,"statfs":{"total":96636764160,"available":96602861568,"internally_reserved":0,"allocated":1966080,"data_stored":994671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574996,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37832,"kb_used_data":2484,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334008,"statfs":{"total":96636764160,"available":96598024192,"internally_reserved":0,"allocated":2543616,"data_stored":1578928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542293,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37252,"kb_used_data":1904,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334588,"statfs":{"total":96636764160,"available":96598618112,"internally_reserved":0,"allocated":1949696,"data_stored":988560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T03:45:29.791 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-21T03:45:29.791 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T03:45:29.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:29.906+0000 7faf9535f640 1 -- 172.21.15.38:0/650260877 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faf90162590 msgr2=0x7faf901629a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:29.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:29.906+0000 7faf9535f640 1 --2- 172.21.15.38:0/650260877 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faf90162590 0x7faf901629a0 secure :-1 s=READY pgs=2516 cs=0 l=1 rev1=1 crypto rx=0x7faf78009830 tx=0x7faf7802a5c0 comp rx=0 tx=0).stop 2024-09-21T03:45:29.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:29.906+0000 7faf9535f640 1 -- 172.21.15.38:0/650260877 shutdown_connections 2024-09-21T03:45:29.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:29.906+0000 7faf9535f640 1 --2- 172.21.15.38:0/650260877 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faf90162590 0x7faf901629a0 unknown :-1 s=CLOSED pgs=2516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:29.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:29.906+0000 7faf9535f640 1 -- 172.21.15.38:0/650260877 >> 172.21.15.38:0/650260877 conn(0x7faf900d4440 msgr2=0x7faf900d4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:29.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:29.906+0000 7faf9535f640 1 -- 172.21.15.38:0/650260877 shutdown_connections 2024-09-21T03:45:29.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:29.906+0000 7faf9535f640 1 -- 172.21.15.38:0/650260877 wait complete. 2024-09-21T03:45:29.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:29.908+0000 7faf9535f640 1 Processor -- start 2024-09-21T03:45:29.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:29.908+0000 7faf9535f640 1 -- start start 2024-09-21T03:45:29.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:29.910+0000 7faf9535f640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faf900b2e60 0x7faf90150040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:29.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:29.910+0000 7faf9535f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faf900cfe00 0x7faf90150580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:29.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:29.910+0000 7faf9535f640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faf90162590 0x7faf901538e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:29.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:29.910+0000 7faf9535f640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7faf900b3b00 con 0x7faf900cfe00 2024-09-21T03:45:29.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:29.910+0000 7faf9535f640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7faf900b3980 con 0x7faf900b2e60 2024-09-21T03:45:29.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:29.910+0000 7faf9535f640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7faf900b4f70 con 0x7faf90162590 2024-09-21T03:45:29.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:29.910+0000 7faf8e7fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faf900cfe00 0x7faf90150580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:29.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:29.910+0000 7faf8e7fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faf900cfe00 0x7faf90150580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:56562/0 (socket says 172.21.15.38:56562) 2024-09-21T03:45:29.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:29.910+0000 7faf8e7fc640 1 -- 172.21.15.38:0/2496529394 learned_addr learned my addr 172.21.15.38:0/2496529394 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:45:29.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:29.910+0000 7faf8effd640 1 --2- 172.21.15.38:0/2496529394 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faf900b2e60 0x7faf90150040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:29.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:29.910+0000 7faf8f7fe640 1 --2- 172.21.15.38:0/2496529394 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faf90162590 0x7faf901538e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:29.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:29.910+0000 7faf8e7fc640 1 -- 172.21.15.38:0/2496529394 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faf90162590 msgr2=0x7faf901538e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:29.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:29.910+0000 7faf8e7fc640 1 --2- 172.21.15.38:0/2496529394 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faf90162590 0x7faf901538e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:29.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:29.910+0000 7faf8e7fc640 1 -- 172.21.15.38:0/2496529394 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faf900b2e60 msgr2=0x7faf90150040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:29.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:29.910+0000 7faf8e7fc640 1 --2- 172.21.15.38:0/2496529394 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faf900b2e60 0x7faf90150040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:29.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:29.910+0000 7faf8e7fc640 1 -- 172.21.15.38:0/2496529394 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf90157d40 con 0x7faf900cfe00 2024-09-21T03:45:29.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:29.910+0000 7faf8e7fc640 1 --2- 172.21.15.38:0/2496529394 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faf900cfe00 0x7faf90150580 secure :-1 s=READY pgs=2517 cs=0 l=1 rev1=1 crypto rx=0x7faf78009950 tx=0x7faf78033c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:29.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:29.911+0000 7faf877fe640 1 -- 172.21.15.38:0/2496529394 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf78031b60 con 0x7faf900cfe00 2024-09-21T03:45:29.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:29.911+0000 7faf877fe640 1 -- 172.21.15.38:0/2496529394 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faf78031df0 con 0x7faf900cfe00 2024-09-21T03:45:29.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:29.911+0000 7faf9535f640 1 -- 172.21.15.38:0/2496529394 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf901571c0 con 0x7faf900cfe00 2024-09-21T03:45:29.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:29.911+0000 7faf877fe640 1 -- 172.21.15.38:0/2496529394 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf78005280 con 0x7faf900cfe00 2024-09-21T03:45:29.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:29.911+0000 7faf9535f640 1 -- 172.21.15.38:0/2496529394 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf90157780 con 0x7faf900cfe00 2024-09-21T03:45:29.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:29.912+0000 7faf9535f640 1 -- 172.21.15.38:0/2496529394 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf901629a0 con 0x7faf900cfe00 2024-09-21T03:45:29.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:29.912+0000 7faf877fe640 1 -- 172.21.15.38:0/2496529394 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7faf78005420 con 0x7faf900cfe00 2024-09-21T03:45:29.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:29.913+0000 7faf877fe640 1 --2- 172.21.15.38:0/2496529394 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7faf70078b90 0x7faf7007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:29.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:29.913+0000 7faf877fe640 1 -- 172.21.15.38:0/2496529394 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 5421+0+0 (secure 0 0 0) 0x7faf780bbe60 con 0x7faf900cfe00 2024-09-21T03:45:29.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:29.913+0000 7faf8effd640 1 --2- 172.21.15.38:0/2496529394 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7faf70078b90 0x7faf7007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:29.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:29.914+0000 7faf8effd640 1 --2- 172.21.15.38:0/2496529394 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7faf70078b90 0x7faf7007b0a0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7faf800012e0 tx=0x7faf80009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:29.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:29.917+0000 7faf877fe640 1 -- 172.21.15.38:0/2496529394 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7faf7803f030 con 0x7faf900cfe00 2024-09-21T03:45:30.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:30.102+0000 7faf9535f640 1 -- 172.21.15.38:0/2496529394 --> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7faf90159100 con 0x7faf70078b90 2024-09-21T03:45:30.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:30.103+0000 7faf877fe640 1 -- 172.21.15.38:0/2496529394 <== mgr.4117 v2:172.21.15.195:6800/1631901344 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13472 (secure 0 0 0) 0x7faf90159100 con 0x7faf70078b90 2024-09-21T03:45:30.105 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:45:30.105 INFO:teuthology.orchestra.run.smithi038.stderr:dumped all 2024-09-21T03:45:30.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:30.105+0000 7faf9535f640 1 -- 172.21.15.38:0/2496529394 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7faf70078b90 msgr2=0x7faf7007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:30.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:30.105+0000 7faf9535f640 1 --2- 172.21.15.38:0/2496529394 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7faf70078b90 0x7faf7007b0a0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7faf800012e0 tx=0x7faf80009040 comp rx=0 tx=0).stop 2024-09-21T03:45:30.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:30.105+0000 7faf9535f640 1 -- 172.21.15.38:0/2496529394 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faf900cfe00 msgr2=0x7faf90150580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:30.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:30.105+0000 7faf9535f640 1 --2- 172.21.15.38:0/2496529394 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faf900cfe00 0x7faf90150580 secure :-1 s=READY pgs=2517 cs=0 l=1 rev1=1 crypto rx=0x7faf78009950 tx=0x7faf78033c10 comp rx=0 tx=0).stop 2024-09-21T03:45:30.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:30.105+0000 7faf9535f640 1 -- 172.21.15.38:0/2496529394 shutdown_connections 2024-09-21T03:45:30.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:30.105+0000 7faf9535f640 1 --2- 172.21.15.38:0/2496529394 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7faf70078b90 0x7faf7007b0a0 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:30.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:30.105+0000 7faf9535f640 1 --2- 172.21.15.38:0/2496529394 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faf90162590 0x7faf901538e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:30.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:30.105+0000 7faf9535f640 1 --2- 172.21.15.38:0/2496529394 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faf900cfe00 0x7faf90150580 unknown :-1 s=CLOSED pgs=2517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:30.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:30.105+0000 7faf9535f640 1 --2- 172.21.15.38:0/2496529394 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faf900b2e60 0x7faf90150040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:30.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:30.105+0000 7faf9535f640 1 -- 172.21.15.38:0/2496529394 >> 172.21.15.38:0/2496529394 conn(0x7faf900d4440 msgr2=0x7faf900b3270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:30.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:30.105+0000 7faf9535f640 1 -- 172.21.15.38:0/2496529394 shutdown_connections 2024-09-21T03:45:30.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:30.105+0000 7faf9535f640 1 -- 172.21.15.38:0/2496529394 wait complete. 2024-09-21T03:45:30.118 INFO:teuthology.orchestra.run.smithi038.stdout:{"pg_ready":true,"pg_map":{"version":508,"stamp":"2024-09-21T03:45:29.180061+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":116,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":307572,"kb_used_data":16456,"kb_used_omap":12,"kb_used_meta":290931,"kb_avail":754667148,"statfs":{"total":773094113280,"available":772779159552,"internally_reserved":0,"allocated":16850944,"data_stored":9107036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12748,"internal_metadata":297913908},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-36145,"num_objects":-27,"num_object_clones":0,"num_object_copies":-54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-27,"num_whiteouts":0,"num_read":-90,"num_read_kb":-92,"num_write":-97,"num_write_kb":-70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-16,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001766"},"pg_stats":[{"pgid":"1.0","version":"17'72","reported_seq":362,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-21T03:45:18.161661+0000","last_change":"2024-09-21T03:33:25.319776+0000","last_active":"2024-09-21T03:45:18.161661+0000","last_peered":"2024-09-21T03:45:18.161661+0000","last_clean":"2024-09-21T03:45:18.161661+0000","last_became_active":"2024-09-21T03:33:25.318399+0000","last_became_peered":"2024-09-21T03:33:25.318399+0000","last_unstale":"2024-09-21T03:45:18.161661+0000","last_undegraded":"2024-09-21T03:45:18.161661+0000","last_fullsized":"2024-09-21T03:45:18.161661+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:33:23.016672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:33:23.016672+0000","last_clean_scrub_stamp":"2024-09-21T03:33:23.016672+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:32:35.906114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":1,"up_from":16,"seq":68719476884,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41980,"kb_used_data":2468,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94329860,"statfs":{"total":96636764160,"available":96593776640,"internally_reserved":0,"allocated":2527232,"data_stored":1566193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509589,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":41436,"kb_used_data":1920,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330404,"statfs":{"total":96636764160,"available":96594333696,"internally_reserved":0,"allocated":1966080,"data_stored":994671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":40434123},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509588,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":41428,"kb_used_data":1920,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330412,"statfs":{"total":96636764160,"available":96594341888,"internally_reserved":0,"allocated":1966080,"data_stored":994671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509589,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37268,"kb_used_data":1920,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334572,"statfs":{"total":96636764160,"available":96598601728,"internally_reserved":0,"allocated":1966080,"data_stored":994671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509588,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37268,"kb_used_data":1920,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334572,"statfs":{"total":96636764160,"available":96598601728,"internally_reserved":0,"allocated":1966080,"data_stored":994671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542292,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33108,"kb_used_data":1920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338732,"statfs":{"total":96636764160,"available":96602861568,"internally_reserved":0,"allocated":1966080,"data_stored":994671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574996,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37832,"kb_used_data":2484,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334008,"statfs":{"total":96636764160,"available":96598024192,"internally_reserved":0,"allocated":2543616,"data_stored":1578928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542293,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37252,"kb_used_data":1904,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334588,"statfs":{"total":96636764160,"available":96598618112,"internally_reserved":0,"allocated":1949696,"data_stored":988560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T03:45:30.119 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T03:45:30.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:30.234+0000 7fc933ede640 1 -- 172.21.15.38:0/2973411081 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc92c162590 msgr2=0x7fc92c1629a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:30.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:30.234+0000 7fc933ede640 1 --2- 172.21.15.38:0/2973411081 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc92c162590 0x7fc92c1629a0 secure :-1 s=READY pgs=2518 cs=0 l=1 rev1=1 crypto rx=0x7fc91c005fd0 tx=0x7fc91c02a5d0 comp rx=0 tx=0).stop 2024-09-21T03:45:30.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:30.234+0000 7fc933ede640 1 -- 172.21.15.38:0/2973411081 shutdown_connections 2024-09-21T03:45:30.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:30.234+0000 7fc933ede640 1 --2- 172.21.15.38:0/2973411081 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc92c162590 0x7fc92c1629a0 unknown :-1 s=CLOSED pgs=2518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:30.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:30.234+0000 7fc933ede640 1 --2- 172.21.15.38:0/2973411081 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc92c0cfe00 0x7fc92c0d2a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:30.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:30.234+0000 7fc933ede640 1 -- 172.21.15.38:0/2973411081 >> 172.21.15.38:0/2973411081 conn(0x7fc92c0d4440 msgr2=0x7fc92c0d4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:30.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:30.234+0000 7fc933ede640 1 -- 172.21.15.38:0/2973411081 shutdown_connections 2024-09-21T03:45:30.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:30.234+0000 7fc933ede640 1 -- 172.21.15.38:0/2973411081 wait complete. 2024-09-21T03:45:30.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:30.236+0000 7fc933ede640 1 Processor -- start 2024-09-21T03:45:30.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:30.236+0000 7fc933ede640 1 -- start start 2024-09-21T03:45:30.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:30.237+0000 7fc933ede640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc92c0b2e60 0x7fc92c1f64c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:30.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:30.237+0000 7fc933ede640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc92c0cfe00 0x7fc92c156900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:30.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:30.237+0000 7fc933ede640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc92c162590 0x7fc92c15e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:30.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:30.237+0000 7fc933ede640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fc92c0b3b00 con 0x7fc92c0cfe00 2024-09-21T03:45:30.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:30.237+0000 7fc933ede640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fc92c0b3980 con 0x7fc92c162590 2024-09-21T03:45:30.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:30.237+0000 7fc933ede640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fc92c0b3f60 con 0x7fc92c0b2e60 2024-09-21T03:45:30.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:30.237+0000 7fc931479640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc92c0cfe00 0x7fc92c156900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:30.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:30.237+0000 7fc931479640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc92c0cfe00 0x7fc92c156900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:56594/0 (socket says 172.21.15.38:56594) 2024-09-21T03:45:30.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:30.237+0000 7fc931479640 1 -- 172.21.15.38:0/1166528607 learned_addr learned my addr 172.21.15.38:0/1166528607 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:45:30.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:30.238+0000 7fc931c7a640 1 --2- 172.21.15.38:0/1166528607 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc92c0b2e60 0x7fc92c1f64c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:30.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:30.238+0000 7fc93247b640 1 --2- 172.21.15.38:0/1166528607 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc92c162590 0x7fc92c15e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:30.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:30.238+0000 7fc931479640 1 -- 172.21.15.38:0/1166528607 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc92c0b2e60 msgr2=0x7fc92c1f64c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:30.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:30.238+0000 7fc931479640 1 --2- 172.21.15.38:0/1166528607 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc92c0b2e60 0x7fc92c1f64c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:30.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:30.238+0000 7fc931479640 1 -- 172.21.15.38:0/1166528607 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc92c162590 msgr2=0x7fc92c15e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:30.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:30.238+0000 7fc931479640 1 --2- 172.21.15.38:0/1166528607 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc92c162590 0x7fc92c15e4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:30.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:30.238+0000 7fc931479640 1 -- 172.21.15.38:0/1166528607 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc92c156e40 con 0x7fc92c0cfe00 2024-09-21T03:45:30.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:30.238+0000 7fc931479640 1 --2- 172.21.15.38:0/1166528607 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc92c0cfe00 0x7fc92c156900 secure :-1 s=READY pgs=2519 cs=0 l=1 rev1=1 crypto rx=0x7fc91c000c00 tx=0x7fc91c0038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:30.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:30.238+0000 7fc9227fc640 1 -- 172.21.15.38:0/1166528607 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc91c03e030 con 0x7fc92c0cfe00 2024-09-21T03:45:30.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:30.238+0000 7fc9227fc640 1 -- 172.21.15.38:0/1166528607 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc91c003ca0 con 0x7fc92c0cfe00 2024-09-21T03:45:30.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:30.238+0000 7fc933ede640 1 -- 172.21.15.38:0/1166528607 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc92c159770 con 0x7fc92c0cfe00 2024-09-21T03:45:30.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:30.238+0000 7fc9227fc640 1 -- 172.21.15.38:0/1166528607 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc91c035040 con 0x7fc92c0cfe00 2024-09-21T03:45:30.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:30.238+0000 7fc933ede640 1 -- 172.21.15.38:0/1166528607 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc92c159d30 con 0x7fc92c0cfe00 2024-09-21T03:45:30.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:30.239+0000 7fc933ede640 1 -- 172.21.15.38:0/1166528607 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc92c0d2a20 con 0x7fc92c0cfe00 2024-09-21T03:45:30.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:30.240+0000 7fc9227fc640 1 -- 172.21.15.38:0/1166528607 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc91c041030 con 0x7fc92c0cfe00 2024-09-21T03:45:30.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:30.240+0000 7fc9227fc640 1 --2- 172.21.15.38:0/1166528607 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc910078bc0 0x7fc91007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:30.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:30.240+0000 7fc9227fc640 1 -- 172.21.15.38:0/1166528607 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 5421+0+0 (secure 0 0 0) 0x7fc91c0bb140 con 0x7fc92c0cfe00 2024-09-21T03:45:30.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:30.241+0000 7fc931c7a640 1 --2- 172.21.15.38:0/1166528607 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc910078bc0 0x7fc91007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:30.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:30.241+0000 7fc931c7a640 1 --2- 172.21.15.38:0/1166528607 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc910078bc0 0x7fc91007b0d0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7fc928005d80 tx=0x7fc928005c70 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:30.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:30.245+0000 7fc9227fc640 1 -- 172.21.15.38:0/1166528607 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fc91c083730 con 0x7fc92c0cfe00 2024-09-21T03:45:30.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:30.434+0000 7fc933ede640 1 -- 172.21.15.38:0/1166528607 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc92c06eaa0 con 0x7fc92c0cfe00 2024-09-21T03:45:30.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:30.435+0000 7fc9227fc640 1 -- 172.21.15.38:0/1166528607 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v152) ==== 74+0+13469 (secure 0 0 0) 0x7fc91c083030 con 0x7fc92c0cfe00 2024-09-21T03:45:30.437 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:45:30.437 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":152,"fsid":"9d50d66f-656b-4fe2-8e3b-fe53b79c4d43","created":"2024-09-21T03:33:10.677865+0000","modified":"2024-09-21T03:45:27.201621+0000","last_up_change":"2024-09-21T03:33:24.023421+0000","last_in_change":"2024-09-21T03:33:14.296193+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T03:33:22.992573+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dbab183f-7e63-48b8-960a-ae55fd5bf565","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6801","nonce":2381639433}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6803","nonce":2381639433}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6807","nonce":2381639433}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":2381639433},{"type":"v1","addr":"172.21.15.38:6805","nonce":2381639433}]},"public_addr":"172.21.15.38:6801/2381639433","cluster_addr":"172.21.15.38:6803/2381639433","heartbeat_back_addr":"172.21.15.38:6807/2381639433","heartbeat_front_addr":"172.21.15.38:6805/2381639433","state":["exists","up"]},{"osd":1,"uuid":"fa1c0e8a-ebc1-4359-8146-bd5342abbc16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6817","nonce":1491818732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6819","nonce":1491818732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6823","nonce":1491818732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":1491818732},{"type":"v1","addr":"172.21.15.38:6821","nonce":1491818732}]},"public_addr":"172.21.15.38:6817/1491818732","cluster_addr":"172.21.15.38:6819/1491818732","heartbeat_back_addr":"172.21.15.38:6823/1491818732","heartbeat_front_addr":"172.21.15.38:6821/1491818732","state":["exists","up"]},{"osd":2,"uuid":"2ed3cd2d-385d-4fc6-9869-214552a84803","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6825","nonce":2577858902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6827","nonce":2577858902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6831","nonce":2577858902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":2577858902},{"type":"v1","addr":"172.21.15.38:6829","nonce":2577858902}]},"public_addr":"172.21.15.38:6825/2577858902","cluster_addr":"172.21.15.38:6827/2577858902","heartbeat_back_addr":"172.21.15.38:6831/2577858902","heartbeat_front_addr":"172.21.15.38:6829/2577858902","state":["exists","up"]},{"osd":3,"uuid":"62d1d95e-919d-4250-be71-1d34cfa70dbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6809","nonce":1286810294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6811","nonce":1286810294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6815","nonce":1286810294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":1286810294},{"type":"v1","addr":"172.21.15.38:6813","nonce":1286810294}]},"public_addr":"172.21.15.38:6809/1286810294","cluster_addr":"172.21.15.38:6811/1286810294","heartbeat_back_addr":"172.21.15.38:6815/1286810294","heartbeat_front_addr":"172.21.15.38:6813/1286810294","state":["exists","up"]},{"osd":4,"uuid":"7c9f9314-99d7-4d73-8392-37809a9f54b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6803","nonce":3401837571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6805","nonce":3401837571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6809","nonce":3401837571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":3401837571},{"type":"v1","addr":"172.21.15.195:6807","nonce":3401837571}]},"public_addr":"172.21.15.195:6803/3401837571","cluster_addr":"172.21.15.195:6805/3401837571","heartbeat_back_addr":"172.21.15.195:6809/3401837571","heartbeat_front_addr":"172.21.15.195:6807/3401837571","state":["exists","up"]},{"osd":5,"uuid":"bde4fe9f-8414-4bba-bf4a-43973d1976bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6811","nonce":2080960180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6813","nonce":2080960180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6817","nonce":2080960180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":2080960180},{"type":"v1","addr":"172.21.15.195:6815","nonce":2080960180}]},"public_addr":"172.21.15.195:6811/2080960180","cluster_addr":"172.21.15.195:6813/2080960180","heartbeat_back_addr":"172.21.15.195:6817/2080960180","heartbeat_front_addr":"172.21.15.195:6815/2080960180","state":["exists","up"]},{"osd":6,"uuid":"6ee6d7f4-1a5f-4a4c-ac16-48fe05f6a002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6827","nonce":4055825675}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6829","nonce":4055825675}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6833","nonce":4055825675}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":4055825675},{"type":"v1","addr":"172.21.15.195:6831","nonce":4055825675}]},"public_addr":"172.21.15.195:6827/4055825675","cluster_addr":"172.21.15.195:6829/4055825675","heartbeat_back_addr":"172.21.15.195:6833/4055825675","heartbeat_front_addr":"172.21.15.195:6831/4055825675","state":["exists","up"]},{"osd":7,"uuid":"efeb5f51-329a-4484-8290-92e7d3503d05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6819","nonce":1606106342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6821","nonce":1606106342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6825","nonce":1606106342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":1606106342},{"type":"v1","addr":"172.21.15.195:6823","nonce":1606106342}]},"public_addr":"172.21.15.195:6819/1606106342","cluster_addr":"172.21.15.195:6821/1606106342","heartbeat_back_addr":"172.21.15.195:6825/1606106342","heartbeat_front_addr":"172.21.15.195:6823/1606106342","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.901343+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:21.115003+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.901623+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.835238+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.132199+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:19.817560+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:20.829690+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:21.087300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.38:6833/3839260519":"2024-09-22T03:45:24.189479+0000","172.21.15.38:6832/3839260519":"2024-09-22T03:45:24.189479+0000","172.21.15.38:6833/3866292839":"2024-09-22T03:45:11.068318+0000","172.21.15.38:6832/3866292839":"2024-09-22T03:45:11.068318+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T03:45:30.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:30.437+0000 7fc933ede640 1 -- 172.21.15.38:0/1166528607 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc910078bc0 msgr2=0x7fc91007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:30.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:30.437+0000 7fc933ede640 1 --2- 172.21.15.38:0/1166528607 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc910078bc0 0x7fc91007b0d0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7fc928005d80 tx=0x7fc928005c70 comp rx=0 tx=0).stop 2024-09-21T03:45:30.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:30.437+0000 7fc933ede640 1 -- 172.21.15.38:0/1166528607 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc92c0cfe00 msgr2=0x7fc92c156900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:30.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:30.437+0000 7fc933ede640 1 --2- 172.21.15.38:0/1166528607 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc92c0cfe00 0x7fc92c156900 secure :-1 s=READY pgs=2519 cs=0 l=1 rev1=1 crypto rx=0x7fc91c000c00 tx=0x7fc91c0038a0 comp rx=0 tx=0).stop 2024-09-21T03:45:30.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:30.437+0000 7fc933ede640 1 -- 172.21.15.38:0/1166528607 shutdown_connections 2024-09-21T03:45:30.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:30.437+0000 7fc933ede640 1 --2- 172.21.15.38:0/1166528607 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc92c162590 0x7fc92c15e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:30.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:30.437+0000 7fc933ede640 1 --2- 172.21.15.38:0/1166528607 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc92c0cfe00 0x7fc92c156900 unknown :-1 s=CLOSED pgs=2519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:30.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:30.437+0000 7fc933ede640 1 --2- 172.21.15.38:0/1166528607 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc910078bc0 0x7fc91007b0d0 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:30.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:30.437+0000 7fc933ede640 1 --2- 172.21.15.38:0/1166528607 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc92c0b2e60 0x7fc92c1f64c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:30.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:30.437+0000 7fc933ede640 1 -- 172.21.15.38:0/1166528607 >> 172.21.15.38:0/1166528607 conn(0x7fc92c0d4440 msgr2=0x7fc92c0b3270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:30.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:30.437+0000 7fc933ede640 1 -- 172.21.15.38:0/1166528607 shutdown_connections 2024-09-21T03:45:30.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:30.437+0000 7fc933ede640 1 -- 172.21.15.38:0/1166528607 wait complete. 2024-09-21T03:45:31.451 INFO:tasks.ceph:Scrubbing osd.0 2024-09-21T03:45:31.452 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-09-21T03:45:31.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:31.586+0000 7f4f7f44d640 1 -- 172.21.15.38:0/2126310238 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f78162de0 msgr2=0x7f4f78169580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:31.589 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:31.586+0000 7f4f7f44d640 1 --2- 172.21.15.38:0/2126310238 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f78162de0 0x7f4f78169580 secure :-1 s=READY pgs=2521 cs=0 l=1 rev1=1 crypto rx=0x7f4f70005fd0 tx=0x7f4f7002a820 comp rx=0 tx=0).stop 2024-09-21T03:45:31.589 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:31.586+0000 7f4f7f44d640 1 -- 172.21.15.38:0/2126310238 shutdown_connections 2024-09-21T03:45:31.589 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:31.586+0000 7f4f7f44d640 1 --2- 172.21.15.38:0/2126310238 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f78162de0 0x7f4f78169580 unknown :-1 s=CLOSED pgs=2521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:31.589 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:31.586+0000 7f4f7f44d640 1 -- 172.21.15.38:0/2126310238 >> 172.21.15.38:0/2126310238 conn(0x7f4f780d44e0 msgr2=0x7f4f780d48f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:31.589 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:31.586+0000 7f4f7f44d640 1 -- 172.21.15.38:0/2126310238 shutdown_connections 2024-09-21T03:45:31.589 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:31.586+0000 7f4f7f44d640 1 -- 172.21.15.38:0/2126310238 wait complete. 2024-09-21T03:45:31.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:31.588+0000 7f4f7f44d640 1 Processor -- start 2024-09-21T03:45:31.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:31.588+0000 7f4f7f44d640 1 -- start start 2024-09-21T03:45:31.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:31.589+0000 7f4f7f44d640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4f7815e7d0 0x7f4f780b94b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:31.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:31.589+0000 7f4f7f44d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f78162de0 0x7f4f7815a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:31.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:31.589+0000 7f4f7f44d640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4f780ca9c0 0x7f4f781575c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:31.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:31.589+0000 7f4f7f44d640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f4f78169cd0 con 0x7f4f78162de0 2024-09-21T03:45:31.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:31.589+0000 7f4f7f44d640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f4f780d37a0 con 0x7f4f7815e7d0 2024-09-21T03:45:31.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:31.589+0000 7f4f7f44d640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f4f78169e50 con 0x7f4f780ca9c0 2024-09-21T03:45:31.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:31.589+0000 7f4f7c9e8640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f78162de0 0x7f4f7815a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:31.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:31.589+0000 7f4f7c9e8640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f78162de0 0x7f4f7815a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:56606/0 (socket says 172.21.15.38:56606) 2024-09-21T03:45:31.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:31.589+0000 7f4f7c9e8640 1 -- 172.21.15.38:0/1963948018 learned_addr learned my addr 172.21.15.38:0/1963948018 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:45:31.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:31.589+0000 7f4f7d1e9640 1 --2- 172.21.15.38:0/1963948018 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4f7815e7d0 0x7f4f780b94b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:31.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:31.590+0000 7f4f7d9ea640 1 --2- 172.21.15.38:0/1963948018 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4f780ca9c0 0x7f4f781575c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:31.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:31.590+0000 7f4f7c9e8640 1 -- 172.21.15.38:0/1963948018 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4f780ca9c0 msgr2=0x7f4f781575c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:31.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:31.590+0000 7f4f7c9e8640 1 --2- 172.21.15.38:0/1963948018 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4f780ca9c0 0x7f4f781575c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:31.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:31.590+0000 7f4f7c9e8640 1 -- 172.21.15.38:0/1963948018 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4f7815e7d0 msgr2=0x7f4f780b94b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:31.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:31.590+0000 7f4f7c9e8640 1 --2- 172.21.15.38:0/1963948018 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4f7815e7d0 0x7f4f780b94b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:31.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:31.590+0000 7f4f7c9e8640 1 -- 172.21.15.38:0/1963948018 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f780b8410 con 0x7f4f78162de0 2024-09-21T03:45:31.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:31.590+0000 7f4f7d9ea640 1 --2- 172.21.15.38:0/1963948018 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4f780ca9c0 0x7f4f781575c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:45:31.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:31.590+0000 7f4f7c9e8640 1 --2- 172.21.15.38:0/1963948018 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f78162de0 0x7f4f7815a760 secure :-1 s=READY pgs=2522 cs=0 l=1 rev1=1 crypto rx=0x7f4f68009fd0 tx=0x7f4f6800b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:31.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:31.590+0000 7f4f6dffb640 1 -- 172.21.15.38:0/1963948018 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f68015020 con 0x7f4f78162de0 2024-09-21T03:45:31.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:31.590+0000 7f4f6dffb640 1 -- 172.21.15.38:0/1963948018 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4f68011920 con 0x7f4f78162de0 2024-09-21T03:45:31.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:31.590+0000 7f4f6dffb640 1 -- 172.21.15.38:0/1963948018 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f68011d50 con 0x7f4f78162de0 2024-09-21T03:45:31.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:31.590+0000 7f4f7f44d640 1 -- 172.21.15.38:0/1963948018 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f780b86f0 con 0x7f4f78162de0 2024-09-21T03:45:31.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:31.590+0000 7f4f7f44d640 1 -- 172.21.15.38:0/1963948018 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f780b8cb0 con 0x7f4f78162de0 2024-09-21T03:45:31.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:31.591+0000 7f4f7f44d640 1 -- 172.21.15.38:0/1963948018 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4f780008d0 con 0x7f4f78162de0 2024-09-21T03:45:31.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:31.592+0000 7f4f6dffb640 1 -- 172.21.15.38:0/1963948018 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f4f68018020 con 0x7f4f78162de0 2024-09-21T03:45:31.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:31.593+0000 7f4f6dffb640 1 --2- 172.21.15.38:0/1963948018 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4f5c078bc0 0x7f4f5c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:31.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:31.593+0000 7f4f6dffb640 1 -- 172.21.15.38:0/1963948018 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 5421+0+0 (secure 0 0 0) 0x7f4f6809aaf0 con 0x7f4f78162de0 2024-09-21T03:45:31.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:31.593+0000 7f4f6dffb640 1 --2- 172.21.15.38:0/1963948018 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x7f4f5c07f150 0x7f4f5c0815e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:31.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:31.593+0000 7f4f6dffb640 1 -- 172.21.15.38:0/1963948018 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4f6800de50 con 0x7f4f5c07f150 2024-09-21T03:45:31.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:31.593+0000 7f4f6dffb640 1 -- 172.21.15.38:0/1963948018 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_get_version_reply(handle=1 version=152) ==== 24+0+0 (secure 0 0 0) 0x7f4f6809f2c0 con 0x7f4f78162de0 2024-09-21T03:45:31.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:31.593+0000 7f4f7d1e9640 1 --2- 172.21.15.38:0/1963948018 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4f5c078bc0 0x7f4f5c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:31.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:31.593+0000 7f4f7d9ea640 1 --2- 172.21.15.38:0/1963948018 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x7f4f5c07f150 0x7f4f5c0815e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:31.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:31.593+0000 7f4f7d9ea640 1 --2- 172.21.15.38:0/1963948018 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x7f4f5c07f150 0x7f4f5c0815e0 crc :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:31.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:31.593+0000 7f4f7d1e9640 1 --2- 172.21.15.38:0/1963948018 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4f5c078bc0 0x7f4f5c07b0d0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f4f74001720 tx=0x7f4f74007e90 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:31.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:31.594+0000 7f4f6dffb640 1 -- 172.21.15.38:0/1963948018 <== osd.0 v2:172.21.15.38:6800/2381639433 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4f6800de50 con 0x7f4f5c07f150 2024-09-21T03:45:31.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:31.607+0000 7f4f7f44d640 1 -- 172.21.15.38:0/1963948018 --> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f4f780b7ec0 con 0x7f4f5c07f150 2024-09-21T03:45:31.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:31.607+0000 7f4f6dffb640 1 -- 172.21.15.38:0/1963948018 <== osd.0 v2:172.21.15.38:6800/2381639433 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f4f780b7ec0 con 0x7f4f5c07f150 2024-09-21T03:45:31.610 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-09-21T03:45:31.610 INFO:teuthology.orchestra.run.smithi038.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-21T03:45:31.610 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-09-21T03:45:31.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:31.608+0000 7f4f7f44d640 1 -- 172.21.15.38:0/1963948018 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x7f4f5c07f150 msgr2=0x7f4f5c0815e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:31.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:31.608+0000 7f4f7f44d640 1 --2- 172.21.15.38:0/1963948018 >> [v2:172.21.15.38:6800/2381639433,v1:172.21.15.38:6801/2381639433] conn(0x7f4f5c07f150 0x7f4f5c0815e0 crc :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:31.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:31.608+0000 7f4f7f44d640 1 -- 172.21.15.38:0/1963948018 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4f5c078bc0 msgr2=0x7f4f5c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:31.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:31.608+0000 7f4f7f44d640 1 --2- 172.21.15.38:0/1963948018 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4f5c078bc0 0x7f4f5c07b0d0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f4f74001720 tx=0x7f4f74007e90 comp rx=0 tx=0).stop 2024-09-21T03:45:31.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:31.608+0000 7f4f7f44d640 1 -- 172.21.15.38:0/1963948018 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f78162de0 msgr2=0x7f4f7815a760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:31.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:31.608+0000 7f4f7f44d640 1 --2- 172.21.15.38:0/1963948018 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f78162de0 0x7f4f7815a760 secure :-1 s=READY pgs=2522 cs=0 l=1 rev1=1 crypto rx=0x7f4f68009fd0 tx=0x7f4f6800b870 comp rx=0 tx=0).stop 2024-09-21T03:45:31.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:31.608+0000 7f4f7d9ea640 1 -- 172.21.15.38:0/1963948018 reap_dead start 2024-09-21T03:45:31.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:31.608+0000 7f4f7f44d640 1 -- 172.21.15.38:0/1963948018 shutdown_connections 2024-09-21T03:45:31.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:31.608+0000 7f4f7f44d640 1 -- 172.21.15.38:0/1963948018 >> 172.21.15.38:0/1963948018 conn(0x7f4f780d44e0 msgr2=0x7f4f780d48f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:31.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:31.608+0000 7f4f7f44d640 1 -- 172.21.15.38:0/1963948018 shutdown_connections 2024-09-21T03:45:31.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:31.608+0000 7f4f7f44d640 1 -- 172.21.15.38:0/1963948018 wait complete. 2024-09-21T03:45:31.621 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-09-21T03:45:31.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:31.743+0000 7ff1ee4cc640 1 -- 172.21.15.38:0/2497493569 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff1e80d0d80 msgr2=0x7ff1e8167550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:31.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:31.743+0000 7ff1ee4cc640 1 --2- 172.21.15.38:0/2497493569 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff1e80d0d80 0x7ff1e8167550 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7ff1e0005fd0 tx=0x7ff1e002a8b0 comp rx=0 tx=0).stop 2024-09-21T03:45:31.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:31.743+0000 7ff1ee4cc640 1 -- 172.21.15.38:0/2497493569 shutdown_connections 2024-09-21T03:45:31.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:31.743+0000 7ff1ee4cc640 1 --2- 172.21.15.38:0/2497493569 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff1e80d0d80 0x7ff1e8167550 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:31.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:31.743+0000 7ff1ee4cc640 1 -- 172.21.15.38:0/2497493569 >> 172.21.15.38:0/2497493569 conn(0x7ff1e80b46e0 msgr2=0x7ff1e80d23e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:31.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:31.744+0000 7ff1ee4cc640 1 -- 172.21.15.38:0/2497493569 shutdown_connections 2024-09-21T03:45:31.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:31.744+0000 7ff1ee4cc640 1 -- 172.21.15.38:0/2497493569 wait complete. 2024-09-21T03:45:31.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:31.745+0000 7ff1ee4cc640 1 Processor -- start 2024-09-21T03:45:31.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:31.745+0000 7ff1ee4cc640 1 -- start start 2024-09-21T03:45:31.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:31.747+0000 7ff1ee4cc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff1e80ccc40 0x7ff1e80c9af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:31.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:31.747+0000 7ff1ee4cc640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff1e815ca90 0x7ff1e814ff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:31.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:31.747+0000 7ff1ee4cc640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff1e8152cc0 0x7ff1e80cb7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:31.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:31.747+0000 7ff1ee4cc640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ff1e8167e20 con 0x7ff1e80ccc40 2024-09-21T03:45:31.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:31.747+0000 7ff1ee4cc640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ff1e8167ca0 con 0x7ff1e815ca90 2024-09-21T03:45:31.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:31.747+0000 7ff1ee4cc640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7ff1e8167fa0 con 0x7ff1e8152cc0 2024-09-21T03:45:31.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:31.747+0000 7ff1e7fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff1e80ccc40 0x7ff1e80c9af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:31.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:31.747+0000 7ff1e7fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff1e80ccc40 0x7ff1e80c9af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:56622/0 (socket says 172.21.15.38:56622) 2024-09-21T03:45:31.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:31.747+0000 7ff1e7fff640 1 -- 172.21.15.38:0/1277819771 learned_addr learned my addr 172.21.15.38:0/1277819771 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:45:31.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:31.747+0000 7ff1e77fe640 1 --2- 172.21.15.38:0/1277819771 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff1e815ca90 0x7ff1e814ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:31.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:31.747+0000 7ff1eca69640 1 --2- 172.21.15.38:0/1277819771 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff1e8152cc0 0x7ff1e80cb7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:31.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:31.747+0000 7ff1e7fff640 1 -- 172.21.15.38:0/1277819771 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff1e8152cc0 msgr2=0x7ff1e80cb7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:31.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:31.747+0000 7ff1e7fff640 1 --2- 172.21.15.38:0/1277819771 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff1e8152cc0 0x7ff1e80cb7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:31.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:31.747+0000 7ff1e7fff640 1 -- 172.21.15.38:0/1277819771 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff1e815ca90 msgr2=0x7ff1e814ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:31.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:31.747+0000 7ff1e7fff640 1 --2- 172.21.15.38:0/1277819771 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff1e815ca90 0x7ff1e814ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:31.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:31.747+0000 7ff1e7fff640 1 -- 172.21.15.38:0/1277819771 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1e8154e40 con 0x7ff1e80ccc40 2024-09-21T03:45:31.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:31.747+0000 7ff1e77fe640 1 --2- 172.21.15.38:0/1277819771 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff1e815ca90 0x7ff1e814ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:45:31.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:31.747+0000 7ff1eca69640 1 --2- 172.21.15.38:0/1277819771 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff1e8152cc0 0x7ff1e80cb7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:45:31.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:31.747+0000 7ff1e7fff640 1 --2- 172.21.15.38:0/1277819771 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff1e80ccc40 0x7ff1e80c9af0 secure :-1 s=READY pgs=2524 cs=0 l=1 rev1=1 crypto rx=0x7ff1d800c430 tx=0x7ff1d800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:31.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:31.748+0000 7ff1e4ff9640 1 -- 172.21.15.38:0/1277819771 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1d8017070 con 0x7ff1e80ccc40 2024-09-21T03:45:31.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:31.748+0000 7ff1e4ff9640 1 -- 172.21.15.38:0/1277819771 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff1d80053c0 con 0x7ff1e80ccc40 2024-09-21T03:45:31.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:31.748+0000 7ff1e4ff9640 1 -- 172.21.15.38:0/1277819771 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1d8003850 con 0x7ff1e80ccc40 2024-09-21T03:45:31.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:31.748+0000 7ff1ee4cc640 1 -- 172.21.15.38:0/1277819771 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1e81504d0 con 0x7ff1e80ccc40 2024-09-21T03:45:31.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:31.748+0000 7ff1ee4cc640 1 -- 172.21.15.38:0/1277819771 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1e8154360 con 0x7ff1e80ccc40 2024-09-21T03:45:31.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:31.749+0000 7ff1ee4cc640 1 -- 172.21.15.38:0/1277819771 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1e80d0e60 con 0x7ff1e80ccc40 2024-09-21T03:45:31.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:31.750+0000 7ff1e4ff9640 1 -- 172.21.15.38:0/1277819771 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ff1d801a070 con 0x7ff1e80ccc40 2024-09-21T03:45:31.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:31.750+0000 7ff1e4ff9640 1 --2- 172.21.15.38:0/1277819771 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff1d4078bc0 0x7ff1d407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:31.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:31.750+0000 7ff1e4ff9640 1 -- 172.21.15.38:0/1277819771 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 5421+0+0 (secure 0 0 0) 0x7ff1d8099f40 con 0x7ff1e80ccc40 2024-09-21T03:45:31.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:31.751+0000 7ff1e77fe640 1 --2- 172.21.15.38:0/1277819771 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff1d4078bc0 0x7ff1d407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:31.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:31.751+0000 7ff1e77fe640 1 --2- 172.21.15.38:0/1277819771 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff1d4078bc0 0x7ff1d407b0d0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7ff1e80b3890 tx=0x7ff1dc009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:31.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:31.755+0000 7ff1e4ff9640 1 -- 172.21.15.38:0/1277819771 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7ff1d80625c0 con 0x7ff1e80ccc40 2024-09-21T03:45:31.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:31.945+0000 7ff1ee4cc640 1 -- 172.21.15.38:0/1277819771 --> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7ff1e80cf880 con 0x7ff1d4078bc0 2024-09-21T03:45:31.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:31.946+0000 7ff1e4ff9640 1 -- 172.21.15.38:0/1277819771 <== mgr.4117 v2:172.21.15.195:6800/1631901344 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7ff1e80cf880 con 0x7ff1d4078bc0 2024-09-21T03:45:31.948 INFO:teuthology.orchestra.run.smithi038.stderr:instructed osd(s) 0 to deep-scrub 2024-09-21T03:45:31.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:31.948+0000 7ff1ee4cc640 1 -- 172.21.15.38:0/1277819771 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff1d4078bc0 msgr2=0x7ff1d407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:31.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:31.948+0000 7ff1ee4cc640 1 --2- 172.21.15.38:0/1277819771 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff1d4078bc0 0x7ff1d407b0d0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7ff1e80b3890 tx=0x7ff1dc009040 comp rx=0 tx=0).stop 2024-09-21T03:45:31.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:31.948+0000 7ff1ee4cc640 1 -- 172.21.15.38:0/1277819771 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff1e80ccc40 msgr2=0x7ff1e80c9af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:31.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:31.948+0000 7ff1ee4cc640 1 --2- 172.21.15.38:0/1277819771 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff1e80ccc40 0x7ff1e80c9af0 secure :-1 s=READY pgs=2524 cs=0 l=1 rev1=1 crypto rx=0x7ff1d800c430 tx=0x7ff1d800c8f0 comp rx=0 tx=0).stop 2024-09-21T03:45:31.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:31.948+0000 7ff1ee4cc640 1 -- 172.21.15.38:0/1277819771 shutdown_connections 2024-09-21T03:45:31.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:31.949+0000 7ff1ee4cc640 1 --2- 172.21.15.38:0/1277819771 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7ff1d4078bc0 0x7ff1d407b0d0 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:31.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:31.949+0000 7ff1ee4cc640 1 --2- 172.21.15.38:0/1277819771 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff1e8152cc0 0x7ff1e80cb7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:31.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:31.949+0000 7ff1ee4cc640 1 --2- 172.21.15.38:0/1277819771 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff1e815ca90 0x7ff1e814ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:31.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:31.949+0000 7ff1ee4cc640 1 --2- 172.21.15.38:0/1277819771 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff1e80ccc40 0x7ff1e80c9af0 unknown :-1 s=CLOSED pgs=2524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:31.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:31.949+0000 7ff1ee4cc640 1 -- 172.21.15.38:0/1277819771 >> 172.21.15.38:0/1277819771 conn(0x7ff1e80b46e0 msgr2=0x7ff1e80d23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:31.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:31.949+0000 7ff1ee4cc640 1 -- 172.21.15.38:0/1277819771 shutdown_connections 2024-09-21T03:45:31.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:31.949+0000 7ff1ee4cc640 1 -- 172.21.15.38:0/1277819771 wait complete. 2024-09-21T03:45:31.962 INFO:tasks.ceph:Scrubbing osd.1 2024-09-21T03:45:31.962 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-09-21T03:45:32.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.083+0000 7fbe184de640 1 -- 172.21.15.38:0/1433003279 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbe100d0e40 msgr2=0x7fbe101627b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:32.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.083+0000 7fbe184de640 1 --2- 172.21.15.38:0/1433003279 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbe100d0e40 0x7fbe101627b0 secure :-1 s=READY pgs=2525 cs=0 l=1 rev1=1 crypto rx=0x7fbe04005fd0 tx=0x7fbe0402a4b0 comp rx=0 tx=0).stop 2024-09-21T03:45:32.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.084+0000 7fbe184de640 1 -- 172.21.15.38:0/1433003279 shutdown_connections 2024-09-21T03:45:32.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.084+0000 7fbe184de640 1 --2- 172.21.15.38:0/1433003279 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbe100d0e40 0x7fbe101627b0 unknown :-1 s=CLOSED pgs=2525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:32.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.084+0000 7fbe184de640 1 --2- 172.21.15.38:0/1433003279 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbe10163350 0x7fbe1015c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:32.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.084+0000 7fbe184de640 1 -- 172.21.15.38:0/1433003279 >> 172.21.15.38:0/1433003279 conn(0x7fbe1006cde0 msgr2=0x7fbe1006d1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:32.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.084+0000 7fbe184de640 1 -- 172.21.15.38:0/1433003279 shutdown_connections 2024-09-21T03:45:32.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.084+0000 7fbe184de640 1 -- 172.21.15.38:0/1433003279 wait complete. 2024-09-21T03:45:32.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.085+0000 7fbe184de640 1 Processor -- start 2024-09-21T03:45:32.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.085+0000 7fbe184de640 1 -- start start 2024-09-21T03:45:32.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.087+0000 7fbe184de640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbe100d0e40 0x7fbe1015a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:32.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.087+0000 7fbe184de640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbe10162cf0 0x7fbe100cb650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:32.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.087+0000 7fbe184de640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbe10163350 0x7fbe101f5e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:32.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.087+0000 7fbe184de640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fbe100b3aa0 con 0x7fbe10163350 2024-09-21T03:45:32.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.087+0000 7fbe184de640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fbe100b3920 con 0x7fbe10162cf0 2024-09-21T03:45:32.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.087+0000 7fbe184de640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fbe100b3c20 con 0x7fbe100d0e40 2024-09-21T03:45:32.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.087+0000 7fbe16a7b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbe10163350 0x7fbe101f5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:32.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.087+0000 7fbe16a7b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbe10163350 0x7fbe101f5e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:56640/0 (socket says 172.21.15.38:56640) 2024-09-21T03:45:32.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.087+0000 7fbe16a7b640 1 -- 172.21.15.38:0/2246475864 learned_addr learned my addr 172.21.15.38:0/2246475864 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:45:32.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.087+0000 7fbe1627a640 1 --2- 172.21.15.38:0/2246475864 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbe100d0e40 0x7fbe1015a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:32.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.087+0000 7fbe15a79640 1 --2- 172.21.15.38:0/2246475864 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbe10162cf0 0x7fbe100cb650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:32.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.087+0000 7fbe16a7b640 1 -- 172.21.15.38:0/2246475864 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbe100d0e40 msgr2=0x7fbe1015a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:32.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.087+0000 7fbe16a7b640 1 --2- 172.21.15.38:0/2246475864 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbe100d0e40 0x7fbe1015a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:32.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.087+0000 7fbe16a7b640 1 -- 172.21.15.38:0/2246475864 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbe10162cf0 msgr2=0x7fbe100cb650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:32.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.087+0000 7fbe16a7b640 1 --2- 172.21.15.38:0/2246475864 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbe10162cf0 0x7fbe100cb650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:32.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.087+0000 7fbe16a7b640 1 -- 172.21.15.38:0/2246475864 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe100ca6f0 con 0x7fbe10163350 2024-09-21T03:45:32.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.087+0000 7fbe15a79640 1 --2- 172.21.15.38:0/2246475864 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbe10162cf0 0x7fbe100cb650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:45:32.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.088+0000 7fbe16a7b640 1 --2- 172.21.15.38:0/2246475864 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbe10163350 0x7fbe101f5e90 secure :-1 s=READY pgs=2526 cs=0 l=1 rev1=1 crypto rx=0x7fbe0000c730 tx=0x7fbe0000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:32.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.088+0000 7fbe0effd640 1 -- 172.21.15.38:0/2246475864 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe00003b70 con 0x7fbe10163350 2024-09-21T03:45:32.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.088+0000 7fbe0effd640 1 -- 172.21.15.38:0/2246475864 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbe00003d30 con 0x7fbe10163350 2024-09-21T03:45:32.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.088+0000 7fbe0effd640 1 -- 172.21.15.38:0/2246475864 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe000052d0 con 0x7fbe10163350 2024-09-21T03:45:32.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.088+0000 7fbe184de640 1 -- 172.21.15.38:0/2246475864 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe100ca9d0 con 0x7fbe10163350 2024-09-21T03:45:32.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.088+0000 7fbe184de640 1 -- 172.21.15.38:0/2246475864 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe100cbc90 con 0x7fbe10163350 2024-09-21T03:45:32.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.089+0000 7fbe184de640 1 -- 172.21.15.38:0/2246475864 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fbe100caef0 con 0x7fbe10163350 2024-09-21T03:45:32.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.090+0000 7fbe0effd640 1 -- 172.21.15.38:0/2246475864 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fbe00005470 con 0x7fbe10163350 2024-09-21T03:45:32.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.090+0000 7fbe0effd640 1 --2- 172.21.15.38:0/2246475864 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fbdfc078b90 0x7fbdfc07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:32.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.090+0000 7fbe0effd640 1 -- 172.21.15.38:0/2246475864 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 5421+0+0 (secure 0 0 0) 0x7fbe0009a840 con 0x7fbe10163350 2024-09-21T03:45:32.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.091+0000 7fbe0effd640 1 --2- 172.21.15.38:0/2246475864 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x7fbdfc07f120 0x7fbdfc0815b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:32.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.091+0000 7fbe0effd640 1 -- 172.21.15.38:0/2246475864 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbe00017ec0 con 0x7fbdfc07f120 2024-09-21T03:45:32.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.091+0000 7fbe1627a640 1 --2- 172.21.15.38:0/2246475864 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fbdfc078b90 0x7fbdfc07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:32.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.091+0000 7fbe0effd640 1 -- 172.21.15.38:0/2246475864 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_get_version_reply(handle=1 version=152) ==== 24+0+0 (secure 0 0 0) 0x7fbe0009f2c0 con 0x7fbe10163350 2024-09-21T03:45:32.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.091+0000 7fbe15a79640 1 --2- 172.21.15.38:0/2246475864 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x7fbdfc07f120 0x7fbdfc0815b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:32.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.091+0000 7fbe1627a640 1 --2- 172.21.15.38:0/2246475864 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fbdfc078b90 0x7fbdfc07b0a0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7fbe08009b30 tx=0x7fbe08005f90 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:32.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.091+0000 7fbe15a79640 1 --2- 172.21.15.38:0/2246475864 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x7fbdfc07f120 0x7fbdfc0815b0 crc :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:32.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.092+0000 7fbe0effd640 1 -- 172.21.15.38:0/2246475864 <== osd.1 v2:172.21.15.38:6816/1491818732 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbe00017ec0 con 0x7fbdfc07f120 2024-09-21T03:45:32.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.107+0000 7fbe184de640 1 -- 172.21.15.38:0/2246475864 --> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fbe101627b0 con 0x7fbdfc07f120 2024-09-21T03:45:32.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.107+0000 7fbe0effd640 1 -- 172.21.15.38:0/2246475864 <== osd.1 v2:172.21.15.38:6816/1491818732 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fbe101627b0 con 0x7fbdfc07f120 2024-09-21T03:45:32.109 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-09-21T03:45:32.109 INFO:teuthology.orchestra.run.smithi038.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-21T03:45:32.109 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-09-21T03:45:32.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.107+0000 7fbe184de640 1 -- 172.21.15.38:0/2246475864 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x7fbdfc07f120 msgr2=0x7fbdfc0815b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:32.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.107+0000 7fbe184de640 1 --2- 172.21.15.38:0/2246475864 >> [v2:172.21.15.38:6816/1491818732,v1:172.21.15.38:6817/1491818732] conn(0x7fbdfc07f120 0x7fbdfc0815b0 crc :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:32.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.107+0000 7fbe184de640 1 -- 172.21.15.38:0/2246475864 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fbdfc078b90 msgr2=0x7fbdfc07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:32.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.107+0000 7fbe184de640 1 --2- 172.21.15.38:0/2246475864 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fbdfc078b90 0x7fbdfc07b0a0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7fbe08009b30 tx=0x7fbe08005f90 comp rx=0 tx=0).stop 2024-09-21T03:45:32.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.107+0000 7fbe184de640 1 -- 172.21.15.38:0/2246475864 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbe10163350 msgr2=0x7fbe101f5e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:32.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.107+0000 7fbe184de640 1 --2- 172.21.15.38:0/2246475864 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbe10163350 0x7fbe101f5e90 secure :-1 s=READY pgs=2526 cs=0 l=1 rev1=1 crypto rx=0x7fbe0000c730 tx=0x7fbe0000cbf0 comp rx=0 tx=0).stop 2024-09-21T03:45:32.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.107+0000 7fbe16a7b640 1 -- 172.21.15.38:0/2246475864 reap_dead start 2024-09-21T03:45:32.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.108+0000 7fbe184de640 1 -- 172.21.15.38:0/2246475864 shutdown_connections 2024-09-21T03:45:32.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.108+0000 7fbe184de640 1 -- 172.21.15.38:0/2246475864 >> 172.21.15.38:0/2246475864 conn(0x7fbe1006cde0 msgr2=0x7fbe1006d1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:32.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.108+0000 7fbe184de640 1 -- 172.21.15.38:0/2246475864 shutdown_connections 2024-09-21T03:45:32.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.108+0000 7fbe184de640 1 -- 172.21.15.38:0/2246475864 wait complete. 2024-09-21T03:45:32.122 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-09-21T03:45:32.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.239+0000 7f373aca9640 1 -- 172.21.15.38:0/2911828109 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3724003a50 msgr2=0x7f3724003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:32.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.239+0000 7f373aca9640 1 --2- 172.21.15.38:0/2911828109 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3724003a50 0x7f3724003ec0 secure :-1 s=READY pgs=2528 cs=0 l=1 rev1=1 crypto rx=0x7f37280016c0 tx=0x7f3724005830 comp rx=0 tx=0).stop 2024-09-21T03:45:32.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.239+0000 7f373aca9640 1 -- 172.21.15.38:0/2911828109 shutdown_connections 2024-09-21T03:45:32.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.239+0000 7f373aca9640 1 --2- 172.21.15.38:0/2911828109 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3724003a50 0x7f3724003ec0 unknown :-1 s=CLOSED pgs=2528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:32.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.239+0000 7f373aca9640 1 -- 172.21.15.38:0/2911828109 >> 172.21.15.38:0/2911828109 conn(0x7f37340cdca0 msgr2=0x7f37340ce0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:32.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.240+0000 7f373aca9640 1 -- 172.21.15.38:0/2911828109 shutdown_connections 2024-09-21T03:45:32.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.240+0000 7f373aca9640 1 -- 172.21.15.38:0/2911828109 wait complete. 2024-09-21T03:45:32.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.242+0000 7f373aca9640 1 Processor -- start 2024-09-21T03:45:32.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.242+0000 7f373aca9640 1 -- start start 2024-09-21T03:45:32.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.243+0000 7f373aca9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3724003a50 0x7f37340cfeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:32.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.243+0000 7f373aca9640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f37340d0840 0x7f3734153730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:32.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.243+0000 7f373aca9640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f37340cc6b0 0x7f37340ccac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:32.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.243+0000 7f373aca9640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f37340b5b20 con 0x7f3724003a50 2024-09-21T03:45:32.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.243+0000 7f373aca9640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f37340aed90 con 0x7f37340cc6b0 2024-09-21T03:45:32.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.243+0000 7f373aca9640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f37340b5ca0 con 0x7f37340d0840 2024-09-21T03:45:32.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.243+0000 7f3738a45640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3724003a50 0x7f37340cfeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:32.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.243+0000 7f3738a45640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3724003a50 0x7f37340cfeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:56650/0 (socket says 172.21.15.38:56650) 2024-09-21T03:45:32.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.243+0000 7f3738a45640 1 -- 172.21.15.38:0/356140388 learned_addr learned my addr 172.21.15.38:0/356140388 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:45:32.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.243+0000 7f3739246640 1 --2- 172.21.15.38:0/356140388 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f37340cc6b0 0x7f37340ccac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:32.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.243+0000 7f3733fff640 1 --2- 172.21.15.38:0/356140388 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f37340d0840 0x7f3734153730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:32.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.244+0000 7f3738a45640 1 -- 172.21.15.38:0/356140388 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f37340d0840 msgr2=0x7f3734153730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:32.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.244+0000 7f3738a45640 1 --2- 172.21.15.38:0/356140388 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f37340d0840 0x7f3734153730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:32.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.244+0000 7f3738a45640 1 -- 172.21.15.38:0/356140388 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f37340cc6b0 msgr2=0x7f37340ccac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:32.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.244+0000 7f3738a45640 1 --2- 172.21.15.38:0/356140388 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f37340cc6b0 0x7f37340ccac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:32.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.244+0000 7f3738a45640 1 -- 172.21.15.38:0/356140388 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3734153c70 con 0x7f3724003a50 2024-09-21T03:45:32.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.244+0000 7f3738a45640 1 --2- 172.21.15.38:0/356140388 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3724003a50 0x7f37340cfeb0 secure :-1 s=READY pgs=2529 cs=0 l=1 rev1=1 crypto rx=0x7f3728000fd0 tx=0x7f37280063a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:32.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.249+0000 7f37317fa640 1 -- 172.21.15.38:0/356140388 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3728041070 con 0x7f3724003a50 2024-09-21T03:45:32.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.249+0000 7f37317fa640 1 -- 172.21.15.38:0/356140388 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3728003af0 con 0x7f3724003a50 2024-09-21T03:45:32.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.249+0000 7f373aca9640 1 -- 172.21.15.38:0/356140388 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37340cd000 con 0x7f3724003a50 2024-09-21T03:45:32.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.249+0000 7f37317fa640 1 -- 172.21.15.38:0/356140388 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3728038070 con 0x7f3724003a50 2024-09-21T03:45:32.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.249+0000 7f373aca9640 1 -- 172.21.15.38:0/356140388 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37341f8c30 con 0x7f3724003a50 2024-09-21T03:45:32.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.250+0000 7f373aca9640 1 -- 172.21.15.38:0/356140388 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37340b2ca0 con 0x7f3724003a50 2024-09-21T03:45:32.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.252+0000 7f37317fa640 1 -- 172.21.15.38:0/356140388 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f3728030070 con 0x7f3724003a50 2024-09-21T03:45:32.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.252+0000 7f37317fa640 1 --2- 172.21.15.38:0/356140388 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3718078b90 0x7f371807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:32.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.252+0000 7f37317fa640 1 -- 172.21.15.38:0/356140388 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 5421+0+0 (secure 0 0 0) 0x7f37280ba150 con 0x7f3724003a50 2024-09-21T03:45:32.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.253+0000 7f3733fff640 1 --2- 172.21.15.38:0/356140388 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3718078b90 0x7f371807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:32.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.253+0000 7f3733fff640 1 --2- 172.21.15.38:0/356140388 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3718078b90 0x7f371807b0a0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f37340b1e30 tx=0x7f3720001510 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:32.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.257+0000 7f37317fa640 1 -- 172.21.15.38:0/356140388 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f37340cd000 con 0x7f3724003a50 2024-09-21T03:45:32.346 DEBUG:teuthology.orchestra.run.smithi038:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T03:45:32.348 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T03:45:32.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.446+0000 7f373aca9640 1 -- 172.21.15.38:0/356140388 --> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7f37340b1e60 con 0x7f3718078b90 2024-09-21T03:45:32.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.447+0000 7f37317fa640 1 -- 172.21.15.38:0/356140388 <== mgr.4117 v2:172.21.15.195:6800/1631901344 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f37340b1e60 con 0x7f3718078b90 2024-09-21T03:45:32.449 INFO:teuthology.orchestra.run.smithi038.stderr:instructed osd(s) 1 to deep-scrub 2024-09-21T03:45:32.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.449+0000 7f373aca9640 1 -- 172.21.15.38:0/356140388 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3718078b90 msgr2=0x7f371807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:32.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.449+0000 7f373aca9640 1 --2- 172.21.15.38:0/356140388 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3718078b90 0x7f371807b0a0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f37340b1e30 tx=0x7f3720001510 comp rx=0 tx=0).stop 2024-09-21T03:45:32.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.449+0000 7f373aca9640 1 -- 172.21.15.38:0/356140388 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3724003a50 msgr2=0x7f37340cfeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:32.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.449+0000 7f373aca9640 1 --2- 172.21.15.38:0/356140388 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3724003a50 0x7f37340cfeb0 secure :-1 s=READY pgs=2529 cs=0 l=1 rev1=1 crypto rx=0x7f3728000fd0 tx=0x7f37280063a0 comp rx=0 tx=0).stop 2024-09-21T03:45:32.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.450+0000 7f373aca9640 1 -- 172.21.15.38:0/356140388 shutdown_connections 2024-09-21T03:45:32.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.450+0000 7f373aca9640 1 --2- 172.21.15.38:0/356140388 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f37340cc6b0 0x7f37340ccac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:32.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.450+0000 7f373aca9640 1 --2- 172.21.15.38:0/356140388 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f3718078b90 0x7f371807b0a0 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:32.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.450+0000 7f373aca9640 1 --2- 172.21.15.38:0/356140388 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f37340d0840 0x7f3734153730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:32.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.450+0000 7f373aca9640 1 --2- 172.21.15.38:0/356140388 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3724003a50 0x7f37340cfeb0 unknown :-1 s=CLOSED pgs=2529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:32.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.450+0000 7f373aca9640 1 -- 172.21.15.38:0/356140388 >> 172.21.15.38:0/356140388 conn(0x7f37340cdca0 msgr2=0x7f37340ce0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:32.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.450+0000 7f373aca9640 1 -- 172.21.15.38:0/356140388 shutdown_connections 2024-09-21T03:45:32.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.450+0000 7f373aca9640 1 -- 172.21.15.38:0/356140388 wait complete. 2024-09-21T03:45:32.463 INFO:tasks.ceph:Scrubbing osd.2 2024-09-21T03:45:32.463 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-09-21T03:45:32.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.584+0000 7fd29749b640 1 -- 172.21.15.38:0/2981054105 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd27c003a50 msgr2=0x7fd27c003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:32.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.584+0000 7fd29749b640 1 --2- 172.21.15.38:0/2981054105 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd27c003a50 0x7fd27c003ec0 secure :-1 s=READY pgs=2531 cs=0 l=1 rev1=1 crypto rx=0x7fd28c009b50 tx=0x7fd27c005830 comp rx=0 tx=0).stop 2024-09-21T03:45:32.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.584+0000 7fd29749b640 1 -- 172.21.15.38:0/2981054105 shutdown_connections 2024-09-21T03:45:32.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.584+0000 7fd29749b640 1 --2- 172.21.15.38:0/2981054105 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd27c003a50 0x7fd27c003ec0 unknown :-1 s=CLOSED pgs=2531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:32.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.584+0000 7fd29749b640 1 -- 172.21.15.38:0/2981054105 >> 172.21.15.38:0/2981054105 conn(0x7fd2900b3ec0 msgr2=0x7fd2900b42d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:32.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.584+0000 7fd29749b640 1 -- 172.21.15.38:0/2981054105 shutdown_connections 2024-09-21T03:45:32.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.584+0000 7fd29749b640 1 -- 172.21.15.38:0/2981054105 wait complete. 2024-09-21T03:45:32.589 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.587+0000 7fd29749b640 1 Processor -- start 2024-09-21T03:45:32.589 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.587+0000 7fd29749b640 1 -- start start 2024-09-21T03:45:32.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.589+0000 7fd29749b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd27c003a50 0x7fd2900b1370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:32.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.589+0000 7fd29749b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd2900b7b00 0x7fd2900cadc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:32.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.589+0000 7fd29749b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd2900cb300 0x7fd2900c9940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:32.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.589+0000 7fd29749b640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fd2900ccdb0 con 0x7fd27c003a50 2024-09-21T03:45:32.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.589+0000 7fd29749b640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd2900ccc30 con 0x7fd2900b7b00 2024-09-21T03:45:32.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.589+0000 7fd295237640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd27c003a50 0x7fd2900b1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:32.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.589+0000 7fd29749b640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fd2900ccf30 con 0x7fd2900cb300 2024-09-21T03:45:32.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.589+0000 7fd295237640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd27c003a50 0x7fd2900b1370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38258/0 (socket says 172.21.15.38:38258) 2024-09-21T03:45:32.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.589+0000 7fd295237640 1 -- 172.21.15.38:0/2577225538 learned_addr learned my addr 172.21.15.38:0/2577225538 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:45:32.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.590+0000 7fd295237640 1 -- 172.21.15.38:0/2577225538 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd2900cb300 msgr2=0x7fd2900c9940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:32.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.590+0000 7fd295237640 1 --2- 172.21.15.38:0/2577225538 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd2900cb300 0x7fd2900c9940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:32.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.590+0000 7fd294a36640 1 --2- 172.21.15.38:0/2577225538 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd2900b7b00 0x7fd2900cadc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:32.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.590+0000 7fd295237640 1 -- 172.21.15.38:0/2577225538 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd2900b7b00 msgr2=0x7fd2900cadc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:32.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.590+0000 7fd295237640 1 --2- 172.21.15.38:0/2577225538 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd2900b7b00 0x7fd2900cadc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:32.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.590+0000 7fd295237640 1 -- 172.21.15.38:0/2577225538 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2900cb710 con 0x7fd27c003a50 2024-09-21T03:45:32.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.590+0000 7fd294a36640 1 --2- 172.21.15.38:0/2577225538 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd2900b7b00 0x7fd2900cadc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:45:32.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.590+0000 7fd295237640 1 --2- 172.21.15.38:0/2577225538 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd27c003a50 0x7fd2900b1370 secure :-1 s=READY pgs=2532 cs=0 l=1 rev1=1 crypto rx=0x7fd28c036040 tx=0x7fd28c0336d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:32.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.590+0000 7fd281ffb640 1 -- 172.21.15.38:0/2577225538 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd28c038040 con 0x7fd27c003a50 2024-09-21T03:45:32.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.590+0000 7fd281ffb640 1 -- 172.21.15.38:0/2577225538 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd28c041070 con 0x7fd27c003a50 2024-09-21T03:45:32.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.590+0000 7fd29749b640 1 -- 172.21.15.38:0/2577225538 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2900b5200 con 0x7fd27c003a50 2024-09-21T03:45:32.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.590+0000 7fd281ffb640 1 -- 172.21.15.38:0/2577225538 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd28c03c430 con 0x7fd27c003a50 2024-09-21T03:45:32.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.590+0000 7fd29749b640 1 -- 172.21.15.38:0/2577225538 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2901f3a10 con 0x7fd27c003a50 2024-09-21T03:45:32.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.591+0000 7fd29749b640 1 -- 172.21.15.38:0/2577225538 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd2900cd490 con 0x7fd27c003a50 2024-09-21T03:45:32.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.592+0000 7fd281ffb640 1 -- 172.21.15.38:0/2577225538 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fd28c02b8a0 con 0x7fd27c003a50 2024-09-21T03:45:32.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.593+0000 7fd281ffb640 1 --2- 172.21.15.38:0/2577225538 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd274078bc0 0x7fd27407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:32.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.593+0000 7fd281ffb640 1 -- 172.21.15.38:0/2577225538 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 5421+0+0 (secure 0 0 0) 0x7fd28c0bc020 con 0x7fd27c003a50 2024-09-21T03:45:32.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.593+0000 7fd281ffb640 1 --2- 172.21.15.38:0/2577225538 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x7fd27407f150 0x7fd2740815e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:32.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.593+0000 7fd281ffb640 1 -- 172.21.15.38:0/2577225538 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd274081cb0 con 0x7fd27407f150 2024-09-21T03:45:32.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.593+0000 7fd281ffb640 1 -- 172.21.15.38:0/2577225538 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_get_version_reply(handle=1 version=152) ==== 24+0+0 (secure 0 0 0) 0x7fd28c0bc4b0 con 0x7fd27c003a50 2024-09-21T03:45:32.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.593+0000 7fd294a36640 1 --2- 172.21.15.38:0/2577225538 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd274078bc0 0x7fd27407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:32.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.593+0000 7fd295a38640 1 --2- 172.21.15.38:0/2577225538 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x7fd27407f150 0x7fd2740815e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:32.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.594+0000 7fd295a38640 1 --2- 172.21.15.38:0/2577225538 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x7fd27407f150 0x7fd2740815e0 crc :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:32.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.594+0000 7fd294a36640 1 --2- 172.21.15.38:0/2577225538 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd274078bc0 0x7fd27407b0d0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fd2900ca7d0 tx=0x7fd284006b70 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:32.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.595+0000 7fd281ffb640 1 -- 172.21.15.38:0/2577225538 <== osd.2 v2:172.21.15.38:6824/2577858902 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd274081cb0 con 0x7fd27407f150 2024-09-21T03:45:32.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.607+0000 7fd29749b640 1 -- 172.21.15.38:0/2577225538 --> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fd29015ddc0 con 0x7fd27407f150 2024-09-21T03:45:32.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.608+0000 7fd281ffb640 1 -- 172.21.15.38:0/2577225538 <== osd.2 v2:172.21.15.38:6824/2577858902 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fd29015ddc0 con 0x7fd27407f150 2024-09-21T03:45:32.610 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-09-21T03:45:32.610 INFO:teuthology.orchestra.run.smithi038.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-21T03:45:32.610 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-09-21T03:45:32.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.608+0000 7fd29749b640 1 -- 172.21.15.38:0/2577225538 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x7fd27407f150 msgr2=0x7fd2740815e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:32.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.608+0000 7fd29749b640 1 --2- 172.21.15.38:0/2577225538 >> [v2:172.21.15.38:6824/2577858902,v1:172.21.15.38:6825/2577858902] conn(0x7fd27407f150 0x7fd2740815e0 crc :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:32.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.608+0000 7fd29749b640 1 -- 172.21.15.38:0/2577225538 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd274078bc0 msgr2=0x7fd27407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:32.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.608+0000 7fd29749b640 1 --2- 172.21.15.38:0/2577225538 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fd274078bc0 0x7fd27407b0d0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fd2900ca7d0 tx=0x7fd284006b70 comp rx=0 tx=0).stop 2024-09-21T03:45:32.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.608+0000 7fd29749b640 1 -- 172.21.15.38:0/2577225538 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd27c003a50 msgr2=0x7fd2900b1370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:32.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.608+0000 7fd29749b640 1 --2- 172.21.15.38:0/2577225538 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd27c003a50 0x7fd2900b1370 secure :-1 s=READY pgs=2532 cs=0 l=1 rev1=1 crypto rx=0x7fd28c036040 tx=0x7fd28c0336d0 comp rx=0 tx=0).stop 2024-09-21T03:45:32.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.608+0000 7fd295a38640 1 -- 172.21.15.38:0/2577225538 reap_dead start 2024-09-21T03:45:32.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.608+0000 7fd29749b640 1 -- 172.21.15.38:0/2577225538 shutdown_connections 2024-09-21T03:45:32.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.608+0000 7fd29749b640 1 -- 172.21.15.38:0/2577225538 >> 172.21.15.38:0/2577225538 conn(0x7fd2900b3ec0 msgr2=0x7fd29015c650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:32.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.608+0000 7fd29749b640 1 -- 172.21.15.38:0/2577225538 shutdown_connections 2024-09-21T03:45:32.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.608+0000 7fd29749b640 1 -- 172.21.15.38:0/2577225538 wait complete. 2024-09-21T03:45:32.621 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-09-21T03:45:32.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.737+0000 7f5c30de6640 1 -- 172.21.15.38:0/1191878343 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5c2c15a300 msgr2=0x7f5c2c06f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:32.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.737+0000 7f5c30de6640 1 --2- 172.21.15.38:0/1191878343 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5c2c15a300 0x7f5c2c06f4c0 secure :-1 s=READY pgs=2534 cs=0 l=1 rev1=1 crypto rx=0x7f5c2c075160 tx=0x7f5c1c02a530 comp rx=0 tx=0).stop 2024-09-21T03:45:32.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.737+0000 7f5c30de6640 1 -- 172.21.15.38:0/1191878343 shutdown_connections 2024-09-21T03:45:32.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.737+0000 7f5c30de6640 1 --2- 172.21.15.38:0/1191878343 >> v2:172.21.15.195:3300/0 conn(0x7f5c2c0b2ca0 0x7f5c2c0b30b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:32.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.737+0000 7f5c30de6640 1 --2- 172.21.15.38:0/1191878343 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5c2c15a300 0x7f5c2c06f4c0 unknown :-1 s=CLOSED pgs=2534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:32.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.737+0000 7f5c30de6640 1 -- 172.21.15.38:0/1191878343 >> 172.21.15.38:0/1191878343 conn(0x7f5c2c0cdca0 msgr2=0x7f5c2c0ce0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:32.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.737+0000 7f5c30de6640 1 -- 172.21.15.38:0/1191878343 shutdown_connections 2024-09-21T03:45:32.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.737+0000 7f5c30de6640 1 -- 172.21.15.38:0/1191878343 wait complete. 2024-09-21T03:45:32.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.739+0000 7f5c30de6640 1 Processor -- start 2024-09-21T03:45:32.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.739+0000 7f5c30de6640 1 -- start start 2024-09-21T03:45:32.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.740+0000 7f5c30de6640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5c2c0ae970 0x7f5c2c15e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:32.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.740+0000 7f5c30de6640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5c2c0b2ca0 0x7f5c2c0c70b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:32.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.740+0000 7f5c30de6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5c2c15a300 0x7f5c2c0c75f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:32.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.740+0000 7f5c30de6640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f5c2c160530 con 0x7f5c2c15a300 2024-09-21T03:45:32.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.740+0000 7f5c30de6640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f5c2c1603b0 con 0x7f5c2c0b2ca0 2024-09-21T03:45:32.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.740+0000 7f5c30de6640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f5c2c1606b0 con 0x7f5c2c0ae970 2024-09-21T03:45:32.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.740+0000 7f5c2ad9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5c2c15a300 0x7f5c2c0c75f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:32.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.741+0000 7f5c2ad9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5c2c15a300 0x7f5c2c0c75f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38276/0 (socket says 172.21.15.38:38276) 2024-09-21T03:45:32.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.741+0000 7f5c2ad9d640 1 -- 172.21.15.38:0/1865275700 learned_addr learned my addr 172.21.15.38:0/1865275700 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:45:32.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.741+0000 7f5c2a59c640 1 --2- 172.21.15.38:0/1865275700 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5c2c0ae970 0x7f5c2c15e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:32.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.741+0000 7f5c29d9b640 1 --2- 172.21.15.38:0/1865275700 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5c2c0b2ca0 0x7f5c2c0c70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:32.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.741+0000 7f5c2ad9d640 1 -- 172.21.15.38:0/1865275700 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5c2c0ae970 msgr2=0x7f5c2c15e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:32.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.741+0000 7f5c2ad9d640 1 --2- 172.21.15.38:0/1865275700 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5c2c0ae970 0x7f5c2c15e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:32.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.741+0000 7f5c2ad9d640 1 -- 172.21.15.38:0/1865275700 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5c2c0b2ca0 msgr2=0x7f5c2c0c70b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:32.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.741+0000 7f5c2ad9d640 1 --2- 172.21.15.38:0/1865275700 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5c2c0b2ca0 0x7f5c2c0c70b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:32.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.741+0000 7f5c2ad9d640 1 -- 172.21.15.38:0/1865275700 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c2c1f5820 con 0x7f5c2c15a300 2024-09-21T03:45:32.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.741+0000 7f5c29d9b640 1 --2- 172.21.15.38:0/1865275700 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5c2c0b2ca0 0x7f5c2c0c70b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:45:32.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.741+0000 7f5c2ad9d640 1 --2- 172.21.15.38:0/1865275700 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5c2c15a300 0x7f5c2c0c75f0 secure :-1 s=READY pgs=2535 cs=0 l=1 rev1=1 crypto rx=0x7f5c2400e990 tx=0x7f5c2400ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:32.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.741+0000 7f5c22ffd640 1 -- 172.21.15.38:0/1865275700 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c24004d20 con 0x7f5c2c15a300 2024-09-21T03:45:32.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.741+0000 7f5c22ffd640 1 -- 172.21.15.38:0/1865275700 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5c2400caa0 con 0x7f5c2c15a300 2024-09-21T03:45:32.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.741+0000 7f5c30de6640 1 -- 172.21.15.38:0/1865275700 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c2c1f5aa0 con 0x7f5c2c15a300 2024-09-21T03:45:32.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.741+0000 7f5c22ffd640 1 -- 172.21.15.38:0/1865275700 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c24005020 con 0x7f5c2c15a300 2024-09-21T03:45:32.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.741+0000 7f5c30de6640 1 -- 172.21.15.38:0/1865275700 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c2c1f6060 con 0x7f5c2c15a300 2024-09-21T03:45:32.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.742+0000 7f5c30de6640 1 -- 172.21.15.38:0/1865275700 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c2c160b60 con 0x7f5c2c15a300 2024-09-21T03:45:32.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.743+0000 7f5c22ffd640 1 -- 172.21.15.38:0/1865275700 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f5c240051c0 con 0x7f5c2c15a300 2024-09-21T03:45:32.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.744+0000 7f5c22ffd640 1 --2- 172.21.15.38:0/1865275700 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5c0c078bc0 0x7f5c0c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:32.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.744+0000 7f5c22ffd640 1 -- 172.21.15.38:0/1865275700 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 5421+0+0 (secure 0 0 0) 0x7f5c2409b470 con 0x7f5c2c15a300 2024-09-21T03:45:32.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.744+0000 7f5c2a59c640 1 --2- 172.21.15.38:0/1865275700 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5c0c078bc0 0x7f5c0c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:32.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.745+0000 7f5c2a59c640 1 --2- 172.21.15.38:0/1865275700 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5c0c078bc0 0x7f5c0c07b0d0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f5c1c034000 tx=0x7f5c1c004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:32.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.749+0000 7f5c22ffd640 1 -- 172.21.15.38:0/1865275700 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f5c2c1f5820 con 0x7f5c2c15a300 2024-09-21T03:45:32.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.936+0000 7f5c30de6640 1 -- 172.21.15.38:0/1865275700 --> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7f5c2c0ce150 con 0x7f5c0c078bc0 2024-09-21T03:45:32.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.937+0000 7f5c22ffd640 1 -- 172.21.15.38:0/1865275700 <== mgr.4117 v2:172.21.15.195:6800/1631901344 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f5c2c0ce150 con 0x7f5c0c078bc0 2024-09-21T03:45:32.939 INFO:teuthology.orchestra.run.smithi038.stderr:instructed osd(s) 2 to deep-scrub 2024-09-21T03:45:32.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.939+0000 7f5c30de6640 1 -- 172.21.15.38:0/1865275700 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5c0c078bc0 msgr2=0x7f5c0c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:32.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.939+0000 7f5c30de6640 1 --2- 172.21.15.38:0/1865275700 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5c0c078bc0 0x7f5c0c07b0d0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f5c1c034000 tx=0x7f5c1c004730 comp rx=0 tx=0).stop 2024-09-21T03:45:32.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.939+0000 7f5c30de6640 1 -- 172.21.15.38:0/1865275700 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5c2c15a300 msgr2=0x7f5c2c0c75f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:32.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.939+0000 7f5c30de6640 1 --2- 172.21.15.38:0/1865275700 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5c2c15a300 0x7f5c2c0c75f0 secure :-1 s=READY pgs=2535 cs=0 l=1 rev1=1 crypto rx=0x7f5c2400e990 tx=0x7f5c2400ee50 comp rx=0 tx=0).stop 2024-09-21T03:45:32.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.939+0000 7f5c30de6640 1 -- 172.21.15.38:0/1865275700 shutdown_connections 2024-09-21T03:45:32.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.939+0000 7f5c30de6640 1 --2- 172.21.15.38:0/1865275700 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5c2c15a300 0x7f5c2c0c75f0 unknown :-1 s=CLOSED pgs=2535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:32.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.939+0000 7f5c30de6640 1 --2- 172.21.15.38:0/1865275700 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5c2c0b2ca0 0x7f5c2c0c70b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:32.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.939+0000 7f5c30de6640 1 --2- 172.21.15.38:0/1865275700 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5c0c078bc0 0x7f5c0c07b0d0 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:32.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.939+0000 7f5c30de6640 1 --2- 172.21.15.38:0/1865275700 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5c2c0ae970 0x7f5c2c15e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:32.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.939+0000 7f5c30de6640 1 -- 172.21.15.38:0/1865275700 >> 172.21.15.38:0/1865275700 conn(0x7f5c2c0cdca0 msgr2=0x7f5c2c0b2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:32.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.939+0000 7f5c30de6640 1 -- 172.21.15.38:0/1865275700 shutdown_connections 2024-09-21T03:45:32.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:32.939+0000 7f5c30de6640 1 -- 172.21.15.38:0/1865275700 wait complete. 2024-09-21T03:45:32.952 INFO:tasks.ceph:Scrubbing osd.3 2024-09-21T03:45:32.952 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-09-21T03:45:33.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.071+0000 7f4269264640 1 -- 172.21.15.38:0/3937770892 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f426415e7d0 msgr2=0x7f42640b5f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:33.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.071+0000 7f4269264640 1 --2- 172.21.15.38:0/3937770892 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f426415e7d0 0x7f42640b5f60 secure :-1 s=READY pgs=2537 cs=0 l=1 rev1=1 crypto rx=0x7f4254005fd0 tx=0x7f425402aa90 comp rx=0 tx=0).stop 2024-09-21T03:45:33.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.071+0000 7f4269264640 1 -- 172.21.15.38:0/3937770892 shutdown_connections 2024-09-21T03:45:33.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.071+0000 7f4269264640 1 --2- 172.21.15.38:0/3937770892 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f42640b64a0 0x7f42641626d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:33.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.071+0000 7f4269264640 1 --2- 172.21.15.38:0/3937770892 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f426415e7d0 0x7f42640b5f60 unknown :-1 s=CLOSED pgs=2537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:33.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.071+0000 7f4269264640 1 -- 172.21.15.38:0/3937770892 >> 172.21.15.38:0/3937770892 conn(0x7f42640d44e0 msgr2=0x7f42640d48f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:33.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.071+0000 7f4269264640 1 -- 172.21.15.38:0/3937770892 shutdown_connections 2024-09-21T03:45:33.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.071+0000 7f4269264640 1 -- 172.21.15.38:0/3937770892 wait complete. 2024-09-21T03:45:33.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.073+0000 7f4269264640 1 Processor -- start 2024-09-21T03:45:33.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.073+0000 7f4269264640 1 -- start start 2024-09-21T03:45:33.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.075+0000 7f4269264640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f42640b64a0 0x7f426415cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:33.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.075+0000 7f4269264640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f426415e7d0 0x7f42641f8c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:33.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.075+0000 7f4269264640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4264162c10 0x7f42641f9140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:33.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.075+0000 7f4269264640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f426406ea10 con 0x7f426415e7d0 2024-09-21T03:45:33.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.075+0000 7f4269264640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f4264163230 con 0x7f4264162c10 2024-09-21T03:45:33.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.075+0000 7f4269264640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f426406eb90 con 0x7f42640b64a0 2024-09-21T03:45:33.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.075+0000 7f42627fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f426415e7d0 0x7f42641f8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:33.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.075+0000 7f42627fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f426415e7d0 0x7f42641f8c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38294/0 (socket says 172.21.15.38:38294) 2024-09-21T03:45:33.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.075+0000 7f42627fc640 1 -- 172.21.15.38:0/3200176341 learned_addr learned my addr 172.21.15.38:0/3200176341 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:45:33.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.075+0000 7f4262ffd640 1 --2- 172.21.15.38:0/3200176341 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f42640b64a0 0x7f426415cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:33.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.075+0000 7f42637fe640 1 --2- 172.21.15.38:0/3200176341 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4264162c10 0x7f42641f9140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:33.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.075+0000 7f42627fc640 1 -- 172.21.15.38:0/3200176341 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f42640b64a0 msgr2=0x7f426415cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:33.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.075+0000 7f42627fc640 1 --2- 172.21.15.38:0/3200176341 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f42640b64a0 0x7f426415cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:33.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.075+0000 7f42627fc640 1 -- 172.21.15.38:0/3200176341 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4264162c10 msgr2=0x7f42641f9140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:33.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.075+0000 7f42627fc640 1 --2- 172.21.15.38:0/3200176341 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4264162c10 0x7f42641f9140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:33.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.075+0000 7f42627fc640 1 -- 172.21.15.38:0/3200176341 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f426415d040 con 0x7f426415e7d0 2024-09-21T03:45:33.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.075+0000 7f42627fc640 1 --2- 172.21.15.38:0/3200176341 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f426415e7d0 0x7f42641f8c00 secure :-1 s=READY pgs=2538 cs=0 l=1 rev1=1 crypto rx=0x7f425800c970 tx=0x7f425800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:33.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.076+0000 7f424b7fe640 1 -- 172.21.15.38:0/3200176341 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4258007bb0 con 0x7f426415e7d0 2024-09-21T03:45:33.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.076+0000 7f424b7fe640 1 -- 172.21.15.38:0/3200176341 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4258007e40 con 0x7f426415e7d0 2024-09-21T03:45:33.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.076+0000 7f424b7fe640 1 -- 172.21.15.38:0/3200176341 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f425800f690 con 0x7f426415e7d0 2024-09-21T03:45:33.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.076+0000 7f4269264640 1 -- 172.21.15.38:0/3200176341 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42640abaa0 con 0x7f426415e7d0 2024-09-21T03:45:33.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.076+0000 7f4269264640 1 -- 172.21.15.38:0/3200176341 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42640b0230 con 0x7f426415e7d0 2024-09-21T03:45:33.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.077+0000 7f4269264640 1 -- 172.21.15.38:0/3200176341 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f426406f060 con 0x7f426415e7d0 2024-09-21T03:45:33.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.078+0000 7f424b7fe640 1 -- 172.21.15.38:0/3200176341 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f4258018400 con 0x7f426415e7d0 2024-09-21T03:45:33.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.078+0000 7f424b7fe640 1 --2- 172.21.15.38:0/3200176341 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4250078b90 0x7f425007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:33.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.078+0000 7f424b7fe640 1 -- 172.21.15.38:0/3200176341 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 5421+0+0 (secure 0 0 0) 0x7f425809bb90 con 0x7f426415e7d0 2024-09-21T03:45:33.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.078+0000 7f424b7fe640 1 --2- 172.21.15.38:0/3200176341 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x7f425007f120 0x7f42500815b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:33.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.078+0000 7f424b7fe640 1 -- 172.21.15.38:0/3200176341 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f425800de50 con 0x7f425007f120 2024-09-21T03:45:33.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.078+0000 7f424b7fe640 1 -- 172.21.15.38:0/3200176341 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_get_version_reply(handle=1 version=152) ==== 24+0+0 (secure 0 0 0) 0x7f425809c020 con 0x7f426415e7d0 2024-09-21T03:45:33.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.078+0000 7f4262ffd640 1 --2- 172.21.15.38:0/3200176341 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4250078b90 0x7f425007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:33.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.078+0000 7f42637fe640 1 --2- 172.21.15.38:0/3200176341 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x7f425007f120 0x7f42500815b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:33.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.079+0000 7f42637fe640 1 --2- 172.21.15.38:0/3200176341 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x7f425007f120 0x7f42500815b0 crc :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:33.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.079+0000 7f4262ffd640 1 --2- 172.21.15.38:0/3200176341 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4250078b90 0x7f425007b0a0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f4254035000 tx=0x7f4254038040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:33.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.079+0000 7f424b7fe640 1 -- 172.21.15.38:0/3200176341 <== osd.3 v2:172.21.15.38:6808/1286810294 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f425800de50 con 0x7f425007f120 2024-09-21T03:45:33.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.092+0000 7f4269264640 1 -- 172.21.15.38:0/3200176341 --> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f42640d3c40 con 0x7f425007f120 2024-09-21T03:45:33.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.092+0000 7f424b7fe640 1 -- 172.21.15.38:0/3200176341 <== osd.3 v2:172.21.15.38:6808/1286810294 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f42640d3c40 con 0x7f425007f120 2024-09-21T03:45:33.094 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-09-21T03:45:33.095 INFO:teuthology.orchestra.run.smithi038.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-21T03:45:33.095 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-09-21T03:45:33.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.093+0000 7f4269264640 1 -- 172.21.15.38:0/3200176341 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x7f425007f120 msgr2=0x7f42500815b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:33.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.093+0000 7f4269264640 1 --2- 172.21.15.38:0/3200176341 >> [v2:172.21.15.38:6808/1286810294,v1:172.21.15.38:6809/1286810294] conn(0x7f425007f120 0x7f42500815b0 crc :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:33.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.093+0000 7f4269264640 1 -- 172.21.15.38:0/3200176341 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4250078b90 msgr2=0x7f425007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:33.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.093+0000 7f4269264640 1 --2- 172.21.15.38:0/3200176341 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f4250078b90 0x7f425007b0a0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f4254035000 tx=0x7f4254038040 comp rx=0 tx=0).stop 2024-09-21T03:45:33.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.093+0000 7f4269264640 1 -- 172.21.15.38:0/3200176341 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f426415e7d0 msgr2=0x7f42641f8c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:33.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.093+0000 7f4269264640 1 --2- 172.21.15.38:0/3200176341 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f426415e7d0 0x7f42641f8c00 secure :-1 s=READY pgs=2538 cs=0 l=1 rev1=1 crypto rx=0x7f425800c970 tx=0x7f425800ce30 comp rx=0 tx=0).stop 2024-09-21T03:45:33.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.093+0000 7f42637fe640 1 -- 172.21.15.38:0/3200176341 reap_dead start 2024-09-21T03:45:33.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.093+0000 7f4269264640 1 -- 172.21.15.38:0/3200176341 shutdown_connections 2024-09-21T03:45:33.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.093+0000 7f4269264640 1 -- 172.21.15.38:0/3200176341 >> 172.21.15.38:0/3200176341 conn(0x7f42640d44e0 msgr2=0x7f4264160fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:33.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.093+0000 7f4269264640 1 -- 172.21.15.38:0/3200176341 shutdown_connections 2024-09-21T03:45:33.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.093+0000 7f4269264640 1 -- 172.21.15.38:0/3200176341 wait complete. 2024-09-21T03:45:33.105 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-09-21T03:45:33.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.221+0000 7f075840e640 1 -- 172.21.15.38:0/3529653388 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f07500ccc40 msgr2=0x7f075015ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:33.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.221+0000 7f075840e640 1 --2- 172.21.15.38:0/3529653388 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f07500ccc40 0x7f075015ca90 secure :-1 s=READY pgs=2539 cs=0 l=1 rev1=1 crypto rx=0x7f074c005fd0 tx=0x7f074c02a540 comp rx=0 tx=0).stop 2024-09-21T03:45:33.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.221+0000 7f075840e640 1 -- 172.21.15.38:0/3529653388 shutdown_connections 2024-09-21T03:45:33.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.221+0000 7f075840e640 1 --2- 172.21.15.38:0/3529653388 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f07500d0d80 0x7f07500b5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:33.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.221+0000 7f075840e640 1 --2- 172.21.15.38:0/3529653388 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f07500ccc40 0x7f075015ca90 unknown :-1 s=CLOSED pgs=2539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:33.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.221+0000 7f075840e640 1 -- 172.21.15.38:0/3529653388 >> 172.21.15.38:0/3529653388 conn(0x7f07500b46e0 msgr2=0x7f07500d23e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:33.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.221+0000 7f075840e640 1 -- 172.21.15.38:0/3529653388 shutdown_connections 2024-09-21T03:45:33.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.221+0000 7f075840e640 1 -- 172.21.15.38:0/3529653388 wait complete. 2024-09-21T03:45:33.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.223+0000 7f075840e640 1 Processor -- start 2024-09-21T03:45:33.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.223+0000 7f075840e640 1 -- start start 2024-09-21T03:45:33.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.224+0000 7f075840e640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f07500ccc40 0x7f075006def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:33.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.224+0000 7f075840e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f07500cd440 0x7f07500cba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:33.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.224+0000 7f075840e640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f07501f8170 0x7f07500cbf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:33.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.224+0000 7f075840e640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f07500b6650 con 0x7f07500cd440 2024-09-21T03:45:33.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.224+0000 7f075840e640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f07500b64d0 con 0x7f07500ccc40 2024-09-21T03:45:33.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.224+0000 7f075840e640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f07500b67d0 con 0x7f07501f8170 2024-09-21T03:45:33.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.224+0000 7f07559a9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f07500cd440 0x7f07500cba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:33.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.224+0000 7f07559a9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f07500cd440 0x7f07500cba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38326/0 (socket says 172.21.15.38:38326) 2024-09-21T03:45:33.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.224+0000 7f07559a9640 1 -- 172.21.15.38:0/2597842466 learned_addr learned my addr 172.21.15.38:0/2597842466 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:45:33.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.225+0000 7f07561aa640 1 --2- 172.21.15.38:0/2597842466 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f07500ccc40 0x7f075006def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:33.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.225+0000 7f07569ab640 1 --2- 172.21.15.38:0/2597842466 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f07501f8170 0x7f07500cbf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:33.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.225+0000 7f07559a9640 1 -- 172.21.15.38:0/2597842466 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f07501f8170 msgr2=0x7f07500cbf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:33.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.225+0000 7f07559a9640 1 --2- 172.21.15.38:0/2597842466 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f07501f8170 0x7f07500cbf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:33.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.225+0000 7f07559a9640 1 -- 172.21.15.38:0/2597842466 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f07500ccc40 msgr2=0x7f075006def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:33.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.225+0000 7f07559a9640 1 --2- 172.21.15.38:0/2597842466 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f07500ccc40 0x7f075006def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:33.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.225+0000 7f07559a9640 1 -- 172.21.15.38:0/2597842466 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f075006ee90 con 0x7f07500cd440 2024-09-21T03:45:33.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.225+0000 7f07569ab640 1 --2- 172.21.15.38:0/2597842466 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f07501f8170 0x7f07500cbf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:45:33.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.225+0000 7f07559a9640 1 --2- 172.21.15.38:0/2597842466 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f07500cd440 0x7f07500cba50 secure :-1 s=READY pgs=2540 cs=0 l=1 rev1=1 crypto rx=0x7f074000c3f0 tx=0x7f074000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:33.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.225+0000 7f0746ffd640 1 -- 172.21.15.38:0/2597842466 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07400052d0 con 0x7f07500cd440 2024-09-21T03:45:33.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.225+0000 7f0746ffd640 1 -- 172.21.15.38:0/2597842466 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0740007020 con 0x7f07500cd440 2024-09-21T03:45:33.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.225+0000 7f075840e640 1 -- 172.21.15.38:0/2597842466 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f075006e8d0 con 0x7f07500cd440 2024-09-21T03:45:33.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.225+0000 7f0746ffd640 1 -- 172.21.15.38:0/2597842466 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0740007470 con 0x7f07500cd440 2024-09-21T03:45:33.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.225+0000 7f075840e640 1 -- 172.21.15.38:0/2597842466 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f075015cfe0 con 0x7f07500cd440 2024-09-21T03:45:33.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.226+0000 7f075840e640 1 -- 172.21.15.38:0/2597842466 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f075015c660 con 0x7f07500cd440 2024-09-21T03:45:33.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.227+0000 7f0746ffd640 1 -- 172.21.15.38:0/2597842466 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f07400067d0 con 0x7f07500cd440 2024-09-21T03:45:33.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.228+0000 7f0746ffd640 1 --2- 172.21.15.38:0/2597842466 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0734078b90 0x7f073407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:33.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.228+0000 7f0746ffd640 1 -- 172.21.15.38:0/2597842466 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 5421+0+0 (secure 0 0 0) 0x7f074009a010 con 0x7f07500cd440 2024-09-21T03:45:33.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.228+0000 7f07561aa640 1 --2- 172.21.15.38:0/2597842466 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0734078b90 0x7f073407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:33.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.229+0000 7f07561aa640 1 --2- 172.21.15.38:0/2597842466 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0734078b90 0x7f073407b0a0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f074c033000 tx=0x7f074c004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:33.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.233+0000 7f0746ffd640 1 -- 172.21.15.38:0/2597842466 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f075015c660 con 0x7f07500cd440 2024-09-21T03:45:33.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.421+0000 7f075840e640 1 -- 172.21.15.38:0/2597842466 --> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7f07500b6cf0 con 0x7f0734078b90 2024-09-21T03:45:33.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.422+0000 7f0746ffd640 1 -- 172.21.15.38:0/2597842466 <== mgr.4117 v2:172.21.15.195:6800/1631901344 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f07500b6cf0 con 0x7f0734078b90 2024-09-21T03:45:33.424 INFO:teuthology.orchestra.run.smithi038.stderr:instructed osd(s) 3 to deep-scrub 2024-09-21T03:45:33.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.424+0000 7f075840e640 1 -- 172.21.15.38:0/2597842466 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0734078b90 msgr2=0x7f073407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:33.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.424+0000 7f075840e640 1 --2- 172.21.15.38:0/2597842466 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0734078b90 0x7f073407b0a0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f074c033000 tx=0x7f074c004730 comp rx=0 tx=0).stop 2024-09-21T03:45:33.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.424+0000 7f075840e640 1 -- 172.21.15.38:0/2597842466 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f07500cd440 msgr2=0x7f07500cba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:33.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.424+0000 7f075840e640 1 --2- 172.21.15.38:0/2597842466 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f07500cd440 0x7f07500cba50 secure :-1 s=READY pgs=2540 cs=0 l=1 rev1=1 crypto rx=0x7f074000c3f0 tx=0x7f074000c8b0 comp rx=0 tx=0).stop 2024-09-21T03:45:33.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.424+0000 7f075840e640 1 -- 172.21.15.38:0/2597842466 shutdown_connections 2024-09-21T03:45:33.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.424+0000 7f075840e640 1 --2- 172.21.15.38:0/2597842466 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0734078b90 0x7f073407b0a0 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:33.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.424+0000 7f075840e640 1 --2- 172.21.15.38:0/2597842466 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f07501f8170 0x7f07500cbf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:33.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.424+0000 7f075840e640 1 --2- 172.21.15.38:0/2597842466 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f07500cd440 0x7f07500cba50 unknown :-1 s=CLOSED pgs=2540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:33.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.424+0000 7f075840e640 1 --2- 172.21.15.38:0/2597842466 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f07500ccc40 0x7f075006def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:33.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.424+0000 7f075840e640 1 -- 172.21.15.38:0/2597842466 >> 172.21.15.38:0/2597842466 conn(0x7f07500b46e0 msgr2=0x7f07500b78c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:33.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.424+0000 7f075840e640 1 -- 172.21.15.38:0/2597842466 shutdown_connections 2024-09-21T03:45:33.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.424+0000 7f075840e640 1 -- 172.21.15.38:0/2597842466 wait complete. 2024-09-21T03:45:33.437 INFO:tasks.ceph:Scrubbing osd.4 2024-09-21T03:45:33.437 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-09-21T03:45:33.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.556+0000 7f58b3b39640 1 -- 172.21.15.38:0/327365654 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f58ac06e900 msgr2=0x7f58ac06ed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:33.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.556+0000 7f58b3b39640 1 --2- 172.21.15.38:0/327365654 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f58ac06e900 0x7f58ac06ed10 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f58a4005fd0 tx=0x7f58a402a8b0 comp rx=0 tx=0).stop 2024-09-21T03:45:33.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.556+0000 7f58b3b39640 1 -- 172.21.15.38:0/327365654 shutdown_connections 2024-09-21T03:45:33.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.556+0000 7f58b3b39640 1 --2- 172.21.15.38:0/327365654 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f58ac06e900 0x7f58ac06ed10 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:33.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.556+0000 7f58b3b39640 1 --2- 172.21.15.38:0/327365654 >> v2:172.21.15.195:3300/0 conn(0x7f58ac071af0 0x7f58ac0ce2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:33.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.556+0000 7f58b3b39640 1 -- 172.21.15.38:0/327365654 >> 172.21.15.38:0/327365654 conn(0x7f58ac09a9e0 msgr2=0x7f58ac0b6050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:33.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.556+0000 7f58b3b39640 1 -- 172.21.15.38:0/327365654 shutdown_connections 2024-09-21T03:45:33.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.556+0000 7f58b3b39640 1 -- 172.21.15.38:0/327365654 wait complete. 2024-09-21T03:45:33.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.558+0000 7f58b3b39640 1 Processor -- start 2024-09-21T03:45:33.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.558+0000 7f58b3b39640 1 -- start start 2024-09-21T03:45:33.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.559+0000 7f58b3b39640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f58ac06e900 0x7f58ac09b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:33.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.559+0000 7f58b3b39640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f58ac071af0 0x7f58ac1f3e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:33.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.559+0000 7f58b3b39640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f58ac15b4d0 0x7f58ac15dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:33.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.559+0000 7f58b3b39640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f58ac06f250 con 0x7f58ac071af0 2024-09-21T03:45:33.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.559+0000 7f58b3b39640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f58ac0cea20 con 0x7f58ac15b4d0 2024-09-21T03:45:33.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.559+0000 7f58b3b39640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f58ac06f3d0 con 0x7f58ac06e900 2024-09-21T03:45:33.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.559+0000 7f58b10d4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f58ac071af0 0x7f58ac1f3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:33.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.559+0000 7f58b10d4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f58ac071af0 0x7f58ac1f3e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38340/0 (socket says 172.21.15.38:38340) 2024-09-21T03:45:33.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.559+0000 7f58b10d4640 1 -- 172.21.15.38:0/283849955 learned_addr learned my addr 172.21.15.38:0/283849955 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:45:33.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.559+0000 7f58b18d5640 1 --2- 172.21.15.38:0/283849955 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f58ac06e900 0x7f58ac09b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:33.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.559+0000 7f58b20d6640 1 --2- 172.21.15.38:0/283849955 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f58ac15b4d0 0x7f58ac15dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:33.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.560+0000 7f58b10d4640 1 -- 172.21.15.38:0/283849955 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f58ac06e900 msgr2=0x7f58ac09b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:33.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.560+0000 7f58b10d4640 1 --2- 172.21.15.38:0/283849955 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f58ac06e900 0x7f58ac09b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:33.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.560+0000 7f58b10d4640 1 -- 172.21.15.38:0/283849955 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f58ac15b4d0 msgr2=0x7f58ac15dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:33.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.560+0000 7f58b10d4640 1 --2- 172.21.15.38:0/283849955 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f58ac15b4d0 0x7f58ac15dcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:33.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.560+0000 7f58b10d4640 1 -- 172.21.15.38:0/283849955 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58ac09b6a0 con 0x7f58ac071af0 2024-09-21T03:45:33.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.560+0000 7f58b10d4640 1 --2- 172.21.15.38:0/283849955 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f58ac071af0 0x7f58ac1f3e50 secure :-1 s=READY pgs=2541 cs=0 l=1 rev1=1 crypto rx=0x7f589c00e9f0 tx=0x7f589c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:33.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.560+0000 7f58a27fc640 1 -- 172.21.15.38:0/283849955 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f589c00cd80 con 0x7f58ac071af0 2024-09-21T03:45:33.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.560+0000 7f58a27fc640 1 -- 172.21.15.38:0/283849955 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f589c004020 con 0x7f58ac071af0 2024-09-21T03:45:33.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.560+0000 7f58b3b39640 1 -- 172.21.15.38:0/283849955 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58ac09bbb0 con 0x7f58ac071af0 2024-09-21T03:45:33.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.560+0000 7f58a27fc640 1 -- 172.21.15.38:0/283849955 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f589c004470 con 0x7f58ac071af0 2024-09-21T03:45:33.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.560+0000 7f58b3b39640 1 -- 172.21.15.38:0/283849955 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58ac09c170 con 0x7f58ac071af0 2024-09-21T03:45:33.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.561+0000 7f58b3b39640 1 -- 172.21.15.38:0/283849955 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f58ac06f850 con 0x7f58ac071af0 2024-09-21T03:45:33.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.562+0000 7f58a27fc640 1 -- 172.21.15.38:0/283849955 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f589c0035f0 con 0x7f58ac071af0 2024-09-21T03:45:33.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.563+0000 7f58a27fc640 1 --2- 172.21.15.38:0/283849955 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5890078b90 0x7f589007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:33.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.563+0000 7f58a27fc640 1 -- 172.21.15.38:0/283849955 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 5421+0+0 (secure 0 0 0) 0x7f589c09b890 con 0x7f58ac071af0 2024-09-21T03:45:33.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.563+0000 7f58a27fc640 1 --2- 172.21.15.38:0/283849955 >> [v2:172.21.15.195:6802/3401837571,v1:172.21.15.195:6803/3401837571] conn(0x7f589007f120 0x7f58900815b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:33.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.563+0000 7f58a27fc640 1 -- 172.21.15.38:0/283849955 --> [v2:172.21.15.195:6802/3401837571,v1:172.21.15.195:6803/3401837571] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f589c023e50 con 0x7f589007f120 2024-09-21T03:45:33.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.563+0000 7f58a27fc640 1 -- 172.21.15.38:0/283849955 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_get_version_reply(handle=1 version=152) ==== 24+0+0 (secure 0 0 0) 0x7f589c09bd20 con 0x7f58ac071af0 2024-09-21T03:45:33.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.563+0000 7f58b18d5640 1 --2- 172.21.15.38:0/283849955 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5890078b90 0x7f589007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:33.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.563+0000 7f58b20d6640 1 --2- 172.21.15.38:0/283849955 >> [v2:172.21.15.195:6802/3401837571,v1:172.21.15.195:6803/3401837571] conn(0x7f589007f120 0x7f58900815b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:33.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.564+0000 7f58b18d5640 1 --2- 172.21.15.38:0/283849955 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5890078b90 0x7f589007b0a0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f58a8001720 tx=0x7f58a8007e90 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:33.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.564+0000 7f58b20d6640 1 --2- 172.21.15.38:0/283849955 >> [v2:172.21.15.195:6802/3401837571,v1:172.21.15.195:6803/3401837571] conn(0x7f589007f120 0x7f58900815b0 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:33.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.565+0000 7f58a27fc640 1 -- 172.21.15.38:0/283849955 <== osd.4 v2:172.21.15.195:6802/3401837571 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f589c023e50 con 0x7f589007f120 2024-09-21T03:45:33.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.577+0000 7f58b3b39640 1 -- 172.21.15.38:0/283849955 --> [v2:172.21.15.195:6802/3401837571,v1:172.21.15.195:6803/3401837571] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f58ac09b960 con 0x7f589007f120 2024-09-21T03:45:33.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.578+0000 7f58a27fc640 1 -- 172.21.15.38:0/283849955 <== osd.4 v2:172.21.15.195:6802/3401837571 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f58ac09b960 con 0x7f589007f120 2024-09-21T03:45:33.580 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-09-21T03:45:33.580 INFO:teuthology.orchestra.run.smithi038.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-21T03:45:33.580 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-09-21T03:45:33.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.578+0000 7f58b3b39640 1 -- 172.21.15.38:0/283849955 >> [v2:172.21.15.195:6802/3401837571,v1:172.21.15.195:6803/3401837571] conn(0x7f589007f120 msgr2=0x7f58900815b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:33.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.578+0000 7f58b3b39640 1 --2- 172.21.15.38:0/283849955 >> [v2:172.21.15.195:6802/3401837571,v1:172.21.15.195:6803/3401837571] conn(0x7f589007f120 0x7f58900815b0 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:33.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.578+0000 7f58b3b39640 1 -- 172.21.15.38:0/283849955 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5890078b90 msgr2=0x7f589007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:33.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.578+0000 7f58b3b39640 1 --2- 172.21.15.38:0/283849955 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f5890078b90 0x7f589007b0a0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f58a8001720 tx=0x7f58a8007e90 comp rx=0 tx=0).stop 2024-09-21T03:45:33.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.578+0000 7f58b3b39640 1 -- 172.21.15.38:0/283849955 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f58ac071af0 msgr2=0x7f58ac1f3e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:33.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.578+0000 7f58b3b39640 1 --2- 172.21.15.38:0/283849955 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f58ac071af0 0x7f58ac1f3e50 secure :-1 s=READY pgs=2541 cs=0 l=1 rev1=1 crypto rx=0x7f589c00e9f0 tx=0x7f589c00eeb0 comp rx=0 tx=0).stop 2024-09-21T03:45:33.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.578+0000 7f58b20d6640 1 -- 172.21.15.38:0/283849955 reap_dead start 2024-09-21T03:45:33.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.578+0000 7f58b3b39640 1 -- 172.21.15.38:0/283849955 shutdown_connections 2024-09-21T03:45:33.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.578+0000 7f58b3b39640 1 -- 172.21.15.38:0/283849955 >> 172.21.15.38:0/283849955 conn(0x7f58ac09a9e0 msgr2=0x7f58ac0ccf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:33.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.578+0000 7f58b3b39640 1 -- 172.21.15.38:0/283849955 shutdown_connections 2024-09-21T03:45:33.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.578+0000 7f58b3b39640 1 -- 172.21.15.38:0/283849955 wait complete. 2024-09-21T03:45:33.591 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-09-21T03:45:33.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.706+0000 7f62d344d640 1 -- 172.21.15.38:0/315013093 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f62cc0d0d80 msgr2=0x7f62cc167550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:33.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.706+0000 7f62d344d640 1 --2- 172.21.15.38:0/315013093 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f62cc0d0d80 0x7f62cc167550 secure :-1 s=READY pgs=2543 cs=0 l=1 rev1=1 crypto rx=0x7f62c4009830 tx=0x7f62c402a380 comp rx=0 tx=0).stop 2024-09-21T03:45:33.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.706+0000 7f62d344d640 1 -- 172.21.15.38:0/315013093 shutdown_connections 2024-09-21T03:45:33.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.706+0000 7f62d344d640 1 --2- 172.21.15.38:0/315013093 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f62cc0d0d80 0x7f62cc167550 unknown :-1 s=CLOSED pgs=2543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:33.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.706+0000 7f62d344d640 1 -- 172.21.15.38:0/315013093 >> 172.21.15.38:0/315013093 conn(0x7f62cc0b46e0 msgr2=0x7f62cc0d23e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:33.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.706+0000 7f62d344d640 1 -- 172.21.15.38:0/315013093 shutdown_connections 2024-09-21T03:45:33.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.707+0000 7f62d344d640 1 -- 172.21.15.38:0/315013093 wait complete. 2024-09-21T03:45:33.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.708+0000 7f62d344d640 1 Processor -- start 2024-09-21T03:45:33.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.708+0000 7f62d344d640 1 -- start start 2024-09-21T03:45:33.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.710+0000 7f62d344d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f62cc0ccc40 0x7f62cc1515a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:33.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.710+0000 7f62d344d640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f62cc15ca90 0x7f62cc164890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:33.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.710+0000 7f62d344d640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f62cc071720 0x7f62cc164dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:33.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.710+0000 7f62d344d640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f62cc167e20 con 0x7f62cc0ccc40 2024-09-21T03:45:33.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.710+0000 7f62d344d640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f62cc167ca0 con 0x7f62cc15ca90 2024-09-21T03:45:33.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.710+0000 7f62d344d640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f62cc167fa0 con 0x7f62cc071720 2024-09-21T03:45:33.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.710+0000 7f62d11e9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f62cc0ccc40 0x7f62cc1515a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:33.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.710+0000 7f62d11e9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f62cc0ccc40 0x7f62cc1515a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38356/0 (socket says 172.21.15.38:38356) 2024-09-21T03:45:33.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.710+0000 7f62d11e9640 1 -- 172.21.15.38:0/2964168805 learned_addr learned my addr 172.21.15.38:0/2964168805 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:45:33.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.710+0000 7f62d09e8640 1 --2- 172.21.15.38:0/2964168805 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f62cc15ca90 0x7f62cc164890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:33.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.710+0000 7f62d19ea640 1 --2- 172.21.15.38:0/2964168805 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f62cc071720 0x7f62cc164dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:33.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.710+0000 7f62d11e9640 1 -- 172.21.15.38:0/2964168805 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f62cc071720 msgr2=0x7f62cc164dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:33.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.710+0000 7f62d11e9640 1 --2- 172.21.15.38:0/2964168805 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f62cc071720 0x7f62cc164dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:33.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.710+0000 7f62d11e9640 1 -- 172.21.15.38:0/2964168805 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f62cc15ca90 msgr2=0x7f62cc164890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:33.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.710+0000 7f62d11e9640 1 --2- 172.21.15.38:0/2964168805 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f62cc15ca90 0x7f62cc164890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:33.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.710+0000 7f62d11e9640 1 -- 172.21.15.38:0/2964168805 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62cc1f5bb0 con 0x7f62cc0ccc40 2024-09-21T03:45:33.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.710+0000 7f62d11e9640 1 --2- 172.21.15.38:0/2964168805 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f62cc0ccc40 0x7f62cc1515a0 secure :-1 s=READY pgs=2544 cs=0 l=1 rev1=1 crypto rx=0x7f62bc00bab0 tx=0x7f62bc00bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:33.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.710+0000 7f62c9ffb640 1 -- 172.21.15.38:0/2964168805 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62bc00cc60 con 0x7f62cc0ccc40 2024-09-21T03:45:33.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.711+0000 7f62c9ffb640 1 -- 172.21.15.38:0/2964168805 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f62bc003970 con 0x7f62cc0ccc40 2024-09-21T03:45:33.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.711+0000 7f62d344d640 1 -- 172.21.15.38:0/2964168805 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62cc1f5e90 con 0x7f62cc0ccc40 2024-09-21T03:45:33.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.711+0000 7f62c9ffb640 1 -- 172.21.15.38:0/2964168805 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62bc003dc0 con 0x7f62cc0ccc40 2024-09-21T03:45:33.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.711+0000 7f62d344d640 1 -- 172.21.15.38:0/2964168805 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62cc1f6450 con 0x7f62cc0ccc40 2024-09-21T03:45:33.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.712+0000 7f62d344d640 1 -- 172.21.15.38:0/2964168805 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62cc0cf940 con 0x7f62cc0ccc40 2024-09-21T03:45:33.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.712+0000 7f62c9ffb640 1 -- 172.21.15.38:0/2964168805 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f62bc004020 con 0x7f62cc0ccc40 2024-09-21T03:45:33.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.713+0000 7f62c9ffb640 1 --2- 172.21.15.38:0/2964168805 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f62b8078b90 0x7f62b807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:33.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.713+0000 7f62c9ffb640 1 -- 172.21.15.38:0/2964168805 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 5421+0+0 (secure 0 0 0) 0x7f62bc09aa60 con 0x7f62cc0ccc40 2024-09-21T03:45:33.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.713+0000 7f62d09e8640 1 --2- 172.21.15.38:0/2964168805 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f62b8078b90 0x7f62b807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:33.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.714+0000 7f62d09e8640 1 --2- 172.21.15.38:0/2964168805 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f62b8078b90 0x7f62b807b0a0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f62cc0b3850 tx=0x7f62c0009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:33.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.717+0000 7f62c9ffb640 1 -- 172.21.15.38:0/2964168805 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f62bc062970 con 0x7f62cc0ccc40 2024-09-21T03:45:33.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.906+0000 7f62d344d640 1 -- 172.21.15.38:0/2964168805 --> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) -- 0x7f62cc0008d0 con 0x7f62b8078b90 2024-09-21T03:45:33.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.907+0000 7f62c9ffb640 1 -- 172.21.15.38:0/2964168805 <== mgr.4117 v2:172.21.15.195:6800/1631901344 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f62cc0008d0 con 0x7f62b8078b90 2024-09-21T03:45:33.909 INFO:teuthology.orchestra.run.smithi038.stderr:instructed osd(s) 4 to deep-scrub 2024-09-21T03:45:33.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.909+0000 7f62d344d640 1 -- 172.21.15.38:0/2964168805 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f62b8078b90 msgr2=0x7f62b807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:33.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.909+0000 7f62d344d640 1 --2- 172.21.15.38:0/2964168805 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f62b8078b90 0x7f62b807b0a0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f62cc0b3850 tx=0x7f62c0009040 comp rx=0 tx=0).stop 2024-09-21T03:45:33.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.909+0000 7f62d344d640 1 -- 172.21.15.38:0/2964168805 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f62cc0ccc40 msgr2=0x7f62cc1515a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:33.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.909+0000 7f62d344d640 1 --2- 172.21.15.38:0/2964168805 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f62cc0ccc40 0x7f62cc1515a0 secure :-1 s=READY pgs=2544 cs=0 l=1 rev1=1 crypto rx=0x7f62bc00bab0 tx=0x7f62bc00bf70 comp rx=0 tx=0).stop 2024-09-21T03:45:33.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.909+0000 7f62d344d640 1 -- 172.21.15.38:0/2964168805 shutdown_connections 2024-09-21T03:45:33.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.909+0000 7f62d344d640 1 --2- 172.21.15.38:0/2964168805 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f62b8078b90 0x7f62b807b0a0 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:33.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.909+0000 7f62d344d640 1 --2- 172.21.15.38:0/2964168805 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f62cc071720 0x7f62cc164dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:33.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.909+0000 7f62d344d640 1 --2- 172.21.15.38:0/2964168805 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f62cc15ca90 0x7f62cc164890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:33.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.909+0000 7f62d344d640 1 --2- 172.21.15.38:0/2964168805 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f62cc0ccc40 0x7f62cc1515a0 unknown :-1 s=CLOSED pgs=2544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:33.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.909+0000 7f62d344d640 1 -- 172.21.15.38:0/2964168805 >> 172.21.15.38:0/2964168805 conn(0x7f62cc0b46e0 msgr2=0x7f62cc0c5ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:33.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.909+0000 7f62d344d640 1 -- 172.21.15.38:0/2964168805 shutdown_connections 2024-09-21T03:45:33.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:33.909+0000 7f62d344d640 1 -- 172.21.15.38:0/2964168805 wait complete. 2024-09-21T03:45:33.922 INFO:tasks.ceph:Scrubbing osd.5 2024-09-21T03:45:33.922 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-09-21T03:45:34.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.041+0000 7fc988238640 1 -- 172.21.15.38:0/2394356572 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc98006def0 msgr2=0x7fc98006e300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:34.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.041+0000 7fc988238640 1 --2- 172.21.15.38:0/2394356572 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc98006def0 0x7fc98006e300 secure :-1 s=READY pgs=2546 cs=0 l=1 rev1=1 crypto rx=0x7fc980072e80 tx=0x7fc97802a530 comp rx=0 tx=0).stop 2024-09-21T03:45:34.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.041+0000 7fc988238640 1 -- 172.21.15.38:0/2394356572 shutdown_connections 2024-09-21T03:45:34.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.041+0000 7fc988238640 1 --2- 172.21.15.38:0/2394356572 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc98006def0 0x7fc98006e300 unknown :-1 s=CLOSED pgs=2546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:34.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.041+0000 7fc988238640 1 --2- 172.21.15.38:0/2394356572 >> v2:172.21.15.195:3300/0 conn(0x7fc980070940 0x7fc9800cbd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:34.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.041+0000 7fc988238640 1 -- 172.21.15.38:0/2394356572 >> 172.21.15.38:0/2394356572 conn(0x7fc9800ccac0 msgr2=0x7fc9800cced0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:34.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.041+0000 7fc988238640 1 -- 172.21.15.38:0/2394356572 shutdown_connections 2024-09-21T03:45:34.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.041+0000 7fc988238640 1 -- 172.21.15.38:0/2394356572 wait complete. 2024-09-21T03:45:34.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.043+0000 7fc988238640 1 Processor -- start 2024-09-21T03:45:34.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.043+0000 7fc988238640 1 -- start start 2024-09-21T03:45:34.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.044+0000 7fc988238640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc98006def0 0x7fc98015c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:34.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.045+0000 7fc988238640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc980070940 0x7fc9800c99f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:34.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.045+0000 7fc988238640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc9800cd790 0x7fc98009cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:34.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.045+0000 7fc988238640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fc98006f5c0 con 0x7fc980070940 2024-09-21T03:45:34.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.045+0000 7fc988238640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fc98006f440 con 0x7fc98006def0 2024-09-21T03:45:34.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.045+0000 7fc988238640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fc98006f740 con 0x7fc9800cd790 2024-09-21T03:45:34.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.045+0000 7fc9857d3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc980070940 0x7fc9800c99f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:34.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.045+0000 7fc9857d3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc980070940 0x7fc9800c99f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38378/0 (socket says 172.21.15.38:38378) 2024-09-21T03:45:34.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.045+0000 7fc9857d3640 1 -- 172.21.15.38:0/28849263 learned_addr learned my addr 172.21.15.38:0/28849263 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:45:34.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.045+0000 7fc985fd4640 1 --2- 172.21.15.38:0/28849263 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc98006def0 0x7fc98015c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:34.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.045+0000 7fc9867d5640 1 --2- 172.21.15.38:0/28849263 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc9800cd790 0x7fc98009cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:34.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.045+0000 7fc9857d3640 1 -- 172.21.15.38:0/28849263 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc9800cd790 msgr2=0x7fc98009cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:34.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.045+0000 7fc9857d3640 1 --2- 172.21.15.38:0/28849263 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc9800cd790 0x7fc98009cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:34.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.045+0000 7fc9857d3640 1 -- 172.21.15.38:0/28849263 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc98006def0 msgr2=0x7fc98015c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:34.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.045+0000 7fc9857d3640 1 --2- 172.21.15.38:0/28849263 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc98006def0 0x7fc98015c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:34.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.045+0000 7fc9857d3640 1 -- 172.21.15.38:0/28849263 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9800c3b90 con 0x7fc980070940 2024-09-21T03:45:34.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.045+0000 7fc9857d3640 1 --2- 172.21.15.38:0/28849263 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc980070940 0x7fc9800c99f0 secure :-1 s=READY pgs=2547 cs=0 l=1 rev1=1 crypto rx=0x7fc97000d740 tx=0x7fc97000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:34.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.045+0000 7fc9767fc640 1 -- 172.21.15.38:0/28849263 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc970017070 con 0x7fc980070940 2024-09-21T03:45:34.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.045+0000 7fc9767fc640 1 -- 172.21.15.38:0/28849263 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc970009d20 con 0x7fc980070940 2024-09-21T03:45:34.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.045+0000 7fc988238640 1 -- 172.21.15.38:0/28849263 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc98015b060 con 0x7fc980070940 2024-09-21T03:45:34.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.045+0000 7fc9767fc640 1 -- 172.21.15.38:0/28849263 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9700106c0 con 0x7fc980070940 2024-09-21T03:45:34.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.046+0000 7fc988238640 1 -- 172.21.15.38:0/28849263 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9800c44b0 con 0x7fc980070940 2024-09-21T03:45:34.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.046+0000 7fc988238640 1 -- 172.21.15.38:0/28849263 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc9800c4260 con 0x7fc980070940 2024-09-21T03:45:34.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.047+0000 7fc9767fc640 1 -- 172.21.15.38:0/28849263 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc97001a070 con 0x7fc980070940 2024-09-21T03:45:34.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.048+0000 7fc9767fc640 1 --2- 172.21.15.38:0/28849263 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc964078bc0 0x7fc96407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:34.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.048+0000 7fc9767fc640 1 -- 172.21.15.38:0/28849263 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 5421+0+0 (secure 0 0 0) 0x7fc97009b3a0 con 0x7fc980070940 2024-09-21T03:45:34.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.048+0000 7fc9767fc640 1 --2- 172.21.15.38:0/28849263 >> [v2:172.21.15.195:6810/2080960180,v1:172.21.15.195:6811/2080960180] conn(0x7fc96407f150 0x7fc9640815e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:34.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.048+0000 7fc9767fc640 1 -- 172.21.15.38:0/28849263 --> [v2:172.21.15.195:6810/2080960180,v1:172.21.15.195:6811/2080960180] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc970013e50 con 0x7fc96407f150 2024-09-21T03:45:34.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.048+0000 7fc9767fc640 1 -- 172.21.15.38:0/28849263 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_get_version_reply(handle=1 version=152) ==== 24+0+0 (secure 0 0 0) 0x7fc97009b800 con 0x7fc980070940 2024-09-21T03:45:34.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.048+0000 7fc985fd4640 1 --2- 172.21.15.38:0/28849263 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc964078bc0 0x7fc96407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:34.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.048+0000 7fc9867d5640 1 --2- 172.21.15.38:0/28849263 >> [v2:172.21.15.195:6810/2080960180,v1:172.21.15.195:6811/2080960180] conn(0x7fc96407f150 0x7fc9640815e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:34.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.049+0000 7fc985fd4640 1 --2- 172.21.15.38:0/28849263 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc964078bc0 0x7fc96407b0d0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7fc97c001790 tx=0x7fc97c001e90 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:34.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.049+0000 7fc9867d5640 1 --2- 172.21.15.38:0/28849263 >> [v2:172.21.15.195:6810/2080960180,v1:172.21.15.195:6811/2080960180] conn(0x7fc96407f150 0x7fc9640815e0 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:34.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.050+0000 7fc9767fc640 1 -- 172.21.15.38:0/28849263 <== osd.5 v2:172.21.15.195:6810/2080960180 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc970013e50 con 0x7fc96407f150 2024-09-21T03:45:34.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.062+0000 7fc988238640 1 -- 172.21.15.38:0/28849263 --> [v2:172.21.15.195:6810/2080960180,v1:172.21.15.195:6811/2080960180] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fc9801f4570 con 0x7fc96407f150 2024-09-21T03:45:34.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.063+0000 7fc9767fc640 1 -- 172.21.15.38:0/28849263 <== osd.5 v2:172.21.15.195:6810/2080960180 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fc9801f4570 con 0x7fc96407f150 2024-09-21T03:45:34.065 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-09-21T03:45:34.065 INFO:teuthology.orchestra.run.smithi038.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-21T03:45:34.065 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-09-21T03:45:34.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.063+0000 7fc988238640 1 -- 172.21.15.38:0/28849263 >> [v2:172.21.15.195:6810/2080960180,v1:172.21.15.195:6811/2080960180] conn(0x7fc96407f150 msgr2=0x7fc9640815e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:34.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.063+0000 7fc988238640 1 --2- 172.21.15.38:0/28849263 >> [v2:172.21.15.195:6810/2080960180,v1:172.21.15.195:6811/2080960180] conn(0x7fc96407f150 0x7fc9640815e0 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:34.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.063+0000 7fc988238640 1 -- 172.21.15.38:0/28849263 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc964078bc0 msgr2=0x7fc96407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:34.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.063+0000 7fc988238640 1 --2- 172.21.15.38:0/28849263 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fc964078bc0 0x7fc96407b0d0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7fc97c001790 tx=0x7fc97c001e90 comp rx=0 tx=0).stop 2024-09-21T03:45:34.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.063+0000 7fc988238640 1 -- 172.21.15.38:0/28849263 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc980070940 msgr2=0x7fc9800c99f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:34.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.063+0000 7fc988238640 1 --2- 172.21.15.38:0/28849263 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc980070940 0x7fc9800c99f0 secure :-1 s=READY pgs=2547 cs=0 l=1 rev1=1 crypto rx=0x7fc97000d740 tx=0x7fc97000dc00 comp rx=0 tx=0).stop 2024-09-21T03:45:34.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.063+0000 7fc9867d5640 1 -- 172.21.15.38:0/28849263 reap_dead start 2024-09-21T03:45:34.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.063+0000 7fc988238640 1 -- 172.21.15.38:0/28849263 shutdown_connections 2024-09-21T03:45:34.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.063+0000 7fc988238640 1 -- 172.21.15.38:0/28849263 >> 172.21.15.38:0/28849263 conn(0x7fc9800ccac0 msgr2=0x7fc98015a710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:34.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.063+0000 7fc988238640 1 -- 172.21.15.38:0/28849263 shutdown_connections 2024-09-21T03:45:34.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.063+0000 7fc988238640 1 -- 172.21.15.38:0/28849263 wait complete. 2024-09-21T03:45:34.076 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-09-21T03:45:34.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.191+0000 7f81c27a1640 1 -- 172.21.15.38:0/2580288815 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f81a4003a00 msgr2=0x7f81a4003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:34.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.191+0000 7f81c27a1640 1 --2- 172.21.15.38:0/2580288815 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f81a4003a00 0x7f81a4003e70 secure :-1 s=READY pgs=2549 cs=0 l=1 rev1=1 crypto rx=0x7f81ac009b80 tx=0x7f81a4005830 comp rx=0 tx=0).stop 2024-09-21T03:45:34.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.191+0000 7f81c0d3e640 1 -- 172.21.15.38:0/2580288815 reap_dead start 2024-09-21T03:45:34.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.191+0000 7f81c27a1640 1 -- 172.21.15.38:0/2580288815 shutdown_connections 2024-09-21T03:45:34.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.191+0000 7f81c27a1640 1 -- 172.21.15.38:0/2580288815 >> 172.21.15.38:0/2580288815 conn(0x7f81bc0b46e0 msgr2=0x7f81bc0d23e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:34.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.191+0000 7f81c27a1640 1 -- 172.21.15.38:0/2580288815 shutdown_connections 2024-09-21T03:45:34.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.191+0000 7f81c27a1640 1 -- 172.21.15.38:0/2580288815 wait complete. 2024-09-21T03:45:34.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.193+0000 7f81c27a1640 1 Processor -- start 2024-09-21T03:45:34.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.193+0000 7f81c27a1640 1 -- start start 2024-09-21T03:45:34.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.195+0000 7f81c27a1640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f81a4003a00 0x7f81bc159940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:34.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.195+0000 7f81c27a1640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f81bc0cb970 0x7f81bc0cecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:34.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.195+0000 7f81c27a1640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f81bc0cbd80 0x7f81bc0cf230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:34.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.195+0000 7f81c27a1640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f81bc0d0670 con 0x7f81bc0cbd80 2024-09-21T03:45:34.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.195+0000 7f81c27a1640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f81bc0b6e00 con 0x7f81bc0cb970 2024-09-21T03:45:34.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.195+0000 7f81c27a1640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f81bc0d07f0 con 0x7f81a4003a00 2024-09-21T03:45:34.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.195+0000 7f81c0d3e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f81bc0cbd80 0x7f81bc0cf230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:34.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.195+0000 7f81c0d3e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f81bc0cbd80 0x7f81bc0cf230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38408/0 (socket says 172.21.15.38:38408) 2024-09-21T03:45:34.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.195+0000 7f81c0d3e640 1 -- 172.21.15.38:0/2367937750 learned_addr learned my addr 172.21.15.38:0/2367937750 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:45:34.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.195+0000 7f81bb7fe640 1 --2- 172.21.15.38:0/2367937750 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f81bc0cb970 0x7f81bc0cecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:34.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.195+0000 7f81bbfff640 1 --2- 172.21.15.38:0/2367937750 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f81a4003a00 0x7f81bc159940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:34.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.195+0000 7f81c0d3e640 1 -- 172.21.15.38:0/2367937750 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f81a4003a00 msgr2=0x7f81bc159940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:34.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.195+0000 7f81c0d3e640 1 --2- 172.21.15.38:0/2367937750 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f81a4003a00 0x7f81bc159940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:34.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.195+0000 7f81c0d3e640 1 -- 172.21.15.38:0/2367937750 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f81bc0cb970 msgr2=0x7f81bc0cecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:34.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.195+0000 7f81c0d3e640 1 --2- 172.21.15.38:0/2367937750 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f81bc0cb970 0x7f81bc0cecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:34.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.195+0000 7f81c0d3e640 1 -- 172.21.15.38:0/2367937750 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81bc0a94f0 con 0x7f81bc0cbd80 2024-09-21T03:45:34.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.195+0000 7f81c0d3e640 1 --2- 172.21.15.38:0/2367937750 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f81bc0cbd80 0x7f81bc0cf230 secure :-1 s=READY pgs=2550 cs=0 l=1 rev1=1 crypto rx=0x7f81b4002230 tx=0x7f81b4004f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:34.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.196+0000 7f81b8ff9640 1 -- 172.21.15.38:0/2367937750 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81b4010890 con 0x7f81bc0cbd80 2024-09-21T03:45:34.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.196+0000 7f81b8ff9640 1 -- 172.21.15.38:0/2367937750 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f81b4017070 con 0x7f81bc0cbd80 2024-09-21T03:45:34.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.196+0000 7f81c27a1640 1 -- 172.21.15.38:0/2367937750 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81bc0ae140 con 0x7f81bc0cbd80 2024-09-21T03:45:34.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.196+0000 7f81b8ff9640 1 -- 172.21.15.38:0/2367937750 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81b4010de0 con 0x7f81bc0cbd80 2024-09-21T03:45:34.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.196+0000 7f81c27a1640 1 -- 172.21.15.38:0/2367937750 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81bc0a9760 con 0x7f81bc0cbd80 2024-09-21T03:45:34.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.197+0000 7f81c27a1640 1 -- 172.21.15.38:0/2367937750 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81bc0ae310 con 0x7f81bc0cbd80 2024-09-21T03:45:34.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.197+0000 7f81b8ff9640 1 -- 172.21.15.38:0/2367937750 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f81b401c460 con 0x7f81bc0cbd80 2024-09-21T03:45:34.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.198+0000 7f81b8ff9640 1 --2- 172.21.15.38:0/2367937750 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f81a8078bc0 0x7f81a807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:34.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.198+0000 7f81b8ff9640 1 -- 172.21.15.38:0/2367937750 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 5421+0+0 (secure 0 0 0) 0x7f81b409b490 con 0x7f81bc0cbd80 2024-09-21T03:45:34.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.198+0000 7f81bbfff640 1 --2- 172.21.15.38:0/2367937750 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f81a8078bc0 0x7f81a807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:34.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.199+0000 7f81bbfff640 1 --2- 172.21.15.38:0/2367937750 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f81a8078bc0 0x7f81a807b0d0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f81ac035000 tx=0x7f81ac02d040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:34.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.203+0000 7f81b8ff9640 1 -- 172.21.15.38:0/2367937750 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f81bc0d0fc0 con 0x7f81bc0cbd80 2024-09-21T03:45:34.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.394+0000 7f81c27a1640 1 -- 172.21.15.38:0/2367937750 --> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) -- 0x7f81bc0c8b60 con 0x7f81a8078bc0 2024-09-21T03:45:34.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.395+0000 7f81b8ff9640 1 -- 172.21.15.38:0/2367937750 <== mgr.4117 v2:172.21.15.195:6800/1631901344 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f81ac02b9f0 con 0x7f81a8078bc0 2024-09-21T03:45:34.397 INFO:teuthology.orchestra.run.smithi038.stderr:instructed osd(s) 5 to deep-scrub 2024-09-21T03:45:34.399 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.397+0000 7f81c27a1640 1 -- 172.21.15.38:0/2367937750 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f81a8078bc0 msgr2=0x7f81a807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:34.399 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.397+0000 7f81c27a1640 1 --2- 172.21.15.38:0/2367937750 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f81a8078bc0 0x7f81a807b0d0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f81ac035000 tx=0x7f81ac02d040 comp rx=0 tx=0).stop 2024-09-21T03:45:34.399 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.397+0000 7f81c27a1640 1 -- 172.21.15.38:0/2367937750 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f81bc0cbd80 msgr2=0x7f81bc0cf230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:34.399 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.397+0000 7f81c27a1640 1 --2- 172.21.15.38:0/2367937750 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f81bc0cbd80 0x7f81bc0cf230 secure :-1 s=READY pgs=2550 cs=0 l=1 rev1=1 crypto rx=0x7f81b4002230 tx=0x7f81b4004f90 comp rx=0 tx=0).stop 2024-09-21T03:45:34.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.397+0000 7f81c27a1640 1 -- 172.21.15.38:0/2367937750 shutdown_connections 2024-09-21T03:45:34.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.397+0000 7f81c27a1640 1 --2- 172.21.15.38:0/2367937750 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f81bc0cbd80 0x7f81bc0cf230 unknown :-1 s=CLOSED pgs=2550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:34.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.397+0000 7f81c27a1640 1 --2- 172.21.15.38:0/2367937750 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f81bc0cb970 0x7f81bc0cecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:34.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.397+0000 7f81c27a1640 1 --2- 172.21.15.38:0/2367937750 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f81a8078bc0 0x7f81a807b0d0 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:34.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.397+0000 7f81c27a1640 1 --2- 172.21.15.38:0/2367937750 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f81a4003a00 0x7f81bc159940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:34.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.397+0000 7f81c27a1640 1 -- 172.21.15.38:0/2367937750 >> 172.21.15.38:0/2367937750 conn(0x7f81bc0b46e0 msgr2=0x7f81bc0b3300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:34.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.397+0000 7f81c27a1640 1 -- 172.21.15.38:0/2367937750 shutdown_connections 2024-09-21T03:45:34.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.397+0000 7f81c27a1640 1 -- 172.21.15.38:0/2367937750 wait complete. 2024-09-21T03:45:34.410 INFO:tasks.ceph:Scrubbing osd.6 2024-09-21T03:45:34.411 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-09-21T03:45:34.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.529+0000 7fce6a01d640 1 -- 172.21.15.38:0/563521704 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fce640cdbc0 msgr2=0x7fce640cdfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:34.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.529+0000 7fce6a01d640 1 --2- 172.21.15.38:0/563521704 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fce640cdbc0 0x7fce640cdfd0 secure :-1 s=READY pgs=2552 cs=0 l=1 rev1=1 crypto rx=0x7fce640721b0 tx=0x7fce5802a530 comp rx=0 tx=0).stop 2024-09-21T03:45:34.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.529+0000 7fce6a01d640 1 -- 172.21.15.38:0/563521704 shutdown_connections 2024-09-21T03:45:34.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.529+0000 7fce6a01d640 1 --2- 172.21.15.38:0/563521704 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fce640cdbc0 0x7fce640cdfd0 unknown :-1 s=CLOSED pgs=2552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:34.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.529+0000 7fce6a01d640 1 --2- 172.21.15.38:0/563521704 >> v2:172.21.15.195:3300/0 conn(0x7fce6415b4d0 0x7fce64071af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:34.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.529+0000 7fce6a01d640 1 -- 172.21.15.38:0/563521704 >> 172.21.15.38:0/563521704 conn(0x7fce6409a9e0 msgr2=0x7fce640b6050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:34.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.529+0000 7fce6a01d640 1 -- 172.21.15.38:0/563521704 shutdown_connections 2024-09-21T03:45:34.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.529+0000 7fce6a01d640 1 -- 172.21.15.38:0/563521704 wait complete. 2024-09-21T03:45:34.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.531+0000 7fce6a01d640 1 Processor -- start 2024-09-21T03:45:34.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.531+0000 7fce6a01d640 1 -- start start 2024-09-21T03:45:34.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.532+0000 7fce6a01d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fce640cdbc0 0x7fce641f79a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:34.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.532+0000 7fce6a01d640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fce640ce510 0x7fce641484d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:34.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.533+0000 7fce6a01d640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fce6415b4d0 0x7fce6415dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:34.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.533+0000 7fce6a01d640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fce6415f760 con 0x7fce640cdbc0 2024-09-21T03:45:34.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.533+0000 7fce6a01d640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fce640705a0 con 0x7fce640ce510 2024-09-21T03:45:34.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.533+0000 7fce6a01d640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fce6415f8e0 con 0x7fce6415b4d0 2024-09-21T03:45:34.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.533+0000 7fce637fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fce640cdbc0 0x7fce641f79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:34.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.533+0000 7fce637fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fce640cdbc0 0x7fce641f79a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38428/0 (socket says 172.21.15.38:38428) 2024-09-21T03:45:34.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.533+0000 7fce637fe640 1 -- 172.21.15.38:0/2440270792 learned_addr learned my addr 172.21.15.38:0/2440270792 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:45:34.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.533+0000 7fce62ffd640 1 --2- 172.21.15.38:0/2440270792 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fce640ce510 0x7fce641484d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:34.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.533+0000 7fce63fff640 1 --2- 172.21.15.38:0/2440270792 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fce6415b4d0 0x7fce6415dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:34.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.533+0000 7fce637fe640 1 -- 172.21.15.38:0/2440270792 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fce6415b4d0 msgr2=0x7fce6415dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:34.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.533+0000 7fce637fe640 1 --2- 172.21.15.38:0/2440270792 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fce6415b4d0 0x7fce6415dce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:34.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.533+0000 7fce637fe640 1 -- 172.21.15.38:0/2440270792 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fce640ce510 msgr2=0x7fce641484d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:34.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.533+0000 7fce637fe640 1 --2- 172.21.15.38:0/2440270792 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fce640ce510 0x7fce641484d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:34.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.533+0000 7fce637fe640 1 -- 172.21.15.38:0/2440270792 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce6414b180 con 0x7fce640cdbc0 2024-09-21T03:45:34.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.533+0000 7fce637fe640 1 --2- 172.21.15.38:0/2440270792 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fce640cdbc0 0x7fce641f79a0 secure :-1 s=READY pgs=2553 cs=0 l=1 rev1=1 crypto rx=0x7fce5400c9a0 tx=0x7fce5400ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:34.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.533+0000 7fce4bfff640 1 -- 172.21.15.38:0/2440270792 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce54007bb0 con 0x7fce640cdbc0 2024-09-21T03:45:34.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.533+0000 7fce6a01d640 1 -- 172.21.15.38:0/2440270792 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce6414b460 con 0x7fce640cdbc0 2024-09-21T03:45:34.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.533+0000 7fce4bfff640 1 -- 172.21.15.38:0/2440270792 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fce54007e40 con 0x7fce640cdbc0 2024-09-21T03:45:34.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.533+0000 7fce4bfff640 1 -- 172.21.15.38:0/2440270792 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce54005280 con 0x7fce640cdbc0 2024-09-21T03:45:34.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.533+0000 7fce6a01d640 1 -- 172.21.15.38:0/2440270792 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce6414ba20 con 0x7fce640cdbc0 2024-09-21T03:45:34.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.534+0000 7fce6a01d640 1 -- 172.21.15.38:0/2440270792 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fce64071bf0 con 0x7fce640cdbc0 2024-09-21T03:45:34.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.535+0000 7fce4bfff640 1 -- 172.21.15.38:0/2440270792 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fce54005420 con 0x7fce640cdbc0 2024-09-21T03:45:34.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.536+0000 7fce4bfff640 1 --2- 172.21.15.38:0/2440270792 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fce50078b90 0x7fce5007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:34.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.536+0000 7fce4bfff640 1 -- 172.21.15.38:0/2440270792 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 5421+0+0 (secure 0 0 0) 0x7fce5409b0b0 con 0x7fce640cdbc0 2024-09-21T03:45:34.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.536+0000 7fce4bfff640 1 --2- 172.21.15.38:0/2440270792 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x7fce5007f120 0x7fce500815b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:34.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.536+0000 7fce4bfff640 1 -- 172.21.15.38:0/2440270792 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fce54004490 con 0x7fce5007f120 2024-09-21T03:45:34.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.536+0000 7fce62ffd640 1 --2- 172.21.15.38:0/2440270792 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fce50078b90 0x7fce5007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:34.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.536+0000 7fce4bfff640 1 -- 172.21.15.38:0/2440270792 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_get_version_reply(handle=1 version=152) ==== 24+0+0 (secure 0 0 0) 0x7fce540a2050 con 0x7fce640cdbc0 2024-09-21T03:45:34.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.536+0000 7fce63fff640 1 --2- 172.21.15.38:0/2440270792 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x7fce5007f120 0x7fce500815b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:34.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.537+0000 7fce62ffd640 1 --2- 172.21.15.38:0/2440270792 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fce50078b90 0x7fce5007b0a0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fce58034000 tx=0x7fce58004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:34.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.537+0000 7fce63fff640 1 --2- 172.21.15.38:0/2440270792 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x7fce5007f120 0x7fce500815b0 crc :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:34.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.538+0000 7fce4bfff640 1 -- 172.21.15.38:0/2440270792 <== osd.6 v2:172.21.15.195:6826/4055825675 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fce54004490 con 0x7fce5007f120 2024-09-21T03:45:34.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.550+0000 7fce6a01d640 1 -- 172.21.15.38:0/2440270792 --> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fce6415fd90 con 0x7fce5007f120 2024-09-21T03:45:34.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.551+0000 7fce4bfff640 1 -- 172.21.15.38:0/2440270792 <== osd.6 v2:172.21.15.195:6826/4055825675 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fce6415fd90 con 0x7fce5007f120 2024-09-21T03:45:34.553 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-09-21T03:45:34.553 INFO:teuthology.orchestra.run.smithi038.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-21T03:45:34.553 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-09-21T03:45:34.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.551+0000 7fce6a01d640 1 -- 172.21.15.38:0/2440270792 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x7fce5007f120 msgr2=0x7fce500815b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:34.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.551+0000 7fce6a01d640 1 --2- 172.21.15.38:0/2440270792 >> [v2:172.21.15.195:6826/4055825675,v1:172.21.15.195:6827/4055825675] conn(0x7fce5007f120 0x7fce500815b0 crc :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:34.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.551+0000 7fce6a01d640 1 -- 172.21.15.38:0/2440270792 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fce50078b90 msgr2=0x7fce5007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:34.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.551+0000 7fce6a01d640 1 --2- 172.21.15.38:0/2440270792 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fce50078b90 0x7fce5007b0a0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fce58034000 tx=0x7fce58004730 comp rx=0 tx=0).stop 2024-09-21T03:45:34.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.551+0000 7fce6a01d640 1 -- 172.21.15.38:0/2440270792 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fce640cdbc0 msgr2=0x7fce641f79a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:34.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.551+0000 7fce6a01d640 1 --2- 172.21.15.38:0/2440270792 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fce640cdbc0 0x7fce641f79a0 secure :-1 s=READY pgs=2553 cs=0 l=1 rev1=1 crypto rx=0x7fce5400c9a0 tx=0x7fce5400ce60 comp rx=0 tx=0).stop 2024-09-21T03:45:34.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.551+0000 7fce63fff640 1 -- 172.21.15.38:0/2440270792 reap_dead start 2024-09-21T03:45:34.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.551+0000 7fce6a01d640 1 -- 172.21.15.38:0/2440270792 shutdown_connections 2024-09-21T03:45:34.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.551+0000 7fce6a01d640 1 -- 172.21.15.38:0/2440270792 >> 172.21.15.38:0/2440270792 conn(0x7fce6409a9e0 msgr2=0x7fce640d0dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:34.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.552+0000 7fce6a01d640 1 -- 172.21.15.38:0/2440270792 shutdown_connections 2024-09-21T03:45:34.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.552+0000 7fce6a01d640 1 -- 172.21.15.38:0/2440270792 wait complete. 2024-09-21T03:45:34.564 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-09-21T03:45:34.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.680+0000 7f144a6a2640 1 -- 172.21.15.38:0/2519762492 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f142c004800 msgr2=0x7f142c004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:34.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.680+0000 7f144a6a2640 1 --2- 172.21.15.38:0/2519762492 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f142c004800 0x7f142c004c10 secure :-1 s=READY pgs=2555 cs=0 l=1 rev1=1 crypto rx=0x7f1438001690 tx=0x7f142c005830 comp rx=0 tx=0).stop 2024-09-21T03:45:34.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.680+0000 7f144a6a2640 1 -- 172.21.15.38:0/2519762492 shutdown_connections 2024-09-21T03:45:34.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.680+0000 7f144a6a2640 1 --2- 172.21.15.38:0/2519762492 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f142c004800 0x7f142c004c10 unknown :-1 s=CLOSED pgs=2555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:34.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.680+0000 7f144a6a2640 1 -- 172.21.15.38:0/2519762492 >> 172.21.15.38:0/2519762492 conn(0x7f14440b46e0 msgr2=0x7f14440d23e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:34.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.680+0000 7f144a6a2640 1 -- 172.21.15.38:0/2519762492 shutdown_connections 2024-09-21T03:45:34.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.680+0000 7f144a6a2640 1 -- 172.21.15.38:0/2519762492 wait complete. 2024-09-21T03:45:34.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.682+0000 7f144a6a2640 1 Processor -- start 2024-09-21T03:45:34.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.682+0000 7f144a6a2640 1 -- start start 2024-09-21T03:45:34.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.684+0000 7f144a6a2640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f142c004800 0x7f1444154120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:34.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.684+0000 7f144a6a2640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f144415cfe0 0x7f144415d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:34.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.684+0000 7f144a6a2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f144415e0c0 0x7f144415e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:34.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.684+0000 7f144a6a2640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f14440d0670 con 0x7f144415e0c0 2024-09-21T03:45:34.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.684+0000 7f144a6a2640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f14440cda30 con 0x7f144415cfe0 2024-09-21T03:45:34.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.684+0000 7f144a6a2640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f14440d07f0 con 0x7f142c004800 2024-09-21T03:45:34.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.684+0000 7f1448c3f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f144415e0c0 0x7f144415e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:34.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.684+0000 7f1448c3f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f144415e0c0 0x7f144415e4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38442/0 (socket says 172.21.15.38:38442) 2024-09-21T03:45:34.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.684+0000 7f1448c3f640 1 -- 172.21.15.38:0/715933648 learned_addr learned my addr 172.21.15.38:0/715933648 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:45:34.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.684+0000 7f1443fff640 1 --2- 172.21.15.38:0/715933648 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f142c004800 0x7f1444154120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:34.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.684+0000 7f14437fe640 1 --2- 172.21.15.38:0/715933648 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f144415cfe0 0x7f144415d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:34.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.684+0000 7f1448c3f640 1 -- 172.21.15.38:0/715933648 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f142c004800 msgr2=0x7f1444154120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:34.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.684+0000 7f1448c3f640 1 --2- 172.21.15.38:0/715933648 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f142c004800 0x7f1444154120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:34.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.684+0000 7f1448c3f640 1 -- 172.21.15.38:0/715933648 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f144415cfe0 msgr2=0x7f144415d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:34.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.684+0000 7f1448c3f640 1 --2- 172.21.15.38:0/715933648 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f144415cfe0 0x7f144415d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:34.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.684+0000 7f1448c3f640 1 -- 172.21.15.38:0/715933648 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1444153360 con 0x7f144415e0c0 2024-09-21T03:45:34.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.684+0000 7f1448c3f640 1 --2- 172.21.15.38:0/715933648 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f144415e0c0 0x7f144415e4d0 secure :-1 s=READY pgs=2556 cs=0 l=1 rev1=1 crypto rx=0x7f14440d34b0 tx=0x7f143c00f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:34.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.685+0000 7f1440ff9640 1 -- 172.21.15.38:0/715933648 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f143c003d80 con 0x7f144415e0c0 2024-09-21T03:45:34.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.685+0000 7f1440ff9640 1 -- 172.21.15.38:0/715933648 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f143c004b20 con 0x7f144415e0c0 2024-09-21T03:45:34.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.685+0000 7f144a6a2640 1 -- 172.21.15.38:0/715933648 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1444153640 con 0x7f144415e0c0 2024-09-21T03:45:34.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.685+0000 7f1440ff9640 1 -- 172.21.15.38:0/715933648 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f143c004030 con 0x7f144415e0c0 2024-09-21T03:45:34.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.685+0000 7f144a6a2640 1 -- 172.21.15.38:0/715933648 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1444153c00 con 0x7f144415e0c0 2024-09-21T03:45:34.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.686+0000 7f144a6a2640 1 -- 172.21.15.38:0/715933648 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14440d0d50 con 0x7f144415e0c0 2024-09-21T03:45:34.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.686+0000 7f1440ff9640 1 -- 172.21.15.38:0/715933648 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f143c0041d0 con 0x7f144415e0c0 2024-09-21T03:45:34.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.687+0000 7f1440ff9640 1 --2- 172.21.15.38:0/715933648 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1430078bc0 0x7f143007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:34.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.687+0000 7f1440ff9640 1 -- 172.21.15.38:0/715933648 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 5421+0+0 (secure 0 0 0) 0x7f143c09b9b0 con 0x7f144415e0c0 2024-09-21T03:45:34.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.687+0000 7f1443fff640 1 --2- 172.21.15.38:0/715933648 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1430078bc0 0x7f143007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:34.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.688+0000 7f1443fff640 1 --2- 172.21.15.38:0/715933648 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1430078bc0 0x7f143007b0d0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f1434001010 tx=0x7f14340067a0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:34.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.692+0000 7f1440ff9640 1 -- 172.21.15.38:0/715933648 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f143c063950 con 0x7f144415e0c0 2024-09-21T03:45:34.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.878+0000 7f144a6a2640 1 -- 172.21.15.38:0/715933648 --> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) -- 0x7f1444155be0 con 0x7f1430078bc0 2024-09-21T03:45:34.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.879+0000 7f1440ff9640 1 -- 172.21.15.38:0/715933648 <== mgr.4117 v2:172.21.15.195:6800/1631901344 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f1444155be0 con 0x7f1430078bc0 2024-09-21T03:45:34.881 INFO:teuthology.orchestra.run.smithi038.stderr:instructed osd(s) 6 to deep-scrub 2024-09-21T03:45:34.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.881+0000 7f144a6a2640 1 -- 172.21.15.38:0/715933648 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1430078bc0 msgr2=0x7f143007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:34.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.881+0000 7f144a6a2640 1 --2- 172.21.15.38:0/715933648 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1430078bc0 0x7f143007b0d0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f1434001010 tx=0x7f14340067a0 comp rx=0 tx=0).stop 2024-09-21T03:45:34.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.881+0000 7f144a6a2640 1 -- 172.21.15.38:0/715933648 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f144415e0c0 msgr2=0x7f144415e4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:34.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.881+0000 7f144a6a2640 1 --2- 172.21.15.38:0/715933648 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f144415e0c0 0x7f144415e4d0 secure :-1 s=READY pgs=2556 cs=0 l=1 rev1=1 crypto rx=0x7f14440d34b0 tx=0x7f143c00f790 comp rx=0 tx=0).stop 2024-09-21T03:45:34.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.881+0000 7f144a6a2640 1 -- 172.21.15.38:0/715933648 shutdown_connections 2024-09-21T03:45:34.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.881+0000 7f144a6a2640 1 --2- 172.21.15.38:0/715933648 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f144415e0c0 0x7f144415e4d0 unknown :-1 s=CLOSED pgs=2556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:34.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.881+0000 7f144a6a2640 1 --2- 172.21.15.38:0/715933648 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f144415cfe0 0x7f144415d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:34.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.881+0000 7f144a6a2640 1 --2- 172.21.15.38:0/715933648 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f1430078bc0 0x7f143007b0d0 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:34.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.881+0000 7f144a6a2640 1 --2- 172.21.15.38:0/715933648 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f142c004800 0x7f1444154120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:34.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.881+0000 7f144a6a2640 1 -- 172.21.15.38:0/715933648 >> 172.21.15.38:0/715933648 conn(0x7f14440b46e0 msgr2=0x7f14440b7830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:34.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.881+0000 7f144a6a2640 1 -- 172.21.15.38:0/715933648 shutdown_connections 2024-09-21T03:45:34.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:34.881+0000 7f144a6a2640 1 -- 172.21.15.38:0/715933648 wait complete. 2024-09-21T03:45:34.894 INFO:tasks.ceph:Scrubbing osd.7 2024-09-21T03:45:34.895 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-09-21T03:45:35.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.013+0000 7f7546c4d640 1 -- 172.21.15.38:0/2130633638 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7540161ed0 msgr2=0x7f75400cf200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:35.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.013+0000 7f7546c4d640 1 --2- 172.21.15.38:0/2130633638 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7540161ed0 0x7f75400cf200 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f7534009860 tx=0x7f753402a5a0 comp rx=0 tx=0).stop 2024-09-21T03:45:35.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.013+0000 7f7546c4d640 1 -- 172.21.15.38:0/2130633638 shutdown_connections 2024-09-21T03:45:35.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.013+0000 7f7546c4d640 1 --2- 172.21.15.38:0/2130633638 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7540161ed0 0x7f75400cf200 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:35.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.013+0000 7f7546c4d640 1 -- 172.21.15.38:0/2130633638 >> 172.21.15.38:0/2130633638 conn(0x7f75400d4510 msgr2=0x7f75400d4920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:35.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.013+0000 7f7546c4d640 1 -- 172.21.15.38:0/2130633638 shutdown_connections 2024-09-21T03:45:35.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.013+0000 7f7546c4d640 1 -- 172.21.15.38:0/2130633638 wait complete. 2024-09-21T03:45:35.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.015+0000 7f7546c4d640 1 Processor -- start 2024-09-21T03:45:35.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.015+0000 7f7546c4d640 1 -- start start 2024-09-21T03:45:35.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.016+0000 7f7546c4d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f75400cf740 0x7f75400ce3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:35.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.016+0000 7f7546c4d640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f75400d0460 0x7f754014e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:35.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.016+0000 7f7546c4d640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7540161ed0 0x7f75400b94d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:35.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.016+0000 7f7546c4d640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f7540162e90 con 0x7f75400cf740 2024-09-21T03:45:35.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.017+0000 7f7546c4d640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f7540162d10 con 0x7f75400d0460 2024-09-21T03:45:35.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.017+0000 7f7546c4d640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f7540163010 con 0x7f7540161ed0 2024-09-21T03:45:35.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.017+0000 7f75449e9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f75400cf740 0x7f75400ce3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:35.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.017+0000 7f75449e9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f75400cf740 0x7f75400ce3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38450/0 (socket says 172.21.15.38:38450) 2024-09-21T03:45:35.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.017+0000 7f75449e9640 1 -- 172.21.15.38:0/2644257908 learned_addr learned my addr 172.21.15.38:0/2644257908 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:45:35.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.017+0000 7f753ffff640 1 --2- 172.21.15.38:0/2644257908 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f75400d0460 0x7f754014e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:35.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.017+0000 7f75451ea640 1 --2- 172.21.15.38:0/2644257908 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7540161ed0 0x7f75400b94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:35.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.017+0000 7f75449e9640 1 -- 172.21.15.38:0/2644257908 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7540161ed0 msgr2=0x7f75400b94d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:35.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.017+0000 7f75449e9640 1 --2- 172.21.15.38:0/2644257908 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7540161ed0 0x7f75400b94d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:35.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.017+0000 7f75449e9640 1 -- 172.21.15.38:0/2644257908 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f75400d0460 msgr2=0x7f754014e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:35.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.017+0000 7f75449e9640 1 --2- 172.21.15.38:0/2644257908 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f75400d0460 0x7f754014e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:35.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.017+0000 7f75449e9640 1 -- 172.21.15.38:0/2644257908 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75400b3e90 con 0x7f75400cf740 2024-09-21T03:45:35.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.017+0000 7f75449e9640 1 --2- 172.21.15.38:0/2644257908 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f75400cf740 0x7f75400ce3b0 secure :-1 s=READY pgs=2558 cs=0 l=1 rev1=1 crypto rx=0x7f753000c430 tx=0x7f753000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:35.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.017+0000 7f753d7fa640 1 -- 172.21.15.38:0/2644257908 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7530011850 con 0x7f75400cf740 2024-09-21T03:45:35.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.017+0000 7f753d7fa640 1 -- 172.21.15.38:0/2644257908 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7530017070 con 0x7f75400cf740 2024-09-21T03:45:35.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.017+0000 7f753d7fa640 1 -- 172.21.15.38:0/2644257908 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7530011da0 con 0x7f75400cf740 2024-09-21T03:45:35.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.017+0000 7f7546c4d640 1 -- 172.21.15.38:0/2644257908 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75400b4170 con 0x7f75400cf740 2024-09-21T03:45:35.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.018+0000 7f7546c4d640 1 -- 172.21.15.38:0/2644257908 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75400b4730 con 0x7f75400cf740 2024-09-21T03:45:35.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.018+0000 7f7546c4d640 1 -- 172.21.15.38:0/2644257908 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7540163400 con 0x7f75400cf740 2024-09-21T03:45:35.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.019+0000 7f753d7fa640 1 -- 172.21.15.38:0/2644257908 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f753001c460 con 0x7f75400cf740 2024-09-21T03:45:35.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.020+0000 7f753d7fa640 1 --2- 172.21.15.38:0/2644257908 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f752c078bc0 0x7f752c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:35.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.020+0000 7f753d7fa640 1 -- 172.21.15.38:0/2644257908 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 5421+0+0 (secure 0 0 0) 0x7f753009b3e0 con 0x7f75400cf740 2024-09-21T03:45:35.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.020+0000 7f753ffff640 1 --2- 172.21.15.38:0/2644257908 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f752c078bc0 0x7f752c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:35.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.020+0000 7f753d7fa640 1 --2- 172.21.15.38:0/2644257908 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x7f752c07f150 0x7f752c0815e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:35.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.020+0000 7f753d7fa640 1 -- 172.21.15.38:0/2644257908 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7530010e50 con 0x7f752c07f150 2024-09-21T03:45:35.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.020+0000 7f753d7fa640 1 -- 172.21.15.38:0/2644257908 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_get_version_reply(handle=1 version=152) ==== 24+0+0 (secure 0 0 0) 0x7f753009b790 con 0x7f75400cf740 2024-09-21T03:45:35.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.020+0000 7f75451ea640 1 --2- 172.21.15.38:0/2644257908 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x7f752c07f150 0x7f752c0815e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:35.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.021+0000 7f753ffff640 1 --2- 172.21.15.38:0/2644257908 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f752c078bc0 0x7f752c07b0d0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f7534034000 tx=0x7f7534004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:35.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.021+0000 7f75451ea640 1 --2- 172.21.15.38:0/2644257908 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x7f752c07f150 0x7f752c0815e0 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:35.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.022+0000 7f753d7fa640 1 -- 172.21.15.38:0/2644257908 <== osd.7 v2:172.21.15.195:6818/1606106342 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7530010e50 con 0x7f752c07f150 2024-09-21T03:45:35.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.034+0000 7f7546c4d640 1 -- 172.21.15.38:0/2644257908 --> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f75400cfe20 con 0x7f752c07f150 2024-09-21T03:45:35.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.035+0000 7f753d7fa640 1 -- 172.21.15.38:0/2644257908 <== osd.7 v2:172.21.15.195:6818/1606106342 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f75400cfe20 con 0x7f752c07f150 2024-09-21T03:45:35.039 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-09-21T03:45:35.039 INFO:teuthology.orchestra.run.smithi038.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-21T03:45:35.039 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-09-21T03:45:35.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.035+0000 7f7546c4d640 1 -- 172.21.15.38:0/2644257908 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x7f752c07f150 msgr2=0x7f752c0815e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:35.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.035+0000 7f7546c4d640 1 --2- 172.21.15.38:0/2644257908 >> [v2:172.21.15.195:6818/1606106342,v1:172.21.15.195:6819/1606106342] conn(0x7f752c07f150 0x7f752c0815e0 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:35.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.035+0000 7f7546c4d640 1 -- 172.21.15.38:0/2644257908 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f752c078bc0 msgr2=0x7f752c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:35.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.035+0000 7f7546c4d640 1 --2- 172.21.15.38:0/2644257908 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f752c078bc0 0x7f752c07b0d0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f7534034000 tx=0x7f7534004730 comp rx=0 tx=0).stop 2024-09-21T03:45:35.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.035+0000 7f7546c4d640 1 -- 172.21.15.38:0/2644257908 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f75400cf740 msgr2=0x7f75400ce3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:35.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.035+0000 7f7546c4d640 1 --2- 172.21.15.38:0/2644257908 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f75400cf740 0x7f75400ce3b0 secure :-1 s=READY pgs=2558 cs=0 l=1 rev1=1 crypto rx=0x7f753000c430 tx=0x7f753000c8f0 comp rx=0 tx=0).stop 2024-09-21T03:45:35.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.035+0000 7f75451ea640 1 -- 172.21.15.38:0/2644257908 reap_dead start 2024-09-21T03:45:35.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.035+0000 7f7546c4d640 1 -- 172.21.15.38:0/2644257908 shutdown_connections 2024-09-21T03:45:35.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.035+0000 7f7546c4d640 1 -- 172.21.15.38:0/2644257908 >> 172.21.15.38:0/2644257908 conn(0x7f75400d4510 msgr2=0x7f754015dbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:35.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.035+0000 7f7546c4d640 1 -- 172.21.15.38:0/2644257908 shutdown_connections 2024-09-21T03:45:35.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.035+0000 7f7546c4d640 1 -- 172.21.15.38:0/2644257908 wait complete. 2024-09-21T03:45:35.048 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-09-21T03:45:35.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.163+0000 7ff0001ad640 1 -- 172.21.15.38:0/3935972291 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feff80d0170 msgr2=0x7feff80d0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:35.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.163+0000 7ff0001ad640 1 --2- 172.21.15.38:0/3935972291 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feff80d0170 0x7feff80d0be0 secure :-1 s=READY pgs=2560 cs=0 l=1 rev1=1 crypto rx=0x7fefec009820 tx=0x7fefec02a3f0 comp rx=0 tx=0).stop 2024-09-21T03:45:35.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.163+0000 7ff0001ad640 1 -- 172.21.15.38:0/3935972291 shutdown_connections 2024-09-21T03:45:35.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.163+0000 7ff0001ad640 1 --2- 172.21.15.38:0/3935972291 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feff80d0170 0x7feff80d0be0 unknown :-1 s=CLOSED pgs=2560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:35.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.163+0000 7ff0001ad640 1 -- 172.21.15.38:0/3935972291 >> 172.21.15.38:0/3935972291 conn(0x7feff809d830 msgr2=0x7feff80b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:35.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.163+0000 7ff0001ad640 1 -- 172.21.15.38:0/3935972291 shutdown_connections 2024-09-21T03:45:35.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.163+0000 7ff0001ad640 1 -- 172.21.15.38:0/3935972291 wait complete. 2024-09-21T03:45:35.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.165+0000 7ff0001ad640 1 Processor -- start 2024-09-21T03:45:35.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.165+0000 7ff0001ad640 1 -- start start 2024-09-21T03:45:35.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.166+0000 7ff0001ad640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feff80d0170 0x7feff806f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:35.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.166+0000 7ff0001ad640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feff80d1120 0x7feff806cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:35.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.166+0000 7ff0001ad640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7feff815ee80 0x7feff8153cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:35.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.166+0000 7ff0001ad640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7feff80b6910 con 0x7feff80d0170 2024-09-21T03:45:35.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.166+0000 7ff0001ad640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7feff80b6790 con 0x7feff80d1120 2024-09-21T03:45:35.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.166+0000 7ff0001ad640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7feff80b6a90 con 0x7feff815ee80 2024-09-21T03:45:35.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.166+0000 7feffdf49640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feff80d0170 0x7feff806f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:35.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.166+0000 7feffdf49640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feff80d0170 0x7feff806f030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38466/0 (socket says 172.21.15.38:38466) 2024-09-21T03:45:35.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.166+0000 7feffdf49640 1 -- 172.21.15.38:0/2200127527 learned_addr learned my addr 172.21.15.38:0/2200127527 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:45:35.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.167+0000 7feffd748640 1 --2- 172.21.15.38:0/2200127527 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feff80d1120 0x7feff806cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:35.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.167+0000 7feffe74a640 1 --2- 172.21.15.38:0/2200127527 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7feff815ee80 0x7feff8153cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:35.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.167+0000 7feffdf49640 1 -- 172.21.15.38:0/2200127527 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7feff815ee80 msgr2=0x7feff8153cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:35.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.167+0000 7feffdf49640 1 --2- 172.21.15.38:0/2200127527 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7feff815ee80 0x7feff8153cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:35.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.167+0000 7feffdf49640 1 -- 172.21.15.38:0/2200127527 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feff80d1120 msgr2=0x7feff806cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:35.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.167+0000 7feffdf49640 1 --2- 172.21.15.38:0/2200127527 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feff80d1120 0x7feff806cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:35.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.167+0000 7feffdf49640 1 -- 172.21.15.38:0/2200127527 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feff806d320 con 0x7feff80d0170 2024-09-21T03:45:35.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.167+0000 7feffe74a640 1 --2- 172.21.15.38:0/2200127527 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7feff815ee80 0x7feff8153cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:45:35.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.167+0000 7feffdf49640 1 --2- 172.21.15.38:0/2200127527 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feff80d0170 0x7feff806f030 secure :-1 s=READY pgs=2561 cs=0 l=1 rev1=1 crypto rx=0x7fefe800cbc0 tx=0x7fefe800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:35.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.167+0000 7feff67fc640 1 -- 172.21.15.38:0/2200127527 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefe8016020 con 0x7feff80d0170 2024-09-21T03:45:35.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.167+0000 7feff67fc640 1 -- 172.21.15.38:0/2200127527 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fefe800ad90 con 0x7feff80d0170 2024-09-21T03:45:35.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.167+0000 7ff0001ad640 1 -- 172.21.15.38:0/2200127527 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feff806f980 con 0x7feff80d0170 2024-09-21T03:45:35.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.167+0000 7feff67fc640 1 -- 172.21.15.38:0/2200127527 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefe8012680 con 0x7feff80d0170 2024-09-21T03:45:35.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.167+0000 7ff0001ad640 1 -- 172.21.15.38:0/2200127527 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feff8157f10 con 0x7feff80d0170 2024-09-21T03:45:35.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.168+0000 7ff0001ad640 1 -- 172.21.15.38:0/2200127527 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feff80d0be0 con 0x7feff80d0170 2024-09-21T03:45:35.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.169+0000 7feff67fc640 1 -- 172.21.15.38:0/2200127527 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fefe8019020 con 0x7feff80d0170 2024-09-21T03:45:35.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.170+0000 7feff67fc640 1 --2- 172.21.15.38:0/2200127527 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fefe4078b90 0x7fefe407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:35.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.170+0000 7feff67fc640 1 -- 172.21.15.38:0/2200127527 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 5421+0+0 (secure 0 0 0) 0x7fefe809b1b0 con 0x7feff80d0170 2024-09-21T03:45:35.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.170+0000 7feffd748640 1 --2- 172.21.15.38:0/2200127527 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fefe4078b90 0x7fefe407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:35.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.171+0000 7feffd748640 1 --2- 172.21.15.38:0/2200127527 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fefe4078b90 0x7fefe407b0a0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7fefec034000 tx=0x7fefec004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:35.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.175+0000 7feff67fc640 1 -- 172.21.15.38:0/2200127527 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fefe8020020 con 0x7feff80d0170 2024-09-21T03:45:35.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.363+0000 7ff0001ad640 1 -- 172.21.15.38:0/2200127527 --> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) -- 0x7feff806fc60 con 0x7fefe4078b90 2024-09-21T03:45:35.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.364+0000 7feff67fc640 1 -- 172.21.15.38:0/2200127527 <== mgr.4117 v2:172.21.15.195:6800/1631901344 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7feff806fc60 con 0x7fefe4078b90 2024-09-21T03:45:35.366 INFO:teuthology.orchestra.run.smithi038.stderr:instructed osd(s) 7 to deep-scrub 2024-09-21T03:45:35.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.366+0000 7ff0001ad640 1 -- 172.21.15.38:0/2200127527 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fefe4078b90 msgr2=0x7fefe407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:35.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.366+0000 7ff0001ad640 1 --2- 172.21.15.38:0/2200127527 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fefe4078b90 0x7fefe407b0a0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7fefec034000 tx=0x7fefec004730 comp rx=0 tx=0).stop 2024-09-21T03:45:35.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.366+0000 7ff0001ad640 1 -- 172.21.15.38:0/2200127527 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feff80d0170 msgr2=0x7feff806f030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:35.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.366+0000 7ff0001ad640 1 --2- 172.21.15.38:0/2200127527 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feff80d0170 0x7feff806f030 secure :-1 s=READY pgs=2561 cs=0 l=1 rev1=1 crypto rx=0x7fefe800cbc0 tx=0x7fefe800a060 comp rx=0 tx=0).stop 2024-09-21T03:45:35.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.366+0000 7ff0001ad640 1 -- 172.21.15.38:0/2200127527 shutdown_connections 2024-09-21T03:45:35.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.366+0000 7ff0001ad640 1 --2- 172.21.15.38:0/2200127527 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fefe4078b90 0x7fefe407b0a0 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:35.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.366+0000 7ff0001ad640 1 --2- 172.21.15.38:0/2200127527 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7feff815ee80 0x7feff8153cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:35.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.366+0000 7ff0001ad640 1 --2- 172.21.15.38:0/2200127527 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feff80d1120 0x7feff806cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:35.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.366+0000 7ff0001ad640 1 --2- 172.21.15.38:0/2200127527 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feff80d0170 0x7feff806f030 unknown :-1 s=CLOSED pgs=2561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:35.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.366+0000 7ff0001ad640 1 -- 172.21.15.38:0/2200127527 >> 172.21.15.38:0/2200127527 conn(0x7feff809d830 msgr2=0x7feff80b31a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:35.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.366+0000 7ff0001ad640 1 -- 172.21.15.38:0/2200127527 shutdown_connections 2024-09-21T03:45:35.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.366+0000 7ff0001ad640 1 -- 172.21.15.38:0/2200127527 wait complete. 2024-09-21T03:45:35.379 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T03:45:35.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.493+0000 7fb1c2939640 1 -- 172.21.15.38:0/3618849982 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb1bc06e1d0 msgr2=0x7fb1bc0afd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:35.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.493+0000 7fb1c2939640 1 --2- 172.21.15.38:0/3618849982 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb1bc06e1d0 0x7fb1bc0afd70 secure :-1 s=READY pgs=2563 cs=0 l=1 rev1=1 crypto rx=0x7fb1bc0b4430 tx=0x7fb1ac02a530 comp rx=0 tx=0).stop 2024-09-21T03:45:35.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.493+0000 7fb1c2939640 1 -- 172.21.15.38:0/3618849982 shutdown_connections 2024-09-21T03:45:35.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.493+0000 7fb1c2939640 1 --2- 172.21.15.38:0/3618849982 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb1bc0b4b50 0x7fb1bc15ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:35.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.493+0000 7fb1c2939640 1 --2- 172.21.15.38:0/3618849982 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb1bc06e1d0 0x7fb1bc0afd70 unknown :-1 s=CLOSED pgs=2563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:35.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.493+0000 7fb1c2939640 1 -- 172.21.15.38:0/3618849982 >> 172.21.15.38:0/3618849982 conn(0x7fb1bc0d0a30 msgr2=0x7fb1bc0d0e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:35.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.493+0000 7fb1c2939640 1 -- 172.21.15.38:0/3618849982 shutdown_connections 2024-09-21T03:45:35.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.493+0000 7fb1c2939640 1 -- 172.21.15.38:0/3618849982 wait complete. 2024-09-21T03:45:35.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.494+0000 7fb1c2939640 1 Processor -- start 2024-09-21T03:45:35.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.495+0000 7fb1c2939640 1 -- start start 2024-09-21T03:45:35.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.496+0000 7fb1c2939640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb1bc06e1d0 0x7fb1bc0aef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:35.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.496+0000 7fb1c2939640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb1bc0b4740 0x7fb1bc1476d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:35.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.496+0000 7fb1c2939640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb1bc0b4b50 0x7fb1bc0b1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:35.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.496+0000 7fb1c2939640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fb1bc15b1f0 con 0x7fb1bc0b4b50 2024-09-21T03:45:35.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.496+0000 7fb1c2939640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fb1bc15b070 con 0x7fb1bc06e1d0 2024-09-21T03:45:35.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.496+0000 7fb1c2939640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fb1bc15b370 con 0x7fb1bc0b4740 2024-09-21T03:45:35.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.496+0000 7fb1c0ed6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb1bc0b4b50 0x7fb1bc0b1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:35.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.496+0000 7fb1c0ed6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb1bc0b4b50 0x7fb1bc0b1720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38482/0 (socket says 172.21.15.38:38482) 2024-09-21T03:45:35.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.496+0000 7fb1c0ed6640 1 -- 172.21.15.38:0/833544411 learned_addr learned my addr 172.21.15.38:0/833544411 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:45:35.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.496+0000 7fb1bbfff640 1 --2- 172.21.15.38:0/833544411 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb1bc06e1d0 0x7fb1bc0aef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:35.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.496+0000 7fb1bb7fe640 1 --2- 172.21.15.38:0/833544411 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb1bc0b4740 0x7fb1bc1476d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:35.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.496+0000 7fb1c0ed6640 1 -- 172.21.15.38:0/833544411 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb1bc0b4740 msgr2=0x7fb1bc1476d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:35.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.496+0000 7fb1c0ed6640 1 --2- 172.21.15.38:0/833544411 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb1bc0b4740 0x7fb1bc1476d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:35.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.496+0000 7fb1c0ed6640 1 -- 172.21.15.38:0/833544411 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb1bc06e1d0 msgr2=0x7fb1bc0aef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:35.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.496+0000 7fb1c0ed6640 1 --2- 172.21.15.38:0/833544411 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb1bc06e1d0 0x7fb1bc0aef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:35.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.497+0000 7fb1c0ed6640 1 -- 172.21.15.38:0/833544411 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1bc0af4b0 con 0x7fb1bc0b4b50 2024-09-21T03:45:35.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.497+0000 7fb1bb7fe640 1 --2- 172.21.15.38:0/833544411 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb1bc0b4740 0x7fb1bc1476d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:45:35.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.497+0000 7fb1c0ed6640 1 --2- 172.21.15.38:0/833544411 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb1bc0b4b50 0x7fb1bc0b1720 secure :-1 s=READY pgs=2564 cs=0 l=1 rev1=1 crypto rx=0x7fb1b400b9e0 tx=0x7fb1b400bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:35.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.497+0000 7fb1b8ff9640 1 -- 172.21.15.38:0/833544411 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1b4019070 con 0x7fb1bc0b4b50 2024-09-21T03:45:35.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.497+0000 7fb1b8ff9640 1 -- 172.21.15.38:0/833544411 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb1b400cc30 con 0x7fb1bc0b4b50 2024-09-21T03:45:35.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.497+0000 7fb1c2939640 1 -- 172.21.15.38:0/833544411 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1bc1519d0 con 0x7fb1bc0b4b50 2024-09-21T03:45:35.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.497+0000 7fb1b8ff9640 1 -- 172.21.15.38:0/833544411 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1b40146a0 con 0x7fb1bc0b4b50 2024-09-21T03:45:35.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.497+0000 7fb1c2939640 1 -- 172.21.15.38:0/833544411 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1bc148520 con 0x7fb1bc0b4b50 2024-09-21T03:45:35.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.498+0000 7fb1c2939640 1 -- 172.21.15.38:0/833544411 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1bc0afd70 con 0x7fb1bc0b4b50 2024-09-21T03:45:35.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.499+0000 7fb1b8ff9640 1 -- 172.21.15.38:0/833544411 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb1b401c070 con 0x7fb1bc0b4b50 2024-09-21T03:45:35.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.499+0000 7fb1b8ff9640 1 --2- 172.21.15.38:0/833544411 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb1a8078b90 0x7fb1a807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:35.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.499+0000 7fb1b8ff9640 1 -- 172.21.15.38:0/833544411 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 5421+0+0 (secure 0 0 0) 0x7fb1b409a960 con 0x7fb1bc0b4b50 2024-09-21T03:45:35.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.500+0000 7fb1bbfff640 1 --2- 172.21.15.38:0/833544411 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb1a8078b90 0x7fb1a807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:35.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.500+0000 7fb1bbfff640 1 --2- 172.21.15.38:0/833544411 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb1a8078b90 0x7fb1a807b0a0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7fb1ac033000 tx=0x7fb1ac004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:35.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.504+0000 7fb1b8ff9640 1 -- 172.21.15.38:0/833544411 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fb1b400ce20 con 0x7fb1bc0b4b50 2024-09-21T03:45:35.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.687+0000 7fb1c2939640 1 -- 172.21.15.38:0/833544411 --> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb1bc0aec70 con 0x7fb1a8078b90 2024-09-21T03:45:35.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.688+0000 7fb1b8ff9640 1 -- 172.21.15.38:0/833544411 <== mgr.4117 v2:172.21.15.195:6800/1631901344 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13393 (secure 0 0 0) 0x7fb1ac004040 con 0x7fb1a8078b90 2024-09-21T03:45:35.691 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:45:35.691 INFO:teuthology.orchestra.run.smithi038.stderr:dumped all 2024-09-21T03:45:35.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.691+0000 7fb1c2939640 1 -- 172.21.15.38:0/833544411 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb1a8078b90 msgr2=0x7fb1a807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:35.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.691+0000 7fb1c2939640 1 --2- 172.21.15.38:0/833544411 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb1a8078b90 0x7fb1a807b0a0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7fb1ac033000 tx=0x7fb1ac004730 comp rx=0 tx=0).stop 2024-09-21T03:45:35.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.691+0000 7fb1c2939640 1 -- 172.21.15.38:0/833544411 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb1bc0b4b50 msgr2=0x7fb1bc0b1720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:35.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.691+0000 7fb1c2939640 1 --2- 172.21.15.38:0/833544411 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb1bc0b4b50 0x7fb1bc0b1720 secure :-1 s=READY pgs=2564 cs=0 l=1 rev1=1 crypto rx=0x7fb1b400b9e0 tx=0x7fb1b400bea0 comp rx=0 tx=0).stop 2024-09-21T03:45:35.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.691+0000 7fb1c2939640 1 -- 172.21.15.38:0/833544411 shutdown_connections 2024-09-21T03:45:35.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.691+0000 7fb1c2939640 1 --2- 172.21.15.38:0/833544411 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb1bc0b4b50 0x7fb1bc0b1720 unknown :-1 s=CLOSED pgs=2564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:35.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.691+0000 7fb1c2939640 1 --2- 172.21.15.38:0/833544411 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7fb1a8078b90 0x7fb1a807b0a0 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:35.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.691+0000 7fb1c2939640 1 --2- 172.21.15.38:0/833544411 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb1bc0b4740 0x7fb1bc1476d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:35.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.691+0000 7fb1c2939640 1 --2- 172.21.15.38:0/833544411 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb1bc06e1d0 0x7fb1bc0aef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:35.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.691+0000 7fb1c2939640 1 -- 172.21.15.38:0/833544411 >> 172.21.15.38:0/833544411 conn(0x7fb1bc0d0a30 msgr2=0x7fb1bc0d0e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:35.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.691+0000 7fb1c2939640 1 -- 172.21.15.38:0/833544411 shutdown_connections 2024-09-21T03:45:35.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.691+0000 7fb1c2939640 1 -- 172.21.15.38:0/833544411 wait complete. 2024-09-21T03:45:35.704 INFO:teuthology.orchestra.run.smithi038.stdout:{"pg_ready":true,"pg_map":{"version":511,"stamp":"2024-09-21T03:45:35.181354+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":307632,"kb_used_data":16520,"kb_used_omap":12,"kb_used_meta":290931,"kb_avail":754667088,"statfs":{"total":773094113280,"available":772779098112,"internally_reserved":0,"allocated":16916480,"data_stored":9138104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12748,"internal_metadata":297913908},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":-2,"num_object_clones":0,"num_object_copies":-4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-2,"num_whiteouts":0,"num_read":-4,"num_read_kb":-4,"num_write":-5,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.979176"},"pg_stats":[{"pgid":"1.0","version":"152'74","reported_seq":378,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-21T03:45:32.902820+0000","last_change":"2024-09-21T03:45:32.902820+0000","last_active":"2024-09-21T03:45:32.902820+0000","last_peered":"2024-09-21T03:45:32.902820+0000","last_clean":"2024-09-21T03:45:32.902820+0000","last_became_active":"2024-09-21T03:33:25.318399+0000","last_became_peered":"2024-09-21T03:33:25.318399+0000","last_unstale":"2024-09-21T03:45:32.902820+0000","last_undegraded":"2024-09-21T03:45:32.902820+0000","last_fullsized":"2024-09-21T03:45:32.902820+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"152'74","last_scrub_stamp":"2024-09-21T03:45:32.902691+0000","last_deep_scrub":"152'74","last_deep_scrub_stamp":"2024-09-21T03:45:32.902691+0000","last_clean_scrub_stamp":"2024-09-21T03:45:32.902691+0000","objects_scrubbed":2,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:16:14.797792+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":1,"up_from":16,"seq":68719476886,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42004,"kb_used_data":2500,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94329836,"statfs":{"total":96636764160,"available":96593752064,"internally_reserved":0,"allocated":2560000,"data_stored":1585039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509590,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":41436,"kb_used_data":1920,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330404,"statfs":{"total":96636764160,"available":96594333696,"internally_reserved":0,"allocated":1966080,"data_stored":994671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":40434123},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509589,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":41428,"kb_used_data":1920,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330412,"statfs":{"total":96636764160,"available":96594341888,"internally_reserved":0,"allocated":1966080,"data_stored":994671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509590,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37268,"kb_used_data":1920,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334572,"statfs":{"total":96636764160,"available":96598601728,"internally_reserved":0,"allocated":1966080,"data_stored":994671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509589,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37268,"kb_used_data":1920,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334572,"statfs":{"total":96636764160,"available":96598601728,"internally_reserved":0,"allocated":1966080,"data_stored":994671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542293,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33108,"kb_used_data":1920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338732,"statfs":{"total":96636764160,"available":96602861568,"internally_reserved":0,"allocated":1966080,"data_stored":994671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574997,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37852,"kb_used_data":2500,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333988,"statfs":{"total":96636764160,"available":96598003712,"internally_reserved":0,"allocated":2560000,"data_stored":1585039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542294,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37268,"kb_used_data":1920,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334572,"statfs":{"total":96636764160,"available":96598601728,"internally_reserved":0,"allocated":1966080,"data_stored":994671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T03:45:35.707 DEBUG:teuthology.orchestra.run.smithi038:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-09-21T03:45:35.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.821+0000 7f42fd215640 1 -- 172.21.15.38:0/4164357278 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f42f80ccc40 msgr2=0x7f42f815ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:35.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.821+0000 7f42fd215640 1 --2- 172.21.15.38:0/4164357278 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f42f80ccc40 0x7f42f815ca90 secure :-1 s=READY pgs=2566 cs=0 l=1 rev1=1 crypto rx=0x7f42f80cd050 tx=0x7f42f002a560 comp rx=0 tx=0).stop 2024-09-21T03:45:35.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.821+0000 7f42fd215640 1 -- 172.21.15.38:0/4164357278 shutdown_connections 2024-09-21T03:45:35.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.821+0000 7f42fd215640 1 --2- 172.21.15.38:0/4164357278 >> v2:172.21.15.195:3300/0 conn(0x7f42f80d0d80 0x7f42f80b5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:35.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.821+0000 7f42fd215640 1 --2- 172.21.15.38:0/4164357278 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f42f80ccc40 0x7f42f815ca90 unknown :-1 s=CLOSED pgs=2566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:35.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.821+0000 7f42fd215640 1 -- 172.21.15.38:0/4164357278 >> 172.21.15.38:0/4164357278 conn(0x7f42f80b46e0 msgr2=0x7f42f80d23e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:35.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.821+0000 7f42fd215640 1 -- 172.21.15.38:0/4164357278 shutdown_connections 2024-09-21T03:45:35.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.821+0000 7f42fd215640 1 -- 172.21.15.38:0/4164357278 wait complete. 2024-09-21T03:45:35.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.823+0000 7f42fd215640 1 Processor -- start 2024-09-21T03:45:35.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.823+0000 7f42fd215640 1 -- start start 2024-09-21T03:45:35.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.825+0000 7f42fd215640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f42f80ccc40 0x7f42f814e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:35.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.825+0000 7f42fd215640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f42f80cd440 0x7f42f80cb660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:35.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.825+0000 7f42fd215640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f42f815fd30 0x7f42f80cbba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:35.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.825+0000 7f42fd215640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f42f80b6650 con 0x7f42f80ccc40 2024-09-21T03:45:35.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.825+0000 7f42fd215640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f42f80b64d0 con 0x7f42f815fd30 2024-09-21T03:45:35.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.825+0000 7f42fd215640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f42f80b67d0 con 0x7f42f80cd440 2024-09-21T03:45:35.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.825+0000 7f42f6d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f42f80ccc40 0x7f42f814e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:35.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.825+0000 7f42f6d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f42f80ccc40 0x7f42f814e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38500/0 (socket says 172.21.15.38:38500) 2024-09-21T03:45:35.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.825+0000 7f42f6d9d640 1 -- 172.21.15.38:0/573831075 learned_addr learned my addr 172.21.15.38:0/573831075 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:45:35.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.825+0000 7f42f759e640 1 --2- 172.21.15.38:0/573831075 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f42f815fd30 0x7f42f80cbba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:35.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.825+0000 7f42f659c640 1 --2- 172.21.15.38:0/573831075 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f42f80cd440 0x7f42f80cb660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:35.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.825+0000 7f42f6d9d640 1 -- 172.21.15.38:0/573831075 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f42f80cd440 msgr2=0x7f42f80cb660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:35.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.825+0000 7f42f6d9d640 1 --2- 172.21.15.38:0/573831075 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f42f80cd440 0x7f42f80cb660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:35.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.825+0000 7f42f6d9d640 1 -- 172.21.15.38:0/573831075 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f42f815fd30 msgr2=0x7f42f80cbba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:35.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.825+0000 7f42f6d9d640 1 --2- 172.21.15.38:0/573831075 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f42f815fd30 0x7f42f80cbba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:35.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.825+0000 7f42f6d9d640 1 -- 172.21.15.38:0/573831075 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42f815d640 con 0x7f42f80ccc40 2024-09-21T03:45:35.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.825+0000 7f42f6d9d640 1 --2- 172.21.15.38:0/573831075 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f42f80ccc40 0x7f42f814e720 secure :-1 s=READY pgs=2567 cs=0 l=1 rev1=1 crypto rx=0x7f42f0000c00 tx=0x7f42f00039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:35.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.825+0000 7f42ef7fe640 1 -- 172.21.15.38:0/573831075 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42f003e030 con 0x7f42f80ccc40 2024-09-21T03:45:35.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.825+0000 7f42ef7fe640 1 -- 172.21.15.38:0/573831075 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f42f00047a0 con 0x7f42f80ccc40 2024-09-21T03:45:35.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.825+0000 7f42fd215640 1 -- 172.21.15.38:0/573831075 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42f815d920 con 0x7f42f80ccc40 2024-09-21T03:45:35.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.826+0000 7f42ef7fe640 1 -- 172.21.15.38:0/573831075 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42f0035040 con 0x7f42f80ccc40 2024-09-21T03:45:35.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.826+0000 7f42fd215640 1 -- 172.21.15.38:0/573831075 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42f815dee0 con 0x7f42f80ccc40 2024-09-21T03:45:35.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.827+0000 7f42fd215640 1 -- 172.21.15.38:0/573831075 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42f80b6d10 con 0x7f42f80ccc40 2024-09-21T03:45:35.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.827+0000 7f42ef7fe640 1 -- 172.21.15.38:0/573831075 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f42f0041030 con 0x7f42f80ccc40 2024-09-21T03:45:35.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.828+0000 7f42ef7fe640 1 --2- 172.21.15.38:0/573831075 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f42d8078bc0 0x7f42d807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:35.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.828+0000 7f42ef7fe640 1 -- 172.21.15.38:0/573831075 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 5421+0+0 (secure 0 0 0) 0x7f42f00bb3d0 con 0x7f42f80ccc40 2024-09-21T03:45:35.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.828+0000 7f42f659c640 1 --2- 172.21.15.38:0/573831075 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f42d8078bc0 0x7f42d807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:35.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.829+0000 7f42f659c640 1 --2- 172.21.15.38:0/573831075 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f42d8078bc0 0x7f42d807b0d0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f42f815b060 tx=0x7f42e0009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:35.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:35.832+0000 7f42ef7fe640 1 -- 172.21.15.38:0/573831075 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f42f00832d0 con 0x7f42f80ccc40 2024-09-21T03:45:36.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:36.021+0000 7f42fd215640 1 -- 172.21.15.38:0/573831075 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7f42f80d1dd0 con 0x7f42f80ccc40 2024-09-21T03:45:36.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:36.029+0000 7f42ef7fe640 1 -- 172.21.15.38:0/573831075 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v51)=0 v51) ==== 123+0+0 (secure 0 0 0) 0x7f42f0087d00 con 0x7f42f80ccc40 2024-09-21T03:45:36.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:36.030+0000 7f42ef7fe640 1 -- 172.21.15.38:0/573831075 <== mon.0 v2:172.21.15.38:3300/0 8 ==== config(6 keys) ==== 172+0+0 (secure 0 0 0) 0x7f42f0082de0 con 0x7f42f80ccc40 2024-09-21T03:45:36.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:36.032+0000 7f42fd215640 1 -- 172.21.15.38:0/573831075 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f42d8078bc0 msgr2=0x7f42d807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:36.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:36.032+0000 7f42fd215640 1 --2- 172.21.15.38:0/573831075 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f42d8078bc0 0x7f42d807b0d0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f42f815b060 tx=0x7f42e0009040 comp rx=0 tx=0).stop 2024-09-21T03:45:36.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:36.032+0000 7f42fd215640 1 -- 172.21.15.38:0/573831075 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f42f80ccc40 msgr2=0x7f42f814e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:36.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:36.032+0000 7f42fd215640 1 --2- 172.21.15.38:0/573831075 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f42f80ccc40 0x7f42f814e720 secure :-1 s=READY pgs=2567 cs=0 l=1 rev1=1 crypto rx=0x7f42f0000c00 tx=0x7f42f00039e0 comp rx=0 tx=0).stop 2024-09-21T03:45:36.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:36.032+0000 7f42fd215640 1 -- 172.21.15.38:0/573831075 shutdown_connections 2024-09-21T03:45:36.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:36.032+0000 7f42fd215640 1 --2- 172.21.15.38:0/573831075 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f42f815fd30 0x7f42f80cbba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:36.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:36.032+0000 7f42fd215640 1 --2- 172.21.15.38:0/573831075 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f42d8078bc0 0x7f42d807b0d0 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:36.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:36.032+0000 7f42fd215640 1 --2- 172.21.15.38:0/573831075 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f42f80cd440 0x7f42f80cb660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:36.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:36.032+0000 7f42fd215640 1 --2- 172.21.15.38:0/573831075 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f42f80ccc40 0x7f42f814e720 unknown :-1 s=CLOSED pgs=2567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:36.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:36.032+0000 7f42fd215640 1 -- 172.21.15.38:0/573831075 >> 172.21.15.38:0/573831075 conn(0x7f42f80b46e0 msgr2=0x7f42f815bb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:36.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:36.033+0000 7f42fd215640 1 -- 172.21.15.38:0/573831075 shutdown_connections 2024-09-21T03:45:36.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:36.033+0000 7f42fd215640 1 -- 172.21.15.38:0/573831075 wait complete. 2024-09-21T03:45:36.046 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-21T03:45:36.046 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T03:45:36.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:36.160+0000 7f056cbf1640 1 -- 172.21.15.38:0/2046869902 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0568074530 msgr2=0x7f056815e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:36.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:36.160+0000 7f056cbf1640 1 --2- 172.21.15.38:0/2046869902 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0568074530 0x7f056815e800 secure :-1 s=READY pgs=2569 cs=0 l=1 rev1=1 crypto rx=0x7f055800adb0 tx=0x7f055802b9c0 comp rx=0 tx=0).stop 2024-09-21T03:45:36.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:36.160+0000 7f056cbf1640 1 -- 172.21.15.38:0/2046869902 shutdown_connections 2024-09-21T03:45:36.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:36.160+0000 7f056cbf1640 1 --2- 172.21.15.38:0/2046869902 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0568074530 0x7f056815e800 unknown :-1 s=CLOSED pgs=2569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:36.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:36.160+0000 7f056cbf1640 1 --2- 172.21.15.38:0/2046869902 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f05680ca610 0x7f05680cdb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:36.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:36.160+0000 7f056cbf1640 1 -- 172.21.15.38:0/2046869902 >> 172.21.15.38:0/2046869902 conn(0x7f05680cca40 msgr2=0x7f05680cce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:36.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:36.160+0000 7f056cbf1640 1 -- 172.21.15.38:0/2046869902 shutdown_connections 2024-09-21T03:45:36.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:36.160+0000 7f056cbf1640 1 -- 172.21.15.38:0/2046869902 wait complete. 2024-09-21T03:45:36.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:36.162+0000 7f056cbf1640 1 Processor -- start 2024-09-21T03:45:36.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:36.162+0000 7f056cbf1640 1 -- start start 2024-09-21T03:45:36.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:36.163+0000 7f056cbf1640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0568074530 0x7f05680b2e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:36.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:36.163+0000 7f056cbf1640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f05680ca610 0x7f0568154e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:36.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:36.163+0000 7f056cbf1640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f05680ce0d0 0x7f05681f62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:36.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:36.163+0000 7f056cbf1640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f056806e2a0 con 0x7f05680ca610 2024-09-21T03:45:36.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:36.163+0000 7f056cbf1640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f056806e120 con 0x7f05680ce0d0 2024-09-21T03:45:36.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:36.163+0000 7f056cbf1640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f056806e420 con 0x7f0568074530 2024-09-21T03:45:36.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:36.163+0000 7f0565d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f05680ca610 0x7f0568154e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:36.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:36.164+0000 7f0565d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f05680ca610 0x7f0568154e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38530/0 (socket says 172.21.15.38:38530) 2024-09-21T03:45:36.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:36.164+0000 7f0565d9b640 1 -- 172.21.15.38:0/928477084 learned_addr learned my addr 172.21.15.38:0/928477084 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-21T03:45:36.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:36.164+0000 7f0566d9d640 1 --2- 172.21.15.38:0/928477084 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f05680ce0d0 0x7f05681f62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:36.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:36.164+0000 7f056659c640 1 --2- 172.21.15.38:0/928477084 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0568074530 0x7f05680b2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:36.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:36.164+0000 7f0565d9b640 1 -- 172.21.15.38:0/928477084 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0568074530 msgr2=0x7f05680b2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:36.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:36.164+0000 7f0565d9b640 1 --2- 172.21.15.38:0/928477084 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0568074530 0x7f05680b2e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:36.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:36.164+0000 7f0565d9b640 1 -- 172.21.15.38:0/928477084 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f05680ce0d0 msgr2=0x7f05681f62d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:36.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:36.164+0000 7f0565d9b640 1 --2- 172.21.15.38:0/928477084 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f05680ce0d0 0x7f05681f62d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:36.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:36.164+0000 7f0565d9b640 1 -- 172.21.15.38:0/928477084 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0568155350 con 0x7f05680ca610 2024-09-21T03:45:36.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:36.164+0000 7f056659c640 1 --2- 172.21.15.38:0/928477084 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0568074530 0x7f05680b2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:45:36.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:36.164+0000 7f0565d9b640 1 --2- 172.21.15.38:0/928477084 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f05680ca610 0x7f0568154e10 secure :-1 s=READY pgs=2570 cs=0 l=1 rev1=1 crypto rx=0x7f055000c3d0 tx=0x7f055000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:36.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:36.164+0000 7f055effd640 1 -- 172.21.15.38:0/928477084 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0550017070 con 0x7f05680ca610 2024-09-21T03:45:36.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:36.164+0000 7f055effd640 1 -- 172.21.15.38:0/928477084 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(6 keys) ==== 172+0+0 (secure 0 0 0) 0x7f05500052d0 con 0x7f05680ca610 2024-09-21T03:45:36.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:36.164+0000 7f056cbf1640 1 -- 172.21.15.38:0/928477084 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05680b5540 con 0x7f05680ca610 2024-09-21T03:45:36.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:36.164+0000 7f055effd640 1 -- 172.21.15.38:0/928477084 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0550003850 con 0x7f05680ca610 2024-09-21T03:45:36.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:36.164+0000 7f056cbf1640 1 -- 172.21.15.38:0/928477084 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05680b5b00 con 0x7f05680ca610 2024-09-21T03:45:36.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:36.165+0000 7f056cbf1640 1 -- 172.21.15.38:0/928477084 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f056815e800 con 0x7f05680ca610 2024-09-21T03:45:36.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:36.166+0000 7f055effd640 1 -- 172.21.15.38:0/928477084 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f055001a070 con 0x7f05680ca610 2024-09-21T03:45:36.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:36.167+0000 7f055effd640 1 --2- 172.21.15.38:0/928477084 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0548078b90 0x7f054807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:36.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:36.167+0000 7f055effd640 1 -- 172.21.15.38:0/928477084 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 5421+0+0 (secure 0 0 0) 0x7f0550099e70 con 0x7f05680ca610 2024-09-21T03:45:36.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:36.167+0000 7f056659c640 1 --2- 172.21.15.38:0/928477084 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0548078b90 0x7f054807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:36.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:36.168+0000 7f056659c640 1 --2- 172.21.15.38:0/928477084 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0548078b90 0x7f054807b0a0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f0560005fd0 tx=0x7f056000f360 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:36.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:36.171+0000 7f055effd640 1 -- 172.21.15.38:0/928477084 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f0550061e10 con 0x7f05680ca610 2024-09-21T03:45:36.398 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:36.396+0000 7f056cbf1640 1 -- 172.21.15.38:0/928477084 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f056809d960 con 0x7f05680ca610 2024-09-21T03:45:36.399 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:36.397+0000 7f055effd640 1 -- 172.21.15.38:0/928477084 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 351 v351) ==== 95+0+3920 (secure 0 0 0) 0x7f0550066840 con 0x7f05680ca610 2024-09-21T03:45:36.399 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-21T03:45:36.399 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":351,"btime":"2024-09-21T03:45:25:192941+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8399,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/3606957623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":3606957623},{"type":"v1","addr":"172.21.15.38:6835","nonce":3606957623}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":338},{"gid":8405,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/456651486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":456651486},{"type":"v1","addr":"172.21.15.195:6837","nonce":456651486}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":338},{"gid":9604,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/2649311840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":2649311840},{"type":"v1","addr":"172.21.15.195:6835","nonce":2649311840}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":338},{"gid":11202,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/2685246673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":2685246673},{"type":"v1","addr":"172.21.15.38:6833","nonce":2685246673}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":351}],"filesystems":[]} 2024-09-21T03:45:36.399 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 351 2024-09-21T03:45:36.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:36.399+0000 7f056cbf1640 1 -- 172.21.15.38:0/928477084 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0548078b90 msgr2=0x7f054807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:36.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:36.399+0000 7f056cbf1640 1 --2- 172.21.15.38:0/928477084 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0548078b90 0x7f054807b0a0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f0560005fd0 tx=0x7f056000f360 comp rx=0 tx=0).stop 2024-09-21T03:45:36.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:36.399+0000 7f056cbf1640 1 -- 172.21.15.38:0/928477084 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f05680ca610 msgr2=0x7f0568154e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:36.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:36.399+0000 7f056cbf1640 1 --2- 172.21.15.38:0/928477084 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f05680ca610 0x7f0568154e10 secure :-1 s=READY pgs=2570 cs=0 l=1 rev1=1 crypto rx=0x7f055000c3d0 tx=0x7f055000c890 comp rx=0 tx=0).stop 2024-09-21T03:45:36.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:36.399+0000 7f056cbf1640 1 -- 172.21.15.38:0/928477084 shutdown_connections 2024-09-21T03:45:36.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:36.399+0000 7f056cbf1640 1 --2- 172.21.15.38:0/928477084 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f05680ce0d0 0x7f05681f62d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:36.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:36.399+0000 7f056cbf1640 1 --2- 172.21.15.38:0/928477084 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f05680ca610 0x7f0568154e10 unknown :-1 s=CLOSED pgs=2570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:36.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:36.399+0000 7f056cbf1640 1 --2- 172.21.15.38:0/928477084 >> [v2:172.21.15.195:6800/1631901344,v1:172.21.15.195:6801/1631901344] conn(0x7f0548078b90 0x7f054807b0a0 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:36.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:36.399+0000 7f056cbf1640 1 --2- 172.21.15.38:0/928477084 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0568074530 0x7f05680b2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:36.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:36.399+0000 7f056cbf1640 1 -- 172.21.15.38:0/928477084 >> 172.21.15.38:0/928477084 conn(0x7f05680cca40 msgr2=0x7f05681e3df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:36.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:36.399+0000 7f056cbf1640 1 -- 172.21.15.38:0/928477084 shutdown_connections 2024-09-21T03:45:36.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-21T03:45:36.399+0000 7f056cbf1640 1 -- 172.21.15.38:0/928477084 wait complete. 2024-09-21T03:45:36.413 DEBUG:tasks.cephfs.filesystem:already dead... 2024-09-21T03:45:36.413 INFO:teuthology.misc:Shutting down mds daemons... 2024-09-21T03:45:36.413 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-21T03:45:36.413 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T03:45:36.422 INFO:tasks.ceph.mds.a:Stopped 2024-09-21T03:45:36.422 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-21T03:45:36.422 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T03:45:36.476 INFO:tasks.ceph.mds.c:Stopped 2024-09-21T03:45:36.476 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-21T03:45:36.476 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T03:45:36.488 INFO:tasks.ceph.mds.b:Stopped 2024-09-21T03:45:36.488 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-21T03:45:36.488 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T03:45:36.540 INFO:tasks.ceph.mds.d:Stopped 2024-09-21T03:45:36.540 INFO:teuthology.misc:Shutting down osd daemons... 2024-09-21T03:45:36.540 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-21T03:45:36.540 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T03:45:36.580 INFO:tasks.ceph.osd.0:Stopped 2024-09-21T03:45:36.581 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-21T03:45:36.581 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T03:45:36.660 INFO:tasks.ceph.osd.1:Stopped 2024-09-21T03:45:36.661 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-21T03:45:36.661 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T03:45:36.729 INFO:tasks.ceph.osd.2:Stopped 2024-09-21T03:45:36.729 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-09-21T03:45:36.729 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T03:45:36.797 INFO:tasks.ceph.osd.3:Stopped 2024-09-21T03:45:36.798 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-09-21T03:45:36.798 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T03:45:36.836 INFO:tasks.ceph.osd.4:Stopped 2024-09-21T03:45:36.836 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-09-21T03:45:36.836 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T03:45:36.907 INFO:tasks.ceph.osd.5:Stopped 2024-09-21T03:45:36.908 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-09-21T03:45:36.908 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T03:45:36.984 INFO:tasks.ceph.osd.6:Stopped 2024-09-21T03:45:36.984 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-09-21T03:45:36.984 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T03:45:37.057 INFO:tasks.ceph.osd.7:Stopped 2024-09-21T03:45:37.058 INFO:teuthology.misc:Shutting down mgr daemons... 2024-09-21T03:45:37.058 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-09-21T03:45:37.058 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T03:45:37.087 INFO:tasks.ceph.mgr.y:Stopped 2024-09-21T03:45:37.088 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-09-21T03:45:37.088 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T03:45:37.128 INFO:tasks.ceph.mgr.x:Stopped 2024-09-21T03:45:37.128 INFO:teuthology.misc:Shutting down mon daemons... 2024-09-21T03:45:37.128 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-21T03:45:37.128 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T03:45:37.142 INFO:tasks.ceph.mon.a:Stopped 2024-09-21T03:45:37.142 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-21T03:45:37.142 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T03:45:37.180 INFO:tasks.ceph.mon.b:Stopped 2024-09-21T03:45:37.181 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-21T03:45:37.181 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T03:45:37.233 INFO:tasks.ceph.mon.c:Stopped 2024-09-21T03:45:37.233 INFO:tasks.ceph:Checking cluster log for badness... 2024-09-21T03:45:37.234 DEBUG:teuthology.orchestra.run.smithi038:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v 'fs.*is degraded' | egrep -v 'filesystem is degraded' | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v 'filesystem is offline' | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v 'insufficient standby MDS daemons available' | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'bad backtrace' | egrep -v 'object missing on disk' | egrep -v 'error reading table object' | egrep -v 'error reading sessionmap' | egrep -v 'Error loading MDS rank' | egrep -v 'missing journal object' | egrep -v 'Error recovering journal' | egrep -v 'error decoding table object' | egrep -v 'failed to read JournalPointer' | egrep -v 'Corrupt directory entry' | egrep -v 'Corrupt fnode header' | egrep -v 'corrupt sessionmap header' | egrep -v 'Corrupt dentry' | egrep -v 'Scrub error on inode' | egrep -v 'Metadata damage detected' | egrep -v MDS_READ_ONLY | egrep -v 'force file system read-only' | egrep -v 'with standby daemon mds' | egrep -v 'MDS abort because newly corrupt dentry' | head -n 1 2024-09-21T03:45:37.281 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi038.front.sepia.ceph.com 2024-09-21T03:45:37.281 DEBUG:teuthology.orchestra.run.smithi038:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-09-21T03:45:37.511 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi038.front.sepia.ceph.com 2024-09-21T03:45:37.511 DEBUG:teuthology.orchestra.run.smithi038:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-09-21T03:45:37.595 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi038.front.sepia.ceph.com 2024-09-21T03:45:37.595 DEBUG:teuthology.orchestra.run.smithi038:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-09-21T03:45:37.657 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi038.front.sepia.ceph.com 2024-09-21T03:45:37.658 DEBUG:teuthology.orchestra.run.smithi038:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-09-21T03:45:37.713 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi195.front.sepia.ceph.com 2024-09-21T03:45:37.714 DEBUG:teuthology.orchestra.run.smithi195:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-09-21T03:45:37.883 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi195.front.sepia.ceph.com 2024-09-21T03:45:37.883 DEBUG:teuthology.orchestra.run.smithi195:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-09-21T03:45:37.954 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi195.front.sepia.ceph.com 2024-09-21T03:45:37.954 DEBUG:teuthology.orchestra.run.smithi195:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-09-21T03:45:38.011 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi195.front.sepia.ceph.com 2024-09-21T03:45:38.011 DEBUG:teuthology.orchestra.run.smithi195:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-09-21T03:45:38.061 INFO:tasks.ceph:Archiving mon data... 2024-09-21T03:45:38.062 DEBUG:teuthology.misc:Transferring archived files from smithi038:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913473/data/mon.a.tgz 2024-09-21T03:45:38.063 DEBUG:teuthology.orchestra.run.smithi038:> mktemp 2024-09-21T03:45:38.080 INFO:teuthology.orchestra.run.smithi038.stdout:/tmp/tmp.oFCm8SvnEe 2024-09-21T03:45:38.081 DEBUG:teuthology.orchestra.run.smithi038:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.oFCm8SvnEe 2024-09-21T03:45:38.290 DEBUG:teuthology.orchestra.run.smithi038:> sudo chmod 0666 /tmp/tmp.oFCm8SvnEe 2024-09-21T03:45:38.375 DEBUG:teuthology.orchestra.remote:smithi038:/tmp/tmp.oFCm8SvnEe is 645KB 2024-09-21T03:45:38.448 DEBUG:teuthology.orchestra.run.smithi038:> rm -fr /tmp/tmp.oFCm8SvnEe 2024-09-21T03:45:38.463 DEBUG:teuthology.misc:Transferring archived files from smithi195:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913473/data/mon.b.tgz 2024-09-21T03:45:38.464 DEBUG:teuthology.orchestra.run.smithi195:> mktemp 2024-09-21T03:45:38.481 INFO:teuthology.orchestra.run.smithi195.stdout:/tmp/tmp.UxQdz6qzoA 2024-09-21T03:45:38.482 DEBUG:teuthology.orchestra.run.smithi195:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.UxQdz6qzoA 2024-09-21T03:45:38.682 DEBUG:teuthology.orchestra.run.smithi195:> sudo chmod 0666 /tmp/tmp.UxQdz6qzoA 2024-09-21T03:45:38.765 DEBUG:teuthology.orchestra.remote:smithi195:/tmp/tmp.UxQdz6qzoA is 665KB 2024-09-21T03:45:38.835 DEBUG:teuthology.orchestra.run.smithi195:> rm -fr /tmp/tmp.UxQdz6qzoA 2024-09-21T03:45:38.850 DEBUG:teuthology.misc:Transferring archived files from smithi195:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913473/data/mon.c.tgz 2024-09-21T03:45:38.850 DEBUG:teuthology.orchestra.run.smithi195:> mktemp 2024-09-21T03:45:38.904 INFO:teuthology.orchestra.run.smithi195.stdout:/tmp/tmp.z4eqvHMikc 2024-09-21T03:45:38.905 DEBUG:teuthology.orchestra.run.smithi195:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.z4eqvHMikc 2024-09-21T03:45:39.091 DEBUG:teuthology.orchestra.run.smithi195:> sudo chmod 0666 /tmp/tmp.z4eqvHMikc 2024-09-21T03:45:39.175 DEBUG:teuthology.orchestra.remote:smithi195:/tmp/tmp.z4eqvHMikc is 665KB 2024-09-21T03:45:39.275 DEBUG:teuthology.orchestra.run.smithi195:> rm -fr /tmp/tmp.z4eqvHMikc 2024-09-21T03:45:39.290 INFO:tasks.ceph:Cleaning ceph cluster... 2024-09-21T03:45:39.291 DEBUG:teuthology.orchestra.run.smithi038:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-21T03:45:39.293 DEBUG:teuthology.orchestra.run.smithi195:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-21T03:45:39.379 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-21T03:45:39.410 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-21T03:45:39.410 INFO:tasks.ceph:Archiving crash dumps... 2024-09-21T03:45:39.412 DEBUG:teuthology.misc:Transferring archived files from smithi038:/var/lib/ceph/crash to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913473/remote/smithi038/crash 2024-09-21T03:45:39.413 DEBUG:teuthology.orchestra.run.smithi038:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-21T03:45:39.491 DEBUG:teuthology.misc:Transferring archived files from smithi195:/var/lib/ceph/crash to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913473/remote/smithi195/crash 2024-09-21T03:45:39.492 DEBUG:teuthology.orchestra.run.smithi195:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-21T03:45:39.516 INFO:tasks.ceph:Shutting down logrotate 2024-09-21T03:45:39.516 DEBUG:teuthology.orchestra.run.smithi038:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T03:45:39.519 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T03:45:39.608 DEBUG:teuthology.orchestra.run.smithi038:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-09-21T03:45:39.638 DEBUG:teuthology.orchestra.run.smithi195:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-09-21T03:45:39.662 INFO:tasks.ceph:Compressing logs... 2024-09-21T03:45:39.663 DEBUG:teuthology.orchestra.run.smithi038:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-21T03:45:39.682 DEBUG:teuthology.orchestra.run.smithi195:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-21T03:45:39.708 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-09-21T03:45:39.708 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71282.log 2024-09-21T03:45:39.709 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52128.log 2024-09-21T03:45:39.710 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-osd.0.log: /var/log/ceph/ceph-client.admin.71282.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56794.log 2024-09-21T03:45:39.711 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.52128.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47350.log 2024-09-21T03:45:39.711 INFO:teuthology.orchestra.run.smithi038.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.71282.log.gz 2024-09-21T03:45:39.711 INFO:teuthology.orchestra.run.smithi038.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.52128.log.gz 2024-09-21T03:45:39.711 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.56794.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44702.log 2024-09-21T03:45:39.711 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.47350.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53555.log 2024-09-21T03:45:39.711 INFO:teuthology.orchestra.run.smithi038.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.56794.log.gz 2024-09-21T03:45:39.711 INFO:teuthology.orchestra.run.smithi038.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.47350.log.gz 2024-09-21T03:45:39.711 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.44702.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67385.log 2024-09-21T03:45:39.711 INFO:teuthology.orchestra.run.smithi038.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.44702.log.gz 2024-09-21T03:45:39.720 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.53555.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69727.log 2024-09-21T03:45:39.720 INFO:teuthology.orchestra.run.smithi038.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.53555.log.gz 2024-09-21T03:45:39.720 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.67385.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44618.log 2024-09-21T03:45:39.721 INFO:teuthology.orchestra.run.smithi038.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.67385.log.gz 2024-09-21T03:45:39.721 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.69727.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78509.log 2024-09-21T03:45:39.721 INFO:teuthology.orchestra.run.smithi038.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.69727.log.gz 2024-09-21T03:45:39.721 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.44618.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79247.log 2024-09-21T03:45:39.721 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.78509.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.44618.log.gz 2024-09-21T03:45:39.721 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37460.log 2024-09-21T03:45:39.721 INFO:teuthology.orchestra.run.smithi038.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.78509.log.gz 2024-09-21T03:45:39.721 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.79247.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52788.log 2024-09-21T03:45:39.721 INFO:teuthology.orchestra.run.smithi038.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.79247.log.gz 2024-09-21T03:45:39.721 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.37460.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91219.log 2024-09-21T03:45:39.721 INFO:teuthology.orchestra.run.smithi038.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.37460.log.gz 2024-09-21T03:45:39.721 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.52788.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67249.log 2024-09-21T03:45:39.722 INFO:teuthology.orchestra.run.smithi038.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.52788.log.gz 2024-09-21T03:45:39.722 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.91219.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53471.log 2024-09-21T03:45:39.722 INFO:teuthology.orchestra.run.smithi038.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.91219.log.gz 2024-09-21T03:45:39.722 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.67249.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87143.log 2024-09-21T03:45:39.722 INFO:teuthology.orchestra.run.smithi038.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.67249.log.gz 2024-09-21T03:45:39.722 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.53471.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91918.log 2024-09-21T03:45:39.722 INFO:teuthology.orchestra.run.smithi038.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.53471.log.gz 2024-09-21T03:45:39.722 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.87143.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53122.log 2024-09-21T03:45:39.722 INFO:teuthology.orchestra.run.smithi038.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.87143.log.gz 2024-09-21T03:45:39.722 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.91918.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91050.log 2024-09-21T03:45:39.722 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.53122.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.91918.log.gz 2024-09-21T03:45:39.722 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56089.log 2024-09-21T03:45:39.722 INFO:teuthology.orchestra.run.smithi038.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.53122.log.gz 2024-09-21T03:45:39.723 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.91050.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59577.log 2024-09-21T03:45:39.723 INFO:teuthology.orchestra.run.smithi038.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.91050.log.gz 2024-09-21T03:45:39.723 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.56089.log: -- /var/log/ceph/ceph-client.admin.51755.log 2024-09-21T03:45:39.723 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.59577.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.56089.log.gz 2024-09-21T03:45:39.723 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78180.log 2024-09-21T03:45:39.723 INFO:teuthology.orchestra.run.smithi038.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.59577.log.gz 2024-09-21T03:45:39.723 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.51755.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68343.log 2024-09-21T03:45:39.723 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.78180.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60580.log 2024-09-21T03:45:39.723 INFO:teuthology.orchestra.run.smithi038.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.51755.log.gz 2024-09-21T03:45:39.723 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.68343.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.78180.log.gz 2024-09-21T03:45:39.723 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60710.log 2024-09-21T03:45:39.723 INFO:teuthology.orchestra.run.smithi038.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.68343.log.gz 2024-09-21T03:45:39.724 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.60580.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62209.log 2024-09-21T03:45:39.724 INFO:teuthology.orchestra.run.smithi038.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.60580.log.gz 2024-09-21T03:45:39.724 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.60710.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59914.log 2024-09-21T03:45:39.724 INFO:teuthology.orchestra.run.smithi038.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.60710.log.gz 2024-09-21T03:45:39.724 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.62209.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65802.log 2024-09-21T03:45:39.724 INFO:teuthology.orchestra.run.smithi038.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.62209.log.gz 2024-09-21T03:45:39.724 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.59914.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52441.log 2024-09-21T03:45:39.724 INFO:teuthology.orchestra.run.smithi038.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.59914.log.gz 2024-09-21T03:45:39.724 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.65802.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61900.log 2024-09-21T03:45:39.724 INFO:teuthology.orchestra.run.smithi038.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.65802.log.gz 2024-09-21T03:45:39.724 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.52441.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55494.log 2024-09-21T03:45:39.724 INFO:teuthology.orchestra.run.smithi038.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.52441.log.gz 2024-09-21T03:45:39.724 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.61900.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89478.log 2024-09-21T03:45:39.725 INFO:teuthology.orchestra.run.smithi038.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.61900.log.gz 2024-09-21T03:45:39.725 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.55494.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60626.log 2024-09-21T03:45:39.725 INFO:teuthology.orchestra.run.smithi038.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.55494.log.gz 2024-09-21T03:45:39.725 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.89478.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90460.log 2024-09-21T03:45:39.745 INFO:teuthology.orchestra.run.smithi195.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.3.41737.log 2024-09-21T03:45:39.745 INFO:teuthology.orchestra.run.smithi195.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-09-21T03:45:39.745 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ceph-client.3.41737.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36278.log 2024-09-21T03:45:39.745 INFO:teuthology.orchestra.run.smithi195.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.3.41737.log.gz 2024-09-21T03:45:39.745 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36398.log 2024-09-21T03:45:39.745 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ceph-client.admin.36278.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-09-21T03:45:39.745 INFO:teuthology.orchestra.run.smithi195.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.36278.log.gz 2024-09-21T03:45:39.745 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ceph-client.admin.36398.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-09-21T03:45:39.745 INFO:teuthology.orchestra.run.smithi195.stderr: 86.8%/var/log/ceph/ceph-mds.b.log: -- replaced with /var/log/ceph/ceph-client.admin.36398.log.gz 2024-09-21T03:45:39.745 INFO:teuthology.orchestra.run.smithi195.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-21T03:45:39.745 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36438.log 2024-09-21T03:45:39.745 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36478.log 2024-09-21T03:45:39.746 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ceph-client.admin.36438.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.41332.log 2024-09-21T03:45:39.746 INFO:teuthology.orchestra.run.smithi195.stderr: 86.6%/var/log/ceph/ceph-client.admin.36478.log: -- replaced with /var/log/ceph/ceph-client.admin.36438.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-09-21T03:45:39.746 INFO:teuthology.orchestra.run.smithi195.stderr: 2024-09-21T03:45:39.746 INFO:teuthology.orchestra.run.smithi195.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.36478.log.gz 2024-09-21T03:45:39.746 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ceph-client.1.41332.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36518.log 0.0% 2024-09-21T03:45:39.746 INFO:teuthology.orchestra.run.smithi195.stderr: -- replaced with /var/log/ceph/ceph-client.1.41332.log.gz 2024-09-21T03:45:39.746 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-09-21T03:45:39.746 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ceph-client.admin.36518.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.3.41733.log 2024-09-21T03:45:39.746 INFO:teuthology.orchestra.run.smithi195.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.36518.log.gz 2024-09-21T03:45:39.746 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-09-21T03:45:39.746 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ceph-client.3.41733.log: 0.0% -- replaced with /var/log/ceph/ceph-client.3.41733.log.gzgzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-21T03:45:39.746 INFO:teuthology.orchestra.run.smithi195.stderr: 2024-09-21T03:45:39.746 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ceph-osd.5.log: 94.0% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-21T03:45:39.746 INFO:teuthology.orchestra.run.smithi195.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-09-21T03:45:39.747 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-09-21T03:45:39.747 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ceph-mds.d.log: 92.7% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-21T03:45:39.757 INFO:teuthology.orchestra.run.smithi195.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36318.log 2024-09-21T03:45:39.768 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.41337.log 2024-09-21T03:45:39.782 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ceph-client.admin.36318.log: 86.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36358.log 2024-09-21T03:45:39.782 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ceph-client.1.41337.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.41337.log.gz 2024-09-21T03:45:39.782 INFO:teuthology.orchestra.run.smithi195.stderr: -- replaced with /var/log/ceph/ceph-client.admin.36318.log.gz 2024-09-21T03:45:39.792 INFO:teuthology.orchestra.run.smithi195.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-09-21T03:45:39.805 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ceph-client.admin.36358.log: 91.7% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-09-21T03:45:39.805 INFO:teuthology.orchestra.run.smithi195.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.36358.log.gz 2024-09-21T03:45:39.805 INFO:teuthology.orchestra.run.smithi195.stderr: 92.7% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-09-21T03:45:39.812 INFO:teuthology.orchestra.run.smithi195.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36558.log 2024-09-21T03:45:39.812 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ceph-mon.b.log: /var/log/ceph/ceph-client.admin.36558.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.36558.log.gz 2024-09-21T03:45:39.848 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.60626.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.89478.log.gz 2024-09-21T03:45:39.848 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86178.log 2024-09-21T03:45:39.848 INFO:teuthology.orchestra.run.smithi038.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.60626.log.gz 2024-09-21T03:45:39.848 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.90460.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46064.log 2024-09-21T03:45:39.848 INFO:teuthology.orchestra.run.smithi038.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.90460.log.gz 2024-09-21T03:45:39.848 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.86178.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79863.log 2024-09-21T03:45:39.849 INFO:teuthology.orchestra.run.smithi038.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.86178.log.gz 2024-09-21T03:45:39.849 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.46064.log: /var/log/ceph/ceph-client.admin.36187.log 2024-09-21T03:45:39.849 INFO:teuthology.orchestra.run.smithi038.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46064.log.gz 2024-09-21T03:45:39.849 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.79863.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53387.log 2024-09-21T03:45:39.849 INFO:teuthology.orchestra.run.smithi038.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.79863.log.gz 2024-09-21T03:45:39.849 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.36187.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89971.log 2024-09-21T03:45:39.849 INFO:teuthology.orchestra.run.smithi038.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36187.log.gz 2024-09-21T03:45:39.849 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.53387.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56752.log 2024-09-21T03:45:39.849 INFO:teuthology.orchestra.run.smithi038.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.53387.log.gz 2024-09-21T03:45:39.849 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.89971.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51096.log 2024-09-21T03:45:39.849 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.56752.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56314.log 2024-09-21T03:45:39.849 INFO:teuthology.orchestra.run.smithi038.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.56752.log.gz 2024-09-21T03:45:39.849 INFO:teuthology.orchestra.run.smithi038.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.89971.log.gz 2024-09-21T03:45:39.849 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.51096.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48810.log 2024-09-21T03:45:39.849 INFO:teuthology.orchestra.run.smithi038.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.51096.log.gz 2024-09-21T03:45:39.850 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.56314.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51173.log 2024-09-21T03:45:39.850 INFO:teuthology.orchestra.run.smithi038.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.56314.log.gz 2024-09-21T03:45:39.850 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82242.log 2024-09-21T03:45:39.850 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.48810.log: /var/log/ceph/ceph-client.admin.51173.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.48810.log.gz 2024-09-21T03:45:39.850 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91752.log 2024-09-21T03:45:39.850 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.82242.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84555.log 2024-09-21T03:45:39.850 INFO:teuthology.orchestra.run.smithi038.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.51173.log.gz 2024-09-21T03:45:39.850 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53513.log 2024-09-21T03:45:39.850 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.91752.log: /var/log/ceph/ceph-client.admin.84555.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.91752.log.gz 2024-09-21T03:45:39.850 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74631.log 2024-09-21T03:45:39.850 INFO:teuthology.orchestra.run.smithi038.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.84555.log.gz 2024-09-21T03:45:39.850 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75853.log 2024-09-21T03:45:39.851 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.53513.log: 92.5% -- replaced with /var/log/ceph/ceph-client.admin.82242.log.gz 2024-09-21T03:45:39.851 INFO:teuthology.orchestra.run.smithi038.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.53513.log.gz 2024-09-21T03:45:39.851 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.74631.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90966.log 2024-09-21T03:45:39.851 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.75853.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.74631.log.gz 2024-09-21T03:45:39.851 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81999.log 2024-09-21T03:45:39.851 INFO:teuthology.orchestra.run.smithi038.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.75853.log.gz 2024-09-21T03:45:39.851 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.90966.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73233.log 2024-09-21T03:45:39.851 INFO:teuthology.orchestra.run.smithi038.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.90966.log.gz 2024-09-21T03:45:39.851 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.81999.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48294.log 2024-09-21T03:45:39.851 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.73233.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.81999.log.gz 2024-09-21T03:45:39.851 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91134.log 2024-09-21T03:45:39.851 INFO:teuthology.orchestra.run.smithi038.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.73233.log.gz 2024-09-21T03:45:39.851 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.48294.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91794.log 2024-09-21T03:45:39.851 INFO:teuthology.orchestra.run.smithi038.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48294.log.gz 2024-09-21T03:45:39.851 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73191.log 2024-09-21T03:45:39.852 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.91134.log: /var/log/ceph/ceph-client.admin.91794.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.91134.log.gz 2024-09-21T03:45:39.852 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71222.log 2024-09-21T03:45:39.852 INFO:teuthology.orchestra.run.smithi038.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.91794.log.gz 2024-09-21T03:45:39.852 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.73191.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85237.log 2024-09-21T03:45:39.852 INFO:teuthology.orchestra.run.smithi038.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.73191.log.gz 2024-09-21T03:45:39.852 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.71222.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76666.log 2024-09-21T03:45:39.852 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.85237.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.71222.log.gz 2024-09-21T03:45:39.852 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60440.log 2024-09-21T03:45:39.852 INFO:teuthology.orchestra.run.smithi038.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.85237.log.gz 2024-09-21T03:45:39.852 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.76666.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67661.log 2024-09-21T03:45:39.852 INFO:teuthology.orchestra.run.smithi038.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.76666.log.gz 2024-09-21T03:45:39.852 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.60440.log: -- /var/log/ceph/ceph-client.admin.44968.log 2024-09-21T03:45:39.852 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.67661.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.60440.log.gz 2024-09-21T03:45:39.852 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73626.log 2024-09-21T03:45:39.853 INFO:teuthology.orchestra.run.smithi038.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.67661.log.gz 2024-09-21T03:45:39.853 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.44968.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88012.log 2024-09-21T03:45:39.853 INFO:teuthology.orchestra.run.smithi038.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.44968.log.gz 2024-09-21T03:45:39.853 INFO:teuthology.orchestra.run.smithi038.stderr:gzip/var/log/ceph/ceph-client.admin.73626.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.72924.log 2024-09-21T03:45:39.853 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.88012.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.73626.log.gz 2024-09-21T03:45:39.853 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90079.log 2024-09-21T03:45:39.853 INFO:teuthology.orchestra.run.smithi038.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.88012.log.gz 2024-09-21T03:45:39.853 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.72924.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67586.log 2024-09-21T03:45:39.853 INFO:teuthology.orchestra.run.smithi038.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.72924.log.gz 2024-09-21T03:45:39.853 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43457.log 2024-09-21T03:45:39.853 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.90079.log: /var/log/ceph/ceph-client.admin.67586.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42586.log 2024-09-21T03:45:39.853 INFO:teuthology.orchestra.run.smithi038.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.67586.log.gz 2024-09-21T03:45:39.853 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.43457.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.90079.log.gz 2024-09-21T03:45:39.853 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82335.log 2024-09-21T03:45:39.853 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76222.log 2024-09-21T03:45:39.854 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.42586.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.43457.log.gz 2024-09-21T03:45:39.854 INFO:teuthology.orchestra.run.smithi038.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.42586.log.gz 2024-09-21T03:45:39.854 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.82335.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77367.log 2024-09-21T03:45:39.854 INFO:teuthology.orchestra.run.smithi038.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.82335.log.gz 2024-09-21T03:45:39.854 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.76222.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54702.log 2024-09-21T03:45:39.854 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.77367.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.76222.log.gz 2024-09-21T03:45:39.854 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63934.log 2024-09-21T03:45:39.854 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.54702.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56184.log 2024-09-21T03:45:39.854 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.63934.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.54702.log.gz 2024-09-21T03:45:39.854 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68595.log 2024-09-21T03:45:39.854 INFO:teuthology.orchestra.run.smithi038.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.63934.log.gz 2024-09-21T03:45:39.854 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75732.log 2024-09-21T03:45:39.854 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.56184.log: 92.4% -- replaced with /var/log/ceph/ceph-client.admin.77367.log.gz 2024-09-21T03:45:39.855 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.68595.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.56184.log.gz 2024-09-21T03:45:39.855 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71736.log 2024-09-21T03:45:39.855 INFO:teuthology.orchestra.run.smithi038.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.68595.log.gz 2024-09-21T03:45:39.855 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.75732.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53639.log 2024-09-21T03:45:39.855 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.71736.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.75732.log.gz 2024-09-21T03:45:39.855 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66745.log 2024-09-21T03:45:39.855 INFO:teuthology.orchestra.run.smithi038.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.71736.log.gz 2024-09-21T03:45:39.855 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.53639.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36547.log 2024-09-21T03:45:39.855 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.66745.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.53639.log.gz 2024-09-21T03:45:39.855 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64914.log 2024-09-21T03:45:39.855 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.36547.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45278.log 2024-09-21T03:45:39.855 INFO:teuthology.orchestra.run.smithi038.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.66745.log.gz 2024-09-21T03:45:39.855 INFO:teuthology.orchestra.run.smithi038.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.36547.log.gz 2024-09-21T03:45:39.856 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.64914.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62885.log 2024-09-21T03:45:39.856 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.45278.log: 89.2%gzip -- replaced with /var/log/ceph/ceph-client.admin.64914.log.gz 2024-09-21T03:45:39.856 INFO:teuthology.orchestra.run.smithi038.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.56860.log 2024-09-21T03:45:39.856 INFO:teuthology.orchestra.run.smithi038.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.45278.log.gz 2024-09-21T03:45:39.856 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.62885.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73710.log 2024-09-21T03:45:39.856 INFO:teuthology.orchestra.run.smithi038.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.62885.log.gz 2024-09-21T03:45:39.856 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.56860.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81028.log 2024-09-21T03:45:39.856 INFO:teuthology.orchestra.run.smithi038.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.56860.log.gz 2024-09-21T03:45:39.856 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.73710.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.54019.log 2024-09-21T03:45:39.856 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.81028.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87570.log 2024-09-21T03:45:39.856 INFO:teuthology.orchestra.run.smithi038.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.81028.log.gz 2024-09-21T03:45:39.856 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.0.54019.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.54019.log.gz 2024-09-21T03:45:39.856 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72234.log 2024-09-21T03:45:39.857 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.87570.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72088.log 2024-09-21T03:45:39.857 INFO:teuthology.orchestra.run.smithi038.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.87570.log.gz 2024-09-21T03:45:39.857 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.72234.log: 92.5% -- replaced with /var/log/ceph/ceph-client.admin.73710.log.gz 2024-09-21T03:45:39.857 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88054.log 2024-09-21T03:45:39.857 INFO:teuthology.orchestra.run.smithi038.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.72234.log.gz 2024-09-21T03:45:39.857 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.72088.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74715.log 2024-09-21T03:45:39.857 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.88054.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86838.log 2024-09-21T03:45:39.857 INFO:teuthology.orchestra.run.smithi038.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.88054.log.gz 2024-09-21T03:45:39.857 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.74715.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75319.log 2024-09-21T03:45:39.857 INFO:teuthology.orchestra.run.smithi038.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.74715.log.gz 2024-09-21T03:45:39.857 INFO:teuthology.orchestra.run.smithi038.stderr: 92.5% -- replaced with /var/log/ceph/ceph-client.admin.72088.log.gz 2024-09-21T03:45:39.857 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.86838.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83223.log 2024-09-21T03:45:39.857 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.75319.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86441.log 2024-09-21T03:45:39.858 INFO:teuthology.orchestra.run.smithi038.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.86838.log.gz 2024-09-21T03:45:39.858 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.83223.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.75319.log.gz 2024-09-21T03:45:39.858 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90731.log 2024-09-21T03:45:39.858 INFO:teuthology.orchestra.run.smithi038.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.83223.log.gz 2024-09-21T03:45:39.858 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.86441.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55799.log 2024-09-21T03:45:39.858 INFO:teuthology.orchestra.run.smithi038.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.86441.log.gz 2024-09-21T03:45:39.858 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.90731.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66024.log 2024-09-21T03:45:39.858 INFO:teuthology.orchestra.run.smithi038.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.90731.log.gz 2024-09-21T03:45:39.858 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66098.log 2024-09-21T03:45:39.858 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.55799.log: /var/log/ceph/ceph-client.admin.66024.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.55799.log.gz 2024-09-21T03:45:39.858 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48000.log 2024-09-21T03:45:39.858 INFO:teuthology.orchestra.run.smithi038.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.66024.log.gz 2024-09-21T03:45:39.859 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.66098.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44967.log 2024-09-21T03:45:39.859 INFO:teuthology.orchestra.run.smithi038.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.66098.log.gz 2024-09-21T03:45:39.859 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56524.log 2024-09-21T03:45:39.859 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.48000.log: /var/log/ceph/ceph-client.admin.44967.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.48000.log.gz 2024-09-21T03:45:39.859 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-09-21T03:45:39.859 INFO:teuthology.orchestra.run.smithi038.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.44967.log.gz 2024-09-21T03:45:39.859 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.56524.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50115.log 2024-09-21T03:45:39.859 INFO:teuthology.orchestra.run.smithi038.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.56524.log.gz 2024-09-21T03:45:39.859 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44576.log 2024-09-21T03:45:39.859 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.50115.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73509.log 2024-09-21T03:45:39.859 INFO:teuthology.orchestra.run.smithi038.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.50115.log.gz 2024-09-21T03:45:39.859 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.44576.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57887.log 2024-09-21T03:45:39.859 INFO:teuthology.orchestra.run.smithi038.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.44576.log.gz/var/log/ceph/ceph-client.admin.73509.log: 2024-09-21T03:45:39.860 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52868.log 2024-09-21T03:45:39.860 INFO:teuthology.orchestra.run.smithi038.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.73509.log.gz 2024-09-21T03:45:39.860 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.57887.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64951.log 2024-09-21T03:45:39.860 INFO:teuthology.orchestra.run.smithi038.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.57887.log.gz 2024-09-21T03:45:39.860 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.52868.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78467.log 2024-09-21T03:45:39.860 INFO:teuthology.orchestra.run.smithi038.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.52868.log.gz 2024-09-21T03:45:39.860 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.64951.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68207.log 2024-09-21T03:45:39.860 INFO:teuthology.orchestra.run.smithi038.stderr:gzip 89.1% -5 -- replaced with /var/log/ceph/ceph-client.admin.64951.log.gz --verbose 2024-09-21T03:45:39.860 INFO:teuthology.orchestra.run.smithi038.stderr: -- /var/log/ceph/ceph.tmp-client.admin.30141.log 2024-09-21T03:45:39.860 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.78467.log: /var/log/ceph/ceph-client.admin.68207.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.78467.log.gz 2024-09-21T03:45:39.860 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76718.log 2024-09-21T03:45:39.860 INFO:teuthology.orchestra.run.smithi038.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.68207.log.gz 2024-09-21T03:45:39.860 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54810.log 2024-09-21T03:45:39.860 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph.tmp-client.admin.30141.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.30141.log.gz 2024-09-21T03:45:39.861 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.76718.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55452.log 2024-09-21T03:45:39.861 INFO:teuthology.orchestra.run.smithi038.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.76718.log.gz 2024-09-21T03:45:39.861 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.54810.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80293.log 2024-09-21T03:45:39.861 INFO:teuthology.orchestra.run.smithi038.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.54810.log.gz 2024-09-21T03:45:39.861 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.55452.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76278.log 2024-09-21T03:45:39.861 INFO:teuthology.orchestra.run.smithi038.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.55452.log.gz 2024-09-21T03:45:39.861 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.80293.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66283.log 2024-09-21T03:45:39.861 INFO:teuthology.orchestra.run.smithi038.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.80293.log.gz 2024-09-21T03:45:39.861 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68891.log 2024-09-21T03:45:39.861 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.76278.log: /var/log/ceph/ceph-client.admin.66283.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36790.log 2024-09-21T03:45:39.861 INFO:teuthology.orchestra.run.smithi038.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.76278.log.gz 2024-09-21T03:45:39.861 INFO:teuthology.orchestra.run.smithi038.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.66283.log.gz 2024-09-21T03:45:39.861 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.68891.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.61426.log 2024-09-21T03:45:39.862 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56268.log 2024-09-21T03:45:39.862 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.36790.log: /var/log/ceph/ceph-client.0.61426.log: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.36790.log.gz 2024-09-21T03:45:39.862 INFO:teuthology.orchestra.run.smithi038.stderr: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.0.61426.log.gz -5 2024-09-21T03:45:39.862 INFO:teuthology.orchestra.run.smithi038.stderr: --verbose -- /var/log/ceph/ceph-client.admin.72887.log 2024-09-21T03:45:39.862 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.56268.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86670.log 2024-09-21T03:45:39.862 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.72887.log: 87.5% 92.5% -- replaced with /var/log/ceph/ceph-client.admin.56268.log.gz 2024-09-21T03:45:39.862 INFO:teuthology.orchestra.run.smithi038.stderr: -- replaced with /var/log/ceph/ceph-client.admin.68891.log.gz 2024-09-21T03:45:39.862 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81747.log 2024-09-21T03:45:39.862 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.86670.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56440.log 2024-09-21T03:45:39.862 INFO:teuthology.orchestra.run.smithi038.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.86670.log.gz 2024-09-21T03:45:39.862 INFO:teuthology.orchestra.run.smithi038.stderr:gzip/var/log/ceph/ceph-client.admin.81747.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.68114.log 2024-09-21T03:45:39.862 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.56440.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.81747.log.gz 2024-09-21T03:45:39.863 INFO:teuthology.orchestra.run.smithi038.stderr: 92.6% -- replaced with /var/log/ceph/ceph-client.admin.72887.log.gz 2024-09-21T03:45:39.863 INFO:teuthology.orchestra.run.smithi038.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.56440.log.gz 2024-09-21T03:45:39.863 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77226.log 2024-09-21T03:45:39.863 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.68114.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36244.log 2024-09-21T03:45:39.863 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.77226.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76120.log 2024-09-21T03:45:39.863 INFO:teuthology.orchestra.run.smithi038.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.77226.log.gz 2024-09-21T03:45:39.863 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.36244.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.82734.log 2024-09-21T03:45:39.863 INFO:teuthology.orchestra.run.smithi038.stderr: 76.9% -- replaced with /var/log/ceph/ceph-client.admin.36244.log.gz 2024-09-21T03:45:39.863 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.76120.log: 92.6% -- replaced with /var/log/ceph/ceph-client.admin.68114.log.gz 2024-09-21T03:45:39.863 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45283.log 2024-09-21T03:45:39.863 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.0.82734.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.76120.log.gz 2024-09-21T03:45:39.863 INFO:teuthology.orchestra.run.smithi038.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.82734.log.gz 2024-09-21T03:45:39.863 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79386.log 2024-09-21T03:45:39.864 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.45283.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67130.log 2024-09-21T03:45:39.864 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.79386.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.45283.log.gz 2024-09-21T03:45:39.864 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65469.log 2024-09-21T03:45:39.864 INFO:teuthology.orchestra.run.smithi038.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.79386.log.gz 2024-09-21T03:45:39.864 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63179.log 2024-09-21T03:45:39.864 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.67130.log: gzip -5/var/log/ceph/ceph-client.admin.65469.log: --verbose -- /var/log/ceph/ceph-client.admin.57612.log 2024-09-21T03:45:39.864 INFO:teuthology.orchestra.run.smithi038.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.67130.log.gz 2024-09-21T03:45:39.864 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.63179.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.65469.log.gz 2024-09-21T03:45:39.864 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84363.log 2024-09-21T03:45:39.864 INFO:teuthology.orchestra.run.smithi038.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.63179.log.gz 2024-09-21T03:45:39.864 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85537.log 2024-09-21T03:45:39.864 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.57612.log: /var/log/ceph/ceph-client.admin.84363.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.57612.log.gz 2024-09-21T03:45:39.864 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89425.log 2024-09-21T03:45:39.865 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.85537.log: /var/log/ceph/ceph-client.admin.68539.log 2024-09-21T03:45:39.865 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.89425.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.85537.log.gz 2024-09-21T03:45:39.865 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81530.log 2024-09-21T03:45:39.865 INFO:teuthology.orchestra.run.smithi038.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.89425.log.gz 2024-09-21T03:45:39.865 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.68539.log: 92.6% -- replaced with /var/log/ceph/ceph-client.admin.84363.log.gz 2024-09-21T03:45:39.865 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71044.log 2024-09-21T03:45:39.865 INFO:teuthology.orchestra.run.smithi038.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.68539.log.gz 2024-09-21T03:45:39.865 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.81530.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52170.log 2024-09-21T03:45:39.865 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.71044.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61610.log 2024-09-21T03:45:39.865 INFO:teuthology.orchestra.run.smithi038.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.71044.log.gz 2024-09-21T03:45:39.865 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.52170.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.81530.log.gz 2024-09-21T03:45:39.865 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53429.log 2024-09-21T03:45:39.865 INFO:teuthology.orchestra.run.smithi038.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.52170.log.gz 2024-09-21T03:45:39.866 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42781.log 2024-09-21T03:45:39.866 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.61610.log: /var/log/ceph/ceph-client.admin.53429.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91876.log 2024-09-21T03:45:39.866 INFO:teuthology.orchestra.run.smithi038.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.53429.log.gz 2024-09-21T03:45:39.866 INFO:teuthology.orchestra.run.smithi038.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.61610.log.gz 2024-09-21T03:45:39.866 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.42781.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80681.log 2024-09-21T03:45:39.866 INFO:teuthology.orchestra.run.smithi038.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.42781.log.gz 2024-09-21T03:45:39.866 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58083.log 2024-09-21T03:45:39.866 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.91876.log: /var/log/ceph/ceph-client.admin.80681.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86060.log 2024-09-21T03:45:39.866 INFO:teuthology.orchestra.run.smithi038.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.91876.log.gz 2024-09-21T03:45:39.866 INFO:teuthology.orchestra.run.smithi038.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.80681.log.gz 2024-09-21T03:45:39.866 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.58083.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81093.log 2024-09-21T03:45:39.866 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.82730.log 2024-09-21T03:45:39.866 INFO:teuthology.orchestra.run.smithi038.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.58083.log.gz 2024-09-21T03:45:39.867 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.86060.log: /var/log/ceph/ceph-client.admin.81093.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.86060.log.gz 2024-09-21T03:45:39.867 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63519.log 2024-09-21T03:45:39.867 INFO:teuthology.orchestra.run.smithi038.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.81093.log.gz 2024-09-21T03:45:39.867 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60177.log 2024-09-21T03:45:39.867 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.0.82730.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.82730.log.gz 2024-09-21T03:45:39.867 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.63519.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49159.log 2024-09-21T03:45:39.867 INFO:teuthology.orchestra.run.smithi038.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.63519.log.gz 2024-09-21T03:45:39.867 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.60177.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46148.log 2024-09-21T03:45:39.867 INFO:teuthology.orchestra.run.smithi038.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.60177.log.gz 2024-09-21T03:45:39.867 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.49159.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66320.log 2024-09-21T03:45:39.867 INFO:teuthology.orchestra.run.smithi038.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.49159.log.gz 2024-09-21T03:45:39.867 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71657.log 2024-09-21T03:45:39.867 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.46148.log: /var/log/ceph/ceph-client.admin.66320.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76414.log 2024-09-21T03:45:39.868 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.71657.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.66320.log.gz 2024-09-21T03:45:39.868 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48336.log 2024-09-21T03:45:39.868 INFO:teuthology.orchestra.run.smithi038.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.46148.log.gz 2024-09-21T03:45:39.868 INFO:teuthology.orchestra.run.smithi038.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.71657.log.gz 2024-09-21T03:45:39.868 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65247.log 2024-09-21T03:45:39.868 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.76414.log: /var/log/ceph/ceph-client.admin.48336.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54322.log 2024-09-21T03:45:39.868 INFO:teuthology.orchestra.run.smithi038.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.76414.log.gz 2024-09-21T03:45:39.868 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.65247.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.48336.log.gz 2024-09-21T03:45:39.868 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44366.log 2024-09-21T03:45:39.868 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.54322.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.65247.log.gz 2024-09-21T03:45:39.868 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84204.log 2024-09-21T03:45:39.868 INFO:teuthology.orchestra.run.smithi038.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.54322.log.gz 2024-09-21T03:45:39.868 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85672.log 2024-09-21T03:45:39.869 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.44366.log: gzip/var/log/ceph/ceph-client.admin.84204.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.61840.log 2024-09-21T03:45:39.869 INFO:teuthology.orchestra.run.smithi038.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.44366.log.gz 2024-09-21T03:45:39.869 INFO:teuthology.orchestra.run.smithi038.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.84204.log.gz 2024-09-21T03:45:39.869 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.85672.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74911.log 2024-09-21T03:45:39.869 INFO:teuthology.orchestra.run.smithi038.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.85672.log.gz 2024-09-21T03:45:39.869 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72299.log 2024-09-21T03:45:39.869 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.61840.log: /var/log/ceph/ceph-client.admin.74911.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76550.log 2024-09-21T03:45:39.869 INFO:teuthology.orchestra.run.smithi038.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.61840.log.gz 2024-09-21T03:45:39.869 INFO:teuthology.orchestra.run.smithi038.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.74911.log.gz 2024-09-21T03:45:39.869 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.72299.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88816.log 2024-09-21T03:45:39.869 INFO:teuthology.orchestra.run.smithi038.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.72299.log.gz 2024-09-21T03:45:39.869 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70525.log 2024-09-21T03:45:39.869 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.76550.log: /var/log/ceph/ceph-client.admin.88816.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56044.log 2024-09-21T03:45:39.870 INFO:teuthology.orchestra.run.smithi038.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.76550.log.gz 2024-09-21T03:45:39.870 INFO:teuthology.orchestra.run.smithi038.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.88816.log.gz 2024-09-21T03:45:39.870 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.70525.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59703.log 2024-09-21T03:45:39.870 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79718.log 2024-09-21T03:45:39.870 INFO:teuthology.orchestra.run.smithi038.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.70525.log.gz 2024-09-21T03:45:39.870 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.56044.log: /var/log/ceph/ceph-client.admin.59703.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.56044.log.gz 2024-09-21T03:45:39.870 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78532.log 2024-09-21T03:45:39.870 INFO:teuthology.orchestra.run.smithi038.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.59703.log.gz 2024-09-21T03:45:39.870 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71930.log 2024-09-21T03:45:39.870 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.79718.log: /var/log/ceph/ceph-client.admin.78532.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86262.log 2024-09-21T03:45:39.870 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.71930.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65728.log 2024-09-21T03:45:39.870 INFO:teuthology.orchestra.run.smithi038.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.71930.log.gz 2024-09-21T03:45:39.870 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.86262.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48936.log 2024-09-21T03:45:39.871 INFO:teuthology.orchestra.run.smithi038.stderr: 87.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45652.log 2024-09-21T03:45:39.871 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.65728.log: -- replaced with /var/log/ceph/ceph-client.admin.86262.log.gz 2024-09-21T03:45:39.871 INFO:teuthology.orchestra.run.smithi038.stderr: 92.5% -- replaced with /var/log/ceph/ceph-client.admin.79718.log.gz 2024-09-21T03:45:39.871 INFO:teuthology.orchestra.run.smithi038.stderr: 92.5% -- replaced with /var/log/ceph/ceph-client.admin.78532.log.gz 2024-09-21T03:45:39.871 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.48936.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90292.log 2024-09-21T03:45:39.871 INFO:teuthology.orchestra.run.smithi038.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.65728.log.gz 2024-09-21T03:45:39.871 INFO:teuthology.orchestra.run.smithi038.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.48936.log.gz 2024-09-21T03:45:39.871 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.45652.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73803.log 2024-09-21T03:45:39.871 INFO:teuthology.orchestra.run.smithi038.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45652.log.gz 2024-09-21T03:45:39.871 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.90292.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86003.log 2024-09-21T03:45:39.871 INFO:teuthology.orchestra.run.smithi038.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.90292.log.gz 2024-09-21T03:45:39.871 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.73803.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70581.log 2024-09-21T03:45:39.871 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.86003.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.73803.log.gz 2024-09-21T03:45:39.871 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63095.log 2024-09-21T03:45:39.872 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.70581.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.86003.log.gz 2024-09-21T03:45:39.872 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50254.log 2024-09-21T03:45:39.872 INFO:teuthology.orchestra.run.smithi038.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.70581.log.gz/var/log/ceph/ceph-client.admin.63095.log: 2024-09-21T03:45:39.872 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68479.log 2024-09-21T03:45:39.872 INFO:teuthology.orchestra.run.smithi038.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.63095.log.gz 2024-09-21T03:45:39.872 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.50254.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69606.log 2024-09-21T03:45:39.872 INFO:teuthology.orchestra.run.smithi038.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.50254.log.gz 2024-09-21T03:45:39.872 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.68479.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78078.log 2024-09-21T03:45:39.872 INFO:teuthology.orchestra.run.smithi038.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.68479.log.gz 2024-09-21T03:45:39.872 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74014.log 2024-09-21T03:45:39.872 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.69606.log: /var/log/ceph/ceph-client.admin.78078.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76838.log 2024-09-21T03:45:39.872 INFO:teuthology.orchestra.run.smithi038.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.69606.log.gz 2024-09-21T03:45:39.872 INFO:teuthology.orchestra.run.smithi038.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.78078.log.gz 2024-09-21T03:45:39.873 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.74014.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90689.log 2024-09-21T03:45:39.873 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74420.log 2024-09-21T03:45:39.873 INFO:teuthology.orchestra.run.smithi038.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.74014.log.gz 2024-09-21T03:45:39.873 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.76838.log: /var/log/ceph/ceph-client.admin.90689.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.76838.log.gz 2024-09-21T03:45:39.873 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68984.log 2024-09-21T03:45:39.873 INFO:teuthology.orchestra.run.smithi038.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.90689.log.gz 2024-09-21T03:45:39.873 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76778.log 2024-09-21T03:45:39.873 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.74420.log: /var/log/ceph/ceph-client.admin.68984.log: gzip 87.2% -5 -- replaced with /var/log/ceph/ceph-client.admin.74420.log.gz 2024-09-21T03:45:39.873 INFO:teuthology.orchestra.run.smithi038.stderr: --verbose -- /var/log/ceph/ceph-client.admin.52950.log 2024-09-21T03:45:39.873 INFO:teuthology.orchestra.run.smithi038.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.68984.log.gz 2024-09-21T03:45:39.873 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.76778.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64803.log 2024-09-21T03:45:39.873 INFO:teuthology.orchestra.run.smithi038.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.76778.log.gz 2024-09-21T03:45:39.873 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67438.log 2024-09-21T03:45:39.873 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.52950.log: /var/log/ceph/ceph-client.admin.64803.log: gzip 87.0% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.52950.log.gz -- 2024-09-21T03:45:39.874 INFO:teuthology.orchestra.run.smithi038.stderr: /var/log/ceph/ceph-client.admin.80429.log 2024-09-21T03:45:39.874 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.67438.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64144.log 2024-09-21T03:45:39.874 INFO:teuthology.orchestra.run.smithi038.stderr: 91.8% -- replaced with /var/log/ceph/ceph-client.admin.64803.log.gz 2024-09-21T03:45:39.874 INFO:teuthology.orchestra.run.smithi038.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.67438.log.gz 2024-09-21T03:45:39.874 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47266.log 2024-09-21T03:45:39.874 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.80429.log: /var/log/ceph/ceph-client.admin.64144.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76508.log 2024-09-21T03:45:39.874 INFO:teuthology.orchestra.run.smithi038.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.80429.log.gz 2024-09-21T03:45:39.874 INFO:teuthology.orchestra.run.smithi038.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.64144.log.gz 2024-09-21T03:45:39.874 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.47266.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64187.log 2024-09-21T03:45:39.874 INFO:teuthology.orchestra.run.smithi038.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.47266.log.gz 2024-09-21T03:45:39.874 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.76508.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.49933.log 2024-09-21T03:45:39.875 INFO:teuthology.orchestra.run.smithi038.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.76508.log.gz 2024-09-21T03:45:39.875 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.57052.log 2024-09-21T03:45:39.875 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.64187.log: /var/log/ceph/ceph-client.0.49933.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.49933.log.gz 2024-09-21T03:45:39.875 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66061.log 2024-09-21T03:45:39.875 INFO:teuthology.orchestra.run.smithi038.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.64187.log.gz 2024-09-21T03:45:39.875 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.57496.log 2024-09-21T03:45:39.875 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.0.57052.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.57052.log.gz 2024-09-21T03:45:39.875 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.66061.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84003.log 2024-09-21T03:45:39.875 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.0.57496.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.66061.log.gz 2024-09-21T03:45:39.875 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.79039.log 2024-09-21T03:45:39.875 INFO:teuthology.orchestra.run.smithi038.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.57496.log.gz 2024-09-21T03:45:39.875 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77404.log 2024-09-21T03:45:39.875 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.84003.log: /var/log/ceph/ceph-client.0.79039.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.84003.log.gz 2024-09-21T03:45:39.876 INFO:teuthology.orchestra.run.smithi038.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.79039.log.gz 2024-09-21T03:45:39.876 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.46638.log 2024-09-21T03:45:39.876 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36384.log 2024-09-21T03:45:39.876 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.77404.log: /var/log/ceph/ceph-client.0.46638.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83322.log 2024-09-21T03:45:39.876 INFO:teuthology.orchestra.run.smithi038.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.46638.log.gz 2024-09-21T03:45:39.876 INFO:teuthology.orchestra.run.smithi038.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.77404.log.gz 2024-09-21T03:45:39.876 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.36384.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62927.log 2024-09-21T03:45:39.876 INFO:teuthology.orchestra.run.smithi038.stderr: 77.5% -- replaced with /var/log/ceph/ceph-client.admin.36384.log.gz 2024-09-21T03:45:39.876 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.83322.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61797.log 2024-09-21T03:45:39.876 INFO:teuthology.orchestra.run.smithi038.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.83322.log.gz 2024-09-21T03:45:39.876 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.62927.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63713.log 2024-09-21T03:45:39.876 INFO:teuthology.orchestra.run.smithi038.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.62927.log.gz 2024-09-21T03:45:39.876 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.61797.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77302.log 2024-09-21T03:45:39.877 INFO:teuthology.orchestra.run.smithi038.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.61797.log.gz 2024-09-21T03:45:39.877 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.63713.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.88396.log 2024-09-21T03:45:39.877 INFO:teuthology.orchestra.run.smithi038.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.63713.log.gz 2024-09-21T03:45:39.877 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.77302.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42905.log 2024-09-21T03:45:39.877 INFO:teuthology.orchestra.run.smithi038.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.77302.log.gz 2024-09-21T03:45:39.877 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.0.88396.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84162.log 2024-09-21T03:45:39.877 INFO:teuthology.orchestra.run.smithi038.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.88396.log.gz 2024-09-21T03:45:39.877 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.42905.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75816.log 2024-09-21T03:45:39.877 INFO:teuthology.orchestra.run.smithi038.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42905.log.gz 2024-09-21T03:45:39.877 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.84162.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49511.log 2024-09-21T03:45:39.877 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 87.6% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.84162.log.gz /var/log/ceph/ceph-client.admin.64840.log 2024-09-21T03:45:39.877 INFO:teuthology.orchestra.run.smithi038.stderr: 2024-09-21T03:45:39.877 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.75816.log: /var/log/ceph/ceph-client.admin.49511.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75909.log 2024-09-21T03:45:39.878 INFO:teuthology.orchestra.run.smithi038.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.49511.log.gz 2024-09-21T03:45:39.878 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.64840.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65358.log 2024-09-21T03:45:39.878 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52357.log 2024-09-21T03:45:39.878 INFO:teuthology.orchestra.run.smithi038.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.64840.log.gz 2024-09-21T03:45:39.878 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.75909.log: /var/log/ceph/ceph-client.admin.65358.log: 92.5% -- replaced with /var/log/ceph/ceph-client.admin.75816.log.gz 2024-09-21T03:45:39.878 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73686.log 2024-09-21T03:45:39.878 INFO:teuthology.orchestra.run.smithi038.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.75909.log.gz 2024-09-21T03:45:39.878 INFO:teuthology.orchestra.run.smithi038.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.65358.log.gz 2024-09-21T03:45:39.878 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5/var/log/ceph/ceph-client.admin.52357.log: --verbose -- /var/log/ceph/ceph-client.admin.56226.log 2024-09-21T03:45:39.878 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.73686.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.52357.log.gz 2024-09-21T03:45:39.878 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77512.log 2024-09-21T03:45:39.878 INFO:teuthology.orchestra.run.smithi038.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.73686.log.gz 2024-09-21T03:45:39.878 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.56226.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69900.log 2024-09-21T03:45:39.879 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.77512.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.56226.log.gz 2024-09-21T03:45:39.879 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43077.log 2024-09-21T03:45:39.879 INFO:teuthology.orchestra.run.smithi038.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.77512.log.gz 2024-09-21T03:45:39.879 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.69900.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58251.log 2024-09-21T03:45:39.879 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.43077.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.69900.log.gz 2024-09-21T03:45:39.879 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43077.log.gz 2024-09-21T03:45:39.879 INFO:teuthology.orchestra.run.smithi038.stderr: /var/log/ceph/ceph-client.admin.77344.log 2024-09-21T03:45:39.879 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.58251.log: 87.3%gzip -- replaced with /var/log/ceph/ceph-client.admin.58251.log.gz 2024-09-21T03:45:39.879 INFO:teuthology.orchestra.run.smithi038.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.68091.log 2024-09-21T03:45:39.879 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.77344.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65506.log 2024-09-21T03:45:39.879 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.68091.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.77344.log.gz 2024-09-21T03:45:39.879 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83826.log 2024-09-21T03:45:39.879 INFO:teuthology.orchestra.run.smithi038.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.68091.log.gz 2024-09-21T03:45:39.879 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.65506.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69942.log 2024-09-21T03:45:39.880 INFO:teuthology.orchestra.run.smithi038.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.65506.log.gz 2024-09-21T03:45:39.880 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.83826.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48894.log 2024-09-21T03:45:39.880 INFO:teuthology.orchestra.run.smithi038.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.83826.log.gz 2024-09-21T03:45:39.880 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.69942.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82969.log 2024-09-21T03:45:39.880 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87048.log 2024-09-21T03:45:39.880 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.48894.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.69942.log.gz 2024-09-21T03:45:39.880 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.82969.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36515.log 2024-09-21T03:45:39.880 INFO:teuthology.orchestra.run.smithi038.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48894.log.gz 2024-09-21T03:45:39.880 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.87048.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.82969.log.gz 2024-09-21T03:45:39.880 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88924.log 2024-09-21T03:45:39.880 INFO:teuthology.orchestra.run.smithi038.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.87048.log.gz 2024-09-21T03:45:39.880 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.36515.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51891.log 2024-09-21T03:45:39.880 INFO:teuthology.orchestra.run.smithi038.stderr: 85.7% -- replaced with /var/log/ceph/ceph-client.admin.36515.log.gz 2024-09-21T03:45:39.881 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.88924.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.2.47091.log 2024-09-21T03:45:39.881 INFO:teuthology.orchestra.run.smithi038.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.88924.log.gz 2024-09-21T03:45:39.881 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.51891.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67502.log 2024-09-21T03:45:39.881 INFO:teuthology.orchestra.run.smithi038.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.51891.log.gz 2024-09-21T03:45:39.881 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.2.47091.log: 0.0% -- replaced with /var/log/ceph/ceph-client.2.47091.log.gz 2024-09-21T03:45:39.881 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59451.log 2024-09-21T03:45:39.881 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.67502.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87388.log 2024-09-21T03:45:39.881 INFO:teuthology.orchestra.run.smithi038.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.67502.log.gz 2024-09-21T03:45:39.881 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.59451.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52525.log 2024-09-21T03:45:39.881 INFO:teuthology.orchestra.run.smithi038.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.59451.log.gz 2024-09-21T03:45:39.881 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87477.log 2024-09-21T03:45:39.881 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.87388.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.87388.log.gz 2024-09-21T03:45:39.881 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.52525.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36644.log 2024-09-21T03:45:39.882 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62497.log 2024-09-21T03:45:39.882 INFO:teuthology.orchestra.run.smithi038.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.52525.log.gz 2024-09-21T03:45:39.882 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.87477.log: /var/log/ceph/ceph-client.admin.36644.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.87477.log.gz 2024-09-21T03:45:39.882 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66488.log 2024-09-21T03:45:39.882 INFO:teuthology.orchestra.run.smithi038.stderr: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.36644.log.gz 2024-09-21T03:45:39.882 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51239.log 2024-09-21T03:45:39.882 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.62497.log: /var/log/ceph/ceph-client.admin.66488.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.62497.log.gzgzip 2024-09-21T03:45:39.882 INFO:teuthology.orchestra.run.smithi038.stderr: -5 --verbose -- /var/log/ceph/ceph-client.0.72595.log 2024-09-21T03:45:39.882 INFO:teuthology.orchestra.run.smithi038.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.66488.log.gz 2024-09-21T03:45:39.882 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.51239.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45980.log 2024-09-21T03:45:39.882 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65432.log 2024-09-21T03:45:39.883 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.0.72595.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.72595.log.gz 2024-09-21T03:45:39.883 INFO:teuthology.orchestra.run.smithi038.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.51239.log.gz 2024-09-21T03:45:39.883 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.45980.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60920.log 2024-09-21T03:45:39.883 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.65432.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.45980.log.gz 2024-09-21T03:45:39.883 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71391.log 2024-09-21T03:45:39.883 INFO:teuthology.orchestra.run.smithi038.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.65432.log.gz 2024-09-21T03:45:39.883 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.60920.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80083.log 2024-09-21T03:45:39.883 INFO:teuthology.orchestra.run.smithi038.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.60920.log.gz 2024-09-21T03:45:39.883 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.71391.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80199.log 2024-09-21T03:45:39.883 INFO:teuthology.orchestra.run.smithi038.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.71391.log.gz 2024-09-21T03:45:39.883 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.80083.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57242.log 2024-09-21T03:45:39.883 INFO:teuthology.orchestra.run.smithi038.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.80083.log.gz 2024-09-21T03:45:39.883 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.80199.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70446.log 2024-09-21T03:45:39.884 INFO:teuthology.orchestra.run.smithi038.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.80199.log.gz 2024-09-21T03:45:39.884 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.57242.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74874.log 2024-09-21T03:45:39.884 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.70446.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55307.log 2024-09-21T03:45:39.884 INFO:teuthology.orchestra.run.smithi038.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.57242.log.gz 2024-09-21T03:45:39.884 INFO:teuthology.orchestra.run.smithi038.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.70446.log.gz 2024-09-21T03:45:39.884 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36595.log 2024-09-21T03:45:39.884 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.74874.log: /var/log/ceph/ceph-client.admin.55307.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91008.log 2024-09-21T03:45:39.884 INFO:teuthology.orchestra.run.smithi038.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.55307.log.gz 2024-09-21T03:45:39.884 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.57048.log 2024-09-21T03:45:39.884 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.36595.log: /var/log/ceph/ceph-client.admin.91008.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.36595.log.gz 2024-09-21T03:45:39.884 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58395.log 2024-09-21T03:45:39.884 INFO:teuthology.orchestra.run.smithi038.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.91008.log.gz 2024-09-21T03:45:39.884 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.0.57048.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55960.log 2024-09-21T03:45:39.885 INFO:teuthology.orchestra.run.smithi038.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.57048.log.gz 2024-09-21T03:45:39.885 INFO:teuthology.orchestra.run.smithi038.stderr: 92.5% -- replaced with /var/log/ceph/ceph-client.admin.74874.log.gz 2024-09-21T03:45:39.885 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.58395.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80335.log 2024-09-21T03:45:39.885 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.55960.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.58395.log.gz 2024-09-21T03:45:39.885 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48664.log 2024-09-21T03:45:39.885 INFO:teuthology.orchestra.run.smithi038.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.55960.log.gz 2024-09-21T03:45:39.885 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.80335.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83276.log 2024-09-21T03:45:39.885 INFO:teuthology.orchestra.run.smithi038.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.80335.log.gz 2024-09-21T03:45:39.885 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.48664.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66657.log 2024-09-21T03:45:39.885 INFO:teuthology.orchestra.run.smithi038.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48664.log.gz 2024-09-21T03:45:39.885 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.83276.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63053.log 2024-09-21T03:45:39.885 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- 87.2% /var/log/ceph/ceph-client.admin.57975.log 2024-09-21T03:45:39.885 INFO:teuthology.orchestra.run.smithi038.stderr: -- replaced with /var/log/ceph/ceph-client.admin.83276.log.gz 2024-09-21T03:45:39.885 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.66657.log: /var/log/ceph/ceph-client.admin.63053.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.66657.log.gz 2024-09-21T03:45:39.886 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.54023.log 2024-09-21T03:45:39.886 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85336.log 2024-09-21T03:45:39.886 INFO:teuthology.orchestra.run.smithi038.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.63053.log.gz 2024-09-21T03:45:39.886 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.57975.log: /var/log/ceph/ceph-client.0.54023.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82829.log 2024-09-21T03:45:39.886 INFO:teuthology.orchestra.run.smithi038.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.54023.log.gz 2024-09-21T03:45:39.886 INFO:teuthology.orchestra.run.smithi038.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.57975.log.gz 2024-09-21T03:45:39.886 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.85336.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65654.log 2024-09-21T03:45:39.886 INFO:teuthology.orchestra.run.smithi038.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.85336.log.gz 2024-09-21T03:45:39.886 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.82829.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70675.log 2024-09-21T03:45:39.886 INFO:teuthology.orchestra.run.smithi038.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.82829.log.gz 2024-09-21T03:45:39.886 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.65654.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57731.log 2024-09-21T03:45:39.886 INFO:teuthology.orchestra.run.smithi038.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.65654.log.gz 2024-09-21T03:45:39.887 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.70675.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47958.log 2024-09-21T03:45:39.887 INFO:teuthology.orchestra.run.smithi038.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.70675.log.gz 2024-09-21T03:45:39.887 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.57731.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83046.log 2024-09-21T03:45:39.887 INFO:teuthology.orchestra.run.smithi038.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.57731.log.gz 2024-09-21T03:45:39.887 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.47958.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64701.log 2024-09-21T03:45:39.887 INFO:teuthology.orchestra.run.smithi038.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.47958.log.gz 2024-09-21T03:45:39.887 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.83046.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49075.log 2024-09-21T03:45:39.887 INFO:teuthology.orchestra.run.smithi038.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.83046.log.gz 2024-09-21T03:45:39.887 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66511.log 2024-09-21T03:45:39.887 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.64701.log: /var/log/ceph/ceph-client.admin.49075.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-21T03:45:39.887 INFO:teuthology.orchestra.run.smithi038.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.64701.log.gz 2024-09-21T03:45:39.887 INFO:teuthology.orchestra.run.smithi038.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49075.log.gz 2024-09-21T03:45:39.888 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.66511.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91626.log 2024-09-21T03:45:39.888 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87910.log 2024-09-21T03:45:39.888 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph.audit.log: /var/log/ceph/ceph-client.admin.91626.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74336.log 2024-09-21T03:45:39.888 INFO:teuthology.orchestra.run.smithi038.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.91626.log.gz 2024-09-21T03:45:39.888 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.87910.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.50793.log 2024-09-21T03:45:39.888 INFO:teuthology.orchestra.run.smithi038.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.87910.log.gz 2024-09-21T03:45:39.888 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.74336.log: 92.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73402.log 2024-09-21T03:45:39.888 INFO:teuthology.orchestra.run.smithi038.stderr: -- replaced with /var/log/ceph/ceph-client.admin.66511.log.gz 2024-09-21T03:45:39.888 INFO:teuthology.orchestra.run.smithi038.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.74336.log.gz 2024-09-21T03:45:39.888 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.0.50793.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42863.log 2024-09-21T03:45:39.888 INFO:teuthology.orchestra.run.smithi038.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.50793.log.gz 2024-09-21T03:45:39.888 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.73402.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80041.log 2024-09-21T03:45:39.888 INFO:teuthology.orchestra.run.smithi038.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.73402.log.gz 2024-09-21T03:45:39.889 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.42863.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62843.log 2024-09-21T03:45:39.889 INFO:teuthology.orchestra.run.smithi038.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.42863.log.gz 2024-09-21T03:45:39.889 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.80041.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82542.log 2024-09-21T03:45:39.889 INFO:teuthology.orchestra.run.smithi038.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.80041.log.gz 2024-09-21T03:45:39.889 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.62843.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89108.log 2024-09-21T03:45:39.889 INFO:teuthology.orchestra.run.smithi038.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.62843.log.gz 2024-09-21T03:45:39.889 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.82542.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75020.log 2024-09-21T03:45:39.889 INFO:teuthology.orchestra.run.smithi038.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.82542.log.gz 2024-09-21T03:45:39.889 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.89108.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79653.log 2024-09-21T03:45:39.889 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.75020.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91331.log 2024-09-21T03:45:39.889 INFO:teuthology.orchestra.run.smithi038.stderr: 89.7% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-21T03:45:39.889 INFO:teuthology.orchestra.run.smithi038.stderr: 87.3%/var/log/ceph/ceph-client.admin.79653.log: -- replaced with /var/log/ceph/ceph-client.admin.75020.log.gz 2024-09-21T03:45:39.889 INFO:teuthology.orchestra.run.smithi038.stderr:gzip 90.9% -5 -- replaced with /var/log/ceph/ceph-client.admin.89108.log.gz 2024-09-21T03:45:39.889 INFO:teuthology.orchestra.run.smithi038.stderr: --verbose -- /var/log/ceph/ceph-client.admin.59956.log 2024-09-21T03:45:39.890 INFO:teuthology.orchestra.run.smithi038.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.79653.log.gz 2024-09-21T03:45:39.890 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84279.log 2024-09-21T03:45:39.890 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.91331.log: /var/log/ceph/ceph-client.admin.59956.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60040.log 2024-09-21T03:45:39.890 INFO:teuthology.orchestra.run.smithi038.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.91331.log.gz 2024-09-21T03:45:39.890 INFO:teuthology.orchestra.run.smithi038.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.59956.log.gz/var/log/ceph/ceph-client.admin.84279.log: 2024-09-21T03:45:39.890 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87826.log 2024-09-21T03:45:39.890 INFO:teuthology.orchestra.run.smithi038.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.84279.log.gz 2024-09-21T03:45:39.890 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.60040.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69783.log 2024-09-21T03:45:39.890 INFO:teuthology.orchestra.run.smithi038.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.60040.log.gz 2024-09-21T03:45:39.890 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73897.log 2024-09-21T03:45:39.890 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.87826.log: /var/log/ceph/ceph-client.admin.69783.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56002.log 2024-09-21T03:45:39.890 INFO:teuthology.orchestra.run.smithi038.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.87826.log.gz 2024-09-21T03:45:39.890 INFO:teuthology.orchestra.run.smithi038.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.69783.log.gz 2024-09-21T03:45:39.891 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.73897.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62251.log 2024-09-21T03:45:39.891 INFO:teuthology.orchestra.run.smithi038.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.73897.log.gz 2024-09-21T03:45:39.891 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48126.log 2024-09-21T03:45:39.891 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.56002.log: /var/log/ceph/ceph-client.admin.62251.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70327.log 2024-09-21T03:45:39.891 INFO:teuthology.orchestra.run.smithi038.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.56002.log.gz 2024-09-21T03:45:39.891 INFO:teuthology.orchestra.run.smithi038.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.62251.log.gz 2024-09-21T03:45:39.891 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.48126.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61755.log 2024-09-21T03:45:39.891 INFO:teuthology.orchestra.run.smithi038.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.48126.log.gz 2024-09-21T03:45:39.891 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52254.log 2024-09-21T03:45:39.891 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.70327.log: /var/log/ceph/ceph-client.admin.61755.log: 87.5%gzip -- replaced with /var/log/ceph/ceph-client.admin.70327.log.gz 2024-09-21T03:45:39.891 INFO:teuthology.orchestra.run.smithi038.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.70857.log 2024-09-21T03:45:39.891 INFO:teuthology.orchestra.run.smithi038.stderr: 85.2% -- replaced with /var/log/ceph/ceph-client.admin.61755.log.gz 2024-09-21T03:45:39.892 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54119.log 2024-09-21T03:45:39.892 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.52254.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49745.log 2024-09-21T03:45:39.892 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.70857.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.52254.log.gz 2024-09-21T03:45:39.892 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.54119.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59661.log 2024-09-21T03:45:39.892 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.49745.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54504.log 2024-09-21T03:45:39.892 INFO:teuthology.orchestra.run.smithi038.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.54119.log.gz 2024-09-21T03:45:39.892 INFO:teuthology.orchestra.run.smithi038.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49745.log.gz 2024-09-21T03:45:39.892 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.59661.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90544.log 2024-09-21T03:45:39.892 INFO:teuthology.orchestra.run.smithi038.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.59661.log.gz 2024-09-21T03:45:39.892 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.54504.log: 92.5%gzip -- replaced with /var/log/ceph/ceph-client.admin.70857.log.gz 2024-09-21T03:45:39.892 INFO:teuthology.orchestra.run.smithi038.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.69144.log 2024-09-21T03:45:39.892 INFO:teuthology.orchestra.run.smithi038.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.54504.log.gz 2024-09-21T03:45:39.892 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87006.log 2024-09-21T03:45:39.893 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.90544.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87658.log/var/log/ceph/ceph-client.admin.69144.log: 2024-09-21T03:45:39.893 INFO:teuthology.orchestra.run.smithi038.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.90544.log.gz 2024-09-21T03:45:39.893 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.87006.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.46633.log 2024-09-21T03:45:39.893 INFO:teuthology.orchestra.run.smithi038.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.87006.log.gz 2024-09-21T03:45:39.893 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.87658.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58437.log 91.1% 2024-09-21T03:45:39.893 INFO:teuthology.orchestra.run.smithi038.stderr: -- replaced with /var/log/ceph/ceph-client.admin.69144.log.gz 2024-09-21T03:45:39.893 INFO:teuthology.orchestra.run.smithi038.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.87658.log.gz 2024-09-21T03:45:39.893 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.0.46633.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60668.log 2024-09-21T03:45:39.893 INFO:teuthology.orchestra.run.smithi038.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.46633.log.gz 2024-09-21T03:45:39.893 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.58437.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49340.log 2024-09-21T03:45:39.893 INFO:teuthology.orchestra.run.smithi038.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.58437.log.gz 2024-09-21T03:45:39.893 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.60668.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44966.log 2024-09-21T03:45:39.893 INFO:teuthology.orchestra.run.smithi038.stderr: 87.4%gzip -- replaced with /var/log/ceph/ceph-client.admin.60668.log.gz 2024-09-21T03:45:39.893 INFO:teuthology.orchestra.run.smithi038.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.36271.log 2024-09-21T03:45:39.894 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.49340.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.49340.log.gz 2024-09-21T03:45:39.894 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.44966.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63808.log 2024-09-21T03:45:39.894 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83749.log 2024-09-21T03:45:39.894 INFO:teuthology.orchestra.run.smithi038.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.44966.log.gz 2024-09-21T03:45:39.894 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.36271.log: 76.3% -- replaced with /var/log/ceph/ceph-client.admin.36271.log.gz 2024-09-21T03:45:39.894 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.63808.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59197.log 2024-09-21T03:45:39.894 INFO:teuthology.orchestra.run.smithi038.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.63808.log.gz 2024-09-21T03:45:39.894 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65062.log 2024-09-21T03:45:39.894 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.83749.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.83749.log.gz 2024-09-21T03:45:39.894 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.59197.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60257.log 2024-09-21T03:45:39.894 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48084.log 2024-09-21T03:45:39.894 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.65062.log: /var/log/ceph/ceph-client.admin.60257.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.69402.log 2024-09-21T03:45:39.894 INFO:teuthology.orchestra.run.smithi038.stderr: 90.9% 89.2% -- replaced with /var/log/ceph/ceph-client.admin.65062.log.gz -- replaced with /var/log/ceph/ceph-client.admin.59197.log.gz 2024-09-21T03:45:39.895 INFO:teuthology.orchestra.run.smithi038.stderr: 2024-09-21T03:45:39.895 INFO:teuthology.orchestra.run.smithi038.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.60257.log.gz 2024-09-21T03:45:39.895 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.48084.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44744.log 2024-09-21T03:45:39.895 INFO:teuthology.orchestra.run.smithi038.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.48084.log.gz 2024-09-21T03:45:39.895 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.0.69402.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49290.log 2024-09-21T03:45:39.895 INFO:teuthology.orchestra.run.smithi038.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.69402.log.gz 2024-09-21T03:45:39.895 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47308.log 2024-09-21T03:45:39.895 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.44744.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.44744.log.gz 2024-09-21T03:45:39.895 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.49290.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52026.log 2024-09-21T03:45:39.895 INFO:teuthology.orchestra.run.smithi038.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.49290.log.gz 2024-09-21T03:45:39.895 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43247.log 2024-09-21T03:45:39.895 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.47308.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47308.log.gz 2024-09-21T03:45:39.895 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.52026.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60529.log 2024-09-21T03:45:39.895 INFO:teuthology.orchestra.run.smithi038.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.52026.log.gz 2024-09-21T03:45:39.896 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89551.log 2024-09-21T03:45:39.896 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.43247.log: /var/log/ceph/ceph-client.admin.60529.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89701.log 2024-09-21T03:45:39.896 INFO:teuthology.orchestra.run.smithi038.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.60529.log.gz 2024-09-21T03:45:39.896 INFO:teuthology.orchestra.run.smithi038.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.43247.log.gz 2024-09-21T03:45:39.896 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.89551.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52609.log 2024-09-21T03:45:39.896 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.89701.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.89551.log.gz 2024-09-21T03:45:39.896 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69529.log 2024-09-21T03:45:39.896 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.52609.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72726.log 2024-09-21T03:45:39.896 INFO:teuthology.orchestra.run.smithi038.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.89701.log.gz 2024-09-21T03:45:39.896 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42991.log 2024-09-21T03:45:39.896 INFO:teuthology.orchestra.run.smithi038.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.52609.log.gz 2024-09-21T03:45:39.896 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.69529.log: /var/log/ceph/ceph-client.admin.72726.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.69529.log.gz 2024-09-21T03:45:39.896 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84456.log 2024-09-21T03:45:39.897 INFO:teuthology.orchestra.run.smithi038.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.72726.log.gz 2024-09-21T03:45:39.897 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59493.log 2024-09-21T03:45:39.897 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.42991.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.42991.log.gz 2024-09-21T03:45:39.897 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.84456.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59788.log 2024-09-21T03:45:39.897 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47392.log 2024-09-21T03:45:39.897 INFO:teuthology.orchestra.run.smithi038.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.84456.log.gz 2024-09-21T03:45:39.897 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.59493.log: /var/log/ceph/ceph-client.admin.59788.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.59493.log.gz 2024-09-21T03:45:39.897 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62591.log 2024-09-21T03:45:39.897 INFO:teuthology.orchestra.run.smithi038.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.59788.log.gz 2024-09-21T03:45:39.897 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74134.log 2024-09-21T03:45:39.897 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.47392.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.47392.log.gz 2024-09-21T03:45:39.897 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.62591.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.70200.log 2024-09-21T03:45:39.897 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62027.log 87.4% 2024-09-21T03:45:39.898 INFO:teuthology.orchestra.run.smithi038.stderr: -- replaced with /var/log/ceph/ceph-client.admin.62591.log.gz 2024-09-21T03:45:39.898 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.74134.log: /var/log/ceph/ceph-client.0.70200.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.74134.log.gz 2024-09-21T03:45:39.898 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88754.log 2024-09-21T03:45:39.898 INFO:teuthology.orchestra.run.smithi038.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.70200.log.gz 2024-09-21T03:45:39.898 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.62027.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58479.log 2024-09-21T03:45:39.898 INFO:teuthology.orchestra.run.smithi038.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.62027.log.gz 2024-09-21T03:45:39.898 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.88754.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89821.log 2024-09-21T03:45:39.898 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78569.log 2024-09-21T03:45:39.898 INFO:teuthology.orchestra.run.smithi038.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.88754.log.gz 2024-09-21T03:45:39.898 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.58479.log: /var/log/ceph/ceph-client.admin.89821.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.58479.log.gz 2024-09-21T03:45:39.898 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78236.log 2024-09-21T03:45:39.898 INFO:teuthology.orchestra.run.smithi038.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.89821.log.gz 2024-09-21T03:45:39.898 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67745.log 2024-09-21T03:45:39.899 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.78569.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.78569.log.gz 2024-09-21T03:45:39.899 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.78236.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80251.log 2024-09-21T03:45:39.899 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49469.log 2024-09-21T03:45:39.899 INFO:teuthology.orchestra.run.smithi038.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.78236.log.gz 2024-09-21T03:45:39.899 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.67745.log: /var/log/ceph/ceph-client.admin.80251.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80587.log 2024-09-21T03:45:39.899 INFO:teuthology.orchestra.run.smithi038.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.80251.log.gz 2024-09-21T03:45:39.899 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.49469.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66811.log 2024-09-21T03:45:39.899 INFO:teuthology.orchestra.run.smithi038.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.49469.log.gz 2024-09-21T03:45:39.899 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.80587.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62378.log 2024-09-21T03:45:39.899 INFO:teuthology.orchestra.run.smithi038.stderr: 92.6% -- replaced with /var/log/ceph/ceph-client.admin.67745.log.gz 2024-09-21T03:45:39.899 INFO:teuthology.orchestra.run.smithi038.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.80587.log.gz 2024-09-21T03:45:39.899 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.66811.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74462.log 2024-09-21T03:45:39.899 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62633.log 2024-09-21T03:45:39.900 INFO:teuthology.orchestra.run.smithi038.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.66811.log.gz 2024-09-21T03:45:39.900 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.62378.log: /var/log/ceph/ceph-client.admin.74462.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58641.log 2024-09-21T03:45:39.900 INFO:teuthology.orchestra.run.smithi038.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.62378.log.gz 2024-09-21T03:45:39.900 INFO:teuthology.orchestra.run.smithi038.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.74462.log.gz 2024-09-21T03:45:39.900 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.62633.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46772.log 2024-09-21T03:45:39.900 INFO:teuthology.orchestra.run.smithi038.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.62633.log.gz 2024-09-21T03:45:39.900 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.58641.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74242.log 2024-09-21T03:45:39.900 INFO:teuthology.orchestra.run.smithi038.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.58641.log.gz 2024-09-21T03:45:39.900 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85784.log 2024-09-21T03:45:39.900 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.46772.log: /var/log/ceph/ceph-client.admin.74242.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82388.log 2024-09-21T03:45:39.900 INFO:teuthology.orchestra.run.smithi038.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46772.log.gz 2024-09-21T03:45:39.900 INFO:teuthology.orchestra.run.smithi038.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.74242.log.gz 2024-09-21T03:45:39.900 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.85784.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58907.log 2024-09-21T03:45:39.901 INFO:teuthology.orchestra.run.smithi038.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.85784.log.gz 2024-09-21T03:45:39.901 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.82388.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86754.log 2024-09-21T03:45:39.901 INFO:teuthology.orchestra.run.smithi038.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.82388.log.gz 2024-09-21T03:45:39.901 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91500.log 2024-09-21T03:45:39.901 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.58907.log: /var/log/ceph/ceph-client.admin.86754.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76979.log 87.7% 2024-09-21T03:45:39.901 INFO:teuthology.orchestra.run.smithi038.stderr: -- replaced with /var/log/ceph/ceph-client.admin.58907.log.gz 2024-09-21T03:45:39.901 INFO:teuthology.orchestra.run.smithi038.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.86754.log.gz 2024-09-21T03:45:39.901 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.91500.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82871.log 2024-09-21T03:45:39.901 INFO:teuthology.orchestra.run.smithi038.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.91500.log.gz 2024-09-21T03:45:39.901 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36299.log 2024-09-21T03:45:39.901 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.76979.log: /var/log/ceph/ceph-client.admin.82871.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68049.log 2024-09-21T03:45:39.901 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.36299.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70834.log 2024-09-21T03:45:39.902 INFO:teuthology.orchestra.run.smithi038.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36299.log.gz 2024-09-21T03:45:39.902 INFO:teuthology.orchestra.run.smithi038.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.82871.log.gz 2024-09-21T03:45:39.902 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.68049.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65395.log 2024-09-21T03:45:39.902 INFO:teuthology.orchestra.run.smithi038.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.68049.log.gz 2024-09-21T03:45:39.902 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55093.log 2024-09-21T03:45:39.902 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.70834.log: 92.5% -- replaced with /var/log/ceph/ceph-client.admin.76979.log.gz 2024-09-21T03:45:39.902 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.65395.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73939.log 2024-09-21T03:45:39.902 INFO:teuthology.orchestra.run.smithi038.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.70834.log.gz 2024-09-21T03:45:39.902 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.55093.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68648.log 2024-09-21T03:45:39.902 INFO:teuthology.orchestra.run.smithi038.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.65395.log.gz 2024-09-21T03:45:39.902 INFO:teuthology.orchestra.run.smithi038.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.55093.log.gz 2024-09-21T03:45:39.902 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.73939.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72341.log 2024-09-21T03:45:39.903 INFO:teuthology.orchestra.run.smithi038.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.73939.log.gz 2024-09-21T03:45:39.903 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.68648.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84120.log 2024-09-21T03:45:39.903 INFO:teuthology.orchestra.run.smithi038.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.68648.log.gz 2024-09-21T03:45:39.903 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79755.log 2024-09-21T03:45:39.903 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.72341.log: /var/log/ceph/ceph-client.admin.84120.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81135.log 2024-09-21T03:45:39.903 INFO:teuthology.orchestra.run.smithi038.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.84120.log.gz 2024-09-21T03:45:39.903 INFO:teuthology.orchestra.run.smithi038.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.72341.log.gz 2024-09-21T03:45:39.903 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83947.log 2024-09-21T03:45:39.903 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.79755.log: /var/log/ceph/ceph-client.admin.81135.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.79755.log.gz 2024-09-21T03:45:39.903 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58745.log 2024-09-21T03:45:39.903 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48852.log 2024-09-21T03:45:39.903 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.83947.log: /var/log/ceph/ceph-client.admin.58745.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.81135.log.gz 2024-09-21T03:45:39.903 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- 89.0% -- replaced with /var/log/ceph/ceph-client.admin.83947.log.gz 2024-09-21T03:45:39.904 INFO:teuthology.orchestra.run.smithi038.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.58745.log.gz 2024-09-21T03:45:39.904 INFO:teuthology.orchestra.run.smithi038.stderr: /var/log/ceph/ceph-client.admin.52212.log 2024-09-21T03:45:39.904 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.48852.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48852.log.gz 2024-09-21T03:45:39.904 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44963.log 2024-09-21T03:45:39.904 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.52212.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63137.log 2024-09-21T03:45:39.904 INFO:teuthology.orchestra.run.smithi038.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.52212.log.gz 2024-09-21T03:45:39.904 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.44963.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80143.log 2024-09-21T03:45:39.904 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55662.log 2024-09-21T03:45:39.904 INFO:teuthology.orchestra.run.smithi038.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.44963.log.gz 2024-09-21T03:45:39.904 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.63137.log: /var/log/ceph/ceph-client.admin.80143.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.63137.log.gz 2024-09-21T03:45:39.904 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65839.log 2024-09-21T03:45:39.904 INFO:teuthology.orchestra.run.smithi038.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.80143.log.gz 2024-09-21T03:45:39.904 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81882.log 2024-09-21T03:45:39.905 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.55662.log: /var/log/ceph/ceph-client.admin.65839.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80494.log 2024-09-21T03:45:39.905 INFO:teuthology.orchestra.run.smithi038.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.55662.log.gz 2024-09-21T03:45:39.905 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.81882.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.65839.log.gz 2024-09-21T03:45:39.905 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58335.log 2024-09-21T03:45:39.905 INFO:teuthology.orchestra.run.smithi038.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.81882.log.gz 2024-09-21T03:45:39.905 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.80494.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61238.log 2024-09-21T03:45:39.905 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.61430.log 2024-09-21T03:45:39.905 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.58335.log: /var/log/ceph/ceph-client.admin.61238.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66135.log 2024-09-21T03:45:39.905 INFO:teuthology.orchestra.run.smithi038.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.58335.log.gz 2024-09-21T03:45:39.905 INFO:teuthology.orchestra.run.smithi038.stderr: 87.7%gzip/var/log/ceph/ceph-client.0.61430.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.43523.log 2024-09-21T03:45:39.905 INFO:teuthology.orchestra.run.smithi038.stderr: -- replaced with /var/log/ceph/ceph-client.admin.61238.log.gz 2024-09-21T03:45:39.905 INFO:teuthology.orchestra.run.smithi038.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.61430.log.gz 2024-09-21T03:45:39.906 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.66135.log: 92.6% -- replaced with /var/log/ceph/ceph-client.admin.80494.log.gz 2024-09-21T03:45:39.906 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62147.log 2024-09-21T03:45:39.906 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.43523.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76914.log 2024-09-21T03:45:39.906 INFO:teuthology.orchestra.run.smithi038.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.66135.log.gz 2024-09-21T03:45:39.906 INFO:teuthology.orchestra.run.smithi038.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43523.log.gz 2024-09-21T03:45:39.906 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.62147.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51323.log 2024-09-21T03:45:39.906 INFO:teuthology.orchestra.run.smithi038.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.62147.log.gz/var/log/ceph/ceph-client.admin.76914.log: 2024-09-21T03:45:39.906 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60398.log 2024-09-21T03:45:39.906 INFO:teuthology.orchestra.run.smithi038.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.76914.log.gz 2024-09-21T03:45:39.906 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.51323.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90037.log 2024-09-21T03:45:39.932 INFO:teuthology.orchestra.run.smithi195.stderr: 91.3% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-09-21T03:45:40.110 INFO:teuthology.orchestra.run.smithi195.stderr: 91.1% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-09-21T03:45:40.120 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.60398.log: gzip -5 91.0% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.51323.log.gz 2024-09-21T03:45:40.120 INFO:teuthology.orchestra.run.smithi038.stderr: /var/log/ceph/ceph-client.admin.87868.log 2024-09-21T03:45:40.120 INFO:teuthology.orchestra.run.smithi038.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.60398.log.gz 2024-09-21T03:45:40.120 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48580.log 2024-09-21T03:45:40.120 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.90037.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.90037.log.gz 2024-09-21T03:45:40.120 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.87868.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84663.log 2024-09-21T03:45:40.120 INFO:teuthology.orchestra.run.smithi038.stderr:gzip 86.8% -- replaced with /var/log/ceph/ceph-client.admin.87868.log.gz -5 2024-09-21T03:45:40.120 INFO:teuthology.orchestra.run.smithi038.stderr: --verbose -- /var/log/ceph/ceph-client.admin.79536.log 2024-09-21T03:45:40.120 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.48580.log: /var/log/ceph/ceph-client.admin.84663.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48580.log.gz 2024-09-21T03:45:40.121 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60485.log 2024-09-21T03:45:40.121 INFO:teuthology.orchestra.run.smithi038.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.84663.log.gz 2024-09-21T03:45:40.121 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74851.log 2024-09-21T03:45:40.121 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.79536.log: /var/log/ceph/ceph-client.admin.60485.log: gzip 87.6% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.79536.log.gz -- 2024-09-21T03:45:40.121 INFO:teuthology.orchestra.run.smithi038.stderr: /var/log/ceph/ceph-client.admin.51389.log 2024-09-21T03:45:40.121 INFO:teuthology.orchestra.run.smithi038.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.60485.log.gz 2024-09-21T03:45:40.121 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.74851.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74284.log 2024-09-21T03:45:40.121 INFO:teuthology.orchestra.run.smithi038.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.74851.log.gz 2024-09-21T03:45:40.121 INFO:teuthology.orchestra.run.smithi038.stderr:gzip/var/log/ceph/ceph-client.admin.51389.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.44450.log 2024-09-21T03:45:40.121 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.74284.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62293.log 2024-09-21T03:45:40.121 INFO:teuthology.orchestra.run.smithi038.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.51389.log.gz 2024-09-21T03:45:40.121 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.44450.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.74284.log.gz 2024-09-21T03:45:40.121 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68301.log 2024-09-21T03:45:40.122 INFO:teuthology.orchestra.run.smithi038.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.44450.log.gz 2024-09-21T03:45:40.122 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.62293.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60794.log 2024-09-21T03:45:40.122 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.68301.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80639.log 2024-09-21T03:45:40.122 INFO:teuthology.orchestra.run.smithi038.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.62293.log.gz 2024-09-21T03:45:40.122 INFO:teuthology.orchestra.run.smithi038.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.68301.log.gz 2024-09-21T03:45:40.122 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45282.log 2024-09-21T03:45:40.122 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.60794.log: /var/log/ceph/ceph-client.admin.80639.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.60794.log.gz 2024-09-21T03:45:40.122 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44961.log 2024-09-21T03:45:40.122 INFO:teuthology.orchestra.run.smithi038.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.80639.log.gz 2024-09-21T03:45:40.122 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.45282.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55051.log 2024-09-21T03:45:40.122 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.44961.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.45282.log.gz 2024-09-21T03:45:40.122 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68826.log 2024-09-21T03:45:40.122 INFO:teuthology.orchestra.run.smithi038.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.44961.log.gz 2024-09-21T03:45:40.123 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.55051.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90334.log 2024-09-21T03:45:40.123 INFO:teuthology.orchestra.run.smithi038.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.55051.log.gz 2024-09-21T03:45:40.123 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.68826.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82434.log 2024-09-21T03:45:40.123 INFO:teuthology.orchestra.run.smithi038.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.68826.log.gz 2024-09-21T03:45:40.123 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.90334.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72980.log 2024-09-21T03:45:40.123 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.82434.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.90334.log.gz 2024-09-21T03:45:40.123 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85579.log 2024-09-21T03:45:40.123 INFO:teuthology.orchestra.run.smithi038.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.82434.log.gz 2024-09-21T03:45:40.123 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.72980.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86220.log 2024-09-21T03:45:40.123 INFO:teuthology.orchestra.run.smithi038.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.72980.log.gz 2024-09-21T03:45:40.123 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.85579.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71845.log 2024-09-21T03:45:40.123 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.86220.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90586.log 2024-09-21T03:45:40.123 INFO:teuthology.orchestra.run.smithi038.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.86220.log.gz 2024-09-21T03:45:40.124 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-09-21T03:45:40.124 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.71845.log: /var/log/ceph/ceph-client.admin.90586.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.71845.log.gz 2024-09-21T03:45:40.124 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84400.log 2024-09-21T03:45:40.124 INFO:teuthology.orchestra.run.smithi038.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.90586.log.gz 2024-09-21T03:45:40.124 INFO:teuthology.orchestra.run.smithi038.stderr: 92.5%/var/log/ceph/ceph-mds.a.log: -- replaced with /var/log/ceph/ceph-client.admin.85579.log.gz 2024-09-21T03:45:40.124 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85102.log 2024-09-21T03:45:40.124 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.84400.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67932.log 2024-09-21T03:45:40.124 INFO:teuthology.orchestra.run.smithi038.stderr: 89.1%/var/log/ceph/ceph-client.admin.85102.log: -- replaced with /var/log/ceph/ceph-client.admin.84400.log.gz 2024-09-21T03:45:40.124 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80106.log 2024-09-21T03:45:40.124 INFO:teuthology.orchestra.run.smithi038.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.85102.log.gz 2024-09-21T03:45:40.124 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.67932.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81789.log 2024-09-21T03:45:40.124 INFO:teuthology.orchestra.run.smithi038.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.67932.log.gz 2024-09-21T03:45:40.124 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.80106.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53215.log 2024-09-21T03:45:40.125 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65913.log 2024-09-21T03:45:40.125 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.81789.log: /var/log/ceph/ceph-client.admin.53215.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83868.log 2024-09-21T03:45:40.125 INFO:teuthology.orchestra.run.smithi038.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.53215.log.gz/var/log/ceph/ceph-client.admin.65913.log: 2024-09-21T03:45:40.125 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66246.log 2024-09-21T03:45:40.125 INFO:teuthology.orchestra.run.smithi038.stderr: 89.1% 92.5% -- replaced with /var/log/ceph/ceph-client.admin.80106.log.gz 2024-09-21T03:45:40.125 INFO:teuthology.orchestra.run.smithi038.stderr: -- replaced with /var/log/ceph/ceph-client.admin.65913.log.gz 2024-09-21T03:45:40.125 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70008.log 2024-09-21T03:45:40.125 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.83868.log: /var/log/ceph/ceph-client.admin.66246.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64877.log 2024-09-21T03:45:40.125 INFO:teuthology.orchestra.run.smithi038.stderr: 92.5% -- replaced with /var/log/ceph/ceph-client.admin.81789.log.gz 2024-09-21T03:45:40.125 INFO:teuthology.orchestra.run.smithi038.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.83868.log.gz 2024-09-21T03:45:40.125 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.70008.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91177.log 2024-09-21T03:45:40.125 INFO:teuthology.orchestra.run.smithi038.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.66246.log.gz 2024-09-21T03:45:40.125 INFO:teuthology.orchestra.run.smithi038.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.70008.log.gz 2024-09-21T03:45:40.125 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80471.log 2024-09-21T03:45:40.126 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.64877.log: /var/log/ceph/ceph-client.admin.91177.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63011.log 2024-09-21T03:45:40.126 INFO:teuthology.orchestra.run.smithi038.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.64877.log.gz 2024-09-21T03:45:40.126 INFO:teuthology.orchestra.run.smithi038.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.91177.log.gz 2024-09-21T03:45:40.126 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.80471.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78851.log 2024-09-21T03:45:40.126 INFO:teuthology.orchestra.run.smithi038.stderr: 87.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74190.log 2024-09-21T03:45:40.126 INFO:teuthology.orchestra.run.smithi038.stderr: -- replaced with /var/log/ceph/ceph-client.admin.80471.log.gz/var/log/ceph/ceph-client.admin.63011.log: 2024-09-21T03:45:40.126 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.78851.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.63011.log.gz 2024-09-21T03:45:40.126 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49024.log 2024-09-21T03:45:40.126 INFO:teuthology.orchestra.run.smithi038.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.78851.log.gz 2024-09-21T03:45:40.126 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.74190.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.50789.log 2024-09-21T03:45:40.126 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88138.log 2024-09-21T03:45:40.126 INFO:teuthology.orchestra.run.smithi038.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.74190.log.gz 2024-09-21T03:45:40.126 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.49024.log: /var/log/ceph/ceph-client.0.50789.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73551.log 2024-09-21T03:45:40.126 INFO:teuthology.orchestra.run.smithi038.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.49024.log.gz 2024-09-21T03:45:40.127 INFO:teuthology.orchestra.run.smithi038.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.50789.log.gz 2024-09-21T03:45:40.127 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.88138.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36730.log 2024-09-21T03:45:40.127 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.73551.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67544.log 2024-09-21T03:45:40.127 INFO:teuthology.orchestra.run.smithi038.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.73551.log.gz 2024-09-21T03:45:40.127 INFO:teuthology.orchestra.run.smithi038.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.88138.log.gz 2024-09-21T03:45:40.127 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.36730.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91836.log 2024-09-21T03:45:40.127 INFO:teuthology.orchestra.run.smithi038.stderr: 85.9% -- replaced with /var/log/ceph/ceph-client.admin.36730.log.gz 2024-09-21T03:45:40.127 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54894.log 2024-09-21T03:45:40.127 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.67544.log: /var/log/ceph/ceph-client.admin.91836.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61508.log 2024-09-21T03:45:40.127 INFO:teuthology.orchestra.run.smithi038.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.67544.log.gz 2024-09-21T03:45:40.127 INFO:teuthology.orchestra.run.smithi038.stderr: 87.2%/var/log/ceph/ceph-client.admin.54894.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51984.log 2024-09-21T03:45:40.127 INFO:teuthology.orchestra.run.smithi038.stderr: -- replaced with /var/log/ceph/ceph-client.admin.91836.log.gz 2024-09-21T03:45:40.128 INFO:teuthology.orchestra.run.smithi038.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.54894.log.gz 2024-09-21T03:45:40.128 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.61508.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78678.log 2024-09-21T03:45:40.128 INFO:teuthology.orchestra.run.smithi038.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.61508.log.gz 2024-09-21T03:45:40.128 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60356.log 2024-09-21T03:45:40.128 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.51984.log: /var/log/ceph/ceph-client.admin.78678.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58521.log 2024-09-21T03:45:40.128 INFO:teuthology.orchestra.run.smithi038.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.51984.log.gz 2024-09-21T03:45:40.128 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.60356.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88096.log 2024-09-21T03:45:40.128 INFO:teuthology.orchestra.run.smithi038.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.78678.log.gz 2024-09-21T03:45:40.128 INFO:teuthology.orchestra.run.smithi038.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.60356.log.gz 2024-09-21T03:45:40.128 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.58521.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77690.log 2024-09-21T03:45:40.128 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- 87.1% /var/log/ceph/ceph-client.admin.69690.log 2024-09-21T03:45:40.128 INFO:teuthology.orchestra.run.smithi038.stderr: -- replaced with /var/log/ceph/ceph-client.admin.58521.log.gz 2024-09-21T03:45:40.128 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.88096.log: /var/log/ceph/ceph-client.admin.77690.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.88096.log.gz 2024-09-21T03:45:40.128 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45654.log 2024-09-21T03:45:40.129 INFO:teuthology.orchestra.run.smithi038.stderr: 87.2%/var/log/ceph/ceph-client.admin.69690.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48378.log 2024-09-21T03:45:40.129 INFO:teuthology.orchestra.run.smithi038.stderr: -- replaced with /var/log/ceph/ceph-client.admin.77690.log.gz 2024-09-21T03:45:40.129 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.45654.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86483.log 2024-09-21T03:45:40.129 INFO:teuthology.orchestra.run.smithi038.stderr: 87.5%/var/log/ceph/ceph-client.admin.48378.log: -- replaced with /var/log/ceph/ceph-client.admin.45654.log.gz 2024-09-21T03:45:40.129 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90773.log 2024-09-21T03:45:40.129 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.86483.log: 87.3%gzip -- replaced with /var/log/ceph/ceph-client.admin.48378.log.gz -5 2024-09-21T03:45:40.129 INFO:teuthology.orchestra.run.smithi038.stderr: --verbose -- /var/log/ceph/ceph-client.admin.48980.log 2024-09-21T03:45:40.129 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.90773.log: 87.2%gzip -- replaced with /var/log/ceph/ceph-client.admin.86483.log.gz 2024-09-21T03:45:40.129 INFO:teuthology.orchestra.run.smithi038.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.78743.log 2024-09-21T03:45:40.129 INFO:teuthology.orchestra.run.smithi038.stderr: 92.6% -- replaced with /var/log/ceph/ceph-client.admin.69690.log.gz 2024-09-21T03:45:40.129 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.48980.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76045.log 2024-09-21T03:45:40.129 INFO:teuthology.orchestra.run.smithi038.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.90773.log.gz 2024-09-21T03:45:40.129 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.78743.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.48980.log.gz 2024-09-21T03:45:40.130 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53681.log 2024-09-21T03:45:40.130 INFO:teuthology.orchestra.run.smithi038.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.78743.log.gz 2024-09-21T03:45:40.130 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.76045.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79170.log 2024-09-21T03:45:40.130 INFO:teuthology.orchestra.run.smithi038.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.76045.log.gz 2024-09-21T03:45:40.130 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64060.log 2024-09-21T03:45:40.130 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.53681.log: /var/log/ceph/ceph-client.admin.79170.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71792.log 2024-09-21T03:45:40.130 INFO:teuthology.orchestra.run.smithi038.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.53681.log.gz 2024-09-21T03:45:40.130 INFO:teuthology.orchestra.run.smithi038.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.79170.log.gz 2024-09-21T03:45:40.130 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.64060.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67782.log 2024-09-21T03:45:40.130 INFO:teuthology.orchestra.run.smithi038.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.64060.log.gz 2024-09-21T03:45:40.130 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.71792.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47434.log 2024-09-21T03:45:40.130 INFO:teuthology.orchestra.run.smithi038.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.71792.log.gz 2024-09-21T03:45:40.130 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.67782.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64988.log 2024-09-21T03:45:40.131 INFO:teuthology.orchestra.run.smithi038.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.67782.log.gz 2024-09-21T03:45:40.131 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.47434.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54227.log 2024-09-21T03:45:40.131 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.64988.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.47434.log.gz 2024-09-21T03:45:40.131 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81201.log 2024-09-21T03:45:40.131 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.54227.log: gzip -5 --verbose 89.4% -- -- replaced with /var/log/ceph/ceph-client.admin.64988.log.gz /var/log/ceph/ceph-client.admin.78120.log 2024-09-21T03:45:40.131 INFO:teuthology.orchestra.run.smithi038.stderr: 2024-09-21T03:45:40.131 INFO:teuthology.orchestra.run.smithi038.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.54227.log.gz 2024-09-21T03:45:40.131 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.81201.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75655.log 2024-09-21T03:45:40.131 INFO:teuthology.orchestra.run.smithi038.stderr:gzip 87.6% -5 -- replaced with /var/log/ceph/ceph-client.admin.81201.log.gz --verbose 2024-09-21T03:45:40.131 INFO:teuthology.orchestra.run.smithi038.stderr: -- /var/log/ceph/ceph-client.0.64441.log 2024-09-21T03:45:40.131 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.78120.log: /var/log/ceph/ceph-client.admin.75655.log: 87.2%gzip -- replaced with /var/log/ceph/ceph-client.admin.78120.log.gz -5 2024-09-21T03:45:40.131 INFO:teuthology.orchestra.run.smithi038.stderr: --verbose -- /var/log/ceph/ceph-client.admin.45277.log 2024-09-21T03:45:40.131 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.0.64441.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85181.log 2024-09-21T03:45:40.131 INFO:teuthology.orchestra.run.smithi038.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.75655.log.gz 2024-09-21T03:45:40.132 INFO:teuthology.orchestra.run.smithi038.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.64441.log.gz 2024-09-21T03:45:40.132 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91373.log 2024-09-21T03:45:40.132 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.45277.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89174.log 2024-09-21T03:45:40.132 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.85181.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.45277.log.gz 2024-09-21T03:45:40.132 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.91373.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72803.log 2024-09-21T03:45:40.132 INFO:teuthology.orchestra.run.smithi038.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.85181.log.gz 2024-09-21T03:45:40.132 INFO:teuthology.orchestra.run.smithi038.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.91373.log.gz 2024-09-21T03:45:40.132 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67890.log 2024-09-21T03:45:40.132 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.89174.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.59095.log 2024-09-21T03:45:40.132 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.72803.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.89174.log.gz 2024-09-21T03:45:40.132 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.67890.log: 87.2%gzip -- replaced with /var/log/ceph/ceph-client.admin.72803.log.gz 2024-09-21T03:45:40.132 INFO:teuthology.orchestra.run.smithi038.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.79289.log 2024-09-21T03:45:40.132 INFO:teuthology.orchestra.run.smithi038.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.67890.log.gz 2024-09-21T03:45:40.132 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.0.59095.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65210.log 2024-09-21T03:45:40.133 INFO:teuthology.orchestra.run.smithi038.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.59095.log.gz 2024-09-21T03:45:40.133 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73032.log 2024-09-21T03:45:40.133 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.79289.log: /var/log/ceph/ceph-client.admin.65210.log: gzip 87.4% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.79289.log.gz -- 2024-09-21T03:45:40.133 INFO:teuthology.orchestra.run.smithi038.stderr: /var/log/ceph/ceph-client.admin.60878.log 2024-09-21T03:45:40.133 INFO:teuthology.orchestra.run.smithi038.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.65210.log.gz 2024-09-21T03:45:40.133 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.73032.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65173.log 2024-09-21T03:45:40.133 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91269.log 2024-09-21T03:45:40.133 INFO:teuthology.orchestra.run.smithi038.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.73032.log.gz 2024-09-21T03:45:40.133 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.60878.log: /var/log/ceph/ceph-client.admin.65173.log: gzip -5 87.3% -- replaced with /var/log/ceph/ceph-client.admin.60878.log.gz 2024-09-21T03:45:40.133 INFO:teuthology.orchestra.run.smithi038.stderr: --verbose -- /var/log/ceph/ceph-client.0.81393.log 2024-09-21T03:45:40.133 INFO:teuthology.orchestra.run.smithi038.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.65173.log.gz 2024-09-21T03:45:40.133 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.91269.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74485.log 2024-09-21T03:45:40.133 INFO:teuthology.orchestra.run.smithi038.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.91269.log.gz 2024-09-21T03:45:40.133 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83430.log 2024-09-21T03:45:40.134 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.0.81393.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.81393.log.gz 2024-09-21T03:45:40.134 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.74485.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86796.log 2024-09-21T03:45:40.134 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.83430.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90376.log 2024-09-21T03:45:40.134 INFO:teuthology.orchestra.run.smithi038.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.83430.log.gz 2024-09-21T03:45:40.134 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.86796.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77900.log 2024-09-21T03:45:40.134 INFO:teuthology.orchestra.run.smithi038.stderr: 92.5% 87.4% -- replaced with /var/log/ceph/ceph-client.admin.86796.log.gz 2024-09-21T03:45:40.134 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.90376.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49595.log 2024-09-21T03:45:40.134 INFO:teuthology.orchestra.run.smithi038.stderr: -- replaced with /var/log/ceph/ceph-client.admin.74485.log.gz 2024-09-21T03:45:40.134 INFO:teuthology.orchestra.run.smithi038.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.90376.log.gz 2024-09-21T03:45:40.134 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.77900.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45657.log 2024-09-21T03:45:40.134 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.49595.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.77900.log.gz 2024-09-21T03:45:40.134 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78785.log 2024-09-21T03:45:40.135 INFO:teuthology.orchestra.run.smithi038.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.49595.log.gz 2024-09-21T03:45:40.135 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.45657.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.57500.log 2024-09-21T03:45:40.135 INFO:teuthology.orchestra.run.smithi038.stderr: 87.4%gzip -- replaced with /var/log/ceph/ceph-client.admin.45657.log.gz 2024-09-21T03:45:40.135 INFO:teuthology.orchestra.run.smithi038.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.71540.log 2024-09-21T03:45:40.135 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.78785.log: /var/log/ceph/ceph-client.0.57500.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-09-21T03:45:40.135 INFO:teuthology.orchestra.run.smithi038.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.57500.log.gz 2024-09-21T03:45:40.135 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.71540.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76610.log 2024-09-21T03:45:40.135 INFO:teuthology.orchestra.run.smithi038.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.78785.log.gz 2024-09-21T03:45:40.135 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-mon.a.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.71540.log.gz 2024-09-21T03:45:40.135 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82200.log 2024-09-21T03:45:40.135 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.76610.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46313.log 2024-09-21T03:45:40.135 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.82200.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64253.log 2024-09-21T03:45:40.136 INFO:teuthology.orchestra.run.smithi038.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.76610.log.gz 2024-09-21T03:45:40.136 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.46313.log: 87.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86922.log 2024-09-21T03:45:40.136 INFO:teuthology.orchestra.run.smithi038.stderr: -- replaced with /var/log/ceph/ceph-client.admin.82200.log.gz 2024-09-21T03:45:40.136 INFO:teuthology.orchestra.run.smithi038.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46313.log.gz 2024-09-21T03:45:40.136 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.64253.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75080.log 2024-09-21T03:45:40.136 INFO:teuthology.orchestra.run.smithi038.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.64253.log.gz 2024-09-21T03:45:40.136 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73074.log 2024-09-21T03:45:40.136 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.86922.log: /var/log/ceph/ceph-client.admin.75080.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45656.log 2024-09-21T03:45:40.136 INFO:teuthology.orchestra.run.smithi038.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.86922.log.gz 2024-09-21T03:45:40.136 INFO:teuthology.orchestra.run.smithi038.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.75080.log.gz 2024-09-21T03:45:40.136 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68868.log 2024-09-21T03:45:40.136 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.73074.log: /var/log/ceph/ceph-client.admin.45656.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77614.log 2024-09-21T03:45:40.136 INFO:teuthology.orchestra.run.smithi038.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.45656.log.gz/var/log/ceph/ceph-client.admin.68868.log: 2024-09-21T03:45:40.136 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84597.log 2024-09-21T03:45:40.137 INFO:teuthology.orchestra.run.smithi038.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.73074.log.gz 2024-09-21T03:45:40.137 INFO:teuthology.orchestra.run.smithi038.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.68868.log.gz 2024-09-21T03:45:40.137 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.77614.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.83618.log 2024-09-21T03:45:40.137 INFO:teuthology.orchestra.run.smithi038.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.77614.log.gz 2024-09-21T03:45:40.137 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.84597.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78000.log 2024-09-21T03:45:40.137 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.0.83618.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.83618.log.gz 2024-09-21T03:45:40.137 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57845.log 2024-09-21T03:45:40.137 INFO:teuthology.orchestra.run.smithi038.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.84597.log.gz 2024-09-21T03:45:40.137 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.78000.log: /var/log/ceph/ceph-client.admin.58787.log 2024-09-21T03:45:40.137 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.57845.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.78000.log.gz 2024-09-21T03:45:40.137 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89020.log 2024-09-21T03:45:40.137 INFO:teuthology.orchestra.run.smithi038.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.57845.log.gz 2024-09-21T03:45:40.138 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.58787.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52068.log 2024-09-21T03:45:40.138 INFO:teuthology.orchestra.run.smithi038.stderr:gzip/var/log/ceph/ceph-client.admin.89020.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.55704.log 2024-09-21T03:45:40.138 INFO:teuthology.orchestra.run.smithi038.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.58787.log.gz 2024-09-21T03:45:40.138 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.52068.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.89020.log.gz 2024-09-21T03:45:40.138 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81826.log 2024-09-21T03:45:40.138 INFO:teuthology.orchestra.run.smithi038.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.52068.log.gz 2024-09-21T03:45:40.138 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.55704.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.89362.log 2024-09-21T03:45:40.138 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.81826.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53597.log 2024-09-21T03:45:40.138 INFO:teuthology.orchestra.run.smithi038.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.55704.log.gz 2024-09-21T03:45:40.138 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.0.89362.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58159.log 2024-09-21T03:45:40.138 INFO:teuthology.orchestra.run.smithi038.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.81826.log.gz 2024-09-21T03:45:40.138 INFO:teuthology.orchestra.run.smithi038.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.89362.log.gz 2024-09-21T03:45:40.138 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.53597.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78289.log 2024-09-21T03:45:40.139 INFO:teuthology.orchestra.run.smithi038.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.53597.log.gz 2024-09-21T03:45:40.139 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.58159.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79442.log 2024-09-21T03:45:40.139 INFO:teuthology.orchestra.run.smithi038.stderr: 85.3% -- replaced with /var/log/ceph/ceph-client.admin.58159.log.gz 2024-09-21T03:45:40.139 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.78289.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.84851.log 2024-09-21T03:45:40.139 INFO:teuthology.orchestra.run.smithi038.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.78289.log.gz 2024-09-21T03:45:40.139 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.79442.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57792.log 2024-09-21T03:45:40.139 INFO:teuthology.orchestra.run.smithi038.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.79442.log.gz 2024-09-21T03:45:40.139 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.0.84851.log: gzip -5 --verbose 0.0% -- /var/log/ceph/ceph-client.admin.79811.log 2024-09-21T03:45:40.139 INFO:teuthology.orchestra.run.smithi038.stderr: -- replaced with /var/log/ceph/ceph-client.0.84851.log.gz 2024-09-21T03:45:40.139 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.57792.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88204.log 2024-09-21T03:45:40.139 INFO:teuthology.orchestra.run.smithi038.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.57792.log.gz 2024-09-21T03:45:40.139 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.79811.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72181.log 2024-09-21T03:45:40.139 INFO:teuthology.orchestra.run.smithi038.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.79811.log.gz 2024-09-21T03:45:40.140 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.88204.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47476.log 2024-09-21T03:45:40.140 INFO:teuthology.orchestra.run.smithi038.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.88204.log.gz 2024-09-21T03:45:40.140 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.72181.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68928.log 2024-09-21T03:45:40.140 INFO:teuthology.orchestra.run.smithi038.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.72181.log.gz 2024-09-21T03:45:40.140 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.47476.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59872.log 2024-09-21T03:45:40.140 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.68928.log: gzip -5 --verbose 87.4% -- /var/log/ceph/ceph-client.admin.74522.log -- replaced with /var/log/ceph/ceph-client.admin.47476.log.gz 2024-09-21T03:45:40.140 INFO:teuthology.orchestra.run.smithi038.stderr: 2024-09-21T03:45:40.140 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.59872.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50192.log 2024-09-21T03:45:40.140 INFO:teuthology.orchestra.run.smithi038.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.68928.log.gz 2024-09-21T03:45:40.140 INFO:teuthology.orchestra.run.smithi038.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.59872.log.gz 2024-09-21T03:45:40.140 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53303.log 2024-09-21T03:45:40.140 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.74522.log: /var/log/ceph/ceph-client.admin.50192.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52399.log 2024-09-21T03:45:40.140 INFO:teuthology.orchestra.run.smithi038.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.50192.log.gz 89.3% -- replaced with /var/log/ceph/ceph-client.admin.74522.log.gz 2024-09-21T03:45:40.140 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.53303.log: 2024-09-21T03:45:40.141 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48500.log 2024-09-21T03:45:40.141 INFO:teuthology.orchestra.run.smithi038.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.53303.log.gz 2024-09-21T03:45:40.141 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71615.log 2024-09-21T03:45:40.141 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.52399.log: /var/log/ceph/ceph-client.admin.48500.log: gzip -5 87.0% -- replaced with /var/log/ceph/ceph-client.admin.52399.log.gz 2024-09-21T03:45:40.141 INFO:teuthology.orchestra.run.smithi038.stderr: --verbose -- /var/log/ceph/ceph-client.admin.43163.log 2024-09-21T03:45:40.141 INFO:teuthology.orchestra.run.smithi038.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48500.log.gz 2024-09-21T03:45:40.141 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.71615.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43035.log 2024-09-21T03:45:40.141 INFO:teuthology.orchestra.run.smithi038.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.71615.log.gz 2024-09-21T03:45:40.141 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.43163.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59998.log 2024-09-21T03:45:40.141 INFO:teuthology.orchestra.run.smithi038.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.43163.log.gz 2024-09-21T03:45:40.141 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.43035.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54185.log 2024-09-21T03:45:40.141 INFO:teuthology.orchestra.run.smithi038.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.43035.log.gz 2024-09-21T03:45:40.141 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.59998.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50351.log 2024-09-21T03:45:40.142 INFO:teuthology.orchestra.run.smithi038.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.59998.log.gz 2024-09-21T03:45:40.142 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.54185.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59830.log 2024-09-21T03:45:40.142 INFO:teuthology.orchestra.run.smithi038.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.54185.log.gz 2024-09-21T03:45:40.142 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.50351.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81935.log 2024-09-21T03:45:40.142 INFO:teuthology.orchestra.run.smithi038.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.50351.log.gz 2024-09-21T03:45:40.142 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.59830.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91668.log 2024-09-21T03:45:40.142 INFO:teuthology.orchestra.run.smithi038.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.59830.log.gz 2024-09-21T03:45:40.142 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.81935.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84982.log 2024-09-21T03:45:40.142 INFO:teuthology.orchestra.run.smithi038.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.81935.log.gz 2024-09-21T03:45:40.142 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.91668.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47602.log 2024-09-21T03:45:40.142 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.84982.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78143.log 2024-09-21T03:45:40.142 INFO:teuthology.orchestra.run.smithi038.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.91668.log.gz 2024-09-21T03:45:40.142 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.47602.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86399.log 2024-09-21T03:45:40.143 INFO:teuthology.orchestra.run.smithi038.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.84982.log.gz 2024-09-21T03:45:40.143 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.78143.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36477.log 2024-09-21T03:45:40.143 INFO:teuthology.orchestra.run.smithi038.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.47602.log.gz 2024-09-21T03:45:40.143 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73293.log 2024-09-21T03:45:40.143 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.86399.log: /var/log/ceph/ceph-client.admin.36477.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65987.log 2024-09-21T03:45:40.143 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.73293.log: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.36477.log.gz 2024-09-21T03:45:40.143 INFO:teuthology.orchestra.run.smithi038.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.86399.log.gz 2024-09-21T03:45:40.143 INFO:teuthology.orchestra.run.smithi038.stderr: 92.5% -- replaced with /var/log/ceph/ceph-client.admin.78143.log.gz 2024-09-21T03:45:40.143 INFO:teuthology.orchestra.run.smithi038.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.73293.log.gz 2024-09-21T03:45:40.143 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89863.log 2024-09-21T03:45:40.143 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.65987.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44959.log 2024-09-21T03:45:40.143 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.89863.log: 89.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52296.log 2024-09-21T03:45:40.143 INFO:teuthology.orchestra.run.smithi038.stderr: -- replaced with /var/log/ceph/ceph-client.admin.65987.log.gz 2024-09-21T03:45:40.143 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.44959.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77072.log 2024-09-21T03:45:40.144 INFO:teuthology.orchestra.run.smithi038.stderr: 87.3% 91.0% -- replaced with /var/log/ceph/ceph-client.admin.89863.log.gz 2024-09-21T03:45:40.144 INFO:teuthology.orchestra.run.smithi038.stderr: -- replaced with /var/log/ceph/ceph-client.admin.44959.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.0.67003.log 2024-09-21T03:45:40.144 INFO:teuthology.orchestra.run.smithi038.stderr: 2024-09-21T03:45:40.144 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.52296.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75774.log 2024-09-21T03:45:40.144 INFO:teuthology.orchestra.run.smithi038.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.52296.log.gz 2024-09-21T03:45:40.144 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.77072.log: /var/log/ceph/ceph-client.0.67003.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82083.log 2024-09-21T03:45:40.144 INFO:teuthology.orchestra.run.smithi038.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.67003.log.gz 2024-09-21T03:45:40.144 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.75774.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87700.log 2024-09-21T03:45:40.144 INFO:teuthology.orchestra.run.smithi038.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.77072.log.gz 2024-09-21T03:45:40.144 INFO:teuthology.orchestra.run.smithi038.stderr: 87.3%/var/log/ceph/ceph-client.admin.82083.log: -- replaced with /var/log/ceph/ceph-client.admin.75774.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71498.log 2024-09-21T03:45:40.144 INFO:teuthology.orchestra.run.smithi038.stderr: 2024-09-21T03:45:40.144 INFO:teuthology.orchestra.run.smithi038.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.82083.log.gz 2024-09-21T03:45:40.144 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.87700.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85378.log 2024-09-21T03:45:40.145 INFO:teuthology.orchestra.run.smithi038.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.87700.log.gz 2024-09-21T03:45:40.145 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66548.log 2024-09-21T03:45:40.145 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.71498.log: /var/log/ceph/ceph-client.admin.85378.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67207.log 2024-09-21T03:45:40.145 INFO:teuthology.orchestra.run.smithi038.stderr: 87.4% 87.5% -- replaced with /var/log/ceph/ceph-client.admin.71498.log.gz 2024-09-21T03:45:40.145 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.66548.log: -- replaced with /var/log/ceph/ceph-client.admin.85378.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70633.log 2024-09-21T03:45:40.145 INFO:teuthology.orchestra.run.smithi038.stderr: 2024-09-21T03:45:40.145 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.67207.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.66548.log.gz 2024-09-21T03:45:40.145 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46022.log 2024-09-21T03:45:40.145 INFO:teuthology.orchestra.run.smithi038.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.67207.log.gz 2024-09-21T03:45:40.145 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.70633.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60962.log 2024-09-21T03:45:40.145 INFO:teuthology.orchestra.run.smithi038.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.70633.log.gz 2024-09-21T03:45:40.145 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.46022.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85144.log 2024-09-21T03:45:40.145 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.60962.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-09-21T03:45:40.145 INFO:teuthology.orchestra.run.smithi038.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46022.log.gz 2024-09-21T03:45:40.146 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.85144.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86357.log 2024-09-21T03:45:40.146 INFO:teuthology.orchestra.run.smithi038.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.60962.log.gz 2024-09-21T03:45:40.146 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose/var/log/ceph/ceph-osd.2.log: -- /var/log/ceph/ceph-client.admin.58041.log 2024-09-21T03:45:40.146 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.86357.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87528.log 2024-09-21T03:45:40.146 INFO:teuthology.orchestra.run.smithi038.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.86357.log.gz 2024-09-21T03:45:40.146 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.58041.log: 92.5% -- replaced with /var/log/ceph/ceph-client.admin.85144.log.gz 2024-09-21T03:45:40.146 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66357.log 2024-09-21T03:45:40.146 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.87528.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.58041.log.gz 2024-09-21T03:45:40.146 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67703.log 2024-09-21T03:45:40.146 INFO:teuthology.orchestra.run.smithi038.stderr: 87.4%/var/log/ceph/ceph-client.admin.66357.log: -- replaced with /var/log/ceph/ceph-client.admin.87528.log.gzgzip 2024-09-21T03:45:40.146 INFO:teuthology.orchestra.run.smithi038.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.86586.log 2024-09-21T03:45:40.146 INFO:teuthology.orchestra.run.smithi038.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.66357.log.gz 2024-09-21T03:45:40.146 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.67703.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.67703.log.gz 2024-09-21T03:45:40.147 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78625.log 2024-09-21T03:45:40.147 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.86586.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.64445.log 2024-09-21T03:45:40.147 INFO:teuthology.orchestra.run.smithi038.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.86586.log.gz 2024-09-21T03:45:40.147 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.78625.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85290.log 2024-09-21T03:45:40.147 INFO:teuthology.orchestra.run.smithi038.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.78625.log.gz 2024-09-21T03:45:40.147 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.0.64445.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.64445.log.gz 2024-09-21T03:45:40.147 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45284.log 2024-09-21T03:45:40.147 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.85290.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53831.log 2024-09-21T03:45:40.147 INFO:teuthology.orchestra.run.smithi038.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.85290.log.gz/var/log/ceph/ceph-client.admin.45284.log: 2024-09-21T03:45:40.147 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42947.log 2024-09-21T03:45:40.147 INFO:teuthology.orchestra.run.smithi038.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.45284.log.gz 2024-09-21T03:45:40.147 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43119.log 2024-09-21T03:45:40.147 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.53831.log: /var/log/ceph/ceph-client.admin.42947.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51657.log 2024-09-21T03:45:40.148 INFO:teuthology.orchestra.run.smithi038.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.53831.log.gz/var/log/ceph/ceph-client.admin.43119.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.42947.log.gz 2024-09-21T03:45:40.148 INFO:teuthology.orchestra.run.smithi038.stderr: 2024-09-21T03:45:40.148 INFO:teuthology.orchestra.run.smithi038.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.43119.log.gz 2024-09-21T03:45:40.148 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80882.log 2024-09-21T03:45:40.148 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.51657.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85943.log 2024-09-21T03:45:40.148 INFO:teuthology.orchestra.run.smithi038.stderr: 89.3%/var/log/ceph/ceph-client.admin.80882.log: -- replaced with /var/log/ceph/ceph-client.admin.51657.log.gz 2024-09-21T03:45:40.148 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36815.log 2024-09-21T03:45:40.148 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.85943.log: -- /var/log/ceph/ceph-client.admin.37371.log 2024-09-21T03:45:40.148 INFO:teuthology.orchestra.run.smithi038.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.85943.log.gz 2024-09-21T03:45:40.148 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.36815.log: gzip -5 92.6% -- replaced with /var/log/ceph/ceph-client.admin.80882.log.gz 2024-09-21T03:45:40.148 INFO:teuthology.orchestra.run.smithi038.stderr: --verbose -- /var/log/ceph/ceph-client.0.81389.log 2024-09-21T03:45:40.148 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.37371.log: 85.8% -- replaced with /var/log/ceph/ceph-client.admin.36815.log.gz 2024-09-21T03:45:40.148 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45655.log 2024-09-21T03:45:40.149 INFO:teuthology.orchestra.run.smithi038.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.37371.log.gz 2024-09-21T03:45:40.149 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.0.81389.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.0.81389.log.gz 2024-09-21T03:45:40.149 INFO:teuthology.orchestra.run.smithi038.stderr: --verbose -- /var/log/ceph/ceph-client.admin.44408.log 2024-09-21T03:45:40.149 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.45655.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.83622.log 87.4% -- replaced with /var/log/ceph/ceph-client.admin.45655.log.gz 2024-09-21T03:45:40.149 INFO:teuthology.orchestra.run.smithi038.stderr: 2024-09-21T03:45:40.149 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.44408.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45938.log 87.5% -- replaced with /var/log/ceph/ceph-client.admin.44408.log.gz 2024-09-21T03:45:40.149 INFO:teuthology.orchestra.run.smithi038.stderr: 2024-09-21T03:45:40.149 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.0.83622.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.83622.log.gz 2024-09-21T03:45:40.149 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78373.log 2024-09-21T03:45:40.149 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.45938.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49679.log 2024-09-21T03:45:40.149 INFO:teuthology.orchestra.run.smithi038.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45938.log.gz/var/log/ceph/ceph-client.admin.78373.log: 2024-09-21T03:45:40.149 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- 87.1% -- replaced with /var/log/ceph/ceph-client.admin.78373.log.gz 2024-09-21T03:45:40.149 INFO:teuthology.orchestra.run.smithi038.stderr: /var/log/ceph/ceph-client.admin.63892.log 2024-09-21T03:45:40.150 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.49679.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56566.log 2024-09-21T03:45:40.150 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.63892.log: gzip 91.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.49679.log.gz --verbose 2024-09-21T03:45:40.150 INFO:teuthology.orchestra.run.smithi038.stderr: -- /var/log/ceph/ceph-client.admin.76372.log 2024-09-21T03:45:40.150 INFO:teuthology.orchestra.run.smithi038.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.63892.log.gz 2024-09-21T03:45:40.150 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.56566.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.56566.log.gz 2024-09-21T03:45:40.150 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53078.log 2024-09-21T03:45:40.150 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.76372.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84321.log 2024-09-21T03:45:40.150 INFO:teuthology.orchestra.run.smithi038.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.76372.log.gz 2024-09-21T03:45:40.150 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.53078.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89929.log 2024-09-21T03:45:40.150 INFO:teuthology.orchestra.run.smithi038.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.53078.log.gz 2024-09-21T03:45:40.150 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58144.log 2024-09-21T03:45:40.150 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.84321.log: /var/log/ceph/ceph-client.admin.89929.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70404.log 2024-09-21T03:45:40.150 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.58144.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.84321.log.gz 2024-09-21T03:45:40.151 INFO:teuthology.orchestra.run.smithi038.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.89929.log.gz 2024-09-21T03:45:40.151 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85495.log 2024-09-21T03:45:40.151 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88677.log 2024-09-21T03:45:40.151 INFO:teuthology.orchestra.run.smithi038.stderr: 92.3% -- replaced with /var/log/ceph/ceph-client.admin.58144.log.gz 2024-09-21T03:45:40.151 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.70404.log: /var/log/ceph/ceph-client.admin.85495.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.70404.log.gzgzip -5 --verbose -- 87.1% -- replaced with /var/log/ceph/ceph-client.admin.85495.log.gz 2024-09-21T03:45:40.151 INFO:teuthology.orchestra.run.smithi038.stderr: /var/log/ceph/ceph-client.admin.44492.log 2024-09-21T03:45:40.151 INFO:teuthology.orchestra.run.smithi038.stderr: 2024-09-21T03:45:40.151 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.88677.log: gzip -5 --verbose -- 87.4% -- replaced with /var/log/ceph/ceph-client.admin.88677.log.gz 2024-09-21T03:45:40.151 INFO:teuthology.orchestra.run.smithi038.stderr: /var/log/ceph/ceph-client.admin.79695.log 2024-09-21T03:45:40.151 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.44492.log: gzip -5 --verbose -- 87.0% -- replaced with /var/log/ceph/ceph-client.admin.44492.log.gz 2024-09-21T03:45:40.151 INFO:teuthology.orchestra.run.smithi038.stderr: /var/log/ceph/ceph-client.admin.87346.log 2024-09-21T03:45:40.151 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.79695.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70717.log 2024-09-21T03:45:40.151 INFO:teuthology.orchestra.run.smithi038.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.79695.log.gz 2024-09-21T03:45:40.152 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.87346.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77848.log 2024-09-21T03:45:40.152 INFO:teuthology.orchestra.run.smithi038.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.87346.log.gz 2024-09-21T03:45:40.152 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-09-21T03:45:40.152 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.70717.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47518.log 2024-09-21T03:45:40.152 INFO:teuthology.orchestra.run.smithi038.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.70717.log.gz/var/log/ceph/ceph-client.admin.77848.log: 2024-09-21T03:45:40.152 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-mgr.y.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.77848.log.gz 2024-09-21T03:45:40.152 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46106.log 2024-09-21T03:45:40.152 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.47518.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.47518.log.gz 2024-09-21T03:45:40.152 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81705.log 2024-09-21T03:45:40.152 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.46106.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91584.log 2024-09-21T03:45:40.152 INFO:teuthology.orchestra.run.smithi038.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.46106.log.gz 2024-09-21T03:45:40.152 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.81705.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55181.log 2024-09-21T03:45:40.152 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.91584.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.81705.log.gz 2024-09-21T03:45:40.152 INFO:teuthology.orchestra.run.smithi038.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.91584.log.gz 2024-09-21T03:45:40.153 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43289.log 2024-09-21T03:45:40.153 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.55181.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72845.log 2024-09-21T03:45:40.153 INFO:teuthology.orchestra.run.smithi038.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.55181.log.gz 2024-09-21T03:45:40.153 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.43289.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85724.log 2024-09-21T03:45:40.153 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.72845.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43289.log.gz 2024-09-21T03:45:40.153 INFO:teuthology.orchestra.run.smithi038.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.72845.log.gz 2024-09-21T03:45:40.153 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63976.log 2024-09-21T03:45:40.153 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.85724.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61943.log 2024-09-21T03:45:40.153 INFO:teuthology.orchestra.run.smithi038.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.85724.log.gz 2024-09-21T03:45:40.153 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.63976.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66703.log 2024-09-21T03:45:40.153 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.61943.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.63976.log.gz 2024-09-21T03:45:40.153 INFO:teuthology.orchestra.run.smithi038.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.61943.log.gz 2024-09-21T03:45:40.153 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44964.log 2024-09-21T03:45:40.154 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.66703.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85826.log 2024-09-21T03:45:40.154 INFO:teuthology.orchestra.run.smithi038.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.66703.log.gz 2024-09-21T03:45:40.154 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.44964.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55368.log 2024-09-21T03:45:40.154 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.85826.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.44964.log.gz 2024-09-21T03:45:40.154 INFO:teuthology.orchestra.run.smithi038.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.85826.log.gz 2024-09-21T03:45:40.154 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86543.log 2024-09-21T03:45:40.154 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.55368.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.66999.log 2024-09-21T03:45:40.154 INFO:teuthology.orchestra.run.smithi038.stderr: 94.4% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-09-21T03:45:40.154 INFO:teuthology.orchestra.run.smithi038.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.55368.log.gz 2024-09-21T03:45:40.154 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.86543.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61676.log 2024-09-21T03:45:40.154 INFO:teuthology.orchestra.run.smithi038.stderr: 87.6%/var/log/ceph/ceph-client.0.66999.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55879.log 2024-09-21T03:45:40.154 INFO:teuthology.orchestra.run.smithi038.stderr: -- replaced with /var/log/ceph/ceph-client.admin.86543.log.gz 2024-09-21T03:45:40.154 INFO:teuthology.orchestra.run.smithi038.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.66999.log.gz 2024-09-21T03:45:40.155 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.61676.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73256.log 2024-09-21T03:45:40.155 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.55879.log: gzip 87.0% -- replaced with /var/log/ceph/ceph-client.admin.61676.log.gz 2024-09-21T03:45:40.155 INFO:teuthology.orchestra.run.smithi038.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.87742.log 2024-09-21T03:45:40.155 INFO:teuthology.orchestra.run.smithi038.stderr: 87.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75127.log 2024-09-21T03:45:40.155 INFO:teuthology.orchestra.run.smithi038.stderr: -- replaced with /var/log/ceph/ceph-client.admin.55879.log.gz 2024-09-21T03:45:40.155 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.73256.log: /var/log/ceph/ceph-client.admin.87742.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78331.log 2024-09-21T03:45:40.155 INFO:teuthology.orchestra.run.smithi038.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.87742.log.gz 2024-09-21T03:45:40.155 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.75127.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.75127.log.gzgzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-21T03:45:40.155 INFO:teuthology.orchestra.run.smithi038.stderr: 2024-09-21T03:45:40.155 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.78331.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55265.log 2024-09-21T03:45:40.155 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.78331.log.gz 2024-09-21T03:45:40.155 INFO:teuthology.orchestra.run.smithi038.stderr: 92.5% -- replaced with /var/log/ceph/ceph-client.admin.73256.log.gz 2024-09-21T03:45:40.155 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69210.log 2024-09-21T03:45:40.155 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.55265.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.55265.log.gz 2024-09-21T03:45:40.156 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65321.log 2024-09-21T03:45:40.156 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.69210.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89748.log 2024-09-21T03:45:40.156 INFO:teuthology.orchestra.run.smithi038.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.69210.log.gz 2024-09-21T03:45:40.156 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.65321.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75211.log 2024-09-21T03:45:40.156 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.89748.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.65321.log.gz 2024-09-21T03:45:40.156 INFO:teuthology.orchestra.run.smithi038.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.89748.log.gz 2024-09-21T03:45:40.156 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76185.log 2024-09-21T03:45:40.156 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.75211.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73116.log 2024-09-21T03:45:40.156 INFO:teuthology.orchestra.run.smithi038.stderr: 87.1% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-21T03:45:40.156 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.76185.log: gzip 90.8% -- replaced with /var/log/ceph/ceph-client.admin.75211.log.gz 2024-09-21T03:45:40.156 INFO:teuthology.orchestra.run.smithi038.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.67974.log 2024-09-21T03:45:40.156 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.73116.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48210.log 2024-09-21T03:45:40.156 INFO:teuthology.orchestra.run.smithi038.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.73116.log.gz 2024-09-21T03:45:40.157 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.67974.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83088.log 2024-09-21T03:45:40.157 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.48210.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.67974.log.gz 2024-09-21T03:45:40.157 INFO:teuthology.orchestra.run.smithi038.stderr: 92.5% -- replaced with /var/log/ceph/ceph-client.admin.76185.log.gz 2024-09-21T03:45:40.157 INFO:teuthology.orchestra.run.smithi038.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.48210.log.gz 2024-09-21T03:45:40.157 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66209.log 2024-09-21T03:45:40.157 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.83088.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77554.log 2024-09-21T03:45:40.157 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.66209.log: gzip -5 87.4% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.83088.log.gz /var/log/ceph/ceph-client.admin.91416.log 2024-09-21T03:45:40.157 INFO:teuthology.orchestra.run.smithi038.stderr: 2024-09-21T03:45:40.157 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71338.log 2024-09-21T03:45:40.157 INFO:teuthology.orchestra.run.smithi038.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.66209.log.gz 2024-09-21T03:45:40.157 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.77554.log: /var/log/ceph/ceph-client.admin.91416.log: 87.2%gzip -5 87.2% -- replaced with /var/log/ceph/ceph-client.admin.91416.log.gz 2024-09-21T03:45:40.157 INFO:teuthology.orchestra.run.smithi038.stderr: --verbose -- /var/log/ceph/ceph-client.admin.36823.log 2024-09-21T03:45:40.157 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.71338.log: -- replaced with /var/log/ceph/ceph-client.admin.77554.log.gz 2024-09-21T03:45:40.157 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- 87.4% -- replaced with /var/log/ceph/ceph-client.admin.71338.log.gz 2024-09-21T03:45:40.158 INFO:teuthology.orchestra.run.smithi038.stderr: /var/log/ceph/ceph-client.admin.80741.log 2024-09-21T03:45:40.158 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.36823.log: gzip -5 --verbose -- 85.7% -- replaced with /var/log/ceph/ceph-client.admin.36823.log.gz 2024-09-21T03:45:40.158 INFO:teuthology.orchestra.run.smithi038.stderr: /var/log/ceph/ceph-client.admin.76956.log 2024-09-21T03:45:40.158 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.80741.log: gzip -5 --verbose -- 87.3% -- replaced with /var/log/ceph/ceph-client.admin.80741.log.gz 2024-09-21T03:45:40.158 INFO:teuthology.orchestra.run.smithi038.stderr: /var/log/ceph/ceph-client.admin.85901.log 2024-09-21T03:45:40.158 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.76956.log: gzip -5 --verbose 87.2% -- /var/log/ceph/ceph-client.admin.47224.log 2024-09-21T03:45:40.158 INFO:teuthology.orchestra.run.smithi038.stderr: -- replaced with /var/log/ceph/ceph-client.admin.76956.log.gz 2024-09-21T03:45:40.158 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.85901.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.75511.log 2024-09-21T03:45:40.158 INFO:teuthology.orchestra.run.smithi038.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.85901.log.gz 2024-09-21T03:45:40.158 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55620.log 2024-09-21T03:45:40.158 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.47224.log: /var/log/ceph/ceph-client.0.75511.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71245.log 2024-09-21T03:45:40.158 INFO:teuthology.orchestra.run.smithi038.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.75511.log.gz 2024-09-21T03:45:40.158 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.55620.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47224.log.gz 2024-09-21T03:45:40.159 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54546.log 2024-09-21T03:45:40.159 INFO:teuthology.orchestra.run.smithi038.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.55620.log.gz 2024-09-21T03:45:40.159 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.71245.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64624.log 2024-09-21T03:45:40.159 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.54546.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88839.log 2024-09-21T03:45:40.159 INFO:teuthology.orchestra.run.smithi038.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.54546.log.gz 2024-09-21T03:45:40.159 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.64624.log: gzip -5 92.6% -- replaced with /var/log/ceph/ceph-client.admin.71245.log.gz 2024-09-21T03:45:40.159 INFO:teuthology.orchestra.run.smithi038.stderr: --verbose -- /var/log/ceph/ceph-client.admin.54852.log 2024-09-21T03:45:40.159 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.88839.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.64624.log.gz 2024-09-21T03:45:40.159 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54744.log 2024-09-21T03:45:40.159 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81628.log 2024-09-21T03:45:40.159 INFO:teuthology.orchestra.run.smithi038.stderr: 91.8%/var/log/ceph/ceph-client.admin.54744.log: -- replaced with /var/log/ceph/ceph-client.admin.88839.log.gz 2024-09-21T03:45:40.159 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.54852.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89066.log 2024-09-21T03:45:40.159 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.81628.log: 91.0% 87.5% -- replaced with /var/log/ceph/ceph-client.admin.54852.log.gz 2024-09-21T03:45:40.160 INFO:teuthology.orchestra.run.smithi038.stderr: -- replaced with /var/log/ceph/ceph-client.admin.54744.log.gz 2024-09-21T03:45:40.160 INFO:teuthology.orchestra.run.smithi038.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.81628.log.gz 2024-09-21T03:45:40.160 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80975.log 2024-09-21T03:45:40.160 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55009.log 2024-09-21T03:45:40.160 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.89066.log: /var/log/ceph/ceph-client.admin.80975.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77755.log 2024-09-21T03:45:40.160 INFO:teuthology.orchestra.run.smithi038.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.89066.log.gz 2024-09-21T03:45:40.160 INFO:teuthology.orchestra.run.smithi038.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.80975.log.gz 2024-09-21T03:45:40.160 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.55009.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59535.log 2024-09-21T03:45:40.160 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.77755.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87433.log 2024-09-21T03:45:40.160 INFO:teuthology.orchestra.run.smithi038.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.55009.log.gz 2024-09-21T03:45:40.160 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.59535.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67838.log 2024-09-21T03:45:40.160 INFO:teuthology.orchestra.run.smithi038.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.59535.log.gz 2024-09-21T03:45:40.160 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.87433.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.87433.log.gz 2024-09-21T03:45:40.160 INFO:teuthology.orchestra.run.smithi038.stderr: 92.5% -- replaced with /var/log/ceph/ceph-client.admin.77755.log.gz 2024-09-21T03:45:40.161 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49426.log 2024-09-21T03:45:40.161 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85420.log 2024-09-21T03:45:40.161 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.67838.log: gzip/var/log/ceph/ceph-client.admin.49426.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.65136.log 2024-09-21T03:45:40.161 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.85420.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.49426.log.gz 2024-09-21T03:45:40.161 INFO:teuthology.orchestra.run.smithi038.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.67838.log.gz 2024-09-21T03:45:40.161 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 87.2% -- replaced with /var/log/ceph/ceph-client.admin.85420.log.gz 2024-09-21T03:45:40.161 INFO:teuthology.orchestra.run.smithi038.stderr: --verbose -- /var/log/ceph/ceph-client.admin.62675.log 2024-09-21T03:45:40.161 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.65136.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68750.log 2024-09-21T03:45:40.161 INFO:teuthology.orchestra.run.smithi038.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.65136.log.gz 2024-09-21T03:45:40.161 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.62675.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65950.log 2024-09-21T03:45:40.161 INFO:teuthology.orchestra.run.smithi038.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.62675.log.gz 2024-09-21T03:45:40.162 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36814.log 2024-09-21T03:45:40.162 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.68750.log: /var/log/ceph/ceph-client.admin.65950.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67329.log 2024-09-21T03:45:40.162 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.36814.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.68750.log.gz 89.1% -- replaced with /var/log/ceph/ceph-client.admin.65950.log.gz 2024-09-21T03:45:40.162 INFO:teuthology.orchestra.run.smithi038.stderr: 2024-09-21T03:45:40.162 INFO:teuthology.orchestra.run.smithi038.stderr: 85.7% -- replaced with /var/log/ceph/ceph-client.admin.36814.log.gz 2024-09-21T03:45:40.162 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48420.log 2024-09-21T03:45:40.162 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.67329.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62717.log 2024-09-21T03:45:40.162 INFO:teuthology.orchestra.run.smithi038.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.67329.log.gz 2024-09-21T03:45:40.162 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.48420.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89593.log 2024-09-21T03:45:40.162 INFO:teuthology.orchestra.run.smithi038.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.48420.log.gz 2024-09-21T03:45:40.163 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56668.log 2024-09-21T03:45:40.163 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.62717.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83364.log 2024-09-21T03:45:40.163 INFO:teuthology.orchestra.run.smithi038.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.62717.log.gz/var/log/ceph/ceph-client.admin.89593.log: 2024-09-21T03:45:40.163 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.56668.log: gzip 87.4% -- replaced with /var/log/ceph/ceph-client.admin.56668.log.gz 2024-09-21T03:45:40.163 INFO:teuthology.orchestra.run.smithi038.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.42685.log 2024-09-21T03:45:40.163 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.83364.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68259.log 2024-09-21T03:45:40.163 INFO:teuthology.orchestra.run.smithi038.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.89593.log.gz 2024-09-21T03:45:40.163 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61004.log 2024-09-21T03:45:40.163 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.42685.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.83364.log.gz 2024-09-21T03:45:40.163 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.68259.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64018.log 2024-09-21T03:45:40.163 INFO:teuthology.orchestra.run.smithi038.stderr: 87.5% 87.5% -- replaced with /var/log/ceph/ceph-client.admin.68259.log.gz 2024-09-21T03:45:40.163 INFO:teuthology.orchestra.run.smithi038.stderr: -- replaced with /var/log/ceph/ceph-client.admin.42685.log.gz 2024-09-21T03:45:40.164 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.61004.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54606.log 2024-09-21T03:45:40.164 INFO:teuthology.orchestra.run.smithi038.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.61004.log.gz 2024-09-21T03:45:40.164 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.64018.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86880.log 87.1% -- replaced with /var/log/ceph/ceph-client.admin.64018.log.gz 2024-09-21T03:45:40.164 INFO:teuthology.orchestra.run.smithi038.stderr: 2024-09-21T03:45:40.164 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.54606.log: gzip -5 --verbose -- 87.4% -- replaced with /var/log/ceph/ceph-client.admin.54606.log.gz 2024-09-21T03:45:40.164 INFO:teuthology.orchestra.run.smithi038.stderr: /var/log/ceph/ceph-client.admin.91458.log 2024-09-21T03:45:40.164 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.86880.log: gzip -5 --verbose -- 87.7% /var/log/ceph/ceph-client.admin.44189.log 2024-09-21T03:45:40.164 INFO:teuthology.orchestra.run.smithi038.stderr: -- replaced with /var/log/ceph/ceph-client.admin.86880.log.gz 2024-09-21T03:45:40.164 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.91458.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55223.log 2024-09-21T03:45:40.164 INFO:teuthology.orchestra.run.smithi038.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.91458.log.gz 2024-09-21T03:45:40.164 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49637.log 2024-09-21T03:45:40.164 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.44189.log: /var/log/ceph/ceph-client.admin.55223.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47881.log 2024-09-21T03:45:40.164 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.49637.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.55223.log.gz 2024-09-21T03:45:40.165 INFO:teuthology.orchestra.run.smithi038.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44189.log.gz 2024-09-21T03:45:40.165 INFO:teuthology.orchestra.run.smithi038.stderr: 87.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77732.log 2024-09-21T03:45:40.165 INFO:teuthology.orchestra.run.smithi038.stderr: -- replaced with /var/log/ceph/ceph-client.admin.49637.log.gz 2024-09-21T03:45:40.165 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.47881.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90418.log 2024-09-21T03:45:40.165 INFO:teuthology.orchestra.run.smithi038.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.47881.log.gz 2024-09-21T03:45:40.165 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86113.log 2024-09-21T03:45:40.165 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.77732.log: /var/log/ceph/ceph-client.admin.90418.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82158.log 2024-09-21T03:45:40.165 INFO:teuthology.orchestra.run.smithi038.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.77732.log.gz 2024-09-21T03:45:40.165 INFO:teuthology.orchestra.run.smithi038.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.90418.log.gz 2024-09-21T03:45:40.165 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.86113.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86628.log 2024-09-21T03:45:40.165 INFO:teuthology.orchestra.run.smithi038.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.86113.log.gz 2024-09-21T03:45:40.165 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.82158.log: -- /var/log/ceph/ceph-client.admin.36819.log 2024-09-21T03:45:40.166 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.86628.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.82158.log.gz 2024-09-21T03:45:40.166 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52992.log 2024-09-21T03:45:40.166 INFO:teuthology.orchestra.run.smithi038.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.86628.log.gz 2024-09-21T03:45:40.166 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44957.log 2024-09-21T03:45:40.166 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.36819.log: /var/log/ceph/ceph-client.admin.52992.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45304.log 2024-09-21T03:45:40.166 INFO:teuthology.orchestra.run.smithi038.stderr: 85.9%/var/log/ceph/ceph-client.admin.44957.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.52992.log.gz 2024-09-21T03:45:40.166 INFO:teuthology.orchestra.run.smithi038.stderr: -- replaced with /var/log/ceph/ceph-client.admin.36819.log.gz 2024-09-21T03:45:40.166 INFO:teuthology.orchestra.run.smithi038.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.44957.log.gz 2024-09-21T03:45:40.166 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50601.log 2024-09-21T03:45:40.166 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.45304.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74967.log 2024-09-21T03:45:40.166 INFO:teuthology.orchestra.run.smithi038.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.45304.log.gz 2024-09-21T03:45:40.166 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.50601.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80817.log 2024-09-21T03:45:40.166 INFO:teuthology.orchestra.run.smithi038.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.50601.log.gz 2024-09-21T03:45:40.167 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.74967.log: -- /var/log/ceph/ceph-client.admin.69648.log 2024-09-21T03:45:40.167 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.80817.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.74967.log.gz 2024-09-21T03:45:40.167 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50947.log 2024-09-21T03:45:40.167 INFO:teuthology.orchestra.run.smithi038.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.80817.log.gz 2024-09-21T03:45:40.167 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70488.log 2024-09-21T03:45:40.167 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.69648.log: /var/log/ceph/ceph-client.admin.50947.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56710.log 2024-09-21T03:45:40.167 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.70488.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.69648.log.gz 2024-09-21T03:45:40.167 INFO:teuthology.orchestra.run.smithi038.stderr:gzip 90.9% -- replaced with /var/log/ceph/ceph-client.admin.50947.log.gz 2024-09-21T03:45:40.167 INFO:teuthology.orchestra.run.smithi038.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.56482.log 2024-09-21T03:45:40.167 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73855.log 2024-09-21T03:45:40.167 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.56710.log: /var/log/ceph/ceph-client.admin.56482.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.56710.log.gz 2024-09-21T03:45:40.167 INFO:teuthology.orchestra.run.smithi038.stderr: 92.6% -- replaced with /var/log/ceph/ceph-client.admin.70488.log.gz 2024-09-21T03:45:40.167 INFO:teuthology.orchestra.run.smithi038.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.56482.log.gz 2024-09-21T03:45:40.168 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88978.log 2024-09-21T03:45:40.168 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.73855.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59316.log 2024-09-21T03:45:40.168 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.88978.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47560.log 2024-09-21T03:45:40.168 INFO:teuthology.orchestra.run.smithi038.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.73855.log.gz 2024-09-21T03:45:40.168 INFO:teuthology.orchestra.run.smithi038.stderr: 87.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50277.log 2024-09-21T03:45:40.168 INFO:teuthology.orchestra.run.smithi038.stderr: -- replaced with /var/log/ceph/ceph-client.admin.88978.log.gz 2024-09-21T03:45:40.168 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.59316.log: /var/log/ceph/ceph-client.admin.47560.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.75507.log 2024-09-21T03:45:40.168 INFO:teuthology.orchestra.run.smithi038.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.59316.log.gz 2024-09-21T03:45:40.168 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.50277.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.47560.log.gz 2024-09-21T03:45:40.168 INFO:teuthology.orchestra.run.smithi038.stderr:gzip 89.1% -- replaced with /var/log/ceph/ceph-client.admin.50277.log.gz 2024-09-21T03:45:40.168 INFO:teuthology.orchestra.run.smithi038.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.49117.log 2024-09-21T03:45:40.168 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.0.75507.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.75507.log.gz 2024-09-21T03:45:40.168 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63603.log 2024-09-21T03:45:40.169 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.49117.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61088.log 2024-09-21T03:45:40.169 INFO:teuthology.orchestra.run.smithi038.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49117.log.gz 2024-09-21T03:45:40.169 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.63603.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44534.log 2024-09-21T03:45:40.169 INFO:teuthology.orchestra.run.smithi038.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.63603.log.gz 2024-09-21T03:45:40.169 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55578.log 2024-09-21T03:45:40.169 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.61088.log: 87.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73467.log 2024-09-21T03:45:40.169 INFO:teuthology.orchestra.run.smithi038.stderr: -- replaced with /var/log/ceph/ceph-client.admin.61088.log.gz 2024-09-21T03:45:40.169 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.44534.log: /var/log/ceph/ceph-client.admin.55578.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55410.log 2024-09-21T03:45:40.169 INFO:teuthology.orchestra.run.smithi038.stderr: 87.4% 87.2% -- replaced with /var/log/ceph/ceph-client.admin.55578.log.gz/var/log/ceph/ceph-client.admin.73467.log: 2024-09-21T03:45:40.169 INFO:teuthology.orchestra.run.smithi038.stderr: -- replaced with /var/log/ceph/ceph-client.admin.44534.log.gz 2024-09-21T03:45:40.169 INFO:teuthology.orchestra.run.smithi038.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.73467.log.gz 2024-09-21T03:45:40.169 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72125.log 2024-09-21T03:45:40.169 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36781.log 2024-09-21T03:45:40.170 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.55410.log: gzip/var/log/ceph/ceph-client.admin.72125.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.53173.log 2024-09-21T03:45:40.170 INFO:teuthology.orchestra.run.smithi038.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.55410.log.gz 2024-09-21T03:45:40.170 INFO:teuthology.orchestra.run.smithi038.stderr: 89.3%/var/log/ceph/ceph-client.admin.36781.log: -- replaced with /var/log/ceph/ceph-client.admin.72125.log.gz 2024-09-21T03:45:40.170 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 85.7% -- replaced with /var/log/ceph/ceph-client.admin.36781.log.gz 2024-09-21T03:45:40.170 INFO:teuthology.orchestra.run.smithi038.stderr: --verbose -- /var/log/ceph/ceph-client.0.79043.log 2024-09-21T03:45:40.170 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.53173.log: gzip -5 --verbose -- 87.5% /var/log/ceph/ceph-client.admin.71456.log 2024-09-21T03:45:40.170 INFO:teuthology.orchestra.run.smithi038.stderr: -- replaced with /var/log/ceph/ceph-client.admin.53173.log.gz 2024-09-21T03:45:40.170 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.0.79043.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.79043.log.gz 2024-09-21T03:45:40.170 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76573.log 2024-09-21T03:45:40.170 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59619.log 2024-09-21T03:45:40.170 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.71456.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48708.log 2024-09-21T03:45:40.170 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.76573.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.71456.log.gz 2024-09-21T03:45:40.170 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.59619.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36811.log 2024-09-21T03:45:40.171 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.48708.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.59619.log.gz 2024-09-21T03:45:40.171 INFO:teuthology.orchestra.run.smithi038.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48708.log.gz 2024-09-21T03:45:40.171 INFO:teuthology.orchestra.run.smithi038.stderr: 92.5% -- replaced with /var/log/ceph/ceph-client.admin.76573.log.gzgzip 2024-09-21T03:45:40.171 INFO:teuthology.orchestra.run.smithi038.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.87784.log 2024-09-21T03:45:40.171 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83130.log 2024-09-21T03:45:40.171 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.36811.log: 91.0% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-09-21T03:45:40.171 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51713.log 2024-09-21T03:45:40.171 INFO:teuthology.orchestra.run.smithi038.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.36811.log.gz 2024-09-21T03:45:40.171 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.87784.log: /var/log/ceph/ceph-client.admin.83130.log: gzip -5 --verbose 87.1% -- replaced with /var/log/ceph/ceph-client.admin.87784.log.gz 2024-09-21T03:45:40.171 INFO:teuthology.orchestra.run.smithi038.stderr: -- /var/log/ceph/ceph-client.0.51577.log 2024-09-21T03:45:40.171 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.51713.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72065.log 87.2% -- replaced with /var/log/ceph/ceph-client.admin.51713.log.gz 2024-09-21T03:45:40.171 INFO:teuthology.orchestra.run.smithi038.stderr: 2024-09-21T03:45:40.171 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.0.51577.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.51577.log.gz 2024-09-21T03:45:40.171 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44324.log 2024-09-21T03:45:40.172 INFO:teuthology.orchestra.run.smithi038.stderr: 92.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84074.log 2024-09-21T03:45:40.172 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.72065.log: -- replaced with /var/log/ceph/ceph-client.admin.83130.log.gz 2024-09-21T03:45:40.172 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.44324.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.72065.log.gz 2024-09-21T03:45:40.172 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60836.log 2024-09-21T03:45:40.172 INFO:teuthology.orchestra.run.smithi038.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44324.log.gz 2024-09-21T03:45:40.172 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.72599.log 2024-09-21T03:45:40.172 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.84074.log: /var/log/ceph/ceph-client.admin.60836.log: gzip -5 87.5% -- replaced with /var/log/ceph/ceph-client.admin.84074.log.gz 2024-09-21T03:45:40.172 INFO:teuthology.orchestra.run.smithi038.stderr: --verbose -- /var/log/ceph/ceph-client.admin.74578.log 2024-09-21T03:45:40.172 INFO:teuthology.orchestra.run.smithi038.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.60836.log.gz 2024-09-21T03:45:40.172 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.0.72599.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.72599.log.gz 2024-09-21T03:45:40.172 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77942.log 2024-09-21T03:45:40.172 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.74578.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86712.log 2024-09-21T03:45:40.172 INFO:teuthology.orchestra.run.smithi038.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.74578.log.gz 2024-09-21T03:45:40.173 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.77942.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68437.log 2024-09-21T03:45:40.173 INFO:teuthology.orchestra.run.smithi038.stderr: 87.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91542.log 2024-09-21T03:45:40.173 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.86712.log: -- replaced with /var/log/ceph/ceph-client.admin.77942.log.gz 2024-09-21T03:45:40.173 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.68437.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.86712.log.gz 2024-09-21T03:45:40.173 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52567.log 2024-09-21T03:45:40.173 INFO:teuthology.orchestra.run.smithi038.stderr: 87.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76330.log 2024-09-21T03:45:40.173 INFO:teuthology.orchestra.run.smithi038.stderr: -- replaced with /var/log/ceph/ceph-client.admin.68437.log.gz 2024-09-21T03:45:40.173 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.91542.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71699.log 2024-09-21T03:45:40.173 INFO:teuthology.orchestra.run.smithi038.stderr: 87.1%/var/log/ceph/ceph-client.admin.52567.log: -- replaced with /var/log/ceph/ceph-client.admin.91542.log.gz 2024-09-21T03:45:40.173 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44660.log 2024-09-21T03:45:40.173 INFO:teuthology.orchestra.run.smithi038.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.52567.log.gz 2024-09-21T03:45:40.173 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.76330.log: /var/log/ceph/ceph-client.admin.71699.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56356.log 2024-09-21T03:45:40.173 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74809.log 2024-09-21T03:45:40.174 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.44660.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.76330.log.gz 2024-09-21T03:45:40.174 INFO:teuthology.orchestra.run.smithi038.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44660.log.gz 2024-09-21T03:45:40.174 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.56356.log: gzip -5 87.5% --verbose -- /var/log/ceph/ceph-client.admin.45658.log 2024-09-21T03:45:40.174 INFO:teuthology.orchestra.run.smithi038.stderr: -- replaced with /var/log/ceph/ceph-client.admin.56356.log.gz 2024-09-21T03:45:40.174 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.74809.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63561.log 2024-09-21T03:45:40.174 INFO:teuthology.orchestra.run.smithi038.stderr: 86.9% 92.5% -- replaced with /var/log/ceph/ceph-client.admin.71699.log.gz 2024-09-21T03:45:40.174 INFO:teuthology.orchestra.run.smithi038.stderr: -- replaced with /var/log/ceph/ceph-client.admin.74809.log.gz/var/log/ceph/ceph-client.admin.45658.log: 2024-09-21T03:45:40.174 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55139.log 2024-09-21T03:45:40.174 INFO:teuthology.orchestra.run.smithi038.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45658.log.gz 2024-09-21T03:45:40.174 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56608.log 2024-09-21T03:45:40.174 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.63561.log: /var/log/ceph/ceph-client.admin.55139.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68502.log 2024-09-21T03:45:40.174 INFO:teuthology.orchestra.run.smithi038.stderr: 87.5%/var/log/ceph/ceph-client.admin.56608.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76003.log 2024-09-21T03:45:40.174 INFO:teuthology.orchestra.run.smithi038.stderr: -- replaced with /var/log/ceph/ceph-client.admin.55139.log.gz 2024-09-21T03:45:40.174 INFO:teuthology.orchestra.run.smithi038.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.63561.log.gz 2024-09-21T03:45:40.175 INFO:teuthology.orchestra.run.smithi038.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.56608.log.gz 2024-09-21T03:45:40.175 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.68502.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69037.log 2024-09-21T03:45:40.175 INFO:teuthology.orchestra.run.smithi038.stderr:gzip/var/log/ceph/ceph-client.admin.76003.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.85966.log 2024-09-21T03:45:40.175 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.69037.log: gzip -5 87.3% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.76003.log.gz 2024-09-21T03:45:40.175 INFO:teuthology.orchestra.run.smithi038.stderr: /var/log/ceph/ceph-client.admin.62759.log 2024-09-21T03:45:40.175 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65876.log 2024-09-21T03:45:40.175 INFO:teuthology.orchestra.run.smithi038.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.69037.log.gz 2024-09-21T03:45:40.175 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.85966.log: 92.5% -- replaced with /var/log/ceph/ceph-client.admin.68502.log.gz 2024-09-21T03:45:40.175 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.62759.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61550.log 2024-09-21T03:45:40.175 INFO:teuthology.orchestra.run.smithi038.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.62759.log.gz 2024-09-21T03:45:40.175 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37170.log 2024-09-21T03:45:40.175 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.65876.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70792.log 2024-09-21T03:45:40.175 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.61550.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.65876.log.gz 2024-09-21T03:45:40.176 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.37170.log: 92.6% -- replaced with /var/log/ceph/ceph-client.admin.85966.log.gz 87.4% 2024-09-21T03:45:40.176 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36327.log 2024-09-21T03:45:40.176 INFO:teuthology.orchestra.run.smithi038.stderr: -- replaced with /var/log/ceph/ceph-client.admin.61550.log.gz 2024-09-21T03:45:40.176 INFO:teuthology.orchestra.run.smithi038.stderr: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.37170.log.gz 2024-09-21T03:45:40.176 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5/var/log/ceph/ceph-client.admin.70792.log: --verbose -- /var/log/ceph/ceph-client.admin.45518.log 2024-09-21T03:45:40.176 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.36327.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.70792.log.gz 2024-09-21T03:45:40.176 INFO:teuthology.orchestra.run.smithi038.stderr:gzip 77.9% -- replaced with /var/log/ceph/ceph-client.admin.36327.log.gz 2024-09-21T03:45:40.176 INFO:teuthology.orchestra.run.smithi038.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.64743.log 2024-09-21T03:45:40.176 INFO:teuthology.orchestra.run.smithi038.stderr:gzip/var/log/ceph/ceph-client.admin.45518.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.65617.log 2024-09-21T03:45:40.176 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.64743.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.45518.log.gz 2024-09-21T03:45:40.176 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.2.47087.log 2024-09-21T03:45:40.176 INFO:teuthology.orchestra.run.smithi038.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.64743.log.gz 2024-09-21T03:45:40.176 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.65617.log: gzip -5 --verbose -- 89.3% -- replaced with /var/log/ceph/ceph-client.admin.65617.log.gz 2024-09-21T03:45:40.177 INFO:teuthology.orchestra.run.smithi038.stderr: /var/log/ceph/ceph-client.admin.77166.log 2024-09-21T03:45:40.177 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.2.47087.log: 0.0% -- replaced with /var/log/ceph/ceph-client.2.47087.log.gz 2024-09-21T03:45:40.177 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74673.log 2024-09-21T03:45:40.177 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.77166.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66604.log 2024-09-21T03:45:40.177 INFO:teuthology.orchestra.run.smithi038.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.77166.log.gz 2024-09-21T03:45:40.177 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.74673.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.84855.log 2024-09-21T03:45:40.177 INFO:teuthology.orchestra.run.smithi038.stderr: 87.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54400.log 2024-09-21T03:45:40.177 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.66604.log: -- replaced with /var/log/ceph/ceph-client.admin.74673.log.gz 2024-09-21T03:45:40.177 INFO:teuthology.orchestra.run.smithi038.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.66604.log.gz 2024-09-21T03:45:40.177 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.0.84855.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61172.log 2024-09-21T03:45:40.177 INFO:teuthology.orchestra.run.smithi038.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.84855.log.gz 2024-09-21T03:45:40.177 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.54400.log: -- /var/log/ceph/ceph-client.admin.79965.log 2024-09-21T03:45:40.177 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.61172.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.54400.log.gz 2024-09-21T03:45:40.178 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62801.log 2024-09-21T03:45:40.178 INFO:teuthology.orchestra.run.smithi038.stderr:gzip/var/log/ceph/ceph-client.admin.79965.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.84509.log 2024-09-21T03:45:40.178 INFO:teuthology.orchestra.run.smithi038.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.61172.log.gz 2024-09-21T03:45:40.178 INFO:teuthology.orchestra.run.smithi038.stderr: 87.6%/var/log/ceph/ceph-client.admin.62801.log: -- replaced with /var/log/ceph/ceph-client.admin.79965.log.gz 2024-09-21T03:45:40.178 INFO:teuthology.orchestra.run.smithi038.stderr:gzip 87.2% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.62801.log.gz -- 2024-09-21T03:45:40.178 INFO:teuthology.orchestra.run.smithi038.stderr: /var/log/ceph/ceph-client.admin.53034.log 2024-09-21T03:45:40.178 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45854.log 2024-09-21T03:45:40.178 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.84509.log: /var/log/ceph/ceph-client.admin.53034.log: 87.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77460.log 2024-09-21T03:45:40.178 INFO:teuthology.orchestra.run.smithi038.stderr: -- replaced with /var/log/ceph/ceph-client.admin.84509.log.gz 2024-09-21T03:45:40.178 INFO:teuthology.orchestra.run.smithi038.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.53034.log.gz 2024-09-21T03:45:40.178 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.45854.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63263.log 2024-09-21T03:45:40.178 INFO:teuthology.orchestra.run.smithi038.stderr: 87.1%/var/log/ceph/ceph-client.admin.77460.log: -- replaced with /var/log/ceph/ceph-client.admin.45854.log.gz 2024-09-21T03:45:40.178 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77016.log 2024-09-21T03:45:40.179 INFO:teuthology.orchestra.run.smithi038.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.77460.log.gz 2024-09-21T03:45:40.179 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.63263.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55536.log 2024-09-21T03:45:40.179 INFO:teuthology.orchestra.run.smithi038.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.63263.log.gz 2024-09-21T03:45:40.179 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.77016.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48168.log 2024-09-21T03:45:40.179 INFO:teuthology.orchestra.run.smithi038.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.77016.log.gz 2024-09-21T03:45:40.179 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.55536.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65580.log 2024-09-21T03:45:40.179 INFO:teuthology.orchestra.run.smithi038.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.55536.log.gz 2024-09-21T03:45:40.179 INFO:teuthology.orchestra.run.smithi038.stderr:gzip/var/log/ceph/ceph-client.admin.48168.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.53765.log 2024-09-21T03:45:40.179 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.65580.log: gzip 87.8% -5 --verbose -- /var/log/ceph/ceph-client.admin.63221.log 2024-09-21T03:45:40.179 INFO:teuthology.orchestra.run.smithi038.stderr: -- replaced with /var/log/ceph/ceph-client.admin.48168.log.gz 2024-09-21T03:45:40.179 INFO:teuthology.orchestra.run.smithi038.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.65580.log.gz 2024-09-21T03:45:40.179 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.53765.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44786.log 2024-09-21T03:45:40.179 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.63221.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63765.log 2024-09-21T03:45:40.179 INFO:teuthology.orchestra.run.smithi038.stderr: 90.8%/var/log/ceph/ceph-client.admin.44786.log: -- replaced with /var/log/ceph/ceph-client.admin.53765.log.gz 87.5% -- replaced with /var/log/ceph/ceph-client.admin.63221.log.gz 2024-09-21T03:45:40.180 INFO:teuthology.orchestra.run.smithi038.stderr: 2024-09-21T03:45:40.180 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.59099.log 2024-09-21T03:45:40.180 INFO:teuthology.orchestra.run.smithi038.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44786.log.gz 2024-09-21T03:45:40.180 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.63765.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75169.log 2024-09-21T03:45:40.180 INFO:teuthology.orchestra.run.smithi038.stderr: 87.4%/var/log/ceph/ceph-client.0.59099.log: -- replaced with /var/log/ceph/ceph-client.admin.63765.log.gz 2024-09-21T03:45:40.180 INFO:teuthology.orchestra.run.smithi038.stderr:gzip 0.0% -- replaced with /var/log/ceph/ceph-client.0.59099.log.gz 2024-09-21T03:45:40.180 INFO:teuthology.orchestra.run.smithi038.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.50447.log 2024-09-21T03:45:40.180 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.75169.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60082.log 2024-09-21T03:45:40.180 INFO:teuthology.orchestra.run.smithi038.stderr: 87.4%/var/log/ceph/ceph-client.admin.50447.log: -- replaced with /var/log/ceph/ceph-client.admin.75169.log.gz 2024-09-21T03:45:40.180 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85616.log 2024-09-21T03:45:40.180 INFO:teuthology.orchestra.run.smithi038.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.50447.log.gz 2024-09-21T03:45:40.180 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.60082.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50535.log 2024-09-21T03:45:40.180 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.85616.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.60082.log.gz 2024-09-21T03:45:40.180 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81488.log 2024-09-21T03:45:40.180 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.50535.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.85616.log.gz 2024-09-21T03:45:40.181 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75277.log 2024-09-21T03:45:40.181 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5/var/log/ceph/ceph-client.admin.81488.log: --verbose -- /var/log/ceph/ceph-client.admin.45281.log 2024-09-21T03:45:40.181 INFO:teuthology.orchestra.run.smithi038.stderr: 91.0%/var/log/ceph/ceph-client.admin.75277.log: -- replaced with /var/log/ceph/ceph-client.admin.50535.log.gz 2024-09-21T03:45:40.181 INFO:teuthology.orchestra.run.smithi038.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.81488.log.gz 2024-09-21T03:45:40.181 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70950.log 87.6% -- replaced with /var/log/ceph/ceph-client.admin.75277.log.gz 2024-09-21T03:45:40.181 INFO:teuthology.orchestra.run.smithi038.stderr: 2024-09-21T03:45:40.181 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.45281.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43205.log 2024-09-21T03:45:40.181 INFO:teuthology.orchestra.run.smithi038.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.45281.log.gz 2024-09-21T03:45:40.181 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.70950.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61130.log 87.3% -- replaced with /var/log/ceph/ceph-client.admin.70950.log.gz 2024-09-21T03:45:40.181 INFO:teuthology.orchestra.run.smithi038.stderr: 2024-09-21T03:45:40.181 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.43205.log: gzip -5 --verbose -- 87.4% -- replaced with /var/log/ceph/ceph-client.admin.43205.log.gz 2024-09-21T03:45:40.181 INFO:teuthology.orchestra.run.smithi038.stderr: /var/log/ceph/ceph-client.admin.82041.log 2024-09-21T03:45:40.181 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.61130.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44247.log 2024-09-21T03:45:40.181 INFO:teuthology.orchestra.run.smithi038.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.61130.log.gz 2024-09-21T03:45:40.181 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.82041.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69102.log 2024-09-21T03:45:40.182 INFO:teuthology.orchestra.run.smithi038.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.82041.log.gz 2024-09-21T03:45:40.182 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.44247.log: gzip -5 --verbose 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44247.log.gz 2024-09-21T03:45:40.182 INFO:teuthology.orchestra.run.smithi038.stderr: -- /var/log/ceph/ceph-client.admin.36813.log 2024-09-21T03:45:40.182 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.69102.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49205.log 2024-09-21T03:45:40.182 INFO:teuthology.orchestra.run.smithi038.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.69102.log.gz 2024-09-21T03:45:40.182 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.36813.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53345.log 2024-09-21T03:45:40.182 INFO:teuthology.orchestra.run.smithi038.stderr: 86.0%/var/log/ceph/ceph-client.admin.49205.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71002.log 2024-09-21T03:45:40.182 INFO:teuthology.orchestra.run.smithi038.stderr: -- replaced with /var/log/ceph/ceph-client.admin.36813.log.gz 2024-09-21T03:45:40.182 INFO:teuthology.orchestra.run.smithi038.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.49205.log.gz 2024-09-21T03:45:40.182 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.53345.log: /var/log/ceph/ceph-client.0.70196.log 2024-09-21T03:45:40.182 INFO:teuthology.orchestra.run.smithi038.stderr:gzip 87.0% -- replaced with /var/log/ceph/ceph-client.admin.53345.log.gz 2024-09-21T03:45:40.182 INFO:teuthology.orchestra.run.smithi038.stderr: -5 --verbose --/var/log/ceph/ceph-client.admin.71002.log: /var/log/ceph/ceph-client.admin.37130.log 2024-09-21T03:45:40.182 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.0.70196.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.70196.log.gz 2024-09-21T03:45:40.182 INFO:teuthology.orchestra.run.smithi038.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.71002.log.gz 2024-09-21T03:45:40.183 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82476.log 2024-09-21T03:45:40.183 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49384.log 2024-09-21T03:45:40.183 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.37130.log: /var/log/ceph/ceph-client.admin.82476.log: gzip 86.7% -- replaced with /var/log/ceph/ceph-client.admin.37130.log.gz 2024-09-21T03:45:40.183 INFO:teuthology.orchestra.run.smithi038.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.45279.log 2024-09-21T03:45:40.183 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.49384.log: gzip 91.0% -5 --verbose -- /var/log/ceph/ceph-client.admin.52651.log 2024-09-21T03:45:40.183 INFO:teuthology.orchestra.run.smithi038.stderr: -- replaced with /var/log/ceph/ceph-client.admin.82476.log.gz 2024-09-21T03:45:40.183 INFO:teuthology.orchestra.run.smithi038.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.49384.log.gz 2024-09-21T03:45:40.183 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.45279.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91710.log 2024-09-21T03:45:40.183 INFO:teuthology.orchestra.run.smithi038.stderr: 87.4%/var/log/ceph/ceph-client.admin.52651.log: -- replaced with /var/log/ceph/ceph-client.admin.45279.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52483.log 2024-09-21T03:45:40.183 INFO:teuthology.orchestra.run.smithi038.stderr: 2024-09-21T03:45:40.183 INFO:teuthology.orchestra.run.smithi038.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.52651.log.gz 2024-09-21T03:45:40.183 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.91710.log: /var/log/ceph/ceph-client.admin.68151.log 2024-09-21T03:45:40.183 INFO:teuthology.orchestra.run.smithi038.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.91710.log.gz 2024-09-21T03:45:40.183 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65025.log 2024-09-21T03:45:40.184 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.52483.log: /var/log/ceph/ceph-client.admin.68151.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48252.log 2024-09-21T03:45:40.184 INFO:teuthology.orchestra.run.smithi038.stderr: 87.3%/var/log/ceph/ceph-client.admin.65025.log: 89.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65691.log 2024-09-21T03:45:40.184 INFO:teuthology.orchestra.run.smithi038.stderr: -- replaced with /var/log/ceph/ceph-client.admin.68151.log.gz 2024-09-21T03:45:40.184 INFO:teuthology.orchestra.run.smithi038.stderr: -- replaced with /var/log/ceph/ceph-client.admin.52483.log.gz 2024-09-21T03:45:40.184 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.48252.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.65025.log.gz 2024-09-21T03:45:40.184 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.89366.log 2024-09-21T03:45:40.184 INFO:teuthology.orchestra.run.smithi038.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.48252.log.gz 2024-09-21T03:45:40.184 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.65691.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74056.log 2024-09-21T03:45:40.184 INFO:teuthology.orchestra.run.smithi038.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.65691.log.gz 2024-09-21T03:45:40.184 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79578.log 2024-09-21T03:45:40.184 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.0.89366.log: /var/log/ceph/ceph-client.admin.74056.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.89366.log.gz 2024-09-21T03:45:40.184 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63438.log 2024-09-21T03:45:40.184 INFO:teuthology.orchestra.run.smithi038.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.74056.log.gz 2024-09-21T03:45:40.184 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.79578.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71888.log 2024-09-21T03:45:40.185 INFO:teuthology.orchestra.run.smithi038.stderr: 87.5%/var/log/ceph/ceph-client.admin.63438.log: -- replaced with /var/log/ceph/ceph-client.admin.79578.log.gz 2024-09-21T03:45:40.185 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57308.log 2024-09-21T03:45:40.185 INFO:teuthology.orchestra.run.smithi038.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.63438.log.gz 2024-09-21T03:45:40.185 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.71888.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91092.log 2024-09-21T03:45:40.185 INFO:teuthology.orchestra.run.smithi038.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.71888.log.gz 2024-09-21T03:45:40.185 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58703.log 2024-09-21T03:45:40.185 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.57308.log: /var/log/ceph/ceph-client.admin.91092.log: gzip -5 87.2% -- replaced with /var/log/ceph/ceph-client.admin.57308.log.gz 2024-09-21T03:45:40.185 INFO:teuthology.orchestra.run.smithi038.stderr: --verbose -- /var/log/ceph/ceph-client.admin.77792.log 2024-09-21T03:45:40.185 INFO:teuthology.orchestra.run.smithi038.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.91092.log.gz 2024-09-21T03:45:40.185 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.58703.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43373.log 87.4% -- replaced with /var/log/ceph/ceph-client.admin.58703.log.gz 2024-09-21T03:45:40.185 INFO:teuthology.orchestra.run.smithi038.stderr: 2024-09-21T03:45:40.185 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.77792.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57165.log 2024-09-21T03:45:40.185 INFO:teuthology.orchestra.run.smithi038.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.77792.log.gz 2024-09-21T03:45:40.186 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.43373.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82279.log 2024-09-21T03:45:40.186 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.57165.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43373.log.gz 2024-09-21T03:45:40.186 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58293.log 2024-09-21T03:45:40.186 INFO:teuthology.orchestra.run.smithi038.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.57165.log.gz 2024-09-21T03:45:40.186 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63850.log 2024-09-21T03:45:40.186 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.82279.log: /var/log/ceph/ceph-client.admin.58293.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65099.log 2024-09-21T03:45:40.186 INFO:teuthology.orchestra.run.smithi038.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.82279.log.gz 2024-09-21T03:45:40.186 INFO:teuthology.orchestra.run.smithi038.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.58293.log.gz 2024-09-21T03:45:40.186 INFO:teuthology.orchestra.run.smithi038.stderr:gzip/var/log/ceph/ceph-client.admin.63850.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.83910.log 2024-09-21T03:45:40.186 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.65099.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.63850.log.gz 2024-09-21T03:45:40.186 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79494.log 2024-09-21T03:45:40.186 INFO:teuthology.orchestra.run.smithi038.stderr: 89.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90502.log 2024-09-21T03:45:40.186 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.83910.log: -- replaced with /var/log/ceph/ceph-client.admin.65099.log.gz 2024-09-21T03:45:40.187 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.79494.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71086.log 2024-09-21T03:45:40.187 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50493.log 2024-09-21T03:45:40.187 INFO:teuthology.orchestra.run.smithi038.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.79494.log.gz 2024-09-21T03:45:40.187 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.90502.log: gzip/var/log/ceph/ceph-client.admin.71086.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.48622.log 2024-09-21T03:45:40.187 INFO:teuthology.orchestra.run.smithi038.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.71086.log.gz/var/log/ceph/ceph-client.admin.50493.log: 2024-09-21T03:45:40.187 INFO:teuthology.orchestra.run.smithi038.stderr: 92.5% -- replaced with /var/log/ceph/ceph-client.admin.83910.log.gz 2024-09-21T03:45:40.187 INFO:teuthology.orchestra.run.smithi038.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.90502.log.gz 2024-09-21T03:45:40.187 INFO:teuthology.orchestra.run.smithi038.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.50493.log.gz 2024-09-21T03:45:40.187 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45653.log 2024-09-21T03:45:40.187 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.88392.log 2024-09-21T03:45:40.187 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.48622.log: /var/log/ceph/ceph-client.admin.45653.log: 87.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.0.69398.log 2024-09-21T03:45:40.188 INFO:teuthology.orchestra.run.smithi038.stderr: -- replaced with /var/log/ceph/ceph-client.admin.48622.log.gz 2024-09-21T03:45:40.188 INFO:teuthology.orchestra.run.smithi038.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45653.log.gz 2024-09-21T03:45:40.188 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.0.88392.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64766.log 2024-09-21T03:45:40.188 INFO:teuthology.orchestra.run.smithi038.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.88392.log.gz 2024-09-21T03:45:40.188 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.0.69398.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59746.log 2024-09-21T03:45:40.188 INFO:teuthology.orchestra.run.smithi038.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.69398.log.gz 2024-09-21T03:45:40.188 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.64766.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70894.log 2024-09-21T03:45:40.188 INFO:teuthology.orchestra.run.smithi038.stderr:gzip/var/log/ceph/ceph-client.admin.59746.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.50405.log 2024-09-21T03:45:40.188 INFO:teuthology.orchestra.run.smithi038.stderr: 87.4%/var/log/ceph/ceph-client.admin.70894.log: -- replaced with /var/log/ceph/ceph-client.admin.59746.log.gz 2024-09-21T03:45:40.188 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56133.log 2024-09-21T03:45:40.188 INFO:teuthology.orchestra.run.smithi038.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.70894.log.gz 2024-09-21T03:45:40.188 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79331.log 2024-09-21T03:45:40.188 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.50405.log: 92.9% -- replaced with /var/log/ceph/ceph-client.admin.64766.log.gz 2024-09-21T03:45:40.189 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.56133.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76162.log 2024-09-21T03:45:40.189 INFO:teuthology.orchestra.run.smithi038.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.50405.log.gz 86.9% 2024-09-21T03:45:40.189 INFO:teuthology.orchestra.run.smithi038.stderr: -- replaced with /var/log/ceph/ceph-client.admin.56133.log.gz 2024-09-21T03:45:40.189 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.79331.log: -- /var/log/ceph/ceph-client.admin.63649.log 2024-09-21T03:45:40.189 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.76162.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87970.log 87.4% 2024-09-21T03:45:40.189 INFO:teuthology.orchestra.run.smithi038.stderr: -- replaced with /var/log/ceph/ceph-client.admin.76162.log.gz 2024-09-21T03:45:40.189 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.63649.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72407.log 2024-09-21T03:45:40.189 INFO:teuthology.orchestra.run.smithi038.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.63649.log.gz 2024-09-21T03:45:40.189 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.87970.log: 92.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43331.log 2024-09-21T03:45:40.189 INFO:teuthology.orchestra.run.smithi038.stderr: -- replaced with /var/log/ceph/ceph-client.admin.79331.log.gz 2024-09-21T03:45:40.189 INFO:teuthology.orchestra.run.smithi038.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.87970.log.gz 2024-09-21T03:45:40.189 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.72407.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87223.log 2024-09-21T03:45:40.189 INFO:teuthology.orchestra.run.smithi038.stderr:gzip 87.4% -- replaced with /var/log/ceph/ceph-client.admin.72407.log.gz 2024-09-21T03:45:40.190 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.43331.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.85060.log 2024-09-21T03:45:40.190 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.87223.log: gzip 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43331.log.gz 2024-09-21T03:45:40.190 INFO:teuthology.orchestra.run.smithi038.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.47713.log 2024-09-21T03:45:40.190 INFO:teuthology.orchestra.run.smithi038.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.87223.log.gz 2024-09-21T03:45:40.190 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.85060.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80859.log 2024-09-21T03:45:40.190 INFO:teuthology.orchestra.run.smithi038.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.85060.log.gz 2024-09-21T03:45:40.190 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.47713.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90647.log 2024-09-21T03:45:40.190 INFO:teuthology.orchestra.run.smithi038.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.47713.log.gz 2024-09-21T03:45:40.190 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.80859.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43963.log 2024-09-21T03:45:40.190 INFO:teuthology.orchestra.run.smithi038.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.80859.log.gz 2024-09-21T03:45:40.190 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.90647.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65765.log 2024-09-21T03:45:40.190 INFO:teuthology.orchestra.run.smithi038.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.90647.log.gz 2024-09-21T03:45:40.191 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.43963.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65284.log 2024-09-21T03:45:40.191 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.65765.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.51581.log 2024-09-21T03:45:40.191 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.65284.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.65765.log.gz 2024-09-21T03:45:40.191 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-09-21T03:45:40.191 INFO:teuthology.orchestra.run.smithi038.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.65284.log.gz/var/log/ceph/ceph-client.0.51581.log: 2024-09-21T03:45:40.191 INFO:teuthology.orchestra.run.smithi038.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.51581.log.gz 2024-09-21T03:45:40.191 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87612.log 2024-09-21T03:45:40.191 INFO:teuthology.orchestra.run.smithi038.stderr: 92.3% -- replaced with /var/log/ceph/ceph-client.admin.43963.log.gz/var/log/ceph/ceph-osd.3.log: 2024-09-21T03:45:40.191 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50905.log 2024-09-21T03:45:40.191 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.87612.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68690.log 2024-09-21T03:45:40.191 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.50905.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75961.log 2024-09-21T03:45:40.191 INFO:teuthology.orchestra.run.smithi038.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.87612.log.gz 2024-09-21T03:45:40.192 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67292.log 2024-09-21T03:45:40.192 INFO:teuthology.orchestra.run.smithi038.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.50905.log.gz 2024-09-21T03:45:40.192 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.68690.log: /var/log/ceph/ceph-client.admin.75961.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.68690.log.gz 2024-09-21T03:45:40.192 INFO:teuthology.orchestra.run.smithi038.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.75961.log.gz 2024-09-21T03:45:40.192 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36356.log 2024-09-21T03:45:40.192 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.67292.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87304.log 2024-09-21T03:45:40.192 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.36356.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64102.log 2024-09-21T03:45:40.192 INFO:teuthology.orchestra.run.smithi038.stderr: 78.0% -- replaced with /var/log/ceph/ceph-client.admin.36356.log.gz 2024-09-21T03:45:40.192 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.87304.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.87304.log.gz 2024-09-21T03:45:40.192 INFO:teuthology.orchestra.run.smithi038.stderr: 92.6% -- replaced with /var/log/ceph/ceph-client.admin.67292.log.gz 2024-09-21T03:45:40.192 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54442.log 2024-09-21T03:45:40.192 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.64102.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-09-21T03:45:40.192 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.54442.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61985.log 2024-09-21T03:45:40.193 INFO:teuthology.orchestra.run.smithi038.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.64102.log.gz 2024-09-21T03:45:40.193 INFO:teuthology.orchestra.run.smithi038.stderr: 87.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53261.log 2024-09-21T03:45:40.193 INFO:teuthology.orchestra.run.smithi038.stderr: -- replaced with /var/log/ceph/ceph-client.admin.54442.log.gz 2024-09-21T03:45:40.193 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-mds.c.log: /var/log/ceph/ceph-client.admin.61985.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.61985.log.gz 2024-09-21T03:45:40.193 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43415.log 2024-09-21T03:45:40.193 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.53261.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69854.log 2024-09-21T03:45:40.193 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.43415.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.53261.log.gz 2024-09-21T03:45:40.193 INFO:teuthology.orchestra.run.smithi038.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.43415.log.gz 2024-09-21T03:45:40.193 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80919.log 2024-09-21T03:45:40.193 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.69854.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.69854.log.gz 2024-09-21T03:45:40.193 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66446.log 2024-09-21T03:45:40.193 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.80919.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86964.log 2024-09-21T03:45:40.193 INFO:teuthology.orchestra.run.smithi038.stderr: 89.1%/var/log/ceph/ceph-client.admin.66446.log: -- replaced with /var/log/ceph/ceph-client.admin.80919.log.gz 2024-09-21T03:45:40.194 INFO:teuthology.orchestra.run.smithi038.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.66446.log.gz 2024-09-21T03:45:40.194 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57689.log 2024-09-21T03:45:40.194 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.86964.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.86964.log.gz 2024-09-21T03:45:40.194 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47791.log 2024-09-21T03:45:40.194 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.57689.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73747.log 2024-09-21T03:45:40.194 INFO:teuthology.orchestra.run.smithi038.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.57689.log.gz 2024-09-21T03:45:40.194 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.47791.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90198.log 2024-09-21T03:45:40.194 INFO:teuthology.orchestra.run.smithi038.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.47791.log.gz 2024-09-21T03:45:40.194 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.73747.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.73747.log.gz 2024-09-21T03:45:40.194 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72005.log 2024-09-21T03:45:40.194 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.90198.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73349.log 2024-09-21T03:45:40.194 INFO:teuthology.orchestra.run.smithi038.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.90198.log.gz 2024-09-21T03:45:40.195 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.72005.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66172.log 2024-09-21T03:45:40.195 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.73349.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.72005.log.gz 2024-09-21T03:45:40.195 INFO:teuthology.orchestra.run.smithi038.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.73349.log.gz 2024-09-21T03:45:40.195 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61046.log 2024-09-21T03:45:40.195 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.66172.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.49937.log 2024-09-21T03:45:40.195 INFO:teuthology.orchestra.run.smithi038.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.66172.log.gz 2024-09-21T03:45:40.195 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.61046.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49553.log 2024-09-21T03:45:40.195 INFO:teuthology.orchestra.run.smithi038.stderr: 86.9%/var/log/ceph/ceph-client.0.49937.log: -- replaced with /var/log/ceph/ceph-client.admin.61046.log.gz 2024-09-21T03:45:40.195 INFO:teuthology.orchestra.run.smithi038.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.49937.log.gz 2024-09-21T03:45:40.195 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62969.log 2024-09-21T03:45:40.195 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.49553.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48768.log 2024-09-21T03:45:40.195 INFO:teuthology.orchestra.run.smithi038.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.49553.log.gz 2024-09-21T03:45:40.195 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.62969.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79905.log 2024-09-21T03:45:40.195 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.48768.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.62969.log.gz 2024-09-21T03:45:40.196 INFO:teuthology.orchestra.run.smithi038.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48768.log.gz 2024-09-21T03:45:40.196 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54648.log 2024-09-21T03:45:40.196 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.79905.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56398.log 2024-09-21T03:45:40.196 INFO:teuthology.orchestra.run.smithi038.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.79905.log.gz 2024-09-21T03:45:40.196 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.54648.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59409.log 2024-09-21T03:45:40.196 INFO:teuthology.orchestra.run.smithi038.stderr: 87.3%/var/log/ceph/ceph-client.admin.56398.log: -- replaced with /var/log/ceph/ceph-client.admin.54648.log.gz 2024-09-21T03:45:40.196 INFO:teuthology.orchestra.run.smithi038.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.56398.log.gz 2024-09-21T03:45:40.196 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49247.log 2024-09-21T03:45:40.196 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.59409.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52693.log 2024-09-21T03:45:40.196 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.49247.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.59409.log.gz 2024-09-21T03:45:40.196 INFO:teuthology.orchestra.run.smithi038.stderr: 87.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71180.log 2024-09-21T03:45:40.196 INFO:teuthology.orchestra.run.smithi038.stderr: -- replaced with /var/log/ceph/ceph-client.admin.49247.log.gz 2024-09-21T03:45:40.196 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.52693.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77124.log 2024-09-21T03:45:40.197 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.71180.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.52693.log.gz 2024-09-21T03:45:40.197 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65543.log 2024-09-21T03:45:40.197 INFO:teuthology.orchestra.run.smithi038.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.71180.log.gz 2024-09-21T03:45:40.197 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.77124.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45896.log 2024-09-21T03:45:40.197 INFO:teuthology.orchestra.run.smithi038.stderr: 87.5%/var/log/ceph/ceph-client.admin.65543.log: -- replaced with /var/log/ceph/ceph-client.admin.77124.log.gz 2024-09-21T03:45:40.197 INFO:teuthology.orchestra.run.smithi038.stderr: 89.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63358.log 2024-09-21T03:45:40.197 INFO:teuthology.orchestra.run.smithi038.stderr: -- replaced with /var/log/ceph/ceph-client.admin.65543.log.gz 2024-09-21T03:45:40.197 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.45896.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57933.log 2024-09-21T03:45:40.197 INFO:teuthology.orchestra.run.smithi038.stderr: 87.0%/var/log/ceph/ceph-client.admin.63358.log: -- replaced with /var/log/ceph/ceph-client.admin.45896.log.gz 2024-09-21T03:45:40.197 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83167.log 2024-09-21T03:45:40.197 INFO:teuthology.orchestra.run.smithi038.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.63358.log.gz 2024-09-21T03:45:40.197 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.57933.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60752.log 2024-09-21T03:45:40.197 INFO:teuthology.orchestra.run.smithi038.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.57933.log.gz/var/log/ceph/ceph-client.admin.83167.log: 2024-09-21T03:45:40.198 INFO:teuthology.orchestra.run.smithi038.stderr:gzip 89.5% -- replaced with /var/log/ceph/ceph-client.admin.83167.log.gz 2024-09-21T03:45:40.198 INFO:teuthology.orchestra.run.smithi038.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.48042.log 2024-09-21T03:45:40.198 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.60752.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89659.log 2024-09-21T03:45:40.198 INFO:teuthology.orchestra.run.smithi038.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.60752.log.gz 2024-09-21T03:45:40.198 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.48042.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.48042.log.gz 2024-09-21T03:45:40.198 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80531.log 2024-09-21T03:45:40.198 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36812.log 2024-09-21T03:45:40.198 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.89659.log: /var/log/ceph/ceph-client.admin.80531.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.89659.log.gz 2024-09-21T03:45:40.198 INFO:teuthology.orchestra.run.smithi038.stderr: 89.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74097.log 2024-09-21T03:45:40.198 INFO:teuthology.orchestra.run.smithi038.stderr: -- replaced with /var/log/ceph/ceph-client.admin.80531.log.gz 2024-09-21T03:45:40.198 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.36812.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.36812.log.gz 2024-09-21T03:45:40.198 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53723.log 2024-09-21T03:45:40.198 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.74097.log: /var/log/ceph/ceph-client.admin.53723.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.53723.log.gz 2024-09-21T03:45:40.198 INFO:teuthology.orchestra.run.smithi038.stderr: 92.5% -- replaced with /var/log/ceph/ceph-client.admin.74097.log.gz 2024-09-21T03:45:40.237 INFO:teuthology.orchestra.run.smithi195.stderr: 90.9% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-09-21T03:45:40.450 INFO:teuthology.orchestra.run.smithi038.stderr: 91.7% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-09-21T03:45:40.777 INFO:teuthology.orchestra.run.smithi038.stderr: 90.5% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-09-21T03:45:41.171 INFO:teuthology.orchestra.run.smithi195.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-09-21T03:45:41.194 INFO:teuthology.orchestra.run.smithi195.stderr: 93.3% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-09-21T03:45:41.347 INFO:teuthology.orchestra.run.smithi038.stderr: 93.1% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-09-21T03:45:41.480 INFO:teuthology.orchestra.run.smithi195.stderr: 92.9% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-09-21T03:45:41.596 INFO:teuthology.orchestra.run.smithi038.stderr: 93.2% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-09-21T03:45:41.642 INFO:teuthology.orchestra.run.smithi195.stderr: 93.1% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-09-21T03:45:41.644 INFO:teuthology.orchestra.run.smithi195.stderr: 2024-09-21T03:45:41.644 INFO:teuthology.orchestra.run.smithi195.stderr:real 0m1.927s 2024-09-21T03:45:41.644 INFO:teuthology.orchestra.run.smithi195.stderr:user 0m7.444s 2024-09-21T03:45:41.644 INFO:teuthology.orchestra.run.smithi195.stderr:sys 0m0.284s 2024-09-21T03:45:41.823 INFO:teuthology.orchestra.run.smithi038.stderr: 92.9% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-09-21T03:45:41.907 INFO:teuthology.orchestra.run.smithi038.stderr: 92.9% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-09-21T03:45:41.909 INFO:teuthology.orchestra.run.smithi038.stderr: 2024-09-21T03:45:41.909 INFO:teuthology.orchestra.run.smithi038.stderr:real 0m2.213s 2024-09-21T03:45:41.909 INFO:teuthology.orchestra.run.smithi038.stderr:user 0m8.868s 2024-09-21T03:45:41.910 INFO:teuthology.orchestra.run.smithi038.stderr:sys 0m0.864s 2024-09-21T03:45:41.910 INFO:tasks.ceph:Archiving logs... 2024-09-21T03:45:41.910 DEBUG:teuthology.misc:Transferring archived files from smithi038:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913473/remote/smithi038/log 2024-09-21T03:45:41.911 DEBUG:teuthology.orchestra.run.smithi038:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-21T03:45:43.793 DEBUG:teuthology.misc:Transferring archived files from smithi195:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913473/remote/smithi195/log 2024-09-21T03:45:43.794 DEBUG:teuthology.orchestra.run.smithi195:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-21T03:45:44.825 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-21T03:45:44.832 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-21T03:45:44.832 DEBUG:teuthology.orchestra.run.smithi038:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-21T03:45:44.835 DEBUG:teuthology.orchestra.run.smithi195:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-21T03:45:44.930 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-21T03:45:44.945 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-21T03:45:44.945 DEBUG:teuthology.orchestra.run.smithi038:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T03:45:44.948 DEBUG:teuthology.orchestra.run.smithi195:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T03:45:44.963 INFO:teuthology.orchestra.run.smithi195.stderr:bash: line 1: ntpq: command not found 2024-09-21T03:45:44.963 INFO:teuthology.orchestra.run.smithi038.stderr:bash: line 1: ntpq: command not found 2024-09-21T03:45:44.968 INFO:teuthology.orchestra.run.smithi038.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T03:45:44.968 INFO:teuthology.orchestra.run.smithi038.stdout:=============================================================================== 2024-09-21T03:45:44.968 INFO:teuthology.orchestra.run.smithi038.stdout:^* hv01.front.sepia.ceph.com 3 6 377 54 -461us[ -466us] +/- 58ms 2024-09-21T03:45:44.968 INFO:teuthology.orchestra.run.smithi038.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 54 +616us[ +616us] +/- 55ms 2024-09-21T03:45:44.968 INFO:teuthology.orchestra.run.smithi038.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 56 -235us[ -240us] +/- 89ms 2024-09-21T03:45:44.968 INFO:teuthology.orchestra.run.smithi038.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T03:45:44.969 INFO:teuthology.orchestra.run.smithi195.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T03:45:44.969 INFO:teuthology.orchestra.run.smithi195.stdout:=============================================================================== 2024-09-21T03:45:44.969 INFO:teuthology.orchestra.run.smithi195.stdout:^* hv01.front.sepia.ceph.com 3 6 377 54 -455us[ -445us] +/- 58ms 2024-09-21T03:45:44.969 INFO:teuthology.orchestra.run.smithi195.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 58 +614us[ +624us] +/- 55ms 2024-09-21T03:45:44.970 INFO:teuthology.orchestra.run.smithi195.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 56 -229us[ -218us] +/- 89ms 2024-09-21T03:45:44.970 INFO:teuthology.orchestra.run.smithi195.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T03:45:44.970 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-21T03:45:44.978 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-21T03:45:44.978 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-21T03:45:44.986 DEBUG:teuthology.orchestra.run.smithi038:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-21T03:45:45.109 DEBUG:teuthology.orchestra.run.smithi195:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-21T03:45:45.180 DEBUG:teuthology.orchestra.run.smithi038:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-21T03:45:45.212 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T03:45:45.212 DEBUG:teuthology.orchestra.run.smithi195:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-21T03:45:45.242 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T03:45:45.243 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-21T03:45:45.252 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-21T03:45:45.261 INFO:teuthology.task.internal:Duration was 1133.396355 seconds 2024-09-21T03:45:45.261 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-21T03:45:45.270 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-21T03:45:45.270 DEBUG:teuthology.orchestra.run.smithi038:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-21T03:45:45.273 DEBUG:teuthology.orchestra.run.smithi195:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-21T03:45:45.313 INFO:teuthology.orchestra.run.smithi038.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T03:45:45.324 INFO:teuthology.orchestra.run.smithi195.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T03:45:45.609 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-21T03:45:45.610 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi038.front.sepia.ceph.com 2024-09-21T03:45:45.610 DEBUG:teuthology.orchestra.run.smithi038:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-21T03:45:45.632 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi195.front.sepia.ceph.com 2024-09-21T03:45:45.632 DEBUG:teuthology.orchestra.run.smithi195:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-21T03:45:45.673 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-21T03:45:45.674 DEBUG:teuthology.orchestra.run.smithi038:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-21T03:45:45.676 DEBUG:teuthology.orchestra.run.smithi195:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-21T03:45:45.741 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-21T03:45:45.741 DEBUG:teuthology.orchestra.run.smithi038:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-21T03:45:45.744 DEBUG:teuthology.orchestra.run.smithi195:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-21T03:45:45.889 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-21T03:45:45.898 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-21T03:45:45.899 DEBUG:teuthology.orchestra.run.smithi038:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-21T03:45:45.924 DEBUG:teuthology.orchestra.run.smithi195:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-21T03:45:45.949 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-21T03:45:45.959 DEBUG:teuthology.orchestra.run.smithi038:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-21T03:45:45.967 DEBUG:teuthology.orchestra.run.smithi195:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-21T03:45:45.989 INFO:teuthology.orchestra.run.smithi038.stdout:kernel.core_pattern = core 2024-09-21T03:45:46.013 INFO:teuthology.orchestra.run.smithi195.stdout:kernel.core_pattern = core 2024-09-21T03:45:46.026 DEBUG:teuthology.orchestra.run.smithi038:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-21T03:45:46.057 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T03:45:46.058 DEBUG:teuthology.orchestra.run.smithi195:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-21T03:45:46.080 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T03:45:46.080 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-21T03:45:46.090 INFO:teuthology.task.internal:Transferring archived files... 2024-09-21T03:45:46.090 DEBUG:teuthology.misc:Transferring archived files from smithi038:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913473/remote/smithi038 2024-09-21T03:45:46.090 DEBUG:teuthology.orchestra.run.smithi038:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-21T03:45:46.134 DEBUG:teuthology.misc:Transferring archived files from smithi195:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913473/remote/smithi195 2024-09-21T03:45:46.134 DEBUG:teuthology.orchestra.run.smithi195:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-21T03:45:46.167 INFO:teuthology.task.internal:Removing archive directory... 2024-09-21T03:45:46.168 DEBUG:teuthology.orchestra.run.smithi038:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-21T03:45:46.171 DEBUG:teuthology.orchestra.run.smithi195:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-21T03:45:46.217 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-21T03:45:46.228 INFO:teuthology.task.internal:Not uploading archives. 2024-09-21T03:45:46.228 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-21T03:45:46.237 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-21T03:45:46.237 DEBUG:teuthology.orchestra.run.smithi038:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-21T03:45:46.240 DEBUG:teuthology.orchestra.run.smithi195:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-21T03:45:46.252 INFO:teuthology.orchestra.run.smithi038.stdout: 265487 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 21 03:45 /home/ubuntu/cephtest 2024-09-21T03:45:46.272 INFO:teuthology.orchestra.run.smithi195.stdout: 265324 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 21 03:45 /home/ubuntu/cephtest 2024-09-21T03:45:46.273 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-21T03:45:46.282 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-21T03:45:46.304 INFO:teuthology.run:Summary data: description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/damage} duration: 1133.3963553905487 flavor: default owner: scheduled_pdonnell@teuthology success: true 2024-09-21T03:45:46.304 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-21T03:45:46.420 INFO:teuthology.run:pass